2024-04-25T09:56:52.073 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T09:56:52.073 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_neesingh@teuthology --name neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --verbose --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} --archive /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673050 -- /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673050/orig.config.yaml 2024-04-25T09:56:52.119 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T09:56:52.205 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673050 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7673050' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec kclient: syntax: v1 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1124 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN2iYC7Ugex++yKosbzMawOwzxL38DS3dqI7MUwotBAPrDEPKnVJV6b+sq62SjlSANplv7au1YWEDpu7Eqc6gng= smithi186.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA+Cc6WkzlJ+cSqMPMFeH+1li8tOLIdJJ4YKeh6QMd6OYTqIhlhT9xtmFNC+tjuTgMktZjWOZzrs6ibdQ63aeNE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_09:36:50 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T09:56:52.205 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa; will attempt to use it 2024-04-25T09:56:52.206 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa/tasks 2024-04-25T09:56:52.206 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T09:56:52.207 INFO:teuthology.task.internal:Checking packages... 2024-04-25T09:56:52.231 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T09:56:52.231 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T09:56:52.231 INFO:teuthology.packaging:ref: None 2024-04-25T09:56:52.231 INFO:teuthology.packaging:tag: None 2024-04-25T09:56:52.231 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T09:56:52.231 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:56:52.231 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T09:56:52.397 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628.g97ae6cc9 2024-04-25T09:56:52.399 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T09:56:52.406 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T09:56:52.406 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T09:56:52.421 INFO:teuthology.task.internal:Saving configuration 2024-04-25T09:56:52.437 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T09:56:52.442 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T09:56:52.462 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673050', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:51:46.879036', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN2iYC7Ugex++yKosbzMawOwzxL38DS3dqI7MUwotBAPrDEPKnVJV6b+sq62SjlSANplv7au1YWEDpu7Eqc6gng='} 2024-04-25T09:56:52.484 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673050', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:51:46.880514', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA+Cc6WkzlJ+cSqMPMFeH+1li8tOLIdJJ4YKeh6QMd6OYTqIhlhT9xtmFNC+tjuTgMktZjWOZzrs6ibdQ63aeNE='} 2024-04-25T09:56:52.484 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T09:56:52.491 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T09:56:52.491 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T09:56:52.491 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T09:56:52.560 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6fdfa3e160>, signals=[15]) 2024-04-25T09:56:52.561 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T09:56:52.567 INFO:teuthology.task.internal:Opening connections... 2024-04-25T09:56:52.567 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T09:56:52.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:56:52.644 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi186.front.sepia.ceph.com 2024-04-25T09:56:52.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:56:52.717 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T09:56:52.724 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-04-25T09:56:52.743 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-04-25T09:56:52.743 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-04-25T09:56:52.801 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-04-25T09:56:52.801 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-04-25T09:56:52.801 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-04-25T09:56:52.801 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-04-25T09:56:52.801 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-04-25T09:56:52.801 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-04-25T09:56:52.801 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T09:56:52.801 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-04-25T09:56:52.802 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-04-25T09:56:52.802 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T09:56:52.802 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-04-25T09:56:52.802 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T09:56:52.802 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T09:56:52.802 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T09:56:52.802 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-04-25T09:56:52.837 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2024-04-25T09:56:52.855 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2024-04-25T09:56:52.855 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2024-04-25T09:56:52.912 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="CentOS Stream" 2024-04-25T09:56:52.912 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="9" 2024-04-25T09:56:52.912 INFO:teuthology.orchestra.run.smithi186.stdout:ID="centos" 2024-04-25T09:56:52.912 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="rhel fedora" 2024-04-25T09:56:52.912 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="9" 2024-04-25T09:56:52.913 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el9" 2024-04-25T09:56:52.913 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T09:56:52.913 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2024-04-25T09:56:52.913 INFO:teuthology.orchestra.run.smithi186.stdout:LOGO="fedora-logo-icon" 2024-04-25T09:56:52.913 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T09:56:52.913 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://centos.org/" 2024-04-25T09:56:52.913 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T09:56:52.913 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T09:56:52.913 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T09:56:52.913 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2024-04-25T09:56:52.936 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T09:56:52.946 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T09:56:52.951 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T09:56:52.951 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:56:52.954 DEBUG:teuthology.orchestra.run.smithi186:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:56:52.969 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T09:56:52.976 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T09:56:52.976 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:56:53.011 DEBUG:teuthology.orchestra.run.smithi186:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:56:53.166 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T09:56:53.219 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T09:56:53.233 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T09:56:53.233 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-25T09:56:53.233 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-25T09:56:53.234 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:56:53.234 DEBUG:teuthology.orchestra.run.smithi186:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:56:53.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:56:53.251 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-04-25T09:56:53.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:56:53.252 DEBUG:teuthology.orchestra.run.smithi186:> uname -r 2024-04-25T09:56:53.309 INFO:teuthology.orchestra.run.smithi186.stdout:5.14.0-437.el9.x86_64 2024-04-25T09:56:53.309 INFO:teuthology.task.kernel:Running kernel on smithi186: 5.14.0-437.el9.x86_64 2024-04-25T09:56:53.309 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum install -y kernel 2024-04-25T09:56:53.310 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-437.el9.x86_64 2024-04-25T09:56:53.310 INFO:teuthology.task.kernel:Running kernel on smithi016: 5.14.0-437.el9.x86_64 2024-04-25T09:56:53.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2024-04-25T09:56:55.790 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 9.0 MB/s | 8.0 MB 00:00 2024-04-25T09:56:56.347 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.0 MB 00:01 2024-04-25T09:56:58.573 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-25T09:56:58.795 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 19 MB 00:01 2024-04-25T09:57:02.657 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T09:57:03.146 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.2 MB 00:00 2024-04-25T09:57:04.739 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-04-25T09:57:07.840 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 7.0 MB/s | 21 MB 00:02 2024-04-25T09:57:13.098 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-25T09:57:15.087 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T09:57:15.088 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T09:57:15.194 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-25T09:57:15.197 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-25T09:57:15.198 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-25T09:57:15.296 DEBUG:teuthology.orchestra.run.smithi016:> echo no | sudo yum reinstall kernel || true 2024-04-25T09:57:15.817 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 09:57:13 AM UTC. 2024-04-25T09:57:15.948 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-25T09:57:15.955 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-25T09:57:15.955 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-04-25T09:57:15.955 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-25T09:57:15.955 INFO:teuthology.orchestra.run.smithi016.stdout:Reinstalling: 2024-04-25T09:57:15.955 INFO:teuthology.orchestra.run.smithi016.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T09:57:15.955 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T09:57:15.955 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-25T09:57:15.955 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-25T09:57:15.955 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T09:57:15.955 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 255 k 2024-04-25T09:57:15.955 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 0 2024-04-25T09:57:15.956 INFO:teuthology.orchestra.run.smithi016.stderr:Operation aborted. 2024-04-25T09:57:16.011 INFO:teuthology.orchestra.run.smithi016.stdout:Is this ok [y/N]: 2024-04-25T09:57:16.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum reinstall -y kernel || true 2024-04-25T09:57:16.530 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 09:57:13 AM UTC. 2024-04-25T09:57:16.651 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-25T09:57:16.651 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-25T09:57:16.652 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-04-25T09:57:16.652 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-25T09:57:16.652 INFO:teuthology.orchestra.run.smithi016.stdout:Reinstalling: 2024-04-25T09:57:16.652 INFO:teuthology.orchestra.run.smithi016.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T09:57:16.652 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T09:57:16.652 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-25T09:57:16.652 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-25T09:57:16.652 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T09:57:16.652 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 255 k 2024-04-25T09:57:16.653 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 0 2024-04-25T09:57:16.653 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-25T09:57:17.034 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.0 MB/s | 255 kB 00:00 2024-04-25T09:57:17.034 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:57:17.034 INFO:teuthology.orchestra.run.smithi016.stdout:Total 669 kB/s | 255 kB 00:00 2024-04-25T09:57:17.052 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-25T09:57:17.224 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-25T09:57:17.224 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-25T09:57:17.229 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-25T09:57:17.230 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-25T09:57:17.370 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-25T09:57:17.459 INFO:teuthology.orchestra.run.smithi016.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T09:57:17.459 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:57:18.107 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:57:18.107 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T09:57:18.794 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:57:18.794 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T09:57:18.795 INFO:teuthology.orchestra.run.smithi016.stdout:Reinstalled: 2024-04-25T09:57:18.795 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T09:57:18.795 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T09:57:18.795 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-25T09:57:18.997 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T09:57:19.045 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T09:57:19.045 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T09:57:19.045 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T09:57:19.045 DEBUG:teuthology.parallel:result is None 2024-04-25T09:57:52.527 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - Extras packages 344 B/s | 16 kB 00:48 2024-04-25T09:57:53.862 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-25T09:57:59.003 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 6.6 kB/s | 1.7 kB 00:00 2024-04-25T09:58:00.884 INFO:teuthology.orchestra.run.smithi186.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T09:58:00.885 INFO:teuthology.orchestra.run.smithi186.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T09:58:00.976 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-25T09:58:00.976 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2024-04-25T09:58:00.977 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-25T09:58:01.085 DEBUG:teuthology.orchestra.run.smithi186:> echo no | sudo yum reinstall kernel || true 2024-04-25T09:58:01.622 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 09:57:58 AM UTC. 2024-04-25T09:58:01.741 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-25T09:58:01.752 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-25T09:58:01.752 INFO:teuthology.orchestra.run.smithi186.stdout: Package Architecture Version Repository Size 2024-04-25T09:58:01.752 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-25T09:58:01.752 INFO:teuthology.orchestra.run.smithi186.stdout:Reinstalling: 2024-04-25T09:58:01.752 INFO:teuthology.orchestra.run.smithi186.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T09:58:01.752 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T09:58:01.752 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-25T09:58:01.752 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-25T09:58:01.752 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T09:58:01.752 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 255 k 2024-04-25T09:58:01.752 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 0 2024-04-25T09:58:01.753 INFO:teuthology.orchestra.run.smithi186.stderr:Operation aborted. 2024-04-25T09:58:01.802 INFO:teuthology.orchestra.run.smithi186.stdout:Is this ok [y/N]: 2024-04-25T09:58:01.802 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum reinstall -y kernel || true 2024-04-25T09:58:02.295 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 09:57:58 AM UTC. 2024-04-25T09:58:02.411 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-25T09:58:02.412 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-25T09:58:02.413 INFO:teuthology.orchestra.run.smithi186.stdout: Package Architecture Version Repository Size 2024-04-25T09:58:02.413 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-25T09:58:02.413 INFO:teuthology.orchestra.run.smithi186.stdout:Reinstalling: 2024-04-25T09:58:02.413 INFO:teuthology.orchestra.run.smithi186.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T09:58:02.413 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T09:58:02.413 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-25T09:58:02.413 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-25T09:58:02.413 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T09:58:02.413 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 255 k 2024-04-25T09:58:02.413 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 0 2024-04-25T09:58:02.413 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-25T09:58:02.672 INFO:teuthology.orchestra.run.smithi186.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.3 MB/s | 255 kB 00:00 2024-04-25T09:58:02.672 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:58:02.672 INFO:teuthology.orchestra.run.smithi186.stdout:Total 983 kB/s | 255 kB 00:00 2024-04-25T09:58:02.688 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-25T09:58:02.903 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-25T09:58:02.903 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-25T09:58:02.908 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-25T09:58:02.908 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-25T09:58:03.037 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-25T09:58:03.115 INFO:teuthology.orchestra.run.smithi186.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T09:58:03.115 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:58:03.577 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:58:03.577 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T09:58:04.236 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:58:04.237 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T09:58:04.237 INFO:teuthology.orchestra.run.smithi186.stdout:Reinstalled: 2024-04-25T09:58:04.237 INFO:teuthology.orchestra.run.smithi186.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T09:58:04.237 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T09:58:04.237 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-25T09:58:04.396 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T09:58:04.443 INFO:teuthology.orchestra.run.smithi186.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T09:58:04.443 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T09:58:04.443 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T09:58:04.444 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:04.444 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T09:58:04.454 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T09:58:04.455 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:58:04.458 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:58:04.474 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T09:58:04.481 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T09:58:04.492 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T09:58:04.492 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:58:04.516 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:58:04.551 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T09:58:04.558 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T09:58:04.558 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:58:04.587 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:58:04.614 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:58:04.630 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:58:04.638 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:58:04.659 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:58:04.661 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T09:58:04.669 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T09:58:04.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:58:04.672 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:58:04.697 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T09:58:04.706 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T09:58:04.706 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:58:04.739 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:58:04.754 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:58:04.841 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:58:04.894 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:58:04.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:58:05.019 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T09:58:05.019 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:58:05.084 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:58:05.124 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:58:05.173 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:58:05.256 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:58:05.293 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T09:58:05.293 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:58:05.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-04-25T09:58:05.359 DEBUG:teuthology.orchestra.run.smithi186:> sudo service rsyslog restart 2024-04-25T09:58:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T09:58:05.434 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T09:58:05.919 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T09:58:05.927 INFO:teuthology.task.internal:Starting timer... 2024-04-25T09:58:05.927 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T09:58:05.972 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T09:58:05.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-04-25T09:58:06.056 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-04-25T09:58:06.058 DEBUG:teuthology.orchestra.run.smithi186:> sudo service auditd rotate 2024-04-25T09:58:06.110 INFO:teuthology.orchestra.run.smithi186.stdout:Rotating logs: 2024-04-25T09:58:06.112 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T09:58:06.112 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-04-25T09:58:06.146 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-04-25T09:58:06.146 DEBUG:teuthology.orchestra.run.smithi186:> /usr/sbin/getenforce 2024-04-25T09:58:06.172 INFO:teuthology.orchestra.run.smithi186.stdout:Permissive 2024-04-25T09:58:06.172 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi186.front.sepia.ceph.com': 'permissive'} 2024-04-25T09:58:06.172 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T09:58:06.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:58:06.214 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T09:58:06.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:58:06.237 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T09:58:06.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-04-25T09:58:06.279 DEBUG:teuthology.orchestra.run.smithi186:> sudo /usr/sbin/setenforce permissive 2024-04-25T09:58:06.303 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T09:58:06.314 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T09:58:06.320 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T09:58:06.450 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T09:58:06.466 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T09:58:06.467 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi186.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T10:01:35.639 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi186.front.sepia.ceph.com')] 2024-04-25T10:01:35.641 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-04-25T10:01:35.643 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:01:35.723 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-04-25T10:01:35.809 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-04-25T10:01:35.810 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-04-25T10:01:35.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:01:35.884 DEBUG:teuthology.orchestra.run.smithi186:> true 2024-04-25T10:01:35.924 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-04-25T10:01:35.925 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T10:01:35.936 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T10:01:35.936 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:01:35.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:01:35.939 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:01:35.939 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:01:35.966 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T10:01:35.980 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T10:01:35.992 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T10:01:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-04-25T10:01:36.005 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T10:01:36.014 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-04-25T10:01:36.028 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: ntpd: command not found 2024-04-25T10:01:36.029 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T10:01:36.038 INFO:teuthology.orchestra.run.smithi186.stdout:506 Cannot talk to daemon 2024-04-25T10:01:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T10:01:36.051 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T10:01:36.064 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T10:01:36.102 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:01:36.105 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:01:36.105 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-04-25T10:01:36.105 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:01:36.105 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:01:36.105 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:01:36.105 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:01:36.120 INFO:teuthology.orchestra.run.smithi186.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:01:36.123 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:01:36.123 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2024-04-25T10:01:36.123 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:01:36.123 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:01:36.123 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:01:36.124 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:01:36.125 INFO:teuthology.run_tasks:Running task install... 2024-04-25T10:01:36.134 DEBUG:teuthology.task.install:project ceph 2024-04-25T10:01:36.134 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T10:01:36.134 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T10:01:36.134 INFO:teuthology.task.install:Using flavor: default 2024-04-25T10:01:36.140 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T10:01:36.140 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T10:01:36.141 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T10:01:36.142 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:01:36.144 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T10:01:36.144 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:01:36.292 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T10:01:36.292 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T10:01:36.378 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T10:01:36.378 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T10:01:36.497 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T10:01:36.497 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:01:36.497 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T10:01:36.521 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T10:01:36.521 DEBUG:teuthology.orchestra.run.smithi186:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T10:01:36.548 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T10:01:36.549 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:01:36.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T10:01:36.575 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T10:01:36.575 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T10:01:36.586 DEBUG:teuthology.orchestra.run.smithi186:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T10:01:36.641 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T10:01:36.659 DEBUG:teuthology.orchestra.run.smithi186:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T10:01:36.716 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T10:01:36.722 INFO:teuthology.orchestra.run.smithi186.stdout:check_obsoletes = 1 2024-04-25T10:01:36.724 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum clean all 2024-04-25T10:01:36.780 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-04-25T10:01:36.782 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-04-25T10:01:36.947 INFO:teuthology.orchestra.run.smithi186.stdout:48 files removed 2024-04-25T10:01:36.975 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T10:01:37.014 INFO:teuthology.orchestra.run.smithi016.stdout:48 files removed 2024-04-25T10:01:37.041 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T10:01:37.582 INFO:teuthology.orchestra.run.smithi186.stdout:ceph packages for x86_64 236 kB/s | 82 kB 00:00 2024-04-25T10:01:37.646 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 247 kB/s | 82 kB 00:00 2024-04-25T10:01:37.891 INFO:teuthology.orchestra.run.smithi186.stdout:ceph noarch packages 38 kB/s | 10 kB 00:00 2024-04-25T10:01:37.989 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 34 kB/s | 10 kB 00:00 2024-04-25T10:01:38.209 INFO:teuthology.orchestra.run.smithi186.stdout:ceph source packages 6.7 kB/s | 1.9 kB 00:00 2024-04-25T10:01:38.348 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-04-25T10:01:46.310 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - BaseOS 1.0 MB/s | 8.0 MB 00:08 2024-04-25T10:01:53.705 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 538 kB/s | 8.0 MB 00:15 2024-04-25T10:01:54.101 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - AppStream 2.8 MB/s | 19 MB 00:06 2024-04-25T10:01:58.727 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - CRB 6.0 MB/s | 6.2 MB 00:01 2024-04-25T10:02:08.877 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - Extras packages 1.8 kB/s | 16 kB 00:08 2024-04-25T10:02:19.388 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 2.0 MB/s | 21 MB 00:10 2024-04-25T10:02:20.212 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 767 kB/s | 19 MB 00:25 2024-04-25T10:02:24.419 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-25T10:02:26.536 INFO:teuthology.orchestra.run.smithi186.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:02:26.537 INFO:teuthology.orchestra.run.smithi186.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:02:26.548 INFO:teuthology.orchestra.run.smithi186.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T10:02:26.548 INFO:teuthology.orchestra.run.smithi186.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T10:02:26.549 INFO:teuthology.orchestra.run.smithi186.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T10:02:26.550 INFO:teuthology.orchestra.run.smithi186.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T10:02:26.552 INFO:teuthology.orchestra.run.smithi186.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T10:02:26.554 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T10:02:26.555 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T10:02:26.616 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-25T10:02:26.625 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================= 2024-04-25T10:02:26.625 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2024-04-25T10:02:26.625 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================= 2024-04-25T10:02:26.625 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-25T10:02:26.625 INFO:teuthology.orchestra.run.smithi186.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T10:02:26.625 INFO:teuthology.orchestra.run.smithi186.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T10:02:26.625 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T10:02:26.625 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T10:02:26.625 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T10:02:26.625 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi186.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T10:02:26.627 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T10:02:26.627 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T10:02:26.627 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T10:02:26.627 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T10:02:26.627 INFO:teuthology.orchestra.run.smithi186.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T10:02:26.627 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T10:02:26.627 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T10:02:26.627 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T10:02:26.627 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T10:02:26.627 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T10:02:26.627 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T10:02:26.627 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T10:02:26.627 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T10:02:26.628 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T10:02:26.628 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T10:02:26.628 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T10:02:26.628 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T10:02:26.628 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T10:02:26.628 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrading: 2024-04-25T10:02:26.628 INFO:teuthology.orchestra.run.smithi186.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T10:02:26.628 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T10:02:26.628 INFO:teuthology.orchestra.run.smithi186.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T10:02:26.628 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T10:02:26.628 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2024-04-25T10:02:26.628 INFO:teuthology.orchestra.run.smithi186.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T10:02:26.628 INFO:teuthology.orchestra.run.smithi186.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T10:02:26.628 INFO:teuthology.orchestra.run.smithi186.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T10:02:26.629 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T10:02:26.629 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T10:02:26.629 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T10:02:26.629 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T10:02:26.629 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T10:02:26.629 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T10:02:26.629 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T10:02:26.629 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T10:02:26.629 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T10:02:26.629 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T10:02:26.629 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T10:02:26.629 INFO:teuthology.orchestra.run.smithi186.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T10:02:26.629 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T10:02:26.630 INFO:teuthology.orchestra.run.smithi186.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T10:02:26.630 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T10:02:26.630 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T10:02:26.630 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T10:02:26.630 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T10:02:26.630 INFO:teuthology.orchestra.run.smithi186.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T10:02:26.630 INFO:teuthology.orchestra.run.smithi186.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T10:02:26.630 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T10:02:26.630 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T10:02:26.630 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T10:02:26.630 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T10:02:26.630 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T10:02:26.630 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi186.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi186.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi186.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi186.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi186.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi186.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi186.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T10:02:26.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T10:02:26.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T10:02:26.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T10:02:26.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T10:02:26.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T10:02:26.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T10:02:26.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T10:02:26.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T10:02:26.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T10:02:26.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T10:02:26.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T10:02:26.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T10:02:26.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T10:02:26.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T10:02:26.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T10:02:26.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T10:02:26.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T10:02:26.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T10:02:26.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T10:02:26.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T10:02:26.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T10:02:26.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T10:02:26.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T10:02:26.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T10:02:26.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T10:02:26.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T10:02:26.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T10:02:26.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T10:02:26.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T10:02:26.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T10:02:26.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T10:02:26.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T10:02:26.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T10:02:26.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T10:02:26.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T10:02:26.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T10:02:26.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T10:02:26.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T10:02:26.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T10:02:26.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T10:02:26.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T10:02:26.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T10:02:26.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T10:02:26.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T10:02:26.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T10:02:26.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T10:02:26.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T10:02:26.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T10:02:26.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T10:02:26.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T10:02:26.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T10:02:26.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T10:02:26.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T10:02:26.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T10:02:26.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T10:02:26.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T10:02:26.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T10:02:26.636 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T10:02:26.636 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T10:02:26.636 INFO:teuthology.orchestra.run.smithi186.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T10:02:26.636 INFO:teuthology.orchestra.run.smithi186.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T10:02:26.636 INFO:teuthology.orchestra.run.smithi186.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T10:02:26.636 INFO:teuthology.orchestra.run.smithi186.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T10:02:26.636 INFO:teuthology.orchestra.run.smithi186.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T10:02:26.636 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T10:02:26.636 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T10:02:26.636 INFO:teuthology.orchestra.run.smithi186.stdout:Installing weak dependencies: 2024-04-25T10:02:26.636 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T10:02:26.636 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:02:26.636 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-25T10:02:26.636 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================= 2024-04-25T10:02:26.636 INFO:teuthology.orchestra.run.smithi186.stdout:Install 148 Packages 2024-04-25T10:02:26.637 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrade 4 Packages 2024-04-25T10:02:26.637 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:02:26.637 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 215 M 2024-04-25T10:02:26.637 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-25T10:02:32.821 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 712 kB/s | 6.2 MB 00:08 2024-04-25T10:02:34.313 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 16 kB 00:00 2024-04-25T10:02:36.226 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-25T10:02:41.473 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-25T10:02:42.377 INFO:teuthology.orchestra.run.smithi186.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-04-25T10:02:42.844 INFO:teuthology.orchestra.run.smithi186.stdout:(2/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 6.3 MB/s | 5.3 MB 00:00 2024-04-25T10:02:42.877 INFO:teuthology.orchestra.run.smithi186.stdout:(3/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.7 MB/s | 887 kB 00:00 2024-04-25T10:02:42.903 INFO:teuthology.orchestra.run.smithi186.stdout:(4/152): ceph-immutable-object-cache-19.0.0-162 2.4 MB/s | 145 kB 00:00 2024-04-25T10:02:43.045 INFO:teuthology.orchestra.run.smithi186.stdout:(5/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 11 MB/s | 1.5 MB 00:00 2024-04-25T10:02:43.428 INFO:teuthology.orchestra.run.smithi186.stdout:(6/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 12 MB/s | 4.7 MB 00:00 2024-04-25T10:02:43.679 INFO:teuthology.orchestra.run.smithi186.stdout:(7/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 2.7 MB/s | 2.1 MB 00:00 2024-04-25T10:02:43.683 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:02:43.684 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:02:43.695 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T10:02:43.696 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T10:02:43.697 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T10:02:43.698 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T10:02:43.699 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T10:02:43.702 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T10:02:43.702 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T10:02:43.766 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T10:02:43.777 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T10:02:43.777 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T10:02:43.777 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T10:02:43.777 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T10:02:43.777 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T10:02:43.777 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T10:02:43.777 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T10:02:43.777 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T10:02:43.777 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T10:02:43.777 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T10:02:43.777 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T10:02:43.777 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T10:02:43.777 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T10:02:43.780 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T10:02:43.780 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T10:02:43.780 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T10:02:43.780 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T10:02:43.780 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T10:02:43.780 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T10:02:43.780 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T10:02:43.780 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T10:02:43.780 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T10:02:43.780 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T10:02:43.780 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T10:02:43.780 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T10:02:43.780 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T10:02:43.780 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T10:02:43.782 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T10:02:43.782 INFO:teuthology.orchestra.run.smithi016.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T10:02:43.782 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T10:02:43.782 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T10:02:43.782 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T10:02:43.782 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T10:02:43.782 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T10:02:43.782 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T10:02:43.782 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T10:02:43.782 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T10:02:43.782 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T10:02:43.782 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T10:02:43.782 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T10:02:43.783 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T10:02:43.783 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T10:02:43.783 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T10:02:43.783 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T10:02:43.783 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T10:02:43.783 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T10:02:43.783 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T10:02:43.783 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T10:02:43.783 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T10:02:43.783 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T10:02:43.783 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T10:02:43.783 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T10:02:43.783 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T10:02:43.783 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T10:02:43.783 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T10:02:43.784 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T10:02:43.784 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T10:02:43.784 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T10:02:43.784 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T10:02:43.784 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T10:02:43.784 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T10:02:43.784 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T10:02:43.784 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T10:02:43.784 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T10:02:43.784 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T10:02:43.784 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T10:02:43.784 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T10:02:43.784 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T10:02:43.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T10:02:43.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T10:02:43.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T10:02:43.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T10:02:43.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T10:02:43.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T10:02:43.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T10:02:43.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T10:02:43.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T10:02:43.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T10:02:43.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T10:02:43.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T10:02:43.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T10:02:43.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T10:02:43.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T10:02:43.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T10:02:43.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T10:02:43.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T10:02:43.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T10:02:43.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T10:02:43.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T10:02:43.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T10:02:43.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T10:02:43.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T10:02:43.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T10:02:43.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T10:02:43.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T10:02:43.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T10:02:43.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T10:02:43.787 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T10:02:43.787 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T10:02:43.787 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T10:02:43.787 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T10:02:43.787 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T10:02:43.787 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T10:02:43.787 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T10:02:43.787 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T10:02:43.787 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T10:02:43.787 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T10:02:43.787 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-04-25T10:02:43.787 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T10:02:43.787 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:02:43.787 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-25T10:02:43.787 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-04-25T10:02:43.787 INFO:teuthology.orchestra.run.smithi016.stdout:Install 148 Packages 2024-04-25T10:02:43.788 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 4 Packages 2024-04-25T10:02:43.788 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:02:43.788 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 215 M 2024-04-25T10:02:43.788 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-25T10:02:44.246 INFO:teuthology.orchestra.run.smithi186.stdout:(8/152): ceph-common-19.0.0-1628.g97ae6cc9.el9. 8.8 MB/s | 20 MB 00:02 2024-04-25T10:02:44.304 INFO:teuthology.orchestra.run.smithi186.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 435 kB/s | 25 kB 00:00 2024-04-25T10:02:44.830 INFO:teuthology.orchestra.run.smithi186.stdout:(10/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x8 12 MB/s | 17 MB 00:01 2024-04-25T10:02:44.905 INFO:teuthology.orchestra.run.smithi186.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 2.9 MB/s | 220 kB 00:00 2024-04-25T10:02:44.963 INFO:teuthology.orchestra.run.smithi186.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 465 kB/s | 27 kB 00:00 2024-04-25T10:02:44.997 INFO:teuthology.orchestra.run.smithi186.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 942 kB/s | 31 kB 00:00 2024-04-25T10:02:45.081 INFO:teuthology.orchestra.run.smithi186.stdout:(14/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 8.7 MB/s | 744 kB 00:00 2024-04-25T10:02:45.131 INFO:teuthology.orchestra.run.smithi186.stdout:(15/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 3.2 MB/s | 163 kB 00:00 2024-04-25T10:02:45.181 INFO:teuthology.orchestra.run.smithi186.stdout:(16/152): librados-devel-19.0.0-1628.g97ae6cc9. 2.5 MB/s | 127 kB 00:00 2024-04-25T10:02:45.282 INFO:teuthology.orchestra.run.smithi186.stdout:(17/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 6.1 MB/s | 9.8 MB 00:01 2024-04-25T10:02:45.315 INFO:teuthology.orchestra.run.smithi186.stdout:(18/152): libradosstriper1-19.0.0-1628.g97ae6cc 3.7 MB/s | 501 kB 00:00 2024-04-25T10:02:45.390 INFO:teuthology.orchestra.run.smithi186.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 603 kB/s | 45 kB 00:00 2024-04-25T10:02:45.466 INFO:teuthology.orchestra.run.smithi186.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 1.7 MB/s | 131 kB 00:00 2024-04-25T10:02:45.551 INFO:teuthology.orchestra.run.smithi186.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 1.9 MB/s | 162 kB 00:00 2024-04-25T10:02:45.652 INFO:teuthology.orchestra.run.smithi186.stdout:(22/152): python3-rados-19.0.0-1628.g97ae6cc9.e 3.1 MB/s | 321 kB 00:00 2024-04-25T10:02:45.735 INFO:teuthology.orchestra.run.smithi186.stdout:(23/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 3.5 MB/s | 298 kB 00:00 2024-04-25T10:02:45.802 INFO:teuthology.orchestra.run.smithi186.stdout:(24/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 9.8 MB/s | 5.1 MB 00:00 2024-04-25T10:02:45.827 INFO:teuthology.orchestra.run.smithi186.stdout:(25/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 1.1 MB/s | 99 kB 00:00 2024-04-25T10:02:45.869 INFO:teuthology.orchestra.run.smithi186.stdout:(26/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-25T10:02:45.986 INFO:teuthology.orchestra.run.smithi186.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 1.4 MB/s | 170 kB 00:00 2024-04-25T10:02:46.078 INFO:teuthology.orchestra.run.smithi186.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 344 kB/s | 31 kB 00:00 2024-04-25T10:02:46.136 INFO:teuthology.orchestra.run.smithi186.stdout:(29/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 10 MB/s | 3.1 MB 00:00 2024-04-25T10:02:46.162 INFO:teuthology.orchestra.run.smithi186.stdout:(30/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 1.6 MB/s | 138 kB 00:00 2024-04-25T10:02:46.362 INFO:teuthology.orchestra.run.smithi186.stdout:(31/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 7.9 MB/s | 1.8 MB 00:00 2024-04-25T10:02:46.438 INFO:teuthology.orchestra.run.smithi186.stdout:(32/152): ceph-mgr-modules-core-19.0.0-1628.g97 3.2 MB/s | 248 kB 00:00 2024-04-25T10:02:46.512 INFO:teuthology.orchestra.run.smithi186.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 660 kB/s | 49 kB 00:00 2024-04-25T10:02:46.580 INFO:teuthology.orchestra.run.smithi186.stdout:(34/152): ceph-prometheus-alerts-19.0.0-1628.g9 230 kB/s | 15 kB 00:00 2024-04-25T10:02:46.663 INFO:teuthology.orchestra.run.smithi186.stdout:(35/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 3.1 MB/s | 264 kB 00:00 2024-04-25T10:02:46.822 INFO:teuthology.orchestra.run.smithi186.stdout:(36/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 4.6 MB/s | 748 kB 00:00 2024-04-25T10:02:46.906 INFO:teuthology.orchestra.run.smithi186.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 9.9 MB/s | 7.4 MB 00:00 2024-04-25T10:02:46.931 INFO:teuthology.orchestra.run.smithi186.stdout:(38/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 168 kB/s | 18 kB 00:00 2024-04-25T10:02:46.998 INFO:teuthology.orchestra.run.smithi186.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 877 kB/s | 80 kB 00:00 2024-04-25T10:02:47.048 INFO:teuthology.orchestra.run.smithi186.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.7 MB/s | 803 kB 00:00 2024-04-25T10:02:47.090 INFO:teuthology.orchestra.run.smithi186.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-04-25T10:02:47.140 INFO:teuthology.orchestra.run.smithi186.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-25T10:02:47.207 INFO:teuthology.orchestra.run.smithi186.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-25T10:02:47.241 INFO:teuthology.orchestra.run.smithi186.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 8.1 MB/s | 1.2 MB 00:00 2024-04-25T10:02:47.299 INFO:teuthology.orchestra.run.smithi186.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-04-25T10:02:47.333 INFO:teuthology.orchestra.run.smithi186.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-04-25T10:02:47.392 INFO:teuthology.orchestra.run.smithi186.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-25T10:02:48.009 INFO:teuthology.orchestra.run.smithi186.stdout:(48/152): ceph-test-19.0.0-1628.g97ae6cc9.el9.x 13 MB/s | 47 MB 00:03 2024-04-25T10:02:48.033 INFO:teuthology.orchestra.run.smithi186.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 983 kB/s | 689 kB 00:00 2024-04-25T10:02:48.101 INFO:teuthology.orchestra.run.smithi186.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-04-25T10:02:48.124 INFO:teuthology.orchestra.run.smithi016.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 34 kB/s | 6.6 kB 00:00 2024-04-25T10:02:48.143 INFO:teuthology.orchestra.run.smithi186.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-04-25T10:02:48.210 INFO:teuthology.orchestra.run.smithi186.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 491 kB/s | 33 kB 00:00 2024-04-25T10:02:48.235 INFO:teuthology.orchestra.run.smithi186.stdout:(53/152): boost-program-options-1.75.0-8.el9.x8 473 kB/s | 107 kB 00:00 2024-04-25T10:02:48.294 INFO:teuthology.orchestra.run.smithi186.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 1.6 MB/s | 313 kB 00:00 2024-04-25T10:02:48.319 INFO:teuthology.orchestra.run.smithi186.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 218 kB/s | 18 kB 00:00 2024-04-25T10:02:48.369 INFO:teuthology.orchestra.run.smithi186.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-04-25T10:02:48.444 INFO:teuthology.orchestra.run.smithi186.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-04-25T10:02:48.469 INFO:teuthology.orchestra.run.smithi186.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 452 kB/s | 45 kB 00:00 2024-04-25T10:02:48.661 INFO:teuthology.orchestra.run.smithi186.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.6 MB/s | 3.0 MB 00:00 2024-04-25T10:02:48.725 INFO:teuthology.orchestra.run.smithi016.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.4 MB/s | 887 kB 00:00 2024-04-25T10:02:48.762 INFO:teuthology.orchestra.run.smithi186.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 255 kB/s | 25 kB 00:00 2024-04-25T10:02:48.787 INFO:teuthology.orchestra.run.smithi186.stdout:(61/152): librdkafka-1.6.1-102.el9.x86_64.rpm 1.9 MB/s | 662 kB 00:00 2024-04-25T10:02:48.812 INFO:teuthology.orchestra.run.smithi186.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 1.7 MB/s | 582 kB 00:00 2024-04-25T10:02:48.837 INFO:teuthology.orchestra.run.smithi186.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-04-25T10:02:48.892 INFO:teuthology.orchestra.run.smithi016.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 873 kB/s | 145 kB 00:00 2024-04-25T10:02:48.921 INFO:teuthology.orchestra.run.smithi186.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-04-25T10:02:48.946 INFO:teuthology.orchestra.run.smithi186.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 324 kB/s | 35 kB 00:00 2024-04-25T10:02:48.971 INFO:teuthology.orchestra.run.smithi186.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-04-25T10:02:49.180 INFO:teuthology.orchestra.run.smithi186.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-04-25T10:02:49.314 INFO:teuthology.orchestra.run.smithi186.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 6.7 MB/s | 911 kB 00:00 2024-04-25T10:02:49.389 INFO:teuthology.orchestra.run.smithi186.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.7 MB/s | 133 kB 00:00 2024-04-25T10:02:49.567 INFO:teuthology.orchestra.run.smithi016.stdout:(4/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 3.2 MB/s | 2.1 MB 00:00 2024-04-25T10:02:50.017 INFO:teuthology.orchestra.run.smithi016.stdout:(5/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 3.3 MB/s | 1.5 MB 00:00 2024-04-25T10:02:50.147 INFO:teuthology.orchestra.run.smithi186.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 7.6 MB/s | 5.8 MB 00:00 2024-04-25T10:02:50.231 INFO:teuthology.orchestra.run.smithi186.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.4 MB/s | 4.4 MB 00:01 2024-04-25T10:02:50.323 INFO:teuthology.orchestra.run.smithi186.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 956 kB/s | 87 kB 00:00 2024-04-25T10:02:50.356 INFO:teuthology.orchestra.run.smithi186.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-04-25T10:02:50.415 INFO:teuthology.orchestra.run.smithi186.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-25T10:02:50.474 INFO:teuthology.orchestra.run.smithi186.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-25T10:02:50.541 INFO:teuthology.orchestra.run.smithi186.stdout:(76/152): openblas-openmp-0.3.21-2.el9.x86_64.r 2.9 MB/s | 4.7 MB 00:01 2024-04-25T10:02:50.557 INFO:teuthology.orchestra.run.smithi186.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 307 kB/s | 26 kB 00:00 2024-04-25T10:02:50.583 INFO:teuthology.orchestra.run.smithi186.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 458 kB/s | 19 kB 00:00 2024-04-25T10:02:50.616 INFO:teuthology.orchestra.run.smithi186.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T10:02:50.642 INFO:teuthology.orchestra.run.smithi186.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-04-25T10:02:50.734 INFO:teuthology.orchestra.run.smithi186.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-25T10:02:50.817 INFO:teuthology.orchestra.run.smithi186.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-04-25T10:02:50.868 INFO:teuthology.orchestra.run.smithi186.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T10:02:50.918 INFO:teuthology.orchestra.run.smithi186.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 848 kB/s | 42 kB 00:00 2024-04-25T10:02:51.002 INFO:teuthology.orchestra.run.smithi186.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-04-25T10:02:51.060 INFO:teuthology.orchestra.run.smithi186.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-25T10:02:51.169 INFO:teuthology.orchestra.run.smithi186.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-04-25T10:02:51.200 INFO:teuthology.orchestra.run.smithi016.stdout:(6/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 4.0 MB/s | 4.7 MB 00:01 2024-04-25T10:02:51.252 INFO:teuthology.orchestra.run.smithi186.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 613 kB/s | 51 kB 00:00 2024-04-25T10:02:51.319 INFO:teuthology.orchestra.run.smithi186.stdout:(89/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.3 MB/s | 5.1 MB 00:00 2024-04-25T10:02:51.369 INFO:teuthology.orchestra.run.smithi186.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 460 kB/s | 54 kB 00:00 2024-04-25T10:02:51.428 INFO:teuthology.orchestra.run.smithi186.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 713 kB/s | 42 kB 00:00 2024-04-25T10:02:51.495 INFO:teuthology.orchestra.run.smithi186.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 664 kB/s | 44 kB 00:00 2024-04-25T10:02:51.620 INFO:teuthology.orchestra.run.smithi186.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-04-25T10:02:51.679 INFO:teuthology.orchestra.run.smithi186.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 617 kB/s | 36 kB 00:00 2024-04-25T10:02:51.729 INFO:teuthology.orchestra.run.smithi186.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-25T10:02:51.838 INFO:teuthology.orchestra.run.smithi186.stdout:(96/152): python3-babel-2.9.1-2.el9.noarch.rpm 3.9 MB/s | 6.0 MB 00:01 2024-04-25T10:02:51.863 INFO:teuthology.orchestra.run.smithi186.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 305 kB 00:00 2024-04-25T10:02:51.897 INFO:teuthology.orchestra.run.smithi186.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 988 kB/s | 58 kB 00:00 2024-04-25T10:02:51.922 INFO:teuthology.orchestra.run.smithi186.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T10:02:51.964 INFO:teuthology.orchestra.run.smithi186.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 953 kB/s | 64 kB 00:00 2024-04-25T10:02:51.989 INFO:teuthology.orchestra.run.smithi186.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 669 kB/s | 45 kB 00:00 2024-04-25T10:02:52.031 INFO:teuthology.orchestra.run.smithi186.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 334 kB/s | 22 kB 00:00 2024-04-25T10:02:52.290 INFO:teuthology.orchestra.run.smithi186.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.8 MB/s | 863 kB 00:00 2024-04-25T10:02:53.068 INFO:teuthology.orchestra.run.smithi016.stdout:(7/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x86 9.0 MB/s | 17 MB 00:01 2024-04-25T10:02:53.218 INFO:teuthology.orchestra.run.smithi016.stdout:(8/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 1.0 MB/s | 5.3 MB 00:05 2024-04-25T10:02:53.310 INFO:teuthology.orchestra.run.smithi016.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 275 kB/s | 25 kB 00:00 2024-04-25T10:02:53.623 INFO:teuthology.orchestra.run.smithi186.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.1 MB/s | 16 MB 00:02 2024-04-25T10:02:53.810 INFO:teuthology.orchestra.run.smithi016.stdout:(10/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 13 MB/s | 9.8 MB 00:00 2024-04-25T10:02:53.860 INFO:teuthology.orchestra.run.smithi016.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 4.3 MB/s | 220 kB 00:00 2024-04-25T10:02:53.927 INFO:teuthology.orchestra.run.smithi016.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 407 kB/s | 27 kB 00:00 2024-04-25T10:02:53.994 INFO:teuthology.orchestra.run.smithi016.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 468 kB/s | 31 kB 00:00 2024-04-25T10:02:54.177 INFO:teuthology.orchestra.run.smithi016.stdout:(14/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 4.0 MB/s | 744 kB 00:00 2024-04-25T10:02:54.269 INFO:teuthology.orchestra.run.smithi016.stdout:(15/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 1.7 MB/s | 163 kB 00:00 2024-04-25T10:02:54.370 INFO:teuthology.orchestra.run.smithi016.stdout:(16/152): librados-devel-19.0.0-1628.g97ae6cc9. 1.2 MB/s | 127 kB 00:00 2024-04-25T10:02:54.603 INFO:teuthology.orchestra.run.smithi016.stdout:(17/152): ceph-common-19.0.0-1628.g97ae6cc9.el9 3.0 MB/s | 20 MB 00:06 2024-04-25T10:02:54.637 INFO:teuthology.orchestra.run.smithi016.stdout:(18/152): libradosstriper1-19.0.0-1628.g97ae6cc 1.8 MB/s | 501 kB 00:00 2024-04-25T10:02:54.687 INFO:teuthology.orchestra.run.smithi016.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 905 kB/s | 45 kB 00:00 2024-04-25T10:02:54.746 INFO:teuthology.orchestra.run.smithi016.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 2.2 MB/s | 131 kB 00:00 2024-04-25T10:02:54.804 INFO:teuthology.orchestra.run.smithi016.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 2.7 MB/s | 162 kB 00:00 2024-04-25T10:02:54.896 INFO:teuthology.orchestra.run.smithi016.stdout:(22/152): python3-rados-19.0.0-1628.g97ae6cc9.e 3.4 MB/s | 321 kB 00:00 2024-04-25T10:02:54.947 INFO:teuthology.orchestra.run.smithi016.stdout:(23/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 5.8 MB/s | 298 kB 00:00 2024-04-25T10:02:55.005 INFO:teuthology.orchestra.run.smithi016.stdout:(24/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 1.7 MB/s | 99 kB 00:00 2024-04-25T10:02:55.047 INFO:teuthology.orchestra.run.smithi016.stdout:(25/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 2.0 MB/s | 85 kB 00:00 2024-04-25T10:02:55.440 INFO:teuthology.orchestra.run.smithi016.stdout:(26/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 7.9 MB/s | 3.1 MB 00:00 2024-04-25T10:02:55.507 INFO:teuthology.orchestra.run.smithi016.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 2.5 MB/s | 170 kB 00:00 2024-04-25T10:02:55.557 INFO:teuthology.orchestra.run.smithi016.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 631 kB/s | 31 kB 00:00 2024-04-25T10:02:55.632 INFO:teuthology.orchestra.run.smithi016.stdout:(29/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 5.0 MB/s | 5.1 MB 00:01 2024-04-25T10:02:55.658 INFO:teuthology.orchestra.run.smithi016.stdout:(30/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 1.3 MB/s | 138 kB 00:00 2024-04-25T10:02:56.100 INFO:teuthology.orchestra.run.smithi016.stdout:(31/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 3.8 MB/s | 1.8 MB 00:00 2024-04-25T10:02:56.200 INFO:teuthology.orchestra.run.smithi016.stdout:(32/152): ceph-mgr-modules-core-19.0.0-1628.g97 2.4 MB/s | 248 kB 00:00 2024-04-25T10:02:56.333 INFO:teuthology.orchestra.run.smithi016.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 370 kB/s | 49 kB 00:00 2024-04-25T10:02:56.417 INFO:teuthology.orchestra.run.smithi016.stdout:(34/152): ceph-prometheus-alerts-19.0.0-1628.g9 184 kB/s | 15 kB 00:00 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi016.stdout:(35/152): ceph-mgr-diskprediction-local-19.0.0- 6.2 MB/s | 7.4 MB 00:01 2024-04-25T10:02:57.034 INFO:teuthology.orchestra.run.smithi016.stdout:(36/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 4.0 MB/s | 748 kB 00:00 2024-04-25T10:02:57.110 INFO:teuthology.orchestra.run.smithi016.stdout:(37/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 243 kB/s | 18 kB 00:00 2024-04-25T10:02:57.235 INFO:teuthology.orchestra.run.smithi016.stdout:(38/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 323 kB/s | 264 kB 00:00 2024-04-25T10:02:57.293 INFO:teuthology.orchestra.run.smithi016.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 438 kB/s | 80 kB 00:00 2024-04-25T10:02:57.394 INFO:teuthology.orchestra.run.smithi016.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-25T10:02:57.486 INFO:teuthology.orchestra.run.smithi016.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-25T10:02:57.552 INFO:teuthology.orchestra.run.smithi016.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-04-25T10:02:57.611 INFO:teuthology.orchestra.run.smithi016.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T10:02:57.670 INFO:teuthology.orchestra.run.smithi016.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T10:02:57.712 INFO:teuthology.orchestra.run.smithi016.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 5.4 MB/s | 1.2 MB 00:00 2024-04-25T10:02:57.737 INFO:teuthology.orchestra.run.smithi016.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-25T10:02:57.771 INFO:teuthology.orchestra.run.smithi016.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-25T10:02:57.888 INFO:teuthology.orchestra.run.smithi016.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 4.5 MB/s | 689 kB 00:00 2024-04-25T10:02:57.946 INFO:teuthology.orchestra.run.smithi016.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 5.3 MB/s | 943 kB 00:00 2024-04-25T10:02:57.971 INFO:teuthology.orchestra.run.smithi016.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-04-25T10:02:57.997 INFO:teuthology.orchestra.run.smithi016.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 428 kB/s | 21 kB 00:00 2024-04-25T10:02:58.055 INFO:teuthology.orchestra.run.smithi016.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 564 kB/s | 33 kB 00:00 2024-04-25T10:02:58.081 INFO:teuthology.orchestra.run.smithi016.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-25T10:02:58.140 INFO:teuthology.orchestra.run.smithi016.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-04-25T10:02:58.198 INFO:teuthology.orchestra.run.smithi016.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-04-25T10:02:58.324 INFO:teuthology.orchestra.run.smithi016.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-04-25T10:02:58.407 INFO:teuthology.orchestra.run.smithi016.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 548 kB/s | 45 kB 00:00 2024-04-25T10:02:58.465 INFO:teuthology.orchestra.run.smithi016.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.3 MB/s | 3.0 MB 00:00 2024-04-25T10:02:58.599 INFO:teuthology.orchestra.run.smithi016.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-04-25T10:02:58.625 INFO:teuthology.orchestra.run.smithi016.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 3.6 MB/s | 582 kB 00:00 2024-04-25T10:02:58.658 INFO:teuthology.orchestra.run.smithi016.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-04-25T10:02:58.692 INFO:teuthology.orchestra.run.smithi016.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 710 kB/s | 47 kB 00:00 2024-04-25T10:02:58.726 INFO:teuthology.orchestra.run.smithi016.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-04-25T10:02:58.759 INFO:teuthology.orchestra.run.smithi016.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 188 kB 00:00 2024-04-25T10:02:58.784 INFO:teuthology.orchestra.run.smithi016.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 603 kB/s | 35 kB 00:00 2024-04-25T10:02:59.151 INFO:teuthology.orchestra.run.smithi016.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-04-25T10:02:59.311 INFO:teuthology.orchestra.run.smithi016.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 8.4 MB/s | 4.4 MB 00:00 2024-04-25T10:02:59.351 INFO:teuthology.orchestra.run.smithi016.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-04-25T10:02:59.435 INFO:teuthology.orchestra.run.smithi016.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-25T10:02:59.477 INFO:teuthology.orchestra.run.smithi016.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 5.4 MB/s | 911 kB 00:00 2024-04-25T10:02:59.602 INFO:teuthology.orchestra.run.smithi016.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-04-25T10:02:59.670 INFO:teuthology.orchestra.run.smithi016.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-25T10:02:59.828 INFO:teuthology.orchestra.run.smithi016.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 15 MB/s | 5.8 MB 00:00 2024-04-25T10:02:59.904 INFO:teuthology.orchestra.run.smithi016.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-25T10:02:59.979 INFO:teuthology.orchestra.run.smithi016.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-25T10:03:00.037 INFO:teuthology.orchestra.run.smithi016.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 440 kB/s | 26 kB 00:00 2024-04-25T10:03:00.096 INFO:teuthology.orchestra.run.smithi016.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-04-25T10:03:00.205 INFO:teuthology.orchestra.run.smithi016.stdout:(78/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-25T10:03:00.230 INFO:teuthology.orchestra.run.smithi016.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-04-25T10:03:00.264 INFO:teuthology.orchestra.run.smithi016.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-04-25T10:03:00.334 INFO:teuthology.orchestra.run.smithi016.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-25T10:03:00.589 INFO:teuthology.orchestra.run.smithi016.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 860 kB/s | 222 kB 00:00 2024-04-25T10:03:00.656 INFO:teuthology.orchestra.run.smithi016.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-25T10:03:00.681 INFO:teuthology.orchestra.run.smithi016.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 843 kB/s | 77 kB 00:00 2024-04-25T10:03:00.715 INFO:teuthology.orchestra.run.smithi016.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 725 kB/s | 42 kB 00:00 2024-04-25T10:03:00.740 INFO:teuthology.orchestra.run.smithi016.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-25T10:03:00.776 INFO:teuthology.orchestra.run.smithi016.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-04-25T10:03:00.809 INFO:teuthology.orchestra.run.smithi016.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.0 MB/s | 279 kB 00:00 2024-04-25T10:03:00.835 INFO:teuthology.orchestra.run.smithi016.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 870 kB/s | 51 kB 00:00 2024-04-25T10:03:00.868 INFO:teuthology.orchestra.run.smithi016.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 915 kB/s | 54 kB 00:00 2024-04-25T10:03:00.927 INFO:teuthology.orchestra.run.smithi016.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 715 kB/s | 42 kB 00:00 2024-04-25T10:03:00.994 INFO:teuthology.orchestra.run.smithi016.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 664 kB/s | 44 kB 00:00 2024-04-25T10:03:01.061 INFO:teuthology.orchestra.run.smithi016.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-25T10:03:01.111 INFO:teuthology.orchestra.run.smithi016.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 720 kB/s | 36 kB 00:00 2024-04-25T10:03:01.170 INFO:teuthology.orchestra.run.smithi016.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-04-25T10:03:01.245 INFO:teuthology.orchestra.run.smithi016.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-25T10:03:01.304 INFO:teuthology.orchestra.run.smithi016.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 990 kB/s | 58 kB 00:00 2024-04-25T10:03:01.371 INFO:teuthology.orchestra.run.smithi016.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-25T10:03:01.546 INFO:teuthology.orchestra.run.smithi016.stdout:(99/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-04-25T10:03:01.563 INFO:teuthology.orchestra.run.smithi016.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 332 kB/s | 64 kB 00:00 2024-04-25T10:03:01.597 INFO:teuthology.orchestra.run.smithi016.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-04-25T10:03:01.613 INFO:teuthology.orchestra.run.smithi016.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-04-25T10:03:01.697 INFO:teuthology.orchestra.run.smithi016.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 8.4 MB/s | 863 kB 00:00 2024-04-25T10:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-25T10:03:02.174 INFO:teuthology.orchestra.run.smithi016.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 690 kB/s | 309 kB 00:00 2024-04-25T10:03:02.214 INFO:teuthology.orchestra.run.smithi016.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-04-25T10:03:02.281 INFO:teuthology.orchestra.run.smithi016.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 389 kB/s | 26 kB 00:00 2024-04-25T10:03:02.357 INFO:teuthology.orchestra.run.smithi016.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 649 kB/s | 49 kB 00:00 2024-04-25T10:03:02.432 INFO:teuthology.orchestra.run.smithi016.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 897 kB/s | 67 kB 00:00 2024-04-25T10:03:02.507 INFO:teuthology.orchestra.run.smithi016.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi016.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-04-25T10:03:02.599 INFO:teuthology.orchestra.run.smithi016.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-04-25T10:03:02.633 INFO:teuthology.orchestra.run.smithi016.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-04-25T10:03:02.675 INFO:teuthology.orchestra.run.smithi016.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 578 kB/s | 43 kB 00:00 2024-04-25T10:03:02.708 INFO:teuthology.orchestra.run.smithi016.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 430 kB/s | 32 kB 00:00 2024-04-25T10:03:02.759 INFO:teuthology.orchestra.run.smithi016.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-04-25T10:03:02.793 INFO:teuthology.orchestra.run.smithi016.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-25T10:03:02.835 INFO:teuthology.orchestra.run.smithi016.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-25T10:03:02.885 INFO:teuthology.orchestra.run.smithi016.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-25T10:03:02.910 INFO:teuthology.orchestra.run.smithi016.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 469 kB/s | 35 kB 00:00 2024-04-25T10:03:02.969 INFO:teuthology.orchestra.run.smithi016.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-25T10:03:03.019 INFO:teuthology.orchestra.run.smithi016.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-04-25T10:03:03.053 INFO:teuthology.orchestra.run.smithi016.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-04-25T10:03:03.094 INFO:teuthology.orchestra.run.smithi016.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-25T10:03:03.128 INFO:teuthology.orchestra.run.smithi016.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-04-25T10:03:03.170 INFO:teuthology.orchestra.run.smithi016.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 308 kB/s | 23 kB 00:00 2024-04-25T10:03:03.204 INFO:teuthology.orchestra.run.smithi016.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-04-25T10:03:03.245 INFO:teuthology.orchestra.run.smithi016.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-04-25T10:03:03.321 INFO:teuthology.orchestra.run.smithi016.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 617 kB/s | 46 kB 00:00 2024-04-25T10:03:03.363 INFO:teuthology.orchestra.run.smithi016.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-04-25T10:03:03.405 INFO:teuthology.orchestra.run.smithi016.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 945 kB/s | 79 kB 00:00 2024-04-25T10:03:03.438 INFO:teuthology.orchestra.run.smithi016.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 769 kB/s | 58 kB 00:00 2024-04-25T10:03:03.480 INFO:teuthology.orchestra.run.smithi016.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-25T10:03:03.514 INFO:teuthology.orchestra.run.smithi016.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-04-25T10:03:03.556 INFO:teuthology.orchestra.run.smithi016.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-25T10:03:03.581 INFO:teuthology.orchestra.run.smithi186.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 1.9 kB/s | 19 kB 00:09 2024-04-25T10:03:03.589 INFO:teuthology.orchestra.run.smithi016.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-04-25T10:03:03.631 INFO:teuthology.orchestra.run.smithi016.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-04-25T10:03:03.665 INFO:teuthology.orchestra.run.smithi016.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-04-25T10:03:03.707 INFO:teuthology.orchestra.run.smithi016.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 787 kB/s | 59 kB 00:00 2024-04-25T10:03:03.732 INFO:teuthology.orchestra.run.smithi016.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 534 kB/s | 36 kB 00:00 2024-04-25T10:03:03.773 INFO:teuthology.orchestra.run.smithi186.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 27 kB/s | 309 kB 00:11 2024-04-25T10:03:03.782 INFO:teuthology.orchestra.run.smithi016.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 679 kB/s | 51 kB 00:00 2024-04-25T10:03:03.816 INFO:teuthology.orchestra.run.smithi016.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-25T10:03:03.858 INFO:teuthology.orchestra.run.smithi016.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-25T10:03:03.900 INFO:teuthology.orchestra.run.smithi016.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-25T10:03:03.933 INFO:teuthology.orchestra.run.smithi016.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-25T10:03:04.367 INFO:teuthology.orchestra.run.smithi016.stdout:(146/152): ceph-test-19.0.0-1628.g97ae6cc9.el9. 4.3 MB/s | 47 MB 00:11 2024-04-25T10:03:04.414 INFO:teuthology.orchestra.run.smithi016.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 372 kB/s | 192 kB 00:00 2024-04-25T10:03:04.448 INFO:teuthology.orchestra.run.smithi186.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 38 kB/s | 26 kB 00:00 2024-04-25T10:03:04.459 INFO:teuthology.orchestra.run.smithi016.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 3.0 MB/s | 1.6 MB 00:00 2024-04-25T10:03:04.473 INFO:teuthology.orchestra.run.smithi186.stdout:(108/152): abseil-cpp-20211102.0-3.el9.x86_64.r 44 kB/s | 548 kB 00:12 2024-04-25T10:03:04.632 INFO:teuthology.orchestra.run.smithi186.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 264 kB/s | 49 kB 00:00 2024-04-25T10:03:04.685 INFO:teuthology.orchestra.run.smithi016.stdout:(149/152): librados2-19.0.0-1628.g97ae6cc9.el9. 11 MB/s | 3.4 MB 00:00 2024-04-25T10:03:04.798 INFO:teuthology.orchestra.run.smithi186.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 207 kB/s | 67 kB 00:00 2024-04-25T10:03:04.852 INFO:teuthology.orchestra.run.smithi016.stdout:(150/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 7.2 MB/s | 3.1 MB 00:00 2024-04-25T10:03:05.149 INFO:teuthology.orchestra.run.smithi186.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 292 kB/s | 151 kB 00:00 2024-04-25T10:03:05.291 INFO:teuthology.orchestra.run.smithi186.stdout:(112/152): libarrow-9.0.0-10.el9.x86_64.rpm 2.6 MB/s | 4.4 MB 00:01 2024-04-25T10:03:05.524 INFO:teuthology.orchestra.run.smithi186.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 186 kB/s | 43 kB 00:00 2024-04-25T10:03:05.593 INFO:teuthology.orchestra.run.smithi016.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.6 MB/s | 2.4 MB 00:00 2024-04-25T10:03:05.627 INFO:teuthology.orchestra.run.smithi016.stdout:(152/152): openssl-3.2.1-1.el9.x86_64.rpm 1.2 MB/s | 1.3 MB 00:01 2024-04-25T10:03:05.633 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:03:05.633 INFO:teuthology.orchestra.run.smithi016.stdout:Total 9.9 MB/s | 215 MB 00:21 2024-04-25T10:03:05.633 INFO:teuthology.orchestra.run.smithi186.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.0 MB/s | 838 kB 00:00 2024-04-25T10:03:05.675 INFO:teuthology.orchestra.run.smithi186.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 214 kB/s | 32 kB 00:00 2024-04-25T10:03:05.734 INFO:teuthology.orchestra.run.smithi186.stdout:(116/152): python3-asyncssh-2.13.2-5.el9.noarch 938 kB/s | 548 kB 00:00 2024-04-25T10:03:05.750 INFO:teuthology.orchestra.run.smithi186.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-04-25T10:03:06.192 INFO:teuthology.orchestra.run.smithi186.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 332 kB/s | 172 kB 00:00 2024-04-25T10:03:06.217 INFO:teuthology.orchestra.run.smithi186.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 741 kB/s | 358 kB 00:00 2024-04-25T10:03:06.268 INFO:teuthology.orchestra.run.smithi186.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 470 kB/s | 35 kB 00:00 2024-04-25T10:03:06.318 INFO:teuthology.orchestra.run.smithi186.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-04-25T10:03:06.418 INFO:teuthology.orchestra.run.smithi186.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-25T10:03:06.443 INFO:teuthology.orchestra.run.smithi186.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-25T10:03:06.719 INFO:teuthology.orchestra.run.smithi186.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 36 kB/s | 11 kB 00:00 2024-04-25T10:03:06.744 INFO:teuthology.orchestra.run.smithi186.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 59 kB/s | 18 kB 00:00 2024-04-25T10:03:06.767 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-25T10:03:06.803 INFO:teuthology.orchestra.run.smithi186.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 278 kB/s | 23 kB 00:00 2024-04-25T10:03:06.828 INFO:teuthology.orchestra.run.smithi186.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-04-25T10:03:06.878 INFO:teuthology.orchestra.run.smithi186.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-04-25T10:03:06.926 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-25T10:03:06.926 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-25T10:03:06.978 INFO:teuthology.orchestra.run.smithi186.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-04-25T10:03:07.037 INFO:teuthology.orchestra.run.smithi186.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-04-25T10:03:07.062 INFO:teuthology.orchestra.run.smithi186.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 945 kB/s | 79 kB 00:00 2024-04-25T10:03:07.138 INFO:teuthology.orchestra.run.smithi186.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 576 kB/s | 58 kB 00:00 2024-04-25T10:03:07.162 INFO:teuthology.orchestra.run.smithi186.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-25T10:03:07.621 INFO:teuthology.orchestra.run.smithi186.stdout:(134/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 196 kB/s | 90 kB 00:00 2024-04-25T10:03:07.646 INFO:teuthology.orchestra.run.smithi186.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 32 kB/s | 16 kB 00:00 2024-04-25T10:03:07.671 INFO:teuthology.orchestra.run.smithi186.stdout:(136/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 158 kB/s | 303 kB 00:01 2024-04-25T10:03:07.905 INFO:teuthology.orchestra.run.smithi186.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 119 kB/s | 31 kB 00:00 2024-04-25T10:03:07.931 INFO:teuthology.orchestra.run.smithi186.stdout:(138/152): python3-pyperclip-1.8.2-2.el9.noarch 91 kB/s | 28 kB 00:00 2024-04-25T10:03:08.348 INFO:teuthology.orchestra.run.smithi186.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 86 kB/s | 36 kB 00:00 2024-04-25T10:03:08.373 INFO:teuthology.orchestra.run.smithi186.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 126 kB/s | 59 kB 00:00 2024-04-25T10:03:08.456 INFO:teuthology.orchestra.run.smithi186.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 470 kB/s | 51 kB 00:00 2024-04-25T10:03:08.757 INFO:teuthology.orchestra.run.smithi186.stdout:(142/152): python3-websocket-client-1.2.3-2.el9 299 kB/s | 90 kB 00:00 2024-04-25T10:03:08.766 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-25T10:03:08.766 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-25T10:03:08.782 INFO:teuthology.orchestra.run.smithi186.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 561 kB/s | 230 kB 00:00 2024-04-25T10:03:08.916 INFO:teuthology.orchestra.run.smithi186.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 150 kB/s | 20 kB 00:00 2024-04-25T10:03:08.941 INFO:teuthology.orchestra.run.smithi186.stdout:(145/152): python3-routes-2.5.1-5.el9.noarch.rp 149 kB/s | 188 kB 00:01 2024-04-25T10:03:08.966 INFO:teuthology.orchestra.run.smithi186.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.0 MB/s | 427 kB 00:00 2024-04-25T10:03:09.025 INFO:teuthology.orchestra.run.smithi186.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-25T10:03:09.333 INFO:teuthology.orchestra.run.smithi186.stdout:(148/152): librados2-19.0.0-1628.g97ae6cc9.el9. 9.2 MB/s | 3.4 MB 00:00 2024-04-25T10:03:09.384 INFO:teuthology.orchestra.run.smithi186.stdout:(149/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 8.7 MB/s | 3.1 MB 00:00 2024-04-25T10:03:09.434 INFO:teuthology.orchestra.run.smithi186.stdout:(150/152): thrift-0.15.0-2.el9.x86_64.rpm 3.2 MB/s | 1.6 MB 00:00 2024-04-25T10:03:09.551 INFO:teuthology.orchestra.run.smithi186.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 6.2 MB/s | 1.3 MB 00:00 2024-04-25T10:03:09.609 INFO:teuthology.orchestra.run.smithi186.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-04-25T10:03:09.616 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:03:09.616 INFO:teuthology.orchestra.run.smithi186.stdout:Total 5.0 MB/s | 215 MB 00:42 2024-04-25T10:03:10.656 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-25T10:03:10.801 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-25T10:03:10.801 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-25T10:03:12.694 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-25T10:03:12.694 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-25T10:03:14.890 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-25T10:03:14.989 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T10:03:15.205 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T10:03:15.267 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T10:03:15.308 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T10:03:15.332 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T10:03:15.378 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T10:03:15.729 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T10:03:15.834 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T10:03:15.958 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T10:03:15.984 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T10:03:16.069 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T10:03:16.106 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T10:03:16.262 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T10:03:16.311 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T10:03:16.339 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T10:03:16.444 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T10:03:16.473 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T10:03:16.536 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T10:03:16.557 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:03:16.583 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:03:16.784 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:03:16.816 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:03:16.927 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T10:03:17.027 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T10:03:17.065 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T10:03:17.101 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T10:03:17.176 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T10:03:17.217 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T10:03:17.260 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T10:03:17.292 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T10:03:17.320 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T10:03:17.393 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T10:03:17.451 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T10:03:17.490 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T10:03:17.533 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T10:03:17.611 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T10:03:17.644 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T10:03:17.688 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T10:03:17.716 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T10:03:17.788 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T10:03:17.831 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T10:03:17.908 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T10:03:18.053 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T10:03:18.109 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T10:03:18.625 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-25T10:03:18.662 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T10:03:18.696 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T10:03:18.743 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T10:03:18.756 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T10:03:18.791 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T10:03:18.797 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T10:03:18.830 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T10:03:18.834 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T10:03:18.855 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T10:03:18.872 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T10:03:18.889 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T10:03:18.914 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T10:03:19.082 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T10:03:19.134 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T10:03:19.190 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T10:03:19.259 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T10:03:19.264 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T10:03:19.300 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T10:03:19.337 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T10:03:19.365 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T10:03:19.385 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T10:03:19.420 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T10:03:19.463 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T10:03:19.486 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T10:03:19.505 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T10:03:19.516 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T10:03:19.589 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T10:03:19.620 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T10:03:19.651 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T10:03:19.694 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T10:03:19.724 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T10:03:19.766 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T10:03:19.775 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T10:03:19.833 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T10:03:19.861 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T10:03:19.872 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T10:03:19.907 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T10:03:19.948 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T10:03:19.964 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T10:03:19.981 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T10:03:19.987 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T10:03:20.006 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T10:03:20.049 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T10:03:20.071 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:03:20.075 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T10:03:20.092 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:03:20.214 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T10:03:20.246 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T10:03:20.271 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:03:20.274 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T10:03:20.304 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:03:20.309 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T10:03:20.348 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T10:03:20.388 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T10:03:20.413 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T10:03:20.418 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T10:03:20.452 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T10:03:20.485 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T10:03:20.517 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T10:03:20.527 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T10:03:20.554 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T10:03:20.561 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T10:03:20.589 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T10:03:20.651 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T10:03:20.652 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T10:03:20.687 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T10:03:20.700 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T10:03:20.731 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T10:03:20.731 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T10:03:20.763 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T10:03:20.779 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T10:03:20.791 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T10:03:20.860 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T10:03:20.892 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T10:03:20.913 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T10:03:20.944 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T10:03:20.951 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T10:03:20.980 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T10:03:21.055 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T10:03:21.090 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T10:03:21.133 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T10:03:21.162 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T10:03:21.248 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T10:03:21.293 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T10:03:21.367 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T10:03:21.500 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T10:03:21.554 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T10:03:21.709 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T10:03:21.755 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T10:03:21.789 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T10:03:21.899 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T10:03:21.947 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T10:03:22.030 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T10:03:22.066 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T10:03:22.077 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T10:03:22.158 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T10:03:22.195 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T10:03:22.230 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T10:03:22.252 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T10:03:22.268 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T10:03:22.280 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T10:03:22.309 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T10:03:22.360 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T10:03:22.478 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T10:03:22.554 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T10:03:22.609 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T10:03:22.681 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T10:03:22.713 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T10:03:22.750 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T10:03:22.796 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T10:03:22.832 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T10:03:22.875 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T10:03:22.917 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T10:03:23.041 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T10:03:23.055 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T10:03:23.098 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T10:03:23.128 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T10:03:23.150 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T10:03:23.169 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T10:03:23.262 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T10:03:23.294 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T10:03:23.335 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T10:03:23.360 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T10:03:23.385 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T10:03:23.459 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T10:03:23.595 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T10:03:23.634 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T10:03:23.661 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T10:03:23.695 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T10:03:23.735 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T10:03:23.766 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T10:03:23.797 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T10:03:23.823 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T10:03:23.856 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T10:03:23.898 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T10:03:23.932 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T10:03:24.019 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T10:03:24.067 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T10:03:24.305 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T10:03:24.348 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T10:03:24.446 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T10:03:24.451 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T10:03:24.494 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T10:03:24.526 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T10:03:24.554 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T10:03:24.587 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T10:03:24.935 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T10:03:25.191 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T10:03:25.217 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T10:03:25.267 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T10:03:25.283 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T10:03:25.300 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T10:03:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T10:03:25.341 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T10:03:25.398 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T10:03:25.466 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T10:03:25.557 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T10:03:25.602 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T10:03:25.783 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T10:03:25.817 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T10:03:25.821 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T10:03:25.853 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T10:03:25.895 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T10:03:25.924 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T10:03:25.954 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T10:03:26.544 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T10:03:26.657 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T10:03:27.894 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T10:03:27.979 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T10:03:28.006 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T10:03:28.040 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T10:03:28.264 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T10:03:28.301 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T10:03:28.352 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T10:03:28.386 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T10:03:28.477 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T10:03:28.507 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T10:03:28.543 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T10:03:28.870 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T10:03:28.913 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T10:03:28.995 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T10:03:28.995 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T10:03:28.995 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T10:03:28.995 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:03:28.998 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T10:03:29.022 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T10:03:29.039 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T10:03:29.109 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T10:03:29.132 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T10:03:31.304 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T10:03:31.353 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T10:03:31.894 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T10:03:31.940 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T10:03:32.038 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T10:03:32.039 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T10:03:32.039 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T10:03:32.039 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:03:32.066 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T10:03:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T10:03:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-04-25T10:03:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-04-25T10:03:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-04-25T10:03:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-04-25T10:03:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-04-25T10:03:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-04-25T10:03:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-04-25T10:03:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:03:40.991 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T10:03:41.008 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T10:03:41.008 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:03:41.008 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:03:41.008 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:03:42.546 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T10:03:42.573 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T10:03:42.573 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:03:42.574 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:03:42.574 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:03:42.574 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:03:42.574 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:03:42.796 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T10:03:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T10:03:42.821 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:03:42.821 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:03:42.821 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:03:42.821 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:03:42.821 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:03:43.289 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T10:03:43.314 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T10:03:43.314 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:03:43.314 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:03:43.314 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:03:43.314 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:03:43.314 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:03:43.407 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T10:03:43.407 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /sys 2024-04-25T10:03:43.407 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /proc 2024-04-25T10:03:43.407 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /mnt 2024-04-25T10:03:43.407 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /var/tmp 2024-04-25T10:03:43.407 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /home 2024-04-25T10:03:43.408 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /root 2024-04-25T10:03:43.408 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /tmp 2024-04-25T10:03:43.408 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:03:43.428 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:03:43.486 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:03:43.550 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:03:43.598 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T10:03:43.617 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T10:03:43.617 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:03:43.617 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:03:43.617 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:03:43.746 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:03:43.799 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:03:43.878 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:03:43.970 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:03:44.111 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:03:44.157 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:03:44.188 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:03:44.216 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:03:44.551 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T10:03:44.581 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T10:03:44.605 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T10:03:44.668 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T10:03:44.696 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T10:03:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T10:03:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T10:03:45.080 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T10:03:45.106 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T10:03:45.106 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:03:45.106 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:03:45.106 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:03:45.106 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:03:45.106 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:03:45.314 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T10:03:45.336 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T10:03:45.337 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:03:45.337 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:03:45.337 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:03:45.337 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:03:45.337 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:03:45.782 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T10:03:45.803 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T10:03:45.804 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:03:45.804 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:03:45.804 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:03:45.804 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:03:45.804 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:03:45.903 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:03:45.951 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:03:45.981 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T10:03:46.006 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T10:03:46.006 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:03:46.112 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T10:03:46.193 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:03:46.248 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:03:46.276 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T10:03:46.307 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T10:03:46.309 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:03:46.393 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T10:03:46.398 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:03:46.404 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T10:03:46.504 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T10:03:46.537 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:03:46.557 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T10:03:46.574 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:03:46.605 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:03:46.633 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:03:47.042 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T10:03:47.082 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T10:03:47.114 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T10:03:47.183 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T10:03:47.214 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T10:03:47.443 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T10:03:47.470 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:03:47.472 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T10:03:47.503 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T10:03:47.527 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T10:03:47.527 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:03:47.527 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:03:47.527 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:03:47.527 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:03:47.527 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:03:47.570 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T10:03:47.593 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T10:03:47.593 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:03:47.593 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:03:47.593 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:03:47.637 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T10:03:47.661 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T10:03:47.661 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:03:47.661 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:03:47.661 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:03:47.661 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:03:47.661 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:03:47.979 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T10:03:48.004 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T10:03:48.004 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:03:48.004 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:03:48.004 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:03:48.004 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:03:48.004 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:03:48.442 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T10:03:48.464 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T10:03:48.565 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T10:03:48.728 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T10:03:48.758 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T10:03:48.781 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T10:03:48.781 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:03:48.781 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:03:48.781 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:03:48.781 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:03:48.781 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:03:48.825 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T10:03:48.838 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T10:03:48.933 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T10:03:48.990 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T10:03:49.841 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:03:49.869 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T10:03:49.892 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T10:03:49.892 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:03:49.892 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:03:49.892 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:03:49.892 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:03:49.892 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:03:49.937 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T10:03:49.958 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T10:03:49.958 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:03:49.958 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:03:49.959 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:03:50.004 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T10:03:50.026 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T10:03:50.026 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:03:50.026 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:03:50.026 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:03:50.026 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:03:50.026 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:03:50.338 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T10:03:50.360 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T10:03:50.361 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:03:50.361 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:03:50.361 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:03:50.361 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:03:50.361 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:03:52.683 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T10:03:52.725 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T10:03:52.775 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T10:03:52.806 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:03:52.833 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:03:53.025 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T10:03:53.071 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:03:53.100 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T10:03:53.685 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:03:53.723 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:03:53.744 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:03:54.641 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T10:03:54.728 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:03:54.766 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T10:03:54.770 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:03:54.807 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T10:03:54.848 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T10:03:54.854 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:03:54.881 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:03:54.907 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:03:54.907 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:03:54.908 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:03:55.000 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:03:55.033 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:03:55.033 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:03:55.089 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T10:03:55.137 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:03:55.139 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:03:55.167 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T10:03:55.217 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:03:55.795 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:03:55.831 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:03:55.850 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:03:56.696 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T10:03:56.794 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:03:56.838 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:03:56.915 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:03:56.956 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:03:56.956 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:03:57.042 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:03:57.073 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:03:57.073 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:03:57.159 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:03:57.229 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:04:00.657 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:04:00.657 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T10:04:00.658 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T10:04:00.658 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T10:04:00.658 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T10:04:00.658 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T10:04:00.658 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T10:04:00.658 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T10:04:00.658 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T10:04:00.658 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T10:04:00.658 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T10:04:00.658 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T10:04:00.658 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T10:04:00.658 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T10:04:00.659 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T10:04:00.659 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T10:04:00.659 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T10:04:00.659 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T10:04:00.659 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T10:04:00.659 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T10:04:00.659 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T10:04:00.659 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T10:04:00.659 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T10:04:00.659 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T10:04:00.659 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T10:04:00.659 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T10:04:00.659 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T10:04:00.659 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T10:04:00.660 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T10:04:00.660 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T10:04:00.660 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T10:04:00.660 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T10:04:00.660 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T10:04:00.660 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T10:04:00.660 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T10:04:00.660 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T10:04:00.660 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T10:04:00.660 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T10:04:00.660 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T10:04:00.660 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T10:04:00.660 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:04:00.660 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:04:00.661 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:04:00.661 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:04:00.661 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:04:00.661 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:04:00.661 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:04:00.661 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:04:00.661 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:04:00.661 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:04:00.661 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:04:00.661 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:04:00.661 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:04:00.661 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:04:00.661 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:04:00.661 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:04:00.661 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:04:00.662 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:04:00.662 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:04:00.662 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:04:00.662 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:04:00.662 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:04:00.662 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:04:00.662 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:04:00.662 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:04:00.662 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:04:00.662 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:04:00.662 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:04:00.662 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:04:00.662 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:04:00.663 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:04:00.663 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:04:00.663 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:04:00.663 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:04:00.663 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:04:00.663 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:04:00.663 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:04:00.663 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:04:00.663 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:04:00.663 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:04:00.663 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:04:00.663 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:04:00.663 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:04:00.663 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:04:00.664 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:04:00.664 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:04:00.664 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:04:00.664 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:04:00.664 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:04:00.664 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:04:00.664 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:04:00.664 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:04:00.664 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:04:00.664 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:04:00.664 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:04:00.664 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:04:00.664 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:04:00.664 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:04:00.664 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:04:00.665 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:04:00.665 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:04:00.665 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:04:00.665 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:04:00.665 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:04:00.665 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:04:00.665 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:04:00.665 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:04:00.665 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:04:00.665 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:04:00.665 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:04:00.665 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:04:00.665 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:04:00.665 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:04:00.665 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:04:00.666 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:04:00.666 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:04:00.666 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:04:00.666 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:04:00.666 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:04:00.666 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:04:00.666 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:04:00.666 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:04:00.666 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:04:00.666 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:04:00.666 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:04:00.666 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:04:00.666 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:04:00.666 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:04:00.667 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:04:00.667 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:04:00.667 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:04:00.667 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:04:00.667 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:04:00.667 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:04:00.667 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:04:00.667 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:04:00.667 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:04:00.667 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:04:00.667 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:04:00.667 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:04:00.667 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:04:00.668 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:04:00.668 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:04:00.668 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:04:00.668 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:04:00.668 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:04:00.668 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:04:00.668 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:04:00.668 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:04:00.668 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T10:04:00.668 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:04:00.668 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T10:04:00.668 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:04:00.668 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:04:00.668 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:04:00.669 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:04:02.573 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:04:02.573 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T10:04:02.573 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T10:04:02.573 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T10:04:02.573 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T10:04:02.573 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T10:04:02.573 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T10:04:02.573 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T10:04:02.574 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T10:04:02.574 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T10:04:02.574 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T10:04:02.574 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T10:04:02.574 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T10:04:02.574 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T10:04:02.574 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T10:04:02.574 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T10:04:02.574 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T10:04:02.574 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T10:04:02.574 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T10:04:02.574 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T10:04:02.574 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T10:04:02.574 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T10:04:02.575 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T10:04:02.575 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T10:04:02.575 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T10:04:02.575 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T10:04:02.575 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T10:04:02.575 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T10:04:02.575 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T10:04:02.575 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T10:04:02.575 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T10:04:02.575 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T10:04:02.575 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T10:04:02.575 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T10:04:02.575 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T10:04:02.575 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T10:04:02.576 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T10:04:02.576 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T10:04:02.576 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T10:04:02.576 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T10:04:02.576 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:04:02.576 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:04:02.576 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:04:02.576 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:04:02.576 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:04:02.576 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:04:02.576 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:04:02.576 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:04:02.576 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:04:02.576 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:04:02.577 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:04:02.577 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:04:02.577 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:04:02.577 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:04:02.577 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:04:02.577 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:04:02.577 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:04:02.577 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:04:02.577 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:04:02.577 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:04:02.577 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:04:02.577 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:04:02.577 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:04:02.577 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:04:02.577 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:04:02.579 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:04:02.579 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:04:02.579 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:04:02.579 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:04:02.579 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:04:02.579 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:04:02.579 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:04:02.581 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:04:02.581 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:04:02.581 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:04:02.581 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:04:02.581 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:04:02.581 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:04:02.581 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:04:02.581 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:04:02.581 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:04:02.581 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:04:02.581 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:04:02.582 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:04:02.582 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:04:02.582 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:04:02.582 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:04:02.582 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:04:02.582 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:04:02.582 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:04:02.582 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:04:02.582 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:04:02.582 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:04:02.582 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:04:02.582 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:04:02.582 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:04:02.585 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:04:02.585 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:04:02.585 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:04:02.585 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:04:02.585 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:04:02.585 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:04:02.585 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:04:02.585 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:04:02.585 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:04:02.585 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:04:02.585 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:04:02.585 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:04:02.585 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:04:02.585 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:04:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:04:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:02.893 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-04-25T10:04:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:04:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:04:02.894 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-25T10:04:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:04:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:04:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:04:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:04:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:04:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:04:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:04:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:04:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:04:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:04:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:04:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:04:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:04:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:04:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:04:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:04:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:04:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:04:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:04:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:04:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:04:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:04:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:04:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:04:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:04:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:04:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:04:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:04:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:04:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:04:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:04:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:04:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:04:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:04:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:04:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:04:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:04:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:04:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:04:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:04:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:04:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:04:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:04:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:04:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:04:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:04:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:04:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:04:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:04:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:04:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:04:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:04:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:04:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:04:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:04:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:04:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:04:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:04:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:04:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:04:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:04:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:04:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:04:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:04:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:04:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:04:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:04:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:04:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:04:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:04:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:04:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:04:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:04:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:04:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:04:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:04:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:04:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:04:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:04:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:04:02.902 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:04:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:04:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:04:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:04:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:04:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:04:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:04:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:04:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:04:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:04:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:04:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:04:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:04:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:04:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:04:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:04:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:04:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:04:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:04:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:04:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:04:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:04:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:04:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:04:02.904 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:04:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:04:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:04:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:04:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:04:02.905 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:02.905 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-25T10:04:03.260 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:04.623 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:04:04.623 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:04.623 INFO:teuthology.orchestra.run.smithi186.stdout:Upgraded: 2024-04-25T10:04:04.623 INFO:teuthology.orchestra.run.smithi186.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.623 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:04:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:04:04.625 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-25T10:04:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:04:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:04:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:04:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:04:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:04:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:04:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:04:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:04:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:04:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:04:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:04:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:04:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:04:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:04:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:04:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:04:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:04:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:04:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:04:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:04:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:04:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:04:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:04:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:04:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:04:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:04:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:04:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:04:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:04:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:04:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:04:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:04:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:04:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:04:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:04:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:04:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:04:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:04:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:04:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:04:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:04:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:04:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:04:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:04:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:04:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:04:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:04:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:04:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:04:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:04:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:04:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:04:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:04:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:04:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:04:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:04:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:04:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:04:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:04:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:04:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:04:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:04:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:04:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:04:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:04:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:04:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:04:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:04:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:04:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:04:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:04:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:04:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:04:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:04:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:04:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:04:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:04:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:04:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:04:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:04:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:04:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:04:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:04:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:04:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:04:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:04:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:04:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:04:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:04:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:04:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:04:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:04:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:04:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:04:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:04:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:04:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:04:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:04:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:04:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:04:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:04:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:04:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:04:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:04:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:04:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:04:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:04:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:04:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:04:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:04:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:04:04.640 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:04.640 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-25T10:04:04.983 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:04.984 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:04:05.131 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:04:05.153 INFO:teuthology.orchestra.run.smithi016.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T10:04:05.154 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T10:04:05.154 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T10:04:05.156 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:04:05.295 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:04:05.315 INFO:teuthology.orchestra.run.smithi186.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T10:04:05.316 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T10:04:05.316 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T10:04:05.317 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T10:04:05.318 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:05.318 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:04:05.345 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:05.345 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:04:05.384 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T10:04:05.385 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:05.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:04:05.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:04:05.473 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:05.473 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:04:05.497 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:04:05.560 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T10:04:05.561 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:05.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:04:05.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:04:05.648 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:05.649 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:04:05.673 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:04:05.739 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T10:04:05.740 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:05.740 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:04:05.765 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:04:05.828 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:05.828 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:04:05.851 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:04:05.915 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T10:04:06.008 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T10:04:06.009 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /var/log/ceph 2024-04-25T10:04:06.011 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 777 /var/log/ceph 2024-04-25T10:04:06.035 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T10:04:06.035 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:04:06.076 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:04:06.100 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T10:04:06.100 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:04:06.141 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:04:06.172 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T10:04:06.172 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T10:04:06.172 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T10:04:06.173 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:06.173 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:04:06.219 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:06.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:06.369 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:06.369 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:04:06.384 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:06.494 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:06.537 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T10:04:06.537 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T10:04:06.538 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673050', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7673050', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'kclient': {'syntax': 'v1'}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1124, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi016.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN2iYC7Ugex++yKosbzMawOwzxL38DS3dqI7MUwotBAPrDEPKnVJV6b+sq62SjlSANplv7au1YWEDpu7Eqc6gng=', 'smithi186.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA+Cc6WkzlJ+cSqMPMFeH+1li8tOLIdJJ4YKeh6QMd6OYTqIhlhT9xtmFNC+tjuTgMktZjWOZzrs6ibdQ63aeNE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_09:36:50', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T10:04:06.538 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:04:06.541 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:04:06.587 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:04:06.597 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:04:06.651 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:06.652 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:04:06.676 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:04:06.676 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-04-25T10:04:06.731 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:04:06.731 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:04:06.731 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T10:04:06.732 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:04:06.732 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:04:06.732 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-04-25 10:03:55.580030685 +0000 2024-04-25T10:04:06.732 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-04-25 10:01:17.477760958 +0000 2024-04-25T10:04:06.732 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-04-25 10:01:17.477760958 +0000 2024-04-25T10:04:06.732 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-04-25 10:01:17.477760958 +0000 2024-04-25T10:04:06.732 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:04:06.795 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-04-25T10:04:06.795 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-04-25T10:04:06.795 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000104377 s, 4.9 MB/s 2024-04-25T10:04:06.796 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:04:06.853 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-04-25T10:04:06.909 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:04:06.909 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:04:06.909 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T10:04:06.909 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:04:06.910 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:04:06.910 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-04-25 10:03:55.581030670 +0000 2024-04-25T10:04:06.910 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-04-25 10:01:17.825755378 +0000 2024-04-25T10:04:06.910 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-04-25 10:01:17.825755378 +0000 2024-04-25T10:04:06.910 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-04-25 10:01:17.825755378 +0000 2024-04-25T10:04:06.910 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:04:06.972 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-04-25T10:04:06.972 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-04-25T10:04:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 8.739e-05 s, 5.9 MB/s 2024-04-25T10:04:06.974 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:04:07.031 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-04-25T10:04:07.086 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:04:07.086 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:04:07.087 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T10:04:07.087 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:04:07.087 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:04:07.087 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-04-25 10:03:55.581030670 +0000 2024-04-25T10:04:07.087 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-04-25 10:01:18.165749926 +0000 2024-04-25T10:04:07.087 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-04-25 10:01:18.165749926 +0000 2024-04-25T10:04:07.087 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-04-25 10:01:18.165749926 +0000 2024-04-25T10:04:07.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:04:07.148 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-04-25T10:04:07.148 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-04-25T10:04:07.148 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000100156 s, 5.1 MB/s 2024-04-25T10:04:07.149 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:04:07.206 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-04-25T10:04:07.261 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:04:07.261 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:04:07.261 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T10:04:07.261 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:04:07.261 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:04:07.261 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-04-25 10:03:55.581030670 +0000 2024-04-25T10:04:07.261 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-04-25 10:01:18.513744345 +0000 2024-04-25T10:04:07.261 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-04-25 10:01:18.513744345 +0000 2024-04-25T10:04:07.261 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-04-25 10:01:18.513744345 +0000 2024-04-25T10:04:07.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:04:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-04-25T10:04:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-04-25T10:04:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000120871 s, 4.2 MB/s 2024-04-25T10:04:07.324 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:04:07.380 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:07.381 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:07.381 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:04:07.396 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:04:07.396 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_1 2024-04-25T10:04:07.450 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:04:07.451 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:04:07.451 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T10:04:07.451 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:04:07.451 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:04:07.451 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-25 10:03:57.591113842 +0000 2024-04-25T10:04:07.451 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-25 10:01:15.078638238 +0000 2024-04-25T10:04:07.451 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-25 10:01:15.078638238 +0000 2024-04-25T10:04:07.451 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-25 10:01:15.078638238 +0000 2024-04-25T10:04:07.451 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:04:07.513 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-25T10:04:07.513 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-25T10:04:07.513 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000274751 s, 1.9 MB/s 2024-04-25T10:04:07.514 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:04:07.571 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_2 2024-04-25T10:04:07.626 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:04:07.626 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:04:07.626 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-25T10:04:07.626 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:04:07.626 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:04:07.626 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-25 10:03:57.591113842 +0000 2024-04-25T10:04:07.626 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-25 10:01:15.414631603 +0000 2024-04-25T10:04:07.626 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-25 10:01:15.414631603 +0000 2024-04-25T10:04:07.626 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-25 10:01:15.414631603 +0000 2024-04-25T10:04:07.627 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:04:07.688 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-25T10:04:07.688 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-25T10:04:07.688 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000194788 s, 2.6 MB/s 2024-04-25T10:04:07.689 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:04:07.745 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_3 2024-04-25T10:04:07.799 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:04:07.800 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:04:07.800 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-25T10:04:07.800 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:04:07.800 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:04:07.800 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-25 10:03:57.592113823 +0000 2024-04-25T10:04:07.800 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-25 10:01:15.759624791 +0000 2024-04-25T10:04:07.800 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-25 10:01:15.759624791 +0000 2024-04-25T10:04:07.800 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-25 10:01:15.759624791 +0000 2024-04-25T10:04:07.800 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:04:07.862 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-25T10:04:07.862 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-25T10:04:07.862 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000233946 s, 2.2 MB/s 2024-04-25T10:04:07.863 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:04:07.919 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_4 2024-04-25T10:04:07.975 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:04:07.975 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:04:07.975 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T10:04:07.975 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:04:07.975 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:04:07.975 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-25 10:03:57.592113823 +0000 2024-04-25T10:04:07.975 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-25 10:01:16.084618373 +0000 2024-04-25T10:04:07.975 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-25 10:01:16.084618373 +0000 2024-04-25T10:04:07.976 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-25 10:01:16.084618373 +0000 2024-04-25T10:04:07.976 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:04:08.037 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-25T10:04:08.037 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-25T10:04:08.038 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000226778 s, 2.3 MB/s 2024-04-25T10:04:08.039 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:04:08.095 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:08.095 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi186.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:04:08.095 INFO:tasks.ceph:Generating config... 2024-04-25T10:04:08.096 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T10:04:08.096 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T10:04:08.096 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T10:04:08.096 INFO:tasks.ceph:[client] pid file = 2024-04-25T10:04:08.097 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T10:04:08.097 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T10:04:08.097 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T10:04:08.097 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T10:04:08.097 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T10:04:08.097 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T10:04:08.097 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T10:04:08.097 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T10:04:08.097 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T10:04:08.097 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T10:04:08.097 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T10:04:08.097 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T10:04:08.097 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T10:04:08.098 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T10:04:08.098 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T10:04:08.098 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T10:04:08.098 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T10:04:08.098 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T10:04:08.098 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T10:04:08.098 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-25T10:04:08.098 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-25T10:04:08.098 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-25T10:04:08.098 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T10:04:08.098 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T10:04:08.098 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-25T10:04:08.098 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-25T10:04:08.099 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T10:04:08.099 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T10:04:08.099 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-25T10:04:08.099 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T10:04:08.099 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T10:04:08.099 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T10:04:08.099 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T10:04:08.099 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T10:04:08.099 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T10:04:08.099 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T10:04:08.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T10:04:08.151 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T10:04:08.154 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T10:04:08.198 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:04:08.264 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.16'), ('mon.b', '172.21.15.186'), ('mon.c', '[v2:172.21.15.186:3301,v1:172.21.15.186:6790]')] 2024-04-25T10:04:08.264 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.16,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T10:04:08.265 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:08.265 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:04:08.320 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.16 --add b 172.21.15.186 --addv c '[v2:172.21.15.186:3301,v1:172.21.15.186:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:08.407 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:08.408 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: generated fsid 34bc5eca-3a21-42a6-a26f-07da0905e8bc 2024-04-25T10:04:08.408 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = quincy 2024-04-25T10:04:08.408 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 0 2024-04-25T10:04:08.408 INFO:teuthology.orchestra.run.smithi016.stdout:fsid 34bc5eca-3a21-42a6-a26f-07da0905e8bc 2024-04-25T10:04:08.408 INFO:teuthology.orchestra.run.smithi016.stdout:last_changed 2024-04-25T10:04:08.403794+0000 2024-04-25T10:04:08.408 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-25T10:04:08.403794+0000 2024-04-25T10:04:08.408 INFO:teuthology.orchestra.run.smithi016.stdout:min_mon_release 17 (quincy) 2024-04-25T10:04:08.408 INFO:teuthology.orchestra.run.smithi016.stdout:election_strategy: 1 2024-04-25T10:04:08.408 INFO:teuthology.orchestra.run.smithi016.stdout:0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-04-25T10:04:08.408 INFO:teuthology.orchestra.run.smithi016.stdout:1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.b 2024-04-25T10:04:08.408 INFO:teuthology.orchestra.run.smithi016.stdout:2: [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] mon.c 2024-04-25T10:04:08.408 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T10:04:08.410 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:04:08.425 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 34bc5eca-3a21-42a6-a26f-07da0905e8bc... 2024-04-25T10:04:08.426 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:04:08.468 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:04:08.507 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-04-25T10:04:08.507 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2024-04-25T10:04:08.507 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:04:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2024-04-25T10:04:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2024-04-25T10:04:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2024-04-25T10:04:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2024-04-25T10:04:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2024-04-25T10:04:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2024-04-25T10:04:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2024-04-25T10:04:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:04:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2024-04-25T10:04:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:04:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:04:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2024-04-25T10:04:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2024-04-25T10:04:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2024-04-25T10:04:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:04:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:04:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2024-04-25T10:04:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:04:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2024-04-25T10:04:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:04:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:04:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2024-04-25T10:04:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:04:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2024-04-25T10:04:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2024-04-25T10:04:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2024-04-25T10:04:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2024-04-25T10:04:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2024-04-25T10:04:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]" 2024-04-25T10:04:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2024-04-25T10:04:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 34bc5eca-3a21-42a6-a26f-07da0905e8bc 2024-04-25T10:04:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.510 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: bdev async discard = True 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: bdev enable discard = True 2024-04-25T10:04:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore allocator = bitmap 2024-04-25T10:04:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2024-04-25T10:04:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2024-04-25T10:04:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 1/20 2024-04-25T10:04:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 1/20 2024-04-25T10:04:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:04:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2024-04-25T10:04:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 4/10 2024-04-25T10:04:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:04:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2024-04-25T10:04:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:04:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:04:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2024-04-25T10:04:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-04-25T10:04:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.514 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2024-04-25T10:04:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:04:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2024-04-25T10:04:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-04-25T10:04:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-04-25T10:04:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:04:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:04:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2024-04-25T10:04:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.514 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2024-04-25T10:04:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:04:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-04-25T10:04:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2024-04-25T10:04:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-04-25T10:04:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2024-04-25T10:04:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2024-04-25T10:04:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:04:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2024-04-25T10:04:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2024-04-25T10:04:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:04:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:04:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:04:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:04:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:04:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:04:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:04:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2024-04-25T10:04:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:04:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:04:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2024-04-25T10:04:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:08.517 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2024-04-25T10:04:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2024-04-25T10:04:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2024-04-25T10:04:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2024-04-25T10:04:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:04:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:04:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2024-04-25T10:04:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-04-25T10:04:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:04:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2024-04-25T10:04:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-04-25T10:04:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-04-25T10:04:08.518 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2024-04-25T10:04:08.518 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2024-04-25T10:04:08.518 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2024-04-25T10:04:08.518 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-04-25T10:04:08.518 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-04-25T10:04:08.518 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-04-25T10:04:08.518 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-04-25T10:04:08.518 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2024-04-25T10:04:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2024-04-25T10:04:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2024-04-25T10:04:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:04:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2024-04-25T10:04:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2024-04-25T10:04:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2024-04-25T10:04:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2024-04-25T10:04:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-04-25T10:04:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-04-25T10:04:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-04-25T10:04:08.520 INFO:teuthology.orchestra.run.smithi186.stdout:[global] 2024-04-25T10:04:08.521 INFO:teuthology.orchestra.run.smithi186.stdout: chdir = "" 2024-04-25T10:04:08.521 INFO:teuthology.orchestra.run.smithi186.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:04:08.521 INFO:teuthology.orchestra.run.smithi186.stdout: auth supported = cephx 2024-04-25T10:04:08.521 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.521 INFO:teuthology.orchestra.run.smithi186.stdout: filestore xattr use omap = true 2024-04-25T10:04:08.521 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.521 INFO:teuthology.orchestra.run.smithi186.stdout: mon clock drift allowed = 1.000 2024-04-25T10:04:08.521 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.521 INFO:teuthology.orchestra.run.smithi186.stdout: osd crush chooseleaf type = 0 2024-04-25T10:04:08.521 INFO:teuthology.orchestra.run.smithi186.stdout: auth debug = true 2024-04-25T10:04:08.521 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.521 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on old message = true 2024-04-25T10:04:08.521 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on bug = true 2024-04-25T10:04:08.522 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.522 INFO:teuthology.orchestra.run.smithi186.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:04:08.522 INFO:teuthology.orchestra.run.smithi186.stdout: mon pg warn max object skew = 0 2024-04-25T10:04:08.522 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.522 INFO:teuthology.orchestra.run.smithi186.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:04:08.522 INFO:teuthology.orchestra.run.smithi186.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:04:08.522 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.522 INFO:teuthology.orchestra.run.smithi186.stdout: osd pool default size = 2 2024-04-25T10:04:08.522 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.522 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd allow primary affinity = true 2024-04-25T10:04:08.522 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd allow pg remap = true 2024-04-25T10:04:08.522 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:04:08.522 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:04:08.522 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on no sortbitwise = false 2024-04-25T10:04:08.522 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:04:08.523 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on too few osds = false 2024-04-25T10:04:08.523 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:04:08.523 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:04:08.523 INFO:teuthology.orchestra.run.smithi186.stdout: mon_allow_pool_size_one = true 2024-04-25T10:04:08.523 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.523 INFO:teuthology.orchestra.run.smithi186.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:04:08.523 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.523 INFO:teuthology.orchestra.run.smithi186.stdout: osd default data pool replay window = 5 2024-04-25T10:04:08.523 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.523 INFO:teuthology.orchestra.run.smithi186.stdout: mon allow pool delete = true 2024-04-25T10:04:08.523 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.523 INFO:teuthology.orchestra.run.smithi186.stdout: mon cluster log file level = debug 2024-04-25T10:04:08.523 INFO:teuthology.orchestra.run.smithi186.stdout: debug asserts on shutdown = true 2024-04-25T10:04:08.523 INFO:teuthology.orchestra.run.smithi186.stdout: mon health detail to clog = false 2024-04-25T10:04:08.523 INFO:teuthology.orchestra.run.smithi186.stdout: mon host = "172.21.15.16,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]" 2024-04-25T10:04:08.523 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on skipped message = False 2024-04-25T10:04:08.524 INFO:teuthology.orchestra.run.smithi186.stdout: fsid = 34bc5eca-3a21-42a6-a26f-07da0905e8bc 2024-04-25T10:04:08.524 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.524 INFO:teuthology.orchestra.run.smithi186.stdout:[osd] 2024-04-25T10:04:08.524 INFO:teuthology.orchestra.run.smithi186.stdout: osd journal size = 100 2024-04-25T10:04:08.524 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.524 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub load threshold = 5.0 2024-04-25T10:04:08.524 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub max interval = 600 2024-04-25T10:04:08.524 INFO:teuthology.orchestra.run.smithi186.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:04:08.524 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.524 INFO:teuthology.orchestra.run.smithi186.stdout: osd recover clone overlap = true 2024-04-25T10:04:08.524 INFO:teuthology.orchestra.run.smithi186.stdout: osd recovery max chunk = 1048576 2024-04-25T10:04:08.524 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.524 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug shutdown = true 2024-04-25T10:04:08.524 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug op order = true 2024-04-25T10:04:08.524 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug verify stray on activate = true 2024-04-25T10:04:08.525 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug trim objects = true 2024-04-25T10:04:08.525 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.525 INFO:teuthology.orchestra.run.smithi186.stdout: osd open classes on start = true 2024-04-25T10:04:08.525 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug pg log writeout = true 2024-04-25T10:04:08.525 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.525 INFO:teuthology.orchestra.run.smithi186.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:04:08.525 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.525 INFO:teuthology.orchestra.run.smithi186.stdout: osd map max advance = 10 2024-04-25T10:04:08.525 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.525 INFO:teuthology.orchestra.run.smithi186.stdout: journal zero on create = true 2024-04-25T10:04:08.525 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.525 INFO:teuthology.orchestra.run.smithi186.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:04:08.525 INFO:teuthology.orchestra.run.smithi186.stdout: filestore apply finisher threads = 3 2024-04-25T10:04:08.525 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.525 INFO:teuthology.orchestra.run.smithi186.stdout: bdev debug aio = true 2024-04-25T10:04:08.526 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug misdirected ops = true 2024-04-25T10:04:08.526 INFO:teuthology.orchestra.run.smithi186.stdout: bdev async discard = True 2024-04-25T10:04:08.526 INFO:teuthology.orchestra.run.smithi186.stdout: bdev enable discard = True 2024-04-25T10:04:08.526 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore allocator = bitmap 2024-04-25T10:04:08.526 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore block size = 96636764160 2024-04-25T10:04:08.526 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore fsck on mount = True 2024-04-25T10:04:08.526 INFO:teuthology.orchestra.run.smithi186.stdout: debug bluefs = 1/20 2024-04-25T10:04:08.526 INFO:teuthology.orchestra.run.smithi186.stdout: debug bluestore = 1/20 2024-04-25T10:04:08.526 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:04:08.526 INFO:teuthology.orchestra.run.smithi186.stdout: debug osd = 20 2024-04-25T10:04:08.526 INFO:teuthology.orchestra.run.smithi186.stdout: debug rocksdb = 4/10 2024-04-25T10:04:08.526 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:04:08.526 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd full ratio = 0.9 2024-04-25T10:04:08.526 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:04:08.526 INFO:teuthology.orchestra.run.smithi186.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:04:08.526 INFO:teuthology.orchestra.run.smithi186.stdout: osd objectstore = bluestore 2024-04-25T10:04:08.527 INFO:teuthology.orchestra.run.smithi186.stdout: osd op complaint time = 180 2024-04-25T10:04:08.527 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.527 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr] 2024-04-25T10:04:08.527 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:04:08.527 INFO:teuthology.orchestra.run.smithi186.stdout: debug mgr = 20 2024-04-25T10:04:08.527 INFO:teuthology.orchestra.run.smithi186.stdout: debug mon = 20 2024-04-25T10:04:08.527 INFO:teuthology.orchestra.run.smithi186.stdout: debug auth = 20 2024-04-25T10:04:08.527 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:04:08.527 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:04:08.527 INFO:teuthology.orchestra.run.smithi186.stdout: mgr/telemetry/nag = false 2024-04-25T10:04:08.527 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.527 INFO:teuthology.orchestra.run.smithi186.stdout:[mon] 2024-04-25T10:04:08.527 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:04:08.527 INFO:teuthology.orchestra.run.smithi186.stdout: debug mon = 20 2024-04-25T10:04:08.527 INFO:teuthology.orchestra.run.smithi186.stdout: debug paxos = 20 2024-04-25T10:04:08.528 INFO:teuthology.orchestra.run.smithi186.stdout: debug auth = 20 2024-04-25T10:04:08.528 INFO:teuthology.orchestra.run.smithi186.stdout: mon data avail warn = 5 2024-04-25T10:04:08.528 INFO:teuthology.orchestra.run.smithi186.stdout: mon mgr mkfs grace = 240 2024-04-25T10:04:08.528 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:04:08.528 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd reporter subtree level = osd 2024-04-25T10:04:08.528 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd prime pg temp = true 2024-04-25T10:04:08.528 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:04:08.528 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.528 INFO:teuthology.orchestra.run.smithi186.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:04:08.528 INFO:teuthology.orchestra.run.smithi186.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:04:08.528 INFO:teuthology.orchestra.run.smithi186.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:04:08.528 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.528 INFO:teuthology.orchestra.run.smithi186.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:04:08.528 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:04:08.529 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:04:08.529 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.529 INFO:teuthology.orchestra.run.smithi186.stdout: # 1m isn't quite enough 2024-04-25T10:04:08.529 INFO:teuthology.orchestra.run.smithi186.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:04:08.529 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.529 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:04:08.529 INFO:teuthology.orchestra.run.smithi186.stdout: mon op complaint time = 120 2024-04-25T10:04:08.529 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:04:08.529 INFO:teuthology.orchestra.run.smithi186.stdout:[client] 2024-04-25T10:04:08.529 INFO:teuthology.orchestra.run.smithi186.stdout: rgw cache enabled = true 2024-04-25T10:04:08.529 INFO:teuthology.orchestra.run.smithi186.stdout: rgw enable ops log = true 2024-04-25T10:04:08.529 INFO:teuthology.orchestra.run.smithi186.stdout: rgw enable usage log = true 2024-04-25T10:04:08.529 INFO:teuthology.orchestra.run.smithi186.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:04:08.529 INFO:teuthology.orchestra.run.smithi186.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:04:08.529 INFO:teuthology.orchestra.run.smithi186.stdout: client mount timeout = 600 2024-04-25T10:04:08.530 INFO:teuthology.orchestra.run.smithi186.stdout: debug client = 20 2024-04-25T10:04:08.530 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:04:08.530 INFO:teuthology.orchestra.run.smithi186.stdout: pid file = "" 2024-04-25T10:04:08.530 INFO:teuthology.orchestra.run.smithi186.stdout: rados mon op timeout = 900 2024-04-25T10:04:08.530 INFO:teuthology.orchestra.run.smithi186.stdout: rados osd op timeout = 900 2024-04-25T10:04:08.530 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.a] 2024-04-25T10:04:08.530 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.b] 2024-04-25T10:04:08.530 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.c] 2024-04-25T10:04:08.530 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.a] 2024-04-25T10:04:08.530 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.c] 2024-04-25T10:04:08.530 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.b] 2024-04-25T10:04:08.530 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.d] 2024-04-25T10:04:08.530 INFO:teuthology.orchestra.run.smithi186.stdout:[mds] 2024-04-25T10:04:08.530 INFO:teuthology.orchestra.run.smithi186.stdout: debug mds = 20 2024-04-25T10:04:08.531 INFO:teuthology.orchestra.run.smithi186.stdout: debug mds balancer = 20 2024-04-25T10:04:08.531 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:04:08.531 INFO:teuthology.orchestra.run.smithi186.stdout: mds debug frag = True 2024-04-25T10:04:08.531 INFO:teuthology.orchestra.run.smithi186.stdout: mds debug scatterstat = True 2024-04-25T10:04:08.531 INFO:teuthology.orchestra.run.smithi186.stdout: mds op complaint time = 180 2024-04-25T10:04:08.531 INFO:teuthology.orchestra.run.smithi186.stdout: mds verify scatter = True 2024-04-25T10:04:08.531 INFO:teuthology.orchestra.run.smithi186.stdout: osd op complaint time = 180 2024-04-25T10:04:08.531 INFO:teuthology.orchestra.run.smithi186.stdout: rados mon op timeout = 900 2024-04-25T10:04:08.531 INFO:teuthology.orchestra.run.smithi186.stdout: rados osd op timeout = 900 2024-04-25T10:04:08.531 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T10:04:08.532 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T10:04:08.613 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T10:04:08.614 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:08.614 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T10:04:08.629 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:08.629 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T10:04:08.684 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:04:08.684 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:08.684 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:04:08.684 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:04:08.758 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:08.758 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:08.812 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi186.front.sepia.ceph.com 2024-04-25T10:04:08.813 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:08.813 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:04:08.813 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:04:08.846 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:08.846 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:08.901 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T10:04:08.901 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T10:04:08.901 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:04:08.950 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:04:08.953 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:04:09.025 INFO:teuthology.orchestra.run.smithi186.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:04:09.028 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T10:04:09.028 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:04:09.077 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:04:09.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T10:04:09.103 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:04:09.193 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:04:09.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T10:04:09.219 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:04:09.268 INFO:teuthology.orchestra.run.smithi186.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:04:09.270 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T10:04:09.294 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:04:09.382 INFO:teuthology.orchestra.run.smithi186.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:04:09.384 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T10:04:09.408 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T10:04:09.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T10:04:09.448 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T10:04:09.460 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T10:04:09.540 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T10:04:09.553 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T10:04:09.592 INFO:teuthology.orchestra.run.smithi186.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T10:04:09.603 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T10:04:09.683 INFO:teuthology.orchestra.run.smithi186.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T10:04:09.694 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T10:04:09.694 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi186.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:04:09.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T10:04:09.719 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:09.719 INFO:tasks.ceph:role: osd.0 2024-04-25T10:04:09.719 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:04:09.719 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:04:09.783 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:04:09.784 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:04:09.784 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:04:09.784 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:04:09.784 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:04:09.784 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:04:09.784 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:04:09.784 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:04:09.784 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:04:09.784 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:04:09.823 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-04-25T10:04:09.825 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-04-25T10:04:09.825 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T10:04:09.893 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T10:04:09.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T10:04:10.025 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:10.025 INFO:tasks.ceph:role: osd.1 2024-04-25T10:04:10.025 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:04:10.025 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:04:10.091 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:04:10.091 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:04:10.091 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:04:10.091 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:04:10.091 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:04:10.091 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:04:10.091 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:04:10.092 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:04:10.092 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:04:10.092 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:04:10.131 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-04-25T10:04:10.133 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-04-25T10:04:10.133 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T10:04:10.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T10:04:10.272 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T10:04:10.338 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:10.338 INFO:tasks.ceph:role: osd.2 2024-04-25T10:04:10.338 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:04:10.338 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:04:10.402 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:04:10.403 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:04:10.403 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:04:10.403 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:04:10.403 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:04:10.403 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:04:10.403 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:04:10.403 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:04:10.403 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:04:10.403 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:04:10.447 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-04-25T10:04:10.449 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-04-25T10:04:10.449 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T10:04:10.522 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T10:04:10.590 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T10:04:10.655 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:10.655 INFO:tasks.ceph:role: osd.3 2024-04-25T10:04:10.655 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:04:10.655 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:04:10.720 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:04:10.721 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:04:10.721 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:04:10.721 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:04:10.721 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:04:10.721 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:04:10.721 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:04:10.721 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:04:10.721 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:04:10.721 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:04:10.759 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-04-25T10:04:10.761 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-04-25T10:04:10.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T10:04:10.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T10:04:10.897 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:10.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:10.990+0000 7f36c4b1a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T10:04:10.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:10.991+0000 7f36c4b1a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T10:04:10.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:10.991+0000 7f36c4b1a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:04:10.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:10.991+0000 7f36c4b1a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:04:10.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:10.991+0000 7f36c4b1a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:04:10.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:10.991+0000 7f36c4b1a640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T10:04:13.844 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T10:04:13.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:13.959+0000 7f773742e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T10:04:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:13.959+0000 7f773742e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T10:04:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:13.959+0000 7f773742e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:04:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:13.959+0000 7f773742e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:04:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:13.959+0000 7f773742e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:04:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:13.960+0000 7f773742e640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T10:04:16.825 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T10:04:16.854 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:16.939+0000 7f64dec9a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T10:04:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:16.939+0000 7f64dec9a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T10:04:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:16.939+0000 7f64dec9a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:04:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:16.939+0000 7f64dec9a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:04:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:16.939+0000 7f64dec9a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:04:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:16.939+0000 7f64dec9a640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T10:04:19.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T10:04:19.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:19.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:19.908+0000 7fddd2140640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T10:04:19.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:19.908+0000 7fddd2140640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T10:04:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:19.908+0000 7fddd2140640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:04:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:19.908+0000 7fddd2140640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:04:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:19.908+0000 7fddd2140640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:04:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:19.909+0000 7fddd2140640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T10:04:22.760 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T10:04:22.788 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T10:04:22.817 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:22.817 INFO:tasks.ceph:role: osd.4 2024-04-25T10:04:22.817 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi186.front.sepia.ceph.com 2024-04-25T10:04:22.817 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:04:22.892 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:04:22.892 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:04:22.892 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:04:22.892 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:04:22.892 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:04:22.892 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:04:22.892 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:04:22.892 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:04:22.892 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:04:22.893 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:04:23.389 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2024-04-25T10:04:23.391 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2024-04-25T10:04:23.391 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T10:04:23.511 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T10:04:23.541 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T10:04:23.604 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:23.604 INFO:tasks.ceph:role: osd.5 2024-04-25T10:04:23.604 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi186.front.sepia.ceph.com 2024-04-25T10:04:23.605 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:04:23.676 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:04:23.676 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:04:23.676 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:04:23.676 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:04:23.676 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:04:23.676 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:04:23.676 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:04:23.677 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:04:23.677 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:04:23.677 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:04:24.137 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2024-04-25T10:04:24.139 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2024-04-25T10:04:24.139 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T10:04:24.228 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T10:04:24.255 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T10:04:24.318 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:24.318 INFO:tasks.ceph:role: osd.6 2024-04-25T10:04:24.318 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi186.front.sepia.ceph.com 2024-04-25T10:04:24.318 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:04:24.389 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:04:24.389 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:04:24.389 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:04:24.389 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:04:24.389 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:04:24.389 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:04:24.389 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:04:24.390 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:04:24.390 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:04:24.390 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:04:24.874 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2024-04-25T10:04:24.877 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2024-04-25T10:04:24.877 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T10:04:25.013 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T10:04:25.042 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T10:04:25.107 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:25.107 INFO:tasks.ceph:role: osd.7 2024-04-25T10:04:25.107 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi186.front.sepia.ceph.com 2024-04-25T10:04:25.107 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:04:25.178 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:04:25.178 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:04:25.179 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:04:25.179 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:04:25.179 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:04:25.179 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:04:25.179 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:04:25.179 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:04:25.179 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:04:25.179 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:04:25.688 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2024-04-25T10:04:25.690 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2024-04-25T10:04:25.690 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T10:04:25.818 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T10:04:25.847 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:25.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:25.939+0000 7f7ae5a1a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T10:04:25.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:25.939+0000 7f7ae5a1a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T10:04:25.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:25.939+0000 7f7ae5a1a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:04:25.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:25.939+0000 7f7ae5a1a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:04:25.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:25.939+0000 7f7ae5a1a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:04:25.945 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:25.940+0000 7f7ae5a1a640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T10:04:28.799 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T10:04:28.827 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:28.919 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:28.914+0000 7fae45614640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T10:04:28.919 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:28.914+0000 7fae45614640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T10:04:28.920 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:28.914+0000 7fae45614640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:04:28.920 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:28.914+0000 7fae45614640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:04:28.920 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:28.914+0000 7fae45614640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:04:28.920 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:28.914+0000 7fae45614640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T10:04:31.786 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T10:04:31.814 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:31.904 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:31.899+0000 7fb5c00dd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T10:04:31.905 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:31.900+0000 7fb5c00dd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T10:04:31.905 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:31.900+0000 7fb5c00dd640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:04:31.905 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:31.900+0000 7fb5c00dd640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:04:31.905 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:31.900+0000 7fb5c00dd640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:04:31.905 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:31.900+0000 7fb5c00dd640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T10:04:34.753 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T10:04:34.782 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:34.867 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:34.862+0000 7f7a3981e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T10:04:34.867 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:34.862+0000 7f7a3981e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T10:04:34.867 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:34.863+0000 7f7a3981e640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:04:34.867 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:34.863+0000 7f7a3981e640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:04:34.867 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:34.863+0000 7f7a3981e640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:04:34.868 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:34.863+0000 7f7a3981e640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T10:04:36.539 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:36.544 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:37.723 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T10:04:37.750 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T10:04:37.751 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:37.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T10:04:37.780 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:37.780 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T10:04:37.849 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:37.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T10:04:37.917 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:37.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T10:04:37.984 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:37.985 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T10:04:38.051 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:38.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T10:04:38.117 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:38.117 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T10:04:38.182 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:38.182 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T10:04:38.211 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:38.211 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T10:04:38.279 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:38.279 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T10:04:38.345 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:38.345 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T10:04:38.410 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:38.411 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T10:04:38.476 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:38.476 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T10:04:38.540 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:38.541 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T10:04:38.604 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:38.604 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T10:04:38.623 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:38.623 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T10:04:38.681 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:38.681 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T10:04:38.697 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:38.697 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T10:04:38.753 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T10:04:38.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:04:38.756 DEBUG:teuthology.orchestra.run.smithi186:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:04:38.796 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.y] 2024-04-25T10:04:38.797 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCYKipmghhyOBAAcCm0hQzU+Wlw45yZQETTTg== 2024-04-25T10:04:38.797 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-04-25T10:04:38.797 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCZKipmj+xmBBAAaWdHmoiYcXzFnMVQSioT2Q== 2024-04-25T10:04:38.797 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-04-25T10:04:38.797 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCZKipm+LtFCxAAGG79kv8eWzly8l5qzQwfww== 2024-04-25T10:04:38.797 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.0] 2024-04-25T10:04:38.797 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCaKipm2+UdOxAAP8GBunvYsue8EILYV5Uq7A== 2024-04-25T10:04:38.797 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.1] 2024-04-25T10:04:38.797 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCdKipmSv87ORAAigVU7nyzkcXqMEHKsSAsqg== 2024-04-25T10:04:38.797 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.2] 2024-04-25T10:04:38.797 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCgKipm2VMLOBAAJETFfWgq7H/imQAfw93Ruw== 2024-04-25T10:04:38.797 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.3] 2024-04-25T10:04:38.798 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCjKipmb1g0NhAADH2uHQPhW84KLYyWbOjQdw== 2024-04-25T10:04:38.798 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.x] 2024-04-25T10:04:38.798 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCZKipmiPVHARAAGPbv3P3yVj3tlo0b2gmcyw== 2024-04-25T10:04:38.798 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-04-25T10:04:38.798 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCZKipmUES+DxAAu5It7dQmb7/BM/4x1fUYXA== 2024-04-25T10:04:38.798 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-04-25T10:04:38.798 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCZKipmqbSSFhAAZtrLsAGSyaMGgv45Hy0WpA== 2024-04-25T10:04:38.798 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.4] 2024-04-25T10:04:38.798 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCpKipmnH0ROBAAEb8i4nfswYQ/w5mEw48kIg== 2024-04-25T10:04:38.798 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.5] 2024-04-25T10:04:38.798 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCsKipm8HCRNhAA3FhcMlUWxMAiFRXbBWWvNA== 2024-04-25T10:04:38.798 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.6] 2024-04-25T10:04:38.798 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCvKipmfgqpNRAAtPhpSDU9q23Sfa+V7m+3NA== 2024-04-25T10:04:38.799 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.7] 2024-04-25T10:04:38.799 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCyKipm3ndyMxAAeMNODGxNupb043CsA/SP1A== 2024-04-25T10:04:38.799 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-04-25T10:04:38.799 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCZKipmAph/GhAAG3x1zC98yHQ13fjpmrlPkg== 2024-04-25T10:04:38.799 INFO:teuthology.orchestra.run.smithi016.stdout:[client.2] 2024-04-25T10:04:38.799 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCZKipmsFACIBAAFuWsmuQlF58yzIcYUx/6Iw== 2024-04-25T10:04:38.799 INFO:teuthology.orchestra.run.smithi016.stdout:[client.1] 2024-04-25T10:04:38.799 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCZKipmrccTIxAAuDZamFXTe6J2d+HAcBe+DA== 2024-04-25T10:04:38.799 INFO:teuthology.orchestra.run.smithi016.stdout:[client.3] 2024-04-25T10:04:38.799 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCZKipmAe1/KBAAXzLkscuwpCwMnrIKQny9+Q== 2024-04-25T10:04:38.816 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr.y] 2024-04-25T10:04:38.816 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCYKipmghhyOBAAcCm0hQzU+Wlw45yZQETTTg== 2024-04-25T10:04:38.816 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.a] 2024-04-25T10:04:38.816 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCZKipmj+xmBBAAaWdHmoiYcXzFnMVQSioT2Q== 2024-04-25T10:04:38.816 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.c] 2024-04-25T10:04:38.816 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCZKipm+LtFCxAAGG79kv8eWzly8l5qzQwfww== 2024-04-25T10:04:38.817 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.0] 2024-04-25T10:04:38.817 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCaKipm2+UdOxAAP8GBunvYsue8EILYV5Uq7A== 2024-04-25T10:04:38.817 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.1] 2024-04-25T10:04:38.817 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCdKipmSv87ORAAigVU7nyzkcXqMEHKsSAsqg== 2024-04-25T10:04:38.817 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.2] 2024-04-25T10:04:38.817 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCgKipm2VMLOBAAJETFfWgq7H/imQAfw93Ruw== 2024-04-25T10:04:38.817 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.3] 2024-04-25T10:04:38.817 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCjKipmb1g0NhAADH2uHQPhW84KLYyWbOjQdw== 2024-04-25T10:04:38.817 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr.x] 2024-04-25T10:04:38.817 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCZKipmiPVHARAAGPbv3P3yVj3tlo0b2gmcyw== 2024-04-25T10:04:38.817 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.b] 2024-04-25T10:04:38.817 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCZKipmUES+DxAAu5It7dQmb7/BM/4x1fUYXA== 2024-04-25T10:04:38.817 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.d] 2024-04-25T10:04:38.817 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCZKipmqbSSFhAAZtrLsAGSyaMGgv45Hy0WpA== 2024-04-25T10:04:38.818 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.4] 2024-04-25T10:04:38.818 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCpKipmnH0ROBAAEb8i4nfswYQ/w5mEw48kIg== 2024-04-25T10:04:38.818 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.5] 2024-04-25T10:04:38.818 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCsKipm8HCRNhAA3FhcMlUWxMAiFRXbBWWvNA== 2024-04-25T10:04:38.818 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.6] 2024-04-25T10:04:38.818 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCvKipmfgqpNRAAtPhpSDU9q23Sfa+V7m+3NA== 2024-04-25T10:04:38.818 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.7] 2024-04-25T10:04:38.818 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCyKipm3ndyMxAAeMNODGxNupb043CsA/SP1A== 2024-04-25T10:04:38.818 INFO:teuthology.orchestra.run.smithi186.stdout:[client.0] 2024-04-25T10:04:38.818 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCZKipmAph/GhAAG3x1zC98yHQ13fjpmrlPkg== 2024-04-25T10:04:38.818 INFO:teuthology.orchestra.run.smithi186.stdout:[client.2] 2024-04-25T10:04:38.818 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCZKipmsFACIBAAFuWsmuQlF58yzIcYUx/6Iw== 2024-04-25T10:04:38.818 INFO:teuthology.orchestra.run.smithi186.stdout:[client.1] 2024-04-25T10:04:38.819 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCZKipmrccTIxAAuDZamFXTe6J2d+HAcBe+DA== 2024-04-25T10:04:38.819 INFO:teuthology.orchestra.run.smithi186.stdout:[client.3] 2024-04-25T10:04:38.819 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCZKipmAe1/KBAAXzLkscuwpCwMnrIKQny9+Q== 2024-04-25T10:04:38.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:04:38.844 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:04:38.910 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:04:38.913 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:04:38.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:04:38.969 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:04:39.021 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:39.023 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:39.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:39.073 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:39.122 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:39.124 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:39.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:39.175 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:39.223 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:04:39.225 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:04:39.275 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:04:39.278 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:04:39.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:04:39.332 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:04:39.384 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:39.386 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:39.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:39.437 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:39.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:39.488 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:39.537 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:39.539 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:39.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:04:39.590 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:04:39.642 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:04:39.644 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:04:39.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:04:39.698 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:04:39.749 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:04:39.751 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:04:39.803 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T10:04:39.803 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T10:04:39.827 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:04:39.923 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T10:04:39.947 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T10:04:39.971 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:04:40.065 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T10:04:40.089 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T10:04:40.153 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:04:40.246 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T10:04:40.269 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:40.272 DEBUG:teuthology.orchestra.run.smithi186:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:40.323 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T10:04:40.324 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T10:04:40.324 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T10:04:40.327 INFO:tasks.ceph.mon.a:Started 2024-04-25T10:04:40.327 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T10:04:40.327 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T10:04:40.366 INFO:tasks.ceph.mon.b:Started 2024-04-25T10:04:40.366 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T10:04:40.366 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T10:04:40.368 INFO:tasks.ceph.mon.c:Started 2024-04-25T10:04:40.368 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T10:04:40.368 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T10:04:40.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T10:04:40.370 INFO:tasks.ceph.mgr.y:Started 2024-04-25T10:04:40.371 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T10:04:40.371 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T10:04:40.372 INFO:tasks.ceph.mgr.x:Started 2024-04-25T10:04:40.373 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T10:04:40.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T10:04:40.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.469+0000 7f73544b3640 1 Processor -- start 2024-04-25T10:04:40.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.469+0000 7f73544b3640 1 -- start start 2024-04-25T10:04:40.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.470+0000 7f73544b3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 0x7f734c0b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.470+0000 7f73544b3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734c161500 con 0x7f734c0d2560 2024-04-25T10:04:40.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.470+0000 7f73544b3640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734c161eb0 con 0x7f734c161120 2024-04-25T10:04:40.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.470+0000 7f73544b3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734c0b5ef0 con 0x7f734c0b5550 2024-04-25T10:04:40.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.470+0000 7f7351a4e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 0x7f734c0b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.470+0000 7f7351a4e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 0x7f734c0b59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60044/0 (socket says 172.21.15.16:60044) 2024-04-25T10:04:40.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.470+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 learned_addr learned my addr 172.21.15.16:0/2031372195 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:04:40.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.470+0000 7f7352a50640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f734c0d2560 0x7f734c0d2980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41784/0 (socket says 172.21.15.16:41784) 2024-04-25T10:04:40.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.470+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 msgr2=0x7f734c0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:04:40.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.470+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 msgr2=0x7f734c0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:40.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.470+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 0x7f734c0b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:40.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.470+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 0x7f734c0b59b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:40.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.671+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 0x7f734c0b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.671+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 msgr2=0x7f734c0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:04:40.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.671+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 msgr2=0x7f734c0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:40.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.671+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 0x7f734c0b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:40.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:40.671+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 0x7f734c0b59b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:04:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:41.072+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 0x7f734c0b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:41.072+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 msgr2=0x7f734c0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:04:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:41.072+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 msgr2=0x7f734c0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:41.072+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 0x7f734c0b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:41.072+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 0x7f734c0b59b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:04:41.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:41.873+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 0x7f734c0b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:41.873+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 msgr2=0x7f734c0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:04:41.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:41.873+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 msgr2=0x7f734c0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:41.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:41.873+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 0x7f734c0b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:41.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:41.873+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 0x7f734c0b59b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:04:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.470+0000 7f7343fff640 1 -- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 msgr2=0x7f734c0b59b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:04:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.470+0000 7f7343fff640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.16:3300/0 conn(0x7f734c0b5550 0x7f734c0b59b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.470+0000 7f7343fff640 1 -- 172.21.15.16:0/2031372195 >> v1:172.21.15.186:6789/0 conn(0x7f734c161120 legacy=0x7f734c1617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.470+0000 7f7343fff640 1 -- 172.21.15.16:0/2031372195 >> v1:172.21.15.16:6789/0 conn(0x7f734c0d2560 legacy=0x7f734c0d2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.470+0000 7f7343fff640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 0x7f7330005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.470+0000 7f7343fff640 1 --2- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 0x7f7330008620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.470+0000 7f7343fff640 1 -- 172.21.15.16:0/2031372195 --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7330008d20 con 0x7f73300036c0 2024-04-25T10:04:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.470+0000 7f7343fff640 1 -- 172.21.15.16:0/2031372195 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7330008ed0 con 0x7f7330006180 2024-04-25T10:04:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.470+0000 7f735224f640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 0x7f7330005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.470+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 0x7f7330008620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.470+0000 7f735224f640 1 -- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 msgr2=0x7f7330005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:04:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.470+0000 7f735224f640 1 -- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 msgr2=0x7f7330005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.470+0000 7f735224f640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 0x7f7330005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.470+0000 7f735224f640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 0x7f7330005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.470+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 msgr2=0x7f7330008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:04:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.470+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 msgr2=0x7f7330008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.470+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 0x7f7330008620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.470+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 0x7f7330008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:43.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.671+0000 7f735224f640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 0x7f7330005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:43.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.671+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 0x7f7330008620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:43.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.671+0000 7f735224f640 1 -- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 msgr2=0x7f7330005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:04:43.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.671+0000 7f735224f640 1 -- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 msgr2=0x7f7330005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:43.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.671+0000 7f735224f640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 0x7f7330005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:43.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.671+0000 7f735224f640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 0x7f7330005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:04:43.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.671+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 msgr2=0x7f7330008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:04:43.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.671+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 msgr2=0x7f7330008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:43.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.671+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 0x7f7330008620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:43.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:43.671+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 0x7f7330008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:04:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.072+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 0x7f7330008620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.072+0000 7f735224f640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 0x7f7330005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.072+0000 7f735224f640 1 -- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 msgr2=0x7f7330005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:04:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.072+0000 7f735224f640 1 -- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 msgr2=0x7f7330005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.072+0000 7f735224f640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 0x7f7330005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.072+0000 7f735224f640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 0x7f7330005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:04:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.072+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 msgr2=0x7f7330008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:04:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.072+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 msgr2=0x7f7330008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:44.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.072+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 0x7f7330008620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:44.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.072+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 0x7f7330008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:04:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.874+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 0x7f7330008620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.874+0000 7f735224f640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 0x7f7330005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.874+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 msgr2=0x7f7330008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:04:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.874+0000 7f735224f640 1 -- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 msgr2=0x7f7330005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:04:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.874+0000 7f735224f640 1 -- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 msgr2=0x7f7330005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.874+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 msgr2=0x7f7330008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.874+0000 7f735224f640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 0x7f7330005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.874+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 0x7f7330008620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.874+0000 7f735224f640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 0x7f7330005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:04:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:44.874+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 0x7f7330008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:04:45.764 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:45.759+0000 7fc684930040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:04:45.768 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:45.764+0000 7fc470923040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:04:45.907 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:45.902+0000 7fc470923040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:04:45.917 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:45.912+0000 7fc684930040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:04:45.958 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:45.954+0000 7fc470923040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:04:45.973 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:45.968+0000 7fc684930040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:04:46.193 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:46.188+0000 7fc470923040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:04:46.229 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:46.224+0000 7fc684930040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:04:46.237 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:46.233+0000 7fc470923040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:04:46.276 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:46.271+0000 7fc684930040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:04:46.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.470+0000 7f7343fff640 1 -- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 msgr2=0x7f7330008620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:04:46.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.470+0000 7f7343fff640 1 --2- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330006180 0x7f7330008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:46.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.470+0000 7f7343fff640 1 -- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 msgr2=0x7f7330005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:04:46.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.470+0000 7f7343fff640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f73300036c0 0x7f7330005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.470+0000 7f7352a50640 1 -- 172.21.15.16:0/2031372195 reap_dead start 2024-04-25T10:04:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.470+0000 7f7343fff640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f7330003b30 0x7f7330006610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.470+0000 7f7343fff640 1 -- 172.21.15.16:0/2031372195 --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734c162180 con 0x7f734c0d2560 2024-04-25T10:04:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.470+0000 7f7343fff640 1 -- 172.21.15.16:0/2031372195 --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7330008870 con 0x7f7330006b50 2024-04-25T10:04:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.470+0000 7f7343fff640 1 -- 172.21.15.16:0/2031372195 --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7330008d20 con 0x7f7330003b30 2024-04-25T10:04:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.471+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> v2:172.21.15.186:3300/0 conn(0x7f7330003b30 0x7f7330006610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.471+0000 7f735124d640 1 -- 172.21.15.16:0/2031372195 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274392042 0 0) 0x7f734c162180 con 0x7f734c0d2560 2024-04-25T10:04:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.471+0000 7f735124d640 1 -- 172.21.15.16:0/2031372195 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7338003520 con 0x7f734c0d2560 2024-04-25T10:04:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.471+0000 7f735124d640 1 -- 172.21.15.16:0/2031372195 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635501719 0 0) 0x7f7330008870 con 0x7f7330006b50 2024-04-25T10:04:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.471+0000 7f735124d640 1 -- 172.21.15.16:0/2031372195 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f734c162180 con 0x7f7330006b50 2024-04-25T10:04:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.471+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 >> v1:172.21.15.186:6789/0 conn(0x7f7330006b50 legacy=0x7f7330007000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.471+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 >> v1:172.21.15.16:6789/0 conn(0x7f734c0d2560 legacy=0x7f733000b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.471+0000 7f7351a4e640 1 -- 172.21.15.16:0/2031372195 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7330010ad0 con 0x7f7330003b30 2024-04-25T10:04:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.472+0000 7f7351a4e640 1 --2- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7330003b30 0x7f7330006610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f733c009100 tx=0x7f733c02bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34640570d3b31eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.472+0000 7f735124d640 1 -- 172.21.15.16:0/2031372195 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c039070 con 0x7f7330003b30 2024-04-25T10:04:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.472+0000 7f735124d640 1 -- 172.21.15.16:0/2031372195 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733c02ca50 con 0x7f7330003b30 2024-04-25T10:04:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.472+0000 7f735124d640 1 -- 172.21.15.16:0/2031372195 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c00f040 con 0x7f7330003b30 2024-04-25T10:04:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.473+0000 7f73544b3640 1 -- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7330003b30 msgr2=0x7f7330006610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.473+0000 7f73544b3640 1 --2- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7330003b30 0x7f7330006610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f733c009100 tx=0x7f733c02bf50 comp rx=0 tx=0).stop 2024-04-25T10:04:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.473+0000 7f73544b3640 1 -- 172.21.15.16:0/2031372195 shutdown_connections 2024-04-25T10:04:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.473+0000 7f73544b3640 1 --2- 172.21.15.16:0/2031372195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7330003b30 0x7f7330006610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.473+0000 7f73544b3640 1 -- 172.21.15.16:0/2031372195 >> 172.21.15.16:0/2031372195 conn(0x7f734c0d0550 msgr2=0x7f734c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.473+0000 7f73544b3640 1 -- 172.21.15.16:0/2031372195 shutdown_connections 2024-04-25T10:04:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.473+0000 7f73544b3640 1 -- 172.21.15.16:0/2031372195 wait complete. 2024-04-25T10:04:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.474+0000 7f73544b3640 1 Processor -- start 2024-04-25T10:04:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.474+0000 7f73544b3640 1 -- start start 2024-04-25T10:04:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f73544b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330003b30 0x7f734c1e7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f73544b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7330006b50 0x7f734c0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f73544b3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f734c0d2560 0x7f734c0c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f73544b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c0d1d60 con 0x7f7330006b50 2024-04-25T10:04:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f73544b3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c162740 con 0x7f734c0d2560 2024-04-25T10:04:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f73544b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734c0d2040 con 0x7f7330003b30 2024-04-25T10:04:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f7351a4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7330006b50 0x7f734c0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f7351a4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7330006b50 0x7f734c0caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57036/0 (socket says 172.21.15.16:57036) 2024-04-25T10:04:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f7351a4e640 1 -- 172.21.15.16:0/1361914597 learned_addr learned my addr 172.21.15.16:0/1361914597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:04:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f7352a50640 1 --2- 172.21.15.16:0/1361914597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f734c0d2560 0x7f734c0c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f735224f640 1 --2- 172.21.15.16:0/1361914597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330003b30 0x7f734c1e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f735224f640 1 -- 172.21.15.16:0/1361914597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330003b30 msgr2=0x7f734c1e7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T10:04:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f735224f640 1 -- 172.21.15.16:0/1361914597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330003b30 msgr2=0x7f734c1e7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:04:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f735224f640 1 --2- 172.21.15.16:0/1361914597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330003b30 0x7f734c1e7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f735224f640 1 --2- 172.21.15.16:0/1361914597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330003b30 0x7f734c1e7600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f7351a4e640 1 -- 172.21.15.16:0/1361914597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330003b30 msgr2=0x7f734c1e7600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:04:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f7351a4e640 1 --2- 172.21.15.16:0/1361914597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330003b30 0x7f734c1e7600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f7351a4e640 1 -- 172.21.15.16:0/1361914597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f734c0d2560 msgr2=0x7f734c0c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f7351a4e640 1 --2- 172.21.15.16:0/1361914597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f734c0d2560 0x7f734c0c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f7351a4e640 1 -- 172.21.15.16:0/1361914597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c0c6c60 con 0x7f7330006b50 2024-04-25T10:04:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.475+0000 7f7351a4e640 1 --2- 172.21.15.16:0/1361914597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7330006b50 0x7f734c0caba0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f733c016a80 tx=0x7f733c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.476+0000 7f7342ffd640 1 -- 172.21.15.16:0/1361914597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c044020 con 0x7f7330006b50 2024-04-25T10:04:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.476+0000 7f7342ffd640 1 -- 172.21.15.16:0/1361914597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733c03f470 con 0x7f7330006b50 2024-04-25T10:04:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.476+0000 7f73544b3640 1 -- 172.21.15.16:0/1361914597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734c1e7c90 con 0x7f7330006b50 2024-04-25T10:04:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.476+0000 7f7342ffd640 1 -- 172.21.15.16:0/1361914597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c039040 con 0x7f7330006b50 2024-04-25T10:04:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.476+0000 7f73544b3640 1 -- 172.21.15.16:0/1361914597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734c1e8250 con 0x7f7330006b50 2024-04-25T10:04:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.476+0000 7f7342ffd640 1 -- 172.21.15.16:0/1361914597 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f733c047020 con 0x7f7330006b50 2024-04-25T10:04:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.476+0000 7f7342ffd640 1 -- 172.21.15.16:0/1361914597 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f733c03fc60 con 0x7f7330006b50 2024-04-25T10:04:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.477+0000 7f73544b3640 1 -- 172.21.15.16:0/1361914597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734c0750b0 con 0x7f7330006b50 2024-04-25T10:04:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.479+0000 7f7342ffd640 1 -- 172.21.15.16:0/1361914597 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f733c04e030 con 0x7f7330006b50 2024-04-25T10:04:46.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.546+0000 7f73544b3640 1 -- 172.21.15.16:0/1361914597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f734c0b6010 con 0x7f7330006b50 2024-04-25T10:04:46.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.547+0000 7f7342ffd640 1 -- 172.21.15.16:0/1361914597 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f734c0c6c60 con 0x7f7330006b50 2024-04-25T10:04:46.552 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-04-25T10:04:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.548+0000 7f73544b3640 1 -- 172.21.15.16:0/1361914597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7330006b50 msgr2=0x7f734c0caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.548+0000 7f73544b3640 1 --2- 172.21.15.16:0/1361914597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7330006b50 0x7f734c0caba0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f733c016a80 tx=0x7f733c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.548+0000 7f73544b3640 1 -- 172.21.15.16:0/1361914597 shutdown_connections 2024-04-25T10:04:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.548+0000 7f73544b3640 1 --2- 172.21.15.16:0/1361914597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f734c0d2560 0x7f734c0c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.548+0000 7f73544b3640 1 --2- 172.21.15.16:0/1361914597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7330006b50 0x7f734c0caba0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.548+0000 7f73544b3640 1 --2- 172.21.15.16:0/1361914597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7330003b30 0x7f734c1e7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.548+0000 7f73544b3640 1 -- 172.21.15.16:0/1361914597 >> 172.21.15.16:0/1361914597 conn(0x7f734c0d0550 msgr2=0x7f734c0b8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.548+0000 7f73544b3640 1 -- 172.21.15.16:0/1361914597 shutdown_connections 2024-04-25T10:04:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:46.548+0000 7f73544b3640 1 -- 172.21.15.16:0/1361914597 wait complete. 2024-04-25T10:04:46.564 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T10:04:46.564 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T10:04:46.565 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:46.565 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T10:04:46.589 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:46.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T10:04:46.654 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:46.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T10:04:46.719 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:04:46.719 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T10:04:46.783 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:46.783 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T10:04:46.807 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:46.807 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T10:04:46.871 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:46.871 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T10:04:46.935 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:04:46.935 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T10:04:46.958 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:46.953+0000 7fc470923040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:04:46.980 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 40dcfc39-064c-4b17-bcf1-3f45d2e26b86 0 2024-04-25T10:04:47.044 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:47.039+0000 7fc684930040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:04:47.049 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:47.045+0000 7fc470923040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:04:47.094 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:47.089+0000 7fc470923040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:04:47.118 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.113+0000 7fd3c21e0640 1 Processor -- start 2024-04-25T10:04:47.118 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.113+0000 7fd3c21e0640 1 -- start start 2024-04-25T10:04:47.127 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.115+0000 7fd3c21e0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc15a900 0x7fd3bc0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.127 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.115+0000 7fd3c21e0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3bc0af5f0 con 0x7fd3bc06f010 2024-04-25T10:04:47.127 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.115+0000 7fd3c21e0640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3bc06e530 con 0x7fd3bc0cc120 2024-04-25T10:04:47.127 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.115+0000 7fd3c21e0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc0af8d0 con 0x7fd3bc15a900 2024-04-25T10:04:47.127 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.115+0000 7fd3bb7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc15a900 0x7fd3bc0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.127 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.115+0000 7fd3baffd640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fd3bc0cc120 0x7fd3bc0b3350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.186:34522/0 (socket says 172.21.15.186:34522) 2024-04-25T10:04:47.127 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.115+0000 7fd3baffd640 1 -- 172.21.15.186:0/2651935520 learned_addr learned my addr 172.21.15.186:0/2651935520 (peer_addr_for_me v1:172.21.15.186:0/0) 2024-04-25T10:04:47.127 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.115+0000 7fd3bb7fe640 1 -- 172.21.15.186:0/2651935520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc15a900 msgr2=0x7fd3bc0b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:04:47.127 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.115+0000 7fd3bb7fe640 1 -- 172.21.15.186:0/2651935520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc15a900 msgr2=0x7fd3bc0b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:47.127 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.115+0000 7fd3bb7fe640 1 --2- 172.21.15.186:0/2651935520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc15a900 0x7fd3bc0b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:47.127 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.115+0000 7fd3bb7fe640 1 --2- 172.21.15.186:0/2651935520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc15a900 0x7fd3bc0b2e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:47.127 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.115+0000 7fd3ba7fc640 1 -- 172.21.15.186:0/2651935520 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957307219 0 0) 0x7fd3bc06e530 con 0x7fd3bc0cc120 2024-04-25T10:04:47.128 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.115+0000 7fd3ba7fc640 1 -- 172.21.15.186:0/2651935520 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3a4003520 con 0x7fd3bc0cc120 2024-04-25T10:04:47.128 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.115+0000 7fd3ba7fc640 1 -- 172.21.15.186:0/2651935520 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3552161787 0 0) 0x7fd3bc0af5f0 con 0x7fd3bc06f010 2024-04-25T10:04:47.128 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.115+0000 7fd3ba7fc640 1 -- 172.21.15.186:0/2651935520 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3bc06e530 con 0x7fd3bc06f010 2024-04-25T10:04:47.128 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3ba7fc640 1 -- 172.21.15.186:0/2651935520 <== mon.1 v1:172.21.15.186:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3363108380 0 0) 0x7fd3a4003520 con 0x7fd3bc0cc120 2024-04-25T10:04:47.128 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3ba7fc640 1 -- 172.21.15.186:0/2651935520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc15a900 msgr2=0x7fd3bc0b2e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:04:47.128 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3ba7fc640 1 --2- 172.21.15.186:0/2651935520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc15a900 0x7fd3bc0b2e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.128 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3ba7fc640 1 -- 172.21.15.186:0/2651935520 >> v1:172.21.15.16:6789/0 conn(0x7fd3bc06f010 legacy=0x7fd3bc0ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:47.128 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3ba7fc640 1 -- 172.21.15.186:0/2651935520 --> v1:172.21.15.186:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3bc06e810 con 0x7fd3bc0cc120 2024-04-25T10:04:47.128 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3ba7fc640 1 -- 172.21.15.186:0/2651935520 <== mon.1 v1:172.21.15.186:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fd3ac001670 con 0x7fd3bc0cc120 2024-04-25T10:04:47.128 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3ba7fc640 1 -- 172.21.15.186:0/2651935520 >> v1:172.21.15.186:6789/0 conn(0x7fd3bc0cc120 legacy=0x7fd3bc0b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:47.128 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3ba7fc640 1 --2- 172.21.15.186:0/2651935520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a4003b40 0x7fd3a4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.128 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3ba7fc640 1 --2- 172.21.15.186:0/2651935520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc15a900 0x7fd3a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.128 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3ba7fc640 1 --2- 172.21.15.186:0/2651935520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3a4004d40 0x7fd3a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3ba7fc640 1 -- 172.21.15.186:0/2651935520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc0af8d0 con 0x7fd3a4003b40 2024-04-25T10:04:47.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3ba7fc640 1 -- 172.21.15.186:0/2651935520 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a400bc40 con 0x7fd3a4004d40 2024-04-25T10:04:47.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3ba7fc640 1 -- 172.21.15.186:0/2651935520 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a400be40 con 0x7fd3bc15a900 2024-04-25T10:04:47.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3baffd640 1 --2- 172.21.15.186:0/2651935520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc15a900 0x7fd3a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3bbfff640 1 --2- 172.21.15.186:0/2651935520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3a4004d40 0x7fd3a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3bb7fe640 1 --2- 172.21.15.186:0/2651935520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a4003b40 0x7fd3a4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3baffd640 1 -- 172.21.15.186:0/2651935520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3a4004d40 msgr2=0x7fd3a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:47.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3baffd640 1 --2- 172.21.15.186:0/2651935520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3a4004d40 0x7fd3a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3baffd640 1 -- 172.21.15.186:0/2651935520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a4003b40 msgr2=0x7fd3a4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:47.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3baffd640 1 --2- 172.21.15.186:0/2651935520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a4003b40 0x7fd3a4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3baffd640 1 -- 172.21.15.186:0/2651935520 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a4003520 con 0x7fd3bc15a900 2024-04-25T10:04:47.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.116+0000 7fd3bbfff640 1 --2- 172.21.15.186:0/2651935520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3a4004d40 0x7fd3a40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:47.129 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.117+0000 7fd3baffd640 1 --2- 172.21.15.186:0/2651935520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc15a900 0x7fd3a4004800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00c1c0 tx=0x7fd3a4005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad24273b8c89ca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:47.130 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.117+0000 7fd3ba7fc640 1 -- 172.21.15.186:0/2651935520 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac007cc0 con 0x7fd3bc15a900 2024-04-25T10:04:47.130 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.117+0000 7fd3ba7fc640 1 -- 172.21.15.186:0/2651935520 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac005ad0 con 0x7fd3bc15a900 2024-04-25T10:04:47.130 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.117+0000 7fd3ba7fc640 1 -- 172.21.15.186:0/2651935520 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac007070 con 0x7fd3bc15a900 2024-04-25T10:04:47.130 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.118+0000 7fd3c21e0640 1 -- 172.21.15.186:0/2651935520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc15a900 msgr2=0x7fd3a4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:47.130 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.118+0000 7fd3c21e0640 1 --2- 172.21.15.186:0/2651935520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc15a900 0x7fd3a4004800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00c1c0 tx=0x7fd3a4005830 comp rx=0 tx=0).stop 2024-04-25T10:04:47.130 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.118+0000 7fd3bbfff640 1 -- 172.21.15.186:0/2651935520 reap_dead start 2024-04-25T10:04:47.130 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.118+0000 7fd3c21e0640 1 -- 172.21.15.186:0/2651935520 shutdown_connections 2024-04-25T10:04:47.130 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.118+0000 7fd3c21e0640 1 -- 172.21.15.186:0/2651935520 >> 172.21.15.186:0/2651935520 conn(0x7fd3bc0cff20 msgr2=0x7fd3bc0d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:47.130 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.118+0000 7fd3c21e0640 1 -- 172.21.15.186:0/2651935520 shutdown_connections 2024-04-25T10:04:47.130 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.118+0000 7fd3c21e0640 1 -- 172.21.15.186:0/2651935520 wait complete. 2024-04-25T10:04:47.130 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.119+0000 7fd3c21e0640 1 Processor -- start 2024-04-25T10:04:47.130 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.119+0000 7fd3c21e0640 1 -- start start 2024-04-25T10:04:47.131 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3c21e0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3bc15a900 0x7fd3bc1f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.131 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3c21e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3bc1f7f40 0x7fd3bc1550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.131 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3c21e0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc151ce0 0x7fd3bc1e7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.131 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3c21e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc0b2e10 con 0x7fd3bc1f7f40 2024-04-25T10:04:47.131 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3c21e0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc06eb80 con 0x7fd3bc15a900 2024-04-25T10:04:47.131 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3c21e0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc0b2f90 con 0x7fd3bc151ce0 2024-04-25T10:04:47.131 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3bb7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3bc15a900 0x7fd3bc1f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.131 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3bbfff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc151ce0 0x7fd3bc1e7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.131 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3bb7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3bc15a900 0x7fd3bc1f4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58690/0 (socket says 172.21.15.186:58690) 2024-04-25T10:04:47.131 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3bb7fe640 1 -- 172.21.15.186:0/3690223964 learned_addr learned my addr 172.21.15.186:0/3690223964 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T10:04:47.131 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3bbfff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc151ce0 0x7fd3bc1e7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:45280/0 (socket says 172.21.15.186:45280) 2024-04-25T10:04:47.131 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3baffd640 1 --2- 172.21.15.186:0/3690223964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3bc1f7f40 0x7fd3bc1550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.131 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3bbfff640 1 -- 172.21.15.186:0/3690223964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc151ce0 msgr2=0x7fd3bc1e7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-25T10:04:47.131 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3bbfff640 1 -- 172.21.15.186:0/3690223964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc151ce0 msgr2=0x7fd3bc1e7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:04:47.132 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3bbfff640 1 --2- 172.21.15.186:0/3690223964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc151ce0 0x7fd3bc1e7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:47.132 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3bbfff640 1 --2- 172.21.15.186:0/3690223964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc151ce0 0x7fd3bc1e7470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:47.132 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3bb7fe640 1 -- 172.21.15.186:0/3690223964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc151ce0 msgr2=0x7fd3bc1e7470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:04:47.132 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3bb7fe640 1 --2- 172.21.15.186:0/3690223964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc151ce0 0x7fd3bc1e7470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.132 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3bb7fe640 1 -- 172.21.15.186:0/3690223964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3bc1f7f40 msgr2=0x7fd3bc1550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:47.132 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3bb7fe640 1 --2- 172.21.15.186:0/3690223964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3bc1f7f40 0x7fd3bc1550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.132 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3bb7fe640 1 -- 172.21.15.186:0/3690223964 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3bc155620 con 0x7fd3bc15a900 2024-04-25T10:04:47.132 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.120+0000 7fd3bb7fe640 1 --2- 172.21.15.186:0/3690223964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3bc15a900 0x7fd3bc1f4920 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0010920 tx=0x7fd3b0010de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:47.132 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.121+0000 7fd3a3fff640 1 -- 172.21.15.186:0/3690223964 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b001a020 con 0x7fd3bc15a900 2024-04-25T10:04:47.132 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.121+0000 7fd3a3fff640 1 -- 172.21.15.186:0/3690223964 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b000e9c0 con 0x7fd3bc15a900 2024-04-25T10:04:47.132 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.121+0000 7fd3c21e0640 1 -- 172.21.15.186:0/3690223964 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3bc1558c0 con 0x7fd3bc15a900 2024-04-25T10:04:47.132 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.121+0000 7fd3a3fff640 1 -- 172.21.15.186:0/3690223964 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b000ed90 con 0x7fd3bc15a900 2024-04-25T10:04:47.132 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.121+0000 7fd3c21e0640 1 -- 172.21.15.186:0/3690223964 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3bc1f7740 con 0x7fd3bc15a900 2024-04-25T10:04:47.133 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.121+0000 7fd3a3fff640 1 -- 172.21.15.186:0/3690223964 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd3b001d020 con 0x7fd3bc15a900 2024-04-25T10:04:47.133 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.121+0000 7fd3a3fff640 1 -- 172.21.15.186:0/3690223964 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd3b0014790 con 0x7fd3bc15a900 2024-04-25T10:04:47.133 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.122+0000 7fd3c21e0640 1 -- 172.21.15.186:0/3690223964 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3bc075090 con 0x7fd3bc15a900 2024-04-25T10:04:47.133 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.124+0000 7fd3a3fff640 1 -- 172.21.15.186:0/3690223964 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd3b0024020 con 0x7fd3bc15a900 2024-04-25T10:04:47.136 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:47.131+0000 7fc684930040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:04:47.181 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:47.176+0000 7fc684930040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:04:47.200 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.195+0000 7fd3c21e0640 1 -- 172.21.15.186:0/3690223964 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd new", "uuid": "40dcfc39-064c-4b17-bcf1-3f45d2e26b86", "id": 0} v 0) v1 -- 0x7fd3bc0cf1c0 con 0x7fd3bc15a900 2024-04-25T10:04:47.203 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.199+0000 7fd3a3fff640 1 -- 172.21.15.186:0/3690223964 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "40dcfc39-064c-4b17-bcf1-3f45d2e26b86", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd3b0016a40 con 0x7fd3bc15a900 2024-04-25T10:04:47.203 INFO:teuthology.orchestra.run.smithi186.stdout:0 2024-04-25T10:04:47.204 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.199+0000 7fd3c21e0640 1 -- 172.21.15.186:0/3690223964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3bc15a900 msgr2=0x7fd3bc1f4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:47.205 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.199+0000 7fd3c21e0640 1 --2- 172.21.15.186:0/3690223964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3bc15a900 0x7fd3bc1f4920 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0010920 tx=0x7fd3b0010de0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.205 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.200+0000 7fd3c21e0640 1 -- 172.21.15.186:0/3690223964 shutdown_connections 2024-04-25T10:04:47.205 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.200+0000 7fd3c21e0640 1 --2- 172.21.15.186:0/3690223964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3bc151ce0 0x7fd3bc1e7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.205 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.200+0000 7fd3c21e0640 1 --2- 172.21.15.186:0/3690223964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3bc1f7f40 0x7fd3bc1550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.205 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.200+0000 7fd3c21e0640 1 --2- 172.21.15.186:0/3690223964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3bc15a900 0x7fd3bc1f4920 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.205 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.200+0000 7fd3c21e0640 1 -- 172.21.15.186:0/3690223964 >> 172.21.15.186:0/3690223964 conn(0x7fd3bc0cff20 msgr2=0x7fd3bc0ae190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:47.205 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.200+0000 7fd3c21e0640 1 -- 172.21.15.186:0/3690223964 shutdown_connections 2024-04-25T10:04:47.205 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.200+0000 7fd3c21e0640 1 -- 172.21.15.186:0/3690223964 wait complete. 2024-04-25T10:04:47.214 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new eeff61d1-2c4f-4879-ac01-b803de113532 1 2024-04-25T10:04:47.267 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:47.262+0000 7fc470923040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:04:47.326 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:47.322+0000 7fc470923040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:04:47.352 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.347+0000 7f190f7e2640 1 Processor -- start 2024-04-25T10:04:47.352 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.348+0000 7f190f7e2640 1 -- start start 2024-04-25T10:04:47.353 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.348+0000 7f190f7e2640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f19080cd5f0 0x7f190815a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.354 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.348+0000 7f190f7e2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f190815a990 0x7f19080cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.354 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.348+0000 7f190f7e2640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19080cf870 con 0x7f19080cd210 2024-04-25T10:04:47.354 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.348+0000 7f190f7e2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19080cec60 con 0x7f190815a990 2024-04-25T10:04:47.354 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.348+0000 7f190f7e2640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19080cef40 con 0x7f19080cd5f0 2024-04-25T10:04:47.354 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.348+0000 7f190cd7d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f19080cd5f0 0x7f190815a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.354 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.348+0000 7f190d57e640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f19080cd210 0x7f190815baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.186:34532/0 (socket says 172.21.15.186:34532) 2024-04-25T10:04:47.354 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.348+0000 7f190cd7d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f19080cd5f0 0x7f190815a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58698/0 (socket says 172.21.15.186:58698) 2024-04-25T10:04:47.354 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.348+0000 7f190d57e640 1 -- 172.21.15.186:0/3174565733 learned_addr learned my addr 172.21.15.186:0/3174565733 (peer_addr_for_me v1:172.21.15.186:0/0) 2024-04-25T10:04:47.354 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.349+0000 7f190dd7f640 1 --2- 172.21.15.186:0/3174565733 >> v2:172.21.15.16:3300/0 conn(0x7f190815a990 0x7f19080cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.354 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.349+0000 7f190cd7d640 1 -- 172.21.15.186:0/3174565733 >> v2:172.21.15.16:3300/0 conn(0x7f190815a990 msgr2=0x7f19080cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:47.354 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.349+0000 7f190cd7d640 1 --2- 172.21.15.186:0/3174565733 >> v2:172.21.15.16:3300/0 conn(0x7f190815a990 0x7f19080cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.354 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.349+0000 7f190cd7d640 1 -- 172.21.15.186:0/3174565733 >> v1:172.21.15.186:6789/0 conn(0x7f19080cd210 legacy=0x7f190815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:47.354 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.349+0000 7f190cd7d640 1 -- 172.21.15.186:0/3174565733 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19080cf220 con 0x7f19080cd5f0 2024-04-25T10:04:47.355 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:47.349+0000 7fc684930040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:04:47.359 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.349+0000 7f190cd7d640 1 --2- 172.21.15.186:0/3174565733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19080cd5f0 0x7f190815a450 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f19080cfec0 tx=0x7f190402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c49cdd2d1a9ccd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:47.359 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.350+0000 7f18fbfff640 1 -- 172.21.15.186:0/3174565733 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904037070 con 0x7f19080cd5f0 2024-04-25T10:04:47.359 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.350+0000 7f18fbfff640 1 -- 172.21.15.186:0/3174565733 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1904032440 con 0x7f19080cd5f0 2024-04-25T10:04:47.359 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.350+0000 7f18fbfff640 1 -- 172.21.15.186:0/3174565733 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904032810 con 0x7f19080cd5f0 2024-04-25T10:04:47.359 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.350+0000 7f190f7e2640 1 -- 172.21.15.186:0/3174565733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19080cd5f0 msgr2=0x7f190815a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:47.359 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.350+0000 7f190f7e2640 1 --2- 172.21.15.186:0/3174565733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19080cd5f0 0x7f190815a450 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f19080cfec0 tx=0x7f190402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.359 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.350+0000 7f190f7e2640 1 -- 172.21.15.186:0/3174565733 shutdown_connections 2024-04-25T10:04:47.359 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.350+0000 7f190f7e2640 1 --2- 172.21.15.186:0/3174565733 >> v2:172.21.15.16:3300/0 conn(0x7f190815a990 0x7f19080cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.359 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.350+0000 7f190f7e2640 1 --2- 172.21.15.186:0/3174565733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19080cd5f0 0x7f190815a450 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.359 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.350+0000 7f190f7e2640 1 -- 172.21.15.186:0/3174565733 >> 172.21.15.186:0/3174565733 conn(0x7f1908099a40 msgr2=0x7f19080d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:47.360 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.350+0000 7f190f7e2640 1 -- 172.21.15.186:0/3174565733 shutdown_connections 2024-04-25T10:04:47.360 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.350+0000 7f190f7e2640 1 -- 172.21.15.186:0/3174565733 wait complete. 2024-04-25T10:04:47.360 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.352+0000 7f190f7e2640 1 Processor -- start 2024-04-25T10:04:47.360 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.352+0000 7f190f7e2640 1 -- start start 2024-04-25T10:04:47.360 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.352+0000 7f190f7e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19080cd210 0x7f190809e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.360 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.352+0000 7f190f7e2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19080cd5f0 0x7f1908151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.360 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.352+0000 7f190f7e2640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f190815a990 0x7f190806aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.360 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.352+0000 7f190f7e2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190815ae20 con 0x7f19080cd210 2024-04-25T10:04:47.360 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.352+0000 7f190f7e2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19080d1430 con 0x7f19080cd5f0 2024-04-25T10:04:47.361 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.352+0000 7f190f7e2640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19080cda30 con 0x7f190815a990 2024-04-25T10:04:47.361 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.352+0000 7f190cd7d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19080cd5f0 0x7f1908151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.361 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.352+0000 7f190dd7f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f190815a990 0x7f190806aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.361 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.352+0000 7f190cd7d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19080cd5f0 0x7f1908151b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58710/0 (socket says 172.21.15.186:58710) 2024-04-25T10:04:47.361 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.352+0000 7f190cd7d640 1 -- 172.21.15.186:0/1207400021 learned_addr learned my addr 172.21.15.186:0/1207400021 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T10:04:47.361 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.352+0000 7f190d57e640 1 --2- 172.21.15.186:0/1207400021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19080cd210 0x7f190809e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.361 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.353+0000 7f190dd7f640 1 -- 172.21.15.186:0/1207400021 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f190815a990 msgr2=0x7f190806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-25T10:04:47.361 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.353+0000 7f190dd7f640 1 -- 172.21.15.186:0/1207400021 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f190815a990 msgr2=0x7f190806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:04:47.361 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.353+0000 7f190dd7f640 1 --2- 172.21.15.186:0/1207400021 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f190815a990 0x7f190806aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:47.361 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.353+0000 7f190dd7f640 1 --2- 172.21.15.186:0/1207400021 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f190815a990 0x7f190806aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:47.361 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.353+0000 7f190cd7d640 1 -- 172.21.15.186:0/1207400021 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f190815a990 msgr2=0x7f190806aba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:04:47.361 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.353+0000 7f190cd7d640 1 --2- 172.21.15.186:0/1207400021 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f190815a990 0x7f190806aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.361 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.353+0000 7f190cd7d640 1 -- 172.21.15.186:0/1207400021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19080cd210 msgr2=0x7f190809e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:47.361 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.353+0000 7f190cd7d640 1 --2- 172.21.15.186:0/1207400021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19080cd210 0x7f190809e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.362 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.353+0000 7f190cd7d640 1 -- 172.21.15.186:0/1207400021 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190809e5b0 con 0x7f19080cd5f0 2024-04-25T10:04:47.362 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.353+0000 7f190cd7d640 1 --2- 172.21.15.186:0/1207400021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19080cd5f0 0x7f1908151b60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1904000c00 tx=0x7f19040046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:47.362 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.353+0000 7f18f9ffb640 1 -- 172.21.15.186:0/1207400021 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904041070 con 0x7f19080cd5f0 2024-04-25T10:04:47.362 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.353+0000 7f18f9ffb640 1 -- 172.21.15.186:0/1207400021 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190403c890 con 0x7f19080cd5f0 2024-04-25T10:04:47.362 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.353+0000 7f18f9ffb640 1 -- 172.21.15.186:0/1207400021 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904037040 con 0x7f19080cd5f0 2024-04-25T10:04:47.362 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.353+0000 7f190f7e2640 1 -- 172.21.15.186:0/1207400021 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190809a8d0 con 0x7f19080cd5f0 2024-04-25T10:04:47.362 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.353+0000 7f190f7e2640 1 -- 172.21.15.186:0/1207400021 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190809ae90 con 0x7f19080cd5f0 2024-04-25T10:04:47.362 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.354+0000 7f18f9ffb640 1 -- 172.21.15.186:0/1207400021 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1904044070 con 0x7f19080cd5f0 2024-04-25T10:04:47.362 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.354+0000 7f18f9ffb640 1 -- 172.21.15.186:0/1207400021 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f1904046df0 con 0x7f19080cd5f0 2024-04-25T10:04:47.362 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.354+0000 7f190f7e2640 1 -- 172.21.15.186:0/1207400021 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19080750b0 con 0x7f19080cd5f0 2024-04-25T10:04:47.363 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.356+0000 7f18f9ffb640 1 -- 172.21.15.186:0/1207400021 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f190403cc80 con 0x7f19080cd5f0 2024-04-25T10:04:47.383 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:47.378+0000 7fc470923040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:04:47.416 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:47.411+0000 7fc684930040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:04:47.434 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.429+0000 7f190f7e2640 1 -- 172.21.15.186:0/1207400021 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd new", "uuid": "eeff61d1-2c4f-4879-ac01-b803de113532", "id": 1} v 0) v1 -- 0x7f19080ccff0 con 0x7f19080cd5f0 2024-04-25T10:04:47.438 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.433+0000 7f18f9ffb640 1 -- 172.21.15.186:0/1207400021 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "eeff61d1-2c4f-4879-ac01-b803de113532", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1904046450 con 0x7f19080cd5f0 2024-04-25T10:04:47.438 INFO:teuthology.orchestra.run.smithi186.stdout:1 2024-04-25T10:04:47.439 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.433+0000 7f190f7e2640 1 -- 172.21.15.186:0/1207400021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19080cd5f0 msgr2=0x7f1908151b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:47.439 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.433+0000 7f190f7e2640 1 --2- 172.21.15.186:0/1207400021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19080cd5f0 0x7f1908151b60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1904000c00 tx=0x7f19040046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.439 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.434+0000 7f190f7e2640 1 -- 172.21.15.186:0/1207400021 shutdown_connections 2024-04-25T10:04:47.439 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.434+0000 7f190f7e2640 1 --2- 172.21.15.186:0/1207400021 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f190815a990 0x7f190806aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.439 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.434+0000 7f190f7e2640 1 --2- 172.21.15.186:0/1207400021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19080cd5f0 0x7f1908151b60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.439 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.434+0000 7f190f7e2640 1 --2- 172.21.15.186:0/1207400021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19080cd210 0x7f190809e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.440 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.434+0000 7f190f7e2640 1 -- 172.21.15.186:0/1207400021 >> 172.21.15.186:0/1207400021 conn(0x7f1908099a40 msgr2=0x7f19080cb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:47.440 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.434+0000 7f190f7e2640 1 -- 172.21.15.186:0/1207400021 shutdown_connections 2024-04-25T10:04:47.440 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.434+0000 7f190f7e2640 1 -- 172.21.15.186:0/1207400021 wait complete. 2024-04-25T10:04:47.448 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 49fff454-4f8d-46c8-b700-ca1380a49ca7 2 2024-04-25T10:04:47.471 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:47.466+0000 7fc684930040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:04:47.499 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:47.494+0000 7fc470923040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:04:47.586 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.582+0000 7f0772590640 1 Processor -- start 2024-04-25T10:04:47.586 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.582+0000 7f0772590640 1 -- start start 2024-04-25T10:04:47.588 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.582+0000 7f0772590640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f076c1617a0 0x7f076c0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.588 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.582+0000 7f0772590640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f076c0b3350 0x7f076c06b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.588 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.582+0000 7f0772590640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f076c161d00 con 0x7f076c0af570 2024-04-25T10:04:47.588 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.582+0000 7f0772590640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f076c06aba0 con 0x7f076c1617a0 2024-04-25T10:04:47.588 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.582+0000 7f0772590640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f076c161fe0 con 0x7f076c0b3350 2024-04-25T10:04:47.589 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.583+0000 7f0770b2d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f076c0b3350 0x7f076c06b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.589 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.583+0000 7f0770b2d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f076c0b3350 0x7f076c06b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58716/0 (socket says 172.21.15.186:58716) 2024-04-25T10:04:47.589 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.583+0000 7f0770b2d640 1 -- 172.21.15.186:0/549696994 learned_addr learned my addr 172.21.15.186:0/549696994 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T10:04:47.589 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.583+0000 7f076b7fe640 1 --2- 172.21.15.186:0/549696994 >> v2:172.21.15.16:3300/0 conn(0x7f076c1617a0 0x7f076c0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.589 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.583+0000 7f0770b2d640 1 -- 172.21.15.186:0/549696994 >> v2:172.21.15.16:3300/0 conn(0x7f076c1617a0 msgr2=0x7f076c0b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:47.589 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.583+0000 7f0770b2d640 1 --2- 172.21.15.186:0/549696994 >> v2:172.21.15.16:3300/0 conn(0x7f076c1617a0 0x7f076c0b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.589 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.583+0000 7f0770b2d640 1 -- 172.21.15.186:0/549696994 >> v1:172.21.15.186:6789/0 conn(0x7f076c0af570 legacy=0x7f076c15ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:47.589 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.583+0000 7f0770b2d640 1 -- 172.21.15.186:0/549696994 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076c1622c0 con 0x7f076c0b3350 2024-04-25T10:04:47.589 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.584+0000 7f0770b2d640 1 --2- 172.21.15.186:0/549696994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f076c0b3350 0x7f076c06b030 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f076c071910 tx=0x7f076402a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93db66bf83021510 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:47.589 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.584+0000 7f076affd640 1 -- 172.21.15.186:0/549696994 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764036070 con 0x7f076c0b3350 2024-04-25T10:04:47.589 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.584+0000 7f076affd640 1 -- 172.21.15.186:0/549696994 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0764031600 con 0x7f076c0b3350 2024-04-25T10:04:47.589 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.584+0000 7f076affd640 1 -- 172.21.15.186:0/549696994 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07640319d0 con 0x7f076c0b3350 2024-04-25T10:04:47.590 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:47.583+0000 7fc684930040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:04:47.593 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.584+0000 7f0772590640 1 -- 172.21.15.186:0/549696994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f076c0b3350 msgr2=0x7f076c06b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:47.593 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.584+0000 7f0772590640 1 --2- 172.21.15.186:0/549696994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f076c0b3350 0x7f076c06b030 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f076c071910 tx=0x7f076402a730 comp rx=0 tx=0).stop 2024-04-25T10:04:47.593 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.584+0000 7f0772590640 1 -- 172.21.15.186:0/549696994 shutdown_connections 2024-04-25T10:04:47.594 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.584+0000 7f0772590640 1 --2- 172.21.15.186:0/549696994 >> v2:172.21.15.16:3300/0 conn(0x7f076c1617a0 0x7f076c0b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.594 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.584+0000 7f0772590640 1 --2- 172.21.15.186:0/549696994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f076c0b3350 0x7f076c06b030 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.594 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.584+0000 7f0772590640 1 -- 172.21.15.186:0/549696994 >> 172.21.15.186:0/549696994 conn(0x7f076c0cff20 msgr2=0x7f076c0d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:47.594 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.584+0000 7f0772590640 1 -- 172.21.15.186:0/549696994 shutdown_connections 2024-04-25T10:04:47.594 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.584+0000 7f0772590640 1 -- 172.21.15.186:0/549696994 wait complete. 2024-04-25T10:04:47.594 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.586+0000 7f0772590640 1 Processor -- start 2024-04-25T10:04:47.594 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.586+0000 7f0772590640 1 -- start start 2024-04-25T10:04:47.594 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.586+0000 7f0772590640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f076c0af570 0x7f076c0c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.594 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.586+0000 7f0772590640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f076c0b3350 0x7f076c0c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.594 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.586+0000 7f0772590640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f076c1617a0 0x7f076c0c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.594 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.586+0000 7f0772590640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c06b6f0 con 0x7f076c1617a0 2024-04-25T10:04:47.594 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.586+0000 7f0772590640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c06b570 con 0x7f076c0af570 2024-04-25T10:04:47.594 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.586+0000 7f0772590640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076c06b870 con 0x7f076c0b3350 2024-04-25T10:04:47.594 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f076bfff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f076c0af570 0x7f076c0c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f076bfff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f076c0af570 0x7f076c0c5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58724/0 (socket says 172.21.15.186:58724) 2024-04-25T10:04:47.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f076b7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f076c0b3350 0x7f076c0c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f076bfff640 1 -- 172.21.15.186:0/2951472592 learned_addr learned my addr 172.21.15.186:0/2951472592 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T10:04:47.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f0770b2d640 1 --2- 172.21.15.186:0/2951472592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f076c1617a0 0x7f076c0c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f076b7fe640 1 -- 172.21.15.186:0/2951472592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f076c0b3350 msgr2=0x7f076c0c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T10:04:47.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f076b7fe640 1 -- 172.21.15.186:0/2951472592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f076c0b3350 msgr2=0x7f076c0c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:04:47.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f076b7fe640 1 --2- 172.21.15.186:0/2951472592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f076c0b3350 0x7f076c0c79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:47.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f076b7fe640 1 --2- 172.21.15.186:0/2951472592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f076c0b3350 0x7f076c0c79e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:47.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f0770b2d640 1 -- 172.21.15.186:0/2951472592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f076c0b3350 msgr2=0x7f076c0c79e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:04:47.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f0770b2d640 1 --2- 172.21.15.186:0/2951472592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f076c0b3350 0x7f076c0c79e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f0770b2d640 1 -- 172.21.15.186:0/2951472592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f076c0af570 msgr2=0x7f076c0c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:47.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f0770b2d640 1 --2- 172.21.15.186:0/2951472592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f076c0af570 0x7f076c0c5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f0770b2d640 1 -- 172.21.15.186:0/2951472592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076c0c7050 con 0x7f076c1617a0 2024-04-25T10:04:47.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f076bfff640 1 --2- 172.21.15.186:0/2951472592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f076c0af570 0x7f076c0c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:47.596 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f0770b2d640 1 --2- 172.21.15.186:0/2951472592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f076c1617a0 0x7f076c0c7f20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0764030000 tx=0x7f0764004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:47.596 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f0768ff9640 1 -- 172.21.15.186:0/2951472592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076403f020 con 0x7f076c1617a0 2024-04-25T10:04:47.596 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f0768ff9640 1 -- 172.21.15.186:0/2951472592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0764037040 con 0x7f076c1617a0 2024-04-25T10:04:47.596 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f0772590640 1 -- 172.21.15.186:0/2951472592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f076c0c6a90 con 0x7f076c1617a0 2024-04-25T10:04:47.596 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f0768ff9640 1 -- 172.21.15.186:0/2951472592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764036070 con 0x7f076c1617a0 2024-04-25T10:04:47.596 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.587+0000 7f0772590640 1 -- 172.21.15.186:0/2951472592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f076c0cd5e0 con 0x7f076c1617a0 2024-04-25T10:04:47.596 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.588+0000 7f0768ff9640 1 -- 172.21.15.186:0/2951472592 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0764042020 con 0x7f076c1617a0 2024-04-25T10:04:47.596 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.588+0000 7f0768ff9640 1 -- 172.21.15.186:0/2951472592 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f0764044910 con 0x7f076c1617a0 2024-04-25T10:04:47.596 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.588+0000 7f0772590640 1 -- 172.21.15.186:0/2951472592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f076c06b030 con 0x7f076c1617a0 2024-04-25T10:04:47.596 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.591+0000 7f0768ff9640 1 -- 172.21.15.186:0/2951472592 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f076c06b030 con 0x7f076c1617a0 2024-04-25T10:04:47.669 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.664+0000 7f0772590640 1 -- 172.21.15.186:0/2951472592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd new", "uuid": "49fff454-4f8d-46c8-b700-ca1380a49ca7", "id": 2} v 0) v1 -- 0x7f076c0750b0 con 0x7f076c1617a0 2024-04-25T10:04:47.670 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.666+0000 7f0768ff9640 1 -- 172.21.15.186:0/2951472592 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "49fff454-4f8d-46c8-b700-ca1380a49ca7", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0764049030 con 0x7f076c1617a0 2024-04-25T10:04:47.671 INFO:teuthology.orchestra.run.smithi186.stdout:2 2024-04-25T10:04:47.672 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.667+0000 7f0772590640 1 -- 172.21.15.186:0/2951472592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f076c1617a0 msgr2=0x7f076c0c7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:47.672 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.667+0000 7f0772590640 1 --2- 172.21.15.186:0/2951472592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f076c1617a0 0x7f076c0c7f20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0764030000 tx=0x7f0764004a40 comp rx=0 tx=0).stop 2024-04-25T10:04:47.672 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.667+0000 7f0772590640 1 -- 172.21.15.186:0/2951472592 shutdown_connections 2024-04-25T10:04:47.672 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.667+0000 7f0772590640 1 --2- 172.21.15.186:0/2951472592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f076c1617a0 0x7f076c0c7f20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.672 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.667+0000 7f0772590640 1 --2- 172.21.15.186:0/2951472592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f076c0b3350 0x7f076c0c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.672 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.667+0000 7f0772590640 1 --2- 172.21.15.186:0/2951472592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f076c0af570 0x7f076c0c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.672 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.667+0000 7f0772590640 1 -- 172.21.15.186:0/2951472592 >> 172.21.15.186:0/2951472592 conn(0x7f076c0cff20 msgr2=0x7f076c0b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:47.672 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.667+0000 7f0772590640 1 -- 172.21.15.186:0/2951472592 shutdown_connections 2024-04-25T10:04:47.672 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.667+0000 7f0772590640 1 -- 172.21.15.186:0/2951472592 wait complete. 2024-04-25T10:04:47.682 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 6ed9721d-2101-49a5-9ac4-61696c382866 3 2024-04-25T10:04:47.820 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.816+0000 7fa8e469e640 1 Processor -- start 2024-04-25T10:04:47.821 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.816+0000 7fa8e469e640 1 -- start start 2024-04-25T10:04:47.823 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e469e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa8dc15de70 0x7fa8dc0cead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.823 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e469e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa8dc0b0730 0x7fa8dc0b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.823 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e469e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 0x7fa8dc0b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.823 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e469e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8dc15e250 con 0x7fa8dc0b0730 2024-04-25T10:04:47.824 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e469e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8dc0b4a00 con 0x7fa8dc15de70 2024-04-25T10:04:47.824 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e469e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc0cf010 con 0x7fa8dc0b0b90 2024-04-25T10:04:47.824 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e243a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa8dc15de70 0x7fa8dc0cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.824 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e2c3b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 0x7fa8dc0b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.824 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e243a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa8dc15de70 0x7fa8dc0cead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58734/0 (socket says 172.21.15.186:58734) 2024-04-25T10:04:47.824 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e243a640 1 -- 172.21.15.186:0/2901279233 learned_addr learned my addr 172.21.15.186:0/2901279233 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T10:04:47.824 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e1c39640 1 --2- 172.21.15.186:0/2901279233 >> v2:172.21.15.16:3300/0 conn(0x7fa8dc0b0730 0x7fa8dc0b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.824 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e2c3b640 1 -- 172.21.15.186:0/2901279233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 msgr2=0x7fa8dc0b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:04:47.824 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e2c3b640 1 -- 172.21.15.186:0/2901279233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 msgr2=0x7fa8dc0b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:47.824 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e2c3b640 1 --2- 172.21.15.186:0/2901279233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 0x7fa8dc0b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:47.824 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e2c3b640 1 --2- 172.21.15.186:0/2901279233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 0x7fa8dc0b5af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:47.824 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e243a640 1 -- 172.21.15.186:0/2901279233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 msgr2=0x7fa8dc0b5af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:04:47.824 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e243a640 1 --2- 172.21.15.186:0/2901279233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 0x7fa8dc0b5af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.824 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e243a640 1 -- 172.21.15.186:0/2901279233 >> v2:172.21.15.16:3300/0 conn(0x7fa8dc0b0730 msgr2=0x7fa8dc0b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:47.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e243a640 1 --2- 172.21.15.186:0/2901279233 >> v2:172.21.15.16:3300/0 conn(0x7fa8dc0b0730 0x7fa8dc0b55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.817+0000 7fa8e243a640 1 -- 172.21.15.186:0/2901279233 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc06bcb0 con 0x7fa8dc15de70 2024-04-25T10:04:47.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.818+0000 7fa8e243a640 1 --2- 172.21.15.186:0/2901279233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8dc15de70 0x7fa8dc0cead0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0709e0 tx=0x7fa8d802a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b9117cd5720c07c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:47.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.818+0000 7fa8e1438640 1 -- 172.21.15.186:0/2901279233 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8037070 con 0x7fa8dc15de70 2024-04-25T10:04:47.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.818+0000 7fa8e1438640 1 -- 172.21.15.186:0/2901279233 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d8032440 con 0x7fa8dc15de70 2024-04-25T10:04:47.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.818+0000 7fa8e1438640 1 -- 172.21.15.186:0/2901279233 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8032810 con 0x7fa8dc15de70 2024-04-25T10:04:47.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.819+0000 7fa8e469e640 1 -- 172.21.15.186:0/2901279233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8dc15de70 msgr2=0x7fa8dc0cead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:47.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.819+0000 7fa8e469e640 1 --2- 172.21.15.186:0/2901279233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8dc15de70 0x7fa8dc0cead0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0709e0 tx=0x7fa8d802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.819+0000 7fa8e469e640 1 -- 172.21.15.186:0/2901279233 shutdown_connections 2024-04-25T10:04:47.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.819+0000 7fa8e469e640 1 --2- 172.21.15.186:0/2901279233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 0x7fa8dc0b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.819+0000 7fa8e469e640 1 --2- 172.21.15.186:0/2901279233 >> v2:172.21.15.16:3300/0 conn(0x7fa8dc0b0730 0x7fa8dc0b55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.819+0000 7fa8e469e640 1 --2- 172.21.15.186:0/2901279233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8dc15de70 0x7fa8dc0cead0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.819+0000 7fa8e469e640 1 -- 172.21.15.186:0/2901279233 >> 172.21.15.186:0/2901279233 conn(0x7fa8dc0b3500 msgr2=0x7fa8dc0b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:47.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.819+0000 7fa8e469e640 1 -- 172.21.15.186:0/2901279233 shutdown_connections 2024-04-25T10:04:47.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.819+0000 7fa8e469e640 1 -- 172.21.15.186:0/2901279233 wait complete. 2024-04-25T10:04:47.829 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.820+0000 7fa8e469e640 1 Processor -- start 2024-04-25T10:04:47.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.820+0000 7fa8e469e640 1 -- start start 2024-04-25T10:04:47.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.820+0000 7fa8e469e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8dc0b0730 0x7fa8dc15fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e469e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 0x7fa8dc0cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e469e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc15de70 0x7fa8dc0a8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:47.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e469e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc06c550 con 0x7fa8dc15de70 2024-04-25T10:04:47.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e469e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc06c270 con 0x7fa8dc0b0730 2024-04-25T10:04:47.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e469e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc06c830 con 0x7fa8dc0b0b90 2024-04-25T10:04:47.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e243a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8dc0b0730 0x7fa8dc15fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e1c39640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 0x7fa8dc0cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e1c39640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 0x7fa8dc0cabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:45322/0 (socket says 172.21.15.186:45322) 2024-04-25T10:04:47.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e1c39640 1 -- 172.21.15.186:0/2780226922 learned_addr learned my addr 172.21.15.186:0/2780226922 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T10:04:47.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e1c39640 1 -- 172.21.15.186:0/2780226922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 msgr2=0x7fa8dc0cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T10:04:47.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e2c3b640 1 --2- 172.21.15.186:0/2780226922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc15de70 0x7fa8dc0a8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:47.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e1c39640 1 -- 172.21.15.186:0/2780226922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 msgr2=0x7fa8dc0cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:04:47.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e1c39640 1 --2- 172.21.15.186:0/2780226922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 0x7fa8dc0cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:47.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e1c39640 1 --2- 172.21.15.186:0/2780226922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 0x7fa8dc0cabc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:47.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e243a640 1 -- 172.21.15.186:0/2780226922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 msgr2=0x7fa8dc0cabc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:04:47.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e243a640 1 --2- 172.21.15.186:0/2780226922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 0x7fa8dc0cabc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e243a640 1 -- 172.21.15.186:0/2780226922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc15de70 msgr2=0x7fa8dc0a8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:47.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e243a640 1 --2- 172.21.15.186:0/2780226922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc15de70 0x7fa8dc0a8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e243a640 1 -- 172.21.15.186:0/2780226922 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc0cb680 con 0x7fa8dc0b0730 2024-04-25T10:04:47.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e243a640 1 --2- 172.21.15.186:0/2780226922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8dc0b0730 0x7fa8dc15fb40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8031000 tx=0x7fa8d80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:47.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8d2ffd640 1 -- 172.21.15.186:0/2780226922 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8041030 con 0x7fa8dc0b0730 2024-04-25T10:04:47.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8d2ffd640 1 -- 172.21.15.186:0/2780226922 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d8030440 con 0x7fa8dc0b0730 2024-04-25T10:04:47.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e469e640 1 -- 172.21.15.186:0/2780226922 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8dc0cb960 con 0x7fa8dc0b0730 2024-04-25T10:04:47.832 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8d2ffd640 1 -- 172.21.15.186:0/2780226922 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8037040 con 0x7fa8dc0b0730 2024-04-25T10:04:47.832 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.821+0000 7fa8e469e640 1 -- 172.21.15.186:0/2780226922 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8dc0cc650 con 0x7fa8dc0b0730 2024-04-25T10:04:47.832 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.822+0000 7fa8d2ffd640 1 -- 172.21.15.186:0/2780226922 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa8d8044070 con 0x7fa8dc0b0730 2024-04-25T10:04:47.832 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.822+0000 7fa8d2ffd640 1 -- 172.21.15.186:0/2780226922 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fa8d803da40 con 0x7fa8dc0b0730 2024-04-25T10:04:47.832 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.822+0000 7fa8e469e640 1 -- 172.21.15.186:0/2780226922 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8dc0750b0 con 0x7fa8dc0b0730 2024-04-25T10:04:47.832 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.824+0000 7fa8d2ffd640 1 -- 172.21.15.186:0/2780226922 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa8d803dcd0 con 0x7fa8dc0b0730 2024-04-25T10:04:47.897 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:47.892+0000 7fc470923040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:04:47.899 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.895+0000 7fa8e469e640 1 -- 172.21.15.186:0/2780226922 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6ed9721d-2101-49a5-9ac4-61696c382866", "id": 3} v 0) v1 -- 0x7fa8dc0cc820 con 0x7fa8dc0b0730 2024-04-25T10:04:47.902 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.898+0000 7fa8d2ffd640 1 -- 172.21.15.186:0/2780226922 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6ed9721d-2101-49a5-9ac4-61696c382866", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa8d803d5b0 con 0x7fa8dc0b0730 2024-04-25T10:04:47.902 INFO:teuthology.orchestra.run.smithi186.stdout:3 2024-04-25T10:04:47.904 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.899+0000 7fa8e469e640 1 -- 172.21.15.186:0/2780226922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8dc0b0730 msgr2=0x7fa8dc15fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:47.904 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.899+0000 7fa8e469e640 1 --2- 172.21.15.186:0/2780226922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8dc0b0730 0x7fa8dc15fb40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8031000 tx=0x7fa8d80046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.904 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.899+0000 7fa8e469e640 1 -- 172.21.15.186:0/2780226922 shutdown_connections 2024-04-25T10:04:47.904 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.899+0000 7fa8e469e640 1 --2- 172.21.15.186:0/2780226922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc15de70 0x7fa8dc0a8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.904 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.899+0000 7fa8e469e640 1 --2- 172.21.15.186:0/2780226922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8dc0b0b90 0x7fa8dc0cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.904 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.899+0000 7fa8e469e640 1 --2- 172.21.15.186:0/2780226922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8dc0b0730 0x7fa8dc15fb40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:47.904 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.899+0000 7fa8e469e640 1 -- 172.21.15.186:0/2780226922 >> 172.21.15.186:0/2780226922 conn(0x7fa8dc0b3500 msgr2=0x7fa8dc161cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:47.904 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.899+0000 7fa8e469e640 1 -- 172.21.15.186:0/2780226922 shutdown_connections 2024-04-25T10:04:47.904 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:47.899+0000 7fa8e469e640 1 -- 172.21.15.186:0/2780226922 wait complete. 2024-04-25T10:04:47.914 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 46edf17f-ad1d-404a-b94a-9ed0cc9a7226 4 2024-04-25T10:04:47.989 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:47.985+0000 7fc684930040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:04:48.010 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.006+0000 7fbec97ce640 1 Processor -- start 2024-04-25T10:04:48.011 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.006+0000 7fbec97ce640 1 -- start start 2024-04-25T10:04:48.013 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.006+0000 7fbec97ce640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbec415db50 0x7fbec40cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.013 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.006+0000 7fbec97ce640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec40d0200 con 0x7fbec40d12a0 2024-04-25T10:04:48.013 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.006+0000 7fbec97ce640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec40d04e0 con 0x7fbec415df30 2024-04-25T10:04:48.013 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.006+0000 7fbec97ce640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec406c020 con 0x7fbec415db50 2024-04-25T10:04:48.013 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.007+0000 7fbec27fc640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fbec415df30 0x7fbec406cba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.186:34540/0 (socket says 172.21.15.186:34540) 2024-04-25T10:04:48.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.007+0000 7fbec27fc640 1 -- 172.21.15.186:0/1682187156 learned_addr learned my addr 172.21.15.186:0/1682187156 (peer_addr_for_me v1:172.21.15.186:0/0) 2024-04-25T10:04:48.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.007+0000 7fbec2ffd640 1 --2- 172.21.15.186:0/1682187156 >> v2:172.21.15.186:3300/0 conn(0x7fbec415db50 0x7fbec40cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.007+0000 7fbec1ffb640 1 -- 172.21.15.186:0/1682187156 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608317245 0 0) 0x7fbec40d04e0 con 0x7fbec415df30 2024-04-25T10:04:48.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.007+0000 7fbec1ffb640 1 -- 172.21.15.186:0/1682187156 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeb0003520 con 0x7fbec415df30 2024-04-25T10:04:48.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.007+0000 7fbec2ffd640 1 -- 172.21.15.186:0/1682187156 >> v1:172.21.15.186:6789/0 conn(0x7fbec415df30 legacy=0x7fbec406cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.007+0000 7fbec2ffd640 1 -- 172.21.15.186:0/1682187156 >> v1:172.21.15.16:6789/0 conn(0x7fbec40d12a0 legacy=0x7fbec4099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.007+0000 7fbec2ffd640 1 -- 172.21.15.186:0/1682187156 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec40d07c0 con 0x7fbec415db50 2024-04-25T10:04:48.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.008+0000 7fbec2ffd640 1 --2- 172.21.15.186:0/1682187156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbec415db50 0x7fbec40cfb80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0097a0 tx=0x7fbeac02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f05a3011a0ba6e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:48.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.008+0000 7fbec1ffb640 1 -- 172.21.15.186:0/1682187156 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac037070 con 0x7fbec415db50 2024-04-25T10:04:48.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.008+0000 7fbec1ffb640 1 -- 172.21.15.186:0/1682187156 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeac032440 con 0x7fbec415db50 2024-04-25T10:04:48.014 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.008+0000 7fbec1ffb640 1 -- 172.21.15.186:0/1682187156 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac032810 con 0x7fbec415db50 2024-04-25T10:04:48.015 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.008+0000 7fbec97ce640 1 -- 172.21.15.186:0/1682187156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbec415db50 msgr2=0x7fbec40cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.015 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.008+0000 7fbec97ce640 1 --2- 172.21.15.186:0/1682187156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbec415db50 0x7fbec40cfb80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0097a0 tx=0x7fbeac02a230 comp rx=0 tx=0).stop 2024-04-25T10:04:48.015 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.008+0000 7fbec97ce640 1 -- 172.21.15.186:0/1682187156 shutdown_connections 2024-04-25T10:04:48.015 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.008+0000 7fbec97ce640 1 --2- 172.21.15.186:0/1682187156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbec415db50 0x7fbec40cfb80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.015 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.008+0000 7fbec97ce640 1 -- 172.21.15.186:0/1682187156 >> 172.21.15.186:0/1682187156 conn(0x7fbec40b3430 msgr2=0x7fbec40b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:48.015 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.008+0000 7fbec97ce640 1 -- 172.21.15.186:0/1682187156 shutdown_connections 2024-04-25T10:04:48.015 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.008+0000 7fbec97ce640 1 -- 172.21.15.186:0/1682187156 wait complete. 2024-04-25T10:04:48.018 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.010+0000 7fbec97ce640 1 Processor -- start 2024-04-25T10:04:48.018 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.010+0000 7fbec97ce640 1 -- start start 2024-04-25T10:04:48.018 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.010+0000 7fbec97ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec40d12a0 0x7fbec4158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.018 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.010+0000 7fbec97ce640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbec415db50 0x7fbec40c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.018 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.010+0000 7fbec97ce640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbec415df30 0x7fbec41603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.019 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.010+0000 7fbec97ce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec409a150 con 0x7fbec40d12a0 2024-04-25T10:04:48.019 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.010+0000 7fbec97ce640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec40d0d80 con 0x7fbec415db50 2024-04-25T10:04:48.019 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.010+0000 7fbec97ce640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec409a2d0 con 0x7fbec415df30 2024-04-25T10:04:48.019 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbec27fc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbec415db50 0x7fbec40c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.019 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbec27fc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbec415db50 0x7fbec40c7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58770/0 (socket says 172.21.15.186:58770) 2024-04-25T10:04:48.019 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbec27fc640 1 -- 172.21.15.186:0/3152709350 learned_addr learned my addr 172.21.15.186:0/3152709350 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T10:04:48.019 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbec37fe640 1 --2- 172.21.15.186:0/3152709350 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbec415df30 0x7fbec41603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.019 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbec2ffd640 1 --2- 172.21.15.186:0/3152709350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec40d12a0 0x7fbec4158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.019 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbec37fe640 1 -- 172.21.15.186:0/3152709350 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbec415df30 msgr2=0x7fbec41603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-25T10:04:48.019 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbec37fe640 1 -- 172.21.15.186:0/3152709350 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbec415df30 msgr2=0x7fbec41603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:04:48.019 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbec37fe640 1 --2- 172.21.15.186:0/3152709350 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbec415df30 0x7fbec41603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:48.019 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbec37fe640 1 --2- 172.21.15.186:0/3152709350 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbec415df30 0x7fbec41603e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:48.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbec27fc640 1 -- 172.21.15.186:0/3152709350 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbec415df30 msgr2=0x7fbec41603e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:04:48.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbec27fc640 1 --2- 172.21.15.186:0/3152709350 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbec415df30 0x7fbec41603e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbec27fc640 1 -- 172.21.15.186:0/3152709350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec40d12a0 msgr2=0x7fbec4158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbec27fc640 1 --2- 172.21.15.186:0/3152709350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec40d12a0 0x7fbec4158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbec27fc640 1 -- 172.21.15.186:0/3152709350 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec4158d70 con 0x7fbec415db50 2024-04-25T10:04:48.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbec27fc640 1 --2- 172.21.15.186:0/3152709350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbec415db50 0x7fbec40c7cd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400c410 tx=0x7fbeb400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:48.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbebb7fe640 1 -- 172.21.15.186:0/3152709350 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4017070 con 0x7fbec415db50 2024-04-25T10:04:48.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbebb7fe640 1 -- 172.21.15.186:0/3152709350 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb4005ce0 con 0x7fbec415db50 2024-04-25T10:04:48.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbec97ce640 1 -- 172.21.15.186:0/3152709350 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec41f4410 con 0x7fbec415db50 2024-04-25T10:04:48.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbebb7fe640 1 -- 172.21.15.186:0/3152709350 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb40072d0 con 0x7fbec415db50 2024-04-25T10:04:48.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbec97ce640 1 -- 172.21.15.186:0/3152709350 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec40b6400 con 0x7fbec415db50 2024-04-25T10:04:48.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbebb7fe640 1 -- 172.21.15.186:0/3152709350 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbeb401a070 con 0x7fbec415db50 2024-04-25T10:04:48.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.011+0000 7fbebb7fe640 1 -- 172.21.15.186:0/3152709350 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fbeb401cc10 con 0x7fbec415db50 2024-04-25T10:04:48.021 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.012+0000 7fbec97ce640 1 -- 172.21.15.186:0/3152709350 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec40750b0 con 0x7fbec415db50 2024-04-25T10:04:48.021 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.016+0000 7fbebb7fe640 1 -- 172.21.15.186:0/3152709350 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbec4158d70 con 0x7fbec415db50 2024-04-25T10:04:48.093 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.088+0000 7fbec97ce640 1 -- 172.21.15.186:0/3152709350 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd new", "uuid": "46edf17f-ad1d-404a-b94a-9ed0cc9a7226", "id": 4} v 0) v1 -- 0x7fbec409a980 con 0x7fbec415db50 2024-04-25T10:04:48.096 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.092+0000 7fbebb7fe640 1 -- 172.21.15.186:0/3152709350 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "46edf17f-ad1d-404a-b94a-9ed0cc9a7226", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbeb4007470 con 0x7fbec415db50 2024-04-25T10:04:48.097 INFO:teuthology.orchestra.run.smithi186.stdout:4 2024-04-25T10:04:48.098 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.093+0000 7fbec97ce640 1 -- 172.21.15.186:0/3152709350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbec415db50 msgr2=0x7fbec40c7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.098 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.093+0000 7fbec97ce640 1 --2- 172.21.15.186:0/3152709350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbec415db50 0x7fbec40c7cd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400c410 tx=0x7fbeb400c8d0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.098 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.093+0000 7fbec97ce640 1 -- 172.21.15.186:0/3152709350 shutdown_connections 2024-04-25T10:04:48.098 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.093+0000 7fbec97ce640 1 --2- 172.21.15.186:0/3152709350 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbec415df30 0x7fbec41603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.099 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.093+0000 7fbec97ce640 1 --2- 172.21.15.186:0/3152709350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbec415db50 0x7fbec40c7cd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.099 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.093+0000 7fbec97ce640 1 --2- 172.21.15.186:0/3152709350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec40d12a0 0x7fbec4158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.099 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.093+0000 7fbec97ce640 1 -- 172.21.15.186:0/3152709350 >> 172.21.15.186:0/3152709350 conn(0x7fbec40b3430 msgr2=0x7fbec40ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:48.099 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.093+0000 7fbec97ce640 1 -- 172.21.15.186:0/3152709350 shutdown_connections 2024-04-25T10:04:48.099 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.093+0000 7fbec97ce640 1 -- 172.21.15.186:0/3152709350 wait complete. 2024-04-25T10:04:48.108 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new f4824f88-e2a1-4c26-8198-1bc0cedaa1c4 5 2024-04-25T10:04:48.170 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:48.165+0000 7fc470923040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:04:48.206 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.201+0000 7fdf1eab3640 1 Processor -- start 2024-04-25T10:04:48.206 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.201+0000 7fdf1eab3640 1 -- start start 2024-04-25T10:04:48.208 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.202+0000 7fdf1eab3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdf180ad000 0x7fdf180af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.208 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.202+0000 7fdf1eab3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf18144bf0 0x7fdf18141930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.208 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.202+0000 7fdf1eab3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf180af720 con 0x7fdf1813f140 2024-04-25T10:04:48.208 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.202+0000 7fdf1eab3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf18160010 con 0x7fdf180ad000 2024-04-25T10:04:48.208 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.202+0000 7fdf1eab3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf181602f0 con 0x7fdf18144bf0 2024-04-25T10:04:48.209 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.202+0000 7fdf1d050640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf18144bf0 0x7fdf18141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.209 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.202+0000 7fdf1d050640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf18144bf0 0x7fdf18141930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:45334/0 (socket says 172.21.15.186:45334) 2024-04-25T10:04:48.209 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.202+0000 7fdf1d050640 1 -- 172.21.15.186:0/3912734114 learned_addr learned my addr 172.21.15.186:0/3912734114 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T10:04:48.209 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.202+0000 7fdf1d050640 1 -- 172.21.15.186:0/3912734114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf18144bf0 msgr2=0x7fdf18141930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:04:48.209 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.202+0000 7fdf1d050640 1 -- 172.21.15.186:0/3912734114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf18144bf0 msgr2=0x7fdf18141930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:48.209 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.202+0000 7fdf1d050640 1 --2- 172.21.15.186:0/3912734114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf18144bf0 0x7fdf18141930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:48.209 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.202+0000 7fdf1c84f640 1 --2- 172.21.15.186:0/3912734114 >> v2:172.21.15.16:3300/0 conn(0x7fdf180ad000 0x7fdf180af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.209 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.203+0000 7fdf1d050640 1 --2- 172.21.15.186:0/3912734114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf18144bf0 0x7fdf18141930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:48.209 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.203+0000 7fdf1c84f640 1 -- 172.21.15.186:0/3912734114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf18144bf0 msgr2=0x7fdf18141930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:04:48.209 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.203+0000 7fdf1c84f640 1 --2- 172.21.15.186:0/3912734114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf18144bf0 0x7fdf18141930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.209 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.203+0000 7fdf1c84f640 1 -- 172.21.15.186:0/3912734114 >> v1:172.21.15.16:6789/0 conn(0x7fdf1813f140 legacy=0x7fdf180ad3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.209 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.203+0000 7fdf1c84f640 1 -- 172.21.15.186:0/3912734114 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1806ac40 con 0x7fdf180ad000 2024-04-25T10:04:48.209 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.203+0000 7fdf1c84f640 1 --2- 172.21.15.186:0/3912734114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf180ad000 0x7fdf180af1e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0097a0 tx=0x7fdf0c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8409578ef37cd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:48.209 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.204+0000 7fdf177fe640 1 -- 172.21.15.186:0/3912734114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c037070 con 0x7fdf180ad000 2024-04-25T10:04:48.210 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.204+0000 7fdf177fe640 1 -- 172.21.15.186:0/3912734114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0c032440 con 0x7fdf180ad000 2024-04-25T10:04:48.210 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.204+0000 7fdf177fe640 1 -- 172.21.15.186:0/3912734114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c032810 con 0x7fdf180ad000 2024-04-25T10:04:48.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.204+0000 7fdf1eab3640 1 -- 172.21.15.186:0/3912734114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf180ad000 msgr2=0x7fdf180af1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.204+0000 7fdf1eab3640 1 --2- 172.21.15.186:0/3912734114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf180ad000 0x7fdf180af1e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0097a0 tx=0x7fdf0c02a180 comp rx=0 tx=0).stop 2024-04-25T10:04:48.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.204+0000 7fdf1eab3640 1 -- 172.21.15.186:0/3912734114 shutdown_connections 2024-04-25T10:04:48.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.204+0000 7fdf1eab3640 1 --2- 172.21.15.186:0/3912734114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf18144bf0 0x7fdf18141930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.204+0000 7fdf1eab3640 1 --2- 172.21.15.186:0/3912734114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf180ad000 0x7fdf180af1e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.204+0000 7fdf1eab3640 1 -- 172.21.15.186:0/3912734114 >> 172.21.15.186:0/3912734114 conn(0x7fdf180cff20 msgr2=0x7fdf180d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:48.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.204+0000 7fdf1eab3640 1 -- 172.21.15.186:0/3912734114 shutdown_connections 2024-04-25T10:04:48.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.204+0000 7fdf1eab3640 1 -- 172.21.15.186:0/3912734114 wait complete. 2024-04-25T10:04:48.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.205+0000 7fdf1eab3640 1 Processor -- start 2024-04-25T10:04:48.213 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.206+0000 7fdf1eab3640 1 -- start start 2024-04-25T10:04:48.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.206+0000 7fdf1eab3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf180ad000 0x7fdf1815cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.206+0000 7fdf1eab3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf1813f140 0x7fdf181e5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.206+0000 7fdf1eab3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf18144bf0 0x7fdf1815e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.206+0000 7fdf1eab3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf18141e70 con 0x7fdf1813f140 2024-04-25T10:04:48.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.206+0000 7fdf1eab3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1806b200 con 0x7fdf18144bf0 2024-04-25T10:04:48.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.206+0000 7fdf1eab3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf18141ff0 con 0x7fdf180ad000 2024-04-25T10:04:48.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.206+0000 7fdf1c84f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf180ad000 0x7fdf1815cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf1c84f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf180ad000 0x7fdf1815cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:45346/0 (socket says 172.21.15.186:45346) 2024-04-25T10:04:48.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf1c84f640 1 -- 172.21.15.186:0/1825461146 learned_addr learned my addr 172.21.15.186:0/1825461146 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T10:04:48.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf1d050640 1 --2- 172.21.15.186:0/1825461146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf18144bf0 0x7fdf1815e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf1c84f640 1 -- 172.21.15.186:0/1825461146 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf180ad000 msgr2=0x7fdf1815cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-25T10:04:48.214 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf1c84f640 1 -- 172.21.15.186:0/1825461146 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf180ad000 msgr2=0x7fdf1815cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:04:48.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf1c84f640 1 --2- 172.21.15.186:0/1825461146 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf180ad000 0x7fdf1815cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:48.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf1c84f640 1 --2- 172.21.15.186:0/1825461146 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf180ad000 0x7fdf1815cc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:48.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf1d050640 1 -- 172.21.15.186:0/1825461146 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf180ad000 msgr2=0x7fdf1815cc40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:04:48.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf1d050640 1 --2- 172.21.15.186:0/1825461146 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf180ad000 0x7fdf1815cc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf1d050640 1 -- 172.21.15.186:0/1825461146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf1813f140 msgr2=0x7fdf181e5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf1d050640 1 --2- 172.21.15.186:0/1825461146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf1813f140 0x7fdf181e5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf1d050640 1 -- 172.21.15.186:0/1825461146 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf181e5990 con 0x7fdf18144bf0 2024-04-25T10:04:48.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf1d050640 1 --2- 172.21.15.186:0/1825461146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf18144bf0 0x7fdf1815e3a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009fd0 tx=0x7fdf10007ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:48.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf157fa640 1 -- 172.21.15.186:0/1825461146 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1001a070 con 0x7fdf18144bf0 2024-04-25T10:04:48.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf157fa640 1 -- 172.21.15.186:0/1825461146 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1000dbe0 con 0x7fdf18144bf0 2024-04-25T10:04:48.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf1eab3640 1 -- 172.21.15.186:0/1825461146 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1815d300 con 0x7fdf18144bf0 2024-04-25T10:04:48.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf157fa640 1 -- 172.21.15.186:0/1825461146 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf10015400 con 0x7fdf18144bf0 2024-04-25T10:04:48.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf1eab3640 1 -- 172.21.15.186:0/1825461146 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1815d8c0 con 0x7fdf18144bf0 2024-04-25T10:04:48.215 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf157fa640 1 -- 172.21.15.186:0/1825461146 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fdf1001d070 con 0x7fdf18144bf0 2024-04-25T10:04:48.216 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.207+0000 7fdf157fa640 1 -- 172.21.15.186:0/1825461146 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fdf1001e930 con 0x7fdf18144bf0 2024-04-25T10:04:48.216 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.208+0000 7fdf1eab3640 1 -- 172.21.15.186:0/1825461146 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf180750b0 con 0x7fdf18144bf0 2024-04-25T10:04:48.216 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.212+0000 7fdf157fa640 1 -- 172.21.15.186:0/1825461146 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fdf10024020 con 0x7fdf18144bf0 2024-04-25T10:04:48.226 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:48.222+0000 7fc470923040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:04:48.261 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:48.256+0000 7fc684930040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:04:48.289 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.284+0000 7fdf1eab3640 1 -- 172.21.15.186:0/1825461146 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f4824f88-e2a1-4c26-8198-1bc0cedaa1c4", "id": 5} v 0) v1 -- 0x7fdf181443b0 con 0x7fdf18144bf0 2024-04-25T10:04:48.292 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.288+0000 7fdf157fa640 1 -- 172.21.15.186:0/1825461146 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f4824f88-e2a1-4c26-8198-1bc0cedaa1c4", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdf10010070 con 0x7fdf18144bf0 2024-04-25T10:04:48.293 INFO:teuthology.orchestra.run.smithi186.stdout:5 2024-04-25T10:04:48.294 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.289+0000 7fdf1eab3640 1 -- 172.21.15.186:0/1825461146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf18144bf0 msgr2=0x7fdf1815e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.294 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.289+0000 7fdf1eab3640 1 --2- 172.21.15.186:0/1825461146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf18144bf0 0x7fdf1815e3a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009fd0 tx=0x7fdf10007ef0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.294 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.289+0000 7fdf1eab3640 1 -- 172.21.15.186:0/1825461146 shutdown_connections 2024-04-25T10:04:48.294 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.289+0000 7fdf1eab3640 1 --2- 172.21.15.186:0/1825461146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf18144bf0 0x7fdf1815e3a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.294 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.289+0000 7fdf1eab3640 1 --2- 172.21.15.186:0/1825461146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf1813f140 0x7fdf181e5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.294 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.289+0000 7fdf1eab3640 1 --2- 172.21.15.186:0/1825461146 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf180ad000 0x7fdf1815cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.294 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.289+0000 7fdf1eab3640 1 -- 172.21.15.186:0/1825461146 >> 172.21.15.186:0/1825461146 conn(0x7fdf180cff20 msgr2=0x7fdf180b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:48.294 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.289+0000 7fdf1eab3640 1 -- 172.21.15.186:0/1825461146 shutdown_connections 2024-04-25T10:04:48.295 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.289+0000 7fdf1eab3640 1 -- 172.21.15.186:0/1825461146 wait complete. 2024-04-25T10:04:48.304 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e 6 2024-04-25T10:04:48.316 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:48.311+0000 7fc684930040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:04:48.349 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:48.344+0000 7fc470923040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:04:48.401 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.397+0000 7ff89ceb6640 1 Processor -- start 2024-04-25T10:04:48.401 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.397+0000 7ff89ceb6640 1 -- start start 2024-04-25T10:04:48.403 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff89ceb6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff898165770 0x7ff89806be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.403 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff89ceb6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff89806c3a0 0x7ff8980c4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.404 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff89ceb6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89809a4e0 0x7ff8980c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.404 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff89ceb6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8980cbf10 con 0x7ff89806c3a0 2024-04-25T10:04:48.404 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff89ceb6640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff89809a9f0 con 0x7ff898165770 2024-04-25T10:04:48.404 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff89ceb6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89809acd0 con 0x7ff89809a4e0 2024-04-25T10:04:48.404 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff89659c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff898165770 0x7ff89806be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.404 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff896d9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89809a4e0 0x7ff8980c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.404 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff89659c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff898165770 0x7ff89806be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58792/0 (socket says 172.21.15.186:58792) 2024-04-25T10:04:48.404 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff896d9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89809a4e0 0x7ff8980c4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:45352/0 (socket says 172.21.15.186:45352) 2024-04-25T10:04:48.404 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff89659c640 1 -- 172.21.15.186:0/93206544 learned_addr learned my addr 172.21.15.186:0/93206544 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T10:04:48.404 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff895d9b640 1 --2- 172.21.15.186:0/93206544 >> v2:172.21.15.16:3300/0 conn(0x7ff89806c3a0 0x7ff8980c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.404 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff896d9d640 1 -- 172.21.15.186:0/93206544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89809a4e0 msgr2=0x7ff8980c4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:04:48.404 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff896d9d640 1 -- 172.21.15.186:0/93206544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89809a4e0 msgr2=0x7ff8980c4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:48.404 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff896d9d640 1 --2- 172.21.15.186:0/93206544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89809a4e0 0x7ff8980c4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:48.404 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff896d9d640 1 --2- 172.21.15.186:0/93206544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89809a4e0 0x7ff8980c4dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:48.405 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff89659c640 1 -- 172.21.15.186:0/93206544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89809a4e0 msgr2=0x7ff8980c4dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:04:48.405 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff89659c640 1 --2- 172.21.15.186:0/93206544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89809a4e0 0x7ff8980c4dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.405 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff89659c640 1 -- 172.21.15.186:0/93206544 >> v2:172.21.15.16:3300/0 conn(0x7ff89806c3a0 msgr2=0x7ff8980c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:48.405 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff89659c640 1 --2- 172.21.15.186:0/93206544 >> v2:172.21.15.16:3300/0 conn(0x7ff89806c3a0 0x7ff8980c4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.405 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.398+0000 7ff89659c640 1 -- 172.21.15.186:0/93206544 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8980cc160 con 0x7ff898165770 2024-04-25T10:04:48.405 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.399+0000 7ff89659c640 1 --2- 172.21.15.186:0/93206544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff898165770 0x7ff89806be60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff8980c53c0 tx=0x7ff88802a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f463d8063c6b9233 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:48.405 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.399+0000 7ff89559a640 1 -- 172.21.15.186:0/93206544 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888037070 con 0x7ff898165770 2024-04-25T10:04:48.405 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.399+0000 7ff89559a640 1 -- 172.21.15.186:0/93206544 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88802ae00 con 0x7ff898165770 2024-04-25T10:04:48.405 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.399+0000 7ff89559a640 1 -- 172.21.15.186:0/93206544 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888032660 con 0x7ff898165770 2024-04-25T10:04:48.408 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.400+0000 7ff89ceb6640 1 -- 172.21.15.186:0/93206544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff898165770 msgr2=0x7ff89806be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.409 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.400+0000 7ff89ceb6640 1 --2- 172.21.15.186:0/93206544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff898165770 0x7ff89806be60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff8980c53c0 tx=0x7ff88802a0b0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.409 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.400+0000 7ff89ceb6640 1 -- 172.21.15.186:0/93206544 shutdown_connections 2024-04-25T10:04:48.409 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.400+0000 7ff89ceb6640 1 --2- 172.21.15.186:0/93206544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89809a4e0 0x7ff8980c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.409 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.400+0000 7ff89ceb6640 1 --2- 172.21.15.186:0/93206544 >> v2:172.21.15.16:3300/0 conn(0x7ff89806c3a0 0x7ff8980c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.409 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.400+0000 7ff89ceb6640 1 --2- 172.21.15.186:0/93206544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff898165770 0x7ff89806be60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.409 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.400+0000 7ff89ceb6640 1 -- 172.21.15.186:0/93206544 >> 172.21.15.186:0/93206544 conn(0x7ff8980b33b0 msgr2=0x7ff8980b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:48.409 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.400+0000 7ff89ceb6640 1 -- 172.21.15.186:0/93206544 shutdown_connections 2024-04-25T10:04:48.409 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.400+0000 7ff89ceb6640 1 -- 172.21.15.186:0/93206544 wait complete. 2024-04-25T10:04:48.409 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.401+0000 7ff89ceb6640 1 Processor -- start 2024-04-25T10:04:48.409 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.401+0000 7ff89ceb6640 1 -- start start 2024-04-25T10:04:48.409 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff89ceb6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89806c3a0 0x7ff89809e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.409 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff89ceb6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89809a4e0 0x7ff898163090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.409 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff89ceb6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff898165770 0x7ff89809c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.410 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff89ceb6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff898165ff0 con 0x7ff89809a4e0 2024-04-25T10:04:48.410 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff89ceb6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff898165e70 con 0x7ff898165770 2024-04-25T10:04:48.410 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff89ceb6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff898166170 con 0x7ff89806c3a0 2024-04-25T10:04:48.410 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff89659c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89806c3a0 0x7ff89809e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.410 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff89659c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89806c3a0 0x7ff89809e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:45356/0 (socket says 172.21.15.186:45356) 2024-04-25T10:04:48.410 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff89659c640 1 -- 172.21.15.186:0/3055544023 learned_addr learned my addr 172.21.15.186:0/3055544023 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T10:04:48.410 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff896d9d640 1 --2- 172.21.15.186:0/3055544023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff898165770 0x7ff89809c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.410 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff895d9b640 1 --2- 172.21.15.186:0/3055544023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89809a4e0 0x7ff898163090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.410 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff89659c640 1 -- 172.21.15.186:0/3055544023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89806c3a0 msgr2=0x7ff89809e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-25T10:04:48.410 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff89659c640 1 -- 172.21.15.186:0/3055544023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89806c3a0 msgr2=0x7ff89809e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:04:48.410 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff89659c640 1 --2- 172.21.15.186:0/3055544023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89806c3a0 0x7ff89809e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:48.410 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff89659c640 1 --2- 172.21.15.186:0/3055544023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89806c3a0 0x7ff89809e700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:48.410 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff895d9b640 1 -- 172.21.15.186:0/3055544023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89806c3a0 msgr2=0x7ff89809e700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:04:48.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff895d9b640 1 --2- 172.21.15.186:0/3055544023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89806c3a0 0x7ff89809e700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff895d9b640 1 -- 172.21.15.186:0/3055544023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff898165770 msgr2=0x7ff89809c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff895d9b640 1 --2- 172.21.15.186:0/3055544023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff898165770 0x7ff89809c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff895d9b640 1 -- 172.21.15.186:0/3055544023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89809dae0 con 0x7ff89809a4e0 2024-04-25T10:04:48.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.402+0000 7ff896d9d640 1 --2- 172.21.15.186:0/3055544023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff898165770 0x7ff89809c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:48.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.403+0000 7ff895d9b640 1 --2- 172.21.15.186:0/3055544023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89809a4e0 0x7ff898163090 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00e870 tx=0x7ff88c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:48.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.403+0000 7ff886ffd640 1 -- 172.21.15.186:0/3055544023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c017070 con 0x7ff89809a4e0 2024-04-25T10:04:48.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.403+0000 7ff886ffd640 1 -- 172.21.15.186:0/3055544023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c00c940 con 0x7ff89809a4e0 2024-04-25T10:04:48.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.403+0000 7ff886ffd640 1 -- 172.21.15.186:0/3055544023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c00ccf0 con 0x7ff89809a4e0 2024-04-25T10:04:48.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.403+0000 7ff89ceb6640 1 -- 172.21.15.186:0/3055544023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8980c81a0 con 0x7ff89809a4e0 2024-04-25T10:04:48.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.403+0000 7ff89ceb6640 1 -- 172.21.15.186:0/3055544023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff89809d2e0 con 0x7ff89809a4e0 2024-04-25T10:04:48.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.403+0000 7ff886ffd640 1 -- 172.21.15.186:0/3055544023 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff88c01a070 con 0x7ff89809a4e0 2024-04-25T10:04:48.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.403+0000 7ff886ffd640 1 -- 172.21.15.186:0/3055544023 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7ff88c006570 con 0x7ff89809a4e0 2024-04-25T10:04:48.411 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.404+0000 7ff89ceb6640 1 -- 172.21.15.186:0/3055544023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8980750b0 con 0x7ff89809a4e0 2024-04-25T10:04:48.412 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.408+0000 7ff886ffd640 1 -- 172.21.15.186:0/3055544023 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff88c006850 con 0x7ff89809a4e0 2024-04-25T10:04:48.438 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:48.434+0000 7fc684930040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:04:48.486 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.481+0000 7ff89ceb6640 1 -- 172.21.15.186:0/3055544023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e", "id": 6} v 0) v1 -- 0x7ff898070eb0 con 0x7ff89809a4e0 2024-04-25T10:04:48.487 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.483+0000 7ff886ffd640 1 -- 172.21.15.186:0/3055544023 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff88c005a60 con 0x7ff89809a4e0 2024-04-25T10:04:48.488 INFO:teuthology.orchestra.run.smithi186.stdout:6 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.484+0000 7ff89ceb6640 1 -- 172.21.15.186:0/3055544023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89809a4e0 msgr2=0x7ff898163090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.484+0000 7ff89ceb6640 1 --2- 172.21.15.186:0/3055544023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89809a4e0 0x7ff898163090 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00e870 tx=0x7ff88c00ed30 comp rx=0 tx=0).stop 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.484+0000 7ff89ceb6640 1 -- 172.21.15.186:0/3055544023 shutdown_connections 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.484+0000 7ff89ceb6640 1 --2- 172.21.15.186:0/3055544023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff898165770 0x7ff89809c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.484+0000 7ff89ceb6640 1 --2- 172.21.15.186:0/3055544023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff89809a4e0 0x7ff898163090 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.484+0000 7ff89ceb6640 1 --2- 172.21.15.186:0/3055544023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89806c3a0 0x7ff89809e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.490 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.484+0000 7ff89ceb6640 1 -- 172.21.15.186:0/3055544023 >> 172.21.15.186:0/3055544023 conn(0x7ff8980b33b0 msgr2=0x7ff8980b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:48.490 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.484+0000 7ff89ceb6640 1 -- 172.21.15.186:0/3055544023 shutdown_connections 2024-04-25T10:04:48.490 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.484+0000 7ff89ceb6640 1 -- 172.21.15.186:0/3055544023 wait complete. 2024-04-25T10:04:48.499 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new cb71fc85-240b-40b2-95cb-e39b8570f4bb 7 2024-04-25T10:04:48.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.590+0000 7f1b09edd640 1 Processor -- start 2024-04-25T10:04:48.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.590+0000 7f1b09edd640 1 -- start start 2024-04-25T10:04:48.597 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.591+0000 7f1b09edd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 0x7f1b040d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.597 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.591+0000 7f1b09edd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1b0415e150 0x7f1b0406aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.597 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.591+0000 7f1b09edd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b040d0390 con 0x7f1b0415f1b0 2024-04-25T10:04:48.597 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.591+0000 7f1b09edd640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0415e6d0 con 0x7f1b0415e150 2024-04-25T10:04:48.597 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.591+0000 7f1b09edd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0415e9b0 con 0x7f1b0415dd70 2024-04-25T10:04:48.597 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.591+0000 7f1b03fff640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1b0415e150 0x7f1b0406aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.591+0000 7f1b037fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 0x7f1b040d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.591+0000 7f1b03fff640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1b0415e150 0x7f1b0406aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58802/0 (socket says 172.21.15.186:58802) 2024-04-25T10:04:48.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.591+0000 7f1b03fff640 1 -- 172.21.15.186:0/2018172240 learned_addr learned my addr 172.21.15.186:0/2018172240 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T10:04:48.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.592+0000 7f1b037fe640 1 -- 172.21.15.186:0/2018172240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 msgr2=0x7f1b040d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:04:48.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.592+0000 7f1b037fe640 1 -- 172.21.15.186:0/2018172240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 msgr2=0x7f1b040d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:48.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.592+0000 7f1b037fe640 1 --2- 172.21.15.186:0/2018172240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 0x7f1b040d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:48.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.592+0000 7f1b037fe640 1 --2- 172.21.15.186:0/2018172240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 0x7f1b040d1300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:48.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.592+0000 7f1b03fff640 1 -- 172.21.15.186:0/2018172240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 msgr2=0x7f1b040d1300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:04:48.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.592+0000 7f1b03fff640 1 --2- 172.21.15.186:0/2018172240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 0x7f1b040d1300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.592+0000 7f1b03fff640 1 -- 172.21.15.186:0/2018172240 >> v1:172.21.15.16:6789/0 conn(0x7f1b0415f1b0 legacy=0x7f1b040cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.592+0000 7f1b03fff640 1 -- 172.21.15.186:0/2018172240 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0415ec90 con 0x7f1b0415e150 2024-04-25T10:04:48.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.592+0000 7f1b03fff640 1 --2- 172.21.15.186:0/2018172240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b0415e150 0x7f1b0406aba0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1afc009e50 tx=0x7f1afc02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71423e3f2301f7a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:48.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.592+0000 7f1b02ffd640 1 -- 172.21.15.186:0/2018172240 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc036070 con 0x7f1b0415e150 2024-04-25T10:04:48.599 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.592+0000 7f1b02ffd640 1 -- 172.21.15.186:0/2018172240 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1afc031600 con 0x7f1b0415e150 2024-04-25T10:04:48.599 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.593+0000 7f1b02ffd640 1 -- 172.21.15.186:0/2018172240 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc0319d0 con 0x7f1b0415e150 2024-04-25T10:04:48.602 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.593+0000 7f1b09edd640 1 -- 172.21.15.186:0/2018172240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b0415e150 msgr2=0x7f1b0406aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.602 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.593+0000 7f1b09edd640 1 --2- 172.21.15.186:0/2018172240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b0415e150 0x7f1b0406aba0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1afc009e50 tx=0x7f1afc02a680 comp rx=0 tx=0).stop 2024-04-25T10:04:48.602 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.593+0000 7f1b09edd640 1 -- 172.21.15.186:0/2018172240 shutdown_connections 2024-04-25T10:04:48.602 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.593+0000 7f1b09edd640 1 --2- 172.21.15.186:0/2018172240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b0415e150 0x7f1b0406aba0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.602 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.593+0000 7f1b09edd640 1 --2- 172.21.15.186:0/2018172240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 0x7f1b040d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.602 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.593+0000 7f1b09edd640 1 -- 172.21.15.186:0/2018172240 >> 172.21.15.186:0/2018172240 conn(0x7f1b04099a40 msgr2=0x7f1b040d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:48.602 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.593+0000 7f1b09edd640 1 -- 172.21.15.186:0/2018172240 shutdown_connections 2024-04-25T10:04:48.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.593+0000 7f1b09edd640 1 -- 172.21.15.186:0/2018172240 wait complete. 2024-04-25T10:04:48.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.594+0000 7f1b09edd640 1 Processor -- start 2024-04-25T10:04:48.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.594+0000 7f1b09edd640 1 -- start start 2024-04-25T10:04:48.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b09edd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 0x7f1b040ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b09edd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b0415e150 0x7f1b040c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b09edd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b0415f1b0 0x7f1b040d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b09edd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0406b470 con 0x7f1b0415e150 2024-04-25T10:04:48.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b09edd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0406b2f0 con 0x7f1b0415f1b0 2024-04-25T10:04:48.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b09edd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b04161a00 con 0x7f1b0415dd70 2024-04-25T10:04:48.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b03fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b0415f1b0 0x7f1b040d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b037fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 0x7f1b040ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b03fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b0415f1b0 0x7f1b040d07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58804/0 (socket says 172.21.15.186:58804) 2024-04-25T10:04:48.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b037fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 0x7f1b040ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:45370/0 (socket says 172.21.15.186:45370) 2024-04-25T10:04:48.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b03fff640 1 -- 172.21.15.186:0/727166704 learned_addr learned my addr 172.21.15.186:0/727166704 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-25T10:04:48.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1afbfff640 1 --2- 172.21.15.186:0/727166704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b0415e150 0x7f1b040c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b037fe640 1 -- 172.21.15.186:0/727166704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 msgr2=0x7f1b040ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-25T10:04:48.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b037fe640 1 -- 172.21.15.186:0/727166704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 msgr2=0x7f1b040ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:04:48.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b037fe640 1 --2- 172.21.15.186:0/727166704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 0x7f1b040ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:48.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b037fe640 1 --2- 172.21.15.186:0/727166704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 0x7f1b040ca650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:48.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b03fff640 1 -- 172.21.15.186:0/727166704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 msgr2=0x7f1b040ca650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:04:48.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b03fff640 1 --2- 172.21.15.186:0/727166704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 0x7f1b040ca650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b03fff640 1 -- 172.21.15.186:0/727166704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b0415e150 msgr2=0x7f1b040c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b03fff640 1 --2- 172.21.15.186:0/727166704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b0415e150 0x7f1b040c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.595+0000 7f1b03fff640 1 -- 172.21.15.186:0/727166704 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b040c91c0 con 0x7f1b0415f1b0 2024-04-25T10:04:48.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.596+0000 7f1b03fff640 1 --2- 172.21.15.186:0/727166704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b0415f1b0 0x7f1b040d07e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1afc009e50 tx=0x7f1afc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:48.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.596+0000 7f1b00ff9640 1 -- 172.21.15.186:0/727166704 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc036050 con 0x7f1b0415f1b0 2024-04-25T10:04:48.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.596+0000 7f1b00ff9640 1 -- 172.21.15.186:0/727166704 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1afc042070 con 0x7f1b0415f1b0 2024-04-25T10:04:48.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.596+0000 7f1b09edd640 1 -- 172.21.15.186:0/727166704 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b041f7390 con 0x7f1b0415f1b0 2024-04-25T10:04:48.605 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.596+0000 7f1b00ff9640 1 -- 172.21.15.186:0/727166704 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc009510 con 0x7f1b0415f1b0 2024-04-25T10:04:48.605 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.596+0000 7f1b09edd640 1 -- 172.21.15.186:0/727166704 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b041f7650 con 0x7f1b0415f1b0 2024-04-25T10:04:48.605 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.596+0000 7f1b00ff9640 1 -- 172.21.15.186:0/727166704 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1afc007bd0 con 0x7f1b0415f1b0 2024-04-25T10:04:48.605 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.596+0000 7f1b00ff9640 1 -- 172.21.15.186:0/727166704 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f1afc047af0 con 0x7f1b0415f1b0 2024-04-25T10:04:48.605 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.597+0000 7f1b09edd640 1 -- 172.21.15.186:0/727166704 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b040750b0 con 0x7f1b0415f1b0 2024-04-25T10:04:48.605 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.599+0000 7f1b00ff9640 1 -- 172.21.15.186:0/727166704 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1afc047460 con 0x7f1b0415f1b0 2024-04-25T10:04:48.642 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:48.637+0000 7fc470923040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:04:48.676 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.671+0000 7f1b09edd640 1 -- 172.21.15.186:0/727166704 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cb71fc85-240b-40b2-95cb-e39b8570f4bb", "id": 7} v 0) v1 -- 0x7f1b040b6780 con 0x7f1b0415f1b0 2024-04-25T10:04:48.680 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.676+0000 7f1b00ff9640 1 -- 172.21.15.186:0/727166704 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cb71fc85-240b-40b2-95cb-e39b8570f4bb", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1afc047640 con 0x7f1b0415f1b0 2024-04-25T10:04:48.680 INFO:teuthology.orchestra.run.smithi186.stdout:7 2024-04-25T10:04:48.682 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.677+0000 7f1b09edd640 1 -- 172.21.15.186:0/727166704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b0415f1b0 msgr2=0x7f1b040d07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.682 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.677+0000 7f1b09edd640 1 --2- 172.21.15.186:0/727166704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b0415f1b0 0x7f1b040d07e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1afc009e50 tx=0x7f1afc004a40 comp rx=0 tx=0).stop 2024-04-25T10:04:48.682 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.677+0000 7f1b09edd640 1 -- 172.21.15.186:0/727166704 shutdown_connections 2024-04-25T10:04:48.682 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.677+0000 7f1b09edd640 1 --2- 172.21.15.186:0/727166704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b0415f1b0 0x7f1b040d07e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.682 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.677+0000 7f1b09edd640 1 --2- 172.21.15.186:0/727166704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b0415e150 0x7f1b040c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.682 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.677+0000 7f1b09edd640 1 --2- 172.21.15.186:0/727166704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0415dd70 0x7f1b040ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.682 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.677+0000 7f1b09edd640 1 -- 172.21.15.186:0/727166704 >> 172.21.15.186:0/727166704 conn(0x7f1b04099a40 msgr2=0x7f1b040cbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:48.682 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.677+0000 7f1b09edd640 1 -- 172.21.15.186:0/727166704 shutdown_connections 2024-04-25T10:04:48.682 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-25T10:04:48.677+0000 7f1b09edd640 1 -- 172.21.15.186:0/727166704 wait complete. 2024-04-25T10:04:48.692 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T10:04:48.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T10:04:48.693 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:48.687+0000 7fc470923040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:04:48.694 INFO:tasks.ceph.osd.0:Started 2024-04-25T10:04:48.695 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T10:04:48.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T10:04:48.697 INFO:tasks.ceph.osd.1:Started 2024-04-25T10:04:48.697 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T10:04:48.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T10:04:48.699 INFO:tasks.ceph.osd.2:Started 2024-04-25T10:04:48.699 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T10:04:48.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T10:04:48.701 INFO:tasks.ceph.osd.3:Started 2024-04-25T10:04:48.701 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T10:04:48.701 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T10:04:48.703 INFO:tasks.ceph.osd.4:Started 2024-04-25T10:04:48.703 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T10:04:48.704 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T10:04:48.705 INFO:tasks.ceph.osd.5:Started 2024-04-25T10:04:48.706 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T10:04:48.706 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T10:04:48.708 INFO:tasks.ceph.osd.6:Started 2024-04-25T10:04:48.708 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T10:04:48.708 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T10:04:48.710 INFO:tasks.ceph.osd.7:Started 2024-04-25T10:04:48.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:48.747 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:48.742+0000 7fc684930040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:04:48.834 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:48.829+0000 7fc684930040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:04:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.853+0000 7fe382dd5640 1 Processor -- start 2024-04-25T10:04:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.853+0000 7fe382dd5640 1 -- start start 2024-04-25T10:04:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.853+0000 7fe382dd5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe37c06cdd0 0x7fe37c0b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.853+0000 7fe382dd5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe37c0b1960 0x7fe37c0d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.853+0000 7fe382dd5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37c06c1f0 con 0x7fe37c15dc30 2024-04-25T10:04:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.853+0000 7fe382dd5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe37c0b1e10 con 0x7fe37c06cdd0 2024-04-25T10:04:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.853+0000 7fe382dd5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe37c0cfb40 con 0x7fe37c0b1960 2024-04-25T10:04:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.854+0000 7fe37bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe37c06cdd0 0x7fe37c0b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.854+0000 7fe37bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe37c06cdd0 0x7fe37c0b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57058/0 (socket says 172.21.15.16:57058) 2024-04-25T10:04:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.854+0000 7fe37bfff640 1 -- 172.21.15.16:0/3147723899 learned_addr learned my addr 172.21.15.16:0/3147723899 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:04:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.854+0000 7fe381372640 1 --2- 172.21.15.16:0/3147723899 >> v2:172.21.15.186:3300/0 conn(0x7fe37c0b1960 0x7fe37c0d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.854+0000 7fe37bfff640 1 -- 172.21.15.16:0/3147723899 >> v2:172.21.15.186:3300/0 conn(0x7fe37c0b1960 msgr2=0x7fe37c0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:48.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.854+0000 7fe37bfff640 1 --2- 172.21.15.16:0/3147723899 >> v2:172.21.15.186:3300/0 conn(0x7fe37c0b1960 0x7fe37c0d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.854+0000 7fe37bfff640 1 -- 172.21.15.16:0/3147723899 >> v1:172.21.15.186:6789/0 conn(0x7fe37c15dc30 legacy=0x7fe37c06c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:48.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.854+0000 7fe37bfff640 1 -- 172.21.15.16:0/3147723899 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c0b2060 con 0x7fe37c06cdd0 2024-04-25T10:04:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.854+0000 7fe37bfff640 1 --2- 172.21.15.16:0/3147723899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c06cdd0 0x7fe37c0b1420 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0c8980 tx=0x7fe36802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a027e90cb7cd7495 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.855+0000 7fe37b7fe640 1 -- 172.21.15.16:0/3147723899 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368037070 con 0x7fe37c06cdd0 2024-04-25T10:04:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.855+0000 7fe37b7fe640 1 -- 172.21.15.16:0/3147723899 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe368032440 con 0x7fe37c06cdd0 2024-04-25T10:04:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.855+0000 7fe37b7fe640 1 -- 172.21.15.16:0/3147723899 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368032810 con 0x7fe37c06cdd0 2024-04-25T10:04:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.855+0000 7fe382dd5640 1 -- 172.21.15.16:0/3147723899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c06cdd0 msgr2=0x7fe37c0b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.855+0000 7fe382dd5640 1 --2- 172.21.15.16:0/3147723899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c06cdd0 0x7fe37c0b1420 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0c8980 tx=0x7fe36802a230 comp rx=0 tx=0).stop 2024-04-25T10:04:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.855+0000 7fe382dd5640 1 -- 172.21.15.16:0/3147723899 shutdown_connections 2024-04-25T10:04:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.855+0000 7fe382dd5640 1 --2- 172.21.15.16:0/3147723899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c06cdd0 0x7fe37c0b1420 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.855+0000 7fe382dd5640 1 --2- 172.21.15.16:0/3147723899 >> v2:172.21.15.186:3300/0 conn(0x7fe37c0b1960 0x7fe37c0d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.855+0000 7fe382dd5640 1 -- 172.21.15.16:0/3147723899 >> 172.21.15.16:0/3147723899 conn(0x7fe37c0d1900 msgr2=0x7fe37c0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.855+0000 7fe382dd5640 1 -- 172.21.15.16:0/3147723899 shutdown_connections 2024-04-25T10:04:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.855+0000 7fe382dd5640 1 -- 172.21.15.16:0/3147723899 wait complete. 2024-04-25T10:04:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.856+0000 7fe382dd5640 1 Processor -- start 2024-04-25T10:04:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.857+0000 7fe382dd5640 1 -- start start 2024-04-25T10:04:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.857+0000 7fe382dd5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe37c06cdd0 0x7fe37c0b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.857+0000 7fe382dd5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c0b1960 0x7fe37c158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.857+0000 7fe382dd5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe37c15dc30 0x7fe37c0b6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.857+0000 7fe382dd5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c0cfde0 con 0x7fe37c0b1960 2024-04-25T10:04:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.857+0000 7fe382dd5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c0b2620 con 0x7fe37c06cdd0 2024-04-25T10:04:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.857+0000 7fe382dd5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c15e010 con 0x7fe37c15dc30 2024-04-25T10:04:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.857+0000 7fe37bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c0b1960 0x7fe37c158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe37bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c0b1960 0x7fe37c158190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57066/0 (socket says 172.21.15.16:57066) 2024-04-25T10:04:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe37bfff640 1 -- 172.21.15.16:0/3426662303 learned_addr learned my addr 172.21.15.16:0/3426662303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:04:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe381372640 1 --2- 172.21.15.16:0/3426662303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe37c15dc30 0x7fe37c0b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe380b71640 1 --2- 172.21.15.16:0/3426662303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe37c06cdd0 0x7fe37c0b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe381372640 1 -- 172.21.15.16:0/3426662303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe37c15dc30 msgr2=0x7fe37c0b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-25T10:04:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe381372640 1 -- 172.21.15.16:0/3426662303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe37c15dc30 msgr2=0x7fe37c0b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:04:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe381372640 1 --2- 172.21.15.16:0/3426662303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe37c15dc30 0x7fe37c0b6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe381372640 1 --2- 172.21.15.16:0/3426662303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe37c15dc30 0x7fe37c0b6960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe37bfff640 1 -- 172.21.15.16:0/3426662303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe37c15dc30 msgr2=0x7fe37c0b6960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:04:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe37bfff640 1 --2- 172.21.15.16:0/3426662303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe37c15dc30 0x7fe37c0b6960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe37bfff640 1 -- 172.21.15.16:0/3426662303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe37c06cdd0 msgr2=0x7fe37c0b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe37bfff640 1 --2- 172.21.15.16:0/3426662303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe37c06cdd0 0x7fe37c0b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe37bfff640 1 -- 172.21.15.16:0/3426662303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c0c6c90 con 0x7fe37c0b1960 2024-04-25T10:04:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe380b71640 1 --2- 172.21.15.16:0/3426662303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe37c06cdd0 0x7fe37c0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe37bfff640 1 --2- 172.21.15.16:0/3426662303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c0b1960 0x7fe37c158190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe368000c00 tx=0x7fe3680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe3797fa640 1 -- 172.21.15.16:0/3426662303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368041020 con 0x7fe37c0b1960 2024-04-25T10:04:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe3797fa640 1 -- 172.21.15.16:0/3426662303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3680306c0 con 0x7fe37c0b1960 2024-04-25T10:04:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe3797fa640 1 -- 172.21.15.16:0/3426662303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368037040 con 0x7fe37c0b1960 2024-04-25T10:04:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe382dd5640 1 -- 172.21.15.16:0/3426662303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37c0c6f70 con 0x7fe37c0b1960 2024-04-25T10:04:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe382dd5640 1 -- 172.21.15.16:0/3426662303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37c15e440 con 0x7fe37c0b1960 2024-04-25T10:04:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe3797fa640 1 -- 172.21.15.16:0/3426662303 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe368044020 con 0x7fe37c0b1960 2024-04-25T10:04:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.858+0000 7fe3797fa640 1 -- 172.21.15.16:0/3426662303 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fe36803dc40 con 0x7fe37c0b1960 2024-04-25T10:04:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.859+0000 7fe382dd5640 1 -- 172.21.15.16:0/3426662303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37c067850 con 0x7fe37c0b1960 2024-04-25T10:04:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.861+0000 7fe3797fa640 1 -- 172.21.15.16:0/3426662303 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe36804b030 con 0x7fe37c0b1960 2024-04-25T10:04:48.876 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:48.872+0000 7fc470923040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:04:48.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.923+0000 7fe382dd5640 1 -- 172.21.15.16:0/3426662303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe37c0749b0 con 0x7fe37c0b1960 2024-04-25T10:04:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.924+0000 7fe3797fa640 1 -- 172.21.15.16:0/3426662303 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fe37c06c1f0 con 0x7fe37c0b1960 2024-04-25T10:04:48.929 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:04:48.929 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:04:48.672971+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.924+0000 7fe382dd5640 1 -- 172.21.15.16:0/3426662303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c0b1960 msgr2=0x7fe37c158190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.924+0000 7fe382dd5640 1 --2- 172.21.15.16:0/3426662303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c0b1960 0x7fe37c158190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe368000c00 tx=0x7fe3680046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.925+0000 7fe382dd5640 1 -- 172.21.15.16:0/3426662303 shutdown_connections 2024-04-25T10:04:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.925+0000 7fe382dd5640 1 --2- 172.21.15.16:0/3426662303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe37c15dc30 0x7fe37c0b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.925+0000 7fe382dd5640 1 --2- 172.21.15.16:0/3426662303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe37c0b1960 0x7fe37c158190 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.925+0000 7fe382dd5640 1 --2- 172.21.15.16:0/3426662303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe37c06cdd0 0x7fe37c0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.925+0000 7fe382dd5640 1 -- 172.21.15.16:0/3426662303 >> 172.21.15.16:0/3426662303 conn(0x7fe37c0d1900 msgr2=0x7fe37c0b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.925+0000 7fe382dd5640 1 -- 172.21.15.16:0/3426662303 shutdown_connections 2024-04-25T10:04:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:04:48.925+0000 7fe382dd5640 1 -- 172.21.15.16:0/3426662303 wait complete. 2024-04-25T10:04:48.940 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T10:04:48.940 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T10:04:49.002 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:48.997+0000 7fc684930040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:04:49.012 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:49.007+0000 7fc470923040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:04:49.063 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:49.058+0000 7fc470923040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:04:49.133 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:49.128+0000 7fc470923040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:04:49.145 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:49.140+0000 7fc684930040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:04:49.177 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:49.172+0000 7fc470923040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:04:49.197 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:49.192+0000 7fc684930040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:04:49.270 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:49.266+0000 7fc684930040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:04:49.301 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-04-25T10:04:49.296+0000 7fef31fa1640 -1 Falling back to public interface 2024-04-25T10:04:49.304 INFO:tasks.ceph.osd.5.smithi186.stderr:2024-04-25T10:04:49.299+0000 7fc9765cb640 -1 Falling back to public interface 2024-04-25T10:04:49.305 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-04-25T10:04:49.300+0000 7f920680c640 -1 Falling back to public interface 2024-04-25T10:04:49.313 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-04-25T10:04:49.308+0000 7fcdf2b33640 -1 Falling back to public interface 2024-04-25T10:04:49.316 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-04-25T10:04:49.311+0000 7f9fdb5a7640 -1 Falling back to public interface 2024-04-25T10:04:49.319 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:49.314+0000 7fc684930040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:04:49.320 INFO:tasks.ceph.osd.7.smithi186.stderr:2024-04-25T10:04:49.315+0000 7f5d2fcaf640 -1 Falling back to public interface 2024-04-25T10:04:49.322 INFO:tasks.ceph.osd.6.smithi186.stderr:2024-04-25T10:04:49.317+0000 7f12fe40c640 -1 Falling back to public interface 2024-04-25T10:04:49.326 INFO:tasks.ceph.osd.4.smithi186.stderr:2024-04-25T10:04:49.322+0000 7f4d9e41c640 -1 Falling back to public interface 2024-04-25T10:04:49.659 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:49.654+0000 7fc470923040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:04:49.746 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:49.771 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:49.767+0000 7fc470923040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:04:49.827 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:49.822+0000 7fc684930040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:04:49.842 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.837+0000 7f62eb3b6640 1 Processor -- start 2024-04-25T10:04:49.842 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.837+0000 7f62eb3b6640 1 -- start start 2024-04-25T10:04:49.844 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.838+0000 7f62eb3b6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62e415df20 0x7f62e406c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.844 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.838+0000 7f62eb3b6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e406cad0 0x7f62e4099d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.844 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.838+0000 7f62eb3b6640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e406ceb0 con 0x7f62e409a250 2024-04-25T10:04:49.844 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.838+0000 7f62eb3b6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e40cbe80 con 0x7f62e415df20 2024-04-25T10:04:49.844 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.838+0000 7f62eb3b6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e40cc9c0 con 0x7f62e406cad0 2024-04-25T10:04:49.844 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.838+0000 7f62e9152640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62e415df20 0x7f62e406c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.845 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.838+0000 7f62e9152640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62e415df20 0x7f62e406c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57068/0 (socket says 172.21.15.16:57068) 2024-04-25T10:04:49.845 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.838+0000 7f62e9152640 1 -- 172.21.15.16:0/3760807039 learned_addr learned my addr 172.21.15.16:0/3760807039 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:04:49.845 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.838+0000 7f62e8951640 1 --2- 172.21.15.16:0/3760807039 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e406cad0 0x7f62e4099d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.845 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.838+0000 7f62e8951640 1 -- 172.21.15.16:0/3760807039 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e406cad0 msgr2=0x7f62e4099d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:04:49.845 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.838+0000 7f62e8951640 1 -- 172.21.15.16:0/3760807039 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e406cad0 msgr2=0x7f62e4099d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:49.845 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.838+0000 7f62e8951640 1 --2- 172.21.15.16:0/3760807039 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e406cad0 0x7f62e4099d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:49.845 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.838+0000 7f62e9152640 1 -- 172.21.15.16:0/3760807039 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e406cad0 msgr2=0x7f62e4099d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:04:49.845 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.838+0000 7f62e8951640 1 --2- 172.21.15.16:0/3760807039 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e406cad0 0x7f62e4099d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:49.845 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.838+0000 7f62e9152640 1 --2- 172.21.15.16:0/3760807039 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e406cad0 0x7f62e4099d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.845 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.839+0000 7f62e9152640 1 -- 172.21.15.16:0/3760807039 >> v1:172.21.15.186:6789/0 conn(0x7f62e409a250 legacy=0x7f62e40cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.845 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.839+0000 7f62e9152640 1 -- 172.21.15.16:0/3760807039 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e40cc0b0 con 0x7f62e415df20 2024-04-25T10:04:49.845 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.839+0000 7f62e9152640 1 --2- 172.21.15.16:0/3760807039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e415df20 0x7f62e406c590 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009870 tx=0x7f62e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901d47d0a43ae06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.845 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.839+0000 7f62d7fff640 1 -- 172.21.15.16:0/3760807039 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e0037070 con 0x7f62e415df20 2024-04-25T10:04:49.845 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.839+0000 7f62d7fff640 1 -- 172.21.15.16:0/3760807039 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e0032440 con 0x7f62e415df20 2024-04-25T10:04:49.846 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.839+0000 7f62d7fff640 1 -- 172.21.15.16:0/3760807039 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e0032810 con 0x7f62e415df20 2024-04-25T10:04:49.849 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.840+0000 7f62eb3b6640 1 -- 172.21.15.16:0/3760807039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e415df20 msgr2=0x7f62e406c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.849 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.840+0000 7f62eb3b6640 1 --2- 172.21.15.16:0/3760807039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e415df20 0x7f62e406c590 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009870 tx=0x7f62e002a270 comp rx=0 tx=0).stop 2024-04-25T10:04:49.849 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.840+0000 7f62eb3b6640 1 -- 172.21.15.16:0/3760807039 shutdown_connections 2024-04-25T10:04:49.849 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.840+0000 7f62eb3b6640 1 --2- 172.21.15.16:0/3760807039 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e406cad0 0x7f62e4099d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.849 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.840+0000 7f62eb3b6640 1 --2- 172.21.15.16:0/3760807039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e415df20 0x7f62e406c590 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.849 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.840+0000 7f62eb3b6640 1 -- 172.21.15.16:0/3760807039 >> 172.21.15.16:0/3760807039 conn(0x7f62e40b3320 msgr2=0x7f62e40b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.849 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.840+0000 7f62eb3b6640 1 -- 172.21.15.16:0/3760807039 shutdown_connections 2024-04-25T10:04:49.849 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.840+0000 7f62eb3b6640 1 -- 172.21.15.16:0/3760807039 wait complete. 2024-04-25T10:04:49.849 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.841+0000 7f62eb3b6640 1 Processor -- start 2024-04-25T10:04:49.849 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.841+0000 7f62eb3b6640 1 -- start start 2024-04-25T10:04:49.849 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62eb3b6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e406cad0 0x7f62e40c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.850 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62eb3b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e409a250 0x7f62e415ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.850 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62eb3b6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62e414fed0 0x7f62e40cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.850 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62eb3b6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e409a6e0 con 0x7f62e409a250 2024-04-25T10:04:49.850 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62eb3b6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e40cc650 con 0x7f62e414fed0 2024-04-25T10:04:49.850 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62eb3b6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e40b0dc0 con 0x7f62e406cad0 2024-04-25T10:04:49.850 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62e8951640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e409a250 0x7f62e415ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.850 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62e8951640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e409a250 0x7f62e415ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57082/0 (socket says 172.21.15.16:57082) 2024-04-25T10:04:49.850 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62e8951640 1 -- 172.21.15.16:0/799081497 learned_addr learned my addr 172.21.15.16:0/799081497 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:04:49.850 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62e9152640 1 --2- 172.21.15.16:0/799081497 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e406cad0 0x7f62e40c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.850 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62e9953640 1 --2- 172.21.15.16:0/799081497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62e414fed0 0x7f62e40cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.850 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62e9152640 1 -- 172.21.15.16:0/799081497 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e406cad0 msgr2=0x7f62e40c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-25T10:04:49.850 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62e9152640 1 -- 172.21.15.16:0/799081497 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e406cad0 msgr2=0x7f62e40c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T10:04:49.850 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62e9152640 1 --2- 172.21.15.16:0/799081497 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e406cad0 0x7f62e40c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:49.851 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62e9152640 1 --2- 172.21.15.16:0/799081497 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e406cad0 0x7f62e40c96d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:49.851 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62e8951640 1 -- 172.21.15.16:0/799081497 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e406cad0 msgr2=0x7f62e40c96d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:04:49.851 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62e8951640 1 --2- 172.21.15.16:0/799081497 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e406cad0 0x7f62e40c96d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.851 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62e8951640 1 -- 172.21.15.16:0/799081497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62e414fed0 msgr2=0x7f62e40cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.851 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.842+0000 7f62e8951640 1 --2- 172.21.15.16:0/799081497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62e414fed0 0x7f62e40cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.851 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.843+0000 7f62e8951640 1 -- 172.21.15.16:0/799081497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e4160510 con 0x7f62e409a250 2024-04-25T10:04:49.851 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.843+0000 7f62e9953640 1 --2- 172.21.15.16:0/799081497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62e414fed0 0x7f62e40cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:49.851 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.843+0000 7f62e8951640 1 --2- 172.21.15.16:0/799081497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e409a250 0x7f62e415ffd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f62d800d800 tx=0x7f62d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.851 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.843+0000 7f62d5ffb640 1 -- 172.21.15.16:0/799081497 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d800b6c0 con 0x7f62e409a250 2024-04-25T10:04:49.851 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.843+0000 7f62d5ffb640 1 -- 172.21.15.16:0/799081497 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d8019070 con 0x7f62e409a250 2024-04-25T10:04:49.851 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.843+0000 7f62eb3b6640 1 -- 172.21.15.16:0/799081497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e41607b0 con 0x7f62e409a250 2024-04-25T10:04:49.851 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.843+0000 7f62d5ffb640 1 -- 172.21.15.16:0/799081497 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d800bac0 con 0x7f62e409a250 2024-04-25T10:04:49.851 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.843+0000 7f62eb3b6640 1 -- 172.21.15.16:0/799081497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e4160d70 con 0x7f62e409a250 2024-04-25T10:04:49.851 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.844+0000 7f62d5ffb640 1 -- 172.21.15.16:0/799081497 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 2) v1 ==== 44915+0+0 (secure 0 0 0) 0x7f62d800bc60 con 0x7f62e409a250 2024-04-25T10:04:49.852 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.844+0000 7f62d5ffb640 1 -- 172.21.15.16:0/799081497 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f62d804a130 con 0x7f62e409a250 2024-04-25T10:04:49.852 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.844+0000 7f62eb3b6640 1 -- 172.21.15.16:0/799081497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e4068530 con 0x7f62e409a250 2024-04-25T10:04:49.852 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.846+0000 7f62d5ffb640 1 -- 172.21.15.16:0/799081497 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f62d801d3c0 con 0x7f62e409a250 2024-04-25T10:04:49.914 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.909+0000 7f62eb3b6640 1 -- 172.21.15.16:0/799081497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62e40cdd00 con 0x7f62e409a250 2024-04-25T10:04:49.914 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.909+0000 7f62d5ffb640 1 -- 172.21.15.16:0/799081497 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f62d8013e00 con 0x7f62e409a250 2024-04-25T10:04:49.915 INFO:teuthology.misc.health.smithi016.stdout: 2024-04-25T10:04:49.915 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":10,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:04:48.672971+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:49.916 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.910+0000 7f62eb3b6640 1 -- 172.21.15.16:0/799081497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e409a250 msgr2=0x7f62e415ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.916 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.910+0000 7f62eb3b6640 1 --2- 172.21.15.16:0/799081497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e409a250 0x7f62e415ffd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f62d800d800 tx=0x7f62d800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.916 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.910+0000 7f62eb3b6640 1 -- 172.21.15.16:0/799081497 shutdown_connections 2024-04-25T10:04:49.916 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.910+0000 7f62eb3b6640 1 --2- 172.21.15.16:0/799081497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62e414fed0 0x7f62e40cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.916 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.910+0000 7f62eb3b6640 1 --2- 172.21.15.16:0/799081497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e409a250 0x7f62e415ffd0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.916 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.910+0000 7f62eb3b6640 1 --2- 172.21.15.16:0/799081497 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e406cad0 0x7f62e40c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.916 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.910+0000 7f62eb3b6640 1 -- 172.21.15.16:0/799081497 >> 172.21.15.16:0/799081497 conn(0x7f62e40b3320 msgr2=0x7f62e4070400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.916 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.911+0000 7f62eb3b6640 1 -- 172.21.15.16:0/799081497 shutdown_connections 2024-04-25T10:04:49.916 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:49.911+0000 7f62eb3b6640 1 -- 172.21.15.16:0/799081497 wait complete. 2024-04-25T10:04:49.925 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:04:49.951 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-04-25T10:04:49.946+0000 7fc684930040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:04:50.786 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:50.782+0000 7fc4114cf640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T10:04:50.789 INFO:tasks.ceph.mgr.x.smithi186.stderr:2024-04-25T10:04:50.785+0000 7fc4114cf640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T10:04:52.757 INFO:tasks.ceph.osd.5.smithi186.stderr:2024-04-25T10:04:52.752+0000 7fc9765cb640 -1 osd.5 0 log_to_monitors true 2024-04-25T10:04:52.783 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-04-25T10:04:52.778+0000 7fef31fa1640 -1 osd.2 0 log_to_monitors true 2024-04-25T10:04:52.823 INFO:tasks.ceph.osd.6.smithi186.stderr:2024-04-25T10:04:52.819+0000 7f12fe40c640 -1 osd.6 0 log_to_monitors true 2024-04-25T10:04:52.827 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-04-25T10:04:52.822+0000 7f920680c640 -1 osd.1 0 log_to_monitors true 2024-04-25T10:04:52.841 INFO:tasks.ceph.osd.4.smithi186.stderr:2024-04-25T10:04:52.837+0000 7f4d9e41c640 -1 osd.4 0 log_to_monitors true 2024-04-25T10:04:52.845 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-04-25T10:04:52.840+0000 7fcdf2b33640 -1 osd.0 0 log_to_monitors true 2024-04-25T10:04:52.847 INFO:tasks.ceph.osd.7.smithi186.stderr:2024-04-25T10:04:52.843+0000 7f5d2fcaf640 -1 osd.7 0 log_to_monitors true 2024-04-25T10:04:52.847 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-04-25T10:04:52.843+0000 7f9fdb5a7640 -1 osd.3 0 log_to_monitors true 2024-04-25T10:04:54.643 INFO:tasks.ceph.osd.5.smithi186.stderr:2024-04-25T10:04:54.638+0000 7fc971d3e640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T10:04:54.659 INFO:tasks.ceph.osd.5.smithi186.stderr:2024-04-25T10:04:54.655+0000 7fc96a370640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:04:56.388 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-04-25T10:04:56.383+0000 7fcdee2a6640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T10:04:56.399 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-04-25T10:04:56.394+0000 7fcde68d8640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:04:56.478 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-04-25T10:04:56.473+0000 7f92018ac640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T10:04:56.479 INFO:tasks.ceph.osd.6.smithi186.stderr:2024-04-25T10:04:56.474+0000 7f12f94ac640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T10:04:56.489 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-04-25T10:04:56.485+0000 7f91f9ede640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:04:56.492 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-04-25T10:04:56.487+0000 7f9fd6d1a640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T10:04:56.494 INFO:tasks.ceph.osd.6.smithi186.stderr:2024-04-25T10:04:56.489+0000 7f12f1ade640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:04:56.501 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-04-25T10:04:56.496+0000 7fef2d714640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T10:04:56.502 INFO:tasks.ceph.osd.4.smithi186.stderr:2024-04-25T10:04:56.497+0000 7f4d99636640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T10:04:56.505 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-04-25T10:04:56.501+0000 7f9fcf34c640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:04:56.518 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-04-25T10:04:56.514+0000 7fef25d46640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:04:56.524 INFO:tasks.ceph.osd.4.smithi186.stderr:2024-04-25T10:04:56.519+0000 7f4d91c68640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:04:56.529 INFO:tasks.ceph.osd.7.smithi186.stderr:2024-04-25T10:04:56.524+0000 7f5d2b422640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T10:04:56.545 INFO:tasks.ceph.osd.7.smithi186.stderr:2024-04-25T10:04:56.540+0000 7f5d23a54640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:04:56.731 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:56.824 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.819+0000 7f2f86f5c640 1 Processor -- start 2024-04-25T10:04:56.824 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.819+0000 7f2f86f5c640 1 -- start start 2024-04-25T10:04:56.828 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.820+0000 7f2f86f5c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2f8015de50 0x7f2f800d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.828 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.820+0000 7f2f86f5c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f800d2bf0 con 0x7f2f800d0970 2024-04-25T10:04:56.828 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.820+0000 7f2f86f5c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f800d2ed0 con 0x7f2f800d2590 2024-04-25T10:04:56.828 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.820+0000 7f2f86f5c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f800d3800 con 0x7f2f8015de50 2024-04-25T10:04:56.828 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.820+0000 7f2f7ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2f800d0970 0x7f2f800d1e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34846/0 (socket says 172.21.15.16:34846) 2024-04-25T10:04:56.828 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.820+0000 7f2f7ffff640 1 -- 172.21.15.16:0/1024767239 learned_addr learned my addr 172.21.15.16:0/1024767239 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:04:56.829 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.820+0000 7f2f7f7fe640 1 -- 172.21.15.16:0/1024767239 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3380316054 0 0) 0x7f2f800d2bf0 con 0x7f2f800d0970 2024-04-25T10:04:56.829 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.820+0000 7f2f84cf8640 1 --2- 172.21.15.16:0/1024767239 >> v2:172.21.15.186:3300/0 conn(0x7f2f8015de50 0x7f2f800d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.829 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.820+0000 7f2f7f7fe640 1 -- 172.21.15.16:0/1024767239 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f68003520 con 0x7f2f800d0970 2024-04-25T10:04:56.829 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f7f7fe640 1 -- 172.21.15.16:0/1024767239 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1536094040 0 0) 0x7f2f68003520 con 0x7f2f800d0970 2024-04-25T10:04:56.829 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f7f7fe640 1 -- 172.21.15.16:0/1024767239 >> v2:172.21.15.186:3300/0 conn(0x7f2f8015de50 msgr2=0x7f2f800d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:56.829 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f7f7fe640 1 --2- 172.21.15.16:0/1024767239 >> v2:172.21.15.186:3300/0 conn(0x7f2f8015de50 0x7f2f800d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.829 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f7f7fe640 1 -- 172.21.15.16:0/1024767239 >> v1:172.21.15.186:6789/0 conn(0x7f2f800d2590 legacy=0x7f2f800b7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.829 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f7f7fe640 1 -- 172.21.15.16:0/1024767239 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f800d31b0 con 0x7f2f800d0970 2024-04-25T10:04:56.829 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f7f7fe640 1 -- 172.21.15.16:0/1024767239 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f2f74001670 con 0x7f2f800d0970 2024-04-25T10:04:56.829 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f7f7fe640 1 -- 172.21.15.16:0/1024767239 >> v1:172.21.15.16:6789/0 conn(0x7f2f800d0970 legacy=0x7f2f800d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.829 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f7f7fe640 1 --2- 172.21.15.16:0/1024767239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f680039b0 0x7f2f68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.829 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f7f7fe640 1 --2- 172.21.15.16:0/1024767239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f68004800 0x7f2f68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.829 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f7f7fe640 1 --2- 172.21.15.16:0/1024767239 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f680051c0 0x7f2f6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.829 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f7f7fe640 1 -- 172.21.15.16:0/1024767239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f800d3800 con 0x7f2f680039b0 2024-04-25T10:04:56.830 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f7f7fe640 1 -- 172.21.15.16:0/1024767239 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6800c060 con 0x7f2f68004800 2024-04-25T10:04:56.830 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f7f7fe640 1 -- 172.21.15.16:0/1024767239 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6800c260 con 0x7f2f680051c0 2024-04-25T10:04:56.830 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f84cf8640 1 --2- 172.21.15.16:0/1024767239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f680039b0 0x7f2f68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.830 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f7ffff640 1 --2- 172.21.15.16:0/1024767239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f68004800 0x7f2f68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.830 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f854f9640 1 --2- 172.21.15.16:0/1024767239 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f680051c0 0x7f2f6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.830 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f84cf8640 1 -- 172.21.15.16:0/1024767239 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f680051c0 msgr2=0x7f2f6800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:56.830 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f84cf8640 1 --2- 172.21.15.16:0/1024767239 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f680051c0 0x7f2f6800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.830 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f84cf8640 1 -- 172.21.15.16:0/1024767239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f68004800 msgr2=0x7f2f68004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:56.830 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f84cf8640 1 --2- 172.21.15.16:0/1024767239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f68004800 0x7f2f68004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.830 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f84cf8640 1 -- 172.21.15.16:0/1024767239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f68003520 con 0x7f2f680039b0 2024-04-25T10:04:56.830 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f854f9640 1 --2- 172.21.15.16:0/1024767239 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f680051c0 0x7f2f6800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:56.830 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.821+0000 7f2f854f9640 1 -- 172.21.15.16:0/1024767239 reap_dead start 2024-04-25T10:04:56.830 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.822+0000 7f2f84cf8640 1 --2- 172.21.15.16:0/1024767239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f680039b0 0x7f2f68003dd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009ac0 tx=0x7f2f68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d23f7ebd46b8d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:56.830 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.822+0000 7f2f7f7fe640 1 -- 172.21.15.16:0/1024767239 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f70036070 con 0x7f2f680039b0 2024-04-25T10:04:56.831 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.822+0000 7f2f7f7fe640 1 -- 172.21.15.16:0/1024767239 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f70031400 con 0x7f2f680039b0 2024-04-25T10:04:56.831 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.822+0000 7f2f7f7fe640 1 -- 172.21.15.16:0/1024767239 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f70009690 con 0x7f2f680039b0 2024-04-25T10:04:56.831 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.822+0000 7f2f86f5c640 1 -- 172.21.15.16:0/1024767239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f680039b0 msgr2=0x7f2f68003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.831 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.822+0000 7f2f86f5c640 1 --2- 172.21.15.16:0/1024767239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f680039b0 0x7f2f68003dd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009ac0 tx=0x7f2f68005830 comp rx=0 tx=0).stop 2024-04-25T10:04:56.831 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.823+0000 7f2f86f5c640 1 -- 172.21.15.16:0/1024767239 shutdown_connections 2024-04-25T10:04:56.831 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.823+0000 7f2f86f5c640 1 --2- 172.21.15.16:0/1024767239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f680039b0 0x7f2f68003dd0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.831 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.823+0000 7f2f86f5c640 1 -- 172.21.15.16:0/1024767239 >> 172.21.15.16:0/1024767239 conn(0x7f2f8006bfd0 msgr2=0x7f2f8006c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:56.831 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.823+0000 7f2f86f5c640 1 -- 172.21.15.16:0/1024767239 shutdown_connections 2024-04-25T10:04:56.831 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.823+0000 7f2f86f5c640 1 -- 172.21.15.16:0/1024767239 wait complete. 2024-04-25T10:04:56.834 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.824+0000 7f2f86f5c640 1 Processor -- start 2024-04-25T10:04:56.834 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.824+0000 7f2f86f5c640 1 -- start start 2024-04-25T10:04:56.834 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f86f5c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f680039b0 0x7f2f80144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.834 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f86f5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f801602f0 0x7f2f800ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.834 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f86f5c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f800c91d0 0x7f2f800c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.834 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f86f5c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8015e3b0 con 0x7f2f801602f0 2024-04-25T10:04:56.834 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f86f5c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8015e230 con 0x7f2f680039b0 2024-04-25T10:04:56.834 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f86f5c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f800d0430 con 0x7f2f800c91d0 2024-04-25T10:04:56.834 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f7ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f801602f0 0x7f2f800ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.835 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f7ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f801602f0 0x7f2f800ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38348/0 (socket says 172.21.15.16:38348) 2024-04-25T10:04:56.835 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f7ffff640 1 -- 172.21.15.16:0/3146986515 learned_addr learned my addr 172.21.15.16:0/3146986515 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:04:56.835 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f84cf8640 1 --2- 172.21.15.16:0/3146986515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f680039b0 0x7f2f80144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.835 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f854f9640 1 --2- 172.21.15.16:0/3146986515 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f800c91d0 0x7f2f800c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.835 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f7ffff640 1 -- 172.21.15.16:0/3146986515 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f800c91d0 msgr2=0x7f2f800c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.835 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f7ffff640 1 --2- 172.21.15.16:0/3146986515 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f800c91d0 0x7f2f800c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.835 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f7ffff640 1 -- 172.21.15.16:0/3146986515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f680039b0 msgr2=0x7f2f80144150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.835 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f7ffff640 1 --2- 172.21.15.16:0/3146986515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f680039b0 0x7f2f80144150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.835 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f7ffff640 1 -- 172.21.15.16:0/3146986515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f80160ad0 con 0x7f2f801602f0 2024-04-25T10:04:56.835 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f854f9640 1 --2- 172.21.15.16:0/3146986515 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f800c91d0 0x7f2f800c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:56.835 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f7ffff640 1 --2- 172.21.15.16:0/3146986515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f801602f0 0x7f2f800ccf70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2f740072f0 tx=0x7f2f740077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:56.835 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f7d7fa640 1 -- 172.21.15.16:0/3146986515 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7401a070 con 0x7f2f801602f0 2024-04-25T10:04:56.835 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f7d7fa640 1 -- 172.21.15.16:0/3146986515 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7400f9c0 con 0x7f2f801602f0 2024-04-25T10:04:56.835 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f86f5c640 1 -- 172.21.15.16:0/3146986515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f80160db0 con 0x7f2f801602f0 2024-04-25T10:04:56.836 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f7d7fa640 1 -- 172.21.15.16:0/3146986515 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7400fd70 con 0x7f2f801602f0 2024-04-25T10:04:56.836 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.825+0000 7f2f86f5c640 1 -- 172.21.15.16:0/3146986515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f80161370 con 0x7f2f801602f0 2024-04-25T10:04:56.836 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.826+0000 7f2f86f5c640 1 -- 172.21.15.16:0/3146986515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f800d0760 con 0x7f2f801602f0 2024-04-25T10:04:56.836 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.827+0000 7f2f7d7fa640 1 -- 172.21.15.16:0/3146986515 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f2f7401d070 con 0x7f2f801602f0 2024-04-25T10:04:56.836 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.827+0000 7f2f7d7fa640 1 --2- 172.21.15.16:0/3146986515 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2f6c06b470 0x7f2f6c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.836 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.827+0000 7f2f7d7fa640 1 -- 172.21.15.16:0/3146986515 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2511+0+0 (secure 0 0 0) 0x7f2f74088f60 con 0x7f2f801602f0 2024-04-25T10:04:56.836 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.827+0000 7f2f84cf8640 1 --2- 172.21.15.16:0/3146986515 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2f6c06b470 0x7f2f6c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.836 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.828+0000 7f2f84cf8640 1 --2- 172.21.15.16:0/3146986515 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2f6c06b470 0x7f2f6c06d930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2f70033000 tx=0x7f2f70008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:56.836 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:56.832+0000 7f2f7d7fa640 1 -- 172.21.15.16:0/3146986515 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f800d0760 con 0x7f2f801602f0 2024-04-25T10:04:57.017 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:57.012+0000 7f2f86f5c640 1 -- 172.21.15.16:0/3146986515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f80068530 con 0x7f2f801602f0 2024-04-25T10:04:57.018 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:57.013+0000 7f2f7d7fa640 1 -- 172.21.15.16:0/3146986515 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+7590 (secure 0 0 0) 0x7f2f74058b40 con 0x7f2f801602f0 2024-04-25T10:04:57.018 INFO:teuthology.misc.health.smithi016.stdout: 2024-04-25T10:04:57.018 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":14,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:04:55.948949+0000","last_up_change":"2024-04-25T10:04:54.784206+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:57.020 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:57.014+0000 7f2f86f5c640 1 -- 172.21.15.16:0/3146986515 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2f6c06b470 msgr2=0x7f2f6c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.020 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:57.014+0000 7f2f86f5c640 1 --2- 172.21.15.16:0/3146986515 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2f6c06b470 0x7f2f6c06d930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2f70033000 tx=0x7f2f70008030 comp rx=0 tx=0).stop 2024-04-25T10:04:57.020 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:57.014+0000 7f2f86f5c640 1 -- 172.21.15.16:0/3146986515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f801602f0 msgr2=0x7f2f800ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.020 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:57.014+0000 7f2f86f5c640 1 --2- 172.21.15.16:0/3146986515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f801602f0 0x7f2f800ccf70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2f740072f0 tx=0x7f2f740077b0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.020 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:57.015+0000 7f2f86f5c640 1 -- 172.21.15.16:0/3146986515 shutdown_connections 2024-04-25T10:04:57.020 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:57.015+0000 7f2f86f5c640 1 --2- 172.21.15.16:0/3146986515 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2f6c06b470 0x7f2f6c06d930 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.020 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:57.015+0000 7f2f86f5c640 1 --2- 172.21.15.16:0/3146986515 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f800c91d0 0x7f2f800c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.020 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:57.015+0000 7f2f86f5c640 1 --2- 172.21.15.16:0/3146986515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f801602f0 0x7f2f800ccf70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.020 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:57.015+0000 7f2f86f5c640 1 --2- 172.21.15.16:0/3146986515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f680039b0 0x7f2f80144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.020 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:57.015+0000 7f2f86f5c640 1 -- 172.21.15.16:0/3146986515 >> 172.21.15.16:0/3146986515 conn(0x7f2f8006bfd0 msgr2=0x7f2f800b8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:57.020 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:57.015+0000 7f2f86f5c640 1 -- 172.21.15.16:0/3146986515 shutdown_connections 2024-04-25T10:04:57.021 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:04:57.015+0000 7f2f86f5c640 1 -- 172.21.15.16:0/3146986515 wait complete. 2024-04-25T10:04:57.029 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-04-25T10:05:03.836 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:03.941 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.935+0000 7fb33e007640 1 Processor -- start 2024-04-25T10:05:03.941 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.935+0000 7fb33e007640 1 -- start start 2024-04-25T10:05:03.943 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.936+0000 7fb33e007640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb3380d0b40 0x7fb3380d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.943 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.936+0000 7fb33e007640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb3380d23c0 0x7fb3380b7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.943 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.936+0000 7fb33e007640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3380d2bf0 con 0x7fb33815de50 2024-04-25T10:05:03.943 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.936+0000 7fb33e007640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3380d3800 con 0x7fb3380d23c0 2024-04-25T10:05:03.944 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.936+0000 7fb33e007640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3380d2ed0 con 0x7fb3380d0b40 2024-04-25T10:05:03.944 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.936+0000 7fb337fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb3380d23c0 0x7fb3380b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.944 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.936+0000 7fb337fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb3380d23c0 0x7fb3380b7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56940/0 (socket says 172.21.15.16:56940) 2024-04-25T10:05:03.944 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.936+0000 7fb337fff640 1 -- 172.21.15.16:0/479332354 learned_addr learned my addr 172.21.15.16:0/479332354 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:03.944 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.936+0000 7fb3377fe640 1 --1- 172.21.15.16:0/479332354 >> v1:172.21.15.16:6789/0 conn(0x7fb33815de50 0x7fb3380d0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43172/0 (socket says 172.21.15.16:43172) 2024-04-25T10:05:03.944 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.936+0000 7fb336ffd640 1 --2- 172.21.15.16:0/479332354 >> v2:172.21.15.186:3300/0 conn(0x7fb3380d0b40 0x7fb3380d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.944 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.936+0000 7fb337fff640 1 -- 172.21.15.16:0/479332354 >> v2:172.21.15.186:3300/0 conn(0x7fb3380d0b40 msgr2=0x7fb3380d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:03.944 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.936+0000 7fb337fff640 1 --2- 172.21.15.16:0/479332354 >> v2:172.21.15.186:3300/0 conn(0x7fb3380d0b40 0x7fb3380d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.944 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.936+0000 7fb337fff640 1 -- 172.21.15.16:0/479332354 >> v1:172.21.15.16:6789/0 conn(0x7fb33815de50 legacy=0x7fb3380d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.944 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.936+0000 7fb3367fc640 1 -- 172.21.15.16:0/479332354 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145937677 0 0) 0x7fb3380d2bf0 con 0x7fb33815de50 2024-04-25T10:05:03.944 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.936+0000 7fb337fff640 1 -- 172.21.15.16:0/479332354 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3380d31b0 con 0x7fb3380d23c0 2024-04-25T10:05:03.944 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.937+0000 7fb337fff640 1 --2- 172.21.15.16:0/479332354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3380d23c0 0x7fb3380b7080 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb3380d3980 tx=0x7fb33002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b5dadc419382e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:03.944 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.937+0000 7fb3367fc640 1 -- 172.21.15.16:0/479332354 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330036070 con 0x7fb3380d23c0 2024-04-25T10:05:03.945 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.937+0000 7fb3367fc640 1 -- 172.21.15.16:0/479332354 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb330031600 con 0x7fb3380d23c0 2024-04-25T10:05:03.945 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.937+0000 7fb3367fc640 1 -- 172.21.15.16:0/479332354 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3300319d0 con 0x7fb3380d23c0 2024-04-25T10:05:03.947 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.938+0000 7fb33e007640 1 -- 172.21.15.16:0/479332354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3380d23c0 msgr2=0x7fb3380b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.947 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.938+0000 7fb33e007640 1 --2- 172.21.15.16:0/479332354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3380d23c0 0x7fb3380b7080 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb3380d3980 tx=0x7fb33002a730 comp rx=0 tx=0).stop 2024-04-25T10:05:03.948 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.938+0000 7fb33e007640 1 -- 172.21.15.16:0/479332354 shutdown_connections 2024-04-25T10:05:03.948 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.938+0000 7fb33e007640 1 --2- 172.21.15.16:0/479332354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3380d23c0 0x7fb3380b7080 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.948 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.938+0000 7fb33e007640 1 --2- 172.21.15.16:0/479332354 >> v2:172.21.15.186:3300/0 conn(0x7fb3380d0b40 0x7fb3380d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.948 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.938+0000 7fb33e007640 1 -- 172.21.15.16:0/479332354 >> 172.21.15.16:0/479332354 conn(0x7fb33806bfd0 msgr2=0x7fb33806c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:03.948 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.938+0000 7fb33e007640 1 -- 172.21.15.16:0/479332354 shutdown_connections 2024-04-25T10:05:03.948 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.938+0000 7fb33e007640 1 -- 172.21.15.16:0/479332354 wait complete. 2024-04-25T10:05:03.948 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.939+0000 7fb33e007640 1 Processor -- start 2024-04-25T10:05:03.948 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.939+0000 7fb33e007640 1 -- start start 2024-04-25T10:05:03.948 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.940+0000 7fb33e007640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3380d0b40 0x7fb3381502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.948 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.940+0000 7fb33e007640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3380d23c0 0x7fb3381548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.948 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.940+0000 7fb33e007640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb33815de50 0x7fb33815ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.948 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.940+0000 7fb33e007640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380d29b0 con 0x7fb3380d23c0 2024-04-25T10:05:03.948 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.940+0000 7fb33e007640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380d2830 con 0x7fb33815de50 2024-04-25T10:05:03.948 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.940+0000 7fb33e007640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33815e230 con 0x7fb3380d0b40 2024-04-25T10:05:03.949 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.940+0000 7fb336ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3380d23c0 0x7fb3381548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.949 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.940+0000 7fb336ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3380d23c0 0x7fb3381548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56954/0 (socket says 172.21.15.16:56954) 2024-04-25T10:05:03.949 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.940+0000 7fb336ffd640 1 -- 172.21.15.16:0/2961413790 learned_addr learned my addr 172.21.15.16:0/2961413790 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:03.949 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.940+0000 7fb3377fe640 1 --2- 172.21.15.16:0/2961413790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3380d0b40 0x7fb3381502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.949 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.941+0000 7fb337fff640 1 --2- 172.21.15.16:0/2961413790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb33815de50 0x7fb33815ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.949 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.941+0000 7fb336ffd640 1 -- 172.21.15.16:0/2961413790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3380d0b40 msgr2=0x7fb3381502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.949 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.941+0000 7fb336ffd640 1 --2- 172.21.15.16:0/2961413790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3380d0b40 0x7fb3381502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.949 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.941+0000 7fb336ffd640 1 -- 172.21.15.16:0/2961413790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb33815de50 msgr2=0x7fb33815ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.949 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.941+0000 7fb336ffd640 1 --2- 172.21.15.16:0/2961413790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb33815de50 0x7fb33815ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.949 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.941+0000 7fb336ffd640 1 -- 172.21.15.16:0/2961413790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3381603d0 con 0x7fb3380d23c0 2024-04-25T10:05:03.949 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.941+0000 7fb336ffd640 1 --2- 172.21.15.16:0/2961413790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3380d23c0 0x7fb3381548b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00c7f0 tx=0x7fb32c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:03.949 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.941+0000 7fb31ffff640 1 -- 172.21.15.16:0/2961413790 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c015070 con 0x7fb3380d23c0 2024-04-25T10:05:03.949 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.941+0000 7fb31ffff640 1 -- 172.21.15.16:0/2961413790 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c004240 con 0x7fb3380d23c0 2024-04-25T10:05:03.949 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.941+0000 7fb33e007640 1 -- 172.21.15.16:0/2961413790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3381514d0 con 0x7fb3380d23c0 2024-04-25T10:05:03.950 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.941+0000 7fb31ffff640 1 -- 172.21.15.16:0/2961413790 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c007620 con 0x7fb3380d23c0 2024-04-25T10:05:03.950 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.941+0000 7fb33e007640 1 -- 172.21.15.16:0/2961413790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb338151a90 con 0x7fb3380d23c0 2024-04-25T10:05:03.950 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.942+0000 7fb33e007640 1 -- 172.21.15.16:0/2961413790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb338068530 con 0x7fb3380d23c0 2024-04-25T10:05:03.951 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.942+0000 7fb31ffff640 1 -- 172.21.15.16:0/2961413790 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb32c0079d0 con 0x7fb3380d23c0 2024-04-25T10:05:03.951 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.943+0000 7fb31ffff640 1 --2- 172.21.15.16:0/2961413790 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb32406b4d0 0x7fb32406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.951 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.943+0000 7fb31ffff640 1 -- 172.21.15.16:0/2961413790 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb32c089f20 con 0x7fb3380d23c0 2024-04-25T10:05:03.951 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.943+0000 7fb3377fe640 1 --2- 172.21.15.16:0/2961413790 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb32406b4d0 0x7fb32406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.951 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.944+0000 7fb3377fe640 1 --2- 172.21.15.16:0/2961413790 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb32406b4d0 0x7fb32406d990 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb328000f60 tx=0x7fb328007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:03.953 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:03.947+0000 7fb31ffff640 1 -- 172.21.15.16:0/2961413790 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb32c0590e0 con 0x7fb3380d23c0 2024-04-25T10:05:04.132 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:04.126+0000 7fb33e007640 1 -- 172.21.15.16:0/2961413790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb33815b740 con 0x7fb3380d23c0 2024-04-25T10:05:04.134 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:04.127+0000 7fb31ffff640 1 -- 172.21.15.16:0/2961413790 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12884 (secure 0 0 0) 0x7fb32c0589e0 con 0x7fb3380d23c0 2024-04-25T10:05:04.134 INFO:teuthology.misc.health.smithi016.stdout: 2024-04-25T10:05:04.134 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":18,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:00.401566+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:04.135 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:04.129+0000 7fb33e007640 1 -- 172.21.15.16:0/2961413790 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb32406b4d0 msgr2=0x7fb32406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.135 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:04.129+0000 7fb33e007640 1 --2- 172.21.15.16:0/2961413790 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb32406b4d0 0x7fb32406d990 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb328000f60 tx=0x7fb328007c00 comp rx=0 tx=0).stop 2024-04-25T10:05:04.135 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:04.129+0000 7fb33e007640 1 -- 172.21.15.16:0/2961413790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3380d23c0 msgr2=0x7fb3381548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.135 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:04.129+0000 7fb33e007640 1 --2- 172.21.15.16:0/2961413790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3380d23c0 0x7fb3381548b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00c7f0 tx=0x7fb32c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.136 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:04.129+0000 7fb33e007640 1 -- 172.21.15.16:0/2961413790 shutdown_connections 2024-04-25T10:05:04.136 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:04.129+0000 7fb33e007640 1 --2- 172.21.15.16:0/2961413790 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb32406b4d0 0x7fb32406d990 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.136 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:04.129+0000 7fb33e007640 1 --2- 172.21.15.16:0/2961413790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb33815de50 0x7fb33815ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.136 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:04.129+0000 7fb33e007640 1 --2- 172.21.15.16:0/2961413790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3380d23c0 0x7fb3381548b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.136 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:04.129+0000 7fb33e007640 1 --2- 172.21.15.16:0/2961413790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3380d0b40 0x7fb3381502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.136 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:04.129+0000 7fb33e007640 1 -- 172.21.15.16:0/2961413790 >> 172.21.15.16:0/2961413790 conn(0x7fb33806bfd0 msgr2=0x7fb33815bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:04.136 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:04.129+0000 7fb33e007640 1 -- 172.21.15.16:0/2961413790 shutdown_connections 2024-04-25T10:05:04.136 INFO:teuthology.misc.health.smithi016.stderr:2024-04-25T10:05:04.129+0000 7fb33e007640 1 -- 172.21.15.16:0/2961413790 wait complete. 2024-04-25T10:05:04.145 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T10:05:04.145 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T10:05:04.145 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:05:04.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:05:04.148 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:05:04.148 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:05:04.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:05:04.150 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:05:04.150 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:05:04.150 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:05:04.153 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:05:04.153 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:05:04.153 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:05:04.155 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:05:04.156 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T10:05:04.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:04.222 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-04-25T10:05:04.223 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-04-25T10:05:04.236 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2024-04-25T10:05:04.238 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2024-04-25T10:05:04.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.255+0000 7f02f0db0640 1 Processor -- start 2024-04-25T10:05:04.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.255+0000 7f02f0db0640 1 -- start start 2024-04-25T10:05:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.255+0000 7f02f0db0640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f02ec15dce0 0x7f02ec0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02f0db0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ec15e0c0 con 0x7f02ec0d35b0 2024-04-25T10:05:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02f0db0640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ec0d2510 con 0x7f02ec0b5b60 2024-04-25T10:05:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02f0db0640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ec0d27f0 con 0x7f02ec15dce0 2024-04-25T10:05:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02e9d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f02ec0d35b0 0x7f02ec0b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43202/0 (socket says 172.21.15.16:43202) 2024-04-25T10:05:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02e9d9b640 1 -- 172.21.15.16:0/2700376165 learned_addr learned my addr 172.21.15.16:0/2700376165 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02e959a640 1 -- 172.21.15.16:0/2700376165 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1428027852 0 0) 0x7f02ec15e0c0 con 0x7f02ec0d35b0 2024-04-25T10:05:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02e959a640 1 -- 172.21.15.16:0/2700376165 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02d4003520 con 0x7f02ec0d35b0 2024-04-25T10:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02ea59c640 1 --2- 172.21.15.16:0/2700376165 >> v2:172.21.15.186:3300/0 conn(0x7f02ec15dce0 0x7f02ec0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02e959a640 1 -- 172.21.15.16:0/2700376165 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1078848706 0 0) 0x7f02d4003520 con 0x7f02ec0d35b0 2024-04-25T10:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02e959a640 1 -- 172.21.15.16:0/2700376165 >> v2:172.21.15.186:3300/0 conn(0x7f02ec15dce0 msgr2=0x7f02ec0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02e959a640 1 --2- 172.21.15.16:0/2700376165 >> v2:172.21.15.186:3300/0 conn(0x7f02ec15dce0 0x7f02ec0d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02e959a640 1 -- 172.21.15.16:0/2700376165 >> v1:172.21.15.186:6789/0 conn(0x7f02ec0b5b60 legacy=0x7f02ec0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02e959a640 1 -- 172.21.15.16:0/2700376165 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ec0d2ad0 con 0x7f02ec0d35b0 2024-04-25T10:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02e959a640 1 -- 172.21.15.16:0/2700376165 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f02e0001670 con 0x7f02ec0d35b0 2024-04-25T10:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02ea59c640 1 --2- 172.21.15.16:0/2700376165 >> v2:172.21.15.186:3300/0 conn(0x7f02ec15dce0 0x7f02ec0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02e959a640 1 -- 172.21.15.16:0/2700376165 >> v1:172.21.15.16:6789/0 conn(0x7f02ec0d35b0 legacy=0x7f02ec0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02e959a640 1 --2- 172.21.15.16:0/2700376165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d40039b0 0x7f02d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02e959a640 1 --2- 172.21.15.16:0/2700376165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02d4004800 0x7f02d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02e959a640 1 --2- 172.21.15.16:0/2700376165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d40051c0 0x7f02d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02e959a640 1 -- 172.21.15.16:0/2700376165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec0d27f0 con 0x7f02d40051c0 2024-04-25T10:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02e959a640 1 -- 172.21.15.16:0/2700376165 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d400c110 con 0x7f02d4004800 2024-04-25T10:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02e959a640 1 -- 172.21.15.16:0/2700376165 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d400c310 con 0x7f02d40039b0 2024-04-25T10:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02ead9d640 1 --2- 172.21.15.16:0/2700376165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d40051c0 0x7f02d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02e9d9b640 1 --2- 172.21.15.16:0/2700376165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02d4004800 0x7f02d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.256+0000 7f02ea59c640 1 --2- 172.21.15.16:0/2700376165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d40039b0 0x7f02d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.257+0000 7f02ead9d640 1 -- 172.21.15.16:0/2700376165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d40039b0 msgr2=0x7f02d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.257+0000 7f02ead9d640 1 --2- 172.21.15.16:0/2700376165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d40039b0 0x7f02d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.257+0000 7f02ead9d640 1 -- 172.21.15.16:0/2700376165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02d4004800 msgr2=0x7f02d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.257+0000 7f02ead9d640 1 --2- 172.21.15.16:0/2700376165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02d4004800 0x7f02d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.257+0000 7f02ead9d640 1 -- 172.21.15.16:0/2700376165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d4003520 con 0x7f02d40051c0 2024-04-25T10:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.257+0000 7f02ead9d640 1 -- 172.21.15.16:0/2700376165 reap_dead start 2024-04-25T10:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.257+0000 7f02ead9d640 1 --2- 172.21.15.16:0/2700376165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d40051c0 0x7f02d400b9c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f02dc001750 tx=0x7f02d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fb7dc1ce101965 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.257+0000 7f02e959a640 1 -- 172.21.15.16:0/2700376165 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc001370 con 0x7f02d40051c0 2024-04-25T10:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.257+0000 7f02e959a640 1 -- 172.21.15.16:0/2700376165 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc005310 con 0x7f02d40051c0 2024-04-25T10:05:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.257+0000 7f02e959a640 1 -- 172.21.15.16:0/2700376165 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc02abe0 con 0x7f02d40051c0 2024-04-25T10:05:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.258+0000 7f02f0db0640 1 -- 172.21.15.16:0/2700376165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d40051c0 msgr2=0x7f02d400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.258+0000 7f02f0db0640 1 --2- 172.21.15.16:0/2700376165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d40051c0 0x7f02d400b9c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f02dc001750 tx=0x7f02d4005830 comp rx=0 tx=0).stop 2024-04-25T10:05:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.258+0000 7f02f0db0640 1 -- 172.21.15.16:0/2700376165 shutdown_connections 2024-04-25T10:05:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.258+0000 7f02f0db0640 1 --2- 172.21.15.16:0/2700376165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d40051c0 0x7f02d400b9c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.258+0000 7f02f0db0640 1 -- 172.21.15.16:0/2700376165 >> 172.21.15.16:0/2700376165 conn(0x7f02ec072900 msgr2=0x7f02ec0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.258+0000 7f02f0db0640 1 -- 172.21.15.16:0/2700376165 shutdown_connections 2024-04-25T10:05:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.258+0000 7f02f0db0640 1 -- 172.21.15.16:0/2700376165 wait complete. 2024-04-25T10:05:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.259+0000 7f02f0db0640 1 Processor -- start 2024-04-25T10:05:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.260+0000 7f02f0db0640 1 -- start start 2024-04-25T10:05:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.260+0000 7f02f0db0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d40051c0 0x7f02ec15a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.260+0000 7f02f0db0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02ec160e30 0x7f02ec15f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.260+0000 7f02f0db0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02ec151260 0x7f02ec0b7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.260+0000 7f02f0db0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec0b9820 con 0x7f02d40051c0 2024-04-25T10:05:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.260+0000 7f02f0db0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec0d3090 con 0x7f02ec151260 2024-04-25T10:05:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.260+0000 7f02f0db0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec0b99a0 con 0x7f02ec160e30 2024-04-25T10:05:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.260+0000 7f02ea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d40051c0 0x7f02ec15a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.260+0000 7f02ea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d40051c0 0x7f02ec15a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57008/0 (socket says 172.21.15.16:57008) 2024-04-25T10:05:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.260+0000 7f02ea59c640 1 -- 172.21.15.16:0/1388477007 learned_addr learned my addr 172.21.15.16:0/1388477007 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.260+0000 7f02e9d9b640 1 --2- 172.21.15.16:0/1388477007 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02ec160e30 0x7f02ec15f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.261+0000 7f02ead9d640 1 --2- 172.21.15.16:0/1388477007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02ec151260 0x7f02ec0b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.261+0000 7f02ea59c640 1 -- 172.21.15.16:0/1388477007 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02ec160e30 msgr2=0x7f02ec15f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.261+0000 7f02ea59c640 1 --2- 172.21.15.16:0/1388477007 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02ec160e30 0x7f02ec15f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.261+0000 7f02ea59c640 1 -- 172.21.15.16:0/1388477007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02ec151260 msgr2=0x7f02ec0b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.261+0000 7f02ea59c640 1 --2- 172.21.15.16:0/1388477007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02ec151260 0x7f02ec0b7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.261+0000 7f02ea59c640 1 -- 172.21.15.16:0/1388477007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ec15fd90 con 0x7f02d40051c0 2024-04-25T10:05:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.261+0000 7f02ead9d640 1 --2- 172.21.15.16:0/1388477007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02ec151260 0x7f02ec0b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.261+0000 7f02ea59c640 1 --2- 172.21.15.16:0/1388477007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d40051c0 0x7f02ec15a100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f02e4009fa0 tx=0x7f02e40105f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.261+0000 7f02daffd640 1 -- 172.21.15.16:0/1388477007 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e4003990 con 0x7f02d40051c0 2024-04-25T10:05:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.261+0000 7f02daffd640 1 -- 172.21.15.16:0/1388477007 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e4003b30 con 0x7f02d40051c0 2024-04-25T10:05:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.261+0000 7f02daffd640 1 -- 172.21.15.16:0/1388477007 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e4003d20 con 0x7f02d40051c0 2024-04-25T10:05:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.261+0000 7f02f0db0640 1 -- 172.21.15.16:0/1388477007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ec160070 con 0x7f02d40051c0 2024-04-25T10:05:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.261+0000 7f02f0db0640 1 -- 172.21.15.16:0/1388477007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ec160630 con 0x7f02d40051c0 2024-04-25T10:05:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.262+0000 7f02f0db0640 1 -- 172.21.15.16:0/1388477007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ec075070 con 0x7f02d40051c0 2024-04-25T10:05:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.263+0000 7f02daffd640 1 -- 172.21.15.16:0/1388477007 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f02e401e020 con 0x7f02d40051c0 2024-04-25T10:05:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.263+0000 7f02daffd640 1 --2- 172.21.15.16:0/1388477007 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f02cc06b500 0x7f02cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.263+0000 7f02daffd640 1 -- 172.21.15.16:0/1388477007 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f02e408a6a0 con 0x7f02d40051c0 2024-04-25T10:05:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.263+0000 7f02e9d9b640 1 --2- 172.21.15.16:0/1388477007 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f02cc06b500 0x7f02cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.264+0000 7f02e9d9b640 1 --2- 172.21.15.16:0/1388477007 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f02cc06b500 0x7f02cc06d9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f02ec0cd4f0 tx=0x7f02e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.267+0000 7f02daffd640 1 -- 172.21.15.16:0/1388477007 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f02ec075070 con 0x7f02d40051c0 2024-04-25T10:05:04.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.450+0000 7f02f0db0640 1 -- 172.21.15.16:0/1388477007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02ec0749b0 con 0x7f02d40051c0 2024-04-25T10:05:04.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.451+0000 7f02daffd640 1 -- 172.21.15.16:0/1388477007 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12884 (secure 0 0 0) 0x7f02e4025020 con 0x7f02d40051c0 2024-04-25T10:05:04.457 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:04.458 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":18,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:00.401566+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.453+0000 7f02f0db0640 1 -- 172.21.15.16:0/1388477007 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f02cc06b500 msgr2=0x7f02cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.453+0000 7f02f0db0640 1 --2- 172.21.15.16:0/1388477007 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f02cc06b500 0x7f02cc06d9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f02ec0cd4f0 tx=0x7f02e0009040 comp rx=0 tx=0).stop 2024-04-25T10:05:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.453+0000 7f02f0db0640 1 -- 172.21.15.16:0/1388477007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d40051c0 msgr2=0x7f02ec15a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.453+0000 7f02f0db0640 1 --2- 172.21.15.16:0/1388477007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d40051c0 0x7f02ec15a100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f02e4009fa0 tx=0x7f02e40105f0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.453+0000 7f02f0db0640 1 -- 172.21.15.16:0/1388477007 shutdown_connections 2024-04-25T10:05:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.453+0000 7f02f0db0640 1 --2- 172.21.15.16:0/1388477007 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f02cc06b500 0x7f02cc06d9c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.453+0000 7f02f0db0640 1 --2- 172.21.15.16:0/1388477007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02ec151260 0x7f02ec0b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.453+0000 7f02f0db0640 1 --2- 172.21.15.16:0/1388477007 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02ec160e30 0x7f02ec15f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.453+0000 7f02f0db0640 1 --2- 172.21.15.16:0/1388477007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d40051c0 0x7f02ec15a100 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.453+0000 7f02f0db0640 1 -- 172.21.15.16:0/1388477007 >> 172.21.15.16:0/1388477007 conn(0x7f02ec072900 msgr2=0x7f02ec0d1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.453+0000 7f02f0db0640 1 -- 172.21.15.16:0/1388477007 shutdown_connections 2024-04-25T10:05:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.453+0000 7f02f0db0640 1 -- 172.21.15.16:0/1388477007 wait complete. 2024-04-25T10:05:04.470 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:04.470 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.568+0000 7fc7d7d8b640 1 Processor -- start 2024-04-25T10:05:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.568+0000 7fc7d7d8b640 1 -- start start 2024-04-25T10:05:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.569+0000 7fc7d7d8b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc7d015dc80 0x7fc7d0161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.569+0000 7fc7d7d8b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc7d00b97c0 0x7fc7d00b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.569+0000 7fc7d7d8b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7d00b5a30 0x7fc7d006bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.569+0000 7fc7d7d8b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d00b5e60 con 0x7fc7d015dc80 2024-04-25T10:05:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.569+0000 7fc7d7d8b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d01627a0 con 0x7fc7d00b97c0 2024-04-25T10:05:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.569+0000 7fc7d7d8b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d01621e0 con 0x7fc7d00b5a30 2024-04-25T10:05:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.569+0000 7fc7d5b27640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc7d015dc80 0x7fc7d0161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.569+0000 7fc7d5b27640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc7d015dc80 0x7fc7d0161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57020/0 (socket says 172.21.15.16:57020) 2024-04-25T10:05:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.569+0000 7fc7d5b27640 1 -- 172.21.15.16:0/3365416065 learned_addr learned my addr 172.21.15.16:0/3365416065 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.569+0000 7fc7d6328640 1 --2- 172.21.15.16:0/3365416065 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7d00b5a30 0x7fc7d006bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.569+0000 7fc7d5326640 1 --2- 172.21.15.16:0/3365416065 >> v2:172.21.15.186:3300/0 conn(0x7fc7d00b97c0 0x7fc7d00b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.569+0000 7fc7d5b27640 1 -- 172.21.15.16:0/3365416065 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7d00b5a30 msgr2=0x7fc7d006bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.569+0000 7fc7d5b27640 1 --2- 172.21.15.16:0/3365416065 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7d00b5a30 0x7fc7d006bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.569+0000 7fc7d5b27640 1 -- 172.21.15.16:0/3365416065 >> v2:172.21.15.186:3300/0 conn(0x7fc7d00b97c0 msgr2=0x7fc7d00b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.569+0000 7fc7d5b27640 1 --2- 172.21.15.16:0/3365416065 >> v2:172.21.15.186:3300/0 conn(0x7fc7d00b97c0 0x7fc7d00b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.569+0000 7fc7d5b27640 1 -- 172.21.15.16:0/3365416065 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d01624c0 con 0x7fc7d015dc80 2024-04-25T10:05:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.570+0000 7fc7d5b27640 1 --2- 172.21.15.16:0/3365416065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d015dc80 0x7fc7d0161870 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc7d006ba70 tx=0x7fc7cc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa368dbd1e8545a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.570+0000 7fc7d4b25640 1 -- 172.21.15.16:0/3365416065 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc037070 con 0x7fc7d015dc80 2024-04-25T10:05:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.570+0000 7fc7d4b25640 1 -- 172.21.15.16:0/3365416065 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc032440 con 0x7fc7d015dc80 2024-04-25T10:05:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.570+0000 7fc7d4b25640 1 -- 172.21.15.16:0/3365416065 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc032810 con 0x7fc7d015dc80 2024-04-25T10:05:04.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.571+0000 7fc7d7d8b640 1 -- 172.21.15.16:0/3365416065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d015dc80 msgr2=0x7fc7d0161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.571+0000 7fc7d7d8b640 1 --2- 172.21.15.16:0/3365416065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d015dc80 0x7fc7d0161870 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc7d006ba70 tx=0x7fc7cc02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.571+0000 7fc7d7d8b640 1 -- 172.21.15.16:0/3365416065 shutdown_connections 2024-04-25T10:05:04.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.571+0000 7fc7d7d8b640 1 --2- 172.21.15.16:0/3365416065 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7d00b5a30 0x7fc7d006bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.571+0000 7fc7d7d8b640 1 --2- 172.21.15.16:0/3365416065 >> v2:172.21.15.186:3300/0 conn(0x7fc7d00b97c0 0x7fc7d00b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.571+0000 7fc7d7d8b640 1 --2- 172.21.15.16:0/3365416065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d015dc80 0x7fc7d0161870 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.571+0000 7fc7d7d8b640 1 -- 172.21.15.16:0/3365416065 >> 172.21.15.16:0/3365416065 conn(0x7fc7d00d04f0 msgr2=0x7fc7d00d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.571+0000 7fc7d7d8b640 1 -- 172.21.15.16:0/3365416065 shutdown_connections 2024-04-25T10:05:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.571+0000 7fc7d7d8b640 1 -- 172.21.15.16:0/3365416065 wait complete. 2024-04-25T10:05:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.572+0000 7fc7d7d8b640 1 Processor -- start 2024-04-25T10:05:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.572+0000 7fc7d7d8b640 1 -- start start 2024-04-25T10:05:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d7d8b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7d00b5a30 0x7fc7d015f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d7d8b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7d00b97c0 0x7fc7d01f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d7d8b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d015dc80 0x7fc7d01f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d7d8b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d0161db0 con 0x7fc7d015dc80 2024-04-25T10:05:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d7d8b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d015e060 con 0x7fc7d00b97c0 2024-04-25T10:05:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d7d8b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d0161f30 con 0x7fc7d00b5a30 2024-04-25T10:05:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d6328640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d015dc80 0x7fc7d01f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d6328640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d015dc80 0x7fc7d01f2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57030/0 (socket says 172.21.15.16:57030) 2024-04-25T10:05:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d6328640 1 -- 172.21.15.16:0/3871928790 learned_addr learned my addr 172.21.15.16:0/3871928790 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d5b27640 1 --2- 172.21.15.16:0/3871928790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7d00b5a30 0x7fc7d015f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d5326640 1 --2- 172.21.15.16:0/3871928790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7d00b97c0 0x7fc7d01f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d6328640 1 -- 172.21.15.16:0/3871928790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7d00b5a30 msgr2=0x7fc7d015f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d6328640 1 --2- 172.21.15.16:0/3871928790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7d00b5a30 0x7fc7d015f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d6328640 1 -- 172.21.15.16:0/3871928790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7d00b97c0 msgr2=0x7fc7d01f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d6328640 1 --2- 172.21.15.16:0/3871928790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7d00b97c0 0x7fc7d01f76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d6328640 1 -- 172.21.15.16:0/3871928790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d01f3510 con 0x7fc7d015dc80 2024-04-25T10:05:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d5b27640 1 --2- 172.21.15.16:0/3871928790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7d00b5a30 0x7fc7d015f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d6328640 1 --2- 172.21.15.16:0/3871928790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d015dc80 0x7fc7d01f2fd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8007e60 tx=0x7fc7c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7c67fc640 1 -- 172.21.15.16:0/3871928790 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8014620 con 0x7fc7d015dc80 2024-04-25T10:05:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7c67fc640 1 -- 172.21.15.16:0/3871928790 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c8019070 con 0x7fc7d015dc80 2024-04-25T10:05:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7d7d8b640 1 -- 172.21.15.16:0/3871928790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d015fed0 con 0x7fc7d015dc80 2024-04-25T10:05:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.573+0000 7fc7c67fc640 1 -- 172.21.15.16:0/3871928790 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8014a20 con 0x7fc7d015dc80 2024-04-25T10:05:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.574+0000 7fc7d7d8b640 1 -- 172.21.15.16:0/3871928790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d0160490 con 0x7fc7d015dc80 2024-04-25T10:05:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.574+0000 7fc7d7d8b640 1 -- 172.21.15.16:0/3871928790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d00750b0 con 0x7fc7d015dc80 2024-04-25T10:05:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.575+0000 7fc7c67fc640 1 -- 172.21.15.16:0/3871928790 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7c8014bc0 con 0x7fc7d015dc80 2024-04-25T10:05:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.575+0000 7fc7c67fc640 1 --2- 172.21.15.16:0/3871928790 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc7b406b4d0 0x7fc7b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.575+0000 7fc7c67fc640 1 -- 172.21.15.16:0/3871928790 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc7c808a7c0 con 0x7fc7d015dc80 2024-04-25T10:05:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.575+0000 7fc7d5b27640 1 --2- 172.21.15.16:0/3871928790 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc7b406b4d0 0x7fc7b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.576+0000 7fc7d5b27640 1 --2- 172.21.15.16:0/3871928790 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc7b406b4d0 0x7fc7b406d990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc034000 tx=0x7fc7cc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.579+0000 7fc7c67fc640 1 -- 172.21.15.16:0/3871928790 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7d01f3510 con 0x7fc7d015dc80 2024-04-25T10:05:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.757+0000 7fc7d7d8b640 1 -- 172.21.15.16:0/3871928790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7d00d1780 con 0x7fc7d015dc80 2024-04-25T10:05:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.758+0000 7fc7c67fc640 1 -- 172.21.15.16:0/3871928790 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc7c80598f0 con 0x7fc7d015dc80 2024-04-25T10:05:04.764 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:05:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.759+0000 7fc7d7d8b640 1 -- 172.21.15.16:0/3871928790 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc7b406b4d0 msgr2=0x7fc7b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.759+0000 7fc7d7d8b640 1 --2- 172.21.15.16:0/3871928790 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc7b406b4d0 0x7fc7b406d990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc034000 tx=0x7fc7cc02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.759+0000 7fc7d7d8b640 1 -- 172.21.15.16:0/3871928790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d015dc80 msgr2=0x7fc7d01f2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.759+0000 7fc7d7d8b640 1 --2- 172.21.15.16:0/3871928790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d015dc80 0x7fc7d01f2fd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8007e60 tx=0x7fc7c800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.760+0000 7fc7d7d8b640 1 -- 172.21.15.16:0/3871928790 shutdown_connections 2024-04-25T10:05:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.760+0000 7fc7d7d8b640 1 --2- 172.21.15.16:0/3871928790 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc7b406b4d0 0x7fc7b406d990 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.760+0000 7fc7d7d8b640 1 --2- 172.21.15.16:0/3871928790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d015dc80 0x7fc7d01f2fd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.760+0000 7fc7d7d8b640 1 --2- 172.21.15.16:0/3871928790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7d00b97c0 0x7fc7d01f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.760+0000 7fc7d7d8b640 1 --2- 172.21.15.16:0/3871928790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7d00b5a30 0x7fc7d015f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.760+0000 7fc7d7d8b640 1 -- 172.21.15.16:0/3871928790 >> 172.21.15.16:0/3871928790 conn(0x7fc7d00d04f0 msgr2=0x7fc7d00b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:04.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.760+0000 7fc7d7d8b640 1 -- 172.21.15.16:0/3871928790 shutdown_connections 2024-04-25T10:05:04.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.760+0000 7fc7d7d8b640 1 -- 172.21.15.16:0/3871928790 wait complete. 2024-04-25T10:05:04.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:04.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.870+0000 7f00bb288640 1 Processor -- start 2024-04-25T10:05:04.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.870+0000 7f00bb288640 1 -- start start 2024-04-25T10:05:04.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.871+0000 7f00bb288640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00b40d35b0 0x7f00b40b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.871+0000 7f00bb288640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f00b40b5990 0x7f00b40b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.871+0000 7f00bb288640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b415e0c0 con 0x7f00b415dce0 2024-04-25T10:05:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.871+0000 7f00bb288640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b40b5de0 con 0x7f00b40b5990 2024-04-25T10:05:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.871+0000 7f00bb288640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b40d2510 con 0x7f00b40d35b0 2024-04-25T10:05:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.871+0000 7f00b9024640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f00b415dce0 0x7f00b40d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43206/0 (socket says 172.21.15.16:43206) 2024-04-25T10:05:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.871+0000 7f00b9825640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f00b40b5990 0x7f00b40b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.871+0000 7f00b9024640 1 -- 172.21.15.16:0/3223243894 learned_addr learned my addr 172.21.15.16:0/3223243894 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.871+0000 7f00b8823640 1 --2- 172.21.15.16:0/3223243894 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00b40d35b0 0x7f00b40b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.871+0000 7f00b3fff640 1 -- 172.21.15.16:0/3223243894 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920050680 0 0) 0x7f00b415e0c0 con 0x7f00b415dce0 2024-04-25T10:05:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.871+0000 7f00b3fff640 1 -- 172.21.15.16:0/3223243894 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f009c003520 con 0x7f00b415dce0 2024-04-25T10:05:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.872+0000 7f00b9825640 1 -- 172.21.15.16:0/3223243894 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00b40d35b0 msgr2=0x7f00b40b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.872+0000 7f00b9825640 1 --2- 172.21.15.16:0/3223243894 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00b40d35b0 0x7f00b40b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.872+0000 7f00b9825640 1 -- 172.21.15.16:0/3223243894 >> v1:172.21.15.16:6789/0 conn(0x7f00b415dce0 legacy=0x7f00b40d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.872+0000 7f00b9825640 1 -- 172.21.15.16:0/3223243894 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b40d27f0 con 0x7f00b40b5990 2024-04-25T10:05:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.872+0000 7f00b9825640 1 --2- 172.21.15.16:0/3223243894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b40b5990 0x7f00b40b6b00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0097a0 tx=0x7f00ac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32053bc023173c1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.872+0000 7f00b3fff640 1 -- 172.21.15.16:0/3223243894 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac037070 con 0x7f00b40b5990 2024-04-25T10:05:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.872+0000 7f00b3fff640 1 -- 172.21.15.16:0/3223243894 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac032440 con 0x7f00b40b5990 2024-04-25T10:05:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.872+0000 7f00b3fff640 1 -- 172.21.15.16:0/3223243894 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac032810 con 0x7f00b40b5990 2024-04-25T10:05:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.873+0000 7f00bb288640 1 -- 172.21.15.16:0/3223243894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b40b5990 msgr2=0x7f00b40b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.873+0000 7f00bb288640 1 --2- 172.21.15.16:0/3223243894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b40b5990 0x7f00b40b6b00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0097a0 tx=0x7f00ac02a180 comp rx=0 tx=0).stop 2024-04-25T10:05:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.873+0000 7f00bb288640 1 -- 172.21.15.16:0/3223243894 shutdown_connections 2024-04-25T10:05:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.873+0000 7f00bb288640 1 --2- 172.21.15.16:0/3223243894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b40b5990 0x7f00b40b6b00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.873+0000 7f00bb288640 1 --2- 172.21.15.16:0/3223243894 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00b40d35b0 0x7f00b40b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.873+0000 7f00bb288640 1 -- 172.21.15.16:0/3223243894 >> 172.21.15.16:0/3223243894 conn(0x7f00b4072900 msgr2=0x7f00b40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.873+0000 7f00bb288640 1 -- 172.21.15.16:0/3223243894 shutdown_connections 2024-04-25T10:05:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.873+0000 7f00bb288640 1 -- 172.21.15.16:0/3223243894 wait complete. 2024-04-25T10:05:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.874+0000 7f00bb288640 1 Processor -- start 2024-04-25T10:05:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.874+0000 7f00bb288640 1 -- start start 2024-04-25T10:05:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.875+0000 7f00bb288640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00b40b5990 0x7f00b40caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.875+0000 7f00bb288640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00b40d35b0 0x7f00b4157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.875+0000 7f00bb288640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b415dce0 0x7f00b40c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.875+0000 7f00bb288640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b40d3090 con 0x7f00b415dce0 2024-04-25T10:05:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.875+0000 7f00bb288640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b40d2db0 con 0x7f00b40b5990 2024-04-25T10:05:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.875+0000 7f00bb288640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b40d2160 con 0x7f00b40d35b0 2024-04-25T10:05:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.875+0000 7f00b9825640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b415dce0 0x7f00b40c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.875+0000 7f00b9825640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b415dce0 0x7f00b40c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57042/0 (socket says 172.21.15.16:57042) 2024-04-25T10:05:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.875+0000 7f00b9825640 1 -- 172.21.15.16:0/1562047361 learned_addr learned my addr 172.21.15.16:0/1562047361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.875+0000 7f00b8823640 1 --2- 172.21.15.16:0/1562047361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00b40d35b0 0x7f00b4157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.875+0000 7f00b9024640 1 --2- 172.21.15.16:0/1562047361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00b40b5990 0x7f00b40caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.875+0000 7f00b9825640 1 -- 172.21.15.16:0/1562047361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00b40d35b0 msgr2=0x7f00b4157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.875+0000 7f00b9825640 1 --2- 172.21.15.16:0/1562047361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00b40d35b0 0x7f00b4157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.875+0000 7f00b9825640 1 -- 172.21.15.16:0/1562047361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00b40b5990 msgr2=0x7f00b40caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.875+0000 7f00b9825640 1 --2- 172.21.15.16:0/1562047361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00b40b5990 0x7f00b40caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.875+0000 7f00b9825640 1 -- 172.21.15.16:0/1562047361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b41f71f0 con 0x7f00b415dce0 2024-04-25T10:05:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.875+0000 7f00b8823640 1 --2- 172.21.15.16:0/1562047361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00b40d35b0 0x7f00b4157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.876+0000 7f00b9825640 1 --2- 172.21.15.16:0/1562047361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b415dce0 0x7f00b40c89d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f00ac035040 tx=0x7f00ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.876+0000 7f00b1ffb640 1 -- 172.21.15.16:0/1562047361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac041020 con 0x7f00b415dce0 2024-04-25T10:05:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.876+0000 7f00b1ffb640 1 -- 172.21.15.16:0/1562047361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac03d640 con 0x7f00b415dce0 2024-04-25T10:05:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.876+0000 7f00bb288640 1 -- 172.21.15.16:0/1562047361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b41f74d0 con 0x7f00b415dce0 2024-04-25T10:05:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.876+0000 7f00b1ffb640 1 -- 172.21.15.16:0/1562047361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac037040 con 0x7f00b415dce0 2024-04-25T10:05:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.876+0000 7f00bb288640 1 -- 172.21.15.16:0/1562047361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b41f7a90 con 0x7f00b415dce0 2024-04-25T10:05:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.877+0000 7f00bb288640 1 -- 172.21.15.16:0/1562047361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b40749b0 con 0x7f00b415dce0 2024-04-25T10:05:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.877+0000 7f00b1ffb640 1 -- 172.21.15.16:0/1562047361 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f00ac044020 con 0x7f00b415dce0 2024-04-25T10:05:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.878+0000 7f00b1ffb640 1 --2- 172.21.15.16:0/1562047361 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f00a006b4d0 0x7f00a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.878+0000 7f00b1ffb640 1 -- 172.21.15.16:0/1562047361 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f00ac0aa5e0 con 0x7f00b415dce0 2024-04-25T10:05:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.878+0000 7f00b9024640 1 --2- 172.21.15.16:0/1562047361 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f00a006b4d0 0x7f00a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.878+0000 7f00b9024640 1 --2- 172.21.15.16:0/1562047361 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f00a006b4d0 0x7f00a006d990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f00a4005c00 tx=0x7f00a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:04.882+0000 7f00b1ffb640 1 -- 172.21.15.16:0/1562047361 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f00ac04b030 con 0x7f00b415dce0 2024-04-25T10:05:05.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:05.097+0000 7f00bb288640 1 -- 172.21.15.16:0/1562047361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00b415d010 con 0x7f00b415dce0 2024-04-25T10:05:05.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:05.098+0000 7f00b1ffb640 1 -- 172.21.15.16:0/1562047361 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f00ac0797c0 con 0x7f00b415dce0 2024-04-25T10:05:05.104 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:05.104 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1965168813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1965168813},{"type":"v1","addr":"172.21.15.16:6834","nonce":1965168813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:05:05.104 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2024-04-25T10:05:05.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:05.100+0000 7f00bb288640 1 -- 172.21.15.16:0/1562047361 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f00a006b4d0 msgr2=0x7f00a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:05.100+0000 7f00bb288640 1 --2- 172.21.15.16:0/1562047361 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f00a006b4d0 0x7f00a006d990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f00a4005c00 tx=0x7f00a4009040 comp rx=0 tx=0).stop 2024-04-25T10:05:05.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:05.100+0000 7f00bb288640 1 -- 172.21.15.16:0/1562047361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b415dce0 msgr2=0x7f00b40c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:05.100+0000 7f00bb288640 1 --2- 172.21.15.16:0/1562047361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b415dce0 0x7f00b40c89d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f00ac035040 tx=0x7f00ac0046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:05.100+0000 7f00bb288640 1 -- 172.21.15.16:0/1562047361 shutdown_connections 2024-04-25T10:05:05.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:05.100+0000 7f00bb288640 1 --2- 172.21.15.16:0/1562047361 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f00a006b4d0 0x7f00a006d990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:05.100+0000 7f00bb288640 1 --2- 172.21.15.16:0/1562047361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00b415dce0 0x7f00b40c89d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:05.100+0000 7f00bb288640 1 --2- 172.21.15.16:0/1562047361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00b40d35b0 0x7f00b4157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:05.100+0000 7f00bb288640 1 --2- 172.21.15.16:0/1562047361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00b40b5990 0x7f00b40caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:05.100+0000 7f00bb288640 1 -- 172.21.15.16:0/1562047361 >> 172.21.15.16:0/1562047361 conn(0x7f00b4072900 msgr2=0x7f00b40d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:05.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:05.100+0000 7f00bb288640 1 -- 172.21.15.16:0/1562047361 shutdown_connections 2024-04-25T10:05:05.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:05.100+0000 7f00bb288640 1 -- 172.21.15.16:0/1562047361 wait complete. 2024-04-25T10:05:06.618 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:06.621 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:07.117 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.225+0000 7f378006a640 1 Processor -- start 2024-04-25T10:05:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.225+0000 7f378006a640 1 -- start start 2024-04-25T10:05:07.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.226+0000 7f378006a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f377815dce0 0x7f37780d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.226+0000 7f378006a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37780d35b0 0x7f37780b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.226+0000 7f378006a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377815e0c0 con 0x7f37780d1f90 2024-04-25T10:05:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.226+0000 7f378006a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37780b7b10 con 0x7f377815dce0 2024-04-25T10:05:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.226+0000 7f378006a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37780d2510 con 0x7f37780d35b0 2024-04-25T10:05:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.226+0000 7f377de06640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f377815dce0 0x7f37780d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.226+0000 7f377de06640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f377815dce0 0x7f37780d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:43998/0 (socket says 172.21.15.16:43998) 2024-04-25T10:05:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.226+0000 7f377de06640 1 -- 172.21.15.16:0/3735512563 learned_addr learned my addr 172.21.15.16:0/3735512563 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.226+0000 7f377d605640 1 --2- 172.21.15.16:0/3735512563 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37780d35b0 0x7f37780b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.226+0000 7f377e607640 1 --1- 172.21.15.16:0/3735512563 >> v1:172.21.15.186:6789/0 conn(0x7f37780d1f90 0x7f377806bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:45654/0 (socket says 172.21.15.16:45654) 2024-04-25T10:05:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.227+0000 7f377d605640 1 -- 172.21.15.16:0/3735512563 >> v2:172.21.15.186:3300/0 conn(0x7f377815dce0 msgr2=0x7f37780d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.227+0000 7f377d605640 1 --2- 172.21.15.16:0/3735512563 >> v2:172.21.15.186:3300/0 conn(0x7f377815dce0 0x7f37780d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.227+0000 7f377d605640 1 -- 172.21.15.16:0/3735512563 >> v1:172.21.15.186:6789/0 conn(0x7f37780d1f90 legacy=0x7f377806bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.227+0000 7f377d605640 1 -- 172.21.15.16:0/3735512563 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37780d27f0 con 0x7f37780d35b0 2024-04-25T10:05:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.227+0000 7f377de06640 1 --2- 172.21.15.16:0/3735512563 >> v2:172.21.15.186:3300/0 conn(0x7f377815dce0 0x7f37780d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.228+0000 7f377d605640 1 --2- 172.21.15.16:0/3735512563 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37780d35b0 0x7f37780b5450 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f37680097a0 tx=0x7f376802a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=846a26dc189422f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.228+0000 7f377ce04640 1 -- 172.21.15.16:0/3735512563 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3768037070 con 0x7f37780d35b0 2024-04-25T10:05:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.228+0000 7f377ce04640 1 -- 172.21.15.16:0/3735512563 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376802ac90 con 0x7f37780d35b0 2024-04-25T10:05:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.228+0000 7f377ce04640 1 -- 172.21.15.16:0/3735512563 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3768032660 con 0x7f37780d35b0 2024-04-25T10:05:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.228+0000 7f378006a640 1 -- 172.21.15.16:0/3735512563 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37780d35b0 msgr2=0x7f37780b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.228+0000 7f378006a640 1 --2- 172.21.15.16:0/3735512563 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37780d35b0 0x7f37780b5450 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f37680097a0 tx=0x7f376802a000 comp rx=0 tx=0).stop 2024-04-25T10:05:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.228+0000 7f378006a640 1 -- 172.21.15.16:0/3735512563 shutdown_connections 2024-04-25T10:05:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.228+0000 7f378006a640 1 --2- 172.21.15.16:0/3735512563 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37780d35b0 0x7f37780b5450 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.228+0000 7f378006a640 1 --2- 172.21.15.16:0/3735512563 >> v2:172.21.15.186:3300/0 conn(0x7f377815dce0 0x7f37780d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.228+0000 7f378006a640 1 -- 172.21.15.16:0/3735512563 >> 172.21.15.16:0/3735512563 conn(0x7f3778072900 msgr2=0x7f37780d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.229+0000 7f378006a640 1 -- 172.21.15.16:0/3735512563 shutdown_connections 2024-04-25T10:05:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.229+0000 7f378006a640 1 -- 172.21.15.16:0/3735512563 wait complete. 2024-04-25T10:05:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.230+0000 7f378006a640 1 Processor -- start 2024-04-25T10:05:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.230+0000 7f378006a640 1 -- start start 2024-04-25T10:05:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.230+0000 7f378006a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37780d1f90 0x7f3778153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.230+0000 7f378006a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37780d35b0 0x7f37781600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.230+0000 7f378006a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f377815dce0 0x7f37780b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.230+0000 7f378006a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37780d3090 con 0x7f37780d1f90 2024-04-25T10:05:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.230+0000 7f378006a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37780d2db0 con 0x7f377815dce0 2024-04-25T10:05:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.230+0000 7f378006a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37780b9820 con 0x7f37780d35b0 2024-04-25T10:05:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.231+0000 7f377de06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37780d1f90 0x7f3778153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.231+0000 7f377de06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37780d1f90 0x7f3778153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57060/0 (socket says 172.21.15.16:57060) 2024-04-25T10:05:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.231+0000 7f377de06640 1 -- 172.21.15.16:0/1517815831 learned_addr learned my addr 172.21.15.16:0/1517815831 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.231+0000 7f377d605640 1 --2- 172.21.15.16:0/1517815831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37780d35b0 0x7f37781600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.231+0000 7f377e607640 1 --2- 172.21.15.16:0/1517815831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f377815dce0 0x7f37780b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.231+0000 7f377de06640 1 -- 172.21.15.16:0/1517815831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37780d35b0 msgr2=0x7f37781600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.231+0000 7f377de06640 1 --2- 172.21.15.16:0/1517815831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37780d35b0 0x7f37781600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.231+0000 7f377de06640 1 -- 172.21.15.16:0/1517815831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f377815dce0 msgr2=0x7f37780b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.231+0000 7f377de06640 1 --2- 172.21.15.16:0/1517815831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f377815dce0 0x7f37780b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.231+0000 7f377de06640 1 -- 172.21.15.16:0/1517815831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3778158b20 con 0x7f37780d1f90 2024-04-25T10:05:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.231+0000 7f377de06640 1 --2- 172.21.15.16:0/1517815831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37780d1f90 0x7f3778153ab0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f377400e9d0 tx=0x7f377400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.231+0000 7f376e7fc640 1 -- 172.21.15.16:0/1517815831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774017070 con 0x7f37780d1f90 2024-04-25T10:05:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.231+0000 7f376e7fc640 1 -- 172.21.15.16:0/1517815831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377400ca60 con 0x7f37780d1f90 2024-04-25T10:05:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.231+0000 7f376e7fc640 1 -- 172.21.15.16:0/1517815831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377400ce10 con 0x7f37780d1f90 2024-04-25T10:05:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.231+0000 7f378006a640 1 -- 172.21.15.16:0/1517815831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3778158e00 con 0x7f37780d1f90 2024-04-25T10:05:07.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.231+0000 7f378006a640 1 -- 172.21.15.16:0/1517815831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37781593c0 con 0x7f37780d1f90 2024-04-25T10:05:07.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.232+0000 7f378006a640 1 -- 172.21.15.16:0/1517815831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37780749b0 con 0x7f37780d1f90 2024-04-25T10:05:07.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.233+0000 7f376e7fc640 1 -- 172.21.15.16:0/1517815831 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f377401a070 con 0x7f37780d1f90 2024-04-25T10:05:07.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.233+0000 7f376e7fc640 1 --2- 172.21.15.16:0/1517815831 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f375c06b4d0 0x7f375c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.233+0000 7f376e7fc640 1 -- 172.21.15.16:0/1517815831 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f377408a210 con 0x7f37780d1f90 2024-04-25T10:05:07.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.234+0000 7f377d605640 1 --2- 172.21.15.16:0/1517815831 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f375c06b4d0 0x7f375c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.234+0000 7f377d605640 1 --2- 172.21.15.16:0/1517815831 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f375c06b4d0 0x7f375c06d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3768034000 tx=0x7f376802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.238+0000 7f376e7fc640 1 -- 172.21.15.16:0/1517815831 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3774059340 con 0x7f37780d1f90 2024-04-25T10:05:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.459+0000 7f378006a640 1 -- 172.21.15.16:0/1517815831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37780b92f0 con 0x7f37780d1f90 2024-04-25T10:05:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.459+0000 7f376e7fc640 1 -- 172.21.15.16:0/1517815831 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3699 (secure 0 0 0) 0x7f3774058c40 con 0x7f37780d1f90 2024-04-25T10:05:07.466 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:07.466 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1965168813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1965168813},{"type":"v1","addr":"172.21.15.16:6834","nonce":1965168813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/437791378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":437791378},{"type":"v1","addr":"172.21.15.16:6835","nonce":437791378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34145,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3044177642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3044177642},{"type":"v1","addr":"172.21.15.186:6835","nonce":3044177642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1763853189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1763853189},{"type":"v1","addr":"172.21.15.186:6837","nonce":1763853189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-25T10:05:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 3 2024-04-25T10:05:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.462+0000 7f378006a640 1 -- 172.21.15.16:0/1517815831 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f375c06b4d0 msgr2=0x7f375c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.462+0000 7f378006a640 1 --2- 172.21.15.16:0/1517815831 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f375c06b4d0 0x7f375c06d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3768034000 tx=0x7f376802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.462+0000 7f378006a640 1 -- 172.21.15.16:0/1517815831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37780d1f90 msgr2=0x7f3778153ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.462+0000 7f378006a640 1 --2- 172.21.15.16:0/1517815831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37780d1f90 0x7f3778153ab0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f377400e9d0 tx=0x7f377400ee90 comp rx=0 tx=0).stop 2024-04-25T10:05:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.462+0000 7f378006a640 1 -- 172.21.15.16:0/1517815831 shutdown_connections 2024-04-25T10:05:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.462+0000 7f378006a640 1 --2- 172.21.15.16:0/1517815831 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f375c06b4d0 0x7f375c06d990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.462+0000 7f378006a640 1 --2- 172.21.15.16:0/1517815831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f377815dce0 0x7f37780b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.462+0000 7f378006a640 1 --2- 172.21.15.16:0/1517815831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37780d35b0 0x7f37781600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.462+0000 7f378006a640 1 --2- 172.21.15.16:0/1517815831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37780d1f90 0x7f3778153ab0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.462+0000 7f378006a640 1 -- 172.21.15.16:0/1517815831 >> 172.21.15.16:0/1517815831 conn(0x7f3778072900 msgr2=0x7f37780d4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.462+0000 7f378006a640 1 -- 172.21.15.16:0/1517815831 shutdown_connections 2024-04-25T10:05:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.462+0000 7f378006a640 1 -- 172.21.15.16:0/1517815831 wait complete. 2024-04-25T10:05:07.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.572+0000 7ff594cfc640 1 Processor -- start 2024-04-25T10:05:07.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.572+0000 7ff594cfc640 1 -- start start 2024-04-25T10:05:07.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.573+0000 7ff594cfc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff5900b5b60 0x7ff5901687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.573+0000 7ff594cfc640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59015e0c0 con 0x7ff5900d35b0 2024-04-25T10:05:07.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.573+0000 7ff594cfc640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5900d2510 con 0x7ff59015dce0 2024-04-25T10:05:07.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.573+0000 7ff594cfc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5900b7b10 con 0x7ff5900b5b60 2024-04-25T10:05:07.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.573+0000 7ff58dd9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff5900d35b0 0x7ff5900b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43210/0 (socket says 172.21.15.16:43210) 2024-04-25T10:05:07.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.573+0000 7ff58dd9b640 1 -- 172.21.15.16:0/83075517 learned_addr learned my addr 172.21.15.16:0/83075517 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:07.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.573+0000 7ff58ed9d640 1 --2- 172.21.15.16:0/83075517 >> v2:172.21.15.16:3300/0 conn(0x7ff5900b5b60 0x7ff5901687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.574+0000 7ff58d59a640 1 -- 172.21.15.16:0/83075517 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300745461 0 0) 0x7ff59015e0c0 con 0x7ff5900d35b0 2024-04-25T10:05:07.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.574+0000 7ff58d59a640 1 -- 172.21.15.16:0/83075517 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff578003520 con 0x7ff5900d35b0 2024-04-25T10:05:07.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.574+0000 7ff58ed9d640 1 -- 172.21.15.16:0/83075517 >> v1:172.21.15.186:6789/0 conn(0x7ff59015dce0 legacy=0x7ff5900d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.574+0000 7ff58ed9d640 1 -- 172.21.15.16:0/83075517 >> v1:172.21.15.16:6789/0 conn(0x7ff5900d35b0 legacy=0x7ff5900b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.574+0000 7ff58ed9d640 1 -- 172.21.15.16:0/83075517 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5900d27f0 con 0x7ff5900b5b60 2024-04-25T10:05:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.574+0000 7ff58ed9d640 1 --2- 172.21.15.16:0/83075517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5900b5b60 0x7ff5901687f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff580009d90 tx=0x7ff58002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167bb1574b816282 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.574+0000 7ff58d59a640 1 -- 172.21.15.16:0/83075517 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580007740 con 0x7ff5900b5b60 2024-04-25T10:05:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.575+0000 7ff58d59a640 1 -- 172.21.15.16:0/83075517 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5800078e0 con 0x7ff5900b5b60 2024-04-25T10:05:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.575+0000 7ff58d59a640 1 -- 172.21.15.16:0/83075517 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580007cb0 con 0x7ff5900b5b60 2024-04-25T10:05:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.575+0000 7ff594cfc640 1 -- 172.21.15.16:0/83075517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5900b5b60 msgr2=0x7ff5901687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.575+0000 7ff594cfc640 1 --2- 172.21.15.16:0/83075517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5900b5b60 0x7ff5901687f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff580009d90 tx=0x7ff58002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.575+0000 7ff594cfc640 1 -- 172.21.15.16:0/83075517 shutdown_connections 2024-04-25T10:05:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.575+0000 7ff594cfc640 1 --2- 172.21.15.16:0/83075517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5900b5b60 0x7ff5901687f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.575+0000 7ff594cfc640 1 -- 172.21.15.16:0/83075517 >> 172.21.15.16:0/83075517 conn(0x7ff590072900 msgr2=0x7ff5900d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.575+0000 7ff594cfc640 1 -- 172.21.15.16:0/83075517 shutdown_connections 2024-04-25T10:05:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.575+0000 7ff594cfc640 1 -- 172.21.15.16:0/83075517 wait complete. 2024-04-25T10:05:07.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.576+0000 7ff594cfc640 1 Processor -- start 2024-04-25T10:05:07.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.577+0000 7ff594cfc640 1 -- start start 2024-04-25T10:05:07.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.577+0000 7ff594cfc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5900b5b60 0x7ff59014e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.577+0000 7ff594cfc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5900d35b0 0x7ff5900cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.577+0000 7ff594cfc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff59015dce0 0x7ff5901f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.577+0000 7ff594cfc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5900d3090 con 0x7ff5900d35b0 2024-04-25T10:05:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.577+0000 7ff594cfc640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5900d2db0 con 0x7ff59015dce0 2024-04-25T10:05:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.577+0000 7ff594cfc640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5900d2160 con 0x7ff5900b5b60 2024-04-25T10:05:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.577+0000 7ff58dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5900d35b0 0x7ff5900cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.577+0000 7ff58dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5900d35b0 0x7ff5900cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57074/0 (socket says 172.21.15.16:57074) 2024-04-25T10:05:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.577+0000 7ff58dd9b640 1 -- 172.21.15.16:0/500902160 learned_addr learned my addr 172.21.15.16:0/500902160 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.578+0000 7ff58e59c640 1 --2- 172.21.15.16:0/500902160 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5900b5b60 0x7ff59014e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.578+0000 7ff58ed9d640 1 --2- 172.21.15.16:0/500902160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff59015dce0 0x7ff5901f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.578+0000 7ff58dd9b640 1 -- 172.21.15.16:0/500902160 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5900b5b60 msgr2=0x7ff59014e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.578+0000 7ff58dd9b640 1 --2- 172.21.15.16:0/500902160 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5900b5b60 0x7ff59014e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.578+0000 7ff58dd9b640 1 -- 172.21.15.16:0/500902160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff59015dce0 msgr2=0x7ff5901f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.578+0000 7ff58dd9b640 1 --2- 172.21.15.16:0/500902160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff59015dce0 0x7ff5901f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.578+0000 7ff58dd9b640 1 -- 172.21.15.16:0/500902160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59014ed20 con 0x7ff5900d35b0 2024-04-25T10:05:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.578+0000 7ff58ed9d640 1 --2- 172.21.15.16:0/500902160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff59015dce0 0x7ff5901f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.578+0000 7ff58dd9b640 1 --2- 172.21.15.16:0/500902160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5900d35b0 0x7ff5900cc480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff58400c500 tx=0x7ff58400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.578+0000 7ff57effd640 1 -- 172.21.15.16:0/500902160 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584017070 con 0x7ff5900d35b0 2024-04-25T10:05:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.578+0000 7ff57effd640 1 -- 172.21.15.16:0/500902160 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff584005ce0 con 0x7ff5900d35b0 2024-04-25T10:05:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.578+0000 7ff594cfc640 1 -- 172.21.15.16:0/500902160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5900caf80 con 0x7ff5900d35b0 2024-04-25T10:05:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.578+0000 7ff57effd640 1 -- 172.21.15.16:0/500902160 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5840072f0 con 0x7ff5900d35b0 2024-04-25T10:05:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.578+0000 7ff594cfc640 1 -- 172.21.15.16:0/500902160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5900cb540 con 0x7ff5900d35b0 2024-04-25T10:05:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.579+0000 7ff594cfc640 1 -- 172.21.15.16:0/500902160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5900749b0 con 0x7ff5900d35b0 2024-04-25T10:05:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.580+0000 7ff57effd640 1 -- 172.21.15.16:0/500902160 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff58401a070 con 0x7ff5900d35b0 2024-04-25T10:05:07.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.580+0000 7ff57effd640 1 --2- 172.21.15.16:0/500902160 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff57006b500 0x7ff57006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.580+0000 7ff57effd640 1 -- 172.21.15.16:0/500902160 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff5840891e0 con 0x7ff5900d35b0 2024-04-25T10:05:07.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.580+0000 7ff58e59c640 1 --2- 172.21.15.16:0/500902160 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff57006b500 0x7ff57006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.581+0000 7ff58e59c640 1 --2- 172.21.15.16:0/500902160 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff57006b500 0x7ff57006d9c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff588007fd0 tx=0x7ff588009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.584+0000 7ff57effd640 1 -- 172.21.15.16:0/500902160 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff584057d50 con 0x7ff5900d35b0 2024-04-25T10:05:07.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.765+0000 7ff594cfc640 1 -- 172.21.15.16:0/500902160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff59006b080 con 0x7ff5900d35b0 2024-04-25T10:05:07.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.766+0000 7ff57effd640 1 -- 172.21.15.16:0/500902160 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12884 (secure 0 0 0) 0x7ff58405bdf0 con 0x7ff5900d35b0 2024-04-25T10:05:07.772 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:07.772 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":18,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:00.401566+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.768+0000 7ff594cfc640 1 -- 172.21.15.16:0/500902160 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff57006b500 msgr2=0x7ff57006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.768+0000 7ff594cfc640 1 --2- 172.21.15.16:0/500902160 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff57006b500 0x7ff57006d9c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff588007fd0 tx=0x7ff588009040 comp rx=0 tx=0).stop 2024-04-25T10:05:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.768+0000 7ff594cfc640 1 -- 172.21.15.16:0/500902160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5900d35b0 msgr2=0x7ff5900cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.768+0000 7ff594cfc640 1 --2- 172.21.15.16:0/500902160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5900d35b0 0x7ff5900cc480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff58400c500 tx=0x7ff58400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.768+0000 7ff594cfc640 1 -- 172.21.15.16:0/500902160 shutdown_connections 2024-04-25T10:05:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.768+0000 7ff594cfc640 1 --2- 172.21.15.16:0/500902160 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff57006b500 0x7ff57006d9c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.768+0000 7ff594cfc640 1 --2- 172.21.15.16:0/500902160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff59015dce0 0x7ff5901f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.768+0000 7ff594cfc640 1 --2- 172.21.15.16:0/500902160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5900d35b0 0x7ff5900cc480 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.768+0000 7ff594cfc640 1 --2- 172.21.15.16:0/500902160 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5900b5b60 0x7ff59014e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.768+0000 7ff594cfc640 1 -- 172.21.15.16:0/500902160 >> 172.21.15.16:0/500902160 conn(0x7ff590072900 msgr2=0x7ff5900ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.768+0000 7ff594cfc640 1 -- 172.21.15.16:0/500902160 shutdown_connections 2024-04-25T10:05:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.768+0000 7ff594cfc640 1 -- 172.21.15.16:0/500902160 wait complete. 2024-04-25T10:05:07.785 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:07.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:07.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.883+0000 7f8930330640 1 Processor -- start 2024-04-25T10:05:07.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.883+0000 7f8930330640 1 -- start start 2024-04-25T10:05:07.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.884+0000 7f8930330640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f89280b97c0 0x7f89280b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.884+0000 7f8930330640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89280b5e40 con 0x7f892815dc80 2024-04-25T10:05:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.884+0000 7f8930330640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89281627a0 con 0x7f89280b5a30 2024-04-25T10:05:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.884+0000 7f8930330640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8928161f80 con 0x7f89280b97c0 2024-04-25T10:05:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.884+0000 7f892d8cb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f89280b97c0 0x7f89280b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.884+0000 7f892e0cc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f892815dc80 0x7f8928161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43212/0 (socket says 172.21.15.16:43212) 2024-04-25T10:05:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.884+0000 7f892e0cc640 1 -- 172.21.15.16:0/3312206055 learned_addr learned my addr 172.21.15.16:0/3312206055 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.884+0000 7f892d8cb640 1 -- 172.21.15.16:0/3312206055 >> v1:172.21.15.186:6789/0 conn(0x7f89280b5a30 legacy=0x7f89280d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.884+0000 7f892d8cb640 1 -- 172.21.15.16:0/3312206055 >> v1:172.21.15.16:6789/0 conn(0x7f892815dc80 legacy=0x7f8928161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.884+0000 7f892d0ca640 1 -- 172.21.15.16:0/3312206055 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724232143 0 0) 0x7f8924001110 con 0x7f892815dc80 2024-04-25T10:05:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.884+0000 7f892d8cb640 1 -- 172.21.15.16:0/3312206055 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89281621e0 con 0x7f89280b97c0 2024-04-25T10:05:07.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.885+0000 7f892d8cb640 1 --2- 172.21.15.16:0/3312206055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89280b97c0 0x7f89280b54f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89180097a0 tx=0x7f891802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca52b37fcdc58160 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.885+0000 7f892d0ca640 1 -- 172.21.15.16:0/3312206055 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8918037070 con 0x7f89280b97c0 2024-04-25T10:05:07.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.885+0000 7f892d0ca640 1 -- 172.21.15.16:0/3312206055 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8918032440 con 0x7f89280b97c0 2024-04-25T10:05:07.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.885+0000 7f892d0ca640 1 -- 172.21.15.16:0/3312206055 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8918032810 con 0x7f89280b97c0 2024-04-25T10:05:07.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.885+0000 7f8930330640 1 -- 172.21.15.16:0/3312206055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89280b97c0 msgr2=0x7f89280b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.885+0000 7f8930330640 1 --2- 172.21.15.16:0/3312206055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89280b97c0 0x7f89280b54f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89180097a0 tx=0x7f891802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.885+0000 7f8930330640 1 -- 172.21.15.16:0/3312206055 shutdown_connections 2024-04-25T10:05:07.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.885+0000 7f8930330640 1 --2- 172.21.15.16:0/3312206055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89280b97c0 0x7f89280b54f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.885+0000 7f8930330640 1 -- 172.21.15.16:0/3312206055 >> 172.21.15.16:0/3312206055 conn(0x7f89280d04f0 msgr2=0x7f89280d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:07.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.885+0000 7f8930330640 1 -- 172.21.15.16:0/3312206055 shutdown_connections 2024-04-25T10:05:07.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.885+0000 7f8930330640 1 -- 172.21.15.16:0/3312206055 wait complete. 2024-04-25T10:05:07.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.887+0000 7f8930330640 1 Processor -- start 2024-04-25T10:05:07.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.887+0000 7f8930330640 1 -- start start 2024-04-25T10:05:07.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.887+0000 7f8930330640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89280b5a30 0x7f892815be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.887+0000 7f8930330640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89280b97c0 0x7f8928160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.887+0000 7f8930330640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f892815dc80 0x7f892815f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.887+0000 7f8930330640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892815e060 con 0x7f89280b97c0 2024-04-25T10:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.887+0000 7f8930330640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89280d1c70 con 0x7f89280b5a30 2024-04-25T10:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.887+0000 7f8930330640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89280d2b80 con 0x7f892815dc80 2024-04-25T10:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.887+0000 7f892d8cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89280b97c0 0x7f8928160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.887+0000 7f892d8cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89280b97c0 0x7f8928160d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57090/0 (socket says 172.21.15.16:57090) 2024-04-25T10:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.887+0000 7f892d8cb640 1 -- 172.21.15.16:0/2003255996 learned_addr learned my addr 172.21.15.16:0/2003255996 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.888+0000 7f892e0cc640 1 --2- 172.21.15.16:0/2003255996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89280b5a30 0x7f892815be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.888+0000 7f892e8cd640 1 --2- 172.21.15.16:0/2003255996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f892815dc80 0x7f892815f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.888+0000 7f892d8cb640 1 -- 172.21.15.16:0/2003255996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f892815dc80 msgr2=0x7f892815f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.888+0000 7f892d8cb640 1 --2- 172.21.15.16:0/2003255996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f892815dc80 0x7f892815f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.888+0000 7f892d8cb640 1 -- 172.21.15.16:0/2003255996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89280b5a30 msgr2=0x7f892815be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.888+0000 7f892d8cb640 1 --2- 172.21.15.16:0/2003255996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89280b5a30 0x7f892815be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.888+0000 7f892d8cb640 1 -- 172.21.15.16:0/2003255996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892815c3a0 con 0x7f89280b97c0 2024-04-25T10:05:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.888+0000 7f892e0cc640 1 --2- 172.21.15.16:0/2003255996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89280b5a30 0x7f892815be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.888+0000 7f892e8cd640 1 --2- 172.21.15.16:0/2003255996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f892815dc80 0x7f892815f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.888+0000 7f892d8cb640 1 --2- 172.21.15.16:0/2003255996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89280b97c0 0x7f8928160d10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f89180097a0 tx=0x7f89180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.888+0000 7f891effd640 1 -- 172.21.15.16:0/2003255996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8918037040 con 0x7f89280b97c0 2024-04-25T10:05:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.888+0000 7f891effd640 1 -- 172.21.15.16:0/2003255996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8918004250 con 0x7f89280b97c0 2024-04-25T10:05:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.888+0000 7f8930330640 1 -- 172.21.15.16:0/2003255996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892815eba0 con 0x7f89280b97c0 2024-04-25T10:05:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.888+0000 7f891effd640 1 -- 172.21.15.16:0/2003255996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8918032e20 con 0x7f89280b97c0 2024-04-25T10:05:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.888+0000 7f8930330640 1 -- 172.21.15.16:0/2003255996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89281e9ed0 con 0x7f89280b97c0 2024-04-25T10:05:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.889+0000 7f8930330640 1 -- 172.21.15.16:0/2003255996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89280750b0 con 0x7f89280b97c0 2024-04-25T10:05:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.890+0000 7f891effd640 1 -- 172.21.15.16:0/2003255996 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8918030770 con 0x7f89280b97c0 2024-04-25T10:05:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.890+0000 7f891effd640 1 --2- 172.21.15.16:0/2003255996 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f890c06b4d0 0x7f890c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.890+0000 7f891effd640 1 -- 172.21.15.16:0/2003255996 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f89180ab160 con 0x7f89280b97c0 2024-04-25T10:05:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.890+0000 7f892e0cc640 1 --2- 172.21.15.16:0/2003255996 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f890c06b4d0 0x7f890c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.891+0000 7f892e0cc640 1 --2- 172.21.15.16:0/2003255996 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f890c06b4d0 0x7f890c06d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8924000f60 tx=0x7f8924009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:07.894+0000 7f891effd640 1 -- 172.21.15.16:0/2003255996 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f891807a330 con 0x7f89280b97c0 2024-04-25T10:05:08.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.073+0000 7f8930330640 1 -- 172.21.15.16:0/2003255996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f89280d1780 con 0x7f89280b97c0 2024-04-25T10:05:08.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.073+0000 7f891effd640 1 -- 172.21.15.16:0/2003255996 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8918079c30 con 0x7f89280b97c0 2024-04-25T10:05:08.080 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:05:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.075+0000 7f8930330640 1 -- 172.21.15.16:0/2003255996 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f890c06b4d0 msgr2=0x7f890c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.075+0000 7f8930330640 1 --2- 172.21.15.16:0/2003255996 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f890c06b4d0 0x7f890c06d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8924000f60 tx=0x7f8924009040 comp rx=0 tx=0).stop 2024-04-25T10:05:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.075+0000 7f8930330640 1 -- 172.21.15.16:0/2003255996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89280b97c0 msgr2=0x7f8928160d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.075+0000 7f8930330640 1 --2- 172.21.15.16:0/2003255996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89280b97c0 0x7f8928160d10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f89180097a0 tx=0x7f89180046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.075+0000 7f8930330640 1 -- 172.21.15.16:0/2003255996 shutdown_connections 2024-04-25T10:05:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.075+0000 7f8930330640 1 --2- 172.21.15.16:0/2003255996 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f890c06b4d0 0x7f890c06d990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.075+0000 7f8930330640 1 --2- 172.21.15.16:0/2003255996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f892815dc80 0x7f892815f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.075+0000 7f8930330640 1 --2- 172.21.15.16:0/2003255996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89280b97c0 0x7f8928160d10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.075+0000 7f8930330640 1 --2- 172.21.15.16:0/2003255996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89280b5a30 0x7f892815be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.075+0000 7f8930330640 1 -- 172.21.15.16:0/2003255996 >> 172.21.15.16:0/2003255996 conn(0x7f89280d04f0 msgr2=0x7f89280d2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.075+0000 7f8930330640 1 -- 172.21.15.16:0/2003255996 shutdown_connections 2024-04-25T10:05:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.075+0000 7f8930330640 1 -- 172.21.15.16:0/2003255996 wait complete. 2024-04-25T10:05:08.092 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:05:08.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.186+0000 7f0e8e29a640 1 Processor -- start 2024-04-25T10:05:08.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.186+0000 7f0e8e29a640 1 -- start start 2024-04-25T10:05:08.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.187+0000 7f0e8e29a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e8815dce0 0x7f0e880d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.187+0000 7f0e8e29a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0e880b5b60 0x7f0e880b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.187+0000 7f0e8e29a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8815e0c0 con 0x7f0e880d35b0 2024-04-25T10:05:08.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.187+0000 7f0e8e29a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e880d2510 con 0x7f0e880b5b60 2024-04-25T10:05:08.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.187+0000 7f0e8e29a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880d27f0 con 0x7f0e8815dce0 2024-04-25T10:05:08.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.187+0000 7f0e8c837640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0e880b5b60 0x7f0e880b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.187+0000 7f0e8c837640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0e880b5b60 0x7f0e880b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57104/0 (socket says 172.21.15.16:57104) 2024-04-25T10:05:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.187+0000 7f0e8c837640 1 -- 172.21.15.16:0/3669851765 learned_addr learned my addr 172.21.15.16:0/3669851765 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.187+0000 7f0e87fff640 1 --2- 172.21.15.16:0/3669851765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e8815dce0 0x7f0e880d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.187+0000 7f0e8c837640 1 -- 172.21.15.16:0/3669851765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e8815dce0 msgr2=0x7f0e880d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.187+0000 7f0e8c837640 1 --2- 172.21.15.16:0/3669851765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e8815dce0 0x7f0e880d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.187+0000 7f0e8c837640 1 -- 172.21.15.16:0/3669851765 >> v1:172.21.15.186:6789/0 conn(0x7f0e880d35b0 legacy=0x7f0e880b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.187+0000 7f0e8c837640 1 -- 172.21.15.16:0/3669851765 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e880d2ad0 con 0x7f0e880b5b60 2024-04-25T10:05:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.187+0000 7f0e87fff640 1 --2- 172.21.15.16:0/3669851765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e8815dce0 0x7f0e880d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.188+0000 7f0e8c837640 1 --2- 172.21.15.16:0/3669851765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e880b5b60 0x7f0e880b6b00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009e50 tx=0x7f0e8002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58e4369c7a18210 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.188+0000 7f0e86ffd640 1 -- 172.21.15.16:0/3669851765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80036070 con 0x7f0e880b5b60 2024-04-25T10:05:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.188+0000 7f0e86ffd640 1 -- 172.21.15.16:0/3669851765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e80031600 con 0x7f0e880b5b60 2024-04-25T10:05:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.188+0000 7f0e86ffd640 1 -- 172.21.15.16:0/3669851765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e800319d0 con 0x7f0e880b5b60 2024-04-25T10:05:08.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.189+0000 7f0e8e29a640 1 -- 172.21.15.16:0/3669851765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e880b5b60 msgr2=0x7f0e880b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.189+0000 7f0e8e29a640 1 --2- 172.21.15.16:0/3669851765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e880b5b60 0x7f0e880b6b00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009e50 tx=0x7f0e8002a6c0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.189+0000 7f0e8e29a640 1 -- 172.21.15.16:0/3669851765 shutdown_connections 2024-04-25T10:05:08.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.189+0000 7f0e8e29a640 1 --2- 172.21.15.16:0/3669851765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e880b5b60 0x7f0e880b6b00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.189+0000 7f0e8e29a640 1 --2- 172.21.15.16:0/3669851765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e8815dce0 0x7f0e880d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.189+0000 7f0e8e29a640 1 -- 172.21.15.16:0/3669851765 >> 172.21.15.16:0/3669851765 conn(0x7f0e88072900 msgr2=0x7f0e880d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.189+0000 7f0e8e29a640 1 -- 172.21.15.16:0/3669851765 shutdown_connections 2024-04-25T10:05:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.189+0000 7f0e8e29a640 1 -- 172.21.15.16:0/3669851765 wait complete. 2024-04-25T10:05:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.190+0000 7f0e8e29a640 1 Processor -- start 2024-04-25T10:05:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.190+0000 7f0e8e29a640 1 -- start start 2024-04-25T10:05:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.191+0000 7f0e8e29a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e880b5b60 0x7f0e880cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.191+0000 7f0e8e29a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e880d35b0 0x7f0e8815e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.191+0000 7f0e8e29a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e8815dce0 0x7f0e8815e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.191+0000 7f0e8e29a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880b9820 con 0x7f0e880d35b0 2024-04-25T10:05:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.191+0000 7f0e8e29a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880d3090 con 0x7f0e8815dce0 2024-04-25T10:05:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.191+0000 7f0e8e29a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880b99a0 con 0x7f0e880b5b60 2024-04-25T10:05:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.191+0000 7f0e877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e880d35b0 0x7f0e8815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.191+0000 7f0e877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e880d35b0 0x7f0e8815e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57110/0 (socket says 172.21.15.16:57110) 2024-04-25T10:05:08.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.191+0000 7f0e877fe640 1 -- 172.21.15.16:0/1764914996 learned_addr learned my addr 172.21.15.16:0/1764914996 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:08.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.191+0000 7f0e87fff640 1 --2- 172.21.15.16:0/1764914996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e880b5b60 0x7f0e880cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.191+0000 7f0e877fe640 1 -- 172.21.15.16:0/1764914996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e880b5b60 msgr2=0x7f0e880cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.191+0000 7f0e877fe640 1 --2- 172.21.15.16:0/1764914996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e880b5b60 0x7f0e880cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.191+0000 7f0e877fe640 1 -- 172.21.15.16:0/1764914996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e8815dce0 msgr2=0x7f0e8815e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.191+0000 7f0e8c837640 1 --2- 172.21.15.16:0/1764914996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e8815dce0 0x7f0e8815e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.191+0000 7f0e877fe640 1 --2- 172.21.15.16:0/1764914996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e8815dce0 0x7f0e8815e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.191+0000 7f0e877fe640 1 -- 172.21.15.16:0/1764914996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e880cd4c0 con 0x7f0e880d35b0 2024-04-25T10:05:08.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.191+0000 7f0e877fe640 1 --2- 172.21.15.16:0/1764914996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e880d35b0 0x7f0e8815e290 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c430 tx=0x7f0e7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:08.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.192+0000 7f0e84ff9640 1 -- 172.21.15.16:0/1764914996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e78011790 con 0x7f0e880d35b0 2024-04-25T10:05:08.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.192+0000 7f0e84ff9640 1 -- 172.21.15.16:0/1764914996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e78017070 con 0x7f0e880d35b0 2024-04-25T10:05:08.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.192+0000 7f0e8e29a640 1 -- 172.21.15.16:0/1764914996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e880cc420 con 0x7f0e880d35b0 2024-04-25T10:05:08.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.192+0000 7f0e84ff9640 1 -- 172.21.15.16:0/1764914996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e78011bb0 con 0x7f0e880d35b0 2024-04-25T10:05:08.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.192+0000 7f0e8e29a640 1 -- 172.21.15.16:0/1764914996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e880cccc0 con 0x7f0e880d35b0 2024-04-25T10:05:08.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.193+0000 7f0e8e29a640 1 -- 172.21.15.16:0/1764914996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e880d2070 con 0x7f0e880d35b0 2024-04-25T10:05:08.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.193+0000 7f0e84ff9640 1 -- 172.21.15.16:0/1764914996 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e7801c460 con 0x7f0e880d35b0 2024-04-25T10:05:08.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.194+0000 7f0e84ff9640 1 --2- 172.21.15.16:0/1764914996 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0e6806b4d0 0x7f0e6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.194+0000 7f0e84ff9640 1 -- 172.21.15.16:0/1764914996 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0e78089e20 con 0x7f0e880d35b0 2024-04-25T10:05:08.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.194+0000 7f0e87fff640 1 --2- 172.21.15.16:0/1764914996 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0e6806b4d0 0x7f0e6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.194+0000 7f0e87fff640 1 --2- 172.21.15.16:0/1764914996 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0e6806b4d0 0x7f0e6806d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0e7000b840 tx=0x7f0e70008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:08.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.198+0000 7f0e84ff9640 1 -- 172.21.15.16:0/1764914996 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0e880d2070 con 0x7f0e880d35b0 2024-04-25T10:05:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.380+0000 7f0e8e29a640 1 -- 172.21.15.16:0/1764914996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0e880749b0 con 0x7f0e880d35b0 2024-04-25T10:05:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.380+0000 7f0e84ff9640 1 -- 172.21.15.16:0/1764914996 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0e78058f60 con 0x7f0e880d35b0 2024-04-25T10:05:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:08.387 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-04-25T10:05:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:05:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-04-25T10:05:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-04-25T10:05:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:05:08.387 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-04-25T10:05:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.382+0000 7f0e8e29a640 1 -- 172.21.15.16:0/1764914996 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0e6806b4d0 msgr2=0x7f0e6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.382+0000 7f0e8e29a640 1 --2- 172.21.15.16:0/1764914996 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0e6806b4d0 0x7f0e6806d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0e7000b840 tx=0x7f0e70008040 comp rx=0 tx=0).stop 2024-04-25T10:05:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.382+0000 7f0e8e29a640 1 -- 172.21.15.16:0/1764914996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e880d35b0 msgr2=0x7f0e8815e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.382+0000 7f0e8e29a640 1 --2- 172.21.15.16:0/1764914996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e880d35b0 0x7f0e8815e290 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c430 tx=0x7f0e7800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.382+0000 7f0e8e29a640 1 -- 172.21.15.16:0/1764914996 shutdown_connections 2024-04-25T10:05:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.382+0000 7f0e8e29a640 1 --2- 172.21.15.16:0/1764914996 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0e6806b4d0 0x7f0e6806d990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.382+0000 7f0e8e29a640 1 --2- 172.21.15.16:0/1764914996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e8815dce0 0x7f0e8815e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.382+0000 7f0e8e29a640 1 --2- 172.21.15.16:0/1764914996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e880d35b0 0x7f0e8815e290 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.382+0000 7f0e8e29a640 1 --2- 172.21.15.16:0/1764914996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e880b5b60 0x7f0e880cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.382+0000 7f0e8e29a640 1 -- 172.21.15.16:0/1764914996 >> 172.21.15.16:0/1764914996 conn(0x7f0e88072900 msgr2=0x7f0e880ce250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.382+0000 7f0e8e29a640 1 -- 172.21.15.16:0/1764914996 shutdown_connections 2024-04-25T10:05:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.382+0000 7f0e8e29a640 1 -- 172.21.15.16:0/1764914996 wait complete. 2024-04-25T10:05:08.399 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:05:08.399 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:05:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.498+0000 7f96faae2640 1 Processor -- start 2024-04-25T10:05:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.498+0000 7f96faae2640 1 -- start start 2024-04-25T10:05:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.499+0000 7f96faae2640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f96f4161090 0x7f96f4161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.499+0000 7f96faae2640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96f40b9720 0x7f96f40ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.499+0000 7f96faae2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f4161f80 con 0x7f96f40cece0 2024-04-25T10:05:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.499+0000 7f96faae2640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f40cf130 con 0x7f96f4161090 2024-04-25T10:05:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.499+0000 7f96faae2640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f40d21a0 con 0x7f96f40b9720 2024-04-25T10:05:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.499+0000 7f96f907f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f96f40cece0 0x7f96f406c0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43220/0 (socket says 172.21.15.16:43220) 2024-04-25T10:05:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.499+0000 7f96f907f640 1 -- 172.21.15.16:0/1225758162 learned_addr learned my addr 172.21.15.16:0/1225758162 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.499+0000 7f96f3fff640 1 --2- 172.21.15.16:0/1225758162 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96f40b9720 0x7f96f40ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.499+0000 7f96f37fe640 1 -- 172.21.15.16:0/1225758162 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2382436190 0 0) 0x7f96f4161f80 con 0x7f96f40cece0 2024-04-25T10:05:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.499+0000 7f96f887e640 1 --2- 172.21.15.16:0/1225758162 >> v2:172.21.15.186:3300/0 conn(0x7f96f4161090 0x7f96f4161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.499+0000 7f96f37fe640 1 -- 172.21.15.16:0/1225758162 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96dc003520 con 0x7f96f40cece0 2024-04-25T10:05:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.499+0000 7f96f37fe640 1 -- 172.21.15.16:0/1225758162 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2555498446 0 0) 0x7f96dc003520 con 0x7f96f40cece0 2024-04-25T10:05:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.499+0000 7f96f37fe640 1 -- 172.21.15.16:0/1225758162 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96f40b9720 msgr2=0x7f96f40ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.499+0000 7f96f37fe640 1 --2- 172.21.15.16:0/1225758162 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96f40b9720 0x7f96f40ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f37fe640 1 -- 172.21.15.16:0/1225758162 >> v2:172.21.15.186:3300/0 conn(0x7f96f4161090 msgr2=0x7f96f4161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f37fe640 1 --2- 172.21.15.16:0/1225758162 >> v2:172.21.15.186:3300/0 conn(0x7f96f4161090 0x7f96f4161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f37fe640 1 -- 172.21.15.16:0/1225758162 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f4162260 con 0x7f96f40cece0 2024-04-25T10:05:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f37fe640 1 -- 172.21.15.16:0/1225758162 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f96e4001670 con 0x7f96f40cece0 2024-04-25T10:05:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f37fe640 1 -- 172.21.15.16:0/1225758162 >> v1:172.21.15.16:6789/0 conn(0x7f96f40cece0 legacy=0x7f96f406c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f37fe640 1 --2- 172.21.15.16:0/1225758162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96dc0039b0 0x7f96dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f37fe640 1 --2- 172.21.15.16:0/1225758162 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96f40b9720 0x7f96dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f37fe640 1 --2- 172.21.15.16:0/1225758162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96dc004d40 0x7f96dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f37fe640 1 -- 172.21.15.16:0/1225758162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f40cf130 con 0x7f96dc004d40 2024-04-25T10:05:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f37fe640 1 -- 172.21.15.16:0/1225758162 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f40d21a0 con 0x7f96dc0039b0 2024-04-25T10:05:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f37fe640 1 -- 172.21.15.16:0/1225758162 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc00b9c0 con 0x7f96f40b9720 2024-04-25T10:05:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f907f640 1 --2- 172.21.15.16:0/1225758162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96dc004d40 0x7f96dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f887e640 1 --2- 172.21.15.16:0/1225758162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96dc0039b0 0x7f96dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f3fff640 1 --2- 172.21.15.16:0/1225758162 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96f40b9720 0x7f96dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f907f640 1 -- 172.21.15.16:0/1225758162 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96f40b9720 msgr2=0x7f96dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f907f640 1 --2- 172.21.15.16:0/1225758162 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96f40b9720 0x7f96dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f907f640 1 -- 172.21.15.16:0/1225758162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96dc0039b0 msgr2=0x7f96dc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f907f640 1 --2- 172.21.15.16:0/1225758162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96dc0039b0 0x7f96dc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.500+0000 7f96f907f640 1 -- 172.21.15.16:0/1225758162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc003520 con 0x7f96dc004d40 2024-04-25T10:05:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.501+0000 7f96f907f640 1 --2- 172.21.15.16:0/1225758162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96dc004d40 0x7f96dc0051c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f96e400c160 tx=0x7f96dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be85ec179aad3e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.501+0000 7f96f37fe640 1 -- 172.21.15.16:0/1225758162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4007c80 con 0x7f96dc004d40 2024-04-25T10:05:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.501+0000 7f96f37fe640 1 -- 172.21.15.16:0/1225758162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e4005af0 con 0x7f96dc004d40 2024-04-25T10:05:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.501+0000 7f96f37fe640 1 -- 172.21.15.16:0/1225758162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4007070 con 0x7f96dc004d40 2024-04-25T10:05:08.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.501+0000 7f96faae2640 1 -- 172.21.15.16:0/1225758162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96dc004d40 msgr2=0x7f96dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.501+0000 7f96faae2640 1 --2- 172.21.15.16:0/1225758162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96dc004d40 0x7f96dc0051c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f96e400c160 tx=0x7f96dc005830 comp rx=0 tx=0).stop 2024-04-25T10:05:08.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.501+0000 7f96f907f640 1 -- 172.21.15.16:0/1225758162 reap_dead start 2024-04-25T10:05:08.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.501+0000 7f96faae2640 1 -- 172.21.15.16:0/1225758162 shutdown_connections 2024-04-25T10:05:08.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.501+0000 7f96faae2640 1 -- 172.21.15.16:0/1225758162 >> 172.21.15.16:0/1225758162 conn(0x7f96f40b55e0 msgr2=0x7f96f40b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:08.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.501+0000 7f96faae2640 1 -- 172.21.15.16:0/1225758162 shutdown_connections 2024-04-25T10:05:08.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.501+0000 7f96faae2640 1 -- 172.21.15.16:0/1225758162 wait complete. 2024-04-25T10:05:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.503+0000 7f96faae2640 1 Processor -- start 2024-04-25T10:05:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.503+0000 7f96faae2640 1 -- start start 2024-04-25T10:05:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.503+0000 7f96faae2640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96dc004d40 0x7f96f40cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.503+0000 7f96faae2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96f4152200 0x7f96f40cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.503+0000 7f96faae2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96f415e1e0 0x7f96f415e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.503+0000 7f96faae2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f4161470 con 0x7f96f415e1e0 2024-04-25T10:05:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.503+0000 7f96faae2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f40d1e30 con 0x7f96f4152200 2024-04-25T10:05:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.503+0000 7f96faae2640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f4161c70 con 0x7f96dc004d40 2024-04-25T10:05:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.504+0000 7f96f907f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96f415e1e0 0x7f96f415e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.504+0000 7f96f907f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96f415e1e0 0x7f96f415e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57122/0 (socket says 172.21.15.16:57122) 2024-04-25T10:05:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.504+0000 7f96f907f640 1 -- 172.21.15.16:0/757475954 learned_addr learned my addr 172.21.15.16:0/757475954 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.504+0000 7f96f887e640 1 --2- 172.21.15.16:0/757475954 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96dc004d40 0x7f96f40cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.504+0000 7f96f3fff640 1 --2- 172.21.15.16:0/757475954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96f4152200 0x7f96f40cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.504+0000 7f96f907f640 1 -- 172.21.15.16:0/757475954 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96dc004d40 msgr2=0x7f96f40cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.504+0000 7f96f907f640 1 --2- 172.21.15.16:0/757475954 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96dc004d40 0x7f96f40cb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.504+0000 7f96f907f640 1 -- 172.21.15.16:0/757475954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96f4152200 msgr2=0x7f96f40cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.504+0000 7f96f907f640 1 --2- 172.21.15.16:0/757475954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96f4152200 0x7f96f40cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.504+0000 7f96f907f640 1 -- 172.21.15.16:0/757475954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f40c8260 con 0x7f96f415e1e0 2024-04-25T10:05:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.504+0000 7f96f887e640 1 --2- 172.21.15.16:0/757475954 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96dc004d40 0x7f96f40cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.504+0000 7f96f3fff640 1 --2- 172.21.15.16:0/757475954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96f4152200 0x7f96f40cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.504+0000 7f96f907f640 1 --2- 172.21.15.16:0/757475954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96f415e1e0 0x7f96f415e690 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f96e400c160 tx=0x7f96f40d1b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.504+0000 7f96f17fa640 1 -- 172.21.15.16:0/757475954 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4037040 con 0x7f96f415e1e0 2024-04-25T10:05:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.504+0000 7f96f17fa640 1 -- 172.21.15.16:0/757475954 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e4041070 con 0x7f96f415e1e0 2024-04-25T10:05:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.504+0000 7f96faae2640 1 -- 172.21.15.16:0/757475954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f40c8540 con 0x7f96f415e1e0 2024-04-25T10:05:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.504+0000 7f96f17fa640 1 -- 172.21.15.16:0/757475954 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e40072b0 con 0x7f96f415e1e0 2024-04-25T10:05:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.504+0000 7f96faae2640 1 -- 172.21.15.16:0/757475954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f40c8b00 con 0x7f96f415e1e0 2024-04-25T10:05:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.505+0000 7f96faae2640 1 -- 172.21.15.16:0/757475954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f40750b0 con 0x7f96f415e1e0 2024-04-25T10:05:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.506+0000 7f96f17fa640 1 -- 172.21.15.16:0/757475954 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96e4007450 con 0x7f96f415e1e0 2024-04-25T10:05:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.506+0000 7f96f17fa640 1 --2- 172.21.15.16:0/757475954 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f96e006b500 0x7f96e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.506+0000 7f96f17fa640 1 -- 172.21.15.16:0/757475954 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f96e40a9d40 con 0x7f96f415e1e0 2024-04-25T10:05:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.506+0000 7f96f887e640 1 --2- 172.21.15.16:0/757475954 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f96e006b500 0x7f96e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.507+0000 7f96f887e640 1 --2- 172.21.15.16:0/757475954 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f96e006b500 0x7f96e006d9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f96f41f7310 tx=0x7f96ec00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.510+0000 7f96f17fa640 1 -- 172.21.15.16:0/757475954 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96e40790c0 con 0x7f96f415e1e0 2024-04-25T10:05:08.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:08.692+0000 7f96faae2640 1 -- 172.21.15.16:0/757475954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f96f40cae50 con 0x7f96f415e1e0 2024-04-25T10:05:09.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.432+0000 7f96f17fa640 1 -- 172.21.15.16:0/757475954 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7f96e40789c0 con 0x7f96f415e1e0 2024-04-25T10:05:09.439 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-04-25T10:05:09.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.438+0000 7f96faae2640 1 -- 172.21.15.16:0/757475954 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f96e006b500 msgr2=0x7f96e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:09.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.438+0000 7f96faae2640 1 --2- 172.21.15.16:0/757475954 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f96e006b500 0x7f96e006d9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f96f41f7310 tx=0x7f96ec00b040 comp rx=0 tx=0).stop 2024-04-25T10:05:09.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.438+0000 7f96faae2640 1 -- 172.21.15.16:0/757475954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96f415e1e0 msgr2=0x7f96f415e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:09.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.438+0000 7f96faae2640 1 --2- 172.21.15.16:0/757475954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96f415e1e0 0x7f96f415e690 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f96e400c160 tx=0x7f96f40d1b50 comp rx=0 tx=0).stop 2024-04-25T10:05:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.439+0000 7f96faae2640 1 -- 172.21.15.16:0/757475954 shutdown_connections 2024-04-25T10:05:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.439+0000 7f96faae2640 1 --2- 172.21.15.16:0/757475954 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f96e006b500 0x7f96e006d9c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.439+0000 7f96faae2640 1 --2- 172.21.15.16:0/757475954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96f415e1e0 0x7f96f415e690 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.439+0000 7f96faae2640 1 --2- 172.21.15.16:0/757475954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96f4152200 0x7f96f40cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.439+0000 7f96faae2640 1 --2- 172.21.15.16:0/757475954 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96dc004d40 0x7f96f40cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.439+0000 7f96faae2640 1 -- 172.21.15.16:0/757475954 >> 172.21.15.16:0/757475954 conn(0x7f96f40b55e0 msgr2=0x7f96f40b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.439+0000 7f96faae2640 1 -- 172.21.15.16:0/757475954 shutdown_connections 2024-04-25T10:05:09.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.440+0000 7f96faae2640 1 -- 172.21.15.16:0/757475954 wait complete. 2024-04-25T10:05:09.461 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:05:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.569+0000 7f845bf6a640 1 Processor -- start 2024-04-25T10:05:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.569+0000 7f845bf6a640 1 -- start start 2024-04-25T10:05:09.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.570+0000 7f845bf6a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84540ccec0 0x7f845415b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.570+0000 7f845bf6a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84540b2cf0 0x7f84540cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.570+0000 7f845bf6a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84540cd7e0 con 0x7f845415e560 2024-04-25T10:05:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.570+0000 7f845bf6a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84540cf840 con 0x7f84540b2cf0 2024-04-25T10:05:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.570+0000 7f845bf6a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845415bf00 con 0x7f84540ccec0 2024-04-25T10:05:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.570+0000 7f8459505640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84540b2cf0 0x7f84540cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.570+0000 7f8459505640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84540b2cf0 0x7f84540cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57130/0 (socket says 172.21.15.16:57130) 2024-04-25T10:05:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.570+0000 7f845a507640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f845415e560 0x7f845415e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43230/0 (socket says 172.21.15.16:43230) 2024-04-25T10:05:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.570+0000 7f8459505640 1 -- 172.21.15.16:0/903405589 learned_addr learned my addr 172.21.15.16:0/903405589 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.570+0000 7f8459d06640 1 --2- 172.21.15.16:0/903405589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84540ccec0 0x7f845415b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.570+0000 7f8459505640 1 -- 172.21.15.16:0/903405589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84540ccec0 msgr2=0x7f845415b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.570+0000 7f8459505640 1 --2- 172.21.15.16:0/903405589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84540ccec0 0x7f845415b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.570+0000 7f8459505640 1 -- 172.21.15.16:0/903405589 >> v1:172.21.15.16:6789/0 conn(0x7f845415e560 legacy=0x7f845415e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.570+0000 7f8459505640 1 -- 172.21.15.16:0/903405589 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84540ce7a0 con 0x7f84540b2cf0 2024-04-25T10:05:09.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.570+0000 7f8459505640 1 --2- 172.21.15.16:0/903405589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84540b2cf0 0x7f84540cd2a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f84480098a0 tx=0x7f844802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8da3e7a7d127fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.570+0000 7f8458d04640 1 -- 172.21.15.16:0/903405589 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8448037070 con 0x7f84540b2cf0 2024-04-25T10:05:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.570+0000 7f8458d04640 1 -- 172.21.15.16:0/903405589 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8448032440 con 0x7f84540b2cf0 2024-04-25T10:05:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.570+0000 7f8458d04640 1 -- 172.21.15.16:0/903405589 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8448032810 con 0x7f84540b2cf0 2024-04-25T10:05:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.571+0000 7f845bf6a640 1 -- 172.21.15.16:0/903405589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84540b2cf0 msgr2=0x7f84540cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.571+0000 7f845bf6a640 1 --2- 172.21.15.16:0/903405589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84540b2cf0 0x7f84540cd2a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f84480098a0 tx=0x7f844802a200 comp rx=0 tx=0).stop 2024-04-25T10:05:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.571+0000 7f845bf6a640 1 -- 172.21.15.16:0/903405589 shutdown_connections 2024-04-25T10:05:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.571+0000 7f845bf6a640 1 --2- 172.21.15.16:0/903405589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84540b2cf0 0x7f84540cd2a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.571+0000 7f845bf6a640 1 --2- 172.21.15.16:0/903405589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84540ccec0 0x7f845415b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.571+0000 7f845bf6a640 1 -- 172.21.15.16:0/903405589 >> 172.21.15.16:0/903405589 conn(0x7f8454099a40 msgr2=0x7f84540b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.571+0000 7f845bf6a640 1 -- 172.21.15.16:0/903405589 shutdown_connections 2024-04-25T10:05:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.571+0000 7f845bf6a640 1 -- 172.21.15.16:0/903405589 wait complete. 2024-04-25T10:05:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.572+0000 7f845bf6a640 1 Processor -- start 2024-04-25T10:05:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.572+0000 7f845bf6a640 1 -- start start 2024-04-25T10:05:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.573+0000 7f845bf6a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84540b2cf0 0x7f845414ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.573+0000 7f845bf6a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84540ccec0 0x7f845415ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.573+0000 7f845bf6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845415e560 0x7f8454155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.573+0000 7f845bf6a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84540cf040 con 0x7f845415e560 2024-04-25T10:05:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.573+0000 7f845bf6a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84540ced60 con 0x7f84540b2cf0 2024-04-25T10:05:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.573+0000 7f845bf6a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84540ce450 con 0x7f84540ccec0 2024-04-25T10:05:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.573+0000 7f845a507640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845415e560 0x7f8454155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.573+0000 7f845a507640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845415e560 0x7f8454155b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57138/0 (socket says 172.21.15.16:57138) 2024-04-25T10:05:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.573+0000 7f845a507640 1 -- 172.21.15.16:0/3994108089 learned_addr learned my addr 172.21.15.16:0/3994108089 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.573+0000 7f8459505640 1 --2- 172.21.15.16:0/3994108089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84540ccec0 0x7f845415ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.573+0000 7f8459d06640 1 --2- 172.21.15.16:0/3994108089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84540b2cf0 0x7f845414ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.573+0000 7f845a507640 1 -- 172.21.15.16:0/3994108089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84540ccec0 msgr2=0x7f845415ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.573+0000 7f845a507640 1 --2- 172.21.15.16:0/3994108089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84540ccec0 0x7f845415ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.573+0000 7f845a507640 1 -- 172.21.15.16:0/3994108089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84540b2cf0 msgr2=0x7f845414ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.573+0000 7f845a507640 1 --2- 172.21.15.16:0/3994108089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84540b2cf0 0x7f845414ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.573+0000 7f845a507640 1 -- 172.21.15.16:0/3994108089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8454153150 con 0x7f845415e560 2024-04-25T10:05:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.573+0000 7f8459d06640 1 --2- 172.21.15.16:0/3994108089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84540b2cf0 0x7f845414ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.574+0000 7f845a507640 1 --2- 172.21.15.16:0/3994108089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845415e560 0x7f8454155b70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f844400c730 tx=0x7f844400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.574+0000 7f84527fc640 1 -- 172.21.15.16:0/3994108089 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444018070 con 0x7f845415e560 2024-04-25T10:05:09.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.574+0000 7f84527fc640 1 -- 172.21.15.16:0/3994108089 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8444004d60 con 0x7f845415e560 2024-04-25T10:05:09.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.574+0000 7f845bf6a640 1 -- 172.21.15.16:0/3994108089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8454153430 con 0x7f845415e560 2024-04-25T10:05:09.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.574+0000 7f84527fc640 1 -- 172.21.15.16:0/3994108089 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844400a880 con 0x7f845415e560 2024-04-25T10:05:09.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.574+0000 7f845bf6a640 1 -- 172.21.15.16:0/3994108089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84541539f0 con 0x7f845415e560 2024-04-25T10:05:09.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.575+0000 7f845bf6a640 1 -- 172.21.15.16:0/3994108089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84540749b0 con 0x7f845415e560 2024-04-25T10:05:09.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.575+0000 7f84527fc640 1 -- 172.21.15.16:0/3994108089 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f844401b070 con 0x7f845415e560 2024-04-25T10:05:09.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.576+0000 7f84527fc640 1 --2- 172.21.15.16:0/3994108089 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f844006b500 0x7f844006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:09.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.576+0000 7f84527fc640 1 -- 172.21.15.16:0/3994108089 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f84440894b0 con 0x7f845415e560 2024-04-25T10:05:09.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.576+0000 7f8459d06640 1 --2- 172.21.15.16:0/3994108089 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f844006b500 0x7f844006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:09.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.578+0000 7f8459d06640 1 --2- 172.21.15.16:0/3994108089 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f844006b500 0x7f844006d9c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f844c006fd0 tx=0x7f844c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:09.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.581+0000 7f84527fc640 1 -- 172.21.15.16:0/3994108089 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8444058510 con 0x7f845415e560 2024-04-25T10:05:09.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:09.761+0000 7f845bf6a640 1 -- 172.21.15.16:0/3994108089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f845415a440 con 0x7f845415e560 2024-04-25T10:05:10.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.438+0000 7f84527fc640 1 -- 172.21.15.16:0/3994108089 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8444057e10 con 0x7f845415e560 2024-04-25T10:05:10.444 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-04-25T10:05:10.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.443+0000 7f845bf6a640 1 -- 172.21.15.16:0/3994108089 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f844006b500 msgr2=0x7f844006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.443+0000 7f845bf6a640 1 --2- 172.21.15.16:0/3994108089 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f844006b500 0x7f844006d9c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f844c006fd0 tx=0x7f844c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:10.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.443+0000 7f845bf6a640 1 -- 172.21.15.16:0/3994108089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845415e560 msgr2=0x7f8454155b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.443+0000 7f845bf6a640 1 --2- 172.21.15.16:0/3994108089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845415e560 0x7f8454155b70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f844400c730 tx=0x7f844400cb30 comp rx=0 tx=0).stop 2024-04-25T10:05:10.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.445+0000 7f845bf6a640 1 -- 172.21.15.16:0/3994108089 shutdown_connections 2024-04-25T10:05:10.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.445+0000 7f845bf6a640 1 --2- 172.21.15.16:0/3994108089 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f844006b500 0x7f844006d9c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.445+0000 7f845bf6a640 1 --2- 172.21.15.16:0/3994108089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845415e560 0x7f8454155b70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.445+0000 7f845bf6a640 1 --2- 172.21.15.16:0/3994108089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84540ccec0 0x7f845415ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.445+0000 7f845bf6a640 1 --2- 172.21.15.16:0/3994108089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84540b2cf0 0x7f845414ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.445+0000 7f845bf6a640 1 -- 172.21.15.16:0/3994108089 >> 172.21.15.16:0/3994108089 conn(0x7f8454099a40 msgr2=0x7f845406e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:10.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.449+0000 7f845bf6a640 1 -- 172.21.15.16:0/3994108089 shutdown_connections 2024-04-25T10:05:10.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.454+0000 7f845bf6a640 1 -- 172.21.15.16:0/3994108089 wait complete. 2024-04-25T10:05:10.480 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:05:10.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.582+0000 7f2ddc89a640 1 Processor -- start 2024-04-25T10:05:10.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.582+0000 7f2ddc89a640 1 -- start start 2024-04-25T10:05:10.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.583+0000 7f2ddc89a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd80d0490 0x7f2dd815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.583+0000 7f2ddc89a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2dd80ce1a0 0x7f2dd80ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.583+0000 7f2ddc89a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd806c150 con 0x7f2dd80b2740 2024-04-25T10:05:10.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.583+0000 7f2ddc89a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd80d2000 con 0x7f2dd80ce1a0 2024-04-25T10:05:10.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.583+0000 7f2ddc89a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd806cd60 con 0x7f2dd80d0490 2024-04-25T10:05:10.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.583+0000 7f2dd6d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2dd80ce1a0 0x7f2dd80ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.583+0000 7f2dd5d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2dd80b2740 0x7f2dd80d0870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43232/0 (socket says 172.21.15.16:43232) 2024-04-25T10:05:10.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.583+0000 7f2dd6d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2dd80ce1a0 0x7f2dd80ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57140/0 (socket says 172.21.15.16:57140) 2024-04-25T10:05:10.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.583+0000 7f2dd5d9b640 1 -- 172.21.15.16:0/456825302 learned_addr learned my addr 172.21.15.16:0/456825302 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:10.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.583+0000 7f2dd659c640 1 --2- 172.21.15.16:0/456825302 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd80d0490 0x7f2dd815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.584+0000 7f2dd6d9d640 1 -- 172.21.15.16:0/456825302 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd80d0490 msgr2=0x7f2dd815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:10.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.584+0000 7f2dd6d9d640 1 --2- 172.21.15.16:0/456825302 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd80d0490 0x7f2dd815ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.584+0000 7f2dd6d9d640 1 -- 172.21.15.16:0/456825302 >> v1:172.21.15.16:6789/0 conn(0x7f2dd80b2740 legacy=0x7f2dd80d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.584+0000 7f2dd6d9d640 1 -- 172.21.15.16:0/456825302 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd806c430 con 0x7f2dd80ce1a0 2024-04-25T10:05:10.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.584+0000 7f2dd6d9d640 1 --2- 172.21.15.16:0/456825302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd80ce1a0 0x7f2dd80ce630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009dc0 tx=0x7f2dc802a500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb9979ce43bef0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:10.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.584+0000 7f2dd559a640 1 -- 172.21.15.16:0/456825302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8007740 con 0x7f2dd80ce1a0 2024-04-25T10:05:10.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.584+0000 7f2dd559a640 1 -- 172.21.15.16:0/456825302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc80078e0 con 0x7f2dd80ce1a0 2024-04-25T10:05:10.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.584+0000 7f2dd559a640 1 -- 172.21.15.16:0/456825302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8007cb0 con 0x7f2dd80ce1a0 2024-04-25T10:05:10.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.585+0000 7f2ddc89a640 1 -- 172.21.15.16:0/456825302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd80ce1a0 msgr2=0x7f2dd80ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.585+0000 7f2ddc89a640 1 --2- 172.21.15.16:0/456825302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd80ce1a0 0x7f2dd80ce630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009dc0 tx=0x7f2dc802a500 comp rx=0 tx=0).stop 2024-04-25T10:05:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.585+0000 7f2ddc89a640 1 -- 172.21.15.16:0/456825302 shutdown_connections 2024-04-25T10:05:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.585+0000 7f2ddc89a640 1 --2- 172.21.15.16:0/456825302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd80ce1a0 0x7f2dd80ce630 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.585+0000 7f2ddc89a640 1 --2- 172.21.15.16:0/456825302 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd80d0490 0x7f2dd815ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.585+0000 7f2ddc89a640 1 -- 172.21.15.16:0/456825302 >> 172.21.15.16:0/456825302 conn(0x7f2dd80b5660 msgr2=0x7f2dd80b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.585+0000 7f2ddc89a640 1 -- 172.21.15.16:0/456825302 shutdown_connections 2024-04-25T10:05:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.585+0000 7f2ddc89a640 1 -- 172.21.15.16:0/456825302 wait complete. 2024-04-25T10:05:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.586+0000 7f2ddc89a640 1 Processor -- start 2024-04-25T10:05:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.586+0000 7f2ddc89a640 1 -- start start 2024-04-25T10:05:10.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.587+0000 7f2ddc89a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd80b2740 0x7f2dd80cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.587+0000 7f2ddc89a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd80ce1a0 0x7f2dd80abd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.587+0000 7f2ddc89a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dd80d0490 0x7f2dd80aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.587+0000 7f2ddc89a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd806c9f0 con 0x7f2dd80b2740 2024-04-25T10:05:10.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.587+0000 7f2ddc89a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd815f1d0 con 0x7f2dd80d0490 2024-04-25T10:05:10.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.587+0000 7f2ddc89a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd815e020 con 0x7f2dd80ce1a0 2024-04-25T10:05:10.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.587+0000 7f2dd659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd80b2740 0x7f2dd80cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.587+0000 7f2dd659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd80b2740 0x7f2dd80cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57156/0 (socket says 172.21.15.16:57156) 2024-04-25T10:05:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.587+0000 7f2dd659c640 1 -- 172.21.15.16:0/1715512019 learned_addr learned my addr 172.21.15.16:0/1715512019 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.587+0000 7f2dd5d9b640 1 --2- 172.21.15.16:0/1715512019 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd80ce1a0 0x7f2dd80abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.588+0000 7f2dd6d9d640 1 --2- 172.21.15.16:0/1715512019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dd80d0490 0x7f2dd80aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.588+0000 7f2dd659c640 1 -- 172.21.15.16:0/1715512019 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd80ce1a0 msgr2=0x7f2dd80abd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.588+0000 7f2dd659c640 1 --2- 172.21.15.16:0/1715512019 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd80ce1a0 0x7f2dd80abd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.588+0000 7f2dd659c640 1 -- 172.21.15.16:0/1715512019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dd80d0490 msgr2=0x7f2dd80aa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.588+0000 7f2dd659c640 1 --2- 172.21.15.16:0/1715512019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dd80d0490 0x7f2dd80aa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.588+0000 7f2dd659c640 1 -- 172.21.15.16:0/1715512019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd80aaa00 con 0x7f2dd80b2740 2024-04-25T10:05:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.588+0000 7f2dd5d9b640 1 --2- 172.21.15.16:0/1715512019 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd80ce1a0 0x7f2dd80abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.588+0000 7f2dd659c640 1 --2- 172.21.15.16:0/1715512019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd80b2740 0x7f2dd80cc280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2dd000d620 tx=0x7f2dd000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.588+0000 7f2dceffd640 1 -- 172.21.15.16:0/1715512019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd000b620 con 0x7f2dd80b2740 2024-04-25T10:05:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.588+0000 7f2dceffd640 1 -- 172.21.15.16:0/1715512019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd0019070 con 0x7f2dd80b2740 2024-04-25T10:05:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.588+0000 7f2dceffd640 1 -- 172.21.15.16:0/1715512019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd000ba00 con 0x7f2dd80b2740 2024-04-25T10:05:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.588+0000 7f2ddc89a640 1 -- 172.21.15.16:0/1715512019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd80aaca0 con 0x7f2dd80b2740 2024-04-25T10:05:10.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.588+0000 7f2ddc89a640 1 -- 172.21.15.16:0/1715512019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd80ab260 con 0x7f2dd80b2740 2024-04-25T10:05:10.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.589+0000 7f2ddc89a640 1 -- 172.21.15.16:0/1715512019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd80750b0 con 0x7f2dd80b2740 2024-04-25T10:05:10.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.589+0000 7f2dceffd640 1 -- 172.21.15.16:0/1715512019 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2dd000bba0 con 0x7f2dd80b2740 2024-04-25T10:05:10.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.590+0000 7f2dceffd640 1 --2- 172.21.15.16:0/1715512019 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2db806b4d0 0x7f2db806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.590+0000 7f2dceffd640 1 -- 172.21.15.16:0/1715512019 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f2dd008a5c0 con 0x7f2dd80b2740 2024-04-25T10:05:10.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.590+0000 7f2dd5d9b640 1 --2- 172.21.15.16:0/1715512019 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2db806b4d0 0x7f2db806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.591+0000 7f2dd5d9b640 1 --2- 172.21.15.16:0/1715512019 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2db806b4d0 0x7f2db806d990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80cecb0 tx=0x7f2dc0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:10.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.595+0000 7f2dceffd640 1 -- 172.21.15.16:0/1715512019 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2dd00594a0 con 0x7f2dd80b2740 2024-04-25T10:05:10.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:10.812+0000 7f2ddc89a640 1 -- 172.21.15.16:0/1715512019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2dd80d18d0 con 0x7f2dd80b2740 2024-04-25T10:05:11.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.440+0000 7f2dceffd640 1 -- 172.21.15.16:0/1715512019 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2dd0058da0 con 0x7f2dd80b2740 2024-04-25T10:05:11.446 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T10:05:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.445+0000 7f2ddc89a640 1 -- 172.21.15.16:0/1715512019 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2db806b4d0 msgr2=0x7f2db806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.445+0000 7f2ddc89a640 1 --2- 172.21.15.16:0/1715512019 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2db806b4d0 0x7f2db806d990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80cecb0 tx=0x7f2dc0009040 comp rx=0 tx=0).stop 2024-04-25T10:05:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.446+0000 7f2ddc89a640 1 -- 172.21.15.16:0/1715512019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd80b2740 msgr2=0x7f2dd80cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.446+0000 7f2ddc89a640 1 --2- 172.21.15.16:0/1715512019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd80b2740 0x7f2dd80cc280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2dd000d620 tx=0x7f2dd000dae0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.446+0000 7f2ddc89a640 1 -- 172.21.15.16:0/1715512019 shutdown_connections 2024-04-25T10:05:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.446+0000 7f2ddc89a640 1 --2- 172.21.15.16:0/1715512019 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2db806b4d0 0x7f2db806d990 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.446+0000 7f2ddc89a640 1 --2- 172.21.15.16:0/1715512019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dd80d0490 0x7f2dd80aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.446+0000 7f2ddc89a640 1 --2- 172.21.15.16:0/1715512019 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd80ce1a0 0x7f2dd80abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.446+0000 7f2ddc89a640 1 --2- 172.21.15.16:0/1715512019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd80b2740 0x7f2dd80cc280 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.446+0000 7f2ddc89a640 1 -- 172.21.15.16:0/1715512019 >> 172.21.15.16:0/1715512019 conn(0x7f2dd80b5660 msgr2=0x7f2dd80cef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.447+0000 7f2ddc89a640 1 -- 172.21.15.16:0/1715512019 shutdown_connections 2024-04-25T10:05:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.447+0000 7f2ddc89a640 1 -- 172.21.15.16:0/1715512019 wait complete. 2024-04-25T10:05:11.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.585+0000 7fb9caff9640 1 Processor -- start 2024-04-25T10:05:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.585+0000 7fb9caff9640 1 -- start start 2024-04-25T10:05:11.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.586+0000 7fb9caff9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9c40d35b0 0x7fb9c40b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.586+0000 7fb9caff9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c415e0c0 con 0x7fb9c415dce0 2024-04-25T10:05:11.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.586+0000 7fb9caff9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c40d2510 con 0x7fb9c40b5990 2024-04-25T10:05:11.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.586+0000 7fb9caff9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c40b5de0 con 0x7fb9c40d35b0 2024-04-25T10:05:11.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.586+0000 7fb9c8d95640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb9c415dce0 0x7fb9c40d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43246/0 (socket says 172.21.15.16:43246) 2024-04-25T10:05:11.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.586+0000 7fb9c8d95640 1 -- 172.21.15.16:0/466098083 learned_addr learned my addr 172.21.15.16:0/466098083 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:11.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.586+0000 7fb9c3fff640 1 --2- 172.21.15.16:0/466098083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9c40d35b0 0x7fb9c40b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.586+0000 7fb9c37fe640 1 -- 172.21.15.16:0/466098083 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373664918 0 0) 0x7fb9c415e0c0 con 0x7fb9c415dce0 2024-04-25T10:05:11.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.586+0000 7fb9c37fe640 1 -- 172.21.15.16:0/466098083 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9b4003520 con 0x7fb9c415dce0 2024-04-25T10:05:11.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.586+0000 7fb9c37fe640 1 -- 172.21.15.16:0/466098083 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1209998840 0 0) 0x7fb9c40d2510 con 0x7fb9c40b5990 2024-04-25T10:05:11.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.586+0000 7fb9c37fe640 1 -- 172.21.15.16:0/466098083 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9c415e0c0 con 0x7fb9c40b5990 2024-04-25T10:05:11.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.587+0000 7fb9c3fff640 1 -- 172.21.15.16:0/466098083 >> v1:172.21.15.186:6789/0 conn(0x7fb9c40b5990 legacy=0x7fb9c40b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.587+0000 7fb9c3fff640 1 -- 172.21.15.16:0/466098083 >> v1:172.21.15.16:6789/0 conn(0x7fb9c415dce0 legacy=0x7fb9c40d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.587+0000 7fb9c37fe640 1 -- 172.21.15.16:0/466098083 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1930377073 0 0) 0x7fb9b4003520 con 0x7fb9c415dce0 2024-04-25T10:05:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.587+0000 7fb9c3fff640 1 -- 172.21.15.16:0/466098083 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c40d27f0 con 0x7fb9c40d35b0 2024-04-25T10:05:11.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.587+0000 7fb9c3fff640 1 --2- 172.21.15.16:0/466098083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9c40d35b0 0x7fb9c40b5450 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00097d0 tx=0x7fb9b002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b1b5914747decf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.587+0000 7fb9c37fe640 1 -- 172.21.15.16:0/466098083 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0037070 con 0x7fb9c40d35b0 2024-04-25T10:05:11.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.587+0000 7fb9c37fe640 1 -- 172.21.15.16:0/466098083 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b002ac30 con 0x7fb9c40d35b0 2024-04-25T10:05:11.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.587+0000 7fb9c37fe640 1 -- 172.21.15.16:0/466098083 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8001670 con 0x7fb9c40d35b0 2024-04-25T10:05:11.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.588+0000 7fb9caff9640 1 -- 172.21.15.16:0/466098083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9c40d35b0 msgr2=0x7fb9c40b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.588+0000 7fb9caff9640 1 --2- 172.21.15.16:0/466098083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9c40d35b0 0x7fb9c40b5450 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00097d0 tx=0x7fb9b002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.588+0000 7fb9caff9640 1 -- 172.21.15.16:0/466098083 shutdown_connections 2024-04-25T10:05:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.588+0000 7fb9caff9640 1 --2- 172.21.15.16:0/466098083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9c40d35b0 0x7fb9c40b5450 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.588+0000 7fb9caff9640 1 -- 172.21.15.16:0/466098083 >> 172.21.15.16:0/466098083 conn(0x7fb9c4072900 msgr2=0x7fb9c40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.588+0000 7fb9caff9640 1 -- 172.21.15.16:0/466098083 shutdown_connections 2024-04-25T10:05:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.588+0000 7fb9caff9640 1 -- 172.21.15.16:0/466098083 wait complete. 2024-04-25T10:05:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.589+0000 7fb9caff9640 1 Processor -- start 2024-04-25T10:05:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.589+0000 7fb9caff9640 1 -- start start 2024-04-25T10:05:11.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.590+0000 7fb9caff9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c40b5990 0x7fb9c40c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.590+0000 7fb9caff9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9c40d35b0 0x7fb9c4156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.590+0000 7fb9caff9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9c415dce0 0x7fb9c4156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.590+0000 7fb9caff9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c40d3090 con 0x7fb9c40b5990 2024-04-25T10:05:11.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.590+0000 7fb9caff9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c40d2db0 con 0x7fb9c40d35b0 2024-04-25T10:05:11.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.590+0000 7fb9caff9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c40d2160 con 0x7fb9c415dce0 2024-04-25T10:05:11.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.590+0000 7fb9c8d95640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c40b5990 0x7fb9c40c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.590+0000 7fb9c8d95640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c40b5990 0x7fb9c40c9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57160/0 (socket says 172.21.15.16:57160) 2024-04-25T10:05:11.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.590+0000 7fb9c8d95640 1 -- 172.21.15.16:0/3438610592 learned_addr learned my addr 172.21.15.16:0/3438610592 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:11.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.590+0000 7fb9c3fff640 1 --2- 172.21.15.16:0/3438610592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9c40d35b0 0x7fb9c4156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.590+0000 7fb9c9596640 1 --2- 172.21.15.16:0/3438610592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9c415dce0 0x7fb9c4156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.591+0000 7fb9c8d95640 1 -- 172.21.15.16:0/3438610592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9c415dce0 msgr2=0x7fb9c4156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.591+0000 7fb9c8d95640 1 --2- 172.21.15.16:0/3438610592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9c415dce0 0x7fb9c4156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.591+0000 7fb9c8d95640 1 -- 172.21.15.16:0/3438610592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9c40d35b0 msgr2=0x7fb9c4156900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.591+0000 7fb9c8d95640 1 --2- 172.21.15.16:0/3438610592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9c40d35b0 0x7fb9c4156900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.591+0000 7fb9c8d95640 1 -- 172.21.15.16:0/3438610592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c415ef70 con 0x7fb9c40b5990 2024-04-25T10:05:11.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.591+0000 7fb9c8d95640 1 --2- 172.21.15.16:0/3438610592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c40b5990 0x7fb9c40c9ba0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800c4f0 tx=0x7fb9b800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.591+0000 7fb9c17fa640 1 -- 172.21.15.16:0/3438610592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8017070 con 0x7fb9c40b5990 2024-04-25T10:05:11.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.591+0000 7fb9c17fa640 1 -- 172.21.15.16:0/3438610592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b8005ce0 con 0x7fb9c40b5990 2024-04-25T10:05:11.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.591+0000 7fb9caff9640 1 -- 172.21.15.16:0/3438610592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c415f250 con 0x7fb9c40b5990 2024-04-25T10:05:11.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.591+0000 7fb9c17fa640 1 -- 172.21.15.16:0/3438610592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8007290 con 0x7fb9c40b5990 2024-04-25T10:05:11.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.591+0000 7fb9caff9640 1 -- 172.21.15.16:0/3438610592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c40c76a0 con 0x7fb9c40b5990 2024-04-25T10:05:11.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.592+0000 7fb9caff9640 1 -- 172.21.15.16:0/3438610592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c40749b0 con 0x7fb9c40b5990 2024-04-25T10:05:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.593+0000 7fb9c17fa640 1 -- 172.21.15.16:0/3438610592 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb9b801a070 con 0x7fb9c40b5990 2024-04-25T10:05:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.593+0000 7fb9c17fa640 1 --2- 172.21.15.16:0/3438610592 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb9a806b500 0x7fb9a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.593+0000 7fb9c17fa640 1 -- 172.21.15.16:0/3438610592 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9b8089540 con 0x7fb9c40b5990 2024-04-25T10:05:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.593+0000 7fb9c3fff640 1 --2- 172.21.15.16:0/3438610592 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb9a806b500 0x7fb9a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.594+0000 7fb9c3fff640 1 --2- 172.21.15.16:0/3438610592 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb9a806b500 0x7fb9a806d9c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0034000 tx=0x7fb9b002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.597+0000 7fb9c17fa640 1 -- 172.21.15.16:0/3438610592 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb9b8057e00 con 0x7fb9c40b5990 2024-04-25T10:05:11.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.777+0000 7fb9caff9640 1 -- 172.21.15.16:0/3438610592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9c40b73a0 con 0x7fb9c40b5990 2024-04-25T10:05:11.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.778+0000 7fb9c17fa640 1 -- 172.21.15.16:0/3438610592 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16622 (secure 0 0 0) 0x7fb9b805bea0 con 0x7fb9c40b5990 2024-04-25T10:05:11.784 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:11.784 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:10.813976+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.780+0000 7fb9caff9640 1 -- 172.21.15.16:0/3438610592 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb9a806b500 msgr2=0x7fb9a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.780+0000 7fb9caff9640 1 --2- 172.21.15.16:0/3438610592 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb9a806b500 0x7fb9a806d9c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0034000 tx=0x7fb9b002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.780+0000 7fb9caff9640 1 -- 172.21.15.16:0/3438610592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c40b5990 msgr2=0x7fb9c40c9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.780+0000 7fb9caff9640 1 --2- 172.21.15.16:0/3438610592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c40b5990 0x7fb9c40c9ba0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800c4f0 tx=0x7fb9b800c9b0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.780+0000 7fb9caff9640 1 -- 172.21.15.16:0/3438610592 shutdown_connections 2024-04-25T10:05:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.780+0000 7fb9caff9640 1 --2- 172.21.15.16:0/3438610592 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb9a806b500 0x7fb9a806d9c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.780+0000 7fb9caff9640 1 --2- 172.21.15.16:0/3438610592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9c415dce0 0x7fb9c4156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.780+0000 7fb9caff9640 1 --2- 172.21.15.16:0/3438610592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9c40d35b0 0x7fb9c4156900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.780+0000 7fb9caff9640 1 --2- 172.21.15.16:0/3438610592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c40b5990 0x7fb9c40c9ba0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.780+0000 7fb9caff9640 1 -- 172.21.15.16:0/3438610592 >> 172.21.15.16:0/3438610592 conn(0x7fb9c4072900 msgr2=0x7fb9c40b7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.780+0000 7fb9caff9640 1 -- 172.21.15.16:0/3438610592 shutdown_connections 2024-04-25T10:05:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.780+0000 7fb9caff9640 1 -- 172.21.15.16:0/3438610592 wait complete. 2024-04-25T10:05:11.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:11.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.892+0000 7f69d822a640 1 Processor -- start 2024-04-25T10:05:11.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.892+0000 7f69d822a640 1 -- start start 2024-04-25T10:05:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.893+0000 7f69d822a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69d015dce0 0x7f69d00d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.893+0000 7f69d822a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f69d00b5b60 0x7f69d00b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.893+0000 7f69d822a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d015e0c0 con 0x7f69d00d35b0 2024-04-25T10:05:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.893+0000 7f69d822a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d00d2510 con 0x7f69d00b5b60 2024-04-25T10:05:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.893+0000 7f69d822a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d00d27f0 con 0x7f69d015dce0 2024-04-25T10:05:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.893+0000 7f69d67c7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f69d00b5b60 0x7f69d00b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.893+0000 7f69d57c5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f69d00d35b0 0x7f69d00b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43260/0 (socket says 172.21.15.16:43260) 2024-04-25T10:05:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.893+0000 7f69d57c5640 1 -- 172.21.15.16:0/1365602544 learned_addr learned my addr 172.21.15.16:0/1365602544 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.893+0000 7f69d5fc6640 1 --2- 172.21.15.16:0/1365602544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69d015dce0 0x7f69d00d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.893+0000 7f69d67c7640 1 -- 172.21.15.16:0/1365602544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69d015dce0 msgr2=0x7f69d00d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.893+0000 7f69d67c7640 1 --2- 172.21.15.16:0/1365602544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69d015dce0 0x7f69d00d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.893+0000 7f69d67c7640 1 -- 172.21.15.16:0/1365602544 >> v1:172.21.15.16:6789/0 conn(0x7f69d00d35b0 legacy=0x7f69d00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.893+0000 7f69d67c7640 1 -- 172.21.15.16:0/1365602544 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d00d2ad0 con 0x7f69d00b5b60 2024-04-25T10:05:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.893+0000 7f69d5fc6640 1 --2- 172.21.15.16:0/1365602544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69d015dce0 0x7f69d00d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.894+0000 7f69d67c7640 1 --2- 172.21.15.16:0/1365602544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d00b5b60 0x7f69d00b6b00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009d90 tx=0x7f69c802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf63d462a8c3e0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.894+0000 7f69d4fc4640 1 -- 172.21.15.16:0/1365602544 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8007740 con 0x7f69d00b5b60 2024-04-25T10:05:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.894+0000 7f69d4fc4640 1 -- 172.21.15.16:0/1365602544 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c80078e0 con 0x7f69d00b5b60 2024-04-25T10:05:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.894+0000 7f69d4fc4640 1 -- 172.21.15.16:0/1365602544 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8007cb0 con 0x7f69d00b5b60 2024-04-25T10:05:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.894+0000 7f69d822a640 1 -- 172.21.15.16:0/1365602544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d00b5b60 msgr2=0x7f69d00b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.894+0000 7f69d822a640 1 --2- 172.21.15.16:0/1365602544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d00b5b60 0x7f69d00b6b00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009d90 tx=0x7f69c802a660 comp rx=0 tx=0).stop 2024-04-25T10:05:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.895+0000 7f69d822a640 1 -- 172.21.15.16:0/1365602544 shutdown_connections 2024-04-25T10:05:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.895+0000 7f69d822a640 1 --2- 172.21.15.16:0/1365602544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d00b5b60 0x7f69d00b6b00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.895+0000 7f69d822a640 1 --2- 172.21.15.16:0/1365602544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69d015dce0 0x7f69d00d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.895+0000 7f69d822a640 1 -- 172.21.15.16:0/1365602544 >> 172.21.15.16:0/1365602544 conn(0x7f69d0072900 msgr2=0x7f69d00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.895+0000 7f69d822a640 1 -- 172.21.15.16:0/1365602544 shutdown_connections 2024-04-25T10:05:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.895+0000 7f69d822a640 1 -- 172.21.15.16:0/1365602544 wait complete. 2024-04-25T10:05:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.896+0000 7f69d822a640 1 Processor -- start 2024-04-25T10:05:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.896+0000 7f69d822a640 1 -- start start 2024-04-25T10:05:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.896+0000 7f69d822a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69d00b5b60 0x7f69d01f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.896+0000 7f69d822a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d00d35b0 0x7f69d00ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.896+0000 7f69d822a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69d015dce0 0x7f69d00cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.896+0000 7f69d822a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d00b9820 con 0x7f69d00d35b0 2024-04-25T10:05:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69d822a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d00d3090 con 0x7f69d00b5b60 2024-04-25T10:05:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69d822a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d00b99a0 con 0x7f69d015dce0 2024-04-25T10:05:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69d57c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d00d35b0 0x7f69d00ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69d57c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d00d35b0 0x7f69d00ccf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57168/0 (socket says 172.21.15.16:57168) 2024-04-25T10:05:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69d57c5640 1 -- 172.21.15.16:0/3378650718 learned_addr learned my addr 172.21.15.16:0/3378650718 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69d67c7640 1 --2- 172.21.15.16:0/3378650718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69d015dce0 0x7f69d00cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69d5fc6640 1 --2- 172.21.15.16:0/3378650718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69d00b5b60 0x7f69d01f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69d57c5640 1 -- 172.21.15.16:0/3378650718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69d015dce0 msgr2=0x7f69d00cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69d57c5640 1 --2- 172.21.15.16:0/3378650718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69d015dce0 0x7f69d00cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69d57c5640 1 -- 172.21.15.16:0/3378650718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69d00b5b60 msgr2=0x7f69d01f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69d57c5640 1 --2- 172.21.15.16:0/3378650718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69d00b5b60 0x7f69d01f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69d57c5640 1 -- 172.21.15.16:0/3378650718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d00cac20 con 0x7f69d00d35b0 2024-04-25T10:05:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69d67c7640 1 --2- 172.21.15.16:0/3378650718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69d015dce0 0x7f69d00cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69d5fc6640 1 --2- 172.21.15.16:0/3378650718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69d00b5b60 0x7f69d01f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69d57c5640 1 --2- 172.21.15.16:0/3378650718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d00d35b0 0x7f69d00ccf20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f69c000c430 tx=0x7f69c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69c67fc640 1 -- 172.21.15.16:0/3378650718 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0017070 con 0x7f69d00d35b0 2024-04-25T10:05:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69c67fc640 1 -- 172.21.15.16:0/3378650718 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c0005ce0 con 0x7f69d00d35b0 2024-04-25T10:05:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69d822a640 1 -- 172.21.15.16:0/3378650718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d0160230 con 0x7f69d00d35b0 2024-04-25T10:05:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69c67fc640 1 -- 172.21.15.16:0/3378650718 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c00072f0 con 0x7f69d00d35b0 2024-04-25T10:05:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.897+0000 7f69d822a640 1 -- 172.21.15.16:0/3378650718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d01f4320 con 0x7f69d00d35b0 2024-04-25T10:05:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.898+0000 7f69d822a640 1 -- 172.21.15.16:0/3378650718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d01f45e0 con 0x7f69d00d35b0 2024-04-25T10:05:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.899+0000 7f69c67fc640 1 -- 172.21.15.16:0/3378650718 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69c001a070 con 0x7f69d00d35b0 2024-04-25T10:05:11.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.899+0000 7f69c67fc640 1 --2- 172.21.15.16:0/3378650718 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f69b406b4d0 0x7f69b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.899+0000 7f69c67fc640 1 -- 172.21.15.16:0/3378650718 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69c00896c0 con 0x7f69d00d35b0 2024-04-25T10:05:11.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.900+0000 7f69d5fc6640 1 --2- 172.21.15.16:0/3378650718 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f69b406b4d0 0x7f69b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.900+0000 7f69d5fc6640 1 --2- 172.21.15.16:0/3378650718 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f69b406b4d0 0x7f69b406d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00a990 tx=0x7f69cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:11.904+0000 7f69c67fc640 1 -- 172.21.15.16:0/3378650718 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69c00584a0 con 0x7f69d00d35b0 2024-04-25T10:05:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.088+0000 7f69d822a640 1 -- 172.21.15.16:0/3378650718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69d00749b0 con 0x7f69d00d35b0 2024-04-25T10:05:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.089+0000 7f69c67fc640 1 -- 172.21.15.16:0/3378650718 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16622 (secure 0 0 0) 0x7f69c0057da0 con 0x7f69d00d35b0 2024-04-25T10:05:12.095 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:12.095 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:10.813976+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.091+0000 7f69d822a640 1 -- 172.21.15.16:0/3378650718 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f69b406b4d0 msgr2=0x7f69b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.091+0000 7f69d822a640 1 --2- 172.21.15.16:0/3378650718 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f69b406b4d0 0x7f69b406d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00a990 tx=0x7f69cc008040 comp rx=0 tx=0).stop 2024-04-25T10:05:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.091+0000 7f69d822a640 1 -- 172.21.15.16:0/3378650718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d00d35b0 msgr2=0x7f69d00ccf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.091+0000 7f69d822a640 1 --2- 172.21.15.16:0/3378650718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d00d35b0 0x7f69d00ccf20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f69c000c430 tx=0x7f69c000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.091+0000 7f69d822a640 1 -- 172.21.15.16:0/3378650718 shutdown_connections 2024-04-25T10:05:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.091+0000 7f69d822a640 1 --2- 172.21.15.16:0/3378650718 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f69b406b4d0 0x7f69b406d990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.091+0000 7f69d822a640 1 --2- 172.21.15.16:0/3378650718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69d015dce0 0x7f69d00cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.091+0000 7f69d822a640 1 --2- 172.21.15.16:0/3378650718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69d00d35b0 0x7f69d00ccf20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.091+0000 7f69d822a640 1 --2- 172.21.15.16:0/3378650718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69d00b5b60 0x7f69d01f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.091+0000 7f69d822a640 1 -- 172.21.15.16:0/3378650718 >> 172.21.15.16:0/3378650718 conn(0x7f69d0072900 msgr2=0x7f69d006bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.091+0000 7f69d822a640 1 -- 172.21.15.16:0/3378650718 shutdown_connections 2024-04-25T10:05:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.091+0000 7f69d822a640 1 -- 172.21.15.16:0/3378650718 wait complete. 2024-04-25T10:05:12.107 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:05:12.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.205+0000 7f2d89e6f640 1 Processor -- start 2024-04-25T10:05:12.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.206+0000 7f2d89e6f640 1 -- start start 2024-04-25T10:05:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.206+0000 7f2d89e6f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d840cd5f0 0x7f2d8415a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.206+0000 7f2d89e6f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d8415a990 0x7f2d840cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.206+0000 7f2d89e6f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d840cf870 con 0x7f2d840cd210 2024-04-25T10:05:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.206+0000 7f2d89e6f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d840cec60 con 0x7f2d8415a990 2024-04-25T10:05:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.206+0000 7f2d89e6f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d840cef40 con 0x7f2d840cd5f0 2024-04-25T10:05:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.206+0000 7f2d83fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d8415a990 0x7f2d840cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.206+0000 7f2d83fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d8415a990 0x7f2d840cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57180/0 (socket says 172.21.15.16:57180) 2024-04-25T10:05:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.207+0000 7f2d83fff640 1 -- 172.21.15.16:0/1823219283 learned_addr learned my addr 172.21.15.16:0/1823219283 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:12.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.207+0000 7f2d82ffd640 1 --2- 172.21.15.16:0/1823219283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d840cd5f0 0x7f2d8415a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.207+0000 7f2d83fff640 1 -- 172.21.15.16:0/1823219283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d840cd5f0 msgr2=0x7f2d8415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.207+0000 7f2d83fff640 1 --2- 172.21.15.16:0/1823219283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d840cd5f0 0x7f2d8415a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.207+0000 7f2d83fff640 1 -- 172.21.15.16:0/1823219283 >> v1:172.21.15.186:6789/0 conn(0x7f2d840cd210 legacy=0x7f2d8415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.207+0000 7f2d83fff640 1 -- 172.21.15.16:0/1823219283 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d840cf220 con 0x7f2d8415a990 2024-04-25T10:05:12.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.207+0000 7f2d83fff640 1 --2- 172.21.15.16:0/1823219283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8415a990 0x7f2d840cb7f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009e50 tx=0x7f2d7c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e06aef23976210e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.208+0000 7f2d827fc640 1 -- 172.21.15.16:0/1823219283 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c036070 con 0x7f2d8415a990 2024-04-25T10:05:12.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.208+0000 7f2d827fc640 1 -- 172.21.15.16:0/1823219283 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7c031600 con 0x7f2d8415a990 2024-04-25T10:05:12.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.208+0000 7f2d827fc640 1 -- 172.21.15.16:0/1823219283 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c0319d0 con 0x7f2d8415a990 2024-04-25T10:05:12.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.208+0000 7f2d89e6f640 1 -- 172.21.15.16:0/1823219283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8415a990 msgr2=0x7f2d840cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.208+0000 7f2d89e6f640 1 --2- 172.21.15.16:0/1823219283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8415a990 0x7f2d840cb7f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009e50 tx=0x7f2d7c02a6c0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.208+0000 7f2d89e6f640 1 -- 172.21.15.16:0/1823219283 shutdown_connections 2024-04-25T10:05:12.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.208+0000 7f2d89e6f640 1 --2- 172.21.15.16:0/1823219283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8415a990 0x7f2d840cb7f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.208+0000 7f2d89e6f640 1 --2- 172.21.15.16:0/1823219283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d840cd5f0 0x7f2d8415a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.208+0000 7f2d89e6f640 1 -- 172.21.15.16:0/1823219283 >> 172.21.15.16:0/1823219283 conn(0x7f2d84099a40 msgr2=0x7f2d840d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.208+0000 7f2d89e6f640 1 -- 172.21.15.16:0/1823219283 shutdown_connections 2024-04-25T10:05:12.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.208+0000 7f2d89e6f640 1 -- 172.21.15.16:0/1823219283 wait complete. 2024-04-25T10:05:12.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.209+0000 7f2d89e6f640 1 Processor -- start 2024-04-25T10:05:12.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.210+0000 7f2d89e6f640 1 -- start start 2024-04-25T10:05:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.210+0000 7f2d89e6f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d840cd210 0x7f2d84147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.210+0000 7f2d89e6f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d840cd5f0 0x7f2d840c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.210+0000 7f2d89e6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8415a990 0x7f2d840c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.210+0000 7f2d89e6f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8415ae20 con 0x7f2d8415a990 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.210+0000 7f2d89e6f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d840d1430 con 0x7f2d840cd210 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.210+0000 7f2d89e6f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d840cda30 con 0x7f2d840cd5f0 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.210+0000 7f2d83fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8415a990 0x7f2d840c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.211+0000 7f2d83fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8415a990 0x7f2d840c46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57182/0 (socket says 172.21.15.16:57182) 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.211+0000 7f2d83fff640 1 -- 172.21.15.16:0/493449944 learned_addr learned my addr 172.21.15.16:0/493449944 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.211+0000 7f2d837fe640 1 --2- 172.21.15.16:0/493449944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d840cd210 0x7f2d84147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.211+0000 7f2d82ffd640 1 --2- 172.21.15.16:0/493449944 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d840cd5f0 0x7f2d840c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.211+0000 7f2d83fff640 1 -- 172.21.15.16:0/493449944 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d840cd5f0 msgr2=0x7f2d840c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.211+0000 7f2d83fff640 1 --2- 172.21.15.16:0/493449944 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d840cd5f0 0x7f2d840c41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.211+0000 7f2d83fff640 1 -- 172.21.15.16:0/493449944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d840cd210 msgr2=0x7f2d84147320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.211+0000 7f2d83fff640 1 --2- 172.21.15.16:0/493449944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d840cd210 0x7f2d84147320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.211+0000 7f2d83fff640 1 -- 172.21.15.16:0/493449944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d840c7030 con 0x7f2d8415a990 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.211+0000 7f2d83fff640 1 --2- 172.21.15.16:0/493449944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8415a990 0x7f2d840c46f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c034040 tx=0x7f2d7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.211+0000 7f2d6bfff640 1 -- 172.21.15.16:0/493449944 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c041020 con 0x7f2d8415a990 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.211+0000 7f2d6bfff640 1 -- 172.21.15.16:0/493449944 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7c0077a0 con 0x7f2d8415a990 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.211+0000 7f2d89e6f640 1 -- 172.21.15.16:0/493449944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d840c7310 con 0x7f2d8415a990 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.211+0000 7f2d6bfff640 1 -- 172.21.15.16:0/493449944 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c036040 con 0x7f2d8415a990 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.211+0000 7f2d89e6f640 1 -- 172.21.15.16:0/493449944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d84147d70 con 0x7f2d8415a990 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.212+0000 7f2d89e6f640 1 -- 172.21.15.16:0/493449944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d840750b0 con 0x7f2d8415a990 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.213+0000 7f2d6bfff640 1 -- 172.21.15.16:0/493449944 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d7c044020 con 0x7f2d8415a990 2024-04-25T10:05:12.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.213+0000 7f2d6bfff640 1 --2- 172.21.15.16:0/493449944 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2d7006b4d0 0x7f2d7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.213+0000 7f2d6bfff640 1 -- 172.21.15.16:0/493449944 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d7c0aae70 con 0x7f2d8415a990 2024-04-25T10:05:12.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.213+0000 7f2d837fe640 1 --2- 172.21.15.16:0/493449944 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2d7006b4d0 0x7f2d7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.214+0000 7f2d837fe640 1 --2- 172.21.15.16:0/493449944 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2d7006b4d0 0x7f2d7006d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2d74006fd0 tx=0x7f2d74008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.218+0000 7f2d6bfff640 1 -- 172.21.15.16:0/493449944 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d7c04b030 con 0x7f2d8415a990 2024-04-25T10:05:12.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.435+0000 7f2d89e6f640 1 -- 172.21.15.16:0/493449944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2d840cc900 con 0x7f2d8415a990 2024-04-25T10:05:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.452+0000 7f2d6bfff640 1 -- 172.21.15.16:0/493449944 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2d840cf870 con 0x7f2d8415a990 2024-04-25T10:05:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.455+0000 7f2d89e6f640 1 -- 172.21.15.16:0/493449944 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2d7006b4d0 msgr2=0x7f2d7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.455+0000 7f2d89e6f640 1 --2- 172.21.15.16:0/493449944 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2d7006b4d0 0x7f2d7006d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2d74006fd0 tx=0x7f2d74008040 comp rx=0 tx=0).stop 2024-04-25T10:05:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.455+0000 7f2d89e6f640 1 -- 172.21.15.16:0/493449944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8415a990 msgr2=0x7f2d840c46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.455+0000 7f2d89e6f640 1 --2- 172.21.15.16:0/493449944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8415a990 0x7f2d840c46f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c034040 tx=0x7f2d7c004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.455+0000 7f2d89e6f640 1 -- 172.21.15.16:0/493449944 shutdown_connections 2024-04-25T10:05:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.455+0000 7f2d89e6f640 1 --2- 172.21.15.16:0/493449944 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2d7006b4d0 0x7f2d7006d990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.455+0000 7f2d89e6f640 1 --2- 172.21.15.16:0/493449944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8415a990 0x7f2d840c46f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.455+0000 7f2d89e6f640 1 --2- 172.21.15.16:0/493449944 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d840cd5f0 0x7f2d840c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.455+0000 7f2d89e6f640 1 --2- 172.21.15.16:0/493449944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d840cd210 0x7f2d84147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.455+0000 7f2d89e6f640 1 -- 172.21.15.16:0/493449944 >> 172.21.15.16:0/493449944 conn(0x7f2d84099a40 msgr2=0x7f2d8415a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.456+0000 7f2d89e6f640 1 -- 172.21.15.16:0/493449944 shutdown_connections 2024-04-25T10:05:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.456+0000 7f2d89e6f640 1 -- 172.21.15.16:0/493449944 wait complete. 2024-04-25T10:05:12.471 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:05:12.471 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:05:12.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.570+0000 7f658d607640 1 Processor -- start 2024-04-25T10:05:12.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.570+0000 7f658d607640 1 -- start start 2024-04-25T10:05:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.570+0000 7f658d607640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f658815db50 0x7f65880cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.570+0000 7f658d607640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65880d12a0 0x7f6588099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.570+0000 7f658d607640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65880d0200 con 0x7f658815df30 2024-04-25T10:05:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.570+0000 7f658d607640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f658806c020 con 0x7f658815db50 2024-04-25T10:05:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.570+0000 7f658d607640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65880d04e0 con 0x7f65880d12a0 2024-04-25T10:05:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.571+0000 7f6586ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f658815db50 0x7f65880cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.571+0000 7f6586ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f658815db50 0x7f65880cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57198/0 (socket says 172.21.15.16:57198) 2024-04-25T10:05:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.571+0000 7f6586ffd640 1 -- 172.21.15.16:0/3258402349 learned_addr learned my addr 172.21.15.16:0/3258402349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.571+0000 7f65877fe640 1 --2- 172.21.15.16:0/3258402349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65880d12a0 0x7f6588099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.571+0000 7f6586ffd640 1 -- 172.21.15.16:0/3258402349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65880d12a0 msgr2=0x7f6588099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.571+0000 7f6586ffd640 1 --2- 172.21.15.16:0/3258402349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65880d12a0 0x7f6588099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.571+0000 7f6586ffd640 1 -- 172.21.15.16:0/3258402349 >> v1:172.21.15.186:6789/0 conn(0x7f658815df30 legacy=0x7f658806cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.571+0000 7f6586ffd640 1 -- 172.21.15.16:0/3258402349 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65880d07c0 con 0x7f658815db50 2024-04-25T10:05:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.571+0000 7f65877fe640 1 --2- 172.21.15.16:0/3258402349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65880d12a0 0x7f6588099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.572+0000 7f6586ffd640 1 --2- 172.21.15.16:0/3258402349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f658815db50 0x7f65880cfb80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6578009870 tx=0x7f657802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df79987f5232c913 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.572+0000 7f65867fc640 1 -- 172.21.15.16:0/3258402349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6578037070 con 0x7f658815db50 2024-04-25T10:05:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.572+0000 7f65867fc640 1 -- 172.21.15.16:0/3258402349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6578032440 con 0x7f658815db50 2024-04-25T10:05:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.572+0000 7f65867fc640 1 -- 172.21.15.16:0/3258402349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6578032810 con 0x7f658815db50 2024-04-25T10:05:12.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.572+0000 7f658d607640 1 -- 172.21.15.16:0/3258402349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f658815db50 msgr2=0x7f65880cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.572+0000 7f658d607640 1 --2- 172.21.15.16:0/3258402349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f658815db50 0x7f65880cfb80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6578009870 tx=0x7f657802a180 comp rx=0 tx=0).stop 2024-04-25T10:05:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.572+0000 7f658d607640 1 -- 172.21.15.16:0/3258402349 shutdown_connections 2024-04-25T10:05:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.572+0000 7f658d607640 1 --2- 172.21.15.16:0/3258402349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65880d12a0 0x7f6588099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.572+0000 7f658d607640 1 --2- 172.21.15.16:0/3258402349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f658815db50 0x7f65880cfb80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.572+0000 7f658d607640 1 -- 172.21.15.16:0/3258402349 >> 172.21.15.16:0/3258402349 conn(0x7f65880b3430 msgr2=0x7f65880b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.572+0000 7f658d607640 1 -- 172.21.15.16:0/3258402349 shutdown_connections 2024-04-25T10:05:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.572+0000 7f658d607640 1 -- 172.21.15.16:0/3258402349 wait complete. 2024-04-25T10:05:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.573+0000 7f658d607640 1 Processor -- start 2024-04-25T10:05:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.573+0000 7f658d607640 1 -- start start 2024-04-25T10:05:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.574+0000 7f658d607640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65880d12a0 0x7f6588151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.574+0000 7f658d607640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f658815db50 0x7f65880c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.574+0000 7f658d607640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f658815df30 0x7f6588158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.574+0000 7f658d607640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6588099f80 con 0x7f658815df30 2024-04-25T10:05:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.574+0000 7f658d607640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65880d0d80 con 0x7f658815db50 2024-04-25T10:05:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.574+0000 7f658d607640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658809a100 con 0x7f65880d12a0 2024-04-25T10:05:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.574+0000 7f65877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f658815df30 0x7f6588158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.574+0000 7f65877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f658815df30 0x7f6588158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57200/0 (socket says 172.21.15.16:57200) 2024-04-25T10:05:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.574+0000 7f65877fe640 1 -- 172.21.15.16:0/1004938280 learned_addr learned my addr 172.21.15.16:0/1004938280 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.574+0000 7f657ffff640 1 --2- 172.21.15.16:0/1004938280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f658815db50 0x7f65880c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.574+0000 7f6586ffd640 1 --2- 172.21.15.16:0/1004938280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65880d12a0 0x7f6588151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.575+0000 7f65877fe640 1 -- 172.21.15.16:0/1004938280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65880d12a0 msgr2=0x7f6588151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.575+0000 7f65877fe640 1 --2- 172.21.15.16:0/1004938280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65880d12a0 0x7f6588151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.575+0000 7f65877fe640 1 -- 172.21.15.16:0/1004938280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f658815db50 msgr2=0x7f65880c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.575+0000 7f65877fe640 1 --2- 172.21.15.16:0/1004938280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f658815db50 0x7f65880c6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.575+0000 7f65877fe640 1 -- 172.21.15.16:0/1004938280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65881f7a40 con 0x7f658815df30 2024-04-25T10:05:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.575+0000 7f65877fe640 1 --2- 172.21.15.16:0/1004938280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f658815df30 0x7f6588158b70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f658000cef0 tx=0x7f658000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.575+0000 7f657f7fe640 1 -- 172.21.15.16:0/1004938280 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658001a070 con 0x7f658815df30 2024-04-25T10:05:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.575+0000 7f657f7fe640 1 -- 172.21.15.16:0/1004938280 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658000d860 con 0x7f658815df30 2024-04-25T10:05:12.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.575+0000 7f658d607640 1 -- 172.21.15.16:0/1004938280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65881f7d20 con 0x7f658815df30 2024-04-25T10:05:12.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.575+0000 7f657f7fe640 1 -- 172.21.15.16:0/1004938280 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658000dc50 con 0x7f658815df30 2024-04-25T10:05:12.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.575+0000 7f658d607640 1 -- 172.21.15.16:0/1004938280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65881f82e0 con 0x7f658815df30 2024-04-25T10:05:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.576+0000 7f658d607640 1 -- 172.21.15.16:0/1004938280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65880750b0 con 0x7f658815df30 2024-04-25T10:05:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.576+0000 7f657f7fe640 1 -- 172.21.15.16:0/1004938280 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f658001d070 con 0x7f658815df30 2024-04-25T10:05:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.577+0000 7f657f7fe640 1 --2- 172.21.15.16:0/1004938280 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f656806b4d0 0x7f656806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.577+0000 7f657f7fe640 1 -- 172.21.15.16:0/1004938280 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f658008aa80 con 0x7f658815df30 2024-04-25T10:05:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.577+0000 7f6586ffd640 1 --2- 172.21.15.16:0/1004938280 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f656806b4d0 0x7f656806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.578+0000 7f6586ffd640 1 --2- 172.21.15.16:0/1004938280 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f656806b4d0 0x7f656806d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6578034000 tx=0x7f657802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.581+0000 7f657f7fe640 1 -- 172.21.15.16:0/1004938280 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65800599c0 con 0x7f658815df30 2024-04-25T10:05:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:12.803+0000 7f658d607640 1 -- 172.21.15.16:0/1004938280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f658809aed0 con 0x7f658815df30 2024-04-25T10:05:13.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.458+0000 7f657f7fe640 1 -- 172.21.15.16:0/1004938280 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f65800592c0 con 0x7f658815df30 2024-04-25T10:05:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.460+0000 7f658d607640 1 -- 172.21.15.16:0/1004938280 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f656806b4d0 msgr2=0x7f656806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.460+0000 7f658d607640 1 --2- 172.21.15.16:0/1004938280 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f656806b4d0 0x7f656806d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6578034000 tx=0x7f657802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.460+0000 7f658d607640 1 -- 172.21.15.16:0/1004938280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f658815df30 msgr2=0x7f6588158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.460+0000 7f658d607640 1 --2- 172.21.15.16:0/1004938280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f658815df30 0x7f6588158b70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f658000cef0 tx=0x7f658000a710 comp rx=0 tx=0).stop 2024-04-25T10:05:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.460+0000 7f658d607640 1 -- 172.21.15.16:0/1004938280 shutdown_connections 2024-04-25T10:05:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.460+0000 7f658d607640 1 --2- 172.21.15.16:0/1004938280 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f656806b4d0 0x7f656806d990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.460+0000 7f658d607640 1 --2- 172.21.15.16:0/1004938280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f658815df30 0x7f6588158b70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.460+0000 7f658d607640 1 --2- 172.21.15.16:0/1004938280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f658815db50 0x7f65880c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.460+0000 7f658d607640 1 --2- 172.21.15.16:0/1004938280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65880d12a0 0x7f6588151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.461+0000 7f658d607640 1 -- 172.21.15.16:0/1004938280 >> 172.21.15.16:0/1004938280 conn(0x7f65880b3430 msgr2=0x7f6588070310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.461+0000 7f658d607640 1 -- 172.21.15.16:0/1004938280 shutdown_connections 2024-04-25T10:05:13.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.461+0000 7f658d607640 1 -- 172.21.15.16:0/1004938280 wait complete. 2024-04-25T10:05:13.478 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:05:13.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.581+0000 7fb95870a640 1 Processor -- start 2024-04-25T10:05:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.581+0000 7fb95870a640 1 -- start start 2024-04-25T10:05:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.582+0000 7fb95870a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9500d35b0 0x7fb9500b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.582+0000 7fb95870a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb9500b5990 0x7fb9500b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.582+0000 7fb95870a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95015e0c0 con 0x7fb95015dce0 2024-04-25T10:05:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.582+0000 7fb95870a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9500b5de0 con 0x7fb9500b5990 2024-04-25T10:05:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.582+0000 7fb95870a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9500d2510 con 0x7fb9500d35b0 2024-04-25T10:05:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.582+0000 7fb956ca7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb9500b5990 0x7fb9500b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.582+0000 7fb956ca7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb9500b5990 0x7fb9500b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41710/0 (socket says 172.21.15.16:41710) 2024-04-25T10:05:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.582+0000 7fb956ca7640 1 -- 172.21.15.16:0/898880972 learned_addr learned my addr 172.21.15.16:0/898880972 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.582+0000 7fb955ca5640 1 --2- 172.21.15.16:0/898880972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9500d35b0 0x7fb9500b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.582+0000 7fb956ca7640 1 -- 172.21.15.16:0/898880972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9500d35b0 msgr2=0x7fb9500b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.582+0000 7fb956ca7640 1 --2- 172.21.15.16:0/898880972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9500d35b0 0x7fb9500b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.582+0000 7fb956ca7640 1 -- 172.21.15.16:0/898880972 >> v1:172.21.15.186:6789/0 conn(0x7fb95015dce0 legacy=0x7fb9500d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.582+0000 7fb956ca7640 1 -- 172.21.15.16:0/898880972 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9500d27f0 con 0x7fb9500b5990 2024-04-25T10:05:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.582+0000 7fb955ca5640 1 --2- 172.21.15.16:0/898880972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9500d35b0 0x7fb9500b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.583+0000 7fb956ca7640 1 --2- 172.21.15.16:0/898880972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9500b5990 0x7fb9500b6b00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb948009e50 tx=0x7fb94802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f362f4867dbf4e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.583+0000 7fb9554a4640 1 -- 172.21.15.16:0/898880972 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948036070 con 0x7fb9500b5990 2024-04-25T10:05:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.583+0000 7fb9554a4640 1 -- 172.21.15.16:0/898880972 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb948031600 con 0x7fb9500b5990 2024-04-25T10:05:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.583+0000 7fb9554a4640 1 -- 172.21.15.16:0/898880972 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9480319d0 con 0x7fb9500b5990 2024-04-25T10:05:13.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.583+0000 7fb95870a640 1 -- 172.21.15.16:0/898880972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9500b5990 msgr2=0x7fb9500b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.583+0000 7fb95870a640 1 --2- 172.21.15.16:0/898880972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9500b5990 0x7fb9500b6b00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb948009e50 tx=0x7fb94802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.583+0000 7fb95870a640 1 -- 172.21.15.16:0/898880972 shutdown_connections 2024-04-25T10:05:13.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.583+0000 7fb95870a640 1 --2- 172.21.15.16:0/898880972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9500b5990 0x7fb9500b6b00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.583+0000 7fb95870a640 1 --2- 172.21.15.16:0/898880972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9500d35b0 0x7fb9500b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.584+0000 7fb95870a640 1 -- 172.21.15.16:0/898880972 >> 172.21.15.16:0/898880972 conn(0x7fb950072900 msgr2=0x7fb9500d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.584+0000 7fb95870a640 1 -- 172.21.15.16:0/898880972 shutdown_connections 2024-04-25T10:05:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.584+0000 7fb95870a640 1 -- 172.21.15.16:0/898880972 wait complete. 2024-04-25T10:05:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.585+0000 7fb95870a640 1 Processor -- start 2024-04-25T10:05:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.585+0000 7fb95870a640 1 -- start start 2024-04-25T10:05:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.585+0000 7fb95870a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9500b5990 0x7fb9500c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.585+0000 7fb95870a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9500d35b0 0x7fb9500cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.585+0000 7fb95870a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb95015dce0 0x7fb9501517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.585+0000 7fb95870a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9500d3090 con 0x7fb9500d35b0 2024-04-25T10:05:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.585+0000 7fb95870a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9500d2db0 con 0x7fb95015dce0 2024-04-25T10:05:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.585+0000 7fb95870a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9500d2160 con 0x7fb9500b5990 2024-04-25T10:05:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.586+0000 7fb955ca5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9500d35b0 0x7fb9500cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.586+0000 7fb955ca5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9500d35b0 0x7fb9500cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41724/0 (socket says 172.21.15.16:41724) 2024-04-25T10:05:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.586+0000 7fb955ca5640 1 -- 172.21.15.16:0/3522249977 learned_addr learned my addr 172.21.15.16:0/3522249977 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.586+0000 7fb9564a6640 1 --2- 172.21.15.16:0/3522249977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9500b5990 0x7fb9500c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.586+0000 7fb956ca7640 1 --2- 172.21.15.16:0/3522249977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb95015dce0 0x7fb9501517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.586+0000 7fb955ca5640 1 -- 172.21.15.16:0/3522249977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9500b5990 msgr2=0x7fb9500c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.586+0000 7fb955ca5640 1 --2- 172.21.15.16:0/3522249977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9500b5990 0x7fb9500c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.586+0000 7fb955ca5640 1 -- 172.21.15.16:0/3522249977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb95015dce0 msgr2=0x7fb9501517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.586+0000 7fb955ca5640 1 --2- 172.21.15.16:0/3522249977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb95015dce0 0x7fb9501517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.586+0000 7fb955ca5640 1 -- 172.21.15.16:0/3522249977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9500cd690 con 0x7fb9500d35b0 2024-04-25T10:05:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.586+0000 7fb956ca7640 1 --2- 172.21.15.16:0/3522249977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb95015dce0 0x7fb9501517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.586+0000 7fb955ca5640 1 --2- 172.21.15.16:0/3522249977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9500d35b0 0x7fb9500cd150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb94000c7f0 tx=0x7fb94000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.586+0000 7fb946ffd640 1 -- 172.21.15.16:0/3522249977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb940015070 con 0x7fb9500d35b0 2024-04-25T10:05:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.586+0000 7fb946ffd640 1 -- 172.21.15.16:0/3522249977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb940007a30 con 0x7fb9500d35b0 2024-04-25T10:05:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.586+0000 7fb946ffd640 1 -- 172.21.15.16:0/3522249977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb940007e00 con 0x7fb9500d35b0 2024-04-25T10:05:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.586+0000 7fb95870a640 1 -- 172.21.15.16:0/3522249977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9501f7010 con 0x7fb9500d35b0 2024-04-25T10:05:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.586+0000 7fb95870a640 1 -- 172.21.15.16:0/3522249977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9501f75d0 con 0x7fb9500d35b0 2024-04-25T10:05:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.587+0000 7fb95870a640 1 -- 172.21.15.16:0/3522249977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95015d540 con 0x7fb9500d35b0 2024-04-25T10:05:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.588+0000 7fb946ffd640 1 -- 172.21.15.16:0/3522249977 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb940018070 con 0x7fb9500d35b0 2024-04-25T10:05:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.588+0000 7fb946ffd640 1 --2- 172.21.15.16:0/3522249977 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb93406b4d0 0x7fb93406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.588+0000 7fb946ffd640 1 -- 172.21.15.16:0/3522249977 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb94008a120 con 0x7fb9500d35b0 2024-04-25T10:05:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.588+0000 7fb9564a6640 1 --2- 172.21.15.16:0/3522249977 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb93406b4d0 0x7fb93406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.589+0000 7fb9564a6640 1 --2- 172.21.15.16:0/3522249977 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb93406b4d0 0x7fb93406d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb94c006fd0 tx=0x7fb94c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.592+0000 7fb946ffd640 1 -- 172.21.15.16:0/3522249977 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb95015d540 con 0x7fb9500d35b0 2024-04-25T10:05:13.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.810+0000 7fb95870a640 1 -- 172.21.15.16:0/3522249977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9500cf5f0 con 0x7fb9500d35b0 2024-04-25T10:05:13.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.810+0000 7fb946ffd640 1 -- 172.21.15.16:0/3522249977 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4939 (secure 0 0 0) 0x7fb9500cf5f0 con 0x7fb9500d35b0 2024-04-25T10:05:13.815 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:13.815 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1965168813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1965168813},{"type":"v1","addr":"172.21.15.16:6834","nonce":1965168813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/437791378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":437791378},{"type":"v1","addr":"172.21.15.16:6835","nonce":437791378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34145,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3044177642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3044177642},{"type":"v1","addr":"172.21.15.186:6835","nonce":3044177642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:10.814281+0000","modified":"2024-04-25T10:05:12.805138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34148":{"gid":34148,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/1763853189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1763853189},{"type":"v1","addr":"172.21.15.186:6837","nonce":1763853189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:05:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-04-25T10:05:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.812+0000 7fb95870a640 1 -- 172.21.15.16:0/3522249977 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb93406b4d0 msgr2=0x7fb93406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.812+0000 7fb95870a640 1 --2- 172.21.15.16:0/3522249977 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb93406b4d0 0x7fb93406d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb94c006fd0 tx=0x7fb94c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:13.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.812+0000 7fb95870a640 1 -- 172.21.15.16:0/3522249977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9500d35b0 msgr2=0x7fb9500cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.812+0000 7fb95870a640 1 --2- 172.21.15.16:0/3522249977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9500d35b0 0x7fb9500cd150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb94000c7f0 tx=0x7fb94000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.812+0000 7fb95870a640 1 -- 172.21.15.16:0/3522249977 shutdown_connections 2024-04-25T10:05:13.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.812+0000 7fb95870a640 1 --2- 172.21.15.16:0/3522249977 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb93406b4d0 0x7fb93406d990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.812+0000 7fb95870a640 1 --2- 172.21.15.16:0/3522249977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb95015dce0 0x7fb9501517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.812+0000 7fb95870a640 1 --2- 172.21.15.16:0/3522249977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9500d35b0 0x7fb9500cd150 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.812+0000 7fb95870a640 1 --2- 172.21.15.16:0/3522249977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9500b5990 0x7fb9500c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.812+0000 7fb95870a640 1 -- 172.21.15.16:0/3522249977 >> 172.21.15.16:0/3522249977 conn(0x7fb950072900 msgr2=0x7fb9500c6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.812+0000 7fb95870a640 1 -- 172.21.15.16:0/3522249977 shutdown_connections 2024-04-25T10:05:13.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.812+0000 7fb95870a640 1 -- 172.21.15.16:0/3522249977 wait complete. 2024-04-25T10:05:13.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.923+0000 7fccb56cb640 1 Processor -- start 2024-04-25T10:05:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.923+0000 7fccb56cb640 1 -- start start 2024-04-25T10:05:13.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.924+0000 7fccb56cb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fccb00d35b0 0x7fccb00b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.924+0000 7fccb56cb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fccb00b5990 0x7fccb00b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.924+0000 7fccb56cb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb015e0c0 con 0x7fccb015dce0 2024-04-25T10:05:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.924+0000 7fccb56cb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb00d2510 con 0x7fccb00d35b0 2024-04-25T10:05:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.924+0000 7fccb56cb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb00d27f0 con 0x7fccb00b5990 2024-04-25T10:05:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.924+0000 7fccae7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fccb00d35b0 0x7fccb00b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.924+0000 7fccaeffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fccb015dce0 0x7fccb00d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34780/0 (socket says 172.21.15.16:34780) 2024-04-25T10:05:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.924+0000 7fccaeffd640 1 -- 172.21.15.16:0/1568932540 learned_addr learned my addr 172.21.15.16:0/1568932540 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.924+0000 7fccaf7fe640 1 --2- 172.21.15.16:0/1568932540 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fccb00b5990 0x7fccb00b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.924+0000 7fccae7fc640 1 -- 172.21.15.16:0/1568932540 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fccb00b5990 msgr2=0x7fccb00b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.924+0000 7fccae7fc640 1 --2- 172.21.15.16:0/1568932540 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fccb00b5990 0x7fccb00b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.924+0000 7fccae7fc640 1 -- 172.21.15.16:0/1568932540 >> v1:172.21.15.16:6789/0 conn(0x7fccb015dce0 legacy=0x7fccb00d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.925+0000 7fccae7fc640 1 -- 172.21.15.16:0/1568932540 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb00d2ad0 con 0x7fccb00d35b0 2024-04-25T10:05:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.924+0000 7fccadffb640 1 -- 172.21.15.16:0/1568932540 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977925807 0 0) 0x7fccb015e0c0 con 0x7fccb015dce0 2024-04-25T10:05:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.925+0000 7fccaf7fe640 1 --2- 172.21.15.16:0/1568932540 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fccb00b5990 0x7fccb00b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.925+0000 7fccae7fc640 1 --2- 172.21.15.16:0/1568932540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccb00d35b0 0x7fccb00b5450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcc980097a0 tx=0x7fcc9802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a446c027b4702a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.925+0000 7fccadffb640 1 -- 172.21.15.16:0/1568932540 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc98037070 con 0x7fccb00d35b0 2024-04-25T10:05:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.925+0000 7fccadffb640 1 -- 172.21.15.16:0/1568932540 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc98032440 con 0x7fccb00d35b0 2024-04-25T10:05:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.925+0000 7fccadffb640 1 -- 172.21.15.16:0/1568932540 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc98032810 con 0x7fccb00d35b0 2024-04-25T10:05:13.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.926+0000 7fccb56cb640 1 -- 172.21.15.16:0/1568932540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccb00d35b0 msgr2=0x7fccb00b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.926+0000 7fccb56cb640 1 --2- 172.21.15.16:0/1568932540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccb00d35b0 0x7fccb00b5450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcc980097a0 tx=0x7fcc9802a180 comp rx=0 tx=0).stop 2024-04-25T10:05:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.926+0000 7fccb56cb640 1 -- 172.21.15.16:0/1568932540 shutdown_connections 2024-04-25T10:05:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.926+0000 7fccb56cb640 1 --2- 172.21.15.16:0/1568932540 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fccb00b5990 0x7fccb00b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.926+0000 7fccb56cb640 1 --2- 172.21.15.16:0/1568932540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccb00d35b0 0x7fccb00b5450 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.926+0000 7fccb56cb640 1 -- 172.21.15.16:0/1568932540 >> 172.21.15.16:0/1568932540 conn(0x7fccb0072900 msgr2=0x7fccb00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.926+0000 7fccb56cb640 1 -- 172.21.15.16:0/1568932540 shutdown_connections 2024-04-25T10:05:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.926+0000 7fccb56cb640 1 -- 172.21.15.16:0/1568932540 wait complete. 2024-04-25T10:05:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.927+0000 7fccb56cb640 1 Processor -- start 2024-04-25T10:05:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.927+0000 7fccb56cb640 1 -- start start 2024-04-25T10:05:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.928+0000 7fccb56cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccb00b5990 0x7fccb0156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.928+0000 7fccb56cb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccb00d35b0 0x7fccb01f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.928+0000 7fccb56cb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fccb015dce0 0x7fccb01600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.928+0000 7fccb56cb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb00d2160 con 0x7fccb00b5990 2024-04-25T10:05:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.928+0000 7fccb56cb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb00d3090 con 0x7fccb00d35b0 2024-04-25T10:05:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.928+0000 7fccb56cb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb015fc60 con 0x7fccb015dce0 2024-04-25T10:05:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.928+0000 7fccaeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccb00b5990 0x7fccb0156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.928+0000 7fccaeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccb00b5990 0x7fccb0156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41746/0 (socket says 172.21.15.16:41746) 2024-04-25T10:05:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.928+0000 7fccaeffd640 1 -- 172.21.15.16:0/2409811437 learned_addr learned my addr 172.21.15.16:0/2409811437 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.928+0000 7fccaf7fe640 1 --2- 172.21.15.16:0/2409811437 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fccb015dce0 0x7fccb01600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.928+0000 7fccae7fc640 1 --2- 172.21.15.16:0/2409811437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccb00d35b0 0x7fccb01f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.928+0000 7fccaeffd640 1 -- 172.21.15.16:0/2409811437 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fccb015dce0 msgr2=0x7fccb01600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.928+0000 7fccaeffd640 1 --2- 172.21.15.16:0/2409811437 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fccb015dce0 0x7fccb01600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.928+0000 7fccaeffd640 1 -- 172.21.15.16:0/2409811437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccb00d35b0 msgr2=0x7fccb01f3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.928+0000 7fccaeffd640 1 --2- 172.21.15.16:0/2409811437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccb00d35b0 0x7fccb01f3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.928+0000 7fccaeffd640 1 -- 172.21.15.16:0/2409811437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb015f050 con 0x7fccb00b5990 2024-04-25T10:05:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.928+0000 7fccae7fc640 1 --2- 172.21.15.16:0/2409811437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccb00d35b0 0x7fccb01f3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.929+0000 7fccaeffd640 1 --2- 172.21.15.16:0/2409811437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccb00b5990 0x7fccb0156d70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcca800c460 tx=0x7fcca800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.929+0000 7fcca77fe640 1 -- 172.21.15.16:0/2409811437 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca8018070 con 0x7fccb00b5990 2024-04-25T10:05:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.929+0000 7fcca77fe640 1 -- 172.21.15.16:0/2409811437 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca8007500 con 0x7fccb00b5990 2024-04-25T10:05:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.929+0000 7fccb56cb640 1 -- 172.21.15.16:0/2409811437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb015f330 con 0x7fccb00b5990 2024-04-25T10:05:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.929+0000 7fcca77fe640 1 -- 172.21.15.16:0/2409811437 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca8004a20 con 0x7fccb00b5990 2024-04-25T10:05:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.929+0000 7fccb56cb640 1 -- 172.21.15.16:0/2409811437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb015f8f0 con 0x7fccb00b5990 2024-04-25T10:05:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.930+0000 7fccb56cb640 1 -- 172.21.15.16:0/2409811437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb00749b0 con 0x7fccb00b5990 2024-04-25T10:05:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.930+0000 7fcca77fe640 1 -- 172.21.15.16:0/2409811437 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcca801b070 con 0x7fccb00b5990 2024-04-25T10:05:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.930+0000 7fcca77fe640 1 --2- 172.21.15.16:0/2409811437 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcc9006b4d0 0x7fcc9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.931+0000 7fcca77fe640 1 -- 172.21.15.16:0/2409811437 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcca80894d0 con 0x7fccb00b5990 2024-04-25T10:05:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.931+0000 7fccae7fc640 1 --2- 172.21.15.16:0/2409811437 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcc9006b4d0 0x7fcc9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.931+0000 7fccae7fc640 1 --2- 172.21.15.16:0/2409811437 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcc9006b4d0 0x7fcc9006d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcc98034000 tx=0x7fcc9802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:13.935+0000 7fcca77fe640 1 -- 172.21.15.16:0/2409811437 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcca8058350 con 0x7fccb00b5990 2024-04-25T10:05:14.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.119+0000 7fccb56cb640 1 -- 172.21.15.16:0/2409811437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccb00d1a50 con 0x7fccb00b5990 2024-04-25T10:05:14.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.120+0000 7fcca77fe640 1 -- 172.21.15.16:0/2409811437 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16609 (secure 0 0 0) 0x7fcca8087e20 con 0x7fccb00b5990 2024-04-25T10:05:14.125 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:14.125 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:12.449183+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:14.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.122+0000 7fccb56cb640 1 -- 172.21.15.16:0/2409811437 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcc9006b4d0 msgr2=0x7fcc9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.122+0000 7fccb56cb640 1 --2- 172.21.15.16:0/2409811437 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcc9006b4d0 0x7fcc9006d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcc98034000 tx=0x7fcc9802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:14.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.122+0000 7fccb56cb640 1 -- 172.21.15.16:0/2409811437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccb00b5990 msgr2=0x7fccb0156d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.122+0000 7fccb56cb640 1 --2- 172.21.15.16:0/2409811437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccb00b5990 0x7fccb0156d70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcca800c460 tx=0x7fcca800c920 comp rx=0 tx=0).stop 2024-04-25T10:05:14.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.122+0000 7fccb56cb640 1 -- 172.21.15.16:0/2409811437 shutdown_connections 2024-04-25T10:05:14.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.122+0000 7fccb56cb640 1 --2- 172.21.15.16:0/2409811437 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcc9006b4d0 0x7fcc9006d990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.122+0000 7fccb56cb640 1 --2- 172.21.15.16:0/2409811437 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fccb015dce0 0x7fccb01600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.122+0000 7fccb56cb640 1 --2- 172.21.15.16:0/2409811437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccb00d35b0 0x7fccb01f3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.122+0000 7fccb56cb640 1 --2- 172.21.15.16:0/2409811437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccb00b5990 0x7fccb0156d70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.122+0000 7fccb56cb640 1 -- 172.21.15.16:0/2409811437 >> 172.21.15.16:0/2409811437 conn(0x7fccb0072900 msgr2=0x7fccb00b7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.122+0000 7fccb56cb640 1 -- 172.21.15.16:0/2409811437 shutdown_connections 2024-04-25T10:05:14.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.122+0000 7fccb56cb640 1 -- 172.21.15.16:0/2409811437 wait complete. 2024-04-25T10:05:14.138 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:05:14.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:14.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.233+0000 7f11305a5640 1 Processor -- start 2024-04-25T10:05:14.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.233+0000 7f11305a5640 1 -- start start 2024-04-25T10:05:14.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.234+0000 7f11305a5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f112815dce0 0x7f11280d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.234+0000 7f11305a5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f11280d35b0 0x7f11280b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.234+0000 7f11305a5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f112815e0c0 con 0x7f11280d1f90 2024-04-25T10:05:14.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.234+0000 7f11305a5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11280b7b10 con 0x7f112815dce0 2024-04-25T10:05:14.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.234+0000 7f11305a5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11280d2510 con 0x7f11280d35b0 2024-04-25T10:05:14.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.234+0000 7f112e341640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f112815dce0 0x7f11280d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.234+0000 7f112eb42640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f11280d1f90 0x7f112806bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34794/0 (socket says 172.21.15.16:34794) 2024-04-25T10:05:14.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.234+0000 7f112eb42640 1 -- 172.21.15.16:0/1875110759 learned_addr learned my addr 172.21.15.16:0/1875110759 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:14.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.234+0000 7f112db40640 1 --2- 172.21.15.16:0/1875110759 >> v2:172.21.15.186:3300/0 conn(0x7f11280d35b0 0x7f11280b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.235+0000 7f112e341640 1 -- 172.21.15.16:0/1875110759 >> v2:172.21.15.186:3300/0 conn(0x7f11280d35b0 msgr2=0x7f11280b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.235+0000 7f112e341640 1 --2- 172.21.15.16:0/1875110759 >> v2:172.21.15.186:3300/0 conn(0x7f11280d35b0 0x7f11280b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.235+0000 7f112e341640 1 -- 172.21.15.16:0/1875110759 >> v1:172.21.15.16:6789/0 conn(0x7f11280d1f90 legacy=0x7f112806bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.235+0000 7f112d33f640 1 -- 172.21.15.16:0/1875110759 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2213994756 0 0) 0x7f112815e0c0 con 0x7f11280d1f90 2024-04-25T10:05:14.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.235+0000 7f112e341640 1 -- 172.21.15.16:0/1875110759 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11280d27f0 con 0x7f112815dce0 2024-04-25T10:05:14.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.235+0000 7f112e341640 1 --2- 172.21.15.16:0/1875110759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f112815dce0 0x7f11280d1a50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f11280cdfa0 tx=0x7f112002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76beea0604f2383 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.235+0000 7f112d33f640 1 -- 172.21.15.16:0/1875110759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1120037070 con 0x7f112815dce0 2024-04-25T10:05:14.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.236+0000 7f112d33f640 1 -- 172.21.15.16:0/1875110759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1120032440 con 0x7f112815dce0 2024-04-25T10:05:14.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.236+0000 7f112d33f640 1 -- 172.21.15.16:0/1875110759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1120032810 con 0x7f112815dce0 2024-04-25T10:05:14.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.236+0000 7f11305a5640 1 -- 172.21.15.16:0/1875110759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f112815dce0 msgr2=0x7f11280d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.236+0000 7f11305a5640 1 --2- 172.21.15.16:0/1875110759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f112815dce0 0x7f11280d1a50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f11280cdfa0 tx=0x7f112002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.236+0000 7f11305a5640 1 -- 172.21.15.16:0/1875110759 shutdown_connections 2024-04-25T10:05:14.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.236+0000 7f11305a5640 1 --2- 172.21.15.16:0/1875110759 >> v2:172.21.15.186:3300/0 conn(0x7f11280d35b0 0x7f11280b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.236+0000 7f11305a5640 1 --2- 172.21.15.16:0/1875110759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f112815dce0 0x7f11280d1a50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.236+0000 7f11305a5640 1 -- 172.21.15.16:0/1875110759 >> 172.21.15.16:0/1875110759 conn(0x7f1128072900 msgr2=0x7f11280d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.236+0000 7f11305a5640 1 -- 172.21.15.16:0/1875110759 shutdown_connections 2024-04-25T10:05:14.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.236+0000 7f11305a5640 1 -- 172.21.15.16:0/1875110759 wait complete. 2024-04-25T10:05:14.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.237+0000 7f11305a5640 1 Processor -- start 2024-04-25T10:05:14.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.237+0000 7f11305a5640 1 -- start start 2024-04-25T10:05:14.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.238+0000 7f11305a5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11280d1f90 0x7f1128159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.238+0000 7f11305a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11280d35b0 0x7f1128152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.238+0000 7f11305a5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f112815dce0 0x7f11281540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.238+0000 7f11305a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11280d3090 con 0x7f11280d35b0 2024-04-25T10:05:14.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.238+0000 7f11305a5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11280d2db0 con 0x7f112815dce0 2024-04-25T10:05:14.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.238+0000 7f11305a5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11280b9820 con 0x7f11280d1f90 2024-04-25T10:05:14.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.238+0000 7f112db40640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11280d35b0 0x7f1128152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.238+0000 7f112db40640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11280d35b0 0x7f1128152ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41760/0 (socket says 172.21.15.16:41760) 2024-04-25T10:05:14.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.238+0000 7f112db40640 1 -- 172.21.15.16:0/4163438949 learned_addr learned my addr 172.21.15.16:0/4163438949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:14.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.238+0000 7f112eb42640 1 --2- 172.21.15.16:0/4163438949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f112815dce0 0x7f11281540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.238+0000 7f112e341640 1 --2- 172.21.15.16:0/4163438949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11280d1f90 0x7f1128159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.238+0000 7f112db40640 1 -- 172.21.15.16:0/4163438949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11280d1f90 msgr2=0x7f1128159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.238+0000 7f112db40640 1 --2- 172.21.15.16:0/4163438949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11280d1f90 0x7f1128159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.238+0000 7f112db40640 1 -- 172.21.15.16:0/4163438949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f112815dce0 msgr2=0x7f11281540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.238+0000 7f112db40640 1 --2- 172.21.15.16:0/4163438949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f112815dce0 0x7f11281540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.238+0000 7f112db40640 1 -- 172.21.15.16:0/4163438949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11281f3e00 con 0x7f11280d35b0 2024-04-25T10:05:14.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.239+0000 7f112db40640 1 --2- 172.21.15.16:0/4163438949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11280d35b0 0x7f1128152ff0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f111c00e980 tx=0x7f111c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.239+0000 7f1126ffd640 1 -- 172.21.15.16:0/4163438949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c017070 con 0x7f11280d35b0 2024-04-25T10:05:14.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.239+0000 7f1126ffd640 1 -- 172.21.15.16:0/4163438949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111c00ca00 con 0x7f11280d35b0 2024-04-25T10:05:14.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.239+0000 7f11305a5640 1 -- 172.21.15.16:0/4163438949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11281f40e0 con 0x7f11280d35b0 2024-04-25T10:05:14.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.239+0000 7f1126ffd640 1 -- 172.21.15.16:0/4163438949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c00cdd0 con 0x7f11280d35b0 2024-04-25T10:05:14.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.239+0000 7f11305a5640 1 -- 172.21.15.16:0/4163438949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11281f46a0 con 0x7f11280d35b0 2024-04-25T10:05:14.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.240+0000 7f11305a5640 1 -- 172.21.15.16:0/4163438949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11280749b0 con 0x7f11280d35b0 2024-04-25T10:05:14.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.240+0000 7f1126ffd640 1 -- 172.21.15.16:0/4163438949 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f111c01a070 con 0x7f11280d35b0 2024-04-25T10:05:14.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.241+0000 7f1126ffd640 1 --2- 172.21.15.16:0/4163438949 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f111406b4d0 0x7f111406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.241+0000 7f1126ffd640 1 -- 172.21.15.16:0/4163438949 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f111c08a230 con 0x7f11280d35b0 2024-04-25T10:05:14.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.241+0000 7f112e341640 1 --2- 172.21.15.16:0/4163438949 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f111406b4d0 0x7f111406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.241+0000 7f112e341640 1 --2- 172.21.15.16:0/4163438949 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f111406b4d0 0x7f111406d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1120034000 tx=0x7f112002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.245+0000 7f1126ffd640 1 -- 172.21.15.16:0/4163438949 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f111c0590b0 con 0x7f11280d35b0 2024-04-25T10:05:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.421+0000 7f11305a5640 1 -- 172.21.15.16:0/4163438949 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f11280d1d10 con 0x7f111406b4d0 2024-04-25T10:05:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.428+0000 7f1126ffd640 1 -- 172.21.15.16:0/4163438949 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249961 (secure 0 0 0) 0x7f11280d1d10 con 0x7f111406b4d0 2024-04-25T10:05:14.436 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:05:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.430+0000 7f11305a5640 1 -- 172.21.15.16:0/4163438949 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f111406b4d0 msgr2=0x7f111406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.430+0000 7f11305a5640 1 --2- 172.21.15.16:0/4163438949 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f111406b4d0 0x7f111406d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1120034000 tx=0x7f112002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.430+0000 7f11305a5640 1 -- 172.21.15.16:0/4163438949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11280d35b0 msgr2=0x7f1128152ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.430+0000 7f11305a5640 1 --2- 172.21.15.16:0/4163438949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11280d35b0 0x7f1128152ff0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f111c00e980 tx=0x7f111c00ee40 comp rx=0 tx=0).stop 2024-04-25T10:05:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.430+0000 7f11305a5640 1 -- 172.21.15.16:0/4163438949 shutdown_connections 2024-04-25T10:05:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.430+0000 7f11305a5640 1 --2- 172.21.15.16:0/4163438949 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f111406b4d0 0x7f111406d990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.430+0000 7f11305a5640 1 --2- 172.21.15.16:0/4163438949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f112815dce0 0x7f11281540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.430+0000 7f11305a5640 1 --2- 172.21.15.16:0/4163438949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11280d35b0 0x7f1128152ff0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.430+0000 7f11305a5640 1 --2- 172.21.15.16:0/4163438949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11280d1f90 0x7f1128159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.430+0000 7f11305a5640 1 -- 172.21.15.16:0/4163438949 >> 172.21.15.16:0/4163438949 conn(0x7f1128072900 msgr2=0x7f11280ced50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.430+0000 7f11305a5640 1 -- 172.21.15.16:0/4163438949 shutdown_connections 2024-04-25T10:05:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.430+0000 7f11305a5640 1 -- 172.21.15.16:0/4163438949 wait complete. 2024-04-25T10:05:14.446 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-25T10:05:13.781767+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235812,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738908,"statfs":{"total":773094113280,"available":772852641792,"internally_reserved":0,"allocated":2891776,"data_stored":1735009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.346535"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.475316+0000","last_change":"2024-04-25T10:05:11.473253+0000","last_active":"2024-04-25T10:05:11.475316+0000","last_peered":"2024-04-25T10:05:11.475316+0000","last_clean":"2024-04-25T10:05:11.475316+0000","last_became_active":"2024-04-25T10:05:11.472940+0000","last_became_peered":"2024-04-25T10:05:11.472940+0000","last_unstale":"2024-04-25T10:05:11.475316+0000","last_undegraded":"2024-04-25T10:05:11.475316+0000","last_fullsized":"2024-04-25T10:05:11.475316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:52.014434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455508+0000","last_change":"2024-04-25T10:05:11.461498+0000","last_active":"2024-04-25T10:05:12.455508+0000","last_peered":"2024-04-25T10:05:12.455508+0000","last_clean":"2024-04-25T10:05:12.455508+0000","last_became_active":"2024-04-25T10:05:11.461241+0000","last_became_peered":"2024-04-25T10:05:11.461241+0000","last_unstale":"2024-04-25T10:05:12.455508+0000","last_undegraded":"2024-04-25T10:05:12.455508+0000","last_fullsized":"2024-04-25T10:05:12.455508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472722+0000","last_change":"2024-04-25T10:05:11.469281+0000","last_active":"2024-04-25T10:05:11.472722+0000","last_peered":"2024-04-25T10:05:11.472722+0000","last_clean":"2024-04-25T10:05:11.472722+0000","last_became_active":"2024-04-25T10:05:11.468809+0000","last_became_peered":"2024-04-25T10:05:11.468809+0000","last_unstale":"2024-04-25T10:05:11.472722+0000","last_undegraded":"2024-04-25T10:05:11.472722+0000","last_fullsized":"2024-04-25T10:05:11.472722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:30.789969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455529+0000","last_change":"2024-04-25T10:05:11.466531+0000","last_active":"2024-04-25T10:05:12.455529+0000","last_peered":"2024-04-25T10:05:12.455529+0000","last_clean":"2024-04-25T10:05:12.455529+0000","last_became_active":"2024-04-25T10:05:11.466126+0000","last_became_peered":"2024-04-25T10:05:11.466126+0000","last_unstale":"2024-04-25T10:05:12.455529+0000","last_undegraded":"2024-04-25T10:05:12.455529+0000","last_fullsized":"2024-04-25T10:05:12.455529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469389+0000","last_change":"2024-04-25T10:05:11.466336+0000","last_active":"2024-04-25T10:05:11.469389+0000","last_peered":"2024-04-25T10:05:11.469389+0000","last_clean":"2024-04-25T10:05:11.469389+0000","last_became_active":"2024-04-25T10:05:11.466032+0000","last_became_peered":"2024-04-25T10:05:11.466032+0000","last_unstale":"2024-04-25T10:05:11.469389+0000","last_undegraded":"2024-04-25T10:05:11.469389+0000","last_fullsized":"2024-04-25T10:05:11.469389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:12.926321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456654+0000","last_change":"2024-04-25T10:05:11.461997+0000","last_active":"2024-04-25T10:05:12.456654+0000","last_peered":"2024-04-25T10:05:12.456654+0000","last_clean":"2024-04-25T10:05:12.456654+0000","last_became_active":"2024-04-25T10:05:11.461658+0000","last_became_peered":"2024-04-25T10:05:11.461658+0000","last_unstale":"2024-04-25T10:05:12.456654+0000","last_undegraded":"2024-04-25T10:05:12.456654+0000","last_fullsized":"2024-04-25T10:05:12.456654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.463064+0000","last_change":"2024-04-25T10:05:11.461631+0000","last_active":"2024-04-25T10:05:11.463064+0000","last_peered":"2024-04-25T10:05:11.463064+0000","last_clean":"2024-04-25T10:05:11.463064+0000","last_became_active":"2024-04-25T10:05:11.461327+0000","last_became_peered":"2024-04-25T10:05:11.461327+0000","last_unstale":"2024-04-25T10:05:11.463064+0000","last_undegraded":"2024-04-25T10:05:11.463064+0000","last_fullsized":"2024-04-25T10:05:11.463064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.473963+0000","last_change":"2024-04-25T10:05:11.473044+0000","last_active":"2024-04-25T10:05:11.473963+0000","last_peered":"2024-04-25T10:05:11.473963+0000","last_clean":"2024-04-25T10:05:11.473963+0000","last_became_active":"2024-04-25T10:05:11.472747+0000","last_became_peered":"2024-04-25T10:05:11.472747+0000","last_unstale":"2024-04-25T10:05:11.473963+0000","last_undegraded":"2024-04-25T10:05:11.473963+0000","last_fullsized":"2024-04-25T10:05:11.473963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474718+0000","last_change":"2024-04-25T10:05:11.470219+0000","last_active":"2024-04-25T10:05:11.474718+0000","last_peered":"2024-04-25T10:05:11.474718+0000","last_clean":"2024-04-25T10:05:11.474718+0000","last_became_active":"2024-04-25T10:05:11.469066+0000","last_became_peered":"2024-04-25T10:05:11.469066+0000","last_unstale":"2024-04-25T10:05:11.474718+0000","last_undegraded":"2024-04-25T10:05:11.474718+0000","last_fullsized":"2024-04-25T10:05:11.474718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:54.934726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.466367+0000","last_change":"2024-04-25T10:05:11.463262+0000","last_active":"2024-04-25T10:05:11.466367+0000","last_peered":"2024-04-25T10:05:11.466367+0000","last_clean":"2024-04-25T10:05:11.466367+0000","last_became_active":"2024-04-25T10:05:11.462970+0000","last_became_peered":"2024-04-25T10:05:11.462970+0000","last_unstale":"2024-04-25T10:05:11.466367+0000","last_undegraded":"2024-04-25T10:05:11.466367+0000","last_fullsized":"2024-04-25T10:05:11.466367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.464756+0000","last_change":"2024-04-25T10:05:11.458310+0000","last_active":"2024-04-25T10:05:11.464756+0000","last_peered":"2024-04-25T10:05:11.464756+0000","last_clean":"2024-04-25T10:05:11.464756+0000","last_became_active":"2024-04-25T10:05:11.457672+0000","last_became_peered":"2024-04-25T10:05:11.457672+0000","last_unstale":"2024-04-25T10:05:11.464756+0000","last_undegraded":"2024-04-25T10:05:11.464756+0000","last_fullsized":"2024-04-25T10:05:11.464756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:02.561557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471639+0000","last_change":"2024-04-25T10:05:11.470743+0000","last_active":"2024-04-25T10:05:11.471639+0000","last_peered":"2024-04-25T10:05:11.471639+0000","last_clean":"2024-04-25T10:05:11.471639+0000","last_became_active":"2024-04-25T10:05:11.470447+0000","last_became_peered":"2024-04-25T10:05:11.470447+0000","last_unstale":"2024-04-25T10:05:11.471639+0000","last_undegraded":"2024-04-25T10:05:11.471639+0000","last_fullsized":"2024-04-25T10:05:11.471639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:41.766168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.473455+0000","last_change":"2024-04-25T10:05:11.472530+0000","last_active":"2024-04-25T10:05:11.473455+0000","last_peered":"2024-04-25T10:05:11.473455+0000","last_clean":"2024-04-25T10:05:11.473455+0000","last_became_active":"2024-04-25T10:05:11.472164+0000","last_became_peered":"2024-04-25T10:05:11.472164+0000","last_unstale":"2024-04-25T10:05:11.473455+0000","last_undegraded":"2024-04-25T10:05:11.473455+0000","last_fullsized":"2024-04-25T10:05:11.473455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:08.525515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471968+0000","last_change":"2024-04-25T10:05:11.464186+0000","last_active":"2024-04-25T10:05:11.471968+0000","last_peered":"2024-04-25T10:05:11.471968+0000","last_clean":"2024-04-25T10:05:11.471968+0000","last_became_active":"2024-04-25T10:05:11.463925+0000","last_became_peered":"2024-04-25T10:05:11.463925+0000","last_unstale":"2024-04-25T10:05:11.471968+0000","last_undegraded":"2024-04-25T10:05:11.471968+0000","last_fullsized":"2024-04-25T10:05:11.471968+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:55.300183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474686+0000","last_change":"2024-04-25T10:05:11.473554+0000","last_active":"2024-04-25T10:05:11.474686+0000","last_peered":"2024-04-25T10:05:11.474686+0000","last_clean":"2024-04-25T10:05:11.474686+0000","last_became_active":"2024-04-25T10:05:11.473271+0000","last_became_peered":"2024-04-25T10:05:11.473271+0000","last_unstale":"2024-04-25T10:05:11.474686+0000","last_undegraded":"2024-04-25T10:05:11.474686+0000","last_fullsized":"2024-04-25T10:05:11.474686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:35.076762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474740+0000","last_change":"2024-04-25T10:05:11.471477+0000","last_active":"2024-04-25T10:05:11.474740+0000","last_peered":"2024-04-25T10:05:11.474740+0000","last_clean":"2024-04-25T10:05:11.474740+0000","last_became_active":"2024-04-25T10:05:11.471219+0000","last_became_peered":"2024-04-25T10:05:11.471219+0000","last_unstale":"2024-04-25T10:05:11.474740+0000","last_undegraded":"2024-04-25T10:05:11.474740+0000","last_fullsized":"2024-04-25T10:05:11.474740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456655+0000","last_change":"2024-04-25T10:05:11.464410+0000","last_active":"2024-04-25T10:05:12.456655+0000","last_peered":"2024-04-25T10:05:12.456655+0000","last_clean":"2024-04-25T10:05:12.456655+0000","last_became_active":"2024-04-25T10:05:11.464003+0000","last_became_peered":"2024-04-25T10:05:11.464003+0000","last_unstale":"2024-04-25T10:05:12.456655+0000","last_undegraded":"2024-04-25T10:05:12.456655+0000","last_fullsized":"2024-04-25T10:05:12.456655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:52.014434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456342+0000","last_change":"2024-04-25T10:05:11.462487+0000","last_active":"2024-04-25T10:05:12.456342+0000","last_peered":"2024-04-25T10:05:12.456342+0000","last_clean":"2024-04-25T10:05:12.456342+0000","last_became_active":"2024-04-25T10:05:11.462191+0000","last_became_peered":"2024-04-25T10:05:11.462191+0000","last_unstale":"2024-04-25T10:05:12.456342+0000","last_undegraded":"2024-04-25T10:05:12.456342+0000","last_fullsized":"2024-04-25T10:05:12.456342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469823+0000","last_change":"2024-04-25T10:05:11.467406+0000","last_active":"2024-04-25T10:05:11.469823+0000","last_peered":"2024-04-25T10:05:11.469823+0000","last_clean":"2024-04-25T10:05:11.469823+0000","last_became_active":"2024-04-25T10:05:11.467126+0000","last_became_peered":"2024-04-25T10:05:11.467126+0000","last_unstale":"2024-04-25T10:05:11.469823+0000","last_undegraded":"2024-04-25T10:05:11.469823+0000","last_fullsized":"2024-04-25T10:05:11.469823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:52.014434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455642+0000","last_change":"2024-04-25T10:05:11.469452+0000","last_active":"2024-04-25T10:05:12.455642+0000","last_peered":"2024-04-25T10:05:12.455642+0000","last_clean":"2024-04-25T10:05:12.455642+0000","last_became_active":"2024-04-25T10:05:11.465896+0000","last_became_peered":"2024-04-25T10:05:11.465896+0000","last_unstale":"2024-04-25T10:05:12.455642+0000","last_undegraded":"2024-04-25T10:05:12.455642+0000","last_fullsized":"2024-04-25T10:05:12.455642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:14.506306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.460957+0000","last_change":"2024-04-25T10:05:11.457595+0000","last_active":"2024-04-25T10:05:11.460957+0000","last_peered":"2024-04-25T10:05:11.460957+0000","last_clean":"2024-04-25T10:05:11.460957+0000","last_became_active":"2024-04-25T10:05:11.457320+0000","last_became_peered":"2024-04-25T10:05:11.457320+0000","last_unstale":"2024-04-25T10:05:11.460957+0000","last_undegraded":"2024-04-25T10:05:11.460957+0000","last_fullsized":"2024-04-25T10:05:11.460957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:40.831935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472125+0000","last_change":"2024-04-25T10:05:11.469113+0000","last_active":"2024-04-25T10:05:11.472125+0000","last_peered":"2024-04-25T10:05:11.472125+0000","last_clean":"2024-04-25T10:05:11.472125+0000","last_became_active":"2024-04-25T10:05:11.468759+0000","last_became_peered":"2024-04-25T10:05:11.468759+0000","last_unstale":"2024-04-25T10:05:11.472125+0000","last_undegraded":"2024-04-25T10:05:11.472125+0000","last_fullsized":"2024-04-25T10:05:11.472125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471666+0000","last_change":"2024-04-25T10:05:11.470055+0000","last_active":"2024-04-25T10:05:11.471666+0000","last_peered":"2024-04-25T10:05:11.471666+0000","last_clean":"2024-04-25T10:05:11.471666+0000","last_became_active":"2024-04-25T10:05:11.468974+0000","last_became_peered":"2024-04-25T10:05:11.468974+0000","last_unstale":"2024-04-25T10:05:11.471666+0000","last_undegraded":"2024-04-25T10:05:11.471666+0000","last_fullsized":"2024-04-25T10:05:11.471666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.228600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469311+0000","last_change":"2024-04-25T10:05:11.462234+0000","last_active":"2024-04-25T10:05:11.469311+0000","last_peered":"2024-04-25T10:05:11.469311+0000","last_clean":"2024-04-25T10:05:11.469311+0000","last_became_active":"2024-04-25T10:05:11.461975+0000","last_became_peered":"2024-04-25T10:05:11.461975+0000","last_unstale":"2024-04-25T10:05:11.469311+0000","last_undegraded":"2024-04-25T10:05:11.469311+0000","last_fullsized":"2024-04-25T10:05:11.469311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:02.561557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455396+0000","last_change":"2024-04-25T10:05:11.463406+0000","last_active":"2024-04-25T10:05:12.455396+0000","last_peered":"2024-04-25T10:05:12.455396+0000","last_clean":"2024-04-25T10:05:12.455396+0000","last_became_active":"2024-04-25T10:05:11.463139+0000","last_became_peered":"2024-04-25T10:05:11.463139+0000","last_unstale":"2024-04-25T10:05:12.455396+0000","last_undegraded":"2024-04-25T10:05:12.455396+0000","last_fullsized":"2024-04-25T10:05:12.455396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456384+0000","last_change":"2024-04-25T10:05:11.453168+0000","last_active":"2024-04-25T10:05:12.456384+0000","last_peered":"2024-04-25T10:05:12.456384+0000","last_clean":"2024-04-25T10:05:12.456384+0000","last_became_active":"2024-04-25T10:05:11.452238+0000","last_became_peered":"2024-04-25T10:05:11.452238+0000","last_unstale":"2024-04-25T10:05:12.456384+0000","last_undegraded":"2024-04-25T10:05:12.456384+0000","last_fullsized":"2024-04-25T10:05:12.456384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-04-25T10:05:09.441702+0000","last_change":"2024-04-25T10:05:09.441702+0000","last_active":"2024-04-25T10:05:09.429642+0000","last_peered":"2024-04-25T10:05:09.429642+0000","last_clean":"2024-04-25T10:05:09.429642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:09.441702+0000","last_undegraded":"2024-04-25T10:05:09.441702+0000","last_fullsized":"2024-04-25T10:05:09.441702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456962+0000","last_change":"2024-04-25T10:05:11.469760+0000","last_active":"2024-04-25T10:05:12.456962+0000","last_peered":"2024-04-25T10:05:12.456962+0000","last_clean":"2024-04-25T10:05:12.456962+0000","last_became_active":"2024-04-25T10:05:11.469490+0000","last_became_peered":"2024-04-25T10:05:11.469490+0000","last_unstale":"2024-04-25T10:05:12.456962+0000","last_undegraded":"2024-04-25T10:05:12.456962+0000","last_fullsized":"2024-04-25T10:05:12.456962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.228600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.475721+0000","last_change":"2024-04-25T10:05:10.461940+0000","last_active":"2024-04-25T10:05:11.475721+0000","last_peered":"2024-04-25T10:05:11.475721+0000","last_clean":"2024-04-25T10:05:11.475721+0000","last_became_active":"2024-04-25T10:05:10.461633+0000","last_became_peered":"2024-04-25T10:05:10.461633+0000","last_unstale":"2024-04-25T10:05:11.475721+0000","last_undegraded":"2024-04-25T10:05:11.475721+0000","last_fullsized":"2024-04-25T10:05:11.475721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.456539+0000","last_change":"2024-04-25T10:05:10.460823+0000","last_active":"2024-04-25T10:05:11.456539+0000","last_peered":"2024-04-25T10:05:11.456539+0000","last_clean":"2024-04-25T10:05:11.456539+0000","last_became_active":"2024-04-25T10:05:10.460480+0000","last_became_peered":"2024-04-25T10:05:10.460480+0000","last_unstale":"2024-04-25T10:05:11.456539+0000","last_undegraded":"2024-04-25T10:05:11.456539+0000","last_fullsized":"2024-04-25T10:05:11.456539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:31.375084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.450458+0000","last_change":"2024-04-25T10:05:10.461713+0000","last_active":"2024-04-25T10:05:11.450458+0000","last_peered":"2024-04-25T10:05:11.450458+0000","last_clean":"2024-04-25T10:05:11.450458+0000","last_became_active":"2024-04-25T10:05:10.461379+0000","last_became_peered":"2024-04-25T10:05:10.461379+0000","last_unstale":"2024-04-25T10:05:11.450458+0000","last_undegraded":"2024-04-25T10:05:11.450458+0000","last_fullsized":"2024-04-25T10:05:11.450458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456510+0000","last_change":"2024-04-25T10:05:11.466408+0000","last_active":"2024-04-25T10:05:12.456510+0000","last_peered":"2024-04-25T10:05:12.456510+0000","last_clean":"2024-04-25T10:05:12.456510+0000","last_became_active":"2024-04-25T10:05:11.465611+0000","last_became_peered":"2024-04-25T10:05:11.465611+0000","last_unstale":"2024-04-25T10:05:12.456510+0000","last_undegraded":"2024-04-25T10:05:12.456510+0000","last_fullsized":"2024-04-25T10:05:12.456510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456657+0000","last_change":"2024-04-25T10:05:10.449054+0000","last_active":"2024-04-25T10:05:12.456657+0000","last_peered":"2024-04-25T10:05:12.456657+0000","last_clean":"2024-04-25T10:05:12.456657+0000","last_became_active":"2024-04-25T10:05:10.448743+0000","last_became_peered":"2024-04-25T10:05:10.448743+0000","last_unstale":"2024-04-25T10:05:12.456657+0000","last_undegraded":"2024-04-25T10:05:12.456657+0000","last_fullsized":"2024-04-25T10:05:12.456657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:22.897294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455686+0000","last_change":"2024-04-25T10:05:11.453268+0000","last_active":"2024-04-25T10:05:12.455686+0000","last_peered":"2024-04-25T10:05:12.455686+0000","last_clean":"2024-04-25T10:05:12.455686+0000","last_became_active":"2024-04-25T10:05:11.452971+0000","last_became_peered":"2024-04-25T10:05:11.452971+0000","last_unstale":"2024-04-25T10:05:12.455686+0000","last_undegraded":"2024-04-25T10:05:12.455686+0000","last_fullsized":"2024-04-25T10:05:12.455686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:56.142030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.452585+0000","last_change":"2024-04-25T10:05:10.452247+0000","last_active":"2024-04-25T10:05:11.452585+0000","last_peered":"2024-04-25T10:05:11.452585+0000","last_clean":"2024-04-25T10:05:11.452585+0000","last_became_active":"2024-04-25T10:05:10.451925+0000","last_became_peered":"2024-04-25T10:05:10.451925+0000","last_unstale":"2024-04-25T10:05:11.452585+0000","last_undegraded":"2024-04-25T10:05:11.452585+0000","last_fullsized":"2024-04-25T10:05:11.452585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:53.640804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472671+0000","last_change":"2024-04-25T10:05:10.459898+0000","last_active":"2024-04-25T10:05:11.472671+0000","last_peered":"2024-04-25T10:05:11.472671+0000","last_clean":"2024-04-25T10:05:11.472671+0000","last_became_active":"2024-04-25T10:05:10.459250+0000","last_became_peered":"2024-04-25T10:05:10.459250+0000","last_unstale":"2024-04-25T10:05:11.472671+0000","last_undegraded":"2024-04-25T10:05:11.472671+0000","last_fullsized":"2024-04-25T10:05:11.472671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:41.255007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.473052+0000","last_change":"2024-04-25T10:05:11.471498+0000","last_active":"2024-04-25T10:05:11.473052+0000","last_peered":"2024-04-25T10:05:11.473052+0000","last_clean":"2024-04-25T10:05:11.473052+0000","last_became_active":"2024-04-25T10:05:11.471193+0000","last_became_peered":"2024-04-25T10:05:11.471193+0000","last_unstale":"2024-04-25T10:05:11.473052+0000","last_undegraded":"2024-04-25T10:05:11.473052+0000","last_fullsized":"2024-04-25T10:05:11.473052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:45.980553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455560+0000","last_change":"2024-04-25T10:05:10.450534+0000","last_active":"2024-04-25T10:05:12.455560+0000","last_peered":"2024-04-25T10:05:12.455560+0000","last_clean":"2024-04-25T10:05:12.455560+0000","last_became_active":"2024-04-25T10:05:10.450251+0000","last_became_peered":"2024-04-25T10:05:10.450251+0000","last_unstale":"2024-04-25T10:05:12.455560+0000","last_undegraded":"2024-04-25T10:05:12.455560+0000","last_fullsized":"2024-04-25T10:05:12.455560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:22.897294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.468770+0000","last_change":"2024-04-25T10:05:11.463480+0000","last_active":"2024-04-25T10:05:11.468770+0000","last_peered":"2024-04-25T10:05:11.468770+0000","last_clean":"2024-04-25T10:05:11.468770+0000","last_became_active":"2024-04-25T10:05:11.460995+0000","last_became_peered":"2024-04-25T10:05:11.460995+0000","last_unstale":"2024-04-25T10:05:11.468770+0000","last_undegraded":"2024-04-25T10:05:11.468770+0000","last_fullsized":"2024-04-25T10:05:11.468770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:56.142030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.464540+0000","last_change":"2024-04-25T10:05:10.462323+0000","last_active":"2024-04-25T10:05:11.464540+0000","last_peered":"2024-04-25T10:05:11.464540+0000","last_clean":"2024-04-25T10:05:11.464540+0000","last_became_active":"2024-04-25T10:05:10.462118+0000","last_became_peered":"2024-04-25T10:05:10.462118+0000","last_unstale":"2024-04-25T10:05:11.464540+0000","last_undegraded":"2024-04-25T10:05:11.464540+0000","last_fullsized":"2024-04-25T10:05:11.464540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:39.826605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471805+0000","last_change":"2024-04-25T10:05:11.467396+0000","last_active":"2024-04-25T10:05:11.471805+0000","last_peered":"2024-04-25T10:05:11.471805+0000","last_clean":"2024-04-25T10:05:11.471805+0000","last_became_active":"2024-04-25T10:05:11.467112+0000","last_became_peered":"2024-04-25T10:05:11.467112+0000","last_unstale":"2024-04-25T10:05:11.471805+0000","last_undegraded":"2024-04-25T10:05:11.471805+0000","last_fullsized":"2024-04-25T10:05:11.471805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.451714+0000","last_change":"2024-04-25T10:05:10.458429+0000","last_active":"2024-04-25T10:05:11.451714+0000","last_peered":"2024-04-25T10:05:11.451714+0000","last_clean":"2024-04-25T10:05:11.451714+0000","last_became_active":"2024-04-25T10:05:10.457958+0000","last_became_peered":"2024-04-25T10:05:10.457958+0000","last_unstale":"2024-04-25T10:05:11.451714+0000","last_undegraded":"2024-04-25T10:05:11.451714+0000","last_fullsized":"2024-04-25T10:05:11.451714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:23.585636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469949+0000","last_change":"2024-04-25T10:05:11.465006+0000","last_active":"2024-04-25T10:05:11.469949+0000","last_peered":"2024-04-25T10:05:11.469949+0000","last_clean":"2024-04-25T10:05:11.469949+0000","last_became_active":"2024-04-25T10:05:11.464719+0000","last_became_peered":"2024-04-25T10:05:11.464719+0000","last_unstale":"2024-04-25T10:05:11.469949+0000","last_undegraded":"2024-04-25T10:05:11.469949+0000","last_fullsized":"2024-04-25T10:05:11.469949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:54.934726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.459177+0000","last_change":"2024-04-25T10:05:10.456938+0000","last_active":"2024-04-25T10:05:11.459177+0000","last_peered":"2024-04-25T10:05:11.459177+0000","last_clean":"2024-04-25T10:05:11.459177+0000","last_became_active":"2024-04-25T10:05:10.456644+0000","last_became_peered":"2024-04-25T10:05:10.456644+0000","last_unstale":"2024-04-25T10:05:11.459177+0000","last_undegraded":"2024-04-25T10:05:11.459177+0000","last_fullsized":"2024-04-25T10:05:11.459177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456188+0000","last_change":"2024-04-25T10:05:11.456178+0000","last_active":"2024-04-25T10:05:12.456188+0000","last_peered":"2024-04-25T10:05:12.456188+0000","last_clean":"2024-04-25T10:05:12.456188+0000","last_became_active":"2024-04-25T10:05:11.455879+0000","last_became_peered":"2024-04-25T10:05:11.455879+0000","last_unstale":"2024-04-25T10:05:12.456188+0000","last_undegraded":"2024-04-25T10:05:12.456188+0000","last_fullsized":"2024-04-25T10:05:12.456188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:55.300183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456275+0000","last_change":"2024-04-25T10:05:10.455684+0000","last_active":"2024-04-25T10:05:12.456275+0000","last_peered":"2024-04-25T10:05:12.456275+0000","last_clean":"2024-04-25T10:05:12.456275+0000","last_became_active":"2024-04-25T10:05:10.455342+0000","last_became_peered":"2024-04-25T10:05:10.455342+0000","last_unstale":"2024-04-25T10:05:12.456275+0000","last_undegraded":"2024-04-25T10:05:12.456275+0000","last_fullsized":"2024-04-25T10:05:12.456275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:01.556228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469903+0000","last_change":"2024-04-25T10:05:11.466622+0000","last_active":"2024-04-25T10:05:11.469903+0000","last_peered":"2024-04-25T10:05:11.469903+0000","last_clean":"2024-04-25T10:05:11.469903+0000","last_became_active":"2024-04-25T10:05:11.466346+0000","last_became_peered":"2024-04-25T10:05:11.466346+0000","last_unstale":"2024-04-25T10:05:11.469903+0000","last_undegraded":"2024-04-25T10:05:11.469903+0000","last_fullsized":"2024-04-25T10:05:11.469903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.468144+0000","last_change":"2024-04-25T10:05:10.457614+0000","last_active":"2024-04-25T10:05:11.468144+0000","last_peered":"2024-04-25T10:05:11.468144+0000","last_clean":"2024-04-25T10:05:11.468144+0000","last_became_active":"2024-04-25T10:05:10.457332+0000","last_became_peered":"2024-04-25T10:05:10.457332+0000","last_unstale":"2024-04-25T10:05:11.468144+0000","last_undegraded":"2024-04-25T10:05:11.468144+0000","last_fullsized":"2024-04-25T10:05:11.468144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:01.556228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455780+0000","last_change":"2024-04-25T10:05:11.460589+0000","last_active":"2024-04-25T10:05:12.455780+0000","last_peered":"2024-04-25T10:05:12.455780+0000","last_clean":"2024-04-25T10:05:12.455780+0000","last_became_active":"2024-04-25T10:05:11.459433+0000","last_became_peered":"2024-04-25T10:05:11.459433+0000","last_unstale":"2024-04-25T10:05:12.455780+0000","last_undegraded":"2024-04-25T10:05:12.455780+0000","last_fullsized":"2024-04-25T10:05:12.455780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:55.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456910+0000","last_change":"2024-04-25T10:05:10.458692+0000","last_active":"2024-04-25T10:05:12.456910+0000","last_peered":"2024-04-25T10:05:12.456910+0000","last_clean":"2024-04-25T10:05:12.456910+0000","last_became_active":"2024-04-25T10:05:10.458090+0000","last_became_peered":"2024-04-25T10:05:10.458090+0000","last_unstale":"2024-04-25T10:05:12.456910+0000","last_undegraded":"2024-04-25T10:05:12.456910+0000","last_fullsized":"2024-04-25T10:05:12.456910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:23.585636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.468810+0000","last_change":"2024-04-25T10:05:11.464532+0000","last_active":"2024-04-25T10:05:11.468810+0000","last_peered":"2024-04-25T10:05:11.468810+0000","last_clean":"2024-04-25T10:05:11.468810+0000","last_became_active":"2024-04-25T10:05:11.464186+0000","last_became_peered":"2024-04-25T10:05:11.464186+0000","last_unstale":"2024-04-25T10:05:11.468810+0000","last_undegraded":"2024-04-25T10:05:11.468810+0000","last_fullsized":"2024-04-25T10:05:11.468810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:55.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-04-25T10:05:09.441695+0000","last_change":"2024-04-25T10:05:09.441695+0000","last_active":"2024-04-25T10:05:09.429642+0000","last_peered":"2024-04-25T10:05:09.429642+0000","last_clean":"2024-04-25T10:05:09.429642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:09.441695+0000","last_undegraded":"2024-04-25T10:05:09.441695+0000","last_fullsized":"2024-04-25T10:05:09.441695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.475897+0000","last_change":"2024-04-25T10:05:11.474519+0000","last_active":"2024-04-25T10:05:11.475897+0000","last_peered":"2024-04-25T10:05:11.475897+0000","last_clean":"2024-04-25T10:05:11.475897+0000","last_became_active":"2024-04-25T10:05:11.472675+0000","last_became_peered":"2024-04-25T10:05:11.472675+0000","last_unstale":"2024-04-25T10:05:11.475897+0000","last_undegraded":"2024-04-25T10:05:11.475897+0000","last_fullsized":"2024-04-25T10:05:11.475897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.799101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.458209+0000","last_change":"2024-04-25T10:05:10.453364+0000","last_active":"2024-04-25T10:05:11.458209+0000","last_peered":"2024-04-25T10:05:11.458209+0000","last_clean":"2024-04-25T10:05:11.458209+0000","last_became_active":"2024-04-25T10:05:10.453072+0000","last_became_peered":"2024-04-25T10:05:10.453072+0000","last_unstale":"2024-04-25T10:05:11.458209+0000","last_undegraded":"2024-04-25T10:05:11.458209+0000","last_fullsized":"2024-04-25T10:05:11.458209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:53.640804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469818+0000","last_change":"2024-04-25T10:05:11.467814+0000","last_active":"2024-04-25T10:05:11.469818+0000","last_peered":"2024-04-25T10:05:11.469818+0000","last_clean":"2024-04-25T10:05:11.469818+0000","last_became_active":"2024-04-25T10:05:11.467496+0000","last_became_peered":"2024-04-25T10:05:11.467496+0000","last_unstale":"2024-04-25T10:05:11.469818+0000","last_undegraded":"2024-04-25T10:05:11.469818+0000","last_fullsized":"2024-04-25T10:05:11.469818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.799101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456108+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:05:12.456108+0000","last_peered":"2024-04-25T10:05:12.456108+0000","last_clean":"2024-04-25T10:05:12.456108+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:05:12.456108+0000","last_undegraded":"2024-04-25T10:05:12.456108+0000","last_fullsized":"2024-04-25T10:05:12.456108+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.452172+0000","last_change":"2024-04-25T10:05:10.458170+0000","last_active":"2024-04-25T10:05:11.452172+0000","last_peered":"2024-04-25T10:05:11.452172+0000","last_clean":"2024-04-25T10:05:11.452172+0000","last_became_active":"2024-04-25T10:05:10.457794+0000","last_became_peered":"2024-04-25T10:05:10.457794+0000","last_unstale":"2024-04-25T10:05:11.452172+0000","last_undegraded":"2024-04-25T10:05:11.452172+0000","last_fullsized":"2024-04-25T10:05:11.452172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:11.920992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456600+0000","last_change":"2024-04-25T10:05:11.462574+0000","last_active":"2024-04-25T10:05:12.456600+0000","last_peered":"2024-04-25T10:05:12.456600+0000","last_clean":"2024-04-25T10:05:12.456600+0000","last_became_active":"2024-04-25T10:05:11.461617+0000","last_became_peered":"2024-04-25T10:05:11.461617+0000","last_unstale":"2024-04-25T10:05:12.456600+0000","last_undegraded":"2024-04-25T10:05:12.456600+0000","last_fullsized":"2024-04-25T10:05:12.456600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.451455+0000","last_change":"2024-04-25T10:05:10.464334+0000","last_active":"2024-04-25T10:05:11.451455+0000","last_peered":"2024-04-25T10:05:11.451455+0000","last_clean":"2024-04-25T10:05:11.451455+0000","last_became_active":"2024-04-25T10:05:10.463991+0000","last_became_peered":"2024-04-25T10:05:10.463991+0000","last_unstale":"2024-04-25T10:05:11.451455+0000","last_undegraded":"2024-04-25T10:05:11.451455+0000","last_fullsized":"2024-04-25T10:05:11.451455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469159+0000","last_change":"2024-04-25T10:05:11.464866+0000","last_active":"2024-04-25T10:05:11.469159+0000","last_peered":"2024-04-25T10:05:11.469159+0000","last_clean":"2024-04-25T10:05:11.469159+0000","last_became_active":"2024-04-25T10:05:11.464504+0000","last_became_peered":"2024-04-25T10:05:11.464504+0000","last_unstale":"2024-04-25T10:05:11.469159+0000","last_undegraded":"2024-04-25T10:05:11.469159+0000","last_fullsized":"2024-04-25T10:05:11.469159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.449849+0000","last_change":"2024-04-25T10:05:10.458220+0000","last_active":"2024-04-25T10:05:11.449849+0000","last_peered":"2024-04-25T10:05:11.449849+0000","last_clean":"2024-04-25T10:05:11.449849+0000","last_became_active":"2024-04-25T10:05:10.457936+0000","last_became_peered":"2024-04-25T10:05:10.457936+0000","last_unstale":"2024-04-25T10:05:11.449849+0000","last_undegraded":"2024-04-25T10:05:11.449849+0000","last_fullsized":"2024-04-25T10:05:11.449849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:31.375084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.451883+0000","last_change":"2024-04-25T10:05:10.463388+0000","last_active":"2024-04-25T10:05:11.451883+0000","last_peered":"2024-04-25T10:05:11.451883+0000","last_clean":"2024-04-25T10:05:11.451883+0000","last_became_active":"2024-04-25T10:05:10.463041+0000","last_became_peered":"2024-04-25T10:05:10.463041+0000","last_unstale":"2024-04-25T10:05:11.451883+0000","last_undegraded":"2024-04-25T10:05:11.451883+0000","last_fullsized":"2024-04-25T10:05:11.451883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:13.500977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.463844+0000","last_change":"2024-04-25T10:05:11.462360+0000","last_active":"2024-04-25T10:05:11.463844+0000","last_peered":"2024-04-25T10:05:11.463844+0000","last_clean":"2024-04-25T10:05:11.463844+0000","last_became_active":"2024-04-25T10:05:11.462050+0000","last_became_peered":"2024-04-25T10:05:11.462050+0000","last_unstale":"2024-04-25T10:05:11.463844+0000","last_undegraded":"2024-04-25T10:05:11.463844+0000","last_fullsized":"2024-04-25T10:05:11.463844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:14.506306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-04-25T10:05:09.441740+0000","last_change":"2024-04-25T10:05:09.441740+0000","last_active":"2024-04-25T10:05:09.429642+0000","last_peered":"2024-04-25T10:05:09.429642+0000","last_clean":"2024-04-25T10:05:09.429642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:09.441740+0000","last_undegraded":"2024-04-25T10:05:09.441740+0000","last_fullsized":"2024-04-25T10:05:09.441740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455433+0000","last_change":"2024-04-25T10:05:11.464184+0000","last_active":"2024-04-25T10:05:12.455433+0000","last_peered":"2024-04-25T10:05:12.455433+0000","last_clean":"2024-04-25T10:05:12.455433+0000","last_became_active":"2024-04-25T10:05:11.463894+0000","last_became_peered":"2024-04-25T10:05:11.463894+0000","last_unstale":"2024-04-25T10:05:12.455433+0000","last_undegraded":"2024-04-25T10:05:12.455433+0000","last_fullsized":"2024-04-25T10:05:12.455433+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.452412+0000","last_change":"2024-04-25T10:05:10.460261+0000","last_active":"2024-04-25T10:05:11.452412+0000","last_peered":"2024-04-25T10:05:11.452412+0000","last_clean":"2024-04-25T10:05:11.452412+0000","last_became_active":"2024-04-25T10:05:10.459949+0000","last_became_peered":"2024-04-25T10:05:10.459949+0000","last_unstale":"2024-04-25T10:05:11.452412+0000","last_undegraded":"2024-04-25T10:05:11.452412+0000","last_fullsized":"2024-04-25T10:05:11.452412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:29.784640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.463941+0000","last_change":"2024-04-25T10:05:11.458748+0000","last_active":"2024-04-25T10:05:11.463941+0000","last_peered":"2024-04-25T10:05:11.463941+0000","last_clean":"2024-04-25T10:05:11.463941+0000","last_became_active":"2024-04-25T10:05:11.458440+0000","last_became_peered":"2024-04-25T10:05:11.458440+0000","last_unstale":"2024-04-25T10:05:11.463941+0000","last_undegraded":"2024-04-25T10:05:11.463941+0000","last_fullsized":"2024-04-25T10:05:11.463941+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:55.300183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456842+0000","last_change":"2024-04-25T10:05:10.455250+0000","last_active":"2024-04-25T10:05:12.456842+0000","last_peered":"2024-04-25T10:05:12.456842+0000","last_clean":"2024-04-25T10:05:12.456842+0000","last_became_active":"2024-04-25T10:05:10.454833+0000","last_became_peered":"2024-04-25T10:05:10.454833+0000","last_unstale":"2024-04-25T10:05:12.456842+0000","last_undegraded":"2024-04-25T10:05:12.456842+0000","last_fullsized":"2024-04-25T10:05:12.456842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.457011+0000","last_change":"2024-04-25T10:05:11.466533+0000","last_active":"2024-04-25T10:05:12.457011+0000","last_peered":"2024-04-25T10:05:12.457011+0000","last_clean":"2024-04-25T10:05:12.457011+0000","last_became_active":"2024-04-25T10:05:11.466256+0000","last_became_peered":"2024-04-25T10:05:11.466256+0000","last_unstale":"2024-04-25T10:05:12.457011+0000","last_undegraded":"2024-04-25T10:05:12.457011+0000","last_fullsized":"2024-04-25T10:05:12.457011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:41.766168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.476424+0000","last_change":"2024-04-25T10:05:10.462808+0000","last_active":"2024-04-25T10:05:11.476424+0000","last_peered":"2024-04-25T10:05:11.476424+0000","last_clean":"2024-04-25T10:05:11.476424+0000","last_became_active":"2024-04-25T10:05:10.462421+0000","last_became_peered":"2024-04-25T10:05:10.462421+0000","last_unstale":"2024-04-25T10:05:11.476424+0000","last_undegraded":"2024-04-25T10:05:11.476424+0000","last_fullsized":"2024-04-25T10:05:11.476424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:55.136701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456680+0000","last_change":"2024-04-25T10:05:11.470381+0000","last_active":"2024-04-25T10:05:12.456680+0000","last_peered":"2024-04-25T10:05:12.456680+0000","last_clean":"2024-04-25T10:05:12.456680+0000","last_became_active":"2024-04-25T10:05:11.470031+0000","last_became_peered":"2024-04-25T10:05:11.470031+0000","last_unstale":"2024-04-25T10:05:12.456680+0000","last_undegraded":"2024-04-25T10:05:12.456680+0000","last_fullsized":"2024-04-25T10:05:12.456680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.799101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.454513+0000","last_change":"2024-04-25T10:05:10.461232+0000","last_active":"2024-04-25T10:05:11.454513+0000","last_peered":"2024-04-25T10:05:11.454513+0000","last_clean":"2024-04-25T10:05:11.454513+0000","last_became_active":"2024-04-25T10:05:10.460905+0000","last_became_peered":"2024-04-25T10:05:10.460905+0000","last_unstale":"2024-04-25T10:05:11.454513+0000","last_undegraded":"2024-04-25T10:05:11.454513+0000","last_fullsized":"2024-04-25T10:05:11.454513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:54.294853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474543+0000","last_change":"2024-04-25T10:05:11.472902+0000","last_active":"2024-04-25T10:05:11.474543+0000","last_peered":"2024-04-25T10:05:11.474543+0000","last_clean":"2024-04-25T10:05:11.474543+0000","last_became_active":"2024-04-25T10:05:11.472622+0000","last_became_peered":"2024-04-25T10:05:11.472622+0000","last_unstale":"2024-04-25T10:05:11.474543+0000","last_undegraded":"2024-04-25T10:05:11.474543+0000","last_fullsized":"2024-04-25T10:05:11.474543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:24.616832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456461+0000","last_change":"2024-04-25T10:05:10.460759+0000","last_active":"2024-04-25T10:05:12.456461+0000","last_peered":"2024-04-25T10:05:12.456461+0000","last_clean":"2024-04-25T10:05:12.456461+0000","last_became_active":"2024-04-25T10:05:10.460485+0000","last_became_peered":"2024-04-25T10:05:10.460485+0000","last_unstale":"2024-04-25T10:05:12.456461+0000","last_undegraded":"2024-04-25T10:05:12.456461+0000","last_fullsized":"2024-04-25T10:05:12.456461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456829+0000","last_change":"2024-04-25T10:05:11.470739+0000","last_active":"2024-04-25T10:05:12.456829+0000","last_peered":"2024-04-25T10:05:12.456829+0000","last_clean":"2024-04-25T10:05:12.456829+0000","last_became_active":"2024-04-25T10:05:11.470408+0000","last_became_peered":"2024-04-25T10:05:11.470408+0000","last_unstale":"2024-04-25T10:05:12.456829+0000","last_undegraded":"2024-04-25T10:05:12.456829+0000","last_fullsized":"2024-04-25T10:05:12.456829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:17.789806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.457380+0000","last_change":"2024-04-25T10:05:10.458496+0000","last_active":"2024-04-25T10:05:11.457380+0000","last_peered":"2024-04-25T10:05:11.457380+0000","last_clean":"2024-04-25T10:05:11.457380+0000","last_became_active":"2024-04-25T10:05:10.458168+0000","last_became_peered":"2024-04-25T10:05:10.458168+0000","last_unstale":"2024-04-25T10:05:11.457380+0000","last_undegraded":"2024-04-25T10:05:11.457380+0000","last_fullsized":"2024-04-25T10:05:11.457380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:41.255007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.461196+0000","last_change":"2024-04-25T10:05:11.459826+0000","last_active":"2024-04-25T10:05:11.461196+0000","last_peered":"2024-04-25T10:05:11.461196+0000","last_clean":"2024-04-25T10:05:11.461196+0000","last_became_active":"2024-04-25T10:05:11.459570+0000","last_became_peered":"2024-04-25T10:05:11.459570+0000","last_unstale":"2024-04-25T10:05:11.461196+0000","last_undegraded":"2024-04-25T10:05:11.461196+0000","last_fullsized":"2024-04-25T10:05:11.461196+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:54.646134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474457+0000","last_change":"2024-04-25T10:05:10.455137+0000","last_active":"2024-04-25T10:05:11.474457+0000","last_peered":"2024-04-25T10:05:11.474457+0000","last_clean":"2024-04-25T10:05:11.474457+0000","last_became_active":"2024-04-25T10:05:10.454827+0000","last_became_peered":"2024-04-25T10:05:10.454827+0000","last_unstale":"2024-04-25T10:05:11.474457+0000","last_undegraded":"2024-04-25T10:05:11.474457+0000","last_fullsized":"2024-04-25T10:05:11.474457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:54.691904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.470977+0000","last_change":"2024-04-25T10:05:11.469441+0000","last_active":"2024-04-25T10:05:11.470977+0000","last_peered":"2024-04-25T10:05:11.470977+0000","last_clean":"2024-04-25T10:05:11.470977+0000","last_became_active":"2024-04-25T10:05:11.468649+0000","last_became_peered":"2024-04-25T10:05:11.468649+0000","last_unstale":"2024-04-25T10:05:11.470977+0000","last_undegraded":"2024-04-25T10:05:11.470977+0000","last_fullsized":"2024-04-25T10:05:11.470977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:17.789806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469799+0000","last_change":"2024-04-25T10:05:11.465690+0000","last_active":"2024-04-25T10:05:11.469799+0000","last_peered":"2024-04-25T10:05:11.469799+0000","last_clean":"2024-04-25T10:05:11.469799+0000","last_became_active":"2024-04-25T10:05:11.465422+0000","last_became_peered":"2024-04-25T10:05:11.465422+0000","last_unstale":"2024-04-25T10:05:11.469799+0000","last_undegraded":"2024-04-25T10:05:11.469799+0000","last_fullsized":"2024-04-25T10:05:11.469799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472427+0000","last_change":"2024-04-25T10:05:10.450133+0000","last_active":"2024-04-25T10:05:11.472427+0000","last_peered":"2024-04-25T10:05:11.472427+0000","last_clean":"2024-04-25T10:05:11.472427+0000","last_became_active":"2024-04-25T10:05:10.449482+0000","last_became_peered":"2024-04-25T10:05:10.449482+0000","last_unstale":"2024-04-25T10:05:11.472427+0000","last_undegraded":"2024-04-25T10:05:11.472427+0000","last_fullsized":"2024-04-25T10:05:11.472427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456402+0000","last_change":"2024-04-25T10:05:11.469301+0000","last_active":"2024-04-25T10:05:12.456402+0000","last_peered":"2024-04-25T10:05:12.456402+0000","last_clean":"2024-04-25T10:05:12.456402+0000","last_became_active":"2024-04-25T10:05:11.468980+0000","last_became_peered":"2024-04-25T10:05:11.468980+0000","last_unstale":"2024-04-25T10:05:12.456402+0000","last_undegraded":"2024-04-25T10:05:12.456402+0000","last_fullsized":"2024-04-25T10:05:12.456402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471641+0000","last_change":"2024-04-25T10:05:10.456749+0000","last_active":"2024-04-25T10:05:11.471641+0000","last_peered":"2024-04-25T10:05:11.471641+0000","last_clean":"2024-04-25T10:05:11.471641+0000","last_became_active":"2024-04-25T10:05:10.456397+0000","last_became_peered":"2024-04-25T10:05:10.456397+0000","last_unstale":"2024-04-25T10:05:11.471641+0000","last_undegraded":"2024-04-25T10:05:11.471641+0000","last_fullsized":"2024-04-25T10:05:11.471641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-04-25T10:05:09.441671+0000","last_change":"2024-04-25T10:05:09.441671+0000","last_active":"2024-04-25T10:05:09.429642+0000","last_peered":"2024-04-25T10:05:09.429642+0000","last_clean":"2024-04-25T10:05:09.429642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:09.441671+0000","last_undegraded":"2024-04-25T10:05:09.441671+0000","last_fullsized":"2024-04-25T10:05:09.441671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.457002+0000","last_change":"2024-04-25T10:05:10.454280+0000","last_active":"2024-04-25T10:05:12.457002+0000","last_peered":"2024-04-25T10:05:12.457002+0000","last_clean":"2024-04-25T10:05:12.457002+0000","last_became_active":"2024-04-25T10:05:10.453982+0000","last_became_peered":"2024-04-25T10:05:10.453982+0000","last_unstale":"2024-04-25T10:05:12.457002+0000","last_undegraded":"2024-04-25T10:05:12.457002+0000","last_fullsized":"2024-04-25T10:05:12.457002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":104,"acting":104,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":72548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":70846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":71876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":667002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":70320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":765952,"data_stored":641765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:14.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:14.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.582+0000 7ff18298b640 1 Processor -- start 2024-04-25T10:05:14.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.582+0000 7ff18298b640 1 -- start start 2024-04-25T10:05:14.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.583+0000 7ff18298b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff17c15dce0 0x7ff17c0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.583+0000 7ff18298b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff17c0d35b0 0x7ff17c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.583+0000 7ff18298b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff17c0d1f90 0x7ff17c0b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.583+0000 7ff18298b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff17c15e0c0 con 0x7ff17c15dce0 2024-04-25T10:05:14.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.583+0000 7ff18298b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff17c0d2510 con 0x7ff17c0d1f90 2024-04-25T10:05:14.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.583+0000 7ff18298b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c0d27f0 con 0x7ff17c0d35b0 2024-04-25T10:05:14.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.583+0000 7ff17bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff17c15dce0 0x7ff17c0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.583+0000 7ff17bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff17c15dce0 0x7ff17c0d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41774/0 (socket says 172.21.15.16:41774) 2024-04-25T10:05:14.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.583+0000 7ff17bfff640 1 -- 172.21.15.16:0/1214457853 learned_addr learned my addr 172.21.15.16:0/1214457853 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:14.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.583+0000 7ff17b7fe640 1 --2- 172.21.15.16:0/1214457853 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff17c0d35b0 0x7ff17c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.583+0000 7ff180f28640 1 --2- 172.21.15.16:0/1214457853 >> v2:172.21.15.186:3300/0 conn(0x7ff17c0d1f90 0x7ff17c0b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.583+0000 7ff17bfff640 1 -- 172.21.15.16:0/1214457853 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff17c0d35b0 msgr2=0x7ff17c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.583+0000 7ff17bfff640 1 --2- 172.21.15.16:0/1214457853 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff17c0d35b0 0x7ff17c0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.583+0000 7ff17bfff640 1 -- 172.21.15.16:0/1214457853 >> v2:172.21.15.186:3300/0 conn(0x7ff17c0d1f90 msgr2=0x7ff17c0b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.583+0000 7ff17bfff640 1 --2- 172.21.15.16:0/1214457853 >> v2:172.21.15.186:3300/0 conn(0x7ff17c0d1f90 0x7ff17c0b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.583+0000 7ff17bfff640 1 -- 172.21.15.16:0/1214457853 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c0d2ad0 con 0x7ff17c15dce0 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.584+0000 7ff17bfff640 1 --2- 172.21.15.16:0/1214457853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff17c15dce0 0x7ff17c0d1a50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff17c161710 tx=0x7ff16c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309b98292bd7ac02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.584+0000 7ff17affd640 1 -- 172.21.15.16:0/1214457853 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c037070 con 0x7ff17c15dce0 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.584+0000 7ff17affd640 1 -- 172.21.15.16:0/1214457853 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16c032440 con 0x7ff17c15dce0 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.584+0000 7ff17affd640 1 -- 172.21.15.16:0/1214457853 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c032810 con 0x7ff17c15dce0 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.584+0000 7ff18298b640 1 -- 172.21.15.16:0/1214457853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff17c15dce0 msgr2=0x7ff17c0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.584+0000 7ff18298b640 1 --2- 172.21.15.16:0/1214457853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff17c15dce0 0x7ff17c0d1a50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff17c161710 tx=0x7ff16c02a140 comp rx=0 tx=0).stop 2024-04-25T10:05:14.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.584+0000 7ff18298b640 1 -- 172.21.15.16:0/1214457853 shutdown_connections 2024-04-25T10:05:14.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.584+0000 7ff18298b640 1 --2- 172.21.15.16:0/1214457853 >> v2:172.21.15.186:3300/0 conn(0x7ff17c0d1f90 0x7ff17c0b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.584+0000 7ff18298b640 1 --2- 172.21.15.16:0/1214457853 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff17c0d35b0 0x7ff17c0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.584+0000 7ff18298b640 1 --2- 172.21.15.16:0/1214457853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff17c15dce0 0x7ff17c0d1a50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.584+0000 7ff18298b640 1 -- 172.21.15.16:0/1214457853 >> 172.21.15.16:0/1214457853 conn(0x7ff17c072900 msgr2=0x7ff17c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.584+0000 7ff18298b640 1 -- 172.21.15.16:0/1214457853 shutdown_connections 2024-04-25T10:05:14.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.584+0000 7ff18298b640 1 -- 172.21.15.16:0/1214457853 wait complete. 2024-04-25T10:05:14.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.586+0000 7ff18298b640 1 Processor -- start 2024-04-25T10:05:14.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.586+0000 7ff18298b640 1 -- start start 2024-04-25T10:05:14.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.586+0000 7ff18298b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff17c0d1f90 0x7ff17c160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.586+0000 7ff18298b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff17c0d35b0 0x7ff17c1e60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.586+0000 7ff18298b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff17c15dce0 0x7ff17c1e6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.586+0000 7ff18298b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c0b9820 con 0x7ff17c15dce0 2024-04-25T10:05:14.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.586+0000 7ff18298b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c0d3090 con 0x7ff17c0d35b0 2024-04-25T10:05:14.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.586+0000 7ff18298b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c0b99a0 con 0x7ff17c0d1f90 2024-04-25T10:05:14.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.587+0000 7ff180f28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff17c15dce0 0x7ff17c1e6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.587+0000 7ff180f28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff17c15dce0 0x7ff17c1e6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41780/0 (socket says 172.21.15.16:41780) 2024-04-25T10:05:14.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.587+0000 7ff180f28640 1 -- 172.21.15.16:0/2643101045 learned_addr learned my addr 172.21.15.16:0/2643101045 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:14.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.587+0000 7ff17bfff640 1 --2- 172.21.15.16:0/2643101045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff17c0d1f90 0x7ff17c160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.587+0000 7ff17b7fe640 1 --2- 172.21.15.16:0/2643101045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff17c0d35b0 0x7ff17c1e60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.587+0000 7ff180f28640 1 -- 172.21.15.16:0/2643101045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff17c0d1f90 msgr2=0x7ff17c160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.587+0000 7ff180f28640 1 --2- 172.21.15.16:0/2643101045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff17c0d1f90 0x7ff17c160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.587+0000 7ff180f28640 1 -- 172.21.15.16:0/2643101045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff17c0d35b0 msgr2=0x7ff17c1e60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.587+0000 7ff180f28640 1 --2- 172.21.15.16:0/2643101045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff17c0d35b0 0x7ff17c1e60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.587+0000 7ff180f28640 1 -- 172.21.15.16:0/2643101045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c0c8760 con 0x7ff17c15dce0 2024-04-25T10:05:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.587+0000 7ff17bfff640 1 --2- 172.21.15.16:0/2643101045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff17c0d1f90 0x7ff17c160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.587+0000 7ff180f28640 1 --2- 172.21.15.16:0/2643101045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff17c15dce0 0x7ff17c1e6630 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff174007d50 tx=0x7ff17400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.587+0000 7ff178ff9640 1 -- 172.21.15.16:0/2643101045 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174014420 con 0x7ff17c15dce0 2024-04-25T10:05:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.587+0000 7ff178ff9640 1 -- 172.21.15.16:0/2643101045 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff174019070 con 0x7ff17c15dce0 2024-04-25T10:05:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.587+0000 7ff18298b640 1 -- 172.21.15.16:0/2643101045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17c0c8a40 con 0x7ff17c15dce0 2024-04-25T10:05:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.587+0000 7ff178ff9640 1 -- 172.21.15.16:0/2643101045 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174014820 con 0x7ff17c15dce0 2024-04-25T10:05:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.587+0000 7ff18298b640 1 -- 172.21.15.16:0/2643101045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17c0c9000 con 0x7ff17c15dce0 2024-04-25T10:05:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.588+0000 7ff18298b640 1 -- 172.21.15.16:0/2643101045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17c0749b0 con 0x7ff17c15dce0 2024-04-25T10:05:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.589+0000 7ff178ff9640 1 -- 172.21.15.16:0/2643101045 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff1740149c0 con 0x7ff17c15dce0 2024-04-25T10:05:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.589+0000 7ff178ff9640 1 --2- 172.21.15.16:0/2643101045 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff16806b4d0 0x7ff16806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.589+0000 7ff178ff9640 1 -- 172.21.15.16:0/2643101045 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff17408a970 con 0x7ff17c15dce0 2024-04-25T10:05:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.589+0000 7ff17bfff640 1 --2- 172.21.15.16:0/2643101045 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff16806b4d0 0x7ff16806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.590+0000 7ff17bfff640 1 --2- 172.21.15.16:0/2643101045 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff16806b4d0 0x7ff16806d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff16c034000 tx=0x7ff16c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.593+0000 7ff178ff9640 1 -- 172.21.15.16:0/2643101045 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff1740597e0 con 0x7ff17c15dce0 2024-04-25T10:05:14.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.770+0000 7ff18298b640 1 -- 172.21.15.16:0/2643101045 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff17c0cf2d0 con 0x7ff16806b4d0 2024-04-25T10:05:14.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.777+0000 7ff178ff9640 1 -- 172.21.15.16:0/2643101045 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249961 (secure 0 0 0) 0x7ff17c0cf2d0 con 0x7ff16806b4d0 2024-04-25T10:05:14.785 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:14.786 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:05:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.780+0000 7ff18298b640 1 -- 172.21.15.16:0/2643101045 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff16806b4d0 msgr2=0x7ff16806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.780+0000 7ff18298b640 1 --2- 172.21.15.16:0/2643101045 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff16806b4d0 0x7ff16806d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff16c034000 tx=0x7ff16c02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.780+0000 7ff18298b640 1 -- 172.21.15.16:0/2643101045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff17c15dce0 msgr2=0x7ff17c1e6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.780+0000 7ff18298b640 1 --2- 172.21.15.16:0/2643101045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff17c15dce0 0x7ff17c1e6630 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff174007d50 tx=0x7ff17400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.780+0000 7ff18298b640 1 -- 172.21.15.16:0/2643101045 shutdown_connections 2024-04-25T10:05:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.780+0000 7ff18298b640 1 --2- 172.21.15.16:0/2643101045 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff16806b4d0 0x7ff16806d990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.780+0000 7ff18298b640 1 --2- 172.21.15.16:0/2643101045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff17c15dce0 0x7ff17c1e6630 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.780+0000 7ff18298b640 1 --2- 172.21.15.16:0/2643101045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff17c0d35b0 0x7ff17c1e60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.780+0000 7ff18298b640 1 --2- 172.21.15.16:0/2643101045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff17c0d1f90 0x7ff17c160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.780+0000 7ff18298b640 1 -- 172.21.15.16:0/2643101045 >> 172.21.15.16:0/2643101045 conn(0x7ff17c072900 msgr2=0x7ff17c0b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.780+0000 7ff18298b640 1 -- 172.21.15.16:0/2643101045 shutdown_connections 2024-04-25T10:05:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.780+0000 7ff18298b640 1 -- 172.21.15.16:0/2643101045 wait complete. 2024-04-25T10:05:14.796 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-25T10:05:13.781767+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235812,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738908,"statfs":{"total":773094113280,"available":772852641792,"internally_reserved":0,"allocated":2891776,"data_stored":1735009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.346535"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.475316+0000","last_change":"2024-04-25T10:05:11.473253+0000","last_active":"2024-04-25T10:05:11.475316+0000","last_peered":"2024-04-25T10:05:11.475316+0000","last_clean":"2024-04-25T10:05:11.475316+0000","last_became_active":"2024-04-25T10:05:11.472940+0000","last_became_peered":"2024-04-25T10:05:11.472940+0000","last_unstale":"2024-04-25T10:05:11.475316+0000","last_undegraded":"2024-04-25T10:05:11.475316+0000","last_fullsized":"2024-04-25T10:05:11.475316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:52.014434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455508+0000","last_change":"2024-04-25T10:05:11.461498+0000","last_active":"2024-04-25T10:05:12.455508+0000","last_peered":"2024-04-25T10:05:12.455508+0000","last_clean":"2024-04-25T10:05:12.455508+0000","last_became_active":"2024-04-25T10:05:11.461241+0000","last_became_peered":"2024-04-25T10:05:11.461241+0000","last_unstale":"2024-04-25T10:05:12.455508+0000","last_undegraded":"2024-04-25T10:05:12.455508+0000","last_fullsized":"2024-04-25T10:05:12.455508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472722+0000","last_change":"2024-04-25T10:05:11.469281+0000","last_active":"2024-04-25T10:05:11.472722+0000","last_peered":"2024-04-25T10:05:11.472722+0000","last_clean":"2024-04-25T10:05:11.472722+0000","last_became_active":"2024-04-25T10:05:11.468809+0000","last_became_peered":"2024-04-25T10:05:11.468809+0000","last_unstale":"2024-04-25T10:05:11.472722+0000","last_undegraded":"2024-04-25T10:05:11.472722+0000","last_fullsized":"2024-04-25T10:05:11.472722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:30.789969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455529+0000","last_change":"2024-04-25T10:05:11.466531+0000","last_active":"2024-04-25T10:05:12.455529+0000","last_peered":"2024-04-25T10:05:12.455529+0000","last_clean":"2024-04-25T10:05:12.455529+0000","last_became_active":"2024-04-25T10:05:11.466126+0000","last_became_peered":"2024-04-25T10:05:11.466126+0000","last_unstale":"2024-04-25T10:05:12.455529+0000","last_undegraded":"2024-04-25T10:05:12.455529+0000","last_fullsized":"2024-04-25T10:05:12.455529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469389+0000","last_change":"2024-04-25T10:05:11.466336+0000","last_active":"2024-04-25T10:05:11.469389+0000","last_peered":"2024-04-25T10:05:11.469389+0000","last_clean":"2024-04-25T10:05:11.469389+0000","last_became_active":"2024-04-25T10:05:11.466032+0000","last_became_peered":"2024-04-25T10:05:11.466032+0000","last_unstale":"2024-04-25T10:05:11.469389+0000","last_undegraded":"2024-04-25T10:05:11.469389+0000","last_fullsized":"2024-04-25T10:05:11.469389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:12.926321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456654+0000","last_change":"2024-04-25T10:05:11.461997+0000","last_active":"2024-04-25T10:05:12.456654+0000","last_peered":"2024-04-25T10:05:12.456654+0000","last_clean":"2024-04-25T10:05:12.456654+0000","last_became_active":"2024-04-25T10:05:11.461658+0000","last_became_peered":"2024-04-25T10:05:11.461658+0000","last_unstale":"2024-04-25T10:05:12.456654+0000","last_undegraded":"2024-04-25T10:05:12.456654+0000","last_fullsized":"2024-04-25T10:05:12.456654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.463064+0000","last_change":"2024-04-25T10:05:11.461631+0000","last_active":"2024-04-25T10:05:11.463064+0000","last_peered":"2024-04-25T10:05:11.463064+0000","last_clean":"2024-04-25T10:05:11.463064+0000","last_became_active":"2024-04-25T10:05:11.461327+0000","last_became_peered":"2024-04-25T10:05:11.461327+0000","last_unstale":"2024-04-25T10:05:11.463064+0000","last_undegraded":"2024-04-25T10:05:11.463064+0000","last_fullsized":"2024-04-25T10:05:11.463064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.473963+0000","last_change":"2024-04-25T10:05:11.473044+0000","last_active":"2024-04-25T10:05:11.473963+0000","last_peered":"2024-04-25T10:05:11.473963+0000","last_clean":"2024-04-25T10:05:11.473963+0000","last_became_active":"2024-04-25T10:05:11.472747+0000","last_became_peered":"2024-04-25T10:05:11.472747+0000","last_unstale":"2024-04-25T10:05:11.473963+0000","last_undegraded":"2024-04-25T10:05:11.473963+0000","last_fullsized":"2024-04-25T10:05:11.473963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474718+0000","last_change":"2024-04-25T10:05:11.470219+0000","last_active":"2024-04-25T10:05:11.474718+0000","last_peered":"2024-04-25T10:05:11.474718+0000","last_clean":"2024-04-25T10:05:11.474718+0000","last_became_active":"2024-04-25T10:05:11.469066+0000","last_became_peered":"2024-04-25T10:05:11.469066+0000","last_unstale":"2024-04-25T10:05:11.474718+0000","last_undegraded":"2024-04-25T10:05:11.474718+0000","last_fullsized":"2024-04-25T10:05:11.474718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:54.934726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.466367+0000","last_change":"2024-04-25T10:05:11.463262+0000","last_active":"2024-04-25T10:05:11.466367+0000","last_peered":"2024-04-25T10:05:11.466367+0000","last_clean":"2024-04-25T10:05:11.466367+0000","last_became_active":"2024-04-25T10:05:11.462970+0000","last_became_peered":"2024-04-25T10:05:11.462970+0000","last_unstale":"2024-04-25T10:05:11.466367+0000","last_undegraded":"2024-04-25T10:05:11.466367+0000","last_fullsized":"2024-04-25T10:05:11.466367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.464756+0000","last_change":"2024-04-25T10:05:11.458310+0000","last_active":"2024-04-25T10:05:11.464756+0000","last_peered":"2024-04-25T10:05:11.464756+0000","last_clean":"2024-04-25T10:05:11.464756+0000","last_became_active":"2024-04-25T10:05:11.457672+0000","last_became_peered":"2024-04-25T10:05:11.457672+0000","last_unstale":"2024-04-25T10:05:11.464756+0000","last_undegraded":"2024-04-25T10:05:11.464756+0000","last_fullsized":"2024-04-25T10:05:11.464756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:02.561557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471639+0000","last_change":"2024-04-25T10:05:11.470743+0000","last_active":"2024-04-25T10:05:11.471639+0000","last_peered":"2024-04-25T10:05:11.471639+0000","last_clean":"2024-04-25T10:05:11.471639+0000","last_became_active":"2024-04-25T10:05:11.470447+0000","last_became_peered":"2024-04-25T10:05:11.470447+0000","last_unstale":"2024-04-25T10:05:11.471639+0000","last_undegraded":"2024-04-25T10:05:11.471639+0000","last_fullsized":"2024-04-25T10:05:11.471639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:41.766168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.473455+0000","last_change":"2024-04-25T10:05:11.472530+0000","last_active":"2024-04-25T10:05:11.473455+0000","last_peered":"2024-04-25T10:05:11.473455+0000","last_clean":"2024-04-25T10:05:11.473455+0000","last_became_active":"2024-04-25T10:05:11.472164+0000","last_became_peered":"2024-04-25T10:05:11.472164+0000","last_unstale":"2024-04-25T10:05:11.473455+0000","last_undegraded":"2024-04-25T10:05:11.473455+0000","last_fullsized":"2024-04-25T10:05:11.473455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:08.525515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471968+0000","last_change":"2024-04-25T10:05:11.464186+0000","last_active":"2024-04-25T10:05:11.471968+0000","last_peered":"2024-04-25T10:05:11.471968+0000","last_clean":"2024-04-25T10:05:11.471968+0000","last_became_active":"2024-04-25T10:05:11.463925+0000","last_became_peered":"2024-04-25T10:05:11.463925+0000","last_unstale":"2024-04-25T10:05:11.471968+0000","last_undegraded":"2024-04-25T10:05:11.471968+0000","last_fullsized":"2024-04-25T10:05:11.471968+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:55.300183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474686+0000","last_change":"2024-04-25T10:05:11.473554+0000","last_active":"2024-04-25T10:05:11.474686+0000","last_peered":"2024-04-25T10:05:11.474686+0000","last_clean":"2024-04-25T10:05:11.474686+0000","last_became_active":"2024-04-25T10:05:11.473271+0000","last_became_peered":"2024-04-25T10:05:11.473271+0000","last_unstale":"2024-04-25T10:05:11.474686+0000","last_undegraded":"2024-04-25T10:05:11.474686+0000","last_fullsized":"2024-04-25T10:05:11.474686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:35.076762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474740+0000","last_change":"2024-04-25T10:05:11.471477+0000","last_active":"2024-04-25T10:05:11.474740+0000","last_peered":"2024-04-25T10:05:11.474740+0000","last_clean":"2024-04-25T10:05:11.474740+0000","last_became_active":"2024-04-25T10:05:11.471219+0000","last_became_peered":"2024-04-25T10:05:11.471219+0000","last_unstale":"2024-04-25T10:05:11.474740+0000","last_undegraded":"2024-04-25T10:05:11.474740+0000","last_fullsized":"2024-04-25T10:05:11.474740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456655+0000","last_change":"2024-04-25T10:05:11.464410+0000","last_active":"2024-04-25T10:05:12.456655+0000","last_peered":"2024-04-25T10:05:12.456655+0000","last_clean":"2024-04-25T10:05:12.456655+0000","last_became_active":"2024-04-25T10:05:11.464003+0000","last_became_peered":"2024-04-25T10:05:11.464003+0000","last_unstale":"2024-04-25T10:05:12.456655+0000","last_undegraded":"2024-04-25T10:05:12.456655+0000","last_fullsized":"2024-04-25T10:05:12.456655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:52.014434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456342+0000","last_change":"2024-04-25T10:05:11.462487+0000","last_active":"2024-04-25T10:05:12.456342+0000","last_peered":"2024-04-25T10:05:12.456342+0000","last_clean":"2024-04-25T10:05:12.456342+0000","last_became_active":"2024-04-25T10:05:11.462191+0000","last_became_peered":"2024-04-25T10:05:11.462191+0000","last_unstale":"2024-04-25T10:05:12.456342+0000","last_undegraded":"2024-04-25T10:05:12.456342+0000","last_fullsized":"2024-04-25T10:05:12.456342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469823+0000","last_change":"2024-04-25T10:05:11.467406+0000","last_active":"2024-04-25T10:05:11.469823+0000","last_peered":"2024-04-25T10:05:11.469823+0000","last_clean":"2024-04-25T10:05:11.469823+0000","last_became_active":"2024-04-25T10:05:11.467126+0000","last_became_peered":"2024-04-25T10:05:11.467126+0000","last_unstale":"2024-04-25T10:05:11.469823+0000","last_undegraded":"2024-04-25T10:05:11.469823+0000","last_fullsized":"2024-04-25T10:05:11.469823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:52.014434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455642+0000","last_change":"2024-04-25T10:05:11.469452+0000","last_active":"2024-04-25T10:05:12.455642+0000","last_peered":"2024-04-25T10:05:12.455642+0000","last_clean":"2024-04-25T10:05:12.455642+0000","last_became_active":"2024-04-25T10:05:11.465896+0000","last_became_peered":"2024-04-25T10:05:11.465896+0000","last_unstale":"2024-04-25T10:05:12.455642+0000","last_undegraded":"2024-04-25T10:05:12.455642+0000","last_fullsized":"2024-04-25T10:05:12.455642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:14.506306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.460957+0000","last_change":"2024-04-25T10:05:11.457595+0000","last_active":"2024-04-25T10:05:11.460957+0000","last_peered":"2024-04-25T10:05:11.460957+0000","last_clean":"2024-04-25T10:05:11.460957+0000","last_became_active":"2024-04-25T10:05:11.457320+0000","last_became_peered":"2024-04-25T10:05:11.457320+0000","last_unstale":"2024-04-25T10:05:11.460957+0000","last_undegraded":"2024-04-25T10:05:11.460957+0000","last_fullsized":"2024-04-25T10:05:11.460957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:40.831935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472125+0000","last_change":"2024-04-25T10:05:11.469113+0000","last_active":"2024-04-25T10:05:11.472125+0000","last_peered":"2024-04-25T10:05:11.472125+0000","last_clean":"2024-04-25T10:05:11.472125+0000","last_became_active":"2024-04-25T10:05:11.468759+0000","last_became_peered":"2024-04-25T10:05:11.468759+0000","last_unstale":"2024-04-25T10:05:11.472125+0000","last_undegraded":"2024-04-25T10:05:11.472125+0000","last_fullsized":"2024-04-25T10:05:11.472125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471666+0000","last_change":"2024-04-25T10:05:11.470055+0000","last_active":"2024-04-25T10:05:11.471666+0000","last_peered":"2024-04-25T10:05:11.471666+0000","last_clean":"2024-04-25T10:05:11.471666+0000","last_became_active":"2024-04-25T10:05:11.468974+0000","last_became_peered":"2024-04-25T10:05:11.468974+0000","last_unstale":"2024-04-25T10:05:11.471666+0000","last_undegraded":"2024-04-25T10:05:11.471666+0000","last_fullsized":"2024-04-25T10:05:11.471666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.228600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469311+0000","last_change":"2024-04-25T10:05:11.462234+0000","last_active":"2024-04-25T10:05:11.469311+0000","last_peered":"2024-04-25T10:05:11.469311+0000","last_clean":"2024-04-25T10:05:11.469311+0000","last_became_active":"2024-04-25T10:05:11.461975+0000","last_became_peered":"2024-04-25T10:05:11.461975+0000","last_unstale":"2024-04-25T10:05:11.469311+0000","last_undegraded":"2024-04-25T10:05:11.469311+0000","last_fullsized":"2024-04-25T10:05:11.469311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:02.561557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455396+0000","last_change":"2024-04-25T10:05:11.463406+0000","last_active":"2024-04-25T10:05:12.455396+0000","last_peered":"2024-04-25T10:05:12.455396+0000","last_clean":"2024-04-25T10:05:12.455396+0000","last_became_active":"2024-04-25T10:05:11.463139+0000","last_became_peered":"2024-04-25T10:05:11.463139+0000","last_unstale":"2024-04-25T10:05:12.455396+0000","last_undegraded":"2024-04-25T10:05:12.455396+0000","last_fullsized":"2024-04-25T10:05:12.455396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456384+0000","last_change":"2024-04-25T10:05:11.453168+0000","last_active":"2024-04-25T10:05:12.456384+0000","last_peered":"2024-04-25T10:05:12.456384+0000","last_clean":"2024-04-25T10:05:12.456384+0000","last_became_active":"2024-04-25T10:05:11.452238+0000","last_became_peered":"2024-04-25T10:05:11.452238+0000","last_unstale":"2024-04-25T10:05:12.456384+0000","last_undegraded":"2024-04-25T10:05:12.456384+0000","last_fullsized":"2024-04-25T10:05:12.456384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-04-25T10:05:09.441702+0000","last_change":"2024-04-25T10:05:09.441702+0000","last_active":"2024-04-25T10:05:09.429642+0000","last_peered":"2024-04-25T10:05:09.429642+0000","last_clean":"2024-04-25T10:05:09.429642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:09.441702+0000","last_undegraded":"2024-04-25T10:05:09.441702+0000","last_fullsized":"2024-04-25T10:05:09.441702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456962+0000","last_change":"2024-04-25T10:05:11.469760+0000","last_active":"2024-04-25T10:05:12.456962+0000","last_peered":"2024-04-25T10:05:12.456962+0000","last_clean":"2024-04-25T10:05:12.456962+0000","last_became_active":"2024-04-25T10:05:11.469490+0000","last_became_peered":"2024-04-25T10:05:11.469490+0000","last_unstale":"2024-04-25T10:05:12.456962+0000","last_undegraded":"2024-04-25T10:05:12.456962+0000","last_fullsized":"2024-04-25T10:05:12.456962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.228600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.475721+0000","last_change":"2024-04-25T10:05:10.461940+0000","last_active":"2024-04-25T10:05:11.475721+0000","last_peered":"2024-04-25T10:05:11.475721+0000","last_clean":"2024-04-25T10:05:11.475721+0000","last_became_active":"2024-04-25T10:05:10.461633+0000","last_became_peered":"2024-04-25T10:05:10.461633+0000","last_unstale":"2024-04-25T10:05:11.475721+0000","last_undegraded":"2024-04-25T10:05:11.475721+0000","last_fullsized":"2024-04-25T10:05:11.475721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.456539+0000","last_change":"2024-04-25T10:05:10.460823+0000","last_active":"2024-04-25T10:05:11.456539+0000","last_peered":"2024-04-25T10:05:11.456539+0000","last_clean":"2024-04-25T10:05:11.456539+0000","last_became_active":"2024-04-25T10:05:10.460480+0000","last_became_peered":"2024-04-25T10:05:10.460480+0000","last_unstale":"2024-04-25T10:05:11.456539+0000","last_undegraded":"2024-04-25T10:05:11.456539+0000","last_fullsized":"2024-04-25T10:05:11.456539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:31.375084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.450458+0000","last_change":"2024-04-25T10:05:10.461713+0000","last_active":"2024-04-25T10:05:11.450458+0000","last_peered":"2024-04-25T10:05:11.450458+0000","last_clean":"2024-04-25T10:05:11.450458+0000","last_became_active":"2024-04-25T10:05:10.461379+0000","last_became_peered":"2024-04-25T10:05:10.461379+0000","last_unstale":"2024-04-25T10:05:11.450458+0000","last_undegraded":"2024-04-25T10:05:11.450458+0000","last_fullsized":"2024-04-25T10:05:11.450458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456510+0000","last_change":"2024-04-25T10:05:11.466408+0000","last_active":"2024-04-25T10:05:12.456510+0000","last_peered":"2024-04-25T10:05:12.456510+0000","last_clean":"2024-04-25T10:05:12.456510+0000","last_became_active":"2024-04-25T10:05:11.465611+0000","last_became_peered":"2024-04-25T10:05:11.465611+0000","last_unstale":"2024-04-25T10:05:12.456510+0000","last_undegraded":"2024-04-25T10:05:12.456510+0000","last_fullsized":"2024-04-25T10:05:12.456510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456657+0000","last_change":"2024-04-25T10:05:10.449054+0000","last_active":"2024-04-25T10:05:12.456657+0000","last_peered":"2024-04-25T10:05:12.456657+0000","last_clean":"2024-04-25T10:05:12.456657+0000","last_became_active":"2024-04-25T10:05:10.448743+0000","last_became_peered":"2024-04-25T10:05:10.448743+0000","last_unstale":"2024-04-25T10:05:12.456657+0000","last_undegraded":"2024-04-25T10:05:12.456657+0000","last_fullsized":"2024-04-25T10:05:12.456657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:22.897294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455686+0000","last_change":"2024-04-25T10:05:11.453268+0000","last_active":"2024-04-25T10:05:12.455686+0000","last_peered":"2024-04-25T10:05:12.455686+0000","last_clean":"2024-04-25T10:05:12.455686+0000","last_became_active":"2024-04-25T10:05:11.452971+0000","last_became_peered":"2024-04-25T10:05:11.452971+0000","last_unstale":"2024-04-25T10:05:12.455686+0000","last_undegraded":"2024-04-25T10:05:12.455686+0000","last_fullsized":"2024-04-25T10:05:12.455686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:56.142030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.452585+0000","last_change":"2024-04-25T10:05:10.452247+0000","last_active":"2024-04-25T10:05:11.452585+0000","last_peered":"2024-04-25T10:05:11.452585+0000","last_clean":"2024-04-25T10:05:11.452585+0000","last_became_active":"2024-04-25T10:05:10.451925+0000","last_became_peered":"2024-04-25T10:05:10.451925+0000","last_unstale":"2024-04-25T10:05:11.452585+0000","last_undegraded":"2024-04-25T10:05:11.452585+0000","last_fullsized":"2024-04-25T10:05:11.452585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:53.640804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472671+0000","last_change":"2024-04-25T10:05:10.459898+0000","last_active":"2024-04-25T10:05:11.472671+0000","last_peered":"2024-04-25T10:05:11.472671+0000","last_clean":"2024-04-25T10:05:11.472671+0000","last_became_active":"2024-04-25T10:05:10.459250+0000","last_became_peered":"2024-04-25T10:05:10.459250+0000","last_unstale":"2024-04-25T10:05:11.472671+0000","last_undegraded":"2024-04-25T10:05:11.472671+0000","last_fullsized":"2024-04-25T10:05:11.472671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:41.255007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.473052+0000","last_change":"2024-04-25T10:05:11.471498+0000","last_active":"2024-04-25T10:05:11.473052+0000","last_peered":"2024-04-25T10:05:11.473052+0000","last_clean":"2024-04-25T10:05:11.473052+0000","last_became_active":"2024-04-25T10:05:11.471193+0000","last_became_peered":"2024-04-25T10:05:11.471193+0000","last_unstale":"2024-04-25T10:05:11.473052+0000","last_undegraded":"2024-04-25T10:05:11.473052+0000","last_fullsized":"2024-04-25T10:05:11.473052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:45.980553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455560+0000","last_change":"2024-04-25T10:05:10.450534+0000","last_active":"2024-04-25T10:05:12.455560+0000","last_peered":"2024-04-25T10:05:12.455560+0000","last_clean":"2024-04-25T10:05:12.455560+0000","last_became_active":"2024-04-25T10:05:10.450251+0000","last_became_peered":"2024-04-25T10:05:10.450251+0000","last_unstale":"2024-04-25T10:05:12.455560+0000","last_undegraded":"2024-04-25T10:05:12.455560+0000","last_fullsized":"2024-04-25T10:05:12.455560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:22.897294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.468770+0000","last_change":"2024-04-25T10:05:11.463480+0000","last_active":"2024-04-25T10:05:11.468770+0000","last_peered":"2024-04-25T10:05:11.468770+0000","last_clean":"2024-04-25T10:05:11.468770+0000","last_became_active":"2024-04-25T10:05:11.460995+0000","last_became_peered":"2024-04-25T10:05:11.460995+0000","last_unstale":"2024-04-25T10:05:11.468770+0000","last_undegraded":"2024-04-25T10:05:11.468770+0000","last_fullsized":"2024-04-25T10:05:11.468770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:56.142030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.464540+0000","last_change":"2024-04-25T10:05:10.462323+0000","last_active":"2024-04-25T10:05:11.464540+0000","last_peered":"2024-04-25T10:05:11.464540+0000","last_clean":"2024-04-25T10:05:11.464540+0000","last_became_active":"2024-04-25T10:05:10.462118+0000","last_became_peered":"2024-04-25T10:05:10.462118+0000","last_unstale":"2024-04-25T10:05:11.464540+0000","last_undegraded":"2024-04-25T10:05:11.464540+0000","last_fullsized":"2024-04-25T10:05:11.464540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:39.826605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471805+0000","last_change":"2024-04-25T10:05:11.467396+0000","last_active":"2024-04-25T10:05:11.471805+0000","last_peered":"2024-04-25T10:05:11.471805+0000","last_clean":"2024-04-25T10:05:11.471805+0000","last_became_active":"2024-04-25T10:05:11.467112+0000","last_became_peered":"2024-04-25T10:05:11.467112+0000","last_unstale":"2024-04-25T10:05:11.471805+0000","last_undegraded":"2024-04-25T10:05:11.471805+0000","last_fullsized":"2024-04-25T10:05:11.471805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.451714+0000","last_change":"2024-04-25T10:05:10.458429+0000","last_active":"2024-04-25T10:05:11.451714+0000","last_peered":"2024-04-25T10:05:11.451714+0000","last_clean":"2024-04-25T10:05:11.451714+0000","last_became_active":"2024-04-25T10:05:10.457958+0000","last_became_peered":"2024-04-25T10:05:10.457958+0000","last_unstale":"2024-04-25T10:05:11.451714+0000","last_undegraded":"2024-04-25T10:05:11.451714+0000","last_fullsized":"2024-04-25T10:05:11.451714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:23.585636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469949+0000","last_change":"2024-04-25T10:05:11.465006+0000","last_active":"2024-04-25T10:05:11.469949+0000","last_peered":"2024-04-25T10:05:11.469949+0000","last_clean":"2024-04-25T10:05:11.469949+0000","last_became_active":"2024-04-25T10:05:11.464719+0000","last_became_peered":"2024-04-25T10:05:11.464719+0000","last_unstale":"2024-04-25T10:05:11.469949+0000","last_undegraded":"2024-04-25T10:05:11.469949+0000","last_fullsized":"2024-04-25T10:05:11.469949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:54.934726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.459177+0000","last_change":"2024-04-25T10:05:10.456938+0000","last_active":"2024-04-25T10:05:11.459177+0000","last_peered":"2024-04-25T10:05:11.459177+0000","last_clean":"2024-04-25T10:05:11.459177+0000","last_became_active":"2024-04-25T10:05:10.456644+0000","last_became_peered":"2024-04-25T10:05:10.456644+0000","last_unstale":"2024-04-25T10:05:11.459177+0000","last_undegraded":"2024-04-25T10:05:11.459177+0000","last_fullsized":"2024-04-25T10:05:11.459177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456188+0000","last_change":"2024-04-25T10:05:11.456178+0000","last_active":"2024-04-25T10:05:12.456188+0000","last_peered":"2024-04-25T10:05:12.456188+0000","last_clean":"2024-04-25T10:05:12.456188+0000","last_became_active":"2024-04-25T10:05:11.455879+0000","last_became_peered":"2024-04-25T10:05:11.455879+0000","last_unstale":"2024-04-25T10:05:12.456188+0000","last_undegraded":"2024-04-25T10:05:12.456188+0000","last_fullsized":"2024-04-25T10:05:12.456188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:55.300183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456275+0000","last_change":"2024-04-25T10:05:10.455684+0000","last_active":"2024-04-25T10:05:12.456275+0000","last_peered":"2024-04-25T10:05:12.456275+0000","last_clean":"2024-04-25T10:05:12.456275+0000","last_became_active":"2024-04-25T10:05:10.455342+0000","last_became_peered":"2024-04-25T10:05:10.455342+0000","last_unstale":"2024-04-25T10:05:12.456275+0000","last_undegraded":"2024-04-25T10:05:12.456275+0000","last_fullsized":"2024-04-25T10:05:12.456275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:01.556228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469903+0000","last_change":"2024-04-25T10:05:11.466622+0000","last_active":"2024-04-25T10:05:11.469903+0000","last_peered":"2024-04-25T10:05:11.469903+0000","last_clean":"2024-04-25T10:05:11.469903+0000","last_became_active":"2024-04-25T10:05:11.466346+0000","last_became_peered":"2024-04-25T10:05:11.466346+0000","last_unstale":"2024-04-25T10:05:11.469903+0000","last_undegraded":"2024-04-25T10:05:11.469903+0000","last_fullsized":"2024-04-25T10:05:11.469903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.468144+0000","last_change":"2024-04-25T10:05:10.457614+0000","last_active":"2024-04-25T10:05:11.468144+0000","last_peered":"2024-04-25T10:05:11.468144+0000","last_clean":"2024-04-25T10:05:11.468144+0000","last_became_active":"2024-04-25T10:05:10.457332+0000","last_became_peered":"2024-04-25T10:05:10.457332+0000","last_unstale":"2024-04-25T10:05:11.468144+0000","last_undegraded":"2024-04-25T10:05:11.468144+0000","last_fullsized":"2024-04-25T10:05:11.468144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:01.556228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455780+0000","last_change":"2024-04-25T10:05:11.460589+0000","last_active":"2024-04-25T10:05:12.455780+0000","last_peered":"2024-04-25T10:05:12.455780+0000","last_clean":"2024-04-25T10:05:12.455780+0000","last_became_active":"2024-04-25T10:05:11.459433+0000","last_became_peered":"2024-04-25T10:05:11.459433+0000","last_unstale":"2024-04-25T10:05:12.455780+0000","last_undegraded":"2024-04-25T10:05:12.455780+0000","last_fullsized":"2024-04-25T10:05:12.455780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:55.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456910+0000","last_change":"2024-04-25T10:05:10.458692+0000","last_active":"2024-04-25T10:05:12.456910+0000","last_peered":"2024-04-25T10:05:12.456910+0000","last_clean":"2024-04-25T10:05:12.456910+0000","last_became_active":"2024-04-25T10:05:10.458090+0000","last_became_peered":"2024-04-25T10:05:10.458090+0000","last_unstale":"2024-04-25T10:05:12.456910+0000","last_undegraded":"2024-04-25T10:05:12.456910+0000","last_fullsized":"2024-04-25T10:05:12.456910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:23.585636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.468810+0000","last_change":"2024-04-25T10:05:11.464532+0000","last_active":"2024-04-25T10:05:11.468810+0000","last_peered":"2024-04-25T10:05:11.468810+0000","last_clean":"2024-04-25T10:05:11.468810+0000","last_became_active":"2024-04-25T10:05:11.464186+0000","last_became_peered":"2024-04-25T10:05:11.464186+0000","last_unstale":"2024-04-25T10:05:11.468810+0000","last_undegraded":"2024-04-25T10:05:11.468810+0000","last_fullsized":"2024-04-25T10:05:11.468810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:55.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-04-25T10:05:09.441695+0000","last_change":"2024-04-25T10:05:09.441695+0000","last_active":"2024-04-25T10:05:09.429642+0000","last_peered":"2024-04-25T10:05:09.429642+0000","last_clean":"2024-04-25T10:05:09.429642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:09.441695+0000","last_undegraded":"2024-04-25T10:05:09.441695+0000","last_fullsized":"2024-04-25T10:05:09.441695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.475897+0000","last_change":"2024-04-25T10:05:11.474519+0000","last_active":"2024-04-25T10:05:11.475897+0000","last_peered":"2024-04-25T10:05:11.475897+0000","last_clean":"2024-04-25T10:05:11.475897+0000","last_became_active":"2024-04-25T10:05:11.472675+0000","last_became_peered":"2024-04-25T10:05:11.472675+0000","last_unstale":"2024-04-25T10:05:11.475897+0000","last_undegraded":"2024-04-25T10:05:11.475897+0000","last_fullsized":"2024-04-25T10:05:11.475897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.799101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.458209+0000","last_change":"2024-04-25T10:05:10.453364+0000","last_active":"2024-04-25T10:05:11.458209+0000","last_peered":"2024-04-25T10:05:11.458209+0000","last_clean":"2024-04-25T10:05:11.458209+0000","last_became_active":"2024-04-25T10:05:10.453072+0000","last_became_peered":"2024-04-25T10:05:10.453072+0000","last_unstale":"2024-04-25T10:05:11.458209+0000","last_undegraded":"2024-04-25T10:05:11.458209+0000","last_fullsized":"2024-04-25T10:05:11.458209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:53.640804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469818+0000","last_change":"2024-04-25T10:05:11.467814+0000","last_active":"2024-04-25T10:05:11.469818+0000","last_peered":"2024-04-25T10:05:11.469818+0000","last_clean":"2024-04-25T10:05:11.469818+0000","last_became_active":"2024-04-25T10:05:11.467496+0000","last_became_peered":"2024-04-25T10:05:11.467496+0000","last_unstale":"2024-04-25T10:05:11.469818+0000","last_undegraded":"2024-04-25T10:05:11.469818+0000","last_fullsized":"2024-04-25T10:05:11.469818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.799101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456108+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:05:12.456108+0000","last_peered":"2024-04-25T10:05:12.456108+0000","last_clean":"2024-04-25T10:05:12.456108+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:05:12.456108+0000","last_undegraded":"2024-04-25T10:05:12.456108+0000","last_fullsized":"2024-04-25T10:05:12.456108+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.452172+0000","last_change":"2024-04-25T10:05:10.458170+0000","last_active":"2024-04-25T10:05:11.452172+0000","last_peered":"2024-04-25T10:05:11.452172+0000","last_clean":"2024-04-25T10:05:11.452172+0000","last_became_active":"2024-04-25T10:05:10.457794+0000","last_became_peered":"2024-04-25T10:05:10.457794+0000","last_unstale":"2024-04-25T10:05:11.452172+0000","last_undegraded":"2024-04-25T10:05:11.452172+0000","last_fullsized":"2024-04-25T10:05:11.452172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:11.920992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456600+0000","last_change":"2024-04-25T10:05:11.462574+0000","last_active":"2024-04-25T10:05:12.456600+0000","last_peered":"2024-04-25T10:05:12.456600+0000","last_clean":"2024-04-25T10:05:12.456600+0000","last_became_active":"2024-04-25T10:05:11.461617+0000","last_became_peered":"2024-04-25T10:05:11.461617+0000","last_unstale":"2024-04-25T10:05:12.456600+0000","last_undegraded":"2024-04-25T10:05:12.456600+0000","last_fullsized":"2024-04-25T10:05:12.456600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.451455+0000","last_change":"2024-04-25T10:05:10.464334+0000","last_active":"2024-04-25T10:05:11.451455+0000","last_peered":"2024-04-25T10:05:11.451455+0000","last_clean":"2024-04-25T10:05:11.451455+0000","last_became_active":"2024-04-25T10:05:10.463991+0000","last_became_peered":"2024-04-25T10:05:10.463991+0000","last_unstale":"2024-04-25T10:05:11.451455+0000","last_undegraded":"2024-04-25T10:05:11.451455+0000","last_fullsized":"2024-04-25T10:05:11.451455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469159+0000","last_change":"2024-04-25T10:05:11.464866+0000","last_active":"2024-04-25T10:05:11.469159+0000","last_peered":"2024-04-25T10:05:11.469159+0000","last_clean":"2024-04-25T10:05:11.469159+0000","last_became_active":"2024-04-25T10:05:11.464504+0000","last_became_peered":"2024-04-25T10:05:11.464504+0000","last_unstale":"2024-04-25T10:05:11.469159+0000","last_undegraded":"2024-04-25T10:05:11.469159+0000","last_fullsized":"2024-04-25T10:05:11.469159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.449849+0000","last_change":"2024-04-25T10:05:10.458220+0000","last_active":"2024-04-25T10:05:11.449849+0000","last_peered":"2024-04-25T10:05:11.449849+0000","last_clean":"2024-04-25T10:05:11.449849+0000","last_became_active":"2024-04-25T10:05:10.457936+0000","last_became_peered":"2024-04-25T10:05:10.457936+0000","last_unstale":"2024-04-25T10:05:11.449849+0000","last_undegraded":"2024-04-25T10:05:11.449849+0000","last_fullsized":"2024-04-25T10:05:11.449849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:31.375084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.451883+0000","last_change":"2024-04-25T10:05:10.463388+0000","last_active":"2024-04-25T10:05:11.451883+0000","last_peered":"2024-04-25T10:05:11.451883+0000","last_clean":"2024-04-25T10:05:11.451883+0000","last_became_active":"2024-04-25T10:05:10.463041+0000","last_became_peered":"2024-04-25T10:05:10.463041+0000","last_unstale":"2024-04-25T10:05:11.451883+0000","last_undegraded":"2024-04-25T10:05:11.451883+0000","last_fullsized":"2024-04-25T10:05:11.451883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:13.500977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.463844+0000","last_change":"2024-04-25T10:05:11.462360+0000","last_active":"2024-04-25T10:05:11.463844+0000","last_peered":"2024-04-25T10:05:11.463844+0000","last_clean":"2024-04-25T10:05:11.463844+0000","last_became_active":"2024-04-25T10:05:11.462050+0000","last_became_peered":"2024-04-25T10:05:11.462050+0000","last_unstale":"2024-04-25T10:05:11.463844+0000","last_undegraded":"2024-04-25T10:05:11.463844+0000","last_fullsized":"2024-04-25T10:05:11.463844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:14.506306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-04-25T10:05:09.441740+0000","last_change":"2024-04-25T10:05:09.441740+0000","last_active":"2024-04-25T10:05:09.429642+0000","last_peered":"2024-04-25T10:05:09.429642+0000","last_clean":"2024-04-25T10:05:09.429642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:09.441740+0000","last_undegraded":"2024-04-25T10:05:09.441740+0000","last_fullsized":"2024-04-25T10:05:09.441740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455433+0000","last_change":"2024-04-25T10:05:11.464184+0000","last_active":"2024-04-25T10:05:12.455433+0000","last_peered":"2024-04-25T10:05:12.455433+0000","last_clean":"2024-04-25T10:05:12.455433+0000","last_became_active":"2024-04-25T10:05:11.463894+0000","last_became_peered":"2024-04-25T10:05:11.463894+0000","last_unstale":"2024-04-25T10:05:12.455433+0000","last_undegraded":"2024-04-25T10:05:12.455433+0000","last_fullsized":"2024-04-25T10:05:12.455433+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.452412+0000","last_change":"2024-04-25T10:05:10.460261+0000","last_active":"2024-04-25T10:05:11.452412+0000","last_peered":"2024-04-25T10:05:11.452412+0000","last_clean":"2024-04-25T10:05:11.452412+0000","last_became_active":"2024-04-25T10:05:10.459949+0000","last_became_peered":"2024-04-25T10:05:10.459949+0000","last_unstale":"2024-04-25T10:05:11.452412+0000","last_undegraded":"2024-04-25T10:05:11.452412+0000","last_fullsized":"2024-04-25T10:05:11.452412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:29.784640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.463941+0000","last_change":"2024-04-25T10:05:11.458748+0000","last_active":"2024-04-25T10:05:11.463941+0000","last_peered":"2024-04-25T10:05:11.463941+0000","last_clean":"2024-04-25T10:05:11.463941+0000","last_became_active":"2024-04-25T10:05:11.458440+0000","last_became_peered":"2024-04-25T10:05:11.458440+0000","last_unstale":"2024-04-25T10:05:11.463941+0000","last_undegraded":"2024-04-25T10:05:11.463941+0000","last_fullsized":"2024-04-25T10:05:11.463941+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:55.300183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456842+0000","last_change":"2024-04-25T10:05:10.455250+0000","last_active":"2024-04-25T10:05:12.456842+0000","last_peered":"2024-04-25T10:05:12.456842+0000","last_clean":"2024-04-25T10:05:12.456842+0000","last_became_active":"2024-04-25T10:05:10.454833+0000","last_became_peered":"2024-04-25T10:05:10.454833+0000","last_unstale":"2024-04-25T10:05:12.456842+0000","last_undegraded":"2024-04-25T10:05:12.456842+0000","last_fullsized":"2024-04-25T10:05:12.456842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.457011+0000","last_change":"2024-04-25T10:05:11.466533+0000","last_active":"2024-04-25T10:05:12.457011+0000","last_peered":"2024-04-25T10:05:12.457011+0000","last_clean":"2024-04-25T10:05:12.457011+0000","last_became_active":"2024-04-25T10:05:11.466256+0000","last_became_peered":"2024-04-25T10:05:11.466256+0000","last_unstale":"2024-04-25T10:05:12.457011+0000","last_undegraded":"2024-04-25T10:05:12.457011+0000","last_fullsized":"2024-04-25T10:05:12.457011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:41.766168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.476424+0000","last_change":"2024-04-25T10:05:10.462808+0000","last_active":"2024-04-25T10:05:11.476424+0000","last_peered":"2024-04-25T10:05:11.476424+0000","last_clean":"2024-04-25T10:05:11.476424+0000","last_became_active":"2024-04-25T10:05:10.462421+0000","last_became_peered":"2024-04-25T10:05:10.462421+0000","last_unstale":"2024-04-25T10:05:11.476424+0000","last_undegraded":"2024-04-25T10:05:11.476424+0000","last_fullsized":"2024-04-25T10:05:11.476424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:55.136701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456680+0000","last_change":"2024-04-25T10:05:11.470381+0000","last_active":"2024-04-25T10:05:12.456680+0000","last_peered":"2024-04-25T10:05:12.456680+0000","last_clean":"2024-04-25T10:05:12.456680+0000","last_became_active":"2024-04-25T10:05:11.470031+0000","last_became_peered":"2024-04-25T10:05:11.470031+0000","last_unstale":"2024-04-25T10:05:12.456680+0000","last_undegraded":"2024-04-25T10:05:12.456680+0000","last_fullsized":"2024-04-25T10:05:12.456680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.799101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.454513+0000","last_change":"2024-04-25T10:05:10.461232+0000","last_active":"2024-04-25T10:05:11.454513+0000","last_peered":"2024-04-25T10:05:11.454513+0000","last_clean":"2024-04-25T10:05:11.454513+0000","last_became_active":"2024-04-25T10:05:10.460905+0000","last_became_peered":"2024-04-25T10:05:10.460905+0000","last_unstale":"2024-04-25T10:05:11.454513+0000","last_undegraded":"2024-04-25T10:05:11.454513+0000","last_fullsized":"2024-04-25T10:05:11.454513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:54.294853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474543+0000","last_change":"2024-04-25T10:05:11.472902+0000","last_active":"2024-04-25T10:05:11.474543+0000","last_peered":"2024-04-25T10:05:11.474543+0000","last_clean":"2024-04-25T10:05:11.474543+0000","last_became_active":"2024-04-25T10:05:11.472622+0000","last_became_peered":"2024-04-25T10:05:11.472622+0000","last_unstale":"2024-04-25T10:05:11.474543+0000","last_undegraded":"2024-04-25T10:05:11.474543+0000","last_fullsized":"2024-04-25T10:05:11.474543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:24.616832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456461+0000","last_change":"2024-04-25T10:05:10.460759+0000","last_active":"2024-04-25T10:05:12.456461+0000","last_peered":"2024-04-25T10:05:12.456461+0000","last_clean":"2024-04-25T10:05:12.456461+0000","last_became_active":"2024-04-25T10:05:10.460485+0000","last_became_peered":"2024-04-25T10:05:10.460485+0000","last_unstale":"2024-04-25T10:05:12.456461+0000","last_undegraded":"2024-04-25T10:05:12.456461+0000","last_fullsized":"2024-04-25T10:05:12.456461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456829+0000","last_change":"2024-04-25T10:05:11.470739+0000","last_active":"2024-04-25T10:05:12.456829+0000","last_peered":"2024-04-25T10:05:12.456829+0000","last_clean":"2024-04-25T10:05:12.456829+0000","last_became_active":"2024-04-25T10:05:11.470408+0000","last_became_peered":"2024-04-25T10:05:11.470408+0000","last_unstale":"2024-04-25T10:05:12.456829+0000","last_undegraded":"2024-04-25T10:05:12.456829+0000","last_fullsized":"2024-04-25T10:05:12.456829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:17.789806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.457380+0000","last_change":"2024-04-25T10:05:10.458496+0000","last_active":"2024-04-25T10:05:11.457380+0000","last_peered":"2024-04-25T10:05:11.457380+0000","last_clean":"2024-04-25T10:05:11.457380+0000","last_became_active":"2024-04-25T10:05:10.458168+0000","last_became_peered":"2024-04-25T10:05:10.458168+0000","last_unstale":"2024-04-25T10:05:11.457380+0000","last_undegraded":"2024-04-25T10:05:11.457380+0000","last_fullsized":"2024-04-25T10:05:11.457380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:41.255007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.461196+0000","last_change":"2024-04-25T10:05:11.459826+0000","last_active":"2024-04-25T10:05:11.461196+0000","last_peered":"2024-04-25T10:05:11.461196+0000","last_clean":"2024-04-25T10:05:11.461196+0000","last_became_active":"2024-04-25T10:05:11.459570+0000","last_became_peered":"2024-04-25T10:05:11.459570+0000","last_unstale":"2024-04-25T10:05:11.461196+0000","last_undegraded":"2024-04-25T10:05:11.461196+0000","last_fullsized":"2024-04-25T10:05:11.461196+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:54.646134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474457+0000","last_change":"2024-04-25T10:05:10.455137+0000","last_active":"2024-04-25T10:05:11.474457+0000","last_peered":"2024-04-25T10:05:11.474457+0000","last_clean":"2024-04-25T10:05:11.474457+0000","last_became_active":"2024-04-25T10:05:10.454827+0000","last_became_peered":"2024-04-25T10:05:10.454827+0000","last_unstale":"2024-04-25T10:05:11.474457+0000","last_undegraded":"2024-04-25T10:05:11.474457+0000","last_fullsized":"2024-04-25T10:05:11.474457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:54.691904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.470977+0000","last_change":"2024-04-25T10:05:11.469441+0000","last_active":"2024-04-25T10:05:11.470977+0000","last_peered":"2024-04-25T10:05:11.470977+0000","last_clean":"2024-04-25T10:05:11.470977+0000","last_became_active":"2024-04-25T10:05:11.468649+0000","last_became_peered":"2024-04-25T10:05:11.468649+0000","last_unstale":"2024-04-25T10:05:11.470977+0000","last_undegraded":"2024-04-25T10:05:11.470977+0000","last_fullsized":"2024-04-25T10:05:11.470977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:17.789806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469799+0000","last_change":"2024-04-25T10:05:11.465690+0000","last_active":"2024-04-25T10:05:11.469799+0000","last_peered":"2024-04-25T10:05:11.469799+0000","last_clean":"2024-04-25T10:05:11.469799+0000","last_became_active":"2024-04-25T10:05:11.465422+0000","last_became_peered":"2024-04-25T10:05:11.465422+0000","last_unstale":"2024-04-25T10:05:11.469799+0000","last_undegraded":"2024-04-25T10:05:11.469799+0000","last_fullsized":"2024-04-25T10:05:11.469799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472427+0000","last_change":"2024-04-25T10:05:10.450133+0000","last_active":"2024-04-25T10:05:11.472427+0000","last_peered":"2024-04-25T10:05:11.472427+0000","last_clean":"2024-04-25T10:05:11.472427+0000","last_became_active":"2024-04-25T10:05:10.449482+0000","last_became_peered":"2024-04-25T10:05:10.449482+0000","last_unstale":"2024-04-25T10:05:11.472427+0000","last_undegraded":"2024-04-25T10:05:11.472427+0000","last_fullsized":"2024-04-25T10:05:11.472427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456402+0000","last_change":"2024-04-25T10:05:11.469301+0000","last_active":"2024-04-25T10:05:12.456402+0000","last_peered":"2024-04-25T10:05:12.456402+0000","last_clean":"2024-04-25T10:05:12.456402+0000","last_became_active":"2024-04-25T10:05:11.468980+0000","last_became_peered":"2024-04-25T10:05:11.468980+0000","last_unstale":"2024-04-25T10:05:12.456402+0000","last_undegraded":"2024-04-25T10:05:12.456402+0000","last_fullsized":"2024-04-25T10:05:12.456402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471641+0000","last_change":"2024-04-25T10:05:10.456749+0000","last_active":"2024-04-25T10:05:11.471641+0000","last_peered":"2024-04-25T10:05:11.471641+0000","last_clean":"2024-04-25T10:05:11.471641+0000","last_became_active":"2024-04-25T10:05:10.456397+0000","last_became_peered":"2024-04-25T10:05:10.456397+0000","last_unstale":"2024-04-25T10:05:11.471641+0000","last_undegraded":"2024-04-25T10:05:11.471641+0000","last_fullsized":"2024-04-25T10:05:11.471641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-04-25T10:05:09.441671+0000","last_change":"2024-04-25T10:05:09.441671+0000","last_active":"2024-04-25T10:05:09.429642+0000","last_peered":"2024-04-25T10:05:09.429642+0000","last_clean":"2024-04-25T10:05:09.429642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:09.441671+0000","last_undegraded":"2024-04-25T10:05:09.441671+0000","last_fullsized":"2024-04-25T10:05:09.441671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.457002+0000","last_change":"2024-04-25T10:05:10.454280+0000","last_active":"2024-04-25T10:05:12.457002+0000","last_peered":"2024-04-25T10:05:12.457002+0000","last_clean":"2024-04-25T10:05:12.457002+0000","last_became_active":"2024-04-25T10:05:10.453982+0000","last_became_peered":"2024-04-25T10:05:10.453982+0000","last_unstale":"2024-04-25T10:05:12.457002+0000","last_undegraded":"2024-04-25T10:05:12.457002+0000","last_fullsized":"2024-04-25T10:05:12.457002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":104,"acting":104,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":72548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":70846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":71876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":667002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":70320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":765952,"data_stored":641765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:14.800 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:14.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.931+0000 7f8c0d78b640 1 Processor -- start 2024-04-25T10:05:14.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.931+0000 7f8c0d78b640 1 -- start start 2024-04-25T10:05:14.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c0d78b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8c0815dc30 0x7f8c0806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c0d78b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c080b1b30 0x7f8c080d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c0d78b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c0806c1f0 con 0x7f8c0806cc00 2024-04-25T10:05:14.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c0d78b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c080cfb40 con 0x7f8c0815dc30 2024-04-25T10:05:14.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c0d78b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c080b2060 con 0x7f8c080b1b30 2024-04-25T10:05:14.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c067fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8c0806cc00 0x7f8c080b1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34796/0 (socket says 172.21.15.16:34796) 2024-04-25T10:05:14.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c067fc640 1 -- 172.21.15.16:0/3854984174 learned_addr learned my addr 172.21.15.16:0/3854984174 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:14.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c05ffb640 1 -- 172.21.15.16:0/3854984174 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636126311 0 0) 0x7f8bfc001110 con 0x7f8c0806cc00 2024-04-25T10:05:14.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c05ffb640 1 -- 172.21.15.16:0/3854984174 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bf0003520 con 0x7f8c0806cc00 2024-04-25T10:05:14.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c06ffd640 1 --2- 172.21.15.16:0/3854984174 >> v2:172.21.15.186:3300/0 conn(0x7f8c0815dc30 0x7f8c0806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c077fe640 1 --2- 172.21.15.16:0/3854984174 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c080b1b30 0x7f8c080d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c05ffb640 1 -- 172.21.15.16:0/3854984174 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2634579814 0 0) 0x7f8bf0003520 con 0x7f8c0806cc00 2024-04-25T10:05:14.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c05ffb640 1 -- 172.21.15.16:0/3854984174 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c080b1b30 msgr2=0x7f8c080d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c05ffb640 1 --2- 172.21.15.16:0/3854984174 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c080b1b30 0x7f8c080d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c05ffb640 1 -- 172.21.15.16:0/3854984174 >> v2:172.21.15.186:3300/0 conn(0x7f8c0815dc30 msgr2=0x7f8c0806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c05ffb640 1 --2- 172.21.15.16:0/3854984174 >> v2:172.21.15.186:3300/0 conn(0x7f8c0815dc30 0x7f8c0806c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c05ffb640 1 -- 172.21.15.16:0/3854984174 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c080b2340 con 0x7f8c0806cc00 2024-04-25T10:05:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c05ffb640 1 -- 172.21.15.16:0/3854984174 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f8bfc001840 con 0x7f8c0806cc00 2024-04-25T10:05:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.932+0000 7f8c05ffb640 1 -- 172.21.15.16:0/3854984174 >> v1:172.21.15.16:6789/0 conn(0x7f8c0806cc00 legacy=0x7f8c080b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.933+0000 7f8c05ffb640 1 --2- 172.21.15.16:0/3854984174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf00039b0 0x7f8bf0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.933+0000 7f8c05ffb640 1 --2- 172.21.15.16:0/3854984174 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c080b1b30 0x7f8bf0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.933+0000 7f8c05ffb640 1 --2- 172.21.15.16:0/3854984174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf0004d40 0x7f8bf00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.933+0000 7f8c05ffb640 1 -- 172.21.15.16:0/3854984174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c080cfb40 con 0x7f8bf0004d40 2024-04-25T10:05:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.933+0000 7f8c05ffb640 1 -- 172.21.15.16:0/3854984174 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c080b2060 con 0x7f8bf00039b0 2024-04-25T10:05:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.933+0000 7f8c05ffb640 1 -- 172.21.15.16:0/3854984174 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf000ba70 con 0x7f8c080b1b30 2024-04-25T10:05:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.933+0000 7f8c077fe640 1 --2- 172.21.15.16:0/3854984174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf0004d40 0x7f8bf00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.933+0000 7f8c067fc640 1 --2- 172.21.15.16:0/3854984174 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c080b1b30 0x7f8bf0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.933+0000 7f8c06ffd640 1 --2- 172.21.15.16:0/3854984174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf00039b0 0x7f8bf0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.933+0000 7f8c077fe640 1 -- 172.21.15.16:0/3854984174 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c080b1b30 msgr2=0x7f8bf0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.933+0000 7f8c077fe640 1 --2- 172.21.15.16:0/3854984174 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c080b1b30 0x7f8bf0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.933+0000 7f8c077fe640 1 -- 172.21.15.16:0/3854984174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf00039b0 msgr2=0x7f8bf0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.933+0000 7f8c077fe640 1 --2- 172.21.15.16:0/3854984174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf00039b0 0x7f8bf0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.933+0000 7f8c077fe640 1 -- 172.21.15.16:0/3854984174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf0003520 con 0x7f8bf0004d40 2024-04-25T10:05:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.934+0000 7f8c077fe640 1 --2- 172.21.15.16:0/3854984174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf0004d40 0x7f8bf00051c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8c00007390 tx=0x7f8bf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d0305fc0f08bcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.934+0000 7f8c05ffb640 1 -- 172.21.15.16:0/3854984174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00009be0 con 0x7f8bf0004d40 2024-04-25T10:05:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.934+0000 7f8c05ffb640 1 -- 172.21.15.16:0/3854984174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0002e620 con 0x7f8bf0004d40 2024-04-25T10:05:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.934+0000 7f8c05ffb640 1 -- 172.21.15.16:0/3854984174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0002e9f0 con 0x7f8bf0004d40 2024-04-25T10:05:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.934+0000 7f8c0d78b640 1 -- 172.21.15.16:0/3854984174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf0004d40 msgr2=0x7f8bf00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.934+0000 7f8c0d78b640 1 --2- 172.21.15.16:0/3854984174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf0004d40 0x7f8bf00051c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8c00007390 tx=0x7f8bf0005830 comp rx=0 tx=0).stop 2024-04-25T10:05:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.934+0000 7f8c077fe640 1 -- 172.21.15.16:0/3854984174 reap_dead start 2024-04-25T10:05:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.934+0000 7f8c0d78b640 1 -- 172.21.15.16:0/3854984174 shutdown_connections 2024-04-25T10:05:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.934+0000 7f8c0d78b640 1 -- 172.21.15.16:0/3854984174 >> 172.21.15.16:0/3854984174 conn(0x7f8c080d1900 msgr2=0x7f8c080d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.935+0000 7f8c0d78b640 1 -- 172.21.15.16:0/3854984174 shutdown_connections 2024-04-25T10:05:14.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.935+0000 7f8c0d78b640 1 -- 172.21.15.16:0/3854984174 wait complete. 2024-04-25T10:05:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.936+0000 7f8c0d78b640 1 Processor -- start 2024-04-25T10:05:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.936+0000 7f8c0d78b640 1 -- start start 2024-04-25T10:05:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.937+0000 7f8c0d78b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8bf0004d40 0x7f8c081f4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.937+0000 7f8c0d78b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c08153fe0 0x7f8c080b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.937+0000 7f8c0d78b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c0815e190 0x7f8c0815e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.937+0000 7f8c0d78b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0815e010 con 0x7f8c0815e190 2024-04-25T10:05:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.937+0000 7f8c0d78b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c080cfde0 con 0x7f8c08153fe0 2024-04-25T10:05:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.937+0000 7f8c0d78b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c080d0300 con 0x7f8bf0004d40 2024-04-25T10:05:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.937+0000 7f8c077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c0815e190 0x7f8c0815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.937+0000 7f8c077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c0815e190 0x7f8c0815e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41802/0 (socket says 172.21.15.16:41802) 2024-04-25T10:05:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.937+0000 7f8c077fe640 1 -- 172.21.15.16:0/3784593935 learned_addr learned my addr 172.21.15.16:0/3784593935 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.937+0000 7f8c06ffd640 1 --2- 172.21.15.16:0/3784593935 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8bf0004d40 0x7f8c081f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.937+0000 7f8c067fc640 1 --2- 172.21.15.16:0/3784593935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c08153fe0 0x7f8c080b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.937+0000 7f8c077fe640 1 -- 172.21.15.16:0/3784593935 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8bf0004d40 msgr2=0x7f8c081f4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.937+0000 7f8c077fe640 1 --2- 172.21.15.16:0/3784593935 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8bf0004d40 0x7f8c081f4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.937+0000 7f8c077fe640 1 -- 172.21.15.16:0/3784593935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c08153fe0 msgr2=0x7f8c080b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.937+0000 7f8c077fe640 1 --2- 172.21.15.16:0/3784593935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c08153fe0 0x7f8c080b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.937+0000 7f8c077fe640 1 -- 172.21.15.16:0/3784593935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c080b7290 con 0x7f8c0815e190 2024-04-25T10:05:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.937+0000 7f8c077fe640 1 --2- 172.21.15.16:0/3784593935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c0815e190 0x7f8c0815e640 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8c00002230 tx=0x7f8c00004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.937+0000 7f8bef7fe640 1 -- 172.21.15.16:0/3784593935 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00009be0 con 0x7f8c0815e190 2024-04-25T10:05:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.938+0000 7f8bef7fe640 1 -- 172.21.15.16:0/3784593935 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0002e420 con 0x7f8c0815e190 2024-04-25T10:05:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.938+0000 7f8c0d78b640 1 -- 172.21.15.16:0/3784593935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0815fa30 con 0x7f8c0815e190 2024-04-25T10:05:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.938+0000 7f8bef7fe640 1 -- 172.21.15.16:0/3784593935 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00038280 con 0x7f8c0815e190 2024-04-25T10:05:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.938+0000 7f8c0d78b640 1 -- 172.21.15.16:0/3784593935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0815fff0 con 0x7f8c0815e190 2024-04-25T10:05:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.939+0000 7f8c0d78b640 1 -- 172.21.15.16:0/3784593935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c08067850 con 0x7f8c0815e190 2024-04-25T10:05:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.939+0000 7f8bef7fe640 1 -- 172.21.15.16:0/3784593935 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c00038870 con 0x7f8c0815e190 2024-04-25T10:05:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.940+0000 7f8bef7fe640 1 --2- 172.21.15.16:0/3784593935 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8bf406b500 0x7f8bf406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.940+0000 7f8bef7fe640 1 -- 172.21.15.16:0/3784593935 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c000aa7e0 con 0x7f8c0815e190 2024-04-25T10:05:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.940+0000 7f8c06ffd640 1 --2- 172.21.15.16:0/3784593935 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8bf406b500 0x7f8bf406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.940+0000 7f8c06ffd640 1 --2- 172.21.15.16:0/3784593935 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8bf406b500 0x7f8bf406d9c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8c081f18c0 tx=0x7f8bf8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:14.944+0000 7f8bef7fe640 1 -- 172.21.15.16:0/3784593935 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c00032030 con 0x7f8c0815e190 2024-04-25T10:05:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.119+0000 7f8c0d78b640 1 -- 172.21.15.16:0/3784593935 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8c080b7030 con 0x7f8bf406b500 2024-04-25T10:05:15.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.126+0000 7f8bef7fe640 1 -- 172.21.15.16:0/3784593935 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249961 (secure 0 0 0) 0x7f8bf800b7c0 con 0x7f8bf406b500 2024-04-25T10:05:15.134 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:15.135 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:05:15.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.128+0000 7f8c0d78b640 1 -- 172.21.15.16:0/3784593935 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8bf406b500 msgr2=0x7f8bf406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.128+0000 7f8c0d78b640 1 --2- 172.21.15.16:0/3784593935 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8bf406b500 0x7f8bf406d9c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8c081f18c0 tx=0x7f8bf8008040 comp rx=0 tx=0).stop 2024-04-25T10:05:15.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.128+0000 7f8c0d78b640 1 -- 172.21.15.16:0/3784593935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c0815e190 msgr2=0x7f8c0815e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.128+0000 7f8c0d78b640 1 --2- 172.21.15.16:0/3784593935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c0815e190 0x7f8c0815e640 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8c00002230 tx=0x7f8c00004ab0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.128+0000 7f8c0d78b640 1 -- 172.21.15.16:0/3784593935 shutdown_connections 2024-04-25T10:05:15.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.128+0000 7f8c0d78b640 1 --2- 172.21.15.16:0/3784593935 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8bf406b500 0x7f8bf406d9c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.128+0000 7f8c0d78b640 1 --2- 172.21.15.16:0/3784593935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c0815e190 0x7f8c0815e640 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.128+0000 7f8c0d78b640 1 --2- 172.21.15.16:0/3784593935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c08153fe0 0x7f8c080b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.128+0000 7f8c0d78b640 1 --2- 172.21.15.16:0/3784593935 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8bf0004d40 0x7f8c081f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.128+0000 7f8c0d78b640 1 -- 172.21.15.16:0/3784593935 >> 172.21.15.16:0/3784593935 conn(0x7f8c080d1900 msgr2=0x7f8c080ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.128+0000 7f8c0d78b640 1 -- 172.21.15.16:0/3784593935 shutdown_connections 2024-04-25T10:05:15.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.128+0000 7f8c0d78b640 1 -- 172.21.15.16:0/3784593935 wait complete. 2024-04-25T10:05:15.144 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-25T10:05:13.781767+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235812,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738908,"statfs":{"total":773094113280,"available":772852641792,"internally_reserved":0,"allocated":2891776,"data_stored":1735009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.346535"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.475316+0000","last_change":"2024-04-25T10:05:11.473253+0000","last_active":"2024-04-25T10:05:11.475316+0000","last_peered":"2024-04-25T10:05:11.475316+0000","last_clean":"2024-04-25T10:05:11.475316+0000","last_became_active":"2024-04-25T10:05:11.472940+0000","last_became_peered":"2024-04-25T10:05:11.472940+0000","last_unstale":"2024-04-25T10:05:11.475316+0000","last_undegraded":"2024-04-25T10:05:11.475316+0000","last_fullsized":"2024-04-25T10:05:11.475316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:52.014434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455508+0000","last_change":"2024-04-25T10:05:11.461498+0000","last_active":"2024-04-25T10:05:12.455508+0000","last_peered":"2024-04-25T10:05:12.455508+0000","last_clean":"2024-04-25T10:05:12.455508+0000","last_became_active":"2024-04-25T10:05:11.461241+0000","last_became_peered":"2024-04-25T10:05:11.461241+0000","last_unstale":"2024-04-25T10:05:12.455508+0000","last_undegraded":"2024-04-25T10:05:12.455508+0000","last_fullsized":"2024-04-25T10:05:12.455508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472722+0000","last_change":"2024-04-25T10:05:11.469281+0000","last_active":"2024-04-25T10:05:11.472722+0000","last_peered":"2024-04-25T10:05:11.472722+0000","last_clean":"2024-04-25T10:05:11.472722+0000","last_became_active":"2024-04-25T10:05:11.468809+0000","last_became_peered":"2024-04-25T10:05:11.468809+0000","last_unstale":"2024-04-25T10:05:11.472722+0000","last_undegraded":"2024-04-25T10:05:11.472722+0000","last_fullsized":"2024-04-25T10:05:11.472722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:30.789969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455529+0000","last_change":"2024-04-25T10:05:11.466531+0000","last_active":"2024-04-25T10:05:12.455529+0000","last_peered":"2024-04-25T10:05:12.455529+0000","last_clean":"2024-04-25T10:05:12.455529+0000","last_became_active":"2024-04-25T10:05:11.466126+0000","last_became_peered":"2024-04-25T10:05:11.466126+0000","last_unstale":"2024-04-25T10:05:12.455529+0000","last_undegraded":"2024-04-25T10:05:12.455529+0000","last_fullsized":"2024-04-25T10:05:12.455529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469389+0000","last_change":"2024-04-25T10:05:11.466336+0000","last_active":"2024-04-25T10:05:11.469389+0000","last_peered":"2024-04-25T10:05:11.469389+0000","last_clean":"2024-04-25T10:05:11.469389+0000","last_became_active":"2024-04-25T10:05:11.466032+0000","last_became_peered":"2024-04-25T10:05:11.466032+0000","last_unstale":"2024-04-25T10:05:11.469389+0000","last_undegraded":"2024-04-25T10:05:11.469389+0000","last_fullsized":"2024-04-25T10:05:11.469389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:12.926321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456654+0000","last_change":"2024-04-25T10:05:11.461997+0000","last_active":"2024-04-25T10:05:12.456654+0000","last_peered":"2024-04-25T10:05:12.456654+0000","last_clean":"2024-04-25T10:05:12.456654+0000","last_became_active":"2024-04-25T10:05:11.461658+0000","last_became_peered":"2024-04-25T10:05:11.461658+0000","last_unstale":"2024-04-25T10:05:12.456654+0000","last_undegraded":"2024-04-25T10:05:12.456654+0000","last_fullsized":"2024-04-25T10:05:12.456654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.463064+0000","last_change":"2024-04-25T10:05:11.461631+0000","last_active":"2024-04-25T10:05:11.463064+0000","last_peered":"2024-04-25T10:05:11.463064+0000","last_clean":"2024-04-25T10:05:11.463064+0000","last_became_active":"2024-04-25T10:05:11.461327+0000","last_became_peered":"2024-04-25T10:05:11.461327+0000","last_unstale":"2024-04-25T10:05:11.463064+0000","last_undegraded":"2024-04-25T10:05:11.463064+0000","last_fullsized":"2024-04-25T10:05:11.463064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.473963+0000","last_change":"2024-04-25T10:05:11.473044+0000","last_active":"2024-04-25T10:05:11.473963+0000","last_peered":"2024-04-25T10:05:11.473963+0000","last_clean":"2024-04-25T10:05:11.473963+0000","last_became_active":"2024-04-25T10:05:11.472747+0000","last_became_peered":"2024-04-25T10:05:11.472747+0000","last_unstale":"2024-04-25T10:05:11.473963+0000","last_undegraded":"2024-04-25T10:05:11.473963+0000","last_fullsized":"2024-04-25T10:05:11.473963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474718+0000","last_change":"2024-04-25T10:05:11.470219+0000","last_active":"2024-04-25T10:05:11.474718+0000","last_peered":"2024-04-25T10:05:11.474718+0000","last_clean":"2024-04-25T10:05:11.474718+0000","last_became_active":"2024-04-25T10:05:11.469066+0000","last_became_peered":"2024-04-25T10:05:11.469066+0000","last_unstale":"2024-04-25T10:05:11.474718+0000","last_undegraded":"2024-04-25T10:05:11.474718+0000","last_fullsized":"2024-04-25T10:05:11.474718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:54.934726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.466367+0000","last_change":"2024-04-25T10:05:11.463262+0000","last_active":"2024-04-25T10:05:11.466367+0000","last_peered":"2024-04-25T10:05:11.466367+0000","last_clean":"2024-04-25T10:05:11.466367+0000","last_became_active":"2024-04-25T10:05:11.462970+0000","last_became_peered":"2024-04-25T10:05:11.462970+0000","last_unstale":"2024-04-25T10:05:11.466367+0000","last_undegraded":"2024-04-25T10:05:11.466367+0000","last_fullsized":"2024-04-25T10:05:11.466367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.464756+0000","last_change":"2024-04-25T10:05:11.458310+0000","last_active":"2024-04-25T10:05:11.464756+0000","last_peered":"2024-04-25T10:05:11.464756+0000","last_clean":"2024-04-25T10:05:11.464756+0000","last_became_active":"2024-04-25T10:05:11.457672+0000","last_became_peered":"2024-04-25T10:05:11.457672+0000","last_unstale":"2024-04-25T10:05:11.464756+0000","last_undegraded":"2024-04-25T10:05:11.464756+0000","last_fullsized":"2024-04-25T10:05:11.464756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:02.561557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471639+0000","last_change":"2024-04-25T10:05:11.470743+0000","last_active":"2024-04-25T10:05:11.471639+0000","last_peered":"2024-04-25T10:05:11.471639+0000","last_clean":"2024-04-25T10:05:11.471639+0000","last_became_active":"2024-04-25T10:05:11.470447+0000","last_became_peered":"2024-04-25T10:05:11.470447+0000","last_unstale":"2024-04-25T10:05:11.471639+0000","last_undegraded":"2024-04-25T10:05:11.471639+0000","last_fullsized":"2024-04-25T10:05:11.471639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:41.766168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.473455+0000","last_change":"2024-04-25T10:05:11.472530+0000","last_active":"2024-04-25T10:05:11.473455+0000","last_peered":"2024-04-25T10:05:11.473455+0000","last_clean":"2024-04-25T10:05:11.473455+0000","last_became_active":"2024-04-25T10:05:11.472164+0000","last_became_peered":"2024-04-25T10:05:11.472164+0000","last_unstale":"2024-04-25T10:05:11.473455+0000","last_undegraded":"2024-04-25T10:05:11.473455+0000","last_fullsized":"2024-04-25T10:05:11.473455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:08.525515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471968+0000","last_change":"2024-04-25T10:05:11.464186+0000","last_active":"2024-04-25T10:05:11.471968+0000","last_peered":"2024-04-25T10:05:11.471968+0000","last_clean":"2024-04-25T10:05:11.471968+0000","last_became_active":"2024-04-25T10:05:11.463925+0000","last_became_peered":"2024-04-25T10:05:11.463925+0000","last_unstale":"2024-04-25T10:05:11.471968+0000","last_undegraded":"2024-04-25T10:05:11.471968+0000","last_fullsized":"2024-04-25T10:05:11.471968+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:55.300183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474686+0000","last_change":"2024-04-25T10:05:11.473554+0000","last_active":"2024-04-25T10:05:11.474686+0000","last_peered":"2024-04-25T10:05:11.474686+0000","last_clean":"2024-04-25T10:05:11.474686+0000","last_became_active":"2024-04-25T10:05:11.473271+0000","last_became_peered":"2024-04-25T10:05:11.473271+0000","last_unstale":"2024-04-25T10:05:11.474686+0000","last_undegraded":"2024-04-25T10:05:11.474686+0000","last_fullsized":"2024-04-25T10:05:11.474686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:35.076762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474740+0000","last_change":"2024-04-25T10:05:11.471477+0000","last_active":"2024-04-25T10:05:11.474740+0000","last_peered":"2024-04-25T10:05:11.474740+0000","last_clean":"2024-04-25T10:05:11.474740+0000","last_became_active":"2024-04-25T10:05:11.471219+0000","last_became_peered":"2024-04-25T10:05:11.471219+0000","last_unstale":"2024-04-25T10:05:11.474740+0000","last_undegraded":"2024-04-25T10:05:11.474740+0000","last_fullsized":"2024-04-25T10:05:11.474740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456655+0000","last_change":"2024-04-25T10:05:11.464410+0000","last_active":"2024-04-25T10:05:12.456655+0000","last_peered":"2024-04-25T10:05:12.456655+0000","last_clean":"2024-04-25T10:05:12.456655+0000","last_became_active":"2024-04-25T10:05:11.464003+0000","last_became_peered":"2024-04-25T10:05:11.464003+0000","last_unstale":"2024-04-25T10:05:12.456655+0000","last_undegraded":"2024-04-25T10:05:12.456655+0000","last_fullsized":"2024-04-25T10:05:12.456655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:52.014434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456342+0000","last_change":"2024-04-25T10:05:11.462487+0000","last_active":"2024-04-25T10:05:12.456342+0000","last_peered":"2024-04-25T10:05:12.456342+0000","last_clean":"2024-04-25T10:05:12.456342+0000","last_became_active":"2024-04-25T10:05:11.462191+0000","last_became_peered":"2024-04-25T10:05:11.462191+0000","last_unstale":"2024-04-25T10:05:12.456342+0000","last_undegraded":"2024-04-25T10:05:12.456342+0000","last_fullsized":"2024-04-25T10:05:12.456342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469823+0000","last_change":"2024-04-25T10:05:11.467406+0000","last_active":"2024-04-25T10:05:11.469823+0000","last_peered":"2024-04-25T10:05:11.469823+0000","last_clean":"2024-04-25T10:05:11.469823+0000","last_became_active":"2024-04-25T10:05:11.467126+0000","last_became_peered":"2024-04-25T10:05:11.467126+0000","last_unstale":"2024-04-25T10:05:11.469823+0000","last_undegraded":"2024-04-25T10:05:11.469823+0000","last_fullsized":"2024-04-25T10:05:11.469823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:52.014434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455642+0000","last_change":"2024-04-25T10:05:11.469452+0000","last_active":"2024-04-25T10:05:12.455642+0000","last_peered":"2024-04-25T10:05:12.455642+0000","last_clean":"2024-04-25T10:05:12.455642+0000","last_became_active":"2024-04-25T10:05:11.465896+0000","last_became_peered":"2024-04-25T10:05:11.465896+0000","last_unstale":"2024-04-25T10:05:12.455642+0000","last_undegraded":"2024-04-25T10:05:12.455642+0000","last_fullsized":"2024-04-25T10:05:12.455642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:14.506306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.460957+0000","last_change":"2024-04-25T10:05:11.457595+0000","last_active":"2024-04-25T10:05:11.460957+0000","last_peered":"2024-04-25T10:05:11.460957+0000","last_clean":"2024-04-25T10:05:11.460957+0000","last_became_active":"2024-04-25T10:05:11.457320+0000","last_became_peered":"2024-04-25T10:05:11.457320+0000","last_unstale":"2024-04-25T10:05:11.460957+0000","last_undegraded":"2024-04-25T10:05:11.460957+0000","last_fullsized":"2024-04-25T10:05:11.460957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:40.831935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472125+0000","last_change":"2024-04-25T10:05:11.469113+0000","last_active":"2024-04-25T10:05:11.472125+0000","last_peered":"2024-04-25T10:05:11.472125+0000","last_clean":"2024-04-25T10:05:11.472125+0000","last_became_active":"2024-04-25T10:05:11.468759+0000","last_became_peered":"2024-04-25T10:05:11.468759+0000","last_unstale":"2024-04-25T10:05:11.472125+0000","last_undegraded":"2024-04-25T10:05:11.472125+0000","last_fullsized":"2024-04-25T10:05:11.472125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471666+0000","last_change":"2024-04-25T10:05:11.470055+0000","last_active":"2024-04-25T10:05:11.471666+0000","last_peered":"2024-04-25T10:05:11.471666+0000","last_clean":"2024-04-25T10:05:11.471666+0000","last_became_active":"2024-04-25T10:05:11.468974+0000","last_became_peered":"2024-04-25T10:05:11.468974+0000","last_unstale":"2024-04-25T10:05:11.471666+0000","last_undegraded":"2024-04-25T10:05:11.471666+0000","last_fullsized":"2024-04-25T10:05:11.471666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.228600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469311+0000","last_change":"2024-04-25T10:05:11.462234+0000","last_active":"2024-04-25T10:05:11.469311+0000","last_peered":"2024-04-25T10:05:11.469311+0000","last_clean":"2024-04-25T10:05:11.469311+0000","last_became_active":"2024-04-25T10:05:11.461975+0000","last_became_peered":"2024-04-25T10:05:11.461975+0000","last_unstale":"2024-04-25T10:05:11.469311+0000","last_undegraded":"2024-04-25T10:05:11.469311+0000","last_fullsized":"2024-04-25T10:05:11.469311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:02.561557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455396+0000","last_change":"2024-04-25T10:05:11.463406+0000","last_active":"2024-04-25T10:05:12.455396+0000","last_peered":"2024-04-25T10:05:12.455396+0000","last_clean":"2024-04-25T10:05:12.455396+0000","last_became_active":"2024-04-25T10:05:11.463139+0000","last_became_peered":"2024-04-25T10:05:11.463139+0000","last_unstale":"2024-04-25T10:05:12.455396+0000","last_undegraded":"2024-04-25T10:05:12.455396+0000","last_fullsized":"2024-04-25T10:05:12.455396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456384+0000","last_change":"2024-04-25T10:05:11.453168+0000","last_active":"2024-04-25T10:05:12.456384+0000","last_peered":"2024-04-25T10:05:12.456384+0000","last_clean":"2024-04-25T10:05:12.456384+0000","last_became_active":"2024-04-25T10:05:11.452238+0000","last_became_peered":"2024-04-25T10:05:11.452238+0000","last_unstale":"2024-04-25T10:05:12.456384+0000","last_undegraded":"2024-04-25T10:05:12.456384+0000","last_fullsized":"2024-04-25T10:05:12.456384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-04-25T10:05:09.441702+0000","last_change":"2024-04-25T10:05:09.441702+0000","last_active":"2024-04-25T10:05:09.429642+0000","last_peered":"2024-04-25T10:05:09.429642+0000","last_clean":"2024-04-25T10:05:09.429642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:09.441702+0000","last_undegraded":"2024-04-25T10:05:09.441702+0000","last_fullsized":"2024-04-25T10:05:09.441702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456962+0000","last_change":"2024-04-25T10:05:11.469760+0000","last_active":"2024-04-25T10:05:12.456962+0000","last_peered":"2024-04-25T10:05:12.456962+0000","last_clean":"2024-04-25T10:05:12.456962+0000","last_became_active":"2024-04-25T10:05:11.469490+0000","last_became_peered":"2024-04-25T10:05:11.469490+0000","last_unstale":"2024-04-25T10:05:12.456962+0000","last_undegraded":"2024-04-25T10:05:12.456962+0000","last_fullsized":"2024-04-25T10:05:12.456962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.228600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.475721+0000","last_change":"2024-04-25T10:05:10.461940+0000","last_active":"2024-04-25T10:05:11.475721+0000","last_peered":"2024-04-25T10:05:11.475721+0000","last_clean":"2024-04-25T10:05:11.475721+0000","last_became_active":"2024-04-25T10:05:10.461633+0000","last_became_peered":"2024-04-25T10:05:10.461633+0000","last_unstale":"2024-04-25T10:05:11.475721+0000","last_undegraded":"2024-04-25T10:05:11.475721+0000","last_fullsized":"2024-04-25T10:05:11.475721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.456539+0000","last_change":"2024-04-25T10:05:10.460823+0000","last_active":"2024-04-25T10:05:11.456539+0000","last_peered":"2024-04-25T10:05:11.456539+0000","last_clean":"2024-04-25T10:05:11.456539+0000","last_became_active":"2024-04-25T10:05:10.460480+0000","last_became_peered":"2024-04-25T10:05:10.460480+0000","last_unstale":"2024-04-25T10:05:11.456539+0000","last_undegraded":"2024-04-25T10:05:11.456539+0000","last_fullsized":"2024-04-25T10:05:11.456539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:31.375084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.450458+0000","last_change":"2024-04-25T10:05:10.461713+0000","last_active":"2024-04-25T10:05:11.450458+0000","last_peered":"2024-04-25T10:05:11.450458+0000","last_clean":"2024-04-25T10:05:11.450458+0000","last_became_active":"2024-04-25T10:05:10.461379+0000","last_became_peered":"2024-04-25T10:05:10.461379+0000","last_unstale":"2024-04-25T10:05:11.450458+0000","last_undegraded":"2024-04-25T10:05:11.450458+0000","last_fullsized":"2024-04-25T10:05:11.450458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456510+0000","last_change":"2024-04-25T10:05:11.466408+0000","last_active":"2024-04-25T10:05:12.456510+0000","last_peered":"2024-04-25T10:05:12.456510+0000","last_clean":"2024-04-25T10:05:12.456510+0000","last_became_active":"2024-04-25T10:05:11.465611+0000","last_became_peered":"2024-04-25T10:05:11.465611+0000","last_unstale":"2024-04-25T10:05:12.456510+0000","last_undegraded":"2024-04-25T10:05:12.456510+0000","last_fullsized":"2024-04-25T10:05:12.456510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456657+0000","last_change":"2024-04-25T10:05:10.449054+0000","last_active":"2024-04-25T10:05:12.456657+0000","last_peered":"2024-04-25T10:05:12.456657+0000","last_clean":"2024-04-25T10:05:12.456657+0000","last_became_active":"2024-04-25T10:05:10.448743+0000","last_became_peered":"2024-04-25T10:05:10.448743+0000","last_unstale":"2024-04-25T10:05:12.456657+0000","last_undegraded":"2024-04-25T10:05:12.456657+0000","last_fullsized":"2024-04-25T10:05:12.456657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:22.897294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455686+0000","last_change":"2024-04-25T10:05:11.453268+0000","last_active":"2024-04-25T10:05:12.455686+0000","last_peered":"2024-04-25T10:05:12.455686+0000","last_clean":"2024-04-25T10:05:12.455686+0000","last_became_active":"2024-04-25T10:05:11.452971+0000","last_became_peered":"2024-04-25T10:05:11.452971+0000","last_unstale":"2024-04-25T10:05:12.455686+0000","last_undegraded":"2024-04-25T10:05:12.455686+0000","last_fullsized":"2024-04-25T10:05:12.455686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:56.142030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.452585+0000","last_change":"2024-04-25T10:05:10.452247+0000","last_active":"2024-04-25T10:05:11.452585+0000","last_peered":"2024-04-25T10:05:11.452585+0000","last_clean":"2024-04-25T10:05:11.452585+0000","last_became_active":"2024-04-25T10:05:10.451925+0000","last_became_peered":"2024-04-25T10:05:10.451925+0000","last_unstale":"2024-04-25T10:05:11.452585+0000","last_undegraded":"2024-04-25T10:05:11.452585+0000","last_fullsized":"2024-04-25T10:05:11.452585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:53.640804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472671+0000","last_change":"2024-04-25T10:05:10.459898+0000","last_active":"2024-04-25T10:05:11.472671+0000","last_peered":"2024-04-25T10:05:11.472671+0000","last_clean":"2024-04-25T10:05:11.472671+0000","last_became_active":"2024-04-25T10:05:10.459250+0000","last_became_peered":"2024-04-25T10:05:10.459250+0000","last_unstale":"2024-04-25T10:05:11.472671+0000","last_undegraded":"2024-04-25T10:05:11.472671+0000","last_fullsized":"2024-04-25T10:05:11.472671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:41.255007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.473052+0000","last_change":"2024-04-25T10:05:11.471498+0000","last_active":"2024-04-25T10:05:11.473052+0000","last_peered":"2024-04-25T10:05:11.473052+0000","last_clean":"2024-04-25T10:05:11.473052+0000","last_became_active":"2024-04-25T10:05:11.471193+0000","last_became_peered":"2024-04-25T10:05:11.471193+0000","last_unstale":"2024-04-25T10:05:11.473052+0000","last_undegraded":"2024-04-25T10:05:11.473052+0000","last_fullsized":"2024-04-25T10:05:11.473052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:45.980553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455560+0000","last_change":"2024-04-25T10:05:10.450534+0000","last_active":"2024-04-25T10:05:12.455560+0000","last_peered":"2024-04-25T10:05:12.455560+0000","last_clean":"2024-04-25T10:05:12.455560+0000","last_became_active":"2024-04-25T10:05:10.450251+0000","last_became_peered":"2024-04-25T10:05:10.450251+0000","last_unstale":"2024-04-25T10:05:12.455560+0000","last_undegraded":"2024-04-25T10:05:12.455560+0000","last_fullsized":"2024-04-25T10:05:12.455560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:22.897294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.468770+0000","last_change":"2024-04-25T10:05:11.463480+0000","last_active":"2024-04-25T10:05:11.468770+0000","last_peered":"2024-04-25T10:05:11.468770+0000","last_clean":"2024-04-25T10:05:11.468770+0000","last_became_active":"2024-04-25T10:05:11.460995+0000","last_became_peered":"2024-04-25T10:05:11.460995+0000","last_unstale":"2024-04-25T10:05:11.468770+0000","last_undegraded":"2024-04-25T10:05:11.468770+0000","last_fullsized":"2024-04-25T10:05:11.468770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:56.142030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.464540+0000","last_change":"2024-04-25T10:05:10.462323+0000","last_active":"2024-04-25T10:05:11.464540+0000","last_peered":"2024-04-25T10:05:11.464540+0000","last_clean":"2024-04-25T10:05:11.464540+0000","last_became_active":"2024-04-25T10:05:10.462118+0000","last_became_peered":"2024-04-25T10:05:10.462118+0000","last_unstale":"2024-04-25T10:05:11.464540+0000","last_undegraded":"2024-04-25T10:05:11.464540+0000","last_fullsized":"2024-04-25T10:05:11.464540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:39.826605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471805+0000","last_change":"2024-04-25T10:05:11.467396+0000","last_active":"2024-04-25T10:05:11.471805+0000","last_peered":"2024-04-25T10:05:11.471805+0000","last_clean":"2024-04-25T10:05:11.471805+0000","last_became_active":"2024-04-25T10:05:11.467112+0000","last_became_peered":"2024-04-25T10:05:11.467112+0000","last_unstale":"2024-04-25T10:05:11.471805+0000","last_undegraded":"2024-04-25T10:05:11.471805+0000","last_fullsized":"2024-04-25T10:05:11.471805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.451714+0000","last_change":"2024-04-25T10:05:10.458429+0000","last_active":"2024-04-25T10:05:11.451714+0000","last_peered":"2024-04-25T10:05:11.451714+0000","last_clean":"2024-04-25T10:05:11.451714+0000","last_became_active":"2024-04-25T10:05:10.457958+0000","last_became_peered":"2024-04-25T10:05:10.457958+0000","last_unstale":"2024-04-25T10:05:11.451714+0000","last_undegraded":"2024-04-25T10:05:11.451714+0000","last_fullsized":"2024-04-25T10:05:11.451714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:23.585636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469949+0000","last_change":"2024-04-25T10:05:11.465006+0000","last_active":"2024-04-25T10:05:11.469949+0000","last_peered":"2024-04-25T10:05:11.469949+0000","last_clean":"2024-04-25T10:05:11.469949+0000","last_became_active":"2024-04-25T10:05:11.464719+0000","last_became_peered":"2024-04-25T10:05:11.464719+0000","last_unstale":"2024-04-25T10:05:11.469949+0000","last_undegraded":"2024-04-25T10:05:11.469949+0000","last_fullsized":"2024-04-25T10:05:11.469949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:54.934726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.459177+0000","last_change":"2024-04-25T10:05:10.456938+0000","last_active":"2024-04-25T10:05:11.459177+0000","last_peered":"2024-04-25T10:05:11.459177+0000","last_clean":"2024-04-25T10:05:11.459177+0000","last_became_active":"2024-04-25T10:05:10.456644+0000","last_became_peered":"2024-04-25T10:05:10.456644+0000","last_unstale":"2024-04-25T10:05:11.459177+0000","last_undegraded":"2024-04-25T10:05:11.459177+0000","last_fullsized":"2024-04-25T10:05:11.459177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456188+0000","last_change":"2024-04-25T10:05:11.456178+0000","last_active":"2024-04-25T10:05:12.456188+0000","last_peered":"2024-04-25T10:05:12.456188+0000","last_clean":"2024-04-25T10:05:12.456188+0000","last_became_active":"2024-04-25T10:05:11.455879+0000","last_became_peered":"2024-04-25T10:05:11.455879+0000","last_unstale":"2024-04-25T10:05:12.456188+0000","last_undegraded":"2024-04-25T10:05:12.456188+0000","last_fullsized":"2024-04-25T10:05:12.456188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:55.300183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456275+0000","last_change":"2024-04-25T10:05:10.455684+0000","last_active":"2024-04-25T10:05:12.456275+0000","last_peered":"2024-04-25T10:05:12.456275+0000","last_clean":"2024-04-25T10:05:12.456275+0000","last_became_active":"2024-04-25T10:05:10.455342+0000","last_became_peered":"2024-04-25T10:05:10.455342+0000","last_unstale":"2024-04-25T10:05:12.456275+0000","last_undegraded":"2024-04-25T10:05:12.456275+0000","last_fullsized":"2024-04-25T10:05:12.456275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:01.556228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469903+0000","last_change":"2024-04-25T10:05:11.466622+0000","last_active":"2024-04-25T10:05:11.469903+0000","last_peered":"2024-04-25T10:05:11.469903+0000","last_clean":"2024-04-25T10:05:11.469903+0000","last_became_active":"2024-04-25T10:05:11.466346+0000","last_became_peered":"2024-04-25T10:05:11.466346+0000","last_unstale":"2024-04-25T10:05:11.469903+0000","last_undegraded":"2024-04-25T10:05:11.469903+0000","last_fullsized":"2024-04-25T10:05:11.469903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.468144+0000","last_change":"2024-04-25T10:05:10.457614+0000","last_active":"2024-04-25T10:05:11.468144+0000","last_peered":"2024-04-25T10:05:11.468144+0000","last_clean":"2024-04-25T10:05:11.468144+0000","last_became_active":"2024-04-25T10:05:10.457332+0000","last_became_peered":"2024-04-25T10:05:10.457332+0000","last_unstale":"2024-04-25T10:05:11.468144+0000","last_undegraded":"2024-04-25T10:05:11.468144+0000","last_fullsized":"2024-04-25T10:05:11.468144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:01.556228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455780+0000","last_change":"2024-04-25T10:05:11.460589+0000","last_active":"2024-04-25T10:05:12.455780+0000","last_peered":"2024-04-25T10:05:12.455780+0000","last_clean":"2024-04-25T10:05:12.455780+0000","last_became_active":"2024-04-25T10:05:11.459433+0000","last_became_peered":"2024-04-25T10:05:11.459433+0000","last_unstale":"2024-04-25T10:05:12.455780+0000","last_undegraded":"2024-04-25T10:05:12.455780+0000","last_fullsized":"2024-04-25T10:05:12.455780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:55.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456910+0000","last_change":"2024-04-25T10:05:10.458692+0000","last_active":"2024-04-25T10:05:12.456910+0000","last_peered":"2024-04-25T10:05:12.456910+0000","last_clean":"2024-04-25T10:05:12.456910+0000","last_became_active":"2024-04-25T10:05:10.458090+0000","last_became_peered":"2024-04-25T10:05:10.458090+0000","last_unstale":"2024-04-25T10:05:12.456910+0000","last_undegraded":"2024-04-25T10:05:12.456910+0000","last_fullsized":"2024-04-25T10:05:12.456910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:23.585636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.468810+0000","last_change":"2024-04-25T10:05:11.464532+0000","last_active":"2024-04-25T10:05:11.468810+0000","last_peered":"2024-04-25T10:05:11.468810+0000","last_clean":"2024-04-25T10:05:11.468810+0000","last_became_active":"2024-04-25T10:05:11.464186+0000","last_became_peered":"2024-04-25T10:05:11.464186+0000","last_unstale":"2024-04-25T10:05:11.468810+0000","last_undegraded":"2024-04-25T10:05:11.468810+0000","last_fullsized":"2024-04-25T10:05:11.468810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:55.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-04-25T10:05:09.441695+0000","last_change":"2024-04-25T10:05:09.441695+0000","last_active":"2024-04-25T10:05:09.429642+0000","last_peered":"2024-04-25T10:05:09.429642+0000","last_clean":"2024-04-25T10:05:09.429642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:09.441695+0000","last_undegraded":"2024-04-25T10:05:09.441695+0000","last_fullsized":"2024-04-25T10:05:09.441695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.475897+0000","last_change":"2024-04-25T10:05:11.474519+0000","last_active":"2024-04-25T10:05:11.475897+0000","last_peered":"2024-04-25T10:05:11.475897+0000","last_clean":"2024-04-25T10:05:11.475897+0000","last_became_active":"2024-04-25T10:05:11.472675+0000","last_became_peered":"2024-04-25T10:05:11.472675+0000","last_unstale":"2024-04-25T10:05:11.475897+0000","last_undegraded":"2024-04-25T10:05:11.475897+0000","last_fullsized":"2024-04-25T10:05:11.475897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.799101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.458209+0000","last_change":"2024-04-25T10:05:10.453364+0000","last_active":"2024-04-25T10:05:11.458209+0000","last_peered":"2024-04-25T10:05:11.458209+0000","last_clean":"2024-04-25T10:05:11.458209+0000","last_became_active":"2024-04-25T10:05:10.453072+0000","last_became_peered":"2024-04-25T10:05:10.453072+0000","last_unstale":"2024-04-25T10:05:11.458209+0000","last_undegraded":"2024-04-25T10:05:11.458209+0000","last_fullsized":"2024-04-25T10:05:11.458209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:53.640804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469818+0000","last_change":"2024-04-25T10:05:11.467814+0000","last_active":"2024-04-25T10:05:11.469818+0000","last_peered":"2024-04-25T10:05:11.469818+0000","last_clean":"2024-04-25T10:05:11.469818+0000","last_became_active":"2024-04-25T10:05:11.467496+0000","last_became_peered":"2024-04-25T10:05:11.467496+0000","last_unstale":"2024-04-25T10:05:11.469818+0000","last_undegraded":"2024-04-25T10:05:11.469818+0000","last_fullsized":"2024-04-25T10:05:11.469818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.799101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456108+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:05:12.456108+0000","last_peered":"2024-04-25T10:05:12.456108+0000","last_clean":"2024-04-25T10:05:12.456108+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:05:12.456108+0000","last_undegraded":"2024-04-25T10:05:12.456108+0000","last_fullsized":"2024-04-25T10:05:12.456108+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.452172+0000","last_change":"2024-04-25T10:05:10.458170+0000","last_active":"2024-04-25T10:05:11.452172+0000","last_peered":"2024-04-25T10:05:11.452172+0000","last_clean":"2024-04-25T10:05:11.452172+0000","last_became_active":"2024-04-25T10:05:10.457794+0000","last_became_peered":"2024-04-25T10:05:10.457794+0000","last_unstale":"2024-04-25T10:05:11.452172+0000","last_undegraded":"2024-04-25T10:05:11.452172+0000","last_fullsized":"2024-04-25T10:05:11.452172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:11.920992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456600+0000","last_change":"2024-04-25T10:05:11.462574+0000","last_active":"2024-04-25T10:05:12.456600+0000","last_peered":"2024-04-25T10:05:12.456600+0000","last_clean":"2024-04-25T10:05:12.456600+0000","last_became_active":"2024-04-25T10:05:11.461617+0000","last_became_peered":"2024-04-25T10:05:11.461617+0000","last_unstale":"2024-04-25T10:05:12.456600+0000","last_undegraded":"2024-04-25T10:05:12.456600+0000","last_fullsized":"2024-04-25T10:05:12.456600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.451455+0000","last_change":"2024-04-25T10:05:10.464334+0000","last_active":"2024-04-25T10:05:11.451455+0000","last_peered":"2024-04-25T10:05:11.451455+0000","last_clean":"2024-04-25T10:05:11.451455+0000","last_became_active":"2024-04-25T10:05:10.463991+0000","last_became_peered":"2024-04-25T10:05:10.463991+0000","last_unstale":"2024-04-25T10:05:11.451455+0000","last_undegraded":"2024-04-25T10:05:11.451455+0000","last_fullsized":"2024-04-25T10:05:11.451455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469159+0000","last_change":"2024-04-25T10:05:11.464866+0000","last_active":"2024-04-25T10:05:11.469159+0000","last_peered":"2024-04-25T10:05:11.469159+0000","last_clean":"2024-04-25T10:05:11.469159+0000","last_became_active":"2024-04-25T10:05:11.464504+0000","last_became_peered":"2024-04-25T10:05:11.464504+0000","last_unstale":"2024-04-25T10:05:11.469159+0000","last_undegraded":"2024-04-25T10:05:11.469159+0000","last_fullsized":"2024-04-25T10:05:11.469159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.449849+0000","last_change":"2024-04-25T10:05:10.458220+0000","last_active":"2024-04-25T10:05:11.449849+0000","last_peered":"2024-04-25T10:05:11.449849+0000","last_clean":"2024-04-25T10:05:11.449849+0000","last_became_active":"2024-04-25T10:05:10.457936+0000","last_became_peered":"2024-04-25T10:05:10.457936+0000","last_unstale":"2024-04-25T10:05:11.449849+0000","last_undegraded":"2024-04-25T10:05:11.449849+0000","last_fullsized":"2024-04-25T10:05:11.449849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:31.375084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.451883+0000","last_change":"2024-04-25T10:05:10.463388+0000","last_active":"2024-04-25T10:05:11.451883+0000","last_peered":"2024-04-25T10:05:11.451883+0000","last_clean":"2024-04-25T10:05:11.451883+0000","last_became_active":"2024-04-25T10:05:10.463041+0000","last_became_peered":"2024-04-25T10:05:10.463041+0000","last_unstale":"2024-04-25T10:05:11.451883+0000","last_undegraded":"2024-04-25T10:05:11.451883+0000","last_fullsized":"2024-04-25T10:05:11.451883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:13.500977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.463844+0000","last_change":"2024-04-25T10:05:11.462360+0000","last_active":"2024-04-25T10:05:11.463844+0000","last_peered":"2024-04-25T10:05:11.463844+0000","last_clean":"2024-04-25T10:05:11.463844+0000","last_became_active":"2024-04-25T10:05:11.462050+0000","last_became_peered":"2024-04-25T10:05:11.462050+0000","last_unstale":"2024-04-25T10:05:11.463844+0000","last_undegraded":"2024-04-25T10:05:11.463844+0000","last_fullsized":"2024-04-25T10:05:11.463844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:14.506306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-04-25T10:05:09.441740+0000","last_change":"2024-04-25T10:05:09.441740+0000","last_active":"2024-04-25T10:05:09.429642+0000","last_peered":"2024-04-25T10:05:09.429642+0000","last_clean":"2024-04-25T10:05:09.429642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:09.441740+0000","last_undegraded":"2024-04-25T10:05:09.441740+0000","last_fullsized":"2024-04-25T10:05:09.441740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455433+0000","last_change":"2024-04-25T10:05:11.464184+0000","last_active":"2024-04-25T10:05:12.455433+0000","last_peered":"2024-04-25T10:05:12.455433+0000","last_clean":"2024-04-25T10:05:12.455433+0000","last_became_active":"2024-04-25T10:05:11.463894+0000","last_became_peered":"2024-04-25T10:05:11.463894+0000","last_unstale":"2024-04-25T10:05:12.455433+0000","last_undegraded":"2024-04-25T10:05:12.455433+0000","last_fullsized":"2024-04-25T10:05:12.455433+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.452412+0000","last_change":"2024-04-25T10:05:10.460261+0000","last_active":"2024-04-25T10:05:11.452412+0000","last_peered":"2024-04-25T10:05:11.452412+0000","last_clean":"2024-04-25T10:05:11.452412+0000","last_became_active":"2024-04-25T10:05:10.459949+0000","last_became_peered":"2024-04-25T10:05:10.459949+0000","last_unstale":"2024-04-25T10:05:11.452412+0000","last_undegraded":"2024-04-25T10:05:11.452412+0000","last_fullsized":"2024-04-25T10:05:11.452412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:29.784640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.463941+0000","last_change":"2024-04-25T10:05:11.458748+0000","last_active":"2024-04-25T10:05:11.463941+0000","last_peered":"2024-04-25T10:05:11.463941+0000","last_clean":"2024-04-25T10:05:11.463941+0000","last_became_active":"2024-04-25T10:05:11.458440+0000","last_became_peered":"2024-04-25T10:05:11.458440+0000","last_unstale":"2024-04-25T10:05:11.463941+0000","last_undegraded":"2024-04-25T10:05:11.463941+0000","last_fullsized":"2024-04-25T10:05:11.463941+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:55.300183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456842+0000","last_change":"2024-04-25T10:05:10.455250+0000","last_active":"2024-04-25T10:05:12.456842+0000","last_peered":"2024-04-25T10:05:12.456842+0000","last_clean":"2024-04-25T10:05:12.456842+0000","last_became_active":"2024-04-25T10:05:10.454833+0000","last_became_peered":"2024-04-25T10:05:10.454833+0000","last_unstale":"2024-04-25T10:05:12.456842+0000","last_undegraded":"2024-04-25T10:05:12.456842+0000","last_fullsized":"2024-04-25T10:05:12.456842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.457011+0000","last_change":"2024-04-25T10:05:11.466533+0000","last_active":"2024-04-25T10:05:12.457011+0000","last_peered":"2024-04-25T10:05:12.457011+0000","last_clean":"2024-04-25T10:05:12.457011+0000","last_became_active":"2024-04-25T10:05:11.466256+0000","last_became_peered":"2024-04-25T10:05:11.466256+0000","last_unstale":"2024-04-25T10:05:12.457011+0000","last_undegraded":"2024-04-25T10:05:12.457011+0000","last_fullsized":"2024-04-25T10:05:12.457011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:41.766168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.476424+0000","last_change":"2024-04-25T10:05:10.462808+0000","last_active":"2024-04-25T10:05:11.476424+0000","last_peered":"2024-04-25T10:05:11.476424+0000","last_clean":"2024-04-25T10:05:11.476424+0000","last_became_active":"2024-04-25T10:05:10.462421+0000","last_became_peered":"2024-04-25T10:05:10.462421+0000","last_unstale":"2024-04-25T10:05:11.476424+0000","last_undegraded":"2024-04-25T10:05:11.476424+0000","last_fullsized":"2024-04-25T10:05:11.476424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:55.136701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456680+0000","last_change":"2024-04-25T10:05:11.470381+0000","last_active":"2024-04-25T10:05:12.456680+0000","last_peered":"2024-04-25T10:05:12.456680+0000","last_clean":"2024-04-25T10:05:12.456680+0000","last_became_active":"2024-04-25T10:05:11.470031+0000","last_became_peered":"2024-04-25T10:05:11.470031+0000","last_unstale":"2024-04-25T10:05:12.456680+0000","last_undegraded":"2024-04-25T10:05:12.456680+0000","last_fullsized":"2024-04-25T10:05:12.456680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.799101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.454513+0000","last_change":"2024-04-25T10:05:10.461232+0000","last_active":"2024-04-25T10:05:11.454513+0000","last_peered":"2024-04-25T10:05:11.454513+0000","last_clean":"2024-04-25T10:05:11.454513+0000","last_became_active":"2024-04-25T10:05:10.460905+0000","last_became_peered":"2024-04-25T10:05:10.460905+0000","last_unstale":"2024-04-25T10:05:11.454513+0000","last_undegraded":"2024-04-25T10:05:11.454513+0000","last_fullsized":"2024-04-25T10:05:11.454513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:54.294853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474543+0000","last_change":"2024-04-25T10:05:11.472902+0000","last_active":"2024-04-25T10:05:11.474543+0000","last_peered":"2024-04-25T10:05:11.474543+0000","last_clean":"2024-04-25T10:05:11.474543+0000","last_became_active":"2024-04-25T10:05:11.472622+0000","last_became_peered":"2024-04-25T10:05:11.472622+0000","last_unstale":"2024-04-25T10:05:11.474543+0000","last_undegraded":"2024-04-25T10:05:11.474543+0000","last_fullsized":"2024-04-25T10:05:11.474543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:24.616832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456461+0000","last_change":"2024-04-25T10:05:10.460759+0000","last_active":"2024-04-25T10:05:12.456461+0000","last_peered":"2024-04-25T10:05:12.456461+0000","last_clean":"2024-04-25T10:05:12.456461+0000","last_became_active":"2024-04-25T10:05:10.460485+0000","last_became_peered":"2024-04-25T10:05:10.460485+0000","last_unstale":"2024-04-25T10:05:12.456461+0000","last_undegraded":"2024-04-25T10:05:12.456461+0000","last_fullsized":"2024-04-25T10:05:12.456461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456829+0000","last_change":"2024-04-25T10:05:11.470739+0000","last_active":"2024-04-25T10:05:12.456829+0000","last_peered":"2024-04-25T10:05:12.456829+0000","last_clean":"2024-04-25T10:05:12.456829+0000","last_became_active":"2024-04-25T10:05:11.470408+0000","last_became_peered":"2024-04-25T10:05:11.470408+0000","last_unstale":"2024-04-25T10:05:12.456829+0000","last_undegraded":"2024-04-25T10:05:12.456829+0000","last_fullsized":"2024-04-25T10:05:12.456829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:17.789806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.457380+0000","last_change":"2024-04-25T10:05:10.458496+0000","last_active":"2024-04-25T10:05:11.457380+0000","last_peered":"2024-04-25T10:05:11.457380+0000","last_clean":"2024-04-25T10:05:11.457380+0000","last_became_active":"2024-04-25T10:05:10.458168+0000","last_became_peered":"2024-04-25T10:05:10.458168+0000","last_unstale":"2024-04-25T10:05:11.457380+0000","last_undegraded":"2024-04-25T10:05:11.457380+0000","last_fullsized":"2024-04-25T10:05:11.457380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:41.255007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.461196+0000","last_change":"2024-04-25T10:05:11.459826+0000","last_active":"2024-04-25T10:05:11.461196+0000","last_peered":"2024-04-25T10:05:11.461196+0000","last_clean":"2024-04-25T10:05:11.461196+0000","last_became_active":"2024-04-25T10:05:11.459570+0000","last_became_peered":"2024-04-25T10:05:11.459570+0000","last_unstale":"2024-04-25T10:05:11.461196+0000","last_undegraded":"2024-04-25T10:05:11.461196+0000","last_fullsized":"2024-04-25T10:05:11.461196+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:54.646134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474457+0000","last_change":"2024-04-25T10:05:10.455137+0000","last_active":"2024-04-25T10:05:11.474457+0000","last_peered":"2024-04-25T10:05:11.474457+0000","last_clean":"2024-04-25T10:05:11.474457+0000","last_became_active":"2024-04-25T10:05:10.454827+0000","last_became_peered":"2024-04-25T10:05:10.454827+0000","last_unstale":"2024-04-25T10:05:11.474457+0000","last_undegraded":"2024-04-25T10:05:11.474457+0000","last_fullsized":"2024-04-25T10:05:11.474457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:54.691904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.470977+0000","last_change":"2024-04-25T10:05:11.469441+0000","last_active":"2024-04-25T10:05:11.470977+0000","last_peered":"2024-04-25T10:05:11.470977+0000","last_clean":"2024-04-25T10:05:11.470977+0000","last_became_active":"2024-04-25T10:05:11.468649+0000","last_became_peered":"2024-04-25T10:05:11.468649+0000","last_unstale":"2024-04-25T10:05:11.470977+0000","last_undegraded":"2024-04-25T10:05:11.470977+0000","last_fullsized":"2024-04-25T10:05:11.470977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:17.789806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469799+0000","last_change":"2024-04-25T10:05:11.465690+0000","last_active":"2024-04-25T10:05:11.469799+0000","last_peered":"2024-04-25T10:05:11.469799+0000","last_clean":"2024-04-25T10:05:11.469799+0000","last_became_active":"2024-04-25T10:05:11.465422+0000","last_became_peered":"2024-04-25T10:05:11.465422+0000","last_unstale":"2024-04-25T10:05:11.469799+0000","last_undegraded":"2024-04-25T10:05:11.469799+0000","last_fullsized":"2024-04-25T10:05:11.469799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472427+0000","last_change":"2024-04-25T10:05:10.450133+0000","last_active":"2024-04-25T10:05:11.472427+0000","last_peered":"2024-04-25T10:05:11.472427+0000","last_clean":"2024-04-25T10:05:11.472427+0000","last_became_active":"2024-04-25T10:05:10.449482+0000","last_became_peered":"2024-04-25T10:05:10.449482+0000","last_unstale":"2024-04-25T10:05:11.472427+0000","last_undegraded":"2024-04-25T10:05:11.472427+0000","last_fullsized":"2024-04-25T10:05:11.472427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456402+0000","last_change":"2024-04-25T10:05:11.469301+0000","last_active":"2024-04-25T10:05:12.456402+0000","last_peered":"2024-04-25T10:05:12.456402+0000","last_clean":"2024-04-25T10:05:12.456402+0000","last_became_active":"2024-04-25T10:05:11.468980+0000","last_became_peered":"2024-04-25T10:05:11.468980+0000","last_unstale":"2024-04-25T10:05:12.456402+0000","last_undegraded":"2024-04-25T10:05:12.456402+0000","last_fullsized":"2024-04-25T10:05:12.456402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471641+0000","last_change":"2024-04-25T10:05:10.456749+0000","last_active":"2024-04-25T10:05:11.471641+0000","last_peered":"2024-04-25T10:05:11.471641+0000","last_clean":"2024-04-25T10:05:11.471641+0000","last_became_active":"2024-04-25T10:05:10.456397+0000","last_became_peered":"2024-04-25T10:05:10.456397+0000","last_unstale":"2024-04-25T10:05:11.471641+0000","last_undegraded":"2024-04-25T10:05:11.471641+0000","last_fullsized":"2024-04-25T10:05:11.471641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-04-25T10:05:09.441671+0000","last_change":"2024-04-25T10:05:09.441671+0000","last_active":"2024-04-25T10:05:09.429642+0000","last_peered":"2024-04-25T10:05:09.429642+0000","last_clean":"2024-04-25T10:05:09.429642+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:09.441671+0000","last_undegraded":"2024-04-25T10:05:09.441671+0000","last_fullsized":"2024-04-25T10:05:09.441671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:10.434972+0000","last_change":"2024-04-25T10:05:10.434972+0000","last_active":"2024-04-25T10:05:10.434972+0000","last_peered":"2024-04-25T10:05:10.434972+0000","last_clean":"2024-04-25T10:05:10.434972+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:10.434972+0000","last_undegraded":"2024-04-25T10:05:10.434972+0000","last_fullsized":"2024-04-25T10:05:10.434972+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.457002+0000","last_change":"2024-04-25T10:05:10.454280+0000","last_active":"2024-04-25T10:05:12.457002+0000","last_peered":"2024-04-25T10:05:12.457002+0000","last_clean":"2024-04-25T10:05:12.457002+0000","last_became_active":"2024-04-25T10:05:10.453982+0000","last_became_peered":"2024-04-25T10:05:10.453982+0000","last_unstale":"2024-04-25T10:05:12.457002+0000","last_undegraded":"2024-04-25T10:05:12.457002+0000","last_fullsized":"2024-04-25T10:05:12.457002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":104,"acting":104,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":72548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":70846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":71876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":667002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":70320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":765952,"data_stored":641765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:15.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2024-04-25T10:05:15.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:10.434972+0000', 'last_change': '2024-04-25T10:05:10.434972+0000', 'last_active': '2024-04-25T10:05:10.434972+0000', 'last_peered': '2024-04-25T10:05:10.434972+0000', 'last_clean': '2024-04-25T10:05:10.434972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:10.434972+0000', 'last_undegraded': '2024-04-25T10:05:10.434972+0000', 'last_fullsized': '2024-04-25T10:05:10.434972+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:15.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2024-04-25T10:05:15.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:10.434972+0000', 'last_change': '2024-04-25T10:05:10.434972+0000', 'last_active': '2024-04-25T10:05:10.434972+0000', 'last_peered': '2024-04-25T10:05:10.434972+0000', 'last_clean': '2024-04-25T10:05:10.434972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:10.434972+0000', 'last_undegraded': '2024-04-25T10:05:10.434972+0000', 'last_fullsized': '2024-04-25T10:05:10.434972+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:15.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2024-04-25T10:05:15.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:10.434972+0000', 'last_change': '2024-04-25T10:05:10.434972+0000', 'last_active': '2024-04-25T10:05:10.434972+0000', 'last_peered': '2024-04-25T10:05:10.434972+0000', 'last_clean': '2024-04-25T10:05:10.434972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:10.434972+0000', 'last_undegraded': '2024-04-25T10:05:10.434972+0000', 'last_fullsized': '2024-04-25T10:05:10.434972+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:15.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2024-04-25T10:05:15.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:10.434972+0000', 'last_change': '2024-04-25T10:05:10.434972+0000', 'last_active': '2024-04-25T10:05:10.434972+0000', 'last_peered': '2024-04-25T10:05:10.434972+0000', 'last_clean': '2024-04-25T10:05:10.434972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:10.434972+0000', 'last_undegraded': '2024-04-25T10:05:10.434972+0000', 'last_fullsized': '2024-04-25T10:05:10.434972+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:15.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2024-04-25T10:05:15.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:10.434972+0000', 'last_change': '2024-04-25T10:05:10.434972+0000', 'last_active': '2024-04-25T10:05:10.434972+0000', 'last_peered': '2024-04-25T10:05:10.434972+0000', 'last_clean': '2024-04-25T10:05:10.434972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:10.434972+0000', 'last_undegraded': '2024-04-25T10:05:10.434972+0000', 'last_fullsized': '2024-04-25T10:05:10.434972+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:15.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2024-04-25T10:05:15.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:10.434972+0000', 'last_change': '2024-04-25T10:05:10.434972+0000', 'last_active': '2024-04-25T10:05:10.434972+0000', 'last_peered': '2024-04-25T10:05:10.434972+0000', 'last_clean': '2024-04-25T10:05:10.434972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:10.434972+0000', 'last_undegraded': '2024-04-25T10:05:10.434972+0000', 'last_fullsized': '2024-04-25T10:05:10.434972+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:15.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2024-04-25T10:05:15.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:10.434972+0000', 'last_change': '2024-04-25T10:05:10.434972+0000', 'last_active': '2024-04-25T10:05:10.434972+0000', 'last_peered': '2024-04-25T10:05:10.434972+0000', 'last_clean': '2024-04-25T10:05:10.434972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:10.434972+0000', 'last_undegraded': '2024-04-25T10:05:10.434972+0000', 'last_fullsized': '2024-04-25T10:05:10.434972+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:15.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1b is not active+clean 2024-04-25T10:05:15.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:09.441702+0000', 'last_change': '2024-04-25T10:05:09.441702+0000', 'last_active': '2024-04-25T10:05:09.429642+0000', 'last_peered': '2024-04-25T10:05:09.429642+0000', 'last_clean': '2024-04-25T10:05:09.429642+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:09.441702+0000', 'last_undegraded': '2024-04-25T10:05:09.441702+0000', 'last_fullsized': '2024-04-25T10:05:09.441702+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:09.429642+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:09.429642+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:09.429642+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:05:15.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2024-04-25T10:05:15.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:10.434972+0000', 'last_change': '2024-04-25T10:05:10.434972+0000', 'last_active': '2024-04-25T10:05:10.434972+0000', 'last_peered': '2024-04-25T10:05:10.434972+0000', 'last_clean': '2024-04-25T10:05:10.434972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:10.434972+0000', 'last_undegraded': '2024-04-25T10:05:10.434972+0000', 'last_fullsized': '2024-04-25T10:05:10.434972+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:15.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2024-04-25T10:05:15.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:10.434972+0000', 'last_change': '2024-04-25T10:05:10.434972+0000', 'last_active': '2024-04-25T10:05:10.434972+0000', 'last_peered': '2024-04-25T10:05:10.434972+0000', 'last_clean': '2024-04-25T10:05:10.434972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:10.434972+0000', 'last_undegraded': '2024-04-25T10:05:10.434972+0000', 'last_fullsized': '2024-04-25T10:05:10.434972+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:15.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2024-04-25T10:05:15.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:09.441695+0000', 'last_change': '2024-04-25T10:05:09.441695+0000', 'last_active': '2024-04-25T10:05:09.429642+0000', 'last_peered': '2024-04-25T10:05:09.429642+0000', 'last_clean': '2024-04-25T10:05:09.429642+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:09.441695+0000', 'last_undegraded': '2024-04-25T10:05:09.441695+0000', 'last_fullsized': '2024-04-25T10:05:09.441695+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:09.429642+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:09.429642+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:09.429642+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:05:15.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2024-04-25T10:05:15.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:10.434972+0000', 'last_change': '2024-04-25T10:05:10.434972+0000', 'last_active': '2024-04-25T10:05:10.434972+0000', 'last_peered': '2024-04-25T10:05:10.434972+0000', 'last_clean': '2024-04-25T10:05:10.434972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:10.434972+0000', 'last_undegraded': '2024-04-25T10:05:10.434972+0000', 'last_fullsized': '2024-04-25T10:05:10.434972+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:15.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2024-04-25T10:05:15.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:09.441740+0000', 'last_change': '2024-04-25T10:05:09.441740+0000', 'last_active': '2024-04-25T10:05:09.429642+0000', 'last_peered': '2024-04-25T10:05:09.429642+0000', 'last_clean': '2024-04-25T10:05:09.429642+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:09.441740+0000', 'last_undegraded': '2024-04-25T10:05:09.441740+0000', 'last_fullsized': '2024-04-25T10:05:09.441740+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:09.429642+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:09.429642+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:09.429642+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:05:15.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2024-04-25T10:05:15.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:10.434972+0000', 'last_change': '2024-04-25T10:05:10.434972+0000', 'last_active': '2024-04-25T10:05:10.434972+0000', 'last_peered': '2024-04-25T10:05:10.434972+0000', 'last_clean': '2024-04-25T10:05:10.434972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:10.434972+0000', 'last_undegraded': '2024-04-25T10:05:10.434972+0000', 'last_fullsized': '2024-04-25T10:05:10.434972+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:15.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1e is not active+clean 2024-04-25T10:05:15.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:09.441671+0000', 'last_change': '2024-04-25T10:05:09.441671+0000', 'last_active': '2024-04-25T10:05:09.429642+0000', 'last_peered': '2024-04-25T10:05:09.429642+0000', 'last_clean': '2024-04-25T10:05:09.429642+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:09.441671+0000', 'last_undegraded': '2024-04-25T10:05:09.441671+0000', 'last_fullsized': '2024-04-25T10:05:09.441671+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:09.429642+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:09.429642+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:09.429642+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:05:15.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2024-04-25T10:05:15.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:10.434972+0000', 'last_change': '2024-04-25T10:05:10.434972+0000', 'last_active': '2024-04-25T10:05:10.434972+0000', 'last_peered': '2024-04-25T10:05:10.434972+0000', 'last_clean': '2024-04-25T10:05:10.434972+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:10.434972+0000', 'last_undegraded': '2024-04-25T10:05:10.434972+0000', 'last_fullsized': '2024-04-25T10:05:10.434972+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:10.434972+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:15.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:05:15.236 INFO:tasks.ceph.mds.b.smithi186.stderr:2024-04-25T10:05:15.231+0000 7f8ddbd57640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:05:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.279+0000 7faa49113640 1 Processor -- start 2024-04-25T10:05:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.279+0000 7faa49113640 1 -- start start 2024-04-25T10:05:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.280+0000 7faa49113640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7faa4415dce0 0x7faa440d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.280+0000 7faa49113640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440d35b0 0x7faa440b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.280+0000 7faa49113640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa440d1f90 0x7faa440b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.280+0000 7faa49113640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa4415e0c0 con 0x7faa440d1f90 2024-04-25T10:05:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.280+0000 7faa49113640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa440d2510 con 0x7faa4415dce0 2024-04-25T10:05:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.280+0000 7faa49113640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa440d27f0 con 0x7faa440d35b0 2024-04-25T10:05:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.280+0000 7faa4359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa440d1f90 0x7faa440b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.280+0000 7faa4359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa440d1f90 0x7faa440b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41818/0 (socket says 172.21.15.16:41818) 2024-04-25T10:05:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.280+0000 7faa4359e640 1 -- 172.21.15.16:0/4062535982 learned_addr learned my addr 172.21.15.16:0/4062535982 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.281+0000 7faa3bfff640 1 --2- 172.21.15.16:0/4062535982 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440d35b0 0x7faa440b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.281+0000 7faa42d9d640 1 --2- 172.21.15.16:0/4062535982 >> v2:172.21.15.186:3300/0 conn(0x7faa4415dce0 0x7faa440d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.281+0000 7faa4359e640 1 -- 172.21.15.16:0/4062535982 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440d35b0 msgr2=0x7faa440b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.281+0000 7faa4359e640 1 --2- 172.21.15.16:0/4062535982 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440d35b0 0x7faa440b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.281+0000 7faa4359e640 1 -- 172.21.15.16:0/4062535982 >> v2:172.21.15.186:3300/0 conn(0x7faa4415dce0 msgr2=0x7faa440d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.281+0000 7faa4359e640 1 --2- 172.21.15.16:0/4062535982 >> v2:172.21.15.186:3300/0 conn(0x7faa4415dce0 0x7faa440d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.281+0000 7faa4359e640 1 -- 172.21.15.16:0/4062535982 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa440d2ad0 con 0x7faa440d1f90 2024-04-25T10:05:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.281+0000 7faa4359e640 1 --2- 172.21.15.16:0/4062535982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa440d1f90 0x7faa440b5990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faa440ce220 tx=0x7faa3c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6351f3112513a409 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.282+0000 7faa4259c640 1 -- 172.21.15.16:0/4062535982 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c036070 con 0x7faa440d1f90 2024-04-25T10:05:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.282+0000 7faa4259c640 1 -- 172.21.15.16:0/4062535982 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3c031600 con 0x7faa440d1f90 2024-04-25T10:05:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.282+0000 7faa4259c640 1 -- 172.21.15.16:0/4062535982 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c0319d0 con 0x7faa440d1f90 2024-04-25T10:05:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.282+0000 7faa49113640 1 -- 172.21.15.16:0/4062535982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa440d1f90 msgr2=0x7faa440b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.282+0000 7faa49113640 1 --2- 172.21.15.16:0/4062535982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa440d1f90 0x7faa440b5990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faa440ce220 tx=0x7faa3c02a610 comp rx=0 tx=0).stop 2024-04-25T10:05:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.282+0000 7faa49113640 1 -- 172.21.15.16:0/4062535982 shutdown_connections 2024-04-25T10:05:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.282+0000 7faa49113640 1 --2- 172.21.15.16:0/4062535982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa440d1f90 0x7faa440b5990 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.282+0000 7faa49113640 1 --2- 172.21.15.16:0/4062535982 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440d35b0 0x7faa440b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.282+0000 7faa49113640 1 --2- 172.21.15.16:0/4062535982 >> v2:172.21.15.186:3300/0 conn(0x7faa4415dce0 0x7faa440d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.282+0000 7faa49113640 1 -- 172.21.15.16:0/4062535982 >> 172.21.15.16:0/4062535982 conn(0x7faa44072900 msgr2=0x7faa440d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.282+0000 7faa49113640 1 -- 172.21.15.16:0/4062535982 shutdown_connections 2024-04-25T10:05:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.282+0000 7faa49113640 1 -- 172.21.15.16:0/4062535982 wait complete. 2024-04-25T10:05:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.283+0000 7faa49113640 1 Processor -- start 2024-04-25T10:05:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.284+0000 7faa49113640 1 -- start start 2024-04-25T10:05:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.284+0000 7faa49113640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440d1f90 0x7faa44159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.284+0000 7faa49113640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa440d35b0 0x7faa440c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.284+0000 7faa49113640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa4415dce0 0x7faa440ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.284+0000 7faa49113640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa440b9820 con 0x7faa440d35b0 2024-04-25T10:05:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.284+0000 7faa49113640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa440d3090 con 0x7faa4415dce0 2024-04-25T10:05:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.284+0000 7faa49113640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa440b99a0 con 0x7faa440d1f90 2024-04-25T10:05:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.284+0000 7faa3bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa440d35b0 0x7faa440c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.284+0000 7faa3bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa440d35b0 0x7faa440c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41830/0 (socket says 172.21.15.16:41830) 2024-04-25T10:05:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.284+0000 7faa3bfff640 1 -- 172.21.15.16:0/588294287 learned_addr learned my addr 172.21.15.16:0/588294287 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.284+0000 7faa42d9d640 1 --2- 172.21.15.16:0/588294287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440d1f90 0x7faa44159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.284+0000 7faa4359e640 1 --2- 172.21.15.16:0/588294287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa4415dce0 0x7faa440ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.285+0000 7faa3bfff640 1 -- 172.21.15.16:0/588294287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440d1f90 msgr2=0x7faa44159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.285+0000 7faa3bfff640 1 --2- 172.21.15.16:0/588294287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440d1f90 0x7faa44159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.285+0000 7faa3bfff640 1 -- 172.21.15.16:0/588294287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa4415dce0 msgr2=0x7faa440ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.285+0000 7faa3bfff640 1 --2- 172.21.15.16:0/588294287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa4415dce0 0x7faa440ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.285+0000 7faa3bfff640 1 -- 172.21.15.16:0/588294287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa440cbaf0 con 0x7faa440d35b0 2024-04-25T10:05:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.285+0000 7faa3bfff640 1 --2- 172.21.15.16:0/588294287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa440d35b0 0x7faa440c9b40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00e8a0 tx=0x7faa2c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.285+0000 7faa3b7fe640 1 -- 172.21.15.16:0/588294287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c017070 con 0x7faa440d35b0 2024-04-25T10:05:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.285+0000 7faa3b7fe640 1 -- 172.21.15.16:0/588294287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2c00ee90 con 0x7faa440d35b0 2024-04-25T10:05:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.285+0000 7faa49113640 1 -- 172.21.15.16:0/588294287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa440cbdd0 con 0x7faa440d35b0 2024-04-25T10:05:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.285+0000 7faa3b7fe640 1 -- 172.21.15.16:0/588294287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c00cb80 con 0x7faa440d35b0 2024-04-25T10:05:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.285+0000 7faa49113640 1 -- 172.21.15.16:0/588294287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa440cc390 con 0x7faa440d35b0 2024-04-25T10:05:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.286+0000 7faa49113640 1 -- 172.21.15.16:0/588294287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa440d1a50 con 0x7faa440d35b0 2024-04-25T10:05:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.286+0000 7faa3b7fe640 1 -- 172.21.15.16:0/588294287 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa2c01a070 con 0x7faa440d35b0 2024-04-25T10:05:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.287+0000 7faa3b7fe640 1 --2- 172.21.15.16:0/588294287 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7faa2406b4d0 0x7faa2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.287+0000 7faa3b7fe640 1 -- 172.21.15.16:0/588294287 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa2c08a280 con 0x7faa440d35b0 2024-04-25T10:05:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.287+0000 7faa42d9d640 1 --2- 172.21.15.16:0/588294287 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7faa2406b4d0 0x7faa2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.288+0000 7faa42d9d640 1 --2- 172.21.15.16:0/588294287 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7faa2406b4d0 0x7faa2406d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa34009b40 tx=0x7faa34009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.291+0000 7faa3b7fe640 1 -- 172.21.15.16:0/588294287 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa440d1a50 con 0x7faa440d35b0 2024-04-25T10:05:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.540+0000 7faa49113640 1 -- 172.21.15.16:0/588294287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7faa44159cc0 con 0x7faa440d35b0 2024-04-25T10:05:15.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.540+0000 7faa3b7fe640 1 -- 172.21.15.16:0/588294287 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1174 (secure 0 0 0) 0x7faa2c058b40 con 0x7faa440d35b0 2024-04-25T10:05:15.546 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:15.546 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":28,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":22,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039497,"num_in_osds":8,"osd_in_since":1714039488,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":81},{"state_name":"unknown","count":12},{"state_name":"creating+peering","count":4}],"num_pgs":97,"num_pools":3,"num_objects":22,"data_bytes":592708,"bytes_used":241471488,"bytes_avail":772852641792,"bytes_total":773094113280,"unknown_pgs_ratio":0.12371134012937546,"inactive_pgs_ratio":0.041237112134695053,"write_bytes_sec":3062,"read_op_per_sec":0,"write_op_per_sec":9},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":34148}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-04-25T10:04:45.622387+0000","services":{}},"progress_events":{}} 2024-04-25T10:05:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.542+0000 7faa49113640 1 -- 172.21.15.16:0/588294287 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7faa2406b4d0 msgr2=0x7faa2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.542+0000 7faa49113640 1 --2- 172.21.15.16:0/588294287 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7faa2406b4d0 0x7faa2406d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa34009b40 tx=0x7faa34009190 comp rx=0 tx=0).stop 2024-04-25T10:05:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.542+0000 7faa49113640 1 -- 172.21.15.16:0/588294287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa440d35b0 msgr2=0x7faa440c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.542+0000 7faa49113640 1 --2- 172.21.15.16:0/588294287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa440d35b0 0x7faa440c9b40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00e8a0 tx=0x7faa2c00ed60 comp rx=0 tx=0).stop 2024-04-25T10:05:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.542+0000 7faa49113640 1 -- 172.21.15.16:0/588294287 shutdown_connections 2024-04-25T10:05:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.542+0000 7faa49113640 1 --2- 172.21.15.16:0/588294287 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7faa2406b4d0 0x7faa2406d990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.542+0000 7faa49113640 1 --2- 172.21.15.16:0/588294287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa4415dce0 0x7faa440ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.542+0000 7faa49113640 1 --2- 172.21.15.16:0/588294287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa440d35b0 0x7faa440c9b40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.542+0000 7faa49113640 1 --2- 172.21.15.16:0/588294287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440d1f90 0x7faa44159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.542+0000 7faa49113640 1 -- 172.21.15.16:0/588294287 >> 172.21.15.16:0/588294287 conn(0x7faa44072900 msgr2=0x7faa440cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.542+0000 7faa49113640 1 -- 172.21.15.16:0/588294287 shutdown_connections 2024-04-25T10:05:15.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.542+0000 7faa49113640 1 -- 172.21.15.16:0/588294287 wait complete. 2024-04-25T10:05:15.558 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:05:15.558 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.654+0000 7f30c9f29640 1 Processor -- start 2024-04-25T10:05:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.654+0000 7f30c9f29640 1 -- start start 2024-04-25T10:05:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.654+0000 7f30c9f29640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f30c40d35b0 0x7f30c40b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.654+0000 7f30c9f29640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f30c40b5990 0x7f30c40b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.654+0000 7f30c9f29640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c415e0c0 con 0x7f30c415dce0 2024-04-25T10:05:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.654+0000 7f30c9f29640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c40d2510 con 0x7f30c40d35b0 2024-04-25T10:05:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.654+0000 7f30c9f29640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c40d27f0 con 0x7f30c40b5990 2024-04-25T10:05:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.655+0000 7f30c2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f30c40d35b0 0x7f30c40b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.655+0000 7f30c2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f30c40d35b0 0x7f30c40b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41846/0 (socket says 172.21.15.16:41846) 2024-04-25T10:05:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.655+0000 7f30c2ffd640 1 -- 172.21.15.16:0/1449379353 learned_addr learned my addr 172.21.15.16:0/1449379353 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.655+0000 7f30c3fff640 1 --2- 172.21.15.16:0/1449379353 >> v2:172.21.15.186:3300/0 conn(0x7f30c40b5990 0x7f30c40b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.655+0000 7f30c2ffd640 1 -- 172.21.15.16:0/1449379353 >> v2:172.21.15.186:3300/0 conn(0x7f30c40b5990 msgr2=0x7f30c40b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.655+0000 7f30c2ffd640 1 --2- 172.21.15.16:0/1449379353 >> v2:172.21.15.186:3300/0 conn(0x7f30c40b5990 0x7f30c40b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.655+0000 7f30c2ffd640 1 -- 172.21.15.16:0/1449379353 >> v1:172.21.15.186:6789/0 conn(0x7f30c415dce0 legacy=0x7f30c40d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.655+0000 7f30c2ffd640 1 -- 172.21.15.16:0/1449379353 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c40d2ad0 con 0x7f30c40d35b0 2024-04-25T10:05:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.656+0000 7f30c2ffd640 1 --2- 172.21.15.16:0/1449379353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c40d35b0 0x7f30c40b5450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f30c40b3390 tx=0x7f30b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f780a110dd9e11ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.656+0000 7f30c27fc640 1 -- 172.21.15.16:0/1449379353 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4037070 con 0x7f30c40d35b0 2024-04-25T10:05:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.656+0000 7f30c27fc640 1 -- 172.21.15.16:0/1449379353 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b4032440 con 0x7f30c40d35b0 2024-04-25T10:05:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.656+0000 7f30c27fc640 1 -- 172.21.15.16:0/1449379353 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4032810 con 0x7f30c40d35b0 2024-04-25T10:05:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.656+0000 7f30c9f29640 1 -- 172.21.15.16:0/1449379353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c40d35b0 msgr2=0x7f30c40b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.656+0000 7f30c9f29640 1 --2- 172.21.15.16:0/1449379353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c40d35b0 0x7f30c40b5450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f30c40b3390 tx=0x7f30b402a270 comp rx=0 tx=0).stop 2024-04-25T10:05:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.656+0000 7f30c9f29640 1 -- 172.21.15.16:0/1449379353 shutdown_connections 2024-04-25T10:05:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.656+0000 7f30c9f29640 1 --2- 172.21.15.16:0/1449379353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c40d35b0 0x7f30c40b5450 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.656+0000 7f30c9f29640 1 --2- 172.21.15.16:0/1449379353 >> v2:172.21.15.186:3300/0 conn(0x7f30c40b5990 0x7f30c40b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.656+0000 7f30c9f29640 1 -- 172.21.15.16:0/1449379353 >> 172.21.15.16:0/1449379353 conn(0x7f30c4072900 msgr2=0x7f30c40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.656+0000 7f30c9f29640 1 -- 172.21.15.16:0/1449379353 shutdown_connections 2024-04-25T10:05:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.656+0000 7f30c9f29640 1 -- 172.21.15.16:0/1449379353 wait complete. 2024-04-25T10:05:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.657+0000 7f30c9f29640 1 Processor -- start 2024-04-25T10:05:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.658+0000 7f30c9f29640 1 -- start start 2024-04-25T10:05:15.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.658+0000 7f30c9f29640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c40b5990 0x7f30c40c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.658+0000 7f30c9f29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c40d35b0 0x7f30c415a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.658+0000 7f30c9f29640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30c415dce0 0x7f30c4156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.658+0000 7f30c9f29640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c40d2160 con 0x7f30c40d35b0 2024-04-25T10:05:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.658+0000 7f30c9f29640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c40d3090 con 0x7f30c40b5990 2024-04-25T10:05:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.658+0000 7f30c9f29640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c415e2b0 con 0x7f30c415dce0 2024-04-25T10:05:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.658+0000 7f30c2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c40d35b0 0x7f30c415a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.658+0000 7f30c2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c40d35b0 0x7f30c415a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41856/0 (socket says 172.21.15.16:41856) 2024-04-25T10:05:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.658+0000 7f30c2ffd640 1 -- 172.21.15.16:0/2424117164 learned_addr learned my addr 172.21.15.16:0/2424117164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.659+0000 7f30c3fff640 1 --2- 172.21.15.16:0/2424117164 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30c415dce0 0x7f30c4156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.659+0000 7f30c37fe640 1 --2- 172.21.15.16:0/2424117164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c40b5990 0x7f30c40c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.659+0000 7f30c2ffd640 1 -- 172.21.15.16:0/2424117164 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30c415dce0 msgr2=0x7f30c4156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.659+0000 7f30c2ffd640 1 --2- 172.21.15.16:0/2424117164 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30c415dce0 0x7f30c4156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.659+0000 7f30c2ffd640 1 -- 172.21.15.16:0/2424117164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c40b5990 msgr2=0x7f30c40c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.659+0000 7f30c2ffd640 1 --2- 172.21.15.16:0/2424117164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c40b5990 0x7f30c40c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.659+0000 7f30c2ffd640 1 -- 172.21.15.16:0/2424117164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c415e500 con 0x7f30c40d35b0 2024-04-25T10:05:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.659+0000 7f30c2ffd640 1 --2- 172.21.15.16:0/2424117164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c40d35b0 0x7f30c415a590 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f30b402a2b0 tx=0x7f30b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.659+0000 7f30abfff640 1 -- 172.21.15.16:0/2424117164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4037040 con 0x7f30c40d35b0 2024-04-25T10:05:15.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.659+0000 7f30abfff640 1 -- 172.21.15.16:0/2424117164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b4042070 con 0x7f30c40d35b0 2024-04-25T10:05:15.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.659+0000 7f30c9f29640 1 -- 172.21.15.16:0/2424117164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c415e7e0 con 0x7f30c40d35b0 2024-04-25T10:05:15.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.659+0000 7f30abfff640 1 -- 172.21.15.16:0/2424117164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4030880 con 0x7f30c40d35b0 2024-04-25T10:05:15.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.659+0000 7f30c9f29640 1 -- 172.21.15.16:0/2424117164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c415aad0 con 0x7f30c40d35b0 2024-04-25T10:05:15.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.660+0000 7f30c9f29640 1 -- 172.21.15.16:0/2424117164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c40749b0 con 0x7f30c40d35b0 2024-04-25T10:05:15.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.661+0000 7f30abfff640 1 -- 172.21.15.16:0/2424117164 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30b4030a20 con 0x7f30c40d35b0 2024-04-25T10:05:15.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.661+0000 7f30abfff640 1 --2- 172.21.15.16:0/2424117164 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f30b006b4d0 0x7f30b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.661+0000 7f30abfff640 1 -- 172.21.15.16:0/2424117164 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30b40aaa60 con 0x7f30c40d35b0 2024-04-25T10:05:15.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.661+0000 7f30c37fe640 1 --2- 172.21.15.16:0/2424117164 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f30b006b4d0 0x7f30b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.662+0000 7f30c37fe640 1 --2- 172.21.15.16:0/2424117164 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f30b006b4d0 0x7f30b006d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f30b8007fd0 tx=0x7f30b8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.665+0000 7f30abfff640 1 -- 172.21.15.16:0/2424117164 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30b4079970 con 0x7f30c40d35b0 2024-04-25T10:05:15.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.840+0000 7f30c9f29640 1 -- 172.21.15.16:0/2424117164 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f30c415d390 con 0x7f30b006b4d0 2024-04-25T10:05:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.847+0000 7f30abfff640 1 -- 172.21.15.16:0/2424117164 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251906 (secure 0 0 0) 0x7f30c415d390 con 0x7f30b006b4d0 2024-04-25T10:05:15.855 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:15.856 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:05:15.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.850+0000 7f30c9f29640 1 -- 172.21.15.16:0/2424117164 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f30b006b4d0 msgr2=0x7f30b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.850+0000 7f30c9f29640 1 --2- 172.21.15.16:0/2424117164 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f30b006b4d0 0x7f30b006d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f30b8007fd0 tx=0x7f30b8009040 comp rx=0 tx=0).stop 2024-04-25T10:05:15.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.850+0000 7f30c9f29640 1 -- 172.21.15.16:0/2424117164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c40d35b0 msgr2=0x7f30c415a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.850+0000 7f30c9f29640 1 --2- 172.21.15.16:0/2424117164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c40d35b0 0x7f30c415a590 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f30b402a2b0 tx=0x7f30b40046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.850+0000 7f30c9f29640 1 -- 172.21.15.16:0/2424117164 shutdown_connections 2024-04-25T10:05:15.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.850+0000 7f30c9f29640 1 --2- 172.21.15.16:0/2424117164 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f30b006b4d0 0x7f30b006d990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.850+0000 7f30c9f29640 1 --2- 172.21.15.16:0/2424117164 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30c415dce0 0x7f30c4156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.850+0000 7f30c9f29640 1 --2- 172.21.15.16:0/2424117164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30c40d35b0 0x7f30c415a590 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.850+0000 7f30c9f29640 1 --2- 172.21.15.16:0/2424117164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c40b5990 0x7f30c40c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.850+0000 7f30c9f29640 1 -- 172.21.15.16:0/2424117164 >> 172.21.15.16:0/2424117164 conn(0x7f30c4072900 msgr2=0x7f30c40d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.850+0000 7f30c9f29640 1 -- 172.21.15.16:0/2424117164 shutdown_connections 2024-04-25T10:05:15.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:15.850+0000 7f30c9f29640 1 -- 172.21.15.16:0/2424117164 wait complete. 2024-04-25T10:05:15.866 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-25T10:05:15.782427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235672,"kb_used_data":2868,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739048,"statfs":{"total":773094113280,"available":772852785152,"internally_reserved":0,"allocated":2936832,"data_stored":1753968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001630"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.475316+0000","last_change":"2024-04-25T10:05:11.473253+0000","last_active":"2024-04-25T10:05:11.475316+0000","last_peered":"2024-04-25T10:05:11.475316+0000","last_clean":"2024-04-25T10:05:11.475316+0000","last_became_active":"2024-04-25T10:05:11.472940+0000","last_became_peered":"2024-04-25T10:05:11.472940+0000","last_unstale":"2024-04-25T10:05:11.475316+0000","last_undegraded":"2024-04-25T10:05:11.475316+0000","last_fullsized":"2024-04-25T10:05:11.475316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:52.014434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786817+0000","last_change":"2024-04-25T10:05:11.460499+0000","last_active":"2024-04-25T10:05:14.786817+0000","last_peered":"2024-04-25T10:05:14.786817+0000","last_clean":"2024-04-25T10:05:14.786817+0000","last_became_active":"2024-04-25T10:05:11.458776+0000","last_became_peered":"2024-04-25T10:05:11.458776+0000","last_unstale":"2024-04-25T10:05:14.786817+0000","last_undegraded":"2024-04-25T10:05:14.786817+0000","last_fullsized":"2024-04-25T10:05:14.786817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:17.247685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786678+0000","last_change":"2024-04-25T10:05:11.466721+0000","last_active":"2024-04-25T10:05:14.786678+0000","last_peered":"2024-04-25T10:05:14.786678+0000","last_clean":"2024-04-25T10:05:14.786678+0000","last_became_active":"2024-04-25T10:05:11.466430+0000","last_became_peered":"2024-04-25T10:05:11.466430+0000","last_unstale":"2024-04-25T10:05:14.786678+0000","last_undegraded":"2024-04-25T10:05:14.786678+0000","last_fullsized":"2024-04-25T10:05:14.786678+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:34.227544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455508+0000","last_change":"2024-04-25T10:05:11.461498+0000","last_active":"2024-04-25T10:05:12.455508+0000","last_peered":"2024-04-25T10:05:12.455508+0000","last_clean":"2024-04-25T10:05:12.455508+0000","last_became_active":"2024-04-25T10:05:11.461241+0000","last_became_peered":"2024-04-25T10:05:11.461241+0000","last_unstale":"2024-04-25T10:05:12.455508+0000","last_undegraded":"2024-04-25T10:05:12.455508+0000","last_fullsized":"2024-04-25T10:05:12.455508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472722+0000","last_change":"2024-04-25T10:05:11.469281+0000","last_active":"2024-04-25T10:05:11.472722+0000","last_peered":"2024-04-25T10:05:11.472722+0000","last_clean":"2024-04-25T10:05:11.472722+0000","last_became_active":"2024-04-25T10:05:11.468809+0000","last_became_peered":"2024-04-25T10:05:11.468809+0000","last_unstale":"2024-04-25T10:05:11.472722+0000","last_undegraded":"2024-04-25T10:05:11.472722+0000","last_fullsized":"2024-04-25T10:05:11.472722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:30.789969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455529+0000","last_change":"2024-04-25T10:05:11.466531+0000","last_active":"2024-04-25T10:05:12.455529+0000","last_peered":"2024-04-25T10:05:12.455529+0000","last_clean":"2024-04-25T10:05:12.455529+0000","last_became_active":"2024-04-25T10:05:11.466126+0000","last_became_peered":"2024-04-25T10:05:11.466126+0000","last_unstale":"2024-04-25T10:05:12.455529+0000","last_undegraded":"2024-04-25T10:05:12.455529+0000","last_fullsized":"2024-04-25T10:05:12.455529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469389+0000","last_change":"2024-04-25T10:05:11.466336+0000","last_active":"2024-04-25T10:05:11.469389+0000","last_peered":"2024-04-25T10:05:11.469389+0000","last_clean":"2024-04-25T10:05:11.469389+0000","last_became_active":"2024-04-25T10:05:11.466032+0000","last_became_peered":"2024-04-25T10:05:11.466032+0000","last_unstale":"2024-04-25T10:05:11.469389+0000","last_undegraded":"2024-04-25T10:05:11.469389+0000","last_fullsized":"2024-04-25T10:05:11.469389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:12.926321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786457+0000","last_change":"2024-04-25T10:05:11.461892+0000","last_active":"2024-04-25T10:05:14.786457+0000","last_peered":"2024-04-25T10:05:14.786457+0000","last_clean":"2024-04-25T10:05:14.786457+0000","last_became_active":"2024-04-25T10:05:11.461264+0000","last_became_peered":"2024-04-25T10:05:11.461264+0000","last_unstale":"2024-04-25T10:05:14.786457+0000","last_undegraded":"2024-04-25T10:05:14.786457+0000","last_fullsized":"2024-04-25T10:05:14.786457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:58.263921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456654+0000","last_change":"2024-04-25T10:05:11.461997+0000","last_active":"2024-04-25T10:05:12.456654+0000","last_peered":"2024-04-25T10:05:12.456654+0000","last_clean":"2024-04-25T10:05:12.456654+0000","last_became_active":"2024-04-25T10:05:11.461658+0000","last_became_peered":"2024-04-25T10:05:11.461658+0000","last_unstale":"2024-04-25T10:05:12.456654+0000","last_undegraded":"2024-04-25T10:05:12.456654+0000","last_fullsized":"2024-04-25T10:05:12.456654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.463064+0000","last_change":"2024-04-25T10:05:11.461631+0000","last_active":"2024-04-25T10:05:11.463064+0000","last_peered":"2024-04-25T10:05:11.463064+0000","last_clean":"2024-04-25T10:05:11.463064+0000","last_became_active":"2024-04-25T10:05:11.461327+0000","last_became_peered":"2024-04-25T10:05:11.461327+0000","last_unstale":"2024-04-25T10:05:11.463064+0000","last_undegraded":"2024-04-25T10:05:11.463064+0000","last_fullsized":"2024-04-25T10:05:11.463064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.473963+0000","last_change":"2024-04-25T10:05:11.473044+0000","last_active":"2024-04-25T10:05:11.473963+0000","last_peered":"2024-04-25T10:05:11.473963+0000","last_clean":"2024-04-25T10:05:11.473963+0000","last_became_active":"2024-04-25T10:05:11.472747+0000","last_became_peered":"2024-04-25T10:05:11.472747+0000","last_unstale":"2024-04-25T10:05:11.473963+0000","last_undegraded":"2024-04-25T10:05:11.473963+0000","last_fullsized":"2024-04-25T10:05:11.473963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474718+0000","last_change":"2024-04-25T10:05:11.470219+0000","last_active":"2024-04-25T10:05:11.474718+0000","last_peered":"2024-04-25T10:05:11.474718+0000","last_clean":"2024-04-25T10:05:11.474718+0000","last_became_active":"2024-04-25T10:05:11.469066+0000","last_became_peered":"2024-04-25T10:05:11.469066+0000","last_unstale":"2024-04-25T10:05:11.474718+0000","last_undegraded":"2024-04-25T10:05:11.474718+0000","last_fullsized":"2024-04-25T10:05:11.474718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:54.934726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.466367+0000","last_change":"2024-04-25T10:05:11.463262+0000","last_active":"2024-04-25T10:05:11.466367+0000","last_peered":"2024-04-25T10:05:11.466367+0000","last_clean":"2024-04-25T10:05:11.466367+0000","last_became_active":"2024-04-25T10:05:11.462970+0000","last_became_peered":"2024-04-25T10:05:11.462970+0000","last_unstale":"2024-04-25T10:05:11.466367+0000","last_undegraded":"2024-04-25T10:05:11.466367+0000","last_fullsized":"2024-04-25T10:05:11.466367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.464756+0000","last_change":"2024-04-25T10:05:11.458310+0000","last_active":"2024-04-25T10:05:11.464756+0000","last_peered":"2024-04-25T10:05:11.464756+0000","last_clean":"2024-04-25T10:05:11.464756+0000","last_became_active":"2024-04-25T10:05:11.457672+0000","last_became_peered":"2024-04-25T10:05:11.457672+0000","last_unstale":"2024-04-25T10:05:11.464756+0000","last_undegraded":"2024-04-25T10:05:11.464756+0000","last_fullsized":"2024-04-25T10:05:11.464756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:02.561557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786423+0000","last_change":"2024-04-25T10:05:11.461506+0000","last_active":"2024-04-25T10:05:14.786423+0000","last_peered":"2024-04-25T10:05:14.786423+0000","last_clean":"2024-04-25T10:05:14.786423+0000","last_became_active":"2024-04-25T10:05:11.460963+0000","last_became_peered":"2024-04-25T10:05:11.460963+0000","last_unstale":"2024-04-25T10:05:14.786423+0000","last_undegraded":"2024-04-25T10:05:14.786423+0000","last_fullsized":"2024-04-25T10:05:14.786423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:45.096476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471639+0000","last_change":"2024-04-25T10:05:11.470743+0000","last_active":"2024-04-25T10:05:11.471639+0000","last_peered":"2024-04-25T10:05:11.471639+0000","last_clean":"2024-04-25T10:05:11.471639+0000","last_became_active":"2024-04-25T10:05:11.470447+0000","last_became_peered":"2024-04-25T10:05:11.470447+0000","last_unstale":"2024-04-25T10:05:11.471639+0000","last_undegraded":"2024-04-25T10:05:11.471639+0000","last_fullsized":"2024-04-25T10:05:11.471639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:41.766168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.473455+0000","last_change":"2024-04-25T10:05:11.472530+0000","last_active":"2024-04-25T10:05:11.473455+0000","last_peered":"2024-04-25T10:05:11.473455+0000","last_clean":"2024-04-25T10:05:11.473455+0000","last_became_active":"2024-04-25T10:05:11.472164+0000","last_became_peered":"2024-04-25T10:05:11.472164+0000","last_unstale":"2024-04-25T10:05:11.473455+0000","last_undegraded":"2024-04-25T10:05:11.473455+0000","last_fullsized":"2024-04-25T10:05:11.473455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:08.525515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471968+0000","last_change":"2024-04-25T10:05:11.464186+0000","last_active":"2024-04-25T10:05:11.471968+0000","last_peered":"2024-04-25T10:05:11.471968+0000","last_clean":"2024-04-25T10:05:11.471968+0000","last_became_active":"2024-04-25T10:05:11.463925+0000","last_became_peered":"2024-04-25T10:05:11.463925+0000","last_unstale":"2024-04-25T10:05:11.471968+0000","last_undegraded":"2024-04-25T10:05:11.471968+0000","last_fullsized":"2024-04-25T10:05:11.471968+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:55.300183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474686+0000","last_change":"2024-04-25T10:05:11.473554+0000","last_active":"2024-04-25T10:05:11.474686+0000","last_peered":"2024-04-25T10:05:11.474686+0000","last_clean":"2024-04-25T10:05:11.474686+0000","last_became_active":"2024-04-25T10:05:11.473271+0000","last_became_peered":"2024-04-25T10:05:11.473271+0000","last_unstale":"2024-04-25T10:05:11.474686+0000","last_undegraded":"2024-04-25T10:05:11.474686+0000","last_fullsized":"2024-04-25T10:05:11.474686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:35.076762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474740+0000","last_change":"2024-04-25T10:05:11.471477+0000","last_active":"2024-04-25T10:05:11.474740+0000","last_peered":"2024-04-25T10:05:11.474740+0000","last_clean":"2024-04-25T10:05:11.474740+0000","last_became_active":"2024-04-25T10:05:11.471219+0000","last_became_peered":"2024-04-25T10:05:11.471219+0000","last_unstale":"2024-04-25T10:05:11.474740+0000","last_undegraded":"2024-04-25T10:05:11.474740+0000","last_fullsized":"2024-04-25T10:05:11.474740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456655+0000","last_change":"2024-04-25T10:05:11.464410+0000","last_active":"2024-04-25T10:05:12.456655+0000","last_peered":"2024-04-25T10:05:12.456655+0000","last_clean":"2024-04-25T10:05:12.456655+0000","last_became_active":"2024-04-25T10:05:11.464003+0000","last_became_peered":"2024-04-25T10:05:11.464003+0000","last_unstale":"2024-04-25T10:05:12.456655+0000","last_undegraded":"2024-04-25T10:05:12.456655+0000","last_fullsized":"2024-04-25T10:05:12.456655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:52.014434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456342+0000","last_change":"2024-04-25T10:05:11.462487+0000","last_active":"2024-04-25T10:05:12.456342+0000","last_peered":"2024-04-25T10:05:12.456342+0000","last_clean":"2024-04-25T10:05:12.456342+0000","last_became_active":"2024-04-25T10:05:11.462191+0000","last_became_peered":"2024-04-25T10:05:11.462191+0000","last_unstale":"2024-04-25T10:05:12.456342+0000","last_undegraded":"2024-04-25T10:05:12.456342+0000","last_fullsized":"2024-04-25T10:05:12.456342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786682+0000","last_change":"2024-04-25T10:05:11.450886+0000","last_active":"2024-04-25T10:05:14.786682+0000","last_peered":"2024-04-25T10:05:14.786682+0000","last_clean":"2024-04-25T10:05:14.786682+0000","last_became_active":"2024-04-25T10:05:11.450616+0000","last_became_peered":"2024-04-25T10:05:11.450616+0000","last_unstale":"2024-04-25T10:05:14.786682+0000","last_undegraded":"2024-04-25T10:05:14.786682+0000","last_fullsized":"2024-04-25T10:05:14.786682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:47.607490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469823+0000","last_change":"2024-04-25T10:05:11.467406+0000","last_active":"2024-04-25T10:05:11.469823+0000","last_peered":"2024-04-25T10:05:11.469823+0000","last_clean":"2024-04-25T10:05:11.469823+0000","last_became_active":"2024-04-25T10:05:11.467126+0000","last_became_peered":"2024-04-25T10:05:11.467126+0000","last_unstale":"2024-04-25T10:05:11.469823+0000","last_undegraded":"2024-04-25T10:05:11.469823+0000","last_fullsized":"2024-04-25T10:05:11.469823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:52.014434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786879+0000","last_change":"2024-04-25T10:05:11.453135+0000","last_active":"2024-04-25T10:05:14.786879+0000","last_peered":"2024-04-25T10:05:14.786879+0000","last_clean":"2024-04-25T10:05:14.786879+0000","last_became_active":"2024-04-25T10:05:11.452053+0000","last_became_peered":"2024-04-25T10:05:11.452053+0000","last_unstale":"2024-04-25T10:05:14.786879+0000","last_undegraded":"2024-04-25T10:05:14.786879+0000","last_fullsized":"2024-04-25T10:05:14.786879+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:03.147551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455642+0000","last_change":"2024-04-25T10:05:11.469452+0000","last_active":"2024-04-25T10:05:12.455642+0000","last_peered":"2024-04-25T10:05:12.455642+0000","last_clean":"2024-04-25T10:05:12.455642+0000","last_became_active":"2024-04-25T10:05:11.465896+0000","last_became_peered":"2024-04-25T10:05:11.465896+0000","last_unstale":"2024-04-25T10:05:12.455642+0000","last_undegraded":"2024-04-25T10:05:12.455642+0000","last_fullsized":"2024-04-25T10:05:12.455642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:14.506306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.460957+0000","last_change":"2024-04-25T10:05:11.457595+0000","last_active":"2024-04-25T10:05:11.460957+0000","last_peered":"2024-04-25T10:05:11.460957+0000","last_clean":"2024-04-25T10:05:11.460957+0000","last_became_active":"2024-04-25T10:05:11.457320+0000","last_became_peered":"2024-04-25T10:05:11.457320+0000","last_unstale":"2024-04-25T10:05:11.460957+0000","last_undegraded":"2024-04-25T10:05:11.460957+0000","last_fullsized":"2024-04-25T10:05:11.460957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:40.831935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472125+0000","last_change":"2024-04-25T10:05:11.469113+0000","last_active":"2024-04-25T10:05:11.472125+0000","last_peered":"2024-04-25T10:05:11.472125+0000","last_clean":"2024-04-25T10:05:11.472125+0000","last_became_active":"2024-04-25T10:05:11.468759+0000","last_became_peered":"2024-04-25T10:05:11.468759+0000","last_unstale":"2024-04-25T10:05:11.472125+0000","last_undegraded":"2024-04-25T10:05:11.472125+0000","last_fullsized":"2024-04-25T10:05:11.472125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786630+0000","last_change":"2024-04-25T10:05:11.457486+0000","last_active":"2024-04-25T10:05:14.786630+0000","last_peered":"2024-04-25T10:05:14.786630+0000","last_clean":"2024-04-25T10:05:14.786630+0000","last_became_active":"2024-04-25T10:05:11.454855+0000","last_became_peered":"2024-04-25T10:05:11.454855+0000","last_unstale":"2024-04-25T10:05:14.786630+0000","last_undegraded":"2024-04-25T10:05:14.786630+0000","last_fullsized":"2024-04-25T10:05:14.786630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:45.863831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471666+0000","last_change":"2024-04-25T10:05:11.470055+0000","last_active":"2024-04-25T10:05:11.471666+0000","last_peered":"2024-04-25T10:05:11.471666+0000","last_clean":"2024-04-25T10:05:11.471666+0000","last_became_active":"2024-04-25T10:05:11.468974+0000","last_became_peered":"2024-04-25T10:05:11.468974+0000","last_unstale":"2024-04-25T10:05:11.471666+0000","last_undegraded":"2024-04-25T10:05:11.471666+0000","last_fullsized":"2024-04-25T10:05:11.471666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.228600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469311+0000","last_change":"2024-04-25T10:05:11.462234+0000","last_active":"2024-04-25T10:05:11.469311+0000","last_peered":"2024-04-25T10:05:11.469311+0000","last_clean":"2024-04-25T10:05:11.469311+0000","last_became_active":"2024-04-25T10:05:11.461975+0000","last_became_peered":"2024-04-25T10:05:11.461975+0000","last_unstale":"2024-04-25T10:05:11.469311+0000","last_undegraded":"2024-04-25T10:05:11.469311+0000","last_fullsized":"2024-04-25T10:05:11.469311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:02.561557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455396+0000","last_change":"2024-04-25T10:05:11.463406+0000","last_active":"2024-04-25T10:05:12.455396+0000","last_peered":"2024-04-25T10:05:12.455396+0000","last_clean":"2024-04-25T10:05:12.455396+0000","last_became_active":"2024-04-25T10:05:11.463139+0000","last_became_peered":"2024-04-25T10:05:11.463139+0000","last_unstale":"2024-04-25T10:05:12.455396+0000","last_undegraded":"2024-04-25T10:05:12.455396+0000","last_fullsized":"2024-04-25T10:05:12.455396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456384+0000","last_change":"2024-04-25T10:05:11.453168+0000","last_active":"2024-04-25T10:05:12.456384+0000","last_peered":"2024-04-25T10:05:12.456384+0000","last_clean":"2024-04-25T10:05:12.456384+0000","last_became_active":"2024-04-25T10:05:11.452238+0000","last_became_peered":"2024-04-25T10:05:11.452238+0000","last_unstale":"2024-04-25T10:05:12.456384+0000","last_undegraded":"2024-04-25T10:05:12.456384+0000","last_fullsized":"2024-04-25T10:05:12.456384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786706+0000","last_change":"2024-04-25T10:05:10.455995+0000","last_active":"2024-04-25T10:05:14.786706+0000","last_peered":"2024-04-25T10:05:14.786706+0000","last_clean":"2024-04-25T10:05:14.786706+0000","last_became_active":"2024-04-25T10:05:10.455718+0000","last_became_peered":"2024-04-25T10:05:10.455718+0000","last_unstale":"2024-04-25T10:05:14.786706+0000","last_undegraded":"2024-04-25T10:05:14.786706+0000","last_fullsized":"2024-04-25T10:05:14.786706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:47.448482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456962+0000","last_change":"2024-04-25T10:05:11.469760+0000","last_active":"2024-04-25T10:05:12.456962+0000","last_peered":"2024-04-25T10:05:12.456962+0000","last_clean":"2024-04-25T10:05:12.456962+0000","last_became_active":"2024-04-25T10:05:11.469490+0000","last_became_peered":"2024-04-25T10:05:11.469490+0000","last_unstale":"2024-04-25T10:05:12.456962+0000","last_undegraded":"2024-04-25T10:05:12.456962+0000","last_fullsized":"2024-04-25T10:05:12.456962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.228600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.475721+0000","last_change":"2024-04-25T10:05:10.461940+0000","last_active":"2024-04-25T10:05:11.475721+0000","last_peered":"2024-04-25T10:05:11.475721+0000","last_clean":"2024-04-25T10:05:11.475721+0000","last_became_active":"2024-04-25T10:05:10.461633+0000","last_became_peered":"2024-04-25T10:05:10.461633+0000","last_unstale":"2024-04-25T10:05:11.475721+0000","last_undegraded":"2024-04-25T10:05:11.475721+0000","last_fullsized":"2024-04-25T10:05:11.475721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.456539+0000","last_change":"2024-04-25T10:05:10.460823+0000","last_active":"2024-04-25T10:05:11.456539+0000","last_peered":"2024-04-25T10:05:11.456539+0000","last_clean":"2024-04-25T10:05:11.456539+0000","last_became_active":"2024-04-25T10:05:10.460480+0000","last_became_peered":"2024-04-25T10:05:10.460480+0000","last_unstale":"2024-04-25T10:05:11.456539+0000","last_undegraded":"2024-04-25T10:05:11.456539+0000","last_fullsized":"2024-04-25T10:05:11.456539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:31.375084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786743+0000","last_change":"2024-04-25T10:05:11.460164+0000","last_active":"2024-04-25T10:05:14.786743+0000","last_peered":"2024-04-25T10:05:14.786743+0000","last_clean":"2024-04-25T10:05:14.786743+0000","last_became_active":"2024-04-25T10:05:11.459615+0000","last_became_peered":"2024-04-25T10:05:11.459615+0000","last_unstale":"2024-04-25T10:05:14.786743+0000","last_undegraded":"2024-04-25T10:05:14.786743+0000","last_fullsized":"2024-04-25T10:05:14.786743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:00.275933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.450458+0000","last_change":"2024-04-25T10:05:10.461713+0000","last_active":"2024-04-25T10:05:11.450458+0000","last_peered":"2024-04-25T10:05:11.450458+0000","last_clean":"2024-04-25T10:05:11.450458+0000","last_became_active":"2024-04-25T10:05:10.461379+0000","last_became_peered":"2024-04-25T10:05:10.461379+0000","last_unstale":"2024-04-25T10:05:11.450458+0000","last_undegraded":"2024-04-25T10:05:11.450458+0000","last_fullsized":"2024-04-25T10:05:11.450458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456510+0000","last_change":"2024-04-25T10:05:11.466408+0000","last_active":"2024-04-25T10:05:12.456510+0000","last_peered":"2024-04-25T10:05:12.456510+0000","last_clean":"2024-04-25T10:05:12.456510+0000","last_became_active":"2024-04-25T10:05:11.465611+0000","last_became_peered":"2024-04-25T10:05:11.465611+0000","last_unstale":"2024-04-25T10:05:12.456510+0000","last_undegraded":"2024-04-25T10:05:12.456510+0000","last_fullsized":"2024-04-25T10:05:12.456510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456657+0000","last_change":"2024-04-25T10:05:10.449054+0000","last_active":"2024-04-25T10:05:12.456657+0000","last_peered":"2024-04-25T10:05:12.456657+0000","last_clean":"2024-04-25T10:05:12.456657+0000","last_became_active":"2024-04-25T10:05:10.448743+0000","last_became_peered":"2024-04-25T10:05:10.448743+0000","last_unstale":"2024-04-25T10:05:12.456657+0000","last_undegraded":"2024-04-25T10:05:12.456657+0000","last_fullsized":"2024-04-25T10:05:12.456657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:22.897294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455686+0000","last_change":"2024-04-25T10:05:11.453268+0000","last_active":"2024-04-25T10:05:12.455686+0000","last_peered":"2024-04-25T10:05:12.455686+0000","last_clean":"2024-04-25T10:05:12.455686+0000","last_became_active":"2024-04-25T10:05:11.452971+0000","last_became_peered":"2024-04-25T10:05:11.452971+0000","last_unstale":"2024-04-25T10:05:12.455686+0000","last_undegraded":"2024-04-25T10:05:12.455686+0000","last_fullsized":"2024-04-25T10:05:12.455686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:56.142030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.452585+0000","last_change":"2024-04-25T10:05:10.452247+0000","last_active":"2024-04-25T10:05:11.452585+0000","last_peered":"2024-04-25T10:05:11.452585+0000","last_clean":"2024-04-25T10:05:11.452585+0000","last_became_active":"2024-04-25T10:05:10.451925+0000","last_became_peered":"2024-04-25T10:05:10.451925+0000","last_unstale":"2024-04-25T10:05:11.452585+0000","last_undegraded":"2024-04-25T10:05:11.452585+0000","last_fullsized":"2024-04-25T10:05:11.452585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:53.640804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786996+0000","last_change":"2024-04-25T10:05:11.467490+0000","last_active":"2024-04-25T10:05:14.786996+0000","last_peered":"2024-04-25T10:05:14.786996+0000","last_clean":"2024-04-25T10:05:14.786996+0000","last_became_active":"2024-04-25T10:05:11.467217+0000","last_became_peered":"2024-04-25T10:05:11.467217+0000","last_unstale":"2024-04-25T10:05:14.786996+0000","last_undegraded":"2024-04-25T10:05:14.786996+0000","last_fullsized":"2024-04-25T10:05:14.786996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:26.821659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472671+0000","last_change":"2024-04-25T10:05:10.459898+0000","last_active":"2024-04-25T10:05:11.472671+0000","last_peered":"2024-04-25T10:05:11.472671+0000","last_clean":"2024-04-25T10:05:11.472671+0000","last_became_active":"2024-04-25T10:05:10.459250+0000","last_became_peered":"2024-04-25T10:05:10.459250+0000","last_unstale":"2024-04-25T10:05:11.472671+0000","last_undegraded":"2024-04-25T10:05:11.472671+0000","last_fullsized":"2024-04-25T10:05:11.472671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:41.255007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.473052+0000","last_change":"2024-04-25T10:05:11.471498+0000","last_active":"2024-04-25T10:05:11.473052+0000","last_peered":"2024-04-25T10:05:11.473052+0000","last_clean":"2024-04-25T10:05:11.473052+0000","last_became_active":"2024-04-25T10:05:11.471193+0000","last_became_peered":"2024-04-25T10:05:11.471193+0000","last_unstale":"2024-04-25T10:05:11.473052+0000","last_undegraded":"2024-04-25T10:05:11.473052+0000","last_fullsized":"2024-04-25T10:05:11.473052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:45.980553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455560+0000","last_change":"2024-04-25T10:05:10.450534+0000","last_active":"2024-04-25T10:05:12.455560+0000","last_peered":"2024-04-25T10:05:12.455560+0000","last_clean":"2024-04-25T10:05:12.455560+0000","last_became_active":"2024-04-25T10:05:10.450251+0000","last_became_peered":"2024-04-25T10:05:10.450251+0000","last_unstale":"2024-04-25T10:05:12.455560+0000","last_undegraded":"2024-04-25T10:05:12.455560+0000","last_fullsized":"2024-04-25T10:05:12.455560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:22.897294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.468770+0000","last_change":"2024-04-25T10:05:11.463480+0000","last_active":"2024-04-25T10:05:11.468770+0000","last_peered":"2024-04-25T10:05:11.468770+0000","last_clean":"2024-04-25T10:05:11.468770+0000","last_became_active":"2024-04-25T10:05:11.460995+0000","last_became_peered":"2024-04-25T10:05:11.460995+0000","last_unstale":"2024-04-25T10:05:11.468770+0000","last_undegraded":"2024-04-25T10:05:11.468770+0000","last_fullsized":"2024-04-25T10:05:11.468770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:56.142030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.464540+0000","last_change":"2024-04-25T10:05:10.462323+0000","last_active":"2024-04-25T10:05:11.464540+0000","last_peered":"2024-04-25T10:05:11.464540+0000","last_clean":"2024-04-25T10:05:11.464540+0000","last_became_active":"2024-04-25T10:05:10.462118+0000","last_became_peered":"2024-04-25T10:05:10.462118+0000","last_unstale":"2024-04-25T10:05:11.464540+0000","last_undegraded":"2024-04-25T10:05:11.464540+0000","last_fullsized":"2024-04-25T10:05:11.464540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:39.826605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471805+0000","last_change":"2024-04-25T10:05:11.467396+0000","last_active":"2024-04-25T10:05:11.471805+0000","last_peered":"2024-04-25T10:05:11.471805+0000","last_clean":"2024-04-25T10:05:11.471805+0000","last_became_active":"2024-04-25T10:05:11.467112+0000","last_became_peered":"2024-04-25T10:05:11.467112+0000","last_unstale":"2024-04-25T10:05:11.471805+0000","last_undegraded":"2024-04-25T10:05:11.471805+0000","last_fullsized":"2024-04-25T10:05:11.471805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.451714+0000","last_change":"2024-04-25T10:05:10.458429+0000","last_active":"2024-04-25T10:05:11.451714+0000","last_peered":"2024-04-25T10:05:11.451714+0000","last_clean":"2024-04-25T10:05:11.451714+0000","last_became_active":"2024-04-25T10:05:10.457958+0000","last_became_peered":"2024-04-25T10:05:10.457958+0000","last_unstale":"2024-04-25T10:05:11.451714+0000","last_undegraded":"2024-04-25T10:05:11.451714+0000","last_fullsized":"2024-04-25T10:05:11.451714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:23.585636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469949+0000","last_change":"2024-04-25T10:05:11.465006+0000","last_active":"2024-04-25T10:05:11.469949+0000","last_peered":"2024-04-25T10:05:11.469949+0000","last_clean":"2024-04-25T10:05:11.469949+0000","last_became_active":"2024-04-25T10:05:11.464719+0000","last_became_peered":"2024-04-25T10:05:11.464719+0000","last_unstale":"2024-04-25T10:05:11.469949+0000","last_undegraded":"2024-04-25T10:05:11.469949+0000","last_fullsized":"2024-04-25T10:05:11.469949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:54.934726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.459177+0000","last_change":"2024-04-25T10:05:10.456938+0000","last_active":"2024-04-25T10:05:11.459177+0000","last_peered":"2024-04-25T10:05:11.459177+0000","last_clean":"2024-04-25T10:05:11.459177+0000","last_became_active":"2024-04-25T10:05:10.456644+0000","last_became_peered":"2024-04-25T10:05:10.456644+0000","last_unstale":"2024-04-25T10:05:11.459177+0000","last_undegraded":"2024-04-25T10:05:11.459177+0000","last_fullsized":"2024-04-25T10:05:11.459177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456188+0000","last_change":"2024-04-25T10:05:11.456178+0000","last_active":"2024-04-25T10:05:12.456188+0000","last_peered":"2024-04-25T10:05:12.456188+0000","last_clean":"2024-04-25T10:05:12.456188+0000","last_became_active":"2024-04-25T10:05:11.455879+0000","last_became_peered":"2024-04-25T10:05:11.455879+0000","last_unstale":"2024-04-25T10:05:12.456188+0000","last_undegraded":"2024-04-25T10:05:12.456188+0000","last_fullsized":"2024-04-25T10:05:12.456188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:55.300183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456275+0000","last_change":"2024-04-25T10:05:10.455684+0000","last_active":"2024-04-25T10:05:12.456275+0000","last_peered":"2024-04-25T10:05:12.456275+0000","last_clean":"2024-04-25T10:05:12.456275+0000","last_became_active":"2024-04-25T10:05:10.455342+0000","last_became_peered":"2024-04-25T10:05:10.455342+0000","last_unstale":"2024-04-25T10:05:12.456275+0000","last_undegraded":"2024-04-25T10:05:12.456275+0000","last_fullsized":"2024-04-25T10:05:12.456275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:01.556228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469903+0000","last_change":"2024-04-25T10:05:11.466622+0000","last_active":"2024-04-25T10:05:11.469903+0000","last_peered":"2024-04-25T10:05:11.469903+0000","last_clean":"2024-04-25T10:05:11.469903+0000","last_became_active":"2024-04-25T10:05:11.466346+0000","last_became_peered":"2024-04-25T10:05:11.466346+0000","last_unstale":"2024-04-25T10:05:11.469903+0000","last_undegraded":"2024-04-25T10:05:11.469903+0000","last_fullsized":"2024-04-25T10:05:11.469903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.468144+0000","last_change":"2024-04-25T10:05:10.457614+0000","last_active":"2024-04-25T10:05:11.468144+0000","last_peered":"2024-04-25T10:05:11.468144+0000","last_clean":"2024-04-25T10:05:11.468144+0000","last_became_active":"2024-04-25T10:05:10.457332+0000","last_became_peered":"2024-04-25T10:05:10.457332+0000","last_unstale":"2024-04-25T10:05:11.468144+0000","last_undegraded":"2024-04-25T10:05:11.468144+0000","last_fullsized":"2024-04-25T10:05:11.468144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:01.556228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455780+0000","last_change":"2024-04-25T10:05:11.460589+0000","last_active":"2024-04-25T10:05:12.455780+0000","last_peered":"2024-04-25T10:05:12.455780+0000","last_clean":"2024-04-25T10:05:12.455780+0000","last_became_active":"2024-04-25T10:05:11.459433+0000","last_became_peered":"2024-04-25T10:05:11.459433+0000","last_unstale":"2024-04-25T10:05:12.455780+0000","last_undegraded":"2024-04-25T10:05:12.455780+0000","last_fullsized":"2024-04-25T10:05:12.455780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:55.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456910+0000","last_change":"2024-04-25T10:05:10.458692+0000","last_active":"2024-04-25T10:05:12.456910+0000","last_peered":"2024-04-25T10:05:12.456910+0000","last_clean":"2024-04-25T10:05:12.456910+0000","last_became_active":"2024-04-25T10:05:10.458090+0000","last_became_peered":"2024-04-25T10:05:10.458090+0000","last_unstale":"2024-04-25T10:05:12.456910+0000","last_undegraded":"2024-04-25T10:05:12.456910+0000","last_fullsized":"2024-04-25T10:05:12.456910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:23.585636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.468810+0000","last_change":"2024-04-25T10:05:11.464532+0000","last_active":"2024-04-25T10:05:11.468810+0000","last_peered":"2024-04-25T10:05:11.468810+0000","last_clean":"2024-04-25T10:05:11.468810+0000","last_became_active":"2024-04-25T10:05:11.464186+0000","last_became_peered":"2024-04-25T10:05:11.464186+0000","last_unstale":"2024-04-25T10:05:11.468810+0000","last_undegraded":"2024-04-25T10:05:11.468810+0000","last_fullsized":"2024-04-25T10:05:11.468810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:55.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786543+0000","last_change":"2024-04-25T10:05:10.460107+0000","last_active":"2024-04-25T10:05:14.786543+0000","last_peered":"2024-04-25T10:05:14.786543+0000","last_clean":"2024-04-25T10:05:14.786543+0000","last_became_active":"2024-04-25T10:05:10.459754+0000","last_became_peered":"2024-04-25T10:05:10.459754+0000","last_unstale":"2024-04-25T10:05:14.786543+0000","last_undegraded":"2024-04-25T10:05:14.786543+0000","last_fullsized":"2024-04-25T10:05:14.786543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:48.214248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.475897+0000","last_change":"2024-04-25T10:05:11.474519+0000","last_active":"2024-04-25T10:05:11.475897+0000","last_peered":"2024-04-25T10:05:11.475897+0000","last_clean":"2024-04-25T10:05:11.475897+0000","last_became_active":"2024-04-25T10:05:11.472675+0000","last_became_peered":"2024-04-25T10:05:11.472675+0000","last_unstale":"2024-04-25T10:05:11.475897+0000","last_undegraded":"2024-04-25T10:05:11.475897+0000","last_fullsized":"2024-04-25T10:05:11.475897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.799101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.458209+0000","last_change":"2024-04-25T10:05:10.453364+0000","last_active":"2024-04-25T10:05:11.458209+0000","last_peered":"2024-04-25T10:05:11.458209+0000","last_clean":"2024-04-25T10:05:11.458209+0000","last_became_active":"2024-04-25T10:05:10.453072+0000","last_became_peered":"2024-04-25T10:05:10.453072+0000","last_unstale":"2024-04-25T10:05:11.458209+0000","last_undegraded":"2024-04-25T10:05:11.458209+0000","last_fullsized":"2024-04-25T10:05:11.458209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:53.640804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469818+0000","last_change":"2024-04-25T10:05:11.467814+0000","last_active":"2024-04-25T10:05:11.469818+0000","last_peered":"2024-04-25T10:05:11.469818+0000","last_clean":"2024-04-25T10:05:11.469818+0000","last_became_active":"2024-04-25T10:05:11.467496+0000","last_became_peered":"2024-04-25T10:05:11.467496+0000","last_unstale":"2024-04-25T10:05:11.469818+0000","last_undegraded":"2024-04-25T10:05:11.469818+0000","last_fullsized":"2024-04-25T10:05:11.469818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.799101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456108+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:05:12.456108+0000","last_peered":"2024-04-25T10:05:12.456108+0000","last_clean":"2024-04-25T10:05:12.456108+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:05:12.456108+0000","last_undegraded":"2024-04-25T10:05:12.456108+0000","last_fullsized":"2024-04-25T10:05:12.456108+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.452172+0000","last_change":"2024-04-25T10:05:10.458170+0000","last_active":"2024-04-25T10:05:11.452172+0000","last_peered":"2024-04-25T10:05:11.452172+0000","last_clean":"2024-04-25T10:05:11.452172+0000","last_became_active":"2024-04-25T10:05:10.457794+0000","last_became_peered":"2024-04-25T10:05:10.457794+0000","last_unstale":"2024-04-25T10:05:11.452172+0000","last_undegraded":"2024-04-25T10:05:11.452172+0000","last_fullsized":"2024-04-25T10:05:11.452172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:11.920992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456600+0000","last_change":"2024-04-25T10:05:11.462574+0000","last_active":"2024-04-25T10:05:12.456600+0000","last_peered":"2024-04-25T10:05:12.456600+0000","last_clean":"2024-04-25T10:05:12.456600+0000","last_became_active":"2024-04-25T10:05:11.461617+0000","last_became_peered":"2024-04-25T10:05:11.461617+0000","last_unstale":"2024-04-25T10:05:12.456600+0000","last_undegraded":"2024-04-25T10:05:12.456600+0000","last_fullsized":"2024-04-25T10:05:12.456600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.451455+0000","last_change":"2024-04-25T10:05:10.464334+0000","last_active":"2024-04-25T10:05:11.451455+0000","last_peered":"2024-04-25T10:05:11.451455+0000","last_clean":"2024-04-25T10:05:11.451455+0000","last_became_active":"2024-04-25T10:05:10.463991+0000","last_became_peered":"2024-04-25T10:05:10.463991+0000","last_unstale":"2024-04-25T10:05:11.451455+0000","last_undegraded":"2024-04-25T10:05:11.451455+0000","last_fullsized":"2024-04-25T10:05:11.451455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469159+0000","last_change":"2024-04-25T10:05:11.464866+0000","last_active":"2024-04-25T10:05:11.469159+0000","last_peered":"2024-04-25T10:05:11.469159+0000","last_clean":"2024-04-25T10:05:11.469159+0000","last_became_active":"2024-04-25T10:05:11.464504+0000","last_became_peered":"2024-04-25T10:05:11.464504+0000","last_unstale":"2024-04-25T10:05:11.469159+0000","last_undegraded":"2024-04-25T10:05:11.469159+0000","last_fullsized":"2024-04-25T10:05:11.469159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.449849+0000","last_change":"2024-04-25T10:05:10.458220+0000","last_active":"2024-04-25T10:05:11.449849+0000","last_peered":"2024-04-25T10:05:11.449849+0000","last_clean":"2024-04-25T10:05:11.449849+0000","last_became_active":"2024-04-25T10:05:10.457936+0000","last_became_peered":"2024-04-25T10:05:10.457936+0000","last_unstale":"2024-04-25T10:05:11.449849+0000","last_undegraded":"2024-04-25T10:05:11.449849+0000","last_fullsized":"2024-04-25T10:05:11.449849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:31.375084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786598+0000","last_change":"2024-04-25T10:05:11.457665+0000","last_active":"2024-04-25T10:05:14.786598+0000","last_peered":"2024-04-25T10:05:14.786598+0000","last_clean":"2024-04-25T10:05:14.786598+0000","last_became_active":"2024-04-25T10:05:11.455078+0000","last_became_peered":"2024-04-25T10:05:11.455078+0000","last_unstale":"2024-04-25T10:05:14.786598+0000","last_undegraded":"2024-04-25T10:05:14.786598+0000","last_fullsized":"2024-04-25T10:05:14.786598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:21.453409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.451883+0000","last_change":"2024-04-25T10:05:10.463388+0000","last_active":"2024-04-25T10:05:11.451883+0000","last_peered":"2024-04-25T10:05:11.451883+0000","last_clean":"2024-04-25T10:05:11.451883+0000","last_became_active":"2024-04-25T10:05:10.463041+0000","last_became_peered":"2024-04-25T10:05:10.463041+0000","last_unstale":"2024-04-25T10:05:11.451883+0000","last_undegraded":"2024-04-25T10:05:11.451883+0000","last_fullsized":"2024-04-25T10:05:11.451883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:13.500977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.463844+0000","last_change":"2024-04-25T10:05:11.462360+0000","last_active":"2024-04-25T10:05:11.463844+0000","last_peered":"2024-04-25T10:05:11.463844+0000","last_clean":"2024-04-25T10:05:11.463844+0000","last_became_active":"2024-04-25T10:05:11.462050+0000","last_became_peered":"2024-04-25T10:05:11.462050+0000","last_unstale":"2024-04-25T10:05:11.463844+0000","last_undegraded":"2024-04-25T10:05:11.463844+0000","last_fullsized":"2024-04-25T10:05:11.463844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:14.506306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786571+0000","last_change":"2024-04-25T10:05:10.456403+0000","last_active":"2024-04-25T10:05:14.786571+0000","last_peered":"2024-04-25T10:05:14.786571+0000","last_clean":"2024-04-25T10:05:14.786571+0000","last_became_active":"2024-04-25T10:05:10.456060+0000","last_became_peered":"2024-04-25T10:05:10.456060+0000","last_unstale":"2024-04-25T10:05:14.786571+0000","last_undegraded":"2024-04-25T10:05:14.786571+0000","last_fullsized":"2024-04-25T10:05:14.786571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:50.107731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455433+0000","last_change":"2024-04-25T10:05:11.464184+0000","last_active":"2024-04-25T10:05:12.455433+0000","last_peered":"2024-04-25T10:05:12.455433+0000","last_clean":"2024-04-25T10:05:12.455433+0000","last_became_active":"2024-04-25T10:05:11.463894+0000","last_became_peered":"2024-04-25T10:05:11.463894+0000","last_unstale":"2024-04-25T10:05:12.455433+0000","last_undegraded":"2024-04-25T10:05:12.455433+0000","last_fullsized":"2024-04-25T10:05:12.455433+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.452412+0000","last_change":"2024-04-25T10:05:10.460261+0000","last_active":"2024-04-25T10:05:11.452412+0000","last_peered":"2024-04-25T10:05:11.452412+0000","last_clean":"2024-04-25T10:05:11.452412+0000","last_became_active":"2024-04-25T10:05:10.459949+0000","last_became_peered":"2024-04-25T10:05:10.459949+0000","last_unstale":"2024-04-25T10:05:11.452412+0000","last_undegraded":"2024-04-25T10:05:11.452412+0000","last_fullsized":"2024-04-25T10:05:11.452412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:29.784640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.463941+0000","last_change":"2024-04-25T10:05:11.458748+0000","last_active":"2024-04-25T10:05:11.463941+0000","last_peered":"2024-04-25T10:05:11.463941+0000","last_clean":"2024-04-25T10:05:11.463941+0000","last_became_active":"2024-04-25T10:05:11.458440+0000","last_became_peered":"2024-04-25T10:05:11.458440+0000","last_unstale":"2024-04-25T10:05:11.463941+0000","last_undegraded":"2024-04-25T10:05:11.463941+0000","last_fullsized":"2024-04-25T10:05:11.463941+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:55.300183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456842+0000","last_change":"2024-04-25T10:05:10.455250+0000","last_active":"2024-04-25T10:05:12.456842+0000","last_peered":"2024-04-25T10:05:12.456842+0000","last_clean":"2024-04-25T10:05:12.456842+0000","last_became_active":"2024-04-25T10:05:10.454833+0000","last_became_peered":"2024-04-25T10:05:10.454833+0000","last_unstale":"2024-04-25T10:05:12.456842+0000","last_undegraded":"2024-04-25T10:05:12.456842+0000","last_fullsized":"2024-04-25T10:05:12.456842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.457011+0000","last_change":"2024-04-25T10:05:11.466533+0000","last_active":"2024-04-25T10:05:12.457011+0000","last_peered":"2024-04-25T10:05:12.457011+0000","last_clean":"2024-04-25T10:05:12.457011+0000","last_became_active":"2024-04-25T10:05:11.466256+0000","last_became_peered":"2024-04-25T10:05:11.466256+0000","last_unstale":"2024-04-25T10:05:12.457011+0000","last_undegraded":"2024-04-25T10:05:12.457011+0000","last_fullsized":"2024-04-25T10:05:12.457011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:41.766168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.476424+0000","last_change":"2024-04-25T10:05:10.462808+0000","last_active":"2024-04-25T10:05:11.476424+0000","last_peered":"2024-04-25T10:05:11.476424+0000","last_clean":"2024-04-25T10:05:11.476424+0000","last_became_active":"2024-04-25T10:05:10.462421+0000","last_became_peered":"2024-04-25T10:05:10.462421+0000","last_unstale":"2024-04-25T10:05:11.476424+0000","last_undegraded":"2024-04-25T10:05:11.476424+0000","last_fullsized":"2024-04-25T10:05:11.476424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:55.136701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456680+0000","last_change":"2024-04-25T10:05:11.470381+0000","last_active":"2024-04-25T10:05:12.456680+0000","last_peered":"2024-04-25T10:05:12.456680+0000","last_clean":"2024-04-25T10:05:12.456680+0000","last_became_active":"2024-04-25T10:05:11.470031+0000","last_became_peered":"2024-04-25T10:05:11.470031+0000","last_unstale":"2024-04-25T10:05:12.456680+0000","last_undegraded":"2024-04-25T10:05:12.456680+0000","last_fullsized":"2024-04-25T10:05:12.456680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.799101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.454513+0000","last_change":"2024-04-25T10:05:10.461232+0000","last_active":"2024-04-25T10:05:11.454513+0000","last_peered":"2024-04-25T10:05:11.454513+0000","last_clean":"2024-04-25T10:05:11.454513+0000","last_became_active":"2024-04-25T10:05:10.460905+0000","last_became_peered":"2024-04-25T10:05:10.460905+0000","last_unstale":"2024-04-25T10:05:11.454513+0000","last_undegraded":"2024-04-25T10:05:11.454513+0000","last_fullsized":"2024-04-25T10:05:11.454513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:54.294853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474543+0000","last_change":"2024-04-25T10:05:11.472902+0000","last_active":"2024-04-25T10:05:11.474543+0000","last_peered":"2024-04-25T10:05:11.474543+0000","last_clean":"2024-04-25T10:05:11.474543+0000","last_became_active":"2024-04-25T10:05:11.472622+0000","last_became_peered":"2024-04-25T10:05:11.472622+0000","last_unstale":"2024-04-25T10:05:11.474543+0000","last_undegraded":"2024-04-25T10:05:11.474543+0000","last_fullsized":"2024-04-25T10:05:11.474543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:24.616832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456461+0000","last_change":"2024-04-25T10:05:10.460759+0000","last_active":"2024-04-25T10:05:12.456461+0000","last_peered":"2024-04-25T10:05:12.456461+0000","last_clean":"2024-04-25T10:05:12.456461+0000","last_became_active":"2024-04-25T10:05:10.460485+0000","last_became_peered":"2024-04-25T10:05:10.460485+0000","last_unstale":"2024-04-25T10:05:12.456461+0000","last_undegraded":"2024-04-25T10:05:12.456461+0000","last_fullsized":"2024-04-25T10:05:12.456461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456829+0000","last_change":"2024-04-25T10:05:11.470739+0000","last_active":"2024-04-25T10:05:12.456829+0000","last_peered":"2024-04-25T10:05:12.456829+0000","last_clean":"2024-04-25T10:05:12.456829+0000","last_became_active":"2024-04-25T10:05:11.470408+0000","last_became_peered":"2024-04-25T10:05:11.470408+0000","last_unstale":"2024-04-25T10:05:12.456829+0000","last_undegraded":"2024-04-25T10:05:12.456829+0000","last_fullsized":"2024-04-25T10:05:12.456829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:17.789806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.457380+0000","last_change":"2024-04-25T10:05:10.458496+0000","last_active":"2024-04-25T10:05:11.457380+0000","last_peered":"2024-04-25T10:05:11.457380+0000","last_clean":"2024-04-25T10:05:11.457380+0000","last_became_active":"2024-04-25T10:05:10.458168+0000","last_became_peered":"2024-04-25T10:05:10.458168+0000","last_unstale":"2024-04-25T10:05:11.457380+0000","last_undegraded":"2024-04-25T10:05:11.457380+0000","last_fullsized":"2024-04-25T10:05:11.457380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:41.255007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.461196+0000","last_change":"2024-04-25T10:05:11.459826+0000","last_active":"2024-04-25T10:05:11.461196+0000","last_peered":"2024-04-25T10:05:11.461196+0000","last_clean":"2024-04-25T10:05:11.461196+0000","last_became_active":"2024-04-25T10:05:11.459570+0000","last_became_peered":"2024-04-25T10:05:11.459570+0000","last_unstale":"2024-04-25T10:05:11.461196+0000","last_undegraded":"2024-04-25T10:05:11.461196+0000","last_fullsized":"2024-04-25T10:05:11.461196+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:54.646134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474457+0000","last_change":"2024-04-25T10:05:10.455137+0000","last_active":"2024-04-25T10:05:11.474457+0000","last_peered":"2024-04-25T10:05:11.474457+0000","last_clean":"2024-04-25T10:05:11.474457+0000","last_became_active":"2024-04-25T10:05:10.454827+0000","last_became_peered":"2024-04-25T10:05:10.454827+0000","last_unstale":"2024-04-25T10:05:11.474457+0000","last_undegraded":"2024-04-25T10:05:11.474457+0000","last_fullsized":"2024-04-25T10:05:11.474457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:54.691904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.470977+0000","last_change":"2024-04-25T10:05:11.469441+0000","last_active":"2024-04-25T10:05:11.470977+0000","last_peered":"2024-04-25T10:05:11.470977+0000","last_clean":"2024-04-25T10:05:11.470977+0000","last_became_active":"2024-04-25T10:05:11.468649+0000","last_became_peered":"2024-04-25T10:05:11.468649+0000","last_unstale":"2024-04-25T10:05:11.470977+0000","last_undegraded":"2024-04-25T10:05:11.470977+0000","last_fullsized":"2024-04-25T10:05:11.470977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:17.789806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469799+0000","last_change":"2024-04-25T10:05:11.465690+0000","last_active":"2024-04-25T10:05:11.469799+0000","last_peered":"2024-04-25T10:05:11.469799+0000","last_clean":"2024-04-25T10:05:11.469799+0000","last_became_active":"2024-04-25T10:05:11.465422+0000","last_became_peered":"2024-04-25T10:05:11.465422+0000","last_unstale":"2024-04-25T10:05:11.469799+0000","last_undegraded":"2024-04-25T10:05:11.469799+0000","last_fullsized":"2024-04-25T10:05:11.469799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472427+0000","last_change":"2024-04-25T10:05:10.450133+0000","last_active":"2024-04-25T10:05:11.472427+0000","last_peered":"2024-04-25T10:05:11.472427+0000","last_clean":"2024-04-25T10:05:11.472427+0000","last_became_active":"2024-04-25T10:05:10.449482+0000","last_became_peered":"2024-04-25T10:05:10.449482+0000","last_unstale":"2024-04-25T10:05:11.472427+0000","last_undegraded":"2024-04-25T10:05:11.472427+0000","last_fullsized":"2024-04-25T10:05:11.472427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456402+0000","last_change":"2024-04-25T10:05:11.469301+0000","last_active":"2024-04-25T10:05:12.456402+0000","last_peered":"2024-04-25T10:05:12.456402+0000","last_clean":"2024-04-25T10:05:12.456402+0000","last_became_active":"2024-04-25T10:05:11.468980+0000","last_became_peered":"2024-04-25T10:05:11.468980+0000","last_unstale":"2024-04-25T10:05:12.456402+0000","last_undegraded":"2024-04-25T10:05:12.456402+0000","last_fullsized":"2024-04-25T10:05:12.456402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471641+0000","last_change":"2024-04-25T10:05:10.456749+0000","last_active":"2024-04-25T10:05:11.471641+0000","last_peered":"2024-04-25T10:05:11.471641+0000","last_clean":"2024-04-25T10:05:11.471641+0000","last_became_active":"2024-04-25T10:05:10.456397+0000","last_became_peered":"2024-04-25T10:05:10.456397+0000","last_unstale":"2024-04-25T10:05:11.471641+0000","last_undegraded":"2024-04-25T10:05:11.471641+0000","last_fullsized":"2024-04-25T10:05:11.471641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786907+0000","last_change":"2024-04-25T10:05:11.453961+0000","last_active":"2024-04-25T10:05:14.786907+0000","last_peered":"2024-04-25T10:05:14.786907+0000","last_clean":"2024-04-25T10:05:14.786907+0000","last_became_active":"2024-04-25T10:05:11.453443+0000","last_became_peered":"2024-04-25T10:05:11.453443+0000","last_unstale":"2024-04-25T10:05:14.786907+0000","last_undegraded":"2024-04-25T10:05:14.786907+0000","last_fullsized":"2024-04-25T10:05:14.786907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:06.235586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786886+0000","last_change":"2024-04-25T10:05:10.458501+0000","last_active":"2024-04-25T10:05:14.786886+0000","last_peered":"2024-04-25T10:05:14.786886+0000","last_clean":"2024-04-25T10:05:14.786886+0000","last_became_active":"2024-04-25T10:05:10.457915+0000","last_became_peered":"2024-04-25T10:05:10.457915+0000","last_unstale":"2024-04-25T10:05:14.786886+0000","last_undegraded":"2024-04-25T10:05:14.786886+0000","last_fullsized":"2024-04-25T10:05:14.786886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:15.560541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786839+0000","last_change":"2024-04-25T10:05:11.451812+0000","last_active":"2024-04-25T10:05:14.786839+0000","last_peered":"2024-04-25T10:05:14.786839+0000","last_clean":"2024-04-25T10:05:14.786839+0000","last_became_active":"2024-04-25T10:05:11.451318+0000","last_became_peered":"2024-04-25T10:05:11.451318+0000","last_unstale":"2024-04-25T10:05:14.786839+0000","last_undegraded":"2024-04-25T10:05:14.786839+0000","last_fullsized":"2024-04-25T10:05:14.786839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:28.693266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.457002+0000","last_change":"2024-04-25T10:05:10.454280+0000","last_active":"2024-04-25T10:05:12.457002+0000","last_peered":"2024-04-25T10:05:12.457002+0000","last_clean":"2024-04-25T10:05:12.457002+0000","last_became_active":"2024-04-25T10:05:10.453982+0000","last_became_peered":"2024-04-25T10:05:10.453982+0000","last_unstale":"2024-04-25T10:05:12.457002+0000","last_undegraded":"2024-04-25T10:05:12.457002+0000","last_fullsized":"2024-04-25T10:05:12.457002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":72548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":70846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":71876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":667002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":70320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":660724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:18.870 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:18.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.978+0000 7f80e7ee1640 1 Processor -- start 2024-04-25T10:05:18.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.978+0000 7f80e7ee1640 1 -- start start 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.979+0000 7f80e7ee1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80e015dce0 0x7f80e00d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.979+0000 7f80e7ee1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80e00b5b60 0x7f80e00b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.979+0000 7f80e7ee1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e015e0c0 con 0x7f80e00d35b0 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.979+0000 7f80e7ee1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e00d2510 con 0x7f80e015dce0 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.979+0000 7f80e7ee1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e00d27f0 con 0x7f80e00b5b60 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.979+0000 7f80e5c7d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80e015dce0 0x7f80e00d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.979+0000 7f80e547c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f80e00d35b0 0x7f80e00b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34798/0 (socket says 172.21.15.16:34798) 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.979+0000 7f80e547c640 1 -- 172.21.15.16:0/2271723606 learned_addr learned my addr 172.21.15.16:0/2271723606 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.979+0000 7f80e647e640 1 --2- 172.21.15.16:0/2271723606 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80e00b5b60 0x7f80e00b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.980+0000 7f80e5c7d640 1 -- 172.21.15.16:0/2271723606 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80e00b5b60 msgr2=0x7f80e00b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.980+0000 7f80e5c7d640 1 --2- 172.21.15.16:0/2271723606 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80e00b5b60 0x7f80e00b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.980+0000 7f80e5c7d640 1 -- 172.21.15.16:0/2271723606 >> v1:172.21.15.16:6789/0 conn(0x7f80e00d35b0 legacy=0x7f80e00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.980+0000 7f80e5c7d640 1 -- 172.21.15.16:0/2271723606 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e00d2ad0 con 0x7f80e015dce0 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.980+0000 7f80e5c7d640 1 --2- 172.21.15.16:0/2271723606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e015dce0 0x7f80e00d1a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0097a0 tx=0x7f80dc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81989b968dc0615 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.980+0000 7f80e4c7b640 1 -- 172.21.15.16:0/2271723606 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc037070 con 0x7f80e015dce0 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.980+0000 7f80e4c7b640 1 -- 172.21.15.16:0/2271723606 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80dc032440 con 0x7f80e015dce0 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.981+0000 7f80e4c7b640 1 -- 172.21.15.16:0/2271723606 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc032810 con 0x7f80e015dce0 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.981+0000 7f80e7ee1640 1 -- 172.21.15.16:0/2271723606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e015dce0 msgr2=0x7f80e00d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.981+0000 7f80e7ee1640 1 --2- 172.21.15.16:0/2271723606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e015dce0 0x7f80e00d1a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0097a0 tx=0x7f80dc02a180 comp rx=0 tx=0).stop 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.981+0000 7f80e7ee1640 1 -- 172.21.15.16:0/2271723606 shutdown_connections 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.981+0000 7f80e7ee1640 1 --2- 172.21.15.16:0/2271723606 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80e00b5b60 0x7f80e00b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.981+0000 7f80e7ee1640 1 --2- 172.21.15.16:0/2271723606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e015dce0 0x7f80e00d1a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.981+0000 7f80e7ee1640 1 -- 172.21.15.16:0/2271723606 >> 172.21.15.16:0/2271723606 conn(0x7f80e0072900 msgr2=0x7f80e00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.981+0000 7f80e7ee1640 1 -- 172.21.15.16:0/2271723606 shutdown_connections 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.981+0000 7f80e7ee1640 1 -- 172.21.15.16:0/2271723606 wait complete. 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.982+0000 7f80e7ee1640 1 Processor -- start 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.982+0000 7f80e7ee1640 1 -- start start 2024-04-25T10:05:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.983+0000 7f80e7ee1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e00b5b60 0x7f80e014dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.983+0000 7f80e7ee1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80e00d35b0 0x7f80e01f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.983+0000 7f80e7ee1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80e015dce0 0x7f80e00b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.983+0000 7f80e7ee1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e00b9820 con 0x7f80e00b5b60 2024-04-25T10:05:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.983+0000 7f80e7ee1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e00d3090 con 0x7f80e00d35b0 2024-04-25T10:05:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.983+0000 7f80e7ee1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e00b99a0 con 0x7f80e015dce0 2024-04-25T10:05:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.983+0000 7f80e5c7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e00b5b60 0x7f80e014dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.983+0000 7f80e5c7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e00b5b60 0x7f80e014dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41874/0 (socket says 172.21.15.16:41874) 2024-04-25T10:05:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.983+0000 7f80e5c7d640 1 -- 172.21.15.16:0/588173973 learned_addr learned my addr 172.21.15.16:0/588173973 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.983+0000 7f80e647e640 1 --2- 172.21.15.16:0/588173973 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80e015dce0 0x7f80e00b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.983+0000 7f80e547c640 1 --2- 172.21.15.16:0/588173973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80e00d35b0 0x7f80e01f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.983+0000 7f80e5c7d640 1 -- 172.21.15.16:0/588173973 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80e015dce0 msgr2=0x7f80e00b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.983+0000 7f80e5c7d640 1 --2- 172.21.15.16:0/588173973 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80e015dce0 0x7f80e00b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.983+0000 7f80e5c7d640 1 -- 172.21.15.16:0/588173973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80e00d35b0 msgr2=0x7f80e01f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.983+0000 7f80e5c7d640 1 --2- 172.21.15.16:0/588173973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80e00d35b0 0x7f80e01f39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.984+0000 7f80e5c7d640 1 -- 172.21.15.16:0/588173973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e0161070 con 0x7f80e00b5b60 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.984+0000 7f80e547c640 1 --2- 172.21.15.16:0/588173973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80e00d35b0 0x7f80e01f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.984+0000 7f80e5c7d640 1 --2- 172.21.15.16:0/588173973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e00b5b60 0x7f80e014dce0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f80dc035040 tx=0x7f80dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.984+0000 7f80d67fc640 1 -- 172.21.15.16:0/588173973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc037040 con 0x7f80e00b5b60 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.984+0000 7f80d67fc640 1 -- 172.21.15.16:0/588173973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80dc030770 con 0x7f80e00b5b60 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.984+0000 7f80e7ee1640 1 -- 172.21.15.16:0/588173973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e00c8dc0 con 0x7f80e00b5b60 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.984+0000 7f80d67fc640 1 -- 172.21.15.16:0/588173973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc030b80 con 0x7f80e00b5b60 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.984+0000 7f80e7ee1640 1 -- 172.21.15.16:0/588173973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e00c9380 con 0x7f80e00b5b60 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.985+0000 7f80e7ee1640 1 -- 172.21.15.16:0/588173973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e00749b0 con 0x7f80e00b5b60 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.985+0000 7f80d67fc640 1 -- 172.21.15.16:0/588173973 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f80dc044020 con 0x7f80e00b5b60 2024-04-25T10:05:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.986+0000 7f80d67fc640 1 --2- 172.21.15.16:0/588173973 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f80c406b4d0 0x7f80c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.986+0000 7f80d67fc640 1 -- 172.21.15.16:0/588173973 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80dc0aa6e0 con 0x7f80e00b5b60 2024-04-25T10:05:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.986+0000 7f80e547c640 1 --2- 172.21.15.16:0/588173973 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f80c406b4d0 0x7f80c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.987+0000 7f80e547c640 1 --2- 172.21.15.16:0/588173973 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f80c406b4d0 0x7f80c406d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f80e00b7d90 tx=0x7f80d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:18.990+0000 7f80d67fc640 1 -- 172.21.15.16:0/588173973 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f80dc032e20 con 0x7f80e00b5b60 2024-04-25T10:05:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.166+0000 7f80e7ee1640 1 -- 172.21.15.16:0/588173973 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f80e00b7a00 con 0x7f80c406b4d0 2024-04-25T10:05:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.173+0000 7f80d67fc640 1 -- 172.21.15.16:0/588173973 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251906 (secure 0 0 0) 0x7f80e00b7a00 con 0x7f80c406b4d0 2024-04-25T10:05:19.180 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:19.182 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:05:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.176+0000 7f80e7ee1640 1 -- 172.21.15.16:0/588173973 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f80c406b4d0 msgr2=0x7f80c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.176+0000 7f80e7ee1640 1 --2- 172.21.15.16:0/588173973 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f80c406b4d0 0x7f80c406d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f80e00b7d90 tx=0x7f80d0009040 comp rx=0 tx=0).stop 2024-04-25T10:05:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.176+0000 7f80e7ee1640 1 -- 172.21.15.16:0/588173973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e00b5b60 msgr2=0x7f80e014dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.176+0000 7f80e7ee1640 1 --2- 172.21.15.16:0/588173973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e00b5b60 0x7f80e014dce0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f80dc035040 tx=0x7f80dc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.176+0000 7f80e7ee1640 1 -- 172.21.15.16:0/588173973 shutdown_connections 2024-04-25T10:05:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.176+0000 7f80e7ee1640 1 --2- 172.21.15.16:0/588173973 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f80c406b4d0 0x7f80c406d990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.176+0000 7f80e7ee1640 1 --2- 172.21.15.16:0/588173973 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80e015dce0 0x7f80e00b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.176+0000 7f80e7ee1640 1 --2- 172.21.15.16:0/588173973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80e00d35b0 0x7f80e01f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.176+0000 7f80e7ee1640 1 --2- 172.21.15.16:0/588173973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e00b5b60 0x7f80e014dce0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.176+0000 7f80e7ee1640 1 -- 172.21.15.16:0/588173973 >> 172.21.15.16:0/588173973 conn(0x7f80e0072900 msgr2=0x7f80e00b7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.176+0000 7f80e7ee1640 1 -- 172.21.15.16:0/588173973 shutdown_connections 2024-04-25T10:05:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.176+0000 7f80e7ee1640 1 -- 172.21.15.16:0/588173973 wait complete. 2024-04-25T10:05:19.192 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-04-25T10:05:17.782865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235672,"kb_used_data":2868,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739048,"statfs":{"total":773094113280,"available":772852785152,"internally_reserved":0,"allocated":2936832,"data_stored":1753968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.342370"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.475316+0000","last_change":"2024-04-25T10:05:11.473253+0000","last_active":"2024-04-25T10:05:11.475316+0000","last_peered":"2024-04-25T10:05:11.475316+0000","last_clean":"2024-04-25T10:05:11.475316+0000","last_became_active":"2024-04-25T10:05:11.472940+0000","last_became_peered":"2024-04-25T10:05:11.472940+0000","last_unstale":"2024-04-25T10:05:11.475316+0000","last_undegraded":"2024-04-25T10:05:11.475316+0000","last_fullsized":"2024-04-25T10:05:11.475316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:52.014434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786817+0000","last_change":"2024-04-25T10:05:11.460499+0000","last_active":"2024-04-25T10:05:14.786817+0000","last_peered":"2024-04-25T10:05:14.786817+0000","last_clean":"2024-04-25T10:05:14.786817+0000","last_became_active":"2024-04-25T10:05:11.458776+0000","last_became_peered":"2024-04-25T10:05:11.458776+0000","last_unstale":"2024-04-25T10:05:14.786817+0000","last_undegraded":"2024-04-25T10:05:14.786817+0000","last_fullsized":"2024-04-25T10:05:14.786817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:17.247685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786678+0000","last_change":"2024-04-25T10:05:11.466721+0000","last_active":"2024-04-25T10:05:14.786678+0000","last_peered":"2024-04-25T10:05:14.786678+0000","last_clean":"2024-04-25T10:05:14.786678+0000","last_became_active":"2024-04-25T10:05:11.466430+0000","last_became_peered":"2024-04-25T10:05:11.466430+0000","last_unstale":"2024-04-25T10:05:14.786678+0000","last_undegraded":"2024-04-25T10:05:14.786678+0000","last_fullsized":"2024-04-25T10:05:14.786678+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:34.227544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455508+0000","last_change":"2024-04-25T10:05:11.461498+0000","last_active":"2024-04-25T10:05:12.455508+0000","last_peered":"2024-04-25T10:05:12.455508+0000","last_clean":"2024-04-25T10:05:12.455508+0000","last_became_active":"2024-04-25T10:05:11.461241+0000","last_became_peered":"2024-04-25T10:05:11.461241+0000","last_unstale":"2024-04-25T10:05:12.455508+0000","last_undegraded":"2024-04-25T10:05:12.455508+0000","last_fullsized":"2024-04-25T10:05:12.455508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472722+0000","last_change":"2024-04-25T10:05:11.469281+0000","last_active":"2024-04-25T10:05:11.472722+0000","last_peered":"2024-04-25T10:05:11.472722+0000","last_clean":"2024-04-25T10:05:11.472722+0000","last_became_active":"2024-04-25T10:05:11.468809+0000","last_became_peered":"2024-04-25T10:05:11.468809+0000","last_unstale":"2024-04-25T10:05:11.472722+0000","last_undegraded":"2024-04-25T10:05:11.472722+0000","last_fullsized":"2024-04-25T10:05:11.472722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:30.789969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455529+0000","last_change":"2024-04-25T10:05:11.466531+0000","last_active":"2024-04-25T10:05:12.455529+0000","last_peered":"2024-04-25T10:05:12.455529+0000","last_clean":"2024-04-25T10:05:12.455529+0000","last_became_active":"2024-04-25T10:05:11.466126+0000","last_became_peered":"2024-04-25T10:05:11.466126+0000","last_unstale":"2024-04-25T10:05:12.455529+0000","last_undegraded":"2024-04-25T10:05:12.455529+0000","last_fullsized":"2024-04-25T10:05:12.455529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469389+0000","last_change":"2024-04-25T10:05:11.466336+0000","last_active":"2024-04-25T10:05:11.469389+0000","last_peered":"2024-04-25T10:05:11.469389+0000","last_clean":"2024-04-25T10:05:11.469389+0000","last_became_active":"2024-04-25T10:05:11.466032+0000","last_became_peered":"2024-04-25T10:05:11.466032+0000","last_unstale":"2024-04-25T10:05:11.469389+0000","last_undegraded":"2024-04-25T10:05:11.469389+0000","last_fullsized":"2024-04-25T10:05:11.469389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:12.926321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786457+0000","last_change":"2024-04-25T10:05:11.461892+0000","last_active":"2024-04-25T10:05:14.786457+0000","last_peered":"2024-04-25T10:05:14.786457+0000","last_clean":"2024-04-25T10:05:14.786457+0000","last_became_active":"2024-04-25T10:05:11.461264+0000","last_became_peered":"2024-04-25T10:05:11.461264+0000","last_unstale":"2024-04-25T10:05:14.786457+0000","last_undegraded":"2024-04-25T10:05:14.786457+0000","last_fullsized":"2024-04-25T10:05:14.786457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:58.263921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456654+0000","last_change":"2024-04-25T10:05:11.461997+0000","last_active":"2024-04-25T10:05:12.456654+0000","last_peered":"2024-04-25T10:05:12.456654+0000","last_clean":"2024-04-25T10:05:12.456654+0000","last_became_active":"2024-04-25T10:05:11.461658+0000","last_became_peered":"2024-04-25T10:05:11.461658+0000","last_unstale":"2024-04-25T10:05:12.456654+0000","last_undegraded":"2024-04-25T10:05:12.456654+0000","last_fullsized":"2024-04-25T10:05:12.456654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.463064+0000","last_change":"2024-04-25T10:05:11.461631+0000","last_active":"2024-04-25T10:05:11.463064+0000","last_peered":"2024-04-25T10:05:11.463064+0000","last_clean":"2024-04-25T10:05:11.463064+0000","last_became_active":"2024-04-25T10:05:11.461327+0000","last_became_peered":"2024-04-25T10:05:11.461327+0000","last_unstale":"2024-04-25T10:05:11.463064+0000","last_undegraded":"2024-04-25T10:05:11.463064+0000","last_fullsized":"2024-04-25T10:05:11.463064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.473963+0000","last_change":"2024-04-25T10:05:11.473044+0000","last_active":"2024-04-25T10:05:11.473963+0000","last_peered":"2024-04-25T10:05:11.473963+0000","last_clean":"2024-04-25T10:05:11.473963+0000","last_became_active":"2024-04-25T10:05:11.472747+0000","last_became_peered":"2024-04-25T10:05:11.472747+0000","last_unstale":"2024-04-25T10:05:11.473963+0000","last_undegraded":"2024-04-25T10:05:11.473963+0000","last_fullsized":"2024-04-25T10:05:11.473963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474718+0000","last_change":"2024-04-25T10:05:11.470219+0000","last_active":"2024-04-25T10:05:11.474718+0000","last_peered":"2024-04-25T10:05:11.474718+0000","last_clean":"2024-04-25T10:05:11.474718+0000","last_became_active":"2024-04-25T10:05:11.469066+0000","last_became_peered":"2024-04-25T10:05:11.469066+0000","last_unstale":"2024-04-25T10:05:11.474718+0000","last_undegraded":"2024-04-25T10:05:11.474718+0000","last_fullsized":"2024-04-25T10:05:11.474718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:54.934726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.466367+0000","last_change":"2024-04-25T10:05:11.463262+0000","last_active":"2024-04-25T10:05:11.466367+0000","last_peered":"2024-04-25T10:05:11.466367+0000","last_clean":"2024-04-25T10:05:11.466367+0000","last_became_active":"2024-04-25T10:05:11.462970+0000","last_became_peered":"2024-04-25T10:05:11.462970+0000","last_unstale":"2024-04-25T10:05:11.466367+0000","last_undegraded":"2024-04-25T10:05:11.466367+0000","last_fullsized":"2024-04-25T10:05:11.466367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.464756+0000","last_change":"2024-04-25T10:05:11.458310+0000","last_active":"2024-04-25T10:05:11.464756+0000","last_peered":"2024-04-25T10:05:11.464756+0000","last_clean":"2024-04-25T10:05:11.464756+0000","last_became_active":"2024-04-25T10:05:11.457672+0000","last_became_peered":"2024-04-25T10:05:11.457672+0000","last_unstale":"2024-04-25T10:05:11.464756+0000","last_undegraded":"2024-04-25T10:05:11.464756+0000","last_fullsized":"2024-04-25T10:05:11.464756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:02.561557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786423+0000","last_change":"2024-04-25T10:05:11.461506+0000","last_active":"2024-04-25T10:05:14.786423+0000","last_peered":"2024-04-25T10:05:14.786423+0000","last_clean":"2024-04-25T10:05:14.786423+0000","last_became_active":"2024-04-25T10:05:11.460963+0000","last_became_peered":"2024-04-25T10:05:11.460963+0000","last_unstale":"2024-04-25T10:05:14.786423+0000","last_undegraded":"2024-04-25T10:05:14.786423+0000","last_fullsized":"2024-04-25T10:05:14.786423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:45.096476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471639+0000","last_change":"2024-04-25T10:05:11.470743+0000","last_active":"2024-04-25T10:05:11.471639+0000","last_peered":"2024-04-25T10:05:11.471639+0000","last_clean":"2024-04-25T10:05:11.471639+0000","last_became_active":"2024-04-25T10:05:11.470447+0000","last_became_peered":"2024-04-25T10:05:11.470447+0000","last_unstale":"2024-04-25T10:05:11.471639+0000","last_undegraded":"2024-04-25T10:05:11.471639+0000","last_fullsized":"2024-04-25T10:05:11.471639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:41.766168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.473455+0000","last_change":"2024-04-25T10:05:11.472530+0000","last_active":"2024-04-25T10:05:11.473455+0000","last_peered":"2024-04-25T10:05:11.473455+0000","last_clean":"2024-04-25T10:05:11.473455+0000","last_became_active":"2024-04-25T10:05:11.472164+0000","last_became_peered":"2024-04-25T10:05:11.472164+0000","last_unstale":"2024-04-25T10:05:11.473455+0000","last_undegraded":"2024-04-25T10:05:11.473455+0000","last_fullsized":"2024-04-25T10:05:11.473455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:08.525515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471968+0000","last_change":"2024-04-25T10:05:11.464186+0000","last_active":"2024-04-25T10:05:11.471968+0000","last_peered":"2024-04-25T10:05:11.471968+0000","last_clean":"2024-04-25T10:05:11.471968+0000","last_became_active":"2024-04-25T10:05:11.463925+0000","last_became_peered":"2024-04-25T10:05:11.463925+0000","last_unstale":"2024-04-25T10:05:11.471968+0000","last_undegraded":"2024-04-25T10:05:11.471968+0000","last_fullsized":"2024-04-25T10:05:11.471968+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:55.300183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474686+0000","last_change":"2024-04-25T10:05:11.473554+0000","last_active":"2024-04-25T10:05:11.474686+0000","last_peered":"2024-04-25T10:05:11.474686+0000","last_clean":"2024-04-25T10:05:11.474686+0000","last_became_active":"2024-04-25T10:05:11.473271+0000","last_became_peered":"2024-04-25T10:05:11.473271+0000","last_unstale":"2024-04-25T10:05:11.474686+0000","last_undegraded":"2024-04-25T10:05:11.474686+0000","last_fullsized":"2024-04-25T10:05:11.474686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:35.076762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474740+0000","last_change":"2024-04-25T10:05:11.471477+0000","last_active":"2024-04-25T10:05:11.474740+0000","last_peered":"2024-04-25T10:05:11.474740+0000","last_clean":"2024-04-25T10:05:11.474740+0000","last_became_active":"2024-04-25T10:05:11.471219+0000","last_became_peered":"2024-04-25T10:05:11.471219+0000","last_unstale":"2024-04-25T10:05:11.474740+0000","last_undegraded":"2024-04-25T10:05:11.474740+0000","last_fullsized":"2024-04-25T10:05:11.474740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456655+0000","last_change":"2024-04-25T10:05:11.464410+0000","last_active":"2024-04-25T10:05:12.456655+0000","last_peered":"2024-04-25T10:05:12.456655+0000","last_clean":"2024-04-25T10:05:12.456655+0000","last_became_active":"2024-04-25T10:05:11.464003+0000","last_became_peered":"2024-04-25T10:05:11.464003+0000","last_unstale":"2024-04-25T10:05:12.456655+0000","last_undegraded":"2024-04-25T10:05:12.456655+0000","last_fullsized":"2024-04-25T10:05:12.456655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:52.014434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456342+0000","last_change":"2024-04-25T10:05:11.462487+0000","last_active":"2024-04-25T10:05:12.456342+0000","last_peered":"2024-04-25T10:05:12.456342+0000","last_clean":"2024-04-25T10:05:12.456342+0000","last_became_active":"2024-04-25T10:05:11.462191+0000","last_became_peered":"2024-04-25T10:05:11.462191+0000","last_unstale":"2024-04-25T10:05:12.456342+0000","last_undegraded":"2024-04-25T10:05:12.456342+0000","last_fullsized":"2024-04-25T10:05:12.456342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786682+0000","last_change":"2024-04-25T10:05:11.450886+0000","last_active":"2024-04-25T10:05:14.786682+0000","last_peered":"2024-04-25T10:05:14.786682+0000","last_clean":"2024-04-25T10:05:14.786682+0000","last_became_active":"2024-04-25T10:05:11.450616+0000","last_became_peered":"2024-04-25T10:05:11.450616+0000","last_unstale":"2024-04-25T10:05:14.786682+0000","last_undegraded":"2024-04-25T10:05:14.786682+0000","last_fullsized":"2024-04-25T10:05:14.786682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:47.607490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469823+0000","last_change":"2024-04-25T10:05:11.467406+0000","last_active":"2024-04-25T10:05:11.469823+0000","last_peered":"2024-04-25T10:05:11.469823+0000","last_clean":"2024-04-25T10:05:11.469823+0000","last_became_active":"2024-04-25T10:05:11.467126+0000","last_became_peered":"2024-04-25T10:05:11.467126+0000","last_unstale":"2024-04-25T10:05:11.469823+0000","last_undegraded":"2024-04-25T10:05:11.469823+0000","last_fullsized":"2024-04-25T10:05:11.469823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:52.014434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786879+0000","last_change":"2024-04-25T10:05:11.453135+0000","last_active":"2024-04-25T10:05:14.786879+0000","last_peered":"2024-04-25T10:05:14.786879+0000","last_clean":"2024-04-25T10:05:14.786879+0000","last_became_active":"2024-04-25T10:05:11.452053+0000","last_became_peered":"2024-04-25T10:05:11.452053+0000","last_unstale":"2024-04-25T10:05:14.786879+0000","last_undegraded":"2024-04-25T10:05:14.786879+0000","last_fullsized":"2024-04-25T10:05:14.786879+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:03.147551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455642+0000","last_change":"2024-04-25T10:05:11.469452+0000","last_active":"2024-04-25T10:05:12.455642+0000","last_peered":"2024-04-25T10:05:12.455642+0000","last_clean":"2024-04-25T10:05:12.455642+0000","last_became_active":"2024-04-25T10:05:11.465896+0000","last_became_peered":"2024-04-25T10:05:11.465896+0000","last_unstale":"2024-04-25T10:05:12.455642+0000","last_undegraded":"2024-04-25T10:05:12.455642+0000","last_fullsized":"2024-04-25T10:05:12.455642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:14.506306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.460957+0000","last_change":"2024-04-25T10:05:11.457595+0000","last_active":"2024-04-25T10:05:11.460957+0000","last_peered":"2024-04-25T10:05:11.460957+0000","last_clean":"2024-04-25T10:05:11.460957+0000","last_became_active":"2024-04-25T10:05:11.457320+0000","last_became_peered":"2024-04-25T10:05:11.457320+0000","last_unstale":"2024-04-25T10:05:11.460957+0000","last_undegraded":"2024-04-25T10:05:11.460957+0000","last_fullsized":"2024-04-25T10:05:11.460957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:40.831935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472125+0000","last_change":"2024-04-25T10:05:11.469113+0000","last_active":"2024-04-25T10:05:11.472125+0000","last_peered":"2024-04-25T10:05:11.472125+0000","last_clean":"2024-04-25T10:05:11.472125+0000","last_became_active":"2024-04-25T10:05:11.468759+0000","last_became_peered":"2024-04-25T10:05:11.468759+0000","last_unstale":"2024-04-25T10:05:11.472125+0000","last_undegraded":"2024-04-25T10:05:11.472125+0000","last_fullsized":"2024-04-25T10:05:11.472125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786630+0000","last_change":"2024-04-25T10:05:11.457486+0000","last_active":"2024-04-25T10:05:14.786630+0000","last_peered":"2024-04-25T10:05:14.786630+0000","last_clean":"2024-04-25T10:05:14.786630+0000","last_became_active":"2024-04-25T10:05:11.454855+0000","last_became_peered":"2024-04-25T10:05:11.454855+0000","last_unstale":"2024-04-25T10:05:14.786630+0000","last_undegraded":"2024-04-25T10:05:14.786630+0000","last_fullsized":"2024-04-25T10:05:14.786630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:45.863831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471666+0000","last_change":"2024-04-25T10:05:11.470055+0000","last_active":"2024-04-25T10:05:11.471666+0000","last_peered":"2024-04-25T10:05:11.471666+0000","last_clean":"2024-04-25T10:05:11.471666+0000","last_became_active":"2024-04-25T10:05:11.468974+0000","last_became_peered":"2024-04-25T10:05:11.468974+0000","last_unstale":"2024-04-25T10:05:11.471666+0000","last_undegraded":"2024-04-25T10:05:11.471666+0000","last_fullsized":"2024-04-25T10:05:11.471666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.228600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469311+0000","last_change":"2024-04-25T10:05:11.462234+0000","last_active":"2024-04-25T10:05:11.469311+0000","last_peered":"2024-04-25T10:05:11.469311+0000","last_clean":"2024-04-25T10:05:11.469311+0000","last_became_active":"2024-04-25T10:05:11.461975+0000","last_became_peered":"2024-04-25T10:05:11.461975+0000","last_unstale":"2024-04-25T10:05:11.469311+0000","last_undegraded":"2024-04-25T10:05:11.469311+0000","last_fullsized":"2024-04-25T10:05:11.469311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:02.561557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455396+0000","last_change":"2024-04-25T10:05:11.463406+0000","last_active":"2024-04-25T10:05:12.455396+0000","last_peered":"2024-04-25T10:05:12.455396+0000","last_clean":"2024-04-25T10:05:12.455396+0000","last_became_active":"2024-04-25T10:05:11.463139+0000","last_became_peered":"2024-04-25T10:05:11.463139+0000","last_unstale":"2024-04-25T10:05:12.455396+0000","last_undegraded":"2024-04-25T10:05:12.455396+0000","last_fullsized":"2024-04-25T10:05:12.455396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456384+0000","last_change":"2024-04-25T10:05:11.453168+0000","last_active":"2024-04-25T10:05:12.456384+0000","last_peered":"2024-04-25T10:05:12.456384+0000","last_clean":"2024-04-25T10:05:12.456384+0000","last_became_active":"2024-04-25T10:05:11.452238+0000","last_became_peered":"2024-04-25T10:05:11.452238+0000","last_unstale":"2024-04-25T10:05:12.456384+0000","last_undegraded":"2024-04-25T10:05:12.456384+0000","last_fullsized":"2024-04-25T10:05:12.456384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786706+0000","last_change":"2024-04-25T10:05:10.455995+0000","last_active":"2024-04-25T10:05:14.786706+0000","last_peered":"2024-04-25T10:05:14.786706+0000","last_clean":"2024-04-25T10:05:14.786706+0000","last_became_active":"2024-04-25T10:05:10.455718+0000","last_became_peered":"2024-04-25T10:05:10.455718+0000","last_unstale":"2024-04-25T10:05:14.786706+0000","last_undegraded":"2024-04-25T10:05:14.786706+0000","last_fullsized":"2024-04-25T10:05:14.786706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:47.448482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456962+0000","last_change":"2024-04-25T10:05:11.469760+0000","last_active":"2024-04-25T10:05:12.456962+0000","last_peered":"2024-04-25T10:05:12.456962+0000","last_clean":"2024-04-25T10:05:12.456962+0000","last_became_active":"2024-04-25T10:05:11.469490+0000","last_became_peered":"2024-04-25T10:05:11.469490+0000","last_unstale":"2024-04-25T10:05:12.456962+0000","last_undegraded":"2024-04-25T10:05:12.456962+0000","last_fullsized":"2024-04-25T10:05:12.456962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.228600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.475721+0000","last_change":"2024-04-25T10:05:10.461940+0000","last_active":"2024-04-25T10:05:11.475721+0000","last_peered":"2024-04-25T10:05:11.475721+0000","last_clean":"2024-04-25T10:05:11.475721+0000","last_became_active":"2024-04-25T10:05:10.461633+0000","last_became_peered":"2024-04-25T10:05:10.461633+0000","last_unstale":"2024-04-25T10:05:11.475721+0000","last_undegraded":"2024-04-25T10:05:11.475721+0000","last_fullsized":"2024-04-25T10:05:11.475721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.456539+0000","last_change":"2024-04-25T10:05:10.460823+0000","last_active":"2024-04-25T10:05:11.456539+0000","last_peered":"2024-04-25T10:05:11.456539+0000","last_clean":"2024-04-25T10:05:11.456539+0000","last_became_active":"2024-04-25T10:05:10.460480+0000","last_became_peered":"2024-04-25T10:05:10.460480+0000","last_unstale":"2024-04-25T10:05:11.456539+0000","last_undegraded":"2024-04-25T10:05:11.456539+0000","last_fullsized":"2024-04-25T10:05:11.456539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:31.375084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786743+0000","last_change":"2024-04-25T10:05:11.460164+0000","last_active":"2024-04-25T10:05:14.786743+0000","last_peered":"2024-04-25T10:05:14.786743+0000","last_clean":"2024-04-25T10:05:14.786743+0000","last_became_active":"2024-04-25T10:05:11.459615+0000","last_became_peered":"2024-04-25T10:05:11.459615+0000","last_unstale":"2024-04-25T10:05:14.786743+0000","last_undegraded":"2024-04-25T10:05:14.786743+0000","last_fullsized":"2024-04-25T10:05:14.786743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:00.275933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.450458+0000","last_change":"2024-04-25T10:05:10.461713+0000","last_active":"2024-04-25T10:05:11.450458+0000","last_peered":"2024-04-25T10:05:11.450458+0000","last_clean":"2024-04-25T10:05:11.450458+0000","last_became_active":"2024-04-25T10:05:10.461379+0000","last_became_peered":"2024-04-25T10:05:10.461379+0000","last_unstale":"2024-04-25T10:05:11.450458+0000","last_undegraded":"2024-04-25T10:05:11.450458+0000","last_fullsized":"2024-04-25T10:05:11.450458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456510+0000","last_change":"2024-04-25T10:05:11.466408+0000","last_active":"2024-04-25T10:05:12.456510+0000","last_peered":"2024-04-25T10:05:12.456510+0000","last_clean":"2024-04-25T10:05:12.456510+0000","last_became_active":"2024-04-25T10:05:11.465611+0000","last_became_peered":"2024-04-25T10:05:11.465611+0000","last_unstale":"2024-04-25T10:05:12.456510+0000","last_undegraded":"2024-04-25T10:05:12.456510+0000","last_fullsized":"2024-04-25T10:05:12.456510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456657+0000","last_change":"2024-04-25T10:05:10.449054+0000","last_active":"2024-04-25T10:05:12.456657+0000","last_peered":"2024-04-25T10:05:12.456657+0000","last_clean":"2024-04-25T10:05:12.456657+0000","last_became_active":"2024-04-25T10:05:10.448743+0000","last_became_peered":"2024-04-25T10:05:10.448743+0000","last_unstale":"2024-04-25T10:05:12.456657+0000","last_undegraded":"2024-04-25T10:05:12.456657+0000","last_fullsized":"2024-04-25T10:05:12.456657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:22.897294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455686+0000","last_change":"2024-04-25T10:05:11.453268+0000","last_active":"2024-04-25T10:05:12.455686+0000","last_peered":"2024-04-25T10:05:12.455686+0000","last_clean":"2024-04-25T10:05:12.455686+0000","last_became_active":"2024-04-25T10:05:11.452971+0000","last_became_peered":"2024-04-25T10:05:11.452971+0000","last_unstale":"2024-04-25T10:05:12.455686+0000","last_undegraded":"2024-04-25T10:05:12.455686+0000","last_fullsized":"2024-04-25T10:05:12.455686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:56.142030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.452585+0000","last_change":"2024-04-25T10:05:10.452247+0000","last_active":"2024-04-25T10:05:11.452585+0000","last_peered":"2024-04-25T10:05:11.452585+0000","last_clean":"2024-04-25T10:05:11.452585+0000","last_became_active":"2024-04-25T10:05:10.451925+0000","last_became_peered":"2024-04-25T10:05:10.451925+0000","last_unstale":"2024-04-25T10:05:11.452585+0000","last_undegraded":"2024-04-25T10:05:11.452585+0000","last_fullsized":"2024-04-25T10:05:11.452585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:53.640804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786996+0000","last_change":"2024-04-25T10:05:11.467490+0000","last_active":"2024-04-25T10:05:14.786996+0000","last_peered":"2024-04-25T10:05:14.786996+0000","last_clean":"2024-04-25T10:05:14.786996+0000","last_became_active":"2024-04-25T10:05:11.467217+0000","last_became_peered":"2024-04-25T10:05:11.467217+0000","last_unstale":"2024-04-25T10:05:14.786996+0000","last_undegraded":"2024-04-25T10:05:14.786996+0000","last_fullsized":"2024-04-25T10:05:14.786996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:26.821659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472671+0000","last_change":"2024-04-25T10:05:10.459898+0000","last_active":"2024-04-25T10:05:11.472671+0000","last_peered":"2024-04-25T10:05:11.472671+0000","last_clean":"2024-04-25T10:05:11.472671+0000","last_became_active":"2024-04-25T10:05:10.459250+0000","last_became_peered":"2024-04-25T10:05:10.459250+0000","last_unstale":"2024-04-25T10:05:11.472671+0000","last_undegraded":"2024-04-25T10:05:11.472671+0000","last_fullsized":"2024-04-25T10:05:11.472671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:41.255007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.473052+0000","last_change":"2024-04-25T10:05:11.471498+0000","last_active":"2024-04-25T10:05:11.473052+0000","last_peered":"2024-04-25T10:05:11.473052+0000","last_clean":"2024-04-25T10:05:11.473052+0000","last_became_active":"2024-04-25T10:05:11.471193+0000","last_became_peered":"2024-04-25T10:05:11.471193+0000","last_unstale":"2024-04-25T10:05:11.473052+0000","last_undegraded":"2024-04-25T10:05:11.473052+0000","last_fullsized":"2024-04-25T10:05:11.473052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:45.980553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455560+0000","last_change":"2024-04-25T10:05:10.450534+0000","last_active":"2024-04-25T10:05:12.455560+0000","last_peered":"2024-04-25T10:05:12.455560+0000","last_clean":"2024-04-25T10:05:12.455560+0000","last_became_active":"2024-04-25T10:05:10.450251+0000","last_became_peered":"2024-04-25T10:05:10.450251+0000","last_unstale":"2024-04-25T10:05:12.455560+0000","last_undegraded":"2024-04-25T10:05:12.455560+0000","last_fullsized":"2024-04-25T10:05:12.455560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:22.897294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.468770+0000","last_change":"2024-04-25T10:05:11.463480+0000","last_active":"2024-04-25T10:05:11.468770+0000","last_peered":"2024-04-25T10:05:11.468770+0000","last_clean":"2024-04-25T10:05:11.468770+0000","last_became_active":"2024-04-25T10:05:11.460995+0000","last_became_peered":"2024-04-25T10:05:11.460995+0000","last_unstale":"2024-04-25T10:05:11.468770+0000","last_undegraded":"2024-04-25T10:05:11.468770+0000","last_fullsized":"2024-04-25T10:05:11.468770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:56.142030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.464540+0000","last_change":"2024-04-25T10:05:10.462323+0000","last_active":"2024-04-25T10:05:11.464540+0000","last_peered":"2024-04-25T10:05:11.464540+0000","last_clean":"2024-04-25T10:05:11.464540+0000","last_became_active":"2024-04-25T10:05:10.462118+0000","last_became_peered":"2024-04-25T10:05:10.462118+0000","last_unstale":"2024-04-25T10:05:11.464540+0000","last_undegraded":"2024-04-25T10:05:11.464540+0000","last_fullsized":"2024-04-25T10:05:11.464540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:39.826605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471805+0000","last_change":"2024-04-25T10:05:11.467396+0000","last_active":"2024-04-25T10:05:11.471805+0000","last_peered":"2024-04-25T10:05:11.471805+0000","last_clean":"2024-04-25T10:05:11.471805+0000","last_became_active":"2024-04-25T10:05:11.467112+0000","last_became_peered":"2024-04-25T10:05:11.467112+0000","last_unstale":"2024-04-25T10:05:11.471805+0000","last_undegraded":"2024-04-25T10:05:11.471805+0000","last_fullsized":"2024-04-25T10:05:11.471805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.451714+0000","last_change":"2024-04-25T10:05:10.458429+0000","last_active":"2024-04-25T10:05:11.451714+0000","last_peered":"2024-04-25T10:05:11.451714+0000","last_clean":"2024-04-25T10:05:11.451714+0000","last_became_active":"2024-04-25T10:05:10.457958+0000","last_became_peered":"2024-04-25T10:05:10.457958+0000","last_unstale":"2024-04-25T10:05:11.451714+0000","last_undegraded":"2024-04-25T10:05:11.451714+0000","last_fullsized":"2024-04-25T10:05:11.451714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:23.585636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469949+0000","last_change":"2024-04-25T10:05:11.465006+0000","last_active":"2024-04-25T10:05:11.469949+0000","last_peered":"2024-04-25T10:05:11.469949+0000","last_clean":"2024-04-25T10:05:11.469949+0000","last_became_active":"2024-04-25T10:05:11.464719+0000","last_became_peered":"2024-04-25T10:05:11.464719+0000","last_unstale":"2024-04-25T10:05:11.469949+0000","last_undegraded":"2024-04-25T10:05:11.469949+0000","last_fullsized":"2024-04-25T10:05:11.469949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:54.934726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.459177+0000","last_change":"2024-04-25T10:05:10.456938+0000","last_active":"2024-04-25T10:05:11.459177+0000","last_peered":"2024-04-25T10:05:11.459177+0000","last_clean":"2024-04-25T10:05:11.459177+0000","last_became_active":"2024-04-25T10:05:10.456644+0000","last_became_peered":"2024-04-25T10:05:10.456644+0000","last_unstale":"2024-04-25T10:05:11.459177+0000","last_undegraded":"2024-04-25T10:05:11.459177+0000","last_fullsized":"2024-04-25T10:05:11.459177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456188+0000","last_change":"2024-04-25T10:05:11.456178+0000","last_active":"2024-04-25T10:05:12.456188+0000","last_peered":"2024-04-25T10:05:12.456188+0000","last_clean":"2024-04-25T10:05:12.456188+0000","last_became_active":"2024-04-25T10:05:11.455879+0000","last_became_peered":"2024-04-25T10:05:11.455879+0000","last_unstale":"2024-04-25T10:05:12.456188+0000","last_undegraded":"2024-04-25T10:05:12.456188+0000","last_fullsized":"2024-04-25T10:05:12.456188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:55.300183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456275+0000","last_change":"2024-04-25T10:05:10.455684+0000","last_active":"2024-04-25T10:05:12.456275+0000","last_peered":"2024-04-25T10:05:12.456275+0000","last_clean":"2024-04-25T10:05:12.456275+0000","last_became_active":"2024-04-25T10:05:10.455342+0000","last_became_peered":"2024-04-25T10:05:10.455342+0000","last_unstale":"2024-04-25T10:05:12.456275+0000","last_undegraded":"2024-04-25T10:05:12.456275+0000","last_fullsized":"2024-04-25T10:05:12.456275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:01.556228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469903+0000","last_change":"2024-04-25T10:05:11.466622+0000","last_active":"2024-04-25T10:05:11.469903+0000","last_peered":"2024-04-25T10:05:11.469903+0000","last_clean":"2024-04-25T10:05:11.469903+0000","last_became_active":"2024-04-25T10:05:11.466346+0000","last_became_peered":"2024-04-25T10:05:11.466346+0000","last_unstale":"2024-04-25T10:05:11.469903+0000","last_undegraded":"2024-04-25T10:05:11.469903+0000","last_fullsized":"2024-04-25T10:05:11.469903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.468144+0000","last_change":"2024-04-25T10:05:10.457614+0000","last_active":"2024-04-25T10:05:11.468144+0000","last_peered":"2024-04-25T10:05:11.468144+0000","last_clean":"2024-04-25T10:05:11.468144+0000","last_became_active":"2024-04-25T10:05:10.457332+0000","last_became_peered":"2024-04-25T10:05:10.457332+0000","last_unstale":"2024-04-25T10:05:11.468144+0000","last_undegraded":"2024-04-25T10:05:11.468144+0000","last_fullsized":"2024-04-25T10:05:11.468144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:01.556228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455780+0000","last_change":"2024-04-25T10:05:11.460589+0000","last_active":"2024-04-25T10:05:12.455780+0000","last_peered":"2024-04-25T10:05:12.455780+0000","last_clean":"2024-04-25T10:05:12.455780+0000","last_became_active":"2024-04-25T10:05:11.459433+0000","last_became_peered":"2024-04-25T10:05:11.459433+0000","last_unstale":"2024-04-25T10:05:12.455780+0000","last_undegraded":"2024-04-25T10:05:12.455780+0000","last_fullsized":"2024-04-25T10:05:12.455780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:55.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456910+0000","last_change":"2024-04-25T10:05:10.458692+0000","last_active":"2024-04-25T10:05:12.456910+0000","last_peered":"2024-04-25T10:05:12.456910+0000","last_clean":"2024-04-25T10:05:12.456910+0000","last_became_active":"2024-04-25T10:05:10.458090+0000","last_became_peered":"2024-04-25T10:05:10.458090+0000","last_unstale":"2024-04-25T10:05:12.456910+0000","last_undegraded":"2024-04-25T10:05:12.456910+0000","last_fullsized":"2024-04-25T10:05:12.456910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:23.585636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.468810+0000","last_change":"2024-04-25T10:05:11.464532+0000","last_active":"2024-04-25T10:05:11.468810+0000","last_peered":"2024-04-25T10:05:11.468810+0000","last_clean":"2024-04-25T10:05:11.468810+0000","last_became_active":"2024-04-25T10:05:11.464186+0000","last_became_peered":"2024-04-25T10:05:11.464186+0000","last_unstale":"2024-04-25T10:05:11.468810+0000","last_undegraded":"2024-04-25T10:05:11.468810+0000","last_fullsized":"2024-04-25T10:05:11.468810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:55.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786543+0000","last_change":"2024-04-25T10:05:10.460107+0000","last_active":"2024-04-25T10:05:14.786543+0000","last_peered":"2024-04-25T10:05:14.786543+0000","last_clean":"2024-04-25T10:05:14.786543+0000","last_became_active":"2024-04-25T10:05:10.459754+0000","last_became_peered":"2024-04-25T10:05:10.459754+0000","last_unstale":"2024-04-25T10:05:14.786543+0000","last_undegraded":"2024-04-25T10:05:14.786543+0000","last_fullsized":"2024-04-25T10:05:14.786543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:48.214248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.475897+0000","last_change":"2024-04-25T10:05:11.474519+0000","last_active":"2024-04-25T10:05:11.475897+0000","last_peered":"2024-04-25T10:05:11.475897+0000","last_clean":"2024-04-25T10:05:11.475897+0000","last_became_active":"2024-04-25T10:05:11.472675+0000","last_became_peered":"2024-04-25T10:05:11.472675+0000","last_unstale":"2024-04-25T10:05:11.475897+0000","last_undegraded":"2024-04-25T10:05:11.475897+0000","last_fullsized":"2024-04-25T10:05:11.475897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.799101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.458209+0000","last_change":"2024-04-25T10:05:10.453364+0000","last_active":"2024-04-25T10:05:11.458209+0000","last_peered":"2024-04-25T10:05:11.458209+0000","last_clean":"2024-04-25T10:05:11.458209+0000","last_became_active":"2024-04-25T10:05:10.453072+0000","last_became_peered":"2024-04-25T10:05:10.453072+0000","last_unstale":"2024-04-25T10:05:11.458209+0000","last_undegraded":"2024-04-25T10:05:11.458209+0000","last_fullsized":"2024-04-25T10:05:11.458209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:53.640804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469818+0000","last_change":"2024-04-25T10:05:11.467814+0000","last_active":"2024-04-25T10:05:11.469818+0000","last_peered":"2024-04-25T10:05:11.469818+0000","last_clean":"2024-04-25T10:05:11.469818+0000","last_became_active":"2024-04-25T10:05:11.467496+0000","last_became_peered":"2024-04-25T10:05:11.467496+0000","last_unstale":"2024-04-25T10:05:11.469818+0000","last_undegraded":"2024-04-25T10:05:11.469818+0000","last_fullsized":"2024-04-25T10:05:11.469818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.799101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456108+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:05:12.456108+0000","last_peered":"2024-04-25T10:05:12.456108+0000","last_clean":"2024-04-25T10:05:12.456108+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:05:12.456108+0000","last_undegraded":"2024-04-25T10:05:12.456108+0000","last_fullsized":"2024-04-25T10:05:12.456108+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.452172+0000","last_change":"2024-04-25T10:05:10.458170+0000","last_active":"2024-04-25T10:05:11.452172+0000","last_peered":"2024-04-25T10:05:11.452172+0000","last_clean":"2024-04-25T10:05:11.452172+0000","last_became_active":"2024-04-25T10:05:10.457794+0000","last_became_peered":"2024-04-25T10:05:10.457794+0000","last_unstale":"2024-04-25T10:05:11.452172+0000","last_undegraded":"2024-04-25T10:05:11.452172+0000","last_fullsized":"2024-04-25T10:05:11.452172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:11.920992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456600+0000","last_change":"2024-04-25T10:05:11.462574+0000","last_active":"2024-04-25T10:05:12.456600+0000","last_peered":"2024-04-25T10:05:12.456600+0000","last_clean":"2024-04-25T10:05:12.456600+0000","last_became_active":"2024-04-25T10:05:11.461617+0000","last_became_peered":"2024-04-25T10:05:11.461617+0000","last_unstale":"2024-04-25T10:05:12.456600+0000","last_undegraded":"2024-04-25T10:05:12.456600+0000","last_fullsized":"2024-04-25T10:05:12.456600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.451455+0000","last_change":"2024-04-25T10:05:10.464334+0000","last_active":"2024-04-25T10:05:11.451455+0000","last_peered":"2024-04-25T10:05:11.451455+0000","last_clean":"2024-04-25T10:05:11.451455+0000","last_became_active":"2024-04-25T10:05:10.463991+0000","last_became_peered":"2024-04-25T10:05:10.463991+0000","last_unstale":"2024-04-25T10:05:11.451455+0000","last_undegraded":"2024-04-25T10:05:11.451455+0000","last_fullsized":"2024-04-25T10:05:11.451455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469159+0000","last_change":"2024-04-25T10:05:11.464866+0000","last_active":"2024-04-25T10:05:11.469159+0000","last_peered":"2024-04-25T10:05:11.469159+0000","last_clean":"2024-04-25T10:05:11.469159+0000","last_became_active":"2024-04-25T10:05:11.464504+0000","last_became_peered":"2024-04-25T10:05:11.464504+0000","last_unstale":"2024-04-25T10:05:11.469159+0000","last_undegraded":"2024-04-25T10:05:11.469159+0000","last_fullsized":"2024-04-25T10:05:11.469159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.449849+0000","last_change":"2024-04-25T10:05:10.458220+0000","last_active":"2024-04-25T10:05:11.449849+0000","last_peered":"2024-04-25T10:05:11.449849+0000","last_clean":"2024-04-25T10:05:11.449849+0000","last_became_active":"2024-04-25T10:05:10.457936+0000","last_became_peered":"2024-04-25T10:05:10.457936+0000","last_unstale":"2024-04-25T10:05:11.449849+0000","last_undegraded":"2024-04-25T10:05:11.449849+0000","last_fullsized":"2024-04-25T10:05:11.449849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:31.375084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786598+0000","last_change":"2024-04-25T10:05:11.457665+0000","last_active":"2024-04-25T10:05:14.786598+0000","last_peered":"2024-04-25T10:05:14.786598+0000","last_clean":"2024-04-25T10:05:14.786598+0000","last_became_active":"2024-04-25T10:05:11.455078+0000","last_became_peered":"2024-04-25T10:05:11.455078+0000","last_unstale":"2024-04-25T10:05:14.786598+0000","last_undegraded":"2024-04-25T10:05:14.786598+0000","last_fullsized":"2024-04-25T10:05:14.786598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:21.453409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.451883+0000","last_change":"2024-04-25T10:05:10.463388+0000","last_active":"2024-04-25T10:05:11.451883+0000","last_peered":"2024-04-25T10:05:11.451883+0000","last_clean":"2024-04-25T10:05:11.451883+0000","last_became_active":"2024-04-25T10:05:10.463041+0000","last_became_peered":"2024-04-25T10:05:10.463041+0000","last_unstale":"2024-04-25T10:05:11.451883+0000","last_undegraded":"2024-04-25T10:05:11.451883+0000","last_fullsized":"2024-04-25T10:05:11.451883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:13.500977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.463844+0000","last_change":"2024-04-25T10:05:11.462360+0000","last_active":"2024-04-25T10:05:11.463844+0000","last_peered":"2024-04-25T10:05:11.463844+0000","last_clean":"2024-04-25T10:05:11.463844+0000","last_became_active":"2024-04-25T10:05:11.462050+0000","last_became_peered":"2024-04-25T10:05:11.462050+0000","last_unstale":"2024-04-25T10:05:11.463844+0000","last_undegraded":"2024-04-25T10:05:11.463844+0000","last_fullsized":"2024-04-25T10:05:11.463844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:14.506306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786571+0000","last_change":"2024-04-25T10:05:10.456403+0000","last_active":"2024-04-25T10:05:14.786571+0000","last_peered":"2024-04-25T10:05:14.786571+0000","last_clean":"2024-04-25T10:05:14.786571+0000","last_became_active":"2024-04-25T10:05:10.456060+0000","last_became_peered":"2024-04-25T10:05:10.456060+0000","last_unstale":"2024-04-25T10:05:14.786571+0000","last_undegraded":"2024-04-25T10:05:14.786571+0000","last_fullsized":"2024-04-25T10:05:14.786571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:50.107731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455433+0000","last_change":"2024-04-25T10:05:11.464184+0000","last_active":"2024-04-25T10:05:12.455433+0000","last_peered":"2024-04-25T10:05:12.455433+0000","last_clean":"2024-04-25T10:05:12.455433+0000","last_became_active":"2024-04-25T10:05:11.463894+0000","last_became_peered":"2024-04-25T10:05:11.463894+0000","last_unstale":"2024-04-25T10:05:12.455433+0000","last_undegraded":"2024-04-25T10:05:12.455433+0000","last_fullsized":"2024-04-25T10:05:12.455433+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.452412+0000","last_change":"2024-04-25T10:05:10.460261+0000","last_active":"2024-04-25T10:05:11.452412+0000","last_peered":"2024-04-25T10:05:11.452412+0000","last_clean":"2024-04-25T10:05:11.452412+0000","last_became_active":"2024-04-25T10:05:10.459949+0000","last_became_peered":"2024-04-25T10:05:10.459949+0000","last_unstale":"2024-04-25T10:05:11.452412+0000","last_undegraded":"2024-04-25T10:05:11.452412+0000","last_fullsized":"2024-04-25T10:05:11.452412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:29.784640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.463941+0000","last_change":"2024-04-25T10:05:11.458748+0000","last_active":"2024-04-25T10:05:11.463941+0000","last_peered":"2024-04-25T10:05:11.463941+0000","last_clean":"2024-04-25T10:05:11.463941+0000","last_became_active":"2024-04-25T10:05:11.458440+0000","last_became_peered":"2024-04-25T10:05:11.458440+0000","last_unstale":"2024-04-25T10:05:11.463941+0000","last_undegraded":"2024-04-25T10:05:11.463941+0000","last_fullsized":"2024-04-25T10:05:11.463941+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:55.300183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456842+0000","last_change":"2024-04-25T10:05:10.455250+0000","last_active":"2024-04-25T10:05:12.456842+0000","last_peered":"2024-04-25T10:05:12.456842+0000","last_clean":"2024-04-25T10:05:12.456842+0000","last_became_active":"2024-04-25T10:05:10.454833+0000","last_became_peered":"2024-04-25T10:05:10.454833+0000","last_unstale":"2024-04-25T10:05:12.456842+0000","last_undegraded":"2024-04-25T10:05:12.456842+0000","last_fullsized":"2024-04-25T10:05:12.456842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.457011+0000","last_change":"2024-04-25T10:05:11.466533+0000","last_active":"2024-04-25T10:05:12.457011+0000","last_peered":"2024-04-25T10:05:12.457011+0000","last_clean":"2024-04-25T10:05:12.457011+0000","last_became_active":"2024-04-25T10:05:11.466256+0000","last_became_peered":"2024-04-25T10:05:11.466256+0000","last_unstale":"2024-04-25T10:05:12.457011+0000","last_undegraded":"2024-04-25T10:05:12.457011+0000","last_fullsized":"2024-04-25T10:05:12.457011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:41.766168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.476424+0000","last_change":"2024-04-25T10:05:10.462808+0000","last_active":"2024-04-25T10:05:11.476424+0000","last_peered":"2024-04-25T10:05:11.476424+0000","last_clean":"2024-04-25T10:05:11.476424+0000","last_became_active":"2024-04-25T10:05:10.462421+0000","last_became_peered":"2024-04-25T10:05:10.462421+0000","last_unstale":"2024-04-25T10:05:11.476424+0000","last_undegraded":"2024-04-25T10:05:11.476424+0000","last_fullsized":"2024-04-25T10:05:11.476424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:55.136701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456680+0000","last_change":"2024-04-25T10:05:11.470381+0000","last_active":"2024-04-25T10:05:12.456680+0000","last_peered":"2024-04-25T10:05:12.456680+0000","last_clean":"2024-04-25T10:05:12.456680+0000","last_became_active":"2024-04-25T10:05:11.470031+0000","last_became_peered":"2024-04-25T10:05:11.470031+0000","last_unstale":"2024-04-25T10:05:12.456680+0000","last_undegraded":"2024-04-25T10:05:12.456680+0000","last_fullsized":"2024-04-25T10:05:12.456680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.799101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.454513+0000","last_change":"2024-04-25T10:05:10.461232+0000","last_active":"2024-04-25T10:05:11.454513+0000","last_peered":"2024-04-25T10:05:11.454513+0000","last_clean":"2024-04-25T10:05:11.454513+0000","last_became_active":"2024-04-25T10:05:10.460905+0000","last_became_peered":"2024-04-25T10:05:10.460905+0000","last_unstale":"2024-04-25T10:05:11.454513+0000","last_undegraded":"2024-04-25T10:05:11.454513+0000","last_fullsized":"2024-04-25T10:05:11.454513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:54.294853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474543+0000","last_change":"2024-04-25T10:05:11.472902+0000","last_active":"2024-04-25T10:05:11.474543+0000","last_peered":"2024-04-25T10:05:11.474543+0000","last_clean":"2024-04-25T10:05:11.474543+0000","last_became_active":"2024-04-25T10:05:11.472622+0000","last_became_peered":"2024-04-25T10:05:11.472622+0000","last_unstale":"2024-04-25T10:05:11.474543+0000","last_undegraded":"2024-04-25T10:05:11.474543+0000","last_fullsized":"2024-04-25T10:05:11.474543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:24.616832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456461+0000","last_change":"2024-04-25T10:05:10.460759+0000","last_active":"2024-04-25T10:05:12.456461+0000","last_peered":"2024-04-25T10:05:12.456461+0000","last_clean":"2024-04-25T10:05:12.456461+0000","last_became_active":"2024-04-25T10:05:10.460485+0000","last_became_peered":"2024-04-25T10:05:10.460485+0000","last_unstale":"2024-04-25T10:05:12.456461+0000","last_undegraded":"2024-04-25T10:05:12.456461+0000","last_fullsized":"2024-04-25T10:05:12.456461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456829+0000","last_change":"2024-04-25T10:05:11.470739+0000","last_active":"2024-04-25T10:05:12.456829+0000","last_peered":"2024-04-25T10:05:12.456829+0000","last_clean":"2024-04-25T10:05:12.456829+0000","last_became_active":"2024-04-25T10:05:11.470408+0000","last_became_peered":"2024-04-25T10:05:11.470408+0000","last_unstale":"2024-04-25T10:05:12.456829+0000","last_undegraded":"2024-04-25T10:05:12.456829+0000","last_fullsized":"2024-04-25T10:05:12.456829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:17.789806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.457380+0000","last_change":"2024-04-25T10:05:10.458496+0000","last_active":"2024-04-25T10:05:11.457380+0000","last_peered":"2024-04-25T10:05:11.457380+0000","last_clean":"2024-04-25T10:05:11.457380+0000","last_became_active":"2024-04-25T10:05:10.458168+0000","last_became_peered":"2024-04-25T10:05:10.458168+0000","last_unstale":"2024-04-25T10:05:11.457380+0000","last_undegraded":"2024-04-25T10:05:11.457380+0000","last_fullsized":"2024-04-25T10:05:11.457380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:41.255007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.461196+0000","last_change":"2024-04-25T10:05:11.459826+0000","last_active":"2024-04-25T10:05:11.461196+0000","last_peered":"2024-04-25T10:05:11.461196+0000","last_clean":"2024-04-25T10:05:11.461196+0000","last_became_active":"2024-04-25T10:05:11.459570+0000","last_became_peered":"2024-04-25T10:05:11.459570+0000","last_unstale":"2024-04-25T10:05:11.461196+0000","last_undegraded":"2024-04-25T10:05:11.461196+0000","last_fullsized":"2024-04-25T10:05:11.461196+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:54.646134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.474457+0000","last_change":"2024-04-25T10:05:10.455137+0000","last_active":"2024-04-25T10:05:11.474457+0000","last_peered":"2024-04-25T10:05:11.474457+0000","last_clean":"2024-04-25T10:05:11.474457+0000","last_became_active":"2024-04-25T10:05:10.454827+0000","last_became_peered":"2024-04-25T10:05:10.454827+0000","last_unstale":"2024-04-25T10:05:11.474457+0000","last_undegraded":"2024-04-25T10:05:11.474457+0000","last_fullsized":"2024-04-25T10:05:11.474457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:54.691904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.470977+0000","last_change":"2024-04-25T10:05:11.469441+0000","last_active":"2024-04-25T10:05:11.470977+0000","last_peered":"2024-04-25T10:05:11.470977+0000","last_clean":"2024-04-25T10:05:11.470977+0000","last_became_active":"2024-04-25T10:05:11.468649+0000","last_became_peered":"2024-04-25T10:05:11.468649+0000","last_unstale":"2024-04-25T10:05:11.470977+0000","last_undegraded":"2024-04-25T10:05:11.470977+0000","last_fullsized":"2024-04-25T10:05:11.470977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:17.789806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.469799+0000","last_change":"2024-04-25T10:05:11.465690+0000","last_active":"2024-04-25T10:05:11.469799+0000","last_peered":"2024-04-25T10:05:11.469799+0000","last_clean":"2024-04-25T10:05:11.469799+0000","last_became_active":"2024-04-25T10:05:11.465422+0000","last_became_peered":"2024-04-25T10:05:11.465422+0000","last_unstale":"2024-04-25T10:05:11.469799+0000","last_undegraded":"2024-04-25T10:05:11.469799+0000","last_fullsized":"2024-04-25T10:05:11.469799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.472427+0000","last_change":"2024-04-25T10:05:10.450133+0000","last_active":"2024-04-25T10:05:11.472427+0000","last_peered":"2024-04-25T10:05:11.472427+0000","last_clean":"2024-04-25T10:05:11.472427+0000","last_became_active":"2024-04-25T10:05:10.449482+0000","last_became_peered":"2024-04-25T10:05:10.449482+0000","last_unstale":"2024-04-25T10:05:11.472427+0000","last_undegraded":"2024-04-25T10:05:11.472427+0000","last_fullsized":"2024-04-25T10:05:11.472427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456402+0000","last_change":"2024-04-25T10:05:11.469301+0000","last_active":"2024-04-25T10:05:12.456402+0000","last_peered":"2024-04-25T10:05:12.456402+0000","last_clean":"2024-04-25T10:05:12.456402+0000","last_became_active":"2024-04-25T10:05:11.468980+0000","last_became_peered":"2024-04-25T10:05:11.468980+0000","last_unstale":"2024-04-25T10:05:12.456402+0000","last_undegraded":"2024-04-25T10:05:12.456402+0000","last_fullsized":"2024-04-25T10:05:12.456402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:05:11.471641+0000","last_change":"2024-04-25T10:05:10.456749+0000","last_active":"2024-04-25T10:05:11.471641+0000","last_peered":"2024-04-25T10:05:11.471641+0000","last_clean":"2024-04-25T10:05:11.471641+0000","last_became_active":"2024-04-25T10:05:10.456397+0000","last_became_peered":"2024-04-25T10:05:10.456397+0000","last_unstale":"2024-04-25T10:05:11.471641+0000","last_undegraded":"2024-04-25T10:05:11.471641+0000","last_fullsized":"2024-04-25T10:05:11.471641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786907+0000","last_change":"2024-04-25T10:05:11.453961+0000","last_active":"2024-04-25T10:05:14.786907+0000","last_peered":"2024-04-25T10:05:14.786907+0000","last_clean":"2024-04-25T10:05:14.786907+0000","last_became_active":"2024-04-25T10:05:11.453443+0000","last_became_peered":"2024-04-25T10:05:11.453443+0000","last_unstale":"2024-04-25T10:05:14.786907+0000","last_undegraded":"2024-04-25T10:05:14.786907+0000","last_fullsized":"2024-04-25T10:05:14.786907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:06.235586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786886+0000","last_change":"2024-04-25T10:05:10.458501+0000","last_active":"2024-04-25T10:05:14.786886+0000","last_peered":"2024-04-25T10:05:14.786886+0000","last_clean":"2024-04-25T10:05:14.786886+0000","last_became_active":"2024-04-25T10:05:10.457915+0000","last_became_peered":"2024-04-25T10:05:10.457915+0000","last_unstale":"2024-04-25T10:05:14.786886+0000","last_undegraded":"2024-04-25T10:05:14.786886+0000","last_fullsized":"2024-04-25T10:05:14.786886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:15.560541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786839+0000","last_change":"2024-04-25T10:05:11.451812+0000","last_active":"2024-04-25T10:05:14.786839+0000","last_peered":"2024-04-25T10:05:14.786839+0000","last_clean":"2024-04-25T10:05:14.786839+0000","last_became_active":"2024-04-25T10:05:11.451318+0000","last_became_peered":"2024-04-25T10:05:11.451318+0000","last_unstale":"2024-04-25T10:05:14.786839+0000","last_undegraded":"2024-04-25T10:05:14.786839+0000","last_fullsized":"2024-04-25T10:05:14.786839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:28.693266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.457002+0000","last_change":"2024-04-25T10:05:10.454280+0000","last_active":"2024-04-25T10:05:12.457002+0000","last_peered":"2024-04-25T10:05:12.457002+0000","last_clean":"2024-04-25T10:05:12.457002+0000","last_became_active":"2024-04-25T10:05:10.453982+0000","last_became_peered":"2024-04-25T10:05:10.453982+0000","last_unstale":"2024-04-25T10:05:12.457002+0000","last_undegraded":"2024-04-25T10:05:12.457002+0000","last_fullsized":"2024-04-25T10:05:12.457002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":72548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":70846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":71876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":667002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":70320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":660724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:19.195 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:05:19.196 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T10:05:19.196 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T10:05:19.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T10:05:19.197 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T10:05:19.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.327+0000 7fac7d529640 1 Processor -- start 2024-04-25T10:05:19.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.327+0000 7fac7d529640 1 -- start start 2024-04-25T10:05:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.328+0000 7fac7d529640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fac7815dce0 0x7fac780d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.328+0000 7fac7d529640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fac780b5b60 0x7fac780b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.328+0000 7fac7d529640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac7815e0c0 con 0x7fac780d35b0 2024-04-25T10:05:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.328+0000 7fac7d529640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac780d2510 con 0x7fac780b5b60 2024-04-25T10:05:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.328+0000 7fac7d529640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac780d27f0 con 0x7fac7815dce0 2024-04-25T10:05:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.328+0000 7fac777fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fac780b5b60 0x7fac780b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.328+0000 7fac767fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fac780d35b0 0x7fac780b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34802/0 (socket says 172.21.15.16:34802) 2024-04-25T10:05:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.328+0000 7fac767fc640 1 -- 172.21.15.16:0/600356756 learned_addr learned my addr 172.21.15.16:0/600356756 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.328+0000 7fac76ffd640 1 --2- 172.21.15.16:0/600356756 >> v2:172.21.15.186:3300/0 conn(0x7fac7815dce0 0x7fac780d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.328+0000 7fac777fe640 1 -- 172.21.15.16:0/600356756 >> v2:172.21.15.186:3300/0 conn(0x7fac7815dce0 msgr2=0x7fac780d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.328+0000 7fac777fe640 1 --2- 172.21.15.16:0/600356756 >> v2:172.21.15.186:3300/0 conn(0x7fac7815dce0 0x7fac780d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.328+0000 7fac777fe640 1 -- 172.21.15.16:0/600356756 >> v1:172.21.15.16:6789/0 conn(0x7fac780d35b0 legacy=0x7fac780b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.328+0000 7fac777fe640 1 -- 172.21.15.16:0/600356756 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac780d2ad0 con 0x7fac780b5b60 2024-04-25T10:05:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.328+0000 7fac75ffb640 1 -- 172.21.15.16:0/600356756 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701861086 0 0) 0x7fac7815e0c0 con 0x7fac780d35b0 2024-04-25T10:05:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.329+0000 7fac777fe640 1 --2- 172.21.15.16:0/600356756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac780b5b60 0x7fac780b6b00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fac780ce220 tx=0x7fac6802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceaf527c75e6f1fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.329+0000 7fac75ffb640 1 -- 172.21.15.16:0/600356756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac68007740 con 0x7fac780b5b60 2024-04-25T10:05:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.329+0000 7fac75ffb640 1 -- 172.21.15.16:0/600356756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac680078e0 con 0x7fac780b5b60 2024-04-25T10:05:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.329+0000 7fac75ffb640 1 -- 172.21.15.16:0/600356756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac68007cb0 con 0x7fac780b5b60 2024-04-25T10:05:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.330+0000 7fac7d529640 1 -- 172.21.15.16:0/600356756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac780b5b60 msgr2=0x7fac780b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.330+0000 7fac7d529640 1 --2- 172.21.15.16:0/600356756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac780b5b60 0x7fac780b6b00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fac780ce220 tx=0x7fac6802a680 comp rx=0 tx=0).stop 2024-04-25T10:05:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.330+0000 7fac7d529640 1 -- 172.21.15.16:0/600356756 shutdown_connections 2024-04-25T10:05:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.330+0000 7fac7d529640 1 --2- 172.21.15.16:0/600356756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac780b5b60 0x7fac780b6b00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.330+0000 7fac7d529640 1 --2- 172.21.15.16:0/600356756 >> v2:172.21.15.186:3300/0 conn(0x7fac7815dce0 0x7fac780d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.330+0000 7fac7d529640 1 -- 172.21.15.16:0/600356756 >> 172.21.15.16:0/600356756 conn(0x7fac78072900 msgr2=0x7fac780d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:19.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.330+0000 7fac7d529640 1 -- 172.21.15.16:0/600356756 shutdown_connections 2024-04-25T10:05:19.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.330+0000 7fac7d529640 1 -- 172.21.15.16:0/600356756 wait complete. 2024-04-25T10:05:19.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.331+0000 7fac7d529640 1 Processor -- start 2024-04-25T10:05:19.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.331+0000 7fac7d529640 1 -- start start 2024-04-25T10:05:19.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac7d529640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac780b5b60 0x7fac7815e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac7d529640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac780d35b0 0x7fac78159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac7d529640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac7815dce0 0x7fac780b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac7d529640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac780b9820 con 0x7fac7815dce0 2024-04-25T10:05:19.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac7d529640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac780d3090 con 0x7fac780d35b0 2024-04-25T10:05:19.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac7d529640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac780b99a0 con 0x7fac780b5b60 2024-04-25T10:05:19.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac7815dce0 0x7fac780b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac7815dce0 0x7fac780b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41890/0 (socket says 172.21.15.16:41890) 2024-04-25T10:05:19.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac777fe640 1 -- 172.21.15.16:0/1926529973 learned_addr learned my addr 172.21.15.16:0/1926529973 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:19.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac76ffd640 1 --2- 172.21.15.16:0/1926529973 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac780b5b60 0x7fac7815e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac767fc640 1 --2- 172.21.15.16:0/1926529973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac780d35b0 0x7fac78159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac777fe640 1 -- 172.21.15.16:0/1926529973 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac780b5b60 msgr2=0x7fac7815e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac777fe640 1 --2- 172.21.15.16:0/1926529973 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac780b5b60 0x7fac7815e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac777fe640 1 -- 172.21.15.16:0/1926529973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac780d35b0 msgr2=0x7fac78159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac777fe640 1 --2- 172.21.15.16:0/1926529973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac780d35b0 0x7fac78159d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac777fe640 1 -- 172.21.15.16:0/1926529973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac780b50c0 con 0x7fac7815dce0 2024-04-25T10:05:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac76ffd640 1 --2- 172.21.15.16:0/1926529973 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac780b5b60 0x7fac7815e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac767fc640 1 --2- 172.21.15.16:0/1926529973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac780d35b0 0x7fac78159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.332+0000 7fac777fe640 1 --2- 172.21.15.16:0/1926529973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac7815dce0 0x7fac780b3c30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fac6802ac70 tx=0x7fac68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.333+0000 7fac6f7fe640 1 -- 172.21.15.16:0/1926529973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac68040070 con 0x7fac7815dce0 2024-04-25T10:05:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.333+0000 7fac6f7fe640 1 -- 172.21.15.16:0/1926529973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac68031c40 con 0x7fac7815dce0 2024-04-25T10:05:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.333+0000 7fac7d529640 1 -- 172.21.15.16:0/1926529973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac7815e970 con 0x7fac7815dce0 2024-04-25T10:05:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.333+0000 7fac6f7fe640 1 -- 172.21.15.16:0/1926529973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac68003990 con 0x7fac7815dce0 2024-04-25T10:05:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.333+0000 7fac7d529640 1 -- 172.21.15.16:0/1926529973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7815ef30 con 0x7fac7815dce0 2024-04-25T10:05:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.334+0000 7fac7d529640 1 -- 172.21.15.16:0/1926529973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac780749b0 con 0x7fac7815dce0 2024-04-25T10:05:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.334+0000 7fac6f7fe640 1 -- 172.21.15.16:0/1926529973 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac68039040 con 0x7fac7815dce0 2024-04-25T10:05:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.334+0000 7fac6f7fe640 1 --2- 172.21.15.16:0/1926529973 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fac5806b4d0 0x7fac5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.335+0000 7fac6f7fe640 1 -- 172.21.15.16:0/1926529973 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac680ab1f0 con 0x7fac7815dce0 2024-04-25T10:05:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.335+0000 7fac76ffd640 1 --2- 172.21.15.16:0/1926529973 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fac5806b4d0 0x7fac5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.335+0000 7fac76ffd640 1 --2- 172.21.15.16:0/1926529973 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fac5806b4d0 0x7fac5806d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fac70006fd0 tx=0x7fac70008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.339+0000 7fac6f7fe640 1 -- 172.21.15.16:0/1926529973 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fac6807a070 con 0x7fac7815dce0 2024-04-25T10:05:19.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.575+0000 7fac7d529640 1 -- 172.21.15.16:0/1926529973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fac780d14a0 con 0x7fac7815dce0 2024-04-25T10:05:19.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.578+0000 7fac6f7fe640 1 -- 172.21.15.16:0/1926529973 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171791 (secure 0 0 0) 0x7fac68079970 con 0x7fac7815dce0 2024-04-25T10:05:19.585 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.580+0000 7fac7d529640 1 -- 172.21.15.16:0/1926529973 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fac5806b4d0 msgr2=0x7fac5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.580+0000 7fac7d529640 1 --2- 172.21.15.16:0/1926529973 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fac5806b4d0 0x7fac5806d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fac70006fd0 tx=0x7fac70008040 comp rx=0 tx=0).stop 2024-04-25T10:05:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.580+0000 7fac7d529640 1 -- 172.21.15.16:0/1926529973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac7815dce0 msgr2=0x7fac780b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.580+0000 7fac7d529640 1 --2- 172.21.15.16:0/1926529973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac7815dce0 0x7fac780b3c30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fac6802ac70 tx=0x7fac68004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.580+0000 7fac7d529640 1 -- 172.21.15.16:0/1926529973 shutdown_connections 2024-04-25T10:05:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.580+0000 7fac7d529640 1 --2- 172.21.15.16:0/1926529973 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fac5806b4d0 0x7fac5806d990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.580+0000 7fac7d529640 1 --2- 172.21.15.16:0/1926529973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac7815dce0 0x7fac780b3c30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.580+0000 7fac7d529640 1 --2- 172.21.15.16:0/1926529973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac780d35b0 0x7fac78159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.580+0000 7fac7d529640 1 --2- 172.21.15.16:0/1926529973 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac780b5b60 0x7fac7815e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.580+0000 7fac7d529640 1 -- 172.21.15.16:0/1926529973 >> 172.21.15.16:0/1926529973 conn(0x7fac78072900 msgr2=0x7fac780ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.580+0000 7fac7d529640 1 -- 172.21.15.16:0/1926529973 shutdown_connections 2024-04-25T10:05:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.581+0000 7fac7d529640 1 -- 172.21.15.16:0/1926529973 wait complete. 2024-04-25T10:05:19.596 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":4291145543},{"type":"v1","addr":"172.21.15.186:6833","nonce":4291145543}]},"active_addr":"172.21.15.186:6833/4291145543","active_change":"2024-04-25T10:04:49.769882+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.186:0","nonce":694476157}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.186:0","nonce":3561657578}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.186:0","nonce":2313233527}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.186:0","nonce":2835888935}]}]} 2024-04-25T10:05:19.599 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T10:05:19.600 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T10:05:19.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:19.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.695+0000 7f292cd5c640 1 Processor -- start 2024-04-25T10:05:19.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.695+0000 7f292cd5c640 1 -- start start 2024-04-25T10:05:19.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.696+0000 7f292cd5c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f292815dce0 0x7f29280d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.696+0000 7f292cd5c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f292815e0c0 con 0x7f29280b5b60 2024-04-25T10:05:19.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.696+0000 7f292cd5c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29280d2510 con 0x7f29280d35b0 2024-04-25T10:05:19.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.696+0000 7f292cd5c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29280d27f0 con 0x7f292815dce0 2024-04-25T10:05:19.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.696+0000 7f292659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f292815dce0 0x7f29280d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.696+0000 7f2926d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f29280b5b60 0x7f29280b6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34810/0 (socket says 172.21.15.16:34810) 2024-04-25T10:05:19.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.696+0000 7f2926d9d640 1 -- 172.21.15.16:0/2136012989 learned_addr learned my addr 172.21.15.16:0/2136012989 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:19.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.696+0000 7f292659c640 1 -- 172.21.15.16:0/2136012989 >> v1:172.21.15.186:6789/0 conn(0x7f29280d35b0 legacy=0x7f29280b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:19.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.696+0000 7f292659c640 1 -- 172.21.15.16:0/2136012989 >> v1:172.21.15.16:6789/0 conn(0x7f29280b5b60 legacy=0x7f29280b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.696+0000 7f292659c640 1 -- 172.21.15.16:0/2136012989 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29280d2ad0 con 0x7f292815dce0 2024-04-25T10:05:19.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.696+0000 7f2925d9b640 1 -- 172.21.15.16:0/2136012989 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 794840139 0 0) 0x7f292815e0c0 con 0x7f29280b5b60 2024-04-25T10:05:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.697+0000 7f292659c640 1 --2- 172.21.15.16:0/2136012989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f292815dce0 0x7f29280d1a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f29180097a0 tx=0x7f291802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515788ad4e3fa4b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.697+0000 7f2925d9b640 1 -- 172.21.15.16:0/2136012989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918037070 con 0x7f292815dce0 2024-04-25T10:05:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.697+0000 7f2925d9b640 1 -- 172.21.15.16:0/2136012989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2918032440 con 0x7f292815dce0 2024-04-25T10:05:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.697+0000 7f2925d9b640 1 -- 172.21.15.16:0/2136012989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918032810 con 0x7f292815dce0 2024-04-25T10:05:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.697+0000 7f292cd5c640 1 -- 172.21.15.16:0/2136012989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f292815dce0 msgr2=0x7f29280d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.697+0000 7f292cd5c640 1 --2- 172.21.15.16:0/2136012989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f292815dce0 0x7f29280d1a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f29180097a0 tx=0x7f291802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.697+0000 7f292cd5c640 1 -- 172.21.15.16:0/2136012989 shutdown_connections 2024-04-25T10:05:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.697+0000 7f292cd5c640 1 --2- 172.21.15.16:0/2136012989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f292815dce0 0x7f29280d1a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.697+0000 7f292cd5c640 1 -- 172.21.15.16:0/2136012989 >> 172.21.15.16:0/2136012989 conn(0x7f2928072900 msgr2=0x7f29280d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:19.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.697+0000 7f292cd5c640 1 -- 172.21.15.16:0/2136012989 shutdown_connections 2024-04-25T10:05:19.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.697+0000 7f292cd5c640 1 -- 172.21.15.16:0/2136012989 wait complete. 2024-04-25T10:05:19.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.699+0000 7f292cd5c640 1 Processor -- start 2024-04-25T10:05:19.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.699+0000 7f292cd5c640 1 -- start start 2024-04-25T10:05:19.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f292cd5c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29280b5b60 0x7f2928153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f292cd5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29280d35b0 0x7f2928157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f292cd5c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f292815dce0 0x7f2928159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f292cd5c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29280b9820 con 0x7f29280d35b0 2024-04-25T10:05:19.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f292cd5c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29280d3090 con 0x7f292815dce0 2024-04-25T10:05:19.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f292cd5c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29280b99a0 con 0x7f29280b5b60 2024-04-25T10:05:19.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f291dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29280d35b0 0x7f2928157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f291dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29280d35b0 0x7f2928157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41910/0 (socket says 172.21.15.16:41910) 2024-04-25T10:05:19.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f291dd9b640 1 -- 172.21.15.16:0/2598600692 learned_addr learned my addr 172.21.15.16:0/2598600692 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f292659c640 1 --2- 172.21.15.16:0/2598600692 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29280b5b60 0x7f2928153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f2926d9d640 1 --2- 172.21.15.16:0/2598600692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f292815dce0 0x7f2928159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f291dd9b640 1 -- 172.21.15.16:0/2598600692 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29280b5b60 msgr2=0x7f2928153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f291dd9b640 1 --2- 172.21.15.16:0/2598600692 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29280b5b60 0x7f2928153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f291dd9b640 1 -- 172.21.15.16:0/2598600692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f292815dce0 msgr2=0x7f2928159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f291dd9b640 1 --2- 172.21.15.16:0/2598600692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f292815dce0 0x7f2928159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f291dd9b640 1 -- 172.21.15.16:0/2598600692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2928153860 con 0x7f29280d35b0 2024-04-25T10:05:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f291dd9b640 1 --2- 172.21.15.16:0/2598600692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29280d35b0 0x7f2928157680 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f291000c400 tx=0x7f291000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f291f7fe640 1 -- 172.21.15.16:0/2598600692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910011790 con 0x7f29280d35b0 2024-04-25T10:05:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f291f7fe640 1 -- 172.21.15.16:0/2598600692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2910017070 con 0x7f29280d35b0 2024-04-25T10:05:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f292cd5c640 1 -- 172.21.15.16:0/2598600692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2928158410 con 0x7f29280d35b0 2024-04-25T10:05:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f291f7fe640 1 -- 172.21.15.16:0/2598600692 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910011bb0 con 0x7f29280d35b0 2024-04-25T10:05:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.700+0000 7f292cd5c640 1 -- 172.21.15.16:0/2598600692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29281551d0 con 0x7f29280d35b0 2024-04-25T10:05:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.701+0000 7f292cd5c640 1 -- 172.21.15.16:0/2598600692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29280d20b0 con 0x7f29280d35b0 2024-04-25T10:05:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.702+0000 7f291f7fe640 1 -- 172.21.15.16:0/2598600692 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f291001c460 con 0x7f29280d35b0 2024-04-25T10:05:19.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.702+0000 7f291f7fe640 1 --2- 172.21.15.16:0/2598600692 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f290806b4d0 0x7f290806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.702+0000 7f291f7fe640 1 -- 172.21.15.16:0/2598600692 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f291008a0b0 con 0x7f29280d35b0 2024-04-25T10:05:19.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.702+0000 7f292659c640 1 --2- 172.21.15.16:0/2598600692 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f290806b4d0 0x7f290806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.703+0000 7f292659c640 1 --2- 172.21.15.16:0/2598600692 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f290806b4d0 0x7f290806d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2918034000 tx=0x7f291802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.707+0000 7f291f7fe640 1 -- 172.21.15.16:0/2598600692 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f29280d20b0 con 0x7f29280d35b0 2024-04-25T10:05:19.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.886+0000 7f292cd5c640 1 -- 172.21.15.16:0/2598600692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29280749b0 con 0x7f29280d35b0 2024-04-25T10:05:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.887+0000 7f291f7fe640 1 -- 172.21.15.16:0/2598600692 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f2910058f20 con 0x7f29280d35b0 2024-04-25T10:05:19.893 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:19.893 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:12.449183+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:19.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.889+0000 7f292cd5c640 1 -- 172.21.15.16:0/2598600692 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f290806b4d0 msgr2=0x7f290806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.889+0000 7f292cd5c640 1 --2- 172.21.15.16:0/2598600692 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f290806b4d0 0x7f290806d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2918034000 tx=0x7f291802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:19.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.889+0000 7f292cd5c640 1 -- 172.21.15.16:0/2598600692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29280d35b0 msgr2=0x7f2928157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.889+0000 7f292cd5c640 1 --2- 172.21.15.16:0/2598600692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29280d35b0 0x7f2928157680 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f291000c400 tx=0x7f291000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.889+0000 7f292cd5c640 1 -- 172.21.15.16:0/2598600692 shutdown_connections 2024-04-25T10:05:19.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.889+0000 7f292cd5c640 1 --2- 172.21.15.16:0/2598600692 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f290806b4d0 0x7f290806d990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.889+0000 7f292cd5c640 1 --2- 172.21.15.16:0/2598600692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f292815dce0 0x7f2928159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.889+0000 7f292cd5c640 1 --2- 172.21.15.16:0/2598600692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29280d35b0 0x7f2928157680 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.889+0000 7f292cd5c640 1 --2- 172.21.15.16:0/2598600692 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29280b5b60 0x7f2928153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.889+0000 7f292cd5c640 1 -- 172.21.15.16:0/2598600692 >> 172.21.15.16:0/2598600692 conn(0x7f2928072900 msgr2=0x7f29280b7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:19.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.889+0000 7f292cd5c640 1 -- 172.21.15.16:0/2598600692 shutdown_connections 2024-04-25T10:05:19.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:19.889+0000 7f292cd5c640 1 -- 172.21.15.16:0/2598600692 wait complete. 2024-04-25T10:05:19.905 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T10:05:19.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.001+0000 7fac48bb7640 1 Processor -- start 2024-04-25T10:05:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.001+0000 7fac48bb7640 1 -- start start 2024-04-25T10:05:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.002+0000 7fac48bb7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fac4415dce0 0x7fac440d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.002+0000 7fac48bb7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac4415e0c0 con 0x7fac440b5b60 2024-04-25T10:05:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.002+0000 7fac48bb7640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac440d2510 con 0x7fac440d35b0 2024-04-25T10:05:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.002+0000 7fac48bb7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac440d27f0 con 0x7fac4415dce0 2024-04-25T10:05:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.002+0000 7fac4259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fac4415dce0 0x7fac440d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.002+0000 7fac42d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fac440b5b60 0x7fac440b6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34812/0 (socket says 172.21.15.16:34812) 2024-04-25T10:05:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.002+0000 7fac42d9d640 1 -- 172.21.15.16:0/3342161582 learned_addr learned my addr 172.21.15.16:0/3342161582 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.002+0000 7fac4259c640 1 -- 172.21.15.16:0/3342161582 >> v1:172.21.15.186:6789/0 conn(0x7fac440d35b0 legacy=0x7fac440b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.002+0000 7fac4259c640 1 -- 172.21.15.16:0/3342161582 >> v1:172.21.15.16:6789/0 conn(0x7fac440b5b60 legacy=0x7fac440b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.002+0000 7fac4259c640 1 -- 172.21.15.16:0/3342161582 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac440d2ad0 con 0x7fac4415dce0 2024-04-25T10:05:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.002+0000 7fac41d9b640 1 -- 172.21.15.16:0/3342161582 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3654602037 0 0) 0x7fac4415e0c0 con 0x7fac440b5b60 2024-04-25T10:05:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.003+0000 7fac4259c640 1 --2- 172.21.15.16:0/3342161582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4415dce0 0x7fac440d1a50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fac340097a0 tx=0x7fac3402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7de9d3e50d3dd7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.003+0000 7fac41d9b640 1 -- 172.21.15.16:0/3342161582 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac34037070 con 0x7fac4415dce0 2024-04-25T10:05:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.003+0000 7fac41d9b640 1 -- 172.21.15.16:0/3342161582 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac34032440 con 0x7fac4415dce0 2024-04-25T10:05:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.003+0000 7fac41d9b640 1 -- 172.21.15.16:0/3342161582 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac34032810 con 0x7fac4415dce0 2024-04-25T10:05:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.004+0000 7fac48bb7640 1 -- 172.21.15.16:0/3342161582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4415dce0 msgr2=0x7fac440d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.004+0000 7fac48bb7640 1 --2- 172.21.15.16:0/3342161582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4415dce0 0x7fac440d1a50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fac340097a0 tx=0x7fac3402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.004+0000 7fac48bb7640 1 -- 172.21.15.16:0/3342161582 shutdown_connections 2024-04-25T10:05:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.004+0000 7fac48bb7640 1 --2- 172.21.15.16:0/3342161582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4415dce0 0x7fac440d1a50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.004+0000 7fac48bb7640 1 -- 172.21.15.16:0/3342161582 >> 172.21.15.16:0/3342161582 conn(0x7fac44072900 msgr2=0x7fac440d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.004+0000 7fac48bb7640 1 -- 172.21.15.16:0/3342161582 shutdown_connections 2024-04-25T10:05:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.004+0000 7fac48bb7640 1 -- 172.21.15.16:0/3342161582 wait complete. 2024-04-25T10:05:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.005+0000 7fac48bb7640 1 Processor -- start 2024-04-25T10:05:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.005+0000 7fac48bb7640 1 -- start start 2024-04-25T10:05:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac48bb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac440b5b60 0x7fac44153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac48bb7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac440d35b0 0x7fac44157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac48bb7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac4415dce0 0x7fac44159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac48bb7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac440b9820 con 0x7fac440b5b60 2024-04-25T10:05:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac48bb7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac440d3090 con 0x7fac4415dce0 2024-04-25T10:05:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac48bb7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac440b99a0 con 0x7fac440d35b0 2024-04-25T10:05:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac4259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac440b5b60 0x7fac44153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac4259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac440b5b60 0x7fac44153320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41928/0 (socket says 172.21.15.16:41928) 2024-04-25T10:05:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac4259c640 1 -- 172.21.15.16:0/1467404399 learned_addr learned my addr 172.21.15.16:0/1467404399 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac3bfff640 1 --2- 172.21.15.16:0/1467404399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac440d35b0 0x7fac44157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac42d9d640 1 --2- 172.21.15.16:0/1467404399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac4415dce0 0x7fac44159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac4259c640 1 -- 172.21.15.16:0/1467404399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac440d35b0 msgr2=0x7fac44157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac4259c640 1 --2- 172.21.15.16:0/1467404399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac440d35b0 0x7fac44157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac4259c640 1 -- 172.21.15.16:0/1467404399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac4415dce0 msgr2=0x7fac44159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac4259c640 1 --2- 172.21.15.16:0/1467404399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac4415dce0 0x7fac44159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac4259c640 1 -- 172.21.15.16:0/1467404399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac44153860 con 0x7fac440b5b60 2024-04-25T10:05:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac3bfff640 1 --2- 172.21.15.16:0/1467404399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac440d35b0 0x7fac44157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac42d9d640 1 --2- 172.21.15.16:0/1467404399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac4415dce0 0x7fac44159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.006+0000 7fac4259c640 1 --2- 172.21.15.16:0/1467404399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac440b5b60 0x7fac44153320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fac340097a0 tx=0x7fac340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.007+0000 7fac3affd640 1 -- 172.21.15.16:0/1467404399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac34041070 con 0x7fac440b5b60 2024-04-25T10:05:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.007+0000 7fac3affd640 1 -- 172.21.15.16:0/1467404399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3403c870 con 0x7fac440b5b60 2024-04-25T10:05:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.007+0000 7fac48bb7640 1 -- 172.21.15.16:0/1467404399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac44158410 con 0x7fac440b5b60 2024-04-25T10:05:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.007+0000 7fac3affd640 1 -- 172.21.15.16:0/1467404399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac34037040 con 0x7fac440b5b60 2024-04-25T10:05:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.007+0000 7fac48bb7640 1 -- 172.21.15.16:0/1467404399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac441551d0 con 0x7fac440b5b60 2024-04-25T10:05:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.008+0000 7fac48bb7640 1 -- 172.21.15.16:0/1467404399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac440d20b0 con 0x7fac440b5b60 2024-04-25T10:05:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.008+0000 7fac3affd640 1 -- 172.21.15.16:0/1467404399 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac34044070 con 0x7fac440b5b60 2024-04-25T10:05:20.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.008+0000 7fac3affd640 1 --2- 172.21.15.16:0/1467404399 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fac2406b4d0 0x7fac2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.009+0000 7fac3affd640 1 -- 172.21.15.16:0/1467404399 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac340aade0 con 0x7fac440b5b60 2024-04-25T10:05:20.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.009+0000 7fac3bfff640 1 --2- 172.21.15.16:0/1467404399 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fac2406b4d0 0x7fac2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.009+0000 7fac3bfff640 1 --2- 172.21.15.16:0/1467404399 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fac2406b4d0 0x7fac2406d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac2c007fd0 tx=0x7fac2c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.013+0000 7fac3affd640 1 -- 172.21.15.16:0/1467404399 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fac440d20b0 con 0x7fac440b5b60 2024-04-25T10:05:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.194+0000 7fac48bb7640 1 -- 172.21.15.16:0/1467404399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac440c9e50 con 0x7fac440b5b60 2024-04-25T10:05:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.195+0000 7fac3affd640 1 -- 172.21.15.16:0/1467404399 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16609 (secure 0 0 0) 0x7fac440c9e50 con 0x7fac440b5b60 2024-04-25T10:05:20.201 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:20.201 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:12.449183+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:20.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.197+0000 7fac48bb7640 1 -- 172.21.15.16:0/1467404399 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fac2406b4d0 msgr2=0x7fac2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.197+0000 7fac48bb7640 1 --2- 172.21.15.16:0/1467404399 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fac2406b4d0 0x7fac2406d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac2c007fd0 tx=0x7fac2c009040 comp rx=0 tx=0).stop 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.197+0000 7fac48bb7640 1 -- 172.21.15.16:0/1467404399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac440b5b60 msgr2=0x7fac44153320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.197+0000 7fac48bb7640 1 --2- 172.21.15.16:0/1467404399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac440b5b60 0x7fac44153320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fac340097a0 tx=0x7fac340046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.197+0000 7fac48bb7640 1 -- 172.21.15.16:0/1467404399 shutdown_connections 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.197+0000 7fac48bb7640 1 --2- 172.21.15.16:0/1467404399 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fac2406b4d0 0x7fac2406d990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.197+0000 7fac48bb7640 1 --2- 172.21.15.16:0/1467404399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac4415dce0 0x7fac44159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.197+0000 7fac48bb7640 1 --2- 172.21.15.16:0/1467404399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac440d35b0 0x7fac44157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.197+0000 7fac48bb7640 1 --2- 172.21.15.16:0/1467404399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac440b5b60 0x7fac44153320 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.197+0000 7fac48bb7640 1 -- 172.21.15.16:0/1467404399 >> 172.21.15.16:0/1467404399 conn(0x7fac44072900 msgr2=0x7fac440b7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.197+0000 7fac48bb7640 1 -- 172.21.15.16:0/1467404399 shutdown_connections 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.197+0000 7fac48bb7640 1 -- 172.21.15.16:0/1467404399 wait complete. 2024-04-25T10:05:20.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T10:05:20.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T10:05:20.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T10:05:20.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T10:05:20.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T10:05:20.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T10:05:20.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T10:05:20.217 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T10:05:20.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.383+0000 7fdfbe95f640 1 Processor -- start 2024-04-25T10:05:20.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.384+0000 7fdfbe95f640 1 -- start start 2024-04-25T10:05:20.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.385+0000 7f105541b640 1 Processor -- start 2024-04-25T10:05:20.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7f68b6d1d640 1 Processor -- start 2024-04-25T10:05:20.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.385+0000 7fdfbe95f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fdfb0049710 0x7fdfb0048f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7f68b6d1d640 1 -- start start 2024-04-25T10:05:20.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.385+0000 7f105541b640 1 -- start start 2024-04-25T10:05:20.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.385+0000 7fdfbe95f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb0048cb0 con 0x7fdfb0051430 2024-04-25T10:05:20.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.385+0000 7fdfbe95f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb00538c0 con 0x7fdfb0006ae0 2024-04-25T10:05:20.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.385+0000 7fdfbe95f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb0053ba0 con 0x7fdfb0049710 2024-04-25T10:05:20.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.385+0000 7fdfbe15e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdfb0051430 0x7fdfb00543a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34822/0 (socket says 172.21.15.16:34822) 2024-04-25T10:05:20.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.385+0000 7fdfbe15e640 1 -- 172.21.15.16:0/217008116 learned_addr learned my addr 172.21.15.16:0/217008116 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:20.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7fdfbc95b640 1 -- 172.21.15.16:0/217008116 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940471012 0 0) 0x7fdfb00538c0 con 0x7fdfb0006ae0 2024-04-25T10:05:20.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7fdfbc95b640 1 -- 172.21.15.16:0/217008116 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfa4003520 con 0x7fdfb0006ae0 2024-04-25T10:05:20.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7fdfbc95b640 1 -- 172.21.15.16:0/217008116 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010237432 0 0) 0x7fdfb0048cb0 con 0x7fdfb0051430 2024-04-25T10:05:20.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7ff775ea7640 1 Processor -- start 2024-04-25T10:05:20.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7f70ea438640 1 Processor -- start 2024-04-25T10:05:20.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7f70ea438640 1 -- start start 2024-04-25T10:05:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7f105541b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f10500cbcb0 0x7f10500cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7f105541b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f105015e630 0x7f105015eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7f105541b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10500cf110 con 0x7f105015aad0 2024-04-25T10:05:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7f105541b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10500cf3f0 con 0x7f10500cbcb0 2024-04-25T10:05:20.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7f105541b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10500cb4b0 con 0x7f105015e630 2024-04-25T10:05:20.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f104effd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f105015aad0 0x7f105006e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34836/0 (socket says 172.21.15.16:34836) 2024-04-25T10:05:20.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f104effd640 1 -- 172.21.15.16:0/3360689730 learned_addr learned my addr 172.21.15.16:0/3360689730 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:20.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f29aebcf640 1 Processor -- start 2024-04-25T10:05:20.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f104f7fe640 1 --2- 172.21.15.16:0/3360689730 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f105015e630 0x7f105015eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7fdfbc95b640 1 -- 172.21.15.16:0/217008116 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfb00538c0 con 0x7fdfb0051430 2024-04-25T10:05:20.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7fdfbc95b640 1 -- 172.21.15.16:0/217008116 <== mon.1 v1:172.21.15.186:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4223131394 0 0) 0x7fdfa4003520 con 0x7fdfb0006ae0 2024-04-25T10:05:20.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7fdfbc95b640 1 -- 172.21.15.16:0/217008116 >> v2:172.21.15.186:3300/0 conn(0x7fdfb0049710 msgr2=0x7fdfb0048f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:05:20.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7fdfbc95b640 1 --2- 172.21.15.16:0/217008116 >> v2:172.21.15.186:3300/0 conn(0x7fdfb0049710 0x7fdfb0048f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7fdfbc95b640 1 -- 172.21.15.16:0/217008116 >> v1:172.21.15.16:6789/0 conn(0x7fdfb0051430 legacy=0x7fdfb00543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7fdfbc95b640 1 -- 172.21.15.16:0/217008116 --> v1:172.21.15.186:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb0053e80 con 0x7fdfb0006ae0 2024-04-25T10:05:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7fdfbc95b640 1 -- 172.21.15.16:0/217008116 <== mon.1 v1:172.21.15.186:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fdfa8001670 con 0x7fdfb0006ae0 2024-04-25T10:05:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7fdfbc95b640 1 -- 172.21.15.16:0/217008116 >> v1:172.21.15.186:6789/0 conn(0x7fdfb0006ae0 legacy=0x7fdfb0006ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7fdfbc95b640 1 --2- 172.21.15.16:0/217008116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfa4003b40 0x7fdfa4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7fdfbc95b640 1 --2- 172.21.15.16:0/217008116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa4004800 0x7fdfa4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7fdfbc95b640 1 --2- 172.21.15.16:0/217008116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa4005370 0x7fdfa400b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7fdfbc95b640 1 -- 172.21.15.16:0/217008116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb0053ba0 con 0x7fdfa4004800 2024-04-25T10:05:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7fdfbc95b640 1 -- 172.21.15.16:0/217008116 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa400c2c0 con 0x7fdfa4005370 2024-04-25T10:05:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.386+0000 7fdfbc95b640 1 -- 172.21.15.16:0/217008116 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa400c4c0 con 0x7fdfa4003b40 2024-04-25T10:05:20.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f104e7fc640 1 --2- 172.21.15.16:0/3360689730 >> v2:172.21.15.186:3300/0 conn(0x7f10500cbcb0 0x7f10500cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7fdfbd15c640 1 --2- 172.21.15.16:0/217008116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa4004800 0x7fdfa4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f104e7fc640 1 -- 172.21.15.16:0/3360689730 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f105015e630 msgr2=0x7f105015eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f104e7fc640 1 --2- 172.21.15.16:0/3360689730 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f105015e630 0x7f105015eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f104e7fc640 1 -- 172.21.15.16:0/3360689730 >> v1:172.21.15.16:6789/0 conn(0x7f105015aad0 legacy=0x7f105006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f104e7fc640 1 -- 172.21.15.16:0/3360689730 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10500cb790 con 0x7f10500cbcb0 2024-04-25T10:05:20.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7ff775ea7640 1 -- start start 2024-04-25T10:05:20.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7ff775ea7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff77006bcb0 0x7ff7700cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7ff775ea7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77006c340 con 0x7ff7700c4630 2024-04-25T10:05:20.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7ff775ea7640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff770165af0 con 0x7ff7701665d0 2024-04-25T10:05:20.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7ff775ea7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff770166140 con 0x7ff77006bcb0 2024-04-25T10:05:20.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7ff76ffff640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7ff7701665d0 0x7ff770162de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:42658/0 (socket says 172.21.15.16:42658) 2024-04-25T10:05:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7f70ea438640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f70e41451f0 0x7f70e415c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7f70ea438640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f70e40ccb20 0x7f70e40ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7f70ea438640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70e40ccf80 0x7f70e40cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7f70ea438640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e40cf3a0 con 0x7f70e41451f0 2024-04-25T10:05:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7f70ea438640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e406d710 con 0x7f70e40ccb20 2024-04-25T10:05:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7f70ea438640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e406d9f0 con 0x7f70e40ccf80 2024-04-25T10:05:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7f70e37fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f70e40ccb20 0x7f70e40ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7f70e37fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f70e40ccb20 0x7f70e40ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:53424/0 (socket says 172.21.15.16:53424) 2024-04-25T10:05:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7f70e37fe640 1 -- 172.21.15.16:0/3263855619 learned_addr learned my addr 172.21.15.16:0/3263855619 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7f70e89d5640 1 --2- 172.21.15.16:0/3263855619 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70e40ccf80 0x7f70e40cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f68b6d1d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f68b0071fc0 0x7f68b006d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f68b6d1d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68b006dee0 0x7f68b00b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f68b6d1d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f68b00b3780 0x7f68b0165b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f68b6d1d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b006e3f0 con 0x7f68b0071fc0 2024-04-25T10:05:20.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f68b6d1d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b006e6d0 con 0x7f68b00b3780 2024-04-25T10:05:20.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f68b6d1d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b006e9b0 con 0x7f68b006dee0 2024-04-25T10:05:20.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f68b4ab9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f68b0071fc0 0x7f68b006d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f68b4ab9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f68b0071fc0 0x7f68b006d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41954/0 (socket says 172.21.15.16:41954) 2024-04-25T10:05:20.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f68b4ab9640 1 -- 172.21.15.16:0/335232822 learned_addr learned my addr 172.21.15.16:0/335232822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7ff76ffff640 1 -- 172.21.15.16:0/3342973904 learned_addr learned my addr 172.21.15.16:0/3342973904 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:20.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7f68b4ab9640 1 -- 172.21.15.16:0/335232822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68b006dee0 msgr2=0x7f68b00b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7f68b4ab9640 1 --2- 172.21.15.16:0/335232822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68b006dee0 0x7f68b00b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7f68b4ab9640 1 -- 172.21.15.16:0/335232822 >> v2:172.21.15.186:3300/0 conn(0x7f68b00b3780 msgr2=0x7f68b0165b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7f68b4ab9640 1 --2- 172.21.15.16:0/335232822 >> v2:172.21.15.186:3300/0 conn(0x7f68b00b3780 0x7f68b0165b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7f68b4ab9640 1 -- 172.21.15.16:0/335232822 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b0162910 con 0x7f68b0071fc0 2024-04-25T10:05:20.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f104e7fc640 1 --2- 172.21.15.16:0/3360689730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10500cbcb0 0x7f10500cc160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1048009d90 tx=0x7f104802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ad67e1284865337 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f427cd3b640 1 Processor -- start 2024-04-25T10:05:20.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f427cd3b640 1 -- start start 2024-04-25T10:05:20.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f70e37fe640 1 -- 172.21.15.16:0/3263855619 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70e40ccf80 msgr2=0x7f70e40cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f70e37fe640 1 --2- 172.21.15.16:0/3263855619 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70e40ccf80 0x7f70e40cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f70e37fe640 1 -- 172.21.15.16:0/3263855619 >> v2:172.21.15.16:3300/0 conn(0x7f70e41451f0 msgr2=0x7f70e415c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f70e37fe640 1 --2- 172.21.15.16:0/3263855619 >> v2:172.21.15.16:3300/0 conn(0x7f70e41451f0 0x7f70e415c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f70e37fe640 1 -- 172.21.15.16:0/3263855619 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e406dcd0 con 0x7f70e40ccb20 2024-04-25T10:05:20.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.387+0000 7f29aebcf640 1 -- start start 2024-04-25T10:05:20.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f29aebcf640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f29a8161d60 0x7f29a8166380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f29aebcf640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29a80cea30 0x7f29a80b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f29aebcf640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a8162140 con 0x7f29a806f0c0 2024-04-25T10:05:20.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f29aebcf640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a80b2470 con 0x7f29a8161d60 2024-04-25T10:05:20.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f29aebcf640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a8165ef0 con 0x7f29a80cea30 2024-04-25T10:05:20.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f104dffb640 1 -- 172.21.15.16:0/3360689730 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048007740 con 0x7f10500cbcb0 2024-04-25T10:05:20.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7ff7756a6640 1 --2- 172.21.15.16:0/3342973904 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff77006bcb0 0x7ff7700cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7ff76f7fe640 1 -- 172.21.15.16:0/3342973904 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1634038066 0 0) 0x7ff770165af0 con 0x7ff7701665d0 2024-04-25T10:05:20.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7ff76f7fe640 1 -- 172.21.15.16:0/3342973904 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff758003520 con 0x7ff7701665d0 2024-04-25T10:05:20.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f104dffb640 1 -- 172.21.15.16:0/3360689730 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10480078e0 con 0x7f10500cbcb0 2024-04-25T10:05:20.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f104dffb640 1 -- 172.21.15.16:0/3360689730 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048007cf0 con 0x7f10500cbcb0 2024-04-25T10:05:20.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f29ad3cc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29a80cea30 0x7f29a80b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f29ad3cc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29a80cea30 0x7f29a80b2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:34022/0 (socket says 172.21.15.16:34022) 2024-04-25T10:05:20.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f29ad3cc640 1 -- 172.21.15.16:0/4105026429 learned_addr learned my addr 172.21.15.16:0/4105026429 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7fdfbe15e640 1 --2- 172.21.15.16:0/217008116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa4005370 0x7fdfa400b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7fdfbd95d640 1 --2- 172.21.15.16:0/217008116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfa4003b40 0x7fdfa4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7fdfbe15e640 1 -- 172.21.15.16:0/217008116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfa4003b40 msgr2=0x7fdfa4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7fdfbe15e640 1 --2- 172.21.15.16:0/217008116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfa4003b40 0x7fdfa4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7fdfbe15e640 1 -- 172.21.15.16:0/217008116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa4004800 msgr2=0x7fdfa4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7fdfbe15e640 1 --2- 172.21.15.16:0/217008116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa4004800 0x7fdfa4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.388+0000 7fdfbe15e640 1 -- 172.21.15.16:0/217008116 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa4003520 con 0x7fdfa4005370 2024-04-25T10:05:20.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7fdfbe15e640 1 -- 172.21.15.16:0/217008116 reap_dead start 2024-04-25T10:05:20.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7fdfbe15e640 1 --2- 172.21.15.16:0/217008116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa4005370 0x7fdfa400b9e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8062780 tx=0x7fdfa4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25611663fc864e8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7fdfbc95b640 1 -- 172.21.15.16:0/217008116 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb804f490 con 0x7fdfa4005370 2024-04-25T10:05:20.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7fdfbc95b640 1 -- 172.21.15.16:0/217008116 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb80a9070 con 0x7fdfa4005370 2024-04-25T10:05:20.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f29adbcd640 1 --2- 172.21.15.16:0/4105026429 >> v2:172.21.15.186:3300/0 conn(0x7f29a8161d60 0x7f29a8166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7ff76f7fe640 1 -- 172.21.15.16:0/3342973904 <== mon.1 v1:172.21.15.186:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1040478081 0 0) 0x7ff758003520 con 0x7ff7701665d0 2024-04-25T10:05:20.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7ff76f7fe640 1 -- 172.21.15.16:0/3342973904 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff77006bcb0 msgr2=0x7ff7700cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7ff76f7fe640 1 --2- 172.21.15.16:0/3342973904 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff77006bcb0 0x7ff7700cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f70e37fe640 1 --2- 172.21.15.16:0/3263855619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70e40ccb20 0x7f70e40ce920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f70e40b4de0 tx=0x7f70d802a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4974f1561384745 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7ff76f7fe640 1 -- 172.21.15.16:0/3342973904 >> v1:172.21.15.16:6789/0 conn(0x7ff7700c4630 legacy=0x7ff7700c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.389+0000 7f68b4ab9640 1 --2- 172.21.15.16:0/335232822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b0071fc0 0x7f68b006d9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f68b006d760 tx=0x7f68a802a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc77a3d8338db3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f68af7fe640 1 -- 172.21.15.16:0/335232822 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8036070 con 0x7f68b0071fc0 2024-04-25T10:05:20.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7ff76f7fe640 1 -- 172.21.15.16:0/3342973904 --> v1:172.21.15.186:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff770165dd0 con 0x7ff7701665d0 2024-04-25T10:05:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f68af7fe640 1 -- 172.21.15.16:0/335232822 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a8031600 con 0x7f68b0071fc0 2024-04-25T10:05:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f68af7fe640 1 -- 172.21.15.16:0/335232822 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a80319d0 con 0x7f68b0071fc0 2024-04-25T10:05:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f105541b640 1 -- 172.21.15.16:0/3360689730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10500cbcb0 msgr2=0x7f10500cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f105541b640 1 --2- 172.21.15.16:0/3360689730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10500cbcb0 0x7f10500cc160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1048009d90 tx=0x7f104802a680 comp rx=0 tx=0).stop 2024-04-25T10:05:20.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7ff76f7fe640 1 -- 172.21.15.16:0/3342973904 <== mon.1 v1:172.21.15.186:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7ff764001670 con 0x7ff7701665d0 2024-04-25T10:05:20.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f70e2ffd640 1 -- 172.21.15.16:0/3263855619 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8037070 con 0x7f70e40ccb20 2024-04-25T10:05:20.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f70e2ffd640 1 -- 172.21.15.16:0/3263855619 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d8032440 con 0x7f70e40ccb20 2024-04-25T10:05:20.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7ff76f7fe640 1 -- 172.21.15.16:0/3342973904 >> v1:172.21.15.186:6789/0 conn(0x7ff7701665d0 legacy=0x7ff770162de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f29ad3cc640 1 -- 172.21.15.16:0/4105026429 >> v2:172.21.15.186:3300/0 conn(0x7f29a8161d60 msgr2=0x7f29a8166380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f29ad3cc640 1 --2- 172.21.15.16:0/4105026429 >> v2:172.21.15.186:3300/0 conn(0x7f29a8161d60 0x7f29a8166380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f29ad3cc640 1 -- 172.21.15.16:0/4105026429 >> v1:172.21.15.186:6789/0 conn(0x7f29a806f0c0 legacy=0x7f29a806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f29ad3cc640 1 -- 172.21.15.16:0/4105026429 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a80cf210 con 0x7f29a80cea30 2024-04-25T10:05:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f427cd3b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f42700eb050 0x7f42700552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f427cd3b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f42700eb430 0x7f42700eb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f427cd3b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42700387d0 con 0x7f42700e7a60 2024-04-25T10:05:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f427cd3b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4270057e20 con 0x7f42700eb430 2024-04-25T10:05:20.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f427cd3b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4270057990 con 0x7f42700eb050 2024-04-25T10:05:20.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f105541b640 1 -- 172.21.15.16:0/3360689730 shutdown_connections 2024-04-25T10:05:20.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f105541b640 1 --2- 172.21.15.16:0/3360689730 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f105015e630 0x7f105015eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f105541b640 1 --2- 172.21.15.16:0/3360689730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10500cbcb0 0x7f10500cc160 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f105541b640 1 -- 172.21.15.16:0/3360689730 >> 172.21.15.16:0/3360689730 conn(0x7f10500d0750 msgr2=0x7f10500d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7fdfbe95f640 1 -- 172.21.15.16:0/217008116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa4005370 msgr2=0x7fdfa400b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7fdfbe95f640 1 --2- 172.21.15.16:0/217008116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa4005370 0x7fdfa400b9e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8062780 tx=0x7fdfa4005830 comp rx=0 tx=0).stop 2024-04-25T10:05:20.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7fdfbe95f640 1 -- 172.21.15.16:0/217008116 shutdown_connections 2024-04-25T10:05:20.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7fdfbe95f640 1 --2- 172.21.15.16:0/217008116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa4005370 0x7fdfa400b9e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7fdfbe95f640 1 -- 172.21.15.16:0/217008116 >> 172.21.15.16:0/217008116 conn(0x7fdfb0055bc0 msgr2=0x7fdfb0055fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7ff76f7fe640 1 --2- 172.21.15.16:0/3342973904 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff77006bcb0 0x7ff7580039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7ff76f7fe640 1 --2- 172.21.15.16:0/3342973904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff758003ef0 0x7ff758004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7ff76f7fe640 1 --2- 172.21.15.16:0/3342973904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff758004d40 0x7ff7580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7ff76f7fe640 1 -- 172.21.15.16:0/3342973904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff770166140 con 0x7ff758003ef0 2024-04-25T10:05:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7ff76f7fe640 1 -- 172.21.15.16:0/3342973904 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75800ba70 con 0x7ff758004d40 2024-04-25T10:05:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7ff7756a6640 1 --2- 172.21.15.16:0/3342973904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff758004d40 0x7ff7580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7ff76f7fe640 1 -- 172.21.15.16:0/3342973904 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75800bc40 con 0x7ff77006bcb0 2024-04-25T10:05:20.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f70e2ffd640 1 -- 172.21.15.16:0/3263855619 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8032810 con 0x7f70e40ccb20 2024-04-25T10:05:20.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f4276d9d640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f42700e7a60 0x7f42700e7ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:42664/0 (socket says 172.21.15.16:42664) 2024-04-25T10:05:20.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f4276d9d640 1 -- 172.21.15.16:0/951154839 learned_addr learned my addr 172.21.15.16:0/951154839 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:20.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.390+0000 7f427659c640 1 --2- 172.21.15.16:0/951154839 >> v2:172.21.15.186:3300/0 conn(0x7f42700eb050 0x7f42700552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f4275d9b640 1 --2- 172.21.15.16:0/951154839 >> v2:172.21.15.16:3300/0 conn(0x7f42700eb430 0x7f42700eb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f427559a640 1 -- 172.21.15.16:0/951154839 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132987875 0 0) 0x7f427804f490 con 0x7f42700e7a60 2024-04-25T10:05:20.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f427559a640 1 -- 172.21.15.16:0/951154839 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4260003520 con 0x7f42700e7a60 2024-04-25T10:05:20.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f105541b640 1 -- 172.21.15.16:0/3360689730 shutdown_connections 2024-04-25T10:05:20.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7fdfbe95f640 1 -- 172.21.15.16:0/217008116 shutdown_connections 2024-04-25T10:05:20.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7fdfbe95f640 1 -- 172.21.15.16:0/217008116 wait complete. 2024-04-25T10:05:20.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f427659c640 1 -- 172.21.15.16:0/951154839 >> v2:172.21.15.16:3300/0 conn(0x7f42700eb430 msgr2=0x7f42700eb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f427659c640 1 --2- 172.21.15.16:0/951154839 >> v2:172.21.15.16:3300/0 conn(0x7f42700eb430 0x7f42700eb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f427659c640 1 -- 172.21.15.16:0/951154839 >> v1:172.21.15.186:6789/0 conn(0x7f42700e7a60 legacy=0x7f42700e7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f70ea438640 1 -- 172.21.15.16:0/3263855619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70e40ccb20 msgr2=0x7f70e40ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f70ea438640 1 --2- 172.21.15.16:0/3263855619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70e40ccb20 0x7f70e40ce920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f70e40b4de0 tx=0x7f70d802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f427559a640 1 -- 172.21.15.16:0/951154839 <== mon.1 v1:172.21.15.186:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 605468300 0 0) 0x7f4260003520 con 0x7f42700e7a60 2024-04-25T10:05:20.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f29ad3cc640 1 --2- 172.21.15.16:0/4105026429 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29a80cea30 0x7f29a80b2df0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009870 tx=0x7f29a402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5541d321544615a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f70ea438640 1 -- 172.21.15.16:0/3263855619 shutdown_connections 2024-04-25T10:05:20.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f70ea438640 1 --2- 172.21.15.16:0/3263855619 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70e40ccf80 0x7f70e40cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f70ea438640 1 --2- 172.21.15.16:0/3263855619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70e40ccb20 0x7f70e40ce920 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f70ea438640 1 --2- 172.21.15.16:0/3263855619 >> v2:172.21.15.16:3300/0 conn(0x7f70e41451f0 0x7f70e415c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f70ea438640 1 -- 172.21.15.16:0/3263855619 >> 172.21.15.16:0/3263855619 conn(0x7f70e406aba0 msgr2=0x7f70e406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f105541b640 1 -- 172.21.15.16:0/3360689730 wait complete. 2024-04-25T10:05:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7ff76ffff640 1 --2- 172.21.15.16:0/3342973904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff758003ef0 0x7ff758004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f29acbcb640 1 -- 172.21.15.16:0/4105026429 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4037070 con 0x7f29a80cea30 2024-04-25T10:05:20.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.392+0000 7ff7756a6640 1 -- 172.21.15.16:0/3342973904 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff77006bcb0 msgr2=0x7ff7580039b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:05:20.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.392+0000 7ff7756a6640 1 --2- 172.21.15.16:0/3342973904 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff77006bcb0 0x7ff7580039b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.392+0000 7f29acbcb640 1 -- 172.21.15.16:0/4105026429 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a4032440 con 0x7f29a80cea30 2024-04-25T10:05:20.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.392+0000 7f29acbcb640 1 -- 172.21.15.16:0/4105026429 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a40327f0 con 0x7f29a80cea30 2024-04-25T10:05:20.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.392+0000 7ff7756a6640 1 -- 172.21.15.16:0/3342973904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff758003ef0 msgr2=0x7ff758004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.391+0000 7f427659c640 1 -- 172.21.15.16:0/951154839 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4270056d80 con 0x7f42700eb050 2024-04-25T10:05:20.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.392+0000 7ff7756a6640 1 --2- 172.21.15.16:0/3342973904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff758003ef0 0x7ff758004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.392+0000 7ff7756a6640 1 -- 172.21.15.16:0/3342973904 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff758003520 con 0x7ff758004d40 2024-04-25T10:05:20.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.392+0000 7f68b6d1d640 1 -- 172.21.15.16:0/335232822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b0071fc0 msgr2=0x7f68b006d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.392+0000 7f70ea438640 1 -- 172.21.15.16:0/3263855619 shutdown_connections 2024-04-25T10:05:20.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.392+0000 7f68b6d1d640 1 --2- 172.21.15.16:0/335232822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b0071fc0 0x7f68b006d9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f68b006d760 tx=0x7f68a802a690 comp rx=0 tx=0).stop 2024-04-25T10:05:20.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.392+0000 7f427659c640 1 --2- 172.21.15.16:0/951154839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42700eb050 0x7f42700552f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4270055930 tx=0x7f426c02a010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a90d524caefe899 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.392+0000 7f68b6d1d640 1 -- 172.21.15.16:0/335232822 shutdown_connections 2024-04-25T10:05:20.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.392+0000 7f68b6d1d640 1 --2- 172.21.15.16:0/335232822 >> v2:172.21.15.186:3300/0 conn(0x7f68b00b3780 0x7f68b0165b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7f68b6d1d640 1 --2- 172.21.15.16:0/335232822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68b006dee0 0x7f68b00b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7f68b6d1d640 1 --2- 172.21.15.16:0/335232822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b0071fc0 0x7f68b006d9a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7f68b6d1d640 1 -- 172.21.15.16:0/335232822 >> 172.21.15.16:0/335232822 conn(0x7f68b006aba0 msgr2=0x7f68b006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7f427559a640 1 -- 172.21.15.16:0/951154839 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c037070 con 0x7f42700eb050 2024-04-25T10:05:20.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.392+0000 7f70ea438640 1 -- 172.21.15.16:0/3263855619 wait complete. 2024-04-25T10:05:20.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7f68b6d1d640 1 -- 172.21.15.16:0/335232822 shutdown_connections 2024-04-25T10:05:20.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7f68b6d1d640 1 -- 172.21.15.16:0/335232822 wait complete. 2024-04-25T10:05:20.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7f427559a640 1 -- 172.21.15.16:0/951154839 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c02aaa0 con 0x7f42700eb050 2024-04-25T10:05:20.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7f29aebcf640 1 -- 172.21.15.16:0/4105026429 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29a80cea30 msgr2=0x7f29a80b2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7f105541b640 1 Processor -- start 2024-04-25T10:05:20.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7f29aebcf640 1 --2- 172.21.15.16:0/4105026429 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29a80cea30 0x7f29a80b2df0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009870 tx=0x7f29a402a210 comp rx=0 tx=0).stop 2024-04-25T10:05:20.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7f105541b640 1 -- start start 2024-04-25T10:05:20.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7f29aebcf640 1 -- 172.21.15.16:0/4105026429 shutdown_connections 2024-04-25T10:05:20.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7f29aebcf640 1 --2- 172.21.15.16:0/4105026429 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29a80cea30 0x7f29a80b2df0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7f29aebcf640 1 --2- 172.21.15.16:0/4105026429 >> v2:172.21.15.186:3300/0 conn(0x7f29a8161d60 0x7f29a8166380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7f29aebcf640 1 -- 172.21.15.16:0/4105026429 >> 172.21.15.16:0/4105026429 conn(0x7f29a8066ff0 msgr2=0x7f29a80b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7fdfbe95f640 1 Processor -- start 2024-04-25T10:05:20.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7fdfbe95f640 1 -- start start 2024-04-25T10:05:20.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7ff7756a6640 1 --2- 172.21.15.16:0/3342973904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff758004d40 0x7ff7580051c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff768009fc0 tx=0x7ff758005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2353714469153130 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7f427559a640 1 -- 172.21.15.16:0/951154839 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42780627f0 con 0x7f42700eb050 2024-04-25T10:05:20.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7f29aebcf640 1 -- 172.21.15.16:0/4105026429 shutdown_connections 2024-04-25T10:05:20.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7f29aebcf640 1 -- 172.21.15.16:0/4105026429 wait complete. 2024-04-25T10:05:20.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.393+0000 7fd45759e640 1 Processor -- start 2024-04-25T10:05:20.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7fd45759e640 1 -- start start 2024-04-25T10:05:20.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7f427cd3b640 1 -- 172.21.15.16:0/951154839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42700eb050 msgr2=0x7f42700552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7f427cd3b640 1 --2- 172.21.15.16:0/951154839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42700eb050 0x7f42700552f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4270055930 tx=0x7f426c02a010 comp rx=0 tx=0).stop 2024-04-25T10:05:20.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7ff76f7fe640 1 -- 172.21.15.16:0/3342973904 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768009c10 con 0x7ff758004d40 2024-04-25T10:05:20.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7fdfbe95f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa4005370 0x7fdfb00d9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7fdfbe95f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfb00d8f40 0x7fdfb00e1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7ff76f7fe640 1 -- 172.21.15.16:0/3342973904 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff768036070 con 0x7ff758004d40 2024-04-25T10:05:20.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7fdfbe95f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb00e8bc0 0x7fdfb004eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7ff76f7fe640 1 -- 172.21.15.16:0/3342973904 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768031890 con 0x7ff758004d40 2024-04-25T10:05:20.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7fdfbe95f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb0048f90 con 0x7fdfb00e8bc0 2024-04-25T10:05:20.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7fdfbe95f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb00558e0 con 0x7fdfa4005370 2024-04-25T10:05:20.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7fdfbe95f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb0049110 con 0x7fdfb00d8f40 2024-04-25T10:05:20.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7fdfbd95d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa4005370 0x7fdfb00d9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7f70ea438640 1 Processor -- start 2024-04-25T10:05:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7f70ea438640 1 -- start start 2024-04-25T10:05:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7f105541b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10500cbcb0 0x7f105016c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7fdfbd95d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa4005370 0x7fdfb00d9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:53460/0 (socket says 172.21.15.16:53460) 2024-04-25T10:05:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7fdfbd95d640 1 -- 172.21.15.16:0/1258957911 learned_addr learned my addr 172.21.15.16:0/1258957911 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.394+0000 7fdfbe15e640 1 --2- 172.21.15.16:0/1258957911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb00e8bc0 0x7fdfb004eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7fdfbd15c640 1 --2- 172.21.15.16:0/1258957911 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfb00d8f40 0x7fdfb00e1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7f105541b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f105015aad0 0x7f1050144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7f105541b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105015e630 0x7f10501505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7f105541b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105015aeb0 con 0x7f105015e630 2024-04-25T10:05:20.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7fdfbd95d640 1 -- 172.21.15.16:0/1258957911 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfb00d8f40 msgr2=0x7fdfb00e1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7fdfbd95d640 1 --2- 172.21.15.16:0/1258957911 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfb00d8f40 0x7fdfb00e1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7fdfbd95d640 1 -- 172.21.15.16:0/1258957911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb00e8bc0 msgr2=0x7fdfb004eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7fdfbd95d640 1 --2- 172.21.15.16:0/1258957911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb00e8bc0 0x7fdfb004eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7fdfbd95d640 1 -- 172.21.15.16:0/1258957911 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb00d93c0 con 0x7fdfa4005370 2024-04-25T10:05:20.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7f68b6d1d640 1 Processor -- start 2024-04-25T10:05:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7f427cd3b640 1 -- 172.21.15.16:0/951154839 shutdown_connections 2024-04-25T10:05:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7f427cd3b640 1 --2- 172.21.15.16:0/951154839 >> v2:172.21.15.16:3300/0 conn(0x7f42700eb430 0x7f42700eb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7f427cd3b640 1 --2- 172.21.15.16:0/951154839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42700eb050 0x7f42700552f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7f427cd3b640 1 -- 172.21.15.16:0/951154839 >> 172.21.15.16:0/951154839 conn(0x7f42700559d0 msgr2=0x7f4270055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7f104e7fc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f105015aad0 0x7f1050144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7f104e7fc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f105015aad0 0x7f1050144900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:53466/0 (socket says 172.21.15.16:53466) 2024-04-25T10:05:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7f104e7fc640 1 -- 172.21.15.16:0/4080453851 learned_addr learned my addr 172.21.15.16:0/4080453851 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7f104f7fe640 1 --2- 172.21.15.16:0/4080453851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105015e630 0x7f10501505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7f68b6d1d640 1 -- start start 2024-04-25T10:05:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7f105541b640 1 -- 172.21.15.16:0/4080453851 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105006eac0 con 0x7f105015aad0 2024-04-25T10:05:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7fdfbd95d640 1 --2- 172.21.15.16:0/1258957911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa4005370 0x7fdfb00d9600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800f850 tx=0x7fdfa800fd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7f105541b640 1 -- 172.21.15.16:0/4080453851 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10500ced30 con 0x7f10500cbcb0 2024-04-25T10:05:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7f427cd3b640 1 -- 172.21.15.16:0/951154839 shutdown_connections 2024-04-25T10:05:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7f427cd3b640 1 -- 172.21.15.16:0/951154839 wait complete. 2024-04-25T10:05:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7fd45759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd45806b280 0x7fd45806b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7f70ea438640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70e40ccb20 0x7f70e4143920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7fd45759e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4580cec90 con 0x7fd4580ccf50 2024-04-25T10:05:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7fd45759e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4580c3550 con 0x7fd45806aea0 2024-04-25T10:05:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7fd45759e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4581625d0 con 0x7fd45806b280 2024-04-25T10:05:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7f70ea438640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70e40ccf80 0x7f70e40ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7f70ea438640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e41451f0 0x7f70e40ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7f29aebcf640 1 Processor -- start 2024-04-25T10:05:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7f70ea438640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e415c600 con 0x7f70e41451f0 2024-04-25T10:05:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7f70ea438640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e41455d0 con 0x7f70e40ccb20 2024-04-25T10:05:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7f70ea438640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e40cfe90 con 0x7f70e40ccf80 2024-04-25T10:05:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7f70e3fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70e40ccb20 0x7f70e4143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7f29aebcf640 1 -- start start 2024-04-25T10:05:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7f70e3fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70e40ccb20 0x7f70e4143920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:53478/0 (socket says 172.21.15.16:53478) 2024-04-25T10:05:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7f70e3fff640 1 -- 172.21.15.16:0/126556966 learned_addr learned my addr 172.21.15.16:0/126556966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7fdfae7fc640 1 -- 172.21.15.16:0/1258957911 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa80052e0 con 0x7fdfa4005370 2024-04-25T10:05:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.395+0000 7fdfae7fc640 1 -- 172.21.15.16:0/1258957911 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa8018070 con 0x7fdfa4005370 2024-04-25T10:05:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7fd45659c640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fd45806aea0 0x7fd4580cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:42668/0 (socket says 172.21.15.16:42668) 2024-04-25T10:05:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7fd45659c640 1 -- 172.21.15.16:0/2129866865 learned_addr learned my addr 172.21.15.16:0/2129866865 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7f70e3fff640 1 -- 172.21.15.16:0/126556966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70e40ccf80 msgr2=0x7f70e40ad110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:05:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7f70e37fe640 1 --2- 172.21.15.16:0/126556966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70e40ccf80 0x7f70e40ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7fd455d9b640 1 --2- 172.21.15.16:0/2129866865 >> v2:172.21.15.16:3300/0 conn(0x7fd45806b280 0x7fd45806b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7ff775ea7640 1 -- 172.21.15.16:0/3342973904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff758004d40 msgr2=0x7ff7580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7ff775ea7640 1 --2- 172.21.15.16:0/3342973904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff758004d40 0x7ff7580051c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff768009fc0 tx=0x7ff758005830 comp rx=0 tx=0).stop 2024-04-25T10:05:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7f70e3fff640 1 --2- 172.21.15.16:0/126556966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70e40ccf80 0x7f70e40ad110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7f70e3fff640 1 -- 172.21.15.16:0/126556966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e41451f0 msgr2=0x7f70e40ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7f70e3fff640 1 --2- 172.21.15.16:0/126556966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e41451f0 0x7f70e40ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7f70e3fff640 1 -- 172.21.15.16:0/126556966 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e4143e60 con 0x7f70e40ccb20 2024-04-25T10:05:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7ff7756a6640 1 -- 172.21.15.16:0/3342973904 reap_dead start 2024-04-25T10:05:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.396+0000 7f104effd640 1 --2- 172.21.15.16:0/4080453851 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10500cbcb0 0x7f105016c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7fdfae7fc640 1 -- 172.21.15.16:0/1258957911 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa800d970 con 0x7fdfa4005370 2024-04-25T10:05:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7fdfbe95f640 1 -- 172.21.15.16:0/1258957911 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb00e7e00 con 0x7fdfa4005370 2024-04-25T10:05:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7fdfbe95f640 1 -- 172.21.15.16:0/1258957911 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb00e83c0 con 0x7fdfa4005370 2024-04-25T10:05:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f104f7fe640 1 -- 172.21.15.16:0/4080453851 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10500cbcb0 msgr2=0x7f105016c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f29aebcf640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29a806f0c0 0x7f29a8164650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f29aebcf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a80cea30 0x7f29a815cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f68b6d1d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68b006dee0 0x7f68b015ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7fd45559a640 1 -- 172.21.15.16:0/2129866865 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3691409158 0 0) 0x7fd4580c3550 con 0x7fd45806aea0 2024-04-25T10:05:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7fd45559a640 1 -- 172.21.15.16:0/2129866865 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd440003520 con 0x7fd45806aea0 2024-04-25T10:05:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f104f7fe640 1 --2- 172.21.15.16:0/4080453851 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10500cbcb0 0x7f105016c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f104f7fe640 1 -- 172.21.15.16:0/4080453851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f105015aad0 msgr2=0x7f1050144900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f104f7fe640 1 --2- 172.21.15.16:0/4080453851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f105015aad0 0x7f1050144900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f68b6d1d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b0071fc0 0x7f68b015d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7ff775ea7640 1 -- 172.21.15.16:0/3342973904 shutdown_connections 2024-04-25T10:05:20.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7ff775ea7640 1 -- 172.21.15.16:0/3342973904 >> 172.21.15.16:0/3342973904 conn(0x7ff7700b3150 msgr2=0x7ff7700b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f68b4ab9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68b006dee0 0x7f68b015ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f104f7fe640 1 -- 172.21.15.16:0/4080453851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10500c6da0 con 0x7f105015e630 2024-04-25T10:05:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f29aebcf640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29a8161d60 0x7f29a8162320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f29aebcf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a81658a0 con 0x7f29a80cea30 2024-04-25T10:05:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f29aebcf640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a80b29a0 con 0x7f29a8161d60 2024-04-25T10:05:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f29aebcf640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a8165b80 con 0x7f29a806f0c0 2024-04-25T10:05:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f68b4ab9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68b006dee0 0x7f68b015ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:53486/0 (socket says 172.21.15.16:53486) 2024-04-25T10:05:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f68b4ab9640 1 -- 172.21.15.16:0/2164844211 learned_addr learned my addr 172.21.15.16:0/2164844211 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f68b6d1d640 1 --2- 172.21.15.16:0/2164844211 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68b00b3780 0x7f68b014fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f29ae3ce640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29a8161d60 0x7f29a8162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7fd455d9b640 1 -- 172.21.15.16:0/2129866865 >> v1:172.21.15.186:6789/0 conn(0x7fd45806aea0 legacy=0x7fd4580cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7fd455d9b640 1 -- 172.21.15.16:0/2129866865 >> v1:172.21.15.16:6789/0 conn(0x7fd4580ccf50 legacy=0x7fd4580b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7fd455d9b640 1 -- 172.21.15.16:0/2129866865 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4580c3830 con 0x7fd45806b280 2024-04-25T10:05:20.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f29ae3ce640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29a8161d60 0x7f29a8162320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:53496/0 (socket says 172.21.15.16:53496) 2024-04-25T10:05:20.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f29ae3ce640 1 -- 172.21.15.16:0/1033477381 learned_addr learned my addr 172.21.15.16:0/1033477381 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f68b6d1d640 1 -- 172.21.15.16:0/2164844211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b00713c0 con 0x7f68b0071fc0 2024-04-25T10:05:20.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f68b6d1d640 1 -- 172.21.15.16:0/2164844211 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b0162590 con 0x7f68b006dee0 2024-04-25T10:05:20.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f68b6d1d640 1 -- 172.21.15.16:0/2164844211 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b0071540 con 0x7f68b00b3780 2024-04-25T10:05:20.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7ff775ea7640 1 -- 172.21.15.16:0/3342973904 shutdown_connections 2024-04-25T10:05:20.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f68b52ba640 1 --2- 172.21.15.16:0/2164844211 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68b00b3780 0x7f68b014fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f29ad3cc640 1 --2- 172.21.15.16:0/1033477381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a80cea30 0x7f29a815cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f68affff640 1 --2- 172.21.15.16:0/2164844211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b0071fc0 0x7f68b015d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f29adbcd640 1 --2- 172.21.15.16:0/1033477381 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29a806f0c0 0x7f29a8164650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f70e3fff640 1 --2- 172.21.15.16:0/126556966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70e40ccb20 0x7f70e4143920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f70d400b410 tx=0x7f70d400b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f427cd3b640 1 Processor -- start 2024-04-25T10:05:20.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.397+0000 7f427cd3b640 1 -- start start 2024-04-25T10:05:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f29ae3ce640 1 -- 172.21.15.16:0/1033477381 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29a806f0c0 msgr2=0x7f29a8164650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f29ae3ce640 1 --2- 172.21.15.16:0/1033477381 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29a806f0c0 0x7f29a8164650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f29ae3ce640 1 -- 172.21.15.16:0/1033477381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a80cea30 msgr2=0x7f29a815cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f29ae3ce640 1 --2- 172.21.15.16:0/1033477381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a80cea30 0x7f29a815cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f29ae3ce640 1 -- 172.21.15.16:0/1033477381 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a815db40 con 0x7f29a8161d60 2024-04-25T10:05:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f68b4ab9640 1 -- 172.21.15.16:0/2164844211 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68b00b3780 msgr2=0x7f68b014fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f68b4ab9640 1 --2- 172.21.15.16:0/2164844211 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68b00b3780 0x7f68b014fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f68b4ab9640 1 -- 172.21.15.16:0/2164844211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b0071fc0 msgr2=0x7f68b015d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f68b4ab9640 1 --2- 172.21.15.16:0/2164844211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b0071fc0 0x7f68b015d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f68b4ab9640 1 -- 172.21.15.16:0/2164844211 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b015cc40 con 0x7f68b006dee0 2024-04-25T10:05:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7ff775ea7640 1 -- 172.21.15.16:0/3342973904 wait complete. 2024-04-25T10:05:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f104f7fe640 1 --2- 172.21.15.16:0/4080453851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105015e630 0x7f10501505b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f104400e820 tx=0x7f104400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7fdfbe95f640 1 -- 172.21.15.16:0/1258957911 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdfb00057d0 con 0x7fdfa4005370 2024-04-25T10:05:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f70e0ff9640 1 -- 172.21.15.16:0/126556966 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4015020 con 0x7f70e40ccb20 2024-04-25T10:05:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f70e0ff9640 1 -- 172.21.15.16:0/126556966 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d4011850 con 0x7f70e40ccb20 2024-04-25T10:05:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f70e0ff9640 1 -- 172.21.15.16:0/126556966 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4011c40 con 0x7f70e40ccb20 2024-04-25T10:05:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f70ea438640 1 -- 172.21.15.16:0/126556966 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e40a92f0 con 0x7f70e40ccb20 2024-04-25T10:05:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f10377fe640 1 -- 172.21.15.16:0/4080453851 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1044017070 con 0x7f105015e630 2024-04-25T10:05:20.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.398+0000 7f29ae3ce640 1 --2- 172.21.15.16:0/1033477381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29a8161d60 0x7f29a8162320 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f29a000c710 tx=0x7f29a000cbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7f70ea438640 1 -- 172.21.15.16:0/126556966 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e40a98b0 con 0x7f70e40ccb20 2024-04-25T10:05:20.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7f10377fe640 1 -- 172.21.15.16:0/4080453851 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104400c940 con 0x7f105015e630 2024-04-25T10:05:20.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7f427cd3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42700e7a60 0x7f42700e04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7f427cd3b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42700eb050 0x7f427017d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7f427cd3b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42700eb430 0x7f4270181670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7f427cd3b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4270057620 con 0x7f42700e7a60 2024-04-25T10:05:20.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7f427cd3b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4270057340 con 0x7f42700eb050 2024-04-25T10:05:20.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7f427cd3b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42700e1e00 con 0x7f42700eb430 2024-04-25T10:05:20.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7f299e7fc640 1 -- 172.21.15.16:0/1033477381 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0016020 con 0x7f29a8161d60 2024-04-25T10:05:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7f299e7fc640 1 -- 172.21.15.16:0/1033477381 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a0006d50 con 0x7f29a8161d60 2024-04-25T10:05:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7f299e7fc640 1 -- 172.21.15.16:0/1033477381 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0003bb0 con 0x7f29a8161d60 2024-04-25T10:05:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7f29aebcf640 1 -- 172.21.15.16:0/1033477381 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a8164ce0 con 0x7f29a8161d60 2024-04-25T10:05:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7f4276d9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42700eb430 0x7f4270181670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7f4276d9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42700eb430 0x7f4270181670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:34090/0 (socket says 172.21.15.16:34090) 2024-04-25T10:05:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7f4276d9d640 1 -- 172.21.15.16:0/99488681 learned_addr learned my addr 172.21.15.16:0/99488681 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7f29aebcf640 1 -- 172.21.15.16:0/1033477381 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a80ac4d0 con 0x7f29a8161d60 2024-04-25T10:05:20.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7f4275d9b640 1 --2- 172.21.15.16:0/99488681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42700eb050 0x7f427017d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7f427659c640 1 --2- 172.21.15.16:0/99488681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42700e7a60 0x7f42700e04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7fd455d9b640 1 --2- 172.21.15.16:0/2129866865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45806b280 0x7fd45806b710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0097a0 tx=0x7fd44c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6544a8f96851f318 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7f10377fe640 1 -- 172.21.15.16:0/4080453851 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10440094a0 con 0x7f105015e630 2024-04-25T10:05:20.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7ff775ea7640 1 Processor -- start 2024-04-25T10:05:20.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7f4275d9b640 1 -- 172.21.15.16:0/99488681 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42700eb430 msgr2=0x7f4270181670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7f4275d9b640 1 --2- 172.21.15.16:0/99488681 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42700eb430 0x7f4270181670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7f4275d9b640 1 -- 172.21.15.16:0/99488681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42700e7a60 msgr2=0x7f42700e04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7f4275d9b640 1 --2- 172.21.15.16:0/99488681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42700e7a60 0x7f42700e04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7f4275d9b640 1 -- 172.21.15.16:0/99488681 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4270180ee0 con 0x7f42700eb050 2024-04-25T10:05:20.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7f427659c640 1 --2- 172.21.15.16:0/99488681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42700e7a60 0x7f42700e04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:20.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.399+0000 7fdfae7fc640 1 -- 172.21.15.16:0/1258957911 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdfa800db10 con 0x7fdfa4005370 2024-04-25T10:05:20.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7f105541b640 1 -- 172.21.15.16:0/4080453851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105015bb70 con 0x7f105015e630 2024-04-25T10:05:20.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7f68b4ab9640 1 --2- 172.21.15.16:0/2164844211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68b006dee0 0x7f68b015ba50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009f40 tx=0x7f68a8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7fd45559a640 1 -- 172.21.15.16:0/2129866865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c037070 con 0x7fd45806b280 2024-04-25T10:05:20.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7fd45559a640 1 -- 172.21.15.16:0/2129866865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c032440 con 0x7fd45806b280 2024-04-25T10:05:20.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7fd45559a640 1 -- 172.21.15.16:0/2129866865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c032810 con 0x7fd45806b280 2024-04-25T10:05:20.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7fdfae7fc640 1 --2- 172.21.15.16:0/1258957911 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdf9c06b4d0 0x7fdf9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7f105541b640 1 -- 172.21.15.16:0/4080453851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105015c130 con 0x7f105015e630 2024-04-25T10:05:20.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.401+0000 7f29aebcf640 1 -- 172.21.15.16:0/1033477381 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2990001030 con 0x7f29a8161d60 2024-04-25T10:05:20.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.401+0000 7f4275d9b640 1 --2- 172.21.15.16:0/99488681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42700eb050 0x7f427017d830 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f426800e870 tx=0x7f426800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7ff775ea7640 1 -- start start 2024-04-25T10:05:20.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7fdfae7fc640 1 -- 172.21.15.16:0/1258957911 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdfa808a1b0 con 0x7fdfa4005370 2024-04-25T10:05:20.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7fdfae7fc640 1 --2- 172.21.15.16:0/1258957911 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x7fdf9c0736f0 0x7fdf9c075b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7fdfae7fc640 1 -- 172.21.15.16:0/1258957911 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdfa800ae50 con 0x7fdf9c0736f0 2024-04-25T10:05:20.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7fdfae7fc640 1 -- 172.21.15.16:0/1258957911 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfa8091050 con 0x7fdfa4005370 2024-04-25T10:05:20.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.401+0000 7fd45759e640 1 -- 172.21.15.16:0/2129866865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45806b280 msgr2=0x7fd45806b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.401+0000 7fd45759e640 1 --2- 172.21.15.16:0/2129866865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45806b280 0x7fd45806b710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0097a0 tx=0x7fd44c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.401+0000 7f105541b640 1 -- 172.21.15.16:0/4080453851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f10500c7080 con 0x7f105015e630 2024-04-25T10:05:20.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.401+0000 7fd45759e640 1 -- 172.21.15.16:0/2129866865 shutdown_connections 2024-04-25T10:05:20.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.401+0000 7fd45759e640 1 --2- 172.21.15.16:0/2129866865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45806b280 0x7fd45806b710 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7fd45759e640 1 -- 172.21.15.16:0/2129866865 >> 172.21.15.16:0/2129866865 conn(0x7fd458099a40 msgr2=0x7fd4580b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.400+0000 7f70ea438640 1 -- 172.21.15.16:0/126556966 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f70e40750b0 con 0x7f70e40ccb20 2024-04-25T10:05:20.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.401+0000 7f70e0ff9640 1 -- 172.21.15.16:0/126556966 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70d4018020 con 0x7f70e40ccb20 2024-04-25T10:05:20.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.401+0000 7fdfbe15e640 1 --2- 172.21.15.16:0/1258957911 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x7fdf9c0736f0 0x7fdf9c075b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7ff775ea7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff758004d40 0x7ff7701ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7ff775ea7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7700b1750 0x7ff7700b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7ff775ea7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7700ca4c0 0x7ff7700ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7ff775ea7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77015c8e0 con 0x7ff7700ca4c0 2024-04-25T10:05:20.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7ff775ea7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77015c760 con 0x7ff7700b1750 2024-04-25T10:05:20.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7ff775ea7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7700cbcb0 con 0x7ff758004d40 2024-04-25T10:05:20.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.401+0000 7f68ad7fa640 1 -- 172.21.15.16:0/2164844211 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8041020 con 0x7f68b006dee0 2024-04-25T10:05:20.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.401+0000 7f68ad7fa640 1 -- 172.21.15.16:0/2164844211 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a8037040 con 0x7f68b006dee0 2024-04-25T10:05:20.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7ff7756a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7700ca4c0 0x7ff7700ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7f299e7fc640 1 -- 172.21.15.16:0/1033477381 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29a000a850 con 0x7f29a8161d60 2024-04-25T10:05:20.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7ff7756a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7700ca4c0 0x7ff7700ca940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42076/0 (socket says 172.21.15.16:42076) 2024-04-25T10:05:20.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7ff7756a6640 1 -- 172.21.15.16:0/3100934719 learned_addr learned my addr 172.21.15.16:0/3100934719 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7fdfbe15e640 1 --2- 172.21.15.16:0/1258957911 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x7fdf9c0736f0 0x7fdf9c075b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.401+0000 7f68ad7fa640 1 -- 172.21.15.16:0/2164844211 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8036070 con 0x7f68b006dee0 2024-04-25T10:05:20.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7fd45759e640 1 -- 172.21.15.16:0/2129866865 shutdown_connections 2024-04-25T10:05:20.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7f68b6d1d640 1 -- 172.21.15.16:0/2164844211 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b015ac90 con 0x7f68b006dee0 2024-04-25T10:05:20.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7f68b6d1d640 1 -- 172.21.15.16:0/2164844211 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b015b250 con 0x7f68b006dee0 2024-04-25T10:05:20.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7fd45759e640 1 -- 172.21.15.16:0/2129866865 wait complete. 2024-04-25T10:05:20.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7fdfbd15c640 1 --2- 172.21.15.16:0/1258957911 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdf9c06b4d0 0x7fdf9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7f299e7fc640 1 --2- 172.21.15.16:0/1033477381 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f298c06b5a0 0x7f298c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7f70e0ff9640 1 --2- 172.21.15.16:0/126556966 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f70d006f9b0 0x7f70d0071e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7f70e0ff9640 1 -- 172.21.15.16:0/126556966 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70d408aa40 con 0x7f70e40ccb20 2024-04-25T10:05:20.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7f70e0ff9640 1 --2- 172.21.15.16:0/126556966 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x7f70d0077bd0 0x7f70d0079ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7f70e0ff9640 1 -- 172.21.15.16:0/126556966 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f70d400fe50 con 0x7f70d0077bd0 2024-04-25T10:05:20.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7f70e0ff9640 1 -- 172.21.15.16:0/126556966 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70d408ae30 con 0x7f70e40ccb20 2024-04-25T10:05:20.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7f70e89d5640 1 --2- 172.21.15.16:0/126556966 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x7f70d0077bd0 0x7f70d0079ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7f299e7fc640 1 -- 172.21.15.16:0/1033477381 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29a0089e90 con 0x7f29a8161d60 2024-04-25T10:05:20.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7f299e7fc640 1 --2- 172.21.15.16:0/1033477381 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x7f298c073740 0x7f298c075b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7f4266ffd640 1 -- 172.21.15.16:0/99488681 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4268017070 con 0x7f42700eb050 2024-04-25T10:05:20.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7f299e7fc640 1 -- 172.21.15.16:0/1033477381 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29a001eec0 con 0x7f298c073740 2024-04-25T10:05:20.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7f29adbcd640 1 --2- 172.21.15.16:0/1033477381 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f298c06b5a0 0x7f298c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7f10377fe640 1 -- 172.21.15.16:0/4080453851 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f104401a070 con 0x7f105015e630 2024-04-25T10:05:20.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7f299e7fc640 1 -- 172.21.15.16:0/1033477381 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29a0054920 con 0x7f29a8161d60 2024-04-25T10:05:20.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7fdfae7fc640 1 -- 172.21.15.16:0/1258957911 <== osd.1 v2:172.21.15.16:6808/1916704159 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fdfa800ae50 con 0x7fdf9c0736f0 2024-04-25T10:05:20.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7f70e37fe640 1 --2- 172.21.15.16:0/126556966 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f70d006f9b0 0x7f70d0071e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.404+0000 7fdfbd15c640 1 --2- 172.21.15.16:0/1258957911 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdf9c06b4d0 0x7fdf9c06d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4006fd0 tx=0x7fdfb4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7f68b6d1d640 1 -- 172.21.15.16:0/2164844211 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f68b00cb940 con 0x7f68b006dee0 2024-04-25T10:05:20.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7f427cd3b640 1 -- 172.21.15.16:0/99488681 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427017e0f0 con 0x7f42700eb050 2024-04-25T10:05:20.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7f427cd3b640 1 -- 172.21.15.16:0/99488681 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427017e6b0 con 0x7f42700eb050 2024-04-25T10:05:20.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.404+0000 7f4266ffd640 1 -- 172.21.15.16:0/99488681 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426800ee60 con 0x7f42700eb050 2024-04-25T10:05:20.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.404+0000 7f4266ffd640 1 -- 172.21.15.16:0/99488681 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426800cb60 con 0x7f42700eb050 2024-04-25T10:05:20.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7f29ad3cc640 1 --2- 172.21.15.16:0/1033477381 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x7f298c073740 0x7f298c075b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.404+0000 7f70e89d5640 1 --2- 172.21.15.16:0/126556966 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x7f70d0077bd0 0x7f70d0079ff0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.404+0000 7f29adbcd640 1 --2- 172.21.15.16:0/1033477381 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f298c06b5a0 0x7f298c06da60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2998006fd0 tx=0x7f2998008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7ff7756a6640 1 -- 172.21.15.16:0/3100934719 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff758004d40 msgr2=0x7ff7701ffc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7ff7756a6640 1 --2- 172.21.15.16:0/3100934719 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff758004d40 0x7ff7701ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.404+0000 7f10377fe640 1 --2- 172.21.15.16:0/4080453851 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f103c06b5a0 0x7f103c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7ff7756a6640 1 -- 172.21.15.16:0/3100934719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7700b1750 msgr2=0x7ff7700b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7ff7756a6640 1 --2- 172.21.15.16:0/3100934719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7700b1750 0x7ff7700b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.402+0000 7ff7756a6640 1 -- 172.21.15.16:0/3100934719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7700cae80 con 0x7ff7700ca4c0 2024-04-25T10:05:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7ff7756a6640 1 --2- 172.21.15.16:0/3100934719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7700ca4c0 0x7ff7700ca940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff76800e200 tx=0x7ff768009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7ff76d7fa640 1 -- 172.21.15.16:0/3100934719 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768009c10 con 0x7ff7700ca4c0 2024-04-25T10:05:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7ff76d7fa640 1 -- 172.21.15.16:0/3100934719 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff768040030 con 0x7ff7700ca4c0 2024-04-25T10:05:20.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.404+0000 7f10377fe640 1 -- 172.21.15.16:0/4080453851 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f104408a010 con 0x7f105015e630 2024-04-25T10:05:20.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.403+0000 7ff76d7fa640 1 -- 172.21.15.16:0/3100934719 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76803c840 con 0x7ff7700ca4c0 2024-04-25T10:05:20.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.404+0000 7f29ad3cc640 1 --2- 172.21.15.16:0/1033477381 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x7f298c073740 0x7f298c075b60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.404+0000 7f10377fe640 1 --2- 172.21.15.16:0/4080453851 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x7f103c0737c0 0x7f103c075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.404+0000 7f104effd640 1 --2- 172.21.15.16:0/4080453851 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f103c06b5a0 0x7f103c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.404+0000 7f427cd3b640 1 -- 172.21.15.16:0/99488681 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f425c001030 con 0x7f42700eb050 2024-04-25T10:05:20.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.405+0000 7f10377fe640 1 -- 172.21.15.16:0/4080453851 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1044012e50 con 0x7f103c0737c0 2024-04-25T10:05:20.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.405+0000 7f10377fe640 1 -- 172.21.15.16:0/4080453851 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1044091050 con 0x7f105015e630 2024-04-25T10:05:20.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.405+0000 7f68ad7fa640 1 -- 172.21.15.16:0/2164844211 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68a8066070 con 0x7f68b006dee0 2024-04-25T10:05:20.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.405+0000 7fd45759e640 1 Processor -- start 2024-04-25T10:05:20.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.405+0000 7ff775ea7640 1 -- 172.21.15.16:0/3100934719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff770157fb0 con 0x7ff7700ca4c0 2024-04-25T10:05:20.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.405+0000 7ff775ea7640 1 -- 172.21.15.16:0/3100934719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7700cb3d0 con 0x7ff7700ca4c0 2024-04-25T10:05:20.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.405+0000 7fd45759e640 1 -- start start 2024-04-25T10:05:20.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.405+0000 7f70e0ff9640 1 -- 172.21.15.16:0/126556966 <== osd.4 v2:172.21.15.186:6824/3091472889 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f70d400fe50 con 0x7f70d0077bd0 2024-04-25T10:05:20.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.406+0000 7f299e7fc640 1 -- 172.21.15.16:0/1033477381 <== osd.2 v2:172.21.15.16:6800/2094356376 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f29a001eec0 con 0x7f298c073740 2024-04-25T10:05:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.406+0000 7f68ad7fa640 1 --2- 172.21.15.16:0/2164844211 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f689c06b5a0 0x7f689c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.406+0000 7f70e37fe640 1 --2- 172.21.15.16:0/126556966 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f70d006f9b0 0x7f70d0071e70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f70d8034000 tx=0x7f70d802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.406+0000 7f68ad7fa640 1 -- 172.21.15.16:0/2164844211 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68a80abd80 con 0x7f68b006dee0 2024-04-25T10:05:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.405+0000 7f4266ffd640 1 -- 172.21.15.16:0/99488681 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f426801a070 con 0x7f42700eb050 2024-04-25T10:05:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.406+0000 7f68ad7fa640 1 --2- 172.21.15.16:0/2164844211 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x7f689c0737c0 0x7f689c075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.406+0000 7f68ad7fa640 1 -- 172.21.15.16:0/2164844211 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f689c076290 con 0x7f689c0737c0 2024-04-25T10:05:20.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.406+0000 7f4266ffd640 1 --2- 172.21.15.16:0/99488681 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f425806b5a0 0x7f425806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.406+0000 7f104e7fc640 1 --2- 172.21.15.16:0/4080453851 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x7f103c0737c0 0x7f103c075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.406+0000 7f68ad7fa640 1 -- 172.21.15.16:0/2164844211 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68a80ac170 con 0x7f68b006dee0 2024-04-25T10:05:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.406+0000 7f104e7fc640 1 --2- 172.21.15.16:0/4080453851 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x7f103c0737c0 0x7f103c075be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.406+0000 7ff775ea7640 1 -- 172.21.15.16:0/3100934719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff7700749b0 con 0x7ff7700ca4c0 2024-04-25T10:05:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.406+0000 7f427659c640 1 --2- 172.21.15.16:0/99488681 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f425806b5a0 0x7f425806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.407+0000 7fd45759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45806aea0 0x7fd458151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.407+0000 7f4266ffd640 1 -- 172.21.15.16:0/99488681 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4268089f50 con 0x7f42700eb050 2024-04-25T10:05:20.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.407+0000 7fd45759e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd45806b280 0x7fd458164360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.407+0000 7fd45759e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4580ccf50 0x7fd4580ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.407+0000 7fd45759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4580cf210 con 0x7fd45806aea0 2024-04-25T10:05:20.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.407+0000 7fd45659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45806aea0 0x7fd458151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.407+0000 7fd45759e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4580cf4f0 con 0x7fd45806b280 2024-04-25T10:05:20.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.407+0000 7fd45659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45806aea0 0x7fd458151b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42082/0 (socket says 172.21.15.16:42082) 2024-04-25T10:05:20.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.407+0000 7fd45759e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4580cdba0 con 0x7fd4580ccf50 2024-04-25T10:05:20.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.407+0000 7fd45659c640 1 -- 172.21.15.16:0/2635431470 learned_addr learned my addr 172.21.15.16:0/2635431470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.407+0000 7fd455d9b640 1 --2- 172.21.15.16:0/2635431470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd45806b280 0x7fd458164360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.407+0000 7fd456d9d640 1 --2- 172.21.15.16:0/2635431470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4580ccf50 0x7fd4580ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.407+0000 7f427659c640 1 --2- 172.21.15.16:0/99488681 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f425806b5a0 0x7f425806da60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f426c034000 tx=0x7f426c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.407+0000 7f68affff640 1 --2- 172.21.15.16:0/2164844211 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f689c06b5a0 0x7f689c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.407+0000 7f4266ffd640 1 --2- 172.21.15.16:0/99488681 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x7f42580737c0 0x7f4258075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.407+0000 7f4266ffd640 1 -- 172.21.15.16:0/99488681 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4268022e50 con 0x7f42580737c0 2024-04-25T10:05:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.407+0000 7f4266ffd640 1 -- 172.21.15.16:0/99488681 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f426808c5b0 con 0x7f42700eb050 2024-04-25T10:05:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.408+0000 7f10377fe640 1 -- 172.21.15.16:0/4080453851 <== osd.0 v2:172.21.15.16:6816/2167750256 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1044012e50 con 0x7f103c0737c0 2024-04-25T10:05:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.408+0000 7f68b52ba640 1 --2- 172.21.15.16:0/2164844211 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x7f689c0737c0 0x7f689c075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.408+0000 7f4276d9d640 1 --2- 172.21.15.16:0/99488681 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x7f42580737c0 0x7f4258075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.408+0000 7fd455d9b640 1 -- 172.21.15.16:0/2635431470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4580ccf50 msgr2=0x7fd4580ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.408+0000 7fd455d9b640 1 --2- 172.21.15.16:0/2635431470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4580ccf50 0x7fd4580ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.408+0000 7fd455d9b640 1 -- 172.21.15.16:0/2635431470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45806aea0 msgr2=0x7fd458151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.408+0000 7fd455d9b640 1 --2- 172.21.15.16:0/2635431470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45806aea0 0x7fd458151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.408+0000 7fd455d9b640 1 -- 172.21.15.16:0/2635431470 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4581520c0 con 0x7fd45806b280 2024-04-25T10:05:20.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.408+0000 7ff76d7fa640 1 -- 172.21.15.16:0/3100934719 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff768005d10 con 0x7ff7700ca4c0 2024-04-25T10:05:20.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.408+0000 7f104effd640 1 --2- 172.21.15.16:0/4080453851 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f103c06b5a0 0x7f103c06da60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f10500c7250 tx=0x7f1040009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.408+0000 7f68affff640 1 --2- 172.21.15.16:0/2164844211 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f689c06b5a0 0x7f689c06da60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f68b01600b0 tx=0x7f68a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.408+0000 7fd455d9b640 1 --2- 172.21.15.16:0/2635431470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd45806b280 0x7fd458164360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd458072f70 tx=0x7fd44c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.408+0000 7f68b52ba640 1 --2- 172.21.15.16:0/2164844211 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x7f689c0737c0 0x7f689c075be0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.408+0000 7fd446ffd640 1 -- 172.21.15.16:0/2635431470 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c037050 con 0x7fd45806b280 2024-04-25T10:05:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.408+0000 7f4276d9d640 1 --2- 172.21.15.16:0/99488681 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x7f42580737c0 0x7f4258075be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.408+0000 7fd446ffd640 1 -- 172.21.15.16:0/2635431470 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c038040 con 0x7fd45806b280 2024-04-25T10:05:20.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.409+0000 7fd45759e640 1 -- 172.21.15.16:0/2635431470 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45815e290 con 0x7fd45806b280 2024-04-25T10:05:20.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.409+0000 7fd45759e640 1 -- 172.21.15.16:0/2635431470 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45815d4d0 con 0x7fd45806b280 2024-04-25T10:05:20.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.409+0000 7ff76d7fa640 1 --2- 172.21.15.16:0/3100934719 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff75c06b5a0 0x7ff75c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.409+0000 7fd446ffd640 1 -- 172.21.15.16:0/2635431470 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c03c8f0 con 0x7fd45806b280 2024-04-25T10:05:20.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.409+0000 7ff76d7fa640 1 -- 172.21.15.16:0/3100934719 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7680ab6e0 con 0x7ff7700ca4c0 2024-04-25T10:05:20.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.409+0000 7ff76d7fa640 1 --2- 172.21.15.16:0/3100934719 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x7ff75c073820 0x7ff75c075c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.409+0000 7ff76d7fa640 1 -- 172.21.15.16:0/3100934719 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff768038e50 con 0x7ff75c073820 2024-04-25T10:05:20.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.409+0000 7ff76d7fa640 1 -- 172.21.15.16:0/3100934719 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7680abad0 con 0x7ff7700ca4c0 2024-04-25T10:05:20.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.410+0000 7f4266ffd640 1 -- 172.21.15.16:0/99488681 <== osd.7 v2:172.21.15.186:6808/1627307043 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4268022e50 con 0x7f42580737c0 2024-04-25T10:05:20.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.410+0000 7ff76ffff640 1 --2- 172.21.15.16:0/3100934719 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x7ff75c073820 0x7ff75c075c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.410+0000 7ff76ffff640 1 --2- 172.21.15.16:0/3100934719 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x7ff75c073820 0x7ff75c075c40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.410+0000 7ff774ea5640 1 --2- 172.21.15.16:0/3100934719 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff75c06b5a0 0x7ff75c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.411+0000 7fd45759e640 1 -- 172.21.15.16:0/2635431470 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd43c001030 con 0x7fd45806b280 2024-04-25T10:05:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.411+0000 7fd446ffd640 1 -- 172.21.15.16:0/2635431470 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd44c045020 con 0x7fd45806b280 2024-04-25T10:05:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.411+0000 7fd446ffd640 1 --2- 172.21.15.16:0/2635431470 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd43806b4d0 0x7fd43806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.411+0000 7fd446ffd640 1 -- 172.21.15.16:0/2635431470 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd44c0aa350 con 0x7fd45806b280 2024-04-25T10:05:20.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.411+0000 7fd446ffd640 1 --2- 172.21.15.16:0/2635431470 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x7fd4380736f0 0x7fd438075b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.411+0000 7fd446ffd640 1 -- 172.21.15.16:0/2635431470 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd44c001d30 con 0x7fd4380736f0 2024-04-25T10:05:20.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.411+0000 7fd446ffd640 1 -- 172.21.15.16:0/2635431470 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd44c0b1050 con 0x7fd45806b280 2024-04-25T10:05:20.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.412+0000 7ff76d7fa640 1 -- 172.21.15.16:0/3100934719 <== osd.3 v2:172.21.15.16:6824/256577261 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff768038e50 con 0x7ff75c073820 2024-04-25T10:05:20.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.412+0000 7f68ad7fa640 1 -- 172.21.15.16:0/2164844211 <== osd.5 v2:172.21.15.186:6800/1827108421 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f689c076290 con 0x7f689c0737c0 2024-04-25T10:05:20.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.412+0000 7fd45659c640 1 --2- 172.21.15.16:0/2635431470 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd43806b4d0 0x7fd43806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.412+0000 7ff774ea5640 1 --2- 172.21.15.16:0/3100934719 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff75c06b5a0 0x7ff75c06da60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff7600014e0 tx=0x7ff760007990 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.412+0000 7fd45659c640 1 --2- 172.21.15.16:0/2635431470 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd43806b4d0 0x7fd43806d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd448005880 tx=0x7fd448009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.413+0000 7fd456d9d640 1 --2- 172.21.15.16:0/2635431470 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x7fd4380736f0 0x7fd438075b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.413+0000 7fd456d9d640 1 --2- 172.21.15.16:0/2635431470 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x7fd4380736f0 0x7fd438075b10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.414+0000 7fd446ffd640 1 -- 172.21.15.16:0/2635431470 <== osd.6 v2:172.21.15.186:6816/1123185664 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd44c001d30 con 0x7fd4380736f0 2024-04-25T10:05:20.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.437+0000 7f29aebcf640 1 -- 172.21.15.16:0/1033477381 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2990002dd0 con 0x7f298c073740 2024-04-25T10:05:20.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.437+0000 7f299e7fc640 1 -- 172.21.15.16:0/1033477381 <== osd.2 v2:172.21.15.16:6800/2094356376 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2990002dd0 con 0x7f298c073740 2024-04-25T10:05:20.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.437+0000 7f70ea438640 1 -- 172.21.15.16:0/126556966 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f70e406f520 con 0x7f70d0077bd0 2024-04-25T10:05:20.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.438+0000 7f70e0ff9640 1 -- 172.21.15.16:0/126556966 <== osd.4 v2:172.21.15.186:6824/3091472889 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f70e406f520 con 0x7f70d0077bd0 2024-04-25T10:05:20.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.438+0000 7f298bfff640 1 -- 172.21.15.16:0/1033477381 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x7f298c073740 msgr2=0x7f298c075b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.438+0000 7f298bfff640 1 --2- 172.21.15.16:0/1033477381 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x7f298c073740 0x7f298c075b60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.439+0000 7f298bfff640 1 -- 172.21.15.16:0/1033477381 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f298c06b5a0 msgr2=0x7f298c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.439+0000 7f298bfff640 1 --2- 172.21.15.16:0/1033477381 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f298c06b5a0 0x7f298c06da60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2998006fd0 tx=0x7f2998008040 comp rx=0 tx=0).stop 2024-04-25T10:05:20.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.439+0000 7f298bfff640 1 -- 172.21.15.16:0/1033477381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29a8161d60 msgr2=0x7f29a8162320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.439+0000 7f298bfff640 1 --2- 172.21.15.16:0/1033477381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29a8161d60 0x7f29a8162320 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f29a000c710 tx=0x7f29a000cbd0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.439+0000 7f70ea438640 1 -- 172.21.15.16:0/126556966 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x7f70d0077bd0 msgr2=0x7f70d0079ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.439+0000 7f70ea438640 1 --2- 172.21.15.16:0/126556966 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x7f70d0077bd0 0x7f70d0079ff0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.439+0000 7f29ae3ce640 1 -- 172.21.15.16:0/1033477381 reap_dead start 2024-04-25T10:05:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.439+0000 7f70ea438640 1 -- 172.21.15.16:0/126556966 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f70d006f9b0 msgr2=0x7f70d0071e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.439+0000 7f70ea438640 1 --2- 172.21.15.16:0/126556966 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f70d006f9b0 0x7f70d0071e70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f70d8034000 tx=0x7f70d802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.439+0000 7f70ea438640 1 -- 172.21.15.16:0/126556966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70e40ccb20 msgr2=0x7f70e4143920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.439+0000 7f70ea438640 1 --2- 172.21.15.16:0/126556966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70e40ccb20 0x7f70e4143920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f70d400b410 tx=0x7f70d400b8d0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.439+0000 7f70e89d5640 1 -- 172.21.15.16:0/126556966 reap_dead start 2024-04-25T10:05:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.439+0000 7f70ea438640 1 -- 172.21.15.16:0/126556966 shutdown_connections 2024-04-25T10:05:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.439+0000 7f70ea438640 1 -- 172.21.15.16:0/126556966 >> 172.21.15.16:0/126556966 conn(0x7f70e406aba0 msgr2=0x7f70e40afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.439+0000 7f298bfff640 1 -- 172.21.15.16:0/1033477381 shutdown_connections 2024-04-25T10:05:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.439+0000 7f298bfff640 1 -- 172.21.15.16:0/1033477381 >> 172.21.15.16:0/1033477381 conn(0x7f29a8066ff0 msgr2=0x7f29a80af0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.439+0000 7f70ea438640 1 -- 172.21.15.16:0/126556966 shutdown_connections 2024-04-25T10:05:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.439+0000 7f70ea438640 1 -- 172.21.15.16:0/126556966 wait complete. 2024-04-25T10:05:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.440+0000 7f298bfff640 1 -- 172.21.15.16:0/1033477381 shutdown_connections 2024-04-25T10:05:20.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.440+0000 7f298bfff640 1 -- 172.21.15.16:0/1033477381 wait complete. 2024-04-25T10:05:20.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.440+0000 7fdfbe95f640 1 -- 172.21.15.16:0/1258957911 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdfb00553b0 con 0x7fdf9c0736f0 2024-04-25T10:05:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.441+0000 7fdfae7fc640 1 -- 172.21.15.16:0/1258957911 <== osd.1 v2:172.21.15.16:6808/1916704159 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdfb00553b0 con 0x7fdf9c0736f0 2024-04-25T10:05:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.442+0000 7fdf9bfff640 1 -- 172.21.15.16:0/1258957911 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x7fdf9c0736f0 msgr2=0x7fdf9c075b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.442+0000 7fdf9bfff640 1 --2- 172.21.15.16:0/1258957911 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x7fdf9c0736f0 0x7fdf9c075b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.442+0000 7f105541b640 1 -- 172.21.15.16:0/4080453851 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1050145260 con 0x7f103c0737c0 2024-04-25T10:05:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.442+0000 7fdf9bfff640 1 -- 172.21.15.16:0/1258957911 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdf9c06b4d0 msgr2=0x7fdf9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.442+0000 7fdf9bfff640 1 --2- 172.21.15.16:0/1258957911 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdf9c06b4d0 0x7fdf9c06d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4006fd0 tx=0x7fdfb4008040 comp rx=0 tx=0).stop 2024-04-25T10:05:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.442+0000 7fdf9bfff640 1 -- 172.21.15.16:0/1258957911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa4005370 msgr2=0x7fdfb00d9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.442+0000 7fdf9bfff640 1 --2- 172.21.15.16:0/1258957911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa4005370 0x7fdfb00d9600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800f850 tx=0x7fdfa800fd10 comp rx=0 tx=0).stop 2024-04-25T10:05:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.442+0000 7fdfbe15e640 1 -- 172.21.15.16:0/1258957911 reap_dead start 2024-04-25T10:05:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.442+0000 7fdf9bfff640 1 -- 172.21.15.16:0/1258957911 shutdown_connections 2024-04-25T10:05:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.442+0000 7fdf9bfff640 1 -- 172.21.15.16:0/1258957911 >> 172.21.15.16:0/1258957911 conn(0x7fdfb0055bc0 msgr2=0x7fdfb0051880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.442+0000 7f10377fe640 1 -- 172.21.15.16:0/4080453851 <== osd.0 v2:172.21.15.16:6816/2167750256 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1050145260 con 0x7f103c0737c0 2024-04-25T10:05:20.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.442+0000 7fdf9bfff640 1 -- 172.21.15.16:0/1258957911 shutdown_connections 2024-04-25T10:05:20.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.443+0000 7fdf9bfff640 1 -- 172.21.15.16:0/1258957911 wait complete. 2024-04-25T10:05:20.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.443+0000 7f68b6d1d640 1 -- 172.21.15.16:0/2164844211 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f68b00750b0 con 0x7f689c0737c0 2024-04-25T10:05:20.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.443+0000 7f105541b640 1 -- 172.21.15.16:0/4080453851 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x7f103c0737c0 msgr2=0x7f103c075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.443+0000 7f105541b640 1 --2- 172.21.15.16:0/4080453851 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x7f103c0737c0 0x7f103c075be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.444+0000 7f105541b640 1 -- 172.21.15.16:0/4080453851 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f103c06b5a0 msgr2=0x7f103c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.444+0000 7f105541b640 1 --2- 172.21.15.16:0/4080453851 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f103c06b5a0 0x7f103c06da60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f10500c7250 tx=0x7f1040009040 comp rx=0 tx=0).stop 2024-04-25T10:05:20.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.444+0000 7f105541b640 1 -- 172.21.15.16:0/4080453851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105015e630 msgr2=0x7f10501505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.444+0000 7f105541b640 1 --2- 172.21.15.16:0/4080453851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105015e630 0x7f10501505b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f104400e820 tx=0x7f104400ece0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.444+0000 7ff775ea7640 1 -- 172.21.15.16:0/3100934719 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff7700b2110 con 0x7ff75c073820 2024-04-25T10:05:20.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.444+0000 7f104f7fe640 1 -- 172.21.15.16:0/4080453851 reap_dead start 2024-04-25T10:05:20.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.444+0000 7f427cd3b640 1 -- 172.21.15.16:0/99488681 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f425c002dd0 con 0x7f42580737c0 2024-04-25T10:05:20.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.444+0000 7f68ad7fa640 1 -- 172.21.15.16:0/2164844211 <== osd.5 v2:172.21.15.186:6800/1827108421 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f68b00750b0 con 0x7f689c0737c0 2024-04-25T10:05:20.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.444+0000 7ff76d7fa640 1 -- 172.21.15.16:0/3100934719 <== osd.3 v2:172.21.15.16:6824/256577261 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff7700b2110 con 0x7ff75c073820 2024-04-25T10:05:20.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.444+0000 7f105541b640 1 -- 172.21.15.16:0/4080453851 shutdown_connections 2024-04-25T10:05:20.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.444+0000 7f105541b640 1 -- 172.21.15.16:0/4080453851 >> 172.21.15.16:0/4080453851 conn(0x7f10500d0750 msgr2=0x7f10500ada10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.444+0000 7f105541b640 1 -- 172.21.15.16:0/4080453851 shutdown_connections 2024-04-25T10:05:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.444+0000 7f105541b640 1 -- 172.21.15.16:0/4080453851 wait complete. 2024-04-25T10:05:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7ff756ffd640 1 -- 172.21.15.16:0/3100934719 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x7ff75c073820 msgr2=0x7ff75c075c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7ff756ffd640 1 --2- 172.21.15.16:0/3100934719 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x7ff75c073820 0x7ff75c075c40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7ff756ffd640 1 -- 172.21.15.16:0/3100934719 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff75c06b5a0 msgr2=0x7ff75c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7ff756ffd640 1 --2- 172.21.15.16:0/3100934719 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff75c06b5a0 0x7ff75c06da60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff7600014e0 tx=0x7ff760007990 comp rx=0 tx=0).stop 2024-04-25T10:05:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7f68b6d1d640 1 -- 172.21.15.16:0/2164844211 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x7f689c0737c0 msgr2=0x7f689c075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7f68b6d1d640 1 --2- 172.21.15.16:0/2164844211 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x7f689c0737c0 0x7f689c075be0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7f68b6d1d640 1 -- 172.21.15.16:0/2164844211 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f689c06b5a0 msgr2=0x7f689c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7f68b6d1d640 1 --2- 172.21.15.16:0/2164844211 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f689c06b5a0 0x7f689c06da60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f68b01600b0 tx=0x7f68a0008040 comp rx=0 tx=0).stop 2024-04-25T10:05:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7f68b6d1d640 1 -- 172.21.15.16:0/2164844211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68b006dee0 msgr2=0x7f68b015ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7f68b6d1d640 1 --2- 172.21.15.16:0/2164844211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68b006dee0 0x7f68b015ba50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009f40 tx=0x7f68a8004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7f68b52ba640 1 -- 172.21.15.16:0/2164844211 reap_dead start 2024-04-25T10:05:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7ff756ffd640 1 -- 172.21.15.16:0/3100934719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7700ca4c0 msgr2=0x7ff7700ca940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7ff756ffd640 1 --2- 172.21.15.16:0/3100934719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7700ca4c0 0x7ff7700ca940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff76800e200 tx=0x7ff768009750 comp rx=0 tx=0).stop 2024-04-25T10:05:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7f4266ffd640 1 -- 172.21.15.16:0/99488681 <== osd.7 v2:172.21.15.186:6808/1627307043 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f425c002dd0 con 0x7f42580737c0 2024-04-25T10:05:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7ff7756a6640 1 -- 172.21.15.16:0/3100934719 reap_dead start 2024-04-25T10:05:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7f4264ff9640 1 -- 172.21.15.16:0/99488681 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x7f42580737c0 msgr2=0x7f4258075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7f4264ff9640 1 --2- 172.21.15.16:0/99488681 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x7f42580737c0 0x7f4258075be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7f4264ff9640 1 -- 172.21.15.16:0/99488681 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f425806b5a0 msgr2=0x7f425806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.445+0000 7f4264ff9640 1 --2- 172.21.15.16:0/99488681 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f425806b5a0 0x7f425806da60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f426c034000 tx=0x7f426c02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.446+0000 7f4264ff9640 1 -- 172.21.15.16:0/99488681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42700eb050 msgr2=0x7f427017d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.446+0000 7f4264ff9640 1 --2- 172.21.15.16:0/99488681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42700eb050 0x7f427017d830 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f426800e870 tx=0x7f426800ed30 comp rx=0 tx=0).stop 2024-04-25T10:05:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.446+0000 7f4276d9d640 1 -- 172.21.15.16:0/99488681 reap_dead start 2024-04-25T10:05:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.446+0000 7ff756ffd640 1 -- 172.21.15.16:0/3100934719 shutdown_connections 2024-04-25T10:05:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.446+0000 7ff756ffd640 1 -- 172.21.15.16:0/3100934719 >> 172.21.15.16:0/3100934719 conn(0x7ff7700b3150 msgr2=0x7ff7700cfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.446+0000 7f4264ff9640 1 -- 172.21.15.16:0/99488681 shutdown_connections 2024-04-25T10:05:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.446+0000 7f4264ff9640 1 -- 172.21.15.16:0/99488681 >> 172.21.15.16:0/99488681 conn(0x7f42700559d0 msgr2=0x7f42700d7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.446+0000 7ff756ffd640 1 -- 172.21.15.16:0/3100934719 shutdown_connections 2024-04-25T10:05:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.446+0000 7f68b6d1d640 1 -- 172.21.15.16:0/2164844211 shutdown_connections 2024-04-25T10:05:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.446+0000 7f68b6d1d640 1 -- 172.21.15.16:0/2164844211 >> 172.21.15.16:0/2164844211 conn(0x7f68b006aba0 msgr2=0x7f68b00b22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.446+0000 7ff756ffd640 1 -- 172.21.15.16:0/3100934719 wait complete. 2024-04-25T10:05:20.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.446+0000 7f4264ff9640 1 -- 172.21.15.16:0/99488681 shutdown_connections 2024-04-25T10:05:20.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.446+0000 7f4264ff9640 1 -- 172.21.15.16:0/99488681 wait complete. 2024-04-25T10:05:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.446+0000 7f68b6d1d640 1 -- 172.21.15.16:0/2164844211 shutdown_connections 2024-04-25T10:05:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.447+0000 7f68b6d1d640 1 -- 172.21.15.16:0/2164844211 wait complete. 2024-04-25T10:05:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.447+0000 7fd45759e640 1 -- 172.21.15.16:0/2635431470 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd43c002dd0 con 0x7fd4380736f0 2024-04-25T10:05:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.447+0000 7fd446ffd640 1 -- 172.21.15.16:0/2635431470 <== osd.6 v2:172.21.15.186:6816/1123185664 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd43c002dd0 con 0x7fd4380736f0 2024-04-25T10:05:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.448+0000 7fd45759e640 1 -- 172.21.15.16:0/2635431470 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x7fd4380736f0 msgr2=0x7fd438075b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.448+0000 7fd45759e640 1 --2- 172.21.15.16:0/2635431470 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x7fd4380736f0 0x7fd438075b10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.449+0000 7fd45759e640 1 -- 172.21.15.16:0/2635431470 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd43806b4d0 msgr2=0x7fd43806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.449+0000 7fd45759e640 1 --2- 172.21.15.16:0/2635431470 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd43806b4d0 0x7fd43806d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd448005880 tx=0x7fd448009040 comp rx=0 tx=0).stop 2024-04-25T10:05:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.449+0000 7fd45759e640 1 -- 172.21.15.16:0/2635431470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd45806b280 msgr2=0x7fd458164360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.449+0000 7fd45759e640 1 --2- 172.21.15.16:0/2635431470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd45806b280 0x7fd458164360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd458072f70 tx=0x7fd44c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.449+0000 7fd456d9d640 1 -- 172.21.15.16:0/2635431470 reap_dead start 2024-04-25T10:05:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.449+0000 7fd45759e640 1 -- 172.21.15.16:0/2635431470 shutdown_connections 2024-04-25T10:05:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.449+0000 7fd45759e640 1 -- 172.21.15.16:0/2635431470 >> 172.21.15.16:0/2635431470 conn(0x7fd458099a40 msgr2=0x7fd4580adf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.449+0000 7fd45759e640 1 -- 172.21.15.16:0/2635431470 shutdown_connections 2024-04-25T10:05:20.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.449+0000 7fd45759e640 1 -- 172.21.15.16:0/2635431470 wait complete. 2024-04-25T10:05:20.555 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-04-25T10:05:20.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:05:20.557 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-04-25T10:05:20.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:05:20.559 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-04-25T10:05:20.559 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:05:20.561 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-04-25T10:05:20.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:05:20.562 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-04-25T10:05:20.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:05:20.564 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2024-04-25T10:05:20.565 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:05:20.566 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-04-25T10:05:20.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:05:20.568 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-04-25T10:05:20.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:05:20.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.764+0000 7f596083c640 1 Processor -- start 2024-04-25T10:05:20.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.765+0000 7fa206568640 1 Processor -- start 2024-04-25T10:05:20.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.765+0000 7fa206568640 1 -- start start 2024-04-25T10:05:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.765+0000 7f40e9797640 1 Processor -- start 2024-04-25T10:05:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.765+0000 7f596083c640 1 -- start start 2024-04-25T10:05:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.765+0000 7f40e9797640 1 -- start start 2024-04-25T10:05:20.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.766+0000 7f596083c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f595c1545e0 0x7f595c0b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.766+0000 7f596083c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f595c0b4a10 0x7f595c06bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.766+0000 7f596083c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f595c15b420 con 0x7f595c06c0f0 2024-04-25T10:05:20.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.766+0000 7f596083c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f595c0b4f30 con 0x7f595c0b4a10 2024-04-25T10:05:20.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.766+0000 7fa206568640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa1f80e1050 0x7fa1f80e2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.766+0000 7fa206568640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f803a160 con 0x7fa1f8033ea0 2024-04-25T10:05:20.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.766+0000 7fa206568640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f803a440 con 0x7fa1f80384a0 2024-04-25T10:05:20.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.766+0000 7fa206568640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f803aa90 con 0x7fa1f80e1050 2024-04-25T10:05:20.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.766+0000 7f596083c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595c0b2bc0 con 0x7f595c1545e0 2024-04-25T10:05:20.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.765+0000 7fdbb1a93640 1 Processor -- start 2024-04-25T10:05:20.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.765+0000 7fdbb1a93640 1 -- start start 2024-04-25T10:05:20.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7f595bfff640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f595c06c0f0 0x7f595c0b2ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:42682/0 (socket says 172.21.15.16:42682) 2024-04-25T10:05:20.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7f595bfff640 1 -- 172.21.15.16:0/113074155 learned_addr learned my addr 172.21.15.16:0/113074155 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:20.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7f595affd640 1 --2- 172.21.15.16:0/113074155 >> v2:172.21.15.186:3300/0 conn(0x7f595c0b4a10 0x7f595c06bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7f595b7fe640 1 --2- 172.21.15.16:0/113074155 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f595c1545e0 0x7f595c0b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.766+0000 7fdbb1a93640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdbac06f140 0x7fdbac13fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7fdbb1a93640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdbac15aa30 0x7fdbac15d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7fdbb1a93640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbac15dff0 con 0x7fdbac15db50 2024-04-25T10:05:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7fdbb1a93640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbac140300 con 0x7fdbac06f140 2024-04-25T10:05:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7fdbb1a93640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac15e2d0 con 0x7fdbac15aa30 2024-04-25T10:05:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7fdbab7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdbac06f140 0x7fdbac13fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7fdbab7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdbac06f140 0x7fdbac13fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42096/0 (socket says 172.21.15.16:42096) 2024-04-25T10:05:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7fdbab7fe640 1 -- 172.21.15.16:0/1810235752 learned_addr learned my addr 172.21.15.16:0/1810235752 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7fdbaaffd640 1 --2- 172.21.15.16:0/1810235752 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdbac15aa30 0x7fdbac15d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7fa1ff7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa1f8033ea0 0x7fa1f8037d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34860/0 (socket says 172.21.15.16:34860) 2024-04-25T10:05:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7fa1ff7fe640 1 -- 172.21.15.16:0/2617685640 learned_addr learned my addr 172.21.15.16:0/2617685640 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7fa1fffff640 1 --2- 172.21.15.16:0/2617685640 >> v2:172.21.15.186:3300/0 conn(0x7fa1f80e1050 0x7fa1f80e2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7fb4ddc51640 1 Processor -- start 2024-04-25T10:05:20.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7fdbaa7fc640 1 -- 172.21.15.16:0/1810235752 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 961831615 0 0) 0x7fdbac15dff0 con 0x7fdbac15db50 2024-04-25T10:05:20.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7fdbaa7fc640 1 -- 172.21.15.16:0/1810235752 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb94003520 con 0x7fdbac15db50 2024-04-25T10:05:20.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7fa1feffd640 1 -- 172.21.15.16:0/2617685640 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3922612451 0 0) 0x7fa1f803a160 con 0x7fa1f8033ea0 2024-04-25T10:05:20.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7fa1feffd640 1 -- 172.21.15.16:0/2617685640 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1f0003520 con 0x7fa1f8033ea0 2024-04-25T10:05:20.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7f40e9797640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40dc0dfee0 0x7f40dc0085c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7f595b7fe640 1 -- 172.21.15.16:0/113074155 >> v2:172.21.15.186:3300/0 conn(0x7f595c0b4a10 msgr2=0x7f595c06bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7f595b7fe640 1 --2- 172.21.15.16:0/113074155 >> v2:172.21.15.186:3300/0 conn(0x7f595c0b4a10 0x7f595c06bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7f595b7fe640 1 -- 172.21.15.16:0/113074155 >> v1:172.21.15.186:6789/0 conn(0x7f595c06c0f0 legacy=0x7f595c0b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7f595b7fe640 1 -- 172.21.15.16:0/113074155 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c15a660 con 0x7f595c1545e0 2024-04-25T10:05:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7fdbaaffd640 1 -- 172.21.15.16:0/1810235752 >> v2:172.21.15.16:3300/0 conn(0x7fdbac06f140 msgr2=0x7fdbac13fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7fdbaaffd640 1 --2- 172.21.15.16:0/1810235752 >> v2:172.21.15.16:3300/0 conn(0x7fdbac06f140 0x7fdbac13fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7fdbaaffd640 1 -- 172.21.15.16:0/1810235752 >> v1:172.21.15.16:6789/0 conn(0x7fdbac15db50 legacy=0x7fdbac0b17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7f40e9797640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f40dc0e02c0 0x7f40dc0e3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7fdbaaffd640 1 -- 172.21.15.16:0/1810235752 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbac15a230 con 0x7fdbac15aa30 2024-04-25T10:05:20.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7f40e3fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40dc0dfee0 0x7f40dc0085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7f40e3fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40dc0dfee0 0x7f40dc0085c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:34136/0 (socket says 172.21.15.16:34136) 2024-04-25T10:05:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.767+0000 7fb4ddc51640 1 -- start start 2024-04-25T10:05:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7f40e3fff640 1 -- 172.21.15.16:0/3128046098 learned_addr learned my addr 172.21.15.16:0/3128046098 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7f40e9797640 1 --2- 172.21.15.16:0/3128046098 >> v2:172.21.15.186:3300/0 conn(0x7f40dc0e40f0 0x7f40dc03a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7f40e9797640 1 -- 172.21.15.16:0/3128046098 --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40dc0e4540 con 0x7f40dc0e02c0 2024-04-25T10:05:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7f40e9797640 1 -- 172.21.15.16:0/3128046098 --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40dc0094a0 con 0x7f40dc0e40f0 2024-04-25T10:05:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7f40e9797640 1 -- 172.21.15.16:0/3128046098 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc056580 con 0x7f40dc0dfee0 2024-04-25T10:05:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7fb4ddc51640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4d0034010 0x7fb4d004fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7fb4ddc51640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb4d0006a40 0x7fb4d0006e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7fb4ddc51640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d0054c00 con 0x7fb4d004b3b0 2024-04-25T10:05:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7fb4ddc51640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d00062d0 con 0x7fb4d0034010 2024-04-25T10:05:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7fb4ddc51640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d0050110 con 0x7fb4d0006a40 2024-04-25T10:05:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7fa1feffd640 1 -- 172.21.15.16:0/2617685640 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3335283832 0 0) 0x7fa1f0003520 con 0x7fa1f8033ea0 2024-04-25T10:05:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7f595b7fe640 1 --2- 172.21.15.16:0/113074155 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f595c1545e0 0x7f595c0b44d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f595400aea0 tx=0x7f595402c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e6af95fe88a9361 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7fa1feffd640 1 -- 172.21.15.16:0/2617685640 >> v2:172.21.15.186:3300/0 conn(0x7fa1f80e1050 msgr2=0x7fa1f80e2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7fa1feffd640 1 --2- 172.21.15.16:0/2617685640 >> v2:172.21.15.186:3300/0 conn(0x7fa1f80e1050 0x7fa1f80e2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7fa1feffd640 1 -- 172.21.15.16:0/2617685640 >> v1:172.21.15.186:6789/0 conn(0x7fa1f80384a0 legacy=0x7fa1f804fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7fa1feffd640 1 -- 172.21.15.16:0/2617685640 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f803a720 con 0x7fa1f8033ea0 2024-04-25T10:05:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fa1feffd640 1 -- 172.21.15.16:0/2617685640 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fa1ec001670 con 0x7fa1f8033ea0 2024-04-25T10:05:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fa1feffd640 1 -- 172.21.15.16:0/2617685640 >> v1:172.21.15.16:6789/0 conn(0x7fa1f8033ea0 legacy=0x7fa1f8037d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fa1feffd640 1 --2- 172.21.15.16:0/2617685640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f00039b0 0x7fa1f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7f40e8f96640 1 --2- 172.21.15.16:0/3128046098 >> v2:172.21.15.186:3300/0 conn(0x7f40dc0e40f0 0x7f40dc03a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fe8a906b640 1 Processor -- start 2024-04-25T10:05:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7fb4dcc4f640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fb4d004b3b0 0x7fb4d004f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:42690/0 (socket says 172.21.15.16:42690) 2024-04-25T10:05:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7fb4dcc4f640 1 -- 172.21.15.16:0/3326657147 learned_addr learned my addr 172.21.15.16:0/3326657147 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7fb4dd450640 1 --2- 172.21.15.16:0/3326657147 >> v2:172.21.15.186:3300/0 conn(0x7fb4d0006a40 0x7fb4d0006e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fb4d77fe640 1 -- 172.21.15.16:0/3326657147 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812895569 0 0) 0x7fb4d0054c00 con 0x7fb4d004b3b0 2024-04-25T10:05:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fb4d77fe640 1 -- 172.21.15.16:0/3326657147 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4c8003520 con 0x7fb4d004b3b0 2024-04-25T10:05:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.768+0000 7f8e4a20d640 1 Processor -- start 2024-04-25T10:05:20.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7f8e4a20d640 1 -- start start 2024-04-25T10:05:20.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7f5cf2cb4640 1 Processor -- start 2024-04-25T10:05:20.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fa1feffd640 1 --2- 172.21.15.16:0/2617685640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1f0004310 0x7fa1f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fb4d77fe640 1 -- 172.21.15.16:0/3326657147 <== mon.1 v1:172.21.15.186:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3771125115 0 0) 0x7fb4c8003520 con 0x7fb4d004b3b0 2024-04-25T10:05:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fb4d77fe640 1 -- 172.21.15.16:0/3326657147 >> v2:172.21.15.186:3300/0 conn(0x7fb4d0006a40 msgr2=0x7fb4d0006e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fb4d77fe640 1 --2- 172.21.15.16:0/3326657147 >> v2:172.21.15.186:3300/0 conn(0x7fb4d0006a40 0x7fb4d0006e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fb4d77fe640 1 -- 172.21.15.16:0/3326657147 >> v2:172.21.15.16:3300/0 conn(0x7fb4d0034010 msgr2=0x7fb4d004fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fb4d77fe640 1 --2- 172.21.15.16:0/3326657147 >> v2:172.21.15.16:3300/0 conn(0x7fb4d0034010 0x7fb4d004fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fb4d77fe640 1 -- 172.21.15.16:0/3326657147 --> v1:172.21.15.186:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d0053a40 con 0x7fb4d004b3b0 2024-04-25T10:05:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fb4d77fe640 1 -- 172.21.15.16:0/3326657147 <== mon.1 v1:172.21.15.186:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fb4d80627f0 con 0x7fb4d004b3b0 2024-04-25T10:05:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fa1feffd640 1 --2- 172.21.15.16:0/2617685640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1f0004fe0 0x7fa1f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7f595a7fc640 1 -- 172.21.15.16:0/113074155 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954038070 con 0x7f595c1545e0 2024-04-25T10:05:20.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fa1feffd640 1 -- 172.21.15.16:0/2617685640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f803aa90 con 0x7fa1f00039b0 2024-04-25T10:05:20.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fe8a906b640 1 -- start start 2024-04-25T10:05:20.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fdbaaffd640 1 --2- 172.21.15.16:0/1810235752 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdbac15aa30 0x7fdbac15d610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdba00097a0 tx=0x7fdba002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b33c098d88661f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fa1feffd640 1 -- 172.21.15.16:0/2617685640 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f000bf30 con 0x7fa1f0004310 2024-04-25T10:05:20.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fa1feffd640 1 -- 172.21.15.16:0/2617685640 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f000c130 con 0x7fa1f0004fe0 2024-04-25T10:05:20.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7f595a7fc640 1 -- 172.21.15.16:0/113074155 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5954008640 con 0x7f595c1545e0 2024-04-25T10:05:20.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fb4d77fe640 1 -- 172.21.15.16:0/3326657147 >> v1:172.21.15.186:6789/0 conn(0x7fb4d004b3b0 legacy=0x7fb4d004f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fb4d77fe640 1 --2- 172.21.15.16:0/3326657147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4c80039b0 0x7fb4c8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fb4d77fe640 1 --2- 172.21.15.16:0/3326657147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4c8004370 0x7fb4c8004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fb4d77fe640 1 --2- 172.21.15.16:0/3326657147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c8004ff0 0x7fb4c800b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fb4d77fe640 1 -- 172.21.15.16:0/3326657147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d00062d0 con 0x7fb4c8004ff0 2024-04-25T10:05:20.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fb4d77fe640 1 -- 172.21.15.16:0/3326657147 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d0050110 con 0x7fb4c80039b0 2024-04-25T10:05:20.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fb4d77fe640 1 -- 172.21.15.16:0/3326657147 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c800bfa0 con 0x7fb4c8004370 2024-04-25T10:05:20.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fa204b05640 1 --2- 172.21.15.16:0/2617685640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1f0004fe0 0x7fa1f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fdbaa7fc640 1 -- 172.21.15.16:0/1810235752 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0037070 con 0x7fdbac15aa30 2024-04-25T10:05:20.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7f595a7fc640 1 -- 172.21.15.16:0/113074155 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954008a10 con 0x7f595c1545e0 2024-04-25T10:05:20.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fb4dcc4f640 1 --2- 172.21.15.16:0/3326657147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4c80039b0 0x7fb4c8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7fb4d7fff640 1 --2- 172.21.15.16:0/3326657147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4c8004370 0x7fb4c8004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7fdbaa7fc640 1 -- 172.21.15.16:0/1810235752 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba0032440 con 0x7fdbac15aa30 2024-04-25T10:05:20.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7fdbaa7fc640 1 -- 172.21.15.16:0/1810235752 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba00327f0 con 0x7fdbac15aa30 2024-04-25T10:05:20.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7f596083c640 1 -- 172.21.15.16:0/113074155 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f595c1545e0 msgr2=0x7f595c0b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7f596083c640 1 --2- 172.21.15.16:0/113074155 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f595c1545e0 0x7f595c0b44d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f595400aea0 tx=0x7f595402c7e0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f596083c640 1 -- 172.21.15.16:0/113074155 shutdown_connections 2024-04-25T10:05:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f596083c640 1 --2- 172.21.15.16:0/113074155 >> v2:172.21.15.186:3300/0 conn(0x7f595c0b4a10 0x7f595c06bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f596083c640 1 --2- 172.21.15.16:0/113074155 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f595c1545e0 0x7f595c0b44d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f596083c640 1 -- 172.21.15.16:0/113074155 >> 172.21.15.16:0/113074155 conn(0x7f595c071900 msgr2=0x7f595c071d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f596083c640 1 -- 172.21.15.16:0/113074155 shutdown_connections 2024-04-25T10:05:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7fa1fffff640 1 --2- 172.21.15.16:0/2617685640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f00039b0 0x7fa1f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7fa204b05640 1 -- 172.21.15.16:0/2617685640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1f0004310 msgr2=0x7fa1f0004800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:05:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7fa204b05640 1 --2- 172.21.15.16:0/2617685640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1f0004310 0x7fa1f0004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7fa204b05640 1 -- 172.21.15.16:0/2617685640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f00039b0 msgr2=0x7fa1f0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7f40e3fff640 1 -- 172.21.15.16:0/3128046098 >> v2:172.21.15.186:3300/0 conn(0x7f40dc0e40f0 msgr2=0x7f40dc03a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7f40e3fff640 1 --2- 172.21.15.16:0/3128046098 >> v2:172.21.15.186:3300/0 conn(0x7f40dc0e40f0 0x7f40dc03a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7f40e3fff640 1 -- 172.21.15.16:0/3128046098 >> v2:172.21.15.16:3300/0 conn(0x7f40dc0e02c0 msgr2=0x7f40dc0e3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7f40e3fff640 1 --2- 172.21.15.16:0/3128046098 >> v2:172.21.15.16:3300/0 conn(0x7f40dc0e02c0 0x7f40dc0e3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7f40e3fff640 1 -- 172.21.15.16:0/3128046098 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40dc056860 con 0x7f40dc0dfee0 2024-04-25T10:05:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7fb4d7fff640 1 -- 172.21.15.16:0/3326657147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4c80039b0 msgr2=0x7fb4c8003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7fb4d7fff640 1 --2- 172.21.15.16:0/3326657147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4c80039b0 0x7fb4c8003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7fb4d7fff640 1 -- 172.21.15.16:0/3326657147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c8004ff0 msgr2=0x7fb4c800b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7fb4d7fff640 1 --2- 172.21.15.16:0/3326657147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c8004ff0 0x7fb4c800b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7fb4d7fff640 1 -- 172.21.15.16:0/3326657147 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d0053a40 con 0x7fb4c8004370 2024-04-25T10:05:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7f8e4a20d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8e44071830 0x7f8e4415e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7f8e48a0a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8e44071830 0x7f8e4415e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f8e48a0a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8e44071830 0x7f8e4415e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:53610/0 (socket says 172.21.15.16:53610) 2024-04-25T10:05:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f8e48a0a640 1 -- 172.21.15.16:0/3560191395 learned_addr learned my addr 172.21.15.16:0/3560191395 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7f8e4a20d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e440cbae0 con 0x7f8e44159880 2024-04-25T10:05:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f8e4a20d640 1 -- 172.21.15.16:0/3560191395 --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4415d570 con 0x7f8e440cb470 2024-04-25T10:05:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f8e4a20d640 1 -- 172.21.15.16:0/3560191395 --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4415dbc0 con 0x7f8e44071830 2024-04-25T10:05:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.769+0000 7f5cf2cb4640 1 -- start start 2024-04-25T10:05:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7f5cf2cb4640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5cec159880 0x7f5cec15e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7f5cf2cb4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cec0cbae0 con 0x7f5cec0cb470 2024-04-25T10:05:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7f5cf2cb4640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cec15d570 con 0x7f5cec071830 2024-04-25T10:05:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7f5cf2cb4640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cec15dbc0 con 0x7f5cec159880 2024-04-25T10:05:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f5cf1cb2640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5cec159880 0x7f5cec15e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f5cf1cb2640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5cec159880 0x7f5cec15e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:53620/0 (socket says 172.21.15.16:53620) 2024-04-25T10:05:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f5cf1cb2640 1 -- 172.21.15.16:0/1803218606 learned_addr learned my addr 172.21.15.16:0/1803218606 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fdbb1a93640 1 -- 172.21.15.16:0/1810235752 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdbac15aa30 msgr2=0x7fdbac15d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fdbb1a93640 1 --2- 172.21.15.16:0/1810235752 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdbac15aa30 0x7fdbac15d610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdba00097a0 tx=0x7fdba002a210 comp rx=0 tx=0).stop 2024-04-25T10:05:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fdbb1a93640 1 -- 172.21.15.16:0/1810235752 shutdown_connections 2024-04-25T10:05:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fdbb1a93640 1 --2- 172.21.15.16:0/1810235752 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdbac15aa30 0x7fdbac15d610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fdbb1a93640 1 --2- 172.21.15.16:0/1810235752 >> v2:172.21.15.16:3300/0 conn(0x7fdbac06f140 0x7fdbac13fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fdbb1a93640 1 -- 172.21.15.16:0/1810235752 >> 172.21.15.16:0/1810235752 conn(0x7fdbac06aba0 msgr2=0x7fdbac06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7fa204b05640 1 --2- 172.21.15.16:0/2617685640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f00039b0 0x7fa1f0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f40e3fff640 1 --2- 172.21.15.16:0/3128046098 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40dc0dfee0 0x7f40dc0085c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00ae50 tx=0x7f40d802e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c78fc8ed4364b8b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fb4d7fff640 1 --2- 172.21.15.16:0/3326657147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4c8004370 0x7fb4c8004840 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009860 tx=0x7fb4c8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdd9505314efc929 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fdbb1a93640 1 -- 172.21.15.16:0/1810235752 shutdown_connections 2024-04-25T10:05:20.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7fa204b05640 1 -- 172.21.15.16:0/2617685640 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f0003520 con 0x7fa1f0004fe0 2024-04-25T10:05:20.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fa204b05640 1 -- 172.21.15.16:0/2617685640 reap_dead start 2024-04-25T10:05:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f596083c640 1 -- 172.21.15.16:0/113074155 wait complete. 2024-04-25T10:05:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fdbb1a93640 1 -- 172.21.15.16:0/1810235752 wait complete. 2024-04-25T10:05:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.772+0000 7fa204b05640 1 --2- 172.21.15.16:0/2617685640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1f0004fe0 0x7fa1f000b9c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa2000682b0 tx=0x7fa1f0005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76c2c1d47d7d86fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.772+0000 7fa1feffd640 1 -- 172.21.15.16:0/2617685640 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2000b2070 con 0x7fa1f0004fe0 2024-04-25T10:05:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.772+0000 7fa1feffd640 1 -- 172.21.15.16:0/2617685640 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2000a5c90 con 0x7fa1f0004fe0 2024-04-25T10:05:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f40e2ffd640 1 -- 172.21.15.16:0/3128046098 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d803a070 con 0x7f40dc0dfee0 2024-04-25T10:05:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.772+0000 7f40e2ffd640 1 -- 172.21.15.16:0/3128046098 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d8033710 con 0x7f40dc0dfee0 2024-04-25T10:05:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.772+0000 7f40e2ffd640 1 -- 172.21.15.16:0/3128046098 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d8033ac0 con 0x7f40dc0dfee0 2024-04-25T10:05:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7fe8a906b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe8a406dfa0 0x7fe8a415dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7fe8a906b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a40b36a0 con 0x7fe8a406b690 2024-04-25T10:05:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7fe8a906b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a40b3980 con 0x7fe8a4159910 2024-04-25T10:05:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.770+0000 7fe8a906b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a40b3fd0 con 0x7fe8a406dfa0 2024-04-25T10:05:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fe8a2d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe8a406b690 0x7fe8a415d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34880/0 (socket says 172.21.15.16:34880) 2024-04-25T10:05:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fe8a2d9d640 1 -- 172.21.15.16:0/3965834368 learned_addr learned my addr 172.21.15.16:0/3965834368 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fe8a259c640 1 --2- 172.21.15.16:0/3965834368 >> v2:172.21.15.186:3300/0 conn(0x7fe8a406dfa0 0x7fe8a415dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fe8a1d9b640 1 -- 172.21.15.16:0/3965834368 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30669428 0 0) 0x7fe8a40b36a0 con 0x7fe8a406b690 2024-04-25T10:05:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fe8a1d9b640 1 -- 172.21.15.16:0/3965834368 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe88c003520 con 0x7fe8a406b690 2024-04-25T10:05:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fe8a1d9b640 1 -- 172.21.15.16:0/3965834368 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871701330 0 0) 0x7fe8a40b3980 con 0x7fe8a4159910 2024-04-25T10:05:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.772+0000 7fe8a1d9b640 1 -- 172.21.15.16:0/3965834368 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe88c0048e0 con 0x7fe8a4159910 2024-04-25T10:05:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.772+0000 7fe8a259c640 1 -- 172.21.15.16:0/3965834368 >> v1:172.21.15.186:6789/0 conn(0x7fe8a4159910 legacy=0x7fe8a4159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.772+0000 7fe8a259c640 1 -- 172.21.15.16:0/3965834368 >> v1:172.21.15.16:6789/0 conn(0x7fe8a406b690 legacy=0x7fe8a415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.772+0000 7fe8a259c640 1 -- 172.21.15.16:0/3965834368 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a4160b50 con 0x7fe8a406dfa0 2024-04-25T10:05:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f8e43fff640 1 -- 172.21.15.16:0/3560191395 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257452244 0 0) 0x7f8e4415d570 con 0x7f8e440cb470 2024-04-25T10:05:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f8e43fff640 1 -- 172.21.15.16:0/3560191395 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e2c003520 con 0x7f8e440cb470 2024-04-25T10:05:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f8e48a0a640 1 -- 172.21.15.16:0/3560191395 >> v1:172.21.15.186:6789/0 conn(0x7f8e440cb470 legacy=0x7f8e44159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f8e48a0a640 1 -- 172.21.15.16:0/3560191395 >> v1:172.21.15.16:6789/0 conn(0x7f8e44159880 legacy=0x7f8e44071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f5cf1cb2640 1 -- 172.21.15.16:0/1803218606 >> v1:172.21.15.186:6789/0 conn(0x7f5cec071830 legacy=0x7f5cec159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f5cf1cb2640 1 -- 172.21.15.16:0/1803218606 >> v1:172.21.15.16:6789/0 conn(0x7f5cec0cb470 legacy=0x7f5cec071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7f5cf1cb2640 1 -- 172.21.15.16:0/1803218606 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec15d850 con 0x7f5cec159880 2024-04-25T10:05:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fb4d77fe640 1 -- 172.21.15.16:0/3326657147 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc037070 con 0x7fb4c8004370 2024-04-25T10:05:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fb4d77fe640 1 -- 172.21.15.16:0/3326657147 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4cc02aaa0 con 0x7fb4c8004370 2024-04-25T10:05:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fb4d77fe640 1 -- 172.21.15.16:0/3326657147 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc009460 con 0x7fb4c8004370 2024-04-25T10:05:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.771+0000 7fb4dd450640 1 -- 172.21.15.16:0/3326657147 reap_dead start 2024-04-25T10:05:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.772+0000 7fb4ddc51640 1 -- 172.21.15.16:0/3326657147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4c8004370 msgr2=0x7fb4c8004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.772+0000 7fb4ddc51640 1 --2- 172.21.15.16:0/3326657147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4c8004370 0x7fb4c8004840 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009860 tx=0x7fb4c8005830 comp rx=0 tx=0).stop 2024-04-25T10:05:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.772+0000 7fb4ddc51640 1 -- 172.21.15.16:0/3326657147 shutdown_connections 2024-04-25T10:05:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.772+0000 7fb4ddc51640 1 --2- 172.21.15.16:0/3326657147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4c8004370 0x7fb4c8004840 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7f5cf1cb2640 1 --2- 172.21.15.16:0/1803218606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec159880 0x7f5cec15e050 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009870 tx=0x7f5ce802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b530bdde516730a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.772+0000 7fb4ddc51640 1 -- 172.21.15.16:0/3326657147 >> 172.21.15.16:0/3326657147 conn(0x7fb4d000a3f0 msgr2=0x7fb4d000a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7f40e9797640 1 -- 172.21.15.16:0/3128046098 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40dc0dfee0 msgr2=0x7f40dc0085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7f40e9797640 1 --2- 172.21.15.16:0/3128046098 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40dc0dfee0 0x7f40dc0085c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00ae50 tx=0x7f40d802e6c0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7fb4ddc51640 1 -- 172.21.15.16:0/3326657147 shutdown_connections 2024-04-25T10:05:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7f8e48a0a640 1 -- 172.21.15.16:0/3560191395 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4415d850 con 0x7f8e44071830 2024-04-25T10:05:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7fdbb1a93640 1 Processor -- start 2024-04-25T10:05:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7fb4ddc51640 1 -- 172.21.15.16:0/3326657147 wait complete. 2024-04-25T10:05:20.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7fa206568640 1 -- 172.21.15.16:0/2617685640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1f0004fe0 msgr2=0x7fa1f000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7f596083c640 1 Processor -- start 2024-04-25T10:05:20.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7f40e9797640 1 -- 172.21.15.16:0/3128046098 shutdown_connections 2024-04-25T10:05:20.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7f40e9797640 1 --2- 172.21.15.16:0/3128046098 >> v2:172.21.15.186:3300/0 conn(0x7f40dc0e40f0 0x7f40dc03a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7f40e9797640 1 --2- 172.21.15.16:0/3128046098 >> v2:172.21.15.16:3300/0 conn(0x7f40dc0e02c0 0x7f40dc0e3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7f40e9797640 1 --2- 172.21.15.16:0/3128046098 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40dc0dfee0 0x7f40dc0085c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7f40e9797640 1 -- 172.21.15.16:0/3128046098 >> 172.21.15.16:0/3128046098 conn(0x7f40dc01ef30 msgr2=0x7f40dc055c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7fe8a259c640 1 --2- 172.21.15.16:0/3965834368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8a406dfa0 0x7fe8a415dee0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe8980097a0 tx=0x7fe89802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7eb6194fcfc2d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7fa206568640 1 --2- 172.21.15.16:0/2617685640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1f0004fe0 0x7fa1f000b9c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa2000682b0 tx=0x7fa1f0005830 comp rx=0 tx=0).stop 2024-04-25T10:05:20.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7fdbb1a93640 1 -- start start 2024-04-25T10:05:20.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7f596083c640 1 -- start start 2024-04-25T10:05:20.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7fe8a1d9b640 1 -- 172.21.15.16:0/3965834368 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898037070 con 0x7fe8a406dfa0 2024-04-25T10:05:20.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7fe8a1d9b640 1 -- 172.21.15.16:0/3965834368 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe898032440 con 0x7fe8a406dfa0 2024-04-25T10:05:20.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7fe8a1d9b640 1 -- 172.21.15.16:0/3965834368 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898032810 con 0x7fe8a406dfa0 2024-04-25T10:05:20.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7fa206568640 1 -- 172.21.15.16:0/2617685640 shutdown_connections 2024-04-25T10:05:20.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7fa206568640 1 --2- 172.21.15.16:0/2617685640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1f0004fe0 0x7fa1f000b9c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7fa206568640 1 -- 172.21.15.16:0/2617685640 >> 172.21.15.16:0/2617685640 conn(0x7fa1f800a430 msgr2=0x7fa1f800a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7f5cf0cb0640 1 -- 172.21.15.16:0/1803218606 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8037070 con 0x7f5cec159880 2024-04-25T10:05:20.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7f5cf0cb0640 1 -- 172.21.15.16:0/1803218606 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce8032440 con 0x7f5cec159880 2024-04-25T10:05:20.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7f40e9797640 1 -- 172.21.15.16:0/3128046098 shutdown_connections 2024-04-25T10:05:20.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7fdbb1a93640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdbac06f140 0x7fdbac1eeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7fdbb1a93640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbac15aa30 0x7fdbac0c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7fdbb1a93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbac15db50 0x7fdbac0c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7f40e9797640 1 -- 172.21.15.16:0/3128046098 wait complete. 2024-04-25T10:05:20.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7f8e48a0a640 1 --2- 172.21.15.16:0/3560191395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e44071830 0x7f8e4415e050 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8e4415dd40 tx=0x7f8e3402bbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0d69875ee0f36b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7fdbb1a93640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac0ceba0 con 0x7fdbac15db50 2024-04-25T10:05:20.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7fdbb1a93640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac0cea20 con 0x7fdbac15aa30 2024-04-25T10:05:20.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7fdbb1a93640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac0b1ee0 con 0x7fdbac06f140 2024-04-25T10:05:20.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7fe8a906b640 1 -- 172.21.15.16:0/3965834368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8a406dfa0 msgr2=0x7fe8a415dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7fe8a906b640 1 --2- 172.21.15.16:0/3965834368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8a406dfa0 0x7fe8a415dee0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe8980097a0 tx=0x7fe89802a230 comp rx=0 tx=0).stop 2024-04-25T10:05:20.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7f8e43fff640 1 -- 172.21.15.16:0/3560191395 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3402c940 con 0x7f8e44071830 2024-04-25T10:05:20.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7f8e43fff640 1 -- 172.21.15.16:0/3560191395 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3402cae0 con 0x7f8e44071830 2024-04-25T10:05:20.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7f8e43fff640 1 -- 172.21.15.16:0/3560191395 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3400a0d0 con 0x7f8e44071830 2024-04-25T10:05:20.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7fdbabfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbac15db50 0x7fdbac0c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.773+0000 7f5cf0cb0640 1 -- 172.21.15.16:0/1803218606 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8032810 con 0x7f5cec159880 2024-04-25T10:05:20.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7f5cf2cb4640 1 -- 172.21.15.16:0/1803218606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec159880 msgr2=0x7f5cec15e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7f5cf2cb4640 1 --2- 172.21.15.16:0/1803218606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec159880 0x7f5cec15e050 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009870 tx=0x7f5ce802a270 comp rx=0 tx=0).stop 2024-04-25T10:05:20.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7fdbabfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbac15db50 0x7fdbac0c5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42152/0 (socket says 172.21.15.16:42152) 2024-04-25T10:05:20.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7fdbabfff640 1 -- 172.21.15.16:0/129823462 learned_addr learned my addr 172.21.15.16:0/129823462 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.774+0000 7fa206568640 1 -- 172.21.15.16:0/2617685640 shutdown_connections 2024-04-25T10:05:20.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7f596083c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f595c06c0f0 0x7f595c14efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7fb4ddc51640 1 Processor -- start 2024-04-25T10:05:20.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7fdbab7fe640 1 --2- 172.21.15.16:0/129823462 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdbac06f140 0x7fdbac1eeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7fb4ddc51640 1 -- start start 2024-04-25T10:05:20.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7fdbaaffd640 1 --2- 172.21.15.16:0/129823462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbac15aa30 0x7fdbac0c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7f596083c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f595c0b4a10 0x7f595c1f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7fdbabfff640 1 -- 172.21.15.16:0/129823462 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdbac06f140 msgr2=0x7fdbac1eeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7fdbabfff640 1 --2- 172.21.15.16:0/129823462 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdbac06f140 0x7fdbac1eeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7fdbabfff640 1 -- 172.21.15.16:0/129823462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbac15aa30 msgr2=0x7fdbac0c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7fdbabfff640 1 --2- 172.21.15.16:0/129823462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbac15aa30 0x7fdbac0c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7fdbabfff640 1 -- 172.21.15.16:0/129823462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbac0c6600 con 0x7fdbac15db50 2024-04-25T10:05:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7f595b7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f595c06c0f0 0x7f595c14efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7f595b7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f595c06c0f0 0x7f595c14efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:34168/0 (socket says 172.21.15.16:34168) 2024-04-25T10:05:20.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7fdbabfff640 1 --2- 172.21.15.16:0/129823462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbac15db50 0x7fdbac0c5ff0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdba400c6c0 tx=0x7fdba400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7f595b7fe640 1 -- 172.21.15.16:0/1165395397 learned_addr learned my addr 172.21.15.16:0/1165395397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7f596083c640 1 --2- 172.21.15.16:0/1165395397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f595c1545e0 0x7f595c156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7f596083c640 1 -- 172.21.15.16:0/1165395397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c15af00 con 0x7f595c0b4a10 2024-04-25T10:05:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7f596083c640 1 -- 172.21.15.16:0/1165395397 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c15ac20 con 0x7f595c1545e0 2024-04-25T10:05:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7f596083c640 1 -- 172.21.15.16:0/1165395397 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595c0aed00 con 0x7f595c06c0f0 2024-04-25T10:05:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7fdb93fff640 1 -- 172.21.15.16:0/129823462 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba4003b70 con 0x7fdbac15db50 2024-04-25T10:05:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7fdbb1a93640 1 -- 172.21.15.16:0/129823462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbac140ea0 con 0x7fdbac15db50 2024-04-25T10:05:20.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7f8e4a20d640 1 -- 172.21.15.16:0/3560191395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e44071830 msgr2=0x7f8e4415e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7f8e4a20d640 1 --2- 172.21.15.16:0/3560191395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e44071830 0x7f8e4415e050 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8e4415dd40 tx=0x7f8e3402bbf0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7f595bfff640 1 --2- 172.21.15.16:0/1165395397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f595c1545e0 0x7f595c156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7f40e9797640 1 Processor -- start 2024-04-25T10:05:20.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7fb4ddc51640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4c8004370 0x7fb4d00e2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7f595b7fe640 1 -- 172.21.15.16:0/1165395397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f595c1545e0 msgr2=0x7f595c156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7f595b7fe640 1 --2- 172.21.15.16:0/1165395397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f595c1545e0 0x7f595c156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7f595b7fe640 1 -- 172.21.15.16:0/1165395397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f595c0b4a10 msgr2=0x7f595c1f8100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:05:20.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7fe8a906b640 1 -- 172.21.15.16:0/3965834368 shutdown_connections 2024-04-25T10:05:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7fe8a906b640 1 --2- 172.21.15.16:0/3965834368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8a406dfa0 0x7fe8a415dee0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7fe8a906b640 1 -- 172.21.15.16:0/3965834368 >> 172.21.15.16:0/3965834368 conn(0x7fe8a406aba0 msgr2=0x7fe8a406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7fb4ddc51640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4d00e9350 0x7fb4d0038f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7f40e9797640 1 -- start start 2024-04-25T10:05:20.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7fb4ddc51640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4d00e3fc0 0x7fb4d00e4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7fb4d7fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4d00e9350 0x7fb4d0038f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7fb4dd450640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4d00e3fc0 0x7fb4d00e4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7f595affd640 1 --2- 172.21.15.16:0/1165395397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f595c0b4a10 0x7f595c1f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7fa206568640 1 -- 172.21.15.16:0/2617685640 wait complete. 2024-04-25T10:05:20.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7fa206568640 1 Processor -- start 2024-04-25T10:05:20.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7fb4d7fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4d00e9350 0x7fb4d0038f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:34174/0 (socket says 172.21.15.16:34174) 2024-04-25T10:05:20.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7fb4dcc4f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4c8004370 0x7fb4d00e2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7fb4dd450640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4d00e3fc0 0x7fb4d00e4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42174/0 (socket says 172.21.15.16:42174) 2024-04-25T10:05:20.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7fb4d7fff640 1 -- 172.21.15.16:0/171125399 learned_addr learned my addr 172.21.15.16:0/171125399 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7fb4ddc51640 1 -- 172.21.15.16:0/171125399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d004b790 con 0x7fb4d00e3fc0 2024-04-25T10:05:20.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7fb4ddc51640 1 -- 172.21.15.16:0/171125399 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d0006520 con 0x7fb4c8004370 2024-04-25T10:05:20.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7fb4ddc51640 1 -- 172.21.15.16:0/171125399 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d0054820 con 0x7fb4d00e9350 2024-04-25T10:05:20.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7f595b7fe640 1 --2- 172.21.15.16:0/1165395397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f595c0b4a10 0x7f595c1f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.777+0000 7fe8a906b640 1 -- 172.21.15.16:0/3965834368 shutdown_connections 2024-04-25T10:05:20.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7f5cf2cb4640 1 -- 172.21.15.16:0/1803218606 shutdown_connections 2024-04-25T10:05:20.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7f5cf2cb4640 1 --2- 172.21.15.16:0/1803218606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec159880 0x7f5cec15e050 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7f5cf2cb4640 1 -- 172.21.15.16:0/1803218606 >> 172.21.15.16:0/1803218606 conn(0x7f5cec06aba0 msgr2=0x7f5cec06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.776+0000 7fdb93fff640 1 -- 172.21.15.16:0/129823462 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba4003d10 con 0x7fdbac15db50 2024-04-25T10:05:20.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.777+0000 7f8e4a20d640 1 -- 172.21.15.16:0/3560191395 shutdown_connections 2024-04-25T10:05:20.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7f5cf2cb4640 1 -- 172.21.15.16:0/1803218606 shutdown_connections 2024-04-25T10:05:20.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.775+0000 7f5cf2cb4640 1 -- 172.21.15.16:0/1803218606 wait complete. 2024-04-25T10:05:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.777+0000 7f5cf2cb4640 1 Processor -- start 2024-04-25T10:05:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.777+0000 7f8e4a20d640 1 --2- 172.21.15.16:0/3560191395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e44071830 0x7f8e4415e050 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.777+0000 7f595b7fe640 1 -- 172.21.15.16:0/1165395397 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c1f8b30 con 0x7f595c06c0f0 2024-04-25T10:05:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.777+0000 7f595affd640 1 --2- 172.21.15.16:0/1165395397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f595c0b4a10 0x7f595c1f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.777+0000 7fe8a906b640 1 -- 172.21.15.16:0/3965834368 wait complete. 2024-04-25T10:05:20.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.777+0000 7f8e4a20d640 1 -- 172.21.15.16:0/3560191395 >> 172.21.15.16:0/3560191395 conn(0x7f8e4406aba0 msgr2=0x7f8e4406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.777+0000 7f595b7fe640 1 --2- 172.21.15.16:0/1165395397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f595c06c0f0 0x7f595c14efb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f595402c3c0 tx=0x7f5954004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.777+0000 7f8e4a20d640 1 -- 172.21.15.16:0/3560191395 shutdown_connections 2024-04-25T10:05:20.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.777+0000 7f8e4a20d640 1 -- 172.21.15.16:0/3560191395 wait complete. 2024-04-25T10:05:20.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.777+0000 7f5cf2cb4640 1 -- start start 2024-04-25T10:05:20.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.777+0000 7fa206568640 1 -- start start 2024-04-25T10:05:20.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.777+0000 7f40e9797640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40dc0dfee0 0x7f40dc0de770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.777+0000 7fdb93fff640 1 -- 172.21.15.16:0/129823462 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba4005310 con 0x7fdbac15db50 2024-04-25T10:05:20.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7fdbb1a93640 1 -- 172.21.15.16:0/129823462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbac141460 con 0x7fdbac15db50 2024-04-25T10:05:20.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.777+0000 7fb4d7fff640 1 -- 172.21.15.16:0/171125399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4c8004370 msgr2=0x7fb4d00e2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.777+0000 7fb4d7fff640 1 --2- 172.21.15.16:0/171125399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4c8004370 0x7fb4d00e2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7f5943fff640 1 -- 172.21.15.16:0/1165395397 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954038050 con 0x7f595c06c0f0 2024-04-25T10:05:20.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7f40e9797640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40dc0e40f0 0x7f40dc0da370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.777+0000 7fb4d7fff640 1 -- 172.21.15.16:0/171125399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4d00e3fc0 msgr2=0x7fb4d00e4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7f5943fff640 1 -- 172.21.15.16:0/1165395397 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5954039040 con 0x7f595c06c0f0 2024-04-25T10:05:20.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7f40e3fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40dc0dfee0 0x7f40dc0de770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7f596083c640 1 -- 172.21.15.16:0/1165395397 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595c1f8e10 con 0x7f595c06c0f0 2024-04-25T10:05:20.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7f40e3fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40dc0dfee0 0x7f40dc0de770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:53672/0 (socket says 172.21.15.16:53672) 2024-04-25T10:05:20.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7f596083c640 1 -- 172.21.15.16:0/1165395397 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595c154e40 con 0x7f595c06c0f0 2024-04-25T10:05:20.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7f40e37fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40dc0e40f0 0x7f40dc0da370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7f5cf2cb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cec071830 0x7f5cec1559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7f5cf2cb4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec0cb470 0x7f5cec154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7f5cf2cb4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5cec159880 0x7f5cec203a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7f5cf2cb4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec0cdf70 con 0x7f5cec071830 2024-04-25T10:05:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7f5cf2cb4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec0cddf0 con 0x7f5cec0cb470 2024-04-25T10:05:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7f5cf2cb4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec0ce0f0 con 0x7f5cec159880 2024-04-25T10:05:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7fb4d7fff640 1 --2- 172.21.15.16:0/171125399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4d00e3fc0 0x7fb4d00e4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7fb4d7fff640 1 -- 172.21.15.16:0/171125399 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d00e98c0 con 0x7fb4d00e9350 2024-04-25T10:05:20.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7fb4dcc4f640 1 --2- 172.21.15.16:0/171125399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4c8004370 0x7fb4d00e2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:20.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7fb4dd450640 1 --2- 172.21.15.16:0/171125399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4d00e3fc0 0x7fb4d00e4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:20.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7f40e37fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40dc0e40f0 0x7f40dc0da370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:34186/0 (socket says 172.21.15.16:34186) 2024-04-25T10:05:20.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7f40e9797640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40dc0dd090 0x7f40dc1777e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.778+0000 7fb4d7fff640 1 --2- 172.21.15.16:0/171125399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4d00e9350 0x7fb4d0038f50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009950 tx=0x7fb4cc0047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.779+0000 7fa206568640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1f0004fe0 0x7fa1f8035390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.779+0000 7fb4d57fa640 1 -- 172.21.15.16:0/171125399 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc037070 con 0x7fb4d00e9350 2024-04-25T10:05:20.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.779+0000 7f5cf24b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5cec159880 0x7f5cec203a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.779+0000 7fdb93fff640 1 -- 172.21.15.16:0/129823462 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdba40054f0 con 0x7fdbac15db50 2024-04-25T10:05:20.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.779+0000 7fb4d57fa640 1 -- 172.21.15.16:0/171125399 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4cc038040 con 0x7fb4d00e9350 2024-04-25T10:05:20.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.779+0000 7fa206568640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1f804e0e0 0x7fa1f8036aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.779+0000 7f40e8f96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40dc0dd090 0x7f40dc1777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.779+0000 7fa206568640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f80ecd70 0x7fa1f804a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.779+0000 7f40e8f96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40dc0dd090 0x7f40dc1777e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42200/0 (socket says 172.21.15.16:42200) 2024-04-25T10:05:20.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.779+0000 7f40e3fff640 1 -- 172.21.15.16:0/3977823789 learned_addr learned my addr 172.21.15.16:0/3977823789 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.779+0000 7fe8a906b640 1 Processor -- start 2024-04-25T10:05:20.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.779+0000 7f40e9797640 1 -- 172.21.15.16:0/3977823789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc056b40 con 0x7f40dc0dd090 2024-04-25T10:05:20.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.779+0000 7f40e9797640 1 -- 172.21.15.16:0/3977823789 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc009130 con 0x7f40dc0dfee0 2024-04-25T10:05:20.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.779+0000 7f40e9797640 1 -- 172.21.15.16:0/3977823789 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc008b00 con 0x7f40dc0e40f0 2024-04-25T10:05:20.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.779+0000 7fe8a906b640 1 -- start start 2024-04-25T10:05:20.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.779+0000 7fa204b05640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f80ecd70 0x7fa1f804a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7fa204b05640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f80ecd70 0x7fa1f804a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42214/0 (socket says 172.21.15.16:42214) 2024-04-25T10:05:20.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7fdb93fff640 1 --2- 172.21.15.16:0/129823462 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb9806b5a0 0x7fdb9806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7f5943fff640 1 -- 172.21.15.16:0/1165395397 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954003b90 con 0x7f595c06c0f0 2024-04-25T10:05:20.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7f8e4a20d640 1 Processor -- start 2024-04-25T10:05:20.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7fb4d57fa640 1 -- 172.21.15.16:0/171125399 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc003c00 con 0x7fb4d00e9350 2024-04-25T10:05:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7fdb93fff640 1 -- 172.21.15.16:0/129823462 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdba4089790 con 0x7fdbac15db50 2024-04-25T10:05:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7fdbab7fe640 1 --2- 172.21.15.16:0/129823462 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb9806b5a0 0x7fdb9806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7f8e4a20d640 1 -- start start 2024-04-25T10:05:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7f5cf1cb2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cec071830 0x7f5cec1559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7f5cf1cb2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cec071830 0x7f5cec1559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42186/0 (socket says 172.21.15.16:42186) 2024-04-25T10:05:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7f5cf1cb2640 1 -- 172.21.15.16:0/421409675 learned_addr learned my addr 172.21.15.16:0/421409675 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7fb4ddc51640 1 -- 172.21.15.16:0/171125399 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d00e9ba0 con 0x7fb4d00e9350 2024-04-25T10:05:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7fdbb1a93640 1 -- 172.21.15.16:0/129823462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbac0749b0 con 0x7fdbac15db50 2024-04-25T10:05:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7fa206568640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f8034320 con 0x7fa1f80ecd70 2024-04-25T10:05:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7fa206568640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f803af20 con 0x7fa1f804e0e0 2024-04-25T10:05:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7fa206568640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f80344a0 con 0x7fa1f0004fe0 2024-04-25T10:05:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7fa204b05640 1 -- 172.21.15.16:0/2326652357 learned_addr learned my addr 172.21.15.16:0/2326652357 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7fa1fffff640 1 --2- 172.21.15.16:0/2326652357 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1f0004fe0 0x7fa1f8035390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7fa1ff7fe640 1 --2- 172.21.15.16:0/2326652357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1f804e0e0 0x7fa1f8036aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7f40e37fe640 1 -- 172.21.15.16:0/3977823789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40dc0dfee0 msgr2=0x7f40dc0de770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7f40e37fe640 1 --2- 172.21.15.16:0/3977823789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40dc0dfee0 0x7f40dc0de770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7f40e37fe640 1 -- 172.21.15.16:0/3977823789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40dc0dd090 msgr2=0x7f40dc1777e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7f40e37fe640 1 --2- 172.21.15.16:0/3977823789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40dc0dd090 0x7f40dc1777e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7f40e37fe640 1 -- 172.21.15.16:0/3977823789 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40dc0dd570 con 0x7f40dc0e40f0 2024-04-25T10:05:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7f40e8f96640 1 --2- 172.21.15.16:0/3977823789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40dc0dd090 0x7f40dc1777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7fe8a906b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a406b690 0x7fe8a41f0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7fe8a906b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8a406dfa0 0x7fe8a41f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7fe8a906b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8a4159910 0x7fe8a415f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7fe8a906b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a415b4a0 con 0x7fe8a406b690 2024-04-25T10:05:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7fe8a906b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40b3170 con 0x7fe8a4159910 2024-04-25T10:05:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7fe8a906b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a415e420 con 0x7fe8a406dfa0 2024-04-25T10:05:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7fe8a359e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8a4159910 0x7fe8a415f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7fe8a359e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8a4159910 0x7fe8a415f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:53710/0 (socket says 172.21.15.16:53710) 2024-04-25T10:05:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7fe8a359e640 1 -- 172.21.15.16:0/2824136036 learned_addr learned my addr 172.21.15.16:0/2824136036 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7fb4ddc51640 1 -- 172.21.15.16:0/171125399 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d00ea160 con 0x7fb4d00e9350 2024-04-25T10:05:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7f5cf14b1640 1 --2- 172.21.15.16:0/421409675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec0cb470 0x7f5cec154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7f5cf24b3640 1 -- 172.21.15.16:0/421409675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec0cb470 msgr2=0x7f5cec154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7f5cf24b3640 1 --2- 172.21.15.16:0/421409675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec0cb470 0x7f5cec154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7f5cf24b3640 1 -- 172.21.15.16:0/421409675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cec071830 msgr2=0x7f5cec1559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7f5cf24b3640 1 --2- 172.21.15.16:0/421409675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cec071830 0x7f5cec1559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.780+0000 7f5cf24b3640 1 -- 172.21.15.16:0/421409675 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec1561f0 con 0x7f5cec159880 2024-04-25T10:05:20.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7fa1ff7fe640 1 -- 172.21.15.16:0/2326652357 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1f0004fe0 msgr2=0x7fa1f8035390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7fa1ff7fe640 1 --2- 172.21.15.16:0/2326652357 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1f0004fe0 0x7fa1f8035390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7fa1ff7fe640 1 -- 172.21.15.16:0/2326652357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f80ecd70 msgr2=0x7fa1f804a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7fa1ff7fe640 1 --2- 172.21.15.16:0/2326652357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f80ecd70 0x7fa1f804a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7fa1ff7fe640 1 -- 172.21.15.16:0/2326652357 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f8035a00 con 0x7fa1f804e0e0 2024-04-25T10:05:20.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7f5941ffb640 1 -- 172.21.15.16:0/1165395397 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595c0749b0 con 0x7f595c06c0f0 2024-04-25T10:05:20.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7fe8a259c640 1 --2- 172.21.15.16:0/2824136036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8a406dfa0 0x7fe8a41f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7f40e37fe640 1 --2- 172.21.15.16:0/3977823789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40dc0e40f0 0x7f40dc0da370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f40d000b3f0 tx=0x7f40d000b7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7fe8a359e640 1 -- 172.21.15.16:0/2824136036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8a406dfa0 msgr2=0x7fe8a41f2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7fe8a359e640 1 --2- 172.21.15.16:0/2824136036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8a406dfa0 0x7fe8a41f2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7fe8a359e640 1 -- 172.21.15.16:0/2824136036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a406b690 msgr2=0x7fe8a41f0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7fe8a359e640 1 --2- 172.21.15.16:0/2824136036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a406b690 0x7fe8a41f0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7fe8a359e640 1 -- 172.21.15.16:0/2824136036 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a415e900 con 0x7fe8a4159910 2024-04-25T10:05:20.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7f40e0ff9640 1 -- 172.21.15.16:0/3977823789 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d0015020 con 0x7f40dc0e40f0 2024-04-25T10:05:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7fa1ff7fe640 1 --2- 172.21.15.16:0/2326652357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1f804e0e0 0x7fa1f8036aa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00c680 tx=0x7fa1ec00cb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.781+0000 7f40e0ff9640 1 -- 172.21.15.16:0/3977823789 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d0011830 con 0x7f40dc0e40f0 2024-04-25T10:05:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7f40e0ff9640 1 -- 172.21.15.16:0/3977823789 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d0011c20 con 0x7f40dc0e40f0 2024-04-25T10:05:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7f40e9797640 1 -- 172.21.15.16:0/3977823789 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40dc0dd850 con 0x7f40dc0e40f0 2024-04-25T10:05:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7f8e4a20d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e44071830 0x7f8e44150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7fe8a359e640 1 --2- 172.21.15.16:0/2824136036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8a4159910 0x7fe8a415f0c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00c6c0 tx=0x7fe89c00cb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7f40e9797640 1 -- 172.21.15.16:0/3977823789 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40dc0dde10 con 0x7f40dc0e40f0 2024-04-25T10:05:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7f8e4a20d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e440cb470 0x7f8e440c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7f5943fff640 1 -- 172.21.15.16:0/1165395397 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5954033610 con 0x7f595c06c0f0 2024-04-25T10:05:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7fe88b7fe640 1 -- 172.21.15.16:0/2824136036 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c003b70 con 0x7fe8a4159910 2024-04-25T10:05:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7f8e4a20d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e44159880 0x7f8e44157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7f8e4a20d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4406f390 con 0x7f8e440cb470 2024-04-25T10:05:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7f8e4a20d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4406f210 con 0x7f8e44159880 2024-04-25T10:05:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7f8e4a20d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440cddf0 con 0x7f8e44071830 2024-04-25T10:05:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7f5943fff640 1 --2- 172.21.15.16:0/1165395397 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f594806b520 0x7f594806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7fe8a906b640 1 -- 172.21.15.16:0/2824136036 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a41f3540 con 0x7fe8a4159910 2024-04-25T10:05:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7fe88b7fe640 1 -- 172.21.15.16:0/2824136036 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c003d10 con 0x7fe8a4159910 2024-04-25T10:05:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7fe88b7fe640 1 -- 172.21.15.16:0/2824136036 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c005310 con 0x7fe8a4159910 2024-04-25T10:05:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7fa1fcff9640 1 -- 172.21.15.16:0/2326652357 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec018070 con 0x7fa1f804e0e0 2024-04-25T10:05:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7fa206568640 1 -- 172.21.15.16:0/2326652357 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f8035ce0 con 0x7fa1f804e0e0 2024-04-25T10:05:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7fa206568640 1 -- 172.21.15.16:0/2326652357 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f80362a0 con 0x7fa1f804e0e0 2024-04-25T10:05:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7fa1fcff9640 1 -- 172.21.15.16:0/2326652357 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec0072d0 con 0x7fa1f804e0e0 2024-04-25T10:05:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7fa1fcff9640 1 -- 172.21.15.16:0/2326652357 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0047e0 con 0x7fa1f804e0e0 2024-04-25T10:05:20.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7f5943fff640 1 -- 172.21.15.16:0/1165395397 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59540aa530 con 0x7f595c06c0f0 2024-04-25T10:05:20.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7f5cf24b3640 1 --2- 172.21.15.16:0/421409675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5cec159880 0x7f5cec203a20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00c410 tx=0x7f5cdc00c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.783+0000 7fb4d57fa640 1 -- 172.21.15.16:0/171125399 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb4cc03e480 con 0x7fb4d00e9350 2024-04-25T10:05:20.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.783+0000 7fb4d57fa640 1 --2- 172.21.15.16:0/171125399 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4bc06b5a0 0x7fb4bc06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.783+0000 7fb4dcc4f640 1 --2- 172.21.15.16:0/171125399 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4bc06b5a0 0x7fb4bc06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.782+0000 7fe8a906b640 1 -- 172.21.15.16:0/2824136036 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a41f3b00 con 0x7fe8a4159910 2024-04-25T10:05:20.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.784+0000 7f595affd640 1 --2- 172.21.15.16:0/1165395397 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f594806b520 0x7f594806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.784+0000 7fa206568640 1 -- 172.21.15.16:0/2326652357 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e0005290 con 0x7fa1f804e0e0 2024-04-25T10:05:20.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.783+0000 7fb4d57fa640 1 -- 172.21.15.16:0/171125399 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4cc0aa970 con 0x7fb4d00e9350 2024-04-25T10:05:20.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.784+0000 7fdbab7fe640 1 --2- 172.21.15.16:0/129823462 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb9806b5a0 0x7fdb9806da60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c018690 tx=0x7fdb9c00f9e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.784+0000 7fe8a906b640 1 -- 172.21.15.16:0/2824136036 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a40750b0 con 0x7fe8a4159910 2024-04-25T10:05:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.784+0000 7f40e9797640 1 -- 172.21.15.16:0/3977823789 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40dc0057d0 con 0x7f40dc0e40f0 2024-04-25T10:05:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.784+0000 7fb4dcc4f640 1 --2- 172.21.15.16:0/171125399 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4bc06b5a0 0x7fb4bc06da60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8050190 tx=0x7fb4d806e040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.784+0000 7fb4ddc51640 1 -- 172.21.15.16:0/171125399 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d0008e10 con 0x7fb4d00e9350 2024-04-25T10:05:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.784+0000 7f8e4920b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e44071830 0x7f8e44150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.784+0000 7f595affd640 1 --2- 172.21.15.16:0/1165395397 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f594806b520 0x7f594806d9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f595c06b0f0 tx=0x7f5950008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.784+0000 7f8e48a0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e440cb470 0x7f8e440c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.784+0000 7f8e48a0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e440cb470 0x7f8e440c9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42238/0 (socket says 172.21.15.16:42238) 2024-04-25T10:05:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.784+0000 7f8e48a0a640 1 -- 172.21.15.16:0/107625945 learned_addr learned my addr 172.21.15.16:0/107625945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.784+0000 7f8e49a0c640 1 --2- 172.21.15.16:0/107625945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e44159880 0x7f8e44157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.785+0000 7fe88b7fe640 1 -- 172.21.15.16:0/2824136036 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe89c0054b0 con 0x7fe8a4159910 2024-04-25T10:05:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.785+0000 7fa1fcff9640 1 -- 172.21.15.16:0/2326652357 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1ec01b070 con 0x7fa1f804e0e0 2024-04-25T10:05:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.785+0000 7f8e4920b640 1 -- 172.21.15.16:0/107625945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e44159880 msgr2=0x7f8e44157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.785+0000 7f8e4920b640 1 --2- 172.21.15.16:0/107625945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e44159880 0x7f8e44157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.785+0000 7f8e4920b640 1 -- 172.21.15.16:0/107625945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e440cb470 msgr2=0x7f8e440c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.785+0000 7f8e4920b640 1 --2- 172.21.15.16:0/107625945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e440cb470 0x7f8e440c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.785+0000 7f8e4920b640 1 -- 172.21.15.16:0/107625945 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e440c89d0 con 0x7f8e44071830 2024-04-25T10:05:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.785+0000 7f8e49a0c640 1 --2- 172.21.15.16:0/107625945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e44159880 0x7f8e44157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.785+0000 7f8e4920b640 1 --2- 172.21.15.16:0/107625945 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e44071830 0x7f8e44150980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00c390 tx=0x7f8e3c00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.785+0000 7fe88b7fe640 1 --2- 172.21.15.16:0/2824136036 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe89006b550 0x7fe89006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.786+0000 7fa1fcff9640 1 --2- 172.21.15.16:0/2326652357 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa1e406b5a0 0x7fa1e406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.786+0000 7fe88b7fe640 1 -- 172.21.15.16:0/2824136036 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe89c089f40 con 0x7fe8a4159910 2024-04-25T10:05:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.786+0000 7fa1fcff9640 1 -- 172.21.15.16:0/2326652357 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1ec089a10 con 0x7fa1f804e0e0 2024-04-25T10:05:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.786+0000 7fe8a2d9d640 1 --2- 172.21.15.16:0/2824136036 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe89006b550 0x7fe89006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.786+0000 7fa1fffff640 1 --2- 172.21.15.16:0/2326652357 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa1e406b5a0 0x7fa1e406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.786+0000 7fe8a2d9d640 1 --2- 172.21.15.16:0/2824136036 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe89006b550 0x7fe89006da10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe8940011e0 tx=0x7fe894007890 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.786+0000 7fa1fffff640 1 --2- 172.21.15.16:0/2326652357 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa1e406b5a0 0x7fa1e406da60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8037aa0 tx=0x7fa1f4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.788+0000 7fdb93fff640 1 -- 172.21.15.16:0/129823462 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdba408c030 con 0x7fdbac15db50 2024-04-25T10:05:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.791+0000 7f5943fff640 1 -- 172.21.15.16:0/1165395397 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5954079390 con 0x7f595c06c0f0 2024-04-25T10:05:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.791+0000 7f8e41ffb640 1 -- 172.21.15.16:0/107625945 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c017070 con 0x7f8e44071830 2024-04-25T10:05:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.791+0000 7f8e4a20d640 1 -- 172.21.15.16:0/107625945 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e440c8c70 con 0x7f8e44071830 2024-04-25T10:05:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.791+0000 7f8e4a20d640 1 -- 172.21.15.16:0/107625945 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e440c9230 con 0x7f8e44071830 2024-04-25T10:05:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.791+0000 7f5ce27fc640 1 -- 172.21.15.16:0/421409675 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc011790 con 0x7f5cec159880 2024-04-25T10:05:20.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.792+0000 7f5cf2cb4640 1 -- 172.21.15.16:0/421409675 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cec1564d0 con 0x7f5cec159880 2024-04-25T10:05:20.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.792+0000 7f5cf2cb4640 1 -- 172.21.15.16:0/421409675 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec156a90 con 0x7f5cec159880 2024-04-25T10:05:20.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.792+0000 7f8e4a20d640 1 -- 172.21.15.16:0/107625945 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e440749b0 con 0x7f8e44071830 2024-04-25T10:05:20.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.792+0000 7f5cf2cb4640 1 -- 172.21.15.16:0/421409675 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cec0749b0 con 0x7f5cec159880 2024-04-25T10:05:20.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.793+0000 7f40e0ff9640 1 -- 172.21.15.16:0/3977823789 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f40d0018020 con 0x7f40dc0e40f0 2024-04-25T10:05:20.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.793+0000 7f40e0ff9640 1 --2- 172.21.15.16:0/3977823789 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f40c806b520 0x7f40c806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.793+0000 7f40e3fff640 1 --2- 172.21.15.16:0/3977823789 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f40c806b520 0x7f40c806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.794+0000 7f40e3fff640 1 --2- 172.21.15.16:0/3977823789 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f40c806b520 0x7f40c806d9e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009630 tx=0x7f40d8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.794+0000 7fa1fcff9640 1 -- 172.21.15.16:0/2326652357 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1ec022020 con 0x7fa1f804e0e0 2024-04-25T10:05:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.801+0000 7f5ce27fc640 1 -- 172.21.15.16:0/421409675 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cdc017070 con 0x7f5cec159880 2024-04-25T10:05:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.801+0000 7f5ce27fc640 1 -- 172.21.15.16:0/421409675 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc011b30 con 0x7f5cec159880 2024-04-25T10:05:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.801+0000 7f40e0ff9640 1 -- 172.21.15.16:0/3977823789 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40d0056df0 con 0x7f40dc0e40f0 2024-04-25T10:05:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.801+0000 7fb4d57fa640 1 -- 172.21.15.16:0/171125399 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb4cc079180 con 0x7fb4d00e9350 2024-04-25T10:05:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.801+0000 7f8e41ffb640 1 -- 172.21.15.16:0/107625945 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3c0052d0 con 0x7f8e44071830 2024-04-25T10:05:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.801+0000 7f8e41ffb640 1 -- 172.21.15.16:0/107625945 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c007230 con 0x7f8e44071830 2024-04-25T10:05:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.804+0000 7fe88b7fe640 1 -- 172.21.15.16:0/2824136036 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe89c058ea0 con 0x7fe8a4159910 2024-04-25T10:05:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.807+0000 7f40e0ff9640 1 -- 172.21.15.16:0/3977823789 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f40d001f020 con 0x7f40dc0e40f0 2024-04-25T10:05:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.809+0000 7f5ce27fc640 1 -- 172.21.15.16:0/421409675 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5cdc011cd0 con 0x7f5cec159880 2024-04-25T10:05:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.810+0000 7f5ce27fc640 1 --2- 172.21.15.16:0/421409675 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5cd006b4d0 0x7f5cd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.810+0000 7f5ce27fc640 1 -- 172.21.15.16:0/421409675 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5cdc08a960 con 0x7f5cec159880 2024-04-25T10:05:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.810+0000 7f5cf1cb2640 1 --2- 172.21.15.16:0/421409675 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5cd006b4d0 0x7f5cd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.810+0000 7f8e41ffb640 1 -- 172.21.15.16:0/107625945 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e3c01a070 con 0x7f8e44071830 2024-04-25T10:05:20.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.811+0000 7f5cf1cb2640 1 --2- 172.21.15.16:0/421409675 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5cd006b4d0 0x7f5cd006d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5ce802a2b0 tx=0x7f5ce802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.811+0000 7f8e41ffb640 1 --2- 172.21.15.16:0/107625945 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8e3006b4d0 0x7f8e3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.811+0000 7f8e41ffb640 1 -- 172.21.15.16:0/107625945 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e3c089600 con 0x7f8e44071830 2024-04-25T10:05:20.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.812+0000 7f5ce27fc640 1 -- 172.21.15.16:0/421409675 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5cdc059170 con 0x7f5cec159880 2024-04-25T10:05:20.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.812+0000 7f8e48a0a640 1 --2- 172.21.15.16:0/107625945 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8e3006b4d0 0x7f8e3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.813+0000 7f8e48a0a640 1 --2- 172.21.15.16:0/107625945 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8e3006b4d0 0x7f8e3006d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8e441583d0 tx=0x7f8e34030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:20.816+0000 7f8e41ffb640 1 -- 172.21.15.16:0/107625945 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8e3c021080 con 0x7f8e44071830 2024-04-25T10:05:21.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.192+0000 7fdbb1a93640 1 -- 172.21.15.16:0/129823462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fdbac13fef0 con 0x7fdbac15db50 2024-04-25T10:05:21.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.193+0000 7fdb93fff640 1 -- 172.21.15.16:0/129823462 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdba4058690 con 0x7fdbac15db50 2024-04-25T10:05:21.198 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-04-25T10:05:21.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.196+0000 7fdb91ffb640 1 -- 172.21.15.16:0/129823462 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb9806b5a0 msgr2=0x7fdb9806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.196+0000 7fdb91ffb640 1 --2- 172.21.15.16:0/129823462 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb9806b5a0 0x7fdb9806da60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c018690 tx=0x7fdb9c00f9e0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.196+0000 7fdb91ffb640 1 -- 172.21.15.16:0/129823462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbac15db50 msgr2=0x7fdbac0c5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.196+0000 7fdb91ffb640 1 --2- 172.21.15.16:0/129823462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbac15db50 0x7fdbac0c5ff0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdba400c6c0 tx=0x7fdba400cb80 comp rx=0 tx=0).stop 2024-04-25T10:05:21.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.197+0000 7fdb91ffb640 1 -- 172.21.15.16:0/129823462 shutdown_connections 2024-04-25T10:05:21.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.197+0000 7fdb91ffb640 1 --2- 172.21.15.16:0/129823462 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb9806b5a0 0x7fdb9806da60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.197+0000 7fdb91ffb640 1 --2- 172.21.15.16:0/129823462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbac15db50 0x7fdbac0c5ff0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.197+0000 7fdb91ffb640 1 --2- 172.21.15.16:0/129823462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbac15aa30 0x7fdbac0c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.197+0000 7fdb91ffb640 1 --2- 172.21.15.16:0/129823462 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdbac06f140 0x7fdbac1eeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.197+0000 7fdb91ffb640 1 -- 172.21.15.16:0/129823462 >> 172.21.15.16:0/129823462 conn(0x7fdbac06aba0 msgr2=0x7fdbac0b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.197+0000 7fdb91ffb640 1 -- 172.21.15.16:0/129823462 shutdown_connections 2024-04-25T10:05:21.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.198+0000 7fdb91ffb640 1 -- 172.21.15.16:0/129823462 wait complete. 2024-04-25T10:05:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.205+0000 7f5941ffb640 1 -- 172.21.15.16:0/1165395397 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f595c0b3da0 con 0x7f595c06c0f0 2024-04-25T10:05:21.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.206+0000 7f5943fff640 1 -- 172.21.15.16:0/1165395397 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5954078c90 con 0x7f595c06c0f0 2024-04-25T10:05:21.211 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.209+0000 7f596083c640 1 -- 172.21.15.16:0/1165395397 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f594806b520 msgr2=0x7f594806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.209+0000 7f596083c640 1 --2- 172.21.15.16:0/1165395397 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f594806b520 0x7f594806d9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f595c06b0f0 tx=0x7f5950008040 comp rx=0 tx=0).stop 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.210+0000 7f596083c640 1 -- 172.21.15.16:0/1165395397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f595c06c0f0 msgr2=0x7f595c14efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.210+0000 7f596083c640 1 --2- 172.21.15.16:0/1165395397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f595c06c0f0 0x7f595c14efb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f595402c3c0 tx=0x7f5954004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.210+0000 7f596083c640 1 -- 172.21.15.16:0/1165395397 shutdown_connections 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.210+0000 7f596083c640 1 --2- 172.21.15.16:0/1165395397 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f594806b520 0x7f594806d9e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.210+0000 7f596083c640 1 --2- 172.21.15.16:0/1165395397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f595c1545e0 0x7f595c156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.210+0000 7f596083c640 1 --2- 172.21.15.16:0/1165395397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f595c0b4a10 0x7f595c1f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.210+0000 7f596083c640 1 --2- 172.21.15.16:0/1165395397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f595c06c0f0 0x7f595c14efb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.210+0000 7f596083c640 1 -- 172.21.15.16:0/1165395397 >> 172.21.15.16:0/1165395397 conn(0x7f595c071900 msgr2=0x7f595c159530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.210+0000 7f596083c640 1 -- 172.21.15.16:0/1165395397 shutdown_connections 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.210+0000 7f596083c640 1 -- 172.21.15.16:0/1165395397 wait complete. 2024-04-25T10:05:21.220 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.4 2024-04-25T10:05:21.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.218+0000 7fa206568640 1 -- 172.21.15.16:0/2326652357 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa1e0005850 con 0x7fa1f804e0e0 2024-04-25T10:05:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.218+0000 7fa1fcff9640 1 -- 172.21.15.16:0/2326652357 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa1ec0582b0 con 0x7fa1f804e0e0 2024-04-25T10:05:21.224 INFO:teuthology.orchestra.run.smithi016.stdout:64424509446 2024-04-25T10:05:21.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.222+0000 7fa206568640 1 -- 172.21.15.16:0/2326652357 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa1e406b5a0 msgr2=0x7fa1e406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.222+0000 7fa206568640 1 --2- 172.21.15.16:0/2326652357 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa1e406b5a0 0x7fa1e406da60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8037aa0 tx=0x7fa1f4008040 comp rx=0 tx=0).stop 2024-04-25T10:05:21.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.222+0000 7fa206568640 1 -- 172.21.15.16:0/2326652357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1f804e0e0 msgr2=0x7fa1f8036aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.222+0000 7fa206568640 1 --2- 172.21.15.16:0/2326652357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1f804e0e0 0x7fa1f8036aa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00c680 tx=0x7fa1ec00cb40 comp rx=0 tx=0).stop 2024-04-25T10:05:21.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.222+0000 7fa206568640 1 -- 172.21.15.16:0/2326652357 shutdown_connections 2024-04-25T10:05:21.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.222+0000 7fa206568640 1 --2- 172.21.15.16:0/2326652357 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa1e406b5a0 0x7fa1e406da60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.222+0000 7fa206568640 1 --2- 172.21.15.16:0/2326652357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f80ecd70 0x7fa1f804a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.222+0000 7fa206568640 1 --2- 172.21.15.16:0/2326652357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1f804e0e0 0x7fa1f8036aa0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.222+0000 7fa206568640 1 --2- 172.21.15.16:0/2326652357 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1f0004fe0 0x7fa1f8035390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.222+0000 7fa206568640 1 -- 172.21.15.16:0/2326652357 >> 172.21.15.16:0/2326652357 conn(0x7fa1f800a430 msgr2=0x7fa1f804b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.222+0000 7fa206568640 1 -- 172.21.15.16:0/2326652357 shutdown_connections 2024-04-25T10:05:21.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.222+0000 7fa206568640 1 -- 172.21.15.16:0/2326652357 wait complete. 2024-04-25T10:05:21.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.224+0000 7f40e9797640 1 -- 172.21.15.16:0/3977823789 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f40dc04fd60 con 0x7f40dc0e40f0 2024-04-25T10:05:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.224+0000 7f5cf2cb4640 1 -- 172.21.15.16:0/421409675 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5cec0c4c80 con 0x7f5cec159880 2024-04-25T10:05:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.225+0000 7f40e0ff9640 1 -- 172.21.15.16:0/3977823789 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f40d0059450 con 0x7f40dc0e40f0 2024-04-25T10:05:21.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.225+0000 7f5ce27fc640 1 -- 172.21.15.16:0/421409675 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5cdc05d210 con 0x7f5cec159880 2024-04-25T10:05:21.234 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-04-25T10:05:21.234 INFO:teuthology.orchestra.run.smithi016.stdout:64424509446 2024-04-25T10:05:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.226+0000 7fb4ddc51640 1 -- 172.21.15.16:0/171125399 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb4d000aab0 con 0x7fb4d00e9350 2024-04-25T10:05:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.226+0000 7fb4d57fa640 1 -- 172.21.15.16:0/171125399 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb4cc07d220 con 0x7fb4d00e9350 2024-04-25T10:05:21.235 INFO:teuthology.orchestra.run.smithi016.stdout:55834574854 2024-04-25T10:05:21.236 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.2 2024-04-25T10:05:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.228+0000 7fe8a906b640 1 -- 172.21.15.16:0/2824136036 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe8a406d930 con 0x7fe8a4159910 2024-04-25T10:05:21.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.228+0000 7f5cf2cb4640 1 -- 172.21.15.16:0/421409675 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5cd006b4d0 msgr2=0x7f5cd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.228+0000 7f5cf2cb4640 1 --2- 172.21.15.16:0/421409675 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5cd006b4d0 0x7f5cd006d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5ce802a2b0 tx=0x7f5ce802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:21.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.228+0000 7f5cf2cb4640 1 -- 172.21.15.16:0/421409675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5cec159880 msgr2=0x7f5cec203a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.228+0000 7f5cf2cb4640 1 --2- 172.21.15.16:0/421409675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5cec159880 0x7f5cec203a20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00c410 tx=0x7f5cdc00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.228+0000 7f5cf2cb4640 1 -- 172.21.15.16:0/421409675 shutdown_connections 2024-04-25T10:05:21.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.228+0000 7f5cf2cb4640 1 --2- 172.21.15.16:0/421409675 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5cd006b4d0 0x7f5cd006d990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.228+0000 7f5cf2cb4640 1 --2- 172.21.15.16:0/421409675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5cec159880 0x7f5cec203a20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.228+0000 7f5cf2cb4640 1 --2- 172.21.15.16:0/421409675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec0cb470 0x7f5cec154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.229+0000 7f5cf2cb4640 1 --2- 172.21.15.16:0/421409675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cec071830 0x7f5cec1559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.229+0000 7f5cf2cb4640 1 -- 172.21.15.16:0/421409675 >> 172.21.15.16:0/421409675 conn(0x7f5cec06aba0 msgr2=0x7f5cec0cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.229+0000 7f5cf2cb4640 1 -- 172.21.15.16:0/421409675 shutdown_connections 2024-04-25T10:05:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.229+0000 7f40e9797640 1 -- 172.21.15.16:0/3977823789 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f40c806b520 msgr2=0x7f40c806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.229+0000 7f40e9797640 1 --2- 172.21.15.16:0/3977823789 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f40c806b520 0x7f40c806d9e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009630 tx=0x7f40d8008030 comp rx=0 tx=0).stop 2024-04-25T10:05:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.229+0000 7f5cf2cb4640 1 -- 172.21.15.16:0/421409675 wait complete. 2024-04-25T10:05:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.229+0000 7f40e9797640 1 -- 172.21.15.16:0/3977823789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40dc0e40f0 msgr2=0x7f40dc0da370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.229+0000 7f40e9797640 1 --2- 172.21.15.16:0/3977823789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40dc0e40f0 0x7f40dc0da370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f40d000b3f0 tx=0x7f40d000b7f0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.229+0000 7f40e9797640 1 -- 172.21.15.16:0/3977823789 shutdown_connections 2024-04-25T10:05:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.229+0000 7f40e9797640 1 --2- 172.21.15.16:0/3977823789 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f40c806b520 0x7f40c806d9e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.229+0000 7f40e9797640 1 --2- 172.21.15.16:0/3977823789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40dc0dd090 0x7f40dc1777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.229+0000 7f40e9797640 1 --2- 172.21.15.16:0/3977823789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40dc0e40f0 0x7f40dc0da370 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.229+0000 7f40e9797640 1 --2- 172.21.15.16:0/3977823789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40dc0dfee0 0x7f40dc0de770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.229+0000 7fe88b7fe640 1 -- 172.21.15.16:0/2824136036 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe89c0587a0 con 0x7fe8a4159910 2024-04-25T10:05:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.229+0000 7f40e9797640 1 -- 172.21.15.16:0/3977823789 >> 172.21.15.16:0/3977823789 conn(0x7f40dc01ef30 msgr2=0x7f40dc0539e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.241 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-04-25T10:05:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.229+0000 7f40e9797640 1 -- 172.21.15.16:0/3977823789 shutdown_connections 2024-04-25T10:05:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.229+0000 7f40e9797640 1 -- 172.21.15.16:0/3977823789 wait complete. 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.229+0000 7fb4ddc51640 1 -- 172.21.15.16:0/171125399 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4bc06b5a0 msgr2=0x7fb4bc06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.230+0000 7fb4ddc51640 1 --2- 172.21.15.16:0/171125399 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4bc06b5a0 0x7fb4bc06da60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8050190 tx=0x7fb4d806e040 comp rx=0 tx=0).stop 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.230+0000 7fb4ddc51640 1 -- 172.21.15.16:0/171125399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4d00e9350 msgr2=0x7fb4d0038f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.230+0000 7fb4ddc51640 1 --2- 172.21.15.16:0/171125399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4d00e9350 0x7fb4d0038f50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009950 tx=0x7fb4cc0047d0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.230+0000 7fb4ddc51640 1 -- 172.21.15.16:0/171125399 shutdown_connections 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.230+0000 7fb4ddc51640 1 --2- 172.21.15.16:0/171125399 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4bc06b5a0 0x7fb4bc06da60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.230+0000 7fb4ddc51640 1 --2- 172.21.15.16:0/171125399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4d00e3fc0 0x7fb4d00e4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.230+0000 7fb4ddc51640 1 --2- 172.21.15.16:0/171125399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4d00e9350 0x7fb4d0038f50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.230+0000 7fb4ddc51640 1 --2- 172.21.15.16:0/171125399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4c8004370 0x7fb4d00e2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.230+0000 7fb4ddc51640 1 -- 172.21.15.16:0/171125399 >> 172.21.15.16:0/171125399 conn(0x7fb4d000a3f0 msgr2=0x7fb4d00e74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.230+0000 7fb4ddc51640 1 -- 172.21.15.16:0/171125399 shutdown_connections 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.231+0000 7fb4ddc51640 1 -- 172.21.15.16:0/171125399 wait complete. 2024-04-25T10:05:21.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.233+0000 7fe8a906b640 1 -- 172.21.15.16:0/2824136036 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe89006b550 msgr2=0x7fe89006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.233+0000 7fe8a906b640 1 --2- 172.21.15.16:0/2824136036 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe89006b550 0x7fe89006da10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe8940011e0 tx=0x7fe894007890 comp rx=0 tx=0).stop 2024-04-25T10:05:21.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.233+0000 7fe8a906b640 1 -- 172.21.15.16:0/2824136036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8a4159910 msgr2=0x7fe8a415f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.233+0000 7fe8a906b640 1 --2- 172.21.15.16:0/2824136036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8a4159910 0x7fe8a415f0c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00c6c0 tx=0x7fe89c00cb80 comp rx=0 tx=0).stop 2024-04-25T10:05:21.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.233+0000 7fe8a906b640 1 -- 172.21.15.16:0/2824136036 shutdown_connections 2024-04-25T10:05:21.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.233+0000 7fe8a906b640 1 --2- 172.21.15.16:0/2824136036 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe89006b550 0x7fe89006da10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.233+0000 7fe8a906b640 1 --2- 172.21.15.16:0/2824136036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8a4159910 0x7fe8a415f0c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.233+0000 7fe8a906b640 1 --2- 172.21.15.16:0/2824136036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8a406dfa0 0x7fe8a41f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.233+0000 7fe8a906b640 1 --2- 172.21.15.16:0/2824136036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a406b690 0x7fe8a41f0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.233+0000 7fe8a906b640 1 -- 172.21.15.16:0/2824136036 >> 172.21.15.16:0/2824136036 conn(0x7fe8a406aba0 msgr2=0x7fe8a41481d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.233+0000 7fe8a906b640 1 -- 172.21.15.16:0/2824136036 shutdown_connections 2024-04-25T10:05:21.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.233+0000 7fe8a906b640 1 -- 172.21.15.16:0/2824136036 wait complete. 2024-04-25T10:05:21.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.239+0000 7f8e4a20d640 1 -- 172.21.15.16:0/107625945 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f8e44159080 con 0x7f8e44071830 2024-04-25T10:05:21.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.239+0000 7f8e41ffb640 1 -- 172.21.15.16:0/107625945 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8e3c021080 con 0x7f8e44071830 2024-04-25T10:05:21.246 INFO:teuthology.orchestra.run.smithi016.stdout:64424509446 2024-04-25T10:05:21.248 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.1 2024-04-25T10:05:21.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.243+0000 7f8e4a20d640 1 -- 172.21.15.16:0/107625945 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8e3006b4d0 msgr2=0x7f8e3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.243+0000 7f8e4a20d640 1 --2- 172.21.15.16:0/107625945 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8e3006b4d0 0x7f8e3006d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8e441583d0 tx=0x7f8e34030000 comp rx=0 tx=0).stop 2024-04-25T10:05:21.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.243+0000 7f8e4a20d640 1 -- 172.21.15.16:0/107625945 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e44071830 msgr2=0x7f8e44150980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.243+0000 7f8e4a20d640 1 --2- 172.21.15.16:0/107625945 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e44071830 0x7f8e44150980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00c390 tx=0x7f8e3c00c850 comp rx=0 tx=0).stop 2024-04-25T10:05:21.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.243+0000 7f8e4a20d640 1 -- 172.21.15.16:0/107625945 shutdown_connections 2024-04-25T10:05:21.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.243+0000 7f8e4a20d640 1 --2- 172.21.15.16:0/107625945 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8e3006b4d0 0x7f8e3006d990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.243+0000 7f8e4a20d640 1 --2- 172.21.15.16:0/107625945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e44159880 0x7f8e44157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.243+0000 7f8e4a20d640 1 --2- 172.21.15.16:0/107625945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e440cb470 0x7f8e440c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.243+0000 7f8e4a20d640 1 --2- 172.21.15.16:0/107625945 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e44071830 0x7f8e44150980 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.243+0000 7f8e4a20d640 1 -- 172.21.15.16:0/107625945 >> 172.21.15.16:0/107625945 conn(0x7f8e4406aba0 msgr2=0x7f8e4415c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.243+0000 7f8e4a20d640 1 -- 172.21.15.16:0/107625945 shutdown_connections 2024-04-25T10:05:21.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:21.243+0000 7f8e4a20d640 1 -- 172.21.15.16:0/107625945 wait complete. 2024-04-25T10:05:21.251 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2024-04-25T10:05:21.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.5 2024-04-25T10:05:21.253 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.0 2024-04-25T10:05:21.258 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.3 2024-04-25T10:05:21.262 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.7 2024-04-25T10:05:22.220 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:05:22.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:05:22.249 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:05:22.252 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:05:22.252 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:05:22.253 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:05:22.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:05:22.263 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:05:22.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.382+0000 7f88d7fff640 1 Processor -- start 2024-04-25T10:05:22.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.382+0000 7f88d7fff640 1 -- start start 2024-04-25T10:05:22.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.384+0000 7f88d7fff640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f88d00df6e0 0x7f88d0054cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.384+0000 7f88d7fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88d003ad40 0x7f88d00533f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.384+0000 7f88d7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88d0050080 0x7f88d0050510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.384+0000 7f88d7fff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d0053c20 con 0x7f88d0050080 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.384+0000 7f88d7fff640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d0053f00 con 0x7f88d00df6e0 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.384+0000 7f88d7fff640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d00541e0 con 0x7f88d003ad40 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.384+0000 7f88d77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88d0050080 0x7f88d0050510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.384+0000 7f88d77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88d0050080 0x7f88d0050510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42248/0 (socket says 172.21.15.16:42248) 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.384+0000 7f88d77fe640 1 -- 172.21.15.16:0/1245065397 learned_addr learned my addr 172.21.15.16:0/1245065397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.385+0000 7f88d77fe640 1 -- 172.21.15.16:0/1245065397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88d003ad40 msgr2=0x7f88d00533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.385+0000 7f88d77fe640 1 --2- 172.21.15.16:0/1245065397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88d003ad40 0x7f88d00533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.385+0000 7f88d77fe640 1 -- 172.21.15.16:0/1245065397 >> v2:172.21.15.186:3300/0 conn(0x7f88d00df6e0 msgr2=0x7f88d0054cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.385+0000 7f88d77fe640 1 --2- 172.21.15.16:0/1245065397 >> v2:172.21.15.186:3300/0 conn(0x7f88d00df6e0 0x7f88d0054cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.385+0000 7f88d77fe640 1 -- 172.21.15.16:0/1245065397 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d00544c0 con 0x7f88d0050080 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.386+0000 7f88d77fe640 1 --2- 172.21.15.16:0/1245065397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d0050080 0x7f88d0050510 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f88d804f440 tx=0x7f88d80a51b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9262f5eef4a0928 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.386+0000 7f88d5ffb640 1 -- 172.21.15.16:0/1245065397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d80b2070 con 0x7f88d0050080 2024-04-25T10:05:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.386+0000 7f88d5ffb640 1 -- 172.21.15.16:0/1245065397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d80ad440 con 0x7f88d0050080 2024-04-25T10:05:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.387+0000 7f88d7fff640 1 -- 172.21.15.16:0/1245065397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d0050080 msgr2=0x7f88d0050510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.387+0000 7f88d7fff640 1 --2- 172.21.15.16:0/1245065397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d0050080 0x7f88d0050510 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f88d804f440 tx=0x7f88d80a51b0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.388+0000 7f88d7fff640 1 -- 172.21.15.16:0/1245065397 shutdown_connections 2024-04-25T10:05:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.388+0000 7f88d7fff640 1 --2- 172.21.15.16:0/1245065397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d0050080 0x7f88d0050510 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.388+0000 7f88d7fff640 1 --2- 172.21.15.16:0/1245065397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88d003ad40 0x7f88d00533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.388+0000 7f88d7fff640 1 --2- 172.21.15.16:0/1245065397 >> v2:172.21.15.186:3300/0 conn(0x7f88d00df6e0 0x7f88d0054cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.388+0000 7f88d7fff640 1 -- 172.21.15.16:0/1245065397 >> 172.21.15.16:0/1245065397 conn(0x7f88d0055340 msgr2=0x7f88d0055750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.388+0000 7f88d7fff640 1 -- 172.21.15.16:0/1245065397 shutdown_connections 2024-04-25T10:05:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.388+0000 7f88d7fff640 1 -- 172.21.15.16:0/1245065397 wait complete. 2024-04-25T10:05:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.390+0000 7f88d7fff640 1 Processor -- start 2024-04-25T10:05:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.390+0000 7f88d7fff640 1 -- start start 2024-04-25T10:05:22.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.392+0000 7f88d7fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88d003ad40 0x7f88d004ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.392+0000 7f88d7fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88d0050080 0x7f88d00368c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.392+0000 7f88d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d00d7f40 0x7f88d00d8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.392+0000 7f88d7fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d0006ac0 con 0x7f88d00d7f40 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.392+0000 7f88d7fff640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d0053930 con 0x7f88d003ad40 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.392+0000 7f88d7fff640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d0006c40 con 0x7f88d0050080 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.392+0000 7f88d67fc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88d0050080 0x7f88d00368c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.392+0000 7f88d77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d00d7f40 0x7f88d00d8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.392+0000 7f88d77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d00d7f40 0x7f88d00d8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42250/0 (socket says 172.21.15.16:42250) 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.392+0000 7f88d77fe640 1 -- 172.21.15.16:0/1976629909 learned_addr learned my addr 172.21.15.16:0/1976629909 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.393+0000 7f88d77fe640 1 -- 172.21.15.16:0/1976629909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88d0050080 msgr2=0x7f88d00368c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.393+0000 7f88d77fe640 1 --2- 172.21.15.16:0/1976629909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88d0050080 0x7f88d00368c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.393+0000 7f88d77fe640 1 -- 172.21.15.16:0/1976629909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88d003ad40 msgr2=0x7f88d004ac90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.393+0000 7f88d77fe640 1 --2- 172.21.15.16:0/1976629909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88d003ad40 0x7f88d004ac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.393+0000 7f88d77fe640 1 -- 172.21.15.16:0/1976629909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d0037210 con 0x7f88d00d7f40 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.393+0000 7f88d77fe640 1 --2- 172.21.15.16:0/1976629909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d00d7f40 0x7f88d00d8af0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f88d8074ed0 tx=0x7f88d8068980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.394+0000 7f88bf7fe640 1 -- 172.21.15.16:0/1976629909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d80b2030 con 0x7f88d00d7f40 2024-04-25T10:05:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.394+0000 7f88d7fff640 1 -- 172.21.15.16:0/1976629909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d00374f0 con 0x7f88d00d7f40 2024-04-25T10:05:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.395+0000 7f88bf7fe640 1 -- 172.21.15.16:0/1976629909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d8087070 con 0x7f88d00d7f40 2024-04-25T10:05:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.395+0000 7f88bf7fe640 1 -- 172.21.15.16:0/1976629909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8089070 con 0x7f88d00d7f40 2024-04-25T10:05:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.396+0000 7f88d7fff640 1 -- 172.21.15.16:0/1976629909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d004ce50 con 0x7f88d00d7f40 2024-04-25T10:05:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.397+0000 7f88bf7fe640 1 -- 172.21.15.16:0/1976629909 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88d80b5030 con 0x7f88d00d7f40 2024-04-25T10:05:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.398+0000 7f88bf7fe640 1 --2- 172.21.15.16:0/1976629909 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f88c006b660 0x7f88c006db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.398+0000 7f88bf7fe640 1 -- 172.21.15.16:0/1976629909 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88d80ad630 con 0x7f88d00d7f40 2024-04-25T10:05:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.399+0000 7f88d6ffd640 1 --2- 172.21.15.16:0/1976629909 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f88c006b660 0x7f88c006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.400+0000 7f88b57fa640 1 -- 172.21.15.16:0/1976629909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d00057d0 con 0x7f88d00d7f40 2024-04-25T10:05:22.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.409+0000 7f88bf7fe640 1 -- 172.21.15.16:0/1976629909 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88d80e2920 con 0x7f88d00d7f40 2024-04-25T10:05:22.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.409+0000 7f88d6ffd640 1 --2- 172.21.15.16:0/1976629909 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f88c006b660 0x7f88c006db20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f88d0008d70 tx=0x7f88cc006c20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.439+0000 7f8830831640 1 Processor -- start 2024-04-25T10:05:22.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.439+0000 7f8830831640 1 -- start start 2024-04-25T10:05:22.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.439+0000 7fc79cda5640 1 Processor -- start 2024-04-25T10:05:22.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.439+0000 7fc79cda5640 1 -- start start 2024-04-25T10:05:22.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.440+0000 7f390726f640 1 Processor -- start 2024-04-25T10:05:22.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.441+0000 7f8830831640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f88240e1200 0x7f8824036120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.441+0000 7f8830831640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8824038e50 0x7f8824039ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.441+0000 7f882b7fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f88240e1200 0x7f8824036120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.441+0000 7f882b7fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f88240e1200 0x7f8824036120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:53748/0 (socket says 172.21.15.16:53748) 2024-04-25T10:05:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.441+0000 7f882affd640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f8824051d90 0x7f8824038740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:42720/0 (socket says 172.21.15.16:42720) 2024-04-25T10:05:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.441+0000 7fc79cda5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc79806ead0 0x7fc7981599c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.441+0000 7fc79cda5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc7980cf670 0x7fc7980cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.441+0000 7fc79cda5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7980b3c40 con 0x7fc79806f2f0 2024-04-25T10:05:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.441+0000 7fc79cda5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7980cca90 con 0x7fc7980cf670 2024-04-25T10:05:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.441+0000 7fc79cda5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980b3f20 con 0x7fc79806ead0 2024-04-25T10:05:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.441+0000 7f882b7fe640 1 -- 172.21.15.16:0/746196417 learned_addr learned my addr 172.21.15.16:0/746196417 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.441+0000 7f2d43e38640 1 Processor -- start 2024-04-25T10:05:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.441+0000 7f8830831640 1 -- 172.21.15.16:0/746196417 --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88240545e0 con 0x7f8824051d90 2024-04-25T10:05:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.441+0000 7f8830831640 1 -- 172.21.15.16:0/746196417 --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8824039310 con 0x7f8824038e50 2024-04-25T10:05:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.441+0000 7f8830831640 1 -- 172.21.15.16:0/746196417 --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88240395f0 con 0x7f88240e1200 2024-04-25T10:05:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f2d43e38640 1 -- start start 2024-04-25T10:05:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f882bfff640 1 --2- 172.21.15.16:0/746196417 >> v2:172.21.15.16:3300/0 conn(0x7f8824038e50 0x7f8824039ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f882a7fc640 1 -- 172.21.15.16:0/746196417 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 867921601 0 0) 0x7f88240545e0 con 0x7f8824051d90 2024-04-25T10:05:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f882a7fc640 1 -- 172.21.15.16:0/746196417 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f881c003520 con 0x7f8824051d90 2024-04-25T10:05:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f882a7fc640 1 -- 172.21.15.16:0/746196417 <== mon.1 v1:172.21.15.186:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 496476660 0 0) 0x7f881c003520 con 0x7f8824051d90 2024-04-25T10:05:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f882a7fc640 1 -- 172.21.15.16:0/746196417 >> v2:172.21.15.186:3300/0 conn(0x7f88240e1200 msgr2=0x7f8824036120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f882a7fc640 1 --2- 172.21.15.16:0/746196417 >> v2:172.21.15.186:3300/0 conn(0x7f88240e1200 0x7f8824036120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f882a7fc640 1 -- 172.21.15.16:0/746196417 >> v2:172.21.15.16:3300/0 conn(0x7f8824038e50 msgr2=0x7f8824039ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f882a7fc640 1 --2- 172.21.15.16:0/746196417 >> v2:172.21.15.16:3300/0 conn(0x7f8824038e50 0x7f8824039ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f882a7fc640 1 -- 172.21.15.16:0/746196417 --> v1:172.21.15.186:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88240515b0 con 0x7f8824051d90 2024-04-25T10:05:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f882b7fe640 1 --2- 172.21.15.16:0/746196417 >> v2:172.21.15.186:3300/0 conn(0x7f88240e1200 0x7f8824036120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f882a7fc640 1 -- 172.21.15.16:0/746196417 <== mon.1 v1:172.21.15.186:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f8818001670 con 0x7f8824051d90 2024-04-25T10:05:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f882a7fc640 1 -- 172.21.15.16:0/746196417 >> v1:172.21.15.186:6789/0 conn(0x7f8824051d90 legacy=0x7f8824038740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f882a7fc640 1 --2- 172.21.15.16:0/746196417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f881c003a00 0x7f881c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.441+0000 7fc795d9b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc79806ead0 0x7fc7981599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7fc796d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc7980cf670 0x7fc7980cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7fc796d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc7980cf670 0x7fc7980cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42274/0 (socket says 172.21.15.16:42274) 2024-04-25T10:05:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7fc796d9d640 1 -- 172.21.15.16:0/3780137802 learned_addr learned my addr 172.21.15.16:0/3780137802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7fc796d9d640 1 -- 172.21.15.16:0/3780137802 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc79806ead0 msgr2=0x7fc7981599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.441+0000 7fc795d9b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc79806ead0 0x7fc7981599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:34268/0 (socket says 172.21.15.16:34268) 2024-04-25T10:05:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7fc796d9d640 1 --2- 172.21.15.16:0/3780137802 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc79806ead0 0x7fc7981599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7fc796d9d640 1 -- 172.21.15.16:0/3780137802 >> v1:172.21.15.16:6789/0 conn(0x7fc79806f2f0 legacy=0x7fc7980b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7fc796d9d640 1 -- 172.21.15.16:0/3780137802 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980cccc0 con 0x7fc7980cf670 2024-04-25T10:05:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f1455b1c640 1 Processor -- start 2024-04-25T10:05:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.441+0000 7f390726f640 1 -- start start 2024-04-25T10:05:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f390726f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3900159b80 0x7f390015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f390726f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3900159f60 0x7f390015a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f390726f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39000b33c0 con 0x7f390006dfa0 2024-04-25T10:05:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f390726f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39000b3fd0 con 0x7f3900159f60 2024-04-25T10:05:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f390726f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39000b3170 con 0x7f3900159b80 2024-04-25T10:05:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f390480a640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f390006dfa0 0x7f390015dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:42726/0 (socket says 172.21.15.16:42726) 2024-04-25T10:05:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f390480a640 1 -- 172.21.15.16:0/3696976390 learned_addr learned my addr 172.21.15.16:0/3696976390 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:22.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f882a7fc640 1 --2- 172.21.15.16:0/746196417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f881c004800 0x7f881c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f882a7fc640 1 --2- 172.21.15.16:0/746196417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f881c0053c0 0x7f881c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f882a7fc640 1 -- 172.21.15.16:0/746196417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8824039310 con 0x7f881c003a00 2024-04-25T10:05:22.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f882a7fc640 1 -- 172.21.15.16:0/746196417 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88240395f0 con 0x7f881c004800 2024-04-25T10:05:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f2d43e38640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d3c159b80 0x7f2d3c15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f2d43e38640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2d3c159f60 0x7f2d3c15a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f2d413d3640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2d3c06dfa0 0x7f2d3c15dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34900/0 (socket says 172.21.15.16:34900) 2024-04-25T10:05:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f2d413d3640 1 -- 172.21.15.16:0/3111330534 learned_addr learned my addr 172.21.15.16:0/3111330534 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f2d43e38640 1 -- 172.21.15.16:0/3111330534 --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d3c0b33c0 con 0x7f2d3c06dfa0 2024-04-25T10:05:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f2d43e38640 1 -- 172.21.15.16:0/3111330534 --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d3c0b3fd0 con 0x7f2d3c159f60 2024-04-25T10:05:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f2d43e38640 1 -- 172.21.15.16:0/3111330534 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c0b3170 con 0x7f2d3c159b80 2024-04-25T10:05:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f1455b1c640 1 -- start start 2024-04-25T10:05:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f1455b1c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14480078d0 0x7f14480edd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f1455b1c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1448058560 con 0x7f144803e830 2024-04-25T10:05:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f1455b1c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1448058840 con 0x7f14480e30b0 2024-04-25T10:05:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f1455b1c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1448058b20 con 0x7f14480078d0 2024-04-25T10:05:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7fc796d9d640 1 --2- 172.21.15.16:0/3780137802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7980cf670 0x7fc7980cfb00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009870 tx=0x7fc78c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ebab86e2e6fa65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7fc79559a640 1 -- 172.21.15.16:0/3780137802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c037070 con 0x7fc7980cf670 2024-04-25T10:05:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f145531b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14480078d0 0x7f14480edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f1454b1a640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f14480e30b0 0x7f1448057d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:42732/0 (socket says 172.21.15.16:42732) 2024-04-25T10:05:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f1454b1a640 1 -- 172.21.15.16:0/2580859810 learned_addr learned my addr 172.21.15.16:0/2580859810 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7fc79559a640 1 -- 172.21.15.16:0/3780137802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c032440 con 0x7fc7980cf670 2024-04-25T10:05:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f882a7fc640 1 -- 172.21.15.16:0/746196417 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f881c00c2b0 con 0x7f881c0053c0 2024-04-25T10:05:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f882bfff640 1 --2- 172.21.15.16:0/746196417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f881c0053c0 0x7f881c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f882b7fe640 1 --2- 172.21.15.16:0/746196417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f881c003a00 0x7f881c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7fc79559a640 1 -- 172.21.15.16:0/3780137802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c032810 con 0x7fc7980cf670 2024-04-25T10:05:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f882affd640 1 --2- 172.21.15.16:0/746196417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f881c004800 0x7f881c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f882bfff640 1 -- 172.21.15.16:0/746196417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f881c004800 msgr2=0x7f881c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f882bfff640 1 --2- 172.21.15.16:0/746196417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f881c004800 0x7f881c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f882bfff640 1 -- 172.21.15.16:0/746196417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f881c003a00 msgr2=0x7f881c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f882bfff640 1 --2- 172.21.15.16:0/746196417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f881c003a00 0x7f881c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f882bfff640 1 -- 172.21.15.16:0/746196417 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881c003520 con 0x7f881c0053c0 2024-04-25T10:05:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f882bfff640 1 -- 172.21.15.16:0/746196417 reap_dead start 2024-04-25T10:05:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f882bfff640 1 --2- 172.21.15.16:0/746196417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f881c0053c0 0x7f881c00b9c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f882c050700 tx=0x7f881c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=339045e1bb779185 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f390580c640 1 --2- 172.21.15.16:0/3696976390 >> v2:172.21.15.186:3300/0 conn(0x7f3900159f60 0x7f390015a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.442+0000 7f38f7fff640 1 -- 172.21.15.16:0/3696976390 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3031262391 0 0) 0x7f39000b33c0 con 0x7f390006dfa0 2024-04-25T10:05:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f38f7fff640 1 -- 172.21.15.16:0/3696976390 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38ec003520 con 0x7f390006dfa0 2024-04-25T10:05:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f38f7fff640 1 -- 172.21.15.16:0/3696976390 <== mon.1 v1:172.21.15.186:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2872036573 0 0) 0x7f38ec003520 con 0x7f390006dfa0 2024-04-25T10:05:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f38f7fff640 1 -- 172.21.15.16:0/3696976390 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3900159b80 msgr2=0x7f390015d7d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:05:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f38f7fff640 1 --2- 172.21.15.16:0/3696976390 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3900159b80 0x7f390015d7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f38f7fff640 1 -- 172.21.15.16:0/3696976390 >> v2:172.21.15.186:3300/0 conn(0x7f3900159f60 msgr2=0x7f390015a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f38f7fff640 1 --2- 172.21.15.16:0/3696976390 >> v2:172.21.15.186:3300/0 conn(0x7f3900159f60 0x7f390015a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f38f7fff640 1 -- 172.21.15.16:0/3696976390 --> v1:172.21.15.186:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39000b36a0 con 0x7f390006dfa0 2024-04-25T10:05:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f38f7fff640 1 -- 172.21.15.16:0/3696976390 <== mon.1 v1:172.21.15.186:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f38f80019f0 con 0x7f390006dfa0 2024-04-25T10:05:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f38f7fff640 1 -- 172.21.15.16:0/3696976390 >> v1:172.21.15.186:6789/0 conn(0x7f390006dfa0 legacy=0x7f390015dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f38f7fff640 1 --2- 172.21.15.16:0/3696976390 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38ec0039b0 0x7f38ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f38f7fff640 1 --2- 172.21.15.16:0/3696976390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38ec004800 0x7f38ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f390480a640 1 --2- 172.21.15.16:0/3696976390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38ec004800 0x7f38ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.443+0000 7f38f7fff640 1 --2- 172.21.15.16:0/3696976390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38ec005210 0x7f38ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f38f7fff640 1 -- 172.21.15.16:0/3696976390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39000b3170 con 0x7f38ec004800 2024-04-25T10:05:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f38f7fff640 1 -- 172.21.15.16:0/3696976390 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec00c140 con 0x7f38ec005210 2024-04-25T10:05:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f38f7fff640 1 -- 172.21.15.16:0/3696976390 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec00c340 con 0x7f38ec0039b0 2024-04-25T10:05:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f390580c640 1 --2- 172.21.15.16:0/3696976390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38ec005210 0x7f38ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f390580c640 1 -- 172.21.15.16:0/3696976390 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38ec0039b0 msgr2=0x7f38ec003dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:05:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f390580c640 1 --2- 172.21.15.16:0/3696976390 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38ec0039b0 0x7f38ec003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f390580c640 1 -- 172.21.15.16:0/3696976390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38ec004800 msgr2=0x7f38ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f390580c640 1 --2- 172.21.15.16:0/3696976390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38ec004800 0x7f38ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f390580c640 1 -- 172.21.15.16:0/3696976390 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec003520 con 0x7f38ec005210 2024-04-25T10:05:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7f390480a640 1 --2- 172.21.15.16:0/3696976390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38ec004800 0x7f38ec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.444+0000 7fc795d9b640 1 --2- 172.21.15.16:0/3780137802 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc79806ead0 0x7fc7981599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-25T10:05:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7fc79cda5640 1 -- 172.21.15.16:0/3780137802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7980cf670 msgr2=0x7fc7980cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7f144f7fe640 1 -- 172.21.15.16:0/2580859810 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3339807797 0 0) 0x7f1448058840 con 0x7f14480e30b0 2024-04-25T10:05:22.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7f144f7fe640 1 -- 172.21.15.16:0/2580859810 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1440003520 con 0x7f14480e30b0 2024-04-25T10:05:22.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7fdf09c6b640 1 Processor -- start 2024-04-25T10:05:22.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7fc79cda5640 1 --2- 172.21.15.16:0/3780137802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7980cf670 0x7fc7980cfb00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009870 tx=0x7fc78c02a2c0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7fc79cda5640 1 -- 172.21.15.16:0/3780137802 shutdown_connections 2024-04-25T10:05:22.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7f882a7fc640 1 -- 172.21.15.16:0/746196417 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c0b3070 con 0x7f881c0053c0 2024-04-25T10:05:22.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7f882a7fc640 1 -- 172.21.15.16:0/746196417 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882c0ae410 con 0x7f881c0053c0 2024-04-25T10:05:22.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7f882a7fc640 1 -- 172.21.15.16:0/746196417 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c04f490 con 0x7f881c0053c0 2024-04-25T10:05:22.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7fdf09c6b640 1 -- start start 2024-04-25T10:05:22.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7fc79cda5640 1 --2- 172.21.15.16:0/3780137802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7980cf670 0x7fc7980cfb00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7fc79cda5640 1 --2- 172.21.15.16:0/3780137802 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc79806ead0 0x7fc7981599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7fc79cda5640 1 -- 172.21.15.16:0/3780137802 >> 172.21.15.16:0/3780137802 conn(0x7fc79806aba0 msgr2=0x7fc79806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7fc79cda5640 1 -- 172.21.15.16:0/3780137802 shutdown_connections 2024-04-25T10:05:22.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7fc79cda5640 1 -- 172.21.15.16:0/3780137802 wait complete. 2024-04-25T10:05:22.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.446+0000 7f2d41bd4640 1 --2- 172.21.15.16:0/3111330534 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d3c159b80 0x7f2d3c15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7f145531b640 1 -- 172.21.15.16:0/2580859810 >> v1:172.21.15.186:6789/0 conn(0x7f14480e30b0 legacy=0x7f1448057d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7f145531b640 1 -- 172.21.15.16:0/2580859810 >> v1:172.21.15.16:6789/0 conn(0x7f144803e830 legacy=0x7f1448053be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7f145531b640 1 -- 172.21.15.16:0/2580859810 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14480063a0 con 0x7f14480078d0 2024-04-25T10:05:22.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7f390580c640 1 --2- 172.21.15.16:0/3696976390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38ec005210 0x7f38ec00b9c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00ba30 tx=0x7f38ec005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97531a4344790c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.446+0000 7f2d423d5640 1 --2- 172.21.15.16:0/3111330534 >> v2:172.21.15.186:3300/0 conn(0x7f2d3c159f60 0x7f2d3c15a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7f38f7fff640 1 -- 172.21.15.16:0/3696976390 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc052070 con 0x7f38ec005210 2024-04-25T10:05:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7f38f7fff640 1 -- 172.21.15.16:0/3696976390 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc04ce20 con 0x7f38ec005210 2024-04-25T10:05:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.445+0000 7f38f7fff640 1 -- 172.21.15.16:0/3696976390 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc00b600 con 0x7f38ec005210 2024-04-25T10:05:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.446+0000 7f2d40bd2640 1 -- 172.21.15.16:0/3111330534 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895969192 0 0) 0x7f2d3c0b33c0 con 0x7f2d3c06dfa0 2024-04-25T10:05:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.446+0000 7f8830831640 1 -- 172.21.15.16:0/746196417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f881c0053c0 msgr2=0x7f881c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.446+0000 7f8830831640 1 --2- 172.21.15.16:0/746196417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f881c0053c0 0x7f881c00b9c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f882c050700 tx=0x7f881c005830 comp rx=0 tx=0).stop 2024-04-25T10:05:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.446+0000 7f145531b640 1 --2- 172.21.15.16:0/2580859810 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14480078d0 0x7f14480edd80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f14440097d0 tx=0x7f144402a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfee047b471b5043 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.446+0000 7f2d40bd2640 1 -- 172.21.15.16:0/3111330534 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d28003520 con 0x7f2d3c06dfa0 2024-04-25T10:05:22.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.446+0000 7f390726f640 1 -- 172.21.15.16:0/3696976390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38ec005210 msgr2=0x7f38ec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.446+0000 7f2d423d5640 1 -- 172.21.15.16:0/3111330534 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d3c159b80 msgr2=0x7f2d3c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.446+0000 7f7aae9bd640 1 Processor -- start 2024-04-25T10:05:22.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.446+0000 7f2d423d5640 1 --2- 172.21.15.16:0/3111330534 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d3c159b80 0x7f2d3c15d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.446+0000 7f144f7fe640 1 -- 172.21.15.16:0/2580859810 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444037070 con 0x7f14480078d0 2024-04-25T10:05:22.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7f2d423d5640 1 -- 172.21.15.16:0/3111330534 >> v1:172.21.15.16:6789/0 conn(0x7f2d3c06dfa0 legacy=0x7f2d3c15dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.446+0000 7fdf09c6b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fdf0415bc50 0x7fdf040cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7fdf09c6b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdf040c5ba0 0x7fdf04099b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7fdf09c6b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf040cf7a0 con 0x7fdf0415f6b0 2024-04-25T10:05:22.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7fdf09c6b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0415c030 con 0x7fdf040c5ba0 2024-04-25T10:05:22.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7fdf09c6b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf040cc7a0 con 0x7fdf0415bc50 2024-04-25T10:05:22.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.446+0000 7f390726f640 1 --2- 172.21.15.16:0/3696976390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38ec005210 0x7f38ec00b9c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00ba30 tx=0x7f38ec005830 comp rx=0 tx=0).stop 2024-04-25T10:05:22.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7f390580c640 1 -- 172.21.15.16:0/3696976390 reap_dead start 2024-04-25T10:05:22.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7f144f7fe640 1 -- 172.21.15.16:0/2580859810 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1444032640 con 0x7f14480078d0 2024-04-25T10:05:22.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7f390726f640 1 -- 172.21.15.16:0/3696976390 shutdown_connections 2024-04-25T10:05:22.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7f390726f640 1 -- 172.21.15.16:0/3696976390 >> 172.21.15.16:0/3696976390 conn(0x7f390006aba0 msgr2=0x7f390006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7f2d423d5640 1 -- 172.21.15.16:0/3111330534 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3c0b36a0 con 0x7f2d3c159f60 2024-04-25T10:05:22.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7f7aae9bd640 1 -- start start 2024-04-25T10:05:22.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7f8830831640 1 -- 172.21.15.16:0/746196417 shutdown_connections 2024-04-25T10:05:22.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7f8830831640 1 --2- 172.21.15.16:0/746196417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f881c0053c0 0x7f881c00b9c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7f8830831640 1 -- 172.21.15.16:0/746196417 >> 172.21.15.16:0/746196417 conn(0x7f8824053740 msgr2=0x7f8824053b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7fdf03fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdf040c5ba0 0x7fdf04099b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7fdf037fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fdf0415bc50 0x7fdf040cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7fdf037fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fdf0415bc50 0x7fdf040cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:53800/0 (socket says 172.21.15.16:53800) 2024-04-25T10:05:22.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7fdf037fe640 1 -- 172.21.15.16:0/2786188630 learned_addr learned my addr 172.21.15.16:0/2786188630 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:22.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7f144f7fe640 1 -- 172.21.15.16:0/2580859810 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444037070 con 0x7f14480078d0 2024-04-25T10:05:22.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7f390726f640 1 -- 172.21.15.16:0/3696976390 shutdown_connections 2024-04-25T10:05:22.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7f390726f640 1 -- 172.21.15.16:0/3696976390 wait complete. 2024-04-25T10:05:22.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.447+0000 7fc79cda5640 1 Processor -- start 2024-04-25T10:05:22.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7fc79cda5640 1 -- start start 2024-04-25T10:05:22.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7fdf03fff640 1 -- 172.21.15.16:0/2786188630 >> v2:172.21.15.186:3300/0 conn(0x7fdf0415bc50 msgr2=0x7fdf040cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7fdf03fff640 1 --2- 172.21.15.16:0/2786188630 >> v2:172.21.15.186:3300/0 conn(0x7fdf0415bc50 0x7fdf040cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7fdf03fff640 1 -- 172.21.15.16:0/2786188630 >> v1:172.21.15.186:6789/0 conn(0x7fdf0415f6b0 legacy=0x7fdf040c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7f8830831640 1 -- 172.21.15.16:0/746196417 shutdown_connections 2024-04-25T10:05:22.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7f8830831640 1 -- 172.21.15.16:0/746196417 wait complete. 2024-04-25T10:05:22.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7f1455b1c640 1 -- 172.21.15.16:0/2580859810 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14480078d0 msgr2=0x7f14480edd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7f1455b1c640 1 --2- 172.21.15.16:0/2580859810 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14480078d0 0x7f14480edd80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f14440097d0 tx=0x7f144402a2c0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7fdf03fff640 1 -- 172.21.15.16:0/2786188630 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf040cca80 con 0x7fdf040c5ba0 2024-04-25T10:05:22.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7fdf037fe640 1 --2- 172.21.15.16:0/2786188630 >> v2:172.21.15.186:3300/0 conn(0x7fdf0415bc50 0x7fdf040cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:22.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7f7aae9bd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7aa8071830 0x7f7aa8166050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7f7aae9bd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa81666b0 con 0x7f7aa8148290 2024-04-25T10:05:22.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7f7aae9bd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa8162610 con 0x7f7aa81630f0 2024-04-25T10:05:22.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7f7aae9bd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa81628f0 con 0x7f7aa8071830 2024-04-25T10:05:22.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7f7aad9bb640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7aa8148290 0x7f7aa8071120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34912/0 (socket says 172.21.15.16:34912) 2024-04-25T10:05:22.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7f7aad9bb640 1 -- 172.21.15.16:0/255196128 learned_addr learned my addr 172.21.15.16:0/255196128 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:22.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7f7aae1bc640 1 --2- 172.21.15.16:0/255196128 >> v2:172.21.15.16:3300/0 conn(0x7f7aa8071830 0x7f7aa8166050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7fc79cda5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc79806ead0 0x7fc7980aa360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7fc79cda5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79806f2f0 0x7fc798203d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7f7aac9b9640 1 -- 172.21.15.16:0/255196128 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389754480 0 0) 0x7f7aa81666b0 con 0x7f7aa8148290 2024-04-25T10:05:22.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7f7aac9b9640 1 -- 172.21.15.16:0/255196128 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a94003520 con 0x7f7aa8148290 2024-04-25T10:05:22.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7fc79659c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc79806ead0 0x7fc7980aa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7f1455b1c640 1 -- 172.21.15.16:0/2580859810 shutdown_connections 2024-04-25T10:05:22.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7f1455b1c640 1 --2- 172.21.15.16:0/2580859810 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14480078d0 0x7f14480edd80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.448+0000 7f1455b1c640 1 -- 172.21.15.16:0/2580859810 >> 172.21.15.16:0/2580859810 conn(0x7f14480559d0 msgr2=0x7f1448055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7fc795d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79806f2f0 0x7fc798203d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7fc795d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79806f2f0 0x7fc798203d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42308/0 (socket says 172.21.15.16:42308) 2024-04-25T10:05:22.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7fc795d9b640 1 -- 172.21.15.16:0/3816061568 learned_addr learned my addr 172.21.15.16:0/3816061568 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:22.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7f7aac9b9640 1 -- 172.21.15.16:0/255196128 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3834937162 0 0) 0x7f7a94003520 con 0x7f7aa8148290 2024-04-25T10:05:22.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7f7aae1bc640 1 -- 172.21.15.16:0/255196128 >> v1:172.21.15.186:6789/0 conn(0x7f7aa81630f0 legacy=0x7f7aa8165940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:05:22.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7f7aae1bc640 1 -- 172.21.15.16:0/255196128 >> v1:172.21.15.16:6789/0 conn(0x7f7aa8148290 legacy=0x7f7aa8071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7fc79cda5640 1 --2- 172.21.15.16:0/3816061568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7980cf670 0x7fc798204270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7f7aae1bc640 1 -- 172.21.15.16:0/255196128 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa8162bd0 con 0x7f7aa8071830 2024-04-25T10:05:22.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7fdf03fff640 1 --2- 172.21.15.16:0/2786188630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf040c5ba0 0x7fdf04099b20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdf0406f490 tx=0x7fdef802a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483f79f15dc0e35e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7f2d423d5640 1 --2- 172.21.15.16:0/3111330534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d3c159f60 0x7f2d3c15a3a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c00a940 tx=0x7f2d2c02a3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53688dbb9cc146da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7f1455b1c640 1 -- 172.21.15.16:0/2580859810 shutdown_connections 2024-04-25T10:05:22.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7f1455b1c640 1 -- 172.21.15.16:0/2580859810 wait complete. 2024-04-25T10:05:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7fc79cda5640 1 -- 172.21.15.16:0/3816061568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79815a080 con 0x7fc79806f2f0 2024-04-25T10:05:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7fc79cda5640 1 -- 172.21.15.16:0/3816061568 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc798159f00 con 0x7fc79806ead0 2024-04-25T10:05:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.450+0000 7f390726f640 1 Processor -- start 2024-04-25T10:05:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.449+0000 7fc79cda5640 1 -- 172.21.15.16:0/3816061568 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980cd1e0 con 0x7fc7980cf670 2024-04-25T10:05:22.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.450+0000 7f390726f640 1 -- start start 2024-04-25T10:05:22.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.450+0000 7f8830831640 1 Processor -- start 2024-04-25T10:05:22.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.450+0000 7fdf027fc640 1 -- 172.21.15.16:0/2786188630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8037070 con 0x7fdf040c5ba0 2024-04-25T10:05:22.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.450+0000 7fdf027fc640 1 -- 172.21.15.16:0/2786188630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef802ae30 con 0x7fdf040c5ba0 2024-04-25T10:05:22.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.450+0000 7f8830831640 1 -- start start 2024-04-25T10:05:22.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.450+0000 7fc795d9b640 1 -- 172.21.15.16:0/3816061568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7980cf670 msgr2=0x7fc798204270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.450+0000 7fc795d9b640 1 --2- 172.21.15.16:0/3816061568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7980cf670 0x7fc798204270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.450+0000 7fc795d9b640 1 -- 172.21.15.16:0/3816061568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc79806ead0 msgr2=0x7fc7980aa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.450+0000 7fc795d9b640 1 --2- 172.21.15.16:0/3816061568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc79806ead0 0x7fc7980aa360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.450+0000 7fc795d9b640 1 -- 172.21.15.16:0/3816061568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980aad90 con 0x7fc79806f2f0 2024-04-25T10:05:22.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.450+0000 7fc79659c640 1 --2- 172.21.15.16:0/3816061568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc79806ead0 0x7fc7980aa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:22.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7f7aae1bc640 1 --2- 172.21.15.16:0/255196128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aa8071830 0x7f7aa8166050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40097a0 tx=0x7f7aa402a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3471be6fdd551578 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7fc795d9b640 1 --2- 172.21.15.16:0/3816061568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79806f2f0 0x7fc798203d30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc79000ce00 tx=0x7fc79000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7fdf09c6b640 1 -- 172.21.15.16:0/2786188630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf040c5ba0 msgr2=0x7fdf04099b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7fdf09c6b640 1 --2- 172.21.15.16:0/2786188630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf040c5ba0 0x7fdf04099b20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdf0406f490 tx=0x7fdef802a330 comp rx=0 tx=0).stop 2024-04-25T10:05:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7fc786ffd640 1 -- 172.21.15.16:0/3816061568 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79000ae20 con 0x7fc79806f2f0 2024-04-25T10:05:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7fdf09c6b640 1 -- 172.21.15.16:0/2786188630 shutdown_connections 2024-04-25T10:05:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7fdf09c6b640 1 --2- 172.21.15.16:0/2786188630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf040c5ba0 0x7fdf04099b20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7fdf09c6b640 1 --2- 172.21.15.16:0/2786188630 >> v2:172.21.15.186:3300/0 conn(0x7fdf0415bc50 0x7fdf040cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7fdf09c6b640 1 -- 172.21.15.16:0/2786188630 >> 172.21.15.16:0/2786188630 conn(0x7fdf040b3430 msgr2=0x7fdf040b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7fc79cda5640 1 -- 172.21.15.16:0/3816061568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7980ab070 con 0x7fc79806f2f0 2024-04-25T10:05:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7fdf09c6b640 1 -- 172.21.15.16:0/2786188630 shutdown_connections 2024-04-25T10:05:22.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7f390726f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38ec005210 0x7f39000b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7f390726f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f390015bd30 0x7f390015d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7f390726f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39000b2070 0x7f3900143720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7f390726f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390015e420 con 0x7f38ec005210 2024-04-25T10:05:22.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7f390726f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39000b3c60 con 0x7f390015bd30 2024-04-25T10:05:22.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7f390726f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390015e5a0 con 0x7f39000b2070 2024-04-25T10:05:22.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7fc79cda5640 1 -- 172.21.15.16:0/3816061568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7980ab630 con 0x7fc79806f2f0 2024-04-25T10:05:22.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7f2d40bd2640 1 -- 172.21.15.16:0/3111330534 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c037070 con 0x7f2d3c159f60 2024-04-25T10:05:22.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7f2d40bd2640 1 -- 172.21.15.16:0/3111330534 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2c032640 con 0x7f2d3c159f60 2024-04-25T10:05:22.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7f8830831640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f881c0053c0 0x7f88240deba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7f2d40bd2640 1 -- 172.21.15.16:0/3111330534 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c037070 con 0x7f2d3c159f60 2024-04-25T10:05:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7f390500b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38ec005210 0x7f39000b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7f390500b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38ec005210 0x7f39000b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42314/0 (socket says 172.21.15.16:42314) 2024-04-25T10:05:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7f390500b640 1 -- 172.21.15.16:0/576681482 learned_addr learned my addr 172.21.15.16:0/576681482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7fc786ffd640 1 -- 172.21.15.16:0/3816061568 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79000f480 con 0x7fc79806f2f0 2024-04-25T10:05:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.452+0000 7fc786ffd640 1 -- 172.21.15.16:0/3816061568 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79000f830 con 0x7fc79806f2f0 2024-04-25T10:05:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.452+0000 7f1455b1c640 1 Processor -- start 2024-04-25T10:05:22.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.452+0000 7f390500b640 1 -- 172.21.15.16:0/576681482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39000b2070 msgr2=0x7f3900143720 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:05:22.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.452+0000 7f390500b640 1 --2- 172.21.15.16:0/576681482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39000b2070 0x7f3900143720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.452+0000 7f390500b640 1 -- 172.21.15.16:0/576681482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f390015bd30 msgr2=0x7f390015d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.452+0000 7f390500b640 1 --2- 172.21.15.16:0/576681482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f390015bd30 0x7f390015d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.452+0000 7f390500b640 1 -- 172.21.15.16:0/576681482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390015d1f0 con 0x7f38ec005210 2024-04-25T10:05:22.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7fdf09c6b640 1 -- 172.21.15.16:0/2786188630 wait complete. 2024-04-25T10:05:22.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.452+0000 7f390500b640 1 --2- 172.21.15.16:0/576681482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38ec005210 0x7f39000b1b30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f38f000b3f0 tx=0x7f38f000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.452+0000 7f2d43e38640 1 -- 172.21.15.16:0/3111330534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d3c159f60 msgr2=0x7f2d3c15a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7f7aac9b9640 1 -- 172.21.15.16:0/255196128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4037070 con 0x7f7aa8071830 2024-04-25T10:05:22.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.451+0000 7f7aac9b9640 1 -- 172.21.15.16:0/255196128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa402ab60 con 0x7f7aa8071830 2024-04-25T10:05:22.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.452+0000 7f7aac9b9640 1 -- 172.21.15.16:0/255196128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa00019f0 con 0x7f7aa8071830 2024-04-25T10:05:22.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.452+0000 7f7aae9bd640 1 -- 172.21.15.16:0/255196128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aa8071830 msgr2=0x7f7aa8166050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.452+0000 7f7aae9bd640 1 --2- 172.21.15.16:0/255196128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aa8071830 0x7f7aa8166050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40097a0 tx=0x7f7aa402a0d0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.452+0000 7f7aae9bd640 1 -- 172.21.15.16:0/255196128 shutdown_connections 2024-04-25T10:05:22.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.452+0000 7f7aae9bd640 1 --2- 172.21.15.16:0/255196128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aa8071830 0x7f7aa8166050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.452+0000 7f7aae9bd640 1 -- 172.21.15.16:0/255196128 >> 172.21.15.16:0/255196128 conn(0x7f7aa806aba0 msgr2=0x7f7aa806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.453+0000 7f7aae9bd640 1 -- 172.21.15.16:0/255196128 shutdown_connections 2024-04-25T10:05:22.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.453+0000 7f8830831640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88240d4cf0 0x7f88240df0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.453+0000 7f7aae9bd640 1 -- 172.21.15.16:0/255196128 wait complete. 2024-04-25T10:05:22.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.453+0000 7f1455b1c640 1 -- start start 2024-04-25T10:05:22.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.453+0000 7f8830831640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f882404b020 0x7f88240d5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.453+0000 7f38f5ffb640 1 -- 172.21.15.16:0/576681482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0015020 con 0x7f38ec005210 2024-04-25T10:05:22.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.453+0000 7fdf09c6b640 1 Processor -- start 2024-04-25T10:05:22.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.453+0000 7f882b7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f881c0053c0 0x7f88240deba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.453+0000 7f882affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88240d4cf0 0x7f88240df0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.453+0000 7f882b7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f881c0053c0 0x7f88240deba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:53838/0 (socket says 172.21.15.16:53838) 2024-04-25T10:05:22.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.453+0000 7f882affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88240d4cf0 0x7f88240df0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42326/0 (socket says 172.21.15.16:42326) 2024-04-25T10:05:22.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.453+0000 7f882b7fe640 1 -- 172.21.15.16:0/1142485241 learned_addr learned my addr 172.21.15.16:0/1142485241 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:22.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.453+0000 7f8830831640 1 -- 172.21.15.16:0/1142485241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88240564e0 con 0x7f88240d4cf0 2024-04-25T10:05:22.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.453+0000 7f8830831640 1 -- 172.21.15.16:0/1142485241 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8824054260 con 0x7f881c0053c0 2024-04-25T10:05:22.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.453+0000 7f882bfff640 1 --2- 172.21.15.16:0/1142485241 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f882404b020 0x7f88240d5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.453+0000 7f8830831640 1 -- 172.21.15.16:0/1142485241 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8824056660 con 0x7f882404b020 2024-04-25T10:05:22.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7f2d43e38640 1 --2- 172.21.15.16:0/3111330534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d3c159f60 0x7f2d3c15a3a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c00a940 tx=0x7f2d2c02a3a0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7f882affd640 1 -- 172.21.15.16:0/1142485241 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f882404b020 msgr2=0x7f88240d5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7f882affd640 1 --2- 172.21.15.16:0/1142485241 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f882404b020 0x7f88240d5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7f882affd640 1 -- 172.21.15.16:0/1142485241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f881c0053c0 msgr2=0x7f88240deba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7f882affd640 1 --2- 172.21.15.16:0/1142485241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f881c0053c0 0x7f88240deba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7f882affd640 1 -- 172.21.15.16:0/1142485241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88240e1950 con 0x7f88240d4cf0 2024-04-25T10:05:22.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7f1455b1c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14480078d0 0x7f14480d67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.453+0000 7f390726f640 1 -- 172.21.15.16:0/576681482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39000b2530 con 0x7f38ec005210 2024-04-25T10:05:22.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.453+0000 7f390726f640 1 -- 172.21.15.16:0/576681482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390015d490 con 0x7f38ec005210 2024-04-25T10:05:22.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7f38f5ffb640 1 -- 172.21.15.16:0/576681482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f0011830 con 0x7f38ec005210 2024-04-25T10:05:22.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7f38f5ffb640 1 -- 172.21.15.16:0/576681482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0011c20 con 0x7f38ec005210 2024-04-25T10:05:22.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7f1455b1c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144803e830 0x7f14480d8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7f1455b1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14480e30b0 0x7f14480e16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7f1455b1c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14480542f0 con 0x7f14480e30b0 2024-04-25T10:05:22.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7f1455b1c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14480e3490 con 0x7f14480078d0 2024-04-25T10:05:22.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7f1455b1c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1448054470 con 0x7f144803e830 2024-04-25T10:05:22.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7f145531b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14480e30b0 0x7f14480e16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7f145531b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14480e30b0 0x7f14480e16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42340/0 (socket says 172.21.15.16:42340) 2024-04-25T10:05:22.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7f145531b640 1 -- 172.21.15.16:0/2009033817 learned_addr learned my addr 172.21.15.16:0/2009033817 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:22.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7f2d43e38640 1 -- 172.21.15.16:0/3111330534 shutdown_connections 2024-04-25T10:05:22.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7f2d43e38640 1 --2- 172.21.15.16:0/3111330534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d3c159f60 0x7f2d3c15a3a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7f882affd640 1 --2- 172.21.15.16:0/1142485241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88240d4cf0 0x7f88240df0e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8818001050 tx=0x7f881800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7f2d43e38640 1 --2- 172.21.15.16:0/3111330534 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d3c159b80 0x7f2d3c15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7f2d43e38640 1 -- 172.21.15.16:0/3111330534 >> 172.21.15.16:0/3111330534 conn(0x7f2d3c06aba0 msgr2=0x7f2d3c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7f7aae9bd640 1 Processor -- start 2024-04-25T10:05:22.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7fdf09c6b640 1 -- start start 2024-04-25T10:05:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7f145531b640 1 -- 172.21.15.16:0/2009033817 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144803e830 msgr2=0x7f14480d8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7f145531b640 1 --2- 172.21.15.16:0/2009033817 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144803e830 0x7f14480d8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7f145531b640 1 -- 172.21.15.16:0/2009033817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14480078d0 msgr2=0x7f14480d67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7f145531b640 1 --2- 172.21.15.16:0/2009033817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14480078d0 0x7f14480d67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7f145531b640 1 -- 172.21.15.16:0/2009033817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14480d72a0 con 0x7f14480e30b0 2024-04-25T10:05:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7fdf09c6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf040c5ba0 0x7fdf041f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7fdf09c6b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf0415bc50 0x7fdf0406d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.454+0000 7fc784ff9640 1 -- 172.21.15.16:0/3816061568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc770005290 con 0x7fc79806f2f0 2024-04-25T10:05:22.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7fdf09c6b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf0415f6b0 0x7fdf040c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7fdf09c6b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040cdd80 con 0x7fdf040c5ba0 2024-04-25T10:05:22.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7fdf09c6b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040cd040 con 0x7fdf0415bc50 2024-04-25T10:05:22.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7fdf09c6b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040cc450 con 0x7fdf0415f6b0 2024-04-25T10:05:22.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7f145531b640 1 --2- 172.21.15.16:0/2009033817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14480e30b0 0x7f14480e16f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1444009720 tx=0x7f14440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7f2d43e38640 1 -- 172.21.15.16:0/3111330534 shutdown_connections 2024-04-25T10:05:22.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7f2d43e38640 1 -- 172.21.15.16:0/3111330534 wait complete. 2024-04-25T10:05:22.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7f7aae9bd640 1 -- start start 2024-04-25T10:05:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7fdf02ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf0415bc50 0x7fdf0406d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7fdf02ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf0415bc50 0x7fdf0406d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:53844/0 (socket says 172.21.15.16:53844) 2024-04-25T10:05:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7fdf02ffd640 1 -- 172.21.15.16:0/2053160962 learned_addr learned my addr 172.21.15.16:0/2053160962 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7fdf03fff640 1 --2- 172.21.15.16:0/2053160962 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf0415f6b0 0x7fdf040c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.455+0000 7fc786ffd640 1 -- 172.21.15.16:0/3816061568 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc790018400 con 0x7fc79806f2f0 2024-04-25T10:05:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7fc786ffd640 1 --2- 172.21.15.16:0/3816061568 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc77806b4d0 0x7fc77806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7fc786ffd640 1 -- 172.21.15.16:0/3816061568 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc79008b250 con 0x7fc79806f2f0 2024-04-25T10:05:22.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7f7aae9bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aa8071830 0x7f7aa8156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7f7aae9bd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aa8148290 0x7f7aa813afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7fdf037fe640 1 --2- 172.21.15.16:0/2053160962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf040c5ba0 0x7fdf041f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7f7aae9bd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7aa81630f0 0x7f7aa806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7f7aae9bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa816cc40 con 0x7f7aa8071830 2024-04-25T10:05:22.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7f7aae9bd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa816cac0 con 0x7f7aa8148290 2024-04-25T10:05:22.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7f7aae9bd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa816cdc0 con 0x7f7aa81630f0 2024-04-25T10:05:22.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7fc79659c640 1 --2- 172.21.15.16:0/3816061568 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc77806b4d0 0x7fc77806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7fdf037fe640 1 -- 172.21.15.16:0/2053160962 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf0415f6b0 msgr2=0x7fdf040c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7f390726f640 1 -- 172.21.15.16:0/576681482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e0005290 con 0x7f38ec005210 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7fdf037fe640 1 --2- 172.21.15.16:0/2053160962 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf0415f6b0 0x7fdf040c4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7fdf037fe640 1 -- 172.21.15.16:0/2053160962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf0415bc50 msgr2=0x7fdf0406d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7fdf037fe640 1 --2- 172.21.15.16:0/2053160962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf0415bc50 0x7fdf0406d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7fdf037fe640 1 -- 172.21.15.16:0/2053160962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf040c9470 con 0x7fdf040c5ba0 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7f144d7fa640 1 -- 172.21.15.16:0/2009033817 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444041070 con 0x7f14480e30b0 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7f1455b1c640 1 -- 172.21.15.16:0/2009033817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14480d7580 con 0x7f14480e30b0 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7f1455b1c640 1 -- 172.21.15.16:0/2009033817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14480d7b40 con 0x7f14480e30b0 2024-04-25T10:05:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.457+0000 7fdf02ffd640 1 --2- 172.21.15.16:0/2053160962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf0415bc50 0x7fdf0406d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7f7aae1bc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7aa81630f0 0x7f7aa806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7f7aae1bc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7aa81630f0 0x7f7aa806d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:34362/0 (socket says 172.21.15.16:34362) 2024-04-25T10:05:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.456+0000 7f7aae1bc640 1 -- 172.21.15.16:0/3552154854 learned_addr learned my addr 172.21.15.16:0/3552154854 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.457+0000 7f7aad9bb640 1 --2- 172.21.15.16:0/3552154854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aa8071830 0x7f7aa8156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.457+0000 7fdf037fe640 1 --2- 172.21.15.16:0/2053160962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf040c5ba0 0x7fdf041f8a10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdef400e820 tx=0x7fdef400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.457+0000 7fdf03fff640 1 --2- 172.21.15.16:0/2053160962 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf0415f6b0 0x7fdf040c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:22.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.457+0000 7f2d43e38640 1 Processor -- start 2024-04-25T10:05:22.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.457+0000 7f2d43e38640 1 -- start start 2024-04-25T10:05:22.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.457+0000 7f7aae1bc640 1 -- 172.21.15.16:0/3552154854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aa8148290 msgr2=0x7f7aa813afc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:05:22.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.457+0000 7f7aae1bc640 1 --2- 172.21.15.16:0/3552154854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aa8148290 0x7f7aa813afc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.457+0000 7f7aae1bc640 1 -- 172.21.15.16:0/3552154854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aa8071830 msgr2=0x7f7aa8156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.457+0000 7f7aae1bc640 1 --2- 172.21.15.16:0/3552154854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aa8071830 0x7f7aa8156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.457+0000 7f7aae1bc640 1 -- 172.21.15.16:0/3552154854 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa813bc50 con 0x7f7aa81630f0 2024-04-25T10:05:22.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.457+0000 7fc79659c640 1 --2- 172.21.15.16:0/3816061568 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc77806b4d0 0x7fc77806d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc788001750 tx=0x7fc788009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.457+0000 7f7aae1bc640 1 --2- 172.21.15.16:0/3552154854 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7aa81630f0 0x7f7aa806d190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7aa402a6c0 tx=0x7f7aa40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.458+0000 7f8817fff640 1 -- 172.21.15.16:0/1142485241 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881800c690 con 0x7f88240d4cf0 2024-04-25T10:05:22.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.458+0000 7f8830831640 1 -- 172.21.15.16:0/1142485241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88240e1c30 con 0x7f88240d4cf0 2024-04-25T10:05:22.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.458+0000 7f8830831640 1 -- 172.21.15.16:0/1142485241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88240e24d0 con 0x7f88240d4cf0 2024-04-25T10:05:22.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.458+0000 7f38f5ffb640 1 -- 172.21.15.16:0/576681482 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f38f0018020 con 0x7f38ec005210 2024-04-25T10:05:22.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.458+0000 7f7a9e7fc640 1 -- 172.21.15.16:0/3552154854 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4037040 con 0x7f7aa81630f0 2024-04-25T10:05:22.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.458+0000 7f1455b1c640 1 -- 172.21.15.16:0/2009033817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14480057d0 con 0x7f14480e30b0 2024-04-25T10:05:22.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.458+0000 7f7aae9bd640 1 -- 172.21.15.16:0/3552154854 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa813bf30 con 0x7f7aa81630f0 2024-04-25T10:05:22.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.458+0000 7f2d43e38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d3c06dfa0 0x7f2d3c1ea630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.458+0000 7f2d43e38640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d3c159b80 0x7f2d3c155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.458+0000 7f38f5ffb640 1 --2- 172.21.15.16:0/576681482 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f38e406b5a0 0x7f38e406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.458+0000 7f2d43e38640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d3c159f60 0x7f2d3c156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.458+0000 7f2d43e38640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c15e420 con 0x7f2d3c06dfa0 2024-04-25T10:05:22.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.458+0000 7f2d43e38640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c0b3c60 con 0x7f2d3c159f60 2024-04-25T10:05:22.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.458+0000 7f2d43e38640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c15e5a0 con 0x7f2d3c159b80 2024-04-25T10:05:22.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.459+0000 7f390480a640 1 --2- 172.21.15.16:0/576681482 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f38e406b5a0 0x7f38e406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.459+0000 7f38f5ffb640 1 -- 172.21.15.16:0/576681482 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38f008a5d0 con 0x7f38ec005210 2024-04-25T10:05:22.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.459+0000 7f2d413d3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d3c159b80 0x7f2d3c155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.459+0000 7f2d413d3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d3c159b80 0x7f2d3c155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:34366/0 (socket says 172.21.15.16:34366) 2024-04-25T10:05:22.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.459+0000 7f2d413d3640 1 -- 172.21.15.16:0/1092273813 learned_addr learned my addr 172.21.15.16:0/1092273813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:22.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.459+0000 7f390480a640 1 --2- 172.21.15.16:0/576681482 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f38e406b5a0 0x7f38e406da60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f39000ae080 tx=0x7f38f80012d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.458+0000 7f7aae9bd640 1 -- 172.21.15.16:0/3552154854 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa81570d0 con 0x7f7aa81630f0 2024-04-25T10:05:22.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.458+0000 7f7a9e7fc640 1 -- 172.21.15.16:0/3552154854 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa4004230 con 0x7f7aa81630f0 2024-04-25T10:05:22.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.459+0000 7f7a9e7fc640 1 -- 172.21.15.16:0/3552154854 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4003610 con 0x7f7aa81630f0 2024-04-25T10:05:22.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.459+0000 7f8830831640 1 -- 172.21.15.16:0/1142485241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88240e1e00 con 0x7f88240d4cf0 2024-04-25T10:05:22.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.459+0000 7f2d41bd4640 1 --2- 172.21.15.16:0/1092273813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d3c06dfa0 0x7f2d3c1ea630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.459+0000 7f2d423d5640 1 --2- 172.21.15.16:0/1092273813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d3c159f60 0x7f2d3c156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.459+0000 7f7aae9bd640 1 -- 172.21.15.16:0/3552154854 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a90005290 con 0x7f7aa81630f0 2024-04-25T10:05:22.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.460+0000 7f2d413d3640 1 -- 172.21.15.16:0/1092273813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d3c159f60 msgr2=0x7f2d3c156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.460+0000 7f2d413d3640 1 --2- 172.21.15.16:0/1092273813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d3c159f60 0x7f2d3c156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.460+0000 7f2d413d3640 1 -- 172.21.15.16:0/1092273813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d3c06dfa0 msgr2=0x7f2d3c1ea630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.460+0000 7f2d413d3640 1 --2- 172.21.15.16:0/1092273813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d3c06dfa0 0x7f2d3c1ea630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.460+0000 7f2d413d3640 1 -- 172.21.15.16:0/1092273813 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3c0b0c60 con 0x7f2d3c159b80 2024-04-25T10:05:22.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.460+0000 7f2d423d5640 1 --2- 172.21.15.16:0/1092273813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d3c159f60 0x7f2d3c156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:22.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.460+0000 7f2d41bd4640 1 --2- 172.21.15.16:0/1092273813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d3c06dfa0 0x7f2d3c1ea630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:22.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.460+0000 7f2d413d3640 1 --2- 172.21.15.16:0/1092273813 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d3c159b80 0x7f2d3c155d60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2d340019e0 tx=0x7f2d34001ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.460+0000 7f2d327fc640 1 -- 172.21.15.16:0/1092273813 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d34002090 con 0x7f2d3c159b80 2024-04-25T10:05:22.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.460+0000 7f2d327fc640 1 -- 172.21.15.16:0/1092273813 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3400d6c0 con 0x7f2d3c159b80 2024-04-25T10:05:22.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.460+0000 7f2d327fc640 1 -- 172.21.15.16:0/1092273813 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3400dad0 con 0x7f2d3c159b80 2024-04-25T10:05:22.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.460+0000 7f7a9e7fc640 1 -- 172.21.15.16:0/3552154854 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7aa402ad10 con 0x7f7aa81630f0 2024-04-25T10:05:22.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.460+0000 7f2d43e38640 1 -- 172.21.15.16:0/1092273813 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d3c0afea0 con 0x7f2d3c159b80 2024-04-25T10:05:22.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.460+0000 7f2d43e38640 1 -- 172.21.15.16:0/1092273813 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d3c0b0460 con 0x7f2d3c159b80 2024-04-25T10:05:22.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.461+0000 7f7a9e7fc640 1 --2- 172.21.15.16:0/3552154854 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a8c06b520 0x7f7a8c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.461+0000 7f7a9e7fc640 1 -- 172.21.15.16:0/3552154854 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7aa40aabd0 con 0x7f7aa81630f0 2024-04-25T10:05:22.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.461+0000 7f7aad9bb640 1 --2- 172.21.15.16:0/3552154854 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a8c06b520 0x7f7a8c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.461+0000 7f7aad9bb640 1 --2- 172.21.15.16:0/3552154854 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a8c06b520 0x7f7a8c06d9e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0001580 tx=0x7f7aa0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.462+0000 7f2d43e38640 1 -- 172.21.15.16:0/1092273813 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d3c0750b0 con 0x7f2d3c159b80 2024-04-25T10:05:22.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.463+0000 7fdeebfff640 1 -- 172.21.15.16:0/2053160962 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4017070 con 0x7fdf040c5ba0 2024-04-25T10:05:22.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.463+0000 7fdf09c6b640 1 -- 172.21.15.16:0/2053160962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf040c9750 con 0x7fdf040c5ba0 2024-04-25T10:05:22.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.463+0000 7fdf09c6b640 1 -- 172.21.15.16:0/2053160962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf040c9d10 con 0x7fdf040c5ba0 2024-04-25T10:05:22.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.463+0000 7fc786ffd640 1 -- 172.21.15.16:0/3816061568 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc790014030 con 0x7fc79806f2f0 2024-04-25T10:05:22.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.463+0000 7f8817fff640 1 -- 172.21.15.16:0/1142485241 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8818018070 con 0x7f88240d4cf0 2024-04-25T10:05:22.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.463+0000 7f8817fff640 1 -- 172.21.15.16:0/1142485241 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881800cc60 con 0x7f88240d4cf0 2024-04-25T10:05:22.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.463+0000 7f144d7fa640 1 -- 172.21.15.16:0/2009033817 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1444032640 con 0x7f14480e30b0 2024-04-25T10:05:22.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.463+0000 7f144d7fa640 1 -- 172.21.15.16:0/2009033817 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144404b060 con 0x7f14480e30b0 2024-04-25T10:05:22.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.464+0000 7fdf09c6b640 1 -- 172.21.15.16:0/2053160962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf040ca040 con 0x7fdf040c5ba0 2024-04-25T10:05:22.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.465+0000 7f7a9e7fc640 1 -- 172.21.15.16:0/3552154854 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7aa4079bc0 con 0x7f7aa81630f0 2024-04-25T10:05:22.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.466+0000 7f2d327fc640 1 -- 172.21.15.16:0/1092273813 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d3401b020 con 0x7f2d3c159b80 2024-04-25T10:05:22.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.467+0000 7f2d327fc640 1 --2- 172.21.15.16:0/1092273813 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2d2006b4d0 0x7f2d2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.467+0000 7f2d327fc640 1 -- 172.21.15.16:0/1092273813 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d3408a8a0 con 0x7f2d3c159b80 2024-04-25T10:05:22.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.467+0000 7f2d41bd4640 1 --2- 172.21.15.16:0/1092273813 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2d2006b4d0 0x7f2d2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.468+0000 7f38f5ffb640 1 -- 172.21.15.16:0/576681482 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f38f0059430 con 0x7f38ec005210 2024-04-25T10:05:22.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.468+0000 7f2d41bd4640 1 --2- 172.21.15.16:0/1092273813 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2d2006b4d0 0x7f2d2006d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d38009b90 tx=0x7f2d38008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.469+0000 7fdeebfff640 1 -- 172.21.15.16:0/2053160962 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef4004020 con 0x7fdf040c5ba0 2024-04-25T10:05:22.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.469+0000 7fdeebfff640 1 -- 172.21.15.16:0/2053160962 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4021060 con 0x7fdf040c5ba0 2024-04-25T10:05:22.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.470+0000 7f8817fff640 1 -- 172.21.15.16:0/1142485241 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f881801d460 con 0x7f88240d4cf0 2024-04-25T10:05:22.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.471+0000 7f8817fff640 1 --2- 172.21.15.16:0/1142485241 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f881006b4d0 0x7f881006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.471+0000 7fdeebfff640 1 -- 172.21.15.16:0/2053160962 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdef401a070 con 0x7fdf040c5ba0 2024-04-25T10:05:22.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.471+0000 7f882b7fe640 1 --2- 172.21.15.16:0/1142485241 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f881006b4d0 0x7f881006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.471+0000 7f8817fff640 1 -- 172.21.15.16:0/1142485241 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f881808ac40 con 0x7f88240d4cf0 2024-04-25T10:05:22.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.470+0000 7f144d7fa640 1 -- 172.21.15.16:0/2009033817 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1444044070 con 0x7f14480e30b0 2024-04-25T10:05:22.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.471+0000 7f144d7fa640 1 --2- 172.21.15.16:0/2009033817 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f143406b4d0 0x7f143406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.471+0000 7f144d7fa640 1 -- 172.21.15.16:0/2009033817 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14440aaf50 con 0x7f14480e30b0 2024-04-25T10:05:22.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.472+0000 7f882b7fe640 1 --2- 172.21.15.16:0/1142485241 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f881006b4d0 0x7f881006d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8820006b10 tx=0x7f8820008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.471+0000 7fdeebfff640 1 --2- 172.21.15.16:0/2053160962 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdef006b4d0 0x7fdef006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.472+0000 7f1454b1a640 1 --2- 172.21.15.16:0/2009033817 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f143406b4d0 0x7f143406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.472+0000 7fdf02ffd640 1 --2- 172.21.15.16:0/2053160962 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdef006b4d0 0x7fdef006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.472+0000 7f2d327fc640 1 -- 172.21.15.16:0/1092273813 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d34022070 con 0x7f2d3c159b80 2024-04-25T10:05:22.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.473+0000 7fdf02ffd640 1 --2- 172.21.15.16:0/2053160962 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdef006b4d0 0x7fdef006d990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdf0409a5b0 tx=0x7fdefc003e60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.473+0000 7f1454b1a640 1 --2- 172.21.15.16:0/2009033817 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f143406b4d0 0x7f143406d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f143c001570 tx=0x7f143c007930 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.475+0000 7f8817fff640 1 -- 172.21.15.16:0/1142485241 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8818059450 con 0x7f88240d4cf0 2024-04-25T10:05:22.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.484+0000 7f144d7fa640 1 -- 172.21.15.16:0/2009033817 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1444079f60 con 0x7f14480e30b0 2024-04-25T10:05:22.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.484+0000 7fdeebfff640 1 -- 172.21.15.16:0/2053160962 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdef4056c10 con 0x7fdf040c5ba0 2024-04-25T10:05:22.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.493+0000 7fdeebfff640 1 -- 172.21.15.16:0/2053160962 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdef4058b70 con 0x7fdf040c5ba0 2024-04-25T10:05:22.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.802+0000 7f88b57fa640 1 -- 172.21.15.16:0/1976629909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f88d0009880 con 0x7f88d00d7f40 2024-04-25T10:05:22.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.803+0000 7f88bf7fe640 1 -- 172.21.15.16:0/1976629909 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f88d80e2220 con 0x7f88d00d7f40 2024-04-25T10:05:22.808 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-04-25T10:05:22.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.806+0000 7f88d7fff640 1 -- 172.21.15.16:0/1976629909 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f88c006b660 msgr2=0x7f88c006db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.806+0000 7f88d7fff640 1 --2- 172.21.15.16:0/1976629909 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f88c006b660 0x7f88c006db20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f88d0008d70 tx=0x7f88cc006c20 comp rx=0 tx=0).stop 2024-04-25T10:05:22.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.806+0000 7f88d7fff640 1 -- 172.21.15.16:0/1976629909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d00d7f40 msgr2=0x7f88d00d8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.806+0000 7f88d7fff640 1 --2- 172.21.15.16:0/1976629909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d00d7f40 0x7f88d00d8af0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f88d8074ed0 tx=0x7f88d8068980 comp rx=0 tx=0).stop 2024-04-25T10:05:22.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.806+0000 7f88d7fff640 1 -- 172.21.15.16:0/1976629909 shutdown_connections 2024-04-25T10:05:22.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.806+0000 7f88d7fff640 1 --2- 172.21.15.16:0/1976629909 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f88c006b660 0x7f88c006db20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.807+0000 7f88d7fff640 1 --2- 172.21.15.16:0/1976629909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d00d7f40 0x7f88d00d8af0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.807+0000 7f88d7fff640 1 --2- 172.21.15.16:0/1976629909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88d0050080 0x7f88d00368c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.807+0000 7f88d7fff640 1 --2- 172.21.15.16:0/1976629909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88d003ad40 0x7f88d004ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.807+0000 7f88d7fff640 1 -- 172.21.15.16:0/1976629909 >> 172.21.15.16:0/1976629909 conn(0x7f88d0055340 msgr2=0x7f88d00e0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.807+0000 7f88d7fff640 1 -- 172.21.15.16:0/1976629909 shutdown_connections 2024-04-25T10:05:22.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.807+0000 7f88d7fff640 1 -- 172.21.15.16:0/1976629909 wait complete. 2024-04-25T10:05:22.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.4 2024-04-25T10:05:22.829 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:22.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.850+0000 7f7aae9bd640 1 -- 172.21.15.16:0/3552154854 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7a90005850 con 0x7f7aa81630f0 2024-04-25T10:05:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.851+0000 7f7a9e7fc640 1 -- 172.21.15.16:0/3552154854 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7aa40794c0 con 0x7f7aa81630f0 2024-04-25T10:05:22.856 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-04-25T10:05:22.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.853+0000 7f7aae9bd640 1 -- 172.21.15.16:0/3552154854 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a8c06b520 msgr2=0x7f7a8c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.853+0000 7f7aae9bd640 1 --2- 172.21.15.16:0/3552154854 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a8c06b520 0x7f7a8c06d9e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0001580 tx=0x7f7aa0009040 comp rx=0 tx=0).stop 2024-04-25T10:05:22.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.853+0000 7f7aae9bd640 1 -- 172.21.15.16:0/3552154854 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7aa81630f0 msgr2=0x7f7aa806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.853+0000 7f7aae9bd640 1 --2- 172.21.15.16:0/3552154854 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7aa81630f0 0x7f7aa806d190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7aa402a6c0 tx=0x7f7aa40046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.853+0000 7f7aae9bd640 1 -- 172.21.15.16:0/3552154854 shutdown_connections 2024-04-25T10:05:22.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.853+0000 7f7aae9bd640 1 --2- 172.21.15.16:0/3552154854 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a8c06b520 0x7f7a8c06d9e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.853+0000 7f7aae9bd640 1 --2- 172.21.15.16:0/3552154854 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7aa81630f0 0x7f7aa806d190 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.853+0000 7f7aae9bd640 1 --2- 172.21.15.16:0/3552154854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aa8148290 0x7f7aa813afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.853+0000 7f7aae9bd640 1 --2- 172.21.15.16:0/3552154854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aa8071830 0x7f7aa8156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.853+0000 7f7aae9bd640 1 -- 172.21.15.16:0/3552154854 >> 172.21.15.16:0/3552154854 conn(0x7f7aa806aba0 msgr2=0x7f7aa8163d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.853+0000 7f7aae9bd640 1 -- 172.21.15.16:0/3552154854 shutdown_connections 2024-04-25T10:05:22.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.853+0000 7f7aae9bd640 1 -- 172.21.15.16:0/3552154854 wait complete. 2024-04-25T10:05:22.869 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.7 2024-04-25T10:05:22.869 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.884+0000 7fc784ff9640 1 -- 172.21.15.16:0/3816061568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc770005580 con 0x7fc79806f2f0 2024-04-25T10:05:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.884+0000 7fc786ffd640 1 -- 172.21.15.16:0/3816061568 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc79005a040 con 0x7fc79806f2f0 2024-04-25T10:05:22.889 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-04-25T10:05:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.886+0000 7fc784ff9640 1 -- 172.21.15.16:0/3816061568 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc77806b4d0 msgr2=0x7fc77806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.886+0000 7fc784ff9640 1 --2- 172.21.15.16:0/3816061568 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc77806b4d0 0x7fc77806d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc788001750 tx=0x7fc788009040 comp rx=0 tx=0).stop 2024-04-25T10:05:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.886+0000 7fc784ff9640 1 -- 172.21.15.16:0/3816061568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79806f2f0 msgr2=0x7fc798203d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.886+0000 7fc784ff9640 1 --2- 172.21.15.16:0/3816061568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79806f2f0 0x7fc798203d30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc79000ce00 tx=0x7fc79000a650 comp rx=0 tx=0).stop 2024-04-25T10:05:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.886+0000 7fc784ff9640 1 -- 172.21.15.16:0/3816061568 shutdown_connections 2024-04-25T10:05:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.886+0000 7fc784ff9640 1 --2- 172.21.15.16:0/3816061568 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc77806b4d0 0x7fc77806d990 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.886+0000 7fc784ff9640 1 --2- 172.21.15.16:0/3816061568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7980cf670 0x7fc798204270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.886+0000 7fc784ff9640 1 --2- 172.21.15.16:0/3816061568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79806f2f0 0x7fc798203d30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.886+0000 7fc784ff9640 1 --2- 172.21.15.16:0/3816061568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc79806ead0 0x7fc7980aa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.886+0000 7fc784ff9640 1 -- 172.21.15.16:0/3816061568 >> 172.21.15.16:0/3816061568 conn(0x7fc79806aba0 msgr2=0x7fc7980afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.886+0000 7fc784ff9640 1 -- 172.21.15.16:0/3816061568 shutdown_connections 2024-04-25T10:05:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.886+0000 7fc784ff9640 1 -- 172.21.15.16:0/3816061568 wait complete. 2024-04-25T10:05:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.892+0000 7f390726f640 1 -- 172.21.15.16:0/576681482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f38e0005850 con 0x7f38ec005210 2024-04-25T10:05:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.892+0000 7f38f5ffb640 1 -- 172.21.15.16:0/576681482 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f38f0058d30 con 0x7f38ec005210 2024-04-25T10:05:22.898 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-04-25T10:05:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.895+0000 7f390726f640 1 -- 172.21.15.16:0/576681482 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f38e406b5a0 msgr2=0x7f38e406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.895+0000 7f390726f640 1 --2- 172.21.15.16:0/576681482 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f38e406b5a0 0x7f38e406da60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f39000ae080 tx=0x7f38f80012d0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.895+0000 7f390726f640 1 -- 172.21.15.16:0/576681482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38ec005210 msgr2=0x7f39000b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.895+0000 7f390726f640 1 --2- 172.21.15.16:0/576681482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38ec005210 0x7f39000b1b30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f38f000b3f0 tx=0x7f38f000b8b0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.895+0000 7f390726f640 1 -- 172.21.15.16:0/576681482 shutdown_connections 2024-04-25T10:05:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.895+0000 7f390726f640 1 --2- 172.21.15.16:0/576681482 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f38e406b5a0 0x7f38e406da60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.895+0000 7f390726f640 1 --2- 172.21.15.16:0/576681482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39000b2070 0x7f3900143720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.895+0000 7f390726f640 1 --2- 172.21.15.16:0/576681482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f390015bd30 0x7f390015d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.895+0000 7f390726f640 1 --2- 172.21.15.16:0/576681482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38ec005210 0x7f39000b1b30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.895+0000 7f390726f640 1 -- 172.21.15.16:0/576681482 >> 172.21.15.16:0/576681482 conn(0x7f390006aba0 msgr2=0x7f3900156710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.895+0000 7f390726f640 1 -- 172.21.15.16:0/576681482 shutdown_connections 2024-04-25T10:05:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.895+0000 7f390726f640 1 -- 172.21.15.16:0/576681482 wait complete. 2024-04-25T10:05:22.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.897+0000 7f2d43e38640 1 -- 172.21.15.16:0/1092273813 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2d3c06d930 con 0x7f2d3c159b80 2024-04-25T10:05:22.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.897+0000 7f2d327fc640 1 -- 172.21.15.16:0/1092273813 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2d3401b310 con 0x7f2d3c159b80 2024-04-25T10:05:22.902 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-04-25T10:05:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.901+0000 7f2d43e38640 1 -- 172.21.15.16:0/1092273813 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2d2006b4d0 msgr2=0x7f2d2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.901+0000 7f2d43e38640 1 --2- 172.21.15.16:0/1092273813 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2d2006b4d0 0x7f2d2006d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d38009b90 tx=0x7f2d38008040 comp rx=0 tx=0).stop 2024-04-25T10:05:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.901+0000 7f2d43e38640 1 -- 172.21.15.16:0/1092273813 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d3c159b80 msgr2=0x7f2d3c155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.901+0000 7f2d43e38640 1 --2- 172.21.15.16:0/1092273813 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d3c159b80 0x7f2d3c155d60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2d340019e0 tx=0x7f2d34001ea0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.901+0000 7f2d43e38640 1 -- 172.21.15.16:0/1092273813 shutdown_connections 2024-04-25T10:05:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.901+0000 7f2d43e38640 1 --2- 172.21.15.16:0/1092273813 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2d2006b4d0 0x7f2d2006d990 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.901+0000 7f2d43e38640 1 --2- 172.21.15.16:0/1092273813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d3c159f60 0x7f2d3c156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.901+0000 7f2d43e38640 1 --2- 172.21.15.16:0/1092273813 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d3c159b80 0x7f2d3c155d60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.901+0000 7f2d43e38640 1 --2- 172.21.15.16:0/1092273813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d3c06dfa0 0x7f2d3c1ea630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.901+0000 7f2d43e38640 1 -- 172.21.15.16:0/1092273813 >> 172.21.15.16:0/1092273813 conn(0x7f2d3c06aba0 msgr2=0x7f2d3c0b26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.901+0000 7f2d43e38640 1 -- 172.21.15.16:0/1092273813 shutdown_connections 2024-04-25T10:05:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.901+0000 7f2d43e38640 1 -- 172.21.15.16:0/1092273813 wait complete. 2024-04-25T10:05:22.908 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.1 2024-04-25T10:05:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.903+0000 7f8830831640 1 -- 172.21.15.16:0/1142485241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f88240057d0 con 0x7f88240d4cf0 2024-04-25T10:05:22.909 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.904+0000 7f8817fff640 1 -- 172.21.15.16:0/1142485241 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f881805d4f0 con 0x7f88240d4cf0 2024-04-25T10:05:22.909 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-04-25T10:05:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.907+0000 7f8830831640 1 -- 172.21.15.16:0/1142485241 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f881006b4d0 msgr2=0x7f881006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.907+0000 7f8830831640 1 --2- 172.21.15.16:0/1142485241 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f881006b4d0 0x7f881006d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8820006b10 tx=0x7f8820008040 comp rx=0 tx=0).stop 2024-04-25T10:05:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.907+0000 7f8830831640 1 -- 172.21.15.16:0/1142485241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88240d4cf0 msgr2=0x7f88240df0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.907+0000 7f8830831640 1 --2- 172.21.15.16:0/1142485241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88240d4cf0 0x7f88240df0e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8818001050 tx=0x7f881800eb90 comp rx=0 tx=0).stop 2024-04-25T10:05:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.907+0000 7f8830831640 1 -- 172.21.15.16:0/1142485241 shutdown_connections 2024-04-25T10:05:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.907+0000 7f8830831640 1 --2- 172.21.15.16:0/1142485241 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f881006b4d0 0x7f881006d990 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.907+0000 7f8830831640 1 --2- 172.21.15.16:0/1142485241 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f882404b020 0x7f88240d5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.907+0000 7f8830831640 1 --2- 172.21.15.16:0/1142485241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88240d4cf0 0x7f88240df0e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.907+0000 7f8830831640 1 --2- 172.21.15.16:0/1142485241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f881c0053c0 0x7f88240deba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.907+0000 7f8830831640 1 -- 172.21.15.16:0/1142485241 >> 172.21.15.16:0/1142485241 conn(0x7f8824053740 msgr2=0x7f8824039eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.907+0000 7f8830831640 1 -- 172.21.15.16:0/1142485241 shutdown_connections 2024-04-25T10:05:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.907+0000 7f8830831640 1 -- 172.21.15.16:0/1142485241 wait complete. 2024-04-25T10:05:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.912+0000 7f1455b1c640 1 -- 172.21.15.16:0/2009033817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f1448056b30 con 0x7f14480e30b0 2024-04-25T10:05:22.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.913+0000 7fdf09c6b640 1 -- 172.21.15.16:0/2053160962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fdf040750b0 con 0x7fdf040c5ba0 2024-04-25T10:05:22.919 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.0 2024-04-25T10:05:22.919 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:22.921 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.3 2024-04-25T10:05:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.913+0000 7f144d7fa640 1 -- 172.21.15.16:0/2009033817 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1444079860 con 0x7f14480e30b0 2024-04-25T10:05:22.921 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-04-25T10:05:22.922 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.913+0000 7fdeebfff640 1 -- 172.21.15.16:0/2053160962 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdef405cc10 con 0x7fdf040c5ba0 2024-04-25T10:05:22.922 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2024-04-25T10:05:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7f1455b1c640 1 -- 172.21.15.16:0/2009033817 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f143406b4d0 msgr2=0x7f143406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7f1455b1c640 1 --2- 172.21.15.16:0/2009033817 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f143406b4d0 0x7f143406d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f143c001570 tx=0x7f143c007930 comp rx=0 tx=0).stop 2024-04-25T10:05:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7f1455b1c640 1 -- 172.21.15.16:0/2009033817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14480e30b0 msgr2=0x7f14480e16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7f1455b1c640 1 --2- 172.21.15.16:0/2009033817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14480e30b0 0x7f14480e16f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1444009720 tx=0x7f14440046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7fdf09c6b640 1 -- 172.21.15.16:0/2053160962 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdef006b4d0 msgr2=0x7fdef006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7fdf09c6b640 1 --2- 172.21.15.16:0/2053160962 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdef006b4d0 0x7fdef006d990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdf0409a5b0 tx=0x7fdefc003e60 comp rx=0 tx=0).stop 2024-04-25T10:05:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7fdf09c6b640 1 -- 172.21.15.16:0/2053160962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf040c5ba0 msgr2=0x7fdf041f8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7fdf09c6b640 1 --2- 172.21.15.16:0/2053160962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf040c5ba0 0x7fdf041f8a10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdef400e820 tx=0x7fdef400ece0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7fdf09c6b640 1 -- 172.21.15.16:0/2053160962 shutdown_connections 2024-04-25T10:05:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7f1455b1c640 1 -- 172.21.15.16:0/2009033817 shutdown_connections 2024-04-25T10:05:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7fdf09c6b640 1 --2- 172.21.15.16:0/2053160962 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdef006b4d0 0x7fdef006d990 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7fdf09c6b640 1 --2- 172.21.15.16:0/2053160962 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdf0415f6b0 0x7fdf040c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7fdf09c6b640 1 --2- 172.21.15.16:0/2053160962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf0415bc50 0x7fdf0406d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7fdf09c6b640 1 --2- 172.21.15.16:0/2053160962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf040c5ba0 0x7fdf041f8a10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7fdf09c6b640 1 -- 172.21.15.16:0/2053160962 >> 172.21.15.16:0/2053160962 conn(0x7fdf040b3430 msgr2=0x7fdf04099b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7f1455b1c640 1 --2- 172.21.15.16:0/2009033817 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f143406b4d0 0x7f143406d990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7f1455b1c640 1 --2- 172.21.15.16:0/2009033817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14480e30b0 0x7f14480e16f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7f1455b1c640 1 --2- 172.21.15.16:0/2009033817 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144803e830 0x7f14480d8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7f1455b1c640 1 --2- 172.21.15.16:0/2009033817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14480078d0 0x7f14480d67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7f1455b1c640 1 -- 172.21.15.16:0/2009033817 >> 172.21.15.16:0/2009033817 conn(0x7f14480559d0 msgr2=0x7f14480389b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7f1455b1c640 1 -- 172.21.15.16:0/2009033817 shutdown_connections 2024-04-25T10:05:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7f1455b1c640 1 -- 172.21.15.16:0/2009033817 wait complete. 2024-04-25T10:05:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7fdf09c6b640 1 -- 172.21.15.16:0/2053160962 shutdown_connections 2024-04-25T10:05:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:22.915+0000 7fdf09c6b640 1 -- 172.21.15.16:0/2053160962 wait complete. 2024-04-25T10:05:22.927 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.2 2024-04-25T10:05:22.927 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:22.934 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-04-25T10:05:22.934 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.6 2024-04-25T10:05:22.934 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:22.934 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:22.935 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:05:22.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.070+0000 7fbcd3ccd640 1 Processor -- start 2024-04-25T10:05:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.070+0000 7fbcd3ccd640 1 -- start start 2024-04-25T10:05:23.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.071+0000 7fbcd3ccd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc15dc30 0x7fbccc06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.071+0000 7fbcd3ccd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbccc06c1f0 con 0x7fbccc0b1b30 2024-04-25T10:05:23.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.071+0000 7fbcd3ccd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbccc0cfb40 con 0x7fbccc06cc00 2024-04-25T10:05:23.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.071+0000 7fbcd3ccd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0b2060 con 0x7fbccc15dc30 2024-04-25T10:05:23.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.071+0000 7fbcd226a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbccc0b1b30 0x7fbccc0d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35174/0 (socket says 172.21.15.16:35174) 2024-04-25T10:05:23.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.071+0000 7fbcd226a640 1 -- 172.21.15.16:0/2870553107 learned_addr learned my addr 172.21.15.16:0/2870553107 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:23.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.072+0000 7fbcd1a69640 1 --2- 172.21.15.16:0/2870553107 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc15dc30 0x7fbccc06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.072+0000 7fbcd0a67640 1 -- 172.21.15.16:0/2870553107 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831349029 0 0) 0x7fbcbc001490 con 0x7fbccc0b1b30 2024-04-25T10:05:23.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.072+0000 7fbcd0a67640 1 -- 172.21.15.16:0/2870553107 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcb4003520 con 0x7fbccc0b1b30 2024-04-25T10:05:23.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.072+0000 7fbcd0a67640 1 -- 172.21.15.16:0/2870553107 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313356081 0 0) 0x7fbccc0cfb40 con 0x7fbccc06cc00 2024-04-25T10:05:23.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.072+0000 7fbcd0a67640 1 -- 172.21.15.16:0/2870553107 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcbc001490 con 0x7fbccc06cc00 2024-04-25T10:05:23.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.072+0000 7fbcd1a69640 1 -- 172.21.15.16:0/2870553107 >> v1:172.21.15.186:6789/0 conn(0x7fbccc06cc00 legacy=0x7fbccc0b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.072+0000 7fbcd0a67640 1 -- 172.21.15.16:0/2870553107 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1671382274 0 0) 0x7fbcb4003520 con 0x7fbccc0b1b30 2024-04-25T10:05:23.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.072+0000 7fbcd1a69640 1 -- 172.21.15.16:0/2870553107 >> v1:172.21.15.16:6789/0 conn(0x7fbccc0b1b30 legacy=0x7fbccc0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.072+0000 7fbcd1a69640 1 -- 172.21.15.16:0/2870553107 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc0b2340 con 0x7fbccc15dc30 2024-04-25T10:05:23.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.073+0000 7fbcd1a69640 1 --2- 172.21.15.16:0/2870553107 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc15dc30 0x7fbccc06c6c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0cfd10 tx=0x7fbcc402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66cb194e24a9ec3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.073+0000 7fbcd0a67640 1 -- 172.21.15.16:0/2870553107 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4037070 con 0x7fbccc15dc30 2024-04-25T10:05:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.073+0000 7fbcd0a67640 1 -- 172.21.15.16:0/2870553107 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc402ac10 con 0x7fbccc15dc30 2024-04-25T10:05:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.073+0000 7fbcd0a67640 1 -- 172.21.15.16:0/2870553107 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc001bc0 con 0x7fbccc15dc30 2024-04-25T10:05:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.073+0000 7fbcd3ccd640 1 -- 172.21.15.16:0/2870553107 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc15dc30 msgr2=0x7fbccc06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.073+0000 7fbcd3ccd640 1 --2- 172.21.15.16:0/2870553107 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc15dc30 0x7fbccc06c6c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0cfd10 tx=0x7fbcc402a180 comp rx=0 tx=0).stop 2024-04-25T10:05:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.073+0000 7fbcd3ccd640 1 -- 172.21.15.16:0/2870553107 shutdown_connections 2024-04-25T10:05:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.073+0000 7fbcd3ccd640 1 --2- 172.21.15.16:0/2870553107 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc15dc30 0x7fbccc06c6c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.073+0000 7fbcd3ccd640 1 -- 172.21.15.16:0/2870553107 >> 172.21.15.16:0/2870553107 conn(0x7fbccc0d1900 msgr2=0x7fbccc0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.074+0000 7fbcd3ccd640 1 -- 172.21.15.16:0/2870553107 shutdown_connections 2024-04-25T10:05:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.074+0000 7fbcd3ccd640 1 -- 172.21.15.16:0/2870553107 wait complete. 2024-04-25T10:05:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.075+0000 7fbcd3ccd640 1 Processor -- start 2024-04-25T10:05:23.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.075+0000 7fbcd3ccd640 1 -- start start 2024-04-25T10:05:23.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.075+0000 7fbcd3ccd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc06cc00 0x7fbccc160a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.075+0000 7fbcd3ccd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc0b1b30 0x7fbccc157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.075+0000 7fbcd3ccd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbccc15dc30 0x7fbccc0cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.075+0000 7fbcd3ccd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc15e010 con 0x7fbccc0b1b30 2024-04-25T10:05:23.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.076+0000 7fbcd3ccd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0cfde0 con 0x7fbccc15dc30 2024-04-25T10:05:23.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.076+0000 7fbcd3ccd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0d0a10 con 0x7fbccc06cc00 2024-04-25T10:05:23.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.076+0000 7fbcd1268640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc0b1b30 0x7fbccc157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.076+0000 7fbcd1268640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc0b1b30 0x7fbccc157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56172/0 (socket says 172.21.15.16:56172) 2024-04-25T10:05:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.076+0000 7fbcd1268640 1 -- 172.21.15.16:0/7411913 learned_addr learned my addr 172.21.15.16:0/7411913 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.076+0000 7fbcd1a69640 1 --2- 172.21.15.16:0/7411913 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc06cc00 0x7fbccc160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.076+0000 7fbcd1268640 1 -- 172.21.15.16:0/7411913 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc06cc00 msgr2=0x7fbccc160a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.076+0000 7fbcd1268640 1 --2- 172.21.15.16:0/7411913 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc06cc00 0x7fbccc160a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.076+0000 7fbcd1268640 1 -- 172.21.15.16:0/7411913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbccc15dc30 msgr2=0x7fbccc0cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.076+0000 7fbcd1268640 1 --2- 172.21.15.16:0/7411913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbccc15dc30 0x7fbccc0cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.076+0000 7fbcd1268640 1 -- 172.21.15.16:0/7411913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc0c5810 con 0x7fbccc0b1b30 2024-04-25T10:05:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.076+0000 7fbcd1a69640 1 --2- 172.21.15.16:0/7411913 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc06cc00 0x7fbccc160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.076+0000 7fbcd1268640 1 --2- 172.21.15.16:0/7411913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc0b1b30 0x7fbccc157d70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000c4c0 tx=0x7fbcc000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.076+0000 7fbcca7fc640 1 -- 172.21.15.16:0/7411913 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc00052d0 con 0x7fbccc0b1b30 2024-04-25T10:05:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.076+0000 7fbcca7fc640 1 -- 172.21.15.16:0/7411913 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc0005470 con 0x7fbccc0b1b30 2024-04-25T10:05:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.076+0000 7fbcd3ccd640 1 -- 172.21.15.16:0/7411913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbccc0c5af0 con 0x7fbccc0b1b30 2024-04-25T10:05:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.076+0000 7fbcca7fc640 1 -- 172.21.15.16:0/7411913 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0012870 con 0x7fbccc0b1b30 2024-04-25T10:05:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.076+0000 7fbcd3ccd640 1 -- 172.21.15.16:0/7411913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbccc0c60b0 con 0x7fbccc0b1b30 2024-04-25T10:05:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.077+0000 7fbcd3ccd640 1 -- 172.21.15.16:0/7411913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbccc0749b0 con 0x7fbccc0b1b30 2024-04-25T10:05:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.078+0000 7fbcca7fc640 1 -- 172.21.15.16:0/7411913 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbcc0012a10 con 0x7fbccc0b1b30 2024-04-25T10:05:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.078+0000 7fbcca7fc640 1 --2- 172.21.15.16:0/7411913 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbcb806b500 0x7fbcb806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.078+0000 7fbcca7fc640 1 -- 172.21.15.16:0/7411913 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbcc0089f10 con 0x7fbccc0b1b30 2024-04-25T10:05:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.078+0000 7fbcd1a69640 1 --2- 172.21.15.16:0/7411913 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbcb806b500 0x7fbcb806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.079+0000 7fbcd1a69640 1 --2- 172.21.15.16:0/7411913 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbcb806b500 0x7fbcb806d9c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4034000 tx=0x7fbcc402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.082+0000 7fbcca7fc640 1 -- 172.21.15.16:0/7411913 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbcc0058d00 con 0x7fbccc0b1b30 2024-04-25T10:05:23.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.259+0000 7fbcd3ccd640 1 -- 172.21.15.16:0/7411913 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbccc06e330 con 0x7fbcb806b500 2024-04-25T10:05:23.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.265+0000 7fbcca7fc640 1 -- 172.21.15.16:0/7411913 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251833 (secure 0 0 0) 0x7fbccc06e330 con 0x7fbcb806b500 2024-04-25T10:05:23.273 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:23.274 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:05:23.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.268+0000 7fbcd3ccd640 1 -- 172.21.15.16:0/7411913 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbcb806b500 msgr2=0x7fbcb806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.268+0000 7fbcd3ccd640 1 --2- 172.21.15.16:0/7411913 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbcb806b500 0x7fbcb806d9c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4034000 tx=0x7fbcc402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:23.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.268+0000 7fbcd3ccd640 1 -- 172.21.15.16:0/7411913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc0b1b30 msgr2=0x7fbccc157d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.268+0000 7fbcd3ccd640 1 --2- 172.21.15.16:0/7411913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc0b1b30 0x7fbccc157d70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000c4c0 tx=0x7fbcc000c980 comp rx=0 tx=0).stop 2024-04-25T10:05:23.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.268+0000 7fbcd3ccd640 1 -- 172.21.15.16:0/7411913 shutdown_connections 2024-04-25T10:05:23.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.268+0000 7fbcd3ccd640 1 --2- 172.21.15.16:0/7411913 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbcb806b500 0x7fbcb806d9c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.268+0000 7fbcd3ccd640 1 --2- 172.21.15.16:0/7411913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbccc15dc30 0x7fbccc0cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.268+0000 7fbcd3ccd640 1 --2- 172.21.15.16:0/7411913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc0b1b30 0x7fbccc157d70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.268+0000 7fbcd3ccd640 1 --2- 172.21.15.16:0/7411913 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc06cc00 0x7fbccc160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.268+0000 7fbcd3ccd640 1 -- 172.21.15.16:0/7411913 >> 172.21.15.16:0/7411913 conn(0x7fbccc0d1900 msgr2=0x7fbccc0d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.268+0000 7fbcd3ccd640 1 -- 172.21.15.16:0/7411913 shutdown_connections 2024-04-25T10:05:23.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.268+0000 7fbcd3ccd640 1 -- 172.21.15.16:0/7411913 wait complete. 2024-04-25T10:05:23.284 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-04-25T10:05:21.783812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235780,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738940,"statfs":{"total":773094113280,"available":772852674560,"internally_reserved":0,"allocated":3022848,"data_stored":1798068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002578"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.449339+0000","last_change":"2024-04-25T10:05:11.473253+0000","last_active":"2024-04-25T10:05:13.449339+0000","last_peered":"2024-04-25T10:05:13.449339+0000","last_clean":"2024-04-25T10:05:13.449339+0000","last_became_active":"2024-04-25T10:05:11.472940+0000","last_became_peered":"2024-04-25T10:05:11.472940+0000","last_unstale":"2024-04-25T10:05:13.449339+0000","last_undegraded":"2024-04-25T10:05:13.449339+0000","last_fullsized":"2024-04-25T10:05:13.449339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:22.629451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786817+0000","last_change":"2024-04-25T10:05:11.460499+0000","last_active":"2024-04-25T10:05:14.786817+0000","last_peered":"2024-04-25T10:05:14.786817+0000","last_clean":"2024-04-25T10:05:14.786817+0000","last_became_active":"2024-04-25T10:05:11.458776+0000","last_became_peered":"2024-04-25T10:05:11.458776+0000","last_unstale":"2024-04-25T10:05:14.786817+0000","last_undegraded":"2024-04-25T10:05:14.786817+0000","last_fullsized":"2024-04-25T10:05:14.786817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:17.247685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786678+0000","last_change":"2024-04-25T10:05:11.466721+0000","last_active":"2024-04-25T10:05:14.786678+0000","last_peered":"2024-04-25T10:05:14.786678+0000","last_clean":"2024-04-25T10:05:14.786678+0000","last_became_active":"2024-04-25T10:05:11.466430+0000","last_became_peered":"2024-04-25T10:05:11.466430+0000","last_unstale":"2024-04-25T10:05:14.786678+0000","last_undegraded":"2024-04-25T10:05:14.786678+0000","last_fullsized":"2024-04-25T10:05:14.786678+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:34.227544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455508+0000","last_change":"2024-04-25T10:05:11.461498+0000","last_active":"2024-04-25T10:05:12.455508+0000","last_peered":"2024-04-25T10:05:12.455508+0000","last_clean":"2024-04-25T10:05:12.455508+0000","last_became_active":"2024-04-25T10:05:11.461241+0000","last_became_peered":"2024-04-25T10:05:11.461241+0000","last_unstale":"2024-04-25T10:05:12.455508+0000","last_undegraded":"2024-04-25T10:05:12.455508+0000","last_fullsized":"2024-04-25T10:05:12.455508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.448822+0000","last_change":"2024-04-25T10:05:11.469281+0000","last_active":"2024-04-25T10:05:13.448822+0000","last_peered":"2024-04-25T10:05:13.448822+0000","last_clean":"2024-04-25T10:05:13.448822+0000","last_became_active":"2024-04-25T10:05:11.468809+0000","last_became_peered":"2024-04-25T10:05:11.468809+0000","last_unstale":"2024-04-25T10:05:13.448822+0000","last_undegraded":"2024-04-25T10:05:13.448822+0000","last_fullsized":"2024-04-25T10:05:13.448822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:45.980553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455529+0000","last_change":"2024-04-25T10:05:11.466531+0000","last_active":"2024-04-25T10:05:12.455529+0000","last_peered":"2024-04-25T10:05:12.455529+0000","last_clean":"2024-04-25T10:05:12.455529+0000","last_became_active":"2024-04-25T10:05:11.466126+0000","last_became_peered":"2024-04-25T10:05:11.466126+0000","last_unstale":"2024-04-25T10:05:12.455529+0000","last_undegraded":"2024-04-25T10:05:12.455529+0000","last_fullsized":"2024-04-25T10:05:12.455529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.448621+0000","last_change":"2024-04-25T10:05:11.466336+0000","last_active":"2024-04-25T10:05:13.448621+0000","last_peered":"2024-04-25T10:05:13.448621+0000","last_clean":"2024-04-25T10:05:13.448621+0000","last_became_active":"2024-04-25T10:05:11.466032+0000","last_became_peered":"2024-04-25T10:05:11.466032+0000","last_unstale":"2024-04-25T10:05:13.448621+0000","last_undegraded":"2024-04-25T10:05:13.448621+0000","last_fullsized":"2024-04-25T10:05:13.448621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.228600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786457+0000","last_change":"2024-04-25T10:05:11.461892+0000","last_active":"2024-04-25T10:05:14.786457+0000","last_peered":"2024-04-25T10:05:14.786457+0000","last_clean":"2024-04-25T10:05:14.786457+0000","last_became_active":"2024-04-25T10:05:11.461264+0000","last_became_peered":"2024-04-25T10:05:11.461264+0000","last_unstale":"2024-04-25T10:05:14.786457+0000","last_undegraded":"2024-04-25T10:05:14.786457+0000","last_fullsized":"2024-04-25T10:05:14.786457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:58.263921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456654+0000","last_change":"2024-04-25T10:05:11.461997+0000","last_active":"2024-04-25T10:05:12.456654+0000","last_peered":"2024-04-25T10:05:12.456654+0000","last_clean":"2024-04-25T10:05:12.456654+0000","last_became_active":"2024-04-25T10:05:11.461658+0000","last_became_peered":"2024-04-25T10:05:11.461658+0000","last_unstale":"2024-04-25T10:05:12.456654+0000","last_undegraded":"2024-04-25T10:05:12.456654+0000","last_fullsized":"2024-04-25T10:05:12.456654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.449240+0000","last_change":"2024-04-25T10:05:11.461631+0000","last_active":"2024-04-25T10:05:13.449240+0000","last_peered":"2024-04-25T10:05:13.449240+0000","last_clean":"2024-04-25T10:05:13.449240+0000","last_became_active":"2024-04-25T10:05:11.461327+0000","last_became_peered":"2024-04-25T10:05:11.461327+0000","last_unstale":"2024-04-25T10:05:13.449240+0000","last_undegraded":"2024-04-25T10:05:13.449240+0000","last_fullsized":"2024-04-25T10:05:13.449240+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:48.544393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.448945+0000","last_change":"2024-04-25T10:05:11.473044+0000","last_active":"2024-04-25T10:05:13.448945+0000","last_peered":"2024-04-25T10:05:13.448945+0000","last_clean":"2024-04-25T10:05:13.448945+0000","last_became_active":"2024-04-25T10:05:11.472747+0000","last_became_peered":"2024-04-25T10:05:11.472747+0000","last_unstale":"2024-04-25T10:05:13.448945+0000","last_undegraded":"2024-04-25T10:05:13.448945+0000","last_fullsized":"2024-04-25T10:05:13.448945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:35.076762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.448767+0000","last_change":"2024-04-25T10:05:11.470219+0000","last_active":"2024-04-25T10:05:13.448767+0000","last_peered":"2024-04-25T10:05:13.448767+0000","last_clean":"2024-04-25T10:05:13.448767+0000","last_became_active":"2024-04-25T10:05:11.469066+0000","last_became_peered":"2024-04-25T10:05:11.469066+0000","last_unstale":"2024-04-25T10:05:13.448767+0000","last_undegraded":"2024-04-25T10:05:13.448767+0000","last_fullsized":"2024-04-25T10:05:13.448767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:41.766168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.626322+0000","last_change":"2024-04-25T10:05:11.463262+0000","last_active":"2024-04-25T10:05:13.626322+0000","last_peered":"2024-04-25T10:05:13.626322+0000","last_clean":"2024-04-25T10:05:13.626322+0000","last_became_active":"2024-04-25T10:05:11.462970+0000","last_became_peered":"2024-04-25T10:05:11.462970+0000","last_unstale":"2024-04-25T10:05:13.626322+0000","last_undegraded":"2024-04-25T10:05:13.626322+0000","last_fullsized":"2024-04-25T10:05:13.626322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:30.789969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.626079+0000","last_change":"2024-04-25T10:05:11.458310+0000","last_active":"2024-04-25T10:05:13.626079+0000","last_peered":"2024-04-25T10:05:13.626079+0000","last_clean":"2024-04-25T10:05:13.626079+0000","last_became_active":"2024-04-25T10:05:11.457672+0000","last_became_peered":"2024-04-25T10:05:11.457672+0000","last_unstale":"2024-04-25T10:05:13.626079+0000","last_undegraded":"2024-04-25T10:05:13.626079+0000","last_fullsized":"2024-04-25T10:05:13.626079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:55.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786423+0000","last_change":"2024-04-25T10:05:11.461506+0000","last_active":"2024-04-25T10:05:14.786423+0000","last_peered":"2024-04-25T10:05:14.786423+0000","last_clean":"2024-04-25T10:05:14.786423+0000","last_became_active":"2024-04-25T10:05:11.460963+0000","last_became_peered":"2024-04-25T10:05:11.460963+0000","last_unstale":"2024-04-25T10:05:14.786423+0000","last_undegraded":"2024-04-25T10:05:14.786423+0000","last_fullsized":"2024-04-25T10:05:14.786423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:45.096476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750382+0000","last_change":"2024-04-25T10:05:11.470743+0000","last_active":"2024-04-25T10:05:13.750382+0000","last_peered":"2024-04-25T10:05:13.750382+0000","last_clean":"2024-04-25T10:05:13.750382+0000","last_became_active":"2024-04-25T10:05:11.470447+0000","last_became_peered":"2024-04-25T10:05:11.470447+0000","last_unstale":"2024-04-25T10:05:13.750382+0000","last_undegraded":"2024-04-25T10:05:13.750382+0000","last_fullsized":"2024-04-25T10:05:13.750382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:22.629451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.751002+0000","last_change":"2024-04-25T10:05:11.472530+0000","last_active":"2024-04-25T10:05:13.751002+0000","last_peered":"2024-04-25T10:05:13.751002+0000","last_clean":"2024-04-25T10:05:13.751002+0000","last_became_active":"2024-04-25T10:05:11.472164+0000","last_became_peered":"2024-04-25T10:05:11.472164+0000","last_unstale":"2024-04-25T10:05:13.751002+0000","last_undegraded":"2024-04-25T10:05:13.751002+0000","last_fullsized":"2024-04-25T10:05:13.751002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:33.547465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.650020+0000","last_change":"2024-04-25T10:05:11.464186+0000","last_active":"2024-04-25T10:05:13.650020+0000","last_peered":"2024-04-25T10:05:13.650020+0000","last_clean":"2024-04-25T10:05:13.650020+0000","last_became_active":"2024-04-25T10:05:11.463925+0000","last_became_peered":"2024-04-25T10:05:11.463925+0000","last_unstale":"2024-04-25T10:05:13.650020+0000","last_undegraded":"2024-04-25T10:05:13.650020+0000","last_fullsized":"2024-04-25T10:05:13.650020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750917+0000","last_change":"2024-04-25T10:05:11.473554+0000","last_active":"2024-04-25T10:05:13.750917+0000","last_peered":"2024-04-25T10:05:13.750917+0000","last_clean":"2024-04-25T10:05:13.750917+0000","last_became_active":"2024-04-25T10:05:11.473271+0000","last_became_peered":"2024-04-25T10:05:11.473271+0000","last_unstale":"2024-04-25T10:05:13.750917+0000","last_undegraded":"2024-04-25T10:05:13.750917+0000","last_fullsized":"2024-04-25T10:05:13.750917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:07.933937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.448919+0000","last_change":"2024-04-25T10:05:11.471477+0000","last_active":"2024-04-25T10:05:13.448919+0000","last_peered":"2024-04-25T10:05:13.448919+0000","last_clean":"2024-04-25T10:05:13.448919+0000","last_became_active":"2024-04-25T10:05:11.471219+0000","last_became_peered":"2024-04-25T10:05:11.471219+0000","last_unstale":"2024-04-25T10:05:13.448919+0000","last_undegraded":"2024-04-25T10:05:13.448919+0000","last_fullsized":"2024-04-25T10:05:13.448919+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:24.616832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456655+0000","last_change":"2024-04-25T10:05:11.464410+0000","last_active":"2024-04-25T10:05:12.456655+0000","last_peered":"2024-04-25T10:05:12.456655+0000","last_clean":"2024-04-25T10:05:12.456655+0000","last_became_active":"2024-04-25T10:05:11.464003+0000","last_became_peered":"2024-04-25T10:05:11.464003+0000","last_unstale":"2024-04-25T10:05:12.456655+0000","last_undegraded":"2024-04-25T10:05:12.456655+0000","last_fullsized":"2024-04-25T10:05:12.456655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:52.014434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456342+0000","last_change":"2024-04-25T10:05:11.462487+0000","last_active":"2024-04-25T10:05:12.456342+0000","last_peered":"2024-04-25T10:05:12.456342+0000","last_clean":"2024-04-25T10:05:12.456342+0000","last_became_active":"2024-04-25T10:05:11.462191+0000","last_became_peered":"2024-04-25T10:05:11.462191+0000","last_unstale":"2024-04-25T10:05:12.456342+0000","last_undegraded":"2024-04-25T10:05:12.456342+0000","last_fullsized":"2024-04-25T10:05:12.456342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786682+0000","last_change":"2024-04-25T10:05:11.450886+0000","last_active":"2024-04-25T10:05:14.786682+0000","last_peered":"2024-04-25T10:05:14.786682+0000","last_clean":"2024-04-25T10:05:14.786682+0000","last_became_active":"2024-04-25T10:05:11.450616+0000","last_became_peered":"2024-04-25T10:05:11.450616+0000","last_unstale":"2024-04-25T10:05:14.786682+0000","last_undegraded":"2024-04-25T10:05:14.786682+0000","last_fullsized":"2024-04-25T10:05:14.786682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:47.607490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750381+0000","last_change":"2024-04-25T10:05:11.467406+0000","last_active":"2024-04-25T10:05:13.750381+0000","last_peered":"2024-04-25T10:05:13.750381+0000","last_clean":"2024-04-25T10:05:13.750381+0000","last_became_active":"2024-04-25T10:05:11.467126+0000","last_became_peered":"2024-04-25T10:05:11.467126+0000","last_unstale":"2024-04-25T10:05:13.750381+0000","last_undegraded":"2024-04-25T10:05:13.750381+0000","last_fullsized":"2024-04-25T10:05:13.750381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:55.943373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786879+0000","last_change":"2024-04-25T10:05:11.453135+0000","last_active":"2024-04-25T10:05:14.786879+0000","last_peered":"2024-04-25T10:05:14.786879+0000","last_clean":"2024-04-25T10:05:14.786879+0000","last_became_active":"2024-04-25T10:05:11.452053+0000","last_became_peered":"2024-04-25T10:05:11.452053+0000","last_unstale":"2024-04-25T10:05:14.786879+0000","last_undegraded":"2024-04-25T10:05:14.786879+0000","last_fullsized":"2024-04-25T10:05:14.786879+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:03.147551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455642+0000","last_change":"2024-04-25T10:05:11.469452+0000","last_active":"2024-04-25T10:05:12.455642+0000","last_peered":"2024-04-25T10:05:12.455642+0000","last_clean":"2024-04-25T10:05:12.455642+0000","last_became_active":"2024-04-25T10:05:11.465896+0000","last_became_peered":"2024-04-25T10:05:11.465896+0000","last_unstale":"2024-04-25T10:05:12.455642+0000","last_undegraded":"2024-04-25T10:05:12.455642+0000","last_fullsized":"2024-04-25T10:05:12.455642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:14.506306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.626351+0000","last_change":"2024-04-25T10:05:11.457595+0000","last_active":"2024-04-25T10:05:13.626351+0000","last_peered":"2024-04-25T10:05:13.626351+0000","last_clean":"2024-04-25T10:05:13.626351+0000","last_became_active":"2024-04-25T10:05:11.457320+0000","last_became_peered":"2024-04-25T10:05:11.457320+0000","last_unstale":"2024-04-25T10:05:13.626351+0000","last_undegraded":"2024-04-25T10:05:13.626351+0000","last_fullsized":"2024-04-25T10:05:13.626351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.626269+0000","last_change":"2024-04-25T10:05:11.469113+0000","last_active":"2024-04-25T10:05:13.626269+0000","last_peered":"2024-04-25T10:05:13.626269+0000","last_clean":"2024-04-25T10:05:13.626269+0000","last_became_active":"2024-04-25T10:05:11.468759+0000","last_became_peered":"2024-04-25T10:05:11.468759+0000","last_unstale":"2024-04-25T10:05:13.626269+0000","last_undegraded":"2024-04-25T10:05:13.626269+0000","last_fullsized":"2024-04-25T10:05:13.626269+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:12.926321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786630+0000","last_change":"2024-04-25T10:05:11.457486+0000","last_active":"2024-04-25T10:05:14.786630+0000","last_peered":"2024-04-25T10:05:14.786630+0000","last_clean":"2024-04-25T10:05:14.786630+0000","last_became_active":"2024-04-25T10:05:11.454855+0000","last_became_peered":"2024-04-25T10:05:11.454855+0000","last_unstale":"2024-04-25T10:05:14.786630+0000","last_undegraded":"2024-04-25T10:05:14.786630+0000","last_fullsized":"2024-04-25T10:05:14.786630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:45.863831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750531+0000","last_change":"2024-04-25T10:05:11.470055+0000","last_active":"2024-04-25T10:05:13.750531+0000","last_peered":"2024-04-25T10:05:13.750531+0000","last_clean":"2024-04-25T10:05:13.750531+0000","last_became_active":"2024-04-25T10:05:11.468974+0000","last_became_peered":"2024-04-25T10:05:11.468974+0000","last_unstale":"2024-04-25T10:05:13.750531+0000","last_undegraded":"2024-04-25T10:05:13.750531+0000","last_fullsized":"2024-04-25T10:05:13.750531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:20.369758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.649698+0000","last_change":"2024-04-25T10:05:11.462234+0000","last_active":"2024-04-25T10:05:13.649698+0000","last_peered":"2024-04-25T10:05:13.649698+0000","last_clean":"2024-04-25T10:05:13.649698+0000","last_became_active":"2024-04-25T10:05:11.461975+0000","last_became_peered":"2024-04-25T10:05:11.461975+0000","last_unstale":"2024-04-25T10:05:13.649698+0000","last_undegraded":"2024-04-25T10:05:13.649698+0000","last_fullsized":"2024-04-25T10:05:13.649698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455396+0000","last_change":"2024-04-25T10:05:11.463406+0000","last_active":"2024-04-25T10:05:12.455396+0000","last_peered":"2024-04-25T10:05:12.455396+0000","last_clean":"2024-04-25T10:05:12.455396+0000","last_became_active":"2024-04-25T10:05:11.463139+0000","last_became_peered":"2024-04-25T10:05:11.463139+0000","last_unstale":"2024-04-25T10:05:12.455396+0000","last_undegraded":"2024-04-25T10:05:12.455396+0000","last_fullsized":"2024-04-25T10:05:12.455396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456384+0000","last_change":"2024-04-25T10:05:11.453168+0000","last_active":"2024-04-25T10:05:12.456384+0000","last_peered":"2024-04-25T10:05:12.456384+0000","last_clean":"2024-04-25T10:05:12.456384+0000","last_became_active":"2024-04-25T10:05:11.452238+0000","last_became_peered":"2024-04-25T10:05:11.452238+0000","last_unstale":"2024-04-25T10:05:12.456384+0000","last_undegraded":"2024-04-25T10:05:12.456384+0000","last_fullsized":"2024-04-25T10:05:12.456384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786706+0000","last_change":"2024-04-25T10:05:10.455995+0000","last_active":"2024-04-25T10:05:14.786706+0000","last_peered":"2024-04-25T10:05:14.786706+0000","last_clean":"2024-04-25T10:05:14.786706+0000","last_became_active":"2024-04-25T10:05:10.455718+0000","last_became_peered":"2024-04-25T10:05:10.455718+0000","last_unstale":"2024-04-25T10:05:14.786706+0000","last_undegraded":"2024-04-25T10:05:14.786706+0000","last_fullsized":"2024-04-25T10:05:14.786706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:47.448482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456962+0000","last_change":"2024-04-25T10:05:11.469760+0000","last_active":"2024-04-25T10:05:12.456962+0000","last_peered":"2024-04-25T10:05:12.456962+0000","last_clean":"2024-04-25T10:05:12.456962+0000","last_became_active":"2024-04-25T10:05:11.469490+0000","last_became_peered":"2024-04-25T10:05:11.469490+0000","last_unstale":"2024-04-25T10:05:12.456962+0000","last_undegraded":"2024-04-25T10:05:12.456962+0000","last_fullsized":"2024-04-25T10:05:12.456962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.228600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750654+0000","last_change":"2024-04-25T10:05:10.461940+0000","last_active":"2024-04-25T10:05:13.750654+0000","last_peered":"2024-04-25T10:05:13.750654+0000","last_clean":"2024-04-25T10:05:13.750654+0000","last_became_active":"2024-04-25T10:05:10.461633+0000","last_became_peered":"2024-04-25T10:05:10.461633+0000","last_unstale":"2024-04-25T10:05:13.750654+0000","last_undegraded":"2024-04-25T10:05:13.750654+0000","last_fullsized":"2024-04-25T10:05:13.750654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.625954+0000","last_change":"2024-04-25T10:05:10.460823+0000","last_active":"2024-04-25T10:05:13.625954+0000","last_peered":"2024-04-25T10:05:13.625954+0000","last_clean":"2024-04-25T10:05:13.625954+0000","last_became_active":"2024-04-25T10:05:10.460480+0000","last_became_peered":"2024-04-25T10:05:10.460480+0000","last_unstale":"2024-04-25T10:05:13.625954+0000","last_undegraded":"2024-04-25T10:05:13.625954+0000","last_fullsized":"2024-04-25T10:05:13.625954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:31.375084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786743+0000","last_change":"2024-04-25T10:05:11.460164+0000","last_active":"2024-04-25T10:05:14.786743+0000","last_peered":"2024-04-25T10:05:14.786743+0000","last_clean":"2024-04-25T10:05:14.786743+0000","last_became_active":"2024-04-25T10:05:11.459615+0000","last_became_peered":"2024-04-25T10:05:11.459615+0000","last_unstale":"2024-04-25T10:05:14.786743+0000","last_undegraded":"2024-04-25T10:05:14.786743+0000","last_fullsized":"2024-04-25T10:05:14.786743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:00.275933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750579+0000","last_change":"2024-04-25T10:05:10.461713+0000","last_active":"2024-04-25T10:05:13.750579+0000","last_peered":"2024-04-25T10:05:13.750579+0000","last_clean":"2024-04-25T10:05:13.750579+0000","last_became_active":"2024-04-25T10:05:10.461379+0000","last_became_peered":"2024-04-25T10:05:10.461379+0000","last_unstale":"2024-04-25T10:05:13.750579+0000","last_undegraded":"2024-04-25T10:05:13.750579+0000","last_fullsized":"2024-04-25T10:05:13.750579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456510+0000","last_change":"2024-04-25T10:05:11.466408+0000","last_active":"2024-04-25T10:05:12.456510+0000","last_peered":"2024-04-25T10:05:12.456510+0000","last_clean":"2024-04-25T10:05:12.456510+0000","last_became_active":"2024-04-25T10:05:11.465611+0000","last_became_peered":"2024-04-25T10:05:11.465611+0000","last_unstale":"2024-04-25T10:05:12.456510+0000","last_undegraded":"2024-04-25T10:05:12.456510+0000","last_fullsized":"2024-04-25T10:05:12.456510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456657+0000","last_change":"2024-04-25T10:05:10.449054+0000","last_active":"2024-04-25T10:05:12.456657+0000","last_peered":"2024-04-25T10:05:12.456657+0000","last_clean":"2024-04-25T10:05:12.456657+0000","last_became_active":"2024-04-25T10:05:10.448743+0000","last_became_peered":"2024-04-25T10:05:10.448743+0000","last_unstale":"2024-04-25T10:05:12.456657+0000","last_undegraded":"2024-04-25T10:05:12.456657+0000","last_fullsized":"2024-04-25T10:05:12.456657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:22.897294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455686+0000","last_change":"2024-04-25T10:05:11.453268+0000","last_active":"2024-04-25T10:05:12.455686+0000","last_peered":"2024-04-25T10:05:12.455686+0000","last_clean":"2024-04-25T10:05:12.455686+0000","last_became_active":"2024-04-25T10:05:11.452971+0000","last_became_peered":"2024-04-25T10:05:11.452971+0000","last_unstale":"2024-04-25T10:05:12.455686+0000","last_undegraded":"2024-04-25T10:05:12.455686+0000","last_fullsized":"2024-04-25T10:05:12.455686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:56.142030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.449266+0000","last_change":"2024-04-25T10:05:10.452247+0000","last_active":"2024-04-25T10:05:13.449266+0000","last_peered":"2024-04-25T10:05:13.449266+0000","last_clean":"2024-04-25T10:05:13.449266+0000","last_became_active":"2024-04-25T10:05:10.451925+0000","last_became_peered":"2024-04-25T10:05:10.451925+0000","last_unstale":"2024-04-25T10:05:13.449266+0000","last_undegraded":"2024-04-25T10:05:13.449266+0000","last_fullsized":"2024-04-25T10:05:13.449266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:53.640804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786996+0000","last_change":"2024-04-25T10:05:11.467490+0000","last_active":"2024-04-25T10:05:14.786996+0000","last_peered":"2024-04-25T10:05:14.786996+0000","last_clean":"2024-04-25T10:05:14.786996+0000","last_became_active":"2024-04-25T10:05:11.467217+0000","last_became_peered":"2024-04-25T10:05:11.467217+0000","last_unstale":"2024-04-25T10:05:14.786996+0000","last_undegraded":"2024-04-25T10:05:14.786996+0000","last_fullsized":"2024-04-25T10:05:14.786996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:26.821659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.649772+0000","last_change":"2024-04-25T10:05:10.459898+0000","last_active":"2024-04-25T10:05:13.649772+0000","last_peered":"2024-04-25T10:05:13.649772+0000","last_clean":"2024-04-25T10:05:13.649772+0000","last_became_active":"2024-04-25T10:05:10.459250+0000","last_became_peered":"2024-04-25T10:05:10.459250+0000","last_unstale":"2024-04-25T10:05:13.649772+0000","last_undegraded":"2024-04-25T10:05:13.649772+0000","last_fullsized":"2024-04-25T10:05:13.649772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:41.255007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750541+0000","last_change":"2024-04-25T10:05:11.471498+0000","last_active":"2024-04-25T10:05:13.750541+0000","last_peered":"2024-04-25T10:05:13.750541+0000","last_clean":"2024-04-25T10:05:13.750541+0000","last_became_active":"2024-04-25T10:05:11.471193+0000","last_became_peered":"2024-04-25T10:05:11.471193+0000","last_unstale":"2024-04-25T10:05:13.750541+0000","last_undegraded":"2024-04-25T10:05:13.750541+0000","last_fullsized":"2024-04-25T10:05:13.750541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:53.026414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455560+0000","last_change":"2024-04-25T10:05:10.450534+0000","last_active":"2024-04-25T10:05:12.455560+0000","last_peered":"2024-04-25T10:05:12.455560+0000","last_clean":"2024-04-25T10:05:12.455560+0000","last_became_active":"2024-04-25T10:05:10.450251+0000","last_became_peered":"2024-04-25T10:05:10.450251+0000","last_unstale":"2024-04-25T10:05:12.455560+0000","last_undegraded":"2024-04-25T10:05:12.455560+0000","last_fullsized":"2024-04-25T10:05:12.455560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:22.897294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.449211+0000","last_change":"2024-04-25T10:05:11.463480+0000","last_active":"2024-04-25T10:05:13.449211+0000","last_peered":"2024-04-25T10:05:13.449211+0000","last_clean":"2024-04-25T10:05:13.449211+0000","last_became_active":"2024-04-25T10:05:11.460995+0000","last_became_peered":"2024-04-25T10:05:11.460995+0000","last_unstale":"2024-04-25T10:05:13.449211+0000","last_undegraded":"2024-04-25T10:05:13.449211+0000","last_fullsized":"2024-04-25T10:05:13.449211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:55.943373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.649937+0000","last_change":"2024-04-25T10:05:10.462323+0000","last_active":"2024-04-25T10:05:13.649937+0000","last_peered":"2024-04-25T10:05:13.649937+0000","last_clean":"2024-04-25T10:05:13.649937+0000","last_became_active":"2024-04-25T10:05:10.462118+0000","last_became_peered":"2024-04-25T10:05:10.462118+0000","last_unstale":"2024-04-25T10:05:13.649937+0000","last_undegraded":"2024-04-25T10:05:13.649937+0000","last_fullsized":"2024-04-25T10:05:13.649937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:39.826605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.626366+0000","last_change":"2024-04-25T10:05:11.467396+0000","last_active":"2024-04-25T10:05:13.626366+0000","last_peered":"2024-04-25T10:05:13.626366+0000","last_clean":"2024-04-25T10:05:13.626366+0000","last_became_active":"2024-04-25T10:05:11.467112+0000","last_became_peered":"2024-04-25T10:05:11.467112+0000","last_unstale":"2024-04-25T10:05:13.626366+0000","last_undegraded":"2024-04-25T10:05:13.626366+0000","last_fullsized":"2024-04-25T10:05:13.626366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750782+0000","last_change":"2024-04-25T10:05:10.458429+0000","last_active":"2024-04-25T10:05:13.750782+0000","last_peered":"2024-04-25T10:05:13.750782+0000","last_clean":"2024-04-25T10:05:13.750782+0000","last_became_active":"2024-04-25T10:05:10.457958+0000","last_became_peered":"2024-04-25T10:05:10.457958+0000","last_unstale":"2024-04-25T10:05:13.750782+0000","last_undegraded":"2024-04-25T10:05:13.750782+0000","last_fullsized":"2024-04-25T10:05:13.750782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:23.585636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750626+0000","last_change":"2024-04-25T10:05:11.465006+0000","last_active":"2024-04-25T10:05:13.750626+0000","last_peered":"2024-04-25T10:05:13.750626+0000","last_clean":"2024-04-25T10:05:13.750626+0000","last_became_active":"2024-04-25T10:05:11.464719+0000","last_became_peered":"2024-04-25T10:05:11.464719+0000","last_unstale":"2024-04-25T10:05:13.750626+0000","last_undegraded":"2024-04-25T10:05:13.750626+0000","last_fullsized":"2024-04-25T10:05:13.750626+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:12.496324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.448634+0000","last_change":"2024-04-25T10:05:10.456938+0000","last_active":"2024-04-25T10:05:13.448634+0000","last_peered":"2024-04-25T10:05:13.448634+0000","last_clean":"2024-04-25T10:05:13.448634+0000","last_became_active":"2024-04-25T10:05:10.456644+0000","last_became_peered":"2024-04-25T10:05:10.456644+0000","last_unstale":"2024-04-25T10:05:13.448634+0000","last_undegraded":"2024-04-25T10:05:13.448634+0000","last_fullsized":"2024-04-25T10:05:13.448634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456188+0000","last_change":"2024-04-25T10:05:11.456178+0000","last_active":"2024-04-25T10:05:12.456188+0000","last_peered":"2024-04-25T10:05:12.456188+0000","last_clean":"2024-04-25T10:05:12.456188+0000","last_became_active":"2024-04-25T10:05:11.455879+0000","last_became_peered":"2024-04-25T10:05:11.455879+0000","last_unstale":"2024-04-25T10:05:12.456188+0000","last_undegraded":"2024-04-25T10:05:12.456188+0000","last_fullsized":"2024-04-25T10:05:12.456188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:55.300183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456275+0000","last_change":"2024-04-25T10:05:10.455684+0000","last_active":"2024-04-25T10:05:12.456275+0000","last_peered":"2024-04-25T10:05:12.456275+0000","last_clean":"2024-04-25T10:05:12.456275+0000","last_became_active":"2024-04-25T10:05:10.455342+0000","last_became_peered":"2024-04-25T10:05:10.455342+0000","last_unstale":"2024-04-25T10:05:12.456275+0000","last_undegraded":"2024-04-25T10:05:12.456275+0000","last_fullsized":"2024-04-25T10:05:12.456275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:01.556228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750396+0000","last_change":"2024-04-25T10:05:11.466622+0000","last_active":"2024-04-25T10:05:13.750396+0000","last_peered":"2024-04-25T10:05:13.750396+0000","last_clean":"2024-04-25T10:05:13.750396+0000","last_became_active":"2024-04-25T10:05:11.466346+0000","last_became_peered":"2024-04-25T10:05:11.466346+0000","last_unstale":"2024-04-25T10:05:13.750396+0000","last_undegraded":"2024-04-25T10:05:13.750396+0000","last_fullsized":"2024-04-25T10:05:13.750396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:48.544393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.449417+0000","last_change":"2024-04-25T10:05:10.457614+0000","last_active":"2024-04-25T10:05:13.449417+0000","last_peered":"2024-04-25T10:05:13.449417+0000","last_clean":"2024-04-25T10:05:13.449417+0000","last_became_active":"2024-04-25T10:05:10.457332+0000","last_became_peered":"2024-04-25T10:05:10.457332+0000","last_unstale":"2024-04-25T10:05:13.449417+0000","last_undegraded":"2024-04-25T10:05:13.449417+0000","last_fullsized":"2024-04-25T10:05:13.449417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:01.556228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455780+0000","last_change":"2024-04-25T10:05:11.460589+0000","last_active":"2024-04-25T10:05:12.455780+0000","last_peered":"2024-04-25T10:05:12.455780+0000","last_clean":"2024-04-25T10:05:12.455780+0000","last_became_active":"2024-04-25T10:05:11.459433+0000","last_became_peered":"2024-04-25T10:05:11.459433+0000","last_unstale":"2024-04-25T10:05:12.455780+0000","last_undegraded":"2024-04-25T10:05:12.455780+0000","last_fullsized":"2024-04-25T10:05:12.455780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:55.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456910+0000","last_change":"2024-04-25T10:05:10.458692+0000","last_active":"2024-04-25T10:05:12.456910+0000","last_peered":"2024-04-25T10:05:12.456910+0000","last_clean":"2024-04-25T10:05:12.456910+0000","last_became_active":"2024-04-25T10:05:10.458090+0000","last_became_peered":"2024-04-25T10:05:10.458090+0000","last_unstale":"2024-04-25T10:05:12.456910+0000","last_undegraded":"2024-04-25T10:05:12.456910+0000","last_fullsized":"2024-04-25T10:05:12.456910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:23.585636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.449379+0000","last_change":"2024-04-25T10:05:11.464532+0000","last_active":"2024-04-25T10:05:13.449379+0000","last_peered":"2024-04-25T10:05:13.449379+0000","last_clean":"2024-04-25T10:05:13.449379+0000","last_became_active":"2024-04-25T10:05:11.464186+0000","last_became_peered":"2024-04-25T10:05:11.464186+0000","last_unstale":"2024-04-25T10:05:13.449379+0000","last_undegraded":"2024-04-25T10:05:13.449379+0000","last_fullsized":"2024-04-25T10:05:13.449379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:17.888835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786543+0000","last_change":"2024-04-25T10:05:10.460107+0000","last_active":"2024-04-25T10:05:14.786543+0000","last_peered":"2024-04-25T10:05:14.786543+0000","last_clean":"2024-04-25T10:05:14.786543+0000","last_became_active":"2024-04-25T10:05:10.459754+0000","last_became_peered":"2024-04-25T10:05:10.459754+0000","last_unstale":"2024-04-25T10:05:14.786543+0000","last_undegraded":"2024-04-25T10:05:14.786543+0000","last_fullsized":"2024-04-25T10:05:14.786543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:48.214248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.448967+0000","last_change":"2024-04-25T10:05:11.474519+0000","last_active":"2024-04-25T10:05:13.448967+0000","last_peered":"2024-04-25T10:05:13.448967+0000","last_clean":"2024-04-25T10:05:13.448967+0000","last_became_active":"2024-04-25T10:05:11.472675+0000","last_became_peered":"2024-04-25T10:05:11.472675+0000","last_unstale":"2024-04-25T10:05:13.448967+0000","last_undegraded":"2024-04-25T10:05:13.448967+0000","last_fullsized":"2024-04-25T10:05:13.448967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:48.013538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.626392+0000","last_change":"2024-04-25T10:05:10.453364+0000","last_active":"2024-04-25T10:05:13.626392+0000","last_peered":"2024-04-25T10:05:13.626392+0000","last_clean":"2024-04-25T10:05:13.626392+0000","last_became_active":"2024-04-25T10:05:10.453072+0000","last_became_peered":"2024-04-25T10:05:10.453072+0000","last_unstale":"2024-04-25T10:05:13.626392+0000","last_undegraded":"2024-04-25T10:05:13.626392+0000","last_fullsized":"2024-04-25T10:05:13.626392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:53.640804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750741+0000","last_change":"2024-04-25T10:05:11.467814+0000","last_active":"2024-04-25T10:05:13.750741+0000","last_peered":"2024-04-25T10:05:13.750741+0000","last_clean":"2024-04-25T10:05:13.750741+0000","last_became_active":"2024-04-25T10:05:11.467496+0000","last_became_peered":"2024-04-25T10:05:11.467496+0000","last_unstale":"2024-04-25T10:05:13.750741+0000","last_undegraded":"2024-04-25T10:05:13.750741+0000","last_fullsized":"2024-04-25T10:05:13.750741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:18.768171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456108+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:05:12.456108+0000","last_peered":"2024-04-25T10:05:12.456108+0000","last_clean":"2024-04-25T10:05:12.456108+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:05:12.456108+0000","last_undegraded":"2024-04-25T10:05:12.456108+0000","last_fullsized":"2024-04-25T10:05:12.456108+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750724+0000","last_change":"2024-04-25T10:05:10.458170+0000","last_active":"2024-04-25T10:05:13.750724+0000","last_peered":"2024-04-25T10:05:13.750724+0000","last_clean":"2024-04-25T10:05:13.750724+0000","last_became_active":"2024-04-25T10:05:10.457794+0000","last_became_peered":"2024-04-25T10:05:10.457794+0000","last_unstale":"2024-04-25T10:05:13.750724+0000","last_undegraded":"2024-04-25T10:05:13.750724+0000","last_fullsized":"2024-04-25T10:05:13.750724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:11.920992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456600+0000","last_change":"2024-04-25T10:05:11.462574+0000","last_active":"2024-04-25T10:05:12.456600+0000","last_peered":"2024-04-25T10:05:12.456600+0000","last_clean":"2024-04-25T10:05:12.456600+0000","last_became_active":"2024-04-25T10:05:11.461617+0000","last_became_peered":"2024-04-25T10:05:11.461617+0000","last_unstale":"2024-04-25T10:05:12.456600+0000","last_undegraded":"2024-04-25T10:05:12.456600+0000","last_fullsized":"2024-04-25T10:05:12.456600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750513+0000","last_change":"2024-04-25T10:05:10.464334+0000","last_active":"2024-04-25T10:05:13.750513+0000","last_peered":"2024-04-25T10:05:13.750513+0000","last_clean":"2024-04-25T10:05:13.750513+0000","last_became_active":"2024-04-25T10:05:10.463991+0000","last_became_peered":"2024-04-25T10:05:10.463991+0000","last_unstale":"2024-04-25T10:05:13.750513+0000","last_undegraded":"2024-04-25T10:05:13.750513+0000","last_fullsized":"2024-04-25T10:05:13.750513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.626027+0000","last_change":"2024-04-25T10:05:11.464866+0000","last_active":"2024-04-25T10:05:13.626027+0000","last_peered":"2024-04-25T10:05:13.626027+0000","last_clean":"2024-04-25T10:05:13.626027+0000","last_became_active":"2024-04-25T10:05:11.464504+0000","last_became_peered":"2024-04-25T10:05:11.464504+0000","last_unstale":"2024-04-25T10:05:13.626027+0000","last_undegraded":"2024-04-25T10:05:13.626027+0000","last_fullsized":"2024-04-25T10:05:13.626027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:56.142030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.649649+0000","last_change":"2024-04-25T10:05:10.458220+0000","last_active":"2024-04-25T10:05:13.649649+0000","last_peered":"2024-04-25T10:05:13.649649+0000","last_clean":"2024-04-25T10:05:13.649649+0000","last_became_active":"2024-04-25T10:05:10.457936+0000","last_became_peered":"2024-04-25T10:05:10.457936+0000","last_unstale":"2024-04-25T10:05:13.649649+0000","last_undegraded":"2024-04-25T10:05:13.649649+0000","last_fullsized":"2024-04-25T10:05:13.649649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:31.375084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786598+0000","last_change":"2024-04-25T10:05:11.457665+0000","last_active":"2024-04-25T10:05:14.786598+0000","last_peered":"2024-04-25T10:05:14.786598+0000","last_clean":"2024-04-25T10:05:14.786598+0000","last_became_active":"2024-04-25T10:05:11.455078+0000","last_became_peered":"2024-04-25T10:05:11.455078+0000","last_unstale":"2024-04-25T10:05:14.786598+0000","last_undegraded":"2024-04-25T10:05:14.786598+0000","last_fullsized":"2024-04-25T10:05:14.786598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:21.453409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750827+0000","last_change":"2024-04-25T10:05:10.463388+0000","last_active":"2024-04-25T10:05:13.750827+0000","last_peered":"2024-04-25T10:05:13.750827+0000","last_clean":"2024-04-25T10:05:13.750827+0000","last_became_active":"2024-04-25T10:05:10.463041+0000","last_became_peered":"2024-04-25T10:05:10.463041+0000","last_unstale":"2024-04-25T10:05:13.750827+0000","last_undegraded":"2024-04-25T10:05:13.750827+0000","last_fullsized":"2024-04-25T10:05:13.750827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:13.500977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.448829+0000","last_change":"2024-04-25T10:05:11.462360+0000","last_active":"2024-04-25T10:05:13.448829+0000","last_peered":"2024-04-25T10:05:13.448829+0000","last_clean":"2024-04-25T10:05:13.448829+0000","last_became_active":"2024-04-25T10:05:11.462050+0000","last_became_peered":"2024-04-25T10:05:11.462050+0000","last_unstale":"2024-04-25T10:05:13.448829+0000","last_undegraded":"2024-04-25T10:05:13.448829+0000","last_fullsized":"2024-04-25T10:05:13.448829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:08.525515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786571+0000","last_change":"2024-04-25T10:05:10.456403+0000","last_active":"2024-04-25T10:05:14.786571+0000","last_peered":"2024-04-25T10:05:14.786571+0000","last_clean":"2024-04-25T10:05:14.786571+0000","last_became_active":"2024-04-25T10:05:10.456060+0000","last_became_peered":"2024-04-25T10:05:10.456060+0000","last_unstale":"2024-04-25T10:05:14.786571+0000","last_undegraded":"2024-04-25T10:05:14.786571+0000","last_fullsized":"2024-04-25T10:05:14.786571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:50.107731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455433+0000","last_change":"2024-04-25T10:05:11.464184+0000","last_active":"2024-04-25T10:05:12.455433+0000","last_peered":"2024-04-25T10:05:12.455433+0000","last_clean":"2024-04-25T10:05:12.455433+0000","last_became_active":"2024-04-25T10:05:11.463894+0000","last_became_peered":"2024-04-25T10:05:11.463894+0000","last_unstale":"2024-04-25T10:05:12.455433+0000","last_undegraded":"2024-04-25T10:05:12.455433+0000","last_fullsized":"2024-04-25T10:05:12.455433+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750931+0000","last_change":"2024-04-25T10:05:10.460261+0000","last_active":"2024-04-25T10:05:13.750931+0000","last_peered":"2024-04-25T10:05:13.750931+0000","last_clean":"2024-04-25T10:05:13.750931+0000","last_became_active":"2024-04-25T10:05:10.459949+0000","last_became_peered":"2024-04-25T10:05:10.459949+0000","last_unstale":"2024-04-25T10:05:13.750931+0000","last_undegraded":"2024-04-25T10:05:13.750931+0000","last_fullsized":"2024-04-25T10:05:13.750931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:29.784640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.626340+0000","last_change":"2024-04-25T10:05:11.458748+0000","last_active":"2024-04-25T10:05:13.626340+0000","last_peered":"2024-04-25T10:05:13.626340+0000","last_clean":"2024-04-25T10:05:13.626340+0000","last_became_active":"2024-04-25T10:05:11.458440+0000","last_became_peered":"2024-04-25T10:05:11.458440+0000","last_unstale":"2024-04-25T10:05:13.626340+0000","last_undegraded":"2024-04-25T10:05:13.626340+0000","last_fullsized":"2024-04-25T10:05:13.626340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:54.934726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.793557+0000","last_change":"2024-04-25T10:05:10.455250+0000","last_active":"2024-04-25T10:05:12.793557+0000","last_peered":"2024-04-25T10:05:12.793557+0000","last_clean":"2024-04-25T10:05:12.793557+0000","last_became_active":"2024-04-25T10:05:10.454833+0000","last_became_peered":"2024-04-25T10:05:10.454833+0000","last_unstale":"2024-04-25T10:05:12.793557+0000","last_undegraded":"2024-04-25T10:05:12.793557+0000","last_fullsized":"2024-04-25T10:05:12.793557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.457011+0000","last_change":"2024-04-25T10:05:11.466533+0000","last_active":"2024-04-25T10:05:12.457011+0000","last_peered":"2024-04-25T10:05:12.457011+0000","last_clean":"2024-04-25T10:05:12.457011+0000","last_became_active":"2024-04-25T10:05:11.466256+0000","last_became_peered":"2024-04-25T10:05:11.466256+0000","last_unstale":"2024-04-25T10:05:12.457011+0000","last_undegraded":"2024-04-25T10:05:12.457011+0000","last_fullsized":"2024-04-25T10:05:12.457011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:41.766168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750989+0000","last_change":"2024-04-25T10:05:10.462808+0000","last_active":"2024-04-25T10:05:13.750989+0000","last_peered":"2024-04-25T10:05:13.750989+0000","last_clean":"2024-04-25T10:05:13.750989+0000","last_became_active":"2024-04-25T10:05:10.462421+0000","last_became_peered":"2024-04-25T10:05:10.462421+0000","last_unstale":"2024-04-25T10:05:13.750989+0000","last_undegraded":"2024-04-25T10:05:13.750989+0000","last_fullsized":"2024-04-25T10:05:13.750989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:55.136701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456680+0000","last_change":"2024-04-25T10:05:11.470381+0000","last_active":"2024-04-25T10:05:12.456680+0000","last_peered":"2024-04-25T10:05:12.456680+0000","last_clean":"2024-04-25T10:05:12.456680+0000","last_became_active":"2024-04-25T10:05:11.470031+0000","last_became_peered":"2024-04-25T10:05:11.470031+0000","last_unstale":"2024-04-25T10:05:12.456680+0000","last_undegraded":"2024-04-25T10:05:12.456680+0000","last_fullsized":"2024-04-25T10:05:12.456680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.799101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.448417+0000","last_change":"2024-04-25T10:05:10.461232+0000","last_active":"2024-04-25T10:05:13.448417+0000","last_peered":"2024-04-25T10:05:13.448417+0000","last_clean":"2024-04-25T10:05:13.448417+0000","last_became_active":"2024-04-25T10:05:10.460905+0000","last_became_peered":"2024-04-25T10:05:10.460905+0000","last_unstale":"2024-04-25T10:05:13.448417+0000","last_undegraded":"2024-04-25T10:05:13.448417+0000","last_fullsized":"2024-04-25T10:05:13.448417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:54.294853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750425+0000","last_change":"2024-04-25T10:05:11.472902+0000","last_active":"2024-04-25T10:05:13.750425+0000","last_peered":"2024-04-25T10:05:13.750425+0000","last_clean":"2024-04-25T10:05:13.750425+0000","last_became_active":"2024-04-25T10:05:11.472622+0000","last_became_peered":"2024-04-25T10:05:11.472622+0000","last_unstale":"2024-04-25T10:05:13.750425+0000","last_undegraded":"2024-04-25T10:05:13.750425+0000","last_fullsized":"2024-04-25T10:05:13.750425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:17.888835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456461+0000","last_change":"2024-04-25T10:05:10.460759+0000","last_active":"2024-04-25T10:05:12.456461+0000","last_peered":"2024-04-25T10:05:12.456461+0000","last_clean":"2024-04-25T10:05:12.456461+0000","last_became_active":"2024-04-25T10:05:10.460485+0000","last_became_peered":"2024-04-25T10:05:10.460485+0000","last_unstale":"2024-04-25T10:05:12.456461+0000","last_undegraded":"2024-04-25T10:05:12.456461+0000","last_fullsized":"2024-04-25T10:05:12.456461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456829+0000","last_change":"2024-04-25T10:05:11.470739+0000","last_active":"2024-04-25T10:05:12.456829+0000","last_peered":"2024-04-25T10:05:12.456829+0000","last_clean":"2024-04-25T10:05:12.456829+0000","last_became_active":"2024-04-25T10:05:11.470408+0000","last_became_peered":"2024-04-25T10:05:11.470408+0000","last_unstale":"2024-04-25T10:05:12.456829+0000","last_undegraded":"2024-04-25T10:05:12.456829+0000","last_fullsized":"2024-04-25T10:05:12.456829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:17.789806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.626301+0000","last_change":"2024-04-25T10:05:10.458496+0000","last_active":"2024-04-25T10:05:13.626301+0000","last_peered":"2024-04-25T10:05:13.626301+0000","last_clean":"2024-04-25T10:05:13.626301+0000","last_became_active":"2024-04-25T10:05:10.458168+0000","last_became_peered":"2024-04-25T10:05:10.458168+0000","last_unstale":"2024-04-25T10:05:13.626301+0000","last_undegraded":"2024-04-25T10:05:13.626301+0000","last_fullsized":"2024-04-25T10:05:13.626301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:41.255007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.649878+0000","last_change":"2024-04-25T10:05:11.459826+0000","last_active":"2024-04-25T10:05:13.649878+0000","last_peered":"2024-04-25T10:05:13.649878+0000","last_clean":"2024-04-25T10:05:13.649878+0000","last_became_active":"2024-04-25T10:05:11.459570+0000","last_became_peered":"2024-04-25T10:05:11.459570+0000","last_unstale":"2024-04-25T10:05:13.649878+0000","last_undegraded":"2024-04-25T10:05:13.649878+0000","last_fullsized":"2024-04-25T10:05:13.649878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750688+0000","last_change":"2024-04-25T10:05:10.455137+0000","last_active":"2024-04-25T10:05:13.750688+0000","last_peered":"2024-04-25T10:05:13.750688+0000","last_clean":"2024-04-25T10:05:13.750688+0000","last_became_active":"2024-04-25T10:05:10.454827+0000","last_became_peered":"2024-04-25T10:05:10.454827+0000","last_unstale":"2024-04-25T10:05:13.750688+0000","last_undegraded":"2024-04-25T10:05:13.750688+0000","last_fullsized":"2024-04-25T10:05:13.750688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:54.691904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750707+0000","last_change":"2024-04-25T10:05:11.469441+0000","last_active":"2024-04-25T10:05:13.750707+0000","last_peered":"2024-04-25T10:05:13.750707+0000","last_clean":"2024-04-25T10:05:13.750707+0000","last_became_active":"2024-04-25T10:05:11.468649+0000","last_became_peered":"2024-04-25T10:05:11.468649+0000","last_unstale":"2024-04-25T10:05:13.750707+0000","last_undegraded":"2024-04-25T10:05:13.750707+0000","last_fullsized":"2024-04-25T10:05:13.750707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:37.891605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750592+0000","last_change":"2024-04-25T10:05:11.465690+0000","last_active":"2024-04-25T10:05:13.750592+0000","last_peered":"2024-04-25T10:05:13.750592+0000","last_clean":"2024-04-25T10:05:13.750592+0000","last_became_active":"2024-04-25T10:05:11.465422+0000","last_became_peered":"2024-04-25T10:05:11.465422+0000","last_unstale":"2024-04-25T10:05:13.750592+0000","last_undegraded":"2024-04-25T10:05:13.750592+0000","last_fullsized":"2024-04-25T10:05:13.750592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:02.099978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.449083+0000","last_change":"2024-04-25T10:05:10.450133+0000","last_active":"2024-04-25T10:05:13.449083+0000","last_peered":"2024-04-25T10:05:13.449083+0000","last_clean":"2024-04-25T10:05:13.449083+0000","last_became_active":"2024-04-25T10:05:10.449482+0000","last_became_peered":"2024-04-25T10:05:10.449482+0000","last_unstale":"2024-04-25T10:05:13.449083+0000","last_undegraded":"2024-04-25T10:05:13.449083+0000","last_fullsized":"2024-04-25T10:05:13.449083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456402+0000","last_change":"2024-04-25T10:05:11.469301+0000","last_active":"2024-04-25T10:05:12.456402+0000","last_peered":"2024-04-25T10:05:12.456402+0000","last_clean":"2024-04-25T10:05:12.456402+0000","last_became_active":"2024-04-25T10:05:11.468980+0000","last_became_peered":"2024-04-25T10:05:11.468980+0000","last_unstale":"2024-04-25T10:05:12.456402+0000","last_undegraded":"2024-04-25T10:05:12.456402+0000","last_fullsized":"2024-04-25T10:05:12.456402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.449110+0000","last_change":"2024-04-25T10:05:10.456749+0000","last_active":"2024-04-25T10:05:13.449110+0000","last_peered":"2024-04-25T10:05:13.449110+0000","last_clean":"2024-04-25T10:05:13.449110+0000","last_became_active":"2024-04-25T10:05:10.456397+0000","last_became_peered":"2024-04-25T10:05:10.456397+0000","last_unstale":"2024-04-25T10:05:13.449110+0000","last_undegraded":"2024-04-25T10:05:13.449110+0000","last_fullsized":"2024-04-25T10:05:13.449110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786907+0000","last_change":"2024-04-25T10:05:11.453961+0000","last_active":"2024-04-25T10:05:14.786907+0000","last_peered":"2024-04-25T10:05:14.786907+0000","last_clean":"2024-04-25T10:05:14.786907+0000","last_became_active":"2024-04-25T10:05:11.453443+0000","last_became_peered":"2024-04-25T10:05:11.453443+0000","last_unstale":"2024-04-25T10:05:14.786907+0000","last_undegraded":"2024-04-25T10:05:14.786907+0000","last_fullsized":"2024-04-25T10:05:14.786907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:06.235586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786886+0000","last_change":"2024-04-25T10:05:10.458501+0000","last_active":"2024-04-25T10:05:14.786886+0000","last_peered":"2024-04-25T10:05:14.786886+0000","last_clean":"2024-04-25T10:05:14.786886+0000","last_became_active":"2024-04-25T10:05:10.457915+0000","last_became_peered":"2024-04-25T10:05:10.457915+0000","last_unstale":"2024-04-25T10:05:14.786886+0000","last_undegraded":"2024-04-25T10:05:14.786886+0000","last_fullsized":"2024-04-25T10:05:14.786886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:15.560541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786839+0000","last_change":"2024-04-25T10:05:11.451812+0000","last_active":"2024-04-25T10:05:14.786839+0000","last_peered":"2024-04-25T10:05:14.786839+0000","last_clean":"2024-04-25T10:05:14.786839+0000","last_became_active":"2024-04-25T10:05:11.451318+0000","last_became_peered":"2024-04-25T10:05:11.451318+0000","last_unstale":"2024-04-25T10:05:14.786839+0000","last_undegraded":"2024-04-25T10:05:14.786839+0000","last_fullsized":"2024-04-25T10:05:14.786839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:28.693266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.457002+0000","last_change":"2024-04-25T10:05:10.454280+0000","last_active":"2024-04-25T10:05:12.457002+0000","last_peered":"2024-04-25T10:05:12.457002+0000","last_clean":"2024-04-25T10:05:12.457002+0000","last_became_active":"2024-04-25T10:05:10.453982+0000","last_became_peered":"2024-04-25T10:05:10.453982+0000","last_unstale":"2024-04-25T10:05:12.457002+0000","last_undegraded":"2024-04-25T10:05:12.457002+0000","last_fullsized":"2024-04-25T10:05:12.457002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":76688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":76564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":78848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":225280,"data_stored":77146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":233472,"data_stored":78176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":667002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":229376,"data_stored":76620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":667024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:23.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.420+0000 7fe2e346f640 1 Processor -- start 2024-04-25T10:05:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.420+0000 7fe2e346f640 1 -- start start 2024-04-25T10:05:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.421+0000 7fe2e346f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe2dc0cea00 0x7fe2dc0cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.421+0000 7fe2e346f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe2dc0cfbe0 0x7fe2dc0ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.421+0000 7fe2e346f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2dc0d0430 con 0x7fe2dc15f120 2024-04-25T10:05:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.421+0000 7fe2e346f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2dc0d09f0 con 0x7fe2dc0cea00 2024-04-25T10:05:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.421+0000 7fe2e346f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2dc0d0710 con 0x7fe2dc0cfbe0 2024-04-25T10:05:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.421+0000 7fe2e0a0a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe2dc0cea00 0x7fe2dc0cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.421+0000 7fe2e0a0a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe2dc0cea00 0x7fe2dc0cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56188/0 (socket says 172.21.15.16:56188) 2024-04-25T10:05:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.421+0000 7fe2e0a0a640 1 -- 172.21.15.16:0/3989906376 learned_addr learned my addr 172.21.15.16:0/3989906376 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.421+0000 7fe2e1a0c640 1 --2- 172.21.15.16:0/3989906376 >> v2:172.21.15.186:3300/0 conn(0x7fe2dc0cfbe0 0x7fe2dc0ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.421+0000 7fe2e0a0a640 1 -- 172.21.15.16:0/3989906376 >> v2:172.21.15.186:3300/0 conn(0x7fe2dc0cfbe0 msgr2=0x7fe2dc0ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.421+0000 7fe2e0a0a640 1 --2- 172.21.15.16:0/3989906376 >> v2:172.21.15.186:3300/0 conn(0x7fe2dc0cfbe0 0x7fe2dc0ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.421+0000 7fe2e0a0a640 1 -- 172.21.15.16:0/3989906376 >> v1:172.21.15.186:6789/0 conn(0x7fe2dc15f120 legacy=0x7fe2dc0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.421+0000 7fe2e0a0a640 1 -- 172.21.15.16:0/3989906376 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc0cc7d0 con 0x7fe2dc0cea00 2024-04-25T10:05:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.421+0000 7fe2e1a0c640 1 --2- 172.21.15.16:0/3989906376 >> v2:172.21.15.186:3300/0 conn(0x7fe2dc0cfbe0 0x7fe2dc0ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.422+0000 7fe2e0a0a640 1 --2- 172.21.15.16:0/3989906376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc0cea00 0x7fe2dc0cf6a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0d29e0 tx=0x7fe2cc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5209c05eb819bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.422+0000 7fe2d3fff640 1 -- 172.21.15.16:0/3989906376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc037070 con 0x7fe2dc0cea00 2024-04-25T10:05:23.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.422+0000 7fe2d3fff640 1 -- 172.21.15.16:0/3989906376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2cc02ae50 con 0x7fe2dc0cea00 2024-04-25T10:05:23.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.422+0000 7fe2d3fff640 1 -- 172.21.15.16:0/3989906376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc032860 con 0x7fe2dc0cea00 2024-04-25T10:05:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.423+0000 7fe2e346f640 1 -- 172.21.15.16:0/3989906376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc0cea00 msgr2=0x7fe2dc0cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.423+0000 7fe2e346f640 1 --2- 172.21.15.16:0/3989906376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc0cea00 0x7fe2dc0cf6a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0d29e0 tx=0x7fe2cc02a300 comp rx=0 tx=0).stop 2024-04-25T10:05:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.423+0000 7fe2e346f640 1 -- 172.21.15.16:0/3989906376 shutdown_connections 2024-04-25T10:05:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.423+0000 7fe2e346f640 1 --2- 172.21.15.16:0/3989906376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc0cea00 0x7fe2dc0cf6a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.423+0000 7fe2e346f640 1 --2- 172.21.15.16:0/3989906376 >> v2:172.21.15.186:3300/0 conn(0x7fe2dc0cfbe0 0x7fe2dc0ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.423+0000 7fe2e346f640 1 -- 172.21.15.16:0/3989906376 >> 172.21.15.16:0/3989906376 conn(0x7fe2dc0d1900 msgr2=0x7fe2dc0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.423+0000 7fe2e346f640 1 -- 172.21.15.16:0/3989906376 shutdown_connections 2024-04-25T10:05:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.423+0000 7fe2e346f640 1 -- 172.21.15.16:0/3989906376 wait complete. 2024-04-25T10:05:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.424+0000 7fe2e346f640 1 Processor -- start 2024-04-25T10:05:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.424+0000 7fe2e346f640 1 -- start start 2024-04-25T10:05:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.425+0000 7fe2e346f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc0cea00 0x7fe2dc06bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.425+0000 7fe2e346f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2dc0cfbe0 0x7fe2dc1f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.425+0000 7fe2e346f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2dc15f120 0x7fe2dc0ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.425+0000 7fe2e346f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc0c5450 con 0x7fe2dc0cea00 2024-04-25T10:05:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.425+0000 7fe2e346f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc0cc480 con 0x7fe2dc0cfbe0 2024-04-25T10:05:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.425+0000 7fe2e346f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc0d0070 con 0x7fe2dc15f120 2024-04-25T10:05:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.425+0000 7fe2e120b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc0cea00 0x7fe2dc06bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.425+0000 7fe2e120b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc0cea00 0x7fe2dc06bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56202/0 (socket says 172.21.15.16:56202) 2024-04-25T10:05:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.425+0000 7fe2e120b640 1 -- 172.21.15.16:0/1499445080 learned_addr learned my addr 172.21.15.16:0/1499445080 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.425+0000 7fe2e0a0a640 1 --2- 172.21.15.16:0/1499445080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2dc0cfbe0 0x7fe2dc1f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.425+0000 7fe2e1a0c640 1 --2- 172.21.15.16:0/1499445080 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2dc15f120 0x7fe2dc0ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.425+0000 7fe2e120b640 1 -- 172.21.15.16:0/1499445080 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2dc15f120 msgr2=0x7fe2dc0ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.425+0000 7fe2e120b640 1 --2- 172.21.15.16:0/1499445080 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2dc15f120 0x7fe2dc0ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.425+0000 7fe2e120b640 1 -- 172.21.15.16:0/1499445080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2dc0cfbe0 msgr2=0x7fe2dc1f7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.425+0000 7fe2e120b640 1 --2- 172.21.15.16:0/1499445080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2dc0cfbe0 0x7fe2dc1f7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.425+0000 7fe2e120b640 1 -- 172.21.15.16:0/1499445080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc06c420 con 0x7fe2dc0cea00 2024-04-25T10:05:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.425+0000 7fe2e0a0a640 1 --2- 172.21.15.16:0/1499445080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2dc0cfbe0 0x7fe2dc1f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.425+0000 7fe2e120b640 1 --2- 172.21.15.16:0/1499445080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc0cea00 0x7fe2dc06bcb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800c6c0 tx=0x7fe2d800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.426+0000 7fe2d1ffb640 1 -- 172.21.15.16:0/1499445080 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8003b90 con 0x7fe2dc0cea00 2024-04-25T10:05:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.426+0000 7fe2d1ffb640 1 -- 172.21.15.16:0/1499445080 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d8003d30 con 0x7fe2dc0cea00 2024-04-25T10:05:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.426+0000 7fe2e346f640 1 -- 172.21.15.16:0/1499445080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2dc06c700 con 0x7fe2dc0cea00 2024-04-25T10:05:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.426+0000 7fe2d1ffb640 1 -- 172.21.15.16:0/1499445080 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8005310 con 0x7fe2dc0cea00 2024-04-25T10:05:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.426+0000 7fe2e346f640 1 -- 172.21.15.16:0/1499445080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2dc06ccc0 con 0x7fe2dc0cea00 2024-04-25T10:05:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.427+0000 7fe2e346f640 1 -- 172.21.15.16:0/1499445080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2dc067850 con 0x7fe2dc0cea00 2024-04-25T10:05:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.427+0000 7fe2d1ffb640 1 -- 172.21.15.16:0/1499445080 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2d80054b0 con 0x7fe2dc0cea00 2024-04-25T10:05:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.428+0000 7fe2d1ffb640 1 --2- 172.21.15.16:0/1499445080 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe2c006b500 0x7fe2c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.428+0000 7fe2d1ffb640 1 -- 172.21.15.16:0/1499445080 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2d8089fb0 con 0x7fe2dc0cea00 2024-04-25T10:05:23.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.428+0000 7fe2e0a0a640 1 --2- 172.21.15.16:0/1499445080 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe2c006b500 0x7fe2c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.428+0000 7fe2e0a0a640 1 --2- 172.21.15.16:0/1499445080 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe2c006b500 0x7fe2c006d9c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc034000 tx=0x7fe2cc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.432+0000 7fe2d1ffb640 1 -- 172.21.15.16:0/1499445080 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2d8058e30 con 0x7fe2dc0cea00 2024-04-25T10:05:23.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.608+0000 7fe2e346f640 1 -- 172.21.15.16:0/1499445080 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe2dc09a5f0 con 0x7fe2c006b500 2024-04-25T10:05:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.614+0000 7fe2d1ffb640 1 -- 172.21.15.16:0/1499445080 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251833 (secure 0 0 0) 0x7fe2cc07f7d0 con 0x7fe2c006b500 2024-04-25T10:05:23.622 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:23.623 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:05:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.617+0000 7fe2e346f640 1 -- 172.21.15.16:0/1499445080 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe2c006b500 msgr2=0x7fe2c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.617+0000 7fe2e346f640 1 --2- 172.21.15.16:0/1499445080 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe2c006b500 0x7fe2c006d9c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc034000 tx=0x7fe2cc02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.617+0000 7fe2e346f640 1 -- 172.21.15.16:0/1499445080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc0cea00 msgr2=0x7fe2dc06bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.617+0000 7fe2e346f640 1 --2- 172.21.15.16:0/1499445080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc0cea00 0x7fe2dc06bcb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800c6c0 tx=0x7fe2d800cb80 comp rx=0 tx=0).stop 2024-04-25T10:05:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.617+0000 7fe2e346f640 1 -- 172.21.15.16:0/1499445080 shutdown_connections 2024-04-25T10:05:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.617+0000 7fe2e346f640 1 --2- 172.21.15.16:0/1499445080 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe2c006b500 0x7fe2c006d9c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.617+0000 7fe2e346f640 1 --2- 172.21.15.16:0/1499445080 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2dc15f120 0x7fe2dc0ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.617+0000 7fe2e346f640 1 --2- 172.21.15.16:0/1499445080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2dc0cfbe0 0x7fe2dc1f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.617+0000 7fe2e346f640 1 --2- 172.21.15.16:0/1499445080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2dc0cea00 0x7fe2dc06bcb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.617+0000 7fe2e346f640 1 -- 172.21.15.16:0/1499445080 >> 172.21.15.16:0/1499445080 conn(0x7fe2dc0d1900 msgr2=0x7fe2dc0b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.617+0000 7fe2e346f640 1 -- 172.21.15.16:0/1499445080 shutdown_connections 2024-04-25T10:05:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.617+0000 7fe2e346f640 1 -- 172.21.15.16:0/1499445080 wait complete. 2024-04-25T10:05:23.633 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-04-25T10:05:21.783812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235780,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738940,"statfs":{"total":773094113280,"available":772852674560,"internally_reserved":0,"allocated":3022848,"data_stored":1798068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002578"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.449339+0000","last_change":"2024-04-25T10:05:11.473253+0000","last_active":"2024-04-25T10:05:13.449339+0000","last_peered":"2024-04-25T10:05:13.449339+0000","last_clean":"2024-04-25T10:05:13.449339+0000","last_became_active":"2024-04-25T10:05:11.472940+0000","last_became_peered":"2024-04-25T10:05:11.472940+0000","last_unstale":"2024-04-25T10:05:13.449339+0000","last_undegraded":"2024-04-25T10:05:13.449339+0000","last_fullsized":"2024-04-25T10:05:13.449339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:22.629451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786817+0000","last_change":"2024-04-25T10:05:11.460499+0000","last_active":"2024-04-25T10:05:14.786817+0000","last_peered":"2024-04-25T10:05:14.786817+0000","last_clean":"2024-04-25T10:05:14.786817+0000","last_became_active":"2024-04-25T10:05:11.458776+0000","last_became_peered":"2024-04-25T10:05:11.458776+0000","last_unstale":"2024-04-25T10:05:14.786817+0000","last_undegraded":"2024-04-25T10:05:14.786817+0000","last_fullsized":"2024-04-25T10:05:14.786817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:17.247685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786678+0000","last_change":"2024-04-25T10:05:11.466721+0000","last_active":"2024-04-25T10:05:14.786678+0000","last_peered":"2024-04-25T10:05:14.786678+0000","last_clean":"2024-04-25T10:05:14.786678+0000","last_became_active":"2024-04-25T10:05:11.466430+0000","last_became_peered":"2024-04-25T10:05:11.466430+0000","last_unstale":"2024-04-25T10:05:14.786678+0000","last_undegraded":"2024-04-25T10:05:14.786678+0000","last_fullsized":"2024-04-25T10:05:14.786678+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:34.227544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455508+0000","last_change":"2024-04-25T10:05:11.461498+0000","last_active":"2024-04-25T10:05:12.455508+0000","last_peered":"2024-04-25T10:05:12.455508+0000","last_clean":"2024-04-25T10:05:12.455508+0000","last_became_active":"2024-04-25T10:05:11.461241+0000","last_became_peered":"2024-04-25T10:05:11.461241+0000","last_unstale":"2024-04-25T10:05:12.455508+0000","last_undegraded":"2024-04-25T10:05:12.455508+0000","last_fullsized":"2024-04-25T10:05:12.455508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.448822+0000","last_change":"2024-04-25T10:05:11.469281+0000","last_active":"2024-04-25T10:05:13.448822+0000","last_peered":"2024-04-25T10:05:13.448822+0000","last_clean":"2024-04-25T10:05:13.448822+0000","last_became_active":"2024-04-25T10:05:11.468809+0000","last_became_peered":"2024-04-25T10:05:11.468809+0000","last_unstale":"2024-04-25T10:05:13.448822+0000","last_undegraded":"2024-04-25T10:05:13.448822+0000","last_fullsized":"2024-04-25T10:05:13.448822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:45.980553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455529+0000","last_change":"2024-04-25T10:05:11.466531+0000","last_active":"2024-04-25T10:05:12.455529+0000","last_peered":"2024-04-25T10:05:12.455529+0000","last_clean":"2024-04-25T10:05:12.455529+0000","last_became_active":"2024-04-25T10:05:11.466126+0000","last_became_peered":"2024-04-25T10:05:11.466126+0000","last_unstale":"2024-04-25T10:05:12.455529+0000","last_undegraded":"2024-04-25T10:05:12.455529+0000","last_fullsized":"2024-04-25T10:05:12.455529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.448621+0000","last_change":"2024-04-25T10:05:11.466336+0000","last_active":"2024-04-25T10:05:13.448621+0000","last_peered":"2024-04-25T10:05:13.448621+0000","last_clean":"2024-04-25T10:05:13.448621+0000","last_became_active":"2024-04-25T10:05:11.466032+0000","last_became_peered":"2024-04-25T10:05:11.466032+0000","last_unstale":"2024-04-25T10:05:13.448621+0000","last_undegraded":"2024-04-25T10:05:13.448621+0000","last_fullsized":"2024-04-25T10:05:13.448621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.228600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786457+0000","last_change":"2024-04-25T10:05:11.461892+0000","last_active":"2024-04-25T10:05:14.786457+0000","last_peered":"2024-04-25T10:05:14.786457+0000","last_clean":"2024-04-25T10:05:14.786457+0000","last_became_active":"2024-04-25T10:05:11.461264+0000","last_became_peered":"2024-04-25T10:05:11.461264+0000","last_unstale":"2024-04-25T10:05:14.786457+0000","last_undegraded":"2024-04-25T10:05:14.786457+0000","last_fullsized":"2024-04-25T10:05:14.786457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:58.263921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456654+0000","last_change":"2024-04-25T10:05:11.461997+0000","last_active":"2024-04-25T10:05:12.456654+0000","last_peered":"2024-04-25T10:05:12.456654+0000","last_clean":"2024-04-25T10:05:12.456654+0000","last_became_active":"2024-04-25T10:05:11.461658+0000","last_became_peered":"2024-04-25T10:05:11.461658+0000","last_unstale":"2024-04-25T10:05:12.456654+0000","last_undegraded":"2024-04-25T10:05:12.456654+0000","last_fullsized":"2024-04-25T10:05:12.456654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.449240+0000","last_change":"2024-04-25T10:05:11.461631+0000","last_active":"2024-04-25T10:05:13.449240+0000","last_peered":"2024-04-25T10:05:13.449240+0000","last_clean":"2024-04-25T10:05:13.449240+0000","last_became_active":"2024-04-25T10:05:11.461327+0000","last_became_peered":"2024-04-25T10:05:11.461327+0000","last_unstale":"2024-04-25T10:05:13.449240+0000","last_undegraded":"2024-04-25T10:05:13.449240+0000","last_fullsized":"2024-04-25T10:05:13.449240+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:48.544393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.448945+0000","last_change":"2024-04-25T10:05:11.473044+0000","last_active":"2024-04-25T10:05:13.448945+0000","last_peered":"2024-04-25T10:05:13.448945+0000","last_clean":"2024-04-25T10:05:13.448945+0000","last_became_active":"2024-04-25T10:05:11.472747+0000","last_became_peered":"2024-04-25T10:05:11.472747+0000","last_unstale":"2024-04-25T10:05:13.448945+0000","last_undegraded":"2024-04-25T10:05:13.448945+0000","last_fullsized":"2024-04-25T10:05:13.448945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:35.076762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.448767+0000","last_change":"2024-04-25T10:05:11.470219+0000","last_active":"2024-04-25T10:05:13.448767+0000","last_peered":"2024-04-25T10:05:13.448767+0000","last_clean":"2024-04-25T10:05:13.448767+0000","last_became_active":"2024-04-25T10:05:11.469066+0000","last_became_peered":"2024-04-25T10:05:11.469066+0000","last_unstale":"2024-04-25T10:05:13.448767+0000","last_undegraded":"2024-04-25T10:05:13.448767+0000","last_fullsized":"2024-04-25T10:05:13.448767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:41.766168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.626322+0000","last_change":"2024-04-25T10:05:11.463262+0000","last_active":"2024-04-25T10:05:13.626322+0000","last_peered":"2024-04-25T10:05:13.626322+0000","last_clean":"2024-04-25T10:05:13.626322+0000","last_became_active":"2024-04-25T10:05:11.462970+0000","last_became_peered":"2024-04-25T10:05:11.462970+0000","last_unstale":"2024-04-25T10:05:13.626322+0000","last_undegraded":"2024-04-25T10:05:13.626322+0000","last_fullsized":"2024-04-25T10:05:13.626322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:30.789969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.626079+0000","last_change":"2024-04-25T10:05:11.458310+0000","last_active":"2024-04-25T10:05:13.626079+0000","last_peered":"2024-04-25T10:05:13.626079+0000","last_clean":"2024-04-25T10:05:13.626079+0000","last_became_active":"2024-04-25T10:05:11.457672+0000","last_became_peered":"2024-04-25T10:05:11.457672+0000","last_unstale":"2024-04-25T10:05:13.626079+0000","last_undegraded":"2024-04-25T10:05:13.626079+0000","last_fullsized":"2024-04-25T10:05:13.626079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:55.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786423+0000","last_change":"2024-04-25T10:05:11.461506+0000","last_active":"2024-04-25T10:05:14.786423+0000","last_peered":"2024-04-25T10:05:14.786423+0000","last_clean":"2024-04-25T10:05:14.786423+0000","last_became_active":"2024-04-25T10:05:11.460963+0000","last_became_peered":"2024-04-25T10:05:11.460963+0000","last_unstale":"2024-04-25T10:05:14.786423+0000","last_undegraded":"2024-04-25T10:05:14.786423+0000","last_fullsized":"2024-04-25T10:05:14.786423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:45.096476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750382+0000","last_change":"2024-04-25T10:05:11.470743+0000","last_active":"2024-04-25T10:05:13.750382+0000","last_peered":"2024-04-25T10:05:13.750382+0000","last_clean":"2024-04-25T10:05:13.750382+0000","last_became_active":"2024-04-25T10:05:11.470447+0000","last_became_peered":"2024-04-25T10:05:11.470447+0000","last_unstale":"2024-04-25T10:05:13.750382+0000","last_undegraded":"2024-04-25T10:05:13.750382+0000","last_fullsized":"2024-04-25T10:05:13.750382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:22.629451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.751002+0000","last_change":"2024-04-25T10:05:11.472530+0000","last_active":"2024-04-25T10:05:13.751002+0000","last_peered":"2024-04-25T10:05:13.751002+0000","last_clean":"2024-04-25T10:05:13.751002+0000","last_became_active":"2024-04-25T10:05:11.472164+0000","last_became_peered":"2024-04-25T10:05:11.472164+0000","last_unstale":"2024-04-25T10:05:13.751002+0000","last_undegraded":"2024-04-25T10:05:13.751002+0000","last_fullsized":"2024-04-25T10:05:13.751002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:33.547465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.650020+0000","last_change":"2024-04-25T10:05:11.464186+0000","last_active":"2024-04-25T10:05:13.650020+0000","last_peered":"2024-04-25T10:05:13.650020+0000","last_clean":"2024-04-25T10:05:13.650020+0000","last_became_active":"2024-04-25T10:05:11.463925+0000","last_became_peered":"2024-04-25T10:05:11.463925+0000","last_unstale":"2024-04-25T10:05:13.650020+0000","last_undegraded":"2024-04-25T10:05:13.650020+0000","last_fullsized":"2024-04-25T10:05:13.650020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:24.590965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750917+0000","last_change":"2024-04-25T10:05:11.473554+0000","last_active":"2024-04-25T10:05:13.750917+0000","last_peered":"2024-04-25T10:05:13.750917+0000","last_clean":"2024-04-25T10:05:13.750917+0000","last_became_active":"2024-04-25T10:05:11.473271+0000","last_became_peered":"2024-04-25T10:05:11.473271+0000","last_unstale":"2024-04-25T10:05:13.750917+0000","last_undegraded":"2024-04-25T10:05:13.750917+0000","last_fullsized":"2024-04-25T10:05:13.750917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:07.933937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.448919+0000","last_change":"2024-04-25T10:05:11.471477+0000","last_active":"2024-04-25T10:05:13.448919+0000","last_peered":"2024-04-25T10:05:13.448919+0000","last_clean":"2024-04-25T10:05:13.448919+0000","last_became_active":"2024-04-25T10:05:11.471219+0000","last_became_peered":"2024-04-25T10:05:11.471219+0000","last_unstale":"2024-04-25T10:05:13.448919+0000","last_undegraded":"2024-04-25T10:05:13.448919+0000","last_fullsized":"2024-04-25T10:05:13.448919+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:24.616832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456655+0000","last_change":"2024-04-25T10:05:11.464410+0000","last_active":"2024-04-25T10:05:12.456655+0000","last_peered":"2024-04-25T10:05:12.456655+0000","last_clean":"2024-04-25T10:05:12.456655+0000","last_became_active":"2024-04-25T10:05:11.464003+0000","last_became_peered":"2024-04-25T10:05:11.464003+0000","last_unstale":"2024-04-25T10:05:12.456655+0000","last_undegraded":"2024-04-25T10:05:12.456655+0000","last_fullsized":"2024-04-25T10:05:12.456655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:52.014434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456342+0000","last_change":"2024-04-25T10:05:11.462487+0000","last_active":"2024-04-25T10:05:12.456342+0000","last_peered":"2024-04-25T10:05:12.456342+0000","last_clean":"2024-04-25T10:05:12.456342+0000","last_became_active":"2024-04-25T10:05:11.462191+0000","last_became_peered":"2024-04-25T10:05:11.462191+0000","last_unstale":"2024-04-25T10:05:12.456342+0000","last_undegraded":"2024-04-25T10:05:12.456342+0000","last_fullsized":"2024-04-25T10:05:12.456342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786682+0000","last_change":"2024-04-25T10:05:11.450886+0000","last_active":"2024-04-25T10:05:14.786682+0000","last_peered":"2024-04-25T10:05:14.786682+0000","last_clean":"2024-04-25T10:05:14.786682+0000","last_became_active":"2024-04-25T10:05:11.450616+0000","last_became_peered":"2024-04-25T10:05:11.450616+0000","last_unstale":"2024-04-25T10:05:14.786682+0000","last_undegraded":"2024-04-25T10:05:14.786682+0000","last_fullsized":"2024-04-25T10:05:14.786682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:47.607490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750381+0000","last_change":"2024-04-25T10:05:11.467406+0000","last_active":"2024-04-25T10:05:13.750381+0000","last_peered":"2024-04-25T10:05:13.750381+0000","last_clean":"2024-04-25T10:05:13.750381+0000","last_became_active":"2024-04-25T10:05:11.467126+0000","last_became_peered":"2024-04-25T10:05:11.467126+0000","last_unstale":"2024-04-25T10:05:13.750381+0000","last_undegraded":"2024-04-25T10:05:13.750381+0000","last_fullsized":"2024-04-25T10:05:13.750381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:55.943373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786879+0000","last_change":"2024-04-25T10:05:11.453135+0000","last_active":"2024-04-25T10:05:14.786879+0000","last_peered":"2024-04-25T10:05:14.786879+0000","last_clean":"2024-04-25T10:05:14.786879+0000","last_became_active":"2024-04-25T10:05:11.452053+0000","last_became_peered":"2024-04-25T10:05:11.452053+0000","last_unstale":"2024-04-25T10:05:14.786879+0000","last_undegraded":"2024-04-25T10:05:14.786879+0000","last_fullsized":"2024-04-25T10:05:14.786879+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:03.147551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455642+0000","last_change":"2024-04-25T10:05:11.469452+0000","last_active":"2024-04-25T10:05:12.455642+0000","last_peered":"2024-04-25T10:05:12.455642+0000","last_clean":"2024-04-25T10:05:12.455642+0000","last_became_active":"2024-04-25T10:05:11.465896+0000","last_became_peered":"2024-04-25T10:05:11.465896+0000","last_unstale":"2024-04-25T10:05:12.455642+0000","last_undegraded":"2024-04-25T10:05:12.455642+0000","last_fullsized":"2024-04-25T10:05:12.455642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:14.506306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.626351+0000","last_change":"2024-04-25T10:05:11.457595+0000","last_active":"2024-04-25T10:05:13.626351+0000","last_peered":"2024-04-25T10:05:13.626351+0000","last_clean":"2024-04-25T10:05:13.626351+0000","last_became_active":"2024-04-25T10:05:11.457320+0000","last_became_peered":"2024-04-25T10:05:11.457320+0000","last_unstale":"2024-04-25T10:05:13.626351+0000","last_undegraded":"2024-04-25T10:05:13.626351+0000","last_fullsized":"2024-04-25T10:05:13.626351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.626269+0000","last_change":"2024-04-25T10:05:11.469113+0000","last_active":"2024-04-25T10:05:13.626269+0000","last_peered":"2024-04-25T10:05:13.626269+0000","last_clean":"2024-04-25T10:05:13.626269+0000","last_became_active":"2024-04-25T10:05:11.468759+0000","last_became_peered":"2024-04-25T10:05:11.468759+0000","last_unstale":"2024-04-25T10:05:13.626269+0000","last_undegraded":"2024-04-25T10:05:13.626269+0000","last_fullsized":"2024-04-25T10:05:13.626269+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:12.926321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786630+0000","last_change":"2024-04-25T10:05:11.457486+0000","last_active":"2024-04-25T10:05:14.786630+0000","last_peered":"2024-04-25T10:05:14.786630+0000","last_clean":"2024-04-25T10:05:14.786630+0000","last_became_active":"2024-04-25T10:05:11.454855+0000","last_became_peered":"2024-04-25T10:05:11.454855+0000","last_unstale":"2024-04-25T10:05:14.786630+0000","last_undegraded":"2024-04-25T10:05:14.786630+0000","last_fullsized":"2024-04-25T10:05:14.786630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:45.863831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750531+0000","last_change":"2024-04-25T10:05:11.470055+0000","last_active":"2024-04-25T10:05:13.750531+0000","last_peered":"2024-04-25T10:05:13.750531+0000","last_clean":"2024-04-25T10:05:13.750531+0000","last_became_active":"2024-04-25T10:05:11.468974+0000","last_became_peered":"2024-04-25T10:05:11.468974+0000","last_unstale":"2024-04-25T10:05:13.750531+0000","last_undegraded":"2024-04-25T10:05:13.750531+0000","last_fullsized":"2024-04-25T10:05:13.750531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:20.369758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.649698+0000","last_change":"2024-04-25T10:05:11.462234+0000","last_active":"2024-04-25T10:05:13.649698+0000","last_peered":"2024-04-25T10:05:13.649698+0000","last_clean":"2024-04-25T10:05:13.649698+0000","last_became_active":"2024-04-25T10:05:11.461975+0000","last_became_peered":"2024-04-25T10:05:11.461975+0000","last_unstale":"2024-04-25T10:05:13.649698+0000","last_undegraded":"2024-04-25T10:05:13.649698+0000","last_fullsized":"2024-04-25T10:05:13.649698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455396+0000","last_change":"2024-04-25T10:05:11.463406+0000","last_active":"2024-04-25T10:05:12.455396+0000","last_peered":"2024-04-25T10:05:12.455396+0000","last_clean":"2024-04-25T10:05:12.455396+0000","last_became_active":"2024-04-25T10:05:11.463139+0000","last_became_peered":"2024-04-25T10:05:11.463139+0000","last_unstale":"2024-04-25T10:05:12.455396+0000","last_undegraded":"2024-04-25T10:05:12.455396+0000","last_fullsized":"2024-04-25T10:05:12.455396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:27.103164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456384+0000","last_change":"2024-04-25T10:05:11.453168+0000","last_active":"2024-04-25T10:05:12.456384+0000","last_peered":"2024-04-25T10:05:12.456384+0000","last_clean":"2024-04-25T10:05:12.456384+0000","last_became_active":"2024-04-25T10:05:11.452238+0000","last_became_peered":"2024-04-25T10:05:11.452238+0000","last_unstale":"2024-04-25T10:05:12.456384+0000","last_undegraded":"2024-04-25T10:05:12.456384+0000","last_fullsized":"2024-04-25T10:05:12.456384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786706+0000","last_change":"2024-04-25T10:05:10.455995+0000","last_active":"2024-04-25T10:05:14.786706+0000","last_peered":"2024-04-25T10:05:14.786706+0000","last_clean":"2024-04-25T10:05:14.786706+0000","last_became_active":"2024-04-25T10:05:10.455718+0000","last_became_peered":"2024-04-25T10:05:10.455718+0000","last_unstale":"2024-04-25T10:05:14.786706+0000","last_undegraded":"2024-04-25T10:05:14.786706+0000","last_fullsized":"2024-04-25T10:05:14.786706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:47.448482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456962+0000","last_change":"2024-04-25T10:05:11.469760+0000","last_active":"2024-04-25T10:05:12.456962+0000","last_peered":"2024-04-25T10:05:12.456962+0000","last_clean":"2024-04-25T10:05:12.456962+0000","last_became_active":"2024-04-25T10:05:11.469490+0000","last_became_peered":"2024-04-25T10:05:11.469490+0000","last_unstale":"2024-04-25T10:05:12.456962+0000","last_undegraded":"2024-04-25T10:05:12.456962+0000","last_fullsized":"2024-04-25T10:05:12.456962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.228600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750654+0000","last_change":"2024-04-25T10:05:10.461940+0000","last_active":"2024-04-25T10:05:13.750654+0000","last_peered":"2024-04-25T10:05:13.750654+0000","last_clean":"2024-04-25T10:05:13.750654+0000","last_became_active":"2024-04-25T10:05:10.461633+0000","last_became_peered":"2024-04-25T10:05:10.461633+0000","last_unstale":"2024-04-25T10:05:13.750654+0000","last_undegraded":"2024-04-25T10:05:13.750654+0000","last_fullsized":"2024-04-25T10:05:13.750654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.625954+0000","last_change":"2024-04-25T10:05:10.460823+0000","last_active":"2024-04-25T10:05:13.625954+0000","last_peered":"2024-04-25T10:05:13.625954+0000","last_clean":"2024-04-25T10:05:13.625954+0000","last_became_active":"2024-04-25T10:05:10.460480+0000","last_became_peered":"2024-04-25T10:05:10.460480+0000","last_unstale":"2024-04-25T10:05:13.625954+0000","last_undegraded":"2024-04-25T10:05:13.625954+0000","last_fullsized":"2024-04-25T10:05:13.625954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:31.375084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786743+0000","last_change":"2024-04-25T10:05:11.460164+0000","last_active":"2024-04-25T10:05:14.786743+0000","last_peered":"2024-04-25T10:05:14.786743+0000","last_clean":"2024-04-25T10:05:14.786743+0000","last_became_active":"2024-04-25T10:05:11.459615+0000","last_became_peered":"2024-04-25T10:05:11.459615+0000","last_unstale":"2024-04-25T10:05:14.786743+0000","last_undegraded":"2024-04-25T10:05:14.786743+0000","last_fullsized":"2024-04-25T10:05:14.786743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:00.275933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750579+0000","last_change":"2024-04-25T10:05:10.461713+0000","last_active":"2024-04-25T10:05:13.750579+0000","last_peered":"2024-04-25T10:05:13.750579+0000","last_clean":"2024-04-25T10:05:13.750579+0000","last_became_active":"2024-04-25T10:05:10.461379+0000","last_became_peered":"2024-04-25T10:05:10.461379+0000","last_unstale":"2024-04-25T10:05:13.750579+0000","last_undegraded":"2024-04-25T10:05:13.750579+0000","last_fullsized":"2024-04-25T10:05:13.750579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456510+0000","last_change":"2024-04-25T10:05:11.466408+0000","last_active":"2024-04-25T10:05:12.456510+0000","last_peered":"2024-04-25T10:05:12.456510+0000","last_clean":"2024-04-25T10:05:12.456510+0000","last_became_active":"2024-04-25T10:05:11.465611+0000","last_became_peered":"2024-04-25T10:05:11.465611+0000","last_unstale":"2024-04-25T10:05:12.456510+0000","last_undegraded":"2024-04-25T10:05:12.456510+0000","last_fullsized":"2024-04-25T10:05:12.456510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456657+0000","last_change":"2024-04-25T10:05:10.449054+0000","last_active":"2024-04-25T10:05:12.456657+0000","last_peered":"2024-04-25T10:05:12.456657+0000","last_clean":"2024-04-25T10:05:12.456657+0000","last_became_active":"2024-04-25T10:05:10.448743+0000","last_became_peered":"2024-04-25T10:05:10.448743+0000","last_unstale":"2024-04-25T10:05:12.456657+0000","last_undegraded":"2024-04-25T10:05:12.456657+0000","last_fullsized":"2024-04-25T10:05:12.456657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:22.897294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455686+0000","last_change":"2024-04-25T10:05:11.453268+0000","last_active":"2024-04-25T10:05:12.455686+0000","last_peered":"2024-04-25T10:05:12.455686+0000","last_clean":"2024-04-25T10:05:12.455686+0000","last_became_active":"2024-04-25T10:05:11.452971+0000","last_became_peered":"2024-04-25T10:05:11.452971+0000","last_unstale":"2024-04-25T10:05:12.455686+0000","last_undegraded":"2024-04-25T10:05:12.455686+0000","last_fullsized":"2024-04-25T10:05:12.455686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:56.142030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.449266+0000","last_change":"2024-04-25T10:05:10.452247+0000","last_active":"2024-04-25T10:05:13.449266+0000","last_peered":"2024-04-25T10:05:13.449266+0000","last_clean":"2024-04-25T10:05:13.449266+0000","last_became_active":"2024-04-25T10:05:10.451925+0000","last_became_peered":"2024-04-25T10:05:10.451925+0000","last_unstale":"2024-04-25T10:05:13.449266+0000","last_undegraded":"2024-04-25T10:05:13.449266+0000","last_fullsized":"2024-04-25T10:05:13.449266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:53.640804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786996+0000","last_change":"2024-04-25T10:05:11.467490+0000","last_active":"2024-04-25T10:05:14.786996+0000","last_peered":"2024-04-25T10:05:14.786996+0000","last_clean":"2024-04-25T10:05:14.786996+0000","last_became_active":"2024-04-25T10:05:11.467217+0000","last_became_peered":"2024-04-25T10:05:11.467217+0000","last_unstale":"2024-04-25T10:05:14.786996+0000","last_undegraded":"2024-04-25T10:05:14.786996+0000","last_fullsized":"2024-04-25T10:05:14.786996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:26.821659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.649772+0000","last_change":"2024-04-25T10:05:10.459898+0000","last_active":"2024-04-25T10:05:13.649772+0000","last_peered":"2024-04-25T10:05:13.649772+0000","last_clean":"2024-04-25T10:05:13.649772+0000","last_became_active":"2024-04-25T10:05:10.459250+0000","last_became_peered":"2024-04-25T10:05:10.459250+0000","last_unstale":"2024-04-25T10:05:13.649772+0000","last_undegraded":"2024-04-25T10:05:13.649772+0000","last_fullsized":"2024-04-25T10:05:13.649772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:41.255007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750541+0000","last_change":"2024-04-25T10:05:11.471498+0000","last_active":"2024-04-25T10:05:13.750541+0000","last_peered":"2024-04-25T10:05:13.750541+0000","last_clean":"2024-04-25T10:05:13.750541+0000","last_became_active":"2024-04-25T10:05:11.471193+0000","last_became_peered":"2024-04-25T10:05:11.471193+0000","last_unstale":"2024-04-25T10:05:13.750541+0000","last_undegraded":"2024-04-25T10:05:13.750541+0000","last_fullsized":"2024-04-25T10:05:13.750541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:53.026414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455560+0000","last_change":"2024-04-25T10:05:10.450534+0000","last_active":"2024-04-25T10:05:12.455560+0000","last_peered":"2024-04-25T10:05:12.455560+0000","last_clean":"2024-04-25T10:05:12.455560+0000","last_became_active":"2024-04-25T10:05:10.450251+0000","last_became_peered":"2024-04-25T10:05:10.450251+0000","last_unstale":"2024-04-25T10:05:12.455560+0000","last_undegraded":"2024-04-25T10:05:12.455560+0000","last_fullsized":"2024-04-25T10:05:12.455560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:22.897294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.449211+0000","last_change":"2024-04-25T10:05:11.463480+0000","last_active":"2024-04-25T10:05:13.449211+0000","last_peered":"2024-04-25T10:05:13.449211+0000","last_clean":"2024-04-25T10:05:13.449211+0000","last_became_active":"2024-04-25T10:05:11.460995+0000","last_became_peered":"2024-04-25T10:05:11.460995+0000","last_unstale":"2024-04-25T10:05:13.449211+0000","last_undegraded":"2024-04-25T10:05:13.449211+0000","last_fullsized":"2024-04-25T10:05:13.449211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:55.943373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.649937+0000","last_change":"2024-04-25T10:05:10.462323+0000","last_active":"2024-04-25T10:05:13.649937+0000","last_peered":"2024-04-25T10:05:13.649937+0000","last_clean":"2024-04-25T10:05:13.649937+0000","last_became_active":"2024-04-25T10:05:10.462118+0000","last_became_peered":"2024-04-25T10:05:10.462118+0000","last_unstale":"2024-04-25T10:05:13.649937+0000","last_undegraded":"2024-04-25T10:05:13.649937+0000","last_fullsized":"2024-04-25T10:05:13.649937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:39.826605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.626366+0000","last_change":"2024-04-25T10:05:11.467396+0000","last_active":"2024-04-25T10:05:13.626366+0000","last_peered":"2024-04-25T10:05:13.626366+0000","last_clean":"2024-04-25T10:05:13.626366+0000","last_became_active":"2024-04-25T10:05:11.467112+0000","last_became_peered":"2024-04-25T10:05:11.467112+0000","last_unstale":"2024-04-25T10:05:13.626366+0000","last_undegraded":"2024-04-25T10:05:13.626366+0000","last_fullsized":"2024-04-25T10:05:13.626366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:25.122545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750782+0000","last_change":"2024-04-25T10:05:10.458429+0000","last_active":"2024-04-25T10:05:13.750782+0000","last_peered":"2024-04-25T10:05:13.750782+0000","last_clean":"2024-04-25T10:05:13.750782+0000","last_became_active":"2024-04-25T10:05:10.457958+0000","last_became_peered":"2024-04-25T10:05:10.457958+0000","last_unstale":"2024-04-25T10:05:13.750782+0000","last_undegraded":"2024-04-25T10:05:13.750782+0000","last_fullsized":"2024-04-25T10:05:13.750782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:23.585636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750626+0000","last_change":"2024-04-25T10:05:11.465006+0000","last_active":"2024-04-25T10:05:13.750626+0000","last_peered":"2024-04-25T10:05:13.750626+0000","last_clean":"2024-04-25T10:05:13.750626+0000","last_became_active":"2024-04-25T10:05:11.464719+0000","last_became_peered":"2024-04-25T10:05:11.464719+0000","last_unstale":"2024-04-25T10:05:13.750626+0000","last_undegraded":"2024-04-25T10:05:13.750626+0000","last_fullsized":"2024-04-25T10:05:13.750626+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:12.496324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.448634+0000","last_change":"2024-04-25T10:05:10.456938+0000","last_active":"2024-04-25T10:05:13.448634+0000","last_peered":"2024-04-25T10:05:13.448634+0000","last_clean":"2024-04-25T10:05:13.448634+0000","last_became_active":"2024-04-25T10:05:10.456644+0000","last_became_peered":"2024-04-25T10:05:10.456644+0000","last_unstale":"2024-04-25T10:05:13.448634+0000","last_undegraded":"2024-04-25T10:05:13.448634+0000","last_fullsized":"2024-04-25T10:05:13.448634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456188+0000","last_change":"2024-04-25T10:05:11.456178+0000","last_active":"2024-04-25T10:05:12.456188+0000","last_peered":"2024-04-25T10:05:12.456188+0000","last_clean":"2024-04-25T10:05:12.456188+0000","last_became_active":"2024-04-25T10:05:11.455879+0000","last_became_peered":"2024-04-25T10:05:11.455879+0000","last_unstale":"2024-04-25T10:05:12.456188+0000","last_undegraded":"2024-04-25T10:05:12.456188+0000","last_fullsized":"2024-04-25T10:05:12.456188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:55.300183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456275+0000","last_change":"2024-04-25T10:05:10.455684+0000","last_active":"2024-04-25T10:05:12.456275+0000","last_peered":"2024-04-25T10:05:12.456275+0000","last_clean":"2024-04-25T10:05:12.456275+0000","last_became_active":"2024-04-25T10:05:10.455342+0000","last_became_peered":"2024-04-25T10:05:10.455342+0000","last_unstale":"2024-04-25T10:05:12.456275+0000","last_undegraded":"2024-04-25T10:05:12.456275+0000","last_fullsized":"2024-04-25T10:05:12.456275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:01.556228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750396+0000","last_change":"2024-04-25T10:05:11.466622+0000","last_active":"2024-04-25T10:05:13.750396+0000","last_peered":"2024-04-25T10:05:13.750396+0000","last_clean":"2024-04-25T10:05:13.750396+0000","last_became_active":"2024-04-25T10:05:11.466346+0000","last_became_peered":"2024-04-25T10:05:11.466346+0000","last_unstale":"2024-04-25T10:05:13.750396+0000","last_undegraded":"2024-04-25T10:05:13.750396+0000","last_fullsized":"2024-04-25T10:05:13.750396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:48.544393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.449417+0000","last_change":"2024-04-25T10:05:10.457614+0000","last_active":"2024-04-25T10:05:13.449417+0000","last_peered":"2024-04-25T10:05:13.449417+0000","last_clean":"2024-04-25T10:05:13.449417+0000","last_became_active":"2024-04-25T10:05:10.457332+0000","last_became_peered":"2024-04-25T10:05:10.457332+0000","last_unstale":"2024-04-25T10:05:13.449417+0000","last_undegraded":"2024-04-25T10:05:13.449417+0000","last_fullsized":"2024-04-25T10:05:13.449417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:01.556228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455780+0000","last_change":"2024-04-25T10:05:11.460589+0000","last_active":"2024-04-25T10:05:12.455780+0000","last_peered":"2024-04-25T10:05:12.455780+0000","last_clean":"2024-04-25T10:05:12.455780+0000","last_became_active":"2024-04-25T10:05:11.459433+0000","last_became_peered":"2024-04-25T10:05:11.459433+0000","last_unstale":"2024-04-25T10:05:12.455780+0000","last_undegraded":"2024-04-25T10:05:12.455780+0000","last_fullsized":"2024-04-25T10:05:12.455780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:55.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456910+0000","last_change":"2024-04-25T10:05:10.458692+0000","last_active":"2024-04-25T10:05:12.456910+0000","last_peered":"2024-04-25T10:05:12.456910+0000","last_clean":"2024-04-25T10:05:12.456910+0000","last_became_active":"2024-04-25T10:05:10.458090+0000","last_became_peered":"2024-04-25T10:05:10.458090+0000","last_unstale":"2024-04-25T10:05:12.456910+0000","last_undegraded":"2024-04-25T10:05:12.456910+0000","last_fullsized":"2024-04-25T10:05:12.456910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:23.585636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.449379+0000","last_change":"2024-04-25T10:05:11.464532+0000","last_active":"2024-04-25T10:05:13.449379+0000","last_peered":"2024-04-25T10:05:13.449379+0000","last_clean":"2024-04-25T10:05:13.449379+0000","last_became_active":"2024-04-25T10:05:11.464186+0000","last_became_peered":"2024-04-25T10:05:11.464186+0000","last_unstale":"2024-04-25T10:05:13.449379+0000","last_undegraded":"2024-04-25T10:05:13.449379+0000","last_fullsized":"2024-04-25T10:05:13.449379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:17.888835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786543+0000","last_change":"2024-04-25T10:05:10.460107+0000","last_active":"2024-04-25T10:05:14.786543+0000","last_peered":"2024-04-25T10:05:14.786543+0000","last_clean":"2024-04-25T10:05:14.786543+0000","last_became_active":"2024-04-25T10:05:10.459754+0000","last_became_peered":"2024-04-25T10:05:10.459754+0000","last_unstale":"2024-04-25T10:05:14.786543+0000","last_undegraded":"2024-04-25T10:05:14.786543+0000","last_fullsized":"2024-04-25T10:05:14.786543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:48.214248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.448967+0000","last_change":"2024-04-25T10:05:11.474519+0000","last_active":"2024-04-25T10:05:13.448967+0000","last_peered":"2024-04-25T10:05:13.448967+0000","last_clean":"2024-04-25T10:05:13.448967+0000","last_became_active":"2024-04-25T10:05:11.472675+0000","last_became_peered":"2024-04-25T10:05:11.472675+0000","last_unstale":"2024-04-25T10:05:13.448967+0000","last_undegraded":"2024-04-25T10:05:13.448967+0000","last_fullsized":"2024-04-25T10:05:13.448967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:48.013538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.626392+0000","last_change":"2024-04-25T10:05:10.453364+0000","last_active":"2024-04-25T10:05:13.626392+0000","last_peered":"2024-04-25T10:05:13.626392+0000","last_clean":"2024-04-25T10:05:13.626392+0000","last_became_active":"2024-04-25T10:05:10.453072+0000","last_became_peered":"2024-04-25T10:05:10.453072+0000","last_unstale":"2024-04-25T10:05:13.626392+0000","last_undegraded":"2024-04-25T10:05:13.626392+0000","last_fullsized":"2024-04-25T10:05:13.626392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:53.640804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750741+0000","last_change":"2024-04-25T10:05:11.467814+0000","last_active":"2024-04-25T10:05:13.750741+0000","last_peered":"2024-04-25T10:05:13.750741+0000","last_clean":"2024-04-25T10:05:13.750741+0000","last_became_active":"2024-04-25T10:05:11.467496+0000","last_became_peered":"2024-04-25T10:05:11.467496+0000","last_unstale":"2024-04-25T10:05:13.750741+0000","last_undegraded":"2024-04-25T10:05:13.750741+0000","last_fullsized":"2024-04-25T10:05:13.750741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:18.768171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456108+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:05:12.456108+0000","last_peered":"2024-04-25T10:05:12.456108+0000","last_clean":"2024-04-25T10:05:12.456108+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:05:12.456108+0000","last_undegraded":"2024-04-25T10:05:12.456108+0000","last_fullsized":"2024-04-25T10:05:12.456108+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750724+0000","last_change":"2024-04-25T10:05:10.458170+0000","last_active":"2024-04-25T10:05:13.750724+0000","last_peered":"2024-04-25T10:05:13.750724+0000","last_clean":"2024-04-25T10:05:13.750724+0000","last_became_active":"2024-04-25T10:05:10.457794+0000","last_became_peered":"2024-04-25T10:05:10.457794+0000","last_unstale":"2024-04-25T10:05:13.750724+0000","last_undegraded":"2024-04-25T10:05:13.750724+0000","last_fullsized":"2024-04-25T10:05:13.750724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:11.920992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456600+0000","last_change":"2024-04-25T10:05:11.462574+0000","last_active":"2024-04-25T10:05:12.456600+0000","last_peered":"2024-04-25T10:05:12.456600+0000","last_clean":"2024-04-25T10:05:12.456600+0000","last_became_active":"2024-04-25T10:05:11.461617+0000","last_became_peered":"2024-04-25T10:05:11.461617+0000","last_unstale":"2024-04-25T10:05:12.456600+0000","last_undegraded":"2024-04-25T10:05:12.456600+0000","last_fullsized":"2024-04-25T10:05:12.456600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.333704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750513+0000","last_change":"2024-04-25T10:05:10.464334+0000","last_active":"2024-04-25T10:05:13.750513+0000","last_peered":"2024-04-25T10:05:13.750513+0000","last_clean":"2024-04-25T10:05:13.750513+0000","last_became_active":"2024-04-25T10:05:10.463991+0000","last_became_peered":"2024-04-25T10:05:10.463991+0000","last_unstale":"2024-04-25T10:05:13.750513+0000","last_undegraded":"2024-04-25T10:05:13.750513+0000","last_fullsized":"2024-04-25T10:05:13.750513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.626027+0000","last_change":"2024-04-25T10:05:11.464866+0000","last_active":"2024-04-25T10:05:13.626027+0000","last_peered":"2024-04-25T10:05:13.626027+0000","last_clean":"2024-04-25T10:05:13.626027+0000","last_became_active":"2024-04-25T10:05:11.464504+0000","last_became_peered":"2024-04-25T10:05:11.464504+0000","last_unstale":"2024-04-25T10:05:13.626027+0000","last_undegraded":"2024-04-25T10:05:13.626027+0000","last_fullsized":"2024-04-25T10:05:13.626027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:56.142030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.649649+0000","last_change":"2024-04-25T10:05:10.458220+0000","last_active":"2024-04-25T10:05:13.649649+0000","last_peered":"2024-04-25T10:05:13.649649+0000","last_clean":"2024-04-25T10:05:13.649649+0000","last_became_active":"2024-04-25T10:05:10.457936+0000","last_became_peered":"2024-04-25T10:05:10.457936+0000","last_unstale":"2024-04-25T10:05:13.649649+0000","last_undegraded":"2024-04-25T10:05:13.649649+0000","last_fullsized":"2024-04-25T10:05:13.649649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:31.375084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786598+0000","last_change":"2024-04-25T10:05:11.457665+0000","last_active":"2024-04-25T10:05:14.786598+0000","last_peered":"2024-04-25T10:05:14.786598+0000","last_clean":"2024-04-25T10:05:14.786598+0000","last_became_active":"2024-04-25T10:05:11.455078+0000","last_became_peered":"2024-04-25T10:05:11.455078+0000","last_unstale":"2024-04-25T10:05:14.786598+0000","last_undegraded":"2024-04-25T10:05:14.786598+0000","last_fullsized":"2024-04-25T10:05:14.786598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:21.453409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750827+0000","last_change":"2024-04-25T10:05:10.463388+0000","last_active":"2024-04-25T10:05:13.750827+0000","last_peered":"2024-04-25T10:05:13.750827+0000","last_clean":"2024-04-25T10:05:13.750827+0000","last_became_active":"2024-04-25T10:05:10.463041+0000","last_became_peered":"2024-04-25T10:05:10.463041+0000","last_unstale":"2024-04-25T10:05:13.750827+0000","last_undegraded":"2024-04-25T10:05:13.750827+0000","last_fullsized":"2024-04-25T10:05:13.750827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:13.500977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.448829+0000","last_change":"2024-04-25T10:05:11.462360+0000","last_active":"2024-04-25T10:05:13.448829+0000","last_peered":"2024-04-25T10:05:13.448829+0000","last_clean":"2024-04-25T10:05:13.448829+0000","last_became_active":"2024-04-25T10:05:11.462050+0000","last_became_peered":"2024-04-25T10:05:11.462050+0000","last_unstale":"2024-04-25T10:05:13.448829+0000","last_undegraded":"2024-04-25T10:05:13.448829+0000","last_fullsized":"2024-04-25T10:05:13.448829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:08.525515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786571+0000","last_change":"2024-04-25T10:05:10.456403+0000","last_active":"2024-04-25T10:05:14.786571+0000","last_peered":"2024-04-25T10:05:14.786571+0000","last_clean":"2024-04-25T10:05:14.786571+0000","last_became_active":"2024-04-25T10:05:10.456060+0000","last_became_peered":"2024-04-25T10:05:10.456060+0000","last_unstale":"2024-04-25T10:05:14.786571+0000","last_undegraded":"2024-04-25T10:05:14.786571+0000","last_fullsized":"2024-04-25T10:05:14.786571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:50.107731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.455433+0000","last_change":"2024-04-25T10:05:11.464184+0000","last_active":"2024-04-25T10:05:12.455433+0000","last_peered":"2024-04-25T10:05:12.455433+0000","last_clean":"2024-04-25T10:05:12.455433+0000","last_became_active":"2024-04-25T10:05:11.463894+0000","last_became_peered":"2024-04-25T10:05:11.463894+0000","last_unstale":"2024-04-25T10:05:12.455433+0000","last_undegraded":"2024-04-25T10:05:12.455433+0000","last_fullsized":"2024-04-25T10:05:12.455433+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:05.872605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750931+0000","last_change":"2024-04-25T10:05:10.460261+0000","last_active":"2024-04-25T10:05:13.750931+0000","last_peered":"2024-04-25T10:05:13.750931+0000","last_clean":"2024-04-25T10:05:13.750931+0000","last_became_active":"2024-04-25T10:05:10.459949+0000","last_became_peered":"2024-04-25T10:05:10.459949+0000","last_unstale":"2024-04-25T10:05:13.750931+0000","last_undegraded":"2024-04-25T10:05:13.750931+0000","last_fullsized":"2024-04-25T10:05:13.750931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:29.784640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.626340+0000","last_change":"2024-04-25T10:05:11.458748+0000","last_active":"2024-04-25T10:05:13.626340+0000","last_peered":"2024-04-25T10:05:13.626340+0000","last_clean":"2024-04-25T10:05:13.626340+0000","last_became_active":"2024-04-25T10:05:11.458440+0000","last_became_peered":"2024-04-25T10:05:11.458440+0000","last_unstale":"2024-04-25T10:05:13.626340+0000","last_undegraded":"2024-04-25T10:05:13.626340+0000","last_fullsized":"2024-04-25T10:05:13.626340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:54.934726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.793557+0000","last_change":"2024-04-25T10:05:10.455250+0000","last_active":"2024-04-25T10:05:12.793557+0000","last_peered":"2024-04-25T10:05:12.793557+0000","last_clean":"2024-04-25T10:05:12.793557+0000","last_became_active":"2024-04-25T10:05:10.454833+0000","last_became_peered":"2024-04-25T10:05:10.454833+0000","last_unstale":"2024-04-25T10:05:12.793557+0000","last_undegraded":"2024-04-25T10:05:12.793557+0000","last_fullsized":"2024-04-25T10:05:12.793557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.457011+0000","last_change":"2024-04-25T10:05:11.466533+0000","last_active":"2024-04-25T10:05:12.457011+0000","last_peered":"2024-04-25T10:05:12.457011+0000","last_clean":"2024-04-25T10:05:12.457011+0000","last_became_active":"2024-04-25T10:05:11.466256+0000","last_became_peered":"2024-04-25T10:05:11.466256+0000","last_unstale":"2024-04-25T10:05:12.457011+0000","last_undegraded":"2024-04-25T10:05:12.457011+0000","last_fullsized":"2024-04-25T10:05:12.457011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:41.766168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750989+0000","last_change":"2024-04-25T10:05:10.462808+0000","last_active":"2024-04-25T10:05:13.750989+0000","last_peered":"2024-04-25T10:05:13.750989+0000","last_clean":"2024-04-25T10:05:13.750989+0000","last_became_active":"2024-04-25T10:05:10.462421+0000","last_became_peered":"2024-04-25T10:05:10.462421+0000","last_unstale":"2024-04-25T10:05:13.750989+0000","last_undegraded":"2024-04-25T10:05:13.750989+0000","last_fullsized":"2024-04-25T10:05:13.750989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:55.136701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456680+0000","last_change":"2024-04-25T10:05:11.470381+0000","last_active":"2024-04-25T10:05:12.456680+0000","last_peered":"2024-04-25T10:05:12.456680+0000","last_clean":"2024-04-25T10:05:12.456680+0000","last_became_active":"2024-04-25T10:05:11.470031+0000","last_became_peered":"2024-04-25T10:05:11.470031+0000","last_unstale":"2024-04-25T10:05:12.456680+0000","last_undegraded":"2024-04-25T10:05:12.456680+0000","last_fullsized":"2024-04-25T10:05:12.456680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.799101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.448417+0000","last_change":"2024-04-25T10:05:10.461232+0000","last_active":"2024-04-25T10:05:13.448417+0000","last_peered":"2024-04-25T10:05:13.448417+0000","last_clean":"2024-04-25T10:05:13.448417+0000","last_became_active":"2024-04-25T10:05:10.460905+0000","last_became_peered":"2024-04-25T10:05:10.460905+0000","last_unstale":"2024-04-25T10:05:13.448417+0000","last_undegraded":"2024-04-25T10:05:13.448417+0000","last_fullsized":"2024-04-25T10:05:13.448417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:54.294853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750425+0000","last_change":"2024-04-25T10:05:11.472902+0000","last_active":"2024-04-25T10:05:13.750425+0000","last_peered":"2024-04-25T10:05:13.750425+0000","last_clean":"2024-04-25T10:05:13.750425+0000","last_became_active":"2024-04-25T10:05:11.472622+0000","last_became_peered":"2024-04-25T10:05:11.472622+0000","last_unstale":"2024-04-25T10:05:13.750425+0000","last_undegraded":"2024-04-25T10:05:13.750425+0000","last_fullsized":"2024-04-25T10:05:13.750425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:17.888835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456461+0000","last_change":"2024-04-25T10:05:10.460759+0000","last_active":"2024-04-25T10:05:12.456461+0000","last_peered":"2024-04-25T10:05:12.456461+0000","last_clean":"2024-04-25T10:05:12.456461+0000","last_became_active":"2024-04-25T10:05:10.460485+0000","last_became_peered":"2024-04-25T10:05:10.460485+0000","last_unstale":"2024-04-25T10:05:12.456461+0000","last_undegraded":"2024-04-25T10:05:12.456461+0000","last_fullsized":"2024-04-25T10:05:12.456461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456829+0000","last_change":"2024-04-25T10:05:11.470739+0000","last_active":"2024-04-25T10:05:12.456829+0000","last_peered":"2024-04-25T10:05:12.456829+0000","last_clean":"2024-04-25T10:05:12.456829+0000","last_became_active":"2024-04-25T10:05:11.470408+0000","last_became_peered":"2024-04-25T10:05:11.470408+0000","last_unstale":"2024-04-25T10:05:12.456829+0000","last_undegraded":"2024-04-25T10:05:12.456829+0000","last_fullsized":"2024-04-25T10:05:12.456829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:17.789806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.626301+0000","last_change":"2024-04-25T10:05:10.458496+0000","last_active":"2024-04-25T10:05:13.626301+0000","last_peered":"2024-04-25T10:05:13.626301+0000","last_clean":"2024-04-25T10:05:13.626301+0000","last_became_active":"2024-04-25T10:05:10.458168+0000","last_became_peered":"2024-04-25T10:05:10.458168+0000","last_unstale":"2024-04-25T10:05:13.626301+0000","last_undegraded":"2024-04-25T10:05:13.626301+0000","last_fullsized":"2024-04-25T10:05:13.626301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:41.255007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.649878+0000","last_change":"2024-04-25T10:05:11.459826+0000","last_active":"2024-04-25T10:05:13.649878+0000","last_peered":"2024-04-25T10:05:13.649878+0000","last_clean":"2024-04-25T10:05:13.649878+0000","last_became_active":"2024-04-25T10:05:11.459570+0000","last_became_peered":"2024-04-25T10:05:11.459570+0000","last_unstale":"2024-04-25T10:05:13.649878+0000","last_undegraded":"2024-04-25T10:05:13.649878+0000","last_fullsized":"2024-04-25T10:05:13.649878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750688+0000","last_change":"2024-04-25T10:05:10.455137+0000","last_active":"2024-04-25T10:05:13.750688+0000","last_peered":"2024-04-25T10:05:13.750688+0000","last_clean":"2024-04-25T10:05:13.750688+0000","last_became_active":"2024-04-25T10:05:10.454827+0000","last_became_peered":"2024-04-25T10:05:10.454827+0000","last_unstale":"2024-04-25T10:05:13.750688+0000","last_undegraded":"2024-04-25T10:05:13.750688+0000","last_fullsized":"2024-04-25T10:05:13.750688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:54.691904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750707+0000","last_change":"2024-04-25T10:05:11.469441+0000","last_active":"2024-04-25T10:05:13.750707+0000","last_peered":"2024-04-25T10:05:13.750707+0000","last_clean":"2024-04-25T10:05:13.750707+0000","last_became_active":"2024-04-25T10:05:11.468649+0000","last_became_peered":"2024-04-25T10:05:11.468649+0000","last_unstale":"2024-04-25T10:05:13.750707+0000","last_undegraded":"2024-04-25T10:05:13.750707+0000","last_fullsized":"2024-04-25T10:05:13.750707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:37.891605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.750592+0000","last_change":"2024-04-25T10:05:11.465690+0000","last_active":"2024-04-25T10:05:13.750592+0000","last_peered":"2024-04-25T10:05:13.750592+0000","last_clean":"2024-04-25T10:05:13.750592+0000","last_became_active":"2024-04-25T10:05:11.465422+0000","last_became_peered":"2024-04-25T10:05:11.465422+0000","last_unstale":"2024-04-25T10:05:13.750592+0000","last_undegraded":"2024-04-25T10:05:13.750592+0000","last_fullsized":"2024-04-25T10:05:13.750592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:02.099978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.449083+0000","last_change":"2024-04-25T10:05:10.450133+0000","last_active":"2024-04-25T10:05:13.449083+0000","last_peered":"2024-04-25T10:05:13.449083+0000","last_clean":"2024-04-25T10:05:13.449083+0000","last_became_active":"2024-04-25T10:05:10.449482+0000","last_became_peered":"2024-04-25T10:05:10.449482+0000","last_unstale":"2024-04-25T10:05:13.449083+0000","last_undegraded":"2024-04-25T10:05:13.449083+0000","last_fullsized":"2024-04-25T10:05:13.449083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:05.085522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.456402+0000","last_change":"2024-04-25T10:05:11.469301+0000","last_active":"2024-04-25T10:05:12.456402+0000","last_peered":"2024-04-25T10:05:12.456402+0000","last_clean":"2024-04-25T10:05:12.456402+0000","last_became_active":"2024-04-25T10:05:11.468980+0000","last_became_peered":"2024-04-25T10:05:11.468980+0000","last_unstale":"2024-04-25T10:05:12.456402+0000","last_undegraded":"2024-04-25T10:05:12.456402+0000","last_fullsized":"2024-04-25T10:05:12.456402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.449110+0000","last_change":"2024-04-25T10:05:10.456749+0000","last_active":"2024-04-25T10:05:13.449110+0000","last_peered":"2024-04-25T10:05:13.449110+0000","last_clean":"2024-04-25T10:05:13.449110+0000","last_became_active":"2024-04-25T10:05:10.456397+0000","last_became_peered":"2024-04-25T10:05:10.456397+0000","last_unstale":"2024-04-25T10:05:13.449110+0000","last_undegraded":"2024-04-25T10:05:13.449110+0000","last_fullsized":"2024-04-25T10:05:13.449110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:26.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786907+0000","last_change":"2024-04-25T10:05:11.453961+0000","last_active":"2024-04-25T10:05:14.786907+0000","last_peered":"2024-04-25T10:05:14.786907+0000","last_clean":"2024-04-25T10:05:14.786907+0000","last_became_active":"2024-04-25T10:05:11.453443+0000","last_became_peered":"2024-04-25T10:05:11.453443+0000","last_unstale":"2024-04-25T10:05:14.786907+0000","last_undegraded":"2024-04-25T10:05:14.786907+0000","last_fullsized":"2024-04-25T10:05:14.786907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:06.235586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786886+0000","last_change":"2024-04-25T10:05:10.458501+0000","last_active":"2024-04-25T10:05:14.786886+0000","last_peered":"2024-04-25T10:05:14.786886+0000","last_clean":"2024-04-25T10:05:14.786886+0000","last_became_active":"2024-04-25T10:05:10.457915+0000","last_became_peered":"2024-04-25T10:05:10.457915+0000","last_unstale":"2024-04-25T10:05:14.786886+0000","last_undegraded":"2024-04-25T10:05:14.786886+0000","last_fullsized":"2024-04-25T10:05:14.786886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:15.560541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:14.786839+0000","last_change":"2024-04-25T10:05:11.451812+0000","last_active":"2024-04-25T10:05:14.786839+0000","last_peered":"2024-04-25T10:05:14.786839+0000","last_clean":"2024-04-25T10:05:14.786839+0000","last_became_active":"2024-04-25T10:05:11.451318+0000","last_became_peered":"2024-04-25T10:05:11.451318+0000","last_unstale":"2024-04-25T10:05:14.786839+0000","last_undegraded":"2024-04-25T10:05:14.786839+0000","last_fullsized":"2024-04-25T10:05:14.786839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:10.434972+0000","last_clean_scrub_stamp":"2024-04-25T10:05:10.434972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:28.693266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:12.457002+0000","last_change":"2024-04-25T10:05:10.454280+0000","last_active":"2024-04-25T10:05:12.457002+0000","last_peered":"2024-04-25T10:05:12.457002+0000","last_clean":"2024-04-25T10:05:12.457002+0000","last_became_active":"2024-04-25T10:05:10.453982+0000","last_became_peered":"2024-04-25T10:05:10.453982+0000","last_unstale":"2024-04-25T10:05:12.457002+0000","last_undegraded":"2024-04-25T10:05:12.457002+0000","last_fullsized":"2024-04-25T10:05:12.457002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:09.429642+0000","last_clean_scrub_stamp":"2024-04-25T10:05:09.429642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:04.867276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":76688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":76564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":78848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":225280,"data_stored":77146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":233472,"data_stored":78176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":667002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":229376,"data_stored":76620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":667024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:23.637 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:05:23.637 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T10:05:23.637 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T10:05:23.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:05:23.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.768+0000 7f775d5f4640 1 Processor -- start 2024-04-25T10:05:23.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.768+0000 7f775d5f4640 1 -- start start 2024-04-25T10:05:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.769+0000 7f775d5f4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f77580d35b0 0x7f77580b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.769+0000 7f775d5f4640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f77580b5990 0x7f77580b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.769+0000 7f775d5f4640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f775815e0c0 con 0x7f775815dce0 2024-04-25T10:05:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.769+0000 7f775d5f4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77580d2510 con 0x7f77580d35b0 2024-04-25T10:05:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.769+0000 7f775d5f4640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77580d27f0 con 0x7f77580b5990 2024-04-25T10:05:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.769+0000 7f774ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f77580d35b0 0x7f77580b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.769+0000 7f774ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f77580d35b0 0x7f77580b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56210/0 (socket says 172.21.15.16:56210) 2024-04-25T10:05:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.769+0000 7f774ffff640 1 -- 172.21.15.16:0/1905975221 learned_addr learned my addr 172.21.15.16:0/1905975221 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.769+0000 7f77577fe640 1 --2- 172.21.15.16:0/1905975221 >> v2:172.21.15.186:3300/0 conn(0x7f77580b5990 0x7f77580b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.769+0000 7f774ffff640 1 -- 172.21.15.16:0/1905975221 >> v2:172.21.15.186:3300/0 conn(0x7f77580b5990 msgr2=0x7f77580b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.769+0000 7f774ffff640 1 --2- 172.21.15.16:0/1905975221 >> v2:172.21.15.186:3300/0 conn(0x7f77580b5990 0x7f77580b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.769+0000 7f774ffff640 1 -- 172.21.15.16:0/1905975221 >> v1:172.21.15.186:6789/0 conn(0x7f775815dce0 legacy=0x7f77580d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.769+0000 7f774ffff640 1 -- 172.21.15.16:0/1905975221 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77580d2ad0 con 0x7f77580d35b0 2024-04-25T10:05:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.770+0000 7f774ffff640 1 --2- 172.21.15.16:0/1905975221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77580d35b0 0x7f77580b5450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f77580b3390 tx=0x7f774002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd930e8d0cf76d0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.770+0000 7f77567fc640 1 -- 172.21.15.16:0/1905975221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7740037070 con 0x7f77580d35b0 2024-04-25T10:05:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.770+0000 7f77567fc640 1 -- 172.21.15.16:0/1905975221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7740032440 con 0x7f77580d35b0 2024-04-25T10:05:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.770+0000 7f77567fc640 1 -- 172.21.15.16:0/1905975221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7740032810 con 0x7f77580d35b0 2024-04-25T10:05:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.771+0000 7f775d5f4640 1 -- 172.21.15.16:0/1905975221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77580d35b0 msgr2=0x7f77580b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.771+0000 7f775d5f4640 1 --2- 172.21.15.16:0/1905975221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77580d35b0 0x7f77580b5450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f77580b3390 tx=0x7f774002a230 comp rx=0 tx=0).stop 2024-04-25T10:05:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.771+0000 7f775d5f4640 1 -- 172.21.15.16:0/1905975221 shutdown_connections 2024-04-25T10:05:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.771+0000 7f775d5f4640 1 --2- 172.21.15.16:0/1905975221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77580d35b0 0x7f77580b5450 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.771+0000 7f775d5f4640 1 --2- 172.21.15.16:0/1905975221 >> v2:172.21.15.186:3300/0 conn(0x7f77580b5990 0x7f77580b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.771+0000 7f775d5f4640 1 -- 172.21.15.16:0/1905975221 >> 172.21.15.16:0/1905975221 conn(0x7f7758072900 msgr2=0x7f77580d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.771+0000 7f775d5f4640 1 -- 172.21.15.16:0/1905975221 shutdown_connections 2024-04-25T10:05:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.771+0000 7f775d5f4640 1 -- 172.21.15.16:0/1905975221 wait complete. 2024-04-25T10:05:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.772+0000 7f775d5f4640 1 Processor -- start 2024-04-25T10:05:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.772+0000 7f775d5f4640 1 -- start start 2024-04-25T10:05:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.772+0000 7f775d5f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77580b5990 0x7f77580cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.772+0000 7f775d5f4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77580d35b0 0x7f77580cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f775d5f4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775815dce0 0x7f775815f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f775d5f4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77580d2160 con 0x7f77580b5990 2024-04-25T10:05:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f775d5f4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77580d3090 con 0x7f775815dce0 2024-04-25T10:05:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f775d5f4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77581ede40 con 0x7f77580d35b0 2024-04-25T10:05:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f7756ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77580b5990 0x7f77580cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f7756ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77580b5990 0x7f77580cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56226/0 (socket says 172.21.15.16:56226) 2024-04-25T10:05:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f7756ffd640 1 -- 172.21.15.16:0/1487430273 learned_addr learned my addr 172.21.15.16:0/1487430273 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f77577fe640 1 --2- 172.21.15.16:0/1487430273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775815dce0 0x7f775815f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f774ffff640 1 --2- 172.21.15.16:0/1487430273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77580d35b0 0x7f77580cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f7756ffd640 1 -- 172.21.15.16:0/1487430273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77580d35b0 msgr2=0x7f77580cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f7756ffd640 1 --2- 172.21.15.16:0/1487430273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77580d35b0 0x7f77580cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f7756ffd640 1 -- 172.21.15.16:0/1487430273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775815dce0 msgr2=0x7f775815f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f7756ffd640 1 --2- 172.21.15.16:0/1487430273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775815dce0 0x7f775815f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f7756ffd640 1 -- 172.21.15.16:0/1487430273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77580cbf30 con 0x7f77580b5990 2024-04-25T10:05:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f7756ffd640 1 --2- 172.21.15.16:0/1487430273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77580b5990 0x7f77580cb4b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f774800b3f0 tx=0x7f774800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f774f7fe640 1 -- 172.21.15.16:0/1487430273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748015020 con 0x7f77580b5990 2024-04-25T10:05:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f774f7fe640 1 -- 172.21.15.16:0/1487430273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7748011830 con 0x7f77580b5990 2024-04-25T10:05:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f775d5f4640 1 -- 172.21.15.16:0/1487430273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77580cc1e0 con 0x7f77580b5990 2024-04-25T10:05:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f774f7fe640 1 -- 172.21.15.16:0/1487430273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748011c20 con 0x7f77580b5990 2024-04-25T10:05:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.773+0000 7f775d5f4640 1 -- 172.21.15.16:0/1487430273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77581f4f40 con 0x7f77580b5990 2024-04-25T10:05:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.774+0000 7f775d5f4640 1 -- 172.21.15.16:0/1487430273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77580749b0 con 0x7f77580b5990 2024-04-25T10:05:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.775+0000 7f774f7fe640 1 -- 172.21.15.16:0/1487430273 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7748018020 con 0x7f77580b5990 2024-04-25T10:05:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.775+0000 7f774f7fe640 1 --2- 172.21.15.16:0/1487430273 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f773806b4d0 0x7f773806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.776+0000 7f774f7fe640 1 -- 172.21.15.16:0/1487430273 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7748089d70 con 0x7f77580b5990 2024-04-25T10:05:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.776+0000 7f774ffff640 1 --2- 172.21.15.16:0/1487430273 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f773806b4d0 0x7f773806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.776+0000 7f774ffff640 1 --2- 172.21.15.16:0/1487430273 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f773806b4d0 0x7f773806d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7740034000 tx=0x7f774002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:23.780+0000 7f774f7fe640 1 -- 172.21.15.16:0/1487430273 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f774801f030 con 0x7f77580b5990 2024-04-25T10:05:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.000+0000 7f775d5f4640 1 -- 172.21.15.16:0/1487430273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f77580b3430 con 0x7f77580b5990 2024-04-25T10:05:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.000+0000 7f774f7fe640 1 -- 172.21.15.16:0/1487430273 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f77480172c0 con 0x7f77580b5990 2024-04-25T10:05:24.005 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:24.006 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:05:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.002+0000 7f775d5f4640 1 -- 172.21.15.16:0/1487430273 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f773806b4d0 msgr2=0x7f773806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.002+0000 7f775d5f4640 1 --2- 172.21.15.16:0/1487430273 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f773806b4d0 0x7f773806d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7740034000 tx=0x7f774002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.002+0000 7f775d5f4640 1 -- 172.21.15.16:0/1487430273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77580b5990 msgr2=0x7f77580cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.002+0000 7f775d5f4640 1 --2- 172.21.15.16:0/1487430273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77580b5990 0x7f77580cb4b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f774800b3f0 tx=0x7f774800b8b0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.002+0000 7f775d5f4640 1 -- 172.21.15.16:0/1487430273 shutdown_connections 2024-04-25T10:05:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.002+0000 7f775d5f4640 1 --2- 172.21.15.16:0/1487430273 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f773806b4d0 0x7f773806d990 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.002+0000 7f775d5f4640 1 --2- 172.21.15.16:0/1487430273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f775815dce0 0x7f775815f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.002+0000 7f775d5f4640 1 --2- 172.21.15.16:0/1487430273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77580d35b0 0x7f77580cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.002+0000 7f775d5f4640 1 --2- 172.21.15.16:0/1487430273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77580b5990 0x7f77580cb4b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.002+0000 7f775d5f4640 1 -- 172.21.15.16:0/1487430273 >> 172.21.15.16:0/1487430273 conn(0x7f7758072900 msgr2=0x7f77580d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.002+0000 7f775d5f4640 1 -- 172.21.15.16:0/1487430273 shutdown_connections 2024-04-25T10:05:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.002+0000 7f775d5f4640 1 -- 172.21.15.16:0/1487430273 wait complete. 2024-04-25T10:05:24.018 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T10:05:24.018 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:24.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.113+0000 7f9bdff34640 1 Processor -- start 2024-04-25T10:05:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.114+0000 7f9bdff34640 1 -- start start 2024-04-25T10:05:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.114+0000 7f9bdff34640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9bd815dce0 0x7f9bd80d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.114+0000 7f9bdff34640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9bd80b5b60 0x7f9bd80b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.114+0000 7f9bdff34640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd815e0c0 con 0x7f9bd80d35b0 2024-04-25T10:05:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.114+0000 7f9bdff34640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd80d2510 con 0x7f9bd815dce0 2024-04-25T10:05:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.114+0000 7f9bdff34640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd80d27f0 con 0x7f9bd80b5b60 2024-04-25T10:05:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.114+0000 7f9bddcd0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9bd815dce0 0x7f9bd80d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.115+0000 7f9bdd4cf640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9bd80d35b0 0x7f9bd80b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35180/0 (socket says 172.21.15.16:35180) 2024-04-25T10:05:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.115+0000 7f9bdd4cf640 1 -- 172.21.15.16:0/3834628351 learned_addr learned my addr 172.21.15.16:0/3834628351 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.115+0000 7f9bde4d1640 1 --2- 172.21.15.16:0/3834628351 >> v2:172.21.15.186:3300/0 conn(0x7f9bd80b5b60 0x7f9bd80b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.115+0000 7f9bddcd0640 1 -- 172.21.15.16:0/3834628351 >> v2:172.21.15.186:3300/0 conn(0x7f9bd80b5b60 msgr2=0x7f9bd80b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.115+0000 7f9bddcd0640 1 --2- 172.21.15.16:0/3834628351 >> v2:172.21.15.186:3300/0 conn(0x7f9bd80b5b60 0x7f9bd80b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.115+0000 7f9bddcd0640 1 -- 172.21.15.16:0/3834628351 >> v1:172.21.15.16:6789/0 conn(0x7f9bd80d35b0 legacy=0x7f9bd80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.115+0000 7f9bdccce640 1 -- 172.21.15.16:0/3834628351 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361238379 0 0) 0x7f9bd815e0c0 con 0x7f9bd80d35b0 2024-04-25T10:05:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.115+0000 7f9bddcd0640 1 -- 172.21.15.16:0/3834628351 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd80d2ad0 con 0x7f9bd815dce0 2024-04-25T10:05:24.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.115+0000 7f9bddcd0640 1 --2- 172.21.15.16:0/3834628351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd815dce0 0x7f9bd80d1a50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80b3390 tx=0x7f9bc802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c842e8042013f7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.116+0000 7f9bdccce640 1 -- 172.21.15.16:0/3834628351 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8007740 con 0x7f9bd815dce0 2024-04-25T10:05:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.116+0000 7f9bdccce640 1 -- 172.21.15.16:0/3834628351 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc80078e0 con 0x7f9bd815dce0 2024-04-25T10:05:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.116+0000 7f9bdccce640 1 -- 172.21.15.16:0/3834628351 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8007cb0 con 0x7f9bd815dce0 2024-04-25T10:05:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.116+0000 7f9bdff34640 1 -- 172.21.15.16:0/3834628351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd815dce0 msgr2=0x7f9bd80d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.116+0000 7f9bdff34640 1 --2- 172.21.15.16:0/3834628351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd815dce0 0x7f9bd80d1a50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80b3390 tx=0x7f9bc802a680 comp rx=0 tx=0).stop 2024-04-25T10:05:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.116+0000 7f9bdff34640 1 -- 172.21.15.16:0/3834628351 shutdown_connections 2024-04-25T10:05:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.116+0000 7f9bdff34640 1 --2- 172.21.15.16:0/3834628351 >> v2:172.21.15.186:3300/0 conn(0x7f9bd80b5b60 0x7f9bd80b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.116+0000 7f9bdff34640 1 --2- 172.21.15.16:0/3834628351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd815dce0 0x7f9bd80d1a50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.116+0000 7f9bdff34640 1 -- 172.21.15.16:0/3834628351 >> 172.21.15.16:0/3834628351 conn(0x7f9bd8072900 msgr2=0x7f9bd80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.116+0000 7f9bdff34640 1 -- 172.21.15.16:0/3834628351 shutdown_connections 2024-04-25T10:05:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.116+0000 7f9bdff34640 1 -- 172.21.15.16:0/3834628351 wait complete. 2024-04-25T10:05:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.117+0000 7f9bdff34640 1 Processor -- start 2024-04-25T10:05:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.117+0000 7f9bdff34640 1 -- start start 2024-04-25T10:05:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.118+0000 7f9bdff34640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bd80b5b60 0x7f9bd814bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.118+0000 7f9bdff34640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bd80d35b0 0x7f9bd80abe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.118+0000 7f9bdff34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd815dce0 0x7f9bd80ac370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.118+0000 7f9bdff34640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80b9820 con 0x7f9bd815dce0 2024-04-25T10:05:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.118+0000 7f9bdff34640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80d3090 con 0x7f9bd80b5b60 2024-04-25T10:05:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.118+0000 7f9bdff34640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80b99a0 con 0x7f9bd80d35b0 2024-04-25T10:05:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.118+0000 7f9bde4d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd815dce0 0x7f9bd80ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.118+0000 7f9bde4d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd815dce0 0x7f9bd80ac370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56234/0 (socket says 172.21.15.16:56234) 2024-04-25T10:05:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.118+0000 7f9bde4d1640 1 -- 172.21.15.16:0/3362699150 learned_addr learned my addr 172.21.15.16:0/3362699150 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.118+0000 7f9bdd4cf640 1 --2- 172.21.15.16:0/3362699150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bd80d35b0 0x7f9bd80abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.118+0000 7f9bddcd0640 1 --2- 172.21.15.16:0/3362699150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bd80b5b60 0x7f9bd814bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.119+0000 7f9bde4d1640 1 -- 172.21.15.16:0/3362699150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bd80d35b0 msgr2=0x7f9bd80abe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.119+0000 7f9bde4d1640 1 --2- 172.21.15.16:0/3362699150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bd80d35b0 0x7f9bd80abe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.119+0000 7f9bde4d1640 1 -- 172.21.15.16:0/3362699150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bd80b5b60 msgr2=0x7f9bd814bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.119+0000 7f9bde4d1640 1 --2- 172.21.15.16:0/3362699150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bd80b5b60 0x7f9bd814bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.119+0000 7f9bde4d1640 1 -- 172.21.15.16:0/3362699150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd80ab0a0 con 0x7f9bd815dce0 2024-04-25T10:05:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.119+0000 7f9bddcd0640 1 --2- 172.21.15.16:0/3362699150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bd80b5b60 0x7f9bd814bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.119+0000 7f9bde4d1640 1 --2- 172.21.15.16:0/3362699150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd815dce0 0x7f9bd80ac370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000c7f0 tx=0x7f9bd000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.119+0000 7f9bd67fc640 1 -- 172.21.15.16:0/3362699150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0015070 con 0x7f9bd815dce0 2024-04-25T10:05:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.119+0000 7f9bd67fc640 1 -- 172.21.15.16:0/3362699150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd0004220 con 0x7f9bd815dce0 2024-04-25T10:05:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.119+0000 7f9bdff34640 1 -- 172.21.15.16:0/3362699150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd80ab380 con 0x7f9bd815dce0 2024-04-25T10:05:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.119+0000 7f9bd67fc640 1 -- 172.21.15.16:0/3362699150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0007620 con 0x7f9bd815dce0 2024-04-25T10:05:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.119+0000 7f9bdff34640 1 -- 172.21.15.16:0/3362699150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd80ab940 con 0x7f9bd815dce0 2024-04-25T10:05:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.120+0000 7f9bdff34640 1 -- 172.21.15.16:0/3362699150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd80749b0 con 0x7f9bd815dce0 2024-04-25T10:05:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.120+0000 7f9bd67fc640 1 -- 172.21.15.16:0/3362699150 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9bd00079d0 con 0x7f9bd815dce0 2024-04-25T10:05:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.121+0000 7f9bd67fc640 1 --2- 172.21.15.16:0/3362699150 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9bc406b4d0 0x7f9bc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.121+0000 7f9bd67fc640 1 -- 172.21.15.16:0/3362699150 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9bd008a350 con 0x7f9bd815dce0 2024-04-25T10:05:24.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.121+0000 7f9bddcd0640 1 --2- 172.21.15.16:0/3362699150 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9bc406b4d0 0x7f9bc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.123+0000 7f9bddcd0640 1 --2- 172.21.15.16:0/3362699150 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9bc406b4d0 0x7f9bc406d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8034000 tx=0x7f9bc8033040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.125+0000 7f9bd67fc640 1 -- 172.21.15.16:0/3362699150 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9bd0059150 con 0x7f9bd815dce0 2024-04-25T10:05:24.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.306+0000 7f9bdff34640 1 -- 172.21.15.16:0/3362699150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bd80cf020 con 0x7f9bd815dce0 2024-04-25T10:05:24.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.307+0000 7f9bd67fc640 1 -- 172.21.15.16:0/3362699150 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f9bd0058a50 con 0x7f9bd815dce0 2024-04-25T10:05:24.313 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:24.313 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:12.449183+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.309+0000 7f9bdff34640 1 -- 172.21.15.16:0/3362699150 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9bc406b4d0 msgr2=0x7f9bc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.309+0000 7f9bdff34640 1 --2- 172.21.15.16:0/3362699150 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9bc406b4d0 0x7f9bc406d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8034000 tx=0x7f9bc8033040 comp rx=0 tx=0).stop 2024-04-25T10:05:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.309+0000 7f9bdff34640 1 -- 172.21.15.16:0/3362699150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd815dce0 msgr2=0x7f9bd80ac370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.309+0000 7f9bdff34640 1 --2- 172.21.15.16:0/3362699150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd815dce0 0x7f9bd80ac370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000c7f0 tx=0x7f9bd000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.309+0000 7f9bdff34640 1 -- 172.21.15.16:0/3362699150 shutdown_connections 2024-04-25T10:05:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.309+0000 7f9bdff34640 1 --2- 172.21.15.16:0/3362699150 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9bc406b4d0 0x7f9bc406d990 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.309+0000 7f9bdff34640 1 --2- 172.21.15.16:0/3362699150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd815dce0 0x7f9bd80ac370 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.309+0000 7f9bdff34640 1 --2- 172.21.15.16:0/3362699150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bd80d35b0 0x7f9bd80abe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.309+0000 7f9bdff34640 1 --2- 172.21.15.16:0/3362699150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bd80b5b60 0x7f9bd814bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.309+0000 7f9bdff34640 1 -- 172.21.15.16:0/3362699150 >> 172.21.15.16:0/3362699150 conn(0x7f9bd8072900 msgr2=0x7f9bd80b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.309+0000 7f9bdff34640 1 -- 172.21.15.16:0/3362699150 shutdown_connections 2024-04-25T10:05:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.309+0000 7f9bdff34640 1 -- 172.21.15.16:0/3362699150 wait complete. 2024-04-25T10:05:24.325 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:08.724537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:09.793105+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:24.326 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:24.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.424+0000 7fcefc116640 1 Processor -- start 2024-04-25T10:05:24.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.424+0000 7fcefc116640 1 -- start start 2024-04-25T10:05:24.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.425+0000 7fcefc116640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef415dca0 0x7fcef40ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.425+0000 7fcefc116640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcef40d1130 0x7fcef40b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.425+0000 7fcefc116640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef40d4090 con 0x7fcef40d3b90 2024-04-25T10:05:24.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.425+0000 7fcefc116640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef40b6b20 con 0x7fcef40d1130 2024-04-25T10:05:24.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.425+0000 7fcefc116640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef40d4370 con 0x7fcef415dca0 2024-04-25T10:05:24.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.425+0000 7fcefa6b3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcef40d1130 0x7fcef40b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.425+0000 7fcefa6b3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcef40d1130 0x7fcef40b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56238/0 (socket says 172.21.15.16:56238) 2024-04-25T10:05:24.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.425+0000 7fcefa6b3640 1 -- 172.21.15.16:0/639143562 learned_addr learned my addr 172.21.15.16:0/639143562 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:24.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.425+0000 7fcef9eb2640 1 --2- 172.21.15.16:0/639143562 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef415dca0 0x7fcef40ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.425+0000 7fcefa6b3640 1 -- 172.21.15.16:0/639143562 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef415dca0 msgr2=0x7fcef40ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.425+0000 7fcefa6b3640 1 --2- 172.21.15.16:0/639143562 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef415dca0 0x7fcef40ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.425+0000 7fcefa6b3640 1 -- 172.21.15.16:0/639143562 >> v1:172.21.15.186:6789/0 conn(0x7fcef40d3b90 legacy=0x7fcef40d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.425+0000 7fcefa6b3640 1 -- 172.21.15.16:0/639143562 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef40d4650 con 0x7fcef40d1130 2024-04-25T10:05:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.426+0000 7fcefa6b3640 1 --2- 172.21.15.16:0/639143562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef40d1130 0x7fcef40b2500 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fceec009e50 tx=0x7fceec02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6336781e0d37c0af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.426+0000 7fcef8eb0640 1 -- 172.21.15.16:0/639143562 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec036070 con 0x7fcef40d1130 2024-04-25T10:05:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.426+0000 7fcef8eb0640 1 -- 172.21.15.16:0/639143562 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec031600 con 0x7fcef40d1130 2024-04-25T10:05:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.426+0000 7fcef8eb0640 1 -- 172.21.15.16:0/639143562 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0319d0 con 0x7fcef40d1130 2024-04-25T10:05:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.427+0000 7fcefc116640 1 -- 172.21.15.16:0/639143562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef40d1130 msgr2=0x7fcef40b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.427+0000 7fcefc116640 1 --2- 172.21.15.16:0/639143562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef40d1130 0x7fcef40b2500 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fceec009e50 tx=0x7fceec02a6c0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.427+0000 7fcefc116640 1 -- 172.21.15.16:0/639143562 shutdown_connections 2024-04-25T10:05:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.427+0000 7fcefc116640 1 --2- 172.21.15.16:0/639143562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef40d1130 0x7fcef40b2500 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.427+0000 7fcefc116640 1 --2- 172.21.15.16:0/639143562 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef415dca0 0x7fcef40ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.427+0000 7fcefc116640 1 -- 172.21.15.16:0/639143562 >> 172.21.15.16:0/639143562 conn(0x7fcef40726c0 msgr2=0x7fcef4072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.427+0000 7fcefc116640 1 -- 172.21.15.16:0/639143562 shutdown_connections 2024-04-25T10:05:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.427+0000 7fcefc116640 1 -- 172.21.15.16:0/639143562 wait complete. 2024-04-25T10:05:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.428+0000 7fcefc116640 1 Processor -- start 2024-04-25T10:05:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.428+0000 7fcefc116640 1 -- start start 2024-04-25T10:05:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.429+0000 7fcefc116640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef40d1130 0x7fcef4149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.429+0000 7fcefc116640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcef40d3b90 0x7fcef414c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.429+0000 7fcefc116640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef415dca0 0x7fcef40cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.429+0000 7fcefc116640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef415e200 con 0x7fcef40d1130 2024-04-25T10:05:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.429+0000 7fcefc116640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef415e080 con 0x7fcef40d3b90 2024-04-25T10:05:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.429+0000 7fcefc116640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef40cee80 con 0x7fcef415dca0 2024-04-25T10:05:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.429+0000 7fcef9eb2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef40d1130 0x7fcef4149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.429+0000 7fcef9eb2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef40d1130 0x7fcef4149960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56244/0 (socket says 172.21.15.16:56244) 2024-04-25T10:05:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.429+0000 7fcef9eb2640 1 -- 172.21.15.16:0/2729498978 learned_addr learned my addr 172.21.15.16:0/2729498978 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.429+0000 7fcefa6b3640 1 --2- 172.21.15.16:0/2729498978 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef415dca0 0x7fcef40cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.429+0000 7fcef96b1640 1 --2- 172.21.15.16:0/2729498978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcef40d3b90 0x7fcef414c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.430+0000 7fcef9eb2640 1 -- 172.21.15.16:0/2729498978 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef415dca0 msgr2=0x7fcef40cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.430+0000 7fcef9eb2640 1 --2- 172.21.15.16:0/2729498978 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef415dca0 0x7fcef40cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.430+0000 7fcef9eb2640 1 -- 172.21.15.16:0/2729498978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcef40d3b90 msgr2=0x7fcef414c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.430+0000 7fcef9eb2640 1 --2- 172.21.15.16:0/2729498978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcef40d3b90 0x7fcef414c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.430+0000 7fcef9eb2640 1 -- 172.21.15.16:0/2729498978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef4159380 con 0x7fcef40d1130 2024-04-25T10:05:24.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.430+0000 7fcef9eb2640 1 --2- 172.21.15.16:0/2729498978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef40d1130 0x7fcef4149960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcef000d5f0 tx=0x7fcef000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.430+0000 7fceea7fc640 1 -- 172.21.15.16:0/2729498978 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef00125f0 con 0x7fcef40d1130 2024-04-25T10:05:24.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.430+0000 7fceea7fc640 1 -- 172.21.15.16:0/2729498978 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef0019070 con 0x7fcef40d1130 2024-04-25T10:05:24.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.430+0000 7fceea7fc640 1 -- 172.21.15.16:0/2729498978 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef00129d0 con 0x7fcef40d1130 2024-04-25T10:05:24.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.430+0000 7fcefc116640 1 -- 172.21.15.16:0/2729498978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef40cad70 con 0x7fcef40d1130 2024-04-25T10:05:24.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.430+0000 7fcefc116640 1 -- 172.21.15.16:0/2729498978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef414c840 con 0x7fcef40d1130 2024-04-25T10:05:24.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.431+0000 7fcefc116640 1 -- 172.21.15.16:0/2729498978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef40750b0 con 0x7fcef40d1130 2024-04-25T10:05:24.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.431+0000 7fceea7fc640 1 -- 172.21.15.16:0/2729498978 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcef0012b70 con 0x7fcef40d1130 2024-04-25T10:05:24.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.432+0000 7fceea7fc640 1 --2- 172.21.15.16:0/2729498978 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fced806b4d0 0x7fced806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.432+0000 7fceea7fc640 1 -- 172.21.15.16:0/2729498978 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcef008a420 con 0x7fcef40d1130 2024-04-25T10:05:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.432+0000 7fcef96b1640 1 --2- 172.21.15.16:0/2729498978 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fced806b4d0 0x7fced806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.433+0000 7fcef96b1640 1 --2- 172.21.15.16:0/2729498978 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fced806b4d0 0x7fced806d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcef41f7070 tx=0x7fcee4007e30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.436+0000 7fceea7fc640 1 -- 172.21.15.16:0/2729498978 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcef0058ce0 con 0x7fcef40d1130 2024-04-25T10:05:24.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.615+0000 7fcefc116640 1 -- 172.21.15.16:0/2729498978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcef40b8400 con 0x7fcef40d1130 2024-04-25T10:05:24.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.615+0000 7fceea7fc640 1 -- 172.21.15.16:0/2729498978 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcef005cd80 con 0x7fcef40d1130 2024-04-25T10:05:24.620 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:05:24.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.617+0000 7fcefc116640 1 -- 172.21.15.16:0/2729498978 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fced806b4d0 msgr2=0x7fced806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.617+0000 7fcefc116640 1 --2- 172.21.15.16:0/2729498978 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fced806b4d0 0x7fced806d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcef41f7070 tx=0x7fcee4007e30 comp rx=0 tx=0).stop 2024-04-25T10:05:24.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.617+0000 7fcefc116640 1 -- 172.21.15.16:0/2729498978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef40d1130 msgr2=0x7fcef4149960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.617+0000 7fcefc116640 1 --2- 172.21.15.16:0/2729498978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef40d1130 0x7fcef4149960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcef000d5f0 tx=0x7fcef000dab0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.617+0000 7fcefc116640 1 -- 172.21.15.16:0/2729498978 shutdown_connections 2024-04-25T10:05:24.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.617+0000 7fcefc116640 1 --2- 172.21.15.16:0/2729498978 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fced806b4d0 0x7fced806d990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.617+0000 7fcefc116640 1 --2- 172.21.15.16:0/2729498978 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef415dca0 0x7fcef40cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.617+0000 7fcefc116640 1 --2- 172.21.15.16:0/2729498978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcef40d3b90 0x7fcef414c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.617+0000 7fcefc116640 1 --2- 172.21.15.16:0/2729498978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef40d1130 0x7fcef4149960 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.617+0000 7fcefc116640 1 -- 172.21.15.16:0/2729498978 >> 172.21.15.16:0/2729498978 conn(0x7fcef40726c0 msgr2=0x7fcef40d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.617+0000 7fcefc116640 1 -- 172.21.15.16:0/2729498978 shutdown_connections 2024-04-25T10:05:24.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.617+0000 7fcefc116640 1 -- 172.21.15.16:0/2729498978 wait complete. 2024-04-25T10:05:24.633 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:24.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.731+0000 7f3a78aee640 1 Processor -- start 2024-04-25T10:05:24.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.731+0000 7f3a78aee640 1 -- start start 2024-04-25T10:05:24.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.732+0000 7f3a78aee640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a7415df30 0x7f3a740cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.732+0000 7f3a78aee640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a740d0200 con 0x7f3a7415db50 2024-04-25T10:05:24.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.732+0000 7f3a78aee640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a740d04e0 con 0x7f3a740d12a0 2024-04-25T10:05:24.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.732+0000 7f3a78aee640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7406c020 con 0x7f3a7415df30 2024-04-25T10:05:24.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.732+0000 7f3a7259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3a7415db50 0x7f3a7406cba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35188/0 (socket says 172.21.15.16:35188) 2024-04-25T10:05:24.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.732+0000 7f3a7259c640 1 -- 172.21.15.16:0/3471359787 learned_addr learned my addr 172.21.15.16:0/3471359787 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:24.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a71d9b640 1 --2- 172.21.15.16:0/3471359787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a7415df30 0x7f3a740cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a7159a640 1 -- 172.21.15.16:0/3471359787 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2823448285 0 0) 0x7f3a740d0200 con 0x7f3a7415db50 2024-04-25T10:05:24.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a7159a640 1 -- 172.21.15.16:0/3471359787 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a60003520 con 0x7f3a7415db50 2024-04-25T10:05:24.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a7159a640 1 -- 172.21.15.16:0/3471359787 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2082036343 0 0) 0x7f3a60003520 con 0x7f3a7415db50 2024-04-25T10:05:24.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a7159a640 1 -- 172.21.15.16:0/3471359787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a7415df30 msgr2=0x7f3a740cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a7159a640 1 --2- 172.21.15.16:0/3471359787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a7415df30 0x7f3a740cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a7159a640 1 -- 172.21.15.16:0/3471359787 >> v1:172.21.15.186:6789/0 conn(0x7f3a740d12a0 legacy=0x7f3a74099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a7159a640 1 -- 172.21.15.16:0/3471359787 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a740d07c0 con 0x7f3a7415db50 2024-04-25T10:05:24.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a7159a640 1 -- 172.21.15.16:0/3471359787 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f3a6c001670 con 0x7f3a7415db50 2024-04-25T10:05:24.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a7159a640 1 -- 172.21.15.16:0/3471359787 >> v1:172.21.15.16:6789/0 conn(0x7f3a7415db50 legacy=0x7f3a7406cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a7159a640 1 --2- 172.21.15.16:0/3471359787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a600039e0 0x7f3a60003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a7159a640 1 --2- 172.21.15.16:0/3471359787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a60004340 0x7f3a60004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a7159a640 1 --2- 172.21.15.16:0/3471359787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a60005020 0x7f3a6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a7159a640 1 -- 172.21.15.16:0/3471359787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7406c020 con 0x7f3a60005020 2024-04-25T10:05:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a7159a640 1 -- 172.21.15.16:0/3471359787 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6000bf60 con 0x7f3a60004340 2024-04-25T10:05:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a7159a640 1 -- 172.21.15.16:0/3471359787 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6000c160 con 0x7f3a600039e0 2024-04-25T10:05:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a71d9b640 1 --2- 172.21.15.16:0/3471359787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a7415df30 0x7f3a740cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a72d9d640 1 --2- 172.21.15.16:0/3471359787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a60005020 0x7f3a6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a7259c640 1 --2- 172.21.15.16:0/3471359787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a600039e0 0x7f3a60003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a71d9b640 1 --2- 172.21.15.16:0/3471359787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a60004340 0x7f3a60004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a72d9d640 1 -- 172.21.15.16:0/3471359787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a600039e0 msgr2=0x7f3a60003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a72d9d640 1 --2- 172.21.15.16:0/3471359787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a600039e0 0x7f3a60003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a72d9d640 1 -- 172.21.15.16:0/3471359787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a60004340 msgr2=0x7f3a60004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a72d9d640 1 --2- 172.21.15.16:0/3471359787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a60004340 0x7f3a60004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a72d9d640 1 -- 172.21.15.16:0/3471359787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a60003520 con 0x7f3a60005020 2024-04-25T10:05:24.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.733+0000 7f3a7259c640 1 --2- 172.21.15.16:0/3471359787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a600039e0 0x7f3a60003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:24.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.734+0000 7f3a72d9d640 1 --2- 172.21.15.16:0/3471359787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a60005020 0x7f3a6000b9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3a64001f60 tx=0x7f3a60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0e8d279a71b8f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.734+0000 7f3a7159a640 1 -- 172.21.15.16:0/3471359787 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64001b30 con 0x7f3a60005020 2024-04-25T10:05:24.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.734+0000 7f3a7159a640 1 -- 172.21.15.16:0/3471359787 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a64031620 con 0x7f3a60005020 2024-04-25T10:05:24.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.734+0000 7f3a7159a640 1 -- 172.21.15.16:0/3471359787 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64031a10 con 0x7f3a60005020 2024-04-25T10:05:24.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.735+0000 7f3a78aee640 1 -- 172.21.15.16:0/3471359787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a60005020 msgr2=0x7f3a6000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.735+0000 7f3a78aee640 1 --2- 172.21.15.16:0/3471359787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a60005020 0x7f3a6000b9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3a64001f60 tx=0x7f3a60005830 comp rx=0 tx=0).stop 2024-04-25T10:05:24.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.735+0000 7f3a72d9d640 1 -- 172.21.15.16:0/3471359787 reap_dead start 2024-04-25T10:05:24.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.735+0000 7f3a78aee640 1 -- 172.21.15.16:0/3471359787 shutdown_connections 2024-04-25T10:05:24.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.735+0000 7f3a78aee640 1 -- 172.21.15.16:0/3471359787 >> 172.21.15.16:0/3471359787 conn(0x7f3a740b3430 msgr2=0x7f3a740b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.735+0000 7f3a78aee640 1 -- 172.21.15.16:0/3471359787 shutdown_connections 2024-04-25T10:05:24.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.735+0000 7f3a78aee640 1 -- 172.21.15.16:0/3471359787 wait complete. 2024-04-25T10:05:24.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.736+0000 7f3a78aee640 1 Processor -- start 2024-04-25T10:05:24.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.736+0000 7f3a78aee640 1 -- start start 2024-04-25T10:05:24.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.737+0000 7f3a78aee640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a60005020 0x7f3a74157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.737+0000 7f3a78aee640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a740c7e60 0x7f3a7409c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.737+0000 7f3a78aee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a740c82e0 0x7f3a74160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.737+0000 7f3a78aee640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7409a150 con 0x7f3a740c82e0 2024-04-25T10:05:24.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.737+0000 7f3a78aee640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a740d0d80 con 0x7f3a60005020 2024-04-25T10:05:24.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.737+0000 7f3a78aee640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7409a2d0 con 0x7f3a740c7e60 2024-04-25T10:05:24.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.737+0000 7f3a72d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a740c82e0 0x7f3a74160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.737+0000 7f3a72d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a740c82e0 0x7f3a74160140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56266/0 (socket says 172.21.15.16:56266) 2024-04-25T10:05:24.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.737+0000 7f3a72d9d640 1 -- 172.21.15.16:0/2146105768 learned_addr learned my addr 172.21.15.16:0/2146105768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:24.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.737+0000 7f3a72d9d640 1 -- 172.21.15.16:0/2146105768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a740c7e60 msgr2=0x7f3a7409c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.737+0000 7f3a71d9b640 1 --2- 172.21.15.16:0/2146105768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a740c7e60 0x7f3a7409c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.737+0000 7f3a7259c640 1 --2- 172.21.15.16:0/2146105768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a60005020 0x7f3a74157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.737+0000 7f3a72d9d640 1 --2- 172.21.15.16:0/2146105768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a740c7e60 0x7f3a7409c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.737+0000 7f3a72d9d640 1 -- 172.21.15.16:0/2146105768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a60005020 msgr2=0x7f3a74157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.737+0000 7f3a72d9d640 1 --2- 172.21.15.16:0/2146105768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a60005020 0x7f3a74157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.737+0000 7f3a72d9d640 1 -- 172.21.15.16:0/2146105768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a74158a20 con 0x7f3a740c82e0 2024-04-25T10:05:24.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.738+0000 7f3a72d9d640 1 --2- 172.21.15.16:0/2146105768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a740c82e0 0x7f3a74160140 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3a6402aa00 tx=0x7f3a640015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.738+0000 7f3a6affd640 1 -- 172.21.15.16:0/2146105768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64006d60 con 0x7f3a740c82e0 2024-04-25T10:05:24.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.738+0000 7f3a6affd640 1 -- 172.21.15.16:0/2146105768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a640317d0 con 0x7f3a740c82e0 2024-04-25T10:05:24.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.738+0000 7f3a78aee640 1 -- 172.21.15.16:0/2146105768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a74158d00 con 0x7f3a740c82e0 2024-04-25T10:05:24.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.738+0000 7f3a6affd640 1 -- 172.21.15.16:0/2146105768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64031bc0 con 0x7f3a740c82e0 2024-04-25T10:05:24.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.738+0000 7f3a78aee640 1 -- 172.21.15.16:0/2146105768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a741592c0 con 0x7f3a740c82e0 2024-04-25T10:05:24.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.739+0000 7f3a78aee640 1 -- 172.21.15.16:0/2146105768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a740750b0 con 0x7f3a740c82e0 2024-04-25T10:05:24.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.739+0000 7f3a6affd640 1 -- 172.21.15.16:0/2146105768 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a64044020 con 0x7f3a740c82e0 2024-04-25T10:05:24.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.739+0000 7f3a6affd640 1 --2- 172.21.15.16:0/2146105768 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a5406b4d0 0x7f3a5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.740+0000 7f3a6affd640 1 -- 172.21.15.16:0/2146105768 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a640aa830 con 0x7f3a740c82e0 2024-04-25T10:05:24.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.740+0000 7f3a7259c640 1 --2- 172.21.15.16:0/2146105768 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a5406b4d0 0x7f3a5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.740+0000 7f3a7259c640 1 --2- 172.21.15.16:0/2146105768 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a5406b4d0 0x7f3a5406d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c000fd0 tx=0x7f3a6c001170 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.744+0000 7f3a6affd640 1 -- 172.21.15.16:0/2146105768 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a64079620 con 0x7f3a740c82e0 2024-04-25T10:05:24.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.923+0000 7f3a78aee640 1 -- 172.21.15.16:0/2146105768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a7409aed0 con 0x7f3a740c82e0 2024-04-25T10:05:24.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.923+0000 7f3a6affd640 1 -- 172.21.15.16:0/2146105768 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3a64078f20 con 0x7f3a740c82e0 2024-04-25T10:05:24.929 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:05:24.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.925+0000 7f3a78aee640 1 -- 172.21.15.16:0/2146105768 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a5406b4d0 msgr2=0x7f3a5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.925+0000 7f3a78aee640 1 --2- 172.21.15.16:0/2146105768 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a5406b4d0 0x7f3a5406d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c000fd0 tx=0x7f3a6c001170 comp rx=0 tx=0).stop 2024-04-25T10:05:24.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.925+0000 7f3a78aee640 1 -- 172.21.15.16:0/2146105768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a740c82e0 msgr2=0x7f3a74160140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.925+0000 7f3a78aee640 1 --2- 172.21.15.16:0/2146105768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a740c82e0 0x7f3a74160140 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3a6402aa00 tx=0x7f3a640015b0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.926+0000 7f3a78aee640 1 -- 172.21.15.16:0/2146105768 shutdown_connections 2024-04-25T10:05:24.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.926+0000 7f3a78aee640 1 --2- 172.21.15.16:0/2146105768 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a5406b4d0 0x7f3a5406d990 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.926+0000 7f3a78aee640 1 --2- 172.21.15.16:0/2146105768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a740c82e0 0x7f3a74160140 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.926+0000 7f3a78aee640 1 --2- 172.21.15.16:0/2146105768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a740c7e60 0x7f3a7409c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.926+0000 7f3a78aee640 1 --2- 172.21.15.16:0/2146105768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a60005020 0x7f3a74157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.926+0000 7f3a78aee640 1 -- 172.21.15.16:0/2146105768 >> 172.21.15.16:0/2146105768 conn(0x7f3a740b3430 msgr2=0x7f3a7415ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.926+0000 7f3a78aee640 1 -- 172.21.15.16:0/2146105768 shutdown_connections 2024-04-25T10:05:24.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:24.926+0000 7f3a78aee640 1 -- 172.21.15.16:0/2146105768 wait complete. 2024-04-25T10:05:24.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:25.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.040+0000 7f348ae5d640 1 Processor -- start 2024-04-25T10:05:25.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.040+0000 7f348ae5d640 1 -- start start 2024-04-25T10:05:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f348ae5d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f34840b97c0 0x7f34840b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f348ae5d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34840b5e40 con 0x7f348415dc80 2024-04-25T10:05:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f348ae5d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34841627a0 con 0x7f34840b5a30 2024-04-25T10:05:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f348ae5d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3484161f80 con 0x7f34840b97c0 2024-04-25T10:05:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f3488bf9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f348415dc80 0x7f3484161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35200/0 (socket says 172.21.15.16:35200) 2024-04-25T10:05:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f3488bf9640 1 -- 172.21.15.16:0/509811929 learned_addr learned my addr 172.21.15.16:0/509811929 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f3483fff640 1 --2- 172.21.15.16:0/509811929 >> v2:172.21.15.186:3300/0 conn(0x7f34840b97c0 0x7f34840b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f34837fe640 1 -- 172.21.15.16:0/509811929 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402539416 0 0) 0x7f3478001110 con 0x7f348415dc80 2024-04-25T10:05:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f34837fe640 1 -- 172.21.15.16:0/509811929 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3474003520 con 0x7f348415dc80 2024-04-25T10:05:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f34837fe640 1 -- 172.21.15.16:0/509811929 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 564340823 0 0) 0x7f3474003520 con 0x7f348415dc80 2024-04-25T10:05:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f34837fe640 1 -- 172.21.15.16:0/509811929 >> v2:172.21.15.186:3300/0 conn(0x7f34840b97c0 msgr2=0x7f34840b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f34837fe640 1 --2- 172.21.15.16:0/509811929 >> v2:172.21.15.186:3300/0 conn(0x7f34840b97c0 0x7f34840b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f34837fe640 1 -- 172.21.15.16:0/509811929 >> v1:172.21.15.186:6789/0 conn(0x7f34840b5a30 legacy=0x7f34840d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f34837fe640 1 -- 172.21.15.16:0/509811929 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34841621e0 con 0x7f348415dc80 2024-04-25T10:05:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f34837fe640 1 -- 172.21.15.16:0/509811929 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f3478001840 con 0x7f348415dc80 2024-04-25T10:05:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f34837fe640 1 -- 172.21.15.16:0/509811929 >> v1:172.21.15.16:6789/0 conn(0x7f348415dc80 legacy=0x7f3484161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f34837fe640 1 --2- 172.21.15.16:0/509811929 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34740039b0 0x7f3474003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f34837fe640 1 --2- 172.21.15.16:0/509811929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3474004800 0x7f3474004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f34837fe640 1 --2- 172.21.15.16:0/509811929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34740051c0 0x7f347400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f34837fe640 1 -- 172.21.15.16:0/509811929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3484161f80 con 0x7f3474004800 2024-04-25T10:05:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f34837fe640 1 -- 172.21.15.16:0/509811929 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347400c060 con 0x7f34740051c0 2024-04-25T10:05:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f34837fe640 1 -- 172.21.15.16:0/509811929 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347400c260 con 0x7f34740039b0 2024-04-25T10:05:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f3483fff640 1 --2- 172.21.15.16:0/509811929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3474004800 0x7f3474004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f3488bf9640 1 --2- 172.21.15.16:0/509811929 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34740039b0 0x7f3474003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.041+0000 7f34893fa640 1 --2- 172.21.15.16:0/509811929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34740051c0 0x7f347400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.042+0000 7f3483fff640 1 -- 172.21.15.16:0/509811929 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34740039b0 msgr2=0x7f3474003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.042+0000 7f3483fff640 1 --2- 172.21.15.16:0/509811929 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34740039b0 0x7f3474003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.042+0000 7f3483fff640 1 -- 172.21.15.16:0/509811929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34740051c0 msgr2=0x7f347400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.042+0000 7f3483fff640 1 --2- 172.21.15.16:0/509811929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34740051c0 0x7f347400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.042+0000 7f34893fa640 1 -- 172.21.15.16:0/509811929 reap_dead start 2024-04-25T10:05:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.042+0000 7f3483fff640 1 -- 172.21.15.16:0/509811929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3474003520 con 0x7f3474004800 2024-04-25T10:05:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.042+0000 7f3483fff640 1 --2- 172.21.15.16:0/509811929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3474004800 0x7f3474004c80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3470009ac0 tx=0x7f3474005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4345cd28aa2a52cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.042+0000 7f34837fe640 1 -- 172.21.15.16:0/509811929 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470036070 con 0x7f3474004800 2024-04-25T10:05:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.042+0000 7f34837fe640 1 -- 172.21.15.16:0/509811929 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3470031400 con 0x7f3474004800 2024-04-25T10:05:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.042+0000 7f34837fe640 1 -- 172.21.15.16:0/509811929 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470009690 con 0x7f3474004800 2024-04-25T10:05:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.043+0000 7f348ae5d640 1 -- 172.21.15.16:0/509811929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3474004800 msgr2=0x7f3474004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.043+0000 7f348ae5d640 1 --2- 172.21.15.16:0/509811929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3474004800 0x7f3474004c80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3470009ac0 tx=0x7f3474005830 comp rx=0 tx=0).stop 2024-04-25T10:05:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.043+0000 7f348ae5d640 1 -- 172.21.15.16:0/509811929 shutdown_connections 2024-04-25T10:05:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.043+0000 7f348ae5d640 1 --2- 172.21.15.16:0/509811929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3474004800 0x7f3474004c80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.043+0000 7f348ae5d640 1 -- 172.21.15.16:0/509811929 >> 172.21.15.16:0/509811929 conn(0x7f34840d04f0 msgr2=0x7f34840d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.043+0000 7f348ae5d640 1 -- 172.21.15.16:0/509811929 shutdown_connections 2024-04-25T10:05:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.043+0000 7f348ae5d640 1 -- 172.21.15.16:0/509811929 wait complete. 2024-04-25T10:05:25.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.044+0000 7f348ae5d640 1 Processor -- start 2024-04-25T10:05:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.044+0000 7f348ae5d640 1 -- start start 2024-04-25T10:05:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.045+0000 7f348ae5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3474004800 0x7f3484157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.045+0000 7f348ae5d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34841548a0 0x7f348415f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.045+0000 7f348ae5d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3484160b60 0x7f348415fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.045+0000 7f348ae5d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348415e060 con 0x7f3474004800 2024-04-25T10:05:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.045+0000 7f348ae5d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34840d1c70 con 0x7f34841548a0 2024-04-25T10:05:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.045+0000 7f348ae5d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34840b54f0 con 0x7f3484160b60 2024-04-25T10:05:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.045+0000 7f3488bf9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3474004800 0x7f3484157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.045+0000 7f3488bf9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3474004800 0x7f3484157200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56274/0 (socket says 172.21.15.16:56274) 2024-04-25T10:05:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.045+0000 7f3488bf9640 1 -- 172.21.15.16:0/103180622 learned_addr learned my addr 172.21.15.16:0/103180622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.046+0000 7f3483fff640 1 --2- 172.21.15.16:0/103180622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34841548a0 0x7f348415f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.046+0000 7f34893fa640 1 --2- 172.21.15.16:0/103180622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3484160b60 0x7f348415fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.046+0000 7f3488bf9640 1 -- 172.21.15.16:0/103180622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3484160b60 msgr2=0x7f348415fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.046+0000 7f3488bf9640 1 --2- 172.21.15.16:0/103180622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3484160b60 0x7f348415fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.046+0000 7f3488bf9640 1 -- 172.21.15.16:0/103180622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34841548a0 msgr2=0x7f348415f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.046+0000 7f3488bf9640 1 --2- 172.21.15.16:0/103180622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34841548a0 0x7f348415f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.046+0000 7f3488bf9640 1 -- 172.21.15.16:0/103180622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34841f42a0 con 0x7f3474004800 2024-04-25T10:05:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.046+0000 7f3483fff640 1 --2- 172.21.15.16:0/103180622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34841548a0 0x7f348415f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.046+0000 7f3488bf9640 1 --2- 172.21.15.16:0/103180622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3474004800 0x7f3484157200 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f34780019b0 tx=0x7f3478007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.046+0000 7f34817fa640 1 -- 172.21.15.16:0/103180622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347800f8e0 con 0x7f3474004800 2024-04-25T10:05:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.046+0000 7f34817fa640 1 -- 172.21.15.16:0/103180622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3478018070 con 0x7f3474004800 2024-04-25T10:05:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.046+0000 7f34817fa640 1 -- 172.21.15.16:0/103180622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347800fce0 con 0x7f3474004800 2024-04-25T10:05:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.046+0000 7f348ae5d640 1 -- 172.21.15.16:0/103180622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3484161090 con 0x7f3474004800 2024-04-25T10:05:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.046+0000 7f348ae5d640 1 -- 172.21.15.16:0/103180622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34840c7880 con 0x7f3474004800 2024-04-25T10:05:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.047+0000 7f348ae5d640 1 -- 172.21.15.16:0/103180622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34840750b0 con 0x7f3474004800 2024-04-25T10:05:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.047+0000 7f34817fa640 1 -- 172.21.15.16:0/103180622 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f347801d460 con 0x7f3474004800 2024-04-25T10:05:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.048+0000 7f34817fa640 1 --2- 172.21.15.16:0/103180622 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f346806b500 0x7f346806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.048+0000 7f34817fa640 1 -- 172.21.15.16:0/103180622 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f347808a260 con 0x7f3474004800 2024-04-25T10:05:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.048+0000 7f3483fff640 1 --2- 172.21.15.16:0/103180622 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f346806b500 0x7f346806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.049+0000 7f3483fff640 1 --2- 172.21.15.16:0/103180622 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f346806b500 0x7f346806d9c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3470033000 tx=0x7f3470008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.052+0000 7f34817fa640 1 -- 172.21.15.16:0/103180622 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34840b5e40 con 0x7f3474004800 2024-04-25T10:05:25.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.233+0000 7f348ae5d640 1 -- 172.21.15.16:0/103180622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f34840d1780 con 0x7f3474004800 2024-04-25T10:05:25.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.233+0000 7f34817fa640 1 -- 172.21.15.16:0/103180622 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3478059170 con 0x7f3474004800 2024-04-25T10:05:25.239 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:05:25.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.235+0000 7f348ae5d640 1 -- 172.21.15.16:0/103180622 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f346806b500 msgr2=0x7f346806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.235+0000 7f348ae5d640 1 --2- 172.21.15.16:0/103180622 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f346806b500 0x7f346806d9c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3470033000 tx=0x7f3470008030 comp rx=0 tx=0).stop 2024-04-25T10:05:25.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.235+0000 7f348ae5d640 1 -- 172.21.15.16:0/103180622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3474004800 msgr2=0x7f3484157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.235+0000 7f348ae5d640 1 --2- 172.21.15.16:0/103180622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3474004800 0x7f3484157200 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f34780019b0 tx=0x7f3478007720 comp rx=0 tx=0).stop 2024-04-25T10:05:25.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.235+0000 7f348ae5d640 1 -- 172.21.15.16:0/103180622 shutdown_connections 2024-04-25T10:05:25.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.235+0000 7f348ae5d640 1 --2- 172.21.15.16:0/103180622 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f346806b500 0x7f346806d9c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.235+0000 7f348ae5d640 1 --2- 172.21.15.16:0/103180622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3484160b60 0x7f348415fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.235+0000 7f348ae5d640 1 --2- 172.21.15.16:0/103180622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34841548a0 0x7f348415f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.235+0000 7f348ae5d640 1 --2- 172.21.15.16:0/103180622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3474004800 0x7f3484157200 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.235+0000 7f348ae5d640 1 -- 172.21.15.16:0/103180622 >> 172.21.15.16:0/103180622 conn(0x7f34840d04f0 msgr2=0x7f34840b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.235+0000 7f348ae5d640 1 -- 172.21.15.16:0/103180622 shutdown_connections 2024-04-25T10:05:25.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.235+0000 7f348ae5d640 1 -- 172.21.15.16:0/103180622 wait complete. 2024-04-25T10:05:25.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:25.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.347+0000 7f70e6b5c640 1 Processor -- start 2024-04-25T10:05:25.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.347+0000 7f70e6b5c640 1 -- start start 2024-04-25T10:05:25.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.348+0000 7f70e6b5c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f70e015dce0 0x7f70e00d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.348+0000 7f70e6b5c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f70e00b5b60 0x7f70e00b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.348+0000 7f70e6b5c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e015e0c0 con 0x7f70e00d35b0 2024-04-25T10:05:25.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.348+0000 7f70e6b5c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e00d2510 con 0x7f70e00b5b60 2024-04-25T10:05:25.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.348+0000 7f70e6b5c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e00d27f0 con 0x7f70e015dce0 2024-04-25T10:05:25.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.348+0000 7f70e50f9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f70e00b5b60 0x7f70e00b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.348+0000 7f70d7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f70e00d35b0 0x7f70e00b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35204/0 (socket says 172.21.15.16:35204) 2024-04-25T10:05:25.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.348+0000 7f70d7fff640 1 -- 172.21.15.16:0/514504161 learned_addr learned my addr 172.21.15.16:0/514504161 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:25.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.348+0000 7f70e50f9640 1 -- 172.21.15.16:0/514504161 >> v2:172.21.15.186:3300/0 conn(0x7f70e015dce0 msgr2=0x7f70e00d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.348+0000 7f70e48f8640 1 --2- 172.21.15.16:0/514504161 >> v2:172.21.15.186:3300/0 conn(0x7f70e015dce0 0x7f70e00d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.348+0000 7f70e50f9640 1 --2- 172.21.15.16:0/514504161 >> v2:172.21.15.186:3300/0 conn(0x7f70e015dce0 0x7f70e00d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.348+0000 7f70d77fe640 1 -- 172.21.15.16:0/514504161 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970892215 0 0) 0x7f70e015e0c0 con 0x7f70e00d35b0 2024-04-25T10:05:25.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.348+0000 7f70e50f9640 1 -- 172.21.15.16:0/514504161 >> v1:172.21.15.16:6789/0 conn(0x7f70e00d35b0 legacy=0x7f70e00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.348+0000 7f70e50f9640 1 -- 172.21.15.16:0/514504161 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e00d2ad0 con 0x7f70e00b5b60 2024-04-25T10:05:25.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.348+0000 7f70e48f8640 1 --2- 172.21.15.16:0/514504161 >> v2:172.21.15.186:3300/0 conn(0x7f70e015dce0 0x7f70e00d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:25.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.349+0000 7f70e50f9640 1 --2- 172.21.15.16:0/514504161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e00b5b60 0x7f70e00b6b00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f70e00ce220 tx=0x7f70d802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6631b0e4f7e40244 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.349+0000 7f70d77fe640 1 -- 172.21.15.16:0/514504161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8007740 con 0x7f70e00b5b60 2024-04-25T10:05:25.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.349+0000 7f70d77fe640 1 -- 172.21.15.16:0/514504161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d80078e0 con 0x7f70e00b5b60 2024-04-25T10:05:25.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.349+0000 7f70d77fe640 1 -- 172.21.15.16:0/514504161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8007cb0 con 0x7f70e00b5b60 2024-04-25T10:05:25.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.350+0000 7f70e6b5c640 1 -- 172.21.15.16:0/514504161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e00b5b60 msgr2=0x7f70e00b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.350+0000 7f70e6b5c640 1 --2- 172.21.15.16:0/514504161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e00b5b60 0x7f70e00b6b00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f70e00ce220 tx=0x7f70d802a680 comp rx=0 tx=0).stop 2024-04-25T10:05:25.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.350+0000 7f70e6b5c640 1 -- 172.21.15.16:0/514504161 shutdown_connections 2024-04-25T10:05:25.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.350+0000 7f70e6b5c640 1 --2- 172.21.15.16:0/514504161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e00b5b60 0x7f70e00b6b00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.350+0000 7f70e6b5c640 1 --2- 172.21.15.16:0/514504161 >> v2:172.21.15.186:3300/0 conn(0x7f70e015dce0 0x7f70e00d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.350+0000 7f70e6b5c640 1 -- 172.21.15.16:0/514504161 >> 172.21.15.16:0/514504161 conn(0x7f70e0072900 msgr2=0x7f70e00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.350+0000 7f70e6b5c640 1 -- 172.21.15.16:0/514504161 shutdown_connections 2024-04-25T10:05:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.350+0000 7f70e6b5c640 1 -- 172.21.15.16:0/514504161 wait complete. 2024-04-25T10:05:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.351+0000 7f70e6b5c640 1 Processor -- start 2024-04-25T10:05:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.351+0000 7f70e6b5c640 1 -- start start 2024-04-25T10:05:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70e6b5c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70e00b5b60 0x7f70e0152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70e6b5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e00d35b0 0x7f70e015f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70e6b5c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70e015dce0 0x7f70e00b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70e6b5c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00b9820 con 0x7f70e00d35b0 2024-04-25T10:05:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70e6b5c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00d3090 con 0x7f70e00b5b60 2024-04-25T10:05:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70e6b5c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00b99a0 con 0x7f70e015dce0 2024-04-25T10:05:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e00d35b0 0x7f70e015f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e00d35b0 0x7f70e015f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56296/0 (socket says 172.21.15.16:56296) 2024-04-25T10:05:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70d7fff640 1 -- 172.21.15.16:0/3065536059 learned_addr learned my addr 172.21.15.16:0/3065536059 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70e48f8640 1 --2- 172.21.15.16:0/3065536059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70e00b5b60 0x7f70e0152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70e50f9640 1 --2- 172.21.15.16:0/3065536059 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70e015dce0 0x7f70e00b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70d7fff640 1 -- 172.21.15.16:0/3065536059 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70e015dce0 msgr2=0x7f70e00b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70d7fff640 1 --2- 172.21.15.16:0/3065536059 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70e015dce0 0x7f70e00b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70d7fff640 1 -- 172.21.15.16:0/3065536059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70e00b5b60 msgr2=0x7f70e0152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70d7fff640 1 --2- 172.21.15.16:0/3065536059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70e00b5b60 0x7f70e0152ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70d7fff640 1 -- 172.21.15.16:0/3065536059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e00cbf70 con 0x7f70e00d35b0 2024-04-25T10:05:25.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70d7fff640 1 --2- 172.21.15.16:0/3065536059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e00d35b0 0x7f70e015f080 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f70d000c4e0 tx=0x7f70d000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70d57fa640 1 -- 172.21.15.16:0/3065536059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d0018070 con 0x7f70e00d35b0 2024-04-25T10:05:25.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70d57fa640 1 -- 172.21.15.16:0/3065536059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d0007500 con 0x7f70e00d35b0 2024-04-25T10:05:25.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70e6b5c640 1 -- 172.21.15.16:0/3065536059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e00cc250 con 0x7f70e00d35b0 2024-04-25T10:05:25.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70d57fa640 1 -- 172.21.15.16:0/3065536059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d0004a20 con 0x7f70e00d35b0 2024-04-25T10:05:25.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.352+0000 7f70e6b5c640 1 -- 172.21.15.16:0/3065536059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e00cc810 con 0x7f70e00d35b0 2024-04-25T10:05:25.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.353+0000 7f70e6b5c640 1 -- 172.21.15.16:0/3065536059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e00749b0 con 0x7f70e00d35b0 2024-04-25T10:05:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.354+0000 7f70d57fa640 1 -- 172.21.15.16:0/3065536059 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70d001b070 con 0x7f70e00d35b0 2024-04-25T10:05:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.354+0000 7f70d57fa640 1 --2- 172.21.15.16:0/3065536059 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f70cc06b4d0 0x7f70cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.354+0000 7f70d57fa640 1 -- 172.21.15.16:0/3065536059 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70d0089560 con 0x7f70e00d35b0 2024-04-25T10:05:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.354+0000 7f70e48f8640 1 --2- 172.21.15.16:0/3065536059 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f70cc06b4d0 0x7f70cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.355+0000 7f70e48f8640 1 --2- 172.21.15.16:0/3065536059 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f70cc06b4d0 0x7f70cc06d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f70e015a2a0 tx=0x7f70dc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.358+0000 7f70d57fa640 1 -- 172.21.15.16:0/3065536059 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f70d00584a0 con 0x7f70e00d35b0 2024-04-25T10:05:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.582+0000 7f70e6b5c640 1 -- 172.21.15.16:0/3065536059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70e015d1a0 con 0x7f70e00d35b0 2024-04-25T10:05:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.582+0000 7f70d57fa640 1 -- 172.21.15.16:0/3065536059 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4939 (secure 0 0 0) 0x7f70d0057da0 con 0x7f70e00d35b0 2024-04-25T10:05:25.588 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:25.588 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1965168813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1965168813},{"type":"v1","addr":"172.21.15.16:6834","nonce":1965168813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/437791378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":437791378},{"type":"v1","addr":"172.21.15.16:6835","nonce":437791378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34145,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3044177642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3044177642},{"type":"v1","addr":"172.21.15.186:6835","nonce":3044177642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:10.814281+0000","modified":"2024-04-25T10:05:12.805138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34148":{"gid":34148,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/1763853189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1763853189},{"type":"v1","addr":"172.21.15.186:6837","nonce":1763853189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:05:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-04-25T10:05:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.584+0000 7f70e6b5c640 1 -- 172.21.15.16:0/3065536059 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f70cc06b4d0 msgr2=0x7f70cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.584+0000 7f70e6b5c640 1 --2- 172.21.15.16:0/3065536059 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f70cc06b4d0 0x7f70cc06d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f70e015a2a0 tx=0x7f70dc008040 comp rx=0 tx=0).stop 2024-04-25T10:05:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.584+0000 7f70e6b5c640 1 -- 172.21.15.16:0/3065536059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e00d35b0 msgr2=0x7f70e015f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.584+0000 7f70e6b5c640 1 --2- 172.21.15.16:0/3065536059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e00d35b0 0x7f70e015f080 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f70d000c4e0 tx=0x7f70d000c9a0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.584+0000 7f70e6b5c640 1 -- 172.21.15.16:0/3065536059 shutdown_connections 2024-04-25T10:05:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.584+0000 7f70e6b5c640 1 --2- 172.21.15.16:0/3065536059 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f70cc06b4d0 0x7f70cc06d990 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.584+0000 7f70e6b5c640 1 --2- 172.21.15.16:0/3065536059 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70e015dce0 0x7f70e00b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.584+0000 7f70e6b5c640 1 --2- 172.21.15.16:0/3065536059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e00d35b0 0x7f70e015f080 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.584+0000 7f70e6b5c640 1 --2- 172.21.15.16:0/3065536059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70e00b5b60 0x7f70e0152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.585+0000 7f70e6b5c640 1 -- 172.21.15.16:0/3065536059 >> 172.21.15.16:0/3065536059 conn(0x7f70e0072900 msgr2=0x7f70e015be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.585+0000 7f70e6b5c640 1 -- 172.21.15.16:0/3065536059 shutdown_connections 2024-04-25T10:05:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.585+0000 7f70e6b5c640 1 -- 172.21.15.16:0/3065536059 wait complete. 2024-04-25T10:05:25.601 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:25.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.696+0000 7fe8f4a3a640 1 Processor -- start 2024-04-25T10:05:25.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.696+0000 7fe8f4a3a640 1 -- start start 2024-04-25T10:05:25.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.697+0000 7fe8f4a3a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8f00d35b0 0x7fe8f00b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.697+0000 7fe8f4a3a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe8f00b5990 0x7fe8f00b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.697+0000 7fe8f4a3a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f015e0c0 con 0x7fe8f015dce0 2024-04-25T10:05:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.697+0000 7fe8f4a3a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f00b5de0 con 0x7fe8f00b5990 2024-04-25T10:05:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.697+0000 7fe8f4a3a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f00d2510 con 0x7fe8f00d35b0 2024-04-25T10:05:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.697+0000 7fe8edd9b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8f00d35b0 0x7fe8f00b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.697+0000 7fe8eed9d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe8f00b5990 0x7fe8f00b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.697+0000 7fe8edd9b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8f00d35b0 0x7fe8f00b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:36608/0 (socket says 172.21.15.16:36608) 2024-04-25T10:05:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.697+0000 7fe8edd9b640 1 -- 172.21.15.16:0/3147395201 learned_addr learned my addr 172.21.15.16:0/3147395201 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.697+0000 7fe8edd9b640 1 -- 172.21.15.16:0/3147395201 >> v2:172.21.15.186:3300/0 conn(0x7fe8f00b5990 msgr2=0x7fe8f00b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.697+0000 7fe8edd9b640 1 --2- 172.21.15.16:0/3147395201 >> v2:172.21.15.186:3300/0 conn(0x7fe8f00b5990 0x7fe8f00b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.697+0000 7fe8edd9b640 1 -- 172.21.15.16:0/3147395201 >> v1:172.21.15.186:6789/0 conn(0x7fe8f015dce0 legacy=0x7fe8f00d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.697+0000 7fe8edd9b640 1 -- 172.21.15.16:0/3147395201 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f00d27f0 con 0x7fe8f00d35b0 2024-04-25T10:05:25.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.698+0000 7fe8edd9b640 1 --2- 172.21.15.16:0/3147395201 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8f00d35b0 0x7fe8f00b5450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009870 tx=0x7fe8e402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee2607384cb3c640 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.698+0000 7fe8ed59a640 1 -- 172.21.15.16:0/3147395201 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4037070 con 0x7fe8f00d35b0 2024-04-25T10:05:25.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.698+0000 7fe8ed59a640 1 -- 172.21.15.16:0/3147395201 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e4032440 con 0x7fe8f00d35b0 2024-04-25T10:05:25.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.698+0000 7fe8ed59a640 1 -- 172.21.15.16:0/3147395201 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4032810 con 0x7fe8f00d35b0 2024-04-25T10:05:25.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.699+0000 7fe8f4a3a640 1 -- 172.21.15.16:0/3147395201 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8f00d35b0 msgr2=0x7fe8f00b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.699+0000 7fe8f4a3a640 1 --2- 172.21.15.16:0/3147395201 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8f00d35b0 0x7fe8f00b5450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009870 tx=0x7fe8e402a250 comp rx=0 tx=0).stop 2024-04-25T10:05:25.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.699+0000 7fe8f4a3a640 1 -- 172.21.15.16:0/3147395201 shutdown_connections 2024-04-25T10:05:25.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.699+0000 7fe8f4a3a640 1 --2- 172.21.15.16:0/3147395201 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8f00d35b0 0x7fe8f00b5450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.699+0000 7fe8f4a3a640 1 --2- 172.21.15.16:0/3147395201 >> v2:172.21.15.186:3300/0 conn(0x7fe8f00b5990 0x7fe8f00b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.699+0000 7fe8f4a3a640 1 -- 172.21.15.16:0/3147395201 >> 172.21.15.16:0/3147395201 conn(0x7fe8f0072900 msgr2=0x7fe8f00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.699+0000 7fe8f4a3a640 1 -- 172.21.15.16:0/3147395201 shutdown_connections 2024-04-25T10:05:25.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.699+0000 7fe8f4a3a640 1 -- 172.21.15.16:0/3147395201 wait complete. 2024-04-25T10:05:25.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.700+0000 7fe8f4a3a640 1 Processor -- start 2024-04-25T10:05:25.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.700+0000 7fe8f4a3a640 1 -- start start 2024-04-25T10:05:25.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.701+0000 7fe8f4a3a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8f00b5990 0x7fe8f014d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.701+0000 7fe8f4a3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f00d35b0 0x7fe8f015fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.701+0000 7fe8f4a3a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8f015dce0 0x7fe8f0149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.701+0000 7fe8f4a3a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f00d3090 con 0x7fe8f00d35b0 2024-04-25T10:05:25.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.701+0000 7fe8f4a3a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f00d2db0 con 0x7fe8f015dce0 2024-04-25T10:05:25.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.701+0000 7fe8f4a3a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f00d2160 con 0x7fe8f00b5990 2024-04-25T10:05:25.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.701+0000 7fe8edd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f00d35b0 0x7fe8f015fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.701+0000 7fe8edd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f00d35b0 0x7fe8f015fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56302/0 (socket says 172.21.15.16:56302) 2024-04-25T10:05:25.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.701+0000 7fe8edd9b640 1 -- 172.21.15.16:0/3772727765 learned_addr learned my addr 172.21.15.16:0/3772727765 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:25.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.701+0000 7fe8eed9d640 1 --2- 172.21.15.16:0/3772727765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8f015dce0 0x7fe8f0149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.701+0000 7fe8ee59c640 1 --2- 172.21.15.16:0/3772727765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8f00b5990 0x7fe8f014d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.701+0000 7fe8edd9b640 1 -- 172.21.15.16:0/3772727765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8f00b5990 msgr2=0x7fe8f014d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.701+0000 7fe8edd9b640 1 --2- 172.21.15.16:0/3772727765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8f00b5990 0x7fe8f014d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.702+0000 7fe8edd9b640 1 -- 172.21.15.16:0/3772727765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8f015dce0 msgr2=0x7fe8f0149d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.702+0000 7fe8edd9b640 1 --2- 172.21.15.16:0/3772727765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8f015dce0 0x7fe8f0149d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.702+0000 7fe8edd9b640 1 -- 172.21.15.16:0/3772727765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f01611c0 con 0x7fe8f00d35b0 2024-04-25T10:05:25.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.702+0000 7fe8edd9b640 1 --2- 172.21.15.16:0/3772727765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f00d35b0 0x7fe8f015fed0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4000c00 tx=0x7fe8e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.702+0000 7fe8deffd640 1 -- 172.21.15.16:0/3772727765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4037040 con 0x7fe8f00d35b0 2024-04-25T10:05:25.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.702+0000 7fe8deffd640 1 -- 172.21.15.16:0/3772727765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e4042070 con 0x7fe8f00d35b0 2024-04-25T10:05:25.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.702+0000 7fe8deffd640 1 -- 172.21.15.16:0/3772727765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4030880 con 0x7fe8f00d35b0 2024-04-25T10:05:25.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.702+0000 7fe8f4a3a640 1 -- 172.21.15.16:0/3772727765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f0160640 con 0x7fe8f00d35b0 2024-04-25T10:05:25.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.702+0000 7fe8f4a3a640 1 -- 172.21.15.16:0/3772727765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f0160c00 con 0x7fe8f00d35b0 2024-04-25T10:05:25.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.703+0000 7fe8f4a3a640 1 -- 172.21.15.16:0/3772727765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f0067490 con 0x7fe8f00d35b0 2024-04-25T10:05:25.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.703+0000 7fe8deffd640 1 -- 172.21.15.16:0/3772727765 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8e4030a20 con 0x7fe8f00d35b0 2024-04-25T10:05:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.704+0000 7fe8deffd640 1 --2- 172.21.15.16:0/3772727765 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe8d006b4d0 0x7fe8d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.704+0000 7fe8deffd640 1 -- 172.21.15.16:0/3772727765 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8e40aa920 con 0x7fe8f00d35b0 2024-04-25T10:05:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.704+0000 7fe8ee59c640 1 --2- 172.21.15.16:0/3772727765 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe8d006b4d0 0x7fe8d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.705+0000 7fe8ee59c640 1 --2- 172.21.15.16:0/3772727765 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe8d006b4d0 0x7fe8d006d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00016e0 tx=0x7fe8e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.708+0000 7fe8deffd640 1 -- 172.21.15.16:0/3772727765 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8f0067490 con 0x7fe8f00d35b0 2024-04-25T10:05:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.888+0000 7fe8f4a3a640 1 -- 172.21.15.16:0/3772727765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8f00749b0 con 0x7fe8f00d35b0 2024-04-25T10:05:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.889+0000 7fe8deffd640 1 -- 172.21.15.16:0/3772727765 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16609 (secure 0 0 0) 0x7fe8e4079150 con 0x7fe8f00d35b0 2024-04-25T10:05:25.894 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:25.894 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:12.449183+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:25.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.891+0000 7fe8f4a3a640 1 -- 172.21.15.16:0/3772727765 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe8d006b4d0 msgr2=0x7fe8d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.891+0000 7fe8f4a3a640 1 --2- 172.21.15.16:0/3772727765 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe8d006b4d0 0x7fe8d006d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00016e0 tx=0x7fe8e0009040 comp rx=0 tx=0).stop 2024-04-25T10:05:25.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.891+0000 7fe8f4a3a640 1 -- 172.21.15.16:0/3772727765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f00d35b0 msgr2=0x7fe8f015fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.891+0000 7fe8f4a3a640 1 --2- 172.21.15.16:0/3772727765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f00d35b0 0x7fe8f015fed0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4000c00 tx=0x7fe8e40046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.891+0000 7fe8f4a3a640 1 -- 172.21.15.16:0/3772727765 shutdown_connections 2024-04-25T10:05:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.891+0000 7fe8f4a3a640 1 --2- 172.21.15.16:0/3772727765 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe8d006b4d0 0x7fe8d006d990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.891+0000 7fe8f4a3a640 1 --2- 172.21.15.16:0/3772727765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8f015dce0 0x7fe8f0149d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.891+0000 7fe8f4a3a640 1 --2- 172.21.15.16:0/3772727765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f00d35b0 0x7fe8f015fed0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.891+0000 7fe8f4a3a640 1 --2- 172.21.15.16:0/3772727765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8f00b5990 0x7fe8f014d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.891+0000 7fe8f4a3a640 1 -- 172.21.15.16:0/3772727765 >> 172.21.15.16:0/3772727765 conn(0x7fe8f0072900 msgr2=0x7fe8f00d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.891+0000 7fe8f4a3a640 1 -- 172.21.15.16:0/3772727765 shutdown_connections 2024-04-25T10:05:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:25.891+0000 7fe8f4a3a640 1 -- 172.21.15.16:0/3772727765 wait complete. 2024-04-25T10:05:25.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:08.724537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:09.793105+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:25.907 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:26.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.005+0000 7f074d6dd640 1 Processor -- start 2024-04-25T10:05:26.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.005+0000 7f074d6dd640 1 -- start start 2024-04-25T10:05:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.006+0000 7f074d6dd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07480b5c00 0x7f0748168c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.006+0000 7f074d6dd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07481627a0 con 0x7f074815dc80 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.006+0000 7f074d6dd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07481621e0 con 0x7f07480b97c0 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.006+0000 7f074d6dd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0748161f80 con 0x7f07480b5c00 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.006+0000 7f07477fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07480b5c00 0x7f0748168c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.006+0000 7f0746ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f074815dc80 0x7f0748161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35212/0 (socket says 172.21.15.16:35212) 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.006+0000 7f0746ffd640 1 -- 172.21.15.16:0/3964931290 learned_addr learned my addr 172.21.15.16:0/3964931290 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.007+0000 7f07477fe640 1 -- 172.21.15.16:0/3964931290 >> v1:172.21.15.186:6789/0 conn(0x7f07480b97c0 legacy=0x7f07480b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.007+0000 7f07477fe640 1 -- 172.21.15.16:0/3964931290 >> v1:172.21.15.16:6789/0 conn(0x7f074815dc80 legacy=0x7f0748161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.007+0000 7f0745ffb640 1 -- 172.21.15.16:0/3964931290 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445731492 0 0) 0x7f07481627a0 con 0x7f074815dc80 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.007+0000 7f07477fe640 1 -- 172.21.15.16:0/3964931290 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07481624c0 con 0x7f07480b5c00 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.007+0000 7f07477fe640 1 --2- 172.21.15.16:0/3964931290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07480b5c00 0x7f0748168c00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0738009d90 tx=0x7f073802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca67ba95968d0b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.008+0000 7f0745ffb640 1 -- 172.21.15.16:0/3964931290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0738007740 con 0x7f07480b5c00 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.008+0000 7f0745ffb640 1 -- 172.21.15.16:0/3964931290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07380078e0 con 0x7f07480b5c00 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.008+0000 7f0745ffb640 1 -- 172.21.15.16:0/3964931290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0738007cb0 con 0x7f07480b5c00 2024-04-25T10:05:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.008+0000 7f074d6dd640 1 -- 172.21.15.16:0/3964931290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07480b5c00 msgr2=0x7f0748168c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.008+0000 7f074d6dd640 1 --2- 172.21.15.16:0/3964931290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07480b5c00 0x7f0748168c00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0738009d90 tx=0x7f073802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.008+0000 7f074d6dd640 1 -- 172.21.15.16:0/3964931290 shutdown_connections 2024-04-25T10:05:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.008+0000 7f074d6dd640 1 --2- 172.21.15.16:0/3964931290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07480b5c00 0x7f0748168c00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.008+0000 7f074d6dd640 1 -- 172.21.15.16:0/3964931290 >> 172.21.15.16:0/3964931290 conn(0x7f07480d04f0 msgr2=0x7f07480d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.008+0000 7f074d6dd640 1 -- 172.21.15.16:0/3964931290 shutdown_connections 2024-04-25T10:05:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.008+0000 7f074d6dd640 1 -- 172.21.15.16:0/3964931290 wait complete. 2024-04-25T10:05:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.009+0000 7f074d6dd640 1 Processor -- start 2024-04-25T10:05:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.010+0000 7f074d6dd640 1 -- start start 2024-04-25T10:05:26.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.010+0000 7f074d6dd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07480b5c00 0x7f07480b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.010+0000 7f074d6dd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f07480b97c0 0x7f074814ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.010+0000 7f074d6dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f074815dc80 0x7f074814f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.010+0000 7f074d6dd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07480d3480 con 0x7f074815dc80 2024-04-25T10:05:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.010+0000 7f074d6dd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074815e060 con 0x7f07480b5c00 2024-04-25T10:05:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.010+0000 7f074d6dd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07480d3600 con 0x7f07480b97c0 2024-04-25T10:05:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.010+0000 7f07477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f074815dc80 0x7f074814f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.010+0000 7f07477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f074815dc80 0x7f074814f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56316/0 (socket says 172.21.15.16:56316) 2024-04-25T10:05:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.010+0000 7f07477fe640 1 -- 172.21.15.16:0/3376663508 learned_addr learned my addr 172.21.15.16:0/3376663508 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.010+0000 7f0746ffd640 1 --2- 172.21.15.16:0/3376663508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07480b5c00 0x7f07480b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.011+0000 7f07467fc640 1 --2- 172.21.15.16:0/3376663508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f07480b97c0 0x7f074814ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.011+0000 7f07477fe640 1 -- 172.21.15.16:0/3376663508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f07480b97c0 msgr2=0x7f074814ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.011+0000 7f07477fe640 1 --2- 172.21.15.16:0/3376663508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f07480b97c0 0x7f074814ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.011+0000 7f07477fe640 1 -- 172.21.15.16:0/3376663508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07480b5c00 msgr2=0x7f07480b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.011+0000 7f07477fe640 1 --2- 172.21.15.16:0/3376663508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07480b5c00 0x7f07480b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.011+0000 7f07477fe640 1 -- 172.21.15.16:0/3376663508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07480b30b0 con 0x7f074815dc80 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.011+0000 7f0746ffd640 1 --2- 172.21.15.16:0/3376663508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07480b5c00 0x7f07480b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.011+0000 7f07477fe640 1 --2- 172.21.15.16:0/3376663508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f074815dc80 0x7f074814f1f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0738009d90 tx=0x7f0738004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.011+0000 7f073f7fe640 1 -- 172.21.15.16:0/3376663508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0738040070 con 0x7f074815dc80 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.011+0000 7f073f7fe640 1 -- 172.21.15.16:0/3376663508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0738031c40 con 0x7f074815dc80 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.011+0000 7f074d6dd640 1 -- 172.21.15.16:0/3376663508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074814ce70 con 0x7f074815dc80 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.011+0000 7f073f7fe640 1 -- 172.21.15.16:0/3376663508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0738003990 con 0x7f074815dc80 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.011+0000 7f074d6dd640 1 -- 172.21.15.16:0/3376663508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074814d430 con 0x7f074815dc80 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.012+0000 7f074d6dd640 1 -- 172.21.15.16:0/3376663508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07480d1f50 con 0x7f074815dc80 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.013+0000 7f073f7fe640 1 -- 172.21.15.16:0/3376663508 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0738039040 con 0x7f074815dc80 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.013+0000 7f073f7fe640 1 --2- 172.21.15.16:0/3376663508 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f072806b4d0 0x7f072806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.013+0000 7f073f7fe640 1 -- 172.21.15.16:0/3376663508 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07380ab3f0 con 0x7f074815dc80 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.013+0000 7f0746ffd640 1 --2- 172.21.15.16:0/3376663508 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f072806b4d0 0x7f072806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.014+0000 7f0746ffd640 1 --2- 172.21.15.16:0/3376663508 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f072806b4d0 0x7f072806d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0740007fd0 tx=0x7f0740007dc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.017+0000 7f073f7fe640 1 -- 172.21.15.16:0/3376663508 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f07480d1f50 con 0x7f074815dc80 2024-04-25T10:05:26.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.195+0000 7f074d6dd640 1 -- 172.21.15.16:0/3376663508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f07480750b0 con 0x7f074815dc80 2024-04-25T10:05:26.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.196+0000 7f073f7fe640 1 -- 172.21.15.16:0/3376663508 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0738079cb0 con 0x7f074815dc80 2024-04-25T10:05:26.201 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:05:26.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.198+0000 7f074d6dd640 1 -- 172.21.15.16:0/3376663508 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f072806b4d0 msgr2=0x7f072806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.198+0000 7f074d6dd640 1 --2- 172.21.15.16:0/3376663508 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f072806b4d0 0x7f072806d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0740007fd0 tx=0x7f0740007dc0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.198+0000 7f074d6dd640 1 -- 172.21.15.16:0/3376663508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f074815dc80 msgr2=0x7f074814f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.198+0000 7f074d6dd640 1 --2- 172.21.15.16:0/3376663508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f074815dc80 0x7f074814f1f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0738009d90 tx=0x7f0738004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:26.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.198+0000 7f074d6dd640 1 -- 172.21.15.16:0/3376663508 shutdown_connections 2024-04-25T10:05:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.198+0000 7f074d6dd640 1 --2- 172.21.15.16:0/3376663508 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f072806b4d0 0x7f072806d990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.198+0000 7f074d6dd640 1 --2- 172.21.15.16:0/3376663508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f074815dc80 0x7f074814f1f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.198+0000 7f074d6dd640 1 --2- 172.21.15.16:0/3376663508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f07480b97c0 0x7f074814ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.198+0000 7f074d6dd640 1 --2- 172.21.15.16:0/3376663508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07480b5c00 0x7f07480b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.198+0000 7f074d6dd640 1 -- 172.21.15.16:0/3376663508 >> 172.21.15.16:0/3376663508 conn(0x7f07480d04f0 msgr2=0x7f07480d3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.198+0000 7f074d6dd640 1 -- 172.21.15.16:0/3376663508 shutdown_connections 2024-04-25T10:05:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.198+0000 7f074d6dd640 1 -- 172.21.15.16:0/3376663508 wait complete. 2024-04-25T10:05:26.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.312+0000 7f26494db640 1 Processor -- start 2024-04-25T10:05:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.312+0000 7f26494db640 1 -- start start 2024-04-25T10:05:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.313+0000 7f26494db640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f264415dc80 0x7f2644161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.313+0000 7f26494db640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f26440b97c0 0x7f26440b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.313+0000 7f26494db640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26441627a0 con 0x7f26440b5a30 2024-04-25T10:05:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.313+0000 7f26494db640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26440b5ea0 con 0x7f26440b97c0 2024-04-25T10:05:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.313+0000 7f26494db640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26441621e0 con 0x7f264415dc80 2024-04-25T10:05:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.313+0000 7f26437fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f26440b5a30 0x7f264406bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35218/0 (socket says 172.21.15.16:35218) 2024-04-25T10:05:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.313+0000 7f26437fe640 1 -- 172.21.15.16:0/1999131820 learned_addr learned my addr 172.21.15.16:0/1999131820 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.313+0000 7f2642ffd640 1 --2- 172.21.15.16:0/1999131820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f264415dc80 0x7f2644161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.313+0000 7f26427fc640 1 --2- 172.21.15.16:0/1999131820 >> v2:172.21.15.186:3300/0 conn(0x7f26440b97c0 0x7f26440b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.313+0000 7f2641ffb640 1 -- 172.21.15.16:0/1999131820 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344252549 0 0) 0x7f26441627a0 con 0x7f26440b5a30 2024-04-25T10:05:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.313+0000 7f2641ffb640 1 -- 172.21.15.16:0/1999131820 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2630003520 con 0x7f26440b5a30 2024-04-25T10:05:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f2641ffb640 1 -- 172.21.15.16:0/1999131820 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 289808871 0 0) 0x7f2630003520 con 0x7f26440b5a30 2024-04-25T10:05:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f2641ffb640 1 -- 172.21.15.16:0/1999131820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f264415dc80 msgr2=0x7f2644161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f2641ffb640 1 --2- 172.21.15.16:0/1999131820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f264415dc80 0x7f2644161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f2641ffb640 1 -- 172.21.15.16:0/1999131820 >> v2:172.21.15.186:3300/0 conn(0x7f26440b97c0 msgr2=0x7f26440b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f2641ffb640 1 --2- 172.21.15.16:0/1999131820 >> v2:172.21.15.186:3300/0 conn(0x7f26440b97c0 0x7f26440b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f2641ffb640 1 -- 172.21.15.16:0/1999131820 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26441624c0 con 0x7f26440b5a30 2024-04-25T10:05:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f2641ffb640 1 -- 172.21.15.16:0/1999131820 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f26340019f0 con 0x7f26440b5a30 2024-04-25T10:05:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f2641ffb640 1 -- 172.21.15.16:0/1999131820 >> v1:172.21.15.16:6789/0 conn(0x7f26440b5a30 legacy=0x7f264406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f2641ffb640 1 --2- 172.21.15.16:0/1999131820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f264415dc80 0x7f26300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f2641ffb640 1 --2- 172.21.15.16:0/1999131820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2630003f30 0x7f2630004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f2641ffb640 1 --2- 172.21.15.16:0/1999131820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2630004d40 0x7f26300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f2641ffb640 1 -- 172.21.15.16:0/1999131820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26440b5ea0 con 0x7f2630004d40 2024-04-25T10:05:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f2641ffb640 1 -- 172.21.15.16:0/1999131820 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26441621e0 con 0x7f2630003f30 2024-04-25T10:05:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f2641ffb640 1 -- 172.21.15.16:0/1999131820 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263000b9c0 con 0x7f264415dc80 2024-04-25T10:05:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f26437fe640 1 --2- 172.21.15.16:0/1999131820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2630004d40 0x7f26300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f2642ffd640 1 --2- 172.21.15.16:0/1999131820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f264415dc80 0x7f26300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f26427fc640 1 --2- 172.21.15.16:0/1999131820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2630003f30 0x7f2630004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f26437fe640 1 -- 172.21.15.16:0/1999131820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f264415dc80 msgr2=0x7f26300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f26437fe640 1 --2- 172.21.15.16:0/1999131820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f264415dc80 0x7f26300039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f26437fe640 1 -- 172.21.15.16:0/1999131820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2630003f30 msgr2=0x7f2630004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f26437fe640 1 --2- 172.21.15.16:0/1999131820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2630003f30 0x7f2630004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.314+0000 7f26437fe640 1 -- 172.21.15.16:0/1999131820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2630003520 con 0x7f2630004d40 2024-04-25T10:05:26.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.315+0000 7f26437fe640 1 --2- 172.21.15.16:0/1999131820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2630004d40 0x7f26300051c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f263400a870 tx=0x7f2630005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59d8b7e9deed58c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.315+0000 7f2641ffb640 1 -- 172.21.15.16:0/1999131820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263400a4c0 con 0x7f2630004d40 2024-04-25T10:05:26.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.315+0000 7f2641ffb640 1 -- 172.21.15.16:0/1999131820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263402f070 con 0x7f2630004d40 2024-04-25T10:05:26.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.315+0000 7f2641ffb640 1 -- 172.21.15.16:0/1999131820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263402bcb0 con 0x7f2630004d40 2024-04-25T10:05:26.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.315+0000 7f26494db640 1 -- 172.21.15.16:0/1999131820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2630004d40 msgr2=0x7f26300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.315+0000 7f26494db640 1 --2- 172.21.15.16:0/1999131820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2630004d40 0x7f26300051c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f263400a870 tx=0x7f2630005830 comp rx=0 tx=0).stop 2024-04-25T10:05:26.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.315+0000 7f26437fe640 1 -- 172.21.15.16:0/1999131820 reap_dead start 2024-04-25T10:05:26.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.315+0000 7f26494db640 1 -- 172.21.15.16:0/1999131820 shutdown_connections 2024-04-25T10:05:26.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.315+0000 7f26494db640 1 -- 172.21.15.16:0/1999131820 >> 172.21.15.16:0/1999131820 conn(0x7f26440d04f0 msgr2=0x7f26440d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.316+0000 7f26494db640 1 -- 172.21.15.16:0/1999131820 shutdown_connections 2024-04-25T10:05:26.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.316+0000 7f26494db640 1 -- 172.21.15.16:0/1999131820 wait complete. 2024-04-25T10:05:26.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.317+0000 7f26494db640 1 Processor -- start 2024-04-25T10:05:26.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.317+0000 7f26494db640 1 -- start start 2024-04-25T10:05:26.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26494db640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2630004d40 0x7f26441f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26494db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2644152450 0x7f26440cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26494db640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f264414f7f0 0x7f264415f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26494db640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26440d3480 con 0x7f2644152450 2024-04-25T10:05:26.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26494db640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264415e060 con 0x7f264414f7f0 2024-04-25T10:05:26.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26494db640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26440d3600 con 0x7f2630004d40 2024-04-25T10:05:26.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26427fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2644152450 0x7f26440cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26427fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2644152450 0x7f26440cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56338/0 (socket says 172.21.15.16:56338) 2024-04-25T10:05:26.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26427fc640 1 -- 172.21.15.16:0/3514334594 learned_addr learned my addr 172.21.15.16:0/3514334594 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:26.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f2642ffd640 1 --2- 172.21.15.16:0/3514334594 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2630004d40 0x7f26441f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26437fe640 1 --2- 172.21.15.16:0/3514334594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f264414f7f0 0x7f264415f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26427fc640 1 -- 172.21.15.16:0/3514334594 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2630004d40 msgr2=0x7f26441f2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26427fc640 1 --2- 172.21.15.16:0/3514334594 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2630004d40 0x7f26441f2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26427fc640 1 -- 172.21.15.16:0/3514334594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f264414f7f0 msgr2=0x7f264415f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26427fc640 1 --2- 172.21.15.16:0/3514334594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f264414f7f0 0x7f264415f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26427fc640 1 -- 172.21.15.16:0/3514334594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26440cdc00 con 0x7f2644152450 2024-04-25T10:05:26.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26437fe640 1 --2- 172.21.15.16:0/3514334594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f264414f7f0 0x7f264415f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:26.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26427fc640 1 --2- 172.21.15.16:0/3514334594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2644152450 0x7f26440cd6c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f262c011960 tx=0x7f262c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f263b7fe640 1 -- 172.21.15.16:0/3514334594 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c019070 con 0x7f2644152450 2024-04-25T10:05:26.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f263b7fe640 1 -- 172.21.15.16:0/3514334594 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c004240 con 0x7f2644152450 2024-04-25T10:05:26.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f263b7fe640 1 -- 172.21.15.16:0/3514334594 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c00f880 con 0x7f2644152450 2024-04-25T10:05:26.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26494db640 1 -- 172.21.15.16:0/3514334594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f264415e430 con 0x7f2644152450 2024-04-25T10:05:26.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.318+0000 7f26494db640 1 -- 172.21.15.16:0/3514334594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f264415e9f0 con 0x7f2644152450 2024-04-25T10:05:26.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.319+0000 7f26494db640 1 -- 172.21.15.16:0/3514334594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26440750b0 con 0x7f2644152450 2024-04-25T10:05:26.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.320+0000 7f263b7fe640 1 -- 172.21.15.16:0/3514334594 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f262c01c070 con 0x7f2644152450 2024-04-25T10:05:26.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.320+0000 7f263b7fe640 1 --2- 172.21.15.16:0/3514334594 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f262406b500 0x7f262406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.320+0000 7f263b7fe640 1 -- 172.21.15.16:0/3514334594 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f262c08a630 con 0x7f2644152450 2024-04-25T10:05:26.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.320+0000 7f2642ffd640 1 --2- 172.21.15.16:0/3514334594 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f262406b500 0x7f262406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.321+0000 7f2642ffd640 1 --2- 172.21.15.16:0/3514334594 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f262406b500 0x7f262406d9c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f263c009b00 tx=0x7f263c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.324+0000 7f263b7fe640 1 -- 172.21.15.16:0/3514334594 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f262c0594a0 con 0x7f2644152450 2024-04-25T10:05:26.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.507+0000 7f26494db640 1 -- 172.21.15.16:0/3514334594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f26440d1780 con 0x7f2644152450 2024-04-25T10:05:26.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.507+0000 7f263b7fe640 1 -- 172.21.15.16:0/3514334594 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f262c058da0 con 0x7f2644152450 2024-04-25T10:05:26.513 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:05:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.509+0000 7f26494db640 1 -- 172.21.15.16:0/3514334594 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f262406b500 msgr2=0x7f262406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.509+0000 7f26494db640 1 --2- 172.21.15.16:0/3514334594 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f262406b500 0x7f262406d9c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f263c009b00 tx=0x7f263c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.509+0000 7f26494db640 1 -- 172.21.15.16:0/3514334594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2644152450 msgr2=0x7f26440cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.509+0000 7f26494db640 1 --2- 172.21.15.16:0/3514334594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2644152450 0x7f26440cd6c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f262c011960 tx=0x7f262c011e20 comp rx=0 tx=0).stop 2024-04-25T10:05:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.509+0000 7f26494db640 1 -- 172.21.15.16:0/3514334594 shutdown_connections 2024-04-25T10:05:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.509+0000 7f26494db640 1 --2- 172.21.15.16:0/3514334594 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f262406b500 0x7f262406d9c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.509+0000 7f26494db640 1 --2- 172.21.15.16:0/3514334594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f264414f7f0 0x7f264415f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.509+0000 7f26494db640 1 --2- 172.21.15.16:0/3514334594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2644152450 0x7f26440cd6c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.509+0000 7f26494db640 1 --2- 172.21.15.16:0/3514334594 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2630004d40 0x7f26441f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.509+0000 7f26494db640 1 -- 172.21.15.16:0/3514334594 >> 172.21.15.16:0/3514334594 conn(0x7f26440d04f0 msgr2=0x7f26440b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.509+0000 7f26494db640 1 -- 172.21.15.16:0/3514334594 shutdown_connections 2024-04-25T10:05:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.509+0000 7f26494db640 1 -- 172.21.15.16:0/3514334594 wait complete. 2024-04-25T10:05:26.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:26.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.624+0000 7f027dccd640 1 Processor -- start 2024-04-25T10:05:26.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.624+0000 7f027dccd640 1 -- start start 2024-04-25T10:05:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.625+0000 7f027dccd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02780cc7b0 0x7f02780b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.625+0000 7f027dccd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02780cfbd0 con 0x7f02780b07d0 2024-04-25T10:05:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.625+0000 7f027dccd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02780cfeb0 con 0x7f027815bae0 2024-04-25T10:05:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.625+0000 7f027dccd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02780cf980 con 0x7f02780cc7b0 2024-04-25T10:05:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.625+0000 7f0276ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f02780b07d0 0x7f02780cc0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35228/0 (socket says 172.21.15.16:35228) 2024-04-25T10:05:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.625+0000 7f0276ffd640 1 -- 172.21.15.16:0/4274803503 learned_addr learned my addr 172.21.15.16:0/4274803503 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.625+0000 7f0277fff640 1 --2- 172.21.15.16:0/4274803503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02780cc7b0 0x7f02780b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.625+0000 7f02767fc640 1 -- 172.21.15.16:0/4274803503 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2917034556 0 0) 0x7f02780cfbd0 con 0x7f02780b07d0 2024-04-25T10:05:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.625+0000 7f02767fc640 1 -- 172.21.15.16:0/4274803503 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0260003520 con 0x7f02780b07d0 2024-04-25T10:05:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.625+0000 7f02767fc640 1 -- 172.21.15.16:0/4274803503 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1410574784 0 0) 0x7f02780cfeb0 con 0x7f027815bae0 2024-04-25T10:05:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.625+0000 7f02767fc640 1 -- 172.21.15.16:0/4274803503 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02780cfbd0 con 0x7f027815bae0 2024-04-25T10:05:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.625+0000 7f02767fc640 1 -- 172.21.15.16:0/4274803503 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3557903030 0 0) 0x7f0260003520 con 0x7f02780b07d0 2024-04-25T10:05:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.625+0000 7f0277fff640 1 -- 172.21.15.16:0/4274803503 >> v1:172.21.15.186:6789/0 conn(0x7f027815bae0 legacy=0x7f02780d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.625+0000 7f0277fff640 1 -- 172.21.15.16:0/4274803503 >> v1:172.21.15.16:6789/0 conn(0x7f02780b07d0 legacy=0x7f02780cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.625+0000 7f0277fff640 1 -- 172.21.15.16:0/4274803503 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02780d0190 con 0x7f02780cc7b0 2024-04-25T10:05:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.626+0000 7f0277fff640 1 --2- 172.21.15.16:0/4274803503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02780cc7b0 0x7f02780b0c60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f02780cfb00 tx=0x7f027002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c2cf8890831d7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.626+0000 7f02767fc640 1 -- 172.21.15.16:0/4274803503 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c001670 con 0x7f02780cc7b0 2024-04-25T10:05:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.626+0000 7f02767fc640 1 -- 172.21.15.16:0/4274803503 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0270007540 con 0x7f02780cc7b0 2024-04-25T10:05:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.626+0000 7f02767fc640 1 -- 172.21.15.16:0/4274803503 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02700078f0 con 0x7f02780cc7b0 2024-04-25T10:05:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.627+0000 7f027dccd640 1 -- 172.21.15.16:0/4274803503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02780cc7b0 msgr2=0x7f02780b0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.627+0000 7f027dccd640 1 --2- 172.21.15.16:0/4274803503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02780cc7b0 0x7f02780b0c60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f02780cfb00 tx=0x7f027002a660 comp rx=0 tx=0).stop 2024-04-25T10:05:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.627+0000 7f027dccd640 1 -- 172.21.15.16:0/4274803503 shutdown_connections 2024-04-25T10:05:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.627+0000 7f027dccd640 1 --2- 172.21.15.16:0/4274803503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02780cc7b0 0x7f02780b0c60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.627+0000 7f027dccd640 1 -- 172.21.15.16:0/4274803503 >> 172.21.15.16:0/4274803503 conn(0x7f0278070ef0 msgr2=0x7f02780712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.627+0000 7f027dccd640 1 -- 172.21.15.16:0/4274803503 shutdown_connections 2024-04-25T10:05:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.627+0000 7f027dccd640 1 -- 172.21.15.16:0/4274803503 wait complete. 2024-04-25T10:05:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.628+0000 7f027dccd640 1 Processor -- start 2024-04-25T10:05:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.628+0000 7f027dccd640 1 -- start start 2024-04-25T10:05:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.629+0000 7f027dccd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02780b07d0 0x7f027815eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.629+0000 7f027dccd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02780cc7b0 0x7f02780c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.629+0000 7f027dccd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f027815bae0 0x7f02780ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.629+0000 7f027dccd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02780b47e0 con 0x7f02780cc7b0 2024-04-25T10:05:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.629+0000 7f027dccd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027815bec0 con 0x7f02780b07d0 2024-04-25T10:05:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.629+0000 7f027dccd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02780b4960 con 0x7f027815bae0 2024-04-25T10:05:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.629+0000 7f0276ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02780cc7b0 0x7f02780c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.629+0000 7f0276ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02780cc7b0 0x7f02780c6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56350/0 (socket says 172.21.15.16:56350) 2024-04-25T10:05:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.629+0000 7f0276ffd640 1 -- 172.21.15.16:0/1097440908 learned_addr learned my addr 172.21.15.16:0/1097440908 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.629+0000 7f02777fe640 1 --2- 172.21.15.16:0/1097440908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02780b07d0 0x7f027815eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.629+0000 7f0277fff640 1 --2- 172.21.15.16:0/1097440908 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f027815bae0 0x7f02780ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.629+0000 7f0276ffd640 1 -- 172.21.15.16:0/1097440908 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f027815bae0 msgr2=0x7f02780ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.629+0000 7f0276ffd640 1 --2- 172.21.15.16:0/1097440908 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f027815bae0 0x7f02780ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.629+0000 7f0276ffd640 1 -- 172.21.15.16:0/1097440908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02780b07d0 msgr2=0x7f027815eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.629+0000 7f0276ffd640 1 --2- 172.21.15.16:0/1097440908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02780b07d0 0x7f027815eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.629+0000 7f0276ffd640 1 -- 172.21.15.16:0/1097440908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02781f3a90 con 0x7f02780cc7b0 2024-04-25T10:05:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.629+0000 7f0276ffd640 1 --2- 172.21.15.16:0/1097440908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02780cc7b0 0x7f02780c6410 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f026c00c5d0 tx=0x7f026c00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.629+0000 7f025ffff640 1 -- 172.21.15.16:0/1097440908 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c017070 con 0x7f02780cc7b0 2024-04-25T10:05:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.630+0000 7f025ffff640 1 -- 172.21.15.16:0/1097440908 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c005ce0 con 0x7f02780cc7b0 2024-04-25T10:05:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.630+0000 7f027dccd640 1 -- 172.21.15.16:0/1097440908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02780caf40 con 0x7f02780cc7b0 2024-04-25T10:05:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.630+0000 7f025ffff640 1 -- 172.21.15.16:0/1097440908 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c0072b0 con 0x7f02780cc7b0 2024-04-25T10:05:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.630+0000 7f027dccd640 1 -- 172.21.15.16:0/1097440908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02780cb500 con 0x7f02780cc7b0 2024-04-25T10:05:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.631+0000 7f027dccd640 1 -- 172.21.15.16:0/1097440908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02780750b0 con 0x7f02780cc7b0 2024-04-25T10:05:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.631+0000 7f025ffff640 1 -- 172.21.15.16:0/1097440908 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f026c01a070 con 0x7f02780cc7b0 2024-04-25T10:05:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.631+0000 7f025ffff640 1 --2- 172.21.15.16:0/1097440908 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f026406b4d0 0x7f026406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.632+0000 7f025ffff640 1 -- 172.21.15.16:0/1097440908 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f026c089760 con 0x7f02780cc7b0 2024-04-25T10:05:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.632+0000 7f02777fe640 1 --2- 172.21.15.16:0/1097440908 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f026406b4d0 0x7f026406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.632+0000 7f02777fe640 1 --2- 172.21.15.16:0/1097440908 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f026406b4d0 0x7f026406d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f02680011e0 tx=0x7f0268009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.636+0000 7f025ffff640 1 -- 172.21.15.16:0/1097440908 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f026c0585e0 con 0x7f02780cc7b0 2024-04-25T10:05:26.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.817+0000 7f027dccd640 1 -- 172.21.15.16:0/1097440908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02780c6950 con 0x7f02780cc7b0 2024-04-25T10:05:26.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.817+0000 7f025ffff640 1 -- 172.21.15.16:0/1097440908 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f026c057ee0 con 0x7f02780cc7b0 2024-04-25T10:05:26.823 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:05:26.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.820+0000 7f027dccd640 1 -- 172.21.15.16:0/1097440908 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f026406b4d0 msgr2=0x7f026406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.820+0000 7f027dccd640 1 --2- 172.21.15.16:0/1097440908 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f026406b4d0 0x7f026406d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f02680011e0 tx=0x7f0268009040 comp rx=0 tx=0).stop 2024-04-25T10:05:26.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.820+0000 7f027dccd640 1 -- 172.21.15.16:0/1097440908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02780cc7b0 msgr2=0x7f02780c6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.820+0000 7f027dccd640 1 --2- 172.21.15.16:0/1097440908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02780cc7b0 0x7f02780c6410 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f026c00c5d0 tx=0x7f026c00ca90 comp rx=0 tx=0).stop 2024-04-25T10:05:26.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.820+0000 7f027dccd640 1 -- 172.21.15.16:0/1097440908 shutdown_connections 2024-04-25T10:05:26.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.820+0000 7f027dccd640 1 --2- 172.21.15.16:0/1097440908 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f026406b4d0 0x7f026406d990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.820+0000 7f027dccd640 1 --2- 172.21.15.16:0/1097440908 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f027815bae0 0x7f02780ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.820+0000 7f027dccd640 1 --2- 172.21.15.16:0/1097440908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02780cc7b0 0x7f02780c6410 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.820+0000 7f027dccd640 1 --2- 172.21.15.16:0/1097440908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02780b07d0 0x7f027815eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.820+0000 7f027dccd640 1 -- 172.21.15.16:0/1097440908 >> 172.21.15.16:0/1097440908 conn(0x7f0278070ef0 msgr2=0x7f02780b1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.820+0000 7f027dccd640 1 -- 172.21.15.16:0/1097440908 shutdown_connections 2024-04-25T10:05:26.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.820+0000 7f027dccd640 1 -- 172.21.15.16:0/1097440908 wait complete. 2024-04-25T10:05:26.836 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:26.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.931+0000 7f04d3afc640 1 Processor -- start 2024-04-25T10:05:26.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.932+0000 7f04d3afc640 1 -- start start 2024-04-25T10:05:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.932+0000 7f04d3afc640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f04cc162360 0x7f04cc0cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.932+0000 7f04d3afc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04cc0cdf60 0x7f04cc06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.932+0000 7f04d3afc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04cc06b0e0 0x7f04cc15e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.932+0000 7f04d3afc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04cc06b530 con 0x7f04cc0cdf60 2024-04-25T10:05:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.932+0000 7f04d3afc640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04cc161970 con 0x7f04cc162360 2024-04-25T10:05:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.932+0000 7f04d3afc640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc06b780 con 0x7f04cc06b0e0 2024-04-25T10:05:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.932+0000 7f04d1097640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04cc0cdf60 0x7f04cc06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.932+0000 7f04d1097640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04cc0cdf60 0x7f04cc06aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56360/0 (socket says 172.21.15.16:56360) 2024-04-25T10:05:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.932+0000 7f04d1097640 1 -- 172.21.15.16:0/3335468900 learned_addr learned my addr 172.21.15.16:0/3335468900 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.933+0000 7f04d2099640 1 --2- 172.21.15.16:0/3335468900 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04cc06b0e0 0x7f04cc15e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.933+0000 7f04d1898640 1 --2- 172.21.15.16:0/3335468900 >> v2:172.21.15.186:3300/0 conn(0x7f04cc162360 0x7f04cc0cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.933+0000 7f04d1097640 1 -- 172.21.15.16:0/3335468900 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04cc06b0e0 msgr2=0x7f04cc15e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.933+0000 7f04d1097640 1 --2- 172.21.15.16:0/3335468900 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04cc06b0e0 0x7f04cc15e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.933+0000 7f04d1097640 1 -- 172.21.15.16:0/3335468900 >> v2:172.21.15.186:3300/0 conn(0x7f04cc162360 msgr2=0x7f04cc0cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.933+0000 7f04d1097640 1 --2- 172.21.15.16:0/3335468900 >> v2:172.21.15.186:3300/0 conn(0x7f04cc162360 0x7f04cc0cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.933+0000 7f04d1097640 1 -- 172.21.15.16:0/3335468900 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc06ba60 con 0x7f04cc0cdf60 2024-04-25T10:05:26.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.933+0000 7f04d1898640 1 --2- 172.21.15.16:0/3335468900 >> v2:172.21.15.186:3300/0 conn(0x7f04cc162360 0x7f04cc0cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:26.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.933+0000 7f04d1097640 1 --2- 172.21.15.16:0/3335468900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc0cdf60 0x7f04cc06aba0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f04cc071710 tx=0x7f04c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d98a30909c19d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.934+0000 7f04d0896640 1 -- 172.21.15.16:0/3335468900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0037070 con 0x7f04cc0cdf60 2024-04-25T10:05:26.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.934+0000 7f04d0896640 1 -- 172.21.15.16:0/3335468900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c0032440 con 0x7f04cc0cdf60 2024-04-25T10:05:26.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.934+0000 7f04d0896640 1 -- 172.21.15.16:0/3335468900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0032810 con 0x7f04cc0cdf60 2024-04-25T10:05:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.934+0000 7f04d3afc640 1 -- 172.21.15.16:0/3335468900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc0cdf60 msgr2=0x7f04cc06aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.934+0000 7f04d3afc640 1 --2- 172.21.15.16:0/3335468900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc0cdf60 0x7f04cc06aba0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f04cc071710 tx=0x7f04c002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.934+0000 7f04d3afc640 1 -- 172.21.15.16:0/3335468900 shutdown_connections 2024-04-25T10:05:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.934+0000 7f04d3afc640 1 --2- 172.21.15.16:0/3335468900 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04cc06b0e0 0x7f04cc15e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.934+0000 7f04d3afc640 1 --2- 172.21.15.16:0/3335468900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc0cdf60 0x7f04cc06aba0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.934+0000 7f04d3afc640 1 --2- 172.21.15.16:0/3335468900 >> v2:172.21.15.186:3300/0 conn(0x7f04cc162360 0x7f04cc0cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.934+0000 7f04d3afc640 1 -- 172.21.15.16:0/3335468900 >> 172.21.15.16:0/3335468900 conn(0x7f04cc0cc8d0 msgr2=0x7f04cc0ccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.934+0000 7f04d3afc640 1 -- 172.21.15.16:0/3335468900 shutdown_connections 2024-04-25T10:05:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.934+0000 7f04d3afc640 1 -- 172.21.15.16:0/3335468900 wait complete. 2024-04-25T10:05:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.935+0000 7f04d3afc640 1 Processor -- start 2024-04-25T10:05:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.935+0000 7f04d3afc640 1 -- start start 2024-04-25T10:05:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.936+0000 7f04d3afc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc06b0e0 0x7f04cc0c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.936+0000 7f04d3afc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04cc0cdf60 0x7f04cc14e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.936+0000 7f04d3afc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04cc151610 0x7f04cc0c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.936+0000 7f04d3afc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc06c300 con 0x7f04cc06b0e0 2024-04-25T10:05:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.936+0000 7f04d3afc640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc06c020 con 0x7f04cc0cdf60 2024-04-25T10:05:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.936+0000 7f04d3afc640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc15e5b0 con 0x7f04cc151610 2024-04-25T10:05:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.936+0000 7f04d1898640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc06b0e0 0x7f04cc0c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.936+0000 7f04d1898640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc06b0e0 0x7f04cc0c4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56374/0 (socket says 172.21.15.16:56374) 2024-04-25T10:05:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.936+0000 7f04d1898640 1 -- 172.21.15.16:0/1120843083 learned_addr learned my addr 172.21.15.16:0/1120843083 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.936+0000 7f04d1097640 1 --2- 172.21.15.16:0/1120843083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04cc0cdf60 0x7f04cc14e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.936+0000 7f04d2099640 1 --2- 172.21.15.16:0/1120843083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04cc151610 0x7f04cc0c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.937+0000 7f04d1898640 1 -- 172.21.15.16:0/1120843083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04cc151610 msgr2=0x7f04cc0c62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.937+0000 7f04d1898640 1 --2- 172.21.15.16:0/1120843083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04cc151610 0x7f04cc0c62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.937+0000 7f04d1898640 1 -- 172.21.15.16:0/1120843083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04cc0cdf60 msgr2=0x7f04cc14e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.937+0000 7f04d1898640 1 --2- 172.21.15.16:0/1120843083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04cc0cdf60 0x7f04cc14e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.937+0000 7f04d1898640 1 -- 172.21.15.16:0/1120843083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc14ec70 con 0x7f04cc06b0e0 2024-04-25T10:05:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.937+0000 7f04d1898640 1 --2- 172.21.15.16:0/1120843083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc06b0e0 0x7f04cc0c4360 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009fc0 tx=0x7f04c400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.937+0000 7f04ca7fc640 1 -- 172.21.15.16:0/1120843083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c401a070 con 0x7f04cc06b0e0 2024-04-25T10:05:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.937+0000 7f04ca7fc640 1 -- 172.21.15.16:0/1120843083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c4003bb0 con 0x7f04cc06b0e0 2024-04-25T10:05:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.937+0000 7f04d3afc640 1 -- 172.21.15.16:0/1120843083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04cc0c4bb0 con 0x7f04cc06b0e0 2024-04-25T10:05:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.937+0000 7f04ca7fc640 1 -- 172.21.15.16:0/1120843083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c4004030 con 0x7f04cc06b0e0 2024-04-25T10:05:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.937+0000 7f04d3afc640 1 -- 172.21.15.16:0/1120843083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04cc0c48a0 con 0x7f04cc06b0e0 2024-04-25T10:05:26.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.938+0000 7f04d3afc640 1 -- 172.21.15.16:0/1120843083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04cc0749b0 con 0x7f04cc06b0e0 2024-04-25T10:05:26.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.938+0000 7f04ca7fc640 1 -- 172.21.15.16:0/1120843083 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f04c401d070 con 0x7f04cc06b0e0 2024-04-25T10:05:26.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.939+0000 7f04ca7fc640 1 --2- 172.21.15.16:0/1120843083 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f04b806b4d0 0x7f04b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.939+0000 7f04ca7fc640 1 -- 172.21.15.16:0/1120843083 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04c408a3f0 con 0x7f04cc06b0e0 2024-04-25T10:05:26.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.939+0000 7f04d1097640 1 --2- 172.21.15.16:0/1120843083 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f04b806b4d0 0x7f04b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.940+0000 7f04d1097640 1 --2- 172.21.15.16:0/1120843083 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f04b806b4d0 0x7f04b806d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f04c0034000 tx=0x7f04c002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:26.943+0000 7f04ca7fc640 1 -- 172.21.15.16:0/1120843083 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f04c4010070 con 0x7f04cc06b0e0 2024-04-25T10:05:27.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.158+0000 7f04d3afc640 1 -- 172.21.15.16:0/1120843083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04cc066a30 con 0x7f04cc06b0e0 2024-04-25T10:05:27.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.158+0000 7f04ca7fc640 1 -- 172.21.15.16:0/1120843083 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4939 (secure 0 0 0) 0x7f04c4058cb0 con 0x7f04cc06b0e0 2024-04-25T10:05:27.164 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:27.164 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1965168813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1965168813},{"type":"v1","addr":"172.21.15.16:6834","nonce":1965168813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/437791378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":437791378},{"type":"v1","addr":"172.21.15.16:6835","nonce":437791378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34145,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3044177642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3044177642},{"type":"v1","addr":"172.21.15.186:6835","nonce":3044177642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:10.814281+0000","modified":"2024-04-25T10:05:12.805138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34148":{"gid":34148,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/1763853189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1763853189},{"type":"v1","addr":"172.21.15.186:6837","nonce":1763853189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:05:27.164 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-04-25T10:05:27.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.160+0000 7f04d3afc640 1 -- 172.21.15.16:0/1120843083 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f04b806b4d0 msgr2=0x7f04b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.160+0000 7f04d3afc640 1 --2- 172.21.15.16:0/1120843083 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f04b806b4d0 0x7f04b806d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f04c0034000 tx=0x7f04c002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:27.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.160+0000 7f04d3afc640 1 -- 172.21.15.16:0/1120843083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc06b0e0 msgr2=0x7f04cc0c4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.160+0000 7f04d3afc640 1 --2- 172.21.15.16:0/1120843083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc06b0e0 0x7f04cc0c4360 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009fc0 tx=0x7f04c400d560 comp rx=0 tx=0).stop 2024-04-25T10:05:27.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.160+0000 7f04d3afc640 1 -- 172.21.15.16:0/1120843083 shutdown_connections 2024-04-25T10:05:27.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.160+0000 7f04d3afc640 1 --2- 172.21.15.16:0/1120843083 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f04b806b4d0 0x7f04b806d990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.160+0000 7f04d3afc640 1 --2- 172.21.15.16:0/1120843083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04cc151610 0x7f04cc0c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.160+0000 7f04d3afc640 1 --2- 172.21.15.16:0/1120843083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04cc0cdf60 0x7f04cc14e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.160+0000 7f04d3afc640 1 --2- 172.21.15.16:0/1120843083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc06b0e0 0x7f04cc0c4360 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.160+0000 7f04d3afc640 1 -- 172.21.15.16:0/1120843083 >> 172.21.15.16:0/1120843083 conn(0x7f04cc0cc8d0 msgr2=0x7f04cc0adf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.160+0000 7f04d3afc640 1 -- 172.21.15.16:0/1120843083 shutdown_connections 2024-04-25T10:05:27.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.160+0000 7f04d3afc640 1 -- 172.21.15.16:0/1120843083 wait complete. 2024-04-25T10:05:27.176 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:27.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.271+0000 7f0a5441b640 1 Processor -- start 2024-04-25T10:05:27.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.271+0000 7f0a5441b640 1 -- start start 2024-04-25T10:05:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.272+0000 7f0a5441b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a4c0b5b60 0x7f0a4c1687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.272+0000 7f0a5441b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a4c15e0c0 con 0x7f0a4c15dce0 2024-04-25T10:05:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.272+0000 7f0a5441b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a4c0d2510 con 0x7f0a4c0d35b0 2024-04-25T10:05:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.272+0000 7f0a5441b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0b7b10 con 0x7f0a4c0b5b60 2024-04-25T10:05:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.272+0000 7f0a521b7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0a4c15dce0 0x7f0a4c0d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35230/0 (socket says 172.21.15.16:35230) 2024-04-25T10:05:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.272+0000 7f0a521b7640 1 -- 172.21.15.16:0/2534238208 learned_addr learned my addr 172.21.15.16:0/2534238208 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.273+0000 7f0a529b8640 1 --2- 172.21.15.16:0/2534238208 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a4c0b5b60 0x7f0a4c1687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.273+0000 7f0a511b5640 1 -- 172.21.15.16:0/2534238208 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1398415776 0 0) 0x7f0a4c15e0c0 con 0x7f0a4c15dce0 2024-04-25T10:05:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.273+0000 7f0a511b5640 1 -- 172.21.15.16:0/2534238208 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a38003520 con 0x7f0a4c15dce0 2024-04-25T10:05:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.273+0000 7f0a511b5640 1 -- 172.21.15.16:0/2534238208 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4016228517 0 0) 0x7f0a4c0d2510 con 0x7f0a4c0d35b0 2024-04-25T10:05:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.273+0000 7f0a511b5640 1 -- 172.21.15.16:0/2534238208 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a4c15e0c0 con 0x7f0a4c0d35b0 2024-04-25T10:05:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.273+0000 7f0a529b8640 1 -- 172.21.15.16:0/2534238208 >> v1:172.21.15.186:6789/0 conn(0x7f0a4c0d35b0 legacy=0x7f0a4c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.273+0000 7f0a529b8640 1 -- 172.21.15.16:0/2534238208 >> v1:172.21.15.16:6789/0 conn(0x7f0a4c15dce0 legacy=0x7f0a4c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.273+0000 7f0a529b8640 1 -- 172.21.15.16:0/2534238208 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c0d27f0 con 0x7f0a4c0b5b60 2024-04-25T10:05:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.274+0000 7f0a529b8640 1 --2- 172.21.15.16:0/2534238208 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a4c0b5b60 0x7f0a4c1687f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009dc0 tx=0x7f0a4402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=186a98c81afedc82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.274+0000 7f0a511b5640 1 -- 172.21.15.16:0/2534238208 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44007740 con 0x7f0a4c0b5b60 2024-04-25T10:05:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.274+0000 7f0a511b5640 1 -- 172.21.15.16:0/2534238208 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a440078e0 con 0x7f0a4c0b5b60 2024-04-25T10:05:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.274+0000 7f0a511b5640 1 -- 172.21.15.16:0/2534238208 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44007c90 con 0x7f0a4c0b5b60 2024-04-25T10:05:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.274+0000 7f0a5441b640 1 -- 172.21.15.16:0/2534238208 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a4c0b5b60 msgr2=0x7f0a4c1687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.274+0000 7f0a5441b640 1 --2- 172.21.15.16:0/2534238208 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a4c0b5b60 0x7f0a4c1687f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009dc0 tx=0x7f0a4402a6d0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.275+0000 7f0a5441b640 1 -- 172.21.15.16:0/2534238208 shutdown_connections 2024-04-25T10:05:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.275+0000 7f0a5441b640 1 --2- 172.21.15.16:0/2534238208 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a4c0b5b60 0x7f0a4c1687f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.275+0000 7f0a5441b640 1 -- 172.21.15.16:0/2534238208 >> 172.21.15.16:0/2534238208 conn(0x7f0a4c072900 msgr2=0x7f0a4c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.275+0000 7f0a5441b640 1 -- 172.21.15.16:0/2534238208 shutdown_connections 2024-04-25T10:05:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.275+0000 7f0a5441b640 1 -- 172.21.15.16:0/2534238208 wait complete. 2024-04-25T10:05:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.276+0000 7f0a5441b640 1 Processor -- start 2024-04-25T10:05:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.276+0000 7f0a5441b640 1 -- start start 2024-04-25T10:05:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.276+0000 7f0a5441b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a4c0b5b60 0x7f0a4c0cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.276+0000 7f0a5441b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a4c0d35b0 0x7f0a4c160d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.276+0000 7f0a5441b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a4c15dce0 0x7f0a4c143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.276+0000 7f0a5441b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0d3090 con 0x7f0a4c15dce0 2024-04-25T10:05:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.276+0000 7f0a5441b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0d2db0 con 0x7f0a4c0b5b60 2024-04-25T10:05:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.276+0000 7f0a5441b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0d2160 con 0x7f0a4c0d35b0 2024-04-25T10:05:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.277+0000 7f0a529b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a4c15dce0 0x7f0a4c143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.277+0000 7f0a529b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a4c15dce0 0x7f0a4c143c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56380/0 (socket says 172.21.15.16:56380) 2024-04-25T10:05:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.277+0000 7f0a529b8640 1 -- 172.21.15.16:0/3054204015 learned_addr learned my addr 172.21.15.16:0/3054204015 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.277+0000 7f0a521b7640 1 --2- 172.21.15.16:0/3054204015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a4c0b5b60 0x7f0a4c0cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.277+0000 7f0a519b6640 1 --2- 172.21.15.16:0/3054204015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a4c0d35b0 0x7f0a4c160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.277+0000 7f0a529b8640 1 -- 172.21.15.16:0/3054204015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a4c0d35b0 msgr2=0x7f0a4c160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.277+0000 7f0a529b8640 1 --2- 172.21.15.16:0/3054204015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a4c0d35b0 0x7f0a4c160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.277+0000 7f0a529b8640 1 -- 172.21.15.16:0/3054204015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a4c0b5b60 msgr2=0x7f0a4c0cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.277+0000 7f0a529b8640 1 --2- 172.21.15.16:0/3054204015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a4c0b5b60 0x7f0a4c0cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.277+0000 7f0a529b8640 1 -- 172.21.15.16:0/3054204015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c1612a0 con 0x7f0a4c15dce0 2024-04-25T10:05:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.277+0000 7f0a529b8640 1 --2- 172.21.15.16:0/3054204015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a4c15dce0 0x7f0a4c143c90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0a4402acc0 tx=0x7f0a44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.277+0000 7f0a42ffd640 1 -- 172.21.15.16:0/3054204015 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44040070 con 0x7f0a4c15dce0 2024-04-25T10:05:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.277+0000 7f0a42ffd640 1 -- 172.21.15.16:0/3054204015 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a44031c40 con 0x7f0a4c15dce0 2024-04-25T10:05:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.277+0000 7f0a5441b640 1 -- 172.21.15.16:0/3054204015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a4c153f00 con 0x7f0a4c15dce0 2024-04-25T10:05:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.277+0000 7f0a42ffd640 1 -- 172.21.15.16:0/3054204015 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44009450 con 0x7f0a4c15dce0 2024-04-25T10:05:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.277+0000 7f0a5441b640 1 -- 172.21.15.16:0/3054204015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a4c0cc880 con 0x7f0a4c15dce0 2024-04-25T10:05:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.278+0000 7f0a5441b640 1 -- 172.21.15.16:0/3054204015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a4c0749b0 con 0x7f0a4c15dce0 2024-04-25T10:05:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.279+0000 7f0a42ffd640 1 -- 172.21.15.16:0/3054204015 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a44039040 con 0x7f0a4c15dce0 2024-04-25T10:05:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.279+0000 7f0a42ffd640 1 --2- 172.21.15.16:0/3054204015 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0a3006b4d0 0x7f0a3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.279+0000 7f0a42ffd640 1 -- 172.21.15.16:0/3054204015 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a440ab450 con 0x7f0a4c15dce0 2024-04-25T10:05:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.279+0000 7f0a521b7640 1 --2- 172.21.15.16:0/3054204015 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0a3006b4d0 0x7f0a3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.280+0000 7f0a521b7640 1 --2- 172.21.15.16:0/3054204015 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0a3006b4d0 0x7f0a3006d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0a48000f60 tx=0x7f0a48009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.283+0000 7f0a42ffd640 1 -- 172.21.15.16:0/3054204015 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a4404c070 con 0x7f0a4c15dce0 2024-04-25T10:05:27.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.462+0000 7f0a5441b640 1 -- 172.21.15.16:0/3054204015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a4c156df0 con 0x7f0a4c15dce0 2024-04-25T10:05:27.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.463+0000 7f0a42ffd640 1 -- 172.21.15.16:0/3054204015 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f0a44038350 con 0x7f0a4c15dce0 2024-04-25T10:05:27.469 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:27.469 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:12.449183+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.466+0000 7f0a5441b640 1 -- 172.21.15.16:0/3054204015 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0a3006b4d0 msgr2=0x7f0a3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.466+0000 7f0a5441b640 1 --2- 172.21.15.16:0/3054204015 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0a3006b4d0 0x7f0a3006d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0a48000f60 tx=0x7f0a48009040 comp rx=0 tx=0).stop 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.466+0000 7f0a5441b640 1 -- 172.21.15.16:0/3054204015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a4c15dce0 msgr2=0x7f0a4c143c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.466+0000 7f0a5441b640 1 --2- 172.21.15.16:0/3054204015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a4c15dce0 0x7f0a4c143c90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0a4402acc0 tx=0x7f0a44004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.466+0000 7f0a5441b640 1 -- 172.21.15.16:0/3054204015 shutdown_connections 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.466+0000 7f0a5441b640 1 --2- 172.21.15.16:0/3054204015 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0a3006b4d0 0x7f0a3006d990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.466+0000 7f0a5441b640 1 --2- 172.21.15.16:0/3054204015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a4c15dce0 0x7f0a4c143c90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.466+0000 7f0a5441b640 1 --2- 172.21.15.16:0/3054204015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a4c0d35b0 0x7f0a4c160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.466+0000 7f0a5441b640 1 --2- 172.21.15.16:0/3054204015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a4c0b5b60 0x7f0a4c0cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.466+0000 7f0a5441b640 1 -- 172.21.15.16:0/3054204015 >> 172.21.15.16:0/3054204015 conn(0x7f0a4c072900 msgr2=0x7f0a4c0d3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.466+0000 7f0a5441b640 1 -- 172.21.15.16:0/3054204015 shutdown_connections 2024-04-25T10:05:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.466+0000 7f0a5441b640 1 -- 172.21.15.16:0/3054204015 wait complete. 2024-04-25T10:05:27.483 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:27.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.578+0000 7f79f726f640 1 Processor -- start 2024-04-25T10:05:27.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.578+0000 7f79f726f640 1 -- start start 2024-04-25T10:05:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.579+0000 7f79f726f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f79f00d35b0 0x7f79f00b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.579+0000 7f79f726f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f015e0c0 con 0x7f79f00b5990 2024-04-25T10:05:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.579+0000 7f79f726f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f00d2510 con 0x7f79f015dce0 2024-04-25T10:05:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.579+0000 7f79f726f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f00d27f0 con 0x7f79f00d35b0 2024-04-25T10:05:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.579+0000 7f79f580c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f79f00b5990 0x7f79f00b6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35242/0 (socket says 172.21.15.16:35242) 2024-04-25T10:05:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.579+0000 7f79f580c640 1 -- 172.21.15.16:0/1244031812 learned_addr learned my addr 172.21.15.16:0/1244031812 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.579+0000 7f79f480a640 1 --2- 172.21.15.16:0/1244031812 >> v2:172.21.15.186:3300/0 conn(0x7f79f00d35b0 0x7f79f00b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.579+0000 7f79e7fff640 1 -- 172.21.15.16:0/1244031812 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969619160 0 0) 0x7f79f015e0c0 con 0x7f79f00b5990 2024-04-25T10:05:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.579+0000 7f79e7fff640 1 -- 172.21.15.16:0/1244031812 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79dc003520 con 0x7f79f00b5990 2024-04-25T10:05:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79e7fff640 1 -- 172.21.15.16:0/1244031812 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3382095664 0 0) 0x7f79dc003520 con 0x7f79f00b5990 2024-04-25T10:05:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79e7fff640 1 -- 172.21.15.16:0/1244031812 >> v2:172.21.15.186:3300/0 conn(0x7f79f00d35b0 msgr2=0x7f79f00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79e7fff640 1 --2- 172.21.15.16:0/1244031812 >> v2:172.21.15.186:3300/0 conn(0x7f79f00d35b0 0x7f79f00b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79e7fff640 1 -- 172.21.15.16:0/1244031812 >> v1:172.21.15.186:6789/0 conn(0x7f79f015dce0 legacy=0x7f79f00d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79e7fff640 1 -- 172.21.15.16:0/1244031812 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f00d2ad0 con 0x7f79f00b5990 2024-04-25T10:05:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79e7fff640 1 -- 172.21.15.16:0/1244031812 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f79e80019f0 con 0x7f79f00b5990 2024-04-25T10:05:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79e7fff640 1 -- 172.21.15.16:0/1244031812 >> v1:172.21.15.16:6789/0 conn(0x7f79f00b5990 legacy=0x7f79f00b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79e7fff640 1 --2- 172.21.15.16:0/1244031812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79dc0039b0 0x7f79dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79e7fff640 1 --2- 172.21.15.16:0/1244031812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79dc004800 0x7f79dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79e7fff640 1 --2- 172.21.15.16:0/1244031812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79dc0051c0 0x7f79dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79e7fff640 1 -- 172.21.15.16:0/1244031812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00d27f0 con 0x7f79dc0039b0 2024-04-25T10:05:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79e7fff640 1 -- 172.21.15.16:0/1244031812 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc00c060 con 0x7f79dc004800 2024-04-25T10:05:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79e7fff640 1 -- 172.21.15.16:0/1244031812 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc00c260 con 0x7f79dc0051c0 2024-04-25T10:05:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79f500b640 1 --2- 172.21.15.16:0/1244031812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79dc0039b0 0x7f79dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79f480a640 1 --2- 172.21.15.16:0/1244031812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79dc004800 0x7f79dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79f580c640 1 --2- 172.21.15.16:0/1244031812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79dc0051c0 0x7f79dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79f500b640 1 -- 172.21.15.16:0/1244031812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79dc0051c0 msgr2=0x7f79dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79f500b640 1 --2- 172.21.15.16:0/1244031812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79dc0051c0 0x7f79dc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79f500b640 1 -- 172.21.15.16:0/1244031812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79dc004800 msgr2=0x7f79dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79f500b640 1 --2- 172.21.15.16:0/1244031812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79dc004800 0x7f79dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79f500b640 1 -- 172.21.15.16:0/1244031812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc003520 con 0x7f79dc0039b0 2024-04-25T10:05:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.580+0000 7f79f580c640 1 -- 172.21.15.16:0/1244031812 reap_dead start 2024-04-25T10:05:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.581+0000 7f79f500b640 1 --2- 172.21.15.16:0/1244031812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79dc0039b0 0x7f79dc003dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f79ec001750 tx=0x7f79dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fabd0a81385f39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.581+0000 7f79e7fff640 1 -- 172.21.15.16:0/1244031812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec001370 con 0x7f79dc0039b0 2024-04-25T10:05:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.581+0000 7f79e7fff640 1 -- 172.21.15.16:0/1244031812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec0052d0 con 0x7f79dc0039b0 2024-04-25T10:05:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.581+0000 7f79e7fff640 1 -- 172.21.15.16:0/1244031812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec02abe0 con 0x7f79dc0039b0 2024-04-25T10:05:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.582+0000 7f79f726f640 1 -- 172.21.15.16:0/1244031812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79dc0039b0 msgr2=0x7f79dc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.582+0000 7f79f726f640 1 --2- 172.21.15.16:0/1244031812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79dc0039b0 0x7f79dc003dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f79ec001750 tx=0x7f79dc005830 comp rx=0 tx=0).stop 2024-04-25T10:05:27.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.582+0000 7f79f726f640 1 -- 172.21.15.16:0/1244031812 shutdown_connections 2024-04-25T10:05:27.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.582+0000 7f79f726f640 1 --2- 172.21.15.16:0/1244031812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79dc0039b0 0x7f79dc003dd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.582+0000 7f79f726f640 1 -- 172.21.15.16:0/1244031812 >> 172.21.15.16:0/1244031812 conn(0x7f79f0072900 msgr2=0x7f79f00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.582+0000 7f79f726f640 1 -- 172.21.15.16:0/1244031812 shutdown_connections 2024-04-25T10:05:27.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.582+0000 7f79f726f640 1 -- 172.21.15.16:0/1244031812 wait complete. 2024-04-25T10:05:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.583+0000 7f79f726f640 1 Processor -- start 2024-04-25T10:05:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.583+0000 7f79f726f640 1 -- start start 2024-04-25T10:05:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79f726f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79dc0039b0 0x7f79f00b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79f726f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79f014cd90 0x7f79f01608d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79f726f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79f00b4870 0x7f79f00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79f726f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00b9820 con 0x7f79f014cd90 2024-04-25T10:05:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79f726f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00d3090 con 0x7f79f00b4870 2024-04-25T10:05:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79f726f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00b99a0 con 0x7f79dc0039b0 2024-04-25T10:05:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79f480a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79f014cd90 0x7f79f01608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79f480a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79f014cd90 0x7f79f01608d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56404/0 (socket says 172.21.15.16:56404) 2024-04-25T10:05:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79f480a640 1 -- 172.21.15.16:0/1799363128 learned_addr learned my addr 172.21.15.16:0/1799363128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79f500b640 1 --2- 172.21.15.16:0/1799363128 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79dc0039b0 0x7f79f00b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79f580c640 1 --2- 172.21.15.16:0/1799363128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79f00b4870 0x7f79f00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79f480a640 1 -- 172.21.15.16:0/1799363128 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79dc0039b0 msgr2=0x7f79f00b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79f480a640 1 --2- 172.21.15.16:0/1799363128 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79dc0039b0 0x7f79f00b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79f480a640 1 -- 172.21.15.16:0/1799363128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79f00b4870 msgr2=0x7f79f00b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79f480a640 1 --2- 172.21.15.16:0/1799363128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79f00b4870 0x7f79f00b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79f480a640 1 -- 172.21.15.16:0/1799363128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f015e160 con 0x7f79f014cd90 2024-04-25T10:05:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79f480a640 1 --2- 172.21.15.16:0/1799363128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79f014cd90 0x7f79f01608d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f79e0011890 tx=0x7f79e0011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79e5ffb640 1 -- 172.21.15.16:0/1799363128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0019070 con 0x7f79f014cd90 2024-04-25T10:05:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79e5ffb640 1 -- 172.21.15.16:0/1799363128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e0004240 con 0x7f79f014cd90 2024-04-25T10:05:27.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79e5ffb640 1 -- 172.21.15.16:0/1799363128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e000f820 con 0x7f79f014cd90 2024-04-25T10:05:27.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.584+0000 7f79f726f640 1 -- 172.21.15.16:0/1799363128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f00b39d0 con 0x7f79f014cd90 2024-04-25T10:05:27.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.585+0000 7f79f726f640 1 -- 172.21.15.16:0/1799363128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f015e400 con 0x7f79f014cd90 2024-04-25T10:05:27.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.585+0000 7f79f726f640 1 -- 172.21.15.16:0/1799363128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f00749b0 con 0x7f79f014cd90 2024-04-25T10:05:27.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.586+0000 7f79e5ffb640 1 -- 172.21.15.16:0/1799363128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79e001c070 con 0x7f79f014cd90 2024-04-25T10:05:27.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.586+0000 7f79e5ffb640 1 --2- 172.21.15.16:0/1799363128 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f79d406b4d0 0x7f79d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.586+0000 7f79e5ffb640 1 -- 172.21.15.16:0/1799363128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79e008a4f0 con 0x7f79f014cd90 2024-04-25T10:05:27.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.587+0000 7f79f500b640 1 --2- 172.21.15.16:0/1799363128 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f79d406b4d0 0x7f79d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.587+0000 7f79f500b640 1 --2- 172.21.15.16:0/1799363128 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f79d406b4d0 0x7f79d406d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f79ec032000 tx=0x7f79ec02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.591+0000 7f79e5ffb640 1 -- 172.21.15.16:0/1799363128 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79e0058c80 con 0x7f79f014cd90 2024-04-25T10:05:27.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.807+0000 7f79f726f640 1 -- 172.21.15.16:0/1799363128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79f015d2b0 con 0x7f79f014cd90 2024-04-25T10:05:27.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.807+0000 7f79e5ffb640 1 -- 172.21.15.16:0/1799363128 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4939 (secure 0 0 0) 0x7f79e005cd20 con 0x7f79f014cd90 2024-04-25T10:05:27.813 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:27.813 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1965168813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1965168813},{"type":"v1","addr":"172.21.15.16:6834","nonce":1965168813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/437791378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":437791378},{"type":"v1","addr":"172.21.15.16:6835","nonce":437791378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34145,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3044177642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3044177642},{"type":"v1","addr":"172.21.15.186:6835","nonce":3044177642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:10.814281+0000","modified":"2024-04-25T10:05:12.805138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34148":{"gid":34148,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/1763853189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1763853189},{"type":"v1","addr":"172.21.15.186:6837","nonce":1763853189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:05:27.813 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-04-25T10:05:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.809+0000 7f79f726f640 1 -- 172.21.15.16:0/1799363128 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f79d406b4d0 msgr2=0x7f79d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.809+0000 7f79f726f640 1 --2- 172.21.15.16:0/1799363128 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f79d406b4d0 0x7f79d406d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f79ec032000 tx=0x7f79ec02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.809+0000 7f79f726f640 1 -- 172.21.15.16:0/1799363128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79f014cd90 msgr2=0x7f79f01608d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.809+0000 7f79f726f640 1 --2- 172.21.15.16:0/1799363128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79f014cd90 0x7f79f01608d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f79e0011890 tx=0x7f79e0011d50 comp rx=0 tx=0).stop 2024-04-25T10:05:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.809+0000 7f79f726f640 1 -- 172.21.15.16:0/1799363128 shutdown_connections 2024-04-25T10:05:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.809+0000 7f79f726f640 1 --2- 172.21.15.16:0/1799363128 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f79d406b4d0 0x7f79d406d990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.809+0000 7f79f726f640 1 --2- 172.21.15.16:0/1799363128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79f00b4870 0x7f79f00b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.809+0000 7f79f726f640 1 --2- 172.21.15.16:0/1799363128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79f014cd90 0x7f79f01608d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.809+0000 7f79f726f640 1 --2- 172.21.15.16:0/1799363128 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79dc0039b0 0x7f79f00b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.809+0000 7f79f726f640 1 -- 172.21.15.16:0/1799363128 >> 172.21.15.16:0/1799363128 conn(0x7f79f0072900 msgr2=0x7f79f00b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.809+0000 7f79f726f640 1 -- 172.21.15.16:0/1799363128 shutdown_connections 2024-04-25T10:05:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:27.809+0000 7f79f726f640 1 -- 172.21.15.16:0/1799363128 wait complete. 2024-04-25T10:05:28.228 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:05:10.814281+0000', 'modified': '2024-04-25T10:05:12.805138+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34148}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34148': {'gid': 34148, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.186:6837/1763853189', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 1763853189}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 1763853189}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:05:28.228 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:05:28.228 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:05:28.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.327+0000 7f2056996640 1 Processor -- start 2024-04-25T10:05:28.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.327+0000 7f2056996640 1 -- start start 2024-04-25T10:05:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.328+0000 7f2056996640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2050071590 0x7f20500cc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.328+0000 7f2056996640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20500cd940 0x7f20500ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.328+0000 7f2056996640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205006c220 con 0x7f205015dc50 2024-04-25T10:05:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.328+0000 7f2056996640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20500cfe00 con 0x7f2050071590 2024-04-25T10:05:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.328+0000 7f2056996640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20500cec10 con 0x7f20500cd940 2024-04-25T10:05:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.328+0000 7f204ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f205015dc50 0x7f2050070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35252/0 (socket says 172.21.15.16:35252) 2024-04-25T10:05:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.328+0000 7f204ffff640 1 -- 172.21.15.16:0/2515164383 learned_addr learned my addr 172.21.15.16:0/2515164383 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.328+0000 7f204effd640 1 -- 172.21.15.16:0/2515164383 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001834143 0 0) 0x7f205006c220 con 0x7f205015dc50 2024-04-25T10:05:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204f7fe640 1 --2- 172.21.15.16:0/2515164383 >> v2:172.21.15.186:3300/0 conn(0x7f2050071590 0x7f20500cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204effd640 1 -- 172.21.15.16:0/2515164383 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2038003520 con 0x7f205015dc50 2024-04-25T10:05:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f2054f33640 1 --2- 172.21.15.16:0/2515164383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20500cd940 0x7f20500ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204effd640 1 -- 172.21.15.16:0/2515164383 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 626441239 0 0) 0x7f2038003520 con 0x7f205015dc50 2024-04-25T10:05:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204effd640 1 -- 172.21.15.16:0/2515164383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20500cd940 msgr2=0x7f20500ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204effd640 1 --2- 172.21.15.16:0/2515164383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20500cd940 0x7f20500ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204effd640 1 -- 172.21.15.16:0/2515164383 >> v2:172.21.15.186:3300/0 conn(0x7f2050071590 msgr2=0x7f20500cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204effd640 1 --2- 172.21.15.16:0/2515164383 >> v2:172.21.15.186:3300/0 conn(0x7f2050071590 0x7f20500cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204effd640 1 -- 172.21.15.16:0/2515164383 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20500d0050 con 0x7f205015dc50 2024-04-25T10:05:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204effd640 1 -- 172.21.15.16:0/2515164383 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f2044001670 con 0x7f205015dc50 2024-04-25T10:05:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204effd640 1 -- 172.21.15.16:0/2515164383 >> v1:172.21.15.16:6789/0 conn(0x7f205015dc50 legacy=0x7f2050070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204effd640 1 --2- 172.21.15.16:0/2515164383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20500cd940 0x7f20380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204effd640 1 --2- 172.21.15.16:0/2515164383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2038003f30 0x7f2038004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204effd640 1 --2- 172.21.15.16:0/2515164383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2038004d40 0x7f20380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204effd640 1 -- 172.21.15.16:0/2515164383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20500cfe00 con 0x7f2038003f30 2024-04-25T10:05:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204effd640 1 -- 172.21.15.16:0/2515164383 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20500cec10 con 0x7f2038004d40 2024-04-25T10:05:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204effd640 1 -- 172.21.15.16:0/2515164383 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203800ba10 con 0x7f20500cd940 2024-04-25T10:05:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204f7fe640 1 --2- 172.21.15.16:0/2515164383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2038003f30 0x7f2038004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204ffff640 1 --2- 172.21.15.16:0/2515164383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20500cd940 0x7f20380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f2054f33640 1 --2- 172.21.15.16:0/2515164383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2038004d40 0x7f20380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204f7fe640 1 -- 172.21.15.16:0/2515164383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20500cd940 msgr2=0x7f20380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204f7fe640 1 --2- 172.21.15.16:0/2515164383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20500cd940 0x7f20380039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204f7fe640 1 -- 172.21.15.16:0/2515164383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2038004d40 msgr2=0x7f20380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204f7fe640 1 --2- 172.21.15.16:0/2515164383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2038004d40 0x7f20380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.329+0000 7f204f7fe640 1 -- 172.21.15.16:0/2515164383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2038003520 con 0x7f2038003f30 2024-04-25T10:05:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.330+0000 7f204f7fe640 1 --2- 172.21.15.16:0/2515164383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2038003f30 0x7f2038004800 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2040009af0 tx=0x7f2038005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbc4e02205da8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.330+0000 7f204effd640 1 -- 172.21.15.16:0/2515164383 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040036070 con 0x7f2038003f30 2024-04-25T10:05:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.330+0000 7f204effd640 1 -- 172.21.15.16:0/2515164383 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2040031400 con 0x7f2038003f30 2024-04-25T10:05:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.330+0000 7f204effd640 1 -- 172.21.15.16:0/2515164383 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040009690 con 0x7f2038003f30 2024-04-25T10:05:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.331+0000 7f2056996640 1 -- 172.21.15.16:0/2515164383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2038003f30 msgr2=0x7f2038004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.331+0000 7f2056996640 1 --2- 172.21.15.16:0/2515164383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2038003f30 0x7f2038004800 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2040009af0 tx=0x7f2038005830 comp rx=0 tx=0).stop 2024-04-25T10:05:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.331+0000 7f2054f33640 1 -- 172.21.15.16:0/2515164383 reap_dead start 2024-04-25T10:05:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.331+0000 7f2056996640 1 -- 172.21.15.16:0/2515164383 shutdown_connections 2024-04-25T10:05:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.331+0000 7f2056996640 1 -- 172.21.15.16:0/2515164383 >> 172.21.15.16:0/2515164383 conn(0x7f205006f430 msgr2=0x7f205006f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.331+0000 7f2056996640 1 -- 172.21.15.16:0/2515164383 shutdown_connections 2024-04-25T10:05:28.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.331+0000 7f2056996640 1 -- 172.21.15.16:0/2515164383 wait complete. 2024-04-25T10:05:28.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.332+0000 7f2056996640 1 Processor -- start 2024-04-25T10:05:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.332+0000 7f2056996640 1 -- start start 2024-04-25T10:05:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.333+0000 7f2056996640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2038003f30 0x7f205015f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.333+0000 7f2056996640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20500caed0 0x7f20500c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.333+0000 7f2056996640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2050154b50 0x7f20500b6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.333+0000 7f2056996640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20500ce8a0 con 0x7f2038003f30 2024-04-25T10:05:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.333+0000 7f2056996640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20500cc0c0 con 0x7f20500caed0 2024-04-25T10:05:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.333+0000 7f2056996640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205015e030 con 0x7f2050154b50 2024-04-25T10:05:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.333+0000 7f204ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2038003f30 0x7f205015f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.333+0000 7f204ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2038003f30 0x7f205015f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56420/0 (socket says 172.21.15.16:56420) 2024-04-25T10:05:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.333+0000 7f204ffff640 1 -- 172.21.15.16:0/953734688 learned_addr learned my addr 172.21.15.16:0/953734688 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.333+0000 7f2054f33640 1 --2- 172.21.15.16:0/953734688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2050154b50 0x7f20500b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.333+0000 7f204ffff640 1 -- 172.21.15.16:0/953734688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2050154b50 msgr2=0x7f20500b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.333+0000 7f204ffff640 1 --2- 172.21.15.16:0/953734688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2050154b50 0x7f20500b6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.333+0000 7f204f7fe640 1 --2- 172.21.15.16:0/953734688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20500caed0 0x7f20500c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.333+0000 7f204ffff640 1 -- 172.21.15.16:0/953734688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20500caed0 msgr2=0x7f20500c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.333+0000 7f204ffff640 1 --2- 172.21.15.16:0/953734688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20500caed0 0x7f20500c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.333+0000 7f204ffff640 1 -- 172.21.15.16:0/953734688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20500cb330 con 0x7f2038003f30 2024-04-25T10:05:28.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.333+0000 7f204ffff640 1 --2- 172.21.15.16:0/953734688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2038003f30 0x7f205015f380 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f20440073a0 tx=0x7f2044007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.333+0000 7f204cff9640 1 -- 172.21.15.16:0/953734688 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204401a070 con 0x7f2038003f30 2024-04-25T10:05:28.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.334+0000 7f204cff9640 1 -- 172.21.15.16:0/953734688 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204400f960 con 0x7f2038003f30 2024-04-25T10:05:28.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.334+0000 7f2056996640 1 -- 172.21.15.16:0/953734688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20500cb5d0 con 0x7f2038003f30 2024-04-25T10:05:28.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.334+0000 7f204cff9640 1 -- 172.21.15.16:0/953734688 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204400fd30 con 0x7f2038003f30 2024-04-25T10:05:28.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.334+0000 7f2056996640 1 -- 172.21.15.16:0/953734688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20500b47c0 con 0x7f2038003f30 2024-04-25T10:05:28.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.335+0000 7f204cff9640 1 -- 172.21.15.16:0/953734688 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f204401d070 con 0x7f2038003f30 2024-04-25T10:05:28.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.335+0000 7f204cff9640 1 --2- 172.21.15.16:0/953734688 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f203c06b500 0x7f203c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.335+0000 7f204cff9640 1 -- 172.21.15.16:0/953734688 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2044089b60 con 0x7f2038003f30 2024-04-25T10:05:28.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.336+0000 7f204f7fe640 1 --2- 172.21.15.16:0/953734688 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f203c06b500 0x7f203c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.336+0000 7f2059770740 1 Processor -- start 2024-04-25T10:05:28.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.336+0000 7f2059770740 1 -- start start 2024-04-25T10:05:28.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.337+0000 7f204f7fe640 1 --2- 172.21.15.16:0/953734688 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f203c06b500 0x7f203c06d9c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2040033000 tx=0x7f2040008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.337+0000 7f2059770740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2cb230 0x55ca9a2cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.337+0000 7f2059770740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2e8ad0 0x55ca9a2e9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.337+0000 7f2059770740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2a0f10 0x55ca9a2e7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.337+0000 7f2059770740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9a203a10 con 0x55ca9a2a0f10 2024-04-25T10:05:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.337+0000 7f2059770740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9a1acb30 con 0x55ca9a2e8ad0 2024-04-25T10:05:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.337+0000 7f2059770740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca9a1ffc50 con 0x55ca9a2cb230 2024-04-25T10:05:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.337+0000 7f2054f33640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2a0f10 0x55ca9a2e7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.337+0000 7f2054f33640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2a0f10 0x55ca9a2e7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56426/0 (socket says 172.21.15.16:56426) 2024-04-25T10:05:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.337+0000 7f2054f33640 1 -- 172.21.15.16:0/1679621063 learned_addr learned my addr 172.21.15.16:0/1679621063 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.337+0000 7f204ffff640 1 --2- 172.21.15.16:0/1679621063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2cb230 0x55ca9a2cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.337+0000 7f2054f33640 1 -- 172.21.15.16:0/1679621063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2cb230 msgr2=0x55ca9a2cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.337+0000 7f204f7fe640 1 --2- 172.21.15.16:0/1679621063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2e8ad0 0x55ca9a2e9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.337+0000 7f2054f33640 1 --2- 172.21.15.16:0/1679621063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2cb230 0x55ca9a2cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.337+0000 7f2054f33640 1 -- 172.21.15.16:0/1679621063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2e8ad0 msgr2=0x55ca9a2e9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.337+0000 7f2054f33640 1 --2- 172.21.15.16:0/1679621063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2e8ad0 0x55ca9a2e9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.337+0000 7f2054f33640 1 -- 172.21.15.16:0/1679621063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca9a1c1e10 con 0x55ca9a2a0f10 2024-04-25T10:05:28.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.338+0000 7f204f7fe640 1 --2- 172.21.15.16:0/1679621063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2e8ad0 0x55ca9a2e9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:28.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.338+0000 7f2054f33640 1 --2- 172.21.15.16:0/1679621063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2a0f10 0x55ca9a2e7190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2050071ec0 tx=0x7f2048009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c8dabefc02ca90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.338+0000 7f2035ffb640 1 -- 172.21.15.16:0/1679621063 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048003b90 con 0x55ca9a2a0f10 2024-04-25T10:05:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.338+0000 7f2035ffb640 1 -- 172.21.15.16:0/1679621063 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204801d070 con 0x55ca9a2a0f10 2024-04-25T10:05:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.338+0000 7f2035ffb640 1 -- 172.21.15.16:0/1679621063 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048004030 con 0x55ca9a2a0f10 2024-04-25T10:05:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.338+0000 7f2059770740 1 -- 172.21.15.16:0/1679621063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2a0f10 msgr2=0x55ca9a2e7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.338+0000 7f2059770740 1 --2- 172.21.15.16:0/1679621063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2a0f10 0x55ca9a2e7190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2050071ec0 tx=0x7f2048009750 comp rx=0 tx=0).stop 2024-04-25T10:05:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.338+0000 7f2059770740 1 -- 172.21.15.16:0/1679621063 shutdown_connections 2024-04-25T10:05:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.338+0000 7f2059770740 1 --2- 172.21.15.16:0/1679621063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2a0f10 0x55ca9a2e7190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.338+0000 7f2059770740 1 --2- 172.21.15.16:0/1679621063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2e8ad0 0x55ca9a2e9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.338+0000 7f2059770740 1 --2- 172.21.15.16:0/1679621063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2cb230 0x55ca9a2cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.338+0000 7f2059770740 1 -- 172.21.15.16:0/1679621063 >> 172.21.15.16:0/1679621063 conn(0x55ca9a29d840 msgr2=0x55ca9a2e6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.338+0000 7f2059770740 1 -- 172.21.15.16:0/1679621063 shutdown_connections 2024-04-25T10:05:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.338+0000 7f2059770740 1 -- 172.21.15.16:0/1679621063 wait complete. 2024-04-25T10:05:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.339+0000 7f2059770740 1 -- start start 2024-04-25T10:05:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.339+0000 7f2059770740 1 Processor -- start 2024-04-25T10:05:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.340+0000 7f2059770740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2cb230 0x55ca9a2b9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.340+0000 7f2059770740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2e8ad0 0x55ca9a2def60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.340+0000 7f2059770740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2cd130 0x55ca9a2ba1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.340+0000 7f2059770740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9a2a1cb0 con 0x55ca9a2e8ad0 2024-04-25T10:05:28.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.340+0000 7f2059770740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9a1c00c0 con 0x55ca9a2cb230 2024-04-25T10:05:28.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.340+0000 7f2059770740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca9a29b770 con 0x55ca9a2cd130 2024-04-25T10:05:28.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.340+0000 7f2059770740 10 client.0 _ll_register_callbacks cb 0x55ca9a1b3ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:28.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.340+0000 7f204f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2e8ad0 0x55ca9a2def60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.340+0000 7f204f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2e8ad0 0x55ca9a2def60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56432/0 (socket says 172.21.15.16:56432) 2024-04-25T10:05:28.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.340+0000 7f204f7fe640 1 -- 172.21.15.16:0/1922437447 learned_addr learned my addr 172.21.15.16:0/1922437447 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:28.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.340+0000 7f2054f33640 1 --2- 172.21.15.16:0/1922437447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2cd130 0x55ca9a2ba1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.340+0000 7f204ffff640 1 --2- 172.21.15.16:0/1922437447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2cb230 0x55ca9a2b9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.340+0000 7f204f7fe640 1 -- 172.21.15.16:0/1922437447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2cd130 msgr2=0x55ca9a2ba1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.340+0000 7f204f7fe640 1 --2- 172.21.15.16:0/1922437447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2cd130 0x55ca9a2ba1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.340+0000 7f204f7fe640 1 -- 172.21.15.16:0/1922437447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2cb230 msgr2=0x55ca9a2b9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.340+0000 7f204f7fe640 1 --2- 172.21.15.16:0/1922437447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2cb230 0x55ca9a2b9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.340+0000 7f204f7fe640 1 -- 172.21.15.16:0/1922437447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca99f182d0 con 0x55ca9a2e8ad0 2024-04-25T10:05:28.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.341+0000 7f204f7fe640 1 --2- 172.21.15.16:0/1922437447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2e8ad0 0x55ca9a2def60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2040009be0 tx=0x7f204002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49473a3aa19e4c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.341+0000 7f202effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:05:28.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.341+0000 7f202effd640 1 -- 172.21.15.16:0/1922437447 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040036050 con 0x55ca9a2e8ad0 2024-04-25T10:05:28.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.341+0000 7f202effd640 1 -- 172.21.15.16:0/1922437447 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2040003ba0 con 0x55ca9a2e8ad0 2024-04-25T10:05:28.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.341+0000 7f2059770740 1 -- 172.21.15.16:0/1922437447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f204801d070 con 0x55ca9a2e8ad0 2024-04-25T10:05:28.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.341+0000 7f202effd640 1 -- 172.21.15.16:0/1922437447 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2040056050 con 0x55ca9a2e8ad0 2024-04-25T10:05:28.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.341+0000 7f2059770740 10 client.4701 fetch_fsmap learned FSMap version 6 2024-04-25T10:05:28.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.341+0000 7f2059770740 1 -- 172.21.15.16:0/1922437447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55ca99fe26b0 con 0x55ca9a2e8ad0 2024-04-25T10:05:28.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.341+0000 7f202effd640 1 -- 172.21.15.16:0/1922437447 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204801d070 con 0x55ca9a2e8ad0 2024-04-25T10:05:28.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.341+0000 7f202effd640 1 -- 172.21.15.16:0/1922437447 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2040059020 con 0x55ca9a2e8ad0 2024-04-25T10:05:28.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.341+0000 7f2059770740 10 client.4701 fetch_fsmap finished waiting for FSMap version 6 2024-04-25T10:05:28.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.341+0000 7f2059770740 10 client.4701 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:05:28.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.341+0000 7f2059770740 20 client.4701 populate_metadata read hostname 'smithi016' 2024-04-25T10:05:28.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.341+0000 7f2059770740 1 --2- 172.21.15.16:0/1922437447 >> [v2:172.21.15.186:6836/1763853189,v1:172.21.15.186:6837/1763853189] conn(0x55ca9a2cc220 0x55ca9a2cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.341+0000 7f2059770740 4 client.4701 mds_command: new command op to 34148 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:28.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.341+0000 7f2059770740 1 -- 172.21.15.16:0/1922437447 --> [v2:172.21.15.186:6836/1763853189,v1:172.21.15.186:6837/1763853189] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca9a262920 con 0x55ca9a2cc220 2024-04-25T10:05:28.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.341+0000 7f204ffff640 1 --2- 172.21.15.16:0/1922437447 >> [v2:172.21.15.186:6836/1763853189,v1:172.21.15.186:6837/1763853189] conn(0x55ca9a2cc220 0x55ca9a2cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.342+0000 7f202effd640 10 client.4701 ms_handle_connect on v2:172.21.15.186:6836/1763853189 2024-04-25T10:05:28.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.342+0000 7f204ffff640 1 --2- 172.21.15.16:0/1922437447 >> [v2:172.21.15.186:6836/1763853189,v1:172.21.15.186:6837/1763853189] conn(0x55ca9a2cc220 0x55ca9a2cc600 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd9a7c0a1c24ae56 server_cookie=7d50e9fb21211dfd in_seq=0 out_seq=0 2024-04-25T10:05:28.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.343+0000 7f202effd640 1 -- 172.21.15.16:0/1922437447 <== mds.0 v2:172.21.15.186:6836/1763853189 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ca9a262920 con 0x55ca9a2cc220 2024-04-25T10:05:28.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.343+0000 7f202effd640 10 client.4701 handle_command_reply: tid=0 2024-04-25T10:05:28.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.343+0000 7f2059770740 1 client.4701 shutdown 2024-04-25T10:05:28.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.343+0000 7f2059770740 1 -- 172.21.15.16:0/1922437447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2e8ad0 msgr2=0x55ca9a2def60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.343+0000 7f2059770740 1 --2- 172.21.15.16:0/1922437447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2e8ad0 0x55ca9a2def60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2040009be0 tx=0x7f204002c000 comp rx=0 tx=0).stop 2024-04-25T10:05:28.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.343+0000 7f2059770740 1 -- 172.21.15.16:0/1922437447 shutdown_connections 2024-04-25T10:05:28.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.343+0000 7f2059770740 1 --2- 172.21.15.16:0/1922437447 >> [v2:172.21.15.186:6836/1763853189,v1:172.21.15.186:6837/1763853189] conn(0x55ca9a2cc220 0x55ca9a2cc600 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.343+0000 7f2059770740 1 --2- 172.21.15.16:0/1922437447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2cd130 0x55ca9a2ba1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.343+0000 7f2059770740 1 --2- 172.21.15.16:0/1922437447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2e8ad0 0x55ca9a2def60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.343+0000 7f2059770740 1 --2- 172.21.15.16:0/1922437447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2cb230 0x55ca9a2b9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.343+0000 7f202effd640 0 client.4701 ms_handle_reset on v2:172.21.15.186:6836/1763853189 2024-04-25T10:05:28.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.343+0000 7f2059770740 1 -- 172.21.15.16:0/1922437447 >> 172.21.15.16:0/1922437447 conn(0x55ca9a2a0f10 msgr2=0x55ca9a2e57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.343+0000 7f2059770740 1 -- 172.21.15.16:0/1922437447 shutdown_connections 2024-04-25T10:05:28.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.343+0000 7f2059770740 1 -- 172.21.15.16:0/1922437447 wait complete. 2024-04-25T10:05:28.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.343+0000 7f2059770740 20 client.4701 trim_cache size 0 max 16384 2024-04-25T10:05:28.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.359+0000 7f2059770740 1 Processor -- start 2024-04-25T10:05:28.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.359+0000 7f2059770740 1 -- start start 2024-04-25T10:05:28.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.359+0000 7f2059770740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2cb230 0x55ca9a2e0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.359+0000 7f2059770740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2cd130 0x55ca9a2cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.359+0000 7f2059770740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2e8ad0 0x55ca9a2ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.359+0000 7f2059770740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20501ff810 con 0x55ca9a2cb230 2024-04-25T10:05:28.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.359+0000 7f2059770740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20501f8080 con 0x55ca9a2cd130 2024-04-25T10:05:28.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.359+0000 7f2059770740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20500b5c70 con 0x55ca9a2e8ad0 2024-04-25T10:05:28.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.360+0000 7f204ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2cb230 0x55ca9a2e0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.360+0000 7f204ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2cb230 0x55ca9a2e0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56436/0 (socket says 172.21.15.16:56436) 2024-04-25T10:05:28.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.360+0000 7f204ffff640 1 -- 172.21.15.16:0/2638341664 learned_addr learned my addr 172.21.15.16:0/2638341664 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:28.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.360+0000 7f2054f33640 1 --2- 172.21.15.16:0/2638341664 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2e8ad0 0x55ca9a2ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.360+0000 7f204f7fe640 1 --2- 172.21.15.16:0/2638341664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2cd130 0x55ca9a2cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.360+0000 7f204ffff640 1 -- 172.21.15.16:0/2638341664 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2e8ad0 msgr2=0x55ca9a2ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.360+0000 7f204ffff640 1 --2- 172.21.15.16:0/2638341664 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2e8ad0 0x55ca9a2ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.360+0000 7f204ffff640 1 -- 172.21.15.16:0/2638341664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2cd130 msgr2=0x55ca9a2cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.360+0000 7f204ffff640 1 --2- 172.21.15.16:0/2638341664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2cd130 0x55ca9a2cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.360+0000 7f204ffff640 1 -- 172.21.15.16:0/2638341664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca9a2cbb10 con 0x55ca9a2cb230 2024-04-25T10:05:28.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.360+0000 7f204f7fe640 1 --2- 172.21.15.16:0/2638341664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2cd130 0x55ca9a2cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:28.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.360+0000 7f204ffff640 1 --2- 172.21.15.16:0/2638341664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2cb230 0x55ca9a2e0030 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2044088140 tx=0x7f204405b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3938c84198e4256d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.360+0000 7f20357fa640 1 -- 172.21.15.16:0/2638341664 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044085020 con 0x55ca9a2cb230 2024-04-25T10:05:28.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.360+0000 7f20357fa640 1 -- 172.21.15.16:0/2638341664 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2044056670 con 0x55ca9a2cb230 2024-04-25T10:05:28.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.360+0000 7f20357fa640 1 -- 172.21.15.16:0/2638341664 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204405c120 con 0x55ca9a2cb230 2024-04-25T10:05:28.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.361+0000 7f2059770740 1 -- 172.21.15.16:0/2638341664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2cb230 msgr2=0x55ca9a2e0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.361+0000 7f2059770740 1 --2- 172.21.15.16:0/2638341664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2cb230 0x55ca9a2e0030 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2044088140 tx=0x7f204405b040 comp rx=0 tx=0).stop 2024-04-25T10:05:28.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.361+0000 7f2059770740 1 -- 172.21.15.16:0/2638341664 shutdown_connections 2024-04-25T10:05:28.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.361+0000 7f2059770740 1 --2- 172.21.15.16:0/2638341664 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2e8ad0 0x55ca9a2ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.361+0000 7f2059770740 1 --2- 172.21.15.16:0/2638341664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2cd130 0x55ca9a2cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.361+0000 7f2059770740 1 --2- 172.21.15.16:0/2638341664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2cb230 0x55ca9a2e0030 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.361+0000 7f2059770740 1 -- 172.21.15.16:0/2638341664 >> 172.21.15.16:0/2638341664 conn(0x55ca9a2a0f10 msgr2=0x55ca9a2e8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.361+0000 7f2059770740 1 -- 172.21.15.16:0/2638341664 shutdown_connections 2024-04-25T10:05:28.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.361+0000 7f2059770740 1 -- 172.21.15.16:0/2638341664 wait complete. 2024-04-25T10:05:28.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.361+0000 7f2059770740 1 -- start start 2024-04-25T10:05:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.361+0000 7f2059770740 1 Processor -- start 2024-04-25T10:05:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.362+0000 7f2059770740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2cb230 0x55ca9a2cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.362+0000 7f2059770740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2cd130 0x55ca9a2ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.362+0000 7f2059770740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2e8ad0 0x55ca9a29a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.362+0000 7f2059770740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9a2baca0 con 0x55ca9a2cd130 2024-04-25T10:05:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.362+0000 7f2059770740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9a2de480 con 0x55ca9a2cb230 2024-04-25T10:05:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.362+0000 7f2059770740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca9a2ce740 con 0x55ca9a2e8ad0 2024-04-25T10:05:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.362+0000 7f2059770740 10 client.0 _ll_register_callbacks cb 0x55ca9a29bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.362+0000 7f204f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2cd130 0x55ca9a2ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.362+0000 7f204f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2cd130 0x55ca9a2ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56448/0 (socket says 172.21.15.16:56448) 2024-04-25T10:05:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.362+0000 7f204f7fe640 1 -- 172.21.15.16:0/3765011862 learned_addr learned my addr 172.21.15.16:0/3765011862 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.362+0000 7f204ffff640 1 --2- 172.21.15.16:0/3765011862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2cb230 0x55ca9a2cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.362+0000 7f2054f33640 1 --2- 172.21.15.16:0/3765011862 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2e8ad0 0x55ca9a29a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f204f7fe640 1 -- 172.21.15.16:0/3765011862 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2e8ad0 msgr2=0x55ca9a29a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f204f7fe640 1 --2- 172.21.15.16:0/3765011862 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2e8ad0 0x55ca9a29a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f204f7fe640 1 -- 172.21.15.16:0/3765011862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2cb230 msgr2=0x55ca9a2cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f204f7fe640 1 --2- 172.21.15.16:0/3765011862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2cb230 0x55ca9a2cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f204f7fe640 1 -- 172.21.15.16:0/3765011862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca9a2ca2e0 con 0x55ca9a2cd130 2024-04-25T10:05:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f204f7fe640 1 --2- 172.21.15.16:0/3765011862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2cd130 0x55ca9a2ccb40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2040006f80 tx=0x7f20400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fbd850da807037 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f202effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:05:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f202effd640 1 -- 172.21.15.16:0/3765011862 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040036040 con 0x55ca9a2cd130 2024-04-25T10:05:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f202effd640 1 -- 172.21.15.16:0/3765011862 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f204003e730 con 0x55ca9a2cd130 2024-04-25T10:05:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f2059770740 1 -- 172.21.15.16:0/3765011862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2044056670 con 0x55ca9a2cd130 2024-04-25T10:05:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f202effd640 1 -- 172.21.15.16:0/3765011862 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2040068050 con 0x55ca9a2cd130 2024-04-25T10:05:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f2059770740 10 client.4707 fetch_fsmap learned FSMap version 6 2024-04-25T10:05:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f2059770740 1 -- 172.21.15.16:0/3765011862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55ca9a29fd40 con 0x55ca9a2cd130 2024-04-25T10:05:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f202effd640 1 -- 172.21.15.16:0/3765011862 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2044056670 con 0x55ca9a2cd130 2024-04-25T10:05:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f202effd640 1 -- 172.21.15.16:0/3765011862 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f204006b020 con 0x55ca9a2cd130 2024-04-25T10:05:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f2059770740 10 client.4707 fetch_fsmap finished waiting for FSMap version 6 2024-04-25T10:05:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f2059770740 10 client.4707 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:05:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f2059770740 20 client.4707 populate_metadata read hostname 'smithi016' 2024-04-25T10:05:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f2059770740 1 --2- 172.21.15.16:0/3765011862 >> [v2:172.21.15.186:6836/1763853189,v1:172.21.15.186:6837/1763853189] conn(0x55ca9a350760 0x55ca9a350b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f2059770740 4 client.4707 mds_command: new command op to 34148 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:05:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.363+0000 7f2059770740 1 -- 172.21.15.16:0/3765011862 --> [v2:172.21.15.186:6836/1763853189,v1:172.21.15.186:6837/1763853189] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ca9a210f50 con 0x55ca9a350760 2024-04-25T10:05:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.364+0000 7f204ffff640 1 --2- 172.21.15.16:0/3765011862 >> [v2:172.21.15.186:6836/1763853189,v1:172.21.15.186:6837/1763853189] conn(0x55ca9a350760 0x55ca9a350b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.364+0000 7f204ffff640 1 --2- 172.21.15.16:0/3765011862 >> [v2:172.21.15.186:6836/1763853189,v1:172.21.15.186:6837/1763853189] conn(0x55ca9a350760 0x55ca9a350b40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12fb91393808bc3f server_cookie=7a0f82b1385e6142 in_seq=0 out_seq=0 2024-04-25T10:05:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.364+0000 7f202effd640 10 client.4707 ms_handle_connect on v2:172.21.15.186:6836/1763853189 2024-04-25T10:05:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.364+0000 7f202effd640 1 -- 172.21.15.16:0/3765011862 <== mds.0 v2:172.21.15.186:6836/1763853189 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55ca9a210f50 con 0x55ca9a350760 2024-04-25T10:05:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.364+0000 7f202effd640 10 client.4707 handle_command_reply: tid=0 2024-04-25T10:05:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.364+0000 7f2059770740 1 client.4707 shutdown 2024-04-25T10:05:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.364+0000 7f2059770740 1 -- 172.21.15.16:0/3765011862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2cd130 msgr2=0x55ca9a2ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.364+0000 7f2059770740 1 --2- 172.21.15.16:0/3765011862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2cd130 0x55ca9a2ccb40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2040006f80 tx=0x7f20400046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2059770740 1 -- 172.21.15.16:0/3765011862 shutdown_connections 2024-04-25T10:05:28.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2059770740 1 --2- 172.21.15.16:0/3765011862 >> [v2:172.21.15.186:6836/1763853189,v1:172.21.15.186:6837/1763853189] conn(0x55ca9a350760 0x55ca9a350b40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2059770740 1 --2- 172.21.15.16:0/3765011862 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca9a2e8ad0 0x55ca9a29a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f202effd640 0 client.4707 ms_handle_reset on v2:172.21.15.186:6836/1763853189 2024-04-25T10:05:28.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2059770740 1 --2- 172.21.15.16:0/3765011862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca9a2cd130 0x55ca9a2ccb40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2059770740 1 --2- 172.21.15.16:0/3765011862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca9a2cb230 0x55ca9a2cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2059770740 1 -- 172.21.15.16:0/3765011862 >> 172.21.15.16:0/3765011862 conn(0x55ca9a2a0f10 msgr2=0x55ca9a2c7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2059770740 1 -- 172.21.15.16:0/3765011862 shutdown_connections 2024-04-25T10:05:28.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2059770740 1 -- 172.21.15.16:0/3765011862 wait complete. 2024-04-25T10:05:28.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2059770740 20 client.4707 trim_cache size 0 max 16384 2024-04-25T10:05:28.381 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:05:28.381 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "34bc5eca-3a21-42a6-a26f-07da0905e8bc", 2024-04-25T10:05:28.381 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-04-25T10:05:28.381 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 34148, 2024-04-25T10:05:28.381 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-04-25T10:05:28.381 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-04-25T10:05:28.381 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-04-25T10:05:28.381 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 16.923166294000001, 2024-04-25T10:05:28.381 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 6, 2024-04-25T10:05:28.381 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 22, 2024-04-25T10:05:28.381 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 22, 2024-04-25T10:05:28.381 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 24.129838123999999 2024-04-25T10:05:28.381 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:05:28.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2056996640 1 -- 172.21.15.16:0/953734688 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f203c06b500 msgr2=0x7f203c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2056996640 1 --2- 172.21.15.16:0/953734688 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f203c06b500 0x7f203c06d9c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2040033000 tx=0x7f2040008030 comp rx=0 tx=0).stop 2024-04-25T10:05:28.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2056996640 1 -- 172.21.15.16:0/953734688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2038003f30 msgr2=0x7f205015f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2056996640 1 --2- 172.21.15.16:0/953734688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2038003f30 0x7f205015f380 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f20440073a0 tx=0x7f2044007860 comp rx=0 tx=0).stop 2024-04-25T10:05:28.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2056996640 1 -- 172.21.15.16:0/953734688 shutdown_connections 2024-04-25T10:05:28.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2056996640 1 --2- 172.21.15.16:0/953734688 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f203c06b500 0x7f203c06d9c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2056996640 1 --2- 172.21.15.16:0/953734688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2050154b50 0x7f20500b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2056996640 1 --2- 172.21.15.16:0/953734688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20500caed0 0x7f20500c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2056996640 1 --2- 172.21.15.16:0/953734688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2038003f30 0x7f205015f380 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2056996640 1 -- 172.21.15.16:0/953734688 >> 172.21.15.16:0/953734688 conn(0x7f205006f430 msgr2=0x7f20500cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2056996640 1 -- 172.21.15.16:0/953734688 shutdown_connections 2024-04-25T10:05:28.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.365+0000 7f2056996640 1 -- 172.21.15.16:0/953734688 wait complete. 2024-04-25T10:05:28.384 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T10:05:28.411 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T10:05:28.411 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T10:05:28.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:28.411 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:05:28.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:28.427 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-04-25T10:05:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-04-25T10:05:28.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:05:28.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:28.495 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:05:28.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:28.550 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-04-25T10:05:28.616 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-04-25T10:05:28.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:05:28.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:28.618 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2024-04-25T10:05:28.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:28.638 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link delete ceph-brx 2024-04-25T10:05:28.708 INFO:teuthology.orchestra.run.smithi186.stderr:Cannot find device "ceph-brx" 2024-04-25T10:05:28.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:05:28.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:28.710 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2024-04-25T10:05:28.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:28.769 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link delete ceph-brx 2024-04-25T10:05:28.837 INFO:teuthology.orchestra.run.smithi186.stderr:Cannot find device "ceph-brx" 2024-04-25T10:05:28.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:05:28.839 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-25T10:05:28.840 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:05:28.840 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:28.840 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:05:28.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.937+0000 7fd15edf4640 1 Processor -- start 2024-04-25T10:05:28.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.937+0000 7fd15edf4640 1 -- start start 2024-04-25T10:05:28.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.938+0000 7fd15edf4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd15815dce0 0x7fd1580d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.938+0000 7fd15edf4640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd1580d35b0 0x7fd1580b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.938+0000 7fd15edf4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd15815e0c0 con 0x7fd1580d1f90 2024-04-25T10:05:28.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.938+0000 7fd15edf4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1580b7b10 con 0x7fd15815dce0 2024-04-25T10:05:28.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.938+0000 7fd15edf4640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1580d2510 con 0x7fd1580d35b0 2024-04-25T10:05:28.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.938+0000 7fd15cb90640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd15815dce0 0x7fd1580d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.938+0000 7fd15d391640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd1580d1f90 0x7fd15806bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35256/0 (socket says 172.21.15.16:35256) 2024-04-25T10:05:28.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.938+0000 7fd15d391640 1 -- 172.21.15.16:0/2524275152 learned_addr learned my addr 172.21.15.16:0/2524275152 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:28.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.938+0000 7fd14ffff640 1 --2- 172.21.15.16:0/2524275152 >> v2:172.21.15.186:3300/0 conn(0x7fd1580d35b0 0x7fd1580b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.938+0000 7fd15cb90640 1 -- 172.21.15.16:0/2524275152 >> v2:172.21.15.186:3300/0 conn(0x7fd1580d35b0 msgr2=0x7fd1580b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.938+0000 7fd15cb90640 1 --2- 172.21.15.16:0/2524275152 >> v2:172.21.15.186:3300/0 conn(0x7fd1580d35b0 0x7fd1580b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.938+0000 7fd15cb90640 1 -- 172.21.15.16:0/2524275152 >> v1:172.21.15.16:6789/0 conn(0x7fd1580d1f90 legacy=0x7fd15806bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.938+0000 7fd14f7fe640 1 -- 172.21.15.16:0/2524275152 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443605456 0 0) 0x7fd15815e0c0 con 0x7fd1580d1f90 2024-04-25T10:05:28.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.938+0000 7fd15cb90640 1 -- 172.21.15.16:0/2524275152 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1580d27f0 con 0x7fd15815dce0 2024-04-25T10:05:28.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.939+0000 7fd15cb90640 1 --2- 172.21.15.16:0/2524275152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15815dce0 0x7fd1580d1a50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd1580cdfa0 tx=0x7fd15402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9020682c3fec38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.939+0000 7fd14f7fe640 1 -- 172.21.15.16:0/2524275152 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154037070 con 0x7fd15815dce0 2024-04-25T10:05:28.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.939+0000 7fd14f7fe640 1 -- 172.21.15.16:0/2524275152 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd154032440 con 0x7fd15815dce0 2024-04-25T10:05:28.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.939+0000 7fd14f7fe640 1 -- 172.21.15.16:0/2524275152 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154032810 con 0x7fd15815dce0 2024-04-25T10:05:28.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.939+0000 7fd15edf4640 1 -- 172.21.15.16:0/2524275152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15815dce0 msgr2=0x7fd1580d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.939+0000 7fd15edf4640 1 --2- 172.21.15.16:0/2524275152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15815dce0 0x7fd1580d1a50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd1580cdfa0 tx=0x7fd15402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.939+0000 7fd15edf4640 1 -- 172.21.15.16:0/2524275152 shutdown_connections 2024-04-25T10:05:28.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.939+0000 7fd15edf4640 1 --2- 172.21.15.16:0/2524275152 >> v2:172.21.15.186:3300/0 conn(0x7fd1580d35b0 0x7fd1580b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.939+0000 7fd15edf4640 1 --2- 172.21.15.16:0/2524275152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15815dce0 0x7fd1580d1a50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.940+0000 7fd15edf4640 1 -- 172.21.15.16:0/2524275152 >> 172.21.15.16:0/2524275152 conn(0x7fd158072900 msgr2=0x7fd1580d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.940+0000 7fd15edf4640 1 -- 172.21.15.16:0/2524275152 shutdown_connections 2024-04-25T10:05:28.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.940+0000 7fd15edf4640 1 -- 172.21.15.16:0/2524275152 wait complete. 2024-04-25T10:05:28.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.941+0000 7fd15edf4640 1 Processor -- start 2024-04-25T10:05:28.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.941+0000 7fd15edf4640 1 -- start start 2024-04-25T10:05:28.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd15edf4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1580d1f90 0x7fd1581507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd15edf4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1580d35b0 0x7fd1580cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd15edf4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd15815dce0 0x7fd158153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd15edf4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1580d3090 con 0x7fd1580d1f90 2024-04-25T10:05:28.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd15edf4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1580d2db0 con 0x7fd15815dce0 2024-04-25T10:05:28.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd15edf4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1580b9820 con 0x7fd1580d35b0 2024-04-25T10:05:28.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd15cb90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1580d1f90 0x7fd1581507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd15cb90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1580d1f90 0x7fd1581507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56468/0 (socket says 172.21.15.16:56468) 2024-04-25T10:05:28.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd15cb90640 1 -- 172.21.15.16:0/3437953473 learned_addr learned my addr 172.21.15.16:0/3437953473 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd15d391640 1 --2- 172.21.15.16:0/3437953473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd15815dce0 0x7fd158153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd14ffff640 1 --2- 172.21.15.16:0/3437953473 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1580d35b0 0x7fd1580cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd15cb90640 1 -- 172.21.15.16:0/3437953473 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1580d35b0 msgr2=0x7fd1580cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd15cb90640 1 --2- 172.21.15.16:0/3437953473 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1580d35b0 0x7fd1580cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd15cb90640 1 -- 172.21.15.16:0/3437953473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd15815dce0 msgr2=0x7fd158153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd15cb90640 1 --2- 172.21.15.16:0/3437953473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd15815dce0 0x7fd158153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd15cb90640 1 -- 172.21.15.16:0/3437953473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1580cc600 con 0x7fd1580d1f90 2024-04-25T10:05:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd15d391640 1 --2- 172.21.15.16:0/3437953473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd15815dce0 0x7fd158153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd15cb90640 1 --2- 172.21.15.16:0/3437953473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1580d1f90 0x7fd1581507c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd154000c00 tx=0x7fd1540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd14d7fa640 1 -- 172.21.15.16:0/3437953473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154037040 con 0x7fd1580d1f90 2024-04-25T10:05:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd14d7fa640 1 -- 172.21.15.16:0/3437953473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd154042070 con 0x7fd1580d1f90 2024-04-25T10:05:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd15edf4640 1 -- 172.21.15.16:0/3437953473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1580cc8e0 con 0x7fd1580d1f90 2024-04-25T10:05:28.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.942+0000 7fd14d7fa640 1 -- 172.21.15.16:0/3437953473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154030bd0 con 0x7fd1580d1f90 2024-04-25T10:05:28.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.943+0000 7fd15edf4640 1 -- 172.21.15.16:0/3437953473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1580ccea0 con 0x7fd1580d1f90 2024-04-25T10:05:28.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.944+0000 7fd15edf4640 1 -- 172.21.15.16:0/3437953473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1580749b0 con 0x7fd1580d1f90 2024-04-25T10:05:28.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.944+0000 7fd14d7fa640 1 -- 172.21.15.16:0/3437953473 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd154047460 con 0x7fd1580d1f90 2024-04-25T10:05:28.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.944+0000 7fd14d7fa640 1 --2- 172.21.15.16:0/3437953473 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd14406b4d0 0x7fd14406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.944+0000 7fd14d7fa640 1 -- 172.21.15.16:0/3437953473 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1540aacd0 con 0x7fd1580d1f90 2024-04-25T10:05:28.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.944+0000 7fd14ffff640 1 --2- 172.21.15.16:0/3437953473 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd14406b4d0 0x7fd14406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.945+0000 7fd14ffff640 1 --2- 172.21.15.16:0/3437953473 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd14406b4d0 0x7fd14406d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd1580ca4b0 tx=0x7fd148008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:28.948+0000 7fd14d7fa640 1 -- 172.21.15.16:0/3437953473 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd154032e20 con 0x7fd1580d1f90 2024-04-25T10:05:29.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:29.173+0000 7fd15edf4640 1 -- 172.21.15.16:0/3437953473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd15815d170 con 0x7fd1580d1f90 2024-04-25T10:05:29.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:29.174+0000 7fd14d7fa640 1 -- 172.21.15.16:0/3437953473 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd154079be0 con 0x7fd1580d1f90 2024-04-25T10:05:29.179 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:05:29.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:29.176+0000 7fd15edf4640 1 -- 172.21.15.16:0/3437953473 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd14406b4d0 msgr2=0x7fd14406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:29.176+0000 7fd15edf4640 1 --2- 172.21.15.16:0/3437953473 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd14406b4d0 0x7fd14406d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd1580ca4b0 tx=0x7fd148008040 comp rx=0 tx=0).stop 2024-04-25T10:05:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:29.176+0000 7fd15edf4640 1 -- 172.21.15.16:0/3437953473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1580d1f90 msgr2=0x7fd1581507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:29.176+0000 7fd15edf4640 1 --2- 172.21.15.16:0/3437953473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1580d1f90 0x7fd1581507c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd154000c00 tx=0x7fd1540046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:29.176+0000 7fd15edf4640 1 -- 172.21.15.16:0/3437953473 shutdown_connections 2024-04-25T10:05:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:29.176+0000 7fd15edf4640 1 --2- 172.21.15.16:0/3437953473 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd14406b4d0 0x7fd14406d990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:29.176+0000 7fd15edf4640 1 --2- 172.21.15.16:0/3437953473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd15815dce0 0x7fd158153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:29.176+0000 7fd15edf4640 1 --2- 172.21.15.16:0/3437953473 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1580d35b0 0x7fd1580cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:29.176+0000 7fd15edf4640 1 --2- 172.21.15.16:0/3437953473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1580d1f90 0x7fd1581507c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:29.176+0000 7fd15edf4640 1 -- 172.21.15.16:0/3437953473 >> 172.21.15.16:0/3437953473 conn(0x7fd158072900 msgr2=0x7fd1580b8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:29.176+0000 7fd15edf4640 1 -- 172.21.15.16:0/3437953473 shutdown_connections 2024-04-25T10:05:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:29.176+0000 7fd15edf4640 1 -- 172.21.15.16:0/3437953473 wait complete. 2024-04-25T10:05:29.192 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:05:29.192 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:05:29.192 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-04-25T10:05:29.192 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:05:29.192 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:29.192 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:05:29.192 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:05:29.192 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:05:29.192 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:29.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:29.193 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-04-25T10:05:29.208 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:29.208 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:29.208 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:29.208 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:29.208 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:29.208 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:29.208 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:29.208 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:29.208 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-04-25T10:05:29.208 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:29.208 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:29.209 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-04-25T10:05:29.209 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:05:29.209 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:29.209 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-04-25T10:05:29.209 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:05:29.209 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42631sec preferred_lft 42631sec 2024-04-25T10:05:29.209 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-25T10:05:29.209 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:29.209 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:29.209 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:29.209 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-04-25T10:05:29.210 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:05:29.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:29.210 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:05:29.210 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:05:29.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:05:29.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-04-25T10:05:29.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-04-25T10:05:29.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:05:29.210 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:05:29.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:29.351 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:05:29.375 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:05:29.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:29.377 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-04-25T10:05:29.441 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-04-25T10:05:29.441 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:05:29.441 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:05:29.442 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:05:29.442 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:05:29.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:29.442 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:05:29.443 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:05:29.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:05:29.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:05:29.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:05:29.443 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:05:29.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:29.679 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:05:29.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:29.695 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-04-25T10:05:29.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:29.751 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:05:29.751 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:05:29.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:29.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T10:05:29.751 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:05:29.831 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T10:05:29.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:29.832 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:05:29.832 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:05:29.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T10:05:29.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:05:29.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T10:05:29.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T10:05:29.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T10:05:29.832 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:05:29.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:29.969 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:05:29.969 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:05:29.969 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-04-25T10:05:29.969 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:05:29.969 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:05:30.006 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:30.006 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:30.062 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:30.062 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:30.063 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:30.118 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:05:30.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:30.118 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:05:30.207 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:05:30.207 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:05:30.208 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:05:30.313 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:05:30.313 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:05:30.313 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:05:30.313 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:05:30.313 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:05:30.319 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:05:30.319 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:05:30.319 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:05:30.319 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:05:30.319 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:05:30.319 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:05:30.319 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:05:30.319 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:05:30.320 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:05:30.320 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:05:30.320 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:05:30.321 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:05:30.321 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:05:30.322 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:05:30.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:30.323 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:30.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:30.348 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:05:30.410 INFO:teuthology.orchestra.run.smithi016.stdout:client4615 2024-04-25T10:05:30.411 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:05:30.411 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:30.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:30.468 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:05:30.529 INFO:teuthology.orchestra.run.smithi016.stdout:client4615 2024-04-25T10:05:30.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:30.530 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:05:30.592 INFO:teuthology.orchestra.run.smithi016.stdout:client4615 2024-04-25T10:05:30.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:30.592 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/34bc5eca-3a21-42a6-a26f-07da0905e8bc.client4615/status') 2024-04-25T10:05:30.660 INFO:teuthology.orchestra.run.smithi016.stdout:instance: client.4615 (3)172.21.15.16:0/2438497292 2024-04-25T10:05:30.660 INFO:teuthology.orchestra.run.smithi016.stdout:blocklisted: false 2024-04-25T10:05:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records in 2024-04-25T10:05:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records out 2024-04-25T10:05:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:70 bytes copied, 2.4642e-05 s, 2.8 MB/s 2024-04-25T10:05:30.662 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-25T10:05:30.662 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:05:30.663 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:05:30.663 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:05:30.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.798+0000 7f15ee9f9640 1 Processor -- start 2024-04-25T10:05:30.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.798+0000 7f15ee9f9640 1 -- start start 2024-04-25T10:05:30.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.799+0000 7f15ee9f9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f15e806fab0 0x7f15e8070320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.799+0000 7f15ee9f9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15e806ff10 0x7f15e8070860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.799+0000 7f15ee9f9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15e80c6540 con 0x7f15e815ded0 2024-04-25T10:05:30.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.799+0000 7f15ee9f9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e8072bc0 con 0x7f15e806ff10 2024-04-25T10:05:30.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.799+0000 7f15ee9f9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e80ccfc0 con 0x7f15e806fab0 2024-04-25T10:05:30.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.799+0000 7f15ecf96640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15e806ff10 0x7f15e8070860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.799+0000 7f15ecf96640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15e806ff10 0x7f15e8070860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56480/0 (socket says 172.21.15.16:56480) 2024-04-25T10:05:30.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.799+0000 7f15ecf96640 1 -- 172.21.15.16:0/2131667712 learned_addr learned my addr 172.21.15.16:0/2131667712 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:30.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.799+0000 7f15e77fe640 1 --2- 172.21.15.16:0/2131667712 >> v2:172.21.15.186:3300/0 conn(0x7f15e806fab0 0x7f15e8070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.800+0000 7f15ecf96640 1 -- 172.21.15.16:0/2131667712 >> v2:172.21.15.186:3300/0 conn(0x7f15e806fab0 msgr2=0x7f15e8070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.800+0000 7f15ecf96640 1 --2- 172.21.15.16:0/2131667712 >> v2:172.21.15.186:3300/0 conn(0x7f15e806fab0 0x7f15e8070320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.800+0000 7f15ecf96640 1 -- 172.21.15.16:0/2131667712 >> v1:172.21.15.186:6789/0 conn(0x7f15e815ded0 legacy=0x7f15e80cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.800+0000 7f15ecf96640 1 -- 172.21.15.16:0/2131667712 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e8072e10 con 0x7f15e806ff10 2024-04-25T10:05:30.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.800+0000 7f15ecf96640 1 --2- 172.21.15.16:0/2131667712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15e806ff10 0x7f15e8070860 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f15e80b3220 tx=0x7f15e002a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0b5c932ba8fe07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.801+0000 7f15e6ffd640 1 -- 172.21.15.16:0/2131667712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e0036070 con 0x7f15e806ff10 2024-04-25T10:05:30.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.801+0000 7f15e6ffd640 1 -- 172.21.15.16:0/2131667712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e0031600 con 0x7f15e806ff10 2024-04-25T10:05:30.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.801+0000 7f15e6ffd640 1 -- 172.21.15.16:0/2131667712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e00319f0 con 0x7f15e806ff10 2024-04-25T10:05:30.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.801+0000 7f15ee9f9640 1 -- 172.21.15.16:0/2131667712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15e806ff10 msgr2=0x7f15e8070860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.801+0000 7f15ee9f9640 1 --2- 172.21.15.16:0/2131667712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15e806ff10 0x7f15e8070860 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f15e80b3220 tx=0x7f15e002a7c0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.801+0000 7f15ee9f9640 1 -- 172.21.15.16:0/2131667712 shutdown_connections 2024-04-25T10:05:30.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.801+0000 7f15ee9f9640 1 --2- 172.21.15.16:0/2131667712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15e806ff10 0x7f15e8070860 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.801+0000 7f15ee9f9640 1 --2- 172.21.15.16:0/2131667712 >> v2:172.21.15.186:3300/0 conn(0x7f15e806fab0 0x7f15e8070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.801+0000 7f15ee9f9640 1 -- 172.21.15.16:0/2131667712 >> 172.21.15.16:0/2131667712 conn(0x7f15e80b2370 msgr2=0x7f15e80b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.801+0000 7f15ee9f9640 1 -- 172.21.15.16:0/2131667712 shutdown_connections 2024-04-25T10:05:30.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.801+0000 7f15ee9f9640 1 -- 172.21.15.16:0/2131667712 wait complete. 2024-04-25T10:05:30.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.802+0000 7f15ee9f9640 1 Processor -- start 2024-04-25T10:05:30.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.803+0000 7f15ee9f9640 1 -- start start 2024-04-25T10:05:30.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.803+0000 7f15ee9f9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15e806fab0 0x7f15e80c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.803+0000 7f15ee9f9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15e815ded0 0x7f15e81520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.803+0000 7f15ee9f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15e80b3460 0x7f15e8152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.803+0000 7f15ee9f9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e80cd4f0 con 0x7f15e80b3460 2024-04-25T10:05:30.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.803+0000 7f15ee9f9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e80cd210 con 0x7f15e806fab0 2024-04-25T10:05:30.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.803+0000 7f15ee9f9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e80cad90 con 0x7f15e815ded0 2024-04-25T10:05:30.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.803+0000 7f15ecf96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15e80b3460 0x7f15e8152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.804+0000 7f15ecf96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15e80b3460 0x7f15e8152630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56486/0 (socket says 172.21.15.16:56486) 2024-04-25T10:05:30.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.804+0000 7f15ecf96640 1 -- 172.21.15.16:0/2453542928 learned_addr learned my addr 172.21.15.16:0/2453542928 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:30.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.804+0000 7f15e7fff640 1 --2- 172.21.15.16:0/2453542928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15e806fab0 0x7f15e80c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.804+0000 7f15e77fe640 1 --2- 172.21.15.16:0/2453542928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15e815ded0 0x7f15e81520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.804+0000 7f15ecf96640 1 -- 172.21.15.16:0/2453542928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15e815ded0 msgr2=0x7f15e81520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.804+0000 7f15ecf96640 1 --2- 172.21.15.16:0/2453542928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15e815ded0 0x7f15e81520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.804+0000 7f15ecf96640 1 -- 172.21.15.16:0/2453542928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15e806fab0 msgr2=0x7f15e80c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.804+0000 7f15ecf96640 1 --2- 172.21.15.16:0/2453542928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15e806fab0 0x7f15e80c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.804+0000 7f15ecf96640 1 -- 172.21.15.16:0/2453542928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e80ade50 con 0x7f15e80b3460 2024-04-25T10:05:30.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.804+0000 7f15ecf96640 1 --2- 172.21.15.16:0/2453542928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15e80b3460 0x7f15e8152630 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009f40 tx=0x7f15e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.804+0000 7f15e4ff9640 1 -- 172.21.15.16:0/2453542928 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e0036050 con 0x7f15e80b3460 2024-04-25T10:05:30.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.804+0000 7f15e4ff9640 1 -- 172.21.15.16:0/2453542928 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e0037040 con 0x7f15e80b3460 2024-04-25T10:05:30.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.804+0000 7f15ee9f9640 1 -- 172.21.15.16:0/2453542928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e80af140 con 0x7f15e80b3460 2024-04-25T10:05:30.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.804+0000 7f15e4ff9640 1 -- 172.21.15.16:0/2453542928 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e0009510 con 0x7f15e80b3460 2024-04-25T10:05:30.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.804+0000 7f15ee9f9640 1 -- 172.21.15.16:0/2453542928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e80c7d50 con 0x7f15e80b3460 2024-04-25T10:05:30.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.805+0000 7f15ee9f9640 1 -- 172.21.15.16:0/2453542928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e80749b0 con 0x7f15e80b3460 2024-04-25T10:05:30.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.806+0000 7f15e4ff9640 1 -- 172.21.15.16:0/2453542928 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15e0044020 con 0x7f15e80b3460 2024-04-25T10:05:30.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.806+0000 7f15e4ff9640 1 --2- 172.21.15.16:0/2453542928 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f15d406b4d0 0x7f15d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.806+0000 7f15e4ff9640 1 -- 172.21.15.16:0/2453542928 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f15e00aa4f0 con 0x7f15e80b3460 2024-04-25T10:05:30.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.806+0000 7f15e7fff640 1 --2- 172.21.15.16:0/2453542928 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f15d406b4d0 0x7f15d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.807+0000 7f15e7fff640 1 --2- 172.21.15.16:0/2453542928 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f15d406b4d0 0x7f15d406d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f15e80c9430 tx=0x7f15d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:30.810+0000 7f15e4ff9640 1 -- 172.21.15.16:0/2453542928 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15e004b020 con 0x7f15e80b3460 2024-04-25T10:05:31.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:31.033+0000 7f15ee9f9640 1 -- 172.21.15.16:0/2453542928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f15e8072450 con 0x7f15e80b3460 2024-04-25T10:05:31.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:31.033+0000 7f15e4ff9640 1 -- 172.21.15.16:0/2453542928 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f15e0079400 con 0x7f15e80b3460 2024-04-25T10:05:31.039 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:05:31.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:31.035+0000 7f15ee9f9640 1 -- 172.21.15.16:0/2453542928 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f15d406b4d0 msgr2=0x7f15d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:31.035+0000 7f15ee9f9640 1 --2- 172.21.15.16:0/2453542928 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f15d406b4d0 0x7f15d406d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f15e80c9430 tx=0x7f15d8009040 comp rx=0 tx=0).stop 2024-04-25T10:05:31.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:31.035+0000 7f15ee9f9640 1 -- 172.21.15.16:0/2453542928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15e80b3460 msgr2=0x7f15e8152630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:31.035+0000 7f15ee9f9640 1 --2- 172.21.15.16:0/2453542928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15e80b3460 0x7f15e8152630 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009f40 tx=0x7f15e0004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:31.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:31.035+0000 7f15ee9f9640 1 -- 172.21.15.16:0/2453542928 shutdown_connections 2024-04-25T10:05:31.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:31.035+0000 7f15ee9f9640 1 --2- 172.21.15.16:0/2453542928 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f15d406b4d0 0x7f15d406d990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:31.035+0000 7f15ee9f9640 1 --2- 172.21.15.16:0/2453542928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15e80b3460 0x7f15e8152630 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:31.035+0000 7f15ee9f9640 1 --2- 172.21.15.16:0/2453542928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15e815ded0 0x7f15e81520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:31.035+0000 7f15ee9f9640 1 --2- 172.21.15.16:0/2453542928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15e806fab0 0x7f15e80c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:31.035+0000 7f15ee9f9640 1 -- 172.21.15.16:0/2453542928 >> 172.21.15.16:0/2453542928 conn(0x7f15e80b2370 msgr2=0x7f15e80cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:31.036+0000 7f15ee9f9640 1 -- 172.21.15.16:0/2453542928 shutdown_connections 2024-04-25T10:05:31.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:31.036+0000 7f15ee9f9640 1 -- 172.21.15.16:0/2453542928 wait complete. 2024-04-25T10:05:31.051 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:05:31.051 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:05:31.051 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-04-25T10:05:31.052 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T10:05:31.052 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:05:31.052 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:05:31.052 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:05:31.052 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:05:31.052 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:05:31.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:31.052 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-04-25T10:05:31.068 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:31.068 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:31.068 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:31.068 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:31.068 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:31.068 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:31.068 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:31.068 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:31.068 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-04-25T10:05:31.068 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:31.069 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:31.069 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-04-25T10:05:31.069 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:05:31.069 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:31.069 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-04-25T10:05:31.069 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:05:31.069 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42629sec preferred_lft 42629sec 2024-04-25T10:05:31.069 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-25T10:05:31.069 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:31.069 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:31.069 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:31.069 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-04-25T10:05:31.069 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:05:31.069 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:31.069 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:05:31.070 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:31.070 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f06b:fdff:fed4:a335/64 scope link tentative 2024-04-25T10:05:31.070 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:31.070 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:05:31.070 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:31.070 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::90ad:4fff:fe91:3c33/64 scope link tentative 2024-04-25T10:05:31.070 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:31.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:31.070 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:05:31.131 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:05:31.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:31.133 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-04-25T10:05:31.190 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-04-25T10:05:31.190 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:05:31.190 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:05:31.190 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:05:31.190 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:05:31.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:31.190 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:05:31.191 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:05:31.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:05:31.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:05:31.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:05:31.191 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:05:31.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:31.276 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:05:31.292 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:05:31.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:31.293 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-04-25T10:05:31.348 INFO:teuthology.orchestra.run.smithi016.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T10:05:31.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:31.349 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:05:31.349 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:05:31.349 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:31.349 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T10:05:31.349 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:05:31.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:31.425 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:05:31.488 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:31.489 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:31.489 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:31.489 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:31.489 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:31.489 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:31.489 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:05:31.489 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 32:85:01:31:58:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:05:31.489 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:05:31.489 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:31.489 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::3085:1ff:fe31:58ca/64 scope link 2024-04-25T10:05:31.489 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:31.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:31.491 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:05:31.555 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:31.556 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:31.556 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:31.556 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:31.556 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:31.556 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:31.557 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:05:31.557 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 32:85:01:31:58:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:05:31.557 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:05:31.557 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:31.557 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::3085:1ff:fe31:58ca/64 scope link 2024-04-25T10:05:31.557 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:31.560 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T10:05:31.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:31.560 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:05:31.560 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:05:31.560 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T10:05:31.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:05:31.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T10:05:31.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T10:05:31.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T10:05:31.561 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:05:31.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:31.671 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:05:31.671 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:05:31.671 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.1 up 2024-04-25T10:05:31.671 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:05:31.671 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:05:31.708 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:31.708 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T10:05:31.763 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:05:31.763 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:31.763 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:05:31.818 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:05:31.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:31.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:05:31.900 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:05:31.901 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T10:05:31.901 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:05:31.909 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:05:31.909 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T10:05:31.909 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:05:31.909 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:05:31.909 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:05:31.914 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:05:31.914 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T10:05:31.915 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:05:31.915 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:05:31.915 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:05:31.915 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:05:31.915 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:05:31.915 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:05:31.917 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-25T10:05:31.917 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:05:31.917 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:05:31.917 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:05:31.917 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:05:31.918 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:05:31.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:31.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:05:31.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:31.981 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:05:32.039 INFO:teuthology.orchestra.run.smithi016.stdout:client4725 2024-04-25T10:05:32.039 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:05:32.039 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:32.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:32.095 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:05:32.151 INFO:teuthology.orchestra.run.smithi016.stdout:client4725 2024-04-25T10:05:32.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:32.152 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:05:32.208 INFO:teuthology.orchestra.run.smithi016.stdout:client4725 2024-04-25T10:05:32.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:32.209 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/34bc5eca-3a21-42a6-a26f-07da0905e8bc.client4725/status') 2024-04-25T10:05:32.273 INFO:teuthology.orchestra.run.smithi016.stdout:instance: client.4725 (3)192.168.0.2:0/3708345793 2024-04-25T10:05:32.273 INFO:teuthology.orchestra.run.smithi016.stdout:blocklisted: false 2024-04-25T10:05:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records in 2024-04-25T10:05:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records out 2024-04-25T10:05:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:69 bytes copied, 2.5539e-05 s, 2.7 MB/s 2024-04-25T10:05:32.274 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-25T10:05:32.274 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:05:32.275 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:05:32.275 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:05:32.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.410+0000 7f7f9d24a640 1 Processor -- start 2024-04-25T10:05:32.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.410+0000 7f7f9d24a640 1 -- start start 2024-04-25T10:05:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.411+0000 7f7f9d24a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f980d35b0 0x7f7f980b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.411+0000 7f7f9d24a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f980b5990 0x7f7f980b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.411+0000 7f7f9d24a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9815e0c0 con 0x7f7f9815dce0 2024-04-25T10:05:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.411+0000 7f7f9d24a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f980b5de0 con 0x7f7f980b5990 2024-04-25T10:05:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.411+0000 7f7f9d24a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f980d2510 con 0x7f7f980d35b0 2024-04-25T10:05:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.411+0000 7f7f9759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f980b5990 0x7f7f980b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.411+0000 7f7f9759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f980b5990 0x7f7f980b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56502/0 (socket says 172.21.15.16:56502) 2024-04-25T10:05:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.411+0000 7f7f9759e640 1 -- 172.21.15.16:0/1637214455 learned_addr learned my addr 172.21.15.16:0/1637214455 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.411+0000 7f7f8ffff640 1 --2- 172.21.15.16:0/1637214455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f980d35b0 0x7f7f980b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.411+0000 7f7f9759e640 1 -- 172.21.15.16:0/1637214455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f980d35b0 msgr2=0x7f7f980b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.411+0000 7f7f9759e640 1 --2- 172.21.15.16:0/1637214455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f980d35b0 0x7f7f980b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.411+0000 7f7f9759e640 1 -- 172.21.15.16:0/1637214455 >> v1:172.21.15.186:6789/0 conn(0x7f7f9815dce0 legacy=0x7f7f980d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.411+0000 7f7f9759e640 1 -- 172.21.15.16:0/1637214455 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f980d27f0 con 0x7f7f980b5990 2024-04-25T10:05:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.412+0000 7f7f9759e640 1 --2- 172.21.15.16:0/1637214455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f980b5990 0x7f7f980b6b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009e50 tx=0x7f7f9002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af454f27a18cd8e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.412+0000 7f7f9659c640 1 -- 172.21.15.16:0/1637214455 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90036070 con 0x7f7f980b5990 2024-04-25T10:05:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.412+0000 7f7f9659c640 1 -- 172.21.15.16:0/1637214455 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f90031600 con 0x7f7f980b5990 2024-04-25T10:05:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.412+0000 7f7f9659c640 1 -- 172.21.15.16:0/1637214455 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f900319d0 con 0x7f7f980b5990 2024-04-25T10:05:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.413+0000 7f7f9d24a640 1 -- 172.21.15.16:0/1637214455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f980b5990 msgr2=0x7f7f980b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.413+0000 7f7f9d24a640 1 --2- 172.21.15.16:0/1637214455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f980b5990 0x7f7f980b6b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009e50 tx=0x7f7f9002a710 comp rx=0 tx=0).stop 2024-04-25T10:05:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.413+0000 7f7f9d24a640 1 -- 172.21.15.16:0/1637214455 shutdown_connections 2024-04-25T10:05:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.413+0000 7f7f9d24a640 1 --2- 172.21.15.16:0/1637214455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f980b5990 0x7f7f980b6b00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.413+0000 7f7f9d24a640 1 --2- 172.21.15.16:0/1637214455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f980d35b0 0x7f7f980b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.413+0000 7f7f9d24a640 1 -- 172.21.15.16:0/1637214455 >> 172.21.15.16:0/1637214455 conn(0x7f7f98072900 msgr2=0x7f7f980d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.413+0000 7f7f9d24a640 1 -- 172.21.15.16:0/1637214455 shutdown_connections 2024-04-25T10:05:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.413+0000 7f7f9d24a640 1 -- 172.21.15.16:0/1637214455 wait complete. 2024-04-25T10:05:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.414+0000 7f7f9d24a640 1 Processor -- start 2024-04-25T10:05:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.414+0000 7f7f9d24a640 1 -- start start 2024-04-25T10:05:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.415+0000 7f7f9d24a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f980b5990 0x7f7f981603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.415+0000 7f7f9d24a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f980d35b0 0x7f7f981521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.415+0000 7f7f9d24a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f9815dce0 0x7f7f980c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.415+0000 7f7f9d24a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f980d3090 con 0x7f7f9815dce0 2024-04-25T10:05:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.415+0000 7f7f9d24a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f980d2db0 con 0x7f7f980b5990 2024-04-25T10:05:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.415+0000 7f7f9d24a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f980d2160 con 0x7f7f980d35b0 2024-04-25T10:05:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.415+0000 7f7f9759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f9815dce0 0x7f7f980c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.415+0000 7f7f9759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f9815dce0 0x7f7f980c9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56504/0 (socket says 172.21.15.16:56504) 2024-04-25T10:05:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.415+0000 7f7f9759e640 1 -- 172.21.15.16:0/2244156531 learned_addr learned my addr 172.21.15.16:0/2244156531 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.415+0000 7f7f96d9d640 1 --2- 172.21.15.16:0/2244156531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f980b5990 0x7f7f981603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.415+0000 7f7f8ffff640 1 --2- 172.21.15.16:0/2244156531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f980d35b0 0x7f7f981521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.415+0000 7f7f9759e640 1 -- 172.21.15.16:0/2244156531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f980d35b0 msgr2=0x7f7f981521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.415+0000 7f7f9759e640 1 --2- 172.21.15.16:0/2244156531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f980d35b0 0x7f7f981521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.415+0000 7f7f9759e640 1 -- 172.21.15.16:0/2244156531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f980b5990 msgr2=0x7f7f981603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.415+0000 7f7f9759e640 1 --2- 172.21.15.16:0/2244156531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f980b5990 0x7f7f981603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.415+0000 7f7f9759e640 1 -- 172.21.15.16:0/2244156531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f98152720 con 0x7f7f9815dce0 2024-04-25T10:05:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.416+0000 7f7f9759e640 1 --2- 172.21.15.16:0/2244156531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f9815dce0 0x7f7f980c9570 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f90034040 tx=0x7f7f90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.416+0000 7f7f8f7fe640 1 -- 172.21.15.16:0/2244156531 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90036050 con 0x7f7f9815dce0 2024-04-25T10:05:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.416+0000 7f7f8f7fe640 1 -- 172.21.15.16:0/2244156531 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f90042070 con 0x7f7f9815dce0 2024-04-25T10:05:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.416+0000 7f7f9d24a640 1 -- 172.21.15.16:0/2244156531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f981609f0 con 0x7f7f9815dce0 2024-04-25T10:05:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.416+0000 7f7f8f7fe640 1 -- 172.21.15.16:0/2244156531 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90009510 con 0x7f7f9815dce0 2024-04-25T10:05:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.416+0000 7f7f9d24a640 1 -- 172.21.15.16:0/2244156531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f98160fb0 con 0x7f7f9815dce0 2024-04-25T10:05:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.417+0000 7f7f9d24a640 1 -- 172.21.15.16:0/2244156531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f980749b0 con 0x7f7f9815dce0 2024-04-25T10:05:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.417+0000 7f7f8f7fe640 1 -- 172.21.15.16:0/2244156531 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f90007bf0 con 0x7f7f9815dce0 2024-04-25T10:05:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.418+0000 7f7f8f7fe640 1 --2- 172.21.15.16:0/2244156531 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7f7806b4d0 0x7f7f7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.418+0000 7f7f8f7fe640 1 -- 172.21.15.16:0/2244156531 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f900aaea0 con 0x7f7f9815dce0 2024-04-25T10:05:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.418+0000 7f7f96d9d640 1 --2- 172.21.15.16:0/2244156531 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7f7806b4d0 0x7f7f7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.418+0000 7f7f96d9d640 1 --2- 172.21.15.16:0/2244156531 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7f7806b4d0 0x7f7f7806d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7f980cce00 tx=0x7f7f88009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.422+0000 7f7f8f7fe640 1 -- 172.21.15.16:0/2244156531 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f900796d0 con 0x7f7f9815dce0 2024-04-25T10:05:32.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.639+0000 7f7f9d24a640 1 -- 172.21.15.16:0/2244156531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7f980b7ad0 con 0x7f7f9815dce0 2024-04-25T10:05:32.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.639+0000 7f7f8f7fe640 1 -- 172.21.15.16:0/2244156531 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7f9007d770 con 0x7f7f9815dce0 2024-04-25T10:05:32.645 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:05:32.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.642+0000 7f7f9d24a640 1 -- 172.21.15.16:0/2244156531 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7f7806b4d0 msgr2=0x7f7f7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.642+0000 7f7f9d24a640 1 --2- 172.21.15.16:0/2244156531 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7f7806b4d0 0x7f7f7806d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7f980cce00 tx=0x7f7f88009040 comp rx=0 tx=0).stop 2024-04-25T10:05:32.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.642+0000 7f7f9d24a640 1 -- 172.21.15.16:0/2244156531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f9815dce0 msgr2=0x7f7f980c9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.642+0000 7f7f9d24a640 1 --2- 172.21.15.16:0/2244156531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f9815dce0 0x7f7f980c9570 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f90034040 tx=0x7f7f90004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.642+0000 7f7f9d24a640 1 -- 172.21.15.16:0/2244156531 shutdown_connections 2024-04-25T10:05:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.642+0000 7f7f9d24a640 1 --2- 172.21.15.16:0/2244156531 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7f7806b4d0 0x7f7f7806d990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.642+0000 7f7f9d24a640 1 --2- 172.21.15.16:0/2244156531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f9815dce0 0x7f7f980c9570 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.642+0000 7f7f9d24a640 1 --2- 172.21.15.16:0/2244156531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7f980d35b0 0x7f7f981521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.642+0000 7f7f9d24a640 1 --2- 172.21.15.16:0/2244156531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f980b5990 0x7f7f981603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.642+0000 7f7f9d24a640 1 -- 172.21.15.16:0/2244156531 >> 172.21.15.16:0/2244156531 conn(0x7f7f98072900 msgr2=0x7f7f980b7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.642+0000 7f7f9d24a640 1 -- 172.21.15.16:0/2244156531 shutdown_connections 2024-04-25T10:05:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:32.642+0000 7f7f9d24a640 1 -- 172.21.15.16:0/2244156531 wait complete. 2024-04-25T10:05:32.657 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:05:32.657 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:05:32.658 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi186.front.sepia.ceph.com 2024-04-25T10:05:32.658 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T10:05:32.658 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:05:32.658 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:05:32.658 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:05:32.658 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:05:32.658 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:05:32.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:32.658 DEBUG:teuthology.orchestra.run.smithi186:> ip addr 2024-04-25T10:05:32.677 INFO:teuthology.orchestra.run.smithi186.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:32.677 INFO:teuthology.orchestra.run.smithi186.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:32.677 INFO:teuthology.orchestra.run.smithi186.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:32.677 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:32.678 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:32.678 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:32.678 INFO:teuthology.orchestra.run.smithi186.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:32.678 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:d9:38:2c brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:32.678 INFO:teuthology.orchestra.run.smithi186.stdout: altname enp7s0f0 2024-04-25T10:05:32.678 INFO:teuthology.orchestra.run.smithi186.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:32.678 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:d9:38:2d brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:32.678 INFO:teuthology.orchestra.run.smithi186.stdout: altname enp7s0f1 2024-04-25T10:05:32.678 INFO:teuthology.orchestra.run.smithi186.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:32.678 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:88:6d:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:32.678 INFO:teuthology.orchestra.run.smithi186.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:05:32.678 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:88:6d:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:32.678 INFO:teuthology.orchestra.run.smithi186.stdout: inet 172.21.15.186/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:05:32.678 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft 42655sec preferred_lft 42655sec 2024-04-25T10:05:32.679 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::dfbb:5b26:e99a:c329/64 scope link noprefixroute 2024-04-25T10:05:32.679 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:32.679 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:05:32.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:32.679 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2024-04-25T10:05:32.679 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2024-04-25T10:05:32.679 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:05:32.679 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip addr flush dev ceph-brx 2024-04-25T10:05:32.679 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set ceph-brx up 2024-04-25T10:05:32.679 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:05:32.680 DEBUG:teuthology.orchestra.run.smithi186:> ') 2024-04-25T10:05:32.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:32.820 DEBUG:teuthology.orchestra.run.smithi186:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:05:32.847 INFO:teuthology.orchestra.run.smithi186.stdout:1 2024-04-25T10:05:32.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:32.849 DEBUG:teuthology.orchestra.run.smithi186:> route 2024-04-25T10:05:32.940 INFO:teuthology.orchestra.run.smithi186.stdout:Kernel IP routing table 2024-04-25T10:05:32.940 INFO:teuthology.orchestra.run.smithi186.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:05:32.940 INFO:teuthology.orchestra.run.smithi186.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:05:32.940 INFO:teuthology.orchestra.run.smithi186.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:05:32.941 INFO:teuthology.orchestra.run.smithi186.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:05:32.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:32.941 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2024-04-25T10:05:32.941 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2024-04-25T10:05:32.941 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:05:32.941 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:05:32.941 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:05:32.941 DEBUG:teuthology.orchestra.run.smithi186:> ') 2024-04-25T10:05:33.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:33.097 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2024-04-25T10:05:33.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:33.115 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list-id 2024-04-25T10:05:33.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:33.170 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2024-04-25T10:05:33.171 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2024-04-25T10:05:33.171 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:05:33.171 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T10:05:33.171 DEBUG:teuthology.orchestra.run.smithi186:> ') 2024-04-25T10:05:33.247 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T10:05:33.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:33.247 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2024-04-25T10:05:33.247 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2024-04-25T10:05:33.247 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T10:05:33.247 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:05:33.247 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T10:05:33.248 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T10:05:33.248 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T10:05:33.248 DEBUG:teuthology.orchestra.run.smithi186:> ') 2024-04-25T10:05:33.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:33.367 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2024-04-25T10:05:33.367 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2024-04-25T10:05:33.367 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set brx.0 up 2024-04-25T10:05:33.367 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:05:33.367 DEBUG:teuthology.orchestra.run.smithi186:> ') 2024-04-25T10:05:33.403 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:33.403 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T10:05:33.459 INFO:teuthology.orchestra.run.smithi186.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:05:33.460 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:33.460 DEBUG:teuthology.orchestra.run.smithi186:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:05:33.514 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:05:33.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:33.514 DEBUG:teuthology.orchestra.run.smithi186:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:05:33.605 INFO:teuthology.orchestra.run.smithi186.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:05:33.605 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T10:05:33.605 INFO:teuthology.orchestra.run.smithi186.stdout:invalid new device string format 2024-04-25T10:05:33.707 INFO:teuthology.orchestra.run.smithi186.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:05:33.707 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T10:05:33.707 INFO:teuthology.orchestra.run.smithi186.stdout:invalid new device string format 2024-04-25T10:05:33.707 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:05:33.707 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:05:33.712 INFO:teuthology.orchestra.run.smithi186.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:05:33.712 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T10:05:33.712 INFO:teuthology.orchestra.run.smithi186.stdout:invalid new device string format 2024-04-25T10:05:33.712 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:05:33.712 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi186.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi186.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi186.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:05:33.714 INFO:teuthology.orchestra.run.smithi186.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:05:33.714 INFO:teuthology.orchestra.run.smithi186.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:05:33.715 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:05:33.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:33.715 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:05:33.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:33.739 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:05:33.795 INFO:teuthology.orchestra.run.smithi186.stdout:client4630 2024-04-25T10:05:33.796 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:05:33.796 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:33.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:33.851 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:05:33.907 INFO:teuthology.orchestra.run.smithi186.stdout:client4630 2024-04-25T10:05:33.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:33.908 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:05:33.963 INFO:teuthology.orchestra.run.smithi186.stdout:client4630 2024-04-25T10:05:33.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:33.963 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/34bc5eca-3a21-42a6-a26f-07da0905e8bc.client4630/status') 2024-04-25T10:05:34.027 INFO:teuthology.orchestra.run.smithi186.stdout:instance: client.4630 (3)192.168.0.1:0/2165069688 2024-04-25T10:05:34.027 INFO:teuthology.orchestra.run.smithi186.stdout:blocklisted: false 2024-04-25T10:05:34.027 INFO:teuthology.orchestra.run.smithi186.stderr:0+1 records in 2024-04-25T10:05:34.027 INFO:teuthology.orchestra.run.smithi186.stderr:0+1 records out 2024-04-25T10:05:34.027 INFO:teuthology.orchestra.run.smithi186.stderr:69 bytes copied, 2.285e-05 s, 3.0 MB/s 2024-04-25T10:05:34.028 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-25T10:05:34.028 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:05:34.028 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:05:34.029 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:05:34.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.136+0000 7f13b44e1640 1 Processor -- start 2024-04-25T10:05:34.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.136+0000 7f13b44e1640 1 -- start start 2024-04-25T10:05:34.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.137+0000 7f13b44e1640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f13ac15dce0 0x7f13ac0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.137+0000 7f13b44e1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13ac0d35b0 0x7f13ac0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.137+0000 7f13b44e1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13ac0d1f90 0x7f13ac0b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.137+0000 7f13b44e1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ac15e0c0 con 0x7f13ac0d1f90 2024-04-25T10:05:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.137+0000 7f13b44e1640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ac0d2510 con 0x7f13ac15dce0 2024-04-25T10:05:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.137+0000 7f13b44e1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac0d27f0 con 0x7f13ac0d35b0 2024-04-25T10:05:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.138+0000 7f13b2a7e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13ac0d1f90 0x7f13ac0b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.138+0000 7f13b2a7e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13ac0d1f90 0x7f13ac0b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37380/0 (socket says 172.21.15.16:37380) 2024-04-25T10:05:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.138+0000 7f13b2a7e640 1 -- 172.21.15.16:0/2221041949 learned_addr learned my addr 172.21.15.16:0/2221041949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.138+0000 7f13b227d640 1 --2- 172.21.15.16:0/2221041949 >> v2:172.21.15.186:3300/0 conn(0x7f13ac15dce0 0x7f13ac0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.138+0000 7f13b1a7c640 1 --2- 172.21.15.16:0/2221041949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13ac0d35b0 0x7f13ac0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.138+0000 7f13b2a7e640 1 -- 172.21.15.16:0/2221041949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13ac0d35b0 msgr2=0x7f13ac0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.138+0000 7f13b2a7e640 1 --2- 172.21.15.16:0/2221041949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13ac0d35b0 0x7f13ac0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.138+0000 7f13b2a7e640 1 -- 172.21.15.16:0/2221041949 >> v2:172.21.15.186:3300/0 conn(0x7f13ac15dce0 msgr2=0x7f13ac0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.138+0000 7f13b2a7e640 1 --2- 172.21.15.16:0/2221041949 >> v2:172.21.15.186:3300/0 conn(0x7f13ac15dce0 0x7f13ac0d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.138+0000 7f13b2a7e640 1 -- 172.21.15.16:0/2221041949 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac0d2ad0 con 0x7f13ac0d1f90 2024-04-25T10:05:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.139+0000 7f13b2a7e640 1 --2- 172.21.15.16:0/2221041949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac0d1f90 0x7f13ac0b5990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0ce220 tx=0x7f139c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2497a6862e58c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.139+0000 7f13b127b640 1 -- 172.21.15.16:0/2221041949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c037070 con 0x7f13ac0d1f90 2024-04-25T10:05:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.139+0000 7f13b127b640 1 -- 172.21.15.16:0/2221041949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139c032440 con 0x7f13ac0d1f90 2024-04-25T10:05:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.139+0000 7f13b127b640 1 -- 172.21.15.16:0/2221041949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c032810 con 0x7f13ac0d1f90 2024-04-25T10:05:34.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.139+0000 7f13b44e1640 1 -- 172.21.15.16:0/2221041949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac0d1f90 msgr2=0x7f13ac0b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.139+0000 7f13b44e1640 1 --2- 172.21.15.16:0/2221041949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac0d1f90 0x7f13ac0b5990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0ce220 tx=0x7f139c02a140 comp rx=0 tx=0).stop 2024-04-25T10:05:34.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.140+0000 7f13b44e1640 1 -- 172.21.15.16:0/2221041949 shutdown_connections 2024-04-25T10:05:34.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.140+0000 7f13b44e1640 1 --2- 172.21.15.16:0/2221041949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac0d1f90 0x7f13ac0b5990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.140+0000 7f13b44e1640 1 --2- 172.21.15.16:0/2221041949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13ac0d35b0 0x7f13ac0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.140+0000 7f13b44e1640 1 --2- 172.21.15.16:0/2221041949 >> v2:172.21.15.186:3300/0 conn(0x7f13ac15dce0 0x7f13ac0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.140+0000 7f13b44e1640 1 -- 172.21.15.16:0/2221041949 >> 172.21.15.16:0/2221041949 conn(0x7f13ac072900 msgr2=0x7f13ac0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:34.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.140+0000 7f13b44e1640 1 -- 172.21.15.16:0/2221041949 shutdown_connections 2024-04-25T10:05:34.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.140+0000 7f13b44e1640 1 -- 172.21.15.16:0/2221041949 wait complete. 2024-04-25T10:05:34.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.141+0000 7f13b44e1640 1 Processor -- start 2024-04-25T10:05:34.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.141+0000 7f13b44e1640 1 -- start start 2024-04-25T10:05:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13b44e1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13ac0d1f90 0x7f13ac156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13b44e1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13ac0d35b0 0x7f13ac0c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13b44e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac15dce0 0x7f13ac15fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13b44e1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac0b9820 con 0x7f13ac15dce0 2024-04-25T10:05:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13b44e1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac0d3090 con 0x7f13ac0d35b0 2024-04-25T10:05:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13b44e1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac0b99a0 con 0x7f13ac0d1f90 2024-04-25T10:05:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13b2a7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac15dce0 0x7f13ac15fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13b2a7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac15dce0 0x7f13ac15fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37390/0 (socket says 172.21.15.16:37390) 2024-04-25T10:05:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13b2a7e640 1 -- 172.21.15.16:0/4234019381 learned_addr learned my addr 172.21.15.16:0/4234019381 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13b227d640 1 --2- 172.21.15.16:0/4234019381 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13ac0d1f90 0x7f13ac156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13b1a7c640 1 --2- 172.21.15.16:0/4234019381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13ac0d35b0 0x7f13ac0c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13b2a7e640 1 -- 172.21.15.16:0/4234019381 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13ac0d1f90 msgr2=0x7f13ac156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13b2a7e640 1 --2- 172.21.15.16:0/4234019381 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13ac0d1f90 0x7f13ac156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13b2a7e640 1 -- 172.21.15.16:0/4234019381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13ac0d35b0 msgr2=0x7f13ac0c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13b2a7e640 1 --2- 172.21.15.16:0/4234019381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13ac0d35b0 0x7f13ac0c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13b2a7e640 1 -- 172.21.15.16:0/4234019381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac15ed80 con 0x7f13ac15dce0 2024-04-25T10:05:34.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13b2a7e640 1 --2- 172.21.15.16:0/4234019381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac15dce0 0x7f13ac15fe20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f139c0090b0 tx=0x7f139c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13aaffd640 1 -- 172.21.15.16:0/4234019381 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c041020 con 0x7f13ac15dce0 2024-04-25T10:05:34.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.142+0000 7f13aaffd640 1 -- 172.21.15.16:0/4234019381 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139c03d640 con 0x7f13ac15dce0 2024-04-25T10:05:34.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.143+0000 7f13b44e1640 1 -- 172.21.15.16:0/4234019381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ac15f060 con 0x7f13ac15dce0 2024-04-25T10:05:34.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.143+0000 7f13aaffd640 1 -- 172.21.15.16:0/4234019381 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c037040 con 0x7f13ac15dce0 2024-04-25T10:05:34.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.143+0000 7f13b44e1640 1 -- 172.21.15.16:0/4234019381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ac15f620 con 0x7f13ac15dce0 2024-04-25T10:05:34.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.144+0000 7f13b44e1640 1 -- 172.21.15.16:0/4234019381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ac0749b0 con 0x7f13ac15dce0 2024-04-25T10:05:34.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.144+0000 7f13aaffd640 1 -- 172.21.15.16:0/4234019381 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f139c044020 con 0x7f13ac15dce0 2024-04-25T10:05:34.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.145+0000 7f13aaffd640 1 --2- 172.21.15.16:0/4234019381 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f139806b4d0 0x7f139806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.145+0000 7f13aaffd640 1 -- 172.21.15.16:0/4234019381 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f139c0aa6c0 con 0x7f13ac15dce0 2024-04-25T10:05:34.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.145+0000 7f13b227d640 1 --2- 172.21.15.16:0/4234019381 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f139806b4d0 0x7f139806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.145+0000 7f13b227d640 1 --2- 172.21.15.16:0/4234019381 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f139806b4d0 0x7f139806d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13a40113e0 tx=0x7f13a400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.149+0000 7f13aaffd640 1 -- 172.21.15.16:0/4234019381 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f139c04b030 con 0x7f13ac15dce0 2024-04-25T10:05:34.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.366+0000 7f13b44e1640 1 -- 172.21.15.16:0/4234019381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f13ac06cb90 con 0x7f13ac15dce0 2024-04-25T10:05:34.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.366+0000 7f13aaffd640 1 -- 172.21.15.16:0/4234019381 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f139c078ee0 con 0x7f13ac15dce0 2024-04-25T10:05:34.371 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:05:34.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.368+0000 7f13b44e1640 1 -- 172.21.15.16:0/4234019381 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f139806b4d0 msgr2=0x7f139806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.368+0000 7f13b44e1640 1 --2- 172.21.15.16:0/4234019381 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f139806b4d0 0x7f139806d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13a40113e0 tx=0x7f13a400b040 comp rx=0 tx=0).stop 2024-04-25T10:05:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.368+0000 7f13b44e1640 1 -- 172.21.15.16:0/4234019381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac15dce0 msgr2=0x7f13ac15fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.368+0000 7f13b44e1640 1 --2- 172.21.15.16:0/4234019381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac15dce0 0x7f13ac15fe20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f139c0090b0 tx=0x7f139c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.368+0000 7f13b44e1640 1 -- 172.21.15.16:0/4234019381 shutdown_connections 2024-04-25T10:05:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.368+0000 7f13b44e1640 1 --2- 172.21.15.16:0/4234019381 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f139806b4d0 0x7f139806d990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.368+0000 7f13b44e1640 1 --2- 172.21.15.16:0/4234019381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac15dce0 0x7f13ac15fe20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.368+0000 7f13b44e1640 1 --2- 172.21.15.16:0/4234019381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13ac0d35b0 0x7f13ac0c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.368+0000 7f13b44e1640 1 --2- 172.21.15.16:0/4234019381 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13ac0d1f90 0x7f13ac156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.368+0000 7f13b44e1640 1 -- 172.21.15.16:0/4234019381 >> 172.21.15.16:0/4234019381 conn(0x7f13ac072900 msgr2=0x7f13ac0b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.368+0000 7f13b44e1640 1 -- 172.21.15.16:0/4234019381 shutdown_connections 2024-04-25T10:05:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:34.368+0000 7f13b44e1640 1 -- 172.21.15.16:0/4234019381 wait complete. 2024-04-25T10:05:34.384 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:05:34.384 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:05:34.384 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi186.front.sepia.ceph.com 2024-04-25T10:05:34.384 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T10:05:34.384 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:05:34.384 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:05:34.384 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:05:34.384 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:05:34.384 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:05:34.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.385 DEBUG:teuthology.orchestra.run.smithi186:> ip addr 2024-04-25T10:05:34.400 INFO:teuthology.orchestra.run.smithi186.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:34.400 INFO:teuthology.orchestra.run.smithi186.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:34.400 INFO:teuthology.orchestra.run.smithi186.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:34.400 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.400 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:34.400 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.400 INFO:teuthology.orchestra.run.smithi186.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:34.400 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:d9:38:2c brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:34.400 INFO:teuthology.orchestra.run.smithi186.stdout: altname enp7s0f0 2024-04-25T10:05:34.401 INFO:teuthology.orchestra.run.smithi186.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:34.401 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:d9:38:2d brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:34.401 INFO:teuthology.orchestra.run.smithi186.stdout: altname enp7s0f1 2024-04-25T10:05:34.401 INFO:teuthology.orchestra.run.smithi186.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:34.401 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:88:6d:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:34.401 INFO:teuthology.orchestra.run.smithi186.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:05:34.401 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:88:6d:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:34.401 INFO:teuthology.orchestra.run.smithi186.stdout: inet 172.21.15.186/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:05:34.401 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft 42653sec preferred_lft 42653sec 2024-04-25T10:05:34.401 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::dfbb:5b26:e99a:c329/64 scope link noprefixroute 2024-04-25T10:05:34.401 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.401 INFO:teuthology.orchestra.run.smithi186.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:05:34.401 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 46:8d:d7:bc:2c:c9 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:34.401 INFO:teuthology.orchestra.run.smithi186.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:05:34.401 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.402 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::3caa:c1ff:fe6a:4ca4/64 scope link 2024-04-25T10:05:34.402 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.402 INFO:teuthology.orchestra.run.smithi186.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:05:34.402 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 46:8d:d7:bc:2c:c9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:05:34.402 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::448d:d7ff:febc:2cc9/64 scope link tentative 2024-04-25T10:05:34.402 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.402 DEBUG:teuthology.orchestra.run.smithi186:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:05:34.463 INFO:teuthology.orchestra.run.smithi186.stdout:1 2024-04-25T10:05:34.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.465 DEBUG:teuthology.orchestra.run.smithi186:> route 2024-04-25T10:05:34.521 INFO:teuthology.orchestra.run.smithi186.stdout:Kernel IP routing table 2024-04-25T10:05:34.521 INFO:teuthology.orchestra.run.smithi186.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:05:34.521 INFO:teuthology.orchestra.run.smithi186.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:05:34.521 INFO:teuthology.orchestra.run.smithi186.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:05:34.522 INFO:teuthology.orchestra.run.smithi186.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:05:34.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.522 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2024-04-25T10:05:34.522 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2024-04-25T10:05:34.522 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:05:34.522 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:05:34.522 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:05:34.522 DEBUG:teuthology.orchestra.run.smithi186:> ') 2024-04-25T10:05:34.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.607 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2024-04-25T10:05:34.623 INFO:teuthology.orchestra.run.smithi186.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:05:34.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.623 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list-id 2024-04-25T10:05:34.679 INFO:teuthology.orchestra.run.smithi186.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T10:05:34.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.680 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2024-04-25T10:05:34.680 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2024-04-25T10:05:34.680 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:05:34.680 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T10:05:34.680 DEBUG:teuthology.orchestra.run.smithi186:> ') 2024-04-25T10:05:34.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.755 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:05:34.818 INFO:teuthology.orchestra.run.smithi186.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:34.819 INFO:teuthology.orchestra.run.smithi186.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:34.819 INFO:teuthology.orchestra.run.smithi186.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:34.819 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.819 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:34.819 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.819 INFO:teuthology.orchestra.run.smithi186.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:05:34.819 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 16:ec:6c:2f:10:e9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:05:34.819 INFO:teuthology.orchestra.run.smithi186.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:05:34.819 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.819 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::14ec:6cff:fe2f:10e9/64 scope link tentative 2024-04-25T10:05:34.819 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.820 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:05:34.884 INFO:teuthology.orchestra.run.smithi186.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:34.884 INFO:teuthology.orchestra.run.smithi186.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:34.884 INFO:teuthology.orchestra.run.smithi186.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:34.884 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.884 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:34.885 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.885 INFO:teuthology.orchestra.run.smithi186.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:05:34.885 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 16:ec:6c:2f:10:e9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:05:34.885 INFO:teuthology.orchestra.run.smithi186.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:05:34.885 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.885 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::14ec:6cff:fe2f:10e9/64 scope link tentative 2024-04-25T10:05:34.885 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.886 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T10:05:34.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.887 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2024-04-25T10:05:34.887 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2024-04-25T10:05:34.887 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T10:05:34.887 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:05:34.887 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T10:05:34.887 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T10:05:34.887 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T10:05:34.887 DEBUG:teuthology.orchestra.run.smithi186:> ') 2024-04-25T10:05:34.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.995 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2024-04-25T10:05:34.995 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2024-04-25T10:05:34.995 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set brx.1 up 2024-04-25T10:05:34.995 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:05:34.995 DEBUG:teuthology.orchestra.run.smithi186:> ') 2024-04-25T10:05:35.029 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:35.029 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T10:05:35.085 INFO:teuthology.orchestra.run.smithi186.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:05:35.085 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:35.086 DEBUG:teuthology.orchestra.run.smithi186:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:05:35.140 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:05:35.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:35.140 DEBUG:teuthology.orchestra.run.smithi186:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:05:35.221 INFO:teuthology.orchestra.run.smithi186.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:05:35.222 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T10:05:35.222 INFO:teuthology.orchestra.run.smithi186.stdout:invalid new device string format 2024-04-25T10:05:35.230 INFO:teuthology.orchestra.run.smithi186.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:05:35.230 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T10:05:35.230 INFO:teuthology.orchestra.run.smithi186.stdout:invalid new device string format 2024-04-25T10:05:35.230 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:05:35.230 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:05:35.235 INFO:teuthology.orchestra.run.smithi186.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:05:35.235 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T10:05:35.235 INFO:teuthology.orchestra.run.smithi186.stdout:invalid new device string format 2024-04-25T10:05:35.235 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:05:35.235 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:05:35.235 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:05:35.235 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:05:35.236 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:05:35.236 INFO:teuthology.orchestra.run.smithi186.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-25T10:05:35.236 INFO:teuthology.orchestra.run.smithi186.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:05:35.236 INFO:teuthology.orchestra.run.smithi186.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:05:35.236 INFO:teuthology.orchestra.run.smithi186.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:05:35.236 INFO:teuthology.orchestra.run.smithi186.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:05:35.238 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:05:35.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:35.238 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:05:35.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:35.301 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:05:35.358 INFO:teuthology.orchestra.run.smithi186.stdout:client4740 2024-04-25T10:05:35.358 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:05:35.358 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:35.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:35.413 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:05:35.469 INFO:teuthology.orchestra.run.smithi186.stdout:client4740 2024-04-25T10:05:35.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:35.470 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:05:35.525 INFO:teuthology.orchestra.run.smithi186.stdout:client4740 2024-04-25T10:05:35.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:35.526 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/34bc5eca-3a21-42a6-a26f-07da0905e8bc.client4740/status') 2024-04-25T10:05:35.589 INFO:teuthology.orchestra.run.smithi186.stdout:instance: client.4740 (3)172.21.15.186:0/953529884 2024-04-25T10:05:35.590 INFO:teuthology.orchestra.run.smithi186.stdout:blocklisted: false 2024-04-25T10:05:35.591 INFO:teuthology.orchestra.run.smithi186.stderr:0+1 records in 2024-04-25T10:05:35.591 INFO:teuthology.orchestra.run.smithi186.stderr:0+1 records out 2024-04-25T10:05:35.592 INFO:teuthology.orchestra.run.smithi186.stderr:70 bytes copied, 2.146e-05 s, 3.3 MB/s 2024-04-25T10:05:35.592 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T10:05:35.614 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:35.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.721+0000 7f620051d640 1 Processor -- start 2024-04-25T10:05:35.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.721+0000 7f620051d640 1 -- start start 2024-04-25T10:05:35.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.722+0000 7f620051d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61f815dce0 0x7f61f80d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.722+0000 7f620051d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f61f80b5b60 0x7f61f80b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.722+0000 7f620051d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f815e0c0 con 0x7f61f80d35b0 2024-04-25T10:05:35.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.722+0000 7f620051d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f80d2510 con 0x7f61f815dce0 2024-04-25T10:05:35.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.722+0000 7f620051d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f80d27f0 con 0x7f61f80b5b60 2024-04-25T10:05:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.722+0000 7f61fe2b9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61f815dce0 0x7f61f80d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.722+0000 7f61fdab8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f61f80d35b0 0x7f61f80b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40890/0 (socket says 172.21.15.16:40890) 2024-04-25T10:05:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.722+0000 7f61fdab8640 1 -- 172.21.15.16:0/3879986763 learned_addr learned my addr 172.21.15.16:0/3879986763 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.722+0000 7f61feaba640 1 --2- 172.21.15.16:0/3879986763 >> v2:172.21.15.186:3300/0 conn(0x7f61f80b5b60 0x7f61f80b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.723+0000 7f61fe2b9640 1 -- 172.21.15.16:0/3879986763 >> v2:172.21.15.186:3300/0 conn(0x7f61f80b5b60 msgr2=0x7f61f80b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.723+0000 7f61fe2b9640 1 --2- 172.21.15.16:0/3879986763 >> v2:172.21.15.186:3300/0 conn(0x7f61f80b5b60 0x7f61f80b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.723+0000 7f61fe2b9640 1 -- 172.21.15.16:0/3879986763 >> v1:172.21.15.16:6789/0 conn(0x7f61f80d35b0 legacy=0x7f61f80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.723+0000 7f61fd2b7640 1 -- 172.21.15.16:0/3879986763 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933015182 0 0) 0x7f61f815e0c0 con 0x7f61f80d35b0 2024-04-25T10:05:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.723+0000 7f61fe2b9640 1 -- 172.21.15.16:0/3879986763 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f80d2ad0 con 0x7f61f815dce0 2024-04-25T10:05:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.723+0000 7f61fe2b9640 1 --2- 172.21.15.16:0/3879986763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f815dce0 0x7f61f80d1a50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f61f80b3390 tx=0x7f61f402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2a1d0abb0cdc00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.724+0000 7f61fd2b7640 1 -- 172.21.15.16:0/3879986763 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4037070 con 0x7f61f815dce0 2024-04-25T10:05:35.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.724+0000 7f61fd2b7640 1 -- 172.21.15.16:0/3879986763 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f4032440 con 0x7f61f815dce0 2024-04-25T10:05:35.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.724+0000 7f61fd2b7640 1 -- 172.21.15.16:0/3879986763 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4032810 con 0x7f61f815dce0 2024-04-25T10:05:35.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.724+0000 7f620051d640 1 -- 172.21.15.16:0/3879986763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f815dce0 msgr2=0x7f61f80d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.724+0000 7f620051d640 1 --2- 172.21.15.16:0/3879986763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f815dce0 0x7f61f80d1a50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f61f80b3390 tx=0x7f61f402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.724+0000 7f620051d640 1 -- 172.21.15.16:0/3879986763 shutdown_connections 2024-04-25T10:05:35.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.724+0000 7f620051d640 1 --2- 172.21.15.16:0/3879986763 >> v2:172.21.15.186:3300/0 conn(0x7f61f80b5b60 0x7f61f80b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.724+0000 7f620051d640 1 --2- 172.21.15.16:0/3879986763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f815dce0 0x7f61f80d1a50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.724+0000 7f620051d640 1 -- 172.21.15.16:0/3879986763 >> 172.21.15.16:0/3879986763 conn(0x7f61f8072900 msgr2=0x7f61f80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.724+0000 7f620051d640 1 -- 172.21.15.16:0/3879986763 shutdown_connections 2024-04-25T10:05:35.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.724+0000 7f620051d640 1 -- 172.21.15.16:0/3879986763 wait complete. 2024-04-25T10:05:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.725+0000 7f620051d640 1 Processor -- start 2024-04-25T10:05:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.725+0000 7f620051d640 1 -- start start 2024-04-25T10:05:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.726+0000 7f620051d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61f80b5b60 0x7f61f8158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.726+0000 7f620051d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61f80d35b0 0x7f61f814abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.726+0000 7f620051d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f815dce0 0x7f61f80b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.726+0000 7f620051d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f80b9820 con 0x7f61f815dce0 2024-04-25T10:05:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.726+0000 7f620051d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f80d3090 con 0x7f61f80b5b60 2024-04-25T10:05:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.726+0000 7f620051d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f80b99a0 con 0x7f61f80d35b0 2024-04-25T10:05:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.726+0000 7f61feaba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f815dce0 0x7f61f80b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.726+0000 7f61feaba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f815dce0 0x7f61f80b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37402/0 (socket says 172.21.15.16:37402) 2024-04-25T10:05:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.726+0000 7f61feaba640 1 -- 172.21.15.16:0/3415797187 learned_addr learned my addr 172.21.15.16:0/3415797187 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.727+0000 7f61feaba640 1 -- 172.21.15.16:0/3415797187 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61f80d35b0 msgr2=0x7f61f814abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.727+0000 7f61feaba640 1 --2- 172.21.15.16:0/3415797187 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61f80d35b0 0x7f61f814abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.727+0000 7f61feaba640 1 -- 172.21.15.16:0/3415797187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61f80b5b60 msgr2=0x7f61f8158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.727+0000 7f61feaba640 1 --2- 172.21.15.16:0/3415797187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61f80b5b60 0x7f61f8158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.727+0000 7f61feaba640 1 -- 172.21.15.16:0/3415797187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f81577a0 con 0x7f61f815dce0 2024-04-25T10:05:35.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.727+0000 7f61feaba640 1 --2- 172.21.15.16:0/3415797187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f815dce0 0x7f61f80b3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f61f000efc0 tx=0x7f61f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.727+0000 7f61eeffd640 1 -- 172.21.15.16:0/3415797187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0018020 con 0x7f61f815dce0 2024-04-25T10:05:35.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.727+0000 7f61eeffd640 1 -- 172.21.15.16:0/3415797187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f0014480 con 0x7f61f815dce0 2024-04-25T10:05:35.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.727+0000 7f61eeffd640 1 -- 172.21.15.16:0/3415797187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0014870 con 0x7f61f815dce0 2024-04-25T10:05:35.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.727+0000 7f620051d640 1 -- 172.21.15.16:0/3415797187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f8157a40 con 0x7f61f815dce0 2024-04-25T10:05:35.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.727+0000 7f620051d640 1 -- 172.21.15.16:0/3415797187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f8158000 con 0x7f61f815dce0 2024-04-25T10:05:35.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.728+0000 7f620051d640 1 -- 172.21.15.16:0/3415797187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f80d2020 con 0x7f61f815dce0 2024-04-25T10:05:35.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.728+0000 7f61eeffd640 1 -- 172.21.15.16:0/3415797187 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f61f0005d10 con 0x7f61f815dce0 2024-04-25T10:05:35.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.729+0000 7f61eeffd640 1 --2- 172.21.15.16:0/3415797187 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f61dc06b4d0 0x7f61dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.729+0000 7f61eeffd640 1 -- 172.21.15.16:0/3415797187 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61f008b080 con 0x7f61f815dce0 2024-04-25T10:05:35.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.729+0000 7f61fe2b9640 1 --2- 172.21.15.16:0/3415797187 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f61dc06b4d0 0x7f61dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.730+0000 7f61fe2b9640 1 --2- 172.21.15.16:0/3415797187 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f61dc06b4d0 0x7f61dc06d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f61f4034000 tx=0x7f61f402c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.733+0000 7f61eeffd640 1 -- 172.21.15.16:0/3415797187 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f61f80d2020 con 0x7f61f815dce0 2024-04-25T10:05:35.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.915+0000 7f620051d640 1 -- 172.21.15.16:0/3415797187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61f8161210 con 0x7f61f815dce0 2024-04-25T10:05:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.917+0000 7f61eeffd640 1 -- 172.21.15.16:0/3415797187 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f61f0022030 con 0x7f61f815dce0 2024-04-25T10:05:35.922 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:35.923 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:12.449183+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:35.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.918+0000 7f620051d640 1 -- 172.21.15.16:0/3415797187 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f61dc06b4d0 msgr2=0x7f61dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.918+0000 7f620051d640 1 --2- 172.21.15.16:0/3415797187 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f61dc06b4d0 0x7f61dc06d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f61f4034000 tx=0x7f61f402c000 comp rx=0 tx=0).stop 2024-04-25T10:05:35.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.918+0000 7f620051d640 1 -- 172.21.15.16:0/3415797187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f815dce0 msgr2=0x7f61f80b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.918+0000 7f620051d640 1 --2- 172.21.15.16:0/3415797187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f815dce0 0x7f61f80b3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f61f000efc0 tx=0x7f61f000c490 comp rx=0 tx=0).stop 2024-04-25T10:05:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.918+0000 7f620051d640 1 -- 172.21.15.16:0/3415797187 shutdown_connections 2024-04-25T10:05:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.918+0000 7f620051d640 1 --2- 172.21.15.16:0/3415797187 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f61dc06b4d0 0x7f61dc06d990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.918+0000 7f620051d640 1 --2- 172.21.15.16:0/3415797187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f815dce0 0x7f61f80b3710 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.918+0000 7f620051d640 1 --2- 172.21.15.16:0/3415797187 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61f80d35b0 0x7f61f814abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.918+0000 7f620051d640 1 --2- 172.21.15.16:0/3415797187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61f80b5b60 0x7f61f8158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.918+0000 7f620051d640 1 -- 172.21.15.16:0/3415797187 >> 172.21.15.16:0/3415797187 conn(0x7f61f8072900 msgr2=0x7f61f80b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.919+0000 7f620051d640 1 -- 172.21.15.16:0/3415797187 shutdown_connections 2024-04-25T10:05:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:35.919+0000 7f620051d640 1 -- 172.21.15.16:0/3415797187 wait complete. 2024-04-25T10:05:35.935 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:08.724537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:09.793105+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:35.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:36.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.033+0000 7fcef82e1640 1 Processor -- start 2024-04-25T10:05:36.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.033+0000 7fcef82e1640 1 -- start start 2024-04-25T10:05:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.034+0000 7fcef82e1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcef015dca0 0x7fcef00ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.034+0000 7fcef82e1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef00d3b90 0x7fcef00d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.034+0000 7fcef82e1640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fcef00cee80 0x7fcef00d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.034+0000 7fcef82e1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef00b6b20 con 0x7fcef015dca0 2024-04-25T10:05:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.034+0000 7fcef82e1640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef00d4090 con 0x7fcef00cee80 2024-04-25T10:05:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.034+0000 7fcef82e1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00d4370 con 0x7fcef00d3b90 2024-04-25T10:05:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.034+0000 7fcef607d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcef015dca0 0x7fcef00ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.034+0000 7fcef607d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcef015dca0 0x7fcef00ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37406/0 (socket says 172.21.15.16:37406) 2024-04-25T10:05:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.034+0000 7fcef607d640 1 -- 172.21.15.16:0/1232807680 learned_addr learned my addr 172.21.15.16:0/1232807680 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.034+0000 7fcef687e640 1 --2- 172.21.15.16:0/1232807680 >> v2:172.21.15.186:3300/0 conn(0x7fcef00cee80 0x7fcef00d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.034+0000 7fcef587c640 1 --2- 172.21.15.16:0/1232807680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef00d3b90 0x7fcef00d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.035+0000 7fcef607d640 1 -- 172.21.15.16:0/1232807680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef00d3b90 msgr2=0x7fcef00d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.035+0000 7fcef607d640 1 --2- 172.21.15.16:0/1232807680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef00d3b90 0x7fcef00d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.035+0000 7fcef607d640 1 -- 172.21.15.16:0/1232807680 >> v2:172.21.15.186:3300/0 conn(0x7fcef00cee80 msgr2=0x7fcef00d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.035+0000 7fcef607d640 1 --2- 172.21.15.16:0/1232807680 >> v2:172.21.15.186:3300/0 conn(0x7fcef00cee80 0x7fcef00d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.035+0000 7fcef607d640 1 -- 172.21.15.16:0/1232807680 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef00d4650 con 0x7fcef015dca0 2024-04-25T10:05:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.035+0000 7fcef587c640 1 --2- 172.21.15.16:0/1232807680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef00d3b90 0x7fcef00d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.035+0000 7fcef607d640 1 --2- 172.21.15.16:0/1232807680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef015dca0 0x7fcef00ce940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcef00d04c0 tx=0x7fcee802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f01777e76ae38c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.035+0000 7fcef507b640 1 -- 172.21.15.16:0/1232807680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8037070 con 0x7fcef015dca0 2024-04-25T10:05:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.035+0000 7fcef507b640 1 -- 172.21.15.16:0/1232807680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee8032440 con 0x7fcef015dca0 2024-04-25T10:05:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.035+0000 7fcef507b640 1 -- 172.21.15.16:0/1232807680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8032810 con 0x7fcef015dca0 2024-04-25T10:05:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.036+0000 7fcef82e1640 1 -- 172.21.15.16:0/1232807680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef015dca0 msgr2=0x7fcef00ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.036+0000 7fcef82e1640 1 --2- 172.21.15.16:0/1232807680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef015dca0 0x7fcef00ce940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcef00d04c0 tx=0x7fcee802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.036+0000 7fcef82e1640 1 -- 172.21.15.16:0/1232807680 shutdown_connections 2024-04-25T10:05:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.036+0000 7fcef82e1640 1 --2- 172.21.15.16:0/1232807680 >> v2:172.21.15.186:3300/0 conn(0x7fcef00cee80 0x7fcef00d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.036+0000 7fcef82e1640 1 --2- 172.21.15.16:0/1232807680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef00d3b90 0x7fcef00d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.036+0000 7fcef82e1640 1 --2- 172.21.15.16:0/1232807680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef015dca0 0x7fcef00ce940 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.036+0000 7fcef82e1640 1 -- 172.21.15.16:0/1232807680 >> 172.21.15.16:0/1232807680 conn(0x7fcef00726c0 msgr2=0x7fcef0072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.036+0000 7fcef82e1640 1 -- 172.21.15.16:0/1232807680 shutdown_connections 2024-04-25T10:05:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.036+0000 7fcef82e1640 1 -- 172.21.15.16:0/1232807680 wait complete. 2024-04-25T10:05:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.037+0000 7fcef82e1640 1 Processor -- start 2024-04-25T10:05:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.037+0000 7fcef82e1640 1 -- start start 2024-04-25T10:05:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.038+0000 7fcef82e1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcef00cee80 0x7fcef0159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.038+0000 7fcef82e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef00d3b90 0x7fcef0158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.038+0000 7fcef82e1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef015dca0 0x7fcef0160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.038+0000 7fcef82e1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef015e200 con 0x7fcef00d3b90 2024-04-25T10:05:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.038+0000 7fcef82e1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef015e080 con 0x7fcef00cee80 2024-04-25T10:05:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.038+0000 7fcef82e1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00b2500 con 0x7fcef015dca0 2024-04-25T10:05:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.038+0000 7fcef587c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef00d3b90 0x7fcef0158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.038+0000 7fcef587c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef00d3b90 0x7fcef0158bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37408/0 (socket says 172.21.15.16:37408) 2024-04-25T10:05:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.038+0000 7fcef587c640 1 -- 172.21.15.16:0/3449788962 learned_addr learned my addr 172.21.15.16:0/3449788962 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.038+0000 7fcef607d640 1 --2- 172.21.15.16:0/3449788962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcef00cee80 0x7fcef0159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.038+0000 7fcef687e640 1 --2- 172.21.15.16:0/3449788962 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef015dca0 0x7fcef0160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.038+0000 7fcef587c640 1 -- 172.21.15.16:0/3449788962 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef015dca0 msgr2=0x7fcef0160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.038+0000 7fcef587c640 1 --2- 172.21.15.16:0/3449788962 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef015dca0 0x7fcef0160ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.038+0000 7fcef587c640 1 -- 172.21.15.16:0/3449788962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcef00cee80 msgr2=0x7fcef0159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.038+0000 7fcef587c640 1 --2- 172.21.15.16:0/3449788962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcef00cee80 0x7fcef0159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.038+0000 7fcef587c640 1 -- 172.21.15.16:0/3449788962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef015e420 con 0x7fcef00d3b90 2024-04-25T10:05:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.039+0000 7fcef587c640 1 --2- 172.21.15.16:0/3449788962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef00d3b90 0x7fcef0158bb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcee400e810 tx=0x7fcee400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.039+0000 7fceeeffd640 1 -- 172.21.15.16:0/3449788962 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee400f040 con 0x7fcef00d3b90 2024-04-25T10:05:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.039+0000 7fceeeffd640 1 -- 172.21.15.16:0/3449788962 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee4004030 con 0x7fcef00d3b90 2024-04-25T10:05:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.039+0000 7fcef82e1640 1 -- 172.21.15.16:0/3449788962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef0159f10 con 0x7fcef00d3b90 2024-04-25T10:05:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.039+0000 7fceeeffd640 1 -- 172.21.15.16:0/3449788962 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4004400 con 0x7fcef00d3b90 2024-04-25T10:05:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.039+0000 7fcef82e1640 1 -- 172.21.15.16:0/3449788962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef015e6c0 con 0x7fcef00d3b90 2024-04-25T10:05:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.040+0000 7fcef82e1640 1 -- 172.21.15.16:0/3449788962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef0159150 con 0x7fcef00d3b90 2024-04-25T10:05:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.040+0000 7fceeeffd640 1 -- 172.21.15.16:0/3449788962 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcee400c6d0 con 0x7fcef00d3b90 2024-04-25T10:05:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.041+0000 7fceeeffd640 1 --2- 172.21.15.16:0/3449788962 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcedc06b500 0x7fcedc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.041+0000 7fceeeffd640 1 -- 172.21.15.16:0/3449788962 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcee4089d80 con 0x7fcef00d3b90 2024-04-25T10:05:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.041+0000 7fcef607d640 1 --2- 172.21.15.16:0/3449788962 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcedc06b500 0x7fcedc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.041+0000 7fcef607d640 1 --2- 172.21.15.16:0/3449788962 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcedc06b500 0x7fcedc06d9c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcee8034000 tx=0x7fcee802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.045+0000 7fceeeffd640 1 -- 172.21.15.16:0/3449788962 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcee4017020 con 0x7fcef00d3b90 2024-04-25T10:05:36.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.228+0000 7fcef82e1640 1 -- 172.21.15.16:0/3449788962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcef00750b0 con 0x7fcef00d3b90 2024-04-25T10:05:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.228+0000 7fceeeffd640 1 -- 172.21.15.16:0/3449788962 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcee4058640 con 0x7fcef00d3b90 2024-04-25T10:05:36.234 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:05:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.230+0000 7fcef82e1640 1 -- 172.21.15.16:0/3449788962 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcedc06b500 msgr2=0x7fcedc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.230+0000 7fcef82e1640 1 --2- 172.21.15.16:0/3449788962 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcedc06b500 0x7fcedc06d9c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcee8034000 tx=0x7fcee802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.230+0000 7fcef82e1640 1 -- 172.21.15.16:0/3449788962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef00d3b90 msgr2=0x7fcef0158bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.230+0000 7fcef82e1640 1 --2- 172.21.15.16:0/3449788962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef00d3b90 0x7fcef0158bb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcee400e810 tx=0x7fcee400ecd0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.230+0000 7fcef82e1640 1 -- 172.21.15.16:0/3449788962 shutdown_connections 2024-04-25T10:05:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.230+0000 7fcef82e1640 1 --2- 172.21.15.16:0/3449788962 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcedc06b500 0x7fcedc06d9c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.230+0000 7fcef82e1640 1 --2- 172.21.15.16:0/3449788962 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef015dca0 0x7fcef0160ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.230+0000 7fcef82e1640 1 --2- 172.21.15.16:0/3449788962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef00d3b90 0x7fcef0158bb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.230+0000 7fcef82e1640 1 --2- 172.21.15.16:0/3449788962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcef00cee80 0x7fcef0159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.230+0000 7fcef82e1640 1 -- 172.21.15.16:0/3449788962 >> 172.21.15.16:0/3449788962 conn(0x7fcef00726c0 msgr2=0x7fcef00ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.230+0000 7fcef82e1640 1 -- 172.21.15.16:0/3449788962 shutdown_connections 2024-04-25T10:05:36.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.230+0000 7fcef82e1640 1 -- 172.21.15.16:0/3449788962 wait complete. 2024-04-25T10:05:36.246 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.345+0000 7f68872c4640 1 Processor -- start 2024-04-25T10:05:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.345+0000 7f68872c4640 1 -- start start 2024-04-25T10:05:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.346+0000 7f68872c4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688015b940 0x7f688015f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.346+0000 7f68872c4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68800c4100 con 0x7f68800b1e90 2024-04-25T10:05:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.346+0000 7f68872c4640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f688015fe40 con 0x7f688006aba0 2024-04-25T10:05:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.346+0000 7f68872c4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68801606e0 con 0x7f688015b940 2024-04-25T10:05:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.346+0000 7f6885861640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f68800b1e90 0x7f68800b0660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40906/0 (socket says 172.21.15.16:40906) 2024-04-25T10:05:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.346+0000 7f6885861640 1 -- 172.21.15.16:0/761135398 learned_addr learned my addr 172.21.15.16:0/761135398 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.346+0000 7f6885060640 1 --2- 172.21.15.16:0/761135398 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688015b940 0x7f688015f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.347+0000 7f687ffff640 1 -- 172.21.15.16:0/761135398 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2347420647 0 0) 0x7f68800c4100 con 0x7f68800b1e90 2024-04-25T10:05:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.347+0000 7f687ffff640 1 -- 172.21.15.16:0/761135398 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6868003520 con 0x7f68800b1e90 2024-04-25T10:05:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.347+0000 7f6885060640 1 -- 172.21.15.16:0/761135398 >> v1:172.21.15.186:6789/0 conn(0x7f688006aba0 legacy=0x7f68800b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.347+0000 7f6885060640 1 -- 172.21.15.16:0/761135398 >> v1:172.21.15.16:6789/0 conn(0x7f68800b1e90 legacy=0x7f68800b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.347+0000 7f6885060640 1 -- 172.21.15.16:0/761135398 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6880160120 con 0x7f688015b940 2024-04-25T10:05:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.347+0000 7f6885060640 1 --2- 172.21.15.16:0/761135398 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688015b940 0x7f688015f740 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6878009dc0 tx=0x7f687802a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9bf2c5584f55cef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.348+0000 7f687ffff640 1 -- 172.21.15.16:0/761135398 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687802ae60 con 0x7f688015b940 2024-04-25T10:05:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.348+0000 7f687ffff640 1 -- 172.21.15.16:0/761135398 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6878007740 con 0x7f688015b940 2024-04-25T10:05:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.348+0000 7f687ffff640 1 -- 172.21.15.16:0/761135398 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6878007af0 con 0x7f688015b940 2024-04-25T10:05:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.348+0000 7f68872c4640 1 -- 172.21.15.16:0/761135398 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688015b940 msgr2=0x7f688015f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.348+0000 7f68872c4640 1 --2- 172.21.15.16:0/761135398 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688015b940 0x7f688015f740 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6878009dc0 tx=0x7f687802a640 comp rx=0 tx=0).stop 2024-04-25T10:05:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.348+0000 7f68872c4640 1 -- 172.21.15.16:0/761135398 shutdown_connections 2024-04-25T10:05:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.348+0000 7f68872c4640 1 --2- 172.21.15.16:0/761135398 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688015b940 0x7f688015f740 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.348+0000 7f68872c4640 1 -- 172.21.15.16:0/761135398 >> 172.21.15.16:0/761135398 conn(0x7f68800d07d0 msgr2=0x7f68800d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.348+0000 7f68872c4640 1 -- 172.21.15.16:0/761135398 shutdown_connections 2024-04-25T10:05:36.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.348+0000 7f68872c4640 1 -- 172.21.15.16:0/761135398 wait complete. 2024-04-25T10:05:36.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.349+0000 7f68872c4640 1 Processor -- start 2024-04-25T10:05:36.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.350+0000 7f68872c4640 1 -- start start 2024-04-25T10:05:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.350+0000 7f68872c4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688006aba0 0x7f68801f1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.350+0000 7f68872c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68800b1e90 0x7f68800c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.350+0000 7f68872c4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f688015b940 0x7f68800ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.350+0000 7f68872c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688015cd30 con 0x7f68800b1e90 2024-04-25T10:05:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.350+0000 7f68872c4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68800b0160 con 0x7f688015b940 2024-04-25T10:05:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.350+0000 7f68872c4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688015ceb0 con 0x7f688006aba0 2024-04-25T10:05:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.350+0000 7f688485f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68800b1e90 0x7f68800c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.350+0000 7f688485f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68800b1e90 0x7f68800c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37412/0 (socket says 172.21.15.16:37412) 2024-04-25T10:05:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.350+0000 7f688485f640 1 -- 172.21.15.16:0/3901904055 learned_addr learned my addr 172.21.15.16:0/3901904055 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.350+0000 7f6885060640 1 --2- 172.21.15.16:0/3901904055 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688006aba0 0x7f68801f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.350+0000 7f6885861640 1 --2- 172.21.15.16:0/3901904055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f688015b940 0x7f68800ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.351+0000 7f688485f640 1 -- 172.21.15.16:0/3901904055 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688006aba0 msgr2=0x7f68801f1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.351+0000 7f688485f640 1 --2- 172.21.15.16:0/3901904055 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688006aba0 0x7f68801f1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.351+0000 7f688485f640 1 -- 172.21.15.16:0/3901904055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f688015b940 msgr2=0x7f68800ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.351+0000 7f688485f640 1 --2- 172.21.15.16:0/3901904055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f688015b940 0x7f68800ae550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.351+0000 7f688485f640 1 -- 172.21.15.16:0/3901904055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68800c8280 con 0x7f68800b1e90 2024-04-25T10:05:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.351+0000 7f688485f640 1 --2- 172.21.15.16:0/3901904055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68800b1e90 0x7f68800c78a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f687400c3c0 tx=0x7f687400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.351+0000 7f687dffb640 1 -- 172.21.15.16:0/3901904055 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68740052d0 con 0x7f68800b1e90 2024-04-25T10:05:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.351+0000 7f687dffb640 1 -- 172.21.15.16:0/3901904055 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6874005490 con 0x7f68800b1e90 2024-04-25T10:05:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.351+0000 7f68872c4640 1 -- 172.21.15.16:0/3901904055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68800c8560 con 0x7f68800b1e90 2024-04-25T10:05:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.351+0000 7f687dffb640 1 -- 172.21.15.16:0/3901904055 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6874007260 con 0x7f68800b1e90 2024-04-25T10:05:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.351+0000 7f68872c4640 1 -- 172.21.15.16:0/3901904055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68800c8b20 con 0x7f68800b1e90 2024-04-25T10:05:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.352+0000 7f68872c4640 1 -- 172.21.15.16:0/3901904055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68800750b0 con 0x7f68800b1e90 2024-04-25T10:05:36.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.353+0000 7f687dffb640 1 -- 172.21.15.16:0/3901904055 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6874007400 con 0x7f68800b1e90 2024-04-25T10:05:36.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.353+0000 7f687dffb640 1 --2- 172.21.15.16:0/3901904055 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f686c06b500 0x7f686c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.353+0000 7f687dffb640 1 -- 172.21.15.16:0/3901904055 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68740895a0 con 0x7f68800b1e90 2024-04-25T10:05:36.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.353+0000 7f6885060640 1 --2- 172.21.15.16:0/3901904055 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f686c06b500 0x7f686c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.354+0000 7f6885060640 1 --2- 172.21.15.16:0/3901904055 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f686c06b500 0x7f686c06d9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6878034000 tx=0x7f687802f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.357+0000 7f687dffb640 1 -- 172.21.15.16:0/3901904055 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6874057dd0 con 0x7f68800b1e90 2024-04-25T10:05:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.540+0000 7f68872c4640 1 -- 172.21.15.16:0/3901904055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f688015c6e0 con 0x7f68800b1e90 2024-04-25T10:05:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.541+0000 7f687dffb640 1 -- 172.21.15.16:0/3901904055 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f687405be70 con 0x7f68800b1e90 2024-04-25T10:05:36.546 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:05:36.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.543+0000 7f68872c4640 1 -- 172.21.15.16:0/3901904055 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f686c06b500 msgr2=0x7f686c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.543+0000 7f68872c4640 1 --2- 172.21.15.16:0/3901904055 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f686c06b500 0x7f686c06d9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6878034000 tx=0x7f687802f000 comp rx=0 tx=0).stop 2024-04-25T10:05:36.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.543+0000 7f68872c4640 1 -- 172.21.15.16:0/3901904055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68800b1e90 msgr2=0x7f68800c78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.543+0000 7f68872c4640 1 --2- 172.21.15.16:0/3901904055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68800b1e90 0x7f68800c78a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f687400c3c0 tx=0x7f687400c880 comp rx=0 tx=0).stop 2024-04-25T10:05:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.543+0000 7f68872c4640 1 -- 172.21.15.16:0/3901904055 shutdown_connections 2024-04-25T10:05:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.543+0000 7f68872c4640 1 --2- 172.21.15.16:0/3901904055 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f686c06b500 0x7f686c06d9c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.543+0000 7f68872c4640 1 --2- 172.21.15.16:0/3901904055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f688015b940 0x7f68800ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.543+0000 7f68872c4640 1 --2- 172.21.15.16:0/3901904055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68800b1e90 0x7f68800c78a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.543+0000 7f68872c4640 1 --2- 172.21.15.16:0/3901904055 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f688006aba0 0x7f68801f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.543+0000 7f68872c4640 1 -- 172.21.15.16:0/3901904055 >> 172.21.15.16:0/3901904055 conn(0x7f68800d07d0 msgr2=0x7f688006f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.543+0000 7f68872c4640 1 -- 172.21.15.16:0/3901904055 shutdown_connections 2024-04-25T10:05:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.543+0000 7f68872c4640 1 -- 172.21.15.16:0/3901904055 wait complete. 2024-04-25T10:05:36.559 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:36.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.658+0000 7fae718cd640 1 Processor -- start 2024-04-25T10:05:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.658+0000 7fae718cd640 1 -- start start 2024-04-25T10:05:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.659+0000 7fae718cd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fae6c15dc80 0x7fae6c161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.659+0000 7fae718cd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae6c1627a0 con 0x7fae6c0b5c00 2024-04-25T10:05:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.659+0000 7fae718cd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae6c1621e0 con 0x7fae6c0b97c0 2024-04-25T10:05:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.659+0000 7fae718cd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae6c1624c0 con 0x7fae6c15dc80 2024-04-25T10:05:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.659+0000 7fae6affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fae6c15dc80 0x7fae6c161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.659+0000 7fae6b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fae6c0b5c00 0x7fae6c0d2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40910/0 (socket says 172.21.15.16:40910) 2024-04-25T10:05:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.659+0000 7fae6b7fe640 1 -- 172.21.15.16:0/2821905960 learned_addr learned my addr 172.21.15.16:0/2821905960 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.659+0000 7fae6affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fae6c15dc80 0x7fae6c161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37418/0 (socket says 172.21.15.16:37418) 2024-04-25T10:05:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.659+0000 7fae6affd640 1 -- 172.21.15.16:0/2821905960 >> v1:172.21.15.186:6789/0 conn(0x7fae6c0b97c0 legacy=0x7fae6c0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.659+0000 7fae6affd640 1 -- 172.21.15.16:0/2821905960 >> v1:172.21.15.16:6789/0 conn(0x7fae6c0b5c00 legacy=0x7fae6c0d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.659+0000 7fae6affd640 1 -- 172.21.15.16:0/2821905960 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6c0d1c70 con 0x7fae6c15dc80 2024-04-25T10:05:36.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.660+0000 7fae6affd640 1 --2- 172.21.15.16:0/2821905960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae6c15dc80 0x7fae6c161870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fae640097a0 tx=0x7fae6402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68e2159802a3f69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.660+0000 7fae69ffb640 1 -- 172.21.15.16:0/2821905960 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae64037070 con 0x7fae6c15dc80 2024-04-25T10:05:36.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.660+0000 7fae69ffb640 1 -- 172.21.15.16:0/2821905960 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae64032440 con 0x7fae6c15dc80 2024-04-25T10:05:36.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.660+0000 7fae69ffb640 1 -- 172.21.15.16:0/2821905960 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae64032810 con 0x7fae6c15dc80 2024-04-25T10:05:36.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.661+0000 7fae718cd640 1 -- 172.21.15.16:0/2821905960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae6c15dc80 msgr2=0x7fae6c161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.661+0000 7fae718cd640 1 --2- 172.21.15.16:0/2821905960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae6c15dc80 0x7fae6c161870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fae640097a0 tx=0x7fae6402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.661+0000 7fae718cd640 1 -- 172.21.15.16:0/2821905960 shutdown_connections 2024-04-25T10:05:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.661+0000 7fae718cd640 1 --2- 172.21.15.16:0/2821905960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae6c15dc80 0x7fae6c161870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.661+0000 7fae718cd640 1 -- 172.21.15.16:0/2821905960 >> 172.21.15.16:0/2821905960 conn(0x7fae6c0d04f0 msgr2=0x7fae6c0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.661+0000 7fae718cd640 1 -- 172.21.15.16:0/2821905960 shutdown_connections 2024-04-25T10:05:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.661+0000 7fae718cd640 1 -- 172.21.15.16:0/2821905960 wait complete. 2024-04-25T10:05:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.662+0000 7fae718cd640 1 Processor -- start 2024-04-25T10:05:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.662+0000 7fae718cd640 1 -- start start 2024-04-25T10:05:36.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.663+0000 7fae718cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae6c0b5c00 0x7fae6c0cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.663+0000 7fae718cd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae6c0b97c0 0x7fae6c0c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.663+0000 7fae718cd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae6c15dc80 0x7fae6c149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.663+0000 7fae718cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c161f30 con 0x7fae6c0b5c00 2024-04-25T10:05:36.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.663+0000 7fae718cd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c161db0 con 0x7fae6c0b97c0 2024-04-25T10:05:36.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.663+0000 7fae718cd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c0d2b80 con 0x7fae6c15dc80 2024-04-25T10:05:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.663+0000 7fae6affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae6c0b5c00 0x7fae6c0cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.663+0000 7fae6affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae6c0b5c00 0x7fae6c0cd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37430/0 (socket says 172.21.15.16:37430) 2024-04-25T10:05:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.663+0000 7fae6affd640 1 -- 172.21.15.16:0/91527218 learned_addr learned my addr 172.21.15.16:0/91527218 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.663+0000 7fae6affd640 1 -- 172.21.15.16:0/91527218 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae6c15dc80 msgr2=0x7fae6c149940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.663+0000 7fae6affd640 1 --2- 172.21.15.16:0/91527218 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae6c15dc80 0x7fae6c149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.663+0000 7fae6affd640 1 -- 172.21.15.16:0/91527218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae6c0b97c0 msgr2=0x7fae6c0c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.663+0000 7fae6affd640 1 --2- 172.21.15.16:0/91527218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae6c0b97c0 0x7fae6c0c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.663+0000 7fae6affd640 1 -- 172.21.15.16:0/91527218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6c1f74c0 con 0x7fae6c0b5c00 2024-04-25T10:05:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.663+0000 7fae6affd640 1 --2- 172.21.15.16:0/91527218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae6c0b5c00 0x7fae6c0cd370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae640098c0 tx=0x7fae640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.664+0000 7fae63fff640 1 -- 172.21.15.16:0/91527218 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae64037040 con 0x7fae6c0b5c00 2024-04-25T10:05:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.664+0000 7fae63fff640 1 -- 172.21.15.16:0/91527218 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae64042070 con 0x7fae6c0b5c00 2024-04-25T10:05:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.664+0000 7fae718cd640 1 -- 172.21.15.16:0/91527218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae6c1f77a0 con 0x7fae6c0b5c00 2024-04-25T10:05:36.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.664+0000 7fae63fff640 1 -- 172.21.15.16:0/91527218 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae64030a70 con 0x7fae6c0b5c00 2024-04-25T10:05:36.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.664+0000 7fae718cd640 1 -- 172.21.15.16:0/91527218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6c1f7d60 con 0x7fae6c0b5c00 2024-04-25T10:05:36.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.665+0000 7fae718cd640 1 -- 172.21.15.16:0/91527218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae6c0750b0 con 0x7fae6c0b5c00 2024-04-25T10:05:36.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.665+0000 7fae63fff640 1 -- 172.21.15.16:0/91527218 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae64030c10 con 0x7fae6c0b5c00 2024-04-25T10:05:36.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.666+0000 7fae63fff640 1 --2- 172.21.15.16:0/91527218 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae4c06b4d0 0x7fae4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.666+0000 7fae63fff640 1 -- 172.21.15.16:0/91527218 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae640aacf0 con 0x7fae6c0b5c00 2024-04-25T10:05:36.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.666+0000 7fae6a7fc640 1 --2- 172.21.15.16:0/91527218 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae4c06b4d0 0x7fae4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.666+0000 7fae6a7fc640 1 --2- 172.21.15.16:0/91527218 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae4c06b4d0 0x7fae4c06d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fae6c1e8430 tx=0x7fae540079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.670+0000 7fae63fff640 1 -- 172.21.15.16:0/91527218 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae640af070 con 0x7fae6c0b5c00 2024-04-25T10:05:36.685 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:36.688 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:36.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.852+0000 7fae718cd640 1 -- 172.21.15.16:0/91527218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae6c0cff50 con 0x7fae6c0b5c00 2024-04-25T10:05:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.853+0000 7fae63fff640 1 -- 172.21.15.16:0/91527218 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae64045360 con 0x7fae6c0b5c00 2024-04-25T10:05:36.858 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:05:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.854+0000 7fae718cd640 1 -- 172.21.15.16:0/91527218 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae4c06b4d0 msgr2=0x7fae4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.854+0000 7fae718cd640 1 --2- 172.21.15.16:0/91527218 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae4c06b4d0 0x7fae4c06d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fae6c1e8430 tx=0x7fae540079b0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.854+0000 7fae718cd640 1 -- 172.21.15.16:0/91527218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae6c0b5c00 msgr2=0x7fae6c0cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.854+0000 7fae718cd640 1 --2- 172.21.15.16:0/91527218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae6c0b5c00 0x7fae6c0cd370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae640098c0 tx=0x7fae640046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.854+0000 7fae718cd640 1 -- 172.21.15.16:0/91527218 shutdown_connections 2024-04-25T10:05:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.854+0000 7fae718cd640 1 --2- 172.21.15.16:0/91527218 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae4c06b4d0 0x7fae4c06d990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.854+0000 7fae718cd640 1 --2- 172.21.15.16:0/91527218 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae6c15dc80 0x7fae6c149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.854+0000 7fae718cd640 1 --2- 172.21.15.16:0/91527218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae6c0b97c0 0x7fae6c0c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.854+0000 7fae718cd640 1 --2- 172.21.15.16:0/91527218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae6c0b5c00 0x7fae6c0cd370 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.854+0000 7fae718cd640 1 -- 172.21.15.16:0/91527218 >> 172.21.15.16:0/91527218 conn(0x7fae6c0d04f0 msgr2=0x7fae6c0b7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.855+0000 7fae718cd640 1 -- 172.21.15.16:0/91527218 shutdown_connections 2024-04-25T10:05:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.855+0000 7fae718cd640 1 -- 172.21.15.16:0/91527218 wait complete. 2024-04-25T10:05:36.870 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:36.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.966+0000 7f2dc0022640 1 Processor -- start 2024-04-25T10:05:36.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.966+0000 7f2dc0022640 1 -- start start 2024-04-25T10:05:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.967+0000 7f2dc0022640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db815dce0 0x7f2db80d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.967+0000 7f2dc0022640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2db80d35b0 0x7f2db80b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.967+0000 7f2dc0022640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db815e0c0 con 0x7f2db80d1f90 2024-04-25T10:05:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.967+0000 7f2dc0022640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2db80b7b10 con 0x7f2db80d35b0 2024-04-25T10:05:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.967+0000 7f2dc0022640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db80d2510 con 0x7f2db815dce0 2024-04-25T10:05:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.968+0000 7f2dbddbe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db815dce0 0x7f2db80d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.968+0000 7f2dbd5bd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2db80d35b0 0x7f2db80b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.968+0000 7f2dbddbe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db815dce0 0x7f2db80d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:49440/0 (socket says 172.21.15.16:49440) 2024-04-25T10:05:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.968+0000 7f2dbd5bd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2db80d35b0 0x7f2db80b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:48156/0 (socket says 172.21.15.16:48156) 2024-04-25T10:05:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.968+0000 7f2dbddbe640 1 -- 172.21.15.16:0/3423349704 learned_addr learned my addr 172.21.15.16:0/3423349704 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.968+0000 7f2dbe5bf640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f2db80d1f90 0x7f2db806bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:36504/0 (socket says 172.21.15.16:36504) 2024-04-25T10:05:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.968+0000 7f2dbd5bd640 1 -- 172.21.15.16:0/3423349704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db815dce0 msgr2=0x7f2db80d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.968+0000 7f2dbd5bd640 1 --2- 172.21.15.16:0/3423349704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db815dce0 0x7f2db80d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.968+0000 7f2dbd5bd640 1 -- 172.21.15.16:0/3423349704 >> v1:172.21.15.186:6789/0 conn(0x7f2db80d1f90 legacy=0x7f2db806bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.968+0000 7f2dbd5bd640 1 -- 172.21.15.16:0/3423349704 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db80d27f0 con 0x7f2db80d35b0 2024-04-25T10:05:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.968+0000 7f2dbcdbc640 1 -- 172.21.15.16:0/3423349704 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3955147182 0 0) 0x7f2db815e0c0 con 0x7f2db80d1f90 2024-04-25T10:05:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.968+0000 7f2dbddbe640 1 --2- 172.21.15.16:0/3423349704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db815dce0 0x7f2db80d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.969+0000 7f2dbd5bd640 1 --2- 172.21.15.16:0/3423349704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2db80d35b0 0x7f2db80b5450 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0097a0 tx=0x7f2dac02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43cb5240a3de1448 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.969+0000 7f2dbcdbc640 1 -- 172.21.15.16:0/3423349704 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac037070 con 0x7f2db80d35b0 2024-04-25T10:05:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.969+0000 7f2dbcdbc640 1 -- 172.21.15.16:0/3423349704 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dac02ae50 con 0x7f2db80d35b0 2024-04-25T10:05:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.969+0000 7f2dbcdbc640 1 -- 172.21.15.16:0/3423349704 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac032660 con 0x7f2db80d35b0 2024-04-25T10:05:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.970+0000 7f2dc0022640 1 -- 172.21.15.16:0/3423349704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2db80d35b0 msgr2=0x7f2db80b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.970+0000 7f2dc0022640 1 --2- 172.21.15.16:0/3423349704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2db80d35b0 0x7f2db80b5450 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0097a0 tx=0x7f2dac02a100 comp rx=0 tx=0).stop 2024-04-25T10:05:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.970+0000 7f2dc0022640 1 -- 172.21.15.16:0/3423349704 shutdown_connections 2024-04-25T10:05:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.970+0000 7f2dc0022640 1 --2- 172.21.15.16:0/3423349704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2db80d35b0 0x7f2db80b5450 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.970+0000 7f2dc0022640 1 --2- 172.21.15.16:0/3423349704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db815dce0 0x7f2db80d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.970+0000 7f2dc0022640 1 -- 172.21.15.16:0/3423349704 >> 172.21.15.16:0/3423349704 conn(0x7f2db8072900 msgr2=0x7f2db80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.970+0000 7f2dc0022640 1 -- 172.21.15.16:0/3423349704 shutdown_connections 2024-04-25T10:05:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.970+0000 7f2dc0022640 1 -- 172.21.15.16:0/3423349704 wait complete. 2024-04-25T10:05:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.971+0000 7f2dc0022640 1 Processor -- start 2024-04-25T10:05:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.971+0000 7f2dc0022640 1 -- start start 2024-04-25T10:05:36.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.972+0000 7f2dc0022640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db80d1f90 0x7f2db81f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.972+0000 7f2dc0022640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db80d35b0 0x7f2db8157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.972+0000 7f2dc0022640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2db815dce0 0x7f2db815f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.972+0000 7f2dc0022640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db80d3090 con 0x7f2db80d35b0 2024-04-25T10:05:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.972+0000 7f2dc0022640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db80d2db0 con 0x7f2db815dce0 2024-04-25T10:05:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.972+0000 7f2dc0022640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db80b9820 con 0x7f2db80d1f90 2024-04-25T10:05:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.972+0000 7f2dbd5bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db80d35b0 0x7f2db8157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.972+0000 7f2dbd5bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db80d35b0 0x7f2db8157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37444/0 (socket says 172.21.15.16:37444) 2024-04-25T10:05:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.972+0000 7f2dbd5bd640 1 -- 172.21.15.16:0/69469072 learned_addr learned my addr 172.21.15.16:0/69469072 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.972+0000 7f2dbe5bf640 1 --2- 172.21.15.16:0/69469072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2db815dce0 0x7f2db815f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.972+0000 7f2dbddbe640 1 --2- 172.21.15.16:0/69469072 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db80d1f90 0x7f2db81f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.972+0000 7f2dbd5bd640 1 -- 172.21.15.16:0/69469072 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db80d1f90 msgr2=0x7f2db81f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.972+0000 7f2dbd5bd640 1 --2- 172.21.15.16:0/69469072 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db80d1f90 0x7f2db81f46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.972+0000 7f2dbd5bd640 1 -- 172.21.15.16:0/69469072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2db815dce0 msgr2=0x7f2db815f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.972+0000 7f2dbd5bd640 1 --2- 172.21.15.16:0/69469072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2db815dce0 0x7f2db815f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.972+0000 7f2dbd5bd640 1 -- 172.21.15.16:0/69469072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db81f4c00 con 0x7f2db80d35b0 2024-04-25T10:05:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.972+0000 7f2dbd5bd640 1 --2- 172.21.15.16:0/69469072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db80d35b0 0x7f2db8157030 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0098c0 tx=0x7f2dac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.973+0000 7f2db67fc640 1 -- 172.21.15.16:0/69469072 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac041030 con 0x7f2db80d35b0 2024-04-25T10:05:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.973+0000 7f2db67fc640 1 -- 172.21.15.16:0/69469072 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dac02ac50 con 0x7f2db80d35b0 2024-04-25T10:05:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.973+0000 7f2db67fc640 1 -- 172.21.15.16:0/69469072 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac037040 con 0x7f2db80d35b0 2024-04-25T10:05:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.973+0000 7f2dc0022640 1 -- 172.21.15.16:0/69469072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db815fb20 con 0x7f2db80d35b0 2024-04-25T10:05:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.973+0000 7f2dc0022640 1 -- 172.21.15.16:0/69469072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db80c8c30 con 0x7f2db80d35b0 2024-04-25T10:05:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.974+0000 7f2dc0022640 1 -- 172.21.15.16:0/69469072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db80749b0 con 0x7f2db80d35b0 2024-04-25T10:05:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.974+0000 7f2db67fc640 1 -- 172.21.15.16:0/69469072 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2dac044070 con 0x7f2db80d35b0 2024-04-25T10:05:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.974+0000 7f2db67fc640 1 --2- 172.21.15.16:0/69469072 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2da406b500 0x7f2da406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.974+0000 7f2db67fc640 1 -- 172.21.15.16:0/69469072 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2dac0aa5a0 con 0x7f2db80d35b0 2024-04-25T10:05:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.975+0000 7f2dbddbe640 1 --2- 172.21.15.16:0/69469072 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2da406b500 0x7f2da406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.975+0000 7f2dbddbe640 1 --2- 172.21.15.16:0/69469072 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2da406b500 0x7f2da406d9c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2db0006ca0 tx=0x7f2db0009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:36.979+0000 7f2db67fc640 1 -- 172.21.15.16:0/69469072 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2db81f4c00 con 0x7f2db80d35b0 2024-04-25T10:05:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.159+0000 7f2dc0022640 1 -- 172.21.15.16:0/69469072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2db806b080 con 0x7f2db80d35b0 2024-04-25T10:05:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.160+0000 7f2db67fc640 1 -- 172.21.15.16:0/69469072 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f2dac078ef0 con 0x7f2db80d35b0 2024-04-25T10:05:37.166 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:37.166 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:12.449183+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.162+0000 7f2dc0022640 1 -- 172.21.15.16:0/69469072 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2da406b500 msgr2=0x7f2da406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.162+0000 7f2dc0022640 1 --2- 172.21.15.16:0/69469072 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2da406b500 0x7f2da406d9c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2db0006ca0 tx=0x7f2db0009240 comp rx=0 tx=0).stop 2024-04-25T10:05:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.162+0000 7f2dc0022640 1 -- 172.21.15.16:0/69469072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db80d35b0 msgr2=0x7f2db8157030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.162+0000 7f2dc0022640 1 --2- 172.21.15.16:0/69469072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db80d35b0 0x7f2db8157030 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0098c0 tx=0x7f2dac0046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.162+0000 7f2dc0022640 1 -- 172.21.15.16:0/69469072 shutdown_connections 2024-04-25T10:05:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.162+0000 7f2dc0022640 1 --2- 172.21.15.16:0/69469072 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2da406b500 0x7f2da406d9c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.162+0000 7f2dc0022640 1 --2- 172.21.15.16:0/69469072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2db815dce0 0x7f2db815f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.162+0000 7f2dc0022640 1 --2- 172.21.15.16:0/69469072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db80d35b0 0x7f2db8157030 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.162+0000 7f2dc0022640 1 --2- 172.21.15.16:0/69469072 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db80d1f90 0x7f2db81f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.162+0000 7f2dc0022640 1 -- 172.21.15.16:0/69469072 >> 172.21.15.16:0/69469072 conn(0x7f2db8072900 msgr2=0x7f2db80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.162+0000 7f2dc0022640 1 -- 172.21.15.16:0/69469072 shutdown_connections 2024-04-25T10:05:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.162+0000 7f2dc0022640 1 -- 172.21.15.16:0/69469072 wait complete. 2024-04-25T10:05:37.179 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:08.724537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:09.793105+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:37.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:37.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.277+0000 7f4ea53f9640 1 Processor -- start 2024-04-25T10:05:37.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.277+0000 7f4ea53f9640 1 -- start start 2024-04-25T10:05:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.278+0000 7f4ea53f9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ea00b2740 0x7f4ea015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.278+0000 7f4ea53f9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ea00ce1a0 0x7f4ea00ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.278+0000 7f4ea53f9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea006c150 con 0x7f4ea00d0490 2024-04-25T10:05:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.278+0000 7f4ea53f9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea00d2000 con 0x7f4ea00b2740 2024-04-25T10:05:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.278+0000 7f4ea53f9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea006cd60 con 0x7f4ea00ce1a0 2024-04-25T10:05:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.278+0000 7f4e97fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ea00b2740 0x7f4ea015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.278+0000 7f4e97fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ea00b2740 0x7f4ea015ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37448/0 (socket says 172.21.15.16:37448) 2024-04-25T10:05:37.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.278+0000 7f4e97fff640 1 -- 172.21.15.16:0/3084733917 learned_addr learned my addr 172.21.15.16:0/3084733917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:37.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.278+0000 7f4e97fff640 1 -- 172.21.15.16:0/3084733917 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ea00ce1a0 msgr2=0x7f4ea00ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:37.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.278+0000 7f4e97fff640 1 --2- 172.21.15.16:0/3084733917 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ea00ce1a0 0x7f4ea00ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.278+0000 7f4e97fff640 1 -- 172.21.15.16:0/3084733917 >> v1:172.21.15.186:6789/0 conn(0x7f4ea00d0490 legacy=0x7f4ea00d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:37.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.278+0000 7f4e97fff640 1 -- 172.21.15.16:0/3084733917 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea006c430 con 0x7f4ea00b2740 2024-04-25T10:05:37.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.279+0000 7f4e97fff640 1 --2- 172.21.15.16:0/3084733917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea00b2740 0x7f4ea015ec90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4e880098a0 tx=0x7f4e8802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e79740af580e29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.279+0000 7f4e9e7fc640 1 -- 172.21.15.16:0/3084733917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e88037070 con 0x7f4ea00b2740 2024-04-25T10:05:37.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.279+0000 7f4e9e7fc640 1 -- 172.21.15.16:0/3084733917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e88032440 con 0x7f4ea00b2740 2024-04-25T10:05:37.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.279+0000 7f4e9e7fc640 1 -- 172.21.15.16:0/3084733917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e88032810 con 0x7f4ea00b2740 2024-04-25T10:05:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.280+0000 7f4ea53f9640 1 -- 172.21.15.16:0/3084733917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea00b2740 msgr2=0x7f4ea015ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.280+0000 7f4ea53f9640 1 --2- 172.21.15.16:0/3084733917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea00b2740 0x7f4ea015ec90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4e880098a0 tx=0x7f4e8802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.280+0000 7f4ea53f9640 1 -- 172.21.15.16:0/3084733917 shutdown_connections 2024-04-25T10:05:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.280+0000 7f4ea53f9640 1 --2- 172.21.15.16:0/3084733917 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ea00ce1a0 0x7f4ea00ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.280+0000 7f4ea53f9640 1 --2- 172.21.15.16:0/3084733917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea00b2740 0x7f4ea015ec90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.280+0000 7f4ea53f9640 1 -- 172.21.15.16:0/3084733917 >> 172.21.15.16:0/3084733917 conn(0x7f4ea00b5660 msgr2=0x7f4ea00b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.280+0000 7f4ea53f9640 1 -- 172.21.15.16:0/3084733917 shutdown_connections 2024-04-25T10:05:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.280+0000 7f4ea53f9640 1 -- 172.21.15.16:0/3084733917 wait complete. 2024-04-25T10:05:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.281+0000 7f4ea53f9640 1 Processor -- start 2024-04-25T10:05:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.281+0000 7f4ea53f9640 1 -- start start 2024-04-25T10:05:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4ea53f9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ea00b2740 0x7f4ea015b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4ea53f9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea00ce1a0 0x7f4ea0159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4ea53f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea00d0490 0x7f4ea015a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4ea53f9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea006c9f0 con 0x7f4ea00d0490 2024-04-25T10:05:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4ea53f9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea015f1d0 con 0x7f4ea00ce1a0 2024-04-25T10:05:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4ea53f9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea015e020 con 0x7f4ea00b2740 2024-04-25T10:05:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4e9f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea00d0490 0x7f4ea015a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4e9f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea00d0490 0x7f4ea015a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37464/0 (socket says 172.21.15.16:37464) 2024-04-25T10:05:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4e9f7fe640 1 -- 172.21.15.16:0/821066844 learned_addr learned my addr 172.21.15.16:0/821066844 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4e9f7fe640 1 -- 172.21.15.16:0/821066844 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ea00b2740 msgr2=0x7f4ea015b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4e9f7fe640 1 --2- 172.21.15.16:0/821066844 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ea00b2740 0x7f4ea015b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4e9f7fe640 1 -- 172.21.15.16:0/821066844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea00ce1a0 msgr2=0x7f4ea0159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4e9f7fe640 1 --2- 172.21.15.16:0/821066844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea00ce1a0 0x7f4ea0159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4e9f7fe640 1 -- 172.21.15.16:0/821066844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea0146330 con 0x7f4ea00d0490 2024-04-25T10:05:37.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4e9f7fe640 1 --2- 172.21.15.16:0/821066844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea00d0490 0x7f4ea015a460 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800b970 tx=0x7f4e9800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4e977fe640 1 -- 172.21.15.16:0/821066844 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98019070 con 0x7f4ea00d0490 2024-04-25T10:05:37.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4e977fe640 1 -- 172.21.15.16:0/821066844 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9800cb60 con 0x7f4ea00d0490 2024-04-25T10:05:37.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4ea53f9640 1 -- 172.21.15.16:0/821066844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea00cd480 con 0x7f4ea00d0490 2024-04-25T10:05:37.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.282+0000 7f4e977fe640 1 -- 172.21.15.16:0/821066844 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98014440 con 0x7f4ea00d0490 2024-04-25T10:05:37.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.283+0000 7f4ea53f9640 1 -- 172.21.15.16:0/821066844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea0140000 con 0x7f4ea00d0490 2024-04-25T10:05:37.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.283+0000 7f4ea53f9640 1 -- 172.21.15.16:0/821066844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea00750b0 con 0x7f4ea00d0490 2024-04-25T10:05:37.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.284+0000 7f4e977fe640 1 -- 172.21.15.16:0/821066844 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e9801c070 con 0x7f4ea00d0490 2024-04-25T10:05:37.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.284+0000 7f4e977fe640 1 --2- 172.21.15.16:0/821066844 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4e8006b4d0 0x7f4e8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.284+0000 7f4e977fe640 1 -- 172.21.15.16:0/821066844 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e98089e90 con 0x7f4ea00d0490 2024-04-25T10:05:37.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.285+0000 7f4e9effd640 1 --2- 172.21.15.16:0/821066844 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4e8006b4d0 0x7f4e8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.285+0000 7f4e9effd640 1 --2- 172.21.15.16:0/821066844 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4e8006b4d0 0x7f4e8006d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4e90006fd0 tx=0x7f4e90006bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.289+0000 7f4e977fe640 1 -- 172.21.15.16:0/821066844 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ea0146330 con 0x7f4ea00d0490 2024-04-25T10:05:37.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.467+0000 7f4ea53f9640 1 -- 172.21.15.16:0/821066844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4ea00d18d0 con 0x7f4ea00d0490 2024-04-25T10:05:37.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.467+0000 7f4e977fe640 1 -- 172.21.15.16:0/821066844 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e98058da0 con 0x7f4ea00d0490 2024-04-25T10:05:37.473 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:05:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.469+0000 7f4ea53f9640 1 -- 172.21.15.16:0/821066844 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4e8006b4d0 msgr2=0x7f4e8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.469+0000 7f4ea53f9640 1 --2- 172.21.15.16:0/821066844 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4e8006b4d0 0x7f4e8006d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4e90006fd0 tx=0x7f4e90006bc0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.469+0000 7f4ea53f9640 1 -- 172.21.15.16:0/821066844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea00d0490 msgr2=0x7f4ea015a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.469+0000 7f4ea53f9640 1 --2- 172.21.15.16:0/821066844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea00d0490 0x7f4ea015a460 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800b970 tx=0x7f4e9800be30 comp rx=0 tx=0).stop 2024-04-25T10:05:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.469+0000 7f4ea53f9640 1 -- 172.21.15.16:0/821066844 shutdown_connections 2024-04-25T10:05:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.469+0000 7f4ea53f9640 1 --2- 172.21.15.16:0/821066844 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4e8006b4d0 0x7f4e8006d990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.469+0000 7f4ea53f9640 1 --2- 172.21.15.16:0/821066844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea00d0490 0x7f4ea015a460 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.469+0000 7f4ea53f9640 1 --2- 172.21.15.16:0/821066844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea00ce1a0 0x7f4ea0159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.469+0000 7f4ea53f9640 1 --2- 172.21.15.16:0/821066844 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ea00b2740 0x7f4ea015b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.469+0000 7f4ea53f9640 1 -- 172.21.15.16:0/821066844 >> 172.21.15.16:0/821066844 conn(0x7f4ea00b5660 msgr2=0x7f4ea00b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:37.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.469+0000 7f4ea53f9640 1 -- 172.21.15.16:0/821066844 shutdown_connections 2024-04-25T10:05:37.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.469+0000 7f4ea53f9640 1 -- 172.21.15.16:0/821066844 wait complete. 2024-04-25T10:05:37.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:37.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.584+0000 7f9a13b9e640 1 Processor -- start 2024-04-25T10:05:37.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.584+0000 7f9a13b9e640 1 -- start start 2024-04-25T10:05:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.585+0000 7f9a13b9e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a0c0b97c0 0x7f9a0c0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.585+0000 7f9a13b9e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9a0c0b5a30 0x7f9a0c0d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.585+0000 7f9a13b9e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0c0b5e40 con 0x7f9a0c15dc80 2024-04-25T10:05:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.585+0000 7f9a13b9e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c161f80 con 0x7f9a0c0b97c0 2024-04-25T10:05:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.585+0000 7f9a13b9e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c1627a0 con 0x7f9a0c0b5a30 2024-04-25T10:05:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.585+0000 7f9a11139640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a0c0b97c0 0x7f9a0c0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.585+0000 7f9a11139640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a0c0b97c0 0x7f9a0c0b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37472/0 (socket says 172.21.15.16:37472) 2024-04-25T10:05:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.585+0000 7f9a11139640 1 -- 172.21.15.16:0/3272894270 learned_addr learned my addr 172.21.15.16:0/3272894270 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.585+0000 7f9a1193a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9a0c15dc80 0x7f9a0c161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40914/0 (socket says 172.21.15.16:40914) 2024-04-25T10:05:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.585+0000 7f9a11139640 1 -- 172.21.15.16:0/3272894270 >> v2:172.21.15.186:3300/0 conn(0x7f9a0c0b5a30 msgr2=0x7f9a0c0d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.585+0000 7f9a11139640 1 --2- 172.21.15.16:0/3272894270 >> v2:172.21.15.186:3300/0 conn(0x7f9a0c0b5a30 0x7f9a0c0d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.585+0000 7f9a11139640 1 -- 172.21.15.16:0/3272894270 >> v1:172.21.15.16:6789/0 conn(0x7f9a0c15dc80 legacy=0x7f9a0c161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.585+0000 7f9a11139640 1 -- 172.21.15.16:0/3272894270 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c1621e0 con 0x7f9a0c0b97c0 2024-04-25T10:05:37.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.585+0000 7f9a10938640 1 -- 172.21.15.16:0/3272894270 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310686469 0 0) 0x7f9a04001110 con 0x7f9a0c15dc80 2024-04-25T10:05:37.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.586+0000 7f9a11139640 1 --2- 172.21.15.16:0/3272894270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c0b97c0 0x7f9a0c0b54f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0b6300 tx=0x7f9a0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb08a75dfe3c073 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.586+0000 7f9a10938640 1 -- 172.21.15.16:0/3272894270 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00037070 con 0x7f9a0c0b97c0 2024-04-25T10:05:37.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.586+0000 7f9a10938640 1 -- 172.21.15.16:0/3272894270 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a00032440 con 0x7f9a0c0b97c0 2024-04-25T10:05:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.586+0000 7f9a10938640 1 -- 172.21.15.16:0/3272894270 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00032810 con 0x7f9a0c0b97c0 2024-04-25T10:05:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.587+0000 7f9a13b9e640 1 -- 172.21.15.16:0/3272894270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c0b97c0 msgr2=0x7f9a0c0b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.587+0000 7f9a13b9e640 1 --2- 172.21.15.16:0/3272894270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c0b97c0 0x7f9a0c0b54f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0b6300 tx=0x7f9a0002a270 comp rx=0 tx=0).stop 2024-04-25T10:05:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.587+0000 7f9a13b9e640 1 -- 172.21.15.16:0/3272894270 shutdown_connections 2024-04-25T10:05:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.587+0000 7f9a13b9e640 1 --2- 172.21.15.16:0/3272894270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c0b97c0 0x7f9a0c0b54f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.587+0000 7f9a13b9e640 1 --2- 172.21.15.16:0/3272894270 >> v2:172.21.15.186:3300/0 conn(0x7f9a0c0b5a30 0x7f9a0c0d2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.587+0000 7f9a13b9e640 1 -- 172.21.15.16:0/3272894270 >> 172.21.15.16:0/3272894270 conn(0x7f9a0c0d04f0 msgr2=0x7f9a0c0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.587+0000 7f9a13b9e640 1 -- 172.21.15.16:0/3272894270 shutdown_connections 2024-04-25T10:05:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.587+0000 7f9a13b9e640 1 -- 172.21.15.16:0/3272894270 wait complete. 2024-04-25T10:05:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.588+0000 7f9a13b9e640 1 Processor -- start 2024-04-25T10:05:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.588+0000 7f9a13b9e640 1 -- start start 2024-04-25T10:05:37.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.589+0000 7f9a13b9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c0b5a30 0x7f9a0c14bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.589+0000 7f9a13b9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a0c0b97c0 0x7f9a0c15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.589+0000 7f9a13b9e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c15dc80 0x7f9a0c0cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.589+0000 7f9a13b9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c15e060 con 0x7f9a0c0b5a30 2024-04-25T10:05:37.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.589+0000 7f9a13b9e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0d1c70 con 0x7f9a0c15dc80 2024-04-25T10:05:37.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.589+0000 7f9a13b9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0d29b0 con 0x7f9a0c0b97c0 2024-04-25T10:05:37.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.589+0000 7f9a1193a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c0b5a30 0x7f9a0c14bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.589+0000 7f9a1193a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c0b5a30 0x7f9a0c14bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37488/0 (socket says 172.21.15.16:37488) 2024-04-25T10:05:37.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.589+0000 7f9a1193a640 1 -- 172.21.15.16:0/749011154 learned_addr learned my addr 172.21.15.16:0/749011154 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:37.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.589+0000 7f9a1193a640 1 -- 172.21.15.16:0/749011154 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a0c0b97c0 msgr2=0x7f9a0c15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.589+0000 7f9a1193a640 1 --2- 172.21.15.16:0/749011154 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a0c0b97c0 0x7f9a0c15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.589+0000 7f9a1213b640 1 --2- 172.21.15.16:0/749011154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c15dc80 0x7f9a0c0cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.589+0000 7f9a1193a640 1 -- 172.21.15.16:0/749011154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c15dc80 msgr2=0x7f9a0c0cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.589+0000 7f9a1193a640 1 --2- 172.21.15.16:0/749011154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c15dc80 0x7f9a0c0cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.589+0000 7f9a1193a640 1 -- 172.21.15.16:0/749011154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c15f440 con 0x7f9a0c0b5a30 2024-04-25T10:05:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.589+0000 7f9a1213b640 1 --2- 172.21.15.16:0/749011154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c15dc80 0x7f9a0c0cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.590+0000 7f9a1193a640 1 --2- 172.21.15.16:0/749011154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c0b5a30 0x7f9a0c14bf20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400c430 tx=0x7f9a0400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.590+0000 7f9a0a7fc640 1 -- 172.21.15.16:0/749011154 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a040074e0 con 0x7f9a0c0b5a30 2024-04-25T10:05:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.590+0000 7f9a0a7fc640 1 -- 172.21.15.16:0/749011154 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a040047e0 con 0x7f9a0c0b5a30 2024-04-25T10:05:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.590+0000 7f9a13b9e640 1 -- 172.21.15.16:0/749011154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c1e7730 con 0x7f9a0c0b5a30 2024-04-25T10:05:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.590+0000 7f9a0a7fc640 1 -- 172.21.15.16:0/749011154 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04004bd0 con 0x7f9a0c0b5a30 2024-04-25T10:05:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.590+0000 7f9a13b9e640 1 -- 172.21.15.16:0/749011154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c1e7cf0 con 0x7f9a0c0b5a30 2024-04-25T10:05:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.591+0000 7f9a13b9e640 1 -- 172.21.15.16:0/749011154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0c0750b0 con 0x7f9a0c0b5a30 2024-04-25T10:05:37.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.591+0000 7f9a0a7fc640 1 -- 172.21.15.16:0/749011154 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a04011070 con 0x7f9a0c0b5a30 2024-04-25T10:05:37.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.592+0000 7f9a0a7fc640 1 --2- 172.21.15.16:0/749011154 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f99f806b4d0 0x7f99f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.592+0000 7f9a0a7fc640 1 -- 172.21.15.16:0/749011154 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a04089e80 con 0x7f9a0c0b5a30 2024-04-25T10:05:37.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.592+0000 7f9a11139640 1 --2- 172.21.15.16:0/749011154 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f99f806b4d0 0x7f99f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.593+0000 7f9a11139640 1 --2- 172.21.15.16:0/749011154 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f99f806b4d0 0x7f99f806d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9a00034000 tx=0x7f9a0002c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.596+0000 7f9a0a7fc640 1 -- 172.21.15.16:0/749011154 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a0c0b5e40 con 0x7f9a0c0b5a30 2024-04-25T10:05:37.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.775+0000 7f9a13b9e640 1 -- 172.21.15.16:0/749011154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9a0c1ea1a0 con 0x7f9a0c0b5a30 2024-04-25T10:05:37.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.775+0000 7f9a0a7fc640 1 -- 172.21.15.16:0/749011154 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9a04058d90 con 0x7f9a0c0b5a30 2024-04-25T10:05:37.781 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:05:37.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.777+0000 7f9a13b9e640 1 -- 172.21.15.16:0/749011154 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f99f806b4d0 msgr2=0x7f99f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.777+0000 7f9a13b9e640 1 --2- 172.21.15.16:0/749011154 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f99f806b4d0 0x7f99f806d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9a00034000 tx=0x7f9a0002c000 comp rx=0 tx=0).stop 2024-04-25T10:05:37.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.777+0000 7f9a13b9e640 1 -- 172.21.15.16:0/749011154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c0b5a30 msgr2=0x7f9a0c14bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.777+0000 7f9a13b9e640 1 --2- 172.21.15.16:0/749011154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c0b5a30 0x7f9a0c14bf20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400c430 tx=0x7f9a0400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.777+0000 7f9a13b9e640 1 -- 172.21.15.16:0/749011154 shutdown_connections 2024-04-25T10:05:37.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.777+0000 7f9a13b9e640 1 --2- 172.21.15.16:0/749011154 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f99f806b4d0 0x7f99f806d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.777+0000 7f9a13b9e640 1 --2- 172.21.15.16:0/749011154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c15dc80 0x7f9a0c0cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.777+0000 7f9a13b9e640 1 --2- 172.21.15.16:0/749011154 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a0c0b97c0 0x7f9a0c15ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.777+0000 7f9a13b9e640 1 --2- 172.21.15.16:0/749011154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c0b5a30 0x7f9a0c14bf20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.777+0000 7f9a13b9e640 1 -- 172.21.15.16:0/749011154 >> 172.21.15.16:0/749011154 conn(0x7f9a0c0d04f0 msgr2=0x7f9a0c15c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:37.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.778+0000 7f9a13b9e640 1 -- 172.21.15.16:0/749011154 shutdown_connections 2024-04-25T10:05:37.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.778+0000 7f9a13b9e640 1 -- 172.21.15.16:0/749011154 wait complete. 2024-04-25T10:05:37.793 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.892+0000 7f9ea343a640 1 Processor -- start 2024-04-25T10:05:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.892+0000 7f9ea343a640 1 -- start start 2024-04-25T10:05:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.893+0000 7f9ea343a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e9c0d1130 0x7f9e9c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.893+0000 7f9ea343a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e9c0d4090 con 0x7f9e9c0d3b90 2024-04-25T10:05:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.893+0000 7f9ea343a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e9c0d4370 con 0x7f9e9c15dca0 2024-04-25T10:05:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.893+0000 7f9ea343a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e9c0b6b20 con 0x7f9e9c0d1130 2024-04-25T10:05:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.893+0000 7f9ea19d7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e9c0d1130 0x7f9e9c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.893+0000 7f9ea09d5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9e9c0d3b90 0x7f9e9c0d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40930/0 (socket says 172.21.15.16:40930) 2024-04-25T10:05:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.893+0000 7f9ea19d7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e9c0d1130 0x7f9e9c0b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37498/0 (socket says 172.21.15.16:37498) 2024-04-25T10:05:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.893+0000 7f9ea09d5640 1 -- 172.21.15.16:0/411924321 learned_addr learned my addr 172.21.15.16:0/411924321 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.893+0000 7f9ea19d7640 1 -- 172.21.15.16:0/411924321 >> v1:172.21.15.186:6789/0 conn(0x7f9e9c15dca0 legacy=0x7f9e9c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.893+0000 7f9ea19d7640 1 -- 172.21.15.16:0/411924321 >> v1:172.21.15.16:6789/0 conn(0x7f9e9c0d3b90 legacy=0x7f9e9c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.893+0000 7f9ea19d7640 1 -- 172.21.15.16:0/411924321 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c0d4650 con 0x7f9e9c0d1130 2024-04-25T10:05:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.894+0000 7f9ea19d7640 1 --2- 172.21.15.16:0/411924321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e9c0d1130 0x7f9e9c0b2500 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009d90 tx=0x7f9e9402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42318c687d68970 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.894+0000 7f9e93fff640 1 -- 172.21.15.16:0/411924321 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94007740 con 0x7f9e9c0d1130 2024-04-25T10:05:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.894+0000 7f9e93fff640 1 -- 172.21.15.16:0/411924321 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e940078e0 con 0x7f9e9c0d1130 2024-04-25T10:05:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.894+0000 7f9e93fff640 1 -- 172.21.15.16:0/411924321 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94007cb0 con 0x7f9e9c0d1130 2024-04-25T10:05:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.895+0000 7f9ea343a640 1 -- 172.21.15.16:0/411924321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e9c0d1130 msgr2=0x7f9e9c0b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.895+0000 7f9ea343a640 1 --2- 172.21.15.16:0/411924321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e9c0d1130 0x7f9e9c0b2500 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009d90 tx=0x7f9e9402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.895+0000 7f9ea343a640 1 -- 172.21.15.16:0/411924321 shutdown_connections 2024-04-25T10:05:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.895+0000 7f9ea343a640 1 --2- 172.21.15.16:0/411924321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e9c0d1130 0x7f9e9c0b2500 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.895+0000 7f9ea343a640 1 -- 172.21.15.16:0/411924321 >> 172.21.15.16:0/411924321 conn(0x7f9e9c0726c0 msgr2=0x7f9e9c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.895+0000 7f9ea343a640 1 -- 172.21.15.16:0/411924321 shutdown_connections 2024-04-25T10:05:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.895+0000 7f9ea343a640 1 -- 172.21.15.16:0/411924321 wait complete. 2024-04-25T10:05:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.896+0000 7f9ea343a640 1 Processor -- start 2024-04-25T10:05:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.896+0000 7f9ea343a640 1 -- start start 2024-04-25T10:05:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.897+0000 7f9ea343a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e9c0d1130 0x7f9e9c15a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.897+0000 7f9ea343a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e9c0d3b90 0x7f9e9c0c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.897+0000 7f9ea343a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e9c15dca0 0x7f9e9c0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.897+0000 7f9ea343a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c15e080 con 0x7f9e9c15dca0 2024-04-25T10:05:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.897+0000 7f9ea343a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c0cf050 con 0x7f9e9c0d3b90 2024-04-25T10:05:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.897+0000 7f9ea343a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c15e200 con 0x7f9e9c0d1130 2024-04-25T10:05:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.897+0000 7f9ea19d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e9c15dca0 0x7f9e9c0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.897+0000 7f9ea19d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e9c15dca0 0x7f9e9c0cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37500/0 (socket says 172.21.15.16:37500) 2024-04-25T10:05:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.897+0000 7f9ea19d7640 1 -- 172.21.15.16:0/3197017736 learned_addr learned my addr 172.21.15.16:0/3197017736 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.897+0000 7f9ea11d6640 1 --2- 172.21.15.16:0/3197017736 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e9c0d1130 0x7f9e9c15a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.897+0000 7f9ea09d5640 1 --2- 172.21.15.16:0/3197017736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e9c0d3b90 0x7f9e9c0c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.897+0000 7f9ea19d7640 1 -- 172.21.15.16:0/3197017736 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e9c0d1130 msgr2=0x7f9e9c15a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.897+0000 7f9ea19d7640 1 --2- 172.21.15.16:0/3197017736 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e9c0d1130 0x7f9e9c15a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.897+0000 7f9ea19d7640 1 -- 172.21.15.16:0/3197017736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e9c0d3b90 msgr2=0x7f9e9c0c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.897+0000 7f9ea19d7640 1 --2- 172.21.15.16:0/3197017736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e9c0d3b90 0x7f9e9c0c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.897+0000 7f9ea19d7640 1 -- 172.21.15.16:0/3197017736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c0c8a20 con 0x7f9e9c15dca0 2024-04-25T10:05:37.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.898+0000 7f9ea19d7640 1 --2- 172.21.15.16:0/3197017736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e9c15dca0 0x7f9e9c0cc820 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e9402ace0 tx=0x7f9e94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.898+0000 7f9e91ffb640 1 -- 172.21.15.16:0/3197017736 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94040070 con 0x7f9e9c15dca0 2024-04-25T10:05:37.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.898+0000 7f9e91ffb640 1 -- 172.21.15.16:0/3197017736 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e94031c90 con 0x7f9e9c15dca0 2024-04-25T10:05:37.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.898+0000 7f9ea343a640 1 -- 172.21.15.16:0/3197017736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e9c0c8d00 con 0x7f9e9c15dca0 2024-04-25T10:05:37.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.898+0000 7f9e91ffb640 1 -- 172.21.15.16:0/3197017736 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94045680 con 0x7f9e9c15dca0 2024-04-25T10:05:37.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.898+0000 7f9ea343a640 1 -- 172.21.15.16:0/3197017736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e9c0c92c0 con 0x7f9e9c15dca0 2024-04-25T10:05:37.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.899+0000 7f9ea343a640 1 -- 172.21.15.16:0/3197017736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e9c0750b0 con 0x7f9e9c15dca0 2024-04-25T10:05:37.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.899+0000 7f9e91ffb640 1 -- 172.21.15.16:0/3197017736 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e94039040 con 0x7f9e9c15dca0 2024-04-25T10:05:37.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.900+0000 7f9e91ffb640 1 --2- 172.21.15.16:0/3197017736 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9e8006b4d0 0x7f9e8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.900+0000 7f9e91ffb640 1 -- 172.21.15.16:0/3197017736 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e940abbc0 con 0x7f9e9c15dca0 2024-04-25T10:05:37.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.900+0000 7f9ea11d6640 1 --2- 172.21.15.16:0/3197017736 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9e8006b4d0 0x7f9e8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.901+0000 7f9ea11d6640 1 --2- 172.21.15.16:0/3197017736 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9e8006b4d0 0x7f9e8006d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9e980016e0 tx=0x7f9e98009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:37.904+0000 7f9e91ffb640 1 -- 172.21.15.16:0/3197017736 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9e9407a380 con 0x7f9e9c15dca0 2024-04-25T10:05:38.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.083+0000 7f9ea343a640 1 -- 172.21.15.16:0/3197017736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9e9c06b1d0 con 0x7f9e9c15dca0 2024-04-25T10:05:38.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.083+0000 7f9e91ffb640 1 -- 172.21.15.16:0/3197017736 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9e9407e420 con 0x7f9e9c15dca0 2024-04-25T10:05:38.088 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:05:38.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.085+0000 7f9ea343a640 1 -- 172.21.15.16:0/3197017736 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9e8006b4d0 msgr2=0x7f9e8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.085+0000 7f9ea343a640 1 --2- 172.21.15.16:0/3197017736 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9e8006b4d0 0x7f9e8006d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9e980016e0 tx=0x7f9e98009040 comp rx=0 tx=0).stop 2024-04-25T10:05:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.085+0000 7f9ea343a640 1 -- 172.21.15.16:0/3197017736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e9c15dca0 msgr2=0x7f9e9c0cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.085+0000 7f9ea343a640 1 --2- 172.21.15.16:0/3197017736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e9c15dca0 0x7f9e9c0cc820 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e9402ace0 tx=0x7f9e94004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.085+0000 7f9ea343a640 1 -- 172.21.15.16:0/3197017736 shutdown_connections 2024-04-25T10:05:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.085+0000 7f9ea343a640 1 --2- 172.21.15.16:0/3197017736 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9e8006b4d0 0x7f9e8006d990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.085+0000 7f9ea343a640 1 --2- 172.21.15.16:0/3197017736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e9c15dca0 0x7f9e9c0cc820 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.085+0000 7f9ea343a640 1 --2- 172.21.15.16:0/3197017736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e9c0d3b90 0x7f9e9c0c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.085+0000 7f9ea343a640 1 --2- 172.21.15.16:0/3197017736 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e9c0d1130 0x7f9e9c15a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.085+0000 7f9ea343a640 1 -- 172.21.15.16:0/3197017736 >> 172.21.15.16:0/3197017736 conn(0x7f9e9c0726c0 msgr2=0x7f9e9c0d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.085+0000 7f9ea343a640 1 -- 172.21.15.16:0/3197017736 shutdown_connections 2024-04-25T10:05:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.085+0000 7f9ea343a640 1 -- 172.21.15.16:0/3197017736 wait complete. 2024-04-25T10:05:38.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:38.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.196+0000 7f1c716bd640 1 Processor -- start 2024-04-25T10:05:38.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.196+0000 7f1c716bd640 1 -- start start 2024-04-25T10:05:38.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.197+0000 7f1c716bd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c6c162360 0x7f1c6c0cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.197+0000 7f1c716bd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6c161970 con 0x7f1c6c0cdf60 2024-04-25T10:05:38.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.197+0000 7f1c716bd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6c06b780 con 0x7f1c6c06b2b0 2024-04-25T10:05:38.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.197+0000 7f1c716bd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c6c06ba60 con 0x7f1c6c162360 2024-04-25T10:05:38.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.197+0000 7f1c6affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c6c162360 0x7f1c6c0cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.197+0000 7f1c6a7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1c6c0cdf60 0x7f1c6c06aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40942/0 (socket says 172.21.15.16:40942) 2024-04-25T10:05:38.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.197+0000 7f1c6affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c6c162360 0x7f1c6c0cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37502/0 (socket says 172.21.15.16:37502) 2024-04-25T10:05:38.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.197+0000 7f1c6a7fc640 1 -- 172.21.15.16:0/672176430 learned_addr learned my addr 172.21.15.16:0/672176430 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:38.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.198+0000 7f1c6affd640 1 -- 172.21.15.16:0/672176430 >> v1:172.21.15.186:6789/0 conn(0x7f1c6c06b2b0 legacy=0x7f1c6c06dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:38.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.198+0000 7f1c6affd640 1 -- 172.21.15.16:0/672176430 >> v1:172.21.15.16:6789/0 conn(0x7f1c6c0cdf60 legacy=0x7f1c6c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.198+0000 7f1c69ffb640 1 -- 172.21.15.16:0/672176430 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557088554 0 0) 0x7f1c54001110 con 0x7f1c6c0cdf60 2024-04-25T10:05:38.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.198+0000 7f1c6affd640 1 -- 172.21.15.16:0/672176430 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c06bd40 con 0x7f1c6c162360 2024-04-25T10:05:38.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.198+0000 7f1c6affd640 1 --2- 172.21.15.16:0/672176430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c162360 0x7f1c6c0cda20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009d90 tx=0x7f1c5c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84bdfd7997dcb9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.198+0000 7f1c69ffb640 1 -- 172.21.15.16:0/672176430 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c007740 con 0x7f1c6c162360 2024-04-25T10:05:38.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.199+0000 7f1c69ffb640 1 -- 172.21.15.16:0/672176430 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5c0078e0 con 0x7f1c6c162360 2024-04-25T10:05:38.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.199+0000 7f1c69ffb640 1 -- 172.21.15.16:0/672176430 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c007cb0 con 0x7f1c6c162360 2024-04-25T10:05:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.199+0000 7f1c716bd640 1 -- 172.21.15.16:0/672176430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c162360 msgr2=0x7f1c6c0cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.199+0000 7f1c716bd640 1 --2- 172.21.15.16:0/672176430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c162360 0x7f1c6c0cda20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009d90 tx=0x7f1c5c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.199+0000 7f1c716bd640 1 -- 172.21.15.16:0/672176430 shutdown_connections 2024-04-25T10:05:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.199+0000 7f1c716bd640 1 --2- 172.21.15.16:0/672176430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c162360 0x7f1c6c0cda20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.199+0000 7f1c716bd640 1 -- 172.21.15.16:0/672176430 >> 172.21.15.16:0/672176430 conn(0x7f1c6c0cc8d0 msgr2=0x7f1c6c0ccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.199+0000 7f1c716bd640 1 -- 172.21.15.16:0/672176430 shutdown_connections 2024-04-25T10:05:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.199+0000 7f1c716bd640 1 -- 172.21.15.16:0/672176430 wait complete. 2024-04-25T10:05:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.200+0000 7f1c716bd640 1 Processor -- start 2024-04-25T10:05:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.200+0000 7f1c716bd640 1 -- start start 2024-04-25T10:05:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.201+0000 7f1c716bd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c6c06b2b0 0x7f1c6c0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.201+0000 7f1c716bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0cdf60 0x7f1c6c1e7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.201+0000 7f1c716bd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c6c1577a0 0x7f1c6c0ae010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.201+0000 7f1c716bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c06e1c0 con 0x7f1c6c0cdf60 2024-04-25T10:05:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.201+0000 7f1c716bd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c06c300 con 0x7f1c6c1577a0 2024-04-25T10:05:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.201+0000 7f1c716bd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c06e340 con 0x7f1c6c06b2b0 2024-04-25T10:05:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.201+0000 7f1c6a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0cdf60 0x7f1c6c1e7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.201+0000 7f1c6a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0cdf60 0x7f1c6c1e7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37516/0 (socket says 172.21.15.16:37516) 2024-04-25T10:05:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.201+0000 7f1c6a7fc640 1 -- 172.21.15.16:0/2494536984 learned_addr learned my addr 172.21.15.16:0/2494536984 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.201+0000 7f1c6affd640 1 --2- 172.21.15.16:0/2494536984 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c6c06b2b0 0x7f1c6c0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.202+0000 7f1c6a7fc640 1 -- 172.21.15.16:0/2494536984 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c6c06b2b0 msgr2=0x7f1c6c0c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.202+0000 7f1c6a7fc640 1 --2- 172.21.15.16:0/2494536984 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c6c06b2b0 0x7f1c6c0c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.202+0000 7f1c6b7fe640 1 --2- 172.21.15.16:0/2494536984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c6c1577a0 0x7f1c6c0ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.202+0000 7f1c6a7fc640 1 -- 172.21.15.16:0/2494536984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c6c1577a0 msgr2=0x7f1c6c0ae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.202+0000 7f1c6a7fc640 1 --2- 172.21.15.16:0/2494536984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c6c1577a0 0x7f1c6c0ae010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.202+0000 7f1c6a7fc640 1 -- 172.21.15.16:0/2494536984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c1e85e0 con 0x7f1c6c0cdf60 2024-04-25T10:05:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.202+0000 7f1c6affd640 1 --2- 172.21.15.16:0/2494536984 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c6c06b2b0 0x7f1c6c0c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.202+0000 7f1c6b7fe640 1 --2- 172.21.15.16:0/2494536984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c6c1577a0 0x7f1c6c0ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.202+0000 7f1c6a7fc640 1 --2- 172.21.15.16:0/2494536984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0cdf60 0x7f1c6c1e7e70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400c7b0 tx=0x7f1c5400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.202+0000 7f1c637fe640 1 -- 172.21.15.16:0/2494536984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c54018070 con 0x7f1c6c0cdf60 2024-04-25T10:05:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.202+0000 7f1c637fe640 1 -- 172.21.15.16:0/2494536984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c540072d0 con 0x7f1c6c0cdf60 2024-04-25T10:05:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.202+0000 7f1c716bd640 1 -- 172.21.15.16:0/2494536984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6c1e88c0 con 0x7f1c6c0cdf60 2024-04-25T10:05:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.202+0000 7f1c637fe640 1 -- 172.21.15.16:0/2494536984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c540047e0 con 0x7f1c6c0cdf60 2024-04-25T10:05:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.202+0000 7f1c716bd640 1 -- 172.21.15.16:0/2494536984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6c1e8e80 con 0x7f1c6c0cdf60 2024-04-25T10:05:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.203+0000 7f1c716bd640 1 -- 172.21.15.16:0/2494536984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6c0749b0 con 0x7f1c6c0cdf60 2024-04-25T10:05:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.203+0000 7f1c637fe640 1 -- 172.21.15.16:0/2494536984 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c5401b070 con 0x7f1c6c0cdf60 2024-04-25T10:05:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.204+0000 7f1c637fe640 1 --2- 172.21.15.16:0/2494536984 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1c4c06b4d0 0x7f1c4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.204+0000 7f1c637fe640 1 -- 172.21.15.16:0/2494536984 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c54089540 con 0x7f1c6c0cdf60 2024-04-25T10:05:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.204+0000 7f1c6affd640 1 --2- 172.21.15.16:0/2494536984 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1c4c06b4d0 0x7f1c4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.205+0000 7f1c6affd640 1 --2- 172.21.15.16:0/2494536984 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1c4c06b4d0 0x7f1c4c06d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c034000 tx=0x7f1c5c02f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.208+0000 7f1c637fe640 1 -- 172.21.15.16:0/2494536984 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c6c161970 con 0x7f1c6c0cdf60 2024-04-25T10:05:38.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.387+0000 7f1c716bd640 1 -- 172.21.15.16:0/2494536984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c6c066a30 con 0x7f1c6c0cdf60 2024-04-25T10:05:38.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.388+0000 7f1c637fe640 1 -- 172.21.15.16:0/2494536984 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f1c540583c0 con 0x7f1c6c0cdf60 2024-04-25T10:05:38.394 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:38.394 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:12.449183+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.390+0000 7f1c716bd640 1 -- 172.21.15.16:0/2494536984 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1c4c06b4d0 msgr2=0x7f1c4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.390+0000 7f1c716bd640 1 --2- 172.21.15.16:0/2494536984 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1c4c06b4d0 0x7f1c4c06d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c034000 tx=0x7f1c5c02f000 comp rx=0 tx=0).stop 2024-04-25T10:05:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.390+0000 7f1c716bd640 1 -- 172.21.15.16:0/2494536984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0cdf60 msgr2=0x7f1c6c1e7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.390+0000 7f1c716bd640 1 --2- 172.21.15.16:0/2494536984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0cdf60 0x7f1c6c1e7e70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400c7b0 tx=0x7f1c5400cc70 comp rx=0 tx=0).stop 2024-04-25T10:05:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.390+0000 7f1c716bd640 1 -- 172.21.15.16:0/2494536984 shutdown_connections 2024-04-25T10:05:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.390+0000 7f1c716bd640 1 --2- 172.21.15.16:0/2494536984 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1c4c06b4d0 0x7f1c4c06d990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.390+0000 7f1c716bd640 1 --2- 172.21.15.16:0/2494536984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c6c1577a0 0x7f1c6c0ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.390+0000 7f1c716bd640 1 --2- 172.21.15.16:0/2494536984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0cdf60 0x7f1c6c1e7e70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.390+0000 7f1c716bd640 1 --2- 172.21.15.16:0/2494536984 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c6c06b2b0 0x7f1c6c0c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.390+0000 7f1c716bd640 1 -- 172.21.15.16:0/2494536984 >> 172.21.15.16:0/2494536984 conn(0x7f1c6c0cc8d0 msgr2=0x7f1c6c161be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.390+0000 7f1c716bd640 1 -- 172.21.15.16:0/2494536984 shutdown_connections 2024-04-25T10:05:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.390+0000 7f1c716bd640 1 -- 172.21.15.16:0/2494536984 wait complete. 2024-04-25T10:05:38.407 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:08.724537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:09.793105+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:38.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:38.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.506+0000 7f35b561f640 1 Processor -- start 2024-04-25T10:05:38.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.506+0000 7f35b561f640 1 -- start start 2024-04-25T10:05:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.507+0000 7f35b561f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f35b015dca0 0x7f35b00ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.507+0000 7f35b561f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35b00d3b90 0x7f35b00d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.507+0000 7f35b561f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b00d4090 con 0x7f35b00cee80 2024-04-25T10:05:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.507+0000 7f35b561f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b00b6b20 con 0x7f35b015dca0 2024-04-25T10:05:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.507+0000 7f35b561f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b00d4370 con 0x7f35b00d3b90 2024-04-25T10:05:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.507+0000 7f35af7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f35b00cee80 0x7f35b00b2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40956/0 (socket says 172.21.15.16:40956) 2024-04-25T10:05:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.507+0000 7f35af7fe640 1 -- 172.21.15.16:0/3308192378 learned_addr learned my addr 172.21.15.16:0/3308192378 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.507+0000 7f35a7fff640 1 --2- 172.21.15.16:0/3308192378 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35b00d3b90 0x7f35b00d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.507+0000 7f35aeffd640 1 --2- 172.21.15.16:0/3308192378 >> v2:172.21.15.186:3300/0 conn(0x7f35b015dca0 0x7f35b00ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.507+0000 7f35ae7fc640 1 -- 172.21.15.16:0/3308192378 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1022902005 0 0) 0x7f35b00d4090 con 0x7f35b00cee80 2024-04-25T10:05:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.507+0000 7f35ae7fc640 1 -- 172.21.15.16:0/3308192378 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f359c003520 con 0x7f35b00cee80 2024-04-25T10:05:38.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.507+0000 7f35a7fff640 1 -- 172.21.15.16:0/3308192378 >> v2:172.21.15.186:3300/0 conn(0x7f35b015dca0 msgr2=0x7f35b00ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:38.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.507+0000 7f35a7fff640 1 --2- 172.21.15.16:0/3308192378 >> v2:172.21.15.186:3300/0 conn(0x7f35b015dca0 0x7f35b00ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.507+0000 7f35a7fff640 1 -- 172.21.15.16:0/3308192378 >> v1:172.21.15.16:6789/0 conn(0x7f35b00cee80 legacy=0x7f35b00b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.507+0000 7f35a7fff640 1 -- 172.21.15.16:0/3308192378 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b00d4650 con 0x7f35b00d3b90 2024-04-25T10:05:38.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.508+0000 7f35a7fff640 1 --2- 172.21.15.16:0/3308192378 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35b00d3b90 0x7f35b00d0a20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f35980097a0 tx=0x7f359802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f6b82ad38f4078a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.508+0000 7f35ae7fc640 1 -- 172.21.15.16:0/3308192378 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598037070 con 0x7f35b00d3b90 2024-04-25T10:05:38.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.508+0000 7f35ae7fc640 1 -- 172.21.15.16:0/3308192378 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3598032440 con 0x7f35b00d3b90 2024-04-25T10:05:38.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.508+0000 7f35ae7fc640 1 -- 172.21.15.16:0/3308192378 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35980327f0 con 0x7f35b00d3b90 2024-04-25T10:05:38.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.509+0000 7f35b561f640 1 -- 172.21.15.16:0/3308192378 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35b00d3b90 msgr2=0x7f35b00d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.509+0000 7f35b561f640 1 --2- 172.21.15.16:0/3308192378 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35b00d3b90 0x7f35b00d0a20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f35980097a0 tx=0x7f359802a180 comp rx=0 tx=0).stop 2024-04-25T10:05:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.509+0000 7f35b561f640 1 -- 172.21.15.16:0/3308192378 shutdown_connections 2024-04-25T10:05:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.509+0000 7f35b561f640 1 --2- 172.21.15.16:0/3308192378 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35b00d3b90 0x7f35b00d0a20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.509+0000 7f35b561f640 1 --2- 172.21.15.16:0/3308192378 >> v2:172.21.15.186:3300/0 conn(0x7f35b015dca0 0x7f35b00ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.509+0000 7f35b561f640 1 -- 172.21.15.16:0/3308192378 >> 172.21.15.16:0/3308192378 conn(0x7f35b00726c0 msgr2=0x7f35b0072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.509+0000 7f35b561f640 1 -- 172.21.15.16:0/3308192378 shutdown_connections 2024-04-25T10:05:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.509+0000 7f35b561f640 1 -- 172.21.15.16:0/3308192378 wait complete. 2024-04-25T10:05:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.510+0000 7f35b561f640 1 Processor -- start 2024-04-25T10:05:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.510+0000 7f35b561f640 1 -- start start 2024-04-25T10:05:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.511+0000 7f35b561f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35b00cee80 0x7f35b01f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.511+0000 7f35b561f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35b00d3b90 0x7f35b015fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.511+0000 7f35b561f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35b015dca0 0x7f35b0152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.511+0000 7f35b561f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b015e200 con 0x7f35b015dca0 2024-04-25T10:05:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.511+0000 7f35b561f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b015e080 con 0x7f35b00cee80 2024-04-25T10:05:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.511+0000 7f35b561f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b00d0f60 con 0x7f35b00d3b90 2024-04-25T10:05:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.511+0000 7f35af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35b015dca0 0x7f35b0152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.511+0000 7f35af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35b015dca0 0x7f35b0152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37518/0 (socket says 172.21.15.16:37518) 2024-04-25T10:05:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.511+0000 7f35af7fe640 1 -- 172.21.15.16:0/2947648724 learned_addr learned my addr 172.21.15.16:0/2947648724 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.511+0000 7f35a7fff640 1 --2- 172.21.15.16:0/2947648724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35b00d3b90 0x7f35b015fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.511+0000 7f35aeffd640 1 --2- 172.21.15.16:0/2947648724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35b00cee80 0x7f35b01f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.511+0000 7f35af7fe640 1 -- 172.21.15.16:0/2947648724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35b00d3b90 msgr2=0x7f35b015fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.511+0000 7f35af7fe640 1 --2- 172.21.15.16:0/2947648724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35b00d3b90 0x7f35b015fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.511+0000 7f35af7fe640 1 -- 172.21.15.16:0/2947648724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35b00cee80 msgr2=0x7f35b01f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.511+0000 7f35af7fe640 1 --2- 172.21.15.16:0/2947648724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35b00cee80 0x7f35b01f36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.511+0000 7f35af7fe640 1 -- 172.21.15.16:0/2947648724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b01f3cc0 con 0x7f35b015dca0 2024-04-25T10:05:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.511+0000 7f35a7fff640 1 --2- 172.21.15.16:0/2947648724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35b00d3b90 0x7f35b015fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.511+0000 7f35aeffd640 1 --2- 172.21.15.16:0/2947648724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35b00cee80 0x7f35b01f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.512+0000 7f35af7fe640 1 --2- 172.21.15.16:0/2947648724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35b015dca0 0x7f35b0152f90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f35a800cb70 tx=0x7f35a800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.512+0000 7f35a77fe640 1 -- 172.21.15.16:0/2947648724 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8017070 con 0x7f35b015dca0 2024-04-25T10:05:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.512+0000 7f35a77fe640 1 -- 172.21.15.16:0/2947648724 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a800ad40 con 0x7f35b015dca0 2024-04-25T10:05:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.512+0000 7f35b561f640 1 -- 172.21.15.16:0/2947648724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b01f3fa0 con 0x7f35b015dca0 2024-04-25T10:05:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.512+0000 7f35a77fe640 1 -- 172.21.15.16:0/2947648724 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8012620 con 0x7f35b015dca0 2024-04-25T10:05:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.512+0000 7f35b561f640 1 -- 172.21.15.16:0/2947648724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b01f4560 con 0x7f35b015dca0 2024-04-25T10:05:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.513+0000 7f35b561f640 1 -- 172.21.15.16:0/2947648724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b00750b0 con 0x7f35b015dca0 2024-04-25T10:05:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.513+0000 7f35a77fe640 1 -- 172.21.15.16:0/2947648724 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f35a801a070 con 0x7f35b015dca0 2024-04-25T10:05:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.514+0000 7f35a77fe640 1 --2- 172.21.15.16:0/2947648724 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f359006b4d0 0x7f359006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.514+0000 7f35a77fe640 1 -- 172.21.15.16:0/2947648724 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35a808a020 con 0x7f35b015dca0 2024-04-25T10:05:38.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.514+0000 7f35aeffd640 1 --2- 172.21.15.16:0/2947648724 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f359006b4d0 0x7f359006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.515+0000 7f35aeffd640 1 --2- 172.21.15.16:0/2947648724 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f359006b4d0 0x7f359006d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f35a0009a20 tx=0x7f35a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.518+0000 7f35a77fe640 1 -- 172.21.15.16:0/2947648724 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f35a8058ea0 con 0x7f35b015dca0 2024-04-25T10:05:38.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.699+0000 7f35b561f640 1 -- 172.21.15.16:0/2947648724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35b00cdb90 con 0x7f35b015dca0 2024-04-25T10:05:38.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.700+0000 7f35a77fe640 1 -- 172.21.15.16:0/2947648724 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35a80587a0 con 0x7f35b015dca0 2024-04-25T10:05:38.705 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:05:38.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.702+0000 7f35b561f640 1 -- 172.21.15.16:0/2947648724 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f359006b4d0 msgr2=0x7f359006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.702+0000 7f35b561f640 1 --2- 172.21.15.16:0/2947648724 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f359006b4d0 0x7f359006d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f35a0009a20 tx=0x7f35a0008040 comp rx=0 tx=0).stop 2024-04-25T10:05:38.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.702+0000 7f35b561f640 1 -- 172.21.15.16:0/2947648724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35b015dca0 msgr2=0x7f35b0152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.702+0000 7f35b561f640 1 --2- 172.21.15.16:0/2947648724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35b015dca0 0x7f35b0152f90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f35a800cb70 tx=0x7f35a800a430 comp rx=0 tx=0).stop 2024-04-25T10:05:38.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.702+0000 7f35b561f640 1 -- 172.21.15.16:0/2947648724 shutdown_connections 2024-04-25T10:05:38.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.702+0000 7f35b561f640 1 --2- 172.21.15.16:0/2947648724 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f359006b4d0 0x7f359006d990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.702+0000 7f35b561f640 1 --2- 172.21.15.16:0/2947648724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35b015dca0 0x7f35b0152f90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.702+0000 7f35b561f640 1 --2- 172.21.15.16:0/2947648724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35b00d3b90 0x7f35b015fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.702+0000 7f35b561f640 1 --2- 172.21.15.16:0/2947648724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35b00cee80 0x7f35b01f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.702+0000 7f35b561f640 1 -- 172.21.15.16:0/2947648724 >> 172.21.15.16:0/2947648724 conn(0x7f35b00726c0 msgr2=0x7f35b00d1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:38.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.702+0000 7f35b561f640 1 -- 172.21.15.16:0/2947648724 shutdown_connections 2024-04-25T10:05:38.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.702+0000 7f35b561f640 1 -- 172.21.15.16:0/2947648724 wait complete. 2024-04-25T10:05:38.718 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.816+0000 7f8b5c76a640 1 Processor -- start 2024-04-25T10:05:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.816+0000 7f8b5c76a640 1 -- start start 2024-04-25T10:05:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.817+0000 7f8b5c76a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8b540d3b90 0x7f8b540d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.817+0000 7f8b5c76a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b540d4090 con 0x7f8b5415dca0 2024-04-25T10:05:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.817+0000 7f8b5c76a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b540d4370 con 0x7f8b540d0f60 2024-04-25T10:05:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.817+0000 7f8b5c76a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b540b6b20 con 0x7f8b540d3b90 2024-04-25T10:05:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.817+0000 7f8b5a506640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8b5415dca0 0x7f8b540ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40968/0 (socket says 172.21.15.16:40968) 2024-04-25T10:05:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.817+0000 7f8b5a506640 1 -- 172.21.15.16:0/883567173 learned_addr learned my addr 172.21.15.16:0/883567173 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.817+0000 7f8b59504640 1 -- 172.21.15.16:0/883567173 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966179013 0 0) 0x7f8b540d4090 con 0x7f8b5415dca0 2024-04-25T10:05:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.817+0000 7f8b59504640 1 -- 172.21.15.16:0/883567173 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b40003520 con 0x7f8b5415dca0 2024-04-25T10:05:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.817+0000 7f8b59d05640 1 --2- 172.21.15.16:0/883567173 >> v2:172.21.15.186:3300/0 conn(0x7f8b540d3b90 0x7f8b540d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b59504640 1 -- 172.21.15.16:0/883567173 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2159700392 0 0) 0x7f8b40003520 con 0x7f8b5415dca0 2024-04-25T10:05:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b59504640 1 -- 172.21.15.16:0/883567173 >> v2:172.21.15.186:3300/0 conn(0x7f8b540d3b90 msgr2=0x7f8b540d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b59504640 1 --2- 172.21.15.16:0/883567173 >> v2:172.21.15.186:3300/0 conn(0x7f8b540d3b90 0x7f8b540d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b59504640 1 -- 172.21.15.16:0/883567173 >> v1:172.21.15.186:6789/0 conn(0x7f8b540d0f60 legacy=0x7f8b540b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b59504640 1 -- 172.21.15.16:0/883567173 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b540d4650 con 0x7f8b5415dca0 2024-04-25T10:05:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b59504640 1 -- 172.21.15.16:0/883567173 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f8b4c001670 con 0x7f8b5415dca0 2024-04-25T10:05:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b59504640 1 -- 172.21.15.16:0/883567173 >> v1:172.21.15.16:6789/0 conn(0x7f8b5415dca0 legacy=0x7f8b540ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b59504640 1 --2- 172.21.15.16:0/883567173 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b400039b0 0x7f8b40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b59504640 1 --2- 172.21.15.16:0/883567173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b40004800 0x7f8b40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b59504640 1 --2- 172.21.15.16:0/883567173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b400051c0 0x7f8b4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b59504640 1 -- 172.21.15.16:0/883567173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4000c030 con 0x7f8b400051c0 2024-04-25T10:05:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b59504640 1 -- 172.21.15.16:0/883567173 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4000c1e0 con 0x7f8b40004800 2024-04-25T10:05:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b59504640 1 -- 172.21.15.16:0/883567173 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4000c3e0 con 0x7f8b400039b0 2024-04-25T10:05:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b5ad07640 1 --2- 172.21.15.16:0/883567173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b400051c0 0x7f8b4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b59d05640 1 --2- 172.21.15.16:0/883567173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b40004800 0x7f8b40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b5a506640 1 --2- 172.21.15.16:0/883567173 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b400039b0 0x7f8b40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b5ad07640 1 -- 172.21.15.16:0/883567173 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b400039b0 msgr2=0x7f8b40003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b5ad07640 1 --2- 172.21.15.16:0/883567173 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b400039b0 0x7f8b40003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b5ad07640 1 -- 172.21.15.16:0/883567173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b40004800 msgr2=0x7f8b40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b5ad07640 1 --2- 172.21.15.16:0/883567173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b40004800 0x7f8b40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.818+0000 7f8b5ad07640 1 -- 172.21.15.16:0/883567173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b40003520 con 0x7f8b400051c0 2024-04-25T10:05:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.819+0000 7f8b5ad07640 1 -- 172.21.15.16:0/883567173 reap_dead start 2024-04-25T10:05:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.819+0000 7f8b5ad07640 1 --2- 172.21.15.16:0/883567173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b400051c0 0x7f8b4000b9c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8b50001e50 tx=0x7f8b40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d573fa200c8818b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.819+0000 7f8b59504640 1 -- 172.21.15.16:0/883567173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b50001a70 con 0x7f8b400051c0 2024-04-25T10:05:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.819+0000 7f8b59504640 1 -- 172.21.15.16:0/883567173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b50003970 con 0x7f8b400051c0 2024-04-25T10:05:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.819+0000 7f8b59504640 1 -- 172.21.15.16:0/883567173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b50003d60 con 0x7f8b400051c0 2024-04-25T10:05:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.820+0000 7f8b5c76a640 1 -- 172.21.15.16:0/883567173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b400051c0 msgr2=0x7f8b4000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.820+0000 7f8b5c76a640 1 --2- 172.21.15.16:0/883567173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b400051c0 0x7f8b4000b9c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8b50001e50 tx=0x7f8b40005830 comp rx=0 tx=0).stop 2024-04-25T10:05:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.820+0000 7f8b5c76a640 1 -- 172.21.15.16:0/883567173 shutdown_connections 2024-04-25T10:05:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.820+0000 7f8b5c76a640 1 --2- 172.21.15.16:0/883567173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b400051c0 0x7f8b4000b9c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.820+0000 7f8b5c76a640 1 -- 172.21.15.16:0/883567173 >> 172.21.15.16:0/883567173 conn(0x7f8b540726c0 msgr2=0x7f8b54072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.820+0000 7f8b5c76a640 1 -- 172.21.15.16:0/883567173 shutdown_connections 2024-04-25T10:05:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.820+0000 7f8b5c76a640 1 -- 172.21.15.16:0/883567173 wait complete. 2024-04-25T10:05:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.821+0000 7f8b5c76a640 1 Processor -- start 2024-04-25T10:05:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.821+0000 7f8b5c76a640 1 -- start start 2024-04-25T10:05:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b5c76a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b400051c0 0x7f8b540cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b5c76a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b540cb570 0x7f8b541605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b5c76a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b540cb9f0 0x7f8b54160af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b5c76a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5415e080 con 0x7f8b400051c0 2024-04-25T10:05:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b5c76a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b540d1390 con 0x7f8b540cb9f0 2024-04-25T10:05:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b5c76a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5415e200 con 0x7f8b540cb570 2024-04-25T10:05:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b5a506640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b400051c0 0x7f8b540cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b5a506640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b400051c0 0x7f8b540cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37526/0 (socket says 172.21.15.16:37526) 2024-04-25T10:05:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b5a506640 1 -- 172.21.15.16:0/2315659483 learned_addr learned my addr 172.21.15.16:0/2315659483 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b59d05640 1 --2- 172.21.15.16:0/2315659483 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b540cb570 0x7f8b541605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b5ad07640 1 --2- 172.21.15.16:0/2315659483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b540cb9f0 0x7f8b54160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b5a506640 1 -- 172.21.15.16:0/2315659483 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b540cb570 msgr2=0x7f8b541605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b5a506640 1 --2- 172.21.15.16:0/2315659483 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b540cb570 0x7f8b541605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b5a506640 1 -- 172.21.15.16:0/2315659483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b540cb9f0 msgr2=0x7f8b54160af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b5a506640 1 --2- 172.21.15.16:0/2315659483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b540cb9f0 0x7f8b54160af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b5a506640 1 -- 172.21.15.16:0/2315659483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b540cc590 con 0x7f8b400051c0 2024-04-25T10:05:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b5a506640 1 --2- 172.21.15.16:0/2315659483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b400051c0 0x7f8b540cd630 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c004fa0 tx=0x7f8b4c0074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b4affd640 1 -- 172.21.15.16:0/2315659483 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c007740 con 0x7f8b400051c0 2024-04-25T10:05:38.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b4affd640 1 -- 172.21.15.16:0/2315659483 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4c018070 con 0x7f8b400051c0 2024-04-25T10:05:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b5c76a640 1 -- 172.21.15.16:0/2315659483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b540cc870 con 0x7f8b400051c0 2024-04-25T10:05:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b4affd640 1 -- 172.21.15.16:0/2315659483 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c00fbb0 con 0x7f8b400051c0 2024-04-25T10:05:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.822+0000 7f8b5c76a640 1 -- 172.21.15.16:0/2315659483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b540cce30 con 0x7f8b400051c0 2024-04-25T10:05:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.823+0000 7f8b5c76a640 1 -- 172.21.15.16:0/2315659483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b540750b0 con 0x7f8b400051c0 2024-04-25T10:05:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.824+0000 7f8b4affd640 1 -- 172.21.15.16:0/2315659483 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b4c00fd50 con 0x7f8b400051c0 2024-04-25T10:05:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.824+0000 7f8b4affd640 1 --2- 172.21.15.16:0/2315659483 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8b3806b670 0x7f8b3806db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.824+0000 7f8b4affd640 1 -- 172.21.15.16:0/2315659483 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b4c08a1b0 con 0x7f8b400051c0 2024-04-25T10:05:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.824+0000 7f8b59d05640 1 --2- 172.21.15.16:0/2315659483 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8b3806b670 0x7f8b3806db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.825+0000 7f8b59d05640 1 --2- 172.21.15.16:0/2315659483 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8b3806b670 0x7f8b3806db30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8b540cbf70 tx=0x7f8b44008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:38.829+0000 7f8b4affd640 1 -- 172.21.15.16:0/2315659483 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b4c0589e0 con 0x7f8b400051c0 2024-04-25T10:05:39.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.006+0000 7f8b5c76a640 1 -- 172.21.15.16:0/2315659483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8b540b7e30 con 0x7f8b400051c0 2024-04-25T10:05:39.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.007+0000 7f8b4affd640 1 -- 172.21.15.16:0/2315659483 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8b4c05ca80 con 0x7f8b400051c0 2024-04-25T10:05:39.012 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:05:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.008+0000 7f8b5c76a640 1 -- 172.21.15.16:0/2315659483 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8b3806b670 msgr2=0x7f8b3806db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.009+0000 7f8b5c76a640 1 --2- 172.21.15.16:0/2315659483 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8b3806b670 0x7f8b3806db30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8b540cbf70 tx=0x7f8b44008040 comp rx=0 tx=0).stop 2024-04-25T10:05:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.009+0000 7f8b5c76a640 1 -- 172.21.15.16:0/2315659483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b400051c0 msgr2=0x7f8b540cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.009+0000 7f8b5c76a640 1 --2- 172.21.15.16:0/2315659483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b400051c0 0x7f8b540cd630 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c004fa0 tx=0x7f8b4c0074f0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.009+0000 7f8b5c76a640 1 -- 172.21.15.16:0/2315659483 shutdown_connections 2024-04-25T10:05:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.009+0000 7f8b5c76a640 1 --2- 172.21.15.16:0/2315659483 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8b3806b670 0x7f8b3806db30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.009+0000 7f8b5c76a640 1 --2- 172.21.15.16:0/2315659483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b540cb9f0 0x7f8b54160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.009+0000 7f8b5c76a640 1 --2- 172.21.15.16:0/2315659483 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b540cb570 0x7f8b541605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.009+0000 7f8b5c76a640 1 --2- 172.21.15.16:0/2315659483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b400051c0 0x7f8b540cd630 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.009+0000 7f8b5c76a640 1 -- 172.21.15.16:0/2315659483 >> 172.21.15.16:0/2315659483 conn(0x7f8b540726c0 msgr2=0x7f8b540ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.009+0000 7f8b5c76a640 1 -- 172.21.15.16:0/2315659483 shutdown_connections 2024-04-25T10:05:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.009+0000 7f8b5c76a640 1 -- 172.21.15.16:0/2315659483 wait complete. 2024-04-25T10:05:39.024 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.123+0000 7fae7e53b640 1 Processor -- start 2024-04-25T10:05:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.123+0000 7fae7e53b640 1 -- start start 2024-04-25T10:05:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.124+0000 7fae7e53b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fae780d3b90 0x7fae780d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.124+0000 7fae7e53b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae780d0f60 0x7fae780b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.124+0000 7fae7e53b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae780d4090 con 0x7fae7815dca0 2024-04-25T10:05:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.124+0000 7fae7e53b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae780b6b20 con 0x7fae780d3b90 2024-04-25T10:05:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.124+0000 7fae7e53b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae780d1390 con 0x7fae780d0f60 2024-04-25T10:05:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.124+0000 7fae777fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fae780d3b90 0x7fae780d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.124+0000 7fae777fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fae780d3b90 0x7fae780d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37540/0 (socket says 172.21.15.16:37540) 2024-04-25T10:05:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.124+0000 7fae777fe640 1 -- 172.21.15.16:0/1418726724 learned_addr learned my addr 172.21.15.16:0/1418726724 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.124+0000 7fae7cad8640 1 --2- 172.21.15.16:0/1418726724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae780d0f60 0x7fae780b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.124+0000 7fae777fe640 1 -- 172.21.15.16:0/1418726724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae780d0f60 msgr2=0x7fae780b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.124+0000 7fae777fe640 1 --2- 172.21.15.16:0/1418726724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae780d0f60 0x7fae780b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.124+0000 7fae777fe640 1 -- 172.21.15.16:0/1418726724 >> v1:172.21.15.186:6789/0 conn(0x7fae7815dca0 legacy=0x7fae780ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.124+0000 7fae777fe640 1 -- 172.21.15.16:0/1418726724 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae780d4370 con 0x7fae780d3b90 2024-04-25T10:05:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.125+0000 7fae777fe640 1 --2- 172.21.15.16:0/1418726724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae780d3b90 0x7fae780d0a20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fae680098a0 tx=0x7fae6802a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d112680eda952848 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.125+0000 7fae76ffd640 1 -- 172.21.15.16:0/1418726724 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68037070 con 0x7fae780d3b90 2024-04-25T10:05:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.125+0000 7fae76ffd640 1 -- 172.21.15.16:0/1418726724 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae68032440 con 0x7fae780d3b90 2024-04-25T10:05:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.125+0000 7fae76ffd640 1 -- 172.21.15.16:0/1418726724 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68032810 con 0x7fae780d3b90 2024-04-25T10:05:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.125+0000 7fae7e53b640 1 -- 172.21.15.16:0/1418726724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae780d3b90 msgr2=0x7fae780d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.125+0000 7fae7e53b640 1 --2- 172.21.15.16:0/1418726724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae780d3b90 0x7fae780d0a20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fae680098a0 tx=0x7fae6802a290 comp rx=0 tx=0).stop 2024-04-25T10:05:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.126+0000 7fae7e53b640 1 -- 172.21.15.16:0/1418726724 shutdown_connections 2024-04-25T10:05:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.126+0000 7fae7e53b640 1 --2- 172.21.15.16:0/1418726724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae780d0f60 0x7fae780b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.126+0000 7fae7e53b640 1 --2- 172.21.15.16:0/1418726724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae780d3b90 0x7fae780d0a20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.126+0000 7fae7e53b640 1 -- 172.21.15.16:0/1418726724 >> 172.21.15.16:0/1418726724 conn(0x7fae780726c0 msgr2=0x7fae78072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.126+0000 7fae7e53b640 1 -- 172.21.15.16:0/1418726724 shutdown_connections 2024-04-25T10:05:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.126+0000 7fae7e53b640 1 -- 172.21.15.16:0/1418726724 wait complete. 2024-04-25T10:05:39.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.127+0000 7fae7e53b640 1 Processor -- start 2024-04-25T10:05:39.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.127+0000 7fae7e53b640 1 -- start start 2024-04-25T10:05:39.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae7e53b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae780d0f60 0x7fae7815a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae7e53b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae780d3b90 0x7fae781f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae7e53b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae7815dca0 0x7fae781f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae7e53b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae780cf050 con 0x7fae780d0f60 2024-04-25T10:05:39.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae7e53b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae780b6d50 con 0x7fae780d3b90 2024-04-25T10:05:39.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae7e53b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7815e080 con 0x7fae7815dca0 2024-04-25T10:05:39.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae77fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae780d0f60 0x7fae7815a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae77fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae780d0f60 0x7fae7815a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37552/0 (socket says 172.21.15.16:37552) 2024-04-25T10:05:39.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae77fff640 1 -- 172.21.15.16:0/130232318 learned_addr learned my addr 172.21.15.16:0/130232318 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:39.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae777fe640 1 --2- 172.21.15.16:0/130232318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae780d3b90 0x7fae781f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae7cad8640 1 --2- 172.21.15.16:0/130232318 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae7815dca0 0x7fae781f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae77fff640 1 -- 172.21.15.16:0/130232318 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae7815dca0 msgr2=0x7fae781f4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae77fff640 1 --2- 172.21.15.16:0/130232318 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae7815dca0 0x7fae781f4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae77fff640 1 -- 172.21.15.16:0/130232318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae780d3b90 msgr2=0x7fae781f4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae77fff640 1 --2- 172.21.15.16:0/130232318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae780d3b90 0x7fae781f4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae77fff640 1 -- 172.21.15.16:0/130232318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7815e800 con 0x7fae780d0f60 2024-04-25T10:05:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae777fe640 1 --2- 172.21.15.16:0/130232318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae780d3b90 0x7fae781f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae7cad8640 1 --2- 172.21.15.16:0/130232318 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae7815dca0 0x7fae781f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae77fff640 1 --2- 172.21.15.16:0/130232318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae780d0f60 0x7fae7815a120 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00c3c0 tx=0x7fae6c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae74ff9640 1 -- 172.21.15.16:0/130232318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c0052d0 con 0x7fae780d0f60 2024-04-25T10:05:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae74ff9640 1 -- 172.21.15.16:0/130232318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6c005490 con 0x7fae780d0f60 2024-04-25T10:05:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae7e53b640 1 -- 172.21.15.16:0/130232318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7815eae0 con 0x7fae780d0f60 2024-04-25T10:05:39.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae74ff9640 1 -- 172.21.15.16:0/130232318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c007260 con 0x7fae780d0f60 2024-04-25T10:05:39.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.128+0000 7fae7e53b640 1 -- 172.21.15.16:0/130232318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7815f0a0 con 0x7fae780d0f60 2024-04-25T10:05:39.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.130+0000 7fae7e53b640 1 -- 172.21.15.16:0/130232318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae780750b0 con 0x7fae780d0f60 2024-04-25T10:05:39.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.130+0000 7fae74ff9640 1 -- 172.21.15.16:0/130232318 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae6c007400 con 0x7fae780d0f60 2024-04-25T10:05:39.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.131+0000 7fae74ff9640 1 --2- 172.21.15.16:0/130232318 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae6406b500 0x7fae6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.131+0000 7fae74ff9640 1 -- 172.21.15.16:0/130232318 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae6c0896e0 con 0x7fae780d0f60 2024-04-25T10:05:39.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.131+0000 7fae777fe640 1 --2- 172.21.15.16:0/130232318 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae6406b500 0x7fae6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.131+0000 7fae777fe640 1 --2- 172.21.15.16:0/130232318 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae6406b500 0x7fae6406d9c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fae68034000 tx=0x7fae6802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.135+0000 7fae74ff9640 1 -- 172.21.15.16:0/130232318 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae6c0584d0 con 0x7fae780d0f60 2024-04-25T10:05:39.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.314+0000 7fae7e53b640 1 -- 172.21.15.16:0/130232318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae780cdb90 con 0x7fae780d0f60 2024-04-25T10:05:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.315+0000 7fae74ff9640 1 -- 172.21.15.16:0/130232318 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae6c057dd0 con 0x7fae780d0f60 2024-04-25T10:05:39.321 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:05:39.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.317+0000 7fae7e53b640 1 -- 172.21.15.16:0/130232318 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae6406b500 msgr2=0x7fae6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.317+0000 7fae7e53b640 1 --2- 172.21.15.16:0/130232318 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae6406b500 0x7fae6406d9c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fae68034000 tx=0x7fae6802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:39.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.317+0000 7fae7e53b640 1 -- 172.21.15.16:0/130232318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae780d0f60 msgr2=0x7fae7815a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.317+0000 7fae7e53b640 1 --2- 172.21.15.16:0/130232318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae780d0f60 0x7fae7815a120 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00c3c0 tx=0x7fae6c00c880 comp rx=0 tx=0).stop 2024-04-25T10:05:39.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.317+0000 7fae7e53b640 1 -- 172.21.15.16:0/130232318 shutdown_connections 2024-04-25T10:05:39.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.317+0000 7fae7e53b640 1 --2- 172.21.15.16:0/130232318 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae6406b500 0x7fae6406d9c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.317+0000 7fae7e53b640 1 --2- 172.21.15.16:0/130232318 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae7815dca0 0x7fae781f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.317+0000 7fae7e53b640 1 --2- 172.21.15.16:0/130232318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae780d3b90 0x7fae781f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.317+0000 7fae7e53b640 1 --2- 172.21.15.16:0/130232318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae780d0f60 0x7fae7815a120 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.317+0000 7fae7e53b640 1 -- 172.21.15.16:0/130232318 >> 172.21.15.16:0/130232318 conn(0x7fae780726c0 msgr2=0x7fae780b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.317+0000 7fae7e53b640 1 -- 172.21.15.16:0/130232318 shutdown_connections 2024-04-25T10:05:39.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.317+0000 7fae7e53b640 1 -- 172.21.15.16:0/130232318 wait complete. 2024-04-25T10:05:39.333 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.430+0000 7f4af4170640 1 Processor -- start 2024-04-25T10:05:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.430+0000 7f4af4170640 1 -- start start 2024-04-25T10:05:39.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.430+0000 7f4af4170640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4aec15dce0 0x7f4aec0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.431+0000 7f4af4170640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aec15e0c0 con 0x7f4aec0b5b60 2024-04-25T10:05:39.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.431+0000 7f4af4170640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aec0d2510 con 0x7f4aec0d35b0 2024-04-25T10:05:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.431+0000 7f4af4170640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aec0d27f0 con 0x7f4aec15dce0 2024-04-25T10:05:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.431+0000 7f4af1f0c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4aec15dce0 0x7f4aec0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.431+0000 7f4af270d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4aec0b5b60 0x7f4aec0b6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40980/0 (socket says 172.21.15.16:40980) 2024-04-25T10:05:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.431+0000 7f4af270d640 1 -- 172.21.15.16:0/1512764758 learned_addr learned my addr 172.21.15.16:0/1512764758 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.431+0000 7f4af1f0c640 1 -- 172.21.15.16:0/1512764758 >> v1:172.21.15.186:6789/0 conn(0x7f4aec0d35b0 legacy=0x7f4aec0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.431+0000 7f4af1f0c640 1 -- 172.21.15.16:0/1512764758 >> v1:172.21.15.16:6789/0 conn(0x7f4aec0b5b60 legacy=0x7f4aec0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.431+0000 7f4af1f0c640 1 -- 172.21.15.16:0/1512764758 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec0d2ad0 con 0x7f4aec15dce0 2024-04-25T10:05:39.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.431+0000 7f4af0f0a640 1 -- 172.21.15.16:0/1512764758 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 723651568 0 0) 0x7f4aec15e0c0 con 0x7f4aec0b5b60 2024-04-25T10:05:39.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.432+0000 7f4af1f0c640 1 --2- 172.21.15.16:0/1512764758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aec15dce0 0x7f4aec0d1a50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80097a0 tx=0x7f4ae802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633253db6d76fc4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.432+0000 7f4af0f0a640 1 -- 172.21.15.16:0/1512764758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8037070 con 0x7f4aec15dce0 2024-04-25T10:05:39.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.432+0000 7f4af0f0a640 1 -- 172.21.15.16:0/1512764758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae8032440 con 0x7f4aec15dce0 2024-04-25T10:05:39.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.432+0000 7f4af0f0a640 1 -- 172.21.15.16:0/1512764758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8032810 con 0x7f4aec15dce0 2024-04-25T10:05:39.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.432+0000 7f4af4170640 1 -- 172.21.15.16:0/1512764758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aec15dce0 msgr2=0x7f4aec0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.432+0000 7f4af4170640 1 --2- 172.21.15.16:0/1512764758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aec15dce0 0x7f4aec0d1a50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80097a0 tx=0x7f4ae802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.432+0000 7f4af4170640 1 -- 172.21.15.16:0/1512764758 shutdown_connections 2024-04-25T10:05:39.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.432+0000 7f4af4170640 1 --2- 172.21.15.16:0/1512764758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aec15dce0 0x7f4aec0d1a50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.432+0000 7f4af4170640 1 -- 172.21.15.16:0/1512764758 >> 172.21.15.16:0/1512764758 conn(0x7f4aec072900 msgr2=0x7f4aec0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.433+0000 7f4af4170640 1 -- 172.21.15.16:0/1512764758 shutdown_connections 2024-04-25T10:05:39.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.433+0000 7f4af4170640 1 -- 172.21.15.16:0/1512764758 wait complete. 2024-04-25T10:05:39.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.434+0000 7f4af4170640 1 Processor -- start 2024-04-25T10:05:39.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.434+0000 7f4af4170640 1 -- start start 2024-04-25T10:05:39.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.434+0000 7f4af4170640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aec0b5b60 0x7f4aec1535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.434+0000 7f4af4170640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4aec0d35b0 0x7f4aec1576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.434+0000 7f4af4170640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aec15dce0 0x7f4aec159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4af4170640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec0b9820 con 0x7f4aec15dce0 2024-04-25T10:05:39.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4af4170640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec0d3090 con 0x7f4aec0b5b60 2024-04-25T10:05:39.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4af4170640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec0b99a0 con 0x7f4aec0d35b0 2024-04-25T10:05:39.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4af270d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aec15dce0 0x7f4aec159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4af270d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aec15dce0 0x7f4aec159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37568/0 (socket says 172.21.15.16:37568) 2024-04-25T10:05:39.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4af270d640 1 -- 172.21.15.16:0/1279588568 learned_addr learned my addr 172.21.15.16:0/1279588568 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:39.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4af1f0c640 1 --2- 172.21.15.16:0/1279588568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aec0b5b60 0x7f4aec1535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4af170b640 1 --2- 172.21.15.16:0/1279588568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4aec0d35b0 0x7f4aec1576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4af270d640 1 -- 172.21.15.16:0/1279588568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4aec0d35b0 msgr2=0x7f4aec1576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4af270d640 1 --2- 172.21.15.16:0/1279588568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4aec0d35b0 0x7f4aec1576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4af270d640 1 -- 172.21.15.16:0/1279588568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aec0b5b60 msgr2=0x7f4aec1535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4af270d640 1 --2- 172.21.15.16:0/1279588568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aec0b5b60 0x7f4aec1535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4af270d640 1 -- 172.21.15.16:0/1279588568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec1f4a30 con 0x7f4aec15dce0 2024-04-25T10:05:39.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4af1f0c640 1 --2- 172.21.15.16:0/1279588568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aec0b5b60 0x7f4aec1535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:39.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4af270d640 1 --2- 172.21.15.16:0/1279588568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aec15dce0 0x7f4aec159e40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400c7a0 tx=0x7f4ae400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4ae27fc640 1 -- 172.21.15.16:0/1279588568 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4018070 con 0x7f4aec15dce0 2024-04-25T10:05:39.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4ae27fc640 1 -- 172.21.15.16:0/1279588568 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae400a640 con 0x7f4aec15dce0 2024-04-25T10:05:39.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4ae27fc640 1 -- 172.21.15.16:0/1279588568 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae400aa70 con 0x7f4aec15dce0 2024-04-25T10:05:39.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4af4170640 1 -- 172.21.15.16:0/1279588568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aec1f3eb0 con 0x7f4aec15dce0 2024-04-25T10:05:39.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.435+0000 7f4af4170640 1 -- 172.21.15.16:0/1279588568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aec1f4750 con 0x7f4aec15dce0 2024-04-25T10:05:39.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.436+0000 7f4af4170640 1 -- 172.21.15.16:0/1279588568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aec0cf2e0 con 0x7f4aec15dce0 2024-04-25T10:05:39.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.437+0000 7f4ae27fc640 1 -- 172.21.15.16:0/1279588568 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ae401b070 con 0x7f4aec15dce0 2024-04-25T10:05:39.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.437+0000 7f4ae27fc640 1 --2- 172.21.15.16:0/1279588568 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4ad006b4d0 0x7f4ad006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.437+0000 7f4ae27fc640 1 -- 172.21.15.16:0/1279588568 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ae4089720 con 0x7f4aec15dce0 2024-04-25T10:05:39.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.438+0000 7f4af1f0c640 1 --2- 172.21.15.16:0/1279588568 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4ad006b4d0 0x7f4ad006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.438+0000 7f4af1f0c640 1 --2- 172.21.15.16:0/1279588568 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4ad006b4d0 0x7f4ad006d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8034000 tx=0x7f4ae802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.442+0000 7f4ae27fc640 1 -- 172.21.15.16:0/1279588568 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4aec0cf2e0 con 0x7f4aec15dce0 2024-04-25T10:05:39.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.622+0000 7f4af4170640 1 -- 172.21.15.16:0/1279588568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4aec0749b0 con 0x7f4aec15dce0 2024-04-25T10:05:39.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.623+0000 7f4ae27fc640 1 -- 172.21.15.16:0/1279588568 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f4ae4058630 con 0x7f4aec15dce0 2024-04-25T10:05:39.629 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:39.629 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:12.449183+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:39.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.625+0000 7f4af4170640 1 -- 172.21.15.16:0/1279588568 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4ad006b4d0 msgr2=0x7f4ad006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.625+0000 7f4af4170640 1 --2- 172.21.15.16:0/1279588568 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4ad006b4d0 0x7f4ad006d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8034000 tx=0x7f4ae802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:39.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.625+0000 7f4af4170640 1 -- 172.21.15.16:0/1279588568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aec15dce0 msgr2=0x7f4aec159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.625+0000 7f4af4170640 1 --2- 172.21.15.16:0/1279588568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aec15dce0 0x7f4aec159e40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400c7a0 tx=0x7f4ae400cc60 comp rx=0 tx=0).stop 2024-04-25T10:05:39.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.625+0000 7f4af4170640 1 -- 172.21.15.16:0/1279588568 shutdown_connections 2024-04-25T10:05:39.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.625+0000 7f4af4170640 1 --2- 172.21.15.16:0/1279588568 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4ad006b4d0 0x7f4ad006d990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.625+0000 7f4af4170640 1 --2- 172.21.15.16:0/1279588568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aec15dce0 0x7f4aec159e40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.625+0000 7f4af4170640 1 --2- 172.21.15.16:0/1279588568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4aec0d35b0 0x7f4aec1576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.625+0000 7f4af4170640 1 --2- 172.21.15.16:0/1279588568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aec0b5b60 0x7f4aec1535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.625+0000 7f4af4170640 1 -- 172.21.15.16:0/1279588568 >> 172.21.15.16:0/1279588568 conn(0x7f4aec072900 msgr2=0x7f4aec15cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.625+0000 7f4af4170640 1 -- 172.21.15.16:0/1279588568 shutdown_connections 2024-04-25T10:05:39.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.625+0000 7f4af4170640 1 -- 172.21.15.16:0/1279588568 wait complete. 2024-04-25T10:05:39.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:08.724537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:09.793105+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:39.641 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:39.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.740+0000 7fb5162ab640 1 Processor -- start 2024-04-25T10:05:39.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.740+0000 7fb5162ab640 1 -- start start 2024-04-25T10:05:39.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.741+0000 7fb5162ab640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb5100d3b90 0x7fb5100d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.741+0000 7fb5162ab640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5100d4090 con 0x7fb5100d0f60 2024-04-25T10:05:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.741+0000 7fb5162ab640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5100d4370 con 0x7fb51015dca0 2024-04-25T10:05:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.741+0000 7fb5162ab640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5100b6b20 con 0x7fb5100d3b90 2024-04-25T10:05:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.741+0000 7fb50f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb5100d3b90 0x7fb5100d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.741+0000 7fb514848640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb5100d0f60 0x7fb5100b2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40984/0 (socket says 172.21.15.16:40984) 2024-04-25T10:05:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.741+0000 7fb514848640 1 -- 172.21.15.16:0/1365373149 learned_addr learned my addr 172.21.15.16:0/1365373149 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.741+0000 7fb50f7fe640 1 -- 172.21.15.16:0/1365373149 >> v1:172.21.15.186:6789/0 conn(0x7fb51015dca0 legacy=0x7fb5100ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.741+0000 7fb50f7fe640 1 -- 172.21.15.16:0/1365373149 >> v1:172.21.15.16:6789/0 conn(0x7fb5100d0f60 legacy=0x7fb5100b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.741+0000 7fb50f7fe640 1 -- 172.21.15.16:0/1365373149 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5100d4650 con 0x7fb5100d3b90 2024-04-25T10:05:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.742+0000 7fb50f7fe640 1 --2- 172.21.15.16:0/1365373149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5100d3b90 0x7fb5100d0a20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb5040097a0 tx=0x7fb50402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555f3b4568dd10c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.742+0000 7fb50effd640 1 -- 172.21.15.16:0/1365373149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504037070 con 0x7fb5100d3b90 2024-04-25T10:05:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.742+0000 7fb50effd640 1 -- 172.21.15.16:0/1365373149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb504032440 con 0x7fb5100d3b90 2024-04-25T10:05:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.742+0000 7fb50effd640 1 -- 172.21.15.16:0/1365373149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504032810 con 0x7fb5100d3b90 2024-04-25T10:05:39.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.743+0000 7fb5162ab640 1 -- 172.21.15.16:0/1365373149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5100d3b90 msgr2=0x7fb5100d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.743+0000 7fb5162ab640 1 --2- 172.21.15.16:0/1365373149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5100d3b90 0x7fb5100d0a20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb5040097a0 tx=0x7fb50402a230 comp rx=0 tx=0).stop 2024-04-25T10:05:39.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.743+0000 7fb5162ab640 1 -- 172.21.15.16:0/1365373149 shutdown_connections 2024-04-25T10:05:39.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.743+0000 7fb5162ab640 1 --2- 172.21.15.16:0/1365373149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5100d3b90 0x7fb5100d0a20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.743+0000 7fb5162ab640 1 -- 172.21.15.16:0/1365373149 >> 172.21.15.16:0/1365373149 conn(0x7fb5100726c0 msgr2=0x7fb510072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.743+0000 7fb5162ab640 1 -- 172.21.15.16:0/1365373149 shutdown_connections 2024-04-25T10:05:39.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.743+0000 7fb5162ab640 1 -- 172.21.15.16:0/1365373149 wait complete. 2024-04-25T10:05:39.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.744+0000 7fb5162ab640 1 Processor -- start 2024-04-25T10:05:39.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.744+0000 7fb5162ab640 1 -- start start 2024-04-25T10:05:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.745+0000 7fb5162ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5100d0f60 0x7fb5100c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.745+0000 7fb5162ab640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5100d3b90 0x7fb5100ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.745+0000 7fb5162ab640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb51015dca0 0x7fb51014fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.745+0000 7fb5162ab640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5100cf050 con 0x7fb5100d0f60 2024-04-25T10:05:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.745+0000 7fb5162ab640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5100d1390 con 0x7fb5100d3b90 2024-04-25T10:05:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.745+0000 7fb5162ab640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51015e080 con 0x7fb51015dca0 2024-04-25T10:05:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.745+0000 7fb50ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5100d0f60 0x7fb5100c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.745+0000 7fb50ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5100d0f60 0x7fb5100c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37598/0 (socket says 172.21.15.16:37598) 2024-04-25T10:05:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.745+0000 7fb50ffff640 1 -- 172.21.15.16:0/3358556256 learned_addr learned my addr 172.21.15.16:0/3358556256 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.745+0000 7fb514848640 1 --2- 172.21.15.16:0/3358556256 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb51015dca0 0x7fb51014fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.745+0000 7fb50f7fe640 1 --2- 172.21.15.16:0/3358556256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5100d3b90 0x7fb5100ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.745+0000 7fb50ffff640 1 -- 172.21.15.16:0/3358556256 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb51015dca0 msgr2=0x7fb51014fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.745+0000 7fb50ffff640 1 --2- 172.21.15.16:0/3358556256 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb51015dca0 0x7fb51014fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.745+0000 7fb50ffff640 1 -- 172.21.15.16:0/3358556256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5100d3b90 msgr2=0x7fb5100ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.745+0000 7fb50ffff640 1 --2- 172.21.15.16:0/3358556256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5100d3b90 0x7fb5100ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.745+0000 7fb50ffff640 1 -- 172.21.15.16:0/3358556256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51015fc90 con 0x7fb5100d0f60 2024-04-25T10:05:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.745+0000 7fb50f7fe640 1 --2- 172.21.15.16:0/3358556256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5100d3b90 0x7fb5100ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.746+0000 7fb50ffff640 1 --2- 172.21.15.16:0/3358556256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5100d0f60 0x7fb5100c75c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb50000c3f0 tx=0x7fb50000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.746+0000 7fb50cff9640 1 -- 172.21.15.16:0/3358556256 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5000052d0 con 0x7fb5100d0f60 2024-04-25T10:05:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.746+0000 7fb50cff9640 1 -- 172.21.15.16:0/3358556256 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb500005490 con 0x7fb5100d0f60 2024-04-25T10:05:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.746+0000 7fb5162ab640 1 -- 172.21.15.16:0/3358556256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51015ff70 con 0x7fb5100d0f60 2024-04-25T10:05:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.746+0000 7fb50cff9640 1 -- 172.21.15.16:0/3358556256 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb500007260 con 0x7fb5100d0f60 2024-04-25T10:05:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.746+0000 7fb5162ab640 1 -- 172.21.15.16:0/3358556256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51015a0a0 con 0x7fb5100d0f60 2024-04-25T10:05:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.747+0000 7fb5162ab640 1 -- 172.21.15.16:0/3358556256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5100cfc40 con 0x7fb5100d0f60 2024-04-25T10:05:39.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.747+0000 7fb50cff9640 1 -- 172.21.15.16:0/3358556256 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb500007400 con 0x7fb5100d0f60 2024-04-25T10:05:39.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.748+0000 7fb50cff9640 1 --2- 172.21.15.16:0/3358556256 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4fc06b4d0 0x7fb4fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.748+0000 7fb50cff9640 1 -- 172.21.15.16:0/3358556256 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5000896d0 con 0x7fb5100d0f60 2024-04-25T10:05:39.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.748+0000 7fb50f7fe640 1 --2- 172.21.15.16:0/3358556256 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4fc06b4d0 0x7fb4fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.749+0000 7fb50f7fe640 1 --2- 172.21.15.16:0/3358556256 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4fc06b4d0 0x7fb4fc06d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb504034000 tx=0x7fb50402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.752+0000 7fb50cff9640 1 -- 172.21.15.16:0/3358556256 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5100cfc40 con 0x7fb5100d0f60 2024-04-25T10:05:39.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.931+0000 7fb5162ab640 1 -- 172.21.15.16:0/3358556256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5100750b0 con 0x7fb5100d0f60 2024-04-25T10:05:39.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.931+0000 7fb50cff9640 1 -- 172.21.15.16:0/3358556256 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb500058560 con 0x7fb5100d0f60 2024-04-25T10:05:39.937 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:05:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.933+0000 7fb5162ab640 1 -- 172.21.15.16:0/3358556256 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4fc06b4d0 msgr2=0x7fb4fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.933+0000 7fb5162ab640 1 --2- 172.21.15.16:0/3358556256 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4fc06b4d0 0x7fb4fc06d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb504034000 tx=0x7fb50402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.933+0000 7fb5162ab640 1 -- 172.21.15.16:0/3358556256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5100d0f60 msgr2=0x7fb5100c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.933+0000 7fb5162ab640 1 --2- 172.21.15.16:0/3358556256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5100d0f60 0x7fb5100c75c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb50000c3f0 tx=0x7fb50000c8b0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.933+0000 7fb5162ab640 1 -- 172.21.15.16:0/3358556256 shutdown_connections 2024-04-25T10:05:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.933+0000 7fb5162ab640 1 --2- 172.21.15.16:0/3358556256 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4fc06b4d0 0x7fb4fc06d990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.933+0000 7fb5162ab640 1 --2- 172.21.15.16:0/3358556256 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb51015dca0 0x7fb51014fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.933+0000 7fb5162ab640 1 --2- 172.21.15.16:0/3358556256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5100d3b90 0x7fb5100ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.933+0000 7fb5162ab640 1 --2- 172.21.15.16:0/3358556256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5100d0f60 0x7fb5100c75c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.933+0000 7fb5162ab640 1 -- 172.21.15.16:0/3358556256 >> 172.21.15.16:0/3358556256 conn(0x7fb5100726c0 msgr2=0x7fb5100b7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.933+0000 7fb5162ab640 1 -- 172.21.15.16:0/3358556256 shutdown_connections 2024-04-25T10:05:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:39.933+0000 7fb5162ab640 1 -- 172.21.15.16:0/3358556256 wait complete. 2024-04-25T10:05:39.949 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.048+0000 7f8427246640 1 Processor -- start 2024-04-25T10:05:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.048+0000 7f8427246640 1 -- start start 2024-04-25T10:05:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.049+0000 7f8427246640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f842015dc80 0x7f8420161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.049+0000 7f8427246640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84200b97c0 0x7f84200b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.049+0000 7f8427246640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84201627a0 con 0x7f84200b5a30 2024-04-25T10:05:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.049+0000 7f8427246640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84200b5ea0 con 0x7f84200b97c0 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.049+0000 7f8427246640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84201621e0 con 0x7f842015dc80 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.049+0000 7f841ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84200b97c0 0x7f84200b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.049+0000 7f841ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84200b97c0 0x7f84200b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37612/0 (socket says 172.21.15.16:37612) 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.049+0000 7f84257e3640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f84200b5a30 0x7f842006bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40994/0 (socket says 172.21.15.16:40994) 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.049+0000 7f841ffff640 1 -- 172.21.15.16:0/1408396766 learned_addr learned my addr 172.21.15.16:0/1408396766 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.049+0000 7f8424fe2640 1 --2- 172.21.15.16:0/1408396766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f842015dc80 0x7f8420161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.050+0000 7f841ffff640 1 -- 172.21.15.16:0/1408396766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f842015dc80 msgr2=0x7f8420161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.050+0000 7f841ffff640 1 --2- 172.21.15.16:0/1408396766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f842015dc80 0x7f8420161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.050+0000 7f841ffff640 1 -- 172.21.15.16:0/1408396766 >> v1:172.21.15.16:6789/0 conn(0x7f84200b5a30 legacy=0x7f842006bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.050+0000 7f841ffff640 1 -- 172.21.15.16:0/1408396766 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84201624c0 con 0x7f84200b97c0 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.050+0000 7f8424fe2640 1 --2- 172.21.15.16:0/1408396766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f842015dc80 0x7f8420161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.050+0000 7f841ffff640 1 --2- 172.21.15.16:0/1408396766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84200b97c0 0x7f84200b54f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8414009870 tx=0x7f841402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fce0faf039974a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.050+0000 7f841f7fe640 1 -- 172.21.15.16:0/1408396766 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8414037070 con 0x7f84200b97c0 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.050+0000 7f841f7fe640 1 -- 172.21.15.16:0/1408396766 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8414032440 con 0x7f84200b97c0 2024-04-25T10:05:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.050+0000 7f841f7fe640 1 -- 172.21.15.16:0/1408396766 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8414032810 con 0x7f84200b97c0 2024-04-25T10:05:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.051+0000 7f8427246640 1 -- 172.21.15.16:0/1408396766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84200b97c0 msgr2=0x7f84200b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.051+0000 7f8427246640 1 --2- 172.21.15.16:0/1408396766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84200b97c0 0x7f84200b54f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8414009870 tx=0x7f841402a270 comp rx=0 tx=0).stop 2024-04-25T10:05:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.051+0000 7f8427246640 1 -- 172.21.15.16:0/1408396766 shutdown_connections 2024-04-25T10:05:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.051+0000 7f8427246640 1 --2- 172.21.15.16:0/1408396766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84200b97c0 0x7f84200b54f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.051+0000 7f8427246640 1 --2- 172.21.15.16:0/1408396766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f842015dc80 0x7f8420161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.051+0000 7f8427246640 1 -- 172.21.15.16:0/1408396766 >> 172.21.15.16:0/1408396766 conn(0x7f84200d04f0 msgr2=0x7f84200d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.051+0000 7f8427246640 1 -- 172.21.15.16:0/1408396766 shutdown_connections 2024-04-25T10:05:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.051+0000 7f8427246640 1 -- 172.21.15.16:0/1408396766 wait complete. 2024-04-25T10:05:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.052+0000 7f8427246640 1 Processor -- start 2024-04-25T10:05:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.052+0000 7f8427246640 1 -- start start 2024-04-25T10:05:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.053+0000 7f8427246640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84200b5a30 0x7f84200cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.053+0000 7f8427246640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84200b97c0 0x7f84200cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.053+0000 7f8427246640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f842015dc80 0x7f84200c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.053+0000 7f8427246640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8420161db0 con 0x7f842015dc80 2024-04-25T10:05:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.053+0000 7f8427246640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842015e060 con 0x7f84200b5a30 2024-04-25T10:05:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.053+0000 7f8427246640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8420161f30 con 0x7f84200b97c0 2024-04-25T10:05:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.053+0000 7f84257e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f842015dc80 0x7f84200c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.053+0000 7f84257e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f842015dc80 0x7f84200c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37624/0 (socket says 172.21.15.16:37624) 2024-04-25T10:05:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.053+0000 7f84257e3640 1 -- 172.21.15.16:0/1375811552 learned_addr learned my addr 172.21.15.16:0/1375811552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.053+0000 7f841ffff640 1 --2- 172.21.15.16:0/1375811552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84200b97c0 0x7f84200cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.053+0000 7f8424fe2640 1 --2- 172.21.15.16:0/1375811552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84200b5a30 0x7f84200cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.053+0000 7f84257e3640 1 -- 172.21.15.16:0/1375811552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84200b97c0 msgr2=0x7f84200cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.053+0000 7f84257e3640 1 --2- 172.21.15.16:0/1375811552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84200b97c0 0x7f84200cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.053+0000 7f84257e3640 1 -- 172.21.15.16:0/1375811552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84200b5a30 msgr2=0x7f84200cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.053+0000 7f84257e3640 1 --2- 172.21.15.16:0/1375811552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84200b5a30 0x7f84200cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.054+0000 7f84257e3640 1 -- 172.21.15.16:0/1375811552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842014cd70 con 0x7f842015dc80 2024-04-25T10:05:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.054+0000 7f841ffff640 1 --2- 172.21.15.16:0/1375811552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84200b97c0 0x7f84200cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.054+0000 7f84257e3640 1 --2- 172.21.15.16:0/1375811552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f842015dc80 0x7f84200c8c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f841000c400 tx=0x7f841000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.054+0000 7f841d7fa640 1 -- 172.21.15.16:0/1375811552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84100074e0 con 0x7f842015dc80 2024-04-25T10:05:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.054+0000 7f841d7fa640 1 -- 172.21.15.16:0/1375811552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8410007680 con 0x7f842015dc80 2024-04-25T10:05:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.054+0000 7f841d7fa640 1 -- 172.21.15.16:0/1375811552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84100049e0 con 0x7f842015dc80 2024-04-25T10:05:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.054+0000 7f8427246640 1 -- 172.21.15.16:0/1375811552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842014d050 con 0x7f842015dc80 2024-04-25T10:05:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.054+0000 7f8427246640 1 -- 172.21.15.16:0/1375811552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842014d610 con 0x7f842015dc80 2024-04-25T10:05:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.055+0000 7f8427246640 1 -- 172.21.15.16:0/1375811552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84200b54f0 con 0x7f842015dc80 2024-04-25T10:05:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.055+0000 7f841d7fa640 1 -- 172.21.15.16:0/1375811552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f841001a020 con 0x7f842015dc80 2024-04-25T10:05:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.056+0000 7f841d7fa640 1 --2- 172.21.15.16:0/1375811552 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f840c06b4d0 0x7f840c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.056+0000 7f841d7fa640 1 -- 172.21.15.16:0/1375811552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84100899f0 con 0x7f842015dc80 2024-04-25T10:05:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.056+0000 7f8424fe2640 1 --2- 172.21.15.16:0/1375811552 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f840c06b4d0 0x7f840c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.057+0000 7f8424fe2640 1 --2- 172.21.15.16:0/1375811552 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f840c06b4d0 0x7f840c06d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8418007510 tx=0x7f84180074a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.060+0000 7f841d7fa640 1 -- 172.21.15.16:0/1375811552 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84200b54f0 con 0x7f842015dc80 2024-04-25T10:05:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.239+0000 7f8427246640 1 -- 172.21.15.16:0/1375811552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f84200d1780 con 0x7f842015dc80 2024-04-25T10:05:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.240+0000 7f841d7fa640 1 -- 172.21.15.16:0/1375811552 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8410058870 con 0x7f842015dc80 2024-04-25T10:05:40.245 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:05:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.241+0000 7f8427246640 1 -- 172.21.15.16:0/1375811552 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f840c06b4d0 msgr2=0x7f840c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.241+0000 7f8427246640 1 --2- 172.21.15.16:0/1375811552 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f840c06b4d0 0x7f840c06d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8418007510 tx=0x7f84180074a0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.241+0000 7f8427246640 1 -- 172.21.15.16:0/1375811552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f842015dc80 msgr2=0x7f84200c8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.241+0000 7f8427246640 1 --2- 172.21.15.16:0/1375811552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f842015dc80 0x7f84200c8c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f841000c400 tx=0x7f841000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.242+0000 7f8427246640 1 -- 172.21.15.16:0/1375811552 shutdown_connections 2024-04-25T10:05:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.242+0000 7f8427246640 1 --2- 172.21.15.16:0/1375811552 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f840c06b4d0 0x7f840c06d990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.242+0000 7f8427246640 1 --2- 172.21.15.16:0/1375811552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f842015dc80 0x7f84200c8c10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.242+0000 7f8427246640 1 --2- 172.21.15.16:0/1375811552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84200b97c0 0x7f84200cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.242+0000 7f8427246640 1 --2- 172.21.15.16:0/1375811552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84200b5a30 0x7f84200cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.242+0000 7f8427246640 1 -- 172.21.15.16:0/1375811552 >> 172.21.15.16:0/1375811552 conn(0x7f84200d04f0 msgr2=0x7f84200cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.242+0000 7f8427246640 1 -- 172.21.15.16:0/1375811552 shutdown_connections 2024-04-25T10:05:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.242+0000 7f8427246640 1 -- 172.21.15.16:0/1375811552 wait complete. 2024-04-25T10:05:40.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:40.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.356+0000 7f4c97968640 1 Processor -- start 2024-04-25T10:05:40.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.356+0000 7f4c97968640 1 -- start start 2024-04-25T10:05:40.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.357+0000 7f4c97968640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4c9015b7b0 0x7f4c900ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.357+0000 7f4c97968640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c900cf760 0x7f4c900cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.357+0000 7f4c97968640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9015f000 con 0x7f4c9015e110 2024-04-25T10:05:40.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.357+0000 7f4c97968640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c900d0100 con 0x7f4c9015b7b0 2024-04-25T10:05:40.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.357+0000 7f4c97968640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900ced00 con 0x7f4c900cf760 2024-04-25T10:05:40.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.357+0000 7f4c95f05640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4c9015e110 0x7f4c900cc080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41010/0 (socket says 172.21.15.16:41010) 2024-04-25T10:05:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.357+0000 7f4c95f05640 1 -- 172.21.15.16:0/4034901203 learned_addr learned my addr 172.21.15.16:0/4034901203 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.357+0000 7f4c95704640 1 --2- 172.21.15.16:0/4034901203 >> v2:172.21.15.186:3300/0 conn(0x7f4c9015b7b0 0x7f4c900ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.357+0000 7f4c94f03640 1 --2- 172.21.15.16:0/4034901203 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c900cf760 0x7f4c900cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.357+0000 7f4c8ffff640 1 -- 172.21.15.16:0/4034901203 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3356392860 0 0) 0x7f4c9015f000 con 0x7f4c9015e110 2024-04-25T10:05:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.357+0000 7f4c8ffff640 1 -- 172.21.15.16:0/4034901203 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c78003520 con 0x7f4c9015e110 2024-04-25T10:05:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c8ffff640 1 -- 172.21.15.16:0/4034901203 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1806114614 0 0) 0x7f4c78003520 con 0x7f4c9015e110 2024-04-25T10:05:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c8ffff640 1 -- 172.21.15.16:0/4034901203 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c900cf760 msgr2=0x7f4c900cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c8ffff640 1 --2- 172.21.15.16:0/4034901203 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c900cf760 0x7f4c900cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c8ffff640 1 -- 172.21.15.16:0/4034901203 >> v2:172.21.15.186:3300/0 conn(0x7f4c9015b7b0 msgr2=0x7f4c900ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c8ffff640 1 --2- 172.21.15.16:0/4034901203 >> v2:172.21.15.186:3300/0 conn(0x7f4c9015b7b0 0x7f4c900ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c8ffff640 1 -- 172.21.15.16:0/4034901203 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9015ed20 con 0x7f4c9015e110 2024-04-25T10:05:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c8ffff640 1 -- 172.21.15.16:0/4034901203 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f4c880019f0 con 0x7f4c9015e110 2024-04-25T10:05:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c8ffff640 1 -- 172.21.15.16:0/4034901203 >> v1:172.21.15.16:6789/0 conn(0x7f4c9015e110 legacy=0x7f4c900cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c8ffff640 1 --2- 172.21.15.16:0/4034901203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c780039b0 0x7f4c78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c8ffff640 1 --2- 172.21.15.16:0/4034901203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c78004800 0x7f4c78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c8ffff640 1 --2- 172.21.15.16:0/4034901203 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c900cf760 0x7f4c780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c8ffff640 1 -- 172.21.15.16:0/4034901203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900ced00 con 0x7f4c78004800 2024-04-25T10:05:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c8ffff640 1 -- 172.21.15.16:0/4034901203 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7800b9c0 con 0x7f4c780039b0 2024-04-25T10:05:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c8ffff640 1 -- 172.21.15.16:0/4034901203 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7800bb40 con 0x7f4c900cf760 2024-04-25T10:05:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c94f03640 1 --2- 172.21.15.16:0/4034901203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c78004800 0x7f4c78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c95704640 1 --2- 172.21.15.16:0/4034901203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c780039b0 0x7f4c78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c95f05640 1 --2- 172.21.15.16:0/4034901203 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c900cf760 0x7f4c780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c94f03640 1 -- 172.21.15.16:0/4034901203 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c900cf760 msgr2=0x7f4c780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c94f03640 1 --2- 172.21.15.16:0/4034901203 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c900cf760 0x7f4c780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c94f03640 1 -- 172.21.15.16:0/4034901203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c780039b0 msgr2=0x7f4c78003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c94f03640 1 --2- 172.21.15.16:0/4034901203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c780039b0 0x7f4c78003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.358+0000 7f4c94f03640 1 -- 172.21.15.16:0/4034901203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c78003520 con 0x7f4c78004800 2024-04-25T10:05:40.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.359+0000 7f4c94f03640 1 --2- 172.21.15.16:0/4034901203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c78004800 0x7f4c78004c80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009ac0 tx=0x7f4c78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8cb2aa461905263 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.359+0000 7f4c8ffff640 1 -- 172.21.15.16:0/4034901203 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84036070 con 0x7f4c78004800 2024-04-25T10:05:40.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.359+0000 7f4c8ffff640 1 -- 172.21.15.16:0/4034901203 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c84031400 con 0x7f4c78004800 2024-04-25T10:05:40.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.359+0000 7f4c8ffff640 1 -- 172.21.15.16:0/4034901203 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84009690 con 0x7f4c78004800 2024-04-25T10:05:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.359+0000 7f4c97968640 1 -- 172.21.15.16:0/4034901203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c78004800 msgr2=0x7f4c78004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.359+0000 7f4c97968640 1 --2- 172.21.15.16:0/4034901203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c78004800 0x7f4c78004c80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009ac0 tx=0x7f4c78005830 comp rx=0 tx=0).stop 2024-04-25T10:05:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.360+0000 7f4c95f05640 1 -- 172.21.15.16:0/4034901203 reap_dead start 2024-04-25T10:05:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.360+0000 7f4c97968640 1 -- 172.21.15.16:0/4034901203 shutdown_connections 2024-04-25T10:05:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.360+0000 7f4c97968640 1 -- 172.21.15.16:0/4034901203 >> 172.21.15.16:0/4034901203 conn(0x7f4c9006f4a0 msgr2=0x7f4c9006f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.360+0000 7f4c97968640 1 -- 172.21.15.16:0/4034901203 shutdown_connections 2024-04-25T10:05:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.360+0000 7f4c97968640 1 -- 172.21.15.16:0/4034901203 wait complete. 2024-04-25T10:05:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.361+0000 7f4c97968640 1 Processor -- start 2024-04-25T10:05:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.361+0000 7f4c97968640 1 -- start start 2024-04-25T10:05:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c97968640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c90147fd0 0x7f4c900b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c97968640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c901483b0 0x7f4c901585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c97968640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c90148830 0x7f4c90158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c97968640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900cdfc0 con 0x7f4c90147fd0 2024-04-25T10:05:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c97968640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900ce330 con 0x7f4c90148830 2024-04-25T10:05:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c97968640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c901619f0 con 0x7f4c901483b0 2024-04-25T10:05:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c95704640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c90147fd0 0x7f4c900b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c95704640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c90147fd0 0x7f4c900b2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37634/0 (socket says 172.21.15.16:37634) 2024-04-25T10:05:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c95704640 1 -- 172.21.15.16:0/2065640794 learned_addr learned my addr 172.21.15.16:0/2065640794 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c95704640 1 -- 172.21.15.16:0/2065640794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c901483b0 msgr2=0x7f4c901585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c95704640 1 --2- 172.21.15.16:0/2065640794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c901483b0 0x7f4c901585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c95704640 1 -- 172.21.15.16:0/2065640794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c90148830 msgr2=0x7f4c90158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c95704640 1 --2- 172.21.15.16:0/2065640794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c90148830 0x7f4c90158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c95704640 1 -- 172.21.15.16:0/2065640794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c900b3470 con 0x7f4c90147fd0 2024-04-25T10:05:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c95704640 1 --2- 172.21.15.16:0/2065640794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c90147fd0 0x7f4c900b2de0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4c80011a70 tx=0x7f4c80011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c8dffb640 1 -- 172.21.15.16:0/2065640794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000fb50 con 0x7f4c90147fd0 2024-04-25T10:05:40.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c8dffb640 1 -- 172.21.15.16:0/2065640794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8000fcf0 con 0x7f4c90147fd0 2024-04-25T10:05:40.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c97968640 1 -- 172.21.15.16:0/2065640794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c900b3750 con 0x7f4c90147fd0 2024-04-25T10:05:40.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.362+0000 7f4c8dffb640 1 -- 172.21.15.16:0/2065640794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000fea0 con 0x7f4c90147fd0 2024-04-25T10:05:40.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.363+0000 7f4c97968640 1 -- 172.21.15.16:0/2065640794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c900b3d10 con 0x7f4c90147fd0 2024-04-25T10:05:40.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.364+0000 7f4c97968640 1 -- 172.21.15.16:0/2065640794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c900750b0 con 0x7f4c90147fd0 2024-04-25T10:05:40.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.364+0000 7f4c8dffb640 1 -- 172.21.15.16:0/2065640794 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c80012040 con 0x7f4c90147fd0 2024-04-25T10:05:40.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.364+0000 7f4c8dffb640 1 --2- 172.21.15.16:0/2065640794 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4c7c06b4d0 0x7f4c7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.365+0000 7f4c8dffb640 1 -- 172.21.15.16:0/2065640794 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c8008a800 con 0x7f4c90147fd0 2024-04-25T10:05:40.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.365+0000 7f4c94f03640 1 --2- 172.21.15.16:0/2065640794 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4c7c06b4d0 0x7f4c7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.365+0000 7f4c94f03640 1 --2- 172.21.15.16:0/2065640794 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4c7c06b4d0 0x7f4c7c06d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4c84033000 tx=0x7f4c8402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.369+0000 7f4c8dffb640 1 -- 172.21.15.16:0/2065640794 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c80019030 con 0x7f4c90147fd0 2024-04-25T10:05:40.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.549+0000 7f4c97968640 1 -- 172.21.15.16:0/2065640794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4c900b7870 con 0x7f4c90147fd0 2024-04-25T10:05:40.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.549+0000 7f4c8dffb640 1 -- 172.21.15.16:0/2065640794 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4c80059680 con 0x7f4c90147fd0 2024-04-25T10:05:40.555 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:05:40.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.551+0000 7f4c97968640 1 -- 172.21.15.16:0/2065640794 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4c7c06b4d0 msgr2=0x7f4c7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.551+0000 7f4c97968640 1 --2- 172.21.15.16:0/2065640794 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4c7c06b4d0 0x7f4c7c06d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4c84033000 tx=0x7f4c8402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:40.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.551+0000 7f4c97968640 1 -- 172.21.15.16:0/2065640794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c90147fd0 msgr2=0x7f4c900b2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.551+0000 7f4c97968640 1 --2- 172.21.15.16:0/2065640794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c90147fd0 0x7f4c900b2de0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4c80011a70 tx=0x7f4c80011f30 comp rx=0 tx=0).stop 2024-04-25T10:05:40.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.551+0000 7f4c97968640 1 -- 172.21.15.16:0/2065640794 shutdown_connections 2024-04-25T10:05:40.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.551+0000 7f4c97968640 1 --2- 172.21.15.16:0/2065640794 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4c7c06b4d0 0x7f4c7c06d990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.551+0000 7f4c97968640 1 --2- 172.21.15.16:0/2065640794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c90148830 0x7f4c90158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.551+0000 7f4c97968640 1 --2- 172.21.15.16:0/2065640794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c901483b0 0x7f4c901585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.551+0000 7f4c97968640 1 --2- 172.21.15.16:0/2065640794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c90147fd0 0x7f4c900b2de0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.551+0000 7f4c97968640 1 -- 172.21.15.16:0/2065640794 >> 172.21.15.16:0/2065640794 conn(0x7f4c9006f4a0 msgr2=0x7f4c9006f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.552+0000 7f4c97968640 1 -- 172.21.15.16:0/2065640794 shutdown_connections 2024-04-25T10:05:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.552+0000 7f4c97968640 1 -- 172.21.15.16:0/2065640794 wait complete. 2024-04-25T10:05:40.576 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:05:40.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:40.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.672+0000 7f720e94f640 1 Processor -- start 2024-04-25T10:05:40.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.672+0000 7f720e94f640 1 -- start start 2024-04-25T10:05:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.673+0000 7f720e94f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f720815dc30 0x7f720806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.673+0000 7f720e94f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f720806c1f0 con 0x7f720806cc00 2024-04-25T10:05:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.673+0000 7f720e94f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72080cfb40 con 0x7f72080b1b30 2024-04-25T10:05:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.673+0000 7f720e94f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72080b2060 con 0x7f720815dc30 2024-04-25T10:05:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.673+0000 7f72077fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f720806cc00 0x7f72080b1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41024/0 (socket says 172.21.15.16:41024) 2024-04-25T10:05:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.673+0000 7f72077fe640 1 -- 172.21.15.16:0/1453033617 learned_addr learned my addr 172.21.15.16:0/1453033617 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7207fff640 1 --2- 172.21.15.16:0/1453033617 >> v2:172.21.15.186:3300/0 conn(0x7f720815dc30 0x7f720806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7206ffd640 1 -- 172.21.15.16:0/1453033617 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665393776 0 0) 0x7f71fc001110 con 0x7f720806cc00 2024-04-25T10:05:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7206ffd640 1 -- 172.21.15.16:0/1453033617 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71f0003520 con 0x7f720806cc00 2024-04-25T10:05:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7206ffd640 1 -- 172.21.15.16:0/1453033617 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1863325788 0 0) 0x7f71f0003520 con 0x7f720806cc00 2024-04-25T10:05:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7206ffd640 1 -- 172.21.15.16:0/1453033617 >> v2:172.21.15.186:3300/0 conn(0x7f720815dc30 msgr2=0x7f720806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7206ffd640 1 --2- 172.21.15.16:0/1453033617 >> v2:172.21.15.186:3300/0 conn(0x7f720815dc30 0x7f720806c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7206ffd640 1 -- 172.21.15.16:0/1453033617 >> v1:172.21.15.186:6789/0 conn(0x7f72080b1b30 legacy=0x7f72080d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7206ffd640 1 -- 172.21.15.16:0/1453033617 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72080b2340 con 0x7f720806cc00 2024-04-25T10:05:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7206ffd640 1 -- 172.21.15.16:0/1453033617 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f71fc001840 con 0x7f720806cc00 2024-04-25T10:05:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7206ffd640 1 -- 172.21.15.16:0/1453033617 >> v1:172.21.15.16:6789/0 conn(0x7f720806cc00 legacy=0x7f72080b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7206ffd640 1 --2- 172.21.15.16:0/1453033617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71f00039b0 0x7f71f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7206ffd640 1 --2- 172.21.15.16:0/1453033617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71f0004800 0x7f71f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7206ffd640 1 --2- 172.21.15.16:0/1453033617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71f00051c0 0x7f71f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7206ffd640 1 -- 172.21.15.16:0/1453033617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72080b2060 con 0x7f71f00039b0 2024-04-25T10:05:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7206ffd640 1 -- 172.21.15.16:0/1453033617 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f000c060 con 0x7f71f00051c0 2024-04-25T10:05:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7206ffd640 1 -- 172.21.15.16:0/1453033617 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f000c260 con 0x7f71f0004800 2024-04-25T10:05:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7207fff640 1 --2- 172.21.15.16:0/1453033617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71f00039b0 0x7f71f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f72077fe640 1 --2- 172.21.15.16:0/1453033617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71f0004800 0x7f71f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f720ceec640 1 --2- 172.21.15.16:0/1453033617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71f00051c0 0x7f71f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7207fff640 1 -- 172.21.15.16:0/1453033617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71f0004800 msgr2=0x7f71f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7207fff640 1 --2- 172.21.15.16:0/1453033617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71f0004800 0x7f71f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7207fff640 1 -- 172.21.15.16:0/1453033617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71f00051c0 msgr2=0x7f71f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7207fff640 1 --2- 172.21.15.16:0/1453033617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71f00051c0 0x7f71f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f720ceec640 1 -- 172.21.15.16:0/1453033617 reap_dead start 2024-04-25T10:05:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.674+0000 7f7207fff640 1 -- 172.21.15.16:0/1453033617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f0003520 con 0x7f71f00039b0 2024-04-25T10:05:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.675+0000 7f72077fe640 1 --2- 172.21.15.16:0/1453033617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71f0004800 0x7f71f0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.675+0000 7f7207fff640 1 --2- 172.21.15.16:0/1453033617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71f00039b0 0x7f71f0003dd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009ac0 tx=0x7f71f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d962383b192de186 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.675+0000 7f7206ffd640 1 -- 172.21.15.16:0/1453033617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8036070 con 0x7f71f00039b0 2024-04-25T10:05:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.675+0000 7f7206ffd640 1 -- 172.21.15.16:0/1453033617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f8031400 con 0x7f71f00039b0 2024-04-25T10:05:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.675+0000 7f7206ffd640 1 -- 172.21.15.16:0/1453033617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8009690 con 0x7f71f00039b0 2024-04-25T10:05:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.676+0000 7f720e94f640 1 -- 172.21.15.16:0/1453033617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71f00039b0 msgr2=0x7f71f0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.676+0000 7f720e94f640 1 --2- 172.21.15.16:0/1453033617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71f00039b0 0x7f71f0003dd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009ac0 tx=0x7f71f0005830 comp rx=0 tx=0).stop 2024-04-25T10:05:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.676+0000 7f720e94f640 1 -- 172.21.15.16:0/1453033617 shutdown_connections 2024-04-25T10:05:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.676+0000 7f720e94f640 1 --2- 172.21.15.16:0/1453033617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71f00039b0 0x7f71f0003dd0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.676+0000 7f720e94f640 1 -- 172.21.15.16:0/1453033617 >> 172.21.15.16:0/1453033617 conn(0x7f72080d1900 msgr2=0x7f72080d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.676+0000 7f720e94f640 1 -- 172.21.15.16:0/1453033617 shutdown_connections 2024-04-25T10:05:40.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.676+0000 7f720e94f640 1 -- 172.21.15.16:0/1453033617 wait complete. 2024-04-25T10:05:40.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.677+0000 7f720e94f640 1 Processor -- start 2024-04-25T10:05:40.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.677+0000 7f720e94f640 1 -- start start 2024-04-25T10:05:40.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.678+0000 7f720e94f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71f00039b0 0x7f72080c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.678+0000 7f720e94f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72080cb520 0x7f7208153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.678+0000 7f720e94f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7208154140 0x7f720815f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.678+0000 7f720e94f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720815e010 con 0x7f7208154140 2024-04-25T10:05:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.678+0000 7f720e94f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72080cfde0 con 0x7f72080cb520 2024-04-25T10:05:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.678+0000 7f720e94f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720806c6c0 con 0x7f71f00039b0 2024-04-25T10:05:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.678+0000 7f720ceec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7208154140 0x7f720815f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.678+0000 7f720ceec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7208154140 0x7f720815f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37664/0 (socket says 172.21.15.16:37664) 2024-04-25T10:05:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.678+0000 7f720ceec640 1 -- 172.21.15.16:0/2579550542 learned_addr learned my addr 172.21.15.16:0/2579550542 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.678+0000 7f7207fff640 1 --2- 172.21.15.16:0/2579550542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71f00039b0 0x7f72080c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.678+0000 7f72077fe640 1 --2- 172.21.15.16:0/2579550542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72080cb520 0x7f7208153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.679+0000 7f720ceec640 1 -- 172.21.15.16:0/2579550542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71f00039b0 msgr2=0x7f72080c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.679+0000 7f720ceec640 1 --2- 172.21.15.16:0/2579550542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71f00039b0 0x7f72080c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.679+0000 7f720ceec640 1 -- 172.21.15.16:0/2579550542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72080cb520 msgr2=0x7f7208153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.679+0000 7f720ceec640 1 --2- 172.21.15.16:0/2579550542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72080cb520 0x7f7208153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.679+0000 7f720ceec640 1 -- 172.21.15.16:0/2579550542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72080c69e0 con 0x7f7208154140 2024-04-25T10:05:40.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.679+0000 7f720ceec640 1 --2- 172.21.15.16:0/2579550542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7208154140 0x7f720815f7c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f720806d210 tx=0x7f720000fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.679+0000 7f7204ff9640 1 -- 172.21.15.16:0/2579550542 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720001a070 con 0x7f7208154140 2024-04-25T10:05:40.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.679+0000 7f7204ff9640 1 -- 172.21.15.16:0/2579550542 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7200006d60 con 0x7f7208154140 2024-04-25T10:05:40.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.679+0000 7f720e94f640 1 -- 172.21.15.16:0/2579550542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72080cb9c0 con 0x7f7208154140 2024-04-25T10:05:40.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.679+0000 7f7204ff9640 1 -- 172.21.15.16:0/2579550542 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200003bb0 con 0x7f7208154140 2024-04-25T10:05:40.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.679+0000 7f720e94f640 1 -- 172.21.15.16:0/2579550542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72080b44f0 con 0x7f7208154140 2024-04-25T10:05:40.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.680+0000 7f720e94f640 1 -- 172.21.15.16:0/2579550542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7208067850 con 0x7f7208154140 2024-04-25T10:05:40.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.680+0000 7f7204ff9640 1 -- 172.21.15.16:0/2579550542 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f720001d070 con 0x7f7208154140 2024-04-25T10:05:40.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.681+0000 7f7204ff9640 1 --2- 172.21.15.16:0/2579550542 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f71f406b4d0 0x7f71f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.681+0000 7f7204ff9640 1 -- 172.21.15.16:0/2579550542 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72000897a0 con 0x7f7208154140 2024-04-25T10:05:40.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.681+0000 7f7207fff640 1 --2- 172.21.15.16:0/2579550542 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f71f406b4d0 0x7f71f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.682+0000 7f7207fff640 1 --2- 172.21.15.16:0/2579550542 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f71f406b4d0 0x7f71f406d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f71f8033000 tx=0x7f71f8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.685+0000 7f7204ff9640 1 -- 172.21.15.16:0/2579550542 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7200058620 con 0x7f7208154140 2024-04-25T10:05:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.868+0000 7f720e94f640 1 -- 172.21.15.16:0/2579550542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72080749b0 con 0x7f7208154140 2024-04-25T10:05:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.869+0000 7f7204ff9640 1 -- 172.21.15.16:0/2579550542 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f7200057f20 con 0x7f7208154140 2024-04-25T10:05:40.874 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:40.875 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:12.449183+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.871+0000 7f720e94f640 1 -- 172.21.15.16:0/2579550542 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f71f406b4d0 msgr2=0x7f71f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.871+0000 7f720e94f640 1 --2- 172.21.15.16:0/2579550542 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f71f406b4d0 0x7f71f406d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f71f8033000 tx=0x7f71f8008030 comp rx=0 tx=0).stop 2024-04-25T10:05:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.871+0000 7f720e94f640 1 -- 172.21.15.16:0/2579550542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7208154140 msgr2=0x7f720815f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.871+0000 7f720e94f640 1 --2- 172.21.15.16:0/2579550542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7208154140 0x7f720815f7c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f720806d210 tx=0x7f720000fc00 comp rx=0 tx=0).stop 2024-04-25T10:05:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.871+0000 7f720e94f640 1 -- 172.21.15.16:0/2579550542 shutdown_connections 2024-04-25T10:05:40.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.871+0000 7f720e94f640 1 --2- 172.21.15.16:0/2579550542 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f71f406b4d0 0x7f71f406d990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.871+0000 7f720e94f640 1 --2- 172.21.15.16:0/2579550542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7208154140 0x7f720815f7c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.871+0000 7f720e94f640 1 --2- 172.21.15.16:0/2579550542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72080cb520 0x7f7208153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.871+0000 7f720e94f640 1 --2- 172.21.15.16:0/2579550542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71f00039b0 0x7f72080c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.871+0000 7f720e94f640 1 -- 172.21.15.16:0/2579550542 >> 172.21.15.16:0/2579550542 conn(0x7f72080d1900 msgr2=0x7f72080b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.871+0000 7f720e94f640 1 -- 172.21.15.16:0/2579550542 shutdown_connections 2024-04-25T10:05:40.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.871+0000 7f720e94f640 1 -- 172.21.15.16:0/2579550542 wait complete. 2024-04-25T10:05:40.887 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:08.724537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:09.793105+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:40.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:40.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.986+0000 7fb3f7cc4640 1 Processor -- start 2024-04-25T10:05:40.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.986+0000 7fb3f7cc4640 1 -- start start 2024-04-25T10:05:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.987+0000 7fb3f7cc4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3f015dc80 0x7fb3f0161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.987+0000 7fb3f7cc4640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb3f00b97c0 0x7fb3f00b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.987+0000 7fb3f7cc4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f01627a0 con 0x7fb3f00b5a30 2024-04-25T10:05:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.987+0000 7fb3f7cc4640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f00b5ea0 con 0x7fb3f00b97c0 2024-04-25T10:05:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.987+0000 7fb3f7cc4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f01621e0 con 0x7fb3f015dc80 2024-04-25T10:05:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.987+0000 7fb3f6261640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb3f00b5a30 0x7fb3f006bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41040/0 (socket says 172.21.15.16:41040) 2024-04-25T10:05:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.987+0000 7fb3f6261640 1 -- 172.21.15.16:0/907504972 learned_addr learned my addr 172.21.15.16:0/907504972 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.987+0000 7fb3f525f640 1 --2- 172.21.15.16:0/907504972 >> v2:172.21.15.186:3300/0 conn(0x7fb3f00b97c0 0x7fb3f00b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.987+0000 7fb3f5a60640 1 --2- 172.21.15.16:0/907504972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3f015dc80 0x7fb3f0161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.987+0000 7fb3f4a5e640 1 -- 172.21.15.16:0/907504972 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130113476 0 0) 0x7fb3f01627a0 con 0x7fb3f00b5a30 2024-04-25T10:05:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.987+0000 7fb3f4a5e640 1 -- 172.21.15.16:0/907504972 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3dc003520 con 0x7fb3f00b5a30 2024-04-25T10:05:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.987+0000 7fb3f525f640 1 -- 172.21.15.16:0/907504972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3f015dc80 msgr2=0x7fb3f0161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.987+0000 7fb3f525f640 1 --2- 172.21.15.16:0/907504972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3f015dc80 0x7fb3f0161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.987+0000 7fb3f525f640 1 -- 172.21.15.16:0/907504972 >> v1:172.21.15.16:6789/0 conn(0x7fb3f00b5a30 legacy=0x7fb3f006bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.987+0000 7fb3f525f640 1 -- 172.21.15.16:0/907504972 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f01624c0 con 0x7fb3f00b97c0 2024-04-25T10:05:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.988+0000 7fb3f525f640 1 --2- 172.21.15.16:0/907504972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3f00b97c0 0x7fb3f00b54f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00097a0 tx=0x7fb3e002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b57c96d53739fda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.988+0000 7fb3f4a5e640 1 -- 172.21.15.16:0/907504972 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0037070 con 0x7fb3f00b97c0 2024-04-25T10:05:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.988+0000 7fb3f4a5e640 1 -- 172.21.15.16:0/907504972 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e0032440 con 0x7fb3f00b97c0 2024-04-25T10:05:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.988+0000 7fb3f4a5e640 1 -- 172.21.15.16:0/907504972 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e00327f0 con 0x7fb3f00b97c0 2024-04-25T10:05:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.989+0000 7fb3f7cc4640 1 -- 172.21.15.16:0/907504972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3f00b97c0 msgr2=0x7fb3f00b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.989+0000 7fb3f7cc4640 1 --2- 172.21.15.16:0/907504972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3f00b97c0 0x7fb3f00b54f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00097a0 tx=0x7fb3e002a180 comp rx=0 tx=0).stop 2024-04-25T10:05:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.989+0000 7fb3f7cc4640 1 -- 172.21.15.16:0/907504972 shutdown_connections 2024-04-25T10:05:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.989+0000 7fb3f7cc4640 1 --2- 172.21.15.16:0/907504972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3f00b97c0 0x7fb3f00b54f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.989+0000 7fb3f7cc4640 1 --2- 172.21.15.16:0/907504972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3f015dc80 0x7fb3f0161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.989+0000 7fb3f7cc4640 1 -- 172.21.15.16:0/907504972 >> 172.21.15.16:0/907504972 conn(0x7fb3f00d04f0 msgr2=0x7fb3f00d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.989+0000 7fb3f7cc4640 1 -- 172.21.15.16:0/907504972 shutdown_connections 2024-04-25T10:05:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.989+0000 7fb3f7cc4640 1 -- 172.21.15.16:0/907504972 wait complete. 2024-04-25T10:05:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.990+0000 7fb3f7cc4640 1 Processor -- start 2024-04-25T10:05:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.990+0000 7fb3f7cc4640 1 -- start start 2024-04-25T10:05:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.991+0000 7fb3f7cc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f00b5a30 0x7fb3f015be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.991+0000 7fb3f7cc4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3f00b97c0 0x7fb3f015f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.991+0000 7fb3f7cc4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3f015dc80 0x7fb3f00cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.991+0000 7fb3f7cc4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f0161db0 con 0x7fb3f00b5a30 2024-04-25T10:05:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.991+0000 7fb3f7cc4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f015e060 con 0x7fb3f00b97c0 2024-04-25T10:05:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.991+0000 7fb3f7cc4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f0161f30 con 0x7fb3f015dc80 2024-04-25T10:05:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.991+0000 7fb3f5a60640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f00b5a30 0x7fb3f015be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.991+0000 7fb3f5a60640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f00b5a30 0x7fb3f015be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37678/0 (socket says 172.21.15.16:37678) 2024-04-25T10:05:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.991+0000 7fb3f5a60640 1 -- 172.21.15.16:0/1446640671 learned_addr learned my addr 172.21.15.16:0/1446640671 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.991+0000 7fb3f6261640 1 --2- 172.21.15.16:0/1446640671 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3f015dc80 0x7fb3f00cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.991+0000 7fb3f525f640 1 --2- 172.21.15.16:0/1446640671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3f00b97c0 0x7fb3f015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.991+0000 7fb3f5a60640 1 -- 172.21.15.16:0/1446640671 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3f015dc80 msgr2=0x7fb3f00cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.991+0000 7fb3f5a60640 1 --2- 172.21.15.16:0/1446640671 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3f015dc80 0x7fb3f00cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.991+0000 7fb3f5a60640 1 -- 172.21.15.16:0/1446640671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3f00b97c0 msgr2=0x7fb3f015f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.991+0000 7fb3f5a60640 1 --2- 172.21.15.16:0/1446640671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3f00b97c0 0x7fb3f015f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.991+0000 7fb3f5a60640 1 -- 172.21.15.16:0/1446640671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f015d250 con 0x7fb3f00b5a30 2024-04-25T10:05:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.991+0000 7fb3f525f640 1 --2- 172.21.15.16:0/1446640671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3f00b97c0 0x7fb3f015f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.991+0000 7fb3f6261640 1 --2- 172.21.15.16:0/1446640671 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3f015dc80 0x7fb3f00cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.992+0000 7fb3f5a60640 1 --2- 172.21.15.16:0/1446640671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f00b5a30 0x7fb3f015be60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00e9b0 tx=0x7fb3ec00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.992+0000 7fb3e67fc640 1 -- 172.21.15.16:0/1446640671 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec017070 con 0x7fb3f00b5a30 2024-04-25T10:05:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.992+0000 7fb3e67fc640 1 -- 172.21.15.16:0/1446640671 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ec00c9a0 con 0x7fb3f00b5a30 2024-04-25T10:05:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.992+0000 7fb3e67fc640 1 -- 172.21.15.16:0/1446640671 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec00cd90 con 0x7fb3f00b5a30 2024-04-25T10:05:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.992+0000 7fb3f7cc4640 1 -- 172.21.15.16:0/1446640671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f015ec90 con 0x7fb3f00b5a30 2024-04-25T10:05:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.992+0000 7fb3f7cc4640 1 -- 172.21.15.16:0/1446640671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f00cbcf0 con 0x7fb3f00b5a30 2024-04-25T10:05:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.993+0000 7fb3f7cc4640 1 -- 172.21.15.16:0/1446640671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f00750b0 con 0x7fb3f00b5a30 2024-04-25T10:05:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.993+0000 7fb3e67fc640 1 -- 172.21.15.16:0/1446640671 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3ec01a070 con 0x7fb3f00b5a30 2024-04-25T10:05:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.994+0000 7fb3e67fc640 1 --2- 172.21.15.16:0/1446640671 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb3d406b500 0x7fb3d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.994+0000 7fb3e67fc640 1 -- 172.21.15.16:0/1446640671 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3ec08a1f0 con 0x7fb3f00b5a30 2024-04-25T10:05:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.994+0000 7fb3f525f640 1 --2- 172.21.15.16:0/1446640671 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb3d406b500 0x7fb3d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.995+0000 7fb3f525f640 1 --2- 172.21.15.16:0/1446640671 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb3d406b500 0x7fb3d406d9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0034000 tx=0x7fb3e002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:40.998+0000 7fb3e67fc640 1 -- 172.21.15.16:0/1446640671 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3ec059070 con 0x7fb3f00b5a30 2024-04-25T10:05:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.179+0000 7fb3f7cc4640 1 -- 172.21.15.16:0/1446640671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb3f00d1780 con 0x7fb3f00b5a30 2024-04-25T10:05:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.180+0000 7fb3e67fc640 1 -- 172.21.15.16:0/1446640671 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb3ec058970 con 0x7fb3f00b5a30 2024-04-25T10:05:41.185 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:05:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.182+0000 7fb3f7cc4640 1 -- 172.21.15.16:0/1446640671 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb3d406b500 msgr2=0x7fb3d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.182+0000 7fb3f7cc4640 1 --2- 172.21.15.16:0/1446640671 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb3d406b500 0x7fb3d406d9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0034000 tx=0x7fb3e002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.182+0000 7fb3f7cc4640 1 -- 172.21.15.16:0/1446640671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f00b5a30 msgr2=0x7fb3f015be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.182+0000 7fb3f7cc4640 1 --2- 172.21.15.16:0/1446640671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f00b5a30 0x7fb3f015be60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00e9b0 tx=0x7fb3ec00ee70 comp rx=0 tx=0).stop 2024-04-25T10:05:41.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.182+0000 7fb3f7cc4640 1 -- 172.21.15.16:0/1446640671 shutdown_connections 2024-04-25T10:05:41.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.182+0000 7fb3f7cc4640 1 --2- 172.21.15.16:0/1446640671 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb3d406b500 0x7fb3d406d9c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.182+0000 7fb3f7cc4640 1 --2- 172.21.15.16:0/1446640671 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3f015dc80 0x7fb3f00cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.182+0000 7fb3f7cc4640 1 --2- 172.21.15.16:0/1446640671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3f00b97c0 0x7fb3f015f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.182+0000 7fb3f7cc4640 1 --2- 172.21.15.16:0/1446640671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f00b5a30 0x7fb3f015be60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.182+0000 7fb3f7cc4640 1 -- 172.21.15.16:0/1446640671 >> 172.21.15.16:0/1446640671 conn(0x7fb3f00d04f0 msgr2=0x7fb3f00cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:41.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.182+0000 7fb3f7cc4640 1 -- 172.21.15.16:0/1446640671 shutdown_connections 2024-04-25T10:05:41.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.182+0000 7fb3f7cc4640 1 -- 172.21.15.16:0/1446640671 wait complete. 2024-04-25T10:05:41.198 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:41.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.296+0000 7fbc8a153640 1 Processor -- start 2024-04-25T10:05:41.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.296+0000 7fbc8a153640 1 -- start start 2024-04-25T10:05:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.297+0000 7fbc8a153640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbc8415dc80 0x7fbc84161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.297+0000 7fbc8a153640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbc840b97c0 0x7fbc840b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.297+0000 7fbc8a153640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc840b5a30 0x7fbc8406bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.297+0000 7fbc8a153640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc840b5e60 con 0x7fbc840b97c0 2024-04-25T10:05:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.297+0000 7fbc8a153640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc841627a0 con 0x7fbc8415dc80 2024-04-25T10:05:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.297+0000 7fbc8a153640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc841621e0 con 0x7fbc840b5a30 2024-04-25T10:05:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.297+0000 7fbc82ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbc840b97c0 0x7fbc840b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.297+0000 7fbc82ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbc840b97c0 0x7fbc840b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37680/0 (socket says 172.21.15.16:37680) 2024-04-25T10:05:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.297+0000 7fbc82ffd640 1 -- 172.21.15.16:0/1206769428 learned_addr learned my addr 172.21.15.16:0/1206769428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.297+0000 7fbc837fe640 1 --2- 172.21.15.16:0/1206769428 >> v2:172.21.15.186:3300/0 conn(0x7fbc8415dc80 0x7fbc84161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.297+0000 7fbc83fff640 1 --2- 172.21.15.16:0/1206769428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc840b5a30 0x7fbc8406bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.297+0000 7fbc82ffd640 1 -- 172.21.15.16:0/1206769428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc840b5a30 msgr2=0x7fbc8406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.297+0000 7fbc82ffd640 1 --2- 172.21.15.16:0/1206769428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc840b5a30 0x7fbc8406bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.297+0000 7fbc82ffd640 1 -- 172.21.15.16:0/1206769428 >> v2:172.21.15.186:3300/0 conn(0x7fbc8415dc80 msgr2=0x7fbc84161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.297+0000 7fbc82ffd640 1 --2- 172.21.15.16:0/1206769428 >> v2:172.21.15.186:3300/0 conn(0x7fbc8415dc80 0x7fbc84161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.297+0000 7fbc82ffd640 1 -- 172.21.15.16:0/1206769428 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc841624c0 con 0x7fbc840b97c0 2024-04-25T10:05:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.297+0000 7fbc83fff640 1 --2- 172.21.15.16:0/1206769428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc840b5a30 0x7fbc8406bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.298+0000 7fbc82ffd640 1 --2- 172.21.15.16:0/1206769428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc840b97c0 0x7fbc840b54f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbc8406dab0 tx=0x7fbc7802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21b9500993c1e1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.298+0000 7fbc827fc640 1 -- 172.21.15.16:0/1206769428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78037070 con 0x7fbc840b97c0 2024-04-25T10:05:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.298+0000 7fbc827fc640 1 -- 172.21.15.16:0/1206769428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc78032440 con 0x7fbc840b97c0 2024-04-25T10:05:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.298+0000 7fbc827fc640 1 -- 172.21.15.16:0/1206769428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78032810 con 0x7fbc840b97c0 2024-04-25T10:05:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.299+0000 7fbc8a153640 1 -- 172.21.15.16:0/1206769428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc840b97c0 msgr2=0x7fbc840b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.299+0000 7fbc8a153640 1 --2- 172.21.15.16:0/1206769428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc840b97c0 0x7fbc840b54f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbc8406dab0 tx=0x7fbc7802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.299+0000 7fbc8a153640 1 -- 172.21.15.16:0/1206769428 shutdown_connections 2024-04-25T10:05:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.299+0000 7fbc8a153640 1 --2- 172.21.15.16:0/1206769428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc840b5a30 0x7fbc8406bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.299+0000 7fbc8a153640 1 --2- 172.21.15.16:0/1206769428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc840b97c0 0x7fbc840b54f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.299+0000 7fbc8a153640 1 --2- 172.21.15.16:0/1206769428 >> v2:172.21.15.186:3300/0 conn(0x7fbc8415dc80 0x7fbc84161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.299+0000 7fbc8a153640 1 -- 172.21.15.16:0/1206769428 >> 172.21.15.16:0/1206769428 conn(0x7fbc840d04f0 msgr2=0x7fbc840d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.299+0000 7fbc8a153640 1 -- 172.21.15.16:0/1206769428 shutdown_connections 2024-04-25T10:05:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.299+0000 7fbc8a153640 1 -- 172.21.15.16:0/1206769428 wait complete. 2024-04-25T10:05:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.300+0000 7fbc8a153640 1 Processor -- start 2024-04-25T10:05:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.300+0000 7fbc8a153640 1 -- start start 2024-04-25T10:05:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.301+0000 7fbc8a153640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc840b5a30 0x7fbc8415c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.301+0000 7fbc8a153640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc840b97c0 0x7fbc8414fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.301+0000 7fbc8a153640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc8415dc80 0x7fbc8415e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.301+0000 7fbc8a153640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc84161db0 con 0x7fbc840b5a30 2024-04-25T10:05:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.301+0000 7fbc8a153640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8415e060 con 0x7fbc840b97c0 2024-04-25T10:05:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.301+0000 7fbc8a153640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc84161f30 con 0x7fbc8415dc80 2024-04-25T10:05:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.301+0000 7fbc837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc840b5a30 0x7fbc8415c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.301+0000 7fbc837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc840b5a30 0x7fbc8415c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37690/0 (socket says 172.21.15.16:37690) 2024-04-25T10:05:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.301+0000 7fbc837fe640 1 -- 172.21.15.16:0/1939183598 learned_addr learned my addr 172.21.15.16:0/1939183598 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.301+0000 7fbc83fff640 1 --2- 172.21.15.16:0/1939183598 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc8415dc80 0x7fbc8415e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.301+0000 7fbc82ffd640 1 --2- 172.21.15.16:0/1939183598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc840b97c0 0x7fbc8414fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.301+0000 7fbc837fe640 1 -- 172.21.15.16:0/1939183598 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc8415dc80 msgr2=0x7fbc8415e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.301+0000 7fbc837fe640 1 --2- 172.21.15.16:0/1939183598 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc8415dc80 0x7fbc8415e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.302+0000 7fbc837fe640 1 -- 172.21.15.16:0/1939183598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc840b97c0 msgr2=0x7fbc8414fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.302+0000 7fbc837fe640 1 --2- 172.21.15.16:0/1939183598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc840b97c0 0x7fbc8414fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.302+0000 7fbc837fe640 1 -- 172.21.15.16:0/1939183598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc84150210 con 0x7fbc840b5a30 2024-04-25T10:05:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.302+0000 7fbc837fe640 1 --2- 172.21.15.16:0/1939183598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc840b5a30 0x7fbc8415c760 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400e870 tx=0x7fbc7400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.302+0000 7fbc6bfff640 1 -- 172.21.15.16:0/1939183598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc74017070 con 0x7fbc840b5a30 2024-04-25T10:05:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.302+0000 7fbc6bfff640 1 -- 172.21.15.16:0/1939183598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7400c940 con 0x7fbc840b5a30 2024-04-25T10:05:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.302+0000 7fbc8a153640 1 -- 172.21.15.16:0/1939183598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc841504e0 con 0x7fbc840b5a30 2024-04-25T10:05:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.302+0000 7fbc6bfff640 1 -- 172.21.15.16:0/1939183598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7400cd30 con 0x7fbc840b5a30 2024-04-25T10:05:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.302+0000 7fbc8a153640 1 -- 172.21.15.16:0/1939183598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc84150aa0 con 0x7fbc840b5a30 2024-04-25T10:05:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.303+0000 7fbc8a153640 1 -- 172.21.15.16:0/1939183598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc840750b0 con 0x7fbc840b5a30 2024-04-25T10:05:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.303+0000 7fbc6bfff640 1 -- 172.21.15.16:0/1939183598 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc7401a070 con 0x7fbc840b5a30 2024-04-25T10:05:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.304+0000 7fbc6bfff640 1 --2- 172.21.15.16:0/1939183598 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbc7006b4d0 0x7fbc7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.304+0000 7fbc6bfff640 1 -- 172.21.15.16:0/1939183598 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc7408a0b0 con 0x7fbc840b5a30 2024-04-25T10:05:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.304+0000 7fbc82ffd640 1 --2- 172.21.15.16:0/1939183598 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbc7006b4d0 0x7fbc7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.305+0000 7fbc82ffd640 1 --2- 172.21.15.16:0/1939183598 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbc7006b4d0 0x7fbc7006d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbc78034000 tx=0x7fbc7802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.308+0000 7fbc6bfff640 1 -- 172.21.15.16:0/1939183598 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc74058840 con 0x7fbc840b5a30 2024-04-25T10:05:41.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.486+0000 7fbc8a153640 1 -- 172.21.15.16:0/1939183598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbc840d1780 con 0x7fbc840b5a30 2024-04-25T10:05:41.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.486+0000 7fbc6bfff640 1 -- 172.21.15.16:0/1939183598 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbc7405c8e0 con 0x7fbc840b5a30 2024-04-25T10:05:41.492 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:05:41.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.488+0000 7fbc8a153640 1 -- 172.21.15.16:0/1939183598 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbc7006b4d0 msgr2=0x7fbc7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.488+0000 7fbc8a153640 1 --2- 172.21.15.16:0/1939183598 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbc7006b4d0 0x7fbc7006d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbc78034000 tx=0x7fbc7802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:41.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.488+0000 7fbc8a153640 1 -- 172.21.15.16:0/1939183598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc840b5a30 msgr2=0x7fbc8415c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.488+0000 7fbc8a153640 1 --2- 172.21.15.16:0/1939183598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc840b5a30 0x7fbc8415c760 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400e870 tx=0x7fbc7400ed30 comp rx=0 tx=0).stop 2024-04-25T10:05:41.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.488+0000 7fbc8a153640 1 -- 172.21.15.16:0/1939183598 shutdown_connections 2024-04-25T10:05:41.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.488+0000 7fbc8a153640 1 --2- 172.21.15.16:0/1939183598 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbc7006b4d0 0x7fbc7006d990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.488+0000 7fbc8a153640 1 --2- 172.21.15.16:0/1939183598 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc8415dc80 0x7fbc8415e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.488+0000 7fbc8a153640 1 --2- 172.21.15.16:0/1939183598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc840b97c0 0x7fbc8414fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.488+0000 7fbc8a153640 1 --2- 172.21.15.16:0/1939183598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc840b5a30 0x7fbc8415c760 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.489+0000 7fbc8a153640 1 -- 172.21.15.16:0/1939183598 >> 172.21.15.16:0/1939183598 conn(0x7fbc840d04f0 msgr2=0x7fbc8406bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.489+0000 7fbc8a153640 1 -- 172.21.15.16:0/1939183598 shutdown_connections 2024-04-25T10:05:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.489+0000 7fbc8a153640 1 -- 172.21.15.16:0/1939183598 wait complete. 2024-04-25T10:05:41.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.603+0000 7f6a2ceb3640 1 Processor -- start 2024-04-25T10:05:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.603+0000 7f6a2ceb3640 1 -- start start 2024-04-25T10:05:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.604+0000 7f6a2ceb3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6a2815dc80 0x7f6a28161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.604+0000 7f6a2ceb3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6a280b97c0 0x7f6a280b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.604+0000 7f6a2ceb3640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a280b5e40 con 0x7f6a280b5a30 2024-04-25T10:05:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.604+0000 7f6a2ceb3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a281627a0 con 0x7f6a2815dc80 2024-04-25T10:05:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.604+0000 7f6a2ceb3640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a281621e0 con 0x7f6a280b97c0 2024-04-25T10:05:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.604+0000 7f6a2659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6a2815dc80 0x7f6a28161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.604+0000 7f6a2659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6a2815dc80 0x7f6a28161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37700/0 (socket says 172.21.15.16:37700) 2024-04-25T10:05:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.604+0000 7f6a2659c640 1 -- 172.21.15.16:0/1928227156 learned_addr learned my addr 172.21.15.16:0/1928227156 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.605+0000 7f6a2659c640 1 -- 172.21.15.16:0/1928227156 >> v2:172.21.15.186:3300/0 conn(0x7f6a280b97c0 msgr2=0x7f6a280b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.605+0000 7f6a25d9b640 1 --2- 172.21.15.16:0/1928227156 >> v2:172.21.15.186:3300/0 conn(0x7f6a280b97c0 0x7f6a280b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.605+0000 7f6a2659c640 1 --2- 172.21.15.16:0/1928227156 >> v2:172.21.15.186:3300/0 conn(0x7f6a280b97c0 0x7f6a280b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.605+0000 7f6a2659c640 1 -- 172.21.15.16:0/1928227156 >> v1:172.21.15.186:6789/0 conn(0x7f6a280b5a30 legacy=0x7f6a2806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.605+0000 7f6a2659c640 1 -- 172.21.15.16:0/1928227156 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a281624c0 con 0x7f6a2815dc80 2024-04-25T10:05:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.605+0000 7f6a25d9b640 1 --2- 172.21.15.16:0/1928227156 >> v2:172.21.15.186:3300/0 conn(0x7f6a280b97c0 0x7f6a280b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.606+0000 7f6a2659c640 1 --2- 172.21.15.16:0/1928227156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a2815dc80 0x7f6a28161870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6a2806ba70 tx=0x7f6a2002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba78bf67b10666e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.606+0000 7f6a2559a640 1 -- 172.21.15.16:0/1928227156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20037070 con 0x7f6a2815dc80 2024-04-25T10:05:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.606+0000 7f6a2559a640 1 -- 172.21.15.16:0/1928227156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a20032440 con 0x7f6a2815dc80 2024-04-25T10:05:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.606+0000 7f6a2559a640 1 -- 172.21.15.16:0/1928227156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20032810 con 0x7f6a2815dc80 2024-04-25T10:05:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.606+0000 7f6a2ceb3640 1 -- 172.21.15.16:0/1928227156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a2815dc80 msgr2=0x7f6a28161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.606+0000 7f6a2ceb3640 1 --2- 172.21.15.16:0/1928227156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a2815dc80 0x7f6a28161870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6a2806ba70 tx=0x7f6a2002a230 comp rx=0 tx=0).stop 2024-04-25T10:05:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.606+0000 7f6a2ceb3640 1 -- 172.21.15.16:0/1928227156 shutdown_connections 2024-04-25T10:05:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.606+0000 7f6a2ceb3640 1 --2- 172.21.15.16:0/1928227156 >> v2:172.21.15.186:3300/0 conn(0x7f6a280b97c0 0x7f6a280b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.606+0000 7f6a2ceb3640 1 --2- 172.21.15.16:0/1928227156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a2815dc80 0x7f6a28161870 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.606+0000 7f6a2ceb3640 1 -- 172.21.15.16:0/1928227156 >> 172.21.15.16:0/1928227156 conn(0x7f6a280d04f0 msgr2=0x7f6a280d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.606+0000 7f6a2ceb3640 1 -- 172.21.15.16:0/1928227156 shutdown_connections 2024-04-25T10:05:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.606+0000 7f6a2ceb3640 1 -- 172.21.15.16:0/1928227156 wait complete. 2024-04-25T10:05:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.608+0000 7f6a2ceb3640 1 Processor -- start 2024-04-25T10:05:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.608+0000 7f6a2ceb3640 1 -- start start 2024-04-25T10:05:41.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.608+0000 7f6a2ceb3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a280b5a30 0x7f6a280cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.608+0000 7f6a2ceb3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a280b97c0 0x7f6a2815ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.608+0000 7f6a2ceb3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a2815dc80 0x7f6a280c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.608+0000 7f6a2ceb3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a28161db0 con 0x7f6a280b5a30 2024-04-25T10:05:41.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.608+0000 7f6a2ceb3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2815e060 con 0x7f6a2815dc80 2024-04-25T10:05:41.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.608+0000 7f6a2ceb3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a28161f30 con 0x7f6a280b97c0 2024-04-25T10:05:41.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.609+0000 7f6a2659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a280b5a30 0x7f6a280cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.609+0000 7f6a2659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a280b5a30 0x7f6a280cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37708/0 (socket says 172.21.15.16:37708) 2024-04-25T10:05:41.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.609+0000 7f6a2659c640 1 -- 172.21.15.16:0/4120031946 learned_addr learned my addr 172.21.15.16:0/4120031946 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:41.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.609+0000 7f6a25d9b640 1 --2- 172.21.15.16:0/4120031946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a280b97c0 0x7f6a2815ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.609+0000 7f6a26d9d640 1 --2- 172.21.15.16:0/4120031946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a2815dc80 0x7f6a280c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.609+0000 7f6a2659c640 1 -- 172.21.15.16:0/4120031946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a280b97c0 msgr2=0x7f6a2815ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.609+0000 7f6a2659c640 1 --2- 172.21.15.16:0/4120031946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a280b97c0 0x7f6a2815ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.609+0000 7f6a2659c640 1 -- 172.21.15.16:0/4120031946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a2815dc80 msgr2=0x7f6a280c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.609+0000 7f6a2659c640 1 --2- 172.21.15.16:0/4120031946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a2815dc80 0x7f6a280c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.609+0000 7f6a2659c640 1 -- 172.21.15.16:0/4120031946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a281588c0 con 0x7f6a280b5a30 2024-04-25T10:05:41.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.609+0000 7f6a2659c640 1 --2- 172.21.15.16:0/4120031946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a280b5a30 0x7f6a280cab40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6a20000c00 tx=0x7f6a200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.609+0000 7f6a1effd640 1 -- 172.21.15.16:0/4120031946 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20041020 con 0x7f6a280b5a30 2024-04-25T10:05:41.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.609+0000 7f6a1effd640 1 -- 172.21.15.16:0/4120031946 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a20030970 con 0x7f6a280b5a30 2024-04-25T10:05:41.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.609+0000 7f6a1effd640 1 -- 172.21.15.16:0/4120031946 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20037040 con 0x7f6a280b5a30 2024-04-25T10:05:41.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.609+0000 7f6a2ceb3640 1 -- 172.21.15.16:0/4120031946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a28158ba0 con 0x7f6a280b5a30 2024-04-25T10:05:41.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.609+0000 7f6a2ceb3640 1 -- 172.21.15.16:0/4120031946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a28159160 con 0x7f6a280b5a30 2024-04-25T10:05:41.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.611+0000 7f6a2ceb3640 1 -- 172.21.15.16:0/4120031946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a280750b0 con 0x7f6a280b5a30 2024-04-25T10:05:41.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.611+0000 7f6a1effd640 1 -- 172.21.15.16:0/4120031946 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a20044020 con 0x7f6a280b5a30 2024-04-25T10:05:41.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.611+0000 7f6a1effd640 1 --2- 172.21.15.16:0/4120031946 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6a0806b500 0x7f6a0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.611+0000 7f6a1effd640 1 -- 172.21.15.16:0/4120031946 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a200aab10 con 0x7f6a280b5a30 2024-04-25T10:05:41.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.612+0000 7f6a25d9b640 1 --2- 172.21.15.16:0/4120031946 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6a0806b500 0x7f6a0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.612+0000 7f6a25d9b640 1 --2- 172.21.15.16:0/4120031946 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6a0806b500 0x7f6a0806d9c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6a280b6810 tx=0x7f6a10008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.616+0000 7f6a1effd640 1 -- 172.21.15.16:0/4120031946 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a2004b030 con 0x7f6a280b5a30 2024-04-25T10:05:41.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.795+0000 7f6a2ceb3640 1 -- 172.21.15.16:0/4120031946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6a280d1780 con 0x7f6a280b5a30 2024-04-25T10:05:41.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.795+0000 7f6a1effd640 1 -- 172.21.15.16:0/4120031946 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6a280b5e40 con 0x7f6a280b5a30 2024-04-25T10:05:41.801 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:05:41.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.797+0000 7f6a2ceb3640 1 -- 172.21.15.16:0/4120031946 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6a0806b500 msgr2=0x7f6a0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.797+0000 7f6a2ceb3640 1 --2- 172.21.15.16:0/4120031946 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6a0806b500 0x7f6a0806d9c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6a280b6810 tx=0x7f6a10008040 comp rx=0 tx=0).stop 2024-04-25T10:05:41.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.797+0000 7f6a2ceb3640 1 -- 172.21.15.16:0/4120031946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a280b5a30 msgr2=0x7f6a280cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.797+0000 7f6a2ceb3640 1 --2- 172.21.15.16:0/4120031946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a280b5a30 0x7f6a280cab40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6a20000c00 tx=0x7f6a200046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.797+0000 7f6a2ceb3640 1 -- 172.21.15.16:0/4120031946 shutdown_connections 2024-04-25T10:05:41.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.797+0000 7f6a2ceb3640 1 --2- 172.21.15.16:0/4120031946 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6a0806b500 0x7f6a0806d9c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.797+0000 7f6a2ceb3640 1 --2- 172.21.15.16:0/4120031946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a2815dc80 0x7f6a280c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.797+0000 7f6a2ceb3640 1 --2- 172.21.15.16:0/4120031946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a280b97c0 0x7f6a2815ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.797+0000 7f6a2ceb3640 1 --2- 172.21.15.16:0/4120031946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a280b5a30 0x7f6a280cab40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.797+0000 7f6a2ceb3640 1 -- 172.21.15.16:0/4120031946 >> 172.21.15.16:0/4120031946 conn(0x7f6a280d04f0 msgr2=0x7f6a280cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:41.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.797+0000 7f6a2ceb3640 1 -- 172.21.15.16:0/4120031946 shutdown_connections 2024-04-25T10:05:41.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.797+0000 7f6a2ceb3640 1 -- 172.21.15.16:0/4120031946 wait complete. 2024-04-25T10:05:41.813 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-25T10:05:41.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.909+0000 7ff2599c4640 1 Processor -- start 2024-04-25T10:05:41.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.909+0000 7ff2599c4640 1 -- start start 2024-04-25T10:05:41.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.910+0000 7ff2599c4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff25415a870 0x7ff2540cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.910+0000 7ff2599c4640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff2540cfc40 0x7ff25415ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.910+0000 7ff2599c4640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2540cdb10 con 0x7ff25406e2e0 2024-04-25T10:05:41.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.910+0000 7ff2599c4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2540ce720 con 0x7ff25415a870 2024-04-25T10:05:41.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.910+0000 7ff2599c4640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2540cddf0 con 0x7ff2540cfc40 2024-04-25T10:05:41.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.910+0000 7ff252ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff25415a870 0x7ff2540cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.910+0000 7ff252ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff25415a870 0x7ff2540cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37710/0 (socket says 172.21.15.16:37710) 2024-04-25T10:05:41.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.910+0000 7ff252ffd640 1 -- 172.21.15.16:0/743660130 learned_addr learned my addr 172.21.15.16:0/743660130 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:41.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.910+0000 7ff252ffd640 1 -- 172.21.15.16:0/743660130 >> v2:172.21.15.186:3300/0 conn(0x7ff2540cfc40 msgr2=0x7ff25415ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:41.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.910+0000 7ff252ffd640 1 --2- 172.21.15.16:0/743660130 >> v2:172.21.15.186:3300/0 conn(0x7ff2540cfc40 0x7ff25415ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.910+0000 7ff252ffd640 1 -- 172.21.15.16:0/743660130 >> v1:172.21.15.186:6789/0 conn(0x7ff25406e2e0 legacy=0x7ff2540cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:41.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.910+0000 7ff252ffd640 1 -- 172.21.15.16:0/743660130 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2540ce0d0 con 0x7ff25415a870 2024-04-25T10:05:41.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.911+0000 7ff252ffd640 1 --2- 172.21.15.16:0/743660130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25415a870 0x7ff2540cebb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff244009870 tx=0x7ff24402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ac8859b9a63052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.911+0000 7ff2527fc640 1 -- 172.21.15.16:0/743660130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244037070 con 0x7ff25415a870 2024-04-25T10:05:41.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.911+0000 7ff2527fc640 1 -- 172.21.15.16:0/743660130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff244032440 con 0x7ff25415a870 2024-04-25T10:05:41.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.911+0000 7ff2527fc640 1 -- 172.21.15.16:0/743660130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244032810 con 0x7ff25415a870 2024-04-25T10:05:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.912+0000 7ff2599c4640 1 -- 172.21.15.16:0/743660130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25415a870 msgr2=0x7ff2540cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.912+0000 7ff2599c4640 1 --2- 172.21.15.16:0/743660130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25415a870 0x7ff2540cebb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff244009870 tx=0x7ff24402a270 comp rx=0 tx=0).stop 2024-04-25T10:05:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.912+0000 7ff2599c4640 1 -- 172.21.15.16:0/743660130 shutdown_connections 2024-04-25T10:05:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.912+0000 7ff2599c4640 1 --2- 172.21.15.16:0/743660130 >> v2:172.21.15.186:3300/0 conn(0x7ff2540cfc40 0x7ff25415ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.912+0000 7ff2599c4640 1 --2- 172.21.15.16:0/743660130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25415a870 0x7ff2540cebb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.912+0000 7ff2599c4640 1 -- 172.21.15.16:0/743660130 >> 172.21.15.16:0/743660130 conn(0x7ff25406aba0 msgr2=0x7ff25406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.912+0000 7ff2599c4640 1 -- 172.21.15.16:0/743660130 shutdown_connections 2024-04-25T10:05:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.912+0000 7ff2599c4640 1 -- 172.21.15.16:0/743660130 wait complete. 2024-04-25T10:05:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.913+0000 7ff2599c4640 1 Processor -- start 2024-04-25T10:05:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.913+0000 7ff2599c4640 1 -- start start 2024-04-25T10:05:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.914+0000 7ff2599c4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff25406e2e0 0x7ff2541f7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.914+0000 7ff2599c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2540cfc40 0x7ff254158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.914+0000 7ff2599c4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff25415a870 0x7ff2541f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.914+0000 7ff2599c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540cd700 con 0x7ff2540cfc40 2024-04-25T10:05:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.914+0000 7ff2599c4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540cd580 con 0x7ff25406e2e0 2024-04-25T10:05:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.914+0000 7ff2599c4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540cd880 con 0x7ff25415a870 2024-04-25T10:05:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.914+0000 7ff24bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2540cfc40 0x7ff254158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.914+0000 7ff24bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2540cfc40 0x7ff254158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37722/0 (socket says 172.21.15.16:37722) 2024-04-25T10:05:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.914+0000 7ff24bfff640 1 -- 172.21.15.16:0/1189224452 learned_addr learned my addr 172.21.15.16:0/1189224452 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.914+0000 7ff252ffd640 1 --2- 172.21.15.16:0/1189224452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff25406e2e0 0x7ff2541f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.914+0000 7ff2537fe640 1 --2- 172.21.15.16:0/1189224452 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff25415a870 0x7ff2541f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.914+0000 7ff24bfff640 1 -- 172.21.15.16:0/1189224452 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff25415a870 msgr2=0x7ff2541f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.914+0000 7ff24bfff640 1 --2- 172.21.15.16:0/1189224452 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff25415a870 0x7ff2541f4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.914+0000 7ff24bfff640 1 -- 172.21.15.16:0/1189224452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff25406e2e0 msgr2=0x7ff2541f7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.914+0000 7ff24bfff640 1 --2- 172.21.15.16:0/1189224452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff25406e2e0 0x7ff2541f7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.914+0000 7ff24bfff640 1 -- 172.21.15.16:0/1189224452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2541f7830 con 0x7ff2540cfc40 2024-04-25T10:05:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.915+0000 7ff24bfff640 1 --2- 172.21.15.16:0/1189224452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2540cfc40 0x7ff254158150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00c410 tx=0x7ff23c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.915+0000 7ff250ff9640 1 -- 172.21.15.16:0/1189224452 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c017070 con 0x7ff2540cfc40 2024-04-25T10:05:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.915+0000 7ff250ff9640 1 -- 172.21.15.16:0/1189224452 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c0035f0 con 0x7ff2540cfc40 2024-04-25T10:05:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.915+0000 7ff2599c4640 1 -- 172.21.15.16:0/1189224452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2541f7b10 con 0x7ff2540cfc40 2024-04-25T10:05:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.915+0000 7ff250ff9640 1 -- 172.21.15.16:0/1189224452 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c003a20 con 0x7ff2540cfc40 2024-04-25T10:05:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.915+0000 7ff2599c4640 1 -- 172.21.15.16:0/1189224452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2541e3330 con 0x7ff2540cfc40 2024-04-25T10:05:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.916+0000 7ff2599c4640 1 -- 172.21.15.16:0/1189224452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2541e3530 con 0x7ff2540cfc40 2024-04-25T10:05:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.916+0000 7ff250ff9640 1 -- 172.21.15.16:0/1189224452 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff23c01a070 con 0x7ff2540cfc40 2024-04-25T10:05:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.916+0000 7ff250ff9640 1 --2- 172.21.15.16:0/1189224452 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff23406b4d0 0x7ff23406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.917+0000 7ff250ff9640 1 -- 172.21.15.16:0/1189224452 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff23c089730 con 0x7ff2540cfc40 2024-04-25T10:05:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.917+0000 7ff252ffd640 1 --2- 172.21.15.16:0/1189224452 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff23406b4d0 0x7ff23406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.917+0000 7ff252ffd640 1 --2- 172.21.15.16:0/1189224452 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff23406b4d0 0x7ff23406d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff244034000 tx=0x7ff24402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:41.921+0000 7ff250ff9640 1 -- 172.21.15.16:0/1189224452 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff2541e3530 con 0x7ff2540cfc40 2024-04-25T10:05:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.167+0000 7ff2599c4640 1 -- 172.21.15.16:0/1189224452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7ff2541f8030 con 0x7ff2540cfc40 2024-04-25T10:05:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.550+0000 7ff250ff9640 1 -- 172.21.15.16:0/1189224452 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v52) v1 ==== 145+0+0 (secure 0 0 0) 0x7ff23c058510 con 0x7ff2540cfc40 2024-04-25T10:05:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.552+0000 7ff2599c4640 1 -- 172.21.15.16:0/1189224452 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff23406b4d0 msgr2=0x7ff23406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.552+0000 7ff2599c4640 1 --2- 172.21.15.16:0/1189224452 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff23406b4d0 0x7ff23406d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff244034000 tx=0x7ff24402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.552+0000 7ff2599c4640 1 -- 172.21.15.16:0/1189224452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2540cfc40 msgr2=0x7ff254158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.552+0000 7ff2599c4640 1 --2- 172.21.15.16:0/1189224452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2540cfc40 0x7ff254158150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00c410 tx=0x7ff23c00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.552+0000 7ff2599c4640 1 -- 172.21.15.16:0/1189224452 shutdown_connections 2024-04-25T10:05:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.552+0000 7ff2599c4640 1 --2- 172.21.15.16:0/1189224452 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff23406b4d0 0x7ff23406d990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.552+0000 7ff2599c4640 1 --2- 172.21.15.16:0/1189224452 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff25415a870 0x7ff2541f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.552+0000 7ff2599c4640 1 --2- 172.21.15.16:0/1189224452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2540cfc40 0x7ff254158150 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.552+0000 7ff2599c4640 1 --2- 172.21.15.16:0/1189224452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff25406e2e0 0x7ff2541f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.553+0000 7ff2599c4640 1 -- 172.21.15.16:0/1189224452 >> 172.21.15.16:0/1189224452 conn(0x7ff25406aba0 msgr2=0x7ff2540b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.553+0000 7ff2599c4640 1 -- 172.21.15.16:0/1189224452 shutdown_connections 2024-04-25T10:05:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.553+0000 7ff2599c4640 1 -- 172.21.15.16:0/1189224452 wait complete. 2024-04-25T10:05:42.570 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:05:42.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.678+0000 7f9645dd5640 1 Processor -- start 2024-04-25T10:05:42.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.678+0000 7f9645dd5640 1 -- start start 2024-04-25T10:05:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.679+0000 7f9645dd5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f964015bb60 0x7f96400d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.679+0000 7f9645dd5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96400cf590 0x7f96400cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.679+0000 7f9645dd5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96400cc820 con 0x7f9640099b20 2024-04-25T10:05:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.679+0000 7f9645dd5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96400cd430 con 0x7f964015bb60 2024-04-25T10:05:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.679+0000 7f9645dd5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96400ccb00 con 0x7f96400cf590 2024-04-25T10:05:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.679+0000 7f963f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f964015bb60 0x7f96400d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.679+0000 7f963ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9640099b20 0x7f96400c5410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41042/0 (socket says 172.21.15.16:41042) 2024-04-25T10:05:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.679+0000 7f963ffff640 1 -- 172.21.15.16:0/4291073061 learned_addr learned my addr 172.21.15.16:0/4291073061 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.680+0000 7f963effd640 1 --2- 172.21.15.16:0/4291073061 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96400cf590 0x7f96400cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.680+0000 7f963f7fe640 1 -- 172.21.15.16:0/4291073061 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96400cf590 msgr2=0x7f96400cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.680+0000 7f963f7fe640 1 --2- 172.21.15.16:0/4291073061 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96400cf590 0x7f96400cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.680+0000 7f963f7fe640 1 -- 172.21.15.16:0/4291073061 >> v1:172.21.15.16:6789/0 conn(0x7f9640099b20 legacy=0x7f96400c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.680+0000 7f963f7fe640 1 -- 172.21.15.16:0/4291073061 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400ccde0 con 0x7f964015bb60 2024-04-25T10:05:42.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.680+0000 7f963e7fc640 1 -- 172.21.15.16:0/4291073061 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806969749 0 0) 0x7f96400cc820 con 0x7f9640099b20 2024-04-25T10:05:42.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.680+0000 7f963f7fe640 1 --2- 172.21.15.16:0/4291073061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964015bb60 0x7f96400d0380 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f96300097a0 tx=0x7f963002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781f2e93addaf63e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.681+0000 7f963e7fc640 1 -- 172.21.15.16:0/4291073061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630037070 con 0x7f964015bb60 2024-04-25T10:05:42.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.681+0000 7f963e7fc640 1 -- 172.21.15.16:0/4291073061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9630032440 con 0x7f964015bb60 2024-04-25T10:05:42.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.681+0000 7f963e7fc640 1 -- 172.21.15.16:0/4291073061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630032810 con 0x7f964015bb60 2024-04-25T10:05:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.681+0000 7f9645dd5640 1 -- 172.21.15.16:0/4291073061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964015bb60 msgr2=0x7f96400d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.681+0000 7f9645dd5640 1 --2- 172.21.15.16:0/4291073061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964015bb60 0x7f96400d0380 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f96300097a0 tx=0x7f963002a180 comp rx=0 tx=0).stop 2024-04-25T10:05:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.681+0000 7f9645dd5640 1 -- 172.21.15.16:0/4291073061 shutdown_connections 2024-04-25T10:05:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.681+0000 7f9645dd5640 1 --2- 172.21.15.16:0/4291073061 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96400cf590 0x7f96400cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.681+0000 7f9645dd5640 1 --2- 172.21.15.16:0/4291073061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964015bb60 0x7f96400d0380 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.681+0000 7f9645dd5640 1 -- 172.21.15.16:0/4291073061 >> 172.21.15.16:0/4291073061 conn(0x7f96400b33b0 msgr2=0x7f96400b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.681+0000 7f9645dd5640 1 -- 172.21.15.16:0/4291073061 shutdown_connections 2024-04-25T10:05:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.681+0000 7f9645dd5640 1 -- 172.21.15.16:0/4291073061 wait complete. 2024-04-25T10:05:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.682+0000 7f9645dd5640 1 Processor -- start 2024-04-25T10:05:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.683+0000 7f9645dd5640 1 -- start start 2024-04-25T10:05:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.683+0000 7f9645dd5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9640099b20 0x7f96401581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.683+0000 7f9645dd5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96400cf590 0x7f96401f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.683+0000 7f9645dd5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964015bb60 0x7f96401f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.683+0000 7f9645dd5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96400cde00 con 0x7f964015bb60 2024-04-25T10:05:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.683+0000 7f9645dd5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96400d08c0 con 0x7f9640099b20 2024-04-25T10:05:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.683+0000 7f9645dd5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96400cc440 con 0x7f96400cf590 2024-04-25T10:05:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.683+0000 7f963ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964015bb60 0x7f96401f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.683+0000 7f963ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964015bb60 0x7f96401f16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37732/0 (socket says 172.21.15.16:37732) 2024-04-25T10:05:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.683+0000 7f963ffff640 1 -- 172.21.15.16:0/1663204387 learned_addr learned my addr 172.21.15.16:0/1663204387 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.684+0000 7f963f7fe640 1 --2- 172.21.15.16:0/1663204387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9640099b20 0x7f96401581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.684+0000 7f963effd640 1 --2- 172.21.15.16:0/1663204387 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96400cf590 0x7f96401f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.684+0000 7f963ffff640 1 -- 172.21.15.16:0/1663204387 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96400cf590 msgr2=0x7f96401f55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.684+0000 7f963ffff640 1 --2- 172.21.15.16:0/1663204387 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96400cf590 0x7f96401f55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.684+0000 7f963ffff640 1 -- 172.21.15.16:0/1663204387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9640099b20 msgr2=0x7f96401581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.684+0000 7f963ffff640 1 --2- 172.21.15.16:0/1663204387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9640099b20 0x7f96401581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.684+0000 7f963ffff640 1 -- 172.21.15.16:0/1663204387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96401f20b0 con 0x7f964015bb60 2024-04-25T10:05:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.684+0000 7f963f7fe640 1 --2- 172.21.15.16:0/1663204387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9640099b20 0x7f96401581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.684+0000 7f963ffff640 1 --2- 172.21.15.16:0/1663204387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964015bb60 0x7f96401f16d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f963800c7a0 tx=0x7f963800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.684+0000 7f9627fff640 1 -- 172.21.15.16:0/1663204387 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9638018070 con 0x7f964015bb60 2024-04-25T10:05:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.684+0000 7f9627fff640 1 -- 172.21.15.16:0/1663204387 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963800a640 con 0x7f964015bb60 2024-04-25T10:05:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.684+0000 7f9645dd5640 1 -- 172.21.15.16:0/1663204387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96401f2390 con 0x7f964015bb60 2024-04-25T10:05:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.684+0000 7f9627fff640 1 -- 172.21.15.16:0/1663204387 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963800aab0 con 0x7f964015bb60 2024-04-25T10:05:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.684+0000 7f9645dd5640 1 -- 172.21.15.16:0/1663204387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96401f5090 con 0x7f964015bb60 2024-04-25T10:05:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.685+0000 7f9645dd5640 1 -- 172.21.15.16:0/1663204387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96401f5290 con 0x7f964015bb60 2024-04-25T10:05:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.686+0000 7f9627fff640 1 -- 172.21.15.16:0/1663204387 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f963801b070 con 0x7f964015bb60 2024-04-25T10:05:42.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.686+0000 7f9627fff640 1 --2- 172.21.15.16:0/1663204387 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f962c06b4d0 0x7f962c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.686+0000 7f9627fff640 1 -- 172.21.15.16:0/1663204387 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96380897c0 con 0x7f964015bb60 2024-04-25T10:05:42.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.686+0000 7f963f7fe640 1 --2- 172.21.15.16:0/1663204387 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f962c06b4d0 0x7f962c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.687+0000 7f963f7fe640 1 --2- 172.21.15.16:0/1663204387 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f962c06b4d0 0x7f962c06d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9630034000 tx=0x7f963002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.690+0000 7f9627fff640 1 -- 172.21.15.16:0/1663204387 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96401f5290 con 0x7f964015bb60 2024-04-25T10:05:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.868+0000 7f9645dd5640 1 -- 172.21.15.16:0/1663204387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f964009afb0 con 0x7f964015bb60 2024-04-25T10:05:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.871+0000 7f9627fff640 1 -- 172.21.15.16:0/1663204387 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f96380586d0 con 0x7f964015bb60 2024-04-25T10:05:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.873+0000 7f9645dd5640 1 -- 172.21.15.16:0/1663204387 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f962c06b4d0 msgr2=0x7f962c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.873+0000 7f9645dd5640 1 --2- 172.21.15.16:0/1663204387 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f962c06b4d0 0x7f962c06d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9630034000 tx=0x7f963002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.873+0000 7f9645dd5640 1 -- 172.21.15.16:0/1663204387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964015bb60 msgr2=0x7f96401f16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.873+0000 7f9645dd5640 1 --2- 172.21.15.16:0/1663204387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964015bb60 0x7f96401f16d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f963800c7a0 tx=0x7f963800cc60 comp rx=0 tx=0).stop 2024-04-25T10:05:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.873+0000 7f9645dd5640 1 -- 172.21.15.16:0/1663204387 shutdown_connections 2024-04-25T10:05:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.873+0000 7f9645dd5640 1 --2- 172.21.15.16:0/1663204387 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f962c06b4d0 0x7f962c06d990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.873+0000 7f9645dd5640 1 --2- 172.21.15.16:0/1663204387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964015bb60 0x7f96401f16d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.873+0000 7f9645dd5640 1 --2- 172.21.15.16:0/1663204387 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96400cf590 0x7f96401f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.873+0000 7f9645dd5640 1 --2- 172.21.15.16:0/1663204387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9640099b20 0x7f96401581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.873+0000 7f9645dd5640 1 -- 172.21.15.16:0/1663204387 >> 172.21.15.16:0/1663204387 conn(0x7f96400b33b0 msgr2=0x7f96400c5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.873+0000 7f9645dd5640 1 -- 172.21.15.16:0/1663204387 shutdown_connections 2024-04-25T10:05:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:42.873+0000 7f9645dd5640 1 -- 172.21.15.16:0/1663204387 wait complete. 2024-04-25T10:05:42.889 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:05:42.890 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:42.937 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:42.977 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:05:42.977 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:05:42.978 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:42.994 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:05:42.994 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:43.049 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:05:43.049 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:43.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:43.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.241+0000 7f01374dd640 1 Processor -- start 2024-04-25T10:05:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.241+0000 7f01374dd640 1 -- start start 2024-04-25T10:05:43.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.242+0000 7f01374dd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f01300b5b60 0x7f01301687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.242+0000 7f01374dd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f013015e0c0 con 0x7f01300d35b0 2024-04-25T10:05:43.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.242+0000 7f01374dd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01300d2510 con 0x7f013015dce0 2024-04-25T10:05:43.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.242+0000 7f01374dd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01300b7b10 con 0x7f01300b5b60 2024-04-25T10:05:43.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.242+0000 7f0135a7a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f01300b5b60 0x7f01301687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.242+0000 7f0135a7a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f01300b5b60 0x7f01301687f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42396/0 (socket says 172.21.15.16:42396) 2024-04-25T10:05:43.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.242+0000 7f0135a7a640 1 -- 172.21.15.16:0/4206297823 learned_addr learned my addr 172.21.15.16:0/4206297823 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:43.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.242+0000 7f0134a78640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f01300d35b0 0x7f01300b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51392/0 (socket says 172.21.15.16:51392) 2024-04-25T10:05:43.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.242+0000 7f0135a7a640 1 -- 172.21.15.16:0/4206297823 >> v1:172.21.15.186:6789/0 conn(0x7f013015dce0 legacy=0x7f01300d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.242+0000 7f0135a7a640 1 -- 172.21.15.16:0/4206297823 >> v1:172.21.15.16:6789/0 conn(0x7f01300d35b0 legacy=0x7f01300b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.242+0000 7f0135a7a640 1 -- 172.21.15.16:0/4206297823 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01300d27f0 con 0x7f01300b5b60 2024-04-25T10:05:43.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.243+0000 7f0135a7a640 1 --2- 172.21.15.16:0/4206297823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01300b5b60 0x7f01301687f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0128009e50 tx=0x7f012802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c924268da980e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.243+0000 7f0127fff640 1 -- 172.21.15.16:0/4206297823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0128036070 con 0x7f01300b5b60 2024-04-25T10:05:43.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.243+0000 7f0127fff640 1 -- 172.21.15.16:0/4206297823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0128031600 con 0x7f01300b5b60 2024-04-25T10:05:43.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.243+0000 7f0127fff640 1 -- 172.21.15.16:0/4206297823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01280319d0 con 0x7f01300b5b60 2024-04-25T10:05:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.244+0000 7f01374dd640 1 -- 172.21.15.16:0/4206297823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01300b5b60 msgr2=0x7f01301687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.244+0000 7f01374dd640 1 --2- 172.21.15.16:0/4206297823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01300b5b60 0x7f01301687f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0128009e50 tx=0x7f012802a760 comp rx=0 tx=0).stop 2024-04-25T10:05:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.244+0000 7f01374dd640 1 -- 172.21.15.16:0/4206297823 shutdown_connections 2024-04-25T10:05:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.244+0000 7f01374dd640 1 --2- 172.21.15.16:0/4206297823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01300b5b60 0x7f01301687f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.244+0000 7f01374dd640 1 -- 172.21.15.16:0/4206297823 >> 172.21.15.16:0/4206297823 conn(0x7f0130072900 msgr2=0x7f01300d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.244+0000 7f01374dd640 1 -- 172.21.15.16:0/4206297823 shutdown_connections 2024-04-25T10:05:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.244+0000 7f01374dd640 1 -- 172.21.15.16:0/4206297823 wait complete. 2024-04-25T10:05:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.245+0000 7f01374dd640 1 Processor -- start 2024-04-25T10:05:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.245+0000 7f01374dd640 1 -- start start 2024-04-25T10:05:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f01374dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01300b5b60 0x7f013015ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f01374dd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01300d35b0 0x7f013014ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f01374dd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f013015dce0 0x7f0130160740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f01374dd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01300d3090 con 0x7f01300b5b60 2024-04-25T10:05:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f01374dd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01300d2db0 con 0x7f01300d35b0 2024-04-25T10:05:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f01374dd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01300d2160 con 0x7f013015dce0 2024-04-25T10:05:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f0135279640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01300b5b60 0x7f013015ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f0135279640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01300b5b60 0x7f013015ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42412/0 (socket says 172.21.15.16:42412) 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f0135279640 1 -- 172.21.15.16:0/1635228477 learned_addr learned my addr 172.21.15.16:0/1635228477 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f0134a78640 1 --2- 172.21.15.16:0/1635228477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01300d35b0 0x7f013014ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f0135a7a640 1 --2- 172.21.15.16:0/1635228477 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f013015dce0 0x7f0130160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f0135279640 1 -- 172.21.15.16:0/1635228477 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f013015dce0 msgr2=0x7f0130160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f0135279640 1 --2- 172.21.15.16:0/1635228477 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f013015dce0 0x7f0130160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f0135279640 1 -- 172.21.15.16:0/1635228477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01300d35b0 msgr2=0x7f013014ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f0135279640 1 --2- 172.21.15.16:0/1635228477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01300d35b0 0x7f013014ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f0135279640 1 -- 172.21.15.16:0/1635228477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013014d340 con 0x7f01300b5b60 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f0134a78640 1 --2- 172.21.15.16:0/1635228477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01300d35b0 0x7f013014ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f0135a7a640 1 --2- 172.21.15.16:0/1635228477 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f013015dce0 0x7f0130160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f0135279640 1 --2- 172.21.15.16:0/1635228477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01300b5b60 0x7f013015ec30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f012c00b3e0 tx=0x7f012c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f0125ffb640 1 -- 172.21.15.16:0/1635228477 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c015020 con 0x7f01300b5b60 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f0125ffb640 1 -- 172.21.15.16:0/1635228477 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c011850 con 0x7f01300b5b60 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f01374dd640 1 -- 172.21.15.16:0/1635228477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01300cc6a0 con 0x7f01300b5b60 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.246+0000 7f0125ffb640 1 -- 172.21.15.16:0/1635228477 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c011c40 con 0x7f01300b5b60 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.247+0000 7f01374dd640 1 -- 172.21.15.16:0/1635228477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f013015f2a0 con 0x7f01300b5b60 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.248+0000 7f01374dd640 1 -- 172.21.15.16:0/1635228477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01300749b0 con 0x7f01300b5b60 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.248+0000 7f0125ffb640 1 -- 172.21.15.16:0/1635228477 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f012c018020 con 0x7f01300b5b60 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.248+0000 7f0125ffb640 1 --2- 172.21.15.16:0/1635228477 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f011406b4d0 0x7f011406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.248+0000 7f0125ffb640 1 -- 172.21.15.16:0/1635228477 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f012c08a060 con 0x7f01300b5b60 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.249+0000 7f0134a78640 1 --2- 172.21.15.16:0/1635228477 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f011406b4d0 0x7f011406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.249+0000 7f0134a78640 1 --2- 172.21.15.16:0/1635228477 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f011406b4d0 0x7f011406d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0120007fd0 tx=0x7f0120007d60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.252+0000 7f0125ffb640 1 -- 172.21.15.16:0/1635228477 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f012c01f020 con 0x7f01300b5b60 2024-04-25T10:05:43.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.433+0000 7f01374dd640 1 -- 172.21.15.16:0/1635228477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f013015e550 con 0x7f01300b5b60 2024-04-25T10:05:43.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.434+0000 7f0125ffb640 1 -- 172.21.15.16:0/1635228477 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f013014d340 con 0x7f01300b5b60 2024-04-25T10:05:43.440 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:43.440 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:12.449183+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:43.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.436+0000 7f01374dd640 1 -- 172.21.15.16:0/1635228477 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f011406b4d0 msgr2=0x7f011406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.436+0000 7f01374dd640 1 --2- 172.21.15.16:0/1635228477 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f011406b4d0 0x7f011406d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0120007fd0 tx=0x7f0120007d60 comp rx=0 tx=0).stop 2024-04-25T10:05:43.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.436+0000 7f01374dd640 1 -- 172.21.15.16:0/1635228477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01300b5b60 msgr2=0x7f013015ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.436+0000 7f01374dd640 1 --2- 172.21.15.16:0/1635228477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01300b5b60 0x7f013015ec30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f012c00b3e0 tx=0x7f012c00b8a0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.436+0000 7f01374dd640 1 -- 172.21.15.16:0/1635228477 shutdown_connections 2024-04-25T10:05:43.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.436+0000 7f01374dd640 1 --2- 172.21.15.16:0/1635228477 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f011406b4d0 0x7f011406d990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.436+0000 7f01374dd640 1 --2- 172.21.15.16:0/1635228477 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f013015dce0 0x7f0130160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.436+0000 7f01374dd640 1 --2- 172.21.15.16:0/1635228477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01300d35b0 0x7f013014ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.436+0000 7f01374dd640 1 --2- 172.21.15.16:0/1635228477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01300b5b60 0x7f013015ec30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.436+0000 7f01374dd640 1 -- 172.21.15.16:0/1635228477 >> 172.21.15.16:0/1635228477 conn(0x7f0130072900 msgr2=0x7f01300ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.436+0000 7f01374dd640 1 -- 172.21.15.16:0/1635228477 shutdown_connections 2024-04-25T10:05:43.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.436+0000 7f01374dd640 1 -- 172.21.15.16:0/1635228477 wait complete. 2024-04-25T10:05:43.452 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:08.724537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:09.793105+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:43.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:43.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.554+0000 7f16c8a4d640 1 Processor -- start 2024-04-25T10:05:43.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.554+0000 7f16c8a4d640 1 -- start start 2024-04-25T10:05:43.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.555+0000 7f16c8a4d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f16c40ad000 0x7f16c40af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.555+0000 7f16c8a4d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16c413f140 0x7f16c40ad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.555+0000 7f16c8a4d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c40af720 con 0x7f16c4144bf0 2024-04-25T10:05:43.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.555+0000 7f16c8a4d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c40ad920 con 0x7f16c413f140 2024-04-25T10:05:43.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.555+0000 7f16c8a4d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c4160010 con 0x7f16c40ad000 2024-04-25T10:05:43.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.555+0000 7f16bbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16c413f140 0x7f16c40ad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.555+0000 7f16bbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16c413f140 0x7f16c40ad3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42422/0 (socket says 172.21.15.16:42422) 2024-04-25T10:05:43.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.555+0000 7f16bbfff640 1 -- 172.21.15.16:0/2020145143 learned_addr learned my addr 172.21.15.16:0/2020145143 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.555+0000 7f16c259c640 1 --2- 172.21.15.16:0/2020145143 >> v2:172.21.15.186:3300/0 conn(0x7f16c40ad000 0x7f16c40af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.555+0000 7f16bbfff640 1 -- 172.21.15.16:0/2020145143 >> v2:172.21.15.186:3300/0 conn(0x7f16c40ad000 msgr2=0x7f16c40af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.555+0000 7f16bbfff640 1 --2- 172.21.15.16:0/2020145143 >> v2:172.21.15.186:3300/0 conn(0x7f16c40ad000 0x7f16c40af1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.555+0000 7f16bbfff640 1 -- 172.21.15.16:0/2020145143 >> v1:172.21.15.186:6789/0 conn(0x7f16c4144bf0 legacy=0x7f16c406b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.555+0000 7f16bbfff640 1 -- 172.21.15.16:0/2020145143 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c41602f0 con 0x7f16c413f140 2024-04-25T10:05:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.555+0000 7f16c259c640 1 --2- 172.21.15.16:0/2020145143 >> v2:172.21.15.186:3300/0 conn(0x7f16c40ad000 0x7f16c40af1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.556+0000 7f16bbfff640 1 --2- 172.21.15.16:0/2020145143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16c413f140 0x7f16c40ad3e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f16c4066ec0 tx=0x7f16ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0c4de05d6c79c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.556+0000 7f16c1d9b640 1 -- 172.21.15.16:0/2020145143 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac037070 con 0x7f16c413f140 2024-04-25T10:05:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.556+0000 7f16c1d9b640 1 -- 172.21.15.16:0/2020145143 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ac032440 con 0x7f16c413f140 2024-04-25T10:05:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.556+0000 7f16c1d9b640 1 -- 172.21.15.16:0/2020145143 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac032810 con 0x7f16c413f140 2024-04-25T10:05:43.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.556+0000 7f16c8a4d640 1 -- 172.21.15.16:0/2020145143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16c413f140 msgr2=0x7f16c40ad3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.556+0000 7f16c8a4d640 1 --2- 172.21.15.16:0/2020145143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16c413f140 0x7f16c40ad3e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f16c4066ec0 tx=0x7f16ac02a270 comp rx=0 tx=0).stop 2024-04-25T10:05:43.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.556+0000 7f16c8a4d640 1 -- 172.21.15.16:0/2020145143 shutdown_connections 2024-04-25T10:05:43.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.556+0000 7f16c8a4d640 1 --2- 172.21.15.16:0/2020145143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16c413f140 0x7f16c40ad3e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.556+0000 7f16c8a4d640 1 --2- 172.21.15.16:0/2020145143 >> v2:172.21.15.186:3300/0 conn(0x7f16c40ad000 0x7f16c40af1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.556+0000 7f16c8a4d640 1 -- 172.21.15.16:0/2020145143 >> 172.21.15.16:0/2020145143 conn(0x7f16c40cff20 msgr2=0x7f16c40d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.557+0000 7f16c8a4d640 1 -- 172.21.15.16:0/2020145143 shutdown_connections 2024-04-25T10:05:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.557+0000 7f16c8a4d640 1 -- 172.21.15.16:0/2020145143 wait complete. 2024-04-25T10:05:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.558+0000 7f16c8a4d640 1 Processor -- start 2024-04-25T10:05:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.558+0000 7f16c8a4d640 1 -- start start 2024-04-25T10:05:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16c8a4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16c40ad000 0x7f16c409b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16c8a4d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c413f140 0x7f16c406f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16c8a4d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f16c4144bf0 0x7f16c4140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16c8a4d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c406b200 con 0x7f16c40ad000 2024-04-25T10:05:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16c8a4d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c406af20 con 0x7f16c413f140 2024-04-25T10:05:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16c8a4d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c4141930 con 0x7f16c4144bf0 2024-04-25T10:05:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16c259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16c40ad000 0x7f16c409b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16c259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16c40ad000 0x7f16c409b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42424/0 (socket says 172.21.15.16:42424) 2024-04-25T10:05:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16c259c640 1 -- 172.21.15.16:0/313609555 learned_addr learned my addr 172.21.15.16:0/313609555 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16bbfff640 1 --2- 172.21.15.16:0/313609555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c413f140 0x7f16c406f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16c2d9d640 1 --2- 172.21.15.16:0/313609555 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f16c4144bf0 0x7f16c4140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16c259c640 1 -- 172.21.15.16:0/313609555 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f16c4144bf0 msgr2=0x7f16c4140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16c259c640 1 --2- 172.21.15.16:0/313609555 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f16c4144bf0 0x7f16c4140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16c259c640 1 -- 172.21.15.16:0/313609555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c413f140 msgr2=0x7f16c406f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16c259c640 1 --2- 172.21.15.16:0/313609555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c413f140 0x7f16c406f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16c259c640 1 -- 172.21.15.16:0/313609555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c4099be0 con 0x7f16c40ad000 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16c259c640 1 --2- 172.21.15.16:0/313609555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16c40ad000 0x7f16c409b410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f16b400d7d0 tx=0x7f16b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16baffd640 1 -- 172.21.15.16:0/313609555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b4004030 con 0x7f16c40ad000 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16baffd640 1 -- 172.21.15.16:0/313609555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b40041d0 con 0x7f16c40ad000 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16c8a4d640 1 -- 172.21.15.16:0/313609555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c41f7ab0 con 0x7f16c40ad000 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16baffd640 1 -- 172.21.15.16:0/313609555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b400b6c0 con 0x7f16c40ad000 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.559+0000 7f16c8a4d640 1 -- 172.21.15.16:0/313609555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c4099ec0 con 0x7f16c40ad000 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.560+0000 7f16c8a4d640 1 -- 172.21.15.16:0/313609555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c41f7e90 con 0x7f16c40ad000 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.561+0000 7f16baffd640 1 -- 172.21.15.16:0/313609555 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16b400b860 con 0x7f16c40ad000 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.561+0000 7f16baffd640 1 --2- 172.21.15.16:0/313609555 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f16a406b4d0 0x7f16a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.561+0000 7f16baffd640 1 -- 172.21.15.16:0/313609555 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16b408a4c0 con 0x7f16c40ad000 2024-04-25T10:05:43.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.562+0000 7f16bbfff640 1 --2- 172.21.15.16:0/313609555 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f16a406b4d0 0x7f16a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.562+0000 7f16bbfff640 1 --2- 172.21.15.16:0/313609555 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f16a406b4d0 0x7f16a406d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f16ac034000 tx=0x7f16ac02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.565+0000 7f16baffd640 1 -- 172.21.15.16:0/313609555 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16c41f7e90 con 0x7f16c40ad000 2024-04-25T10:05:43.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.745+0000 7f16c8a4d640 1 -- 172.21.15.16:0/313609555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16c406f3a0 con 0x7f16c40ad000 2024-04-25T10:05:43.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.745+0000 7f16baffd640 1 -- 172.21.15.16:0/313609555 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16b4059340 con 0x7f16c40ad000 2024-04-25T10:05:43.750 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:05:43.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.747+0000 7f16c8a4d640 1 -- 172.21.15.16:0/313609555 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f16a406b4d0 msgr2=0x7f16a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.747+0000 7f16c8a4d640 1 --2- 172.21.15.16:0/313609555 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f16a406b4d0 0x7f16a406d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f16ac034000 tx=0x7f16ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:43.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.747+0000 7f16c8a4d640 1 -- 172.21.15.16:0/313609555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16c40ad000 msgr2=0x7f16c409b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.747+0000 7f16c8a4d640 1 --2- 172.21.15.16:0/313609555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16c40ad000 0x7f16c409b410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f16b400d7d0 tx=0x7f16b400dc90 comp rx=0 tx=0).stop 2024-04-25T10:05:43.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.747+0000 7f16c8a4d640 1 -- 172.21.15.16:0/313609555 shutdown_connections 2024-04-25T10:05:43.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.747+0000 7f16c8a4d640 1 --2- 172.21.15.16:0/313609555 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f16a406b4d0 0x7f16a406d990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.747+0000 7f16c8a4d640 1 --2- 172.21.15.16:0/313609555 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f16c4144bf0 0x7f16c4140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.747+0000 7f16c8a4d640 1 --2- 172.21.15.16:0/313609555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c413f140 0x7f16c406f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.747+0000 7f16c8a4d640 1 --2- 172.21.15.16:0/313609555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16c40ad000 0x7f16c409b410 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.747+0000 7f16c8a4d640 1 -- 172.21.15.16:0/313609555 >> 172.21.15.16:0/313609555 conn(0x7f16c40cff20 msgr2=0x7f16c40b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.747+0000 7f16c8a4d640 1 -- 172.21.15.16:0/313609555 shutdown_connections 2024-04-25T10:05:43.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.747+0000 7f16c8a4d640 1 -- 172.21.15.16:0/313609555 wait complete. 2024-04-25T10:05:43.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.862+0000 7f7b783e0640 1 Processor -- start 2024-04-25T10:05:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.862+0000 7f7b783e0640 1 -- start start 2024-04-25T10:05:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.863+0000 7f7b783e0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b7015dbb0 0x7f7b700b9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.863+0000 7f7b783e0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7b700b5c70 0x7f7b700b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.863+0000 7f7b783e0640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b700d2670 con 0x7f7b700d1e20 2024-04-25T10:05:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.863+0000 7f7b783e0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b700d2950 con 0x7f7b7015dbb0 2024-04-25T10:05:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.863+0000 7f7b783e0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b700d2c30 con 0x7f7b700b5c70 2024-04-25T10:05:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.863+0000 7f7b7617c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b7015dbb0 0x7f7b700b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.863+0000 7f7b7617c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b7015dbb0 0x7f7b700b9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42428/0 (socket says 172.21.15.16:42428) 2024-04-25T10:05:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.863+0000 7f7b7617c640 1 -- 172.21.15.16:0/582177043 learned_addr learned my addr 172.21.15.16:0/582177043 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.863+0000 7f7b7697d640 1 --2- 172.21.15.16:0/582177043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7b700b5c70 0x7f7b700b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.864+0000 7f7b7617c640 1 -- 172.21.15.16:0/582177043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7b700b5c70 msgr2=0x7f7b700b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.864+0000 7f7b7617c640 1 --2- 172.21.15.16:0/582177043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7b700b5c70 0x7f7b700b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.864+0000 7f7b7617c640 1 -- 172.21.15.16:0/582177043 >> v1:172.21.15.186:6789/0 conn(0x7f7b700d1e20 legacy=0x7f7b700b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.864+0000 7f7b7617c640 1 -- 172.21.15.16:0/582177043 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b700d2f10 con 0x7f7b7015dbb0 2024-04-25T10:05:43.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.864+0000 7f7b7617c640 1 --2- 172.21.15.16:0/582177043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b7015dbb0 0x7f7b700b9720 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7b60009e50 tx=0x7f7b6002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bc687687f09fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.864+0000 7f7b7517a640 1 -- 172.21.15.16:0/582177043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b60036070 con 0x7f7b7015dbb0 2024-04-25T10:05:43.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.864+0000 7f7b7517a640 1 -- 172.21.15.16:0/582177043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b60031600 con 0x7f7b7015dbb0 2024-04-25T10:05:43.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.864+0000 7f7b7517a640 1 -- 172.21.15.16:0/582177043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b600319d0 con 0x7f7b7015dbb0 2024-04-25T10:05:43.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.865+0000 7f7b783e0640 1 -- 172.21.15.16:0/582177043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b7015dbb0 msgr2=0x7f7b700b9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.865+0000 7f7b783e0640 1 --2- 172.21.15.16:0/582177043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b7015dbb0 0x7f7b700b9720 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7b60009e50 tx=0x7f7b6002a710 comp rx=0 tx=0).stop 2024-04-25T10:05:43.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.865+0000 7f7b783e0640 1 -- 172.21.15.16:0/582177043 shutdown_connections 2024-04-25T10:05:43.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.865+0000 7f7b783e0640 1 --2- 172.21.15.16:0/582177043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7b700b5c70 0x7f7b700b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.865+0000 7f7b783e0640 1 --2- 172.21.15.16:0/582177043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b7015dbb0 0x7f7b700b9720 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.865+0000 7f7b783e0640 1 -- 172.21.15.16:0/582177043 >> 172.21.15.16:0/582177043 conn(0x7f7b700d0560 msgr2=0x7f7b700d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.865+0000 7f7b783e0640 1 -- 172.21.15.16:0/582177043 shutdown_connections 2024-04-25T10:05:43.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.865+0000 7f7b783e0640 1 -- 172.21.15.16:0/582177043 wait complete. 2024-04-25T10:05:43.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.866+0000 7f7b783e0640 1 Processor -- start 2024-04-25T10:05:43.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.866+0000 7f7b783e0640 1 -- start start 2024-04-25T10:05:43.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.867+0000 7f7b783e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b700b5c70 0x7f7b701e4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.867+0000 7f7b783e0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7b700d1e20 0x7f7b70156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.867+0000 7f7b783e0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b7015dbb0 0x7f7b700b7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.867+0000 7f7b783e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7015e110 con 0x7f7b700b5c70 2024-04-25T10:05:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.867+0000 7f7b783e0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7015df90 con 0x7f7b7015dbb0 2024-04-25T10:05:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.867+0000 7f7b783e0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b700d2290 con 0x7f7b700d1e20 2024-04-25T10:05:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.867+0000 7f7b7617c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b700b5c70 0x7f7b701e4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.867+0000 7f7b7617c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b700b5c70 0x7f7b701e4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42434/0 (socket says 172.21.15.16:42434) 2024-04-25T10:05:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.867+0000 7f7b7617c640 1 -- 172.21.15.16:0/472574110 learned_addr learned my addr 172.21.15.16:0/472574110 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.867+0000 7f7b7617c640 1 -- 172.21.15.16:0/472574110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7b700d1e20 msgr2=0x7f7b70156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.867+0000 7f7b7617c640 1 --2- 172.21.15.16:0/472574110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7b700d1e20 0x7f7b70156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.867+0000 7f7b7617c640 1 -- 172.21.15.16:0/472574110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b7015dbb0 msgr2=0x7f7b700b7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.867+0000 7f7b7697d640 1 --2- 172.21.15.16:0/472574110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b7015dbb0 0x7f7b700b7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.867+0000 7f7b7617c640 1 --2- 172.21.15.16:0/472574110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b7015dbb0 0x7f7b700b7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.867+0000 7f7b7617c640 1 -- 172.21.15.16:0/472574110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b701e49c0 con 0x7f7b700b5c70 2024-04-25T10:05:43.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.868+0000 7f7b7617c640 1 --2- 172.21.15.16:0/472574110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b700b5c70 0x7f7b701e4480 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7b60034040 tx=0x7f7b60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.868+0000 7f7b6effd640 1 -- 172.21.15.16:0/472574110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b60036050 con 0x7f7b700b5c70 2024-04-25T10:05:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.868+0000 7f7b6effd640 1 -- 172.21.15.16:0/472574110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b60042070 con 0x7f7b700b5c70 2024-04-25T10:05:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.868+0000 7f7b783e0640 1 -- 172.21.15.16:0/472574110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b70156c10 con 0x7f7b700b5c70 2024-04-25T10:05:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.868+0000 7f7b6effd640 1 -- 172.21.15.16:0/472574110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b60009510 con 0x7f7b700b5c70 2024-04-25T10:05:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.868+0000 7f7b783e0640 1 -- 172.21.15.16:0/472574110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b701571d0 con 0x7f7b700b5c70 2024-04-25T10:05:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.869+0000 7f7b783e0640 1 -- 172.21.15.16:0/472574110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b700750b0 con 0x7f7b700b5c70 2024-04-25T10:05:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.869+0000 7f7b6effd640 1 -- 172.21.15.16:0/472574110 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b60007c70 con 0x7f7b700b5c70 2024-04-25T10:05:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.870+0000 7f7b6effd640 1 --2- 172.21.15.16:0/472574110 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7b5c06b500 0x7f7b5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.870+0000 7f7b6effd640 1 -- 172.21.15.16:0/472574110 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b600ab010 con 0x7f7b700b5c70 2024-04-25T10:05:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.870+0000 7f7b7597b640 1 --2- 172.21.15.16:0/472574110 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7b5c06b500 0x7f7b5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.871+0000 7f7b7597b640 1 --2- 172.21.15.16:0/472574110 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7b5c06b500 0x7f7b5c06d9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7b64000f60 tx=0x7f7b640079d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:43.874+0000 7f7b6effd640 1 -- 172.21.15.16:0/472574110 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b60079e90 con 0x7f7b700b5c70 2024-04-25T10:05:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.055+0000 7f7b783e0640 1 -- 172.21.15.16:0/472574110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7b700b6f90 con 0x7f7b700b5c70 2024-04-25T10:05:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.056+0000 7f7b6effd640 1 -- 172.21.15.16:0/472574110 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7b60079790 con 0x7f7b700b5c70 2024-04-25T10:05:44.061 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:05:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.057+0000 7f7b783e0640 1 -- 172.21.15.16:0/472574110 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7b5c06b500 msgr2=0x7f7b5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.057+0000 7f7b783e0640 1 --2- 172.21.15.16:0/472574110 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7b5c06b500 0x7f7b5c06d9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7b64000f60 tx=0x7f7b640079d0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.057+0000 7f7b783e0640 1 -- 172.21.15.16:0/472574110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b700b5c70 msgr2=0x7f7b701e4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.058+0000 7f7b783e0640 1 --2- 172.21.15.16:0/472574110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b700b5c70 0x7f7b701e4480 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7b60034040 tx=0x7f7b60004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.058+0000 7f7b783e0640 1 -- 172.21.15.16:0/472574110 shutdown_connections 2024-04-25T10:05:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.058+0000 7f7b783e0640 1 --2- 172.21.15.16:0/472574110 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7b5c06b500 0x7f7b5c06d9c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.058+0000 7f7b783e0640 1 --2- 172.21.15.16:0/472574110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b7015dbb0 0x7f7b700b7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.058+0000 7f7b783e0640 1 --2- 172.21.15.16:0/472574110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7b700d1e20 0x7f7b70156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.058+0000 7f7b783e0640 1 --2- 172.21.15.16:0/472574110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b700b5c70 0x7f7b701e4480 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.058+0000 7f7b783e0640 1 -- 172.21.15.16:0/472574110 >> 172.21.15.16:0/472574110 conn(0x7f7b700d0560 msgr2=0x7f7b7006c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:44.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.058+0000 7f7b783e0640 1 -- 172.21.15.16:0/472574110 shutdown_connections 2024-04-25T10:05:44.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.058+0000 7f7b783e0640 1 -- 172.21.15.16:0/472574110 wait complete. 2024-04-25T10:05:44.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:44.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.172+0000 7ff5ebfa5640 1 Processor -- start 2024-04-25T10:05:44.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.172+0000 7ff5ebfa5640 1 -- start start 2024-04-25T10:05:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.173+0000 7ff5ebfa5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff5e415dca0 0x7ff5e40ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.173+0000 7ff5ebfa5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e40d4090 con 0x7ff5e40d3b90 2024-04-25T10:05:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.173+0000 7ff5ebfa5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e40d4370 con 0x7ff5e40d1130 2024-04-25T10:05:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.173+0000 7ff5ebfa5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e40b6b20 con 0x7ff5e415dca0 2024-04-25T10:05:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.173+0000 7ff5e9540640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff5e40d3b90 0x7ff5e40d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51398/0 (socket says 172.21.15.16:51398) 2024-04-25T10:05:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.173+0000 7ff5e9540640 1 -- 172.21.15.16:0/3950704622 learned_addr learned my addr 172.21.15.16:0/3950704622 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.173+0000 7ff5e8d3f640 1 -- 172.21.15.16:0/3950704622 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 979055547 0 0) 0x7ff5e40d4090 con 0x7ff5e40d3b90 2024-04-25T10:05:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.173+0000 7ff5e9d41640 1 --2- 172.21.15.16:0/3950704622 >> v2:172.21.15.186:3300/0 conn(0x7ff5e415dca0 0x7ff5e40ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.173+0000 7ff5e8d3f640 1 -- 172.21.15.16:0/3950704622 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5cc003520 con 0x7ff5e40d3b90 2024-04-25T10:05:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e8d3f640 1 -- 172.21.15.16:0/3950704622 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 897058343 0 0) 0x7ff5cc003520 con 0x7ff5e40d3b90 2024-04-25T10:05:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e8d3f640 1 -- 172.21.15.16:0/3950704622 >> v2:172.21.15.186:3300/0 conn(0x7ff5e415dca0 msgr2=0x7ff5e40ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e8d3f640 1 --2- 172.21.15.16:0/3950704622 >> v2:172.21.15.186:3300/0 conn(0x7ff5e415dca0 0x7ff5e40ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e8d3f640 1 -- 172.21.15.16:0/3950704622 >> v1:172.21.15.186:6789/0 conn(0x7ff5e40d1130 legacy=0x7ff5e40b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e8d3f640 1 -- 172.21.15.16:0/3950704622 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e40d4650 con 0x7ff5e40d3b90 2024-04-25T10:05:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e8d3f640 1 -- 172.21.15.16:0/3950704622 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7ff5d8001670 con 0x7ff5e40d3b90 2024-04-25T10:05:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e8d3f640 1 -- 172.21.15.16:0/3950704622 >> v1:172.21.15.16:6789/0 conn(0x7ff5e40d3b90 legacy=0x7ff5e40d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e8d3f640 1 --2- 172.21.15.16:0/3950704622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5cc0039b0 0x7ff5cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e8d3f640 1 --2- 172.21.15.16:0/3950704622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5cc004800 0x7ff5cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e8d3f640 1 --2- 172.21.15.16:0/3950704622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5cc0051c0 0x7ff5cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e8d3f640 1 -- 172.21.15.16:0/3950704622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc00c030 con 0x7ff5cc004800 2024-04-25T10:05:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e8d3f640 1 -- 172.21.15.16:0/3950704622 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc00c1e0 con 0x7ff5cc0039b0 2024-04-25T10:05:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e8d3f640 1 -- 172.21.15.16:0/3950704622 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc00c3e0 con 0x7ff5cc0051c0 2024-04-25T10:05:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e9540640 1 --2- 172.21.15.16:0/3950704622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5cc004800 0x7ff5cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e9d41640 1 --2- 172.21.15.16:0/3950704622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5cc0039b0 0x7ff5cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5ea542640 1 --2- 172.21.15.16:0/3950704622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5cc0051c0 0x7ff5cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e9540640 1 -- 172.21.15.16:0/3950704622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5cc0051c0 msgr2=0x7ff5cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e9540640 1 --2- 172.21.15.16:0/3950704622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5cc0051c0 0x7ff5cc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e9540640 1 -- 172.21.15.16:0/3950704622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5cc0039b0 msgr2=0x7ff5cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e9540640 1 --2- 172.21.15.16:0/3950704622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5cc0039b0 0x7ff5cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5e9540640 1 -- 172.21.15.16:0/3950704622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5cc003520 con 0x7ff5cc004800 2024-04-25T10:05:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.174+0000 7ff5ea542640 1 -- 172.21.15.16:0/3950704622 reap_dead start 2024-04-25T10:05:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.175+0000 7ff5e9540640 1 --2- 172.21.15.16:0/3950704622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5cc004800 0x7ff5cc004c80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c210 tx=0x7ff5cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c1f4ab804742ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.175+0000 7ff5e8d3f640 1 -- 172.21.15.16:0/3950704622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8007cf0 con 0x7ff5cc004800 2024-04-25T10:05:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.175+0000 7ff5e8d3f640 1 -- 172.21.15.16:0/3950704622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d8005af0 con 0x7ff5cc004800 2024-04-25T10:05:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.175+0000 7ff5e8d3f640 1 -- 172.21.15.16:0/3950704622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8007070 con 0x7ff5cc004800 2024-04-25T10:05:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.176+0000 7ff5ebfa5640 1 -- 172.21.15.16:0/3950704622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5cc004800 msgr2=0x7ff5cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.176+0000 7ff5ebfa5640 1 --2- 172.21.15.16:0/3950704622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5cc004800 0x7ff5cc004c80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c210 tx=0x7ff5cc005830 comp rx=0 tx=0).stop 2024-04-25T10:05:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.176+0000 7ff5ebfa5640 1 -- 172.21.15.16:0/3950704622 shutdown_connections 2024-04-25T10:05:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.176+0000 7ff5ebfa5640 1 --2- 172.21.15.16:0/3950704622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5cc004800 0x7ff5cc004c80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.176+0000 7ff5ebfa5640 1 -- 172.21.15.16:0/3950704622 >> 172.21.15.16:0/3950704622 conn(0x7ff5e40726c0 msgr2=0x7ff5e4072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.176+0000 7ff5ebfa5640 1 -- 172.21.15.16:0/3950704622 shutdown_connections 2024-04-25T10:05:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.176+0000 7ff5ebfa5640 1 -- 172.21.15.16:0/3950704622 wait complete. 2024-04-25T10:05:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.177+0000 7ff5ebfa5640 1 Processor -- start 2024-04-25T10:05:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.177+0000 7ff5ebfa5640 1 -- start start 2024-04-25T10:05:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5ebfa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5cc004800 0x7ff5e4154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5ebfa5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e4151d40 0x7ff5e41514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5ebfa5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5e415ab60 0x7ff5e415afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5ebfa5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e415e200 con 0x7ff5cc004800 2024-04-25T10:05:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5ebfa5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e415e080 con 0x7ff5e4151d40 2024-04-25T10:05:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5ebfa5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e40d0a20 con 0x7ff5e415ab60 2024-04-25T10:05:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5e9d41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5cc004800 0x7ff5e4154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5e9d41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5cc004800 0x7ff5e4154a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42456/0 (socket says 172.21.15.16:42456) 2024-04-25T10:05:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5e9d41640 1 -- 172.21.15.16:0/350811450 learned_addr learned my addr 172.21.15.16:0/350811450 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5e9540640 1 --2- 172.21.15.16:0/350811450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e4151d40 0x7ff5e41514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5ea542640 1 --2- 172.21.15.16:0/350811450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5e415ab60 0x7ff5e415afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5e9d41640 1 -- 172.21.15.16:0/350811450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5e415ab60 msgr2=0x7ff5e415afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5e9d41640 1 --2- 172.21.15.16:0/350811450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5e415ab60 0x7ff5e415afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5e9d41640 1 -- 172.21.15.16:0/350811450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e4151d40 msgr2=0x7ff5e41514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5e9d41640 1 --2- 172.21.15.16:0/350811450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e4151d40 0x7ff5e41514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5e9d41640 1 -- 172.21.15.16:0/350811450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e4152600 con 0x7ff5cc004800 2024-04-25T10:05:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5ea542640 1 --2- 172.21.15.16:0/350811450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5e415ab60 0x7ff5e415afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5e9d41640 1 --2- 172.21.15.16:0/350811450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5cc004800 0x7ff5e4154a80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4009fc0 tx=0x7ff5d400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5e27fc640 1 -- 172.21.15.16:0/350811450 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d400fea0 con 0x7ff5cc004800 2024-04-25T10:05:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5e27fc640 1 -- 172.21.15.16:0/350811450 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d4015630 con 0x7ff5cc004800 2024-04-25T10:05:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5ebfa5640 1 -- 172.21.15.16:0/350811450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e41528e0 con 0x7ff5cc004800 2024-04-25T10:05:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.178+0000 7ff5e27fc640 1 -- 172.21.15.16:0/350811450 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4015a20 con 0x7ff5cc004800 2024-04-25T10:05:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.179+0000 7ff5ebfa5640 1 -- 172.21.15.16:0/350811450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e4152ea0 con 0x7ff5cc004800 2024-04-25T10:05:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.179+0000 7ff5ebfa5640 1 -- 172.21.15.16:0/350811450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e40750b0 con 0x7ff5cc004800 2024-04-25T10:05:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.180+0000 7ff5e27fc640 1 -- 172.21.15.16:0/350811450 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5d4015bc0 con 0x7ff5cc004800 2024-04-25T10:05:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.180+0000 7ff5e27fc640 1 --2- 172.21.15.16:0/350811450 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff5d006b4d0 0x7ff5d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.180+0000 7ff5e27fc640 1 -- 172.21.15.16:0/350811450 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5d408b340 con 0x7ff5cc004800 2024-04-25T10:05:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.180+0000 7ff5e9540640 1 --2- 172.21.15.16:0/350811450 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff5d006b4d0 0x7ff5d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.181+0000 7ff5e9540640 1 --2- 172.21.15.16:0/350811450 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff5d006b4d0 0x7ff5d006d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40d3fc0 tx=0x7ff5d8034040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:44.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.184+0000 7ff5e27fc640 1 -- 172.21.15.16:0/350811450 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5d4019030 con 0x7ff5cc004800 2024-04-25T10:05:44.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.363+0000 7ff5ebfa5640 1 -- 172.21.15.16:0/350811450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff5e40b75b0 con 0x7ff5cc004800 2024-04-25T10:05:44.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.363+0000 7ff5e27fc640 1 -- 172.21.15.16:0/350811450 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff5d4010040 con 0x7ff5cc004800 2024-04-25T10:05:44.369 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:05:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.365+0000 7ff5ebfa5640 1 -- 172.21.15.16:0/350811450 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff5d006b4d0 msgr2=0x7ff5d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.365+0000 7ff5ebfa5640 1 --2- 172.21.15.16:0/350811450 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff5d006b4d0 0x7ff5d006d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40d3fc0 tx=0x7ff5d8034040 comp rx=0 tx=0).stop 2024-04-25T10:05:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.365+0000 7ff5ebfa5640 1 -- 172.21.15.16:0/350811450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5cc004800 msgr2=0x7ff5e4154a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.365+0000 7ff5ebfa5640 1 --2- 172.21.15.16:0/350811450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5cc004800 0x7ff5e4154a80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4009fc0 tx=0x7ff5d400f580 comp rx=0 tx=0).stop 2024-04-25T10:05:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.365+0000 7ff5ebfa5640 1 -- 172.21.15.16:0/350811450 shutdown_connections 2024-04-25T10:05:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.365+0000 7ff5ebfa5640 1 --2- 172.21.15.16:0/350811450 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff5d006b4d0 0x7ff5d006d990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.365+0000 7ff5ebfa5640 1 --2- 172.21.15.16:0/350811450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5e415ab60 0x7ff5e415afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.365+0000 7ff5ebfa5640 1 --2- 172.21.15.16:0/350811450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e4151d40 0x7ff5e41514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.365+0000 7ff5ebfa5640 1 --2- 172.21.15.16:0/350811450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5cc004800 0x7ff5e4154a80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.365+0000 7ff5ebfa5640 1 -- 172.21.15.16:0/350811450 >> 172.21.15.16:0/350811450 conn(0x7ff5e40726c0 msgr2=0x7ff5e40b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.365+0000 7ff5ebfa5640 1 -- 172.21.15.16:0/350811450 shutdown_connections 2024-04-25T10:05:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.365+0000 7ff5ebfa5640 1 -- 172.21.15.16:0/350811450 wait complete. 2024-04-25T10:05:44.382 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:05:44.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.478+0000 7f48327d5640 1 Processor -- start 2024-04-25T10:05:44.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.478+0000 7f48327d5640 1 -- start start 2024-04-25T10:05:44.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.479+0000 7f48327d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0aedc0 0x7f482c15ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.479+0000 7f48327d5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f482c06d260 0x7f482c0b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.479+0000 7f48327d5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f482c0b4730 0x7f482c0b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.479+0000 7f48327d5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f482c0b4ba0 con 0x7f482c0b4730 2024-04-25T10:05:44.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.479+0000 7f48327d5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f482c15c050 con 0x7f482c06d260 2024-04-25T10:05:44.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.479+0000 7f48327d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0af1a0 con 0x7f482c0aedc0 2024-04-25T10:05:44.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.479+0000 7f4830d72640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f482c0b4730 0x7f482c0b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.479+0000 7f4830d72640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f482c0b4730 0x7f482c0b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42470/0 (socket says 172.21.15.16:42470) 2024-04-25T10:05:44.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.479+0000 7f4830d72640 1 -- 172.21.15.16:0/2335248620 learned_addr learned my addr 172.21.15.16:0/2335248620 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:44.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.480+0000 7f482b7fe640 1 --2- 172.21.15.16:0/2335248620 >> v2:172.21.15.186:3300/0 conn(0x7f482c06d260 0x7f482c0b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.480+0000 7f482bfff640 1 --2- 172.21.15.16:0/2335248620 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0aedc0 0x7f482c15ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.480+0000 7f4830d72640 1 -- 172.21.15.16:0/2335248620 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0aedc0 msgr2=0x7f482c15ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:44.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.480+0000 7f4830d72640 1 --2- 172.21.15.16:0/2335248620 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0aedc0 0x7f482c15ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.480+0000 7f4830d72640 1 -- 172.21.15.16:0/2335248620 >> v2:172.21.15.186:3300/0 conn(0x7f482c06d260 msgr2=0x7f482c0b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:44.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.480+0000 7f4830d72640 1 --2- 172.21.15.16:0/2335248620 >> v2:172.21.15.186:3300/0 conn(0x7f482c06d260 0x7f482c0b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.480+0000 7f4830d72640 1 -- 172.21.15.16:0/2335248620 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c06da40 con 0x7f482c0b4730 2024-04-25T10:05:44.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.481+0000 7f4830d72640 1 --2- 172.21.15.16:0/2335248620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f482c0b4730 0x7f482c0b26f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f482c06d0e0 tx=0x7f482402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948693724c5c9a7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:44.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.481+0000 7f482affd640 1 -- 172.21.15.16:0/2335248620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824036070 con 0x7f482c0b4730 2024-04-25T10:05:44.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.481+0000 7f482affd640 1 -- 172.21.15.16:0/2335248620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4824031600 con 0x7f482c0b4730 2024-04-25T10:05:44.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.481+0000 7f482affd640 1 -- 172.21.15.16:0/2335248620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48240319d0 con 0x7f482c0b4730 2024-04-25T10:05:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.481+0000 7f48327d5640 1 -- 172.21.15.16:0/2335248620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f482c0b4730 msgr2=0x7f482c0b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.481+0000 7f48327d5640 1 --2- 172.21.15.16:0/2335248620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f482c0b4730 0x7f482c0b26f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f482c06d0e0 tx=0x7f482402a630 comp rx=0 tx=0).stop 2024-04-25T10:05:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.481+0000 7f48327d5640 1 -- 172.21.15.16:0/2335248620 shutdown_connections 2024-04-25T10:05:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.481+0000 7f48327d5640 1 --2- 172.21.15.16:0/2335248620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f482c0b4730 0x7f482c0b26f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.481+0000 7f48327d5640 1 --2- 172.21.15.16:0/2335248620 >> v2:172.21.15.186:3300/0 conn(0x7f482c06d260 0x7f482c0b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.481+0000 7f48327d5640 1 --2- 172.21.15.16:0/2335248620 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0aedc0 0x7f482c15ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.481+0000 7f48327d5640 1 -- 172.21.15.16:0/2335248620 >> 172.21.15.16:0/2335248620 conn(0x7f482c06aba0 msgr2=0x7f482c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.481+0000 7f48327d5640 1 -- 172.21.15.16:0/2335248620 shutdown_connections 2024-04-25T10:05:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.481+0000 7f48327d5640 1 -- 172.21.15.16:0/2335248620 wait complete. 2024-04-25T10:05:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.482+0000 7f48327d5640 1 Processor -- start 2024-04-25T10:05:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.483+0000 7f48327d5640 1 -- start start 2024-04-25T10:05:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.483+0000 7f48327d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c06d260 0x7f482c1f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.483+0000 7f48327d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f482c0aedc0 0x7f482c0c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.483+0000 7f48327d5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f482c0b4730 0x7f482c0b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.483+0000 7f48327d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c15c2a0 con 0x7f482c0aedc0 2024-04-25T10:05:44.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.483+0000 7f48327d5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c06e000 con 0x7f482c0b4730 2024-04-25T10:05:44.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.483+0000 7f48327d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482c06d6c0 con 0x7f482c06d260 2024-04-25T10:05:44.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.483+0000 7f482b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f482c0aedc0 0x7f482c0c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.483+0000 7f482b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f482c0aedc0 0x7f482c0c48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42482/0 (socket says 172.21.15.16:42482) 2024-04-25T10:05:44.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.483+0000 7f482b7fe640 1 -- 172.21.15.16:0/1307706244 learned_addr learned my addr 172.21.15.16:0/1307706244 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:44.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.484+0000 7f482bfff640 1 --2- 172.21.15.16:0/1307706244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c06d260 0x7f482c1f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.484+0000 7f4830d72640 1 --2- 172.21.15.16:0/1307706244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f482c0b4730 0x7f482c0b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.484+0000 7f482b7fe640 1 -- 172.21.15.16:0/1307706244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c06d260 msgr2=0x7f482c1f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:44.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.484+0000 7f482b7fe640 1 --2- 172.21.15.16:0/1307706244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c06d260 0x7f482c1f46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.484+0000 7f482b7fe640 1 -- 172.21.15.16:0/1307706244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f482c0b4730 msgr2=0x7f482c0b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:44.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.484+0000 7f482b7fe640 1 --2- 172.21.15.16:0/1307706244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f482c0b4730 0x7f482c0b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.484+0000 7f482b7fe640 1 -- 172.21.15.16:0/1307706244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c0b2880 con 0x7f482c0aedc0 2024-04-25T10:05:44.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.484+0000 7f482b7fe640 1 --2- 172.21.15.16:0/1307706244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f482c0aedc0 0x7f482c0c48d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f482000e890 tx=0x7f482000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:44.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.484+0000 7f4828ff9640 1 -- 172.21.15.16:0/1307706244 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4820017070 con 0x7f482c0aedc0 2024-04-25T10:05:44.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.484+0000 7f4828ff9640 1 -- 172.21.15.16:0/1307706244 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482000c940 con 0x7f482c0aedc0 2024-04-25T10:05:44.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.484+0000 7f48327d5640 1 -- 172.21.15.16:0/1307706244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482c0acb00 con 0x7f482c0aedc0 2024-04-25T10:05:44.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.484+0000 7f4828ff9640 1 -- 172.21.15.16:0/1307706244 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482000cd10 con 0x7f482c0aedc0 2024-04-25T10:05:44.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.484+0000 7f48327d5640 1 -- 172.21.15.16:0/1307706244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f482c0ad0c0 con 0x7f482c0aedc0 2024-04-25T10:05:44.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.485+0000 7f48327d5640 1 -- 172.21.15.16:0/1307706244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482c0750b0 con 0x7f482c0aedc0 2024-04-25T10:05:44.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.486+0000 7f4828ff9640 1 -- 172.21.15.16:0/1307706244 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f482001a070 con 0x7f482c0aedc0 2024-04-25T10:05:44.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.486+0000 7f4828ff9640 1 --2- 172.21.15.16:0/1307706244 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f481806b4d0 0x7f481806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.486+0000 7f4828ff9640 1 -- 172.21.15.16:0/1307706244 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f482008a550 con 0x7f482c0aedc0 2024-04-25T10:05:44.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.486+0000 7f482bfff640 1 --2- 172.21.15.16:0/1307706244 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f481806b4d0 0x7f481806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.487+0000 7f482bfff640 1 --2- 172.21.15.16:0/1307706244 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f481806b4d0 0x7f481806d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f481c009ad0 tx=0x7f481c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:44.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.490+0000 7f4828ff9640 1 -- 172.21.15.16:0/1307706244 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48200593d0 con 0x7f482c0aedc0 2024-04-25T10:05:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.682+0000 7f48327d5640 1 -- 172.21.15.16:0/1307706244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f482c0742c0 con 0x7f482c0aedc0 2024-04-25T10:05:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.683+0000 7f4828ff9640 1 -- 172.21.15.16:0/1307706244 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4820058cd0 con 0x7f482c0aedc0 2024-04-25T10:05:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-04-25T10:05:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.685+0000 7f48327d5640 1 -- 172.21.15.16:0/1307706244 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f481806b4d0 msgr2=0x7f481806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.685+0000 7f48327d5640 1 --2- 172.21.15.16:0/1307706244 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f481806b4d0 0x7f481806d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f481c009ad0 tx=0x7f481c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.685+0000 7f48327d5640 1 -- 172.21.15.16:0/1307706244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f482c0aedc0 msgr2=0x7f482c0c48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.685+0000 7f48327d5640 1 --2- 172.21.15.16:0/1307706244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f482c0aedc0 0x7f482c0c48d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f482000e890 tx=0x7f482000ed50 comp rx=0 tx=0).stop 2024-04-25T10:05:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.685+0000 7f48327d5640 1 -- 172.21.15.16:0/1307706244 shutdown_connections 2024-04-25T10:05:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.685+0000 7f48327d5640 1 --2- 172.21.15.16:0/1307706244 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f481806b4d0 0x7f481806d990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.685+0000 7f48327d5640 1 --2- 172.21.15.16:0/1307706244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f482c0b4730 0x7f482c0b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.685+0000 7f48327d5640 1 --2- 172.21.15.16:0/1307706244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f482c0aedc0 0x7f482c0c48d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.685+0000 7f48327d5640 1 --2- 172.21.15.16:0/1307706244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c06d260 0x7f482c1f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.685+0000 7f48327d5640 1 -- 172.21.15.16:0/1307706244 >> 172.21.15.16:0/1307706244 conn(0x7f482c06aba0 msgr2=0x7f482c0cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.685+0000 7f48327d5640 1 -- 172.21.15.16:0/1307706244 shutdown_connections 2024-04-25T10:05:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:44.685+0000 7f48327d5640 1 -- 172.21.15.16:0/1307706244 wait complete. 2024-04-25T10:05:44.701 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:05:44.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:44.701 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T10:05:44.726 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:05:44.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:44.726 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:05:44.787 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:05:44.787 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:44.806 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:05:44.806 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:44.865 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:05:44.865 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:44.923 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:45.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.018+0000 7fc4f3ff5640 1 Processor -- start 2024-04-25T10:05:45.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.019+0000 7fc4f3ff5640 1 -- start start 2024-04-25T10:05:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.019+0000 7fc4f3ff5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc4ec15dc30 0x7fc4ec06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.019+0000 7fc4f3ff5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4ec06cc00 0x7fc4ec0b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.019+0000 7fc4f3ff5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4ec0b1960 0x7fc4ec099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.019+0000 7fc4f3ff5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4ec06c1f0 con 0x7fc4ec06cc00 2024-04-25T10:05:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.019+0000 7fc4f3ff5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4ec0cfb40 con 0x7fc4ec15dc30 2024-04-25T10:05:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.019+0000 7fc4f3ff5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec0b1db0 con 0x7fc4ec0b1960 2024-04-25T10:05:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.020+0000 7fc4f1590640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4ec06cc00 0x7fc4ec0b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.020+0000 7fc4f1590640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4ec06cc00 0x7fc4ec0b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42498/0 (socket says 172.21.15.16:42498) 2024-04-25T10:05:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.020+0000 7fc4f1590640 1 -- 172.21.15.16:0/1046326740 learned_addr learned my addr 172.21.15.16:0/1046326740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.020+0000 7fc4f2592640 1 --2- 172.21.15.16:0/1046326740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4ec0b1960 0x7fc4ec099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.020+0000 7fc4f1d91640 1 --2- 172.21.15.16:0/1046326740 >> v2:172.21.15.186:3300/0 conn(0x7fc4ec15dc30 0x7fc4ec06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.020+0000 7fc4f1590640 1 -- 172.21.15.16:0/1046326740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4ec0b1960 msgr2=0x7fc4ec099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.020+0000 7fc4f1590640 1 --2- 172.21.15.16:0/1046326740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4ec0b1960 0x7fc4ec099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.020+0000 7fc4f1590640 1 -- 172.21.15.16:0/1046326740 >> v2:172.21.15.186:3300/0 conn(0x7fc4ec15dc30 msgr2=0x7fc4ec06c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.020+0000 7fc4f1590640 1 --2- 172.21.15.16:0/1046326740 >> v2:172.21.15.186:3300/0 conn(0x7fc4ec15dc30 0x7fc4ec06c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.020+0000 7fc4f1590640 1 -- 172.21.15.16:0/1046326740 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ec0b2060 con 0x7fc4ec06cc00 2024-04-25T10:05:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.021+0000 7fc4f1590640 1 --2- 172.21.15.16:0/1046326740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4ec06cc00 0x7fc4ec0b1420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec09ab70 tx=0x7fc4dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc96ccbe6e601d53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.021+0000 7fc4f0d8f640 1 -- 172.21.15.16:0/1046326740 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc037070 con 0x7fc4ec06cc00 2024-04-25T10:05:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.021+0000 7fc4f0d8f640 1 -- 172.21.15.16:0/1046326740 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4dc032440 con 0x7fc4ec06cc00 2024-04-25T10:05:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.021+0000 7fc4f0d8f640 1 -- 172.21.15.16:0/1046326740 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc032810 con 0x7fc4ec06cc00 2024-04-25T10:05:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.021+0000 7fc4f3ff5640 1 -- 172.21.15.16:0/1046326740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4ec06cc00 msgr2=0x7fc4ec0b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.021+0000 7fc4f3ff5640 1 --2- 172.21.15.16:0/1046326740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4ec06cc00 0x7fc4ec0b1420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec09ab70 tx=0x7fc4dc02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.021+0000 7fc4f3ff5640 1 -- 172.21.15.16:0/1046326740 shutdown_connections 2024-04-25T10:05:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.021+0000 7fc4f3ff5640 1 --2- 172.21.15.16:0/1046326740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4ec0b1960 0x7fc4ec099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.021+0000 7fc4f3ff5640 1 --2- 172.21.15.16:0/1046326740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4ec06cc00 0x7fc4ec0b1420 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.021+0000 7fc4f3ff5640 1 --2- 172.21.15.16:0/1046326740 >> v2:172.21.15.186:3300/0 conn(0x7fc4ec15dc30 0x7fc4ec06c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.021+0000 7fc4f3ff5640 1 -- 172.21.15.16:0/1046326740 >> 172.21.15.16:0/1046326740 conn(0x7fc4ec0d1900 msgr2=0x7fc4ec0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.021+0000 7fc4f3ff5640 1 -- 172.21.15.16:0/1046326740 shutdown_connections 2024-04-25T10:05:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.021+0000 7fc4f3ff5640 1 -- 172.21.15.16:0/1046326740 wait complete. 2024-04-25T10:05:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.023+0000 7fc4f3ff5640 1 Processor -- start 2024-04-25T10:05:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.023+0000 7fc4f3ff5640 1 -- start start 2024-04-25T10:05:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.023+0000 7fc4f3ff5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4ec06cc00 0x7fc4ec154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.023+0000 7fc4f3ff5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4ec0b1960 0x7fc4ec0b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.023+0000 7fc4f3ff5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4ec15dc30 0x7fc4ec0b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.023+0000 7fc4f3ff5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec0cfde0 con 0x7fc4ec0b1960 2024-04-25T10:05:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.023+0000 7fc4f3ff5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec0b2620 con 0x7fc4ec06cc00 2024-04-25T10:05:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.023+0000 7fc4f3ff5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec15e010 con 0x7fc4ec15dc30 2024-04-25T10:05:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.023+0000 7fc4f1590640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4ec0b1960 0x7fc4ec0b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.024+0000 7fc4f1590640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4ec0b1960 0x7fc4ec0b57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42510/0 (socket says 172.21.15.16:42510) 2024-04-25T10:05:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.024+0000 7fc4f1590640 1 -- 172.21.15.16:0/4124251474 learned_addr learned my addr 172.21.15.16:0/4124251474 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.024+0000 7fc4f1d91640 1 --2- 172.21.15.16:0/4124251474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4ec06cc00 0x7fc4ec154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.024+0000 7fc4f2592640 1 --2- 172.21.15.16:0/4124251474 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4ec15dc30 0x7fc4ec0b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.024+0000 7fc4f1590640 1 -- 172.21.15.16:0/4124251474 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4ec15dc30 msgr2=0x7fc4ec0b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.024+0000 7fc4f1590640 1 --2- 172.21.15.16:0/4124251474 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4ec15dc30 0x7fc4ec0b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.024+0000 7fc4f1590640 1 -- 172.21.15.16:0/4124251474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4ec06cc00 msgr2=0x7fc4ec154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.024+0000 7fc4f1590640 1 --2- 172.21.15.16:0/4124251474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4ec06cc00 0x7fc4ec154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.024+0000 7fc4f1590640 1 -- 172.21.15.16:0/4124251474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ec1553a0 con 0x7fc4ec0b1960 2024-04-25T10:05:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.024+0000 7fc4f1d91640 1 --2- 172.21.15.16:0/4124251474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4ec06cc00 0x7fc4ec154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.024+0000 7fc4f1590640 1 --2- 172.21.15.16:0/4124251474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4ec0b1960 0x7fc4ec0b57f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc02a220 tx=0x7fc4dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.024+0000 7fc4e27fc640 1 -- 172.21.15.16:0/4124251474 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc041020 con 0x7fc4ec0b1960 2024-04-25T10:05:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.024+0000 7fc4e27fc640 1 -- 172.21.15.16:0/4124251474 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4dc0306c0 con 0x7fc4ec0b1960 2024-04-25T10:05:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.024+0000 7fc4f3ff5640 1 -- 172.21.15.16:0/4124251474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ec1e7ae0 con 0x7fc4ec0b1960 2024-04-25T10:05:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.024+0000 7fc4e27fc640 1 -- 172.21.15.16:0/4124251474 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc037040 con 0x7fc4ec0b1960 2024-04-25T10:05:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.024+0000 7fc4f3ff5640 1 -- 172.21.15.16:0/4124251474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ec1e80a0 con 0x7fc4ec0b1960 2024-04-25T10:05:45.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.025+0000 7fc4f3ff5640 1 -- 172.21.15.16:0/4124251474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4ec067850 con 0x7fc4ec0b1960 2024-04-25T10:05:45.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.026+0000 7fc4e27fc640 1 -- 172.21.15.16:0/4124251474 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4dc044020 con 0x7fc4ec0b1960 2024-04-25T10:05:45.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.026+0000 7fc4e27fc640 1 --2- 172.21.15.16:0/4124251474 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc4d006b4d0 0x7fc4d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.026+0000 7fc4e27fc640 1 -- 172.21.15.16:0/4124251474 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4dc0aaa20 con 0x7fc4ec0b1960 2024-04-25T10:05:45.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.026+0000 7fc4f1d91640 1 --2- 172.21.15.16:0/4124251474 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc4d006b4d0 0x7fc4d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.027+0000 7fc4f1d91640 1 --2- 172.21.15.16:0/4124251474 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc4d006b4d0 0x7fc4d006d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8006fd0 tx=0x7fc4e8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.030+0000 7fc4e27fc640 1 -- 172.21.15.16:0/4124251474 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4dc04b030 con 0x7fc4ec0b1960 2024-04-25T10:05:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.212+0000 7fc4f3ff5640 1 -- 172.21.15.16:0/4124251474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4ec0cec10 con 0x7fc4ec0b1960 2024-04-25T10:05:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.213+0000 7fc4e27fc640 1 -- 172.21.15.16:0/4124251474 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16609 (secure 0 0 0) 0x7fc4dc0798a0 con 0x7fc4ec0b1960 2024-04-25T10:05:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:45.219 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:12.449183+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.215+0000 7fc4f3ff5640 1 -- 172.21.15.16:0/4124251474 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc4d006b4d0 msgr2=0x7fc4d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.215+0000 7fc4f3ff5640 1 --2- 172.21.15.16:0/4124251474 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc4d006b4d0 0x7fc4d006d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8006fd0 tx=0x7fc4e8008040 comp rx=0 tx=0).stop 2024-04-25T10:05:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.215+0000 7fc4f3ff5640 1 -- 172.21.15.16:0/4124251474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4ec0b1960 msgr2=0x7fc4ec0b57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.215+0000 7fc4f3ff5640 1 --2- 172.21.15.16:0/4124251474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4ec0b1960 0x7fc4ec0b57f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc02a220 tx=0x7fc4dc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.215+0000 7fc4f3ff5640 1 -- 172.21.15.16:0/4124251474 shutdown_connections 2024-04-25T10:05:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.215+0000 7fc4f3ff5640 1 --2- 172.21.15.16:0/4124251474 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc4d006b4d0 0x7fc4d006d990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.215+0000 7fc4f3ff5640 1 --2- 172.21.15.16:0/4124251474 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4ec15dc30 0x7fc4ec0b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.215+0000 7fc4f3ff5640 1 --2- 172.21.15.16:0/4124251474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4ec0b1960 0x7fc4ec0b57f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.215+0000 7fc4f3ff5640 1 --2- 172.21.15.16:0/4124251474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4ec06cc00 0x7fc4ec154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.215+0000 7fc4f3ff5640 1 -- 172.21.15.16:0/4124251474 >> 172.21.15.16:0/4124251474 conn(0x7fc4ec0d1900 msgr2=0x7fc4ec0d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.215+0000 7fc4f3ff5640 1 -- 172.21.15.16:0/4124251474 shutdown_connections 2024-04-25T10:05:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.215+0000 7fc4f3ff5640 1 -- 172.21.15.16:0/4124251474 wait complete. 2024-04-25T10:05:45.231 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:08.724537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:09.793105+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:45.232 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:45.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.330+0000 7fedb335a640 1 Processor -- start 2024-04-25T10:05:45.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.330+0000 7fedb335a640 1 -- start start 2024-04-25T10:05:45.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.331+0000 7fedb335a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedac15dc80 0x7fedac161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.331+0000 7fedb335a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedac1627a0 con 0x7fedac0b97c0 2024-04-25T10:05:45.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.331+0000 7fedb335a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedac1621e0 con 0x7fedac0b5c00 2024-04-25T10:05:45.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.331+0000 7fedb335a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedac1624c0 con 0x7fedac15dc80 2024-04-25T10:05:45.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.331+0000 7fedb08f5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fedac0b97c0 0x7fedac0b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51410/0 (socket says 172.21.15.16:51410) 2024-04-25T10:05:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.331+0000 7fedb08f5640 1 -- 172.21.15.16:0/1816122190 learned_addr learned my addr 172.21.15.16:0/1816122190 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.331+0000 7fedb10f6640 1 --2- 172.21.15.16:0/1816122190 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedac15dc80 0x7fedac161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.332+0000 7fedabfff640 1 -- 172.21.15.16:0/1816122190 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980469549 0 0) 0x7fedac1627a0 con 0x7fedac0b97c0 2024-04-25T10:05:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.332+0000 7fedabfff640 1 -- 172.21.15.16:0/1816122190 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed94003520 con 0x7fedac0b97c0 2024-04-25T10:05:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.332+0000 7fedabfff640 1 -- 172.21.15.16:0/1816122190 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 913630275 0 0) 0x7fedac1621e0 con 0x7fedac0b5c00 2024-04-25T10:05:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.332+0000 7fedabfff640 1 -- 172.21.15.16:0/1816122190 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedac1627a0 con 0x7fedac0b5c00 2024-04-25T10:05:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.332+0000 7fedabfff640 1 -- 172.21.15.16:0/1816122190 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 518582155 0 0) 0x7fed94003520 con 0x7fedac0b97c0 2024-04-25T10:05:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.332+0000 7fedb10f6640 1 -- 172.21.15.16:0/1816122190 >> v1:172.21.15.186:6789/0 conn(0x7fedac0b5c00 legacy=0x7fedac0d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.332+0000 7fedb10f6640 1 -- 172.21.15.16:0/1816122190 >> v1:172.21.15.16:6789/0 conn(0x7fedac0b97c0 legacy=0x7fedac0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.332+0000 7fedb10f6640 1 -- 172.21.15.16:0/1816122190 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac0d1c70 con 0x7fedac15dc80 2024-04-25T10:05:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.333+0000 7fedb10f6640 1 --2- 172.21.15.16:0/1816122190 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedac15dc80 0x7fedac161870 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7feda40097d0 tx=0x7feda402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d34f74a6bdaea3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.333+0000 7fedabfff640 1 -- 172.21.15.16:0/1816122190 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda4037070 con 0x7fedac15dc80 2024-04-25T10:05:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.333+0000 7fedabfff640 1 -- 172.21.15.16:0/1816122190 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda402ac30 con 0x7fedac15dc80 2024-04-25T10:05:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.333+0000 7fedabfff640 1 -- 172.21.15.16:0/1816122190 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0001670 con 0x7fedac15dc80 2024-04-25T10:05:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.333+0000 7fedb335a640 1 -- 172.21.15.16:0/1816122190 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedac15dc80 msgr2=0x7fedac161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.333+0000 7fedb335a640 1 --2- 172.21.15.16:0/1816122190 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedac15dc80 0x7fedac161870 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7feda40097d0 tx=0x7feda402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.333+0000 7fedb335a640 1 -- 172.21.15.16:0/1816122190 shutdown_connections 2024-04-25T10:05:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.333+0000 7fedb335a640 1 --2- 172.21.15.16:0/1816122190 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedac15dc80 0x7fedac161870 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.333+0000 7fedb335a640 1 -- 172.21.15.16:0/1816122190 >> 172.21.15.16:0/1816122190 conn(0x7fedac0d04f0 msgr2=0x7fedac0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.334+0000 7fedb335a640 1 -- 172.21.15.16:0/1816122190 shutdown_connections 2024-04-25T10:05:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.334+0000 7fedb335a640 1 -- 172.21.15.16:0/1816122190 wait complete. 2024-04-25T10:05:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.335+0000 7fedb335a640 1 Processor -- start 2024-04-25T10:05:45.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.335+0000 7fedb335a640 1 -- start start 2024-04-25T10:05:45.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.335+0000 7fedb335a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedac0b5c00 0x7fedac0cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.335+0000 7fedb335a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac0b97c0 0x7fedac160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.335+0000 7fedb335a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedac15dc80 0x7fedac0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.335+0000 7fedb335a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac161f30 con 0x7fedac0b97c0 2024-04-25T10:05:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.335+0000 7fedb335a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac161db0 con 0x7fedac0b5c00 2024-04-25T10:05:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.335+0000 7fedb335a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedac0d2b80 con 0x7fedac15dc80 2024-04-25T10:05:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.336+0000 7fedb08f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac0b97c0 0x7fedac160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.336+0000 7fedb08f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac0b97c0 0x7fedac160ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42516/0 (socket says 172.21.15.16:42516) 2024-04-25T10:05:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.336+0000 7fedb08f5640 1 -- 172.21.15.16:0/400332882 learned_addr learned my addr 172.21.15.16:0/400332882 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.336+0000 7fedb10f6640 1 --2- 172.21.15.16:0/400332882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedac0b5c00 0x7fedac0cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.336+0000 7fedb18f7640 1 --2- 172.21.15.16:0/400332882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedac15dc80 0x7fedac0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.336+0000 7fedb08f5640 1 -- 172.21.15.16:0/400332882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedac15dc80 msgr2=0x7fedac0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.336+0000 7fedb08f5640 1 --2- 172.21.15.16:0/400332882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedac15dc80 0x7fedac0cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.336+0000 7fedb08f5640 1 -- 172.21.15.16:0/400332882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedac0b5c00 msgr2=0x7fedac0cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.336+0000 7fedb08f5640 1 --2- 172.21.15.16:0/400332882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedac0b5c00 0x7fedac0cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.336+0000 7fedb08f5640 1 -- 172.21.15.16:0/400332882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac15e8f0 con 0x7fedac0b97c0 2024-04-25T10:05:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.336+0000 7fedb08f5640 1 --2- 172.21.15.16:0/400332882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac0b97c0 0x7fedac160ca0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feda000c750 tx=0x7feda000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.336+0000 7feda9ffb640 1 -- 172.21.15.16:0/400332882 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0017070 con 0x7fedac0b97c0 2024-04-25T10:05:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.336+0000 7feda9ffb640 1 -- 172.21.15.16:0/400332882 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda0005ad0 con 0x7fedac0b97c0 2024-04-25T10:05:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.336+0000 7feda9ffb640 1 -- 172.21.15.16:0/400332882 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0007070 con 0x7fedac0b97c0 2024-04-25T10:05:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.336+0000 7fedb335a640 1 -- 172.21.15.16:0/400332882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedac15ebd0 con 0x7fedac0b97c0 2024-04-25T10:05:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.337+0000 7fedb335a640 1 -- 172.21.15.16:0/400332882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedac1611e0 con 0x7fedac0b97c0 2024-04-25T10:05:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.338+0000 7fedb335a640 1 -- 172.21.15.16:0/400332882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedac0750b0 con 0x7fedac0b97c0 2024-04-25T10:05:45.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.338+0000 7feda9ffb640 1 -- 172.21.15.16:0/400332882 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feda001a070 con 0x7fedac0b97c0 2024-04-25T10:05:45.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.338+0000 7feda9ffb640 1 --2- 172.21.15.16:0/400332882 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed9806b4d0 0x7fed9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.338+0000 7feda9ffb640 1 -- 172.21.15.16:0/400332882 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feda0089560 con 0x7fedac0b97c0 2024-04-25T10:05:45.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.339+0000 7fedb10f6640 1 --2- 172.21.15.16:0/400332882 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed9806b4d0 0x7fed9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.339+0000 7fedb10f6640 1 --2- 172.21.15.16:0/400332882 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed9806b4d0 0x7fed9806d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feda4034000 tx=0x7feda402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.342+0000 7feda9ffb640 1 -- 172.21.15.16:0/400332882 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feda0057e20 con 0x7fedac0b97c0 2024-04-25T10:05:45.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.522+0000 7fedb335a640 1 -- 172.21.15.16:0/400332882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fedac0d1780 con 0x7fedac0b97c0 2024-04-25T10:05:45.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.523+0000 7feda9ffb640 1 -- 172.21.15.16:0/400332882 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7feda005bec0 con 0x7fedac0b97c0 2024-04-25T10:05:45.528 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:05:45.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.524+0000 7fedb335a640 1 -- 172.21.15.16:0/400332882 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed9806b4d0 msgr2=0x7fed9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.524+0000 7fedb335a640 1 --2- 172.21.15.16:0/400332882 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed9806b4d0 0x7fed9806d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feda4034000 tx=0x7feda402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:45.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.524+0000 7fedb335a640 1 -- 172.21.15.16:0/400332882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac0b97c0 msgr2=0x7fedac160ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.524+0000 7fedb335a640 1 --2- 172.21.15.16:0/400332882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac0b97c0 0x7fedac160ca0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feda000c750 tx=0x7feda000cc10 comp rx=0 tx=0).stop 2024-04-25T10:05:45.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.524+0000 7fedb335a640 1 -- 172.21.15.16:0/400332882 shutdown_connections 2024-04-25T10:05:45.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.524+0000 7fedb335a640 1 --2- 172.21.15.16:0/400332882 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed9806b4d0 0x7fed9806d990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.524+0000 7fedb335a640 1 --2- 172.21.15.16:0/400332882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedac15dc80 0x7fedac0cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.524+0000 7fedb335a640 1 --2- 172.21.15.16:0/400332882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac0b97c0 0x7fedac160ca0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.524+0000 7fedb335a640 1 --2- 172.21.15.16:0/400332882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedac0b5c00 0x7fedac0cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.524+0000 7fedb335a640 1 -- 172.21.15.16:0/400332882 >> 172.21.15.16:0/400332882 conn(0x7fedac0d04f0 msgr2=0x7fedac0b7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.525+0000 7fedb335a640 1 -- 172.21.15.16:0/400332882 shutdown_connections 2024-04-25T10:05:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.525+0000 7fedb335a640 1 -- 172.21.15.16:0/400332882 wait complete. 2024-04-25T10:05:45.541 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:45.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.639+0000 7f2d06a09640 1 Processor -- start 2024-04-25T10:05:45.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.639+0000 7f2d06a09640 1 -- start start 2024-04-25T10:05:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.640+0000 7f2d06a09640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d0015b940 0x7f2d0015f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.640+0000 7f2d06a09640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2d0006aba0 0x7f2d000cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.640+0000 7f2d06a09640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d000b1780 0x7f2d000b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.640+0000 7f2d06a09640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d001606e0 con 0x7f2d000b1780 2024-04-25T10:05:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.640+0000 7f2d06a09640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d000cca20 con 0x7f2d0006aba0 2024-04-25T10:05:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.640+0000 7f2d06a09640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d000b2110 con 0x7f2d0015b940 2024-04-25T10:05:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.640+0000 7f2d04fa6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d000b1780 0x7f2d000b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.640+0000 7f2d04fa6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d000b1780 0x7f2d000b1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42530/0 (socket says 172.21.15.16:42530) 2024-04-25T10:05:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.640+0000 7f2d04fa6640 1 -- 172.21.15.16:0/2632934365 learned_addr learned my addr 172.21.15.16:0/2632934365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.640+0000 7f2cff7fe640 1 --2- 172.21.15.16:0/2632934365 >> v2:172.21.15.186:3300/0 conn(0x7f2d0006aba0 0x7f2d000cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.640+0000 7f2cfffff640 1 --2- 172.21.15.16:0/2632934365 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d0015b940 0x7f2d0015f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.640+0000 7f2d04fa6640 1 -- 172.21.15.16:0/2632934365 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d0015b940 msgr2=0x7f2d0015f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.640+0000 7f2d04fa6640 1 --2- 172.21.15.16:0/2632934365 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d0015b940 0x7f2d0015f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.640+0000 7f2d04fa6640 1 -- 172.21.15.16:0/2632934365 >> v2:172.21.15.186:3300/0 conn(0x7f2d0006aba0 msgr2=0x7f2d000cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.640+0000 7f2d04fa6640 1 --2- 172.21.15.16:0/2632934365 >> v2:172.21.15.186:3300/0 conn(0x7f2d0006aba0 0x7f2d000cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.640+0000 7f2d04fa6640 1 -- 172.21.15.16:0/2632934365 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d000c4100 con 0x7f2d000b1780 2024-04-25T10:05:45.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.641+0000 7f2d04fa6640 1 --2- 172.21.15.16:0/2632934365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d000b1780 0x7f2d000b1bd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2d00073410 tx=0x7f2cf802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9271b3e8855e8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.641+0000 7f2cfeffd640 1 -- 172.21.15.16:0/2632934365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8036070 con 0x7f2d000b1780 2024-04-25T10:05:45.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.641+0000 7f2cfeffd640 1 -- 172.21.15.16:0/2632934365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf8031600 con 0x7f2d000b1780 2024-04-25T10:05:45.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.641+0000 7f2cfeffd640 1 -- 172.21.15.16:0/2632934365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf80319d0 con 0x7f2d000b1780 2024-04-25T10:05:45.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.642+0000 7f2d06a09640 1 -- 172.21.15.16:0/2632934365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d000b1780 msgr2=0x7f2d000b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.642+0000 7f2d06a09640 1 --2- 172.21.15.16:0/2632934365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d000b1780 0x7f2d000b1bd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2d00073410 tx=0x7f2cf802a610 comp rx=0 tx=0).stop 2024-04-25T10:05:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.642+0000 7f2d06a09640 1 -- 172.21.15.16:0/2632934365 shutdown_connections 2024-04-25T10:05:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.642+0000 7f2d06a09640 1 --2- 172.21.15.16:0/2632934365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d000b1780 0x7f2d000b1bd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.642+0000 7f2d06a09640 1 --2- 172.21.15.16:0/2632934365 >> v2:172.21.15.186:3300/0 conn(0x7f2d0006aba0 0x7f2d000cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.642+0000 7f2d06a09640 1 --2- 172.21.15.16:0/2632934365 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d0015b940 0x7f2d0015f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.642+0000 7f2d06a09640 1 -- 172.21.15.16:0/2632934365 >> 172.21.15.16:0/2632934365 conn(0x7f2d000d07d0 msgr2=0x7f2d000d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.642+0000 7f2d06a09640 1 -- 172.21.15.16:0/2632934365 shutdown_connections 2024-04-25T10:05:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.642+0000 7f2d06a09640 1 -- 172.21.15.16:0/2632934365 wait complete. 2024-04-25T10:05:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.643+0000 7f2d06a09640 1 Processor -- start 2024-04-25T10:05:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.643+0000 7f2d06a09640 1 -- start start 2024-04-25T10:05:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.644+0000 7f2d06a09640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d0006aba0 0x7f2d000c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.644+0000 7f2d06a09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d000b1780 0x7f2d0014cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.644+0000 7f2d06a09640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d0015b940 0x7f2d000b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.644+0000 7f2d06a09640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d00160400 con 0x7f2d000b1780 2024-04-25T10:05:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.644+0000 7f2d06a09640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d00160120 con 0x7f2d0006aba0 2024-04-25T10:05:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.644+0000 7f2d06a09640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d000b0160 con 0x7f2d0015b940 2024-04-25T10:05:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.644+0000 7f2cff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d000b1780 0x7f2d0014cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.644+0000 7f2cff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d000b1780 0x7f2d0014cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42532/0 (socket says 172.21.15.16:42532) 2024-04-25T10:05:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.644+0000 7f2cff7fe640 1 -- 172.21.15.16:0/1300786053 learned_addr learned my addr 172.21.15.16:0/1300786053 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.644+0000 7f2cfffff640 1 --2- 172.21.15.16:0/1300786053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d0006aba0 0x7f2d000c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.644+0000 7f2d04fa6640 1 --2- 172.21.15.16:0/1300786053 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d0015b940 0x7f2d000b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.644+0000 7f2cff7fe640 1 -- 172.21.15.16:0/1300786053 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d0015b940 msgr2=0x7f2d000b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.644+0000 7f2cff7fe640 1 --2- 172.21.15.16:0/1300786053 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d0015b940 0x7f2d000b0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.644+0000 7f2cff7fe640 1 -- 172.21.15.16:0/1300786053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d0006aba0 msgr2=0x7f2d000c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.644+0000 7f2cff7fe640 1 --2- 172.21.15.16:0/1300786053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d0006aba0 0x7f2d000c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.644+0000 7f2cff7fe640 1 -- 172.21.15.16:0/1300786053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d000b33c0 con 0x7f2d000b1780 2024-04-25T10:05:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.645+0000 7f2cff7fe640 1 --2- 172.21.15.16:0/1300786053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d000b1780 0x7f2d0014cbc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000e870 tx=0x7f2cf000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.645+0000 7f2cfcff9640 1 -- 172.21.15.16:0/1300786053 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0017070 con 0x7f2d000b1780 2024-04-25T10:05:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.645+0000 7f2cfcff9640 1 -- 172.21.15.16:0/1300786053 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf000ee60 con 0x7f2d000b1780 2024-04-25T10:05:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.645+0000 7f2cfcff9640 1 -- 172.21.15.16:0/1300786053 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf000cb80 con 0x7f2d000b1780 2024-04-25T10:05:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.645+0000 7f2d06a09640 1 -- 172.21.15.16:0/1300786053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d000b36a0 con 0x7f2d000b1780 2024-04-25T10:05:45.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.645+0000 7f2d06a09640 1 -- 172.21.15.16:0/1300786053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d000b3c60 con 0x7f2d000b1780 2024-04-25T10:05:45.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.646+0000 7f2d06a09640 1 -- 172.21.15.16:0/1300786053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d000750b0 con 0x7f2d000b1780 2024-04-25T10:05:45.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.646+0000 7f2cfcff9640 1 -- 172.21.15.16:0/1300786053 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2cf001a070 con 0x7f2d000b1780 2024-04-25T10:05:45.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.646+0000 7f2cfcff9640 1 --2- 172.21.15.16:0/1300786053 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ce006b4d0 0x7f2ce006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.647+0000 7f2cfcff9640 1 -- 172.21.15.16:0/1300786053 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2cf008a350 con 0x7f2d000b1780 2024-04-25T10:05:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.647+0000 7f2cfffff640 1 --2- 172.21.15.16:0/1300786053 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ce006b4d0 0x7f2ce006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.647+0000 7f2cfffff640 1 --2- 172.21.15.16:0/1300786053 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ce006b4d0 0x7f2ce006d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009ad0 tx=0x7f2ce8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.651+0000 7f2cfcff9640 1 -- 172.21.15.16:0/1300786053 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2cf0059260 con 0x7f2d000b1780 2024-04-25T10:05:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.830+0000 7f2d06a09640 1 -- 172.21.15.16:0/1300786053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d000b6750 con 0x7f2d000b1780 2024-04-25T10:05:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.830+0000 7f2cfcff9640 1 -- 172.21.15.16:0/1300786053 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2cf0058b60 con 0x7f2d000b1780 2024-04-25T10:05:45.836 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:05:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.832+0000 7f2d06a09640 1 -- 172.21.15.16:0/1300786053 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ce006b4d0 msgr2=0x7f2ce006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.832+0000 7f2d06a09640 1 --2- 172.21.15.16:0/1300786053 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ce006b4d0 0x7f2ce006d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009ad0 tx=0x7f2ce8008040 comp rx=0 tx=0).stop 2024-04-25T10:05:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.832+0000 7f2d06a09640 1 -- 172.21.15.16:0/1300786053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d000b1780 msgr2=0x7f2d0014cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.832+0000 7f2d06a09640 1 --2- 172.21.15.16:0/1300786053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d000b1780 0x7f2d0014cbc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000e870 tx=0x7f2cf000ed30 comp rx=0 tx=0).stop 2024-04-25T10:05:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.832+0000 7f2d06a09640 1 -- 172.21.15.16:0/1300786053 shutdown_connections 2024-04-25T10:05:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.832+0000 7f2d06a09640 1 --2- 172.21.15.16:0/1300786053 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ce006b4d0 0x7f2ce006d990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.832+0000 7f2d06a09640 1 --2- 172.21.15.16:0/1300786053 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d0015b940 0x7f2d000b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.832+0000 7f2d06a09640 1 --2- 172.21.15.16:0/1300786053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d000b1780 0x7f2d0014cbc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.832+0000 7f2d06a09640 1 --2- 172.21.15.16:0/1300786053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d0006aba0 0x7f2d000c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.832+0000 7f2d06a09640 1 -- 172.21.15.16:0/1300786053 >> 172.21.15.16:0/1300786053 conn(0x7f2d000d07d0 msgr2=0x7f2d000cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.832+0000 7f2d06a09640 1 -- 172.21.15.16:0/1300786053 shutdown_connections 2024-04-25T10:05:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.832+0000 7f2d06a09640 1 -- 172.21.15.16:0/1300786053 wait complete. 2024-04-25T10:05:45.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:45.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.948+0000 7f95522b3640 1 Processor -- start 2024-04-25T10:05:45.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.948+0000 7f95522b3640 1 -- start start 2024-04-25T10:05:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.949+0000 7f95522b3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f954c0b97c0 0x7f954c0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.949+0000 7f95522b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f954c0b5a30 0x7f954c0d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.949+0000 7f95522b3640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f954c1627a0 con 0x7f954c15dc80 2024-04-25T10:05:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.949+0000 7f95522b3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f954c161f80 con 0x7f954c0b97c0 2024-04-25T10:05:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.949+0000 7f95522b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954c0b5e60 con 0x7f954c0b5a30 2024-04-25T10:05:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.949+0000 7f954b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f954c0b97c0 0x7f954c0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.949+0000 7f954b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f954c0b97c0 0x7f954c0b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42542/0 (socket says 172.21.15.16:42542) 2024-04-25T10:05:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.949+0000 7f954b7fe640 1 -- 172.21.15.16:0/603045344 learned_addr learned my addr 172.21.15.16:0/603045344 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.949+0000 7f9550850640 1 --2- 172.21.15.16:0/603045344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f954c0b5a30 0x7f954c0d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.949+0000 7f954b7fe640 1 -- 172.21.15.16:0/603045344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f954c0b5a30 msgr2=0x7f954c0d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.949+0000 7f954b7fe640 1 --2- 172.21.15.16:0/603045344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f954c0b5a30 0x7f954c0d2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.949+0000 7f954b7fe640 1 -- 172.21.15.16:0/603045344 >> v1:172.21.15.186:6789/0 conn(0x7f954c15dc80 legacy=0x7f954c161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.949+0000 7f954b7fe640 1 -- 172.21.15.16:0/603045344 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c1621e0 con 0x7f954c0b97c0 2024-04-25T10:05:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.950+0000 7f954b7fe640 1 --2- 172.21.15.16:0/603045344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c0b97c0 0x7f954c0b54f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f953c0098a0 tx=0x7f953c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21240c6906fb4d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.950+0000 7f954affd640 1 -- 172.21.15.16:0/603045344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c037070 con 0x7f954c0b97c0 2024-04-25T10:05:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.950+0000 7f954affd640 1 -- 172.21.15.16:0/603045344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c032440 con 0x7f954c0b97c0 2024-04-25T10:05:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.950+0000 7f954affd640 1 -- 172.21.15.16:0/603045344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c032810 con 0x7f954c0b97c0 2024-04-25T10:05:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.951+0000 7f95522b3640 1 -- 172.21.15.16:0/603045344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c0b97c0 msgr2=0x7f954c0b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.951+0000 7f95522b3640 1 --2- 172.21.15.16:0/603045344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c0b97c0 0x7f954c0b54f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f953c0098a0 tx=0x7f953c02a250 comp rx=0 tx=0).stop 2024-04-25T10:05:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.951+0000 7f95522b3640 1 -- 172.21.15.16:0/603045344 shutdown_connections 2024-04-25T10:05:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.951+0000 7f95522b3640 1 --2- 172.21.15.16:0/603045344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f954c0b5a30 0x7f954c0d2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.951+0000 7f95522b3640 1 --2- 172.21.15.16:0/603045344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c0b97c0 0x7f954c0b54f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.951+0000 7f95522b3640 1 -- 172.21.15.16:0/603045344 >> 172.21.15.16:0/603045344 conn(0x7f954c0d04f0 msgr2=0x7f954c0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.951+0000 7f95522b3640 1 -- 172.21.15.16:0/603045344 shutdown_connections 2024-04-25T10:05:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.951+0000 7f95522b3640 1 -- 172.21.15.16:0/603045344 wait complete. 2024-04-25T10:05:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.952+0000 7f95522b3640 1 Processor -- start 2024-04-25T10:05:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.952+0000 7f95522b3640 1 -- start start 2024-04-25T10:05:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.953+0000 7f95522b3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f954c0b5a30 0x7f954c1496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.953+0000 7f95522b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f954c0b97c0 0x7f954c0c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.953+0000 7f95522b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c15dc80 0x7f954c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.953+0000 7f95522b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c15e060 con 0x7f954c15dc80 2024-04-25T10:05:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.953+0000 7f95522b3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c0d1c70 con 0x7f954c0b5a30 2024-04-25T10:05:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.953+0000 7f95522b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954c0d29b0 con 0x7f954c0b97c0 2024-04-25T10:05:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.953+0000 7f9550850640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c15dc80 0x7f954c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.953+0000 7f9550850640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c15dc80 0x7f954c0cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42556/0 (socket says 172.21.15.16:42556) 2024-04-25T10:05:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.953+0000 7f9550850640 1 -- 172.21.15.16:0/1494067359 learned_addr learned my addr 172.21.15.16:0/1494067359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.953+0000 7f954b7fe640 1 --2- 172.21.15.16:0/1494067359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f954c0b97c0 0x7f954c0c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.953+0000 7f954bfff640 1 --2- 172.21.15.16:0/1494067359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f954c0b5a30 0x7f954c1496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.953+0000 7f9550850640 1 -- 172.21.15.16:0/1494067359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f954c0b97c0 msgr2=0x7f954c0c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.953+0000 7f9550850640 1 --2- 172.21.15.16:0/1494067359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f954c0b97c0 0x7f954c0c8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.953+0000 7f9550850640 1 -- 172.21.15.16:0/1494067359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f954c0b5a30 msgr2=0x7f954c1496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.953+0000 7f9550850640 1 --2- 172.21.15.16:0/1494067359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f954c0b5a30 0x7f954c1496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.953+0000 7f9550850640 1 -- 172.21.15.16:0/1494067359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c0c7d80 con 0x7f954c15dc80 2024-04-25T10:05:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.953+0000 7f954bfff640 1 --2- 172.21.15.16:0/1494067359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f954c0b5a30 0x7f954c1496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.953+0000 7f9550850640 1 --2- 172.21.15.16:0/1494067359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c15dc80 0x7f954c0cd500 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9544007e40 tx=0x7f954400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.954+0000 7f9548ff9640 1 -- 172.21.15.16:0/1494067359 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544014620 con 0x7f954c15dc80 2024-04-25T10:05:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.954+0000 7f9548ff9640 1 -- 172.21.15.16:0/1494067359 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9544019070 con 0x7f954c15dc80 2024-04-25T10:05:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.954+0000 7f95522b3640 1 -- 172.21.15.16:0/1494067359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954c0c8060 con 0x7f954c15dc80 2024-04-25T10:05:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.954+0000 7f9548ff9640 1 -- 172.21.15.16:0/1494067359 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544014a20 con 0x7f954c15dc80 2024-04-25T10:05:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.954+0000 7f95522b3640 1 -- 172.21.15.16:0/1494067359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954c0c8620 con 0x7f954c15dc80 2024-04-25T10:05:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.955+0000 7f95522b3640 1 -- 172.21.15.16:0/1494067359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954c0750b0 con 0x7f954c15dc80 2024-04-25T10:05:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.955+0000 7f9548ff9640 1 -- 172.21.15.16:0/1494067359 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9544014bc0 con 0x7f954c15dc80 2024-04-25T10:05:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.955+0000 7f9548ff9640 1 --2- 172.21.15.16:0/1494067359 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f953806b4d0 0x7f953806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.955+0000 7f9548ff9640 1 -- 172.21.15.16:0/1494067359 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f954408a9e0 con 0x7f954c15dc80 2024-04-25T10:05:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.956+0000 7f954bfff640 1 --2- 172.21.15.16:0/1494067359 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f953806b4d0 0x7f953806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.956+0000 7f954bfff640 1 --2- 172.21.15.16:0/1494067359 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f953806b4d0 0x7f953806d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9540006fd0 tx=0x7f9540008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:45.960+0000 7f9548ff9640 1 -- 172.21.15.16:0/1494067359 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f95440598f0 con 0x7f954c15dc80 2024-04-25T10:05:46.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.142+0000 7f95522b3640 1 -- 172.21.15.16:0/1494067359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f954c0d1780 con 0x7f954c15dc80 2024-04-25T10:05:46.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.143+0000 7f9548ff9640 1 -- 172.21.15.16:0/1494067359 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f95440591f0 con 0x7f954c15dc80 2024-04-25T10:05:46.148 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:05:46.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.145+0000 7f95522b3640 1 -- 172.21.15.16:0/1494067359 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f953806b4d0 msgr2=0x7f953806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.145+0000 7f95522b3640 1 --2- 172.21.15.16:0/1494067359 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f953806b4d0 0x7f953806d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9540006fd0 tx=0x7f9540008040 comp rx=0 tx=0).stop 2024-04-25T10:05:46.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.145+0000 7f95522b3640 1 -- 172.21.15.16:0/1494067359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c15dc80 msgr2=0x7f954c0cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.145+0000 7f95522b3640 1 --2- 172.21.15.16:0/1494067359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c15dc80 0x7f954c0cd500 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9544007e40 tx=0x7f954400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.145+0000 7f95522b3640 1 -- 172.21.15.16:0/1494067359 shutdown_connections 2024-04-25T10:05:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.145+0000 7f95522b3640 1 --2- 172.21.15.16:0/1494067359 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f953806b4d0 0x7f953806d990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.145+0000 7f95522b3640 1 --2- 172.21.15.16:0/1494067359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c15dc80 0x7f954c0cd500 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.145+0000 7f95522b3640 1 --2- 172.21.15.16:0/1494067359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f954c0b97c0 0x7f954c0c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.145+0000 7f95522b3640 1 --2- 172.21.15.16:0/1494067359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f954c0b5a30 0x7f954c1496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.145+0000 7f95522b3640 1 -- 172.21.15.16:0/1494067359 >> 172.21.15.16:0/1494067359 conn(0x7f954c0d04f0 msgr2=0x7f954c0b8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.145+0000 7f95522b3640 1 -- 172.21.15.16:0/1494067359 shutdown_connections 2024-04-25T10:05:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.145+0000 7f95522b3640 1 -- 172.21.15.16:0/1494067359 wait complete. 2024-04-25T10:05:46.161 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:05:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.257+0000 7f9d83c93640 1 Processor -- start 2024-04-25T10:05:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.257+0000 7f9d83c93640 1 -- start start 2024-04-25T10:05:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.258+0000 7f9d83c93640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9d7c0cef40 0x7f9d7c0b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.258+0000 7f9d83c93640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9d7c1618d0 0x7f9d7c0d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.258+0000 7f9d83c93640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d7c162040 con 0x7f9d7c1610b0 2024-04-25T10:05:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.258+0000 7f9d83c93640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d7c0d1ea0 con 0x7f9d7c0cef40 2024-04-25T10:05:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.258+0000 7f9d83c93640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d7c162320 con 0x7f9d7c1618d0 2024-04-25T10:05:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.258+0000 7f9d8122e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9d7c0cef40 0x7f9d7c0b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.258+0000 7f9d81a2f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9d7c1610b0 0x7f9d7c0ce830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51424/0 (socket says 172.21.15.16:51424) 2024-04-25T10:05:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.258+0000 7f9d81a2f640 1 -- 172.21.15.16:0/3916937054 learned_addr learned my addr 172.21.15.16:0/3916937054 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.258+0000 7f9d82230640 1 --2- 172.21.15.16:0/3916937054 >> v2:172.21.15.186:3300/0 conn(0x7f9d7c1618d0 0x7f9d7c0d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.259+0000 7f9d8122e640 1 -- 172.21.15.16:0/3916937054 >> v2:172.21.15.186:3300/0 conn(0x7f9d7c1618d0 msgr2=0x7f9d7c0d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.259+0000 7f9d8122e640 1 --2- 172.21.15.16:0/3916937054 >> v2:172.21.15.186:3300/0 conn(0x7f9d7c1618d0 0x7f9d7c0d2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.259+0000 7f9d8122e640 1 -- 172.21.15.16:0/3916937054 >> v1:172.21.15.16:6789/0 conn(0x7f9d7c1610b0 legacy=0x7f9d7c0ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.259+0000 7f9d8122e640 1 -- 172.21.15.16:0/3916937054 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7c162600 con 0x7f9d7c0cef40 2024-04-25T10:05:46.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.259+0000 7f9d80a2d640 1 -- 172.21.15.16:0/3916937054 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827358852 0 0) 0x7f9d7c162040 con 0x7f9d7c1610b0 2024-04-25T10:05:46.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.259+0000 7f9d82230640 1 --2- 172.21.15.16:0/3916937054 >> v2:172.21.15.186:3300/0 conn(0x7f9d7c1618d0 0x7f9d7c0d2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.259+0000 7f9d8122e640 1 --2- 172.21.15.16:0/3916937054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7c0cef40 0x7f9d7c0b9580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0b9190 tx=0x7f9d6c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70866a4acf40d659 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.259+0000 7f9d80a2d640 1 -- 172.21.15.16:0/3916937054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c037070 con 0x7f9d7c0cef40 2024-04-25T10:05:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.259+0000 7f9d80a2d640 1 -- 172.21.15.16:0/3916937054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c032440 con 0x7f9d7c0cef40 2024-04-25T10:05:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.259+0000 7f9d80a2d640 1 -- 172.21.15.16:0/3916937054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c02ae60 con 0x7f9d7c0cef40 2024-04-25T10:05:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.260+0000 7f9d83c93640 1 -- 172.21.15.16:0/3916937054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7c0cef40 msgr2=0x7f9d7c0b9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.260+0000 7f9d83c93640 1 --2- 172.21.15.16:0/3916937054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7c0cef40 0x7f9d7c0b9580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0b9190 tx=0x7f9d6c02a110 comp rx=0 tx=0).stop 2024-04-25T10:05:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.260+0000 7f9d83c93640 1 -- 172.21.15.16:0/3916937054 shutdown_connections 2024-04-25T10:05:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.260+0000 7f9d83c93640 1 --2- 172.21.15.16:0/3916937054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7c0cef40 0x7f9d7c0b9580 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.260+0000 7f9d83c93640 1 --2- 172.21.15.16:0/3916937054 >> v2:172.21.15.186:3300/0 conn(0x7f9d7c1618d0 0x7f9d7c0d2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.260+0000 7f9d83c93640 1 -- 172.21.15.16:0/3916937054 >> 172.21.15.16:0/3916937054 conn(0x7f9d7c0b5570 msgr2=0x7f9d7c0b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.260+0000 7f9d83c93640 1 -- 172.21.15.16:0/3916937054 shutdown_connections 2024-04-25T10:05:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.260+0000 7f9d83c93640 1 -- 172.21.15.16:0/3916937054 wait complete. 2024-04-25T10:05:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.261+0000 7f9d83c93640 1 Processor -- start 2024-04-25T10:05:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.261+0000 7f9d83c93640 1 -- start start 2024-04-25T10:05:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.262+0000 7f9d83c93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7c0cef40 0x7f9d7c15bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.262+0000 7f9d83c93640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9d7c1610b0 0x7f9d7c159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.262+0000 7f9d83c93640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d7c1618d0 0x7f9d7c06c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.262+0000 7f9d83c93640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c161d40 con 0x7f9d7c0cef40 2024-04-25T10:05:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.262+0000 7f9d83c93640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c0b6c00 con 0x7f9d7c1618d0 2024-04-25T10:05:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.262+0000 7f9d83c93640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c161490 con 0x7f9d7c1610b0 2024-04-25T10:05:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.262+0000 7f9d81a2f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7c0cef40 0x7f9d7c15bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.262+0000 7f9d81a2f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7c0cef40 0x7f9d7c15bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42570/0 (socket says 172.21.15.16:42570) 2024-04-25T10:05:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.262+0000 7f9d81a2f640 1 -- 172.21.15.16:0/3819598123 learned_addr learned my addr 172.21.15.16:0/3819598123 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.262+0000 7f9d8122e640 1 --2- 172.21.15.16:0/3819598123 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9d7c1610b0 0x7f9d7c159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.262+0000 7f9d82230640 1 --2- 172.21.15.16:0/3819598123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d7c1618d0 0x7f9d7c06c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.262+0000 7f9d81a2f640 1 -- 172.21.15.16:0/3819598123 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9d7c1610b0 msgr2=0x7f9d7c159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.262+0000 7f9d81a2f640 1 --2- 172.21.15.16:0/3819598123 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9d7c1610b0 0x7f9d7c159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.262+0000 7f9d81a2f640 1 -- 172.21.15.16:0/3819598123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d7c1618d0 msgr2=0x7f9d7c06c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.262+0000 7f9d81a2f640 1 --2- 172.21.15.16:0/3819598123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d7c1618d0 0x7f9d7c06c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.262+0000 7f9d81a2f640 1 -- 172.21.15.16:0/3819598123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7c159e70 con 0x7f9d7c0cef40 2024-04-25T10:05:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.263+0000 7f9d81a2f640 1 --2- 172.21.15.16:0/3819598123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7c0cef40 0x7f9d7c15bdd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800c4d0 tx=0x7f9d7800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.263+0000 7f9d727fc640 1 -- 172.21.15.16:0/3819598123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78018070 con 0x7f9d7c0cef40 2024-04-25T10:05:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.263+0000 7f9d727fc640 1 -- 172.21.15.16:0/3819598123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d78007500 con 0x7f9d7c0cef40 2024-04-25T10:05:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.263+0000 7f9d727fc640 1 -- 172.21.15.16:0/3819598123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78004a20 con 0x7f9d7c0cef40 2024-04-25T10:05:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.263+0000 7f9d83c93640 1 -- 172.21.15.16:0/3819598123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7c15a110 con 0x7f9d7c0cef40 2024-04-25T10:05:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.263+0000 7f9d83c93640 1 -- 172.21.15.16:0/3819598123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7c15a6d0 con 0x7f9d7c0cef40 2024-04-25T10:05:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.264+0000 7f9d83c93640 1 -- 172.21.15.16:0/3819598123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7c0750b0 con 0x7f9d7c0cef40 2024-04-25T10:05:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.264+0000 7f9d727fc640 1 -- 172.21.15.16:0/3819598123 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d7801b070 con 0x7f9d7c0cef40 2024-04-25T10:05:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.265+0000 7f9d727fc640 1 --2- 172.21.15.16:0/3819598123 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9d6006b4d0 0x7f9d6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.265+0000 7f9d727fc640 1 -- 172.21.15.16:0/3819598123 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d78089570 con 0x7f9d7c0cef40 2024-04-25T10:05:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.265+0000 7f9d8122e640 1 --2- 172.21.15.16:0/3819598123 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9d6006b4d0 0x7f9d6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.266+0000 7f9d8122e640 1 --2- 172.21.15.16:0/3819598123 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9d6006b4d0 0x7f9d6006d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c034000 tx=0x7f9d6c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.269+0000 7f9d727fc640 1 -- 172.21.15.16:0/3819598123 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d780583e0 con 0x7f9d7c0cef40 2024-04-25T10:05:46.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.464+0000 7f9d83c93640 1 -- 172.21.15.16:0/3819598123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9d7c0d1770 con 0x7f9d7c0cef40 2024-04-25T10:05:46.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.464+0000 7f9d727fc640 1 -- 172.21.15.16:0/3819598123 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9d78087e20 con 0x7f9d7c0cef40 2024-04-25T10:05:46.470 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-04-25T10:05:46.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.466+0000 7f9d83c93640 1 -- 172.21.15.16:0/3819598123 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9d6006b4d0 msgr2=0x7f9d6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.466+0000 7f9d83c93640 1 --2- 172.21.15.16:0/3819598123 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9d6006b4d0 0x7f9d6006d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c034000 tx=0x7f9d6c02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.466+0000 7f9d83c93640 1 -- 172.21.15.16:0/3819598123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7c0cef40 msgr2=0x7f9d7c15bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.466+0000 7f9d83c93640 1 --2- 172.21.15.16:0/3819598123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7c0cef40 0x7f9d7c15bdd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800c4d0 tx=0x7f9d7800c990 comp rx=0 tx=0).stop 2024-04-25T10:05:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.466+0000 7f9d83c93640 1 -- 172.21.15.16:0/3819598123 shutdown_connections 2024-04-25T10:05:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.466+0000 7f9d83c93640 1 --2- 172.21.15.16:0/3819598123 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9d6006b4d0 0x7f9d6006d990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.466+0000 7f9d83c93640 1 --2- 172.21.15.16:0/3819598123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d7c1618d0 0x7f9d7c06c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.466+0000 7f9d83c93640 1 --2- 172.21.15.16:0/3819598123 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9d7c1610b0 0x7f9d7c159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.466+0000 7f9d83c93640 1 --2- 172.21.15.16:0/3819598123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7c0cef40 0x7f9d7c15bdd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.466+0000 7f9d83c93640 1 -- 172.21.15.16:0/3819598123 >> 172.21.15.16:0/3819598123 conn(0x7f9d7c0b5570 msgr2=0x7f9d7c15dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.466+0000 7f9d83c93640 1 -- 172.21.15.16:0/3819598123 shutdown_connections 2024-04-25T10:05:46.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.466+0000 7f9d83c93640 1 -- 172.21.15.16:0/3819598123 wait complete. 2024-04-25T10:05:46.482 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T10:05:46.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.483 DEBUG:teuthology.orchestra.run.smithi186:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-25T10:05:46.514 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi186.front.sepia.ceph.com 2024-04-25T10:05:46.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.515 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:05:46.663 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:05:46.663 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:46.681 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:05:46.681 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:46.737 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:05:46.738 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:46.795 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.929+0000 7f8085eee640 1 Processor -- start 2024-04-25T10:05:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.929+0000 7f8085eee640 1 -- start start 2024-04-25T10:05:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.930+0000 7f8085eee640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f808015dce0 0x7f80800d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.930+0000 7f8085eee640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f808015e0c0 con 0x7f80800b5b60 2024-04-25T10:05:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.930+0000 7f8085eee640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80800d2510 con 0x7f80800d35b0 2024-04-25T10:05:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.930+0000 7f8085eee640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80800d27f0 con 0x7f808015dce0 2024-04-25T10:05:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.930+0000 7f807ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f80800b5b60 0x7f80800b6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51426/0 (socket says 172.21.15.16:51426) 2024-04-25T10:05:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.930+0000 7f807ffff640 1 -- 172.21.15.16:0/1690346949 learned_addr learned my addr 172.21.15.16:0/1690346949 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.930+0000 7f807e7fc640 1 -- 172.21.15.16:0/1690346949 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1444211547 0 0) 0x7f808015e0c0 con 0x7f80800b5b60 2024-04-25T10:05:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.930+0000 7f807e7fc640 1 -- 172.21.15.16:0/1690346949 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8068003520 con 0x7f80800b5b60 2024-04-25T10:05:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.930+0000 7f807f7fe640 1 --2- 172.21.15.16:0/1690346949 >> v2:172.21.15.186:3300/0 conn(0x7f808015dce0 0x7f80800d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.930+0000 7f807e7fc640 1 -- 172.21.15.16:0/1690346949 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1574829962 0 0) 0x7f8068003520 con 0x7f80800b5b60 2024-04-25T10:05:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.930+0000 7f807e7fc640 1 -- 172.21.15.16:0/1690346949 >> v2:172.21.15.186:3300/0 conn(0x7f808015dce0 msgr2=0x7f80800d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.930+0000 7f807e7fc640 1 --2- 172.21.15.16:0/1690346949 >> v2:172.21.15.186:3300/0 conn(0x7f808015dce0 0x7f80800d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.930+0000 7f807e7fc640 1 -- 172.21.15.16:0/1690346949 >> v1:172.21.15.186:6789/0 conn(0x7f80800d35b0 legacy=0x7f80800b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.930+0000 7f807e7fc640 1 -- 172.21.15.16:0/1690346949 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80800d2ad0 con 0x7f80800b5b60 2024-04-25T10:05:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.930+0000 7f807e7fc640 1 -- 172.21.15.16:0/1690346949 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f80780019f0 con 0x7f80800b5b60 2024-04-25T10:05:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.931+0000 7f807e7fc640 1 -- 172.21.15.16:0/1690346949 >> v1:172.21.15.16:6789/0 conn(0x7f80800b5b60 legacy=0x7f80800b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.931+0000 7f807e7fc640 1 --2- 172.21.15.16:0/1690346949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80680039b0 0x7f8068003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.931+0000 7f807e7fc640 1 --2- 172.21.15.16:0/1690346949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8068004800 0x7f8068004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.931+0000 7f807e7fc640 1 --2- 172.21.15.16:0/1690346949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80680051c0 0x7f806800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.931+0000 7f807e7fc640 1 -- 172.21.15.16:0/1690346949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80800d27f0 con 0x7f80680051c0 2024-04-25T10:05:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.931+0000 7f807e7fc640 1 -- 172.21.15.16:0/1690346949 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806800c0f0 con 0x7f8068004800 2024-04-25T10:05:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.931+0000 7f807e7fc640 1 -- 172.21.15.16:0/1690346949 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806800c2f0 con 0x7f80680039b0 2024-04-25T10:05:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.931+0000 7f807ffff640 1 --2- 172.21.15.16:0/1690346949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80680051c0 0x7f806800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.931+0000 7f807effd640 1 --2- 172.21.15.16:0/1690346949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8068004800 0x7f8068004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.931+0000 7f807f7fe640 1 --2- 172.21.15.16:0/1690346949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80680039b0 0x7f8068003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.931+0000 7f807ffff640 1 -- 172.21.15.16:0/1690346949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80680039b0 msgr2=0x7f8068003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.931+0000 7f807ffff640 1 --2- 172.21.15.16:0/1690346949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80680039b0 0x7f8068003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.931+0000 7f807ffff640 1 -- 172.21.15.16:0/1690346949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8068004800 msgr2=0x7f8068004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.931+0000 7f807ffff640 1 --2- 172.21.15.16:0/1690346949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8068004800 0x7f8068004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.931+0000 7f807ffff640 1 -- 172.21.15.16:0/1690346949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8068003520 con 0x7f80680051c0 2024-04-25T10:05:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.932+0000 7f807ffff640 1 -- 172.21.15.16:0/1690346949 reap_dead start 2024-04-25T10:05:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.932+0000 7f807ffff640 1 --2- 172.21.15.16:0/1690346949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80680051c0 0x7f806800b9c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f807800a850 tx=0x7f8068005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17eb327144cd037f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.932+0000 7f807e7fc640 1 -- 172.21.15.16:0/1690346949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807800a4a0 con 0x7f80680051c0 2024-04-25T10:05:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.932+0000 7f807e7fc640 1 -- 172.21.15.16:0/1690346949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807802f070 con 0x7f80680051c0 2024-04-25T10:05:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.932+0000 7f807e7fc640 1 -- 172.21.15.16:0/1690346949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807802bcb0 con 0x7f80680051c0 2024-04-25T10:05:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.932+0000 7f8085eee640 1 -- 172.21.15.16:0/1690346949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80680051c0 msgr2=0x7f806800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.932+0000 7f8085eee640 1 --2- 172.21.15.16:0/1690346949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80680051c0 0x7f806800b9c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f807800a850 tx=0x7f8068005830 comp rx=0 tx=0).stop 2024-04-25T10:05:46.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.932+0000 7f8085eee640 1 -- 172.21.15.16:0/1690346949 shutdown_connections 2024-04-25T10:05:46.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.932+0000 7f8085eee640 1 --2- 172.21.15.16:0/1690346949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80680051c0 0x7f806800b9c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.932+0000 7f8085eee640 1 -- 172.21.15.16:0/1690346949 >> 172.21.15.16:0/1690346949 conn(0x7f8080072900 msgr2=0x7f80800d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:46.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.932+0000 7f8085eee640 1 -- 172.21.15.16:0/1690346949 shutdown_connections 2024-04-25T10:05:46.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.932+0000 7f8085eee640 1 -- 172.21.15.16:0/1690346949 wait complete. 2024-04-25T10:05:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.934+0000 7f8085eee640 1 Processor -- start 2024-04-25T10:05:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.934+0000 7f8085eee640 1 -- start start 2024-04-25T10:05:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f8085eee640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80680051c0 0x7f808015e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f8085eee640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8080160160 0x7f80801f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f8085eee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80801605c0 0x7f80801f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f8085eee640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80800b9820 con 0x7f80801605c0 2024-04-25T10:05:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f8085eee640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80800d3090 con 0x7f8080160160 2024-04-25T10:05:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f8085eee640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80800b99a0 con 0x7f80680051c0 2024-04-25T10:05:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f807ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80801605c0 0x7f80801f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f807ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80801605c0 0x7f80801f76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42588/0 (socket says 172.21.15.16:42588) 2024-04-25T10:05:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f807ffff640 1 -- 172.21.15.16:0/520785805 learned_addr learned my addr 172.21.15.16:0/520785805 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f807f7fe640 1 --2- 172.21.15.16:0/520785805 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80680051c0 0x7f808015e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f807effd640 1 --2- 172.21.15.16:0/520785805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8080160160 0x7f80801f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f807ffff640 1 -- 172.21.15.16:0/520785805 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80680051c0 msgr2=0x7f808015e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f807ffff640 1 --2- 172.21.15.16:0/520785805 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80680051c0 0x7f808015e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f807ffff640 1 -- 172.21.15.16:0/520785805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8080160160 msgr2=0x7f80801f7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f807ffff640 1 --2- 172.21.15.16:0/520785805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8080160160 0x7f80801f7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f807ffff640 1 -- 172.21.15.16:0/520785805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8080157000 con 0x7f80801605c0 2024-04-25T10:05:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f807ffff640 1 --2- 172.21.15.16:0/520785805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80801605c0 0x7f80801f76b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f807802b500 tx=0x7f807802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f8067fff640 1 -- 172.21.15.16:0/520785805 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807800a4a0 con 0x7f80801605c0 2024-04-25T10:05:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f8067fff640 1 -- 172.21.15.16:0/520785805 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8078034470 con 0x7f80801605c0 2024-04-25T10:05:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.935+0000 7f8085eee640 1 -- 172.21.15.16:0/520785805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80800ccf90 con 0x7f80801605c0 2024-04-25T10:05:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.936+0000 7f8067fff640 1 -- 172.21.15.16:0/520785805 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078034820 con 0x7f80801605c0 2024-04-25T10:05:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.936+0000 7f8085eee640 1 -- 172.21.15.16:0/520785805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80801572a0 con 0x7f80801605c0 2024-04-25T10:05:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.937+0000 7f8085eee640 1 -- 172.21.15.16:0/520785805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80800749b0 con 0x7f80801605c0 2024-04-25T10:05:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.937+0000 7f8067fff640 1 -- 172.21.15.16:0/520785805 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f80780349c0 con 0x7f80801605c0 2024-04-25T10:05:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.937+0000 7f8067fff640 1 --2- 172.21.15.16:0/520785805 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f806c06b4d0 0x7f806c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.938+0000 7f8067fff640 1 -- 172.21.15.16:0/520785805 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80780aa750 con 0x7f80801605c0 2024-04-25T10:05:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.938+0000 7f807f7fe640 1 --2- 172.21.15.16:0/520785805 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f806c06b4d0 0x7f806c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.938+0000 7f807f7fe640 1 --2- 172.21.15.16:0/520785805 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f806c06b4d0 0x7f806c06d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8070006fd0 tx=0x7f8070008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:46.942+0000 7f8067fff640 1 -- 172.21.15.16:0/520785805 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8078078ef0 con 0x7f80801605c0 2024-04-25T10:05:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.126+0000 7f8085eee640 1 -- 172.21.15.16:0/520785805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80800cd4b0 con 0x7f80801605c0 2024-04-25T10:05:47.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.127+0000 7f8067fff640 1 -- 172.21.15.16:0/520785805 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f807807cf90 con 0x7f80801605c0 2024-04-25T10:05:47.133 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:47.133 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:12.449183+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.129+0000 7f8085eee640 1 -- 172.21.15.16:0/520785805 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f806c06b4d0 msgr2=0x7f806c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.129+0000 7f8085eee640 1 --2- 172.21.15.16:0/520785805 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f806c06b4d0 0x7f806c06d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8070006fd0 tx=0x7f8070008040 comp rx=0 tx=0).stop 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.129+0000 7f8085eee640 1 -- 172.21.15.16:0/520785805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80801605c0 msgr2=0x7f80801f76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.129+0000 7f8085eee640 1 --2- 172.21.15.16:0/520785805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80801605c0 0x7f80801f76b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f807802b500 tx=0x7f807802d070 comp rx=0 tx=0).stop 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.129+0000 7f8085eee640 1 -- 172.21.15.16:0/520785805 shutdown_connections 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.129+0000 7f8085eee640 1 --2- 172.21.15.16:0/520785805 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f806c06b4d0 0x7f806c06d990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.129+0000 7f8085eee640 1 --2- 172.21.15.16:0/520785805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80801605c0 0x7f80801f76b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.129+0000 7f8085eee640 1 --2- 172.21.15.16:0/520785805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8080160160 0x7f80801f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.129+0000 7f8085eee640 1 --2- 172.21.15.16:0/520785805 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80680051c0 0x7f808015e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.129+0000 7f8085eee640 1 -- 172.21.15.16:0/520785805 >> 172.21.15.16:0/520785805 conn(0x7f8080072900 msgr2=0x7f80800b4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.129+0000 7f8085eee640 1 -- 172.21.15.16:0/520785805 shutdown_connections 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.129+0000 7f8085eee640 1 -- 172.21.15.16:0/520785805 wait complete. 2024-04-25T10:05:47.145 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:08.724537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:09.793105+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:47.146 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:47.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.244+0000 7f736259b640 1 Processor -- start 2024-04-25T10:05:47.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.245+0000 7f736259b640 1 -- start start 2024-04-25T10:05:47.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.245+0000 7f736259b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f735c0c8070 0x7f735c1668a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.245+0000 7f736259b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735c09ab60 con 0x7f735c15bdb0 2024-04-25T10:05:47.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.245+0000 7f736259b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735c09ae30 con 0x7f735c0cf830 2024-04-25T10:05:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.245+0000 7f736259b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f735c09b110 con 0x7f735c0c8070 2024-04-25T10:05:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.245+0000 7f735bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f735c15bdb0 0x7f735c09a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51434/0 (socket says 172.21.15.16:51434) 2024-04-25T10:05:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.245+0000 7f735bfff640 1 -- 172.21.15.16:0/2044892639 learned_addr learned my addr 172.21.15.16:0/2044892639 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.246+0000 7f7360b38640 1 --2- 172.21.15.16:0/2044892639 >> v2:172.21.15.186:3300/0 conn(0x7f735c0c8070 0x7f735c1668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.246+0000 7f735affd640 1 -- 172.21.15.16:0/2044892639 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 678198304 0 0) 0x7f735c09ab60 con 0x7f735c15bdb0 2024-04-25T10:05:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.246+0000 7f735affd640 1 -- 172.21.15.16:0/2044892639 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7348003520 con 0x7f735c15bdb0 2024-04-25T10:05:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.246+0000 7f735affd640 1 -- 172.21.15.16:0/2044892639 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151931175 0 0) 0x7f735c09ae30 con 0x7f735c0cf830 2024-04-25T10:05:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.246+0000 7f735affd640 1 -- 172.21.15.16:0/2044892639 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f735c09ab60 con 0x7f735c0cf830 2024-04-25T10:05:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.246+0000 7f7360b38640 1 -- 172.21.15.16:0/2044892639 >> v1:172.21.15.186:6789/0 conn(0x7f735c0cf830 legacy=0x7f735c0cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.246+0000 7f7360b38640 1 -- 172.21.15.16:0/2044892639 >> v1:172.21.15.16:6789/0 conn(0x7f735c15bdb0 legacy=0x7f735c09a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.246+0000 7f7360b38640 1 -- 172.21.15.16:0/2044892639 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c0c5580 con 0x7f735c0c8070 2024-04-25T10:05:47.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.247+0000 7f7360b38640 1 --2- 172.21.15.16:0/2044892639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f735c0c8070 0x7f735c1668a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7354009d90 tx=0x7f735402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9546110f80fe41a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.247+0000 7f735affd640 1 -- 172.21.15.16:0/2044892639 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354007740 con 0x7f735c0c8070 2024-04-25T10:05:47.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.247+0000 7f735affd640 1 -- 172.21.15.16:0/2044892639 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73540078e0 con 0x7f735c0c8070 2024-04-25T10:05:47.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.247+0000 7f735affd640 1 -- 172.21.15.16:0/2044892639 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354007c90 con 0x7f735c0c8070 2024-04-25T10:05:47.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.248+0000 7f736259b640 1 -- 172.21.15.16:0/2044892639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f735c0c8070 msgr2=0x7f735c1668a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.248+0000 7f736259b640 1 --2- 172.21.15.16:0/2044892639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f735c0c8070 0x7f735c1668a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7354009d90 tx=0x7f735402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.248+0000 7f736259b640 1 -- 172.21.15.16:0/2044892639 shutdown_connections 2024-04-25T10:05:47.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.248+0000 7f736259b640 1 --2- 172.21.15.16:0/2044892639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f735c0c8070 0x7f735c1668a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.248+0000 7f736259b640 1 -- 172.21.15.16:0/2044892639 >> 172.21.15.16:0/2044892639 conn(0x7f735c0b33f0 msgr2=0x7f735c0b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:47.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.248+0000 7f736259b640 1 -- 172.21.15.16:0/2044892639 shutdown_connections 2024-04-25T10:05:47.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.248+0000 7f736259b640 1 -- 172.21.15.16:0/2044892639 wait complete. 2024-04-25T10:05:47.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.249+0000 7f736259b640 1 Processor -- start 2024-04-25T10:05:47.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.249+0000 7f736259b640 1 -- start start 2024-04-25T10:05:47.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.250+0000 7f736259b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f735c0c8070 0x7f735c0b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.250+0000 7f736259b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f735c0cf830 0x7f735c0cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.250+0000 7f736259b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f735c15bdb0 0x7f735c159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.250+0000 7f736259b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c15c190 con 0x7f735c0c8070 2024-04-25T10:05:47.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.250+0000 7f736259b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c0c5bd0 con 0x7f735c15bdb0 2024-04-25T10:05:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.250+0000 7f736259b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735c0d05f0 con 0x7f735c0cf830 2024-04-25T10:05:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.250+0000 7f735bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f735c0c8070 0x7f735c0b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.250+0000 7f735bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f735c0c8070 0x7f735c0b44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42590/0 (socket says 172.21.15.16:42590) 2024-04-25T10:05:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.250+0000 7f735bfff640 1 -- 172.21.15.16:0/2910140107 learned_addr learned my addr 172.21.15.16:0/2910140107 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.250+0000 7f735b7fe640 1 --2- 172.21.15.16:0/2910140107 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f735c0cf830 0x7f735c0cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.250+0000 7f735bfff640 1 -- 172.21.15.16:0/2910140107 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f735c0cf830 msgr2=0x7f735c0cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.250+0000 7f735bfff640 1 --2- 172.21.15.16:0/2910140107 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f735c0cf830 0x7f735c0cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.250+0000 7f735bfff640 1 -- 172.21.15.16:0/2910140107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f735c15bdb0 msgr2=0x7f735c159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.250+0000 7f735bfff640 1 --2- 172.21.15.16:0/2910140107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f735c15bdb0 0x7f735c159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.250+0000 7f735bfff640 1 -- 172.21.15.16:0/2910140107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c15a1b0 con 0x7f735c0c8070 2024-04-25T10:05:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.250+0000 7f735bfff640 1 --2- 172.21.15.16:0/2910140107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f735c0c8070 0x7f735c0b44f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f734c00c430 tx=0x7f734c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.250+0000 7f7358ff9640 1 -- 172.21.15.16:0/2910140107 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c017070 con 0x7f735c0c8070 2024-04-25T10:05:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.251+0000 7f7358ff9640 1 -- 172.21.15.16:0/2910140107 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734c005ce0 con 0x7f735c0c8070 2024-04-25T10:05:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.251+0000 7f736259b640 1 -- 172.21.15.16:0/2910140107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735c0b4ad0 con 0x7f735c0c8070 2024-04-25T10:05:47.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.251+0000 7f7358ff9640 1 -- 172.21.15.16:0/2910140107 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c0072f0 con 0x7f735c0c8070 2024-04-25T10:05:47.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.251+0000 7f736259b640 1 -- 172.21.15.16:0/2910140107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735c0b5090 con 0x7f735c0c8070 2024-04-25T10:05:47.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.252+0000 7f736259b640 1 -- 172.21.15.16:0/2910140107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735c0750b0 con 0x7f735c0c8070 2024-04-25T10:05:47.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.252+0000 7f7358ff9640 1 -- 172.21.15.16:0/2910140107 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f734c01a070 con 0x7f735c0c8070 2024-04-25T10:05:47.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.252+0000 7f7358ff9640 1 --2- 172.21.15.16:0/2910140107 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f733c06b4d0 0x7f733c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.252+0000 7f7358ff9640 1 -- 172.21.15.16:0/2910140107 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f734c089730 con 0x7f735c0c8070 2024-04-25T10:05:47.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.253+0000 7f735b7fe640 1 --2- 172.21.15.16:0/2910140107 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f733c06b4d0 0x7f733c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.253+0000 7f735b7fe640 1 --2- 172.21.15.16:0/2910140107 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f733c06b4d0 0x7f733c06d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7344007fd0 tx=0x7f7344009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.257+0000 7f7358ff9640 1 -- 172.21.15.16:0/2910140107 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f734c058530 con 0x7f735c0c8070 2024-04-25T10:05:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.435+0000 7f736259b640 1 -- 172.21.15.16:0/2910140107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f735c070f80 con 0x7f735c0c8070 2024-04-25T10:05:47.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.435+0000 7f7358ff9640 1 -- 172.21.15.16:0/2910140107 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f734c057e30 con 0x7f735c0c8070 2024-04-25T10:05:47.441 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:05:47.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.437+0000 7f736259b640 1 -- 172.21.15.16:0/2910140107 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f733c06b4d0 msgr2=0x7f733c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.437+0000 7f736259b640 1 --2- 172.21.15.16:0/2910140107 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f733c06b4d0 0x7f733c06d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7344007fd0 tx=0x7f7344009040 comp rx=0 tx=0).stop 2024-04-25T10:05:47.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.437+0000 7f736259b640 1 -- 172.21.15.16:0/2910140107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f735c0c8070 msgr2=0x7f735c0b44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.437+0000 7f736259b640 1 --2- 172.21.15.16:0/2910140107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f735c0c8070 0x7f735c0b44f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f734c00c430 tx=0x7f734c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.437+0000 7f736259b640 1 -- 172.21.15.16:0/2910140107 shutdown_connections 2024-04-25T10:05:47.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.437+0000 7f736259b640 1 --2- 172.21.15.16:0/2910140107 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f733c06b4d0 0x7f733c06d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.437+0000 7f736259b640 1 --2- 172.21.15.16:0/2910140107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f735c15bdb0 0x7f735c159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.437+0000 7f736259b640 1 --2- 172.21.15.16:0/2910140107 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f735c0cf830 0x7f735c0cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.437+0000 7f736259b640 1 --2- 172.21.15.16:0/2910140107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f735c0c8070 0x7f735c0b44f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.437+0000 7f736259b640 1 -- 172.21.15.16:0/2910140107 >> 172.21.15.16:0/2910140107 conn(0x7f735c0b33f0 msgr2=0x7f735c15a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.437+0000 7f736259b640 1 -- 172.21.15.16:0/2910140107 shutdown_connections 2024-04-25T10:05:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.437+0000 7f736259b640 1 -- 172.21.15.16:0/2910140107 wait complete. 2024-04-25T10:05:47.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.552+0000 7fd0d58a1640 1 Processor -- start 2024-04-25T10:05:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.552+0000 7fd0d58a1640 1 -- start start 2024-04-25T10:05:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.553+0000 7fd0d58a1640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd0d015dc80 0x7fd0d0161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.553+0000 7fd0d58a1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd0d00b5c00 0x7fd0d00d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.553+0000 7fd0d58a1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d01627a0 con 0x7fd0d00b97c0 2024-04-25T10:05:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.553+0000 7fd0d58a1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d01621e0 con 0x7fd0d00b5c00 2024-04-25T10:05:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.553+0000 7fd0d58a1640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d01624c0 con 0x7fd0d015dc80 2024-04-25T10:05:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.553+0000 7fd0cf7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd0d00b5c00 0x7fd0d00d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.553+0000 7fd0cf7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd0d00b5c00 0x7fd0d00d2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42606/0 (socket says 172.21.15.16:42606) 2024-04-25T10:05:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.553+0000 7fd0cf7fe640 1 -- 172.21.15.16:0/2274464933 learned_addr learned my addr 172.21.15.16:0/2274464933 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.553+0000 7fd0c7fff640 1 --1- 172.21.15.16:0/2274464933 >> v1:172.21.15.16:6789/0 conn(0x7fd0d00b97c0 0x7fd0d00b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51446/0 (socket says 172.21.15.16:51446) 2024-04-25T10:05:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.553+0000 7fd0ceffd640 1 --2- 172.21.15.16:0/2274464933 >> v2:172.21.15.186:3300/0 conn(0x7fd0d015dc80 0x7fd0d0161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.553+0000 7fd0cf7fe640 1 -- 172.21.15.16:0/2274464933 >> v2:172.21.15.186:3300/0 conn(0x7fd0d015dc80 msgr2=0x7fd0d0161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.553+0000 7fd0cf7fe640 1 --2- 172.21.15.16:0/2274464933 >> v2:172.21.15.186:3300/0 conn(0x7fd0d015dc80 0x7fd0d0161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.553+0000 7fd0cf7fe640 1 -- 172.21.15.16:0/2274464933 >> v1:172.21.15.16:6789/0 conn(0x7fd0d00b97c0 legacy=0x7fd0d00b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.553+0000 7fd0ce7fc640 1 -- 172.21.15.16:0/2274464933 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783593384 0 0) 0x7fd0d01627a0 con 0x7fd0d00b97c0 2024-04-25T10:05:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.553+0000 7fd0cf7fe640 1 -- 172.21.15.16:0/2274464933 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d00d1c70 con 0x7fd0d00b5c00 2024-04-25T10:05:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.554+0000 7fd0cf7fe640 1 --2- 172.21.15.16:0/2274464933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0d00b5c00 0x7fd0d00d2470 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd0d006dab0 tx=0x7fd0c802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d3270ac8068f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.554+0000 7fd0ce7fc640 1 -- 172.21.15.16:0/2274464933 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8036070 con 0x7fd0d00b5c00 2024-04-25T10:05:47.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.554+0000 7fd0ce7fc640 1 -- 172.21.15.16:0/2274464933 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c8031600 con 0x7fd0d00b5c00 2024-04-25T10:05:47.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.554+0000 7fd0ce7fc640 1 -- 172.21.15.16:0/2274464933 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c80319d0 con 0x7fd0d00b5c00 2024-04-25T10:05:47.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.555+0000 7fd0d58a1640 1 -- 172.21.15.16:0/2274464933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0d00b5c00 msgr2=0x7fd0d00d2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.555+0000 7fd0d58a1640 1 --2- 172.21.15.16:0/2274464933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0d00b5c00 0x7fd0d00d2470 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd0d006dab0 tx=0x7fd0c802a730 comp rx=0 tx=0).stop 2024-04-25T10:05:47.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.555+0000 7fd0d58a1640 1 -- 172.21.15.16:0/2274464933 shutdown_connections 2024-04-25T10:05:47.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.555+0000 7fd0d58a1640 1 --2- 172.21.15.16:0/2274464933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0d00b5c00 0x7fd0d00d2470 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.555+0000 7fd0d58a1640 1 --2- 172.21.15.16:0/2274464933 >> v2:172.21.15.186:3300/0 conn(0x7fd0d015dc80 0x7fd0d0161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.555+0000 7fd0d58a1640 1 -- 172.21.15.16:0/2274464933 >> 172.21.15.16:0/2274464933 conn(0x7fd0d00d04f0 msgr2=0x7fd0d00d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:47.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.555+0000 7fd0d58a1640 1 -- 172.21.15.16:0/2274464933 shutdown_connections 2024-04-25T10:05:47.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.555+0000 7fd0d58a1640 1 -- 172.21.15.16:0/2274464933 wait complete. 2024-04-25T10:05:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.556+0000 7fd0d58a1640 1 Processor -- start 2024-04-25T10:05:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.556+0000 7fd0d58a1640 1 -- start start 2024-04-25T10:05:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0d58a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0d00b5c00 0x7fd0d0157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0d58a1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0d00b97c0 0x7fd0d01f7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0d58a1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0d015dc80 0x7fd0d00c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0d58a1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d0161f30 con 0x7fd0d00b5c00 2024-04-25T10:05:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0d58a1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d0161db0 con 0x7fd0d015dc80 2024-04-25T10:05:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0d58a1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d00d29b0 con 0x7fd0d00b97c0 2024-04-25T10:05:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0d00b5c00 0x7fd0d0157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0d00b5c00 0x7fd0d0157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42608/0 (socket says 172.21.15.16:42608) 2024-04-25T10:05:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0ceffd640 1 -- 172.21.15.16:0/449062707 learned_addr learned my addr 172.21.15.16:0/449062707 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0cf7fe640 1 --2- 172.21.15.16:0/449062707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0d015dc80 0x7fd0d00c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0c7fff640 1 --2- 172.21.15.16:0/449062707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0d00b97c0 0x7fd0d01f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0ceffd640 1 -- 172.21.15.16:0/449062707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0d00b97c0 msgr2=0x7fd0d01f7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0ceffd640 1 --2- 172.21.15.16:0/449062707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0d00b97c0 0x7fd0d01f7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0ceffd640 1 -- 172.21.15.16:0/449062707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0d015dc80 msgr2=0x7fd0d00c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0ceffd640 1 --2- 172.21.15.16:0/449062707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0d015dc80 0x7fd0d00c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0ceffd640 1 -- 172.21.15.16:0/449062707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d00cbf90 con 0x7fd0d00b5c00 2024-04-25T10:05:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0ceffd640 1 --2- 172.21.15.16:0/449062707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0d00b5c00 0x7fd0d0157b00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000d5f0 tx=0x7fd0c000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0ccff9640 1 -- 172.21.15.16:0/449062707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00125f0 con 0x7fd0d00b5c00 2024-04-25T10:05:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0ccff9640 1 -- 172.21.15.16:0/449062707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c0019070 con 0x7fd0d00b5c00 2024-04-25T10:05:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.557+0000 7fd0ccff9640 1 -- 172.21.15.16:0/449062707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00129d0 con 0x7fd0d00b5c00 2024-04-25T10:05:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.558+0000 7fd0d58a1640 1 -- 172.21.15.16:0/449062707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d00cc270 con 0x7fd0d00b5c00 2024-04-25T10:05:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.558+0000 7fd0d58a1640 1 -- 172.21.15.16:0/449062707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d01f7330 con 0x7fd0d00b5c00 2024-04-25T10:05:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.558+0000 7fd0d58a1640 1 -- 172.21.15.16:0/449062707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d00750b0 con 0x7fd0d00b5c00 2024-04-25T10:05:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.559+0000 7fd0ccff9640 1 -- 172.21.15.16:0/449062707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd0c0012b70 con 0x7fd0d00b5c00 2024-04-25T10:05:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.559+0000 7fd0ccff9640 1 --2- 172.21.15.16:0/449062707 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd0b006b4d0 0x7fd0b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.560+0000 7fd0ccff9640 1 -- 172.21.15.16:0/449062707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0c008a5c0 con 0x7fd0d00b5c00 2024-04-25T10:05:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.560+0000 7fd0c7fff640 1 --2- 172.21.15.16:0/449062707 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd0b006b4d0 0x7fd0b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.560+0000 7fd0c7fff640 1 --2- 172.21.15.16:0/449062707 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd0b006b4d0 0x7fd0b006d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0161090 tx=0x7fd0b8007b70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.564+0000 7fd0ccff9640 1 -- 172.21.15.16:0/449062707 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd0c0059440 con 0x7fd0d00b5c00 2024-04-25T10:05:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.744+0000 7fd0d58a1640 1 -- 172.21.15.16:0/449062707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd0d00d1780 con 0x7fd0d00b5c00 2024-04-25T10:05:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.744+0000 7fd0ccff9640 1 -- 172.21.15.16:0/449062707 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd0c0058d40 con 0x7fd0d00b5c00 2024-04-25T10:05:47.750 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:05:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.746+0000 7fd0d58a1640 1 -- 172.21.15.16:0/449062707 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd0b006b4d0 msgr2=0x7fd0b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.746+0000 7fd0d58a1640 1 --2- 172.21.15.16:0/449062707 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd0b006b4d0 0x7fd0b006d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0161090 tx=0x7fd0b8007b70 comp rx=0 tx=0).stop 2024-04-25T10:05:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.746+0000 7fd0d58a1640 1 -- 172.21.15.16:0/449062707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0d00b5c00 msgr2=0x7fd0d0157b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.746+0000 7fd0d58a1640 1 --2- 172.21.15.16:0/449062707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0d00b5c00 0x7fd0d0157b00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000d5f0 tx=0x7fd0c000dab0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.746+0000 7fd0d58a1640 1 -- 172.21.15.16:0/449062707 shutdown_connections 2024-04-25T10:05:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.746+0000 7fd0d58a1640 1 --2- 172.21.15.16:0/449062707 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd0b006b4d0 0x7fd0b006d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.746+0000 7fd0d58a1640 1 --2- 172.21.15.16:0/449062707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0d015dc80 0x7fd0d00c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.746+0000 7fd0d58a1640 1 --2- 172.21.15.16:0/449062707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0d00b97c0 0x7fd0d01f7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.746+0000 7fd0d58a1640 1 --2- 172.21.15.16:0/449062707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0d00b5c00 0x7fd0d0157b00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.746+0000 7fd0d58a1640 1 -- 172.21.15.16:0/449062707 >> 172.21.15.16:0/449062707 conn(0x7fd0d00d04f0 msgr2=0x7fd0d015ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.746+0000 7fd0d58a1640 1 -- 172.21.15.16:0/449062707 shutdown_connections 2024-04-25T10:05:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.746+0000 7fd0d58a1640 1 -- 172.21.15.16:0/449062707 wait complete. 2024-04-25T10:05:47.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:47.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.862+0000 7f3a3cccd640 1 Processor -- start 2024-04-25T10:05:47.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.862+0000 7f3a3cccd640 1 -- start start 2024-04-25T10:05:47.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.863+0000 7f3a3cccd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a380d3b90 0x7f3a380d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.863+0000 7f3a3cccd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3a380d0f60 0x7f3a380b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.863+0000 7f3a3cccd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a380d4090 con 0x7f3a3815dca0 2024-04-25T10:05:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.863+0000 7f3a3cccd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a380b6b20 con 0x7f3a380d3b90 2024-04-25T10:05:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.863+0000 7f3a3cccd640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a380d1390 con 0x7f3a380d0f60 2024-04-25T10:05:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.863+0000 7f3a2ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a380d3b90 0x7f3a380d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.863+0000 7f3a2ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a380d3b90 0x7f3a380d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42620/0 (socket says 172.21.15.16:42620) 2024-04-25T10:05:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.863+0000 7f3a2ffff640 1 -- 172.21.15.16:0/1771115066 learned_addr learned my addr 172.21.15.16:0/1771115066 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.863+0000 7f3a2ffff640 1 -- 172.21.15.16:0/1771115066 >> v2:172.21.15.186:3300/0 conn(0x7f3a380d0f60 msgr2=0x7f3a380b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.863+0000 7f3a36d9d640 1 --2- 172.21.15.16:0/1771115066 >> v2:172.21.15.186:3300/0 conn(0x7f3a380d0f60 0x7f3a380b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.863+0000 7f3a2ffff640 1 --2- 172.21.15.16:0/1771115066 >> v2:172.21.15.186:3300/0 conn(0x7f3a380d0f60 0x7f3a380b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.863+0000 7f3a2ffff640 1 -- 172.21.15.16:0/1771115066 >> v1:172.21.15.186:6789/0 conn(0x7f3a3815dca0 legacy=0x7f3a380ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:47.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.863+0000 7f3a2ffff640 1 -- 172.21.15.16:0/1771115066 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a380d4370 con 0x7f3a380d3b90 2024-04-25T10:05:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.864+0000 7f3a2ffff640 1 --2- 172.21.15.16:0/1771115066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a380d3b90 0x7f3a380d0a20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3a380b9640 tx=0x7f3a2002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38183a009cafc27b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.864+0000 7f3a35d9b640 1 -- 172.21.15.16:0/1771115066 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20037070 con 0x7f3a380d3b90 2024-04-25T10:05:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.864+0000 7f3a35d9b640 1 -- 172.21.15.16:0/1771115066 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a20032440 con 0x7f3a380d3b90 2024-04-25T10:05:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.864+0000 7f3a35d9b640 1 -- 172.21.15.16:0/1771115066 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20032810 con 0x7f3a380d3b90 2024-04-25T10:05:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.864+0000 7f3a3cccd640 1 -- 172.21.15.16:0/1771115066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a380d3b90 msgr2=0x7f3a380d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.864+0000 7f3a3cccd640 1 --2- 172.21.15.16:0/1771115066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a380d3b90 0x7f3a380d0a20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3a380b9640 tx=0x7f3a2002a270 comp rx=0 tx=0).stop 2024-04-25T10:05:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.864+0000 7f3a3cccd640 1 -- 172.21.15.16:0/1771115066 shutdown_connections 2024-04-25T10:05:47.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.864+0000 7f3a3cccd640 1 --2- 172.21.15.16:0/1771115066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a380d3b90 0x7f3a380d0a20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.864+0000 7f3a3cccd640 1 --2- 172.21.15.16:0/1771115066 >> v2:172.21.15.186:3300/0 conn(0x7f3a380d0f60 0x7f3a380b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.864+0000 7f3a3cccd640 1 -- 172.21.15.16:0/1771115066 >> 172.21.15.16:0/1771115066 conn(0x7f3a380726c0 msgr2=0x7f3a38072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.864+0000 7f3a3cccd640 1 -- 172.21.15.16:0/1771115066 shutdown_connections 2024-04-25T10:05:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.865+0000 7f3a3cccd640 1 -- 172.21.15.16:0/1771115066 wait complete. 2024-04-25T10:05:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.866+0000 7f3a3cccd640 1 Processor -- start 2024-04-25T10:05:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.866+0000 7f3a3cccd640 1 -- start start 2024-04-25T10:05:47.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.866+0000 7f3a3cccd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a380d0f60 0x7f3a380cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.866+0000 7f3a3cccd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a380d3b90 0x7f3a380ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.866+0000 7f3a3cccd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a3815dca0 0x7f3a380cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a3cccd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a380cf050 con 0x7f3a380d3b90 2024-04-25T10:05:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a3cccd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a380b6d50 con 0x7f3a3815dca0 2024-04-25T10:05:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a3cccd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3815e080 con 0x7f3a380d0f60 2024-04-25T10:05:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a2ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a380d3b90 0x7f3a380ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a2ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a380d3b90 0x7f3a380ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42630/0 (socket says 172.21.15.16:42630) 2024-04-25T10:05:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a2ffff640 1 -- 172.21.15.16:0/3817686907 learned_addr learned my addr 172.21.15.16:0/3817686907 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a3659c640 1 --2- 172.21.15.16:0/3817686907 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a380d0f60 0x7f3a380cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a36d9d640 1 --2- 172.21.15.16:0/3817686907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a3815dca0 0x7f3a380cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a2ffff640 1 -- 172.21.15.16:0/3817686907 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a380d0f60 msgr2=0x7f3a380cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a2ffff640 1 --2- 172.21.15.16:0/3817686907 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a380d0f60 0x7f3a380cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a2ffff640 1 -- 172.21.15.16:0/3817686907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a3815dca0 msgr2=0x7f3a380cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a2ffff640 1 --2- 172.21.15.16:0/3817686907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a3815dca0 0x7f3a380cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a2ffff640 1 -- 172.21.15.16:0/3817686907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a380cb340 con 0x7f3a380d3b90 2024-04-25T10:05:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a2ffff640 1 --2- 172.21.15.16:0/3817686907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a380d3b90 0x7f3a380ccb40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3a200097f0 tx=0x7f3a200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a2effd640 1 -- 172.21.15.16:0/3817686907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20037040 con 0x7f3a380d3b90 2024-04-25T10:05:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a2effd640 1 -- 172.21.15.16:0/3817686907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a20004230 con 0x7f3a380d3b90 2024-04-25T10:05:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a3cccd640 1 -- 172.21.15.16:0/3817686907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a380cdac0 con 0x7f3a380d3b90 2024-04-25T10:05:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a2effd640 1 -- 172.21.15.16:0/3817686907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20030620 con 0x7f3a380d3b90 2024-04-25T10:05:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.867+0000 7f3a3cccd640 1 -- 172.21.15.16:0/3817686907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a380c7c90 con 0x7f3a380d3b90 2024-04-25T10:05:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.868+0000 7f3a3cccd640 1 -- 172.21.15.16:0/3817686907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a380750b0 con 0x7f3a380d3b90 2024-04-25T10:05:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.869+0000 7f3a2effd640 1 -- 172.21.15.16:0/3817686907 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a200307c0 con 0x7f3a380d3b90 2024-04-25T10:05:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.869+0000 7f3a2effd640 1 --2- 172.21.15.16:0/3817686907 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a1806b4d0 0x7f3a1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.869+0000 7f3a2effd640 1 -- 172.21.15.16:0/3817686907 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a200ab210 con 0x7f3a380d3b90 2024-04-25T10:05:47.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.870+0000 7f3a3659c640 1 --2- 172.21.15.16:0/3817686907 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a1806b4d0 0x7f3a1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.870+0000 7f3a3659c640 1 --2- 172.21.15.16:0/3817686907 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a1806b4d0 0x7f3a1806d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3a28006fd0 tx=0x7f3a28008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:47.874+0000 7f3a2effd640 1 -- 172.21.15.16:0/3817686907 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a2007a090 con 0x7f3a380d3b90 2024-04-25T10:05:48.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.056+0000 7f3a3cccd640 1 -- 172.21.15.16:0/3817686907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3a380b51a0 con 0x7f3a380d3b90 2024-04-25T10:05:48.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.057+0000 7f3a2effd640 1 -- 172.21.15.16:0/3817686907 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3a20079990 con 0x7f3a380d3b90 2024-04-25T10:05:48.062 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:05:48.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.059+0000 7f3a3cccd640 1 -- 172.21.15.16:0/3817686907 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a1806b4d0 msgr2=0x7f3a1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.059+0000 7f3a3cccd640 1 --2- 172.21.15.16:0/3817686907 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a1806b4d0 0x7f3a1806d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3a28006fd0 tx=0x7f3a28008040 comp rx=0 tx=0).stop 2024-04-25T10:05:48.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.059+0000 7f3a3cccd640 1 -- 172.21.15.16:0/3817686907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a380d3b90 msgr2=0x7f3a380ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.059+0000 7f3a3cccd640 1 --2- 172.21.15.16:0/3817686907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a380d3b90 0x7f3a380ccb40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3a200097f0 tx=0x7f3a200046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.059+0000 7f3a3cccd640 1 -- 172.21.15.16:0/3817686907 shutdown_connections 2024-04-25T10:05:48.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.059+0000 7f3a3cccd640 1 --2- 172.21.15.16:0/3817686907 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a1806b4d0 0x7f3a1806d990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.059+0000 7f3a3cccd640 1 --2- 172.21.15.16:0/3817686907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a3815dca0 0x7f3a380cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.059+0000 7f3a3cccd640 1 --2- 172.21.15.16:0/3817686907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a380d3b90 0x7f3a380ccb40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.059+0000 7f3a3cccd640 1 --2- 172.21.15.16:0/3817686907 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a380d0f60 0x7f3a380cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.059+0000 7f3a3cccd640 1 -- 172.21.15.16:0/3817686907 >> 172.21.15.16:0/3817686907 conn(0x7f3a380726c0 msgr2=0x7f3a380d2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.059+0000 7f3a3cccd640 1 -- 172.21.15.16:0/3817686907 shutdown_connections 2024-04-25T10:05:48.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.059+0000 7f3a3cccd640 1 -- 172.21.15.16:0/3817686907 wait complete. 2024-04-25T10:05:48.075 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:05:48.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.172+0000 7f369895c640 1 Processor -- start 2024-04-25T10:05:48.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.172+0000 7f369895c640 1 -- start start 2024-04-25T10:05:48.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.173+0000 7f369895c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3694161090 0x7f3694161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.173+0000 7f369895c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36940ceeb0 0x7f36940d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.173+0000 7f369895c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3694161f80 con 0x7f36940b9720 2024-04-25T10:05:48.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.173+0000 7f369895c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36940d21a0 con 0x7f3694161090 2024-04-25T10:05:48.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.173+0000 7f369895c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3694162260 con 0x7f36940ceeb0 2024-04-25T10:05:48.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.173+0000 7f369259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3694161090 0x7f3694161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.173+0000 7f369259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3694161090 0x7f3694161730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42638/0 (socket says 172.21.15.16:42638) 2024-04-25T10:05:48.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.173+0000 7f369259c640 1 -- 172.21.15.16:0/1450763725 learned_addr learned my addr 172.21.15.16:0/1450763725 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:48.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.173+0000 7f3692d9d640 1 --2- 172.21.15.16:0/1450763725 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36940ceeb0 0x7f36940d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.174+0000 7f369259c640 1 -- 172.21.15.16:0/1450763725 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36940ceeb0 msgr2=0x7f36940d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.174+0000 7f369259c640 1 --2- 172.21.15.16:0/1450763725 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36940ceeb0 0x7f36940d2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.174+0000 7f369259c640 1 -- 172.21.15.16:0/1450763725 >> v1:172.21.15.186:6789/0 conn(0x7f36940b9720 legacy=0x7f36940ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.174+0000 7f369259c640 1 -- 172.21.15.16:0/1450763725 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3694162540 con 0x7f3694161090 2024-04-25T10:05:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.174+0000 7f369259c640 1 --2- 172.21.15.16:0/1450763725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3694161090 0x7f3694161730 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f368c009870 tx=0x7f368c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64830284182aba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.174+0000 7f369159a640 1 -- 172.21.15.16:0/1450763725 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c037070 con 0x7f3694161090 2024-04-25T10:05:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.174+0000 7f369159a640 1 -- 172.21.15.16:0/1450763725 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c032440 con 0x7f3694161090 2024-04-25T10:05:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.174+0000 7f369159a640 1 -- 172.21.15.16:0/1450763725 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c032810 con 0x7f3694161090 2024-04-25T10:05:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.175+0000 7f369895c640 1 -- 172.21.15.16:0/1450763725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3694161090 msgr2=0x7f3694161730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.175+0000 7f369895c640 1 --2- 172.21.15.16:0/1450763725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3694161090 0x7f3694161730 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f368c009870 tx=0x7f368c02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.175+0000 7f369895c640 1 -- 172.21.15.16:0/1450763725 shutdown_connections 2024-04-25T10:05:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.175+0000 7f369895c640 1 --2- 172.21.15.16:0/1450763725 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36940ceeb0 0x7f36940d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.175+0000 7f369895c640 1 --2- 172.21.15.16:0/1450763725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3694161090 0x7f3694161730 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.175+0000 7f369895c640 1 -- 172.21.15.16:0/1450763725 >> 172.21.15.16:0/1450763725 conn(0x7f36940b55e0 msgr2=0x7f36940b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.175+0000 7f369895c640 1 -- 172.21.15.16:0/1450763725 shutdown_connections 2024-04-25T10:05:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.175+0000 7f369895c640 1 -- 172.21.15.16:0/1450763725 wait complete. 2024-04-25T10:05:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.176+0000 7f369895c640 1 Processor -- start 2024-04-25T10:05:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.176+0000 7f369895c640 1 -- start start 2024-04-25T10:05:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.177+0000 7f369895c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36940b9720 0x7f369415d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.177+0000 7f369895c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36940ceeb0 0x7f36941f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.177+0000 7f369895c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3694161090 0x7f369415a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.177+0000 7f369895c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3694161c70 con 0x7f36940ceeb0 2024-04-25T10:05:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.177+0000 7f369895c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3694161470 con 0x7f3694161090 2024-04-25T10:05:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.177+0000 7f369895c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36940d2b70 con 0x7f36940b9720 2024-04-25T10:05:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.177+0000 7f3691d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36940ceeb0 0x7f36941f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.177+0000 7f3691d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36940ceeb0 0x7f36941f30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42648/0 (socket says 172.21.15.16:42648) 2024-04-25T10:05:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.177+0000 7f3691d9b640 1 -- 172.21.15.16:0/2746668247 learned_addr learned my addr 172.21.15.16:0/2746668247 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.177+0000 7f369259c640 1 --2- 172.21.15.16:0/2746668247 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36940b9720 0x7f369415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.177+0000 7f3692d9d640 1 --2- 172.21.15.16:0/2746668247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3694161090 0x7f369415a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.178+0000 7f3691d9b640 1 -- 172.21.15.16:0/2746668247 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36940b9720 msgr2=0x7f369415d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.178+0000 7f3691d9b640 1 --2- 172.21.15.16:0/2746668247 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36940b9720 0x7f369415d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.178+0000 7f3691d9b640 1 -- 172.21.15.16:0/2746668247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3694161090 msgr2=0x7f369415a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.178+0000 7f3691d9b640 1 --2- 172.21.15.16:0/2746668247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3694161090 0x7f369415a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.178+0000 7f3691d9b640 1 -- 172.21.15.16:0/2746668247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36941f3aa0 con 0x7f36940ceeb0 2024-04-25T10:05:48.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.178+0000 7f3691d9b640 1 --2- 172.21.15.16:0/2746668247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36940ceeb0 0x7f36941f30d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f368800c460 tx=0x7f368800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.178+0000 7f3682ffd640 1 -- 172.21.15.16:0/2746668247 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36880052d0 con 0x7f36940ceeb0 2024-04-25T10:05:48.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.178+0000 7f3682ffd640 1 -- 172.21.15.16:0/2746668247 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3688012630 con 0x7f36940ceeb0 2024-04-25T10:05:48.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.178+0000 7f3682ffd640 1 -- 172.21.15.16:0/2746668247 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36880054b0 con 0x7f36940ceeb0 2024-04-25T10:05:48.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.178+0000 7f369895c640 1 -- 172.21.15.16:0/2746668247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36941f3d80 con 0x7f36940ceeb0 2024-04-25T10:05:48.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.178+0000 7f369895c640 1 -- 172.21.15.16:0/2746668247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36941f4340 con 0x7f36940ceeb0 2024-04-25T10:05:48.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.179+0000 7f369895c640 1 -- 172.21.15.16:0/2746668247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36940750b0 con 0x7f36940ceeb0 2024-04-25T10:05:48.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.179+0000 7f3682ffd640 1 -- 172.21.15.16:0/2746668247 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3688012a20 con 0x7f36940ceeb0 2024-04-25T10:05:48.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.180+0000 7f3682ffd640 1 --2- 172.21.15.16:0/2746668247 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f367406b4d0 0x7f367406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.180+0000 7f3682ffd640 1 -- 172.21.15.16:0/2746668247 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3688089ca0 con 0x7f36940ceeb0 2024-04-25T10:05:48.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.180+0000 7f369259c640 1 --2- 172.21.15.16:0/2746668247 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f367406b4d0 0x7f367406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.181+0000 7f369259c640 1 --2- 172.21.15.16:0/2746668247 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f367406b4d0 0x7f367406d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f368c034000 tx=0x7f368c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.184+0000 7f3682ffd640 1 -- 172.21.15.16:0/2746668247 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f36880584d0 con 0x7f36940ceeb0 2024-04-25T10:05:48.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.377+0000 7f369895c640 1 -- 172.21.15.16:0/2746668247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f36940cf3d0 con 0x7f36940ceeb0 2024-04-25T10:05:48.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.377+0000 7f3682ffd640 1 -- 172.21.15.16:0/2746668247 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f368805c570 con 0x7f36940ceeb0 2024-04-25T10:05:48.382 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-04-25T10:05:48.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.379+0000 7f369895c640 1 -- 172.21.15.16:0/2746668247 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f367406b4d0 msgr2=0x7f367406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.379+0000 7f369895c640 1 --2- 172.21.15.16:0/2746668247 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f367406b4d0 0x7f367406d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f368c034000 tx=0x7f368c02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.379+0000 7f369895c640 1 -- 172.21.15.16:0/2746668247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36940ceeb0 msgr2=0x7f36941f30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.379+0000 7f369895c640 1 --2- 172.21.15.16:0/2746668247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36940ceeb0 0x7f36941f30d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f368800c460 tx=0x7f368800c920 comp rx=0 tx=0).stop 2024-04-25T10:05:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.379+0000 7f369895c640 1 -- 172.21.15.16:0/2746668247 shutdown_connections 2024-04-25T10:05:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.379+0000 7f369895c640 1 --2- 172.21.15.16:0/2746668247 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f367406b4d0 0x7f367406d990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.379+0000 7f369895c640 1 --2- 172.21.15.16:0/2746668247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3694161090 0x7f369415a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.379+0000 7f369895c640 1 --2- 172.21.15.16:0/2746668247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36940ceeb0 0x7f36941f30d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.379+0000 7f369895c640 1 --2- 172.21.15.16:0/2746668247 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36940b9720 0x7f369415d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.379+0000 7f369895c640 1 -- 172.21.15.16:0/2746668247 >> 172.21.15.16:0/2746668247 conn(0x7f36940b55e0 msgr2=0x7f36940b84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.379+0000 7f369895c640 1 -- 172.21.15.16:0/2746668247 shutdown_connections 2024-04-25T10:05:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.379+0000 7f369895c640 1 -- 172.21.15.16:0/2746668247 wait complete. 2024-04-25T10:05:48.395 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T10:05:48.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:48.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-25T10:05:48.421 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:05:48.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:48.421 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:05:48.477 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:05:48.477 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:48.496 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:05:48.496 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:48.555 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:05:48.555 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:48.616 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:48.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.713+0000 7fc17fb7e640 1 Processor -- start 2024-04-25T10:05:48.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.713+0000 7fc17fb7e640 1 -- start start 2024-04-25T10:05:48.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.714+0000 7fc17fb7e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1780d35b0 0x7fc1780b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.714+0000 7fc17fb7e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc1780b5990 0x7fc1780b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.714+0000 7fc17fb7e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17815e0c0 con 0x7fc17815dce0 2024-04-25T10:05:48.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.714+0000 7fc17fb7e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1780b5de0 con 0x7fc1780b5990 2024-04-25T10:05:48.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.714+0000 7fc17fb7e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780d2510 con 0x7fc1780d35b0 2024-04-25T10:05:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.714+0000 7fc17e11b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc1780b5990 0x7fc1780b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.714+0000 7fc17e11b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc1780b5990 0x7fc1780b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42664/0 (socket says 172.21.15.16:42664) 2024-04-25T10:05:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.714+0000 7fc17e11b640 1 -- 172.21.15.16:0/4240736689 learned_addr learned my addr 172.21.15.16:0/4240736689 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.715+0000 7fc17d119640 1 --2- 172.21.15.16:0/4240736689 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1780d35b0 0x7fc1780b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.715+0000 7fc17e11b640 1 -- 172.21.15.16:0/4240736689 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1780d35b0 msgr2=0x7fc1780b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.715+0000 7fc17e11b640 1 --2- 172.21.15.16:0/4240736689 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1780d35b0 0x7fc1780b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.715+0000 7fc17e11b640 1 -- 172.21.15.16:0/4240736689 >> v1:172.21.15.186:6789/0 conn(0x7fc17815dce0 legacy=0x7fc1780d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.715+0000 7fc17e11b640 1 -- 172.21.15.16:0/4240736689 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1780d27f0 con 0x7fc1780b5990 2024-04-25T10:05:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.715+0000 7fc17d119640 1 --2- 172.21.15.16:0/4240736689 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1780d35b0 0x7fc1780b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.715+0000 7fc17e11b640 1 --2- 172.21.15.16:0/4240736689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1780b5990 0x7fc1780b6b00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc170009870 tx=0x7fc17002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7365e8ec81a8187a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.716+0000 7fc17c918640 1 -- 172.21.15.16:0/4240736689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc170037070 con 0x7fc1780b5990 2024-04-25T10:05:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.716+0000 7fc17c918640 1 -- 172.21.15.16:0/4240736689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc170032440 con 0x7fc1780b5990 2024-04-25T10:05:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.716+0000 7fc17c918640 1 -- 172.21.15.16:0/4240736689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc170032810 con 0x7fc1780b5990 2024-04-25T10:05:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.716+0000 7fc17fb7e640 1 -- 172.21.15.16:0/4240736689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1780b5990 msgr2=0x7fc1780b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.716+0000 7fc17fb7e640 1 --2- 172.21.15.16:0/4240736689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1780b5990 0x7fc1780b6b00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc170009870 tx=0x7fc17002a270 comp rx=0 tx=0).stop 2024-04-25T10:05:48.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.716+0000 7fc17fb7e640 1 -- 172.21.15.16:0/4240736689 shutdown_connections 2024-04-25T10:05:48.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.716+0000 7fc17fb7e640 1 --2- 172.21.15.16:0/4240736689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1780b5990 0x7fc1780b6b00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.716+0000 7fc17fb7e640 1 --2- 172.21.15.16:0/4240736689 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1780d35b0 0x7fc1780b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.716+0000 7fc17fb7e640 1 -- 172.21.15.16:0/4240736689 >> 172.21.15.16:0/4240736689 conn(0x7fc178072900 msgr2=0x7fc1780d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.716+0000 7fc17fb7e640 1 -- 172.21.15.16:0/4240736689 shutdown_connections 2024-04-25T10:05:48.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.716+0000 7fc17fb7e640 1 -- 172.21.15.16:0/4240736689 wait complete. 2024-04-25T10:05:48.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.717+0000 7fc17fb7e640 1 Processor -- start 2024-04-25T10:05:48.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.717+0000 7fc17fb7e640 1 -- start start 2024-04-25T10:05:48.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.718+0000 7fc17fb7e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1780b5990 0x7fc178152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.718+0000 7fc17fb7e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1780d35b0 0x7fc1780b7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.718+0000 7fc17fb7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc17815dce0 0x7fc1780b7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.718+0000 7fc17fb7e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780d3090 con 0x7fc17815dce0 2024-04-25T10:05:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.718+0000 7fc17fb7e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780d2db0 con 0x7fc1780d35b0 2024-04-25T10:05:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.718+0000 7fc17fb7e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780d2160 con 0x7fc1780b5990 2024-04-25T10:05:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.718+0000 7fc17e11b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc17815dce0 0x7fc1780b7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.718+0000 7fc17e11b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc17815dce0 0x7fc1780b7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42674/0 (socket says 172.21.15.16:42674) 2024-04-25T10:05:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.718+0000 7fc17e11b640 1 -- 172.21.15.16:0/2606448436 learned_addr learned my addr 172.21.15.16:0/2606448436 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.718+0000 7fc17d119640 1 --2- 172.21.15.16:0/2606448436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1780d35b0 0x7fc1780b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.718+0000 7fc17d91a640 1 --2- 172.21.15.16:0/2606448436 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1780b5990 0x7fc178152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.718+0000 7fc17e11b640 1 -- 172.21.15.16:0/2606448436 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1780b5990 msgr2=0x7fc178152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.718+0000 7fc17e11b640 1 --2- 172.21.15.16:0/2606448436 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1780b5990 0x7fc178152200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.719+0000 7fc17e11b640 1 -- 172.21.15.16:0/2606448436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1780d35b0 msgr2=0x7fc1780b7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.719+0000 7fc17e11b640 1 --2- 172.21.15.16:0/2606448436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1780d35b0 0x7fc1780b7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.719+0000 7fc17e11b640 1 -- 172.21.15.16:0/2606448436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc178152740 con 0x7fc17815dce0 2024-04-25T10:05:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.719+0000 7fc17d91a640 1 --2- 172.21.15.16:0/2606448436 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1780b5990 0x7fc178152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.719+0000 7fc17e11b640 1 --2- 172.21.15.16:0/2606448436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc17815dce0 0x7fc1780b7ab0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc170009870 tx=0x7fc1700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.719+0000 7fc1767fc640 1 -- 172.21.15.16:0/2606448436 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc170037040 con 0x7fc17815dce0 2024-04-25T10:05:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.719+0000 7fc1767fc640 1 -- 172.21.15.16:0/2606448436 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc170042070 con 0x7fc17815dce0 2024-04-25T10:05:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.719+0000 7fc17fb7e640 1 -- 172.21.15.16:0/2606448436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1780c81f0 con 0x7fc17815dce0 2024-04-25T10:05:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.719+0000 7fc1767fc640 1 -- 172.21.15.16:0/2606448436 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc170030920 con 0x7fc17815dce0 2024-04-25T10:05:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.719+0000 7fc17fb7e640 1 -- 172.21.15.16:0/2606448436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1780c87b0 con 0x7fc17815dce0 2024-04-25T10:05:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.720+0000 7fc17fb7e640 1 -- 172.21.15.16:0/2606448436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1780c89f0 con 0x7fc17815dce0 2024-04-25T10:05:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.720+0000 7fc1767fc640 1 -- 172.21.15.16:0/2606448436 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc170030ac0 con 0x7fc17815dce0 2024-04-25T10:05:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.721+0000 7fc1767fc640 1 --2- 172.21.15.16:0/2606448436 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc16406b4d0 0x7fc16406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.721+0000 7fc1767fc640 1 -- 172.21.15.16:0/2606448436 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1700aaa90 con 0x7fc17815dce0 2024-04-25T10:05:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.721+0000 7fc17d91a640 1 --2- 172.21.15.16:0/2606448436 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc16406b4d0 0x7fc16406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.721+0000 7fc17d91a640 1 --2- 172.21.15.16:0/2606448436 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc16406b4d0 0x7fc16406d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc1780cda50 tx=0x7fc168009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.725+0000 7fc1767fc640 1 -- 172.21.15.16:0/2606448436 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc1780c89f0 con 0x7fc17815dce0 2024-04-25T10:05:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.906+0000 7fc17fb7e640 1 -- 172.21.15.16:0/2606448436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1780749b0 con 0x7fc17815dce0 2024-04-25T10:05:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.908+0000 7fc1767fc640 1 -- 172.21.15.16:0/2606448436 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16609 (secure 0 0 0) 0x7fc1700792c0 con 0x7fc17815dce0 2024-04-25T10:05:48.913 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:48.913 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:12.449183+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.910+0000 7fc17fb7e640 1 -- 172.21.15.16:0/2606448436 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc16406b4d0 msgr2=0x7fc16406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.910+0000 7fc17fb7e640 1 --2- 172.21.15.16:0/2606448436 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc16406b4d0 0x7fc16406d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc1780cda50 tx=0x7fc168009040 comp rx=0 tx=0).stop 2024-04-25T10:05:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.910+0000 7fc17fb7e640 1 -- 172.21.15.16:0/2606448436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc17815dce0 msgr2=0x7fc1780b7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.910+0000 7fc17fb7e640 1 --2- 172.21.15.16:0/2606448436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc17815dce0 0x7fc1780b7ab0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc170009870 tx=0x7fc1700046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.910+0000 7fc17fb7e640 1 -- 172.21.15.16:0/2606448436 shutdown_connections 2024-04-25T10:05:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.910+0000 7fc17fb7e640 1 --2- 172.21.15.16:0/2606448436 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc16406b4d0 0x7fc16406d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.910+0000 7fc17fb7e640 1 --2- 172.21.15.16:0/2606448436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc17815dce0 0x7fc1780b7ab0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.910+0000 7fc17fb7e640 1 --2- 172.21.15.16:0/2606448436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1780d35b0 0x7fc1780b7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.910+0000 7fc17fb7e640 1 --2- 172.21.15.16:0/2606448436 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1780b5990 0x7fc178152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.910+0000 7fc17fb7e640 1 -- 172.21.15.16:0/2606448436 >> 172.21.15.16:0/2606448436 conn(0x7fc178072900 msgr2=0x7fc1780c6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.910+0000 7fc17fb7e640 1 -- 172.21.15.16:0/2606448436 shutdown_connections 2024-04-25T10:05:48.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:48.910+0000 7fc17fb7e640 1 -- 172.21.15.16:0/2606448436 wait complete. 2024-04-25T10:05:48.926 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:08.724537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:09.793105+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:48.926 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:49.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.025+0000 7f5ca9996640 1 Processor -- start 2024-04-25T10:05:49.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.025+0000 7f5ca9996640 1 -- start start 2024-04-25T10:05:49.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.026+0000 7f5ca9996640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5ca415ec90 0x7f5ca40ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.026+0000 7f5ca9996640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca406c150 con 0x7f5ca40b2740 2024-04-25T10:05:49.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.026+0000 7f5ca9996640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca406c430 con 0x7f5ca40d0490 2024-04-25T10:05:49.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.026+0000 7f5ca9996640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca40d2000 con 0x7f5ca415ec90 2024-04-25T10:05:49.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.026+0000 7f5ca37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5ca415ec90 0x7f5ca40ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.026+0000 7f5ca27fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5ca40b2740 0x7f5ca40ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51450/0 (socket says 172.21.15.16:51450) 2024-04-25T10:05:49.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.026+0000 7f5ca37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5ca415ec90 0x7f5ca40ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42684/0 (socket says 172.21.15.16:42684) 2024-04-25T10:05:49.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.026+0000 7f5ca27fc640 1 -- 172.21.15.16:0/1056261278 learned_addr learned my addr 172.21.15.16:0/1056261278 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:49.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.026+0000 7f5ca37fe640 1 -- 172.21.15.16:0/1056261278 >> v1:172.21.15.186:6789/0 conn(0x7f5ca40d0490 legacy=0x7f5ca40d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:49.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.026+0000 7f5ca37fe640 1 -- 172.21.15.16:0/1056261278 >> v1:172.21.15.16:6789/0 conn(0x7f5ca40b2740 legacy=0x7f5ca40ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.026+0000 7f5ca37fe640 1 -- 172.21.15.16:0/1056261278 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca406c710 con 0x7f5ca415ec90 2024-04-25T10:05:49.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.027+0000 7f5ca37fe640 1 --2- 172.21.15.16:0/1056261278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca415ec90 0x7f5ca40ce8b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009d90 tx=0x7f5c9402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a30a18d74bb7e83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.027+0000 7f5ca1ffb640 1 -- 172.21.15.16:0/1056261278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94031600 con 0x7f5ca415ec90 2024-04-25T10:05:49.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.027+0000 7f5ca1ffb640 1 -- 172.21.15.16:0/1056261278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c940317a0 con 0x7f5ca415ec90 2024-04-25T10:05:49.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.027+0000 7f5ca1ffb640 1 -- 172.21.15.16:0/1056261278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94031b70 con 0x7f5ca415ec90 2024-04-25T10:05:49.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.028+0000 7f5ca9996640 1 -- 172.21.15.16:0/1056261278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca415ec90 msgr2=0x7f5ca40ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.028+0000 7f5ca9996640 1 --2- 172.21.15.16:0/1056261278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca415ec90 0x7f5ca40ce8b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009d90 tx=0x7f5c9402a610 comp rx=0 tx=0).stop 2024-04-25T10:05:49.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.028+0000 7f5ca9996640 1 -- 172.21.15.16:0/1056261278 shutdown_connections 2024-04-25T10:05:49.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.028+0000 7f5ca9996640 1 --2- 172.21.15.16:0/1056261278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca415ec90 0x7f5ca40ce8b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.028+0000 7f5ca9996640 1 -- 172.21.15.16:0/1056261278 >> 172.21.15.16:0/1056261278 conn(0x7f5ca40b5660 msgr2=0x7f5ca40b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:49.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.028+0000 7f5ca9996640 1 -- 172.21.15.16:0/1056261278 shutdown_connections 2024-04-25T10:05:49.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.028+0000 7f5ca9996640 1 -- 172.21.15.16:0/1056261278 wait complete. 2024-04-25T10:05:49.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.029+0000 7f5ca9996640 1 Processor -- start 2024-04-25T10:05:49.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.029+0000 7f5ca9996640 1 -- start start 2024-04-25T10:05:49.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.030+0000 7f5ca9996640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ca40b2740 0x7f5ca41f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.030+0000 7f5ca9996640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca40d0490 0x7f5ca40ce770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.030+0000 7f5ca9996640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ca415ec90 0x7f5ca40ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.030+0000 7f5ca9996640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca415f120 con 0x7f5ca40d0490 2024-04-25T10:05:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.030+0000 7f5ca9996640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca415e020 con 0x7f5ca40b2740 2024-04-25T10:05:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.030+0000 7f5ca9996640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca40cedf0 con 0x7f5ca415ec90 2024-04-25T10:05:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.030+0000 7f5ca27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca40d0490 0x7f5ca40ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.030+0000 7f5ca27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca40d0490 0x7f5ca40ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42688/0 (socket says 172.21.15.16:42688) 2024-04-25T10:05:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.030+0000 7f5ca27fc640 1 -- 172.21.15.16:0/2042788945 learned_addr learned my addr 172.21.15.16:0/2042788945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.030+0000 7f5ca2ffd640 1 --2- 172.21.15.16:0/2042788945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ca40b2740 0x7f5ca41f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.030+0000 7f5ca37fe640 1 --2- 172.21.15.16:0/2042788945 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ca415ec90 0x7f5ca40ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.030+0000 7f5ca27fc640 1 -- 172.21.15.16:0/2042788945 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ca415ec90 msgr2=0x7f5ca40ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.030+0000 7f5ca27fc640 1 --2- 172.21.15.16:0/2042788945 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ca415ec90 0x7f5ca40ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.030+0000 7f5ca27fc640 1 -- 172.21.15.16:0/2042788945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ca40b2740 msgr2=0x7f5ca41f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.030+0000 7f5ca27fc640 1 --2- 172.21.15.16:0/2042788945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ca40b2740 0x7f5ca41f7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.030+0000 7f5ca27fc640 1 -- 172.21.15.16:0/2042788945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca40ca9e0 con 0x7f5ca40d0490 2024-04-25T10:05:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.030+0000 7f5ca37fe640 1 --2- 172.21.15.16:0/2042788945 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ca415ec90 0x7f5ca40ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.030+0000 7f5ca27fc640 1 --2- 172.21.15.16:0/2042788945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca40d0490 0x7f5ca40ce770 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00c430 tx=0x7f5c8c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.031+0000 7f5c9bfff640 1 -- 172.21.15.16:0/2042788945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c017070 con 0x7f5ca40d0490 2024-04-25T10:05:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.031+0000 7f5c9bfff640 1 -- 172.21.15.16:0/2042788945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8c005ce0 con 0x7f5ca40d0490 2024-04-25T10:05:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.031+0000 7f5ca9996640 1 -- 172.21.15.16:0/2042788945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca41f7af0 con 0x7f5ca40d0490 2024-04-25T10:05:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.031+0000 7f5c9bfff640 1 -- 172.21.15.16:0/2042788945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c0072f0 con 0x7f5ca40d0490 2024-04-25T10:05:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.031+0000 7f5ca9996640 1 -- 172.21.15.16:0/2042788945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca41f80b0 con 0x7f5ca40d0490 2024-04-25T10:05:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.032+0000 7f5ca9996640 1 -- 172.21.15.16:0/2042788945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca40750b0 con 0x7f5ca40d0490 2024-04-25T10:05:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.032+0000 7f5c9bfff640 1 -- 172.21.15.16:0/2042788945 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c8c01a070 con 0x7f5ca40d0490 2024-04-25T10:05:49.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.033+0000 7f5c9bfff640 1 --2- 172.21.15.16:0/2042788945 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5c8406b4d0 0x7f5c8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.033+0000 7f5c9bfff640 1 -- 172.21.15.16:0/2042788945 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c8c0896c0 con 0x7f5ca40d0490 2024-04-25T10:05:49.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.033+0000 7f5ca2ffd640 1 --2- 172.21.15.16:0/2042788945 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5c8406b4d0 0x7f5c8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.033+0000 7f5ca2ffd640 1 --2- 172.21.15.16:0/2042788945 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5c8406b4d0 0x7f5c8406d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c004ea0 tx=0x7f5c9c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.037+0000 7f5c9bfff640 1 -- 172.21.15.16:0/2042788945 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c8c0584a0 con 0x7f5ca40d0490 2024-04-25T10:05:49.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.214+0000 7f5ca9996640 1 -- 172.21.15.16:0/2042788945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5ca40d18d0 con 0x7f5ca40d0490 2024-04-25T10:05:49.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.214+0000 7f5c9bfff640 1 -- 172.21.15.16:0/2042788945 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c8c057da0 con 0x7f5ca40d0490 2024-04-25T10:05:49.220 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:05:49.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.216+0000 7f5ca9996640 1 -- 172.21.15.16:0/2042788945 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5c8406b4d0 msgr2=0x7f5c8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.216+0000 7f5ca9996640 1 --2- 172.21.15.16:0/2042788945 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5c8406b4d0 0x7f5c8406d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c004ea0 tx=0x7f5c9c009040 comp rx=0 tx=0).stop 2024-04-25T10:05:49.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.216+0000 7f5ca9996640 1 -- 172.21.15.16:0/2042788945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca40d0490 msgr2=0x7f5ca40ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.216+0000 7f5ca9996640 1 --2- 172.21.15.16:0/2042788945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca40d0490 0x7f5ca40ce770 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00c430 tx=0x7f5c8c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.216+0000 7f5ca9996640 1 -- 172.21.15.16:0/2042788945 shutdown_connections 2024-04-25T10:05:49.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.216+0000 7f5ca9996640 1 --2- 172.21.15.16:0/2042788945 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5c8406b4d0 0x7f5c8406d990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.216+0000 7f5ca9996640 1 --2- 172.21.15.16:0/2042788945 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ca415ec90 0x7f5ca40ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.216+0000 7f5ca9996640 1 --2- 172.21.15.16:0/2042788945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca40d0490 0x7f5ca40ce770 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.216+0000 7f5ca9996640 1 --2- 172.21.15.16:0/2042788945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ca40b2740 0x7f5ca41f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.216+0000 7f5ca9996640 1 -- 172.21.15.16:0/2042788945 >> 172.21.15.16:0/2042788945 conn(0x7f5ca40b5660 msgr2=0x7f5ca4159310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:49.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.216+0000 7f5ca9996640 1 -- 172.21.15.16:0/2042788945 shutdown_connections 2024-04-25T10:05:49.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.216+0000 7f5ca9996640 1 -- 172.21.15.16:0/2042788945 wait complete. 2024-04-25T10:05:49.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:49.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.332+0000 7f39c18cb640 1 Processor -- start 2024-04-25T10:05:49.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.332+0000 7f39c18cb640 1 -- start start 2024-04-25T10:05:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.333+0000 7f39c18cb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39bc15dc80 0x7f39bc161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.334+0000 7f39c18cb640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f39bc0b5c00 0x7f39bc0d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.334+0000 7f39c18cb640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39bc1627a0 con 0x7f39bc0b97c0 2024-04-25T10:05:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.334+0000 7f39c18cb640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39bc1621e0 con 0x7f39bc0b5c00 2024-04-25T10:05:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.334+0000 7f39c18cb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc1624c0 con 0x7f39bc15dc80 2024-04-25T10:05:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.334+0000 7f39baffd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39bc15dc80 0x7f39bc161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.334+0000 7f39bb7fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f39bc0b5c00 0x7f39bc0d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.334+0000 7f39ba7fc640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f39bc0b97c0 0x7f39bc0b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:43230/0 (socket says 172.21.15.16:43230) 2024-04-25T10:05:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.334+0000 7f39ba7fc640 1 -- 172.21.15.16:0/3557795184 learned_addr learned my addr 172.21.15.16:0/3557795184 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.334+0000 7f39bb7fe640 1 -- 172.21.15.16:0/3557795184 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39bc15dc80 msgr2=0x7f39bc161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.334+0000 7f39bb7fe640 1 --2- 172.21.15.16:0/3557795184 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39bc15dc80 0x7f39bc161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.334+0000 7f39bb7fe640 1 -- 172.21.15.16:0/3557795184 >> v1:172.21.15.186:6789/0 conn(0x7f39bc0b97c0 legacy=0x7f39bc0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.334+0000 7f39bb7fe640 1 -- 172.21.15.16:0/3557795184 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc0d1c70 con 0x7f39bc0b5c00 2024-04-25T10:05:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.334+0000 7f39b9ffb640 1 -- 172.21.15.16:0/3557795184 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436266799 0 0) 0x7f39bc1627a0 con 0x7f39bc0b97c0 2024-04-25T10:05:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.334+0000 7f39baffd640 1 --2- 172.21.15.16:0/3557795184 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39bc15dc80 0x7f39bc161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.335+0000 7f39bb7fe640 1 --2- 172.21.15.16:0/3557795184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc0b5c00 0x7f39bc0d2470 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009d90 tx=0x7f39b402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37bacc463287dc58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.335+0000 7f39b9ffb640 1 -- 172.21.15.16:0/3557795184 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4007740 con 0x7f39bc0b5c00 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.335+0000 7f39b9ffb640 1 -- 172.21.15.16:0/3557795184 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b40078e0 con 0x7f39bc0b5c00 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.335+0000 7f39b9ffb640 1 -- 172.21.15.16:0/3557795184 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4007cb0 con 0x7f39bc0b5c00 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.336+0000 7f39c18cb640 1 -- 172.21.15.16:0/3557795184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc0b5c00 msgr2=0x7f39bc0d2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.336+0000 7f39c18cb640 1 --2- 172.21.15.16:0/3557795184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc0b5c00 0x7f39bc0d2470 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009d90 tx=0x7f39b402a660 comp rx=0 tx=0).stop 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.336+0000 7f39c18cb640 1 -- 172.21.15.16:0/3557795184 shutdown_connections 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.336+0000 7f39c18cb640 1 --2- 172.21.15.16:0/3557795184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc0b5c00 0x7f39bc0d2470 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.336+0000 7f39c18cb640 1 --2- 172.21.15.16:0/3557795184 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39bc15dc80 0x7f39bc161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.336+0000 7f39c18cb640 1 -- 172.21.15.16:0/3557795184 >> 172.21.15.16:0/3557795184 conn(0x7f39bc0d04f0 msgr2=0x7f39bc0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.336+0000 7f39c18cb640 1 -- 172.21.15.16:0/3557795184 shutdown_connections 2024-04-25T10:05:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.336+0000 7f39c18cb640 1 -- 172.21.15.16:0/3557795184 wait complete. 2024-04-25T10:05:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.337+0000 7f39c18cb640 1 Processor -- start 2024-04-25T10:05:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.337+0000 7f39c18cb640 1 -- start start 2024-04-25T10:05:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.338+0000 7f39c18cb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc0b5c00 0x7f39bc160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.338+0000 7f39c18cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39bc0b97c0 0x7f39bc0aa450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.338+0000 7f39c18cb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39bc15dc80 0x7f39bc14ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.338+0000 7f39c18cb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc161f30 con 0x7f39bc0b97c0 2024-04-25T10:05:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.338+0000 7f39c18cb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc161db0 con 0x7f39bc0b5c00 2024-04-25T10:05:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.338+0000 7f39c18cb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0d29b0 con 0x7f39bc15dc80 2024-04-25T10:05:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.338+0000 7f39ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39bc0b97c0 0x7f39bc0aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.338+0000 7f39ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39bc0b97c0 0x7f39bc0aa450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42692/0 (socket says 172.21.15.16:42692) 2024-04-25T10:05:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.338+0000 7f39ba7fc640 1 -- 172.21.15.16:0/2144474994 learned_addr learned my addr 172.21.15.16:0/2144474994 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.338+0000 7f39bb7fe640 1 --2- 172.21.15.16:0/2144474994 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39bc15dc80 0x7f39bc14ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.338+0000 7f39baffd640 1 --2- 172.21.15.16:0/2144474994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc0b5c00 0x7f39bc160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.339+0000 7f39ba7fc640 1 -- 172.21.15.16:0/2144474994 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39bc15dc80 msgr2=0x7f39bc14ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.339+0000 7f39ba7fc640 1 --2- 172.21.15.16:0/2144474994 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39bc15dc80 0x7f39bc14ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.339+0000 7f39ba7fc640 1 -- 172.21.15.16:0/2144474994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc0b5c00 msgr2=0x7f39bc160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.339+0000 7f39ba7fc640 1 --2- 172.21.15.16:0/2144474994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc0b5c00 0x7f39bc160bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.339+0000 7f39ba7fc640 1 -- 172.21.15.16:0/2144474994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc0c8990 con 0x7f39bc0b97c0 2024-04-25T10:05:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.339+0000 7f39ba7fc640 1 --2- 172.21.15.16:0/2144474994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39bc0b97c0 0x7f39bc0aa450 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00c4e0 tx=0x7f39ac00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.339+0000 7f39a3fff640 1 -- 172.21.15.16:0/2144474994 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac018070 con 0x7f39bc0b97c0 2024-04-25T10:05:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.339+0000 7f39a3fff640 1 -- 172.21.15.16:0/2144474994 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac007500 con 0x7f39bc0b97c0 2024-04-25T10:05:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.339+0000 7f39a3fff640 1 -- 172.21.15.16:0/2144474994 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac004a20 con 0x7f39bc0b97c0 2024-04-25T10:05:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.339+0000 7f39c18cb640 1 -- 172.21.15.16:0/2144474994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc0c8c70 con 0x7f39bc0b97c0 2024-04-25T10:05:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.339+0000 7f39c18cb640 1 -- 172.21.15.16:0/2144474994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39bc0c9230 con 0x7f39bc0b97c0 2024-04-25T10:05:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.341+0000 7f39a3fff640 1 -- 172.21.15.16:0/2144474994 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f39ac01b070 con 0x7f39bc0b97c0 2024-04-25T10:05:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.341+0000 7f39c18cb640 1 -- 172.21.15.16:0/2144474994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39bc0750b0 con 0x7f39bc0b97c0 2024-04-25T10:05:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.341+0000 7f39a3fff640 1 --2- 172.21.15.16:0/2144474994 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f39a806b4d0 0x7f39a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.341+0000 7f39a3fff640 1 -- 172.21.15.16:0/2144474994 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39ac0894b0 con 0x7f39bc0b97c0 2024-04-25T10:05:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.341+0000 7f39baffd640 1 --2- 172.21.15.16:0/2144474994 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f39a806b4d0 0x7f39a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.342+0000 7f39baffd640 1 --2- 172.21.15.16:0/2144474994 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f39a806b4d0 0x7f39a806d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009560 tx=0x7f39b0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.345+0000 7f39a3fff640 1 -- 172.21.15.16:0/2144474994 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f39ac057da0 con 0x7f39bc0b97c0 2024-04-25T10:05:49.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.547+0000 7f39c18cb640 1 -- 172.21.15.16:0/2144474994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f39bc0b8d30 con 0x7f39bc0b97c0 2024-04-25T10:05:49.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.548+0000 7f39a3fff640 1 -- 172.21.15.16:0/2144474994 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f39ac05be40 con 0x7f39bc0b97c0 2024-04-25T10:05:49.553 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:05:49.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.550+0000 7f39c18cb640 1 -- 172.21.15.16:0/2144474994 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f39a806b4d0 msgr2=0x7f39a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.550+0000 7f39c18cb640 1 --2- 172.21.15.16:0/2144474994 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f39a806b4d0 0x7f39a806d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009560 tx=0x7f39b0009190 comp rx=0 tx=0).stop 2024-04-25T10:05:49.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.550+0000 7f39c18cb640 1 -- 172.21.15.16:0/2144474994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39bc0b97c0 msgr2=0x7f39bc0aa450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.550+0000 7f39c18cb640 1 --2- 172.21.15.16:0/2144474994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39bc0b97c0 0x7f39bc0aa450 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00c4e0 tx=0x7f39ac00c9a0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.550+0000 7f39c18cb640 1 -- 172.21.15.16:0/2144474994 shutdown_connections 2024-04-25T10:05:49.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.550+0000 7f39c18cb640 1 --2- 172.21.15.16:0/2144474994 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f39a806b4d0 0x7f39a806d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.550+0000 7f39c18cb640 1 --2- 172.21.15.16:0/2144474994 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39bc15dc80 0x7f39bc14ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.550+0000 7f39c18cb640 1 --2- 172.21.15.16:0/2144474994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39bc0b97c0 0x7f39bc0aa450 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.550+0000 7f39c18cb640 1 --2- 172.21.15.16:0/2144474994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39bc0b5c00 0x7f39bc160bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.550+0000 7f39c18cb640 1 -- 172.21.15.16:0/2144474994 >> 172.21.15.16:0/2144474994 conn(0x7f39bc0d04f0 msgr2=0x7f39bc0b8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:49.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.550+0000 7f39c18cb640 1 -- 172.21.15.16:0/2144474994 shutdown_connections 2024-04-25T10:05:49.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.550+0000 7f39c18cb640 1 -- 172.21.15.16:0/2144474994 wait complete. 2024-04-25T10:05:49.566 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.666+0000 7fc12f1b0640 1 Processor -- start 2024-04-25T10:05:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.667+0000 7fc12f1b0640 1 -- start start 2024-04-25T10:05:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.667+0000 7fc12f1b0640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc1280b97c0 0x7fc1280b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.667+0000 7fc12f1b0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1280b5e40 con 0x7fc1280b5a30 2024-04-25T10:05:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.667+0000 7fc12f1b0640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1281627a0 con 0x7fc12815dc80 2024-04-25T10:05:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.667+0000 7fc12f1b0640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc128161f80 con 0x7fc1280b97c0 2024-04-25T10:05:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.668+0000 7fc12d74d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc1280b5a30 0x7fc1280d2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51460/0 (socket says 172.21.15.16:51460) 2024-04-25T10:05:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.668+0000 7fc12d74d640 1 -- 172.21.15.16:0/624750484 learned_addr learned my addr 172.21.15.16:0/624750484 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.668+0000 7fc127fff640 1 --2- 172.21.15.16:0/624750484 >> v2:172.21.15.186:3300/0 conn(0x7fc1280b97c0 0x7fc1280b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.668+0000 7fc1277fe640 1 -- 172.21.15.16:0/624750484 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3887013859 0 0) 0x7fc120001490 con 0x7fc1280b5a30 2024-04-25T10:05:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.668+0000 7fc1277fe640 1 -- 172.21.15.16:0/624750484 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc118003520 con 0x7fc1280b5a30 2024-04-25T10:05:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.668+0000 7fc127fff640 1 -- 172.21.15.16:0/624750484 >> v1:172.21.15.186:6789/0 conn(0x7fc12815dc80 legacy=0x7fc128161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.668+0000 7fc1277fe640 1 -- 172.21.15.16:0/624750484 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 203412071 0 0) 0x7fc118003520 con 0x7fc1280b5a30 2024-04-25T10:05:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.668+0000 7fc127fff640 1 -- 172.21.15.16:0/624750484 >> v1:172.21.15.16:6789/0 conn(0x7fc1280b5a30 legacy=0x7fc1280d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.668+0000 7fc127fff640 1 -- 172.21.15.16:0/624750484 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1281621e0 con 0x7fc1280b97c0 2024-04-25T10:05:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.669+0000 7fc127fff640 1 --2- 172.21.15.16:0/624750484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1280b97c0 0x7fc1280b54f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc1140097a0 tx=0x7fc11402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7299b3bdc769ecb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.669+0000 7fc1277fe640 1 -- 172.21.15.16:0/624750484 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114037070 con 0x7fc1280b97c0 2024-04-25T10:05:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.669+0000 7fc1277fe640 1 -- 172.21.15.16:0/624750484 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11402ac50 con 0x7fc1280b97c0 2024-04-25T10:05:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.669+0000 7fc1277fe640 1 -- 172.21.15.16:0/624750484 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120001bc0 con 0x7fc1280b97c0 2024-04-25T10:05:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.670+0000 7fc12f1b0640 1 -- 172.21.15.16:0/624750484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1280b97c0 msgr2=0x7fc1280b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.670+0000 7fc12f1b0640 1 --2- 172.21.15.16:0/624750484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1280b97c0 0x7fc1280b54f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc1140097a0 tx=0x7fc11402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.670+0000 7fc12f1b0640 1 -- 172.21.15.16:0/624750484 shutdown_connections 2024-04-25T10:05:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.670+0000 7fc12f1b0640 1 --2- 172.21.15.16:0/624750484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1280b97c0 0x7fc1280b54f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.670+0000 7fc12f1b0640 1 -- 172.21.15.16:0/624750484 >> 172.21.15.16:0/624750484 conn(0x7fc1280d04f0 msgr2=0x7fc1280d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.670+0000 7fc12f1b0640 1 -- 172.21.15.16:0/624750484 shutdown_connections 2024-04-25T10:05:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.670+0000 7fc12f1b0640 1 -- 172.21.15.16:0/624750484 wait complete. 2024-04-25T10:05:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.671+0000 7fc12f1b0640 1 Processor -- start 2024-04-25T10:05:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.671+0000 7fc12f1b0640 1 -- start start 2024-04-25T10:05:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc12f1b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1280b5a30 0x7fc1280c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc12f1b0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1280b97c0 0x7fc1281f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc12f1b0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc12815dc80 0x7fc1281499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc12f1b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12815e060 con 0x7fc1280b5a30 2024-04-25T10:05:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc12f1b0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1280d1c70 con 0x7fc12815dc80 2024-04-25T10:05:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc12f1b0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1280d2b80 con 0x7fc1280b97c0 2024-04-25T10:05:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc12cf4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1280b5a30 0x7fc1280c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc12cf4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1280b5a30 0x7fc1280c9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42702/0 (socket says 172.21.15.16:42702) 2024-04-25T10:05:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc12cf4c640 1 -- 172.21.15.16:0/2621414014 learned_addr learned my addr 172.21.15.16:0/2621414014 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc127fff640 1 --2- 172.21.15.16:0/2621414014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1280b97c0 0x7fc1281f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc12d74d640 1 --2- 172.21.15.16:0/2621414014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc12815dc80 0x7fc1281499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc12cf4c640 1 -- 172.21.15.16:0/2621414014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1280b97c0 msgr2=0x7fc1281f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc12cf4c640 1 --2- 172.21.15.16:0/2621414014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1280b97c0 0x7fc1281f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc12cf4c640 1 -- 172.21.15.16:0/2621414014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc12815dc80 msgr2=0x7fc1281499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc12cf4c640 1 --2- 172.21.15.16:0/2621414014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc12815dc80 0x7fc1281499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc12cf4c640 1 -- 172.21.15.16:0/2621414014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1280c8a80 con 0x7fc1280b5a30 2024-04-25T10:05:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc12d74d640 1 --2- 172.21.15.16:0/2621414014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc12815dc80 0x7fc1281499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc127fff640 1 --2- 172.21.15.16:0/2621414014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1280b97c0 0x7fc1281f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.672+0000 7fc12cf4c640 1 --2- 172.21.15.16:0/2621414014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1280b5a30 0x7fc1280c9b20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00c400 tx=0x7fc11c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.673+0000 7fc1257fa640 1 -- 172.21.15.16:0/2621414014 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c017070 con 0x7fc1280b5a30 2024-04-25T10:05:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.673+0000 7fc1257fa640 1 -- 172.21.15.16:0/2621414014 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11c0052d0 con 0x7fc1280b5a30 2024-04-25T10:05:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.673+0000 7fc12f1b0640 1 -- 172.21.15.16:0/2621414014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1280c8d60 con 0x7fc1280b5a30 2024-04-25T10:05:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.673+0000 7fc1257fa640 1 -- 172.21.15.16:0/2621414014 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c003830 con 0x7fc1280b5a30 2024-04-25T10:05:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.673+0000 7fc12f1b0640 1 -- 172.21.15.16:0/2621414014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1280c9320 con 0x7fc1280b5a30 2024-04-25T10:05:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.674+0000 7fc12f1b0640 1 -- 172.21.15.16:0/2621414014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1280750b0 con 0x7fc1280b5a30 2024-04-25T10:05:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.674+0000 7fc1257fa640 1 -- 172.21.15.16:0/2621414014 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc11c01a070 con 0x7fc1280b5a30 2024-04-25T10:05:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.675+0000 7fc1257fa640 1 --2- 172.21.15.16:0/2621414014 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc10c06b4d0 0x7fc10c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.675+0000 7fc1257fa640 1 -- 172.21.15.16:0/2621414014 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc11c089580 con 0x7fc1280b5a30 2024-04-25T10:05:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.675+0000 7fc127fff640 1 --2- 172.21.15.16:0/2621414014 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc10c06b4d0 0x7fc10c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.676+0000 7fc127fff640 1 --2- 172.21.15.16:0/2621414014 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc10c06b4d0 0x7fc10c06d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc114034000 tx=0x7fc11402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.679+0000 7fc1257fa640 1 -- 172.21.15.16:0/2621414014 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc11c058400 con 0x7fc1280b5a30 2024-04-25T10:05:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.859+0000 7fc12f1b0640 1 -- 172.21.15.16:0/2621414014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc1280d1780 con 0x7fc1280b5a30 2024-04-25T10:05:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.859+0000 7fc1257fa640 1 -- 172.21.15.16:0/2621414014 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc11c057d00 con 0x7fc1280b5a30 2024-04-25T10:05:49.865 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:05:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.861+0000 7fc12f1b0640 1 -- 172.21.15.16:0/2621414014 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc10c06b4d0 msgr2=0x7fc10c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.861+0000 7fc12f1b0640 1 --2- 172.21.15.16:0/2621414014 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc10c06b4d0 0x7fc10c06d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc114034000 tx=0x7fc11402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.861+0000 7fc12f1b0640 1 -- 172.21.15.16:0/2621414014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1280b5a30 msgr2=0x7fc1280c9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.861+0000 7fc12f1b0640 1 --2- 172.21.15.16:0/2621414014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1280b5a30 0x7fc1280c9b20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00c400 tx=0x7fc11c00c8c0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.861+0000 7fc12f1b0640 1 -- 172.21.15.16:0/2621414014 shutdown_connections 2024-04-25T10:05:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.861+0000 7fc12f1b0640 1 --2- 172.21.15.16:0/2621414014 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc10c06b4d0 0x7fc10c06d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.861+0000 7fc12f1b0640 1 --2- 172.21.15.16:0/2621414014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc12815dc80 0x7fc1281499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.861+0000 7fc12f1b0640 1 --2- 172.21.15.16:0/2621414014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1280b97c0 0x7fc1281f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.861+0000 7fc12f1b0640 1 --2- 172.21.15.16:0/2621414014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1280b5a30 0x7fc1280c9b20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.861+0000 7fc12f1b0640 1 -- 172.21.15.16:0/2621414014 >> 172.21.15.16:0/2621414014 conn(0x7fc1280d04f0 msgr2=0x7fc1280b4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.862+0000 7fc12f1b0640 1 -- 172.21.15.16:0/2621414014 shutdown_connections 2024-04-25T10:05:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.862+0000 7fc12f1b0640 1 -- 172.21.15.16:0/2621414014 wait complete. 2024-04-25T10:05:49.883 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:05:49.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.981+0000 7f5487a4d640 1 Processor -- start 2024-04-25T10:05:49.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.982+0000 7f5487a4d640 1 -- start start 2024-04-25T10:05:49.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.982+0000 7f5487a4d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5480099c70 0x7f548009a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.982+0000 7f5487a4d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548015bfc0 con 0x7f54800ce2c0 2024-04-25T10:05:49.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.982+0000 7f5487a4d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548009a9e0 con 0x7f548015bbe0 2024-04-25T10:05:49.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.982+0000 7f5487a4d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54800b21e0 con 0x7f5480099c70 2024-04-25T10:05:49.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5484fe8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f54800ce2c0 0x7f54800ce720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51472/0 (socket says 172.21.15.16:51472) 2024-04-25T10:05:49.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5484fe8640 1 -- 172.21.15.16:0/2786222500 learned_addr learned my addr 172.21.15.16:0/2786222500 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:49.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5485fea640 1 --2- 172.21.15.16:0/2786222500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5480099c70 0x7f548009a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5473fff640 1 -- 172.21.15.16:0/2786222500 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379762507 0 0) 0x7f548015bfc0 con 0x7f54800ce2c0 2024-04-25T10:05:49.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5473fff640 1 -- 172.21.15.16:0/2786222500 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f546c003520 con 0x7f54800ce2c0 2024-04-25T10:05:49.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5473fff640 1 -- 172.21.15.16:0/2786222500 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 302309733 0 0) 0x7f546c003520 con 0x7f54800ce2c0 2024-04-25T10:05:49.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5473fff640 1 -- 172.21.15.16:0/2786222500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5480099c70 msgr2=0x7f548009a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:49.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5473fff640 1 --2- 172.21.15.16:0/2786222500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5480099c70 0x7f548009a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5473fff640 1 -- 172.21.15.16:0/2786222500 >> v1:172.21.15.186:6789/0 conn(0x7f548015bbe0 legacy=0x7f54800cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5473fff640 1 -- 172.21.15.16:0/2786222500 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54800b18b0 con 0x7f54800ce2c0 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5473fff640 1 -- 172.21.15.16:0/2786222500 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f5474001670 con 0x7f54800ce2c0 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5473fff640 1 -- 172.21.15.16:0/2786222500 >> v1:172.21.15.16:6789/0 conn(0x7f54800ce2c0 legacy=0x7f54800ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5473fff640 1 --2- 172.21.15.16:0/2786222500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f546c0039e0 0x7f546c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5473fff640 1 --2- 172.21.15.16:0/2786222500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f546c004800 0x7f546c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5473fff640 1 --2- 172.21.15.16:0/2786222500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5480099c70 0x7f546c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5473fff640 1 -- 172.21.15.16:0/2786222500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54800b21e0 con 0x7f546c004800 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5473fff640 1 -- 172.21.15.16:0/2786222500 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c00ba70 con 0x7f546c0039e0 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5473fff640 1 -- 172.21.15.16:0/2786222500 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546c00bc70 con 0x7f5480099c70 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f5484fe8640 1 --2- 172.21.15.16:0/2786222500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f546c004800 0x7f546c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.983+0000 7f54857e9640 1 --2- 172.21.15.16:0/2786222500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f546c0039e0 0x7f546c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.984+0000 7f5485fea640 1 --2- 172.21.15.16:0/2786222500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5480099c70 0x7f546c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.984+0000 7f5484fe8640 1 -- 172.21.15.16:0/2786222500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5480099c70 msgr2=0x7f546c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.984+0000 7f5484fe8640 1 --2- 172.21.15.16:0/2786222500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5480099c70 0x7f546c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.984+0000 7f5484fe8640 1 -- 172.21.15.16:0/2786222500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f546c0039e0 msgr2=0x7f546c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.984+0000 7f5484fe8640 1 --2- 172.21.15.16:0/2786222500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f546c0039e0 0x7f546c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.984+0000 7f5484fe8640 1 -- 172.21.15.16:0/2786222500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546c003520 con 0x7f546c004800 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.984+0000 7f5485fea640 1 --2- 172.21.15.16:0/2786222500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5480099c70 0x7f546c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.984+0000 7f5484fe8640 1 --2- 172.21.15.16:0/2786222500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f546c004800 0x7f546c004c80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5474001eb0 tx=0x7f546c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3888e0f362ae5e2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.985+0000 7f5473fff640 1 -- 172.21.15.16:0/2786222500 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474007d30 con 0x7f546c004800 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.985+0000 7f5473fff640 1 -- 172.21.15.16:0/2786222500 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5474005af0 con 0x7f546c004800 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.985+0000 7f5473fff640 1 -- 172.21.15.16:0/2786222500 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474007070 con 0x7f546c004800 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.985+0000 7f5487a4d640 1 -- 172.21.15.16:0/2786222500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f546c004800 msgr2=0x7f546c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.985+0000 7f5487a4d640 1 --2- 172.21.15.16:0/2786222500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f546c004800 0x7f546c004c80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5474001eb0 tx=0x7f546c005830 comp rx=0 tx=0).stop 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.985+0000 7f5485fea640 1 -- 172.21.15.16:0/2786222500 reap_dead start 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.985+0000 7f5487a4d640 1 -- 172.21.15.16:0/2786222500 shutdown_connections 2024-04-25T10:05:49.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.985+0000 7f5487a4d640 1 -- 172.21.15.16:0/2786222500 >> 172.21.15.16:0/2786222500 conn(0x7f54800d1940 msgr2=0x7f54800d1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:49.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.985+0000 7f5487a4d640 1 -- 172.21.15.16:0/2786222500 shutdown_connections 2024-04-25T10:05:49.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.985+0000 7f5487a4d640 1 -- 172.21.15.16:0/2786222500 wait complete. 2024-04-25T10:05:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.986+0000 7f5487a4d640 1 Processor -- start 2024-04-25T10:05:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.987+0000 7f5487a4d640 1 -- start start 2024-04-25T10:05:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.987+0000 7f5487a4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f546c004800 0x7f548006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.987+0000 7f5487a4d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548006dae0 0x7f548006bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.987+0000 7f5487a4d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548015eda0 0x7f5480070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.987+0000 7f5487a4d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548009a100 con 0x7f546c004800 2024-04-25T10:05:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.987+0000 7f5487a4d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54800b1e70 con 0x7f548006dae0 2024-04-25T10:05:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.987+0000 7f5487a4d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548009a280 con 0x7f548015eda0 2024-04-25T10:05:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.987+0000 7f54857e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f546c004800 0x7f548006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.987+0000 7f54857e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f546c004800 0x7f548006d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42712/0 (socket says 172.21.15.16:42712) 2024-04-25T10:05:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.987+0000 7f54857e9640 1 -- 172.21.15.16:0/228872017 learned_addr learned my addr 172.21.15.16:0/228872017 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.987+0000 7f5485fea640 1 --2- 172.21.15.16:0/228872017 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548015eda0 0x7f5480070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.987+0000 7f5484fe8640 1 --2- 172.21.15.16:0/228872017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548006dae0 0x7f548006bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.988+0000 7f54857e9640 1 -- 172.21.15.16:0/228872017 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548015eda0 msgr2=0x7f5480070730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.988+0000 7f54857e9640 1 --2- 172.21.15.16:0/228872017 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548015eda0 0x7f5480070730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.988+0000 7f54857e9640 1 -- 172.21.15.16:0/228872017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548006dae0 msgr2=0x7f548006bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.988+0000 7f54857e9640 1 --2- 172.21.15.16:0/228872017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548006dae0 0x7f548006bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.988+0000 7f54857e9640 1 -- 172.21.15.16:0/228872017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548015c600 con 0x7f546c004800 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.988+0000 7f5484fe8640 1 --2- 172.21.15.16:0/228872017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548006dae0 0x7f548006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.988+0000 7f54857e9640 1 --2- 172.21.15.16:0/228872017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f546c004800 0x7f548006d5a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f547c00f620 tx=0x7f547c00fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.988+0000 7f5471ffb640 1 -- 172.21.15.16:0/228872017 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c0052e0 con 0x7f546c004800 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.988+0000 7f5471ffb640 1 -- 172.21.15.16:0/228872017 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c018070 con 0x7f546c004800 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.988+0000 7f5487a4d640 1 -- 172.21.15.16:0/228872017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548015c8e0 con 0x7f546c004800 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.988+0000 7f5471ffb640 1 -- 172.21.15.16:0/228872017 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c003810 con 0x7f546c004800 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.988+0000 7f5487a4d640 1 -- 172.21.15.16:0/228872017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548015cea0 con 0x7f546c004800 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.989+0000 7f5487a4d640 1 -- 172.21.15.16:0/228872017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54800750b0 con 0x7f546c004800 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.990+0000 7f5471ffb640 1 -- 172.21.15.16:0/228872017 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f547c0039b0 con 0x7f546c004800 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.990+0000 7f5471ffb640 1 --2- 172.21.15.16:0/228872017 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f546406b500 0x7f546406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.990+0000 7f5471ffb640 1 -- 172.21.15.16:0/228872017 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f547c089f80 con 0x7f546c004800 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.991+0000 7f5484fe8640 1 --2- 172.21.15.16:0/228872017 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f546406b500 0x7f546406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.991+0000 7f5484fe8640 1 --2- 172.21.15.16:0/228872017 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f546406b500 0x7f546406d9c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5474032000 tx=0x7f547402d000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:49.995+0000 7f5471ffb640 1 -- 172.21.15.16:0/228872017 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f547c058df0 con 0x7f546c004800 2024-04-25T10:05:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.190+0000 7f5487a4d640 1 -- 172.21.15.16:0/228872017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f54800b78c0 con 0x7f546c004800 2024-04-25T10:05:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.190+0000 7f5471ffb640 1 -- 172.21.15.16:0/228872017 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f547c0586f0 con 0x7f546c004800 2024-04-25T10:05:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-04-25T10:05:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.192+0000 7f5487a4d640 1 -- 172.21.15.16:0/228872017 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f546406b500 msgr2=0x7f546406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.192+0000 7f5487a4d640 1 --2- 172.21.15.16:0/228872017 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f546406b500 0x7f546406d9c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5474032000 tx=0x7f547402d000 comp rx=0 tx=0).stop 2024-04-25T10:05:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.192+0000 7f5487a4d640 1 -- 172.21.15.16:0/228872017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f546c004800 msgr2=0x7f548006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.192+0000 7f5487a4d640 1 --2- 172.21.15.16:0/228872017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f546c004800 0x7f548006d5a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f547c00f620 tx=0x7f547c00fae0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.192+0000 7f5487a4d640 1 -- 172.21.15.16:0/228872017 shutdown_connections 2024-04-25T10:05:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.192+0000 7f5487a4d640 1 --2- 172.21.15.16:0/228872017 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f546406b500 0x7f546406d9c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.192+0000 7f5487a4d640 1 --2- 172.21.15.16:0/228872017 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548015eda0 0x7f5480070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.192+0000 7f5487a4d640 1 --2- 172.21.15.16:0/228872017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548006dae0 0x7f548006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.192+0000 7f5487a4d640 1 --2- 172.21.15.16:0/228872017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f546c004800 0x7f548006d5a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.192+0000 7f5487a4d640 1 -- 172.21.15.16:0/228872017 >> 172.21.15.16:0/228872017 conn(0x7f54800d1940 msgr2=0x7f54800ae7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.192+0000 7f5487a4d640 1 -- 172.21.15.16:0/228872017 shutdown_connections 2024-04-25T10:05:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.192+0000 7f5487a4d640 1 -- 172.21.15.16:0/228872017 wait complete. 2024-04-25T10:05:50.208 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T10:05:50.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:50.208 DEBUG:teuthology.orchestra.run.smithi186:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-25T10:05:50.239 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi186.front.sepia.ceph.com 2024-04-25T10:05:50.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:50.239 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:05:50.297 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:05:50.298 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.393+0000 7f9f77b90640 1 Processor -- start 2024-04-25T10:05:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.393+0000 7f9f77b90640 1 -- start start 2024-04-25T10:05:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.394+0000 7f9f77b90640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f7006b2b0 0x7f9f700ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.394+0000 7f9f77b90640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f70073210 con 0x7f9f700cc0a0 2024-04-25T10:05:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.394+0000 7f9f77b90640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f700cecc0 con 0x7f9f7015baf0 2024-04-25T10:05:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.394+0000 7f9f77b90640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f700cefa0 con 0x7f9f7006b2b0 2024-04-25T10:05:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.394+0000 7f9f7512b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9f700cc0a0 0x7f9f7006aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51476/0 (socket says 172.21.15.16:51476) 2024-04-25T10:05:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.394+0000 7f9f7512b640 1 -- 172.21.15.16:0/1862907517 learned_addr learned my addr 172.21.15.16:0/1862907517 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.395+0000 7f9f7612d640 1 --2- 172.21.15.16:0/1862907517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f7006b2b0 0x7f9f700ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.395+0000 7f9f7492a640 1 -- 172.21.15.16:0/1862907517 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256325542 0 0) 0x7f9f70073210 con 0x7f9f700cc0a0 2024-04-25T10:05:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.395+0000 7f9f7492a640 1 -- 172.21.15.16:0/1862907517 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f5c003520 con 0x7f9f700cc0a0 2024-04-25T10:05:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.395+0000 7f9f7492a640 1 -- 172.21.15.16:0/1862907517 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873029971 0 0) 0x7f9f700cecc0 con 0x7f9f7015baf0 2024-04-25T10:05:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.395+0000 7f9f7492a640 1 -- 172.21.15.16:0/1862907517 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f70073210 con 0x7f9f7015baf0 2024-04-25T10:05:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.395+0000 7f9f7612d640 1 -- 172.21.15.16:0/1862907517 >> v1:172.21.15.186:6789/0 conn(0x7f9f7015baf0 legacy=0x7f9f700cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.395+0000 7f9f7612d640 1 -- 172.21.15.16:0/1862907517 >> v1:172.21.15.16:6789/0 conn(0x7f9f700cc0a0 legacy=0x7f9f7006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.395+0000 7f9f7612d640 1 -- 172.21.15.16:0/1862907517 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f700cf280 con 0x7f9f7006b2b0 2024-04-25T10:05:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.396+0000 7f9f7612d640 1 --2- 172.21.15.16:0/1862907517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f7006b2b0 0x7f9f700ccb90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009dc0 tx=0x7f9f6802a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b2049083cb587f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.396+0000 7f9f7492a640 1 -- 172.21.15.16:0/1862907517 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f68007740 con 0x7f9f7006b2b0 2024-04-25T10:05:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.396+0000 7f9f7492a640 1 -- 172.21.15.16:0/1862907517 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f680078e0 con 0x7f9f7006b2b0 2024-04-25T10:05:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.396+0000 7f9f7492a640 1 -- 172.21.15.16:0/1862907517 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f68007cb0 con 0x7f9f7006b2b0 2024-04-25T10:05:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.397+0000 7f9f77b90640 1 -- 172.21.15.16:0/1862907517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f7006b2b0 msgr2=0x7f9f700ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.397+0000 7f9f77b90640 1 --2- 172.21.15.16:0/1862907517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f7006b2b0 0x7f9f700ccb90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009dc0 tx=0x7f9f6802a5b0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.397+0000 7f9f77b90640 1 -- 172.21.15.16:0/1862907517 shutdown_connections 2024-04-25T10:05:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.397+0000 7f9f77b90640 1 --2- 172.21.15.16:0/1862907517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f7006b2b0 0x7f9f700ccb90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.397+0000 7f9f77b90640 1 -- 172.21.15.16:0/1862907517 >> 172.21.15.16:0/1862907517 conn(0x7f9f700b2370 msgr2=0x7f9f700b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.397+0000 7f9f77b90640 1 -- 172.21.15.16:0/1862907517 shutdown_connections 2024-04-25T10:05:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.397+0000 7f9f77b90640 1 -- 172.21.15.16:0/1862907517 wait complete. 2024-04-25T10:05:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.398+0000 7f9f77b90640 1 Processor -- start 2024-04-25T10:05:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.398+0000 7f9f77b90640 1 -- start start 2024-04-25T10:05:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f77b90640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f7006b2b0 0x7f9f7014b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f77b90640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f700cc0a0 0x7f9f700c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f77b90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f7015baf0 0x7f9f700c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f77b90640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f70070b90 con 0x7f9f7015baf0 2024-04-25T10:05:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f77b90640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f70070a10 con 0x7f9f700cc0a0 2024-04-25T10:05:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f77b90640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f70072b10 con 0x7f9f7006b2b0 2024-04-25T10:05:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f7612d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f7015baf0 0x7f9f700c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f7612d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f7015baf0 0x7f9f700c9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42726/0 (socket says 172.21.15.16:42726) 2024-04-25T10:05:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f7612d640 1 -- 172.21.15.16:0/99387955 learned_addr learned my addr 172.21.15.16:0/99387955 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f7592c640 1 --2- 172.21.15.16:0/99387955 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f7006b2b0 0x7f9f7014b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f7512b640 1 --2- 172.21.15.16:0/99387955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f700cc0a0 0x7f9f700c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f7612d640 1 -- 172.21.15.16:0/99387955 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f7006b2b0 msgr2=0x7f9f7014b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f7612d640 1 --2- 172.21.15.16:0/99387955 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f7006b2b0 0x7f9f7014b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f7612d640 1 -- 172.21.15.16:0/99387955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f700cc0a0 msgr2=0x7f9f700c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f7612d640 1 --2- 172.21.15.16:0/99387955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f700cc0a0 0x7f9f700c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f7612d640 1 -- 172.21.15.16:0/99387955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7014b7c0 con 0x7f9f7015baf0 2024-04-25T10:05:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f7592c640 1 --2- 172.21.15.16:0/99387955 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f7006b2b0 0x7f9f7014b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f7512b640 1 --2- 172.21.15.16:0/99387955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f700cc0a0 0x7f9f700c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f7612d640 1 --2- 172.21.15.16:0/99387955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f7015baf0 0x7f9f700c9a00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9f6802aba0 tx=0x7f9f68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.399+0000 7f9f667fc640 1 -- 172.21.15.16:0/99387955 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f68040070 con 0x7f9f7015baf0 2024-04-25T10:05:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.400+0000 7f9f667fc640 1 -- 172.21.15.16:0/99387955 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f68031bc0 con 0x7f9f7015baf0 2024-04-25T10:05:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.400+0000 7f9f77b90640 1 -- 172.21.15.16:0/99387955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f700c5fb0 con 0x7f9f7015baf0 2024-04-25T10:05:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.400+0000 7f9f667fc640 1 -- 172.21.15.16:0/99387955 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f68009450 con 0x7f9f7015baf0 2024-04-25T10:05:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.400+0000 7f9f77b90640 1 -- 172.21.15.16:0/99387955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f700c6570 con 0x7f9f7015baf0 2024-04-25T10:05:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.401+0000 7f9f77b90640 1 -- 172.21.15.16:0/99387955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f700749b0 con 0x7f9f7015baf0 2024-04-25T10:05:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.401+0000 7f9f667fc640 1 -- 172.21.15.16:0/99387955 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f68039040 con 0x7f9f7015baf0 2024-04-25T10:05:50.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.402+0000 7f9f667fc640 1 --2- 172.21.15.16:0/99387955 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9f5406b4d0 0x7f9f5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.402+0000 7f9f667fc640 1 -- 172.21.15.16:0/99387955 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f680ab960 con 0x7f9f7015baf0 2024-04-25T10:05:50.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.402+0000 7f9f7592c640 1 --2- 172.21.15.16:0/99387955 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9f5406b4d0 0x7f9f5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.403+0000 7f9f7592c640 1 --2- 172.21.15.16:0/99387955 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9f5406b4d0 0x7f9f5406d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f701f7850 tx=0x7f9f60009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.406+0000 7f9f667fc640 1 -- 172.21.15.16:0/99387955 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f7014b7c0 con 0x7f9f7015baf0 2024-04-25T10:05:50.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.631+0000 7f9f77b90640 1 -- 172.21.15.16:0/99387955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f70155920 con 0x7f9f7015baf0 2024-04-25T10:05:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.632+0000 7f9f667fc640 1 -- 172.21.15.16:0/99387955 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4939 (secure 0 0 0) 0x7f9f6807a880 con 0x7f9f7015baf0 2024-04-25T10:05:50.637 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:50.637 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1965168813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1965168813},{"type":"v1","addr":"172.21.15.16:6834","nonce":1965168813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/437791378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":437791378},{"type":"v1","addr":"172.21.15.16:6835","nonce":437791378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34145,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3044177642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3044177642},{"type":"v1","addr":"172.21.15.186:6835","nonce":3044177642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:10.814281+0000","modified":"2024-04-25T10:05:12.805138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34148":{"gid":34148,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/1763853189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1763853189},{"type":"v1","addr":"172.21.15.186:6837","nonce":1763853189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:05:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-04-25T10:05:50.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.633+0000 7f9f77b90640 1 -- 172.21.15.16:0/99387955 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9f5406b4d0 msgr2=0x7f9f5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.633+0000 7f9f77b90640 1 --2- 172.21.15.16:0/99387955 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9f5406b4d0 0x7f9f5406d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f701f7850 tx=0x7f9f60009040 comp rx=0 tx=0).stop 2024-04-25T10:05:50.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.633+0000 7f9f77b90640 1 -- 172.21.15.16:0/99387955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f7015baf0 msgr2=0x7f9f700c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.633+0000 7f9f77b90640 1 --2- 172.21.15.16:0/99387955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f7015baf0 0x7f9f700c9a00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9f6802aba0 tx=0x7f9f68004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:50.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.634+0000 7f9f77b90640 1 -- 172.21.15.16:0/99387955 shutdown_connections 2024-04-25T10:05:50.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.634+0000 7f9f77b90640 1 --2- 172.21.15.16:0/99387955 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9f5406b4d0 0x7f9f5406d990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.634+0000 7f9f77b90640 1 --2- 172.21.15.16:0/99387955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f7015baf0 0x7f9f700c9a00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.634+0000 7f9f77b90640 1 --2- 172.21.15.16:0/99387955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f700cc0a0 0x7f9f700c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.634+0000 7f9f77b90640 1 --2- 172.21.15.16:0/99387955 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f7006b2b0 0x7f9f7014b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.634+0000 7f9f77b90640 1 -- 172.21.15.16:0/99387955 >> 172.21.15.16:0/99387955 conn(0x7f9f700b2370 msgr2=0x7f9f7014bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.634+0000 7f9f77b90640 1 -- 172.21.15.16:0/99387955 shutdown_connections 2024-04-25T10:05:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.634+0000 7f9f77b90640 1 -- 172.21.15.16:0/99387955 wait complete. 2024-04-25T10:05:50.650 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.745+0000 7f5b9ceb6640 1 Processor -- start 2024-04-25T10:05:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.745+0000 7f5b9ceb6640 1 -- start start 2024-04-25T10:05:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.746+0000 7f5b9ceb6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b9809a150 0x7f5b9806bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.746+0000 7f5b9ceb6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b9806c1f0 0x7f5b980cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.746+0000 7f5b9ceb6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b9806c680 con 0x7f5b9815ba10 2024-04-25T10:05:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.746+0000 7f5b9ceb6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b980cbe50 con 0x7f5b9809a150 2024-04-25T10:05:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.746+0000 7f5b9ceb6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b980cfe80 con 0x7f5b9806c1f0 2024-04-25T10:05:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.746+0000 7f5b8ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b9809a150 0x7f5b9806bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.746+0000 7f5b9659c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5b9815ba10 0x7f5b98099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51484/0 (socket says 172.21.15.16:51484) 2024-04-25T10:05:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.746+0000 7f5b9659c640 1 -- 172.21.15.16:0/3981159255 learned_addr learned my addr 172.21.15.16:0/3981159255 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.746+0000 7f5b96d9d640 1 --2- 172.21.15.16:0/3981159255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b9806c1f0 0x7f5b980cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.746+0000 7f5b95d9b640 1 -- 172.21.15.16:0/3981159255 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294704297 0 0) 0x7f5b88001110 con 0x7f5b9815ba10 2024-04-25T10:05:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.746+0000 7f5b95d9b640 1 -- 172.21.15.16:0/3981159255 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b84003520 con 0x7f5b9815ba10 2024-04-25T10:05:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.747+0000 7f5b8ffff640 1 -- 172.21.15.16:0/3981159255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b9806c1f0 msgr2=0x7f5b980cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.747+0000 7f5b8ffff640 1 --2- 172.21.15.16:0/3981159255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b9806c1f0 0x7f5b980cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.747+0000 7f5b8ffff640 1 -- 172.21.15.16:0/3981159255 >> v1:172.21.15.16:6789/0 conn(0x7f5b9815ba10 legacy=0x7f5b98099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.747+0000 7f5b8ffff640 1 -- 172.21.15.16:0/3981159255 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9806c900 con 0x7f5b9809a150 2024-04-25T10:05:50.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.747+0000 7f5b8ffff640 1 --2- 172.21.15.16:0/3981159255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9809a150 0x7f5b9806bcb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5b800097a0 tx=0x7f5b8002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3846db7b06cc1f94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.747+0000 7f5b95d9b640 1 -- 172.21.15.16:0/3981159255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80037070 con 0x7f5b9809a150 2024-04-25T10:05:50.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.747+0000 7f5b95d9b640 1 -- 172.21.15.16:0/3981159255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b80032440 con 0x7f5b9809a150 2024-04-25T10:05:50.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.747+0000 7f5b95d9b640 1 -- 172.21.15.16:0/3981159255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80032810 con 0x7f5b9809a150 2024-04-25T10:05:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.748+0000 7f5b9ceb6640 1 -- 172.21.15.16:0/3981159255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9809a150 msgr2=0x7f5b9806bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.748+0000 7f5b9ceb6640 1 --2- 172.21.15.16:0/3981159255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9809a150 0x7f5b9806bcb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5b800097a0 tx=0x7f5b8002a180 comp rx=0 tx=0).stop 2024-04-25T10:05:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.748+0000 7f5b9ceb6640 1 -- 172.21.15.16:0/3981159255 shutdown_connections 2024-04-25T10:05:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.748+0000 7f5b9ceb6640 1 --2- 172.21.15.16:0/3981159255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b9806c1f0 0x7f5b980cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.748+0000 7f5b9ceb6640 1 --2- 172.21.15.16:0/3981159255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9809a150 0x7f5b9806bcb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.748+0000 7f5b9ceb6640 1 -- 172.21.15.16:0/3981159255 >> 172.21.15.16:0/3981159255 conn(0x7f5b980b32f0 msgr2=0x7f5b980b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.748+0000 7f5b9ceb6640 1 -- 172.21.15.16:0/3981159255 shutdown_connections 2024-04-25T10:05:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.748+0000 7f5b9ceb6640 1 -- 172.21.15.16:0/3981159255 wait complete. 2024-04-25T10:05:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.749+0000 7f5b9ceb6640 1 Processor -- start 2024-04-25T10:05:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.749+0000 7f5b9ceb6640 1 -- start start 2024-04-25T10:05:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.750+0000 7f5b9ceb6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b9806c1f0 0x7f5b981588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.750+0000 7f5b9ceb6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9809a150 0x7f5b980af080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.750+0000 7f5b9ceb6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b9815ba10 0x7f5b980afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.750+0000 7f5b9ceb6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b980cd760 con 0x7f5b9809a150 2024-04-25T10:05:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.750+0000 7f5b9ceb6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9806cec0 con 0x7f5b9815ba10 2024-04-25T10:05:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.750+0000 7f5b9ceb6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b980cd8e0 con 0x7f5b9806c1f0 2024-04-25T10:05:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.750+0000 7f5b8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9809a150 0x7f5b980af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.750+0000 7f5b8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9809a150 0x7f5b980af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42744/0 (socket says 172.21.15.16:42744) 2024-04-25T10:05:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.750+0000 7f5b8ffff640 1 -- 172.21.15.16:0/1109711277 learned_addr learned my addr 172.21.15.16:0/1109711277 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.750+0000 7f5b8ffff640 1 -- 172.21.15.16:0/1109711277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b9806c1f0 msgr2=0x7f5b981588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.750+0000 7f5b9659c640 1 --2- 172.21.15.16:0/1109711277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b9806c1f0 0x7f5b981588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.750+0000 7f5b96d9d640 1 --2- 172.21.15.16:0/1109711277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b9815ba10 0x7f5b980afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.750+0000 7f5b8ffff640 1 --2- 172.21.15.16:0/1109711277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b9806c1f0 0x7f5b981588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.750+0000 7f5b8ffff640 1 -- 172.21.15.16:0/1109711277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b9815ba10 msgr2=0x7f5b980afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.750+0000 7f5b8ffff640 1 --2- 172.21.15.16:0/1109711277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b9815ba10 0x7f5b980afb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.750+0000 7f5b8ffff640 1 -- 172.21.15.16:0/1109711277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b98158de0 con 0x7f5b9809a150 2024-04-25T10:05:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.751+0000 7f5b8ffff640 1 --2- 172.21.15.16:0/1109711277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9809a150 0x7f5b980af080 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5b80035040 tx=0x7f5b800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.751+0000 7f5b8effd640 1 -- 172.21.15.16:0/1109711277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80037040 con 0x7f5b9809a150 2024-04-25T10:05:50.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.751+0000 7f5b8effd640 1 -- 172.21.15.16:0/1109711277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b80030500 con 0x7f5b9809a150 2024-04-25T10:05:50.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.751+0000 7f5b8effd640 1 -- 172.21.15.16:0/1109711277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80030910 con 0x7f5b9809a150 2024-04-25T10:05:50.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.751+0000 7f5b9ceb6640 1 -- 172.21.15.16:0/1109711277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b980ae5e0 con 0x7f5b9809a150 2024-04-25T10:05:50.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.751+0000 7f5b9ceb6640 1 -- 172.21.15.16:0/1109711277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b980aeb60 con 0x7f5b9809a150 2024-04-25T10:05:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.752+0000 7f5b9ceb6640 1 -- 172.21.15.16:0/1109711277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b980749b0 con 0x7f5b9809a150 2024-04-25T10:05:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.752+0000 7f5b8effd640 1 -- 172.21.15.16:0/1109711277 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b80044020 con 0x7f5b9809a150 2024-04-25T10:05:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.753+0000 7f5b8effd640 1 --2- 172.21.15.16:0/1109711277 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5b7806b500 0x7f5b7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.753+0000 7f5b8effd640 1 -- 172.21.15.16:0/1109711277 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5b800aa460 con 0x7f5b9809a150 2024-04-25T10:05:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.753+0000 7f5b9659c640 1 --2- 172.21.15.16:0/1109711277 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5b7806b500 0x7f5b7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.754+0000 7f5b9659c640 1 --2- 172.21.15.16:0/1109711277 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5b7806b500 0x7f5b7806d9c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5b880012b0 tx=0x7f5b88009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.757+0000 7f5b8effd640 1 -- 172.21.15.16:0/1109711277 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b98158de0 con 0x7f5b9809a150 2024-04-25T10:05:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.939+0000 7f5b9ceb6640 1 -- 172.21.15.16:0/1109711277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b980b4120 con 0x7f5b9809a150 2024-04-25T10:05:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.940+0000 7f5b8effd640 1 -- 172.21.15.16:0/1109711277 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16658 (secure 0 0 0) 0x7f5b80079350 con 0x7f5b9809a150 2024-04-25T10:05:50.945 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:50.945 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":23,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:50.578043+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.36","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:50.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.942+0000 7f5b9ceb6640 1 -- 172.21.15.16:0/1109711277 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5b7806b500 msgr2=0x7f5b7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.942+0000 7f5b9ceb6640 1 --2- 172.21.15.16:0/1109711277 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5b7806b500 0x7f5b7806d9c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5b880012b0 tx=0x7f5b88009040 comp rx=0 tx=0).stop 2024-04-25T10:05:50.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.942+0000 7f5b9ceb6640 1 -- 172.21.15.16:0/1109711277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9809a150 msgr2=0x7f5b980af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.942+0000 7f5b9ceb6640 1 --2- 172.21.15.16:0/1109711277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9809a150 0x7f5b980af080 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5b80035040 tx=0x7f5b800046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.942+0000 7f5b9ceb6640 1 -- 172.21.15.16:0/1109711277 shutdown_connections 2024-04-25T10:05:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.942+0000 7f5b9ceb6640 1 --2- 172.21.15.16:0/1109711277 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5b7806b500 0x7f5b7806d9c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.942+0000 7f5b9ceb6640 1 --2- 172.21.15.16:0/1109711277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b9815ba10 0x7f5b980afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.942+0000 7f5b9ceb6640 1 --2- 172.21.15.16:0/1109711277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9809a150 0x7f5b980af080 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.942+0000 7f5b9ceb6640 1 --2- 172.21.15.16:0/1109711277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b9806c1f0 0x7f5b981588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.942+0000 7f5b9ceb6640 1 -- 172.21.15.16:0/1109711277 >> 172.21.15.16:0/1109711277 conn(0x7f5b980b32f0 msgr2=0x7f5b980ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.942+0000 7f5b9ceb6640 1 -- 172.21.15.16:0/1109711277 shutdown_connections 2024-04-25T10:05:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:50.942+0000 7f5b9ceb6640 1 -- 172.21.15.16:0/1109711277 wait complete. 2024-04-25T10:05:50.958 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:08.724537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:09.793105+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:50.958 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:51.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.057+0000 7f1acd412640 1 Processor -- start 2024-04-25T10:05:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.057+0000 7f1acd412640 1 -- start start 2024-04-25T10:05:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.058+0000 7f1acd412640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ac815dca0 0x7f1ac80ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.058+0000 7f1acd412640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1ac80d3b90 0x7f1ac80d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.058+0000 7f1acd412640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ac80cee80 0x7f1ac80d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.058+0000 7f1acd412640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac80b6b20 con 0x7f1ac80cee80 2024-04-25T10:05:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.058+0000 7f1acd412640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac80d4090 con 0x7f1ac80d3b90 2024-04-25T10:05:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.058+0000 7f1acd412640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac80d4370 con 0x7f1ac815dca0 2024-04-25T10:05:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.058+0000 7f1ac77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ac80cee80 0x7f1ac80d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.058+0000 7f1ac77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ac80cee80 0x7f1ac80d0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42746/0 (socket says 172.21.15.16:42746) 2024-04-25T10:05:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.058+0000 7f1ac77fe640 1 -- 172.21.15.16:0/1944982681 learned_addr learned my addr 172.21.15.16:0/1944982681 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.058+0000 7f1ac6ffd640 1 --2- 172.21.15.16:0/1944982681 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ac815dca0 0x7f1ac80ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.058+0000 7f1ac67fc640 1 --2- 172.21.15.16:0/1944982681 >> v2:172.21.15.186:3300/0 conn(0x7f1ac80d3b90 0x7f1ac80d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.058+0000 7f1ac77fe640 1 -- 172.21.15.16:0/1944982681 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ac815dca0 msgr2=0x7f1ac80ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.058+0000 7f1ac77fe640 1 --2- 172.21.15.16:0/1944982681 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ac815dca0 0x7f1ac80ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.058+0000 7f1ac77fe640 1 -- 172.21.15.16:0/1944982681 >> v2:172.21.15.186:3300/0 conn(0x7f1ac80d3b90 msgr2=0x7f1ac80d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.058+0000 7f1ac77fe640 1 --2- 172.21.15.16:0/1944982681 >> v2:172.21.15.186:3300/0 conn(0x7f1ac80d3b90 0x7f1ac80d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.058+0000 7f1ac77fe640 1 -- 172.21.15.16:0/1944982681 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac80d4650 con 0x7f1ac80cee80 2024-04-25T10:05:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.059+0000 7f1ac77fe640 1 --2- 172.21.15.16:0/1944982681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ac80cee80 0x7f1ac80d0f60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1ac806d210 tx=0x7f1ac002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1feb565a2737a80e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.059+0000 7f1ac5ffb640 1 -- 172.21.15.16:0/1944982681 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0036070 con 0x7f1ac80cee80 2024-04-25T10:05:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.059+0000 7f1ac5ffb640 1 -- 172.21.15.16:0/1944982681 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac0031600 con 0x7f1ac80cee80 2024-04-25T10:05:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.059+0000 7f1ac5ffb640 1 -- 172.21.15.16:0/1944982681 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00319d0 con 0x7f1ac80cee80 2024-04-25T10:05:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.060+0000 7f1acd412640 1 -- 172.21.15.16:0/1944982681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ac80cee80 msgr2=0x7f1ac80d0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.060+0000 7f1acd412640 1 --2- 172.21.15.16:0/1944982681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ac80cee80 0x7f1ac80d0f60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1ac806d210 tx=0x7f1ac002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.060+0000 7f1acd412640 1 -- 172.21.15.16:0/1944982681 shutdown_connections 2024-04-25T10:05:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.060+0000 7f1acd412640 1 --2- 172.21.15.16:0/1944982681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ac80cee80 0x7f1ac80d0f60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.060+0000 7f1acd412640 1 --2- 172.21.15.16:0/1944982681 >> v2:172.21.15.186:3300/0 conn(0x7f1ac80d3b90 0x7f1ac80d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.060+0000 7f1acd412640 1 --2- 172.21.15.16:0/1944982681 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ac815dca0 0x7f1ac80ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.060+0000 7f1acd412640 1 -- 172.21.15.16:0/1944982681 >> 172.21.15.16:0/1944982681 conn(0x7f1ac80726c0 msgr2=0x7f1ac8072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:51.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.060+0000 7f1acd412640 1 -- 172.21.15.16:0/1944982681 shutdown_connections 2024-04-25T10:05:51.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.060+0000 7f1acd412640 1 -- 172.21.15.16:0/1944982681 wait complete. 2024-04-25T10:05:51.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.061+0000 7f1acd412640 1 Processor -- start 2024-04-25T10:05:51.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.061+0000 7f1acd412640 1 -- start start 2024-04-25T10:05:51.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.062+0000 7f1acd412640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ac80cee80 0x7f1ac81526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.062+0000 7f1acd412640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ac80d3b90 0x7f1ac80cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.062+0000 7f1acd412640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ac815dca0 0x7f1ac80cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.062+0000 7f1acd412640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac815e200 con 0x7f1ac80d3b90 2024-04-25T10:05:51.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.062+0000 7f1acd412640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac815e080 con 0x7f1ac80cee80 2024-04-25T10:05:51.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.062+0000 7f1acd412640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac80b2500 con 0x7f1ac815dca0 2024-04-25T10:05:51.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.062+0000 7f1ac67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ac80d3b90 0x7f1ac80cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.062+0000 7f1ac67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ac80d3b90 0x7f1ac80cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42750/0 (socket says 172.21.15.16:42750) 2024-04-25T10:05:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.062+0000 7f1ac67fc640 1 -- 172.21.15.16:0/670322147 learned_addr learned my addr 172.21.15.16:0/670322147 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.062+0000 7f1ac6ffd640 1 --2- 172.21.15.16:0/670322147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ac80cee80 0x7f1ac81526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.062+0000 7f1ac77fe640 1 --2- 172.21.15.16:0/670322147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ac815dca0 0x7f1ac80cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.062+0000 7f1ac67fc640 1 -- 172.21.15.16:0/670322147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ac815dca0 msgr2=0x7f1ac80cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.062+0000 7f1ac67fc640 1 --2- 172.21.15.16:0/670322147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ac815dca0 0x7f1ac80cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.062+0000 7f1ac67fc640 1 -- 172.21.15.16:0/670322147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ac80cee80 msgr2=0x7f1ac81526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.062+0000 7f1ac67fc640 1 --2- 172.21.15.16:0/670322147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ac80cee80 0x7f1ac81526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.062+0000 7f1ac67fc640 1 -- 172.21.15.16:0/670322147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac8153240 con 0x7f1ac80d3b90 2024-04-25T10:05:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.063+0000 7f1ac67fc640 1 --2- 172.21.15.16:0/670322147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ac80d3b90 0x7f1ac80cd060 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00d800 tx=0x7f1abc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.063+0000 7f1aaf7fe640 1 -- 172.21.15.16:0/670322147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc018020 con 0x7f1ac80d3b90 2024-04-25T10:05:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.063+0000 7f1aaf7fe640 1 -- 172.21.15.16:0/670322147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1abc0096a0 con 0x7f1ac80d3b90 2024-04-25T10:05:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.063+0000 7f1acd412640 1 -- 172.21.15.16:0/670322147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac815f3a0 con 0x7f1ac80d3b90 2024-04-25T10:05:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.063+0000 7f1aaf7fe640 1 -- 172.21.15.16:0/670322147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc009a70 con 0x7f1ac80d3b90 2024-04-25T10:05:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.063+0000 7f1acd412640 1 -- 172.21.15.16:0/670322147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac815f960 con 0x7f1ac80d3b90 2024-04-25T10:05:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.064+0000 7f1acd412640 1 -- 172.21.15.16:0/670322147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac80d0c90 con 0x7f1ac80d3b90 2024-04-25T10:05:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.064+0000 7f1aaf7fe640 1 -- 172.21.15.16:0/670322147 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1abc01b020 con 0x7f1ac80d3b90 2024-04-25T10:05:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.064+0000 7f1aaf7fe640 1 --2- 172.21.15.16:0/670322147 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1ab406b4d0 0x7f1ab406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.065+0000 7f1aaf7fe640 1 -- 172.21.15.16:0/670322147 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1abc08a6e0 con 0x7f1ac80d3b90 2024-04-25T10:05:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.065+0000 7f1ac6ffd640 1 --2- 172.21.15.16:0/670322147 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1ab406b4d0 0x7f1ab406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.065+0000 7f1ac6ffd640 1 --2- 172.21.15.16:0/670322147 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1ab406b4d0 0x7f1ab406d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8006fd0 tx=0x7f1ab8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.069+0000 7f1aaf7fe640 1 -- 172.21.15.16:0/670322147 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ac80d0c90 con 0x7f1ac80d3b90 2024-04-25T10:05:51.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.248+0000 7f1acd412640 1 -- 172.21.15.16:0/670322147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ac80ca7e0 con 0x7f1ac80d3b90 2024-04-25T10:05:51.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.248+0000 7f1aaf7fe640 1 -- 172.21.15.16:0/670322147 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1abc022020 con 0x7f1ac80d3b90 2024-04-25T10:05:51.253 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:05:51.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.250+0000 7f1acd412640 1 -- 172.21.15.16:0/670322147 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1ab406b4d0 msgr2=0x7f1ab406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.250+0000 7f1acd412640 1 --2- 172.21.15.16:0/670322147 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1ab406b4d0 0x7f1ab406d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8006fd0 tx=0x7f1ab8008040 comp rx=0 tx=0).stop 2024-04-25T10:05:51.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.250+0000 7f1acd412640 1 -- 172.21.15.16:0/670322147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ac80d3b90 msgr2=0x7f1ac80cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.250+0000 7f1acd412640 1 --2- 172.21.15.16:0/670322147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ac80d3b90 0x7f1ac80cd060 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00d800 tx=0x7f1abc00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.250+0000 7f1acd412640 1 -- 172.21.15.16:0/670322147 shutdown_connections 2024-04-25T10:05:51.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.250+0000 7f1acd412640 1 --2- 172.21.15.16:0/670322147 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1ab406b4d0 0x7f1ab406d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.250+0000 7f1acd412640 1 --2- 172.21.15.16:0/670322147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ac815dca0 0x7f1ac80cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.250+0000 7f1acd412640 1 --2- 172.21.15.16:0/670322147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ac80d3b90 0x7f1ac80cd060 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.250+0000 7f1acd412640 1 --2- 172.21.15.16:0/670322147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ac80cee80 0x7f1ac81526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.250+0000 7f1acd412640 1 -- 172.21.15.16:0/670322147 >> 172.21.15.16:0/670322147 conn(0x7f1ac80726c0 msgr2=0x7f1ac80b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:51.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.250+0000 7f1acd412640 1 -- 172.21.15.16:0/670322147 shutdown_connections 2024-04-25T10:05:51.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.250+0000 7f1acd412640 1 -- 172.21.15.16:0/670322147 wait complete. 2024-04-25T10:05:51.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.364+0000 7f3bb9615640 1 Processor -- start 2024-04-25T10:05:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.364+0000 7f3bb9615640 1 -- start start 2024-04-25T10:05:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.365+0000 7f3bb9615640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3bb415baf0 0x7f3bb406aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.365+0000 7f3bb9615640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bb40b13c0 0x7f3bb406b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.365+0000 7f3bb9615640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb40cd450 con 0x7f3bb40ccee0 2024-04-25T10:05:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.365+0000 7f3bb9615640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb40ce060 con 0x7f3bb415baf0 2024-04-25T10:05:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.365+0000 7f3bb9615640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb40cd730 con 0x7f3bb40b13c0 2024-04-25T10:05:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.365+0000 7f3bb2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3bb415baf0 0x7f3bb406aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.365+0000 7f3bb2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3bb415baf0 0x7f3bb406aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42756/0 (socket says 172.21.15.16:42756) 2024-04-25T10:05:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.365+0000 7f3bb2ffd640 1 -- 172.21.15.16:0/3422093135 learned_addr learned my addr 172.21.15.16:0/3422093135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.366+0000 7f3bb37fe640 1 --2- 172.21.15.16:0/3422093135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bb40b13c0 0x7f3bb406b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.366+0000 7f3bb2ffd640 1 -- 172.21.15.16:0/3422093135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bb40b13c0 msgr2=0x7f3bb406b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.366+0000 7f3bb2ffd640 1 --2- 172.21.15.16:0/3422093135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bb40b13c0 0x7f3bb406b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.366+0000 7f3bb2ffd640 1 -- 172.21.15.16:0/3422093135 >> v1:172.21.15.186:6789/0 conn(0x7f3bb40ccee0 legacy=0x7f3bb415f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.366+0000 7f3bb2ffd640 1 -- 172.21.15.16:0/3422093135 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb40cda10 con 0x7f3bb415baf0 2024-04-25T10:05:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.366+0000 7f3bb37fe640 1 --2- 172.21.15.16:0/3422093135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bb40b13c0 0x7f3bb406b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.366+0000 7f3bb2ffd640 1 --2- 172.21.15.16:0/3422093135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb415baf0 0x7f3bb406aba0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40ce1e0 tx=0x7f3bac02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a9e4b2c7272273 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.366+0000 7f3bb1ffb640 1 -- 172.21.15.16:0/3422093135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac036070 con 0x7f3bb415baf0 2024-04-25T10:05:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.367+0000 7f3bb1ffb640 1 -- 172.21.15.16:0/3422093135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bac031600 con 0x7f3bb415baf0 2024-04-25T10:05:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.367+0000 7f3bb1ffb640 1 -- 172.21.15.16:0/3422093135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac0319d0 con 0x7f3bb415baf0 2024-04-25T10:05:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.367+0000 7f3bb9615640 1 -- 172.21.15.16:0/3422093135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb415baf0 msgr2=0x7f3bb406aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.367+0000 7f3bb9615640 1 --2- 172.21.15.16:0/3422093135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb415baf0 0x7f3bb406aba0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40ce1e0 tx=0x7f3bac02a6c0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.367+0000 7f3bb9615640 1 -- 172.21.15.16:0/3422093135 shutdown_connections 2024-04-25T10:05:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.367+0000 7f3bb9615640 1 --2- 172.21.15.16:0/3422093135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bb40b13c0 0x7f3bb406b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.367+0000 7f3bb9615640 1 --2- 172.21.15.16:0/3422093135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb415baf0 0x7f3bb406aba0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.367+0000 7f3bb9615640 1 -- 172.21.15.16:0/3422093135 >> 172.21.15.16:0/3422093135 conn(0x7f3bb40b2390 msgr2=0x7f3bb40b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.367+0000 7f3bb9615640 1 -- 172.21.15.16:0/3422093135 shutdown_connections 2024-04-25T10:05:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.367+0000 7f3bb9615640 1 -- 172.21.15.16:0/3422093135 wait complete. 2024-04-25T10:05:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.368+0000 7f3bb9615640 1 Processor -- start 2024-04-25T10:05:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.369+0000 7f3bb9615640 1 -- start start 2024-04-25T10:05:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.369+0000 7f3bb9615640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb40b13c0 0x7f3bb414fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.369+0000 7f3bb9615640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bb40ccee0 0x7f3bb40aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.369+0000 7f3bb9615640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bb415baf0 0x7f3bb40af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.369+0000 7f3bb9615640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb40c41a0 con 0x7f3bb40b13c0 2024-04-25T10:05:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.369+0000 7f3bb9615640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb40c4020 con 0x7f3bb40ccee0 2024-04-25T10:05:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.369+0000 7f3bb9615640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb415bed0 con 0x7f3bb415baf0 2024-04-25T10:05:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.369+0000 7f3bb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb40b13c0 0x7f3bb414fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.369+0000 7f3bb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb40b13c0 0x7f3bb414fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42768/0 (socket says 172.21.15.16:42768) 2024-04-25T10:05:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.369+0000 7f3bb2ffd640 1 -- 172.21.15.16:0/3358340338 learned_addr learned my addr 172.21.15.16:0/3358340338 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.370+0000 7f3bb37fe640 1 --2- 172.21.15.16:0/3358340338 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bb415baf0 0x7f3bb40af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.369+0000 7f3bb27fc640 1 --2- 172.21.15.16:0/3358340338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bb40ccee0 0x7f3bb40aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.370+0000 7f3bb2ffd640 1 -- 172.21.15.16:0/3358340338 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bb415baf0 msgr2=0x7f3bb40af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.370+0000 7f3bb2ffd640 1 --2- 172.21.15.16:0/3358340338 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bb415baf0 0x7f3bb40af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.370+0000 7f3bb2ffd640 1 -- 172.21.15.16:0/3358340338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bb40ccee0 msgr2=0x7f3bb40aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.370+0000 7f3bb2ffd640 1 --2- 172.21.15.16:0/3358340338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bb40ccee0 0x7f3bb40aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.370+0000 7f3bb2ffd640 1 -- 172.21.15.16:0/3358340338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb40c7a90 con 0x7f3bb40b13c0 2024-04-25T10:05:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.370+0000 7f3bb2ffd640 1 --2- 172.21.15.16:0/3358340338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb40b13c0 0x7f3bb414fe60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3bac034040 tx=0x7f3bac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.370+0000 7f3bab7fe640 1 -- 172.21.15.16:0/3358340338 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac036050 con 0x7f3bb40b13c0 2024-04-25T10:05:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.370+0000 7f3bab7fe640 1 -- 172.21.15.16:0/3358340338 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bac042070 con 0x7f3bb40b13c0 2024-04-25T10:05:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.370+0000 7f3bb9615640 1 -- 172.21.15.16:0/3358340338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb40c7d70 con 0x7f3bb40b13c0 2024-04-25T10:05:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.370+0000 7f3bab7fe640 1 -- 172.21.15.16:0/3358340338 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac009510 con 0x7f3bb40b13c0 2024-04-25T10:05:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.370+0000 7f3bb9615640 1 -- 172.21.15.16:0/3358340338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb40c8330 con 0x7f3bb40b13c0 2024-04-25T10:05:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.371+0000 7f3bb9615640 1 -- 172.21.15.16:0/3358340338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb40750b0 con 0x7f3bb40b13c0 2024-04-25T10:05:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.372+0000 7f3bab7fe640 1 -- 172.21.15.16:0/3358340338 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3bac007c70 con 0x7f3bb40b13c0 2024-04-25T10:05:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.372+0000 7f3bab7fe640 1 --2- 172.21.15.16:0/3358340338 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b9406b4d0 0x7f3b9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.372+0000 7f3bab7fe640 1 -- 172.21.15.16:0/3358340338 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f3bac0ab000 con 0x7f3bb40b13c0 2024-04-25T10:05:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.372+0000 7f3bb27fc640 1 --2- 172.21.15.16:0/3358340338 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b9406b4d0 0x7f3b9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.373+0000 7f3bb27fc640 1 --2- 172.21.15.16:0/3358340338 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b9406b4d0 0x7f3b9406d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3bb406fe40 tx=0x7f3b9c007e30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.376+0000 7f3bab7fe640 1 -- 172.21.15.16:0/3358340338 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3bac0798a0 con 0x7f3bb40b13c0 2024-04-25T10:05:51.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.561+0000 7f3bb9615640 1 -- 172.21.15.16:0/3358340338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3bb40b67c0 con 0x7f3bb40b13c0 2024-04-25T10:05:51.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.561+0000 7f3bab7fe640 1 -- 172.21.15.16:0/3358340338 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3bac07d940 con 0x7f3bb40b13c0 2024-04-25T10:05:51.567 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:05:51.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.563+0000 7f3bb9615640 1 -- 172.21.15.16:0/3358340338 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b9406b4d0 msgr2=0x7f3b9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.563+0000 7f3bb9615640 1 --2- 172.21.15.16:0/3358340338 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b9406b4d0 0x7f3b9406d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3bb406fe40 tx=0x7f3b9c007e30 comp rx=0 tx=0).stop 2024-04-25T10:05:51.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.563+0000 7f3bb9615640 1 -- 172.21.15.16:0/3358340338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb40b13c0 msgr2=0x7f3bb414fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.563+0000 7f3bb9615640 1 --2- 172.21.15.16:0/3358340338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb40b13c0 0x7f3bb414fe60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3bac034040 tx=0x7f3bac004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:51.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.564+0000 7f3bb9615640 1 -- 172.21.15.16:0/3358340338 shutdown_connections 2024-04-25T10:05:51.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.564+0000 7f3bb9615640 1 --2- 172.21.15.16:0/3358340338 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b9406b4d0 0x7f3b9406d990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.564+0000 7f3bb9615640 1 --2- 172.21.15.16:0/3358340338 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bb415baf0 0x7f3bb40af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.564+0000 7f3bb9615640 1 --2- 172.21.15.16:0/3358340338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bb40ccee0 0x7f3bb40aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.564+0000 7f3bb9615640 1 --2- 172.21.15.16:0/3358340338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb40b13c0 0x7f3bb414fe60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.564+0000 7f3bb9615640 1 -- 172.21.15.16:0/3358340338 >> 172.21.15.16:0/3358340338 conn(0x7f3bb40b2390 msgr2=0x7f3bb4159800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:51.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.564+0000 7f3bb9615640 1 -- 172.21.15.16:0/3358340338 shutdown_connections 2024-04-25T10:05:51.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.564+0000 7f3bb9615640 1 -- 172.21.15.16:0/3358340338 wait complete. 2024-04-25T10:05:51.579 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:51.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.679+0000 7fe7a253b640 1 Processor -- start 2024-04-25T10:05:51.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.679+0000 7fe7a253b640 1 -- start start 2024-04-25T10:05:51.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.680+0000 7fe7a253b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe79c0ce410 0x7fe79c0cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.680+0000 7fe7a253b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe79c0cee20 0x7fe79c0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.680+0000 7fe7a253b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe79c0cb350 con 0x7fe79c15e040 2024-04-25T10:05:51.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.680+0000 7fe7a253b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe79c0cf7b0 con 0x7fe79c0ce410 2024-04-25T10:05:51.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.680+0000 7fe7a253b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe79c0ce9d0 con 0x7fe79c0cee20 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.680+0000 7fe79b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe79c0ce410 0x7fe79c0cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.680+0000 7fe79bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe79c15e040 0x7fe79c0cdbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51498/0 (socket says 172.21.15.16:51498) 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.680+0000 7fe79bfff640 1 -- 172.21.15.16:0/4159679242 learned_addr learned my addr 172.21.15.16:0/4159679242 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.681+0000 7fe7a0ad8640 1 --2- 172.21.15.16:0/4159679242 >> v2:172.21.15.186:3300/0 conn(0x7fe79c0cee20 0x7fe79c0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.681+0000 7fe79b7fe640 1 -- 172.21.15.16:0/4159679242 >> v2:172.21.15.186:3300/0 conn(0x7fe79c0cee20 msgr2=0x7fe79c0cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.681+0000 7fe79b7fe640 1 --2- 172.21.15.16:0/4159679242 >> v2:172.21.15.186:3300/0 conn(0x7fe79c0cee20 0x7fe79c0cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.681+0000 7fe79b7fe640 1 -- 172.21.15.16:0/4159679242 >> v1:172.21.15.16:6789/0 conn(0x7fe79c15e040 legacy=0x7fe79c0cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.681+0000 7fe79affd640 1 -- 172.21.15.16:0/4159679242 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711571782 0 0) 0x7fe79c0cb350 con 0x7fe79c15e040 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.681+0000 7fe79b7fe640 1 -- 172.21.15.16:0/4159679242 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79c0cb5f0 con 0x7fe79c0ce410 2024-04-25T10:05:51.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.682+0000 7fe79b7fe640 1 --2- 172.21.15.16:0/4159679242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79c0ce410 0x7fe79c0cc0d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe7900097a0 tx=0x7fe79002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3436593ce9f138 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.682+0000 7fe79affd640 1 -- 172.21.15.16:0/4159679242 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790037070 con 0x7fe79c0ce410 2024-04-25T10:05:51.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.682+0000 7fe79affd640 1 -- 172.21.15.16:0/4159679242 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe790032440 con 0x7fe79c0ce410 2024-04-25T10:05:51.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.682+0000 7fe79affd640 1 -- 172.21.15.16:0/4159679242 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790032810 con 0x7fe79c0ce410 2024-04-25T10:05:51.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.682+0000 7fe7a253b640 1 -- 172.21.15.16:0/4159679242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79c0ce410 msgr2=0x7fe79c0cc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.682+0000 7fe7a253b640 1 --2- 172.21.15.16:0/4159679242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79c0ce410 0x7fe79c0cc0d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe7900097a0 tx=0x7fe79002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.682+0000 7fe7a253b640 1 -- 172.21.15.16:0/4159679242 shutdown_connections 2024-04-25T10:05:51.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.682+0000 7fe7a253b640 1 --2- 172.21.15.16:0/4159679242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79c0ce410 0x7fe79c0cc0d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.682+0000 7fe7a253b640 1 --2- 172.21.15.16:0/4159679242 >> v2:172.21.15.186:3300/0 conn(0x7fe79c0cee20 0x7fe79c0cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.682+0000 7fe7a253b640 1 -- 172.21.15.16:0/4159679242 >> 172.21.15.16:0/4159679242 conn(0x7fe79c0b2390 msgr2=0x7fe79c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:51.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.682+0000 7fe7a253b640 1 -- 172.21.15.16:0/4159679242 shutdown_connections 2024-04-25T10:05:51.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.682+0000 7fe7a253b640 1 -- 172.21.15.16:0/4159679242 wait complete. 2024-04-25T10:05:51.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.683+0000 7fe7a253b640 1 Processor -- start 2024-04-25T10:05:51.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.684+0000 7fe7a253b640 1 -- start start 2024-04-25T10:05:51.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.684+0000 7fe7a253b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe79c0ce410 0x7fe79c0c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.684+0000 7fe7a253b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79c0cee20 0x7fe79c1f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.684+0000 7fe7a253b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe79c15e040 0x7fe79c159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.684+0000 7fe7a253b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c15e420 con 0x7fe79c0cee20 2024-04-25T10:05:51.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.684+0000 7fe7a253b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c0cbbb0 con 0x7fe79c15e040 2024-04-25T10:05:51.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.684+0000 7fe7a253b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c0cc610 con 0x7fe79c0ce410 2024-04-25T10:05:51.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.684+0000 7fe79b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79c0cee20 0x7fe79c1f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.684+0000 7fe79b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79c0cee20 0x7fe79c1f46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42792/0 (socket says 172.21.15.16:42792) 2024-04-25T10:05:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.684+0000 7fe79b7fe640 1 -- 172.21.15.16:0/1267368273 learned_addr learned my addr 172.21.15.16:0/1267368273 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.685+0000 7fe79bfff640 1 --2- 172.21.15.16:0/1267368273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe79c0ce410 0x7fe79c0c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.685+0000 7fe7a0ad8640 1 --2- 172.21.15.16:0/1267368273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe79c15e040 0x7fe79c159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.685+0000 7fe79b7fe640 1 -- 172.21.15.16:0/1267368273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe79c0ce410 msgr2=0x7fe79c0c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.685+0000 7fe79b7fe640 1 --2- 172.21.15.16:0/1267368273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe79c0ce410 0x7fe79c0c6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.685+0000 7fe79b7fe640 1 -- 172.21.15.16:0/1267368273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe79c15e040 msgr2=0x7fe79c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.685+0000 7fe79b7fe640 1 --2- 172.21.15.16:0/1267368273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe79c15e040 0x7fe79c159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.685+0000 7fe79b7fe640 1 -- 172.21.15.16:0/1267368273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79c0c4930 con 0x7fe79c0cee20 2024-04-25T10:05:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.685+0000 7fe79b7fe640 1 --2- 172.21.15.16:0/1267368273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79c0cee20 0x7fe79c1f46c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe790035040 tx=0x7fe7900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.685+0000 7fe798ff9640 1 -- 172.21.15.16:0/1267368273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790037040 con 0x7fe79c0cee20 2024-04-25T10:05:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.685+0000 7fe798ff9640 1 -- 172.21.15.16:0/1267368273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe790030770 con 0x7fe79c0cee20 2024-04-25T10:05:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.685+0000 7fe7a253b640 1 -- 172.21.15.16:0/1267368273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79c0c4c10 con 0x7fe79c0cee20 2024-04-25T10:05:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.685+0000 7fe798ff9640 1 -- 172.21.15.16:0/1267368273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790030b80 con 0x7fe79c0cee20 2024-04-25T10:05:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.685+0000 7fe7a253b640 1 -- 172.21.15.16:0/1267368273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79c0c7170 con 0x7fe79c0cee20 2024-04-25T10:05:51.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.686+0000 7fe7a253b640 1 -- 172.21.15.16:0/1267368273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79c0750b0 con 0x7fe79c0cee20 2024-04-25T10:05:51.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.687+0000 7fe798ff9640 1 -- 172.21.15.16:0/1267368273 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe790044020 con 0x7fe79c0cee20 2024-04-25T10:05:51.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.687+0000 7fe798ff9640 1 --2- 172.21.15.16:0/1267368273 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe78806b4d0 0x7fe78806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.687+0000 7fe798ff9640 1 -- 172.21.15.16:0/1267368273 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe7900aa790 con 0x7fe79c0cee20 2024-04-25T10:05:51.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.687+0000 7fe79bfff640 1 --2- 172.21.15.16:0/1267368273 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe78806b4d0 0x7fe78806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.688+0000 7fe79bfff640 1 --2- 172.21.15.16:0/1267368273 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe78806b4d0 0x7fe78806d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe78c000f60 tx=0x7fe78c007b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.691+0000 7fe798ff9640 1 -- 172.21.15.16:0/1267368273 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe790032e20 con 0x7fe79c0cee20 2024-04-25T10:05:51.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.871+0000 7fe7a253b640 1 -- 172.21.15.16:0/1267368273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe79c0d0620 con 0x7fe79c0cee20 2024-04-25T10:05:51.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.872+0000 7fe798ff9640 1 -- 172.21.15.16:0/1267368273 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe7900795f0 con 0x7fe79c0cee20 2024-04-25T10:05:51.877 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:05:51.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.873+0000 7fe7a253b640 1 -- 172.21.15.16:0/1267368273 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe78806b4d0 msgr2=0x7fe78806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.873+0000 7fe7a253b640 1 --2- 172.21.15.16:0/1267368273 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe78806b4d0 0x7fe78806d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe78c000f60 tx=0x7fe78c007b50 comp rx=0 tx=0).stop 2024-04-25T10:05:51.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.873+0000 7fe7a253b640 1 -- 172.21.15.16:0/1267368273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79c0cee20 msgr2=0x7fe79c1f46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.873+0000 7fe7a253b640 1 --2- 172.21.15.16:0/1267368273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79c0cee20 0x7fe79c1f46c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe790035040 tx=0x7fe7900046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.873+0000 7fe7a253b640 1 -- 172.21.15.16:0/1267368273 shutdown_connections 2024-04-25T10:05:51.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.873+0000 7fe7a253b640 1 --2- 172.21.15.16:0/1267368273 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe78806b4d0 0x7fe78806d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.873+0000 7fe7a253b640 1 --2- 172.21.15.16:0/1267368273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe79c15e040 0x7fe79c159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.873+0000 7fe7a253b640 1 --2- 172.21.15.16:0/1267368273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79c0cee20 0x7fe79c1f46c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.873+0000 7fe7a253b640 1 --2- 172.21.15.16:0/1267368273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe79c0ce410 0x7fe79c0c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.873+0000 7fe7a253b640 1 -- 172.21.15.16:0/1267368273 >> 172.21.15.16:0/1267368273 conn(0x7fe79c0b2390 msgr2=0x7fe79c0aeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:51.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.874+0000 7fe7a253b640 1 -- 172.21.15.16:0/1267368273 shutdown_connections 2024-04-25T10:05:51.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.874+0000 7fe7a253b640 1 -- 172.21.15.16:0/1267368273 wait complete. 2024-04-25T10:05:51.890 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:51.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.985+0000 7f0867709640 1 Processor -- start 2024-04-25T10:05:51.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.986+0000 7f0867709640 1 -- start start 2024-04-25T10:05:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.986+0000 7f0867709640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f086015dce0 0x7f08600d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.986+0000 7f0867709640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f08600d35b0 0x7f08600b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.986+0000 7f0867709640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f086015e0c0 con 0x7f08600d1f90 2024-04-25T10:05:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.986+0000 7f0867709640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08600b7b10 con 0x7f08600d35b0 2024-04-25T10:05:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.986+0000 7f0867709640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08600d2510 con 0x7f086015dce0 2024-04-25T10:05:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.987+0000 7f0864ca4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f08600d35b0 0x7f08600b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.987+0000 7f0864ca4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f08600d35b0 0x7f08600b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42800/0 (socket says 172.21.15.16:42800) 2024-04-25T10:05:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.987+0000 7f0864ca4640 1 -- 172.21.15.16:0/2827450751 learned_addr learned my addr 172.21.15.16:0/2827450751 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.987+0000 7f0864ca4640 1 -- 172.21.15.16:0/2827450751 >> v2:172.21.15.186:3300/0 conn(0x7f086015dce0 msgr2=0x7f08600d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.987+0000 7f08654a5640 1 --2- 172.21.15.16:0/2827450751 >> v2:172.21.15.186:3300/0 conn(0x7f086015dce0 0x7f08600d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.987+0000 7f0864ca4640 1 --2- 172.21.15.16:0/2827450751 >> v2:172.21.15.186:3300/0 conn(0x7f086015dce0 0x7f08600d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.987+0000 7f0864ca4640 1 -- 172.21.15.16:0/2827450751 >> v1:172.21.15.186:6789/0 conn(0x7f08600d1f90 legacy=0x7f086006bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.987+0000 7f0864ca4640 1 -- 172.21.15.16:0/2827450751 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08600d27f0 con 0x7f08600d35b0 2024-04-25T10:05:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.987+0000 7f08654a5640 1 --2- 172.21.15.16:0/2827450751 >> v2:172.21.15.186:3300/0 conn(0x7f086015dce0 0x7f08600d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.988+0000 7f0864ca4640 1 --2- 172.21.15.16:0/2827450751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08600d35b0 0x7f08600b5450 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f08600ce220 tx=0x7f085002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2be0c224b279bf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.988+0000 7f0857fff640 1 -- 172.21.15.16:0/2827450751 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850037070 con 0x7f08600d35b0 2024-04-25T10:05:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.988+0000 7f0857fff640 1 -- 172.21.15.16:0/2827450751 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0850032440 con 0x7f08600d35b0 2024-04-25T10:05:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.988+0000 7f0857fff640 1 -- 172.21.15.16:0/2827450751 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850032810 con 0x7f08600d35b0 2024-04-25T10:05:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.988+0000 7f0867709640 1 -- 172.21.15.16:0/2827450751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08600d35b0 msgr2=0x7f08600b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.988+0000 7f0867709640 1 --2- 172.21.15.16:0/2827450751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08600d35b0 0x7f08600b5450 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f08600ce220 tx=0x7f085002a230 comp rx=0 tx=0).stop 2024-04-25T10:05:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.988+0000 7f0867709640 1 -- 172.21.15.16:0/2827450751 shutdown_connections 2024-04-25T10:05:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.988+0000 7f0867709640 1 --2- 172.21.15.16:0/2827450751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08600d35b0 0x7f08600b5450 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.988+0000 7f0867709640 1 --2- 172.21.15.16:0/2827450751 >> v2:172.21.15.186:3300/0 conn(0x7f086015dce0 0x7f08600d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.988+0000 7f0867709640 1 -- 172.21.15.16:0/2827450751 >> 172.21.15.16:0/2827450751 conn(0x7f0860072900 msgr2=0x7f08600d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.988+0000 7f0867709640 1 -- 172.21.15.16:0/2827450751 shutdown_connections 2024-04-25T10:05:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.988+0000 7f0867709640 1 -- 172.21.15.16:0/2827450751 wait complete. 2024-04-25T10:05:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.990+0000 7f0867709640 1 Processor -- start 2024-04-25T10:05:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.990+0000 7f0867709640 1 -- start start 2024-04-25T10:05:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.990+0000 7f0867709640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08600d1f90 0x7f086015f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.990+0000 7f0867709640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08600d35b0 0x7f0860157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.990+0000 7f0867709640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f086015dce0 0x7f086015a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.990+0000 7f0867709640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08600d3090 con 0x7f08600d1f90 2024-04-25T10:05:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.990+0000 7f0867709640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08600d2db0 con 0x7f086015dce0 2024-04-25T10:05:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.990+0000 7f0867709640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08600b9820 con 0x7f08600d35b0 2024-04-25T10:05:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.990+0000 7f08654a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08600d1f90 0x7f086015f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.990+0000 7f08654a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08600d1f90 0x7f086015f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42814/0 (socket says 172.21.15.16:42814) 2024-04-25T10:05:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.990+0000 7f08654a5640 1 -- 172.21.15.16:0/881985313 learned_addr learned my addr 172.21.15.16:0/881985313 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.991+0000 7f0864ca4640 1 --2- 172.21.15.16:0/881985313 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08600d35b0 0x7f0860157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.991+0000 7f0865ca6640 1 --2- 172.21.15.16:0/881985313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f086015dce0 0x7f086015a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.991+0000 7f08654a5640 1 -- 172.21.15.16:0/881985313 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08600d35b0 msgr2=0x7f0860157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.991+0000 7f08654a5640 1 --2- 172.21.15.16:0/881985313 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08600d35b0 0x7f0860157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.991+0000 7f08654a5640 1 -- 172.21.15.16:0/881985313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f086015dce0 msgr2=0x7f086015a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.991+0000 7f08654a5640 1 --2- 172.21.15.16:0/881985313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f086015dce0 0x7f086015a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.991+0000 7f08654a5640 1 -- 172.21.15.16:0/881985313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0860157a30 con 0x7f08600d1f90 2024-04-25T10:05:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.991+0000 7f08654a5640 1 --2- 172.21.15.16:0/881985313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08600d1f90 0x7f086015f3e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f085c00c7f0 tx=0x7f085c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.991+0000 7f0855ffb640 1 -- 172.21.15.16:0/881985313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c015070 con 0x7f08600d1f90 2024-04-25T10:05:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.991+0000 7f0855ffb640 1 -- 172.21.15.16:0/881985313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085c007820 con 0x7f08600d1f90 2024-04-25T10:05:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.991+0000 7f0867709640 1 -- 172.21.15.16:0/881985313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f086015fca0 con 0x7f08600d1f90 2024-04-25T10:05:52.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.991+0000 7f0855ffb640 1 -- 172.21.15.16:0/881985313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c007bd0 con 0x7f08600d1f90 2024-04-25T10:05:52.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.991+0000 7f0867709640 1 -- 172.21.15.16:0/881985313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0860157f90 con 0x7f08600d1f90 2024-04-25T10:05:52.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.992+0000 7f0867709640 1 -- 172.21.15.16:0/881985313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08600749b0 con 0x7f08600d1f90 2024-04-25T10:05:52.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.993+0000 7f0855ffb640 1 -- 172.21.15.16:0/881985313 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f085c018070 con 0x7f08600d1f90 2024-04-25T10:05:52.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.993+0000 7f0855ffb640 1 --2- 172.21.15.16:0/881985313 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f084406b500 0x7f084406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.993+0000 7f0855ffb640 1 -- 172.21.15.16:0/881985313 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f085c08a280 con 0x7f08600d1f90 2024-04-25T10:05:52.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.993+0000 7f0864ca4640 1 --2- 172.21.15.16:0/881985313 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f084406b500 0x7f084406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.994+0000 7f0864ca4640 1 --2- 172.21.15.16:0/881985313 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f084406b500 0x7f084406d9c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0850034000 tx=0x7f085002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:51.997+0000 7f0855ffb640 1 -- 172.21.15.16:0/881985313 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0860157a30 con 0x7f08600d1f90 2024-04-25T10:05:52.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.215+0000 7f0867709640 1 -- 172.21.15.16:0/881985313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f086015d2c0 con 0x7f08600d1f90 2024-04-25T10:05:52.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.216+0000 7f0855ffb640 1 -- 172.21.15.16:0/881985313 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4939 (secure 0 0 0) 0x7f085c058a90 con 0x7f08600d1f90 2024-04-25T10:05:52.221 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:52.221 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1965168813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1965168813},{"type":"v1","addr":"172.21.15.16:6834","nonce":1965168813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/437791378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":437791378},{"type":"v1","addr":"172.21.15.16:6835","nonce":437791378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34145,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3044177642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3044177642},{"type":"v1","addr":"172.21.15.186:6835","nonce":3044177642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:10.814281+0000","modified":"2024-04-25T10:05:12.805138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34148":{"gid":34148,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/1763853189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1763853189},{"type":"v1","addr":"172.21.15.186:6837","nonce":1763853189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:05:52.222 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-04-25T10:05:52.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.218+0000 7f0867709640 1 -- 172.21.15.16:0/881985313 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f084406b500 msgr2=0x7f084406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.218+0000 7f0867709640 1 --2- 172.21.15.16:0/881985313 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f084406b500 0x7f084406d9c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0850034000 tx=0x7f085002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:52.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.218+0000 7f0867709640 1 -- 172.21.15.16:0/881985313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08600d1f90 msgr2=0x7f086015f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.218+0000 7f0867709640 1 --2- 172.21.15.16:0/881985313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08600d1f90 0x7f086015f3e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f085c00c7f0 tx=0x7f085c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.218+0000 7f0867709640 1 -- 172.21.15.16:0/881985313 shutdown_connections 2024-04-25T10:05:52.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.218+0000 7f0867709640 1 --2- 172.21.15.16:0/881985313 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f084406b500 0x7f084406d9c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.218+0000 7f0867709640 1 --2- 172.21.15.16:0/881985313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f086015dce0 0x7f086015a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.218+0000 7f0867709640 1 --2- 172.21.15.16:0/881985313 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08600d35b0 0x7f0860157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.218+0000 7f0867709640 1 --2- 172.21.15.16:0/881985313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08600d1f90 0x7f086015f3e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.218+0000 7f0867709640 1 -- 172.21.15.16:0/881985313 >> 172.21.15.16:0/881985313 conn(0x7f0860072900 msgr2=0x7f08600d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.218+0000 7f0867709640 1 -- 172.21.15.16:0/881985313 shutdown_connections 2024-04-25T10:05:52.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.218+0000 7f0867709640 1 -- 172.21.15.16:0/881985313 wait complete. 2024-04-25T10:05:52.234 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:52.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.330+0000 7f8ecfa9e640 1 Processor -- start 2024-04-25T10:05:52.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.330+0000 7f8ecfa9e640 1 -- start start 2024-04-25T10:05:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.331+0000 7f8ecfa9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ec815dce0 0x7f8ec80d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.331+0000 7f8ecfa9e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8ec80b5b60 0x7f8ec80b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.331+0000 7f8ecfa9e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec815e0c0 con 0x7f8ec80d35b0 2024-04-25T10:05:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.331+0000 7f8ecfa9e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec80d2510 con 0x7f8ec80b5b60 2024-04-25T10:05:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.331+0000 7f8ecfa9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec80d27f0 con 0x7f8ec815dce0 2024-04-25T10:05:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.332+0000 7f8ecd83a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ec815dce0 0x7f8ec80d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.332+0000 7f8ece03b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8ec80b5b60 0x7f8ec80b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.332+0000 7f8ecd039640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f8ec80d35b0 0x7f8ec80b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:43270/0 (socket says 172.21.15.16:43270) 2024-04-25T10:05:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.332+0000 7f8ecd83a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ec815dce0 0x7f8ec80d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:59208/0 (socket says 172.21.15.16:59208) 2024-04-25T10:05:52.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.332+0000 7f8ece03b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8ec80b5b60 0x7f8ec80b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:43098/0 (socket says 172.21.15.16:43098) 2024-04-25T10:05:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.332+0000 7f8ecd039640 1 -- 172.21.15.16:0/1225755010 learned_addr learned my addr 172.21.15.16:0/1225755010 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.332+0000 7f8ecc838640 1 -- 172.21.15.16:0/1225755010 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884913672 0 0) 0x7f8ec815e0c0 con 0x7f8ec80d35b0 2024-04-25T10:05:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.332+0000 7f8ecc838640 1 -- 172.21.15.16:0/1225755010 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8eb4003520 con 0x7f8ec80d35b0 2024-04-25T10:05:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.332+0000 7f8ecd83a640 1 -- 172.21.15.16:0/1225755010 >> v2:172.21.15.186:3300/0 conn(0x7f8ec80b5b60 msgr2=0x7f8ec80b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.332+0000 7f8ecd83a640 1 --2- 172.21.15.16:0/1225755010 >> v2:172.21.15.186:3300/0 conn(0x7f8ec80b5b60 0x7f8ec80b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.332+0000 7f8ecd83a640 1 -- 172.21.15.16:0/1225755010 >> v1:172.21.15.186:6789/0 conn(0x7f8ec80d35b0 legacy=0x7f8ec80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.332+0000 7f8ecd83a640 1 -- 172.21.15.16:0/1225755010 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec80d2ad0 con 0x7f8ec815dce0 2024-04-25T10:05:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.332+0000 7f8ece03b640 1 --2- 172.21.15.16:0/1225755010 >> v2:172.21.15.186:3300/0 conn(0x7f8ec80b5b60 0x7f8ec80b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.333+0000 7f8ecd83a640 1 --2- 172.21.15.16:0/1225755010 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ec815dce0 0x7f8ec80d1a50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ec00097a0 tx=0x7f8ec002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7714b59c052437f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.333+0000 7f8ecc838640 1 -- 172.21.15.16:0/1225755010 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0037070 con 0x7f8ec815dce0 2024-04-25T10:05:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.333+0000 7f8ecc838640 1 -- 172.21.15.16:0/1225755010 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec0032440 con 0x7f8ec815dce0 2024-04-25T10:05:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.333+0000 7f8ecc838640 1 -- 172.21.15.16:0/1225755010 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0032810 con 0x7f8ec815dce0 2024-04-25T10:05:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.333+0000 7f8ecfa9e640 1 -- 172.21.15.16:0/1225755010 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ec815dce0 msgr2=0x7f8ec80d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.333+0000 7f8ecfa9e640 1 --2- 172.21.15.16:0/1225755010 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ec815dce0 0x7f8ec80d1a50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ec00097a0 tx=0x7f8ec002a180 comp rx=0 tx=0).stop 2024-04-25T10:05:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.334+0000 7f8ecfa9e640 1 -- 172.21.15.16:0/1225755010 shutdown_connections 2024-04-25T10:05:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.334+0000 7f8ecfa9e640 1 --2- 172.21.15.16:0/1225755010 >> v2:172.21.15.186:3300/0 conn(0x7f8ec80b5b60 0x7f8ec80b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.334+0000 7f8ecfa9e640 1 --2- 172.21.15.16:0/1225755010 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ec815dce0 0x7f8ec80d1a50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.334+0000 7f8ecfa9e640 1 -- 172.21.15.16:0/1225755010 >> 172.21.15.16:0/1225755010 conn(0x7f8ec8072900 msgr2=0x7f8ec80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.334+0000 7f8ecfa9e640 1 -- 172.21.15.16:0/1225755010 shutdown_connections 2024-04-25T10:05:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.334+0000 7f8ecfa9e640 1 -- 172.21.15.16:0/1225755010 wait complete. 2024-04-25T10:05:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.335+0000 7f8ecfa9e640 1 Processor -- start 2024-04-25T10:05:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.335+0000 7f8ecfa9e640 1 -- start start 2024-04-25T10:05:52.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.335+0000 7f8ecfa9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec80b5b60 0x7f8ec814fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.335+0000 7f8ecfa9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ec80d35b0 0x7f8ec80ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.335+0000 7f8ecfa9e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ec815dce0 0x7f8ec81608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.335+0000 7f8ecfa9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec80b9820 con 0x7f8ec80b5b60 2024-04-25T10:05:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.335+0000 7f8ecfa9e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec80d3090 con 0x7f8ec815dce0 2024-04-25T10:05:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.335+0000 7f8ecfa9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec80b99a0 con 0x7f8ec80d35b0 2024-04-25T10:05:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.336+0000 7f8ecd83a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec80b5b60 0x7f8ec814fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.336+0000 7f8ecd83a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec80b5b60 0x7f8ec814fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42818/0 (socket says 172.21.15.16:42818) 2024-04-25T10:05:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.336+0000 7f8ecd83a640 1 -- 172.21.15.16:0/1041958131 learned_addr learned my addr 172.21.15.16:0/1041958131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.336+0000 7f8ece03b640 1 --2- 172.21.15.16:0/1041958131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ec815dce0 0x7f8ec81608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.336+0000 7f8ecd039640 1 --2- 172.21.15.16:0/1041958131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ec80d35b0 0x7f8ec80ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.336+0000 7f8ecd83a640 1 -- 172.21.15.16:0/1041958131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ec80d35b0 msgr2=0x7f8ec80ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.336+0000 7f8ecd83a640 1 --2- 172.21.15.16:0/1041958131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ec80d35b0 0x7f8ec80ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.336+0000 7f8ecd83a640 1 -- 172.21.15.16:0/1041958131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ec815dce0 msgr2=0x7f8ec81608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.336+0000 7f8ecd83a640 1 --2- 172.21.15.16:0/1041958131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ec815dce0 0x7f8ec81608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.336+0000 7f8ecd83a640 1 -- 172.21.15.16:0/1041958131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec81f4570 con 0x7f8ec80b5b60 2024-04-25T10:05:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.336+0000 7f8ecd83a640 1 --2- 172.21.15.16:0/1041958131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec80b5b60 0x7f8ec814fdc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0035040 tx=0x7f8ec00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.336+0000 7f8ebe7fc640 1 -- 172.21.15.16:0/1041958131 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0041030 con 0x7f8ec80b5b60 2024-04-25T10:05:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.336+0000 7f8ebe7fc640 1 -- 172.21.15.16:0/1041958131 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec003d420 con 0x7f8ec80b5b60 2024-04-25T10:05:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.336+0000 7f8ebe7fc640 1 -- 172.21.15.16:0/1041958131 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0037040 con 0x7f8ec80b5b60 2024-04-25T10:05:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.336+0000 7f8ecfa9e640 1 -- 172.21.15.16:0/1041958131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec80cbc90 con 0x7f8ec80b5b60 2024-04-25T10:05:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.336+0000 7f8ecfa9e640 1 -- 172.21.15.16:0/1041958131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec80cc250 con 0x7f8ec80b5b60 2024-04-25T10:05:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.338+0000 7f8ecfa9e640 1 -- 172.21.15.16:0/1041958131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec80749b0 con 0x7f8ec80b5b60 2024-04-25T10:05:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.338+0000 7f8ebe7fc640 1 -- 172.21.15.16:0/1041958131 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ec0044070 con 0x7f8ec80b5b60 2024-04-25T10:05:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.338+0000 7f8ebe7fc640 1 --2- 172.21.15.16:0/1041958131 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8eac06b4d0 0x7f8eac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.339+0000 7f8ebe7fc640 1 -- 172.21.15.16:0/1041958131 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8ec00aa6c0 con 0x7f8ec80b5b60 2024-04-25T10:05:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.339+0000 7f8ecd039640 1 --2- 172.21.15.16:0/1041958131 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8eac06b4d0 0x7f8eac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.339+0000 7f8ecd039640 1 --2- 172.21.15.16:0/1041958131 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8eac06b4d0 0x7f8eac06d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8ec815e540 tx=0x7f8eb8007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.343+0000 7f8ebe7fc640 1 -- 172.21.15.16:0/1041958131 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ec0079650 con 0x7f8ec80b5b60 2024-04-25T10:05:52.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.523+0000 7f8ecfa9e640 1 -- 172.21.15.16:0/1041958131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ec815d250 con 0x7f8ec80b5b60 2024-04-25T10:05:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.524+0000 7f8ebe7fc640 1 -- 172.21.15.16:0/1041958131 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16658 (secure 0 0 0) 0x7f8ec0078f50 con 0x7f8ec80b5b60 2024-04-25T10:05:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:52.529 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:51.582946+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.36","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:52.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.526+0000 7f8ecfa9e640 1 -- 172.21.15.16:0/1041958131 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8eac06b4d0 msgr2=0x7f8eac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.526+0000 7f8ecfa9e640 1 --2- 172.21.15.16:0/1041958131 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8eac06b4d0 0x7f8eac06d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8ec815e540 tx=0x7f8eb8007d30 comp rx=0 tx=0).stop 2024-04-25T10:05:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.526+0000 7f8ecfa9e640 1 -- 172.21.15.16:0/1041958131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec80b5b60 msgr2=0x7f8ec814fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.526+0000 7f8ecfa9e640 1 --2- 172.21.15.16:0/1041958131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec80b5b60 0x7f8ec814fdc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0035040 tx=0x7f8ec00046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.526+0000 7f8ecfa9e640 1 -- 172.21.15.16:0/1041958131 shutdown_connections 2024-04-25T10:05:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.526+0000 7f8ecfa9e640 1 --2- 172.21.15.16:0/1041958131 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8eac06b4d0 0x7f8eac06d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.526+0000 7f8ecfa9e640 1 --2- 172.21.15.16:0/1041958131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ec815dce0 0x7f8ec81608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.526+0000 7f8ecfa9e640 1 --2- 172.21.15.16:0/1041958131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ec80d35b0 0x7f8ec80ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.526+0000 7f8ecfa9e640 1 --2- 172.21.15.16:0/1041958131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec80b5b60 0x7f8ec814fdc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.526+0000 7f8ecfa9e640 1 -- 172.21.15.16:0/1041958131 >> 172.21.15.16:0/1041958131 conn(0x7f8ec8072900 msgr2=0x7f8ec80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.526+0000 7f8ecfa9e640 1 -- 172.21.15.16:0/1041958131 shutdown_connections 2024-04-25T10:05:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.526+0000 7f8ecfa9e640 1 -- 172.21.15.16:0/1041958131 wait complete. 2024-04-25T10:05:52.542 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:05:52.542 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:52.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.639+0000 7fd59f65a640 1 Processor -- start 2024-04-25T10:05:52.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.639+0000 7fd59f65a640 1 -- start start 2024-04-25T10:05:52.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.640+0000 7fd59f65a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd59815a870 0x7fd5980cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.640+0000 7fd59f65a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5980cdb10 con 0x7fd5980cfc40 2024-04-25T10:05:52.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.640+0000 7fd59f65a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5980cddf0 con 0x7fd59806e2e0 2024-04-25T10:05:52.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.640+0000 7fd59f65a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5980ce720 con 0x7fd59815a870 2024-04-25T10:05:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.640+0000 7fd59dbf7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd5980cfc40 0x7fd59815bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51502/0 (socket says 172.21.15.16:51502) 2024-04-25T10:05:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.640+0000 7fd59dbf7640 1 -- 172.21.15.16:0/553763123 learned_addr learned my addr 172.21.15.16:0/553763123 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.640+0000 7fd59d3f6640 1 --2- 172.21.15.16:0/553763123 >> v2:172.21.15.186:3300/0 conn(0x7fd59815a870 0x7fd5980cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd597fff640 1 -- 172.21.15.16:0/553763123 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884449650 0 0) 0x7fd5980cdb10 con 0x7fd5980cfc40 2024-04-25T10:05:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd597fff640 1 -- 172.21.15.16:0/553763123 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd580003520 con 0x7fd5980cfc40 2024-04-25T10:05:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd597fff640 1 -- 172.21.15.16:0/553763123 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293564679 0 0) 0x7fd5980cddf0 con 0x7fd59806e2e0 2024-04-25T10:05:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd597fff640 1 -- 172.21.15.16:0/553763123 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5980cdb10 con 0x7fd59806e2e0 2024-04-25T10:05:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd597fff640 1 -- 172.21.15.16:0/553763123 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 172251860 0 0) 0x7fd580003520 con 0x7fd5980cfc40 2024-04-25T10:05:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd597fff640 1 -- 172.21.15.16:0/553763123 >> v2:172.21.15.186:3300/0 conn(0x7fd59815a870 msgr2=0x7fd5980cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd597fff640 1 --2- 172.21.15.16:0/553763123 >> v2:172.21.15.186:3300/0 conn(0x7fd59815a870 0x7fd5980cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd597fff640 1 -- 172.21.15.16:0/553763123 >> v1:172.21.15.186:6789/0 conn(0x7fd59806e2e0 legacy=0x7fd5980cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd597fff640 1 -- 172.21.15.16:0/553763123 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5980ce0d0 con 0x7fd5980cfc40 2024-04-25T10:05:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd597fff640 1 -- 172.21.15.16:0/553763123 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fd5880019f0 con 0x7fd5980cfc40 2024-04-25T10:05:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd597fff640 1 -- 172.21.15.16:0/553763123 >> v1:172.21.15.16:6789/0 conn(0x7fd5980cfc40 legacy=0x7fd59815bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd597fff640 1 --2- 172.21.15.16:0/553763123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd580003b40 0x7fd580003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd597fff640 1 --2- 172.21.15.16:0/553763123 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd580004800 0x7fd580004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd597fff640 1 --2- 172.21.15.16:0/553763123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5800051c0 0x7fd58000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd597fff640 1 -- 172.21.15.16:0/553763123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980ce720 con 0x7fd580003b40 2024-04-25T10:05:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd597fff640 1 -- 172.21.15.16:0/553763123 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58000c040 con 0x7fd5800051c0 2024-04-25T10:05:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd597fff640 1 -- 172.21.15.16:0/553763123 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58000c240 con 0x7fd580004800 2024-04-25T10:05:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd59d3f6640 1 --2- 172.21.15.16:0/553763123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd580003b40 0x7fd580003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd59dbf7640 1 --2- 172.21.15.16:0/553763123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5800051c0 0x7fd58000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd59cbf5640 1 --2- 172.21.15.16:0/553763123 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd580004800 0x7fd580004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd59d3f6640 1 -- 172.21.15.16:0/553763123 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd580004800 msgr2=0x7fd580004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd59d3f6640 1 --2- 172.21.15.16:0/553763123 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd580004800 0x7fd580004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd59d3f6640 1 -- 172.21.15.16:0/553763123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5800051c0 msgr2=0x7fd58000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd59d3f6640 1 --2- 172.21.15.16:0/553763123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5800051c0 0x7fd58000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd59d3f6640 1 -- 172.21.15.16:0/553763123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd580003520 con 0x7fd580003b40 2024-04-25T10:05:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd59dbf7640 1 --2- 172.21.15.16:0/553763123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5800051c0 0x7fd58000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.641+0000 7fd59dbf7640 1 -- 172.21.15.16:0/553763123 reap_dead start 2024-04-25T10:05:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.642+0000 7fd59d3f6640 1 --2- 172.21.15.16:0/553763123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd580003b40 0x7fd580003f40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd590009ac0 tx=0x7fd580005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1040398a10029236 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.642+0000 7fd597fff640 1 -- 172.21.15.16:0/553763123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590036070 con 0x7fd580003b40 2024-04-25T10:05:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.642+0000 7fd597fff640 1 -- 172.21.15.16:0/553763123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd590031400 con 0x7fd580003b40 2024-04-25T10:05:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.642+0000 7fd597fff640 1 -- 172.21.15.16:0/553763123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590009690 con 0x7fd580003b40 2024-04-25T10:05:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.643+0000 7fd59f65a640 1 -- 172.21.15.16:0/553763123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd580003b40 msgr2=0x7fd580003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.643+0000 7fd59f65a640 1 --2- 172.21.15.16:0/553763123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd580003b40 0x7fd580003f40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd590009ac0 tx=0x7fd580005830 comp rx=0 tx=0).stop 2024-04-25T10:05:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.643+0000 7fd59f65a640 1 -- 172.21.15.16:0/553763123 shutdown_connections 2024-04-25T10:05:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.643+0000 7fd59f65a640 1 --2- 172.21.15.16:0/553763123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd580003b40 0x7fd580003f40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.643+0000 7fd59f65a640 1 -- 172.21.15.16:0/553763123 >> 172.21.15.16:0/553763123 conn(0x7fd59806aba0 msgr2=0x7fd59806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.643+0000 7fd59f65a640 1 -- 172.21.15.16:0/553763123 shutdown_connections 2024-04-25T10:05:52.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.643+0000 7fd59f65a640 1 -- 172.21.15.16:0/553763123 wait complete. 2024-04-25T10:05:52.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.644+0000 7fd59f65a640 1 Processor -- start 2024-04-25T10:05:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.644+0000 7fd59f65a640 1 -- start start 2024-04-25T10:05:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.645+0000 7fd59f65a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd580003b40 0x7fd598143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.645+0000 7fd59f65a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd598155360 0x7fd598150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.645+0000 7fd59f65a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5981520d0 0x7fd5980b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.645+0000 7fd59f65a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980cd700 con 0x7fd598155360 2024-04-25T10:05:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.645+0000 7fd59f65a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980cd580 con 0x7fd5981520d0 2024-04-25T10:05:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.645+0000 7fd59f65a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980cd880 con 0x7fd580003b40 2024-04-25T10:05:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.645+0000 7fd59cbf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd598155360 0x7fd598150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.645+0000 7fd59cbf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd598155360 0x7fd598150ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42842/0 (socket says 172.21.15.16:42842) 2024-04-25T10:05:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.645+0000 7fd59cbf5640 1 -- 172.21.15.16:0/3093473573 learned_addr learned my addr 172.21.15.16:0/3093473573 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.645+0000 7fd59d3f6640 1 --2- 172.21.15.16:0/3093473573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd580003b40 0x7fd598143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.645+0000 7fd59cbf5640 1 -- 172.21.15.16:0/3093473573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd580003b40 msgr2=0x7fd598143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.645+0000 7fd59cbf5640 1 --2- 172.21.15.16:0/3093473573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd580003b40 0x7fd598143120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.645+0000 7fd59cbf5640 1 -- 172.21.15.16:0/3093473573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5981520d0 msgr2=0x7fd5980b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.645+0000 7fd59dbf7640 1 --2- 172.21.15.16:0/3093473573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5981520d0 0x7fd5980b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.645+0000 7fd59cbf5640 1 --2- 172.21.15.16:0/3093473573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5981520d0 0x7fd5980b2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.645+0000 7fd59cbf5640 1 -- 172.21.15.16:0/3093473573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd598151380 con 0x7fd598155360 2024-04-25T10:05:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.645+0000 7fd59cbf5640 1 --2- 172.21.15.16:0/3093473573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd598155360 0x7fd598150ae0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00e7b0 tx=0x7fd58c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.646+0000 7fd595ffb640 1 -- 172.21.15.16:0/3093473573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c00ee60 con 0x7fd598155360 2024-04-25T10:05:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.646+0000 7fd595ffb640 1 -- 172.21.15.16:0/3093473573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58c018070 con 0x7fd598155360 2024-04-25T10:05:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.646+0000 7fd59f65a640 1 -- 172.21.15.16:0/3093473573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5981515f0 con 0x7fd598155360 2024-04-25T10:05:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.646+0000 7fd595ffb640 1 -- 172.21.15.16:0/3093473573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c00c660 con 0x7fd598155360 2024-04-25T10:05:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.646+0000 7fd59f65a640 1 -- 172.21.15.16:0/3093473573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd598151bb0 con 0x7fd598155360 2024-04-25T10:05:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.647+0000 7fd59f65a640 1 -- 172.21.15.16:0/3093473573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5980749b0 con 0x7fd598155360 2024-04-25T10:05:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.647+0000 7fd595ffb640 1 -- 172.21.15.16:0/3093473573 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd58c00c800 con 0x7fd598155360 2024-04-25T10:05:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.648+0000 7fd595ffb640 1 --2- 172.21.15.16:0/3093473573 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd58406b4d0 0x7fd58406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.648+0000 7fd595ffb640 1 -- 172.21.15.16:0/3093473573 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd58c089ff0 con 0x7fd598155360 2024-04-25T10:05:52.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.648+0000 7fd59d3f6640 1 --2- 172.21.15.16:0/3093473573 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd58406b4d0 0x7fd58406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.648+0000 7fd59d3f6640 1 --2- 172.21.15.16:0/3093473573 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd58406b4d0 0x7fd58406d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd590033000 tx=0x7fd590008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.652+0000 7fd595ffb640 1 -- 172.21.15.16:0/3093473573 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd58c01a310 con 0x7fd598155360 2024-04-25T10:05:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.869+0000 7fd59f65a640 1 -- 172.21.15.16:0/3093473573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5980b4410 con 0x7fd598155360 2024-04-25T10:05:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.870+0000 7fd595ffb640 1 -- 172.21.15.16:0/3093473573 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4939 (secure 0 0 0) 0x7fd58c058e50 con 0x7fd598155360 2024-04-25T10:05:52.875 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:52.876 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1965168813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1965168813},{"type":"v1","addr":"172.21.15.16:6834","nonce":1965168813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/437791378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":437791378},{"type":"v1","addr":"172.21.15.16:6835","nonce":437791378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34145,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3044177642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3044177642},{"type":"v1","addr":"172.21.15.186:6835","nonce":3044177642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:10.814281+0000","modified":"2024-04-25T10:05:12.805138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34148":{"gid":34148,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/1763853189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1763853189},{"type":"v1","addr":"172.21.15.186:6837","nonce":1763853189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:05:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-04-25T10:05:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.872+0000 7fd59f65a640 1 -- 172.21.15.16:0/3093473573 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd58406b4d0 msgr2=0x7fd58406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.872+0000 7fd59f65a640 1 --2- 172.21.15.16:0/3093473573 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd58406b4d0 0x7fd58406d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd590033000 tx=0x7fd590008030 comp rx=0 tx=0).stop 2024-04-25T10:05:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.872+0000 7fd59f65a640 1 -- 172.21.15.16:0/3093473573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd598155360 msgr2=0x7fd598150ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.872+0000 7fd59f65a640 1 --2- 172.21.15.16:0/3093473573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd598155360 0x7fd598150ae0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00e7b0 tx=0x7fd58c00ec70 comp rx=0 tx=0).stop 2024-04-25T10:05:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.872+0000 7fd59f65a640 1 -- 172.21.15.16:0/3093473573 shutdown_connections 2024-04-25T10:05:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.872+0000 7fd59f65a640 1 --2- 172.21.15.16:0/3093473573 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd58406b4d0 0x7fd58406d990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.872+0000 7fd59f65a640 1 --2- 172.21.15.16:0/3093473573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5981520d0 0x7fd5980b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.872+0000 7fd59f65a640 1 --2- 172.21.15.16:0/3093473573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd598155360 0x7fd598150ae0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.872+0000 7fd59f65a640 1 --2- 172.21.15.16:0/3093473573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd580003b40 0x7fd598143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.872+0000 7fd59f65a640 1 -- 172.21.15.16:0/3093473573 >> 172.21.15.16:0/3093473573 conn(0x7fd59806aba0 msgr2=0x7fd5980b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.872+0000 7fd59f65a640 1 -- 172.21.15.16:0/3093473573 shutdown_connections 2024-04-25T10:05:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.872+0000 7fd59f65a640 1 -- 172.21.15.16:0/3093473573 wait complete. 2024-04-25T10:05:52.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.983+0000 7f69b91a5640 1 Processor -- start 2024-04-25T10:05:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.983+0000 7f69b91a5640 1 -- start start 2024-04-25T10:05:52.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.984+0000 7f69b91a5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f69b4144450 0x7f69b40c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.984+0000 7f69b91a5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f69b40ccdc0 0x7f69b40722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.984+0000 7f69b91a5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b40c75c0 con 0x7f69b40727e0 2024-04-25T10:05:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.984+0000 7f69b91a5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b40cdb60 con 0x7f69b40ccdc0 2024-04-25T10:05:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.984+0000 7f69b91a5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b40cd2c0 con 0x7f69b4144450 2024-04-25T10:05:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.984+0000 7f69b259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f69b40ccdc0 0x7f69b40722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.984+0000 7f69b359e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f69b40727e0 0x7f69b415eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45422/0 (socket says 172.21.15.16:45422) 2024-04-25T10:05:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.984+0000 7f69b359e640 1 -- 172.21.15.16:0/2308234703 learned_addr learned my addr 172.21.15.16:0/2308234703 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.984+0000 7f69b2d9d640 1 --2- 172.21.15.16:0/2308234703 >> v2:172.21.15.186:3300/0 conn(0x7f69b4144450 0x7f69b40c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.985+0000 7f69b259c640 1 -- 172.21.15.16:0/2308234703 >> v2:172.21.15.186:3300/0 conn(0x7f69b4144450 msgr2=0x7f69b40c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.985+0000 7f69b259c640 1 --2- 172.21.15.16:0/2308234703 >> v2:172.21.15.186:3300/0 conn(0x7f69b4144450 0x7f69b40c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.985+0000 7f69b259c640 1 -- 172.21.15.16:0/2308234703 >> v1:172.21.15.16:6789/0 conn(0x7f69b40727e0 legacy=0x7f69b415eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.985+0000 7f69b259c640 1 -- 172.21.15.16:0/2308234703 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b40cd5a0 con 0x7f69b40ccdc0 2024-04-25T10:05:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.985+0000 7f69b2d9d640 1 --2- 172.21.15.16:0/2308234703 >> v2:172.21.15.186:3300/0 conn(0x7f69b4144450 0x7f69b40c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.985+0000 7f69b259c640 1 --2- 172.21.15.16:0/2308234703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b40ccdc0 0x7f69b40722a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f69b40ce2c0 tx=0x7f69a802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e8c77b51614b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.985+0000 7f69b1d9b640 1 -- 172.21.15.16:0/2308234703 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a8037070 con 0x7f69b40ccdc0 2024-04-25T10:05:52.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.985+0000 7f69b1d9b640 1 -- 172.21.15.16:0/2308234703 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a8032440 con 0x7f69b40ccdc0 2024-04-25T10:05:52.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.985+0000 7f69b1d9b640 1 -- 172.21.15.16:0/2308234703 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a802ae60 con 0x7f69b40ccdc0 2024-04-25T10:05:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.986+0000 7f69b91a5640 1 -- 172.21.15.16:0/2308234703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b40ccdc0 msgr2=0x7f69b40722a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.986+0000 7f69b91a5640 1 --2- 172.21.15.16:0/2308234703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b40ccdc0 0x7f69b40722a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f69b40ce2c0 tx=0x7f69a802a110 comp rx=0 tx=0).stop 2024-04-25T10:05:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.986+0000 7f69b91a5640 1 -- 172.21.15.16:0/2308234703 shutdown_connections 2024-04-25T10:05:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.986+0000 7f69b91a5640 1 --2- 172.21.15.16:0/2308234703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b40ccdc0 0x7f69b40722a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.986+0000 7f69b91a5640 1 --2- 172.21.15.16:0/2308234703 >> v2:172.21.15.186:3300/0 conn(0x7f69b4144450 0x7f69b40c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.986+0000 7f69b91a5640 1 -- 172.21.15.16:0/2308234703 >> 172.21.15.16:0/2308234703 conn(0x7f69b40b2370 msgr2=0x7f69b40b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.986+0000 7f69b91a5640 1 -- 172.21.15.16:0/2308234703 shutdown_connections 2024-04-25T10:05:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.986+0000 7f69b91a5640 1 -- 172.21.15.16:0/2308234703 wait complete. 2024-04-25T10:05:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.987+0000 7f69b91a5640 1 Processor -- start 2024-04-25T10:05:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.988+0000 7f69b91a5640 1 -- start start 2024-04-25T10:05:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.988+0000 7f69b91a5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69b40727e0 0x7f69b40ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.988+0000 7f69b91a5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69b40ccdc0 0x7f69b415cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.988+0000 7f69b91a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b4144450 0x7f69b4148cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.988+0000 7f69b91a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b415f1b0 con 0x7f69b4144450 2024-04-25T10:05:52.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.988+0000 7f69b91a5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b415e580 con 0x7f69b40727e0 2024-04-25T10:05:52.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.988+0000 7f69b91a5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b415f330 con 0x7f69b40ccdc0 2024-04-25T10:05:52.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.988+0000 7f69b359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b4144450 0x7f69b4148cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.988+0000 7f69b359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b4144450 0x7f69b4148cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36044/0 (socket says 172.21.15.16:36044) 2024-04-25T10:05:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.988+0000 7f69b2d9d640 1 --2- 172.21.15.16:0/4189442503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69b40727e0 0x7f69b40ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.988+0000 7f69b359e640 1 -- 172.21.15.16:0/4189442503 learned_addr learned my addr 172.21.15.16:0/4189442503 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.988+0000 7f69b259c640 1 --2- 172.21.15.16:0/4189442503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69b40ccdc0 0x7f69b415cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.989+0000 7f69b359e640 1 -- 172.21.15.16:0/4189442503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69b40ccdc0 msgr2=0x7f69b415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.989+0000 7f69b359e640 1 --2- 172.21.15.16:0/4189442503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69b40ccdc0 0x7f69b415cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.989+0000 7f69b359e640 1 -- 172.21.15.16:0/4189442503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69b40727e0 msgr2=0x7f69b40ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.989+0000 7f69b359e640 1 --2- 172.21.15.16:0/4189442503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69b40727e0 0x7f69b40ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.989+0000 7f69b359e640 1 -- 172.21.15.16:0/4189442503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b415c850 con 0x7f69b4144450 2024-04-25T10:05:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.989+0000 7f69b359e640 1 --2- 172.21.15.16:0/4189442503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b4144450 0x7f69b4148cc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f69a400cbc0 tx=0x7f69a400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.989+0000 7f699b7fe640 1 -- 172.21.15.16:0/4189442503 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a4017070 con 0x7f69b4144450 2024-04-25T10:05:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.989+0000 7f699b7fe640 1 -- 172.21.15.16:0/4189442503 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a400ad40 con 0x7f69b4144450 2024-04-25T10:05:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.989+0000 7f69b91a5640 1 -- 172.21.15.16:0/4189442503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b40b36b0 con 0x7f69b4144450 2024-04-25T10:05:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.989+0000 7f699b7fe640 1 -- 172.21.15.16:0/4189442503 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a4012600 con 0x7f69b4144450 2024-04-25T10:05:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.989+0000 7f69b91a5640 1 -- 172.21.15.16:0/4189442503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b415d530 con 0x7f69b4144450 2024-04-25T10:05:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.990+0000 7f69b91a5640 1 -- 172.21.15.16:0/4189442503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b40b3bd0 con 0x7f69b4144450 2024-04-25T10:05:53.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.991+0000 7f699b7fe640 1 -- 172.21.15.16:0/4189442503 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69a401a070 con 0x7f69b4144450 2024-04-25T10:05:53.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.991+0000 7f699b7fe640 1 --2- 172.21.15.16:0/4189442503 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f69a006b4d0 0x7f69a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.991+0000 7f699b7fe640 1 -- 172.21.15.16:0/4189442503 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f69a4089ed0 con 0x7f69b4144450 2024-04-25T10:05:53.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.991+0000 7f69b2d9d640 1 --2- 172.21.15.16:0/4189442503 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f69a006b4d0 0x7f69a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.992+0000 7f69b2d9d640 1 --2- 172.21.15.16:0/4189442503 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f69a006b4d0 0x7f69a006d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f69ac006e80 tx=0x7f69ac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:52.995+0000 7f699b7fe640 1 -- 172.21.15.16:0/4189442503 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69b415c850 con 0x7f69b4144450 2024-04-25T10:05:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.211+0000 7f69b91a5640 1 -- 172.21.15.16:0/4189442503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69b40749b0 con 0x7f69b4144450 2024-04-25T10:05:53.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.212+0000 7f699b7fe640 1 -- 172.21.15.16:0/4189442503 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4939 (secure 0 0 0) 0x7f69a40586e0 con 0x7f69b4144450 2024-04-25T10:05:53.217 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:53.217 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1965168813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1965168813},{"type":"v1","addr":"172.21.15.16:6834","nonce":1965168813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/437791378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":437791378},{"type":"v1","addr":"172.21.15.16:6835","nonce":437791378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34145,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3044177642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3044177642},{"type":"v1","addr":"172.21.15.186:6835","nonce":3044177642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:10.814281+0000","modified":"2024-04-25T10:05:12.805138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34148":{"gid":34148,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/1763853189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1763853189},{"type":"v1","addr":"172.21.15.186:6837","nonce":1763853189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:05:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-04-25T10:05:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.214+0000 7f69b91a5640 1 -- 172.21.15.16:0/4189442503 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f69a006b4d0 msgr2=0x7f69a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.214+0000 7f69b91a5640 1 --2- 172.21.15.16:0/4189442503 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f69a006b4d0 0x7f69a006d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f69ac006e80 tx=0x7f69ac008040 comp rx=0 tx=0).stop 2024-04-25T10:05:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.214+0000 7f69b91a5640 1 -- 172.21.15.16:0/4189442503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b4144450 msgr2=0x7f69b4148cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.214+0000 7f69b91a5640 1 --2- 172.21.15.16:0/4189442503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b4144450 0x7f69b4148cc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f69a400cbc0 tx=0x7f69a400a430 comp rx=0 tx=0).stop 2024-04-25T10:05:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.214+0000 7f69b91a5640 1 -- 172.21.15.16:0/4189442503 shutdown_connections 2024-04-25T10:05:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.214+0000 7f69b91a5640 1 --2- 172.21.15.16:0/4189442503 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f69a006b4d0 0x7f69a006d990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.214+0000 7f69b91a5640 1 --2- 172.21.15.16:0/4189442503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b4144450 0x7f69b4148cc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.214+0000 7f69b91a5640 1 --2- 172.21.15.16:0/4189442503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69b40ccdc0 0x7f69b415cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.214+0000 7f69b91a5640 1 --2- 172.21.15.16:0/4189442503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69b40727e0 0x7f69b40ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.214+0000 7f69b91a5640 1 -- 172.21.15.16:0/4189442503 >> 172.21.15.16:0/4189442503 conn(0x7f69b40b2370 msgr2=0x7f69b4072d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.214+0000 7f69b91a5640 1 -- 172.21.15.16:0/4189442503 shutdown_connections 2024-04-25T10:05:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.214+0000 7f69b91a5640 1 -- 172.21.15.16:0/4189442503 wait complete. 2024-04-25T10:05:53.230 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:53.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.325+0000 7f3699fb7640 1 Processor -- start 2024-04-25T10:05:53.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.326+0000 7f3699fb7640 1 -- start start 2024-04-25T10:05:53.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.326+0000 7f3699fb7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f369415dc30 0x7f369406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.326+0000 7f3699fb7640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f369406cc00 0x7f36940b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.326+0000 7f3699fb7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36940b1960 0x7f3694099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.326+0000 7f3699fb7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f369406c1f0 con 0x7f36940b1960 2024-04-25T10:05:53.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.326+0000 7f3699fb7640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36940cfb40 con 0x7f369406cc00 2024-04-25T10:05:53.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.326+0000 7f3699fb7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36940b2060 con 0x7f369415dc30 2024-04-25T10:05:53.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.326+0000 7f3693fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36940b1960 0x7f3694099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.327+0000 7f3693fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36940b1960 0x7f3694099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36054/0 (socket says 172.21.15.16:36054) 2024-04-25T10:05:53.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.327+0000 7f3693fff640 1 -- 172.21.15.16:0/3756381646 learned_addr learned my addr 172.21.15.16:0/3756381646 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:53.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.327+0000 7f3692ffd640 1 --2- 172.21.15.16:0/3756381646 >> v2:172.21.15.186:3300/0 conn(0x7f369406cc00 0x7f36940b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.327+0000 7f36937fe640 1 --2- 172.21.15.16:0/3756381646 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f369415dc30 0x7f369406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.327+0000 7f3693fff640 1 -- 172.21.15.16:0/3756381646 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f369415dc30 msgr2=0x7f369406c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:53.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.327+0000 7f3693fff640 1 --2- 172.21.15.16:0/3756381646 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f369415dc30 0x7f369406c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.327+0000 7f3693fff640 1 -- 172.21.15.16:0/3756381646 >> v2:172.21.15.186:3300/0 conn(0x7f369406cc00 msgr2=0x7f36940b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:53.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.327+0000 7f3693fff640 1 --2- 172.21.15.16:0/3756381646 >> v2:172.21.15.186:3300/0 conn(0x7f369406cc00 0x7f36940b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.327+0000 7f3693fff640 1 -- 172.21.15.16:0/3756381646 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36940b2340 con 0x7f36940b1960 2024-04-25T10:05:53.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.327+0000 7f36937fe640 1 --2- 172.21.15.16:0/3756381646 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f369415dc30 0x7f369406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:53.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.328+0000 7f3693fff640 1 --2- 172.21.15.16:0/3756381646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36940b1960 0x7f3694099a40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3694071de0 tx=0x7f368c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c04827ce83c920 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.328+0000 7f36927fc640 1 -- 172.21.15.16:0/3756381646 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c036070 con 0x7f36940b1960 2024-04-25T10:05:53.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.328+0000 7f36927fc640 1 -- 172.21.15.16:0/3756381646 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c031600 con 0x7f36940b1960 2024-04-25T10:05:53.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.328+0000 7f36927fc640 1 -- 172.21.15.16:0/3756381646 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c0319d0 con 0x7f36940b1960 2024-04-25T10:05:53.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.328+0000 7f3699fb7640 1 -- 172.21.15.16:0/3756381646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36940b1960 msgr2=0x7f3694099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.328+0000 7f3699fb7640 1 --2- 172.21.15.16:0/3756381646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36940b1960 0x7f3694099a40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3694071de0 tx=0x7f368c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.328+0000 7f3699fb7640 1 -- 172.21.15.16:0/3756381646 shutdown_connections 2024-04-25T10:05:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.328+0000 7f3699fb7640 1 --2- 172.21.15.16:0/3756381646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36940b1960 0x7f3694099a40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.328+0000 7f3699fb7640 1 --2- 172.21.15.16:0/3756381646 >> v2:172.21.15.186:3300/0 conn(0x7f369406cc00 0x7f36940b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.328+0000 7f3699fb7640 1 --2- 172.21.15.16:0/3756381646 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f369415dc30 0x7f369406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.328+0000 7f3699fb7640 1 -- 172.21.15.16:0/3756381646 >> 172.21.15.16:0/3756381646 conn(0x7f36940d1900 msgr2=0x7f36940d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.328+0000 7f3699fb7640 1 -- 172.21.15.16:0/3756381646 shutdown_connections 2024-04-25T10:05:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.328+0000 7f3699fb7640 1 -- 172.21.15.16:0/3756381646 wait complete. 2024-04-25T10:05:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.329+0000 7f3699fb7640 1 Processor -- start 2024-04-25T10:05:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.330+0000 7f3699fb7640 1 -- start start 2024-04-25T10:05:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.330+0000 7f3699fb7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f369406cc00 0x7f369415f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.330+0000 7f3699fb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36940b1960 0x7f36940c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.330+0000 7f3699fb7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f369415dc30 0x7f36941f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.330+0000 7f3699fb7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36940b1d80 con 0x7f36940b1960 2024-04-25T10:05:53.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.330+0000 7f3699fb7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36940cfde0 con 0x7f369406cc00 2024-04-25T10:05:53.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.330+0000 7f3699fb7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369415e010 con 0x7f369415dc30 2024-04-25T10:05:53.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.330+0000 7f3692ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36940b1960 0x7f36940c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.331+0000 7f3692ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36940b1960 0x7f36940c66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36060/0 (socket says 172.21.15.16:36060) 2024-04-25T10:05:53.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.331+0000 7f3692ffd640 1 -- 172.21.15.16:0/1865082397 learned_addr learned my addr 172.21.15.16:0/1865082397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:53.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.331+0000 7f3693fff640 1 --2- 172.21.15.16:0/1865082397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f369415dc30 0x7f36941f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.331+0000 7f36937fe640 1 --2- 172.21.15.16:0/1865082397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f369406cc00 0x7f369415f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.331+0000 7f3692ffd640 1 -- 172.21.15.16:0/1865082397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f369415dc30 msgr2=0x7f36941f70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.331+0000 7f3692ffd640 1 --2- 172.21.15.16:0/1865082397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f369415dc30 0x7f36941f70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.331+0000 7f3692ffd640 1 -- 172.21.15.16:0/1865082397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f369406cc00 msgr2=0x7f369415f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.331+0000 7f3692ffd640 1 --2- 172.21.15.16:0/1865082397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f369406cc00 0x7f369415f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.331+0000 7f3692ffd640 1 -- 172.21.15.16:0/1865082397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36940cb850 con 0x7f36940b1960 2024-04-25T10:05:53.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.331+0000 7f3692ffd640 1 --2- 172.21.15.16:0/1865082397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36940b1960 0x7f36940c66e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f368800d830 tx=0x7f368800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.331+0000 7f367bfff640 1 -- 172.21.15.16:0/1865082397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3688018030 con 0x7f36940b1960 2024-04-25T10:05:53.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.331+0000 7f367bfff640 1 -- 172.21.15.16:0/1865082397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368800be10 con 0x7f36940b1960 2024-04-25T10:05:53.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.331+0000 7f3699fb7640 1 -- 172.21.15.16:0/1865082397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36940cb290 con 0x7f36940b1960 2024-04-25T10:05:53.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.331+0000 7f367bfff640 1 -- 172.21.15.16:0/1865082397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36880098b0 con 0x7f36940b1960 2024-04-25T10:05:53.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.331+0000 7f3699fb7640 1 -- 172.21.15.16:0/1865082397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3694156b80 con 0x7f36940b1960 2024-04-25T10:05:53.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.332+0000 7f3699fb7640 1 -- 172.21.15.16:0/1865082397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3694067850 con 0x7f36940b1960 2024-04-25T10:05:53.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.333+0000 7f367bfff640 1 -- 172.21.15.16:0/1865082397 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f368801b070 con 0x7f36940b1960 2024-04-25T10:05:53.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.333+0000 7f367bfff640 1 --2- 172.21.15.16:0/1865082397 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f368006b4d0 0x7f368006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.333+0000 7f367bfff640 1 -- 172.21.15.16:0/1865082397 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f368808a3b0 con 0x7f36940b1960 2024-04-25T10:05:53.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.333+0000 7f36937fe640 1 --2- 172.21.15.16:0/1865082397 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f368006b4d0 0x7f368006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.334+0000 7f36937fe640 1 --2- 172.21.15.16:0/1865082397 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f368006b4d0 0x7f368006d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f368400a990 tx=0x7f3684008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.338+0000 7f367bfff640 1 -- 172.21.15.16:0/1865082397 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3688059210 con 0x7f36940b1960 2024-04-25T10:05:53.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.520+0000 7f3699fb7640 1 -- 172.21.15.16:0/1865082397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36940749b0 con 0x7f36940b1960 2024-04-25T10:05:53.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.521+0000 7f367bfff640 1 -- 172.21.15.16:0/1865082397 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16658 (secure 0 0 0) 0x7f3688058b10 con 0x7f36940b1960 2024-04-25T10:05:53.527 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:05:53.527 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:05:51.582946+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:08.724537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:09.793105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.36","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:53.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.524+0000 7f3699fb7640 1 -- 172.21.15.16:0/1865082397 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f368006b4d0 msgr2=0x7f368006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.524+0000 7f3699fb7640 1 --2- 172.21.15.16:0/1865082397 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f368006b4d0 0x7f368006d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f368400a990 tx=0x7f3684008040 comp rx=0 tx=0).stop 2024-04-25T10:05:53.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.524+0000 7f3699fb7640 1 -- 172.21.15.16:0/1865082397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36940b1960 msgr2=0x7f36940c66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.524+0000 7f3699fb7640 1 --2- 172.21.15.16:0/1865082397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36940b1960 0x7f36940c66e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f368800d830 tx=0x7f368800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.524+0000 7f3699fb7640 1 -- 172.21.15.16:0/1865082397 shutdown_connections 2024-04-25T10:05:53.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.524+0000 7f3699fb7640 1 --2- 172.21.15.16:0/1865082397 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f368006b4d0 0x7f368006d990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.524+0000 7f3699fb7640 1 --2- 172.21.15.16:0/1865082397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f369415dc30 0x7f36941f70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.524+0000 7f3699fb7640 1 --2- 172.21.15.16:0/1865082397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36940b1960 0x7f36940c66e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.524+0000 7f3699fb7640 1 --2- 172.21.15.16:0/1865082397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f369406cc00 0x7f369415f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.524+0000 7f3699fb7640 1 -- 172.21.15.16:0/1865082397 >> 172.21.15.16:0/1865082397 conn(0x7f36940d1900 msgr2=0x7f36940d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:53.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.524+0000 7f3699fb7640 1 -- 172.21.15.16:0/1865082397 shutdown_connections 2024-04-25T10:05:53.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.524+0000 7f3699fb7640 1 -- 172.21.15.16:0/1865082397 wait complete. 2024-04-25T10:05:53.540 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:05:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.637+0000 7f808ac38640 1 Processor -- start 2024-04-25T10:05:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.637+0000 7f808ac38640 1 -- start start 2024-04-25T10:05:53.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.638+0000 7f808ac38640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80840b9720 0x7f80840ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.638+0000 7f808ac38640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80840cf0f0 con 0x7f80840cece0 2024-04-25T10:05:53.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.638+0000 7f808ac38640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8084161f80 con 0x7f8084161090 2024-04-25T10:05:53.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.638+0000 7f808ac38640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80840d21a0 con 0x7f80840b9720 2024-04-25T10:05:53.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.638+0000 7f8083fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80840b9720 0x7f80840ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.638+0000 7f80891d5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f80840cece0 0x7f80840d2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45430/0 (socket says 172.21.15.16:45430) 2024-04-25T10:05:53.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.638+0000 7f8083fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80840b9720 0x7f80840ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36070/0 (socket says 172.21.15.16:36070) 2024-04-25T10:05:53.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.638+0000 7f80891d5640 1 -- 172.21.15.16:0/1920849235 learned_addr learned my addr 172.21.15.16:0/1920849235 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:53.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.638+0000 7f8083fff640 1 -- 172.21.15.16:0/1920849235 >> v1:172.21.15.186:6789/0 conn(0x7f8084161090 legacy=0x7f8084161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:53.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.638+0000 7f8083fff640 1 -- 172.21.15.16:0/1920849235 >> v1:172.21.15.16:6789/0 conn(0x7f80840cece0 legacy=0x7f80840d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.638+0000 7f8083fff640 1 -- 172.21.15.16:0/1920849235 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8084162260 con 0x7f80840b9720 2024-04-25T10:05:53.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.638+0000 7f80837fe640 1 -- 172.21.15.16:0/1920849235 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2194392854 0 0) 0x7f807c001490 con 0x7f80840cece0 2024-04-25T10:05:53.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.639+0000 7f8083fff640 1 --2- 172.21.15.16:0/1920849235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840b9720 0x7f80840ce7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f80840d2320 tx=0x7f807002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e786713b38d1abad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.639+0000 7f80837fe640 1 -- 172.21.15.16:0/1920849235 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070037070 con 0x7f80840b9720 2024-04-25T10:05:53.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.639+0000 7f80837fe640 1 -- 172.21.15.16:0/1920849235 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807002ae40 con 0x7f80840b9720 2024-04-25T10:05:53.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.639+0000 7f80837fe640 1 -- 172.21.15.16:0/1920849235 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070032660 con 0x7f80840b9720 2024-04-25T10:05:53.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.639+0000 7f808ac38640 1 -- 172.21.15.16:0/1920849235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840b9720 msgr2=0x7f80840ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.639+0000 7f808ac38640 1 --2- 172.21.15.16:0/1920849235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840b9720 0x7f80840ce7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f80840d2320 tx=0x7f807002a130 comp rx=0 tx=0).stop 2024-04-25T10:05:53.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.639+0000 7f808ac38640 1 -- 172.21.15.16:0/1920849235 shutdown_connections 2024-04-25T10:05:53.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.639+0000 7f808ac38640 1 --2- 172.21.15.16:0/1920849235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840b9720 0x7f80840ce7a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.639+0000 7f808ac38640 1 -- 172.21.15.16:0/1920849235 >> 172.21.15.16:0/1920849235 conn(0x7f80840b55e0 msgr2=0x7f80840b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:53.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.639+0000 7f808ac38640 1 -- 172.21.15.16:0/1920849235 shutdown_connections 2024-04-25T10:05:53.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.640+0000 7f808ac38640 1 -- 172.21.15.16:0/1920849235 wait complete. 2024-04-25T10:05:53.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.641+0000 7f808ac38640 1 Processor -- start 2024-04-25T10:05:53.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.641+0000 7f808ac38640 1 -- start start 2024-04-25T10:05:53.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f808ac38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840b9720 0x7f808415ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f808ac38640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80840cece0 0x7f808415f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f808ac38640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8084161090 0x7f808415cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f808ac38640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8084161470 con 0x7f80840b9720 2024-04-25T10:05:53.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f808ac38640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80840d1e30 con 0x7f8084161090 2024-04-25T10:05:53.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f808ac38640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80840d2d40 con 0x7f80840cece0 2024-04-25T10:05:53.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f80889d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840b9720 0x7f808415ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f80889d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840b9720 0x7f808415ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36078/0 (socket says 172.21.15.16:36078) 2024-04-25T10:05:53.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f80889d4640 1 -- 172.21.15.16:0/3840988295 learned_addr learned my addr 172.21.15.16:0/3840988295 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:53.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f8083fff640 1 --2- 172.21.15.16:0/3840988295 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80840cece0 0x7f808415f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f80891d5640 1 --2- 172.21.15.16:0/3840988295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8084161090 0x7f808415cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f80889d4640 1 -- 172.21.15.16:0/3840988295 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80840cece0 msgr2=0x7f808415f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f80889d4640 1 --2- 172.21.15.16:0/3840988295 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80840cece0 0x7f808415f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f80889d4640 1 -- 172.21.15.16:0/3840988295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8084161090 msgr2=0x7f808415cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f80889d4640 1 --2- 172.21.15.16:0/3840988295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8084161090 0x7f808415cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f80889d4640 1 -- 172.21.15.16:0/3840988295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808415d7a0 con 0x7f80840b9720 2024-04-25T10:05:53.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f80889d4640 1 --2- 172.21.15.16:0/3840988295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840b9720 0x7f808415ec70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f807800b420 tx=0x7f807800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f80817fa640 1 -- 172.21.15.16:0/3840988295 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078011830 con 0x7f80840b9720 2024-04-25T10:05:53.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f80817fa640 1 -- 172.21.15.16:0/3840988295 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8078016070 con 0x7f80840b9720 2024-04-25T10:05:53.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f80817fa640 1 -- 172.21.15.16:0/3840988295 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078011c10 con 0x7f80840b9720 2024-04-25T10:05:53.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f808ac38640 1 -- 172.21.15.16:0/3840988295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808415da80 con 0x7f80840b9720 2024-04-25T10:05:53.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.642+0000 7f808ac38640 1 -- 172.21.15.16:0/3840988295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808415e040 con 0x7f80840b9720 2024-04-25T10:05:53.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.643+0000 7f808ac38640 1 -- 172.21.15.16:0/3840988295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80840750b0 con 0x7f80840b9720 2024-04-25T10:05:53.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.644+0000 7f80817fa640 1 -- 172.21.15.16:0/3840988295 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f807801b460 con 0x7f80840b9720 2024-04-25T10:05:53.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.644+0000 7f80817fa640 1 --2- 172.21.15.16:0/3840988295 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f806806b4d0 0x7f806806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.644+0000 7f80817fa640 1 -- 172.21.15.16:0/3840988295 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8078089cb0 con 0x7f80840b9720 2024-04-25T10:05:53.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.644+0000 7f8083fff640 1 --2- 172.21.15.16:0/3840988295 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f806806b4d0 0x7f806806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.645+0000 7f8083fff640 1 --2- 172.21.15.16:0/3840988295 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f806806b4d0 0x7f806806d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8070034000 tx=0x7f807002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.648+0000 7f80817fa640 1 -- 172.21.15.16:0/3840988295 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8078020020 con 0x7f80840b9720 2024-04-25T10:05:53.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.867+0000 7f808ac38640 1 -- 172.21.15.16:0/3840988295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f80840b32b0 con 0x7f80840b9720 2024-04-25T10:05:53.873 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-04-25T10:05:53.868+0000 7ff13f9bb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:05:53.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.869+0000 7f80817fa640 1 -- 172.21.15.16:0/3840988295 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8078058ba0 con 0x7f80840b9720 2024-04-25T10:05:53.874 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:05:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.872+0000 7f808ac38640 1 -- 172.21.15.16:0/3840988295 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f806806b4d0 msgr2=0x7f806806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.872+0000 7f808ac38640 1 --2- 172.21.15.16:0/3840988295 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f806806b4d0 0x7f806806d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8070034000 tx=0x7f807002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.872+0000 7f808ac38640 1 -- 172.21.15.16:0/3840988295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840b9720 msgr2=0x7f808415ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.872+0000 7f808ac38640 1 --2- 172.21.15.16:0/3840988295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840b9720 0x7f808415ec70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f807800b420 tx=0x7f807800b8e0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.872+0000 7f808ac38640 1 -- 172.21.15.16:0/3840988295 shutdown_connections 2024-04-25T10:05:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.872+0000 7f808ac38640 1 --2- 172.21.15.16:0/3840988295 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f806806b4d0 0x7f806806d990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.872+0000 7f808ac38640 1 --2- 172.21.15.16:0/3840988295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8084161090 0x7f808415cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.872+0000 7f808ac38640 1 --2- 172.21.15.16:0/3840988295 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80840cece0 0x7f808415f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.872+0000 7f808ac38640 1 --2- 172.21.15.16:0/3840988295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840b9720 0x7f808415ec70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.872+0000 7f808ac38640 1 -- 172.21.15.16:0/3840988295 >> 172.21.15.16:0/3840988295 conn(0x7f80840b55e0 msgr2=0x7f80840d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:53.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.872+0000 7f808ac38640 1 -- 172.21.15.16:0/3840988295 shutdown_connections 2024-04-25T10:05:53.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.872+0000 7f808ac38640 1 -- 172.21.15.16:0/3840988295 wait complete. 2024-04-25T10:05:53.884 INFO:tasks.ceph.mds.b.smithi186.stderr: -6885> 2024-04-25T10:05:15.231+0000 7f8ddbd57640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:05:53.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:05:53.917 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2024-04-25T10:05:53.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.987+0000 7f61fd5f5640 1 Processor -- start 2024-04-25T10:05:53.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.987+0000 7f61fd5f5640 1 -- start start 2024-04-25T10:05:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.988+0000 7f61fd5f5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f61f80cfe70 0x7f61f80ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.988+0000 7f61fd5f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61f815e110 0x7f61f815e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.988+0000 7f61fd5f5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f815eaa0 con 0x7f61f815b7b0 2024-04-25T10:05:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.988+0000 7f61fd5f5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f80ced00 con 0x7f61f80cfe70 2024-04-25T10:05:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.988+0000 7f61fd5f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f80ce330 con 0x7f61f815e110 2024-04-25T10:05:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.988+0000 7f61f6ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f61f815b7b0 0x7f61f80cf760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45444/0 (socket says 172.21.15.16:45444) 2024-04-25T10:05:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.988+0000 7f61f6ffd640 1 -- 172.21.15.16:0/3768306541 learned_addr learned my addr 172.21.15.16:0/3768306541 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.988+0000 7f61effff640 1 --2- 172.21.15.16:0/3768306541 >> v2:172.21.15.186:3300/0 conn(0x7f61f80cfe70 0x7f61f80ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.988+0000 7f61f77fe640 1 --2- 172.21.15.16:0/3768306541 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61f815e110 0x7f61f815e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.988+0000 7f61f67fc640 1 -- 172.21.15.16:0/3768306541 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554995061 0 0) 0x7f61e8001110 con 0x7f61f815b7b0 2024-04-25T10:05:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.988+0000 7f61f67fc640 1 -- 172.21.15.16:0/3768306541 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61e4003520 con 0x7f61f815b7b0 2024-04-25T10:05:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.988+0000 7f61effff640 1 -- 172.21.15.16:0/3768306541 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61f815e110 msgr2=0x7f61f815e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:53.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.988+0000 7f61effff640 1 --2- 172.21.15.16:0/3768306541 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61f815e110 0x7f61f815e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.989+0000 7f61effff640 1 -- 172.21.15.16:0/3768306541 >> v1:172.21.15.16:6789/0 conn(0x7f61f815b7b0 legacy=0x7f61f80cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.989+0000 7f61effff640 1 -- 172.21.15.16:0/3768306541 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f815f000 con 0x7f61f80cfe70 2024-04-25T10:05:53.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.989+0000 7f61f77fe640 1 --2- 172.21.15.16:0/3768306541 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61f815e110 0x7f61f815e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:53.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.989+0000 7f61effff640 1 --2- 172.21.15.16:0/3768306541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61f80cfe70 0x7f61f80ce7c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f61e00097a0 tx=0x7f61e002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1e9e79e9f25d300 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.989+0000 7f61f67fc640 1 -- 172.21.15.16:0/3768306541 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0037070 con 0x7f61f80cfe70 2024-04-25T10:05:53.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.990+0000 7f61f67fc640 1 -- 172.21.15.16:0/3768306541 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e0032440 con 0x7f61f80cfe70 2024-04-25T10:05:53.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.990+0000 7f61f67fc640 1 -- 172.21.15.16:0/3768306541 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0032810 con 0x7f61f80cfe70 2024-04-25T10:05:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.990+0000 7f61fd5f5640 1 -- 172.21.15.16:0/3768306541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61f80cfe70 msgr2=0x7f61f80ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.990+0000 7f61fd5f5640 1 --2- 172.21.15.16:0/3768306541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61f80cfe70 0x7f61f80ce7c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f61e00097a0 tx=0x7f61e002a180 comp rx=0 tx=0).stop 2024-04-25T10:05:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.990+0000 7f61fd5f5640 1 -- 172.21.15.16:0/3768306541 shutdown_connections 2024-04-25T10:05:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.990+0000 7f61fd5f5640 1 --2- 172.21.15.16:0/3768306541 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61f815e110 0x7f61f815e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.990+0000 7f61fd5f5640 1 --2- 172.21.15.16:0/3768306541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61f80cfe70 0x7f61f80ce7c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.990+0000 7f61fd5f5640 1 -- 172.21.15.16:0/3768306541 >> 172.21.15.16:0/3768306541 conn(0x7f61f806f4a0 msgr2=0x7f61f806f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.990+0000 7f61fd5f5640 1 -- 172.21.15.16:0/3768306541 shutdown_connections 2024-04-25T10:05:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.990+0000 7f61fd5f5640 1 -- 172.21.15.16:0/3768306541 wait complete. 2024-04-25T10:05:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.991+0000 7f61fd5f5640 1 Processor -- start 2024-04-25T10:05:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.991+0000 7f61fd5f5640 1 -- start start 2024-04-25T10:05:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.992+0000 7f61fd5f5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61f80cfe70 0x7f61f80b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.992+0000 7f61fd5f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f815b7b0 0x7f61f80b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.992+0000 7f61fd5f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61f815e110 0x7f61f80b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.992+0000 7f61fd5f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f80cdfc0 con 0x7f61f815b7b0 2024-04-25T10:05:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.992+0000 7f61fd5f5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f80cdce0 con 0x7f61f80cfe70 2024-04-25T10:05:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.992+0000 7f61fd5f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f81619f0 con 0x7f61f815e110 2024-04-25T10:05:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.992+0000 7f61effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f815b7b0 0x7f61f80b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.992+0000 7f61effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f815b7b0 0x7f61f80b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36088/0 (socket says 172.21.15.16:36088) 2024-04-25T10:05:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.992+0000 7f61effff640 1 -- 172.21.15.16:0/3391453554 learned_addr learned my addr 172.21.15.16:0/3391453554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.992+0000 7f61f77fe640 1 --2- 172.21.15.16:0/3391453554 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61f815e110 0x7f61f80b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.992+0000 7f61f6ffd640 1 --2- 172.21.15.16:0/3391453554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61f80cfe70 0x7f61f80b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.993+0000 7f61effff640 1 -- 172.21.15.16:0/3391453554 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61f815e110 msgr2=0x7f61f80b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.993+0000 7f61effff640 1 --2- 172.21.15.16:0/3391453554 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61f815e110 0x7f61f80b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.993+0000 7f61effff640 1 -- 172.21.15.16:0/3391453554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61f80cfe70 msgr2=0x7f61f80b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.993+0000 7f61effff640 1 --2- 172.21.15.16:0/3391453554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61f80cfe70 0x7f61f80b6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.993+0000 7f61effff640 1 -- 172.21.15.16:0/3391453554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f81f7400 con 0x7f61f815b7b0 2024-04-25T10:05:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.993+0000 7f61f77fe640 1 --2- 172.21.15.16:0/3391453554 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61f815e110 0x7f61f80b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.993+0000 7f61effff640 1 --2- 172.21.15.16:0/3391453554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f815b7b0 0x7f61f80b3860 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f61e0035040 tx=0x7f61e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.993+0000 7f61ef7fe640 1 -- 172.21.15.16:0/3391453554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0041030 con 0x7f61f815b7b0 2024-04-25T10:05:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.993+0000 7f61ef7fe640 1 -- 172.21.15.16:0/3391453554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e003d4e0 con 0x7f61f815b7b0 2024-04-25T10:05:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.993+0000 7f61ef7fe640 1 -- 172.21.15.16:0/3391453554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0037040 con 0x7f61f815b7b0 2024-04-25T10:05:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.993+0000 7f61fd5f5640 1 -- 172.21.15.16:0/3391453554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f81f76e0 con 0x7f61f815b7b0 2024-04-25T10:05:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.993+0000 7f61fd5f5640 1 -- 172.21.15.16:0/3391453554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f81f7ca0 con 0x7f61f815b7b0 2024-04-25T10:05:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.994+0000 7f61fd5f5640 1 -- 172.21.15.16:0/3391453554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f8067570 con 0x7f61f815b7b0 2024-04-25T10:05:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.994+0000 7f61ef7fe640 1 -- 172.21.15.16:0/3391453554 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f61e0044070 con 0x7f61f815b7b0 2024-04-25T10:05:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.995+0000 7f61ef7fe640 1 --2- 172.21.15.16:0/3391453554 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f61d806b4d0 0x7f61d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.995+0000 7f61ef7fe640 1 -- 172.21.15.16:0/3391453554 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f61e00aa890 con 0x7f61f815b7b0 2024-04-25T10:05:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.995+0000 7f61f6ffd640 1 --2- 172.21.15.16:0/3391453554 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f61d806b4d0 0x7f61d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.996+0000 7f61f6ffd640 1 --2- 172.21.15.16:0/3391453554 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f61d806b4d0 0x7f61d806d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f61e8005880 tx=0x7f61e8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:54.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:53.999+0000 7f61ef7fe640 1 -- 172.21.15.16:0/3391453554 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f61f8067570 con 0x7f61f815b7b0 2024-04-25T10:05:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:54.217+0000 7f61fd5f5640 1 -- 172.21.15.16:0/3391453554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f61f80750b0 con 0x7f61f815b7b0 2024-04-25T10:05:54.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:54.871+0000 7f61ef7fe640 1 -- 172.21.15.16:0/3391453554 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f61e00797d0 con 0x7f61f815b7b0 2024-04-25T10:05:54.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:54.873+0000 7f61fd5f5640 1 -- 172.21.15.16:0/3391453554 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f61d806b4d0 msgr2=0x7f61d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:54.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:54.873+0000 7f61fd5f5640 1 --2- 172.21.15.16:0/3391453554 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f61d806b4d0 0x7f61d806d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f61e8005880 tx=0x7f61e8009040 comp rx=0 tx=0).stop 2024-04-25T10:05:54.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:54.873+0000 7f61fd5f5640 1 -- 172.21.15.16:0/3391453554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f815b7b0 msgr2=0x7f61f80b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:54.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:54.873+0000 7f61fd5f5640 1 --2- 172.21.15.16:0/3391453554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f815b7b0 0x7f61f80b3860 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f61e0035040 tx=0x7f61e00046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:54.873+0000 7f61fd5f5640 1 -- 172.21.15.16:0/3391453554 shutdown_connections 2024-04-25T10:05:54.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:54.873+0000 7f61fd5f5640 1 --2- 172.21.15.16:0/3391453554 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f61d806b4d0 0x7f61d806d990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:54.873+0000 7f61fd5f5640 1 --2- 172.21.15.16:0/3391453554 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61f815e110 0x7f61f80b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:54.873+0000 7f61fd5f5640 1 --2- 172.21.15.16:0/3391453554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f815b7b0 0x7f61f80b3860 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:54.873+0000 7f61fd5f5640 1 --2- 172.21.15.16:0/3391453554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61f80cfe70 0x7f61f80b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:54.873+0000 7f61fd5f5640 1 -- 172.21.15.16:0/3391453554 >> 172.21.15.16:0/3391453554 conn(0x7f61f806f4a0 msgr2=0x7f61f806c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:54.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:54.873+0000 7f61fd5f5640 1 -- 172.21.15.16:0/3391453554 shutdown_connections 2024-04-25T10:05:54.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:54.873+0000 7f61fd5f5640 1 -- 172.21.15.16:0/3391453554 wait complete. 2024-04-25T10:05:54.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:05:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.000+0000 7f5e6c876640 1 Processor -- start 2024-04-25T10:05:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.000+0000 7f5e6c876640 1 -- start start 2024-04-25T10:05:55.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.001+0000 7f5e6c876640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e680b9580 0x7f5e681618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.001+0000 7f5e6c876640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e68162040 con 0x7f5e680cef40 2024-04-25T10:05:55.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.001+0000 7f5e6c876640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e68162320 con 0x7f5e681610b0 2024-04-25T10:05:55.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.001+0000 7f5e6c876640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e68161e10 con 0x7f5e680b9580 2024-04-25T10:05:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.001+0000 7f5e66d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e680b9580 0x7f5e681618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.001+0000 7f5e65d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5e680cef40 0x7f5e680d2610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45452/0 (socket says 172.21.15.16:45452) 2024-04-25T10:05:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.001+0000 7f5e65d9b640 1 -- 172.21.15.16:0/2700144702 learned_addr learned my addr 172.21.15.16:0/2700144702 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.001+0000 7f5e66d9d640 1 -- 172.21.15.16:0/2700144702 >> v1:172.21.15.186:6789/0 conn(0x7f5e681610b0 legacy=0x7f5e680ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.001+0000 7f5e66d9d640 1 -- 172.21.15.16:0/2700144702 >> v1:172.21.15.16:6789/0 conn(0x7f5e680cef40 legacy=0x7f5e680d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.001+0000 7f5e66d9d640 1 -- 172.21.15.16:0/2700144702 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e68162600 con 0x7f5e680b9580 2024-04-25T10:05:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.002+0000 7f5e66d9d640 1 --2- 172.21.15.16:0/2700144702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e680b9580 0x7f5e681618d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5e60009d90 tx=0x7f5e6002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74812d3dc7ab679 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.002+0000 7f5e6559a640 1 -- 172.21.15.16:0/2700144702 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e60031600 con 0x7f5e680b9580 2024-04-25T10:05:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.002+0000 7f5e6559a640 1 -- 172.21.15.16:0/2700144702 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e600317a0 con 0x7f5e680b9580 2024-04-25T10:05:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.002+0000 7f5e6559a640 1 -- 172.21.15.16:0/2700144702 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e60031b70 con 0x7f5e680b9580 2024-04-25T10:05:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.003+0000 7f5e6c876640 1 -- 172.21.15.16:0/2700144702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e680b9580 msgr2=0x7f5e681618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.003+0000 7f5e6c876640 1 --2- 172.21.15.16:0/2700144702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e680b9580 0x7f5e681618d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5e60009d90 tx=0x7f5e6002a610 comp rx=0 tx=0).stop 2024-04-25T10:05:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.003+0000 7f5e6c876640 1 -- 172.21.15.16:0/2700144702 shutdown_connections 2024-04-25T10:05:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.003+0000 7f5e6c876640 1 --2- 172.21.15.16:0/2700144702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e680b9580 0x7f5e681618d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.003+0000 7f5e6c876640 1 -- 172.21.15.16:0/2700144702 >> 172.21.15.16:0/2700144702 conn(0x7f5e680b5570 msgr2=0x7f5e680b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.003+0000 7f5e6c876640 1 -- 172.21.15.16:0/2700144702 shutdown_connections 2024-04-25T10:05:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.003+0000 7f5e6c876640 1 -- 172.21.15.16:0/2700144702 wait complete. 2024-04-25T10:05:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.004+0000 7f5e6c876640 1 Processor -- start 2024-04-25T10:05:55.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.004+0000 7f5e6c876640 1 -- start start 2024-04-25T10:05:55.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.005+0000 7f5e6c876640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e680b9580 0x7f5e680ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.005+0000 7f5e6c876640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e680cef40 0x7f5e6813ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.005+0000 7f5e6c876640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e681610b0 0x7f5e6813d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.005+0000 7f5e6c876640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e680b99f0 con 0x7f5e681610b0 2024-04-25T10:05:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.005+0000 7f5e6c876640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e680d1ea0 con 0x7f5e680cef40 2024-04-25T10:05:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.005+0000 7f5e6c876640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e680b6c00 con 0x7f5e680b9580 2024-04-25T10:05:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.005+0000 7f5e66d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e681610b0 0x7f5e6813d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.005+0000 7f5e66d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e681610b0 0x7f5e6813d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36102/0 (socket says 172.21.15.16:36102) 2024-04-25T10:05:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.005+0000 7f5e66d9d640 1 -- 172.21.15.16:0/3184888773 learned_addr learned my addr 172.21.15.16:0/3184888773 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.005+0000 7f5e65d9b640 1 --2- 172.21.15.16:0/3184888773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e680cef40 0x7f5e6813ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.005+0000 7f5e6659c640 1 --2- 172.21.15.16:0/3184888773 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e680b9580 0x7f5e680ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.005+0000 7f5e66d9d640 1 -- 172.21.15.16:0/3184888773 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e680b9580 msgr2=0x7f5e680ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.005+0000 7f5e66d9d640 1 --2- 172.21.15.16:0/3184888773 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e680b9580 0x7f5e680ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.005+0000 7f5e66d9d640 1 -- 172.21.15.16:0/3184888773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e680cef40 msgr2=0x7f5e6813ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.005+0000 7f5e66d9d640 1 --2- 172.21.15.16:0/3184888773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e680cef40 0x7f5e6813ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.005+0000 7f5e66d9d640 1 -- 172.21.15.16:0/3184888773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6813da10 con 0x7f5e681610b0 2024-04-25T10:05:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.005+0000 7f5e6659c640 1 --2- 172.21.15.16:0/3184888773 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e680b9580 0x7f5e680ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.006+0000 7f5e66d9d640 1 --2- 172.21.15.16:0/3184888773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e681610b0 0x7f5e6813d1f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5e60009d90 tx=0x7f5e60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.006+0000 7f5e56ffd640 1 -- 172.21.15.16:0/3184888773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e60040070 con 0x7f5e681610b0 2024-04-25T10:05:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.006+0000 7f5e56ffd640 1 -- 172.21.15.16:0/3184888773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e60030b60 con 0x7f5e681610b0 2024-04-25T10:05:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.006+0000 7f5e56ffd640 1 -- 172.21.15.16:0/3184888773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6003c420 con 0x7f5e681610b0 2024-04-25T10:05:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.006+0000 7f5e6c876640 1 -- 172.21.15.16:0/3184888773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e6813dcf0 con 0x7f5e681610b0 2024-04-25T10:05:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.006+0000 7f5e6c876640 1 -- 172.21.15.16:0/3184888773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e6813d730 con 0x7f5e681610b0 2024-04-25T10:05:55.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.007+0000 7f5e6c876640 1 -- 172.21.15.16:0/3184888773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e680cad90 con 0x7f5e681610b0 2024-04-25T10:05:55.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.007+0000 7f5e56ffd640 1 -- 172.21.15.16:0/3184888773 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e60039040 con 0x7f5e681610b0 2024-04-25T10:05:55.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.008+0000 7f5e56ffd640 1 --2- 172.21.15.16:0/3184888773 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5e4806b500 0x7f5e4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.008+0000 7f5e56ffd640 1 -- 172.21.15.16:0/3184888773 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f5e600ab390 con 0x7f5e681610b0 2024-04-25T10:05:55.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.008+0000 7f5e6659c640 1 --2- 172.21.15.16:0/3184888773 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5e4806b500 0x7f5e4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.009+0000 7f5e6659c640 1 --2- 172.21.15.16:0/3184888773 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5e4806b500 0x7f5e4806d9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800a830 tx=0x7f5e58008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:55.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.012+0000 7f5e56ffd640 1 -- 172.21.15.16:0/3184888773 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e60079b90 con 0x7f5e681610b0 2024-04-25T10:05:55.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.192+0000 7f5e6c876640 1 -- 172.21.15.16:0/3184888773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5e680750b0 con 0x7f5e681610b0 2024-04-25T10:05:55.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.872+0000 7f5e56ffd640 1 -- 172.21.15.16:0/3184888773 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5e6007dc30 con 0x7f5e681610b0 2024-04-25T10:05:55.878 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:05:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.877+0000 7f5e6c876640 1 -- 172.21.15.16:0/3184888773 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5e4806b500 msgr2=0x7f5e4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.877+0000 7f5e6c876640 1 --2- 172.21.15.16:0/3184888773 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5e4806b500 0x7f5e4806d9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800a830 tx=0x7f5e58008040 comp rx=0 tx=0).stop 2024-04-25T10:05:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.877+0000 7f5e6c876640 1 -- 172.21.15.16:0/3184888773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e681610b0 msgr2=0x7f5e6813d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.877+0000 7f5e6c876640 1 --2- 172.21.15.16:0/3184888773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e681610b0 0x7f5e6813d1f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5e60009d90 tx=0x7f5e60004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.878+0000 7f5e6c876640 1 -- 172.21.15.16:0/3184888773 shutdown_connections 2024-04-25T10:05:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.878+0000 7f5e6c876640 1 --2- 172.21.15.16:0/3184888773 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5e4806b500 0x7f5e4806d9c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.878+0000 7f5e6c876640 1 --2- 172.21.15.16:0/3184888773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e681610b0 0x7f5e6813d1f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.878+0000 7f5e6c876640 1 --2- 172.21.15.16:0/3184888773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e680cef40 0x7f5e6813ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.878+0000 7f5e6c876640 1 --2- 172.21.15.16:0/3184888773 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e680b9580 0x7f5e680ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.878+0000 7f5e6c876640 1 -- 172.21.15.16:0/3184888773 >> 172.21.15.16:0/3184888773 conn(0x7f5e680b5570 msgr2=0x7f5e680d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:55.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.878+0000 7f5e6c876640 1 -- 172.21.15.16:0/3184888773 shutdown_connections 2024-04-25T10:05:55.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:55.878+0000 7f5e6c876640 1 -- 172.21.15.16:0/3184888773 wait complete. 2024-04-25T10:05:55.899 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:05:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.004+0000 7f0e65c9e640 1 Processor -- start 2024-04-25T10:05:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.004+0000 7f0e65c9e640 1 -- start start 2024-04-25T10:05:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.005+0000 7f0e65c9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e600b9720 0x7f0e600ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.005+0000 7f0e65c9e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0e600cece0 0x7f0e600d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.005+0000 7f0e65c9e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e60161f80 con 0x7f0e60161090 2024-04-25T10:05:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.005+0000 7f0e65c9e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e600cf130 con 0x7f0e600cece0 2024-04-25T10:05:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.005+0000 7f0e65c9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e600d21a0 con 0x7f0e600b9720 2024-04-25T10:05:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.005+0000 7f0e5ffff640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0e600cece0 0x7f0e600d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.005+0000 7f0e5effd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e600b9720 0x7f0e600ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.005+0000 7f0e5f7fe640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f0e60161090 0x7f0e60161730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:55022/0 (socket says 172.21.15.16:55022) 2024-04-25T10:05:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.005+0000 7f0e5f7fe640 1 -- 172.21.15.16:0/2491158970 learned_addr learned my addr 172.21.15.16:0/2491158970 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:05:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.005+0000 7f0e5effd640 1 -- 172.21.15.16:0/2491158970 >> v2:172.21.15.186:3300/0 conn(0x7f0e600cece0 msgr2=0x7f0e600d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.005+0000 7f0e5effd640 1 --2- 172.21.15.16:0/2491158970 >> v2:172.21.15.186:3300/0 conn(0x7f0e600cece0 0x7f0e600d2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.006+0000 7f0e5effd640 1 -- 172.21.15.16:0/2491158970 >> v1:172.21.15.186:6789/0 conn(0x7f0e60161090 legacy=0x7f0e60161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.006+0000 7f0e5effd640 1 -- 172.21.15.16:0/2491158970 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e60162260 con 0x7f0e600b9720 2024-04-25T10:05:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.006+0000 7f0e5ffff640 1 --2- 172.21.15.16:0/2491158970 >> v2:172.21.15.186:3300/0 conn(0x7f0e600cece0 0x7f0e600d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.006+0000 7f0e5effd640 1 --2- 172.21.15.16:0/2491158970 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e600b9720 0x7f0e600ce7a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0e540097a0 tx=0x7f0e5402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17b9307f1f595bb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.006+0000 7f0e5e7fc640 1 -- 172.21.15.16:0/2491158970 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54037070 con 0x7f0e600b9720 2024-04-25T10:05:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.007+0000 7f0e5e7fc640 1 -- 172.21.15.16:0/2491158970 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e54032440 con 0x7f0e600b9720 2024-04-25T10:05:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.007+0000 7f0e5e7fc640 1 -- 172.21.15.16:0/2491158970 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54032810 con 0x7f0e600b9720 2024-04-25T10:05:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.007+0000 7f0e65c9e640 1 -- 172.21.15.16:0/2491158970 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e600b9720 msgr2=0x7f0e600ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.007+0000 7f0e65c9e640 1 --2- 172.21.15.16:0/2491158970 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e600b9720 0x7f0e600ce7a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0e540097a0 tx=0x7f0e5402a180 comp rx=0 tx=0).stop 2024-04-25T10:05:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.007+0000 7f0e65c9e640 1 -- 172.21.15.16:0/2491158970 shutdown_connections 2024-04-25T10:05:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.007+0000 7f0e65c9e640 1 --2- 172.21.15.16:0/2491158970 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e600b9720 0x7f0e600ce7a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.007+0000 7f0e65c9e640 1 --2- 172.21.15.16:0/2491158970 >> v2:172.21.15.186:3300/0 conn(0x7f0e600cece0 0x7f0e600d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.007+0000 7f0e65c9e640 1 -- 172.21.15.16:0/2491158970 >> 172.21.15.16:0/2491158970 conn(0x7f0e600b55e0 msgr2=0x7f0e600b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.007+0000 7f0e65c9e640 1 -- 172.21.15.16:0/2491158970 shutdown_connections 2024-04-25T10:05:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.007+0000 7f0e65c9e640 1 -- 172.21.15.16:0/2491158970 wait complete. 2024-04-25T10:05:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.008+0000 7f0e65c9e640 1 Processor -- start 2024-04-25T10:05:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.008+0000 7f0e65c9e640 1 -- start start 2024-04-25T10:05:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.009+0000 7f0e65c9e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e600b9720 0x7f0e6015f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.009+0000 7f0e65c9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e600cece0 0x7f0e601f7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.009+0000 7f0e65c9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e60161090 0x7f0e60151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.009+0000 7f0e65c9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e60161470 con 0x7f0e600cece0 2024-04-25T10:05:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.009+0000 7f0e65c9e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e600d1e30 con 0x7f0e600b9720 2024-04-25T10:05:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.009+0000 7f0e65c9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e600d2b70 con 0x7f0e60161090 2024-04-25T10:05:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.009+0000 7f0e5effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e600cece0 0x7f0e601f7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.009+0000 7f0e5effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e600cece0 0x7f0e601f7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36114/0 (socket says 172.21.15.16:36114) 2024-04-25T10:05:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.009+0000 7f0e5effd640 1 -- 172.21.15.16:0/1357779144 learned_addr learned my addr 172.21.15.16:0/1357779144 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:05:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.009+0000 7f0e5f7fe640 1 --2- 172.21.15.16:0/1357779144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e600b9720 0x7f0e6015f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.009+0000 7f0e5ffff640 1 --2- 172.21.15.16:0/1357779144 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e60161090 0x7f0e60151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.010+0000 7f0e5effd640 1 -- 172.21.15.16:0/1357779144 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e60161090 msgr2=0x7f0e60151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.010+0000 7f0e5effd640 1 --2- 172.21.15.16:0/1357779144 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e60161090 0x7f0e60151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.010+0000 7f0e5effd640 1 -- 172.21.15.16:0/1357779144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e600b9720 msgr2=0x7f0e6015f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.010+0000 7f0e5effd640 1 --2- 172.21.15.16:0/1357779144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e600b9720 0x7f0e6015f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.010+0000 7f0e5effd640 1 -- 172.21.15.16:0/1357779144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e600cacd0 con 0x7f0e600cece0 2024-04-25T10:05:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.010+0000 7f0e5f7fe640 1 --2- 172.21.15.16:0/1357779144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e600b9720 0x7f0e6015f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.010+0000 7f0e5ffff640 1 --2- 172.21.15.16:0/1357779144 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e60161090 0x7f0e60151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.010+0000 7f0e5effd640 1 --2- 172.21.15.16:0/1357779144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e600cece0 0x7f0e601f7ec0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0e54035040 tx=0x7f0e540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.010+0000 7f0e47fff640 1 -- 172.21.15.16:0/1357779144 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54037040 con 0x7f0e600cece0 2024-04-25T10:05:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.010+0000 7f0e47fff640 1 -- 172.21.15.16:0/1357779144 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e54030500 con 0x7f0e600cece0 2024-04-25T10:05:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.010+0000 7f0e65c9e640 1 -- 172.21.15.16:0/1357779144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e600cafb0 con 0x7f0e600cece0 2024-04-25T10:05:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.010+0000 7f0e47fff640 1 -- 172.21.15.16:0/1357779144 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54030910 con 0x7f0e600cece0 2024-04-25T10:05:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.010+0000 7f0e65c9e640 1 -- 172.21.15.16:0/1357779144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e600cb570 con 0x7f0e600cece0 2024-04-25T10:05:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.011+0000 7f0e65c9e640 1 -- 172.21.15.16:0/1357779144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e600750b0 con 0x7f0e600cece0 2024-04-25T10:05:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.012+0000 7f0e47fff640 1 -- 172.21.15.16:0/1357779144 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e54044020 con 0x7f0e600cece0 2024-04-25T10:05:56.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.012+0000 7f0e47fff640 1 --2- 172.21.15.16:0/1357779144 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0e4c06b4d0 0x7f0e4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.012+0000 7f0e47fff640 1 -- 172.21.15.16:0/1357779144 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5386+0+0 (secure 0 0 0) 0x7f0e540aa2e0 con 0x7f0e600cece0 2024-04-25T10:05:56.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.012+0000 7f0e5f7fe640 1 --2- 172.21.15.16:0/1357779144 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0e4c06b4d0 0x7f0e4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.013+0000 7f0e5f7fe640 1 --2- 172.21.15.16:0/1357779144 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0e4c06b4d0 0x7f0e4c06d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0e50000f60 tx=0x7f0e50009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.016+0000 7f0e47fff640 1 -- 172.21.15.16:0/1357779144 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0e540792b0 con 0x7f0e600cece0 2024-04-25T10:05:56.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.204+0000 7f0e65c9e640 1 -- 172.21.15.16:0/1357779144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0e600cf710 con 0x7f0e600cece0 2024-04-25T10:05:56.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.877+0000 7f0e47fff640 1 -- 172.21.15.16:0/1357779144 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0e54078bb0 con 0x7f0e600cece0 2024-04-25T10:05:56.883 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-04-25T10:05:56.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.880+0000 7f0e65c9e640 1 -- 172.21.15.16:0/1357779144 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0e4c06b4d0 msgr2=0x7f0e4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.880+0000 7f0e65c9e640 1 --2- 172.21.15.16:0/1357779144 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0e4c06b4d0 0x7f0e4c06d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0e50000f60 tx=0x7f0e50009040 comp rx=0 tx=0).stop 2024-04-25T10:05:56.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.880+0000 7f0e65c9e640 1 -- 172.21.15.16:0/1357779144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e600cece0 msgr2=0x7f0e601f7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.880+0000 7f0e65c9e640 1 --2- 172.21.15.16:0/1357779144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e600cece0 0x7f0e601f7ec0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0e54035040 tx=0x7f0e540046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.881+0000 7f0e65c9e640 1 -- 172.21.15.16:0/1357779144 shutdown_connections 2024-04-25T10:05:56.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.881+0000 7f0e65c9e640 1 --2- 172.21.15.16:0/1357779144 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0e4c06b4d0 0x7f0e4c06d990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.881+0000 7f0e65c9e640 1 --2- 172.21.15.16:0/1357779144 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e60161090 0x7f0e60151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.881+0000 7f0e65c9e640 1 --2- 172.21.15.16:0/1357779144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e600cece0 0x7f0e601f7ec0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.881+0000 7f0e65c9e640 1 --2- 172.21.15.16:0/1357779144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e600b9720 0x7f0e6015f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.881+0000 7f0e65c9e640 1 -- 172.21.15.16:0/1357779144 >> 172.21.15.16:0/1357779144 conn(0x7f0e600b55e0 msgr2=0x7f0e600b84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:56.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.882+0000 7f0e65c9e640 1 -- 172.21.15.16:0/1357779144 shutdown_connections 2024-04-25T10:05:56.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:05:56.882+0000 7f0e65c9e640 1 -- 172.21.15.16:0/1357779144 wait complete. 2024-04-25T10:05:56.905 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:05:56.905 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:05:56.905 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:05:56.905 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:56.905 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:05:56.905 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:05:56.906 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:05:56.906 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:56.906 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:05:56.906 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:05:56.906 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:05:56.906 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:56.906 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:05:56.906 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:05:56.907 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:05:56.907 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:56.916 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:05:56.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:05:56.917 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:05:56.917 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:05:56.920 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:05:56.921 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:05:56.959 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:05:56.959 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:56.959 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:05:56.960 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:56.963 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:05:56.963 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:57.039 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-04-25T10:05:57.040 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-04-25T10:05:57.043 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2024-04-25T10:06:01.096 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T10:06:02.921 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:06:02.921 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:06:02.924 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:06:02.924 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:02.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:06:03.006 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2024-04-25T10:06:03.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.034+0000 7fba5ace1640 1 Processor -- start 2024-04-25T10:06:03.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.034+0000 7fba5ace1640 1 -- start start 2024-04-25T10:06:03.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.035+0000 7fba5ace1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba5415ae20 0x7fba540b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.035+0000 7fba5ace1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba5406da40 con 0x7fba540aedc0 2024-04-25T10:06:03.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.035+0000 7fba5ace1640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba5406dd20 con 0x7fba540b4730 2024-04-25T10:06:03.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.035+0000 7fba5ace1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5415b280 con 0x7fba5415ae20 2024-04-25T10:06:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.035+0000 7fba58a7d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fba540aedc0 0x7fba5406d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38498/0 (socket says 172.21.15.16:38498) 2024-04-25T10:06:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.035+0000 7fba58a7d640 1 -- 172.21.15.16:0/1732702722 learned_addr learned my addr 172.21.15.16:0/1732702722 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.036+0000 7fba53fff640 1 --2- 172.21.15.16:0/1732702722 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba5415ae20 0x7fba540b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.036+0000 7fba537fe640 1 -- 172.21.15.16:0/1732702722 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780144304 0 0) 0x7fba5406da40 con 0x7fba540aedc0 2024-04-25T10:06:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.036+0000 7fba537fe640 1 -- 172.21.15.16:0/1732702722 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba3c003520 con 0x7fba540aedc0 2024-04-25T10:06:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.036+0000 7fba537fe640 1 -- 172.21.15.16:0/1732702722 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4074144862 0 0) 0x7fba5406dd20 con 0x7fba540b4730 2024-04-25T10:06:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.036+0000 7fba537fe640 1 -- 172.21.15.16:0/1732702722 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba5406da40 con 0x7fba540b4730 2024-04-25T10:06:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.036+0000 7fba53fff640 1 -- 172.21.15.16:0/1732702722 >> v1:172.21.15.186:6789/0 conn(0x7fba540b4730 legacy=0x7fba5415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.036+0000 7fba53fff640 1 -- 172.21.15.16:0/1732702722 >> v1:172.21.15.16:6789/0 conn(0x7fba540aedc0 legacy=0x7fba5406d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.036+0000 7fba537fe640 1 -- 172.21.15.16:0/1732702722 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2697900927 0 0) 0x7fba3c003520 con 0x7fba540aedc0 2024-04-25T10:06:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.036+0000 7fba53fff640 1 -- 172.21.15.16:0/1732702722 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5406e000 con 0x7fba5415ae20 2024-04-25T10:06:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.037+0000 7fba53fff640 1 --2- 172.21.15.16:0/1732702722 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba5415ae20 0x7fba540b41f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fba480097d0 tx=0x7fba4802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b684ddefbffcbfc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.037+0000 7fba537fe640 1 -- 172.21.15.16:0/1732702722 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48037070 con 0x7fba5415ae20 2024-04-25T10:06:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.037+0000 7fba537fe640 1 -- 172.21.15.16:0/1732702722 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4802ab90 con 0x7fba5415ae20 2024-04-25T10:06:03.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.037+0000 7fba537fe640 1 -- 172.21.15.16:0/1732702722 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c0019f0 con 0x7fba5415ae20 2024-04-25T10:06:03.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.038+0000 7fba5ace1640 1 -- 172.21.15.16:0/1732702722 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba5415ae20 msgr2=0x7fba540b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.038+0000 7fba5ace1640 1 --2- 172.21.15.16:0/1732702722 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba5415ae20 0x7fba540b41f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fba480097d0 tx=0x7fba4802a100 comp rx=0 tx=0).stop 2024-04-25T10:06:03.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.038+0000 7fba5ace1640 1 -- 172.21.15.16:0/1732702722 shutdown_connections 2024-04-25T10:06:03.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.038+0000 7fba5ace1640 1 --2- 172.21.15.16:0/1732702722 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba5415ae20 0x7fba540b41f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.038+0000 7fba5ace1640 1 -- 172.21.15.16:0/1732702722 >> 172.21.15.16:0/1732702722 conn(0x7fba5406aba0 msgr2=0x7fba5406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:03.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.038+0000 7fba5ace1640 1 -- 172.21.15.16:0/1732702722 shutdown_connections 2024-04-25T10:06:03.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.038+0000 7fba5ace1640 1 -- 172.21.15.16:0/1732702722 wait complete. 2024-04-25T10:06:03.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.039+0000 7fba5ace1640 1 Processor -- start 2024-04-25T10:06:03.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.039+0000 7fba5ace1640 1 -- start start 2024-04-25T10:06:03.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.040+0000 7fba5ace1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba540aedc0 0x7fba5414e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.040+0000 7fba5ace1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba540b4730 0x7fba540cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.040+0000 7fba5ace1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba5415ae20 0x7fba5414f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.040+0000 7fba5ace1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba540af1a0 con 0x7fba540b4730 2024-04-25T10:06:03.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.040+0000 7fba5ace1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5415c050 con 0x7fba540aedc0 2024-04-25T10:06:03.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.040+0000 7fba5ace1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5415ced0 con 0x7fba5415ae20 2024-04-25T10:06:03.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.040+0000 7fba53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba540b4730 0x7fba540cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.040+0000 7fba53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba540b4730 0x7fba540cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34348/0 (socket says 172.21.15.16:34348) 2024-04-25T10:06:03.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.040+0000 7fba53fff640 1 -- 172.21.15.16:0/29657715 learned_addr learned my addr 172.21.15.16:0/29657715 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:03.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.040+0000 7fba58a7d640 1 --2- 172.21.15.16:0/29657715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba540aedc0 0x7fba5414e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.040+0000 7fba5927e640 1 --2- 172.21.15.16:0/29657715 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba5415ae20 0x7fba5414f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.040+0000 7fba53fff640 1 -- 172.21.15.16:0/29657715 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba5415ae20 msgr2=0x7fba5414f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.040+0000 7fba53fff640 1 --2- 172.21.15.16:0/29657715 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba5415ae20 0x7fba5414f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.040+0000 7fba53fff640 1 -- 172.21.15.16:0/29657715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba540aedc0 msgr2=0x7fba5414e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.040+0000 7fba53fff640 1 --2- 172.21.15.16:0/29657715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba540aedc0 0x7fba5414e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.040+0000 7fba53fff640 1 -- 172.21.15.16:0/29657715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba541f6f40 con 0x7fba540b4730 2024-04-25T10:06:03.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.040+0000 7fba5927e640 1 --2- 172.21.15.16:0/29657715 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba5415ae20 0x7fba5414f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:03.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.041+0000 7fba53fff640 1 --2- 172.21.15.16:0/29657715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba540b4730 0x7fba540cade0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fba48035040 tx=0x7fba480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.041+0000 7fba517fa640 1 -- 172.21.15.16:0/29657715 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48041020 con 0x7fba540b4730 2024-04-25T10:06:03.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.041+0000 7fba5ace1640 1 -- 172.21.15.16:0/29657715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba541f7220 con 0x7fba540b4730 2024-04-25T10:06:03.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.041+0000 7fba517fa640 1 -- 172.21.15.16:0/29657715 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4803d640 con 0x7fba540b4730 2024-04-25T10:06:03.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.041+0000 7fba517fa640 1 -- 172.21.15.16:0/29657715 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48037040 con 0x7fba540b4730 2024-04-25T10:06:03.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.041+0000 7fba5ace1640 1 -- 172.21.15.16:0/29657715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba541f77e0 con 0x7fba540b4730 2024-04-25T10:06:03.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.042+0000 7fba5ace1640 1 -- 172.21.15.16:0/29657715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba540750b0 con 0x7fba540b4730 2024-04-25T10:06:03.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.042+0000 7fba517fa640 1 -- 172.21.15.16:0/29657715 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba48044020 con 0x7fba540b4730 2024-04-25T10:06:03.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.043+0000 7fba517fa640 1 --2- 172.21.15.16:0/29657715 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fba4006b4d0 0x7fba4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.043+0000 7fba517fa640 1 -- 172.21.15.16:0/29657715 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fba480aa530 con 0x7fba540b4730 2024-04-25T10:06:03.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.043+0000 7fba58a7d640 1 --2- 172.21.15.16:0/29657715 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fba4006b4d0 0x7fba4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.044+0000 7fba58a7d640 1 --2- 172.21.15.16:0/29657715 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fba4006b4d0 0x7fba4006d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fba4c001560 tx=0x7fba4c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.047+0000 7fba517fa640 1 -- 172.21.15.16:0/29657715 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba4804b030 con 0x7fba540b4730 2024-04-25T10:06:03.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:03.238+0000 7fba5ace1640 1 -- 172.21.15.16:0/29657715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fba5414ed80 con 0x7fba540b4730 2024-04-25T10:06:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:04.899+0000 7fba517fa640 1 -- 172.21.15.16:0/29657715 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7fba48079590 con 0x7fba540b4730 2024-04-25T10:06:04.906 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-04-25T10:06:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:04.902+0000 7fba5ace1640 1 -- 172.21.15.16:0/29657715 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fba4006b4d0 msgr2=0x7fba4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:04.902+0000 7fba5ace1640 1 --2- 172.21.15.16:0/29657715 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fba4006b4d0 0x7fba4006d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fba4c001560 tx=0x7fba4c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:04.902+0000 7fba5ace1640 1 -- 172.21.15.16:0/29657715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba540b4730 msgr2=0x7fba540cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:04.902+0000 7fba5ace1640 1 --2- 172.21.15.16:0/29657715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba540b4730 0x7fba540cade0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fba48035040 tx=0x7fba480046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:04.902+0000 7fba5ace1640 1 -- 172.21.15.16:0/29657715 shutdown_connections 2024-04-25T10:06:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:04.902+0000 7fba5ace1640 1 --2- 172.21.15.16:0/29657715 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fba4006b4d0 0x7fba4006d990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:04.902+0000 7fba5ace1640 1 --2- 172.21.15.16:0/29657715 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba5415ae20 0x7fba5414f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:04.902+0000 7fba5ace1640 1 --2- 172.21.15.16:0/29657715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba540b4730 0x7fba540cade0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:04.902+0000 7fba5ace1640 1 --2- 172.21.15.16:0/29657715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba540aedc0 0x7fba5414e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:04.902+0000 7fba5ace1640 1 -- 172.21.15.16:0/29657715 >> 172.21.15.16:0/29657715 conn(0x7fba5406aba0 msgr2=0x7fba5406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:04.902+0000 7fba5ace1640 1 -- 172.21.15.16:0/29657715 shutdown_connections 2024-04-25T10:06:04.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:04.902+0000 7fba5ace1640 1 -- 172.21.15.16:0/29657715 wait complete. 2024-04-25T10:06:04.923 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:06:05.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.024+0000 7fa9c035c640 1 Processor -- start 2024-04-25T10:06:05.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.024+0000 7fa9c035c640 1 -- start start 2024-04-25T10:06:05.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.025+0000 7fa9c035c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9b80d35b0 0x7fa9b80b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.025+0000 7fa9c035c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9b80b5990 0x7fa9b80b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.025+0000 7fa9c035c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b815e0c0 con 0x7fa9b815dce0 2024-04-25T10:06:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.025+0000 7fa9c035c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b80d2510 con 0x7fa9b80d35b0 2024-04-25T10:06:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.025+0000 7fa9c035c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b80d27f0 con 0x7fa9b80b5990 2024-04-25T10:06:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.025+0000 7fa9bd8f7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9b80d35b0 0x7fa9b80b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.025+0000 7fa9bd8f7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9b80d35b0 0x7fa9b80b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34350/0 (socket says 172.21.15.16:34350) 2024-04-25T10:06:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.025+0000 7fa9bd8f7640 1 -- 172.21.15.16:0/1475456995 learned_addr learned my addr 172.21.15.16:0/1475456995 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.025+0000 7fa9be8f9640 1 --2- 172.21.15.16:0/1475456995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9b80b5990 0x7fa9b80b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.025+0000 7fa9bd8f7640 1 -- 172.21.15.16:0/1475456995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9b80b5990 msgr2=0x7fa9b80b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.025+0000 7fa9bd8f7640 1 --2- 172.21.15.16:0/1475456995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9b80b5990 0x7fa9b80b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.025+0000 7fa9bd8f7640 1 -- 172.21.15.16:0/1475456995 >> v1:172.21.15.186:6789/0 conn(0x7fa9b815dce0 legacy=0x7fa9b80d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.025+0000 7fa9bd8f7640 1 -- 172.21.15.16:0/1475456995 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b80d2ad0 con 0x7fa9b80d35b0 2024-04-25T10:06:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.025+0000 7fa9be8f9640 1 --2- 172.21.15.16:0/1475456995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9b80b5990 0x7fa9b80b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:05.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.026+0000 7fa9bd8f7640 1 --2- 172.21.15.16:0/1475456995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9b80d35b0 0x7fa9b80b5450 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009870 tx=0x7fa9a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221d98d001c1426b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.026+0000 7fa9bd0f6640 1 -- 172.21.15.16:0/1475456995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8037070 con 0x7fa9b80d35b0 2024-04-25T10:06:05.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.026+0000 7fa9bd0f6640 1 -- 172.21.15.16:0/1475456995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a8032440 con 0x7fa9b80d35b0 2024-04-25T10:06:05.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.026+0000 7fa9bd0f6640 1 -- 172.21.15.16:0/1475456995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8032810 con 0x7fa9b80d35b0 2024-04-25T10:06:05.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.027+0000 7fa9c035c640 1 -- 172.21.15.16:0/1475456995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9b80d35b0 msgr2=0x7fa9b80b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.027+0000 7fa9c035c640 1 --2- 172.21.15.16:0/1475456995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9b80d35b0 0x7fa9b80b5450 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009870 tx=0x7fa9a802a270 comp rx=0 tx=0).stop 2024-04-25T10:06:05.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.027+0000 7fa9c035c640 1 -- 172.21.15.16:0/1475456995 shutdown_connections 2024-04-25T10:06:05.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.027+0000 7fa9c035c640 1 --2- 172.21.15.16:0/1475456995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9b80b5990 0x7fa9b80b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.027+0000 7fa9c035c640 1 --2- 172.21.15.16:0/1475456995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9b80d35b0 0x7fa9b80b5450 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.027+0000 7fa9c035c640 1 -- 172.21.15.16:0/1475456995 >> 172.21.15.16:0/1475456995 conn(0x7fa9b8072900 msgr2=0x7fa9b80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:05.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.027+0000 7fa9c035c640 1 -- 172.21.15.16:0/1475456995 shutdown_connections 2024-04-25T10:06:05.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.027+0000 7fa9c035c640 1 -- 172.21.15.16:0/1475456995 wait complete. 2024-04-25T10:06:05.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.028+0000 7fa9c035c640 1 Processor -- start 2024-04-25T10:06:05.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.028+0000 7fa9c035c640 1 -- start start 2024-04-25T10:06:05.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9c035c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9b80b5990 0x7fa9b814fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9c035c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9b80d35b0 0x7fa9b80cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9c035c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9b815dce0 0x7fa9b815e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9c035c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b80d2160 con 0x7fa9b80d35b0 2024-04-25T10:06:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9c035c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b80d3090 con 0x7fa9b80b5990 2024-04-25T10:06:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9c035c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b80c7f20 con 0x7fa9b815dce0 2024-04-25T10:06:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9bd8f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9b80d35b0 0x7fa9b80cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9bd8f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9b80d35b0 0x7fa9b80cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34358/0 (socket says 172.21.15.16:34358) 2024-04-25T10:06:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9bd8f7640 1 -- 172.21.15.16:0/1198851212 learned_addr learned my addr 172.21.15.16:0/1198851212 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9be8f9640 1 --2- 172.21.15.16:0/1198851212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9b815dce0 0x7fa9b815e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9be0f8640 1 --2- 172.21.15.16:0/1198851212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9b80b5990 0x7fa9b814fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9bd8f7640 1 -- 172.21.15.16:0/1198851212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9b815dce0 msgr2=0x7fa9b815e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9bd8f7640 1 --2- 172.21.15.16:0/1198851212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9b815dce0 0x7fa9b815e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9bd8f7640 1 -- 172.21.15.16:0/1198851212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9b80b5990 msgr2=0x7fa9b814fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9bd8f7640 1 --2- 172.21.15.16:0/1198851212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9b80b5990 0x7fa9b814fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9bd8f7640 1 -- 172.21.15.16:0/1198851212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b80c8200 con 0x7fa9b80d35b0 2024-04-25T10:06:05.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9be0f8640 1 --2- 172.21.15.16:0/1198851212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9b80b5990 0x7fa9b814fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:05.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9be8f9640 1 --2- 172.21.15.16:0/1198851212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9b815dce0 0x7fa9b815e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:05.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9bd8f7640 1 --2- 172.21.15.16:0/1198851212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9b80d35b0 0x7fa9b80cccb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa9a802a2b0 tx=0x7fa9a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9aeffd640 1 -- 172.21.15.16:0/1198851212 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8041020 con 0x7fa9b80d35b0 2024-04-25T10:06:05.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9aeffd640 1 -- 172.21.15.16:0/1198851212 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a80306c0 con 0x7fa9b80d35b0 2024-04-25T10:06:05.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.029+0000 7fa9c035c640 1 -- 172.21.15.16:0/1198851212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b80c84e0 con 0x7fa9b80d35b0 2024-04-25T10:06:05.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.030+0000 7fa9aeffd640 1 -- 172.21.15.16:0/1198851212 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8037040 con 0x7fa9b80d35b0 2024-04-25T10:06:05.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.030+0000 7fa9c035c640 1 -- 172.21.15.16:0/1198851212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b80c8aa0 con 0x7fa9b80d35b0 2024-04-25T10:06:05.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.030+0000 7fa9c035c640 1 -- 172.21.15.16:0/1198851212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b8150430 con 0x7fa9b80d35b0 2024-04-25T10:06:05.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.031+0000 7fa9aeffd640 1 -- 172.21.15.16:0/1198851212 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9a8044020 con 0x7fa9b80d35b0 2024-04-25T10:06:05.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.031+0000 7fa9aeffd640 1 --2- 172.21.15.16:0/1198851212 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa99c06b4d0 0x7fa99c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.031+0000 7fa9aeffd640 1 -- 172.21.15.16:0/1198851212 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa9a80aa740 con 0x7fa9b80d35b0 2024-04-25T10:06:05.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.032+0000 7fa9be0f8640 1 --2- 172.21.15.16:0/1198851212 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa99c06b4d0 0x7fa99c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.032+0000 7fa9be0f8640 1 --2- 172.21.15.16:0/1198851212 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa99c06b4d0 0x7fa99c06d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4001660 tx=0x7fa9b4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.036+0000 7fa9aeffd640 1 -- 172.21.15.16:0/1198851212 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9a804b030 con 0x7fa9b80d35b0 2024-04-25T10:06:05.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.285+0000 7fa9c035c640 1 -- 172.21.15.16:0/1198851212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa9b80749b0 con 0x7fa9b80d35b0 2024-04-25T10:06:05.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.286+0000 7fa9aeffd640 1 -- 172.21.15.16:0/1198851212 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v5) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fa9a8079880 con 0x7fa9b80d35b0 2024-04-25T10:06:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:05.292 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:06:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-04-25T10:06:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-04-25T10:06:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmj+xmBBAAaWdHmoiYcXzFnMVQSioT2Q==", 2024-04-25T10:06:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:06:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:06:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:06:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-04-25T10:06:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmUES+DxAAu5It7dQmb7/BM/4x1fUYXA==", 2024-04-25T10:06:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:06:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:06:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:06:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-04-25T10:06:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipm+LtFCxAAGG79kv8eWzly8l5qzQwfww==", 2024-04-25T10:06:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:06:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:06:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:06:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-04-25T10:06:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmqbSSFhAAZtrLsAGSyaMGgv45Hy0WpA==", 2024-04-25T10:06:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:06:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:06:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:06:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-04-25T10:06:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCaKipm2+UdOxAAP8GBunvYsue8EILYV5Uq7A==", 2024-04-25T10:06:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:06:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-04-25T10:06:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCdKipmSv87ORAAigVU7nyzkcXqMEHKsSAsqg==", 2024-04-25T10:06:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:06:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-04-25T10:06:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCgKipm2VMLOBAAJETFfWgq7H/imQAfw93Ruw==", 2024-04-25T10:06:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:06:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-04-25T10:06:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCjKipmb1g0NhAADH2uHQPhW84KLYyWbOjQdw==", 2024-04-25T10:06:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:06:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-04-25T10:06:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCpKipmnH0ROBAAEb8i4nfswYQ/w5mEw48kIg==", 2024-04-25T10:06:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:06:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-04-25T10:06:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCsKipm8HCRNhAA3FhcMlUWxMAiFRXbBWWvNA==", 2024-04-25T10:06:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:06:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-04-25T10:06:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCvKipmfgqpNRAAtPhpSDU9q23Sfa+V7m+3NA==", 2024-04-25T10:06:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:06:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-04-25T10:06:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCyKipm3ndyMxAAeMNODGxNupb043CsA/SP1A==", 2024-04-25T10:06:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:06:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-04-25T10:06:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmAph/GhAAG3x1zC98yHQ13fjpmrlPkg==", 2024-04-25T10:06:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:06:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:06:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:06:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:06:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-04-25T10:06:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmrccTIxAAuDZamFXTe6J2d+HAcBe+DA==", 2024-04-25T10:06:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:06:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:06:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:06:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:06:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-04-25T10:06:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmsFACIBAAFuWsmuQlF58yzIcYUx/6Iw==", 2024-04-25T10:06:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:06:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:06:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:06:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:06:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-04-25T10:06:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmAe1/KBAAXzLkscuwpCwMnrIKQny9+Q==", 2024-04-25T10:06:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:06:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:06:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:06:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:06:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-04-25T10:06:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCYKipmVFQ2JBAAdqG2p/M746PZPMDcpQjwfw==", 2024-04-25T10:06:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:06:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:06:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:06:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mds", 2024-04-25T10:06:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC9Kipma1kLJRAA9JQAYqHjFavgrNYYkTl6fA==", 2024-04-25T10:06:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T10:06:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T10:06:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC9Kipm4GELJRAA3csJVjzPdkUGoVGhFMP2uA==", 2024-04-25T10:06:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T10:06:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-osd", 2024-04-25T10:06:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC9KipmVGkLJRAAkhKig0RBFUJBifFTS4Vp6w==", 2024-04-25T10:06:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T10:06:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T10:06:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC9KipmcHALJRAAWdCA9RBBz3YdoLQoAk05LA==", 2024-04-25T10:06:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T10:06:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T10:06:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC9Kipm5nwLJRAAbrYO8LGblfFZikAyAGs20w==", 2024-04-25T10:06:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T10:06:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T10:06:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC9KipmoIQLJRAAJnLZi+5cIqPLyPu+cIjYfQ==", 2024-04-25T10:06:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T10:06:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-04-25T10:06:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmiPVHARAAGPbv3P3yVj3tlo0b2gmcyw==", 2024-04-25T10:06:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:06:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-04-25T10:06:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-04-25T10:06:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCYKipmghhyOBAAcCm0hQzU+Wlw45yZQETTTg==", 2024-04-25T10:06:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:06:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-04-25T10:06:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-04-25T10:06:05.308 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:06:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.288+0000 7fa9c035c640 1 -- 172.21.15.16:0/1198851212 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa99c06b4d0 msgr2=0x7fa99c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.288+0000 7fa9c035c640 1 --2- 172.21.15.16:0/1198851212 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa99c06b4d0 0x7fa99c06d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4001660 tx=0x7fa9b4009040 comp rx=0 tx=0).stop 2024-04-25T10:06:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.288+0000 7fa9c035c640 1 -- 172.21.15.16:0/1198851212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9b80d35b0 msgr2=0x7fa9b80cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.288+0000 7fa9c035c640 1 --2- 172.21.15.16:0/1198851212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9b80d35b0 0x7fa9b80cccb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa9a802a2b0 tx=0x7fa9a80046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.288+0000 7fa9c035c640 1 -- 172.21.15.16:0/1198851212 shutdown_connections 2024-04-25T10:06:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.288+0000 7fa9c035c640 1 --2- 172.21.15.16:0/1198851212 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa99c06b4d0 0x7fa99c06d990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.288+0000 7fa9c035c640 1 --2- 172.21.15.16:0/1198851212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9b815dce0 0x7fa9b815e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.288+0000 7fa9c035c640 1 --2- 172.21.15.16:0/1198851212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9b80d35b0 0x7fa9b80cccb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.288+0000 7fa9c035c640 1 --2- 172.21.15.16:0/1198851212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9b80b5990 0x7fa9b814fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.288+0000 7fa9c035c640 1 -- 172.21.15.16:0/1198851212 >> 172.21.15.16:0/1198851212 conn(0x7fa9b8072900 msgr2=0x7fa9b80b6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:05.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.288+0000 7fa9c035c640 1 -- 172.21.15.16:0/1198851212 shutdown_connections 2024-04-25T10:06:05.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.288+0000 7fa9c035c640 1 -- 172.21.15.16:0/1198851212 wait complete. 2024-04-25T10:06:05.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T10:06:05.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.406+0000 7f0f91370640 1 Processor -- start 2024-04-25T10:06:05.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.406+0000 7f0f91370640 1 -- start start 2024-04-25T10:06:05.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.407+0000 7f0f91370640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f8c0b3f80 0x7f0f8c162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.407+0000 7f0f91370640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f8c161400 0x7f0f8c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.407+0000 7f0f91370640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f8c0cfed0 con 0x7f0f8c0b35e0 2024-04-25T10:06:05.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.407+0000 7f0f91370640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f8c162890 con 0x7f0f8c0b3f80 2024-04-25T10:06:05.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.407+0000 7f0f91370640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0cf850 con 0x7f0f8c161400 2024-04-25T10:06:05.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.407+0000 7f0f8a7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f8c0b3f80 0x7f0f8c162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.407+0000 7f0f8affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0f8c0b35e0 0x7f0f8c160cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38504/0 (socket says 172.21.15.16:38504) 2024-04-25T10:06:05.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.407+0000 7f0f8affd640 1 -- 172.21.15.16:0/3090968767 learned_addr learned my addr 172.21.15.16:0/3090968767 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:05.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.407+0000 7f0f89ffb640 1 -- 172.21.15.16:0/3090968767 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089185381 0 0) 0x7f0f8c0cfed0 con 0x7f0f8c0b35e0 2024-04-25T10:06:05.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.407+0000 7f0f89ffb640 1 -- 172.21.15.16:0/3090968767 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f78003520 con 0x7f0f8c0b35e0 2024-04-25T10:06:05.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.407+0000 7f0f8b7fe640 1 --2- 172.21.15.16:0/3090968767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f8c161400 0x7f0f8c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.408+0000 7f0f8a7fc640 1 -- 172.21.15.16:0/3090968767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f8c161400 msgr2=0x7f0f8c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:05.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.408+0000 7f0f8a7fc640 1 --2- 172.21.15.16:0/3090968767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f8c161400 0x7f0f8c0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.408+0000 7f0f8a7fc640 1 -- 172.21.15.16:0/3090968767 >> v1:172.21.15.16:6789/0 conn(0x7f0f8c0b35e0 legacy=0x7f0f8c160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.408+0000 7f0f8a7fc640 1 -- 172.21.15.16:0/3090968767 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c0cfb30 con 0x7f0f8c0b3f80 2024-04-25T10:06:05.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.408+0000 7f0f8b7fe640 1 --2- 172.21.15.16:0/3090968767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f8c161400 0x7f0f8c0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:05.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.408+0000 7f0f8a7fc640 1 --2- 172.21.15.16:0/3090968767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8c0b3f80 0x7f0f8c162350 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0f740097a0 tx=0x7f0f7402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0b7fae7da9925c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.408+0000 7f0f89ffb640 1 -- 172.21.15.16:0/3090968767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f74037070 con 0x7f0f8c0b3f80 2024-04-25T10:06:05.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.408+0000 7f0f89ffb640 1 -- 172.21.15.16:0/3090968767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7402ae50 con 0x7f0f8c0b3f80 2024-04-25T10:06:05.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.408+0000 7f0f89ffb640 1 -- 172.21.15.16:0/3090968767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f74032660 con 0x7f0f8c0b3f80 2024-04-25T10:06:05.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.409+0000 7f0f91370640 1 -- 172.21.15.16:0/3090968767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8c0b3f80 msgr2=0x7f0f8c162350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.409+0000 7f0f91370640 1 --2- 172.21.15.16:0/3090968767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8c0b3f80 0x7f0f8c162350 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0f740097a0 tx=0x7f0f7402a100 comp rx=0 tx=0).stop 2024-04-25T10:06:05.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.409+0000 7f0f91370640 1 -- 172.21.15.16:0/3090968767 shutdown_connections 2024-04-25T10:06:05.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.409+0000 7f0f91370640 1 --2- 172.21.15.16:0/3090968767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f8c161400 0x7f0f8c0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.409+0000 7f0f91370640 1 --2- 172.21.15.16:0/3090968767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8c0b3f80 0x7f0f8c162350 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.409+0000 7f0f91370640 1 -- 172.21.15.16:0/3090968767 >> 172.21.15.16:0/3090968767 conn(0x7f0f8c06f4c0 msgr2=0x7f0f8c06f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:05.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.409+0000 7f0f91370640 1 -- 172.21.15.16:0/3090968767 shutdown_connections 2024-04-25T10:06:05.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.409+0000 7f0f91370640 1 -- 172.21.15.16:0/3090968767 wait complete. 2024-04-25T10:06:05.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.410+0000 7f0f91370640 1 Processor -- start 2024-04-25T10:06:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.410+0000 7f0f91370640 1 -- start start 2024-04-25T10:06:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.411+0000 7f0f91370640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f8c0b35e0 0x7f0f8c157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.411+0000 7f0f91370640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8c0b3f80 0x7f0f8c0cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.411+0000 7f0f91370640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c161400 0x7f0f8c0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.411+0000 7f0f91370640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0d05d0 con 0x7f0f8c0b3f80 2024-04-25T10:06:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.411+0000 7f0f91370640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0d0450 con 0x7f0f8c161400 2024-04-25T10:06:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.411+0000 7f0f91370640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0b39c0 con 0x7f0f8c0b35e0 2024-04-25T10:06:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.411+0000 7f0f8a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8c0b3f80 0x7f0f8c0cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.411+0000 7f0f8a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8c0b3f80 0x7f0f8c0cde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34380/0 (socket says 172.21.15.16:34380) 2024-04-25T10:06:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.411+0000 7f0f8a7fc640 1 -- 172.21.15.16:0/1658203217 learned_addr learned my addr 172.21.15.16:0/1658203217 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.411+0000 7f0f8affd640 1 --2- 172.21.15.16:0/1658203217 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f8c0b35e0 0x7f0f8c157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.411+0000 7f0f8b7fe640 1 --2- 172.21.15.16:0/1658203217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c161400 0x7f0f8c0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.411+0000 7f0f8a7fc640 1 -- 172.21.15.16:0/1658203217 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f8c0b35e0 msgr2=0x7f0f8c157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.411+0000 7f0f8a7fc640 1 --2- 172.21.15.16:0/1658203217 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f8c0b35e0 0x7f0f8c157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.411+0000 7f0f8a7fc640 1 -- 172.21.15.16:0/1658203217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c161400 msgr2=0x7f0f8c0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.411+0000 7f0f8a7fc640 1 --2- 172.21.15.16:0/1658203217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c161400 0x7f0f8c0ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.411+0000 7f0f8a7fc640 1 -- 172.21.15.16:0/1658203217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c0b6ab0 con 0x7f0f8c0b3f80 2024-04-25T10:06:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.411+0000 7f0f8affd640 1 --2- 172.21.15.16:0/1658203217 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f8c0b35e0 0x7f0f8c157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.412+0000 7f0f8a7fc640 1 --2- 172.21.15.16:0/1658203217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8c0b3f80 0x7f0f8c0cde40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0f740098c0 tx=0x7f0f740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.412+0000 7f0f837fe640 1 -- 172.21.15.16:0/1658203217 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f74041030 con 0x7f0f8c0b3f80 2024-04-25T10:06:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.412+0000 7f0f837fe640 1 -- 172.21.15.16:0/1658203217 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7402ac50 con 0x7f0f8c0b3f80 2024-04-25T10:06:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.412+0000 7f0f91370640 1 -- 172.21.15.16:0/1658203217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8c0b6210 con 0x7f0f8c0b3f80 2024-04-25T10:06:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.412+0000 7f0f837fe640 1 -- 172.21.15.16:0/1658203217 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f74037040 con 0x7f0f8c0b3f80 2024-04-25T10:06:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.412+0000 7f0f91370640 1 -- 172.21.15.16:0/1658203217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8c0b67d0 con 0x7f0f8c0b3f80 2024-04-25T10:06:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.413+0000 7f0f91370640 1 -- 172.21.15.16:0/1658203217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8c0750b0 con 0x7f0f8c0b3f80 2024-04-25T10:06:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.413+0000 7f0f837fe640 1 -- 172.21.15.16:0/1658203217 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f74044070 con 0x7f0f8c0b3f80 2024-04-25T10:06:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.414+0000 7f0f837fe640 1 --2- 172.21.15.16:0/1658203217 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0f6c06b4d0 0x7f0f6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.414+0000 7f0f837fe640 1 -- 172.21.15.16:0/1658203217 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0f740aa3a0 con 0x7f0f8c0b3f80 2024-04-25T10:06:05.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.414+0000 7f0f8affd640 1 --2- 172.21.15.16:0/1658203217 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0f6c06b4d0 0x7f0f6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.415+0000 7f0f8affd640 1 --2- 172.21.15.16:0/1658203217 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0f6c06b4d0 0x7f0f6c06d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0f84007fd0 tx=0x7f0f84009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.418+0000 7f0f837fe640 1 -- 172.21.15.16:0/1658203217 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f74044e40 con 0x7f0f8c0b3f80 2024-04-25T10:06:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.664+0000 7f0f91370640 1 -- 172.21.15.16:0/1658203217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f0f8c0cce80 con 0x7f0f8c0b3f80 2024-04-25T10:06:05.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.666+0000 7f0f837fe640 1 -- 172.21.15.16:0/1658203217 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0f74079720 con 0x7f0f8c0b3f80 2024-04-25T10:06:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.667+0000 7f0f91370640 1 -- 172.21.15.16:0/1658203217 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0f6c06b4d0 msgr2=0x7f0f6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.667+0000 7f0f91370640 1 --2- 172.21.15.16:0/1658203217 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0f6c06b4d0 0x7f0f6c06d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0f84007fd0 tx=0x7f0f84009040 comp rx=0 tx=0).stop 2024-04-25T10:06:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.667+0000 7f0f91370640 1 -- 172.21.15.16:0/1658203217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8c0b3f80 msgr2=0x7f0f8c0cde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.667+0000 7f0f91370640 1 --2- 172.21.15.16:0/1658203217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8c0b3f80 0x7f0f8c0cde40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0f740098c0 tx=0x7f0f740046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.667+0000 7f0f91370640 1 -- 172.21.15.16:0/1658203217 shutdown_connections 2024-04-25T10:06:05.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.667+0000 7f0f91370640 1 --2- 172.21.15.16:0/1658203217 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0f6c06b4d0 0x7f0f6c06d990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.667+0000 7f0f91370640 1 --2- 172.21.15.16:0/1658203217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f8c161400 0x7f0f8c0ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.667+0000 7f0f91370640 1 --2- 172.21.15.16:0/1658203217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8c0b3f80 0x7f0f8c0cde40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.667+0000 7f0f91370640 1 --2- 172.21.15.16:0/1658203217 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f8c0b35e0 0x7f0f8c157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.667+0000 7f0f91370640 1 -- 172.21.15.16:0/1658203217 >> 172.21.15.16:0/1658203217 conn(0x7f0f8c06f4c0 msgr2=0x7f0f8c074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:05.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.668+0000 7f0f91370640 1 -- 172.21.15.16:0/1658203217 shutdown_connections 2024-04-25T10:06:05.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.668+0000 7f0f91370640 1 -- 172.21.15.16:0/1658203217 wait complete. 2024-04-25T10:06:05.685 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T10:06:05.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.780+0000 7fc90913b640 1 Processor -- start 2024-04-25T10:06:05.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.780+0000 7fc90913b640 1 -- start start 2024-04-25T10:06:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.781+0000 7fc90913b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc904071590 0x7fc9040cc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.781+0000 7fc90913b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc9040cd940 0x7fc9040ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.781+0000 7fc90913b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90406c220 con 0x7fc90415dc50 2024-04-25T10:06:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.781+0000 7fc90913b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9040cfe00 con 0x7fc9040cd940 2024-04-25T10:06:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.781+0000 7fc90913b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9040cec10 con 0x7fc904071590 2024-04-25T10:06:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.781+0000 7fc902d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc90415dc50 0x7fc904070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38520/0 (socket says 172.21.15.16:38520) 2024-04-25T10:06:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.781+0000 7fc902d9d640 1 -- 172.21.15.16:0/605230105 learned_addr learned my addr 172.21.15.16:0/605230105 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.781+0000 7fc901d9b640 1 -- 172.21.15.16:0/605230105 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3520695232 0 0) 0x7fc90406c220 con 0x7fc90415dc50 2024-04-25T10:06:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.781+0000 7fc901d9b640 1 -- 172.21.15.16:0/605230105 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8ec003520 con 0x7fc90415dc50 2024-04-25T10:06:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.781+0000 7fc90359e640 1 --2- 172.21.15.16:0/605230105 >> v2:172.21.15.186:3300/0 conn(0x7fc9040cd940 0x7fc9040ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc90259c640 1 --2- 172.21.15.16:0/605230105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc904071590 0x7fc9040cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc901d9b640 1 -- 172.21.15.16:0/605230105 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3387903595 0 0) 0x7fc8ec003520 con 0x7fc90415dc50 2024-04-25T10:06:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc901d9b640 1 -- 172.21.15.16:0/605230105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc904071590 msgr2=0x7fc9040cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc901d9b640 1 --2- 172.21.15.16:0/605230105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc904071590 0x7fc9040cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc901d9b640 1 -- 172.21.15.16:0/605230105 >> v2:172.21.15.186:3300/0 conn(0x7fc9040cd940 msgr2=0x7fc9040ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc901d9b640 1 --2- 172.21.15.16:0/605230105 >> v2:172.21.15.186:3300/0 conn(0x7fc9040cd940 0x7fc9040ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc901d9b640 1 -- 172.21.15.16:0/605230105 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9040d0050 con 0x7fc90415dc50 2024-04-25T10:06:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc901d9b640 1 -- 172.21.15.16:0/605230105 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fc8fc001670 con 0x7fc90415dc50 2024-04-25T10:06:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc901d9b640 1 -- 172.21.15.16:0/605230105 >> v1:172.21.15.16:6789/0 conn(0x7fc90415dc50 legacy=0x7fc904070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc901d9b640 1 --2- 172.21.15.16:0/605230105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc904071590 0x7fc8ec0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc901d9b640 1 --2- 172.21.15.16:0/605230105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8ec003f30 0x7fc8ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc901d9b640 1 --2- 172.21.15.16:0/605230105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8ec004d40 0x7fc8ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc901d9b640 1 -- 172.21.15.16:0/605230105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9040cfe00 con 0x7fc8ec004d40 2024-04-25T10:06:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc901d9b640 1 -- 172.21.15.16:0/605230105 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9040cec10 con 0x7fc8ec003f30 2024-04-25T10:06:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc901d9b640 1 -- 172.21.15.16:0/605230105 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec00b9c0 con 0x7fc904071590 2024-04-25T10:06:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc90359e640 1 --2- 172.21.15.16:0/605230105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8ec004d40 0x7fc8ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc902d9d640 1 --2- 172.21.15.16:0/605230105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc904071590 0x7fc8ec0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc90259c640 1 --2- 172.21.15.16:0/605230105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8ec003f30 0x7fc8ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc90359e640 1 -- 172.21.15.16:0/605230105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc904071590 msgr2=0x7fc8ec0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:05.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc90359e640 1 --2- 172.21.15.16:0/605230105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc904071590 0x7fc8ec0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc90359e640 1 -- 172.21.15.16:0/605230105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8ec003f30 msgr2=0x7fc8ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:05.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc90359e640 1 --2- 172.21.15.16:0/605230105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8ec003f30 0x7fc8ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.782+0000 7fc90359e640 1 -- 172.21.15.16:0/605230105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ec003520 con 0x7fc8ec004d40 2024-04-25T10:06:05.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.783+0000 7fc90359e640 1 --2- 172.21.15.16:0/605230105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8ec004d40 0x7fc8ec0051c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009fc0 tx=0x7fc8ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8341536a5e93acd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.783+0000 7fc901d9b640 1 -- 172.21.15.16:0/605230105 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4009c10 con 0x7fc8ec004d40 2024-04-25T10:06:05.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.783+0000 7fc901d9b640 1 -- 172.21.15.16:0/605230105 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f4036070 con 0x7fc8ec004d40 2024-04-25T10:06:05.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.783+0000 7fc901d9b640 1 -- 172.21.15.16:0/605230105 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f402d070 con 0x7fc8ec004d40 2024-04-25T10:06:05.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.784+0000 7fc90913b640 1 -- 172.21.15.16:0/605230105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8ec004d40 msgr2=0x7fc8ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.784+0000 7fc90913b640 1 --2- 172.21.15.16:0/605230105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8ec004d40 0x7fc8ec0051c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009fc0 tx=0x7fc8ec005830 comp rx=0 tx=0).stop 2024-04-25T10:06:05.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.784+0000 7fc90359e640 1 -- 172.21.15.16:0/605230105 reap_dead start 2024-04-25T10:06:05.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.784+0000 7fc90913b640 1 -- 172.21.15.16:0/605230105 shutdown_connections 2024-04-25T10:06:05.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.784+0000 7fc90913b640 1 -- 172.21.15.16:0/605230105 >> 172.21.15.16:0/605230105 conn(0x7fc90406f430 msgr2=0x7fc90406f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:05.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.784+0000 7fc90913b640 1 -- 172.21.15.16:0/605230105 shutdown_connections 2024-04-25T10:06:05.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.784+0000 7fc90913b640 1 -- 172.21.15.16:0/605230105 wait complete. 2024-04-25T10:06:05.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.785+0000 7fc90913b640 1 Processor -- start 2024-04-25T10:06:05.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.785+0000 7fc90913b640 1 -- start start 2024-04-25T10:06:05.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.786+0000 7fc90913b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9041576a0 0x7fc9040c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.786+0000 7fc90913b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9041f7db0 0x7fc9040c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.786+0000 7fc90913b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9040c9e80 0x7fc9040ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.786+0000 7fc90913b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9040ce8a0 con 0x7fc9040c9e80 2024-04-25T10:06:05.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.786+0000 7fc90913b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9040cc0c0 con 0x7fc9041f7db0 2024-04-25T10:06:05.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.786+0000 7fc90913b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90415e030 con 0x7fc9041576a0 2024-04-25T10:06:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.786+0000 7fc90359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9040c9e80 0x7fc9040ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.786+0000 7fc90359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9040c9e80 0x7fc9040ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34404/0 (socket says 172.21.15.16:34404) 2024-04-25T10:06:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.786+0000 7fc90359e640 1 -- 172.21.15.16:0/4250832536 learned_addr learned my addr 172.21.15.16:0/4250832536 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.786+0000 7fc902d9d640 1 --2- 172.21.15.16:0/4250832536 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9041576a0 0x7fc9040c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.786+0000 7fc90259c640 1 --2- 172.21.15.16:0/4250832536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9041f7db0 0x7fc9040c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.786+0000 7fc90359e640 1 -- 172.21.15.16:0/4250832536 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9041576a0 msgr2=0x7fc9040c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.786+0000 7fc90359e640 1 --2- 172.21.15.16:0/4250832536 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9041576a0 0x7fc9040c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.786+0000 7fc90359e640 1 -- 172.21.15.16:0/4250832536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9041f7db0 msgr2=0x7fc9040c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.786+0000 7fc90359e640 1 --2- 172.21.15.16:0/4250832536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9041f7db0 0x7fc9040c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.786+0000 7fc90359e640 1 -- 172.21.15.16:0/4250832536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9040c84d0 con 0x7fc9040c9e80 2024-04-25T10:06:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.787+0000 7fc90359e640 1 --2- 172.21.15.16:0/4250832536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9040c9e80 0x7fc9040ca300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4033040 tx=0x7fc8f4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.787+0000 7fc8eb7fe640 1 -- 172.21.15.16:0/4250832536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4009c10 con 0x7fc9040c9e80 2024-04-25T10:06:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.787+0000 7fc8eb7fe640 1 -- 172.21.15.16:0/4250832536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f4003bb0 con 0x7fc9040c9e80 2024-04-25T10:06:05.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.787+0000 7fc90913b640 1 -- 172.21.15.16:0/4250832536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9040c5970 con 0x7fc9040c9e80 2024-04-25T10:06:05.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.787+0000 7fc8eb7fe640 1 -- 172.21.15.16:0/4250832536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4004020 con 0x7fc9040c9e80 2024-04-25T10:06:05.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.787+0000 7fc90913b640 1 -- 172.21.15.16:0/4250832536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9040c5f30 con 0x7fc9040c9e80 2024-04-25T10:06:05.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.788+0000 7fc90913b640 1 -- 172.21.15.16:0/4250832536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9040c5760 con 0x7fc9040c9e80 2024-04-25T10:06:05.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.788+0000 7fc8eb7fe640 1 -- 172.21.15.16:0/4250832536 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8f4005d10 con 0x7fc9040c9e80 2024-04-25T10:06:05.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.789+0000 7fc8eb7fe640 1 --2- 172.21.15.16:0/4250832536 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc8f006b500 0x7fc8f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.789+0000 7fc8eb7fe640 1 -- 172.21.15.16:0/4250832536 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc8f40ab1f0 con 0x7fc9040c9e80 2024-04-25T10:06:05.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.789+0000 7fc902d9d640 1 --2- 172.21.15.16:0/4250832536 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc8f006b500 0x7fc8f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.790+0000 7fc902d9d640 1 --2- 172.21.15.16:0/4250832536 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc8f006b500 0x7fc8f006d9c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc001d30 tx=0x7fc8fc007970 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:05.793+0000 7fc8eb7fe640 1 -- 172.21.15.16:0/4250832536 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc8f404a060 con 0x7fc9040c9e80 2024-04-25T10:06:06.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.042+0000 7fc90913b640 1 -- 172.21.15.16:0/4250832536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fc9040750b0 con 0x7fc9040c9e80 2024-04-25T10:06:06.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.044+0000 7fc8eb7fe640 1 -- 172.21.15.16:0/4250832536 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc8f407a470 con 0x7fc9040c9e80 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.046+0000 7fc90913b640 1 -- 172.21.15.16:0/4250832536 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc8f006b500 msgr2=0x7fc8f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.046+0000 7fc90913b640 1 --2- 172.21.15.16:0/4250832536 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc8f006b500 0x7fc8f006d9c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc001d30 tx=0x7fc8fc007970 comp rx=0 tx=0).stop 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.046+0000 7fc90913b640 1 -- 172.21.15.16:0/4250832536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9040c9e80 msgr2=0x7fc9040ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.046+0000 7fc90913b640 1 --2- 172.21.15.16:0/4250832536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9040c9e80 0x7fc9040ca300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4033040 tx=0x7fc8f4009750 comp rx=0 tx=0).stop 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.046+0000 7fc90913b640 1 -- 172.21.15.16:0/4250832536 shutdown_connections 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.046+0000 7fc90913b640 1 --2- 172.21.15.16:0/4250832536 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc8f006b500 0x7fc8f006d9c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.046+0000 7fc90913b640 1 --2- 172.21.15.16:0/4250832536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9040c9e80 0x7fc9040ca300 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.046+0000 7fc90913b640 1 --2- 172.21.15.16:0/4250832536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9041f7db0 0x7fc9040c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.046+0000 7fc90913b640 1 --2- 172.21.15.16:0/4250832536 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9041576a0 0x7fc9040c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.046+0000 7fc90913b640 1 -- 172.21.15.16:0/4250832536 >> 172.21.15.16:0/4250832536 conn(0x7fc90406f430 msgr2=0x7fc9040cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.046+0000 7fc90913b640 1 -- 172.21.15.16:0/4250832536 shutdown_connections 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.046+0000 7fc90913b640 1 -- 172.21.15.16:0/4250832536 wait complete. 2024-04-25T10:06:06.063 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T10:06:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.158+0000 7f384cc5a640 1 Processor -- start 2024-04-25T10:06:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.158+0000 7f384cc5a640 1 -- start start 2024-04-25T10:06:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.159+0000 7f384cc5a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f384815c090 0x7f38480b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.159+0000 7f384cc5a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f384806aba0 0x7f384806afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.159+0000 7f384cc5a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f384806b4f0 con 0x7f38480d0100 2024-04-25T10:06:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.159+0000 7f384cc5a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384809a1a0 con 0x7f384806aba0 2024-04-25T10:06:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.159+0000 7f384cc5a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38480b3180 con 0x7f384815c090 2024-04-25T10:06:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.160+0000 7f3846d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f384806aba0 0x7f384806afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.160+0000 7f3846d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f384806aba0 0x7f384806afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34412/0 (socket says 172.21.15.16:34412) 2024-04-25T10:06:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.160+0000 7f3846d9d640 1 -- 172.21.15.16:0/1546564842 learned_addr learned my addr 172.21.15.16:0/1546564842 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.160+0000 7f3845d9b640 1 --2- 172.21.15.16:0/1546564842 >> v2:172.21.15.186:3300/0 conn(0x7f384815c090 0x7f38480b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.160+0000 7f3846d9d640 1 -- 172.21.15.16:0/1546564842 >> v2:172.21.15.186:3300/0 conn(0x7f384815c090 msgr2=0x7f38480b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.160+0000 7f3846d9d640 1 --2- 172.21.15.16:0/1546564842 >> v2:172.21.15.186:3300/0 conn(0x7f384815c090 0x7f38480b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.160+0000 7f3846d9d640 1 -- 172.21.15.16:0/1546564842 >> v1:172.21.15.186:6789/0 conn(0x7f38480d0100 legacy=0x7f38480cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.160+0000 7f3846d9d640 1 -- 172.21.15.16:0/1546564842 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384815c5c0 con 0x7f384806aba0 2024-04-25T10:06:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.161+0000 7f3846d9d640 1 --2- 172.21.15.16:0/1546564842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f384806aba0 0x7f384806afb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3848070c90 tx=0x7f384002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4193b18d8a9135 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.161+0000 7f384559a640 1 -- 172.21.15.16:0/1546564842 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840036070 con 0x7f384806aba0 2024-04-25T10:06:06.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.161+0000 7f384559a640 1 -- 172.21.15.16:0/1546564842 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3840031600 con 0x7f384806aba0 2024-04-25T10:06:06.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.161+0000 7f384559a640 1 -- 172.21.15.16:0/1546564842 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38400319d0 con 0x7f384806aba0 2024-04-25T10:06:06.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.161+0000 7f384cc5a640 1 -- 172.21.15.16:0/1546564842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f384806aba0 msgr2=0x7f384806afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.161+0000 7f384cc5a640 1 --2- 172.21.15.16:0/1546564842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f384806aba0 0x7f384806afb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3848070c90 tx=0x7f384002a730 comp rx=0 tx=0).stop 2024-04-25T10:06:06.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.161+0000 7f384cc5a640 1 -- 172.21.15.16:0/1546564842 shutdown_connections 2024-04-25T10:06:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.161+0000 7f384cc5a640 1 --2- 172.21.15.16:0/1546564842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f384806aba0 0x7f384806afb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.162+0000 7f384cc5a640 1 --2- 172.21.15.16:0/1546564842 >> v2:172.21.15.186:3300/0 conn(0x7f384815c090 0x7f38480b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.162+0000 7f384cc5a640 1 -- 172.21.15.16:0/1546564842 >> 172.21.15.16:0/1546564842 conn(0x7f3848099a40 msgr2=0x7f38480b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.162+0000 7f384cc5a640 1 -- 172.21.15.16:0/1546564842 shutdown_connections 2024-04-25T10:06:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.162+0000 7f384cc5a640 1 -- 172.21.15.16:0/1546564842 wait complete. 2024-04-25T10:06:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.163+0000 7f384cc5a640 1 Processor -- start 2024-04-25T10:06:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.163+0000 7f384cc5a640 1 -- start start 2024-04-25T10:06:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.164+0000 7f384cc5a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f384806aba0 0x7f38480c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.164+0000 7f384cc5a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38480d0100 0x7f38480c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.164+0000 7f384cc5a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f384815c090 0x7f38480c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.164+0000 7f384cc5a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384815cb80 con 0x7f38480d0100 2024-04-25T10:06:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.164+0000 7f384cc5a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384815f680 con 0x7f384815c090 2024-04-25T10:06:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.164+0000 7f384cc5a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384815ce60 con 0x7f384806aba0 2024-04-25T10:06:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.164+0000 7f3845d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38480d0100 0x7f38480c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.164+0000 7f3845d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38480d0100 0x7f38480c7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34424/0 (socket says 172.21.15.16:34424) 2024-04-25T10:06:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.164+0000 7f3845d9b640 1 -- 172.21.15.16:0/2483449200 learned_addr learned my addr 172.21.15.16:0/2483449200 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.164+0000 7f384659c640 1 --2- 172.21.15.16:0/2483449200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f384806aba0 0x7f38480c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.164+0000 7f3846d9d640 1 --2- 172.21.15.16:0/2483449200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f384815c090 0x7f38480c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.164+0000 7f3845d9b640 1 -- 172.21.15.16:0/2483449200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f384806aba0 msgr2=0x7f38480c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.164+0000 7f3845d9b640 1 --2- 172.21.15.16:0/2483449200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f384806aba0 0x7f38480c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.164+0000 7f3845d9b640 1 -- 172.21.15.16:0/2483449200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f384815c090 msgr2=0x7f38480c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.164+0000 7f3845d9b640 1 --2- 172.21.15.16:0/2483449200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f384815c090 0x7f38480c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.164+0000 7f3845d9b640 1 -- 172.21.15.16:0/2483449200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38480b4d50 con 0x7f38480d0100 2024-04-25T10:06:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.164+0000 7f3845d9b640 1 --2- 172.21.15.16:0/2483449200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38480d0100 0x7f38480c7a60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f383c00d600 tx=0x7f383c00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.164+0000 7f3836ffd640 1 -- 172.21.15.16:0/2483449200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c0125f0 con 0x7f38480d0100 2024-04-25T10:06:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.165+0000 7f3836ffd640 1 -- 172.21.15.16:0/2483449200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383c019070 con 0x7f38480d0100 2024-04-25T10:06:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.165+0000 7f384cc5a640 1 -- 172.21.15.16:0/2483449200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38480c8840 con 0x7f38480d0100 2024-04-25T10:06:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.165+0000 7f3836ffd640 1 -- 172.21.15.16:0/2483449200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c0129f0 con 0x7f38480d0100 2024-04-25T10:06:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.165+0000 7f384cc5a640 1 -- 172.21.15.16:0/2483449200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38480b52b0 con 0x7f38480d0100 2024-04-25T10:06:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.166+0000 7f384cc5a640 1 -- 172.21.15.16:0/2483449200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38480750b0 con 0x7f38480d0100 2024-04-25T10:06:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.166+0000 7f3836ffd640 1 -- 172.21.15.16:0/2483449200 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f383c012b90 con 0x7f38480d0100 2024-04-25T10:06:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.166+0000 7f3836ffd640 1 --2- 172.21.15.16:0/2483449200 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f382806b4d0 0x7f382806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.167+0000 7f3836ffd640 1 -- 172.21.15.16:0/2483449200 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f383c08a3d0 con 0x7f38480d0100 2024-04-25T10:06:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.167+0000 7f384659c640 1 --2- 172.21.15.16:0/2483449200 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f382806b4d0 0x7f382806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.167+0000 7f384659c640 1 --2- 172.21.15.16:0/2483449200 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f382806b4d0 0x7f382806d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f38380015a0 tx=0x7f3838009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.171+0000 7f3836ffd640 1 -- 172.21.15.16:0/2483449200 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f38480b4d50 con 0x7f38480d0100 2024-04-25T10:06:06.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.421+0000 7f384cc5a640 1 -- 172.21.15.16:0/2483449200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f38480b6700 con 0x7f38480d0100 2024-04-25T10:06:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.424+0000 7f3836ffd640 1 -- 172.21.15.16:0/2483449200 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f383c059500 con 0x7f38480d0100 2024-04-25T10:06:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.426+0000 7f384cc5a640 1 -- 172.21.15.16:0/2483449200 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f382806b4d0 msgr2=0x7f382806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.426+0000 7f384cc5a640 1 --2- 172.21.15.16:0/2483449200 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f382806b4d0 0x7f382806d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f38380015a0 tx=0x7f3838009040 comp rx=0 tx=0).stop 2024-04-25T10:06:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.426+0000 7f384cc5a640 1 -- 172.21.15.16:0/2483449200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38480d0100 msgr2=0x7f38480c7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.426+0000 7f384cc5a640 1 --2- 172.21.15.16:0/2483449200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38480d0100 0x7f38480c7a60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f383c00d600 tx=0x7f383c00dac0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.426+0000 7f384cc5a640 1 -- 172.21.15.16:0/2483449200 shutdown_connections 2024-04-25T10:06:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.426+0000 7f384cc5a640 1 --2- 172.21.15.16:0/2483449200 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f382806b4d0 0x7f382806d990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.426+0000 7f384cc5a640 1 --2- 172.21.15.16:0/2483449200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f384815c090 0x7f38480c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.426+0000 7f384cc5a640 1 --2- 172.21.15.16:0/2483449200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38480d0100 0x7f38480c7a60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.426+0000 7f384cc5a640 1 --2- 172.21.15.16:0/2483449200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f384806aba0 0x7f38480c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.426+0000 7f384cc5a640 1 -- 172.21.15.16:0/2483449200 >> 172.21.15.16:0/2483449200 conn(0x7f3848099a40 msgr2=0x7f38480b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.426+0000 7f384cc5a640 1 -- 172.21.15.16:0/2483449200 shutdown_connections 2024-04-25T10:06:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.426+0000 7f384cc5a640 1 -- 172.21.15.16:0/2483449200 wait complete. 2024-04-25T10:06:06.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T10:06:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.538+0000 7fdcfa5c1640 1 Processor -- start 2024-04-25T10:06:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.538+0000 7fdcfa5c1640 1 -- start start 2024-04-25T10:06:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.539+0000 7fdcfa5c1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcf41610b0 0x7fdcf40b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.539+0000 7fdcfa5c1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdcf40cef40 0x7fdcf40d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.539+0000 7fdcfa5c1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf4162040 con 0x7fdcf41618d0 2024-04-25T10:06:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.539+0000 7fdcfa5c1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf40d1ea0 con 0x7fdcf40cef40 2024-04-25T10:06:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.539+0000 7fdcfa5c1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf4162320 con 0x7fdcf41610b0 2024-04-25T10:06:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.539+0000 7fdcf8b5e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdcf40cef40 0x7fdcf40d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.539+0000 7fdcf37fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdcf41618d0 0x7fdcf40ce830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38530/0 (socket says 172.21.15.16:38530) 2024-04-25T10:06:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.539+0000 7fdcf37fe640 1 -- 172.21.15.16:0/1895832873 learned_addr learned my addr 172.21.15.16:0/1895832873 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.539+0000 7fdcf8b5e640 1 -- 172.21.15.16:0/1895832873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcf41610b0 msgr2=0x7fdcf40b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.539+0000 7fdcf3fff640 1 --2- 172.21.15.16:0/1895832873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcf41610b0 0x7fdcf40b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.539+0000 7fdcf8b5e640 1 --2- 172.21.15.16:0/1895832873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcf41610b0 0x7fdcf40b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.539+0000 7fdcf8b5e640 1 -- 172.21.15.16:0/1895832873 >> v1:172.21.15.16:6789/0 conn(0x7fdcf41618d0 legacy=0x7fdcf40ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.539+0000 7fdcf8b5e640 1 -- 172.21.15.16:0/1895832873 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf4162600 con 0x7fdcf40cef40 2024-04-25T10:06:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.539+0000 7fdcf2ffd640 1 -- 172.21.15.16:0/1895832873 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983425574 0 0) 0x7fdcf4162040 con 0x7fdcf41618d0 2024-04-25T10:06:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.539+0000 7fdcf3fff640 1 --2- 172.21.15.16:0/1895832873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcf41610b0 0x7fdcf40b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.540+0000 7fdcf8b5e640 1 --2- 172.21.15.16:0/1895832873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf40cef40 0x7fdcf40d2610 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009d90 tx=0x7fdcec02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6731eb435fd2fc76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.540+0000 7fdcf2ffd640 1 -- 172.21.15.16:0/1895832873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec031600 con 0x7fdcf40cef40 2024-04-25T10:06:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.540+0000 7fdcf2ffd640 1 -- 172.21.15.16:0/1895832873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec0317a0 con 0x7fdcf40cef40 2024-04-25T10:06:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.540+0000 7fdcf2ffd640 1 -- 172.21.15.16:0/1895832873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec031b70 con 0x7fdcf40cef40 2024-04-25T10:06:06.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.541+0000 7fdcfa5c1640 1 -- 172.21.15.16:0/1895832873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf40cef40 msgr2=0x7fdcf40d2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.541+0000 7fdcfa5c1640 1 --2- 172.21.15.16:0/1895832873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf40cef40 0x7fdcf40d2610 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009d90 tx=0x7fdcec02a5d0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.541+0000 7fdcfa5c1640 1 -- 172.21.15.16:0/1895832873 shutdown_connections 2024-04-25T10:06:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.541+0000 7fdcfa5c1640 1 --2- 172.21.15.16:0/1895832873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf40cef40 0x7fdcf40d2610 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.541+0000 7fdcfa5c1640 1 --2- 172.21.15.16:0/1895832873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcf41610b0 0x7fdcf40b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.541+0000 7fdcfa5c1640 1 -- 172.21.15.16:0/1895832873 >> 172.21.15.16:0/1895832873 conn(0x7fdcf40b5570 msgr2=0x7fdcf40b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.541+0000 7fdcfa5c1640 1 -- 172.21.15.16:0/1895832873 shutdown_connections 2024-04-25T10:06:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.541+0000 7fdcfa5c1640 1 -- 172.21.15.16:0/1895832873 wait complete. 2024-04-25T10:06:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.542+0000 7fdcfa5c1640 1 Processor -- start 2024-04-25T10:06:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.542+0000 7fdcfa5c1640 1 -- start start 2024-04-25T10:06:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.543+0000 7fdcfa5c1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcf40cef40 0x7fdcf41f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.543+0000 7fdcfa5c1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf41610b0 0x7fdcf4152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.543+0000 7fdcfa5c1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcf41618d0 0x7fdcf415f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.543+0000 7fdcfa5c1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf4161d10 con 0x7fdcf41610b0 2024-04-25T10:06:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.543+0000 7fdcfa5c1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40b6c00 con 0x7fdcf41618d0 2024-04-25T10:06:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.543+0000 7fdcfa5c1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf4161490 con 0x7fdcf40cef40 2024-04-25T10:06:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.543+0000 7fdcf37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf41610b0 0x7fdcf4152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.543+0000 7fdcf37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf41610b0 0x7fdcf4152dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34444/0 (socket says 172.21.15.16:34444) 2024-04-25T10:06:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.543+0000 7fdcf37fe640 1 -- 172.21.15.16:0/2744164965 learned_addr learned my addr 172.21.15.16:0/2744164965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.543+0000 7fdcf3fff640 1 --2- 172.21.15.16:0/2744164965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcf40cef40 0x7fdcf41f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.543+0000 7fdcf8b5e640 1 --2- 172.21.15.16:0/2744164965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcf41618d0 0x7fdcf415f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.543+0000 7fdcf37fe640 1 -- 172.21.15.16:0/2744164965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcf40cef40 msgr2=0x7fdcf41f6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.543+0000 7fdcf37fe640 1 --2- 172.21.15.16:0/2744164965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcf40cef40 0x7fdcf41f6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.543+0000 7fdcf37fe640 1 -- 172.21.15.16:0/2744164965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcf41618d0 msgr2=0x7fdcf415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.543+0000 7fdcf37fe640 1 --2- 172.21.15.16:0/2744164965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcf41618d0 0x7fdcf415f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.543+0000 7fdcf37fe640 1 -- 172.21.15.16:0/2744164965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf415c4e0 con 0x7fdcf41610b0 2024-04-25T10:06:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.543+0000 7fdcf8b5e640 1 --2- 172.21.15.16:0/2744164965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcf41618d0 0x7fdcf415f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.543+0000 7fdcf37fe640 1 --2- 172.21.15.16:0/2744164965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf41610b0 0x7fdcf4152dc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdce400c400 tx=0x7fdce400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.544+0000 7fdcf0ff9640 1 -- 172.21.15.16:0/2744164965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4017070 con 0x7fdcf41610b0 2024-04-25T10:06:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.544+0000 7fdcf0ff9640 1 -- 172.21.15.16:0/2744164965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce4005ce0 con 0x7fdcf41610b0 2024-04-25T10:06:06.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.544+0000 7fdcfa5c1640 1 -- 172.21.15.16:0/2744164965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf415c7c0 con 0x7fdcf41610b0 2024-04-25T10:06:06.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.544+0000 7fdcf0ff9640 1 -- 172.21.15.16:0/2744164965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4007310 con 0x7fdcf41610b0 2024-04-25T10:06:06.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.544+0000 7fdcfa5c1640 1 -- 172.21.15.16:0/2744164965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf415cd80 con 0x7fdcf41610b0 2024-04-25T10:06:06.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.545+0000 7fdcfa5c1640 1 -- 172.21.15.16:0/2744164965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf40750b0 con 0x7fdcf41610b0 2024-04-25T10:06:06.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.545+0000 7fdcf0ff9640 1 -- 172.21.15.16:0/2744164965 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdce401a070 con 0x7fdcf41610b0 2024-04-25T10:06:06.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.545+0000 7fdcf0ff9640 1 --2- 172.21.15.16:0/2744164965 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdcd406b4d0 0x7fdcd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.546+0000 7fdcf0ff9640 1 -- 172.21.15.16:0/2744164965 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdce4089260 con 0x7fdcf41610b0 2024-04-25T10:06:06.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.546+0000 7fdcf3fff640 1 --2- 172.21.15.16:0/2744164965 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdcd406b4d0 0x7fdcd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.546+0000 7fdcf3fff640 1 --2- 172.21.15.16:0/2744164965 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdcd406b4d0 0x7fdcd406d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00a990 tx=0x7fdcdc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.550+0000 7fdcf0ff9640 1 -- 172.21.15.16:0/2744164965 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdce40584c0 con 0x7fdcf41610b0 2024-04-25T10:06:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.798+0000 7fdcfa5c1640 1 -- 172.21.15.16:0/2744164965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fdcf40d1770 con 0x7fdcf41610b0 2024-04-25T10:06:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.800+0000 7fdcf0ff9640 1 -- 172.21.15.16:0/2744164965 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fdce4057dc0 con 0x7fdcf41610b0 2024-04-25T10:06:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.802+0000 7fdcfa5c1640 1 -- 172.21.15.16:0/2744164965 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdcd406b4d0 msgr2=0x7fdcd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.802+0000 7fdcfa5c1640 1 --2- 172.21.15.16:0/2744164965 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdcd406b4d0 0x7fdcd406d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00a990 tx=0x7fdcdc008040 comp rx=0 tx=0).stop 2024-04-25T10:06:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.802+0000 7fdcfa5c1640 1 -- 172.21.15.16:0/2744164965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf41610b0 msgr2=0x7fdcf4152dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.802+0000 7fdcfa5c1640 1 --2- 172.21.15.16:0/2744164965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf41610b0 0x7fdcf4152dc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdce400c400 tx=0x7fdce400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.802+0000 7fdcfa5c1640 1 -- 172.21.15.16:0/2744164965 shutdown_connections 2024-04-25T10:06:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.802+0000 7fdcfa5c1640 1 --2- 172.21.15.16:0/2744164965 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdcd406b4d0 0x7fdcd406d990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.802+0000 7fdcfa5c1640 1 --2- 172.21.15.16:0/2744164965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcf41618d0 0x7fdcf415f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.802+0000 7fdcfa5c1640 1 --2- 172.21.15.16:0/2744164965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf41610b0 0x7fdcf4152dc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.802+0000 7fdcfa5c1640 1 --2- 172.21.15.16:0/2744164965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcf40cef40 0x7fdcf41f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.802+0000 7fdcfa5c1640 1 -- 172.21.15.16:0/2744164965 >> 172.21.15.16:0/2744164965 conn(0x7fdcf40b5570 msgr2=0x7fdcf40b7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.802+0000 7fdcfa5c1640 1 -- 172.21.15.16:0/2744164965 shutdown_connections 2024-04-25T10:06:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.802+0000 7fdcfa5c1640 1 -- 172.21.15.16:0/2744164965 wait complete. 2024-04-25T10:06:06.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T10:06:06.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.915+0000 7f5ba5916640 1 Processor -- start 2024-04-25T10:06:06.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.915+0000 7f5ba5916640 1 -- start start 2024-04-25T10:06:06.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.916+0000 7f5ba5916640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ba0161090 0x7f5ba0161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.916+0000 7f5ba5916640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5ba00b9720 0x7f5ba00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.916+0000 7f5ba5916640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba0161f80 con 0x7f5ba00cece0 2024-04-25T10:06:06.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.916+0000 7f5ba5916640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba00cf130 con 0x7f5ba00b9720 2024-04-25T10:06:06.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.916+0000 7f5ba5916640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba00d21a0 con 0x7f5ba0161090 2024-04-25T10:06:06.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.916+0000 7f5b9effd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ba0161090 0x7f5ba0161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.916+0000 7f5b9effd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ba0161090 0x7f5ba0161730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:43032/0 (socket says 172.21.15.16:43032) 2024-04-25T10:06:06.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.916+0000 7f5b9effd640 1 -- 172.21.15.16:0/1506336292 learned_addr learned my addr 172.21.15.16:0/1506336292 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:06.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.916+0000 7f5b9e7fc640 1 --2- 172.21.15.16:0/1506336292 >> v2:172.21.15.186:3300/0 conn(0x7f5ba00b9720 0x7f5ba00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.917+0000 7f5b9effd640 1 -- 172.21.15.16:0/1506336292 >> v2:172.21.15.186:3300/0 conn(0x7f5ba00b9720 msgr2=0x7f5ba00ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.917+0000 7f5b9effd640 1 --2- 172.21.15.16:0/1506336292 >> v2:172.21.15.186:3300/0 conn(0x7f5ba00b9720 0x7f5ba00ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.917+0000 7f5b9effd640 1 -- 172.21.15.16:0/1506336292 >> v1:172.21.15.186:6789/0 conn(0x7f5ba00cece0 legacy=0x7f5ba006c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.917+0000 7f5b9effd640 1 -- 172.21.15.16:0/1506336292 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba0162260 con 0x7f5ba0161090 2024-04-25T10:06:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.917+0000 7f5b9e7fc640 1 --2- 172.21.15.16:0/1506336292 >> v2:172.21.15.186:3300/0 conn(0x7f5ba00b9720 0x7f5ba00ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.917+0000 7f5b9effd640 1 --2- 172.21.15.16:0/1506336292 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ba0161090 0x7f5ba0161730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5b90009870 tx=0x7f5b9002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb6035a785479a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.918+0000 7f5b9dffb640 1 -- 172.21.15.16:0/1506336292 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90037070 con 0x7f5ba0161090 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.918+0000 7f5b9dffb640 1 -- 172.21.15.16:0/1506336292 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b90032440 con 0x7f5ba0161090 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.918+0000 7f5b9dffb640 1 -- 172.21.15.16:0/1506336292 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90032810 con 0x7f5ba0161090 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.918+0000 7f5ba5916640 1 -- 172.21.15.16:0/1506336292 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ba0161090 msgr2=0x7f5ba0161730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.918+0000 7f5ba5916640 1 --2- 172.21.15.16:0/1506336292 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ba0161090 0x7f5ba0161730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5b90009870 tx=0x7f5b9002a270 comp rx=0 tx=0).stop 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.918+0000 7f5ba5916640 1 -- 172.21.15.16:0/1506336292 shutdown_connections 2024-04-25T10:06:06.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.918+0000 7f5ba5916640 1 --2- 172.21.15.16:0/1506336292 >> v2:172.21.15.186:3300/0 conn(0x7f5ba00b9720 0x7f5ba00ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.918+0000 7f5ba5916640 1 --2- 172.21.15.16:0/1506336292 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ba0161090 0x7f5ba0161730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.918+0000 7f5ba5916640 1 -- 172.21.15.16:0/1506336292 >> 172.21.15.16:0/1506336292 conn(0x7f5ba00b55e0 msgr2=0x7f5ba00b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.918+0000 7f5ba5916640 1 -- 172.21.15.16:0/1506336292 shutdown_connections 2024-04-25T10:06:06.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.918+0000 7f5ba5916640 1 -- 172.21.15.16:0/1506336292 wait complete. 2024-04-25T10:06:06.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.919+0000 7f5ba5916640 1 Processor -- start 2024-04-25T10:06:06.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.920+0000 7f5ba5916640 1 -- start start 2024-04-25T10:06:06.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.920+0000 7f5ba5916640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ba00b9720 0x7f5ba00cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.920+0000 7f5ba5916640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ba00cece0 0x7f5ba00cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.920+0000 7f5ba5916640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ba0161090 0x7f5ba01f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.920+0000 7f5ba5916640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba0161470 con 0x7f5ba0161090 2024-04-25T10:06:06.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.920+0000 7f5ba5916640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba00d1e30 con 0x7f5ba00b9720 2024-04-25T10:06:06.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.920+0000 7f5ba5916640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba0161c70 con 0x7f5ba00cece0 2024-04-25T10:06:06.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.920+0000 7f5b9f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ba0161090 0x7f5ba01f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.920+0000 7f5b9f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ba0161090 0x7f5ba01f3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34454/0 (socket says 172.21.15.16:34454) 2024-04-25T10:06:06.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.920+0000 7f5b9f7fe640 1 -- 172.21.15.16:0/4164127802 learned_addr learned my addr 172.21.15.16:0/4164127802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:06.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.921+0000 7f5b9e7fc640 1 --2- 172.21.15.16:0/4164127802 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ba00cece0 0x7f5ba00cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.921+0000 7f5b9effd640 1 --2- 172.21.15.16:0/4164127802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ba00b9720 0x7f5ba00cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.921+0000 7f5b9f7fe640 1 -- 172.21.15.16:0/4164127802 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ba00cece0 msgr2=0x7f5ba00cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.921+0000 7f5b9f7fe640 1 --2- 172.21.15.16:0/4164127802 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ba00cece0 0x7f5ba00cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.921+0000 7f5b9f7fe640 1 -- 172.21.15.16:0/4164127802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ba00b9720 msgr2=0x7f5ba00cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.921+0000 7f5b9f7fe640 1 --2- 172.21.15.16:0/4164127802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ba00b9720 0x7f5ba00cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.921+0000 7f5b9f7fe640 1 -- 172.21.15.16:0/4164127802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba00ccf50 con 0x7f5ba0161090 2024-04-25T10:06:06.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.921+0000 7f5b9f7fe640 1 --2- 172.21.15.16:0/4164127802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ba0161090 0x7f5ba01f3420 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5b9800c7a0 tx=0x7f5b9800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.921+0000 7f5b87fff640 1 -- 172.21.15.16:0/4164127802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b98003b70 con 0x7f5ba0161090 2024-04-25T10:06:06.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.921+0000 7f5b87fff640 1 -- 172.21.15.16:0/4164127802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b98003d30 con 0x7f5ba0161090 2024-04-25T10:06:06.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.921+0000 7f5ba5916640 1 -- 172.21.15.16:0/4164127802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba00cd230 con 0x7f5ba0161090 2024-04-25T10:06:06.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.921+0000 7f5b87fff640 1 -- 172.21.15.16:0/4164127802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9800aa90 con 0x7f5ba0161090 2024-04-25T10:06:06.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.921+0000 7f5ba5916640 1 -- 172.21.15.16:0/4164127802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba015dc40 con 0x7f5ba0161090 2024-04-25T10:06:06.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.922+0000 7f5ba5916640 1 -- 172.21.15.16:0/4164127802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba00750b0 con 0x7f5ba0161090 2024-04-25T10:06:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.923+0000 7f5b87fff640 1 -- 172.21.15.16:0/4164127802 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b9800ac30 con 0x7f5ba0161090 2024-04-25T10:06:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.923+0000 7f5b87fff640 1 --2- 172.21.15.16:0/4164127802 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5b8c06b4d0 0x7f5b8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.923+0000 7f5b87fff640 1 -- 172.21.15.16:0/4164127802 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5b98089c60 con 0x7f5ba0161090 2024-04-25T10:06:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.924+0000 7f5b9effd640 1 --2- 172.21.15.16:0/4164127802 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5b8c06b4d0 0x7f5b8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.924+0000 7f5b9effd640 1 --2- 172.21.15.16:0/4164127802 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5b8c06b4d0 0x7f5b8c06d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5b90034000 tx=0x7f5b9002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:06.928+0000 7f5b87fff640 1 -- 172.21.15.16:0/4164127802 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b980587f0 con 0x7f5ba0161090 2024-04-25T10:06:06.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:06.955 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:07.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.176+0000 7f5ba5916640 1 -- 172.21.15.16:0/4164127802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f5ba00ccd80 con 0x7f5ba0161090 2024-04-25T10:06:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.178+0000 7f5b87fff640 1 -- 172.21.15.16:0/4164127802 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5b9805c890 con 0x7f5ba0161090 2024-04-25T10:06:07.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.180+0000 7f5ba5916640 1 -- 172.21.15.16:0/4164127802 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5b8c06b4d0 msgr2=0x7f5b8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.180+0000 7f5ba5916640 1 --2- 172.21.15.16:0/4164127802 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5b8c06b4d0 0x7f5b8c06d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5b90034000 tx=0x7f5b9002e000 comp rx=0 tx=0).stop 2024-04-25T10:06:07.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.180+0000 7f5ba5916640 1 -- 172.21.15.16:0/4164127802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ba0161090 msgr2=0x7f5ba01f3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.180+0000 7f5ba5916640 1 --2- 172.21.15.16:0/4164127802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ba0161090 0x7f5ba01f3420 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5b9800c7a0 tx=0x7f5b9800cc60 comp rx=0 tx=0).stop 2024-04-25T10:06:07.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.180+0000 7f5ba5916640 1 -- 172.21.15.16:0/4164127802 shutdown_connections 2024-04-25T10:06:07.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.180+0000 7f5ba5916640 1 --2- 172.21.15.16:0/4164127802 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5b8c06b4d0 0x7f5b8c06d990 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.180+0000 7f5ba5916640 1 --2- 172.21.15.16:0/4164127802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ba0161090 0x7f5ba01f3420 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.180+0000 7f5ba5916640 1 --2- 172.21.15.16:0/4164127802 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ba00cece0 0x7f5ba00cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.180+0000 7f5ba5916640 1 --2- 172.21.15.16:0/4164127802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ba00b9720 0x7f5ba00cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.180+0000 7f5ba5916640 1 -- 172.21.15.16:0/4164127802 >> 172.21.15.16:0/4164127802 conn(0x7f5ba00b55e0 msgr2=0x7f5ba00b6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.181+0000 7f5ba5916640 1 -- 172.21.15.16:0/4164127802 shutdown_connections 2024-04-25T10:06:07.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.181+0000 7f5ba5916640 1 -- 172.21.15.16:0/4164127802 wait complete. 2024-04-25T10:06:07.198 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T10:06:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.293+0000 7f911e29e640 1 Processor -- start 2024-04-25T10:06:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.293+0000 7f911e29e640 1 -- start start 2024-04-25T10:06:07.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.294+0000 7f911e29e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9118161090 0x7f9118161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.294+0000 7f911e29e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f91180b9720 0x7f91180ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.294+0000 7f911e29e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91180cf0f0 con 0x7f91180cece0 2024-04-25T10:06:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.294+0000 7f911e29e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91180d21a0 con 0x7f9118161090 2024-04-25T10:06:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.294+0000 7f911e29e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9118161f80 con 0x7f91180b9720 2024-04-25T10:06:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.294+0000 7f9117fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9118161090 0x7f9118161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.294+0000 7f9117fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9118161090 0x7f9118161730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34458/0 (socket says 172.21.15.16:34458) 2024-04-25T10:06:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.294+0000 7f9117fff640 1 -- 172.21.15.16:0/1893392803 learned_addr learned my addr 172.21.15.16:0/1893392803 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.295+0000 7f91177fe640 1 --2- 172.21.15.16:0/1893392803 >> v2:172.21.15.186:3300/0 conn(0x7f91180b9720 0x7f91180ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.295+0000 7f9117fff640 1 -- 172.21.15.16:0/1893392803 >> v2:172.21.15.186:3300/0 conn(0x7f91180b9720 msgr2=0x7f91180ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.295+0000 7f9117fff640 1 --2- 172.21.15.16:0/1893392803 >> v2:172.21.15.186:3300/0 conn(0x7f91180b9720 0x7f91180ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.295+0000 7f9117fff640 1 -- 172.21.15.16:0/1893392803 >> v1:172.21.15.186:6789/0 conn(0x7f91180cece0 legacy=0x7f911806c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.295+0000 7f9117fff640 1 -- 172.21.15.16:0/1893392803 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9118162260 con 0x7f9118161090 2024-04-25T10:06:07.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.295+0000 7f9117fff640 1 --2- 172.21.15.16:0/1893392803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9118161090 0x7f9118161730 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f911808cf10 tx=0x7f910802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b451ca48fa2a9a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.296+0000 7f9116ffd640 1 -- 172.21.15.16:0/1893392803 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108037070 con 0x7f9118161090 2024-04-25T10:06:07.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.296+0000 7f9116ffd640 1 -- 172.21.15.16:0/1893392803 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9108032440 con 0x7f9118161090 2024-04-25T10:06:07.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.296+0000 7f9116ffd640 1 -- 172.21.15.16:0/1893392803 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108032810 con 0x7f9118161090 2024-04-25T10:06:07.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.296+0000 7f911e29e640 1 -- 172.21.15.16:0/1893392803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9118161090 msgr2=0x7f9118161730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.296+0000 7f911e29e640 1 --2- 172.21.15.16:0/1893392803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9118161090 0x7f9118161730 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f911808cf10 tx=0x7f910802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.296+0000 7f911e29e640 1 -- 172.21.15.16:0/1893392803 shutdown_connections 2024-04-25T10:06:07.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.296+0000 7f911e29e640 1 --2- 172.21.15.16:0/1893392803 >> v2:172.21.15.186:3300/0 conn(0x7f91180b9720 0x7f91180ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.296+0000 7f911e29e640 1 --2- 172.21.15.16:0/1893392803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9118161090 0x7f9118161730 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.296+0000 7f911e29e640 1 -- 172.21.15.16:0/1893392803 >> 172.21.15.16:0/1893392803 conn(0x7f91180b55e0 msgr2=0x7f91180b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.296+0000 7f911e29e640 1 -- 172.21.15.16:0/1893392803 shutdown_connections 2024-04-25T10:06:07.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.296+0000 7f911e29e640 1 -- 172.21.15.16:0/1893392803 wait complete. 2024-04-25T10:06:07.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.297+0000 7f911e29e640 1 Processor -- start 2024-04-25T10:06:07.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.298+0000 7f911e29e640 1 -- start start 2024-04-25T10:06:07.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.298+0000 7f911e29e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91180b9720 0x7f9118159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.298+0000 7f911e29e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91180cece0 0x7f91180cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.298+0000 7f911e29e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9118161090 0x7f91181f3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.298+0000 7f911e29e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9118161470 con 0x7f91180cece0 2024-04-25T10:06:07.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.298+0000 7f911e29e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91180d1e30 con 0x7f91180b9720 2024-04-25T10:06:07.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.298+0000 7f911e29e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9118161c70 con 0x7f9118161090 2024-04-25T10:06:07.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.298+0000 7f91177fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91180cece0 0x7f91180cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.298+0000 7f91177fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91180cece0 0x7f91180cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34474/0 (socket says 172.21.15.16:34474) 2024-04-25T10:06:07.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.298+0000 7f91177fe640 1 -- 172.21.15.16:0/851517753 learned_addr learned my addr 172.21.15.16:0/851517753 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:07.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.298+0000 7f9117fff640 1 --2- 172.21.15.16:0/851517753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91180b9720 0x7f9118159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.298+0000 7f911c83b640 1 --2- 172.21.15.16:0/851517753 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9118161090 0x7f91181f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.299+0000 7f91177fe640 1 -- 172.21.15.16:0/851517753 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9118161090 msgr2=0x7f91181f3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.299+0000 7f91177fe640 1 --2- 172.21.15.16:0/851517753 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9118161090 0x7f91181f3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.299+0000 7f91177fe640 1 -- 172.21.15.16:0/851517753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91180b9720 msgr2=0x7f9118159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.299+0000 7f91177fe640 1 --2- 172.21.15.16:0/851517753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91180b9720 0x7f9118159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.299+0000 7f91177fe640 1 -- 172.21.15.16:0/851517753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91180cc6a0 con 0x7f91180cece0 2024-04-25T10:06:07.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.299+0000 7f91177fe640 1 --2- 172.21.15.16:0/851517753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91180cece0 0x7f91180cce60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f910c00d800 tx=0x7f910c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.299+0000 7f9114ff9640 1 -- 172.21.15.16:0/851517753 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c00b6c0 con 0x7f91180cece0 2024-04-25T10:06:07.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.299+0000 7f9114ff9640 1 -- 172.21.15.16:0/851517753 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910c019070 con 0x7f91180cece0 2024-04-25T10:06:07.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.299+0000 7f9114ff9640 1 -- 172.21.15.16:0/851517753 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c00ba80 con 0x7f91180cece0 2024-04-25T10:06:07.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.299+0000 7f911e29e640 1 -- 172.21.15.16:0/851517753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911815de50 con 0x7f91180cece0 2024-04-25T10:06:07.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.299+0000 7f911e29e640 1 -- 172.21.15.16:0/851517753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91180b5020 con 0x7f91180cece0 2024-04-25T10:06:07.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.300+0000 7f911e29e640 1 -- 172.21.15.16:0/851517753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91180ccab0 con 0x7f91180cece0 2024-04-25T10:06:07.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.301+0000 7f9114ff9640 1 -- 172.21.15.16:0/851517753 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f910c00bc20 con 0x7f91180cece0 2024-04-25T10:06:07.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.301+0000 7f9114ff9640 1 --2- 172.21.15.16:0/851517753 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f910406b4d0 0x7f910406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.301+0000 7f9114ff9640 1 -- 172.21.15.16:0/851517753 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f910c08a180 con 0x7f91180cece0 2024-04-25T10:06:07.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.301+0000 7f9117fff640 1 --2- 172.21.15.16:0/851517753 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f910406b4d0 0x7f910406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.302+0000 7f9117fff640 1 --2- 172.21.15.16:0/851517753 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f910406b4d0 0x7f910406d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9108034000 tx=0x7f910802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.305+0000 7f9114ff9640 1 -- 172.21.15.16:0/851517753 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f91180ccab0 con 0x7f91180cece0 2024-04-25T10:06:07.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.558+0000 7f911e29e640 1 -- 172.21.15.16:0/851517753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f91180750b0 con 0x7f91180cece0 2024-04-25T10:06:07.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.560+0000 7f9114ff9640 1 -- 172.21.15.16:0/851517753 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7f910c088e20 con 0x7f91180cece0 2024-04-25T10:06:07.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.562+0000 7f911e29e640 1 -- 172.21.15.16:0/851517753 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f910406b4d0 msgr2=0x7f910406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.562+0000 7f911e29e640 1 --2- 172.21.15.16:0/851517753 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f910406b4d0 0x7f910406d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9108034000 tx=0x7f910802e000 comp rx=0 tx=0).stop 2024-04-25T10:06:07.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.562+0000 7f911e29e640 1 -- 172.21.15.16:0/851517753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91180cece0 msgr2=0x7f91180cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.562+0000 7f911e29e640 1 --2- 172.21.15.16:0/851517753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91180cece0 0x7f91180cce60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f910c00d800 tx=0x7f910c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.562+0000 7f911e29e640 1 -- 172.21.15.16:0/851517753 shutdown_connections 2024-04-25T10:06:07.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.562+0000 7f911e29e640 1 --2- 172.21.15.16:0/851517753 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f910406b4d0 0x7f910406d990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.562+0000 7f911e29e640 1 --2- 172.21.15.16:0/851517753 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9118161090 0x7f91181f3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.562+0000 7f911e29e640 1 --2- 172.21.15.16:0/851517753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91180cece0 0x7f91180cce60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.562+0000 7f911e29e640 1 --2- 172.21.15.16:0/851517753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91180b9720 0x7f9118159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.562+0000 7f911e29e640 1 -- 172.21.15.16:0/851517753 >> 172.21.15.16:0/851517753 conn(0x7f91180b55e0 msgr2=0x7f91180d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.562+0000 7f911e29e640 1 -- 172.21.15.16:0/851517753 shutdown_connections 2024-04-25T10:06:07.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.562+0000 7f911e29e640 1 -- 172.21.15.16:0/851517753 wait complete. 2024-04-25T10:06:07.580 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:07.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.675+0000 7f5d9005d640 1 Processor -- start 2024-04-25T10:06:07.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.675+0000 7f5d9005d640 1 -- start start 2024-04-25T10:06:07.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.676+0000 7f5d9005d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d8815dce0 0x7f5d880d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.676+0000 7f5d9005d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d880d35b0 0x7f5d880b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.676+0000 7f5d9005d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d8815e0c0 con 0x7f5d880d1f90 2024-04-25T10:06:07.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.676+0000 7f5d9005d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d880b7b10 con 0x7f5d8815dce0 2024-04-25T10:06:07.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.676+0000 7f5d9005d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d880d2510 con 0x7f5d880d35b0 2024-04-25T10:06:07.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.676+0000 7f5d8ddf9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d8815dce0 0x7f5d880d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.676+0000 7f5d8ddf9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d8815dce0 0x7f5d880d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34488/0 (socket says 172.21.15.16:34488) 2024-04-25T10:06:07.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.676+0000 7f5d8ddf9640 1 -- 172.21.15.16:0/62544206 learned_addr learned my addr 172.21.15.16:0/62544206 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:07.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.676+0000 7f5d8d5f8640 1 --2- 172.21.15.16:0/62544206 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d880d35b0 0x7f5d880b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.676+0000 7f5d8ddf9640 1 -- 172.21.15.16:0/62544206 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d880d35b0 msgr2=0x7f5d880b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:07.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.676+0000 7f5d8ddf9640 1 --2- 172.21.15.16:0/62544206 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d880d35b0 0x7f5d880b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.676+0000 7f5d8ddf9640 1 -- 172.21.15.16:0/62544206 >> v1:172.21.15.186:6789/0 conn(0x7f5d880d1f90 legacy=0x7f5d8806bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.676+0000 7f5d8ddf9640 1 -- 172.21.15.16:0/62544206 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d880d27f0 con 0x7f5d8815dce0 2024-04-25T10:06:07.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.677+0000 7f5d8ddf9640 1 --2- 172.21.15.16:0/62544206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d8815dce0 0x7f5d880d1a50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009870 tx=0x7f5d8402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb9871573c16ace server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.677+0000 7f5d8cdf7640 1 -- 172.21.15.16:0/62544206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84037070 con 0x7f5d8815dce0 2024-04-25T10:06:07.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.677+0000 7f5d8cdf7640 1 -- 172.21.15.16:0/62544206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d84032440 con 0x7f5d8815dce0 2024-04-25T10:06:07.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.677+0000 7f5d8cdf7640 1 -- 172.21.15.16:0/62544206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84032810 con 0x7f5d8815dce0 2024-04-25T10:06:07.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.678+0000 7f5d9005d640 1 -- 172.21.15.16:0/62544206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d8815dce0 msgr2=0x7f5d880d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.678+0000 7f5d9005d640 1 --2- 172.21.15.16:0/62544206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d8815dce0 0x7f5d880d1a50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009870 tx=0x7f5d8402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.678+0000 7f5d9005d640 1 -- 172.21.15.16:0/62544206 shutdown_connections 2024-04-25T10:06:07.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.678+0000 7f5d9005d640 1 --2- 172.21.15.16:0/62544206 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d880d35b0 0x7f5d880b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.678+0000 7f5d9005d640 1 --2- 172.21.15.16:0/62544206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d8815dce0 0x7f5d880d1a50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.678+0000 7f5d9005d640 1 -- 172.21.15.16:0/62544206 >> 172.21.15.16:0/62544206 conn(0x7f5d88072900 msgr2=0x7f5d880d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.678+0000 7f5d9005d640 1 -- 172.21.15.16:0/62544206 shutdown_connections 2024-04-25T10:06:07.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.678+0000 7f5d9005d640 1 -- 172.21.15.16:0/62544206 wait complete. 2024-04-25T10:06:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.679+0000 7f5d9005d640 1 Processor -- start 2024-04-25T10:06:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.679+0000 7f5d9005d640 1 -- start start 2024-04-25T10:06:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d9005d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d880d1f90 0x7f5d88160c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d9005d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d880d35b0 0x7f5d88161140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d9005d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d8815dce0 0x7f5d8815e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d9005d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d880d3090 con 0x7f5d880d1f90 2024-04-25T10:06:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d9005d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d880d2db0 con 0x7f5d880d35b0 2024-04-25T10:06:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d9005d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d880b9820 con 0x7f5d8815dce0 2024-04-25T10:06:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d8ddf9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d880d1f90 0x7f5d88160c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d8ddf9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d880d1f90 0x7f5d88160c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34498/0 (socket says 172.21.15.16:34498) 2024-04-25T10:06:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d8ddf9640 1 -- 172.21.15.16:0/842738303 learned_addr learned my addr 172.21.15.16:0/842738303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d8d5f8640 1 --2- 172.21.15.16:0/842738303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d880d35b0 0x7f5d88161140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d8e5fa640 1 --2- 172.21.15.16:0/842738303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d8815dce0 0x7f5d8815e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d8ddf9640 1 -- 172.21.15.16:0/842738303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d8815dce0 msgr2=0x7f5d8815e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d8ddf9640 1 --2- 172.21.15.16:0/842738303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d8815dce0 0x7f5d8815e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d8ddf9640 1 -- 172.21.15.16:0/842738303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d880d35b0 msgr2=0x7f5d88161140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d8ddf9640 1 --2- 172.21.15.16:0/842738303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d880d35b0 0x7f5d88161140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d8ddf9640 1 -- 172.21.15.16:0/842738303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d880ccc00 con 0x7f5d880d1f90 2024-04-25T10:06:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d8d5f8640 1 --2- 172.21.15.16:0/842738303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d880d35b0 0x7f5d88161140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d8ddf9640 1 --2- 172.21.15.16:0/842738303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d880d1f90 0x7f5d88160c00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5d84000c00 tx=0x7f5d840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d7a7fc640 1 -- 172.21.15.16:0/842738303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84041020 con 0x7f5d880d1f90 2024-04-25T10:06:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d7a7fc640 1 -- 172.21.15.16:0/842738303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d840306c0 con 0x7f5d880d1f90 2024-04-25T10:06:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.680+0000 7f5d9005d640 1 -- 172.21.15.16:0/842738303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d880ccee0 con 0x7f5d880d1f90 2024-04-25T10:06:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.681+0000 7f5d7a7fc640 1 -- 172.21.15.16:0/842738303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84037040 con 0x7f5d880d1f90 2024-04-25T10:06:07.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.681+0000 7f5d9005d640 1 -- 172.21.15.16:0/842738303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d880cd4a0 con 0x7f5d880d1f90 2024-04-25T10:06:07.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.681+0000 7f5d9005d640 1 -- 172.21.15.16:0/842738303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d880749b0 con 0x7f5d880d1f90 2024-04-25T10:06:07.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.682+0000 7f5d7a7fc640 1 -- 172.21.15.16:0/842738303 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d84044020 con 0x7f5d880d1f90 2024-04-25T10:06:07.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.682+0000 7f5d7a7fc640 1 --2- 172.21.15.16:0/842738303 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5d6c06b500 0x7f5d6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.682+0000 7f5d7a7fc640 1 -- 172.21.15.16:0/842738303 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5d840aa730 con 0x7f5d880d1f90 2024-04-25T10:06:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.683+0000 7f5d8d5f8640 1 --2- 172.21.15.16:0/842738303 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5d6c06b500 0x7f5d6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.683+0000 7f5d8d5f8640 1 --2- 172.21.15.16:0/842738303 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5d6c06b500 0x7f5d6c06d9c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5d8815d090 tx=0x7f5d7c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.687+0000 7f5d7a7fc640 1 -- 172.21.15.16:0/842738303 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d8404b030 con 0x7f5d880d1f90 2024-04-25T10:06:07.811 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T10:06:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.871+0000 7f5d9005d640 1 -- 172.21.15.16:0/842738303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d880ce930 con 0x7f5d880d1f90 2024-04-25T10:06:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.872+0000 7f5d7a7fc640 1 -- 172.21.15.16:0/842738303 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+12891 (secure 0 0 0) 0x7f5d84079880 con 0x7f5d880d1f90 2024-04-25T10:06:07.878 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:07.879 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":32,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:06:04.899685+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.874+0000 7f5d9005d640 1 -- 172.21.15.16:0/842738303 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5d6c06b500 msgr2=0x7f5d6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.874+0000 7f5d9005d640 1 --2- 172.21.15.16:0/842738303 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5d6c06b500 0x7f5d6c06d9c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5d8815d090 tx=0x7f5d7c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.874+0000 7f5d9005d640 1 -- 172.21.15.16:0/842738303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d880d1f90 msgr2=0x7f5d88160c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.874+0000 7f5d9005d640 1 --2- 172.21.15.16:0/842738303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d880d1f90 0x7f5d88160c00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5d84000c00 tx=0x7f5d840046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.874+0000 7f5d9005d640 1 -- 172.21.15.16:0/842738303 shutdown_connections 2024-04-25T10:06:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.874+0000 7f5d9005d640 1 --2- 172.21.15.16:0/842738303 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5d6c06b500 0x7f5d6c06d9c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.874+0000 7f5d9005d640 1 --2- 172.21.15.16:0/842738303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d8815dce0 0x7f5d8815e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.874+0000 7f5d9005d640 1 --2- 172.21.15.16:0/842738303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d880d35b0 0x7f5d88161140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.874+0000 7f5d9005d640 1 --2- 172.21.15.16:0/842738303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d880d1f90 0x7f5d88160c00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.874+0000 7f5d9005d640 1 -- 172.21.15.16:0/842738303 >> 172.21.15.16:0/842738303 conn(0x7f5d88072900 msgr2=0x7f5d880b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.875+0000 7f5d9005d640 1 -- 172.21.15.16:0/842738303 shutdown_connections 2024-04-25T10:06:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.875+0000 7f5d9005d640 1 -- 172.21.15.16:0/842738303 wait complete. 2024-04-25T10:06:07.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:07.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:07.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.990+0000 7f1952215640 1 Processor -- start 2024-04-25T10:06:07.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.990+0000 7f1952215640 1 -- start start 2024-04-25T10:06:07.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.991+0000 7f1952215640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f194c0d3b90 0x7f194c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.991+0000 7f1952215640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f194c0d0f60 0x7f194c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.991+0000 7f1952215640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c0d4090 con 0x7f194c15dca0 2024-04-25T10:06:07.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.991+0000 7f1952215640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194c0b6b20 con 0x7f194c0d3b90 2024-04-25T10:06:07.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.991+0000 7f1952215640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0d1390 con 0x7f194c0d0f60 2024-04-25T10:06:07.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.991+0000 7f194bfff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f194c0d0f60 0x7f194c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.991+0000 7f194b7fe640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f194c15dca0 0x7f194c0ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:54680/0 (socket says 172.21.15.16:54680) 2024-04-25T10:06:07.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.991+0000 7f194affd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f194c0d3b90 0x7f194c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.991+0000 7f194b7fe640 1 -- 172.21.15.16:0/4118175969 learned_addr learned my addr 172.21.15.16:0/4118175969 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:07.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.992+0000 7f194a7fc640 1 -- 172.21.15.16:0/4118175969 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 301848386 0 0) 0x7f194c0d4090 con 0x7f194c15dca0 2024-04-25T10:06:07.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.992+0000 7f194a7fc640 1 -- 172.21.15.16:0/4118175969 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1934003520 con 0x7f194c15dca0 2024-04-25T10:06:07.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.992+0000 7f194bfff640 1 -- 172.21.15.16:0/4118175969 >> v2:172.21.15.186:3300/0 conn(0x7f194c0d3b90 msgr2=0x7f194c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:07.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.992+0000 7f194bfff640 1 --2- 172.21.15.16:0/4118175969 >> v2:172.21.15.186:3300/0 conn(0x7f194c0d3b90 0x7f194c0d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.992+0000 7f194bfff640 1 -- 172.21.15.16:0/4118175969 >> v1:172.21.15.186:6789/0 conn(0x7f194c15dca0 legacy=0x7f194c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.992+0000 7f194bfff640 1 -- 172.21.15.16:0/4118175969 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c0d4370 con 0x7f194c0d0f60 2024-04-25T10:06:08.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.992+0000 7f194affd640 1 --2- 172.21.15.16:0/4118175969 >> v2:172.21.15.186:3300/0 conn(0x7f194c0d3b90 0x7f194c0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:08.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.992+0000 7f194bfff640 1 --2- 172.21.15.16:0/4118175969 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f194c0d0f60 0x7f194c0b2500 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1944009d90 tx=0x7f194402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e18c2a9fec57aa18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.993+0000 7f194a7fc640 1 -- 172.21.15.16:0/4118175969 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944007740 con 0x7f194c0d0f60 2024-04-25T10:06:08.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.993+0000 7f194a7fc640 1 -- 172.21.15.16:0/4118175969 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19440078e0 con 0x7f194c0d0f60 2024-04-25T10:06:08.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.993+0000 7f194a7fc640 1 -- 172.21.15.16:0/4118175969 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944007c90 con 0x7f194c0d0f60 2024-04-25T10:06:08.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.993+0000 7f1952215640 1 -- 172.21.15.16:0/4118175969 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f194c0d0f60 msgr2=0x7f194c0b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.993+0000 7f1952215640 1 --2- 172.21.15.16:0/4118175969 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f194c0d0f60 0x7f194c0b2500 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1944009d90 tx=0x7f194402a660 comp rx=0 tx=0).stop 2024-04-25T10:06:08.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.993+0000 7f1952215640 1 -- 172.21.15.16:0/4118175969 shutdown_connections 2024-04-25T10:06:08.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.993+0000 7f1952215640 1 --2- 172.21.15.16:0/4118175969 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f194c0d0f60 0x7f194c0b2500 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.993+0000 7f1952215640 1 --2- 172.21.15.16:0/4118175969 >> v2:172.21.15.186:3300/0 conn(0x7f194c0d3b90 0x7f194c0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.993+0000 7f1952215640 1 -- 172.21.15.16:0/4118175969 >> 172.21.15.16:0/4118175969 conn(0x7f194c0726c0 msgr2=0x7f194c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.993+0000 7f1952215640 1 -- 172.21.15.16:0/4118175969 shutdown_connections 2024-04-25T10:06:08.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.993+0000 7f1952215640 1 -- 172.21.15.16:0/4118175969 wait complete. 2024-04-25T10:06:08.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.994+0000 7f1952215640 1 Processor -- start 2024-04-25T10:06:08.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.995+0000 7f1952215640 1 -- start start 2024-04-25T10:06:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.995+0000 7f1952215640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f194c0d0f60 0x7f194c0ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.995+0000 7f1952215640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c0d3b90 0x7f194c0aa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.995+0000 7f1952215640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f194c15dca0 0x7f194c14d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.995+0000 7f1952215640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0cf050 con 0x7f194c0d3b90 2024-04-25T10:06:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.995+0000 7f1952215640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0b6d50 con 0x7f194c15dca0 2024-04-25T10:06:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.995+0000 7f1952215640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c15e080 con 0x7f194c0d0f60 2024-04-25T10:06:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.995+0000 7f194affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c0d3b90 0x7f194c0aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.996+0000 7f194affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c0d3b90 0x7f194c0aa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34512/0 (socket says 172.21.15.16:34512) 2024-04-25T10:06:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.996+0000 7f194affd640 1 -- 172.21.15.16:0/3436896448 learned_addr learned my addr 172.21.15.16:0/3436896448 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.996+0000 7f194bfff640 1 --2- 172.21.15.16:0/3436896448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f194c15dca0 0x7f194c14d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.996+0000 7f194b7fe640 1 --2- 172.21.15.16:0/3436896448 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f194c0d0f60 0x7f194c0ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.996+0000 7f194affd640 1 -- 172.21.15.16:0/3436896448 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f194c0d0f60 msgr2=0x7f194c0ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.996+0000 7f194affd640 1 --2- 172.21.15.16:0/3436896448 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f194c0d0f60 0x7f194c0ca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.996+0000 7f194affd640 1 -- 172.21.15.16:0/3436896448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f194c15dca0 msgr2=0x7f194c14d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.996+0000 7f194affd640 1 --2- 172.21.15.16:0/3436896448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f194c15dca0 0x7f194c14d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.996+0000 7f194affd640 1 -- 172.21.15.16:0/3436896448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c14c860 con 0x7f194c0d3b90 2024-04-25T10:06:08.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.996+0000 7f194affd640 1 --2- 172.21.15.16:0/3436896448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c0d3b90 0x7f194c0aa480 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f194000e9e0 tx=0x7f194000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.996+0000 7f1933fff640 1 -- 172.21.15.16:0/3436896448 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940017070 con 0x7f194c0d3b90 2024-04-25T10:06:08.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.996+0000 7f1933fff640 1 -- 172.21.15.16:0/3436896448 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194000ca30 con 0x7f194c0d3b90 2024-04-25T10:06:08.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.996+0000 7f1952215640 1 -- 172.21.15.16:0/3436896448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194c14cb40 con 0x7f194c0d3b90 2024-04-25T10:06:08.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.996+0000 7f1933fff640 1 -- 172.21.15.16:0/3436896448 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194000ce20 con 0x7f194c0d3b90 2024-04-25T10:06:08.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.996+0000 7f1952215640 1 -- 172.21.15.16:0/3436896448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194c14d100 con 0x7f194c0d3b90 2024-04-25T10:06:08.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.997+0000 7f1952215640 1 -- 172.21.15.16:0/3436896448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194c0750b0 con 0x7f194c0d3b90 2024-04-25T10:06:08.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.998+0000 7f1933fff640 1 -- 172.21.15.16:0/3436896448 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f194001a070 con 0x7f194c0d3b90 2024-04-25T10:06:08.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.998+0000 7f1933fff640 1 --2- 172.21.15.16:0/3436896448 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f193806b4d0 0x7f193806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.998+0000 7f1933fff640 1 -- 172.21.15.16:0/3436896448 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1940089d90 con 0x7f194c0d3b90 2024-04-25T10:06:08.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.999+0000 7f194b7fe640 1 --2- 172.21.15.16:0/3436896448 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f193806b4d0 0x7f193806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:07.999+0000 7f194b7fe640 1 --2- 172.21.15.16:0/3436896448 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f193806b4d0 0x7f193806d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f193c005880 tx=0x7f193c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.003+0000 7f1933fff640 1 -- 172.21.15.16:0/3436896448 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1940058910 con 0x7f194c0d3b90 2024-04-25T10:06:08.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.181+0000 7f1952215640 1 -- 172.21.15.16:0/3436896448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f194c0cebb0 con 0x7f194c0d3b90 2024-04-25T10:06:08.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.182+0000 7f1933fff640 1 -- 172.21.15.16:0/3436896448 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7f194005c9b0 con 0x7f194c0d3b90 2024-04-25T10:06:08.188 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:06:08.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.184+0000 7f1952215640 1 -- 172.21.15.16:0/3436896448 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f193806b4d0 msgr2=0x7f193806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.184+0000 7f1952215640 1 --2- 172.21.15.16:0/3436896448 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f193806b4d0 0x7f193806d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f193c005880 tx=0x7f193c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:08.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.184+0000 7f1952215640 1 -- 172.21.15.16:0/3436896448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c0d3b90 msgr2=0x7f194c0aa480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.184+0000 7f1952215640 1 --2- 172.21.15.16:0/3436896448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c0d3b90 0x7f194c0aa480 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f194000e9e0 tx=0x7f194000eea0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.184+0000 7f1952215640 1 -- 172.21.15.16:0/3436896448 shutdown_connections 2024-04-25T10:06:08.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.184+0000 7f1952215640 1 --2- 172.21.15.16:0/3436896448 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f193806b4d0 0x7f193806d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.184+0000 7f1952215640 1 --2- 172.21.15.16:0/3436896448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f194c15dca0 0x7f194c14d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.184+0000 7f1952215640 1 --2- 172.21.15.16:0/3436896448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c0d3b90 0x7f194c0aa480 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.184+0000 7f1952215640 1 --2- 172.21.15.16:0/3436896448 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f194c0d0f60 0x7f194c0ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.184+0000 7f1952215640 1 -- 172.21.15.16:0/3436896448 >> 172.21.15.16:0/3436896448 conn(0x7f194c0726c0 msgr2=0x7f194c15bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.184+0000 7f1952215640 1 -- 172.21.15.16:0/3436896448 shutdown_connections 2024-04-25T10:06:08.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.184+0000 7f1952215640 1 -- 172.21.15.16:0/3436896448 wait complete. 2024-04-25T10:06:08.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:06:08.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.295+0000 7f275cc1f640 1 Processor -- start 2024-04-25T10:06:08.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.295+0000 7f275cc1f640 1 -- start start 2024-04-25T10:06:08.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.296+0000 7f275cc1f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f275815a870 0x7f27580cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.296+0000 7f275cc1f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f275806e2e0 0x7f275815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.296+0000 7f275cc1f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27580cd580 0x7f27580cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.296+0000 7f275cc1f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27580ce720 con 0x7f275815a870 2024-04-25T10:06:08.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.296+0000 7f275cc1f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27580cdb10 con 0x7f275806e2e0 2024-04-25T10:06:08.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.296+0000 7f275cc1f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27580cddf0 con 0x7f27580cd580 2024-04-25T10:06:08.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.296+0000 7f275659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f275815a870 0x7f27580cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.296+0000 7f275659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f275815a870 0x7f27580cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34524/0 (socket says 172.21.15.16:34524) 2024-04-25T10:06:08.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.296+0000 7f275659c640 1 -- 172.21.15.16:0/2472252913 learned_addr learned my addr 172.21.15.16:0/2472252913 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:08.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.296+0000 7f274ffff640 1 --2- 172.21.15.16:0/2472252913 >> v2:172.21.15.186:3300/0 conn(0x7f275806e2e0 0x7f275815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.296+0000 7f2756d9d640 1 --2- 172.21.15.16:0/2472252913 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27580cd580 0x7f27580cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.296+0000 7f275659c640 1 -- 172.21.15.16:0/2472252913 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27580cd580 msgr2=0x7f27580cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:08.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.296+0000 7f275659c640 1 --2- 172.21.15.16:0/2472252913 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27580cd580 0x7f27580cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.296+0000 7f275659c640 1 -- 172.21.15.16:0/2472252913 >> v2:172.21.15.186:3300/0 conn(0x7f275806e2e0 msgr2=0x7f275815ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:08.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.296+0000 7f275659c640 1 --2- 172.21.15.16:0/2472252913 >> v2:172.21.15.186:3300/0 conn(0x7f275806e2e0 0x7f275815ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.296+0000 7f275659c640 1 -- 172.21.15.16:0/2472252913 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27580ce0d0 con 0x7f275815a870 2024-04-25T10:06:08.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.296+0000 7f274ffff640 1 --2- 172.21.15.16:0/2472252913 >> v2:172.21.15.186:3300/0 conn(0x7f275806e2e0 0x7f275815ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:08.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.297+0000 7f275659c640 1 --2- 172.21.15.16:0/2472252913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275815a870 0x7f27580cebb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f27580d0330 tx=0x7f274802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64504bc8166b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.297+0000 7f2755d9b640 1 -- 172.21.15.16:0/2472252913 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748037070 con 0x7f275815a870 2024-04-25T10:06:08.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.297+0000 7f2755d9b640 1 -- 172.21.15.16:0/2472252913 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2748032440 con 0x7f275815a870 2024-04-25T10:06:08.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.297+0000 7f2755d9b640 1 -- 172.21.15.16:0/2472252913 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748032810 con 0x7f275815a870 2024-04-25T10:06:08.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.298+0000 7f275cc1f640 1 -- 172.21.15.16:0/2472252913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275815a870 msgr2=0x7f27580cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.298+0000 7f275cc1f640 1 --2- 172.21.15.16:0/2472252913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275815a870 0x7f27580cebb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f27580d0330 tx=0x7f274802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.298+0000 7f275cc1f640 1 -- 172.21.15.16:0/2472252913 shutdown_connections 2024-04-25T10:06:08.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.298+0000 7f275cc1f640 1 --2- 172.21.15.16:0/2472252913 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27580cd580 0x7f27580cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.298+0000 7f275cc1f640 1 --2- 172.21.15.16:0/2472252913 >> v2:172.21.15.186:3300/0 conn(0x7f275806e2e0 0x7f275815ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.298+0000 7f275cc1f640 1 --2- 172.21.15.16:0/2472252913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275815a870 0x7f27580cebb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.298+0000 7f275cc1f640 1 -- 172.21.15.16:0/2472252913 >> 172.21.15.16:0/2472252913 conn(0x7f275806aba0 msgr2=0x7f275806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.298+0000 7f275cc1f640 1 -- 172.21.15.16:0/2472252913 shutdown_connections 2024-04-25T10:06:08.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.298+0000 7f275cc1f640 1 -- 172.21.15.16:0/2472252913 wait complete. 2024-04-25T10:06:08.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.299+0000 7f275cc1f640 1 Processor -- start 2024-04-25T10:06:08.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.299+0000 7f275cc1f640 1 -- start start 2024-04-25T10:06:08.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f275cc1f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f275806e2e0 0x7f27581490d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f275cc1f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27580cd580 0x7f2758144fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f275cc1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275815a870 0x7f275815c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f275cc1f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27580cfbf0 con 0x7f275815a870 2024-04-25T10:06:08.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f275cc1f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27580cfa70 con 0x7f27580cd580 2024-04-25T10:06:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f275cc1f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27580cfd70 con 0x7f275806e2e0 2024-04-25T10:06:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f2756d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275815a870 0x7f275815c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f2756d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275815a870 0x7f275815c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34532/0 (socket says 172.21.15.16:34532) 2024-04-25T10:06:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f2756d9d640 1 -- 172.21.15.16:0/1399492284 learned_addr learned my addr 172.21.15.16:0/1399492284 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f274ffff640 1 --2- 172.21.15.16:0/1399492284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27580cd580 0x7f2758144fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f275659c640 1 --2- 172.21.15.16:0/1399492284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f275806e2e0 0x7f27581490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f2756d9d640 1 -- 172.21.15.16:0/1399492284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f275806e2e0 msgr2=0x7f27581490d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f2756d9d640 1 --2- 172.21.15.16:0/1399492284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f275806e2e0 0x7f27581490d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f2756d9d640 1 -- 172.21.15.16:0/1399492284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27580cd580 msgr2=0x7f2758144fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f2756d9d640 1 --2- 172.21.15.16:0/1399492284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27580cd580 0x7f2758144fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f2756d9d640 1 -- 172.21.15.16:0/1399492284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275814f560 con 0x7f275815a870 2024-04-25T10:06:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f274ffff640 1 --2- 172.21.15.16:0/1399492284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27580cd580 0x7f2758144fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f2756d9d640 1 --2- 172.21.15.16:0/1399492284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275815a870 0x7f275815c490 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2750007e60 tx=0x7f275000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f274effd640 1 -- 172.21.15.16:0/1399492284 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750014420 con 0x7f275815a870 2024-04-25T10:06:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f274effd640 1 -- 172.21.15.16:0/1399492284 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2750019070 con 0x7f275815a870 2024-04-25T10:06:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f275cc1f640 1 -- 172.21.15.16:0/1399492284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275814f840 con 0x7f275815a870 2024-04-25T10:06:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.300+0000 7f274effd640 1 -- 172.21.15.16:0/1399492284 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750014820 con 0x7f275815a870 2024-04-25T10:06:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.301+0000 7f275cc1f640 1 -- 172.21.15.16:0/1399492284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27581470c0 con 0x7f275815a870 2024-04-25T10:06:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.302+0000 7f275cc1f640 1 -- 172.21.15.16:0/1399492284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27581472c0 con 0x7f275815a870 2024-04-25T10:06:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.302+0000 7f274effd640 1 -- 172.21.15.16:0/1399492284 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27500149c0 con 0x7f275815a870 2024-04-25T10:06:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.302+0000 7f274effd640 1 --2- 172.21.15.16:0/1399492284 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f273806b500 0x7f273806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.302+0000 7f274effd640 1 -- 172.21.15.16:0/1399492284 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f275008a620 con 0x7f275815a870 2024-04-25T10:06:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.303+0000 7f275659c640 1 --2- 172.21.15.16:0/1399492284 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f273806b500 0x7f273806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.303+0000 7f275659c640 1 --2- 172.21.15.16:0/1399492284 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f273806b500 0x7f273806d9c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2748034000 tx=0x7f274802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.307+0000 7f274effd640 1 -- 172.21.15.16:0/1399492284 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f27581472c0 con 0x7f275815a870 2024-04-25T10:06:08.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.488+0000 7f275cc1f640 1 -- 172.21.15.16:0/1399492284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f27580749b0 con 0x7f275815a870 2024-04-25T10:06:08.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.489+0000 7f274effd640 1 -- 172.21.15.16:0/1399492284 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2750059880 con 0x7f275815a870 2024-04-25T10:06:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:08.495 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-04-25T10:06:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-04-25T10:06:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-04-25T10:06:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:08.495 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-04-25T10:06:08.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.490+0000 7f275cc1f640 1 -- 172.21.15.16:0/1399492284 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f273806b500 msgr2=0x7f273806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.490+0000 7f275cc1f640 1 --2- 172.21.15.16:0/1399492284 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f273806b500 0x7f273806d9c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2748034000 tx=0x7f274802e000 comp rx=0 tx=0).stop 2024-04-25T10:06:08.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.490+0000 7f275cc1f640 1 -- 172.21.15.16:0/1399492284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275815a870 msgr2=0x7f275815c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.490+0000 7f275cc1f640 1 --2- 172.21.15.16:0/1399492284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275815a870 0x7f275815c490 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2750007e60 tx=0x7f275000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.490+0000 7f275cc1f640 1 -- 172.21.15.16:0/1399492284 shutdown_connections 2024-04-25T10:06:08.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.490+0000 7f275cc1f640 1 --2- 172.21.15.16:0/1399492284 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f273806b500 0x7f273806d9c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.490+0000 7f275cc1f640 1 --2- 172.21.15.16:0/1399492284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275815a870 0x7f275815c490 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.490+0000 7f275cc1f640 1 --2- 172.21.15.16:0/1399492284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27580cd580 0x7f2758144fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.490+0000 7f275cc1f640 1 --2- 172.21.15.16:0/1399492284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f275806e2e0 0x7f27581490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.490+0000 7f275cc1f640 1 -- 172.21.15.16:0/1399492284 >> 172.21.15.16:0/1399492284 conn(0x7f275806aba0 msgr2=0x7f275815ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.490+0000 7f275cc1f640 1 -- 172.21.15.16:0/1399492284 shutdown_connections 2024-04-25T10:06:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.490+0000 7f275cc1f640 1 -- 172.21.15.16:0/1399492284 wait complete. 2024-04-25T10:06:08.507 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:06:08.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:06:08.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.606+0000 7f2cbade7640 1 Processor -- start 2024-04-25T10:06:08.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.606+0000 7f2cbade7640 1 -- start start 2024-04-25T10:06:08.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.607+0000 7f2cbade7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cb41610b0 0x7f2cb40b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.607+0000 7f2cbade7640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2cb41618d0 0x7f2cb40ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.607+0000 7f2cbade7640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb4162040 con 0x7f2cb40ced70 2024-04-25T10:06:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.607+0000 7f2cbade7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb40cf1c0 con 0x7f2cb41610b0 2024-04-25T10:06:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.607+0000 7f2cbade7640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb40d1ea0 con 0x7f2cb41618d0 2024-04-25T10:06:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.607+0000 7f2cb8b83640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cb41610b0 0x7f2cb40b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.607+0000 7f2cb8b83640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cb41610b0 0x7f2cb40b9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34536/0 (socket says 172.21.15.16:34536) 2024-04-25T10:06:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.607+0000 7f2cb8b83640 1 -- 172.21.15.16:0/3049373784 learned_addr learned my addr 172.21.15.16:0/3049373784 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.607+0000 7f2cabfff640 1 --2- 172.21.15.16:0/3049373784 >> v2:172.21.15.186:3300/0 conn(0x7f2cb41618d0 0x7f2cb40ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.607+0000 7f2cb8b83640 1 -- 172.21.15.16:0/3049373784 >> v2:172.21.15.186:3300/0 conn(0x7f2cb41618d0 msgr2=0x7f2cb40ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.607+0000 7f2cb8b83640 1 --2- 172.21.15.16:0/3049373784 >> v2:172.21.15.186:3300/0 conn(0x7f2cb41618d0 0x7f2cb40ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.607+0000 7f2cb8b83640 1 -- 172.21.15.16:0/3049373784 >> v1:172.21.15.186:6789/0 conn(0x7f2cb40ced70 legacy=0x7f2cb406c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.607+0000 7f2cb8b83640 1 -- 172.21.15.16:0/3049373784 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb4162320 con 0x7f2cb41610b0 2024-04-25T10:06:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.607+0000 7f2cabfff640 1 --2- 172.21.15.16:0/3049373784 >> v2:172.21.15.186:3300/0 conn(0x7f2cb41618d0 0x7f2cb40ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.608+0000 7f2cb8b83640 1 --2- 172.21.15.16:0/3049373784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb41610b0 0x7f2cb40b9580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2cb406d320 tx=0x7f2cb002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3bb633357f777d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.608+0000 7f2cab7fe640 1 -- 172.21.15.16:0/3049373784 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0037070 con 0x7f2cb41610b0 2024-04-25T10:06:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.608+0000 7f2cab7fe640 1 -- 172.21.15.16:0/3049373784 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb0032440 con 0x7f2cb41610b0 2024-04-25T10:06:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.608+0000 7f2cab7fe640 1 -- 172.21.15.16:0/3049373784 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0032810 con 0x7f2cb41610b0 2024-04-25T10:06:08.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.609+0000 7f2cbade7640 1 -- 172.21.15.16:0/3049373784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb41610b0 msgr2=0x7f2cb40b9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.609+0000 7f2cbade7640 1 --2- 172.21.15.16:0/3049373784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb41610b0 0x7f2cb40b9580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2cb406d320 tx=0x7f2cb002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.609+0000 7f2cbade7640 1 -- 172.21.15.16:0/3049373784 shutdown_connections 2024-04-25T10:06:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.609+0000 7f2cbade7640 1 --2- 172.21.15.16:0/3049373784 >> v2:172.21.15.186:3300/0 conn(0x7f2cb41618d0 0x7f2cb40ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.609+0000 7f2cbade7640 1 --2- 172.21.15.16:0/3049373784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb41610b0 0x7f2cb40b9580 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.609+0000 7f2cbade7640 1 -- 172.21.15.16:0/3049373784 >> 172.21.15.16:0/3049373784 conn(0x7f2cb40b5570 msgr2=0x7f2cb40b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.609+0000 7f2cbade7640 1 -- 172.21.15.16:0/3049373784 shutdown_connections 2024-04-25T10:06:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.609+0000 7f2cbade7640 1 -- 172.21.15.16:0/3049373784 wait complete. 2024-04-25T10:06:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.610+0000 7f2cbade7640 1 Processor -- start 2024-04-25T10:06:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.610+0000 7f2cbade7640 1 -- start start 2024-04-25T10:06:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2cbade7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cb40ced70 0x7f2cb415c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2cbade7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cb41610b0 0x7f2cb415fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2cbade7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb41618d0 0x7f2cb4158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2cbade7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb40b6c00 con 0x7f2cb41618d0 2024-04-25T10:06:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2cbade7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb40d20f0 con 0x7f2cb41610b0 2024-04-25T10:06:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2cbade7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb4161d10 con 0x7f2cb40ced70 2024-04-25T10:06:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2cb9384640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb41618d0 0x7f2cb4158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2cb9384640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb41618d0 0x7f2cb4158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34548/0 (socket says 172.21.15.16:34548) 2024-04-25T10:06:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2cb9384640 1 -- 172.21.15.16:0/2664488447 learned_addr learned my addr 172.21.15.16:0/2664488447 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2cabfff640 1 --2- 172.21.15.16:0/2664488447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cb41610b0 0x7f2cb415fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2cb8b83640 1 --2- 172.21.15.16:0/2664488447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cb40ced70 0x7f2cb415c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2cb9384640 1 -- 172.21.15.16:0/2664488447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cb40ced70 msgr2=0x7f2cb415c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2cb9384640 1 --2- 172.21.15.16:0/2664488447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cb40ced70 0x7f2cb415c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2cb9384640 1 -- 172.21.15.16:0/2664488447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cb41610b0 msgr2=0x7f2cb415fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2cb9384640 1 --2- 172.21.15.16:0/2664488447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cb41610b0 0x7f2cb415fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2cb9384640 1 -- 172.21.15.16:0/2664488447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb415ec80 con 0x7f2cb41618d0 2024-04-25T10:06:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2cabfff640 1 --2- 172.21.15.16:0/2664488447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cb41610b0 0x7f2cb415fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2cb9384640 1 --2- 172.21.15.16:0/2664488447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb41618d0 0x7f2cb4158510 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00c7b0 tx=0x7f2cac00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.611+0000 7f2ca97fa640 1 -- 172.21.15.16:0/2664488447 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac004b60 con 0x7f2cb41618d0 2024-04-25T10:06:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.612+0000 7f2ca97fa640 1 -- 172.21.15.16:0/2664488447 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cac004d00 con 0x7f2cb41618d0 2024-04-25T10:06:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.612+0000 7f2cbade7640 1 -- 172.21.15.16:0/2664488447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb415ef60 con 0x7f2cb41618d0 2024-04-25T10:06:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.612+0000 7f2ca97fa640 1 -- 172.21.15.16:0/2664488447 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00aa90 con 0x7f2cb41618d0 2024-04-25T10:06:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.612+0000 7f2cbade7640 1 -- 172.21.15.16:0/2664488447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb415f520 con 0x7f2cb41618d0 2024-04-25T10:06:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.612+0000 7f2cbade7640 1 -- 172.21.15.16:0/2664488447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb415fa00 con 0x7f2cb41618d0 2024-04-25T10:06:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.613+0000 7f2ca97fa640 1 -- 172.21.15.16:0/2664488447 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2cac00ac30 con 0x7f2cb41618d0 2024-04-25T10:06:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.613+0000 7f2ca97fa640 1 --2- 172.21.15.16:0/2664488447 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ca006b4d0 0x7f2ca006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.613+0000 7f2ca97fa640 1 -- 172.21.15.16:0/2664488447 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2cac08a190 con 0x7f2cb41618d0 2024-04-25T10:06:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.614+0000 7f2cb8b83640 1 --2- 172.21.15.16:0/2664488447 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ca006b4d0 0x7f2ca006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.614+0000 7f2cb8b83640 1 --2- 172.21.15.16:0/2664488447 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ca006b4d0 0x7f2ca006d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0034000 tx=0x7f2cb002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.618+0000 7f2ca97fa640 1 -- 172.21.15.16:0/2664488447 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2cb415fa00 con 0x7f2cb41618d0 2024-04-25T10:06:08.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:08.797+0000 7f2cbade7640 1 -- 172.21.15.16:0/2664488447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2cb40750b0 con 0x7f2cb41618d0 2024-04-25T10:06:09.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.564+0000 7f2ca97fa640 1 -- 172.21.15.16:0/2664488447 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2cac020070 con 0x7f2cb41618d0 2024-04-25T10:06:09.570 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-04-25T10:06:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.567+0000 7f2cbade7640 1 -- 172.21.15.16:0/2664488447 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ca006b4d0 msgr2=0x7f2ca006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.567+0000 7f2cbade7640 1 --2- 172.21.15.16:0/2664488447 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ca006b4d0 0x7f2ca006d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0034000 tx=0x7f2cb002e000 comp rx=0 tx=0).stop 2024-04-25T10:06:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.567+0000 7f2cbade7640 1 -- 172.21.15.16:0/2664488447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb41618d0 msgr2=0x7f2cb4158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.567+0000 7f2cbade7640 1 --2- 172.21.15.16:0/2664488447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb41618d0 0x7f2cb4158510 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00c7b0 tx=0x7f2cac00cc70 comp rx=0 tx=0).stop 2024-04-25T10:06:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.567+0000 7f2cbade7640 1 -- 172.21.15.16:0/2664488447 shutdown_connections 2024-04-25T10:06:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.567+0000 7f2cbade7640 1 --2- 172.21.15.16:0/2664488447 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ca006b4d0 0x7f2ca006d990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.567+0000 7f2cbade7640 1 --2- 172.21.15.16:0/2664488447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb41618d0 0x7f2cb4158510 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.567+0000 7f2cbade7640 1 --2- 172.21.15.16:0/2664488447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cb41610b0 0x7f2cb415fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.567+0000 7f2cbade7640 1 --2- 172.21.15.16:0/2664488447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cb40ced70 0x7f2cb415c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.567+0000 7f2cbade7640 1 -- 172.21.15.16:0/2664488447 >> 172.21.15.16:0/2664488447 conn(0x7f2cb40b5570 msgr2=0x7f2cb4156850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.567+0000 7f2cbade7640 1 -- 172.21.15.16:0/2664488447 shutdown_connections 2024-04-25T10:06:09.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.567+0000 7f2cbade7640 1 -- 172.21.15.16:0/2664488447 wait complete. 2024-04-25T10:06:09.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:06:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.698+0000 7f58fdc8f640 1 Processor -- start 2024-04-25T10:06:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.698+0000 7f58fdc8f640 1 -- start start 2024-04-25T10:06:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.698+0000 7f58fdc8f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58f80ce390 0x7f58f806aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.698+0000 7f58fdc8f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58f806b0e0 0x7f58f815d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.699+0000 7f58fdc8f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f80cd0f0 con 0x7f58f80cc7a0 2024-04-25T10:06:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.699+0000 7f58fdc8f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f80cbdb0 con 0x7f58f80ce390 2024-04-25T10:06:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.699+0000 7f58fdc8f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80cda20 con 0x7f58f806b0e0 2024-04-25T10:06:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.699+0000 7f58f6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58f80ce390 0x7f58f806aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.699+0000 7f58f6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58f80ce390 0x7f58f806aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34552/0 (socket says 172.21.15.16:34552) 2024-04-25T10:06:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.699+0000 7f58f6ffd640 1 -- 172.21.15.16:0/116530128 learned_addr learned my addr 172.21.15.16:0/116530128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.699+0000 7f58f6ffd640 1 -- 172.21.15.16:0/116530128 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58f806b0e0 msgr2=0x7f58f815d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.699+0000 7f58f6ffd640 1 --2- 172.21.15.16:0/116530128 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58f806b0e0 0x7f58f815d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.699+0000 7f58f6ffd640 1 -- 172.21.15.16:0/116530128 >> v1:172.21.15.186:6789/0 conn(0x7f58f80cc7a0 legacy=0x7f58f80b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.699+0000 7f58f6ffd640 1 -- 172.21.15.16:0/116530128 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f80cd3d0 con 0x7f58f80ce390 2024-04-25T10:06:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.700+0000 7f58f6ffd640 1 --2- 172.21.15.16:0/116530128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f80ce390 0x7f58f806aba0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009870 tx=0x7f58e802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9fef45c7ad10cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.700+0000 7f58f67fc640 1 -- 172.21.15.16:0/116530128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8037070 con 0x7f58f80ce390 2024-04-25T10:06:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.700+0000 7f58f67fc640 1 -- 172.21.15.16:0/116530128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e8032440 con 0x7f58f80ce390 2024-04-25T10:06:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.700+0000 7f58f67fc640 1 -- 172.21.15.16:0/116530128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8032810 con 0x7f58f80ce390 2024-04-25T10:06:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.700+0000 7f58fdc8f640 1 -- 172.21.15.16:0/116530128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f80ce390 msgr2=0x7f58f806aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.700+0000 7f58fdc8f640 1 --2- 172.21.15.16:0/116530128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f80ce390 0x7f58f806aba0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009870 tx=0x7f58e802a210 comp rx=0 tx=0).stop 2024-04-25T10:06:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.700+0000 7f58fdc8f640 1 -- 172.21.15.16:0/116530128 shutdown_connections 2024-04-25T10:06:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.700+0000 7f58fdc8f640 1 --2- 172.21.15.16:0/116530128 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58f806b0e0 0x7f58f815d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.701+0000 7f58fdc8f640 1 --2- 172.21.15.16:0/116530128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f80ce390 0x7f58f806aba0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.701+0000 7f58fdc8f640 1 -- 172.21.15.16:0/116530128 >> 172.21.15.16:0/116530128 conn(0x7f58f80d0750 msgr2=0x7f58f80d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.701+0000 7f58fdc8f640 1 -- 172.21.15.16:0/116530128 shutdown_connections 2024-04-25T10:06:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.701+0000 7f58fdc8f640 1 -- 172.21.15.16:0/116530128 wait complete. 2024-04-25T10:06:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.702+0000 7f58fdc8f640 1 Processor -- start 2024-04-25T10:06:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.702+0000 7f58fdc8f640 1 -- start start 2024-04-25T10:06:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58fdc8f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58f806b0e0 0x7f58f80c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58fdc8f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58f80cc7a0 0x7f58f8158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58fdc8f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f80ce390 0x7f58f80c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58fdc8f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80ce030 con 0x7f58f80ce390 2024-04-25T10:06:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58fdc8f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80cdeb0 con 0x7f58f806b0e0 2024-04-25T10:06:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58fdc8f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80ccb80 con 0x7f58f80cc7a0 2024-04-25T10:06:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f80ce390 0x7f58f80c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f80ce390 0x7f58f80c9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34566/0 (socket says 172.21.15.16:34566) 2024-04-25T10:06:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58f7fff640 1 -- 172.21.15.16:0/2865589194 learned_addr learned my addr 172.21.15.16:0/2865589194 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58f77fe640 1 --2- 172.21.15.16:0/2865589194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58f806b0e0 0x7f58f80c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58f6ffd640 1 --2- 172.21.15.16:0/2865589194 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58f80cc7a0 0x7f58f8158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58f7fff640 1 -- 172.21.15.16:0/2865589194 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58f80cc7a0 msgr2=0x7f58f8158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58f7fff640 1 --2- 172.21.15.16:0/2865589194 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58f80cc7a0 0x7f58f8158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58f7fff640 1 -- 172.21.15.16:0/2865589194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58f806b0e0 msgr2=0x7f58f80c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58f7fff640 1 --2- 172.21.15.16:0/2865589194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58f806b0e0 0x7f58f80c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58f7fff640 1 -- 172.21.15.16:0/2865589194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f80c6060 con 0x7f58f80ce390 2024-04-25T10:06:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58f7fff640 1 --2- 172.21.15.16:0/2865589194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f80ce390 0x7f58f80c9c80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f58f000b970 tx=0x7f58f000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58dffff640 1 -- 172.21.15.16:0/2865589194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0019070 con 0x7f58f80ce390 2024-04-25T10:06:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58dffff640 1 -- 172.21.15.16:0/2865589194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f000cb60 con 0x7f58f80ce390 2024-04-25T10:06:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58fdc8f640 1 -- 172.21.15.16:0/2865589194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f80c6340 con 0x7f58f80ce390 2024-04-25T10:06:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58dffff640 1 -- 172.21.15.16:0/2865589194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0014440 con 0x7f58f80ce390 2024-04-25T10:06:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.703+0000 7f58fdc8f640 1 -- 172.21.15.16:0/2865589194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f80c6900 con 0x7f58f80ce390 2024-04-25T10:06:09.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.704+0000 7f58fdc8f640 1 -- 172.21.15.16:0/2865589194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f806cc40 con 0x7f58f80ce390 2024-04-25T10:06:09.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.705+0000 7f58dffff640 1 -- 172.21.15.16:0/2865589194 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58f001c070 con 0x7f58f80ce390 2024-04-25T10:06:09.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.705+0000 7f58dffff640 1 --2- 172.21.15.16:0/2865589194 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f58e406b500 0x7f58e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.706+0000 7f58dffff640 1 -- 172.21.15.16:0/2865589194 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f58f0089cd0 con 0x7f58f80ce390 2024-04-25T10:06:09.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.706+0000 7f58f77fe640 1 --2- 172.21.15.16:0/2865589194 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f58e406b500 0x7f58e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.706+0000 7f58f77fe640 1 --2- 172.21.15.16:0/2865589194 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f58e406b500 0x7f58e406d9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f58ec004ea0 tx=0x7f58ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.710+0000 7f58dffff640 1 -- 172.21.15.16:0/2865589194 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58f0058dc0 con 0x7f58f80ce390 2024-04-25T10:06:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:09.890+0000 7f58fdc8f640 1 -- 172.21.15.16:0/2865589194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f58f80b66d0 con 0x7f58f80ce390 2024-04-25T10:06:10.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.569+0000 7f58dffff640 1 -- 172.21.15.16:0/2865589194 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) v1 ==== 170+0+0 (secure 0 0 0) 0x7f58f00586c0 con 0x7f58f80ce390 2024-04-25T10:06:10.576 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-04-25T10:06:10.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.572+0000 7f58fdc8f640 1 -- 172.21.15.16:0/2865589194 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f58e406b500 msgr2=0x7f58e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.572+0000 7f58fdc8f640 1 --2- 172.21.15.16:0/2865589194 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f58e406b500 0x7f58e406d9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f58ec004ea0 tx=0x7f58ec009040 comp rx=0 tx=0).stop 2024-04-25T10:06:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.572+0000 7f58fdc8f640 1 -- 172.21.15.16:0/2865589194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f80ce390 msgr2=0x7f58f80c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.572+0000 7f58fdc8f640 1 --2- 172.21.15.16:0/2865589194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f80ce390 0x7f58f80c9c80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f58f000b970 tx=0x7f58f000be30 comp rx=0 tx=0).stop 2024-04-25T10:06:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.572+0000 7f58fdc8f640 1 -- 172.21.15.16:0/2865589194 shutdown_connections 2024-04-25T10:06:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.572+0000 7f58fdc8f640 1 --2- 172.21.15.16:0/2865589194 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f58e406b500 0x7f58e406d9c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.572+0000 7f58fdc8f640 1 --2- 172.21.15.16:0/2865589194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f80ce390 0x7f58f80c9c80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.572+0000 7f58fdc8f640 1 --2- 172.21.15.16:0/2865589194 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58f80cc7a0 0x7f58f8158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.572+0000 7f58fdc8f640 1 --2- 172.21.15.16:0/2865589194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58f806b0e0 0x7f58f80c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.572+0000 7f58fdc8f640 1 -- 172.21.15.16:0/2865589194 >> 172.21.15.16:0/2865589194 conn(0x7f58f80d0750 msgr2=0x7f58f815adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.572+0000 7f58fdc8f640 1 -- 172.21.15.16:0/2865589194 shutdown_connections 2024-04-25T10:06:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.573+0000 7f58fdc8f640 1 -- 172.21.15.16:0/2865589194 wait complete. 2024-04-25T10:06:10.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:06:10.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.705+0000 7f06ce97e640 1 Processor -- start 2024-04-25T10:06:10.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.705+0000 7f06ce97e640 1 -- start start 2024-04-25T10:06:10.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.706+0000 7f06ce97e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f06c815dc80 0x7f06c8161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.706+0000 7f06ce97e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f06c80b5c00 0x7f06c80d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.706+0000 7f06ce97e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c81627a0 con 0x7f06c80b97c0 2024-04-25T10:06:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.706+0000 7f06ce97e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c81621e0 con 0x7f06c815dc80 2024-04-25T10:06:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.706+0000 7f06ce97e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c81624c0 con 0x7f06c80b5c00 2024-04-25T10:06:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.706+0000 7f06c7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f06c815dc80 0x7f06c8161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.706+0000 7f06c7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f06c815dc80 0x7f06c8161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34578/0 (socket says 172.21.15.16:34578) 2024-04-25T10:06:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.706+0000 7f06c7fff640 1 -- 172.21.15.16:0/841796808 learned_addr learned my addr 172.21.15.16:0/841796808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.706+0000 7f06ccf1b640 1 --2- 172.21.15.16:0/841796808 >> v2:172.21.15.186:3300/0 conn(0x7f06c80b5c00 0x7f06c80d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.707+0000 7f06c7fff640 1 -- 172.21.15.16:0/841796808 >> v2:172.21.15.186:3300/0 conn(0x7f06c80b5c00 msgr2=0x7f06c80d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.707+0000 7f06c7fff640 1 --2- 172.21.15.16:0/841796808 >> v2:172.21.15.186:3300/0 conn(0x7f06c80b5c00 0x7f06c80d2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.707+0000 7f06c7fff640 1 -- 172.21.15.16:0/841796808 >> v1:172.21.15.186:6789/0 conn(0x7f06c80b97c0 legacy=0x7f06c80b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.707+0000 7f06c7fff640 1 -- 172.21.15.16:0/841796808 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c80d1c70 con 0x7f06c815dc80 2024-04-25T10:06:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.707+0000 7f06c7fff640 1 --2- 172.21.15.16:0/841796808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c815dc80 0x7f06c8161870 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f06c80b6300 tx=0x7f06b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7af95d29295995 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.707+0000 7f06c6ffd640 1 -- 172.21.15.16:0/841796808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0037070 con 0x7f06c815dc80 2024-04-25T10:06:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.707+0000 7f06c6ffd640 1 -- 172.21.15.16:0/841796808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b0032440 con 0x7f06c815dc80 2024-04-25T10:06:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.707+0000 7f06c6ffd640 1 -- 172.21.15.16:0/841796808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0032810 con 0x7f06c815dc80 2024-04-25T10:06:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.708+0000 7f06ce97e640 1 -- 172.21.15.16:0/841796808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c815dc80 msgr2=0x7f06c8161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.708+0000 7f06ce97e640 1 --2- 172.21.15.16:0/841796808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c815dc80 0x7f06c8161870 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f06c80b6300 tx=0x7f06b002a270 comp rx=0 tx=0).stop 2024-04-25T10:06:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.708+0000 7f06ce97e640 1 -- 172.21.15.16:0/841796808 shutdown_connections 2024-04-25T10:06:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.708+0000 7f06ce97e640 1 --2- 172.21.15.16:0/841796808 >> v2:172.21.15.186:3300/0 conn(0x7f06c80b5c00 0x7f06c80d2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.708+0000 7f06ce97e640 1 --2- 172.21.15.16:0/841796808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c815dc80 0x7f06c8161870 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.708+0000 7f06ce97e640 1 -- 172.21.15.16:0/841796808 >> 172.21.15.16:0/841796808 conn(0x7f06c80d04f0 msgr2=0x7f06c80d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.708+0000 7f06ce97e640 1 -- 172.21.15.16:0/841796808 shutdown_connections 2024-04-25T10:06:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.708+0000 7f06ce97e640 1 -- 172.21.15.16:0/841796808 wait complete. 2024-04-25T10:06:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.709+0000 7f06ce97e640 1 Processor -- start 2024-04-25T10:06:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.709+0000 7f06ce97e640 1 -- start start 2024-04-25T10:06:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.710+0000 7f06ce97e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f06c80b5c00 0x7f06c81e9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.710+0000 7f06ce97e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06c80b97c0 0x7f06c815e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.710+0000 7f06ce97e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c815dc80 0x7f06c814be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.710+0000 7f06ce97e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c8161f30 con 0x7f06c815dc80 2024-04-25T10:06:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.710+0000 7f06ce97e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c8161db0 con 0x7f06c80b97c0 2024-04-25T10:06:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.710+0000 7f06ce97e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c80d29b0 con 0x7f06c80b5c00 2024-04-25T10:06:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.710+0000 7f06ccf1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c815dc80 0x7f06c814be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.710+0000 7f06ccf1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c815dc80 0x7f06c814be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34584/0 (socket says 172.21.15.16:34584) 2024-04-25T10:06:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.710+0000 7f06ccf1b640 1 -- 172.21.15.16:0/3384402452 learned_addr learned my addr 172.21.15.16:0/3384402452 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.710+0000 7f06c77fe640 1 --2- 172.21.15.16:0/3384402452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06c80b97c0 0x7f06c815e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.710+0000 7f06c7fff640 1 --2- 172.21.15.16:0/3384402452 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f06c80b5c00 0x7f06c81e9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.710+0000 7f06ccf1b640 1 -- 172.21.15.16:0/3384402452 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f06c80b5c00 msgr2=0x7f06c81e9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.710+0000 7f06ccf1b640 1 --2- 172.21.15.16:0/3384402452 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f06c80b5c00 0x7f06c81e9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.710+0000 7f06ccf1b640 1 -- 172.21.15.16:0/3384402452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06c80b97c0 msgr2=0x7f06c815e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.710+0000 7f06ccf1b640 1 --2- 172.21.15.16:0/3384402452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06c80b97c0 0x7f06c815e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.710+0000 7f06ccf1b640 1 -- 172.21.15.16:0/3384402452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c81e92b0 con 0x7f06c815dc80 2024-04-25T10:06:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.711+0000 7f06c7fff640 1 --2- 172.21.15.16:0/3384402452 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f06c80b5c00 0x7f06c81e9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.711+0000 7f06c77fe640 1 --2- 172.21.15.16:0/3384402452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06c80b97c0 0x7f06c815e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.711+0000 7f06ccf1b640 1 --2- 172.21.15.16:0/3384402452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c815dc80 0x7f06c814be60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f06c0007e40 tx=0x7f06c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.711+0000 7f06c4ff9640 1 -- 172.21.15.16:0/3384402452 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c0014620 con 0x7f06c815dc80 2024-04-25T10:06:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.711+0000 7f06c4ff9640 1 -- 172.21.15.16:0/3384402452 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c0019070 con 0x7f06c815dc80 2024-04-25T10:06:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.711+0000 7f06ce97e640 1 -- 172.21.15.16:0/3384402452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c815e7b0 con 0x7f06c815dc80 2024-04-25T10:06:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.711+0000 7f06c4ff9640 1 -- 172.21.15.16:0/3384402452 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c0014a20 con 0x7f06c815dc80 2024-04-25T10:06:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.711+0000 7f06ce97e640 1 -- 172.21.15.16:0/3384402452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c81e9550 con 0x7f06c815dc80 2024-04-25T10:06:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.712+0000 7f06ce97e640 1 -- 172.21.15.16:0/3384402452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c80750b0 con 0x7f06c815dc80 2024-04-25T10:06:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.712+0000 7f06c4ff9640 1 -- 172.21.15.16:0/3384402452 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06c0014bc0 con 0x7f06c815dc80 2024-04-25T10:06:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.713+0000 7f06c4ff9640 1 --2- 172.21.15.16:0/3384402452 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f06a806b500 0x7f06a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.713+0000 7f06c4ff9640 1 -- 172.21.15.16:0/3384402452 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f06c008a960 con 0x7f06c815dc80 2024-04-25T10:06:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.713+0000 7f06c7fff640 1 --2- 172.21.15.16:0/3384402452 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f06a806b500 0x7f06a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.713+0000 7f06c7fff640 1 --2- 172.21.15.16:0/3384402452 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f06a806b500 0x7f06a806d9c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f06b0034000 tx=0x7f06b002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.717+0000 7f06c4ff9640 1 -- 172.21.15.16:0/3384402452 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06c00598d0 con 0x7f06c815dc80 2024-04-25T10:06:10.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:10.937+0000 7f06ce97e640 1 -- 172.21.15.16:0/3384402452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f06c80d1780 con 0x7f06c815dc80 2024-04-25T10:06:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.571+0000 7f06c4ff9640 1 -- 172.21.15.16:0/3384402452 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f06c00591d0 con 0x7f06c815dc80 2024-04-25T10:06:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 4 and data pool 5 2024-04-25T10:06:11.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.576+0000 7f06ce97e640 1 -- 172.21.15.16:0/3384402452 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f06a806b500 msgr2=0x7f06a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.576+0000 7f06ce97e640 1 --2- 172.21.15.16:0/3384402452 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f06a806b500 0x7f06a806d9c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f06b0034000 tx=0x7f06b002e000 comp rx=0 tx=0).stop 2024-04-25T10:06:11.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.576+0000 7f06ce97e640 1 -- 172.21.15.16:0/3384402452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c815dc80 msgr2=0x7f06c814be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.576+0000 7f06ce97e640 1 --2- 172.21.15.16:0/3384402452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c815dc80 0x7f06c814be60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f06c0007e40 tx=0x7f06c000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.577+0000 7f06ce97e640 1 -- 172.21.15.16:0/3384402452 shutdown_connections 2024-04-25T10:06:11.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.577+0000 7f06ce97e640 1 --2- 172.21.15.16:0/3384402452 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f06a806b500 0x7f06a806d9c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.577+0000 7f06ce97e640 1 --2- 172.21.15.16:0/3384402452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c815dc80 0x7f06c814be60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.577+0000 7f06ce97e640 1 --2- 172.21.15.16:0/3384402452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06c80b97c0 0x7f06c815e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.577+0000 7f06ce97e640 1 --2- 172.21.15.16:0/3384402452 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f06c80b5c00 0x7f06c81e9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.577+0000 7f06ce97e640 1 -- 172.21.15.16:0/3384402452 >> 172.21.15.16:0/3384402452 conn(0x7f06c80d04f0 msgr2=0x7f06c80c95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.580+0000 7f06ce97e640 1 -- 172.21.15.16:0/3384402452 shutdown_connections 2024-04-25T10:06:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.582+0000 7f06ce97e640 1 -- 172.21.15.16:0/3384402452 wait complete. 2024-04-25T10:06:11.612 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:11.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.708+0000 7fed18207640 1 Processor -- start 2024-04-25T10:06:11.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.708+0000 7fed18207640 1 -- start start 2024-04-25T10:06:11.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.709+0000 7fed18207640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fed1015baf0 0x7fed100ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.709+0000 7fed18207640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed10070a10 0x7fed10072b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.709+0000 7fed18207640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed100ce810 con 0x7fed100ce380 2024-04-25T10:06:11.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.709+0000 7fed18207640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed10073210 con 0x7fed1015baf0 2024-04-25T10:06:11.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.709+0000 7fed18207640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed100cecc0 con 0x7fed10070a10 2024-04-25T10:06:11.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.709+0000 7fed157a2640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed10070a10 0x7fed10072b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.709+0000 7fed15fa3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fed1015baf0 0x7fed100ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.709+0000 7fed167a4640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fed100ce380 0x7fed100cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:54724/0 (socket says 172.21.15.16:54724) 2024-04-25T10:06:11.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.709+0000 7fed157a2640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed10070a10 0x7fed10072b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:43150/0 (socket says 172.21.15.16:43150) 2024-04-25T10:06:11.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.709+0000 7fed167a4640 1 -- 172.21.15.16:0/305038363 learned_addr learned my addr 172.21.15.16:0/305038363 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:11.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.709+0000 7fed15fa3640 1 -- 172.21.15.16:0/305038363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed10070a10 msgr2=0x7fed10072b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.710+0000 7fed15fa3640 1 --2- 172.21.15.16:0/305038363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed10070a10 0x7fed10072b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.710+0000 7fed15fa3640 1 -- 172.21.15.16:0/305038363 >> v1:172.21.15.186:6789/0 conn(0x7fed100ce380 legacy=0x7fed100cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.710+0000 7fed14fa1640 1 -- 172.21.15.16:0/305038363 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456340265 0 0) 0x7fed08001490 con 0x7fed100ce380 2024-04-25T10:06:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.710+0000 7fed15fa3640 1 -- 172.21.15.16:0/305038363 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed100cefa0 con 0x7fed1015baf0 2024-04-25T10:06:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.710+0000 7fed157a2640 1 --2- 172.21.15.16:0/305038363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed10070a10 0x7fed10072b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.710+0000 7fed15fa3640 1 --2- 172.21.15.16:0/305038363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed1015baf0 0x7fed100ccb90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0097a0 tx=0x7fed0c02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93f4a27d73df9416 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.710+0000 7fed14fa1640 1 -- 172.21.15.16:0/305038363 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c037070 con 0x7fed1015baf0 2024-04-25T10:06:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.711+0000 7fed14fa1640 1 -- 172.21.15.16:0/305038363 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0c02adb0 con 0x7fed1015baf0 2024-04-25T10:06:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.711+0000 7fed14fa1640 1 -- 172.21.15.16:0/305038363 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c032660 con 0x7fed1015baf0 2024-04-25T10:06:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.711+0000 7fed18207640 1 -- 172.21.15.16:0/305038363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed1015baf0 msgr2=0x7fed100ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.711+0000 7fed18207640 1 --2- 172.21.15.16:0/305038363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed1015baf0 0x7fed100ccb90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0097a0 tx=0x7fed0c02a060 comp rx=0 tx=0).stop 2024-04-25T10:06:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.711+0000 7fed18207640 1 -- 172.21.15.16:0/305038363 shutdown_connections 2024-04-25T10:06:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.711+0000 7fed18207640 1 --2- 172.21.15.16:0/305038363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed10070a10 0x7fed10072b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.711+0000 7fed18207640 1 --2- 172.21.15.16:0/305038363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed1015baf0 0x7fed100ccb90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.711+0000 7fed18207640 1 -- 172.21.15.16:0/305038363 >> 172.21.15.16:0/305038363 conn(0x7fed100b2370 msgr2=0x7fed100b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.711+0000 7fed18207640 1 -- 172.21.15.16:0/305038363 shutdown_connections 2024-04-25T10:06:11.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.711+0000 7fed18207640 1 -- 172.21.15.16:0/305038363 wait complete. 2024-04-25T10:06:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.712+0000 7fed18207640 1 Processor -- start 2024-04-25T10:06:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.712+0000 7fed18207640 1 -- start start 2024-04-25T10:06:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.713+0000 7fed18207640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed10070a10 0x7fed101f1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.713+0000 7fed18207640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed100ce380 0x7fed1014e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.713+0000 7fed18207640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed1015baf0 0x7fed100c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.713+0000 7fed18207640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed100cc0a0 con 0x7fed10070a10 2024-04-25T10:06:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.713+0000 7fed18207640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed100704f0 con 0x7fed100ce380 2024-04-25T10:06:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.713+0000 7fed18207640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed100cc220 con 0x7fed1015baf0 2024-04-25T10:06:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.713+0000 7fed15fa3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed10070a10 0x7fed101f1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.713+0000 7fed15fa3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed10070a10 0x7fed101f1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34592/0 (socket says 172.21.15.16:34592) 2024-04-25T10:06:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.713+0000 7fed15fa3640 1 -- 172.21.15.16:0/3065475402 learned_addr learned my addr 172.21.15.16:0/3065475402 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.713+0000 7fed157a2640 1 --2- 172.21.15.16:0/3065475402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed100ce380 0x7fed1014e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.713+0000 7fed167a4640 1 --2- 172.21.15.16:0/3065475402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed1015baf0 0x7fed100c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.714+0000 7fed15fa3640 1 -- 172.21.15.16:0/3065475402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed1015baf0 msgr2=0x7fed100c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.714+0000 7fed15fa3640 1 --2- 172.21.15.16:0/3065475402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed1015baf0 0x7fed100c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.714+0000 7fed15fa3640 1 -- 172.21.15.16:0/3065475402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed100ce380 msgr2=0x7fed1014e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.714+0000 7fed15fa3640 1 --2- 172.21.15.16:0/3065475402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed100ce380 0x7fed1014e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.714+0000 7fed15fa3640 1 -- 172.21.15.16:0/3065475402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1014ef90 con 0x7fed10070a10 2024-04-25T10:06:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.714+0000 7fed15fa3640 1 --2- 172.21.15.16:0/3065475402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed10070a10 0x7fed101f1810 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0098c0 tx=0x7fed0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.714+0000 7fed067fc640 1 -- 172.21.15.16:0/3065475402 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c041030 con 0x7fed10070a10 2024-04-25T10:06:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.714+0000 7fed067fc640 1 -- 172.21.15.16:0/3065475402 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0c02abb0 con 0x7fed10070a10 2024-04-25T10:06:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.714+0000 7fed18207640 1 -- 172.21.15.16:0/3065475402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1014f270 con 0x7fed10070a10 2024-04-25T10:06:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.714+0000 7fed067fc640 1 -- 172.21.15.16:0/3065475402 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c037040 con 0x7fed10070a10 2024-04-25T10:06:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.714+0000 7fed18207640 1 -- 172.21.15.16:0/3065475402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1014f830 con 0x7fed10070a10 2024-04-25T10:06:11.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.715+0000 7fed18207640 1 -- 172.21.15.16:0/3065475402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed100749b0 con 0x7fed10070a10 2024-04-25T10:06:11.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.716+0000 7fed067fc640 1 -- 172.21.15.16:0/3065475402 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed0c044070 con 0x7fed10070a10 2024-04-25T10:06:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.716+0000 7fed067fc640 1 --2- 172.21.15.16:0/3065475402 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fecf406b4d0 0x7fecf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.716+0000 7fed067fc640 1 -- 172.21.15.16:0/3065475402 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed0c0aa8a0 con 0x7fed10070a10 2024-04-25T10:06:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.716+0000 7fed157a2640 1 --2- 172.21.15.16:0/3065475402 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fecf406b4d0 0x7fecf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.717+0000 7fed157a2640 1 --2- 172.21.15.16:0/3065475402 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fecf406b4d0 0x7fecf406d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fed100ca7b0 tx=0x7fed00009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.721+0000 7fed067fc640 1 -- 172.21.15.16:0/3065475402 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed0c079100 con 0x7fed10070a10 2024-04-25T10:06:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.902+0000 7fed18207640 1 -- 172.21.15.16:0/3065475402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed100d1460 con 0x7fed10070a10 2024-04-25T10:06:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.903+0000 7fed067fc640 1 -- 172.21.15.16:0/3065475402 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16622 (secure 0 0 0) 0x7fed0c07d1a0 con 0x7fed10070a10 2024-04-25T10:06:11.910 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:11.910 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":35,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:06:10.940164+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:08.830059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:09.922618+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.905+0000 7fed18207640 1 -- 172.21.15.16:0/3065475402 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fecf406b4d0 msgr2=0x7fecf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.905+0000 7fed18207640 1 --2- 172.21.15.16:0/3065475402 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fecf406b4d0 0x7fecf406d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fed100ca7b0 tx=0x7fed00009190 comp rx=0 tx=0).stop 2024-04-25T10:06:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.905+0000 7fed18207640 1 -- 172.21.15.16:0/3065475402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed10070a10 msgr2=0x7fed101f1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.905+0000 7fed18207640 1 --2- 172.21.15.16:0/3065475402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed10070a10 0x7fed101f1810 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0098c0 tx=0x7fed0c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.905+0000 7fed18207640 1 -- 172.21.15.16:0/3065475402 shutdown_connections 2024-04-25T10:06:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.905+0000 7fed18207640 1 --2- 172.21.15.16:0/3065475402 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fecf406b4d0 0x7fecf406d990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.905+0000 7fed18207640 1 --2- 172.21.15.16:0/3065475402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed1015baf0 0x7fed100c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.905+0000 7fed18207640 1 --2- 172.21.15.16:0/3065475402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed100ce380 0x7fed1014e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.905+0000 7fed18207640 1 --2- 172.21.15.16:0/3065475402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed10070a10 0x7fed101f1810 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.906+0000 7fed18207640 1 -- 172.21.15.16:0/3065475402 >> 172.21.15.16:0/3065475402 conn(0x7fed100b2370 msgr2=0x7fed1015e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.906+0000 7fed18207640 1 -- 172.21.15.16:0/3065475402 shutdown_connections 2024-04-25T10:06:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:11.906+0000 7fed18207640 1 -- 172.21.15.16:0/3065475402 wait complete. 2024-04-25T10:06:11.923 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.017+0000 7f567b742640 1 Processor -- start 2024-04-25T10:06:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.017+0000 7f567b742640 1 -- start start 2024-04-25T10:06:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.018+0000 7f567b742640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f567409a150 0x7f567406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.018+0000 7f567b742640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f567406c1f0 0x7f56740cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.018+0000 7f567b742640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f567406c680 con 0x7f567415ba10 2024-04-25T10:06:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.018+0000 7f567b742640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56740cbe50 con 0x7f567406c1f0 2024-04-25T10:06:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.018+0000 7f567b742640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56740cfe80 con 0x7f567409a150 2024-04-25T10:06:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.018+0000 7f5679cdf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f567406c1f0 0x7f56740cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.018+0000 7f56794de640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f567415ba10 0x7f5674099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38544/0 (socket says 172.21.15.16:38544) 2024-04-25T10:06:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.018+0000 7f56794de640 1 -- 172.21.15.16:0/3140801779 learned_addr learned my addr 172.21.15.16:0/3140801779 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.018+0000 7f5678cdd640 1 --2- 172.21.15.16:0/3140801779 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f567409a150 0x7f567406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.019+0000 7f5679cdf640 1 -- 172.21.15.16:0/3140801779 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f567409a150 msgr2=0x7f567406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.019+0000 7f5679cdf640 1 --2- 172.21.15.16:0/3140801779 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f567409a150 0x7f567406bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.019+0000 7f5679cdf640 1 -- 172.21.15.16:0/3140801779 >> v1:172.21.15.16:6789/0 conn(0x7f567415ba10 legacy=0x7f5674099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.019+0000 7f5679cdf640 1 -- 172.21.15.16:0/3140801779 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567406c900 con 0x7f567406c1f0 2024-04-25T10:06:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.019+0000 7f566bfff640 1 -- 172.21.15.16:0/3140801779 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445759978 0 0) 0x7f5670001110 con 0x7f567415ba10 2024-04-25T10:06:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.019+0000 7f5679cdf640 1 --2- 172.21.15.16:0/3140801779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567406c1f0 0x7f56740cd220 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f566c009d90 tx=0x7f566c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea10de0e4a8d92b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.019+0000 7f566bfff640 1 -- 172.21.15.16:0/3140801779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c007740 con 0x7f567406c1f0 2024-04-25T10:06:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.019+0000 7f566bfff640 1 -- 172.21.15.16:0/3140801779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c0078e0 con 0x7f567406c1f0 2024-04-25T10:06:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.019+0000 7f566bfff640 1 -- 172.21.15.16:0/3140801779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c007cb0 con 0x7f567406c1f0 2024-04-25T10:06:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.020+0000 7f567b742640 1 -- 172.21.15.16:0/3140801779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567406c1f0 msgr2=0x7f56740cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.020+0000 7f567b742640 1 --2- 172.21.15.16:0/3140801779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567406c1f0 0x7f56740cd220 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f566c009d90 tx=0x7f566c02a660 comp rx=0 tx=0).stop 2024-04-25T10:06:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.020+0000 7f567b742640 1 -- 172.21.15.16:0/3140801779 shutdown_connections 2024-04-25T10:06:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.020+0000 7f567b742640 1 --2- 172.21.15.16:0/3140801779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567406c1f0 0x7f56740cd220 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.020+0000 7f567b742640 1 --2- 172.21.15.16:0/3140801779 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f567409a150 0x7f567406bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.020+0000 7f567b742640 1 -- 172.21.15.16:0/3140801779 >> 172.21.15.16:0/3140801779 conn(0x7f56740b32f0 msgr2=0x7f56740b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.020+0000 7f567b742640 1 -- 172.21.15.16:0/3140801779 shutdown_connections 2024-04-25T10:06:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.020+0000 7f567b742640 1 -- 172.21.15.16:0/3140801779 wait complete. 2024-04-25T10:06:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.021+0000 7f567b742640 1 Processor -- start 2024-04-25T10:06:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.021+0000 7f567b742640 1 -- start start 2024-04-25T10:06:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.022+0000 7f567b742640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f567406c1f0 0x7f56741f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.022+0000 7f567b742640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f567409a150 0x7f567415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.022+0000 7f567b742640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567415ba10 0x7f5674161960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.022+0000 7f567b742640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56740cd760 con 0x7f567415ba10 2024-04-25T10:06:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.022+0000 7f567b742640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567406cec0 con 0x7f567406c1f0 2024-04-25T10:06:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.022+0000 7f567b742640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56740cd8e0 con 0x7f567409a150 2024-04-25T10:06:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.022+0000 7f5679cdf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567415ba10 0x7f5674161960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.022+0000 7f5679cdf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567415ba10 0x7f5674161960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34606/0 (socket says 172.21.15.16:34606) 2024-04-25T10:06:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.022+0000 7f5679cdf640 1 -- 172.21.15.16:0/3518429938 learned_addr learned my addr 172.21.15.16:0/3518429938 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.022+0000 7f56794de640 1 --2- 172.21.15.16:0/3518429938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f567406c1f0 0x7f56741f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.022+0000 7f5678cdd640 1 --2- 172.21.15.16:0/3518429938 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f567409a150 0x7f567415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.022+0000 7f5679cdf640 1 -- 172.21.15.16:0/3518429938 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f567409a150 msgr2=0x7f567415bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.022+0000 7f5679cdf640 1 --2- 172.21.15.16:0/3518429938 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f567409a150 0x7f567415bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.022+0000 7f5679cdf640 1 -- 172.21.15.16:0/3518429938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f567406c1f0 msgr2=0x7f56741f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.022+0000 7f5679cdf640 1 --2- 172.21.15.16:0/3518429938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f567406c1f0 0x7f56741f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.022+0000 7f5679cdf640 1 -- 172.21.15.16:0/3518429938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56740c55e0 con 0x7f567415ba10 2024-04-25T10:06:12.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.023+0000 7f5679cdf640 1 --2- 172.21.15.16:0/3518429938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567415ba10 0x7f5674161960 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f566c009d90 tx=0x7f566c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.023+0000 7f5669ffb640 1 -- 172.21.15.16:0/3518429938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c040070 con 0x7f567415ba10 2024-04-25T10:06:12.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.023+0000 7f5669ffb640 1 -- 172.21.15.16:0/3518429938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c031c40 con 0x7f567415ba10 2024-04-25T10:06:12.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.023+0000 7f567b742640 1 -- 172.21.15.16:0/3518429938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56741f7d10 con 0x7f567415ba10 2024-04-25T10:06:12.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.023+0000 7f5669ffb640 1 -- 172.21.15.16:0/3518429938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c009450 con 0x7f567415ba10 2024-04-25T10:06:12.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.023+0000 7f567b742640 1 -- 172.21.15.16:0/3518429938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56741f82d0 con 0x7f567415ba10 2024-04-25T10:06:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.024+0000 7f567b742640 1 -- 172.21.15.16:0/3518429938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56740749b0 con 0x7f567415ba10 2024-04-25T10:06:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.024+0000 7f5669ffb640 1 -- 172.21.15.16:0/3518429938 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f566c039040 con 0x7f567415ba10 2024-04-25T10:06:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.025+0000 7f5669ffb640 1 --2- 172.21.15.16:0/3518429938 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f565806b4d0 0x7f565806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.025+0000 7f5669ffb640 1 -- 172.21.15.16:0/3518429938 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f566c0ab140 con 0x7f567415ba10 2024-04-25T10:06:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.025+0000 7f56794de640 1 --2- 172.21.15.16:0/3518429938 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f565806b4d0 0x7f565806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.026+0000 7f56794de640 1 --2- 172.21.15.16:0/3518429938 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f565806b4d0 0x7f565806d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5670007fd0 tx=0x7f5670007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.029+0000 7f5669ffb640 1 -- 172.21.15.16:0/3518429938 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f566c079970 con 0x7f567415ba10 2024-04-25T10:06:12.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.208+0000 7f567b742640 1 -- 172.21.15.16:0/3518429938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f567409a570 con 0x7f567415ba10 2024-04-25T10:06:12.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.209+0000 7f5669ffb640 1 -- 172.21.15.16:0/3518429938 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16622 (secure 0 0 0) 0x7f566c07da10 con 0x7f567415ba10 2024-04-25T10:06:12.216 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:12.216 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":35,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:06:10.940164+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:08.830059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:09.922618+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.211+0000 7f567b742640 1 -- 172.21.15.16:0/3518429938 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f565806b4d0 msgr2=0x7f565806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.211+0000 7f567b742640 1 --2- 172.21.15.16:0/3518429938 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f565806b4d0 0x7f565806d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5670007fd0 tx=0x7f5670007de0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.211+0000 7f567b742640 1 -- 172.21.15.16:0/3518429938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567415ba10 msgr2=0x7f5674161960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.211+0000 7f567b742640 1 --2- 172.21.15.16:0/3518429938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567415ba10 0x7f5674161960 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f566c009d90 tx=0x7f566c004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.211+0000 7f567b742640 1 -- 172.21.15.16:0/3518429938 shutdown_connections 2024-04-25T10:06:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.211+0000 7f567b742640 1 --2- 172.21.15.16:0/3518429938 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f565806b4d0 0x7f565806d990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.211+0000 7f567b742640 1 --2- 172.21.15.16:0/3518429938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567415ba10 0x7f5674161960 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.211+0000 7f567b742640 1 --2- 172.21.15.16:0/3518429938 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f567409a150 0x7f567415bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.211+0000 7f567b742640 1 --2- 172.21.15.16:0/3518429938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f567406c1f0 0x7f56741f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.211+0000 7f567b742640 1 -- 172.21.15.16:0/3518429938 >> 172.21.15.16:0/3518429938 conn(0x7f56740b32f0 msgr2=0x7f5674070d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.211+0000 7f567b742640 1 -- 172.21.15.16:0/3518429938 shutdown_connections 2024-04-25T10:06:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.211+0000 7f567b742640 1 -- 172.21.15.16:0/3518429938 wait complete. 2024-04-25T10:06:12.229 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:06:12.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.328+0000 7f0601122640 1 Processor -- start 2024-04-25T10:06:12.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.328+0000 7f0601122640 1 -- start start 2024-04-25T10:06:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.329+0000 7f0601122640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc0d3b90 0x7f05fc0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.329+0000 7f0601122640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f05fc0d0f60 0x7f05fc0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.329+0000 7f0601122640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05fc0d4090 con 0x7f05fc15dca0 2024-04-25T10:06:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.329+0000 7f0601122640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05fc0b6b20 con 0x7f05fc0d0f60 2024-04-25T10:06:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.329+0000 7f0601122640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0d4370 con 0x7f05fc0d3b90 2024-04-25T10:06:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.329+0000 7f05fa59c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc0d3b90 0x7f05fc0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.329+0000 7f05fb59e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f05fc0d0f60 0x7f05fc0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.329+0000 7f05fad9d640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f05fc15dca0 0x7f05fc0ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:54730/0 (socket says 172.21.15.16:54730) 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.329+0000 7f05fa59c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc0d3b90 0x7f05fc0d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:43190/0 (socket says 172.21.15.16:43190) 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.329+0000 7f05fad9d640 1 -- 172.21.15.16:0/3928125449 learned_addr learned my addr 172.21.15.16:0/3928125449 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.329+0000 7f05f9d9b640 1 -- 172.21.15.16:0/3928125449 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1000284839 0 0) 0x7f05fc0d4090 con 0x7f05fc15dca0 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.329+0000 7f05fb59e640 1 -- 172.21.15.16:0/3928125449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc0d3b90 msgr2=0x7f05fc0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.329+0000 7f05fb59e640 1 --2- 172.21.15.16:0/3928125449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc0d3b90 0x7f05fc0d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.329+0000 7f05fb59e640 1 -- 172.21.15.16:0/3928125449 >> v1:172.21.15.186:6789/0 conn(0x7f05fc15dca0 legacy=0x7f05fc0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.329+0000 7f05fb59e640 1 -- 172.21.15.16:0/3928125449 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc0d4650 con 0x7f05fc0d0f60 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.329+0000 7f05fa59c640 1 --2- 172.21.15.16:0/3928125449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc0d3b90 0x7f05fc0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:12.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.329+0000 7f05fb59e640 1 --2- 172.21.15.16:0/3928125449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05fc0d0f60 0x7f05fc0b2500 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009d90 tx=0x7f05ec02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f682622bf9efeed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.330+0000 7f05f9d9b640 1 -- 172.21.15.16:0/3928125449 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec007740 con 0x7f05fc0d0f60 2024-04-25T10:06:12.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.330+0000 7f05f9d9b640 1 -- 172.21.15.16:0/3928125449 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ec0078e0 con 0x7f05fc0d0f60 2024-04-25T10:06:12.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.330+0000 7f05f9d9b640 1 -- 172.21.15.16:0/3928125449 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec007cb0 con 0x7f05fc0d0f60 2024-04-25T10:06:12.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.330+0000 7f0601122640 1 -- 172.21.15.16:0/3928125449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05fc0d0f60 msgr2=0x7f05fc0b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.330+0000 7f0601122640 1 --2- 172.21.15.16:0/3928125449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05fc0d0f60 0x7f05fc0b2500 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009d90 tx=0x7f05ec02a660 comp rx=0 tx=0).stop 2024-04-25T10:06:12.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.330+0000 7f0601122640 1 -- 172.21.15.16:0/3928125449 shutdown_connections 2024-04-25T10:06:12.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.330+0000 7f0601122640 1 --2- 172.21.15.16:0/3928125449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc0d3b90 0x7f05fc0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.330+0000 7f0601122640 1 --2- 172.21.15.16:0/3928125449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05fc0d0f60 0x7f05fc0b2500 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.330+0000 7f0601122640 1 -- 172.21.15.16:0/3928125449 >> 172.21.15.16:0/3928125449 conn(0x7f05fc0726c0 msgr2=0x7f05fc072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.330+0000 7f0601122640 1 -- 172.21.15.16:0/3928125449 shutdown_connections 2024-04-25T10:06:12.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.330+0000 7f0601122640 1 -- 172.21.15.16:0/3928125449 wait complete. 2024-04-25T10:06:12.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.332+0000 7f0601122640 1 Processor -- start 2024-04-25T10:06:12.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.332+0000 7f0601122640 1 -- start start 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.332+0000 7f0601122640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05fc0d0f60 0x7f05fc14a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.332+0000 7f0601122640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05fc0d3b90 0x7f05fc159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.332+0000 7f0601122640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc15dca0 0x7f05fc1492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.332+0000 7f0601122640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc15e080 con 0x7f05fc0d0f60 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.332+0000 7f0601122640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0cf050 con 0x7f05fc0d3b90 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.332+0000 7f0601122640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc15e200 con 0x7f05fc15dca0 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.333+0000 7f05fad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05fc0d0f60 0x7f05fc14a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.333+0000 7f05fad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05fc0d0f60 0x7f05fc14a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34620/0 (socket says 172.21.15.16:34620) 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.333+0000 7f05fad9d640 1 -- 172.21.15.16:0/434856898 learned_addr learned my addr 172.21.15.16:0/434856898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.333+0000 7f05fa59c640 1 --2- 172.21.15.16:0/434856898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05fc0d3b90 0x7f05fc159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.333+0000 7f05fb59e640 1 --2- 172.21.15.16:0/434856898 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc15dca0 0x7f05fc1492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.333+0000 7f05fad9d640 1 -- 172.21.15.16:0/434856898 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc15dca0 msgr2=0x7f05fc1492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.333+0000 7f05fad9d640 1 --2- 172.21.15.16:0/434856898 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc15dca0 0x7f05fc1492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.333+0000 7f05fad9d640 1 -- 172.21.15.16:0/434856898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05fc0d3b90 msgr2=0x7f05fc159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.333+0000 7f05fad9d640 1 --2- 172.21.15.16:0/434856898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05fc0d3b90 0x7f05fc159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.333+0000 7f05fad9d640 1 -- 172.21.15.16:0/434856898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc14ac40 con 0x7f05fc0d0f60 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.333+0000 7f05fad9d640 1 --2- 172.21.15.16:0/434856898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05fc0d0f60 0x7f05fc14a700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f05f400c430 tx=0x7f05f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.333+0000 7f05f37fe640 1 -- 172.21.15.16:0/434856898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f40074e0 con 0x7f05fc0d0f60 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.333+0000 7f05f37fe640 1 -- 172.21.15.16:0/434856898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f40047e0 con 0x7f05fc0d0f60 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.333+0000 7f05f37fe640 1 -- 172.21.15.16:0/434856898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4004bd0 con 0x7f05fc0d0f60 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.333+0000 7f0601122640 1 -- 172.21.15.16:0/434856898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc148200 con 0x7f05fc0d0f60 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.333+0000 7f0601122640 1 -- 172.21.15.16:0/434856898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc148aa0 con 0x7f05fc0d0f60 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.334+0000 7f0601122640 1 -- 172.21.15.16:0/434856898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05fc0750b0 con 0x7f05fc0d0f60 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.335+0000 7f05f37fe640 1 -- 172.21.15.16:0/434856898 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05f4011070 con 0x7f05fc0d0f60 2024-04-25T10:06:12.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.335+0000 7f05f37fe640 1 --2- 172.21.15.16:0/434856898 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f05dc06b4d0 0x7f05dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.335+0000 7f05f37fe640 1 -- 172.21.15.16:0/434856898 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05f4089d00 con 0x7f05fc0d0f60 2024-04-25T10:06:12.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.335+0000 7f05fa59c640 1 --2- 172.21.15.16:0/434856898 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f05dc06b4d0 0x7f05dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.336+0000 7f05fa59c640 1 --2- 172.21.15.16:0/434856898 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f05dc06b4d0 0x7f05dc06d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009560 tx=0x7f05e4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.340+0000 7f05f37fe640 1 -- 172.21.15.16:0/434856898 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f05fc14ac40 con 0x7f05fc0d0f60 2024-04-25T10:06:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.557+0000 7f0601122640 1 -- 172.21.15.16:0/434856898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f05fc15d2a0 con 0x7f05fc0d0f60 2024-04-25T10:06:12.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.581+0000 7f05f37fe640 1 -- 172.21.15.16:0/434856898 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) v1 ==== 118+0+0 (secure 0 0 0) 0x7f05f4058530 con 0x7f05fc0d0f60 2024-04-25T10:06:12.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.584+0000 7f0601122640 1 -- 172.21.15.16:0/434856898 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f05dc06b4d0 msgr2=0x7f05dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.584+0000 7f0601122640 1 --2- 172.21.15.16:0/434856898 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f05dc06b4d0 0x7f05dc06d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009560 tx=0x7f05e4009190 comp rx=0 tx=0).stop 2024-04-25T10:06:12.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.584+0000 7f0601122640 1 -- 172.21.15.16:0/434856898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05fc0d0f60 msgr2=0x7f05fc14a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.584+0000 7f0601122640 1 --2- 172.21.15.16:0/434856898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05fc0d0f60 0x7f05fc14a700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f05f400c430 tx=0x7f05f400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.584+0000 7f0601122640 1 -- 172.21.15.16:0/434856898 shutdown_connections 2024-04-25T10:06:12.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.584+0000 7f0601122640 1 --2- 172.21.15.16:0/434856898 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f05dc06b4d0 0x7f05dc06d990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.584+0000 7f0601122640 1 --2- 172.21.15.16:0/434856898 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc15dca0 0x7f05fc1492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.584+0000 7f0601122640 1 --2- 172.21.15.16:0/434856898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05fc0d3b90 0x7f05fc159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.584+0000 7f0601122640 1 --2- 172.21.15.16:0/434856898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05fc0d0f60 0x7f05fc14a700 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.584+0000 7f0601122640 1 -- 172.21.15.16:0/434856898 >> 172.21.15.16:0/434856898 conn(0x7f05fc0726c0 msgr2=0x7f05fc0b88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.584+0000 7f0601122640 1 -- 172.21.15.16:0/434856898 shutdown_connections 2024-04-25T10:06:12.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.584+0000 7f0601122640 1 -- 172.21.15.16:0/434856898 wait complete. 2024-04-25T10:06:12.602 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:06:12.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:06:12.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.699+0000 7faa4981f640 1 Processor -- start 2024-04-25T10:06:12.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.700+0000 7faa4981f640 1 -- start start 2024-04-25T10:06:12.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.700+0000 7faa4981f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440b5580 0x7faa440b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.700+0000 7faa4981f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa4406c4b0 con 0x7faa440b24c0 2024-04-25T10:06:12.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.700+0000 7faa4981f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa4406c790 con 0x7faa440b98b0 2024-04-25T10:06:12.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.700+0000 7faa4981f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4415bb80 con 0x7faa440b5580 2024-04-25T10:06:12.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.700+0000 7faa437fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7faa440b24c0 0x7faa440b2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56500/0 (socket says 172.21.15.16:56500) 2024-04-25T10:06:12.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.700+0000 7faa437fe640 1 -- 172.21.15.16:0/3762968507 learned_addr learned my addr 172.21.15.16:0/3762968507 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:12.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa427fc640 1 --2- 172.21.15.16:0/3762968507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440b5580 0x7faa440b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa41ffb640 1 -- 172.21.15.16:0/3762968507 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4273287784 0 0) 0x7faa4406c4b0 con 0x7faa440b24c0 2024-04-25T10:06:12.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa41ffb640 1 -- 172.21.15.16:0/3762968507 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa30003520 con 0x7faa440b24c0 2024-04-25T10:06:12.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa41ffb640 1 -- 172.21.15.16:0/3762968507 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3639921005 0 0) 0x7faa4406c790 con 0x7faa440b98b0 2024-04-25T10:06:12.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa41ffb640 1 -- 172.21.15.16:0/3762968507 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa4406c4b0 con 0x7faa440b98b0 2024-04-25T10:06:12.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa41ffb640 1 -- 172.21.15.16:0/3762968507 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2414338838 0 0) 0x7faa30003520 con 0x7faa440b24c0 2024-04-25T10:06:12.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa41ffb640 1 -- 172.21.15.16:0/3762968507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440b5580 msgr2=0x7faa440b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa41ffb640 1 --2- 172.21.15.16:0/3762968507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440b5580 0x7faa440b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa41ffb640 1 -- 172.21.15.16:0/3762968507 >> v1:172.21.15.186:6789/0 conn(0x7faa440b98b0 legacy=0x7faa4415eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa41ffb640 1 -- 172.21.15.16:0/3762968507 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4406ca70 con 0x7faa440b24c0 2024-04-25T10:06:12.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa41ffb640 1 -- 172.21.15.16:0/3762968507 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7faa340019f0 con 0x7faa440b24c0 2024-04-25T10:06:12.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa41ffb640 1 -- 172.21.15.16:0/3762968507 >> v1:172.21.15.16:6789/0 conn(0x7faa440b24c0 legacy=0x7faa440b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa41ffb640 1 --2- 172.21.15.16:0/3762968507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440b5580 0x7faa30003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa41ffb640 1 --2- 172.21.15.16:0/3762968507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa300040d0 0x7faa30004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa41ffb640 1 --2- 172.21.15.16:0/3762968507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa30004d40 0x7faa300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa41ffb640 1 -- 172.21.15.16:0/3762968507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4415bb80 con 0x7faa300040d0 2024-04-25T10:06:12.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa41ffb640 1 -- 172.21.15.16:0/3762968507 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3000bbd0 con 0x7faa30004d40 2024-04-25T10:06:12.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa41ffb640 1 -- 172.21.15.16:0/3762968507 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3000bdd0 con 0x7faa440b5580 2024-04-25T10:06:12.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa427fc640 1 --2- 172.21.15.16:0/3762968507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa300040d0 0x7faa30004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa42ffd640 1 --2- 172.21.15.16:0/3762968507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440b5580 0x7faa30003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.701+0000 7faa437fe640 1 --2- 172.21.15.16:0/3762968507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa30004d40 0x7faa300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.702+0000 7faa427fc640 1 -- 172.21.15.16:0/3762968507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440b5580 msgr2=0x7faa30003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.702+0000 7faa427fc640 1 --2- 172.21.15.16:0/3762968507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440b5580 0x7faa30003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.702+0000 7faa427fc640 1 -- 172.21.15.16:0/3762968507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa30004d40 msgr2=0x7faa300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.702+0000 7faa427fc640 1 --2- 172.21.15.16:0/3762968507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa30004d40 0x7faa300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.702+0000 7faa427fc640 1 -- 172.21.15.16:0/3762968507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa30003520 con 0x7faa300040d0 2024-04-25T10:06:12.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.702+0000 7faa427fc640 1 --2- 172.21.15.16:0/3762968507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa300040d0 0x7faa30004800 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009ac0 tx=0x7faa30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e961987f674314e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.702+0000 7faa41ffb640 1 -- 172.21.15.16:0/3762968507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c036070 con 0x7faa300040d0 2024-04-25T10:06:12.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.702+0000 7faa41ffb640 1 -- 172.21.15.16:0/3762968507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2c031400 con 0x7faa300040d0 2024-04-25T10:06:12.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.702+0000 7faa41ffb640 1 -- 172.21.15.16:0/3762968507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c009690 con 0x7faa300040d0 2024-04-25T10:06:12.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.703+0000 7faa4981f640 1 -- 172.21.15.16:0/3762968507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa300040d0 msgr2=0x7faa30004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.703+0000 7faa4981f640 1 --2- 172.21.15.16:0/3762968507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa300040d0 0x7faa30004800 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009ac0 tx=0x7faa30005830 comp rx=0 tx=0).stop 2024-04-25T10:06:12.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.703+0000 7faa437fe640 1 -- 172.21.15.16:0/3762968507 reap_dead start 2024-04-25T10:06:12.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.703+0000 7faa4981f640 1 -- 172.21.15.16:0/3762968507 shutdown_connections 2024-04-25T10:06:12.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.703+0000 7faa4981f640 1 -- 172.21.15.16:0/3762968507 >> 172.21.15.16:0/3762968507 conn(0x7faa440d0580 msgr2=0x7faa440d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.703+0000 7faa4981f640 1 -- 172.21.15.16:0/3762968507 shutdown_connections 2024-04-25T10:06:12.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.703+0000 7faa4981f640 1 -- 172.21.15.16:0/3762968507 wait complete. 2024-04-25T10:06:12.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.704+0000 7faa4981f640 1 Processor -- start 2024-04-25T10:06:12.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.704+0000 7faa4981f640 1 -- start start 2024-04-25T10:06:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.705+0000 7faa4981f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa300040d0 0x7faa44153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.705+0000 7faa4981f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa44158180 0x7faa440cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.705+0000 7faa4981f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440b3e90 0x7faa440b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.705+0000 7faa4981f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa440cec40 con 0x7faa44158180 2024-04-25T10:06:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.705+0000 7faa4981f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4406c140 con 0x7faa300040d0 2024-04-25T10:06:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.705+0000 7faa4981f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa440b3060 con 0x7faa440b3e90 2024-04-25T10:06:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.705+0000 7faa427fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa44158180 0x7faa440cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.705+0000 7faa427fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa44158180 0x7faa440cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49436/0 (socket says 172.21.15.16:49436) 2024-04-25T10:06:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.705+0000 7faa427fc640 1 -- 172.21.15.16:0/1830216335 learned_addr learned my addr 172.21.15.16:0/1830216335 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.705+0000 7faa42ffd640 1 --2- 172.21.15.16:0/1830216335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa300040d0 0x7faa44153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.705+0000 7faa437fe640 1 --2- 172.21.15.16:0/1830216335 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440b3e90 0x7faa440b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.705+0000 7faa427fc640 1 -- 172.21.15.16:0/1830216335 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440b3e90 msgr2=0x7faa440b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.705+0000 7faa427fc640 1 --2- 172.21.15.16:0/1830216335 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440b3e90 0x7faa440b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.705+0000 7faa427fc640 1 -- 172.21.15.16:0/1830216335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa300040d0 msgr2=0x7faa44153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.705+0000 7faa427fc640 1 --2- 172.21.15.16:0/1830216335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa300040d0 0x7faa44153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.705+0000 7faa427fc640 1 -- 172.21.15.16:0/1830216335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa440c7980 con 0x7faa44158180 2024-04-25T10:06:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.706+0000 7faa427fc640 1 --2- 172.21.15.16:0/1830216335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa44158180 0x7faa440cb5e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7faa2c034040 tx=0x7faa2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.706+0000 7faa3bfff640 1 -- 172.21.15.16:0/1830216335 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c040020 con 0x7faa44158180 2024-04-25T10:06:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.706+0000 7faa3bfff640 1 -- 172.21.15.16:0/1830216335 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2c003c60 con 0x7faa44158180 2024-04-25T10:06:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.706+0000 7faa4981f640 1 -- 172.21.15.16:0/1830216335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa440cbf50 con 0x7faa44158180 2024-04-25T10:06:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.706+0000 7faa3bfff640 1 -- 172.21.15.16:0/1830216335 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c036040 con 0x7faa44158180 2024-04-25T10:06:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.706+0000 7faa4981f640 1 -- 172.21.15.16:0/1830216335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa440cc510 con 0x7faa44158180 2024-04-25T10:06:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.707+0000 7faa4981f640 1 -- 172.21.15.16:0/1830216335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa440750b0 con 0x7faa44158180 2024-04-25T10:06:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.707+0000 7faa3bfff640 1 -- 172.21.15.16:0/1830216335 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa2c043020 con 0x7faa44158180 2024-04-25T10:06:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.708+0000 7faa3bfff640 1 --2- 172.21.15.16:0/1830216335 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7faa2406b500 0x7faa2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.708+0000 7faa3bfff640 1 -- 172.21.15.16:0/1830216335 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa2c0aa230 con 0x7faa44158180 2024-04-25T10:06:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.708+0000 7faa42ffd640 1 --2- 172.21.15.16:0/1830216335 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7faa2406b500 0x7faa2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.709+0000 7faa42ffd640 1 --2- 172.21.15.16:0/1830216335 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7faa2406b500 0x7faa2406d9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faa44161860 tx=0x7faa3c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.713+0000 7faa3bfff640 1 -- 172.21.15.16:0/1830216335 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa2c04a070 con 0x7faa44158180 2024-04-25T10:06:12.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:12.929+0000 7faa4981f640 1 -- 172.21.15.16:0/1830216335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7faa440d2000 con 0x7faa44158180 2024-04-25T10:06:13.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.593+0000 7faa3bfff640 1 -- 172.21.15.16:0/1830216335 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7faa2c0044e0 con 0x7faa44158180 2024-04-25T10:06:13.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.596+0000 7faa4981f640 1 -- 172.21.15.16:0/1830216335 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7faa2406b500 msgr2=0x7faa2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.596+0000 7faa4981f640 1 --2- 172.21.15.16:0/1830216335 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7faa2406b500 0x7faa2406d9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faa44161860 tx=0x7faa3c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:13.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.596+0000 7faa4981f640 1 -- 172.21.15.16:0/1830216335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa44158180 msgr2=0x7faa440cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.596+0000 7faa4981f640 1 --2- 172.21.15.16:0/1830216335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa44158180 0x7faa440cb5e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7faa2c034040 tx=0x7faa2c004730 comp rx=0 tx=0).stop 2024-04-25T10:06:13.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.596+0000 7faa4981f640 1 -- 172.21.15.16:0/1830216335 shutdown_connections 2024-04-25T10:06:13.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.596+0000 7faa4981f640 1 --2- 172.21.15.16:0/1830216335 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7faa2406b500 0x7faa2406d9c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.596+0000 7faa4981f640 1 --2- 172.21.15.16:0/1830216335 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa440b3e90 0x7faa440b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.596+0000 7faa4981f640 1 --2- 172.21.15.16:0/1830216335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa44158180 0x7faa440cb5e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.596+0000 7faa4981f640 1 --2- 172.21.15.16:0/1830216335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa300040d0 0x7faa44153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.596+0000 7faa4981f640 1 -- 172.21.15.16:0/1830216335 >> 172.21.15.16:0/1830216335 conn(0x7faa440d0580 msgr2=0x7faa440b5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.596+0000 7faa4981f640 1 -- 172.21.15.16:0/1830216335 shutdown_connections 2024-04-25T10:06:13.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.596+0000 7faa4981f640 1 -- 172.21.15.16:0/1830216335 wait complete. 2024-04-25T10:06:13.614 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:06:13.614 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:13.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.715+0000 7fba61476640 1 Processor -- start 2024-04-25T10:06:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.715+0000 7fba61476640 1 -- start start 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.716+0000 7fba61476640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fba5c0d35b0 0x7fba5c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.716+0000 7fba61476640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fba5c0b5990 0x7fba5c0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.716+0000 7fba61476640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba5c15e0c0 con 0x7fba5c15dce0 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.716+0000 7fba61476640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba5c0d2510 con 0x7fba5c0b5990 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.716+0000 7fba61476640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba5c0d27f0 con 0x7fba5c0d35b0 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.716+0000 7fba5b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fba5c0b5990 0x7fba5c0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.716+0000 7fba5affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fba5c15dce0 0x7fba5c0d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56510/0 (socket says 172.21.15.16:56510) 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.716+0000 7fba5affd640 1 -- 172.21.15.16:0/3648652356 learned_addr learned my addr 172.21.15.16:0/3648652356 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.716+0000 7fba53fff640 1 --2- 172.21.15.16:0/3648652356 >> v2:172.21.15.186:3300/0 conn(0x7fba5c0d35b0 0x7fba5c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.716+0000 7fba5b7fe640 1 -- 172.21.15.16:0/3648652356 >> v2:172.21.15.186:3300/0 conn(0x7fba5c0d35b0 msgr2=0x7fba5c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.716+0000 7fba5b7fe640 1 --2- 172.21.15.16:0/3648652356 >> v2:172.21.15.186:3300/0 conn(0x7fba5c0d35b0 0x7fba5c0b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.716+0000 7fba5b7fe640 1 -- 172.21.15.16:0/3648652356 >> v1:172.21.15.16:6789/0 conn(0x7fba5c15dce0 legacy=0x7fba5c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.716+0000 7fba5b7fe640 1 -- 172.21.15.16:0/3648652356 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c0d2ad0 con 0x7fba5c0b5990 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.716+0000 7fba5a7fc640 1 -- 172.21.15.16:0/3648652356 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134983178 0 0) 0x7fba5c15e0c0 con 0x7fba5c15dce0 2024-04-25T10:06:13.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.717+0000 7fba5b7fe640 1 --2- 172.21.15.16:0/3648652356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba5c0b5990 0x7fba5c0b6b00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fba5c0cdfa0 tx=0x7fba5402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bacb2466725bc5ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.717+0000 7fba5a7fc640 1 -- 172.21.15.16:0/3648652356 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba54007740 con 0x7fba5c0b5990 2024-04-25T10:06:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.717+0000 7fba5a7fc640 1 -- 172.21.15.16:0/3648652356 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba540078e0 con 0x7fba5c0b5990 2024-04-25T10:06:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.717+0000 7fba5a7fc640 1 -- 172.21.15.16:0/3648652356 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba54007cb0 con 0x7fba5c0b5990 2024-04-25T10:06:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.718+0000 7fba61476640 1 -- 172.21.15.16:0/3648652356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba5c0b5990 msgr2=0x7fba5c0b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.718+0000 7fba61476640 1 --2- 172.21.15.16:0/3648652356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba5c0b5990 0x7fba5c0b6b00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fba5c0cdfa0 tx=0x7fba5402a680 comp rx=0 tx=0).stop 2024-04-25T10:06:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.718+0000 7fba61476640 1 -- 172.21.15.16:0/3648652356 shutdown_connections 2024-04-25T10:06:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.718+0000 7fba61476640 1 --2- 172.21.15.16:0/3648652356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba5c0b5990 0x7fba5c0b6b00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.718+0000 7fba61476640 1 --2- 172.21.15.16:0/3648652356 >> v2:172.21.15.186:3300/0 conn(0x7fba5c0d35b0 0x7fba5c0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.718+0000 7fba61476640 1 -- 172.21.15.16:0/3648652356 >> 172.21.15.16:0/3648652356 conn(0x7fba5c072900 msgr2=0x7fba5c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.718+0000 7fba61476640 1 -- 172.21.15.16:0/3648652356 shutdown_connections 2024-04-25T10:06:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.718+0000 7fba61476640 1 -- 172.21.15.16:0/3648652356 wait complete. 2024-04-25T10:06:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.719+0000 7fba61476640 1 Processor -- start 2024-04-25T10:06:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.719+0000 7fba61476640 1 -- start start 2024-04-25T10:06:13.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.720+0000 7fba61476640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba5c0b5990 0x7fba5c14dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.720+0000 7fba61476640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba5c0d35b0 0x7fba5c1589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.720+0000 7fba61476640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba5c15dce0 0x7fba5c158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.720+0000 7fba61476640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c0d2160 con 0x7fba5c15dce0 2024-04-25T10:06:13.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.720+0000 7fba61476640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c0d3090 con 0x7fba5c0d35b0 2024-04-25T10:06:13.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.720+0000 7fba61476640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c1f3b40 con 0x7fba5c0b5990 2024-04-25T10:06:13.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.720+0000 7fba5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba5c15dce0 0x7fba5c158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.720+0000 7fba5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba5c15dce0 0x7fba5c158f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49442/0 (socket says 172.21.15.16:49442) 2024-04-25T10:06:13.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.720+0000 7fba5b7fe640 1 -- 172.21.15.16:0/834600361 learned_addr learned my addr 172.21.15.16:0/834600361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:13.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.720+0000 7fba53fff640 1 --2- 172.21.15.16:0/834600361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba5c0d35b0 0x7fba5c1589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.720+0000 7fba5b7fe640 1 -- 172.21.15.16:0/834600361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba5c0b5990 msgr2=0x7fba5c14dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.720+0000 7fba5b7fe640 1 --2- 172.21.15.16:0/834600361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba5c0b5990 0x7fba5c14dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.720+0000 7fba5b7fe640 1 -- 172.21.15.16:0/834600361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba5c0d35b0 msgr2=0x7fba5c1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.720+0000 7fba5b7fe640 1 --2- 172.21.15.16:0/834600361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba5c0d35b0 0x7fba5c1589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.720+0000 7fba5b7fe640 1 -- 172.21.15.16:0/834600361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c1f3e20 con 0x7fba5c15dce0 2024-04-25T10:06:13.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.720+0000 7fba5b7fe640 1 --2- 172.21.15.16:0/834600361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba5c15dce0 0x7fba5c158f00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fba5402ac70 tx=0x7fba54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.721+0000 7fba537fe640 1 -- 172.21.15.16:0/834600361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba54040070 con 0x7fba5c15dce0 2024-04-25T10:06:13.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.721+0000 7fba537fe640 1 -- 172.21.15.16:0/834600361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba54031bc0 con 0x7fba5c15dce0 2024-04-25T10:06:13.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.721+0000 7fba61476640 1 -- 172.21.15.16:0/834600361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba5c1f4100 con 0x7fba5c15dce0 2024-04-25T10:06:13.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.721+0000 7fba537fe640 1 -- 172.21.15.16:0/834600361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba54045460 con 0x7fba5c15dce0 2024-04-25T10:06:13.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.721+0000 7fba61476640 1 -- 172.21.15.16:0/834600361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba5c1f3820 con 0x7fba5c15dce0 2024-04-25T10:06:13.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.722+0000 7fba61476640 1 -- 172.21.15.16:0/834600361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba5c0cf8d0 con 0x7fba5c15dce0 2024-04-25T10:06:13.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.722+0000 7fba537fe640 1 -- 172.21.15.16:0/834600361 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba54039040 con 0x7fba5c15dce0 2024-04-25T10:06:13.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.723+0000 7fba537fe640 1 --2- 172.21.15.16:0/834600361 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fba3c06b4d0 0x7fba3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.723+0000 7fba537fe640 1 -- 172.21.15.16:0/834600361 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba540abc40 con 0x7fba5c15dce0 2024-04-25T10:06:13.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.723+0000 7fba5affd640 1 --2- 172.21.15.16:0/834600361 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fba3c06b4d0 0x7fba3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.724+0000 7fba5affd640 1 --2- 172.21.15.16:0/834600361 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fba3c06b4d0 0x7fba3c06d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fba4c007fd0 tx=0x7fba4c007960 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.727+0000 7fba537fe640 1 -- 172.21.15.16:0/834600361 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba5c0cf8d0 con 0x7fba5c15dce0 2024-04-25T10:06:13.898 INFO:tasks.ceph.mds.a.smithi016.stderr:2024-04-25T10:06:13.892+0000 7fd9788d7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:06:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.943+0000 7fba61476640 1 -- 172.21.15.16:0/834600361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba5c0749b0 con 0x7fba5c15dce0 2024-04-25T10:06:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.944+0000 7fba537fe640 1 -- 172.21.15.16:0/834600361 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fba5407aac0 con 0x7fba5c15dce0 2024-04-25T10:06:13.949 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:13.949 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3381395304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3381395304},{"type":"v1","addr":"172.21.15.186:6835","nonce":3381395304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2429147785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2429147785},{"type":"v1","addr":"172.21.15.186:6837","nonce":2429147785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5142,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2191514045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2191514045},{"type":"v1","addr":"172.21.15.16:6835","nonce":2191514045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:10.940554+0000","modified":"2024-04-25T10:06:12.930243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34613},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34613":{"gid":34613,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.16:6834/1417313138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1417313138},{"type":"v1","addr":"172.21.15.16:6834","nonce":1417313138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 14 2024-04-25T10:06:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.945+0000 7fba61476640 1 -- 172.21.15.16:0/834600361 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fba3c06b4d0 msgr2=0x7fba3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.945+0000 7fba61476640 1 --2- 172.21.15.16:0/834600361 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fba3c06b4d0 0x7fba3c06d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fba4c007fd0 tx=0x7fba4c007960 comp rx=0 tx=0).stop 2024-04-25T10:06:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.945+0000 7fba61476640 1 -- 172.21.15.16:0/834600361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba5c15dce0 msgr2=0x7fba5c158f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.945+0000 7fba61476640 1 --2- 172.21.15.16:0/834600361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba5c15dce0 0x7fba5c158f00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fba5402ac70 tx=0x7fba54004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.945+0000 7fba61476640 1 -- 172.21.15.16:0/834600361 shutdown_connections 2024-04-25T10:06:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.945+0000 7fba61476640 1 --2- 172.21.15.16:0/834600361 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fba3c06b4d0 0x7fba3c06d990 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.945+0000 7fba61476640 1 --2- 172.21.15.16:0/834600361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba5c15dce0 0x7fba5c158f00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.945+0000 7fba61476640 1 --2- 172.21.15.16:0/834600361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba5c0d35b0 0x7fba5c1589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.945+0000 7fba61476640 1 --2- 172.21.15.16:0/834600361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba5c0b5990 0x7fba5c14dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.945+0000 7fba61476640 1 -- 172.21.15.16:0/834600361 >> 172.21.15.16:0/834600361 conn(0x7fba5c072900 msgr2=0x7fba5c15ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.946+0000 7fba61476640 1 -- 172.21.15.16:0/834600361 shutdown_connections 2024-04-25T10:06:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:13.946+0000 7fba61476640 1 -- 172.21.15.16:0/834600361 wait complete. 2024-04-25T10:06:13.962 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:14.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.057+0000 7f99b3e37640 1 Processor -- start 2024-04-25T10:06:14.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.057+0000 7f99b3e37640 1 -- start start 2024-04-25T10:06:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.058+0000 7f99b3e37640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99ac15ba10 0x7f99ac099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.058+0000 7f99b3e37640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f99ac06c3c0 0x7f99ac0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.058+0000 7f99b3e37640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99ac09a3a0 con 0x7f99ac099f80 2024-04-25T10:06:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.058+0000 7f99b3e37640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99ac0cbe50 con 0x7f99ac06c3c0 2024-04-25T10:06:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.058+0000 7f99b3e37640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac0cfe80 con 0x7f99ac15ba10 2024-04-25T10:06:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.058+0000 7f99b13d2640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f99ac099f80 0x7f99ac06bcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56524/0 (socket says 172.21.15.16:56524) 2024-04-25T10:06:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.058+0000 7f99b13d2640 1 -- 172.21.15.16:0/582756755 learned_addr learned my addr 172.21.15.16:0/582756755 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.058+0000 7f99b0bd1640 1 -- 172.21.15.16:0/582756755 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937406404 0 0) 0x7f99ac09a3a0 con 0x7f99ac099f80 2024-04-25T10:06:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.058+0000 7f99b23d4640 1 --2- 172.21.15.16:0/582756755 >> v2:172.21.15.186:3300/0 conn(0x7f99ac06c3c0 0x7f99ac0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.058+0000 7f99b1bd3640 1 --2- 172.21.15.16:0/582756755 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99ac15ba10 0x7f99ac099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.058+0000 7f99b0bd1640 1 -- 172.21.15.16:0/582756755 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9998003520 con 0x7f99ac099f80 2024-04-25T10:06:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.058+0000 7f99b0bd1640 1 -- 172.21.15.16:0/582756755 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1216070301 0 0) 0x7f9998003520 con 0x7f99ac099f80 2024-04-25T10:06:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.058+0000 7f99b0bd1640 1 -- 172.21.15.16:0/582756755 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99ac15ba10 msgr2=0x7f99ac099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.058+0000 7f99b0bd1640 1 --2- 172.21.15.16:0/582756755 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99ac15ba10 0x7f99ac099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.058+0000 7f99b0bd1640 1 -- 172.21.15.16:0/582756755 >> v2:172.21.15.186:3300/0 conn(0x7f99ac06c3c0 msgr2=0x7f99ac0cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.058+0000 7f99b0bd1640 1 --2- 172.21.15.16:0/582756755 >> v2:172.21.15.186:3300/0 conn(0x7f99ac06c3c0 0x7f99ac0cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.058+0000 7f99b0bd1640 1 -- 172.21.15.16:0/582756755 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac06c900 con 0x7f99ac099f80 2024-04-25T10:06:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.059+0000 7f99b0bd1640 1 -- 172.21.15.16:0/582756755 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f999c001670 con 0x7f99ac099f80 2024-04-25T10:06:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.059+0000 7f99b0bd1640 1 -- 172.21.15.16:0/582756755 >> v1:172.21.15.16:6789/0 conn(0x7f99ac099f80 legacy=0x7f99ac06bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.059+0000 7f99b0bd1640 1 --2- 172.21.15.16:0/582756755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99980039b0 0x7f9998003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.059+0000 7f99b0bd1640 1 --2- 172.21.15.16:0/582756755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9998004800 0x7f9998004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.059+0000 7f99b0bd1640 1 --2- 172.21.15.16:0/582756755 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99ac15ba10 0x7f99980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.059+0000 7f99b0bd1640 1 -- 172.21.15.16:0/582756755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac0cfe80 con 0x7f9998004800 2024-04-25T10:06:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.059+0000 7f99b0bd1640 1 -- 172.21.15.16:0/582756755 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999800b9c0 con 0x7f99980039b0 2024-04-25T10:06:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.059+0000 7f99b0bd1640 1 -- 172.21.15.16:0/582756755 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f999800bb40 con 0x7f99ac15ba10 2024-04-25T10:06:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.059+0000 7f99b13d2640 1 --2- 172.21.15.16:0/582756755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9998004800 0x7f9998004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.059+0000 7f99b1bd3640 1 --2- 172.21.15.16:0/582756755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99980039b0 0x7f9998003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.059+0000 7f99b23d4640 1 --2- 172.21.15.16:0/582756755 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99ac15ba10 0x7f99980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.059+0000 7f99b13d2640 1 -- 172.21.15.16:0/582756755 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99ac15ba10 msgr2=0x7f99980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.059+0000 7f99b13d2640 1 --2- 172.21.15.16:0/582756755 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99ac15ba10 0x7f99980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.059+0000 7f99b13d2640 1 -- 172.21.15.16:0/582756755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99980039b0 msgr2=0x7f9998003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.059+0000 7f99b13d2640 1 --2- 172.21.15.16:0/582756755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99980039b0 0x7f9998003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.059+0000 7f99b13d2640 1 -- 172.21.15.16:0/582756755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9998003520 con 0x7f9998004800 2024-04-25T10:06:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.060+0000 7f99b13d2640 1 --2- 172.21.15.16:0/582756755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9998004800 0x7f9998004c80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f999c001050 tx=0x7f9998005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670b4abd10840393 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.060+0000 7f99b0bd1640 1 -- 172.21.15.16:0/582756755 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c02e070 con 0x7f9998004800 2024-04-25T10:06:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.060+0000 7f99b0bd1640 1 -- 172.21.15.16:0/582756755 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999c005af0 con 0x7f9998004800 2024-04-25T10:06:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.060+0000 7f99b0bd1640 1 -- 172.21.15.16:0/582756755 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c007d70 con 0x7f9998004800 2024-04-25T10:06:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.060+0000 7f99b3e37640 1 -- 172.21.15.16:0/582756755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9998004800 msgr2=0x7f9998004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.060+0000 7f99b3e37640 1 --2- 172.21.15.16:0/582756755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9998004800 0x7f9998004c80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f999c001050 tx=0x7f9998005830 comp rx=0 tx=0).stop 2024-04-25T10:06:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.060+0000 7f99b23d4640 1 -- 172.21.15.16:0/582756755 reap_dead start 2024-04-25T10:06:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.060+0000 7f99b3e37640 1 -- 172.21.15.16:0/582756755 shutdown_connections 2024-04-25T10:06:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.060+0000 7f99b3e37640 1 -- 172.21.15.16:0/582756755 >> 172.21.15.16:0/582756755 conn(0x7f99ac0b32f0 msgr2=0x7f99ac0b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.061+0000 7f99b3e37640 1 -- 172.21.15.16:0/582756755 shutdown_connections 2024-04-25T10:06:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.061+0000 7f99b3e37640 1 -- 172.21.15.16:0/582756755 wait complete. 2024-04-25T10:06:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.062+0000 7f99b3e37640 1 Processor -- start 2024-04-25T10:06:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.062+0000 7f99b3e37640 1 -- start start 2024-04-25T10:06:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.062+0000 7f99b3e37640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9998004800 0x7f99ac15c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.062+0000 7f99b3e37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99ac0b43b0 0x7f99ac15cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.062+0000 7f99b3e37640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99ac0b7180 0x7f99ac0c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.062+0000 7f99b3e37640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac099a40 con 0x7f99ac0b43b0 2024-04-25T10:06:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.062+0000 7f99b3e37640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac06cec0 con 0x7f9998004800 2024-04-25T10:06:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.062+0000 7f99b3e37640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac099bc0 con 0x7f99ac0b7180 2024-04-25T10:06:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.063+0000 7f99b13d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99ac0b43b0 0x7f99ac15cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.063+0000 7f99b13d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99ac0b43b0 0x7f99ac15cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49464/0 (socket says 172.21.15.16:49464) 2024-04-25T10:06:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.063+0000 7f99b13d2640 1 -- 172.21.15.16:0/1862945769 learned_addr learned my addr 172.21.15.16:0/1862945769 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.063+0000 7f99b1bd3640 1 --2- 172.21.15.16:0/1862945769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9998004800 0x7f99ac15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.063+0000 7f99b23d4640 1 --2- 172.21.15.16:0/1862945769 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99ac0b7180 0x7f99ac0c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.063+0000 7f99b13d2640 1 -- 172.21.15.16:0/1862945769 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99ac0b7180 msgr2=0x7f99ac0c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.063+0000 7f99b13d2640 1 --2- 172.21.15.16:0/1862945769 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99ac0b7180 0x7f99ac0c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.063+0000 7f99b13d2640 1 -- 172.21.15.16:0/1862945769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9998004800 msgr2=0x7f99ac15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.063+0000 7f99b13d2640 1 --2- 172.21.15.16:0/1862945769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9998004800 0x7f99ac15c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.063+0000 7f99b13d2640 1 -- 172.21.15.16:0/1862945769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac0c6540 con 0x7f99ac0b43b0 2024-04-25T10:06:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.063+0000 7f99b13d2640 1 --2- 172.21.15.16:0/1862945769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99ac0b43b0 0x7f99ac15cd40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f999c02ae30 tx=0x7f999c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.063+0000 7f99a27fc640 1 -- 172.21.15.16:0/1862945769 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c042070 con 0x7f99ac0b43b0 2024-04-25T10:06:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.063+0000 7f99a27fc640 1 -- 172.21.15.16:0/1862945769 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999c007d50 con 0x7f99ac0b43b0 2024-04-25T10:06:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.063+0000 7f99b3e37640 1 -- 172.21.15.16:0/1862945769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ac0c6820 con 0x7f99ac0b43b0 2024-04-25T10:06:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.063+0000 7f99a27fc640 1 -- 172.21.15.16:0/1862945769 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c001cc0 con 0x7f99ac0b43b0 2024-04-25T10:06:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.063+0000 7f99b3e37640 1 -- 172.21.15.16:0/1862945769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac15ea80 con 0x7f99ac0b43b0 2024-04-25T10:06:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.064+0000 7f99b3e37640 1 -- 172.21.15.16:0/1862945769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99ac0749b0 con 0x7f99ac0b43b0 2024-04-25T10:06:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.065+0000 7f99a27fc640 1 -- 172.21.15.16:0/1862945769 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f999c045070 con 0x7f99ac0b43b0 2024-04-25T10:06:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.065+0000 7f99a27fc640 1 --2- 172.21.15.16:0/1862945769 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f999006b500 0x7f999006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.065+0000 7f99a27fc640 1 -- 172.21.15.16:0/1862945769 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f999c0aafd0 con 0x7f99ac0b43b0 2024-04-25T10:06:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.065+0000 7f99b1bd3640 1 --2- 172.21.15.16:0/1862945769 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f999006b500 0x7f999006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.066+0000 7f99b1bd3640 1 --2- 172.21.15.16:0/1862945769 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f999006b500 0x7f999006d9c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009a60 tx=0x7f99a8009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.069+0000 7f99a27fc640 1 -- 172.21.15.16:0/1862945769 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f999c04c020 con 0x7f99ac0b43b0 2024-04-25T10:06:14.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.249+0000 7f99b3e37640 1 -- 172.21.15.16:0/1862945769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99ac09a570 con 0x7f99ac0b43b0 2024-04-25T10:06:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.250+0000 7f99a27fc640 1 -- 172.21.15.16:0/1862945769 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f999c079dc0 con 0x7f99ac0b43b0 2024-04-25T10:06:14.255 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:14.255 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":36,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:06:12.582767+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:08.830059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:09.922618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.252+0000 7f99b3e37640 1 -- 172.21.15.16:0/1862945769 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f999006b500 msgr2=0x7f999006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.252+0000 7f99b3e37640 1 --2- 172.21.15.16:0/1862945769 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f999006b500 0x7f999006d9c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009a60 tx=0x7f99a8009190 comp rx=0 tx=0).stop 2024-04-25T10:06:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.252+0000 7f99b3e37640 1 -- 172.21.15.16:0/1862945769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99ac0b43b0 msgr2=0x7f99ac15cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.252+0000 7f99b3e37640 1 --2- 172.21.15.16:0/1862945769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99ac0b43b0 0x7f99ac15cd40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f999c02ae30 tx=0x7f999c000fe0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.252+0000 7f99b3e37640 1 -- 172.21.15.16:0/1862945769 shutdown_connections 2024-04-25T10:06:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.252+0000 7f99b3e37640 1 --2- 172.21.15.16:0/1862945769 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f999006b500 0x7f999006d9c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.252+0000 7f99b3e37640 1 --2- 172.21.15.16:0/1862945769 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99ac0b7180 0x7f99ac0c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.252+0000 7f99b3e37640 1 --2- 172.21.15.16:0/1862945769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99ac0b43b0 0x7f99ac15cd40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.252+0000 7f99b3e37640 1 --2- 172.21.15.16:0/1862945769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9998004800 0x7f99ac15c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.252+0000 7f99b3e37640 1 -- 172.21.15.16:0/1862945769 >> 172.21.15.16:0/1862945769 conn(0x7f99ac0b32f0 msgr2=0x7f99ac15ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.252+0000 7f99b3e37640 1 -- 172.21.15.16:0/1862945769 shutdown_connections 2024-04-25T10:06:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.252+0000 7f99b3e37640 1 -- 172.21.15.16:0/1862945769 wait complete. 2024-04-25T10:06:14.268 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:06:14.268 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:14.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.363+0000 7f6b15a76640 1 Processor -- start 2024-04-25T10:06:14.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.363+0000 7f6b15a76640 1 -- start start 2024-04-25T10:06:14.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.364+0000 7f6b15a76640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6b1006bef0 0x7f6b1006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.364+0000 7f6b15a76640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6b1006c2d0 0x7f6b1015c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.364+0000 7f6b15a76640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b1015ce50 con 0x7f6b100acd80 2024-04-25T10:06:14.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.364+0000 7f6b15a76640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1006eb20 con 0x7f6b1006c2d0 2024-04-25T10:06:14.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.364+0000 7f6b15a76640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1006dc70 con 0x7f6b1006bef0 2024-04-25T10:06:14.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.364+0000 7f6b0effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6b1006c2d0 0x7f6b1015c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.364+0000 7f6b0effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6b1006c2d0 0x7f6b1015c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49472/0 (socket says 172.21.15.16:49472) 2024-04-25T10:06:14.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.364+0000 7f6b0effd640 1 -- 172.21.15.16:0/1285625397 learned_addr learned my addr 172.21.15.16:0/1285625397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:14.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.364+0000 7f6b0f7fe640 1 --2- 172.21.15.16:0/1285625397 >> v2:172.21.15.186:3300/0 conn(0x7f6b1006bef0 0x7f6b1006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.364+0000 7f6b0effd640 1 -- 172.21.15.16:0/1285625397 >> v2:172.21.15.186:3300/0 conn(0x7f6b1006bef0 msgr2=0x7f6b1006efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.364+0000 7f6b0effd640 1 --2- 172.21.15.16:0/1285625397 >> v2:172.21.15.186:3300/0 conn(0x7f6b1006bef0 0x7f6b1006efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.364+0000 7f6b0effd640 1 -- 172.21.15.16:0/1285625397 >> v1:172.21.15.186:6789/0 conn(0x7f6b100acd80 legacy=0x7f6b1006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.364+0000 7f6b0effd640 1 -- 172.21.15.16:0/1285625397 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1006df10 con 0x7f6b1006c2d0 2024-04-25T10:06:14.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.365+0000 7f6b0effd640 1 --2- 172.21.15.16:0/1285625397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1006c2d0 0x7f6b1015c910 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6b100af870 tx=0x7f6b0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e8ed2928b2eb16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.365+0000 7f6b0e7fc640 1 -- 172.21.15.16:0/1285625397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04037070 con 0x7f6b1006c2d0 2024-04-25T10:06:14.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.365+0000 7f6b0e7fc640 1 -- 172.21.15.16:0/1285625397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b04032440 con 0x7f6b1006c2d0 2024-04-25T10:06:14.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.365+0000 7f6b0e7fc640 1 -- 172.21.15.16:0/1285625397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04032810 con 0x7f6b1006c2d0 2024-04-25T10:06:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.366+0000 7f6b15a76640 1 -- 172.21.15.16:0/1285625397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1006c2d0 msgr2=0x7f6b1015c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.366+0000 7f6b15a76640 1 --2- 172.21.15.16:0/1285625397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1006c2d0 0x7f6b1015c910 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6b100af870 tx=0x7f6b0402a270 comp rx=0 tx=0).stop 2024-04-25T10:06:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.366+0000 7f6b15a76640 1 -- 172.21.15.16:0/1285625397 shutdown_connections 2024-04-25T10:06:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.366+0000 7f6b15a76640 1 --2- 172.21.15.16:0/1285625397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1006c2d0 0x7f6b1015c910 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.366+0000 7f6b15a76640 1 --2- 172.21.15.16:0/1285625397 >> v2:172.21.15.186:3300/0 conn(0x7f6b1006bef0 0x7f6b1006efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.366+0000 7f6b15a76640 1 -- 172.21.15.16:0/1285625397 >> 172.21.15.16:0/1285625397 conn(0x7f6b100b2220 msgr2=0x7f6b100cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.366+0000 7f6b15a76640 1 -- 172.21.15.16:0/1285625397 shutdown_connections 2024-04-25T10:06:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.366+0000 7f6b15a76640 1 -- 172.21.15.16:0/1285625397 wait complete. 2024-04-25T10:06:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.367+0000 7f6b15a76640 1 Processor -- start 2024-04-25T10:06:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.367+0000 7f6b15a76640 1 -- start start 2024-04-25T10:06:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.368+0000 7f6b15a76640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b1006bef0 0x7f6b101f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.368+0000 7f6b15a76640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b1006c2d0 0x7f6b10157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.368+0000 7f6b15a76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b100acd80 0x7f6b100b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.368+0000 7f6b15a76640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1006e7b0 con 0x7f6b100acd80 2024-04-25T10:06:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.368+0000 7f6b15a76640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1006e4d0 con 0x7f6b1006bef0 2024-04-25T10:06:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.368+0000 7f6b15a76640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b100ada00 con 0x7f6b1006c2d0 2024-04-25T10:06:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.368+0000 7f6b0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b100acd80 0x7f6b100b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.368+0000 7f6b0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b100acd80 0x7f6b100b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49480/0 (socket says 172.21.15.16:49480) 2024-04-25T10:06:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.368+0000 7f6b0ffff640 1 -- 172.21.15.16:0/1216972942 learned_addr learned my addr 172.21.15.16:0/1216972942 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.368+0000 7f6b0f7fe640 1 --2- 172.21.15.16:0/1216972942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b1006bef0 0x7f6b101f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.368+0000 7f6b0effd640 1 --2- 172.21.15.16:0/1216972942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b1006c2d0 0x7f6b10157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.368+0000 7f6b0ffff640 1 -- 172.21.15.16:0/1216972942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b1006c2d0 msgr2=0x7f6b10157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.368+0000 7f6b0ffff640 1 --2- 172.21.15.16:0/1216972942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b1006c2d0 0x7f6b10157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.368+0000 7f6b0ffff640 1 -- 172.21.15.16:0/1216972942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b1006bef0 msgr2=0x7f6b101f7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.368+0000 7f6b0ffff640 1 --2- 172.21.15.16:0/1216972942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b1006bef0 0x7f6b101f7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.368+0000 7f6b0ffff640 1 -- 172.21.15.16:0/1216972942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b100b4c00 con 0x7f6b100acd80 2024-04-25T10:06:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.368+0000 7f6b0ffff640 1 --2- 172.21.15.16:0/1216972942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b100acd80 0x7f6b100b3d30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800c6b0 tx=0x7f6b0800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.369+0000 7f6af7fff640 1 -- 172.21.15.16:0/1216972942 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08003b70 con 0x7f6b100acd80 2024-04-25T10:06:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.369+0000 7f6af7fff640 1 -- 172.21.15.16:0/1216972942 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b08003d30 con 0x7f6b100acd80 2024-04-25T10:06:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.369+0000 7f6b15a76640 1 -- 172.21.15.16:0/1216972942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b100b3250 con 0x7f6b100acd80 2024-04-25T10:06:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.369+0000 7f6af7fff640 1 -- 172.21.15.16:0/1216972942 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08005310 con 0x7f6b100acd80 2024-04-25T10:06:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.369+0000 7f6b15a76640 1 -- 172.21.15.16:0/1216972942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b100b3810 con 0x7f6b100acd80 2024-04-25T10:06:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.370+0000 7f6b15a76640 1 -- 172.21.15.16:0/1216972942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b100d0b70 con 0x7f6b100acd80 2024-04-25T10:06:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.370+0000 7f6af7fff640 1 -- 172.21.15.16:0/1216972942 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b080054b0 con 0x7f6b100acd80 2024-04-25T10:06:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.370+0000 7f6af7fff640 1 --2- 172.21.15.16:0/1216972942 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6afc06b4d0 0x7f6afc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.371+0000 7f6af7fff640 1 -- 172.21.15.16:0/1216972942 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b08089e90 con 0x7f6b100acd80 2024-04-25T10:06:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.371+0000 7f6b0f7fe640 1 --2- 172.21.15.16:0/1216972942 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6afc06b4d0 0x7f6afc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.371+0000 7f6b0f7fe640 1 --2- 172.21.15.16:0/1216972942 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6afc06b4d0 0x7f6afc06d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009b40 tx=0x7f6b00009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.375+0000 7f6af7fff640 1 -- 172.21.15.16:0/1216972942 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b1015ce50 con 0x7f6b100acd80 2024-04-25T10:06:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.551+0000 7f6b15a76640 1 -- 172.21.15.16:0/1216972942 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6b10154090 con 0x7f6afc06b4d0 2024-04-25T10:06:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.558+0000 7f6af7fff640 1 -- 172.21.15.16:0/1216972942 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250788 (secure 0 0 0) 0x7f6b10154090 con 0x7f6afc06b4d0 2024-04-25T10:06:14.566 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:14.568 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:06:14.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.561+0000 7f6b15a76640 1 -- 172.21.15.16:0/1216972942 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6afc06b4d0 msgr2=0x7f6afc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.561+0000 7f6b15a76640 1 --2- 172.21.15.16:0/1216972942 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6afc06b4d0 0x7f6afc06d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009b40 tx=0x7f6b00009190 comp rx=0 tx=0).stop 2024-04-25T10:06:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.561+0000 7f6b15a76640 1 -- 172.21.15.16:0/1216972942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b100acd80 msgr2=0x7f6b100b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.561+0000 7f6b15a76640 1 --2- 172.21.15.16:0/1216972942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b100acd80 0x7f6b100b3d30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800c6b0 tx=0x7f6b0800cb70 comp rx=0 tx=0).stop 2024-04-25T10:06:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.561+0000 7f6b15a76640 1 -- 172.21.15.16:0/1216972942 shutdown_connections 2024-04-25T10:06:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.561+0000 7f6b15a76640 1 --2- 172.21.15.16:0/1216972942 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6afc06b4d0 0x7f6afc06d990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.561+0000 7f6b15a76640 1 --2- 172.21.15.16:0/1216972942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b100acd80 0x7f6b100b3d30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.561+0000 7f6b15a76640 1 --2- 172.21.15.16:0/1216972942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b1006c2d0 0x7f6b10157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.561+0000 7f6b15a76640 1 --2- 172.21.15.16:0/1216972942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b1006bef0 0x7f6b101f7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.561+0000 7f6b15a76640 1 -- 172.21.15.16:0/1216972942 >> 172.21.15.16:0/1216972942 conn(0x7f6b100b2220 msgr2=0x7f6b10154b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.561+0000 7f6b15a76640 1 -- 172.21.15.16:0/1216972942 shutdown_connections 2024-04-25T10:06:14.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.561+0000 7f6b15a76640 1 -- 172.21.15.16:0/1216972942 wait complete. 2024-04-25T10:06:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-04-25T10:06:13.795375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":184,"acting":184,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237044,"kb_used_data":4236,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737676,"statfs":{"total":773094113280,"available":772851380224,"internally_reserved":0,"allocated":4337664,"data_stored":2378543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.228282"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590900+0000","last_change":"2024-04-25T10:06:11.596197+0000","last_active":"2024-04-25T10:06:12.590900+0000","last_peered":"2024-04-25T10:06:12.590900+0000","last_clean":"2024-04-25T10:06:12.590900+0000","last_became_active":"2024-04-25T10:06:11.595896+0000","last_became_peered":"2024-04-25T10:06:11.595896+0000","last_unstale":"2024-04-25T10:06:12.590900+0000","last_undegraded":"2024-04-25T10:06:12.590900+0000","last_fullsized":"2024-04-25T10:06:12.590900+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:50.014644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:10.566145+0000","last_change":"2024-04-25T10:06:10.566145+0000","last_active":"2024-04-25T10:06:10.566145+0000","last_peered":"2024-04-25T10:06:10.566145+0000","last_clean":"2024-04-25T10:06:10.566145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:10.566145+0000","last_undegraded":"2024-04-25T10:06:10.566145+0000","last_fullsized":"2024-04-25T10:06:10.566145+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591774+0000","last_change":"2024-04-25T10:06:11.588804+0000","last_active":"2024-04-25T10:06:12.591774+0000","last_peered":"2024-04-25T10:06:12.591774+0000","last_clean":"2024-04-25T10:06:12.591774+0000","last_became_active":"2024-04-25T10:06:11.588533+0000","last_became_peered":"2024-04-25T10:06:11.588533+0000","last_unstale":"2024-04-25T10:06:12.591774+0000","last_undegraded":"2024-04-25T10:06:12.591774+0000","last_fullsized":"2024-04-25T10:06:12.591774+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:12.136446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610805+0000","last_change":"2024-04-25T10:06:11.603412+0000","last_active":"2024-04-25T10:06:11.610805+0000","last_peered":"2024-04-25T10:06:11.610805+0000","last_clean":"2024-04-25T10:06:11.610805+0000","last_became_active":"2024-04-25T10:06:11.602936+0000","last_became_peered":"2024-04-25T10:06:11.602936+0000","last_unstale":"2024-04-25T10:06:11.610805+0000","last_undegraded":"2024-04-25T10:06:11.610805+0000","last_fullsized":"2024-04-25T10:06:11.610805+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.403695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:10.566145+0000","last_change":"2024-04-25T10:06:10.566145+0000","last_active":"2024-04-25T10:06:10.566145+0000","last_peered":"2024-04-25T10:06:10.566145+0000","last_clean":"2024-04-25T10:06:10.566145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:10.566145+0000","last_undegraded":"2024-04-25T10:06:10.566145+0000","last_fullsized":"2024-04-25T10:06:10.566145+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599412+0000","last_change":"2024-04-25T10:06:11.594811+0000","last_active":"2024-04-25T10:06:11.599412+0000","last_peered":"2024-04-25T10:06:11.599412+0000","last_clean":"2024-04-25T10:06:11.599412+0000","last_became_active":"2024-04-25T10:06:11.594494+0000","last_became_peered":"2024-04-25T10:06:11.594494+0000","last_unstale":"2024-04-25T10:06:11.599412+0000","last_undegraded":"2024-04-25T10:06:11.599412+0000","last_fullsized":"2024-04-25T10:06:11.599412+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:02.231151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590302+0000","last_change":"2024-04-25T10:06:11.585486+0000","last_active":"2024-04-25T10:06:12.590302+0000","last_peered":"2024-04-25T10:06:12.590302+0000","last_clean":"2024-04-25T10:06:12.590302+0000","last_became_active":"2024-04-25T10:06:11.585167+0000","last_became_peered":"2024-04-25T10:06:11.585167+0000","last_unstale":"2024-04-25T10:06:12.590302+0000","last_undegraded":"2024-04-25T10:06:12.590302+0000","last_fullsized":"2024-04-25T10:06:12.590302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590232+0000","last_change":"2024-04-25T10:06:11.599766+0000","last_active":"2024-04-25T10:06:12.590232+0000","last_peered":"2024-04-25T10:06:12.590232+0000","last_clean":"2024-04-25T10:06:12.590232+0000","last_became_active":"2024-04-25T10:06:11.599473+0000","last_became_peered":"2024-04-25T10:06:11.599473+0000","last_unstale":"2024-04-25T10:06:12.590232+0000","last_undegraded":"2024-04-25T10:06:12.590232+0000","last_fullsized":"2024-04-25T10:06:12.590232+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:46.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591773+0000","last_change":"2024-04-25T10:06:11.601172+0000","last_active":"2024-04-25T10:06:12.591773+0000","last_peered":"2024-04-25T10:06:12.591773+0000","last_clean":"2024-04-25T10:06:12.591773+0000","last_became_active":"2024-04-25T10:06:11.600890+0000","last_became_peered":"2024-04-25T10:06:11.600890+0000","last_unstale":"2024-04-25T10:06:12.591773+0000","last_undegraded":"2024-04-25T10:06:12.591773+0000","last_fullsized":"2024-04-25T10:06:12.591773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:33.055338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.614185+0000","last_change":"2024-04-25T10:06:11.611336+0000","last_active":"2024-04-25T10:06:11.614185+0000","last_peered":"2024-04-25T10:06:11.614185+0000","last_clean":"2024-04-25T10:06:11.614185+0000","last_became_active":"2024-04-25T10:06:11.610883+0000","last_became_peered":"2024-04-25T10:06:11.610883+0000","last_unstale":"2024-04-25T10:06:11.614185+0000","last_undegraded":"2024-04-25T10:06:11.614185+0000","last_fullsized":"2024-04-25T10:06:11.614185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:50.014644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.606907+0000","last_change":"2024-04-25T10:06:11.606133+0000","last_active":"2024-04-25T10:06:11.606907+0000","last_peered":"2024-04-25T10:06:11.606907+0000","last_clean":"2024-04-25T10:06:11.606907+0000","last_became_active":"2024-04-25T10:06:11.605855+0000","last_became_peered":"2024-04-25T10:06:11.605855+0000","last_unstale":"2024-04-25T10:06:11.606907+0000","last_undegraded":"2024-04-25T10:06:11.606907+0000","last_fullsized":"2024-04-25T10:06:11.606907+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:35.491727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591678+0000","last_change":"2024-04-25T10:06:11.598791+0000","last_active":"2024-04-25T10:06:12.591678+0000","last_peered":"2024-04-25T10:06:12.591678+0000","last_clean":"2024-04-25T10:06:12.591678+0000","last_became_active":"2024-04-25T10:06:11.598511+0000","last_became_peered":"2024-04-25T10:06:11.598511+0000","last_unstale":"2024-04-25T10:06:12.591678+0000","last_undegraded":"2024-04-25T10:06:12.591678+0000","last_fullsized":"2024-04-25T10:06:12.591678+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:18.899344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592055+0000","last_change":"2024-04-25T10:06:11.602586+0000","last_active":"2024-04-25T10:06:12.592055+0000","last_peered":"2024-04-25T10:06:12.592055+0000","last_clean":"2024-04-25T10:06:12.592055+0000","last_became_active":"2024-04-25T10:06:11.602219+0000","last_became_peered":"2024-04-25T10:06:11.602219+0000","last_unstale":"2024-04-25T10:06:12.592055+0000","last_undegraded":"2024-04-25T10:06:12.592055+0000","last_fullsized":"2024-04-25T10:06:12.592055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610597+0000","last_change":"2024-04-25T10:06:11.599901+0000","last_active":"2024-04-25T10:06:11.610597+0000","last_peered":"2024-04-25T10:06:11.610597+0000","last_clean":"2024-04-25T10:06:11.610597+0000","last_became_active":"2024-04-25T10:06:11.599632+0000","last_became_peered":"2024-04-25T10:06:11.599632+0000","last_unstale":"2024-04-25T10:06:11.610597+0000","last_undegraded":"2024-04-25T10:06:11.610597+0000","last_fullsized":"2024-04-25T10:06:11.610597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:46.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.592021+0000","last_change":"2024-04-25T10:06:11.587570+0000","last_active":"2024-04-25T10:06:11.592021+0000","last_peered":"2024-04-25T10:06:11.592021+0000","last_clean":"2024-04-25T10:06:11.592021+0000","last_became_active":"2024-04-25T10:06:11.587301+0000","last_became_peered":"2024-04-25T10:06:11.587301+0000","last_unstale":"2024-04-25T10:06:11.592021+0000","last_undegraded":"2024-04-25T10:06:11.592021+0000","last_fullsized":"2024-04-25T10:06:11.592021+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.597882+0000","last_change":"2024-04-25T10:06:11.597081+0000","last_active":"2024-04-25T10:06:11.597882+0000","last_peered":"2024-04-25T10:06:11.597882+0000","last_clean":"2024-04-25T10:06:11.597882+0000","last_became_active":"2024-04-25T10:06:11.596781+0000","last_became_peered":"2024-04-25T10:06:11.596781+0000","last_unstale":"2024-04-25T10:06:11.597882+0000","last_undegraded":"2024-04-25T10:06:11.597882+0000","last_fullsized":"2024-04-25T10:06:11.597882+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:11.365920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.589022+0000","last_change":"2024-04-25T10:06:11.587969+0000","last_active":"2024-04-25T10:06:11.589022+0000","last_peered":"2024-04-25T10:06:11.589022+0000","last_clean":"2024-04-25T10:06:11.589022+0000","last_became_active":"2024-04-25T10:06:11.587084+0000","last_became_peered":"2024-04-25T10:06:11.587084+0000","last_unstale":"2024-04-25T10:06:11.589022+0000","last_undegraded":"2024-04-25T10:06:11.589022+0000","last_fullsized":"2024-04-25T10:06:11.589022+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:47.208707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599620+0000","last_change":"2024-04-25T10:06:11.596669+0000","last_active":"2024-04-25T10:06:11.599620+0000","last_peered":"2024-04-25T10:06:11.599620+0000","last_clean":"2024-04-25T10:06:11.599620+0000","last_became_active":"2024-04-25T10:06:11.596091+0000","last_became_peered":"2024-04-25T10:06:11.596091+0000","last_unstale":"2024-04-25T10:06:11.599620+0000","last_undegraded":"2024-04-25T10:06:11.599620+0000","last_fullsized":"2024-04-25T10:06:11.599620+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:18.899344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599443+0000","last_change":"2024-04-25T10:06:11.594701+0000","last_active":"2024-04-25T10:06:11.599443+0000","last_peered":"2024-04-25T10:06:11.599443+0000","last_clean":"2024-04-25T10:06:11.599443+0000","last_became_active":"2024-04-25T10:06:11.594376+0000","last_became_peered":"2024-04-25T10:06:11.594376+0000","last_unstale":"2024-04-25T10:06:11.599443+0000","last_undegraded":"2024-04-25T10:06:11.599443+0000","last_fullsized":"2024-04-25T10:06:11.599443+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:53.157588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592096+0000","last_change":"2024-04-25T10:06:11.597575+0000","last_active":"2024-04-25T10:06:12.592096+0000","last_peered":"2024-04-25T10:06:12.592096+0000","last_clean":"2024-04-25T10:06:12.592096+0000","last_became_active":"2024-04-25T10:06:11.596710+0000","last_became_peered":"2024-04-25T10:06:11.596710+0000","last_unstale":"2024-04-25T10:06:12.592096+0000","last_undegraded":"2024-04-25T10:06:12.592096+0000","last_fullsized":"2024-04-25T10:06:12.592096+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.595406+0000","last_change":"2024-04-25T10:06:11.594514+0000","last_active":"2024-04-25T10:06:11.595406+0000","last_peered":"2024-04-25T10:06:11.595406+0000","last_clean":"2024-04-25T10:06:11.595406+0000","last_became_active":"2024-04-25T10:06:11.594231+0000","last_became_peered":"2024-04-25T10:06:11.594231+0000","last_unstale":"2024-04-25T10:06:11.595406+0000","last_undegraded":"2024-04-25T10:06:11.595406+0000","last_fullsized":"2024-04-25T10:06:11.595406+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:03.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590474+0000","last_change":"2024-04-25T10:06:11.595941+0000","last_active":"2024-04-25T10:06:12.590474+0000","last_peered":"2024-04-25T10:06:12.590474+0000","last_clean":"2024-04-25T10:06:12.590474+0000","last_became_active":"2024-04-25T10:06:11.595598+0000","last_became_peered":"2024-04-25T10:06:11.595598+0000","last_unstale":"2024-04-25T10:06:12.590474+0000","last_undegraded":"2024-04-25T10:06:12.590474+0000","last_fullsized":"2024-04-25T10:06:12.590474+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:40.079857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591540+0000","last_change":"2024-04-25T10:06:11.599549+0000","last_active":"2024-04-25T10:06:12.591540+0000","last_peered":"2024-04-25T10:06:12.591540+0000","last_clean":"2024-04-25T10:06:12.591540+0000","last_became_active":"2024-04-25T10:06:11.599256+0000","last_became_peered":"2024-04-25T10:06:11.599256+0000","last_unstale":"2024-04-25T10:06:12.591540+0000","last_undegraded":"2024-04-25T10:06:12.591540+0000","last_fullsized":"2024-04-25T10:06:12.591540+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:48.190466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590264+0000","last_change":"2024-04-25T10:06:11.605175+0000","last_active":"2024-04-25T10:06:12.590264+0000","last_peered":"2024-04-25T10:06:12.590264+0000","last_clean":"2024-04-25T10:06:12.590264+0000","last_became_active":"2024-04-25T10:06:11.604773+0000","last_became_peered":"2024-04-25T10:06:11.604773+0000","last_unstale":"2024-04-25T10:06:12.590264+0000","last_undegraded":"2024-04-25T10:06:12.590264+0000","last_fullsized":"2024-04-25T10:06:12.590264+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.403695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611074+0000","last_change":"2024-04-25T10:06:11.605194+0000","last_active":"2024-04-25T10:06:11.611074+0000","last_peered":"2024-04-25T10:06:11.611074+0000","last_clean":"2024-04-25T10:06:11.611074+0000","last_became_active":"2024-04-25T10:06:11.604595+0000","last_became_peered":"2024-04-25T10:06:11.604595+0000","last_unstale":"2024-04-25T10:06:11.611074+0000","last_undegraded":"2024-04-25T10:06:11.611074+0000","last_fullsized":"2024-04-25T10:06:11.611074+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:36.054034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611662+0000","last_change":"2024-04-25T10:06:11.603505+0000","last_active":"2024-04-25T10:06:11.611662+0000","last_peered":"2024-04-25T10:06:11.611662+0000","last_clean":"2024-04-25T10:06:11.611662+0000","last_became_active":"2024-04-25T10:06:11.603215+0000","last_became_peered":"2024-04-25T10:06:11.603215+0000","last_unstale":"2024-04-25T10:06:11.611662+0000","last_undegraded":"2024-04-25T10:06:11.611662+0000","last_fullsized":"2024-04-25T10:06:11.611662+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:10.566145+0000","last_change":"2024-04-25T10:06:10.566145+0000","last_active":"2024-04-25T10:06:10.566145+0000","last_peered":"2024-04-25T10:06:10.566145+0000","last_clean":"2024-04-25T10:06:10.566145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:10.566145+0000","last_undegraded":"2024-04-25T10:06:10.566145+0000","last_fullsized":"2024-04-25T10:06:10.566145+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591554+0000","last_change":"2024-04-25T10:06:11.601007+0000","last_active":"2024-04-25T10:06:12.591554+0000","last_peered":"2024-04-25T10:06:12.591554+0000","last_clean":"2024-04-25T10:06:12.591554+0000","last_became_active":"2024-04-25T10:06:11.600297+0000","last_became_peered":"2024-04-25T10:06:11.600297+0000","last_unstale":"2024-04-25T10:06:12.591554+0000","last_undegraded":"2024-04-25T10:06:12.591554+0000","last_fullsized":"2024-04-25T10:06:12.591554+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:18.645477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.601405+0000","last_change":"2024-04-25T10:06:11.600447+0000","last_active":"2024-04-25T10:06:11.601405+0000","last_peered":"2024-04-25T10:06:11.601405+0000","last_clean":"2024-04-25T10:06:11.601405+0000","last_became_active":"2024-04-25T10:06:11.600160+0000","last_became_peered":"2024-04-25T10:06:11.600160+0000","last_unstale":"2024-04-25T10:06:11.601405+0000","last_undegraded":"2024-04-25T10:06:11.601405+0000","last_fullsized":"2024-04-25T10:06:11.601405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:50.014644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.591617+0000","last_change":"2024-04-25T10:06:11.590349+0000","last_active":"2024-04-25T10:06:11.591617+0000","last_peered":"2024-04-25T10:06:11.591617+0000","last_clean":"2024-04-25T10:06:11.591617+0000","last_became_active":"2024-04-25T10:06:11.590090+0000","last_became_peered":"2024-04-25T10:06:11.590090+0000","last_unstale":"2024-04-25T10:06:11.591617+0000","last_undegraded":"2024-04-25T10:06:11.591617+0000","last_fullsized":"2024-04-25T10:06:11.591617+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590344+0000","last_change":"2024-04-25T10:06:11.597956+0000","last_active":"2024-04-25T10:06:12.590344+0000","last_peered":"2024-04-25T10:06:12.590344+0000","last_clean":"2024-04-25T10:06:12.590344+0000","last_became_active":"2024-04-25T10:06:11.597687+0000","last_became_peered":"2024-04-25T10:06:11.597687+0000","last_unstale":"2024-04-25T10:06:12.590344+0000","last_undegraded":"2024-04-25T10:06:12.590344+0000","last_fullsized":"2024-04-25T10:06:12.590344+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.586794+0000","last_change":"2024-04-25T10:06:11.584953+0000","last_active":"2024-04-25T10:06:11.586794+0000","last_peered":"2024-04-25T10:06:11.586794+0000","last_clean":"2024-04-25T10:06:11.586794+0000","last_became_active":"2024-04-25T10:06:11.584666+0000","last_became_peered":"2024-04-25T10:06:11.584666+0000","last_unstale":"2024-04-25T10:06:11.586794+0000","last_undegraded":"2024-04-25T10:06:11.586794+0000","last_fullsized":"2024-04-25T10:06:11.586794+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:55.206873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591906+0000","last_change":"2024-04-25T10:06:11.592933+0000","last_active":"2024-04-25T10:06:12.591906+0000","last_peered":"2024-04-25T10:06:12.591906+0000","last_clean":"2024-04-25T10:06:12.591906+0000","last_became_active":"2024-04-25T10:06:11.592335+0000","last_became_peered":"2024-04-25T10:06:11.592335+0000","last_unstale":"2024-04-25T10:06:12.591906+0000","last_undegraded":"2024-04-25T10:06:12.591906+0000","last_fullsized":"2024-04-25T10:06:12.591906+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.403695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.602910+0000","last_change":"2024-04-25T10:06:10.590142+0000","last_active":"2024-04-25T10:06:11.602910+0000","last_peered":"2024-04-25T10:06:11.602910+0000","last_clean":"2024-04-25T10:06:11.602910+0000","last_became_active":"2024-04-25T10:06:10.589851+0000","last_became_peered":"2024-04-25T10:06:10.589851+0000","last_unstale":"2024-04-25T10:06:11.602910+0000","last_undegraded":"2024-04-25T10:06:11.602910+0000","last_fullsized":"2024-04-25T10:06:11.602910+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:50.142763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590901+0000","last_change":"2024-04-25T10:06:11.586712+0000","last_active":"2024-04-25T10:06:12.590901+0000","last_peered":"2024-04-25T10:06:12.590901+0000","last_clean":"2024-04-25T10:06:12.590901+0000","last_became_active":"2024-04-25T10:06:11.586393+0000","last_became_peered":"2024-04-25T10:06:11.586393+0000","last_unstale":"2024-04-25T10:06:12.590901+0000","last_undegraded":"2024-04-25T10:06:12.590901+0000","last_fullsized":"2024-04-25T10:06:12.590901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:11.365920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611691+0000","last_change":"2024-04-25T10:06:10.590906+0000","last_active":"2024-04-25T10:06:11.611691+0000","last_peered":"2024-04-25T10:06:11.611691+0000","last_clean":"2024-04-25T10:06:11.611691+0000","last_became_active":"2024-04-25T10:06:10.590641+0000","last_became_peered":"2024-04-25T10:06:10.590641+0000","last_unstale":"2024-04-25T10:06:11.611691+0000","last_undegraded":"2024-04-25T10:06:11.611691+0000","last_fullsized":"2024-04-25T10:06:11.611691+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:39.077377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591703+0000","last_change":"2024-04-25T10:06:11.601284+0000","last_active":"2024-04-25T10:06:12.591703+0000","last_peered":"2024-04-25T10:06:12.591703+0000","last_clean":"2024-04-25T10:06:12.591703+0000","last_became_active":"2024-04-25T10:06:11.600739+0000","last_became_peered":"2024-04-25T10:06:11.600739+0000","last_unstale":"2024-04-25T10:06:12.591703+0000","last_undegraded":"2024-04-25T10:06:12.591703+0000","last_fullsized":"2024-04-25T10:06:12.591703+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:03.030946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.594679+0000","last_change":"2024-04-25T10:06:10.589990+0000","last_active":"2024-04-25T10:06:11.594679+0000","last_peered":"2024-04-25T10:06:11.594679+0000","last_clean":"2024-04-25T10:06:11.594679+0000","last_became_active":"2024-04-25T10:06:10.589663+0000","last_became_peered":"2024-04-25T10:06:10.589663+0000","last_unstale":"2024-04-25T10:06:11.594679+0000","last_undegraded":"2024-04-25T10:06:11.594679+0000","last_fullsized":"2024-04-25T10:06:11.594679+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:17.017530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611361+0000","last_change":"2024-04-25T10:06:11.608503+0000","last_active":"2024-04-25T10:06:11.611361+0000","last_peered":"2024-04-25T10:06:11.611361+0000","last_clean":"2024-04-25T10:06:11.611361+0000","last_became_active":"2024-04-25T10:06:11.608234+0000","last_became_peered":"2024-04-25T10:06:11.608234+0000","last_unstale":"2024-04-25T10:06:11.611361+0000","last_undegraded":"2024-04-25T10:06:11.611361+0000","last_fullsized":"2024-04-25T10:06:11.611361+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:11.365920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.604687+0000","last_change":"2024-04-25T10:06:10.590974+0000","last_active":"2024-04-25T10:06:11.604687+0000","last_peered":"2024-04-25T10:06:11.604687+0000","last_clean":"2024-04-25T10:06:11.604687+0000","last_became_active":"2024-04-25T10:06:10.590635+0000","last_became_peered":"2024-04-25T10:06:10.590635+0000","last_unstale":"2024-04-25T10:06:11.604687+0000","last_undegraded":"2024-04-25T10:06:11.604687+0000","last_fullsized":"2024-04-25T10:06:11.604687+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:31.888158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591335+0000","last_change":"2024-04-25T10:06:11.594569+0000","last_active":"2024-04-25T10:06:12.591335+0000","last_peered":"2024-04-25T10:06:12.591335+0000","last_clean":"2024-04-25T10:06:12.591335+0000","last_became_active":"2024-04-25T10:06:11.594016+0000","last_became_peered":"2024-04-25T10:06:11.594016+0000","last_unstale":"2024-04-25T10:06:12.591335+0000","last_undegraded":"2024-04-25T10:06:12.591335+0000","last_fullsized":"2024-04-25T10:06:12.591335+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:38.022779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590761+0000","last_change":"2024-04-25T10:06:10.591527+0000","last_active":"2024-04-25T10:06:12.590761+0000","last_peered":"2024-04-25T10:06:12.590761+0000","last_clean":"2024-04-25T10:06:12.590761+0000","last_became_active":"2024-04-25T10:06:10.591212+0000","last_became_peered":"2024-04-25T10:06:10.591212+0000","last_unstale":"2024-04-25T10:06:12.590761+0000","last_undegraded":"2024-04-25T10:06:12.590761+0000","last_fullsized":"2024-04-25T10:06:12.590761+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:17.896865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590681+0000","last_change":"2024-04-25T10:06:11.597833+0000","last_active":"2024-04-25T10:06:12.590681+0000","last_peered":"2024-04-25T10:06:12.590681+0000","last_clean":"2024-04-25T10:06:12.590681+0000","last_became_active":"2024-04-25T10:06:11.597247+0000","last_became_peered":"2024-04-25T10:06:11.597247+0000","last_unstale":"2024-04-25T10:06:12.590681+0000","last_undegraded":"2024-04-25T10:06:12.590681+0000","last_fullsized":"2024-04-25T10:06:12.590681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:36.054034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590610+0000","last_change":"2024-04-25T10:06:10.593926+0000","last_active":"2024-04-25T10:06:12.590610+0000","last_peered":"2024-04-25T10:06:12.590610+0000","last_clean":"2024-04-25T10:06:12.590610+0000","last_became_active":"2024-04-25T10:06:10.593640+0000","last_became_peered":"2024-04-25T10:06:10.593640+0000","last_unstale":"2024-04-25T10:06:12.590610+0000","last_undegraded":"2024-04-25T10:06:12.590610+0000","last_fullsized":"2024-04-25T10:06:12.590610+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:32.052858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591186+0000","last_change":"2024-04-25T10:06:11.593063+0000","last_active":"2024-04-25T10:06:12.591186+0000","last_peered":"2024-04-25T10:06:12.591186+0000","last_clean":"2024-04-25T10:06:12.591186+0000","last_became_active":"2024-04-25T10:06:11.592772+0000","last_became_peered":"2024-04-25T10:06:11.592772+0000","last_unstale":"2024-04-25T10:06:12.591186+0000","last_undegraded":"2024-04-25T10:06:12.591186+0000","last_fullsized":"2024-04-25T10:06:12.591186+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:12.627498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.584205+0000","last_change":"2024-04-25T10:06:10.594103+0000","last_active":"2024-04-25T10:06:11.584205+0000","last_peered":"2024-04-25T10:06:11.584205+0000","last_clean":"2024-04-25T10:06:11.584205+0000","last_became_active":"2024-04-25T10:06:10.593805+0000","last_became_peered":"2024-04-25T10:06:10.593805+0000","last_unstale":"2024-04-25T10:06:11.584205+0000","last_undegraded":"2024-04-25T10:06:11.584205+0000","last_fullsized":"2024-04-25T10:06:11.584205+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:21.758145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.597476+0000","last_change":"2024-04-25T10:06:11.590223+0000","last_active":"2024-04-25T10:06:11.597476+0000","last_peered":"2024-04-25T10:06:11.597476+0000","last_clean":"2024-04-25T10:06:11.597476+0000","last_became_active":"2024-04-25T10:06:11.589970+0000","last_became_peered":"2024-04-25T10:06:11.589970+0000","last_unstale":"2024-04-25T10:06:11.597476+0000","last_undegraded":"2024-04-25T10:06:11.597476+0000","last_fullsized":"2024-04-25T10:06:11.597476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:44.396817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590364+0000","last_change":"2024-04-25T10:06:10.590741+0000","last_active":"2024-04-25T10:06:12.590364+0000","last_peered":"2024-04-25T10:06:12.590364+0000","last_clean":"2024-04-25T10:06:12.590364+0000","last_became_active":"2024-04-25T10:06:10.590499+0000","last_became_peered":"2024-04-25T10:06:10.590499+0000","last_unstale":"2024-04-25T10:06:12.590364+0000","last_undegraded":"2024-04-25T10:06:12.590364+0000","last_fullsized":"2024-04-25T10:06:12.590364+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:32.676160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599229+0000","last_change":"2024-04-25T10:06:11.594584+0000","last_active":"2024-04-25T10:06:11.599229+0000","last_peered":"2024-04-25T10:06:11.599229+0000","last_clean":"2024-04-25T10:06:11.599229+0000","last_became_active":"2024-04-25T10:06:11.594264+0000","last_became_peered":"2024-04-25T10:06:11.594264+0000","last_unstale":"2024-04-25T10:06:11.599229+0000","last_undegraded":"2024-04-25T10:06:11.599229+0000","last_fullsized":"2024-04-25T10:06:11.599229+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:33.855981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-25T10:06:09.571413+0000","last_change":"2024-04-25T10:06:09.571413+0000","last_active":"2024-04-25T10:06:09.563666+0000","last_peered":"2024-04-25T10:06:09.563666+0000","last_clean":"2024-04-25T10:06:09.563666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:09.571413+0000","last_undegraded":"2024-04-25T10:06:09.571413+0000","last_fullsized":"2024-04-25T10:06:09.571413+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.603137+0000","last_change":"2024-04-25T10:06:11.598368+0000","last_active":"2024-04-25T10:06:11.603137+0000","last_peered":"2024-04-25T10:06:11.603137+0000","last_clean":"2024-04-25T10:06:11.603137+0000","last_became_active":"2024-04-25T10:06:11.597963+0000","last_became_peered":"2024-04-25T10:06:11.597963+0000","last_unstale":"2024-04-25T10:06:11.603137+0000","last_undegraded":"2024-04-25T10:06:11.603137+0000","last_fullsized":"2024-04-25T10:06:11.603137+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:48.190466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-25T10:06:09.571772+0000","last_change":"2024-04-25T10:06:09.571772+0000","last_active":"2024-04-25T10:06:09.563666+0000","last_peered":"2024-04-25T10:06:09.563666+0000","last_clean":"2024-04-25T10:06:09.563666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:09.571772+0000","last_undegraded":"2024-04-25T10:06:09.571772+0000","last_fullsized":"2024-04-25T10:06:09.571772+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610787+0000","last_change":"2024-04-25T10:06:11.603945+0000","last_active":"2024-04-25T10:06:11.610787+0000","last_peered":"2024-04-25T10:06:11.610787+0000","last_clean":"2024-04-25T10:06:11.610787+0000","last_became_active":"2024-04-25T10:06:11.602849+0000","last_became_peered":"2024-04-25T10:06:11.602849+0000","last_unstale":"2024-04-25T10:06:11.610787+0000","last_undegraded":"2024-04-25T10:06:11.610787+0000","last_fullsized":"2024-04-25T10:06:11.610787+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:03.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.601893+0000","last_change":"2024-04-25T10:06:10.579763+0000","last_active":"2024-04-25T10:06:11.601893+0000","last_peered":"2024-04-25T10:06:11.601893+0000","last_clean":"2024-04-25T10:06:11.601893+0000","last_became_active":"2024-04-25T10:06:10.579495+0000","last_became_peered":"2024-04-25T10:06:10.579495+0000","last_unstale":"2024-04-25T10:06:11.601893+0000","last_undegraded":"2024-04-25T10:06:11.601893+0000","last_fullsized":"2024-04-25T10:06:11.601893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:08.212125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.603845+0000","last_change":"2024-04-25T10:06:11.603317+0000","last_active":"2024-04-25T10:06:11.603845+0000","last_peered":"2024-04-25T10:06:11.603845+0000","last_clean":"2024-04-25T10:06:11.603845+0000","last_became_active":"2024-04-25T10:06:11.603066+0000","last_became_peered":"2024-04-25T10:06:11.603066+0000","last_unstale":"2024-04-25T10:06:11.603845+0000","last_undegraded":"2024-04-25T10:06:11.603845+0000","last_fullsized":"2024-04-25T10:06:11.603845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:07.686632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592122+0000","last_change":"2024-04-25T10:06:10.593257+0000","last_active":"2024-04-25T10:06:12.592122+0000","last_peered":"2024-04-25T10:06:12.592122+0000","last_clean":"2024-04-25T10:06:12.592122+0000","last_became_active":"2024-04-25T10:06:10.592737+0000","last_became_peered":"2024-04-25T10:06:10.592737+0000","last_unstale":"2024-04-25T10:06:12.592122+0000","last_undegraded":"2024-04-25T10:06:12.592122+0000","last_fullsized":"2024-04-25T10:06:12.592122+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:38.738216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591639+0000","last_change":"2024-04-25T10:06:11.601746+0000","last_active":"2024-04-25T10:06:12.591639+0000","last_peered":"2024-04-25T10:06:12.591639+0000","last_clean":"2024-04-25T10:06:12.591639+0000","last_became_active":"2024-04-25T10:06:11.600706+0000","last_became_peered":"2024-04-25T10:06:11.600706+0000","last_unstale":"2024-04-25T10:06:12.591639+0000","last_undegraded":"2024-04-25T10:06:12.591639+0000","last_fullsized":"2024-04-25T10:06:12.591639+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:37.328665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590790+0000","last_change":"2024-04-25T10:06:10.589747+0000","last_active":"2024-04-25T10:06:12.590790+0000","last_peered":"2024-04-25T10:06:12.590790+0000","last_clean":"2024-04-25T10:06:12.590790+0000","last_became_active":"2024-04-25T10:06:10.589483+0000","last_became_peered":"2024-04-25T10:06:10.589483+0000","last_unstale":"2024-04-25T10:06:12.590790+0000","last_undegraded":"2024-04-25T10:06:12.590790+0000","last_fullsized":"2024-04-25T10:06:12.590790+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.133966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.605963+0000","last_change":"2024-04-25T10:06:11.599329+0000","last_active":"2024-04-25T10:06:11.605963+0000","last_peered":"2024-04-25T10:06:11.605963+0000","last_clean":"2024-04-25T10:06:11.605963+0000","last_became_active":"2024-04-25T10:06:11.599055+0000","last_became_peered":"2024-04-25T10:06:11.599055+0000","last_unstale":"2024-04-25T10:06:11.605963+0000","last_undegraded":"2024-04-25T10:06:11.605963+0000","last_fullsized":"2024-04-25T10:06:11.605963+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:04.221854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.590118+0000","last_change":"2024-04-25T10:06:10.592105+0000","last_active":"2024-04-25T10:06:11.590118+0000","last_peered":"2024-04-25T10:06:11.590118+0000","last_clean":"2024-04-25T10:06:11.590118+0000","last_became_active":"2024-04-25T10:06:10.591823+0000","last_became_peered":"2024-04-25T10:06:10.591823+0000","last_unstale":"2024-04-25T10:06:11.590118+0000","last_undegraded":"2024-04-25T10:06:11.590118+0000","last_fullsized":"2024-04-25T10:06:11.590118+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:53.425769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590360+0000","last_change":"2024-04-25T10:06:11.585750+0000","last_active":"2024-04-25T10:06:12.590360+0000","last_peered":"2024-04-25T10:06:12.590360+0000","last_clean":"2024-04-25T10:06:12.590360+0000","last_became_active":"2024-04-25T10:06:11.585193+0000","last_became_peered":"2024-04-25T10:06:11.585193+0000","last_unstale":"2024-04-25T10:06:12.590360+0000","last_undegraded":"2024-04-25T10:06:12.590360+0000","last_fullsized":"2024-04-25T10:06:12.590360+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:03.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.589817+0000","last_change":"2024-04-25T10:06:10.590311+0000","last_active":"2024-04-25T10:06:11.589817+0000","last_peered":"2024-04-25T10:06:11.589817+0000","last_clean":"2024-04-25T10:06:11.589817+0000","last_became_active":"2024-04-25T10:06:10.590015+0000","last_became_peered":"2024-04-25T10:06:10.590015+0000","last_unstale":"2024-04-25T10:06:11.589817+0000","last_undegraded":"2024-04-25T10:06:11.589817+0000","last_fullsized":"2024-04-25T10:06:11.589817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:27.524048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.585155+0000","last_change":"2024-04-25T10:06:11.582381+0000","last_active":"2024-04-25T10:06:11.585155+0000","last_peered":"2024-04-25T10:06:11.585155+0000","last_clean":"2024-04-25T10:06:11.585155+0000","last_became_active":"2024-04-25T10:06:11.582109+0000","last_became_peered":"2024-04-25T10:06:11.582109+0000","last_unstale":"2024-04-25T10:06:11.585155+0000","last_undegraded":"2024-04-25T10:06:11.585155+0000","last_fullsized":"2024-04-25T10:06:11.585155+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:41.604772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.604177+0000","last_change":"2024-04-25T10:06:10.589625+0000","last_active":"2024-04-25T10:06:11.604177+0000","last_peered":"2024-04-25T10:06:11.604177+0000","last_clean":"2024-04-25T10:06:11.604177+0000","last_became_active":"2024-04-25T10:06:10.589161+0000","last_became_peered":"2024-04-25T10:06:10.589161+0000","last_unstale":"2024-04-25T10:06:11.604177+0000","last_undegraded":"2024-04-25T10:06:11.604177+0000","last_fullsized":"2024-04-25T10:06:11.604177+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:47.673088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.612662+0000","last_change":"2024-04-25T10:06:11.612101+0000","last_active":"2024-04-25T10:06:11.612662+0000","last_peered":"2024-04-25T10:06:11.612662+0000","last_clean":"2024-04-25T10:06:11.612662+0000","last_became_active":"2024-04-25T10:06:11.611805+0000","last_became_peered":"2024-04-25T10:06:11.611805+0000","last_unstale":"2024-04-25T10:06:11.612662+0000","last_undegraded":"2024-04-25T10:06:11.612662+0000","last_fullsized":"2024-04-25T10:06:11.612662+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:22.287749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.598493+0000","last_change":"2024-04-25T10:06:10.594406+0000","last_active":"2024-04-25T10:06:11.598493+0000","last_peered":"2024-04-25T10:06:11.598493+0000","last_clean":"2024-04-25T10:06:11.598493+0000","last_became_active":"2024-04-25T10:06:10.594129+0000","last_became_peered":"2024-04-25T10:06:10.594129+0000","last_unstale":"2024-04-25T10:06:11.598493+0000","last_undegraded":"2024-04-25T10:06:11.598493+0000","last_fullsized":"2024-04-25T10:06:11.598493+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:19.498453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591710+0000","last_change":"2024-04-25T10:06:11.592494+0000","last_active":"2024-04-25T10:06:12.591710+0000","last_peered":"2024-04-25T10:06:12.591710+0000","last_clean":"2024-04-25T10:06:12.591710+0000","last_became_active":"2024-04-25T10:06:11.592212+0000","last_became_peered":"2024-04-25T10:06:11.592212+0000","last_unstale":"2024-04-25T10:06:12.591710+0000","last_undegraded":"2024-04-25T10:06:12.591710+0000","last_fullsized":"2024-04-25T10:06:12.591710+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:08.065111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590880+0000","last_change":"2024-04-25T10:06:10.588563+0000","last_active":"2024-04-25T10:06:12.590880+0000","last_peered":"2024-04-25T10:06:12.590880+0000","last_clean":"2024-04-25T10:06:12.590880+0000","last_became_active":"2024-04-25T10:06:10.588312+0000","last_became_peered":"2024-04-25T10:06:10.588312+0000","last_unstale":"2024-04-25T10:06:12.590880+0000","last_undegraded":"2024-04-25T10:06:12.590880+0000","last_fullsized":"2024-04-25T10:06:12.590880+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:17.315120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:10.566145+0000","last_change":"2024-04-25T10:06:10.566145+0000","last_active":"2024-04-25T10:06:10.566145+0000","last_peered":"2024-04-25T10:06:10.566145+0000","last_clean":"2024-04-25T10:06:10.566145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:10.566145+0000","last_undegraded":"2024-04-25T10:06:10.566145+0000","last_fullsized":"2024-04-25T10:06:10.566145+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.602393+0000","last_change":"2024-04-25T10:06:10.588662+0000","last_active":"2024-04-25T10:06:11.602393+0000","last_peered":"2024-04-25T10:06:11.602393+0000","last_clean":"2024-04-25T10:06:11.602393+0000","last_became_active":"2024-04-25T10:06:10.588316+0000","last_became_peered":"2024-04-25T10:06:10.588316+0000","last_unstale":"2024-04-25T10:06:11.602393+0000","last_undegraded":"2024-04-25T10:06:11.602393+0000","last_fullsized":"2024-04-25T10:06:11.602393+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:32.850838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599469+0000","last_change":"2024-04-25T10:06:11.595815+0000","last_active":"2024-04-25T10:06:11.599469+0000","last_peered":"2024-04-25T10:06:11.599469+0000","last_clean":"2024-04-25T10:06:11.599469+0000","last_became_active":"2024-04-25T10:06:11.595495+0000","last_became_peered":"2024-04-25T10:06:11.595495+0000","last_unstale":"2024-04-25T10:06:11.599469+0000","last_undegraded":"2024-04-25T10:06:11.599469+0000","last_fullsized":"2024-04-25T10:06:11.599469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:12.627498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.581094+0000","last_change":"2024-04-25T10:06:10.591774+0000","last_active":"2024-04-25T10:06:11.581094+0000","last_peered":"2024-04-25T10:06:11.581094+0000","last_clean":"2024-04-25T10:06:11.581094+0000","last_became_active":"2024-04-25T10:06:10.591412+0000","last_became_peered":"2024-04-25T10:06:10.591412+0000","last_unstale":"2024-04-25T10:06:11.581094+0000","last_undegraded":"2024-04-25T10:06:11.581094+0000","last_fullsized":"2024-04-25T10:06:11.581094+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:17.401215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591675+0000","last_change":"2024-04-25T10:06:11.593318+0000","last_active":"2024-04-25T10:06:12.591675+0000","last_peered":"2024-04-25T10:06:12.591675+0000","last_clean":"2024-04-25T10:06:12.591675+0000","last_became_active":"2024-04-25T10:06:11.593022+0000","last_became_peered":"2024-04-25T10:06:11.593022+0000","last_unstale":"2024-04-25T10:06:12.591675+0000","last_undegraded":"2024-04-25T10:06:12.591675+0000","last_fullsized":"2024-04-25T10:06:12.591675+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:46.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-25T10:06:09.571392+0000","last_change":"2024-04-25T10:06:09.571392+0000","last_active":"2024-04-25T10:06:09.563666+0000","last_peered":"2024-04-25T10:06:09.563666+0000","last_clean":"2024-04-25T10:06:09.563666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:09.571392+0000","last_undegraded":"2024-04-25T10:06:09.571392+0000","last_fullsized":"2024-04-25T10:06:09.571392+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:10.566145+0000","last_change":"2024-04-25T10:06:10.566145+0000","last_active":"2024-04-25T10:06:10.566145+0000","last_peered":"2024-04-25T10:06:10.566145+0000","last_clean":"2024-04-25T10:06:10.566145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:10.566145+0000","last_undegraded":"2024-04-25T10:06:10.566145+0000","last_fullsized":"2024-04-25T10:06:10.566145+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590609+0000","last_change":"2024-04-25T10:06:10.590140+0000","last_active":"2024-04-25T10:06:12.590609+0000","last_peered":"2024-04-25T10:06:12.590609+0000","last_clean":"2024-04-25T10:06:12.590609+0000","last_became_active":"2024-04-25T10:06:10.589015+0000","last_became_peered":"2024-04-25T10:06:10.589015+0000","last_unstale":"2024-04-25T10:06:12.590609+0000","last_undegraded":"2024-04-25T10:06:12.590609+0000","last_fullsized":"2024-04-25T10:06:12.590609+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:18.383218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591617+0000","last_change":"2024-04-25T10:06:11.604928+0000","last_active":"2024-04-25T10:06:12.591617+0000","last_peered":"2024-04-25T10:06:12.591617+0000","last_clean":"2024-04-25T10:06:12.591617+0000","last_became_active":"2024-04-25T10:06:11.604636+0000","last_became_peered":"2024-04-25T10:06:11.604636+0000","last_unstale":"2024-04-25T10:06:12.591617+0000","last_undegraded":"2024-04-25T10:06:12.591617+0000","last_fullsized":"2024-04-25T10:06:12.591617+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:04.221854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591565+0000","last_change":"2024-04-25T10:06:10.590105+0000","last_active":"2024-04-25T10:06:12.591565+0000","last_peered":"2024-04-25T10:06:12.591565+0000","last_clean":"2024-04-25T10:06:12.591565+0000","last_became_active":"2024-04-25T10:06:10.589829+0000","last_became_peered":"2024-04-25T10:06:10.589829+0000","last_unstale":"2024-04-25T10:06:12.591565+0000","last_undegraded":"2024-04-25T10:06:12.591565+0000","last_fullsized":"2024-04-25T10:06:12.591565+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:23.745526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591751+0000","last_change":"2024-04-25T10:06:11.593413+0000","last_active":"2024-04-25T10:06:12.591751+0000","last_peered":"2024-04-25T10:06:12.591751+0000","last_clean":"2024-04-25T10:06:12.591751+0000","last_became_active":"2024-04-25T10:06:11.593155+0000","last_became_peered":"2024-04-25T10:06:11.593155+0000","last_unstale":"2024-04-25T10:06:12.591751+0000","last_undegraded":"2024-04-25T10:06:12.591751+0000","last_fullsized":"2024-04-25T10:06:12.591751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:33.678639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.579429+0000","last_change":"2024-04-25T10:06:10.587062+0000","last_active":"2024-04-25T10:06:11.579429+0000","last_peered":"2024-04-25T10:06:11.579429+0000","last_clean":"2024-04-25T10:06:11.579429+0000","last_became_active":"2024-04-25T10:06:10.586736+0000","last_became_peered":"2024-04-25T10:06:10.586736+0000","last_unstale":"2024-04-25T10:06:11.579429+0000","last_undegraded":"2024-04-25T10:06:11.579429+0000","last_fullsized":"2024-04-25T10:06:11.579429+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:48.572483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599597+0000","last_change":"2024-04-25T10:06:11.595699+0000","last_active":"2024-04-25T10:06:11.599597+0000","last_peered":"2024-04-25T10:06:11.599597+0000","last_clean":"2024-04-25T10:06:11.599597+0000","last_became_active":"2024-04-25T10:06:11.595437+0000","last_became_peered":"2024-04-25T10:06:11.595437+0000","last_unstale":"2024-04-25T10:06:11.599597+0000","last_undegraded":"2024-04-25T10:06:11.599597+0000","last_fullsized":"2024-04-25T10:06:11.599597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:38.022779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591835+0000","last_change":"2024-04-25T10:06:10.588344+0000","last_active":"2024-04-25T10:06:12.591835+0000","last_peered":"2024-04-25T10:06:12.591835+0000","last_clean":"2024-04-25T10:06:12.591835+0000","last_became_active":"2024-04-25T10:06:10.587774+0000","last_became_peered":"2024-04-25T10:06:10.587774+0000","last_unstale":"2024-04-25T10:06:12.591835+0000","last_undegraded":"2024-04-25T10:06:12.591835+0000","last_fullsized":"2024-04-25T10:06:12.591835+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:13.625296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.603613+0000","last_change":"2024-04-25T10:06:11.603042+0000","last_active":"2024-04-25T10:06:11.603613+0000","last_peered":"2024-04-25T10:06:11.603613+0000","last_clean":"2024-04-25T10:06:11.603613+0000","last_became_active":"2024-04-25T10:06:11.598535+0000","last_became_peered":"2024-04-25T10:06:11.598535+0000","last_unstale":"2024-04-25T10:06:11.603613+0000","last_undegraded":"2024-04-25T10:06:11.603613+0000","last_fullsized":"2024-04-25T10:06:11.603613+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:31.070811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-25T10:06:09.571309+0000","last_change":"2024-04-25T10:06:09.571309+0000","last_active":"2024-04-25T10:06:09.563666+0000","last_peered":"2024-04-25T10:06:09.563666+0000","last_clean":"2024-04-25T10:06:09.563666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:09.571309+0000","last_undegraded":"2024-04-25T10:06:09.571309+0000","last_fullsized":"2024-04-25T10:06:09.571309+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.605940+0000","last_change":"2024-04-25T10:06:11.603834+0000","last_active":"2024-04-25T10:06:11.605940+0000","last_peered":"2024-04-25T10:06:11.605940+0000","last_clean":"2024-04-25T10:06:11.605940+0000","last_became_active":"2024-04-25T10:06:11.603541+0000","last_became_peered":"2024-04-25T10:06:11.603541+0000","last_unstale":"2024-04-25T10:06:11.605940+0000","last_undegraded":"2024-04-25T10:06:11.605940+0000","last_fullsized":"2024-04-25T10:06:11.605940+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:08.065111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592173+0000","last_change":"2024-04-25T10:06:10.590856+0000","last_active":"2024-04-25T10:06:12.592173+0000","last_peered":"2024-04-25T10:06:12.592173+0000","last_clean":"2024-04-25T10:06:12.592173+0000","last_became_active":"2024-04-25T10:06:10.590571+0000","last_became_peered":"2024-04-25T10:06:10.590571+0000","last_unstale":"2024-04-25T10:06:12.592173+0000","last_undegraded":"2024-04-25T10:06:12.592173+0000","last_fullsized":"2024-04-25T10:06:12.592173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:01.814874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.596509+0000","last_change":"2024-04-25T10:06:11.595775+0000","last_active":"2024-04-25T10:06:11.596509+0000","last_peered":"2024-04-25T10:06:11.596509+0000","last_clean":"2024-04-25T10:06:11.596509+0000","last_became_active":"2024-04-25T10:06:11.595387+0000","last_became_peered":"2024-04-25T10:06:11.595387+0000","last_unstale":"2024-04-25T10:06:11.596509+0000","last_undegraded":"2024-04-25T10:06:11.596509+0000","last_fullsized":"2024-04-25T10:06:11.596509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:40.079857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592115+0000","last_change":"2024-04-25T10:06:10.588101+0000","last_active":"2024-04-25T10:06:12.592115+0000","last_peered":"2024-04-25T10:06:12.592115+0000","last_clean":"2024-04-25T10:06:12.592115+0000","last_became_active":"2024-04-25T10:06:10.587808+0000","last_became_peered":"2024-04-25T10:06:10.587808+0000","last_unstale":"2024-04-25T10:06:12.592115+0000","last_undegraded":"2024-04-25T10:06:12.592115+0000","last_fullsized":"2024-04-25T10:06:12.592115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.133966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610579+0000","last_change":"2024-04-25T10:06:11.599755+0000","last_active":"2024-04-25T10:06:11.610579+0000","last_peered":"2024-04-25T10:06:11.610579+0000","last_clean":"2024-04-25T10:06:11.610579+0000","last_became_active":"2024-04-25T10:06:11.599317+0000","last_became_peered":"2024-04-25T10:06:11.599317+0000","last_unstale":"2024-04-25T10:06:11.610579+0000","last_undegraded":"2024-04-25T10:06:11.610579+0000","last_fullsized":"2024-04-25T10:06:11.610579+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:37.328665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591917+0000","last_change":"2024-04-25T10:06:10.591156+0000","last_active":"2024-04-25T10:06:12.591917+0000","last_peered":"2024-04-25T10:06:12.591917+0000","last_clean":"2024-04-25T10:06:12.591917+0000","last_became_active":"2024-04-25T10:06:10.590362+0000","last_became_peered":"2024-04-25T10:06:10.590362+0000","last_unstale":"2024-04-25T10:06:12.591917+0000","last_undegraded":"2024-04-25T10:06:12.591917+0000","last_fullsized":"2024-04-25T10:06:12.591917+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:32.052858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610758+0000","last_change":"2024-04-25T10:06:11.603491+0000","last_active":"2024-04-25T10:06:11.610758+0000","last_peered":"2024-04-25T10:06:11.610758+0000","last_clean":"2024-04-25T10:06:11.610758+0000","last_became_active":"2024-04-25T10:06:11.600221+0000","last_became_peered":"2024-04-25T10:06:11.600221+0000","last_unstale":"2024-04-25T10:06:11.610758+0000","last_undegraded":"2024-04-25T10:06:11.610758+0000","last_fullsized":"2024-04-25T10:06:11.610758+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590413+0000","last_change":"2024-04-25T10:06:10.589304+0000","last_active":"2024-04-25T10:06:12.590413+0000","last_peered":"2024-04-25T10:06:12.590413+0000","last_clean":"2024-04-25T10:06:12.590413+0000","last_became_active":"2024-04-25T10:06:10.588934+0000","last_became_peered":"2024-04-25T10:06:10.588934+0000","last_unstale":"2024-04-25T10:06:12.590413+0000","last_undegraded":"2024-04-25T10:06:12.590413+0000","last_fullsized":"2024-04-25T10:06:12.590413+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:07.062632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.602797+0000","last_change":"2024-04-25T10:06:11.597255+0000","last_active":"2024-04-25T10:06:11.602797+0000","last_peered":"2024-04-25T10:06:11.602797+0000","last_clean":"2024-04-25T10:06:11.602797+0000","last_became_active":"2024-04-25T10:06:11.596954+0000","last_became_peered":"2024-04-25T10:06:11.596954+0000","last_unstale":"2024-04-25T10:06:11.602797+0000","last_undegraded":"2024-04-25T10:06:11.602797+0000","last_fullsized":"2024-04-25T10:06:11.602797+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:13.167235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.596522+0000","last_change":"2024-04-25T10:06:10.589615+0000","last_active":"2024-04-25T10:06:11.596522+0000","last_peered":"2024-04-25T10:06:11.596522+0000","last_clean":"2024-04-25T10:06:11.596522+0000","last_became_active":"2024-04-25T10:06:10.589310+0000","last_became_peered":"2024-04-25T10:06:10.589310+0000","last_unstale":"2024-04-25T10:06:11.596522+0000","last_undegraded":"2024-04-25T10:06:11.596522+0000","last_fullsized":"2024-04-25T10:06:11.596522+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:00.780604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.597484+0000","last_change":"2024-04-25T10:06:11.596201+0000","last_active":"2024-04-25T10:06:11.597484+0000","last_peered":"2024-04-25T10:06:11.597484+0000","last_clean":"2024-04-25T10:06:11.597484+0000","last_became_active":"2024-04-25T10:06:11.595880+0000","last_became_peered":"2024-04-25T10:06:11.595880+0000","last_unstale":"2024-04-25T10:06:11.597484+0000","last_undegraded":"2024-04-25T10:06:11.597484+0000","last_fullsized":"2024-04-25T10:06:11.597484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:36.054034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.580878+0000","last_change":"2024-04-25T10:06:10.587821+0000","last_active":"2024-04-25T10:06:11.580878+0000","last_peered":"2024-04-25T10:06:11.580878+0000","last_clean":"2024-04-25T10:06:11.580878+0000","last_became_active":"2024-04-25T10:06:10.587520+0000","last_became_peered":"2024-04-25T10:06:10.587520+0000","last_unstale":"2024-04-25T10:06:11.580878+0000","last_undegraded":"2024-04-25T10:06:11.580878+0000","last_fullsized":"2024-04-25T10:06:11.580878+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:17.031658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":198,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591162+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:06:12.591162+0000","last_peered":"2024-04-25T10:06:12.591162+0000","last_clean":"2024-04-25T10:06:12.591162+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:06:12.591162+0000","last_undegraded":"2024-04-25T10:06:12.591162+0000","last_fullsized":"2024-04-25T10:06:12.591162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":118,"acting":118,"num_store_stats":7},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":150789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":152379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":152401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":405504,"data_stored":151543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":393216,"data_stored":150801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":747505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":393216,"data_stored":150857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":946176,"data_stored":722268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:14.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:14.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.713+0000 7fb5da676640 1 Processor -- start 2024-04-25T10:06:14.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.713+0000 7fb5da676640 1 -- start start 2024-04-25T10:06:14.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.713+0000 7fb5da676640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb5d415ba10 0x7fb5d4099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.713+0000 7fb5da676640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb5d406c3c0 0x7fb5d40cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.713+0000 7fb5da676640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d406c900 con 0x7fb5d4099f80 2024-04-25T10:06:14.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.713+0000 7fb5da676640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d40cbe50 con 0x7fb5d406c3c0 2024-04-25T10:06:14.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.713+0000 7fb5da676640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d409a3d0 con 0x7fb5d415ba10 2024-04-25T10:06:14.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.714+0000 7fb5d8c13640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb5d406c3c0 0x7fb5d40cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.714+0000 7fb5d8c13640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb5d406c3c0 0x7fb5d40cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49486/0 (socket says 172.21.15.16:49486) 2024-04-25T10:06:14.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.714+0000 7fb5d8c13640 1 -- 172.21.15.16:0/3410562520 learned_addr learned my addr 172.21.15.16:0/3410562520 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:14.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.714+0000 7fb5d3fff640 1 --2- 172.21.15.16:0/3410562520 >> v2:172.21.15.186:3300/0 conn(0x7fb5d415ba10 0x7fb5d4099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.714+0000 7fb5d8c13640 1 -- 172.21.15.16:0/3410562520 >> v2:172.21.15.186:3300/0 conn(0x7fb5d415ba10 msgr2=0x7fb5d4099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.714+0000 7fb5d8c13640 1 --2- 172.21.15.16:0/3410562520 >> v2:172.21.15.186:3300/0 conn(0x7fb5d415ba10 0x7fb5d4099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.714+0000 7fb5d8c13640 1 -- 172.21.15.16:0/3410562520 >> v1:172.21.15.186:6789/0 conn(0x7fb5d4099f80 legacy=0x7fb5d406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.714+0000 7fb5d8c13640 1 -- 172.21.15.16:0/3410562520 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d406cbe0 con 0x7fb5d406c3c0 2024-04-25T10:06:14.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.715+0000 7fb5d8c13640 1 --2- 172.21.15.16:0/3410562520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d406c3c0 0x7fb5d40cd220 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb5d409b2f0 tx=0x7fb5cc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c2a4f9a4b01710 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.715+0000 7fb5d2ffd640 1 -- 172.21.15.16:0/3410562520 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc036070 con 0x7fb5d406c3c0 2024-04-25T10:06:14.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.715+0000 7fb5d2ffd640 1 -- 172.21.15.16:0/3410562520 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5cc031600 con 0x7fb5d406c3c0 2024-04-25T10:06:14.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.715+0000 7fb5d2ffd640 1 -- 172.21.15.16:0/3410562520 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc0319d0 con 0x7fb5d406c3c0 2024-04-25T10:06:14.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.715+0000 7fb5da676640 1 -- 172.21.15.16:0/3410562520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d406c3c0 msgr2=0x7fb5d40cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.715+0000 7fb5da676640 1 --2- 172.21.15.16:0/3410562520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d406c3c0 0x7fb5d40cd220 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb5d409b2f0 tx=0x7fb5cc02a730 comp rx=0 tx=0).stop 2024-04-25T10:06:14.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.715+0000 7fb5da676640 1 -- 172.21.15.16:0/3410562520 shutdown_connections 2024-04-25T10:06:14.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.715+0000 7fb5da676640 1 --2- 172.21.15.16:0/3410562520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d406c3c0 0x7fb5d40cd220 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.715+0000 7fb5da676640 1 --2- 172.21.15.16:0/3410562520 >> v2:172.21.15.186:3300/0 conn(0x7fb5d415ba10 0x7fb5d4099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.715+0000 7fb5da676640 1 -- 172.21.15.16:0/3410562520 >> 172.21.15.16:0/3410562520 conn(0x7fb5d40b32f0 msgr2=0x7fb5d40b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.715+0000 7fb5da676640 1 -- 172.21.15.16:0/3410562520 shutdown_connections 2024-04-25T10:06:14.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.715+0000 7fb5da676640 1 -- 172.21.15.16:0/3410562520 wait complete. 2024-04-25T10:06:14.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.717+0000 7fb5da676640 1 Processor -- start 2024-04-25T10:06:14.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.717+0000 7fb5da676640 1 -- start start 2024-04-25T10:06:14.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.717+0000 7fb5da676640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d406c3c0 0x7fb5d41f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.717+0000 7fb5da676640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5d4099f80 0x7fb5d414c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.717+0000 7fb5da676640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb5d415ba10 0x7fb5d4161930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.717+0000 7fb5da676640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40cd760 con 0x7fb5d406c3c0 2024-04-25T10:06:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.717+0000 7fb5da676640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40cfe80 con 0x7fb5d4099f80 2024-04-25T10:06:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.717+0000 7fb5da676640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40cd8e0 con 0x7fb5d415ba10 2024-04-25T10:06:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.718+0000 7fb5d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d406c3c0 0x7fb5d41f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.718+0000 7fb5d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d406c3c0 0x7fb5d41f3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49502/0 (socket says 172.21.15.16:49502) 2024-04-25T10:06:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.718+0000 7fb5d3fff640 1 -- 172.21.15.16:0/1029172766 learned_addr learned my addr 172.21.15.16:0/1029172766 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.718+0000 7fb5d37fe640 1 --2- 172.21.15.16:0/1029172766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5d4099f80 0x7fb5d414c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.718+0000 7fb5d3fff640 1 -- 172.21.15.16:0/1029172766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb5d415ba10 msgr2=0x7fb5d4161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.718+0000 7fb5d8c13640 1 --2- 172.21.15.16:0/1029172766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb5d415ba10 0x7fb5d4161930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.718+0000 7fb5d3fff640 1 --2- 172.21.15.16:0/1029172766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb5d415ba10 0x7fb5d4161930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.718+0000 7fb5d3fff640 1 -- 172.21.15.16:0/1029172766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5d4099f80 msgr2=0x7fb5d414c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.718+0000 7fb5d3fff640 1 --2- 172.21.15.16:0/1029172766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5d4099f80 0x7fb5d414c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.718+0000 7fb5d3fff640 1 -- 172.21.15.16:0/1029172766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d414a790 con 0x7fb5d406c3c0 2024-04-25T10:06:14.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.718+0000 7fb5d37fe640 1 --2- 172.21.15.16:0/1029172766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5d4099f80 0x7fb5d414c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:14.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.718+0000 7fb5d3fff640 1 --2- 172.21.15.16:0/1029172766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d406c3c0 0x7fb5d41f3f70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400d5d0 tx=0x7fb5c400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.718+0000 7fb5d0ff9640 1 -- 172.21.15.16:0/1029172766 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c40125f0 con 0x7fb5d406c3c0 2024-04-25T10:06:14.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.718+0000 7fb5d0ff9640 1 -- 172.21.15.16:0/1029172766 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c4019070 con 0x7fb5d406c3c0 2024-04-25T10:06:14.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.718+0000 7fb5d0ff9640 1 -- 172.21.15.16:0/1029172766 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c40129f0 con 0x7fb5d406c3c0 2024-04-25T10:06:14.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.718+0000 7fb5da676640 1 -- 172.21.15.16:0/1029172766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d414d070 con 0x7fb5d406c3c0 2024-04-25T10:06:14.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.718+0000 7fb5da676640 1 -- 172.21.15.16:0/1029172766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d414d630 con 0x7fb5d406c3c0 2024-04-25T10:06:14.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.719+0000 7fb5da676640 1 -- 172.21.15.16:0/1029172766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d40749b0 con 0x7fb5d406c3c0 2024-04-25T10:06:14.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.720+0000 7fb5d0ff9640 1 -- 172.21.15.16:0/1029172766 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5c4012b90 con 0x7fb5d406c3c0 2024-04-25T10:06:14.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.720+0000 7fb5d0ff9640 1 --2- 172.21.15.16:0/1029172766 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb5c006b500 0x7fb5c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.720+0000 7fb5d0ff9640 1 -- 172.21.15.16:0/1029172766 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5c408a640 con 0x7fb5d406c3c0 2024-04-25T10:06:14.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.720+0000 7fb5d37fe640 1 --2- 172.21.15.16:0/1029172766 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb5c006b500 0x7fb5c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.721+0000 7fb5d37fe640 1 --2- 172.21.15.16:0/1029172766 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb5c006b500 0x7fb5c006d9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40b0d90 tx=0x7fb5c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.724+0000 7fb5d0ff9640 1 -- 172.21.15.16:0/1029172766 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5c40594c0 con 0x7fb5d406c3c0 2024-04-25T10:06:14.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.906+0000 7fb5da676640 1 -- 172.21.15.16:0/1029172766 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb5d415de10 con 0x7fb5c006b500 2024-04-25T10:06:14.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.912+0000 7fb5d0ff9640 1 -- 172.21.15.16:0/1029172766 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250788 (secure 0 0 0) 0x7fb5d415de10 con 0x7fb5c006b500 2024-04-25T10:06:14.921 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:06:14.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.915+0000 7fb5da676640 1 -- 172.21.15.16:0/1029172766 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb5c006b500 msgr2=0x7fb5c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.915+0000 7fb5da676640 1 --2- 172.21.15.16:0/1029172766 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb5c006b500 0x7fb5c006d9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40b0d90 tx=0x7fb5c8009040 comp rx=0 tx=0).stop 2024-04-25T10:06:14.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.915+0000 7fb5da676640 1 -- 172.21.15.16:0/1029172766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d406c3c0 msgr2=0x7fb5d41f3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.915+0000 7fb5da676640 1 --2- 172.21.15.16:0/1029172766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d406c3c0 0x7fb5d41f3f70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400d5d0 tx=0x7fb5c400da90 comp rx=0 tx=0).stop 2024-04-25T10:06:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.915+0000 7fb5da676640 1 -- 172.21.15.16:0/1029172766 shutdown_connections 2024-04-25T10:06:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.915+0000 7fb5da676640 1 --2- 172.21.15.16:0/1029172766 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb5c006b500 0x7fb5c006d9c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.915+0000 7fb5da676640 1 --2- 172.21.15.16:0/1029172766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb5d415ba10 0x7fb5d4161930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.915+0000 7fb5da676640 1 --2- 172.21.15.16:0/1029172766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5d4099f80 0x7fb5d414c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.915+0000 7fb5da676640 1 --2- 172.21.15.16:0/1029172766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d406c3c0 0x7fb5d41f3f70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.915+0000 7fb5da676640 1 -- 172.21.15.16:0/1029172766 >> 172.21.15.16:0/1029172766 conn(0x7fb5d40b32f0 msgr2=0x7fb5d415ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.916+0000 7fb5da676640 1 -- 172.21.15.16:0/1029172766 shutdown_connections 2024-04-25T10:06:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:14.916+0000 7fb5da676640 1 -- 172.21.15.16:0/1029172766 wait complete. 2024-04-25T10:06:14.932 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-04-25T10:06:13.795375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":184,"acting":184,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237044,"kb_used_data":4236,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737676,"statfs":{"total":773094113280,"available":772851380224,"internally_reserved":0,"allocated":4337664,"data_stored":2378543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.228282"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590900+0000","last_change":"2024-04-25T10:06:11.596197+0000","last_active":"2024-04-25T10:06:12.590900+0000","last_peered":"2024-04-25T10:06:12.590900+0000","last_clean":"2024-04-25T10:06:12.590900+0000","last_became_active":"2024-04-25T10:06:11.595896+0000","last_became_peered":"2024-04-25T10:06:11.595896+0000","last_unstale":"2024-04-25T10:06:12.590900+0000","last_undegraded":"2024-04-25T10:06:12.590900+0000","last_fullsized":"2024-04-25T10:06:12.590900+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:50.014644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:10.566145+0000","last_change":"2024-04-25T10:06:10.566145+0000","last_active":"2024-04-25T10:06:10.566145+0000","last_peered":"2024-04-25T10:06:10.566145+0000","last_clean":"2024-04-25T10:06:10.566145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:10.566145+0000","last_undegraded":"2024-04-25T10:06:10.566145+0000","last_fullsized":"2024-04-25T10:06:10.566145+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591774+0000","last_change":"2024-04-25T10:06:11.588804+0000","last_active":"2024-04-25T10:06:12.591774+0000","last_peered":"2024-04-25T10:06:12.591774+0000","last_clean":"2024-04-25T10:06:12.591774+0000","last_became_active":"2024-04-25T10:06:11.588533+0000","last_became_peered":"2024-04-25T10:06:11.588533+0000","last_unstale":"2024-04-25T10:06:12.591774+0000","last_undegraded":"2024-04-25T10:06:12.591774+0000","last_fullsized":"2024-04-25T10:06:12.591774+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:12.136446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610805+0000","last_change":"2024-04-25T10:06:11.603412+0000","last_active":"2024-04-25T10:06:11.610805+0000","last_peered":"2024-04-25T10:06:11.610805+0000","last_clean":"2024-04-25T10:06:11.610805+0000","last_became_active":"2024-04-25T10:06:11.602936+0000","last_became_peered":"2024-04-25T10:06:11.602936+0000","last_unstale":"2024-04-25T10:06:11.610805+0000","last_undegraded":"2024-04-25T10:06:11.610805+0000","last_fullsized":"2024-04-25T10:06:11.610805+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.403695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:10.566145+0000","last_change":"2024-04-25T10:06:10.566145+0000","last_active":"2024-04-25T10:06:10.566145+0000","last_peered":"2024-04-25T10:06:10.566145+0000","last_clean":"2024-04-25T10:06:10.566145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:10.566145+0000","last_undegraded":"2024-04-25T10:06:10.566145+0000","last_fullsized":"2024-04-25T10:06:10.566145+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599412+0000","last_change":"2024-04-25T10:06:11.594811+0000","last_active":"2024-04-25T10:06:11.599412+0000","last_peered":"2024-04-25T10:06:11.599412+0000","last_clean":"2024-04-25T10:06:11.599412+0000","last_became_active":"2024-04-25T10:06:11.594494+0000","last_became_peered":"2024-04-25T10:06:11.594494+0000","last_unstale":"2024-04-25T10:06:11.599412+0000","last_undegraded":"2024-04-25T10:06:11.599412+0000","last_fullsized":"2024-04-25T10:06:11.599412+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:02.231151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590302+0000","last_change":"2024-04-25T10:06:11.585486+0000","last_active":"2024-04-25T10:06:12.590302+0000","last_peered":"2024-04-25T10:06:12.590302+0000","last_clean":"2024-04-25T10:06:12.590302+0000","last_became_active":"2024-04-25T10:06:11.585167+0000","last_became_peered":"2024-04-25T10:06:11.585167+0000","last_unstale":"2024-04-25T10:06:12.590302+0000","last_undegraded":"2024-04-25T10:06:12.590302+0000","last_fullsized":"2024-04-25T10:06:12.590302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590232+0000","last_change":"2024-04-25T10:06:11.599766+0000","last_active":"2024-04-25T10:06:12.590232+0000","last_peered":"2024-04-25T10:06:12.590232+0000","last_clean":"2024-04-25T10:06:12.590232+0000","last_became_active":"2024-04-25T10:06:11.599473+0000","last_became_peered":"2024-04-25T10:06:11.599473+0000","last_unstale":"2024-04-25T10:06:12.590232+0000","last_undegraded":"2024-04-25T10:06:12.590232+0000","last_fullsized":"2024-04-25T10:06:12.590232+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:46.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591773+0000","last_change":"2024-04-25T10:06:11.601172+0000","last_active":"2024-04-25T10:06:12.591773+0000","last_peered":"2024-04-25T10:06:12.591773+0000","last_clean":"2024-04-25T10:06:12.591773+0000","last_became_active":"2024-04-25T10:06:11.600890+0000","last_became_peered":"2024-04-25T10:06:11.600890+0000","last_unstale":"2024-04-25T10:06:12.591773+0000","last_undegraded":"2024-04-25T10:06:12.591773+0000","last_fullsized":"2024-04-25T10:06:12.591773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:33.055338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.614185+0000","last_change":"2024-04-25T10:06:11.611336+0000","last_active":"2024-04-25T10:06:11.614185+0000","last_peered":"2024-04-25T10:06:11.614185+0000","last_clean":"2024-04-25T10:06:11.614185+0000","last_became_active":"2024-04-25T10:06:11.610883+0000","last_became_peered":"2024-04-25T10:06:11.610883+0000","last_unstale":"2024-04-25T10:06:11.614185+0000","last_undegraded":"2024-04-25T10:06:11.614185+0000","last_fullsized":"2024-04-25T10:06:11.614185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:50.014644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.606907+0000","last_change":"2024-04-25T10:06:11.606133+0000","last_active":"2024-04-25T10:06:11.606907+0000","last_peered":"2024-04-25T10:06:11.606907+0000","last_clean":"2024-04-25T10:06:11.606907+0000","last_became_active":"2024-04-25T10:06:11.605855+0000","last_became_peered":"2024-04-25T10:06:11.605855+0000","last_unstale":"2024-04-25T10:06:11.606907+0000","last_undegraded":"2024-04-25T10:06:11.606907+0000","last_fullsized":"2024-04-25T10:06:11.606907+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:35.491727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591678+0000","last_change":"2024-04-25T10:06:11.598791+0000","last_active":"2024-04-25T10:06:12.591678+0000","last_peered":"2024-04-25T10:06:12.591678+0000","last_clean":"2024-04-25T10:06:12.591678+0000","last_became_active":"2024-04-25T10:06:11.598511+0000","last_became_peered":"2024-04-25T10:06:11.598511+0000","last_unstale":"2024-04-25T10:06:12.591678+0000","last_undegraded":"2024-04-25T10:06:12.591678+0000","last_fullsized":"2024-04-25T10:06:12.591678+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:18.899344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592055+0000","last_change":"2024-04-25T10:06:11.602586+0000","last_active":"2024-04-25T10:06:12.592055+0000","last_peered":"2024-04-25T10:06:12.592055+0000","last_clean":"2024-04-25T10:06:12.592055+0000","last_became_active":"2024-04-25T10:06:11.602219+0000","last_became_peered":"2024-04-25T10:06:11.602219+0000","last_unstale":"2024-04-25T10:06:12.592055+0000","last_undegraded":"2024-04-25T10:06:12.592055+0000","last_fullsized":"2024-04-25T10:06:12.592055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610597+0000","last_change":"2024-04-25T10:06:11.599901+0000","last_active":"2024-04-25T10:06:11.610597+0000","last_peered":"2024-04-25T10:06:11.610597+0000","last_clean":"2024-04-25T10:06:11.610597+0000","last_became_active":"2024-04-25T10:06:11.599632+0000","last_became_peered":"2024-04-25T10:06:11.599632+0000","last_unstale":"2024-04-25T10:06:11.610597+0000","last_undegraded":"2024-04-25T10:06:11.610597+0000","last_fullsized":"2024-04-25T10:06:11.610597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:46.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.592021+0000","last_change":"2024-04-25T10:06:11.587570+0000","last_active":"2024-04-25T10:06:11.592021+0000","last_peered":"2024-04-25T10:06:11.592021+0000","last_clean":"2024-04-25T10:06:11.592021+0000","last_became_active":"2024-04-25T10:06:11.587301+0000","last_became_peered":"2024-04-25T10:06:11.587301+0000","last_unstale":"2024-04-25T10:06:11.592021+0000","last_undegraded":"2024-04-25T10:06:11.592021+0000","last_fullsized":"2024-04-25T10:06:11.592021+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.597882+0000","last_change":"2024-04-25T10:06:11.597081+0000","last_active":"2024-04-25T10:06:11.597882+0000","last_peered":"2024-04-25T10:06:11.597882+0000","last_clean":"2024-04-25T10:06:11.597882+0000","last_became_active":"2024-04-25T10:06:11.596781+0000","last_became_peered":"2024-04-25T10:06:11.596781+0000","last_unstale":"2024-04-25T10:06:11.597882+0000","last_undegraded":"2024-04-25T10:06:11.597882+0000","last_fullsized":"2024-04-25T10:06:11.597882+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:11.365920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.589022+0000","last_change":"2024-04-25T10:06:11.587969+0000","last_active":"2024-04-25T10:06:11.589022+0000","last_peered":"2024-04-25T10:06:11.589022+0000","last_clean":"2024-04-25T10:06:11.589022+0000","last_became_active":"2024-04-25T10:06:11.587084+0000","last_became_peered":"2024-04-25T10:06:11.587084+0000","last_unstale":"2024-04-25T10:06:11.589022+0000","last_undegraded":"2024-04-25T10:06:11.589022+0000","last_fullsized":"2024-04-25T10:06:11.589022+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:47.208707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599620+0000","last_change":"2024-04-25T10:06:11.596669+0000","last_active":"2024-04-25T10:06:11.599620+0000","last_peered":"2024-04-25T10:06:11.599620+0000","last_clean":"2024-04-25T10:06:11.599620+0000","last_became_active":"2024-04-25T10:06:11.596091+0000","last_became_peered":"2024-04-25T10:06:11.596091+0000","last_unstale":"2024-04-25T10:06:11.599620+0000","last_undegraded":"2024-04-25T10:06:11.599620+0000","last_fullsized":"2024-04-25T10:06:11.599620+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:18.899344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599443+0000","last_change":"2024-04-25T10:06:11.594701+0000","last_active":"2024-04-25T10:06:11.599443+0000","last_peered":"2024-04-25T10:06:11.599443+0000","last_clean":"2024-04-25T10:06:11.599443+0000","last_became_active":"2024-04-25T10:06:11.594376+0000","last_became_peered":"2024-04-25T10:06:11.594376+0000","last_unstale":"2024-04-25T10:06:11.599443+0000","last_undegraded":"2024-04-25T10:06:11.599443+0000","last_fullsized":"2024-04-25T10:06:11.599443+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:53.157588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592096+0000","last_change":"2024-04-25T10:06:11.597575+0000","last_active":"2024-04-25T10:06:12.592096+0000","last_peered":"2024-04-25T10:06:12.592096+0000","last_clean":"2024-04-25T10:06:12.592096+0000","last_became_active":"2024-04-25T10:06:11.596710+0000","last_became_peered":"2024-04-25T10:06:11.596710+0000","last_unstale":"2024-04-25T10:06:12.592096+0000","last_undegraded":"2024-04-25T10:06:12.592096+0000","last_fullsized":"2024-04-25T10:06:12.592096+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.595406+0000","last_change":"2024-04-25T10:06:11.594514+0000","last_active":"2024-04-25T10:06:11.595406+0000","last_peered":"2024-04-25T10:06:11.595406+0000","last_clean":"2024-04-25T10:06:11.595406+0000","last_became_active":"2024-04-25T10:06:11.594231+0000","last_became_peered":"2024-04-25T10:06:11.594231+0000","last_unstale":"2024-04-25T10:06:11.595406+0000","last_undegraded":"2024-04-25T10:06:11.595406+0000","last_fullsized":"2024-04-25T10:06:11.595406+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:03.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590474+0000","last_change":"2024-04-25T10:06:11.595941+0000","last_active":"2024-04-25T10:06:12.590474+0000","last_peered":"2024-04-25T10:06:12.590474+0000","last_clean":"2024-04-25T10:06:12.590474+0000","last_became_active":"2024-04-25T10:06:11.595598+0000","last_became_peered":"2024-04-25T10:06:11.595598+0000","last_unstale":"2024-04-25T10:06:12.590474+0000","last_undegraded":"2024-04-25T10:06:12.590474+0000","last_fullsized":"2024-04-25T10:06:12.590474+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:40.079857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591540+0000","last_change":"2024-04-25T10:06:11.599549+0000","last_active":"2024-04-25T10:06:12.591540+0000","last_peered":"2024-04-25T10:06:12.591540+0000","last_clean":"2024-04-25T10:06:12.591540+0000","last_became_active":"2024-04-25T10:06:11.599256+0000","last_became_peered":"2024-04-25T10:06:11.599256+0000","last_unstale":"2024-04-25T10:06:12.591540+0000","last_undegraded":"2024-04-25T10:06:12.591540+0000","last_fullsized":"2024-04-25T10:06:12.591540+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:48.190466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590264+0000","last_change":"2024-04-25T10:06:11.605175+0000","last_active":"2024-04-25T10:06:12.590264+0000","last_peered":"2024-04-25T10:06:12.590264+0000","last_clean":"2024-04-25T10:06:12.590264+0000","last_became_active":"2024-04-25T10:06:11.604773+0000","last_became_peered":"2024-04-25T10:06:11.604773+0000","last_unstale":"2024-04-25T10:06:12.590264+0000","last_undegraded":"2024-04-25T10:06:12.590264+0000","last_fullsized":"2024-04-25T10:06:12.590264+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.403695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611074+0000","last_change":"2024-04-25T10:06:11.605194+0000","last_active":"2024-04-25T10:06:11.611074+0000","last_peered":"2024-04-25T10:06:11.611074+0000","last_clean":"2024-04-25T10:06:11.611074+0000","last_became_active":"2024-04-25T10:06:11.604595+0000","last_became_peered":"2024-04-25T10:06:11.604595+0000","last_unstale":"2024-04-25T10:06:11.611074+0000","last_undegraded":"2024-04-25T10:06:11.611074+0000","last_fullsized":"2024-04-25T10:06:11.611074+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:36.054034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611662+0000","last_change":"2024-04-25T10:06:11.603505+0000","last_active":"2024-04-25T10:06:11.611662+0000","last_peered":"2024-04-25T10:06:11.611662+0000","last_clean":"2024-04-25T10:06:11.611662+0000","last_became_active":"2024-04-25T10:06:11.603215+0000","last_became_peered":"2024-04-25T10:06:11.603215+0000","last_unstale":"2024-04-25T10:06:11.611662+0000","last_undegraded":"2024-04-25T10:06:11.611662+0000","last_fullsized":"2024-04-25T10:06:11.611662+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:10.566145+0000","last_change":"2024-04-25T10:06:10.566145+0000","last_active":"2024-04-25T10:06:10.566145+0000","last_peered":"2024-04-25T10:06:10.566145+0000","last_clean":"2024-04-25T10:06:10.566145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:10.566145+0000","last_undegraded":"2024-04-25T10:06:10.566145+0000","last_fullsized":"2024-04-25T10:06:10.566145+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591554+0000","last_change":"2024-04-25T10:06:11.601007+0000","last_active":"2024-04-25T10:06:12.591554+0000","last_peered":"2024-04-25T10:06:12.591554+0000","last_clean":"2024-04-25T10:06:12.591554+0000","last_became_active":"2024-04-25T10:06:11.600297+0000","last_became_peered":"2024-04-25T10:06:11.600297+0000","last_unstale":"2024-04-25T10:06:12.591554+0000","last_undegraded":"2024-04-25T10:06:12.591554+0000","last_fullsized":"2024-04-25T10:06:12.591554+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:18.645477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.601405+0000","last_change":"2024-04-25T10:06:11.600447+0000","last_active":"2024-04-25T10:06:11.601405+0000","last_peered":"2024-04-25T10:06:11.601405+0000","last_clean":"2024-04-25T10:06:11.601405+0000","last_became_active":"2024-04-25T10:06:11.600160+0000","last_became_peered":"2024-04-25T10:06:11.600160+0000","last_unstale":"2024-04-25T10:06:11.601405+0000","last_undegraded":"2024-04-25T10:06:11.601405+0000","last_fullsized":"2024-04-25T10:06:11.601405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:50.014644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.591617+0000","last_change":"2024-04-25T10:06:11.590349+0000","last_active":"2024-04-25T10:06:11.591617+0000","last_peered":"2024-04-25T10:06:11.591617+0000","last_clean":"2024-04-25T10:06:11.591617+0000","last_became_active":"2024-04-25T10:06:11.590090+0000","last_became_peered":"2024-04-25T10:06:11.590090+0000","last_unstale":"2024-04-25T10:06:11.591617+0000","last_undegraded":"2024-04-25T10:06:11.591617+0000","last_fullsized":"2024-04-25T10:06:11.591617+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590344+0000","last_change":"2024-04-25T10:06:11.597956+0000","last_active":"2024-04-25T10:06:12.590344+0000","last_peered":"2024-04-25T10:06:12.590344+0000","last_clean":"2024-04-25T10:06:12.590344+0000","last_became_active":"2024-04-25T10:06:11.597687+0000","last_became_peered":"2024-04-25T10:06:11.597687+0000","last_unstale":"2024-04-25T10:06:12.590344+0000","last_undegraded":"2024-04-25T10:06:12.590344+0000","last_fullsized":"2024-04-25T10:06:12.590344+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.586794+0000","last_change":"2024-04-25T10:06:11.584953+0000","last_active":"2024-04-25T10:06:11.586794+0000","last_peered":"2024-04-25T10:06:11.586794+0000","last_clean":"2024-04-25T10:06:11.586794+0000","last_became_active":"2024-04-25T10:06:11.584666+0000","last_became_peered":"2024-04-25T10:06:11.584666+0000","last_unstale":"2024-04-25T10:06:11.586794+0000","last_undegraded":"2024-04-25T10:06:11.586794+0000","last_fullsized":"2024-04-25T10:06:11.586794+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:55.206873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591906+0000","last_change":"2024-04-25T10:06:11.592933+0000","last_active":"2024-04-25T10:06:12.591906+0000","last_peered":"2024-04-25T10:06:12.591906+0000","last_clean":"2024-04-25T10:06:12.591906+0000","last_became_active":"2024-04-25T10:06:11.592335+0000","last_became_peered":"2024-04-25T10:06:11.592335+0000","last_unstale":"2024-04-25T10:06:12.591906+0000","last_undegraded":"2024-04-25T10:06:12.591906+0000","last_fullsized":"2024-04-25T10:06:12.591906+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.403695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.602910+0000","last_change":"2024-04-25T10:06:10.590142+0000","last_active":"2024-04-25T10:06:11.602910+0000","last_peered":"2024-04-25T10:06:11.602910+0000","last_clean":"2024-04-25T10:06:11.602910+0000","last_became_active":"2024-04-25T10:06:10.589851+0000","last_became_peered":"2024-04-25T10:06:10.589851+0000","last_unstale":"2024-04-25T10:06:11.602910+0000","last_undegraded":"2024-04-25T10:06:11.602910+0000","last_fullsized":"2024-04-25T10:06:11.602910+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:50.142763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590901+0000","last_change":"2024-04-25T10:06:11.586712+0000","last_active":"2024-04-25T10:06:12.590901+0000","last_peered":"2024-04-25T10:06:12.590901+0000","last_clean":"2024-04-25T10:06:12.590901+0000","last_became_active":"2024-04-25T10:06:11.586393+0000","last_became_peered":"2024-04-25T10:06:11.586393+0000","last_unstale":"2024-04-25T10:06:12.590901+0000","last_undegraded":"2024-04-25T10:06:12.590901+0000","last_fullsized":"2024-04-25T10:06:12.590901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:11.365920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611691+0000","last_change":"2024-04-25T10:06:10.590906+0000","last_active":"2024-04-25T10:06:11.611691+0000","last_peered":"2024-04-25T10:06:11.611691+0000","last_clean":"2024-04-25T10:06:11.611691+0000","last_became_active":"2024-04-25T10:06:10.590641+0000","last_became_peered":"2024-04-25T10:06:10.590641+0000","last_unstale":"2024-04-25T10:06:11.611691+0000","last_undegraded":"2024-04-25T10:06:11.611691+0000","last_fullsized":"2024-04-25T10:06:11.611691+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:39.077377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591703+0000","last_change":"2024-04-25T10:06:11.601284+0000","last_active":"2024-04-25T10:06:12.591703+0000","last_peered":"2024-04-25T10:06:12.591703+0000","last_clean":"2024-04-25T10:06:12.591703+0000","last_became_active":"2024-04-25T10:06:11.600739+0000","last_became_peered":"2024-04-25T10:06:11.600739+0000","last_unstale":"2024-04-25T10:06:12.591703+0000","last_undegraded":"2024-04-25T10:06:12.591703+0000","last_fullsized":"2024-04-25T10:06:12.591703+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:03.030946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.594679+0000","last_change":"2024-04-25T10:06:10.589990+0000","last_active":"2024-04-25T10:06:11.594679+0000","last_peered":"2024-04-25T10:06:11.594679+0000","last_clean":"2024-04-25T10:06:11.594679+0000","last_became_active":"2024-04-25T10:06:10.589663+0000","last_became_peered":"2024-04-25T10:06:10.589663+0000","last_unstale":"2024-04-25T10:06:11.594679+0000","last_undegraded":"2024-04-25T10:06:11.594679+0000","last_fullsized":"2024-04-25T10:06:11.594679+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:17.017530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611361+0000","last_change":"2024-04-25T10:06:11.608503+0000","last_active":"2024-04-25T10:06:11.611361+0000","last_peered":"2024-04-25T10:06:11.611361+0000","last_clean":"2024-04-25T10:06:11.611361+0000","last_became_active":"2024-04-25T10:06:11.608234+0000","last_became_peered":"2024-04-25T10:06:11.608234+0000","last_unstale":"2024-04-25T10:06:11.611361+0000","last_undegraded":"2024-04-25T10:06:11.611361+0000","last_fullsized":"2024-04-25T10:06:11.611361+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:11.365920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.604687+0000","last_change":"2024-04-25T10:06:10.590974+0000","last_active":"2024-04-25T10:06:11.604687+0000","last_peered":"2024-04-25T10:06:11.604687+0000","last_clean":"2024-04-25T10:06:11.604687+0000","last_became_active":"2024-04-25T10:06:10.590635+0000","last_became_peered":"2024-04-25T10:06:10.590635+0000","last_unstale":"2024-04-25T10:06:11.604687+0000","last_undegraded":"2024-04-25T10:06:11.604687+0000","last_fullsized":"2024-04-25T10:06:11.604687+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:31.888158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591335+0000","last_change":"2024-04-25T10:06:11.594569+0000","last_active":"2024-04-25T10:06:12.591335+0000","last_peered":"2024-04-25T10:06:12.591335+0000","last_clean":"2024-04-25T10:06:12.591335+0000","last_became_active":"2024-04-25T10:06:11.594016+0000","last_became_peered":"2024-04-25T10:06:11.594016+0000","last_unstale":"2024-04-25T10:06:12.591335+0000","last_undegraded":"2024-04-25T10:06:12.591335+0000","last_fullsized":"2024-04-25T10:06:12.591335+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:38.022779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590761+0000","last_change":"2024-04-25T10:06:10.591527+0000","last_active":"2024-04-25T10:06:12.590761+0000","last_peered":"2024-04-25T10:06:12.590761+0000","last_clean":"2024-04-25T10:06:12.590761+0000","last_became_active":"2024-04-25T10:06:10.591212+0000","last_became_peered":"2024-04-25T10:06:10.591212+0000","last_unstale":"2024-04-25T10:06:12.590761+0000","last_undegraded":"2024-04-25T10:06:12.590761+0000","last_fullsized":"2024-04-25T10:06:12.590761+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:17.896865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590681+0000","last_change":"2024-04-25T10:06:11.597833+0000","last_active":"2024-04-25T10:06:12.590681+0000","last_peered":"2024-04-25T10:06:12.590681+0000","last_clean":"2024-04-25T10:06:12.590681+0000","last_became_active":"2024-04-25T10:06:11.597247+0000","last_became_peered":"2024-04-25T10:06:11.597247+0000","last_unstale":"2024-04-25T10:06:12.590681+0000","last_undegraded":"2024-04-25T10:06:12.590681+0000","last_fullsized":"2024-04-25T10:06:12.590681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:36.054034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590610+0000","last_change":"2024-04-25T10:06:10.593926+0000","last_active":"2024-04-25T10:06:12.590610+0000","last_peered":"2024-04-25T10:06:12.590610+0000","last_clean":"2024-04-25T10:06:12.590610+0000","last_became_active":"2024-04-25T10:06:10.593640+0000","last_became_peered":"2024-04-25T10:06:10.593640+0000","last_unstale":"2024-04-25T10:06:12.590610+0000","last_undegraded":"2024-04-25T10:06:12.590610+0000","last_fullsized":"2024-04-25T10:06:12.590610+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:32.052858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591186+0000","last_change":"2024-04-25T10:06:11.593063+0000","last_active":"2024-04-25T10:06:12.591186+0000","last_peered":"2024-04-25T10:06:12.591186+0000","last_clean":"2024-04-25T10:06:12.591186+0000","last_became_active":"2024-04-25T10:06:11.592772+0000","last_became_peered":"2024-04-25T10:06:11.592772+0000","last_unstale":"2024-04-25T10:06:12.591186+0000","last_undegraded":"2024-04-25T10:06:12.591186+0000","last_fullsized":"2024-04-25T10:06:12.591186+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:12.627498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.584205+0000","last_change":"2024-04-25T10:06:10.594103+0000","last_active":"2024-04-25T10:06:11.584205+0000","last_peered":"2024-04-25T10:06:11.584205+0000","last_clean":"2024-04-25T10:06:11.584205+0000","last_became_active":"2024-04-25T10:06:10.593805+0000","last_became_peered":"2024-04-25T10:06:10.593805+0000","last_unstale":"2024-04-25T10:06:11.584205+0000","last_undegraded":"2024-04-25T10:06:11.584205+0000","last_fullsized":"2024-04-25T10:06:11.584205+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:21.758145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.597476+0000","last_change":"2024-04-25T10:06:11.590223+0000","last_active":"2024-04-25T10:06:11.597476+0000","last_peered":"2024-04-25T10:06:11.597476+0000","last_clean":"2024-04-25T10:06:11.597476+0000","last_became_active":"2024-04-25T10:06:11.589970+0000","last_became_peered":"2024-04-25T10:06:11.589970+0000","last_unstale":"2024-04-25T10:06:11.597476+0000","last_undegraded":"2024-04-25T10:06:11.597476+0000","last_fullsized":"2024-04-25T10:06:11.597476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:44.396817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590364+0000","last_change":"2024-04-25T10:06:10.590741+0000","last_active":"2024-04-25T10:06:12.590364+0000","last_peered":"2024-04-25T10:06:12.590364+0000","last_clean":"2024-04-25T10:06:12.590364+0000","last_became_active":"2024-04-25T10:06:10.590499+0000","last_became_peered":"2024-04-25T10:06:10.590499+0000","last_unstale":"2024-04-25T10:06:12.590364+0000","last_undegraded":"2024-04-25T10:06:12.590364+0000","last_fullsized":"2024-04-25T10:06:12.590364+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:32.676160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599229+0000","last_change":"2024-04-25T10:06:11.594584+0000","last_active":"2024-04-25T10:06:11.599229+0000","last_peered":"2024-04-25T10:06:11.599229+0000","last_clean":"2024-04-25T10:06:11.599229+0000","last_became_active":"2024-04-25T10:06:11.594264+0000","last_became_peered":"2024-04-25T10:06:11.594264+0000","last_unstale":"2024-04-25T10:06:11.599229+0000","last_undegraded":"2024-04-25T10:06:11.599229+0000","last_fullsized":"2024-04-25T10:06:11.599229+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:33.855981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-25T10:06:09.571413+0000","last_change":"2024-04-25T10:06:09.571413+0000","last_active":"2024-04-25T10:06:09.563666+0000","last_peered":"2024-04-25T10:06:09.563666+0000","last_clean":"2024-04-25T10:06:09.563666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:09.571413+0000","last_undegraded":"2024-04-25T10:06:09.571413+0000","last_fullsized":"2024-04-25T10:06:09.571413+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.603137+0000","last_change":"2024-04-25T10:06:11.598368+0000","last_active":"2024-04-25T10:06:11.603137+0000","last_peered":"2024-04-25T10:06:11.603137+0000","last_clean":"2024-04-25T10:06:11.603137+0000","last_became_active":"2024-04-25T10:06:11.597963+0000","last_became_peered":"2024-04-25T10:06:11.597963+0000","last_unstale":"2024-04-25T10:06:11.603137+0000","last_undegraded":"2024-04-25T10:06:11.603137+0000","last_fullsized":"2024-04-25T10:06:11.603137+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:48.190466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-25T10:06:09.571772+0000","last_change":"2024-04-25T10:06:09.571772+0000","last_active":"2024-04-25T10:06:09.563666+0000","last_peered":"2024-04-25T10:06:09.563666+0000","last_clean":"2024-04-25T10:06:09.563666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:09.571772+0000","last_undegraded":"2024-04-25T10:06:09.571772+0000","last_fullsized":"2024-04-25T10:06:09.571772+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610787+0000","last_change":"2024-04-25T10:06:11.603945+0000","last_active":"2024-04-25T10:06:11.610787+0000","last_peered":"2024-04-25T10:06:11.610787+0000","last_clean":"2024-04-25T10:06:11.610787+0000","last_became_active":"2024-04-25T10:06:11.602849+0000","last_became_peered":"2024-04-25T10:06:11.602849+0000","last_unstale":"2024-04-25T10:06:11.610787+0000","last_undegraded":"2024-04-25T10:06:11.610787+0000","last_fullsized":"2024-04-25T10:06:11.610787+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:03.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.601893+0000","last_change":"2024-04-25T10:06:10.579763+0000","last_active":"2024-04-25T10:06:11.601893+0000","last_peered":"2024-04-25T10:06:11.601893+0000","last_clean":"2024-04-25T10:06:11.601893+0000","last_became_active":"2024-04-25T10:06:10.579495+0000","last_became_peered":"2024-04-25T10:06:10.579495+0000","last_unstale":"2024-04-25T10:06:11.601893+0000","last_undegraded":"2024-04-25T10:06:11.601893+0000","last_fullsized":"2024-04-25T10:06:11.601893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:08.212125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.603845+0000","last_change":"2024-04-25T10:06:11.603317+0000","last_active":"2024-04-25T10:06:11.603845+0000","last_peered":"2024-04-25T10:06:11.603845+0000","last_clean":"2024-04-25T10:06:11.603845+0000","last_became_active":"2024-04-25T10:06:11.603066+0000","last_became_peered":"2024-04-25T10:06:11.603066+0000","last_unstale":"2024-04-25T10:06:11.603845+0000","last_undegraded":"2024-04-25T10:06:11.603845+0000","last_fullsized":"2024-04-25T10:06:11.603845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:07.686632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592122+0000","last_change":"2024-04-25T10:06:10.593257+0000","last_active":"2024-04-25T10:06:12.592122+0000","last_peered":"2024-04-25T10:06:12.592122+0000","last_clean":"2024-04-25T10:06:12.592122+0000","last_became_active":"2024-04-25T10:06:10.592737+0000","last_became_peered":"2024-04-25T10:06:10.592737+0000","last_unstale":"2024-04-25T10:06:12.592122+0000","last_undegraded":"2024-04-25T10:06:12.592122+0000","last_fullsized":"2024-04-25T10:06:12.592122+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:38.738216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591639+0000","last_change":"2024-04-25T10:06:11.601746+0000","last_active":"2024-04-25T10:06:12.591639+0000","last_peered":"2024-04-25T10:06:12.591639+0000","last_clean":"2024-04-25T10:06:12.591639+0000","last_became_active":"2024-04-25T10:06:11.600706+0000","last_became_peered":"2024-04-25T10:06:11.600706+0000","last_unstale":"2024-04-25T10:06:12.591639+0000","last_undegraded":"2024-04-25T10:06:12.591639+0000","last_fullsized":"2024-04-25T10:06:12.591639+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:37.328665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590790+0000","last_change":"2024-04-25T10:06:10.589747+0000","last_active":"2024-04-25T10:06:12.590790+0000","last_peered":"2024-04-25T10:06:12.590790+0000","last_clean":"2024-04-25T10:06:12.590790+0000","last_became_active":"2024-04-25T10:06:10.589483+0000","last_became_peered":"2024-04-25T10:06:10.589483+0000","last_unstale":"2024-04-25T10:06:12.590790+0000","last_undegraded":"2024-04-25T10:06:12.590790+0000","last_fullsized":"2024-04-25T10:06:12.590790+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.133966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.605963+0000","last_change":"2024-04-25T10:06:11.599329+0000","last_active":"2024-04-25T10:06:11.605963+0000","last_peered":"2024-04-25T10:06:11.605963+0000","last_clean":"2024-04-25T10:06:11.605963+0000","last_became_active":"2024-04-25T10:06:11.599055+0000","last_became_peered":"2024-04-25T10:06:11.599055+0000","last_unstale":"2024-04-25T10:06:11.605963+0000","last_undegraded":"2024-04-25T10:06:11.605963+0000","last_fullsized":"2024-04-25T10:06:11.605963+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:04.221854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.590118+0000","last_change":"2024-04-25T10:06:10.592105+0000","last_active":"2024-04-25T10:06:11.590118+0000","last_peered":"2024-04-25T10:06:11.590118+0000","last_clean":"2024-04-25T10:06:11.590118+0000","last_became_active":"2024-04-25T10:06:10.591823+0000","last_became_peered":"2024-04-25T10:06:10.591823+0000","last_unstale":"2024-04-25T10:06:11.590118+0000","last_undegraded":"2024-04-25T10:06:11.590118+0000","last_fullsized":"2024-04-25T10:06:11.590118+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:53.425769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590360+0000","last_change":"2024-04-25T10:06:11.585750+0000","last_active":"2024-04-25T10:06:12.590360+0000","last_peered":"2024-04-25T10:06:12.590360+0000","last_clean":"2024-04-25T10:06:12.590360+0000","last_became_active":"2024-04-25T10:06:11.585193+0000","last_became_peered":"2024-04-25T10:06:11.585193+0000","last_unstale":"2024-04-25T10:06:12.590360+0000","last_undegraded":"2024-04-25T10:06:12.590360+0000","last_fullsized":"2024-04-25T10:06:12.590360+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:03.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.589817+0000","last_change":"2024-04-25T10:06:10.590311+0000","last_active":"2024-04-25T10:06:11.589817+0000","last_peered":"2024-04-25T10:06:11.589817+0000","last_clean":"2024-04-25T10:06:11.589817+0000","last_became_active":"2024-04-25T10:06:10.590015+0000","last_became_peered":"2024-04-25T10:06:10.590015+0000","last_unstale":"2024-04-25T10:06:11.589817+0000","last_undegraded":"2024-04-25T10:06:11.589817+0000","last_fullsized":"2024-04-25T10:06:11.589817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:27.524048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.585155+0000","last_change":"2024-04-25T10:06:11.582381+0000","last_active":"2024-04-25T10:06:11.585155+0000","last_peered":"2024-04-25T10:06:11.585155+0000","last_clean":"2024-04-25T10:06:11.585155+0000","last_became_active":"2024-04-25T10:06:11.582109+0000","last_became_peered":"2024-04-25T10:06:11.582109+0000","last_unstale":"2024-04-25T10:06:11.585155+0000","last_undegraded":"2024-04-25T10:06:11.585155+0000","last_fullsized":"2024-04-25T10:06:11.585155+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:41.604772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.604177+0000","last_change":"2024-04-25T10:06:10.589625+0000","last_active":"2024-04-25T10:06:11.604177+0000","last_peered":"2024-04-25T10:06:11.604177+0000","last_clean":"2024-04-25T10:06:11.604177+0000","last_became_active":"2024-04-25T10:06:10.589161+0000","last_became_peered":"2024-04-25T10:06:10.589161+0000","last_unstale":"2024-04-25T10:06:11.604177+0000","last_undegraded":"2024-04-25T10:06:11.604177+0000","last_fullsized":"2024-04-25T10:06:11.604177+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:47.673088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.612662+0000","last_change":"2024-04-25T10:06:11.612101+0000","last_active":"2024-04-25T10:06:11.612662+0000","last_peered":"2024-04-25T10:06:11.612662+0000","last_clean":"2024-04-25T10:06:11.612662+0000","last_became_active":"2024-04-25T10:06:11.611805+0000","last_became_peered":"2024-04-25T10:06:11.611805+0000","last_unstale":"2024-04-25T10:06:11.612662+0000","last_undegraded":"2024-04-25T10:06:11.612662+0000","last_fullsized":"2024-04-25T10:06:11.612662+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:22.287749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.598493+0000","last_change":"2024-04-25T10:06:10.594406+0000","last_active":"2024-04-25T10:06:11.598493+0000","last_peered":"2024-04-25T10:06:11.598493+0000","last_clean":"2024-04-25T10:06:11.598493+0000","last_became_active":"2024-04-25T10:06:10.594129+0000","last_became_peered":"2024-04-25T10:06:10.594129+0000","last_unstale":"2024-04-25T10:06:11.598493+0000","last_undegraded":"2024-04-25T10:06:11.598493+0000","last_fullsized":"2024-04-25T10:06:11.598493+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:19.498453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591710+0000","last_change":"2024-04-25T10:06:11.592494+0000","last_active":"2024-04-25T10:06:12.591710+0000","last_peered":"2024-04-25T10:06:12.591710+0000","last_clean":"2024-04-25T10:06:12.591710+0000","last_became_active":"2024-04-25T10:06:11.592212+0000","last_became_peered":"2024-04-25T10:06:11.592212+0000","last_unstale":"2024-04-25T10:06:12.591710+0000","last_undegraded":"2024-04-25T10:06:12.591710+0000","last_fullsized":"2024-04-25T10:06:12.591710+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:08.065111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590880+0000","last_change":"2024-04-25T10:06:10.588563+0000","last_active":"2024-04-25T10:06:12.590880+0000","last_peered":"2024-04-25T10:06:12.590880+0000","last_clean":"2024-04-25T10:06:12.590880+0000","last_became_active":"2024-04-25T10:06:10.588312+0000","last_became_peered":"2024-04-25T10:06:10.588312+0000","last_unstale":"2024-04-25T10:06:12.590880+0000","last_undegraded":"2024-04-25T10:06:12.590880+0000","last_fullsized":"2024-04-25T10:06:12.590880+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:17.315120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:10.566145+0000","last_change":"2024-04-25T10:06:10.566145+0000","last_active":"2024-04-25T10:06:10.566145+0000","last_peered":"2024-04-25T10:06:10.566145+0000","last_clean":"2024-04-25T10:06:10.566145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:10.566145+0000","last_undegraded":"2024-04-25T10:06:10.566145+0000","last_fullsized":"2024-04-25T10:06:10.566145+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.602393+0000","last_change":"2024-04-25T10:06:10.588662+0000","last_active":"2024-04-25T10:06:11.602393+0000","last_peered":"2024-04-25T10:06:11.602393+0000","last_clean":"2024-04-25T10:06:11.602393+0000","last_became_active":"2024-04-25T10:06:10.588316+0000","last_became_peered":"2024-04-25T10:06:10.588316+0000","last_unstale":"2024-04-25T10:06:11.602393+0000","last_undegraded":"2024-04-25T10:06:11.602393+0000","last_fullsized":"2024-04-25T10:06:11.602393+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:32.850838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599469+0000","last_change":"2024-04-25T10:06:11.595815+0000","last_active":"2024-04-25T10:06:11.599469+0000","last_peered":"2024-04-25T10:06:11.599469+0000","last_clean":"2024-04-25T10:06:11.599469+0000","last_became_active":"2024-04-25T10:06:11.595495+0000","last_became_peered":"2024-04-25T10:06:11.595495+0000","last_unstale":"2024-04-25T10:06:11.599469+0000","last_undegraded":"2024-04-25T10:06:11.599469+0000","last_fullsized":"2024-04-25T10:06:11.599469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:12.627498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.581094+0000","last_change":"2024-04-25T10:06:10.591774+0000","last_active":"2024-04-25T10:06:11.581094+0000","last_peered":"2024-04-25T10:06:11.581094+0000","last_clean":"2024-04-25T10:06:11.581094+0000","last_became_active":"2024-04-25T10:06:10.591412+0000","last_became_peered":"2024-04-25T10:06:10.591412+0000","last_unstale":"2024-04-25T10:06:11.581094+0000","last_undegraded":"2024-04-25T10:06:11.581094+0000","last_fullsized":"2024-04-25T10:06:11.581094+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:17.401215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591675+0000","last_change":"2024-04-25T10:06:11.593318+0000","last_active":"2024-04-25T10:06:12.591675+0000","last_peered":"2024-04-25T10:06:12.591675+0000","last_clean":"2024-04-25T10:06:12.591675+0000","last_became_active":"2024-04-25T10:06:11.593022+0000","last_became_peered":"2024-04-25T10:06:11.593022+0000","last_unstale":"2024-04-25T10:06:12.591675+0000","last_undegraded":"2024-04-25T10:06:12.591675+0000","last_fullsized":"2024-04-25T10:06:12.591675+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:46.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-25T10:06:09.571392+0000","last_change":"2024-04-25T10:06:09.571392+0000","last_active":"2024-04-25T10:06:09.563666+0000","last_peered":"2024-04-25T10:06:09.563666+0000","last_clean":"2024-04-25T10:06:09.563666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:09.571392+0000","last_undegraded":"2024-04-25T10:06:09.571392+0000","last_fullsized":"2024-04-25T10:06:09.571392+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:10.566145+0000","last_change":"2024-04-25T10:06:10.566145+0000","last_active":"2024-04-25T10:06:10.566145+0000","last_peered":"2024-04-25T10:06:10.566145+0000","last_clean":"2024-04-25T10:06:10.566145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:10.566145+0000","last_undegraded":"2024-04-25T10:06:10.566145+0000","last_fullsized":"2024-04-25T10:06:10.566145+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590609+0000","last_change":"2024-04-25T10:06:10.590140+0000","last_active":"2024-04-25T10:06:12.590609+0000","last_peered":"2024-04-25T10:06:12.590609+0000","last_clean":"2024-04-25T10:06:12.590609+0000","last_became_active":"2024-04-25T10:06:10.589015+0000","last_became_peered":"2024-04-25T10:06:10.589015+0000","last_unstale":"2024-04-25T10:06:12.590609+0000","last_undegraded":"2024-04-25T10:06:12.590609+0000","last_fullsized":"2024-04-25T10:06:12.590609+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:18.383218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591617+0000","last_change":"2024-04-25T10:06:11.604928+0000","last_active":"2024-04-25T10:06:12.591617+0000","last_peered":"2024-04-25T10:06:12.591617+0000","last_clean":"2024-04-25T10:06:12.591617+0000","last_became_active":"2024-04-25T10:06:11.604636+0000","last_became_peered":"2024-04-25T10:06:11.604636+0000","last_unstale":"2024-04-25T10:06:12.591617+0000","last_undegraded":"2024-04-25T10:06:12.591617+0000","last_fullsized":"2024-04-25T10:06:12.591617+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:04.221854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591565+0000","last_change":"2024-04-25T10:06:10.590105+0000","last_active":"2024-04-25T10:06:12.591565+0000","last_peered":"2024-04-25T10:06:12.591565+0000","last_clean":"2024-04-25T10:06:12.591565+0000","last_became_active":"2024-04-25T10:06:10.589829+0000","last_became_peered":"2024-04-25T10:06:10.589829+0000","last_unstale":"2024-04-25T10:06:12.591565+0000","last_undegraded":"2024-04-25T10:06:12.591565+0000","last_fullsized":"2024-04-25T10:06:12.591565+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:23.745526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591751+0000","last_change":"2024-04-25T10:06:11.593413+0000","last_active":"2024-04-25T10:06:12.591751+0000","last_peered":"2024-04-25T10:06:12.591751+0000","last_clean":"2024-04-25T10:06:12.591751+0000","last_became_active":"2024-04-25T10:06:11.593155+0000","last_became_peered":"2024-04-25T10:06:11.593155+0000","last_unstale":"2024-04-25T10:06:12.591751+0000","last_undegraded":"2024-04-25T10:06:12.591751+0000","last_fullsized":"2024-04-25T10:06:12.591751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:33.678639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.579429+0000","last_change":"2024-04-25T10:06:10.587062+0000","last_active":"2024-04-25T10:06:11.579429+0000","last_peered":"2024-04-25T10:06:11.579429+0000","last_clean":"2024-04-25T10:06:11.579429+0000","last_became_active":"2024-04-25T10:06:10.586736+0000","last_became_peered":"2024-04-25T10:06:10.586736+0000","last_unstale":"2024-04-25T10:06:11.579429+0000","last_undegraded":"2024-04-25T10:06:11.579429+0000","last_fullsized":"2024-04-25T10:06:11.579429+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:48.572483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599597+0000","last_change":"2024-04-25T10:06:11.595699+0000","last_active":"2024-04-25T10:06:11.599597+0000","last_peered":"2024-04-25T10:06:11.599597+0000","last_clean":"2024-04-25T10:06:11.599597+0000","last_became_active":"2024-04-25T10:06:11.595437+0000","last_became_peered":"2024-04-25T10:06:11.595437+0000","last_unstale":"2024-04-25T10:06:11.599597+0000","last_undegraded":"2024-04-25T10:06:11.599597+0000","last_fullsized":"2024-04-25T10:06:11.599597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:38.022779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591835+0000","last_change":"2024-04-25T10:06:10.588344+0000","last_active":"2024-04-25T10:06:12.591835+0000","last_peered":"2024-04-25T10:06:12.591835+0000","last_clean":"2024-04-25T10:06:12.591835+0000","last_became_active":"2024-04-25T10:06:10.587774+0000","last_became_peered":"2024-04-25T10:06:10.587774+0000","last_unstale":"2024-04-25T10:06:12.591835+0000","last_undegraded":"2024-04-25T10:06:12.591835+0000","last_fullsized":"2024-04-25T10:06:12.591835+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:13.625296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.603613+0000","last_change":"2024-04-25T10:06:11.603042+0000","last_active":"2024-04-25T10:06:11.603613+0000","last_peered":"2024-04-25T10:06:11.603613+0000","last_clean":"2024-04-25T10:06:11.603613+0000","last_became_active":"2024-04-25T10:06:11.598535+0000","last_became_peered":"2024-04-25T10:06:11.598535+0000","last_unstale":"2024-04-25T10:06:11.603613+0000","last_undegraded":"2024-04-25T10:06:11.603613+0000","last_fullsized":"2024-04-25T10:06:11.603613+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:31.070811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-25T10:06:09.571309+0000","last_change":"2024-04-25T10:06:09.571309+0000","last_active":"2024-04-25T10:06:09.563666+0000","last_peered":"2024-04-25T10:06:09.563666+0000","last_clean":"2024-04-25T10:06:09.563666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:09.571309+0000","last_undegraded":"2024-04-25T10:06:09.571309+0000","last_fullsized":"2024-04-25T10:06:09.571309+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.605940+0000","last_change":"2024-04-25T10:06:11.603834+0000","last_active":"2024-04-25T10:06:11.605940+0000","last_peered":"2024-04-25T10:06:11.605940+0000","last_clean":"2024-04-25T10:06:11.605940+0000","last_became_active":"2024-04-25T10:06:11.603541+0000","last_became_peered":"2024-04-25T10:06:11.603541+0000","last_unstale":"2024-04-25T10:06:11.605940+0000","last_undegraded":"2024-04-25T10:06:11.605940+0000","last_fullsized":"2024-04-25T10:06:11.605940+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:08.065111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592173+0000","last_change":"2024-04-25T10:06:10.590856+0000","last_active":"2024-04-25T10:06:12.592173+0000","last_peered":"2024-04-25T10:06:12.592173+0000","last_clean":"2024-04-25T10:06:12.592173+0000","last_became_active":"2024-04-25T10:06:10.590571+0000","last_became_peered":"2024-04-25T10:06:10.590571+0000","last_unstale":"2024-04-25T10:06:12.592173+0000","last_undegraded":"2024-04-25T10:06:12.592173+0000","last_fullsized":"2024-04-25T10:06:12.592173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:01.814874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.596509+0000","last_change":"2024-04-25T10:06:11.595775+0000","last_active":"2024-04-25T10:06:11.596509+0000","last_peered":"2024-04-25T10:06:11.596509+0000","last_clean":"2024-04-25T10:06:11.596509+0000","last_became_active":"2024-04-25T10:06:11.595387+0000","last_became_peered":"2024-04-25T10:06:11.595387+0000","last_unstale":"2024-04-25T10:06:11.596509+0000","last_undegraded":"2024-04-25T10:06:11.596509+0000","last_fullsized":"2024-04-25T10:06:11.596509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:40.079857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592115+0000","last_change":"2024-04-25T10:06:10.588101+0000","last_active":"2024-04-25T10:06:12.592115+0000","last_peered":"2024-04-25T10:06:12.592115+0000","last_clean":"2024-04-25T10:06:12.592115+0000","last_became_active":"2024-04-25T10:06:10.587808+0000","last_became_peered":"2024-04-25T10:06:10.587808+0000","last_unstale":"2024-04-25T10:06:12.592115+0000","last_undegraded":"2024-04-25T10:06:12.592115+0000","last_fullsized":"2024-04-25T10:06:12.592115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.133966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610579+0000","last_change":"2024-04-25T10:06:11.599755+0000","last_active":"2024-04-25T10:06:11.610579+0000","last_peered":"2024-04-25T10:06:11.610579+0000","last_clean":"2024-04-25T10:06:11.610579+0000","last_became_active":"2024-04-25T10:06:11.599317+0000","last_became_peered":"2024-04-25T10:06:11.599317+0000","last_unstale":"2024-04-25T10:06:11.610579+0000","last_undegraded":"2024-04-25T10:06:11.610579+0000","last_fullsized":"2024-04-25T10:06:11.610579+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:37.328665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591917+0000","last_change":"2024-04-25T10:06:10.591156+0000","last_active":"2024-04-25T10:06:12.591917+0000","last_peered":"2024-04-25T10:06:12.591917+0000","last_clean":"2024-04-25T10:06:12.591917+0000","last_became_active":"2024-04-25T10:06:10.590362+0000","last_became_peered":"2024-04-25T10:06:10.590362+0000","last_unstale":"2024-04-25T10:06:12.591917+0000","last_undegraded":"2024-04-25T10:06:12.591917+0000","last_fullsized":"2024-04-25T10:06:12.591917+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:32.052858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610758+0000","last_change":"2024-04-25T10:06:11.603491+0000","last_active":"2024-04-25T10:06:11.610758+0000","last_peered":"2024-04-25T10:06:11.610758+0000","last_clean":"2024-04-25T10:06:11.610758+0000","last_became_active":"2024-04-25T10:06:11.600221+0000","last_became_peered":"2024-04-25T10:06:11.600221+0000","last_unstale":"2024-04-25T10:06:11.610758+0000","last_undegraded":"2024-04-25T10:06:11.610758+0000","last_fullsized":"2024-04-25T10:06:11.610758+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590413+0000","last_change":"2024-04-25T10:06:10.589304+0000","last_active":"2024-04-25T10:06:12.590413+0000","last_peered":"2024-04-25T10:06:12.590413+0000","last_clean":"2024-04-25T10:06:12.590413+0000","last_became_active":"2024-04-25T10:06:10.588934+0000","last_became_peered":"2024-04-25T10:06:10.588934+0000","last_unstale":"2024-04-25T10:06:12.590413+0000","last_undegraded":"2024-04-25T10:06:12.590413+0000","last_fullsized":"2024-04-25T10:06:12.590413+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:07.062632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.602797+0000","last_change":"2024-04-25T10:06:11.597255+0000","last_active":"2024-04-25T10:06:11.602797+0000","last_peered":"2024-04-25T10:06:11.602797+0000","last_clean":"2024-04-25T10:06:11.602797+0000","last_became_active":"2024-04-25T10:06:11.596954+0000","last_became_peered":"2024-04-25T10:06:11.596954+0000","last_unstale":"2024-04-25T10:06:11.602797+0000","last_undegraded":"2024-04-25T10:06:11.602797+0000","last_fullsized":"2024-04-25T10:06:11.602797+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:13.167235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.596522+0000","last_change":"2024-04-25T10:06:10.589615+0000","last_active":"2024-04-25T10:06:11.596522+0000","last_peered":"2024-04-25T10:06:11.596522+0000","last_clean":"2024-04-25T10:06:11.596522+0000","last_became_active":"2024-04-25T10:06:10.589310+0000","last_became_peered":"2024-04-25T10:06:10.589310+0000","last_unstale":"2024-04-25T10:06:11.596522+0000","last_undegraded":"2024-04-25T10:06:11.596522+0000","last_fullsized":"2024-04-25T10:06:11.596522+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:00.780604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.597484+0000","last_change":"2024-04-25T10:06:11.596201+0000","last_active":"2024-04-25T10:06:11.597484+0000","last_peered":"2024-04-25T10:06:11.597484+0000","last_clean":"2024-04-25T10:06:11.597484+0000","last_became_active":"2024-04-25T10:06:11.595880+0000","last_became_peered":"2024-04-25T10:06:11.595880+0000","last_unstale":"2024-04-25T10:06:11.597484+0000","last_undegraded":"2024-04-25T10:06:11.597484+0000","last_fullsized":"2024-04-25T10:06:11.597484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:36.054034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.580878+0000","last_change":"2024-04-25T10:06:10.587821+0000","last_active":"2024-04-25T10:06:11.580878+0000","last_peered":"2024-04-25T10:06:11.580878+0000","last_clean":"2024-04-25T10:06:11.580878+0000","last_became_active":"2024-04-25T10:06:10.587520+0000","last_became_peered":"2024-04-25T10:06:10.587520+0000","last_unstale":"2024-04-25T10:06:11.580878+0000","last_undegraded":"2024-04-25T10:06:11.580878+0000","last_fullsized":"2024-04-25T10:06:11.580878+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:17.031658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":198,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591162+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:06:12.591162+0000","last_peered":"2024-04-25T10:06:12.591162+0000","last_clean":"2024-04-25T10:06:12.591162+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:06:12.591162+0000","last_undegraded":"2024-04-25T10:06:12.591162+0000","last_fullsized":"2024-04-25T10:06:12.591162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":118,"acting":118,"num_store_stats":7},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":150789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":152379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":152401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":405504,"data_stored":151543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":393216,"data_stored":150801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":747505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":393216,"data_stored":150857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":946176,"data_stored":722268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:14.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:15.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.067+0000 7fb4a7ab3640 1 Processor -- start 2024-04-25T10:06:15.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.068+0000 7fb4a7ab3640 1 -- start start 2024-04-25T10:06:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.068+0000 7fb4a7ab3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4a015dce0 0x7fb4a00d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.068+0000 7fb4a7ab3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4a00d35b0 0x7fb4a00b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.068+0000 7fb4a7ab3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb4a00d1f90 0x7fb4a00b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.068+0000 7fb4a7ab3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a015e0c0 con 0x7fb4a015dce0 2024-04-25T10:06:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.068+0000 7fb4a7ab3640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a00d2510 con 0x7fb4a00d1f90 2024-04-25T10:06:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.068+0000 7fb4a7ab3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00d27f0 con 0x7fb4a00d35b0 2024-04-25T10:06:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.068+0000 7fb4a584f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4a015dce0 0x7fb4a00d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.068+0000 7fb4a584f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4a015dce0 0x7fb4a00d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49510/0 (socket says 172.21.15.16:49510) 2024-04-25T10:06:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.068+0000 7fb4a584f640 1 -- 172.21.15.16:0/1133633102 learned_addr learned my addr 172.21.15.16:0/1133633102 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.069+0000 7fb4a6050640 1 --2- 172.21.15.16:0/1133633102 >> v2:172.21.15.186:3300/0 conn(0x7fb4a00d1f90 0x7fb4a00b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.069+0000 7fb4a504e640 1 --2- 172.21.15.16:0/1133633102 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4a00d35b0 0x7fb4a00b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.069+0000 7fb4a584f640 1 -- 172.21.15.16:0/1133633102 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4a00d35b0 msgr2=0x7fb4a00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.069+0000 7fb4a584f640 1 --2- 172.21.15.16:0/1133633102 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4a00d35b0 0x7fb4a00b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.069+0000 7fb4a584f640 1 -- 172.21.15.16:0/1133633102 >> v2:172.21.15.186:3300/0 conn(0x7fb4a00d1f90 msgr2=0x7fb4a00b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.069+0000 7fb4a584f640 1 --2- 172.21.15.16:0/1133633102 >> v2:172.21.15.186:3300/0 conn(0x7fb4a00d1f90 0x7fb4a00b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.069+0000 7fb4a584f640 1 -- 172.21.15.16:0/1133633102 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a00d2ad0 con 0x7fb4a015dce0 2024-04-25T10:06:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.069+0000 7fb4a504e640 1 --2- 172.21.15.16:0/1133633102 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4a00d35b0 0x7fb4a00b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.069+0000 7fb4a584f640 1 --2- 172.21.15.16:0/1133633102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a015dce0 0x7fb4a00d1a50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0161710 tx=0x7fb49802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c038393c559d7b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.070+0000 7fb4a484d640 1 -- 172.21.15.16:0/1133633102 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb498037070 con 0x7fb4a015dce0 2024-04-25T10:06:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.070+0000 7fb4a484d640 1 -- 172.21.15.16:0/1133633102 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb498032440 con 0x7fb4a015dce0 2024-04-25T10:06:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.070+0000 7fb4a484d640 1 -- 172.21.15.16:0/1133633102 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb498032810 con 0x7fb4a015dce0 2024-04-25T10:06:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.070+0000 7fb4a7ab3640 1 -- 172.21.15.16:0/1133633102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a015dce0 msgr2=0x7fb4a00d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.070+0000 7fb4a7ab3640 1 --2- 172.21.15.16:0/1133633102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a015dce0 0x7fb4a00d1a50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0161710 tx=0x7fb49802a140 comp rx=0 tx=0).stop 2024-04-25T10:06:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.070+0000 7fb4a7ab3640 1 -- 172.21.15.16:0/1133633102 shutdown_connections 2024-04-25T10:06:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.070+0000 7fb4a7ab3640 1 --2- 172.21.15.16:0/1133633102 >> v2:172.21.15.186:3300/0 conn(0x7fb4a00d1f90 0x7fb4a00b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.070+0000 7fb4a7ab3640 1 --2- 172.21.15.16:0/1133633102 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4a00d35b0 0x7fb4a00b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.070+0000 7fb4a7ab3640 1 --2- 172.21.15.16:0/1133633102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a015dce0 0x7fb4a00d1a50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.070+0000 7fb4a7ab3640 1 -- 172.21.15.16:0/1133633102 >> 172.21.15.16:0/1133633102 conn(0x7fb4a0072900 msgr2=0x7fb4a00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.070+0000 7fb4a7ab3640 1 -- 172.21.15.16:0/1133633102 shutdown_connections 2024-04-25T10:06:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.070+0000 7fb4a7ab3640 1 -- 172.21.15.16:0/1133633102 wait complete. 2024-04-25T10:06:15.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.071+0000 7fb4a7ab3640 1 Processor -- start 2024-04-25T10:06:15.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.072+0000 7fb4a7ab3640 1 -- start start 2024-04-25T10:06:15.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.072+0000 7fb4a7ab3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a00d1f90 0x7fb4a0160be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.072+0000 7fb4a7ab3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a00d35b0 0x7fb4a00c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.072+0000 7fb4a7ab3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4a015dce0 0x7fb4a00caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.072+0000 7fb4a7ab3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00b9820 con 0x7fb4a00d1f90 2024-04-25T10:06:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.072+0000 7fb4a7ab3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00d3090 con 0x7fb4a00d35b0 2024-04-25T10:06:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.072+0000 7fb4a7ab3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00b99a0 con 0x7fb4a015dce0 2024-04-25T10:06:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.072+0000 7fb4a584f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a00d1f90 0x7fb4a0160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.072+0000 7fb4a584f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a00d1f90 0x7fb4a0160be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49518/0 (socket says 172.21.15.16:49518) 2024-04-25T10:06:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.072+0000 7fb4a584f640 1 -- 172.21.15.16:0/1571124260 learned_addr learned my addr 172.21.15.16:0/1571124260 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.072+0000 7fb4a504e640 1 --2- 172.21.15.16:0/1571124260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a00d35b0 0x7fb4a00c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.072+0000 7fb4a6050640 1 --2- 172.21.15.16:0/1571124260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4a015dce0 0x7fb4a00caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.073+0000 7fb4a584f640 1 -- 172.21.15.16:0/1571124260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4a015dce0 msgr2=0x7fb4a00caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.073+0000 7fb4a584f640 1 --2- 172.21.15.16:0/1571124260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4a015dce0 0x7fb4a00caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.073+0000 7fb4a584f640 1 -- 172.21.15.16:0/1571124260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a00d35b0 msgr2=0x7fb4a00c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.073+0000 7fb4a584f640 1 --2- 172.21.15.16:0/1571124260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a00d35b0 0x7fb4a00c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.073+0000 7fb4a584f640 1 -- 172.21.15.16:0/1571124260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a015fb40 con 0x7fb4a00d1f90 2024-04-25T10:06:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.073+0000 7fb4a584f640 1 --2- 172.21.15.16:0/1571124260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a00d1f90 0x7fb4a0160be0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb498000c00 tx=0x7fb4980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.073+0000 7fb49e7fc640 1 -- 172.21.15.16:0/1571124260 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb498041020 con 0x7fb4a00d1f90 2024-04-25T10:06:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.073+0000 7fb49e7fc640 1 -- 172.21.15.16:0/1571124260 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4980306c0 con 0x7fb4a00d1f90 2024-04-25T10:06:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.073+0000 7fb49e7fc640 1 -- 172.21.15.16:0/1571124260 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb498037040 con 0x7fb4a00d1f90 2024-04-25T10:06:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.073+0000 7fb4a7ab3640 1 -- 172.21.15.16:0/1571124260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a015fe20 con 0x7fb4a00d1f90 2024-04-25T10:06:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.073+0000 7fb4a7ab3640 1 -- 172.21.15.16:0/1571124260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a01603e0 con 0x7fb4a00d1f90 2024-04-25T10:06:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.074+0000 7fb4a7ab3640 1 -- 172.21.15.16:0/1571124260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a00749b0 con 0x7fb4a00d1f90 2024-04-25T10:06:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.075+0000 7fb49e7fc640 1 -- 172.21.15.16:0/1571124260 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb498044020 con 0x7fb4a00d1f90 2024-04-25T10:06:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.075+0000 7fb49e7fc640 1 --2- 172.21.15.16:0/1571124260 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb48c06b4d0 0x7fb48c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.075+0000 7fb49e7fc640 1 -- 172.21.15.16:0/1571124260 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4980aa8e0 con 0x7fb4a00d1f90 2024-04-25T10:06:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.075+0000 7fb4a504e640 1 --2- 172.21.15.16:0/1571124260 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb48c06b4d0 0x7fb48c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.076+0000 7fb4a504e640 1 --2- 172.21.15.16:0/1571124260 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb48c06b4d0 0x7fb48c06d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00b6ba0 tx=0x7fb494008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.079+0000 7fb49e7fc640 1 -- 172.21.15.16:0/1571124260 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb49804b030 con 0x7fb4a00d1f90 2024-04-25T10:06:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.257+0000 7fb4a7ab3640 1 -- 172.21.15.16:0/1571124260 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4a00ce000 con 0x7fb48c06b4d0 2024-04-25T10:06:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.264+0000 7fb49e7fc640 1 -- 172.21.15.16:0/1571124260 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250788 (secure 0 0 0) 0x7fb4a00ce000 con 0x7fb48c06b4d0 2024-04-25T10:06:15.272 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:06:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.267+0000 7fb4a7ab3640 1 -- 172.21.15.16:0/1571124260 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb48c06b4d0 msgr2=0x7fb48c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.267+0000 7fb4a7ab3640 1 --2- 172.21.15.16:0/1571124260 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb48c06b4d0 0x7fb48c06d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00b6ba0 tx=0x7fb494008040 comp rx=0 tx=0).stop 2024-04-25T10:06:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.267+0000 7fb4a7ab3640 1 -- 172.21.15.16:0/1571124260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a00d1f90 msgr2=0x7fb4a0160be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.267+0000 7fb4a7ab3640 1 --2- 172.21.15.16:0/1571124260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a00d1f90 0x7fb4a0160be0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb498000c00 tx=0x7fb4980046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.267+0000 7fb4a7ab3640 1 -- 172.21.15.16:0/1571124260 shutdown_connections 2024-04-25T10:06:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.267+0000 7fb4a7ab3640 1 --2- 172.21.15.16:0/1571124260 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb48c06b4d0 0x7fb48c06d990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.267+0000 7fb4a7ab3640 1 --2- 172.21.15.16:0/1571124260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4a015dce0 0x7fb4a00caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.267+0000 7fb4a7ab3640 1 --2- 172.21.15.16:0/1571124260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a00d35b0 0x7fb4a00c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.267+0000 7fb4a7ab3640 1 --2- 172.21.15.16:0/1571124260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a00d1f90 0x7fb4a0160be0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.267+0000 7fb4a7ab3640 1 -- 172.21.15.16:0/1571124260 >> 172.21.15.16:0/1571124260 conn(0x7fb4a0072900 msgr2=0x7fb4a00b8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.267+0000 7fb4a7ab3640 1 -- 172.21.15.16:0/1571124260 shutdown_connections 2024-04-25T10:06:15.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.267+0000 7fb4a7ab3640 1 -- 172.21.15.16:0/1571124260 wait complete. 2024-04-25T10:06:15.283 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-04-25T10:06:13.795375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":184,"acting":184,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237044,"kb_used_data":4236,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737676,"statfs":{"total":773094113280,"available":772851380224,"internally_reserved":0,"allocated":4337664,"data_stored":2378543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.228282"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590900+0000","last_change":"2024-04-25T10:06:11.596197+0000","last_active":"2024-04-25T10:06:12.590900+0000","last_peered":"2024-04-25T10:06:12.590900+0000","last_clean":"2024-04-25T10:06:12.590900+0000","last_became_active":"2024-04-25T10:06:11.595896+0000","last_became_peered":"2024-04-25T10:06:11.595896+0000","last_unstale":"2024-04-25T10:06:12.590900+0000","last_undegraded":"2024-04-25T10:06:12.590900+0000","last_fullsized":"2024-04-25T10:06:12.590900+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:50.014644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:10.566145+0000","last_change":"2024-04-25T10:06:10.566145+0000","last_active":"2024-04-25T10:06:10.566145+0000","last_peered":"2024-04-25T10:06:10.566145+0000","last_clean":"2024-04-25T10:06:10.566145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:10.566145+0000","last_undegraded":"2024-04-25T10:06:10.566145+0000","last_fullsized":"2024-04-25T10:06:10.566145+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591774+0000","last_change":"2024-04-25T10:06:11.588804+0000","last_active":"2024-04-25T10:06:12.591774+0000","last_peered":"2024-04-25T10:06:12.591774+0000","last_clean":"2024-04-25T10:06:12.591774+0000","last_became_active":"2024-04-25T10:06:11.588533+0000","last_became_peered":"2024-04-25T10:06:11.588533+0000","last_unstale":"2024-04-25T10:06:12.591774+0000","last_undegraded":"2024-04-25T10:06:12.591774+0000","last_fullsized":"2024-04-25T10:06:12.591774+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:12.136446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610805+0000","last_change":"2024-04-25T10:06:11.603412+0000","last_active":"2024-04-25T10:06:11.610805+0000","last_peered":"2024-04-25T10:06:11.610805+0000","last_clean":"2024-04-25T10:06:11.610805+0000","last_became_active":"2024-04-25T10:06:11.602936+0000","last_became_peered":"2024-04-25T10:06:11.602936+0000","last_unstale":"2024-04-25T10:06:11.610805+0000","last_undegraded":"2024-04-25T10:06:11.610805+0000","last_fullsized":"2024-04-25T10:06:11.610805+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.403695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:10.566145+0000","last_change":"2024-04-25T10:06:10.566145+0000","last_active":"2024-04-25T10:06:10.566145+0000","last_peered":"2024-04-25T10:06:10.566145+0000","last_clean":"2024-04-25T10:06:10.566145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:10.566145+0000","last_undegraded":"2024-04-25T10:06:10.566145+0000","last_fullsized":"2024-04-25T10:06:10.566145+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599412+0000","last_change":"2024-04-25T10:06:11.594811+0000","last_active":"2024-04-25T10:06:11.599412+0000","last_peered":"2024-04-25T10:06:11.599412+0000","last_clean":"2024-04-25T10:06:11.599412+0000","last_became_active":"2024-04-25T10:06:11.594494+0000","last_became_peered":"2024-04-25T10:06:11.594494+0000","last_unstale":"2024-04-25T10:06:11.599412+0000","last_undegraded":"2024-04-25T10:06:11.599412+0000","last_fullsized":"2024-04-25T10:06:11.599412+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:02.231151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590302+0000","last_change":"2024-04-25T10:06:11.585486+0000","last_active":"2024-04-25T10:06:12.590302+0000","last_peered":"2024-04-25T10:06:12.590302+0000","last_clean":"2024-04-25T10:06:12.590302+0000","last_became_active":"2024-04-25T10:06:11.585167+0000","last_became_peered":"2024-04-25T10:06:11.585167+0000","last_unstale":"2024-04-25T10:06:12.590302+0000","last_undegraded":"2024-04-25T10:06:12.590302+0000","last_fullsized":"2024-04-25T10:06:12.590302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590232+0000","last_change":"2024-04-25T10:06:11.599766+0000","last_active":"2024-04-25T10:06:12.590232+0000","last_peered":"2024-04-25T10:06:12.590232+0000","last_clean":"2024-04-25T10:06:12.590232+0000","last_became_active":"2024-04-25T10:06:11.599473+0000","last_became_peered":"2024-04-25T10:06:11.599473+0000","last_unstale":"2024-04-25T10:06:12.590232+0000","last_undegraded":"2024-04-25T10:06:12.590232+0000","last_fullsized":"2024-04-25T10:06:12.590232+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:46.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591773+0000","last_change":"2024-04-25T10:06:11.601172+0000","last_active":"2024-04-25T10:06:12.591773+0000","last_peered":"2024-04-25T10:06:12.591773+0000","last_clean":"2024-04-25T10:06:12.591773+0000","last_became_active":"2024-04-25T10:06:11.600890+0000","last_became_peered":"2024-04-25T10:06:11.600890+0000","last_unstale":"2024-04-25T10:06:12.591773+0000","last_undegraded":"2024-04-25T10:06:12.591773+0000","last_fullsized":"2024-04-25T10:06:12.591773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:33.055338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.614185+0000","last_change":"2024-04-25T10:06:11.611336+0000","last_active":"2024-04-25T10:06:11.614185+0000","last_peered":"2024-04-25T10:06:11.614185+0000","last_clean":"2024-04-25T10:06:11.614185+0000","last_became_active":"2024-04-25T10:06:11.610883+0000","last_became_peered":"2024-04-25T10:06:11.610883+0000","last_unstale":"2024-04-25T10:06:11.614185+0000","last_undegraded":"2024-04-25T10:06:11.614185+0000","last_fullsized":"2024-04-25T10:06:11.614185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:50.014644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.606907+0000","last_change":"2024-04-25T10:06:11.606133+0000","last_active":"2024-04-25T10:06:11.606907+0000","last_peered":"2024-04-25T10:06:11.606907+0000","last_clean":"2024-04-25T10:06:11.606907+0000","last_became_active":"2024-04-25T10:06:11.605855+0000","last_became_peered":"2024-04-25T10:06:11.605855+0000","last_unstale":"2024-04-25T10:06:11.606907+0000","last_undegraded":"2024-04-25T10:06:11.606907+0000","last_fullsized":"2024-04-25T10:06:11.606907+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:35.491727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591678+0000","last_change":"2024-04-25T10:06:11.598791+0000","last_active":"2024-04-25T10:06:12.591678+0000","last_peered":"2024-04-25T10:06:12.591678+0000","last_clean":"2024-04-25T10:06:12.591678+0000","last_became_active":"2024-04-25T10:06:11.598511+0000","last_became_peered":"2024-04-25T10:06:11.598511+0000","last_unstale":"2024-04-25T10:06:12.591678+0000","last_undegraded":"2024-04-25T10:06:12.591678+0000","last_fullsized":"2024-04-25T10:06:12.591678+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:18.899344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592055+0000","last_change":"2024-04-25T10:06:11.602586+0000","last_active":"2024-04-25T10:06:12.592055+0000","last_peered":"2024-04-25T10:06:12.592055+0000","last_clean":"2024-04-25T10:06:12.592055+0000","last_became_active":"2024-04-25T10:06:11.602219+0000","last_became_peered":"2024-04-25T10:06:11.602219+0000","last_unstale":"2024-04-25T10:06:12.592055+0000","last_undegraded":"2024-04-25T10:06:12.592055+0000","last_fullsized":"2024-04-25T10:06:12.592055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610597+0000","last_change":"2024-04-25T10:06:11.599901+0000","last_active":"2024-04-25T10:06:11.610597+0000","last_peered":"2024-04-25T10:06:11.610597+0000","last_clean":"2024-04-25T10:06:11.610597+0000","last_became_active":"2024-04-25T10:06:11.599632+0000","last_became_peered":"2024-04-25T10:06:11.599632+0000","last_unstale":"2024-04-25T10:06:11.610597+0000","last_undegraded":"2024-04-25T10:06:11.610597+0000","last_fullsized":"2024-04-25T10:06:11.610597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:46.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.592021+0000","last_change":"2024-04-25T10:06:11.587570+0000","last_active":"2024-04-25T10:06:11.592021+0000","last_peered":"2024-04-25T10:06:11.592021+0000","last_clean":"2024-04-25T10:06:11.592021+0000","last_became_active":"2024-04-25T10:06:11.587301+0000","last_became_peered":"2024-04-25T10:06:11.587301+0000","last_unstale":"2024-04-25T10:06:11.592021+0000","last_undegraded":"2024-04-25T10:06:11.592021+0000","last_fullsized":"2024-04-25T10:06:11.592021+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.597882+0000","last_change":"2024-04-25T10:06:11.597081+0000","last_active":"2024-04-25T10:06:11.597882+0000","last_peered":"2024-04-25T10:06:11.597882+0000","last_clean":"2024-04-25T10:06:11.597882+0000","last_became_active":"2024-04-25T10:06:11.596781+0000","last_became_peered":"2024-04-25T10:06:11.596781+0000","last_unstale":"2024-04-25T10:06:11.597882+0000","last_undegraded":"2024-04-25T10:06:11.597882+0000","last_fullsized":"2024-04-25T10:06:11.597882+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:11.365920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.589022+0000","last_change":"2024-04-25T10:06:11.587969+0000","last_active":"2024-04-25T10:06:11.589022+0000","last_peered":"2024-04-25T10:06:11.589022+0000","last_clean":"2024-04-25T10:06:11.589022+0000","last_became_active":"2024-04-25T10:06:11.587084+0000","last_became_peered":"2024-04-25T10:06:11.587084+0000","last_unstale":"2024-04-25T10:06:11.589022+0000","last_undegraded":"2024-04-25T10:06:11.589022+0000","last_fullsized":"2024-04-25T10:06:11.589022+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:47.208707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599620+0000","last_change":"2024-04-25T10:06:11.596669+0000","last_active":"2024-04-25T10:06:11.599620+0000","last_peered":"2024-04-25T10:06:11.599620+0000","last_clean":"2024-04-25T10:06:11.599620+0000","last_became_active":"2024-04-25T10:06:11.596091+0000","last_became_peered":"2024-04-25T10:06:11.596091+0000","last_unstale":"2024-04-25T10:06:11.599620+0000","last_undegraded":"2024-04-25T10:06:11.599620+0000","last_fullsized":"2024-04-25T10:06:11.599620+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:18.899344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599443+0000","last_change":"2024-04-25T10:06:11.594701+0000","last_active":"2024-04-25T10:06:11.599443+0000","last_peered":"2024-04-25T10:06:11.599443+0000","last_clean":"2024-04-25T10:06:11.599443+0000","last_became_active":"2024-04-25T10:06:11.594376+0000","last_became_peered":"2024-04-25T10:06:11.594376+0000","last_unstale":"2024-04-25T10:06:11.599443+0000","last_undegraded":"2024-04-25T10:06:11.599443+0000","last_fullsized":"2024-04-25T10:06:11.599443+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:53.157588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592096+0000","last_change":"2024-04-25T10:06:11.597575+0000","last_active":"2024-04-25T10:06:12.592096+0000","last_peered":"2024-04-25T10:06:12.592096+0000","last_clean":"2024-04-25T10:06:12.592096+0000","last_became_active":"2024-04-25T10:06:11.596710+0000","last_became_peered":"2024-04-25T10:06:11.596710+0000","last_unstale":"2024-04-25T10:06:12.592096+0000","last_undegraded":"2024-04-25T10:06:12.592096+0000","last_fullsized":"2024-04-25T10:06:12.592096+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.595406+0000","last_change":"2024-04-25T10:06:11.594514+0000","last_active":"2024-04-25T10:06:11.595406+0000","last_peered":"2024-04-25T10:06:11.595406+0000","last_clean":"2024-04-25T10:06:11.595406+0000","last_became_active":"2024-04-25T10:06:11.594231+0000","last_became_peered":"2024-04-25T10:06:11.594231+0000","last_unstale":"2024-04-25T10:06:11.595406+0000","last_undegraded":"2024-04-25T10:06:11.595406+0000","last_fullsized":"2024-04-25T10:06:11.595406+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:03.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590474+0000","last_change":"2024-04-25T10:06:11.595941+0000","last_active":"2024-04-25T10:06:12.590474+0000","last_peered":"2024-04-25T10:06:12.590474+0000","last_clean":"2024-04-25T10:06:12.590474+0000","last_became_active":"2024-04-25T10:06:11.595598+0000","last_became_peered":"2024-04-25T10:06:11.595598+0000","last_unstale":"2024-04-25T10:06:12.590474+0000","last_undegraded":"2024-04-25T10:06:12.590474+0000","last_fullsized":"2024-04-25T10:06:12.590474+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:40.079857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591540+0000","last_change":"2024-04-25T10:06:11.599549+0000","last_active":"2024-04-25T10:06:12.591540+0000","last_peered":"2024-04-25T10:06:12.591540+0000","last_clean":"2024-04-25T10:06:12.591540+0000","last_became_active":"2024-04-25T10:06:11.599256+0000","last_became_peered":"2024-04-25T10:06:11.599256+0000","last_unstale":"2024-04-25T10:06:12.591540+0000","last_undegraded":"2024-04-25T10:06:12.591540+0000","last_fullsized":"2024-04-25T10:06:12.591540+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:48.190466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590264+0000","last_change":"2024-04-25T10:06:11.605175+0000","last_active":"2024-04-25T10:06:12.590264+0000","last_peered":"2024-04-25T10:06:12.590264+0000","last_clean":"2024-04-25T10:06:12.590264+0000","last_became_active":"2024-04-25T10:06:11.604773+0000","last_became_peered":"2024-04-25T10:06:11.604773+0000","last_unstale":"2024-04-25T10:06:12.590264+0000","last_undegraded":"2024-04-25T10:06:12.590264+0000","last_fullsized":"2024-04-25T10:06:12.590264+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.403695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611074+0000","last_change":"2024-04-25T10:06:11.605194+0000","last_active":"2024-04-25T10:06:11.611074+0000","last_peered":"2024-04-25T10:06:11.611074+0000","last_clean":"2024-04-25T10:06:11.611074+0000","last_became_active":"2024-04-25T10:06:11.604595+0000","last_became_peered":"2024-04-25T10:06:11.604595+0000","last_unstale":"2024-04-25T10:06:11.611074+0000","last_undegraded":"2024-04-25T10:06:11.611074+0000","last_fullsized":"2024-04-25T10:06:11.611074+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:36.054034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611662+0000","last_change":"2024-04-25T10:06:11.603505+0000","last_active":"2024-04-25T10:06:11.611662+0000","last_peered":"2024-04-25T10:06:11.611662+0000","last_clean":"2024-04-25T10:06:11.611662+0000","last_became_active":"2024-04-25T10:06:11.603215+0000","last_became_peered":"2024-04-25T10:06:11.603215+0000","last_unstale":"2024-04-25T10:06:11.611662+0000","last_undegraded":"2024-04-25T10:06:11.611662+0000","last_fullsized":"2024-04-25T10:06:11.611662+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:10.566145+0000","last_change":"2024-04-25T10:06:10.566145+0000","last_active":"2024-04-25T10:06:10.566145+0000","last_peered":"2024-04-25T10:06:10.566145+0000","last_clean":"2024-04-25T10:06:10.566145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:10.566145+0000","last_undegraded":"2024-04-25T10:06:10.566145+0000","last_fullsized":"2024-04-25T10:06:10.566145+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591554+0000","last_change":"2024-04-25T10:06:11.601007+0000","last_active":"2024-04-25T10:06:12.591554+0000","last_peered":"2024-04-25T10:06:12.591554+0000","last_clean":"2024-04-25T10:06:12.591554+0000","last_became_active":"2024-04-25T10:06:11.600297+0000","last_became_peered":"2024-04-25T10:06:11.600297+0000","last_unstale":"2024-04-25T10:06:12.591554+0000","last_undegraded":"2024-04-25T10:06:12.591554+0000","last_fullsized":"2024-04-25T10:06:12.591554+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:18.645477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.601405+0000","last_change":"2024-04-25T10:06:11.600447+0000","last_active":"2024-04-25T10:06:11.601405+0000","last_peered":"2024-04-25T10:06:11.601405+0000","last_clean":"2024-04-25T10:06:11.601405+0000","last_became_active":"2024-04-25T10:06:11.600160+0000","last_became_peered":"2024-04-25T10:06:11.600160+0000","last_unstale":"2024-04-25T10:06:11.601405+0000","last_undegraded":"2024-04-25T10:06:11.601405+0000","last_fullsized":"2024-04-25T10:06:11.601405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:50.014644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.591617+0000","last_change":"2024-04-25T10:06:11.590349+0000","last_active":"2024-04-25T10:06:11.591617+0000","last_peered":"2024-04-25T10:06:11.591617+0000","last_clean":"2024-04-25T10:06:11.591617+0000","last_became_active":"2024-04-25T10:06:11.590090+0000","last_became_peered":"2024-04-25T10:06:11.590090+0000","last_unstale":"2024-04-25T10:06:11.591617+0000","last_undegraded":"2024-04-25T10:06:11.591617+0000","last_fullsized":"2024-04-25T10:06:11.591617+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590344+0000","last_change":"2024-04-25T10:06:11.597956+0000","last_active":"2024-04-25T10:06:12.590344+0000","last_peered":"2024-04-25T10:06:12.590344+0000","last_clean":"2024-04-25T10:06:12.590344+0000","last_became_active":"2024-04-25T10:06:11.597687+0000","last_became_peered":"2024-04-25T10:06:11.597687+0000","last_unstale":"2024-04-25T10:06:12.590344+0000","last_undegraded":"2024-04-25T10:06:12.590344+0000","last_fullsized":"2024-04-25T10:06:12.590344+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.586794+0000","last_change":"2024-04-25T10:06:11.584953+0000","last_active":"2024-04-25T10:06:11.586794+0000","last_peered":"2024-04-25T10:06:11.586794+0000","last_clean":"2024-04-25T10:06:11.586794+0000","last_became_active":"2024-04-25T10:06:11.584666+0000","last_became_peered":"2024-04-25T10:06:11.584666+0000","last_unstale":"2024-04-25T10:06:11.586794+0000","last_undegraded":"2024-04-25T10:06:11.586794+0000","last_fullsized":"2024-04-25T10:06:11.586794+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:55.206873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591906+0000","last_change":"2024-04-25T10:06:11.592933+0000","last_active":"2024-04-25T10:06:12.591906+0000","last_peered":"2024-04-25T10:06:12.591906+0000","last_clean":"2024-04-25T10:06:12.591906+0000","last_became_active":"2024-04-25T10:06:11.592335+0000","last_became_peered":"2024-04-25T10:06:11.592335+0000","last_unstale":"2024-04-25T10:06:12.591906+0000","last_undegraded":"2024-04-25T10:06:12.591906+0000","last_fullsized":"2024-04-25T10:06:12.591906+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.403695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.602910+0000","last_change":"2024-04-25T10:06:10.590142+0000","last_active":"2024-04-25T10:06:11.602910+0000","last_peered":"2024-04-25T10:06:11.602910+0000","last_clean":"2024-04-25T10:06:11.602910+0000","last_became_active":"2024-04-25T10:06:10.589851+0000","last_became_peered":"2024-04-25T10:06:10.589851+0000","last_unstale":"2024-04-25T10:06:11.602910+0000","last_undegraded":"2024-04-25T10:06:11.602910+0000","last_fullsized":"2024-04-25T10:06:11.602910+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:50.142763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590901+0000","last_change":"2024-04-25T10:06:11.586712+0000","last_active":"2024-04-25T10:06:12.590901+0000","last_peered":"2024-04-25T10:06:12.590901+0000","last_clean":"2024-04-25T10:06:12.590901+0000","last_became_active":"2024-04-25T10:06:11.586393+0000","last_became_peered":"2024-04-25T10:06:11.586393+0000","last_unstale":"2024-04-25T10:06:12.590901+0000","last_undegraded":"2024-04-25T10:06:12.590901+0000","last_fullsized":"2024-04-25T10:06:12.590901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:11.365920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611691+0000","last_change":"2024-04-25T10:06:10.590906+0000","last_active":"2024-04-25T10:06:11.611691+0000","last_peered":"2024-04-25T10:06:11.611691+0000","last_clean":"2024-04-25T10:06:11.611691+0000","last_became_active":"2024-04-25T10:06:10.590641+0000","last_became_peered":"2024-04-25T10:06:10.590641+0000","last_unstale":"2024-04-25T10:06:11.611691+0000","last_undegraded":"2024-04-25T10:06:11.611691+0000","last_fullsized":"2024-04-25T10:06:11.611691+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:39.077377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591703+0000","last_change":"2024-04-25T10:06:11.601284+0000","last_active":"2024-04-25T10:06:12.591703+0000","last_peered":"2024-04-25T10:06:12.591703+0000","last_clean":"2024-04-25T10:06:12.591703+0000","last_became_active":"2024-04-25T10:06:11.600739+0000","last_became_peered":"2024-04-25T10:06:11.600739+0000","last_unstale":"2024-04-25T10:06:12.591703+0000","last_undegraded":"2024-04-25T10:06:12.591703+0000","last_fullsized":"2024-04-25T10:06:12.591703+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:03.030946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.594679+0000","last_change":"2024-04-25T10:06:10.589990+0000","last_active":"2024-04-25T10:06:11.594679+0000","last_peered":"2024-04-25T10:06:11.594679+0000","last_clean":"2024-04-25T10:06:11.594679+0000","last_became_active":"2024-04-25T10:06:10.589663+0000","last_became_peered":"2024-04-25T10:06:10.589663+0000","last_unstale":"2024-04-25T10:06:11.594679+0000","last_undegraded":"2024-04-25T10:06:11.594679+0000","last_fullsized":"2024-04-25T10:06:11.594679+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:17.017530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611361+0000","last_change":"2024-04-25T10:06:11.608503+0000","last_active":"2024-04-25T10:06:11.611361+0000","last_peered":"2024-04-25T10:06:11.611361+0000","last_clean":"2024-04-25T10:06:11.611361+0000","last_became_active":"2024-04-25T10:06:11.608234+0000","last_became_peered":"2024-04-25T10:06:11.608234+0000","last_unstale":"2024-04-25T10:06:11.611361+0000","last_undegraded":"2024-04-25T10:06:11.611361+0000","last_fullsized":"2024-04-25T10:06:11.611361+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:11.365920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.604687+0000","last_change":"2024-04-25T10:06:10.590974+0000","last_active":"2024-04-25T10:06:11.604687+0000","last_peered":"2024-04-25T10:06:11.604687+0000","last_clean":"2024-04-25T10:06:11.604687+0000","last_became_active":"2024-04-25T10:06:10.590635+0000","last_became_peered":"2024-04-25T10:06:10.590635+0000","last_unstale":"2024-04-25T10:06:11.604687+0000","last_undegraded":"2024-04-25T10:06:11.604687+0000","last_fullsized":"2024-04-25T10:06:11.604687+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:31.888158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591335+0000","last_change":"2024-04-25T10:06:11.594569+0000","last_active":"2024-04-25T10:06:12.591335+0000","last_peered":"2024-04-25T10:06:12.591335+0000","last_clean":"2024-04-25T10:06:12.591335+0000","last_became_active":"2024-04-25T10:06:11.594016+0000","last_became_peered":"2024-04-25T10:06:11.594016+0000","last_unstale":"2024-04-25T10:06:12.591335+0000","last_undegraded":"2024-04-25T10:06:12.591335+0000","last_fullsized":"2024-04-25T10:06:12.591335+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:38.022779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590761+0000","last_change":"2024-04-25T10:06:10.591527+0000","last_active":"2024-04-25T10:06:12.590761+0000","last_peered":"2024-04-25T10:06:12.590761+0000","last_clean":"2024-04-25T10:06:12.590761+0000","last_became_active":"2024-04-25T10:06:10.591212+0000","last_became_peered":"2024-04-25T10:06:10.591212+0000","last_unstale":"2024-04-25T10:06:12.590761+0000","last_undegraded":"2024-04-25T10:06:12.590761+0000","last_fullsized":"2024-04-25T10:06:12.590761+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:17.896865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590681+0000","last_change":"2024-04-25T10:06:11.597833+0000","last_active":"2024-04-25T10:06:12.590681+0000","last_peered":"2024-04-25T10:06:12.590681+0000","last_clean":"2024-04-25T10:06:12.590681+0000","last_became_active":"2024-04-25T10:06:11.597247+0000","last_became_peered":"2024-04-25T10:06:11.597247+0000","last_unstale":"2024-04-25T10:06:12.590681+0000","last_undegraded":"2024-04-25T10:06:12.590681+0000","last_fullsized":"2024-04-25T10:06:12.590681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:36.054034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590610+0000","last_change":"2024-04-25T10:06:10.593926+0000","last_active":"2024-04-25T10:06:12.590610+0000","last_peered":"2024-04-25T10:06:12.590610+0000","last_clean":"2024-04-25T10:06:12.590610+0000","last_became_active":"2024-04-25T10:06:10.593640+0000","last_became_peered":"2024-04-25T10:06:10.593640+0000","last_unstale":"2024-04-25T10:06:12.590610+0000","last_undegraded":"2024-04-25T10:06:12.590610+0000","last_fullsized":"2024-04-25T10:06:12.590610+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:32.052858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591186+0000","last_change":"2024-04-25T10:06:11.593063+0000","last_active":"2024-04-25T10:06:12.591186+0000","last_peered":"2024-04-25T10:06:12.591186+0000","last_clean":"2024-04-25T10:06:12.591186+0000","last_became_active":"2024-04-25T10:06:11.592772+0000","last_became_peered":"2024-04-25T10:06:11.592772+0000","last_unstale":"2024-04-25T10:06:12.591186+0000","last_undegraded":"2024-04-25T10:06:12.591186+0000","last_fullsized":"2024-04-25T10:06:12.591186+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:12.627498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.584205+0000","last_change":"2024-04-25T10:06:10.594103+0000","last_active":"2024-04-25T10:06:11.584205+0000","last_peered":"2024-04-25T10:06:11.584205+0000","last_clean":"2024-04-25T10:06:11.584205+0000","last_became_active":"2024-04-25T10:06:10.593805+0000","last_became_peered":"2024-04-25T10:06:10.593805+0000","last_unstale":"2024-04-25T10:06:11.584205+0000","last_undegraded":"2024-04-25T10:06:11.584205+0000","last_fullsized":"2024-04-25T10:06:11.584205+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:21.758145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.597476+0000","last_change":"2024-04-25T10:06:11.590223+0000","last_active":"2024-04-25T10:06:11.597476+0000","last_peered":"2024-04-25T10:06:11.597476+0000","last_clean":"2024-04-25T10:06:11.597476+0000","last_became_active":"2024-04-25T10:06:11.589970+0000","last_became_peered":"2024-04-25T10:06:11.589970+0000","last_unstale":"2024-04-25T10:06:11.597476+0000","last_undegraded":"2024-04-25T10:06:11.597476+0000","last_fullsized":"2024-04-25T10:06:11.597476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:44.396817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590364+0000","last_change":"2024-04-25T10:06:10.590741+0000","last_active":"2024-04-25T10:06:12.590364+0000","last_peered":"2024-04-25T10:06:12.590364+0000","last_clean":"2024-04-25T10:06:12.590364+0000","last_became_active":"2024-04-25T10:06:10.590499+0000","last_became_peered":"2024-04-25T10:06:10.590499+0000","last_unstale":"2024-04-25T10:06:12.590364+0000","last_undegraded":"2024-04-25T10:06:12.590364+0000","last_fullsized":"2024-04-25T10:06:12.590364+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:32.676160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599229+0000","last_change":"2024-04-25T10:06:11.594584+0000","last_active":"2024-04-25T10:06:11.599229+0000","last_peered":"2024-04-25T10:06:11.599229+0000","last_clean":"2024-04-25T10:06:11.599229+0000","last_became_active":"2024-04-25T10:06:11.594264+0000","last_became_peered":"2024-04-25T10:06:11.594264+0000","last_unstale":"2024-04-25T10:06:11.599229+0000","last_undegraded":"2024-04-25T10:06:11.599229+0000","last_fullsized":"2024-04-25T10:06:11.599229+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:33.855981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-25T10:06:09.571413+0000","last_change":"2024-04-25T10:06:09.571413+0000","last_active":"2024-04-25T10:06:09.563666+0000","last_peered":"2024-04-25T10:06:09.563666+0000","last_clean":"2024-04-25T10:06:09.563666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:09.571413+0000","last_undegraded":"2024-04-25T10:06:09.571413+0000","last_fullsized":"2024-04-25T10:06:09.571413+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.603137+0000","last_change":"2024-04-25T10:06:11.598368+0000","last_active":"2024-04-25T10:06:11.603137+0000","last_peered":"2024-04-25T10:06:11.603137+0000","last_clean":"2024-04-25T10:06:11.603137+0000","last_became_active":"2024-04-25T10:06:11.597963+0000","last_became_peered":"2024-04-25T10:06:11.597963+0000","last_unstale":"2024-04-25T10:06:11.603137+0000","last_undegraded":"2024-04-25T10:06:11.603137+0000","last_fullsized":"2024-04-25T10:06:11.603137+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:48.190466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-25T10:06:09.571772+0000","last_change":"2024-04-25T10:06:09.571772+0000","last_active":"2024-04-25T10:06:09.563666+0000","last_peered":"2024-04-25T10:06:09.563666+0000","last_clean":"2024-04-25T10:06:09.563666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:09.571772+0000","last_undegraded":"2024-04-25T10:06:09.571772+0000","last_fullsized":"2024-04-25T10:06:09.571772+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610787+0000","last_change":"2024-04-25T10:06:11.603945+0000","last_active":"2024-04-25T10:06:11.610787+0000","last_peered":"2024-04-25T10:06:11.610787+0000","last_clean":"2024-04-25T10:06:11.610787+0000","last_became_active":"2024-04-25T10:06:11.602849+0000","last_became_peered":"2024-04-25T10:06:11.602849+0000","last_unstale":"2024-04-25T10:06:11.610787+0000","last_undegraded":"2024-04-25T10:06:11.610787+0000","last_fullsized":"2024-04-25T10:06:11.610787+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:03.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.601893+0000","last_change":"2024-04-25T10:06:10.579763+0000","last_active":"2024-04-25T10:06:11.601893+0000","last_peered":"2024-04-25T10:06:11.601893+0000","last_clean":"2024-04-25T10:06:11.601893+0000","last_became_active":"2024-04-25T10:06:10.579495+0000","last_became_peered":"2024-04-25T10:06:10.579495+0000","last_unstale":"2024-04-25T10:06:11.601893+0000","last_undegraded":"2024-04-25T10:06:11.601893+0000","last_fullsized":"2024-04-25T10:06:11.601893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:08.212125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.603845+0000","last_change":"2024-04-25T10:06:11.603317+0000","last_active":"2024-04-25T10:06:11.603845+0000","last_peered":"2024-04-25T10:06:11.603845+0000","last_clean":"2024-04-25T10:06:11.603845+0000","last_became_active":"2024-04-25T10:06:11.603066+0000","last_became_peered":"2024-04-25T10:06:11.603066+0000","last_unstale":"2024-04-25T10:06:11.603845+0000","last_undegraded":"2024-04-25T10:06:11.603845+0000","last_fullsized":"2024-04-25T10:06:11.603845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:07.686632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592122+0000","last_change":"2024-04-25T10:06:10.593257+0000","last_active":"2024-04-25T10:06:12.592122+0000","last_peered":"2024-04-25T10:06:12.592122+0000","last_clean":"2024-04-25T10:06:12.592122+0000","last_became_active":"2024-04-25T10:06:10.592737+0000","last_became_peered":"2024-04-25T10:06:10.592737+0000","last_unstale":"2024-04-25T10:06:12.592122+0000","last_undegraded":"2024-04-25T10:06:12.592122+0000","last_fullsized":"2024-04-25T10:06:12.592122+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:38.738216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591639+0000","last_change":"2024-04-25T10:06:11.601746+0000","last_active":"2024-04-25T10:06:12.591639+0000","last_peered":"2024-04-25T10:06:12.591639+0000","last_clean":"2024-04-25T10:06:12.591639+0000","last_became_active":"2024-04-25T10:06:11.600706+0000","last_became_peered":"2024-04-25T10:06:11.600706+0000","last_unstale":"2024-04-25T10:06:12.591639+0000","last_undegraded":"2024-04-25T10:06:12.591639+0000","last_fullsized":"2024-04-25T10:06:12.591639+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:37.328665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590790+0000","last_change":"2024-04-25T10:06:10.589747+0000","last_active":"2024-04-25T10:06:12.590790+0000","last_peered":"2024-04-25T10:06:12.590790+0000","last_clean":"2024-04-25T10:06:12.590790+0000","last_became_active":"2024-04-25T10:06:10.589483+0000","last_became_peered":"2024-04-25T10:06:10.589483+0000","last_unstale":"2024-04-25T10:06:12.590790+0000","last_undegraded":"2024-04-25T10:06:12.590790+0000","last_fullsized":"2024-04-25T10:06:12.590790+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.133966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.605963+0000","last_change":"2024-04-25T10:06:11.599329+0000","last_active":"2024-04-25T10:06:11.605963+0000","last_peered":"2024-04-25T10:06:11.605963+0000","last_clean":"2024-04-25T10:06:11.605963+0000","last_became_active":"2024-04-25T10:06:11.599055+0000","last_became_peered":"2024-04-25T10:06:11.599055+0000","last_unstale":"2024-04-25T10:06:11.605963+0000","last_undegraded":"2024-04-25T10:06:11.605963+0000","last_fullsized":"2024-04-25T10:06:11.605963+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:04.221854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.590118+0000","last_change":"2024-04-25T10:06:10.592105+0000","last_active":"2024-04-25T10:06:11.590118+0000","last_peered":"2024-04-25T10:06:11.590118+0000","last_clean":"2024-04-25T10:06:11.590118+0000","last_became_active":"2024-04-25T10:06:10.591823+0000","last_became_peered":"2024-04-25T10:06:10.591823+0000","last_unstale":"2024-04-25T10:06:11.590118+0000","last_undegraded":"2024-04-25T10:06:11.590118+0000","last_fullsized":"2024-04-25T10:06:11.590118+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:53.425769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590360+0000","last_change":"2024-04-25T10:06:11.585750+0000","last_active":"2024-04-25T10:06:12.590360+0000","last_peered":"2024-04-25T10:06:12.590360+0000","last_clean":"2024-04-25T10:06:12.590360+0000","last_became_active":"2024-04-25T10:06:11.585193+0000","last_became_peered":"2024-04-25T10:06:11.585193+0000","last_unstale":"2024-04-25T10:06:12.590360+0000","last_undegraded":"2024-04-25T10:06:12.590360+0000","last_fullsized":"2024-04-25T10:06:12.590360+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:03.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.589817+0000","last_change":"2024-04-25T10:06:10.590311+0000","last_active":"2024-04-25T10:06:11.589817+0000","last_peered":"2024-04-25T10:06:11.589817+0000","last_clean":"2024-04-25T10:06:11.589817+0000","last_became_active":"2024-04-25T10:06:10.590015+0000","last_became_peered":"2024-04-25T10:06:10.590015+0000","last_unstale":"2024-04-25T10:06:11.589817+0000","last_undegraded":"2024-04-25T10:06:11.589817+0000","last_fullsized":"2024-04-25T10:06:11.589817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:27.524048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.585155+0000","last_change":"2024-04-25T10:06:11.582381+0000","last_active":"2024-04-25T10:06:11.585155+0000","last_peered":"2024-04-25T10:06:11.585155+0000","last_clean":"2024-04-25T10:06:11.585155+0000","last_became_active":"2024-04-25T10:06:11.582109+0000","last_became_peered":"2024-04-25T10:06:11.582109+0000","last_unstale":"2024-04-25T10:06:11.585155+0000","last_undegraded":"2024-04-25T10:06:11.585155+0000","last_fullsized":"2024-04-25T10:06:11.585155+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:41.604772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.604177+0000","last_change":"2024-04-25T10:06:10.589625+0000","last_active":"2024-04-25T10:06:11.604177+0000","last_peered":"2024-04-25T10:06:11.604177+0000","last_clean":"2024-04-25T10:06:11.604177+0000","last_became_active":"2024-04-25T10:06:10.589161+0000","last_became_peered":"2024-04-25T10:06:10.589161+0000","last_unstale":"2024-04-25T10:06:11.604177+0000","last_undegraded":"2024-04-25T10:06:11.604177+0000","last_fullsized":"2024-04-25T10:06:11.604177+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:47.673088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.612662+0000","last_change":"2024-04-25T10:06:11.612101+0000","last_active":"2024-04-25T10:06:11.612662+0000","last_peered":"2024-04-25T10:06:11.612662+0000","last_clean":"2024-04-25T10:06:11.612662+0000","last_became_active":"2024-04-25T10:06:11.611805+0000","last_became_peered":"2024-04-25T10:06:11.611805+0000","last_unstale":"2024-04-25T10:06:11.612662+0000","last_undegraded":"2024-04-25T10:06:11.612662+0000","last_fullsized":"2024-04-25T10:06:11.612662+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:22.287749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.598493+0000","last_change":"2024-04-25T10:06:10.594406+0000","last_active":"2024-04-25T10:06:11.598493+0000","last_peered":"2024-04-25T10:06:11.598493+0000","last_clean":"2024-04-25T10:06:11.598493+0000","last_became_active":"2024-04-25T10:06:10.594129+0000","last_became_peered":"2024-04-25T10:06:10.594129+0000","last_unstale":"2024-04-25T10:06:11.598493+0000","last_undegraded":"2024-04-25T10:06:11.598493+0000","last_fullsized":"2024-04-25T10:06:11.598493+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:19.498453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591710+0000","last_change":"2024-04-25T10:06:11.592494+0000","last_active":"2024-04-25T10:06:12.591710+0000","last_peered":"2024-04-25T10:06:12.591710+0000","last_clean":"2024-04-25T10:06:12.591710+0000","last_became_active":"2024-04-25T10:06:11.592212+0000","last_became_peered":"2024-04-25T10:06:11.592212+0000","last_unstale":"2024-04-25T10:06:12.591710+0000","last_undegraded":"2024-04-25T10:06:12.591710+0000","last_fullsized":"2024-04-25T10:06:12.591710+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:08.065111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590880+0000","last_change":"2024-04-25T10:06:10.588563+0000","last_active":"2024-04-25T10:06:12.590880+0000","last_peered":"2024-04-25T10:06:12.590880+0000","last_clean":"2024-04-25T10:06:12.590880+0000","last_became_active":"2024-04-25T10:06:10.588312+0000","last_became_peered":"2024-04-25T10:06:10.588312+0000","last_unstale":"2024-04-25T10:06:12.590880+0000","last_undegraded":"2024-04-25T10:06:12.590880+0000","last_fullsized":"2024-04-25T10:06:12.590880+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:17.315120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:10.566145+0000","last_change":"2024-04-25T10:06:10.566145+0000","last_active":"2024-04-25T10:06:10.566145+0000","last_peered":"2024-04-25T10:06:10.566145+0000","last_clean":"2024-04-25T10:06:10.566145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:10.566145+0000","last_undegraded":"2024-04-25T10:06:10.566145+0000","last_fullsized":"2024-04-25T10:06:10.566145+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.602393+0000","last_change":"2024-04-25T10:06:10.588662+0000","last_active":"2024-04-25T10:06:11.602393+0000","last_peered":"2024-04-25T10:06:11.602393+0000","last_clean":"2024-04-25T10:06:11.602393+0000","last_became_active":"2024-04-25T10:06:10.588316+0000","last_became_peered":"2024-04-25T10:06:10.588316+0000","last_unstale":"2024-04-25T10:06:11.602393+0000","last_undegraded":"2024-04-25T10:06:11.602393+0000","last_fullsized":"2024-04-25T10:06:11.602393+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:32.850838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599469+0000","last_change":"2024-04-25T10:06:11.595815+0000","last_active":"2024-04-25T10:06:11.599469+0000","last_peered":"2024-04-25T10:06:11.599469+0000","last_clean":"2024-04-25T10:06:11.599469+0000","last_became_active":"2024-04-25T10:06:11.595495+0000","last_became_peered":"2024-04-25T10:06:11.595495+0000","last_unstale":"2024-04-25T10:06:11.599469+0000","last_undegraded":"2024-04-25T10:06:11.599469+0000","last_fullsized":"2024-04-25T10:06:11.599469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:12.627498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.581094+0000","last_change":"2024-04-25T10:06:10.591774+0000","last_active":"2024-04-25T10:06:11.581094+0000","last_peered":"2024-04-25T10:06:11.581094+0000","last_clean":"2024-04-25T10:06:11.581094+0000","last_became_active":"2024-04-25T10:06:10.591412+0000","last_became_peered":"2024-04-25T10:06:10.591412+0000","last_unstale":"2024-04-25T10:06:11.581094+0000","last_undegraded":"2024-04-25T10:06:11.581094+0000","last_fullsized":"2024-04-25T10:06:11.581094+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:17.401215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591675+0000","last_change":"2024-04-25T10:06:11.593318+0000","last_active":"2024-04-25T10:06:12.591675+0000","last_peered":"2024-04-25T10:06:12.591675+0000","last_clean":"2024-04-25T10:06:12.591675+0000","last_became_active":"2024-04-25T10:06:11.593022+0000","last_became_peered":"2024-04-25T10:06:11.593022+0000","last_unstale":"2024-04-25T10:06:12.591675+0000","last_undegraded":"2024-04-25T10:06:12.591675+0000","last_fullsized":"2024-04-25T10:06:12.591675+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:46.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-25T10:06:09.571392+0000","last_change":"2024-04-25T10:06:09.571392+0000","last_active":"2024-04-25T10:06:09.563666+0000","last_peered":"2024-04-25T10:06:09.563666+0000","last_clean":"2024-04-25T10:06:09.563666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:09.571392+0000","last_undegraded":"2024-04-25T10:06:09.571392+0000","last_fullsized":"2024-04-25T10:06:09.571392+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:10.566145+0000","last_change":"2024-04-25T10:06:10.566145+0000","last_active":"2024-04-25T10:06:10.566145+0000","last_peered":"2024-04-25T10:06:10.566145+0000","last_clean":"2024-04-25T10:06:10.566145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:10.566145+0000","last_undegraded":"2024-04-25T10:06:10.566145+0000","last_fullsized":"2024-04-25T10:06:10.566145+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590609+0000","last_change":"2024-04-25T10:06:10.590140+0000","last_active":"2024-04-25T10:06:12.590609+0000","last_peered":"2024-04-25T10:06:12.590609+0000","last_clean":"2024-04-25T10:06:12.590609+0000","last_became_active":"2024-04-25T10:06:10.589015+0000","last_became_peered":"2024-04-25T10:06:10.589015+0000","last_unstale":"2024-04-25T10:06:12.590609+0000","last_undegraded":"2024-04-25T10:06:12.590609+0000","last_fullsized":"2024-04-25T10:06:12.590609+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:18.383218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591617+0000","last_change":"2024-04-25T10:06:11.604928+0000","last_active":"2024-04-25T10:06:12.591617+0000","last_peered":"2024-04-25T10:06:12.591617+0000","last_clean":"2024-04-25T10:06:12.591617+0000","last_became_active":"2024-04-25T10:06:11.604636+0000","last_became_peered":"2024-04-25T10:06:11.604636+0000","last_unstale":"2024-04-25T10:06:12.591617+0000","last_undegraded":"2024-04-25T10:06:12.591617+0000","last_fullsized":"2024-04-25T10:06:12.591617+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:04.221854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591565+0000","last_change":"2024-04-25T10:06:10.590105+0000","last_active":"2024-04-25T10:06:12.591565+0000","last_peered":"2024-04-25T10:06:12.591565+0000","last_clean":"2024-04-25T10:06:12.591565+0000","last_became_active":"2024-04-25T10:06:10.589829+0000","last_became_peered":"2024-04-25T10:06:10.589829+0000","last_unstale":"2024-04-25T10:06:12.591565+0000","last_undegraded":"2024-04-25T10:06:12.591565+0000","last_fullsized":"2024-04-25T10:06:12.591565+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:23.745526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591751+0000","last_change":"2024-04-25T10:06:11.593413+0000","last_active":"2024-04-25T10:06:12.591751+0000","last_peered":"2024-04-25T10:06:12.591751+0000","last_clean":"2024-04-25T10:06:12.591751+0000","last_became_active":"2024-04-25T10:06:11.593155+0000","last_became_peered":"2024-04-25T10:06:11.593155+0000","last_unstale":"2024-04-25T10:06:12.591751+0000","last_undegraded":"2024-04-25T10:06:12.591751+0000","last_fullsized":"2024-04-25T10:06:12.591751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:33.678639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.579429+0000","last_change":"2024-04-25T10:06:10.587062+0000","last_active":"2024-04-25T10:06:11.579429+0000","last_peered":"2024-04-25T10:06:11.579429+0000","last_clean":"2024-04-25T10:06:11.579429+0000","last_became_active":"2024-04-25T10:06:10.586736+0000","last_became_peered":"2024-04-25T10:06:10.586736+0000","last_unstale":"2024-04-25T10:06:11.579429+0000","last_undegraded":"2024-04-25T10:06:11.579429+0000","last_fullsized":"2024-04-25T10:06:11.579429+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:48.572483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599597+0000","last_change":"2024-04-25T10:06:11.595699+0000","last_active":"2024-04-25T10:06:11.599597+0000","last_peered":"2024-04-25T10:06:11.599597+0000","last_clean":"2024-04-25T10:06:11.599597+0000","last_became_active":"2024-04-25T10:06:11.595437+0000","last_became_peered":"2024-04-25T10:06:11.595437+0000","last_unstale":"2024-04-25T10:06:11.599597+0000","last_undegraded":"2024-04-25T10:06:11.599597+0000","last_fullsized":"2024-04-25T10:06:11.599597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:38.022779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591835+0000","last_change":"2024-04-25T10:06:10.588344+0000","last_active":"2024-04-25T10:06:12.591835+0000","last_peered":"2024-04-25T10:06:12.591835+0000","last_clean":"2024-04-25T10:06:12.591835+0000","last_became_active":"2024-04-25T10:06:10.587774+0000","last_became_peered":"2024-04-25T10:06:10.587774+0000","last_unstale":"2024-04-25T10:06:12.591835+0000","last_undegraded":"2024-04-25T10:06:12.591835+0000","last_fullsized":"2024-04-25T10:06:12.591835+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:13.625296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.603613+0000","last_change":"2024-04-25T10:06:11.603042+0000","last_active":"2024-04-25T10:06:11.603613+0000","last_peered":"2024-04-25T10:06:11.603613+0000","last_clean":"2024-04-25T10:06:11.603613+0000","last_became_active":"2024-04-25T10:06:11.598535+0000","last_became_peered":"2024-04-25T10:06:11.598535+0000","last_unstale":"2024-04-25T10:06:11.603613+0000","last_undegraded":"2024-04-25T10:06:11.603613+0000","last_fullsized":"2024-04-25T10:06:11.603613+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:31.070811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-25T10:06:09.571309+0000","last_change":"2024-04-25T10:06:09.571309+0000","last_active":"2024-04-25T10:06:09.563666+0000","last_peered":"2024-04-25T10:06:09.563666+0000","last_clean":"2024-04-25T10:06:09.563666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:09.571309+0000","last_undegraded":"2024-04-25T10:06:09.571309+0000","last_fullsized":"2024-04-25T10:06:09.571309+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.605940+0000","last_change":"2024-04-25T10:06:11.603834+0000","last_active":"2024-04-25T10:06:11.605940+0000","last_peered":"2024-04-25T10:06:11.605940+0000","last_clean":"2024-04-25T10:06:11.605940+0000","last_became_active":"2024-04-25T10:06:11.603541+0000","last_became_peered":"2024-04-25T10:06:11.603541+0000","last_unstale":"2024-04-25T10:06:11.605940+0000","last_undegraded":"2024-04-25T10:06:11.605940+0000","last_fullsized":"2024-04-25T10:06:11.605940+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:08.065111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592173+0000","last_change":"2024-04-25T10:06:10.590856+0000","last_active":"2024-04-25T10:06:12.592173+0000","last_peered":"2024-04-25T10:06:12.592173+0000","last_clean":"2024-04-25T10:06:12.592173+0000","last_became_active":"2024-04-25T10:06:10.590571+0000","last_became_peered":"2024-04-25T10:06:10.590571+0000","last_unstale":"2024-04-25T10:06:12.592173+0000","last_undegraded":"2024-04-25T10:06:12.592173+0000","last_fullsized":"2024-04-25T10:06:12.592173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:01.814874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.596509+0000","last_change":"2024-04-25T10:06:11.595775+0000","last_active":"2024-04-25T10:06:11.596509+0000","last_peered":"2024-04-25T10:06:11.596509+0000","last_clean":"2024-04-25T10:06:11.596509+0000","last_became_active":"2024-04-25T10:06:11.595387+0000","last_became_peered":"2024-04-25T10:06:11.595387+0000","last_unstale":"2024-04-25T10:06:11.596509+0000","last_undegraded":"2024-04-25T10:06:11.596509+0000","last_fullsized":"2024-04-25T10:06:11.596509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:40.079857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592115+0000","last_change":"2024-04-25T10:06:10.588101+0000","last_active":"2024-04-25T10:06:12.592115+0000","last_peered":"2024-04-25T10:06:12.592115+0000","last_clean":"2024-04-25T10:06:12.592115+0000","last_became_active":"2024-04-25T10:06:10.587808+0000","last_became_peered":"2024-04-25T10:06:10.587808+0000","last_unstale":"2024-04-25T10:06:12.592115+0000","last_undegraded":"2024-04-25T10:06:12.592115+0000","last_fullsized":"2024-04-25T10:06:12.592115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.133966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610579+0000","last_change":"2024-04-25T10:06:11.599755+0000","last_active":"2024-04-25T10:06:11.610579+0000","last_peered":"2024-04-25T10:06:11.610579+0000","last_clean":"2024-04-25T10:06:11.610579+0000","last_became_active":"2024-04-25T10:06:11.599317+0000","last_became_peered":"2024-04-25T10:06:11.599317+0000","last_unstale":"2024-04-25T10:06:11.610579+0000","last_undegraded":"2024-04-25T10:06:11.610579+0000","last_fullsized":"2024-04-25T10:06:11.610579+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:37.328665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591917+0000","last_change":"2024-04-25T10:06:10.591156+0000","last_active":"2024-04-25T10:06:12.591917+0000","last_peered":"2024-04-25T10:06:12.591917+0000","last_clean":"2024-04-25T10:06:12.591917+0000","last_became_active":"2024-04-25T10:06:10.590362+0000","last_became_peered":"2024-04-25T10:06:10.590362+0000","last_unstale":"2024-04-25T10:06:12.591917+0000","last_undegraded":"2024-04-25T10:06:12.591917+0000","last_fullsized":"2024-04-25T10:06:12.591917+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:32.052858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610758+0000","last_change":"2024-04-25T10:06:11.603491+0000","last_active":"2024-04-25T10:06:11.610758+0000","last_peered":"2024-04-25T10:06:11.610758+0000","last_clean":"2024-04-25T10:06:11.610758+0000","last_became_active":"2024-04-25T10:06:11.600221+0000","last_became_peered":"2024-04-25T10:06:11.600221+0000","last_unstale":"2024-04-25T10:06:11.610758+0000","last_undegraded":"2024-04-25T10:06:11.610758+0000","last_fullsized":"2024-04-25T10:06:11.610758+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590413+0000","last_change":"2024-04-25T10:06:10.589304+0000","last_active":"2024-04-25T10:06:12.590413+0000","last_peered":"2024-04-25T10:06:12.590413+0000","last_clean":"2024-04-25T10:06:12.590413+0000","last_became_active":"2024-04-25T10:06:10.588934+0000","last_became_peered":"2024-04-25T10:06:10.588934+0000","last_unstale":"2024-04-25T10:06:12.590413+0000","last_undegraded":"2024-04-25T10:06:12.590413+0000","last_fullsized":"2024-04-25T10:06:12.590413+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:07.062632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.602797+0000","last_change":"2024-04-25T10:06:11.597255+0000","last_active":"2024-04-25T10:06:11.602797+0000","last_peered":"2024-04-25T10:06:11.602797+0000","last_clean":"2024-04-25T10:06:11.602797+0000","last_became_active":"2024-04-25T10:06:11.596954+0000","last_became_peered":"2024-04-25T10:06:11.596954+0000","last_unstale":"2024-04-25T10:06:11.602797+0000","last_undegraded":"2024-04-25T10:06:11.602797+0000","last_fullsized":"2024-04-25T10:06:11.602797+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:13.167235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.596522+0000","last_change":"2024-04-25T10:06:10.589615+0000","last_active":"2024-04-25T10:06:11.596522+0000","last_peered":"2024-04-25T10:06:11.596522+0000","last_clean":"2024-04-25T10:06:11.596522+0000","last_became_active":"2024-04-25T10:06:10.589310+0000","last_became_peered":"2024-04-25T10:06:10.589310+0000","last_unstale":"2024-04-25T10:06:11.596522+0000","last_undegraded":"2024-04-25T10:06:11.596522+0000","last_fullsized":"2024-04-25T10:06:11.596522+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:00.780604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.597484+0000","last_change":"2024-04-25T10:06:11.596201+0000","last_active":"2024-04-25T10:06:11.597484+0000","last_peered":"2024-04-25T10:06:11.597484+0000","last_clean":"2024-04-25T10:06:11.597484+0000","last_became_active":"2024-04-25T10:06:11.595880+0000","last_became_peered":"2024-04-25T10:06:11.595880+0000","last_unstale":"2024-04-25T10:06:11.597484+0000","last_undegraded":"2024-04-25T10:06:11.597484+0000","last_fullsized":"2024-04-25T10:06:11.597484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:36.054034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.580878+0000","last_change":"2024-04-25T10:06:10.587821+0000","last_active":"2024-04-25T10:06:11.580878+0000","last_peered":"2024-04-25T10:06:11.580878+0000","last_clean":"2024-04-25T10:06:11.580878+0000","last_became_active":"2024-04-25T10:06:10.587520+0000","last_became_peered":"2024-04-25T10:06:10.587520+0000","last_unstale":"2024-04-25T10:06:11.580878+0000","last_undegraded":"2024-04-25T10:06:11.580878+0000","last_fullsized":"2024-04-25T10:06:11.580878+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:17.031658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":198,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591162+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:06:12.591162+0000","last_peered":"2024-04-25T10:06:12.591162+0000","last_clean":"2024-04-25T10:06:12.591162+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:06:12.591162+0000","last_undegraded":"2024-04-25T10:06:12.591162+0000","last_fullsized":"2024-04-25T10:06:12.591162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":118,"acting":118,"num_store_stats":7},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":150789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":152379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":152401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":405504,"data_stored":151543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":393216,"data_stored":150801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":747505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":393216,"data_stored":150857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":946176,"data_stored":722268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:15.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-04-25T10:06:15.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:10.566145+0000', 'last_change': '2024-04-25T10:06:10.566145+0000', 'last_active': '2024-04-25T10:06:10.566145+0000', 'last_peered': '2024-04-25T10:06:10.566145+0000', 'last_clean': '2024-04-25T10:06:10.566145+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:10.566145+0000', 'last_undegraded': '2024-04-25T10:06:10.566145+0000', 'last_fullsized': '2024-04-25T10:06:10.566145+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:10.566145+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:10.566145+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:10.566145+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:15.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-04-25T10:06:15.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:10.566145+0000', 'last_change': '2024-04-25T10:06:10.566145+0000', 'last_active': '2024-04-25T10:06:10.566145+0000', 'last_peered': '2024-04-25T10:06:10.566145+0000', 'last_clean': '2024-04-25T10:06:10.566145+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:10.566145+0000', 'last_undegraded': '2024-04-25T10:06:10.566145+0000', 'last_fullsized': '2024-04-25T10:06:10.566145+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:10.566145+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:10.566145+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:10.566145+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:15.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-04-25T10:06:15.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:10.566145+0000', 'last_change': '2024-04-25T10:06:10.566145+0000', 'last_active': '2024-04-25T10:06:10.566145+0000', 'last_peered': '2024-04-25T10:06:10.566145+0000', 'last_clean': '2024-04-25T10:06:10.566145+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:10.566145+0000', 'last_undegraded': '2024-04-25T10:06:10.566145+0000', 'last_fullsized': '2024-04-25T10:06:10.566145+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:10.566145+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:10.566145+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:10.566145+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:15.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2024-04-25T10:06:15.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:06:09.571413+0000', 'last_change': '2024-04-25T10:06:09.571413+0000', 'last_active': '2024-04-25T10:06:09.563666+0000', 'last_peered': '2024-04-25T10:06:09.563666+0000', 'last_clean': '2024-04-25T10:06:09.563666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:09.571413+0000', 'last_undegraded': '2024-04-25T10:06:09.571413+0000', 'last_fullsized': '2024-04-25T10:06:09.571413+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:09.563666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:09.563666+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:09.563666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:06:15.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2024-04-25T10:06:15.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:06:09.571772+0000', 'last_change': '2024-04-25T10:06:09.571772+0000', 'last_active': '2024-04-25T10:06:09.563666+0000', 'last_peered': '2024-04-25T10:06:09.563666+0000', 'last_clean': '2024-04-25T10:06:09.563666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:09.571772+0000', 'last_undegraded': '2024-04-25T10:06:09.571772+0000', 'last_fullsized': '2024-04-25T10:06:09.571772+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:09.563666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:09.563666+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:09.563666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:06:15.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-25T10:06:15.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:10.566145+0000', 'last_change': '2024-04-25T10:06:10.566145+0000', 'last_active': '2024-04-25T10:06:10.566145+0000', 'last_peered': '2024-04-25T10:06:10.566145+0000', 'last_clean': '2024-04-25T10:06:10.566145+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:10.566145+0000', 'last_undegraded': '2024-04-25T10:06:10.566145+0000', 'last_fullsized': '2024-04-25T10:06:10.566145+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:10.566145+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:10.566145+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:10.566145+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:15.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2024-04-25T10:06:15.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:06:09.571392+0000', 'last_change': '2024-04-25T10:06:09.571392+0000', 'last_active': '2024-04-25T10:06:09.563666+0000', 'last_peered': '2024-04-25T10:06:09.563666+0000', 'last_clean': '2024-04-25T10:06:09.563666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:09.571392+0000', 'last_undegraded': '2024-04-25T10:06:09.571392+0000', 'last_fullsized': '2024-04-25T10:06:09.571392+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:09.563666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:09.563666+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:09.563666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:06:15.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-25T10:06:15.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:10.566145+0000', 'last_change': '2024-04-25T10:06:10.566145+0000', 'last_active': '2024-04-25T10:06:10.566145+0000', 'last_peered': '2024-04-25T10:06:10.566145+0000', 'last_clean': '2024-04-25T10:06:10.566145+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:10.566145+0000', 'last_undegraded': '2024-04-25T10:06:10.566145+0000', 'last_fullsized': '2024-04-25T10:06:10.566145+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:10.566145+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:10.566145+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:10.566145+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:15.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2024-04-25T10:06:15.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:06:09.571309+0000', 'last_change': '2024-04-25T10:06:09.571309+0000', 'last_active': '2024-04-25T10:06:09.563666+0000', 'last_peered': '2024-04-25T10:06:09.563666+0000', 'last_clean': '2024-04-25T10:06:09.563666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:09.571309+0000', 'last_undegraded': '2024-04-25T10:06:09.571309+0000', 'last_fullsized': '2024-04-25T10:06:09.571309+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:09.563666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:09.563666+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:09.563666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:06:15.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:06:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.418+0000 7fa85f0b3640 1 Processor -- start 2024-04-25T10:06:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.418+0000 7fa85f0b3640 1 -- start start 2024-04-25T10:06:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.418+0000 7fa85f0b3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa8580d35b0 0x7fa8580b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.418+0000 7fa85f0b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8580b5990 0x7fa8580b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.418+0000 7fa85f0b3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85815e0c0 con 0x7fa85815dce0 2024-04-25T10:06:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.418+0000 7fa85f0b3640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8580d2510 con 0x7fa8580d35b0 2024-04-25T10:06:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.418+0000 7fa85f0b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580d27f0 con 0x7fa8580b5990 2024-04-25T10:06:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.418+0000 7fa85ce4f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa85815dce0 0x7fa8580d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56534/0 (socket says 172.21.15.16:56534) 2024-04-25T10:06:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa85ce4f640 1 -- 172.21.15.16:0/3310695595 learned_addr learned my addr 172.21.15.16:0/3310695595 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa857fff640 1 --2- 172.21.15.16:0/3310695595 >> v2:172.21.15.186:3300/0 conn(0x7fa8580d35b0 0x7fa8580b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa8577fe640 1 -- 172.21.15.16:0/3310695595 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076206290 0 0) 0x7fa85815e0c0 con 0x7fa85815dce0 2024-04-25T10:06:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa85d650640 1 --2- 172.21.15.16:0/3310695595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8580b5990 0x7fa8580b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa8577fe640 1 -- 172.21.15.16:0/3310695595 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa848003520 con 0x7fa85815dce0 2024-04-25T10:06:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa8577fe640 1 -- 172.21.15.16:0/3310695595 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1893749775 0 0) 0x7fa848003520 con 0x7fa85815dce0 2024-04-25T10:06:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa8577fe640 1 -- 172.21.15.16:0/3310695595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8580b5990 msgr2=0x7fa8580b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa8577fe640 1 --2- 172.21.15.16:0/3310695595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8580b5990 0x7fa8580b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa8577fe640 1 -- 172.21.15.16:0/3310695595 >> v2:172.21.15.186:3300/0 conn(0x7fa8580d35b0 msgr2=0x7fa8580b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa8577fe640 1 --2- 172.21.15.16:0/3310695595 >> v2:172.21.15.186:3300/0 conn(0x7fa8580d35b0 0x7fa8580b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa8577fe640 1 -- 172.21.15.16:0/3310695595 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8580d2ad0 con 0x7fa85815dce0 2024-04-25T10:06:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa8577fe640 1 -- 172.21.15.16:0/3310695595 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fa84c001670 con 0x7fa85815dce0 2024-04-25T10:06:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa8577fe640 1 -- 172.21.15.16:0/3310695595 >> v1:172.21.15.16:6789/0 conn(0x7fa85815dce0 legacy=0x7fa8580d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa8577fe640 1 --2- 172.21.15.16:0/3310695595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8580b5990 0x7fa8480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa8577fe640 1 --2- 172.21.15.16:0/3310695595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa848003f30 0x7fa848004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa8577fe640 1 --2- 172.21.15.16:0/3310695595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa848004d40 0x7fa8480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa8577fe640 1 -- 172.21.15.16:0/3310695595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580d2510 con 0x7fa848004d40 2024-04-25T10:06:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa8577fe640 1 -- 172.21.15.16:0/3310695595 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580d27f0 con 0x7fa848003f30 2024-04-25T10:06:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa8577fe640 1 -- 172.21.15.16:0/3310695595 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84800b9c0 con 0x7fa8580b5990 2024-04-25T10:06:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa85d650640 1 --2- 172.21.15.16:0/3310695595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa848004d40 0x7fa8480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa857fff640 1 --2- 172.21.15.16:0/3310695595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa848003f30 0x7fa848004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.419+0000 7fa85ce4f640 1 --2- 172.21.15.16:0/3310695595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8580b5990 0x7fa8480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.420+0000 7fa85d650640 1 -- 172.21.15.16:0/3310695595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8580b5990 msgr2=0x7fa8480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.420+0000 7fa85d650640 1 --2- 172.21.15.16:0/3310695595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8580b5990 0x7fa8480039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.420+0000 7fa85d650640 1 -- 172.21.15.16:0/3310695595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa848003f30 msgr2=0x7fa848004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.420+0000 7fa85d650640 1 --2- 172.21.15.16:0/3310695595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa848003f30 0x7fa848004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.420+0000 7fa85d650640 1 -- 172.21.15.16:0/3310695595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa848003520 con 0x7fa848004d40 2024-04-25T10:06:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.420+0000 7fa85d650640 1 --2- 172.21.15.16:0/3310695595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa848004d40 0x7fa8480051c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa850009fc0 tx=0x7fa848005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60342384fd40bcb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.420+0000 7fa8577fe640 1 -- 172.21.15.16:0/3310695595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850009c10 con 0x7fa848004d40 2024-04-25T10:06:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.420+0000 7fa8577fe640 1 -- 172.21.15.16:0/3310695595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa850036070 con 0x7fa848004d40 2024-04-25T10:06:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.420+0000 7fa8577fe640 1 -- 172.21.15.16:0/3310695595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850031670 con 0x7fa848004d40 2024-04-25T10:06:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.421+0000 7fa85f0b3640 1 -- 172.21.15.16:0/3310695595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa848004d40 msgr2=0x7fa8480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.421+0000 7fa85f0b3640 1 --2- 172.21.15.16:0/3310695595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa848004d40 0x7fa8480051c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa850009fc0 tx=0x7fa848005830 comp rx=0 tx=0).stop 2024-04-25T10:06:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.421+0000 7fa85d650640 1 -- 172.21.15.16:0/3310695595 reap_dead start 2024-04-25T10:06:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.421+0000 7fa85f0b3640 1 -- 172.21.15.16:0/3310695595 shutdown_connections 2024-04-25T10:06:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.421+0000 7fa85f0b3640 1 -- 172.21.15.16:0/3310695595 >> 172.21.15.16:0/3310695595 conn(0x7fa858072900 msgr2=0x7fa8580d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.421+0000 7fa85f0b3640 1 -- 172.21.15.16:0/3310695595 shutdown_connections 2024-04-25T10:06:15.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.421+0000 7fa85f0b3640 1 -- 172.21.15.16:0/3310695595 wait complete. 2024-04-25T10:06:15.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.422+0000 7fa85f0b3640 1 Processor -- start 2024-04-25T10:06:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.422+0000 7fa85f0b3640 1 -- start start 2024-04-25T10:06:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.423+0000 7fa85f0b3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa848004d40 0x7fa8581f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.423+0000 7fa85f0b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa858153bf0 0x7fa858154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.423+0000 7fa85f0b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8580cbaf0 0x7fa8580cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.423+0000 7fa85f0b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580d2160 con 0x7fa858153bf0 2024-04-25T10:06:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.423+0000 7fa85f0b3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580d3090 con 0x7fa848004d40 2024-04-25T10:06:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.423+0000 7fa85f0b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580cc4b0 con 0x7fa8580cbaf0 2024-04-25T10:06:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.423+0000 7fa857fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa858153bf0 0x7fa858154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.423+0000 7fa857fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa858153bf0 0x7fa858154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49530/0 (socket says 172.21.15.16:49530) 2024-04-25T10:06:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.423+0000 7fa857fff640 1 -- 172.21.15.16:0/940584296 learned_addr learned my addr 172.21.15.16:0/940584296 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.423+0000 7fa85ce4f640 1 --2- 172.21.15.16:0/940584296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa848004d40 0x7fa8581f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.423+0000 7fa85d650640 1 --2- 172.21.15.16:0/940584296 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8580cbaf0 0x7fa8580cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.424+0000 7fa857fff640 1 -- 172.21.15.16:0/940584296 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8580cbaf0 msgr2=0x7fa8580cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.424+0000 7fa857fff640 1 --2- 172.21.15.16:0/940584296 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8580cbaf0 0x7fa8580cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.424+0000 7fa857fff640 1 -- 172.21.15.16:0/940584296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa848004d40 msgr2=0x7fa8581f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.424+0000 7fa857fff640 1 --2- 172.21.15.16:0/940584296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa848004d40 0x7fa8581f67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.424+0000 7fa857fff640 1 -- 172.21.15.16:0/940584296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8581534f0 con 0x7fa858153bf0 2024-04-25T10:06:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.424+0000 7fa857fff640 1 --2- 172.21.15.16:0/940584296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa858153bf0 0x7fa858154070 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa844011960 tx=0x7fa844011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.424+0000 7fa8557fa640 1 -- 172.21.15.16:0/940584296 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844019070 con 0x7fa858153bf0 2024-04-25T10:06:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.424+0000 7fa8557fa640 1 -- 172.21.15.16:0/940584296 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa844004240 con 0x7fa858153bf0 2024-04-25T10:06:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.424+0000 7fa85f0b3640 1 -- 172.21.15.16:0/940584296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8581f6cf0 con 0x7fa858153bf0 2024-04-25T10:06:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.424+0000 7fa8557fa640 1 -- 172.21.15.16:0/940584296 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84400f880 con 0x7fa858153bf0 2024-04-25T10:06:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.424+0000 7fa85f0b3640 1 -- 172.21.15.16:0/940584296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8580cc9b0 con 0x7fa858153bf0 2024-04-25T10:06:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.425+0000 7fa85f0b3640 1 -- 172.21.15.16:0/940584296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8580749b0 con 0x7fa858153bf0 2024-04-25T10:06:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.426+0000 7fa8557fa640 1 -- 172.21.15.16:0/940584296 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa84401c070 con 0x7fa858153bf0 2024-04-25T10:06:15.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.426+0000 7fa8557fa640 1 --2- 172.21.15.16:0/940584296 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa83c06b4d0 0x7fa83c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.426+0000 7fa8557fa640 1 -- 172.21.15.16:0/940584296 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa84408a600 con 0x7fa858153bf0 2024-04-25T10:06:15.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.426+0000 7fa85ce4f640 1 --2- 172.21.15.16:0/940584296 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa83c06b4d0 0x7fa83c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.427+0000 7fa85ce4f640 1 --2- 172.21.15.16:0/940584296 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa83c06b4d0 0x7fa83c06d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa84c001050 tx=0x7fa84c007970 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.430+0000 7fa8557fa640 1 -- 172.21.15.16:0/940584296 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa8581534f0 con 0x7fa858153bf0 2024-04-25T10:06:15.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.680+0000 7fa85f0b3640 1 -- 172.21.15.16:0/940584296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa8580d46a0 con 0x7fa858153bf0 2024-04-25T10:06:15.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.681+0000 7fa8557fa640 1 -- 172.21.15.16:0/940584296 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1724 (secure 0 0 0) 0x7fa844058e30 con 0x7fa858153bf0 2024-04-25T10:06:15.686 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:15.687 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":89,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039497,"num_in_osds":8,"osd_in_since":1714039488,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":88},{"state_name":"unknown","count":5},{"state_name":"creating+peering","count":4}],"num_pgs":97,"num_pools":3,"num_objects":22,"data_bytes":592778,"bytes_used":242733056,"bytes_avail":772851380224,"bytes_total":773094113280,"unknown_pgs_ratio":0.051546391099691391,"inactive_pgs_ratio":0.041237112134695053,"write_bytes_sec":3390,"read_op_per_sec":0,"write_op_per_sec":10},"fsmap":{"epoch":14,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":34613}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-04-25T10:05:59.793070+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:06:15.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.683+0000 7fa85f0b3640 1 -- 172.21.15.16:0/940584296 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa83c06b4d0 msgr2=0x7fa83c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.683+0000 7fa85f0b3640 1 --2- 172.21.15.16:0/940584296 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa83c06b4d0 0x7fa83c06d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa84c001050 tx=0x7fa84c007970 comp rx=0 tx=0).stop 2024-04-25T10:06:15.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.683+0000 7fa85f0b3640 1 -- 172.21.15.16:0/940584296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa858153bf0 msgr2=0x7fa858154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.683+0000 7fa85f0b3640 1 --2- 172.21.15.16:0/940584296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa858153bf0 0x7fa858154070 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa844011960 tx=0x7fa844011e20 comp rx=0 tx=0).stop 2024-04-25T10:06:15.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.683+0000 7fa85f0b3640 1 -- 172.21.15.16:0/940584296 shutdown_connections 2024-04-25T10:06:15.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.683+0000 7fa85f0b3640 1 --2- 172.21.15.16:0/940584296 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa83c06b4d0 0x7fa83c06d990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.683+0000 7fa85f0b3640 1 --2- 172.21.15.16:0/940584296 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8580cbaf0 0x7fa8580cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.683+0000 7fa85f0b3640 1 --2- 172.21.15.16:0/940584296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa858153bf0 0x7fa858154070 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.683+0000 7fa85f0b3640 1 --2- 172.21.15.16:0/940584296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa848004d40 0x7fa8581f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.683+0000 7fa85f0b3640 1 -- 172.21.15.16:0/940584296 >> 172.21.15.16:0/940584296 conn(0x7fa858072900 msgr2=0x7fa8580b8690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.683+0000 7fa85f0b3640 1 -- 172.21.15.16:0/940584296 shutdown_connections 2024-04-25T10:06:15.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.683+0000 7fa85f0b3640 1 -- 172.21.15.16:0/940584296 wait complete. 2024-04-25T10:06:15.699 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:06:15.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:15.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.795+0000 7f992139b640 1 Processor -- start 2024-04-25T10:06:15.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.795+0000 7f992139b640 1 -- start start 2024-04-25T10:06:15.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.795+0000 7f992139b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f991c06c2d0 0x7f991c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.795+0000 7f992139b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c0acd80 0x7f991c0ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.796+0000 7f992139b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991c06dc70 con 0x7f991c06bef0 2024-04-25T10:06:15.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.796+0000 7f992139b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991c06eb20 con 0x7f991c06c2d0 2024-04-25T10:06:15.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.796+0000 7f992139b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c06df10 con 0x7f991c0acd80 2024-04-25T10:06:15.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.796+0000 7f991b7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c0acd80 0x7f991c0ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.796+0000 7f9913fff640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f991c06c2d0 0x7f991c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.796+0000 7f991affd640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f991c06bef0 0x7f991c15c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:36924/0 (socket says 172.21.15.16:36924) 2024-04-25T10:06:15.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.796+0000 7f9913fff640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f991c06c2d0 0x7f991c06efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:51336/0 (socket says 172.21.15.16:51336) 2024-04-25T10:06:15.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.796+0000 7f991b7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c0acd80 0x7f991c0ad210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:42448/0 (socket says 172.21.15.16:42448) 2024-04-25T10:06:15.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.796+0000 7f991affd640 1 -- 172.21.15.16:0/360454608 learned_addr learned my addr 172.21.15.16:0/360454608 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:15.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.796+0000 7f991b7fe640 1 -- 172.21.15.16:0/360454608 >> v2:172.21.15.186:3300/0 conn(0x7f991c06c2d0 msgr2=0x7f991c06efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.796+0000 7f991b7fe640 1 --2- 172.21.15.16:0/360454608 >> v2:172.21.15.186:3300/0 conn(0x7f991c06c2d0 0x7f991c06efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.796+0000 7f991b7fe640 1 -- 172.21.15.16:0/360454608 >> v1:172.21.15.186:6789/0 conn(0x7f991c06bef0 legacy=0x7f991c15c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.796+0000 7f991b7fe640 1 -- 172.21.15.16:0/360454608 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c06e1f0 con 0x7f991c0acd80 2024-04-25T10:06:15.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.796+0000 7f9913fff640 1 --2- 172.21.15.16:0/360454608 >> v2:172.21.15.186:3300/0 conn(0x7f991c06c2d0 0x7f991c06efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:15.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.797+0000 7f991b7fe640 1 --2- 172.21.15.16:0/360454608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c0acd80 0x7f991c0ad210 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f991c06eca0 tx=0x7f991402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5f17c67ca7a47f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.797+0000 7f991a7fc640 1 -- 172.21.15.16:0/360454608 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991402ae60 con 0x7f991c0acd80 2024-04-25T10:06:15.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.797+0000 7f991a7fc640 1 -- 172.21.15.16:0/360454608 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9914007740 con 0x7f991c0acd80 2024-04-25T10:06:15.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.797+0000 7f991a7fc640 1 -- 172.21.15.16:0/360454608 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914007b10 con 0x7f991c0acd80 2024-04-25T10:06:15.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.798+0000 7f992139b640 1 -- 172.21.15.16:0/360454608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c0acd80 msgr2=0x7f991c0ad210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.798+0000 7f992139b640 1 --2- 172.21.15.16:0/360454608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c0acd80 0x7f991c0ad210 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f991c06eca0 tx=0x7f991402a640 comp rx=0 tx=0).stop 2024-04-25T10:06:15.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.798+0000 7f992139b640 1 -- 172.21.15.16:0/360454608 shutdown_connections 2024-04-25T10:06:15.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.798+0000 7f992139b640 1 --2- 172.21.15.16:0/360454608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c0acd80 0x7f991c0ad210 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.798+0000 7f992139b640 1 --2- 172.21.15.16:0/360454608 >> v2:172.21.15.186:3300/0 conn(0x7f991c06c2d0 0x7f991c06efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.798+0000 7f992139b640 1 -- 172.21.15.16:0/360454608 >> 172.21.15.16:0/360454608 conn(0x7f991c0b2220 msgr2=0x7f991c0cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.798+0000 7f992139b640 1 -- 172.21.15.16:0/360454608 shutdown_connections 2024-04-25T10:06:15.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.798+0000 7f992139b640 1 -- 172.21.15.16:0/360454608 wait complete. 2024-04-25T10:06:15.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.799+0000 7f992139b640 1 Processor -- start 2024-04-25T10:06:15.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.799+0000 7f992139b640 1 -- start start 2024-04-25T10:06:15.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.800+0000 7f992139b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f991c06bef0 0x7f991c146940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.800+0000 7f992139b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c06c2d0 0x7f991c157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.800+0000 7f992139b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f991c0acd80 0x7f991c09c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.800+0000 7f992139b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0ada00 con 0x7f991c0acd80 2024-04-25T10:06:15.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.800+0000 7f992139b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c06e7b0 con 0x7f991c06bef0 2024-04-25T10:06:15.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.800+0000 7f992139b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0adb80 con 0x7f991c06c2d0 2024-04-25T10:06:15.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.800+0000 7f991b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f991c0acd80 0x7f991c09c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.800+0000 7f991b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f991c0acd80 0x7f991c09c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49540/0 (socket says 172.21.15.16:49540) 2024-04-25T10:06:15.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.800+0000 7f991b7fe640 1 -- 172.21.15.16:0/1365906741 learned_addr learned my addr 172.21.15.16:0/1365906741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:15.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.800+0000 7f991affd640 1 --2- 172.21.15.16:0/1365906741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f991c06bef0 0x7f991c146940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.800+0000 7f9913fff640 1 --2- 172.21.15.16:0/1365906741 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c06c2d0 0x7f991c157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.800+0000 7f991b7fe640 1 -- 172.21.15.16:0/1365906741 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c06c2d0 msgr2=0x7f991c157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.800+0000 7f991b7fe640 1 --2- 172.21.15.16:0/1365906741 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c06c2d0 0x7f991c157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.800+0000 7f991b7fe640 1 -- 172.21.15.16:0/1365906741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f991c06bef0 msgr2=0x7f991c146940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.800+0000 7f991b7fe640 1 --2- 172.21.15.16:0/1365906741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f991c06bef0 0x7f991c146940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.800+0000 7f991b7fe640 1 -- 172.21.15.16:0/1365906741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c146e80 con 0x7f991c0acd80 2024-04-25T10:06:15.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.801+0000 7f991b7fe640 1 --2- 172.21.15.16:0/1365906741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f991c0acd80 0x7f991c09c430 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9914009e50 tx=0x7f9914004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.801+0000 7f99137fe640 1 -- 172.21.15.16:0/1365906741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914040070 con 0x7f991c0acd80 2024-04-25T10:06:15.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.801+0000 7f99137fe640 1 -- 172.21.15.16:0/1365906741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991402ae60 con 0x7f991c0acd80 2024-04-25T10:06:15.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.801+0000 7f992139b640 1 -- 172.21.15.16:0/1365906741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991c158990 con 0x7f991c0acd80 2024-04-25T10:06:15.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.801+0000 7f99137fe640 1 -- 172.21.15.16:0/1365906741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914031c90 con 0x7f991c0acd80 2024-04-25T10:06:15.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.801+0000 7f992139b640 1 -- 172.21.15.16:0/1365906741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991c099d00 con 0x7f991c0acd80 2024-04-25T10:06:15.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.802+0000 7f992139b640 1 -- 172.21.15.16:0/1365906741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991c0749b0 con 0x7f991c0acd80 2024-04-25T10:06:15.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.802+0000 7f99137fe640 1 -- 172.21.15.16:0/1365906741 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9914039040 con 0x7f991c0acd80 2024-04-25T10:06:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.803+0000 7f99137fe640 1 --2- 172.21.15.16:0/1365906741 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f98fc06b4d0 0x7f98fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.803+0000 7f99137fe640 1 -- 172.21.15.16:0/1365906741 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99140ab4b0 con 0x7f991c0acd80 2024-04-25T10:06:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.803+0000 7f991affd640 1 --2- 172.21.15.16:0/1365906741 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f98fc06b4d0 0x7f98fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.803+0000 7f991affd640 1 --2- 172.21.15.16:0/1365906741 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f98fc06b4d0 0x7f98fc06d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f990c000f60 tx=0x7f990c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.807+0000 7f99137fe640 1 -- 172.21.15.16:0/1365906741 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f991403a3b0 con 0x7f991c0acd80 2024-04-25T10:06:15.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.986+0000 7f992139b640 1 -- 172.21.15.16:0/1365906741 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f991c0d0b70 con 0x7f98fc06b4d0 2024-04-25T10:06:15.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.992+0000 7f99137fe640 1 -- 172.21.15.16:0/1365906741 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251919 (secure 0 0 0) 0x7f991c0d0b70 con 0x7f98fc06b4d0 2024-04-25T10:06:16.000 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:16.001 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:06:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.995+0000 7f992139b640 1 -- 172.21.15.16:0/1365906741 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f98fc06b4d0 msgr2=0x7f98fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.995+0000 7f992139b640 1 --2- 172.21.15.16:0/1365906741 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f98fc06b4d0 0x7f98fc06d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f990c000f60 tx=0x7f990c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.995+0000 7f992139b640 1 -- 172.21.15.16:0/1365906741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f991c0acd80 msgr2=0x7f991c09c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.995+0000 7f992139b640 1 --2- 172.21.15.16:0/1365906741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f991c0acd80 0x7f991c09c430 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9914009e50 tx=0x7f9914004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.995+0000 7f992139b640 1 -- 172.21.15.16:0/1365906741 shutdown_connections 2024-04-25T10:06:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.995+0000 7f992139b640 1 --2- 172.21.15.16:0/1365906741 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f98fc06b4d0 0x7f98fc06d990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.995+0000 7f992139b640 1 --2- 172.21.15.16:0/1365906741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f991c0acd80 0x7f991c09c430 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.995+0000 7f992139b640 1 --2- 172.21.15.16:0/1365906741 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c06c2d0 0x7f991c157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.995+0000 7f992139b640 1 --2- 172.21.15.16:0/1365906741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f991c06bef0 0x7f991c146940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.995+0000 7f992139b640 1 -- 172.21.15.16:0/1365906741 >> 172.21.15.16:0/1365906741 conn(0x7f991c0b2220 msgr2=0x7f991c155940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:16.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.995+0000 7f992139b640 1 -- 172.21.15.16:0/1365906741 shutdown_connections 2024-04-25T10:06:16.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:15.995+0000 7f992139b640 1 -- 172.21.15.16:0/1365906741 wait complete. 2024-04-25T10:06:16.011 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-04-25T10:06:15.795843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237096,"kb_used_data":4288,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737624,"statfs":{"total":773094113280,"available":772851326976,"internally_reserved":0,"allocated":4390912,"data_stored":2404292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001076"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590900+0000","last_change":"2024-04-25T10:06:11.596197+0000","last_active":"2024-04-25T10:06:12.590900+0000","last_peered":"2024-04-25T10:06:12.590900+0000","last_clean":"2024-04-25T10:06:12.590900+0000","last_became_active":"2024-04-25T10:06:11.595896+0000","last_became_peered":"2024-04-25T10:06:11.595896+0000","last_unstale":"2024-04-25T10:06:12.590900+0000","last_undegraded":"2024-04-25T10:06:12.590900+0000","last_fullsized":"2024-04-25T10:06:12.590900+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:50.014644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:13.033669+0000","last_change":"2024-04-25T10:06:11.612418+0000","last_active":"2024-04-25T10:06:13.033669+0000","last_peered":"2024-04-25T10:06:13.033669+0000","last_clean":"2024-04-25T10:06:13.033669+0000","last_became_active":"2024-04-25T10:06:11.612084+0000","last_became_peered":"2024-04-25T10:06:11.612084+0000","last_unstale":"2024-04-25T10:06:13.033669+0000","last_undegraded":"2024-04-25T10:06:13.033669+0000","last_fullsized":"2024-04-25T10:06:13.033669+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:52.424313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591774+0000","last_change":"2024-04-25T10:06:11.588804+0000","last_active":"2024-04-25T10:06:12.591774+0000","last_peered":"2024-04-25T10:06:12.591774+0000","last_clean":"2024-04-25T10:06:12.591774+0000","last_became_active":"2024-04-25T10:06:11.588533+0000","last_became_peered":"2024-04-25T10:06:11.588533+0000","last_unstale":"2024-04-25T10:06:12.591774+0000","last_undegraded":"2024-04-25T10:06:12.591774+0000","last_fullsized":"2024-04-25T10:06:12.591774+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:12.136446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610805+0000","last_change":"2024-04-25T10:06:11.603412+0000","last_active":"2024-04-25T10:06:11.610805+0000","last_peered":"2024-04-25T10:06:11.610805+0000","last_clean":"2024-04-25T10:06:11.610805+0000","last_became_active":"2024-04-25T10:06:11.602936+0000","last_became_peered":"2024-04-25T10:06:11.602936+0000","last_unstale":"2024-04-25T10:06:11.610805+0000","last_undegraded":"2024-04-25T10:06:11.610805+0000","last_fullsized":"2024-04-25T10:06:11.610805+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.403695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:13.033805+0000","last_change":"2024-04-25T10:06:11.593781+0000","last_active":"2024-04-25T10:06:13.033805+0000","last_peered":"2024-04-25T10:06:13.033805+0000","last_clean":"2024-04-25T10:06:13.033805+0000","last_became_active":"2024-04-25T10:06:11.593453+0000","last_became_peered":"2024-04-25T10:06:11.593453+0000","last_unstale":"2024-04-25T10:06:13.033805+0000","last_undegraded":"2024-04-25T10:06:13.033805+0000","last_fullsized":"2024-04-25T10:06:13.033805+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:12.965594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599412+0000","last_change":"2024-04-25T10:06:11.594811+0000","last_active":"2024-04-25T10:06:11.599412+0000","last_peered":"2024-04-25T10:06:11.599412+0000","last_clean":"2024-04-25T10:06:11.599412+0000","last_became_active":"2024-04-25T10:06:11.594494+0000","last_became_peered":"2024-04-25T10:06:11.594494+0000","last_unstale":"2024-04-25T10:06:11.599412+0000","last_undegraded":"2024-04-25T10:06:11.599412+0000","last_fullsized":"2024-04-25T10:06:11.599412+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:02.231151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590302+0000","last_change":"2024-04-25T10:06:11.585486+0000","last_active":"2024-04-25T10:06:12.590302+0000","last_peered":"2024-04-25T10:06:12.590302+0000","last_clean":"2024-04-25T10:06:12.590302+0000","last_became_active":"2024-04-25T10:06:11.585167+0000","last_became_peered":"2024-04-25T10:06:11.585167+0000","last_unstale":"2024-04-25T10:06:12.590302+0000","last_undegraded":"2024-04-25T10:06:12.590302+0000","last_fullsized":"2024-04-25T10:06:12.590302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590232+0000","last_change":"2024-04-25T10:06:11.599766+0000","last_active":"2024-04-25T10:06:12.590232+0000","last_peered":"2024-04-25T10:06:12.590232+0000","last_clean":"2024-04-25T10:06:12.590232+0000","last_became_active":"2024-04-25T10:06:11.599473+0000","last_became_peered":"2024-04-25T10:06:11.599473+0000","last_unstale":"2024-04-25T10:06:12.590232+0000","last_undegraded":"2024-04-25T10:06:12.590232+0000","last_fullsized":"2024-04-25T10:06:12.590232+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:46.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591773+0000","last_change":"2024-04-25T10:06:11.601172+0000","last_active":"2024-04-25T10:06:12.591773+0000","last_peered":"2024-04-25T10:06:12.591773+0000","last_clean":"2024-04-25T10:06:12.591773+0000","last_became_active":"2024-04-25T10:06:11.600890+0000","last_became_peered":"2024-04-25T10:06:11.600890+0000","last_unstale":"2024-04-25T10:06:12.591773+0000","last_undegraded":"2024-04-25T10:06:12.591773+0000","last_fullsized":"2024-04-25T10:06:12.591773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:33.055338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.614185+0000","last_change":"2024-04-25T10:06:11.611336+0000","last_active":"2024-04-25T10:06:11.614185+0000","last_peered":"2024-04-25T10:06:11.614185+0000","last_clean":"2024-04-25T10:06:11.614185+0000","last_became_active":"2024-04-25T10:06:11.610883+0000","last_became_peered":"2024-04-25T10:06:11.610883+0000","last_unstale":"2024-04-25T10:06:11.614185+0000","last_undegraded":"2024-04-25T10:06:11.614185+0000","last_fullsized":"2024-04-25T10:06:11.614185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:50.014644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.606907+0000","last_change":"2024-04-25T10:06:11.606133+0000","last_active":"2024-04-25T10:06:11.606907+0000","last_peered":"2024-04-25T10:06:11.606907+0000","last_clean":"2024-04-25T10:06:11.606907+0000","last_became_active":"2024-04-25T10:06:11.605855+0000","last_became_peered":"2024-04-25T10:06:11.605855+0000","last_unstale":"2024-04-25T10:06:11.606907+0000","last_undegraded":"2024-04-25T10:06:11.606907+0000","last_fullsized":"2024-04-25T10:06:11.606907+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:35.491727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591678+0000","last_change":"2024-04-25T10:06:11.598791+0000","last_active":"2024-04-25T10:06:12.591678+0000","last_peered":"2024-04-25T10:06:12.591678+0000","last_clean":"2024-04-25T10:06:12.591678+0000","last_became_active":"2024-04-25T10:06:11.598511+0000","last_became_peered":"2024-04-25T10:06:11.598511+0000","last_unstale":"2024-04-25T10:06:12.591678+0000","last_undegraded":"2024-04-25T10:06:12.591678+0000","last_fullsized":"2024-04-25T10:06:12.591678+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:18.899344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592055+0000","last_change":"2024-04-25T10:06:11.602586+0000","last_active":"2024-04-25T10:06:12.592055+0000","last_peered":"2024-04-25T10:06:12.592055+0000","last_clean":"2024-04-25T10:06:12.592055+0000","last_became_active":"2024-04-25T10:06:11.602219+0000","last_became_peered":"2024-04-25T10:06:11.602219+0000","last_unstale":"2024-04-25T10:06:12.592055+0000","last_undegraded":"2024-04-25T10:06:12.592055+0000","last_fullsized":"2024-04-25T10:06:12.592055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610597+0000","last_change":"2024-04-25T10:06:11.599901+0000","last_active":"2024-04-25T10:06:11.610597+0000","last_peered":"2024-04-25T10:06:11.610597+0000","last_clean":"2024-04-25T10:06:11.610597+0000","last_became_active":"2024-04-25T10:06:11.599632+0000","last_became_peered":"2024-04-25T10:06:11.599632+0000","last_unstale":"2024-04-25T10:06:11.610597+0000","last_undegraded":"2024-04-25T10:06:11.610597+0000","last_fullsized":"2024-04-25T10:06:11.610597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:46.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.592021+0000","last_change":"2024-04-25T10:06:11.587570+0000","last_active":"2024-04-25T10:06:11.592021+0000","last_peered":"2024-04-25T10:06:11.592021+0000","last_clean":"2024-04-25T10:06:11.592021+0000","last_became_active":"2024-04-25T10:06:11.587301+0000","last_became_peered":"2024-04-25T10:06:11.587301+0000","last_unstale":"2024-04-25T10:06:11.592021+0000","last_undegraded":"2024-04-25T10:06:11.592021+0000","last_fullsized":"2024-04-25T10:06:11.592021+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.597882+0000","last_change":"2024-04-25T10:06:11.597081+0000","last_active":"2024-04-25T10:06:11.597882+0000","last_peered":"2024-04-25T10:06:11.597882+0000","last_clean":"2024-04-25T10:06:11.597882+0000","last_became_active":"2024-04-25T10:06:11.596781+0000","last_became_peered":"2024-04-25T10:06:11.596781+0000","last_unstale":"2024-04-25T10:06:11.597882+0000","last_undegraded":"2024-04-25T10:06:11.597882+0000","last_fullsized":"2024-04-25T10:06:11.597882+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:11.365920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.589022+0000","last_change":"2024-04-25T10:06:11.587969+0000","last_active":"2024-04-25T10:06:11.589022+0000","last_peered":"2024-04-25T10:06:11.589022+0000","last_clean":"2024-04-25T10:06:11.589022+0000","last_became_active":"2024-04-25T10:06:11.587084+0000","last_became_peered":"2024-04-25T10:06:11.587084+0000","last_unstale":"2024-04-25T10:06:11.589022+0000","last_undegraded":"2024-04-25T10:06:11.589022+0000","last_fullsized":"2024-04-25T10:06:11.589022+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:47.208707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599620+0000","last_change":"2024-04-25T10:06:11.596669+0000","last_active":"2024-04-25T10:06:11.599620+0000","last_peered":"2024-04-25T10:06:11.599620+0000","last_clean":"2024-04-25T10:06:11.599620+0000","last_became_active":"2024-04-25T10:06:11.596091+0000","last_became_peered":"2024-04-25T10:06:11.596091+0000","last_unstale":"2024-04-25T10:06:11.599620+0000","last_undegraded":"2024-04-25T10:06:11.599620+0000","last_fullsized":"2024-04-25T10:06:11.599620+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:18.899344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599443+0000","last_change":"2024-04-25T10:06:11.594701+0000","last_active":"2024-04-25T10:06:11.599443+0000","last_peered":"2024-04-25T10:06:11.599443+0000","last_clean":"2024-04-25T10:06:11.599443+0000","last_became_active":"2024-04-25T10:06:11.594376+0000","last_became_peered":"2024-04-25T10:06:11.594376+0000","last_unstale":"2024-04-25T10:06:11.599443+0000","last_undegraded":"2024-04-25T10:06:11.599443+0000","last_fullsized":"2024-04-25T10:06:11.599443+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:53.157588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592096+0000","last_change":"2024-04-25T10:06:11.597575+0000","last_active":"2024-04-25T10:06:12.592096+0000","last_peered":"2024-04-25T10:06:12.592096+0000","last_clean":"2024-04-25T10:06:12.592096+0000","last_became_active":"2024-04-25T10:06:11.596710+0000","last_became_peered":"2024-04-25T10:06:11.596710+0000","last_unstale":"2024-04-25T10:06:12.592096+0000","last_undegraded":"2024-04-25T10:06:12.592096+0000","last_fullsized":"2024-04-25T10:06:12.592096+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.595406+0000","last_change":"2024-04-25T10:06:11.594514+0000","last_active":"2024-04-25T10:06:11.595406+0000","last_peered":"2024-04-25T10:06:11.595406+0000","last_clean":"2024-04-25T10:06:11.595406+0000","last_became_active":"2024-04-25T10:06:11.594231+0000","last_became_peered":"2024-04-25T10:06:11.594231+0000","last_unstale":"2024-04-25T10:06:11.595406+0000","last_undegraded":"2024-04-25T10:06:11.595406+0000","last_fullsized":"2024-04-25T10:06:11.595406+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:03.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590474+0000","last_change":"2024-04-25T10:06:11.595941+0000","last_active":"2024-04-25T10:06:12.590474+0000","last_peered":"2024-04-25T10:06:12.590474+0000","last_clean":"2024-04-25T10:06:12.590474+0000","last_became_active":"2024-04-25T10:06:11.595598+0000","last_became_peered":"2024-04-25T10:06:11.595598+0000","last_unstale":"2024-04-25T10:06:12.590474+0000","last_undegraded":"2024-04-25T10:06:12.590474+0000","last_fullsized":"2024-04-25T10:06:12.590474+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:40.079857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591540+0000","last_change":"2024-04-25T10:06:11.599549+0000","last_active":"2024-04-25T10:06:12.591540+0000","last_peered":"2024-04-25T10:06:12.591540+0000","last_clean":"2024-04-25T10:06:12.591540+0000","last_became_active":"2024-04-25T10:06:11.599256+0000","last_became_peered":"2024-04-25T10:06:11.599256+0000","last_unstale":"2024-04-25T10:06:12.591540+0000","last_undegraded":"2024-04-25T10:06:12.591540+0000","last_fullsized":"2024-04-25T10:06:12.591540+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:48.190466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590264+0000","last_change":"2024-04-25T10:06:11.605175+0000","last_active":"2024-04-25T10:06:12.590264+0000","last_peered":"2024-04-25T10:06:12.590264+0000","last_clean":"2024-04-25T10:06:12.590264+0000","last_became_active":"2024-04-25T10:06:11.604773+0000","last_became_peered":"2024-04-25T10:06:11.604773+0000","last_unstale":"2024-04-25T10:06:12.590264+0000","last_undegraded":"2024-04-25T10:06:12.590264+0000","last_fullsized":"2024-04-25T10:06:12.590264+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.403695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611074+0000","last_change":"2024-04-25T10:06:11.605194+0000","last_active":"2024-04-25T10:06:11.611074+0000","last_peered":"2024-04-25T10:06:11.611074+0000","last_clean":"2024-04-25T10:06:11.611074+0000","last_became_active":"2024-04-25T10:06:11.604595+0000","last_became_peered":"2024-04-25T10:06:11.604595+0000","last_unstale":"2024-04-25T10:06:11.611074+0000","last_undegraded":"2024-04-25T10:06:11.611074+0000","last_fullsized":"2024-04-25T10:06:11.611074+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:36.054034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611662+0000","last_change":"2024-04-25T10:06:11.603505+0000","last_active":"2024-04-25T10:06:11.611662+0000","last_peered":"2024-04-25T10:06:11.611662+0000","last_clean":"2024-04-25T10:06:11.611662+0000","last_became_active":"2024-04-25T10:06:11.603215+0000","last_became_peered":"2024-04-25T10:06:11.603215+0000","last_unstale":"2024-04-25T10:06:11.611662+0000","last_undegraded":"2024-04-25T10:06:11.611662+0000","last_fullsized":"2024-04-25T10:06:11.611662+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:13.033711+0000","last_change":"2024-04-25T10:06:11.593363+0000","last_active":"2024-04-25T10:06:13.033711+0000","last_peered":"2024-04-25T10:06:13.033711+0000","last_clean":"2024-04-25T10:06:13.033711+0000","last_became_active":"2024-04-25T10:06:11.593092+0000","last_became_peered":"2024-04-25T10:06:11.593092+0000","last_unstale":"2024-04-25T10:06:13.033711+0000","last_undegraded":"2024-04-25T10:06:13.033711+0000","last_fullsized":"2024-04-25T10:06:13.033711+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:19.088902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591554+0000","last_change":"2024-04-25T10:06:11.601007+0000","last_active":"2024-04-25T10:06:12.591554+0000","last_peered":"2024-04-25T10:06:12.591554+0000","last_clean":"2024-04-25T10:06:12.591554+0000","last_became_active":"2024-04-25T10:06:11.600297+0000","last_became_peered":"2024-04-25T10:06:11.600297+0000","last_unstale":"2024-04-25T10:06:12.591554+0000","last_undegraded":"2024-04-25T10:06:12.591554+0000","last_fullsized":"2024-04-25T10:06:12.591554+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:18.645477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.601405+0000","last_change":"2024-04-25T10:06:11.600447+0000","last_active":"2024-04-25T10:06:11.601405+0000","last_peered":"2024-04-25T10:06:11.601405+0000","last_clean":"2024-04-25T10:06:11.601405+0000","last_became_active":"2024-04-25T10:06:11.600160+0000","last_became_peered":"2024-04-25T10:06:11.600160+0000","last_unstale":"2024-04-25T10:06:11.601405+0000","last_undegraded":"2024-04-25T10:06:11.601405+0000","last_fullsized":"2024-04-25T10:06:11.601405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:50.014644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.591617+0000","last_change":"2024-04-25T10:06:11.590349+0000","last_active":"2024-04-25T10:06:11.591617+0000","last_peered":"2024-04-25T10:06:11.591617+0000","last_clean":"2024-04-25T10:06:11.591617+0000","last_became_active":"2024-04-25T10:06:11.590090+0000","last_became_peered":"2024-04-25T10:06:11.590090+0000","last_unstale":"2024-04-25T10:06:11.591617+0000","last_undegraded":"2024-04-25T10:06:11.591617+0000","last_fullsized":"2024-04-25T10:06:11.591617+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590344+0000","last_change":"2024-04-25T10:06:11.597956+0000","last_active":"2024-04-25T10:06:12.590344+0000","last_peered":"2024-04-25T10:06:12.590344+0000","last_clean":"2024-04-25T10:06:12.590344+0000","last_became_active":"2024-04-25T10:06:11.597687+0000","last_became_peered":"2024-04-25T10:06:11.597687+0000","last_unstale":"2024-04-25T10:06:12.590344+0000","last_undegraded":"2024-04-25T10:06:12.590344+0000","last_fullsized":"2024-04-25T10:06:12.590344+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.586794+0000","last_change":"2024-04-25T10:06:11.584953+0000","last_active":"2024-04-25T10:06:11.586794+0000","last_peered":"2024-04-25T10:06:11.586794+0000","last_clean":"2024-04-25T10:06:11.586794+0000","last_became_active":"2024-04-25T10:06:11.584666+0000","last_became_peered":"2024-04-25T10:06:11.584666+0000","last_unstale":"2024-04-25T10:06:11.586794+0000","last_undegraded":"2024-04-25T10:06:11.586794+0000","last_fullsized":"2024-04-25T10:06:11.586794+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:55.206873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591906+0000","last_change":"2024-04-25T10:06:11.592933+0000","last_active":"2024-04-25T10:06:12.591906+0000","last_peered":"2024-04-25T10:06:12.591906+0000","last_clean":"2024-04-25T10:06:12.591906+0000","last_became_active":"2024-04-25T10:06:11.592335+0000","last_became_peered":"2024-04-25T10:06:11.592335+0000","last_unstale":"2024-04-25T10:06:12.591906+0000","last_undegraded":"2024-04-25T10:06:12.591906+0000","last_fullsized":"2024-04-25T10:06:12.591906+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.403695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.602910+0000","last_change":"2024-04-25T10:06:10.590142+0000","last_active":"2024-04-25T10:06:11.602910+0000","last_peered":"2024-04-25T10:06:11.602910+0000","last_clean":"2024-04-25T10:06:11.602910+0000","last_became_active":"2024-04-25T10:06:10.589851+0000","last_became_peered":"2024-04-25T10:06:10.589851+0000","last_unstale":"2024-04-25T10:06:11.602910+0000","last_undegraded":"2024-04-25T10:06:11.602910+0000","last_fullsized":"2024-04-25T10:06:11.602910+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:50.142763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590901+0000","last_change":"2024-04-25T10:06:11.586712+0000","last_active":"2024-04-25T10:06:12.590901+0000","last_peered":"2024-04-25T10:06:12.590901+0000","last_clean":"2024-04-25T10:06:12.590901+0000","last_became_active":"2024-04-25T10:06:11.586393+0000","last_became_peered":"2024-04-25T10:06:11.586393+0000","last_unstale":"2024-04-25T10:06:12.590901+0000","last_undegraded":"2024-04-25T10:06:12.590901+0000","last_fullsized":"2024-04-25T10:06:12.590901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:11.365920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611691+0000","last_change":"2024-04-25T10:06:10.590906+0000","last_active":"2024-04-25T10:06:11.611691+0000","last_peered":"2024-04-25T10:06:11.611691+0000","last_clean":"2024-04-25T10:06:11.611691+0000","last_became_active":"2024-04-25T10:06:10.590641+0000","last_became_peered":"2024-04-25T10:06:10.590641+0000","last_unstale":"2024-04-25T10:06:11.611691+0000","last_undegraded":"2024-04-25T10:06:11.611691+0000","last_fullsized":"2024-04-25T10:06:11.611691+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:39.077377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591703+0000","last_change":"2024-04-25T10:06:11.601284+0000","last_active":"2024-04-25T10:06:12.591703+0000","last_peered":"2024-04-25T10:06:12.591703+0000","last_clean":"2024-04-25T10:06:12.591703+0000","last_became_active":"2024-04-25T10:06:11.600739+0000","last_became_peered":"2024-04-25T10:06:11.600739+0000","last_unstale":"2024-04-25T10:06:12.591703+0000","last_undegraded":"2024-04-25T10:06:12.591703+0000","last_fullsized":"2024-04-25T10:06:12.591703+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:03.030946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.594679+0000","last_change":"2024-04-25T10:06:10.589990+0000","last_active":"2024-04-25T10:06:11.594679+0000","last_peered":"2024-04-25T10:06:11.594679+0000","last_clean":"2024-04-25T10:06:11.594679+0000","last_became_active":"2024-04-25T10:06:10.589663+0000","last_became_peered":"2024-04-25T10:06:10.589663+0000","last_unstale":"2024-04-25T10:06:11.594679+0000","last_undegraded":"2024-04-25T10:06:11.594679+0000","last_fullsized":"2024-04-25T10:06:11.594679+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:17.017530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611361+0000","last_change":"2024-04-25T10:06:11.608503+0000","last_active":"2024-04-25T10:06:11.611361+0000","last_peered":"2024-04-25T10:06:11.611361+0000","last_clean":"2024-04-25T10:06:11.611361+0000","last_became_active":"2024-04-25T10:06:11.608234+0000","last_became_peered":"2024-04-25T10:06:11.608234+0000","last_unstale":"2024-04-25T10:06:11.611361+0000","last_undegraded":"2024-04-25T10:06:11.611361+0000","last_fullsized":"2024-04-25T10:06:11.611361+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:11.365920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.604687+0000","last_change":"2024-04-25T10:06:10.590974+0000","last_active":"2024-04-25T10:06:11.604687+0000","last_peered":"2024-04-25T10:06:11.604687+0000","last_clean":"2024-04-25T10:06:11.604687+0000","last_became_active":"2024-04-25T10:06:10.590635+0000","last_became_peered":"2024-04-25T10:06:10.590635+0000","last_unstale":"2024-04-25T10:06:11.604687+0000","last_undegraded":"2024-04-25T10:06:11.604687+0000","last_fullsized":"2024-04-25T10:06:11.604687+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:31.888158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591335+0000","last_change":"2024-04-25T10:06:11.594569+0000","last_active":"2024-04-25T10:06:12.591335+0000","last_peered":"2024-04-25T10:06:12.591335+0000","last_clean":"2024-04-25T10:06:12.591335+0000","last_became_active":"2024-04-25T10:06:11.594016+0000","last_became_peered":"2024-04-25T10:06:11.594016+0000","last_unstale":"2024-04-25T10:06:12.591335+0000","last_undegraded":"2024-04-25T10:06:12.591335+0000","last_fullsized":"2024-04-25T10:06:12.591335+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:38.022779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590761+0000","last_change":"2024-04-25T10:06:10.591527+0000","last_active":"2024-04-25T10:06:12.590761+0000","last_peered":"2024-04-25T10:06:12.590761+0000","last_clean":"2024-04-25T10:06:12.590761+0000","last_became_active":"2024-04-25T10:06:10.591212+0000","last_became_peered":"2024-04-25T10:06:10.591212+0000","last_unstale":"2024-04-25T10:06:12.590761+0000","last_undegraded":"2024-04-25T10:06:12.590761+0000","last_fullsized":"2024-04-25T10:06:12.590761+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:17.896865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590681+0000","last_change":"2024-04-25T10:06:11.597833+0000","last_active":"2024-04-25T10:06:12.590681+0000","last_peered":"2024-04-25T10:06:12.590681+0000","last_clean":"2024-04-25T10:06:12.590681+0000","last_became_active":"2024-04-25T10:06:11.597247+0000","last_became_peered":"2024-04-25T10:06:11.597247+0000","last_unstale":"2024-04-25T10:06:12.590681+0000","last_undegraded":"2024-04-25T10:06:12.590681+0000","last_fullsized":"2024-04-25T10:06:12.590681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:36.054034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590610+0000","last_change":"2024-04-25T10:06:10.593926+0000","last_active":"2024-04-25T10:06:12.590610+0000","last_peered":"2024-04-25T10:06:12.590610+0000","last_clean":"2024-04-25T10:06:12.590610+0000","last_became_active":"2024-04-25T10:06:10.593640+0000","last_became_peered":"2024-04-25T10:06:10.593640+0000","last_unstale":"2024-04-25T10:06:12.590610+0000","last_undegraded":"2024-04-25T10:06:12.590610+0000","last_fullsized":"2024-04-25T10:06:12.590610+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:32.052858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591186+0000","last_change":"2024-04-25T10:06:11.593063+0000","last_active":"2024-04-25T10:06:12.591186+0000","last_peered":"2024-04-25T10:06:12.591186+0000","last_clean":"2024-04-25T10:06:12.591186+0000","last_became_active":"2024-04-25T10:06:11.592772+0000","last_became_peered":"2024-04-25T10:06:11.592772+0000","last_unstale":"2024-04-25T10:06:12.591186+0000","last_undegraded":"2024-04-25T10:06:12.591186+0000","last_fullsized":"2024-04-25T10:06:12.591186+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:12.627498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.584205+0000","last_change":"2024-04-25T10:06:10.594103+0000","last_active":"2024-04-25T10:06:11.584205+0000","last_peered":"2024-04-25T10:06:11.584205+0000","last_clean":"2024-04-25T10:06:11.584205+0000","last_became_active":"2024-04-25T10:06:10.593805+0000","last_became_peered":"2024-04-25T10:06:10.593805+0000","last_unstale":"2024-04-25T10:06:11.584205+0000","last_undegraded":"2024-04-25T10:06:11.584205+0000","last_fullsized":"2024-04-25T10:06:11.584205+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:21.758145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.597476+0000","last_change":"2024-04-25T10:06:11.590223+0000","last_active":"2024-04-25T10:06:11.597476+0000","last_peered":"2024-04-25T10:06:11.597476+0000","last_clean":"2024-04-25T10:06:11.597476+0000","last_became_active":"2024-04-25T10:06:11.589970+0000","last_became_peered":"2024-04-25T10:06:11.589970+0000","last_unstale":"2024-04-25T10:06:11.597476+0000","last_undegraded":"2024-04-25T10:06:11.597476+0000","last_fullsized":"2024-04-25T10:06:11.597476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:44.396817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590364+0000","last_change":"2024-04-25T10:06:10.590741+0000","last_active":"2024-04-25T10:06:12.590364+0000","last_peered":"2024-04-25T10:06:12.590364+0000","last_clean":"2024-04-25T10:06:12.590364+0000","last_became_active":"2024-04-25T10:06:10.590499+0000","last_became_peered":"2024-04-25T10:06:10.590499+0000","last_unstale":"2024-04-25T10:06:12.590364+0000","last_undegraded":"2024-04-25T10:06:12.590364+0000","last_fullsized":"2024-04-25T10:06:12.590364+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:32.676160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599229+0000","last_change":"2024-04-25T10:06:11.594584+0000","last_active":"2024-04-25T10:06:11.599229+0000","last_peered":"2024-04-25T10:06:11.599229+0000","last_clean":"2024-04-25T10:06:11.599229+0000","last_became_active":"2024-04-25T10:06:11.594264+0000","last_became_peered":"2024-04-25T10:06:11.594264+0000","last_unstale":"2024-04-25T10:06:11.599229+0000","last_undegraded":"2024-04-25T10:06:11.599229+0000","last_fullsized":"2024-04-25T10:06:11.599229+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:33.855981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:13.034316+0000","last_change":"2024-04-25T10:06:10.591478+0000","last_active":"2024-04-25T10:06:13.034316+0000","last_peered":"2024-04-25T10:06:13.034316+0000","last_clean":"2024-04-25T10:06:13.034316+0000","last_became_active":"2024-04-25T10:06:10.591156+0000","last_became_peered":"2024-04-25T10:06:10.591156+0000","last_unstale":"2024-04-25T10:06:13.034316+0000","last_undegraded":"2024-04-25T10:06:13.034316+0000","last_fullsized":"2024-04-25T10:06:13.034316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:13.895648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.603137+0000","last_change":"2024-04-25T10:06:11.598368+0000","last_active":"2024-04-25T10:06:11.603137+0000","last_peered":"2024-04-25T10:06:11.603137+0000","last_clean":"2024-04-25T10:06:11.603137+0000","last_became_active":"2024-04-25T10:06:11.597963+0000","last_became_peered":"2024-04-25T10:06:11.597963+0000","last_unstale":"2024-04-25T10:06:11.603137+0000","last_undegraded":"2024-04-25T10:06:11.603137+0000","last_fullsized":"2024-04-25T10:06:11.603137+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:48.190466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:13.033777+0000","last_change":"2024-04-25T10:06:10.594136+0000","last_active":"2024-04-25T10:06:13.033777+0000","last_peered":"2024-04-25T10:06:13.033777+0000","last_clean":"2024-04-25T10:06:13.033777+0000","last_became_active":"2024-04-25T10:06:10.593885+0000","last_became_peered":"2024-04-25T10:06:10.593885+0000","last_unstale":"2024-04-25T10:06:13.033777+0000","last_undegraded":"2024-04-25T10:06:13.033777+0000","last_fullsized":"2024-04-25T10:06:13.033777+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:20.814527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610787+0000","last_change":"2024-04-25T10:06:11.603945+0000","last_active":"2024-04-25T10:06:11.610787+0000","last_peered":"2024-04-25T10:06:11.610787+0000","last_clean":"2024-04-25T10:06:11.610787+0000","last_became_active":"2024-04-25T10:06:11.602849+0000","last_became_peered":"2024-04-25T10:06:11.602849+0000","last_unstale":"2024-04-25T10:06:11.610787+0000","last_undegraded":"2024-04-25T10:06:11.610787+0000","last_fullsized":"2024-04-25T10:06:11.610787+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:03.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.601893+0000","last_change":"2024-04-25T10:06:10.579763+0000","last_active":"2024-04-25T10:06:11.601893+0000","last_peered":"2024-04-25T10:06:11.601893+0000","last_clean":"2024-04-25T10:06:11.601893+0000","last_became_active":"2024-04-25T10:06:10.579495+0000","last_became_peered":"2024-04-25T10:06:10.579495+0000","last_unstale":"2024-04-25T10:06:11.601893+0000","last_undegraded":"2024-04-25T10:06:11.601893+0000","last_fullsized":"2024-04-25T10:06:11.601893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:08.212125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.603845+0000","last_change":"2024-04-25T10:06:11.603317+0000","last_active":"2024-04-25T10:06:11.603845+0000","last_peered":"2024-04-25T10:06:11.603845+0000","last_clean":"2024-04-25T10:06:11.603845+0000","last_became_active":"2024-04-25T10:06:11.603066+0000","last_became_peered":"2024-04-25T10:06:11.603066+0000","last_unstale":"2024-04-25T10:06:11.603845+0000","last_undegraded":"2024-04-25T10:06:11.603845+0000","last_fullsized":"2024-04-25T10:06:11.603845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:07.686632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592122+0000","last_change":"2024-04-25T10:06:10.593257+0000","last_active":"2024-04-25T10:06:12.592122+0000","last_peered":"2024-04-25T10:06:12.592122+0000","last_clean":"2024-04-25T10:06:12.592122+0000","last_became_active":"2024-04-25T10:06:10.592737+0000","last_became_peered":"2024-04-25T10:06:10.592737+0000","last_unstale":"2024-04-25T10:06:12.592122+0000","last_undegraded":"2024-04-25T10:06:12.592122+0000","last_fullsized":"2024-04-25T10:06:12.592122+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:38.738216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591639+0000","last_change":"2024-04-25T10:06:11.601746+0000","last_active":"2024-04-25T10:06:12.591639+0000","last_peered":"2024-04-25T10:06:12.591639+0000","last_clean":"2024-04-25T10:06:12.591639+0000","last_became_active":"2024-04-25T10:06:11.600706+0000","last_became_peered":"2024-04-25T10:06:11.600706+0000","last_unstale":"2024-04-25T10:06:12.591639+0000","last_undegraded":"2024-04-25T10:06:12.591639+0000","last_fullsized":"2024-04-25T10:06:12.591639+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:37.328665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590790+0000","last_change":"2024-04-25T10:06:10.589747+0000","last_active":"2024-04-25T10:06:12.590790+0000","last_peered":"2024-04-25T10:06:12.590790+0000","last_clean":"2024-04-25T10:06:12.590790+0000","last_became_active":"2024-04-25T10:06:10.589483+0000","last_became_peered":"2024-04-25T10:06:10.589483+0000","last_unstale":"2024-04-25T10:06:12.590790+0000","last_undegraded":"2024-04-25T10:06:12.590790+0000","last_fullsized":"2024-04-25T10:06:12.590790+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.133966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.605963+0000","last_change":"2024-04-25T10:06:11.599329+0000","last_active":"2024-04-25T10:06:11.605963+0000","last_peered":"2024-04-25T10:06:11.605963+0000","last_clean":"2024-04-25T10:06:11.605963+0000","last_became_active":"2024-04-25T10:06:11.599055+0000","last_became_peered":"2024-04-25T10:06:11.599055+0000","last_unstale":"2024-04-25T10:06:11.605963+0000","last_undegraded":"2024-04-25T10:06:11.605963+0000","last_fullsized":"2024-04-25T10:06:11.605963+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:04.221854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.590118+0000","last_change":"2024-04-25T10:06:10.592105+0000","last_active":"2024-04-25T10:06:11.590118+0000","last_peered":"2024-04-25T10:06:11.590118+0000","last_clean":"2024-04-25T10:06:11.590118+0000","last_became_active":"2024-04-25T10:06:10.591823+0000","last_became_peered":"2024-04-25T10:06:10.591823+0000","last_unstale":"2024-04-25T10:06:11.590118+0000","last_undegraded":"2024-04-25T10:06:11.590118+0000","last_fullsized":"2024-04-25T10:06:11.590118+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:53.425769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590360+0000","last_change":"2024-04-25T10:06:11.585750+0000","last_active":"2024-04-25T10:06:12.590360+0000","last_peered":"2024-04-25T10:06:12.590360+0000","last_clean":"2024-04-25T10:06:12.590360+0000","last_became_active":"2024-04-25T10:06:11.585193+0000","last_became_peered":"2024-04-25T10:06:11.585193+0000","last_unstale":"2024-04-25T10:06:12.590360+0000","last_undegraded":"2024-04-25T10:06:12.590360+0000","last_fullsized":"2024-04-25T10:06:12.590360+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:03.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.589817+0000","last_change":"2024-04-25T10:06:10.590311+0000","last_active":"2024-04-25T10:06:11.589817+0000","last_peered":"2024-04-25T10:06:11.589817+0000","last_clean":"2024-04-25T10:06:11.589817+0000","last_became_active":"2024-04-25T10:06:10.590015+0000","last_became_peered":"2024-04-25T10:06:10.590015+0000","last_unstale":"2024-04-25T10:06:11.589817+0000","last_undegraded":"2024-04-25T10:06:11.589817+0000","last_fullsized":"2024-04-25T10:06:11.589817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:27.524048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.585155+0000","last_change":"2024-04-25T10:06:11.582381+0000","last_active":"2024-04-25T10:06:11.585155+0000","last_peered":"2024-04-25T10:06:11.585155+0000","last_clean":"2024-04-25T10:06:11.585155+0000","last_became_active":"2024-04-25T10:06:11.582109+0000","last_became_peered":"2024-04-25T10:06:11.582109+0000","last_unstale":"2024-04-25T10:06:11.585155+0000","last_undegraded":"2024-04-25T10:06:11.585155+0000","last_fullsized":"2024-04-25T10:06:11.585155+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:41.604772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.604177+0000","last_change":"2024-04-25T10:06:10.589625+0000","last_active":"2024-04-25T10:06:11.604177+0000","last_peered":"2024-04-25T10:06:11.604177+0000","last_clean":"2024-04-25T10:06:11.604177+0000","last_became_active":"2024-04-25T10:06:10.589161+0000","last_became_peered":"2024-04-25T10:06:10.589161+0000","last_unstale":"2024-04-25T10:06:11.604177+0000","last_undegraded":"2024-04-25T10:06:11.604177+0000","last_fullsized":"2024-04-25T10:06:11.604177+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:47.673088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.612662+0000","last_change":"2024-04-25T10:06:11.612101+0000","last_active":"2024-04-25T10:06:11.612662+0000","last_peered":"2024-04-25T10:06:11.612662+0000","last_clean":"2024-04-25T10:06:11.612662+0000","last_became_active":"2024-04-25T10:06:11.611805+0000","last_became_peered":"2024-04-25T10:06:11.611805+0000","last_unstale":"2024-04-25T10:06:11.612662+0000","last_undegraded":"2024-04-25T10:06:11.612662+0000","last_fullsized":"2024-04-25T10:06:11.612662+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:22.287749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.598493+0000","last_change":"2024-04-25T10:06:10.594406+0000","last_active":"2024-04-25T10:06:11.598493+0000","last_peered":"2024-04-25T10:06:11.598493+0000","last_clean":"2024-04-25T10:06:11.598493+0000","last_became_active":"2024-04-25T10:06:10.594129+0000","last_became_peered":"2024-04-25T10:06:10.594129+0000","last_unstale":"2024-04-25T10:06:11.598493+0000","last_undegraded":"2024-04-25T10:06:11.598493+0000","last_fullsized":"2024-04-25T10:06:11.598493+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:19.498453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591710+0000","last_change":"2024-04-25T10:06:11.592494+0000","last_active":"2024-04-25T10:06:12.591710+0000","last_peered":"2024-04-25T10:06:12.591710+0000","last_clean":"2024-04-25T10:06:12.591710+0000","last_became_active":"2024-04-25T10:06:11.592212+0000","last_became_peered":"2024-04-25T10:06:11.592212+0000","last_unstale":"2024-04-25T10:06:12.591710+0000","last_undegraded":"2024-04-25T10:06:12.591710+0000","last_fullsized":"2024-04-25T10:06:12.591710+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:08.065111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590880+0000","last_change":"2024-04-25T10:06:10.588563+0000","last_active":"2024-04-25T10:06:12.590880+0000","last_peered":"2024-04-25T10:06:12.590880+0000","last_clean":"2024-04-25T10:06:12.590880+0000","last_became_active":"2024-04-25T10:06:10.588312+0000","last_became_peered":"2024-04-25T10:06:10.588312+0000","last_unstale":"2024-04-25T10:06:12.590880+0000","last_undegraded":"2024-04-25T10:06:12.590880+0000","last_fullsized":"2024-04-25T10:06:12.590880+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:17.315120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:13.033642+0000","last_change":"2024-04-25T10:06:11.586327+0000","last_active":"2024-04-25T10:06:13.033642+0000","last_peered":"2024-04-25T10:06:13.033642+0000","last_clean":"2024-04-25T10:06:13.033642+0000","last_became_active":"2024-04-25T10:06:11.586002+0000","last_became_peered":"2024-04-25T10:06:11.586002+0000","last_unstale":"2024-04-25T10:06:13.033642+0000","last_undegraded":"2024-04-25T10:06:13.033642+0000","last_fullsized":"2024-04-25T10:06:13.033642+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:59.859154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.602393+0000","last_change":"2024-04-25T10:06:10.588662+0000","last_active":"2024-04-25T10:06:11.602393+0000","last_peered":"2024-04-25T10:06:11.602393+0000","last_clean":"2024-04-25T10:06:11.602393+0000","last_became_active":"2024-04-25T10:06:10.588316+0000","last_became_peered":"2024-04-25T10:06:10.588316+0000","last_unstale":"2024-04-25T10:06:11.602393+0000","last_undegraded":"2024-04-25T10:06:11.602393+0000","last_fullsized":"2024-04-25T10:06:11.602393+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:32.850838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599469+0000","last_change":"2024-04-25T10:06:11.595815+0000","last_active":"2024-04-25T10:06:11.599469+0000","last_peered":"2024-04-25T10:06:11.599469+0000","last_clean":"2024-04-25T10:06:11.599469+0000","last_became_active":"2024-04-25T10:06:11.595495+0000","last_became_peered":"2024-04-25T10:06:11.595495+0000","last_unstale":"2024-04-25T10:06:11.599469+0000","last_undegraded":"2024-04-25T10:06:11.599469+0000","last_fullsized":"2024-04-25T10:06:11.599469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:12.627498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.581094+0000","last_change":"2024-04-25T10:06:10.591774+0000","last_active":"2024-04-25T10:06:11.581094+0000","last_peered":"2024-04-25T10:06:11.581094+0000","last_clean":"2024-04-25T10:06:11.581094+0000","last_became_active":"2024-04-25T10:06:10.591412+0000","last_became_peered":"2024-04-25T10:06:10.591412+0000","last_unstale":"2024-04-25T10:06:11.581094+0000","last_undegraded":"2024-04-25T10:06:11.581094+0000","last_fullsized":"2024-04-25T10:06:11.581094+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:17.401215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591675+0000","last_change":"2024-04-25T10:06:11.593318+0000","last_active":"2024-04-25T10:06:12.591675+0000","last_peered":"2024-04-25T10:06:12.591675+0000","last_clean":"2024-04-25T10:06:12.591675+0000","last_became_active":"2024-04-25T10:06:11.593022+0000","last_became_peered":"2024-04-25T10:06:11.593022+0000","last_unstale":"2024-04-25T10:06:12.591675+0000","last_undegraded":"2024-04-25T10:06:12.591675+0000","last_fullsized":"2024-04-25T10:06:12.591675+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:46.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:13.033872+0000","last_change":"2024-04-25T10:06:10.590716+0000","last_active":"2024-04-25T10:06:13.033872+0000","last_peered":"2024-04-25T10:06:13.033872+0000","last_clean":"2024-04-25T10:06:13.033872+0000","last_became_active":"2024-04-25T10:06:10.590427+0000","last_became_peered":"2024-04-25T10:06:10.590427+0000","last_unstale":"2024-04-25T10:06:13.033872+0000","last_undegraded":"2024-04-25T10:06:13.033872+0000","last_fullsized":"2024-04-25T10:06:13.033872+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:56.243387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:13.033938+0000","last_change":"2024-04-25T10:06:11.595359+0000","last_active":"2024-04-25T10:06:13.033938+0000","last_peered":"2024-04-25T10:06:13.033938+0000","last_clean":"2024-04-25T10:06:13.033938+0000","last_became_active":"2024-04-25T10:06:11.595037+0000","last_became_peered":"2024-04-25T10:06:11.595037+0000","last_unstale":"2024-04-25T10:06:13.033938+0000","last_undegraded":"2024-04-25T10:06:13.033938+0000","last_fullsized":"2024-04-25T10:06:13.033938+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:49.489109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590609+0000","last_change":"2024-04-25T10:06:10.590140+0000","last_active":"2024-04-25T10:06:12.590609+0000","last_peered":"2024-04-25T10:06:12.590609+0000","last_clean":"2024-04-25T10:06:12.590609+0000","last_became_active":"2024-04-25T10:06:10.589015+0000","last_became_peered":"2024-04-25T10:06:10.589015+0000","last_unstale":"2024-04-25T10:06:12.590609+0000","last_undegraded":"2024-04-25T10:06:12.590609+0000","last_fullsized":"2024-04-25T10:06:12.590609+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:18.383218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591617+0000","last_change":"2024-04-25T10:06:11.604928+0000","last_active":"2024-04-25T10:06:12.591617+0000","last_peered":"2024-04-25T10:06:12.591617+0000","last_clean":"2024-04-25T10:06:12.591617+0000","last_became_active":"2024-04-25T10:06:11.604636+0000","last_became_peered":"2024-04-25T10:06:11.604636+0000","last_unstale":"2024-04-25T10:06:12.591617+0000","last_undegraded":"2024-04-25T10:06:12.591617+0000","last_fullsized":"2024-04-25T10:06:12.591617+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:04.221854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591565+0000","last_change":"2024-04-25T10:06:10.590105+0000","last_active":"2024-04-25T10:06:12.591565+0000","last_peered":"2024-04-25T10:06:12.591565+0000","last_clean":"2024-04-25T10:06:12.591565+0000","last_became_active":"2024-04-25T10:06:10.589829+0000","last_became_peered":"2024-04-25T10:06:10.589829+0000","last_unstale":"2024-04-25T10:06:12.591565+0000","last_undegraded":"2024-04-25T10:06:12.591565+0000","last_fullsized":"2024-04-25T10:06:12.591565+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:23.745526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591751+0000","last_change":"2024-04-25T10:06:11.593413+0000","last_active":"2024-04-25T10:06:12.591751+0000","last_peered":"2024-04-25T10:06:12.591751+0000","last_clean":"2024-04-25T10:06:12.591751+0000","last_became_active":"2024-04-25T10:06:11.593155+0000","last_became_peered":"2024-04-25T10:06:11.593155+0000","last_unstale":"2024-04-25T10:06:12.591751+0000","last_undegraded":"2024-04-25T10:06:12.591751+0000","last_fullsized":"2024-04-25T10:06:12.591751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:33.678639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.579429+0000","last_change":"2024-04-25T10:06:10.587062+0000","last_active":"2024-04-25T10:06:11.579429+0000","last_peered":"2024-04-25T10:06:11.579429+0000","last_clean":"2024-04-25T10:06:11.579429+0000","last_became_active":"2024-04-25T10:06:10.586736+0000","last_became_peered":"2024-04-25T10:06:10.586736+0000","last_unstale":"2024-04-25T10:06:11.579429+0000","last_undegraded":"2024-04-25T10:06:11.579429+0000","last_fullsized":"2024-04-25T10:06:11.579429+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:48.572483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599597+0000","last_change":"2024-04-25T10:06:11.595699+0000","last_active":"2024-04-25T10:06:11.599597+0000","last_peered":"2024-04-25T10:06:11.599597+0000","last_clean":"2024-04-25T10:06:11.599597+0000","last_became_active":"2024-04-25T10:06:11.595437+0000","last_became_peered":"2024-04-25T10:06:11.595437+0000","last_unstale":"2024-04-25T10:06:11.599597+0000","last_undegraded":"2024-04-25T10:06:11.599597+0000","last_fullsized":"2024-04-25T10:06:11.599597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:38.022779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591835+0000","last_change":"2024-04-25T10:06:10.588344+0000","last_active":"2024-04-25T10:06:12.591835+0000","last_peered":"2024-04-25T10:06:12.591835+0000","last_clean":"2024-04-25T10:06:12.591835+0000","last_became_active":"2024-04-25T10:06:10.587774+0000","last_became_peered":"2024-04-25T10:06:10.587774+0000","last_unstale":"2024-04-25T10:06:12.591835+0000","last_undegraded":"2024-04-25T10:06:12.591835+0000","last_fullsized":"2024-04-25T10:06:12.591835+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:13.625296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.603613+0000","last_change":"2024-04-25T10:06:11.603042+0000","last_active":"2024-04-25T10:06:11.603613+0000","last_peered":"2024-04-25T10:06:11.603613+0000","last_clean":"2024-04-25T10:06:11.603613+0000","last_became_active":"2024-04-25T10:06:11.598535+0000","last_became_peered":"2024-04-25T10:06:11.598535+0000","last_unstale":"2024-04-25T10:06:11.603613+0000","last_undegraded":"2024-04-25T10:06:11.603613+0000","last_fullsized":"2024-04-25T10:06:11.603613+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:31.070811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:13.033932+0000","last_change":"2024-04-25T10:06:10.592907+0000","last_active":"2024-04-25T10:06:13.033932+0000","last_peered":"2024-04-25T10:06:13.033932+0000","last_clean":"2024-04-25T10:06:13.033932+0000","last_became_active":"2024-04-25T10:06:10.592637+0000","last_became_peered":"2024-04-25T10:06:10.592637+0000","last_unstale":"2024-04-25T10:06:13.033932+0000","last_undegraded":"2024-04-25T10:06:13.033932+0000","last_fullsized":"2024-04-25T10:06:13.033932+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:52.984279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.605940+0000","last_change":"2024-04-25T10:06:11.603834+0000","last_active":"2024-04-25T10:06:11.605940+0000","last_peered":"2024-04-25T10:06:11.605940+0000","last_clean":"2024-04-25T10:06:11.605940+0000","last_became_active":"2024-04-25T10:06:11.603541+0000","last_became_peered":"2024-04-25T10:06:11.603541+0000","last_unstale":"2024-04-25T10:06:11.605940+0000","last_undegraded":"2024-04-25T10:06:11.605940+0000","last_fullsized":"2024-04-25T10:06:11.605940+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:08.065111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592173+0000","last_change":"2024-04-25T10:06:10.590856+0000","last_active":"2024-04-25T10:06:12.592173+0000","last_peered":"2024-04-25T10:06:12.592173+0000","last_clean":"2024-04-25T10:06:12.592173+0000","last_became_active":"2024-04-25T10:06:10.590571+0000","last_became_peered":"2024-04-25T10:06:10.590571+0000","last_unstale":"2024-04-25T10:06:12.592173+0000","last_undegraded":"2024-04-25T10:06:12.592173+0000","last_fullsized":"2024-04-25T10:06:12.592173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:01.814874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.596509+0000","last_change":"2024-04-25T10:06:11.595775+0000","last_active":"2024-04-25T10:06:11.596509+0000","last_peered":"2024-04-25T10:06:11.596509+0000","last_clean":"2024-04-25T10:06:11.596509+0000","last_became_active":"2024-04-25T10:06:11.595387+0000","last_became_peered":"2024-04-25T10:06:11.595387+0000","last_unstale":"2024-04-25T10:06:11.596509+0000","last_undegraded":"2024-04-25T10:06:11.596509+0000","last_fullsized":"2024-04-25T10:06:11.596509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:40.079857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592115+0000","last_change":"2024-04-25T10:06:10.588101+0000","last_active":"2024-04-25T10:06:12.592115+0000","last_peered":"2024-04-25T10:06:12.592115+0000","last_clean":"2024-04-25T10:06:12.592115+0000","last_became_active":"2024-04-25T10:06:10.587808+0000","last_became_peered":"2024-04-25T10:06:10.587808+0000","last_unstale":"2024-04-25T10:06:12.592115+0000","last_undegraded":"2024-04-25T10:06:12.592115+0000","last_fullsized":"2024-04-25T10:06:12.592115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.133966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610579+0000","last_change":"2024-04-25T10:06:11.599755+0000","last_active":"2024-04-25T10:06:11.610579+0000","last_peered":"2024-04-25T10:06:11.610579+0000","last_clean":"2024-04-25T10:06:11.610579+0000","last_became_active":"2024-04-25T10:06:11.599317+0000","last_became_peered":"2024-04-25T10:06:11.599317+0000","last_unstale":"2024-04-25T10:06:11.610579+0000","last_undegraded":"2024-04-25T10:06:11.610579+0000","last_fullsized":"2024-04-25T10:06:11.610579+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:37.328665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591917+0000","last_change":"2024-04-25T10:06:10.591156+0000","last_active":"2024-04-25T10:06:12.591917+0000","last_peered":"2024-04-25T10:06:12.591917+0000","last_clean":"2024-04-25T10:06:12.591917+0000","last_became_active":"2024-04-25T10:06:10.590362+0000","last_became_peered":"2024-04-25T10:06:10.590362+0000","last_unstale":"2024-04-25T10:06:12.591917+0000","last_undegraded":"2024-04-25T10:06:12.591917+0000","last_fullsized":"2024-04-25T10:06:12.591917+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:32.052858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610758+0000","last_change":"2024-04-25T10:06:11.603491+0000","last_active":"2024-04-25T10:06:11.610758+0000","last_peered":"2024-04-25T10:06:11.610758+0000","last_clean":"2024-04-25T10:06:11.610758+0000","last_became_active":"2024-04-25T10:06:11.600221+0000","last_became_peered":"2024-04-25T10:06:11.600221+0000","last_unstale":"2024-04-25T10:06:11.610758+0000","last_undegraded":"2024-04-25T10:06:11.610758+0000","last_fullsized":"2024-04-25T10:06:11.610758+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590413+0000","last_change":"2024-04-25T10:06:10.589304+0000","last_active":"2024-04-25T10:06:12.590413+0000","last_peered":"2024-04-25T10:06:12.590413+0000","last_clean":"2024-04-25T10:06:12.590413+0000","last_became_active":"2024-04-25T10:06:10.588934+0000","last_became_peered":"2024-04-25T10:06:10.588934+0000","last_unstale":"2024-04-25T10:06:12.590413+0000","last_undegraded":"2024-04-25T10:06:12.590413+0000","last_fullsized":"2024-04-25T10:06:12.590413+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:07.062632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.602797+0000","last_change":"2024-04-25T10:06:11.597255+0000","last_active":"2024-04-25T10:06:11.602797+0000","last_peered":"2024-04-25T10:06:11.602797+0000","last_clean":"2024-04-25T10:06:11.602797+0000","last_became_active":"2024-04-25T10:06:11.596954+0000","last_became_peered":"2024-04-25T10:06:11.596954+0000","last_unstale":"2024-04-25T10:06:11.602797+0000","last_undegraded":"2024-04-25T10:06:11.602797+0000","last_fullsized":"2024-04-25T10:06:11.602797+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:13.167235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.596522+0000","last_change":"2024-04-25T10:06:10.589615+0000","last_active":"2024-04-25T10:06:11.596522+0000","last_peered":"2024-04-25T10:06:11.596522+0000","last_clean":"2024-04-25T10:06:11.596522+0000","last_became_active":"2024-04-25T10:06:10.589310+0000","last_became_peered":"2024-04-25T10:06:10.589310+0000","last_unstale":"2024-04-25T10:06:11.596522+0000","last_undegraded":"2024-04-25T10:06:11.596522+0000","last_fullsized":"2024-04-25T10:06:11.596522+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:00.780604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.597484+0000","last_change":"2024-04-25T10:06:11.596201+0000","last_active":"2024-04-25T10:06:11.597484+0000","last_peered":"2024-04-25T10:06:11.597484+0000","last_clean":"2024-04-25T10:06:11.597484+0000","last_became_active":"2024-04-25T10:06:11.595880+0000","last_became_peered":"2024-04-25T10:06:11.595880+0000","last_unstale":"2024-04-25T10:06:11.597484+0000","last_undegraded":"2024-04-25T10:06:11.597484+0000","last_fullsized":"2024-04-25T10:06:11.597484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:36.054034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.580878+0000","last_change":"2024-04-25T10:06:10.587821+0000","last_active":"2024-04-25T10:06:11.580878+0000","last_peered":"2024-04-25T10:06:11.580878+0000","last_clean":"2024-04-25T10:06:11.580878+0000","last_became_active":"2024-04-25T10:06:10.587520+0000","last_became_peered":"2024-04-25T10:06:10.587520+0000","last_unstale":"2024-04-25T10:06:11.580878+0000","last_undegraded":"2024-04-25T10:06:11.580878+0000","last_fullsized":"2024-04-25T10:06:11.580878+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:17.031658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":198,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591162+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:06:12.591162+0000","last_peered":"2024-04-25T10:06:12.591162+0000","last_clean":"2024-04-25T10:06:12.591162+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:06:12.591162+0000","last_undegraded":"2024-04-25T10:06:12.591162+0000","last_fullsized":"2024-04-25T10:06:12.591162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":150789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":152379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":152401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":405504,"data_stored":151543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":393216,"data_stored":150801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":747505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":393216,"data_stored":150857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":748017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:19.016 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:19.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.125+0000 7f5be1c07640 1 Processor -- start 2024-04-25T10:06:19.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.126+0000 7f5be1c07640 1 -- start start 2024-04-25T10:06:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.126+0000 7f5be1c07640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5bdc15dce0 0x7f5bdc0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.126+0000 7f5be1c07640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5bdc0d35b0 0x7f5bdc0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.126+0000 7f5be1c07640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bdc15e0c0 con 0x7f5bdc0d1f90 2024-04-25T10:06:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.126+0000 7f5be1c07640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bdc0b7b10 con 0x7f5bdc0d35b0 2024-04-25T10:06:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.126+0000 7f5be1c07640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bdc0d2510 con 0x7f5bdc15dce0 2024-04-25T10:06:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.126+0000 7f5bdaffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5bdc0d35b0 0x7f5bdc0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.126+0000 7f5bdbfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5bdc0d1f90 0x7f5bdc06bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56536/0 (socket says 172.21.15.16:56536) 2024-04-25T10:06:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.127+0000 7f5bdbfff640 1 -- 172.21.15.16:0/1402279500 learned_addr learned my addr 172.21.15.16:0/1402279500 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.127+0000 7f5bdb7fe640 1 --2- 172.21.15.16:0/1402279500 >> v2:172.21.15.186:3300/0 conn(0x7f5bdc15dce0 0x7f5bdc0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.127+0000 7f5bdaffd640 1 -- 172.21.15.16:0/1402279500 >> v2:172.21.15.186:3300/0 conn(0x7f5bdc15dce0 msgr2=0x7f5bdc0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.127+0000 7f5bdaffd640 1 --2- 172.21.15.16:0/1402279500 >> v2:172.21.15.186:3300/0 conn(0x7f5bdc15dce0 0x7f5bdc0d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.127+0000 7f5bdaffd640 1 -- 172.21.15.16:0/1402279500 >> v1:172.21.15.16:6789/0 conn(0x7f5bdc0d1f90 legacy=0x7f5bdc06bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.127+0000 7f5bdaffd640 1 -- 172.21.15.16:0/1402279500 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc0d27f0 con 0x7f5bdc0d35b0 2024-04-25T10:06:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.127+0000 7f5bda7fc640 1 -- 172.21.15.16:0/1402279500 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173492841 0 0) 0x7f5bdc15e0c0 con 0x7f5bdc0d1f90 2024-04-25T10:06:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.128+0000 7f5bdaffd640 1 --2- 172.21.15.16:0/1402279500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bdc0d35b0 0x7f5bdc0b5450 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0ce220 tx=0x7f5bd002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13be9b783ea7f03d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.128+0000 7f5bda7fc640 1 -- 172.21.15.16:0/1402279500 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0037070 con 0x7f5bdc0d35b0 2024-04-25T10:06:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.128+0000 7f5bda7fc640 1 -- 172.21.15.16:0/1402279500 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd0032440 con 0x7f5bdc0d35b0 2024-04-25T10:06:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.128+0000 7f5bda7fc640 1 -- 172.21.15.16:0/1402279500 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0032810 con 0x7f5bdc0d35b0 2024-04-25T10:06:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.128+0000 7f5be1c07640 1 -- 172.21.15.16:0/1402279500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bdc0d35b0 msgr2=0x7f5bdc0b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.128+0000 7f5be1c07640 1 --2- 172.21.15.16:0/1402279500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bdc0d35b0 0x7f5bdc0b5450 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0ce220 tx=0x7f5bd002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.128+0000 7f5be1c07640 1 -- 172.21.15.16:0/1402279500 shutdown_connections 2024-04-25T10:06:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.128+0000 7f5be1c07640 1 --2- 172.21.15.16:0/1402279500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bdc0d35b0 0x7f5bdc0b5450 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.128+0000 7f5be1c07640 1 --2- 172.21.15.16:0/1402279500 >> v2:172.21.15.186:3300/0 conn(0x7f5bdc15dce0 0x7f5bdc0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.128+0000 7f5be1c07640 1 -- 172.21.15.16:0/1402279500 >> 172.21.15.16:0/1402279500 conn(0x7f5bdc072900 msgr2=0x7f5bdc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.128+0000 7f5be1c07640 1 -- 172.21.15.16:0/1402279500 shutdown_connections 2024-04-25T10:06:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.128+0000 7f5be1c07640 1 -- 172.21.15.16:0/1402279500 wait complete. 2024-04-25T10:06:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.130+0000 7f5be1c07640 1 Processor -- start 2024-04-25T10:06:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.130+0000 7f5be1c07640 1 -- start start 2024-04-25T10:06:19.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5be1c07640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bdc0d1f90 0x7f5bdc155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5be1c07640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bdc0d35b0 0x7f5bdc0cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5be1c07640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5bdc15dce0 0x7f5bdc153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5be1c07640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0d3090 con 0x7f5bdc0d1f90 2024-04-25T10:06:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5be1c07640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0d2db0 con 0x7f5bdc0d35b0 2024-04-25T10:06:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5be1c07640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0b9820 con 0x7f5bdc15dce0 2024-04-25T10:06:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5bdb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bdc0d1f90 0x7f5bdc155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5bdb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bdc0d1f90 0x7f5bdc155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49544/0 (socket says 172.21.15.16:49544) 2024-04-25T10:06:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5bdb7fe640 1 -- 172.21.15.16:0/248577343 learned_addr learned my addr 172.21.15.16:0/248577343 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5bdaffd640 1 --2- 172.21.15.16:0/248577343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bdc0d35b0 0x7f5bdc0cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5bdb7fe640 1 -- 172.21.15.16:0/248577343 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5bdc15dce0 msgr2=0x7f5bdc153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5bdb7fe640 1 --2- 172.21.15.16:0/248577343 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5bdc15dce0 0x7f5bdc153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5bdb7fe640 1 -- 172.21.15.16:0/248577343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bdc0d35b0 msgr2=0x7f5bdc0cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5bdb7fe640 1 --2- 172.21.15.16:0/248577343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bdc0d35b0 0x7f5bdc0cd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5bdb7fe640 1 -- 172.21.15.16:0/248577343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc155e50 con 0x7f5bdc0d1f90 2024-04-25T10:06:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5bdb7fe640 1 --2- 172.21.15.16:0/248577343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bdc0d1f90 0x7f5bdc155870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00d620 tx=0x7f5bcc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5bc3fff640 1 -- 172.21.15.16:0/248577343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc00b620 con 0x7f5bdc0d1f90 2024-04-25T10:06:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5bc3fff640 1 -- 172.21.15.16:0/248577343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc019070 con 0x7f5bdc0d1f90 2024-04-25T10:06:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5be1c07640 1 -- 172.21.15.16:0/248577343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bdc156130 con 0x7f5bdc0d1f90 2024-04-25T10:06:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.131+0000 7f5bc3fff640 1 -- 172.21.15.16:0/248577343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc00ba00 con 0x7f5bdc0d1f90 2024-04-25T10:06:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.132+0000 7f5be1c07640 1 -- 172.21.15.16:0/248577343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc1566f0 con 0x7f5bdc0d1f90 2024-04-25T10:06:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.132+0000 7f5be1c07640 1 -- 172.21.15.16:0/248577343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bdc0749b0 con 0x7f5bdc0d1f90 2024-04-25T10:06:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.133+0000 7f5bc3fff640 1 -- 172.21.15.16:0/248577343 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5bcc00bba0 con 0x7f5bdc0d1f90 2024-04-25T10:06:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.133+0000 7f5bc3fff640 1 --2- 172.21.15.16:0/248577343 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5bc806b4d0 0x7f5bc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.134+0000 7f5bc3fff640 1 -- 172.21.15.16:0/248577343 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bcc08a670 con 0x7f5bdc0d1f90 2024-04-25T10:06:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.134+0000 7f5bdaffd640 1 --2- 172.21.15.16:0/248577343 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5bc806b4d0 0x7f5bc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.134+0000 7f5bdaffd640 1 --2- 172.21.15.16:0/248577343 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5bc806b4d0 0x7f5bc806d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0034000 tx=0x7f5bd002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.138+0000 7f5bc3fff640 1 -- 172.21.15.16:0/248577343 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5bcc0594f0 con 0x7f5bdc0d1f90 2024-04-25T10:06:19.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.314+0000 7f5be1c07640 1 -- 172.21.15.16:0/248577343 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5bdc0b5dd0 con 0x7f5bc806b4d0 2024-04-25T10:06:19.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.320+0000 7f5bc3fff640 1 -- 172.21.15.16:0/248577343 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251919 (secure 0 0 0) 0x7f5bdc0b5dd0 con 0x7f5bc806b4d0 2024-04-25T10:06:19.329 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:19.333 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:06:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.323+0000 7f5be1c07640 1 -- 172.21.15.16:0/248577343 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5bc806b4d0 msgr2=0x7f5bc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.323+0000 7f5be1c07640 1 --2- 172.21.15.16:0/248577343 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5bc806b4d0 0x7f5bc806d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0034000 tx=0x7f5bd002e000 comp rx=0 tx=0).stop 2024-04-25T10:06:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.323+0000 7f5be1c07640 1 -- 172.21.15.16:0/248577343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bdc0d1f90 msgr2=0x7f5bdc155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.323+0000 7f5be1c07640 1 --2- 172.21.15.16:0/248577343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bdc0d1f90 0x7f5bdc155870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00d620 tx=0x7f5bcc00dae0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.323+0000 7f5be1c07640 1 -- 172.21.15.16:0/248577343 shutdown_connections 2024-04-25T10:06:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.323+0000 7f5be1c07640 1 --2- 172.21.15.16:0/248577343 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5bc806b4d0 0x7f5bc806d990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.323+0000 7f5be1c07640 1 --2- 172.21.15.16:0/248577343 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5bdc15dce0 0x7f5bdc153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.323+0000 7f5be1c07640 1 --2- 172.21.15.16:0/248577343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bdc0d35b0 0x7f5bdc0cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.323+0000 7f5be1c07640 1 --2- 172.21.15.16:0/248577343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bdc0d1f90 0x7f5bdc155870 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.323+0000 7f5be1c07640 1 -- 172.21.15.16:0/248577343 >> 172.21.15.16:0/248577343 conn(0x7f5bdc072900 msgr2=0x7f5bdc0ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.323+0000 7f5be1c07640 1 -- 172.21.15.16:0/248577343 shutdown_connections 2024-04-25T10:06:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.323+0000 7f5be1c07640 1 -- 172.21.15.16:0/248577343 wait complete. 2024-04-25T10:06:19.340 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-04-25T10:06:17.796169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237096,"kb_used_data":4288,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737624,"statfs":{"total":773094113280,"available":772851326976,"internally_reserved":0,"allocated":4390912,"data_stored":2404292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.219197"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590900+0000","last_change":"2024-04-25T10:06:11.596197+0000","last_active":"2024-04-25T10:06:12.590900+0000","last_peered":"2024-04-25T10:06:12.590900+0000","last_clean":"2024-04-25T10:06:12.590900+0000","last_became_active":"2024-04-25T10:06:11.595896+0000","last_became_peered":"2024-04-25T10:06:11.595896+0000","last_unstale":"2024-04-25T10:06:12.590900+0000","last_undegraded":"2024-04-25T10:06:12.590900+0000","last_fullsized":"2024-04-25T10:06:12.590900+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:50.014644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:13.033669+0000","last_change":"2024-04-25T10:06:11.612418+0000","last_active":"2024-04-25T10:06:13.033669+0000","last_peered":"2024-04-25T10:06:13.033669+0000","last_clean":"2024-04-25T10:06:13.033669+0000","last_became_active":"2024-04-25T10:06:11.612084+0000","last_became_peered":"2024-04-25T10:06:11.612084+0000","last_unstale":"2024-04-25T10:06:13.033669+0000","last_undegraded":"2024-04-25T10:06:13.033669+0000","last_fullsized":"2024-04-25T10:06:13.033669+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:52.424313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591774+0000","last_change":"2024-04-25T10:06:11.588804+0000","last_active":"2024-04-25T10:06:12.591774+0000","last_peered":"2024-04-25T10:06:12.591774+0000","last_clean":"2024-04-25T10:06:12.591774+0000","last_became_active":"2024-04-25T10:06:11.588533+0000","last_became_peered":"2024-04-25T10:06:11.588533+0000","last_unstale":"2024-04-25T10:06:12.591774+0000","last_undegraded":"2024-04-25T10:06:12.591774+0000","last_fullsized":"2024-04-25T10:06:12.591774+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:12.136446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610805+0000","last_change":"2024-04-25T10:06:11.603412+0000","last_active":"2024-04-25T10:06:11.610805+0000","last_peered":"2024-04-25T10:06:11.610805+0000","last_clean":"2024-04-25T10:06:11.610805+0000","last_became_active":"2024-04-25T10:06:11.602936+0000","last_became_peered":"2024-04-25T10:06:11.602936+0000","last_unstale":"2024-04-25T10:06:11.610805+0000","last_undegraded":"2024-04-25T10:06:11.610805+0000","last_fullsized":"2024-04-25T10:06:11.610805+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.403695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:13.033805+0000","last_change":"2024-04-25T10:06:11.593781+0000","last_active":"2024-04-25T10:06:13.033805+0000","last_peered":"2024-04-25T10:06:13.033805+0000","last_clean":"2024-04-25T10:06:13.033805+0000","last_became_active":"2024-04-25T10:06:11.593453+0000","last_became_peered":"2024-04-25T10:06:11.593453+0000","last_unstale":"2024-04-25T10:06:13.033805+0000","last_undegraded":"2024-04-25T10:06:13.033805+0000","last_fullsized":"2024-04-25T10:06:13.033805+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:12.965594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599412+0000","last_change":"2024-04-25T10:06:11.594811+0000","last_active":"2024-04-25T10:06:11.599412+0000","last_peered":"2024-04-25T10:06:11.599412+0000","last_clean":"2024-04-25T10:06:11.599412+0000","last_became_active":"2024-04-25T10:06:11.594494+0000","last_became_peered":"2024-04-25T10:06:11.594494+0000","last_unstale":"2024-04-25T10:06:11.599412+0000","last_undegraded":"2024-04-25T10:06:11.599412+0000","last_fullsized":"2024-04-25T10:06:11.599412+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:02.231151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590302+0000","last_change":"2024-04-25T10:06:11.585486+0000","last_active":"2024-04-25T10:06:12.590302+0000","last_peered":"2024-04-25T10:06:12.590302+0000","last_clean":"2024-04-25T10:06:12.590302+0000","last_became_active":"2024-04-25T10:06:11.585167+0000","last_became_peered":"2024-04-25T10:06:11.585167+0000","last_unstale":"2024-04-25T10:06:12.590302+0000","last_undegraded":"2024-04-25T10:06:12.590302+0000","last_fullsized":"2024-04-25T10:06:12.590302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590232+0000","last_change":"2024-04-25T10:06:11.599766+0000","last_active":"2024-04-25T10:06:12.590232+0000","last_peered":"2024-04-25T10:06:12.590232+0000","last_clean":"2024-04-25T10:06:12.590232+0000","last_became_active":"2024-04-25T10:06:11.599473+0000","last_became_peered":"2024-04-25T10:06:11.599473+0000","last_unstale":"2024-04-25T10:06:12.590232+0000","last_undegraded":"2024-04-25T10:06:12.590232+0000","last_fullsized":"2024-04-25T10:06:12.590232+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:46.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591773+0000","last_change":"2024-04-25T10:06:11.601172+0000","last_active":"2024-04-25T10:06:12.591773+0000","last_peered":"2024-04-25T10:06:12.591773+0000","last_clean":"2024-04-25T10:06:12.591773+0000","last_became_active":"2024-04-25T10:06:11.600890+0000","last_became_peered":"2024-04-25T10:06:11.600890+0000","last_unstale":"2024-04-25T10:06:12.591773+0000","last_undegraded":"2024-04-25T10:06:12.591773+0000","last_fullsized":"2024-04-25T10:06:12.591773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:33.055338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.614185+0000","last_change":"2024-04-25T10:06:11.611336+0000","last_active":"2024-04-25T10:06:11.614185+0000","last_peered":"2024-04-25T10:06:11.614185+0000","last_clean":"2024-04-25T10:06:11.614185+0000","last_became_active":"2024-04-25T10:06:11.610883+0000","last_became_peered":"2024-04-25T10:06:11.610883+0000","last_unstale":"2024-04-25T10:06:11.614185+0000","last_undegraded":"2024-04-25T10:06:11.614185+0000","last_fullsized":"2024-04-25T10:06:11.614185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:50.014644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.606907+0000","last_change":"2024-04-25T10:06:11.606133+0000","last_active":"2024-04-25T10:06:11.606907+0000","last_peered":"2024-04-25T10:06:11.606907+0000","last_clean":"2024-04-25T10:06:11.606907+0000","last_became_active":"2024-04-25T10:06:11.605855+0000","last_became_peered":"2024-04-25T10:06:11.605855+0000","last_unstale":"2024-04-25T10:06:11.606907+0000","last_undegraded":"2024-04-25T10:06:11.606907+0000","last_fullsized":"2024-04-25T10:06:11.606907+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:35.491727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591678+0000","last_change":"2024-04-25T10:06:11.598791+0000","last_active":"2024-04-25T10:06:12.591678+0000","last_peered":"2024-04-25T10:06:12.591678+0000","last_clean":"2024-04-25T10:06:12.591678+0000","last_became_active":"2024-04-25T10:06:11.598511+0000","last_became_peered":"2024-04-25T10:06:11.598511+0000","last_unstale":"2024-04-25T10:06:12.591678+0000","last_undegraded":"2024-04-25T10:06:12.591678+0000","last_fullsized":"2024-04-25T10:06:12.591678+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:18.899344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592055+0000","last_change":"2024-04-25T10:06:11.602586+0000","last_active":"2024-04-25T10:06:12.592055+0000","last_peered":"2024-04-25T10:06:12.592055+0000","last_clean":"2024-04-25T10:06:12.592055+0000","last_became_active":"2024-04-25T10:06:11.602219+0000","last_became_peered":"2024-04-25T10:06:11.602219+0000","last_unstale":"2024-04-25T10:06:12.592055+0000","last_undegraded":"2024-04-25T10:06:12.592055+0000","last_fullsized":"2024-04-25T10:06:12.592055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610597+0000","last_change":"2024-04-25T10:06:11.599901+0000","last_active":"2024-04-25T10:06:11.610597+0000","last_peered":"2024-04-25T10:06:11.610597+0000","last_clean":"2024-04-25T10:06:11.610597+0000","last_became_active":"2024-04-25T10:06:11.599632+0000","last_became_peered":"2024-04-25T10:06:11.599632+0000","last_unstale":"2024-04-25T10:06:11.610597+0000","last_undegraded":"2024-04-25T10:06:11.610597+0000","last_fullsized":"2024-04-25T10:06:11.610597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:46.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.592021+0000","last_change":"2024-04-25T10:06:11.587570+0000","last_active":"2024-04-25T10:06:11.592021+0000","last_peered":"2024-04-25T10:06:11.592021+0000","last_clean":"2024-04-25T10:06:11.592021+0000","last_became_active":"2024-04-25T10:06:11.587301+0000","last_became_peered":"2024-04-25T10:06:11.587301+0000","last_unstale":"2024-04-25T10:06:11.592021+0000","last_undegraded":"2024-04-25T10:06:11.592021+0000","last_fullsized":"2024-04-25T10:06:11.592021+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.597882+0000","last_change":"2024-04-25T10:06:11.597081+0000","last_active":"2024-04-25T10:06:11.597882+0000","last_peered":"2024-04-25T10:06:11.597882+0000","last_clean":"2024-04-25T10:06:11.597882+0000","last_became_active":"2024-04-25T10:06:11.596781+0000","last_became_peered":"2024-04-25T10:06:11.596781+0000","last_unstale":"2024-04-25T10:06:11.597882+0000","last_undegraded":"2024-04-25T10:06:11.597882+0000","last_fullsized":"2024-04-25T10:06:11.597882+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:11.365920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.589022+0000","last_change":"2024-04-25T10:06:11.587969+0000","last_active":"2024-04-25T10:06:11.589022+0000","last_peered":"2024-04-25T10:06:11.589022+0000","last_clean":"2024-04-25T10:06:11.589022+0000","last_became_active":"2024-04-25T10:06:11.587084+0000","last_became_peered":"2024-04-25T10:06:11.587084+0000","last_unstale":"2024-04-25T10:06:11.589022+0000","last_undegraded":"2024-04-25T10:06:11.589022+0000","last_fullsized":"2024-04-25T10:06:11.589022+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:47.208707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599620+0000","last_change":"2024-04-25T10:06:11.596669+0000","last_active":"2024-04-25T10:06:11.599620+0000","last_peered":"2024-04-25T10:06:11.599620+0000","last_clean":"2024-04-25T10:06:11.599620+0000","last_became_active":"2024-04-25T10:06:11.596091+0000","last_became_peered":"2024-04-25T10:06:11.596091+0000","last_unstale":"2024-04-25T10:06:11.599620+0000","last_undegraded":"2024-04-25T10:06:11.599620+0000","last_fullsized":"2024-04-25T10:06:11.599620+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:18.899344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599443+0000","last_change":"2024-04-25T10:06:11.594701+0000","last_active":"2024-04-25T10:06:11.599443+0000","last_peered":"2024-04-25T10:06:11.599443+0000","last_clean":"2024-04-25T10:06:11.599443+0000","last_became_active":"2024-04-25T10:06:11.594376+0000","last_became_peered":"2024-04-25T10:06:11.594376+0000","last_unstale":"2024-04-25T10:06:11.599443+0000","last_undegraded":"2024-04-25T10:06:11.599443+0000","last_fullsized":"2024-04-25T10:06:11.599443+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:53.157588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592096+0000","last_change":"2024-04-25T10:06:11.597575+0000","last_active":"2024-04-25T10:06:12.592096+0000","last_peered":"2024-04-25T10:06:12.592096+0000","last_clean":"2024-04-25T10:06:12.592096+0000","last_became_active":"2024-04-25T10:06:11.596710+0000","last_became_peered":"2024-04-25T10:06:11.596710+0000","last_unstale":"2024-04-25T10:06:12.592096+0000","last_undegraded":"2024-04-25T10:06:12.592096+0000","last_fullsized":"2024-04-25T10:06:12.592096+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.595406+0000","last_change":"2024-04-25T10:06:11.594514+0000","last_active":"2024-04-25T10:06:11.595406+0000","last_peered":"2024-04-25T10:06:11.595406+0000","last_clean":"2024-04-25T10:06:11.595406+0000","last_became_active":"2024-04-25T10:06:11.594231+0000","last_became_peered":"2024-04-25T10:06:11.594231+0000","last_unstale":"2024-04-25T10:06:11.595406+0000","last_undegraded":"2024-04-25T10:06:11.595406+0000","last_fullsized":"2024-04-25T10:06:11.595406+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:03.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590474+0000","last_change":"2024-04-25T10:06:11.595941+0000","last_active":"2024-04-25T10:06:12.590474+0000","last_peered":"2024-04-25T10:06:12.590474+0000","last_clean":"2024-04-25T10:06:12.590474+0000","last_became_active":"2024-04-25T10:06:11.595598+0000","last_became_peered":"2024-04-25T10:06:11.595598+0000","last_unstale":"2024-04-25T10:06:12.590474+0000","last_undegraded":"2024-04-25T10:06:12.590474+0000","last_fullsized":"2024-04-25T10:06:12.590474+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:40.079857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591540+0000","last_change":"2024-04-25T10:06:11.599549+0000","last_active":"2024-04-25T10:06:12.591540+0000","last_peered":"2024-04-25T10:06:12.591540+0000","last_clean":"2024-04-25T10:06:12.591540+0000","last_became_active":"2024-04-25T10:06:11.599256+0000","last_became_peered":"2024-04-25T10:06:11.599256+0000","last_unstale":"2024-04-25T10:06:12.591540+0000","last_undegraded":"2024-04-25T10:06:12.591540+0000","last_fullsized":"2024-04-25T10:06:12.591540+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:48.190466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590264+0000","last_change":"2024-04-25T10:06:11.605175+0000","last_active":"2024-04-25T10:06:12.590264+0000","last_peered":"2024-04-25T10:06:12.590264+0000","last_clean":"2024-04-25T10:06:12.590264+0000","last_became_active":"2024-04-25T10:06:11.604773+0000","last_became_peered":"2024-04-25T10:06:11.604773+0000","last_unstale":"2024-04-25T10:06:12.590264+0000","last_undegraded":"2024-04-25T10:06:12.590264+0000","last_fullsized":"2024-04-25T10:06:12.590264+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.403695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611074+0000","last_change":"2024-04-25T10:06:11.605194+0000","last_active":"2024-04-25T10:06:11.611074+0000","last_peered":"2024-04-25T10:06:11.611074+0000","last_clean":"2024-04-25T10:06:11.611074+0000","last_became_active":"2024-04-25T10:06:11.604595+0000","last_became_peered":"2024-04-25T10:06:11.604595+0000","last_unstale":"2024-04-25T10:06:11.611074+0000","last_undegraded":"2024-04-25T10:06:11.611074+0000","last_fullsized":"2024-04-25T10:06:11.611074+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:36.054034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611662+0000","last_change":"2024-04-25T10:06:11.603505+0000","last_active":"2024-04-25T10:06:11.611662+0000","last_peered":"2024-04-25T10:06:11.611662+0000","last_clean":"2024-04-25T10:06:11.611662+0000","last_became_active":"2024-04-25T10:06:11.603215+0000","last_became_peered":"2024-04-25T10:06:11.603215+0000","last_unstale":"2024-04-25T10:06:11.611662+0000","last_undegraded":"2024-04-25T10:06:11.611662+0000","last_fullsized":"2024-04-25T10:06:11.611662+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:13.033711+0000","last_change":"2024-04-25T10:06:11.593363+0000","last_active":"2024-04-25T10:06:13.033711+0000","last_peered":"2024-04-25T10:06:13.033711+0000","last_clean":"2024-04-25T10:06:13.033711+0000","last_became_active":"2024-04-25T10:06:11.593092+0000","last_became_peered":"2024-04-25T10:06:11.593092+0000","last_unstale":"2024-04-25T10:06:13.033711+0000","last_undegraded":"2024-04-25T10:06:13.033711+0000","last_fullsized":"2024-04-25T10:06:13.033711+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:19.088902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591554+0000","last_change":"2024-04-25T10:06:11.601007+0000","last_active":"2024-04-25T10:06:12.591554+0000","last_peered":"2024-04-25T10:06:12.591554+0000","last_clean":"2024-04-25T10:06:12.591554+0000","last_became_active":"2024-04-25T10:06:11.600297+0000","last_became_peered":"2024-04-25T10:06:11.600297+0000","last_unstale":"2024-04-25T10:06:12.591554+0000","last_undegraded":"2024-04-25T10:06:12.591554+0000","last_fullsized":"2024-04-25T10:06:12.591554+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:18.645477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.601405+0000","last_change":"2024-04-25T10:06:11.600447+0000","last_active":"2024-04-25T10:06:11.601405+0000","last_peered":"2024-04-25T10:06:11.601405+0000","last_clean":"2024-04-25T10:06:11.601405+0000","last_became_active":"2024-04-25T10:06:11.600160+0000","last_became_peered":"2024-04-25T10:06:11.600160+0000","last_unstale":"2024-04-25T10:06:11.601405+0000","last_undegraded":"2024-04-25T10:06:11.601405+0000","last_fullsized":"2024-04-25T10:06:11.601405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:50.014644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.591617+0000","last_change":"2024-04-25T10:06:11.590349+0000","last_active":"2024-04-25T10:06:11.591617+0000","last_peered":"2024-04-25T10:06:11.591617+0000","last_clean":"2024-04-25T10:06:11.591617+0000","last_became_active":"2024-04-25T10:06:11.590090+0000","last_became_peered":"2024-04-25T10:06:11.590090+0000","last_unstale":"2024-04-25T10:06:11.591617+0000","last_undegraded":"2024-04-25T10:06:11.591617+0000","last_fullsized":"2024-04-25T10:06:11.591617+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590344+0000","last_change":"2024-04-25T10:06:11.597956+0000","last_active":"2024-04-25T10:06:12.590344+0000","last_peered":"2024-04-25T10:06:12.590344+0000","last_clean":"2024-04-25T10:06:12.590344+0000","last_became_active":"2024-04-25T10:06:11.597687+0000","last_became_peered":"2024-04-25T10:06:11.597687+0000","last_unstale":"2024-04-25T10:06:12.590344+0000","last_undegraded":"2024-04-25T10:06:12.590344+0000","last_fullsized":"2024-04-25T10:06:12.590344+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.859555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.586794+0000","last_change":"2024-04-25T10:06:11.584953+0000","last_active":"2024-04-25T10:06:11.586794+0000","last_peered":"2024-04-25T10:06:11.586794+0000","last_clean":"2024-04-25T10:06:11.586794+0000","last_became_active":"2024-04-25T10:06:11.584666+0000","last_became_peered":"2024-04-25T10:06:11.584666+0000","last_unstale":"2024-04-25T10:06:11.586794+0000","last_undegraded":"2024-04-25T10:06:11.586794+0000","last_fullsized":"2024-04-25T10:06:11.586794+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:55.206873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591906+0000","last_change":"2024-04-25T10:06:11.592933+0000","last_active":"2024-04-25T10:06:12.591906+0000","last_peered":"2024-04-25T10:06:12.591906+0000","last_clean":"2024-04-25T10:06:12.591906+0000","last_became_active":"2024-04-25T10:06:11.592335+0000","last_became_peered":"2024-04-25T10:06:11.592335+0000","last_unstale":"2024-04-25T10:06:12.591906+0000","last_undegraded":"2024-04-25T10:06:12.591906+0000","last_fullsized":"2024-04-25T10:06:12.591906+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.403695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.602910+0000","last_change":"2024-04-25T10:06:10.590142+0000","last_active":"2024-04-25T10:06:11.602910+0000","last_peered":"2024-04-25T10:06:11.602910+0000","last_clean":"2024-04-25T10:06:11.602910+0000","last_became_active":"2024-04-25T10:06:10.589851+0000","last_became_peered":"2024-04-25T10:06:10.589851+0000","last_unstale":"2024-04-25T10:06:11.602910+0000","last_undegraded":"2024-04-25T10:06:11.602910+0000","last_fullsized":"2024-04-25T10:06:11.602910+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:50.142763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590901+0000","last_change":"2024-04-25T10:06:11.586712+0000","last_active":"2024-04-25T10:06:12.590901+0000","last_peered":"2024-04-25T10:06:12.590901+0000","last_clean":"2024-04-25T10:06:12.590901+0000","last_became_active":"2024-04-25T10:06:11.586393+0000","last_became_peered":"2024-04-25T10:06:11.586393+0000","last_unstale":"2024-04-25T10:06:12.590901+0000","last_undegraded":"2024-04-25T10:06:12.590901+0000","last_fullsized":"2024-04-25T10:06:12.590901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:11.365920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611691+0000","last_change":"2024-04-25T10:06:10.590906+0000","last_active":"2024-04-25T10:06:11.611691+0000","last_peered":"2024-04-25T10:06:11.611691+0000","last_clean":"2024-04-25T10:06:11.611691+0000","last_became_active":"2024-04-25T10:06:10.590641+0000","last_became_peered":"2024-04-25T10:06:10.590641+0000","last_unstale":"2024-04-25T10:06:11.611691+0000","last_undegraded":"2024-04-25T10:06:11.611691+0000","last_fullsized":"2024-04-25T10:06:11.611691+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:39.077377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591703+0000","last_change":"2024-04-25T10:06:11.601284+0000","last_active":"2024-04-25T10:06:12.591703+0000","last_peered":"2024-04-25T10:06:12.591703+0000","last_clean":"2024-04-25T10:06:12.591703+0000","last_became_active":"2024-04-25T10:06:11.600739+0000","last_became_peered":"2024-04-25T10:06:11.600739+0000","last_unstale":"2024-04-25T10:06:12.591703+0000","last_undegraded":"2024-04-25T10:06:12.591703+0000","last_fullsized":"2024-04-25T10:06:12.591703+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:03.030946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.594679+0000","last_change":"2024-04-25T10:06:10.589990+0000","last_active":"2024-04-25T10:06:11.594679+0000","last_peered":"2024-04-25T10:06:11.594679+0000","last_clean":"2024-04-25T10:06:11.594679+0000","last_became_active":"2024-04-25T10:06:10.589663+0000","last_became_peered":"2024-04-25T10:06:10.589663+0000","last_unstale":"2024-04-25T10:06:11.594679+0000","last_undegraded":"2024-04-25T10:06:11.594679+0000","last_fullsized":"2024-04-25T10:06:11.594679+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:17.017530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.611361+0000","last_change":"2024-04-25T10:06:11.608503+0000","last_active":"2024-04-25T10:06:11.611361+0000","last_peered":"2024-04-25T10:06:11.611361+0000","last_clean":"2024-04-25T10:06:11.611361+0000","last_became_active":"2024-04-25T10:06:11.608234+0000","last_became_peered":"2024-04-25T10:06:11.608234+0000","last_unstale":"2024-04-25T10:06:11.611361+0000","last_undegraded":"2024-04-25T10:06:11.611361+0000","last_fullsized":"2024-04-25T10:06:11.611361+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:11.365920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.604687+0000","last_change":"2024-04-25T10:06:10.590974+0000","last_active":"2024-04-25T10:06:11.604687+0000","last_peered":"2024-04-25T10:06:11.604687+0000","last_clean":"2024-04-25T10:06:11.604687+0000","last_became_active":"2024-04-25T10:06:10.590635+0000","last_became_peered":"2024-04-25T10:06:10.590635+0000","last_unstale":"2024-04-25T10:06:11.604687+0000","last_undegraded":"2024-04-25T10:06:11.604687+0000","last_fullsized":"2024-04-25T10:06:11.604687+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:31.888158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591335+0000","last_change":"2024-04-25T10:06:11.594569+0000","last_active":"2024-04-25T10:06:12.591335+0000","last_peered":"2024-04-25T10:06:12.591335+0000","last_clean":"2024-04-25T10:06:12.591335+0000","last_became_active":"2024-04-25T10:06:11.594016+0000","last_became_peered":"2024-04-25T10:06:11.594016+0000","last_unstale":"2024-04-25T10:06:12.591335+0000","last_undegraded":"2024-04-25T10:06:12.591335+0000","last_fullsized":"2024-04-25T10:06:12.591335+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:38.022779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590761+0000","last_change":"2024-04-25T10:06:10.591527+0000","last_active":"2024-04-25T10:06:12.590761+0000","last_peered":"2024-04-25T10:06:12.590761+0000","last_clean":"2024-04-25T10:06:12.590761+0000","last_became_active":"2024-04-25T10:06:10.591212+0000","last_became_peered":"2024-04-25T10:06:10.591212+0000","last_unstale":"2024-04-25T10:06:12.590761+0000","last_undegraded":"2024-04-25T10:06:12.590761+0000","last_fullsized":"2024-04-25T10:06:12.590761+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:17.896865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590681+0000","last_change":"2024-04-25T10:06:11.597833+0000","last_active":"2024-04-25T10:06:12.590681+0000","last_peered":"2024-04-25T10:06:12.590681+0000","last_clean":"2024-04-25T10:06:12.590681+0000","last_became_active":"2024-04-25T10:06:11.597247+0000","last_became_peered":"2024-04-25T10:06:11.597247+0000","last_unstale":"2024-04-25T10:06:12.590681+0000","last_undegraded":"2024-04-25T10:06:12.590681+0000","last_fullsized":"2024-04-25T10:06:12.590681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:36.054034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590610+0000","last_change":"2024-04-25T10:06:10.593926+0000","last_active":"2024-04-25T10:06:12.590610+0000","last_peered":"2024-04-25T10:06:12.590610+0000","last_clean":"2024-04-25T10:06:12.590610+0000","last_became_active":"2024-04-25T10:06:10.593640+0000","last_became_peered":"2024-04-25T10:06:10.593640+0000","last_unstale":"2024-04-25T10:06:12.590610+0000","last_undegraded":"2024-04-25T10:06:12.590610+0000","last_fullsized":"2024-04-25T10:06:12.590610+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:32.052858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591186+0000","last_change":"2024-04-25T10:06:11.593063+0000","last_active":"2024-04-25T10:06:12.591186+0000","last_peered":"2024-04-25T10:06:12.591186+0000","last_clean":"2024-04-25T10:06:12.591186+0000","last_became_active":"2024-04-25T10:06:11.592772+0000","last_became_peered":"2024-04-25T10:06:11.592772+0000","last_unstale":"2024-04-25T10:06:12.591186+0000","last_undegraded":"2024-04-25T10:06:12.591186+0000","last_fullsized":"2024-04-25T10:06:12.591186+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:12.627498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.584205+0000","last_change":"2024-04-25T10:06:10.594103+0000","last_active":"2024-04-25T10:06:11.584205+0000","last_peered":"2024-04-25T10:06:11.584205+0000","last_clean":"2024-04-25T10:06:11.584205+0000","last_became_active":"2024-04-25T10:06:10.593805+0000","last_became_peered":"2024-04-25T10:06:10.593805+0000","last_unstale":"2024-04-25T10:06:11.584205+0000","last_undegraded":"2024-04-25T10:06:11.584205+0000","last_fullsized":"2024-04-25T10:06:11.584205+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:21.758145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.597476+0000","last_change":"2024-04-25T10:06:11.590223+0000","last_active":"2024-04-25T10:06:11.597476+0000","last_peered":"2024-04-25T10:06:11.597476+0000","last_clean":"2024-04-25T10:06:11.597476+0000","last_became_active":"2024-04-25T10:06:11.589970+0000","last_became_peered":"2024-04-25T10:06:11.589970+0000","last_unstale":"2024-04-25T10:06:11.597476+0000","last_undegraded":"2024-04-25T10:06:11.597476+0000","last_fullsized":"2024-04-25T10:06:11.597476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:44.396817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590364+0000","last_change":"2024-04-25T10:06:10.590741+0000","last_active":"2024-04-25T10:06:12.590364+0000","last_peered":"2024-04-25T10:06:12.590364+0000","last_clean":"2024-04-25T10:06:12.590364+0000","last_became_active":"2024-04-25T10:06:10.590499+0000","last_became_peered":"2024-04-25T10:06:10.590499+0000","last_unstale":"2024-04-25T10:06:12.590364+0000","last_undegraded":"2024-04-25T10:06:12.590364+0000","last_fullsized":"2024-04-25T10:06:12.590364+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:32.676160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599229+0000","last_change":"2024-04-25T10:06:11.594584+0000","last_active":"2024-04-25T10:06:11.599229+0000","last_peered":"2024-04-25T10:06:11.599229+0000","last_clean":"2024-04-25T10:06:11.599229+0000","last_became_active":"2024-04-25T10:06:11.594264+0000","last_became_peered":"2024-04-25T10:06:11.594264+0000","last_unstale":"2024-04-25T10:06:11.599229+0000","last_undegraded":"2024-04-25T10:06:11.599229+0000","last_fullsized":"2024-04-25T10:06:11.599229+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:33.855981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:13.034316+0000","last_change":"2024-04-25T10:06:10.591478+0000","last_active":"2024-04-25T10:06:13.034316+0000","last_peered":"2024-04-25T10:06:13.034316+0000","last_clean":"2024-04-25T10:06:13.034316+0000","last_became_active":"2024-04-25T10:06:10.591156+0000","last_became_peered":"2024-04-25T10:06:10.591156+0000","last_unstale":"2024-04-25T10:06:13.034316+0000","last_undegraded":"2024-04-25T10:06:13.034316+0000","last_fullsized":"2024-04-25T10:06:13.034316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:13.895648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.603137+0000","last_change":"2024-04-25T10:06:11.598368+0000","last_active":"2024-04-25T10:06:11.603137+0000","last_peered":"2024-04-25T10:06:11.603137+0000","last_clean":"2024-04-25T10:06:11.603137+0000","last_became_active":"2024-04-25T10:06:11.597963+0000","last_became_peered":"2024-04-25T10:06:11.597963+0000","last_unstale":"2024-04-25T10:06:11.603137+0000","last_undegraded":"2024-04-25T10:06:11.603137+0000","last_fullsized":"2024-04-25T10:06:11.603137+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:48.190466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:13.033777+0000","last_change":"2024-04-25T10:06:10.594136+0000","last_active":"2024-04-25T10:06:13.033777+0000","last_peered":"2024-04-25T10:06:13.033777+0000","last_clean":"2024-04-25T10:06:13.033777+0000","last_became_active":"2024-04-25T10:06:10.593885+0000","last_became_peered":"2024-04-25T10:06:10.593885+0000","last_unstale":"2024-04-25T10:06:13.033777+0000","last_undegraded":"2024-04-25T10:06:13.033777+0000","last_fullsized":"2024-04-25T10:06:13.033777+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:20.814527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610787+0000","last_change":"2024-04-25T10:06:11.603945+0000","last_active":"2024-04-25T10:06:11.610787+0000","last_peered":"2024-04-25T10:06:11.610787+0000","last_clean":"2024-04-25T10:06:11.610787+0000","last_became_active":"2024-04-25T10:06:11.602849+0000","last_became_peered":"2024-04-25T10:06:11.602849+0000","last_unstale":"2024-04-25T10:06:11.610787+0000","last_undegraded":"2024-04-25T10:06:11.610787+0000","last_fullsized":"2024-04-25T10:06:11.610787+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:03.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.601893+0000","last_change":"2024-04-25T10:06:10.579763+0000","last_active":"2024-04-25T10:06:11.601893+0000","last_peered":"2024-04-25T10:06:11.601893+0000","last_clean":"2024-04-25T10:06:11.601893+0000","last_became_active":"2024-04-25T10:06:10.579495+0000","last_became_peered":"2024-04-25T10:06:10.579495+0000","last_unstale":"2024-04-25T10:06:11.601893+0000","last_undegraded":"2024-04-25T10:06:11.601893+0000","last_fullsized":"2024-04-25T10:06:11.601893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:08.212125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.603845+0000","last_change":"2024-04-25T10:06:11.603317+0000","last_active":"2024-04-25T10:06:11.603845+0000","last_peered":"2024-04-25T10:06:11.603845+0000","last_clean":"2024-04-25T10:06:11.603845+0000","last_became_active":"2024-04-25T10:06:11.603066+0000","last_became_peered":"2024-04-25T10:06:11.603066+0000","last_unstale":"2024-04-25T10:06:11.603845+0000","last_undegraded":"2024-04-25T10:06:11.603845+0000","last_fullsized":"2024-04-25T10:06:11.603845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:07.686632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592122+0000","last_change":"2024-04-25T10:06:10.593257+0000","last_active":"2024-04-25T10:06:12.592122+0000","last_peered":"2024-04-25T10:06:12.592122+0000","last_clean":"2024-04-25T10:06:12.592122+0000","last_became_active":"2024-04-25T10:06:10.592737+0000","last_became_peered":"2024-04-25T10:06:10.592737+0000","last_unstale":"2024-04-25T10:06:12.592122+0000","last_undegraded":"2024-04-25T10:06:12.592122+0000","last_fullsized":"2024-04-25T10:06:12.592122+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:38.738216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591639+0000","last_change":"2024-04-25T10:06:11.601746+0000","last_active":"2024-04-25T10:06:12.591639+0000","last_peered":"2024-04-25T10:06:12.591639+0000","last_clean":"2024-04-25T10:06:12.591639+0000","last_became_active":"2024-04-25T10:06:11.600706+0000","last_became_peered":"2024-04-25T10:06:11.600706+0000","last_unstale":"2024-04-25T10:06:12.591639+0000","last_undegraded":"2024-04-25T10:06:12.591639+0000","last_fullsized":"2024-04-25T10:06:12.591639+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:37.328665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590790+0000","last_change":"2024-04-25T10:06:10.589747+0000","last_active":"2024-04-25T10:06:12.590790+0000","last_peered":"2024-04-25T10:06:12.590790+0000","last_clean":"2024-04-25T10:06:12.590790+0000","last_became_active":"2024-04-25T10:06:10.589483+0000","last_became_peered":"2024-04-25T10:06:10.589483+0000","last_unstale":"2024-04-25T10:06:12.590790+0000","last_undegraded":"2024-04-25T10:06:12.590790+0000","last_fullsized":"2024-04-25T10:06:12.590790+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.133966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.605963+0000","last_change":"2024-04-25T10:06:11.599329+0000","last_active":"2024-04-25T10:06:11.605963+0000","last_peered":"2024-04-25T10:06:11.605963+0000","last_clean":"2024-04-25T10:06:11.605963+0000","last_became_active":"2024-04-25T10:06:11.599055+0000","last_became_peered":"2024-04-25T10:06:11.599055+0000","last_unstale":"2024-04-25T10:06:11.605963+0000","last_undegraded":"2024-04-25T10:06:11.605963+0000","last_fullsized":"2024-04-25T10:06:11.605963+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:04.221854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.590118+0000","last_change":"2024-04-25T10:06:10.592105+0000","last_active":"2024-04-25T10:06:11.590118+0000","last_peered":"2024-04-25T10:06:11.590118+0000","last_clean":"2024-04-25T10:06:11.590118+0000","last_became_active":"2024-04-25T10:06:10.591823+0000","last_became_peered":"2024-04-25T10:06:10.591823+0000","last_unstale":"2024-04-25T10:06:11.590118+0000","last_undegraded":"2024-04-25T10:06:11.590118+0000","last_fullsized":"2024-04-25T10:06:11.590118+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:53.425769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590360+0000","last_change":"2024-04-25T10:06:11.585750+0000","last_active":"2024-04-25T10:06:12.590360+0000","last_peered":"2024-04-25T10:06:12.590360+0000","last_clean":"2024-04-25T10:06:12.590360+0000","last_became_active":"2024-04-25T10:06:11.585193+0000","last_became_peered":"2024-04-25T10:06:11.585193+0000","last_unstale":"2024-04-25T10:06:12.590360+0000","last_undegraded":"2024-04-25T10:06:12.590360+0000","last_fullsized":"2024-04-25T10:06:12.590360+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:03.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.589817+0000","last_change":"2024-04-25T10:06:10.590311+0000","last_active":"2024-04-25T10:06:11.589817+0000","last_peered":"2024-04-25T10:06:11.589817+0000","last_clean":"2024-04-25T10:06:11.589817+0000","last_became_active":"2024-04-25T10:06:10.590015+0000","last_became_peered":"2024-04-25T10:06:10.590015+0000","last_unstale":"2024-04-25T10:06:11.589817+0000","last_undegraded":"2024-04-25T10:06:11.589817+0000","last_fullsized":"2024-04-25T10:06:11.589817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:27.524048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.585155+0000","last_change":"2024-04-25T10:06:11.582381+0000","last_active":"2024-04-25T10:06:11.585155+0000","last_peered":"2024-04-25T10:06:11.585155+0000","last_clean":"2024-04-25T10:06:11.585155+0000","last_became_active":"2024-04-25T10:06:11.582109+0000","last_became_peered":"2024-04-25T10:06:11.582109+0000","last_unstale":"2024-04-25T10:06:11.585155+0000","last_undegraded":"2024-04-25T10:06:11.585155+0000","last_fullsized":"2024-04-25T10:06:11.585155+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:41.604772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.604177+0000","last_change":"2024-04-25T10:06:10.589625+0000","last_active":"2024-04-25T10:06:11.604177+0000","last_peered":"2024-04-25T10:06:11.604177+0000","last_clean":"2024-04-25T10:06:11.604177+0000","last_became_active":"2024-04-25T10:06:10.589161+0000","last_became_peered":"2024-04-25T10:06:10.589161+0000","last_unstale":"2024-04-25T10:06:11.604177+0000","last_undegraded":"2024-04-25T10:06:11.604177+0000","last_fullsized":"2024-04-25T10:06:11.604177+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:47.673088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.612662+0000","last_change":"2024-04-25T10:06:11.612101+0000","last_active":"2024-04-25T10:06:11.612662+0000","last_peered":"2024-04-25T10:06:11.612662+0000","last_clean":"2024-04-25T10:06:11.612662+0000","last_became_active":"2024-04-25T10:06:11.611805+0000","last_became_peered":"2024-04-25T10:06:11.611805+0000","last_unstale":"2024-04-25T10:06:11.612662+0000","last_undegraded":"2024-04-25T10:06:11.612662+0000","last_fullsized":"2024-04-25T10:06:11.612662+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:22.287749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.598493+0000","last_change":"2024-04-25T10:06:10.594406+0000","last_active":"2024-04-25T10:06:11.598493+0000","last_peered":"2024-04-25T10:06:11.598493+0000","last_clean":"2024-04-25T10:06:11.598493+0000","last_became_active":"2024-04-25T10:06:10.594129+0000","last_became_peered":"2024-04-25T10:06:10.594129+0000","last_unstale":"2024-04-25T10:06:11.598493+0000","last_undegraded":"2024-04-25T10:06:11.598493+0000","last_fullsized":"2024-04-25T10:06:11.598493+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:19.498453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591710+0000","last_change":"2024-04-25T10:06:11.592494+0000","last_active":"2024-04-25T10:06:12.591710+0000","last_peered":"2024-04-25T10:06:12.591710+0000","last_clean":"2024-04-25T10:06:12.591710+0000","last_became_active":"2024-04-25T10:06:11.592212+0000","last_became_peered":"2024-04-25T10:06:11.592212+0000","last_unstale":"2024-04-25T10:06:12.591710+0000","last_undegraded":"2024-04-25T10:06:12.591710+0000","last_fullsized":"2024-04-25T10:06:12.591710+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:08.065111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590880+0000","last_change":"2024-04-25T10:06:10.588563+0000","last_active":"2024-04-25T10:06:12.590880+0000","last_peered":"2024-04-25T10:06:12.590880+0000","last_clean":"2024-04-25T10:06:12.590880+0000","last_became_active":"2024-04-25T10:06:10.588312+0000","last_became_peered":"2024-04-25T10:06:10.588312+0000","last_unstale":"2024-04-25T10:06:12.590880+0000","last_undegraded":"2024-04-25T10:06:12.590880+0000","last_fullsized":"2024-04-25T10:06:12.590880+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:17.315120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:13.033642+0000","last_change":"2024-04-25T10:06:11.586327+0000","last_active":"2024-04-25T10:06:13.033642+0000","last_peered":"2024-04-25T10:06:13.033642+0000","last_clean":"2024-04-25T10:06:13.033642+0000","last_became_active":"2024-04-25T10:06:11.586002+0000","last_became_peered":"2024-04-25T10:06:11.586002+0000","last_unstale":"2024-04-25T10:06:13.033642+0000","last_undegraded":"2024-04-25T10:06:13.033642+0000","last_fullsized":"2024-04-25T10:06:13.033642+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:59.859154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.602393+0000","last_change":"2024-04-25T10:06:10.588662+0000","last_active":"2024-04-25T10:06:11.602393+0000","last_peered":"2024-04-25T10:06:11.602393+0000","last_clean":"2024-04-25T10:06:11.602393+0000","last_became_active":"2024-04-25T10:06:10.588316+0000","last_became_peered":"2024-04-25T10:06:10.588316+0000","last_unstale":"2024-04-25T10:06:11.602393+0000","last_undegraded":"2024-04-25T10:06:11.602393+0000","last_fullsized":"2024-04-25T10:06:11.602393+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:32.850838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599469+0000","last_change":"2024-04-25T10:06:11.595815+0000","last_active":"2024-04-25T10:06:11.599469+0000","last_peered":"2024-04-25T10:06:11.599469+0000","last_clean":"2024-04-25T10:06:11.599469+0000","last_became_active":"2024-04-25T10:06:11.595495+0000","last_became_peered":"2024-04-25T10:06:11.595495+0000","last_unstale":"2024-04-25T10:06:11.599469+0000","last_undegraded":"2024-04-25T10:06:11.599469+0000","last_fullsized":"2024-04-25T10:06:11.599469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:12.627498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.581094+0000","last_change":"2024-04-25T10:06:10.591774+0000","last_active":"2024-04-25T10:06:11.581094+0000","last_peered":"2024-04-25T10:06:11.581094+0000","last_clean":"2024-04-25T10:06:11.581094+0000","last_became_active":"2024-04-25T10:06:10.591412+0000","last_became_peered":"2024-04-25T10:06:10.591412+0000","last_unstale":"2024-04-25T10:06:11.581094+0000","last_undegraded":"2024-04-25T10:06:11.581094+0000","last_fullsized":"2024-04-25T10:06:11.581094+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:17.401215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591675+0000","last_change":"2024-04-25T10:06:11.593318+0000","last_active":"2024-04-25T10:06:12.591675+0000","last_peered":"2024-04-25T10:06:12.591675+0000","last_clean":"2024-04-25T10:06:12.591675+0000","last_became_active":"2024-04-25T10:06:11.593022+0000","last_became_peered":"2024-04-25T10:06:11.593022+0000","last_unstale":"2024-04-25T10:06:12.591675+0000","last_undegraded":"2024-04-25T10:06:12.591675+0000","last_fullsized":"2024-04-25T10:06:12.591675+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:46.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:13.033872+0000","last_change":"2024-04-25T10:06:10.590716+0000","last_active":"2024-04-25T10:06:13.033872+0000","last_peered":"2024-04-25T10:06:13.033872+0000","last_clean":"2024-04-25T10:06:13.033872+0000","last_became_active":"2024-04-25T10:06:10.590427+0000","last_became_peered":"2024-04-25T10:06:10.590427+0000","last_unstale":"2024-04-25T10:06:13.033872+0000","last_undegraded":"2024-04-25T10:06:13.033872+0000","last_fullsized":"2024-04-25T10:06:13.033872+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:56.243387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:13.033938+0000","last_change":"2024-04-25T10:06:11.595359+0000","last_active":"2024-04-25T10:06:13.033938+0000","last_peered":"2024-04-25T10:06:13.033938+0000","last_clean":"2024-04-25T10:06:13.033938+0000","last_became_active":"2024-04-25T10:06:11.595037+0000","last_became_peered":"2024-04-25T10:06:11.595037+0000","last_unstale":"2024-04-25T10:06:13.033938+0000","last_undegraded":"2024-04-25T10:06:13.033938+0000","last_fullsized":"2024-04-25T10:06:13.033938+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:49.489109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590609+0000","last_change":"2024-04-25T10:06:10.590140+0000","last_active":"2024-04-25T10:06:12.590609+0000","last_peered":"2024-04-25T10:06:12.590609+0000","last_clean":"2024-04-25T10:06:12.590609+0000","last_became_active":"2024-04-25T10:06:10.589015+0000","last_became_peered":"2024-04-25T10:06:10.589015+0000","last_unstale":"2024-04-25T10:06:12.590609+0000","last_undegraded":"2024-04-25T10:06:12.590609+0000","last_fullsized":"2024-04-25T10:06:12.590609+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:18.383218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591617+0000","last_change":"2024-04-25T10:06:11.604928+0000","last_active":"2024-04-25T10:06:12.591617+0000","last_peered":"2024-04-25T10:06:12.591617+0000","last_clean":"2024-04-25T10:06:12.591617+0000","last_became_active":"2024-04-25T10:06:11.604636+0000","last_became_peered":"2024-04-25T10:06:11.604636+0000","last_unstale":"2024-04-25T10:06:12.591617+0000","last_undegraded":"2024-04-25T10:06:12.591617+0000","last_fullsized":"2024-04-25T10:06:12.591617+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:04.221854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591565+0000","last_change":"2024-04-25T10:06:10.590105+0000","last_active":"2024-04-25T10:06:12.591565+0000","last_peered":"2024-04-25T10:06:12.591565+0000","last_clean":"2024-04-25T10:06:12.591565+0000","last_became_active":"2024-04-25T10:06:10.589829+0000","last_became_peered":"2024-04-25T10:06:10.589829+0000","last_unstale":"2024-04-25T10:06:12.591565+0000","last_undegraded":"2024-04-25T10:06:12.591565+0000","last_fullsized":"2024-04-25T10:06:12.591565+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:23.745526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591751+0000","last_change":"2024-04-25T10:06:11.593413+0000","last_active":"2024-04-25T10:06:12.591751+0000","last_peered":"2024-04-25T10:06:12.591751+0000","last_clean":"2024-04-25T10:06:12.591751+0000","last_became_active":"2024-04-25T10:06:11.593155+0000","last_became_peered":"2024-04-25T10:06:11.593155+0000","last_unstale":"2024-04-25T10:06:12.591751+0000","last_undegraded":"2024-04-25T10:06:12.591751+0000","last_fullsized":"2024-04-25T10:06:12.591751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:33.678639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.579429+0000","last_change":"2024-04-25T10:06:10.587062+0000","last_active":"2024-04-25T10:06:11.579429+0000","last_peered":"2024-04-25T10:06:11.579429+0000","last_clean":"2024-04-25T10:06:11.579429+0000","last_became_active":"2024-04-25T10:06:10.586736+0000","last_became_peered":"2024-04-25T10:06:10.586736+0000","last_unstale":"2024-04-25T10:06:11.579429+0000","last_undegraded":"2024-04-25T10:06:11.579429+0000","last_fullsized":"2024-04-25T10:06:11.579429+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:48.572483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.599597+0000","last_change":"2024-04-25T10:06:11.595699+0000","last_active":"2024-04-25T10:06:11.599597+0000","last_peered":"2024-04-25T10:06:11.599597+0000","last_clean":"2024-04-25T10:06:11.599597+0000","last_became_active":"2024-04-25T10:06:11.595437+0000","last_became_peered":"2024-04-25T10:06:11.595437+0000","last_unstale":"2024-04-25T10:06:11.599597+0000","last_undegraded":"2024-04-25T10:06:11.599597+0000","last_fullsized":"2024-04-25T10:06:11.599597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:38.022779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591835+0000","last_change":"2024-04-25T10:06:10.588344+0000","last_active":"2024-04-25T10:06:12.591835+0000","last_peered":"2024-04-25T10:06:12.591835+0000","last_clean":"2024-04-25T10:06:12.591835+0000","last_became_active":"2024-04-25T10:06:10.587774+0000","last_became_peered":"2024-04-25T10:06:10.587774+0000","last_unstale":"2024-04-25T10:06:12.591835+0000","last_undegraded":"2024-04-25T10:06:12.591835+0000","last_fullsized":"2024-04-25T10:06:12.591835+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:13.625296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.603613+0000","last_change":"2024-04-25T10:06:11.603042+0000","last_active":"2024-04-25T10:06:11.603613+0000","last_peered":"2024-04-25T10:06:11.603613+0000","last_clean":"2024-04-25T10:06:11.603613+0000","last_became_active":"2024-04-25T10:06:11.598535+0000","last_became_peered":"2024-04-25T10:06:11.598535+0000","last_unstale":"2024-04-25T10:06:11.603613+0000","last_undegraded":"2024-04-25T10:06:11.603613+0000","last_fullsized":"2024-04-25T10:06:11.603613+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:31.070811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:13.033932+0000","last_change":"2024-04-25T10:06:10.592907+0000","last_active":"2024-04-25T10:06:13.033932+0000","last_peered":"2024-04-25T10:06:13.033932+0000","last_clean":"2024-04-25T10:06:13.033932+0000","last_became_active":"2024-04-25T10:06:10.592637+0000","last_became_peered":"2024-04-25T10:06:10.592637+0000","last_unstale":"2024-04-25T10:06:13.033932+0000","last_undegraded":"2024-04-25T10:06:13.033932+0000","last_fullsized":"2024-04-25T10:06:13.033932+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:52.984279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.605940+0000","last_change":"2024-04-25T10:06:11.603834+0000","last_active":"2024-04-25T10:06:11.605940+0000","last_peered":"2024-04-25T10:06:11.605940+0000","last_clean":"2024-04-25T10:06:11.605940+0000","last_became_active":"2024-04-25T10:06:11.603541+0000","last_became_peered":"2024-04-25T10:06:11.603541+0000","last_unstale":"2024-04-25T10:06:11.605940+0000","last_undegraded":"2024-04-25T10:06:11.605940+0000","last_fullsized":"2024-04-25T10:06:11.605940+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:08.065111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592173+0000","last_change":"2024-04-25T10:06:10.590856+0000","last_active":"2024-04-25T10:06:12.592173+0000","last_peered":"2024-04-25T10:06:12.592173+0000","last_clean":"2024-04-25T10:06:12.592173+0000","last_became_active":"2024-04-25T10:06:10.590571+0000","last_became_peered":"2024-04-25T10:06:10.590571+0000","last_unstale":"2024-04-25T10:06:12.592173+0000","last_undegraded":"2024-04-25T10:06:12.592173+0000","last_fullsized":"2024-04-25T10:06:12.592173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:01.814874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.596509+0000","last_change":"2024-04-25T10:06:11.595775+0000","last_active":"2024-04-25T10:06:11.596509+0000","last_peered":"2024-04-25T10:06:11.596509+0000","last_clean":"2024-04-25T10:06:11.596509+0000","last_became_active":"2024-04-25T10:06:11.595387+0000","last_became_peered":"2024-04-25T10:06:11.595387+0000","last_unstale":"2024-04-25T10:06:11.596509+0000","last_undegraded":"2024-04-25T10:06:11.596509+0000","last_fullsized":"2024-04-25T10:06:11.596509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:40.079857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.592115+0000","last_change":"2024-04-25T10:06:10.588101+0000","last_active":"2024-04-25T10:06:12.592115+0000","last_peered":"2024-04-25T10:06:12.592115+0000","last_clean":"2024-04-25T10:06:12.592115+0000","last_became_active":"2024-04-25T10:06:10.587808+0000","last_became_peered":"2024-04-25T10:06:10.587808+0000","last_unstale":"2024-04-25T10:06:12.592115+0000","last_undegraded":"2024-04-25T10:06:12.592115+0000","last_fullsized":"2024-04-25T10:06:12.592115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.133966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610579+0000","last_change":"2024-04-25T10:06:11.599755+0000","last_active":"2024-04-25T10:06:11.610579+0000","last_peered":"2024-04-25T10:06:11.610579+0000","last_clean":"2024-04-25T10:06:11.610579+0000","last_became_active":"2024-04-25T10:06:11.599317+0000","last_became_peered":"2024-04-25T10:06:11.599317+0000","last_unstale":"2024-04-25T10:06:11.610579+0000","last_undegraded":"2024-04-25T10:06:11.610579+0000","last_fullsized":"2024-04-25T10:06:11.610579+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:37.328665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591917+0000","last_change":"2024-04-25T10:06:10.591156+0000","last_active":"2024-04-25T10:06:12.591917+0000","last_peered":"2024-04-25T10:06:12.591917+0000","last_clean":"2024-04-25T10:06:12.591917+0000","last_became_active":"2024-04-25T10:06:10.590362+0000","last_became_peered":"2024-04-25T10:06:10.590362+0000","last_unstale":"2024-04-25T10:06:12.591917+0000","last_undegraded":"2024-04-25T10:06:12.591917+0000","last_fullsized":"2024-04-25T10:06:12.591917+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:32.052858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.610758+0000","last_change":"2024-04-25T10:06:11.603491+0000","last_active":"2024-04-25T10:06:11.610758+0000","last_peered":"2024-04-25T10:06:11.610758+0000","last_clean":"2024-04-25T10:06:11.610758+0000","last_became_active":"2024-04-25T10:06:11.600221+0000","last_became_peered":"2024-04-25T10:06:11.600221+0000","last_unstale":"2024-04-25T10:06:11.610758+0000","last_undegraded":"2024-04-25T10:06:11.610758+0000","last_fullsized":"2024-04-25T10:06:11.610758+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:01.970455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.590413+0000","last_change":"2024-04-25T10:06:10.589304+0000","last_active":"2024-04-25T10:06:12.590413+0000","last_peered":"2024-04-25T10:06:12.590413+0000","last_clean":"2024-04-25T10:06:12.590413+0000","last_became_active":"2024-04-25T10:06:10.588934+0000","last_became_peered":"2024-04-25T10:06:10.588934+0000","last_unstale":"2024-04-25T10:06:12.590413+0000","last_undegraded":"2024-04-25T10:06:12.590413+0000","last_fullsized":"2024-04-25T10:06:12.590413+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:07.062632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.602797+0000","last_change":"2024-04-25T10:06:11.597255+0000","last_active":"2024-04-25T10:06:11.602797+0000","last_peered":"2024-04-25T10:06:11.602797+0000","last_clean":"2024-04-25T10:06:11.602797+0000","last_became_active":"2024-04-25T10:06:11.596954+0000","last_became_peered":"2024-04-25T10:06:11.596954+0000","last_unstale":"2024-04-25T10:06:11.602797+0000","last_undegraded":"2024-04-25T10:06:11.602797+0000","last_fullsized":"2024-04-25T10:06:11.602797+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:13.167235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.596522+0000","last_change":"2024-04-25T10:06:10.589615+0000","last_active":"2024-04-25T10:06:11.596522+0000","last_peered":"2024-04-25T10:06:11.596522+0000","last_clean":"2024-04-25T10:06:11.596522+0000","last_became_active":"2024-04-25T10:06:10.589310+0000","last_became_peered":"2024-04-25T10:06:10.589310+0000","last_unstale":"2024-04-25T10:06:11.596522+0000","last_undegraded":"2024-04-25T10:06:11.596522+0000","last_fullsized":"2024-04-25T10:06:11.596522+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:00.780604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.597484+0000","last_change":"2024-04-25T10:06:11.596201+0000","last_active":"2024-04-25T10:06:11.597484+0000","last_peered":"2024-04-25T10:06:11.597484+0000","last_clean":"2024-04-25T10:06:11.597484+0000","last_became_active":"2024-04-25T10:06:11.595880+0000","last_became_peered":"2024-04-25T10:06:11.595880+0000","last_unstale":"2024-04-25T10:06:11.597484+0000","last_undegraded":"2024-04-25T10:06:11.597484+0000","last_fullsized":"2024-04-25T10:06:11.597484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:10.566145+0000","last_clean_scrub_stamp":"2024-04-25T10:06:10.566145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:36.054034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:11.580878+0000","last_change":"2024-04-25T10:06:10.587821+0000","last_active":"2024-04-25T10:06:11.580878+0000","last_peered":"2024-04-25T10:06:11.580878+0000","last_clean":"2024-04-25T10:06:11.580878+0000","last_became_active":"2024-04-25T10:06:10.587520+0000","last_became_peered":"2024-04-25T10:06:10.587520+0000","last_unstale":"2024-04-25T10:06:11.580878+0000","last_undegraded":"2024-04-25T10:06:11.580878+0000","last_fullsized":"2024-04-25T10:06:11.580878+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:09.563666+0000","last_clean_scrub_stamp":"2024-04-25T10:06:09.563666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:17.031658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":198,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:06:12.591162+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:06:12.591162+0000","last_peered":"2024-04-25T10:06:12.591162+0000","last_clean":"2024-04-25T10:06:12.591162+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:06:12.591162+0000","last_undegraded":"2024-04-25T10:06:12.591162+0000","last_fullsized":"2024-04-25T10:06:12.591162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":150789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":152379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":152401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":405504,"data_stored":151543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":393216,"data_stored":150801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":747505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":393216,"data_stored":150857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":748017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:19.343 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:06:19.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:06:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.484+0000 7f65d8cdd640 1 Processor -- start 2024-04-25T10:06:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.484+0000 7f65d8cdd640 1 -- start start 2024-04-25T10:06:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.485+0000 7f65d8cdd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f65d40d1eb0 0x7f65d41688a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.485+0000 7f65d8cdd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d40cc860 con 0x7f65d415de70 2024-04-25T10:06:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.485+0000 7f65d8cdd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d40ccb40 con 0x7f65d406d620 2024-04-25T10:06:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.485+0000 7f65d8cdd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d40cc620 con 0x7f65d40d1eb0 2024-04-25T10:06:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.485+0000 7f65d2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f65d40d1eb0 0x7f65d41688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.485+0000 7f65d259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f65d415de70 0x7f65d40cbf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56548/0 (socket says 172.21.15.16:56548) 2024-04-25T10:06:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.485+0000 7f65d259c640 1 -- 172.21.15.16:0/4259971103 learned_addr learned my addr 172.21.15.16:0/4259971103 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.486+0000 7f65d159a640 1 -- 172.21.15.16:0/4259971103 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3006896737 0 0) 0x7f65d40cc860 con 0x7f65d415de70 2024-04-25T10:06:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.486+0000 7f65d159a640 1 -- 172.21.15.16:0/4259971103 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65c0003520 con 0x7f65d415de70 2024-04-25T10:06:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.486+0000 7f65d2d9d640 1 -- 172.21.15.16:0/4259971103 >> v1:172.21.15.186:6789/0 conn(0x7f65d406d620 legacy=0x7f65d40d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.486+0000 7f65d2d9d640 1 -- 172.21.15.16:0/4259971103 >> v1:172.21.15.16:6789/0 conn(0x7f65d415de70 legacy=0x7f65d40cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.486+0000 7f65d2d9d640 1 -- 172.21.15.16:0/4259971103 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d40cce20 con 0x7f65d40d1eb0 2024-04-25T10:06:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.486+0000 7f65d2d9d640 1 --2- 172.21.15.16:0/4259971103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65d40d1eb0 0x7f65d41688a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009dc0 tx=0x7f65c402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8664fd2486ee6c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.487+0000 7f65d159a640 1 -- 172.21.15.16:0/4259971103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4037070 con 0x7f65d40d1eb0 2024-04-25T10:06:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.487+0000 7f65d159a640 1 -- 172.21.15.16:0/4259971103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c4007740 con 0x7f65d40d1eb0 2024-04-25T10:06:19.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.487+0000 7f65d159a640 1 -- 172.21.15.16:0/4259971103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4007b10 con 0x7f65d40d1eb0 2024-04-25T10:06:19.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.487+0000 7f65d8cdd640 1 -- 172.21.15.16:0/4259971103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65d40d1eb0 msgr2=0x7f65d41688a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.487+0000 7f65d8cdd640 1 --2- 172.21.15.16:0/4259971103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65d40d1eb0 0x7f65d41688a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009dc0 tx=0x7f65c402a7c0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.487+0000 7f65d8cdd640 1 -- 172.21.15.16:0/4259971103 shutdown_connections 2024-04-25T10:06:19.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.487+0000 7f65d8cdd640 1 --2- 172.21.15.16:0/4259971103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65d40d1eb0 0x7f65d41688a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.487+0000 7f65d8cdd640 1 -- 172.21.15.16:0/4259971103 >> 172.21.15.16:0/4259971103 conn(0x7f65d40b33e0 msgr2=0x7f65d40b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.487+0000 7f65d8cdd640 1 -- 172.21.15.16:0/4259971103 shutdown_connections 2024-04-25T10:06:19.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.487+0000 7f65d8cdd640 1 -- 172.21.15.16:0/4259971103 wait complete. 2024-04-25T10:06:19.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.489+0000 7f65d8cdd640 1 Processor -- start 2024-04-25T10:06:19.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.489+0000 7f65d8cdd640 1 -- start start 2024-04-25T10:06:19.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.489+0000 7f65d8cdd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65d406d620 0x7f65d41e9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.489+0000 7f65d8cdd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65d40d1eb0 0x7f65d4155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.489+0000 7f65d8cdd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65d415de70 0x7f65d40ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.489+0000 7f65d8cdd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d40ce910 con 0x7f65d406d620 2024-04-25T10:06:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.489+0000 7f65d8cdd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d40cd3e0 con 0x7f65d415de70 2024-04-25T10:06:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.489+0000 7f65d8cdd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d40cea90 con 0x7f65d40d1eb0 2024-04-25T10:06:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.489+0000 7f65d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65d406d620 0x7f65d41e9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.489+0000 7f65d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65d406d620 0x7f65d41e9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49552/0 (socket says 172.21.15.16:49552) 2024-04-25T10:06:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.489+0000 7f65d259c640 1 -- 172.21.15.16:0/3343696710 learned_addr learned my addr 172.21.15.16:0/3343696710 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.490+0000 7f65d2d9d640 1 --2- 172.21.15.16:0/3343696710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65d415de70 0x7f65d40ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.490+0000 7f65d259c640 1 -- 172.21.15.16:0/3343696710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65d40d1eb0 msgr2=0x7f65d4155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.490+0000 7f65d259c640 1 --2- 172.21.15.16:0/3343696710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65d40d1eb0 0x7f65d4155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.490+0000 7f65d259c640 1 -- 172.21.15.16:0/3343696710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65d415de70 msgr2=0x7f65d40ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.490+0000 7f65d259c640 1 --2- 172.21.15.16:0/3343696710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65d415de70 0x7f65d40ca4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.490+0000 7f65d259c640 1 -- 172.21.15.16:0/3343696710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d40cac80 con 0x7f65d406d620 2024-04-25T10:06:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.490+0000 7f65d259c640 1 --2- 172.21.15.16:0/3343696710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65d406d620 0x7f65d41e9cf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00c500 tx=0x7f65cc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.490+0000 7f65caffd640 1 -- 172.21.15.16:0/3343696710 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc017070 con 0x7f65d406d620 2024-04-25T10:06:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.490+0000 7f65caffd640 1 -- 172.21.15.16:0/3343696710 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65cc005ce0 con 0x7f65d406d620 2024-04-25T10:06:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.490+0000 7f65caffd640 1 -- 172.21.15.16:0/3343696710 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc007330 con 0x7f65d406d620 2024-04-25T10:06:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.490+0000 7f65d8cdd640 1 -- 172.21.15.16:0/3343696710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d40caf60 con 0x7f65d406d620 2024-04-25T10:06:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.490+0000 7f65d8cdd640 1 -- 172.21.15.16:0/3343696710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d415e530 con 0x7f65d406d620 2024-04-25T10:06:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.491+0000 7f65d8cdd640 1 -- 172.21.15.16:0/3343696710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d406de40 con 0x7f65d406d620 2024-04-25T10:06:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.491+0000 7f65caffd640 1 -- 172.21.15.16:0/3343696710 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65cc01a070 con 0x7f65d406d620 2024-04-25T10:06:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.492+0000 7f65caffd640 1 --2- 172.21.15.16:0/3343696710 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f65b406b500 0x7f65b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.492+0000 7f65caffd640 1 -- 172.21.15.16:0/3343696710 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65cc089650 con 0x7f65d406d620 2024-04-25T10:06:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.492+0000 7f65d1d9b640 1 --2- 172.21.15.16:0/3343696710 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f65b406b500 0x7f65b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.493+0000 7f65d1d9b640 1 --2- 172.21.15.16:0/3343696710 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f65b406b500 0x7f65b406d9c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f65bc000f60 tx=0x7f65bc0079d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.496+0000 7f65caffd640 1 -- 172.21.15.16:0/3343696710 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65cc058580 con 0x7f65d406d620 2024-04-25T10:06:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.752+0000 7f65d8cdd640 1 -- 172.21.15.16:0/3343696710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f65d40ce6f0 con 0x7f65d406d620 2024-04-25T10:06:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.754+0000 7f65caffd640 1 -- 172.21.15.16:0/3343696710 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f65cc057e80 con 0x7f65d406d620 2024-04-25T10:06:19.760 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-04-25T10:06:19.760 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCZKipmAph/GhAAG3x1zC98yHQ13fjpmrlPkg== 2024-04-25T10:06:19.760 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-04-25T10:06:19.760 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-04-25T10:06:19.760 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:06:19.760 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-04-25T10:06:19.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.756+0000 7f65d8cdd640 1 -- 172.21.15.16:0/3343696710 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f65b406b500 msgr2=0x7f65b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.756+0000 7f65d8cdd640 1 --2- 172.21.15.16:0/3343696710 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f65b406b500 0x7f65b406d9c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f65bc000f60 tx=0x7f65bc0079d0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.756+0000 7f65d8cdd640 1 -- 172.21.15.16:0/3343696710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65d406d620 msgr2=0x7f65d41e9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.756+0000 7f65d8cdd640 1 --2- 172.21.15.16:0/3343696710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65d406d620 0x7f65d41e9cf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00c500 tx=0x7f65cc00c9c0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.756+0000 7f65d8cdd640 1 -- 172.21.15.16:0/3343696710 shutdown_connections 2024-04-25T10:06:19.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.756+0000 7f65d8cdd640 1 --2- 172.21.15.16:0/3343696710 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f65b406b500 0x7f65b406d9c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.756+0000 7f65d8cdd640 1 --2- 172.21.15.16:0/3343696710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65d415de70 0x7f65d40ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.756+0000 7f65d8cdd640 1 --2- 172.21.15.16:0/3343696710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65d40d1eb0 0x7f65d4155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.756+0000 7f65d8cdd640 1 --2- 172.21.15.16:0/3343696710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65d406d620 0x7f65d41e9cf0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.756+0000 7f65d8cdd640 1 -- 172.21.15.16:0/3343696710 >> 172.21.15.16:0/3343696710 conn(0x7f65d40b33e0 msgr2=0x7f65d406f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.756+0000 7f65d8cdd640 1 -- 172.21.15.16:0/3343696710 shutdown_connections 2024-04-25T10:06:19.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.756+0000 7f65d8cdd640 1 -- 172.21.15.16:0/3343696710 wait complete. 2024-04-25T10:06:19.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:19.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.868+0000 7f7620c6f640 1 Processor -- start 2024-04-25T10:06:19.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.868+0000 7f7620c6f640 1 -- start start 2024-04-25T10:06:19.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.869+0000 7f7620c6f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f761c15ec10 0x7f761c06dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.869+0000 7f7620c6f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f761c06e1b0 0x7f761c0b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.869+0000 7f7620c6f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f761c06e6c0 con 0x7f761c0b3580 2024-04-25T10:06:19.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.869+0000 7f7620c6f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f761c06f2d0 con 0x7f761c15ec10 2024-04-25T10:06:19.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.869+0000 7f7620c6f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f761c06e9a0 con 0x7f761c06e1b0 2024-04-25T10:06:19.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.869+0000 7f761a59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f761c15ec10 0x7f761c06dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.869+0000 7f761a59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f761c15ec10 0x7f761c06dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49554/0 (socket says 172.21.15.16:49554) 2024-04-25T10:06:19.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.869+0000 7f761a59c640 1 -- 172.21.15.16:0/614064433 learned_addr learned my addr 172.21.15.16:0/614064433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:19.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.869+0000 7f7619d9b640 1 --2- 172.21.15.16:0/614064433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f761c06e1b0 0x7f761c0b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.869+0000 7f761a59c640 1 -- 172.21.15.16:0/614064433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f761c06e1b0 msgr2=0x7f761c0b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.869+0000 7f761a59c640 1 --2- 172.21.15.16:0/614064433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f761c06e1b0 0x7f761c0b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.869+0000 7f761a59c640 1 -- 172.21.15.16:0/614064433 >> v1:172.21.15.186:6789/0 conn(0x7f761c0b3580 legacy=0x7f761c1616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.869+0000 7f761a59c640 1 -- 172.21.15.16:0/614064433 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761c06ec80 con 0x7f761c15ec10 2024-04-25T10:06:19.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.869+0000 7f7619d9b640 1 --2- 172.21.15.16:0/614064433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f761c06e1b0 0x7f761c0b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:19.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.870+0000 7f761a59c640 1 --2- 172.21.15.16:0/614064433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c15ec10 0x7f761c06dc70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f76140098a0 tx=0x7f761402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e55bd17c4b2562 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.870+0000 7f761959a640 1 -- 172.21.15.16:0/614064433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614037070 con 0x7f761c15ec10 2024-04-25T10:06:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.870+0000 7f761959a640 1 -- 172.21.15.16:0/614064433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7614032440 con 0x7f761c15ec10 2024-04-25T10:06:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.870+0000 7f761959a640 1 -- 172.21.15.16:0/614064433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614032810 con 0x7f761c15ec10 2024-04-25T10:06:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.870+0000 7f7620c6f640 1 -- 172.21.15.16:0/614064433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c15ec10 msgr2=0x7f761c06dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.870+0000 7f7620c6f640 1 --2- 172.21.15.16:0/614064433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c15ec10 0x7f761c06dc70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f76140098a0 tx=0x7f761402a160 comp rx=0 tx=0).stop 2024-04-25T10:06:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.871+0000 7f7620c6f640 1 -- 172.21.15.16:0/614064433 shutdown_connections 2024-04-25T10:06:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.871+0000 7f7620c6f640 1 --2- 172.21.15.16:0/614064433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f761c06e1b0 0x7f761c0b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.871+0000 7f7620c6f640 1 --2- 172.21.15.16:0/614064433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c15ec10 0x7f761c06dc70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.871+0000 7f7620c6f640 1 -- 172.21.15.16:0/614064433 >> 172.21.15.16:0/614064433 conn(0x7f761c0b2220 msgr2=0x7f761c0cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.871+0000 7f7620c6f640 1 -- 172.21.15.16:0/614064433 shutdown_connections 2024-04-25T10:06:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.871+0000 7f7620c6f640 1 -- 172.21.15.16:0/614064433 wait complete. 2024-04-25T10:06:19.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.872+0000 7f7620c6f640 1 Processor -- start 2024-04-25T10:06:19.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.872+0000 7f7620c6f640 1 -- start start 2024-04-25T10:06:19.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f7620c6f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f761c06e1b0 0x7f761c159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f7620c6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c0b3580 0x7f761c143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f7620c6f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f761c15ec10 0x7f761c15c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f7620c6f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c15e2e0 con 0x7f761c0b3580 2024-04-25T10:06:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f7620c6f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c15eff0 con 0x7f761c15ec10 2024-04-25T10:06:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f7620c6f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f761c15e460 con 0x7f761c06e1b0 2024-04-25T10:06:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f7619d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c0b3580 0x7f761c143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f7619d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c0b3580 0x7f761c143670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49566/0 (socket says 172.21.15.16:49566) 2024-04-25T10:06:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f7619d9b640 1 -- 172.21.15.16:0/3127247597 learned_addr learned my addr 172.21.15.16:0/3127247597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f761a59c640 1 --2- 172.21.15.16:0/3127247597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f761c06e1b0 0x7f761c159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f761ad9d640 1 --2- 172.21.15.16:0/3127247597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f761c15ec10 0x7f761c15c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f7619d9b640 1 -- 172.21.15.16:0/3127247597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f761c06e1b0 msgr2=0x7f761c159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f7619d9b640 1 --2- 172.21.15.16:0/3127247597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f761c06e1b0 0x7f761c159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f7619d9b640 1 -- 172.21.15.16:0/3127247597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f761c15ec10 msgr2=0x7f761c15c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f7619d9b640 1 --2- 172.21.15.16:0/3127247597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f761c15ec10 0x7f761c15c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f7619d9b640 1 -- 172.21.15.16:0/3127247597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761c1440d0 con 0x7f761c0b3580 2024-04-25T10:06:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f761a59c640 1 --2- 172.21.15.16:0/3127247597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f761c06e1b0 0x7f761c159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f7619d9b640 1 --2- 172.21.15.16:0/3127247597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c0b3580 0x7f761c143670 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f761000c7f0 tx=0x7f761000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.873+0000 7f760affd640 1 -- 172.21.15.16:0/3127247597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610015070 con 0x7f761c0b3580 2024-04-25T10:06:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.874+0000 7f760affd640 1 -- 172.21.15.16:0/3127247597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7610007820 con 0x7f761c0b3580 2024-04-25T10:06:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.874+0000 7f760affd640 1 -- 172.21.15.16:0/3127247597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610007bd0 con 0x7f761c0b3580 2024-04-25T10:06:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.874+0000 7f7620c6f640 1 -- 172.21.15.16:0/3127247597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761c1443b0 con 0x7f761c0b3580 2024-04-25T10:06:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.874+0000 7f7620c6f640 1 -- 172.21.15.16:0/3127247597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761c144970 con 0x7f761c0b3580 2024-04-25T10:06:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.875+0000 7f7620c6f640 1 -- 172.21.15.16:0/3127247597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f761c0749b0 con 0x7f761c0b3580 2024-04-25T10:06:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.875+0000 7f760affd640 1 -- 172.21.15.16:0/3127247597 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7610018070 con 0x7f761c0b3580 2024-04-25T10:06:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.875+0000 7f760affd640 1 --2- 172.21.15.16:0/3127247597 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f75fc06b4d0 0x7f75fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.876+0000 7f760affd640 1 -- 172.21.15.16:0/3127247597 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f761008a2e0 con 0x7f761c0b3580 2024-04-25T10:06:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.876+0000 7f761a59c640 1 --2- 172.21.15.16:0/3127247597 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f75fc06b4d0 0x7f75fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.876+0000 7f761a59c640 1 --2- 172.21.15.16:0/3127247597 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f75fc06b4d0 0x7f75fc06d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7614034000 tx=0x7f761402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:19.880+0000 7f760affd640 1 -- 172.21.15.16:0/3127247597 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76100591f0 con 0x7f761c0b3580 2024-04-25T10:06:20.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.098+0000 7f7620c6f640 1 -- 172.21.15.16:0/3127247597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f761c0d0b00 con 0x7f761c0b3580 2024-04-25T10:06:20.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.098+0000 7f760affd640 1 -- 172.21.15.16:0/3127247597 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f7610059e20 con 0x7f761c0b3580 2024-04-25T10:06:20.104 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:20.104 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3381395304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3381395304},{"type":"v1","addr":"172.21.15.186:6835","nonce":3381395304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2429147785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2429147785},{"type":"v1","addr":"172.21.15.186:6837","nonce":2429147785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5142,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2191514045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2191514045},{"type":"v1","addr":"172.21.15.16:6835","nonce":2191514045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:10.940554+0000","modified":"2024-04-25T10:06:12.930243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34613},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34613":{"gid":34613,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.16:6834/1417313138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1417313138},{"type":"v1","addr":"172.21.15.16:6834","nonce":1417313138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:20.104 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 14 2024-04-25T10:06:20.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.100+0000 7f7620c6f640 1 -- 172.21.15.16:0/3127247597 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f75fc06b4d0 msgr2=0x7f75fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.100+0000 7f7620c6f640 1 --2- 172.21.15.16:0/3127247597 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f75fc06b4d0 0x7f75fc06d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7614034000 tx=0x7f761402e000 comp rx=0 tx=0).stop 2024-04-25T10:06:20.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.100+0000 7f7620c6f640 1 -- 172.21.15.16:0/3127247597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c0b3580 msgr2=0x7f761c143670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.100+0000 7f7620c6f640 1 --2- 172.21.15.16:0/3127247597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c0b3580 0x7f761c143670 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f761000c7f0 tx=0x7f761000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.100+0000 7f7620c6f640 1 -- 172.21.15.16:0/3127247597 shutdown_connections 2024-04-25T10:06:20.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.100+0000 7f7620c6f640 1 --2- 172.21.15.16:0/3127247597 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f75fc06b4d0 0x7f75fc06d990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.100+0000 7f7620c6f640 1 --2- 172.21.15.16:0/3127247597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f761c15ec10 0x7f761c15c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.100+0000 7f7620c6f640 1 --2- 172.21.15.16:0/3127247597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c0b3580 0x7f761c143670 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.100+0000 7f7620c6f640 1 --2- 172.21.15.16:0/3127247597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f761c06e1b0 0x7f761c159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.100+0000 7f7620c6f640 1 -- 172.21.15.16:0/3127247597 >> 172.21.15.16:0/3127247597 conn(0x7f761c0b2220 msgr2=0x7f761c072170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.100+0000 7f7620c6f640 1 -- 172.21.15.16:0/3127247597 shutdown_connections 2024-04-25T10:06:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.100+0000 7f7620c6f640 1 -- 172.21.15.16:0/3127247597 wait complete. 2024-04-25T10:06:20.520 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:06:10.940554+0000', 'modified': '2024-04-25T10:06:12.930243+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34613}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34613': {'gid': 34613, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.16:6834/1417313138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 1417313138}, {'type': 'v1', 'addr': '172.21.15.16:6834', 'nonce': 1417313138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:06:20.520 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:06:20.520 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:06:20.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.620+0000 7fb12d109640 1 Processor -- start 2024-04-25T10:06:20.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.620+0000 7fb12d109640 1 -- start start 2024-04-25T10:06:20.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.621+0000 7fb12d109640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb12815dec0 0x7fb1280d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.621+0000 7fb12d109640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb1280d22e0 0x7fb1280d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.621+0000 7fb12d109640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb1280d0990 0x7fb12806c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.621+0000 7fb12d109640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1280d2760 con 0x7fb1280d22e0 2024-04-25T10:06:20.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.621+0000 7fb12d109640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1280d2a20 con 0x7fb1280d0990 2024-04-25T10:06:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.621+0000 7fb12d109640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1280d2d00 con 0x7fb12815dec0 2024-04-25T10:06:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.621+0000 7fb11ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb1280d22e0 0x7fb1280d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.621+0000 7fb11ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb1280d22e0 0x7fb1280d0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49572/0 (socket says 172.21.15.16:49572) 2024-04-25T10:06:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.621+0000 7fb11ffff640 1 -- 172.21.15.16:0/3584615458 learned_addr learned my addr 172.21.15.16:0/3584615458 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.621+0000 7fb11ffff640 1 -- 172.21.15.16:0/3584615458 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb12815dec0 msgr2=0x7fb1280d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.621+0000 7fb11ffff640 1 --2- 172.21.15.16:0/3584615458 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb12815dec0 0x7fb1280d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.621+0000 7fb11ffff640 1 -- 172.21.15.16:0/3584615458 >> v2:172.21.15.186:3300/0 conn(0x7fb1280d0990 msgr2=0x7fb12806c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.621+0000 7fb12759e640 1 --2- 172.21.15.16:0/3584615458 >> v2:172.21.15.186:3300/0 conn(0x7fb1280d0990 0x7fb12806c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.621+0000 7fb11ffff640 1 --2- 172.21.15.16:0/3584615458 >> v2:172.21.15.186:3300/0 conn(0x7fb1280d0990 0x7fb12806c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.621+0000 7fb11ffff640 1 -- 172.21.15.16:0/3584615458 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1280d2fe0 con 0x7fb1280d22e0 2024-04-25T10:06:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.622+0000 7fb11ffff640 1 --2- 172.21.15.16:0/3584615458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1280d22e0 0x7fb1280d0450 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb1280d4c00 tx=0x7fb11002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3dc1ed4417fc5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.622+0000 7fb12659c640 1 -- 172.21.15.16:0/3584615458 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb110037070 con 0x7fb1280d22e0 2024-04-25T10:06:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.622+0000 7fb12659c640 1 -- 172.21.15.16:0/3584615458 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb110032440 con 0x7fb1280d22e0 2024-04-25T10:06:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.622+0000 7fb12659c640 1 -- 172.21.15.16:0/3584615458 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb110032810 con 0x7fb1280d22e0 2024-04-25T10:06:20.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.622+0000 7fb12d109640 1 -- 172.21.15.16:0/3584615458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1280d22e0 msgr2=0x7fb1280d0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.622+0000 7fb12d109640 1 --2- 172.21.15.16:0/3584615458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1280d22e0 0x7fb1280d0450 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb1280d4c00 tx=0x7fb11002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.623+0000 7fb12d109640 1 -- 172.21.15.16:0/3584615458 shutdown_connections 2024-04-25T10:06:20.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.623+0000 7fb12d109640 1 --2- 172.21.15.16:0/3584615458 >> v2:172.21.15.186:3300/0 conn(0x7fb1280d0990 0x7fb12806c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.623+0000 7fb12d109640 1 --2- 172.21.15.16:0/3584615458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1280d22e0 0x7fb1280d0450 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.623+0000 7fb12d109640 1 --2- 172.21.15.16:0/3584615458 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb12815dec0 0x7fb1280d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.623+0000 7fb12d109640 1 -- 172.21.15.16:0/3584615458 >> 172.21.15.16:0/3584615458 conn(0x7fb128072900 msgr2=0x7fb1280b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.623+0000 7fb12d109640 1 -- 172.21.15.16:0/3584615458 shutdown_connections 2024-04-25T10:06:20.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.623+0000 7fb12d109640 1 -- 172.21.15.16:0/3584615458 wait complete. 2024-04-25T10:06:20.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.624+0000 7fb12d109640 1 Processor -- start 2024-04-25T10:06:20.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.624+0000 7fb12d109640 1 -- start start 2024-04-25T10:06:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb12d109640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1280d0990 0x7fb12814fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb12d109640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1280d22e0 0x7fb1281f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb12d109640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb12815dec0 0x7fb1280cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb12d109640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1280d0e20 con 0x7fb1280d0990 2024-04-25T10:06:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb12d109640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1280d35a0 con 0x7fb1280d22e0 2024-04-25T10:06:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb12d109640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12806c7f0 con 0x7fb12815dec0 2024-04-25T10:06:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb126d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1280d0990 0x7fb12814fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb126d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1280d0990 0x7fb12814fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49578/0 (socket says 172.21.15.16:49578) 2024-04-25T10:06:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb126d9d640 1 -- 172.21.15.16:0/3336662744 learned_addr learned my addr 172.21.15.16:0/3336662744 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb11ffff640 1 --2- 172.21.15.16:0/3336662744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1280d22e0 0x7fb1281f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb126d9d640 1 -- 172.21.15.16:0/3336662744 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb12815dec0 msgr2=0x7fb1280cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb126d9d640 1 --2- 172.21.15.16:0/3336662744 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb12815dec0 0x7fb1280cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb126d9d640 1 -- 172.21.15.16:0/3336662744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1280d22e0 msgr2=0x7fb1281f48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb126d9d640 1 --2- 172.21.15.16:0/3336662744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1280d22e0 0x7fb1281f48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb126d9d640 1 -- 172.21.15.16:0/3336662744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1280cbc40 con 0x7fb1280d0990 2024-04-25T10:06:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb11ffff640 1 --2- 172.21.15.16:0/3336662744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1280d22e0 0x7fb1281f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb126d9d640 1 --2- 172.21.15.16:0/3336662744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1280d0990 0x7fb12814fb70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb11800b410 tx=0x7fb11800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb11f7fe640 1 -- 172.21.15.16:0/3336662744 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118015020 con 0x7fb1280d0990 2024-04-25T10:06:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb11f7fe640 1 -- 172.21.15.16:0/3336662744 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb118011850 con 0x7fb1280d0990 2024-04-25T10:06:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb11f7fe640 1 -- 172.21.15.16:0/3336662744 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118011c40 con 0x7fb1280d0990 2024-04-25T10:06:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.625+0000 7fb12d109640 1 -- 172.21.15.16:0/3336662744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1280cb680 con 0x7fb1280d0990 2024-04-25T10:06:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.626+0000 7fb12d109640 1 -- 172.21.15.16:0/3336662744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12815e340 con 0x7fb1280d0990 2024-04-25T10:06:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.627+0000 7fb11f7fe640 1 -- 172.21.15.16:0/3336662744 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb118018020 con 0x7fb1280d0990 2024-04-25T10:06:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.627+0000 7fb11f7fe640 1 --2- 172.21.15.16:0/3336662744 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb10806b4d0 0x7fb10806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.628+0000 7fb11f7fe640 1 -- 172.21.15.16:0/3336662744 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb11808a130 con 0x7fb1280d0990 2024-04-25T10:06:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.628+0000 7fb11ffff640 1 --2- 172.21.15.16:0/3336662744 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb10806b4d0 0x7fb10806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.628+0000 7fb11ffff640 1 --2- 172.21.15.16:0/3336662744 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb10806b4d0 0x7fb10806d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb110034000 tx=0x7fb11002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.628+0000 7fb12fe14740 1 Processor -- start 2024-04-25T10:06:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.629+0000 7fb12fe14740 1 -- start start 2024-04-25T10:06:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.629+0000 7fb12fe14740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a26023a230 0x55a26023a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.629+0000 7fb12fe14740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a260257ad0 0x55a260258020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.629+0000 7fb12fe14740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26020ff10 0x55a260256190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.629+0000 7fb12fe14740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a260172a10 con 0x55a26020ff10 2024-04-25T10:06:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.629+0000 7fb12fe14740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26011bb30 con 0x55a260257ad0 2024-04-25T10:06:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.629+0000 7fb12fe14740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a26016ec50 con 0x55a26023a230 2024-04-25T10:06:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.629+0000 7fb12759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26020ff10 0x55a260256190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.629+0000 7fb12759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26020ff10 0x55a260256190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49590/0 (socket says 172.21.15.16:49590) 2024-04-25T10:06:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.629+0000 7fb12759e640 1 -- 172.21.15.16:0/983463186 learned_addr learned my addr 172.21.15.16:0/983463186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.629+0000 7fb11ffff640 1 --2- 172.21.15.16:0/983463186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a260257ad0 0x55a260258020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.629+0000 7fb126d9d640 1 --2- 172.21.15.16:0/983463186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a26023a230 0x55a26023a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.630+0000 7fb12759e640 1 -- 172.21.15.16:0/983463186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a26023a230 msgr2=0x55a26023a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.630+0000 7fb12759e640 1 --2- 172.21.15.16:0/983463186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a26023a230 0x55a26023a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.630+0000 7fb12759e640 1 -- 172.21.15.16:0/983463186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a260257ad0 msgr2=0x55a260258020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.630+0000 7fb12759e640 1 --2- 172.21.15.16:0/983463186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a260257ad0 0x55a260258020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.630+0000 7fb12759e640 1 -- 172.21.15.16:0/983463186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a260130e10 con 0x55a26020ff10 2024-04-25T10:06:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.630+0000 7fb12759e640 1 --2- 172.21.15.16:0/983463186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26020ff10 0x55a260256190 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb120007e90 tx=0x7fb12000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8faee607084671f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.630+0000 7fb11cff9640 1 -- 172.21.15.16:0/983463186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb120003b90 con 0x55a26020ff10 2024-04-25T10:06:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.630+0000 7fb11cff9640 1 -- 172.21.15.16:0/983463186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb120003d30 con 0x55a26020ff10 2024-04-25T10:06:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.630+0000 7fb11cff9640 1 -- 172.21.15.16:0/983463186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb120004240 con 0x55a26020ff10 2024-04-25T10:06:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.631+0000 7fb12fe14740 1 -- 172.21.15.16:0/983463186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26020ff10 msgr2=0x55a260256190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.631+0000 7fb12fe14740 1 --2- 172.21.15.16:0/983463186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26020ff10 0x55a260256190 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb120007e90 tx=0x7fb12000c6e0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.631+0000 7fb12fe14740 1 -- 172.21.15.16:0/983463186 shutdown_connections 2024-04-25T10:06:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.631+0000 7fb12fe14740 1 --2- 172.21.15.16:0/983463186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26020ff10 0x55a260256190 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.631+0000 7fb12fe14740 1 --2- 172.21.15.16:0/983463186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a260257ad0 0x55a260258020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.631+0000 7fb12fe14740 1 --2- 172.21.15.16:0/983463186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a26023a230 0x55a26023a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.631+0000 7fb12fe14740 1 -- 172.21.15.16:0/983463186 >> 172.21.15.16:0/983463186 conn(0x55a26020c840 msgr2=0x55a260255040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.631+0000 7fb12fe14740 1 -- 172.21.15.16:0/983463186 shutdown_connections 2024-04-25T10:06:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.631+0000 7fb12fe14740 1 -- 172.21.15.16:0/983463186 wait complete. 2024-04-25T10:06:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.631+0000 7fb12fe14740 1 -- start start 2024-04-25T10:06:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.631+0000 7fb12fe14740 1 Processor -- start 2024-04-25T10:06:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.632+0000 7fb12fe14740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023a230 0x55a260236b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.632+0000 7fb12fe14740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a260257ad0 0x55a26023b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.632+0000 7fb12fe14740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a26023b870 0x55a26024e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.632+0000 7fb12fe14740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a260210cb0 con 0x55a26023a230 2024-04-25T10:06:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.632+0000 7fb12fe14740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26012f0c0 con 0x55a260257ad0 2024-04-25T10:06:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.632+0000 7fb12fe14740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a26020a770 con 0x55a26023b870 2024-04-25T10:06:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.632+0000 7fb12fe14740 10 client.0 _ll_register_callbacks cb 0x55a260122ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.632+0000 7fb126d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023a230 0x55a260236b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.632+0000 7fb126d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023a230 0x55a260236b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49600/0 (socket says 172.21.15.16:49600) 2024-04-25T10:06:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.632+0000 7fb126d9d640 1 -- 172.21.15.16:0/28504086 learned_addr learned my addr 172.21.15.16:0/28504086 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.632+0000 7fb12759e640 1 --2- 172.21.15.16:0/28504086 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a26023b870 0x55a26024e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.632+0000 7fb11ffff640 1 --2- 172.21.15.16:0/28504086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a260257ad0 0x55a26023b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb126d9d640 1 -- 172.21.15.16:0/28504086 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a26023b870 msgr2=0x55a26024e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb126d9d640 1 --2- 172.21.15.16:0/28504086 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a26023b870 0x55a26024e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb126d9d640 1 -- 172.21.15.16:0/28504086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a260257ad0 msgr2=0x55a26023b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:20.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb126d9d640 1 --2- 172.21.15.16:0/28504086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a260257ad0 0x55a26023b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb126d9d640 1 -- 172.21.15.16:0/28504086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a25fe872d0 con 0x55a26023a230 2024-04-25T10:06:20.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb126d9d640 1 --2- 172.21.15.16:0/28504086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023a230 0x55a260236b60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb118058560 tx=0x7fb1180589f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52802550ec2497a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb105ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:06:20.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb105ffb640 1 -- 172.21.15.16:0/28504086 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118086060 con 0x55a26023a230 2024-04-25T10:06:20.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb105ffb640 1 -- 172.21.15.16:0/28504086 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb118070210 con 0x55a26023a230 2024-04-25T10:06:20.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb12fe14740 1 -- 172.21.15.16:0/28504086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb120003d30 con 0x55a26023a230 2024-04-25T10:06:20.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb105ffb640 1 -- 172.21.15.16:0/28504086 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb118075050 con 0x55a26023a230 2024-04-25T10:06:20.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb12fe14740 10 client.5334 fetch_fsmap learned FSMap version 14 2024-04-25T10:06:20.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb12fe14740 1 -- 172.21.15.16:0/28504086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x55a25ff516b0 con 0x55a26023a230 2024-04-25T10:06:20.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb105ffb640 1 -- 172.21.15.16:0/28504086 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb120003d30 con 0x55a26023a230 2024-04-25T10:06:20.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb105ffb640 1 -- 172.21.15.16:0/28504086 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb1180772e0 con 0x55a26023a230 2024-04-25T10:06:20.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb12fe14740 10 client.5334 fetch_fsmap finished waiting for FSMap version 14 2024-04-25T10:06:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb12fe14740 10 client.5334 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:06:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb12fe14740 20 client.5334 populate_metadata read hostname 'smithi016' 2024-04-25T10:06:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb12fe14740 1 --2- 172.21.15.16:0/28504086 >> [v2:172.21.15.16:6833/1417313138,v1:172.21.15.16:6834/1417313138] conn(0x55a26024c740 0x55a260250b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb12fe14740 4 client.5334 mds_command: new command op to 34613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.633+0000 7fb12fe14740 1 -- 172.21.15.16:0/28504086 --> [v2:172.21.15.16:6833/1417313138,v1:172.21.15.16:6834/1417313138] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2601d1920 con 0x55a26024c740 2024-04-25T10:06:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.634+0000 7fb11ffff640 1 --2- 172.21.15.16:0/28504086 >> [v2:172.21.15.16:6833/1417313138,v1:172.21.15.16:6834/1417313138] conn(0x55a26024c740 0x55a260250b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.634+0000 7fb105ffb640 10 client.5334 ms_handle_connect on v2:172.21.15.16:6833/1417313138 2024-04-25T10:06:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.634+0000 7fb11ffff640 1 --2- 172.21.15.16:0/28504086 >> [v2:172.21.15.16:6833/1417313138,v1:172.21.15.16:6834/1417313138] conn(0x55a26024c740 0x55a260250b80 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=954647019ca8af98 server_cookie=8459032cce5d03a9 in_seq=0 out_seq=0 2024-04-25T10:06:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.635+0000 7fb105ffb640 1 -- 172.21.15.16:0/28504086 <== mds.0 v2:172.21.15.16:6833/1417313138 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a2601d1920 con 0x55a26024c740 2024-04-25T10:06:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.635+0000 7fb105ffb640 10 client.5334 handle_command_reply: tid=0 2024-04-25T10:06:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.635+0000 7fb12fe14740 1 client.5334 shutdown 2024-04-25T10:06:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.635+0000 7fb12fe14740 1 -- 172.21.15.16:0/28504086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023a230 msgr2=0x55a260236b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.635+0000 7fb12fe14740 1 --2- 172.21.15.16:0/28504086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023a230 0x55a260236b60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb118058560 tx=0x7fb1180589f0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.635+0000 7fb12fe14740 1 -- 172.21.15.16:0/28504086 shutdown_connections 2024-04-25T10:06:20.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.635+0000 7fb12fe14740 1 --2- 172.21.15.16:0/28504086 >> [v2:172.21.15.16:6833/1417313138,v1:172.21.15.16:6834/1417313138] conn(0x55a26024c740 0x55a260250b80 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.635+0000 7fb12fe14740 1 --2- 172.21.15.16:0/28504086 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a26023b870 0x55a26024e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.635+0000 7fb12fe14740 1 --2- 172.21.15.16:0/28504086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a260257ad0 0x55a26023b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.635+0000 7fb105ffb640 0 client.5334 ms_handle_reset on v2:172.21.15.16:6833/1417313138 2024-04-25T10:06:20.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.635+0000 7fb12fe14740 1 --2- 172.21.15.16:0/28504086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023a230 0x55a260236b60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.635+0000 7fb12fe14740 1 -- 172.21.15.16:0/28504086 >> 172.21.15.16:0/28504086 conn(0x55a26020ff10 msgr2=0x55a260258e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.635+0000 7fb12fe14740 1 -- 172.21.15.16:0/28504086 shutdown_connections 2024-04-25T10:06:20.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.635+0000 7fb12fe14740 1 -- 172.21.15.16:0/28504086 wait complete. 2024-04-25T10:06:20.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.635+0000 7fb12fe14740 20 client.5334 trim_cache size 0 max 16384 2024-04-25T10:06:20.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.651+0000 7fb12fe14740 1 Processor -- start 2024-04-25T10:06:20.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.651+0000 7fb12fe14740 1 -- start start 2024-04-25T10:06:20.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.651+0000 7fb12fe14740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023a230 0x55a26024d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.651+0000 7fb12fe14740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a26023b870 0x55a260253ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.651+0000 7fb12fe14740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a260257ad0 0x55a260253ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.651+0000 7fb12fe14740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1281ff110 con 0x55a26023a230 2024-04-25T10:06:20.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.651+0000 7fb12fe14740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12815f450 con 0x55a260257ad0 2024-04-25T10:06:20.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.651+0000 7fb12fe14740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1281f4370 con 0x55a26023b870 2024-04-25T10:06:20.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.652+0000 7fb126d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023a230 0x55a26024d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.652+0000 7fb126d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023a230 0x55a26024d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49606/0 (socket says 172.21.15.16:49606) 2024-04-25T10:06:20.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.652+0000 7fb126d9d640 1 -- 172.21.15.16:0/4201713826 learned_addr learned my addr 172.21.15.16:0/4201713826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:20.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.652+0000 7fb11ffff640 1 --2- 172.21.15.16:0/4201713826 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a26023b870 0x55a260253ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.652+0000 7fb12759e640 1 --2- 172.21.15.16:0/4201713826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a260257ad0 0x55a260253ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.652+0000 7fb126d9d640 1 -- 172.21.15.16:0/4201713826 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a26023b870 msgr2=0x55a260253ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:20.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.652+0000 7fb126d9d640 1 --2- 172.21.15.16:0/4201713826 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a26023b870 0x55a260253ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.652+0000 7fb126d9d640 1 -- 172.21.15.16:0/4201713826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a260257ad0 msgr2=0x55a260253ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:20.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.652+0000 7fb126d9d640 1 --2- 172.21.15.16:0/4201713826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a260257ad0 0x55a260253ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.652+0000 7fb126d9d640 1 -- 172.21.15.16:0/4201713826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a26020bd00 con 0x55a26023a230 2024-04-25T10:06:20.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.652+0000 7fb126d9d640 1 --2- 172.21.15.16:0/4201713826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023a230 0x55a26024d260 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb118060a90 tx=0x7fb11805b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e84a025b0715a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.652+0000 7fb107fff640 1 -- 172.21.15.16:0/4201713826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11807b070 con 0x55a26023a230 2024-04-25T10:06:20.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.652+0000 7fb107fff640 1 -- 172.21.15.16:0/4201713826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb118054f20 con 0x55a26023a230 2024-04-25T10:06:20.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.652+0000 7fb107fff640 1 -- 172.21.15.16:0/4201713826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118086070 con 0x55a26023a230 2024-04-25T10:06:20.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.653+0000 7fb12fe14740 1 -- 172.21.15.16:0/4201713826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023a230 msgr2=0x55a26024d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.653+0000 7fb12fe14740 1 --2- 172.21.15.16:0/4201713826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023a230 0x55a26024d260 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb118060a90 tx=0x7fb11805b000 comp rx=0 tx=0).stop 2024-04-25T10:06:20.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.653+0000 7fb12fe14740 1 -- 172.21.15.16:0/4201713826 shutdown_connections 2024-04-25T10:06:20.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.653+0000 7fb12fe14740 1 --2- 172.21.15.16:0/4201713826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a260257ad0 0x55a260253ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.653+0000 7fb12fe14740 1 --2- 172.21.15.16:0/4201713826 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a26023b870 0x55a260253ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.653+0000 7fb12fe14740 1 --2- 172.21.15.16:0/4201713826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023a230 0x55a26024d260 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.653+0000 7fb12fe14740 1 -- 172.21.15.16:0/4201713826 >> 172.21.15.16:0/4201713826 conn(0x55a26020ff10 msgr2=0x55a260239790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.653+0000 7fb12fe14740 1 -- 172.21.15.16:0/4201713826 shutdown_connections 2024-04-25T10:06:20.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.653+0000 7fb12fe14740 1 -- 172.21.15.16:0/4201713826 wait complete. 2024-04-25T10:06:20.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.653+0000 7fb12fe14740 1 -- start start 2024-04-25T10:06:20.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.653+0000 7fb12fe14740 1 Processor -- start 2024-04-25T10:06:20.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.654+0000 7fb12fe14740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a26023a230 0x55a2602a6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.654+0000 7fb12fe14740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023b870 0x55a2602682b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.654+0000 7fb12fe14740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a260257ad0 0x55a2602a5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.654+0000 7fb12fe14740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26024c5c0 con 0x55a26023b870 2024-04-25T10:06:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.654+0000 7fb12fe14740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26023c920 con 0x55a26023a230 2024-04-25T10:06:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.654+0000 7fb12fe14740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a26024cb90 con 0x55a260257ad0 2024-04-25T10:06:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.654+0000 7fb12fe14740 10 client.0 _ll_register_callbacks cb 0x55a26020ac50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.654+0000 7fb11ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023b870 0x55a2602682b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.654+0000 7fb11ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023b870 0x55a2602682b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49622/0 (socket says 172.21.15.16:49622) 2024-04-25T10:06:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.654+0000 7fb11ffff640 1 -- 172.21.15.16:0/3914293064 learned_addr learned my addr 172.21.15.16:0/3914293064 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.654+0000 7fb126d9d640 1 --2- 172.21.15.16:0/3914293064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a26023a230 0x55a2602a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.654+0000 7fb12759e640 1 --2- 172.21.15.16:0/3914293064 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a260257ad0 0x55a2602a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.654+0000 7fb11ffff640 1 -- 172.21.15.16:0/3914293064 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a260257ad0 msgr2=0x55a2602a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.654+0000 7fb11ffff640 1 --2- 172.21.15.16:0/3914293064 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a260257ad0 0x55a2602a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.654+0000 7fb11ffff640 1 -- 172.21.15.16:0/3914293064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a26023a230 msgr2=0x55a2602a6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.654+0000 7fb11ffff640 1 --2- 172.21.15.16:0/3914293064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a26023a230 0x55a2602a6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.654+0000 7fb11ffff640 1 -- 172.21.15.16:0/3914293064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2602a5650 con 0x55a26023b870 2024-04-25T10:06:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.655+0000 7fb11ffff640 1 --2- 172.21.15.16:0/3914293064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023b870 0x55a2602682b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb1100090b0 tx=0x7fb11002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86efdfb4c0d25da1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.655+0000 7fb105ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:06:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.655+0000 7fb105ffb640 1 -- 172.21.15.16:0/3914293064 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb110057070 con 0x55a26023b870 2024-04-25T10:06:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.655+0000 7fb105ffb640 1 -- 172.21.15.16:0/3914293064 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb11003eb30 con 0x55a26023b870 2024-04-25T10:06:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.655+0000 7fb12fe14740 1 -- 172.21.15.16:0/3914293064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb118054f20 con 0x55a26023b870 2024-04-25T10:06:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.655+0000 7fb105ffb640 1 -- 172.21.15.16:0/3914293064 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb110062050 con 0x55a26023b870 2024-04-25T10:06:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.655+0000 7fb12fe14740 10 client.5340 fetch_fsmap learned FSMap version 14 2024-04-25T10:06:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.655+0000 7fb12fe14740 1 -- 172.21.15.16:0/3914293064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x55a260277a20 con 0x55a26023b870 2024-04-25T10:06:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.655+0000 7fb105ffb640 1 -- 172.21.15.16:0/3914293064 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb118054f20 con 0x55a26023b870 2024-04-25T10:06:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.655+0000 7fb105ffb640 1 -- 172.21.15.16:0/3914293064 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb110065020 con 0x55a26023b870 2024-04-25T10:06:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.655+0000 7fb12fe14740 10 client.5340 fetch_fsmap finished waiting for FSMap version 14 2024-04-25T10:06:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.655+0000 7fb12fe14740 10 client.5340 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:06:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.655+0000 7fb12fe14740 20 client.5340 populate_metadata read hostname 'smithi016' 2024-04-25T10:06:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.655+0000 7fb12fe14740 1 --2- 172.21.15.16:0/3914293064 >> [v2:172.21.15.16:6833/1417313138,v1:172.21.15.16:6834/1417313138] conn(0x55a26023d900 0x55a26024ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.655+0000 7fb12fe14740 4 client.5340 mds_command: new command op to 34613 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:06:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.655+0000 7fb12fe14740 1 -- 172.21.15.16:0/3914293064 --> [v2:172.21.15.16:6833/1417313138,v1:172.21.15.16:6834/1417313138] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a26017ff50 con 0x55a26023d900 2024-04-25T10:06:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.655+0000 7fb126d9d640 1 --2- 172.21.15.16:0/3914293064 >> [v2:172.21.15.16:6833/1417313138,v1:172.21.15.16:6834/1417313138] conn(0x55a26023d900 0x55a26024ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.656+0000 7fb105ffb640 10 client.5340 ms_handle_connect on v2:172.21.15.16:6833/1417313138 2024-04-25T10:06:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.656+0000 7fb126d9d640 1 --2- 172.21.15.16:0/3914293064 >> [v2:172.21.15.16:6833/1417313138,v1:172.21.15.16:6834/1417313138] conn(0x55a26023d900 0x55a26024ed90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1259fed45b6abb67 server_cookie=392d9fe1600159a2 in_seq=0 out_seq=0 2024-04-25T10:06:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.656+0000 7fb105ffb640 1 -- 172.21.15.16:0/3914293064 <== mds.0 v2:172.21.15.16:6833/1417313138 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55a26017ff50 con 0x55a26023d900 2024-04-25T10:06:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.656+0000 7fb105ffb640 10 client.5340 handle_command_reply: tid=0 2024-04-25T10:06:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.656+0000 7fb12fe14740 1 client.5340 shutdown 2024-04-25T10:06:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.656+0000 7fb12fe14740 1 -- 172.21.15.16:0/3914293064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023b870 msgr2=0x55a2602682b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.656+0000 7fb12fe14740 1 --2- 172.21.15.16:0/3914293064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023b870 0x55a2602682b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb1100090b0 tx=0x7fb11002c070 comp rx=0 tx=0).stop 2024-04-25T10:06:20.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.656+0000 7fb12fe14740 1 -- 172.21.15.16:0/3914293064 shutdown_connections 2024-04-25T10:06:20.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.656+0000 7fb12fe14740 1 --2- 172.21.15.16:0/3914293064 >> [v2:172.21.15.16:6833/1417313138,v1:172.21.15.16:6834/1417313138] conn(0x55a26023d900 0x55a26024ed90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.656+0000 7fb12fe14740 1 --2- 172.21.15.16:0/3914293064 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a260257ad0 0x55a2602a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.656+0000 7fb12fe14740 1 --2- 172.21.15.16:0/3914293064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a26023b870 0x55a2602682b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.656+0000 7fb105ffb640 0 client.5340 ms_handle_reset on v2:172.21.15.16:6833/1417313138 2024-04-25T10:06:20.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.656+0000 7fb12fe14740 1 --2- 172.21.15.16:0/3914293064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a26023a230 0x55a2602a6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.656+0000 7fb12fe14740 1 -- 172.21.15.16:0/3914293064 >> 172.21.15.16:0/3914293064 conn(0x55a26020ff10 msgr2=0x55a260250b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.656+0000 7fb12fe14740 1 -- 172.21.15.16:0/3914293064 shutdown_connections 2024-04-25T10:06:20.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.656+0000 7fb12fe14740 1 -- 172.21.15.16:0/3914293064 wait complete. 2024-04-25T10:06:20.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.656+0000 7fb12fe14740 20 client.5340 trim_cache size 0 max 16384 2024-04-25T10:06:20.669 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:06:20.669 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "34bc5eca-3a21-42a6-a26f-07da0905e8bc", 2024-04-25T10:06:20.669 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-04-25T10:06:20.669 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 34613, 2024-04-25T10:06:20.669 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-04-25T10:06:20.669 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-04-25T10:06:20.669 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-04-25T10:06:20.669 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 9.0829831330000008, 2024-04-25T10:06:20.669 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 14, 2024-04-25T10:06:20.669 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 36, 2024-04-25T10:06:20.669 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 36, 2024-04-25T10:06:20.669 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 23.620313093 2024-04-25T10:06:20.669 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:06:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.657+0000 7fb12d109640 1 -- 172.21.15.16:0/3336662744 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb10806b4d0 msgr2=0x7fb10806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.657+0000 7fb12d109640 1 --2- 172.21.15.16:0/3336662744 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb10806b4d0 0x7fb10806d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb110034000 tx=0x7fb11002e000 comp rx=0 tx=0).stop 2024-04-25T10:06:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.657+0000 7fb12d109640 1 -- 172.21.15.16:0/3336662744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1280d0990 msgr2=0x7fb12814fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.657+0000 7fb12d109640 1 --2- 172.21.15.16:0/3336662744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1280d0990 0x7fb12814fb70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb11800b410 tx=0x7fb11800b8d0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.657+0000 7fb12d109640 1 -- 172.21.15.16:0/3336662744 shutdown_connections 2024-04-25T10:06:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.657+0000 7fb12d109640 1 --2- 172.21.15.16:0/3336662744 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb10806b4d0 0x7fb10806d990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.657+0000 7fb12d109640 1 --2- 172.21.15.16:0/3336662744 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb12815dec0 0x7fb1280cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.657+0000 7fb12d109640 1 --2- 172.21.15.16:0/3336662744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1280d22e0 0x7fb1281f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.657+0000 7fb12d109640 1 --2- 172.21.15.16:0/3336662744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1280d0990 0x7fb12814fb70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.657+0000 7fb12d109640 1 -- 172.21.15.16:0/3336662744 >> 172.21.15.16:0/3336662744 conn(0x7fb128072900 msgr2=0x7fb1280d3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.657+0000 7fb12d109640 1 -- 172.21.15.16:0/3336662744 shutdown_connections 2024-04-25T10:06:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.657+0000 7fb12d109640 1 -- 172.21.15.16:0/3336662744 wait complete. 2024-04-25T10:06:20.673 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:06:20.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.768+0000 7f22eff9e640 1 Processor -- start 2024-04-25T10:06:20.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.769+0000 7f22eff9e640 1 -- start start 2024-04-25T10:06:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.769+0000 7f22eff9e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f22e815dce0 0x7f22e80d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.769+0000 7f22eff9e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e815e0c0 con 0x7f22e80d35b0 2024-04-25T10:06:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.769+0000 7f22eff9e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e80d2510 con 0x7f22e80b5b60 2024-04-25T10:06:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.769+0000 7f22eff9e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e80d27f0 con 0x7f22e815dce0 2024-04-25T10:06:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.769+0000 7f22edd3a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f22e815dce0 0x7f22e80d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.769+0000 7f22ed539640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f22e80d35b0 0x7f22e80b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56556/0 (socket says 172.21.15.16:56556) 2024-04-25T10:06:20.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.769+0000 7f22ed539640 1 -- 172.21.15.16:0/1012752621 learned_addr learned my addr 172.21.15.16:0/1012752621 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:20.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.770+0000 7f22edd3a640 1 -- 172.21.15.16:0/1012752621 >> v1:172.21.15.186:6789/0 conn(0x7f22e80b5b60 legacy=0x7f22e80b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.770+0000 7f22edd3a640 1 -- 172.21.15.16:0/1012752621 >> v1:172.21.15.16:6789/0 conn(0x7f22e80d35b0 legacy=0x7f22e80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.770+0000 7f22edd3a640 1 -- 172.21.15.16:0/1012752621 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e80d2ad0 con 0x7f22e815dce0 2024-04-25T10:06:20.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.770+0000 7f22edd3a640 1 --2- 172.21.15.16:0/1012752621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e815dce0 0x7f22e80d1a50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f22e00097a0 tx=0x7f22e002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deff7217ad3cb82b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.771+0000 7f22ecd38640 1 -- 172.21.15.16:0/1012752621 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0037070 con 0x7f22e815dce0 2024-04-25T10:06:20.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.771+0000 7f22ecd38640 1 -- 172.21.15.16:0/1012752621 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e0032440 con 0x7f22e815dce0 2024-04-25T10:06:20.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.771+0000 7f22ecd38640 1 -- 172.21.15.16:0/1012752621 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0032810 con 0x7f22e815dce0 2024-04-25T10:06:20.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.771+0000 7f22eff9e640 1 -- 172.21.15.16:0/1012752621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e815dce0 msgr2=0x7f22e80d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.771+0000 7f22eff9e640 1 --2- 172.21.15.16:0/1012752621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e815dce0 0x7f22e80d1a50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f22e00097a0 tx=0x7f22e002a230 comp rx=0 tx=0).stop 2024-04-25T10:06:20.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.771+0000 7f22eff9e640 1 -- 172.21.15.16:0/1012752621 shutdown_connections 2024-04-25T10:06:20.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.771+0000 7f22eff9e640 1 --2- 172.21.15.16:0/1012752621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e815dce0 0x7f22e80d1a50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.771+0000 7f22eff9e640 1 -- 172.21.15.16:0/1012752621 >> 172.21.15.16:0/1012752621 conn(0x7f22e8072900 msgr2=0x7f22e80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.771+0000 7f22eff9e640 1 -- 172.21.15.16:0/1012752621 shutdown_connections 2024-04-25T10:06:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.771+0000 7f22eff9e640 1 -- 172.21.15.16:0/1012752621 wait complete. 2024-04-25T10:06:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.772+0000 7f22eff9e640 1 Processor -- start 2024-04-25T10:06:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.773+0000 7f22eff9e640 1 -- start start 2024-04-25T10:06:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.773+0000 7f22eff9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e80b5b60 0x7f22e815eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.773+0000 7f22eff9e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22e80d35b0 0x7f22e81574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.773+0000 7f22eff9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22e815dce0 0x7f22e81579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.773+0000 7f22eff9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e80b9820 con 0x7f22e80b5b60 2024-04-25T10:06:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.773+0000 7f22eff9e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e80d3090 con 0x7f22e80d35b0 2024-04-25T10:06:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.773+0000 7f22eff9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e80b99a0 con 0x7f22e815dce0 2024-04-25T10:06:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.773+0000 7f22edd3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e80b5b60 0x7f22e815eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.773+0000 7f22edd3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e80b5b60 0x7f22e815eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49636/0 (socket says 172.21.15.16:49636) 2024-04-25T10:06:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.773+0000 7f22edd3a640 1 -- 172.21.15.16:0/2540281678 learned_addr learned my addr 172.21.15.16:0/2540281678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.773+0000 7f22ed539640 1 --2- 172.21.15.16:0/2540281678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22e80d35b0 0x7f22e81574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.773+0000 7f22ee53b640 1 --2- 172.21.15.16:0/2540281678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22e815dce0 0x7f22e81579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.774+0000 7f22edd3a640 1 -- 172.21.15.16:0/2540281678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22e815dce0 msgr2=0x7f22e81579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.774+0000 7f22edd3a640 1 --2- 172.21.15.16:0/2540281678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22e815dce0 0x7f22e81579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.774+0000 7f22edd3a640 1 -- 172.21.15.16:0/2540281678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22e80d35b0 msgr2=0x7f22e81574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.774+0000 7f22edd3a640 1 --2- 172.21.15.16:0/2540281678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22e80d35b0 0x7f22e81574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.774+0000 7f22edd3a640 1 -- 172.21.15.16:0/2540281678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e80c8410 con 0x7f22e80b5b60 2024-04-25T10:06:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.774+0000 7f22edd3a640 1 --2- 172.21.15.16:0/2540281678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e80b5b60 0x7f22e815eda0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f22e0035040 tx=0x7f22e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.774+0000 7f22e67fc640 1 -- 172.21.15.16:0/2540281678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0037040 con 0x7f22e80b5b60 2024-04-25T10:06:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.774+0000 7f22e67fc640 1 -- 172.21.15.16:0/2540281678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e0030770 con 0x7f22e80b5b60 2024-04-25T10:06:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.774+0000 7f22eff9e640 1 -- 172.21.15.16:0/2540281678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e80cd750 con 0x7f22e80b5b60 2024-04-25T10:06:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.774+0000 7f22e67fc640 1 -- 172.21.15.16:0/2540281678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0030b80 con 0x7f22e80b5b60 2024-04-25T10:06:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.774+0000 7f22eff9e640 1 -- 172.21.15.16:0/2540281678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e80c86b0 con 0x7f22e80b5b60 2024-04-25T10:06:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.775+0000 7f22eff9e640 1 -- 172.21.15.16:0/2540281678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e80749b0 con 0x7f22e80b5b60 2024-04-25T10:06:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.775+0000 7f22e67fc640 1 -- 172.21.15.16:0/2540281678 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f22e0044020 con 0x7f22e80b5b60 2024-04-25T10:06:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.776+0000 7f22e67fc640 1 --2- 172.21.15.16:0/2540281678 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f22d406b4d0 0x7f22d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.776+0000 7f22e67fc640 1 -- 172.21.15.16:0/2540281678 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22e00aa790 con 0x7f22e80b5b60 2024-04-25T10:06:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.776+0000 7f22ed539640 1 --2- 172.21.15.16:0/2540281678 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f22d406b4d0 0x7f22d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.777+0000 7f22ed539640 1 --2- 172.21.15.16:0/2540281678 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f22d406b4d0 0x7f22d406d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f22e80cda10 tx=0x7f22dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.780+0000 7f22e67fc640 1 -- 172.21.15.16:0/2540281678 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f22e0032e20 con 0x7f22e80b5b60 2024-04-25T10:06:21.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.997+0000 7f22eff9e640 1 -- 172.21.15.16:0/2540281678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f22e806b080 con 0x7f22e80b5b60 2024-04-25T10:06:21.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.998+0000 7f22e67fc640 1 -- 172.21.15.16:0/2540281678 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+73 (secure 0 0 0) 0x7f22e00313a0 con 0x7f22e80b5b60 2024-04-25T10:06:21.003 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:06:21.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.999+0000 7f22eff9e640 1 -- 172.21.15.16:0/2540281678 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f22d406b4d0 msgr2=0x7f22d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.999+0000 7f22eff9e640 1 --2- 172.21.15.16:0/2540281678 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f22d406b4d0 0x7f22d406d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f22e80cda10 tx=0x7f22dc009040 comp rx=0 tx=0).stop 2024-04-25T10:06:21.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.999+0000 7f22eff9e640 1 -- 172.21.15.16:0/2540281678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e80b5b60 msgr2=0x7f22e815eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.999+0000 7f22eff9e640 1 --2- 172.21.15.16:0/2540281678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e80b5b60 0x7f22e815eda0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f22e0035040 tx=0x7f22e00046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.999+0000 7f22eff9e640 1 -- 172.21.15.16:0/2540281678 shutdown_connections 2024-04-25T10:06:21.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.999+0000 7f22eff9e640 1 --2- 172.21.15.16:0/2540281678 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f22d406b4d0 0x7f22d406d990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.999+0000 7f22eff9e640 1 --2- 172.21.15.16:0/2540281678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22e815dce0 0x7f22e81579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.999+0000 7f22eff9e640 1 --2- 172.21.15.16:0/2540281678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22e80d35b0 0x7f22e81574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.999+0000 7f22eff9e640 1 --2- 172.21.15.16:0/2540281678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e80b5b60 0x7f22e815eda0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:20.999+0000 7f22eff9e640 1 -- 172.21.15.16:0/2540281678 >> 172.21.15.16:0/2540281678 conn(0x7f22e8072900 msgr2=0x7f22e80b8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:21.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:21.000+0000 7f22eff9e640 1 -- 172.21.15.16:0/2540281678 shutdown_connections 2024-04-25T10:06:21.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:21.000+0000 7f22eff9e640 1 -- 172.21.15.16:0/2540281678 wait complete. 2024-04-25T10:06:21.016 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:06:21.016 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:06:21.016 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-04-25T10:06:21.016 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:06:21.016 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:21.016 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:06:21.016 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:06:21.016 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:06:21.016 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:21.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:21.017 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-04-25T10:06:21.033 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:06:21.033 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:06:21.033 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:06:21.033 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:21.033 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:06:21.033 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:21.033 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:06:21.033 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:21.034 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-04-25T10:06:21.034 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:06:21.034 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:21.034 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-04-25T10:06:21.034 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:06:21.034 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:21.034 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-04-25T10:06:21.034 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:06:21.034 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42579sec preferred_lft 42579sec 2024-04-25T10:06:21.034 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-25T10:06:21.034 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:21.034 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:06:21.034 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:21.034 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-04-25T10:06:21.035 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:06:21.035 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:21.035 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:06:21.035 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:21.035 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f06b:fdff:fed4:a335/64 scope link 2024-04-25T10:06:21.035 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:21.035 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:06:21.035 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:21.035 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::90ad:4fff:fe91:3c33/64 scope link 2024-04-25T10:06:21.035 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:21.035 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:06:21.035 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether be:14:4a:3c:16:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:21.035 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::bc14:4aff:fe3c:1605/64 scope link 2024-04-25T10:06:21.035 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:21.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:21.036 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:06:21.096 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:06:21.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:21.098 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-04-25T10:06:21.154 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-04-25T10:06:21.154 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:06:21.154 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:06:21.154 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:06:21.155 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:06:21.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:21.155 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:06:21.155 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:06:21.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:06:21.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:06:21.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:06:21.155 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:06:21.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:21.240 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:06:21.255 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:06:21.255 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:06:21.255 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:21.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:21.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-04-25T10:06:21.320 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:06:21.321 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:21.375 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:21.375 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:06:21.375 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:21.430 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:06:21.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:21.430 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:06:21.512 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:06:21.513 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:06:21.513 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:06:21.521 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:06:21.521 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:06:21.521 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:06:21.521 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:06:21.521 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:06:21.526 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:06:21.526 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:06:21.527 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:06:21.527 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:06:21.527 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:06:21.527 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:06:21.527 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:06:21.527 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:06:21.527 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:06:21.527 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:06:21.527 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:06:21.528 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:06:21.528 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:06:21.529 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:06:21.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:21.529 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:21.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:21.595 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:06:21.658 INFO:teuthology.orchestra.run.smithi016.stdout:client34742 2024-04-25T10:06:21.659 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:06:21.659 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:21.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:21.715 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-04-25T10:06:21.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:21.776 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-04-25T10:06:21.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:21.833 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-04-25T10:06:21.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:21.891 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-04-25T10:06:21.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:21.953 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-04-25T10:06:22.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:22.011 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:06:22.070 INFO:teuthology.orchestra.run.smithi016.stdout:./ 2024-04-25T10:06:22.070 INFO:teuthology.orchestra.run.smithi016.stdout:./rfile 2024-04-25T10:06:22.070 INFO:teuthology.orchestra.run.smithi016.stdout:./parentdir 2024-04-25T10:06:22.071 INFO:teuthology.orchestra.run.smithi016.stdout:./parentdir/pfile 2024-04-25T10:06:22.071 INFO:teuthology.orchestra.run.smithi016.stdout:./parentdir/childdir 2024-04-25T10:06:22.071 INFO:teuthology.orchestra.run.smithi016.stdout:./parentdir/childdir/cfile 2024-04-25T10:06:22.071 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:06:22.071 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:06:22.071 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:06:22.071 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:06:22.072 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:06:22.072 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:06:22.177 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:06:22.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:22.187 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:06:22.187 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:06:22.187 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:06:22.188 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:06:22.188 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-04-25T10:06:22.188 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:06:22.295 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627778 2024-04-25T10:06:22.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:22.304 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:06:22.304 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:06:22.304 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:06:22.304 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:06:22.304 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-04-25T10:06:22.304 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:06:22.412 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627776 2024-04-25T10:06:22.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:22.421 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:06:22.421 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:06:22.421 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:06:22.421 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:06:22.421 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-04-25T10:06:22.421 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:06:22.528 INFO:teuthology.orchestra.run.smithi016.stdout:1099511628279 2024-04-25T10:06:22.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:22.537 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:06:22.537 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:06:22.537 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:06:22.537 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:06:22.537 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-04-25T10:06:22.537 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:06:22.644 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627777 2024-04-25T10:06:22.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:22.653 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:06:22.653 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:06:22.653 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:06:22.653 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:06:22.654 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-04-25T10:06:22.654 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:06:22.760 INFO:teuthology.orchestra.run.smithi016.stdout:1099511628280 2024-04-25T10:06:22.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:22.769 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:06:22.769 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:22.825 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:06:22.826 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:22.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:23.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.019+0000 7f5492570640 1 Processor -- start 2024-04-25T10:06:23.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.019+0000 7f5492570640 1 -- start start 2024-04-25T10:06:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.019+0000 7f5492570640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f548c15dce0 0x7f548c0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.019+0000 7f5492570640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548c15e0c0 con 0x7f548c0d35b0 2024-04-25T10:06:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.019+0000 7f5492570640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548c0d2510 con 0x7f548c0b5b60 2024-04-25T10:06:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.019+0000 7f5492570640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f548c0d27f0 con 0x7f548c15dce0 2024-04-25T10:06:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.019+0000 7f548bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f548c15dce0 0x7f548c0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.020+0000 7f548b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f548c0d35b0 0x7f548c0b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49574/0 (socket says 172.21.15.16:49574) 2024-04-25T10:06:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.020+0000 7f548b7fe640 1 -- 172.21.15.16:0/2897762375 learned_addr learned my addr 172.21.15.16:0/2897762375 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.020+0000 7f548bfff640 1 -- 172.21.15.16:0/2897762375 >> v1:172.21.15.186:6789/0 conn(0x7f548c0b5b60 legacy=0x7f548c0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.020+0000 7f548bfff640 1 -- 172.21.15.16:0/2897762375 >> v1:172.21.15.16:6789/0 conn(0x7f548c0d35b0 legacy=0x7f548c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.020+0000 7f548bfff640 1 -- 172.21.15.16:0/2897762375 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c0d2ad0 con 0x7f548c15dce0 2024-04-25T10:06:23.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.020+0000 7f548bfff640 1 --2- 172.21.15.16:0/2897762375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c15dce0 0x7f548c0d1a50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f547c0097a0 tx=0x7f547c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fb273e0bb1de52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.021+0000 7f548affd640 1 -- 172.21.15.16:0/2897762375 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c037070 con 0x7f548c15dce0 2024-04-25T10:06:23.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.021+0000 7f548affd640 1 -- 172.21.15.16:0/2897762375 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c032440 con 0x7f548c15dce0 2024-04-25T10:06:23.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.021+0000 7f548affd640 1 -- 172.21.15.16:0/2897762375 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c032810 con 0x7f548c15dce0 2024-04-25T10:06:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.021+0000 7f5492570640 1 -- 172.21.15.16:0/2897762375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c15dce0 msgr2=0x7f548c0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.021+0000 7f5492570640 1 --2- 172.21.15.16:0/2897762375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c15dce0 0x7f548c0d1a50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f547c0097a0 tx=0x7f547c02a230 comp rx=0 tx=0).stop 2024-04-25T10:06:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.021+0000 7f5492570640 1 -- 172.21.15.16:0/2897762375 shutdown_connections 2024-04-25T10:06:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.021+0000 7f5492570640 1 --2- 172.21.15.16:0/2897762375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c15dce0 0x7f548c0d1a50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.021+0000 7f5492570640 1 -- 172.21.15.16:0/2897762375 >> 172.21.15.16:0/2897762375 conn(0x7f548c072900 msgr2=0x7f548c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.021+0000 7f5492570640 1 -- 172.21.15.16:0/2897762375 shutdown_connections 2024-04-25T10:06:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.021+0000 7f5492570640 1 -- 172.21.15.16:0/2897762375 wait complete. 2024-04-25T10:06:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.022+0000 7f5492570640 1 Processor -- start 2024-04-25T10:06:23.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.023+0000 7f5492570640 1 -- start start 2024-04-25T10:06:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.023+0000 7f5492570640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c0b5b60 0x7f548c154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.023+0000 7f5492570640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548c0d35b0 0x7f548c15f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.023+0000 7f5492570640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548c15dce0 0x7f548c0c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.023+0000 7f5492570640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c0b9820 con 0x7f548c0b5b60 2024-04-25T10:06:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.023+0000 7f5492570640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c0d3090 con 0x7f548c0d35b0 2024-04-25T10:06:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.023+0000 7f5492570640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548c0b99a0 con 0x7f548c15dce0 2024-04-25T10:06:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.023+0000 7f548bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c0b5b60 0x7f548c154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.023+0000 7f548bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c0b5b60 0x7f548c154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49276/0 (socket says 172.21.15.16:49276) 2024-04-25T10:06:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.023+0000 7f548bfff640 1 -- 172.21.15.16:0/4212669552 learned_addr learned my addr 172.21.15.16:0/4212669552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.024+0000 7f5490b0d640 1 --2- 172.21.15.16:0/4212669552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548c15dce0 0x7f548c0c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.024+0000 7f548b7fe640 1 --2- 172.21.15.16:0/4212669552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548c0d35b0 0x7f548c15f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.024+0000 7f548bfff640 1 -- 172.21.15.16:0/4212669552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548c15dce0 msgr2=0x7f548c0c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.024+0000 7f548bfff640 1 --2- 172.21.15.16:0/4212669552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548c15dce0 0x7f548c0c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.024+0000 7f548bfff640 1 -- 172.21.15.16:0/4212669552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548c0d35b0 msgr2=0x7f548c15f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.024+0000 7f548bfff640 1 --2- 172.21.15.16:0/4212669552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548c0d35b0 0x7f548c15f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.024+0000 7f548bfff640 1 -- 172.21.15.16:0/4212669552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c15ff00 con 0x7f548c0b5b60 2024-04-25T10:06:23.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.024+0000 7f548b7fe640 1 --2- 172.21.15.16:0/4212669552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548c0d35b0 0x7f548c15f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:23.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.024+0000 7f5490b0d640 1 --2- 172.21.15.16:0/4212669552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548c15dce0 0x7f548c0c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:23.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.024+0000 7f548bfff640 1 --2- 172.21.15.16:0/4212669552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c0b5b60 0x7f548c154df0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f547c035040 tx=0x7f547c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.024+0000 7f5488ff9640 1 -- 172.21.15.16:0/4212669552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c037040 con 0x7f548c0b5b60 2024-04-25T10:06:23.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.024+0000 7f5488ff9640 1 -- 172.21.15.16:0/4212669552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c030770 con 0x7f548c0b5b60 2024-04-25T10:06:23.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.024+0000 7f5492570640 1 -- 172.21.15.16:0/4212669552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548c1601e0 con 0x7f548c0b5b60 2024-04-25T10:06:23.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.024+0000 7f5488ff9640 1 -- 172.21.15.16:0/4212669552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c030b80 con 0x7f548c0b5b60 2024-04-25T10:06:23.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.024+0000 7f5492570640 1 -- 172.21.15.16:0/4212669552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548c1607a0 con 0x7f548c0b5b60 2024-04-25T10:06:23.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.025+0000 7f5492570640 1 -- 172.21.15.16:0/4212669552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f548c0749b0 con 0x7f548c0b5b60 2024-04-25T10:06:23.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.025+0000 7f5488ff9640 1 -- 172.21.15.16:0/4212669552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f547c044020 con 0x7f548c0b5b60 2024-04-25T10:06:23.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.026+0000 7f5488ff9640 1 --2- 172.21.15.16:0/4212669552 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f547806b500 0x7f547806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.026+0000 7f5488ff9640 1 -- 172.21.15.16:0/4212669552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f547c0aa720 con 0x7f548c0b5b60 2024-04-25T10:06:23.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.026+0000 7f548b7fe640 1 --2- 172.21.15.16:0/4212669552 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f547806b500 0x7f547806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.027+0000 7f548b7fe640 1 --2- 172.21.15.16:0/4212669552 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f547806b500 0x7f547806d9c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f548c15c7e0 tx=0x7f5480009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.030+0000 7f5488ff9640 1 -- 172.21.15.16:0/4212669552 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f547c032e20 con 0x7f548c0b5b60 2024-04-25T10:06:23.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.210+0000 7f5492570640 1 -- 172.21.15.16:0/4212669552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f548c0b72b0 con 0x7f548c0b5b60 2024-04-25T10:06:23.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.211+0000 7f5488ff9640 1 -- 172.21.15.16:0/4212669552 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f547c079630 con 0x7f548c0b5b60 2024-04-25T10:06:23.217 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:23.217 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":36,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:06:12.582767+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:08.830059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:09.922618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:23.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.213+0000 7f5492570640 1 -- 172.21.15.16:0/4212669552 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f547806b500 msgr2=0x7f547806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.213+0000 7f5492570640 1 --2- 172.21.15.16:0/4212669552 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f547806b500 0x7f547806d9c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f548c15c7e0 tx=0x7f5480009040 comp rx=0 tx=0).stop 2024-04-25T10:06:23.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.213+0000 7f5492570640 1 -- 172.21.15.16:0/4212669552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c0b5b60 msgr2=0x7f548c154df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.213+0000 7f5492570640 1 --2- 172.21.15.16:0/4212669552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c0b5b60 0x7f548c154df0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f547c035040 tx=0x7f547c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.213+0000 7f5492570640 1 -- 172.21.15.16:0/4212669552 shutdown_connections 2024-04-25T10:06:23.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.213+0000 7f5492570640 1 --2- 172.21.15.16:0/4212669552 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f547806b500 0x7f547806d9c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.213+0000 7f5492570640 1 --2- 172.21.15.16:0/4212669552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548c15dce0 0x7f548c0c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.213+0000 7f5492570640 1 --2- 172.21.15.16:0/4212669552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548c0d35b0 0x7f548c15f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.213+0000 7f5492570640 1 --2- 172.21.15.16:0/4212669552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c0b5b60 0x7f548c154df0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.213+0000 7f5492570640 1 -- 172.21.15.16:0/4212669552 >> 172.21.15.16:0/4212669552 conn(0x7f548c072900 msgr2=0x7f548c0ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.213+0000 7f5492570640 1 -- 172.21.15.16:0/4212669552 shutdown_connections 2024-04-25T10:06:23.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.213+0000 7f5492570640 1 -- 172.21.15.16:0/4212669552 wait complete. 2024-04-25T10:06:23.231 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:08.830059+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:09.922618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:23.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.330+0000 7f1e276f5640 1 Processor -- start 2024-04-25T10:06:23.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.330+0000 7f1e276f5640 1 -- start start 2024-04-25T10:06:23.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.331+0000 7f1e276f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e200d3b90 0x7f1e200d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.331+0000 7f1e276f5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e200d4090 con 0x7f1e200d0f60 2024-04-25T10:06:23.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.331+0000 7f1e276f5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e200d4370 con 0x7f1e2015dca0 2024-04-25T10:06:23.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.331+0000 7f1e276f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e200b6b20 con 0x7f1e200d3b90 2024-04-25T10:06:23.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.331+0000 7f1e25c92640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1e200d0f60 0x7f1e200b2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49584/0 (socket says 172.21.15.16:49584) 2024-04-25T10:06:23.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.331+0000 7f1e25c92640 1 -- 172.21.15.16:0/1264568855 learned_addr learned my addr 172.21.15.16:0/1264568855 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:23.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.331+0000 7f1e24c90640 1 --2- 172.21.15.16:0/1264568855 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e200d3b90 0x7f1e200d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.331+0000 7f1e1ffff640 1 -- 172.21.15.16:0/1264568855 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127184973 0 0) 0x7f1e200d4090 con 0x7f1e200d0f60 2024-04-25T10:06:23.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e1ffff640 1 -- 172.21.15.16:0/1264568855 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e08003520 con 0x7f1e200d0f60 2024-04-25T10:06:23.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e1ffff640 1 -- 172.21.15.16:0/1264568855 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3971750259 0 0) 0x7f1e08003520 con 0x7f1e200d0f60 2024-04-25T10:06:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e1ffff640 1 -- 172.21.15.16:0/1264568855 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e200d3b90 msgr2=0x7f1e200d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e1ffff640 1 --2- 172.21.15.16:0/1264568855 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e200d3b90 0x7f1e200d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e1ffff640 1 -- 172.21.15.16:0/1264568855 >> v1:172.21.15.186:6789/0 conn(0x7f1e2015dca0 legacy=0x7f1e200ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e1ffff640 1 -- 172.21.15.16:0/1264568855 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e200d4650 con 0x7f1e200d0f60 2024-04-25T10:06:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e1ffff640 1 -- 172.21.15.16:0/1264568855 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f1e100019f0 con 0x7f1e200d0f60 2024-04-25T10:06:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e1ffff640 1 -- 172.21.15.16:0/1264568855 >> v1:172.21.15.16:6789/0 conn(0x7f1e200d0f60 legacy=0x7f1e200b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e1ffff640 1 --2- 172.21.15.16:0/1264568855 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e200d3b90 0x7f1e080039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e1ffff640 1 --2- 172.21.15.16:0/1264568855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e08003f60 0x7f1e08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e1ffff640 1 --2- 172.21.15.16:0/1264568855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e08004d40 0x7f1e080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e1ffff640 1 -- 172.21.15.16:0/1264568855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0800ba70 con 0x7f1e08003f60 2024-04-25T10:06:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e1ffff640 1 -- 172.21.15.16:0/1264568855 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0800bc20 con 0x7f1e08004d40 2024-04-25T10:06:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e1ffff640 1 -- 172.21.15.16:0/1264568855 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0800be20 con 0x7f1e200d3b90 2024-04-25T10:06:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e24c90640 1 --2- 172.21.15.16:0/1264568855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e08003f60 0x7f1e08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e25c92640 1 --2- 172.21.15.16:0/1264568855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e08004d40 0x7f1e080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e25491640 1 --2- 172.21.15.16:0/1264568855 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e200d3b90 0x7f1e080039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e24c90640 1 -- 172.21.15.16:0/1264568855 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e200d3b90 msgr2=0x7f1e080039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e24c90640 1 --2- 172.21.15.16:0/1264568855 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e200d3b90 0x7f1e080039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e24c90640 1 -- 172.21.15.16:0/1264568855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e08004d40 msgr2=0x7f1e080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e24c90640 1 --2- 172.21.15.16:0/1264568855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e08004d40 0x7f1e080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.332+0000 7f1e24c90640 1 -- 172.21.15.16:0/1264568855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e08003520 con 0x7f1e08003f60 2024-04-25T10:06:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.333+0000 7f1e24c90640 1 --2- 172.21.15.16:0/1264568855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e08003f60 0x7f1e08004800 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009a90 tx=0x7f1e08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da47a701432f7b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.333+0000 7f1e1ffff640 1 -- 172.21.15.16:0/1264568855 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14036070 con 0x7f1e08003f60 2024-04-25T10:06:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.333+0000 7f1e1ffff640 1 -- 172.21.15.16:0/1264568855 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e14031400 con 0x7f1e08003f60 2024-04-25T10:06:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.333+0000 7f1e1ffff640 1 -- 172.21.15.16:0/1264568855 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14009690 con 0x7f1e08003f60 2024-04-25T10:06:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.334+0000 7f1e276f5640 1 -- 172.21.15.16:0/1264568855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e08003f60 msgr2=0x7f1e08004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.334+0000 7f1e276f5640 1 --2- 172.21.15.16:0/1264568855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e08003f60 0x7f1e08004800 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009a90 tx=0x7f1e08005830 comp rx=0 tx=0).stop 2024-04-25T10:06:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.334+0000 7f1e25c92640 1 -- 172.21.15.16:0/1264568855 reap_dead start 2024-04-25T10:06:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.334+0000 7f1e276f5640 1 -- 172.21.15.16:0/1264568855 shutdown_connections 2024-04-25T10:06:23.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.334+0000 7f1e276f5640 1 -- 172.21.15.16:0/1264568855 >> 172.21.15.16:0/1264568855 conn(0x7f1e200726c0 msgr2=0x7f1e20072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.334+0000 7f1e276f5640 1 -- 172.21.15.16:0/1264568855 shutdown_connections 2024-04-25T10:06:23.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.334+0000 7f1e276f5640 1 -- 172.21.15.16:0/1264568855 wait complete. 2024-04-25T10:06:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.335+0000 7f1e276f5640 1 Processor -- start 2024-04-25T10:06:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.335+0000 7f1e276f5640 1 -- start start 2024-04-25T10:06:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.336+0000 7f1e276f5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e08003f60 0x7f1e200cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.336+0000 7f1e276f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e20157930 0x7f1e20157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.336+0000 7f1e276f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e2006c080 0x7f1e2006c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.336+0000 7f1e276f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2015e080 con 0x7f1e2006c080 2024-04-25T10:06:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.336+0000 7f1e276f5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e200cf050 con 0x7f1e08003f60 2024-04-25T10:06:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.336+0000 7f1e276f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2015e200 con 0x7f1e20157930 2024-04-25T10:06:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.336+0000 7f1e25c92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e2006c080 0x7f1e2006c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.336+0000 7f1e25c92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e2006c080 0x7f1e2006c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49286/0 (socket says 172.21.15.16:49286) 2024-04-25T10:06:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.336+0000 7f1e25c92640 1 -- 172.21.15.16:0/1259386038 learned_addr learned my addr 172.21.15.16:0/1259386038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.336+0000 7f1e25491640 1 --2- 172.21.15.16:0/1259386038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e08003f60 0x7f1e200cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.336+0000 7f1e24c90640 1 --2- 172.21.15.16:0/1259386038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e20157930 0x7f1e20157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.336+0000 7f1e25c92640 1 -- 172.21.15.16:0/1259386038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e20157930 msgr2=0x7f1e20157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.336+0000 7f1e25c92640 1 --2- 172.21.15.16:0/1259386038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e20157930 0x7f1e20157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.336+0000 7f1e25c92640 1 -- 172.21.15.16:0/1259386038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e08003f60 msgr2=0x7f1e200cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.336+0000 7f1e25c92640 1 --2- 172.21.15.16:0/1259386038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e08003f60 0x7f1e200cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.336+0000 7f1e25c92640 1 -- 172.21.15.16:0/1259386038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e201580a0 con 0x7f1e2006c080 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.336+0000 7f1e25c92640 1 --2- 172.21.15.16:0/1259386038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e2006c080 0x7f1e2006c500 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1e2006d190 tx=0x7f1e1000e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.337+0000 7f1e1dffb640 1 -- 172.21.15.16:0/1259386038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1000eb70 con 0x7f1e2006c080 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.337+0000 7f1e1dffb640 1 -- 172.21.15.16:0/1259386038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e10018070 con 0x7f1e2006c080 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.337+0000 7f1e276f5640 1 -- 172.21.15.16:0/1259386038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e20158380 con 0x7f1e2006c080 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.337+0000 7f1e1dffb640 1 -- 172.21.15.16:0/1259386038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10013420 con 0x7f1e2006c080 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.337+0000 7f1e276f5640 1 -- 172.21.15.16:0/1259386038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e20158940 con 0x7f1e2006c080 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.338+0000 7f1e276f5640 1 -- 172.21.15.16:0/1259386038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e200750b0 con 0x7f1e2006c080 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.338+0000 7f1e1dffb640 1 -- 172.21.15.16:0/1259386038 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e100135c0 con 0x7f1e2006c080 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.338+0000 7f1e1dffb640 1 --2- 172.21.15.16:0/1259386038 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1e0c06b500 0x7f1e0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.339+0000 7f1e1dffb640 1 -- 172.21.15.16:0/1259386038 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e1008a520 con 0x7f1e2006c080 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.339+0000 7f1e25491640 1 --2- 172.21.15.16:0/1259386038 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1e0c06b500 0x7f1e0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.339+0000 7f1e25491640 1 --2- 172.21.15.16:0/1259386038 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1e0c06b500 0x7f1e0c06d9c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1e18007fd0 tx=0x7f1e18009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.343+0000 7f1e1dffb640 1 -- 172.21.15.16:0/1259386038 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1e100593a0 con 0x7f1e2006c080 2024-04-25T10:06:23.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.526+0000 7f1e276f5640 1 -- 172.21.15.16:0/1259386038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e200b7c20 con 0x7f1e2006c080 2024-04-25T10:06:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.526+0000 7f1e1dffb640 1 -- 172.21.15.16:0/1259386038 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e10058ca0 con 0x7f1e2006c080 2024-04-25T10:06:23.532 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:06:23.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.528+0000 7f1e276f5640 1 -- 172.21.15.16:0/1259386038 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1e0c06b500 msgr2=0x7f1e0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.528+0000 7f1e276f5640 1 --2- 172.21.15.16:0/1259386038 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1e0c06b500 0x7f1e0c06d9c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1e18007fd0 tx=0x7f1e18009040 comp rx=0 tx=0).stop 2024-04-25T10:06:23.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.528+0000 7f1e276f5640 1 -- 172.21.15.16:0/1259386038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e2006c080 msgr2=0x7f1e2006c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.528+0000 7f1e276f5640 1 --2- 172.21.15.16:0/1259386038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e2006c080 0x7f1e2006c500 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1e2006d190 tx=0x7f1e1000e160 comp rx=0 tx=0).stop 2024-04-25T10:06:23.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.529+0000 7f1e276f5640 1 -- 172.21.15.16:0/1259386038 shutdown_connections 2024-04-25T10:06:23.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.529+0000 7f1e276f5640 1 --2- 172.21.15.16:0/1259386038 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1e0c06b500 0x7f1e0c06d9c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.529+0000 7f1e276f5640 1 --2- 172.21.15.16:0/1259386038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e2006c080 0x7f1e2006c500 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.529+0000 7f1e276f5640 1 --2- 172.21.15.16:0/1259386038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e20157930 0x7f1e20157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.529+0000 7f1e276f5640 1 --2- 172.21.15.16:0/1259386038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e08003f60 0x7f1e200cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.529+0000 7f1e276f5640 1 -- 172.21.15.16:0/1259386038 >> 172.21.15.16:0/1259386038 conn(0x7f1e200726c0 msgr2=0x7f1e200b4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.529+0000 7f1e276f5640 1 -- 172.21.15.16:0/1259386038 shutdown_connections 2024-04-25T10:06:23.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.529+0000 7f1e276f5640 1 -- 172.21.15.16:0/1259386038 wait complete. 2024-04-25T10:06:23.545 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:23.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.643+0000 7f624889c640 1 Processor -- start 2024-04-25T10:06:23.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.643+0000 7f624889c640 1 -- start start 2024-04-25T10:06:23.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.644+0000 7f624889c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62440aea60 0x7f62440ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.644+0000 7f624889c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f62440cfb80 0x7f62440cab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.644+0000 7f624889c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62440ce060 0x7f624415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.644+0000 7f624889c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f624406b1d0 con 0x7f62440aea60 2024-04-25T10:06:23.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.644+0000 7f624889c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f624406aba0 con 0x7f62440cfb80 2024-04-25T10:06:23.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.644+0000 7f624889c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62440cf360 con 0x7f62440ce060 2024-04-25T10:06:23.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.644+0000 7f624259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62440aea60 0x7f62440ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.644+0000 7f624259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62440aea60 0x7f62440ca5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49300/0 (socket says 172.21.15.16:49300) 2024-04-25T10:06:23.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.644+0000 7f624259c640 1 -- 172.21.15.16:0/3664148397 learned_addr learned my addr 172.21.15.16:0/3664148397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:23.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.644+0000 7f6241d9b640 1 --2- 172.21.15.16:0/3664148397 >> v2:172.21.15.186:3300/0 conn(0x7f62440cfb80 0x7f62440cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.644+0000 7f6242d9d640 1 --2- 172.21.15.16:0/3664148397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62440ce060 0x7f624415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.644+0000 7f624259c640 1 -- 172.21.15.16:0/3664148397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62440ce060 msgr2=0x7f624415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.644+0000 7f624259c640 1 --2- 172.21.15.16:0/3664148397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62440ce060 0x7f624415f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.644+0000 7f624259c640 1 -- 172.21.15.16:0/3664148397 >> v2:172.21.15.186:3300/0 conn(0x7f62440cfb80 msgr2=0x7f62440cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.644+0000 7f624259c640 1 --2- 172.21.15.16:0/3664148397 >> v2:172.21.15.186:3300/0 conn(0x7f62440cfb80 0x7f62440cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.644+0000 7f624259c640 1 -- 172.21.15.16:0/3664148397 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62440cf640 con 0x7f62440aea60 2024-04-25T10:06:23.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.644+0000 7f6241d9b640 1 --2- 172.21.15.16:0/3664148397 >> v2:172.21.15.186:3300/0 conn(0x7f62440cfb80 0x7f62440cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.645+0000 7f624259c640 1 --2- 172.21.15.16:0/3664148397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62440aea60 0x7f62440ca5d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6244071650 tx=0x7f623c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3153091c3faaeda2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.645+0000 7f624159a640 1 -- 172.21.15.16:0/3664148397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c037070 con 0x7f62440aea60 2024-04-25T10:06:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.645+0000 7f624159a640 1 -- 172.21.15.16:0/3664148397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623c032440 con 0x7f62440aea60 2024-04-25T10:06:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.645+0000 7f624159a640 1 -- 172.21.15.16:0/3664148397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c032810 con 0x7f62440aea60 2024-04-25T10:06:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.646+0000 7f624889c640 1 -- 172.21.15.16:0/3664148397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62440aea60 msgr2=0x7f62440ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.646+0000 7f624889c640 1 --2- 172.21.15.16:0/3664148397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62440aea60 0x7f62440ca5d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6244071650 tx=0x7f623c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.646+0000 7f624889c640 1 -- 172.21.15.16:0/3664148397 shutdown_connections 2024-04-25T10:06:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.646+0000 7f624889c640 1 --2- 172.21.15.16:0/3664148397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62440ce060 0x7f624415f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.646+0000 7f624889c640 1 --2- 172.21.15.16:0/3664148397 >> v2:172.21.15.186:3300/0 conn(0x7f62440cfb80 0x7f62440cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.646+0000 7f624889c640 1 --2- 172.21.15.16:0/3664148397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62440aea60 0x7f62440ca5d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.646+0000 7f624889c640 1 -- 172.21.15.16:0/3664148397 >> 172.21.15.16:0/3664148397 conn(0x7f62440cca00 msgr2=0x7f62440cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.646+0000 7f624889c640 1 -- 172.21.15.16:0/3664148397 shutdown_connections 2024-04-25T10:06:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.646+0000 7f624889c640 1 -- 172.21.15.16:0/3664148397 wait complete. 2024-04-25T10:06:23.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.647+0000 7f624889c640 1 Processor -- start 2024-04-25T10:06:23.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.647+0000 7f624889c640 1 -- start start 2024-04-25T10:06:23.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.648+0000 7f624889c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62440aea60 0x7f62440b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.648+0000 7f624889c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62440ce060 0x7f6244148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.648+0000 7f624889c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62440cfb80 0x7f62441494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.648+0000 7f624889c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62440aefc0 con 0x7f62440aea60 2024-04-25T10:06:23.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.648+0000 7f624889c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62440aee40 con 0x7f62440cfb80 2024-04-25T10:06:23.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.648+0000 7f624889c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62440cef80 con 0x7f62440ce060 2024-04-25T10:06:23.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.648+0000 7f624259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62440aea60 0x7f62440b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.648+0000 7f624259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62440aea60 0x7f62440b4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49314/0 (socket says 172.21.15.16:49314) 2024-04-25T10:06:23.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.648+0000 7f624259c640 1 -- 172.21.15.16:0/2283157879 learned_addr learned my addr 172.21.15.16:0/2283157879 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:23.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.648+0000 7f6242d9d640 1 --2- 172.21.15.16:0/2283157879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62440cfb80 0x7f62441494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.648+0000 7f6241d9b640 1 --2- 172.21.15.16:0/2283157879 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62440ce060 0x7f6244148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.649+0000 7f624259c640 1 -- 172.21.15.16:0/2283157879 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62440ce060 msgr2=0x7f6244148fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.649+0000 7f624259c640 1 --2- 172.21.15.16:0/2283157879 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62440ce060 0x7f6244148fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.649+0000 7f624259c640 1 -- 172.21.15.16:0/2283157879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62440cfb80 msgr2=0x7f62441494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.649+0000 7f624259c640 1 --2- 172.21.15.16:0/2283157879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62440cfb80 0x7f62441494e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.649+0000 7f624259c640 1 -- 172.21.15.16:0/2283157879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62441f4a10 con 0x7f62440aea60 2024-04-25T10:06:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.649+0000 7f6241d9b640 1 --2- 172.21.15.16:0/2283157879 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62440ce060 0x7f6244148fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.649+0000 7f624259c640 1 --2- 172.21.15.16:0/2283157879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62440aea60 0x7f62440b4820 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f623c02a220 tx=0x7f623c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.649+0000 7f623affd640 1 -- 172.21.15.16:0/2283157879 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c037040 con 0x7f62440aea60 2024-04-25T10:06:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.649+0000 7f623affd640 1 -- 172.21.15.16:0/2283157879 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623c042070 con 0x7f62440aea60 2024-04-25T10:06:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.649+0000 7f624889c640 1 -- 172.21.15.16:0/2283157879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62441f4cf0 con 0x7f62440aea60 2024-04-25T10:06:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.649+0000 7f623affd640 1 -- 172.21.15.16:0/2283157879 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c030bb0 con 0x7f62440aea60 2024-04-25T10:06:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.649+0000 7f624889c640 1 -- 172.21.15.16:0/2283157879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62441f52b0 con 0x7f62440aea60 2024-04-25T10:06:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.650+0000 7f624889c640 1 -- 172.21.15.16:0/2283157879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62440750b0 con 0x7f62440aea60 2024-04-25T10:06:23.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.650+0000 7f623affd640 1 -- 172.21.15.16:0/2283157879 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f623c047460 con 0x7f62440aea60 2024-04-25T10:06:23.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.651+0000 7f623affd640 1 --2- 172.21.15.16:0/2283157879 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f622406b4d0 0x7f622406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.651+0000 7f623affd640 1 -- 172.21.15.16:0/2283157879 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f623c0aad20 con 0x7f62440aea60 2024-04-25T10:06:23.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.651+0000 7f6241d9b640 1 --2- 172.21.15.16:0/2283157879 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f622406b4d0 0x7f622406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.652+0000 7f6241d9b640 1 --2- 172.21.15.16:0/2283157879 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f622406b4d0 0x7f622406d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6244157580 tx=0x7f622c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.655+0000 7f623affd640 1 -- 172.21.15.16:0/2283157879 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f623c079c30 con 0x7f62440aea60 2024-04-25T10:06:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.835+0000 7f624889c640 1 -- 172.21.15.16:0/2283157879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62440b3db0 con 0x7f62440aea60 2024-04-25T10:06:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.835+0000 7f623affd640 1 -- 172.21.15.16:0/2283157879 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f623c079530 con 0x7f62440aea60 2024-04-25T10:06:23.842 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:06:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.837+0000 7f624889c640 1 -- 172.21.15.16:0/2283157879 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f622406b4d0 msgr2=0x7f622406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.837+0000 7f624889c640 1 --2- 172.21.15.16:0/2283157879 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f622406b4d0 0x7f622406d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6244157580 tx=0x7f622c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.837+0000 7f624889c640 1 -- 172.21.15.16:0/2283157879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62440aea60 msgr2=0x7f62440b4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.837+0000 7f624889c640 1 --2- 172.21.15.16:0/2283157879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62440aea60 0x7f62440b4820 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f623c02a220 tx=0x7f623c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.837+0000 7f624889c640 1 -- 172.21.15.16:0/2283157879 shutdown_connections 2024-04-25T10:06:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.837+0000 7f624889c640 1 --2- 172.21.15.16:0/2283157879 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f622406b4d0 0x7f622406d990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.837+0000 7f624889c640 1 --2- 172.21.15.16:0/2283157879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62440cfb80 0x7f62441494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.837+0000 7f624889c640 1 --2- 172.21.15.16:0/2283157879 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62440ce060 0x7f6244148fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.837+0000 7f624889c640 1 --2- 172.21.15.16:0/2283157879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62440aea60 0x7f62440b4820 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.837+0000 7f624889c640 1 -- 172.21.15.16:0/2283157879 >> 172.21.15.16:0/2283157879 conn(0x7f62440cca00 msgr2=0x7f62440acdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.837+0000 7f624889c640 1 -- 172.21.15.16:0/2283157879 shutdown_connections 2024-04-25T10:06:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.837+0000 7f624889c640 1 -- 172.21.15.16:0/2283157879 wait complete. 2024-04-25T10:06:23.854 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:23.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.952+0000 7fa79dcdd640 1 Processor -- start 2024-04-25T10:06:23.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.952+0000 7fa79dcdd640 1 -- start start 2024-04-25T10:06:23.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.953+0000 7fa79dcdd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa7980b97c0 0x7fa7980b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.953+0000 7fa79dcdd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7980b5e40 con 0x7fa79815dc80 2024-04-25T10:06:23.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.953+0000 7fa79dcdd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7981627a0 con 0x7fa7980b5a30 2024-04-25T10:06:23.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.953+0000 7fa79dcdd640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa798161f80 con 0x7fa7980b97c0 2024-04-25T10:06:23.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.953+0000 7fa7977fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa79815dc80 0x7fa798161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49600/0 (socket says 172.21.15.16:49600) 2024-04-25T10:06:23.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.953+0000 7fa7977fe640 1 -- 172.21.15.16:0/3609629622 learned_addr learned my addr 172.21.15.16:0/3609629622 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:23.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa7967fc640 1 -- 172.21.15.16:0/3609629622 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442788227 0 0) 0x7fa788001110 con 0x7fa79815dc80 2024-04-25T10:06:23.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa7967fc640 1 -- 172.21.15.16:0/3609629622 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa780003520 con 0x7fa79815dc80 2024-04-25T10:06:23.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa796ffd640 1 --2- 172.21.15.16:0/3609629622 >> v2:172.21.15.186:3300/0 conn(0x7fa7980b97c0 0x7fa7980b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa7967fc640 1 -- 172.21.15.16:0/3609629622 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1019491502 0 0) 0x7fa780003520 con 0x7fa79815dc80 2024-04-25T10:06:23.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa7967fc640 1 -- 172.21.15.16:0/3609629622 >> v2:172.21.15.186:3300/0 conn(0x7fa7980b97c0 msgr2=0x7fa7980b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa7967fc640 1 --2- 172.21.15.16:0/3609629622 >> v2:172.21.15.186:3300/0 conn(0x7fa7980b97c0 0x7fa7980b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa7967fc640 1 -- 172.21.15.16:0/3609629622 >> v1:172.21.15.186:6789/0 conn(0x7fa7980b5a30 legacy=0x7fa7980d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa7967fc640 1 -- 172.21.15.16:0/3609629622 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7981621e0 con 0x7fa79815dc80 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa7967fc640 1 -- 172.21.15.16:0/3609629622 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fa788001840 con 0x7fa79815dc80 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa796ffd640 1 --2- 172.21.15.16:0/3609629622 >> v2:172.21.15.186:3300/0 conn(0x7fa7980b97c0 0x7fa7980b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa7967fc640 1 -- 172.21.15.16:0/3609629622 >> v1:172.21.15.16:6789/0 conn(0x7fa79815dc80 legacy=0x7fa798161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa7967fc640 1 --2- 172.21.15.16:0/3609629622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7800039b0 0x7fa780003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa7967fc640 1 --2- 172.21.15.16:0/3609629622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa780004800 0x7fa780004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa7967fc640 1 --2- 172.21.15.16:0/3609629622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7800051c0 0x7fa78000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa7967fc640 1 -- 172.21.15.16:0/3609629622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa798161f80 con 0x7fa780004800 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa7967fc640 1 -- 172.21.15.16:0/3609629622 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78000c110 con 0x7fa7800051c0 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa7967fc640 1 -- 172.21.15.16:0/3609629622 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78000c310 con 0x7fa7800039b0 2024-04-25T10:06:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa796ffd640 1 --2- 172.21.15.16:0/3609629622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa780004800 0x7fa780004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa7977fe640 1 --2- 172.21.15.16:0/3609629622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7800039b0 0x7fa780003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa797fff640 1 --2- 172.21.15.16:0/3609629622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7800051c0 0x7fa78000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa796ffd640 1 -- 172.21.15.16:0/3609629622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7800039b0 msgr2=0x7fa780003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa796ffd640 1 --2- 172.21.15.16:0/3609629622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7800039b0 0x7fa780003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa796ffd640 1 -- 172.21.15.16:0/3609629622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7800051c0 msgr2=0x7fa78000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa796ffd640 1 --2- 172.21.15.16:0/3609629622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7800051c0 0x7fa78000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa796ffd640 1 -- 172.21.15.16:0/3609629622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa780003520 con 0x7fa780004800 2024-04-25T10:06:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa7977fe640 1 --2- 172.21.15.16:0/3609629622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7800039b0 0x7fa780003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa797fff640 1 --2- 172.21.15.16:0/3609629622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7800051c0 0x7fa78000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.954+0000 7fa797fff640 1 -- 172.21.15.16:0/3609629622 reap_dead start 2024-04-25T10:06:23.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.955+0000 7fa796ffd640 1 --2- 172.21.15.16:0/3609629622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa780004800 0x7fa780004c80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0098a0 tx=0x7fa780005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2b32a3ab67f27f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.955+0000 7fa7967fc640 1 -- 172.21.15.16:0/3609629622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c00f040 con 0x7fa780004800 2024-04-25T10:06:23.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.955+0000 7fa7967fc640 1 -- 172.21.15.16:0/3609629622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78c02ace0 con 0x7fa780004800 2024-04-25T10:06:23.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.955+0000 7fa7967fc640 1 -- 172.21.15.16:0/3609629622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c004250 con 0x7fa780004800 2024-04-25T10:06:23.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.956+0000 7fa79dcdd640 1 -- 172.21.15.16:0/3609629622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa780004800 msgr2=0x7fa780004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.956+0000 7fa79dcdd640 1 --2- 172.21.15.16:0/3609629622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa780004800 0x7fa780004c80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0098a0 tx=0x7fa780005830 comp rx=0 tx=0).stop 2024-04-25T10:06:23.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.956+0000 7fa79dcdd640 1 -- 172.21.15.16:0/3609629622 shutdown_connections 2024-04-25T10:06:23.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.956+0000 7fa79dcdd640 1 --2- 172.21.15.16:0/3609629622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa780004800 0x7fa780004c80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.956+0000 7fa79dcdd640 1 -- 172.21.15.16:0/3609629622 >> 172.21.15.16:0/3609629622 conn(0x7fa7980d04f0 msgr2=0x7fa7980d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.956+0000 7fa79dcdd640 1 -- 172.21.15.16:0/3609629622 shutdown_connections 2024-04-25T10:06:23.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.956+0000 7fa79dcdd640 1 -- 172.21.15.16:0/3609629622 wait complete. 2024-04-25T10:06:23.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.957+0000 7fa79dcdd640 1 Processor -- start 2024-04-25T10:06:23.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.957+0000 7fa79dcdd640 1 -- start start 2024-04-25T10:06:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa79dcdd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa780004800 0x7fa7980c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa79dcdd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa798146c80 0x7fa7980c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa79dcdd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7981470e0 0x7fa7980c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa79dcdd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79815e060 con 0x7fa780004800 2024-04-25T10:06:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa79dcdd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7980d1c70 con 0x7fa7981470e0 2024-04-25T10:06:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa79dcdd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7980b54f0 con 0x7fa798146c80 2024-04-25T10:06:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa7977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa780004800 0x7fa7980c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa7977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa780004800 0x7fa7980c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49326/0 (socket says 172.21.15.16:49326) 2024-04-25T10:06:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa7977fe640 1 -- 172.21.15.16:0/219680665 learned_addr learned my addr 172.21.15.16:0/219680665 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa796ffd640 1 --2- 172.21.15.16:0/219680665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa798146c80 0x7fa7980c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa797fff640 1 --2- 172.21.15.16:0/219680665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7981470e0 0x7fa7980c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa7977fe640 1 -- 172.21.15.16:0/219680665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa798146c80 msgr2=0x7fa7980c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa7977fe640 1 --2- 172.21.15.16:0/219680665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa798146c80 0x7fa7980c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa7977fe640 1 -- 172.21.15.16:0/219680665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7981470e0 msgr2=0x7fa7980c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa7977fe640 1 --2- 172.21.15.16:0/219680665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7981470e0 0x7fa7980c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa7977fe640 1 -- 172.21.15.16:0/219680665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79815e1e0 con 0x7fa780004800 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa796ffd640 1 --2- 172.21.15.16:0/219680665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa798146c80 0x7fa7980c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa797fff640 1 --2- 172.21.15.16:0/219680665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7981470e0 0x7fa7980c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.958+0000 7fa7977fe640 1 --2- 172.21.15.16:0/219680665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa780004800 0x7fa7980c8a90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa7880072e0 tx=0x7fa7880077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.959+0000 7fa77ffff640 1 -- 172.21.15.16:0/219680665 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa788019070 con 0x7fa780004800 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.959+0000 7fa77ffff640 1 -- 172.21.15.16:0/219680665 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78800e8f0 con 0x7fa780004800 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.959+0000 7fa79dcdd640 1 -- 172.21.15.16:0/219680665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa798161230 con 0x7fa780004800 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.959+0000 7fa77ffff640 1 -- 172.21.15.16:0/219680665 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78800eca0 con 0x7fa780004800 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.959+0000 7fa79dcdd640 1 -- 172.21.15.16:0/219680665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7980c90a0 con 0x7fa780004800 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.960+0000 7fa79dcdd640 1 -- 172.21.15.16:0/219680665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7980c92a0 con 0x7fa780004800 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.960+0000 7fa77ffff640 1 -- 172.21.15.16:0/219680665 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa788014480 con 0x7fa780004800 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.961+0000 7fa77ffff640 1 --2- 172.21.15.16:0/219680665 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa78406b500 0x7fa78406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.961+0000 7fa77ffff640 1 -- 172.21.15.16:0/219680665 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa788089990 con 0x7fa780004800 2024-04-25T10:06:23.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.961+0000 7fa796ffd640 1 --2- 172.21.15.16:0/219680665 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa78406b500 0x7fa78406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.961+0000 7fa796ffd640 1 --2- 172.21.15.16:0/219680665 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa78406b500 0x7fa78406d9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa78c033000 tx=0x7fa78c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:23.965+0000 7fa77ffff640 1 -- 172.21.15.16:0/219680665 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa7980c92a0 con 0x7fa780004800 2024-04-25T10:06:24.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.144+0000 7fa79dcdd640 1 -- 172.21.15.16:0/219680665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa7980750b0 con 0x7fa780004800 2024-04-25T10:06:24.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.145+0000 7fa77ffff640 1 -- 172.21.15.16:0/219680665 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa7980b5e40 con 0x7fa780004800 2024-04-25T10:06:24.150 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:06:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.147+0000 7fa79dcdd640 1 -- 172.21.15.16:0/219680665 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa78406b500 msgr2=0x7fa78406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.147+0000 7fa79dcdd640 1 --2- 172.21.15.16:0/219680665 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa78406b500 0x7fa78406d9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa78c033000 tx=0x7fa78c02e000 comp rx=0 tx=0).stop 2024-04-25T10:06:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.147+0000 7fa79dcdd640 1 -- 172.21.15.16:0/219680665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa780004800 msgr2=0x7fa7980c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.147+0000 7fa79dcdd640 1 --2- 172.21.15.16:0/219680665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa780004800 0x7fa7980c8a90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa7880072e0 tx=0x7fa7880077a0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.147+0000 7fa79dcdd640 1 -- 172.21.15.16:0/219680665 shutdown_connections 2024-04-25T10:06:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.147+0000 7fa79dcdd640 1 --2- 172.21.15.16:0/219680665 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa78406b500 0x7fa78406d9c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.147+0000 7fa79dcdd640 1 --2- 172.21.15.16:0/219680665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7981470e0 0x7fa7980c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.147+0000 7fa79dcdd640 1 --2- 172.21.15.16:0/219680665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa798146c80 0x7fa7980c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.147+0000 7fa79dcdd640 1 --2- 172.21.15.16:0/219680665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa780004800 0x7fa7980c8a90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.147+0000 7fa79dcdd640 1 -- 172.21.15.16:0/219680665 >> 172.21.15.16:0/219680665 conn(0x7fa7980d04f0 msgr2=0x7fa7980ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.147+0000 7fa79dcdd640 1 -- 172.21.15.16:0/219680665 shutdown_connections 2024-04-25T10:06:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.147+0000 7fa79dcdd640 1 -- 172.21.15.16:0/219680665 wait complete. 2024-04-25T10:06:24.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:06:24.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.259+0000 7f362ee5a640 1 Processor -- start 2024-04-25T10:06:24.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.259+0000 7f362ee5a640 1 -- start start 2024-04-25T10:06:24.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.260+0000 7f362ee5a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f362815dc50 0x7f3628070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.260+0000 7f362ee5a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3628162690 con 0x7f36280cf600 2024-04-25T10:06:24.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.260+0000 7f362ee5a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362806c220 con 0x7f36280cd940 2024-04-25T10:06:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.260+0000 7f362ee5a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36280cfe00 con 0x7f362815dc50 2024-04-25T10:06:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.260+0000 7f362cbf6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f362815dc50 0x7f3628070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.260+0000 7f362d3f7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f36280cf600 0x7f3628161f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49616/0 (socket says 172.21.15.16:49616) 2024-04-25T10:06:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.260+0000 7f362d3f7640 1 -- 172.21.15.16:0/3342037271 learned_addr learned my addr 172.21.15.16:0/3342037271 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.260+0000 7f362cbf6640 1 -- 172.21.15.16:0/3342037271 >> v1:172.21.15.186:6789/0 conn(0x7f36280cd940 legacy=0x7f3628161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.260+0000 7f362cbf6640 1 -- 172.21.15.16:0/3342037271 >> v1:172.21.15.16:6789/0 conn(0x7f36280cf600 legacy=0x7f3628161f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.260+0000 7f362cbf6640 1 -- 172.21.15.16:0/3342037271 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36280cec10 con 0x7f362815dc50 2024-04-25T10:06:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.261+0000 7f362cbf6640 1 --2- 172.21.15.16:0/3342037271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362815dc50 0x7f3628070e80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f36180097a0 tx=0x7f361802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577f7062815a4452 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.261+0000 7f36277fe640 1 -- 172.21.15.16:0/3342037271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618037070 con 0x7f362815dc50 2024-04-25T10:06:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.261+0000 7f36277fe640 1 -- 172.21.15.16:0/3342037271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3618032440 con 0x7f362815dc50 2024-04-25T10:06:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.261+0000 7f36277fe640 1 -- 172.21.15.16:0/3342037271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618032810 con 0x7f362815dc50 2024-04-25T10:06:24.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.262+0000 7f362ee5a640 1 -- 172.21.15.16:0/3342037271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362815dc50 msgr2=0x7f3628070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.262+0000 7f362ee5a640 1 --2- 172.21.15.16:0/3342037271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362815dc50 0x7f3628070e80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f36180097a0 tx=0x7f361802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.262+0000 7f362ee5a640 1 -- 172.21.15.16:0/3342037271 shutdown_connections 2024-04-25T10:06:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.262+0000 7f362ee5a640 1 --2- 172.21.15.16:0/3342037271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362815dc50 0x7f3628070e80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.262+0000 7f362ee5a640 1 -- 172.21.15.16:0/3342037271 >> 172.21.15.16:0/3342037271 conn(0x7f362806f430 msgr2=0x7f362806f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.262+0000 7f362ee5a640 1 -- 172.21.15.16:0/3342037271 shutdown_connections 2024-04-25T10:06:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.262+0000 7f362ee5a640 1 -- 172.21.15.16:0/3342037271 wait complete. 2024-04-25T10:06:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.263+0000 7f362ee5a640 1 Processor -- start 2024-04-25T10:06:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.263+0000 7f362ee5a640 1 -- start start 2024-04-25T10:06:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.264+0000 7f362ee5a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36280cd940 0x7f36281518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.264+0000 7f362ee5a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36280cf600 0x7f36281e8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.264+0000 7f362ee5a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362815dc50 0x7f36281e8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.264+0000 7f362ee5a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36280cc0c0 con 0x7f362815dc50 2024-04-25T10:06:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.264+0000 7f362ee5a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36280d0330 con 0x7f36280cf600 2024-04-25T10:06:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.264+0000 7f362ee5a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36280ce8a0 con 0x7f36280cd940 2024-04-25T10:06:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.264+0000 7f362d3f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362815dc50 0x7f36281e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.264+0000 7f362d3f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362815dc50 0x7f36281e8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49352/0 (socket says 172.21.15.16:49352) 2024-04-25T10:06:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.264+0000 7f362d3f7640 1 -- 172.21.15.16:0/3500626054 learned_addr learned my addr 172.21.15.16:0/3500626054 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.264+0000 7f3627fff640 1 --2- 172.21.15.16:0/3500626054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36280cf600 0x7f36281e8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.264+0000 7f362d3f7640 1 -- 172.21.15.16:0/3500626054 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36280cd940 msgr2=0x7f36281518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.264+0000 7f362d3f7640 1 --2- 172.21.15.16:0/3500626054 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36280cd940 0x7f36281518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.264+0000 7f362d3f7640 1 -- 172.21.15.16:0/3500626054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36280cf600 msgr2=0x7f36281e8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.264+0000 7f362d3f7640 1 --2- 172.21.15.16:0/3500626054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36280cf600 0x7f36281e8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.264+0000 7f362d3f7640 1 -- 172.21.15.16:0/3500626054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3628151e20 con 0x7f362815dc50 2024-04-25T10:06:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.264+0000 7f362d3f7640 1 --2- 172.21.15.16:0/3500626054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362815dc50 0x7f36281e8690 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f362000cbc0 tx=0x7f362000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.265+0000 7f36257fa640 1 -- 172.21.15.16:0/3500626054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620017070 con 0x7f362815dc50 2024-04-25T10:06:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.265+0000 7f36257fa640 1 -- 172.21.15.16:0/3500626054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362000ad40 con 0x7f362815dc50 2024-04-25T10:06:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.265+0000 7f362ee5a640 1 -- 172.21.15.16:0/3500626054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36281e7d70 con 0x7f362815dc50 2024-04-25T10:06:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.265+0000 7f36257fa640 1 -- 172.21.15.16:0/3500626054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620012600 con 0x7f362815dc50 2024-04-25T10:06:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.265+0000 7f362ee5a640 1 -- 172.21.15.16:0/3500626054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36281e7a00 con 0x7f362815dc50 2024-04-25T10:06:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.266+0000 7f362ee5a640 1 -- 172.21.15.16:0/3500626054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36280750b0 con 0x7f362815dc50 2024-04-25T10:06:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.266+0000 7f36257fa640 1 -- 172.21.15.16:0/3500626054 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f362001a070 con 0x7f362815dc50 2024-04-25T10:06:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.266+0000 7f36257fa640 1 --2- 172.21.15.16:0/3500626054 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f361406b4d0 0x7f361406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.267+0000 7f36257fa640 1 -- 172.21.15.16:0/3500626054 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3620089de0 con 0x7f362815dc50 2024-04-25T10:06:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.267+0000 7f362cbf6640 1 --2- 172.21.15.16:0/3500626054 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f361406b4d0 0x7f361406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.267+0000 7f362cbf6640 1 --2- 172.21.15.16:0/3500626054 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f361406b4d0 0x7f361406d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3618034000 tx=0x7f361802c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.271+0000 7f36257fa640 1 -- 172.21.15.16:0/3500626054 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3628151e20 con 0x7f362815dc50 2024-04-25T10:06:24.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.465+0000 7f362ee5a640 1 -- 172.21.15.16:0/3500626054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f362806d180 con 0x7f362815dc50 2024-04-25T10:06:24.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.466+0000 7f36257fa640 1 -- 172.21.15.16:0/3500626054 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7f36200586a0 con 0x7f362815dc50 2024-04-25T10:06:24.471 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-04-25T10:06:24.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.467+0000 7f362ee5a640 1 -- 172.21.15.16:0/3500626054 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f361406b4d0 msgr2=0x7f361406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.467+0000 7f362ee5a640 1 --2- 172.21.15.16:0/3500626054 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f361406b4d0 0x7f361406d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3618034000 tx=0x7f361802c000 comp rx=0 tx=0).stop 2024-04-25T10:06:24.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.467+0000 7f362ee5a640 1 -- 172.21.15.16:0/3500626054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362815dc50 msgr2=0x7f36281e8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.467+0000 7f362ee5a640 1 --2- 172.21.15.16:0/3500626054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362815dc50 0x7f36281e8690 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f362000cbc0 tx=0x7f362000a430 comp rx=0 tx=0).stop 2024-04-25T10:06:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.468+0000 7f362ee5a640 1 -- 172.21.15.16:0/3500626054 shutdown_connections 2024-04-25T10:06:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.468+0000 7f362ee5a640 1 --2- 172.21.15.16:0/3500626054 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f361406b4d0 0x7f361406d990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.468+0000 7f362ee5a640 1 --2- 172.21.15.16:0/3500626054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362815dc50 0x7f36281e8690 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.468+0000 7f362ee5a640 1 --2- 172.21.15.16:0/3500626054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36280cf600 0x7f36281e8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.468+0000 7f362ee5a640 1 --2- 172.21.15.16:0/3500626054 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36280cd940 0x7f36281518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.468+0000 7f362ee5a640 1 -- 172.21.15.16:0/3500626054 >> 172.21.15.16:0/3500626054 conn(0x7f362806f430 msgr2=0x7f362806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.468+0000 7f362ee5a640 1 -- 172.21.15.16:0/3500626054 shutdown_connections 2024-04-25T10:06:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.468+0000 7f362ee5a640 1 -- 172.21.15.16:0/3500626054 wait complete. 2024-04-25T10:06:24.484 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:06:24.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:24.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:24.535 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:06:24.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:24.535 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:06:24.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:24.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.697+0000 7f4236a76640 1 Processor -- start 2024-04-25T10:06:24.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.697+0000 7f4236a76640 1 -- start start 2024-04-25T10:06:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f4236a76640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42300cd320 0x7f42300cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f4236a76640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f42300cfe80 0x7f42300cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f4236a76640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42300cd930 con 0x7f423015df40 2024-04-25T10:06:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f4236a76640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42300cdc10 con 0x7f42300cfe80 2024-04-25T10:06:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f4236a76640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42300cdef0 con 0x7f42300cd320 2024-04-25T10:06:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f4234812640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f423015df40 0x7f42300b19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49622/0 (socket says 172.21.15.16:49622) 2024-04-25T10:06:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f4234812640 1 -- 172.21.15.16:0/3010018182 learned_addr learned my addr 172.21.15.16:0/3010018182 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f422f7fe640 1 -- 172.21.15.16:0/3010018182 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3766809111 0 0) 0x7f42300cd930 con 0x7f423015df40 2024-04-25T10:06:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f422f7fe640 1 -- 172.21.15.16:0/3010018182 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4220003520 con 0x7f423015df40 2024-04-25T10:06:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f422ffff640 1 --2- 172.21.15.16:0/3010018182 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42300cd320 0x7f42300cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f4235013640 1 --2- 172.21.15.16:0/3010018182 >> v2:172.21.15.186:3300/0 conn(0x7f42300cfe80 0x7f42300cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f422f7fe640 1 -- 172.21.15.16:0/3010018182 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2173823225 0 0) 0x7f4220003520 con 0x7f423015df40 2024-04-25T10:06:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f422f7fe640 1 -- 172.21.15.16:0/3010018182 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42300cd320 msgr2=0x7f42300cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f422f7fe640 1 --2- 172.21.15.16:0/3010018182 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42300cd320 0x7f42300cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f422f7fe640 1 -- 172.21.15.16:0/3010018182 >> v2:172.21.15.186:3300/0 conn(0x7f42300cfe80 msgr2=0x7f42300cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f422f7fe640 1 --2- 172.21.15.16:0/3010018182 >> v2:172.21.15.186:3300/0 conn(0x7f42300cfe80 0x7f42300cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f422f7fe640 1 -- 172.21.15.16:0/3010018182 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42300cada0 con 0x7f423015df40 2024-04-25T10:06:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f422f7fe640 1 -- 172.21.15.16:0/3010018182 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f4224001670 con 0x7f423015df40 2024-04-25T10:06:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f422f7fe640 1 -- 172.21.15.16:0/3010018182 >> v1:172.21.15.16:6789/0 conn(0x7f423015df40 legacy=0x7f42300b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.698+0000 7f422f7fe640 1 --2- 172.21.15.16:0/3010018182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42200039b0 0x7f4220003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.699+0000 7f422f7fe640 1 --2- 172.21.15.16:0/3010018182 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42300cd320 0x7f4220004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.699+0000 7f422f7fe640 1 --2- 172.21.15.16:0/3010018182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4220004d40 0x7f42200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.699+0000 7f422f7fe640 1 -- 172.21.15.16:0/3010018182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42300cdc10 con 0x7f4220004d40 2024-04-25T10:06:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.699+0000 7f422f7fe640 1 -- 172.21.15.16:0/3010018182 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42300cdef0 con 0x7f42200039b0 2024-04-25T10:06:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.699+0000 7f422f7fe640 1 -- 172.21.15.16:0/3010018182 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422000bb50 con 0x7f42300cd320 2024-04-25T10:06:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.699+0000 7f4235013640 1 --2- 172.21.15.16:0/3010018182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4220004d40 0x7f42200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.699+0000 7f422ffff640 1 --2- 172.21.15.16:0/3010018182 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42300cd320 0x7f4220004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.699+0000 7f4234812640 1 --2- 172.21.15.16:0/3010018182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42200039b0 0x7f4220003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.699+0000 7f4235013640 1 -- 172.21.15.16:0/3010018182 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42300cd320 msgr2=0x7f4220004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.699+0000 7f4235013640 1 --2- 172.21.15.16:0/3010018182 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42300cd320 0x7f4220004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.699+0000 7f4235013640 1 -- 172.21.15.16:0/3010018182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42200039b0 msgr2=0x7f4220003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.699+0000 7f4235013640 1 --2- 172.21.15.16:0/3010018182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42200039b0 0x7f4220003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.699+0000 7f4235013640 1 -- 172.21.15.16:0/3010018182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4220003520 con 0x7f4220004d40 2024-04-25T10:06:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.700+0000 7f4235013640 1 --2- 172.21.15.16:0/3010018182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4220004d40 0x7f42200051c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4228007360 tx=0x7f4220005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc570ded7d8858b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.700+0000 7f422f7fe640 1 -- 172.21.15.16:0/3010018182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228009be0 con 0x7f4220004d40 2024-04-25T10:06:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.700+0000 7f422f7fe640 1 -- 172.21.15.16:0/3010018182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422802e620 con 0x7f4220004d40 2024-04-25T10:06:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.700+0000 7f422f7fe640 1 -- 172.21.15.16:0/3010018182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422802e9f0 con 0x7f4220004d40 2024-04-25T10:06:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.700+0000 7f4236a76640 1 -- 172.21.15.16:0/3010018182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4220004d40 msgr2=0x7f42200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.700+0000 7f4236a76640 1 --2- 172.21.15.16:0/3010018182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4220004d40 0x7f42200051c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4228007360 tx=0x7f4220005830 comp rx=0 tx=0).stop 2024-04-25T10:06:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.700+0000 7f4235013640 1 -- 172.21.15.16:0/3010018182 reap_dead start 2024-04-25T10:06:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.700+0000 7f4236a76640 1 -- 172.21.15.16:0/3010018182 shutdown_connections 2024-04-25T10:06:24.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.700+0000 7f4236a76640 1 -- 172.21.15.16:0/3010018182 >> 172.21.15.16:0/3010018182 conn(0x7f42300d06b0 msgr2=0x7f42300d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.701+0000 7f4236a76640 1 -- 172.21.15.16:0/3010018182 shutdown_connections 2024-04-25T10:06:24.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.701+0000 7f4236a76640 1 -- 172.21.15.16:0/3010018182 wait complete. 2024-04-25T10:06:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.702+0000 7f4236a76640 1 Processor -- start 2024-04-25T10:06:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.702+0000 7f4236a76640 1 -- start start 2024-04-25T10:06:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4236a76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f423009dcc0 0x7f42300b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4236a76640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f423009a5a0 0x7f423009e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4236a76640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f423009bf00 0x7f42300b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4236a76640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42300c4200 con 0x7f423009dcc0 2024-04-25T10:06:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4236a76640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42300d0310 con 0x7f423009a5a0 2024-04-25T10:06:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4236a76640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42300c4380 con 0x7f423009bf00 2024-04-25T10:06:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4234812640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f423009dcc0 0x7f42300b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4234812640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f423009dcc0 0x7f42300b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49378/0 (socket says 172.21.15.16:49378) 2024-04-25T10:06:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4234812640 1 -- 172.21.15.16:0/4056464179 learned_addr learned my addr 172.21.15.16:0/4056464179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4235013640 1 --2- 172.21.15.16:0/4056464179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f423009bf00 0x7f42300b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f422ffff640 1 --2- 172.21.15.16:0/4056464179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f423009a5a0 0x7f423009e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4234812640 1 -- 172.21.15.16:0/4056464179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f423009bf00 msgr2=0x7f42300b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4234812640 1 --2- 172.21.15.16:0/4056464179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f423009bf00 0x7f42300b50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4234812640 1 -- 172.21.15.16:0/4056464179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f423009a5a0 msgr2=0x7f423009e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4234812640 1 --2- 172.21.15.16:0/4056464179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f423009a5a0 0x7f423009e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4234812640 1 -- 172.21.15.16:0/4056464179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42300b4300 con 0x7f423009dcc0 2024-04-25T10:06:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4235013640 1 --2- 172.21.15.16:0/4056464179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f423009bf00 0x7f42300b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4234812640 1 --2- 172.21.15.16:0/4056464179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f423009dcc0 0x7f42300b5f40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4224004840 tx=0x7f4224007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f422d7fa640 1 -- 172.21.15.16:0/4056464179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422400f990 con 0x7f423009dcc0 2024-04-25T10:06:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f422d7fa640 1 -- 172.21.15.16:0/4056464179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4224018070 con 0x7f423009dcc0 2024-04-25T10:06:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4236a76640 1 -- 172.21.15.16:0/4056464179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42300b45e0 con 0x7f423009dcc0 2024-04-25T10:06:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f422d7fa640 1 -- 172.21.15.16:0/4056464179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422400fdb0 con 0x7f423009dcc0 2024-04-25T10:06:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.703+0000 7f4236a76640 1 -- 172.21.15.16:0/4056464179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42300b4ba0 con 0x7f423009dcc0 2024-04-25T10:06:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.705+0000 7f4236a76640 1 -- 172.21.15.16:0/4056464179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42300749b0 con 0x7f423009dcc0 2024-04-25T10:06:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.705+0000 7f422d7fa640 1 -- 172.21.15.16:0/4056464179 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f422401d460 con 0x7f423009dcc0 2024-04-25T10:06:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.705+0000 7f422d7fa640 1 --2- 172.21.15.16:0/4056464179 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f421406b4d0 0x7f421406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.705+0000 7f422d7fa640 1 -- 172.21.15.16:0/4056464179 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f422408a440 con 0x7f423009dcc0 2024-04-25T10:06:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.706+0000 7f422ffff640 1 --2- 172.21.15.16:0/4056464179 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f421406b4d0 0x7f421406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.706+0000 7f422ffff640 1 --2- 172.21.15.16:0/4056464179 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f421406b4d0 0x7f421406d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f42300b36e0 tx=0x7f421c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.709+0000 7f422d7fa640 1 -- 172.21.15.16:0/4056464179 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f42240592f0 con 0x7f423009dcc0 2024-04-25T10:06:24.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.929+0000 7f4236a76640 1 -- 172.21.15.16:0/4056464179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42300cc530 con 0x7f423009dcc0 2024-04-25T10:06:24.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.930+0000 7f422d7fa640 1 -- 172.21.15.16:0/4056464179 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f4224058bf0 con 0x7f423009dcc0 2024-04-25T10:06:24.935 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:24.935 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3381395304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3381395304},{"type":"v1","addr":"172.21.15.186:6835","nonce":3381395304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2429147785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2429147785},{"type":"v1","addr":"172.21.15.186:6837","nonce":2429147785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5142,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2191514045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2191514045},{"type":"v1","addr":"172.21.15.16:6835","nonce":2191514045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:10.940554+0000","modified":"2024-04-25T10:06:12.930243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34613},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34613":{"gid":34613,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.16:6834/1417313138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1417313138},{"type":"v1","addr":"172.21.15.16:6834","nonce":1417313138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:24.936 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 14 2024-04-25T10:06:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.932+0000 7f4236a76640 1 -- 172.21.15.16:0/4056464179 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f421406b4d0 msgr2=0x7f421406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.932+0000 7f4236a76640 1 --2- 172.21.15.16:0/4056464179 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f421406b4d0 0x7f421406d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f42300b36e0 tx=0x7f421c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.932+0000 7f4236a76640 1 -- 172.21.15.16:0/4056464179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f423009dcc0 msgr2=0x7f42300b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.932+0000 7f4236a76640 1 --2- 172.21.15.16:0/4056464179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f423009dcc0 0x7f42300b5f40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4224004840 tx=0x7f4224007720 comp rx=0 tx=0).stop 2024-04-25T10:06:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.932+0000 7f4236a76640 1 -- 172.21.15.16:0/4056464179 shutdown_connections 2024-04-25T10:06:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.932+0000 7f4236a76640 1 --2- 172.21.15.16:0/4056464179 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f421406b4d0 0x7f421406d990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.932+0000 7f4236a76640 1 --2- 172.21.15.16:0/4056464179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f423009bf00 0x7f42300b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.932+0000 7f4236a76640 1 --2- 172.21.15.16:0/4056464179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f423009a5a0 0x7f423009e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.932+0000 7f4236a76640 1 --2- 172.21.15.16:0/4056464179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f423009dcc0 0x7f42300b5f40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.932+0000 7f4236a76640 1 -- 172.21.15.16:0/4056464179 >> 172.21.15.16:0/4056464179 conn(0x7f42300d06b0 msgr2=0x7f423006e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.932+0000 7f4236a76640 1 -- 172.21.15.16:0/4056464179 shutdown_connections 2024-04-25T10:06:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:24.932+0000 7f4236a76640 1 -- 172.21.15.16:0/4056464179 wait complete. 2024-04-25T10:06:24.949 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:06:25.082 INFO:teuthology.orchestra.run.smithi016.stdout:{"message":"","return_code":0} 2024-04-25T10:06:25.082 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:06:25.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:25.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.178+0000 7f7a4d57e640 1 Processor -- start 2024-04-25T10:06:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.178+0000 7f7a4d57e640 1 -- start start 2024-04-25T10:06:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.179+0000 7f7a4d57e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a480d35b0 0x7f7a480b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.179+0000 7f7a4d57e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a480b5990 0x7f7a480b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.179+0000 7f7a4d57e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4815e0c0 con 0x7f7a4815dce0 2024-04-25T10:06:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.179+0000 7f7a4d57e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a480d2510 con 0x7f7a480d35b0 2024-04-25T10:06:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.179+0000 7f7a4d57e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a480d27f0 con 0x7f7a480b5990 2024-04-25T10:06:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.179+0000 7f7a467fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a480d35b0 0x7f7a480b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.179+0000 7f7a467fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a480d35b0 0x7f7a480b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49386/0 (socket says 172.21.15.16:49386) 2024-04-25T10:06:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.179+0000 7f7a467fc640 1 -- 172.21.15.16:0/3262314949 learned_addr learned my addr 172.21.15.16:0/3262314949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.179+0000 7f7a477fe640 1 --2- 172.21.15.16:0/3262314949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a480b5990 0x7f7a480b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.179+0000 7f7a467fc640 1 -- 172.21.15.16:0/3262314949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a480b5990 msgr2=0x7f7a480b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.179+0000 7f7a467fc640 1 --2- 172.21.15.16:0/3262314949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a480b5990 0x7f7a480b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.179+0000 7f7a467fc640 1 -- 172.21.15.16:0/3262314949 >> v1:172.21.15.186:6789/0 conn(0x7f7a4815dce0 legacy=0x7f7a480d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.179+0000 7f7a467fc640 1 -- 172.21.15.16:0/3262314949 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a480d2ad0 con 0x7f7a480d35b0 2024-04-25T10:06:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.180+0000 7f7a467fc640 1 --2- 172.21.15.16:0/3262314949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a480d35b0 0x7f7a480b5450 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009870 tx=0x7f7a3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9aa16f2f4ec6b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.180+0000 7f7a45ffb640 1 -- 172.21.15.16:0/3262314949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c037070 con 0x7f7a480d35b0 2024-04-25T10:06:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.180+0000 7f7a45ffb640 1 -- 172.21.15.16:0/3262314949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3c032440 con 0x7f7a480d35b0 2024-04-25T10:06:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.180+0000 7f7a45ffb640 1 -- 172.21.15.16:0/3262314949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c032810 con 0x7f7a480d35b0 2024-04-25T10:06:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.181+0000 7f7a4d57e640 1 -- 172.21.15.16:0/3262314949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a480d35b0 msgr2=0x7f7a480b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.181+0000 7f7a4d57e640 1 --2- 172.21.15.16:0/3262314949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a480d35b0 0x7f7a480b5450 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009870 tx=0x7f7a3c02a270 comp rx=0 tx=0).stop 2024-04-25T10:06:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.181+0000 7f7a4d57e640 1 -- 172.21.15.16:0/3262314949 shutdown_connections 2024-04-25T10:06:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.181+0000 7f7a4d57e640 1 --2- 172.21.15.16:0/3262314949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a480b5990 0x7f7a480b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.181+0000 7f7a4d57e640 1 --2- 172.21.15.16:0/3262314949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a480d35b0 0x7f7a480b5450 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.181+0000 7f7a4d57e640 1 -- 172.21.15.16:0/3262314949 >> 172.21.15.16:0/3262314949 conn(0x7f7a48072900 msgr2=0x7f7a480d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.181+0000 7f7a4d57e640 1 -- 172.21.15.16:0/3262314949 shutdown_connections 2024-04-25T10:06:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.181+0000 7f7a4d57e640 1 -- 172.21.15.16:0/3262314949 wait complete. 2024-04-25T10:06:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.182+0000 7f7a4d57e640 1 Processor -- start 2024-04-25T10:06:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.182+0000 7f7a4d57e640 1 -- start start 2024-04-25T10:06:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.183+0000 7f7a4d57e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a480b5990 0x7f7a480c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.183+0000 7f7a4d57e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a480d35b0 0x7f7a48159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.183+0000 7f7a4d57e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a4815dce0 0x7f7a480c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.183+0000 7f7a4d57e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a480d2160 con 0x7f7a480b5990 2024-04-25T10:06:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.183+0000 7f7a4d57e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a480d3090 con 0x7f7a4815dce0 2024-04-25T10:06:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.183+0000 7f7a4d57e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a481614c0 con 0x7f7a480d35b0 2024-04-25T10:06:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.183+0000 7f7a46ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a480b5990 0x7f7a480c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.183+0000 7f7a46ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a480b5990 0x7f7a480c91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49394/0 (socket says 172.21.15.16:49394) 2024-04-25T10:06:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.183+0000 7f7a46ffd640 1 -- 172.21.15.16:0/1828756565 learned_addr learned my addr 172.21.15.16:0/1828756565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.183+0000 7f7a477fe640 1 --2- 172.21.15.16:0/1828756565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a4815dce0 0x7f7a480c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.183+0000 7f7a467fc640 1 --2- 172.21.15.16:0/1828756565 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a480d35b0 0x7f7a48159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.183+0000 7f7a46ffd640 1 -- 172.21.15.16:0/1828756565 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a480d35b0 msgr2=0x7f7a48159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.183+0000 7f7a46ffd640 1 --2- 172.21.15.16:0/1828756565 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a480d35b0 0x7f7a48159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.183+0000 7f7a46ffd640 1 -- 172.21.15.16:0/1828756565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a4815dce0 msgr2=0x7f7a480c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.183+0000 7f7a46ffd640 1 --2- 172.21.15.16:0/1828756565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a4815dce0 0x7f7a480c9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.183+0000 7f7a46ffd640 1 -- 172.21.15.16:0/1828756565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a480c8110 con 0x7f7a480b5990 2024-04-25T10:06:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.183+0000 7f7a477fe640 1 --2- 172.21.15.16:0/1828756565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a4815dce0 0x7f7a480c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.183+0000 7f7a46ffd640 1 --2- 172.21.15.16:0/1828756565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a480b5990 0x7f7a480c91b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800c3c0 tx=0x7f7a3800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.184+0000 7f7a2f7fe640 1 -- 172.21.15.16:0/1828756565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a380052d0 con 0x7f7a480b5990 2024-04-25T10:06:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.184+0000 7f7a2f7fe640 1 -- 172.21.15.16:0/1828756565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a38005490 con 0x7f7a480b5990 2024-04-25T10:06:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.184+0000 7f7a4d57e640 1 -- 172.21.15.16:0/1828756565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a480c83f0 con 0x7f7a480b5990 2024-04-25T10:06:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.184+0000 7f7a2f7fe640 1 -- 172.21.15.16:0/1828756565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a38007260 con 0x7f7a480b5990 2024-04-25T10:06:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.184+0000 7f7a4d57e640 1 -- 172.21.15.16:0/1828756565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a480c89b0 con 0x7f7a480b5990 2024-04-25T10:06:25.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.185+0000 7f7a4d57e640 1 -- 172.21.15.16:0/1828756565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a480749b0 con 0x7f7a480b5990 2024-04-25T10:06:25.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.185+0000 7f7a2f7fe640 1 -- 172.21.15.16:0/1828756565 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a38007400 con 0x7f7a480b5990 2024-04-25T10:06:25.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.186+0000 7f7a2f7fe640 1 --2- 172.21.15.16:0/1828756565 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a3406b500 0x7f7a3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.186+0000 7f7a2f7fe640 1 -- 172.21.15.16:0/1828756565 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a38089650 con 0x7f7a480b5990 2024-04-25T10:06:25.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.186+0000 7f7a467fc640 1 --2- 172.21.15.16:0/1828756565 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a3406b500 0x7f7a3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.186+0000 7f7a467fc640 1 --2- 172.21.15.16:0/1828756565 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a3406b500 0x7f7a3406d9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c034000 tx=0x7f7a3c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.190+0000 7f7a2f7fe640 1 -- 172.21.15.16:0/1828756565 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a380584d0 con 0x7f7a480b5990 2024-04-25T10:06:25.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.405+0000 7f7a4d57e640 1 -- 172.21.15.16:0/1828756565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a4806b080 con 0x7f7a480b5990 2024-04-25T10:06:25.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.405+0000 7f7a2f7fe640 1 -- 172.21.15.16:0/1828756565 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f7a38057dd0 con 0x7f7a480b5990 2024-04-25T10:06:25.411 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:25.411 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3381395304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3381395304},{"type":"v1","addr":"172.21.15.186:6835","nonce":3381395304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2429147785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2429147785},{"type":"v1","addr":"172.21.15.186:6837","nonce":2429147785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5142,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2191514045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2191514045},{"type":"v1","addr":"172.21.15.16:6835","nonce":2191514045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:10.940554+0000","modified":"2024-04-25T10:06:12.930243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34613},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34613":{"gid":34613,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.16:6834/1417313138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1417313138},{"type":"v1","addr":"172.21.15.16:6834","nonce":1417313138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:25.411 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 14 2024-04-25T10:06:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.407+0000 7f7a4d57e640 1 -- 172.21.15.16:0/1828756565 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a3406b500 msgr2=0x7f7a3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.407+0000 7f7a4d57e640 1 --2- 172.21.15.16:0/1828756565 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a3406b500 0x7f7a3406d9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c034000 tx=0x7f7a3c02e000 comp rx=0 tx=0).stop 2024-04-25T10:06:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.407+0000 7f7a4d57e640 1 -- 172.21.15.16:0/1828756565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a480b5990 msgr2=0x7f7a480c91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.407+0000 7f7a4d57e640 1 --2- 172.21.15.16:0/1828756565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a480b5990 0x7f7a480c91b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800c3c0 tx=0x7f7a3800c880 comp rx=0 tx=0).stop 2024-04-25T10:06:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.407+0000 7f7a4d57e640 1 -- 172.21.15.16:0/1828756565 shutdown_connections 2024-04-25T10:06:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.407+0000 7f7a4d57e640 1 --2- 172.21.15.16:0/1828756565 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a3406b500 0x7f7a3406d9c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.407+0000 7f7a4d57e640 1 --2- 172.21.15.16:0/1828756565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a4815dce0 0x7f7a480c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.407+0000 7f7a4d57e640 1 --2- 172.21.15.16:0/1828756565 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a480d35b0 0x7f7a48159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.407+0000 7f7a4d57e640 1 --2- 172.21.15.16:0/1828756565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a480b5990 0x7f7a480c91b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.407+0000 7f7a4d57e640 1 -- 172.21.15.16:0/1828756565 >> 172.21.15.16:0/1828756565 conn(0x7f7a48072900 msgr2=0x7f7a480b7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.407+0000 7f7a4d57e640 1 -- 172.21.15.16:0/1828756565 shutdown_connections 2024-04-25T10:06:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:25.407+0000 7f7a4d57e640 1 -- 172.21.15.16:0/1828756565 wait complete. 2024-04-25T10:06:25.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parentdir mytag 2024-04-25T10:06:25.542 INFO:teuthology.orchestra.run.smithi016.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-04-25T10:06:25.542 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-04-25T10:06:35.543 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-04-25T10:06:35.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.589+0000 7f1c47906f40 1 Processor -- start 2024-04-25T10:06:35.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.589+0000 7f1c47906f40 1 -- start start 2024-04-25T10:06:35.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.590+0000 7f1c47906f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d6a4e95570 0x55d6a4e92b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.590+0000 7f1c47906f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d6a4f3bd90 0x55d6a4f45c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.590+0000 7f1c47906f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6a4f3c6b0 0x55d6a4f461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.590+0000 7f1c47906f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6a4e967d0 con 0x55d6a4f3c6b0 2024-04-25T10:06:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.590+0000 7f1c47906f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6a4e96530 con 0x55d6a4f3bd90 2024-04-25T10:06:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.590+0000 7f1c47906f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d6a4e96a70 con 0x55d6a4e95570 2024-04-25T10:06:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.590+0000 7f1c45c08640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6a4f3c6b0 0x55d6a4f461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.590+0000 7f1c45c08640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6a4f3c6b0 0x55d6a4f461d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51700/0 (socket says 172.21.15.16:51700) 2024-04-25T10:06:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.590+0000 7f1c45c08640 1 -- 172.21.15.16:0/443367436 learned_addr learned my addr 172.21.15.16:0/443367436 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.590+0000 7f1c45407640 1 --2- 172.21.15.16:0/443367436 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d6a4e95570 0x55d6a4e92b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.590+0000 7f1c44c06640 1 --2- 172.21.15.16:0/443367436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d6a4f3bd90 0x55d6a4f45c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.590+0000 7f1c45c08640 1 -- 172.21.15.16:0/443367436 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d6a4e95570 msgr2=0x55d6a4e92b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.590+0000 7f1c45c08640 1 --2- 172.21.15.16:0/443367436 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d6a4e95570 0x55d6a4e92b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.590+0000 7f1c45c08640 1 -- 172.21.15.16:0/443367436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d6a4f3bd90 msgr2=0x55d6a4f45c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.590+0000 7f1c45c08640 1 --2- 172.21.15.16:0/443367436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d6a4f3bd90 0x55d6a4f45c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.590+0000 7f1c45c08640 1 -- 172.21.15.16:0/443367436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6a4da8de0 con 0x55d6a4f3c6b0 2024-04-25T10:06:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.591+0000 7f1c45c08640 1 --2- 172.21.15.16:0/443367436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6a4f3c6b0 0x55d6a4f461d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000c490 tx=0x7f1c4000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0bf9df1c56b940 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.591+0000 7f1c377fe640 1 -- 172.21.15.16:0/443367436 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c400115f0 con 0x55d6a4f3c6b0 2024-04-25T10:06:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.591+0000 7f1c377fe640 1 -- 172.21.15.16:0/443367436 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c40017070 con 0x55d6a4f3c6b0 2024-04-25T10:06:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.591+0000 7f1c377fe640 1 -- 172.21.15.16:0/443367436 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c400119f0 con 0x55d6a4f3c6b0 2024-04-25T10:06:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.591+0000 7f1c47906f40 1 -- 172.21.15.16:0/443367436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6a4f3c6b0 msgr2=0x55d6a4f461d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.591+0000 7f1c47906f40 1 --2- 172.21.15.16:0/443367436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6a4f3c6b0 0x55d6a4f461d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000c490 tx=0x7f1c4000c950 comp rx=0 tx=0).stop 2024-04-25T10:06:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.591+0000 7f1c47906f40 1 -- 172.21.15.16:0/443367436 shutdown_connections 2024-04-25T10:06:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.591+0000 7f1c47906f40 1 --2- 172.21.15.16:0/443367436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6a4f3c6b0 0x55d6a4f461d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.591+0000 7f1c47906f40 1 --2- 172.21.15.16:0/443367436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d6a4f3bd90 0x55d6a4f45c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.591+0000 7f1c47906f40 1 --2- 172.21.15.16:0/443367436 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d6a4e95570 0x55d6a4e92b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.591+0000 7f1c47906f40 1 -- 172.21.15.16:0/443367436 >> 172.21.15.16:0/443367436 conn(0x55d6a4e923d0 msgr2=0x55d6a4f3fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.591+0000 7f1c47906f40 1 -- 172.21.15.16:0/443367436 shutdown_connections 2024-04-25T10:06:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.592+0000 7f1c47906f40 1 -- 172.21.15.16:0/443367436 wait complete. 2024-04-25T10:06:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.592+0000 7f1c47906f40 1 Processor -- start 2024-04-25T10:06:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.592+0000 7f1c47906f40 1 -- start start 2024-04-25T10:06:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.593+0000 7f1c47906f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d6a4e95570 0x55d6a4f34160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.593+0000 7f1c47906f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6a4f3bd90 0x55d6a4f46a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.593+0000 7f1c47906f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d6a4f3c6b0 0x55d6a4f46f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.593+0000 7f1c47906f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6a4f3e890 con 0x55d6a4f3bd90 2024-04-25T10:06:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.593+0000 7f1c47906f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6a4f40490 con 0x55d6a4e95570 2024-04-25T10:06:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.593+0000 7f1c47906f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d6a4e69160 con 0x55d6a4f3c6b0 2024-04-25T10:06:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.593+0000 7f1c44c06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6a4f3bd90 0x55d6a4f46a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.593+0000 7f1c45407640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d6a4e95570 0x55d6a4f34160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.593+0000 7f1c44c06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6a4f3bd90 0x55d6a4f46a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51706/0 (socket says 172.21.15.16:51706) 2024-04-25T10:06:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.593+0000 7f1c44c06640 1 -- 172.21.15.16:0/3380054342 learned_addr learned my addr 172.21.15.16:0/3380054342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.593+0000 7f1c45c08640 1 --2- 172.21.15.16:0/3380054342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d6a4f3c6b0 0x55d6a4f46f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.594+0000 7f1c44c06640 1 -- 172.21.15.16:0/3380054342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d6a4f3c6b0 msgr2=0x55d6a4f46f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.594+0000 7f1c44c06640 1 --2- 172.21.15.16:0/3380054342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d6a4f3c6b0 0x55d6a4f46f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.594+0000 7f1c44c06640 1 -- 172.21.15.16:0/3380054342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d6a4e95570 msgr2=0x55d6a4f34160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.594+0000 7f1c44c06640 1 --2- 172.21.15.16:0/3380054342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d6a4e95570 0x55d6a4f34160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.594+0000 7f1c44c06640 1 -- 172.21.15.16:0/3380054342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6a4d94c50 con 0x55d6a4f3bd90 2024-04-25T10:06:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.594+0000 7f1c44c06640 1 --2- 172.21.15.16:0/3380054342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6a4f3bd90 0x55d6a4f46a10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c005630 tx=0x7f1c3c0084c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.594+0000 7f1c367fc640 1 -- 172.21.15.16:0/3380054342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c015660 con 0x55d6a4f3bd90 2024-04-25T10:06:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.594+0000 7f1c367fc640 1 -- 172.21.15.16:0/3380054342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3c01a070 con 0x55d6a4f3bd90 2024-04-25T10:06:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.594+0000 7f1c47906f40 1 -- 172.21.15.16:0/3380054342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d6a4f346a0 con 0x55d6a4f3bd90 2024-04-25T10:06:35.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.594+0000 7f1c367fc640 1 -- 172.21.15.16:0/3380054342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c015a00 con 0x55d6a4f3bd90 2024-04-25T10:06:35.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.594+0000 7f1c47906f40 1 -- 172.21.15.16:0/3380054342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d6a4fd59a0 con 0x55d6a4f3bd90 2024-04-25T10:06:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.596+0000 7f1c367fc640 1 -- 172.21.15.16:0/3380054342 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c3c013030 con 0x55d6a4f3bd90 2024-04-25T10:06:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.597+0000 7f1c367fc640 1 --2- 172.21.15.16:0/3380054342 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1c28069eb0 0x7f1c2806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.597+0000 7f1c367fc640 1 -- 172.21.15.16:0/3380054342 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c3c088da0 con 0x55d6a4f3bd90 2024-04-25T10:06:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.597+0000 7f1c45407640 1 --2- 172.21.15.16:0/3380054342 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1c28069eb0 0x7f1c2806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.597+0000 7f1c47906f40 1 --2- 172.21.15.16:0/3380054342 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55d6a4e66490 0x55d6a4f35030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.597+0000 7f1c47906f40 1 -- 172.21.15.16:0/3380054342 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) v9 -- 0x55d6a4f3eaa0 con 0x55d6a4e66490 2024-04-25T10:06:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.597+0000 7f1c45c08640 1 --2- 172.21.15.16:0/3380054342 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55d6a4e66490 0x55d6a4f35030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.598+0000 7f1c45407640 1 --2- 172.21.15.16:0/3380054342 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1c28069eb0 0x7f1c2806c370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55d6a4e65320 tx=0x7f1c38050000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.598+0000 7f1c45c08640 1 --2- 172.21.15.16:0/3380054342 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55d6a4e66490 0x55d6a4f35030 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.598+0000 7f1c45c08640 1 -- 172.21.15.16:0/3380054342 <== osd.7 v2:172.21.15.186:6808/1627307043 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f1c40011890 con 0x55d6a4e66490 2024-04-25T10:06:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.598+0000 7f1c47906f40 1 -- 172.21.15.16:0/3380054342 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55d6a4e66490 msgr2=0x55d6a4f35030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.598+0000 7f1c47906f40 1 --2- 172.21.15.16:0/3380054342 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55d6a4e66490 0x55d6a4f35030 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.598+0000 7f1c47906f40 1 -- 172.21.15.16:0/3380054342 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1c28069eb0 msgr2=0x7f1c2806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.598+0000 7f1c47906f40 1 --2- 172.21.15.16:0/3380054342 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1c28069eb0 0x7f1c2806c370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55d6a4e65320 tx=0x7f1c38050000 comp rx=0 tx=0).stop 2024-04-25T10:06:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.598+0000 7f1c47906f40 1 -- 172.21.15.16:0/3380054342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6a4f3bd90 msgr2=0x55d6a4f46a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.598+0000 7f1c47906f40 1 --2- 172.21.15.16:0/3380054342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6a4f3bd90 0x55d6a4f46a10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c005630 tx=0x7f1c3c0084c0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.598+0000 7f1c45c08640 1 -- 172.21.15.16:0/3380054342 reap_dead start 2024-04-25T10:06:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.598+0000 7f1c47906f40 1 -- 172.21.15.16:0/3380054342 shutdown_connections 2024-04-25T10:06:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.598+0000 7f1c47906f40 1 -- 172.21.15.16:0/3380054342 >> 172.21.15.16:0/3380054342 conn(0x55d6a4e923d0 msgr2=0x55d6a4e92c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.599+0000 7f1c47906f40 1 -- 172.21.15.16:0/3380054342 shutdown_connections 2024-04-25T10:06:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.599+0000 7f1c47906f40 1 -- 172.21.15.16:0/3380054342 wait complete. 2024-04-25T10:06:35.607 INFO:teuthology.orchestra.run.smithi016.stdout:mytag 2024-04-25T10:06:35.608 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-04-25T10:06:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.648+0000 7feefeae4f40 1 Processor -- start 2024-04-25T10:06:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.648+0000 7feefeae4f40 1 -- start start 2024-04-25T10:06:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefeae4f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5608299f7880 0x5608299f3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefeae4f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560829a9ab60 0x5608299f1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefeae4f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560829a9d5e0 0x5608299c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefeae4f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560829a9e860 con 0x560829a9d5e0 2024-04-25T10:06:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefeae4f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56082996edf0 con 0x560829a9ab60 2024-04-25T10:06:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefeae4f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560829a9e9e0 con 0x5608299f7880 2024-04-25T10:06:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefcde6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560829a9d5e0 0x5608299c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefcde6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560829a9d5e0 0x5608299c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51726/0 (socket says 172.21.15.16:51726) 2024-04-25T10:06:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefcde6640 1 -- 172.21.15.16:0/2886621999 learned_addr learned my addr 172.21.15.16:0/2886621999 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefbde4640 1 --2- 172.21.15.16:0/2886621999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560829a9ab60 0x5608299f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefc5e5640 1 --2- 172.21.15.16:0/2886621999 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5608299f7880 0x5608299f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefcde6640 1 -- 172.21.15.16:0/2886621999 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5608299f7880 msgr2=0x5608299f3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefcde6640 1 --2- 172.21.15.16:0/2886621999 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5608299f7880 0x5608299f3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefcde6640 1 -- 172.21.15.16:0/2886621999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560829a9ab60 msgr2=0x5608299f1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefcde6640 1 --2- 172.21.15.16:0/2886621999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560829a9ab60 0x5608299f1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefcde6640 1 -- 172.21.15.16:0/2886621999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608298f4c50 con 0x560829a9d5e0 2024-04-25T10:06:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefcde6640 1 --2- 172.21.15.16:0/2886621999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560829a9d5e0 0x5608299c88e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7feef400a4b0 tx=0x7feef400a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5333be6d8ad3ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefade2640 1 -- 172.21.15.16:0/2886621999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4015070 con 0x560829a9d5e0 2024-04-25T10:06:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefade2640 1 -- 172.21.15.16:0/2886621999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef400e710 con 0x560829a9d5e0 2024-04-25T10:06:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.649+0000 7feefade2640 1 -- 172.21.15.16:0/2886621999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef400eb00 con 0x560829a9d5e0 2024-04-25T10:06:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.650+0000 7feefeae4f40 1 -- 172.21.15.16:0/2886621999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560829a9d5e0 msgr2=0x5608299c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.650+0000 7feefeae4f40 1 --2- 172.21.15.16:0/2886621999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560829a9d5e0 0x5608299c88e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7feef400a4b0 tx=0x7feef400a970 comp rx=0 tx=0).stop 2024-04-25T10:06:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.650+0000 7feefeae4f40 1 -- 172.21.15.16:0/2886621999 shutdown_connections 2024-04-25T10:06:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.650+0000 7feefeae4f40 1 --2- 172.21.15.16:0/2886621999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560829a9d5e0 0x5608299c88e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.650+0000 7feefeae4f40 1 --2- 172.21.15.16:0/2886621999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560829a9ab60 0x5608299f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.650+0000 7feefeae4f40 1 --2- 172.21.15.16:0/2886621999 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5608299f7880 0x5608299f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.650+0000 7feefeae4f40 1 -- 172.21.15.16:0/2886621999 >> 172.21.15.16:0/2886621999 conn(0x5608299f23d0 msgr2=0x560829a99a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.650+0000 7feefeae4f40 1 -- 172.21.15.16:0/2886621999 shutdown_connections 2024-04-25T10:06:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.650+0000 7feefeae4f40 1 -- 172.21.15.16:0/2886621999 wait complete. 2024-04-25T10:06:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.651+0000 7feefeae4f40 1 Processor -- start 2024-04-25T10:06:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.651+0000 7feefeae4f40 1 -- start start 2024-04-25T10:06:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.651+0000 7feefeae4f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5608299f7880 0x560829a09f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.651+0000 7feefeae4f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560829a9ab60 0x5608299c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.651+0000 7feefeae4f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560829a9d5e0 0x560829a8ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.651+0000 7feefeae4f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560829a0e2b0 con 0x560829a9ab60 2024-04-25T10:06:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.651+0000 7feefeae4f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560829a0e550 con 0x5608299f7880 2024-04-25T10:06:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.651+0000 7feefeae4f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560829a991d0 con 0x560829a9d5e0 2024-04-25T10:06:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.652+0000 7feefbde4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560829a9ab60 0x5608299c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.652+0000 7feefbde4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560829a9ab60 0x5608299c9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51740/0 (socket says 172.21.15.16:51740) 2024-04-25T10:06:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.652+0000 7feefbde4640 1 -- 172.21.15.16:0/688115238 learned_addr learned my addr 172.21.15.16:0/688115238 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.652+0000 7feefc5e5640 1 --2- 172.21.15.16:0/688115238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5608299f7880 0x560829a09f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.652+0000 7feefcde6640 1 --2- 172.21.15.16:0/688115238 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560829a9d5e0 0x560829a8ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.652+0000 7feefbde4640 1 -- 172.21.15.16:0/688115238 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560829a9d5e0 msgr2=0x560829a8ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.652+0000 7feefbde4640 1 --2- 172.21.15.16:0/688115238 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560829a9d5e0 0x560829a8ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.652+0000 7feefbde4640 1 -- 172.21.15.16:0/688115238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5608299f7880 msgr2=0x560829a09f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.652+0000 7feefbde4640 1 --2- 172.21.15.16:0/688115238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5608299f7880 0x560829a09f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.652+0000 7feefbde4640 1 -- 172.21.15.16:0/688115238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560829a92050 con 0x560829a9ab60 2024-04-25T10:06:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.652+0000 7feefbde4640 1 --2- 172.21.15.16:0/688115238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560829a9ab60 0x5608299c9930 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7feef0007ef0 tx=0x7feef0006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.652+0000 7feef9de0640 1 -- 172.21.15.16:0/688115238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0018070 con 0x560829a9ab60 2024-04-25T10:06:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.652+0000 7feef9de0640 1 -- 172.21.15.16:0/688115238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef0011700 con 0x560829a9ab60 2024-04-25T10:06:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.652+0000 7feef9de0640 1 -- 172.21.15.16:0/688115238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0011ad0 con 0x560829a9ab60 2024-04-25T10:06:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.652+0000 7feefeae4f40 1 -- 172.21.15.16:0/688115238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560829a92330 con 0x560829a9ab60 2024-04-25T10:06:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.652+0000 7feefeae4f40 1 -- 172.21.15.16:0/688115238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560829b354e0 con 0x560829a9ab60 2024-04-25T10:06:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.654+0000 7feef9de0640 1 -- 172.21.15.16:0/688115238 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feef001b070 con 0x560829a9ab60 2024-04-25T10:06:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.654+0000 7feef9de0640 1 --2- 172.21.15.16:0/688115238 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7feee8069ee0 0x7feee806c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.654+0000 7feef9de0640 1 -- 172.21.15.16:0/688115238 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feef00882e0 con 0x560829a9ab60 2024-04-25T10:06:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.654+0000 7feefc5e5640 1 --2- 172.21.15.16:0/688115238 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7feee8069ee0 0x7feee806c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.654+0000 7feefeae4f40 1 --2- 172.21.15.16:0/688115238 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x560829b378f0 0x560829b35ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.654+0000 7feefeae4f40 1 -- 172.21.15.16:0/688115238 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:1 4.7 4:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) v9 -- 0x560829b36220 con 0x560829b378f0 2024-04-25T10:06:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.655+0000 7feefcde6640 1 --2- 172.21.15.16:0/688115238 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x560829b378f0 0x560829b35ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.655+0000 7feefc5e5640 1 --2- 172.21.15.16:0/688115238 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7feee8069ee0 0x7feee806c3a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feeec057000 tx=0x7feeec052000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.655+0000 7feefcde6640 1 --2- 172.21.15.16:0/688115238 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x560829b378f0 0x560829b35ce0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.655+0000 7feefcde6640 1 -- 172.21.15.16:0/688115238 <== osd.6 v2:172.21.15.186:6816/1123185664 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7feef400e070 con 0x560829b378f0 2024-04-25T10:06:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.655+0000 7feefeae4f40 1 -- 172.21.15.16:0/688115238 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x560829b378f0 msgr2=0x560829b35ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.656+0000 7feefeae4f40 1 --2- 172.21.15.16:0/688115238 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x560829b378f0 0x560829b35ce0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.656+0000 7feefeae4f40 1 -- 172.21.15.16:0/688115238 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7feee8069ee0 msgr2=0x7feee806c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.656+0000 7feefeae4f40 1 --2- 172.21.15.16:0/688115238 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7feee8069ee0 0x7feee806c3a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feeec057000 tx=0x7feeec052000 comp rx=0 tx=0).stop 2024-04-25T10:06:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.656+0000 7feefeae4f40 1 -- 172.21.15.16:0/688115238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560829a9ab60 msgr2=0x5608299c9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.656+0000 7feefeae4f40 1 --2- 172.21.15.16:0/688115238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560829a9ab60 0x5608299c9930 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7feef0007ef0 tx=0x7feef0006750 comp rx=0 tx=0).stop 2024-04-25T10:06:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.656+0000 7feefcde6640 1 -- 172.21.15.16:0/688115238 reap_dead start 2024-04-25T10:06:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.656+0000 7feefeae4f40 1 -- 172.21.15.16:0/688115238 shutdown_connections 2024-04-25T10:06:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.656+0000 7feefeae4f40 1 -- 172.21.15.16:0/688115238 >> 172.21.15.16:0/688115238 conn(0x5608299f23d0 msgr2=0x560829a91240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:35.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.656+0000 7feefeae4f40 1 -- 172.21.15.16:0/688115238 shutdown_connections 2024-04-25T10:06:35.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.656+0000 7feefeae4f40 1 -- 172.21.15.16:0/688115238 wait complete. 2024-04-25T10:06:35.665 INFO:teuthology.orchestra.run.smithi016.stdout:mytag 2024-04-25T10:06:35.666 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-25T10:06:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.704+0000 7f47f2fd4f40 1 Processor -- start 2024-04-25T10:06:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.704+0000 7f47f2fd4f40 1 -- start start 2024-04-25T10:06:35.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.705+0000 7f47f2fd4f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932460b00 0x558932506c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.705+0000 7f47f2fd4f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558932460ee0 0x5589324f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.705+0000 7f47f2fd4f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558932509e50 0x55893245c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.705+0000 7f47f2fd4f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589323d7df0 con 0x558932460b00 2024-04-25T10:06:35.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.705+0000 7f47f2fd4f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589323dbd50 con 0x558932460ee0 2024-04-25T10:06:35.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.705+0000 7f47f2fd4f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x558932506540 con 0x558932509e50 2024-04-25T10:06:35.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.705+0000 7f47f0ad5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932460b00 0x558932506c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.705+0000 7f47f0ad5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932460b00 0x558932506c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51752/0 (socket says 172.21.15.16:51752) 2024-04-25T10:06:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.705+0000 7f47f0ad5640 1 -- 172.21.15.16:0/3411800471 learned_addr learned my addr 172.21.15.16:0/3411800471 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.705+0000 7f47f02d4640 1 --2- 172.21.15.16:0/3411800471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558932460ee0 0x5589324f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.705+0000 7f47f12d6640 1 --2- 172.21.15.16:0/3411800471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558932509e50 0x55893245c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47f0ad5640 1 -- 172.21.15.16:0/3411800471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558932509e50 msgr2=0x55893245c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47f0ad5640 1 --2- 172.21.15.16:0/3411800471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558932509e50 0x55893245c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47f0ad5640 1 -- 172.21.15.16:0/3411800471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558932460ee0 msgr2=0x5589324f1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47f0ad5640 1 --2- 172.21.15.16:0/3411800471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558932460ee0 0x5589324f1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47f0ad5640 1 -- 172.21.15.16:0/3411800471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558932478040 con 0x558932460b00 2024-04-25T10:06:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47f02d4640 1 --2- 172.21.15.16:0/3411800471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558932460ee0 0x5589324f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47f0ad5640 1 --2- 172.21.15.16:0/3411800471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932460b00 0x558932506c10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f47e0057040 tx=0x7f47e0051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3146c12974897d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47ef2d2640 1 -- 172.21.15.16:0/3411800471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0063070 con 0x558932460b00 2024-04-25T10:06:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47ef2d2640 1 -- 172.21.15.16:0/3411800471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e00549c0 con 0x558932460b00 2024-04-25T10:06:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47ef2d2640 1 -- 172.21.15.16:0/3411800471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0059040 con 0x558932460b00 2024-04-25T10:06:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47f2fd4f40 1 -- 172.21.15.16:0/3411800471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932460b00 msgr2=0x558932506c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47f2fd4f40 1 --2- 172.21.15.16:0/3411800471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932460b00 0x558932506c10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f47e0057040 tx=0x7f47e0051000 comp rx=0 tx=0).stop 2024-04-25T10:06:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47f2fd4f40 1 -- 172.21.15.16:0/3411800471 shutdown_connections 2024-04-25T10:06:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47f2fd4f40 1 --2- 172.21.15.16:0/3411800471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558932509e50 0x55893245c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47f2fd4f40 1 --2- 172.21.15.16:0/3411800471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558932460ee0 0x5589324f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47f2fd4f40 1 --2- 172.21.15.16:0/3411800471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932460b00 0x558932506c10 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47f2fd4f40 1 -- 172.21.15.16:0/3411800471 >> 172.21.15.16:0/3411800471 conn(0x55893245b3d0 msgr2=0x558932432b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47f2fd4f40 1 -- 172.21.15.16:0/3411800471 shutdown_connections 2024-04-25T10:06:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.706+0000 7f47f2fd4f40 1 -- 172.21.15.16:0/3411800471 wait complete. 2024-04-25T10:06:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.707+0000 7f47f2fd4f40 1 Processor -- start 2024-04-25T10:06:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.707+0000 7f47f2fd4f40 1 -- start start 2024-04-25T10:06:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47f2fd4f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932460b00 0x5589324e8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47f2fd4f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558932460ee0 0x5589324f9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47f2fd4f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558932509e50 0x55893259bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47f2fd4f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589324782e0 con 0x558932460b00 2024-04-25T10:06:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47f2fd4f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589325060d0 con 0x558932460ee0 2024-04-25T10:06:35.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47f2fd4f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589325099b0 con 0x558932509e50 2024-04-25T10:06:35.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47f0ad5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932460b00 0x5589324e8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47f0ad5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932460b00 0x5589324e8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51766/0 (socket says 172.21.15.16:51766) 2024-04-25T10:06:35.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47f0ad5640 1 -- 172.21.15.16:0/1732669505 learned_addr learned my addr 172.21.15.16:0/1732669505 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:35.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47f02d4640 1 --2- 172.21.15.16:0/1732669505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558932460ee0 0x5589324f9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47f0ad5640 1 -- 172.21.15.16:0/1732669505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558932509e50 msgr2=0x55893259bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47f0ad5640 1 --2- 172.21.15.16:0/1732669505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558932509e50 0x55893259bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47f0ad5640 1 -- 172.21.15.16:0/1732669505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558932460ee0 msgr2=0x5589324f9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47f0ad5640 1 --2- 172.21.15.16:0/1732669505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558932460ee0 0x5589324f9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47f0ad5640 1 -- 172.21.15.16:0/1732669505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589324eb030 con 0x558932460b00 2024-04-25T10:06:35.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47f02d4640 1 --2- 172.21.15.16:0/1732669505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558932460ee0 0x5589324f9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:35.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47f0ad5640 1 --2- 172.21.15.16:0/1732669505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932460b00 0x5589324e8dc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f47e00258a0 tx=0x7f47e0003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47ee2d0640 1 -- 172.21.15.16:0/1732669505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0063040 con 0x558932460b00 2024-04-25T10:06:35.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.708+0000 7f47ee2d0640 1 -- 172.21.15.16:0/1732669505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e006d070 con 0x558932460b00 2024-04-25T10:06:35.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.709+0000 7f47f2fd4f40 1 -- 172.21.15.16:0/1732669505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5589324eb310 con 0x558932460b00 2024-04-25T10:06:35.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.709+0000 7f47ee2d0640 1 -- 172.21.15.16:0/1732669505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e005e8c0 con 0x558932460b00 2024-04-25T10:06:35.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.709+0000 7f47f2fd4f40 1 -- 172.21.15.16:0/1732669505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589324e9300 con 0x558932460b00 2024-04-25T10:06:35.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.710+0000 7f47ee2d0640 1 -- 172.21.15.16:0/1732669505 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47e005c070 con 0x558932460b00 2024-04-25T10:06:35.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.710+0000 7f47ee2d0640 1 --2- 172.21.15.16:0/1732669505 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f47d0069ee0 0x7f47d006c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.710+0000 7f47ee2d0640 1 -- 172.21.15.16:0/1732669505 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47e00cd7b0 con 0x558932460b00 2024-04-25T10:06:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.710+0000 7f47f02d4640 1 --2- 172.21.15.16:0/1732669505 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f47d0069ee0 0x7f47d006c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.710+0000 7f47f2fd4f40 1 --2- 172.21.15.16:0/1732669505 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x5589324fa280 0x558932433090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.711+0000 7f47f2fd4f40 1 -- 172.21.15.16:0/1732669505 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:1 5.36 5:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) v9 -- 0x5589324eabe0 con 0x5589324fa280 2024-04-25T10:06:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.711+0000 7f47f12d6640 1 --2- 172.21.15.16:0/1732669505 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x5589324fa280 0x558932433090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.711+0000 7f47f02d4640 1 --2- 172.21.15.16:0/1732669505 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f47d0069ee0 0x7f47d006c3a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f47d8000c00 tx=0x7f47d8002630 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.711+0000 7f47f12d6640 1 --2- 172.21.15.16:0/1732669505 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x5589324fa280 0x558932433090 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.711+0000 7f47f12d6640 1 -- 172.21.15.16:0/1732669505 <== osd.4 v2:172.21.15.186:6824/3091472889 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f47e800f450 con 0x5589324fa280 2024-04-25T10:06:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.711+0000 7f47f2fd4f40 1 -- 172.21.15.16:0/1732669505 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x5589324fa280 msgr2=0x558932433090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.711+0000 7f47f2fd4f40 1 --2- 172.21.15.16:0/1732669505 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x5589324fa280 0x558932433090 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.712+0000 7f47f2fd4f40 1 -- 172.21.15.16:0/1732669505 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f47d0069ee0 msgr2=0x7f47d006c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.712+0000 7f47f2fd4f40 1 --2- 172.21.15.16:0/1732669505 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f47d0069ee0 0x7f47d006c3a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f47d8000c00 tx=0x7f47d8002630 comp rx=0 tx=0).stop 2024-04-25T10:06:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.712+0000 7f47f2fd4f40 1 -- 172.21.15.16:0/1732669505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932460b00 msgr2=0x5589324e8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.712+0000 7f47f2fd4f40 1 --2- 172.21.15.16:0/1732669505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558932460b00 0x5589324e8dc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f47e00258a0 tx=0x7f47e0003330 comp rx=0 tx=0).stop 2024-04-25T10:06:35.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.712+0000 7f47f12d6640 1 -- 172.21.15.16:0/1732669505 reap_dead start 2024-04-25T10:06:35.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.712+0000 7f47f2fd4f40 1 -- 172.21.15.16:0/1732669505 shutdown_connections 2024-04-25T10:06:35.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.712+0000 7f47f2fd4f40 1 -- 172.21.15.16:0/1732669505 >> 172.21.15.16:0/1732669505 conn(0x55893245b3d0 msgr2=0x5589325040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:35.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.712+0000 7f47f2fd4f40 1 -- 172.21.15.16:0/1732669505 shutdown_connections 2024-04-25T10:06:35.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.712+0000 7f47f2fd4f40 1 -- 172.21.15.16:0/1732669505 wait complete. 2024-04-25T10:06:35.723 INFO:teuthology.orchestra.run.smithi016.stdout:mytag 2024-04-25T10:06:35.723 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-25T10:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.761+0000 7f29e351ff40 1 Processor -- start 2024-04-25T10:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.761+0000 7f29e351ff40 1 -- start start 2024-04-25T10:06:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29e351ff40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29cc003630 0x559b109ab600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29e351ff40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559b109a62b0 0x559b108d6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29e351ff40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b109a66f0 0x559b108d7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29e351ff40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b10904ea0 con 0x559b109a66f0 2024-04-25T10:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29e351ff40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1087bdf0 con 0x7f29cc003630 2024-04-25T10:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29e351ff40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b10905020 con 0x559b109a62b0 2024-04-25T10:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29e0d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b109a66f0 0x559b108d7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29e0d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b109a66f0 0x559b108d7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51782/0 (socket says 172.21.15.16:51782) 2024-04-25T10:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29e0d9b640 1 -- 172.21.15.16:0/2800143362 learned_addr learned my addr 172.21.15.16:0/2800143362 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29e059a640 1 --2- 172.21.15.16:0/2800143362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29cc003630 0x559b109ab600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29dfd99640 1 --2- 172.21.15.16:0/2800143362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559b109a62b0 0x559b108d6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29e0d9b640 1 -- 172.21.15.16:0/2800143362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559b109a62b0 msgr2=0x559b108d6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29e0d9b640 1 --2- 172.21.15.16:0/2800143362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559b109a62b0 0x559b108d6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29e0d9b640 1 -- 172.21.15.16:0/2800143362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29cc003630 msgr2=0x559b109ab600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29e0d9b640 1 --2- 172.21.15.16:0/2800143362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29cc003630 0x559b109ab600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29e0d9b640 1 -- 172.21.15.16:0/2800143362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b10801c50 con 0x559b109a66f0 2024-04-25T10:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29dfd99640 1 --2- 172.21.15.16:0/2800143362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559b109a62b0 0x559b108d6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29e0d9b640 1 --2- 172.21.15.16:0/2800143362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b109a66f0 0x559b108d7050 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f29d00260f0 tx=0x7f29d0006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903308564c070bf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29ded97640 1 -- 172.21.15.16:0/2800143362 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d0006070 con 0x559b109a66f0 2024-04-25T10:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29ded97640 1 -- 172.21.15.16:0/2800143362 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d0006210 con 0x559b109a66f0 2024-04-25T10:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.762+0000 7f29ded97640 1 -- 172.21.15.16:0/2800143362 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d00063c0 con 0x559b109a66f0 2024-04-25T10:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.763+0000 7f29e351ff40 1 -- 172.21.15.16:0/2800143362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b109a66f0 msgr2=0x559b108d7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.763+0000 7f29e351ff40 1 --2- 172.21.15.16:0/2800143362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b109a66f0 0x559b108d7050 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f29d00260f0 tx=0x7f29d0006000 comp rx=0 tx=0).stop 2024-04-25T10:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.763+0000 7f29e351ff40 1 -- 172.21.15.16:0/2800143362 shutdown_connections 2024-04-25T10:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.763+0000 7f29e351ff40 1 --2- 172.21.15.16:0/2800143362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b109a66f0 0x559b108d7050 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.763+0000 7f29e351ff40 1 --2- 172.21.15.16:0/2800143362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559b109a62b0 0x559b108d6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.763+0000 7f29e351ff40 1 --2- 172.21.15.16:0/2800143362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29cc003630 0x559b109ab600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.763+0000 7f29e351ff40 1 -- 172.21.15.16:0/2800143362 >> 172.21.15.16:0/2800143362 conn(0x559b108ff3d0 msgr2=0x559b10900070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.763+0000 7f29e351ff40 1 -- 172.21.15.16:0/2800143362 shutdown_connections 2024-04-25T10:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.763+0000 7f29e351ff40 1 -- 172.21.15.16:0/2800143362 wait complete. 2024-04-25T10:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.763+0000 7f29e351ff40 1 Processor -- start 2024-04-25T10:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.763+0000 7f29e351ff40 1 -- start start 2024-04-25T10:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.764+0000 7f29e351ff40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559b109a62b0 0x559b109a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.764+0000 7f29e351ff40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b109a66f0 0x559b1091a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.764+0000 7f29e351ff40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29cc003630 0x559b1099e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.764+0000 7f29e351ff40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b109a6e10 con 0x7f29cc003630 2024-04-25T10:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.764+0000 7f29e351ff40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b109a6bc0 con 0x559b109a66f0 2024-04-25T10:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.764+0000 7f29e351ff40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b108d7590 con 0x559b109a62b0 2024-04-25T10:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.764+0000 7f29e0d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29cc003630 0x559b1099e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.764+0000 7f29e0d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29cc003630 0x559b1099e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51784/0 (socket says 172.21.15.16:51784) 2024-04-25T10:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.764+0000 7f29e0d9b640 1 -- 172.21.15.16:0/2013388022 learned_addr learned my addr 172.21.15.16:0/2013388022 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.764+0000 7f29e059a640 1 --2- 172.21.15.16:0/2013388022 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559b109a62b0 0x559b109a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.764+0000 7f29dfd99640 1 --2- 172.21.15.16:0/2013388022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b109a66f0 0x559b1091a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.764+0000 7f29e0d9b640 1 -- 172.21.15.16:0/2013388022 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559b109a62b0 msgr2=0x559b109a25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.764+0000 7f29e0d9b640 1 --2- 172.21.15.16:0/2013388022 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559b109a62b0 0x559b109a25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.764+0000 7f29e0d9b640 1 -- 172.21.15.16:0/2013388022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b109a66f0 msgr2=0x559b1091a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.764+0000 7f29e0d9b640 1 --2- 172.21.15.16:0/2013388022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b109a66f0 0x559b1091a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.764+0000 7f29e0d9b640 1 -- 172.21.15.16:0/2013388022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b109a2330 con 0x7f29cc003630 2024-04-25T10:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.764+0000 7f29e059a640 1 --2- 172.21.15.16:0/2013388022 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559b109a62b0 0x559b109a25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.765+0000 7f29e0d9b640 1 --2- 172.21.15.16:0/2013388022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29cc003630 0x559b1099e360 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f29d00260c0 tx=0x559b109ab590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.765+0000 7f29ddd95640 1 -- 172.21.15.16:0/2013388022 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d00063c0 con 0x7f29cc003630 2024-04-25T10:06:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.765+0000 7f29ddd95640 1 -- 172.21.15.16:0/2013388022 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d006c070 con 0x7f29cc003630 2024-04-25T10:06:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.765+0000 7f29e351ff40 1 -- 172.21.15.16:0/2013388022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559b109195a0 con 0x7f29cc003630 2024-04-25T10:06:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.765+0000 7f29ddd95640 1 -- 172.21.15.16:0/2013388022 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d00566b0 con 0x7f29cc003630 2024-04-25T10:06:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.765+0000 7f29e351ff40 1 -- 172.21.15.16:0/2013388022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b10919b60 con 0x7f29cc003630 2024-04-25T10:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.766+0000 7f29ddd95640 1 -- 172.21.15.16:0/2013388022 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29d0056850 con 0x7f29cc003630 2024-04-25T10:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.767+0000 7f29ddd95640 1 --2- 172.21.15.16:0/2013388022 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f29c0069eb0 0x7f29c006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.767+0000 7f29ddd95640 1 -- 172.21.15.16:0/2013388022 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29d00cd730 con 0x7f29cc003630 2024-04-25T10:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.767+0000 7f29e059a640 1 --2- 172.21.15.16:0/2013388022 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f29c0069eb0 0x7f29c006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.767+0000 7f29e351ff40 1 --2- 172.21.15.16:0/2013388022 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x559b109acc90 0x559b10a34450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.767+0000 7f29e351ff40 1 -- 172.21.15.16:0/2013388022 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:1 5.19 5:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) v9 -- 0x559b10a34990 con 0x559b109acc90 2024-04-25T10:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.767+0000 7f29dfd99640 1 --2- 172.21.15.16:0/2013388022 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x559b109acc90 0x559b10a34450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.767+0000 7f29e059a640 1 --2- 172.21.15.16:0/2013388022 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f29c0069eb0 0x7f29c006c370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f29d80098e0 tx=0x7f29d800c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.768+0000 7f29dfd99640 1 --2- 172.21.15.16:0/2013388022 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x559b109acc90 0x559b10a34450 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.768+0000 7f29dfd99640 1 -- 172.21.15.16:0/2013388022 <== osd.4 v2:172.21.15.186:6824/3091472889 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f29c8009a20 con 0x559b109acc90 2024-04-25T10:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.768+0000 7f29e351ff40 1 -- 172.21.15.16:0/2013388022 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x559b109acc90 msgr2=0x559b10a34450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.768+0000 7f29e351ff40 1 --2- 172.21.15.16:0/2013388022 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x559b109acc90 0x559b10a34450 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.768+0000 7f29e351ff40 1 -- 172.21.15.16:0/2013388022 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f29c0069eb0 msgr2=0x7f29c006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.768+0000 7f29e351ff40 1 --2- 172.21.15.16:0/2013388022 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f29c0069eb0 0x7f29c006c370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f29d80098e0 tx=0x7f29d800c040 comp rx=0 tx=0).stop 2024-04-25T10:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.768+0000 7f29e351ff40 1 -- 172.21.15.16:0/2013388022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29cc003630 msgr2=0x559b1099e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.768+0000 7f29e351ff40 1 --2- 172.21.15.16:0/2013388022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29cc003630 0x559b1099e360 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f29d00260c0 tx=0x559b109ab590 comp rx=0 tx=0).stop 2024-04-25T10:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.768+0000 7f29e0d9b640 1 -- 172.21.15.16:0/2013388022 reap_dead start 2024-04-25T10:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.768+0000 7f29e351ff40 1 -- 172.21.15.16:0/2013388022 shutdown_connections 2024-04-25T10:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.768+0000 7f29e351ff40 1 -- 172.21.15.16:0/2013388022 >> 172.21.15.16:0/2013388022 conn(0x559b108ff3d0 msgr2=0x559b109a2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.768+0000 7f29e351ff40 1 -- 172.21.15.16:0/2013388022 shutdown_connections 2024-04-25T10:06:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.768+0000 7f29e351ff40 1 -- 172.21.15.16:0/2013388022 wait complete. 2024-04-25T10:06:35.779 INFO:teuthology.orchestra.run.smithi016.stdout:mytag 2024-04-25T10:06:35.779 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-25T10:06:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.857+0000 7fd699b25f40 1 Processor -- start 2024-04-25T10:06:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.857+0000 7fd699b25f40 1 -- start start 2024-04-25T10:06:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.857+0000 7fd699b25f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ee3f14880 0x558ee3fb3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.857+0000 7fd699b25f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ee3fb7b60 0x558ee3fb4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.857+0000 7fd699b25f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558ee3fba5e0 0x558ee3fb6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.857+0000 7fd699b25f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee3e8bdf0 con 0x558ee3f14880 2024-04-25T10:06:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.857+0000 7fd699b25f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee3f151d0 con 0x558ee3fb7b60 2024-04-25T10:06:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.857+0000 7fd699b25f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ee3ee66c0 con 0x558ee3fba5e0 2024-04-25T10:06:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.857+0000 7fd696d0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ee3f14880 0x558ee3fb3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.857+0000 7fd696d0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ee3f14880 0x558ee3fb3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51792/0 (socket says 172.21.15.16:51792) 2024-04-25T10:06:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.858+0000 7fd696d0d640 1 -- 172.21.15.16:0/4231910905 learned_addr learned my addr 172.21.15.16:0/4231910905 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.858+0000 7fd69650c640 1 --2- 172.21.15.16:0/4231910905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ee3fb7b60 0x558ee3fb4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.858+0000 7fd69750e640 1 --2- 172.21.15.16:0/4231910905 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558ee3fba5e0 0x558ee3fb6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.858+0000 7fd696d0d640 1 -- 172.21.15.16:0/4231910905 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558ee3fba5e0 msgr2=0x558ee3fb6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.858+0000 7fd696d0d640 1 --2- 172.21.15.16:0/4231910905 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558ee3fba5e0 0x558ee3fb6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.858+0000 7fd696d0d640 1 -- 172.21.15.16:0/4231910905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ee3fb7b60 msgr2=0x558ee3fb4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.858+0000 7fd696d0d640 1 --2- 172.21.15.16:0/4231910905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ee3fb7b60 0x558ee3fb4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.858+0000 7fd696d0d640 1 -- 172.21.15.16:0/4231910905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ee3e11c50 con 0x558ee3f14880 2024-04-25T10:06:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.858+0000 7fd69750e640 1 --2- 172.21.15.16:0/4231910905 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558ee3fba5e0 0x558ee3fb6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.858+0000 7fd696d0d640 1 --2- 172.21.15.16:0/4231910905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ee3f14880 0x558ee3fb3c30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd6880017e0 tx=0x7fd688001ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f421f3b1f40fd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.858+0000 7fd69550a640 1 -- 172.21.15.16:0/4231910905 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688003650 con 0x558ee3f14880 2024-04-25T10:06:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.858+0000 7fd69550a640 1 -- 172.21.15.16:0/4231910905 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6880037f0 con 0x558ee3f14880 2024-04-25T10:06:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.858+0000 7fd69550a640 1 -- 172.21.15.16:0/4231910905 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688003bc0 con 0x558ee3f14880 2024-04-25T10:06:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.859+0000 7fd699b25f40 1 -- 172.21.15.16:0/4231910905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ee3f14880 msgr2=0x558ee3fb3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.859+0000 7fd699b25f40 1 --2- 172.21.15.16:0/4231910905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ee3f14880 0x558ee3fb3c30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd6880017e0 tx=0x7fd688001ca0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.859+0000 7fd699b25f40 1 -- 172.21.15.16:0/4231910905 shutdown_connections 2024-04-25T10:06:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.859+0000 7fd699b25f40 1 --2- 172.21.15.16:0/4231910905 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558ee3fba5e0 0x558ee3fb6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.859+0000 7fd699b25f40 1 --2- 172.21.15.16:0/4231910905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ee3fb7b60 0x558ee3fb4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.859+0000 7fd699b25f40 1 --2- 172.21.15.16:0/4231910905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ee3f14880 0x558ee3fb3c30 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.859+0000 7fd699b25f40 1 -- 172.21.15.16:0/4231910905 >> 172.21.15.16:0/4231910905 conn(0x558ee3f0f3d0 msgr2=0x558ee3ee2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.859+0000 7fd699b25f40 1 -- 172.21.15.16:0/4231910905 shutdown_connections 2024-04-25T10:06:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.859+0000 7fd699b25f40 1 -- 172.21.15.16:0/4231910905 wait complete. 2024-04-25T10:06:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.859+0000 7fd699b25f40 1 Processor -- start 2024-04-25T10:06:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.859+0000 7fd699b25f40 1 -- start start 2024-04-25T10:06:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.860+0000 7fd699b25f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ee3f14880 0x558ee3ee5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.860+0000 7fd699b25f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558ee3fb7b60 0x558ee3fa7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.860+0000 7fd699b25f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ee3fba5e0 0x558ee3fb4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.860+0000 7fd699b25f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee3fb2c40 con 0x558ee3fba5e0 2024-04-25T10:06:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.860+0000 7fd699b25f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee3ee7f00 con 0x558ee3f14880 2024-04-25T10:06:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.860+0000 7fd699b25f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ee3ee6420 con 0x558ee3fb7b60 2024-04-25T10:06:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.860+0000 7fd69750e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ee3fba5e0 0x558ee3fb4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.860+0000 7fd69750e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ee3fba5e0 0x558ee3fb4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51798/0 (socket says 172.21.15.16:51798) 2024-04-25T10:06:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.860+0000 7fd69750e640 1 -- 172.21.15.16:0/1682584568 learned_addr learned my addr 172.21.15.16:0/1682584568 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.860+0000 7fd696d0d640 1 --2- 172.21.15.16:0/1682584568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ee3f14880 0x558ee3ee5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.860+0000 7fd69650c640 1 --2- 172.21.15.16:0/1682584568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558ee3fb7b60 0x558ee3fa7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.860+0000 7fd69750e640 1 -- 172.21.15.16:0/1682584568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558ee3fb7b60 msgr2=0x558ee3fa7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.860+0000 7fd69750e640 1 --2- 172.21.15.16:0/1682584568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558ee3fb7b60 0x558ee3fa7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.860+0000 7fd69750e640 1 -- 172.21.15.16:0/1682584568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ee3f14880 msgr2=0x558ee3ee5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.860+0000 7fd69750e640 1 --2- 172.21.15.16:0/1682584568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558ee3f14880 0x558ee3ee5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.860+0000 7fd69750e640 1 -- 172.21.15.16:0/1682584568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ee3f2b050 con 0x558ee3fba5e0 2024-04-25T10:06:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.861+0000 7fd69750e640 1 --2- 172.21.15.16:0/1682584568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ee3fba5e0 0x558ee3fb4610 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd69000bea0 tx=0x7fd69000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.861+0000 7fd67ffff640 1 -- 172.21.15.16:0/1682584568 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690007bb0 con 0x558ee3fba5e0 2024-04-25T10:06:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.861+0000 7fd67ffff640 1 -- 172.21.15.16:0/1682584568 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd690007d50 con 0x558ee3fba5e0 2024-04-25T10:06:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.861+0000 7fd67ffff640 1 -- 172.21.15.16:0/1682584568 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690015640 con 0x558ee3fba5e0 2024-04-25T10:06:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.861+0000 7fd699b25f40 1 -- 172.21.15.16:0/1682584568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558ee3fae790 con 0x558ee3fba5e0 2024-04-25T10:06:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.861+0000 7fd699b25f40 1 -- 172.21.15.16:0/1682584568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ee3f2ad00 con 0x558ee3fba5e0 2024-04-25T10:06:35.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:06:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.862+0000 7fd67ffff640 1 -- 172.21.15.16:0/1682584568 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd690009410 con 0x558ee3fba5e0 2024-04-25T10:06:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.863+0000 7fd67ffff640 1 --2- 172.21.15.16:0/1682584568 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd684069eb0 0x7fd68406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.863+0000 7fd67ffff640 1 -- 172.21.15.16:0/1682584568 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6900883e0 con 0x558ee3fba5e0 2024-04-25T10:06:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.863+0000 7fd696d0d640 1 --2- 172.21.15.16:0/1682584568 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd684069eb0 0x7fd68406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.863+0000 7fd699b25f40 1 --2- 172.21.15.16:0/1682584568 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x558ee3fb03c0 0x558ee3fb5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.863+0000 7fd699b25f40 1 -- 172.21.15.16:0/1682584568 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:1 5.12 5:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) v9 -- 0x558ee3fb63f0 con 0x558ee3fb03c0 2024-04-25T10:06:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.863+0000 7fd69650c640 1 --2- 172.21.15.16:0/1682584568 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x558ee3fb03c0 0x558ee3fb5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.863+0000 7fd696d0d640 1 --2- 172.21.15.16:0/1682584568 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd684069eb0 0x7fd68406c370 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd688000f80 tx=0x7fd688004000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.864+0000 7fd69650c640 1 --2- 172.21.15.16:0/1682584568 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x558ee3fb03c0 0x558ee3fb5eb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.864+0000 7fd69650c640 1 -- 172.21.15.16:0/1682584568 <== osd.6 v2:172.21.15.186:6816/1123185664 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd68c05c040 con 0x558ee3fb03c0 2024-04-25T10:06:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-04-25T10:06:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.864+0000 7fd699b25f40 1 -- 172.21.15.16:0/1682584568 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x558ee3fb03c0 msgr2=0x558ee3fb5eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.864+0000 7fd699b25f40 1 --2- 172.21.15.16:0/1682584568 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x558ee3fb03c0 0x558ee3fb5eb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.864+0000 7fd699b25f40 1 -- 172.21.15.16:0/1682584568 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd684069eb0 msgr2=0x7fd68406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.864+0000 7fd699b25f40 1 --2- 172.21.15.16:0/1682584568 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd684069eb0 0x7fd68406c370 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd688000f80 tx=0x7fd688004000 comp rx=0 tx=0).stop 2024-04-25T10:06:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.864+0000 7fd699b25f40 1 -- 172.21.15.16:0/1682584568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ee3fba5e0 msgr2=0x558ee3fb4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.864+0000 7fd699b25f40 1 --2- 172.21.15.16:0/1682584568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ee3fba5e0 0x558ee3fb4610 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd69000bea0 tx=0x7fd69000bf90 comp rx=0 tx=0).stop 2024-04-25T10:06:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.864+0000 7fd69750e640 1 -- 172.21.15.16:0/1682584568 reap_dead start 2024-04-25T10:06:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.864+0000 7fd699b25f40 1 -- 172.21.15.16:0/1682584568 shutdown_connections 2024-04-25T10:06:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.864+0000 7fd699b25f40 1 -- 172.21.15.16:0/1682584568 >> 172.21.15.16:0/1682584568 conn(0x558ee3f0f3d0 msgr2=0x558ee3ee2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.864+0000 7fd699b25f40 1 -- 172.21.15.16:0/1682584568 shutdown_connections 2024-04-25T10:06:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:35.864+0000 7fd699b25f40 1 -- 172.21.15.16:0/1682584568 wait complete. 2024-04-25T10:06:35.876 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:06:35.876 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:35.963 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:36.012 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:06:36.013 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:06:36.013 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:36.028 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:36.028 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:06:36.028 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:36.046 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:36.046 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:06:36.046 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:36.083 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:36.083 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:06:36.084 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:36.103 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:36.103 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.218+0000 7f965bb34640 1 Processor -- start 2024-04-25T10:06:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.218+0000 7f965bb34640 1 -- start start 2024-04-25T10:06:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.219+0000 7f965bb34640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f96540d35b0 0x7f96540b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.219+0000 7f965bb34640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96540b5990 0x7f96540b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.219+0000 7f965bb34640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f965415e0c0 con 0x7f965415dce0 2024-04-25T10:06:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.219+0000 7f965bb34640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96540d2510 con 0x7f96540d35b0 2024-04-25T10:06:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.219+0000 7f965bb34640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96540d27f0 con 0x7f96540b5990 2024-04-25T10:06:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.219+0000 7f96590cf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f96540d35b0 0x7f96540b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.219+0000 7f96598d0640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f965415dce0 0x7f96540d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49116/0 (socket says 172.21.15.16:49116) 2024-04-25T10:06:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.219+0000 7f96598d0640 1 -- 172.21.15.16:0/470921080 learned_addr learned my addr 172.21.15.16:0/470921080 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.219+0000 7f965a0d1640 1 --2- 172.21.15.16:0/470921080 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96540b5990 0x7f96540b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.219+0000 7f96590cf640 1 -- 172.21.15.16:0/470921080 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96540b5990 msgr2=0x7f96540b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.219+0000 7f96590cf640 1 --2- 172.21.15.16:0/470921080 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96540b5990 0x7f96540b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.219+0000 7f96590cf640 1 -- 172.21.15.16:0/470921080 >> v1:172.21.15.16:6789/0 conn(0x7f965415dce0 legacy=0x7f96540d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.219+0000 7f96588ce640 1 -- 172.21.15.16:0/470921080 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 562777260 0 0) 0x7f965415e0c0 con 0x7f965415dce0 2024-04-25T10:06:36.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.219+0000 7f96590cf640 1 -- 172.21.15.16:0/470921080 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96540d2ad0 con 0x7f96540d35b0 2024-04-25T10:06:36.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.220+0000 7f96590cf640 1 --2- 172.21.15.16:0/470921080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96540d35b0 0x7f96540b5450 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f96440097a0 tx=0x7f964402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb7c920cf0ed448 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.220+0000 7f96588ce640 1 -- 172.21.15.16:0/470921080 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644037070 con 0x7f96540d35b0 2024-04-25T10:06:36.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.220+0000 7f96588ce640 1 -- 172.21.15.16:0/470921080 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9644032440 con 0x7f96540d35b0 2024-04-25T10:06:36.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.220+0000 7f96588ce640 1 -- 172.21.15.16:0/470921080 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644032810 con 0x7f96540d35b0 2024-04-25T10:06:36.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.221+0000 7f965bb34640 1 -- 172.21.15.16:0/470921080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96540d35b0 msgr2=0x7f96540b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.221+0000 7f965bb34640 1 --2- 172.21.15.16:0/470921080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96540d35b0 0x7f96540b5450 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f96440097a0 tx=0x7f964402a180 comp rx=0 tx=0).stop 2024-04-25T10:06:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.221+0000 7f965bb34640 1 -- 172.21.15.16:0/470921080 shutdown_connections 2024-04-25T10:06:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.221+0000 7f965bb34640 1 --2- 172.21.15.16:0/470921080 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96540b5990 0x7f96540b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.221+0000 7f965bb34640 1 --2- 172.21.15.16:0/470921080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96540d35b0 0x7f96540b5450 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.221+0000 7f965bb34640 1 -- 172.21.15.16:0/470921080 >> 172.21.15.16:0/470921080 conn(0x7f9654072900 msgr2=0x7f96540d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.221+0000 7f965bb34640 1 -- 172.21.15.16:0/470921080 shutdown_connections 2024-04-25T10:06:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.221+0000 7f965bb34640 1 -- 172.21.15.16:0/470921080 wait complete. 2024-04-25T10:06:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.222+0000 7f965bb34640 1 Processor -- start 2024-04-25T10:06:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.222+0000 7f965bb34640 1 -- start start 2024-04-25T10:06:36.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.223+0000 7f965bb34640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96540b5990 0x7f96540ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.223+0000 7f965bb34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96540d35b0 0x7f96540ac950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.223+0000 7f965bb34640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f965415dce0 0x7f96540af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.223+0000 7f965bb34640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96540d2160 con 0x7f96540d35b0 2024-04-25T10:06:36.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.223+0000 7f965bb34640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96540d3090 con 0x7f965415dce0 2024-04-25T10:06:36.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.223+0000 7f965bb34640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96540ab980 con 0x7f96540b5990 2024-04-25T10:06:36.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.223+0000 7f96590cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96540d35b0 0x7f96540ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.223+0000 7f96590cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96540d35b0 0x7f96540ac950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51812/0 (socket says 172.21.15.16:51812) 2024-04-25T10:06:36.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.223+0000 7f96590cf640 1 -- 172.21.15.16:0/150698537 learned_addr learned my addr 172.21.15.16:0/150698537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:36.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.223+0000 7f96598d0640 1 --2- 172.21.15.16:0/150698537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96540b5990 0x7f96540ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.223+0000 7f96590cf640 1 -- 172.21.15.16:0/150698537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96540b5990 msgr2=0x7f96540ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.223+0000 7f96590cf640 1 --2- 172.21.15.16:0/150698537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96540b5990 0x7f96540ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.223+0000 7f96590cf640 1 -- 172.21.15.16:0/150698537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f965415dce0 msgr2=0x7f96540af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.223+0000 7f96590cf640 1 --2- 172.21.15.16:0/150698537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f965415dce0 0x7f96540af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.223+0000 7f96590cf640 1 -- 172.21.15.16:0/150698537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96540abe90 con 0x7f96540d35b0 2024-04-25T10:06:36.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.223+0000 7f96598d0640 1 --2- 172.21.15.16:0/150698537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96540b5990 0x7f96540ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:36.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.223+0000 7f96590cf640 1 --2- 172.21.15.16:0/150698537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96540d35b0 0x7f96540ac950 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9644035040 tx=0x7f96440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.223+0000 7f964a7fc640 1 -- 172.21.15.16:0/150698537 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644037040 con 0x7f96540d35b0 2024-04-25T10:06:36.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.224+0000 7f964a7fc640 1 -- 172.21.15.16:0/150698537 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9644030770 con 0x7f96540d35b0 2024-04-25T10:06:36.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.224+0000 7f965bb34640 1 -- 172.21.15.16:0/150698537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96540ac170 con 0x7f96540d35b0 2024-04-25T10:06:36.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.224+0000 7f964a7fc640 1 -- 172.21.15.16:0/150698537 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644030b80 con 0x7f96540d35b0 2024-04-25T10:06:36.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.224+0000 7f965bb34640 1 -- 172.21.15.16:0/150698537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96540adb00 con 0x7f96540d35b0 2024-04-25T10:06:36.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.224+0000 7f965bb34640 1 -- 172.21.15.16:0/150698537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96540abc90 con 0x7f96540d35b0 2024-04-25T10:06:36.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.225+0000 7f964a7fc640 1 -- 172.21.15.16:0/150698537 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9644044020 con 0x7f96540d35b0 2024-04-25T10:06:36.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.225+0000 7f964a7fc640 1 --2- 172.21.15.16:0/150698537 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f963806b4d0 0x7f963806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.225+0000 7f964a7fc640 1 -- 172.21.15.16:0/150698537 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96440aa770 con 0x7f96540d35b0 2024-04-25T10:06:36.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.226+0000 7f96598d0640 1 --2- 172.21.15.16:0/150698537 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f963806b4d0 0x7f963806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.226+0000 7f96598d0640 1 --2- 172.21.15.16:0/150698537 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f963806b4d0 0x7f963806d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9650000f60 tx=0x7f9650009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.230+0000 7f964a7fc640 1 -- 172.21.15.16:0/150698537 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9644032e20 con 0x7f96540d35b0 2024-04-25T10:06:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.445+0000 7f965bb34640 1 -- 172.21.15.16:0/150698537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96540749b0 con 0x7f96540d35b0 2024-04-25T10:06:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.445+0000 7f964a7fc640 1 -- 172.21.15.16:0/150698537 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f96440795f0 con 0x7f96540d35b0 2024-04-25T10:06:36.451 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:36.451 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3381395304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3381395304},{"type":"v1","addr":"172.21.15.186:6835","nonce":3381395304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2429147785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2429147785},{"type":"v1","addr":"172.21.15.186:6837","nonce":2429147785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5142,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2191514045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2191514045},{"type":"v1","addr":"172.21.15.16:6835","nonce":2191514045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:10.940554+0000","modified":"2024-04-25T10:06:12.930243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34613},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34613":{"gid":34613,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.16:6834/1417313138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1417313138},{"type":"v1","addr":"172.21.15.16:6834","nonce":1417313138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 14 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.447+0000 7f965bb34640 1 -- 172.21.15.16:0/150698537 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f963806b4d0 msgr2=0x7f963806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.447+0000 7f965bb34640 1 --2- 172.21.15.16:0/150698537 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f963806b4d0 0x7f963806d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9650000f60 tx=0x7f9650009040 comp rx=0 tx=0).stop 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.447+0000 7f965bb34640 1 -- 172.21.15.16:0/150698537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96540d35b0 msgr2=0x7f96540ac950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.447+0000 7f965bb34640 1 --2- 172.21.15.16:0/150698537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96540d35b0 0x7f96540ac950 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9644035040 tx=0x7f96440046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.447+0000 7f965bb34640 1 -- 172.21.15.16:0/150698537 shutdown_connections 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.447+0000 7f965bb34640 1 --2- 172.21.15.16:0/150698537 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f963806b4d0 0x7f963806d990 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.447+0000 7f965bb34640 1 --2- 172.21.15.16:0/150698537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f965415dce0 0x7f96540af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.447+0000 7f965bb34640 1 --2- 172.21.15.16:0/150698537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96540d35b0 0x7f96540ac950 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.447+0000 7f965bb34640 1 --2- 172.21.15.16:0/150698537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96540b5990 0x7f96540ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.447+0000 7f965bb34640 1 -- 172.21.15.16:0/150698537 >> 172.21.15.16:0/150698537 conn(0x7f9654072900 msgr2=0x7f965415cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.447+0000 7f965bb34640 1 -- 172.21.15.16:0/150698537 shutdown_connections 2024-04-25T10:06:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.447+0000 7f965bb34640 1 -- 172.21.15.16:0/150698537 wait complete. 2024-04-25T10:06:36.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:36.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.558+0000 7f73cae58640 1 Processor -- start 2024-04-25T10:06:36.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.559+0000 7f73cae58640 1 -- start start 2024-04-25T10:06:36.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.559+0000 7f73cae58640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73c40d35b0 0x7f73c40b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.559+0000 7f73cae58640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73c415e0c0 con 0x7f73c415dce0 2024-04-25T10:06:36.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.559+0000 7f73cae58640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73c40d2510 con 0x7f73c40b5990 2024-04-25T10:06:36.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.559+0000 7f73cae58640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c40b5de0 con 0x7f73c40d35b0 2024-04-25T10:06:36.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.559+0000 7f73c8bf4640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f73c415dce0 0x7f73c40d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49120/0 (socket says 172.21.15.16:49120) 2024-04-25T10:06:36.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.559+0000 7f73c8bf4640 1 -- 172.21.15.16:0/3949369353 learned_addr learned my addr 172.21.15.16:0/3949369353 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:36.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c3fff640 1 --2- 172.21.15.16:0/3949369353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73c40d35b0 0x7f73c40b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c37fe640 1 -- 172.21.15.16:0/3949369353 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112999530 0 0) 0x7f73c415e0c0 con 0x7f73c415dce0 2024-04-25T10:06:36.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c37fe640 1 -- 172.21.15.16:0/3949369353 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73ac003520 con 0x7f73c415dce0 2024-04-25T10:06:36.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c37fe640 1 -- 172.21.15.16:0/3949369353 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2692535084 0 0) 0x7f73ac003520 con 0x7f73c415dce0 2024-04-25T10:06:36.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c37fe640 1 -- 172.21.15.16:0/3949369353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73c40d35b0 msgr2=0x7f73c40b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c37fe640 1 --2- 172.21.15.16:0/3949369353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73c40d35b0 0x7f73c40b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c37fe640 1 -- 172.21.15.16:0/3949369353 >> v1:172.21.15.186:6789/0 conn(0x7f73c40b5990 legacy=0x7f73c40b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c37fe640 1 -- 172.21.15.16:0/3949369353 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c40d27f0 con 0x7f73c415dce0 2024-04-25T10:06:36.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c37fe640 1 -- 172.21.15.16:0/3949369353 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f73b4001670 con 0x7f73c415dce0 2024-04-25T10:06:36.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c37fe640 1 -- 172.21.15.16:0/3949369353 >> v1:172.21.15.16:6789/0 conn(0x7f73c415dce0 legacy=0x7f73c40d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c37fe640 1 --2- 172.21.15.16:0/3949369353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73c40d35b0 0x7f73ac0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c37fe640 1 --2- 172.21.15.16:0/3949369353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73ac003f20 0x7f73ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c37fe640 1 --2- 172.21.15.16:0/3949369353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73ac004d40 0x7f73ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c37fe640 1 -- 172.21.15.16:0/3949369353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c40b5de0 con 0x7f73ac004d40 2024-04-25T10:06:36.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c37fe640 1 -- 172.21.15.16:0/3949369353 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac00ba20 con 0x7f73ac003f20 2024-04-25T10:06:36.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c37fe640 1 -- 172.21.15.16:0/3949369353 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac00bc20 con 0x7f73c40d35b0 2024-04-25T10:06:36.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c93f5640 1 --2- 172.21.15.16:0/3949369353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73ac004d40 0x7f73ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c8bf4640 1 --2- 172.21.15.16:0/3949369353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73c40d35b0 0x7f73ac0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.560+0000 7f73c3fff640 1 --2- 172.21.15.16:0/3949369353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73ac003f20 0x7f73ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.561+0000 7f73c93f5640 1 -- 172.21.15.16:0/3949369353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73c40d35b0 msgr2=0x7f73ac0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.561+0000 7f73c93f5640 1 --2- 172.21.15.16:0/3949369353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73c40d35b0 0x7f73ac0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.561+0000 7f73c93f5640 1 -- 172.21.15.16:0/3949369353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73ac003f20 msgr2=0x7f73ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.561+0000 7f73c93f5640 1 --2- 172.21.15.16:0/3949369353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73ac003f20 0x7f73ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.561+0000 7f73c93f5640 1 -- 172.21.15.16:0/3949369353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac003520 con 0x7f73ac004d40 2024-04-25T10:06:36.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.561+0000 7f73c3fff640 1 --2- 172.21.15.16:0/3949369353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73ac003f20 0x7f73ac004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:36.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.561+0000 7f73c93f5640 1 --2- 172.21.15.16:0/3949369353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73ac004d40 0x7f73ac0051c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f73bc001f60 tx=0x7f73ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed73581ca9b16d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.561+0000 7f73c37fe640 1 -- 172.21.15.16:0/3949369353 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc001b30 con 0x7f73ac004d40 2024-04-25T10:06:36.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.561+0000 7f73c37fe640 1 -- 172.21.15.16:0/3949369353 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73bc031620 con 0x7f73ac004d40 2024-04-25T10:06:36.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.561+0000 7f73c37fe640 1 -- 172.21.15.16:0/3949369353 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc031a10 con 0x7f73ac004d40 2024-04-25T10:06:36.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.562+0000 7f73cae58640 1 -- 172.21.15.16:0/3949369353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73ac004d40 msgr2=0x7f73ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.562+0000 7f73cae58640 1 --2- 172.21.15.16:0/3949369353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73ac004d40 0x7f73ac0051c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f73bc001f60 tx=0x7f73ac005830 comp rx=0 tx=0).stop 2024-04-25T10:06:36.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.562+0000 7f73c93f5640 1 -- 172.21.15.16:0/3949369353 reap_dead start 2024-04-25T10:06:36.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.562+0000 7f73cae58640 1 -- 172.21.15.16:0/3949369353 shutdown_connections 2024-04-25T10:06:36.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.562+0000 7f73cae58640 1 -- 172.21.15.16:0/3949369353 >> 172.21.15.16:0/3949369353 conn(0x7f73c4072900 msgr2=0x7f73c40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.562+0000 7f73cae58640 1 -- 172.21.15.16:0/3949369353 shutdown_connections 2024-04-25T10:06:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.562+0000 7f73cae58640 1 -- 172.21.15.16:0/3949369353 wait complete. 2024-04-25T10:06:36.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.563+0000 7f73cae58640 1 Processor -- start 2024-04-25T10:06:36.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.563+0000 7f73cae58640 1 -- start start 2024-04-25T10:06:36.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.564+0000 7f73cae58640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73c40c9d40 0x7f73c415e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.564+0000 7f73cae58640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73c40ca120 0x7f73c4142b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.564+0000 7f73cae58640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c41f3700 0x7f73c41f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.564+0000 7f73cae58640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c40d3090 con 0x7f73c41f3700 2024-04-25T10:06:36.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.564+0000 7f73cae58640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c40d2db0 con 0x7f73c40c9d40 2024-04-25T10:06:36.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.564+0000 7f73cae58640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c40d2160 con 0x7f73c40ca120 2024-04-25T10:06:36.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.564+0000 7f73c93f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c41f3700 0x7f73c41f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.564+0000 7f73c93f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c41f3700 0x7f73c41f3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51836/0 (socket says 172.21.15.16:51836) 2024-04-25T10:06:36.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.564+0000 7f73c93f5640 1 -- 172.21.15.16:0/2521014704 learned_addr learned my addr 172.21.15.16:0/2521014704 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:36.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.564+0000 7f73c8bf4640 1 --2- 172.21.15.16:0/2521014704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73c40c9d40 0x7f73c415e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.564+0000 7f73c3fff640 1 --2- 172.21.15.16:0/2521014704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73c40ca120 0x7f73c4142b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.564+0000 7f73c93f5640 1 -- 172.21.15.16:0/2521014704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73c40ca120 msgr2=0x7f73c4142b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.565+0000 7f73c93f5640 1 --2- 172.21.15.16:0/2521014704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73c40ca120 0x7f73c4142b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.565+0000 7f73c93f5640 1 -- 172.21.15.16:0/2521014704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73c40c9d40 msgr2=0x7f73c415e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.565+0000 7f73c93f5640 1 --2- 172.21.15.16:0/2521014704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73c40c9d40 0x7f73c415e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.565+0000 7f73c93f5640 1 -- 172.21.15.16:0/2521014704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c40cbc90 con 0x7f73c41f3700 2024-04-25T10:06:36.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.565+0000 7f73c8bf4640 1 --2- 172.21.15.16:0/2521014704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73c40c9d40 0x7f73c415e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:36.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.565+0000 7f73c93f5640 1 --2- 172.21.15.16:0/2521014704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c41f3700 0x7f73c41f3bb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f73bc02aa00 tx=0x7f73bc0015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.565+0000 7f73c17fa640 1 -- 172.21.15.16:0/2521014704 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc001b30 con 0x7f73c41f3700 2024-04-25T10:06:36.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.565+0000 7f73c17fa640 1 -- 172.21.15.16:0/2521014704 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73bc041070 con 0x7f73c41f3700 2024-04-25T10:06:36.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.565+0000 7f73cae58640 1 -- 172.21.15.16:0/2521014704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c40cbf70 con 0x7f73c41f3700 2024-04-25T10:06:36.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.565+0000 7f73c17fa640 1 -- 172.21.15.16:0/2521014704 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc003b70 con 0x7f73c41f3700 2024-04-25T10:06:36.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.565+0000 7f73cae58640 1 -- 172.21.15.16:0/2521014704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c40cc530 con 0x7f73c41f3700 2024-04-25T10:06:36.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.566+0000 7f73cae58640 1 -- 172.21.15.16:0/2521014704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c40cc730 con 0x7f73c41f3700 2024-04-25T10:06:36.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.566+0000 7f73c17fa640 1 -- 172.21.15.16:0/2521014704 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f73bc003d10 con 0x7f73c41f3700 2024-04-25T10:06:36.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.567+0000 7f73c17fa640 1 --2- 172.21.15.16:0/2521014704 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f73b006b500 0x7f73b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.567+0000 7f73c17fa640 1 -- 172.21.15.16:0/2521014704 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73bc0aaa50 con 0x7f73c41f3700 2024-04-25T10:06:36.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.567+0000 7f73c8bf4640 1 --2- 172.21.15.16:0/2521014704 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f73b006b500 0x7f73b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.568+0000 7f73c8bf4640 1 --2- 172.21.15.16:0/2521014704 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f73b006b500 0x7f73b006d9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f73b4004840 tx=0x7f73b4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.571+0000 7f73c17fa640 1 -- 172.21.15.16:0/2521014704 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f73c40cc730 con 0x7f73c41f3700 2024-04-25T10:06:36.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.753+0000 7f73cae58640 1 -- 172.21.15.16:0/2521014704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73c40749b0 con 0x7f73c41f3700 2024-04-25T10:06:36.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.754+0000 7f73c17fa640 1 -- 172.21.15.16:0/2521014704 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f73bc0798d0 con 0x7f73c41f3700 2024-04-25T10:06:36.760 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:36.760 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":36,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:06:12.582767+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:08.830059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:09.922618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:36.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.756+0000 7f73cae58640 1 -- 172.21.15.16:0/2521014704 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f73b006b500 msgr2=0x7f73b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.756+0000 7f73cae58640 1 --2- 172.21.15.16:0/2521014704 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f73b006b500 0x7f73b006d9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f73b4004840 tx=0x7f73b4009040 comp rx=0 tx=0).stop 2024-04-25T10:06:36.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.756+0000 7f73cae58640 1 -- 172.21.15.16:0/2521014704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c41f3700 msgr2=0x7f73c41f3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.756+0000 7f73cae58640 1 --2- 172.21.15.16:0/2521014704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c41f3700 0x7f73c41f3bb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f73bc02aa00 tx=0x7f73bc0015b0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.756+0000 7f73cae58640 1 -- 172.21.15.16:0/2521014704 shutdown_connections 2024-04-25T10:06:36.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.756+0000 7f73cae58640 1 --2- 172.21.15.16:0/2521014704 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f73b006b500 0x7f73b006d9c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.756+0000 7f73cae58640 1 --2- 172.21.15.16:0/2521014704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c41f3700 0x7f73c41f3bb0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.756+0000 7f73cae58640 1 --2- 172.21.15.16:0/2521014704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73c40ca120 0x7f73c4142b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.756+0000 7f73cae58640 1 --2- 172.21.15.16:0/2521014704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73c40c9d40 0x7f73c415e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.756+0000 7f73cae58640 1 -- 172.21.15.16:0/2521014704 >> 172.21.15.16:0/2521014704 conn(0x7f73c4072900 msgr2=0x7f73c40b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.756+0000 7f73cae58640 1 -- 172.21.15.16:0/2521014704 shutdown_connections 2024-04-25T10:06:36.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.756+0000 7f73cae58640 1 -- 172.21.15.16:0/2521014704 wait complete. 2024-04-25T10:06:36.773 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:08.830059+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:09.922618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:36.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.872+0000 7fc4ceaac640 1 Processor -- start 2024-04-25T10:06:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.872+0000 7fc4ceaac640 1 -- start start 2024-04-25T10:06:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.873+0000 7fc4ceaac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4c80b97c0 0x7fc4c80b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.873+0000 7fc4ceaac640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc4c80b5a30 0x7fc4c80d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.873+0000 7fc4ceaac640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c80b5e40 con 0x7fc4c815dc80 2024-04-25T10:06:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.873+0000 7fc4ceaac640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c8161f80 con 0x7fc4c80b97c0 2024-04-25T10:06:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.873+0000 7fc4ceaac640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c81627a0 con 0x7fc4c80b5a30 2024-04-25T10:06:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.873+0000 7fc4bffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4c80b97c0 0x7fc4c80b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.873+0000 7fc4cc848640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc4c815dc80 0x7fc4c8161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49128/0 (socket says 172.21.15.16:49128) 2024-04-25T10:06:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.873+0000 7fc4cc848640 1 -- 172.21.15.16:0/1332225532 learned_addr learned my addr 172.21.15.16:0/1332225532 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.873+0000 7fc4cd049640 1 --2- 172.21.15.16:0/1332225532 >> v2:172.21.15.186:3300/0 conn(0x7fc4c80b5a30 0x7fc4c80d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.873+0000 7fc4bffff640 1 -- 172.21.15.16:0/1332225532 >> v2:172.21.15.186:3300/0 conn(0x7fc4c80b5a30 msgr2=0x7fc4c80d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.873+0000 7fc4bffff640 1 --2- 172.21.15.16:0/1332225532 >> v2:172.21.15.186:3300/0 conn(0x7fc4c80b5a30 0x7fc4c80d2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.873+0000 7fc4bffff640 1 -- 172.21.15.16:0/1332225532 >> v1:172.21.15.16:6789/0 conn(0x7fc4c815dc80 legacy=0x7fc4c8161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.873+0000 7fc4bf7fe640 1 -- 172.21.15.16:0/1332225532 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514193398 0 0) 0x7fc4c4001110 con 0x7fc4c815dc80 2024-04-25T10:06:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.873+0000 7fc4bffff640 1 -- 172.21.15.16:0/1332225532 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c81621e0 con 0x7fc4c80b97c0 2024-04-25T10:06:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.874+0000 7fc4bffff640 1 --2- 172.21.15.16:0/1332225532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c80b97c0 0x7fc4c80b54f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80b6300 tx=0x7fc4b802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6feee43519d6d161 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.874+0000 7fc4bf7fe640 1 -- 172.21.15.16:0/1332225532 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8037070 con 0x7fc4c80b97c0 2024-04-25T10:06:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.874+0000 7fc4bf7fe640 1 -- 172.21.15.16:0/1332225532 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b8032440 con 0x7fc4c80b97c0 2024-04-25T10:06:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.874+0000 7fc4bf7fe640 1 -- 172.21.15.16:0/1332225532 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8032810 con 0x7fc4c80b97c0 2024-04-25T10:06:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.874+0000 7fc4ceaac640 1 -- 172.21.15.16:0/1332225532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c80b97c0 msgr2=0x7fc4c80b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.874+0000 7fc4ceaac640 1 --2- 172.21.15.16:0/1332225532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c80b97c0 0x7fc4c80b54f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80b6300 tx=0x7fc4b802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.875+0000 7fc4ceaac640 1 -- 172.21.15.16:0/1332225532 shutdown_connections 2024-04-25T10:06:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.875+0000 7fc4ceaac640 1 --2- 172.21.15.16:0/1332225532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c80b97c0 0x7fc4c80b54f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.875+0000 7fc4ceaac640 1 --2- 172.21.15.16:0/1332225532 >> v2:172.21.15.186:3300/0 conn(0x7fc4c80b5a30 0x7fc4c80d2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.875+0000 7fc4ceaac640 1 -- 172.21.15.16:0/1332225532 >> 172.21.15.16:0/1332225532 conn(0x7fc4c80d04f0 msgr2=0x7fc4c80d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.875+0000 7fc4ceaac640 1 -- 172.21.15.16:0/1332225532 shutdown_connections 2024-04-25T10:06:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.875+0000 7fc4ceaac640 1 -- 172.21.15.16:0/1332225532 wait complete. 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.876+0000 7fc4ceaac640 1 Processor -- start 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.876+0000 7fc4ceaac640 1 -- start start 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4ceaac640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4c80b5a30 0x7fc4c8150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4ceaac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c80b97c0 0x7fc4c80aa450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4ceaac640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4c815dc80 0x7fc4c80b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4ceaac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c815e060 con 0x7fc4c80b97c0 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4ceaac640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c80d1c70 con 0x7fc4c815dc80 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4ceaac640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c80d29b0 con 0x7fc4c80b5a30 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c80b97c0 0x7fc4c80aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c80b97c0 0x7fc4c80aa450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51840/0 (socket says 172.21.15.16:51840) 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4bffff640 1 -- 172.21.15.16:0/3409568874 learned_addr learned my addr 172.21.15.16:0/3409568874 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4cc848640 1 --2- 172.21.15.16:0/3409568874 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4c80b5a30 0x7fc4c8150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4cd049640 1 --2- 172.21.15.16:0/3409568874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4c815dc80 0x7fc4c80b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4bffff640 1 -- 172.21.15.16:0/3409568874 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4c80b5a30 msgr2=0x7fc4c8150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4bffff640 1 --2- 172.21.15.16:0/3409568874 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4c80b5a30 0x7fc4c8150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4bffff640 1 -- 172.21.15.16:0/3409568874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4c815dc80 msgr2=0x7fc4c80b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4bffff640 1 --2- 172.21.15.16:0/3409568874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4c815dc80 0x7fc4c80b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4bffff640 1 -- 172.21.15.16:0/3409568874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c80cabe0 con 0x7fc4c80b97c0 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4bffff640 1 --2- 172.21.15.16:0/3409568874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c80b97c0 0x7fc4c80aa450 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8000c00 tx=0x7fc4b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4bd7fa640 1 -- 172.21.15.16:0/3409568874 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8037040 con 0x7fc4c80b97c0 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4bd7fa640 1 -- 172.21.15.16:0/3409568874 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b8042070 con 0x7fc4c80b97c0 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4ceaac640 1 -- 172.21.15.16:0/3409568874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c80caec0 con 0x7fc4c80b97c0 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4bd7fa640 1 -- 172.21.15.16:0/3409568874 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8030920 con 0x7fc4c80b97c0 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.877+0000 7fc4ceaac640 1 -- 172.21.15.16:0/3409568874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c80cb480 con 0x7fc4c80b97c0 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.879+0000 7fc4ceaac640 1 -- 172.21.15.16:0/3409568874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c80750b0 con 0x7fc4c80b97c0 2024-04-25T10:06:36.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.879+0000 7fc4bd7fa640 1 -- 172.21.15.16:0/3409568874 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4b8030ac0 con 0x7fc4c80b97c0 2024-04-25T10:06:36.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.879+0000 7fc4bd7fa640 1 --2- 172.21.15.16:0/3409568874 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc4b406b4d0 0x7fc4b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.879+0000 7fc4bd7fa640 1 -- 172.21.15.16:0/3409568874 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4b80aab20 con 0x7fc4c80b97c0 2024-04-25T10:06:36.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.880+0000 7fc4cc848640 1 --2- 172.21.15.16:0/3409568874 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc4b406b4d0 0x7fc4b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.880+0000 7fc4cc848640 1 --2- 172.21.15.16:0/3409568874 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc4b406b4d0 0x7fc4b406d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4000f60 tx=0x7fc4c4007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:36.884+0000 7fc4bd7fa640 1 -- 172.21.15.16:0/3409568874 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4b8079350 con 0x7fc4c80b97c0 2024-04-25T10:06:37.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.063+0000 7fc4ceaac640 1 -- 172.21.15.16:0/3409568874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4c815c310 con 0x7fc4c80b97c0 2024-04-25T10:06:37.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.063+0000 7fc4bd7fa640 1 -- 172.21.15.16:0/3409568874 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc4b807d3f0 con 0x7fc4c80b97c0 2024-04-25T10:06:37.069 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:06:37.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.065+0000 7fc4ceaac640 1 -- 172.21.15.16:0/3409568874 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc4b406b4d0 msgr2=0x7fc4b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.065+0000 7fc4ceaac640 1 --2- 172.21.15.16:0/3409568874 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc4b406b4d0 0x7fc4b406d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4000f60 tx=0x7fc4c4007c00 comp rx=0 tx=0).stop 2024-04-25T10:06:37.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.065+0000 7fc4ceaac640 1 -- 172.21.15.16:0/3409568874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c80b97c0 msgr2=0x7fc4c80aa450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.065+0000 7fc4ceaac640 1 --2- 172.21.15.16:0/3409568874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c80b97c0 0x7fc4c80aa450 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8000c00 tx=0x7fc4b80046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.065+0000 7fc4ceaac640 1 -- 172.21.15.16:0/3409568874 shutdown_connections 2024-04-25T10:06:37.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.065+0000 7fc4ceaac640 1 --2- 172.21.15.16:0/3409568874 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc4b406b4d0 0x7fc4b406d990 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.065+0000 7fc4ceaac640 1 --2- 172.21.15.16:0/3409568874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4c815dc80 0x7fc4c80b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.065+0000 7fc4ceaac640 1 --2- 172.21.15.16:0/3409568874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c80b97c0 0x7fc4c80aa450 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.065+0000 7fc4ceaac640 1 --2- 172.21.15.16:0/3409568874 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4c80b5a30 0x7fc4c8150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.065+0000 7fc4ceaac640 1 -- 172.21.15.16:0/3409568874 >> 172.21.15.16:0/3409568874 conn(0x7fc4c80d04f0 msgr2=0x7fc4c80d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.065+0000 7fc4ceaac640 1 -- 172.21.15.16:0/3409568874 shutdown_connections 2024-04-25T10:06:37.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.065+0000 7fc4ceaac640 1 -- 172.21.15.16:0/3409568874 wait complete. 2024-04-25T10:06:37.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:37.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:37.127 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:37.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.180+0000 7f8b4cedd640 1 Processor -- start 2024-04-25T10:06:37.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.180+0000 7f8b4cedd640 1 -- start start 2024-04-25T10:06:37.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.181+0000 7f8b4cedd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8b480d0ca0 0x7f8b480d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.181+0000 7f8b4cedd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b480cfee0 con 0x7f8b4809a650 2024-04-25T10:06:37.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.181+0000 7f8b4cedd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b480d01c0 con 0x7f8b4815dff0 2024-04-25T10:06:37.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.181+0000 7f8b4cedd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b480d04a0 con 0x7f8b480d0ca0 2024-04-25T10:06:37.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.181+0000 7f8b46d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8b480d0ca0 0x7f8b480d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.181+0000 7f8b45d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8b4809a650 0x7f8b480cbf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49130/0 (socket says 172.21.15.16:49130) 2024-04-25T10:06:37.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.181+0000 7f8b45d9b640 1 -- 172.21.15.16:0/128096599 learned_addr learned my addr 172.21.15.16:0/128096599 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:37.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.181+0000 7f8b46d9d640 1 -- 172.21.15.16:0/128096599 >> v1:172.21.15.186:6789/0 conn(0x7f8b4815dff0 legacy=0x7f8b4806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.181+0000 7f8b46d9d640 1 -- 172.21.15.16:0/128096599 >> v1:172.21.15.16:6789/0 conn(0x7f8b4809a650 legacy=0x7f8b480cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.181+0000 7f8b46d9d640 1 -- 172.21.15.16:0/128096599 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b480d0780 con 0x7f8b480d0ca0 2024-04-25T10:06:37.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.182+0000 7f8b46d9d640 1 --2- 172.21.15.16:0/128096599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b480d0ca0 0x7f8b480d1110 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009d90 tx=0x7f8b4002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53d34f4d5cdc6fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.182+0000 7f8b4559a640 1 -- 172.21.15.16:0/128096599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40007740 con 0x7f8b480d0ca0 2024-04-25T10:06:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.182+0000 7f8b4559a640 1 -- 172.21.15.16:0/128096599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b400078e0 con 0x7f8b480d0ca0 2024-04-25T10:06:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.182+0000 7f8b4559a640 1 -- 172.21.15.16:0/128096599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40007cb0 con 0x7f8b480d0ca0 2024-04-25T10:06:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.183+0000 7f8b4cedd640 1 -- 172.21.15.16:0/128096599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b480d0ca0 msgr2=0x7f8b480d1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.183+0000 7f8b4cedd640 1 --2- 172.21.15.16:0/128096599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b480d0ca0 0x7f8b480d1110 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009d90 tx=0x7f8b4002a730 comp rx=0 tx=0).stop 2024-04-25T10:06:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.183+0000 7f8b4cedd640 1 -- 172.21.15.16:0/128096599 shutdown_connections 2024-04-25T10:06:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.183+0000 7f8b4cedd640 1 --2- 172.21.15.16:0/128096599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b480d0ca0 0x7f8b480d1110 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.183+0000 7f8b4cedd640 1 -- 172.21.15.16:0/128096599 >> 172.21.15.16:0/128096599 conn(0x7f8b480b33f0 msgr2=0x7f8b480b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.183+0000 7f8b4cedd640 1 -- 172.21.15.16:0/128096599 shutdown_connections 2024-04-25T10:06:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.183+0000 7f8b4cedd640 1 -- 172.21.15.16:0/128096599 wait complete. 2024-04-25T10:06:37.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.184+0000 7f8b4cedd640 1 Processor -- start 2024-04-25T10:06:37.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.184+0000 7f8b4cedd640 1 -- start start 2024-04-25T10:06:37.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.185+0000 7f8b4cedd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b4809a650 0x7f8b4815b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.185+0000 7f8b4cedd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b480d0ca0 0x7f8b480c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.185+0000 7f8b4cedd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b4806feb0 0x7f8b48070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.185+0000 7f8b4cedd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b480cc660 con 0x7f8b480d0ca0 2024-04-25T10:06:37.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.185+0000 7f8b4cedd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b48099a40 con 0x7f8b4806feb0 2024-04-25T10:06:37.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.185+0000 7f8b4cedd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b481f47c0 con 0x7f8b4809a650 2024-04-25T10:06:37.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.185+0000 7f8b45d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b480d0ca0 0x7f8b480c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.185+0000 7f8b45d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b480d0ca0 0x7f8b480c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51858/0 (socket says 172.21.15.16:51858) 2024-04-25T10:06:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.185+0000 7f8b45d9b640 1 -- 172.21.15.16:0/2489008864 learned_addr learned my addr 172.21.15.16:0/2489008864 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.185+0000 7f8b46d9d640 1 --2- 172.21.15.16:0/2489008864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b4806feb0 0x7f8b48070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.185+0000 7f8b4659c640 1 --2- 172.21.15.16:0/2489008864 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b4809a650 0x7f8b4815b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.186+0000 7f8b45d9b640 1 -- 172.21.15.16:0/2489008864 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b4809a650 msgr2=0x7f8b4815b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.186+0000 7f8b45d9b640 1 --2- 172.21.15.16:0/2489008864 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b4809a650 0x7f8b4815b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.186+0000 7f8b45d9b640 1 -- 172.21.15.16:0/2489008864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b4806feb0 msgr2=0x7f8b48070340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.186+0000 7f8b45d9b640 1 --2- 172.21.15.16:0/2489008864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b4806feb0 0x7f8b48070340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.186+0000 7f8b45d9b640 1 -- 172.21.15.16:0/2489008864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b481f44e0 con 0x7f8b480d0ca0 2024-04-25T10:06:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.186+0000 7f8b4659c640 1 --2- 172.21.15.16:0/2489008864 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b4809a650 0x7f8b4815b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.186+0000 7f8b45d9b640 1 --2- 172.21.15.16:0/2489008864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b480d0ca0 0x7f8b480c72f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00c440 tx=0x7f8b3c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.186+0000 7f8b36ffd640 1 -- 172.21.15.16:0/2489008864 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c017070 con 0x7f8b480d0ca0 2024-04-25T10:06:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.186+0000 7f8b36ffd640 1 -- 172.21.15.16:0/2489008864 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3c005ce0 con 0x7f8b480d0ca0 2024-04-25T10:06:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.186+0000 7f8b4cedd640 1 -- 172.21.15.16:0/2489008864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4815a740 con 0x7f8b480d0ca0 2024-04-25T10:06:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.186+0000 7f8b36ffd640 1 -- 172.21.15.16:0/2489008864 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0072f0 con 0x7f8b480d0ca0 2024-04-25T10:06:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.186+0000 7f8b4cedd640 1 -- 172.21.15.16:0/2489008864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b480c7830 con 0x7f8b480d0ca0 2024-04-25T10:06:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.187+0000 7f8b4cedd640 1 -- 172.21.15.16:0/2489008864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b480750b0 con 0x7f8b480d0ca0 2024-04-25T10:06:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.187+0000 7f8b36ffd640 1 -- 172.21.15.16:0/2489008864 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b3c01a070 con 0x7f8b480d0ca0 2024-04-25T10:06:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.188+0000 7f8b36ffd640 1 --2- 172.21.15.16:0/2489008864 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8b2806b4d0 0x7f8b2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.188+0000 7f8b36ffd640 1 -- 172.21.15.16:0/2489008864 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b3c089560 con 0x7f8b480d0ca0 2024-04-25T10:06:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.188+0000 7f8b4659c640 1 --2- 172.21.15.16:0/2489008864 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8b2806b4d0 0x7f8b2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.188+0000 7f8b4659c640 1 --2- 172.21.15.16:0/2489008864 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8b2806b4d0 0x7f8b2806d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800a830 tx=0x7f8b38008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.192+0000 7f8b36ffd640 1 -- 172.21.15.16:0/2489008864 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b3c058470 con 0x7f8b480d0ca0 2024-04-25T10:06:37.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.373+0000 7f8b4cedd640 1 -- 172.21.15.16:0/2489008864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8b480ccef0 con 0x7f8b480d0ca0 2024-04-25T10:06:37.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.374+0000 7f8b36ffd640 1 -- 172.21.15.16:0/2489008864 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8b3c057d70 con 0x7f8b480d0ca0 2024-04-25T10:06:37.379 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:06:37.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.376+0000 7f8b4cedd640 1 -- 172.21.15.16:0/2489008864 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8b2806b4d0 msgr2=0x7f8b2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.376+0000 7f8b4cedd640 1 --2- 172.21.15.16:0/2489008864 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8b2806b4d0 0x7f8b2806d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800a830 tx=0x7f8b38008040 comp rx=0 tx=0).stop 2024-04-25T10:06:37.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.376+0000 7f8b4cedd640 1 -- 172.21.15.16:0/2489008864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b480d0ca0 msgr2=0x7f8b480c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.376+0000 7f8b4cedd640 1 --2- 172.21.15.16:0/2489008864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b480d0ca0 0x7f8b480c72f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00c440 tx=0x7f8b3c001e70 comp rx=0 tx=0).stop 2024-04-25T10:06:37.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.376+0000 7f8b4cedd640 1 -- 172.21.15.16:0/2489008864 shutdown_connections 2024-04-25T10:06:37.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.376+0000 7f8b4cedd640 1 --2- 172.21.15.16:0/2489008864 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8b2806b4d0 0x7f8b2806d990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.376+0000 7f8b4cedd640 1 --2- 172.21.15.16:0/2489008864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b4806feb0 0x7f8b48070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.376+0000 7f8b4cedd640 1 --2- 172.21.15.16:0/2489008864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b480d0ca0 0x7f8b480c72f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.376+0000 7f8b4cedd640 1 --2- 172.21.15.16:0/2489008864 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b4809a650 0x7f8b4815b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.376+0000 7f8b4cedd640 1 -- 172.21.15.16:0/2489008864 >> 172.21.15.16:0/2489008864 conn(0x7f8b480b33f0 msgr2=0x7f8b4815e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.376+0000 7f8b4cedd640 1 -- 172.21.15.16:0/2489008864 shutdown_connections 2024-04-25T10:06:37.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.376+0000 7f8b4cedd640 1 -- 172.21.15.16:0/2489008864 wait complete. 2024-04-25T10:06:37.392 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.530+0000 7f90f864d640 1 Processor -- start 2024-04-25T10:06:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.531+0000 7f90f864d640 1 -- start start 2024-04-25T10:06:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.531+0000 7f90f864d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90f015bbe0 0x7f90f00cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.531+0000 7f90f864d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90f00cf510 0x7f90f00cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.531+0000 7f90f864d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f00cc830 con 0x7f90f00cff00 2024-04-25T10:06:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.531+0000 7f90f864d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f00ccb10 con 0x7f90f015bbe0 2024-04-25T10:06:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.531+0000 7f90f864d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f00ccdf0 con 0x7f90f00cf510 2024-04-25T10:06:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.532+0000 7f90f63e9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90f015bbe0 0x7f90f00cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.532+0000 7f90f6bea640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f90f00cff00 0x7f90f00c5490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49146/0 (socket says 172.21.15.16:49146) 2024-04-25T10:06:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.532+0000 7f90f6bea640 1 -- 172.21.15.16:0/3894313646 learned_addr learned my addr 172.21.15.16:0/3894313646 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.532+0000 7f90f63e9640 1 -- 172.21.15.16:0/3894313646 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90f00cf510 msgr2=0x7f90f00cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.532+0000 7f90f5be8640 1 --2- 172.21.15.16:0/3894313646 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90f00cf510 0x7f90f00cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.532+0000 7f90f63e9640 1 --2- 172.21.15.16:0/3894313646 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90f00cf510 0x7f90f00cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.532+0000 7f90f63e9640 1 -- 172.21.15.16:0/3894313646 >> v1:172.21.15.16:6789/0 conn(0x7f90f00cff00 legacy=0x7f90f00c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.532+0000 7f90f53e7640 1 -- 172.21.15.16:0/3894313646 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2441923058 0 0) 0x7f90f00cc830 con 0x7f90f00cff00 2024-04-25T10:06:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.532+0000 7f90f63e9640 1 -- 172.21.15.16:0/3894313646 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f00cd0d0 con 0x7f90f015bbe0 2024-04-25T10:06:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.532+0000 7f90f5be8640 1 --2- 172.21.15.16:0/3894313646 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90f00cf510 0x7f90f00cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:37.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.533+0000 7f90f63e9640 1 --2- 172.21.15.16:0/3894313646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f015bbe0 0x7f90f00cd8d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f90ec0097a0 tx=0x7f90ec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b5e673ef856749 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.533+0000 7f90f53e7640 1 -- 172.21.15.16:0/3894313646 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec037070 con 0x7f90f015bbe0 2024-04-25T10:06:37.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.533+0000 7f90f53e7640 1 -- 172.21.15.16:0/3894313646 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ec032440 con 0x7f90f015bbe0 2024-04-25T10:06:37.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.533+0000 7f90f53e7640 1 -- 172.21.15.16:0/3894313646 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec032810 con 0x7f90f015bbe0 2024-04-25T10:06:37.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.533+0000 7f90f864d640 1 -- 172.21.15.16:0/3894313646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f015bbe0 msgr2=0x7f90f00cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.533+0000 7f90f864d640 1 --2- 172.21.15.16:0/3894313646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f015bbe0 0x7f90f00cd8d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f90ec0097a0 tx=0x7f90ec02a180 comp rx=0 tx=0).stop 2024-04-25T10:06:37.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.533+0000 7f90f864d640 1 -- 172.21.15.16:0/3894313646 shutdown_connections 2024-04-25T10:06:37.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.533+0000 7f90f864d640 1 --2- 172.21.15.16:0/3894313646 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90f00cf510 0x7f90f00cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.533+0000 7f90f864d640 1 --2- 172.21.15.16:0/3894313646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f015bbe0 0x7f90f00cd8d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.533+0000 7f90f864d640 1 -- 172.21.15.16:0/3894313646 >> 172.21.15.16:0/3894313646 conn(0x7f90f00b3430 msgr2=0x7f90f00b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.534+0000 7f90f864d640 1 -- 172.21.15.16:0/3894313646 shutdown_connections 2024-04-25T10:06:37.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.534+0000 7f90f864d640 1 -- 172.21.15.16:0/3894313646 wait complete. 2024-04-25T10:06:37.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.535+0000 7f90f864d640 1 Processor -- start 2024-04-25T10:06:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.535+0000 7f90f864d640 1 -- start start 2024-04-25T10:06:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.535+0000 7f90f864d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90f00cf510 0x7f90f00c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.535+0000 7f90f864d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90f00cff00 0x7f90f006bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90f864d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f015bbe0 0x7f90f015cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90f864d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f00cde10 con 0x7f90f015bbe0 2024-04-25T10:06:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90f864d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f015bfc0 con 0x7f90f00cff00 2024-04-25T10:06:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90f864d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f00cc450 con 0x7f90f00cf510 2024-04-25T10:06:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90f6bea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f015bbe0 0x7f90f015cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90f6bea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f015bbe0 0x7f90f015cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51890/0 (socket says 172.21.15.16:51890) 2024-04-25T10:06:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90f6bea640 1 -- 172.21.15.16:0/3012330027 learned_addr learned my addr 172.21.15.16:0/3012330027 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90f63e9640 1 --2- 172.21.15.16:0/3012330027 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90f00cf510 0x7f90f00c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90f5be8640 1 --2- 172.21.15.16:0/3012330027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90f00cff00 0x7f90f006bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90f6bea640 1 -- 172.21.15.16:0/3012330027 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90f00cf510 msgr2=0x7f90f00c55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90f6bea640 1 --2- 172.21.15.16:0/3012330027 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90f00cf510 0x7f90f00c55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90f6bea640 1 -- 172.21.15.16:0/3012330027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90f00cff00 msgr2=0x7f90f006bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90f6bea640 1 --2- 172.21.15.16:0/3012330027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90f00cff00 0x7f90f006bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90f6bea640 1 -- 172.21.15.16:0/3012330027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f006c6e0 con 0x7f90f015bbe0 2024-04-25T10:06:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90f6bea640 1 --2- 172.21.15.16:0/3012330027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f015bbe0 0x7f90f015cb50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f90e800c7d0 tx=0x7f90e800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90e6ffd640 1 -- 172.21.15.16:0/3012330027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e8018070 con 0x7f90f015bbe0 2024-04-25T10:06:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90e6ffd640 1 -- 172.21.15.16:0/3012330027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e800a640 con 0x7f90f015bbe0 2024-04-25T10:06:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90e6ffd640 1 -- 172.21.15.16:0/3012330027 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e800aa70 con 0x7f90f015bbe0 2024-04-25T10:06:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90f864d640 1 -- 172.21.15.16:0/3012330027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f006c9c0 con 0x7f90f015bbe0 2024-04-25T10:06:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.536+0000 7f90f864d640 1 -- 172.21.15.16:0/3012330027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f006cf80 con 0x7f90f015bbe0 2024-04-25T10:06:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.537+0000 7f90f864d640 1 -- 172.21.15.16:0/3012330027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f00750b0 con 0x7f90f015bbe0 2024-04-25T10:06:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.538+0000 7f90e6ffd640 1 -- 172.21.15.16:0/3012330027 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90e801b070 con 0x7f90f015bbe0 2024-04-25T10:06:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.538+0000 7f90e6ffd640 1 --2- 172.21.15.16:0/3012330027 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f90d406b4d0 0x7f90d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.538+0000 7f90e6ffd640 1 -- 172.21.15.16:0/3012330027 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90e8089800 con 0x7f90f015bbe0 2024-04-25T10:06:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.539+0000 7f90f63e9640 1 --2- 172.21.15.16:0/3012330027 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f90d406b4d0 0x7f90d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.539+0000 7f90f63e9640 1 --2- 172.21.15.16:0/3012330027 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f90d406b4d0 0x7f90d406d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f90ec034000 tx=0x7f90ec02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.543+0000 7f90e6ffd640 1 -- 172.21.15.16:0/3012330027 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90e8058680 con 0x7f90f015bbe0 2024-04-25T10:06:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.723+0000 7f90f864d640 1 -- 172.21.15.16:0/3012330027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f90f00b42d0 con 0x7f90f015bbe0 2024-04-25T10:06:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.723+0000 7f90e6ffd640 1 -- 172.21.15.16:0/3012330027 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f90e8057f80 con 0x7f90f015bbe0 2024-04-25T10:06:37.729 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:06:37.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.725+0000 7f90f864d640 1 -- 172.21.15.16:0/3012330027 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f90d406b4d0 msgr2=0x7f90d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.725+0000 7f90f864d640 1 --2- 172.21.15.16:0/3012330027 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f90d406b4d0 0x7f90d406d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f90ec034000 tx=0x7f90ec02e000 comp rx=0 tx=0).stop 2024-04-25T10:06:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.725+0000 7f90f864d640 1 -- 172.21.15.16:0/3012330027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f015bbe0 msgr2=0x7f90f015cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.725+0000 7f90f864d640 1 --2- 172.21.15.16:0/3012330027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f015bbe0 0x7f90f015cb50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f90e800c7d0 tx=0x7f90e800cc90 comp rx=0 tx=0).stop 2024-04-25T10:06:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.725+0000 7f90f864d640 1 -- 172.21.15.16:0/3012330027 shutdown_connections 2024-04-25T10:06:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.725+0000 7f90f864d640 1 --2- 172.21.15.16:0/3012330027 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f90d406b4d0 0x7f90d406d990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.725+0000 7f90f864d640 1 --2- 172.21.15.16:0/3012330027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f015bbe0 0x7f90f015cb50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.725+0000 7f90f864d640 1 --2- 172.21.15.16:0/3012330027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90f00cff00 0x7f90f006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.725+0000 7f90f864d640 1 --2- 172.21.15.16:0/3012330027 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90f00cf510 0x7f90f00c55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.725+0000 7f90f864d640 1 -- 172.21.15.16:0/3012330027 >> 172.21.15.16:0/3012330027 conn(0x7f90f00b3430 msgr2=0x7f90f00cafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.725+0000 7f90f864d640 1 -- 172.21.15.16:0/3012330027 shutdown_connections 2024-04-25T10:06:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.725+0000 7f90f864d640 1 -- 172.21.15.16:0/3012330027 wait complete. 2024-04-25T10:06:37.741 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:37.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.837+0000 7fbf5a86b640 1 Processor -- start 2024-04-25T10:06:37.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.837+0000 7fbf5a86b640 1 -- start start 2024-04-25T10:06:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.837+0000 7fbf5a86b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf540cd060 0x7fbf5406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.837+0000 7fbf5a86b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbf5406d690 0x7fbf5406edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.837+0000 7fbf5a86b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf540719f0 con 0x7fbf541476d0 2024-04-25T10:06:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.837+0000 7fbf5a86b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf54071cd0 con 0x7fbf5406d690 2024-04-25T10:06:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.837+0000 7fbf5a86b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf54071fb0 con 0x7fbf540cd060 2024-04-25T10:06:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.838+0000 7fbf537fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf540cd060 0x7fbf5406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.838+0000 7fbf58e08640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbf5406d690 0x7fbf5406edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.838+0000 7fbf53fff640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fbf541476d0 0x7fbf540ca110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:37808/0 (socket says 172.21.15.16:37808) 2024-04-25T10:06:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.838+0000 7fbf537fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf540cd060 0x7fbf5406e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:48152/0 (socket says 172.21.15.16:48152) 2024-04-25T10:06:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.838+0000 7fbf53fff640 1 -- 172.21.15.16:0/3565098612 learned_addr learned my addr 172.21.15.16:0/3565098612 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.838+0000 7fbf537fe640 1 -- 172.21.15.16:0/3565098612 >> v2:172.21.15.186:3300/0 conn(0x7fbf5406d690 msgr2=0x7fbf5406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.838+0000 7fbf537fe640 1 --2- 172.21.15.16:0/3565098612 >> v2:172.21.15.186:3300/0 conn(0x7fbf5406d690 0x7fbf5406edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.838+0000 7fbf537fe640 1 -- 172.21.15.16:0/3565098612 >> v1:172.21.15.186:6789/0 conn(0x7fbf541476d0 legacy=0x7fbf540ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.838+0000 7fbf537fe640 1 -- 172.21.15.16:0/3565098612 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf54072290 con 0x7fbf540cd060 2024-04-25T10:06:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.838+0000 7fbf58e08640 1 --2- 172.21.15.16:0/3565098612 >> v2:172.21.15.186:3300/0 conn(0x7fbf5406d690 0x7fbf5406edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:37.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.839+0000 7fbf537fe640 1 --2- 172.21.15.16:0/3565098612 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf540cd060 0x7fbf5406e8b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbf440097a0 tx=0x7fbf4402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7d595724f39424c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.839+0000 7fbf52ffd640 1 -- 172.21.15.16:0/3565098612 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44037070 con 0x7fbf540cd060 2024-04-25T10:06:37.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.839+0000 7fbf52ffd640 1 -- 172.21.15.16:0/3565098612 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf44032440 con 0x7fbf540cd060 2024-04-25T10:06:37.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.839+0000 7fbf52ffd640 1 -- 172.21.15.16:0/3565098612 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44032810 con 0x7fbf540cd060 2024-04-25T10:06:37.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.840+0000 7fbf5a86b640 1 -- 172.21.15.16:0/3565098612 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf540cd060 msgr2=0x7fbf5406e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.840+0000 7fbf5a86b640 1 --2- 172.21.15.16:0/3565098612 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf540cd060 0x7fbf5406e8b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbf440097a0 tx=0x7fbf4402a210 comp rx=0 tx=0).stop 2024-04-25T10:06:37.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.840+0000 7fbf5a86b640 1 -- 172.21.15.16:0/3565098612 shutdown_connections 2024-04-25T10:06:37.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.840+0000 7fbf5a86b640 1 --2- 172.21.15.16:0/3565098612 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf540cd060 0x7fbf5406e8b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.840+0000 7fbf5a86b640 1 --2- 172.21.15.16:0/3565098612 >> v2:172.21.15.186:3300/0 conn(0x7fbf5406d690 0x7fbf5406edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.840+0000 7fbf5a86b640 1 -- 172.21.15.16:0/3565098612 >> 172.21.15.16:0/3565098612 conn(0x7fbf540cc540 msgr2=0x7fbf540cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.840+0000 7fbf5a86b640 1 -- 172.21.15.16:0/3565098612 shutdown_connections 2024-04-25T10:06:37.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.840+0000 7fbf5a86b640 1 -- 172.21.15.16:0/3565098612 wait complete. 2024-04-25T10:06:37.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.841+0000 7fbf5a86b640 1 Processor -- start 2024-04-25T10:06:37.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.841+0000 7fbf5a86b640 1 -- start start 2024-04-25T10:06:37.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.842+0000 7fbf5a86b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf5406d690 0x7fbf5413b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.842+0000 7fbf5a86b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf540cd060 0x7fbf540b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.842+0000 7fbf5a86b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf541476d0 0x7fbf5413bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.842+0000 7fbf5a86b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf540c93e0 con 0x7fbf5406d690 2024-04-25T10:06:37.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.842+0000 7fbf5a86b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf540c9260 con 0x7fbf541476d0 2024-04-25T10:06:37.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.842+0000 7fbf5a86b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf540c9560 con 0x7fbf540cd060 2024-04-25T10:06:37.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.842+0000 7fbf53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf5406d690 0x7fbf5413b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.842+0000 7fbf53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf5406d690 0x7fbf5413b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51904/0 (socket says 172.21.15.16:51904) 2024-04-25T10:06:37.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.842+0000 7fbf53fff640 1 -- 172.21.15.16:0/141019264 learned_addr learned my addr 172.21.15.16:0/141019264 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:37.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.842+0000 7fbf58e08640 1 --2- 172.21.15.16:0/141019264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf541476d0 0x7fbf5413bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.842+0000 7fbf537fe640 1 --2- 172.21.15.16:0/141019264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf540cd060 0x7fbf540b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.842+0000 7fbf53fff640 1 -- 172.21.15.16:0/141019264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf540cd060 msgr2=0x7fbf540b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.842+0000 7fbf53fff640 1 --2- 172.21.15.16:0/141019264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf540cd060 0x7fbf540b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.842+0000 7fbf53fff640 1 -- 172.21.15.16:0/141019264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf541476d0 msgr2=0x7fbf5413bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.842+0000 7fbf53fff640 1 --2- 172.21.15.16:0/141019264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf541476d0 0x7fbf5413bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.842+0000 7fbf53fff640 1 -- 172.21.15.16:0/141019264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5415e4f0 con 0x7fbf5406d690 2024-04-25T10:06:37.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.842+0000 7fbf537fe640 1 --2- 172.21.15.16:0/141019264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf540cd060 0x7fbf540b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:37.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.842+0000 7fbf53fff640 1 --2- 172.21.15.16:0/141019264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf5406d690 0x7fbf5413b4a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00c3c0 tx=0x7fbf3c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.843+0000 7fbf50ff9640 1 -- 172.21.15.16:0/141019264 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c017070 con 0x7fbf5406d690 2024-04-25T10:06:37.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.843+0000 7fbf50ff9640 1 -- 172.21.15.16:0/141019264 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3c005ce0 con 0x7fbf5406d690 2024-04-25T10:06:37.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.843+0000 7fbf5a86b640 1 -- 172.21.15.16:0/141019264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5415e7d0 con 0x7fbf5406d690 2024-04-25T10:06:37.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.843+0000 7fbf50ff9640 1 -- 172.21.15.16:0/141019264 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c0072f0 con 0x7fbf5406d690 2024-04-25T10:06:37.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.843+0000 7fbf5a86b640 1 -- 172.21.15.16:0/141019264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5415ed90 con 0x7fbf5406d690 2024-04-25T10:06:37.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.844+0000 7fbf5a86b640 1 -- 172.21.15.16:0/141019264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf540749b0 con 0x7fbf5406d690 2024-04-25T10:06:37.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.844+0000 7fbf50ff9640 1 -- 172.21.15.16:0/141019264 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf3c01a070 con 0x7fbf5406d690 2024-04-25T10:06:37.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.845+0000 7fbf50ff9640 1 --2- 172.21.15.16:0/141019264 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbf3406b4d0 0x7fbf3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.845+0000 7fbf50ff9640 1 -- 172.21.15.16:0/141019264 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf3c089510 con 0x7fbf5406d690 2024-04-25T10:06:37.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.845+0000 7fbf537fe640 1 --2- 172.21.15.16:0/141019264 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbf3406b4d0 0x7fbf3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.845+0000 7fbf537fe640 1 --2- 172.21.15.16:0/141019264 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbf3406b4d0 0x7fbf3406d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbf44034000 tx=0x7fbf4402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:37.849+0000 7fbf50ff9640 1 -- 172.21.15.16:0/141019264 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf3c058420 con 0x7fbf5406d690 2024-04-25T10:06:38.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.069+0000 7fbf5a86b640 1 -- 172.21.15.16:0/141019264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf540ccca0 con 0x7fbf5406d690 2024-04-25T10:06:38.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.070+0000 7fbf50ff9640 1 -- 172.21.15.16:0/141019264 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fbf3c057d20 con 0x7fbf5406d690 2024-04-25T10:06:38.076 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:38.076 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3381395304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3381395304},{"type":"v1","addr":"172.21.15.186:6835","nonce":3381395304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2429147785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2429147785},{"type":"v1","addr":"172.21.15.186:6837","nonce":2429147785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5142,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2191514045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2191514045},{"type":"v1","addr":"172.21.15.16:6835","nonce":2191514045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:10.940554+0000","modified":"2024-04-25T10:06:12.930243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34613},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34613":{"gid":34613,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.16:6834/1417313138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1417313138},{"type":"v1","addr":"172.21.15.16:6834","nonce":1417313138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:38.076 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 14 2024-04-25T10:06:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.072+0000 7fbf5a86b640 1 -- 172.21.15.16:0/141019264 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbf3406b4d0 msgr2=0x7fbf3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.072+0000 7fbf5a86b640 1 --2- 172.21.15.16:0/141019264 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbf3406b4d0 0x7fbf3406d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbf44034000 tx=0x7fbf4402e000 comp rx=0 tx=0).stop 2024-04-25T10:06:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.072+0000 7fbf5a86b640 1 -- 172.21.15.16:0/141019264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf5406d690 msgr2=0x7fbf5413b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.072+0000 7fbf5a86b640 1 --2- 172.21.15.16:0/141019264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf5406d690 0x7fbf5413b4a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00c3c0 tx=0x7fbf3c00c880 comp rx=0 tx=0).stop 2024-04-25T10:06:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.072+0000 7fbf5a86b640 1 -- 172.21.15.16:0/141019264 shutdown_connections 2024-04-25T10:06:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.072+0000 7fbf5a86b640 1 --2- 172.21.15.16:0/141019264 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbf3406b4d0 0x7fbf3406d990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.072+0000 7fbf5a86b640 1 --2- 172.21.15.16:0/141019264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf541476d0 0x7fbf5413bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.072+0000 7fbf5a86b640 1 --2- 172.21.15.16:0/141019264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf540cd060 0x7fbf540b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.072+0000 7fbf5a86b640 1 --2- 172.21.15.16:0/141019264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf5406d690 0x7fbf5413b4a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.072+0000 7fbf5a86b640 1 -- 172.21.15.16:0/141019264 >> 172.21.15.16:0/141019264 conn(0x7fbf540cc540 msgr2=0x7fbf540b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.072+0000 7fbf5a86b640 1 -- 172.21.15.16:0/141019264 shutdown_connections 2024-04-25T10:06:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.072+0000 7fbf5a86b640 1 -- 172.21.15.16:0/141019264 wait complete. 2024-04-25T10:06:38.088 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.184+0000 7f989de3f640 1 Processor -- start 2024-04-25T10:06:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.184+0000 7f989de3f640 1 -- start start 2024-04-25T10:06:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.185+0000 7f989de3f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f989815dce0 0x7f98980d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.185+0000 7f989de3f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98980d35b0 0x7f98980b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.185+0000 7f989de3f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989815e0c0 con 0x7f98980d1f90 2024-04-25T10:06:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.185+0000 7f989de3f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98980b7b10 con 0x7f98980d35b0 2024-04-25T10:06:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.185+0000 7f989de3f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98980d2510 con 0x7f989815dce0 2024-04-25T10:06:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.185+0000 7f9896ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98980d35b0 0x7f98980b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.185+0000 7f9897fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f98980d1f90 0x7f989806bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49152/0 (socket says 172.21.15.16:49152) 2024-04-25T10:06:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.185+0000 7f9897fff640 1 -- 172.21.15.16:0/4156522076 learned_addr learned my addr 172.21.15.16:0/4156522076 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.185+0000 7f98977fe640 1 --2- 172.21.15.16:0/4156522076 >> v2:172.21.15.186:3300/0 conn(0x7f989815dce0 0x7f98980d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.186+0000 7f98967fc640 1 -- 172.21.15.16:0/4156522076 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3676877920 0 0) 0x7f989815e0c0 con 0x7f98980d1f90 2024-04-25T10:06:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.186+0000 7f98967fc640 1 -- 172.21.15.16:0/4156522076 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9880003520 con 0x7f98980d1f90 2024-04-25T10:06:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.186+0000 7f9896ffd640 1 -- 172.21.15.16:0/4156522076 >> v2:172.21.15.186:3300/0 conn(0x7f989815dce0 msgr2=0x7f98980d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.186+0000 7f9896ffd640 1 --2- 172.21.15.16:0/4156522076 >> v2:172.21.15.186:3300/0 conn(0x7f989815dce0 0x7f98980d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.186+0000 7f9896ffd640 1 -- 172.21.15.16:0/4156522076 >> v1:172.21.15.16:6789/0 conn(0x7f98980d1f90 legacy=0x7f989806bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.186+0000 7f9896ffd640 1 -- 172.21.15.16:0/4156522076 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98980d27f0 con 0x7f98980d35b0 2024-04-25T10:06:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.186+0000 7f9896ffd640 1 --2- 172.21.15.16:0/4156522076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98980d35b0 0x7f98980b5450 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f98980ce220 tx=0x7f988c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c9d4cbac3e68e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.187+0000 7f98967fc640 1 -- 172.21.15.16:0/4156522076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c037070 con 0x7f98980d35b0 2024-04-25T10:06:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.187+0000 7f98967fc640 1 -- 172.21.15.16:0/4156522076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988c032440 con 0x7f98980d35b0 2024-04-25T10:06:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.187+0000 7f98967fc640 1 -- 172.21.15.16:0/4156522076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c032810 con 0x7f98980d35b0 2024-04-25T10:06:38.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.187+0000 7f989de3f640 1 -- 172.21.15.16:0/4156522076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98980d35b0 msgr2=0x7f98980b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.187+0000 7f989de3f640 1 --2- 172.21.15.16:0/4156522076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98980d35b0 0x7f98980b5450 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f98980ce220 tx=0x7f988c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.187+0000 7f989de3f640 1 -- 172.21.15.16:0/4156522076 shutdown_connections 2024-04-25T10:06:38.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.187+0000 7f989de3f640 1 --2- 172.21.15.16:0/4156522076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98980d35b0 0x7f98980b5450 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.187+0000 7f989de3f640 1 --2- 172.21.15.16:0/4156522076 >> v2:172.21.15.186:3300/0 conn(0x7f989815dce0 0x7f98980d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.187+0000 7f989de3f640 1 -- 172.21.15.16:0/4156522076 >> 172.21.15.16:0/4156522076 conn(0x7f9898072900 msgr2=0x7f98980d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:38.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.187+0000 7f989de3f640 1 -- 172.21.15.16:0/4156522076 shutdown_connections 2024-04-25T10:06:38.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.187+0000 7f989de3f640 1 -- 172.21.15.16:0/4156522076 wait complete. 2024-04-25T10:06:38.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.188+0000 7f989de3f640 1 Processor -- start 2024-04-25T10:06:38.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.188+0000 7f989de3f640 1 -- start start 2024-04-25T10:06:38.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.189+0000 7f989de3f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98980d1f90 0x7f9898160c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.189+0000 7f989de3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98980d35b0 0x7f98981f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.189+0000 7f989de3f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f989815dce0 0x7f98980ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.189+0000 7f989de3f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98980d3090 con 0x7f98980d35b0 2024-04-25T10:06:38.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.189+0000 7f989de3f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98980d2db0 con 0x7f98980d1f90 2024-04-25T10:06:38.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.189+0000 7f989de3f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98980b9820 con 0x7f989815dce0 2024-04-25T10:06:38.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.189+0000 7f9896ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98980d35b0 0x7f98981f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.189+0000 7f9896ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98980d35b0 0x7f98981f7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51918/0 (socket says 172.21.15.16:51918) 2024-04-25T10:06:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.189+0000 7f9896ffd640 1 -- 172.21.15.16:0/386401930 learned_addr learned my addr 172.21.15.16:0/386401930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.189+0000 7f98977fe640 1 --2- 172.21.15.16:0/386401930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98980d1f90 0x7f9898160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.190+0000 7f9897fff640 1 --2- 172.21.15.16:0/386401930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f989815dce0 0x7f98980ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.190+0000 7f9896ffd640 1 -- 172.21.15.16:0/386401930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f989815dce0 msgr2=0x7f98980ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.190+0000 7f9896ffd640 1 --2- 172.21.15.16:0/386401930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f989815dce0 0x7f98980ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.190+0000 7f9896ffd640 1 -- 172.21.15.16:0/386401930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98980d1f90 msgr2=0x7f9898160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.190+0000 7f9896ffd640 1 --2- 172.21.15.16:0/386401930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98980d1f90 0x7f9898160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.190+0000 7f9896ffd640 1 -- 172.21.15.16:0/386401930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98981f77b0 con 0x7f98980d35b0 2024-04-25T10:06:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.190+0000 7f98977fe640 1 --2- 172.21.15.16:0/386401930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98980d1f90 0x7f9898160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.190+0000 7f9896ffd640 1 --2- 172.21.15.16:0/386401930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98980d35b0 0x7f98981f7270 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f988c02a1e0 tx=0x7f988c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.190+0000 7f987ffff640 1 -- 172.21.15.16:0/386401930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c041030 con 0x7f98980d35b0 2024-04-25T10:06:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.190+0000 7f987ffff640 1 -- 172.21.15.16:0/386401930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988c03d420 con 0x7f98980d35b0 2024-04-25T10:06:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.190+0000 7f989de3f640 1 -- 172.21.15.16:0/386401930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9898155730 con 0x7f98980d35b0 2024-04-25T10:06:38.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.190+0000 7f987ffff640 1 -- 172.21.15.16:0/386401930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c037040 con 0x7f98980d35b0 2024-04-25T10:06:38.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.190+0000 7f989de3f640 1 -- 172.21.15.16:0/386401930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9898155cf0 con 0x7f98980d35b0 2024-04-25T10:06:38.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.191+0000 7f989de3f640 1 -- 172.21.15.16:0/386401930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98980749b0 con 0x7f98980d35b0 2024-04-25T10:06:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.192+0000 7f987ffff640 1 -- 172.21.15.16:0/386401930 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f988c044070 con 0x7f98980d35b0 2024-04-25T10:06:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.192+0000 7f987ffff640 1 --2- 172.21.15.16:0/386401930 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f988406b4d0 0x7f988406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.192+0000 7f987ffff640 1 -- 172.21.15.16:0/386401930 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f988c0aa990 con 0x7f98980d35b0 2024-04-25T10:06:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.192+0000 7f98977fe640 1 --2- 172.21.15.16:0/386401930 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f988406b4d0 0x7f988406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.193+0000 7f98977fe640 1 --2- 172.21.15.16:0/386401930 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f988406b4d0 0x7f988406d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9888009af0 tx=0x7f9888008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:38.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.196+0000 7f987ffff640 1 -- 172.21.15.16:0/386401930 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f988c079940 con 0x7f98980d35b0 2024-04-25T10:06:38.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.379+0000 7f989de3f640 1 -- 172.21.15.16:0/386401930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98980d3d80 con 0x7f98980d35b0 2024-04-25T10:06:38.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.380+0000 7f987ffff640 1 -- 172.21.15.16:0/386401930 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f988c079240 con 0x7f98980d35b0 2024-04-25T10:06:38.386 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:38.386 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":36,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:06:12.582767+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:08.830059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:09.922618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:38.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.383+0000 7f989de3f640 1 -- 172.21.15.16:0/386401930 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f988406b4d0 msgr2=0x7f988406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.383+0000 7f989de3f640 1 --2- 172.21.15.16:0/386401930 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f988406b4d0 0x7f988406d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9888009af0 tx=0x7f9888008040 comp rx=0 tx=0).stop 2024-04-25T10:06:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.383+0000 7f989de3f640 1 -- 172.21.15.16:0/386401930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98980d35b0 msgr2=0x7f98981f7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.383+0000 7f989de3f640 1 --2- 172.21.15.16:0/386401930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98980d35b0 0x7f98981f7270 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f988c02a1e0 tx=0x7f988c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.383+0000 7f989de3f640 1 -- 172.21.15.16:0/386401930 shutdown_connections 2024-04-25T10:06:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.383+0000 7f989de3f640 1 --2- 172.21.15.16:0/386401930 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f988406b4d0 0x7f988406d990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.383+0000 7f989de3f640 1 --2- 172.21.15.16:0/386401930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f989815dce0 0x7f98980ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.383+0000 7f989de3f640 1 --2- 172.21.15.16:0/386401930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98980d35b0 0x7f98981f7270 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.383+0000 7f989de3f640 1 --2- 172.21.15.16:0/386401930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98980d1f90 0x7f9898160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.383+0000 7f989de3f640 1 -- 172.21.15.16:0/386401930 >> 172.21.15.16:0/386401930 conn(0x7f9898072900 msgr2=0x7f98980b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.383+0000 7f989de3f640 1 -- 172.21.15.16:0/386401930 shutdown_connections 2024-04-25T10:06:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.383+0000 7f989de3f640 1 -- 172.21.15.16:0/386401930 wait complete. 2024-04-25T10:06:38.399 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:06:38.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.495+0000 7ff30137e640 1 Processor -- start 2024-04-25T10:06:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.495+0000 7ff30137e640 1 -- start start 2024-04-25T10:06:38.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.496+0000 7ff30137e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff2fc15a870 0x7ff2fc0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.496+0000 7ff30137e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2fc06e2e0 0x7ff2fc15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.496+0000 7ff30137e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2fc0cd580 0x7ff2fc0cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.496+0000 7ff30137e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2fc0ce720 con 0x7ff2fc0cd580 2024-04-25T10:06:38.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.496+0000 7ff30137e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2fc0cdb10 con 0x7ff2fc15a870 2024-04-25T10:06:38.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.496+0000 7ff30137e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc0cddf0 con 0x7ff2fc06e2e0 2024-04-25T10:06:38.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.496+0000 7ff2fb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2fc0cd580 0x7ff2fc0cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.496+0000 7ff2fb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2fc0cd580 0x7ff2fc0cf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51924/0 (socket says 172.21.15.16:51924) 2024-04-25T10:06:38.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.496+0000 7ff2fb7fe640 1 -- 172.21.15.16:0/617908017 learned_addr learned my addr 172.21.15.16:0/617908017 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:38.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.496+0000 7ff2faffd640 1 --2- 172.21.15.16:0/617908017 >> v2:172.21.15.186:3300/0 conn(0x7ff2fc15a870 0x7ff2fc0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.496+0000 7ff2fa7fc640 1 --2- 172.21.15.16:0/617908017 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2fc06e2e0 0x7ff2fc15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.496+0000 7ff2fb7fe640 1 -- 172.21.15.16:0/617908017 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2fc06e2e0 msgr2=0x7ff2fc15ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:38.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.496+0000 7ff2fb7fe640 1 --2- 172.21.15.16:0/617908017 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2fc06e2e0 0x7ff2fc15ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.496+0000 7ff2fb7fe640 1 -- 172.21.15.16:0/617908017 >> v2:172.21.15.186:3300/0 conn(0x7ff2fc15a870 msgr2=0x7ff2fc0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:38.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.496+0000 7ff2fb7fe640 1 --2- 172.21.15.16:0/617908017 >> v2:172.21.15.186:3300/0 conn(0x7ff2fc15a870 0x7ff2fc0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.496+0000 7ff2fb7fe640 1 -- 172.21.15.16:0/617908017 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2fc0ce0d0 con 0x7ff2fc0cd580 2024-04-25T10:06:38.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.497+0000 7ff2fb7fe640 1 --2- 172.21.15.16:0/617908017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc0cd580 0x7ff2fc0cf530 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc06ba10 tx=0x7ff2f402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7732d54379bb28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:38.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.497+0000 7ff2f9ffb640 1 -- 172.21.15.16:0/617908017 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4036070 con 0x7ff2fc0cd580 2024-04-25T10:06:38.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.497+0000 7ff2f9ffb640 1 -- 172.21.15.16:0/617908017 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f4031600 con 0x7ff2fc0cd580 2024-04-25T10:06:38.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.497+0000 7ff2f9ffb640 1 -- 172.21.15.16:0/617908017 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40319d0 con 0x7ff2fc0cd580 2024-04-25T10:06:38.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.497+0000 7ff30137e640 1 -- 172.21.15.16:0/617908017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc0cd580 msgr2=0x7ff2fc0cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.497+0000 7ff30137e640 1 --2- 172.21.15.16:0/617908017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc0cd580 0x7ff2fc0cf530 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc06ba10 tx=0x7ff2f402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.498+0000 7ff30137e640 1 -- 172.21.15.16:0/617908017 shutdown_connections 2024-04-25T10:06:38.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.498+0000 7ff30137e640 1 --2- 172.21.15.16:0/617908017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc0cd580 0x7ff2fc0cf530 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.498+0000 7ff30137e640 1 --2- 172.21.15.16:0/617908017 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2fc06e2e0 0x7ff2fc15ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.498+0000 7ff30137e640 1 --2- 172.21.15.16:0/617908017 >> v2:172.21.15.186:3300/0 conn(0x7ff2fc15a870 0x7ff2fc0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.498+0000 7ff30137e640 1 -- 172.21.15.16:0/617908017 >> 172.21.15.16:0/617908017 conn(0x7ff2fc06aba0 msgr2=0x7ff2fc06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:38.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.498+0000 7ff30137e640 1 -- 172.21.15.16:0/617908017 shutdown_connections 2024-04-25T10:06:38.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.498+0000 7ff30137e640 1 -- 172.21.15.16:0/617908017 wait complete. 2024-04-25T10:06:38.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.499+0000 7ff30137e640 1 Processor -- start 2024-04-25T10:06:38.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.499+0000 7ff30137e640 1 -- start start 2024-04-25T10:06:38.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.500+0000 7ff30137e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc06e2e0 0x7ff2fc155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.500+0000 7ff30137e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2fc0cd580 0x7ff2fc152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.500+0000 7ff30137e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2fc15a870 0x7ff2fc1f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.500+0000 7ff30137e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc0cfbf0 con 0x7ff2fc06e2e0 2024-04-25T10:06:38.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.500+0000 7ff30137e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc0cfa70 con 0x7ff2fc0cd580 2024-04-25T10:06:38.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.500+0000 7ff30137e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc0cfd70 con 0x7ff2fc15a870 2024-04-25T10:06:38.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.500+0000 7ff2faffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc06e2e0 0x7ff2fc155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.500+0000 7ff2faffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc06e2e0 0x7ff2fc155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51932/0 (socket says 172.21.15.16:51932) 2024-04-25T10:06:38.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.500+0000 7ff2faffd640 1 -- 172.21.15.16:0/1061184336 learned_addr learned my addr 172.21.15.16:0/1061184336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:38.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.500+0000 7ff2fa7fc640 1 --2- 172.21.15.16:0/1061184336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2fc0cd580 0x7ff2fc152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.500+0000 7ff2fb7fe640 1 --2- 172.21.15.16:0/1061184336 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2fc15a870 0x7ff2fc1f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.500+0000 7ff2faffd640 1 -- 172.21.15.16:0/1061184336 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2fc15a870 msgr2=0x7ff2fc1f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.500+0000 7ff2faffd640 1 --2- 172.21.15.16:0/1061184336 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2fc15a870 0x7ff2fc1f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.500+0000 7ff2faffd640 1 -- 172.21.15.16:0/1061184336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2fc0cd580 msgr2=0x7ff2fc152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.500+0000 7ff2faffd640 1 --2- 172.21.15.16:0/1061184336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2fc0cd580 0x7ff2fc152c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.500+0000 7ff2faffd640 1 -- 172.21.15.16:0/1061184336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2fc153460 con 0x7ff2fc06e2e0 2024-04-25T10:06:38.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.500+0000 7ff2fb7fe640 1 --2- 172.21.15.16:0/1061184336 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2fc15a870 0x7ff2fc1f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:38.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.500+0000 7ff2faffd640 1 --2- 172.21.15.16:0/1061184336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc06e2e0 0x7ff2fc155c90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00e9e0 tx=0x7ff2ec00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:38.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.501+0000 7ff2e37fe640 1 -- 172.21.15.16:0/1061184336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec017070 con 0x7ff2fc06e2e0 2024-04-25T10:06:38.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.501+0000 7ff2e37fe640 1 -- 172.21.15.16:0/1061184336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ec00c9d0 con 0x7ff2fc06e2e0 2024-04-25T10:06:38.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.501+0000 7ff30137e640 1 -- 172.21.15.16:0/1061184336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2fc153740 con 0x7ff2fc06e2e0 2024-04-25T10:06:38.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.501+0000 7ff2e37fe640 1 -- 172.21.15.16:0/1061184336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec00cdc0 con 0x7ff2fc06e2e0 2024-04-25T10:06:38.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.501+0000 7ff30137e640 1 -- 172.21.15.16:0/1061184336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2fc153150 con 0x7ff2fc06e2e0 2024-04-25T10:06:38.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.502+0000 7ff30137e640 1 -- 172.21.15.16:0/1061184336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2fc157960 con 0x7ff2fc06e2e0 2024-04-25T10:06:38.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.502+0000 7ff2e37fe640 1 -- 172.21.15.16:0/1061184336 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff2ec01a070 con 0x7ff2fc06e2e0 2024-04-25T10:06:38.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.503+0000 7ff2e37fe640 1 --2- 172.21.15.16:0/1061184336 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff2e806b4d0 0x7ff2e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.503+0000 7ff2e37fe640 1 -- 172.21.15.16:0/1061184336 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2ec08a100 con 0x7ff2fc06e2e0 2024-04-25T10:06:38.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.503+0000 7ff2fa7fc640 1 --2- 172.21.15.16:0/1061184336 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff2e806b4d0 0x7ff2e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.503+0000 7ff2fa7fc640 1 --2- 172.21.15.16:0/1061184336 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff2e806b4d0 0x7ff2e806d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc154120 tx=0x7ff2f0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:38.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.507+0000 7ff2e37fe640 1 -- 172.21.15.16:0/1061184336 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff2ec058930 con 0x7ff2fc06e2e0 2024-04-25T10:06:38.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.726+0000 7ff30137e640 1 -- 172.21.15.16:0/1061184336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2fc0749b0 con 0x7ff2fc06e2e0 2024-04-25T10:06:38.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.727+0000 7ff2e37fe640 1 -- 172.21.15.16:0/1061184336 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4946 (secure 0 0 0) 0x7ff2ec05c9d0 con 0x7ff2fc06e2e0 2024-04-25T10:06:38.732 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:38.732 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3381395304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3381395304},{"type":"v1","addr":"172.21.15.186:6835","nonce":3381395304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2429147785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2429147785},{"type":"v1","addr":"172.21.15.186:6837","nonce":2429147785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5142,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2191514045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2191514045},{"type":"v1","addr":"172.21.15.16:6835","nonce":2191514045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:10.940554+0000","modified":"2024-04-25T10:06:12.930243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34613},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34613":{"gid":34613,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.16:6834/1417313138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1417313138},{"type":"v1","addr":"172.21.15.16:6834","nonce":1417313138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:38.733 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 14 2024-04-25T10:06:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.728+0000 7ff30137e640 1 -- 172.21.15.16:0/1061184336 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff2e806b4d0 msgr2=0x7ff2e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.728+0000 7ff30137e640 1 --2- 172.21.15.16:0/1061184336 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff2e806b4d0 0x7ff2e806d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc154120 tx=0x7ff2f0009190 comp rx=0 tx=0).stop 2024-04-25T10:06:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.728+0000 7ff30137e640 1 -- 172.21.15.16:0/1061184336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc06e2e0 msgr2=0x7ff2fc155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.728+0000 7ff30137e640 1 --2- 172.21.15.16:0/1061184336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc06e2e0 0x7ff2fc155c90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00e9e0 tx=0x7ff2ec00eea0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.729+0000 7ff30137e640 1 -- 172.21.15.16:0/1061184336 shutdown_connections 2024-04-25T10:06:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.729+0000 7ff30137e640 1 --2- 172.21.15.16:0/1061184336 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff2e806b4d0 0x7ff2e806d990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.729+0000 7ff30137e640 1 --2- 172.21.15.16:0/1061184336 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2fc15a870 0x7ff2fc1f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.729+0000 7ff30137e640 1 --2- 172.21.15.16:0/1061184336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2fc0cd580 0x7ff2fc152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.729+0000 7ff30137e640 1 --2- 172.21.15.16:0/1061184336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc06e2e0 0x7ff2fc155c90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.729+0000 7ff30137e640 1 -- 172.21.15.16:0/1061184336 >> 172.21.15.16:0/1061184336 conn(0x7ff2fc06aba0 msgr2=0x7ff2fc071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.729+0000 7ff30137e640 1 -- 172.21.15.16:0/1061184336 shutdown_connections 2024-04-25T10:06:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.729+0000 7ff30137e640 1 -- 172.21.15.16:0/1061184336 wait complete. 2024-04-25T10:06:38.745 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:38.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.840+0000 7f56443b7640 1 Processor -- start 2024-04-25T10:06:38.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.840+0000 7f56443b7640 1 -- start start 2024-04-25T10:06:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.841+0000 7f56443b7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f563c15dce0 0x7f563c0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.841+0000 7f56443b7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f563c0d35b0 0x7f563c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.841+0000 7f56443b7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563c15e0c0 con 0x7f563c0d1f90 2024-04-25T10:06:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.841+0000 7f56443b7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563c0b7b10 con 0x7f563c0d35b0 2024-04-25T10:06:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.841+0000 7f56443b7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0d2510 con 0x7f563c15dce0 2024-04-25T10:06:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.841+0000 7f5641952640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f563c0d35b0 0x7f563c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.841+0000 7f5642954640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f563c0d1f90 0x7f563c06bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49154/0 (socket says 172.21.15.16:49154) 2024-04-25T10:06:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.841+0000 7f5642954640 1 -- 172.21.15.16:0/3989858587 learned_addr learned my addr 172.21.15.16:0/3989858587 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.841+0000 7f5642153640 1 --2- 172.21.15.16:0/3989858587 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f563c15dce0 0x7f563c0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.841+0000 7f5641952640 1 -- 172.21.15.16:0/3989858587 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f563c15dce0 msgr2=0x7f563c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.841+0000 7f5641952640 1 --2- 172.21.15.16:0/3989858587 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f563c15dce0 0x7f563c0d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.841+0000 7f5641952640 1 -- 172.21.15.16:0/3989858587 >> v1:172.21.15.16:6789/0 conn(0x7f563c0d1f90 legacy=0x7f563c06bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.841+0000 7f5641151640 1 -- 172.21.15.16:0/3989858587 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406795771 0 0) 0x7f563c15e0c0 con 0x7f563c0d1f90 2024-04-25T10:06:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.841+0000 7f5641952640 1 -- 172.21.15.16:0/3989858587 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c0d27f0 con 0x7f563c0d35b0 2024-04-25T10:06:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.842+0000 7f5641952640 1 --2- 172.21.15.16:0/3989858587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f563c0d35b0 0x7f563c0b5450 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f562c0097a0 tx=0x7f562c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1a4f50f433cf28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.842+0000 7f5641151640 1 -- 172.21.15.16:0/3989858587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c037070 con 0x7f563c0d35b0 2024-04-25T10:06:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.842+0000 7f5641151640 1 -- 172.21.15.16:0/3989858587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c02ae50 con 0x7f563c0d35b0 2024-04-25T10:06:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.842+0000 7f5641151640 1 -- 172.21.15.16:0/3989858587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c032660 con 0x7f563c0d35b0 2024-04-25T10:06:38.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.843+0000 7f56443b7640 1 -- 172.21.15.16:0/3989858587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f563c0d35b0 msgr2=0x7f563c0b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.843+0000 7f56443b7640 1 --2- 172.21.15.16:0/3989858587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f563c0d35b0 0x7f563c0b5450 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f562c0097a0 tx=0x7f562c02a100 comp rx=0 tx=0).stop 2024-04-25T10:06:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.843+0000 7f56443b7640 1 -- 172.21.15.16:0/3989858587 shutdown_connections 2024-04-25T10:06:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.843+0000 7f56443b7640 1 --2- 172.21.15.16:0/3989858587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f563c0d35b0 0x7f563c0b5450 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.843+0000 7f56443b7640 1 --2- 172.21.15.16:0/3989858587 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f563c15dce0 0x7f563c0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.843+0000 7f56443b7640 1 -- 172.21.15.16:0/3989858587 >> 172.21.15.16:0/3989858587 conn(0x7f563c072900 msgr2=0x7f563c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.843+0000 7f56443b7640 1 -- 172.21.15.16:0/3989858587 shutdown_connections 2024-04-25T10:06:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.843+0000 7f56443b7640 1 -- 172.21.15.16:0/3989858587 wait complete. 2024-04-25T10:06:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.844+0000 7f56443b7640 1 Processor -- start 2024-04-25T10:06:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.844+0000 7f56443b7640 1 -- start start 2024-04-25T10:06:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f56443b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f563c0d1f90 0x7f563c152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f56443b7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f563c0d35b0 0x7f563c0c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f56443b7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f563c15dce0 0x7f563c0c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f56443b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0d3090 con 0x7f563c0d1f90 2024-04-25T10:06:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f56443b7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0d2db0 con 0x7f563c0d35b0 2024-04-25T10:06:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f56443b7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0b9820 con 0x7f563c15dce0 2024-04-25T10:06:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f5642153640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f563c0d1f90 0x7f563c152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f5642153640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f563c0d1f90 0x7f563c152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51948/0 (socket says 172.21.15.16:51948) 2024-04-25T10:06:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f5642153640 1 -- 172.21.15.16:0/3932249732 learned_addr learned my addr 172.21.15.16:0/3932249732 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f5642153640 1 -- 172.21.15.16:0/3932249732 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f563c15dce0 msgr2=0x7f563c0c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f5642153640 1 --2- 172.21.15.16:0/3932249732 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f563c15dce0 0x7f563c0c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f5642153640 1 -- 172.21.15.16:0/3932249732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f563c0d35b0 msgr2=0x7f563c0c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f5642153640 1 --2- 172.21.15.16:0/3932249732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f563c0d35b0 0x7f563c0c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f5642153640 1 -- 172.21.15.16:0/3932249732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c0cc630 con 0x7f563c0d1f90 2024-04-25T10:06:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f5642153640 1 --2- 172.21.15.16:0/3932249732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f563c0d1f90 0x7f563c152ef0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f563800d830 tx=0x7f563800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f5632ffd640 1 -- 172.21.15.16:0/3932249732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563800b6c0 con 0x7f563c0d1f90 2024-04-25T10:06:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f5632ffd640 1 -- 172.21.15.16:0/3932249732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5638019070 con 0x7f563c0d1f90 2024-04-25T10:06:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f5632ffd640 1 -- 172.21.15.16:0/3932249732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563800ba80 con 0x7f563c0d1f90 2024-04-25T10:06:38.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.845+0000 7f56443b7640 1 -- 172.21.15.16:0/3932249732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563c0cc910 con 0x7f563c0d1f90 2024-04-25T10:06:38.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.846+0000 7f56443b7640 1 -- 172.21.15.16:0/3932249732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c0cced0 con 0x7f563c0d1f90 2024-04-25T10:06:38.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.846+0000 7f56443b7640 1 -- 172.21.15.16:0/3932249732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563c0749b0 con 0x7f563c0d1f90 2024-04-25T10:06:38.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.847+0000 7f5632ffd640 1 -- 172.21.15.16:0/3932249732 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f563801e460 con 0x7f563c0d1f90 2024-04-25T10:06:38.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.847+0000 7f5632ffd640 1 --2- 172.21.15.16:0/3932249732 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f562006b4d0 0x7f562006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.847+0000 7f5632ffd640 1 -- 172.21.15.16:0/3932249732 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f563808a820 con 0x7f563c0d1f90 2024-04-25T10:06:38.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.848+0000 7f5641952640 1 --2- 172.21.15.16:0/3932249732 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f562006b4d0 0x7f562006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.848+0000 7f5641952640 1 --2- 172.21.15.16:0/3932249732 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f562006b4d0 0x7f562006d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f562c034000 tx=0x7f562c02c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:38.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:38.851+0000 7f5632ffd640 1 -- 172.21.15.16:0/3932249732 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5638059730 con 0x7f563c0d1f90 2024-04-25T10:06:39.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.067+0000 7f56443b7640 1 -- 172.21.15.16:0/3932249732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f563c0d1510 con 0x7f563c0d1f90 2024-04-25T10:06:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.067+0000 7f5632ffd640 1 -- 172.21.15.16:0/3932249732 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f5638059030 con 0x7f563c0d1f90 2024-04-25T10:06:39.073 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:39.073 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3381395304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3381395304},{"type":"v1","addr":"172.21.15.186:6835","nonce":3381395304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2429147785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2429147785},{"type":"v1","addr":"172.21.15.186:6837","nonce":2429147785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5142,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2191514045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2191514045},{"type":"v1","addr":"172.21.15.16:6835","nonce":2191514045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:10.940554+0000","modified":"2024-04-25T10:06:12.930243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34613},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34613":{"gid":34613,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.16:6834/1417313138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1417313138},{"type":"v1","addr":"172.21.15.16:6834","nonce":1417313138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 14 2024-04-25T10:06:39.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.069+0000 7f56443b7640 1 -- 172.21.15.16:0/3932249732 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f562006b4d0 msgr2=0x7f562006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.069+0000 7f56443b7640 1 --2- 172.21.15.16:0/3932249732 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f562006b4d0 0x7f562006d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f562c034000 tx=0x7f562c02c000 comp rx=0 tx=0).stop 2024-04-25T10:06:39.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.069+0000 7f56443b7640 1 -- 172.21.15.16:0/3932249732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f563c0d1f90 msgr2=0x7f563c152ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.069+0000 7f56443b7640 1 --2- 172.21.15.16:0/3932249732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f563c0d1f90 0x7f563c152ef0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f563800d830 tx=0x7f563800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.069+0000 7f56443b7640 1 -- 172.21.15.16:0/3932249732 shutdown_connections 2024-04-25T10:06:39.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.069+0000 7f56443b7640 1 --2- 172.21.15.16:0/3932249732 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f562006b4d0 0x7f562006d990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.069+0000 7f56443b7640 1 --2- 172.21.15.16:0/3932249732 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f563c15dce0 0x7f563c0c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.069+0000 7f56443b7640 1 --2- 172.21.15.16:0/3932249732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f563c0d35b0 0x7f563c0c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.069+0000 7f56443b7640 1 --2- 172.21.15.16:0/3932249732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f563c0d1f90 0x7f563c152ef0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.069+0000 7f56443b7640 1 -- 172.21.15.16:0/3932249732 >> 172.21.15.16:0/3932249732 conn(0x7f563c072900 msgr2=0x7f563c0b38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.069+0000 7f56443b7640 1 -- 172.21.15.16:0/3932249732 shutdown_connections 2024-04-25T10:06:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.069+0000 7f56443b7640 1 -- 172.21.15.16:0/3932249732 wait complete. 2024-04-25T10:06:39.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:39.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.180+0000 7ff0a2c5d640 1 Processor -- start 2024-04-25T10:06:39.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.180+0000 7ff0a2c5d640 1 -- start start 2024-04-25T10:06:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.181+0000 7ff0a2c5d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09c0b9460 0x7ff09c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.181+0000 7ff0a2c5d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff09c0d34e0 0x7ff09c0cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.181+0000 7ff0a2c5d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09c0d2440 con 0x7ff09c15dd50 2024-04-25T10:06:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.181+0000 7ff0a2c5d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09c0d2720 con 0x7ff09c0d34e0 2024-04-25T10:06:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.181+0000 7ff0a2c5d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0d2a00 con 0x7ff09c0b9460 2024-04-25T10:06:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.181+0000 7ff0a09f9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff09c15dd50 0x7ff09c0d1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49168/0 (socket says 172.21.15.16:49168) 2024-04-25T10:06:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.181+0000 7ff0a09f9640 1 -- 172.21.15.16:0/1217477613 learned_addr learned my addr 172.21.15.16:0/1217477613 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff0a11fa640 1 --2- 172.21.15.16:0/1217477613 >> v2:172.21.15.186:3300/0 conn(0x7ff09c0d34e0 0x7ff09c0cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09b7fe640 1 -- 172.21.15.16:0/1217477613 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665390574 0 0) 0x7ff09c0d2440 con 0x7ff09c15dd50 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09bfff640 1 --2- 172.21.15.16:0/1217477613 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09c0b9460 0x7ff09c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09b7fe640 1 -- 172.21.15.16:0/1217477613 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff084003520 con 0x7ff09c15dd50 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09b7fe640 1 -- 172.21.15.16:0/1217477613 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3188478039 0 0) 0x7ff084003520 con 0x7ff09c15dd50 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09b7fe640 1 -- 172.21.15.16:0/1217477613 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09c0b9460 msgr2=0x7ff09c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09b7fe640 1 --2- 172.21.15.16:0/1217477613 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09c0b9460 0x7ff09c0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09b7fe640 1 -- 172.21.15.16:0/1217477613 >> v2:172.21.15.186:3300/0 conn(0x7ff09c0d34e0 msgr2=0x7ff09c0cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09b7fe640 1 --2- 172.21.15.16:0/1217477613 >> v2:172.21.15.186:3300/0 conn(0x7ff09c0d34e0 0x7ff09c0cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09b7fe640 1 -- 172.21.15.16:0/1217477613 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c0d2ce0 con 0x7ff09c15dd50 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09b7fe640 1 -- 172.21.15.16:0/1217477613 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7ff08c001670 con 0x7ff09c15dd50 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09b7fe640 1 -- 172.21.15.16:0/1217477613 >> v1:172.21.15.16:6789/0 conn(0x7ff09c15dd50 legacy=0x7ff09c0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09b7fe640 1 --2- 172.21.15.16:0/1217477613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0840039b0 0x7ff084003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09b7fe640 1 --2- 172.21.15.16:0/1217477613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff084004800 0x7ff084004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09b7fe640 1 --2- 172.21.15.16:0/1217477613 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09c0b9460 0x7ff0840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09b7fe640 1 -- 172.21.15.16:0/1217477613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0d2720 con 0x7ff0840039b0 2024-04-25T10:06:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09b7fe640 1 -- 172.21.15.16:0/1217477613 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0d2a00 con 0x7ff084004800 2024-04-25T10:06:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09b7fe640 1 -- 172.21.15.16:0/1217477613 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08400b9c0 con 0x7ff09c0b9460 2024-04-25T10:06:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff0a09f9640 1 --2- 172.21.15.16:0/1217477613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0840039b0 0x7ff084003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff0a11fa640 1 --2- 172.21.15.16:0/1217477613 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09c0b9460 0x7ff0840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09bfff640 1 --2- 172.21.15.16:0/1217477613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff084004800 0x7ff084004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff0a09f9640 1 -- 172.21.15.16:0/1217477613 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09c0b9460 msgr2=0x7ff0840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff0a09f9640 1 --2- 172.21.15.16:0/1217477613 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09c0b9460 0x7ff0840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff0a09f9640 1 -- 172.21.15.16:0/1217477613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff084004800 msgr2=0x7ff084004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff0a09f9640 1 --2- 172.21.15.16:0/1217477613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff084004800 0x7ff084004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff0a09f9640 1 -- 172.21.15.16:0/1217477613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff084003520 con 0x7ff0840039b0 2024-04-25T10:06:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff0a11fa640 1 --2- 172.21.15.16:0/1217477613 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09c0b9460 0x7ff0840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.182+0000 7ff09bfff640 1 --2- 172.21.15.16:0/1217477613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff084004800 0x7ff084004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.183+0000 7ff0a09f9640 1 --2- 172.21.15.16:0/1217477613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0840039b0 0x7ff084003dd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c210 tx=0x7ff084005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1347a3a33ba596f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.183+0000 7ff09b7fe640 1 -- 172.21.15.16:0/1217477613 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c007d70 con 0x7ff0840039b0 2024-04-25T10:06:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.183+0000 7ff09b7fe640 1 -- 172.21.15.16:0/1217477613 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c005af0 con 0x7ff0840039b0 2024-04-25T10:06:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.183+0000 7ff09b7fe640 1 -- 172.21.15.16:0/1217477613 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c007070 con 0x7ff0840039b0 2024-04-25T10:06:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.184+0000 7ff0a2c5d640 1 -- 172.21.15.16:0/1217477613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0840039b0 msgr2=0x7ff084003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.184+0000 7ff0a2c5d640 1 --2- 172.21.15.16:0/1217477613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0840039b0 0x7ff084003dd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c210 tx=0x7ff084005830 comp rx=0 tx=0).stop 2024-04-25T10:06:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.184+0000 7ff0a11fa640 1 -- 172.21.15.16:0/1217477613 reap_dead start 2024-04-25T10:06:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.184+0000 7ff0a2c5d640 1 -- 172.21.15.16:0/1217477613 shutdown_connections 2024-04-25T10:06:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.184+0000 7ff0a2c5d640 1 -- 172.21.15.16:0/1217477613 >> 172.21.15.16:0/1217477613 conn(0x7ff09c0722d0 msgr2=0x7ff09c0726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.184+0000 7ff0a2c5d640 1 -- 172.21.15.16:0/1217477613 shutdown_connections 2024-04-25T10:06:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.184+0000 7ff0a2c5d640 1 -- 172.21.15.16:0/1217477613 wait complete. 2024-04-25T10:06:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.185+0000 7ff0a2c5d640 1 Processor -- start 2024-04-25T10:06:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.185+0000 7ff0a2c5d640 1 -- start start 2024-04-25T10:06:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.186+0000 7ff0a2c5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0840039b0 0x7ff09c1f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.186+0000 7ff0a2c5d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09c15fb40 0x7ff09c15ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.186+0000 7ff0a2c5d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09c0c9340 0x7ff09c0c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.186+0000 7ff0a2c5d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0ce920 con 0x7ff0840039b0 2024-04-25T10:06:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.186+0000 7ff0a2c5d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0ce7a0 con 0x7ff09c0c9340 2024-04-25T10:06:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.186+0000 7ff0a2c5d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0ceaa0 con 0x7ff09c15fb40 2024-04-25T10:06:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.186+0000 7ff0a09f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0840039b0 0x7ff09c1f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.186+0000 7ff0a09f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0840039b0 0x7ff09c1f3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51968/0 (socket says 172.21.15.16:51968) 2024-04-25T10:06:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.186+0000 7ff0a09f9640 1 -- 172.21.15.16:0/1068724269 learned_addr learned my addr 172.21.15.16:0/1068724269 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.186+0000 7ff09bfff640 1 --2- 172.21.15.16:0/1068724269 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09c15fb40 0x7ff09c15ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.186+0000 7ff0a11fa640 1 --2- 172.21.15.16:0/1068724269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09c0c9340 0x7ff09c0c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.186+0000 7ff0a09f9640 1 -- 172.21.15.16:0/1068724269 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09c15fb40 msgr2=0x7ff09c15ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.186+0000 7ff0a09f9640 1 --2- 172.21.15.16:0/1068724269 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09c15fb40 0x7ff09c15ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.186+0000 7ff0a09f9640 1 -- 172.21.15.16:0/1068724269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09c0c9340 msgr2=0x7ff09c0c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.186+0000 7ff0a09f9640 1 --2- 172.21.15.16:0/1068724269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09c0c9340 0x7ff09c0c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.186+0000 7ff0a09f9640 1 -- 172.21.15.16:0/1068724269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c0cac40 con 0x7ff0840039b0 2024-04-25T10:06:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.187+0000 7ff0a09f9640 1 --2- 172.21.15.16:0/1068724269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0840039b0 0x7ff09c1f3510 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff08c034040 tx=0x7ff08c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.187+0000 7ff0997fa640 1 -- 172.21.15.16:0/1068724269 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c042070 con 0x7ff0840039b0 2024-04-25T10:06:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.187+0000 7ff0997fa640 1 -- 172.21.15.16:0/1068724269 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c007480 con 0x7ff0840039b0 2024-04-25T10:06:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.187+0000 7ff0a2c5d640 1 -- 172.21.15.16:0/1068724269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09c154700 con 0x7ff0840039b0 2024-04-25T10:06:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.187+0000 7ff0997fa640 1 -- 172.21.15.16:0/1068724269 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c007d50 con 0x7ff0840039b0 2024-04-25T10:06:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.187+0000 7ff0a2c5d640 1 -- 172.21.15.16:0/1068724269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09c1609e0 con 0x7ff0840039b0 2024-04-25T10:06:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.188+0000 7ff0a2c5d640 1 -- 172.21.15.16:0/1068724269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09c0c9d00 con 0x7ff0840039b0 2024-04-25T10:06:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.188+0000 7ff0997fa640 1 -- 172.21.15.16:0/1068724269 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff08c045070 con 0x7ff0840039b0 2024-04-25T10:06:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.189+0000 7ff0997fa640 1 --2- 172.21.15.16:0/1068724269 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff08806b500 0x7ff08806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.189+0000 7ff0997fa640 1 -- 172.21.15.16:0/1068724269 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff08c0aad70 con 0x7ff0840039b0 2024-04-25T10:06:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.189+0000 7ff09bfff640 1 --2- 172.21.15.16:0/1068724269 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff08806b500 0x7ff08806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.189+0000 7ff09bfff640 1 --2- 172.21.15.16:0/1068724269 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff08806b500 0x7ff08806d9c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0d2220 tx=0x7ff090008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.193+0000 7ff0997fa640 1 -- 172.21.15.16:0/1068724269 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff08c04c030 con 0x7ff0840039b0 2024-04-25T10:06:39.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.377+0000 7ff0a2c5d640 1 -- 172.21.15.16:0/1068724269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff09c067850 con 0x7ff0840039b0 2024-04-25T10:06:39.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.378+0000 7ff0997fa640 1 -- 172.21.15.16:0/1068724269 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16609 (secure 0 0 0) 0x7ff08c03c340 con 0x7ff0840039b0 2024-04-25T10:06:39.383 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:39.384 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":36,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:06:12.582767+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:08.830059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:09.922618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.380+0000 7ff0a2c5d640 1 -- 172.21.15.16:0/1068724269 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff08806b500 msgr2=0x7ff08806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.380+0000 7ff0a2c5d640 1 --2- 172.21.15.16:0/1068724269 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff08806b500 0x7ff08806d9c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0d2220 tx=0x7ff090008040 comp rx=0 tx=0).stop 2024-04-25T10:06:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.380+0000 7ff0a2c5d640 1 -- 172.21.15.16:0/1068724269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0840039b0 msgr2=0x7ff09c1f3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.380+0000 7ff0a2c5d640 1 --2- 172.21.15.16:0/1068724269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0840039b0 0x7ff09c1f3510 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff08c034040 tx=0x7ff08c000fe0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.380+0000 7ff0a2c5d640 1 -- 172.21.15.16:0/1068724269 shutdown_connections 2024-04-25T10:06:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.380+0000 7ff0a2c5d640 1 --2- 172.21.15.16:0/1068724269 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff08806b500 0x7ff08806d9c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.380+0000 7ff0a2c5d640 1 --2- 172.21.15.16:0/1068724269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09c0c9340 0x7ff09c0c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.380+0000 7ff0a2c5d640 1 --2- 172.21.15.16:0/1068724269 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09c15fb40 0x7ff09c15ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.380+0000 7ff0a2c5d640 1 --2- 172.21.15.16:0/1068724269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0840039b0 0x7ff09c1f3510 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.380+0000 7ff0a2c5d640 1 -- 172.21.15.16:0/1068724269 >> 172.21.15.16:0/1068724269 conn(0x7ff09c0722d0 msgr2=0x7ff09c15add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.380+0000 7ff0a2c5d640 1 -- 172.21.15.16:0/1068724269 shutdown_connections 2024-04-25T10:06:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.380+0000 7ff0a2c5d640 1 -- 172.21.15.16:0/1068724269 wait complete. 2024-04-25T10:06:39.398 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:06:39.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.524+0000 7f6ecce6f640 1 Processor -- start 2024-04-25T10:06:39.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.524+0000 7f6ecce6f640 1 -- start start 2024-04-25T10:06:39.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.524+0000 7f6ecce6f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ec815dc50 0x7f6ec8070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.524+0000 7f6ecce6f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec8162690 con 0x7f6ec80cd940 2024-04-25T10:06:39.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.524+0000 7f6ecce6f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec806c220 con 0x7f6ec80cf600 2024-04-25T10:06:39.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.524+0000 7f6ecce6f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec80cfe00 con 0x7f6ec815dc50 2024-04-25T10:06:39.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.525+0000 7f6ebffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6ec80cd940 0x7f6ec8161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49182/0 (socket says 172.21.15.16:49182) 2024-04-25T10:06:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.525+0000 7f6ebffff640 1 -- 172.21.15.16:0/2173034891 learned_addr learned my addr 172.21.15.16:0/2173034891 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.525+0000 7f6ec659c640 1 --2- 172.21.15.16:0/2173034891 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ec815dc50 0x7f6ec8070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.525+0000 7f6ec5d9b640 1 -- 172.21.15.16:0/2173034891 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514120305 0 0) 0x7f6ec8162690 con 0x7f6ec80cd940 2024-04-25T10:06:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.525+0000 7f6ec5d9b640 1 -- 172.21.15.16:0/2173034891 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6eb4003520 con 0x7f6ec80cd940 2024-04-25T10:06:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.525+0000 7f6ec5d9b640 1 -- 172.21.15.16:0/2173034891 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824591504 0 0) 0x7f6ec806c220 con 0x7f6ec80cf600 2024-04-25T10:06:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.525+0000 7f6ec5d9b640 1 -- 172.21.15.16:0/2173034891 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ec8162690 con 0x7f6ec80cf600 2024-04-25T10:06:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.525+0000 7f6ec659c640 1 -- 172.21.15.16:0/2173034891 >> v1:172.21.15.186:6789/0 conn(0x7f6ec80cf600 legacy=0x7f6ec8161f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.525+0000 7f6ec659c640 1 -- 172.21.15.16:0/2173034891 >> v1:172.21.15.16:6789/0 conn(0x7f6ec80cd940 legacy=0x7f6ec8161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.525+0000 7f6ec659c640 1 -- 172.21.15.16:0/2173034891 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec80cec10 con 0x7f6ec815dc50 2024-04-25T10:06:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.526+0000 7f6ec659c640 1 --2- 172.21.15.16:0/2173034891 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ec815dc50 0x7f6ec8070e80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80097d0 tx=0x7f6eb802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f078e372df59610b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.526+0000 7f6ec5d9b640 1 -- 172.21.15.16:0/2173034891 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8037070 con 0x7f6ec815dc50 2024-04-25T10:06:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.526+0000 7f6ec5d9b640 1 -- 172.21.15.16:0/2173034891 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb8032440 con 0x7f6ec815dc50 2024-04-25T10:06:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.526+0000 7f6ec5d9b640 1 -- 172.21.15.16:0/2173034891 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb80327f0 con 0x7f6ec815dc50 2024-04-25T10:06:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.526+0000 7f6ecce6f640 1 -- 172.21.15.16:0/2173034891 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ec815dc50 msgr2=0x7f6ec8070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.526+0000 7f6ecce6f640 1 --2- 172.21.15.16:0/2173034891 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ec815dc50 0x7f6ec8070e80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80097d0 tx=0x7f6eb802a210 comp rx=0 tx=0).stop 2024-04-25T10:06:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.527+0000 7f6ecce6f640 1 -- 172.21.15.16:0/2173034891 shutdown_connections 2024-04-25T10:06:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.527+0000 7f6ecce6f640 1 --2- 172.21.15.16:0/2173034891 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ec815dc50 0x7f6ec8070e80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.527+0000 7f6ecce6f640 1 -- 172.21.15.16:0/2173034891 >> 172.21.15.16:0/2173034891 conn(0x7f6ec806f430 msgr2=0x7f6ec806f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.527+0000 7f6ecce6f640 1 -- 172.21.15.16:0/2173034891 shutdown_connections 2024-04-25T10:06:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.527+0000 7f6ecce6f640 1 -- 172.21.15.16:0/2173034891 wait complete. 2024-04-25T10:06:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.528+0000 7f6ecce6f640 1 Processor -- start 2024-04-25T10:06:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.528+0000 7f6ecce6f640 1 -- start start 2024-04-25T10:06:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ecce6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ec80cd940 0x7f6ec815f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ecce6f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ec80cf600 0x7f6ec80b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ecce6f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ec815dc50 0x7f6ec8160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ecce6f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec80cc0c0 con 0x7f6ec80cd940 2024-04-25T10:06:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ecce6f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec80d0330 con 0x7f6ec815dc50 2024-04-25T10:06:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ecce6f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec80ce8a0 con 0x7f6ec80cf600 2024-04-25T10:06:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ec659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ec80cd940 0x7f6ec815f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ec659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ec80cd940 0x7f6ec815f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51978/0 (socket says 172.21.15.16:51978) 2024-04-25T10:06:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ec659c640 1 -- 172.21.15.16:0/856954837 learned_addr learned my addr 172.21.15.16:0/856954837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ebffff640 1 --2- 172.21.15.16:0/856954837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ec80cf600 0x7f6ec80b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ec6d9d640 1 --2- 172.21.15.16:0/856954837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ec815dc50 0x7f6ec8160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ec659c640 1 -- 172.21.15.16:0/856954837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ec80cf600 msgr2=0x7f6ec80b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ec659c640 1 --2- 172.21.15.16:0/856954837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ec80cf600 0x7f6ec80b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ec659c640 1 -- 172.21.15.16:0/856954837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ec815dc50 msgr2=0x7f6ec8160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ec659c640 1 --2- 172.21.15.16:0/856954837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ec815dc50 0x7f6ec8160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ec659c640 1 -- 172.21.15.16:0/856954837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec80b4910 con 0x7f6ec80cd940 2024-04-25T10:06:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ebffff640 1 --2- 172.21.15.16:0/856954837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ec80cf600 0x7f6ec80b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ec6d9d640 1 --2- 172.21.15.16:0/856954837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ec815dc50 0x7f6ec8160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ec659c640 1 --2- 172.21.15.16:0/856954837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ec80cd940 0x7f6ec815f810 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6eb802a250 tx=0x7f6eb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ebeffd640 1 -- 172.21.15.16:0/856954837 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8041020 con 0x7f6ec80cd940 2024-04-25T10:06:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ebeffd640 1 -- 172.21.15.16:0/856954837 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb803d620 con 0x7f6ec80cd940 2024-04-25T10:06:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ebeffd640 1 -- 172.21.15.16:0/856954837 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8037040 con 0x7f6ec80cd940 2024-04-25T10:06:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.529+0000 7f6ecce6f640 1 -- 172.21.15.16:0/856954837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec80c63a0 con 0x7f6ec80cd940 2024-04-25T10:06:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.530+0000 7f6ecce6f640 1 -- 172.21.15.16:0/856954837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec80c6640 con 0x7f6ec80cd940 2024-04-25T10:06:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.530+0000 7f6ecce6f640 1 -- 172.21.15.16:0/856954837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec80750b0 con 0x7f6ec80cd940 2024-04-25T10:06:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.531+0000 7f6ebeffd640 1 -- 172.21.15.16:0/856954837 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6eb8044020 con 0x7f6ec80cd940 2024-04-25T10:06:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.531+0000 7f6ebeffd640 1 --2- 172.21.15.16:0/856954837 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6ea806b4d0 0x7f6ea806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.531+0000 7f6ebeffd640 1 -- 172.21.15.16:0/856954837 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6eb80aa950 con 0x7f6ec80cd940 2024-04-25T10:06:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.532+0000 7f6ebffff640 1 --2- 172.21.15.16:0/856954837 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6ea806b4d0 0x7f6ea806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.532+0000 7f6ebffff640 1 --2- 172.21.15.16:0/856954837 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6ea806b4d0 0x7f6ea806d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80cd3c0 tx=0x7f6eb0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.536+0000 7f6ebeffd640 1 -- 172.21.15.16:0/856954837 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6eb804b030 con 0x7f6ec80cd940 2024-04-25T10:06:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.755+0000 7f6ecce6f640 1 -- 172.21.15.16:0/856954837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6ec80cf3c0 con 0x7f6ec80cd940 2024-04-25T10:06:39.761 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-04-25T10:06:39.756+0000 7ff13f9bb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:06:39.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.757+0000 7f6ebeffd640 1 -- 172.21.15.16:0/856954837 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6eb80797e0 con 0x7f6ec80cd940 2024-04-25T10:06:39.762 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:06:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.760+0000 7f6ecce6f640 1 -- 172.21.15.16:0/856954837 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6ea806b4d0 msgr2=0x7f6ea806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.760+0000 7f6ecce6f640 1 --2- 172.21.15.16:0/856954837 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6ea806b4d0 0x7f6ea806d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80cd3c0 tx=0x7f6eb0009040 comp rx=0 tx=0).stop 2024-04-25T10:06:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.760+0000 7f6ecce6f640 1 -- 172.21.15.16:0/856954837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ec80cd940 msgr2=0x7f6ec815f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.760+0000 7f6ecce6f640 1 --2- 172.21.15.16:0/856954837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ec80cd940 0x7f6ec815f810 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6eb802a250 tx=0x7f6eb80046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.761+0000 7f6ecce6f640 1 -- 172.21.15.16:0/856954837 shutdown_connections 2024-04-25T10:06:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.761+0000 7f6ecce6f640 1 --2- 172.21.15.16:0/856954837 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6ea806b4d0 0x7f6ea806d990 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.761+0000 7f6ecce6f640 1 --2- 172.21.15.16:0/856954837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ec815dc50 0x7f6ec8160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.761+0000 7f6ecce6f640 1 --2- 172.21.15.16:0/856954837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ec80cf600 0x7f6ec80b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.761+0000 7f6ecce6f640 1 --2- 172.21.15.16:0/856954837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ec80cd940 0x7f6ec815f810 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.761+0000 7f6ecce6f640 1 -- 172.21.15.16:0/856954837 >> 172.21.15.16:0/856954837 conn(0x7f6ec806f430 msgr2=0x7f6ec806c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.761+0000 7f6ecce6f640 1 -- 172.21.15.16:0/856954837 shutdown_connections 2024-04-25T10:06:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.761+0000 7f6ecce6f640 1 -- 172.21.15.16:0/856954837 wait complete. 2024-04-25T10:06:39.769 INFO:tasks.ceph.mds.a.smithi016.stderr: -6572> 2024-04-25T10:06:13.892+0000 7fd9788d7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:06:39.778 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:06:39.801 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-04-25T10:06:39.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.876+0000 7f7460b9e640 1 Processor -- start 2024-04-25T10:06:39.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.876+0000 7f7460b9e640 1 -- start start 2024-04-25T10:06:39.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.877+0000 7f7460b9e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f745c15ccb0 0x7f745c0b4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.877+0000 7f7460b9e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f745c070d60 0x7f745c0711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.877+0000 7f7460b9e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f745c071dc0 con 0x7f745c0d0ce0 2024-04-25T10:06:39.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.877+0000 7f7460b9e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f745c0ce490 con 0x7f745c070d60 2024-04-25T10:06:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.877+0000 7f7460b9e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f745c15d090 con 0x7f745c15ccb0 2024-04-25T10:06:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.877+0000 7f7459d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f745c070d60 0x7f745c0711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.877+0000 7f7459d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f745c070d60 0x7f745c0711e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52002/0 (socket says 172.21.15.16:52002) 2024-04-25T10:06:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.877+0000 7f7459d9b640 1 -- 172.21.15.16:0/3424487039 learned_addr learned my addr 172.21.15.16:0/3424487039 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.878+0000 7f745a59c640 1 --2- 172.21.15.16:0/3424487039 >> v2:172.21.15.186:3300/0 conn(0x7f745c15ccb0 0x7f745c0b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.878+0000 7f7459d9b640 1 -- 172.21.15.16:0/3424487039 >> v2:172.21.15.186:3300/0 conn(0x7f745c15ccb0 msgr2=0x7f745c0b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.878+0000 7f7459d9b640 1 --2- 172.21.15.16:0/3424487039 >> v2:172.21.15.186:3300/0 conn(0x7f745c15ccb0 0x7f745c0b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.878+0000 7f7459d9b640 1 -- 172.21.15.16:0/3424487039 >> v1:172.21.15.186:6789/0 conn(0x7f745c0d0ce0 legacy=0x7f745c0d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.878+0000 7f7459d9b640 1 -- 172.21.15.16:0/3424487039 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c0b4d40 con 0x7f745c070d60 2024-04-25T10:06:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.878+0000 7f7459d9b640 1 --2- 172.21.15.16:0/3424487039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f745c070d60 0x7f745c0711e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f745c0709e0 tx=0x7f745002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c594bb0eb8b9d8d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.879+0000 7f745959a640 1 -- 172.21.15.16:0/3424487039 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450037070 con 0x7f745c070d60 2024-04-25T10:06:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.879+0000 7f745959a640 1 -- 172.21.15.16:0/3424487039 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7450032440 con 0x7f745c070d60 2024-04-25T10:06:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.879+0000 7f745959a640 1 -- 172.21.15.16:0/3424487039 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450032810 con 0x7f745c070d60 2024-04-25T10:06:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.879+0000 7f7460b9e640 1 -- 172.21.15.16:0/3424487039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f745c070d60 msgr2=0x7f745c0711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.879+0000 7f7460b9e640 1 --2- 172.21.15.16:0/3424487039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f745c070d60 0x7f745c0711e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f745c0709e0 tx=0x7f745002a150 comp rx=0 tx=0).stop 2024-04-25T10:06:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.879+0000 7f7460b9e640 1 -- 172.21.15.16:0/3424487039 shutdown_connections 2024-04-25T10:06:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.879+0000 7f7460b9e640 1 --2- 172.21.15.16:0/3424487039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f745c070d60 0x7f745c0711e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.879+0000 7f7460b9e640 1 --2- 172.21.15.16:0/3424487039 >> v2:172.21.15.186:3300/0 conn(0x7f745c15ccb0 0x7f745c0b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.879+0000 7f7460b9e640 1 -- 172.21.15.16:0/3424487039 >> 172.21.15.16:0/3424487039 conn(0x7f745c0b3500 msgr2=0x7f745c0b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.879+0000 7f7460b9e640 1 -- 172.21.15.16:0/3424487039 shutdown_connections 2024-04-25T10:06:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.879+0000 7f7460b9e640 1 -- 172.21.15.16:0/3424487039 wait complete. 2024-04-25T10:06:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.881+0000 7f7460b9e640 1 Processor -- start 2024-04-25T10:06:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.881+0000 7f7460b9e640 1 -- start start 2024-04-25T10:06:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.881+0000 7f7460b9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f745c070d60 0x7f745c155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.881+0000 7f7460b9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f745c0d0ce0 0x7f745c1f1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.881+0000 7f7460b9e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f745c15ccb0 0x7f745c0c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.881+0000 7f7460b9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c0ce730 con 0x7f745c0d0ce0 2024-04-25T10:06:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.881+0000 7f7460b9e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c071ae0 con 0x7f745c15ccb0 2024-04-25T10:06:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.881+0000 7f7460b9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745c15c3a0 con 0x7f745c070d60 2024-04-25T10:06:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.881+0000 7f7459d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f745c0d0ce0 0x7f745c1f1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.882+0000 7f7459d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f745c0d0ce0 0x7f745c1f1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52014/0 (socket says 172.21.15.16:52014) 2024-04-25T10:06:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.882+0000 7f7459d9b640 1 -- 172.21.15.16:0/3701204572 learned_addr learned my addr 172.21.15.16:0/3701204572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.882+0000 7f745a59c640 1 --2- 172.21.15.16:0/3701204572 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f745c070d60 0x7f745c155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.882+0000 7f745ad9d640 1 --2- 172.21.15.16:0/3701204572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f745c15ccb0 0x7f745c0c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.882+0000 7f7459d9b640 1 -- 172.21.15.16:0/3701204572 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f745c070d60 msgr2=0x7f745c155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.882+0000 7f7459d9b640 1 --2- 172.21.15.16:0/3701204572 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f745c070d60 0x7f745c155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.882+0000 7f7459d9b640 1 -- 172.21.15.16:0/3701204572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f745c15ccb0 msgr2=0x7f745c0c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.882+0000 7f7459d9b640 1 --2- 172.21.15.16:0/3701204572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f745c15ccb0 0x7f745c0c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.882+0000 7f7459d9b640 1 -- 172.21.15.16:0/3701204572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c155a80 con 0x7f745c0d0ce0 2024-04-25T10:06:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.882+0000 7f745a59c640 1 --2- 172.21.15.16:0/3701204572 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f745c070d60 0x7f745c155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.882+0000 7f7459d9b640 1 --2- 172.21.15.16:0/3701204572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f745c0d0ce0 0x7f745c1f1670 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7450000c00 tx=0x7f74500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.882+0000 7f744affd640 1 -- 172.21.15.16:0/3701204572 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450041070 con 0x7f745c0d0ce0 2024-04-25T10:06:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.882+0000 7f744affd640 1 -- 172.21.15.16:0/3701204572 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745003c870 con 0x7f745c0d0ce0 2024-04-25T10:06:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.882+0000 7f744affd640 1 -- 172.21.15.16:0/3701204572 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450037040 con 0x7f745c0d0ce0 2024-04-25T10:06:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.882+0000 7f7460b9e640 1 -- 172.21.15.16:0/3701204572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f745c1f2050 con 0x7f745c0d0ce0 2024-04-25T10:06:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.882+0000 7f7460b9e640 1 -- 172.21.15.16:0/3701204572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745c0c68e0 con 0x7f745c0d0ce0 2024-04-25T10:06:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.883+0000 7f7460b9e640 1 -- 172.21.15.16:0/3701204572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745c066c70 con 0x7f745c0d0ce0 2024-04-25T10:06:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.884+0000 7f744affd640 1 -- 172.21.15.16:0/3701204572 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7450044070 con 0x7f745c0d0ce0 2024-04-25T10:06:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.884+0000 7f744affd640 1 --2- 172.21.15.16:0/3701204572 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f743c06b4d0 0x7f743c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.884+0000 7f744affd640 1 -- 172.21.15.16:0/3701204572 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f74500aabd0 con 0x7f745c0d0ce0 2024-04-25T10:06:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.884+0000 7f745a59c640 1 --2- 172.21.15.16:0/3701204572 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f743c06b4d0 0x7f743c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.885+0000 7f745a59c640 1 --2- 172.21.15.16:0/3701204572 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f743c06b4d0 0x7f743c06d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7454006fd0 tx=0x7f7454008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:39.889+0000 7f744affd640 1 -- 172.21.15.16:0/3701204572 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f745c066c70 con 0x7f745c0d0ce0 2024-04-25T10:06:40.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.107+0000 7f7460b9e640 1 -- 172.21.15.16:0/3701204572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f745c0b1000 con 0x7f745c0d0ce0 2024-04-25T10:06:40.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.761+0000 7f744affd640 1 -- 172.21.15.16:0/3701204572 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7450079550 con 0x7f745c0d0ce0 2024-04-25T10:06:40.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.763+0000 7f7460b9e640 1 -- 172.21.15.16:0/3701204572 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f743c06b4d0 msgr2=0x7f743c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.763+0000 7f7460b9e640 1 --2- 172.21.15.16:0/3701204572 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f743c06b4d0 0x7f743c06d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7454006fd0 tx=0x7f7454008040 comp rx=0 tx=0).stop 2024-04-25T10:06:40.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.763+0000 7f7460b9e640 1 -- 172.21.15.16:0/3701204572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f745c0d0ce0 msgr2=0x7f745c1f1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.763+0000 7f7460b9e640 1 --2- 172.21.15.16:0/3701204572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f745c0d0ce0 0x7f745c1f1670 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7450000c00 tx=0x7f74500046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.764+0000 7f7460b9e640 1 -- 172.21.15.16:0/3701204572 shutdown_connections 2024-04-25T10:06:40.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.764+0000 7f7460b9e640 1 --2- 172.21.15.16:0/3701204572 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f743c06b4d0 0x7f743c06d990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.764+0000 7f7460b9e640 1 --2- 172.21.15.16:0/3701204572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f745c15ccb0 0x7f745c0c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.764+0000 7f7460b9e640 1 --2- 172.21.15.16:0/3701204572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f745c0d0ce0 0x7f745c1f1670 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.764+0000 7f7460b9e640 1 --2- 172.21.15.16:0/3701204572 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f745c070d60 0x7f745c155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.764+0000 7f7460b9e640 1 -- 172.21.15.16:0/3701204572 >> 172.21.15.16:0/3701204572 conn(0x7f745c0b3500 msgr2=0x7f745c0cf150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.764+0000 7f7460b9e640 1 -- 172.21.15.16:0/3701204572 shutdown_connections 2024-04-25T10:06:40.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.764+0000 7f7460b9e640 1 -- 172.21.15.16:0/3701204572 wait complete. 2024-04-25T10:06:40.783 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:06:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.931+0000 7ff21d13b640 1 Processor -- start 2024-04-25T10:06:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.931+0000 7ff21d13b640 1 -- start start 2024-04-25T10:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.932+0000 7ff21d13b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff2181610b0 0x7ff2180b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.932+0000 7ff21d13b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2181618d0 0x7ff2180ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.932+0000 7ff21d13b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff218162040 con 0x7ff2180ced70 2024-04-25T10:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.932+0000 7ff21d13b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2180d1ea0 con 0x7ff2181610b0 2024-04-25T10:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.932+0000 7ff21d13b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff218162320 con 0x7ff2181618d0 2024-04-25T10:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.932+0000 7ff21759e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff2180ced70 0x7ff21806c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49200/0 (socket says 172.21.15.16:49200) 2024-04-25T10:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.932+0000 7ff21759e640 1 -- 172.21.15.16:0/816101366 learned_addr learned my addr 172.21.15.16:0/816101366 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.932+0000 7ff216d9d640 1 --2- 172.21.15.16:0/816101366 >> v2:172.21.15.186:3300/0 conn(0x7ff2181610b0 0x7ff2180b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.932+0000 7ff21659c640 1 --2- 172.21.15.16:0/816101366 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2181618d0 0x7ff2180ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.932+0000 7ff215d9b640 1 -- 172.21.15.16:0/816101366 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3401524364 0 0) 0x7ff218162040 con 0x7ff2180ced70 2024-04-25T10:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.932+0000 7ff215d9b640 1 -- 172.21.15.16:0/816101366 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff200003520 con 0x7ff2180ced70 2024-04-25T10:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff215d9b640 1 -- 172.21.15.16:0/816101366 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 51127847 0 0) 0x7ff200003520 con 0x7ff2180ced70 2024-04-25T10:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff215d9b640 1 -- 172.21.15.16:0/816101366 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2181618d0 msgr2=0x7ff2180ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff215d9b640 1 --2- 172.21.15.16:0/816101366 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2181618d0 0x7ff2180ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff215d9b640 1 -- 172.21.15.16:0/816101366 >> v2:172.21.15.186:3300/0 conn(0x7ff2181610b0 msgr2=0x7ff2180b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff215d9b640 1 --2- 172.21.15.16:0/816101366 >> v2:172.21.15.186:3300/0 conn(0x7ff2181610b0 0x7ff2180b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff215d9b640 1 -- 172.21.15.16:0/816101366 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff218162600 con 0x7ff2180ced70 2024-04-25T10:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff215d9b640 1 -- 172.21.15.16:0/816101366 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7ff2080019f0 con 0x7ff2180ced70 2024-04-25T10:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff215d9b640 1 -- 172.21.15.16:0/816101366 >> v1:172.21.15.16:6789/0 conn(0x7ff2180ced70 legacy=0x7ff21806c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff215d9b640 1 --2- 172.21.15.16:0/816101366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2000039b0 0x7ff200003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff215d9b640 1 --2- 172.21.15.16:0/816101366 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2181618d0 0x7ff200004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff215d9b640 1 --2- 172.21.15.16:0/816101366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff200004d40 0x7ff2000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff215d9b640 1 -- 172.21.15.16:0/816101366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180d1ea0 con 0x7ff2000039b0 2024-04-25T10:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff215d9b640 1 -- 172.21.15.16:0/816101366 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff218162320 con 0x7ff200004d40 2024-04-25T10:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff215d9b640 1 -- 172.21.15.16:0/816101366 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20000b9c0 con 0x7ff2181618d0 2024-04-25T10:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff216d9d640 1 --2- 172.21.15.16:0/816101366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2000039b0 0x7ff200003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff21759e640 1 --2- 172.21.15.16:0/816101366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff200004d40 0x7ff2000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff21659c640 1 --2- 172.21.15.16:0/816101366 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2181618d0 0x7ff200004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff216d9d640 1 -- 172.21.15.16:0/816101366 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2181618d0 msgr2=0x7ff200004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff216d9d640 1 --2- 172.21.15.16:0/816101366 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2181618d0 0x7ff200004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff216d9d640 1 -- 172.21.15.16:0/816101366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff200004d40 msgr2=0x7ff2000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff216d9d640 1 --2- 172.21.15.16:0/816101366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff200004d40 0x7ff2000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.933+0000 7ff216d9d640 1 -- 172.21.15.16:0/816101366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff200003520 con 0x7ff2000039b0 2024-04-25T10:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.934+0000 7ff216d9d640 1 --2- 172.21.15.16:0/816101366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2000039b0 0x7ff200003dd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff210009ac0 tx=0x7ff200005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfea3e7657af83cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.934+0000 7ff215d9b640 1 -- 172.21.15.16:0/816101366 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff210036070 con 0x7ff2000039b0 2024-04-25T10:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.934+0000 7ff215d9b640 1 -- 172.21.15.16:0/816101366 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff210031400 con 0x7ff2000039b0 2024-04-25T10:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.934+0000 7ff215d9b640 1 -- 172.21.15.16:0/816101366 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff210009690 con 0x7ff2000039b0 2024-04-25T10:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.935+0000 7ff21d13b640 1 -- 172.21.15.16:0/816101366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2000039b0 msgr2=0x7ff200003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.935+0000 7ff21d13b640 1 --2- 172.21.15.16:0/816101366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2000039b0 0x7ff200003dd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff210009ac0 tx=0x7ff200005830 comp rx=0 tx=0).stop 2024-04-25T10:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.935+0000 7ff21759e640 1 -- 172.21.15.16:0/816101366 reap_dead start 2024-04-25T10:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.935+0000 7ff21d13b640 1 -- 172.21.15.16:0/816101366 shutdown_connections 2024-04-25T10:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.935+0000 7ff21d13b640 1 -- 172.21.15.16:0/816101366 >> 172.21.15.16:0/816101366 conn(0x7ff2180b5570 msgr2=0x7ff2180b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.935+0000 7ff21d13b640 1 -- 172.21.15.16:0/816101366 shutdown_connections 2024-04-25T10:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.935+0000 7ff21d13b640 1 -- 172.21.15.16:0/816101366 wait complete. 2024-04-25T10:06:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.936+0000 7ff21d13b640 1 Processor -- start 2024-04-25T10:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.936+0000 7ff21d13b640 1 -- start start 2024-04-25T10:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.937+0000 7ff21d13b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2000039b0 0x7ff21814b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.937+0000 7ff21d13b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff218152790 0x7ff218152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.937+0000 7ff21d13b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2180cc150 0x7ff2180cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.937+0000 7ff21d13b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff218161d10 con 0x7ff2180cc150 2024-04-25T10:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.937+0000 7ff21d13b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180b6c00 con 0x7ff2000039b0 2024-04-25T10:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.937+0000 7ff21d13b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180d3620 con 0x7ff218152790 2024-04-25T10:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.937+0000 7ff21759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2180cc150 0x7ff2180cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.937+0000 7ff21759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2180cc150 0x7ff2180cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52044/0 (socket says 172.21.15.16:52044) 2024-04-25T10:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.937+0000 7ff21759e640 1 -- 172.21.15.16:0/2290516376 learned_addr learned my addr 172.21.15.16:0/2290516376 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.937+0000 7ff21659c640 1 --2- 172.21.15.16:0/2290516376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff218152790 0x7ff218152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.937+0000 7ff216d9d640 1 --2- 172.21.15.16:0/2290516376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2000039b0 0x7ff21814b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.938+0000 7ff21759e640 1 -- 172.21.15.16:0/2290516376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff218152790 msgr2=0x7ff218152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.938+0000 7ff21759e640 1 --2- 172.21.15.16:0/2290516376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff218152790 0x7ff218152c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.938+0000 7ff21759e640 1 -- 172.21.15.16:0/2290516376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2000039b0 msgr2=0x7ff21814b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.938+0000 7ff21759e640 1 --2- 172.21.15.16:0/2290516376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2000039b0 0x7ff21814b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.938+0000 7ff21759e640 1 -- 172.21.15.16:0/2290516376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff218153b90 con 0x7ff2180cc150 2024-04-25T10:06:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.938+0000 7ff216d9d640 1 --2- 172.21.15.16:0/2290516376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2000039b0 0x7ff21814b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.938+0000 7ff21759e640 1 --2- 172.21.15.16:0/2290516376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2180cc150 0x7ff2180cc5d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff2180cfb90 tx=0x7ff208007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.938+0000 7ff1ff7fe640 1 -- 172.21.15.16:0/2290516376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20800ab20 con 0x7ff2180cc150 2024-04-25T10:06:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.938+0000 7ff1ff7fe640 1 -- 172.21.15.16:0/2290516376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff208018070 con 0x7ff2180cc150 2024-04-25T10:06:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.938+0000 7ff1ff7fe640 1 -- 172.21.15.16:0/2290516376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2080133f0 con 0x7ff2180cc150 2024-04-25T10:06:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.938+0000 7ff21d13b640 1 -- 172.21.15.16:0/2290516376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff218153350 con 0x7ff2180cc150 2024-04-25T10:06:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.938+0000 7ff21d13b640 1 -- 172.21.15.16:0/2290516376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff218153e50 con 0x7ff2180cc150 2024-04-25T10:06:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.939+0000 7ff21d13b640 1 -- 172.21.15.16:0/2290516376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2180750b0 con 0x7ff2180cc150 2024-04-25T10:06:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.939+0000 7ff1ff7fe640 1 -- 172.21.15.16:0/2290516376 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff208013590 con 0x7ff2180cc150 2024-04-25T10:06:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.940+0000 7ff1ff7fe640 1 --2- 172.21.15.16:0/2290516376 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff20406b4d0 0x7ff20406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.940+0000 7ff1ff7fe640 1 -- 172.21.15.16:0/2290516376 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff20808a750 con 0x7ff2180cc150 2024-04-25T10:06:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.940+0000 7ff216d9d640 1 --2- 172.21.15.16:0/2290516376 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff20406b4d0 0x7ff20406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.941+0000 7ff216d9d640 1 --2- 172.21.15.16:0/2290516376 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff20406b4d0 0x7ff20406d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff210033000 tx=0x7ff210008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:40.944+0000 7ff1ff7fe640 1 -- 172.21.15.16:0/2290516376 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff208059570 con 0x7ff2180cc150 2024-04-25T10:06:41.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.124+0000 7ff21d13b640 1 -- 172.21.15.16:0/2290516376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff2180d1770 con 0x7ff2180cc150 2024-04-25T10:06:41.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.762+0000 7ff1ff7fe640 1 -- 172.21.15.16:0/2290516376 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff208058e70 con 0x7ff2180cc150 2024-04-25T10:06:41.768 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:06:41.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.766+0000 7ff21d13b640 1 -- 172.21.15.16:0/2290516376 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff20406b4d0 msgr2=0x7ff20406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.766+0000 7ff21d13b640 1 --2- 172.21.15.16:0/2290516376 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff20406b4d0 0x7ff20406d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff210033000 tx=0x7ff210008030 comp rx=0 tx=0).stop 2024-04-25T10:06:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.766+0000 7ff21d13b640 1 -- 172.21.15.16:0/2290516376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2180cc150 msgr2=0x7ff2180cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.766+0000 7ff21d13b640 1 --2- 172.21.15.16:0/2290516376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2180cc150 0x7ff2180cc5d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff2180cfb90 tx=0x7ff208007a60 comp rx=0 tx=0).stop 2024-04-25T10:06:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.766+0000 7ff21d13b640 1 -- 172.21.15.16:0/2290516376 shutdown_connections 2024-04-25T10:06:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.766+0000 7ff21d13b640 1 --2- 172.21.15.16:0/2290516376 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff20406b4d0 0x7ff20406d990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.766+0000 7ff21d13b640 1 --2- 172.21.15.16:0/2290516376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2180cc150 0x7ff2180cc5d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.766+0000 7ff21d13b640 1 --2- 172.21.15.16:0/2290516376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff218152790 0x7ff218152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.766+0000 7ff21d13b640 1 --2- 172.21.15.16:0/2290516376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2000039b0 0x7ff21814b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.766+0000 7ff21d13b640 1 -- 172.21.15.16:0/2290516376 >> 172.21.15.16:0/2290516376 conn(0x7ff2180b5570 msgr2=0x7ff2180d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.767+0000 7ff21d13b640 1 -- 172.21.15.16:0/2290516376 shutdown_connections 2024-04-25T10:06:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.767+0000 7ff21d13b640 1 -- 172.21.15.16:0/2290516376 wait complete. 2024-04-25T10:06:41.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:06:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.894+0000 7f019f63f640 1 Processor -- start 2024-04-25T10:06:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.894+0000 7f019f63f640 1 -- start start 2024-04-25T10:06:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f019f63f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01980b9580 0x7f01981618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f019f63f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0198162040 con 0x7f01980cef40 2024-04-25T10:06:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f019f63f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0198162320 con 0x7f01981610b0 2024-04-25T10:06:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f019f63f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01980b9a30 con 0x7f01980b9580 2024-04-25T10:06:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f019cbda640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f01980cef40 0x7f01980d2610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49214/0 (socket says 172.21.15.16:49214) 2024-04-25T10:06:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f019cbda640 1 -- 172.21.15.16:0/793504593 learned_addr learned my addr 172.21.15.16:0/793504593 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f018ffff640 1 -- 172.21.15.16:0/793504593 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4134006817 0 0) 0x7f0198162040 con 0x7f01980cef40 2024-04-25T10:06:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f018ffff640 1 -- 172.21.15.16:0/793504593 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0184003520 con 0x7f01980cef40 2024-04-25T10:06:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f019dbdc640 1 --2- 172.21.15.16:0/793504593 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01980b9580 0x7f01981618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f018ffff640 1 -- 172.21.15.16:0/793504593 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1521235021 0 0) 0x7f0184003520 con 0x7f01980cef40 2024-04-25T10:06:41.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f018ffff640 1 -- 172.21.15.16:0/793504593 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01980b9580 msgr2=0x7f01981618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f018ffff640 1 --2- 172.21.15.16:0/793504593 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01980b9580 0x7f01981618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f018ffff640 1 -- 172.21.15.16:0/793504593 >> v1:172.21.15.186:6789/0 conn(0x7f01981610b0 legacy=0x7f01980ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f018ffff640 1 -- 172.21.15.16:0/793504593 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0198162600 con 0x7f01980cef40 2024-04-25T10:06:41.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f018ffff640 1 -- 172.21.15.16:0/793504593 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f0188001670 con 0x7f01980cef40 2024-04-25T10:06:41.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f018ffff640 1 -- 172.21.15.16:0/793504593 >> v1:172.21.15.16:6789/0 conn(0x7f01980cef40 legacy=0x7f01980d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f018ffff640 1 --2- 172.21.15.16:0/793504593 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01980b9580 0x7f01840039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f018ffff640 1 --2- 172.21.15.16:0/793504593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0184003f60 0x7f0184004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f018ffff640 1 --2- 172.21.15.16:0/793504593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0184004d40 0x7f01840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f018ffff640 1 -- 172.21.15.16:0/793504593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01980b9a30 con 0x7f0184003f60 2024-04-25T10:06:41.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f018ffff640 1 -- 172.21.15.16:0/793504593 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018400bb50 con 0x7f0184004d40 2024-04-25T10:06:41.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.895+0000 7f018ffff640 1 -- 172.21.15.16:0/793504593 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018400bd50 con 0x7f01980b9580 2024-04-25T10:06:41.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.896+0000 7f019cbda640 1 --2- 172.21.15.16:0/793504593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0184003f60 0x7f0184004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.896+0000 7f019d3db640 1 --2- 172.21.15.16:0/793504593 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01980b9580 0x7f01840039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.896+0000 7f019dbdc640 1 --2- 172.21.15.16:0/793504593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0184004d40 0x7f01840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.896+0000 7f019cbda640 1 -- 172.21.15.16:0/793504593 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01980b9580 msgr2=0x7f01840039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.896+0000 7f019cbda640 1 --2- 172.21.15.16:0/793504593 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01980b9580 0x7f01840039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.896+0000 7f019cbda640 1 -- 172.21.15.16:0/793504593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0184004d40 msgr2=0x7f01840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.896+0000 7f019cbda640 1 --2- 172.21.15.16:0/793504593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0184004d40 0x7f01840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.896+0000 7f019cbda640 1 -- 172.21.15.16:0/793504593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0184003520 con 0x7f0184003f60 2024-04-25T10:06:41.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.897+0000 7f019cbda640 1 --2- 172.21.15.16:0/793504593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0184003f60 0x7f0184004800 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f018800c240 tx=0x7f0184005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2136f4cab9270767 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.897+0000 7f018ffff640 1 -- 172.21.15.16:0/793504593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188007d10 con 0x7f0184003f60 2024-04-25T10:06:41.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.897+0000 7f018ffff640 1 -- 172.21.15.16:0/793504593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0188005ad0 con 0x7f0184003f60 2024-04-25T10:06:41.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.897+0000 7f018ffff640 1 -- 172.21.15.16:0/793504593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188007070 con 0x7f0184003f60 2024-04-25T10:06:41.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.897+0000 7f019f63f640 1 -- 172.21.15.16:0/793504593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0184003f60 msgr2=0x7f0184004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.897+0000 7f019f63f640 1 --2- 172.21.15.16:0/793504593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0184003f60 0x7f0184004800 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f018800c240 tx=0x7f0184005830 comp rx=0 tx=0).stop 2024-04-25T10:06:41.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.897+0000 7f019dbdc640 1 -- 172.21.15.16:0/793504593 reap_dead start 2024-04-25T10:06:41.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.897+0000 7f019f63f640 1 -- 172.21.15.16:0/793504593 shutdown_connections 2024-04-25T10:06:41.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.897+0000 7f019f63f640 1 -- 172.21.15.16:0/793504593 >> 172.21.15.16:0/793504593 conn(0x7f01980b5570 msgr2=0x7f01980b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.897+0000 7f019f63f640 1 -- 172.21.15.16:0/793504593 shutdown_connections 2024-04-25T10:06:41.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.897+0000 7f019f63f640 1 -- 172.21.15.16:0/793504593 wait complete. 2024-04-25T10:06:41.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.899+0000 7f019f63f640 1 Processor -- start 2024-04-25T10:06:41.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.899+0000 7f019f63f640 1 -- start start 2024-04-25T10:06:41.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.899+0000 7f019f63f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0184003f60 0x7f01980d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.899+0000 7f019f63f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01981f2ef0 0x7f01981f3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.899+0000 7f019f63f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01980cbcb0 0x7f01980cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.899+0000 7f019f63f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01980b6c00 con 0x7f01980cbcb0 2024-04-25T10:06:41.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.899+0000 7f019f63f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01980d1ea0 con 0x7f01981f2ef0 2024-04-25T10:06:41.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.899+0000 7f019f63f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0198161490 con 0x7f0184003f60 2024-04-25T10:06:41.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.900+0000 7f019dbdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01980cbcb0 0x7f01980cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.900+0000 7f019dbdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01980cbcb0 0x7f01980cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52064/0 (socket says 172.21.15.16:52064) 2024-04-25T10:06:41.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.900+0000 7f019dbdc640 1 -- 172.21.15.16:0/2985405451 learned_addr learned my addr 172.21.15.16:0/2985405451 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:41.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.900+0000 7f019cbda640 1 --2- 172.21.15.16:0/2985405451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01981f2ef0 0x7f01981f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.900+0000 7f019d3db640 1 --2- 172.21.15.16:0/2985405451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0184003f60 0x7f01980d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.900+0000 7f019dbdc640 1 -- 172.21.15.16:0/2985405451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0184003f60 msgr2=0x7f01980d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.900+0000 7f019dbdc640 1 --2- 172.21.15.16:0/2985405451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0184003f60 0x7f01980d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.900+0000 7f019dbdc640 1 -- 172.21.15.16:0/2985405451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01981f2ef0 msgr2=0x7f01981f3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.900+0000 7f019dbdc640 1 --2- 172.21.15.16:0/2985405451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01981f2ef0 0x7f01981f3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.900+0000 7f019dbdc640 1 -- 172.21.15.16:0/2985405451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01981f3a40 con 0x7f01980cbcb0 2024-04-25T10:06:41.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.900+0000 7f019d3db640 1 --2- 172.21.15.16:0/2985405451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0184003f60 0x7f01980d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:41.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.900+0000 7f019dbdc640 1 --2- 172.21.15.16:0/2985405451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01980cbcb0 0x7f01980cc130 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0190011bf0 tx=0x7f019000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.900+0000 7f018dffb640 1 -- 172.21.15.16:0/2985405451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019000fe10 con 0x7f01980cbcb0 2024-04-25T10:06:41.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.900+0000 7f018dffb640 1 -- 172.21.15.16:0/2985405451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019001d070 con 0x7f01980cbcb0 2024-04-25T10:06:41.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.900+0000 7f019f63f640 1 -- 172.21.15.16:0/2985405451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01981f3d20 con 0x7f01980cbcb0 2024-04-25T10:06:41.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.900+0000 7f018dffb640 1 -- 172.21.15.16:0/2985405451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190015670 con 0x7f01980cbcb0 2024-04-25T10:06:41.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.901+0000 7f019f63f640 1 -- 172.21.15.16:0/2985405451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01981f42e0 con 0x7f01980cbcb0 2024-04-25T10:06:41.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.902+0000 7f019f63f640 1 -- 172.21.15.16:0/2985405451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01980750b0 con 0x7f01980cbcb0 2024-04-25T10:06:41.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.902+0000 7f018dffb640 1 -- 172.21.15.16:0/2985405451 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0190015810 con 0x7f01980cbcb0 2024-04-25T10:06:41.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.902+0000 7f018dffb640 1 --2- 172.21.15.16:0/2985405451 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f017c06b500 0x7f017c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.902+0000 7f018dffb640 1 -- 172.21.15.16:0/2985405451 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f019008a8a0 con 0x7f01980cbcb0 2024-04-25T10:06:41.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.903+0000 7f019d3db640 1 --2- 172.21.15.16:0/2985405451 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f017c06b500 0x7f017c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.903+0000 7f019d3db640 1 --2- 172.21.15.16:0/2985405451 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f017c06b500 0x7f017c06d9c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f01940036b0 tx=0x7f0194009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:41.907+0000 7f018dffb640 1 -- 172.21.15.16:0/2985405451 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0190010040 con 0x7f01980cbcb0 2024-04-25T10:06:42.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.086+0000 7f019f63f640 1 -- 172.21.15.16:0/2985405451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f01980d1770 con 0x7f01980cbcb0 2024-04-25T10:06:42.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.767+0000 7f018dffb640 1 -- 172.21.15.16:0/2985405451 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0190059880 con 0x7f01980cbcb0 2024-04-25T10:06:42.773 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-04-25T10:06:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.773+0000 7f019f63f640 1 -- 172.21.15.16:0/2985405451 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f017c06b500 msgr2=0x7f017c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.773+0000 7f019f63f640 1 --2- 172.21.15.16:0/2985405451 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f017c06b500 0x7f017c06d9c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f01940036b0 tx=0x7f0194009040 comp rx=0 tx=0).stop 2024-04-25T10:06:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.773+0000 7f019f63f640 1 -- 172.21.15.16:0/2985405451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01980cbcb0 msgr2=0x7f01980cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.773+0000 7f019f63f640 1 --2- 172.21.15.16:0/2985405451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01980cbcb0 0x7f01980cc130 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0190011bf0 tx=0x7f019000f580 comp rx=0 tx=0).stop 2024-04-25T10:06:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.774+0000 7f019f63f640 1 -- 172.21.15.16:0/2985405451 shutdown_connections 2024-04-25T10:06:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.774+0000 7f019f63f640 1 --2- 172.21.15.16:0/2985405451 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f017c06b500 0x7f017c06d9c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.774+0000 7f019f63f640 1 --2- 172.21.15.16:0/2985405451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01980cbcb0 0x7f01980cc130 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.774+0000 7f019f63f640 1 --2- 172.21.15.16:0/2985405451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01981f2ef0 0x7f01981f3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.774+0000 7f019f63f640 1 --2- 172.21.15.16:0/2985405451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0184003f60 0x7f01980d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.774+0000 7f019f63f640 1 -- 172.21.15.16:0/2985405451 >> 172.21.15.16:0/2985405451 conn(0x7f01980b5570 msgr2=0x7f01980ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.774+0000 7f019f63f640 1 -- 172.21.15.16:0/2985405451 shutdown_connections 2024-04-25T10:06:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.774+0000 7f019f63f640 1 -- 172.21.15.16:0/2985405451 wait complete. 2024-04-25T10:06:42.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:06:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.899+0000 7fc0a2e91640 1 Processor -- start 2024-04-25T10:06:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.899+0000 7fc0a2e91640 1 -- start start 2024-04-25T10:06:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.900+0000 7fc0a2e91640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc09c15dca0 0x7fc09c0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.900+0000 7fc0a2e91640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc09c0d1130 0x7fc09c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.900+0000 7fc0a2e91640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc09c0d4090 con 0x7fc09c0d3b90 2024-04-25T10:06:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.900+0000 7fc0a2e91640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc09c0b6b20 con 0x7fc09c15dca0 2024-04-25T10:06:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.900+0000 7fc0a2e91640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc09c0d4370 con 0x7fc09c0d1130 2024-04-25T10:06:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.900+0000 7fc0a0c2d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc09c15dca0 0x7fc09c0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.900+0000 7fc0a0c2d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc09c15dca0 0x7fc09c0ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55968/0 (socket says 172.21.15.16:55968) 2024-04-25T10:06:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.900+0000 7fc0a0c2d640 1 -- 172.21.15.16:0/3069929506 learned_addr learned my addr 172.21.15.16:0/3069929506 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.900+0000 7fc0a142e640 1 --2- 172.21.15.16:0/3069929506 >> v2:172.21.15.186:3300/0 conn(0x7fc09c0d1130 0x7fc09c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.900+0000 7fc0a0c2d640 1 -- 172.21.15.16:0/3069929506 >> v2:172.21.15.186:3300/0 conn(0x7fc09c0d1130 msgr2=0x7fc09c0b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.900+0000 7fc0a0c2d640 1 --2- 172.21.15.16:0/3069929506 >> v2:172.21.15.186:3300/0 conn(0x7fc09c0d1130 0x7fc09c0b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.900+0000 7fc0a0c2d640 1 -- 172.21.15.16:0/3069929506 >> v1:172.21.15.186:6789/0 conn(0x7fc09c0d3b90 legacy=0x7fc09c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.900+0000 7fc0a0c2d640 1 -- 172.21.15.16:0/3069929506 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09c0d4650 con 0x7fc09c15dca0 2024-04-25T10:06:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.901+0000 7fc0a0c2d640 1 --2- 172.21.15.16:0/3069929506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc09c15dca0 0x7fc09c0ce940 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0b9640 tx=0x7fc09002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebbc2e6b7034f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.901+0000 7fc09b7fe640 1 -- 172.21.15.16:0/3069929506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090037070 con 0x7fc09c15dca0 2024-04-25T10:06:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.901+0000 7fc09b7fe640 1 -- 172.21.15.16:0/3069929506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc090032440 con 0x7fc09c15dca0 2024-04-25T10:06:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.901+0000 7fc09b7fe640 1 -- 172.21.15.16:0/3069929506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090032810 con 0x7fc09c15dca0 2024-04-25T10:06:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.902+0000 7fc0a2e91640 1 -- 172.21.15.16:0/3069929506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc09c15dca0 msgr2=0x7fc09c0ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.902+0000 7fc0a2e91640 1 --2- 172.21.15.16:0/3069929506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc09c15dca0 0x7fc09c0ce940 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0b9640 tx=0x7fc09002a230 comp rx=0 tx=0).stop 2024-04-25T10:06:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.902+0000 7fc0a2e91640 1 -- 172.21.15.16:0/3069929506 shutdown_connections 2024-04-25T10:06:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.902+0000 7fc0a2e91640 1 --2- 172.21.15.16:0/3069929506 >> v2:172.21.15.186:3300/0 conn(0x7fc09c0d1130 0x7fc09c0b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.902+0000 7fc0a2e91640 1 --2- 172.21.15.16:0/3069929506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc09c15dca0 0x7fc09c0ce940 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.902+0000 7fc0a2e91640 1 -- 172.21.15.16:0/3069929506 >> 172.21.15.16:0/3069929506 conn(0x7fc09c0726c0 msgr2=0x7fc09c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.902+0000 7fc0a2e91640 1 -- 172.21.15.16:0/3069929506 shutdown_connections 2024-04-25T10:06:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.902+0000 7fc0a2e91640 1 -- 172.21.15.16:0/3069929506 wait complete. 2024-04-25T10:06:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.903+0000 7fc0a2e91640 1 Processor -- start 2024-04-25T10:06:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.903+0000 7fc0a2e91640 1 -- start start 2024-04-25T10:06:42.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc0a2e91640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc09c0d1130 0x7fc09c15c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc0a2e91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc09c0d3b90 0x7fc09c0aa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc0a2e91640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc09c15dca0 0x7fc09c1f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc0a2e91640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c15e200 con 0x7fc09c0d3b90 2024-04-25T10:06:42.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc0a2e91640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c15e080 con 0x7fc09c15dca0 2024-04-25T10:06:42.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc0a2e91640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c0cee80 con 0x7fc09c0d1130 2024-04-25T10:06:42.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc09bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc09c0d3b90 0x7fc09c0aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc09bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc09c0d3b90 0x7fc09c0aa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55970/0 (socket says 172.21.15.16:55970) 2024-04-25T10:06:42.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc09bfff640 1 -- 172.21.15.16:0/3509102106 learned_addr learned my addr 172.21.15.16:0/3509102106 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:42.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc0a142e640 1 --2- 172.21.15.16:0/3509102106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc09c15dca0 0x7fc09c1f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc0a0c2d640 1 --2- 172.21.15.16:0/3509102106 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc09c0d1130 0x7fc09c15c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc09bfff640 1 -- 172.21.15.16:0/3509102106 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc09c0d1130 msgr2=0x7fc09c15c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc09bfff640 1 --2- 172.21.15.16:0/3509102106 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc09c0d1130 0x7fc09c15c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc09bfff640 1 -- 172.21.15.16:0/3509102106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc09c15dca0 msgr2=0x7fc09c1f6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc09bfff640 1 --2- 172.21.15.16:0/3509102106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc09c15dca0 0x7fc09c1f6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc09bfff640 1 -- 172.21.15.16:0/3509102106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09c160520 con 0x7fc09c0d3b90 2024-04-25T10:06:42.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc0a0c2d640 1 --2- 172.21.15.16:0/3509102106 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc09c0d1130 0x7fc09c15c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:42.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc0a142e640 1 --2- 172.21.15.16:0/3509102106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc09c15dca0 0x7fc09c1f6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:42.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc09bfff640 1 --2- 172.21.15.16:0/3509102106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc09c0d3b90 0x7fc09c0aa480 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc08800c400 tx=0x7fc08800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc0997fa640 1 -- 172.21.15.16:0/3509102106 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc088017070 con 0x7fc09c0d3b90 2024-04-25T10:06:42.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.904+0000 7fc0997fa640 1 -- 172.21.15.16:0/3509102106 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0880035f0 con 0x7fc09c0d3b90 2024-04-25T10:06:42.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.905+0000 7fc0a2e91640 1 -- 172.21.15.16:0/3509102106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09c0ab5f0 con 0x7fc09c0d3b90 2024-04-25T10:06:42.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.905+0000 7fc0997fa640 1 -- 172.21.15.16:0/3509102106 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc088003a20 con 0x7fc09c0d3b90 2024-04-25T10:06:42.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.905+0000 7fc0a2e91640 1 -- 172.21.15.16:0/3509102106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc09c1607c0 con 0x7fc09c0d3b90 2024-04-25T10:06:42.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.906+0000 7fc0a2e91640 1 -- 172.21.15.16:0/3509102106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc09c0750b0 con 0x7fc09c0d3b90 2024-04-25T10:06:42.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.906+0000 7fc0997fa640 1 -- 172.21.15.16:0/3509102106 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc08801a070 con 0x7fc09c0d3b90 2024-04-25T10:06:42.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.906+0000 7fc0997fa640 1 --2- 172.21.15.16:0/3509102106 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc08006b4d0 0x7fc08006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.907+0000 7fc0997fa640 1 -- 172.21.15.16:0/3509102106 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc088089220 con 0x7fc09c0d3b90 2024-04-25T10:06:42.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.907+0000 7fc0a0c2d640 1 --2- 172.21.15.16:0/3509102106 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc08006b4d0 0x7fc08006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.908+0000 7fc0a0c2d640 1 --2- 172.21.15.16:0/3509102106 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc08006b4d0 0x7fc08006d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc090034000 tx=0x7fc09002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:42.911+0000 7fc0997fa640 1 -- 172.21.15.16:0/3509102106 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc088057d50 con 0x7fc09c0d3b90 2024-04-25T10:06:43.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.093+0000 7fc0a2e91640 1 -- 172.21.15.16:0/3509102106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc09c0b3740 con 0x7fc09c0d3b90 2024-04-25T10:06:43.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.096+0000 7fc0997fa640 1 -- 172.21.15.16:0/3509102106 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc08805bdf0 con 0x7fc09c0d3b90 2024-04-25T10:06:43.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.098+0000 7fc0a2e91640 1 -- 172.21.15.16:0/3509102106 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc08006b4d0 msgr2=0x7fc08006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.098+0000 7fc0a2e91640 1 --2- 172.21.15.16:0/3509102106 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc08006b4d0 0x7fc08006d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc090034000 tx=0x7fc09002e000 comp rx=0 tx=0).stop 2024-04-25T10:06:43.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.098+0000 7fc0a2e91640 1 -- 172.21.15.16:0/3509102106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc09c0d3b90 msgr2=0x7fc09c0aa480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.098+0000 7fc0a2e91640 1 --2- 172.21.15.16:0/3509102106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc09c0d3b90 0x7fc09c0aa480 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc08800c400 tx=0x7fc08800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.099+0000 7fc0a2e91640 1 -- 172.21.15.16:0/3509102106 shutdown_connections 2024-04-25T10:06:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.099+0000 7fc0a2e91640 1 --2- 172.21.15.16:0/3509102106 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc08006b4d0 0x7fc08006d990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.099+0000 7fc0a2e91640 1 --2- 172.21.15.16:0/3509102106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc09c15dca0 0x7fc09c1f6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.099+0000 7fc0a2e91640 1 --2- 172.21.15.16:0/3509102106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc09c0d3b90 0x7fc09c0aa480 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.099+0000 7fc0a2e91640 1 --2- 172.21.15.16:0/3509102106 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc09c0d1130 0x7fc09c15c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.099+0000 7fc0a2e91640 1 -- 172.21.15.16:0/3509102106 >> 172.21.15.16:0/3509102106 conn(0x7fc09c0726c0 msgr2=0x7fc09c15cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.099+0000 7fc0a2e91640 1 -- 172.21.15.16:0/3509102106 shutdown_connections 2024-04-25T10:06:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.099+0000 7fc0a2e91640 1 -- 172.21.15.16:0/3509102106 wait complete. 2024-04-25T10:06:43.115 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-25T10:06:43.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.213+0000 7fed36953640 1 Processor -- start 2024-04-25T10:06:43.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.213+0000 7fed36953640 1 -- start start 2024-04-25T10:06:43.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.214+0000 7fed36953640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fed3015dce0 0x7fed300d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.214+0000 7fed36953640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed300d35b0 0x7fed300b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.214+0000 7fed36953640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fed300d1f90 0x7fed300b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.214+0000 7fed36953640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed3015e0c0 con 0x7fed3015dce0 2024-04-25T10:06:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.214+0000 7fed36953640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed300d2510 con 0x7fed300d1f90 2024-04-25T10:06:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.214+0000 7fed36953640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed300d27f0 con 0x7fed300d35b0 2024-04-25T10:06:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.214+0000 7fed2ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fed3015dce0 0x7fed300d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.214+0000 7fed2ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fed3015dce0 0x7fed300d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55974/0 (socket says 172.21.15.16:55974) 2024-04-25T10:06:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.214+0000 7fed2ffff640 1 -- 172.21.15.16:0/1190866778 learned_addr learned my addr 172.21.15.16:0/1190866778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.214+0000 7fed34ef0640 1 --2- 172.21.15.16:0/1190866778 >> v2:172.21.15.186:3300/0 conn(0x7fed300d1f90 0x7fed300b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.214+0000 7fed2ffff640 1 -- 172.21.15.16:0/1190866778 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed300d35b0 msgr2=0x7fed300b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.214+0000 7fed2f7fe640 1 --2- 172.21.15.16:0/1190866778 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed300d35b0 0x7fed300b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.214+0000 7fed2ffff640 1 --2- 172.21.15.16:0/1190866778 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed300d35b0 0x7fed300b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.214+0000 7fed2ffff640 1 -- 172.21.15.16:0/1190866778 >> v2:172.21.15.186:3300/0 conn(0x7fed300d1f90 msgr2=0x7fed300b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.214+0000 7fed2ffff640 1 --2- 172.21.15.16:0/1190866778 >> v2:172.21.15.186:3300/0 conn(0x7fed300d1f90 0x7fed300b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.214+0000 7fed2ffff640 1 -- 172.21.15.16:0/1190866778 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed300d2ad0 con 0x7fed3015dce0 2024-04-25T10:06:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.214+0000 7fed34ef0640 1 --2- 172.21.15.16:0/1190866778 >> v2:172.21.15.186:3300/0 conn(0x7fed300d1f90 0x7fed300b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.214+0000 7fed2f7fe640 1 --2- 172.21.15.16:0/1190866778 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed300d35b0 0x7fed300b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.215+0000 7fed2ffff640 1 --2- 172.21.15.16:0/1190866778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed3015dce0 0x7fed300d1a50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fed30161710 tx=0x7fed2402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420115202220c58d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.215+0000 7fed2effd640 1 -- 172.21.15.16:0/1190866778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24037070 con 0x7fed3015dce0 2024-04-25T10:06:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.215+0000 7fed2effd640 1 -- 172.21.15.16:0/1190866778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed24032440 con 0x7fed3015dce0 2024-04-25T10:06:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.215+0000 7fed2effd640 1 -- 172.21.15.16:0/1190866778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24032810 con 0x7fed3015dce0 2024-04-25T10:06:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.216+0000 7fed36953640 1 -- 172.21.15.16:0/1190866778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed3015dce0 msgr2=0x7fed300d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.216+0000 7fed36953640 1 --2- 172.21.15.16:0/1190866778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed3015dce0 0x7fed300d1a50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fed30161710 tx=0x7fed2402a140 comp rx=0 tx=0).stop 2024-04-25T10:06:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.216+0000 7fed36953640 1 -- 172.21.15.16:0/1190866778 shutdown_connections 2024-04-25T10:06:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.216+0000 7fed36953640 1 --2- 172.21.15.16:0/1190866778 >> v2:172.21.15.186:3300/0 conn(0x7fed300d1f90 0x7fed300b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.216+0000 7fed36953640 1 --2- 172.21.15.16:0/1190866778 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed300d35b0 0x7fed300b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.216+0000 7fed36953640 1 --2- 172.21.15.16:0/1190866778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed3015dce0 0x7fed300d1a50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.216+0000 7fed36953640 1 -- 172.21.15.16:0/1190866778 >> 172.21.15.16:0/1190866778 conn(0x7fed30072900 msgr2=0x7fed300d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.216+0000 7fed36953640 1 -- 172.21.15.16:0/1190866778 shutdown_connections 2024-04-25T10:06:43.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.216+0000 7fed36953640 1 -- 172.21.15.16:0/1190866778 wait complete. 2024-04-25T10:06:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.217+0000 7fed36953640 1 Processor -- start 2024-04-25T10:06:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.217+0000 7fed36953640 1 -- start start 2024-04-25T10:06:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.218+0000 7fed36953640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed300d1f90 0x7fed3015b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.218+0000 7fed36953640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed300d35b0 0x7fed300cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.218+0000 7fed36953640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed3015dce0 0x7fed300cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.218+0000 7fed36953640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed300b9820 con 0x7fed300d35b0 2024-04-25T10:06:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.218+0000 7fed36953640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed300d3090 con 0x7fed3015dce0 2024-04-25T10:06:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.218+0000 7fed36953640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed300b99a0 con 0x7fed300d1f90 2024-04-25T10:06:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.218+0000 7fed2f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed300d35b0 0x7fed300cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.218+0000 7fed2f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed300d35b0 0x7fed300cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55976/0 (socket says 172.21.15.16:55976) 2024-04-25T10:06:43.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.218+0000 7fed2f7fe640 1 -- 172.21.15.16:0/4136048966 learned_addr learned my addr 172.21.15.16:0/4136048966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:43.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.218+0000 7fed2ffff640 1 --2- 172.21.15.16:0/4136048966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed300d1f90 0x7fed3015b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.218+0000 7fed34ef0640 1 --2- 172.21.15.16:0/4136048966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed3015dce0 0x7fed300cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.218+0000 7fed2f7fe640 1 -- 172.21.15.16:0/4136048966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed300d1f90 msgr2=0x7fed3015b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.218+0000 7fed2f7fe640 1 --2- 172.21.15.16:0/4136048966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed300d1f90 0x7fed3015b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.218+0000 7fed2f7fe640 1 -- 172.21.15.16:0/4136048966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed3015dce0 msgr2=0x7fed300cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.218+0000 7fed2f7fe640 1 --2- 172.21.15.16:0/4136048966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed3015dce0 0x7fed300cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.218+0000 7fed2f7fe640 1 -- 172.21.15.16:0/4136048966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3015bb20 con 0x7fed300d35b0 2024-04-25T10:06:43.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.218+0000 7fed34ef0640 1 --2- 172.21.15.16:0/4136048966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed3015dce0 0x7fed300cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:43.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.219+0000 7fed2f7fe640 1 --2- 172.21.15.16:0/4136048966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed300d35b0 0x7fed300cbcf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fed2000c7f0 tx=0x7fed2000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.219+0000 7fed2cff9640 1 -- 172.21.15.16:0/4136048966 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20015070 con 0x7fed300d35b0 2024-04-25T10:06:43.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.219+0000 7fed2cff9640 1 -- 172.21.15.16:0/4136048966 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed20007820 con 0x7fed300d35b0 2024-04-25T10:06:43.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.219+0000 7fed2cff9640 1 -- 172.21.15.16:0/4136048966 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20007bd0 con 0x7fed300d35b0 2024-04-25T10:06:43.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.219+0000 7fed36953640 1 -- 172.21.15.16:0/4136048966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3015bdc0 con 0x7fed300d35b0 2024-04-25T10:06:43.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.219+0000 7fed36953640 1 -- 172.21.15.16:0/4136048966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3015c380 con 0x7fed300d35b0 2024-04-25T10:06:43.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.220+0000 7fed36953640 1 -- 172.21.15.16:0/4136048966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed300749b0 con 0x7fed300d35b0 2024-04-25T10:06:43.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.220+0000 7fed2cff9640 1 -- 172.21.15.16:0/4136048966 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed20018070 con 0x7fed300d35b0 2024-04-25T10:06:43.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.221+0000 7fed2cff9640 1 --2- 172.21.15.16:0/4136048966 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed1c06b4d0 0x7fed1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.221+0000 7fed2cff9640 1 -- 172.21.15.16:0/4136048966 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fed2008a0c0 con 0x7fed300d35b0 2024-04-25T10:06:43.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.221+0000 7fed2ffff640 1 --2- 172.21.15.16:0/4136048966 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed1c06b4d0 0x7fed1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.221+0000 7fed2ffff640 1 --2- 172.21.15.16:0/4136048966 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed1c06b4d0 0x7fed1c06d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fed24034000 tx=0x7fed2402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.225+0000 7fed2cff9640 1 -- 172.21.15.16:0/4136048966 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed200591b0 con 0x7fed300d35b0 2024-04-25T10:06:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.477+0000 7fed36953640 1 -- 172.21.15.16:0/4136048966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fed300b7a60 con 0x7fed300d35b0 2024-04-25T10:06:43.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.768+0000 7fed2cff9640 1 -- 172.21.15.16:0/4136048966 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v105) v1 ==== 142+0+0 (secure 0 0 0) 0x7fed20058ab0 con 0x7fed300d35b0 2024-04-25T10:06:43.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.771+0000 7fed36953640 1 -- 172.21.15.16:0/4136048966 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed1c06b4d0 msgr2=0x7fed1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.771+0000 7fed36953640 1 --2- 172.21.15.16:0/4136048966 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed1c06b4d0 0x7fed1c06d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fed24034000 tx=0x7fed2402e000 comp rx=0 tx=0).stop 2024-04-25T10:06:43.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.771+0000 7fed36953640 1 -- 172.21.15.16:0/4136048966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed300d35b0 msgr2=0x7fed300cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.771+0000 7fed36953640 1 --2- 172.21.15.16:0/4136048966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed300d35b0 0x7fed300cbcf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fed2000c7f0 tx=0x7fed2000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.771+0000 7fed36953640 1 -- 172.21.15.16:0/4136048966 shutdown_connections 2024-04-25T10:06:43.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.771+0000 7fed36953640 1 --2- 172.21.15.16:0/4136048966 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed1c06b4d0 0x7fed1c06d990 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.771+0000 7fed36953640 1 --2- 172.21.15.16:0/4136048966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed3015dce0 0x7fed300cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.771+0000 7fed36953640 1 --2- 172.21.15.16:0/4136048966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed300d35b0 0x7fed300cbcf0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.771+0000 7fed36953640 1 --2- 172.21.15.16:0/4136048966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed300d1f90 0x7fed3015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.771+0000 7fed36953640 1 -- 172.21.15.16:0/4136048966 >> 172.21.15.16:0/4136048966 conn(0x7fed30072900 msgr2=0x7fed300b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.771+0000 7fed36953640 1 -- 172.21.15.16:0/4136048966 shutdown_connections 2024-04-25T10:06:43.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.771+0000 7fed36953640 1 -- 172.21.15.16:0/4136048966 wait complete. 2024-04-25T10:06:43.788 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T10:06:43.788 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-04-25T10:06:43.788 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:06:43.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.886+0000 7fea529a5640 1 Processor -- start 2024-04-25T10:06:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.886+0000 7fea529a5640 1 -- start start 2024-04-25T10:06:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.886+0000 7fea529a5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea4c0cea00 0x7fea4c0cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.886+0000 7fea529a5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4c0d0430 con 0x7fea4c15f120 2024-04-25T10:06:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.886+0000 7fea529a5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4c0d0710 con 0x7fea4c0cfbe0 2024-04-25T10:06:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.886+0000 7fea529a5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0d09f0 con 0x7fea4c0cea00 2024-04-25T10:06:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fea4c15f120 0x7fea4c0ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36512/0 (socket says 172.21.15.16:36512) 2024-04-25T10:06:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4bfff640 1 -- 172.21.15.16:0/441320232 learned_addr learned my addr 172.21.15.16:0/441320232 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4affd640 1 -- 172.21.15.16:0/441320232 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3381157811 0 0) 0x7fea4c0d0430 con 0x7fea4c15f120 2024-04-25T10:06:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4b7fe640 1 --2- 172.21.15.16:0/441320232 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea4c0cea00 0x7fea4c0cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4affd640 1 -- 172.21.15.16:0/441320232 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea34003520 con 0x7fea4c15f120 2024-04-25T10:06:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4affd640 1 -- 172.21.15.16:0/441320232 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3112983201 0 0) 0x7fea34003520 con 0x7fea4c15f120 2024-04-25T10:06:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4affd640 1 -- 172.21.15.16:0/441320232 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea4c0cea00 msgr2=0x7fea4c0cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4affd640 1 --2- 172.21.15.16:0/441320232 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea4c0cea00 0x7fea4c0cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4affd640 1 -- 172.21.15.16:0/441320232 >> v1:172.21.15.186:6789/0 conn(0x7fea4c0cfbe0 legacy=0x7fea4c0ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4affd640 1 -- 172.21.15.16:0/441320232 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c0cc7d0 con 0x7fea4c15f120 2024-04-25T10:06:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4affd640 1 -- 172.21.15.16:0/441320232 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fea3c001670 con 0x7fea4c15f120 2024-04-25T10:06:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4affd640 1 -- 172.21.15.16:0/441320232 >> v1:172.21.15.16:6789/0 conn(0x7fea4c15f120 legacy=0x7fea4c0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4affd640 1 --2- 172.21.15.16:0/441320232 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea4c0cea00 0x7fea340039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4affd640 1 --2- 172.21.15.16:0/441320232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea34004010 0x7fea34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4affd640 1 --2- 172.21.15.16:0/441320232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea34004d40 0x7fea340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4affd640 1 -- 172.21.15.16:0/441320232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0d09f0 con 0x7fea34004d40 2024-04-25T10:06:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4affd640 1 -- 172.21.15.16:0/441320232 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3400bc10 con 0x7fea34004010 2024-04-25T10:06:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4affd640 1 -- 172.21.15.16:0/441320232 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3400be10 con 0x7fea4c0cea00 2024-04-25T10:06:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea50f42640 1 --2- 172.21.15.16:0/441320232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea34004d40 0x7fea340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4bfff640 1 --2- 172.21.15.16:0/441320232 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea4c0cea00 0x7fea340039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.887+0000 7fea4b7fe640 1 --2- 172.21.15.16:0/441320232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea34004010 0x7fea34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.888+0000 7fea50f42640 1 -- 172.21.15.16:0/441320232 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea4c0cea00 msgr2=0x7fea340039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.888+0000 7fea50f42640 1 --2- 172.21.15.16:0/441320232 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea4c0cea00 0x7fea340039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.888+0000 7fea50f42640 1 -- 172.21.15.16:0/441320232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea34004010 msgr2=0x7fea34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.888+0000 7fea50f42640 1 --2- 172.21.15.16:0/441320232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea34004010 0x7fea34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.888+0000 7fea50f42640 1 -- 172.21.15.16:0/441320232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea34003520 con 0x7fea34004d40 2024-04-25T10:06:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.888+0000 7fea4b7fe640 1 --2- 172.21.15.16:0/441320232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea34004010 0x7fea34004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.888+0000 7fea50f42640 1 --2- 172.21.15.16:0/441320232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea34004d40 0x7fea340051c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fea44001e50 tx=0x7fea34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4096f57deb6fd4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.888+0000 7fea4affd640 1 -- 172.21.15.16:0/441320232 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44001a70 con 0x7fea34004d40 2024-04-25T10:06:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.888+0000 7fea4affd640 1 -- 172.21.15.16:0/441320232 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea44003970 con 0x7fea34004d40 2024-04-25T10:06:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.888+0000 7fea4affd640 1 -- 172.21.15.16:0/441320232 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44003d60 con 0x7fea34004d40 2024-04-25T10:06:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.889+0000 7fea529a5640 1 -- 172.21.15.16:0/441320232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea34004d40 msgr2=0x7fea340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.889+0000 7fea529a5640 1 --2- 172.21.15.16:0/441320232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea34004d40 0x7fea340051c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fea44001e50 tx=0x7fea34005830 comp rx=0 tx=0).stop 2024-04-25T10:06:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.889+0000 7fea50f42640 1 -- 172.21.15.16:0/441320232 reap_dead start 2024-04-25T10:06:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.889+0000 7fea529a5640 1 -- 172.21.15.16:0/441320232 shutdown_connections 2024-04-25T10:06:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.889+0000 7fea529a5640 1 -- 172.21.15.16:0/441320232 >> 172.21.15.16:0/441320232 conn(0x7fea4c0d1900 msgr2=0x7fea4c0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.889+0000 7fea529a5640 1 -- 172.21.15.16:0/441320232 shutdown_connections 2024-04-25T10:06:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.889+0000 7fea529a5640 1 -- 172.21.15.16:0/441320232 wait complete. 2024-04-25T10:06:43.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.890+0000 7fea529a5640 1 Processor -- start 2024-04-25T10:06:43.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.890+0000 7fea529a5640 1 -- start start 2024-04-25T10:06:43.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.891+0000 7fea529a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea34004d40 0x7fea4c0ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.891+0000 7fea529a5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea4c1589c0 0x7fea4c1f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.891+0000 7fea529a5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea4c1f4f50 0x7fea4c0b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.891+0000 7fea529a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0c5450 con 0x7fea34004d40 2024-04-25T10:06:43.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.891+0000 7fea529a5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0cc480 con 0x7fea4c1f4f50 2024-04-25T10:06:43.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.891+0000 7fea529a5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0d0090 con 0x7fea4c1589c0 2024-04-25T10:06:43.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.891+0000 7fea4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea34004d40 0x7fea4c0ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.891+0000 7fea4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea34004d40 0x7fea4c0ca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55992/0 (socket says 172.21.15.16:55992) 2024-04-25T10:06:43.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.891+0000 7fea4bfff640 1 -- 172.21.15.16:0/673279517 learned_addr learned my addr 172.21.15.16:0/673279517 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:43.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.891+0000 7fea4b7fe640 1 --2- 172.21.15.16:0/673279517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea4c1589c0 0x7fea4c1f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.891+0000 7fea50f42640 1 --2- 172.21.15.16:0/673279517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea4c1f4f50 0x7fea4c0b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.891+0000 7fea4bfff640 1 -- 172.21.15.16:0/673279517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea4c1589c0 msgr2=0x7fea4c1f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.892+0000 7fea4bfff640 1 --2- 172.21.15.16:0/673279517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea4c1589c0 0x7fea4c1f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.892+0000 7fea4bfff640 1 -- 172.21.15.16:0/673279517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea4c1f4f50 msgr2=0x7fea4c0b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.892+0000 7fea4bfff640 1 --2- 172.21.15.16:0/673279517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea4c1f4f50 0x7fea4c0b44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.892+0000 7fea4bfff640 1 -- 172.21.15.16:0/673279517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c158e40 con 0x7fea34004d40 2024-04-25T10:06:43.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.892+0000 7fea4b7fe640 1 --2- 172.21.15.16:0/673279517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea4c1589c0 0x7fea4c1f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:43.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.892+0000 7fea50f42640 1 --2- 172.21.15.16:0/673279517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea4c1f4f50 0x7fea4c0b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:43.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.892+0000 7fea4bfff640 1 --2- 172.21.15.16:0/673279517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea34004d40 0x7fea4c0ca200 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fea3c007320 tx=0x7fea3c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.892+0000 7fea48ff9640 1 -- 172.21.15.16:0/673279517 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c01a070 con 0x7fea34004d40 2024-04-25T10:06:43.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.892+0000 7fea48ff9640 1 -- 172.21.15.16:0/673279517 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c00fa20 con 0x7fea34004d40 2024-04-25T10:06:43.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.892+0000 7fea529a5640 1 -- 172.21.15.16:0/673279517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4c0b7060 con 0x7fea34004d40 2024-04-25T10:06:43.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.892+0000 7fea48ff9640 1 -- 172.21.15.16:0/673279517 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c00fdf0 con 0x7fea34004d40 2024-04-25T10:06:43.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.892+0000 7fea529a5640 1 -- 172.21.15.16:0/673279517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c0c9720 con 0x7fea34004d40 2024-04-25T10:06:43.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.893+0000 7fea529a5640 1 -- 172.21.15.16:0/673279517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4c067850 con 0x7fea34004d40 2024-04-25T10:06:43.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.893+0000 7fea48ff9640 1 -- 172.21.15.16:0/673279517 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea3c01d070 con 0x7fea34004d40 2024-04-25T10:06:43.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.894+0000 7fea48ff9640 1 --2- 172.21.15.16:0/673279517 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fea3806b4d0 0x7fea3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.894+0000 7fea48ff9640 1 -- 172.21.15.16:0/673279517 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fea3c089aa0 con 0x7fea34004d40 2024-04-25T10:06:43.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.894+0000 7fea4b7fe640 1 --2- 172.21.15.16:0/673279517 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fea3806b4d0 0x7fea3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.895+0000 7fea4b7fe640 1 --2- 172.21.15.16:0/673279517 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fea3806b4d0 0x7fea3806d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0c9650 tx=0x7fea40008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:43.898+0000 7fea48ff9640 1 -- 172.21.15.16:0/673279517 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea3c058c20 con 0x7fea34004d40 2024-04-25T10:06:44.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.081+0000 7fea529a5640 1 -- 172.21.15.16:0/673279517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea4c0749b0 con 0x7fea34004d40 2024-04-25T10:06:44.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.082+0000 7fea48ff9640 1 -- 172.21.15.16:0/673279517 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+13020 (secure 0 0 0) 0x7fea3c058520 con 0x7fea34004d40 2024-04-25T10:06:44.087 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:44.087 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:06:42.763920+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6834/1417313138":"2024-04-26T10:06:39.756722+0000","172.21.15.16:6833/1417313138":"2024-04-26T10:06:39.756722+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:44.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.084+0000 7fea529a5640 1 -- 172.21.15.16:0/673279517 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fea3806b4d0 msgr2=0x7fea3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.084+0000 7fea529a5640 1 --2- 172.21.15.16:0/673279517 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fea3806b4d0 0x7fea3806d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0c9650 tx=0x7fea40008040 comp rx=0 tx=0).stop 2024-04-25T10:06:44.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.084+0000 7fea529a5640 1 -- 172.21.15.16:0/673279517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea34004d40 msgr2=0x7fea4c0ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.084+0000 7fea529a5640 1 --2- 172.21.15.16:0/673279517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea34004d40 0x7fea4c0ca200 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fea3c007320 tx=0x7fea3c0077e0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.084+0000 7fea529a5640 1 -- 172.21.15.16:0/673279517 shutdown_connections 2024-04-25T10:06:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.084+0000 7fea529a5640 1 --2- 172.21.15.16:0/673279517 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fea3806b4d0 0x7fea3806d990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.084+0000 7fea529a5640 1 --2- 172.21.15.16:0/673279517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea4c1f4f50 0x7fea4c0b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.084+0000 7fea529a5640 1 --2- 172.21.15.16:0/673279517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea4c1589c0 0x7fea4c1f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.084+0000 7fea529a5640 1 --2- 172.21.15.16:0/673279517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea34004d40 0x7fea4c0ca200 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.084+0000 7fea529a5640 1 -- 172.21.15.16:0/673279517 >> 172.21.15.16:0/673279517 conn(0x7fea4c0d1900 msgr2=0x7fea4c0b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.084+0000 7fea529a5640 1 -- 172.21.15.16:0/673279517 shutdown_connections 2024-04-25T10:06:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.084+0000 7fea529a5640 1 -- 172.21.15.16:0/673279517 wait complete. 2024-04-25T10:06:44.100 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:44.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:44.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.198+0000 7f37a6f10640 1 Processor -- start 2024-04-25T10:06:44.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.199+0000 7f37a6f10640 1 -- start start 2024-04-25T10:06:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.199+0000 7f37a6f10640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a00b97c0 0x7f37a00b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.199+0000 7f37a6f10640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f37a00b5a30 0x7f37a00d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.199+0000 7f37a6f10640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a01627a0 con 0x7f37a015dc80 2024-04-25T10:06:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.199+0000 7f37a6f10640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a00b5ea0 con 0x7f37a00b5a30 2024-04-25T10:06:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.199+0000 7f37a6f10640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a0161f80 con 0x7f37a00b97c0 2024-04-25T10:06:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.200+0000 7f37a54ad640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f37a00b5a30 0x7f37a00d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.200+0000 7f379ffff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a00b97c0 0x7f37a00b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.200+0000 7f37a4cac640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f37a015dc80 0x7f37a0161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:38022/0 (socket says 172.21.15.16:38022) 2024-04-25T10:06:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.200+0000 7f37a4cac640 1 -- 172.21.15.16:0/2711372828 learned_addr learned my addr 172.21.15.16:0/2711372828 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.200+0000 7f379f7fe640 1 -- 172.21.15.16:0/2711372828 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801389908 0 0) 0x7f37a01627a0 con 0x7f37a015dc80 2024-04-25T10:06:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.200+0000 7f379f7fe640 1 -- 172.21.15.16:0/2711372828 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3790003520 con 0x7f37a015dc80 2024-04-25T10:06:44.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.200+0000 7f379ffff640 1 -- 172.21.15.16:0/2711372828 >> v2:172.21.15.186:3300/0 conn(0x7f37a00b5a30 msgr2=0x7f37a00d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.200+0000 7f379ffff640 1 --2- 172.21.15.16:0/2711372828 >> v2:172.21.15.186:3300/0 conn(0x7f37a00b5a30 0x7f37a00d2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.200+0000 7f379ffff640 1 -- 172.21.15.16:0/2711372828 >> v1:172.21.15.186:6789/0 conn(0x7f37a015dc80 legacy=0x7f37a0161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.200+0000 7f379ffff640 1 -- 172.21.15.16:0/2711372828 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a01621e0 con 0x7f37a00b97c0 2024-04-25T10:06:44.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.200+0000 7f37a54ad640 1 --2- 172.21.15.16:0/2711372828 >> v2:172.21.15.186:3300/0 conn(0x7f37a00b5a30 0x7f37a00d2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:44.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.201+0000 7f379ffff640 1 --2- 172.21.15.16:0/2711372828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a00b97c0 0x7f37a00b54f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f378c0097a0 tx=0x7f378c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6adfa903584d5c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.201+0000 7f379f7fe640 1 -- 172.21.15.16:0/2711372828 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c037070 con 0x7f37a00b97c0 2024-04-25T10:06:44.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.201+0000 7f379f7fe640 1 -- 172.21.15.16:0/2711372828 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378c032440 con 0x7f37a00b97c0 2024-04-25T10:06:44.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.201+0000 7f379f7fe640 1 -- 172.21.15.16:0/2711372828 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c0327f0 con 0x7f37a00b97c0 2024-04-25T10:06:44.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.201+0000 7f37a6f10640 1 -- 172.21.15.16:0/2711372828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a00b97c0 msgr2=0x7f37a00b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.201+0000 7f37a6f10640 1 --2- 172.21.15.16:0/2711372828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a00b97c0 0x7f37a00b54f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f378c0097a0 tx=0x7f378c02a180 comp rx=0 tx=0).stop 2024-04-25T10:06:44.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.201+0000 7f37a6f10640 1 -- 172.21.15.16:0/2711372828 shutdown_connections 2024-04-25T10:06:44.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.201+0000 7f37a6f10640 1 --2- 172.21.15.16:0/2711372828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a00b97c0 0x7f37a00b54f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.201+0000 7f37a6f10640 1 --2- 172.21.15.16:0/2711372828 >> v2:172.21.15.186:3300/0 conn(0x7f37a00b5a30 0x7f37a00d2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.201+0000 7f37a6f10640 1 -- 172.21.15.16:0/2711372828 >> 172.21.15.16:0/2711372828 conn(0x7f37a00d04f0 msgr2=0x7f37a00d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.202+0000 7f37a6f10640 1 -- 172.21.15.16:0/2711372828 shutdown_connections 2024-04-25T10:06:44.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.202+0000 7f37a6f10640 1 -- 172.21.15.16:0/2711372828 wait complete. 2024-04-25T10:06:44.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.203+0000 7f37a6f10640 1 Processor -- start 2024-04-25T10:06:44.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.203+0000 7f37a6f10640 1 -- start start 2024-04-25T10:06:44.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.203+0000 7f37a6f10640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37a00b5a30 0x7f37a01f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f37a6f10640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a00b97c0 0x7f37a00c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f37a6f10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a015dc80 0x7f37a01f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f37a6f10640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a015e060 con 0x7f37a015dc80 2024-04-25T10:06:44.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f37a6f10640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a00d1c70 con 0x7f37a00b5a30 2024-04-25T10:06:44.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f37a6f10640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a00d29b0 con 0x7f37a00b97c0 2024-04-25T10:06:44.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f37a54ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a015dc80 0x7f37a01f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f37a54ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a015dc80 0x7f37a01f4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55998/0 (socket says 172.21.15.16:55998) 2024-04-25T10:06:44.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f37a54ad640 1 -- 172.21.15.16:0/3727484420 learned_addr learned my addr 172.21.15.16:0/3727484420 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:44.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f379ffff640 1 --2- 172.21.15.16:0/3727484420 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a00b97c0 0x7f37a00c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f37a4cac640 1 --2- 172.21.15.16:0/3727484420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37a00b5a30 0x7f37a01f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f37a54ad640 1 -- 172.21.15.16:0/3727484420 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a00b97c0 msgr2=0x7f37a00c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f37a54ad640 1 --2- 172.21.15.16:0/3727484420 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a00b97c0 0x7f37a00c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f37a54ad640 1 -- 172.21.15.16:0/3727484420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37a00b5a30 msgr2=0x7f37a01f6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f37a54ad640 1 --2- 172.21.15.16:0/3727484420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37a00b5a30 0x7f37a01f6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f37a54ad640 1 -- 172.21.15.16:0/3727484420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a01f76b0 con 0x7f37a015dc80 2024-04-25T10:06:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f379ffff640 1 --2- 172.21.15.16:0/3727484420 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a00b97c0 0x7f37a00c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f37a54ad640 1 --2- 172.21.15.16:0/3727484420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a015dc80 0x7f37a01f4100 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3798009f00 tx=0x7f379800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f379d7fa640 1 -- 172.21.15.16:0/3727484420 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798018030 con 0x7f37a015dc80 2024-04-25T10:06:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f379d7fa640 1 -- 172.21.15.16:0/3727484420 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379800ce50 con 0x7f37a015dc80 2024-04-25T10:06:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f37a6f10640 1 -- 172.21.15.16:0/3727484420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a01f7990 con 0x7f37a015dc80 2024-04-25T10:06:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f379d7fa640 1 -- 172.21.15.16:0/3727484420 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798003bb0 con 0x7f37a015dc80 2024-04-25T10:06:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.204+0000 7f37a6f10640 1 -- 172.21.15.16:0/3727484420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a01f3be0 con 0x7f37a015dc80 2024-04-25T10:06:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.206+0000 7f37a6f10640 1 -- 172.21.15.16:0/3727484420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a00750b0 con 0x7f37a015dc80 2024-04-25T10:06:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.206+0000 7f379d7fa640 1 -- 172.21.15.16:0/3727484420 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f379801b070 con 0x7f37a015dc80 2024-04-25T10:06:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.206+0000 7f379d7fa640 1 --2- 172.21.15.16:0/3727484420 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f378406b4d0 0x7f378406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.206+0000 7f379d7fa640 1 -- 172.21.15.16:0/3727484420 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f379808a980 con 0x7f37a015dc80 2024-04-25T10:06:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.207+0000 7f37a4cac640 1 --2- 172.21.15.16:0/3727484420 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f378406b4d0 0x7f378406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.207+0000 7f37a4cac640 1 --2- 172.21.15.16:0/3727484420 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f378406b4d0 0x7f378406d990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3794005880 tx=0x7f3794009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.210+0000 7f379d7fa640 1 -- 172.21.15.16:0/3727484420 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3798010040 con 0x7f37a015dc80 2024-04-25T10:06:44.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.396+0000 7f37a6f10640 1 -- 172.21.15.16:0/3727484420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f37a00d1780 con 0x7f37a015dc80 2024-04-25T10:06:44.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.396+0000 7f379d7fa640 1 -- 172.21.15.16:0/3727484420 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3798012340 con 0x7f37a015dc80 2024-04-25T10:06:44.402 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:06:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.398+0000 7f37a6f10640 1 -- 172.21.15.16:0/3727484420 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f378406b4d0 msgr2=0x7f378406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.398+0000 7f37a6f10640 1 --2- 172.21.15.16:0/3727484420 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f378406b4d0 0x7f378406d990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3794005880 tx=0x7f3794009040 comp rx=0 tx=0).stop 2024-04-25T10:06:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.398+0000 7f37a6f10640 1 -- 172.21.15.16:0/3727484420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a015dc80 msgr2=0x7f37a01f4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.398+0000 7f37a6f10640 1 --2- 172.21.15.16:0/3727484420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a015dc80 0x7f37a01f4100 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3798009f00 tx=0x7f379800c490 comp rx=0 tx=0).stop 2024-04-25T10:06:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.398+0000 7f37a6f10640 1 -- 172.21.15.16:0/3727484420 shutdown_connections 2024-04-25T10:06:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.398+0000 7f37a6f10640 1 --2- 172.21.15.16:0/3727484420 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f378406b4d0 0x7f378406d990 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.398+0000 7f37a6f10640 1 --2- 172.21.15.16:0/3727484420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a015dc80 0x7f37a01f4100 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.398+0000 7f37a6f10640 1 --2- 172.21.15.16:0/3727484420 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a00b97c0 0x7f37a00c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.398+0000 7f37a6f10640 1 --2- 172.21.15.16:0/3727484420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37a00b5a30 0x7f37a01f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.398+0000 7f37a6f10640 1 -- 172.21.15.16:0/3727484420 >> 172.21.15.16:0/3727484420 conn(0x7f37a00d04f0 msgr2=0x7f37a015bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.398+0000 7f37a6f10640 1 -- 172.21.15.16:0/3727484420 shutdown_connections 2024-04-25T10:06:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.398+0000 7f37a6f10640 1 -- 172.21.15.16:0/3727484420 wait complete. 2024-04-25T10:06:44.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-25T10:06:44.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.510+0000 7f5d3a4b3640 1 Processor -- start 2024-04-25T10:06:44.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.510+0000 7f5d3a4b3640 1 -- start start 2024-04-25T10:06:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.511+0000 7f5d3a4b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d3415f120 0x7f5d340ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.511+0000 7f5d3a4b3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d340d0430 con 0x7f5d340cfdb0 2024-04-25T10:06:44.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.511+0000 7f5d3a4b3640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d340d0710 con 0x7f5d340ce830 2024-04-25T10:06:44.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.511+0000 7f5d3a4b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d340d09f0 con 0x7f5d3415f120 2024-04-25T10:06:44.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.511+0000 7f5d38a50640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5d340cfdb0 0x7f5d340ccfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36526/0 (socket says 172.21.15.16:36526) 2024-04-25T10:06:44.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.511+0000 7f5d38a50640 1 -- 172.21.15.16:0/1356642825 learned_addr learned my addr 172.21.15.16:0/1356642825 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:44.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.511+0000 7f5d33fff640 1 --2- 172.21.15.16:0/1356642825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d3415f120 0x7f5d340ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.511+0000 7f5d32ffd640 1 -- 172.21.15.16:0/1356642825 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647972655 0 0) 0x7f5d340d0430 con 0x7f5d340cfdb0 2024-04-25T10:06:44.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.511+0000 7f5d32ffd640 1 -- 172.21.15.16:0/1356642825 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d1c003520 con 0x7f5d340cfdb0 2024-04-25T10:06:44.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.511+0000 7f5d32ffd640 1 -- 172.21.15.16:0/1356642825 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2808780494 0 0) 0x7f5d1c003520 con 0x7f5d340cfdb0 2024-04-25T10:06:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.511+0000 7f5d32ffd640 1 -- 172.21.15.16:0/1356642825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d3415f120 msgr2=0x7f5d340ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.511+0000 7f5d32ffd640 1 --2- 172.21.15.16:0/1356642825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d3415f120 0x7f5d340ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.511+0000 7f5d32ffd640 1 -- 172.21.15.16:0/1356642825 >> v1:172.21.15.186:6789/0 conn(0x7f5d340ce830 legacy=0x7f5d340cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.512+0000 7f5d32ffd640 1 -- 172.21.15.16:0/1356642825 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d340cc7d0 con 0x7f5d340cfdb0 2024-04-25T10:06:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.512+0000 7f5d32ffd640 1 -- 172.21.15.16:0/1356642825 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f5d2c0019f0 con 0x7f5d340cfdb0 2024-04-25T10:06:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.512+0000 7f5d32ffd640 1 -- 172.21.15.16:0/1356642825 >> v1:172.21.15.16:6789/0 conn(0x7f5d340cfdb0 legacy=0x7f5d340ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.512+0000 7f5d32ffd640 1 --2- 172.21.15.16:0/1356642825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1c0039e0 0x7f5d1c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.512+0000 7f5d32ffd640 1 --2- 172.21.15.16:0/1356642825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d1c004390 0x7f5d1c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.512+0000 7f5d32ffd640 1 --2- 172.21.15.16:0/1356642825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d3415f120 0x7f5d1c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.512+0000 7f5d32ffd640 1 -- 172.21.15.16:0/1356642825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d340d09f0 con 0x7f5d1c0039e0 2024-04-25T10:06:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.512+0000 7f5d32ffd640 1 -- 172.21.15.16:0/1356642825 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c00b9c0 con 0x7f5d1c004390 2024-04-25T10:06:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.512+0000 7f5d32ffd640 1 -- 172.21.15.16:0/1356642825 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c00bbc0 con 0x7f5d3415f120 2024-04-25T10:06:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.512+0000 7f5d33fff640 1 --2- 172.21.15.16:0/1356642825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1c0039e0 0x7f5d1c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.512+0000 7f5d337fe640 1 --2- 172.21.15.16:0/1356642825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d1c004390 0x7f5d1c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.512+0000 7f5d38a50640 1 --2- 172.21.15.16:0/1356642825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d3415f120 0x7f5d1c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.512+0000 7f5d33fff640 1 -- 172.21.15.16:0/1356642825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d3415f120 msgr2=0x7f5d1c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.512+0000 7f5d33fff640 1 --2- 172.21.15.16:0/1356642825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d3415f120 0x7f5d1c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.512+0000 7f5d33fff640 1 -- 172.21.15.16:0/1356642825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d1c004390 msgr2=0x7f5d1c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.512+0000 7f5d33fff640 1 --2- 172.21.15.16:0/1356642825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d1c004390 0x7f5d1c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.512+0000 7f5d33fff640 1 -- 172.21.15.16:0/1356642825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1c003520 con 0x7f5d1c0039e0 2024-04-25T10:06:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.512+0000 7f5d337fe640 1 --2- 172.21.15.16:0/1356642825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d1c004390 0x7f5d1c004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.513+0000 7f5d33fff640 1 --2- 172.21.15.16:0/1356642825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1c0039e0 0x7f5d1c003dc0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009ac0 tx=0x7f5d1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5d1e45f76b63b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.513+0000 7f5d32ffd640 1 -- 172.21.15.16:0/1356642825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28036070 con 0x7f5d1c0039e0 2024-04-25T10:06:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.513+0000 7f5d32ffd640 1 -- 172.21.15.16:0/1356642825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d28031400 con 0x7f5d1c0039e0 2024-04-25T10:06:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.513+0000 7f5d32ffd640 1 -- 172.21.15.16:0/1356642825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28009690 con 0x7f5d1c0039e0 2024-04-25T10:06:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.513+0000 7f5d3a4b3640 1 -- 172.21.15.16:0/1356642825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1c0039e0 msgr2=0x7f5d1c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.513+0000 7f5d3a4b3640 1 --2- 172.21.15.16:0/1356642825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1c0039e0 0x7f5d1c003dc0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009ac0 tx=0x7f5d1c005830 comp rx=0 tx=0).stop 2024-04-25T10:06:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.513+0000 7f5d38a50640 1 -- 172.21.15.16:0/1356642825 reap_dead start 2024-04-25T10:06:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.513+0000 7f5d3a4b3640 1 -- 172.21.15.16:0/1356642825 shutdown_connections 2024-04-25T10:06:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.513+0000 7f5d3a4b3640 1 -- 172.21.15.16:0/1356642825 >> 172.21.15.16:0/1356642825 conn(0x7f5d340d1900 msgr2=0x7f5d340d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.513+0000 7f5d3a4b3640 1 -- 172.21.15.16:0/1356642825 shutdown_connections 2024-04-25T10:06:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.513+0000 7f5d3a4b3640 1 -- 172.21.15.16:0/1356642825 wait complete. 2024-04-25T10:06:44.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.515+0000 7f5d3a4b3640 1 Processor -- start 2024-04-25T10:06:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.515+0000 7f5d3a4b3640 1 -- start start 2024-04-25T10:06:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.515+0000 7f5d3a4b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d1c0039e0 0x7f5d34159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.515+0000 7f5d3a4b3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d340b0e30 0x7f5d340b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.515+0000 7f5d3a4b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d340b6190 0x7f5d340b6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.515+0000 7f5d3a4b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d340cc480 con 0x7f5d340b6190 2024-04-25T10:06:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.515+0000 7f5d3a4b3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d340cec70 con 0x7f5d340b0e30 2024-04-25T10:06:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.515+0000 7f5d3a4b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d340c5450 con 0x7f5d1c0039e0 2024-04-25T10:06:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.516+0000 7f5d38a50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d340b6190 0x7f5d340b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.516+0000 7f5d38a50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d340b6190 0x7f5d340b6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56022/0 (socket says 172.21.15.16:56022) 2024-04-25T10:06:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.516+0000 7f5d38a50640 1 -- 172.21.15.16:0/664067600 learned_addr learned my addr 172.21.15.16:0/664067600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.516+0000 7f5d33fff640 1 --2- 172.21.15.16:0/664067600 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d1c0039e0 0x7f5d34159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.516+0000 7f5d337fe640 1 --2- 172.21.15.16:0/664067600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d340b0e30 0x7f5d340b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.516+0000 7f5d38a50640 1 -- 172.21.15.16:0/664067600 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d1c0039e0 msgr2=0x7f5d34159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.516+0000 7f5d38a50640 1 --2- 172.21.15.16:0/664067600 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d1c0039e0 0x7f5d34159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.516+0000 7f5d38a50640 1 -- 172.21.15.16:0/664067600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d340b0e30 msgr2=0x7f5d340b5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.516+0000 7f5d38a50640 1 --2- 172.21.15.16:0/664067600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d340b0e30 0x7f5d340b5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.516+0000 7f5d38a50640 1 -- 172.21.15.16:0/664067600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d340cb160 con 0x7f5d340b6190 2024-04-25T10:06:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.516+0000 7f5d33fff640 1 --2- 172.21.15.16:0/664067600 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d1c0039e0 0x7f5d34159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.516+0000 7f5d38a50640 1 --2- 172.21.15.16:0/664067600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d340b6190 0x7f5d340b6610 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5d3409f7d0 tx=0x7f5d2c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.516+0000 7f5d30ff9640 1 -- 172.21.15.16:0/664067600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c00ebc0 con 0x7f5d340b6190 2024-04-25T10:06:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.516+0000 7f5d30ff9640 1 -- 172.21.15.16:0/664067600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2c00ed80 con 0x7f5d340b6190 2024-04-25T10:06:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.516+0000 7f5d3a4b3640 1 -- 172.21.15.16:0/664067600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d340cb440 con 0x7f5d340b6190 2024-04-25T10:06:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.516+0000 7f5d30ff9640 1 -- 172.21.15.16:0/664067600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c014670 con 0x7f5d340b6190 2024-04-25T10:06:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.516+0000 7f5d3a4b3640 1 -- 172.21.15.16:0/664067600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d340b4f00 con 0x7f5d340b6190 2024-04-25T10:06:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.517+0000 7f5d3a4b3640 1 -- 172.21.15.16:0/664067600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d34067850 con 0x7f5d340b6190 2024-04-25T10:06:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.518+0000 7f5d30ff9640 1 -- 172.21.15.16:0/664067600 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d2c014810 con 0x7f5d340b6190 2024-04-25T10:06:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.518+0000 7f5d30ff9640 1 --2- 172.21.15.16:0/664067600 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5d2006b500 0x7f5d2006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.518+0000 7f5d30ff9640 1 -- 172.21.15.16:0/664067600 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5d2c089ca0 con 0x7f5d340b6190 2024-04-25T10:06:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.519+0000 7f5d33fff640 1 --2- 172.21.15.16:0/664067600 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5d2006b500 0x7f5d2006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.519+0000 7f5d33fff640 1 --2- 172.21.15.16:0/664067600 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5d2006b500 0x7f5d2006d9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009ac0 tx=0x7f5d28008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.523+0000 7f5d30ff9640 1 -- 172.21.15.16:0/664067600 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d2c058e30 con 0x7f5d340b6190 2024-04-25T10:06:44.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:44.772+0000 7f5d3a4b3640 1 -- 172.21.15.16:0/664067600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f5d340749b0 con 0x7f5d340b6190 2024-04-25T10:06:45.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.775+0000 7f5d30ff9640 1 -- 172.21.15.16:0/664067600 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v107) v1 ==== 152+0+0 (secure 0 0 0) 0x7f5d2c058730 con 0x7f5d340b6190 2024-04-25T10:06:45.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.778+0000 7f5d3a4b3640 1 -- 172.21.15.16:0/664067600 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5d2006b500 msgr2=0x7f5d2006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.778+0000 7f5d3a4b3640 1 --2- 172.21.15.16:0/664067600 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5d2006b500 0x7f5d2006d9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009ac0 tx=0x7f5d28008030 comp rx=0 tx=0).stop 2024-04-25T10:06:45.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.778+0000 7f5d3a4b3640 1 -- 172.21.15.16:0/664067600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d340b6190 msgr2=0x7f5d340b6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.778+0000 7f5d3a4b3640 1 --2- 172.21.15.16:0/664067600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d340b6190 0x7f5d340b6610 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5d3409f7d0 tx=0x7f5d2c00e530 comp rx=0 tx=0).stop 2024-04-25T10:06:45.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.778+0000 7f5d3a4b3640 1 -- 172.21.15.16:0/664067600 shutdown_connections 2024-04-25T10:06:45.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.778+0000 7f5d3a4b3640 1 --2- 172.21.15.16:0/664067600 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5d2006b500 0x7f5d2006d9c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.778+0000 7f5d3a4b3640 1 --2- 172.21.15.16:0/664067600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d340b6190 0x7f5d340b6610 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.778+0000 7f5d3a4b3640 1 --2- 172.21.15.16:0/664067600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d340b0e30 0x7f5d340b5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.778+0000 7f5d3a4b3640 1 --2- 172.21.15.16:0/664067600 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d1c0039e0 0x7f5d34159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.778+0000 7f5d3a4b3640 1 -- 172.21.15.16:0/664067600 >> 172.21.15.16:0/664067600 conn(0x7f5d340d1900 msgr2=0x7f5d3415e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.778+0000 7f5d3a4b3640 1 -- 172.21.15.16:0/664067600 shutdown_connections 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.779+0000 7f5d3a4b3640 1 -- 172.21.15.16:0/664067600 wait complete. 2024-04-25T10:06:45.797 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:06:45.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.901+0000 7f0dd8134640 1 Processor -- start 2024-04-25T10:06:45.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.901+0000 7f0dd8134640 1 -- start start 2024-04-25T10:06:45.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.902+0000 7f0dd8134640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0dd015db50 0x7f0dd00cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.902+0000 7f0dd8134640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd00d0200 con 0x7f0dd00d12a0 2024-04-25T10:06:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.902+0000 7f0dd8134640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd00d04e0 con 0x7f0dd015df30 2024-04-25T10:06:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.902+0000 7f0dd8134640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd006c020 con 0x7f0dd015db50 2024-04-25T10:06:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.902+0000 7f0dd66d1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0dd00d12a0 0x7f0dd0099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36538/0 (socket says 172.21.15.16:36538) 2024-04-25T10:06:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.902+0000 7f0dd66d1640 1 -- 172.21.15.16:0/526034499 learned_addr learned my addr 172.21.15.16:0/526034499 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.903+0000 7f0dd5ed0640 1 --2- 172.21.15.16:0/526034499 >> v2:172.21.15.186:3300/0 conn(0x7f0dd015db50 0x7f0dd00cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.903+0000 7f0dd4ece640 1 -- 172.21.15.16:0/526034499 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2937839945 0 0) 0x7f0dd00d0200 con 0x7f0dd00d12a0 2024-04-25T10:06:45.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.903+0000 7f0dd4ece640 1 -- 172.21.15.16:0/526034499 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0db8003520 con 0x7f0dd00d12a0 2024-04-25T10:06:45.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.903+0000 7f0dd5ed0640 1 -- 172.21.15.16:0/526034499 >> v1:172.21.15.186:6789/0 conn(0x7f0dd015df30 legacy=0x7f0dd006cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.903+0000 7f0dd5ed0640 1 -- 172.21.15.16:0/526034499 >> v1:172.21.15.16:6789/0 conn(0x7f0dd00d12a0 legacy=0x7f0dd0099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.903+0000 7f0dd4ece640 1 -- 172.21.15.16:0/526034499 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3056100928 0 0) 0x7f0db8003520 con 0x7f0dd00d12a0 2024-04-25T10:06:45.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.903+0000 7f0dd5ed0640 1 -- 172.21.15.16:0/526034499 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd00d07c0 con 0x7f0dd015db50 2024-04-25T10:06:45.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.904+0000 7f0dd5ed0640 1 --2- 172.21.15.16:0/526034499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dd015db50 0x7f0dd00cfb80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80097a0 tx=0x7f0dc802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc81921b8b3bc11f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.904+0000 7f0dd4ece640 1 -- 172.21.15.16:0/526034499 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8037070 con 0x7f0dd015db50 2024-04-25T10:06:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.904+0000 7f0dd4ece640 1 -- 172.21.15.16:0/526034499 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc802ac50 con 0x7f0dd015db50 2024-04-25T10:06:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.904+0000 7f0dd4ece640 1 -- 172.21.15.16:0/526034499 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc00019f0 con 0x7f0dd015db50 2024-04-25T10:06:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.904+0000 7f0dd8134640 1 -- 172.21.15.16:0/526034499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dd015db50 msgr2=0x7f0dd00cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.904+0000 7f0dd8134640 1 --2- 172.21.15.16:0/526034499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dd015db50 0x7f0dd00cfb80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80097a0 tx=0x7f0dc802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.905+0000 7f0dd8134640 1 -- 172.21.15.16:0/526034499 shutdown_connections 2024-04-25T10:06:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.905+0000 7f0dd8134640 1 --2- 172.21.15.16:0/526034499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dd015db50 0x7f0dd00cfb80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.905+0000 7f0dd8134640 1 -- 172.21.15.16:0/526034499 >> 172.21.15.16:0/526034499 conn(0x7f0dd00b3430 msgr2=0x7f0dd00b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.905+0000 7f0dd8134640 1 -- 172.21.15.16:0/526034499 shutdown_connections 2024-04-25T10:06:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.905+0000 7f0dd8134640 1 -- 172.21.15.16:0/526034499 wait complete. 2024-04-25T10:06:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.906+0000 7f0dd8134640 1 Processor -- start 2024-04-25T10:06:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.906+0000 7f0dd8134640 1 -- start start 2024-04-25T10:06:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd8134640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dd00d12a0 0x7f0dd015b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd8134640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd015db50 0x7f0dd01e7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd8134640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0dd015df30 0x7f0dd0158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd8134640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd009a150 con 0x7f0dd015db50 2024-04-25T10:06:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd8134640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd00d0d80 con 0x7f0dd00d12a0 2024-04-25T10:06:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd8134640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd009a2d0 con 0x7f0dd015df30 2024-04-25T10:06:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd56cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd015db50 0x7f0dd01e7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd56cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd015db50 0x7f0dd01e7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56034/0 (socket says 172.21.15.16:56034) 2024-04-25T10:06:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd56cf640 1 -- 172.21.15.16:0/1698529299 learned_addr learned my addr 172.21.15.16:0/1698529299 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd5ed0640 1 --2- 172.21.15.16:0/1698529299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dd00d12a0 0x7f0dd015b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd66d1640 1 --2- 172.21.15.16:0/1698529299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0dd015df30 0x7f0dd0158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd56cf640 1 -- 172.21.15.16:0/1698529299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0dd015df30 msgr2=0x7f0dd0158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd56cf640 1 --2- 172.21.15.16:0/1698529299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0dd015df30 0x7f0dd0158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd56cf640 1 -- 172.21.15.16:0/1698529299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dd00d12a0 msgr2=0x7f0dd015b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd56cf640 1 --2- 172.21.15.16:0/1698529299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dd00d12a0 0x7f0dd015b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd56cf640 1 -- 172.21.15.16:0/1698529299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd01e83e0 con 0x7f0dd015db50 2024-04-25T10:06:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd5ed0640 1 --2- 172.21.15.16:0/1698529299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dd00d12a0 0x7f0dd015b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd56cf640 1 --2- 172.21.15.16:0/1698529299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd015db50 0x7f0dd01e7ea0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400c3c0 tx=0x7f0dc400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dce7fc640 1 -- 172.21.15.16:0/1698529299 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4017070 con 0x7f0dd015db50 2024-04-25T10:06:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dce7fc640 1 -- 172.21.15.16:0/1698529299 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc40052f0 con 0x7f0dd015db50 2024-04-25T10:06:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd8134640 1 -- 172.21.15.16:0/1698529299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd015bc70 con 0x7f0dd015db50 2024-04-25T10:06:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dce7fc640 1 -- 172.21.15.16:0/1698529299 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4003830 con 0x7f0dd015db50 2024-04-25T10:06:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.907+0000 7f0dd8134640 1 -- 172.21.15.16:0/1698529299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd006f800 con 0x7f0dd015db50 2024-04-25T10:06:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.908+0000 7f0dd8134640 1 -- 172.21.15.16:0/1698529299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd00750b0 con 0x7f0dd015db50 2024-04-25T10:06:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.909+0000 7f0dce7fc640 1 -- 172.21.15.16:0/1698529299 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0dc401a070 con 0x7f0dd015db50 2024-04-25T10:06:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.909+0000 7f0dce7fc640 1 --2- 172.21.15.16:0/1698529299 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0dbc06b4d0 0x7f0dbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.909+0000 7f0dce7fc640 1 -- 172.21.15.16:0/1698529299 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0dc4089200 con 0x7f0dd015db50 2024-04-25T10:06:45.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.910+0000 7f0dd5ed0640 1 --2- 172.21.15.16:0/1698529299 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0dbc06b4d0 0x7f0dbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.910+0000 7f0dd5ed0640 1 --2- 172.21.15.16:0/1698529299 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0dbc06b4d0 0x7f0dbc06d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8034000 tx=0x7f0dc802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:45.914+0000 7f0dce7fc640 1 -- 172.21.15.16:0/1698529299 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0dc4058380 con 0x7f0dd015db50 2024-04-25T10:06:46.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.092+0000 7f0dd8134640 1 -- 172.21.15.16:0/1698529299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0dd009aed0 con 0x7f0dd015db50 2024-04-25T10:06:46.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.095+0000 7f0dce7fc640 1 -- 172.21.15.16:0/1698529299 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0dc4057c80 con 0x7f0dd015db50 2024-04-25T10:06:46.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.096+0000 7f0dd8134640 1 -- 172.21.15.16:0/1698529299 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0dbc06b4d0 msgr2=0x7f0dbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.096+0000 7f0dd8134640 1 --2- 172.21.15.16:0/1698529299 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0dbc06b4d0 0x7f0dbc06d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8034000 tx=0x7f0dc802e000 comp rx=0 tx=0).stop 2024-04-25T10:06:46.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.096+0000 7f0dd8134640 1 -- 172.21.15.16:0/1698529299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd015db50 msgr2=0x7f0dd01e7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.096+0000 7f0dd8134640 1 --2- 172.21.15.16:0/1698529299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd015db50 0x7f0dd01e7ea0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400c3c0 tx=0x7f0dc400c880 comp rx=0 tx=0).stop 2024-04-25T10:06:46.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.097+0000 7f0dd8134640 1 -- 172.21.15.16:0/1698529299 shutdown_connections 2024-04-25T10:06:46.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.097+0000 7f0dd8134640 1 --2- 172.21.15.16:0/1698529299 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0dbc06b4d0 0x7f0dbc06d990 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.097+0000 7f0dd8134640 1 --2- 172.21.15.16:0/1698529299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0dd015df30 0x7f0dd0158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.097+0000 7f0dd8134640 1 --2- 172.21.15.16:0/1698529299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd015db50 0x7f0dd01e7ea0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.097+0000 7f0dd8134640 1 --2- 172.21.15.16:0/1698529299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dd00d12a0 0x7f0dd015b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.097+0000 7f0dd8134640 1 -- 172.21.15.16:0/1698529299 >> 172.21.15.16:0/1698529299 conn(0x7f0dd00b3430 msgr2=0x7f0dd015ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.097+0000 7f0dd8134640 1 -- 172.21.15.16:0/1698529299 shutdown_connections 2024-04-25T10:06:46.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.097+0000 7f0dd8134640 1 -- 172.21.15.16:0/1698529299 wait complete. 2024-04-25T10:06:46.113 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:06:46.113 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:46.157 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:46.210 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:06:46.210 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:06:46.210 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:46.226 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:46.226 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:06:46.226 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:46.244 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:46.244 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:06:46.244 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:46.282 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:46.282 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:06:46.282 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:46.300 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:46.300 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:06:46.300 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:46.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.417+0000 7f3f2d7c1640 1 Processor -- start 2024-04-25T10:06:46.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.417+0000 7f3f2d7c1640 1 -- start start 2024-04-25T10:06:46.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.418+0000 7f3f2d7c1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f280b9460 0x7f3f280ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.418+0000 7f3f2d7c1640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3f280d34e0 0x7f3f280cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.418+0000 7f3f2d7c1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f280d2440 con 0x7f3f2815dd50 2024-04-25T10:06:46.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.418+0000 7f3f2d7c1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f280d2720 con 0x7f3f280b9460 2024-04-25T10:06:46.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.418+0000 7f3f2d7c1640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f280d2a00 con 0x7f3f280d34e0 2024-04-25T10:06:46.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.418+0000 7f3f267fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f280b9460 0x7f3f280ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.418+0000 7f3f267fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f280b9460 0x7f3f280ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56048/0 (socket says 172.21.15.16:56048) 2024-04-25T10:06:46.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.418+0000 7f3f267fc640 1 -- 172.21.15.16:0/2647328864 learned_addr learned my addr 172.21.15.16:0/2647328864 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:46.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.418+0000 7f3f26ffd640 1 --1- 172.21.15.16:0/2647328864 >> v1:172.21.15.16:6789/0 conn(0x7f3f2815dd50 0x7f3f280d1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36544/0 (socket says 172.21.15.16:36544) 2024-04-25T10:06:46.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.418+0000 7f3f277fe640 1 --2- 172.21.15.16:0/2647328864 >> v2:172.21.15.186:3300/0 conn(0x7f3f280d34e0 0x7f3f280cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.419+0000 7f3f267fc640 1 -- 172.21.15.16:0/2647328864 >> v2:172.21.15.186:3300/0 conn(0x7f3f280d34e0 msgr2=0x7f3f280cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.419+0000 7f3f267fc640 1 --2- 172.21.15.16:0/2647328864 >> v2:172.21.15.186:3300/0 conn(0x7f3f280d34e0 0x7f3f280cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.419+0000 7f3f267fc640 1 -- 172.21.15.16:0/2647328864 >> v1:172.21.15.16:6789/0 conn(0x7f3f2815dd50 legacy=0x7f3f280d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.419+0000 7f3f267fc640 1 -- 172.21.15.16:0/2647328864 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f280d2ce0 con 0x7f3f280b9460 2024-04-25T10:06:46.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.419+0000 7f3f267fc640 1 --2- 172.21.15.16:0/2647328864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f280b9460 0x7f3f280ce7a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3f280b6a40 tx=0x7f3f1002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8f040e2e6dbb70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.419+0000 7f3f25ffb640 1 -- 172.21.15.16:0/2647328864 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10037070 con 0x7f3f280b9460 2024-04-25T10:06:46.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.419+0000 7f3f25ffb640 1 -- 172.21.15.16:0/2647328864 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f10032440 con 0x7f3f280b9460 2024-04-25T10:06:46.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.419+0000 7f3f25ffb640 1 -- 172.21.15.16:0/2647328864 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10032810 con 0x7f3f280b9460 2024-04-25T10:06:46.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.420+0000 7f3f2d7c1640 1 -- 172.21.15.16:0/2647328864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f280b9460 msgr2=0x7f3f280ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.420+0000 7f3f2d7c1640 1 --2- 172.21.15.16:0/2647328864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f280b9460 0x7f3f280ce7a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3f280b6a40 tx=0x7f3f1002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.420+0000 7f3f2d7c1640 1 -- 172.21.15.16:0/2647328864 shutdown_connections 2024-04-25T10:06:46.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.420+0000 7f3f2d7c1640 1 --2- 172.21.15.16:0/2647328864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f280b9460 0x7f3f280ce7a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.420+0000 7f3f2d7c1640 1 --2- 172.21.15.16:0/2647328864 >> v2:172.21.15.186:3300/0 conn(0x7f3f280d34e0 0x7f3f280cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.420+0000 7f3f2d7c1640 1 -- 172.21.15.16:0/2647328864 >> 172.21.15.16:0/2647328864 conn(0x7f3f280722d0 msgr2=0x7f3f280726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.420+0000 7f3f2d7c1640 1 -- 172.21.15.16:0/2647328864 shutdown_connections 2024-04-25T10:06:46.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.420+0000 7f3f2d7c1640 1 -- 172.21.15.16:0/2647328864 wait complete. 2024-04-25T10:06:46.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.421+0000 7f3f2d7c1640 1 Processor -- start 2024-04-25T10:06:46.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.421+0000 7f3f2d7c1640 1 -- start start 2024-04-25T10:06:46.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.422+0000 7f3f2d7c1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f280b9460 0x7f3f280c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.422+0000 7f3f2d7c1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f280d34e0 0x7f3f2815fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.422+0000 7f3f2d7c1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f2815dd50 0x7f3f28160090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.422+0000 7f3f2d7c1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f280b6df0 con 0x7f3f280d34e0 2024-04-25T10:06:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.422+0000 7f3f2d7c1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f280b6c70 con 0x7f3f2815dd50 2024-04-25T10:06:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.422+0000 7f3f2d7c1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f280b6f70 con 0x7f3f280b9460 2024-04-25T10:06:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.422+0000 7f3f267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f280d34e0 0x7f3f2815fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.422+0000 7f3f267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f280d34e0 0x7f3f2815fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56060/0 (socket says 172.21.15.16:56060) 2024-04-25T10:06:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.422+0000 7f3f267fc640 1 -- 172.21.15.16:0/368653012 learned_addr learned my addr 172.21.15.16:0/368653012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.422+0000 7f3f277fe640 1 --2- 172.21.15.16:0/368653012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f2815dd50 0x7f3f28160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.422+0000 7f3f26ffd640 1 --2- 172.21.15.16:0/368653012 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f280b9460 0x7f3f280c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.422+0000 7f3f267fc640 1 -- 172.21.15.16:0/368653012 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f280b9460 msgr2=0x7f3f280c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.422+0000 7f3f267fc640 1 --2- 172.21.15.16:0/368653012 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f280b9460 0x7f3f280c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.422+0000 7f3f267fc640 1 -- 172.21.15.16:0/368653012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f2815dd50 msgr2=0x7f3f28160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.422+0000 7f3f267fc640 1 --2- 172.21.15.16:0/368653012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f2815dd50 0x7f3f28160090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.422+0000 7f3f267fc640 1 -- 172.21.15.16:0/368653012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f280b5a80 con 0x7f3f280d34e0 2024-04-25T10:06:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.422+0000 7f3f277fe640 1 --2- 172.21.15.16:0/368653012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f2815dd50 0x7f3f28160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.423+0000 7f3f267fc640 1 --2- 172.21.15.16:0/368653012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f280d34e0 0x7f3f2815fb50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3f1002a220 tx=0x7f3f100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.423+0000 7f3f1f7fe640 1 -- 172.21.15.16:0/368653012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10041020 con 0x7f3f280d34e0 2024-04-25T10:06:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.423+0000 7f3f1f7fe640 1 -- 172.21.15.16:0/368653012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f100306c0 con 0x7f3f280d34e0 2024-04-25T10:06:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.423+0000 7f3f2d7c1640 1 -- 172.21.15.16:0/368653012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f280b5d60 con 0x7f3f280d34e0 2024-04-25T10:06:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.423+0000 7f3f1f7fe640 1 -- 172.21.15.16:0/368653012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10037040 con 0x7f3f280d34e0 2024-04-25T10:06:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.423+0000 7f3f2d7c1640 1 -- 172.21.15.16:0/368653012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f280b6320 con 0x7f3f280d34e0 2024-04-25T10:06:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.424+0000 7f3f2d7c1640 1 -- 172.21.15.16:0/368653012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f28067850 con 0x7f3f280d34e0 2024-04-25T10:06:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.424+0000 7f3f1f7fe640 1 -- 172.21.15.16:0/368653012 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f10044020 con 0x7f3f280d34e0 2024-04-25T10:06:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.425+0000 7f3f1f7fe640 1 --2- 172.21.15.16:0/368653012 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3f0806b4d0 0x7f3f0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.425+0000 7f3f1f7fe640 1 -- 172.21.15.16:0/368653012 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3f100aa7b0 con 0x7f3f280d34e0 2024-04-25T10:06:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.425+0000 7f3f26ffd640 1 --2- 172.21.15.16:0/368653012 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3f0806b4d0 0x7f3f0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.426+0000 7f3f26ffd640 1 --2- 172.21.15.16:0/368653012 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3f0806b4d0 0x7f3f0806d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3f20000f60 tx=0x7f3f20009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.429+0000 7f3f1f7fe640 1 -- 172.21.15.16:0/368653012 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f1004b030 con 0x7f3f280d34e0 2024-04-25T10:06:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.648+0000 7f3f2d7c1640 1 -- 172.21.15.16:0/368653012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f28068620 con 0x7f3f280d34e0 2024-04-25T10:06:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.649+0000 7f3f1f7fe640 1 -- 172.21.15.16:0/368653012 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+3701 (secure 0 0 0) 0x7f3f28068620 con 0x7f3f280d34e0 2024-04-25T10:06:46.655 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:46.655 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3381395304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3381395304},{"type":"v1","addr":"172.21.15.186:6835","nonce":3381395304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2429147785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2429147785},{"type":"v1","addr":"172.21.15.186:6837","nonce":2429147785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5142,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2191514045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2191514045},{"type":"v1","addr":"172.21.15.16:6835","nonce":2191514045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":5532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/734089202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":734089202},{"type":"v1","addr":"172.21.15.16:6834","nonce":734089202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":16}],"filesystems":[]} 2024-04-25T10:06:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-04-25T10:06:46.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.651+0000 7f3f2d7c1640 1 -- 172.21.15.16:0/368653012 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3f0806b4d0 msgr2=0x7f3f0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.651+0000 7f3f2d7c1640 1 --2- 172.21.15.16:0/368653012 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3f0806b4d0 0x7f3f0806d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3f20000f60 tx=0x7f3f20009040 comp rx=0 tx=0).stop 2024-04-25T10:06:46.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.651+0000 7f3f2d7c1640 1 -- 172.21.15.16:0/368653012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f280d34e0 msgr2=0x7f3f2815fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.651+0000 7f3f2d7c1640 1 --2- 172.21.15.16:0/368653012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f280d34e0 0x7f3f2815fb50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3f1002a220 tx=0x7f3f100046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.651+0000 7f3f2d7c1640 1 -- 172.21.15.16:0/368653012 shutdown_connections 2024-04-25T10:06:46.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.651+0000 7f3f2d7c1640 1 --2- 172.21.15.16:0/368653012 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3f0806b4d0 0x7f3f0806d990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.651+0000 7f3f2d7c1640 1 --2- 172.21.15.16:0/368653012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f2815dd50 0x7f3f28160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.651+0000 7f3f2d7c1640 1 --2- 172.21.15.16:0/368653012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f280d34e0 0x7f3f2815fb50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.651+0000 7f3f2d7c1640 1 --2- 172.21.15.16:0/368653012 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f280b9460 0x7f3f280c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.651+0000 7f3f2d7c1640 1 -- 172.21.15.16:0/368653012 >> 172.21.15.16:0/368653012 conn(0x7f3f280722d0 msgr2=0x7f3f2815c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.651+0000 7f3f2d7c1640 1 -- 172.21.15.16:0/368653012 shutdown_connections 2024-04-25T10:06:46.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.651+0000 7f3f2d7c1640 1 -- 172.21.15.16:0/368653012 wait complete. 2024-04-25T10:06:46.668 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:06:46.668 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:06:46.668 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:06:46.669 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:46.669 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:06:46.669 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:06:46.669 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:06:46.669 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:46.669 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:06:46.670 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:06:46.670 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:06:46.670 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:46.670 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:06:46.670 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:06:46.670 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:06:46.670 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:46.679 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:06:46.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:06:46.680 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:06:46.681 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:06:46.682 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:06:46.682 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:06:46.683 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:06:46.683 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:06:46.722 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:06:46.722 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:46.723 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:06:46.723 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:46.724 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:06:46.725 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:46.725 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:06:46.725 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:46.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:06:46.797 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-04-25T10:06:46.798 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-04-25T10:06:46.803 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2024-04-25T10:06:46.803 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2024-04-25T10:06:46.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.828+0000 7fcb3e4c4640 1 Processor -- start 2024-04-25T10:06:46.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.828+0000 7fcb3e4c4640 1 -- start start 2024-04-25T10:06:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.829+0000 7fcb3e4c4640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fcb38161400 0x7fcb38162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.829+0000 7fcb3e4c4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb380cfed0 con 0x7fcb380b3f80 2024-04-25T10:06:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.829+0000 7fcb3e4c4640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb380cf850 con 0x7fcb380b35e0 2024-04-25T10:06:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.829+0000 7fcb3e4c4640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb38162890 con 0x7fcb38161400 2024-04-25T10:06:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.829+0000 7fcb377fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcb380b3f80 0x7fcb380cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36566/0 (socket says 172.21.15.16:36566) 2024-04-25T10:06:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.829+0000 7fcb377fe640 1 -- 172.21.15.16:0/3058658222 learned_addr learned my addr 172.21.15.16:0/3058658222 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.829+0000 7fcb3ca61640 1 --2- 172.21.15.16:0/3058658222 >> v2:172.21.15.186:3300/0 conn(0x7fcb38161400 0x7fcb38162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.829+0000 7fcb36ffd640 1 -- 172.21.15.16:0/3058658222 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1236339048 0 0) 0x7fcb380cfed0 con 0x7fcb380b3f80 2024-04-25T10:06:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.829+0000 7fcb36ffd640 1 -- 172.21.15.16:0/3058658222 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb24003520 con 0x7fcb380b3f80 2024-04-25T10:06:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.829+0000 7fcb3ca61640 1 -- 172.21.15.16:0/3058658222 >> v1:172.21.15.186:6789/0 conn(0x7fcb380b35e0 legacy=0x7fcb38160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.830+0000 7fcb36ffd640 1 -- 172.21.15.16:0/3058658222 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722179586 0 0) 0x7fcb380cf850 con 0x7fcb380b35e0 2024-04-25T10:06:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.830+0000 7fcb3ca61640 1 -- 172.21.15.16:0/3058658222 >> v1:172.21.15.16:6789/0 conn(0x7fcb380b3f80 legacy=0x7fcb380cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.830+0000 7fcb3ca61640 1 -- 172.21.15.16:0/3058658222 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb380cfb30 con 0x7fcb38161400 2024-04-25T10:06:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.830+0000 7fcb3ca61640 1 --2- 172.21.15.16:0/3058658222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb38161400 0x7fcb38162350 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009d90 tx=0x7fcb3002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e43fe4feacaa2954 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.831+0000 7fcb36ffd640 1 -- 172.21.15.16:0/3058658222 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30007740 con 0x7fcb38161400 2024-04-25T10:06:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.831+0000 7fcb36ffd640 1 -- 172.21.15.16:0/3058658222 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb300078e0 con 0x7fcb38161400 2024-04-25T10:06:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.831+0000 7fcb36ffd640 1 -- 172.21.15.16:0/3058658222 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30007c90 con 0x7fcb38161400 2024-04-25T10:06:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.831+0000 7fcb3e4c4640 1 -- 172.21.15.16:0/3058658222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb38161400 msgr2=0x7fcb38162350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.831+0000 7fcb3e4c4640 1 --2- 172.21.15.16:0/3058658222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb38161400 0x7fcb38162350 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009d90 tx=0x7fcb3002a660 comp rx=0 tx=0).stop 2024-04-25T10:06:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.831+0000 7fcb3e4c4640 1 -- 172.21.15.16:0/3058658222 shutdown_connections 2024-04-25T10:06:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.831+0000 7fcb3e4c4640 1 --2- 172.21.15.16:0/3058658222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb38161400 0x7fcb38162350 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.831+0000 7fcb3e4c4640 1 -- 172.21.15.16:0/3058658222 >> 172.21.15.16:0/3058658222 conn(0x7fcb3806f4c0 msgr2=0x7fcb3806f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.831+0000 7fcb3e4c4640 1 -- 172.21.15.16:0/3058658222 shutdown_connections 2024-04-25T10:06:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.831+0000 7fcb3e4c4640 1 -- 172.21.15.16:0/3058658222 wait complete. 2024-04-25T10:06:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.833+0000 7fcb3e4c4640 1 Processor -- start 2024-04-25T10:06:46.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.833+0000 7fcb3e4c4640 1 -- start start 2024-04-25T10:06:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.834+0000 7fcb3e4c4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb380b35e0 0x7fcb38152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.834+0000 7fcb3e4c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb380b3f80 0x7fcb380b4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.834+0000 7fcb377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb380b3f80 0x7fcb380b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.834+0000 7fcb377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb380b3f80 0x7fcb380b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56096/0 (socket says 172.21.15.16:56096) 2024-04-25T10:06:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.834+0000 7fcb3e4c4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb38161400 0x7fcb380cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.834+0000 7fcb377fe640 1 -- 172.21.15.16:0/2130064211 learned_addr learned my addr 172.21.15.16:0/2130064211 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.834+0000 7fcb3e4c4640 1 -- 172.21.15.16:0/2130064211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb380d05d0 con 0x7fcb380b3f80 2024-04-25T10:06:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.834+0000 7fcb3e4c4640 1 -- 172.21.15.16:0/2130064211 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb380d0450 con 0x7fcb38161400 2024-04-25T10:06:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.834+0000 7fcb3e4c4640 1 -- 172.21.15.16:0/2130064211 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb380b39c0 con 0x7fcb380b35e0 2024-04-25T10:06:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.834+0000 7fcb37fff640 1 --2- 172.21.15.16:0/2130064211 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb380b35e0 0x7fcb38152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.834+0000 7fcb3ca61640 1 --2- 172.21.15.16:0/2130064211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb38161400 0x7fcb380cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.834+0000 7fcb37fff640 1 -- 172.21.15.16:0/2130064211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb38161400 msgr2=0x7fcb380cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.834+0000 7fcb37fff640 1 --2- 172.21.15.16:0/2130064211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb38161400 0x7fcb380cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.834+0000 7fcb37fff640 1 -- 172.21.15.16:0/2130064211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb380b3f80 msgr2=0x7fcb380b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.834+0000 7fcb37fff640 1 --2- 172.21.15.16:0/2130064211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb380b3f80 0x7fcb380b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.834+0000 7fcb37fff640 1 -- 172.21.15.16:0/2130064211 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb380b4d60 con 0x7fcb380b35e0 2024-04-25T10:06:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.834+0000 7fcb3ca61640 1 --2- 172.21.15.16:0/2130064211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb38161400 0x7fcb380cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.834+0000 7fcb37fff640 1 --2- 172.21.15.16:0/2130064211 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb380b35e0 0x7fcb38152e10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000c450 tx=0x7fcb2000c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.835+0000 7fcb34ff9640 1 -- 172.21.15.16:0/2130064211 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb20017070 con 0x7fcb380b35e0 2024-04-25T10:06:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.835+0000 7fcb34ff9640 1 -- 172.21.15.16:0/2130064211 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb200035f0 con 0x7fcb380b35e0 2024-04-25T10:06:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.835+0000 7fcb34ff9640 1 -- 172.21.15.16:0/2130064211 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb200039e0 con 0x7fcb380b35e0 2024-04-25T10:06:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.835+0000 7fcb3e4c4640 1 -- 172.21.15.16:0/2130064211 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb380b5040 con 0x7fcb380b35e0 2024-04-25T10:06:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.835+0000 7fcb3e4c4640 1 -- 172.21.15.16:0/2130064211 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb380b5600 con 0x7fcb380b35e0 2024-04-25T10:06:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.836+0000 7fcb3e4c4640 1 -- 172.21.15.16:0/2130064211 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb380750b0 con 0x7fcb380b35e0 2024-04-25T10:06:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.836+0000 7fcb34ff9640 1 -- 172.21.15.16:0/2130064211 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb2001a070 con 0x7fcb380b35e0 2024-04-25T10:06:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.837+0000 7fcb34ff9640 1 --2- 172.21.15.16:0/2130064211 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb1806b500 0x7fcb1806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.837+0000 7fcb34ff9640 1 -- 172.21.15.16:0/2130064211 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcb20089460 con 0x7fcb380b35e0 2024-04-25T10:06:46.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.837+0000 7fcb377fe640 1 --2- 172.21.15.16:0/2130064211 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb1806b500 0x7fcb1806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.838+0000 7fcb377fe640 1 --2- 172.21.15.16:0/2130064211 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb1806b500 0x7fcb1806d9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcb380d0c60 tx=0x7fcb28009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:46.841+0000 7fcb34ff9640 1 -- 172.21.15.16:0/2130064211 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb20057f70 con 0x7fcb380b35e0 2024-04-25T10:06:47.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.031+0000 7fcb3e4c4640 1 -- 172.21.15.16:0/2130064211 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcb380b7850 con 0x7fcb380b35e0 2024-04-25T10:06:47.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.791+0000 7fcb34ff9640 1 -- 172.21.15.16:0/2130064211 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcb2005c010 con 0x7fcb380b35e0 2024-04-25T10:06:47.797 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-04-25T10:06:47.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.795+0000 7fcb3e4c4640 1 -- 172.21.15.16:0/2130064211 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb1806b500 msgr2=0x7fcb1806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.795+0000 7fcb3e4c4640 1 --2- 172.21.15.16:0/2130064211 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb1806b500 0x7fcb1806d9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcb380d0c60 tx=0x7fcb28009040 comp rx=0 tx=0).stop 2024-04-25T10:06:47.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.795+0000 7fcb3e4c4640 1 -- 172.21.15.16:0/2130064211 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb380b35e0 msgr2=0x7fcb38152e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.795+0000 7fcb3e4c4640 1 --2- 172.21.15.16:0/2130064211 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb380b35e0 0x7fcb38152e10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000c450 tx=0x7fcb2000c910 comp rx=0 tx=0).stop 2024-04-25T10:06:47.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.795+0000 7fcb3e4c4640 1 -- 172.21.15.16:0/2130064211 shutdown_connections 2024-04-25T10:06:47.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.795+0000 7fcb3e4c4640 1 --2- 172.21.15.16:0/2130064211 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb1806b500 0x7fcb1806d9c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.795+0000 7fcb3e4c4640 1 --2- 172.21.15.16:0/2130064211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb38161400 0x7fcb380cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.795+0000 7fcb3e4c4640 1 --2- 172.21.15.16:0/2130064211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb380b3f80 0x7fcb380b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.795+0000 7fcb3e4c4640 1 --2- 172.21.15.16:0/2130064211 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb380b35e0 0x7fcb38152e10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.795+0000 7fcb3e4c4640 1 -- 172.21.15.16:0/2130064211 >> 172.21.15.16:0/2130064211 conn(0x7fcb3806f4c0 msgr2=0x7fcb38074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.795+0000 7fcb3e4c4640 1 -- 172.21.15.16:0/2130064211 shutdown_connections 2024-04-25T10:06:47.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.795+0000 7fcb3e4c4640 1 -- 172.21.15.16:0/2130064211 wait complete. 2024-04-25T10:06:47.816 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:06:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.917+0000 7fb42afc1640 1 Processor -- start 2024-04-25T10:06:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.917+0000 7fb42afc1640 1 -- start start 2024-04-25T10:06:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.918+0000 7fb42afc1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb42409a150 0x7fb42406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.918+0000 7fb42afc1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb42406c1f0 0x7fb4240cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.918+0000 7fb42afc1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb42406c680 con 0x7fb42415ba10 2024-04-25T10:06:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.918+0000 7fb42afc1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4240cbe50 con 0x7fb42409a150 2024-04-25T10:06:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.918+0000 7fb42afc1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4240cfe80 con 0x7fb42406c1f0 2024-04-25T10:06:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.918+0000 7fb423fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb42409a150 0x7fb42406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.918+0000 7fb423fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb42409a150 0x7fb42406bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56110/0 (socket says 172.21.15.16:56110) 2024-04-25T10:06:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.918+0000 7fb423fff640 1 -- 172.21.15.16:0/2522984628 learned_addr learned my addr 172.21.15.16:0/2522984628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.918+0000 7fb428d5d640 1 --1- 172.21.15.16:0/2522984628 >> v1:172.21.15.16:6789/0 conn(0x7fb42415ba10 0x7fb424099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36570/0 (socket says 172.21.15.16:36570) 2024-04-25T10:06:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.918+0000 7fb42955e640 1 --2- 172.21.15.16:0/2522984628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb42406c1f0 0x7fb4240cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.918+0000 7fb423fff640 1 -- 172.21.15.16:0/2522984628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb42406c1f0 msgr2=0x7fb4240cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.918+0000 7fb423fff640 1 --2- 172.21.15.16:0/2522984628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb42406c1f0 0x7fb4240cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.918+0000 7fb423fff640 1 -- 172.21.15.16:0/2522984628 >> v1:172.21.15.16:6789/0 conn(0x7fb42415ba10 legacy=0x7fb424099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.918+0000 7fb423fff640 1 -- 172.21.15.16:0/2522984628 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42406c900 con 0x7fb42409a150 2024-04-25T10:06:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.918+0000 7fb4237fe640 1 -- 172.21.15.16:0/2522984628 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3589481013 0 0) 0x7fb418001110 con 0x7fb42415ba10 2024-04-25T10:06:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.918+0000 7fb42955e640 1 --2- 172.21.15.16:0/2522984628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb42406c1f0 0x7fb4240cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.919+0000 7fb423fff640 1 --2- 172.21.15.16:0/2522984628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42409a150 0x7fb42406bcb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb410009870 tx=0x7fb41002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4efdfd1cce2939d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.919+0000 7fb4237fe640 1 -- 172.21.15.16:0/2522984628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb410037070 con 0x7fb42409a150 2024-04-25T10:06:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.919+0000 7fb4237fe640 1 -- 172.21.15.16:0/2522984628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb410032440 con 0x7fb42409a150 2024-04-25T10:06:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.919+0000 7fb4237fe640 1 -- 172.21.15.16:0/2522984628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb410032810 con 0x7fb42409a150 2024-04-25T10:06:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.920+0000 7fb42afc1640 1 -- 172.21.15.16:0/2522984628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42409a150 msgr2=0x7fb42406bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.920+0000 7fb42afc1640 1 --2- 172.21.15.16:0/2522984628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42409a150 0x7fb42406bcb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb410009870 tx=0x7fb41002a270 comp rx=0 tx=0).stop 2024-04-25T10:06:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.920+0000 7fb42afc1640 1 -- 172.21.15.16:0/2522984628 shutdown_connections 2024-04-25T10:06:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.920+0000 7fb42afc1640 1 --2- 172.21.15.16:0/2522984628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb42406c1f0 0x7fb4240cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.920+0000 7fb42afc1640 1 --2- 172.21.15.16:0/2522984628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42409a150 0x7fb42406bcb0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.920+0000 7fb42afc1640 1 -- 172.21.15.16:0/2522984628 >> 172.21.15.16:0/2522984628 conn(0x7fb4240b32f0 msgr2=0x7fb4240b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.920+0000 7fb42afc1640 1 -- 172.21.15.16:0/2522984628 shutdown_connections 2024-04-25T10:06:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.920+0000 7fb42afc1640 1 -- 172.21.15.16:0/2522984628 wait complete. 2024-04-25T10:06:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.921+0000 7fb42afc1640 1 Processor -- start 2024-04-25T10:06:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.921+0000 7fb42afc1640 1 -- start start 2024-04-25T10:06:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.922+0000 7fb42afc1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb42406c1f0 0x7fb424158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.922+0000 7fb42afc1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb42409a150 0x7fb424159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.922+0000 7fb42afc1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42415ba10 0x7fb4241e8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.922+0000 7fb42afc1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4240cd760 con 0x7fb42415ba10 2024-04-25T10:06:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.922+0000 7fb42afc1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42406cec0 con 0x7fb42406c1f0 2024-04-25T10:06:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.922+0000 7fb42afc1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4240cd8e0 con 0x7fb42409a150 2024-04-25T10:06:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.922+0000 7fb42955e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42415ba10 0x7fb4241e8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.922+0000 7fb42955e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42415ba10 0x7fb4241e8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56112/0 (socket says 172.21.15.16:56112) 2024-04-25T10:06:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.922+0000 7fb42955e640 1 -- 172.21.15.16:0/2479444977 learned_addr learned my addr 172.21.15.16:0/2479444977 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.922+0000 7fb423fff640 1 --2- 172.21.15.16:0/2479444977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb42409a150 0x7fb424159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.922+0000 7fb428d5d640 1 --2- 172.21.15.16:0/2479444977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb42406c1f0 0x7fb424158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.922+0000 7fb42955e640 1 -- 172.21.15.16:0/2479444977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb42409a150 msgr2=0x7fb424159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.922+0000 7fb42955e640 1 --2- 172.21.15.16:0/2479444977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb42409a150 0x7fb424159d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.922+0000 7fb42955e640 1 -- 172.21.15.16:0/2479444977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb42406c1f0 msgr2=0x7fb424158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.922+0000 7fb42955e640 1 --2- 172.21.15.16:0/2479444977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb42406c1f0 0x7fb424158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.922+0000 7fb42955e640 1 -- 172.21.15.16:0/2479444977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42415a2a0 con 0x7fb42415ba10 2024-04-25T10:06:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.923+0000 7fb42955e640 1 --2- 172.21.15.16:0/2479444977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42415ba10 0x7fb4241e8700 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00cef0 tx=0x7fb41c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.923+0000 7fb4217fa640 1 -- 172.21.15.16:0/2479444977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c01a070 con 0x7fb42415ba10 2024-04-25T10:06:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.923+0000 7fb4217fa640 1 -- 172.21.15.16:0/2479444977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41c00d860 con 0x7fb42415ba10 2024-04-25T10:06:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.923+0000 7fb4217fa640 1 -- 172.21.15.16:0/2479444977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c00dc50 con 0x7fb42415ba10 2024-04-25T10:06:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.923+0000 7fb42afc1640 1 -- 172.21.15.16:0/2479444977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4241e7660 con 0x7fb42415ba10 2024-04-25T10:06:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.923+0000 7fb42afc1640 1 -- 172.21.15.16:0/2479444977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4241e7c20 con 0x7fb42415ba10 2024-04-25T10:06:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.924+0000 7fb42afc1640 1 -- 172.21.15.16:0/2479444977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4240749b0 con 0x7fb42415ba10 2024-04-25T10:06:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.924+0000 7fb4217fa640 1 -- 172.21.15.16:0/2479444977 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb41c01d070 con 0x7fb42415ba10 2024-04-25T10:06:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.925+0000 7fb4217fa640 1 --2- 172.21.15.16:0/2479444977 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb40806b4d0 0x7fb40806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.925+0000 7fb4217fa640 1 -- 172.21.15.16:0/2479444977 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb41c08a790 con 0x7fb42415ba10 2024-04-25T10:06:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.925+0000 7fb428d5d640 1 --2- 172.21.15.16:0/2479444977 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb40806b4d0 0x7fb40806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.925+0000 7fb428d5d640 1 --2- 172.21.15.16:0/2479444977 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb40806b4d0 0x7fb40806d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb418000f60 tx=0x7fb418007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:47.929+0000 7fb4217fa640 1 -- 172.21.15.16:0/2479444977 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb41c059a10 con 0x7fb42415ba10 2024-04-25T10:06:48.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.183+0000 7fb42afc1640 1 -- 172.21.15.16:0/2479444977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb4241612c0 con 0x7fb42415ba10 2024-04-25T10:06:48.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.184+0000 7fb4217fa640 1 -- 172.21.15.16:0/2479444977 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb41c059310 con 0x7fb42415ba10 2024-04-25T10:06:48.190 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:48.190 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:06:48.190 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-04-25T10:06:48.190 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.190 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-04-25T10:06:48.190 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmj+xmBBAAaWdHmoiYcXzFnMVQSioT2Q==", 2024-04-25T10:06:48.190 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.190 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:06:48.190 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:06:48.190 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:06:48.191 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:48.191 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:48.191 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.191 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-04-25T10:06:48.191 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmUES+DxAAu5It7dQmb7/BM/4x1fUYXA==", 2024-04-25T10:06:48.191 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.191 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:06:48.191 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:06:48.191 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:06:48.191 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:48.191 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:48.191 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.192 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-04-25T10:06:48.192 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipm+LtFCxAAGG79kv8eWzly8l5qzQwfww==", 2024-04-25T10:06:48.192 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.192 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:06:48.192 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:06:48.192 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:06:48.192 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:48.192 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:48.192 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.192 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-04-25T10:06:48.192 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmqbSSFhAAZtrLsAGSyaMGgv45Hy0WpA==", 2024-04-25T10:06:48.192 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.192 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:06:48.192 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:06:48.193 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:06:48.193 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:48.193 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.193 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:48.193 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.193 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-04-25T10:06:48.193 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCaKipm2+UdOxAAP8GBunvYsue8EILYV5Uq7A==", 2024-04-25T10:06:48.193 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.193 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:48.193 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:06:48.193 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:48.193 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.193 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:48.193 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.194 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-04-25T10:06:48.194 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCdKipmSv87ORAAigVU7nyzkcXqMEHKsSAsqg==", 2024-04-25T10:06:48.194 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.194 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:48.194 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:06:48.194 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:48.194 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.194 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:48.194 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.194 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-04-25T10:06:48.194 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCgKipm2VMLOBAAJETFfWgq7H/imQAfw93Ruw==", 2024-04-25T10:06:48.194 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.194 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:48.194 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:06:48.195 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:48.195 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:48.195 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.195 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-04-25T10:06:48.195 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCjKipmb1g0NhAADH2uHQPhW84KLYyWbOjQdw==", 2024-04-25T10:06:48.195 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.195 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:48.195 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:06:48.195 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:48.195 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:48.196 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.196 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-04-25T10:06:48.196 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCpKipmnH0ROBAAEb8i4nfswYQ/w5mEw48kIg==", 2024-04-25T10:06:48.196 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.196 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:48.196 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:06:48.196 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:48.196 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:48.196 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.197 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-04-25T10:06:48.197 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCsKipm8HCRNhAA3FhcMlUWxMAiFRXbBWWvNA==", 2024-04-25T10:06:48.197 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.197 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:48.197 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:06:48.197 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:48.197 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:48.197 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.197 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-04-25T10:06:48.197 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCvKipmfgqpNRAAtPhpSDU9q23Sfa+V7m+3NA==", 2024-04-25T10:06:48.197 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.197 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:48.197 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:06:48.198 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:48.198 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:48.198 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.198 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-04-25T10:06:48.198 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCyKipm3ndyMxAAeMNODGxNupb043CsA/SP1A==", 2024-04-25T10:06:48.198 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.198 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:48.198 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:06:48.198 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:48.198 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:48.198 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.199 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-04-25T10:06:48.199 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmAph/GhAAG3x1zC98yHQ13fjpmrlPkg==", 2024-04-25T10:06:48.199 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.199 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:06:48.199 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-04-25T10:06:48.199 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:06:48.199 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:48.199 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.199 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-04-25T10:06:48.199 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmrccTIxAAuDZamFXTe6J2d+HAcBe+DA==", 2024-04-25T10:06:48.199 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.199 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:06:48.199 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:06:48.200 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:06:48.200 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:06:48.200 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:48.200 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.200 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-04-25T10:06:48.200 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmsFACIBAAFuWsmuQlF58yzIcYUx/6Iw==", 2024-04-25T10:06:48.200 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.201 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:06:48.201 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:06:48.201 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:06:48.201 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:06:48.201 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:48.201 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.202 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-04-25T10:06:48.202 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmAe1/KBAAXzLkscuwpCwMnrIKQny9+Q==", 2024-04-25T10:06:48.202 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.202 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:06:48.202 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:06:48.202 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:06:48.202 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:06:48.202 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:48.203 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.203 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-04-25T10:06:48.203 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCYKipmVFQ2JBAAdqG2p/M746PZPMDcpQjwfw==", 2024-04-25T10:06:48.203 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.203 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:06:48.203 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:06:48.203 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:06:48.203 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:48.203 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:48.203 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.203 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-04-25T10:06:48.203 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmiPVHARAAGPbv3P3yVj3tlo0b2gmcyw==", 2024-04-25T10:06:48.203 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.204 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:06:48.204 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-04-25T10:06:48.204 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:48.204 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:06:48.204 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:48.204 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-04-25T10:06:48.204 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCYKipmghhyOBAAcCm0hQzU+Wlw45yZQETTTg==", 2024-04-25T10:06:48.204 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:06:48.204 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:06:48.204 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-04-25T10:06:48.204 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:06:48.204 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.204 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:48.205 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-04-25T10:06:48.205 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:06:48.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.185+0000 7fb42afc1640 1 -- 172.21.15.16:0/2479444977 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb40806b4d0 msgr2=0x7fb40806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.185+0000 7fb42afc1640 1 --2- 172.21.15.16:0/2479444977 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb40806b4d0 0x7fb40806d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb418000f60 tx=0x7fb418007c00 comp rx=0 tx=0).stop 2024-04-25T10:06:48.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.185+0000 7fb42afc1640 1 -- 172.21.15.16:0/2479444977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42415ba10 msgr2=0x7fb4241e8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.185+0000 7fb42afc1640 1 --2- 172.21.15.16:0/2479444977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42415ba10 0x7fb4241e8700 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00cef0 tx=0x7fb41c00a710 comp rx=0 tx=0).stop 2024-04-25T10:06:48.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.186+0000 7fb42afc1640 1 -- 172.21.15.16:0/2479444977 shutdown_connections 2024-04-25T10:06:48.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.186+0000 7fb42afc1640 1 --2- 172.21.15.16:0/2479444977 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb40806b4d0 0x7fb40806d990 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.186+0000 7fb42afc1640 1 --2- 172.21.15.16:0/2479444977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42415ba10 0x7fb4241e8700 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.186+0000 7fb42afc1640 1 --2- 172.21.15.16:0/2479444977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb42409a150 0x7fb424159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.186+0000 7fb42afc1640 1 --2- 172.21.15.16:0/2479444977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb42406c1f0 0x7fb424158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.186+0000 7fb42afc1640 1 -- 172.21.15.16:0/2479444977 >> 172.21.15.16:0/2479444977 conn(0x7fb4240b32f0 msgr2=0x7fb4241616d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:48.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.186+0000 7fb42afc1640 1 -- 172.21.15.16:0/2479444977 shutdown_connections 2024-04-25T10:06:48.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.186+0000 7fb42afc1640 1 -- 172.21.15.16:0/2479444977 wait complete. 2024-04-25T10:06:48.208 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.303+0000 7f487fb5c640 1 Processor -- start 2024-04-25T10:06:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.303+0000 7f487fb5c640 1 -- start start 2024-04-25T10:06:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.304+0000 7f487fb5c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f48780d35b0 0x7f48780b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.304+0000 7f487fb5c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487815e0c0 con 0x7f48780b5990 2024-04-25T10:06:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.304+0000 7f487fb5c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48780d2510 con 0x7f487815dce0 2024-04-25T10:06:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.304+0000 7f487fb5c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48780d27f0 con 0x7f48780d35b0 2024-04-25T10:06:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.304+0000 7f487e0f9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f48780b5990 0x7f48780b6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36572/0 (socket says 172.21.15.16:36572) 2024-04-25T10:06:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.304+0000 7f487e0f9640 1 -- 172.21.15.16:0/2586860801 learned_addr learned my addr 172.21.15.16:0/2586860801 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.304+0000 7f487c8f6640 1 -- 172.21.15.16:0/2586860801 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3222402769 0 0) 0x7f487815e0c0 con 0x7f48780b5990 2024-04-25T10:06:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.304+0000 7f487c8f6640 1 -- 172.21.15.16:0/2586860801 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4864003520 con 0x7f48780b5990 2024-04-25T10:06:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.304+0000 7f487d0f7640 1 --2- 172.21.15.16:0/2586860801 >> v2:172.21.15.186:3300/0 conn(0x7f48780d35b0 0x7f48780b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.304+0000 7f487c8f6640 1 -- 172.21.15.16:0/2586860801 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1133574105 0 0) 0x7f4864003520 con 0x7f48780b5990 2024-04-25T10:06:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.304+0000 7f487c8f6640 1 -- 172.21.15.16:0/2586860801 >> v2:172.21.15.186:3300/0 conn(0x7f48780d35b0 msgr2=0x7f48780b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.304+0000 7f487c8f6640 1 --2- 172.21.15.16:0/2586860801 >> v2:172.21.15.186:3300/0 conn(0x7f48780d35b0 0x7f48780b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.304+0000 7f487c8f6640 1 -- 172.21.15.16:0/2586860801 >> v1:172.21.15.186:6789/0 conn(0x7f487815dce0 legacy=0x7f48780d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.304+0000 7f487c8f6640 1 -- 172.21.15.16:0/2586860801 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48780d2ad0 con 0x7f48780b5990 2024-04-25T10:06:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.304+0000 7f487c8f6640 1 -- 172.21.15.16:0/2586860801 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f48700019f0 con 0x7f48780b5990 2024-04-25T10:06:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.304+0000 7f487c8f6640 1 -- 172.21.15.16:0/2586860801 >> v1:172.21.15.16:6789/0 conn(0x7f48780b5990 legacy=0x7f48780b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.304+0000 7f487c8f6640 1 --2- 172.21.15.16:0/2586860801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48640039b0 0x7f4864003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.304+0000 7f487c8f6640 1 --2- 172.21.15.16:0/2586860801 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4864004800 0x7f4864004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.305+0000 7f487c8f6640 1 --2- 172.21.15.16:0/2586860801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48640051c0 0x7f486400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.305+0000 7f487c8f6640 1 -- 172.21.15.16:0/2586860801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48780d27f0 con 0x7f48640039b0 2024-04-25T10:06:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.305+0000 7f487c8f6640 1 -- 172.21.15.16:0/2586860801 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486400c110 con 0x7f48640051c0 2024-04-25T10:06:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.305+0000 7f487c8f6640 1 -- 172.21.15.16:0/2586860801 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486400c310 con 0x7f4864004800 2024-04-25T10:06:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.305+0000 7f487d8f8640 1 --2- 172.21.15.16:0/2586860801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48640039b0 0x7f4864003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.305+0000 7f487d0f7640 1 --2- 172.21.15.16:0/2586860801 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4864004800 0x7f4864004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.305+0000 7f487e0f9640 1 --2- 172.21.15.16:0/2586860801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48640051c0 0x7f486400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.305+0000 7f487d8f8640 1 -- 172.21.15.16:0/2586860801 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4864004800 msgr2=0x7f4864004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.305+0000 7f487d8f8640 1 --2- 172.21.15.16:0/2586860801 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4864004800 0x7f4864004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.305+0000 7f487d8f8640 1 -- 172.21.15.16:0/2586860801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48640051c0 msgr2=0x7f486400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.305+0000 7f487d8f8640 1 --2- 172.21.15.16:0/2586860801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48640051c0 0x7f486400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.305+0000 7f487d8f8640 1 -- 172.21.15.16:0/2586860801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4864003520 con 0x7f48640039b0 2024-04-25T10:06:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.305+0000 7f487e0f9640 1 --2- 172.21.15.16:0/2586860801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48640051c0 0x7f486400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.305+0000 7f487e0f9640 1 -- 172.21.15.16:0/2586860801 reap_dead start 2024-04-25T10:06:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.306+0000 7f487d8f8640 1 --2- 172.21.15.16:0/2586860801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48640039b0 0x7f4864003dd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4874001750 tx=0x7f4864005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a02b1b105b6a526 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.306+0000 7f487c8f6640 1 -- 172.21.15.16:0/2586860801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874001370 con 0x7f48640039b0 2024-04-25T10:06:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.306+0000 7f487c8f6640 1 -- 172.21.15.16:0/2586860801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48740052d0 con 0x7f48640039b0 2024-04-25T10:06:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.306+0000 7f487c8f6640 1 -- 172.21.15.16:0/2586860801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487402abc0 con 0x7f48640039b0 2024-04-25T10:06:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.306+0000 7f487fb5c640 1 -- 172.21.15.16:0/2586860801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48640039b0 msgr2=0x7f4864003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.306+0000 7f487fb5c640 1 --2- 172.21.15.16:0/2586860801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48640039b0 0x7f4864003dd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4874001750 tx=0x7f4864005830 comp rx=0 tx=0).stop 2024-04-25T10:06:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.306+0000 7f487fb5c640 1 -- 172.21.15.16:0/2586860801 shutdown_connections 2024-04-25T10:06:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.306+0000 7f487fb5c640 1 --2- 172.21.15.16:0/2586860801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48640039b0 0x7f4864003dd0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.306+0000 7f487fb5c640 1 -- 172.21.15.16:0/2586860801 >> 172.21.15.16:0/2586860801 conn(0x7f4878072900 msgr2=0x7f48780d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.306+0000 7f487fb5c640 1 -- 172.21.15.16:0/2586860801 shutdown_connections 2024-04-25T10:06:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.306+0000 7f487fb5c640 1 -- 172.21.15.16:0/2586860801 wait complete. 2024-04-25T10:06:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.308+0000 7f487fb5c640 1 Processor -- start 2024-04-25T10:06:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.308+0000 7f487fb5c640 1 -- start start 2024-04-25T10:06:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.308+0000 7f487fb5c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48640039b0 0x7f4878160f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.308+0000 7f487fb5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4878153a40 0x7f48780cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.308+0000 7f487fb5c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4878153ea0 0x7f48780cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.308+0000 7f487fb5c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48780b9820 con 0x7f4878153a40 2024-04-25T10:06:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.308+0000 7f487fb5c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48780d3090 con 0x7f48640039b0 2024-04-25T10:06:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.308+0000 7f487fb5c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48780b99a0 con 0x7f4878153ea0 2024-04-25T10:06:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.308+0000 7f487d0f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4878153a40 0x7f48780cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.309+0000 7f487d0f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4878153a40 0x7f48780cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56144/0 (socket says 172.21.15.16:56144) 2024-04-25T10:06:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.309+0000 7f487d0f7640 1 -- 172.21.15.16:0/972953468 learned_addr learned my addr 172.21.15.16:0/972953468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.309+0000 7f487d0f7640 1 -- 172.21.15.16:0/972953468 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4878153ea0 msgr2=0x7f48780cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.309+0000 7f487d8f8640 1 --2- 172.21.15.16:0/972953468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48640039b0 0x7f4878160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.309+0000 7f487e0f9640 1 --2- 172.21.15.16:0/972953468 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4878153ea0 0x7f48780cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.309+0000 7f487d0f7640 1 --2- 172.21.15.16:0/972953468 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4878153ea0 0x7f48780cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.309+0000 7f487d0f7640 1 -- 172.21.15.16:0/972953468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48640039b0 msgr2=0x7f4878160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.309+0000 7f487d0f7640 1 --2- 172.21.15.16:0/972953468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48640039b0 0x7f4878160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.309+0000 7f487d0f7640 1 -- 172.21.15.16:0/972953468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487815a350 con 0x7f4878153a40 2024-04-25T10:06:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.309+0000 7f487d0f7640 1 --2- 172.21.15.16:0/972953468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4878153a40 0x7f48780cc0a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f486800b820 tx=0x7f486800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.309+0000 7f486e7fc640 1 -- 172.21.15.16:0/972953468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868018070 con 0x7f4878153a40 2024-04-25T10:06:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.309+0000 7f486e7fc640 1 -- 172.21.15.16:0/972953468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486800be10 con 0x7f4878153a40 2024-04-25T10:06:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.309+0000 7f487fb5c640 1 -- 172.21.15.16:0/972953468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4878159d90 con 0x7f4878153a40 2024-04-25T10:06:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.309+0000 7f486e7fc640 1 -- 172.21.15.16:0/972953468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868010c60 con 0x7f4878153a40 2024-04-25T10:06:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.309+0000 7f487fb5c640 1 -- 172.21.15.16:0/972953468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48781544e0 con 0x7f4878153a40 2024-04-25T10:06:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.310+0000 7f487fb5c640 1 -- 172.21.15.16:0/972953468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48780749b0 con 0x7f4878153a40 2024-04-25T10:06:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.311+0000 7f486e7fc640 1 -- 172.21.15.16:0/972953468 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f486801b070 con 0x7f4878153a40 2024-04-25T10:06:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.311+0000 7f486e7fc640 1 --2- 172.21.15.16:0/972953468 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f485c06b4d0 0x7f485c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.311+0000 7f486e7fc640 1 -- 172.21.15.16:0/972953468 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f486808a2c0 con 0x7f4878153a40 2024-04-25T10:06:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.311+0000 7f487d8f8640 1 --2- 172.21.15.16:0/972953468 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f485c06b4d0 0x7f485c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.312+0000 7f487d8f8640 1 --2- 172.21.15.16:0/972953468 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f485c06b4d0 0x7f485c06d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4874032000 tx=0x7f487402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.315+0000 7f486e7fc640 1 -- 172.21.15.16:0/972953468 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48680593f0 con 0x7f4878153a40 2024-04-25T10:06:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.497+0000 7f487fb5c640 1 -- 172.21.15.16:0/972953468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f487815d0a0 con 0x7f4878153a40 2024-04-25T10:06:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.498+0000 7f486e7fc640 1 -- 172.21.15.16:0/972953468 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+12891 (secure 0 0 0) 0x7f4868058cf0 con 0x7f4878153a40 2024-04-25T10:06:48.504 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:48.504 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":43,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:06:47.797072+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.500+0000 7f487fb5c640 1 -- 172.21.15.16:0/972953468 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f485c06b4d0 msgr2=0x7f485c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.500+0000 7f487fb5c640 1 --2- 172.21.15.16:0/972953468 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f485c06b4d0 0x7f485c06d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4874032000 tx=0x7f487402e000 comp rx=0 tx=0).stop 2024-04-25T10:06:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.500+0000 7f487fb5c640 1 -- 172.21.15.16:0/972953468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4878153a40 msgr2=0x7f48780cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.500+0000 7f487fb5c640 1 --2- 172.21.15.16:0/972953468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4878153a40 0x7f48780cc0a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f486800b820 tx=0x7f486800bce0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.500+0000 7f487fb5c640 1 -- 172.21.15.16:0/972953468 shutdown_connections 2024-04-25T10:06:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.500+0000 7f487fb5c640 1 --2- 172.21.15.16:0/972953468 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f485c06b4d0 0x7f485c06d990 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.500+0000 7f487fb5c640 1 --2- 172.21.15.16:0/972953468 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4878153ea0 0x7f48780cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.500+0000 7f487fb5c640 1 --2- 172.21.15.16:0/972953468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4878153a40 0x7f48780cc0a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.500+0000 7f487fb5c640 1 --2- 172.21.15.16:0/972953468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48640039b0 0x7f4878160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.500+0000 7f487fb5c640 1 -- 172.21.15.16:0/972953468 >> 172.21.15.16:0/972953468 conn(0x7f4878072900 msgr2=0x7f48780b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:48.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.500+0000 7f487fb5c640 1 -- 172.21.15.16:0/972953468 shutdown_connections 2024-04-25T10:06:48.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.500+0000 7f487fb5c640 1 -- 172.21.15.16:0/972953468 wait complete. 2024-04-25T10:06:48.517 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:48.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.615+0000 7f367849e640 1 Processor -- start 2024-04-25T10:06:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.615+0000 7f367849e640 1 -- start start 2024-04-25T10:06:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.616+0000 7f367849e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f367015dca0 0x7f36700ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.616+0000 7f367849e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36700d1130 0x7f36700b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.616+0000 7f367849e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36700d4090 con 0x7f36700d3b90 2024-04-25T10:06:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.616+0000 7f367849e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36700b6b20 con 0x7f36700d1130 2024-04-25T10:06:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.616+0000 7f367849e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36700d4370 con 0x7f367015dca0 2024-04-25T10:06:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.616+0000 7f3676a3b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36700d1130 0x7f36700b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.616+0000 7f3676a3b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36700d1130 0x7f36700b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56158/0 (socket says 172.21.15.16:56158) 2024-04-25T10:06:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.616+0000 7f3676a3b640 1 -- 172.21.15.16:0/2624436205 learned_addr learned my addr 172.21.15.16:0/2624436205 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.616+0000 7f367623a640 1 --2- 172.21.15.16:0/2624436205 >> v2:172.21.15.186:3300/0 conn(0x7f367015dca0 0x7f36700ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.617+0000 7f3676a3b640 1 -- 172.21.15.16:0/2624436205 >> v2:172.21.15.186:3300/0 conn(0x7f367015dca0 msgr2=0x7f36700ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.617+0000 7f3676a3b640 1 --2- 172.21.15.16:0/2624436205 >> v2:172.21.15.186:3300/0 conn(0x7f367015dca0 0x7f36700ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.617+0000 7f3676a3b640 1 -- 172.21.15.16:0/2624436205 >> v1:172.21.15.186:6789/0 conn(0x7f36700d3b90 legacy=0x7f36700d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.617+0000 7f3676a3b640 1 -- 172.21.15.16:0/2624436205 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36700d4650 con 0x7f36700d1130 2024-04-25T10:06:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.617+0000 7f367623a640 1 --2- 172.21.15.16:0/2624436205 >> v2:172.21.15.186:3300/0 conn(0x7f367015dca0 0x7f36700ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.617+0000 7f3676a3b640 1 --2- 172.21.15.16:0/2624436205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36700d1130 0x7f36700b2500 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f36700b6340 tx=0x7f366802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8cd42e074b4a65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.617+0000 7f3675238640 1 -- 172.21.15.16:0/2624436205 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3668037070 con 0x7f36700d1130 2024-04-25T10:06:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.617+0000 7f3675238640 1 -- 172.21.15.16:0/2624436205 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3668032440 con 0x7f36700d1130 2024-04-25T10:06:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.617+0000 7f3675238640 1 -- 172.21.15.16:0/2624436205 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3668032810 con 0x7f36700d1130 2024-04-25T10:06:48.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.618+0000 7f367849e640 1 -- 172.21.15.16:0/2624436205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36700d1130 msgr2=0x7f36700b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.618+0000 7f367849e640 1 --2- 172.21.15.16:0/2624436205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36700d1130 0x7f36700b2500 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f36700b6340 tx=0x7f366802a230 comp rx=0 tx=0).stop 2024-04-25T10:06:48.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.618+0000 7f367849e640 1 -- 172.21.15.16:0/2624436205 shutdown_connections 2024-04-25T10:06:48.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.618+0000 7f367849e640 1 --2- 172.21.15.16:0/2624436205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36700d1130 0x7f36700b2500 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.618+0000 7f367849e640 1 --2- 172.21.15.16:0/2624436205 >> v2:172.21.15.186:3300/0 conn(0x7f367015dca0 0x7f36700ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.618+0000 7f367849e640 1 -- 172.21.15.16:0/2624436205 >> 172.21.15.16:0/2624436205 conn(0x7f36700726c0 msgr2=0x7f3670072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:48.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.618+0000 7f367849e640 1 -- 172.21.15.16:0/2624436205 shutdown_connections 2024-04-25T10:06:48.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.618+0000 7f367849e640 1 -- 172.21.15.16:0/2624436205 wait complete. 2024-04-25T10:06:48.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.619+0000 7f367849e640 1 Processor -- start 2024-04-25T10:06:48.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.619+0000 7f367849e640 1 -- start start 2024-04-25T10:06:48.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.620+0000 7f367849e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36700d1130 0x7f367015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.620+0000 7f367849e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36700d3b90 0x7f36700b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.620+0000 7f367849e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f367015dca0 0x7f36700b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.620+0000 7f367849e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367015e200 con 0x7f367015dca0 2024-04-25T10:06:48.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.620+0000 7f367849e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367015e080 con 0x7f36700d3b90 2024-04-25T10:06:48.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.620+0000 7f367849e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36700cee80 con 0x7f36700d1130 2024-04-25T10:06:48.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.620+0000 7f3676a3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f367015dca0 0x7f36700b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.620+0000 7f3676a3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f367015dca0 0x7f36700b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56166/0 (socket says 172.21.15.16:56166) 2024-04-25T10:06:48.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.620+0000 7f3676a3b640 1 -- 172.21.15.16:0/1462137363 learned_addr learned my addr 172.21.15.16:0/1462137363 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:48.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.620+0000 7f3675a39640 1 --2- 172.21.15.16:0/1462137363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36700d3b90 0x7f36700b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.620+0000 7f367623a640 1 --2- 172.21.15.16:0/1462137363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36700d1130 0x7f367015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.620+0000 7f3676a3b640 1 -- 172.21.15.16:0/1462137363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36700d1130 msgr2=0x7f367015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.620+0000 7f3676a3b640 1 --2- 172.21.15.16:0/1462137363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36700d1130 0x7f367015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.620+0000 7f3676a3b640 1 -- 172.21.15.16:0/1462137363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36700d3b90 msgr2=0x7f36700b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.620+0000 7f3676a3b640 1 --2- 172.21.15.16:0/1462137363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36700d3b90 0x7f36700b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.620+0000 7f3676a3b640 1 -- 172.21.15.16:0/1462137363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36701f38f0 con 0x7f367015dca0 2024-04-25T10:06:48.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.620+0000 7f3675a39640 1 --2- 172.21.15.16:0/1462137363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36700d3b90 0x7f36700b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:48.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.620+0000 7f367623a640 1 --2- 172.21.15.16:0/1462137363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36700d1130 0x7f367015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:48.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.621+0000 7f3676a3b640 1 --2- 172.21.15.16:0/1462137363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f367015dca0 0x7f36700b41c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f36680090b0 tx=0x7f36680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.621+0000 7f366effd640 1 -- 172.21.15.16:0/1462137363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3668041020 con 0x7f367015dca0 2024-04-25T10:06:48.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.621+0000 7f366effd640 1 -- 172.21.15.16:0/1462137363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3668030990 con 0x7f367015dca0 2024-04-25T10:06:48.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.621+0000 7f367849e640 1 -- 172.21.15.16:0/1462137363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36700cc1c0 con 0x7f367015dca0 2024-04-25T10:06:48.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.621+0000 7f366effd640 1 -- 172.21.15.16:0/1462137363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3668037040 con 0x7f367015dca0 2024-04-25T10:06:48.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.621+0000 7f367849e640 1 -- 172.21.15.16:0/1462137363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36700cc780 con 0x7f367015dca0 2024-04-25T10:06:48.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.622+0000 7f367849e640 1 -- 172.21.15.16:0/1462137363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36700750b0 con 0x7f367015dca0 2024-04-25T10:06:48.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.622+0000 7f366effd640 1 -- 172.21.15.16:0/1462137363 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3668044020 con 0x7f367015dca0 2024-04-25T10:06:48.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.623+0000 7f366effd640 1 --2- 172.21.15.16:0/1462137363 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f365c06b4d0 0x7f365c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.623+0000 7f366effd640 1 -- 172.21.15.16:0/1462137363 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f36680aa520 con 0x7f367015dca0 2024-04-25T10:06:48.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.623+0000 7f367623a640 1 --2- 172.21.15.16:0/1462137363 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f365c06b4d0 0x7f365c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.623+0000 7f367623a640 1 --2- 172.21.15.16:0/1462137363 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f365c06b4d0 0x7f365c06d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3660007f40 tx=0x7f3660007490 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.627+0000 7f366effd640 1 -- 172.21.15.16:0/1462137363 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f366804b070 con 0x7f367015dca0 2024-04-25T10:06:48.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.811+0000 7f367849e640 1 -- 172.21.15.16:0/1462137363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f36700c7340 con 0x7f367015dca0 2024-04-25T10:06:48.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.812+0000 7f366effd640 1 -- 172.21.15.16:0/1462137363 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f36680796f0 con 0x7f367015dca0 2024-04-25T10:06:48.817 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:06:48.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.814+0000 7f367849e640 1 -- 172.21.15.16:0/1462137363 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f365c06b4d0 msgr2=0x7f365c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.814+0000 7f367849e640 1 --2- 172.21.15.16:0/1462137363 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f365c06b4d0 0x7f365c06d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3660007f40 tx=0x7f3660007490 comp rx=0 tx=0).stop 2024-04-25T10:06:48.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.814+0000 7f367849e640 1 -- 172.21.15.16:0/1462137363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f367015dca0 msgr2=0x7f36700b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.814+0000 7f367849e640 1 --2- 172.21.15.16:0/1462137363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f367015dca0 0x7f36700b41c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f36680090b0 tx=0x7f36680046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.814+0000 7f367849e640 1 -- 172.21.15.16:0/1462137363 shutdown_connections 2024-04-25T10:06:48.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.814+0000 7f367849e640 1 --2- 172.21.15.16:0/1462137363 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f365c06b4d0 0x7f365c06d990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.814+0000 7f367849e640 1 --2- 172.21.15.16:0/1462137363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f367015dca0 0x7f36700b41c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.814+0000 7f367849e640 1 --2- 172.21.15.16:0/1462137363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36700d3b90 0x7f36700b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.814+0000 7f367849e640 1 --2- 172.21.15.16:0/1462137363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36700d1130 0x7f367015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.814+0000 7f367849e640 1 -- 172.21.15.16:0/1462137363 >> 172.21.15.16:0/1462137363 conn(0x7f36700726c0 msgr2=0x7f36700ce1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:48.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.814+0000 7f367849e640 1 -- 172.21.15.16:0/1462137363 shutdown_connections 2024-04-25T10:06:48.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.814+0000 7f367849e640 1 -- 172.21.15.16:0/1462137363 wait complete. 2024-04-25T10:06:48.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:06:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.926+0000 7f90d3d7e640 1 Processor -- start 2024-04-25T10:06:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.926+0000 7f90d3d7e640 1 -- start start 2024-04-25T10:06:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.927+0000 7f90d3d7e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90cc15dce0 0x7f90cc0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.927+0000 7f90d3d7e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90cc15e0c0 con 0x7f90cc0d35b0 2024-04-25T10:06:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.927+0000 7f90d3d7e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90cc0d2510 con 0x7f90cc0b5b60 2024-04-25T10:06:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.927+0000 7f90d3d7e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90cc0d27f0 con 0x7f90cc15dce0 2024-04-25T10:06:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.927+0000 7f90d1b1a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90cc15dce0 0x7f90cc0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.927+0000 7f90d1319640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f90cc0d35b0 0x7f90cc0b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36586/0 (socket says 172.21.15.16:36586) 2024-04-25T10:06:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.927+0000 7f90d1319640 1 -- 172.21.15.16:0/3198746314 learned_addr learned my addr 172.21.15.16:0/3198746314 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.927+0000 7f90d1b1a640 1 -- 172.21.15.16:0/3198746314 >> v1:172.21.15.186:6789/0 conn(0x7f90cc0b5b60 legacy=0x7f90cc0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.927+0000 7f90d1b1a640 1 -- 172.21.15.16:0/3198746314 >> v1:172.21.15.16:6789/0 conn(0x7f90cc0d35b0 legacy=0x7f90cc0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.927+0000 7f90d1b1a640 1 -- 172.21.15.16:0/3198746314 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc0d2ad0 con 0x7f90cc15dce0 2024-04-25T10:06:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.928+0000 7f90d1b1a640 1 --2- 172.21.15.16:0/3198746314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc15dce0 0x7f90cc0d1a50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f90c80097a0 tx=0x7f90c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9914da90ebe1c97b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.928+0000 7f90d0b18640 1 -- 172.21.15.16:0/3198746314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c8037070 con 0x7f90cc15dce0 2024-04-25T10:06:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.928+0000 7f90d0b18640 1 -- 172.21.15.16:0/3198746314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c8032440 con 0x7f90cc15dce0 2024-04-25T10:06:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.928+0000 7f90d0b18640 1 -- 172.21.15.16:0/3198746314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c8032810 con 0x7f90cc15dce0 2024-04-25T10:06:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.929+0000 7f90d3d7e640 1 -- 172.21.15.16:0/3198746314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc15dce0 msgr2=0x7f90cc0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.929+0000 7f90d3d7e640 1 --2- 172.21.15.16:0/3198746314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc15dce0 0x7f90cc0d1a50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f90c80097a0 tx=0x7f90c802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.929+0000 7f90d3d7e640 1 -- 172.21.15.16:0/3198746314 shutdown_connections 2024-04-25T10:06:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.929+0000 7f90d3d7e640 1 --2- 172.21.15.16:0/3198746314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc15dce0 0x7f90cc0d1a50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.929+0000 7f90d3d7e640 1 -- 172.21.15.16:0/3198746314 >> 172.21.15.16:0/3198746314 conn(0x7f90cc072900 msgr2=0x7f90cc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.929+0000 7f90d3d7e640 1 -- 172.21.15.16:0/3198746314 shutdown_connections 2024-04-25T10:06:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.929+0000 7f90d3d7e640 1 -- 172.21.15.16:0/3198746314 wait complete. 2024-04-25T10:06:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.930+0000 7f90d3d7e640 1 Processor -- start 2024-04-25T10:06:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.930+0000 7f90d3d7e640 1 -- start start 2024-04-25T10:06:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.930+0000 7f90d3d7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc0b5b60 0x7f90cc0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90d3d7e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90cc0d35b0 0x7f90cc159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90d3d7e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90cc15dce0 0x7f90cc0b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90d3d7e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc0b9820 con 0x7f90cc0b5b60 2024-04-25T10:06:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90d3d7e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc0d3090 con 0x7f90cc0d35b0 2024-04-25T10:06:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90d3d7e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc0b99a0 con 0x7f90cc15dce0 2024-04-25T10:06:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90d1b1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc0b5b60 0x7f90cc0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90d1b1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc0b5b60 0x7f90cc0cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56176/0 (socket says 172.21.15.16:56176) 2024-04-25T10:06:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90d1b1a640 1 -- 172.21.15.16:0/2502467746 learned_addr learned my addr 172.21.15.16:0/2502467746 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90d1b1a640 1 -- 172.21.15.16:0/2502467746 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90cc15dce0 msgr2=0x7f90cc0b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90d1b1a640 1 --2- 172.21.15.16:0/2502467746 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90cc15dce0 0x7f90cc0b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90d1b1a640 1 -- 172.21.15.16:0/2502467746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90cc0d35b0 msgr2=0x7f90cc159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90d1b1a640 1 --2- 172.21.15.16:0/2502467746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90cc0d35b0 0x7f90cc159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90d1b1a640 1 -- 172.21.15.16:0/2502467746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc0b21c0 con 0x7f90cc0b5b60 2024-04-25T10:06:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90d1b1a640 1 --2- 172.21.15.16:0/2502467746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc0b5b60 0x7f90cc0cbaf0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f90c80097a0 tx=0x7f90c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90c27fc640 1 -- 172.21.15.16:0/2502467746 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c8041070 con 0x7f90cc0b5b60 2024-04-25T10:06:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90c27fc640 1 -- 172.21.15.16:0/2502467746 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c803c870 con 0x7f90cc0b5b60 2024-04-25T10:06:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90d3d7e640 1 -- 172.21.15.16:0/2502467746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90cc0cc180 con 0x7f90cc0b5b60 2024-04-25T10:06:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90c27fc640 1 -- 172.21.15.16:0/2502467746 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c8037040 con 0x7f90cc0b5b60 2024-04-25T10:06:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.931+0000 7f90d3d7e640 1 -- 172.21.15.16:0/2502467746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90cc0cc740 con 0x7f90cc0b5b60 2024-04-25T10:06:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.932+0000 7f90d3d7e640 1 -- 172.21.15.16:0/2502467746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90cc0749b0 con 0x7f90cc0b5b60 2024-04-25T10:06:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.933+0000 7f90c27fc640 1 -- 172.21.15.16:0/2502467746 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90c8044070 con 0x7f90cc0b5b60 2024-04-25T10:06:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.933+0000 7f90c27fc640 1 --2- 172.21.15.16:0/2502467746 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f90b006b4d0 0x7f90b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.933+0000 7f90c27fc640 1 -- 172.21.15.16:0/2502467746 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f90c80aa9a0 con 0x7f90cc0b5b60 2024-04-25T10:06:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.933+0000 7f90d1319640 1 --2- 172.21.15.16:0/2502467746 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f90b006b4d0 0x7f90b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.934+0000 7f90d1319640 1 --2- 172.21.15.16:0/2502467746 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f90b006b4d0 0x7f90b006d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f90bc000f60 tx=0x7f90bc007a80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:48.937+0000 7f90c27fc640 1 -- 172.21.15.16:0/2502467746 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90c8079c00 con 0x7f90cc0b5b60 2024-04-25T10:06:49.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.119+0000 7f90d3d7e640 1 -- 172.21.15.16:0/2502467746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f90cc0b9460 con 0x7f90cc0b5b60 2024-04-25T10:06:49.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.120+0000 7f90c27fc640 1 -- 172.21.15.16:0/2502467746 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v43) v1 ==== 84+0+65 (secure 0 0 0) 0x7f90c8079500 con 0x7f90cc0b5b60 2024-04-25T10:06:49.125 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:49.125 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-04-25T10:06:49.126 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:06:49.126 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-04-25T10:06:49.126 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-04-25T10:06:49.126 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:06:49.126 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-04-25T10:06:49.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.122+0000 7f90d3d7e640 1 -- 172.21.15.16:0/2502467746 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f90b006b4d0 msgr2=0x7f90b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.122+0000 7f90d3d7e640 1 --2- 172.21.15.16:0/2502467746 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f90b006b4d0 0x7f90b006d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f90bc000f60 tx=0x7f90bc007a80 comp rx=0 tx=0).stop 2024-04-25T10:06:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.122+0000 7f90d3d7e640 1 -- 172.21.15.16:0/2502467746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc0b5b60 msgr2=0x7f90cc0cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.122+0000 7f90d3d7e640 1 --2- 172.21.15.16:0/2502467746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc0b5b60 0x7f90cc0cbaf0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f90c80097a0 tx=0x7f90c80046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.122+0000 7f90d3d7e640 1 -- 172.21.15.16:0/2502467746 shutdown_connections 2024-04-25T10:06:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.122+0000 7f90d3d7e640 1 --2- 172.21.15.16:0/2502467746 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f90b006b4d0 0x7f90b006d990 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.122+0000 7f90d3d7e640 1 --2- 172.21.15.16:0/2502467746 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90cc15dce0 0x7f90cc0b1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.122+0000 7f90d3d7e640 1 --2- 172.21.15.16:0/2502467746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90cc0d35b0 0x7f90cc159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.122+0000 7f90d3d7e640 1 --2- 172.21.15.16:0/2502467746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc0b5b60 0x7f90cc0cbaf0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.122+0000 7f90d3d7e640 1 -- 172.21.15.16:0/2502467746 >> 172.21.15.16:0/2502467746 conn(0x7f90cc072900 msgr2=0x7f90cc0b83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.122+0000 7f90d3d7e640 1 -- 172.21.15.16:0/2502467746 shutdown_connections 2024-04-25T10:06:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.122+0000 7f90d3d7e640 1 -- 172.21.15.16:0/2502467746 wait complete. 2024-04-25T10:06:49.138 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:06:49.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:06:49.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.237+0000 7fa4a0893640 1 Processor -- start 2024-04-25T10:06:49.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.237+0000 7fa4a0893640 1 -- start start 2024-04-25T10:06:49.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.238+0000 7fa4a0893640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49c1610b0 0x7fa49c0b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.238+0000 7fa4a0893640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa49c0cef40 0x7fa49c0d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.238+0000 7fa4a0893640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49c162040 con 0x7fa49c1618d0 2024-04-25T10:06:49.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.238+0000 7fa4a0893640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa49c0d1ea0 con 0x7fa49c0cef40 2024-04-25T10:06:49.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.238+0000 7fa4a0893640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c162320 con 0x7fa49c1610b0 2024-04-25T10:06:49.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.238+0000 7fa49ad9d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa49c0cef40 0x7fa49c0d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.238+0000 7fa49a59c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49c1610b0 0x7fa49c0b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.238+0000 7fa493fff640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fa49c1618d0 0x7fa49c0ce830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:38084/0 (socket says 172.21.15.16:38084) 2024-04-25T10:06:49.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.238+0000 7fa49ad9d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa49c0cef40 0x7fa49c0d2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:47966/0 (socket says 172.21.15.16:47966) 2024-04-25T10:06:49.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.238+0000 7fa493fff640 1 -- 172.21.15.16:0/3218343785 learned_addr learned my addr 172.21.15.16:0/3218343785 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:49.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.238+0000 7fa49ad9d640 1 -- 172.21.15.16:0/3218343785 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49c1610b0 msgr2=0x7fa49c0b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:49.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.238+0000 7fa49ad9d640 1 --2- 172.21.15.16:0/3218343785 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49c1610b0 0x7fa49c0b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.238+0000 7fa49ad9d640 1 -- 172.21.15.16:0/3218343785 >> v1:172.21.15.186:6789/0 conn(0x7fa49c1618d0 legacy=0x7fa49c0ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.238+0000 7fa49ad9d640 1 -- 172.21.15.16:0/3218343785 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c162600 con 0x7fa49c0cef40 2024-04-25T10:06:49.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.239+0000 7fa49a59c640 1 --2- 172.21.15.16:0/3218343785 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49c1610b0 0x7fa49c0b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:49.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.239+0000 7fa49ad9d640 1 --2- 172.21.15.16:0/3218343785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa49c0cef40 0x7fa49c0d2610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa494009d90 tx=0x7fa49402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c81c8e1ef4fe4fb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.239+0000 7fa499d9b640 1 -- 172.21.15.16:0/3218343785 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494031600 con 0x7fa49c0cef40 2024-04-25T10:06:49.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.240+0000 7fa499d9b640 1 -- 172.21.15.16:0/3218343785 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4940317a0 con 0x7fa49c0cef40 2024-04-25T10:06:49.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.240+0000 7fa499d9b640 1 -- 172.21.15.16:0/3218343785 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494031b70 con 0x7fa49c0cef40 2024-04-25T10:06:49.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.240+0000 7fa4a0893640 1 -- 172.21.15.16:0/3218343785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa49c0cef40 msgr2=0x7fa49c0d2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.240+0000 7fa4a0893640 1 --2- 172.21.15.16:0/3218343785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa49c0cef40 0x7fa49c0d2610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa494009d90 tx=0x7fa49402a5d0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.240+0000 7fa4a0893640 1 -- 172.21.15.16:0/3218343785 shutdown_connections 2024-04-25T10:06:49.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.240+0000 7fa4a0893640 1 --2- 172.21.15.16:0/3218343785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa49c0cef40 0x7fa49c0d2610 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.240+0000 7fa4a0893640 1 --2- 172.21.15.16:0/3218343785 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49c1610b0 0x7fa49c0b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.240+0000 7fa4a0893640 1 -- 172.21.15.16:0/3218343785 >> 172.21.15.16:0/3218343785 conn(0x7fa49c0b5570 msgr2=0x7fa49c0b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:49.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.240+0000 7fa4a0893640 1 -- 172.21.15.16:0/3218343785 shutdown_connections 2024-04-25T10:06:49.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.240+0000 7fa4a0893640 1 -- 172.21.15.16:0/3218343785 wait complete. 2024-04-25T10:06:49.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.241+0000 7fa4a0893640 1 Processor -- start 2024-04-25T10:06:49.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.241+0000 7fa4a0893640 1 -- start start 2024-04-25T10:06:49.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.242+0000 7fa4a0893640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49c0cef40 0x7fa49c15d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.242+0000 7fa4a0893640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa49c1610b0 0x7fa49c0cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.242+0000 7fa4a0893640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49c1618d0 0x7fa49c153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.242+0000 7fa4a0893640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c161d10 con 0x7fa49c0cef40 2024-04-25T10:06:49.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.242+0000 7fa4a0893640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c0b6c00 con 0x7fa49c1610b0 2024-04-25T10:06:49.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.242+0000 7fa4a0893640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c161490 con 0x7fa49c1618d0 2024-04-25T10:06:49.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.242+0000 7fa49a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49c0cef40 0x7fa49c15d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.242+0000 7fa49a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49c0cef40 0x7fa49c15d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56180/0 (socket says 172.21.15.16:56180) 2024-04-25T10:06:49.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.242+0000 7fa49a59c640 1 -- 172.21.15.16:0/3538416614 learned_addr learned my addr 172.21.15.16:0/3538416614 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:49.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.242+0000 7fa493fff640 1 --2- 172.21.15.16:0/3538416614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa49c1610b0 0x7fa49c0cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.242+0000 7fa49ad9d640 1 --2- 172.21.15.16:0/3538416614 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49c1618d0 0x7fa49c153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.243+0000 7fa49a59c640 1 -- 172.21.15.16:0/3538416614 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49c1618d0 msgr2=0x7fa49c153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.243+0000 7fa49a59c640 1 --2- 172.21.15.16:0/3538416614 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49c1618d0 0x7fa49c153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.243+0000 7fa49a59c640 1 -- 172.21.15.16:0/3538416614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa49c1610b0 msgr2=0x7fa49c0cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.243+0000 7fa49a59c640 1 --2- 172.21.15.16:0/3538416614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa49c1610b0 0x7fa49c0cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.243+0000 7fa49a59c640 1 -- 172.21.15.16:0/3538416614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c15dde0 con 0x7fa49c0cef40 2024-04-25T10:06:49.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.243+0000 7fa49a59c640 1 --2- 172.21.15.16:0/3538416614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49c0cef40 0x7fa49c15d760 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00ebe0 tx=0x7fa48c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.243+0000 7fa492ffd640 1 -- 172.21.15.16:0/3538416614 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c017070 con 0x7fa49c0cef40 2024-04-25T10:06:49.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.243+0000 7fa492ffd640 1 -- 172.21.15.16:0/3538416614 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c00cc30 con 0x7fa49c0cef40 2024-04-25T10:06:49.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.243+0000 7fa4a0893640 1 -- 172.21.15.16:0/3538416614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49c158e40 con 0x7fa49c0cef40 2024-04-25T10:06:49.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.243+0000 7fa492ffd640 1 -- 172.21.15.16:0/3538416614 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c013470 con 0x7fa49c0cef40 2024-04-25T10:06:49.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.243+0000 7fa4a0893640 1 -- 172.21.15.16:0/3538416614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c159400 con 0x7fa49c0cef40 2024-04-25T10:06:49.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.244+0000 7fa4a0893640 1 -- 172.21.15.16:0/3538416614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49c0750b0 con 0x7fa49c0cef40 2024-04-25T10:06:49.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.244+0000 7fa492ffd640 1 -- 172.21.15.16:0/3538416614 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa48c01a070 con 0x7fa49c0cef40 2024-04-25T10:06:49.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.245+0000 7fa492ffd640 1 --2- 172.21.15.16:0/3538416614 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa47c06b4d0 0x7fa47c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.245+0000 7fa492ffd640 1 -- 172.21.15.16:0/3538416614 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa48c08a080 con 0x7fa49c0cef40 2024-04-25T10:06:49.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.245+0000 7fa493fff640 1 --2- 172.21.15.16:0/3538416614 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa47c06b4d0 0x7fa47c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.246+0000 7fa493fff640 1 --2- 172.21.15.16:0/3538416614 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa47c06b4d0 0x7fa47c06d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0d32e0 tx=0x7fa484007e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.249+0000 7fa492ffd640 1 -- 172.21.15.16:0/3538416614 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa48c058c90 con 0x7fa49c0cef40 2024-04-25T10:06:49.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.431+0000 7fa4a0893640 1 -- 172.21.15.16:0/3538416614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa49c0d1770 con 0x7fa49c0cef40 2024-04-25T10:06:49.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.800+0000 7fa492ffd640 1 -- 172.21.15.16:0/3538416614 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v44) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa48c05cd30 con 0x7fa49c0cef40 2024-04-25T10:06:49.806 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-04-25T10:06:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.803+0000 7fa4a0893640 1 -- 172.21.15.16:0/3538416614 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa47c06b4d0 msgr2=0x7fa47c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.803+0000 7fa4a0893640 1 --2- 172.21.15.16:0/3538416614 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa47c06b4d0 0x7fa47c06d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0d32e0 tx=0x7fa484007e80 comp rx=0 tx=0).stop 2024-04-25T10:06:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.803+0000 7fa4a0893640 1 -- 172.21.15.16:0/3538416614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49c0cef40 msgr2=0x7fa49c15d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.803+0000 7fa4a0893640 1 --2- 172.21.15.16:0/3538416614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49c0cef40 0x7fa49c15d760 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00ebe0 tx=0x7fa48c00c520 comp rx=0 tx=0).stop 2024-04-25T10:06:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.803+0000 7fa4a0893640 1 -- 172.21.15.16:0/3538416614 shutdown_connections 2024-04-25T10:06:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.803+0000 7fa4a0893640 1 --2- 172.21.15.16:0/3538416614 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa47c06b4d0 0x7fa47c06d990 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.803+0000 7fa4a0893640 1 --2- 172.21.15.16:0/3538416614 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49c1618d0 0x7fa49c153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.803+0000 7fa4a0893640 1 --2- 172.21.15.16:0/3538416614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa49c1610b0 0x7fa49c0cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.803+0000 7fa4a0893640 1 --2- 172.21.15.16:0/3538416614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49c0cef40 0x7fa49c15d760 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.803+0000 7fa4a0893640 1 -- 172.21.15.16:0/3538416614 >> 172.21.15.16:0/3538416614 conn(0x7fa49c0b5570 msgr2=0x7fa49c0b7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:49.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.803+0000 7fa4a0893640 1 -- 172.21.15.16:0/3538416614 shutdown_connections 2024-04-25T10:06:49.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.804+0000 7fa4a0893640 1 -- 172.21.15.16:0/3538416614 wait complete. 2024-04-25T10:06:49.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:06:49.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.927+0000 7f5f4ccf7640 1 Processor -- start 2024-04-25T10:06:49.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.927+0000 7f5f4ccf7640 1 -- start start 2024-04-25T10:06:49.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.928+0000 7f5f4ccf7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f480ce390 0x7f5f4806aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.928+0000 7f5f4ccf7640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5f4806b0e0 0x7f5f4815d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.928+0000 7f5f4ccf7640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f480cd0f0 con 0x7f5f480cc7a0 2024-04-25T10:06:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.928+0000 7f5f4ccf7640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f480cbdb0 con 0x7f5f4806b0e0 2024-04-25T10:06:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.928+0000 7f5f4ccf7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480cda20 con 0x7f5f480ce390 2024-04-25T10:06:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.928+0000 7f5f3ffff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f480ce390 0x7f5f4806aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.928+0000 7f5f46d9d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5f4806b0e0 0x7f5f4815d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.928+0000 7f5f3ffff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f480ce390 0x7f5f4806aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:60630/0 (socket says 172.21.15.16:60630) 2024-04-25T10:06:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.928+0000 7f5f4659c640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f5f480cc7a0 0x7f5f480b1890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:38100/0 (socket says 172.21.15.16:38100) 2024-04-25T10:06:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.928+0000 7f5f3ffff640 1 -- 172.21.15.16:0/3675331183 learned_addr learned my addr 172.21.15.16:0/3675331183 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.929+0000 7f5f3ffff640 1 -- 172.21.15.16:0/3675331183 >> v2:172.21.15.186:3300/0 conn(0x7f5f4806b0e0 msgr2=0x7f5f4815d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.929+0000 7f5f3ffff640 1 --2- 172.21.15.16:0/3675331183 >> v2:172.21.15.186:3300/0 conn(0x7f5f4806b0e0 0x7f5f4815d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.929+0000 7f5f3ffff640 1 -- 172.21.15.16:0/3675331183 >> v1:172.21.15.186:6789/0 conn(0x7f5f480cc7a0 legacy=0x7f5f480b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.929+0000 7f5f3ffff640 1 -- 172.21.15.16:0/3675331183 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f480cd3d0 con 0x7f5f480ce390 2024-04-25T10:06:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.929+0000 7f5f46d9d640 1 --2- 172.21.15.16:0/3675331183 >> v2:172.21.15.186:3300/0 conn(0x7f5f4806b0e0 0x7f5f4815d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.929+0000 7f5f3ffff640 1 --2- 172.21.15.16:0/3675331183 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f480ce390 0x7f5f4806aba0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009870 tx=0x7f5f3002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4e811e84c429196 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.929+0000 7f5f45d9b640 1 -- 172.21.15.16:0/3675331183 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30037070 con 0x7f5f480ce390 2024-04-25T10:06:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.930+0000 7f5f45d9b640 1 -- 172.21.15.16:0/3675331183 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f30032440 con 0x7f5f480ce390 2024-04-25T10:06:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.930+0000 7f5f45d9b640 1 -- 172.21.15.16:0/3675331183 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30032810 con 0x7f5f480ce390 2024-04-25T10:06:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.930+0000 7f5f4ccf7640 1 -- 172.21.15.16:0/3675331183 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f480ce390 msgr2=0x7f5f4806aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.930+0000 7f5f4ccf7640 1 --2- 172.21.15.16:0/3675331183 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f480ce390 0x7f5f4806aba0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009870 tx=0x7f5f3002a270 comp rx=0 tx=0).stop 2024-04-25T10:06:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.930+0000 7f5f4ccf7640 1 -- 172.21.15.16:0/3675331183 shutdown_connections 2024-04-25T10:06:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.930+0000 7f5f4ccf7640 1 --2- 172.21.15.16:0/3675331183 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f480ce390 0x7f5f4806aba0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.930+0000 7f5f4ccf7640 1 --2- 172.21.15.16:0/3675331183 >> v2:172.21.15.186:3300/0 conn(0x7f5f4806b0e0 0x7f5f4815d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.930+0000 7f5f4ccf7640 1 -- 172.21.15.16:0/3675331183 >> 172.21.15.16:0/3675331183 conn(0x7f5f480d0750 msgr2=0x7f5f480d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.930+0000 7f5f4ccf7640 1 -- 172.21.15.16:0/3675331183 shutdown_connections 2024-04-25T10:06:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.930+0000 7f5f4ccf7640 1 -- 172.21.15.16:0/3675331183 wait complete. 2024-04-25T10:06:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.931+0000 7f5f4ccf7640 1 Processor -- start 2024-04-25T10:06:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.931+0000 7f5f4ccf7640 1 -- start start 2024-04-25T10:06:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.932+0000 7f5f4ccf7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f4806b0e0 0x7f5f480c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.932+0000 7f5f4ccf7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f480cc7a0 0x7f5f481e8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.932+0000 7f5f4ccf7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f480ce390 0x7f5f481e0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.932+0000 7f5f4ccf7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480ce030 con 0x7f5f480cc7a0 2024-04-25T10:06:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.932+0000 7f5f4ccf7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480cdeb0 con 0x7f5f4806b0e0 2024-04-25T10:06:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.932+0000 7f5f4ccf7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480ccb80 con 0x7f5f480ce390 2024-04-25T10:06:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.932+0000 7f5f3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f480cc7a0 0x7f5f481e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.932+0000 7f5f3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f480cc7a0 0x7f5f481e8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56184/0 (socket says 172.21.15.16:56184) 2024-04-25T10:06:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.932+0000 7f5f3ffff640 1 -- 172.21.15.16:0/1600103678 learned_addr learned my addr 172.21.15.16:0/1600103678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.932+0000 7f5f4659c640 1 --2- 172.21.15.16:0/1600103678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f4806b0e0 0x7f5f480c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.932+0000 7f5f46d9d640 1 --2- 172.21.15.16:0/1600103678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f480ce390 0x7f5f481e0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.933+0000 7f5f3ffff640 1 -- 172.21.15.16:0/1600103678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f480ce390 msgr2=0x7f5f481e0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.933+0000 7f5f3ffff640 1 --2- 172.21.15.16:0/1600103678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f480ce390 0x7f5f481e0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.933+0000 7f5f3ffff640 1 -- 172.21.15.16:0/1600103678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f4806b0e0 msgr2=0x7f5f480c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.933+0000 7f5f3ffff640 1 --2- 172.21.15.16:0/1600103678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f4806b0e0 0x7f5f480c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.933+0000 7f5f3ffff640 1 -- 172.21.15.16:0/1600103678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f480c7820 con 0x7f5f480cc7a0 2024-04-25T10:06:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.933+0000 7f5f4659c640 1 --2- 172.21.15.16:0/1600103678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f4806b0e0 0x7f5f480c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.933+0000 7f5f3ffff640 1 --2- 172.21.15.16:0/1600103678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f480cc7a0 0x7f5f481e8760 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5f30000c00 tx=0x7f5f300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.933+0000 7f5f3effd640 1 -- 172.21.15.16:0/1600103678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30037040 con 0x7f5f480cc7a0 2024-04-25T10:06:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.933+0000 7f5f3effd640 1 -- 172.21.15.16:0/1600103678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f30042070 con 0x7f5f480cc7a0 2024-04-25T10:06:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.933+0000 7f5f4ccf7640 1 -- 172.21.15.16:0/1600103678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f480c7b00 con 0x7f5f480cc7a0 2024-04-25T10:06:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.933+0000 7f5f3effd640 1 -- 172.21.15.16:0/1600103678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30030860 con 0x7f5f480cc7a0 2024-04-25T10:06:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.933+0000 7f5f4ccf7640 1 -- 172.21.15.16:0/1600103678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f480c80c0 con 0x7f5f480cc7a0 2024-04-25T10:06:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.934+0000 7f5f4ccf7640 1 -- 172.21.15.16:0/1600103678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4806cc40 con 0x7f5f480cc7a0 2024-04-25T10:06:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.935+0000 7f5f3effd640 1 -- 172.21.15.16:0/1600103678 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f30030a00 con 0x7f5f480cc7a0 2024-04-25T10:06:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.935+0000 7f5f3effd640 1 --2- 172.21.15.16:0/1600103678 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5f2806b4d0 0x7f5f2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.935+0000 7f5f3effd640 1 -- 172.21.15.16:0/1600103678 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f5f300aa840 con 0x7f5f480cc7a0 2024-04-25T10:06:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.936+0000 7f5f4659c640 1 --2- 172.21.15.16:0/1600103678 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5f2806b4d0 0x7f5f2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.936+0000 7f5f4659c640 1 --2- 172.21.15.16:0/1600103678 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5f2806b4d0 0x7f5f2806d990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5f38007fd0 tx=0x7f5f38007d80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:49.940+0000 7f5f3effd640 1 -- 172.21.15.16:0/1600103678 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5f30045360 con 0x7f5f480cc7a0 2024-04-25T10:06:50.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.122+0000 7f5f4ccf7640 1 -- 172.21.15.16:0/1600103678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5f480749b0 con 0x7f5f480cc7a0 2024-04-25T10:06:50.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.803+0000 7f5f3effd640 1 -- 172.21.15.16:0/1600103678 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v45) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5f30079930 con 0x7f5f480cc7a0 2024-04-25T10:06:50.808 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-04-25T10:06:50.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.808+0000 7f5f4ccf7640 1 -- 172.21.15.16:0/1600103678 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5f2806b4d0 msgr2=0x7f5f2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.808+0000 7f5f4ccf7640 1 --2- 172.21.15.16:0/1600103678 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5f2806b4d0 0x7f5f2806d990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5f38007fd0 tx=0x7f5f38007d80 comp rx=0 tx=0).stop 2024-04-25T10:06:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.808+0000 7f5f4ccf7640 1 -- 172.21.15.16:0/1600103678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f480cc7a0 msgr2=0x7f5f481e8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.808+0000 7f5f4ccf7640 1 --2- 172.21.15.16:0/1600103678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f480cc7a0 0x7f5f481e8760 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5f30000c00 tx=0x7f5f300046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.808+0000 7f5f4ccf7640 1 -- 172.21.15.16:0/1600103678 shutdown_connections 2024-04-25T10:06:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.808+0000 7f5f4ccf7640 1 --2- 172.21.15.16:0/1600103678 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5f2806b4d0 0x7f5f2806d990 secure :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5f38007fd0 tx=0x7f5f38007d80 comp rx=0 tx=0).stop 2024-04-25T10:06:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.808+0000 7f5f4ccf7640 1 --2- 172.21.15.16:0/1600103678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f480ce390 0x7f5f481e0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.808+0000 7f5f4ccf7640 1 --2- 172.21.15.16:0/1600103678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f480cc7a0 0x7f5f481e8760 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.808+0000 7f5f4ccf7640 1 --2- 172.21.15.16:0/1600103678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f4806b0e0 0x7f5f480c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.808+0000 7f5f4ccf7640 1 -- 172.21.15.16:0/1600103678 >> 172.21.15.16:0/1600103678 conn(0x7f5f480d0750 msgr2=0x7f5f480cad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:50.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.817+0000 7f5f4ccf7640 1 -- 172.21.15.16:0/1600103678 shutdown_connections 2024-04-25T10:06:50.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.818+0000 7f5f4ccf7640 1 -- 172.21.15.16:0/1600103678 wait complete. 2024-04-25T10:06:50.841 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:06:50.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.945+0000 7fc7a02ee640 1 Processor -- start 2024-04-25T10:06:50.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.945+0000 7fc7a02ee640 1 -- start start 2024-04-25T10:06:50.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.946+0000 7fc7a02ee640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc7980b42c0 0x7fc79815ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.946+0000 7fc7a02ee640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7980b46a0 0x7fc7981549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.946+0000 7fc7a02ee640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79815b370 con 0x7fc79815aef0 2024-04-25T10:06:50.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.946+0000 7fc7a02ee640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7980d05d0 con 0x7fc7980b42c0 2024-04-25T10:06:50.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.946+0000 7fc7a02ee640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79815b650 con 0x7fc7980b46a0 2024-04-25T10:06:50.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.946+0000 7fc79e08a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc7980b42c0 0x7fc79815ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.946+0000 7fc79e08a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc7980b42c0 0x7fc79815ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56198/0 (socket says 172.21.15.16:56198) 2024-04-25T10:06:50.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.946+0000 7fc79e08a640 1 -- 172.21.15.16:0/3921396146 learned_addr learned my addr 172.21.15.16:0/3921396146 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:50.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.946+0000 7fc79e88b640 1 --2- 172.21.15.16:0/3921396146 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7980b46a0 0x7fc7981549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.947+0000 7fc79e08a640 1 -- 172.21.15.16:0/3921396146 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7980b46a0 msgr2=0x7fc7981549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.947+0000 7fc79e08a640 1 --2- 172.21.15.16:0/3921396146 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7980b46a0 0x7fc7981549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.947+0000 7fc79e08a640 1 -- 172.21.15.16:0/3921396146 >> v1:172.21.15.186:6789/0 conn(0x7fc79815aef0 legacy=0x7fc7980cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.947+0000 7fc79e08a640 1 -- 172.21.15.16:0/3921396146 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79815b930 con 0x7fc7980b42c0 2024-04-25T10:06:50.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.947+0000 7fc79e88b640 1 --2- 172.21.15.16:0/3921396146 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7980b46a0 0x7fc7981549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:50.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.947+0000 7fc79e08a640 1 --2- 172.21.15.16:0/3921396146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7980b42c0 0x7fc79815ef40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc79000bb10 tx=0x7fc79002c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24090ef80d47e3f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.947+0000 7fc79d088640 1 -- 172.21.15.16:0/3921396146 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc790032700 con 0x7fc7980b42c0 2024-04-25T10:06:50.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.947+0000 7fc79d088640 1 -- 172.21.15.16:0/3921396146 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7900328a0 con 0x7fc7980b42c0 2024-04-25T10:06:50.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.947+0000 7fc79d088640 1 -- 172.21.15.16:0/3921396146 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc790032c70 con 0x7fc7980b42c0 2024-04-25T10:06:50.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.948+0000 7fc7a02ee640 1 -- 172.21.15.16:0/3921396146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7980b42c0 msgr2=0x7fc79815ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.948+0000 7fc7a02ee640 1 --2- 172.21.15.16:0/3921396146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7980b42c0 0x7fc79815ef40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc79000bb10 tx=0x7fc79002c800 comp rx=0 tx=0).stop 2024-04-25T10:06:50.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.948+0000 7fc7a02ee640 1 -- 172.21.15.16:0/3921396146 shutdown_connections 2024-04-25T10:06:50.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.948+0000 7fc7a02ee640 1 --2- 172.21.15.16:0/3921396146 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7980b46a0 0x7fc7981549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.948+0000 7fc7a02ee640 1 --2- 172.21.15.16:0/3921396146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7980b42c0 0x7fc79815ef40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.948+0000 7fc7a02ee640 1 -- 172.21.15.16:0/3921396146 >> 172.21.15.16:0/3921396146 conn(0x7fc79806aba0 msgr2=0x7fc79806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:50.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.948+0000 7fc7a02ee640 1 -- 172.21.15.16:0/3921396146 shutdown_connections 2024-04-25T10:06:50.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.948+0000 7fc7a02ee640 1 -- 172.21.15.16:0/3921396146 wait complete. 2024-04-25T10:06:50.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.949+0000 7fc7a02ee640 1 Processor -- start 2024-04-25T10:06:50.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.949+0000 7fc7a02ee640 1 -- start start 2024-04-25T10:06:50.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.950+0000 7fc7a02ee640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7980b42c0 0x7fc798147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.950+0000 7fc7a02ee640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7980b46a0 0x7fc7980c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.950+0000 7fc7a02ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79815aef0 0x7fc7980c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.950+0000 7fc7a02ee640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79806ec60 con 0x7fc79815aef0 2024-04-25T10:06:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.950+0000 7fc7a02ee640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79806eae0 con 0x7fc7980b42c0 2024-04-25T10:06:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.950+0000 7fc7a02ee640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc798154f10 con 0x7fc7980b46a0 2024-04-25T10:06:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.950+0000 7fc79e88b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79815aef0 0x7fc7980c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.950+0000 7fc79e88b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79815aef0 0x7fc7980c69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56204/0 (socket says 172.21.15.16:56204) 2024-04-25T10:06:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.950+0000 7fc79e88b640 1 -- 172.21.15.16:0/2331900076 learned_addr learned my addr 172.21.15.16:0/2331900076 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.950+0000 7fc79e08a640 1 --2- 172.21.15.16:0/2331900076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7980b42c0 0x7fc798147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.950+0000 7fc79d889640 1 --2- 172.21.15.16:0/2331900076 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7980b46a0 0x7fc7980c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.950+0000 7fc79e88b640 1 -- 172.21.15.16:0/2331900076 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7980b46a0 msgr2=0x7fc7980c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.950+0000 7fc79e88b640 1 --2- 172.21.15.16:0/2331900076 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7980b46a0 0x7fc7980c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.950+0000 7fc79e88b640 1 -- 172.21.15.16:0/2331900076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7980b42c0 msgr2=0x7fc798147510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.950+0000 7fc79e88b640 1 --2- 172.21.15.16:0/2331900076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7980b42c0 0x7fc798147510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.950+0000 7fc79e88b640 1 -- 172.21.15.16:0/2331900076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc798147a50 con 0x7fc79815aef0 2024-04-25T10:06:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.951+0000 7fc79e88b640 1 --2- 172.21.15.16:0/2331900076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79815aef0 0x7fc7980c69c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc788009fa0 tx=0x7fc78800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.951+0000 7fc796ffd640 1 -- 172.21.15.16:0/2331900076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78801a070 con 0x7fc79815aef0 2024-04-25T10:06:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.951+0000 7fc796ffd640 1 -- 172.21.15.16:0/2331900076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc788003bb0 con 0x7fc79815aef0 2024-04-25T10:06:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.951+0000 7fc7a02ee640 1 -- 172.21.15.16:0/2331900076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7981e86e0 con 0x7fc79815aef0 2024-04-25T10:06:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.951+0000 7fc796ffd640 1 -- 172.21.15.16:0/2331900076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788004030 con 0x7fc79815aef0 2024-04-25T10:06:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.951+0000 7fc7a02ee640 1 -- 172.21.15.16:0/2331900076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7980c4a80 con 0x7fc79815aef0 2024-04-25T10:06:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.952+0000 7fc7a02ee640 1 -- 172.21.15.16:0/2331900076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7980c45c0 con 0x7fc79815aef0 2024-04-25T10:06:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.953+0000 7fc796ffd640 1 -- 172.21.15.16:0/2331900076 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc78801d070 con 0x7fc79815aef0 2024-04-25T10:06:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.953+0000 7fc796ffd640 1 --2- 172.21.15.16:0/2331900076 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc78406b4d0 0x7fc78406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.953+0000 7fc796ffd640 1 -- 172.21.15.16:0/2331900076 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fc78808a3d0 con 0x7fc79815aef0 2024-04-25T10:06:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.953+0000 7fc79e08a640 1 --2- 172.21.15.16:0/2331900076 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc78406b4d0 0x7fc78406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.954+0000 7fc79e08a640 1 --2- 172.21.15.16:0/2331900076 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc78406b4d0 0x7fc78406d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc790037000 tx=0x7fc790008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:50.957+0000 7fc796ffd640 1 -- 172.21.15.16:0/2331900076 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7980c45c0 con 0x7fc79815aef0 2024-04-25T10:06:51.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.181+0000 7fc7a02ee640 1 -- 172.21.15.16:0/2331900076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc7980750b0 con 0x7fc79815aef0 2024-04-25T10:06:51.830 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 6 and data pool 7 2024-04-25T10:06:51.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.824+0000 7fc796ffd640 1 -- 172.21.15.16:0/2331900076 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v19) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc788010070 con 0x7fc79815aef0 2024-04-25T10:06:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.835+0000 7fc7a02ee640 1 -- 172.21.15.16:0/2331900076 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc78406b4d0 msgr2=0x7fc78406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.835+0000 7fc7a02ee640 1 --2- 172.21.15.16:0/2331900076 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc78406b4d0 0x7fc78406d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc790037000 tx=0x7fc790008040 comp rx=0 tx=0).stop 2024-04-25T10:06:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.835+0000 7fc7a02ee640 1 -- 172.21.15.16:0/2331900076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79815aef0 msgr2=0x7fc7980c69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.835+0000 7fc7a02ee640 1 --2- 172.21.15.16:0/2331900076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79815aef0 0x7fc7980c69c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc788009fa0 tx=0x7fc78800d560 comp rx=0 tx=0).stop 2024-04-25T10:06:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.836+0000 7fc7a02ee640 1 -- 172.21.15.16:0/2331900076 shutdown_connections 2024-04-25T10:06:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.836+0000 7fc7a02ee640 1 --2- 172.21.15.16:0/2331900076 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc78406b4d0 0x7fc78406d990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.836+0000 7fc7a02ee640 1 --2- 172.21.15.16:0/2331900076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79815aef0 0x7fc7980c69c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.836+0000 7fc7a02ee640 1 --2- 172.21.15.16:0/2331900076 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7980b46a0 0x7fc7980c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.836+0000 7fc7a02ee640 1 --2- 172.21.15.16:0/2331900076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7980b42c0 0x7fc798147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.836+0000 7fc7a02ee640 1 -- 172.21.15.16:0/2331900076 >> 172.21.15.16:0/2331900076 conn(0x7fc79806aba0 msgr2=0x7fc7980b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.836+0000 7fc7a02ee640 1 -- 172.21.15.16:0/2331900076 shutdown_connections 2024-04-25T10:06:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.837+0000 7fc7a02ee640 1 -- 172.21.15.16:0/2331900076 wait complete. 2024-04-25T10:06:51.862 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:51.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.961+0000 7f2f8a407640 1 Processor -- start 2024-04-25T10:06:51.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.961+0000 7f2f8a407640 1 -- start start 2024-04-25T10:06:51.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.962+0000 7f2f8a407640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2f840d35b0 0x7f2f840b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.962+0000 7f2f8a407640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f8415e0c0 con 0x7f2f8415dce0 2024-04-25T10:06:51.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.962+0000 7f2f8a407640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f840d2510 con 0x7f2f840b5990 2024-04-25T10:06:51.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.962+0000 7f2f8a407640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f840d27f0 con 0x7f2f840d35b0 2024-04-25T10:06:51.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.962+0000 7f2f837fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2f840d35b0 0x7f2f840b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.962+0000 7f2f837fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2f840d35b0 0x7f2f840b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56208/0 (socket says 172.21.15.16:56208) 2024-04-25T10:06:51.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.962+0000 7f2f83fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2f8415dce0 0x7f2f840d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36596/0 (socket says 172.21.15.16:36596) 2024-04-25T10:06:51.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.962+0000 7f2f837fe640 1 -- 172.21.15.16:0/1236376030 learned_addr learned my addr 172.21.15.16:0/1236376030 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:51.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.963+0000 7f2f837fe640 1 -- 172.21.15.16:0/1236376030 >> v1:172.21.15.186:6789/0 conn(0x7f2f840b5990 legacy=0x7f2f840b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.963+0000 7f2f837fe640 1 -- 172.21.15.16:0/1236376030 >> v1:172.21.15.16:6789/0 conn(0x7f2f8415dce0 legacy=0x7f2f840d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.963+0000 7f2f82ffd640 1 -- 172.21.15.16:0/1236376030 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747866634 0 0) 0x7f2f8415e0c0 con 0x7f2f8415dce0 2024-04-25T10:06:51.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.963+0000 7f2f837fe640 1 -- 172.21.15.16:0/1236376030 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f840d2ad0 con 0x7f2f840d35b0 2024-04-25T10:06:51.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.963+0000 7f2f837fe640 1 --2- 172.21.15.16:0/1236376030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f840d35b0 0x7f2f840b5450 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009870 tx=0x7f2f7802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70079848455ace3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.964+0000 7f2f82ffd640 1 -- 172.21.15.16:0/1236376030 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f78037070 con 0x7f2f840d35b0 2024-04-25T10:06:51.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.964+0000 7f2f82ffd640 1 -- 172.21.15.16:0/1236376030 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f78032440 con 0x7f2f840d35b0 2024-04-25T10:06:51.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.964+0000 7f2f82ffd640 1 -- 172.21.15.16:0/1236376030 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f78032810 con 0x7f2f840d35b0 2024-04-25T10:06:51.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.964+0000 7f2f8a407640 1 -- 172.21.15.16:0/1236376030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f840d35b0 msgr2=0x7f2f840b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.964+0000 7f2f8a407640 1 --2- 172.21.15.16:0/1236376030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f840d35b0 0x7f2f840b5450 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009870 tx=0x7f2f7802a2a0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.964+0000 7f2f8a407640 1 -- 172.21.15.16:0/1236376030 shutdown_connections 2024-04-25T10:06:51.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.964+0000 7f2f8a407640 1 --2- 172.21.15.16:0/1236376030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f840d35b0 0x7f2f840b5450 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.964+0000 7f2f8a407640 1 -- 172.21.15.16:0/1236376030 >> 172.21.15.16:0/1236376030 conn(0x7f2f84072900 msgr2=0x7f2f840d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.964+0000 7f2f8a407640 1 -- 172.21.15.16:0/1236376030 shutdown_connections 2024-04-25T10:06:51.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.964+0000 7f2f8a407640 1 -- 172.21.15.16:0/1236376030 wait complete. 2024-04-25T10:06:51.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.965+0000 7f2f8a407640 1 Processor -- start 2024-04-25T10:06:51.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.966+0000 7f2f8a407640 1 -- start start 2024-04-25T10:06:51.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.966+0000 7f2f8a407640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f840b5990 0x7f2f840b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.966+0000 7f2f8a407640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f840d35b0 0x7f2f84158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.966+0000 7f2f8a407640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f8415dce0 0x7f2f84158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.966+0000 7f2f8a407640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f840d2160 con 0x7f2f8415dce0 2024-04-25T10:06:51.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.966+0000 7f2f8a407640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f840d3090 con 0x7f2f840d35b0 2024-04-25T10:06:51.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.966+0000 7f2f8a407640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f841593a0 con 0x7f2f840b5990 2024-04-25T10:06:51.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.967+0000 7f2f889a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f8415dce0 0x7f2f84158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.967+0000 7f2f889a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f8415dce0 0x7f2f84158e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56216/0 (socket says 172.21.15.16:56216) 2024-04-25T10:06:51.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.967+0000 7f2f889a4640 1 -- 172.21.15.16:0/2080340717 learned_addr learned my addr 172.21.15.16:0/2080340717 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:51.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.967+0000 7f2f83fff640 1 --2- 172.21.15.16:0/2080340717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f840b5990 0x7f2f840b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.967+0000 7f2f837fe640 1 --2- 172.21.15.16:0/2080340717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f840d35b0 0x7f2f84158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.967+0000 7f2f889a4640 1 -- 172.21.15.16:0/2080340717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f840b5990 msgr2=0x7f2f840b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.967+0000 7f2f889a4640 1 --2- 172.21.15.16:0/2080340717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f840b5990 0x7f2f840b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.967+0000 7f2f889a4640 1 -- 172.21.15.16:0/2080340717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f840d35b0 msgr2=0x7f2f84158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.967+0000 7f2f889a4640 1 --2- 172.21.15.16:0/2080340717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f840d35b0 0x7f2f84158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.967+0000 7f2f889a4640 1 -- 172.21.15.16:0/2080340717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f84159620 con 0x7f2f8415dce0 2024-04-25T10:06:51.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.967+0000 7f2f889a4640 1 --2- 172.21.15.16:0/2080340717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f8415dce0 0x7f2f84158e60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00c780 tx=0x7f2f7c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.967+0000 7f2f80ff9640 1 -- 172.21.15.16:0/2080340717 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c004b60 con 0x7f2f8415dce0 2024-04-25T10:06:51.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.967+0000 7f2f80ff9640 1 -- 172.21.15.16:0/2080340717 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7c004d00 con 0x7f2f8415dce0 2024-04-25T10:06:51.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.967+0000 7f2f8a407640 1 -- 172.21.15.16:0/2080340717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f84157a20 con 0x7f2f8415dce0 2024-04-25T10:06:51.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.967+0000 7f2f80ff9640 1 -- 172.21.15.16:0/2080340717 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c00aa30 con 0x7f2f8415dce0 2024-04-25T10:06:51.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.967+0000 7f2f8a407640 1 -- 172.21.15.16:0/2080340717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f84159900 con 0x7f2f8415dce0 2024-04-25T10:06:51.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.968+0000 7f2f8a407640 1 -- 172.21.15.16:0/2080340717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f840749b0 con 0x7f2f8415dce0 2024-04-25T10:06:51.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.969+0000 7f2f80ff9640 1 -- 172.21.15.16:0/2080340717 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f7c00abd0 con 0x7f2f8415dce0 2024-04-25T10:06:51.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.969+0000 7f2f80ff9640 1 --2- 172.21.15.16:0/2080340717 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2f7006b4d0 0x7f2f7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.969+0000 7f2f80ff9640 1 -- 172.21.15.16:0/2080340717 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f7c08a300 con 0x7f2f8415dce0 2024-04-25T10:06:51.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.970+0000 7f2f83fff640 1 --2- 172.21.15.16:0/2080340717 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2f7006b4d0 0x7f2f7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.970+0000 7f2f83fff640 1 --2- 172.21.15.16:0/2080340717 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2f7006b4d0 0x7f2f7006d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2f74007fd0 tx=0x7f2f74007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:51.974+0000 7f2f80ff9640 1 -- 172.21.15.16:0/2080340717 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f7c020070 con 0x7f2f8415dce0 2024-04-25T10:06:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.155+0000 7f2f8a407640 1 -- 172.21.15.16:0/2080340717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f840b3dd0 con 0x7f2f8415dce0 2024-04-25T10:06:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.156+0000 7f2f80ff9640 1 -- 172.21.15.16:0/2080340717 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16622 (secure 0 0 0) 0x7f2f7c058b30 con 0x7f2f8415dce0 2024-04-25T10:06:52.162 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:52.162 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:06:51.182388+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:49.462347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:50.153232+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:52.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.158+0000 7f2f8a407640 1 -- 172.21.15.16:0/2080340717 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2f7006b4d0 msgr2=0x7f2f7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.158+0000 7f2f8a407640 1 --2- 172.21.15.16:0/2080340717 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2f7006b4d0 0x7f2f7006d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2f74007fd0 tx=0x7f2f74007de0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.158+0000 7f2f8a407640 1 -- 172.21.15.16:0/2080340717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f8415dce0 msgr2=0x7f2f84158e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.158+0000 7f2f8a407640 1 --2- 172.21.15.16:0/2080340717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f8415dce0 0x7f2f84158e60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00c780 tx=0x7f2f7c00cc40 comp rx=0 tx=0).stop 2024-04-25T10:06:52.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.158+0000 7f2f8a407640 1 -- 172.21.15.16:0/2080340717 shutdown_connections 2024-04-25T10:06:52.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.158+0000 7f2f8a407640 1 --2- 172.21.15.16:0/2080340717 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2f7006b4d0 0x7f2f7006d990 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.158+0000 7f2f8a407640 1 --2- 172.21.15.16:0/2080340717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f8415dce0 0x7f2f84158e60 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.158+0000 7f2f8a407640 1 --2- 172.21.15.16:0/2080340717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f840d35b0 0x7f2f84158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.158+0000 7f2f8a407640 1 --2- 172.21.15.16:0/2080340717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f840b5990 0x7f2f840b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.159+0000 7f2f8a407640 1 -- 172.21.15.16:0/2080340717 >> 172.21.15.16:0/2080340717 conn(0x7f2f84072900 msgr2=0x7f2f840d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.159+0000 7f2f8a407640 1 -- 172.21.15.16:0/2080340717 shutdown_connections 2024-04-25T10:06:52.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.159+0000 7f2f8a407640 1 -- 172.21.15.16:0/2080340717 wait complete. 2024-04-25T10:06:52.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:52.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.271+0000 7f450f728640 1 Processor -- start 2024-04-25T10:06:52.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.271+0000 7f450f728640 1 -- start start 2024-04-25T10:06:52.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.272+0000 7f450f728640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f45080d35b0 0x7f45080b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.272+0000 7f450f728640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45080b5990 0x7f45080b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.272+0000 7f450f728640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f450815e0c0 con 0x7f450815dce0 2024-04-25T10:06:52.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.272+0000 7f450f728640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45080d2510 con 0x7f45080d35b0 2024-04-25T10:06:52.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.272+0000 7f450f728640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45080d27f0 con 0x7f45080b5990 2024-04-25T10:06:52.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.272+0000 7f450ccc3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f45080d35b0 0x7f45080b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.272+0000 7f450d4c4640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f450815dce0 0x7f45080d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36604/0 (socket says 172.21.15.16:36604) 2024-04-25T10:06:52.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.272+0000 7f450d4c4640 1 -- 172.21.15.16:0/3914222001 learned_addr learned my addr 172.21.15.16:0/3914222001 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:52.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.272+0000 7f450dcc5640 1 --2- 172.21.15.16:0/3914222001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45080b5990 0x7f45080b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.272+0000 7f450ccc3640 1 -- 172.21.15.16:0/3914222001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45080b5990 msgr2=0x7f45080b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.272+0000 7f450ccc3640 1 --2- 172.21.15.16:0/3914222001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45080b5990 0x7f45080b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.272+0000 7f450ccc3640 1 -- 172.21.15.16:0/3914222001 >> v1:172.21.15.16:6789/0 conn(0x7f450815dce0 legacy=0x7f45080d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.272+0000 7f450ccc3640 1 -- 172.21.15.16:0/3914222001 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45080d2ad0 con 0x7f45080d35b0 2024-04-25T10:06:52.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.272+0000 7f44fffff640 1 -- 172.21.15.16:0/3914222001 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840011913 0 0) 0x7f450815e0c0 con 0x7f450815dce0 2024-04-25T10:06:52.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.273+0000 7f450ccc3640 1 --2- 172.21.15.16:0/3914222001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45080d35b0 0x7f45080b5450 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f44f80097a0 tx=0x7f44f802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ab96a4dbb15a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.273+0000 7f44fffff640 1 -- 172.21.15.16:0/3914222001 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f8037070 con 0x7f45080d35b0 2024-04-25T10:06:52.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.273+0000 7f44fffff640 1 -- 172.21.15.16:0/3914222001 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f8032440 con 0x7f45080d35b0 2024-04-25T10:06:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.273+0000 7f44fffff640 1 -- 172.21.15.16:0/3914222001 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f8032810 con 0x7f45080d35b0 2024-04-25T10:06:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.274+0000 7f450f728640 1 -- 172.21.15.16:0/3914222001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45080d35b0 msgr2=0x7f45080b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.274+0000 7f450f728640 1 --2- 172.21.15.16:0/3914222001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45080d35b0 0x7f45080b5450 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f44f80097a0 tx=0x7f44f802a180 comp rx=0 tx=0).stop 2024-04-25T10:06:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.274+0000 7f450f728640 1 -- 172.21.15.16:0/3914222001 shutdown_connections 2024-04-25T10:06:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.274+0000 7f450f728640 1 --2- 172.21.15.16:0/3914222001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45080b5990 0x7f45080b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.274+0000 7f450f728640 1 --2- 172.21.15.16:0/3914222001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45080d35b0 0x7f45080b5450 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.274+0000 7f450f728640 1 -- 172.21.15.16:0/3914222001 >> 172.21.15.16:0/3914222001 conn(0x7f4508072900 msgr2=0x7f45080d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.274+0000 7f450f728640 1 -- 172.21.15.16:0/3914222001 shutdown_connections 2024-04-25T10:06:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.274+0000 7f450f728640 1 -- 172.21.15.16:0/3914222001 wait complete. 2024-04-25T10:06:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.275+0000 7f450f728640 1 Processor -- start 2024-04-25T10:06:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.275+0000 7f450f728640 1 -- start start 2024-04-25T10:06:52.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450f728640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45080b5990 0x7f45080ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450f728640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45080d35b0 0x7f45080ac950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450f728640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f450815dce0 0x7f45080af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450f728640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45080d2160 con 0x7f45080d35b0 2024-04-25T10:06:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450f728640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45080d3090 con 0x7f450815dce0 2024-04-25T10:06:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450f728640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45080ab980 con 0x7f45080b5990 2024-04-25T10:06:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450ccc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45080d35b0 0x7f45080ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450ccc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45080d35b0 0x7f45080ac950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56240/0 (socket says 172.21.15.16:56240) 2024-04-25T10:06:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450ccc3640 1 -- 172.21.15.16:0/1877787285 learned_addr learned my addr 172.21.15.16:0/1877787285 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450d4c4640 1 --2- 172.21.15.16:0/1877787285 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45080b5990 0x7f45080ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450dcc5640 1 --2- 172.21.15.16:0/1877787285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f450815dce0 0x7f45080af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450ccc3640 1 -- 172.21.15.16:0/1877787285 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45080b5990 msgr2=0x7f45080ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450ccc3640 1 --2- 172.21.15.16:0/1877787285 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45080b5990 0x7f45080ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450ccc3640 1 -- 172.21.15.16:0/1877787285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f450815dce0 msgr2=0x7f45080af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450ccc3640 1 --2- 172.21.15.16:0/1877787285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f450815dce0 0x7f45080af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450ccc3640 1 -- 172.21.15.16:0/1877787285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45080abe90 con 0x7f45080d35b0 2024-04-25T10:06:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450dcc5640 1 --2- 172.21.15.16:0/1877787285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f450815dce0 0x7f45080af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450d4c4640 1 --2- 172.21.15.16:0/1877787285 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45080b5990 0x7f45080ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450ccc3640 1 --2- 172.21.15.16:0/1877787285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45080d35b0 0x7f45080ac950 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f44f8035040 tx=0x7f44f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f44fdffb640 1 -- 172.21.15.16:0/1877787285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f8037040 con 0x7f45080d35b0 2024-04-25T10:06:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f44fdffb640 1 -- 172.21.15.16:0/1877787285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f8030770 con 0x7f45080d35b0 2024-04-25T10:06:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f450f728640 1 -- 172.21.15.16:0/1877787285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45080ac170 con 0x7f45080d35b0 2024-04-25T10:06:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.276+0000 7f44fdffb640 1 -- 172.21.15.16:0/1877787285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f8030b80 con 0x7f45080d35b0 2024-04-25T10:06:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.277+0000 7f450f728640 1 -- 172.21.15.16:0/1877787285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45080adb00 con 0x7f45080d35b0 2024-04-25T10:06:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.278+0000 7f450f728640 1 -- 172.21.15.16:0/1877787285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45080749b0 con 0x7f45080d35b0 2024-04-25T10:06:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.278+0000 7f44fdffb640 1 -- 172.21.15.16:0/1877787285 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44f8044020 con 0x7f45080d35b0 2024-04-25T10:06:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.279+0000 7f44fdffb640 1 --2- 172.21.15.16:0/1877787285 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f44ec06b4d0 0x7f44ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.279+0000 7f44fdffb640 1 -- 172.21.15.16:0/1877787285 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44f80aa770 con 0x7f45080d35b0 2024-04-25T10:06:52.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.279+0000 7f450d4c4640 1 --2- 172.21.15.16:0/1877787285 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f44ec06b4d0 0x7f44ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.279+0000 7f450d4c4640 1 --2- 172.21.15.16:0/1877787285 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f44ec06b4d0 0x7f44ec06d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4504000f60 tx=0x7f4504009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.283+0000 7f44fdffb640 1 -- 172.21.15.16:0/1877787285 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44f8032e20 con 0x7f45080d35b0 2024-04-25T10:06:52.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.463+0000 7f450f728640 1 -- 172.21.15.16:0/1877787285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45080b3d90 con 0x7f45080d35b0 2024-04-25T10:06:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.465+0000 7f44fdffb640 1 -- 172.21.15.16:0/1877787285 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16622 (secure 0 0 0) 0x7f44f80795f0 con 0x7f45080d35b0 2024-04-25T10:06:52.470 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:52.470 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:06:51.182388+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:49.462347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:50.153232+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:52.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.466+0000 7f450f728640 1 -- 172.21.15.16:0/1877787285 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f44ec06b4d0 msgr2=0x7f44ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.466+0000 7f450f728640 1 --2- 172.21.15.16:0/1877787285 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f44ec06b4d0 0x7f44ec06d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4504000f60 tx=0x7f4504009040 comp rx=0 tx=0).stop 2024-04-25T10:06:52.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.466+0000 7f450f728640 1 -- 172.21.15.16:0/1877787285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45080d35b0 msgr2=0x7f45080ac950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.466+0000 7f450f728640 1 --2- 172.21.15.16:0/1877787285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45080d35b0 0x7f45080ac950 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f44f8035040 tx=0x7f44f80046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.466+0000 7f450f728640 1 -- 172.21.15.16:0/1877787285 shutdown_connections 2024-04-25T10:06:52.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.466+0000 7f450f728640 1 --2- 172.21.15.16:0/1877787285 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f44ec06b4d0 0x7f44ec06d990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.466+0000 7f450f728640 1 --2- 172.21.15.16:0/1877787285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f450815dce0 0x7f45080af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.467+0000 7f450f728640 1 --2- 172.21.15.16:0/1877787285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45080d35b0 0x7f45080ac950 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.467+0000 7f450f728640 1 --2- 172.21.15.16:0/1877787285 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45080b5990 0x7f45080ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.467+0000 7f450f728640 1 -- 172.21.15.16:0/1877787285 >> 172.21.15.16:0/1877787285 conn(0x7f4508072900 msgr2=0x7f450815cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.467+0000 7f450f728640 1 -- 172.21.15.16:0/1877787285 shutdown_connections 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.467+0000 7f450f728640 1 -- 172.21.15.16:0/1877787285 wait complete. 2024-04-25T10:06:52.483 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:06:52.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.581+0000 7f5192729640 1 Processor -- start 2024-04-25T10:06:52.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.581+0000 7f5192729640 1 -- start start 2024-04-25T10:06:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.582+0000 7f5192729640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f518c0d3b90 0x7f518c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.582+0000 7f5192729640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f518c0d0f60 0x7f518c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.582+0000 7f5192729640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f518c0d4090 con 0x7f518c15dca0 2024-04-25T10:06:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.582+0000 7f5192729640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518c0b6b20 con 0x7f518c0d0f60 2024-04-25T10:06:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.582+0000 7f5192729640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518c0d4370 con 0x7f518c0d3b90 2024-04-25T10:06:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.582+0000 7f5190cc6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f518c0d0f60 0x7f518c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.582+0000 7f518b7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f518c0d3b90 0x7f518c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.582+0000 7f5190cc6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f518c0d0f60 0x7f518c0b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:48030/0 (socket says 172.21.15.16:48030) 2024-04-25T10:06:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.582+0000 7f5190cc6640 1 -- 172.21.15.16:0/3797710196 learned_addr learned my addr 172.21.15.16:0/3797710196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.582+0000 7f518bfff640 1 --1- 172.21.15.16:0/3797710196 >> v1:172.21.15.186:6789/0 conn(0x7f518c15dca0 0x7f518c0ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:38124/0 (socket says 172.21.15.16:38124) 2024-04-25T10:06:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.583+0000 7f5190cc6640 1 -- 172.21.15.16:0/3797710196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f518c0d3b90 msgr2=0x7f518c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.583+0000 7f5190cc6640 1 --2- 172.21.15.16:0/3797710196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f518c0d3b90 0x7f518c0d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.583+0000 7f5190cc6640 1 -- 172.21.15.16:0/3797710196 >> v1:172.21.15.186:6789/0 conn(0x7f518c15dca0 legacy=0x7f518c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.583+0000 7f5190cc6640 1 -- 172.21.15.16:0/3797710196 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c0d4650 con 0x7f518c0d0f60 2024-04-25T10:06:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.583+0000 7f5190cc6640 1 --2- 172.21.15.16:0/3797710196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f518c0d0f60 0x7f518c0b2500 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5184009e50 tx=0x7f518402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8a5689f445d4b52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.583+0000 7f518affd640 1 -- 172.21.15.16:0/3797710196 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5184036070 con 0x7f518c0d0f60 2024-04-25T10:06:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.584+0000 7f518affd640 1 -- 172.21.15.16:0/3797710196 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5184031600 con 0x7f518c0d0f60 2024-04-25T10:06:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.584+0000 7f518affd640 1 -- 172.21.15.16:0/3797710196 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51840319d0 con 0x7f518c0d0f60 2024-04-25T10:06:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.584+0000 7f5192729640 1 -- 172.21.15.16:0/3797710196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f518c0d0f60 msgr2=0x7f518c0b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.584+0000 7f5192729640 1 --2- 172.21.15.16:0/3797710196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f518c0d0f60 0x7f518c0b2500 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5184009e50 tx=0x7f518402a710 comp rx=0 tx=0).stop 2024-04-25T10:06:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.584+0000 7f5192729640 1 -- 172.21.15.16:0/3797710196 shutdown_connections 2024-04-25T10:06:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.584+0000 7f5192729640 1 --2- 172.21.15.16:0/3797710196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f518c0d3b90 0x7f518c0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.584+0000 7f5192729640 1 --2- 172.21.15.16:0/3797710196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f518c0d0f60 0x7f518c0b2500 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.584+0000 7f5192729640 1 -- 172.21.15.16:0/3797710196 >> 172.21.15.16:0/3797710196 conn(0x7f518c0726c0 msgr2=0x7f518c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.584+0000 7f5192729640 1 -- 172.21.15.16:0/3797710196 shutdown_connections 2024-04-25T10:06:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.584+0000 7f5192729640 1 -- 172.21.15.16:0/3797710196 wait complete. 2024-04-25T10:06:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.585+0000 7f5192729640 1 Processor -- start 2024-04-25T10:06:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.585+0000 7f5192729640 1 -- start start 2024-04-25T10:06:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.586+0000 7f5192729640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f518c0d0f60 0x7f518c150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.586+0000 7f5192729640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f518c0d3b90 0x7f518c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.586+0000 7f5192729640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f518c15dca0 0x7f518c06c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.586+0000 7f5192729640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c15e080 con 0x7f518c0d3b90 2024-04-25T10:06:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.586+0000 7f5192729640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c0cf050 con 0x7f518c0d0f60 2024-04-25T10:06:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.586+0000 7f5192729640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518c15e200 con 0x7f518c15dca0 2024-04-25T10:06:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.586+0000 7f518b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f518c0d3b90 0x7f518c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.586+0000 7f518b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f518c0d3b90 0x7f518c06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56248/0 (socket says 172.21.15.16:56248) 2024-04-25T10:06:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.586+0000 7f518b7fe640 1 -- 172.21.15.16:0/2495929266 learned_addr learned my addr 172.21.15.16:0/2495929266 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.586+0000 7f5190cc6640 1 --2- 172.21.15.16:0/2495929266 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f518c15dca0 0x7f518c06c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.586+0000 7f518bfff640 1 --2- 172.21.15.16:0/2495929266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f518c0d0f60 0x7f518c150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.587+0000 7f518b7fe640 1 -- 172.21.15.16:0/2495929266 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f518c15dca0 msgr2=0x7f518c06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.587+0000 7f518b7fe640 1 --2- 172.21.15.16:0/2495929266 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f518c15dca0 0x7f518c06c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.587+0000 7f518b7fe640 1 -- 172.21.15.16:0/2495929266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f518c0d0f60 msgr2=0x7f518c150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.587+0000 7f518b7fe640 1 --2- 172.21.15.16:0/2495929266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f518c0d0f60 0x7f518c150880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.587+0000 7f518b7fe640 1 -- 172.21.15.16:0/2495929266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c1f7080 con 0x7f518c0d3b90 2024-04-25T10:06:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.587+0000 7f518b7fe640 1 --2- 172.21.15.16:0/2495929266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f518c0d3b90 0x7f518c06c080 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f517c00e980 tx=0x7f517c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.587+0000 7f5188ff9640 1 -- 172.21.15.16:0/2495929266 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c017070 con 0x7f518c0d3b90 2024-04-25T10:06:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.587+0000 7f5188ff9640 1 -- 172.21.15.16:0/2495929266 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c00ca00 con 0x7f518c0d3b90 2024-04-25T10:06:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.587+0000 7f5188ff9640 1 -- 172.21.15.16:0/2495929266 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c00cdd0 con 0x7f518c0d3b90 2024-04-25T10:06:52.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.587+0000 7f5192729640 1 -- 172.21.15.16:0/2495929266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518c1f7360 con 0x7f518c0d3b90 2024-04-25T10:06:52.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.587+0000 7f5192729640 1 -- 172.21.15.16:0/2495929266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518c1f7920 con 0x7f518c0d3b90 2024-04-25T10:06:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.588+0000 7f5192729640 1 -- 172.21.15.16:0/2495929266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518c0750b0 con 0x7f518c0d3b90 2024-04-25T10:06:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.589+0000 7f5188ff9640 1 -- 172.21.15.16:0/2495929266 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f517c01a070 con 0x7f518c0d3b90 2024-04-25T10:06:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.589+0000 7f5188ff9640 1 --2- 172.21.15.16:0/2495929266 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f517806b4d0 0x7f517806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.589+0000 7f5188ff9640 1 -- 172.21.15.16:0/2495929266 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f517c08a4f0 con 0x7f518c0d3b90 2024-04-25T10:06:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.589+0000 7f518bfff640 1 --2- 172.21.15.16:0/2495929266 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f517806b4d0 0x7f517806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.590+0000 7f518bfff640 1 --2- 172.21.15.16:0/2495929266 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f517806b4d0 0x7f517806d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5180007fd0 tx=0x7f5180007e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.593+0000 7f5188ff9640 1 -- 172.21.15.16:0/2495929266 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f517c058d30 con 0x7f518c0d3b90 2024-04-25T10:06:52.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.811+0000 7f5192729640 1 -- 172.21.15.16:0/2495929266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f518c150dc0 con 0x7f518c0d3b90 2024-04-25T10:06:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.829+0000 7f5188ff9640 1 -- 172.21.15.16:0/2495929266 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v20) v1 ==== 118+0+0 (secure 0 0 0) 0x7f517c05cdd0 con 0x7f518c0d3b90 2024-04-25T10:06:52.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.832+0000 7f5192729640 1 -- 172.21.15.16:0/2495929266 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f517806b4d0 msgr2=0x7f517806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.832+0000 7f5192729640 1 --2- 172.21.15.16:0/2495929266 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f517806b4d0 0x7f517806d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5180007fd0 tx=0x7f5180007e80 comp rx=0 tx=0).stop 2024-04-25T10:06:52.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.832+0000 7f5192729640 1 -- 172.21.15.16:0/2495929266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f518c0d3b90 msgr2=0x7f518c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.832+0000 7f5192729640 1 --2- 172.21.15.16:0/2495929266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f518c0d3b90 0x7f518c06c080 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f517c00e980 tx=0x7f517c00ee40 comp rx=0 tx=0).stop 2024-04-25T10:06:52.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.832+0000 7f5192729640 1 -- 172.21.15.16:0/2495929266 shutdown_connections 2024-04-25T10:06:52.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.832+0000 7f5192729640 1 --2- 172.21.15.16:0/2495929266 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f517806b4d0 0x7f517806d990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.832+0000 7f5192729640 1 --2- 172.21.15.16:0/2495929266 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f518c15dca0 0x7f518c06c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.832+0000 7f5192729640 1 --2- 172.21.15.16:0/2495929266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f518c0d3b90 0x7f518c06c080 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.832+0000 7f5192729640 1 --2- 172.21.15.16:0/2495929266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f518c0d0f60 0x7f518c150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.832+0000 7f5192729640 1 -- 172.21.15.16:0/2495929266 >> 172.21.15.16:0/2495929266 conn(0x7f518c0726c0 msgr2=0x7f518c0b4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.832+0000 7f5192729640 1 -- 172.21.15.16:0/2495929266 shutdown_connections 2024-04-25T10:06:52.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.833+0000 7f5192729640 1 -- 172.21.15.16:0/2495929266 wait complete. 2024-04-25T10:06:52.849 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:06:52.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:06:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.947+0000 7f0b9474f640 1 Processor -- start 2024-04-25T10:06:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.947+0000 7f0b9474f640 1 -- start start 2024-04-25T10:06:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.948+0000 7f0b9474f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0b8c0b2740 0x7f0b8c15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.948+0000 7f0b9474f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0b8c0ce1a0 0x7f0b8c0ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.948+0000 7f0b9474f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b8c06c150 con 0x7f0b8c0d0490 2024-04-25T10:06:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.948+0000 7f0b9474f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b8c0d2000 con 0x7f0b8c0ce1a0 2024-04-25T10:06:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.948+0000 7f0b9474f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b8c06cd60 con 0x7f0b8c0b2740 2024-04-25T10:06:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.948+0000 7f0b924eb640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0b8c0d0490 0x7f0b8c0d0870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59472/0 (socket says 172.21.15.16:59472) 2024-04-25T10:06:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.948+0000 7f0b924eb640 1 -- 172.21.15.16:0/4042056695 learned_addr learned my addr 172.21.15.16:0/4042056695 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.948+0000 7f0b92cec640 1 --2- 172.21.15.16:0/4042056695 >> v2:172.21.15.16:3300/0 conn(0x7f0b8c0ce1a0 0x7f0b8c0ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.948+0000 7f0b92cec640 1 -- 172.21.15.16:0/4042056695 >> v2:172.21.15.186:3300/0 conn(0x7f0b8c0b2740 msgr2=0x7f0b8c15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.948+0000 7f0b92cec640 1 --2- 172.21.15.16:0/4042056695 >> v2:172.21.15.186:3300/0 conn(0x7f0b8c0b2740 0x7f0b8c15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.948+0000 7f0b92cec640 1 -- 172.21.15.16:0/4042056695 >> v1:172.21.15.16:6789/0 conn(0x7f0b8c0d0490 legacy=0x7f0b8c0d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.948+0000 7f0b92cec640 1 -- 172.21.15.16:0/4042056695 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8c06c430 con 0x7f0b8c0ce1a0 2024-04-25T10:06:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.949+0000 7f0b92cec640 1 --2- 172.21.15.16:0/4042056695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c0ce1a0 0x7f0b8c0ce630 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0d1540 tx=0x7f0b7c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124981d4e6afdafb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.949+0000 7f0b914e9640 1 -- 172.21.15.16:0/4042056695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c007740 con 0x7f0b8c0ce1a0 2024-04-25T10:06:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.949+0000 7f0b914e9640 1 -- 172.21.15.16:0/4042056695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7c0078e0 con 0x7f0b8c0ce1a0 2024-04-25T10:06:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.949+0000 7f0b914e9640 1 -- 172.21.15.16:0/4042056695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c007cb0 con 0x7f0b8c0ce1a0 2024-04-25T10:06:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.950+0000 7f0b9474f640 1 -- 172.21.15.16:0/4042056695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c0ce1a0 msgr2=0x7f0b8c0ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.950+0000 7f0b9474f640 1 --2- 172.21.15.16:0/4042056695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c0ce1a0 0x7f0b8c0ce630 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0d1540 tx=0x7f0b7c02a5f0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.950+0000 7f0b9474f640 1 -- 172.21.15.16:0/4042056695 shutdown_connections 2024-04-25T10:06:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.950+0000 7f0b9474f640 1 --2- 172.21.15.16:0/4042056695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c0ce1a0 0x7f0b8c0ce630 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.950+0000 7f0b9474f640 1 --2- 172.21.15.16:0/4042056695 >> v2:172.21.15.186:3300/0 conn(0x7f0b8c0b2740 0x7f0b8c15ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.950+0000 7f0b9474f640 1 -- 172.21.15.16:0/4042056695 >> 172.21.15.16:0/4042056695 conn(0x7f0b8c0b5660 msgr2=0x7f0b8c0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.950+0000 7f0b9474f640 1 -- 172.21.15.16:0/4042056695 shutdown_connections 2024-04-25T10:06:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.950+0000 7f0b9474f640 1 -- 172.21.15.16:0/4042056695 wait complete. 2024-04-25T10:06:52.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.951+0000 7f0b9474f640 1 Processor -- start 2024-04-25T10:06:52.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.951+0000 7f0b9474f640 1 -- start start 2024-04-25T10:06:52.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.952+0000 7f0b9474f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c0b2740 0x7f0b8c160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.952+0000 7f0b9474f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b8c0ce1a0 0x7f0b8c0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.952+0000 7f0b9474f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b8c0d0490 0x7f0b8c0b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.952+0000 7f0b9474f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c06c9f0 con 0x7f0b8c0b2740 2024-04-25T10:06:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.952+0000 7f0b9474f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c15f1d0 con 0x7f0b8c0ce1a0 2024-04-25T10:06:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.952+0000 7f0b9474f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c15e020 con 0x7f0b8c0d0490 2024-04-25T10:06:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.952+0000 7f0b924eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c0b2740 0x7f0b8c160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.952+0000 7f0b924eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c0b2740 0x7f0b8c160a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50396/0 (socket says 172.21.15.16:50396) 2024-04-25T10:06:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.952+0000 7f0b924eb640 1 -- 172.21.15.16:0/558799642 learned_addr learned my addr 172.21.15.16:0/558799642 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.952+0000 7f0b91cea640 1 --2- 172.21.15.16:0/558799642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b8c0ce1a0 0x7f0b8c0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.952+0000 7f0b924eb640 1 -- 172.21.15.16:0/558799642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b8c0d0490 msgr2=0x7f0b8c0b81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.952+0000 7f0b924eb640 1 --2- 172.21.15.16:0/558799642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b8c0d0490 0x7f0b8c0b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.952+0000 7f0b924eb640 1 -- 172.21.15.16:0/558799642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b8c0ce1a0 msgr2=0x7f0b8c0cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.952+0000 7f0b924eb640 1 --2- 172.21.15.16:0/558799642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b8c0ce1a0 0x7f0b8c0cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.952+0000 7f0b924eb640 1 -- 172.21.15.16:0/558799642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8c0cdee0 con 0x7f0b8c0b2740 2024-04-25T10:06:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.952+0000 7f0b91cea640 1 --2- 172.21.15.16:0/558799642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b8c0ce1a0 0x7f0b8c0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.953+0000 7f0b924eb640 1 --2- 172.21.15.16:0/558799642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c0b2740 0x7f0b8c160a10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400c3c0 tx=0x7f0b8400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.953+0000 7f0b8affd640 1 -- 172.21.15.16:0/558799642 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b84017070 con 0x7f0b8c0b2740 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.953+0000 7f0b8affd640 1 -- 172.21.15.16:0/558799642 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b84005ce0 con 0x7f0b8c0b2740 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.953+0000 7f0b9474f640 1 -- 172.21.15.16:0/558799642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8c1610f0 con 0x7f0b8c0b2740 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.953+0000 7f0b8affd640 1 -- 172.21.15.16:0/558799642 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b840072f0 con 0x7f0b8c0b2740 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.953+0000 7f0b9474f640 1 -- 172.21.15.16:0/558799642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8c1616b0 con 0x7f0b8c0b2740 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.954+0000 7f0b9474f640 1 -- 172.21.15.16:0/558799642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b8c0750b0 con 0x7f0b8c0b2740 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.954+0000 7f0b8affd640 1 -- 172.21.15.16:0/558799642 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b8401a070 con 0x7f0b8c0b2740 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.955+0000 7f0b8affd640 1 --2- 172.21.15.16:0/558799642 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0b7806b4d0 0x7f0b7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.955+0000 7f0b8affd640 1 -- 172.21.15.16:0/558799642 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b84089680 con 0x7f0b8c0b2740 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.955+0000 7f0b91cea640 1 --2- 172.21.15.16:0/558799642 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0b7806b4d0 0x7f0b7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.955+0000 7f0b91cea640 1 --2- 172.21.15.16:0/558799642 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0b7806b4d0 0x7f0b7806d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0b80006fd0 tx=0x7f0b80008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:52.959+0000 7f0b8affd640 1 -- 172.21.15.16:0/558799642 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b84058460 con 0x7f0b8c0b2740 2024-04-25T10:06:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.177+0000 7f0b9474f640 1 -- 172.21.15.16:0/558799642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0b8c0d1de0 con 0x7f0b8c0b2740 2024-04-25T10:06:53.799 INFO:tasks.ceph.mds.b.smithi186.stderr:2024-04-25T10:06:53.793+0000 7f25be5bd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:06:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.834+0000 7f0b8affd640 1 -- 172.21.15.16:0/558799642 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v21) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0b84057d60 con 0x7f0b8c0b2740 2024-04-25T10:06:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.837+0000 7f0b9474f640 1 -- 172.21.15.16:0/558799642 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0b7806b4d0 msgr2=0x7f0b7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.837+0000 7f0b9474f640 1 --2- 172.21.15.16:0/558799642 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0b7806b4d0 0x7f0b7806d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0b80006fd0 tx=0x7f0b80008040 comp rx=0 tx=0).stop 2024-04-25T10:06:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.837+0000 7f0b9474f640 1 -- 172.21.15.16:0/558799642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c0b2740 msgr2=0x7f0b8c160a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.837+0000 7f0b9474f640 1 --2- 172.21.15.16:0/558799642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c0b2740 0x7f0b8c160a10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400c3c0 tx=0x7f0b8400c880 comp rx=0 tx=0).stop 2024-04-25T10:06:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.837+0000 7f0b9474f640 1 -- 172.21.15.16:0/558799642 shutdown_connections 2024-04-25T10:06:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.837+0000 7f0b9474f640 1 --2- 172.21.15.16:0/558799642 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0b7806b4d0 0x7f0b7806d990 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.837+0000 7f0b9474f640 1 --2- 172.21.15.16:0/558799642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b8c0d0490 0x7f0b8c0b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.837+0000 7f0b9474f640 1 --2- 172.21.15.16:0/558799642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b8c0ce1a0 0x7f0b8c0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.837+0000 7f0b9474f640 1 --2- 172.21.15.16:0/558799642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c0b2740 0x7f0b8c160a10 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.837+0000 7f0b9474f640 1 -- 172.21.15.16:0/558799642 >> 172.21.15.16:0/558799642 conn(0x7f0b8c0b5660 msgr2=0x7f0b8c0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.837+0000 7f0b9474f640 1 -- 172.21.15.16:0/558799642 shutdown_connections 2024-04-25T10:06:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.837+0000 7f0b9474f640 1 -- 172.21.15.16:0/558799642 wait complete. 2024-04-25T10:06:53.856 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:06:53.856 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:53.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.958+0000 7f0dc4a07640 1 Processor -- start 2024-04-25T10:06:53.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.958+0000 7f0dc4a07640 1 -- start start 2024-04-25T10:06:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.959+0000 7f0dc4a07640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0dc015ec10 0x7f0dc006dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.959+0000 7f0dc4a07640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc006e6c0 con 0x7f0dc00b3750 2024-04-25T10:06:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.959+0000 7f0dc4a07640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc006e9a0 con 0x7f0dc006e1b0 2024-04-25T10:06:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.959+0000 7f0dc4a07640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc006f2d0 con 0x7f0dc015ec10 2024-04-25T10:06:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.959+0000 7f0dbed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0dc00b3750 0x7f0dc006aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59488/0 (socket says 172.21.15.16:59488) 2024-04-25T10:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.959+0000 7f0dbed9d640 1 -- 172.21.15.16:0/985906745 learned_addr learned my addr 172.21.15.16:0/985906745 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.959+0000 7f0dbe59c640 1 --2- 172.21.15.16:0/985906745 >> v2:172.21.15.186:3300/0 conn(0x7f0dc015ec10 0x7f0dc006dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.959+0000 7f0dbdd9b640 1 -- 172.21.15.16:0/985906745 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531256455 0 0) 0x7f0dc006e6c0 con 0x7f0dc00b3750 2024-04-25T10:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.959+0000 7f0dbdd9b640 1 -- 172.21.15.16:0/985906745 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dac003520 con 0x7f0dc00b3750 2024-04-25T10:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.959+0000 7f0dbdd9b640 1 -- 172.21.15.16:0/985906745 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1018097582 0 0) 0x7f0dac003520 con 0x7f0dc00b3750 2024-04-25T10:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.959+0000 7f0dbdd9b640 1 -- 172.21.15.16:0/985906745 >> v2:172.21.15.186:3300/0 conn(0x7f0dc015ec10 msgr2=0x7f0dc006dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.959+0000 7f0dbdd9b640 1 --2- 172.21.15.16:0/985906745 >> v2:172.21.15.186:3300/0 conn(0x7f0dc015ec10 0x7f0dc006dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.959+0000 7f0dbdd9b640 1 -- 172.21.15.16:0/985906745 >> v1:172.21.15.186:6789/0 conn(0x7f0dc006e1b0 legacy=0x7f0dc00b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.959+0000 7f0dbdd9b640 1 -- 172.21.15.16:0/985906745 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc006ec80 con 0x7f0dc00b3750 2024-04-25T10:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.959+0000 7f0dbdd9b640 1 -- 172.21.15.16:0/985906745 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f0db80019f0 con 0x7f0dc00b3750 2024-04-25T10:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.959+0000 7f0dbdd9b640 1 -- 172.21.15.16:0/985906745 >> v1:172.21.15.16:6789/0 conn(0x7f0dc00b3750 legacy=0x7f0dc006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.959+0000 7f0dbdd9b640 1 --2- 172.21.15.16:0/985906745 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0dac0039e0 0x7f0dac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.959+0000 7f0dbdd9b640 1 --2- 172.21.15.16:0/985906745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dac004800 0x7f0dac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.960+0000 7f0dbdd9b640 1 --2- 172.21.15.16:0/985906745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dac0052e0 0x7f0dac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.960+0000 7f0dbdd9b640 1 -- 172.21.15.16:0/985906745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc006f2d0 con 0x7f0dac0052e0 2024-04-25T10:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.960+0000 7f0dbdd9b640 1 -- 172.21.15.16:0/985906745 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac00c1e0 con 0x7f0dac004800 2024-04-25T10:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.960+0000 7f0dbdd9b640 1 -- 172.21.15.16:0/985906745 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac00c3e0 con 0x7f0dac0039e0 2024-04-25T10:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.960+0000 7f0dbed9d640 1 --2- 172.21.15.16:0/985906745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dac0052e0 0x7f0dac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.960+0000 7f0dbe59c640 1 --2- 172.21.15.16:0/985906745 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0dac0039e0 0x7f0dac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.960+0000 7f0db7fff640 1 --2- 172.21.15.16:0/985906745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dac004800 0x7f0dac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.960+0000 7f0dbed9d640 1 -- 172.21.15.16:0/985906745 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0dac0039e0 msgr2=0x7f0dac003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.960+0000 7f0dbed9d640 1 --2- 172.21.15.16:0/985906745 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0dac0039e0 0x7f0dac003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.960+0000 7f0dbed9d640 1 -- 172.21.15.16:0/985906745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dac004800 msgr2=0x7f0dac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.960+0000 7f0dbed9d640 1 --2- 172.21.15.16:0/985906745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dac004800 0x7f0dac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.960+0000 7f0dbed9d640 1 -- 172.21.15.16:0/985906745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dac003520 con 0x7f0dac0052e0 2024-04-25T10:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.961+0000 7f0dbed9d640 1 -- 172.21.15.16:0/985906745 reap_dead start 2024-04-25T10:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.961+0000 7f0dbed9d640 1 --2- 172.21.15.16:0/985906745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dac0052e0 0x7f0dac00b9c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0db80013d0 tx=0x7f0dac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424b0e2711f156df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.961+0000 7f0dbdd9b640 1 -- 172.21.15.16:0/985906745 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db800a480 con 0x7f0dac0052e0 2024-04-25T10:06:53.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.961+0000 7f0dbdd9b640 1 -- 172.21.15.16:0/985906745 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db802f070 con 0x7f0dac0052e0 2024-04-25T10:06:53.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.961+0000 7f0dbdd9b640 1 -- 172.21.15.16:0/985906745 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db802bc50 con 0x7f0dac0052e0 2024-04-25T10:06:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.961+0000 7f0dc4a07640 1 -- 172.21.15.16:0/985906745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dac0052e0 msgr2=0x7f0dac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.961+0000 7f0dc4a07640 1 --2- 172.21.15.16:0/985906745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dac0052e0 0x7f0dac00b9c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0db80013d0 tx=0x7f0dac005830 comp rx=0 tx=0).stop 2024-04-25T10:06:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.961+0000 7f0dc4a07640 1 -- 172.21.15.16:0/985906745 shutdown_connections 2024-04-25T10:06:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.961+0000 7f0dc4a07640 1 --2- 172.21.15.16:0/985906745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dac0052e0 0x7f0dac00b9c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.961+0000 7f0dc4a07640 1 -- 172.21.15.16:0/985906745 >> 172.21.15.16:0/985906745 conn(0x7f0dc00b2220 msgr2=0x7f0dc00cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.962+0000 7f0dc4a07640 1 -- 172.21.15.16:0/985906745 shutdown_connections 2024-04-25T10:06:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.962+0000 7f0dc4a07640 1 -- 172.21.15.16:0/985906745 wait complete. 2024-04-25T10:06:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.963+0000 7f0dc4a07640 1 Processor -- start 2024-04-25T10:06:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.963+0000 7f0dc4a07640 1 -- start start 2024-04-25T10:06:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.964+0000 7f0dc4a07640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dac0052e0 0x7f0dc014f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.964+0000 7f0dc4a07640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dc014fb20 0x7f0dc0154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.964+0000 7f0dc4a07640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0dc01f0c70 0x7f0dc01545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.964+0000 7f0dc4a07640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc015e2e0 con 0x7f0dac0052e0 2024-04-25T10:06:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.964+0000 7f0dc4a07640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc015eff0 con 0x7f0dc014fb20 2024-04-25T10:06:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.964+0000 7f0dc4a07640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc015e460 con 0x7f0dc01f0c70 2024-04-25T10:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.964+0000 7f0dbe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dac0052e0 0x7f0dc014f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.964+0000 7f0dbe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dac0052e0 0x7f0dc014f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50414/0 (socket says 172.21.15.16:50414) 2024-04-25T10:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.964+0000 7f0dbe59c640 1 -- 172.21.15.16:0/2429892682 learned_addr learned my addr 172.21.15.16:0/2429892682 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.964+0000 7f0dbe59c640 1 -- 172.21.15.16:0/2429892682 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0dc01f0c70 msgr2=0x7f0dc01545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.964+0000 7f0dbe59c640 1 --2- 172.21.15.16:0/2429892682 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0dc01f0c70 0x7f0dc01545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.964+0000 7f0dbe59c640 1 -- 172.21.15.16:0/2429892682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dc014fb20 msgr2=0x7f0dc0154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.964+0000 7f0db7fff640 1 --2- 172.21.15.16:0/2429892682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dc014fb20 0x7f0dc0154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.964+0000 7f0dbe59c640 1 --2- 172.21.15.16:0/2429892682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dc014fb20 0x7f0dc0154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.964+0000 7f0dbe59c640 1 -- 172.21.15.16:0/2429892682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc014ffc0 con 0x7f0dac0052e0 2024-04-25T10:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.964+0000 7f0dbe59c640 1 --2- 172.21.15.16:0/2429892682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dac0052e0 0x7f0dc014f5e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0db00118e0 tx=0x7f0db0011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.964+0000 7f0db6ffd640 1 -- 172.21.15.16:0/2429892682 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db000faf0 con 0x7f0dac0052e0 2024-04-25T10:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.964+0000 7f0db6ffd640 1 -- 172.21.15.16:0/2429892682 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db000fc90 con 0x7f0dac0052e0 2024-04-25T10:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.965+0000 7f0dc4a07640 1 -- 172.21.15.16:0/2429892682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc01502a0 con 0x7f0dac0052e0 2024-04-25T10:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.965+0000 7f0db6ffd640 1 -- 172.21.15.16:0/2429892682 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0014440 con 0x7f0dac0052e0 2024-04-25T10:06:53.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.965+0000 7f0dc4a07640 1 -- 172.21.15.16:0/2429892682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc0150860 con 0x7f0dac0052e0 2024-04-25T10:06:53.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.965+0000 7f0dc4a07640 1 -- 172.21.15.16:0/2429892682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc00749b0 con 0x7f0dac0052e0 2024-04-25T10:06:53.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.966+0000 7f0db6ffd640 1 -- 172.21.15.16:0/2429892682 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0db0012040 con 0x7f0dac0052e0 2024-04-25T10:06:53.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.967+0000 7f0db6ffd640 1 --2- 172.21.15.16:0/2429892682 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0da006b4d0 0x7f0da006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.967+0000 7f0db6ffd640 1 -- 172.21.15.16:0/2429892682 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0db008a9d0 con 0x7f0dac0052e0 2024-04-25T10:06:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.967+0000 7f0db7fff640 1 --2- 172.21.15.16:0/2429892682 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0da006b4d0 0x7f0da006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.968+0000 7f0db7fff640 1 --2- 172.21.15.16:0/2429892682 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0da006b4d0 0x7f0da006d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0da8000c00 tx=0x7f0da8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:53.971+0000 7f0db6ffd640 1 -- 172.21.15.16:0/2429892682 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0db0019030 con 0x7f0dac0052e0 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.187+0000 7f0dc4a07640 1 -- 172.21.15.16:0/2429892682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0dc00d0b00 con 0x7f0dac0052e0 2024-04-25T10:06:54.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.187+0000 7f0db6ffd640 1 -- 172.21.15.16:0/2429892682 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f0db0059200 con 0x7f0dac0052e0 2024-04-25T10:06:54.193 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:54.193 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/791079479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":791079479},{"type":"v1","addr":"172.21.15.16:6835","nonce":791079479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1747921236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1747921236},{"type":"v1","addr":"172.21.15.16:6834","nonce":1747921236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":34880,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/3183779257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3183779257},{"type":"v1","addr":"172.21.15.186:6836","nonce":3183779257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:51.182691+0000","modified":"2024-04-25T10:06:53.179194+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34883},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34883":{"gid":34883,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/83423489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":83423489},{"type":"v1","addr":"172.21.15.186:6837","nonce":83423489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:54.193 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 21 2024-04-25T10:06:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.190+0000 7f0dc4a07640 1 -- 172.21.15.16:0/2429892682 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0da006b4d0 msgr2=0x7f0da006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.190+0000 7f0dc4a07640 1 --2- 172.21.15.16:0/2429892682 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0da006b4d0 0x7f0da006d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0da8000c00 tx=0x7f0da8009040 comp rx=0 tx=0).stop 2024-04-25T10:06:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.190+0000 7f0dc4a07640 1 -- 172.21.15.16:0/2429892682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dac0052e0 msgr2=0x7f0dc014f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.190+0000 7f0dc4a07640 1 --2- 172.21.15.16:0/2429892682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dac0052e0 0x7f0dc014f5e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0db00118e0 tx=0x7f0db0011da0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.190+0000 7f0dc4a07640 1 -- 172.21.15.16:0/2429892682 shutdown_connections 2024-04-25T10:06:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.190+0000 7f0dc4a07640 1 --2- 172.21.15.16:0/2429892682 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0da006b4d0 0x7f0da006d990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.190+0000 7f0dc4a07640 1 --2- 172.21.15.16:0/2429892682 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0dc01f0c70 0x7f0dc01545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.190+0000 7f0dc4a07640 1 --2- 172.21.15.16:0/2429892682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dc014fb20 0x7f0dc0154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.190+0000 7f0dc4a07640 1 --2- 172.21.15.16:0/2429892682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dac0052e0 0x7f0dc014f5e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.190+0000 7f0dc4a07640 1 -- 172.21.15.16:0/2429892682 >> 172.21.15.16:0/2429892682 conn(0x7f0dc00b2220 msgr2=0x7f0dc00cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.190+0000 7f0dc4a07640 1 -- 172.21.15.16:0/2429892682 shutdown_connections 2024-04-25T10:06:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.190+0000 7f0dc4a07640 1 -- 172.21.15.16:0/2429892682 wait complete. 2024-04-25T10:06:54.207 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:54.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.301+0000 7f832b0ac640 1 Processor -- start 2024-04-25T10:06:54.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.301+0000 7f832b0ac640 1 -- start start 2024-04-25T10:06:54.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.302+0000 7f832b0ac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f832409a150 0x7f832406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.302+0000 7f832b0ac640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f832406c1f0 0x7f83240cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.302+0000 7f832b0ac640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832406c680 con 0x7f832415ba10 2024-04-25T10:06:54.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.302+0000 7f832b0ac640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83240cbe50 con 0x7f832409a150 2024-04-25T10:06:54.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.302+0000 7f832b0ac640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83240cfe80 con 0x7f832406c1f0 2024-04-25T10:06:54.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.302+0000 7f8323fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f832409a150 0x7f832406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.302+0000 7f8323fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f832409a150 0x7f832406bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50422/0 (socket says 172.21.15.16:50422) 2024-04-25T10:06:54.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.302+0000 7f8323fff640 1 -- 172.21.15.16:0/332249496 learned_addr learned my addr 172.21.15.16:0/332249496 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:54.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.302+0000 7f8329649640 1 --2- 172.21.15.16:0/332249496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f832406c1f0 0x7f83240cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.303+0000 7f8323fff640 1 -- 172.21.15.16:0/332249496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f832406c1f0 msgr2=0x7f83240cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.303+0000 7f8323fff640 1 --2- 172.21.15.16:0/332249496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f832406c1f0 0x7f83240cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.303+0000 7f8323fff640 1 -- 172.21.15.16:0/332249496 >> v1:172.21.15.186:6789/0 conn(0x7f832415ba10 legacy=0x7f8324099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.303+0000 7f8323fff640 1 -- 172.21.15.16:0/332249496 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832406c900 con 0x7f832409a150 2024-04-25T10:06:54.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.303+0000 7f8323fff640 1 --2- 172.21.15.16:0/332249496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409a150 0x7f832406bcb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f830c009870 tx=0x7f830c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4011fe28ba437986 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.304+0000 7f83237fe640 1 -- 172.21.15.16:0/332249496 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c037070 con 0x7f832409a150 2024-04-25T10:06:54.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.304+0000 7f83237fe640 1 -- 172.21.15.16:0/332249496 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830c032440 con 0x7f832409a150 2024-04-25T10:06:54.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.304+0000 7f83237fe640 1 -- 172.21.15.16:0/332249496 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c032810 con 0x7f832409a150 2024-04-25T10:06:54.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.304+0000 7f832b0ac640 1 -- 172.21.15.16:0/332249496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409a150 msgr2=0x7f832406bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.304+0000 7f832b0ac640 1 --2- 172.21.15.16:0/332249496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409a150 0x7f832406bcb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f830c009870 tx=0x7f830c02a270 comp rx=0 tx=0).stop 2024-04-25T10:06:54.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.304+0000 7f832b0ac640 1 -- 172.21.15.16:0/332249496 shutdown_connections 2024-04-25T10:06:54.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.304+0000 7f832b0ac640 1 --2- 172.21.15.16:0/332249496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f832406c1f0 0x7f83240cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.304+0000 7f832b0ac640 1 --2- 172.21.15.16:0/332249496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409a150 0x7f832406bcb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.304+0000 7f832b0ac640 1 -- 172.21.15.16:0/332249496 >> 172.21.15.16:0/332249496 conn(0x7f83240b32f0 msgr2=0x7f83240b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.304+0000 7f832b0ac640 1 -- 172.21.15.16:0/332249496 shutdown_connections 2024-04-25T10:06:54.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.304+0000 7f832b0ac640 1 -- 172.21.15.16:0/332249496 wait complete. 2024-04-25T10:06:54.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.305+0000 7f832b0ac640 1 Processor -- start 2024-04-25T10:06:54.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.305+0000 7f832b0ac640 1 -- start start 2024-04-25T10:06:54.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.306+0000 7f832b0ac640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f832406c1f0 0x7f83241493c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.306+0000 7f832b0ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409a150 0x7f83240ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.306+0000 7f832b0ac640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f832415ba10 0x7f83240cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.306+0000 7f832b0ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83240cd760 con 0x7f832409a150 2024-04-25T10:06:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.306+0000 7f832b0ac640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832406cec0 con 0x7f832406c1f0 2024-04-25T10:06:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.306+0000 7f832b0ac640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83240cd8e0 con 0x7f832415ba10 2024-04-25T10:06:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.306+0000 7f8323fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409a150 0x7f83240ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.306+0000 7f8323fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409a150 0x7f83240ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50434/0 (socket says 172.21.15.16:50434) 2024-04-25T10:06:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.306+0000 7f8323fff640 1 -- 172.21.15.16:0/4215524930 learned_addr learned my addr 172.21.15.16:0/4215524930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.306+0000 7f8328e48640 1 --2- 172.21.15.16:0/4215524930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f832406c1f0 0x7f83241493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.307+0000 7f8329649640 1 --2- 172.21.15.16:0/4215524930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f832415ba10 0x7f83240cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.307+0000 7f8323fff640 1 -- 172.21.15.16:0/4215524930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f832415ba10 msgr2=0x7f83240cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.307+0000 7f8323fff640 1 --2- 172.21.15.16:0/4215524930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f832415ba10 0x7f83240cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.307+0000 7f8323fff640 1 -- 172.21.15.16:0/4215524930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f832406c1f0 msgr2=0x7f83241493c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.307+0000 7f8323fff640 1 --2- 172.21.15.16:0/4215524930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f832406c1f0 0x7f83241493c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.307+0000 7f8323fff640 1 -- 172.21.15.16:0/4215524930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83240b5000 con 0x7f832409a150 2024-04-25T10:06:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.307+0000 7f8329649640 1 --2- 172.21.15.16:0/4215524930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f832415ba10 0x7f83240cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:54.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.307+0000 7f8323fff640 1 --2- 172.21.15.16:0/4215524930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409a150 0x7f83240ccb30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f830c000c00 tx=0x7f830c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.307+0000 7f83217fa640 1 -- 172.21.15.16:0/4215524930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c037040 con 0x7f832409a150 2024-04-25T10:06:54.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.307+0000 7f83217fa640 1 -- 172.21.15.16:0/4215524930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830c042070 con 0x7f832409a150 2024-04-25T10:06:54.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.307+0000 7f832b0ac640 1 -- 172.21.15.16:0/4215524930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8324149d70 con 0x7f832409a150 2024-04-25T10:06:54.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.307+0000 7f83217fa640 1 -- 172.21.15.16:0/4215524930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c030880 con 0x7f832409a150 2024-04-25T10:06:54.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.307+0000 7f832b0ac640 1 -- 172.21.15.16:0/4215524930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832414a330 con 0x7f832409a150 2024-04-25T10:06:54.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.308+0000 7f832b0ac640 1 -- 172.21.15.16:0/4215524930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83240c8690 con 0x7f832409a150 2024-04-25T10:06:54.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.308+0000 7f83217fa640 1 -- 172.21.15.16:0/4215524930 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f830c030a20 con 0x7f832409a150 2024-04-25T10:06:54.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.309+0000 7f83217fa640 1 --2- 172.21.15.16:0/4215524930 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f830406b4d0 0x7f830406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.309+0000 7f83217fa640 1 -- 172.21.15.16:0/4215524930 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f830c0aa990 con 0x7f832409a150 2024-04-25T10:06:54.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.309+0000 7f8328e48640 1 --2- 172.21.15.16:0/4215524930 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f830406b4d0 0x7f830406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.310+0000 7f8328e48640 1 --2- 172.21.15.16:0/4215524930 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f830406b4d0 0x7f830406d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f831c001e70 tx=0x7f831c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.313+0000 7f83217fa640 1 -- 172.21.15.16:0/4215524930 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83240c8690 con 0x7f832409a150 2024-04-25T10:06:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.493+0000 7f832b0ac640 1 -- 172.21.15.16:0/4215524930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83240749b0 con 0x7f832409a150 2024-04-25T10:06:54.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.495+0000 7f83217fa640 1 -- 172.21.15.16:0/4215524930 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f830c079930 con 0x7f832409a150 2024-04-25T10:06:54.501 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:54.501 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":47,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:06:52.824898+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:49.462347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:50.153232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:54.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.496+0000 7f832b0ac640 1 -- 172.21.15.16:0/4215524930 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f830406b4d0 msgr2=0x7f830406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.496+0000 7f832b0ac640 1 --2- 172.21.15.16:0/4215524930 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f830406b4d0 0x7f830406d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f831c001e70 tx=0x7f831c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:54.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.496+0000 7f832b0ac640 1 -- 172.21.15.16:0/4215524930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409a150 msgr2=0x7f83240ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.496+0000 7f832b0ac640 1 --2- 172.21.15.16:0/4215524930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409a150 0x7f83240ccb30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f830c000c00 tx=0x7f830c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.496+0000 7f832b0ac640 1 -- 172.21.15.16:0/4215524930 shutdown_connections 2024-04-25T10:06:54.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.496+0000 7f832b0ac640 1 --2- 172.21.15.16:0/4215524930 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f830406b4d0 0x7f830406d990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.496+0000 7f832b0ac640 1 --2- 172.21.15.16:0/4215524930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f832415ba10 0x7f83240cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.496+0000 7f832b0ac640 1 --2- 172.21.15.16:0/4215524930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409a150 0x7f83240ccb30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.496+0000 7f832b0ac640 1 --2- 172.21.15.16:0/4215524930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f832406c1f0 0x7f83241493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.497+0000 7f832b0ac640 1 -- 172.21.15.16:0/4215524930 >> 172.21.15.16:0/4215524930 conn(0x7f83240b32f0 msgr2=0x7f8324070180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.497+0000 7f832b0ac640 1 -- 172.21.15.16:0/4215524930 shutdown_connections 2024-04-25T10:06:54.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.497+0000 7f832b0ac640 1 -- 172.21.15.16:0/4215524930 wait complete. 2024-04-25T10:06:54.513 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:06:54.513 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:54.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.609+0000 7ff66446b640 1 Processor -- start 2024-04-25T10:06:54.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.609+0000 7ff66446b640 1 -- start start 2024-04-25T10:06:54.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.610+0000 7ff66446b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff65c0cd320 0x7ff65c0cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.610+0000 7ff66446b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff65c0cfe80 0x7ff65c0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.610+0000 7ff66446b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff65c0cd930 con 0x7ff65c15df40 2024-04-25T10:06:54.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.610+0000 7ff66446b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff65c0cdc10 con 0x7ff65c0cd320 2024-04-25T10:06:54.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.610+0000 7ff66446b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c0cdef0 con 0x7ff65c0cfe80 2024-04-25T10:06:54.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.610+0000 7ff661a06640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff65c0cd320 0x7ff65c0cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.610+0000 7ff661a06640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff65c0cd320 0x7ff65c0cf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50436/0 (socket says 172.21.15.16:50436) 2024-04-25T10:06:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.610+0000 7ff661a06640 1 -- 172.21.15.16:0/2048274414 learned_addr learned my addr 172.21.15.16:0/2048274414 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.610+0000 7ff662a08640 1 --2- 172.21.15.16:0/2048274414 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff65c0cfe80 0x7ff65c0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.611+0000 7ff661a06640 1 -- 172.21.15.16:0/2048274414 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff65c0cfe80 msgr2=0x7ff65c0cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.611+0000 7ff661a06640 1 --2- 172.21.15.16:0/2048274414 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff65c0cfe80 0x7ff65c0cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.611+0000 7ff661a06640 1 -- 172.21.15.16:0/2048274414 >> v1:172.21.15.186:6789/0 conn(0x7ff65c15df40 legacy=0x7ff65c0b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.611+0000 7ff661a06640 1 -- 172.21.15.16:0/2048274414 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c0cada0 con 0x7ff65c0cd320 2024-04-25T10:06:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.611+0000 7ff662a08640 1 --2- 172.21.15.16:0/2048274414 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff65c0cfe80 0x7ff65c0cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:54.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.611+0000 7ff661a06640 1 --2- 172.21.15.16:0/2048274414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c0cd320 0x7ff65c0cf940 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff650009870 tx=0x7ff65002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfd8116b05935bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.611+0000 7ff661205640 1 -- 172.21.15.16:0/2048274414 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff650037070 con 0x7ff65c0cd320 2024-04-25T10:06:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.612+0000 7ff661205640 1 -- 172.21.15.16:0/2048274414 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff650032440 con 0x7ff65c0cd320 2024-04-25T10:06:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.612+0000 7ff661205640 1 -- 172.21.15.16:0/2048274414 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff650032810 con 0x7ff65c0cd320 2024-04-25T10:06:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.612+0000 7ff66446b640 1 -- 172.21.15.16:0/2048274414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c0cd320 msgr2=0x7ff65c0cf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.612+0000 7ff66446b640 1 --2- 172.21.15.16:0/2048274414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c0cd320 0x7ff65c0cf940 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff650009870 tx=0x7ff65002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.612+0000 7ff66446b640 1 -- 172.21.15.16:0/2048274414 shutdown_connections 2024-04-25T10:06:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.612+0000 7ff66446b640 1 --2- 172.21.15.16:0/2048274414 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff65c0cfe80 0x7ff65c0cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.612+0000 7ff66446b640 1 --2- 172.21.15.16:0/2048274414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c0cd320 0x7ff65c0cf940 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.612+0000 7ff66446b640 1 -- 172.21.15.16:0/2048274414 >> 172.21.15.16:0/2048274414 conn(0x7ff65c0d06b0 msgr2=0x7ff65c0d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.612+0000 7ff66446b640 1 -- 172.21.15.16:0/2048274414 shutdown_connections 2024-04-25T10:06:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.612+0000 7ff66446b640 1 -- 172.21.15.16:0/2048274414 wait complete. 2024-04-25T10:06:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.613+0000 7ff66446b640 1 Processor -- start 2024-04-25T10:06:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.613+0000 7ff66446b640 1 -- start start 2024-04-25T10:06:54.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.614+0000 7ff66446b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff65c0cd320 0x7ff65c0c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.614+0000 7ff66446b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c0cfe80 0x7ff65c0b5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.614+0000 7ff66446b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff65c15df40 0x7ff65c15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.614+0000 7ff66446b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c0c4200 con 0x7ff65c0cfe80 2024-04-25T10:06:54.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.614+0000 7ff66446b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c0d02b0 con 0x7ff65c15df40 2024-04-25T10:06:54.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.614+0000 7ff66446b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c0c4380 con 0x7ff65c0cd320 2024-04-25T10:06:54.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.614+0000 7ff661a06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c0cfe80 0x7ff65c0b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.614+0000 7ff661a06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c0cfe80 0x7ff65c0b5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50438/0 (socket says 172.21.15.16:50438) 2024-04-25T10:06:54.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.614+0000 7ff661a06640 1 -- 172.21.15.16:0/2052647276 learned_addr learned my addr 172.21.15.16:0/2052647276 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:54.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.614+0000 7ff662a08640 1 --2- 172.21.15.16:0/2052647276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff65c15df40 0x7ff65c15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.614+0000 7ff662207640 1 --2- 172.21.15.16:0/2052647276 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff65c0cd320 0x7ff65c0c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.614+0000 7ff661a06640 1 -- 172.21.15.16:0/2052647276 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff65c0cd320 msgr2=0x7ff65c0c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.615+0000 7ff661a06640 1 --2- 172.21.15.16:0/2052647276 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff65c0cd320 0x7ff65c0c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.615+0000 7ff661a06640 1 -- 172.21.15.16:0/2052647276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff65c15df40 msgr2=0x7ff65c15ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.615+0000 7ff661a06640 1 --2- 172.21.15.16:0/2052647276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff65c15df40 0x7ff65c15ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.615+0000 7ff661a06640 1 -- 172.21.15.16:0/2052647276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c0ca610 con 0x7ff65c0cfe80 2024-04-25T10:06:54.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.615+0000 7ff662a08640 1 --2- 172.21.15.16:0/2052647276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff65c15df40 0x7ff65c15ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:54.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.615+0000 7ff661a06640 1 --2- 172.21.15.16:0/2052647276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c0cfe80 0x7ff65c0b5350 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff65002a200 tx=0x7ff6500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.615+0000 7ff65affd640 1 -- 172.21.15.16:0/2052647276 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff650041020 con 0x7ff65c0cfe80 2024-04-25T10:06:54.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.615+0000 7ff65affd640 1 -- 172.21.15.16:0/2052647276 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6500306c0 con 0x7ff65c0cfe80 2024-04-25T10:06:54.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.615+0000 7ff66446b640 1 -- 172.21.15.16:0/2052647276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff65c0ca330 con 0x7ff65c0cfe80 2024-04-25T10:06:54.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.615+0000 7ff65affd640 1 -- 172.21.15.16:0/2052647276 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff650037040 con 0x7ff65c0cfe80 2024-04-25T10:06:54.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.615+0000 7ff66446b640 1 -- 172.21.15.16:0/2052647276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65c0c8510 con 0x7ff65c0cfe80 2024-04-25T10:06:54.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.616+0000 7ff66446b640 1 -- 172.21.15.16:0/2052647276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff65c0c8840 con 0x7ff65c0cfe80 2024-04-25T10:06:54.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.616+0000 7ff65affd640 1 -- 172.21.15.16:0/2052647276 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff650044020 con 0x7ff65c0cfe80 2024-04-25T10:06:54.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.617+0000 7ff65affd640 1 --2- 172.21.15.16:0/2052647276 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff64806b4d0 0x7ff64806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.617+0000 7ff65affd640 1 -- 172.21.15.16:0/2052647276 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6500aaa00 con 0x7ff65c0cfe80 2024-04-25T10:06:54.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.617+0000 7ff662207640 1 --2- 172.21.15.16:0/2052647276 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff64806b4d0 0x7ff64806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.617+0000 7ff662207640 1 --2- 172.21.15.16:0/2052647276 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff64806b4d0 0x7ff64806d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff654007fd0 tx=0x7ff654009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.621+0000 7ff65affd640 1 -- 172.21.15.16:0/2052647276 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff65004b030 con 0x7ff65c0cfe80 2024-04-25T10:06:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.800+0000 7ff66446b640 1 -- 172.21.15.16:0/2052647276 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff65c0749b0 con 0x7ff64806b4d0 2024-04-25T10:06:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.807+0000 7ff65affd640 1 -- 172.21.15.16:0/2052647276 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249604 (secure 0 0 0) 0x7ff65c0749b0 con 0x7ff64806b4d0 2024-04-25T10:06:54.815 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:06:54.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.810+0000 7ff66446b640 1 -- 172.21.15.16:0/2052647276 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff64806b4d0 msgr2=0x7ff64806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.810+0000 7ff66446b640 1 --2- 172.21.15.16:0/2052647276 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff64806b4d0 0x7ff64806d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff654007fd0 tx=0x7ff654009040 comp rx=0 tx=0).stop 2024-04-25T10:06:54.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.810+0000 7ff66446b640 1 -- 172.21.15.16:0/2052647276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c0cfe80 msgr2=0x7ff65c0b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.810+0000 7ff66446b640 1 --2- 172.21.15.16:0/2052647276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c0cfe80 0x7ff65c0b5350 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff65002a200 tx=0x7ff6500046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.810+0000 7ff66446b640 1 -- 172.21.15.16:0/2052647276 shutdown_connections 2024-04-25T10:06:54.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.810+0000 7ff66446b640 1 --2- 172.21.15.16:0/2052647276 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff64806b4d0 0x7ff64806d990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.810+0000 7ff66446b640 1 --2- 172.21.15.16:0/2052647276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff65c15df40 0x7ff65c15ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.810+0000 7ff66446b640 1 --2- 172.21.15.16:0/2052647276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c0cfe80 0x7ff65c0b5350 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.810+0000 7ff66446b640 1 --2- 172.21.15.16:0/2052647276 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff65c0cd320 0x7ff65c0c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.810+0000 7ff66446b640 1 -- 172.21.15.16:0/2052647276 >> 172.21.15.16:0/2052647276 conn(0x7ff65c0d06b0 msgr2=0x7ff65c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.810+0000 7ff66446b640 1 -- 172.21.15.16:0/2052647276 shutdown_connections 2024-04-25T10:06:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.810+0000 7ff66446b640 1 -- 172.21.15.16:0/2052647276 wait complete. 2024-04-25T10:06:54.827 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-04-25T10:06:53.804369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":162,"acting":162,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":102,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242276,"kb_used_data":5296,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732444,"statfs":{"total":773094113280,"available":772846022656,"internally_reserved":0,"allocated":5423104,"data_stored":2863059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001193"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.840324+0000","last_change":"2024-04-25T10:06:51.837844+0000","last_active":"2024-04-25T10:06:51.840324+0000","last_peered":"2024-04-25T10:06:51.840324+0000","last_clean":"2024-04-25T10:06:51.840324+0000","last_became_active":"2024-04-25T10:06:51.837567+0000","last_became_peered":"2024-04-25T10:06:51.837567+0000","last_unstale":"2024-04-25T10:06:51.840324+0000","last_undegraded":"2024-04-25T10:06:51.840324+0000","last_fullsized":"2024-04-25T10:06:51.840324+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:03.830929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847252+0000","last_change":"2024-04-25T10:06:51.844389+0000","last_active":"2024-04-25T10:06:51.847252+0000","last_peered":"2024-04-25T10:06:51.847252+0000","last_clean":"2024-04-25T10:06:51.847252+0000","last_became_active":"2024-04-25T10:06:51.844082+0000","last_became_peered":"2024-04-25T10:06:51.844082+0000","last_unstale":"2024-04-25T10:06:51.847252+0000","last_undegraded":"2024-04-25T10:06:51.847252+0000","last_fullsized":"2024-04-25T10:06:51.847252+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:10.566374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847906+0000","last_change":"2024-04-25T10:06:51.845292+0000","last_active":"2024-04-25T10:06:51.847906+0000","last_peered":"2024-04-25T10:06:51.847906+0000","last_clean":"2024-04-25T10:06:51.847906+0000","last_became_active":"2024-04-25T10:06:51.844790+0000","last_became_peered":"2024-04-25T10:06:51.844790+0000","last_unstale":"2024-04-25T10:06:51.847906+0000","last_undegraded":"2024-04-25T10:06:51.847906+0000","last_fullsized":"2024-04-25T10:06:51.847906+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:51.599603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841435+0000","last_change":"2024-04-25T10:06:51.840200+0000","last_active":"2024-04-25T10:06:51.841435+0000","last_peered":"2024-04-25T10:06:51.841435+0000","last_clean":"2024-04-25T10:06:51.841435+0000","last_became_active":"2024-04-25T10:06:51.839897+0000","last_became_peered":"2024-04-25T10:06:51.839897+0000","last_unstale":"2024-04-25T10:06:51.841435+0000","last_undegraded":"2024-04-25T10:06:51.841435+0000","last_fullsized":"2024-04-25T10:06:51.841435+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:23.657828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842607+0000","last_change":"2024-04-25T10:06:51.836510+0000","last_active":"2024-04-25T10:06:51.842607+0000","last_peered":"2024-04-25T10:06:51.842607+0000","last_clean":"2024-04-25T10:06:51.842607+0000","last_became_active":"2024-04-25T10:06:51.836244+0000","last_became_peered":"2024-04-25T10:06:51.836244+0000","last_unstale":"2024-04-25T10:06:51.842607+0000","last_undegraded":"2024-04-25T10:06:51.842607+0000","last_fullsized":"2024-04-25T10:06:51.842607+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:20.690035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841155+0000","last_change":"2024-04-25T10:06:51.837947+0000","last_active":"2024-04-25T10:06:51.841155+0000","last_peered":"2024-04-25T10:06:51.841155+0000","last_clean":"2024-04-25T10:06:51.841155+0000","last_became_active":"2024-04-25T10:06:51.837635+0000","last_became_peered":"2024-04-25T10:06:51.837635+0000","last_unstale":"2024-04-25T10:06:51.841155+0000","last_undegraded":"2024-04-25T10:06:51.841155+0000","last_fullsized":"2024-04-25T10:06:51.841155+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:47.682171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854141+0000","last_change":"2024-04-25T10:06:51.847625+0000","last_active":"2024-04-25T10:06:51.854141+0000","last_peered":"2024-04-25T10:06:51.854141+0000","last_clean":"2024-04-25T10:06:51.854141+0000","last_became_active":"2024-04-25T10:06:51.847179+0000","last_became_peered":"2024-04-25T10:06:51.847179+0000","last_unstale":"2024-04-25T10:06:51.854141+0000","last_undegraded":"2024-04-25T10:06:51.854141+0000","last_fullsized":"2024-04-25T10:06:51.854141+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:40.042571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841895+0000","last_change":"2024-04-25T10:06:51.837753+0000","last_active":"2024-04-25T10:06:51.841895+0000","last_peered":"2024-04-25T10:06:51.841895+0000","last_clean":"2024-04-25T10:06:51.841895+0000","last_became_active":"2024-04-25T10:06:51.837418+0000","last_became_peered":"2024-04-25T10:06:51.837418+0000","last_unstale":"2024-04-25T10:06:51.841895+0000","last_undegraded":"2024-04-25T10:06:51.841895+0000","last_fullsized":"2024-04-25T10:06:51.841895+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:29.774279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845933+0000","last_change":"2024-04-25T10:06:51.843573+0000","last_active":"2024-04-25T10:06:51.845933+0000","last_peered":"2024-04-25T10:06:51.845933+0000","last_clean":"2024-04-25T10:06:51.845933+0000","last_became_active":"2024-04-25T10:06:51.843141+0000","last_became_peered":"2024-04-25T10:06:51.843141+0000","last_unstale":"2024-04-25T10:06:51.845933+0000","last_undegraded":"2024-04-25T10:06:51.845933+0000","last_fullsized":"2024-04-25T10:06:51.845933+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:14.554682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842933+0000","last_change":"2024-04-25T10:06:51.838802+0000","last_active":"2024-04-25T10:06:51.842933+0000","last_peered":"2024-04-25T10:06:51.842933+0000","last_clean":"2024-04-25T10:06:51.842933+0000","last_became_active":"2024-04-25T10:06:51.837871+0000","last_became_peered":"2024-04-25T10:06:51.837871+0000","last_unstale":"2024-04-25T10:06:51.842933+0000","last_undegraded":"2024-04-25T10:06:51.842933+0000","last_fullsized":"2024-04-25T10:06:51.842933+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:50.203747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.844998+0000","last_change":"2024-04-25T10:06:51.842491+0000","last_active":"2024-04-25T10:06:51.844998+0000","last_peered":"2024-04-25T10:06:51.844998+0000","last_clean":"2024-04-25T10:06:51.844998+0000","last_became_active":"2024-04-25T10:06:51.842212+0000","last_became_peered":"2024-04-25T10:06:51.842212+0000","last_unstale":"2024-04-25T10:06:51.844998+0000","last_undegraded":"2024-04-25T10:06:51.844998+0000","last_fullsized":"2024-04-25T10:06:51.844998+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:04.628909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.838191+0000","last_change":"2024-04-25T10:06:51.836207+0000","last_active":"2024-04-25T10:06:51.838191+0000","last_peered":"2024-04-25T10:06:51.838191+0000","last_clean":"2024-04-25T10:06:51.838191+0000","last_became_active":"2024-04-25T10:06:51.835856+0000","last_became_peered":"2024-04-25T10:06:51.835856+0000","last_unstale":"2024-04-25T10:06:51.838191+0000","last_undegraded":"2024-04-25T10:06:51.838191+0000","last_fullsized":"2024-04-25T10:06:51.838191+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:20.690035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.839761+0000","last_change":"2024-04-25T10:06:51.835570+0000","last_active":"2024-04-25T10:06:51.839761+0000","last_peered":"2024-04-25T10:06:51.839761+0000","last_clean":"2024-04-25T10:06:51.839761+0000","last_became_active":"2024-04-25T10:06:51.835287+0000","last_became_peered":"2024-04-25T10:06:51.835287+0000","last_unstale":"2024-04-25T10:06:51.839761+0000","last_undegraded":"2024-04-25T10:06:51.839761+0000","last_fullsized":"2024-04-25T10:06:51.839761+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:58.880955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846553+0000","last_change":"2024-04-25T10:06:51.845653+0000","last_active":"2024-04-25T10:06:51.846553+0000","last_peered":"2024-04-25T10:06:51.846553+0000","last_clean":"2024-04-25T10:06:51.846553+0000","last_became_active":"2024-04-25T10:06:51.845387+0000","last_became_peered":"2024-04-25T10:06:51.845387+0000","last_unstale":"2024-04-25T10:06:51.846553+0000","last_undegraded":"2024-04-25T10:06:51.846553+0000","last_fullsized":"2024-04-25T10:06:51.846553+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:22.306267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841737+0000","last_change":"2024-04-25T10:06:51.836879+0000","last_active":"2024-04-25T10:06:51.841737+0000","last_peered":"2024-04-25T10:06:51.841737+0000","last_clean":"2024-04-25T10:06:51.841737+0000","last_became_active":"2024-04-25T10:06:51.835980+0000","last_became_peered":"2024-04-25T10:06:51.835980+0000","last_unstale":"2024-04-25T10:06:51.841737+0000","last_undegraded":"2024-04-25T10:06:51.841737+0000","last_fullsized":"2024-04-25T10:06:51.841737+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:30.749971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841910+0000","last_change":"2024-04-25T10:06:51.838280+0000","last_active":"2024-04-25T10:06:51.841910+0000","last_peered":"2024-04-25T10:06:51.841910+0000","last_clean":"2024-04-25T10:06:51.841910+0000","last_became_active":"2024-04-25T10:06:51.837813+0000","last_became_peered":"2024-04-25T10:06:51.837813+0000","last_unstale":"2024-04-25T10:06:51.841910+0000","last_undegraded":"2024-04-25T10:06:51.841910+0000","last_fullsized":"2024-04-25T10:06:51.841910+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:58.880955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.833222+0000","last_change":"2024-04-25T10:06:51.831343+0000","last_active":"2024-04-25T10:06:51.833222+0000","last_peered":"2024-04-25T10:06:51.833222+0000","last_clean":"2024-04-25T10:06:51.833222+0000","last_became_active":"2024-04-25T10:06:51.831040+0000","last_became_peered":"2024-04-25T10:06:51.831040+0000","last_unstale":"2024-04-25T10:06:51.833222+0000","last_undegraded":"2024-04-25T10:06:51.833222+0000","last_fullsized":"2024-04-25T10:06:51.833222+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:04.569426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.832424+0000","last_change":"2024-04-25T10:06:51.827202+0000","last_active":"2024-04-25T10:06:51.832424+0000","last_peered":"2024-04-25T10:06:51.832424+0000","last_clean":"2024-04-25T10:06:51.832424+0000","last_became_active":"2024-04-25T10:06:51.826893+0000","last_became_peered":"2024-04-25T10:06:51.826893+0000","last_unstale":"2024-04-25T10:06:51.832424+0000","last_undegraded":"2024-04-25T10:06:51.832424+0000","last_fullsized":"2024-04-25T10:06:51.832424+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:25.546409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.836212+0000","last_change":"2024-04-25T10:06:51.835353+0000","last_active":"2024-04-25T10:06:51.836212+0000","last_peered":"2024-04-25T10:06:51.836212+0000","last_clean":"2024-04-25T10:06:51.836212+0000","last_became_active":"2024-04-25T10:06:51.834941+0000","last_became_peered":"2024-04-25T10:06:51.834941+0000","last_unstale":"2024-04-25T10:06:51.836212+0000","last_undegraded":"2024-04-25T10:06:51.836212+0000","last_fullsized":"2024-04-25T10:06:51.836212+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:48.121882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.833241+0000","last_change":"2024-04-25T10:06:51.830316+0000","last_active":"2024-04-25T10:06:51.833241+0000","last_peered":"2024-04-25T10:06:51.833241+0000","last_clean":"2024-04-25T10:06:51.833241+0000","last_became_active":"2024-04-25T10:06:51.830039+0000","last_became_peered":"2024-04-25T10:06:51.830039+0000","last_unstale":"2024-04-25T10:06:51.833241+0000","last_undegraded":"2024-04-25T10:06:51.833241+0000","last_fullsized":"2024-04-25T10:06:51.833241+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:01.498621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.833261+0000","last_change":"2024-04-25T10:06:51.830470+0000","last_active":"2024-04-25T10:06:51.833261+0000","last_peered":"2024-04-25T10:06:51.833261+0000","last_clean":"2024-04-25T10:06:51.833261+0000","last_became_active":"2024-04-25T10:06:51.830180+0000","last_became_peered":"2024-04-25T10:06:51.830180+0000","last_unstale":"2024-04-25T10:06:51.833261+0000","last_undegraded":"2024-04-25T10:06:51.833261+0000","last_fullsized":"2024-04-25T10:06:51.833261+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:17.127193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.840946+0000","last_change":"2024-04-25T10:06:51.839558+0000","last_active":"2024-04-25T10:06:51.840946+0000","last_peered":"2024-04-25T10:06:51.840946+0000","last_clean":"2024-04-25T10:06:51.840946+0000","last_became_active":"2024-04-25T10:06:51.839271+0000","last_became_peered":"2024-04-25T10:06:51.839271+0000","last_unstale":"2024-04-25T10:06:51.840946+0000","last_undegraded":"2024-04-25T10:06:51.840946+0000","last_fullsized":"2024-04-25T10:06:51.840946+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:11.162077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.832516+0000","last_change":"2024-04-25T10:06:51.828040+0000","last_active":"2024-04-25T10:06:51.832516+0000","last_peered":"2024-04-25T10:06:51.832516+0000","last_clean":"2024-04-25T10:06:51.832516+0000","last_became_active":"2024-04-25T10:06:51.826090+0000","last_became_peered":"2024-04-25T10:06:51.826090+0000","last_unstale":"2024-04-25T10:06:51.832516+0000","last_undegraded":"2024-04-25T10:06:51.832516+0000","last_fullsized":"2024-04-25T10:06:51.832516+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:09.382894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842444+0000","last_change":"2024-04-25T10:06:51.837883+0000","last_active":"2024-04-25T10:06:51.842444+0000","last_peered":"2024-04-25T10:06:51.842444+0000","last_clean":"2024-04-25T10:06:51.842444+0000","last_became_active":"2024-04-25T10:06:51.837631+0000","last_became_peered":"2024-04-25T10:06:51.837631+0000","last_unstale":"2024-04-25T10:06:51.842444+0000","last_undegraded":"2024-04-25T10:06:51.842444+0000","last_fullsized":"2024-04-25T10:06:51.842444+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:14.554682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846377+0000","last_change":"2024-04-25T10:06:51.842565+0000","last_active":"2024-04-25T10:06:51.846377+0000","last_peered":"2024-04-25T10:06:51.846377+0000","last_clean":"2024-04-25T10:06:51.846377+0000","last_became_active":"2024-04-25T10:06:51.842251+0000","last_became_peered":"2024-04-25T10:06:51.842251+0000","last_unstale":"2024-04-25T10:06:51.846377+0000","last_undegraded":"2024-04-25T10:06:51.846377+0000","last_fullsized":"2024-04-25T10:06:51.846377+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:32.311454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854076+0000","last_change":"2024-04-25T10:06:50.815053+0000","last_active":"2024-04-25T10:06:51.854076+0000","last_peered":"2024-04-25T10:06:51.854076+0000","last_clean":"2024-04-25T10:06:51.854076+0000","last_became_active":"2024-04-25T10:06:50.814623+0000","last_became_peered":"2024-04-25T10:06:50.814623+0000","last_unstale":"2024-04-25T10:06:51.854076+0000","last_undegraded":"2024-04-25T10:06:51.854076+0000","last_fullsized":"2024-04-25T10:06:51.854076+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:16.218950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846459+0000","last_change":"2024-04-25T10:06:51.844956+0000","last_active":"2024-04-25T10:06:51.846459+0000","last_peered":"2024-04-25T10:06:51.846459+0000","last_clean":"2024-04-25T10:06:51.846459+0000","last_became_active":"2024-04-25T10:06:51.844611+0000","last_became_peered":"2024-04-25T10:06:51.844611+0000","last_unstale":"2024-04-25T10:06:51.846459+0000","last_undegraded":"2024-04-25T10:06:51.846459+0000","last_fullsized":"2024-04-25T10:06:51.846459+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:50.170873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.848558+0000","last_change":"2024-04-25T10:06:50.822263+0000","last_active":"2024-04-25T10:06:51.848558+0000","last_peered":"2024-04-25T10:06:51.848558+0000","last_clean":"2024-04-25T10:06:51.848558+0000","last_became_active":"2024-04-25T10:06:50.821985+0000","last_became_peered":"2024-04-25T10:06:50.821985+0000","last_unstale":"2024-04-25T10:06:51.848558+0000","last_undegraded":"2024-04-25T10:06:51.848558+0000","last_fullsized":"2024-04-25T10:06:51.848558+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:37.641598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847288+0000","last_change":"2024-04-25T10:06:51.844989+0000","last_active":"2024-04-25T10:06:51.847288+0000","last_peered":"2024-04-25T10:06:51.847288+0000","last_clean":"2024-04-25T10:06:51.847288+0000","last_became_active":"2024-04-25T10:06:51.844614+0000","last_became_peered":"2024-04-25T10:06:51.844614+0000","last_unstale":"2024-04-25T10:06:51.847288+0000","last_undegraded":"2024-04-25T10:06:51.847288+0000","last_fullsized":"2024-04-25T10:06:51.847288+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:50.203747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.853873+0000","last_change":"2024-04-25T10:06:50.818405+0000","last_active":"2024-04-25T10:06:51.853873+0000","last_peered":"2024-04-25T10:06:51.853873+0000","last_clean":"2024-04-25T10:06:51.853873+0000","last_became_active":"2024-04-25T10:06:50.818118+0000","last_became_peered":"2024-04-25T10:06:50.818118+0000","last_unstale":"2024-04-25T10:06:51.853873+0000","last_undegraded":"2024-04-25T10:06:51.853873+0000","last_fullsized":"2024-04-25T10:06:51.853873+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:08.381421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.848517+0000","last_change":"2024-04-25T10:06:50.821564+0000","last_active":"2024-04-25T10:06:51.848517+0000","last_peered":"2024-04-25T10:06:51.848517+0000","last_clean":"2024-04-25T10:06:51.848517+0000","last_became_active":"2024-04-25T10:06:50.821257+0000","last_became_peered":"2024-04-25T10:06:50.821257+0000","last_unstale":"2024-04-25T10:06:51.848517+0000","last_undegraded":"2024-04-25T10:06:51.848517+0000","last_fullsized":"2024-04-25T10:06:51.848517+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:25.919994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834314+0000","last_change":"2024-04-25T10:06:51.832079+0000","last_active":"2024-04-25T10:06:51.834314+0000","last_peered":"2024-04-25T10:06:51.834314+0000","last_clean":"2024-04-25T10:06:51.834314+0000","last_became_active":"2024-04-25T10:06:51.831785+0000","last_became_peered":"2024-04-25T10:06:51.831785+0000","last_unstale":"2024-04-25T10:06:51.834314+0000","last_undegraded":"2024-04-25T10:06:51.834314+0000","last_fullsized":"2024-04-25T10:06:51.834314+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:27.917421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.853752+0000","last_change":"2024-04-25T10:06:50.819361+0000","last_active":"2024-04-25T10:06:51.853752+0000","last_peered":"2024-04-25T10:06:51.853752+0000","last_clean":"2024-04-25T10:06:51.853752+0000","last_became_active":"2024-04-25T10:06:50.819093+0000","last_became_peered":"2024-04-25T10:06:50.819093+0000","last_unstale":"2024-04-25T10:06:51.853752+0000","last_undegraded":"2024-04-25T10:06:51.853752+0000","last_fullsized":"2024-04-25T10:06:51.853752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:24.544937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841867+0000","last_change":"2024-04-25T10:06:51.835101+0000","last_active":"2024-04-25T10:06:51.841867+0000","last_peered":"2024-04-25T10:06:51.841867+0000","last_clean":"2024-04-25T10:06:51.841867+0000","last_became_active":"2024-04-25T10:06:51.834291+0000","last_became_peered":"2024-04-25T10:06:51.834291+0000","last_unstale":"2024-04-25T10:06:51.841867+0000","last_undegraded":"2024-04-25T10:06:51.841867+0000","last_fullsized":"2024-04-25T10:06:51.841867+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:22.306267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845318+0000","last_change":"2024-04-25T10:06:50.822762+0000","last_active":"2024-04-25T10:06:51.845318+0000","last_peered":"2024-04-25T10:06:51.845318+0000","last_clean":"2024-04-25T10:06:51.845318+0000","last_became_active":"2024-04-25T10:06:50.822463+0000","last_became_peered":"2024-04-25T10:06:50.822463+0000","last_unstale":"2024-04-25T10:06:51.845318+0000","last_undegraded":"2024-04-25T10:06:51.845318+0000","last_fullsized":"2024-04-25T10:06:51.845318+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:18.363517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841750+0000","last_change":"2024-04-25T10:06:51.833997+0000","last_active":"2024-04-25T10:06:51.841750+0000","last_peered":"2024-04-25T10:06:51.841750+0000","last_clean":"2024-04-25T10:06:51.841750+0000","last_became_active":"2024-04-25T10:06:51.833694+0000","last_became_peered":"2024-04-25T10:06:51.833694+0000","last_unstale":"2024-04-25T10:06:51.841750+0000","last_undegraded":"2024-04-25T10:06:51.841750+0000","last_fullsized":"2024-04-25T10:06:51.841750+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:50.170873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828476+0000","last_change":"2024-04-25T10:06:50.818945+0000","last_active":"2024-04-25T10:06:51.828476+0000","last_peered":"2024-04-25T10:06:51.828476+0000","last_clean":"2024-04-25T10:06:51.828476+0000","last_became_active":"2024-04-25T10:06:50.818631+0000","last_became_peered":"2024-04-25T10:06:50.818631+0000","last_unstale":"2024-04-25T10:06:51.828476+0000","last_undegraded":"2024-04-25T10:06:51.828476+0000","last_fullsized":"2024-04-25T10:06:51.828476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:49.202275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.839815+0000","last_change":"2024-04-25T10:06:51.837313+0000","last_active":"2024-04-25T10:06:51.839815+0000","last_peered":"2024-04-25T10:06:51.839815+0000","last_clean":"2024-04-25T10:06:51.839815+0000","last_became_active":"2024-04-25T10:06:51.837024+0000","last_became_peered":"2024-04-25T10:06:51.837024+0000","last_unstale":"2024-04-25T10:06:51.839815+0000","last_undegraded":"2024-04-25T10:06:51.839815+0000","last_fullsized":"2024-04-25T10:06:51.839815+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:28.322485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.825554+0000","last_change":"2024-04-25T10:06:50.818743+0000","last_active":"2024-04-25T10:06:51.825554+0000","last_peered":"2024-04-25T10:06:51.825554+0000","last_clean":"2024-04-25T10:06:51.825554+0000","last_became_active":"2024-04-25T10:06:50.818359+0000","last_became_peered":"2024-04-25T10:06:50.818359+0000","last_unstale":"2024-04-25T10:06:51.825554+0000","last_undegraded":"2024-04-25T10:06:51.825554+0000","last_fullsized":"2024-04-25T10:06:51.825554+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:47.120410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.848307+0000","last_change":"2024-04-25T10:06:51.846840+0000","last_active":"2024-04-25T10:06:51.848307+0000","last_peered":"2024-04-25T10:06:51.848307+0000","last_clean":"2024-04-25T10:06:51.848307+0000","last_became_active":"2024-04-25T10:06:51.846314+0000","last_became_peered":"2024-04-25T10:06:51.846314+0000","last_unstale":"2024-04-25T10:06:51.848307+0000","last_undegraded":"2024-04-25T10:06:51.848307+0000","last_fullsized":"2024-04-25T10:06:51.848307+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:29.810440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.823148+0000","last_change":"2024-04-25T10:06:50.819769+0000","last_active":"2024-04-25T10:06:51.823148+0000","last_peered":"2024-04-25T10:06:51.823148+0000","last_clean":"2024-04-25T10:06:51.823148+0000","last_became_active":"2024-04-25T10:06:50.819517+0000","last_became_peered":"2024-04-25T10:06:50.819517+0000","last_unstale":"2024-04-25T10:06:51.823148+0000","last_undegraded":"2024-04-25T10:06:51.823148+0000","last_fullsized":"2024-04-25T10:06:51.823148+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:15.288040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834870+0000","last_change":"2024-04-25T10:06:51.832027+0000","last_active":"2024-04-25T10:06:51.834870+0000","last_peered":"2024-04-25T10:06:51.834870+0000","last_clean":"2024-04-25T10:06:51.834870+0000","last_became_active":"2024-04-25T10:06:51.831718+0000","last_became_peered":"2024-04-25T10:06:51.831718+0000","last_unstale":"2024-04-25T10:06:51.834870+0000","last_undegraded":"2024-04-25T10:06:51.834870+0000","last_fullsized":"2024-04-25T10:06:51.834870+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:30.250121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.825643+0000","last_change":"2024-04-25T10:06:50.825390+0000","last_active":"2024-04-25T10:06:51.825643+0000","last_peered":"2024-04-25T10:06:51.825643+0000","last_clean":"2024-04-25T10:06:51.825643+0000","last_became_active":"2024-04-25T10:06:50.825085+0000","last_became_peered":"2024-04-25T10:06:50.825085+0000","last_unstale":"2024-04-25T10:06:51.825643+0000","last_undegraded":"2024-04-25T10:06:51.825643+0000","last_fullsized":"2024-04-25T10:06:51.825643+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:10.160604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845381+0000","last_change":"2024-04-25T10:06:50.818510+0000","last_active":"2024-04-25T10:06:51.845381+0000","last_peered":"2024-04-25T10:06:51.845381+0000","last_clean":"2024-04-25T10:06:51.845381+0000","last_became_active":"2024-04-25T10:06:50.818217+0000","last_became_peered":"2024-04-25T10:06:50.818217+0000","last_unstale":"2024-04-25T10:06:51.845381+0000","last_undegraded":"2024-04-25T10:06:51.845381+0000","last_fullsized":"2024-04-25T10:06:51.845381+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.521754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841431+0000","last_change":"2024-04-25T10:06:51.838484+0000","last_active":"2024-04-25T10:06:51.841431+0000","last_peered":"2024-04-25T10:06:51.841431+0000","last_clean":"2024-04-25T10:06:51.841431+0000","last_became_active":"2024-04-25T10:06:51.838211+0000","last_became_peered":"2024-04-25T10:06:51.838211+0000","last_unstale":"2024-04-25T10:06:51.841431+0000","last_undegraded":"2024-04-25T10:06:51.841431+0000","last_fullsized":"2024-04-25T10:06:51.841431+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:46.417710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.856308+0000","last_change":"2024-04-25T10:06:50.823945+0000","last_active":"2024-04-25T10:06:51.856308+0000","last_peered":"2024-04-25T10:06:51.856308+0000","last_clean":"2024-04-25T10:06:51.856308+0000","last_became_active":"2024-04-25T10:06:50.823670+0000","last_became_peered":"2024-04-25T10:06:50.823670+0000","last_unstale":"2024-04-25T10:06:51.856308+0000","last_undegraded":"2024-04-25T10:06:51.856308+0000","last_fullsized":"2024-04-25T10:06:51.856308+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:03.627437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845672+0000","last_change":"2024-04-25T10:06:51.843182+0000","last_active":"2024-04-25T10:06:51.845672+0000","last_peered":"2024-04-25T10:06:51.845672+0000","last_clean":"2024-04-25T10:06:51.845672+0000","last_became_active":"2024-04-25T10:06:51.842811+0000","last_became_peered":"2024-04-25T10:06:51.842811+0000","last_unstale":"2024-04-25T10:06:51.845672+0000","last_undegraded":"2024-04-25T10:06:51.845672+0000","last_fullsized":"2024-04-25T10:06:51.845672+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:43.266424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.850017+0000","last_change":"2024-04-25T10:06:50.823653+0000","last_active":"2024-04-25T10:06:51.850017+0000","last_peered":"2024-04-25T10:06:51.850017+0000","last_clean":"2024-04-25T10:06:51.850017+0000","last_became_active":"2024-04-25T10:06:50.823373+0000","last_became_peered":"2024-04-25T10:06:50.823373+0000","last_unstale":"2024-04-25T10:06:51.850017+0000","last_undegraded":"2024-04-25T10:06:51.850017+0000","last_fullsized":"2024-04-25T10:06:51.850017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:19.313862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842243+0000","last_change":"2024-04-25T10:06:51.836404+0000","last_active":"2024-04-25T10:06:51.842243+0000","last_peered":"2024-04-25T10:06:51.842243+0000","last_clean":"2024-04-25T10:06:51.842243+0000","last_became_active":"2024-04-25T10:06:51.835875+0000","last_became_peered":"2024-04-25T10:06:51.835875+0000","last_unstale":"2024-04-25T10:06:51.842243+0000","last_undegraded":"2024-04-25T10:06:51.842243+0000","last_fullsized":"2024-04-25T10:06:51.842243+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:30.250121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828780+0000","last_change":"2024-04-25T10:06:50.824101+0000","last_active":"2024-04-25T10:06:51.828780+0000","last_peered":"2024-04-25T10:06:51.828780+0000","last_clean":"2024-04-25T10:06:51.828780+0000","last_became_active":"2024-04-25T10:06:50.823800+0000","last_became_peered":"2024-04-25T10:06:50.823800+0000","last_unstale":"2024-04-25T10:06:51.828780+0000","last_undegraded":"2024-04-25T10:06:51.828780+0000","last_fullsized":"2024-04-25T10:06:51.828780+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:10.364676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846106+0000","last_change":"2024-04-25T10:06:51.842607+0000","last_active":"2024-04-25T10:06:51.846106+0000","last_peered":"2024-04-25T10:06:51.846106+0000","last_clean":"2024-04-25T10:06:51.846106+0000","last_became_active":"2024-04-25T10:06:51.842296+0000","last_became_peered":"2024-04-25T10:06:51.842296+0000","last_unstale":"2024-04-25T10:06:51.846106+0000","last_undegraded":"2024-04-25T10:06:51.846106+0000","last_fullsized":"2024-04-25T10:06:51.846106+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:26.337568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.831684+0000","last_change":"2024-04-25T10:06:50.826028+0000","last_active":"2024-04-25T10:06:51.831684+0000","last_peered":"2024-04-25T10:06:51.831684+0000","last_clean":"2024-04-25T10:06:51.831684+0000","last_became_active":"2024-04-25T10:06:50.825717+0000","last_became_peered":"2024-04-25T10:06:50.825717+0000","last_unstale":"2024-04-25T10:06:51.831684+0000","last_undegraded":"2024-04-25T10:06:51.831684+0000","last_fullsized":"2024-04-25T10:06:51.831684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:28.284253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834918+0000","last_change":"2024-04-25T10:06:51.833850+0000","last_active":"2024-04-25T10:06:51.834918+0000","last_peered":"2024-04-25T10:06:51.834918+0000","last_clean":"2024-04-25T10:06:51.834918+0000","last_became_active":"2024-04-25T10:06:51.833584+0000","last_became_peered":"2024-04-25T10:06:51.833584+0000","last_unstale":"2024-04-25T10:06:51.834918+0000","last_undegraded":"2024-04-25T10:06:51.834918+0000","last_fullsized":"2024-04-25T10:06:51.834918+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:01.261252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854164+0000","last_change":"2024-04-25T10:06:50.820905+0000","last_active":"2024-04-25T10:06:51.854164+0000","last_peered":"2024-04-25T10:06:51.854164+0000","last_clean":"2024-04-25T10:06:51.854164+0000","last_became_active":"2024-04-25T10:06:50.820642+0000","last_became_peered":"2024-04-25T10:06:50.820642+0000","last_unstale":"2024-04-25T10:06:51.854164+0000","last_undegraded":"2024-04-25T10:06:51.854164+0000","last_fullsized":"2024-04-25T10:06:51.854164+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:22.656356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846398+0000","last_change":"2024-04-25T10:06:51.842872+0000","last_active":"2024-04-25T10:06:51.846398+0000","last_peered":"2024-04-25T10:06:51.846398+0000","last_clean":"2024-04-25T10:06:51.846398+0000","last_became_active":"2024-04-25T10:06:51.842550+0000","last_became_peered":"2024-04-25T10:06:51.842550+0000","last_unstale":"2024-04-25T10:06:51.846398+0000","last_undegraded":"2024-04-25T10:06:51.846398+0000","last_fullsized":"2024-04-25T10:06:51.846398+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:51.599603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:49.800151+0000","last_change":"2024-04-25T10:06:49.800151+0000","last_active":"2024-04-25T10:06:49.800151+0000","last_peered":"2024-04-25T10:06:49.800151+0000","last_clean":"2024-04-25T10:06:49.800151+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:49.800151+0000","last_undegraded":"2024-04-25T10:06:49.800151+0000","last_fullsized":"2024-04-25T10:06:49.800151+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845649+0000","last_change":"2024-04-25T10:06:51.840224+0000","last_active":"2024-04-25T10:06:51.845649+0000","last_peered":"2024-04-25T10:06:51.845649+0000","last_clean":"2024-04-25T10:06:51.845649+0000","last_became_active":"2024-04-25T10:06:51.839917+0000","last_became_peered":"2024-04-25T10:06:51.839917+0000","last_unstale":"2024-04-25T10:06:51.845649+0000","last_undegraded":"2024-04-25T10:06:51.845649+0000","last_fullsized":"2024-04-25T10:06:51.845649+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:43.266424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.859198+0000","last_change":"2024-04-25T10:06:50.818607+0000","last_active":"2024-04-25T10:06:51.859198+0000","last_peered":"2024-04-25T10:06:51.859198+0000","last_clean":"2024-04-25T10:06:51.859198+0000","last_became_active":"2024-04-25T10:06:50.818315+0000","last_became_peered":"2024-04-25T10:06:50.818315+0000","last_unstale":"2024-04-25T10:06:51.859198+0000","last_undegraded":"2024-04-25T10:06:51.859198+0000","last_fullsized":"2024-04-25T10:06:51.859198+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:50.599925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.835690+0000","last_change":"2024-04-25T10:06:51.834754+0000","last_active":"2024-04-25T10:06:51.835690+0000","last_peered":"2024-04-25T10:06:51.835690+0000","last_clean":"2024-04-25T10:06:51.835690+0000","last_became_active":"2024-04-25T10:06:51.834425+0000","last_became_peered":"2024-04-25T10:06:51.834425+0000","last_unstale":"2024-04-25T10:06:51.835690+0000","last_undegraded":"2024-04-25T10:06:51.835690+0000","last_fullsized":"2024-04-25T10:06:51.835690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:17.220423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854430+0000","last_change":"2024-04-25T10:06:50.820485+0000","last_active":"2024-04-25T10:06:51.854430+0000","last_peered":"2024-04-25T10:06:51.854430+0000","last_clean":"2024-04-25T10:06:51.854430+0000","last_became_active":"2024-04-25T10:06:50.820215+0000","last_became_peered":"2024-04-25T10:06:50.820215+0000","last_unstale":"2024-04-25T10:06:51.854430+0000","last_undegraded":"2024-04-25T10:06:51.854430+0000","last_fullsized":"2024-04-25T10:06:51.854430+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:19.313862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847857+0000","last_change":"2024-04-25T10:06:51.845056+0000","last_active":"2024-04-25T10:06:51.847857+0000","last_peered":"2024-04-25T10:06:51.847857+0000","last_clean":"2024-04-25T10:06:51.847857+0000","last_became_active":"2024-04-25T10:06:51.844448+0000","last_became_peered":"2024-04-25T10:06:51.844448+0000","last_unstale":"2024-04-25T10:06:51.847857+0000","last_undegraded":"2024-04-25T10:06:51.847857+0000","last_fullsized":"2024-04-25T10:06:51.847857+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:26.337568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.831449+0000","last_change":"2024-04-25T10:06:50.825791+0000","last_active":"2024-04-25T10:06:51.831449+0000","last_peered":"2024-04-25T10:06:51.831449+0000","last_clean":"2024-04-25T10:06:51.831449+0000","last_became_active":"2024-04-25T10:06:50.825467+0000","last_became_peered":"2024-04-25T10:06:50.825467+0000","last_unstale":"2024-04-25T10:06:51.831449+0000","last_undegraded":"2024-04-25T10:06:51.831449+0000","last_fullsized":"2024-04-25T10:06:51.831449+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.521754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845737+0000","last_change":"2024-04-25T10:06:51.842252+0000","last_active":"2024-04-25T10:06:51.845737+0000","last_peered":"2024-04-25T10:06:51.845737+0000","last_clean":"2024-04-25T10:06:51.845737+0000","last_became_active":"2024-04-25T10:06:51.841999+0000","last_became_peered":"2024-04-25T10:06:51.841999+0000","last_unstale":"2024-04-25T10:06:51.845737+0000","last_undegraded":"2024-04-25T10:06:51.845737+0000","last_fullsized":"2024-04-25T10:06:51.845737+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:15.390698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.851706+0000","last_change":"2024-04-25T10:06:50.821347+0000","last_active":"2024-04-25T10:06:51.851706+0000","last_peered":"2024-04-25T10:06:51.851706+0000","last_clean":"2024-04-25T10:06:51.851706+0000","last_became_active":"2024-04-25T10:06:50.821073+0000","last_became_peered":"2024-04-25T10:06:50.821073+0000","last_unstale":"2024-04-25T10:06:51.851706+0000","last_undegraded":"2024-04-25T10:06:51.851706+0000","last_fullsized":"2024-04-25T10:06:51.851706+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:02.829457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842908+0000","last_change":"2024-04-25T10:06:51.838646+0000","last_active":"2024-04-25T10:06:51.842908+0000","last_peered":"2024-04-25T10:06:51.842908+0000","last_clean":"2024-04-25T10:06:51.842908+0000","last_became_active":"2024-04-25T10:06:51.835104+0000","last_became_peered":"2024-04-25T10:06:51.835104+0000","last_unstale":"2024-04-25T10:06:51.842908+0000","last_undegraded":"2024-04-25T10:06:51.842908+0000","last_fullsized":"2024-04-25T10:06:51.842908+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:10.566374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828795+0000","last_change":"2024-04-25T10:06:50.827168+0000","last_active":"2024-04-25T10:06:51.828795+0000","last_peered":"2024-04-25T10:06:51.828795+0000","last_clean":"2024-04-25T10:06:51.828795+0000","last_became_active":"2024-04-25T10:06:50.826210+0000","last_became_peered":"2024-04-25T10:06:50.826210+0000","last_unstale":"2024-04-25T10:06:51.828795+0000","last_undegraded":"2024-04-25T10:06:51.828795+0000","last_fullsized":"2024-04-25T10:06:51.828795+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.650773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.825362+0000","last_change":"2024-04-25T10:06:50.821404+0000","last_active":"2024-04-25T10:06:51.825362+0000","last_peered":"2024-04-25T10:06:51.825362+0000","last_clean":"2024-04-25T10:06:51.825362+0000","last_became_active":"2024-04-25T10:06:50.821129+0000","last_became_peered":"2024-04-25T10:06:50.821129+0000","last_unstale":"2024-04-25T10:06:51.825362+0000","last_undegraded":"2024-04-25T10:06:51.825362+0000","last_fullsized":"2024-04-25T10:06:51.825362+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.650773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.823231+0000","last_change":"2024-04-25T10:06:50.825554+0000","last_active":"2024-04-25T10:06:51.823231+0000","last_peered":"2024-04-25T10:06:51.823231+0000","last_clean":"2024-04-25T10:06:51.823231+0000","last_became_active":"2024-04-25T10:06:50.825226+0000","last_became_peered":"2024-04-25T10:06:50.825226+0000","last_unstale":"2024-04-25T10:06:51.823231+0000","last_undegraded":"2024-04-25T10:06:51.823231+0000","last_fullsized":"2024-04-25T10:06:51.823231+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:16.990316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834352+0000","last_change":"2024-04-25T10:06:51.832937+0000","last_active":"2024-04-25T10:06:51.834352+0000","last_peered":"2024-04-25T10:06:51.834352+0000","last_clean":"2024-04-25T10:06:51.834352+0000","last_became_active":"2024-04-25T10:06:51.832620+0000","last_became_peered":"2024-04-25T10:06:51.832620+0000","last_unstale":"2024-04-25T10:06:51.834352+0000","last_undegraded":"2024-04-25T10:06:51.834352+0000","last_fullsized":"2024-04-25T10:06:51.834352+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:47.015585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:49.800151+0000","last_change":"2024-04-25T10:06:49.800151+0000","last_active":"2024-04-25T10:06:49.800151+0000","last_peered":"2024-04-25T10:06:49.800151+0000","last_clean":"2024-04-25T10:06:49.800151+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:49.800151+0000","last_undegraded":"2024-04-25T10:06:49.800151+0000","last_fullsized":"2024-04-25T10:06:49.800151+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841715+0000","last_change":"2024-04-25T10:06:51.835802+0000","last_active":"2024-04-25T10:06:51.841715+0000","last_peered":"2024-04-25T10:06:51.841715+0000","last_clean":"2024-04-25T10:06:51.841715+0000","last_became_active":"2024-04-25T10:06:51.835196+0000","last_became_peered":"2024-04-25T10:06:51.835196+0000","last_unstale":"2024-04-25T10:06:51.841715+0000","last_undegraded":"2024-04-25T10:06:51.841715+0000","last_fullsized":"2024-04-25T10:06:51.841715+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:27.916081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828499+0000","last_change":"2024-04-25T10:06:50.826677+0000","last_active":"2024-04-25T10:06:51.828499+0000","last_peered":"2024-04-25T10:06:51.828499+0000","last_clean":"2024-04-25T10:06:51.828499+0000","last_became_active":"2024-04-25T10:06:50.826114+0000","last_became_peered":"2024-04-25T10:06:50.826114+0000","last_unstale":"2024-04-25T10:06:51.828499+0000","last_undegraded":"2024-04-25T10:06:51.828499+0000","last_fullsized":"2024-04-25T10:06:51.828499+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:39.041098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841453+0000","last_change":"2024-04-25T10:06:51.837949+0000","last_active":"2024-04-25T10:06:51.841453+0000","last_peered":"2024-04-25T10:06:51.841453+0000","last_clean":"2024-04-25T10:06:51.841453+0000","last_became_active":"2024-04-25T10:06:51.837681+0000","last_became_peered":"2024-04-25T10:06:51.837681+0000","last_unstale":"2024-04-25T10:06:51.841453+0000","last_undegraded":"2024-04-25T10:06:51.841453+0000","last_fullsized":"2024-04-25T10:06:51.841453+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:17.563413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828714+0000","last_change":"2024-04-25T10:06:50.826236+0000","last_active":"2024-04-25T10:06:51.828714+0000","last_peered":"2024-04-25T10:06:51.828714+0000","last_clean":"2024-04-25T10:06:51.828714+0000","last_became_active":"2024-04-25T10:06:50.825899+0000","last_became_peered":"2024-04-25T10:06:50.825899+0000","last_unstale":"2024-04-25T10:06:51.828714+0000","last_undegraded":"2024-04-25T10:06:51.828714+0000","last_fullsized":"2024-04-25T10:06:51.828714+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:15.288040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845716+0000","last_change":"2024-04-25T10:06:51.840773+0000","last_active":"2024-04-25T10:06:51.845716+0000","last_peered":"2024-04-25T10:06:51.845716+0000","last_clean":"2024-04-25T10:06:51.845716+0000","last_became_active":"2024-04-25T10:06:51.840412+0000","last_became_peered":"2024-04-25T10:06:51.840412+0000","last_unstale":"2024-04-25T10:06:51.845716+0000","last_undegraded":"2024-04-25T10:06:51.845716+0000","last_fullsized":"2024-04-25T10:06:51.845716+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:58.954708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.827378+0000","last_change":"2024-04-25T10:06:50.819732+0000","last_active":"2024-04-25T10:06:51.827378+0000","last_peered":"2024-04-25T10:06:51.827378+0000","last_clean":"2024-04-25T10:06:51.827378+0000","last_became_active":"2024-04-25T10:06:50.819448+0000","last_became_peered":"2024-04-25T10:06:50.819448+0000","last_unstale":"2024-04-25T10:06:51.827378+0000","last_undegraded":"2024-04-25T10:06:51.827378+0000","last_fullsized":"2024-04-25T10:06:51.827378+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:09.564901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.823889+0000","last_change":"2024-04-25T10:06:50.821944+0000","last_active":"2024-04-25T10:06:51.823889+0000","last_peered":"2024-04-25T10:06:51.823889+0000","last_clean":"2024-04-25T10:06:51.823889+0000","last_became_active":"2024-04-25T10:06:50.821425+0000","last_became_peered":"2024-04-25T10:06:50.821425+0000","last_unstale":"2024-04-25T10:06:51.823889+0000","last_undegraded":"2024-04-25T10:06:51.823889+0000","last_fullsized":"2024-04-25T10:06:51.823889+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:41.915878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845619+0000","last_change":"2024-04-25T10:06:51.843538+0000","last_active":"2024-04-25T10:06:51.845619+0000","last_peered":"2024-04-25T10:06:51.845619+0000","last_clean":"2024-04-25T10:06:51.845619+0000","last_became_active":"2024-04-25T10:06:51.842717+0000","last_became_peered":"2024-04-25T10:06:51.842717+0000","last_unstale":"2024-04-25T10:06:51.845619+0000","last_undegraded":"2024-04-25T10:06:51.845619+0000","last_fullsized":"2024-04-25T10:06:51.845619+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:32.311454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.831838+0000","last_change":"2024-04-25T10:06:50.822997+0000","last_active":"2024-04-25T10:06:51.831838+0000","last_peered":"2024-04-25T10:06:51.831838+0000","last_clean":"2024-04-25T10:06:51.831838+0000","last_became_active":"2024-04-25T10:06:50.822554+0000","last_became_peered":"2024-04-25T10:06:50.822554+0000","last_unstale":"2024-04-25T10:06:51.831838+0000","last_undegraded":"2024-04-25T10:06:51.831838+0000","last_fullsized":"2024-04-25T10:06:51.831838+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:00.259780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834087+0000","last_change":"2024-04-25T10:06:51.831355+0000","last_active":"2024-04-25T10:06:51.834087+0000","last_peered":"2024-04-25T10:06:51.834087+0000","last_clean":"2024-04-25T10:06:51.834087+0000","last_became_active":"2024-04-25T10:06:51.831047+0000","last_became_peered":"2024-04-25T10:06:51.831047+0000","last_unstale":"2024-04-25T10:06:51.834087+0000","last_undegraded":"2024-04-25T10:06:51.834087+0000","last_fullsized":"2024-04-25T10:06:51.834087+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:29.285725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.832092+0000","last_change":"2024-04-25T10:06:50.823705+0000","last_active":"2024-04-25T10:06:51.832092+0000","last_peered":"2024-04-25T10:06:51.832092+0000","last_clean":"2024-04-25T10:06:51.832092+0000","last_became_active":"2024-04-25T10:06:50.823452+0000","last_became_peered":"2024-04-25T10:06:50.823452+0000","last_unstale":"2024-04-25T10:06:51.832092+0000","last_undegraded":"2024-04-25T10:06:51.832092+0000","last_fullsized":"2024-04-25T10:06:51.832092+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:50.599925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.849495+0000","last_change":"2024-04-25T10:06:50.823009+0000","last_active":"2024-04-25T10:06:51.849495+0000","last_peered":"2024-04-25T10:06:51.849495+0000","last_clean":"2024-04-25T10:06:51.849495+0000","last_became_active":"2024-04-25T10:06:50.822686+0000","last_became_peered":"2024-04-25T10:06:50.822686+0000","last_unstale":"2024-04-25T10:06:51.849495+0000","last_undegraded":"2024-04-25T10:06:51.849495+0000","last_fullsized":"2024-04-25T10:06:51.849495+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:53.108324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":218,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.822612+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:06:51.822612+0000","last_peered":"2024-04-25T10:06:51.822612+0000","last_clean":"2024-04-25T10:06:51.822612+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:06:51.822612+0000","last_undegraded":"2024-04-25T10:06:51.822612+0000","last_fullsized":"2024-04-25T10:06:51.822612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":100,"acting":100,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":60,"acting":60,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":536576,"data_stored":214001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":540672,"data_stored":214583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509467,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":207192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509467,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":544768,"data_stored":215579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":207192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1130496,"data_stored":804405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":540672,"data_stored":214639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1089536,"data_stored":785468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:54.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:54.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.963+0000 7f7ea4970640 1 Processor -- start 2024-04-25T10:06:54.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.963+0000 7f7ea4970640 1 -- start start 2024-04-25T10:06:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.963+0000 7f7ea4970640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7ea015dce0 0x7f7ea00d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.963+0000 7f7ea4970640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea00d35b0 0x7f7ea00b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.964+0000 7f7ea4970640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea015e0c0 con 0x7f7ea00d1f90 2024-04-25T10:06:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.964+0000 7f7ea4970640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea00b7b10 con 0x7f7ea015dce0 2024-04-25T10:06:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.964+0000 7f7ea4970640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea00d2510 con 0x7f7ea00d35b0 2024-04-25T10:06:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.964+0000 7f7e9dd9b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea00d35b0 0x7f7ea00b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.964+0000 7f7e9e59c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7ea015dce0 0x7f7ea00d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.964+0000 7f7e9dd9b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea00d35b0 0x7f7ea00b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:45524/0 (socket says 172.21.15.16:45524) 2024-04-25T10:06:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.964+0000 7f7e9dd9b640 1 -- 172.21.15.16:0/3588910508 learned_addr learned my addr 172.21.15.16:0/3588910508 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.964+0000 7f7e9e59c640 1 --2- 172.21.15.16:0/3588910508 >> v2:172.21.15.186:3300/0 conn(0x7f7ea015dce0 0x7f7ea00d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:59564/0 (socket says 172.21.15.16:59564) 2024-04-25T10:06:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.964+0000 7f7e9ed9d640 1 --1- 172.21.15.16:0/3588910508 >> v1:172.21.15.186:6789/0 conn(0x7f7ea00d1f90 0x7f7ea006bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:46230/0 (socket says 172.21.15.16:46230) 2024-04-25T10:06:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.964+0000 7f7e9e59c640 1 -- 172.21.15.16:0/3588910508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea00d35b0 msgr2=0x7f7ea00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.964+0000 7f7e9e59c640 1 --2- 172.21.15.16:0/3588910508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea00d35b0 0x7f7ea00b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.964+0000 7f7e9e59c640 1 -- 172.21.15.16:0/3588910508 >> v1:172.21.15.186:6789/0 conn(0x7f7ea00d1f90 legacy=0x7f7ea006bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.964+0000 7f7e9e59c640 1 -- 172.21.15.16:0/3588910508 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea00d27f0 con 0x7f7ea015dce0 2024-04-25T10:06:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.964+0000 7f7e9d59a640 1 -- 172.21.15.16:0/3588910508 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951804837 0 0) 0x7f7ea015e0c0 con 0x7f7ea00d1f90 2024-04-25T10:06:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.964+0000 7f7e9dd9b640 1 --2- 172.21.15.16:0/3588910508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea00d35b0 0x7f7ea00b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.965+0000 7f7e9e59c640 1 --2- 172.21.15.16:0/3588910508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ea015dce0 0x7f7ea00d1a50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7e980097a0 tx=0x7f7e9802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=471e33e1034b022 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.965+0000 7f7e9d59a640 1 -- 172.21.15.16:0/3588910508 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98037070 con 0x7f7ea015dce0 2024-04-25T10:06:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.965+0000 7f7e9d59a640 1 -- 172.21.15.16:0/3588910508 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9802ae50 con 0x7f7ea015dce0 2024-04-25T10:06:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.965+0000 7f7e9d59a640 1 -- 172.21.15.16:0/3588910508 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98032660 con 0x7f7ea015dce0 2024-04-25T10:06:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.966+0000 7f7ea4970640 1 -- 172.21.15.16:0/3588910508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ea015dce0 msgr2=0x7f7ea00d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.966+0000 7f7ea4970640 1 --2- 172.21.15.16:0/3588910508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ea015dce0 0x7f7ea00d1a50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7e980097a0 tx=0x7f7e9802a100 comp rx=0 tx=0).stop 2024-04-25T10:06:54.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.966+0000 7f7ea4970640 1 -- 172.21.15.16:0/3588910508 shutdown_connections 2024-04-25T10:06:54.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.966+0000 7f7ea4970640 1 --2- 172.21.15.16:0/3588910508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea00d35b0 0x7f7ea00b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.966+0000 7f7ea4970640 1 --2- 172.21.15.16:0/3588910508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ea015dce0 0x7f7ea00d1a50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.966+0000 7f7ea4970640 1 -- 172.21.15.16:0/3588910508 >> 172.21.15.16:0/3588910508 conn(0x7f7ea0072900 msgr2=0x7f7ea00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.966+0000 7f7ea4970640 1 -- 172.21.15.16:0/3588910508 shutdown_connections 2024-04-25T10:06:54.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.966+0000 7f7ea4970640 1 -- 172.21.15.16:0/3588910508 wait complete. 2024-04-25T10:06:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.967+0000 7f7ea4970640 1 Processor -- start 2024-04-25T10:06:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.967+0000 7f7ea4970640 1 -- start start 2024-04-25T10:06:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.968+0000 7f7ea4970640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea00d1f90 0x7f7ea01f77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.968+0000 7f7ea4970640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ea00d35b0 0x7f7ea01f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.968+0000 7f7ea4970640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea015dce0 0x7f7ea00c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.968+0000 7f7ea4970640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea00d3090 con 0x7f7ea00d1f90 2024-04-25T10:06:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.968+0000 7f7ea4970640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea00d2db0 con 0x7f7ea00d35b0 2024-04-25T10:06:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.968+0000 7f7ea4970640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea00b9820 con 0x7f7ea015dce0 2024-04-25T10:06:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.968+0000 7f7e9e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea00d1f90 0x7f7ea01f77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.968+0000 7f7e9e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea00d1f90 0x7f7ea01f77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50452/0 (socket says 172.21.15.16:50452) 2024-04-25T10:06:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.968+0000 7f7e9e59c640 1 -- 172.21.15.16:0/952409611 learned_addr learned my addr 172.21.15.16:0/952409611 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.968+0000 7f7e9dd9b640 1 --2- 172.21.15.16:0/952409611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ea00d35b0 0x7f7ea01f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.968+0000 7f7e9ed9d640 1 --2- 172.21.15.16:0/952409611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea015dce0 0x7f7ea00c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.968+0000 7f7e9e59c640 1 -- 172.21.15.16:0/952409611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea015dce0 msgr2=0x7f7ea00c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.968+0000 7f7e9e59c640 1 --2- 172.21.15.16:0/952409611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea015dce0 0x7f7ea00c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.968+0000 7f7e9e59c640 1 -- 172.21.15.16:0/952409611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ea00d35b0 msgr2=0x7f7ea01f37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.968+0000 7f7e9e59c640 1 --2- 172.21.15.16:0/952409611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ea00d35b0 0x7f7ea01f37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.968+0000 7f7e9e59c640 1 -- 172.21.15.16:0/952409611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea015e450 con 0x7f7ea00d1f90 2024-04-25T10:06:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.969+0000 7f7e9e59c640 1 --2- 172.21.15.16:0/952409611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea00d1f90 0x7f7ea01f77e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7e980097a0 tx=0x7f7e980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.969+0000 7f7e8effd640 1 -- 172.21.15.16:0/952409611 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98037040 con 0x7f7ea00d1f90 2024-04-25T10:06:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.969+0000 7f7e8effd640 1 -- 172.21.15.16:0/952409611 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e98004030 con 0x7f7ea00d1f90 2024-04-25T10:06:54.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.969+0000 7f7ea4970640 1 -- 172.21.15.16:0/952409611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea015e730 con 0x7f7ea00d1f90 2024-04-25T10:06:54.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.969+0000 7f7e8effd640 1 -- 172.21.15.16:0/952409611 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98004420 con 0x7f7ea00d1f90 2024-04-25T10:06:54.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.969+0000 7f7ea4970640 1 -- 172.21.15.16:0/952409611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea015ecf0 con 0x7f7ea00d1f90 2024-04-25T10:06:54.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.970+0000 7f7ea4970640 1 -- 172.21.15.16:0/952409611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea00749b0 con 0x7f7ea00d1f90 2024-04-25T10:06:54.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.970+0000 7f7e8effd640 1 -- 172.21.15.16:0/952409611 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e98003610 con 0x7f7ea00d1f90 2024-04-25T10:06:54.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.971+0000 7f7e8effd640 1 --2- 172.21.15.16:0/952409611 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7e8006b4d0 0x7f7e8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.971+0000 7f7e8effd640 1 -- 172.21.15.16:0/952409611 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e980aadd0 con 0x7f7ea00d1f90 2024-04-25T10:06:54.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.971+0000 7f7e9dd9b640 1 --2- 172.21.15.16:0/952409611 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7e8006b4d0 0x7f7e8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.972+0000 7f7e9dd9b640 1 --2- 172.21.15.16:0/952409611 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7e8006b4d0 0x7f7e8006d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7ea01f6d30 tx=0x7f7e94009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:54.975+0000 7f7e8effd640 1 -- 172.21.15.16:0/952409611 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7e98079610 con 0x7f7ea00d1f90 2024-04-25T10:06:55.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.150+0000 7f7ea4970640 1 -- 172.21.15.16:0/952409611 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7ea00d0200 con 0x7f7e8006b4d0 2024-04-25T10:06:55.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.156+0000 7f7e8effd640 1 -- 172.21.15.16:0/952409611 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249604 (secure 0 0 0) 0x7f7ea00d0200 con 0x7f7e8006b4d0 2024-04-25T10:06:55.164 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:55.166 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:06:55.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.159+0000 7f7ea4970640 1 -- 172.21.15.16:0/952409611 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7e8006b4d0 msgr2=0x7f7e8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.159+0000 7f7ea4970640 1 --2- 172.21.15.16:0/952409611 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7e8006b4d0 0x7f7e8006d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7ea01f6d30 tx=0x7f7e94009240 comp rx=0 tx=0).stop 2024-04-25T10:06:55.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.159+0000 7f7ea4970640 1 -- 172.21.15.16:0/952409611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea00d1f90 msgr2=0x7f7ea01f77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.159+0000 7f7ea4970640 1 --2- 172.21.15.16:0/952409611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea00d1f90 0x7f7ea01f77e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7e980097a0 tx=0x7f7e980046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.159+0000 7f7ea4970640 1 -- 172.21.15.16:0/952409611 shutdown_connections 2024-04-25T10:06:55.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.159+0000 7f7ea4970640 1 --2- 172.21.15.16:0/952409611 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7e8006b4d0 0x7f7e8006d990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.159+0000 7f7ea4970640 1 --2- 172.21.15.16:0/952409611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea015dce0 0x7f7ea00c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.159+0000 7f7ea4970640 1 --2- 172.21.15.16:0/952409611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ea00d35b0 0x7f7ea01f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.159+0000 7f7ea4970640 1 --2- 172.21.15.16:0/952409611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea00d1f90 0x7f7ea01f77e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.159+0000 7f7ea4970640 1 -- 172.21.15.16:0/952409611 >> 172.21.15.16:0/952409611 conn(0x7f7ea0072900 msgr2=0x7f7ea00b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.159+0000 7f7ea4970640 1 -- 172.21.15.16:0/952409611 shutdown_connections 2024-04-25T10:06:55.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.159+0000 7f7ea4970640 1 -- 172.21.15.16:0/952409611 wait complete. 2024-04-25T10:06:55.175 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-04-25T10:06:53.804369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":162,"acting":162,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":102,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242276,"kb_used_data":5296,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732444,"statfs":{"total":773094113280,"available":772846022656,"internally_reserved":0,"allocated":5423104,"data_stored":2863059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001193"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.840324+0000","last_change":"2024-04-25T10:06:51.837844+0000","last_active":"2024-04-25T10:06:51.840324+0000","last_peered":"2024-04-25T10:06:51.840324+0000","last_clean":"2024-04-25T10:06:51.840324+0000","last_became_active":"2024-04-25T10:06:51.837567+0000","last_became_peered":"2024-04-25T10:06:51.837567+0000","last_unstale":"2024-04-25T10:06:51.840324+0000","last_undegraded":"2024-04-25T10:06:51.840324+0000","last_fullsized":"2024-04-25T10:06:51.840324+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:03.830929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847252+0000","last_change":"2024-04-25T10:06:51.844389+0000","last_active":"2024-04-25T10:06:51.847252+0000","last_peered":"2024-04-25T10:06:51.847252+0000","last_clean":"2024-04-25T10:06:51.847252+0000","last_became_active":"2024-04-25T10:06:51.844082+0000","last_became_peered":"2024-04-25T10:06:51.844082+0000","last_unstale":"2024-04-25T10:06:51.847252+0000","last_undegraded":"2024-04-25T10:06:51.847252+0000","last_fullsized":"2024-04-25T10:06:51.847252+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:10.566374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847906+0000","last_change":"2024-04-25T10:06:51.845292+0000","last_active":"2024-04-25T10:06:51.847906+0000","last_peered":"2024-04-25T10:06:51.847906+0000","last_clean":"2024-04-25T10:06:51.847906+0000","last_became_active":"2024-04-25T10:06:51.844790+0000","last_became_peered":"2024-04-25T10:06:51.844790+0000","last_unstale":"2024-04-25T10:06:51.847906+0000","last_undegraded":"2024-04-25T10:06:51.847906+0000","last_fullsized":"2024-04-25T10:06:51.847906+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:51.599603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841435+0000","last_change":"2024-04-25T10:06:51.840200+0000","last_active":"2024-04-25T10:06:51.841435+0000","last_peered":"2024-04-25T10:06:51.841435+0000","last_clean":"2024-04-25T10:06:51.841435+0000","last_became_active":"2024-04-25T10:06:51.839897+0000","last_became_peered":"2024-04-25T10:06:51.839897+0000","last_unstale":"2024-04-25T10:06:51.841435+0000","last_undegraded":"2024-04-25T10:06:51.841435+0000","last_fullsized":"2024-04-25T10:06:51.841435+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:23.657828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842607+0000","last_change":"2024-04-25T10:06:51.836510+0000","last_active":"2024-04-25T10:06:51.842607+0000","last_peered":"2024-04-25T10:06:51.842607+0000","last_clean":"2024-04-25T10:06:51.842607+0000","last_became_active":"2024-04-25T10:06:51.836244+0000","last_became_peered":"2024-04-25T10:06:51.836244+0000","last_unstale":"2024-04-25T10:06:51.842607+0000","last_undegraded":"2024-04-25T10:06:51.842607+0000","last_fullsized":"2024-04-25T10:06:51.842607+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:20.690035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841155+0000","last_change":"2024-04-25T10:06:51.837947+0000","last_active":"2024-04-25T10:06:51.841155+0000","last_peered":"2024-04-25T10:06:51.841155+0000","last_clean":"2024-04-25T10:06:51.841155+0000","last_became_active":"2024-04-25T10:06:51.837635+0000","last_became_peered":"2024-04-25T10:06:51.837635+0000","last_unstale":"2024-04-25T10:06:51.841155+0000","last_undegraded":"2024-04-25T10:06:51.841155+0000","last_fullsized":"2024-04-25T10:06:51.841155+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:47.682171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854141+0000","last_change":"2024-04-25T10:06:51.847625+0000","last_active":"2024-04-25T10:06:51.854141+0000","last_peered":"2024-04-25T10:06:51.854141+0000","last_clean":"2024-04-25T10:06:51.854141+0000","last_became_active":"2024-04-25T10:06:51.847179+0000","last_became_peered":"2024-04-25T10:06:51.847179+0000","last_unstale":"2024-04-25T10:06:51.854141+0000","last_undegraded":"2024-04-25T10:06:51.854141+0000","last_fullsized":"2024-04-25T10:06:51.854141+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:40.042571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841895+0000","last_change":"2024-04-25T10:06:51.837753+0000","last_active":"2024-04-25T10:06:51.841895+0000","last_peered":"2024-04-25T10:06:51.841895+0000","last_clean":"2024-04-25T10:06:51.841895+0000","last_became_active":"2024-04-25T10:06:51.837418+0000","last_became_peered":"2024-04-25T10:06:51.837418+0000","last_unstale":"2024-04-25T10:06:51.841895+0000","last_undegraded":"2024-04-25T10:06:51.841895+0000","last_fullsized":"2024-04-25T10:06:51.841895+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:29.774279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845933+0000","last_change":"2024-04-25T10:06:51.843573+0000","last_active":"2024-04-25T10:06:51.845933+0000","last_peered":"2024-04-25T10:06:51.845933+0000","last_clean":"2024-04-25T10:06:51.845933+0000","last_became_active":"2024-04-25T10:06:51.843141+0000","last_became_peered":"2024-04-25T10:06:51.843141+0000","last_unstale":"2024-04-25T10:06:51.845933+0000","last_undegraded":"2024-04-25T10:06:51.845933+0000","last_fullsized":"2024-04-25T10:06:51.845933+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:14.554682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842933+0000","last_change":"2024-04-25T10:06:51.838802+0000","last_active":"2024-04-25T10:06:51.842933+0000","last_peered":"2024-04-25T10:06:51.842933+0000","last_clean":"2024-04-25T10:06:51.842933+0000","last_became_active":"2024-04-25T10:06:51.837871+0000","last_became_peered":"2024-04-25T10:06:51.837871+0000","last_unstale":"2024-04-25T10:06:51.842933+0000","last_undegraded":"2024-04-25T10:06:51.842933+0000","last_fullsized":"2024-04-25T10:06:51.842933+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:50.203747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.844998+0000","last_change":"2024-04-25T10:06:51.842491+0000","last_active":"2024-04-25T10:06:51.844998+0000","last_peered":"2024-04-25T10:06:51.844998+0000","last_clean":"2024-04-25T10:06:51.844998+0000","last_became_active":"2024-04-25T10:06:51.842212+0000","last_became_peered":"2024-04-25T10:06:51.842212+0000","last_unstale":"2024-04-25T10:06:51.844998+0000","last_undegraded":"2024-04-25T10:06:51.844998+0000","last_fullsized":"2024-04-25T10:06:51.844998+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:04.628909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.838191+0000","last_change":"2024-04-25T10:06:51.836207+0000","last_active":"2024-04-25T10:06:51.838191+0000","last_peered":"2024-04-25T10:06:51.838191+0000","last_clean":"2024-04-25T10:06:51.838191+0000","last_became_active":"2024-04-25T10:06:51.835856+0000","last_became_peered":"2024-04-25T10:06:51.835856+0000","last_unstale":"2024-04-25T10:06:51.838191+0000","last_undegraded":"2024-04-25T10:06:51.838191+0000","last_fullsized":"2024-04-25T10:06:51.838191+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:20.690035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.839761+0000","last_change":"2024-04-25T10:06:51.835570+0000","last_active":"2024-04-25T10:06:51.839761+0000","last_peered":"2024-04-25T10:06:51.839761+0000","last_clean":"2024-04-25T10:06:51.839761+0000","last_became_active":"2024-04-25T10:06:51.835287+0000","last_became_peered":"2024-04-25T10:06:51.835287+0000","last_unstale":"2024-04-25T10:06:51.839761+0000","last_undegraded":"2024-04-25T10:06:51.839761+0000","last_fullsized":"2024-04-25T10:06:51.839761+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:58.880955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846553+0000","last_change":"2024-04-25T10:06:51.845653+0000","last_active":"2024-04-25T10:06:51.846553+0000","last_peered":"2024-04-25T10:06:51.846553+0000","last_clean":"2024-04-25T10:06:51.846553+0000","last_became_active":"2024-04-25T10:06:51.845387+0000","last_became_peered":"2024-04-25T10:06:51.845387+0000","last_unstale":"2024-04-25T10:06:51.846553+0000","last_undegraded":"2024-04-25T10:06:51.846553+0000","last_fullsized":"2024-04-25T10:06:51.846553+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:22.306267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841737+0000","last_change":"2024-04-25T10:06:51.836879+0000","last_active":"2024-04-25T10:06:51.841737+0000","last_peered":"2024-04-25T10:06:51.841737+0000","last_clean":"2024-04-25T10:06:51.841737+0000","last_became_active":"2024-04-25T10:06:51.835980+0000","last_became_peered":"2024-04-25T10:06:51.835980+0000","last_unstale":"2024-04-25T10:06:51.841737+0000","last_undegraded":"2024-04-25T10:06:51.841737+0000","last_fullsized":"2024-04-25T10:06:51.841737+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:30.749971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841910+0000","last_change":"2024-04-25T10:06:51.838280+0000","last_active":"2024-04-25T10:06:51.841910+0000","last_peered":"2024-04-25T10:06:51.841910+0000","last_clean":"2024-04-25T10:06:51.841910+0000","last_became_active":"2024-04-25T10:06:51.837813+0000","last_became_peered":"2024-04-25T10:06:51.837813+0000","last_unstale":"2024-04-25T10:06:51.841910+0000","last_undegraded":"2024-04-25T10:06:51.841910+0000","last_fullsized":"2024-04-25T10:06:51.841910+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:58.880955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.833222+0000","last_change":"2024-04-25T10:06:51.831343+0000","last_active":"2024-04-25T10:06:51.833222+0000","last_peered":"2024-04-25T10:06:51.833222+0000","last_clean":"2024-04-25T10:06:51.833222+0000","last_became_active":"2024-04-25T10:06:51.831040+0000","last_became_peered":"2024-04-25T10:06:51.831040+0000","last_unstale":"2024-04-25T10:06:51.833222+0000","last_undegraded":"2024-04-25T10:06:51.833222+0000","last_fullsized":"2024-04-25T10:06:51.833222+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:04.569426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.832424+0000","last_change":"2024-04-25T10:06:51.827202+0000","last_active":"2024-04-25T10:06:51.832424+0000","last_peered":"2024-04-25T10:06:51.832424+0000","last_clean":"2024-04-25T10:06:51.832424+0000","last_became_active":"2024-04-25T10:06:51.826893+0000","last_became_peered":"2024-04-25T10:06:51.826893+0000","last_unstale":"2024-04-25T10:06:51.832424+0000","last_undegraded":"2024-04-25T10:06:51.832424+0000","last_fullsized":"2024-04-25T10:06:51.832424+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:25.546409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.836212+0000","last_change":"2024-04-25T10:06:51.835353+0000","last_active":"2024-04-25T10:06:51.836212+0000","last_peered":"2024-04-25T10:06:51.836212+0000","last_clean":"2024-04-25T10:06:51.836212+0000","last_became_active":"2024-04-25T10:06:51.834941+0000","last_became_peered":"2024-04-25T10:06:51.834941+0000","last_unstale":"2024-04-25T10:06:51.836212+0000","last_undegraded":"2024-04-25T10:06:51.836212+0000","last_fullsized":"2024-04-25T10:06:51.836212+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:48.121882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.833241+0000","last_change":"2024-04-25T10:06:51.830316+0000","last_active":"2024-04-25T10:06:51.833241+0000","last_peered":"2024-04-25T10:06:51.833241+0000","last_clean":"2024-04-25T10:06:51.833241+0000","last_became_active":"2024-04-25T10:06:51.830039+0000","last_became_peered":"2024-04-25T10:06:51.830039+0000","last_unstale":"2024-04-25T10:06:51.833241+0000","last_undegraded":"2024-04-25T10:06:51.833241+0000","last_fullsized":"2024-04-25T10:06:51.833241+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:01.498621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.833261+0000","last_change":"2024-04-25T10:06:51.830470+0000","last_active":"2024-04-25T10:06:51.833261+0000","last_peered":"2024-04-25T10:06:51.833261+0000","last_clean":"2024-04-25T10:06:51.833261+0000","last_became_active":"2024-04-25T10:06:51.830180+0000","last_became_peered":"2024-04-25T10:06:51.830180+0000","last_unstale":"2024-04-25T10:06:51.833261+0000","last_undegraded":"2024-04-25T10:06:51.833261+0000","last_fullsized":"2024-04-25T10:06:51.833261+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:17.127193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.840946+0000","last_change":"2024-04-25T10:06:51.839558+0000","last_active":"2024-04-25T10:06:51.840946+0000","last_peered":"2024-04-25T10:06:51.840946+0000","last_clean":"2024-04-25T10:06:51.840946+0000","last_became_active":"2024-04-25T10:06:51.839271+0000","last_became_peered":"2024-04-25T10:06:51.839271+0000","last_unstale":"2024-04-25T10:06:51.840946+0000","last_undegraded":"2024-04-25T10:06:51.840946+0000","last_fullsized":"2024-04-25T10:06:51.840946+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:11.162077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.832516+0000","last_change":"2024-04-25T10:06:51.828040+0000","last_active":"2024-04-25T10:06:51.832516+0000","last_peered":"2024-04-25T10:06:51.832516+0000","last_clean":"2024-04-25T10:06:51.832516+0000","last_became_active":"2024-04-25T10:06:51.826090+0000","last_became_peered":"2024-04-25T10:06:51.826090+0000","last_unstale":"2024-04-25T10:06:51.832516+0000","last_undegraded":"2024-04-25T10:06:51.832516+0000","last_fullsized":"2024-04-25T10:06:51.832516+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:09.382894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842444+0000","last_change":"2024-04-25T10:06:51.837883+0000","last_active":"2024-04-25T10:06:51.842444+0000","last_peered":"2024-04-25T10:06:51.842444+0000","last_clean":"2024-04-25T10:06:51.842444+0000","last_became_active":"2024-04-25T10:06:51.837631+0000","last_became_peered":"2024-04-25T10:06:51.837631+0000","last_unstale":"2024-04-25T10:06:51.842444+0000","last_undegraded":"2024-04-25T10:06:51.842444+0000","last_fullsized":"2024-04-25T10:06:51.842444+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:14.554682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846377+0000","last_change":"2024-04-25T10:06:51.842565+0000","last_active":"2024-04-25T10:06:51.846377+0000","last_peered":"2024-04-25T10:06:51.846377+0000","last_clean":"2024-04-25T10:06:51.846377+0000","last_became_active":"2024-04-25T10:06:51.842251+0000","last_became_peered":"2024-04-25T10:06:51.842251+0000","last_unstale":"2024-04-25T10:06:51.846377+0000","last_undegraded":"2024-04-25T10:06:51.846377+0000","last_fullsized":"2024-04-25T10:06:51.846377+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:32.311454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854076+0000","last_change":"2024-04-25T10:06:50.815053+0000","last_active":"2024-04-25T10:06:51.854076+0000","last_peered":"2024-04-25T10:06:51.854076+0000","last_clean":"2024-04-25T10:06:51.854076+0000","last_became_active":"2024-04-25T10:06:50.814623+0000","last_became_peered":"2024-04-25T10:06:50.814623+0000","last_unstale":"2024-04-25T10:06:51.854076+0000","last_undegraded":"2024-04-25T10:06:51.854076+0000","last_fullsized":"2024-04-25T10:06:51.854076+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:16.218950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846459+0000","last_change":"2024-04-25T10:06:51.844956+0000","last_active":"2024-04-25T10:06:51.846459+0000","last_peered":"2024-04-25T10:06:51.846459+0000","last_clean":"2024-04-25T10:06:51.846459+0000","last_became_active":"2024-04-25T10:06:51.844611+0000","last_became_peered":"2024-04-25T10:06:51.844611+0000","last_unstale":"2024-04-25T10:06:51.846459+0000","last_undegraded":"2024-04-25T10:06:51.846459+0000","last_fullsized":"2024-04-25T10:06:51.846459+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:50.170873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.848558+0000","last_change":"2024-04-25T10:06:50.822263+0000","last_active":"2024-04-25T10:06:51.848558+0000","last_peered":"2024-04-25T10:06:51.848558+0000","last_clean":"2024-04-25T10:06:51.848558+0000","last_became_active":"2024-04-25T10:06:50.821985+0000","last_became_peered":"2024-04-25T10:06:50.821985+0000","last_unstale":"2024-04-25T10:06:51.848558+0000","last_undegraded":"2024-04-25T10:06:51.848558+0000","last_fullsized":"2024-04-25T10:06:51.848558+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:37.641598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847288+0000","last_change":"2024-04-25T10:06:51.844989+0000","last_active":"2024-04-25T10:06:51.847288+0000","last_peered":"2024-04-25T10:06:51.847288+0000","last_clean":"2024-04-25T10:06:51.847288+0000","last_became_active":"2024-04-25T10:06:51.844614+0000","last_became_peered":"2024-04-25T10:06:51.844614+0000","last_unstale":"2024-04-25T10:06:51.847288+0000","last_undegraded":"2024-04-25T10:06:51.847288+0000","last_fullsized":"2024-04-25T10:06:51.847288+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:50.203747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.853873+0000","last_change":"2024-04-25T10:06:50.818405+0000","last_active":"2024-04-25T10:06:51.853873+0000","last_peered":"2024-04-25T10:06:51.853873+0000","last_clean":"2024-04-25T10:06:51.853873+0000","last_became_active":"2024-04-25T10:06:50.818118+0000","last_became_peered":"2024-04-25T10:06:50.818118+0000","last_unstale":"2024-04-25T10:06:51.853873+0000","last_undegraded":"2024-04-25T10:06:51.853873+0000","last_fullsized":"2024-04-25T10:06:51.853873+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:08.381421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.848517+0000","last_change":"2024-04-25T10:06:50.821564+0000","last_active":"2024-04-25T10:06:51.848517+0000","last_peered":"2024-04-25T10:06:51.848517+0000","last_clean":"2024-04-25T10:06:51.848517+0000","last_became_active":"2024-04-25T10:06:50.821257+0000","last_became_peered":"2024-04-25T10:06:50.821257+0000","last_unstale":"2024-04-25T10:06:51.848517+0000","last_undegraded":"2024-04-25T10:06:51.848517+0000","last_fullsized":"2024-04-25T10:06:51.848517+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:25.919994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834314+0000","last_change":"2024-04-25T10:06:51.832079+0000","last_active":"2024-04-25T10:06:51.834314+0000","last_peered":"2024-04-25T10:06:51.834314+0000","last_clean":"2024-04-25T10:06:51.834314+0000","last_became_active":"2024-04-25T10:06:51.831785+0000","last_became_peered":"2024-04-25T10:06:51.831785+0000","last_unstale":"2024-04-25T10:06:51.834314+0000","last_undegraded":"2024-04-25T10:06:51.834314+0000","last_fullsized":"2024-04-25T10:06:51.834314+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:27.917421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.853752+0000","last_change":"2024-04-25T10:06:50.819361+0000","last_active":"2024-04-25T10:06:51.853752+0000","last_peered":"2024-04-25T10:06:51.853752+0000","last_clean":"2024-04-25T10:06:51.853752+0000","last_became_active":"2024-04-25T10:06:50.819093+0000","last_became_peered":"2024-04-25T10:06:50.819093+0000","last_unstale":"2024-04-25T10:06:51.853752+0000","last_undegraded":"2024-04-25T10:06:51.853752+0000","last_fullsized":"2024-04-25T10:06:51.853752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:24.544937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841867+0000","last_change":"2024-04-25T10:06:51.835101+0000","last_active":"2024-04-25T10:06:51.841867+0000","last_peered":"2024-04-25T10:06:51.841867+0000","last_clean":"2024-04-25T10:06:51.841867+0000","last_became_active":"2024-04-25T10:06:51.834291+0000","last_became_peered":"2024-04-25T10:06:51.834291+0000","last_unstale":"2024-04-25T10:06:51.841867+0000","last_undegraded":"2024-04-25T10:06:51.841867+0000","last_fullsized":"2024-04-25T10:06:51.841867+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:22.306267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845318+0000","last_change":"2024-04-25T10:06:50.822762+0000","last_active":"2024-04-25T10:06:51.845318+0000","last_peered":"2024-04-25T10:06:51.845318+0000","last_clean":"2024-04-25T10:06:51.845318+0000","last_became_active":"2024-04-25T10:06:50.822463+0000","last_became_peered":"2024-04-25T10:06:50.822463+0000","last_unstale":"2024-04-25T10:06:51.845318+0000","last_undegraded":"2024-04-25T10:06:51.845318+0000","last_fullsized":"2024-04-25T10:06:51.845318+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:18.363517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841750+0000","last_change":"2024-04-25T10:06:51.833997+0000","last_active":"2024-04-25T10:06:51.841750+0000","last_peered":"2024-04-25T10:06:51.841750+0000","last_clean":"2024-04-25T10:06:51.841750+0000","last_became_active":"2024-04-25T10:06:51.833694+0000","last_became_peered":"2024-04-25T10:06:51.833694+0000","last_unstale":"2024-04-25T10:06:51.841750+0000","last_undegraded":"2024-04-25T10:06:51.841750+0000","last_fullsized":"2024-04-25T10:06:51.841750+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:50.170873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828476+0000","last_change":"2024-04-25T10:06:50.818945+0000","last_active":"2024-04-25T10:06:51.828476+0000","last_peered":"2024-04-25T10:06:51.828476+0000","last_clean":"2024-04-25T10:06:51.828476+0000","last_became_active":"2024-04-25T10:06:50.818631+0000","last_became_peered":"2024-04-25T10:06:50.818631+0000","last_unstale":"2024-04-25T10:06:51.828476+0000","last_undegraded":"2024-04-25T10:06:51.828476+0000","last_fullsized":"2024-04-25T10:06:51.828476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:49.202275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.839815+0000","last_change":"2024-04-25T10:06:51.837313+0000","last_active":"2024-04-25T10:06:51.839815+0000","last_peered":"2024-04-25T10:06:51.839815+0000","last_clean":"2024-04-25T10:06:51.839815+0000","last_became_active":"2024-04-25T10:06:51.837024+0000","last_became_peered":"2024-04-25T10:06:51.837024+0000","last_unstale":"2024-04-25T10:06:51.839815+0000","last_undegraded":"2024-04-25T10:06:51.839815+0000","last_fullsized":"2024-04-25T10:06:51.839815+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:28.322485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.825554+0000","last_change":"2024-04-25T10:06:50.818743+0000","last_active":"2024-04-25T10:06:51.825554+0000","last_peered":"2024-04-25T10:06:51.825554+0000","last_clean":"2024-04-25T10:06:51.825554+0000","last_became_active":"2024-04-25T10:06:50.818359+0000","last_became_peered":"2024-04-25T10:06:50.818359+0000","last_unstale":"2024-04-25T10:06:51.825554+0000","last_undegraded":"2024-04-25T10:06:51.825554+0000","last_fullsized":"2024-04-25T10:06:51.825554+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:47.120410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.848307+0000","last_change":"2024-04-25T10:06:51.846840+0000","last_active":"2024-04-25T10:06:51.848307+0000","last_peered":"2024-04-25T10:06:51.848307+0000","last_clean":"2024-04-25T10:06:51.848307+0000","last_became_active":"2024-04-25T10:06:51.846314+0000","last_became_peered":"2024-04-25T10:06:51.846314+0000","last_unstale":"2024-04-25T10:06:51.848307+0000","last_undegraded":"2024-04-25T10:06:51.848307+0000","last_fullsized":"2024-04-25T10:06:51.848307+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:29.810440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.823148+0000","last_change":"2024-04-25T10:06:50.819769+0000","last_active":"2024-04-25T10:06:51.823148+0000","last_peered":"2024-04-25T10:06:51.823148+0000","last_clean":"2024-04-25T10:06:51.823148+0000","last_became_active":"2024-04-25T10:06:50.819517+0000","last_became_peered":"2024-04-25T10:06:50.819517+0000","last_unstale":"2024-04-25T10:06:51.823148+0000","last_undegraded":"2024-04-25T10:06:51.823148+0000","last_fullsized":"2024-04-25T10:06:51.823148+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:15.288040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834870+0000","last_change":"2024-04-25T10:06:51.832027+0000","last_active":"2024-04-25T10:06:51.834870+0000","last_peered":"2024-04-25T10:06:51.834870+0000","last_clean":"2024-04-25T10:06:51.834870+0000","last_became_active":"2024-04-25T10:06:51.831718+0000","last_became_peered":"2024-04-25T10:06:51.831718+0000","last_unstale":"2024-04-25T10:06:51.834870+0000","last_undegraded":"2024-04-25T10:06:51.834870+0000","last_fullsized":"2024-04-25T10:06:51.834870+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:30.250121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.825643+0000","last_change":"2024-04-25T10:06:50.825390+0000","last_active":"2024-04-25T10:06:51.825643+0000","last_peered":"2024-04-25T10:06:51.825643+0000","last_clean":"2024-04-25T10:06:51.825643+0000","last_became_active":"2024-04-25T10:06:50.825085+0000","last_became_peered":"2024-04-25T10:06:50.825085+0000","last_unstale":"2024-04-25T10:06:51.825643+0000","last_undegraded":"2024-04-25T10:06:51.825643+0000","last_fullsized":"2024-04-25T10:06:51.825643+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:10.160604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845381+0000","last_change":"2024-04-25T10:06:50.818510+0000","last_active":"2024-04-25T10:06:51.845381+0000","last_peered":"2024-04-25T10:06:51.845381+0000","last_clean":"2024-04-25T10:06:51.845381+0000","last_became_active":"2024-04-25T10:06:50.818217+0000","last_became_peered":"2024-04-25T10:06:50.818217+0000","last_unstale":"2024-04-25T10:06:51.845381+0000","last_undegraded":"2024-04-25T10:06:51.845381+0000","last_fullsized":"2024-04-25T10:06:51.845381+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.521754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841431+0000","last_change":"2024-04-25T10:06:51.838484+0000","last_active":"2024-04-25T10:06:51.841431+0000","last_peered":"2024-04-25T10:06:51.841431+0000","last_clean":"2024-04-25T10:06:51.841431+0000","last_became_active":"2024-04-25T10:06:51.838211+0000","last_became_peered":"2024-04-25T10:06:51.838211+0000","last_unstale":"2024-04-25T10:06:51.841431+0000","last_undegraded":"2024-04-25T10:06:51.841431+0000","last_fullsized":"2024-04-25T10:06:51.841431+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:46.417710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.856308+0000","last_change":"2024-04-25T10:06:50.823945+0000","last_active":"2024-04-25T10:06:51.856308+0000","last_peered":"2024-04-25T10:06:51.856308+0000","last_clean":"2024-04-25T10:06:51.856308+0000","last_became_active":"2024-04-25T10:06:50.823670+0000","last_became_peered":"2024-04-25T10:06:50.823670+0000","last_unstale":"2024-04-25T10:06:51.856308+0000","last_undegraded":"2024-04-25T10:06:51.856308+0000","last_fullsized":"2024-04-25T10:06:51.856308+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:03.627437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845672+0000","last_change":"2024-04-25T10:06:51.843182+0000","last_active":"2024-04-25T10:06:51.845672+0000","last_peered":"2024-04-25T10:06:51.845672+0000","last_clean":"2024-04-25T10:06:51.845672+0000","last_became_active":"2024-04-25T10:06:51.842811+0000","last_became_peered":"2024-04-25T10:06:51.842811+0000","last_unstale":"2024-04-25T10:06:51.845672+0000","last_undegraded":"2024-04-25T10:06:51.845672+0000","last_fullsized":"2024-04-25T10:06:51.845672+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:43.266424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.850017+0000","last_change":"2024-04-25T10:06:50.823653+0000","last_active":"2024-04-25T10:06:51.850017+0000","last_peered":"2024-04-25T10:06:51.850017+0000","last_clean":"2024-04-25T10:06:51.850017+0000","last_became_active":"2024-04-25T10:06:50.823373+0000","last_became_peered":"2024-04-25T10:06:50.823373+0000","last_unstale":"2024-04-25T10:06:51.850017+0000","last_undegraded":"2024-04-25T10:06:51.850017+0000","last_fullsized":"2024-04-25T10:06:51.850017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:19.313862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842243+0000","last_change":"2024-04-25T10:06:51.836404+0000","last_active":"2024-04-25T10:06:51.842243+0000","last_peered":"2024-04-25T10:06:51.842243+0000","last_clean":"2024-04-25T10:06:51.842243+0000","last_became_active":"2024-04-25T10:06:51.835875+0000","last_became_peered":"2024-04-25T10:06:51.835875+0000","last_unstale":"2024-04-25T10:06:51.842243+0000","last_undegraded":"2024-04-25T10:06:51.842243+0000","last_fullsized":"2024-04-25T10:06:51.842243+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:30.250121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828780+0000","last_change":"2024-04-25T10:06:50.824101+0000","last_active":"2024-04-25T10:06:51.828780+0000","last_peered":"2024-04-25T10:06:51.828780+0000","last_clean":"2024-04-25T10:06:51.828780+0000","last_became_active":"2024-04-25T10:06:50.823800+0000","last_became_peered":"2024-04-25T10:06:50.823800+0000","last_unstale":"2024-04-25T10:06:51.828780+0000","last_undegraded":"2024-04-25T10:06:51.828780+0000","last_fullsized":"2024-04-25T10:06:51.828780+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:10.364676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846106+0000","last_change":"2024-04-25T10:06:51.842607+0000","last_active":"2024-04-25T10:06:51.846106+0000","last_peered":"2024-04-25T10:06:51.846106+0000","last_clean":"2024-04-25T10:06:51.846106+0000","last_became_active":"2024-04-25T10:06:51.842296+0000","last_became_peered":"2024-04-25T10:06:51.842296+0000","last_unstale":"2024-04-25T10:06:51.846106+0000","last_undegraded":"2024-04-25T10:06:51.846106+0000","last_fullsized":"2024-04-25T10:06:51.846106+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:26.337568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.831684+0000","last_change":"2024-04-25T10:06:50.826028+0000","last_active":"2024-04-25T10:06:51.831684+0000","last_peered":"2024-04-25T10:06:51.831684+0000","last_clean":"2024-04-25T10:06:51.831684+0000","last_became_active":"2024-04-25T10:06:50.825717+0000","last_became_peered":"2024-04-25T10:06:50.825717+0000","last_unstale":"2024-04-25T10:06:51.831684+0000","last_undegraded":"2024-04-25T10:06:51.831684+0000","last_fullsized":"2024-04-25T10:06:51.831684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:28.284253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834918+0000","last_change":"2024-04-25T10:06:51.833850+0000","last_active":"2024-04-25T10:06:51.834918+0000","last_peered":"2024-04-25T10:06:51.834918+0000","last_clean":"2024-04-25T10:06:51.834918+0000","last_became_active":"2024-04-25T10:06:51.833584+0000","last_became_peered":"2024-04-25T10:06:51.833584+0000","last_unstale":"2024-04-25T10:06:51.834918+0000","last_undegraded":"2024-04-25T10:06:51.834918+0000","last_fullsized":"2024-04-25T10:06:51.834918+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:01.261252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854164+0000","last_change":"2024-04-25T10:06:50.820905+0000","last_active":"2024-04-25T10:06:51.854164+0000","last_peered":"2024-04-25T10:06:51.854164+0000","last_clean":"2024-04-25T10:06:51.854164+0000","last_became_active":"2024-04-25T10:06:50.820642+0000","last_became_peered":"2024-04-25T10:06:50.820642+0000","last_unstale":"2024-04-25T10:06:51.854164+0000","last_undegraded":"2024-04-25T10:06:51.854164+0000","last_fullsized":"2024-04-25T10:06:51.854164+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:22.656356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846398+0000","last_change":"2024-04-25T10:06:51.842872+0000","last_active":"2024-04-25T10:06:51.846398+0000","last_peered":"2024-04-25T10:06:51.846398+0000","last_clean":"2024-04-25T10:06:51.846398+0000","last_became_active":"2024-04-25T10:06:51.842550+0000","last_became_peered":"2024-04-25T10:06:51.842550+0000","last_unstale":"2024-04-25T10:06:51.846398+0000","last_undegraded":"2024-04-25T10:06:51.846398+0000","last_fullsized":"2024-04-25T10:06:51.846398+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:51.599603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:49.800151+0000","last_change":"2024-04-25T10:06:49.800151+0000","last_active":"2024-04-25T10:06:49.800151+0000","last_peered":"2024-04-25T10:06:49.800151+0000","last_clean":"2024-04-25T10:06:49.800151+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:49.800151+0000","last_undegraded":"2024-04-25T10:06:49.800151+0000","last_fullsized":"2024-04-25T10:06:49.800151+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845649+0000","last_change":"2024-04-25T10:06:51.840224+0000","last_active":"2024-04-25T10:06:51.845649+0000","last_peered":"2024-04-25T10:06:51.845649+0000","last_clean":"2024-04-25T10:06:51.845649+0000","last_became_active":"2024-04-25T10:06:51.839917+0000","last_became_peered":"2024-04-25T10:06:51.839917+0000","last_unstale":"2024-04-25T10:06:51.845649+0000","last_undegraded":"2024-04-25T10:06:51.845649+0000","last_fullsized":"2024-04-25T10:06:51.845649+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:43.266424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.859198+0000","last_change":"2024-04-25T10:06:50.818607+0000","last_active":"2024-04-25T10:06:51.859198+0000","last_peered":"2024-04-25T10:06:51.859198+0000","last_clean":"2024-04-25T10:06:51.859198+0000","last_became_active":"2024-04-25T10:06:50.818315+0000","last_became_peered":"2024-04-25T10:06:50.818315+0000","last_unstale":"2024-04-25T10:06:51.859198+0000","last_undegraded":"2024-04-25T10:06:51.859198+0000","last_fullsized":"2024-04-25T10:06:51.859198+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:50.599925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.835690+0000","last_change":"2024-04-25T10:06:51.834754+0000","last_active":"2024-04-25T10:06:51.835690+0000","last_peered":"2024-04-25T10:06:51.835690+0000","last_clean":"2024-04-25T10:06:51.835690+0000","last_became_active":"2024-04-25T10:06:51.834425+0000","last_became_peered":"2024-04-25T10:06:51.834425+0000","last_unstale":"2024-04-25T10:06:51.835690+0000","last_undegraded":"2024-04-25T10:06:51.835690+0000","last_fullsized":"2024-04-25T10:06:51.835690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:17.220423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854430+0000","last_change":"2024-04-25T10:06:50.820485+0000","last_active":"2024-04-25T10:06:51.854430+0000","last_peered":"2024-04-25T10:06:51.854430+0000","last_clean":"2024-04-25T10:06:51.854430+0000","last_became_active":"2024-04-25T10:06:50.820215+0000","last_became_peered":"2024-04-25T10:06:50.820215+0000","last_unstale":"2024-04-25T10:06:51.854430+0000","last_undegraded":"2024-04-25T10:06:51.854430+0000","last_fullsized":"2024-04-25T10:06:51.854430+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:19.313862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847857+0000","last_change":"2024-04-25T10:06:51.845056+0000","last_active":"2024-04-25T10:06:51.847857+0000","last_peered":"2024-04-25T10:06:51.847857+0000","last_clean":"2024-04-25T10:06:51.847857+0000","last_became_active":"2024-04-25T10:06:51.844448+0000","last_became_peered":"2024-04-25T10:06:51.844448+0000","last_unstale":"2024-04-25T10:06:51.847857+0000","last_undegraded":"2024-04-25T10:06:51.847857+0000","last_fullsized":"2024-04-25T10:06:51.847857+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:26.337568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.831449+0000","last_change":"2024-04-25T10:06:50.825791+0000","last_active":"2024-04-25T10:06:51.831449+0000","last_peered":"2024-04-25T10:06:51.831449+0000","last_clean":"2024-04-25T10:06:51.831449+0000","last_became_active":"2024-04-25T10:06:50.825467+0000","last_became_peered":"2024-04-25T10:06:50.825467+0000","last_unstale":"2024-04-25T10:06:51.831449+0000","last_undegraded":"2024-04-25T10:06:51.831449+0000","last_fullsized":"2024-04-25T10:06:51.831449+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.521754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845737+0000","last_change":"2024-04-25T10:06:51.842252+0000","last_active":"2024-04-25T10:06:51.845737+0000","last_peered":"2024-04-25T10:06:51.845737+0000","last_clean":"2024-04-25T10:06:51.845737+0000","last_became_active":"2024-04-25T10:06:51.841999+0000","last_became_peered":"2024-04-25T10:06:51.841999+0000","last_unstale":"2024-04-25T10:06:51.845737+0000","last_undegraded":"2024-04-25T10:06:51.845737+0000","last_fullsized":"2024-04-25T10:06:51.845737+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:15.390698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.851706+0000","last_change":"2024-04-25T10:06:50.821347+0000","last_active":"2024-04-25T10:06:51.851706+0000","last_peered":"2024-04-25T10:06:51.851706+0000","last_clean":"2024-04-25T10:06:51.851706+0000","last_became_active":"2024-04-25T10:06:50.821073+0000","last_became_peered":"2024-04-25T10:06:50.821073+0000","last_unstale":"2024-04-25T10:06:51.851706+0000","last_undegraded":"2024-04-25T10:06:51.851706+0000","last_fullsized":"2024-04-25T10:06:51.851706+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:02.829457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842908+0000","last_change":"2024-04-25T10:06:51.838646+0000","last_active":"2024-04-25T10:06:51.842908+0000","last_peered":"2024-04-25T10:06:51.842908+0000","last_clean":"2024-04-25T10:06:51.842908+0000","last_became_active":"2024-04-25T10:06:51.835104+0000","last_became_peered":"2024-04-25T10:06:51.835104+0000","last_unstale":"2024-04-25T10:06:51.842908+0000","last_undegraded":"2024-04-25T10:06:51.842908+0000","last_fullsized":"2024-04-25T10:06:51.842908+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:10.566374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828795+0000","last_change":"2024-04-25T10:06:50.827168+0000","last_active":"2024-04-25T10:06:51.828795+0000","last_peered":"2024-04-25T10:06:51.828795+0000","last_clean":"2024-04-25T10:06:51.828795+0000","last_became_active":"2024-04-25T10:06:50.826210+0000","last_became_peered":"2024-04-25T10:06:50.826210+0000","last_unstale":"2024-04-25T10:06:51.828795+0000","last_undegraded":"2024-04-25T10:06:51.828795+0000","last_fullsized":"2024-04-25T10:06:51.828795+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.650773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.825362+0000","last_change":"2024-04-25T10:06:50.821404+0000","last_active":"2024-04-25T10:06:51.825362+0000","last_peered":"2024-04-25T10:06:51.825362+0000","last_clean":"2024-04-25T10:06:51.825362+0000","last_became_active":"2024-04-25T10:06:50.821129+0000","last_became_peered":"2024-04-25T10:06:50.821129+0000","last_unstale":"2024-04-25T10:06:51.825362+0000","last_undegraded":"2024-04-25T10:06:51.825362+0000","last_fullsized":"2024-04-25T10:06:51.825362+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.650773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.823231+0000","last_change":"2024-04-25T10:06:50.825554+0000","last_active":"2024-04-25T10:06:51.823231+0000","last_peered":"2024-04-25T10:06:51.823231+0000","last_clean":"2024-04-25T10:06:51.823231+0000","last_became_active":"2024-04-25T10:06:50.825226+0000","last_became_peered":"2024-04-25T10:06:50.825226+0000","last_unstale":"2024-04-25T10:06:51.823231+0000","last_undegraded":"2024-04-25T10:06:51.823231+0000","last_fullsized":"2024-04-25T10:06:51.823231+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:16.990316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834352+0000","last_change":"2024-04-25T10:06:51.832937+0000","last_active":"2024-04-25T10:06:51.834352+0000","last_peered":"2024-04-25T10:06:51.834352+0000","last_clean":"2024-04-25T10:06:51.834352+0000","last_became_active":"2024-04-25T10:06:51.832620+0000","last_became_peered":"2024-04-25T10:06:51.832620+0000","last_unstale":"2024-04-25T10:06:51.834352+0000","last_undegraded":"2024-04-25T10:06:51.834352+0000","last_fullsized":"2024-04-25T10:06:51.834352+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:47.015585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:49.800151+0000","last_change":"2024-04-25T10:06:49.800151+0000","last_active":"2024-04-25T10:06:49.800151+0000","last_peered":"2024-04-25T10:06:49.800151+0000","last_clean":"2024-04-25T10:06:49.800151+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:49.800151+0000","last_undegraded":"2024-04-25T10:06:49.800151+0000","last_fullsized":"2024-04-25T10:06:49.800151+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841715+0000","last_change":"2024-04-25T10:06:51.835802+0000","last_active":"2024-04-25T10:06:51.841715+0000","last_peered":"2024-04-25T10:06:51.841715+0000","last_clean":"2024-04-25T10:06:51.841715+0000","last_became_active":"2024-04-25T10:06:51.835196+0000","last_became_peered":"2024-04-25T10:06:51.835196+0000","last_unstale":"2024-04-25T10:06:51.841715+0000","last_undegraded":"2024-04-25T10:06:51.841715+0000","last_fullsized":"2024-04-25T10:06:51.841715+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:27.916081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828499+0000","last_change":"2024-04-25T10:06:50.826677+0000","last_active":"2024-04-25T10:06:51.828499+0000","last_peered":"2024-04-25T10:06:51.828499+0000","last_clean":"2024-04-25T10:06:51.828499+0000","last_became_active":"2024-04-25T10:06:50.826114+0000","last_became_peered":"2024-04-25T10:06:50.826114+0000","last_unstale":"2024-04-25T10:06:51.828499+0000","last_undegraded":"2024-04-25T10:06:51.828499+0000","last_fullsized":"2024-04-25T10:06:51.828499+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:39.041098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841453+0000","last_change":"2024-04-25T10:06:51.837949+0000","last_active":"2024-04-25T10:06:51.841453+0000","last_peered":"2024-04-25T10:06:51.841453+0000","last_clean":"2024-04-25T10:06:51.841453+0000","last_became_active":"2024-04-25T10:06:51.837681+0000","last_became_peered":"2024-04-25T10:06:51.837681+0000","last_unstale":"2024-04-25T10:06:51.841453+0000","last_undegraded":"2024-04-25T10:06:51.841453+0000","last_fullsized":"2024-04-25T10:06:51.841453+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:17.563413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828714+0000","last_change":"2024-04-25T10:06:50.826236+0000","last_active":"2024-04-25T10:06:51.828714+0000","last_peered":"2024-04-25T10:06:51.828714+0000","last_clean":"2024-04-25T10:06:51.828714+0000","last_became_active":"2024-04-25T10:06:50.825899+0000","last_became_peered":"2024-04-25T10:06:50.825899+0000","last_unstale":"2024-04-25T10:06:51.828714+0000","last_undegraded":"2024-04-25T10:06:51.828714+0000","last_fullsized":"2024-04-25T10:06:51.828714+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:15.288040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845716+0000","last_change":"2024-04-25T10:06:51.840773+0000","last_active":"2024-04-25T10:06:51.845716+0000","last_peered":"2024-04-25T10:06:51.845716+0000","last_clean":"2024-04-25T10:06:51.845716+0000","last_became_active":"2024-04-25T10:06:51.840412+0000","last_became_peered":"2024-04-25T10:06:51.840412+0000","last_unstale":"2024-04-25T10:06:51.845716+0000","last_undegraded":"2024-04-25T10:06:51.845716+0000","last_fullsized":"2024-04-25T10:06:51.845716+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:58.954708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.827378+0000","last_change":"2024-04-25T10:06:50.819732+0000","last_active":"2024-04-25T10:06:51.827378+0000","last_peered":"2024-04-25T10:06:51.827378+0000","last_clean":"2024-04-25T10:06:51.827378+0000","last_became_active":"2024-04-25T10:06:50.819448+0000","last_became_peered":"2024-04-25T10:06:50.819448+0000","last_unstale":"2024-04-25T10:06:51.827378+0000","last_undegraded":"2024-04-25T10:06:51.827378+0000","last_fullsized":"2024-04-25T10:06:51.827378+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:09.564901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.823889+0000","last_change":"2024-04-25T10:06:50.821944+0000","last_active":"2024-04-25T10:06:51.823889+0000","last_peered":"2024-04-25T10:06:51.823889+0000","last_clean":"2024-04-25T10:06:51.823889+0000","last_became_active":"2024-04-25T10:06:50.821425+0000","last_became_peered":"2024-04-25T10:06:50.821425+0000","last_unstale":"2024-04-25T10:06:51.823889+0000","last_undegraded":"2024-04-25T10:06:51.823889+0000","last_fullsized":"2024-04-25T10:06:51.823889+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:41.915878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845619+0000","last_change":"2024-04-25T10:06:51.843538+0000","last_active":"2024-04-25T10:06:51.845619+0000","last_peered":"2024-04-25T10:06:51.845619+0000","last_clean":"2024-04-25T10:06:51.845619+0000","last_became_active":"2024-04-25T10:06:51.842717+0000","last_became_peered":"2024-04-25T10:06:51.842717+0000","last_unstale":"2024-04-25T10:06:51.845619+0000","last_undegraded":"2024-04-25T10:06:51.845619+0000","last_fullsized":"2024-04-25T10:06:51.845619+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:32.311454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.831838+0000","last_change":"2024-04-25T10:06:50.822997+0000","last_active":"2024-04-25T10:06:51.831838+0000","last_peered":"2024-04-25T10:06:51.831838+0000","last_clean":"2024-04-25T10:06:51.831838+0000","last_became_active":"2024-04-25T10:06:50.822554+0000","last_became_peered":"2024-04-25T10:06:50.822554+0000","last_unstale":"2024-04-25T10:06:51.831838+0000","last_undegraded":"2024-04-25T10:06:51.831838+0000","last_fullsized":"2024-04-25T10:06:51.831838+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:00.259780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834087+0000","last_change":"2024-04-25T10:06:51.831355+0000","last_active":"2024-04-25T10:06:51.834087+0000","last_peered":"2024-04-25T10:06:51.834087+0000","last_clean":"2024-04-25T10:06:51.834087+0000","last_became_active":"2024-04-25T10:06:51.831047+0000","last_became_peered":"2024-04-25T10:06:51.831047+0000","last_unstale":"2024-04-25T10:06:51.834087+0000","last_undegraded":"2024-04-25T10:06:51.834087+0000","last_fullsized":"2024-04-25T10:06:51.834087+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:29.285725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.832092+0000","last_change":"2024-04-25T10:06:50.823705+0000","last_active":"2024-04-25T10:06:51.832092+0000","last_peered":"2024-04-25T10:06:51.832092+0000","last_clean":"2024-04-25T10:06:51.832092+0000","last_became_active":"2024-04-25T10:06:50.823452+0000","last_became_peered":"2024-04-25T10:06:50.823452+0000","last_unstale":"2024-04-25T10:06:51.832092+0000","last_undegraded":"2024-04-25T10:06:51.832092+0000","last_fullsized":"2024-04-25T10:06:51.832092+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:50.599925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.849495+0000","last_change":"2024-04-25T10:06:50.823009+0000","last_active":"2024-04-25T10:06:51.849495+0000","last_peered":"2024-04-25T10:06:51.849495+0000","last_clean":"2024-04-25T10:06:51.849495+0000","last_became_active":"2024-04-25T10:06:50.822686+0000","last_became_peered":"2024-04-25T10:06:50.822686+0000","last_unstale":"2024-04-25T10:06:51.849495+0000","last_undegraded":"2024-04-25T10:06:51.849495+0000","last_fullsized":"2024-04-25T10:06:51.849495+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:53.108324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":218,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.822612+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:06:51.822612+0000","last_peered":"2024-04-25T10:06:51.822612+0000","last_clean":"2024-04-25T10:06:51.822612+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:06:51.822612+0000","last_undegraded":"2024-04-25T10:06:51.822612+0000","last_fullsized":"2024-04-25T10:06:51.822612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":100,"acting":100,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":60,"acting":60,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":536576,"data_stored":214001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":540672,"data_stored":214583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509467,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":207192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509467,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":544768,"data_stored":215579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":207192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1130496,"data_stored":804405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":540672,"data_stored":214639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1089536,"data_stored":785468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:55.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.310+0000 7f1347f22640 1 Processor -- start 2024-04-25T10:06:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.311+0000 7f1347f22640 1 -- start start 2024-04-25T10:06:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.311+0000 7f1347f22640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f134015dc30 0x7f134006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.311+0000 7f1347f22640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f134006cc00 0x7f13400b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.311+0000 7f1347f22640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13400b1960 0x7f1340099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.311+0000 7f1347f22640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f134006c1f0 con 0x7f134006cc00 2024-04-25T10:06:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.311+0000 7f1347f22640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13400cfb40 con 0x7f134015dc30 2024-04-25T10:06:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.311+0000 7f1347f22640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13400b1db0 con 0x7f13400b1960 2024-04-25T10:06:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.311+0000 7f13454bd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f134006cc00 0x7f13400b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.312+0000 7f13454bd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f134006cc00 0x7f13400b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50458/0 (socket says 172.21.15.16:50458) 2024-04-25T10:06:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.312+0000 7f13454bd640 1 -- 172.21.15.16:0/2931961946 learned_addr learned my addr 172.21.15.16:0/2931961946 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.312+0000 7f1345cbe640 1 --2- 172.21.15.16:0/2931961946 >> v2:172.21.15.186:3300/0 conn(0x7f134015dc30 0x7f134006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.312+0000 7f13464bf640 1 --2- 172.21.15.16:0/2931961946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13400b1960 0x7f1340099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.312+0000 7f13454bd640 1 -- 172.21.15.16:0/2931961946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13400b1960 msgr2=0x7f1340099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.312+0000 7f13454bd640 1 --2- 172.21.15.16:0/2931961946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13400b1960 0x7f1340099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.312+0000 7f13454bd640 1 -- 172.21.15.16:0/2931961946 >> v2:172.21.15.186:3300/0 conn(0x7f134015dc30 msgr2=0x7f134006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.312+0000 7f13454bd640 1 --2- 172.21.15.16:0/2931961946 >> v2:172.21.15.186:3300/0 conn(0x7f134015dc30 0x7f134006c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.312+0000 7f13454bd640 1 -- 172.21.15.16:0/2931961946 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13400b2060 con 0x7f134006cc00 2024-04-25T10:06:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.312+0000 7f1345cbe640 1 --2- 172.21.15.16:0/2931961946 >> v2:172.21.15.186:3300/0 conn(0x7f134015dc30 0x7f134006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:55.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.313+0000 7f13454bd640 1 --2- 172.21.15.16:0/2931961946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f134006cc00 0x7f13400b1420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f134009ab70 tx=0x7f133002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3aa5273578bc9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.313+0000 7f1344cbc640 1 -- 172.21.15.16:0/2931961946 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1330037070 con 0x7f134006cc00 2024-04-25T10:06:55.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.313+0000 7f1344cbc640 1 -- 172.21.15.16:0/2931961946 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1330032440 con 0x7f134006cc00 2024-04-25T10:06:55.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.313+0000 7f1344cbc640 1 -- 172.21.15.16:0/2931961946 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1330032810 con 0x7f134006cc00 2024-04-25T10:06:55.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.313+0000 7f1347f22640 1 -- 172.21.15.16:0/2931961946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f134006cc00 msgr2=0x7f13400b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.313+0000 7f1347f22640 1 --2- 172.21.15.16:0/2931961946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f134006cc00 0x7f13400b1420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f134009ab70 tx=0x7f133002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.313+0000 7f1347f22640 1 -- 172.21.15.16:0/2931961946 shutdown_connections 2024-04-25T10:06:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.313+0000 7f1347f22640 1 --2- 172.21.15.16:0/2931961946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13400b1960 0x7f1340099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.313+0000 7f1347f22640 1 --2- 172.21.15.16:0/2931961946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f134006cc00 0x7f13400b1420 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.313+0000 7f1347f22640 1 --2- 172.21.15.16:0/2931961946 >> v2:172.21.15.186:3300/0 conn(0x7f134015dc30 0x7f134006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.313+0000 7f1347f22640 1 -- 172.21.15.16:0/2931961946 >> 172.21.15.16:0/2931961946 conn(0x7f13400d1900 msgr2=0x7f13400d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.313+0000 7f1347f22640 1 -- 172.21.15.16:0/2931961946 shutdown_connections 2024-04-25T10:06:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.313+0000 7f1347f22640 1 -- 172.21.15.16:0/2931961946 wait complete. 2024-04-25T10:06:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.315+0000 7f1347f22640 1 Processor -- start 2024-04-25T10:06:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.315+0000 7f1347f22640 1 -- start start 2024-04-25T10:06:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.315+0000 7f1347f22640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f134006cc00 0x7f1340154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.315+0000 7f1347f22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13400b1960 0x7f13400b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.315+0000 7f1347f22640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f134015dc30 0x7f13400b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.315+0000 7f1347f22640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13400cfde0 con 0x7f13400b1960 2024-04-25T10:06:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.315+0000 7f1347f22640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13400b2620 con 0x7f134015dc30 2024-04-25T10:06:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.315+0000 7f1347f22640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134015e010 con 0x7f134006cc00 2024-04-25T10:06:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.315+0000 7f13454bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13400b1960 0x7f13400b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.315+0000 7f13454bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13400b1960 0x7f13400b57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50470/0 (socket says 172.21.15.16:50470) 2024-04-25T10:06:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.315+0000 7f13454bd640 1 -- 172.21.15.16:0/3348459732 learned_addr learned my addr 172.21.15.16:0/3348459732 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.316+0000 7f13464bf640 1 --2- 172.21.15.16:0/3348459732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f134015dc30 0x7f13400b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.316+0000 7f1345cbe640 1 --2- 172.21.15.16:0/3348459732 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f134006cc00 0x7f1340154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.316+0000 7f13454bd640 1 -- 172.21.15.16:0/3348459732 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f134006cc00 msgr2=0x7f1340154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.316+0000 7f13454bd640 1 --2- 172.21.15.16:0/3348459732 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f134006cc00 0x7f1340154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.316+0000 7f13454bd640 1 -- 172.21.15.16:0/3348459732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f134015dc30 msgr2=0x7f13400b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.316+0000 7f13454bd640 1 --2- 172.21.15.16:0/3348459732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f134015dc30 0x7f13400b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.316+0000 7f13454bd640 1 -- 172.21.15.16:0/3348459732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13401553a0 con 0x7f13400b1960 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.316+0000 7f1345cbe640 1 --2- 172.21.15.16:0/3348459732 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f134006cc00 0x7f1340154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.316+0000 7f13464bf640 1 --2- 172.21.15.16:0/3348459732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f134015dc30 0x7f13400b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.316+0000 7f13454bd640 1 --2- 172.21.15.16:0/3348459732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13400b1960 0x7f13400b57f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f133002a220 tx=0x7f13300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.316+0000 7f13367fc640 1 -- 172.21.15.16:0/3348459732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1330041020 con 0x7f13400b1960 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.316+0000 7f13367fc640 1 -- 172.21.15.16:0/3348459732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13300306c0 con 0x7f13400b1960 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.316+0000 7f1347f22640 1 -- 172.21.15.16:0/3348459732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13401e7ae0 con 0x7f13400b1960 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.316+0000 7f13367fc640 1 -- 172.21.15.16:0/3348459732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1330037040 con 0x7f13400b1960 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.316+0000 7f1347f22640 1 -- 172.21.15.16:0/3348459732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13401e80a0 con 0x7f13400b1960 2024-04-25T10:06:55.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.317+0000 7f1347f22640 1 -- 172.21.15.16:0/3348459732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1340067850 con 0x7f13400b1960 2024-04-25T10:06:55.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.318+0000 7f13367fc640 1 -- 172.21.15.16:0/3348459732 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1330044020 con 0x7f13400b1960 2024-04-25T10:06:55.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.318+0000 7f13367fc640 1 --2- 172.21.15.16:0/3348459732 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f132406b4d0 0x7f132406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.318+0000 7f13367fc640 1 -- 172.21.15.16:0/3348459732 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13300aaa30 con 0x7f13400b1960 2024-04-25T10:06:55.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.318+0000 7f1345cbe640 1 --2- 172.21.15.16:0/3348459732 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f132406b4d0 0x7f132406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.319+0000 7f1345cbe640 1 --2- 172.21.15.16:0/3348459732 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f132406b4d0 0x7f132406d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f133c006fd0 tx=0x7f133c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.323+0000 7f13367fc640 1 -- 172.21.15.16:0/3348459732 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f133004b030 con 0x7f13400b1960 2024-04-25T10:06:55.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.499+0000 7f1347f22640 1 -- 172.21.15.16:0/3348459732 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f13400749b0 con 0x7f132406b4d0 2024-04-25T10:06:55.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.506+0000 7f13367fc640 1 -- 172.21.15.16:0/3348459732 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249604 (secure 0 0 0) 0x7f13400749b0 con 0x7f132406b4d0 2024-04-25T10:06:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:55.516 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:06:55.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.509+0000 7f1347f22640 1 -- 172.21.15.16:0/3348459732 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f132406b4d0 msgr2=0x7f132406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.509+0000 7f1347f22640 1 --2- 172.21.15.16:0/3348459732 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f132406b4d0 0x7f132406d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f133c006fd0 tx=0x7f133c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:55.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.509+0000 7f1347f22640 1 -- 172.21.15.16:0/3348459732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13400b1960 msgr2=0x7f13400b57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.509+0000 7f1347f22640 1 --2- 172.21.15.16:0/3348459732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13400b1960 0x7f13400b57f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f133002a220 tx=0x7f13300046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.509+0000 7f1347f22640 1 -- 172.21.15.16:0/3348459732 shutdown_connections 2024-04-25T10:06:55.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.509+0000 7f1347f22640 1 --2- 172.21.15.16:0/3348459732 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f132406b4d0 0x7f132406d990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.509+0000 7f1347f22640 1 --2- 172.21.15.16:0/3348459732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f134015dc30 0x7f13400b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.509+0000 7f1347f22640 1 --2- 172.21.15.16:0/3348459732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13400b1960 0x7f13400b57f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.509+0000 7f1347f22640 1 --2- 172.21.15.16:0/3348459732 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f134006cc00 0x7f1340154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.509+0000 7f1347f22640 1 -- 172.21.15.16:0/3348459732 >> 172.21.15.16:0/3348459732 conn(0x7f13400d1900 msgr2=0x7f13400d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.509+0000 7f1347f22640 1 -- 172.21.15.16:0/3348459732 shutdown_connections 2024-04-25T10:06:55.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.509+0000 7f1347f22640 1 -- 172.21.15.16:0/3348459732 wait complete. 2024-04-25T10:06:55.525 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-04-25T10:06:53.804369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":162,"acting":162,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":102,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242276,"kb_used_data":5296,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732444,"statfs":{"total":773094113280,"available":772846022656,"internally_reserved":0,"allocated":5423104,"data_stored":2863059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001193"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.840324+0000","last_change":"2024-04-25T10:06:51.837844+0000","last_active":"2024-04-25T10:06:51.840324+0000","last_peered":"2024-04-25T10:06:51.840324+0000","last_clean":"2024-04-25T10:06:51.840324+0000","last_became_active":"2024-04-25T10:06:51.837567+0000","last_became_peered":"2024-04-25T10:06:51.837567+0000","last_unstale":"2024-04-25T10:06:51.840324+0000","last_undegraded":"2024-04-25T10:06:51.840324+0000","last_fullsized":"2024-04-25T10:06:51.840324+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:03.830929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847252+0000","last_change":"2024-04-25T10:06:51.844389+0000","last_active":"2024-04-25T10:06:51.847252+0000","last_peered":"2024-04-25T10:06:51.847252+0000","last_clean":"2024-04-25T10:06:51.847252+0000","last_became_active":"2024-04-25T10:06:51.844082+0000","last_became_peered":"2024-04-25T10:06:51.844082+0000","last_unstale":"2024-04-25T10:06:51.847252+0000","last_undegraded":"2024-04-25T10:06:51.847252+0000","last_fullsized":"2024-04-25T10:06:51.847252+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:10.566374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847906+0000","last_change":"2024-04-25T10:06:51.845292+0000","last_active":"2024-04-25T10:06:51.847906+0000","last_peered":"2024-04-25T10:06:51.847906+0000","last_clean":"2024-04-25T10:06:51.847906+0000","last_became_active":"2024-04-25T10:06:51.844790+0000","last_became_peered":"2024-04-25T10:06:51.844790+0000","last_unstale":"2024-04-25T10:06:51.847906+0000","last_undegraded":"2024-04-25T10:06:51.847906+0000","last_fullsized":"2024-04-25T10:06:51.847906+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:51.599603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841435+0000","last_change":"2024-04-25T10:06:51.840200+0000","last_active":"2024-04-25T10:06:51.841435+0000","last_peered":"2024-04-25T10:06:51.841435+0000","last_clean":"2024-04-25T10:06:51.841435+0000","last_became_active":"2024-04-25T10:06:51.839897+0000","last_became_peered":"2024-04-25T10:06:51.839897+0000","last_unstale":"2024-04-25T10:06:51.841435+0000","last_undegraded":"2024-04-25T10:06:51.841435+0000","last_fullsized":"2024-04-25T10:06:51.841435+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:23.657828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842607+0000","last_change":"2024-04-25T10:06:51.836510+0000","last_active":"2024-04-25T10:06:51.842607+0000","last_peered":"2024-04-25T10:06:51.842607+0000","last_clean":"2024-04-25T10:06:51.842607+0000","last_became_active":"2024-04-25T10:06:51.836244+0000","last_became_peered":"2024-04-25T10:06:51.836244+0000","last_unstale":"2024-04-25T10:06:51.842607+0000","last_undegraded":"2024-04-25T10:06:51.842607+0000","last_fullsized":"2024-04-25T10:06:51.842607+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:20.690035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841155+0000","last_change":"2024-04-25T10:06:51.837947+0000","last_active":"2024-04-25T10:06:51.841155+0000","last_peered":"2024-04-25T10:06:51.841155+0000","last_clean":"2024-04-25T10:06:51.841155+0000","last_became_active":"2024-04-25T10:06:51.837635+0000","last_became_peered":"2024-04-25T10:06:51.837635+0000","last_unstale":"2024-04-25T10:06:51.841155+0000","last_undegraded":"2024-04-25T10:06:51.841155+0000","last_fullsized":"2024-04-25T10:06:51.841155+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:47.682171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854141+0000","last_change":"2024-04-25T10:06:51.847625+0000","last_active":"2024-04-25T10:06:51.854141+0000","last_peered":"2024-04-25T10:06:51.854141+0000","last_clean":"2024-04-25T10:06:51.854141+0000","last_became_active":"2024-04-25T10:06:51.847179+0000","last_became_peered":"2024-04-25T10:06:51.847179+0000","last_unstale":"2024-04-25T10:06:51.854141+0000","last_undegraded":"2024-04-25T10:06:51.854141+0000","last_fullsized":"2024-04-25T10:06:51.854141+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:40.042571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841895+0000","last_change":"2024-04-25T10:06:51.837753+0000","last_active":"2024-04-25T10:06:51.841895+0000","last_peered":"2024-04-25T10:06:51.841895+0000","last_clean":"2024-04-25T10:06:51.841895+0000","last_became_active":"2024-04-25T10:06:51.837418+0000","last_became_peered":"2024-04-25T10:06:51.837418+0000","last_unstale":"2024-04-25T10:06:51.841895+0000","last_undegraded":"2024-04-25T10:06:51.841895+0000","last_fullsized":"2024-04-25T10:06:51.841895+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:29.774279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845933+0000","last_change":"2024-04-25T10:06:51.843573+0000","last_active":"2024-04-25T10:06:51.845933+0000","last_peered":"2024-04-25T10:06:51.845933+0000","last_clean":"2024-04-25T10:06:51.845933+0000","last_became_active":"2024-04-25T10:06:51.843141+0000","last_became_peered":"2024-04-25T10:06:51.843141+0000","last_unstale":"2024-04-25T10:06:51.845933+0000","last_undegraded":"2024-04-25T10:06:51.845933+0000","last_fullsized":"2024-04-25T10:06:51.845933+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:14.554682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842933+0000","last_change":"2024-04-25T10:06:51.838802+0000","last_active":"2024-04-25T10:06:51.842933+0000","last_peered":"2024-04-25T10:06:51.842933+0000","last_clean":"2024-04-25T10:06:51.842933+0000","last_became_active":"2024-04-25T10:06:51.837871+0000","last_became_peered":"2024-04-25T10:06:51.837871+0000","last_unstale":"2024-04-25T10:06:51.842933+0000","last_undegraded":"2024-04-25T10:06:51.842933+0000","last_fullsized":"2024-04-25T10:06:51.842933+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:50.203747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.844998+0000","last_change":"2024-04-25T10:06:51.842491+0000","last_active":"2024-04-25T10:06:51.844998+0000","last_peered":"2024-04-25T10:06:51.844998+0000","last_clean":"2024-04-25T10:06:51.844998+0000","last_became_active":"2024-04-25T10:06:51.842212+0000","last_became_peered":"2024-04-25T10:06:51.842212+0000","last_unstale":"2024-04-25T10:06:51.844998+0000","last_undegraded":"2024-04-25T10:06:51.844998+0000","last_fullsized":"2024-04-25T10:06:51.844998+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:04.628909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.838191+0000","last_change":"2024-04-25T10:06:51.836207+0000","last_active":"2024-04-25T10:06:51.838191+0000","last_peered":"2024-04-25T10:06:51.838191+0000","last_clean":"2024-04-25T10:06:51.838191+0000","last_became_active":"2024-04-25T10:06:51.835856+0000","last_became_peered":"2024-04-25T10:06:51.835856+0000","last_unstale":"2024-04-25T10:06:51.838191+0000","last_undegraded":"2024-04-25T10:06:51.838191+0000","last_fullsized":"2024-04-25T10:06:51.838191+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:20.690035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.839761+0000","last_change":"2024-04-25T10:06:51.835570+0000","last_active":"2024-04-25T10:06:51.839761+0000","last_peered":"2024-04-25T10:06:51.839761+0000","last_clean":"2024-04-25T10:06:51.839761+0000","last_became_active":"2024-04-25T10:06:51.835287+0000","last_became_peered":"2024-04-25T10:06:51.835287+0000","last_unstale":"2024-04-25T10:06:51.839761+0000","last_undegraded":"2024-04-25T10:06:51.839761+0000","last_fullsized":"2024-04-25T10:06:51.839761+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:58.880955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846553+0000","last_change":"2024-04-25T10:06:51.845653+0000","last_active":"2024-04-25T10:06:51.846553+0000","last_peered":"2024-04-25T10:06:51.846553+0000","last_clean":"2024-04-25T10:06:51.846553+0000","last_became_active":"2024-04-25T10:06:51.845387+0000","last_became_peered":"2024-04-25T10:06:51.845387+0000","last_unstale":"2024-04-25T10:06:51.846553+0000","last_undegraded":"2024-04-25T10:06:51.846553+0000","last_fullsized":"2024-04-25T10:06:51.846553+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:22.306267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841737+0000","last_change":"2024-04-25T10:06:51.836879+0000","last_active":"2024-04-25T10:06:51.841737+0000","last_peered":"2024-04-25T10:06:51.841737+0000","last_clean":"2024-04-25T10:06:51.841737+0000","last_became_active":"2024-04-25T10:06:51.835980+0000","last_became_peered":"2024-04-25T10:06:51.835980+0000","last_unstale":"2024-04-25T10:06:51.841737+0000","last_undegraded":"2024-04-25T10:06:51.841737+0000","last_fullsized":"2024-04-25T10:06:51.841737+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:30.749971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841910+0000","last_change":"2024-04-25T10:06:51.838280+0000","last_active":"2024-04-25T10:06:51.841910+0000","last_peered":"2024-04-25T10:06:51.841910+0000","last_clean":"2024-04-25T10:06:51.841910+0000","last_became_active":"2024-04-25T10:06:51.837813+0000","last_became_peered":"2024-04-25T10:06:51.837813+0000","last_unstale":"2024-04-25T10:06:51.841910+0000","last_undegraded":"2024-04-25T10:06:51.841910+0000","last_fullsized":"2024-04-25T10:06:51.841910+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:58.880955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.833222+0000","last_change":"2024-04-25T10:06:51.831343+0000","last_active":"2024-04-25T10:06:51.833222+0000","last_peered":"2024-04-25T10:06:51.833222+0000","last_clean":"2024-04-25T10:06:51.833222+0000","last_became_active":"2024-04-25T10:06:51.831040+0000","last_became_peered":"2024-04-25T10:06:51.831040+0000","last_unstale":"2024-04-25T10:06:51.833222+0000","last_undegraded":"2024-04-25T10:06:51.833222+0000","last_fullsized":"2024-04-25T10:06:51.833222+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:04.569426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.832424+0000","last_change":"2024-04-25T10:06:51.827202+0000","last_active":"2024-04-25T10:06:51.832424+0000","last_peered":"2024-04-25T10:06:51.832424+0000","last_clean":"2024-04-25T10:06:51.832424+0000","last_became_active":"2024-04-25T10:06:51.826893+0000","last_became_peered":"2024-04-25T10:06:51.826893+0000","last_unstale":"2024-04-25T10:06:51.832424+0000","last_undegraded":"2024-04-25T10:06:51.832424+0000","last_fullsized":"2024-04-25T10:06:51.832424+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:25.546409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.836212+0000","last_change":"2024-04-25T10:06:51.835353+0000","last_active":"2024-04-25T10:06:51.836212+0000","last_peered":"2024-04-25T10:06:51.836212+0000","last_clean":"2024-04-25T10:06:51.836212+0000","last_became_active":"2024-04-25T10:06:51.834941+0000","last_became_peered":"2024-04-25T10:06:51.834941+0000","last_unstale":"2024-04-25T10:06:51.836212+0000","last_undegraded":"2024-04-25T10:06:51.836212+0000","last_fullsized":"2024-04-25T10:06:51.836212+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:48.121882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.833241+0000","last_change":"2024-04-25T10:06:51.830316+0000","last_active":"2024-04-25T10:06:51.833241+0000","last_peered":"2024-04-25T10:06:51.833241+0000","last_clean":"2024-04-25T10:06:51.833241+0000","last_became_active":"2024-04-25T10:06:51.830039+0000","last_became_peered":"2024-04-25T10:06:51.830039+0000","last_unstale":"2024-04-25T10:06:51.833241+0000","last_undegraded":"2024-04-25T10:06:51.833241+0000","last_fullsized":"2024-04-25T10:06:51.833241+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:01.498621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.833261+0000","last_change":"2024-04-25T10:06:51.830470+0000","last_active":"2024-04-25T10:06:51.833261+0000","last_peered":"2024-04-25T10:06:51.833261+0000","last_clean":"2024-04-25T10:06:51.833261+0000","last_became_active":"2024-04-25T10:06:51.830180+0000","last_became_peered":"2024-04-25T10:06:51.830180+0000","last_unstale":"2024-04-25T10:06:51.833261+0000","last_undegraded":"2024-04-25T10:06:51.833261+0000","last_fullsized":"2024-04-25T10:06:51.833261+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:17.127193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.840946+0000","last_change":"2024-04-25T10:06:51.839558+0000","last_active":"2024-04-25T10:06:51.840946+0000","last_peered":"2024-04-25T10:06:51.840946+0000","last_clean":"2024-04-25T10:06:51.840946+0000","last_became_active":"2024-04-25T10:06:51.839271+0000","last_became_peered":"2024-04-25T10:06:51.839271+0000","last_unstale":"2024-04-25T10:06:51.840946+0000","last_undegraded":"2024-04-25T10:06:51.840946+0000","last_fullsized":"2024-04-25T10:06:51.840946+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:11.162077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.832516+0000","last_change":"2024-04-25T10:06:51.828040+0000","last_active":"2024-04-25T10:06:51.832516+0000","last_peered":"2024-04-25T10:06:51.832516+0000","last_clean":"2024-04-25T10:06:51.832516+0000","last_became_active":"2024-04-25T10:06:51.826090+0000","last_became_peered":"2024-04-25T10:06:51.826090+0000","last_unstale":"2024-04-25T10:06:51.832516+0000","last_undegraded":"2024-04-25T10:06:51.832516+0000","last_fullsized":"2024-04-25T10:06:51.832516+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:09.382894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842444+0000","last_change":"2024-04-25T10:06:51.837883+0000","last_active":"2024-04-25T10:06:51.842444+0000","last_peered":"2024-04-25T10:06:51.842444+0000","last_clean":"2024-04-25T10:06:51.842444+0000","last_became_active":"2024-04-25T10:06:51.837631+0000","last_became_peered":"2024-04-25T10:06:51.837631+0000","last_unstale":"2024-04-25T10:06:51.842444+0000","last_undegraded":"2024-04-25T10:06:51.842444+0000","last_fullsized":"2024-04-25T10:06:51.842444+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:14.554682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846377+0000","last_change":"2024-04-25T10:06:51.842565+0000","last_active":"2024-04-25T10:06:51.846377+0000","last_peered":"2024-04-25T10:06:51.846377+0000","last_clean":"2024-04-25T10:06:51.846377+0000","last_became_active":"2024-04-25T10:06:51.842251+0000","last_became_peered":"2024-04-25T10:06:51.842251+0000","last_unstale":"2024-04-25T10:06:51.846377+0000","last_undegraded":"2024-04-25T10:06:51.846377+0000","last_fullsized":"2024-04-25T10:06:51.846377+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:32.311454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854076+0000","last_change":"2024-04-25T10:06:50.815053+0000","last_active":"2024-04-25T10:06:51.854076+0000","last_peered":"2024-04-25T10:06:51.854076+0000","last_clean":"2024-04-25T10:06:51.854076+0000","last_became_active":"2024-04-25T10:06:50.814623+0000","last_became_peered":"2024-04-25T10:06:50.814623+0000","last_unstale":"2024-04-25T10:06:51.854076+0000","last_undegraded":"2024-04-25T10:06:51.854076+0000","last_fullsized":"2024-04-25T10:06:51.854076+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:16.218950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846459+0000","last_change":"2024-04-25T10:06:51.844956+0000","last_active":"2024-04-25T10:06:51.846459+0000","last_peered":"2024-04-25T10:06:51.846459+0000","last_clean":"2024-04-25T10:06:51.846459+0000","last_became_active":"2024-04-25T10:06:51.844611+0000","last_became_peered":"2024-04-25T10:06:51.844611+0000","last_unstale":"2024-04-25T10:06:51.846459+0000","last_undegraded":"2024-04-25T10:06:51.846459+0000","last_fullsized":"2024-04-25T10:06:51.846459+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:50.170873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.848558+0000","last_change":"2024-04-25T10:06:50.822263+0000","last_active":"2024-04-25T10:06:51.848558+0000","last_peered":"2024-04-25T10:06:51.848558+0000","last_clean":"2024-04-25T10:06:51.848558+0000","last_became_active":"2024-04-25T10:06:50.821985+0000","last_became_peered":"2024-04-25T10:06:50.821985+0000","last_unstale":"2024-04-25T10:06:51.848558+0000","last_undegraded":"2024-04-25T10:06:51.848558+0000","last_fullsized":"2024-04-25T10:06:51.848558+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:37.641598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847288+0000","last_change":"2024-04-25T10:06:51.844989+0000","last_active":"2024-04-25T10:06:51.847288+0000","last_peered":"2024-04-25T10:06:51.847288+0000","last_clean":"2024-04-25T10:06:51.847288+0000","last_became_active":"2024-04-25T10:06:51.844614+0000","last_became_peered":"2024-04-25T10:06:51.844614+0000","last_unstale":"2024-04-25T10:06:51.847288+0000","last_undegraded":"2024-04-25T10:06:51.847288+0000","last_fullsized":"2024-04-25T10:06:51.847288+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:50.203747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.853873+0000","last_change":"2024-04-25T10:06:50.818405+0000","last_active":"2024-04-25T10:06:51.853873+0000","last_peered":"2024-04-25T10:06:51.853873+0000","last_clean":"2024-04-25T10:06:51.853873+0000","last_became_active":"2024-04-25T10:06:50.818118+0000","last_became_peered":"2024-04-25T10:06:50.818118+0000","last_unstale":"2024-04-25T10:06:51.853873+0000","last_undegraded":"2024-04-25T10:06:51.853873+0000","last_fullsized":"2024-04-25T10:06:51.853873+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:08.381421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.848517+0000","last_change":"2024-04-25T10:06:50.821564+0000","last_active":"2024-04-25T10:06:51.848517+0000","last_peered":"2024-04-25T10:06:51.848517+0000","last_clean":"2024-04-25T10:06:51.848517+0000","last_became_active":"2024-04-25T10:06:50.821257+0000","last_became_peered":"2024-04-25T10:06:50.821257+0000","last_unstale":"2024-04-25T10:06:51.848517+0000","last_undegraded":"2024-04-25T10:06:51.848517+0000","last_fullsized":"2024-04-25T10:06:51.848517+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:25.919994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834314+0000","last_change":"2024-04-25T10:06:51.832079+0000","last_active":"2024-04-25T10:06:51.834314+0000","last_peered":"2024-04-25T10:06:51.834314+0000","last_clean":"2024-04-25T10:06:51.834314+0000","last_became_active":"2024-04-25T10:06:51.831785+0000","last_became_peered":"2024-04-25T10:06:51.831785+0000","last_unstale":"2024-04-25T10:06:51.834314+0000","last_undegraded":"2024-04-25T10:06:51.834314+0000","last_fullsized":"2024-04-25T10:06:51.834314+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:27.917421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.853752+0000","last_change":"2024-04-25T10:06:50.819361+0000","last_active":"2024-04-25T10:06:51.853752+0000","last_peered":"2024-04-25T10:06:51.853752+0000","last_clean":"2024-04-25T10:06:51.853752+0000","last_became_active":"2024-04-25T10:06:50.819093+0000","last_became_peered":"2024-04-25T10:06:50.819093+0000","last_unstale":"2024-04-25T10:06:51.853752+0000","last_undegraded":"2024-04-25T10:06:51.853752+0000","last_fullsized":"2024-04-25T10:06:51.853752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:24.544937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841867+0000","last_change":"2024-04-25T10:06:51.835101+0000","last_active":"2024-04-25T10:06:51.841867+0000","last_peered":"2024-04-25T10:06:51.841867+0000","last_clean":"2024-04-25T10:06:51.841867+0000","last_became_active":"2024-04-25T10:06:51.834291+0000","last_became_peered":"2024-04-25T10:06:51.834291+0000","last_unstale":"2024-04-25T10:06:51.841867+0000","last_undegraded":"2024-04-25T10:06:51.841867+0000","last_fullsized":"2024-04-25T10:06:51.841867+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:22.306267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845318+0000","last_change":"2024-04-25T10:06:50.822762+0000","last_active":"2024-04-25T10:06:51.845318+0000","last_peered":"2024-04-25T10:06:51.845318+0000","last_clean":"2024-04-25T10:06:51.845318+0000","last_became_active":"2024-04-25T10:06:50.822463+0000","last_became_peered":"2024-04-25T10:06:50.822463+0000","last_unstale":"2024-04-25T10:06:51.845318+0000","last_undegraded":"2024-04-25T10:06:51.845318+0000","last_fullsized":"2024-04-25T10:06:51.845318+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:18.363517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841750+0000","last_change":"2024-04-25T10:06:51.833997+0000","last_active":"2024-04-25T10:06:51.841750+0000","last_peered":"2024-04-25T10:06:51.841750+0000","last_clean":"2024-04-25T10:06:51.841750+0000","last_became_active":"2024-04-25T10:06:51.833694+0000","last_became_peered":"2024-04-25T10:06:51.833694+0000","last_unstale":"2024-04-25T10:06:51.841750+0000","last_undegraded":"2024-04-25T10:06:51.841750+0000","last_fullsized":"2024-04-25T10:06:51.841750+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:50.170873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828476+0000","last_change":"2024-04-25T10:06:50.818945+0000","last_active":"2024-04-25T10:06:51.828476+0000","last_peered":"2024-04-25T10:06:51.828476+0000","last_clean":"2024-04-25T10:06:51.828476+0000","last_became_active":"2024-04-25T10:06:50.818631+0000","last_became_peered":"2024-04-25T10:06:50.818631+0000","last_unstale":"2024-04-25T10:06:51.828476+0000","last_undegraded":"2024-04-25T10:06:51.828476+0000","last_fullsized":"2024-04-25T10:06:51.828476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:49.202275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.839815+0000","last_change":"2024-04-25T10:06:51.837313+0000","last_active":"2024-04-25T10:06:51.839815+0000","last_peered":"2024-04-25T10:06:51.839815+0000","last_clean":"2024-04-25T10:06:51.839815+0000","last_became_active":"2024-04-25T10:06:51.837024+0000","last_became_peered":"2024-04-25T10:06:51.837024+0000","last_unstale":"2024-04-25T10:06:51.839815+0000","last_undegraded":"2024-04-25T10:06:51.839815+0000","last_fullsized":"2024-04-25T10:06:51.839815+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:28.322485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.825554+0000","last_change":"2024-04-25T10:06:50.818743+0000","last_active":"2024-04-25T10:06:51.825554+0000","last_peered":"2024-04-25T10:06:51.825554+0000","last_clean":"2024-04-25T10:06:51.825554+0000","last_became_active":"2024-04-25T10:06:50.818359+0000","last_became_peered":"2024-04-25T10:06:50.818359+0000","last_unstale":"2024-04-25T10:06:51.825554+0000","last_undegraded":"2024-04-25T10:06:51.825554+0000","last_fullsized":"2024-04-25T10:06:51.825554+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:47.120410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.848307+0000","last_change":"2024-04-25T10:06:51.846840+0000","last_active":"2024-04-25T10:06:51.848307+0000","last_peered":"2024-04-25T10:06:51.848307+0000","last_clean":"2024-04-25T10:06:51.848307+0000","last_became_active":"2024-04-25T10:06:51.846314+0000","last_became_peered":"2024-04-25T10:06:51.846314+0000","last_unstale":"2024-04-25T10:06:51.848307+0000","last_undegraded":"2024-04-25T10:06:51.848307+0000","last_fullsized":"2024-04-25T10:06:51.848307+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:29.810440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.823148+0000","last_change":"2024-04-25T10:06:50.819769+0000","last_active":"2024-04-25T10:06:51.823148+0000","last_peered":"2024-04-25T10:06:51.823148+0000","last_clean":"2024-04-25T10:06:51.823148+0000","last_became_active":"2024-04-25T10:06:50.819517+0000","last_became_peered":"2024-04-25T10:06:50.819517+0000","last_unstale":"2024-04-25T10:06:51.823148+0000","last_undegraded":"2024-04-25T10:06:51.823148+0000","last_fullsized":"2024-04-25T10:06:51.823148+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:15.288040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834870+0000","last_change":"2024-04-25T10:06:51.832027+0000","last_active":"2024-04-25T10:06:51.834870+0000","last_peered":"2024-04-25T10:06:51.834870+0000","last_clean":"2024-04-25T10:06:51.834870+0000","last_became_active":"2024-04-25T10:06:51.831718+0000","last_became_peered":"2024-04-25T10:06:51.831718+0000","last_unstale":"2024-04-25T10:06:51.834870+0000","last_undegraded":"2024-04-25T10:06:51.834870+0000","last_fullsized":"2024-04-25T10:06:51.834870+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:30.250121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.825643+0000","last_change":"2024-04-25T10:06:50.825390+0000","last_active":"2024-04-25T10:06:51.825643+0000","last_peered":"2024-04-25T10:06:51.825643+0000","last_clean":"2024-04-25T10:06:51.825643+0000","last_became_active":"2024-04-25T10:06:50.825085+0000","last_became_peered":"2024-04-25T10:06:50.825085+0000","last_unstale":"2024-04-25T10:06:51.825643+0000","last_undegraded":"2024-04-25T10:06:51.825643+0000","last_fullsized":"2024-04-25T10:06:51.825643+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:10.160604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845381+0000","last_change":"2024-04-25T10:06:50.818510+0000","last_active":"2024-04-25T10:06:51.845381+0000","last_peered":"2024-04-25T10:06:51.845381+0000","last_clean":"2024-04-25T10:06:51.845381+0000","last_became_active":"2024-04-25T10:06:50.818217+0000","last_became_peered":"2024-04-25T10:06:50.818217+0000","last_unstale":"2024-04-25T10:06:51.845381+0000","last_undegraded":"2024-04-25T10:06:51.845381+0000","last_fullsized":"2024-04-25T10:06:51.845381+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.521754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841431+0000","last_change":"2024-04-25T10:06:51.838484+0000","last_active":"2024-04-25T10:06:51.841431+0000","last_peered":"2024-04-25T10:06:51.841431+0000","last_clean":"2024-04-25T10:06:51.841431+0000","last_became_active":"2024-04-25T10:06:51.838211+0000","last_became_peered":"2024-04-25T10:06:51.838211+0000","last_unstale":"2024-04-25T10:06:51.841431+0000","last_undegraded":"2024-04-25T10:06:51.841431+0000","last_fullsized":"2024-04-25T10:06:51.841431+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:46.417710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.856308+0000","last_change":"2024-04-25T10:06:50.823945+0000","last_active":"2024-04-25T10:06:51.856308+0000","last_peered":"2024-04-25T10:06:51.856308+0000","last_clean":"2024-04-25T10:06:51.856308+0000","last_became_active":"2024-04-25T10:06:50.823670+0000","last_became_peered":"2024-04-25T10:06:50.823670+0000","last_unstale":"2024-04-25T10:06:51.856308+0000","last_undegraded":"2024-04-25T10:06:51.856308+0000","last_fullsized":"2024-04-25T10:06:51.856308+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:03.627437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845672+0000","last_change":"2024-04-25T10:06:51.843182+0000","last_active":"2024-04-25T10:06:51.845672+0000","last_peered":"2024-04-25T10:06:51.845672+0000","last_clean":"2024-04-25T10:06:51.845672+0000","last_became_active":"2024-04-25T10:06:51.842811+0000","last_became_peered":"2024-04-25T10:06:51.842811+0000","last_unstale":"2024-04-25T10:06:51.845672+0000","last_undegraded":"2024-04-25T10:06:51.845672+0000","last_fullsized":"2024-04-25T10:06:51.845672+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:43.266424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.850017+0000","last_change":"2024-04-25T10:06:50.823653+0000","last_active":"2024-04-25T10:06:51.850017+0000","last_peered":"2024-04-25T10:06:51.850017+0000","last_clean":"2024-04-25T10:06:51.850017+0000","last_became_active":"2024-04-25T10:06:50.823373+0000","last_became_peered":"2024-04-25T10:06:50.823373+0000","last_unstale":"2024-04-25T10:06:51.850017+0000","last_undegraded":"2024-04-25T10:06:51.850017+0000","last_fullsized":"2024-04-25T10:06:51.850017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:19.313862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842243+0000","last_change":"2024-04-25T10:06:51.836404+0000","last_active":"2024-04-25T10:06:51.842243+0000","last_peered":"2024-04-25T10:06:51.842243+0000","last_clean":"2024-04-25T10:06:51.842243+0000","last_became_active":"2024-04-25T10:06:51.835875+0000","last_became_peered":"2024-04-25T10:06:51.835875+0000","last_unstale":"2024-04-25T10:06:51.842243+0000","last_undegraded":"2024-04-25T10:06:51.842243+0000","last_fullsized":"2024-04-25T10:06:51.842243+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:30.250121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828780+0000","last_change":"2024-04-25T10:06:50.824101+0000","last_active":"2024-04-25T10:06:51.828780+0000","last_peered":"2024-04-25T10:06:51.828780+0000","last_clean":"2024-04-25T10:06:51.828780+0000","last_became_active":"2024-04-25T10:06:50.823800+0000","last_became_peered":"2024-04-25T10:06:50.823800+0000","last_unstale":"2024-04-25T10:06:51.828780+0000","last_undegraded":"2024-04-25T10:06:51.828780+0000","last_fullsized":"2024-04-25T10:06:51.828780+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:10.364676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846106+0000","last_change":"2024-04-25T10:06:51.842607+0000","last_active":"2024-04-25T10:06:51.846106+0000","last_peered":"2024-04-25T10:06:51.846106+0000","last_clean":"2024-04-25T10:06:51.846106+0000","last_became_active":"2024-04-25T10:06:51.842296+0000","last_became_peered":"2024-04-25T10:06:51.842296+0000","last_unstale":"2024-04-25T10:06:51.846106+0000","last_undegraded":"2024-04-25T10:06:51.846106+0000","last_fullsized":"2024-04-25T10:06:51.846106+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:26.337568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.831684+0000","last_change":"2024-04-25T10:06:50.826028+0000","last_active":"2024-04-25T10:06:51.831684+0000","last_peered":"2024-04-25T10:06:51.831684+0000","last_clean":"2024-04-25T10:06:51.831684+0000","last_became_active":"2024-04-25T10:06:50.825717+0000","last_became_peered":"2024-04-25T10:06:50.825717+0000","last_unstale":"2024-04-25T10:06:51.831684+0000","last_undegraded":"2024-04-25T10:06:51.831684+0000","last_fullsized":"2024-04-25T10:06:51.831684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:28.284253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834918+0000","last_change":"2024-04-25T10:06:51.833850+0000","last_active":"2024-04-25T10:06:51.834918+0000","last_peered":"2024-04-25T10:06:51.834918+0000","last_clean":"2024-04-25T10:06:51.834918+0000","last_became_active":"2024-04-25T10:06:51.833584+0000","last_became_peered":"2024-04-25T10:06:51.833584+0000","last_unstale":"2024-04-25T10:06:51.834918+0000","last_undegraded":"2024-04-25T10:06:51.834918+0000","last_fullsized":"2024-04-25T10:06:51.834918+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:01.261252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854164+0000","last_change":"2024-04-25T10:06:50.820905+0000","last_active":"2024-04-25T10:06:51.854164+0000","last_peered":"2024-04-25T10:06:51.854164+0000","last_clean":"2024-04-25T10:06:51.854164+0000","last_became_active":"2024-04-25T10:06:50.820642+0000","last_became_peered":"2024-04-25T10:06:50.820642+0000","last_unstale":"2024-04-25T10:06:51.854164+0000","last_undegraded":"2024-04-25T10:06:51.854164+0000","last_fullsized":"2024-04-25T10:06:51.854164+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:22.656356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846398+0000","last_change":"2024-04-25T10:06:51.842872+0000","last_active":"2024-04-25T10:06:51.846398+0000","last_peered":"2024-04-25T10:06:51.846398+0000","last_clean":"2024-04-25T10:06:51.846398+0000","last_became_active":"2024-04-25T10:06:51.842550+0000","last_became_peered":"2024-04-25T10:06:51.842550+0000","last_unstale":"2024-04-25T10:06:51.846398+0000","last_undegraded":"2024-04-25T10:06:51.846398+0000","last_fullsized":"2024-04-25T10:06:51.846398+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:51.599603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:49.800151+0000","last_change":"2024-04-25T10:06:49.800151+0000","last_active":"2024-04-25T10:06:49.800151+0000","last_peered":"2024-04-25T10:06:49.800151+0000","last_clean":"2024-04-25T10:06:49.800151+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:49.800151+0000","last_undegraded":"2024-04-25T10:06:49.800151+0000","last_fullsized":"2024-04-25T10:06:49.800151+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845649+0000","last_change":"2024-04-25T10:06:51.840224+0000","last_active":"2024-04-25T10:06:51.845649+0000","last_peered":"2024-04-25T10:06:51.845649+0000","last_clean":"2024-04-25T10:06:51.845649+0000","last_became_active":"2024-04-25T10:06:51.839917+0000","last_became_peered":"2024-04-25T10:06:51.839917+0000","last_unstale":"2024-04-25T10:06:51.845649+0000","last_undegraded":"2024-04-25T10:06:51.845649+0000","last_fullsized":"2024-04-25T10:06:51.845649+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:43.266424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.859198+0000","last_change":"2024-04-25T10:06:50.818607+0000","last_active":"2024-04-25T10:06:51.859198+0000","last_peered":"2024-04-25T10:06:51.859198+0000","last_clean":"2024-04-25T10:06:51.859198+0000","last_became_active":"2024-04-25T10:06:50.818315+0000","last_became_peered":"2024-04-25T10:06:50.818315+0000","last_unstale":"2024-04-25T10:06:51.859198+0000","last_undegraded":"2024-04-25T10:06:51.859198+0000","last_fullsized":"2024-04-25T10:06:51.859198+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:50.599925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.835690+0000","last_change":"2024-04-25T10:06:51.834754+0000","last_active":"2024-04-25T10:06:51.835690+0000","last_peered":"2024-04-25T10:06:51.835690+0000","last_clean":"2024-04-25T10:06:51.835690+0000","last_became_active":"2024-04-25T10:06:51.834425+0000","last_became_peered":"2024-04-25T10:06:51.834425+0000","last_unstale":"2024-04-25T10:06:51.835690+0000","last_undegraded":"2024-04-25T10:06:51.835690+0000","last_fullsized":"2024-04-25T10:06:51.835690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:17.220423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854430+0000","last_change":"2024-04-25T10:06:50.820485+0000","last_active":"2024-04-25T10:06:51.854430+0000","last_peered":"2024-04-25T10:06:51.854430+0000","last_clean":"2024-04-25T10:06:51.854430+0000","last_became_active":"2024-04-25T10:06:50.820215+0000","last_became_peered":"2024-04-25T10:06:50.820215+0000","last_unstale":"2024-04-25T10:06:51.854430+0000","last_undegraded":"2024-04-25T10:06:51.854430+0000","last_fullsized":"2024-04-25T10:06:51.854430+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:19.313862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847857+0000","last_change":"2024-04-25T10:06:51.845056+0000","last_active":"2024-04-25T10:06:51.847857+0000","last_peered":"2024-04-25T10:06:51.847857+0000","last_clean":"2024-04-25T10:06:51.847857+0000","last_became_active":"2024-04-25T10:06:51.844448+0000","last_became_peered":"2024-04-25T10:06:51.844448+0000","last_unstale":"2024-04-25T10:06:51.847857+0000","last_undegraded":"2024-04-25T10:06:51.847857+0000","last_fullsized":"2024-04-25T10:06:51.847857+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:26.337568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.831449+0000","last_change":"2024-04-25T10:06:50.825791+0000","last_active":"2024-04-25T10:06:51.831449+0000","last_peered":"2024-04-25T10:06:51.831449+0000","last_clean":"2024-04-25T10:06:51.831449+0000","last_became_active":"2024-04-25T10:06:50.825467+0000","last_became_peered":"2024-04-25T10:06:50.825467+0000","last_unstale":"2024-04-25T10:06:51.831449+0000","last_undegraded":"2024-04-25T10:06:51.831449+0000","last_fullsized":"2024-04-25T10:06:51.831449+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.521754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845737+0000","last_change":"2024-04-25T10:06:51.842252+0000","last_active":"2024-04-25T10:06:51.845737+0000","last_peered":"2024-04-25T10:06:51.845737+0000","last_clean":"2024-04-25T10:06:51.845737+0000","last_became_active":"2024-04-25T10:06:51.841999+0000","last_became_peered":"2024-04-25T10:06:51.841999+0000","last_unstale":"2024-04-25T10:06:51.845737+0000","last_undegraded":"2024-04-25T10:06:51.845737+0000","last_fullsized":"2024-04-25T10:06:51.845737+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:15.390698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.851706+0000","last_change":"2024-04-25T10:06:50.821347+0000","last_active":"2024-04-25T10:06:51.851706+0000","last_peered":"2024-04-25T10:06:51.851706+0000","last_clean":"2024-04-25T10:06:51.851706+0000","last_became_active":"2024-04-25T10:06:50.821073+0000","last_became_peered":"2024-04-25T10:06:50.821073+0000","last_unstale":"2024-04-25T10:06:51.851706+0000","last_undegraded":"2024-04-25T10:06:51.851706+0000","last_fullsized":"2024-04-25T10:06:51.851706+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:02.829457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842908+0000","last_change":"2024-04-25T10:06:51.838646+0000","last_active":"2024-04-25T10:06:51.842908+0000","last_peered":"2024-04-25T10:06:51.842908+0000","last_clean":"2024-04-25T10:06:51.842908+0000","last_became_active":"2024-04-25T10:06:51.835104+0000","last_became_peered":"2024-04-25T10:06:51.835104+0000","last_unstale":"2024-04-25T10:06:51.842908+0000","last_undegraded":"2024-04-25T10:06:51.842908+0000","last_fullsized":"2024-04-25T10:06:51.842908+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:10.566374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828795+0000","last_change":"2024-04-25T10:06:50.827168+0000","last_active":"2024-04-25T10:06:51.828795+0000","last_peered":"2024-04-25T10:06:51.828795+0000","last_clean":"2024-04-25T10:06:51.828795+0000","last_became_active":"2024-04-25T10:06:50.826210+0000","last_became_peered":"2024-04-25T10:06:50.826210+0000","last_unstale":"2024-04-25T10:06:51.828795+0000","last_undegraded":"2024-04-25T10:06:51.828795+0000","last_fullsized":"2024-04-25T10:06:51.828795+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.650773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.825362+0000","last_change":"2024-04-25T10:06:50.821404+0000","last_active":"2024-04-25T10:06:51.825362+0000","last_peered":"2024-04-25T10:06:51.825362+0000","last_clean":"2024-04-25T10:06:51.825362+0000","last_became_active":"2024-04-25T10:06:50.821129+0000","last_became_peered":"2024-04-25T10:06:50.821129+0000","last_unstale":"2024-04-25T10:06:51.825362+0000","last_undegraded":"2024-04-25T10:06:51.825362+0000","last_fullsized":"2024-04-25T10:06:51.825362+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.650773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.823231+0000","last_change":"2024-04-25T10:06:50.825554+0000","last_active":"2024-04-25T10:06:51.823231+0000","last_peered":"2024-04-25T10:06:51.823231+0000","last_clean":"2024-04-25T10:06:51.823231+0000","last_became_active":"2024-04-25T10:06:50.825226+0000","last_became_peered":"2024-04-25T10:06:50.825226+0000","last_unstale":"2024-04-25T10:06:51.823231+0000","last_undegraded":"2024-04-25T10:06:51.823231+0000","last_fullsized":"2024-04-25T10:06:51.823231+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:16.990316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834352+0000","last_change":"2024-04-25T10:06:51.832937+0000","last_active":"2024-04-25T10:06:51.834352+0000","last_peered":"2024-04-25T10:06:51.834352+0000","last_clean":"2024-04-25T10:06:51.834352+0000","last_became_active":"2024-04-25T10:06:51.832620+0000","last_became_peered":"2024-04-25T10:06:51.832620+0000","last_unstale":"2024-04-25T10:06:51.834352+0000","last_undegraded":"2024-04-25T10:06:51.834352+0000","last_fullsized":"2024-04-25T10:06:51.834352+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:47.015585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:49.800151+0000","last_change":"2024-04-25T10:06:49.800151+0000","last_active":"2024-04-25T10:06:49.800151+0000","last_peered":"2024-04-25T10:06:49.800151+0000","last_clean":"2024-04-25T10:06:49.800151+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:49.800151+0000","last_undegraded":"2024-04-25T10:06:49.800151+0000","last_fullsized":"2024-04-25T10:06:49.800151+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841715+0000","last_change":"2024-04-25T10:06:51.835802+0000","last_active":"2024-04-25T10:06:51.841715+0000","last_peered":"2024-04-25T10:06:51.841715+0000","last_clean":"2024-04-25T10:06:51.841715+0000","last_became_active":"2024-04-25T10:06:51.835196+0000","last_became_peered":"2024-04-25T10:06:51.835196+0000","last_unstale":"2024-04-25T10:06:51.841715+0000","last_undegraded":"2024-04-25T10:06:51.841715+0000","last_fullsized":"2024-04-25T10:06:51.841715+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:27.916081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828499+0000","last_change":"2024-04-25T10:06:50.826677+0000","last_active":"2024-04-25T10:06:51.828499+0000","last_peered":"2024-04-25T10:06:51.828499+0000","last_clean":"2024-04-25T10:06:51.828499+0000","last_became_active":"2024-04-25T10:06:50.826114+0000","last_became_peered":"2024-04-25T10:06:50.826114+0000","last_unstale":"2024-04-25T10:06:51.828499+0000","last_undegraded":"2024-04-25T10:06:51.828499+0000","last_fullsized":"2024-04-25T10:06:51.828499+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:39.041098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841453+0000","last_change":"2024-04-25T10:06:51.837949+0000","last_active":"2024-04-25T10:06:51.841453+0000","last_peered":"2024-04-25T10:06:51.841453+0000","last_clean":"2024-04-25T10:06:51.841453+0000","last_became_active":"2024-04-25T10:06:51.837681+0000","last_became_peered":"2024-04-25T10:06:51.837681+0000","last_unstale":"2024-04-25T10:06:51.841453+0000","last_undegraded":"2024-04-25T10:06:51.841453+0000","last_fullsized":"2024-04-25T10:06:51.841453+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:17.563413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828714+0000","last_change":"2024-04-25T10:06:50.826236+0000","last_active":"2024-04-25T10:06:51.828714+0000","last_peered":"2024-04-25T10:06:51.828714+0000","last_clean":"2024-04-25T10:06:51.828714+0000","last_became_active":"2024-04-25T10:06:50.825899+0000","last_became_peered":"2024-04-25T10:06:50.825899+0000","last_unstale":"2024-04-25T10:06:51.828714+0000","last_undegraded":"2024-04-25T10:06:51.828714+0000","last_fullsized":"2024-04-25T10:06:51.828714+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:15.288040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845716+0000","last_change":"2024-04-25T10:06:51.840773+0000","last_active":"2024-04-25T10:06:51.845716+0000","last_peered":"2024-04-25T10:06:51.845716+0000","last_clean":"2024-04-25T10:06:51.845716+0000","last_became_active":"2024-04-25T10:06:51.840412+0000","last_became_peered":"2024-04-25T10:06:51.840412+0000","last_unstale":"2024-04-25T10:06:51.845716+0000","last_undegraded":"2024-04-25T10:06:51.845716+0000","last_fullsized":"2024-04-25T10:06:51.845716+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:58.954708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.827378+0000","last_change":"2024-04-25T10:06:50.819732+0000","last_active":"2024-04-25T10:06:51.827378+0000","last_peered":"2024-04-25T10:06:51.827378+0000","last_clean":"2024-04-25T10:06:51.827378+0000","last_became_active":"2024-04-25T10:06:50.819448+0000","last_became_peered":"2024-04-25T10:06:50.819448+0000","last_unstale":"2024-04-25T10:06:51.827378+0000","last_undegraded":"2024-04-25T10:06:51.827378+0000","last_fullsized":"2024-04-25T10:06:51.827378+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:09.564901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.823889+0000","last_change":"2024-04-25T10:06:50.821944+0000","last_active":"2024-04-25T10:06:51.823889+0000","last_peered":"2024-04-25T10:06:51.823889+0000","last_clean":"2024-04-25T10:06:51.823889+0000","last_became_active":"2024-04-25T10:06:50.821425+0000","last_became_peered":"2024-04-25T10:06:50.821425+0000","last_unstale":"2024-04-25T10:06:51.823889+0000","last_undegraded":"2024-04-25T10:06:51.823889+0000","last_fullsized":"2024-04-25T10:06:51.823889+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:41.915878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845619+0000","last_change":"2024-04-25T10:06:51.843538+0000","last_active":"2024-04-25T10:06:51.845619+0000","last_peered":"2024-04-25T10:06:51.845619+0000","last_clean":"2024-04-25T10:06:51.845619+0000","last_became_active":"2024-04-25T10:06:51.842717+0000","last_became_peered":"2024-04-25T10:06:51.842717+0000","last_unstale":"2024-04-25T10:06:51.845619+0000","last_undegraded":"2024-04-25T10:06:51.845619+0000","last_fullsized":"2024-04-25T10:06:51.845619+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:32.311454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.831838+0000","last_change":"2024-04-25T10:06:50.822997+0000","last_active":"2024-04-25T10:06:51.831838+0000","last_peered":"2024-04-25T10:06:51.831838+0000","last_clean":"2024-04-25T10:06:51.831838+0000","last_became_active":"2024-04-25T10:06:50.822554+0000","last_became_peered":"2024-04-25T10:06:50.822554+0000","last_unstale":"2024-04-25T10:06:51.831838+0000","last_undegraded":"2024-04-25T10:06:51.831838+0000","last_fullsized":"2024-04-25T10:06:51.831838+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:00.259780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834087+0000","last_change":"2024-04-25T10:06:51.831355+0000","last_active":"2024-04-25T10:06:51.834087+0000","last_peered":"2024-04-25T10:06:51.834087+0000","last_clean":"2024-04-25T10:06:51.834087+0000","last_became_active":"2024-04-25T10:06:51.831047+0000","last_became_peered":"2024-04-25T10:06:51.831047+0000","last_unstale":"2024-04-25T10:06:51.834087+0000","last_undegraded":"2024-04-25T10:06:51.834087+0000","last_fullsized":"2024-04-25T10:06:51.834087+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:29.285725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.832092+0000","last_change":"2024-04-25T10:06:50.823705+0000","last_active":"2024-04-25T10:06:51.832092+0000","last_peered":"2024-04-25T10:06:51.832092+0000","last_clean":"2024-04-25T10:06:51.832092+0000","last_became_active":"2024-04-25T10:06:50.823452+0000","last_became_peered":"2024-04-25T10:06:50.823452+0000","last_unstale":"2024-04-25T10:06:51.832092+0000","last_undegraded":"2024-04-25T10:06:51.832092+0000","last_fullsized":"2024-04-25T10:06:51.832092+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:50.599925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:50.801623+0000","last_change":"2024-04-25T10:06:50.801623+0000","last_active":"2024-04-25T10:06:50.801623+0000","last_peered":"2024-04-25T10:06:50.801623+0000","last_clean":"2024-04-25T10:06:50.801623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:50.801623+0000","last_undegraded":"2024-04-25T10:06:50.801623+0000","last_fullsized":"2024-04-25T10:06:50.801623+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.849495+0000","last_change":"2024-04-25T10:06:50.823009+0000","last_active":"2024-04-25T10:06:51.849495+0000","last_peered":"2024-04-25T10:06:51.849495+0000","last_clean":"2024-04-25T10:06:51.849495+0000","last_became_active":"2024-04-25T10:06:50.822686+0000","last_became_peered":"2024-04-25T10:06:50.822686+0000","last_unstale":"2024-04-25T10:06:51.849495+0000","last_undegraded":"2024-04-25T10:06:51.849495+0000","last_fullsized":"2024-04-25T10:06:51.849495+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:53.108324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":218,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.822612+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:06:51.822612+0000","last_peered":"2024-04-25T10:06:51.822612+0000","last_clean":"2024-04-25T10:06:51.822612+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:06:51.822612+0000","last_undegraded":"2024-04-25T10:06:51.822612+0000","last_fullsized":"2024-04-25T10:06:51.822612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":100,"acting":100,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":60,"acting":60,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":536576,"data_stored":214001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":540672,"data_stored":214583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509467,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":207192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509467,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":544768,"data_stored":215579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":207192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1130496,"data_stored":804405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":540672,"data_stored":214639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1089536,"data_stored":785468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:55.530 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2024-04-25T10:06:55.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:50.801623+0000', 'last_change': '2024-04-25T10:06:50.801623+0000', 'last_active': '2024-04-25T10:06:50.801623+0000', 'last_peered': '2024-04-25T10:06:50.801623+0000', 'last_clean': '2024-04-25T10:06:50.801623+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:50.801623+0000', 'last_undegraded': '2024-04-25T10:06:50.801623+0000', 'last_fullsized': '2024-04-25T10:06:50.801623+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:55.530 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2024-04-25T10:06:55.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:50.801623+0000', 'last_change': '2024-04-25T10:06:50.801623+0000', 'last_active': '2024-04-25T10:06:50.801623+0000', 'last_peered': '2024-04-25T10:06:50.801623+0000', 'last_clean': '2024-04-25T10:06:50.801623+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:50.801623+0000', 'last_undegraded': '2024-04-25T10:06:50.801623+0000', 'last_fullsized': '2024-04-25T10:06:50.801623+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:55.530 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2024-04-25T10:06:55.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:50.801623+0000', 'last_change': '2024-04-25T10:06:50.801623+0000', 'last_active': '2024-04-25T10:06:50.801623+0000', 'last_peered': '2024-04-25T10:06:50.801623+0000', 'last_clean': '2024-04-25T10:06:50.801623+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:50.801623+0000', 'last_undegraded': '2024-04-25T10:06:50.801623+0000', 'last_fullsized': '2024-04-25T10:06:50.801623+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:55.530 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2024-04-25T10:06:55.531 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:50.801623+0000', 'last_change': '2024-04-25T10:06:50.801623+0000', 'last_active': '2024-04-25T10:06:50.801623+0000', 'last_peered': '2024-04-25T10:06:50.801623+0000', 'last_clean': '2024-04-25T10:06:50.801623+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:50.801623+0000', 'last_undegraded': '2024-04-25T10:06:50.801623+0000', 'last_fullsized': '2024-04-25T10:06:50.801623+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:55.531 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2024-04-25T10:06:55.531 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:50.801623+0000', 'last_change': '2024-04-25T10:06:50.801623+0000', 'last_active': '2024-04-25T10:06:50.801623+0000', 'last_peered': '2024-04-25T10:06:50.801623+0000', 'last_clean': '2024-04-25T10:06:50.801623+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:50.801623+0000', 'last_undegraded': '2024-04-25T10:06:50.801623+0000', 'last_fullsized': '2024-04-25T10:06:50.801623+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:55.531 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2024-04-25T10:06:55.531 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:50.801623+0000', 'last_change': '2024-04-25T10:06:50.801623+0000', 'last_active': '2024-04-25T10:06:50.801623+0000', 'last_peered': '2024-04-25T10:06:50.801623+0000', 'last_clean': '2024-04-25T10:06:50.801623+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:50.801623+0000', 'last_undegraded': '2024-04-25T10:06:50.801623+0000', 'last_fullsized': '2024-04-25T10:06:50.801623+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:55.531 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2024-04-25T10:06:55.531 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:50.801623+0000', 'last_change': '2024-04-25T10:06:50.801623+0000', 'last_active': '2024-04-25T10:06:50.801623+0000', 'last_peered': '2024-04-25T10:06:50.801623+0000', 'last_clean': '2024-04-25T10:06:50.801623+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:50.801623+0000', 'last_undegraded': '2024-04-25T10:06:50.801623+0000', 'last_fullsized': '2024-04-25T10:06:50.801623+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:55.531 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2024-04-25T10:06:55.531 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:50.801623+0000', 'last_change': '2024-04-25T10:06:50.801623+0000', 'last_active': '2024-04-25T10:06:50.801623+0000', 'last_peered': '2024-04-25T10:06:50.801623+0000', 'last_clean': '2024-04-25T10:06:50.801623+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:50.801623+0000', 'last_undegraded': '2024-04-25T10:06:50.801623+0000', 'last_fullsized': '2024-04-25T10:06:50.801623+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:55.531 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-04-25T10:06:55.531 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:50.801623+0000', 'last_change': '2024-04-25T10:06:50.801623+0000', 'last_active': '2024-04-25T10:06:50.801623+0000', 'last_peered': '2024-04-25T10:06:50.801623+0000', 'last_clean': '2024-04-25T10:06:50.801623+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:50.801623+0000', 'last_undegraded': '2024-04-25T10:06:50.801623+0000', 'last_fullsized': '2024-04-25T10:06:50.801623+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:55.532 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-04-25T10:06:55.532 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:50.801623+0000', 'last_change': '2024-04-25T10:06:50.801623+0000', 'last_active': '2024-04-25T10:06:50.801623+0000', 'last_peered': '2024-04-25T10:06:50.801623+0000', 'last_clean': '2024-04-25T10:06:50.801623+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:50.801623+0000', 'last_undegraded': '2024-04-25T10:06:50.801623+0000', 'last_fullsized': '2024-04-25T10:06:50.801623+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:55.532 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-04-25T10:06:55.532 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:49.800151+0000', 'last_change': '2024-04-25T10:06:49.800151+0000', 'last_active': '2024-04-25T10:06:49.800151+0000', 'last_peered': '2024-04-25T10:06:49.800151+0000', 'last_clean': '2024-04-25T10:06:49.800151+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:49.800151+0000', 'last_undegraded': '2024-04-25T10:06:49.800151+0000', 'last_fullsized': '2024-04-25T10:06:49.800151+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:49.800151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:49.800151+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:49.800151+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:55.532 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-04-25T10:06:55.532 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:50.801623+0000', 'last_change': '2024-04-25T10:06:50.801623+0000', 'last_active': '2024-04-25T10:06:50.801623+0000', 'last_peered': '2024-04-25T10:06:50.801623+0000', 'last_clean': '2024-04-25T10:06:50.801623+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:50.801623+0000', 'last_undegraded': '2024-04-25T10:06:50.801623+0000', 'last_fullsized': '2024-04-25T10:06:50.801623+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:55.532 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2024-04-25T10:06:55.532 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:50.801623+0000', 'last_change': '2024-04-25T10:06:50.801623+0000', 'last_active': '2024-04-25T10:06:50.801623+0000', 'last_peered': '2024-04-25T10:06:50.801623+0000', 'last_clean': '2024-04-25T10:06:50.801623+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:50.801623+0000', 'last_undegraded': '2024-04-25T10:06:50.801623+0000', 'last_fullsized': '2024-04-25T10:06:50.801623+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:55.532 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-04-25T10:06:55.532 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:49.800151+0000', 'last_change': '2024-04-25T10:06:49.800151+0000', 'last_active': '2024-04-25T10:06:49.800151+0000', 'last_peered': '2024-04-25T10:06:49.800151+0000', 'last_clean': '2024-04-25T10:06:49.800151+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:49.800151+0000', 'last_undegraded': '2024-04-25T10:06:49.800151+0000', 'last_fullsized': '2024-04-25T10:06:49.800151+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:49.800151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:49.800151+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:49.800151+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:55.533 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-04-25T10:06:55.533 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:50.801623+0000', 'last_change': '2024-04-25T10:06:50.801623+0000', 'last_active': '2024-04-25T10:06:50.801623+0000', 'last_peered': '2024-04-25T10:06:50.801623+0000', 'last_clean': '2024-04-25T10:06:50.801623+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:50.801623+0000', 'last_undegraded': '2024-04-25T10:06:50.801623+0000', 'last_fullsized': '2024-04-25T10:06:50.801623+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:55.533 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2024-04-25T10:06:55.533 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:06:50.801623+0000', 'last_change': '2024-04-25T10:06:50.801623+0000', 'last_active': '2024-04-25T10:06:50.801623+0000', 'last_peered': '2024-04-25T10:06:50.801623+0000', 'last_clean': '2024-04-25T10:06:50.801623+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:06:50.801623+0000', 'last_undegraded': '2024-04-25T10:06:50.801623+0000', 'last_fullsized': '2024-04-25T10:06:50.801623+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'last_clean_scrub_stamp': '2024-04-25T10:06:50.801623+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:06:55.533 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:06:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.660+0000 7f43d31b0640 1 Processor -- start 2024-04-25T10:06:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.661+0000 7f43d31b0640 1 -- start start 2024-04-25T10:06:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.661+0000 7f43d31b0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43cc0cea00 0x7f43cc0cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.661+0000 7f43d31b0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43cc0d0430 con 0x7f43cc15f120 2024-04-25T10:06:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.661+0000 7f43d31b0640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43cc0d0710 con 0x7f43cc0cfbe0 2024-04-25T10:06:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.661+0000 7f43d31b0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43cc0d09f0 con 0x7f43cc0cea00 2024-04-25T10:06:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.661+0000 7f43cbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43cc0cea00 0x7f43cc0cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.661+0000 7f43cbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43cc0cea00 0x7f43cc0cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50484/0 (socket says 172.21.15.16:50484) 2024-04-25T10:06:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.661+0000 7f43d0f4c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f43cc15f120 0x7f43cc0ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59502/0 (socket says 172.21.15.16:59502) 2024-04-25T10:06:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.661+0000 7f43cbfff640 1 -- 172.21.15.16:0/1073405518 learned_addr learned my addr 172.21.15.16:0/1073405518 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.662+0000 7f43cbfff640 1 -- 172.21.15.16:0/1073405518 >> v1:172.21.15.186:6789/0 conn(0x7f43cc0cfbe0 legacy=0x7f43cc0ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.662+0000 7f43cb7fe640 1 -- 172.21.15.16:0/1073405518 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222761089 0 0) 0x7f43cc0d0430 con 0x7f43cc15f120 2024-04-25T10:06:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.662+0000 7f43cbfff640 1 -- 172.21.15.16:0/1073405518 >> v1:172.21.15.16:6789/0 conn(0x7f43cc15f120 legacy=0x7f43cc0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.662+0000 7f43cbfff640 1 -- 172.21.15.16:0/1073405518 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc0cc7d0 con 0x7f43cc0cea00 2024-04-25T10:06:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.663+0000 7f43cbfff640 1 --2- 172.21.15.16:0/1073405518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43cc0cea00 0x7f43cc0cf6a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009870 tx=0x7f43b802a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f301d63eaf2db4f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.663+0000 7f43cb7fe640 1 -- 172.21.15.16:0/1073405518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8037070 con 0x7f43cc0cea00 2024-04-25T10:06:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.663+0000 7f43cb7fe640 1 -- 172.21.15.16:0/1073405518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b8032640 con 0x7f43cc0cea00 2024-04-25T10:06:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.663+0000 7f43cb7fe640 1 -- 172.21.15.16:0/1073405518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8032a10 con 0x7f43cc0cea00 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.663+0000 7f43d31b0640 1 -- 172.21.15.16:0/1073405518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43cc0cea00 msgr2=0x7f43cc0cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.663+0000 7f43d31b0640 1 --2- 172.21.15.16:0/1073405518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43cc0cea00 0x7f43cc0cf6a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009870 tx=0x7f43b802a330 comp rx=0 tx=0).stop 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.663+0000 7f43d31b0640 1 -- 172.21.15.16:0/1073405518 shutdown_connections 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.663+0000 7f43d31b0640 1 --2- 172.21.15.16:0/1073405518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43cc0cea00 0x7f43cc0cf6a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.663+0000 7f43d31b0640 1 -- 172.21.15.16:0/1073405518 >> 172.21.15.16:0/1073405518 conn(0x7f43cc0d1900 msgr2=0x7f43cc0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.663+0000 7f43d31b0640 1 -- 172.21.15.16:0/1073405518 shutdown_connections 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.663+0000 7f43d31b0640 1 -- 172.21.15.16:0/1073405518 wait complete. 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.664+0000 7f43d31b0640 1 Processor -- start 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.665+0000 7f43d31b0640 1 -- start start 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.665+0000 7f43d31b0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43cc0cea00 0x7f43cc15bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.665+0000 7f43d31b0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43cc0cfbe0 0x7f43cc0c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.665+0000 7f43d31b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43cc15f120 0x7f43cc0c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.665+0000 7f43d31b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0c5450 con 0x7f43cc15f120 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.665+0000 7f43d31b0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0cc480 con 0x7f43cc0cfbe0 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.665+0000 7f43d31b0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0d0070 con 0x7f43cc0cea00 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.665+0000 7f43d174d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43cc15f120 0x7f43cc0c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.666+0000 7f43d174d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43cc15f120 0x7f43cc0c7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50498/0 (socket says 172.21.15.16:50498) 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.666+0000 7f43d174d640 1 -- 172.21.15.16:0/1638757435 learned_addr learned my addr 172.21.15.16:0/1638757435 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.666+0000 7f43cbfff640 1 --2- 172.21.15.16:0/1638757435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43cc0cfbe0 0x7f43cc0c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.666+0000 7f43d0f4c640 1 --2- 172.21.15.16:0/1638757435 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43cc0cea00 0x7f43cc15bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.666+0000 7f43d174d640 1 -- 172.21.15.16:0/1638757435 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43cc0cea00 msgr2=0x7f43cc15bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.666+0000 7f43d174d640 1 --2- 172.21.15.16:0/1638757435 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43cc0cea00 0x7f43cc15bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.666+0000 7f43d174d640 1 -- 172.21.15.16:0/1638757435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43cc0cfbe0 msgr2=0x7f43cc0c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.666+0000 7f43d174d640 1 --2- 172.21.15.16:0/1638757435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43cc0cfbe0 0x7f43cc0c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.666+0000 7f43d174d640 1 -- 172.21.15.16:0/1638757435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc0b51a0 con 0x7f43cc15f120 2024-04-25T10:06:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.666+0000 7f43d174d640 1 --2- 172.21.15.16:0/1638757435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43cc15f120 0x7f43cc0c7070 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f43c400c780 tx=0x7f43c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.666+0000 7f43c97fa640 1 -- 172.21.15.16:0/1638757435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4004b60 con 0x7f43cc15f120 2024-04-25T10:06:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.666+0000 7f43c97fa640 1 -- 172.21.15.16:0/1638757435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c4004d00 con 0x7f43cc15f120 2024-04-25T10:06:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.666+0000 7f43c97fa640 1 -- 172.21.15.16:0/1638757435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c400aa90 con 0x7f43cc15f120 2024-04-25T10:06:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.666+0000 7f43d31b0640 1 -- 172.21.15.16:0/1638757435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43cc0b5480 con 0x7f43cc15f120 2024-04-25T10:06:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.666+0000 7f43d31b0640 1 -- 172.21.15.16:0/1638757435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43cc0b5a40 con 0x7f43cc15f120 2024-04-25T10:06:55.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.667+0000 7f43d31b0640 1 -- 172.21.15.16:0/1638757435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43cc067850 con 0x7f43cc15f120 2024-04-25T10:06:55.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.668+0000 7f43c97fa640 1 -- 172.21.15.16:0/1638757435 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43c400ac30 con 0x7f43cc15f120 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.668+0000 7f43c97fa640 1 --2- 172.21.15.16:0/1638757435 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f43b006b4d0 0x7f43b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.668+0000 7f43c97fa640 1 -- 172.21.15.16:0/1638757435 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43c408a420 con 0x7f43cc15f120 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.668+0000 7f43d0f4c640 1 --2- 172.21.15.16:0/1638757435 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f43b006b4d0 0x7f43b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.669+0000 7f43d0f4c640 1 --2- 172.21.15.16:0/1638757435 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f43b006b4d0 0x7f43b006d990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f43c0000f60 tx=0x7f43c0007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.672+0000 7f43c97fa640 1 -- 172.21.15.16:0/1638757435 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f43c4020070 con 0x7f43cc15f120 2024-04-25T10:06:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.924+0000 7f43d31b0640 1 -- 172.21.15.16:0/1638757435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f43cc09ada0 con 0x7f43cc15f120 2024-04-25T10:06:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.924+0000 7f43c97fa640 1 -- 172.21.15.16:0/1638757435 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1433 (secure 0 0 0) 0x7f43cc09ada0 con 0x7f43cc15f120 2024-04-25T10:06:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:55.930 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":129,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039497,"num_in_osds":8,"osd_in_since":1714039488,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":248152064,"bytes_avail":772845961216,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":21,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"b","status":"up:active","gid":34883}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-04-25T10:06:39.801307+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.926+0000 7f43d31b0640 1 -- 172.21.15.16:0/1638757435 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f43b006b4d0 msgr2=0x7f43b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.926+0000 7f43d31b0640 1 --2- 172.21.15.16:0/1638757435 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f43b006b4d0 0x7f43b006d990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f43c0000f60 tx=0x7f43c0007d50 comp rx=0 tx=0).stop 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.926+0000 7f43d31b0640 1 -- 172.21.15.16:0/1638757435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43cc15f120 msgr2=0x7f43cc0c7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.926+0000 7f43d31b0640 1 --2- 172.21.15.16:0/1638757435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43cc15f120 0x7f43cc0c7070 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f43c400c780 tx=0x7f43c400cc40 comp rx=0 tx=0).stop 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.926+0000 7f43d31b0640 1 -- 172.21.15.16:0/1638757435 shutdown_connections 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.926+0000 7f43d31b0640 1 --2- 172.21.15.16:0/1638757435 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f43b006b4d0 0x7f43b006d990 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.926+0000 7f43d31b0640 1 --2- 172.21.15.16:0/1638757435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43cc15f120 0x7f43cc0c7070 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.926+0000 7f43d31b0640 1 --2- 172.21.15.16:0/1638757435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43cc0cfbe0 0x7f43cc0c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.926+0000 7f43d31b0640 1 --2- 172.21.15.16:0/1638757435 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43cc0cea00 0x7f43cc15bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.926+0000 7f43d31b0640 1 -- 172.21.15.16:0/1638757435 >> 172.21.15.16:0/1638757435 conn(0x7f43cc0d1900 msgr2=0x7f43cc0b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.926+0000 7f43d31b0640 1 -- 172.21.15.16:0/1638757435 shutdown_connections 2024-04-25T10:06:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:55.926+0000 7f43d31b0640 1 -- 172.21.15.16:0/1638757435 wait complete. 2024-04-25T10:06:55.943 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:06:55.943 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.038+0000 7ff34e61f640 1 Processor -- start 2024-04-25T10:06:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.038+0000 7ff34e61f640 1 -- start start 2024-04-25T10:06:56.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.039+0000 7ff34e61f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3480d35b0 0x7ff3480b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.039+0000 7ff34e61f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34815e0c0 con 0x7ff3480b5990 2024-04-25T10:06:56.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.039+0000 7ff34e61f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3480d2510 con 0x7ff34815dce0 2024-04-25T10:06:56.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.039+0000 7ff34e61f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3480b5de0 con 0x7ff3480d35b0 2024-04-25T10:06:56.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.039+0000 7ff34cbbc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff3480b5990 0x7ff3480b6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59516/0 (socket says 172.21.15.16:59516) 2024-04-25T10:06:56.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.039+0000 7ff34cbbc640 1 -- 172.21.15.16:0/3893030505 learned_addr learned my addr 172.21.15.16:0/3893030505 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:06:56.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.039+0000 7ff3477fe640 1 --2- 172.21.15.16:0/3893030505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3480d35b0 0x7ff3480b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff346ffd640 1 -- 172.21.15.16:0/3893030505 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310058111 0 0) 0x7ff34815e0c0 con 0x7ff3480b5990 2024-04-25T10:06:56.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff346ffd640 1 -- 172.21.15.16:0/3893030505 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff334003520 con 0x7ff3480b5990 2024-04-25T10:06:56.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff346ffd640 1 -- 172.21.15.16:0/3893030505 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3720355579 0 0) 0x7ff334003520 con 0x7ff3480b5990 2024-04-25T10:06:56.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff346ffd640 1 -- 172.21.15.16:0/3893030505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3480d35b0 msgr2=0x7ff3480b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff346ffd640 1 --2- 172.21.15.16:0/3893030505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3480d35b0 0x7ff3480b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff346ffd640 1 -- 172.21.15.16:0/3893030505 >> v1:172.21.15.186:6789/0 conn(0x7ff34815dce0 legacy=0x7ff3480d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff346ffd640 1 -- 172.21.15.16:0/3893030505 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3480d27f0 con 0x7ff3480b5990 2024-04-25T10:06:56.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff346ffd640 1 -- 172.21.15.16:0/3893030505 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7ff3400019f0 con 0x7ff3480b5990 2024-04-25T10:06:56.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff346ffd640 1 -- 172.21.15.16:0/3893030505 >> v1:172.21.15.16:6789/0 conn(0x7ff3480b5990 legacy=0x7ff3480b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff346ffd640 1 --2- 172.21.15.16:0/3893030505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3480d35b0 0x7ff3340039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff346ffd640 1 --2- 172.21.15.16:0/3893030505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff334003f20 0x7ff334004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff346ffd640 1 --2- 172.21.15.16:0/3893030505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff334004d40 0x7ff3340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff346ffd640 1 -- 172.21.15.16:0/3893030505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3480b5de0 con 0x7ff334003f20 2024-04-25T10:06:56.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff346ffd640 1 -- 172.21.15.16:0/3893030505 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33400ba20 con 0x7ff334004d40 2024-04-25T10:06:56.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff346ffd640 1 -- 172.21.15.16:0/3893030505 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33400bc20 con 0x7ff3480d35b0 2024-04-25T10:06:56.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff3477fe640 1 --2- 172.21.15.16:0/3893030505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff334003f20 0x7ff334004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff347fff640 1 --2- 172.21.15.16:0/3893030505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3480d35b0 0x7ff3340039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff34cbbc640 1 --2- 172.21.15.16:0/3893030505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff334004d40 0x7ff3340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff3477fe640 1 -- 172.21.15.16:0/3893030505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3480d35b0 msgr2=0x7ff3340039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff3477fe640 1 --2- 172.21.15.16:0/3893030505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3480d35b0 0x7ff3340039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff3477fe640 1 -- 172.21.15.16:0/3893030505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff334004d40 msgr2=0x7ff3340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff3477fe640 1 --2- 172.21.15.16:0/3893030505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff334004d40 0x7ff3340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff3477fe640 1 -- 172.21.15.16:0/3893030505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff334003520 con 0x7ff334003f20 2024-04-25T10:06:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff34cbbc640 1 --2- 172.21.15.16:0/3893030505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff334004d40 0x7ff3340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.040+0000 7ff347fff640 1 --2- 172.21.15.16:0/3893030505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3480d35b0 0x7ff3340039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.041+0000 7ff3477fe640 1 --2- 172.21.15.16:0/3893030505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff334003f20 0x7ff334004800 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff338009ac0 tx=0x7ff334005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8997858e4f625dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.041+0000 7ff346ffd640 1 -- 172.21.15.16:0/3893030505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff338036070 con 0x7ff334003f20 2024-04-25T10:06:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.041+0000 7ff346ffd640 1 -- 172.21.15.16:0/3893030505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff338031400 con 0x7ff334003f20 2024-04-25T10:06:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.041+0000 7ff346ffd640 1 -- 172.21.15.16:0/3893030505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff338009690 con 0x7ff334003f20 2024-04-25T10:06:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.042+0000 7ff34e61f640 1 -- 172.21.15.16:0/3893030505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff334003f20 msgr2=0x7ff334004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.042+0000 7ff34e61f640 1 --2- 172.21.15.16:0/3893030505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff334003f20 0x7ff334004800 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff338009ac0 tx=0x7ff334005830 comp rx=0 tx=0).stop 2024-04-25T10:06:56.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.042+0000 7ff34cbbc640 1 -- 172.21.15.16:0/3893030505 reap_dead start 2024-04-25T10:06:56.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.042+0000 7ff34e61f640 1 -- 172.21.15.16:0/3893030505 shutdown_connections 2024-04-25T10:06:56.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.042+0000 7ff34e61f640 1 -- 172.21.15.16:0/3893030505 >> 172.21.15.16:0/3893030505 conn(0x7ff348072900 msgr2=0x7ff3480d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.042+0000 7ff34e61f640 1 -- 172.21.15.16:0/3893030505 shutdown_connections 2024-04-25T10:06:56.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.042+0000 7ff34e61f640 1 -- 172.21.15.16:0/3893030505 wait complete. 2024-04-25T10:06:56.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.043+0000 7ff34e61f640 1 Processor -- start 2024-04-25T10:06:56.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.043+0000 7ff34e61f640 1 -- start start 2024-04-25T10:06:56.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.044+0000 7ff34e61f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff334003f20 0x7ff34815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.044+0000 7ff34e61f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff348152180 0x7ff348160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.044+0000 7ff34e61f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff34815f9f0 0x7ff348160560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.044+0000 7ff34e61f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3480d3090 con 0x7ff334003f20 2024-04-25T10:06:56.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.044+0000 7ff34e61f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3480d2db0 con 0x7ff34815f9f0 2024-04-25T10:06:56.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.044+0000 7ff34e61f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3480d2160 con 0x7ff348152180 2024-04-25T10:06:56.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.044+0000 7ff347fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff334003f20 0x7ff34815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.044+0000 7ff347fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff334003f20 0x7ff34815f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50514/0 (socket says 172.21.15.16:50514) 2024-04-25T10:06:56.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.044+0000 7ff347fff640 1 -- 172.21.15.16:0/3675500050 learned_addr learned my addr 172.21.15.16:0/3675500050 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:56.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.044+0000 7ff34cbbc640 1 --2- 172.21.15.16:0/3675500050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff34815f9f0 0x7ff348160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.044+0000 7ff3477fe640 1 --2- 172.21.15.16:0/3675500050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff348152180 0x7ff348160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.044+0000 7ff347fff640 1 -- 172.21.15.16:0/3675500050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff348152180 msgr2=0x7ff348160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.044+0000 7ff347fff640 1 --2- 172.21.15.16:0/3675500050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff348152180 0x7ff348160020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.044+0000 7ff347fff640 1 -- 172.21.15.16:0/3675500050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff34815f9f0 msgr2=0x7ff348160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.044+0000 7ff347fff640 1 --2- 172.21.15.16:0/3675500050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff34815f9f0 0x7ff348160560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.044+0000 7ff347fff640 1 -- 172.21.15.16:0/3675500050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3481519b0 con 0x7ff334003f20 2024-04-25T10:06:56.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.045+0000 7ff3477fe640 1 --2- 172.21.15.16:0/3675500050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff348152180 0x7ff348160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:56.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.045+0000 7ff347fff640 1 --2- 172.21.15.16:0/3675500050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff334003f20 0x7ff34815f4b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff33000e7d0 tx=0x7ff33000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.045+0000 7ff344ff9640 1 -- 172.21.15.16:0/3675500050 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33000ee80 con 0x7ff334003f20 2024-04-25T10:06:56.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.045+0000 7ff344ff9640 1 -- 172.21.15.16:0/3675500050 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff330018070 con 0x7ff334003f20 2024-04-25T10:06:56.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.045+0000 7ff34e61f640 1 -- 172.21.15.16:0/3675500050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34815e430 con 0x7ff334003f20 2024-04-25T10:06:56.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.045+0000 7ff344ff9640 1 -- 172.21.15.16:0/3675500050 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33000c660 con 0x7ff334003f20 2024-04-25T10:06:56.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.045+0000 7ff34e61f640 1 -- 172.21.15.16:0/3675500050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34815e9d0 con 0x7ff334003f20 2024-04-25T10:06:56.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.046+0000 7ff34e61f640 1 -- 172.21.15.16:0/3675500050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3480749b0 con 0x7ff334003f20 2024-04-25T10:06:56.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.046+0000 7ff344ff9640 1 -- 172.21.15.16:0/3675500050 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff33000c800 con 0x7ff334003f20 2024-04-25T10:06:56.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.047+0000 7ff344ff9640 1 --2- 172.21.15.16:0/3675500050 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff32806b500 0x7ff32806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.047+0000 7ff344ff9640 1 -- 172.21.15.16:0/3675500050 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff330089fc0 con 0x7ff334003f20 2024-04-25T10:06:56.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.047+0000 7ff3477fe640 1 --2- 172.21.15.16:0/3675500050 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff32806b500 0x7ff32806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.048+0000 7ff3477fe640 1 --2- 172.21.15.16:0/3675500050 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff32806b500 0x7ff32806d9c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff338033000 tx=0x7ff338008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.051+0000 7ff344ff9640 1 -- 172.21.15.16:0/3675500050 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3300587f0 con 0x7ff334003f20 2024-04-25T10:06:56.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.229+0000 7ff34e61f640 1 -- 172.21.15.16:0/3675500050 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff34806b080 con 0x7ff32806b500 2024-04-25T10:06:56.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.236+0000 7ff344ff9640 1 -- 172.21.15.16:0/3675500050 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251905 (secure 0 0 0) 0x7ff34806b080 con 0x7ff32806b500 2024-04-25T10:06:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:06:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.239+0000 7ff34e61f640 1 -- 172.21.15.16:0/3675500050 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff32806b500 msgr2=0x7ff32806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.239+0000 7ff34e61f640 1 --2- 172.21.15.16:0/3675500050 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff32806b500 0x7ff32806d9c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff338033000 tx=0x7ff338008030 comp rx=0 tx=0).stop 2024-04-25T10:06:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.239+0000 7ff34e61f640 1 -- 172.21.15.16:0/3675500050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff334003f20 msgr2=0x7ff34815f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.239+0000 7ff34e61f640 1 --2- 172.21.15.16:0/3675500050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff334003f20 0x7ff34815f4b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff33000e7d0 tx=0x7ff33000ec90 comp rx=0 tx=0).stop 2024-04-25T10:06:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.239+0000 7ff34e61f640 1 -- 172.21.15.16:0/3675500050 shutdown_connections 2024-04-25T10:06:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.239+0000 7ff34e61f640 1 --2- 172.21.15.16:0/3675500050 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff32806b500 0x7ff32806d9c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.239+0000 7ff34e61f640 1 --2- 172.21.15.16:0/3675500050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff34815f9f0 0x7ff348160560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.239+0000 7ff34e61f640 1 --2- 172.21.15.16:0/3675500050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff348152180 0x7ff348160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.239+0000 7ff34e61f640 1 --2- 172.21.15.16:0/3675500050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff334003f20 0x7ff34815f4b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.239+0000 7ff34e61f640 1 -- 172.21.15.16:0/3675500050 >> 172.21.15.16:0/3675500050 conn(0x7ff348072900 msgr2=0x7ff3480b6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.239+0000 7ff34e61f640 1 -- 172.21.15.16:0/3675500050 shutdown_connections 2024-04-25T10:06:56.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:56.239+0000 7ff34e61f640 1 -- 172.21.15.16:0/3675500050 wait complete. 2024-04-25T10:06:56.256 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-04-25T10:06:55.804736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":102,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242336,"kb_used_data":5348,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732384,"statfs":{"total":773094113280,"available":772845961216,"internally_reserved":0,"allocated":5476352,"data_stored":2888248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001156"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.840324+0000","last_change":"2024-04-25T10:06:51.837844+0000","last_active":"2024-04-25T10:06:51.840324+0000","last_peered":"2024-04-25T10:06:51.840324+0000","last_clean":"2024-04-25T10:06:51.840324+0000","last_became_active":"2024-04-25T10:06:51.837567+0000","last_became_peered":"2024-04-25T10:06:51.837567+0000","last_unstale":"2024-04-25T10:06:51.840324+0000","last_undegraded":"2024-04-25T10:06:51.840324+0000","last_fullsized":"2024-04-25T10:06:51.840324+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:03.830929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847252+0000","last_change":"2024-04-25T10:06:51.844389+0000","last_active":"2024-04-25T10:06:51.847252+0000","last_peered":"2024-04-25T10:06:51.847252+0000","last_clean":"2024-04-25T10:06:51.847252+0000","last_became_active":"2024-04-25T10:06:51.844082+0000","last_became_peered":"2024-04-25T10:06:51.844082+0000","last_unstale":"2024-04-25T10:06:51.847252+0000","last_undegraded":"2024-04-25T10:06:51.847252+0000","last_fullsized":"2024-04-25T10:06:51.847252+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:10.566374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847906+0000","last_change":"2024-04-25T10:06:51.845292+0000","last_active":"2024-04-25T10:06:51.847906+0000","last_peered":"2024-04-25T10:06:51.847906+0000","last_clean":"2024-04-25T10:06:51.847906+0000","last_became_active":"2024-04-25T10:06:51.844790+0000","last_became_peered":"2024-04-25T10:06:51.844790+0000","last_unstale":"2024-04-25T10:06:51.847906+0000","last_undegraded":"2024-04-25T10:06:51.847906+0000","last_fullsized":"2024-04-25T10:06:51.847906+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:51.599603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839525+0000","last_change":"2024-04-25T10:06:51.829871+0000","last_active":"2024-04-25T10:06:52.839525+0000","last_peered":"2024-04-25T10:06:52.839525+0000","last_clean":"2024-04-25T10:06:52.839525+0000","last_became_active":"2024-04-25T10:06:51.829563+0000","last_became_peered":"2024-04-25T10:06:51.829563+0000","last_unstale":"2024-04-25T10:06:52.839525+0000","last_undegraded":"2024-04-25T10:06:52.839525+0000","last_fullsized":"2024-04-25T10:06:52.839525+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:36.712935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841435+0000","last_change":"2024-04-25T10:06:51.840200+0000","last_active":"2024-04-25T10:06:51.841435+0000","last_peered":"2024-04-25T10:06:51.841435+0000","last_clean":"2024-04-25T10:06:51.841435+0000","last_became_active":"2024-04-25T10:06:51.839897+0000","last_became_peered":"2024-04-25T10:06:51.839897+0000","last_unstale":"2024-04-25T10:06:51.841435+0000","last_undegraded":"2024-04-25T10:06:51.841435+0000","last_fullsized":"2024-04-25T10:06:51.841435+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:23.657828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842607+0000","last_change":"2024-04-25T10:06:51.836510+0000","last_active":"2024-04-25T10:06:51.842607+0000","last_peered":"2024-04-25T10:06:51.842607+0000","last_clean":"2024-04-25T10:06:51.842607+0000","last_became_active":"2024-04-25T10:06:51.836244+0000","last_became_peered":"2024-04-25T10:06:51.836244+0000","last_unstale":"2024-04-25T10:06:51.842607+0000","last_undegraded":"2024-04-25T10:06:51.842607+0000","last_fullsized":"2024-04-25T10:06:51.842607+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:20.690035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841155+0000","last_change":"2024-04-25T10:06:51.837947+0000","last_active":"2024-04-25T10:06:51.841155+0000","last_peered":"2024-04-25T10:06:51.841155+0000","last_clean":"2024-04-25T10:06:51.841155+0000","last_became_active":"2024-04-25T10:06:51.837635+0000","last_became_peered":"2024-04-25T10:06:51.837635+0000","last_unstale":"2024-04-25T10:06:51.841155+0000","last_undegraded":"2024-04-25T10:06:51.841155+0000","last_fullsized":"2024-04-25T10:06:51.841155+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:47.682171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839202+0000","last_change":"2024-04-25T10:06:51.836550+0000","last_active":"2024-04-25T10:06:52.839202+0000","last_peered":"2024-04-25T10:06:52.839202+0000","last_clean":"2024-04-25T10:06:52.839202+0000","last_became_active":"2024-04-25T10:06:51.836157+0000","last_became_peered":"2024-04-25T10:06:51.836157+0000","last_unstale":"2024-04-25T10:06:52.839202+0000","last_undegraded":"2024-04-25T10:06:52.839202+0000","last_fullsized":"2024-04-25T10:06:52.839202+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:17.748860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854141+0000","last_change":"2024-04-25T10:06:51.847625+0000","last_active":"2024-04-25T10:06:51.854141+0000","last_peered":"2024-04-25T10:06:51.854141+0000","last_clean":"2024-04-25T10:06:51.854141+0000","last_became_active":"2024-04-25T10:06:51.847179+0000","last_became_peered":"2024-04-25T10:06:51.847179+0000","last_unstale":"2024-04-25T10:06:51.854141+0000","last_undegraded":"2024-04-25T10:06:51.854141+0000","last_fullsized":"2024-04-25T10:06:51.854141+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:40.042571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841895+0000","last_change":"2024-04-25T10:06:51.837753+0000","last_active":"2024-04-25T10:06:51.841895+0000","last_peered":"2024-04-25T10:06:51.841895+0000","last_clean":"2024-04-25T10:06:51.841895+0000","last_became_active":"2024-04-25T10:06:51.837418+0000","last_became_peered":"2024-04-25T10:06:51.837418+0000","last_unstale":"2024-04-25T10:06:51.841895+0000","last_undegraded":"2024-04-25T10:06:51.841895+0000","last_fullsized":"2024-04-25T10:06:51.841895+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:29.774279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839652+0000","last_change":"2024-04-25T10:06:51.832514+0000","last_active":"2024-04-25T10:06:52.839652+0000","last_peered":"2024-04-25T10:06:52.839652+0000","last_clean":"2024-04-25T10:06:52.839652+0000","last_became_active":"2024-04-25T10:06:51.832253+0000","last_became_peered":"2024-04-25T10:06:51.832253+0000","last_unstale":"2024-04-25T10:06:52.839652+0000","last_undegraded":"2024-04-25T10:06:52.839652+0000","last_fullsized":"2024-04-25T10:06:52.839652+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:24.885518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845933+0000","last_change":"2024-04-25T10:06:51.843573+0000","last_active":"2024-04-25T10:06:51.845933+0000","last_peered":"2024-04-25T10:06:51.845933+0000","last_clean":"2024-04-25T10:06:51.845933+0000","last_became_active":"2024-04-25T10:06:51.843141+0000","last_became_peered":"2024-04-25T10:06:51.843141+0000","last_unstale":"2024-04-25T10:06:51.845933+0000","last_undegraded":"2024-04-25T10:06:51.845933+0000","last_fullsized":"2024-04-25T10:06:51.845933+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:14.554682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842933+0000","last_change":"2024-04-25T10:06:51.838802+0000","last_active":"2024-04-25T10:06:51.842933+0000","last_peered":"2024-04-25T10:06:51.842933+0000","last_clean":"2024-04-25T10:06:51.842933+0000","last_became_active":"2024-04-25T10:06:51.837871+0000","last_became_peered":"2024-04-25T10:06:51.837871+0000","last_unstale":"2024-04-25T10:06:51.842933+0000","last_undegraded":"2024-04-25T10:06:51.842933+0000","last_fullsized":"2024-04-25T10:06:51.842933+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:50.203747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839394+0000","last_change":"2024-04-25T10:06:51.845427+0000","last_active":"2024-04-25T10:06:52.839394+0000","last_peered":"2024-04-25T10:06:52.839394+0000","last_clean":"2024-04-25T10:06:52.839394+0000","last_became_active":"2024-04-25T10:06:51.845199+0000","last_became_peered":"2024-04-25T10:06:51.845199+0000","last_unstale":"2024-04-25T10:06:52.839394+0000","last_undegraded":"2024-04-25T10:06:52.839394+0000","last_fullsized":"2024-04-25T10:06:52.839394+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:50.203646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839222+0000","last_change":"2024-04-25T10:06:51.841154+0000","last_active":"2024-04-25T10:06:52.839222+0000","last_peered":"2024-04-25T10:06:52.839222+0000","last_clean":"2024-04-25T10:06:52.839222+0000","last_became_active":"2024-04-25T10:06:51.840810+0000","last_became_peered":"2024-04-25T10:06:51.840810+0000","last_unstale":"2024-04-25T10:06:52.839222+0000","last_undegraded":"2024-04-25T10:06:52.839222+0000","last_fullsized":"2024-04-25T10:06:52.839222+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:50.065157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.844998+0000","last_change":"2024-04-25T10:06:51.842491+0000","last_active":"2024-04-25T10:06:51.844998+0000","last_peered":"2024-04-25T10:06:51.844998+0000","last_clean":"2024-04-25T10:06:51.844998+0000","last_became_active":"2024-04-25T10:06:51.842212+0000","last_became_peered":"2024-04-25T10:06:51.842212+0000","last_unstale":"2024-04-25T10:06:51.844998+0000","last_undegraded":"2024-04-25T10:06:51.844998+0000","last_fullsized":"2024-04-25T10:06:51.844998+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:04.628909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.838191+0000","last_change":"2024-04-25T10:06:51.836207+0000","last_active":"2024-04-25T10:06:51.838191+0000","last_peered":"2024-04-25T10:06:51.838191+0000","last_clean":"2024-04-25T10:06:51.838191+0000","last_became_active":"2024-04-25T10:06:51.835856+0000","last_became_peered":"2024-04-25T10:06:51.835856+0000","last_unstale":"2024-04-25T10:06:51.838191+0000","last_undegraded":"2024-04-25T10:06:51.838191+0000","last_fullsized":"2024-04-25T10:06:51.838191+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:20.690035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.839761+0000","last_change":"2024-04-25T10:06:51.835570+0000","last_active":"2024-04-25T10:06:51.839761+0000","last_peered":"2024-04-25T10:06:51.839761+0000","last_clean":"2024-04-25T10:06:51.839761+0000","last_became_active":"2024-04-25T10:06:51.835287+0000","last_became_peered":"2024-04-25T10:06:51.835287+0000","last_unstale":"2024-04-25T10:06:51.839761+0000","last_undegraded":"2024-04-25T10:06:51.839761+0000","last_fullsized":"2024-04-25T10:06:51.839761+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:58.880955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846553+0000","last_change":"2024-04-25T10:06:51.845653+0000","last_active":"2024-04-25T10:06:51.846553+0000","last_peered":"2024-04-25T10:06:51.846553+0000","last_clean":"2024-04-25T10:06:51.846553+0000","last_became_active":"2024-04-25T10:06:51.845387+0000","last_became_peered":"2024-04-25T10:06:51.845387+0000","last_unstale":"2024-04-25T10:06:51.846553+0000","last_undegraded":"2024-04-25T10:06:51.846553+0000","last_fullsized":"2024-04-25T10:06:51.846553+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:22.306267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841737+0000","last_change":"2024-04-25T10:06:51.836879+0000","last_active":"2024-04-25T10:06:51.841737+0000","last_peered":"2024-04-25T10:06:51.841737+0000","last_clean":"2024-04-25T10:06:51.841737+0000","last_became_active":"2024-04-25T10:06:51.835980+0000","last_became_peered":"2024-04-25T10:06:51.835980+0000","last_unstale":"2024-04-25T10:06:51.841737+0000","last_undegraded":"2024-04-25T10:06:51.841737+0000","last_fullsized":"2024-04-25T10:06:51.841737+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:30.749971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839389+0000","last_change":"2024-04-25T10:06:51.826348+0000","last_active":"2024-04-25T10:06:52.839389+0000","last_peered":"2024-04-25T10:06:52.839389+0000","last_clean":"2024-04-25T10:06:52.839389+0000","last_became_active":"2024-04-25T10:06:51.826040+0000","last_became_peered":"2024-04-25T10:06:51.826040+0000","last_unstale":"2024-04-25T10:06:52.839389+0000","last_undegraded":"2024-04-25T10:06:52.839389+0000","last_fullsized":"2024-04-25T10:06:52.839389+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:30.205631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841910+0000","last_change":"2024-04-25T10:06:51.838280+0000","last_active":"2024-04-25T10:06:51.841910+0000","last_peered":"2024-04-25T10:06:51.841910+0000","last_clean":"2024-04-25T10:06:51.841910+0000","last_became_active":"2024-04-25T10:06:51.837813+0000","last_became_peered":"2024-04-25T10:06:51.837813+0000","last_unstale":"2024-04-25T10:06:51.841910+0000","last_undegraded":"2024-04-25T10:06:51.841910+0000","last_fullsized":"2024-04-25T10:06:51.841910+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:58.880955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.833222+0000","last_change":"2024-04-25T10:06:51.831343+0000","last_active":"2024-04-25T10:06:51.833222+0000","last_peered":"2024-04-25T10:06:51.833222+0000","last_clean":"2024-04-25T10:06:51.833222+0000","last_became_active":"2024-04-25T10:06:51.831040+0000","last_became_peered":"2024-04-25T10:06:51.831040+0000","last_unstale":"2024-04-25T10:06:51.833222+0000","last_undegraded":"2024-04-25T10:06:51.833222+0000","last_fullsized":"2024-04-25T10:06:51.833222+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:04.569426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.832424+0000","last_change":"2024-04-25T10:06:51.827202+0000","last_active":"2024-04-25T10:06:51.832424+0000","last_peered":"2024-04-25T10:06:51.832424+0000","last_clean":"2024-04-25T10:06:51.832424+0000","last_became_active":"2024-04-25T10:06:51.826893+0000","last_became_peered":"2024-04-25T10:06:51.826893+0000","last_unstale":"2024-04-25T10:06:51.832424+0000","last_undegraded":"2024-04-25T10:06:51.832424+0000","last_fullsized":"2024-04-25T10:06:51.832424+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:25.546409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.836212+0000","last_change":"2024-04-25T10:06:51.835353+0000","last_active":"2024-04-25T10:06:51.836212+0000","last_peered":"2024-04-25T10:06:51.836212+0000","last_clean":"2024-04-25T10:06:51.836212+0000","last_became_active":"2024-04-25T10:06:51.834941+0000","last_became_peered":"2024-04-25T10:06:51.834941+0000","last_unstale":"2024-04-25T10:06:51.836212+0000","last_undegraded":"2024-04-25T10:06:51.836212+0000","last_fullsized":"2024-04-25T10:06:51.836212+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:48.121882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839669+0000","last_change":"2024-04-25T10:06:51.837299+0000","last_active":"2024-04-25T10:06:52.839669+0000","last_peered":"2024-04-25T10:06:52.839669+0000","last_clean":"2024-04-25T10:06:52.839669+0000","last_became_active":"2024-04-25T10:06:51.836912+0000","last_became_peered":"2024-04-25T10:06:51.836912+0000","last_unstale":"2024-04-25T10:06:52.839669+0000","last_undegraded":"2024-04-25T10:06:52.839669+0000","last_fullsized":"2024-04-25T10:06:52.839669+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:19.276862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.833241+0000","last_change":"2024-04-25T10:06:51.830316+0000","last_active":"2024-04-25T10:06:51.833241+0000","last_peered":"2024-04-25T10:06:51.833241+0000","last_clean":"2024-04-25T10:06:51.833241+0000","last_became_active":"2024-04-25T10:06:51.830039+0000","last_became_peered":"2024-04-25T10:06:51.830039+0000","last_unstale":"2024-04-25T10:06:51.833241+0000","last_undegraded":"2024-04-25T10:06:51.833241+0000","last_fullsized":"2024-04-25T10:06:51.833241+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:01.498621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.833261+0000","last_change":"2024-04-25T10:06:51.830470+0000","last_active":"2024-04-25T10:06:51.833261+0000","last_peered":"2024-04-25T10:06:51.833261+0000","last_clean":"2024-04-25T10:06:51.833261+0000","last_became_active":"2024-04-25T10:06:51.830180+0000","last_became_peered":"2024-04-25T10:06:51.830180+0000","last_unstale":"2024-04-25T10:06:51.833261+0000","last_undegraded":"2024-04-25T10:06:51.833261+0000","last_fullsized":"2024-04-25T10:06:51.833261+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:17.127193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.840946+0000","last_change":"2024-04-25T10:06:51.839558+0000","last_active":"2024-04-25T10:06:51.840946+0000","last_peered":"2024-04-25T10:06:51.840946+0000","last_clean":"2024-04-25T10:06:51.840946+0000","last_became_active":"2024-04-25T10:06:51.839271+0000","last_became_peered":"2024-04-25T10:06:51.839271+0000","last_unstale":"2024-04-25T10:06:51.840946+0000","last_undegraded":"2024-04-25T10:06:51.840946+0000","last_fullsized":"2024-04-25T10:06:51.840946+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:11.162077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.832516+0000","last_change":"2024-04-25T10:06:51.828040+0000","last_active":"2024-04-25T10:06:51.832516+0000","last_peered":"2024-04-25T10:06:51.832516+0000","last_clean":"2024-04-25T10:06:51.832516+0000","last_became_active":"2024-04-25T10:06:51.826090+0000","last_became_peered":"2024-04-25T10:06:51.826090+0000","last_unstale":"2024-04-25T10:06:51.832516+0000","last_undegraded":"2024-04-25T10:06:51.832516+0000","last_fullsized":"2024-04-25T10:06:51.832516+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:09.382894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842444+0000","last_change":"2024-04-25T10:06:51.837883+0000","last_active":"2024-04-25T10:06:51.842444+0000","last_peered":"2024-04-25T10:06:51.842444+0000","last_clean":"2024-04-25T10:06:51.842444+0000","last_became_active":"2024-04-25T10:06:51.837631+0000","last_became_peered":"2024-04-25T10:06:51.837631+0000","last_unstale":"2024-04-25T10:06:51.842444+0000","last_undegraded":"2024-04-25T10:06:51.842444+0000","last_fullsized":"2024-04-25T10:06:51.842444+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:14.554682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839229+0000","last_change":"2024-04-25T10:06:51.838274+0000","last_active":"2024-04-25T10:06:52.839229+0000","last_peered":"2024-04-25T10:06:52.839229+0000","last_clean":"2024-04-25T10:06:52.839229+0000","last_became_active":"2024-04-25T10:06:51.838016+0000","last_became_peered":"2024-04-25T10:06:51.838016+0000","last_unstale":"2024-04-25T10:06:52.839229+0000","last_undegraded":"2024-04-25T10:06:52.839229+0000","last_fullsized":"2024-04-25T10:06:52.839229+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:47.853571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846377+0000","last_change":"2024-04-25T10:06:51.842565+0000","last_active":"2024-04-25T10:06:51.846377+0000","last_peered":"2024-04-25T10:06:51.846377+0000","last_clean":"2024-04-25T10:06:51.846377+0000","last_became_active":"2024-04-25T10:06:51.842251+0000","last_became_peered":"2024-04-25T10:06:51.842251+0000","last_unstale":"2024-04-25T10:06:51.846377+0000","last_undegraded":"2024-04-25T10:06:51.846377+0000","last_fullsized":"2024-04-25T10:06:51.846377+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:32.311454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854076+0000","last_change":"2024-04-25T10:06:50.815053+0000","last_active":"2024-04-25T10:06:51.854076+0000","last_peered":"2024-04-25T10:06:51.854076+0000","last_clean":"2024-04-25T10:06:51.854076+0000","last_became_active":"2024-04-25T10:06:50.814623+0000","last_became_peered":"2024-04-25T10:06:50.814623+0000","last_unstale":"2024-04-25T10:06:51.854076+0000","last_undegraded":"2024-04-25T10:06:51.854076+0000","last_fullsized":"2024-04-25T10:06:51.854076+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:16.218950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846459+0000","last_change":"2024-04-25T10:06:51.844956+0000","last_active":"2024-04-25T10:06:51.846459+0000","last_peered":"2024-04-25T10:06:51.846459+0000","last_clean":"2024-04-25T10:06:51.846459+0000","last_became_active":"2024-04-25T10:06:51.844611+0000","last_became_peered":"2024-04-25T10:06:51.844611+0000","last_unstale":"2024-04-25T10:06:51.846459+0000","last_undegraded":"2024-04-25T10:06:51.846459+0000","last_fullsized":"2024-04-25T10:06:51.846459+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:50.170873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.848558+0000","last_change":"2024-04-25T10:06:50.822263+0000","last_active":"2024-04-25T10:06:51.848558+0000","last_peered":"2024-04-25T10:06:51.848558+0000","last_clean":"2024-04-25T10:06:51.848558+0000","last_became_active":"2024-04-25T10:06:50.821985+0000","last_became_peered":"2024-04-25T10:06:50.821985+0000","last_unstale":"2024-04-25T10:06:51.848558+0000","last_undegraded":"2024-04-25T10:06:51.848558+0000","last_fullsized":"2024-04-25T10:06:51.848558+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:37.641598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847288+0000","last_change":"2024-04-25T10:06:51.844989+0000","last_active":"2024-04-25T10:06:51.847288+0000","last_peered":"2024-04-25T10:06:51.847288+0000","last_clean":"2024-04-25T10:06:51.847288+0000","last_became_active":"2024-04-25T10:06:51.844614+0000","last_became_peered":"2024-04-25T10:06:51.844614+0000","last_unstale":"2024-04-25T10:06:51.847288+0000","last_undegraded":"2024-04-25T10:06:51.847288+0000","last_fullsized":"2024-04-25T10:06:51.847288+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:50.203747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.853873+0000","last_change":"2024-04-25T10:06:50.818405+0000","last_active":"2024-04-25T10:06:51.853873+0000","last_peered":"2024-04-25T10:06:51.853873+0000","last_clean":"2024-04-25T10:06:51.853873+0000","last_became_active":"2024-04-25T10:06:50.818118+0000","last_became_peered":"2024-04-25T10:06:50.818118+0000","last_unstale":"2024-04-25T10:06:51.853873+0000","last_undegraded":"2024-04-25T10:06:51.853873+0000","last_fullsized":"2024-04-25T10:06:51.853873+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:08.381421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839693+0000","last_change":"2024-04-25T10:06:51.832398+0000","last_active":"2024-04-25T10:06:52.839693+0000","last_peered":"2024-04-25T10:06:52.839693+0000","last_clean":"2024-04-25T10:06:52.839693+0000","last_became_active":"2024-04-25T10:06:51.832120+0000","last_became_peered":"2024-04-25T10:06:51.832120+0000","last_unstale":"2024-04-25T10:06:52.839693+0000","last_undegraded":"2024-04-25T10:06:52.839693+0000","last_fullsized":"2024-04-25T10:06:52.839693+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:08.698252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.848517+0000","last_change":"2024-04-25T10:06:50.821564+0000","last_active":"2024-04-25T10:06:51.848517+0000","last_peered":"2024-04-25T10:06:51.848517+0000","last_clean":"2024-04-25T10:06:51.848517+0000","last_became_active":"2024-04-25T10:06:50.821257+0000","last_became_peered":"2024-04-25T10:06:50.821257+0000","last_unstale":"2024-04-25T10:06:51.848517+0000","last_undegraded":"2024-04-25T10:06:51.848517+0000","last_fullsized":"2024-04-25T10:06:51.848517+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:25.919994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834314+0000","last_change":"2024-04-25T10:06:51.832079+0000","last_active":"2024-04-25T10:06:51.834314+0000","last_peered":"2024-04-25T10:06:51.834314+0000","last_clean":"2024-04-25T10:06:51.834314+0000","last_became_active":"2024-04-25T10:06:51.831785+0000","last_became_peered":"2024-04-25T10:06:51.831785+0000","last_unstale":"2024-04-25T10:06:51.834314+0000","last_undegraded":"2024-04-25T10:06:51.834314+0000","last_fullsized":"2024-04-25T10:06:51.834314+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:27.917421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.853752+0000","last_change":"2024-04-25T10:06:50.819361+0000","last_active":"2024-04-25T10:06:51.853752+0000","last_peered":"2024-04-25T10:06:51.853752+0000","last_clean":"2024-04-25T10:06:51.853752+0000","last_became_active":"2024-04-25T10:06:50.819093+0000","last_became_peered":"2024-04-25T10:06:50.819093+0000","last_unstale":"2024-04-25T10:06:51.853752+0000","last_undegraded":"2024-04-25T10:06:51.853752+0000","last_fullsized":"2024-04-25T10:06:51.853752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:24.544937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841867+0000","last_change":"2024-04-25T10:06:51.835101+0000","last_active":"2024-04-25T10:06:51.841867+0000","last_peered":"2024-04-25T10:06:51.841867+0000","last_clean":"2024-04-25T10:06:51.841867+0000","last_became_active":"2024-04-25T10:06:51.834291+0000","last_became_peered":"2024-04-25T10:06:51.834291+0000","last_unstale":"2024-04-25T10:06:51.841867+0000","last_undegraded":"2024-04-25T10:06:51.841867+0000","last_fullsized":"2024-04-25T10:06:51.841867+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:22.306267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845318+0000","last_change":"2024-04-25T10:06:50.822762+0000","last_active":"2024-04-25T10:06:51.845318+0000","last_peered":"2024-04-25T10:06:51.845318+0000","last_clean":"2024-04-25T10:06:51.845318+0000","last_became_active":"2024-04-25T10:06:50.822463+0000","last_became_peered":"2024-04-25T10:06:50.822463+0000","last_unstale":"2024-04-25T10:06:51.845318+0000","last_undegraded":"2024-04-25T10:06:51.845318+0000","last_fullsized":"2024-04-25T10:06:51.845318+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:18.363517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841750+0000","last_change":"2024-04-25T10:06:51.833997+0000","last_active":"2024-04-25T10:06:51.841750+0000","last_peered":"2024-04-25T10:06:51.841750+0000","last_clean":"2024-04-25T10:06:51.841750+0000","last_became_active":"2024-04-25T10:06:51.833694+0000","last_became_peered":"2024-04-25T10:06:51.833694+0000","last_unstale":"2024-04-25T10:06:51.841750+0000","last_undegraded":"2024-04-25T10:06:51.841750+0000","last_fullsized":"2024-04-25T10:06:51.841750+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:50.170873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828476+0000","last_change":"2024-04-25T10:06:50.818945+0000","last_active":"2024-04-25T10:06:51.828476+0000","last_peered":"2024-04-25T10:06:51.828476+0000","last_clean":"2024-04-25T10:06:51.828476+0000","last_became_active":"2024-04-25T10:06:50.818631+0000","last_became_peered":"2024-04-25T10:06:50.818631+0000","last_unstale":"2024-04-25T10:06:51.828476+0000","last_undegraded":"2024-04-25T10:06:51.828476+0000","last_fullsized":"2024-04-25T10:06:51.828476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:49.202275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.839815+0000","last_change":"2024-04-25T10:06:51.837313+0000","last_active":"2024-04-25T10:06:51.839815+0000","last_peered":"2024-04-25T10:06:51.839815+0000","last_clean":"2024-04-25T10:06:51.839815+0000","last_became_active":"2024-04-25T10:06:51.837024+0000","last_became_peered":"2024-04-25T10:06:51.837024+0000","last_unstale":"2024-04-25T10:06:51.839815+0000","last_undegraded":"2024-04-25T10:06:51.839815+0000","last_fullsized":"2024-04-25T10:06:51.839815+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:28.322485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.825554+0000","last_change":"2024-04-25T10:06:50.818743+0000","last_active":"2024-04-25T10:06:51.825554+0000","last_peered":"2024-04-25T10:06:51.825554+0000","last_clean":"2024-04-25T10:06:51.825554+0000","last_became_active":"2024-04-25T10:06:50.818359+0000","last_became_peered":"2024-04-25T10:06:50.818359+0000","last_unstale":"2024-04-25T10:06:51.825554+0000","last_undegraded":"2024-04-25T10:06:51.825554+0000","last_fullsized":"2024-04-25T10:06:51.825554+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:47.120410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.848307+0000","last_change":"2024-04-25T10:06:51.846840+0000","last_active":"2024-04-25T10:06:51.848307+0000","last_peered":"2024-04-25T10:06:51.848307+0000","last_clean":"2024-04-25T10:06:51.848307+0000","last_became_active":"2024-04-25T10:06:51.846314+0000","last_became_peered":"2024-04-25T10:06:51.846314+0000","last_unstale":"2024-04-25T10:06:51.848307+0000","last_undegraded":"2024-04-25T10:06:51.848307+0000","last_fullsized":"2024-04-25T10:06:51.848307+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:29.810440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.823148+0000","last_change":"2024-04-25T10:06:50.819769+0000","last_active":"2024-04-25T10:06:51.823148+0000","last_peered":"2024-04-25T10:06:51.823148+0000","last_clean":"2024-04-25T10:06:51.823148+0000","last_became_active":"2024-04-25T10:06:50.819517+0000","last_became_peered":"2024-04-25T10:06:50.819517+0000","last_unstale":"2024-04-25T10:06:51.823148+0000","last_undegraded":"2024-04-25T10:06:51.823148+0000","last_fullsized":"2024-04-25T10:06:51.823148+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:15.288040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834870+0000","last_change":"2024-04-25T10:06:51.832027+0000","last_active":"2024-04-25T10:06:51.834870+0000","last_peered":"2024-04-25T10:06:51.834870+0000","last_clean":"2024-04-25T10:06:51.834870+0000","last_became_active":"2024-04-25T10:06:51.831718+0000","last_became_peered":"2024-04-25T10:06:51.831718+0000","last_unstale":"2024-04-25T10:06:51.834870+0000","last_undegraded":"2024-04-25T10:06:51.834870+0000","last_fullsized":"2024-04-25T10:06:51.834870+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:30.250121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.825643+0000","last_change":"2024-04-25T10:06:50.825390+0000","last_active":"2024-04-25T10:06:51.825643+0000","last_peered":"2024-04-25T10:06:51.825643+0000","last_clean":"2024-04-25T10:06:51.825643+0000","last_became_active":"2024-04-25T10:06:50.825085+0000","last_became_peered":"2024-04-25T10:06:50.825085+0000","last_unstale":"2024-04-25T10:06:51.825643+0000","last_undegraded":"2024-04-25T10:06:51.825643+0000","last_fullsized":"2024-04-25T10:06:51.825643+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:10.160604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839524+0000","last_change":"2024-04-25T10:06:51.845690+0000","last_active":"2024-04-25T10:06:52.839524+0000","last_peered":"2024-04-25T10:06:52.839524+0000","last_clean":"2024-04-25T10:06:52.839524+0000","last_became_active":"2024-04-25T10:06:51.845262+0000","last_became_peered":"2024-04-25T10:06:51.845262+0000","last_unstale":"2024-04-25T10:06:52.839524+0000","last_undegraded":"2024-04-25T10:06:52.839524+0000","last_fullsized":"2024-04-25T10:06:52.839524+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:20.903422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845381+0000","last_change":"2024-04-25T10:06:50.818510+0000","last_active":"2024-04-25T10:06:51.845381+0000","last_peered":"2024-04-25T10:06:51.845381+0000","last_clean":"2024-04-25T10:06:51.845381+0000","last_became_active":"2024-04-25T10:06:50.818217+0000","last_became_peered":"2024-04-25T10:06:50.818217+0000","last_unstale":"2024-04-25T10:06:51.845381+0000","last_undegraded":"2024-04-25T10:06:51.845381+0000","last_fullsized":"2024-04-25T10:06:51.845381+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.521754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841431+0000","last_change":"2024-04-25T10:06:51.838484+0000","last_active":"2024-04-25T10:06:51.841431+0000","last_peered":"2024-04-25T10:06:51.841431+0000","last_clean":"2024-04-25T10:06:51.841431+0000","last_became_active":"2024-04-25T10:06:51.838211+0000","last_became_peered":"2024-04-25T10:06:51.838211+0000","last_unstale":"2024-04-25T10:06:51.841431+0000","last_undegraded":"2024-04-25T10:06:51.841431+0000","last_fullsized":"2024-04-25T10:06:51.841431+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:46.417710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.856308+0000","last_change":"2024-04-25T10:06:50.823945+0000","last_active":"2024-04-25T10:06:51.856308+0000","last_peered":"2024-04-25T10:06:51.856308+0000","last_clean":"2024-04-25T10:06:51.856308+0000","last_became_active":"2024-04-25T10:06:50.823670+0000","last_became_peered":"2024-04-25T10:06:50.823670+0000","last_unstale":"2024-04-25T10:06:51.856308+0000","last_undegraded":"2024-04-25T10:06:51.856308+0000","last_fullsized":"2024-04-25T10:06:51.856308+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:03.627437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845672+0000","last_change":"2024-04-25T10:06:51.843182+0000","last_active":"2024-04-25T10:06:51.845672+0000","last_peered":"2024-04-25T10:06:51.845672+0000","last_clean":"2024-04-25T10:06:51.845672+0000","last_became_active":"2024-04-25T10:06:51.842811+0000","last_became_peered":"2024-04-25T10:06:51.842811+0000","last_unstale":"2024-04-25T10:06:51.845672+0000","last_undegraded":"2024-04-25T10:06:51.845672+0000","last_fullsized":"2024-04-25T10:06:51.845672+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:43.266424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.850017+0000","last_change":"2024-04-25T10:06:50.823653+0000","last_active":"2024-04-25T10:06:51.850017+0000","last_peered":"2024-04-25T10:06:51.850017+0000","last_clean":"2024-04-25T10:06:51.850017+0000","last_became_active":"2024-04-25T10:06:50.823373+0000","last_became_peered":"2024-04-25T10:06:50.823373+0000","last_unstale":"2024-04-25T10:06:51.850017+0000","last_undegraded":"2024-04-25T10:06:51.850017+0000","last_fullsized":"2024-04-25T10:06:51.850017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:19.313862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842243+0000","last_change":"2024-04-25T10:06:51.836404+0000","last_active":"2024-04-25T10:06:51.842243+0000","last_peered":"2024-04-25T10:06:51.842243+0000","last_clean":"2024-04-25T10:06:51.842243+0000","last_became_active":"2024-04-25T10:06:51.835875+0000","last_became_peered":"2024-04-25T10:06:51.835875+0000","last_unstale":"2024-04-25T10:06:51.842243+0000","last_undegraded":"2024-04-25T10:06:51.842243+0000","last_fullsized":"2024-04-25T10:06:51.842243+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:30.250121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828780+0000","last_change":"2024-04-25T10:06:50.824101+0000","last_active":"2024-04-25T10:06:51.828780+0000","last_peered":"2024-04-25T10:06:51.828780+0000","last_clean":"2024-04-25T10:06:51.828780+0000","last_became_active":"2024-04-25T10:06:50.823800+0000","last_became_peered":"2024-04-25T10:06:50.823800+0000","last_unstale":"2024-04-25T10:06:51.828780+0000","last_undegraded":"2024-04-25T10:06:51.828780+0000","last_fullsized":"2024-04-25T10:06:51.828780+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:10.364676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846106+0000","last_change":"2024-04-25T10:06:51.842607+0000","last_active":"2024-04-25T10:06:51.846106+0000","last_peered":"2024-04-25T10:06:51.846106+0000","last_clean":"2024-04-25T10:06:51.846106+0000","last_became_active":"2024-04-25T10:06:51.842296+0000","last_became_peered":"2024-04-25T10:06:51.842296+0000","last_unstale":"2024-04-25T10:06:51.846106+0000","last_undegraded":"2024-04-25T10:06:51.846106+0000","last_fullsized":"2024-04-25T10:06:51.846106+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:26.337568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.831684+0000","last_change":"2024-04-25T10:06:50.826028+0000","last_active":"2024-04-25T10:06:51.831684+0000","last_peered":"2024-04-25T10:06:51.831684+0000","last_clean":"2024-04-25T10:06:51.831684+0000","last_became_active":"2024-04-25T10:06:50.825717+0000","last_became_peered":"2024-04-25T10:06:50.825717+0000","last_unstale":"2024-04-25T10:06:51.831684+0000","last_undegraded":"2024-04-25T10:06:51.831684+0000","last_fullsized":"2024-04-25T10:06:51.831684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:28.284253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834918+0000","last_change":"2024-04-25T10:06:51.833850+0000","last_active":"2024-04-25T10:06:51.834918+0000","last_peered":"2024-04-25T10:06:51.834918+0000","last_clean":"2024-04-25T10:06:51.834918+0000","last_became_active":"2024-04-25T10:06:51.833584+0000","last_became_peered":"2024-04-25T10:06:51.833584+0000","last_unstale":"2024-04-25T10:06:51.834918+0000","last_undegraded":"2024-04-25T10:06:51.834918+0000","last_fullsized":"2024-04-25T10:06:51.834918+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:01.261252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854164+0000","last_change":"2024-04-25T10:06:50.820905+0000","last_active":"2024-04-25T10:06:51.854164+0000","last_peered":"2024-04-25T10:06:51.854164+0000","last_clean":"2024-04-25T10:06:51.854164+0000","last_became_active":"2024-04-25T10:06:50.820642+0000","last_became_peered":"2024-04-25T10:06:50.820642+0000","last_unstale":"2024-04-25T10:06:51.854164+0000","last_undegraded":"2024-04-25T10:06:51.854164+0000","last_fullsized":"2024-04-25T10:06:51.854164+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:22.656356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846398+0000","last_change":"2024-04-25T10:06:51.842872+0000","last_active":"2024-04-25T10:06:51.846398+0000","last_peered":"2024-04-25T10:06:51.846398+0000","last_clean":"2024-04-25T10:06:51.846398+0000","last_became_active":"2024-04-25T10:06:51.842550+0000","last_became_peered":"2024-04-25T10:06:51.842550+0000","last_unstale":"2024-04-25T10:06:51.846398+0000","last_undegraded":"2024-04-25T10:06:51.846398+0000","last_fullsized":"2024-04-25T10:06:51.846398+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:51.599603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839657+0000","last_change":"2024-04-25T10:06:50.821527+0000","last_active":"2024-04-25T10:06:52.839657+0000","last_peered":"2024-04-25T10:06:52.839657+0000","last_clean":"2024-04-25T10:06:52.839657+0000","last_became_active":"2024-04-25T10:06:50.821259+0000","last_became_peered":"2024-04-25T10:06:50.821259+0000","last_unstale":"2024-04-25T10:06:52.839657+0000","last_undegraded":"2024-04-25T10:06:52.839657+0000","last_fullsized":"2024-04-25T10:06:52.839657+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:42.099958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845649+0000","last_change":"2024-04-25T10:06:51.840224+0000","last_active":"2024-04-25T10:06:51.845649+0000","last_peered":"2024-04-25T10:06:51.845649+0000","last_clean":"2024-04-25T10:06:51.845649+0000","last_became_active":"2024-04-25T10:06:51.839917+0000","last_became_peered":"2024-04-25T10:06:51.839917+0000","last_unstale":"2024-04-25T10:06:51.845649+0000","last_undegraded":"2024-04-25T10:06:51.845649+0000","last_fullsized":"2024-04-25T10:06:51.845649+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:43.266424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.859198+0000","last_change":"2024-04-25T10:06:50.818607+0000","last_active":"2024-04-25T10:06:51.859198+0000","last_peered":"2024-04-25T10:06:51.859198+0000","last_clean":"2024-04-25T10:06:51.859198+0000","last_became_active":"2024-04-25T10:06:50.818315+0000","last_became_peered":"2024-04-25T10:06:50.818315+0000","last_unstale":"2024-04-25T10:06:51.859198+0000","last_undegraded":"2024-04-25T10:06:51.859198+0000","last_fullsized":"2024-04-25T10:06:51.859198+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:50.599925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.835690+0000","last_change":"2024-04-25T10:06:51.834754+0000","last_active":"2024-04-25T10:06:51.835690+0000","last_peered":"2024-04-25T10:06:51.835690+0000","last_clean":"2024-04-25T10:06:51.835690+0000","last_became_active":"2024-04-25T10:06:51.834425+0000","last_became_peered":"2024-04-25T10:06:51.834425+0000","last_unstale":"2024-04-25T10:06:51.835690+0000","last_undegraded":"2024-04-25T10:06:51.835690+0000","last_fullsized":"2024-04-25T10:06:51.835690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:17.220423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854430+0000","last_change":"2024-04-25T10:06:50.820485+0000","last_active":"2024-04-25T10:06:51.854430+0000","last_peered":"2024-04-25T10:06:51.854430+0000","last_clean":"2024-04-25T10:06:51.854430+0000","last_became_active":"2024-04-25T10:06:50.820215+0000","last_became_peered":"2024-04-25T10:06:50.820215+0000","last_unstale":"2024-04-25T10:06:51.854430+0000","last_undegraded":"2024-04-25T10:06:51.854430+0000","last_fullsized":"2024-04-25T10:06:51.854430+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:19.313862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847857+0000","last_change":"2024-04-25T10:06:51.845056+0000","last_active":"2024-04-25T10:06:51.847857+0000","last_peered":"2024-04-25T10:06:51.847857+0000","last_clean":"2024-04-25T10:06:51.847857+0000","last_became_active":"2024-04-25T10:06:51.844448+0000","last_became_peered":"2024-04-25T10:06:51.844448+0000","last_unstale":"2024-04-25T10:06:51.847857+0000","last_undegraded":"2024-04-25T10:06:51.847857+0000","last_fullsized":"2024-04-25T10:06:51.847857+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:26.337568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.831449+0000","last_change":"2024-04-25T10:06:50.825791+0000","last_active":"2024-04-25T10:06:51.831449+0000","last_peered":"2024-04-25T10:06:51.831449+0000","last_clean":"2024-04-25T10:06:51.831449+0000","last_became_active":"2024-04-25T10:06:50.825467+0000","last_became_peered":"2024-04-25T10:06:50.825467+0000","last_unstale":"2024-04-25T10:06:51.831449+0000","last_undegraded":"2024-04-25T10:06:51.831449+0000","last_fullsized":"2024-04-25T10:06:51.831449+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.521754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845737+0000","last_change":"2024-04-25T10:06:51.842252+0000","last_active":"2024-04-25T10:06:51.845737+0000","last_peered":"2024-04-25T10:06:51.845737+0000","last_clean":"2024-04-25T10:06:51.845737+0000","last_became_active":"2024-04-25T10:06:51.841999+0000","last_became_peered":"2024-04-25T10:06:51.841999+0000","last_unstale":"2024-04-25T10:06:51.845737+0000","last_undegraded":"2024-04-25T10:06:51.845737+0000","last_fullsized":"2024-04-25T10:06:51.845737+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:15.390698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.851706+0000","last_change":"2024-04-25T10:06:50.821347+0000","last_active":"2024-04-25T10:06:51.851706+0000","last_peered":"2024-04-25T10:06:51.851706+0000","last_clean":"2024-04-25T10:06:51.851706+0000","last_became_active":"2024-04-25T10:06:50.821073+0000","last_became_peered":"2024-04-25T10:06:50.821073+0000","last_unstale":"2024-04-25T10:06:51.851706+0000","last_undegraded":"2024-04-25T10:06:51.851706+0000","last_fullsized":"2024-04-25T10:06:51.851706+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:02.829457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842908+0000","last_change":"2024-04-25T10:06:51.838646+0000","last_active":"2024-04-25T10:06:51.842908+0000","last_peered":"2024-04-25T10:06:51.842908+0000","last_clean":"2024-04-25T10:06:51.842908+0000","last_became_active":"2024-04-25T10:06:51.835104+0000","last_became_peered":"2024-04-25T10:06:51.835104+0000","last_unstale":"2024-04-25T10:06:51.842908+0000","last_undegraded":"2024-04-25T10:06:51.842908+0000","last_fullsized":"2024-04-25T10:06:51.842908+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:10.566374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828795+0000","last_change":"2024-04-25T10:06:50.827168+0000","last_active":"2024-04-25T10:06:51.828795+0000","last_peered":"2024-04-25T10:06:51.828795+0000","last_clean":"2024-04-25T10:06:51.828795+0000","last_became_active":"2024-04-25T10:06:50.826210+0000","last_became_peered":"2024-04-25T10:06:50.826210+0000","last_unstale":"2024-04-25T10:06:51.828795+0000","last_undegraded":"2024-04-25T10:06:51.828795+0000","last_fullsized":"2024-04-25T10:06:51.828795+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.650773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839389+0000","last_change":"2024-04-25T10:06:51.841005+0000","last_active":"2024-04-25T10:06:52.839389+0000","last_peered":"2024-04-25T10:06:52.839389+0000","last_clean":"2024-04-25T10:06:52.839389+0000","last_became_active":"2024-04-25T10:06:51.840719+0000","last_became_peered":"2024-04-25T10:06:51.840719+0000","last_unstale":"2024-04-25T10:06:52.839389+0000","last_undegraded":"2024-04-25T10:06:52.839389+0000","last_fullsized":"2024-04-25T10:06:52.839389+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:37.185720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.825362+0000","last_change":"2024-04-25T10:06:50.821404+0000","last_active":"2024-04-25T10:06:51.825362+0000","last_peered":"2024-04-25T10:06:51.825362+0000","last_clean":"2024-04-25T10:06:51.825362+0000","last_became_active":"2024-04-25T10:06:50.821129+0000","last_became_peered":"2024-04-25T10:06:50.821129+0000","last_unstale":"2024-04-25T10:06:51.825362+0000","last_undegraded":"2024-04-25T10:06:51.825362+0000","last_fullsized":"2024-04-25T10:06:51.825362+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.650773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839328+0000","last_change":"2024-04-25T10:06:51.846097+0000","last_active":"2024-04-25T10:06:52.839328+0000","last_peered":"2024-04-25T10:06:52.839328+0000","last_clean":"2024-04-25T10:06:52.839328+0000","last_became_active":"2024-04-25T10:06:51.845082+0000","last_became_peered":"2024-04-25T10:06:51.845082+0000","last_unstale":"2024-04-25T10:06:52.839328+0000","last_undegraded":"2024-04-25T10:06:52.839328+0000","last_fullsized":"2024-04-25T10:06:52.839328+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:23.577778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.823231+0000","last_change":"2024-04-25T10:06:50.825554+0000","last_active":"2024-04-25T10:06:51.823231+0000","last_peered":"2024-04-25T10:06:51.823231+0000","last_clean":"2024-04-25T10:06:51.823231+0000","last_became_active":"2024-04-25T10:06:50.825226+0000","last_became_peered":"2024-04-25T10:06:50.825226+0000","last_unstale":"2024-04-25T10:06:51.823231+0000","last_undegraded":"2024-04-25T10:06:51.823231+0000","last_fullsized":"2024-04-25T10:06:51.823231+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:16.990316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834352+0000","last_change":"2024-04-25T10:06:51.832937+0000","last_active":"2024-04-25T10:06:51.834352+0000","last_peered":"2024-04-25T10:06:51.834352+0000","last_clean":"2024-04-25T10:06:51.834352+0000","last_became_active":"2024-04-25T10:06:51.832620+0000","last_became_peered":"2024-04-25T10:06:51.832620+0000","last_unstale":"2024-04-25T10:06:51.834352+0000","last_undegraded":"2024-04-25T10:06:51.834352+0000","last_fullsized":"2024-04-25T10:06:51.834352+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:47.015585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"47'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.841008+0000","last_change":"2024-04-25T10:06:50.825532+0000","last_active":"2024-04-25T10:06:52.841008+0000","last_peered":"2024-04-25T10:06:52.841008+0000","last_clean":"2024-04-25T10:06:52.841008+0000","last_became_active":"2024-04-25T10:06:50.825219+0000","last_became_peered":"2024-04-25T10:06:50.825219+0000","last_unstale":"2024-04-25T10:06:52.841008+0000","last_undegraded":"2024-04-25T10:06:52.841008+0000","last_fullsized":"2024-04-25T10:06:52.841008+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:39.860999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841715+0000","last_change":"2024-04-25T10:06:51.835802+0000","last_active":"2024-04-25T10:06:51.841715+0000","last_peered":"2024-04-25T10:06:51.841715+0000","last_clean":"2024-04-25T10:06:51.841715+0000","last_became_active":"2024-04-25T10:06:51.835196+0000","last_became_peered":"2024-04-25T10:06:51.835196+0000","last_unstale":"2024-04-25T10:06:51.841715+0000","last_undegraded":"2024-04-25T10:06:51.841715+0000","last_fullsized":"2024-04-25T10:06:51.841715+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:27.916081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828499+0000","last_change":"2024-04-25T10:06:50.826677+0000","last_active":"2024-04-25T10:06:51.828499+0000","last_peered":"2024-04-25T10:06:51.828499+0000","last_clean":"2024-04-25T10:06:51.828499+0000","last_became_active":"2024-04-25T10:06:50.826114+0000","last_became_peered":"2024-04-25T10:06:50.826114+0000","last_unstale":"2024-04-25T10:06:51.828499+0000","last_undegraded":"2024-04-25T10:06:51.828499+0000","last_fullsized":"2024-04-25T10:06:51.828499+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:39.041098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841453+0000","last_change":"2024-04-25T10:06:51.837949+0000","last_active":"2024-04-25T10:06:51.841453+0000","last_peered":"2024-04-25T10:06:51.841453+0000","last_clean":"2024-04-25T10:06:51.841453+0000","last_became_active":"2024-04-25T10:06:51.837681+0000","last_became_peered":"2024-04-25T10:06:51.837681+0000","last_unstale":"2024-04-25T10:06:51.841453+0000","last_undegraded":"2024-04-25T10:06:51.841453+0000","last_fullsized":"2024-04-25T10:06:51.841453+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:17.563413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828714+0000","last_change":"2024-04-25T10:06:50.826236+0000","last_active":"2024-04-25T10:06:51.828714+0000","last_peered":"2024-04-25T10:06:51.828714+0000","last_clean":"2024-04-25T10:06:51.828714+0000","last_became_active":"2024-04-25T10:06:50.825899+0000","last_became_peered":"2024-04-25T10:06:50.825899+0000","last_unstale":"2024-04-25T10:06:51.828714+0000","last_undegraded":"2024-04-25T10:06:51.828714+0000","last_fullsized":"2024-04-25T10:06:51.828714+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:15.288040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845716+0000","last_change":"2024-04-25T10:06:51.840773+0000","last_active":"2024-04-25T10:06:51.845716+0000","last_peered":"2024-04-25T10:06:51.845716+0000","last_clean":"2024-04-25T10:06:51.845716+0000","last_became_active":"2024-04-25T10:06:51.840412+0000","last_became_peered":"2024-04-25T10:06:51.840412+0000","last_unstale":"2024-04-25T10:06:51.845716+0000","last_undegraded":"2024-04-25T10:06:51.845716+0000","last_fullsized":"2024-04-25T10:06:51.845716+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:58.954708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.827378+0000","last_change":"2024-04-25T10:06:50.819732+0000","last_active":"2024-04-25T10:06:51.827378+0000","last_peered":"2024-04-25T10:06:51.827378+0000","last_clean":"2024-04-25T10:06:51.827378+0000","last_became_active":"2024-04-25T10:06:50.819448+0000","last_became_peered":"2024-04-25T10:06:50.819448+0000","last_unstale":"2024-04-25T10:06:51.827378+0000","last_undegraded":"2024-04-25T10:06:51.827378+0000","last_fullsized":"2024-04-25T10:06:51.827378+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:09.564901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839296+0000","last_change":"2024-04-25T10:06:51.833076+0000","last_active":"2024-04-25T10:06:52.839296+0000","last_peered":"2024-04-25T10:06:52.839296+0000","last_clean":"2024-04-25T10:06:52.839296+0000","last_became_active":"2024-04-25T10:06:51.832825+0000","last_became_peered":"2024-04-25T10:06:51.832825+0000","last_unstale":"2024-04-25T10:06:52.839296+0000","last_undegraded":"2024-04-25T10:06:52.839296+0000","last_fullsized":"2024-04-25T10:06:52.839296+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:32.521164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.823889+0000","last_change":"2024-04-25T10:06:50.821944+0000","last_active":"2024-04-25T10:06:51.823889+0000","last_peered":"2024-04-25T10:06:51.823889+0000","last_clean":"2024-04-25T10:06:51.823889+0000","last_became_active":"2024-04-25T10:06:50.821425+0000","last_became_peered":"2024-04-25T10:06:50.821425+0000","last_unstale":"2024-04-25T10:06:51.823889+0000","last_undegraded":"2024-04-25T10:06:51.823889+0000","last_fullsized":"2024-04-25T10:06:51.823889+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:41.915878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845619+0000","last_change":"2024-04-25T10:06:51.843538+0000","last_active":"2024-04-25T10:06:51.845619+0000","last_peered":"2024-04-25T10:06:51.845619+0000","last_clean":"2024-04-25T10:06:51.845619+0000","last_became_active":"2024-04-25T10:06:51.842717+0000","last_became_peered":"2024-04-25T10:06:51.842717+0000","last_unstale":"2024-04-25T10:06:51.845619+0000","last_undegraded":"2024-04-25T10:06:51.845619+0000","last_fullsized":"2024-04-25T10:06:51.845619+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:32.311454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.831838+0000","last_change":"2024-04-25T10:06:50.822997+0000","last_active":"2024-04-25T10:06:51.831838+0000","last_peered":"2024-04-25T10:06:51.831838+0000","last_clean":"2024-04-25T10:06:51.831838+0000","last_became_active":"2024-04-25T10:06:50.822554+0000","last_became_peered":"2024-04-25T10:06:50.822554+0000","last_unstale":"2024-04-25T10:06:51.831838+0000","last_undegraded":"2024-04-25T10:06:51.831838+0000","last_fullsized":"2024-04-25T10:06:51.831838+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:00.259780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834087+0000","last_change":"2024-04-25T10:06:51.831355+0000","last_active":"2024-04-25T10:06:51.834087+0000","last_peered":"2024-04-25T10:06:51.834087+0000","last_clean":"2024-04-25T10:06:51.834087+0000","last_became_active":"2024-04-25T10:06:51.831047+0000","last_became_peered":"2024-04-25T10:06:51.831047+0000","last_unstale":"2024-04-25T10:06:51.834087+0000","last_undegraded":"2024-04-25T10:06:51.834087+0000","last_fullsized":"2024-04-25T10:06:51.834087+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:29.285725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.832092+0000","last_change":"2024-04-25T10:06:50.823705+0000","last_active":"2024-04-25T10:06:51.832092+0000","last_peered":"2024-04-25T10:06:51.832092+0000","last_clean":"2024-04-25T10:06:51.832092+0000","last_became_active":"2024-04-25T10:06:50.823452+0000","last_became_peered":"2024-04-25T10:06:50.823452+0000","last_unstale":"2024-04-25T10:06:51.832092+0000","last_undegraded":"2024-04-25T10:06:51.832092+0000","last_fullsized":"2024-04-25T10:06:51.832092+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:50.599925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839685+0000","last_change":"2024-04-25T10:06:51.835572+0000","last_active":"2024-04-25T10:06:52.839685+0000","last_peered":"2024-04-25T10:06:52.839685+0000","last_clean":"2024-04-25T10:06:52.839685+0000","last_became_active":"2024-04-25T10:06:51.835269+0000","last_became_peered":"2024-04-25T10:06:51.835269+0000","last_unstale":"2024-04-25T10:06:52.839685+0000","last_undegraded":"2024-04-25T10:06:52.839685+0000","last_fullsized":"2024-04-25T10:06:52.839685+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:52.423386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.849495+0000","last_change":"2024-04-25T10:06:50.823009+0000","last_active":"2024-04-25T10:06:51.849495+0000","last_peered":"2024-04-25T10:06:51.849495+0000","last_clean":"2024-04-25T10:06:51.849495+0000","last_became_active":"2024-04-25T10:06:50.822686+0000","last_became_peered":"2024-04-25T10:06:50.822686+0000","last_unstale":"2024-04-25T10:06:51.849495+0000","last_undegraded":"2024-04-25T10:06:51.849495+0000","last_fullsized":"2024-04-25T10:06:51.849495+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:53.108324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":218,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.822612+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:06:51.822612+0000","last_peered":"2024-04-25T10:06:51.822612+0000","last_clean":"2024-04-25T10:06:51.822612+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:06:51.822612+0000","last_undegraded":"2024-04-25T10:06:51.822612+0000","last_fullsized":"2024-04-25T10:06:51.822612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":536576,"data_stored":214001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":540672,"data_stored":214583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509467,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":207192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509467,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":544768,"data_stored":215579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":207192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1130496,"data_stored":804405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":540672,"data_stored":214639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":810657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:59.261 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:59.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.369+0000 7f71a6d41640 1 Processor -- start 2024-04-25T10:06:59.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.369+0000 7f71a6d41640 1 -- start start 2024-04-25T10:06:59.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.370+0000 7f71a6d41640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71a015a870 0x7f71a00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.370+0000 7f71a6d41640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71a00cfc40 0x7f71a015ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.370+0000 7f71a6d41640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a00cdb10 con 0x7f71a006e2e0 2024-04-25T10:06:59.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.370+0000 7f71a6d41640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a00ce720 con 0x7f71a015a870 2024-04-25T10:06:59.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.370+0000 7f71a6d41640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00cddf0 con 0x7f71a00cfc40 2024-04-25T10:06:59.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.370+0000 7f71a4add640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71a015a870 0x7f71a00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.370+0000 7f71a4add640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71a015a870 0x7f71a00cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50520/0 (socket says 172.21.15.16:50520) 2024-04-25T10:06:59.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.370+0000 7f71a4add640 1 -- 172.21.15.16:0/1432680948 learned_addr learned my addr 172.21.15.16:0/1432680948 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:59.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.371+0000 7f71a52de640 1 --2- 172.21.15.16:0/1432680948 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71a00cfc40 0x7f71a015ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.371+0000 7f71a4add640 1 -- 172.21.15.16:0/1432680948 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71a00cfc40 msgr2=0x7f71a015ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.371+0000 7f71a4add640 1 --2- 172.21.15.16:0/1432680948 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71a00cfc40 0x7f71a015ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.371+0000 7f71a4add640 1 -- 172.21.15.16:0/1432680948 >> v1:172.21.15.186:6789/0 conn(0x7f71a006e2e0 legacy=0x7f71a00cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.371+0000 7f71a4add640 1 -- 172.21.15.16:0/1432680948 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a00ce0d0 con 0x7f71a015a870 2024-04-25T10:06:59.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.372+0000 7f71a4add640 1 --2- 172.21.15.16:0/1432680948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a015a870 0x7f71a00cebb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7198009e50 tx=0x7f719802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1bc04e8971336e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.372+0000 7f719f7fe640 1 -- 172.21.15.16:0/1432680948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198036070 con 0x7f71a015a870 2024-04-25T10:06:59.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.372+0000 7f719f7fe640 1 -- 172.21.15.16:0/1432680948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7198031600 con 0x7f71a015a870 2024-04-25T10:06:59.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.372+0000 7f719f7fe640 1 -- 172.21.15.16:0/1432680948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71980319d0 con 0x7f71a015a870 2024-04-25T10:06:59.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.372+0000 7f71a6d41640 1 -- 172.21.15.16:0/1432680948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a015a870 msgr2=0x7f71a00cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.372+0000 7f71a6d41640 1 --2- 172.21.15.16:0/1432680948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a015a870 0x7f71a00cebb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7198009e50 tx=0x7f719802a710 comp rx=0 tx=0).stop 2024-04-25T10:06:59.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.372+0000 7f71a6d41640 1 -- 172.21.15.16:0/1432680948 shutdown_connections 2024-04-25T10:06:59.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.372+0000 7f71a6d41640 1 --2- 172.21.15.16:0/1432680948 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71a00cfc40 0x7f71a015ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.372+0000 7f71a6d41640 1 --2- 172.21.15.16:0/1432680948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a015a870 0x7f71a00cebb0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.372+0000 7f71a6d41640 1 -- 172.21.15.16:0/1432680948 >> 172.21.15.16:0/1432680948 conn(0x7f71a006aba0 msgr2=0x7f71a006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.372+0000 7f71a6d41640 1 -- 172.21.15.16:0/1432680948 shutdown_connections 2024-04-25T10:06:59.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.372+0000 7f71a6d41640 1 -- 172.21.15.16:0/1432680948 wait complete. 2024-04-25T10:06:59.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.374+0000 7f71a6d41640 1 Processor -- start 2024-04-25T10:06:59.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.374+0000 7f71a6d41640 1 -- start start 2024-04-25T10:06:59.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.374+0000 7f71a6d41640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71a006e2e0 0x7f71a0146e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.374+0000 7f71a6d41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a00cfc40 0x7f71a00a9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.374+0000 7f71a6d41640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71a015a870 0x7f71a00a9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.374+0000 7f71a6d41640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00cd700 con 0x7f71a00cfc40 2024-04-25T10:06:59.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.374+0000 7f71a6d41640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00cd580 con 0x7f71a015a870 2024-04-25T10:06:59.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.374+0000 7f71a6d41640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00cd880 con 0x7f71a006e2e0 2024-04-25T10:06:59.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.375+0000 7f719ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a00cfc40 0x7f71a00a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.375+0000 7f719ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a00cfc40 0x7f71a00a9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50524/0 (socket says 172.21.15.16:50524) 2024-04-25T10:06:59.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.375+0000 7f719ffff640 1 -- 172.21.15.16:0/3355237565 learned_addr learned my addr 172.21.15.16:0/3355237565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:59.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.375+0000 7f71a52de640 1 --2- 172.21.15.16:0/3355237565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71a015a870 0x7f71a00a9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.375+0000 7f71a4add640 1 --2- 172.21.15.16:0/3355237565 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71a006e2e0 0x7f71a0146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.375+0000 7f719ffff640 1 -- 172.21.15.16:0/3355237565 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71a006e2e0 msgr2=0x7f71a0146e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.375+0000 7f719ffff640 1 --2- 172.21.15.16:0/3355237565 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71a006e2e0 0x7f71a0146e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.375+0000 7f719ffff640 1 -- 172.21.15.16:0/3355237565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71a015a870 msgr2=0x7f71a00a9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.375+0000 7f719ffff640 1 --2- 172.21.15.16:0/3355237565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71a015a870 0x7f71a00a9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.375+0000 7f719ffff640 1 -- 172.21.15.16:0/3355237565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a00aa3c0 con 0x7f71a00cfc40 2024-04-25T10:06:59.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.375+0000 7f71a52de640 1 --2- 172.21.15.16:0/3355237565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71a015a870 0x7f71a00a9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:59.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.375+0000 7f719ffff640 1 --2- 172.21.15.16:0/3355237565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a00cfc40 0x7f71a00a9940 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f719000c400 tx=0x7f719000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.375+0000 7f719d7fa640 1 -- 172.21.15.16:0/3355237565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7190017070 con 0x7f71a00cfc40 2024-04-25T10:06:59.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.375+0000 7f719d7fa640 1 -- 172.21.15.16:0/3355237565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71900052d0 con 0x7f71a00cfc40 2024-04-25T10:06:59.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.375+0000 7f71a6d41640 1 -- 172.21.15.16:0/3355237565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a00aa6d0 con 0x7f71a00cfc40 2024-04-25T10:06:59.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.375+0000 7f719d7fa640 1 -- 172.21.15.16:0/3355237565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7190003830 con 0x7f71a00cfc40 2024-04-25T10:06:59.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.375+0000 7f71a6d41640 1 -- 172.21.15.16:0/3355237565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a00ac610 con 0x7f71a00cfc40 2024-04-25T10:06:59.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.376+0000 7f71a6d41640 1 -- 172.21.15.16:0/3355237565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a00ac810 con 0x7f71a00cfc40 2024-04-25T10:06:59.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.377+0000 7f719d7fa640 1 -- 172.21.15.16:0/3355237565 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f719001a070 con 0x7f71a00cfc40 2024-04-25T10:06:59.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.377+0000 7f719d7fa640 1 --2- 172.21.15.16:0/3355237565 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f718006b4d0 0x7f718006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.377+0000 7f719d7fa640 1 -- 172.21.15.16:0/3355237565 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7190089570 con 0x7f71a00cfc40 2024-04-25T10:06:59.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.378+0000 7f71a4add640 1 --2- 172.21.15.16:0/3355237565 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f718006b4d0 0x7f718006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.378+0000 7f71a4add640 1 --2- 172.21.15.16:0/3355237565 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f718006b4d0 0x7f718006d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7198033000 tx=0x7f7198009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.381+0000 7f719d7fa640 1 -- 172.21.15.16:0/3355237565 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f71a00ac810 con 0x7f71a00cfc40 2024-04-25T10:06:59.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.558+0000 7f71a6d41640 1 -- 172.21.15.16:0/3355237565 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f71a00749b0 con 0x7f718006b4d0 2024-04-25T10:06:59.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.565+0000 7f719d7fa640 1 -- 172.21.15.16:0/3355237565 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251913 (secure 0 0 0) 0x7f71a00749b0 con 0x7f718006b4d0 2024-04-25T10:06:59.573 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:06:59.574 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:06:59.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.567+0000 7f71a6d41640 1 -- 172.21.15.16:0/3355237565 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f718006b4d0 msgr2=0x7f718006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.567+0000 7f71a6d41640 1 --2- 172.21.15.16:0/3355237565 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f718006b4d0 0x7f718006d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7198033000 tx=0x7f7198009000 comp rx=0 tx=0).stop 2024-04-25T10:06:59.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.567+0000 7f71a6d41640 1 -- 172.21.15.16:0/3355237565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a00cfc40 msgr2=0x7f71a00a9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.567+0000 7f71a6d41640 1 --2- 172.21.15.16:0/3355237565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a00cfc40 0x7f71a00a9940 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f719000c400 tx=0x7f719000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.568+0000 7f71a6d41640 1 -- 172.21.15.16:0/3355237565 shutdown_connections 2024-04-25T10:06:59.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.568+0000 7f71a6d41640 1 --2- 172.21.15.16:0/3355237565 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f718006b4d0 0x7f718006d990 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.568+0000 7f71a6d41640 1 --2- 172.21.15.16:0/3355237565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71a015a870 0x7f71a00a9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.568+0000 7f71a6d41640 1 --2- 172.21.15.16:0/3355237565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a00cfc40 0x7f71a00a9940 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.568+0000 7f71a6d41640 1 --2- 172.21.15.16:0/3355237565 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71a006e2e0 0x7f71a0146e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.568+0000 7f71a6d41640 1 -- 172.21.15.16:0/3355237565 >> 172.21.15.16:0/3355237565 conn(0x7f71a006aba0 msgr2=0x7f71a015d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.568+0000 7f71a6d41640 1 -- 172.21.15.16:0/3355237565 shutdown_connections 2024-04-25T10:06:59.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.568+0000 7f71a6d41640 1 -- 172.21.15.16:0/3355237565 wait complete. 2024-04-25T10:06:59.584 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-04-25T10:06:57.805199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242388,"kb_used_data":5392,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732332,"statfs":{"total":773094113280,"available":772845907968,"internally_reserved":0,"allocated":5521408,"data_stored":2903027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.000194"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.840324+0000","last_change":"2024-04-25T10:06:51.837844+0000","last_active":"2024-04-25T10:06:51.840324+0000","last_peered":"2024-04-25T10:06:51.840324+0000","last_clean":"2024-04-25T10:06:51.840324+0000","last_became_active":"2024-04-25T10:06:51.837567+0000","last_became_peered":"2024-04-25T10:06:51.837567+0000","last_unstale":"2024-04-25T10:06:51.840324+0000","last_undegraded":"2024-04-25T10:06:51.840324+0000","last_fullsized":"2024-04-25T10:06:51.840324+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:03.830929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847252+0000","last_change":"2024-04-25T10:06:51.844389+0000","last_active":"2024-04-25T10:06:51.847252+0000","last_peered":"2024-04-25T10:06:51.847252+0000","last_clean":"2024-04-25T10:06:51.847252+0000","last_became_active":"2024-04-25T10:06:51.844082+0000","last_became_peered":"2024-04-25T10:06:51.844082+0000","last_unstale":"2024-04-25T10:06:51.847252+0000","last_undegraded":"2024-04-25T10:06:51.847252+0000","last_fullsized":"2024-04-25T10:06:51.847252+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:10.566374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847906+0000","last_change":"2024-04-25T10:06:51.845292+0000","last_active":"2024-04-25T10:06:51.847906+0000","last_peered":"2024-04-25T10:06:51.847906+0000","last_clean":"2024-04-25T10:06:51.847906+0000","last_became_active":"2024-04-25T10:06:51.844790+0000","last_became_peered":"2024-04-25T10:06:51.844790+0000","last_unstale":"2024-04-25T10:06:51.847906+0000","last_undegraded":"2024-04-25T10:06:51.847906+0000","last_fullsized":"2024-04-25T10:06:51.847906+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:51.599603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839525+0000","last_change":"2024-04-25T10:06:51.829871+0000","last_active":"2024-04-25T10:06:52.839525+0000","last_peered":"2024-04-25T10:06:52.839525+0000","last_clean":"2024-04-25T10:06:52.839525+0000","last_became_active":"2024-04-25T10:06:51.829563+0000","last_became_peered":"2024-04-25T10:06:51.829563+0000","last_unstale":"2024-04-25T10:06:52.839525+0000","last_undegraded":"2024-04-25T10:06:52.839525+0000","last_fullsized":"2024-04-25T10:06:52.839525+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:36.712935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841435+0000","last_change":"2024-04-25T10:06:51.840200+0000","last_active":"2024-04-25T10:06:51.841435+0000","last_peered":"2024-04-25T10:06:51.841435+0000","last_clean":"2024-04-25T10:06:51.841435+0000","last_became_active":"2024-04-25T10:06:51.839897+0000","last_became_peered":"2024-04-25T10:06:51.839897+0000","last_unstale":"2024-04-25T10:06:51.841435+0000","last_undegraded":"2024-04-25T10:06:51.841435+0000","last_fullsized":"2024-04-25T10:06:51.841435+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:23.657828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842607+0000","last_change":"2024-04-25T10:06:51.836510+0000","last_active":"2024-04-25T10:06:51.842607+0000","last_peered":"2024-04-25T10:06:51.842607+0000","last_clean":"2024-04-25T10:06:51.842607+0000","last_became_active":"2024-04-25T10:06:51.836244+0000","last_became_peered":"2024-04-25T10:06:51.836244+0000","last_unstale":"2024-04-25T10:06:51.842607+0000","last_undegraded":"2024-04-25T10:06:51.842607+0000","last_fullsized":"2024-04-25T10:06:51.842607+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:20.690035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841155+0000","last_change":"2024-04-25T10:06:51.837947+0000","last_active":"2024-04-25T10:06:51.841155+0000","last_peered":"2024-04-25T10:06:51.841155+0000","last_clean":"2024-04-25T10:06:51.841155+0000","last_became_active":"2024-04-25T10:06:51.837635+0000","last_became_peered":"2024-04-25T10:06:51.837635+0000","last_unstale":"2024-04-25T10:06:51.841155+0000","last_undegraded":"2024-04-25T10:06:51.841155+0000","last_fullsized":"2024-04-25T10:06:51.841155+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:47.682171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839202+0000","last_change":"2024-04-25T10:06:51.836550+0000","last_active":"2024-04-25T10:06:52.839202+0000","last_peered":"2024-04-25T10:06:52.839202+0000","last_clean":"2024-04-25T10:06:52.839202+0000","last_became_active":"2024-04-25T10:06:51.836157+0000","last_became_peered":"2024-04-25T10:06:51.836157+0000","last_unstale":"2024-04-25T10:06:52.839202+0000","last_undegraded":"2024-04-25T10:06:52.839202+0000","last_fullsized":"2024-04-25T10:06:52.839202+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:17.748860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854141+0000","last_change":"2024-04-25T10:06:51.847625+0000","last_active":"2024-04-25T10:06:51.854141+0000","last_peered":"2024-04-25T10:06:51.854141+0000","last_clean":"2024-04-25T10:06:51.854141+0000","last_became_active":"2024-04-25T10:06:51.847179+0000","last_became_peered":"2024-04-25T10:06:51.847179+0000","last_unstale":"2024-04-25T10:06:51.854141+0000","last_undegraded":"2024-04-25T10:06:51.854141+0000","last_fullsized":"2024-04-25T10:06:51.854141+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:40.042571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.963225+0000","last_change":"2024-04-25T10:06:51.837753+0000","last_active":"2024-04-25T10:06:52.963225+0000","last_peered":"2024-04-25T10:06:52.963225+0000","last_clean":"2024-04-25T10:06:52.963225+0000","last_became_active":"2024-04-25T10:06:51.837418+0000","last_became_peered":"2024-04-25T10:06:51.837418+0000","last_unstale":"2024-04-25T10:06:52.963225+0000","last_undegraded":"2024-04-25T10:06:52.963225+0000","last_fullsized":"2024-04-25T10:06:52.963225+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:38.643070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839652+0000","last_change":"2024-04-25T10:06:51.832514+0000","last_active":"2024-04-25T10:06:52.839652+0000","last_peered":"2024-04-25T10:06:52.839652+0000","last_clean":"2024-04-25T10:06:52.839652+0000","last_became_active":"2024-04-25T10:06:51.832253+0000","last_became_peered":"2024-04-25T10:06:51.832253+0000","last_unstale":"2024-04-25T10:06:52.839652+0000","last_undegraded":"2024-04-25T10:06:52.839652+0000","last_fullsized":"2024-04-25T10:06:52.839652+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:24.885518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845933+0000","last_change":"2024-04-25T10:06:51.843573+0000","last_active":"2024-04-25T10:06:51.845933+0000","last_peered":"2024-04-25T10:06:51.845933+0000","last_clean":"2024-04-25T10:06:51.845933+0000","last_became_active":"2024-04-25T10:06:51.843141+0000","last_became_peered":"2024-04-25T10:06:51.843141+0000","last_unstale":"2024-04-25T10:06:51.845933+0000","last_undegraded":"2024-04-25T10:06:51.845933+0000","last_fullsized":"2024-04-25T10:06:51.845933+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:14.554682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842933+0000","last_change":"2024-04-25T10:06:51.838802+0000","last_active":"2024-04-25T10:06:51.842933+0000","last_peered":"2024-04-25T10:06:51.842933+0000","last_clean":"2024-04-25T10:06:51.842933+0000","last_became_active":"2024-04-25T10:06:51.837871+0000","last_became_peered":"2024-04-25T10:06:51.837871+0000","last_unstale":"2024-04-25T10:06:51.842933+0000","last_undegraded":"2024-04-25T10:06:51.842933+0000","last_fullsized":"2024-04-25T10:06:51.842933+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:50.203747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839394+0000","last_change":"2024-04-25T10:06:51.845427+0000","last_active":"2024-04-25T10:06:52.839394+0000","last_peered":"2024-04-25T10:06:52.839394+0000","last_clean":"2024-04-25T10:06:52.839394+0000","last_became_active":"2024-04-25T10:06:51.845199+0000","last_became_peered":"2024-04-25T10:06:51.845199+0000","last_unstale":"2024-04-25T10:06:52.839394+0000","last_undegraded":"2024-04-25T10:06:52.839394+0000","last_fullsized":"2024-04-25T10:06:52.839394+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:50.203646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839222+0000","last_change":"2024-04-25T10:06:51.841154+0000","last_active":"2024-04-25T10:06:52.839222+0000","last_peered":"2024-04-25T10:06:52.839222+0000","last_clean":"2024-04-25T10:06:52.839222+0000","last_became_active":"2024-04-25T10:06:51.840810+0000","last_became_peered":"2024-04-25T10:06:51.840810+0000","last_unstale":"2024-04-25T10:06:52.839222+0000","last_undegraded":"2024-04-25T10:06:52.839222+0000","last_fullsized":"2024-04-25T10:06:52.839222+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:50.065157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.844998+0000","last_change":"2024-04-25T10:06:51.842491+0000","last_active":"2024-04-25T10:06:51.844998+0000","last_peered":"2024-04-25T10:06:51.844998+0000","last_clean":"2024-04-25T10:06:51.844998+0000","last_became_active":"2024-04-25T10:06:51.842212+0000","last_became_peered":"2024-04-25T10:06:51.842212+0000","last_unstale":"2024-04-25T10:06:51.844998+0000","last_undegraded":"2024-04-25T10:06:51.844998+0000","last_fullsized":"2024-04-25T10:06:51.844998+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:04.628909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.838191+0000","last_change":"2024-04-25T10:06:51.836207+0000","last_active":"2024-04-25T10:06:51.838191+0000","last_peered":"2024-04-25T10:06:51.838191+0000","last_clean":"2024-04-25T10:06:51.838191+0000","last_became_active":"2024-04-25T10:06:51.835856+0000","last_became_peered":"2024-04-25T10:06:51.835856+0000","last_unstale":"2024-04-25T10:06:51.838191+0000","last_undegraded":"2024-04-25T10:06:51.838191+0000","last_fullsized":"2024-04-25T10:06:51.838191+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:20.690035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.839761+0000","last_change":"2024-04-25T10:06:51.835570+0000","last_active":"2024-04-25T10:06:51.839761+0000","last_peered":"2024-04-25T10:06:51.839761+0000","last_clean":"2024-04-25T10:06:51.839761+0000","last_became_active":"2024-04-25T10:06:51.835287+0000","last_became_peered":"2024-04-25T10:06:51.835287+0000","last_unstale":"2024-04-25T10:06:51.839761+0000","last_undegraded":"2024-04-25T10:06:51.839761+0000","last_fullsized":"2024-04-25T10:06:51.839761+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:58.880955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846553+0000","last_change":"2024-04-25T10:06:51.845653+0000","last_active":"2024-04-25T10:06:51.846553+0000","last_peered":"2024-04-25T10:06:51.846553+0000","last_clean":"2024-04-25T10:06:51.846553+0000","last_became_active":"2024-04-25T10:06:51.845387+0000","last_became_peered":"2024-04-25T10:06:51.845387+0000","last_unstale":"2024-04-25T10:06:51.846553+0000","last_undegraded":"2024-04-25T10:06:51.846553+0000","last_fullsized":"2024-04-25T10:06:51.846553+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:22.306267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.962759+0000","last_change":"2024-04-25T10:06:51.836879+0000","last_active":"2024-04-25T10:06:52.962759+0000","last_peered":"2024-04-25T10:06:52.962759+0000","last_clean":"2024-04-25T10:06:52.962759+0000","last_became_active":"2024-04-25T10:06:51.835980+0000","last_became_peered":"2024-04-25T10:06:51.835980+0000","last_unstale":"2024-04-25T10:06:52.962759+0000","last_undegraded":"2024-04-25T10:06:52.962759+0000","last_fullsized":"2024-04-25T10:06:52.962759+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:19.364990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839389+0000","last_change":"2024-04-25T10:06:51.826348+0000","last_active":"2024-04-25T10:06:52.839389+0000","last_peered":"2024-04-25T10:06:52.839389+0000","last_clean":"2024-04-25T10:06:52.839389+0000","last_became_active":"2024-04-25T10:06:51.826040+0000","last_became_peered":"2024-04-25T10:06:51.826040+0000","last_unstale":"2024-04-25T10:06:52.839389+0000","last_undegraded":"2024-04-25T10:06:52.839389+0000","last_fullsized":"2024-04-25T10:06:52.839389+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:30.205631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841910+0000","last_change":"2024-04-25T10:06:51.838280+0000","last_active":"2024-04-25T10:06:51.841910+0000","last_peered":"2024-04-25T10:06:51.841910+0000","last_clean":"2024-04-25T10:06:51.841910+0000","last_became_active":"2024-04-25T10:06:51.837813+0000","last_became_peered":"2024-04-25T10:06:51.837813+0000","last_unstale":"2024-04-25T10:06:51.841910+0000","last_undegraded":"2024-04-25T10:06:51.841910+0000","last_fullsized":"2024-04-25T10:06:51.841910+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:58.880955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.833222+0000","last_change":"2024-04-25T10:06:51.831343+0000","last_active":"2024-04-25T10:06:51.833222+0000","last_peered":"2024-04-25T10:06:51.833222+0000","last_clean":"2024-04-25T10:06:51.833222+0000","last_became_active":"2024-04-25T10:06:51.831040+0000","last_became_peered":"2024-04-25T10:06:51.831040+0000","last_unstale":"2024-04-25T10:06:51.833222+0000","last_undegraded":"2024-04-25T10:06:51.833222+0000","last_fullsized":"2024-04-25T10:06:51.833222+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:04.569426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.832424+0000","last_change":"2024-04-25T10:06:51.827202+0000","last_active":"2024-04-25T10:06:51.832424+0000","last_peered":"2024-04-25T10:06:51.832424+0000","last_clean":"2024-04-25T10:06:51.832424+0000","last_became_active":"2024-04-25T10:06:51.826893+0000","last_became_peered":"2024-04-25T10:06:51.826893+0000","last_unstale":"2024-04-25T10:06:51.832424+0000","last_undegraded":"2024-04-25T10:06:51.832424+0000","last_fullsized":"2024-04-25T10:06:51.832424+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:25.546409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.836212+0000","last_change":"2024-04-25T10:06:51.835353+0000","last_active":"2024-04-25T10:06:51.836212+0000","last_peered":"2024-04-25T10:06:51.836212+0000","last_clean":"2024-04-25T10:06:51.836212+0000","last_became_active":"2024-04-25T10:06:51.834941+0000","last_became_peered":"2024-04-25T10:06:51.834941+0000","last_unstale":"2024-04-25T10:06:51.836212+0000","last_undegraded":"2024-04-25T10:06:51.836212+0000","last_fullsized":"2024-04-25T10:06:51.836212+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:48.121882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839669+0000","last_change":"2024-04-25T10:06:51.837299+0000","last_active":"2024-04-25T10:06:52.839669+0000","last_peered":"2024-04-25T10:06:52.839669+0000","last_clean":"2024-04-25T10:06:52.839669+0000","last_became_active":"2024-04-25T10:06:51.836912+0000","last_became_peered":"2024-04-25T10:06:51.836912+0000","last_unstale":"2024-04-25T10:06:52.839669+0000","last_undegraded":"2024-04-25T10:06:52.839669+0000","last_fullsized":"2024-04-25T10:06:52.839669+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:19.276862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.833241+0000","last_change":"2024-04-25T10:06:51.830316+0000","last_active":"2024-04-25T10:06:51.833241+0000","last_peered":"2024-04-25T10:06:51.833241+0000","last_clean":"2024-04-25T10:06:51.833241+0000","last_became_active":"2024-04-25T10:06:51.830039+0000","last_became_peered":"2024-04-25T10:06:51.830039+0000","last_unstale":"2024-04-25T10:06:51.833241+0000","last_undegraded":"2024-04-25T10:06:51.833241+0000","last_fullsized":"2024-04-25T10:06:51.833241+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:01.498621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.833261+0000","last_change":"2024-04-25T10:06:51.830470+0000","last_active":"2024-04-25T10:06:51.833261+0000","last_peered":"2024-04-25T10:06:51.833261+0000","last_clean":"2024-04-25T10:06:51.833261+0000","last_became_active":"2024-04-25T10:06:51.830180+0000","last_became_peered":"2024-04-25T10:06:51.830180+0000","last_unstale":"2024-04-25T10:06:51.833261+0000","last_undegraded":"2024-04-25T10:06:51.833261+0000","last_fullsized":"2024-04-25T10:06:51.833261+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:17.127193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.840946+0000","last_change":"2024-04-25T10:06:51.839558+0000","last_active":"2024-04-25T10:06:51.840946+0000","last_peered":"2024-04-25T10:06:51.840946+0000","last_clean":"2024-04-25T10:06:51.840946+0000","last_became_active":"2024-04-25T10:06:51.839271+0000","last_became_peered":"2024-04-25T10:06:51.839271+0000","last_unstale":"2024-04-25T10:06:51.840946+0000","last_undegraded":"2024-04-25T10:06:51.840946+0000","last_fullsized":"2024-04-25T10:06:51.840946+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:11.162077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.832516+0000","last_change":"2024-04-25T10:06:51.828040+0000","last_active":"2024-04-25T10:06:51.832516+0000","last_peered":"2024-04-25T10:06:51.832516+0000","last_clean":"2024-04-25T10:06:51.832516+0000","last_became_active":"2024-04-25T10:06:51.826090+0000","last_became_peered":"2024-04-25T10:06:51.826090+0000","last_unstale":"2024-04-25T10:06:51.832516+0000","last_undegraded":"2024-04-25T10:06:51.832516+0000","last_fullsized":"2024-04-25T10:06:51.832516+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:09.382894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842444+0000","last_change":"2024-04-25T10:06:51.837883+0000","last_active":"2024-04-25T10:06:51.842444+0000","last_peered":"2024-04-25T10:06:51.842444+0000","last_clean":"2024-04-25T10:06:51.842444+0000","last_became_active":"2024-04-25T10:06:51.837631+0000","last_became_peered":"2024-04-25T10:06:51.837631+0000","last_unstale":"2024-04-25T10:06:51.842444+0000","last_undegraded":"2024-04-25T10:06:51.842444+0000","last_fullsized":"2024-04-25T10:06:51.842444+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:14.554682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839229+0000","last_change":"2024-04-25T10:06:51.838274+0000","last_active":"2024-04-25T10:06:52.839229+0000","last_peered":"2024-04-25T10:06:52.839229+0000","last_clean":"2024-04-25T10:06:52.839229+0000","last_became_active":"2024-04-25T10:06:51.838016+0000","last_became_peered":"2024-04-25T10:06:51.838016+0000","last_unstale":"2024-04-25T10:06:52.839229+0000","last_undegraded":"2024-04-25T10:06:52.839229+0000","last_fullsized":"2024-04-25T10:06:52.839229+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:47.853571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846377+0000","last_change":"2024-04-25T10:06:51.842565+0000","last_active":"2024-04-25T10:06:51.846377+0000","last_peered":"2024-04-25T10:06:51.846377+0000","last_clean":"2024-04-25T10:06:51.846377+0000","last_became_active":"2024-04-25T10:06:51.842251+0000","last_became_peered":"2024-04-25T10:06:51.842251+0000","last_unstale":"2024-04-25T10:06:51.846377+0000","last_undegraded":"2024-04-25T10:06:51.846377+0000","last_fullsized":"2024-04-25T10:06:51.846377+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:32.311454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854076+0000","last_change":"2024-04-25T10:06:50.815053+0000","last_active":"2024-04-25T10:06:51.854076+0000","last_peered":"2024-04-25T10:06:51.854076+0000","last_clean":"2024-04-25T10:06:51.854076+0000","last_became_active":"2024-04-25T10:06:50.814623+0000","last_became_peered":"2024-04-25T10:06:50.814623+0000","last_unstale":"2024-04-25T10:06:51.854076+0000","last_undegraded":"2024-04-25T10:06:51.854076+0000","last_fullsized":"2024-04-25T10:06:51.854076+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:16.218950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846459+0000","last_change":"2024-04-25T10:06:51.844956+0000","last_active":"2024-04-25T10:06:51.846459+0000","last_peered":"2024-04-25T10:06:51.846459+0000","last_clean":"2024-04-25T10:06:51.846459+0000","last_became_active":"2024-04-25T10:06:51.844611+0000","last_became_peered":"2024-04-25T10:06:51.844611+0000","last_unstale":"2024-04-25T10:06:51.846459+0000","last_undegraded":"2024-04-25T10:06:51.846459+0000","last_fullsized":"2024-04-25T10:06:51.846459+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:50.170873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.848558+0000","last_change":"2024-04-25T10:06:50.822263+0000","last_active":"2024-04-25T10:06:51.848558+0000","last_peered":"2024-04-25T10:06:51.848558+0000","last_clean":"2024-04-25T10:06:51.848558+0000","last_became_active":"2024-04-25T10:06:50.821985+0000","last_became_peered":"2024-04-25T10:06:50.821985+0000","last_unstale":"2024-04-25T10:06:51.848558+0000","last_undegraded":"2024-04-25T10:06:51.848558+0000","last_fullsized":"2024-04-25T10:06:51.848558+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:37.641598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847288+0000","last_change":"2024-04-25T10:06:51.844989+0000","last_active":"2024-04-25T10:06:51.847288+0000","last_peered":"2024-04-25T10:06:51.847288+0000","last_clean":"2024-04-25T10:06:51.847288+0000","last_became_active":"2024-04-25T10:06:51.844614+0000","last_became_peered":"2024-04-25T10:06:51.844614+0000","last_unstale":"2024-04-25T10:06:51.847288+0000","last_undegraded":"2024-04-25T10:06:51.847288+0000","last_fullsized":"2024-04-25T10:06:51.847288+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:50.203747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.853873+0000","last_change":"2024-04-25T10:06:50.818405+0000","last_active":"2024-04-25T10:06:51.853873+0000","last_peered":"2024-04-25T10:06:51.853873+0000","last_clean":"2024-04-25T10:06:51.853873+0000","last_became_active":"2024-04-25T10:06:50.818118+0000","last_became_peered":"2024-04-25T10:06:50.818118+0000","last_unstale":"2024-04-25T10:06:51.853873+0000","last_undegraded":"2024-04-25T10:06:51.853873+0000","last_fullsized":"2024-04-25T10:06:51.853873+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:08.381421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839693+0000","last_change":"2024-04-25T10:06:51.832398+0000","last_active":"2024-04-25T10:06:52.839693+0000","last_peered":"2024-04-25T10:06:52.839693+0000","last_clean":"2024-04-25T10:06:52.839693+0000","last_became_active":"2024-04-25T10:06:51.832120+0000","last_became_peered":"2024-04-25T10:06:51.832120+0000","last_unstale":"2024-04-25T10:06:52.839693+0000","last_undegraded":"2024-04-25T10:06:52.839693+0000","last_fullsized":"2024-04-25T10:06:52.839693+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:08.698252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.848517+0000","last_change":"2024-04-25T10:06:50.821564+0000","last_active":"2024-04-25T10:06:51.848517+0000","last_peered":"2024-04-25T10:06:51.848517+0000","last_clean":"2024-04-25T10:06:51.848517+0000","last_became_active":"2024-04-25T10:06:50.821257+0000","last_became_peered":"2024-04-25T10:06:50.821257+0000","last_unstale":"2024-04-25T10:06:51.848517+0000","last_undegraded":"2024-04-25T10:06:51.848517+0000","last_fullsized":"2024-04-25T10:06:51.848517+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:25.919994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834314+0000","last_change":"2024-04-25T10:06:51.832079+0000","last_active":"2024-04-25T10:06:51.834314+0000","last_peered":"2024-04-25T10:06:51.834314+0000","last_clean":"2024-04-25T10:06:51.834314+0000","last_became_active":"2024-04-25T10:06:51.831785+0000","last_became_peered":"2024-04-25T10:06:51.831785+0000","last_unstale":"2024-04-25T10:06:51.834314+0000","last_undegraded":"2024-04-25T10:06:51.834314+0000","last_fullsized":"2024-04-25T10:06:51.834314+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:27.917421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.853752+0000","last_change":"2024-04-25T10:06:50.819361+0000","last_active":"2024-04-25T10:06:51.853752+0000","last_peered":"2024-04-25T10:06:51.853752+0000","last_clean":"2024-04-25T10:06:51.853752+0000","last_became_active":"2024-04-25T10:06:50.819093+0000","last_became_peered":"2024-04-25T10:06:50.819093+0000","last_unstale":"2024-04-25T10:06:51.853752+0000","last_undegraded":"2024-04-25T10:06:51.853752+0000","last_fullsized":"2024-04-25T10:06:51.853752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:24.544937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.962576+0000","last_change":"2024-04-25T10:06:51.835101+0000","last_active":"2024-04-25T10:06:52.962576+0000","last_peered":"2024-04-25T10:06:52.962576+0000","last_clean":"2024-04-25T10:06:52.962576+0000","last_became_active":"2024-04-25T10:06:51.834291+0000","last_became_peered":"2024-04-25T10:06:51.834291+0000","last_unstale":"2024-04-25T10:06:52.962576+0000","last_undegraded":"2024-04-25T10:06:52.962576+0000","last_fullsized":"2024-04-25T10:06:52.962576+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:26.921467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845318+0000","last_change":"2024-04-25T10:06:50.822762+0000","last_active":"2024-04-25T10:06:51.845318+0000","last_peered":"2024-04-25T10:06:51.845318+0000","last_clean":"2024-04-25T10:06:51.845318+0000","last_became_active":"2024-04-25T10:06:50.822463+0000","last_became_peered":"2024-04-25T10:06:50.822463+0000","last_unstale":"2024-04-25T10:06:51.845318+0000","last_undegraded":"2024-04-25T10:06:51.845318+0000","last_fullsized":"2024-04-25T10:06:51.845318+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:18.363517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.961968+0000","last_change":"2024-04-25T10:06:51.833997+0000","last_active":"2024-04-25T10:06:52.961968+0000","last_peered":"2024-04-25T10:06:52.961968+0000","last_clean":"2024-04-25T10:06:52.961968+0000","last_became_active":"2024-04-25T10:06:51.833694+0000","last_became_peered":"2024-04-25T10:06:51.833694+0000","last_unstale":"2024-04-25T10:06:52.961968+0000","last_undegraded":"2024-04-25T10:06:52.961968+0000","last_fullsized":"2024-04-25T10:06:52.961968+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:06.416821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828476+0000","last_change":"2024-04-25T10:06:50.818945+0000","last_active":"2024-04-25T10:06:51.828476+0000","last_peered":"2024-04-25T10:06:51.828476+0000","last_clean":"2024-04-25T10:06:51.828476+0000","last_became_active":"2024-04-25T10:06:50.818631+0000","last_became_peered":"2024-04-25T10:06:50.818631+0000","last_unstale":"2024-04-25T10:06:51.828476+0000","last_undegraded":"2024-04-25T10:06:51.828476+0000","last_fullsized":"2024-04-25T10:06:51.828476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:49.202275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.839815+0000","last_change":"2024-04-25T10:06:51.837313+0000","last_active":"2024-04-25T10:06:51.839815+0000","last_peered":"2024-04-25T10:06:51.839815+0000","last_clean":"2024-04-25T10:06:51.839815+0000","last_became_active":"2024-04-25T10:06:51.837024+0000","last_became_peered":"2024-04-25T10:06:51.837024+0000","last_unstale":"2024-04-25T10:06:51.839815+0000","last_undegraded":"2024-04-25T10:06:51.839815+0000","last_fullsized":"2024-04-25T10:06:51.839815+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:28.322485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.962019+0000","last_change":"2024-04-25T10:06:50.818743+0000","last_active":"2024-04-25T10:06:52.962019+0000","last_peered":"2024-04-25T10:06:52.962019+0000","last_clean":"2024-04-25T10:06:52.962019+0000","last_became_active":"2024-04-25T10:06:50.818359+0000","last_became_peered":"2024-04-25T10:06:50.818359+0000","last_unstale":"2024-04-25T10:06:52.962019+0000","last_undegraded":"2024-04-25T10:06:52.962019+0000","last_fullsized":"2024-04-25T10:06:52.962019+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:47.120410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.848307+0000","last_change":"2024-04-25T10:06:51.846840+0000","last_active":"2024-04-25T10:06:51.848307+0000","last_peered":"2024-04-25T10:06:51.848307+0000","last_clean":"2024-04-25T10:06:51.848307+0000","last_became_active":"2024-04-25T10:06:51.846314+0000","last_became_peered":"2024-04-25T10:06:51.846314+0000","last_unstale":"2024-04-25T10:06:51.848307+0000","last_undegraded":"2024-04-25T10:06:51.848307+0000","last_fullsized":"2024-04-25T10:06:51.848307+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:29.810440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.823148+0000","last_change":"2024-04-25T10:06:50.819769+0000","last_active":"2024-04-25T10:06:51.823148+0000","last_peered":"2024-04-25T10:06:51.823148+0000","last_clean":"2024-04-25T10:06:51.823148+0000","last_became_active":"2024-04-25T10:06:50.819517+0000","last_became_peered":"2024-04-25T10:06:50.819517+0000","last_unstale":"2024-04-25T10:06:51.823148+0000","last_undegraded":"2024-04-25T10:06:51.823148+0000","last_fullsized":"2024-04-25T10:06:51.823148+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:15.288040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834870+0000","last_change":"2024-04-25T10:06:51.832027+0000","last_active":"2024-04-25T10:06:51.834870+0000","last_peered":"2024-04-25T10:06:51.834870+0000","last_clean":"2024-04-25T10:06:51.834870+0000","last_became_active":"2024-04-25T10:06:51.831718+0000","last_became_peered":"2024-04-25T10:06:51.831718+0000","last_unstale":"2024-04-25T10:06:51.834870+0000","last_undegraded":"2024-04-25T10:06:51.834870+0000","last_fullsized":"2024-04-25T10:06:51.834870+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:30.250121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.963167+0000","last_change":"2024-04-25T10:06:50.825390+0000","last_active":"2024-04-25T10:06:52.963167+0000","last_peered":"2024-04-25T10:06:52.963167+0000","last_clean":"2024-04-25T10:06:52.963167+0000","last_became_active":"2024-04-25T10:06:50.825085+0000","last_became_peered":"2024-04-25T10:06:50.825085+0000","last_unstale":"2024-04-25T10:06:52.963167+0000","last_undegraded":"2024-04-25T10:06:52.963167+0000","last_fullsized":"2024-04-25T10:06:52.963167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:10.160604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839524+0000","last_change":"2024-04-25T10:06:51.845690+0000","last_active":"2024-04-25T10:06:52.839524+0000","last_peered":"2024-04-25T10:06:52.839524+0000","last_clean":"2024-04-25T10:06:52.839524+0000","last_became_active":"2024-04-25T10:06:51.845262+0000","last_became_peered":"2024-04-25T10:06:51.845262+0000","last_unstale":"2024-04-25T10:06:52.839524+0000","last_undegraded":"2024-04-25T10:06:52.839524+0000","last_fullsized":"2024-04-25T10:06:52.839524+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:20.903422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845381+0000","last_change":"2024-04-25T10:06:50.818510+0000","last_active":"2024-04-25T10:06:51.845381+0000","last_peered":"2024-04-25T10:06:51.845381+0000","last_clean":"2024-04-25T10:06:51.845381+0000","last_became_active":"2024-04-25T10:06:50.818217+0000","last_became_peered":"2024-04-25T10:06:50.818217+0000","last_unstale":"2024-04-25T10:06:51.845381+0000","last_undegraded":"2024-04-25T10:06:51.845381+0000","last_fullsized":"2024-04-25T10:06:51.845381+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.521754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841431+0000","last_change":"2024-04-25T10:06:51.838484+0000","last_active":"2024-04-25T10:06:51.841431+0000","last_peered":"2024-04-25T10:06:51.841431+0000","last_clean":"2024-04-25T10:06:51.841431+0000","last_became_active":"2024-04-25T10:06:51.838211+0000","last_became_peered":"2024-04-25T10:06:51.838211+0000","last_unstale":"2024-04-25T10:06:51.841431+0000","last_undegraded":"2024-04-25T10:06:51.841431+0000","last_fullsized":"2024-04-25T10:06:51.841431+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:46.417710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.962921+0000","last_change":"2024-04-25T10:06:50.823945+0000","last_active":"2024-04-25T10:06:52.962921+0000","last_peered":"2024-04-25T10:06:52.962921+0000","last_clean":"2024-04-25T10:06:52.962921+0000","last_became_active":"2024-04-25T10:06:50.823670+0000","last_became_peered":"2024-04-25T10:06:50.823670+0000","last_unstale":"2024-04-25T10:06:52.962921+0000","last_undegraded":"2024-04-25T10:06:52.962921+0000","last_fullsized":"2024-04-25T10:06:52.962921+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:03.627437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845672+0000","last_change":"2024-04-25T10:06:51.843182+0000","last_active":"2024-04-25T10:06:51.845672+0000","last_peered":"2024-04-25T10:06:51.845672+0000","last_clean":"2024-04-25T10:06:51.845672+0000","last_became_active":"2024-04-25T10:06:51.842811+0000","last_became_peered":"2024-04-25T10:06:51.842811+0000","last_unstale":"2024-04-25T10:06:51.845672+0000","last_undegraded":"2024-04-25T10:06:51.845672+0000","last_fullsized":"2024-04-25T10:06:51.845672+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:43.266424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.850017+0000","last_change":"2024-04-25T10:06:50.823653+0000","last_active":"2024-04-25T10:06:51.850017+0000","last_peered":"2024-04-25T10:06:51.850017+0000","last_clean":"2024-04-25T10:06:51.850017+0000","last_became_active":"2024-04-25T10:06:50.823373+0000","last_became_peered":"2024-04-25T10:06:50.823373+0000","last_unstale":"2024-04-25T10:06:51.850017+0000","last_undegraded":"2024-04-25T10:06:51.850017+0000","last_fullsized":"2024-04-25T10:06:51.850017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:19.313862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842243+0000","last_change":"2024-04-25T10:06:51.836404+0000","last_active":"2024-04-25T10:06:51.842243+0000","last_peered":"2024-04-25T10:06:51.842243+0000","last_clean":"2024-04-25T10:06:51.842243+0000","last_became_active":"2024-04-25T10:06:51.835875+0000","last_became_peered":"2024-04-25T10:06:51.835875+0000","last_unstale":"2024-04-25T10:06:51.842243+0000","last_undegraded":"2024-04-25T10:06:51.842243+0000","last_fullsized":"2024-04-25T10:06:51.842243+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:30.250121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828780+0000","last_change":"2024-04-25T10:06:50.824101+0000","last_active":"2024-04-25T10:06:51.828780+0000","last_peered":"2024-04-25T10:06:51.828780+0000","last_clean":"2024-04-25T10:06:51.828780+0000","last_became_active":"2024-04-25T10:06:50.823800+0000","last_became_peered":"2024-04-25T10:06:50.823800+0000","last_unstale":"2024-04-25T10:06:51.828780+0000","last_undegraded":"2024-04-25T10:06:51.828780+0000","last_fullsized":"2024-04-25T10:06:51.828780+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:10.364676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846106+0000","last_change":"2024-04-25T10:06:51.842607+0000","last_active":"2024-04-25T10:06:51.846106+0000","last_peered":"2024-04-25T10:06:51.846106+0000","last_clean":"2024-04-25T10:06:51.846106+0000","last_became_active":"2024-04-25T10:06:51.842296+0000","last_became_peered":"2024-04-25T10:06:51.842296+0000","last_unstale":"2024-04-25T10:06:51.846106+0000","last_undegraded":"2024-04-25T10:06:51.846106+0000","last_fullsized":"2024-04-25T10:06:51.846106+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:26.337568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.831684+0000","last_change":"2024-04-25T10:06:50.826028+0000","last_active":"2024-04-25T10:06:51.831684+0000","last_peered":"2024-04-25T10:06:51.831684+0000","last_clean":"2024-04-25T10:06:51.831684+0000","last_became_active":"2024-04-25T10:06:50.825717+0000","last_became_peered":"2024-04-25T10:06:50.825717+0000","last_unstale":"2024-04-25T10:06:51.831684+0000","last_undegraded":"2024-04-25T10:06:51.831684+0000","last_fullsized":"2024-04-25T10:06:51.831684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:28.284253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834918+0000","last_change":"2024-04-25T10:06:51.833850+0000","last_active":"2024-04-25T10:06:51.834918+0000","last_peered":"2024-04-25T10:06:51.834918+0000","last_clean":"2024-04-25T10:06:51.834918+0000","last_became_active":"2024-04-25T10:06:51.833584+0000","last_became_peered":"2024-04-25T10:06:51.833584+0000","last_unstale":"2024-04-25T10:06:51.834918+0000","last_undegraded":"2024-04-25T10:06:51.834918+0000","last_fullsized":"2024-04-25T10:06:51.834918+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:01.261252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":22,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.962096+0000","last_change":"2024-04-25T10:06:50.820905+0000","last_active":"2024-04-25T10:06:52.962096+0000","last_peered":"2024-04-25T10:06:52.962096+0000","last_clean":"2024-04-25T10:06:52.962096+0000","last_became_active":"2024-04-25T10:06:50.820642+0000","last_became_peered":"2024-04-25T10:06:50.820642+0000","last_unstale":"2024-04-25T10:06:52.962096+0000","last_undegraded":"2024-04-25T10:06:52.962096+0000","last_fullsized":"2024-04-25T10:06:52.962096+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:22.656356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.846398+0000","last_change":"2024-04-25T10:06:51.842872+0000","last_active":"2024-04-25T10:06:51.846398+0000","last_peered":"2024-04-25T10:06:51.846398+0000","last_clean":"2024-04-25T10:06:51.846398+0000","last_became_active":"2024-04-25T10:06:51.842550+0000","last_became_peered":"2024-04-25T10:06:51.842550+0000","last_unstale":"2024-04-25T10:06:51.846398+0000","last_undegraded":"2024-04-25T10:06:51.846398+0000","last_fullsized":"2024-04-25T10:06:51.846398+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:51.599603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839657+0000","last_change":"2024-04-25T10:06:50.821527+0000","last_active":"2024-04-25T10:06:52.839657+0000","last_peered":"2024-04-25T10:06:52.839657+0000","last_clean":"2024-04-25T10:06:52.839657+0000","last_became_active":"2024-04-25T10:06:50.821259+0000","last_became_peered":"2024-04-25T10:06:50.821259+0000","last_unstale":"2024-04-25T10:06:52.839657+0000","last_undegraded":"2024-04-25T10:06:52.839657+0000","last_fullsized":"2024-04-25T10:06:52.839657+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:42.099958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845649+0000","last_change":"2024-04-25T10:06:51.840224+0000","last_active":"2024-04-25T10:06:51.845649+0000","last_peered":"2024-04-25T10:06:51.845649+0000","last_clean":"2024-04-25T10:06:51.845649+0000","last_became_active":"2024-04-25T10:06:51.839917+0000","last_became_peered":"2024-04-25T10:06:51.839917+0000","last_unstale":"2024-04-25T10:06:51.845649+0000","last_undegraded":"2024-04-25T10:06:51.845649+0000","last_fullsized":"2024-04-25T10:06:51.845649+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:43.266424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.859198+0000","last_change":"2024-04-25T10:06:50.818607+0000","last_active":"2024-04-25T10:06:51.859198+0000","last_peered":"2024-04-25T10:06:51.859198+0000","last_clean":"2024-04-25T10:06:51.859198+0000","last_became_active":"2024-04-25T10:06:50.818315+0000","last_became_peered":"2024-04-25T10:06:50.818315+0000","last_unstale":"2024-04-25T10:06:51.859198+0000","last_undegraded":"2024-04-25T10:06:51.859198+0000","last_fullsized":"2024-04-25T10:06:51.859198+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:50.599925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.835690+0000","last_change":"2024-04-25T10:06:51.834754+0000","last_active":"2024-04-25T10:06:51.835690+0000","last_peered":"2024-04-25T10:06:51.835690+0000","last_clean":"2024-04-25T10:06:51.835690+0000","last_became_active":"2024-04-25T10:06:51.834425+0000","last_became_peered":"2024-04-25T10:06:51.834425+0000","last_unstale":"2024-04-25T10:06:51.835690+0000","last_undegraded":"2024-04-25T10:06:51.835690+0000","last_fullsized":"2024-04-25T10:06:51.835690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:17.220423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.854430+0000","last_change":"2024-04-25T10:06:50.820485+0000","last_active":"2024-04-25T10:06:51.854430+0000","last_peered":"2024-04-25T10:06:51.854430+0000","last_clean":"2024-04-25T10:06:51.854430+0000","last_became_active":"2024-04-25T10:06:50.820215+0000","last_became_peered":"2024-04-25T10:06:50.820215+0000","last_unstale":"2024-04-25T10:06:51.854430+0000","last_undegraded":"2024-04-25T10:06:51.854430+0000","last_fullsized":"2024-04-25T10:06:51.854430+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:19.313862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.847857+0000","last_change":"2024-04-25T10:06:51.845056+0000","last_active":"2024-04-25T10:06:51.847857+0000","last_peered":"2024-04-25T10:06:51.847857+0000","last_clean":"2024-04-25T10:06:51.847857+0000","last_became_active":"2024-04-25T10:06:51.844448+0000","last_became_peered":"2024-04-25T10:06:51.844448+0000","last_unstale":"2024-04-25T10:06:51.847857+0000","last_undegraded":"2024-04-25T10:06:51.847857+0000","last_fullsized":"2024-04-25T10:06:51.847857+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:26.337568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.831449+0000","last_change":"2024-04-25T10:06:50.825791+0000","last_active":"2024-04-25T10:06:51.831449+0000","last_peered":"2024-04-25T10:06:51.831449+0000","last_clean":"2024-04-25T10:06:51.831449+0000","last_became_active":"2024-04-25T10:06:50.825467+0000","last_became_peered":"2024-04-25T10:06:50.825467+0000","last_unstale":"2024-04-25T10:06:51.831449+0000","last_undegraded":"2024-04-25T10:06:51.831449+0000","last_fullsized":"2024-04-25T10:06:51.831449+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.521754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.962071+0000","last_change":"2024-04-25T10:06:51.842252+0000","last_active":"2024-04-25T10:06:52.962071+0000","last_peered":"2024-04-25T10:06:52.962071+0000","last_clean":"2024-04-25T10:06:52.962071+0000","last_became_active":"2024-04-25T10:06:51.841999+0000","last_became_peered":"2024-04-25T10:06:51.841999+0000","last_unstale":"2024-04-25T10:06:52.962071+0000","last_undegraded":"2024-04-25T10:06:52.962071+0000","last_fullsized":"2024-04-25T10:06:52.962071+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:22.244523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.962584+0000","last_change":"2024-04-25T10:06:50.821347+0000","last_active":"2024-04-25T10:06:52.962584+0000","last_peered":"2024-04-25T10:06:52.962584+0000","last_clean":"2024-04-25T10:06:52.962584+0000","last_became_active":"2024-04-25T10:06:50.821073+0000","last_became_peered":"2024-04-25T10:06:50.821073+0000","last_unstale":"2024-04-25T10:06:52.962584+0000","last_undegraded":"2024-04-25T10:06:52.962584+0000","last_fullsized":"2024-04-25T10:06:52.962584+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:02.829457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.842908+0000","last_change":"2024-04-25T10:06:51.838646+0000","last_active":"2024-04-25T10:06:51.842908+0000","last_peered":"2024-04-25T10:06:51.842908+0000","last_clean":"2024-04-25T10:06:51.842908+0000","last_became_active":"2024-04-25T10:06:51.835104+0000","last_became_peered":"2024-04-25T10:06:51.835104+0000","last_unstale":"2024-04-25T10:06:51.842908+0000","last_undegraded":"2024-04-25T10:06:51.842908+0000","last_fullsized":"2024-04-25T10:06:51.842908+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:10.566374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828795+0000","last_change":"2024-04-25T10:06:50.827168+0000","last_active":"2024-04-25T10:06:51.828795+0000","last_peered":"2024-04-25T10:06:51.828795+0000","last_clean":"2024-04-25T10:06:51.828795+0000","last_became_active":"2024-04-25T10:06:50.826210+0000","last_became_peered":"2024-04-25T10:06:50.826210+0000","last_unstale":"2024-04-25T10:06:51.828795+0000","last_undegraded":"2024-04-25T10:06:51.828795+0000","last_fullsized":"2024-04-25T10:06:51.828795+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.650773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839389+0000","last_change":"2024-04-25T10:06:51.841005+0000","last_active":"2024-04-25T10:06:52.839389+0000","last_peered":"2024-04-25T10:06:52.839389+0000","last_clean":"2024-04-25T10:06:52.839389+0000","last_became_active":"2024-04-25T10:06:51.840719+0000","last_became_peered":"2024-04-25T10:06:51.840719+0000","last_unstale":"2024-04-25T10:06:52.839389+0000","last_undegraded":"2024-04-25T10:06:52.839389+0000","last_fullsized":"2024-04-25T10:06:52.839389+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:37.185720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.961852+0000","last_change":"2024-04-25T10:06:50.821404+0000","last_active":"2024-04-25T10:06:52.961852+0000","last_peered":"2024-04-25T10:06:52.961852+0000","last_clean":"2024-04-25T10:06:52.961852+0000","last_became_active":"2024-04-25T10:06:50.821129+0000","last_became_peered":"2024-04-25T10:06:50.821129+0000","last_unstale":"2024-04-25T10:06:52.961852+0000","last_undegraded":"2024-04-25T10:06:52.961852+0000","last_fullsized":"2024-04-25T10:06:52.961852+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.650773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839328+0000","last_change":"2024-04-25T10:06:51.846097+0000","last_active":"2024-04-25T10:06:52.839328+0000","last_peered":"2024-04-25T10:06:52.839328+0000","last_clean":"2024-04-25T10:06:52.839328+0000","last_became_active":"2024-04-25T10:06:51.845082+0000","last_became_peered":"2024-04-25T10:06:51.845082+0000","last_unstale":"2024-04-25T10:06:52.839328+0000","last_undegraded":"2024-04-25T10:06:52.839328+0000","last_fullsized":"2024-04-25T10:06:52.839328+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:23.577778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.823231+0000","last_change":"2024-04-25T10:06:50.825554+0000","last_active":"2024-04-25T10:06:51.823231+0000","last_peered":"2024-04-25T10:06:51.823231+0000","last_clean":"2024-04-25T10:06:51.823231+0000","last_became_active":"2024-04-25T10:06:50.825226+0000","last_became_peered":"2024-04-25T10:06:50.825226+0000","last_unstale":"2024-04-25T10:06:51.823231+0000","last_undegraded":"2024-04-25T10:06:51.823231+0000","last_fullsized":"2024-04-25T10:06:51.823231+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:16.990316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834352+0000","last_change":"2024-04-25T10:06:51.832937+0000","last_active":"2024-04-25T10:06:51.834352+0000","last_peered":"2024-04-25T10:06:51.834352+0000","last_clean":"2024-04-25T10:06:51.834352+0000","last_became_active":"2024-04-25T10:06:51.832620+0000","last_became_peered":"2024-04-25T10:06:51.832620+0000","last_unstale":"2024-04-25T10:06:51.834352+0000","last_undegraded":"2024-04-25T10:06:51.834352+0000","last_fullsized":"2024-04-25T10:06:51.834352+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:47.015585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"47'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.841008+0000","last_change":"2024-04-25T10:06:50.825532+0000","last_active":"2024-04-25T10:06:52.841008+0000","last_peered":"2024-04-25T10:06:52.841008+0000","last_clean":"2024-04-25T10:06:52.841008+0000","last_became_active":"2024-04-25T10:06:50.825219+0000","last_became_peered":"2024-04-25T10:06:50.825219+0000","last_unstale":"2024-04-25T10:06:52.841008+0000","last_undegraded":"2024-04-25T10:06:52.841008+0000","last_fullsized":"2024-04-25T10:06:52.841008+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:39.860999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.962970+0000","last_change":"2024-04-25T10:06:51.835802+0000","last_active":"2024-04-25T10:06:52.962970+0000","last_peered":"2024-04-25T10:06:52.962970+0000","last_clean":"2024-04-25T10:06:52.962970+0000","last_became_active":"2024-04-25T10:06:51.835196+0000","last_became_peered":"2024-04-25T10:06:51.835196+0000","last_unstale":"2024-04-25T10:06:52.962970+0000","last_undegraded":"2024-04-25T10:06:52.962970+0000","last_fullsized":"2024-04-25T10:06:52.962970+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:42.917350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828499+0000","last_change":"2024-04-25T10:06:50.826677+0000","last_active":"2024-04-25T10:06:51.828499+0000","last_peered":"2024-04-25T10:06:51.828499+0000","last_clean":"2024-04-25T10:06:51.828499+0000","last_became_active":"2024-04-25T10:06:50.826114+0000","last_became_peered":"2024-04-25T10:06:50.826114+0000","last_unstale":"2024-04-25T10:06:51.828499+0000","last_undegraded":"2024-04-25T10:06:51.828499+0000","last_fullsized":"2024-04-25T10:06:51.828499+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:39.041098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.841453+0000","last_change":"2024-04-25T10:06:51.837949+0000","last_active":"2024-04-25T10:06:51.841453+0000","last_peered":"2024-04-25T10:06:51.841453+0000","last_clean":"2024-04-25T10:06:51.841453+0000","last_became_active":"2024-04-25T10:06:51.837681+0000","last_became_peered":"2024-04-25T10:06:51.837681+0000","last_unstale":"2024-04-25T10:06:51.841453+0000","last_undegraded":"2024-04-25T10:06:51.841453+0000","last_fullsized":"2024-04-25T10:06:51.841453+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:17.563413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.828714+0000","last_change":"2024-04-25T10:06:50.826236+0000","last_active":"2024-04-25T10:06:51.828714+0000","last_peered":"2024-04-25T10:06:51.828714+0000","last_clean":"2024-04-25T10:06:51.828714+0000","last_became_active":"2024-04-25T10:06:50.825899+0000","last_became_peered":"2024-04-25T10:06:50.825899+0000","last_unstale":"2024-04-25T10:06:51.828714+0000","last_undegraded":"2024-04-25T10:06:51.828714+0000","last_fullsized":"2024-04-25T10:06:51.828714+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:15.288040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.962500+0000","last_change":"2024-04-25T10:06:51.840773+0000","last_active":"2024-04-25T10:06:52.962500+0000","last_peered":"2024-04-25T10:06:52.962500+0000","last_clean":"2024-04-25T10:06:52.962500+0000","last_became_active":"2024-04-25T10:06:51.840412+0000","last_became_peered":"2024-04-25T10:06:51.840412+0000","last_unstale":"2024-04-25T10:06:52.962500+0000","last_undegraded":"2024-04-25T10:06:52.962500+0000","last_fullsized":"2024-04-25T10:06:52.962500+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:17.991788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.827378+0000","last_change":"2024-04-25T10:06:50.819732+0000","last_active":"2024-04-25T10:06:51.827378+0000","last_peered":"2024-04-25T10:06:51.827378+0000","last_clean":"2024-04-25T10:06:51.827378+0000","last_became_active":"2024-04-25T10:06:50.819448+0000","last_became_peered":"2024-04-25T10:06:50.819448+0000","last_unstale":"2024-04-25T10:06:51.827378+0000","last_undegraded":"2024-04-25T10:06:51.827378+0000","last_fullsized":"2024-04-25T10:06:51.827378+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:09.564901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839296+0000","last_change":"2024-04-25T10:06:51.833076+0000","last_active":"2024-04-25T10:06:52.839296+0000","last_peered":"2024-04-25T10:06:52.839296+0000","last_clean":"2024-04-25T10:06:52.839296+0000","last_became_active":"2024-04-25T10:06:51.832825+0000","last_became_peered":"2024-04-25T10:06:51.832825+0000","last_unstale":"2024-04-25T10:06:52.839296+0000","last_undegraded":"2024-04-25T10:06:52.839296+0000","last_fullsized":"2024-04-25T10:06:52.839296+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:32.521164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.823889+0000","last_change":"2024-04-25T10:06:50.821944+0000","last_active":"2024-04-25T10:06:51.823889+0000","last_peered":"2024-04-25T10:06:51.823889+0000","last_clean":"2024-04-25T10:06:51.823889+0000","last_became_active":"2024-04-25T10:06:50.821425+0000","last_became_peered":"2024-04-25T10:06:50.821425+0000","last_unstale":"2024-04-25T10:06:51.823889+0000","last_undegraded":"2024-04-25T10:06:51.823889+0000","last_fullsized":"2024-04-25T10:06:51.823889+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:41.915878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.845619+0000","last_change":"2024-04-25T10:06:51.843538+0000","last_active":"2024-04-25T10:06:51.845619+0000","last_peered":"2024-04-25T10:06:51.845619+0000","last_clean":"2024-04-25T10:06:51.845619+0000","last_became_active":"2024-04-25T10:06:51.842717+0000","last_became_peered":"2024-04-25T10:06:51.842717+0000","last_unstale":"2024-04-25T10:06:51.845619+0000","last_undegraded":"2024-04-25T10:06:51.845619+0000","last_fullsized":"2024-04-25T10:06:51.845619+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:32.311454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.831838+0000","last_change":"2024-04-25T10:06:50.822997+0000","last_active":"2024-04-25T10:06:51.831838+0000","last_peered":"2024-04-25T10:06:51.831838+0000","last_clean":"2024-04-25T10:06:51.831838+0000","last_became_active":"2024-04-25T10:06:50.822554+0000","last_became_peered":"2024-04-25T10:06:50.822554+0000","last_unstale":"2024-04-25T10:06:51.831838+0000","last_undegraded":"2024-04-25T10:06:51.831838+0000","last_fullsized":"2024-04-25T10:06:51.831838+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:00.259780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.834087+0000","last_change":"2024-04-25T10:06:51.831355+0000","last_active":"2024-04-25T10:06:51.834087+0000","last_peered":"2024-04-25T10:06:51.834087+0000","last_clean":"2024-04-25T10:06:51.834087+0000","last_became_active":"2024-04-25T10:06:51.831047+0000","last_became_peered":"2024-04-25T10:06:51.831047+0000","last_unstale":"2024-04-25T10:06:51.834087+0000","last_undegraded":"2024-04-25T10:06:51.834087+0000","last_fullsized":"2024-04-25T10:06:51.834087+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:29.285725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.832092+0000","last_change":"2024-04-25T10:06:50.823705+0000","last_active":"2024-04-25T10:06:51.832092+0000","last_peered":"2024-04-25T10:06:51.832092+0000","last_clean":"2024-04-25T10:06:51.832092+0000","last_became_active":"2024-04-25T10:06:50.823452+0000","last_became_peered":"2024-04-25T10:06:50.823452+0000","last_unstale":"2024-04-25T10:06:51.832092+0000","last_undegraded":"2024-04-25T10:06:51.832092+0000","last_fullsized":"2024-04-25T10:06:51.832092+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:50.599925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:06:52.839685+0000","last_change":"2024-04-25T10:06:51.835572+0000","last_active":"2024-04-25T10:06:52.839685+0000","last_peered":"2024-04-25T10:06:52.839685+0000","last_clean":"2024-04-25T10:06:52.839685+0000","last_became_active":"2024-04-25T10:06:51.835269+0000","last_became_peered":"2024-04-25T10:06:51.835269+0000","last_unstale":"2024-04-25T10:06:52.839685+0000","last_undegraded":"2024-04-25T10:06:52.839685+0000","last_fullsized":"2024-04-25T10:06:52.839685+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:50.801623+0000","last_clean_scrub_stamp":"2024-04-25T10:06:50.801623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:52.423386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.849495+0000","last_change":"2024-04-25T10:06:50.823009+0000","last_active":"2024-04-25T10:06:51.849495+0000","last_peered":"2024-04-25T10:06:51.849495+0000","last_clean":"2024-04-25T10:06:51.849495+0000","last_became_active":"2024-04-25T10:06:50.822686+0000","last_became_peered":"2024-04-25T10:06:50.822686+0000","last_unstale":"2024-04-25T10:06:51.849495+0000","last_undegraded":"2024-04-25T10:06:51.849495+0000","last_fullsized":"2024-04-25T10:06:51.849495+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:49.800151+0000","last_clean_scrub_stamp":"2024-04-25T10:06:49.800151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:53.108324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":218,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T10:06:51.822612+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:06:51.822612+0000","last_peered":"2024-04-25T10:06:51.822612+0000","last_clean":"2024-04-25T10:06:51.822612+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:06:51.822612+0000","last_undegraded":"2024-04-25T10:06:51.822612+0000","last_fullsized":"2024-04-25T10:06:51.822612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":536576,"data_stored":214001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":540672,"data_stored":214583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":565248,"data_stored":221971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509467,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":544768,"data_stored":215579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":207192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1130496,"data_stored":804405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":540672,"data_stored":214639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":810657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:59.589 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:06:59.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:06:59.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.727+0000 7fd8b5f0a640 1 Processor -- start 2024-04-25T10:06:59.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.727+0000 7fd8b5f0a640 1 -- start start 2024-04-25T10:06:59.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.727+0000 7fd8b5f0a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8b0144550 0x7fd8b006d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.727+0000 7fd8b5f0a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8b00cf620 0x7fd8b00cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.727+0000 7fd8b5f0a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b015fdd0 con 0x7fd8b0144d00 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.727+0000 7fd8b5f0a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b00cfff0 con 0x7fd8b0144550 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.727+0000 7fd8b5f0a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b01600b0 con 0x7fd8b00cf620 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.728+0000 7fd8af7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8b0144550 0x7fd8b006d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.728+0000 7fd8af7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8b0144550 0x7fd8b006d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50540/0 (socket says 172.21.15.16:50540) 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.728+0000 7fd8af7fe640 1 -- 172.21.15.16:0/2550641108 learned_addr learned my addr 172.21.15.16:0/2550641108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.728+0000 7fd8af7fe640 1 -- 172.21.15.16:0/2550641108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8b00cf620 msgr2=0x7fd8b00cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.728+0000 7fd8affff640 1 --2- 172.21.15.16:0/2550641108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8b00cf620 0x7fd8b00cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.728+0000 7fd8af7fe640 1 --2- 172.21.15.16:0/2550641108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8b00cf620 0x7fd8b00cfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.728+0000 7fd8af7fe640 1 -- 172.21.15.16:0/2550641108 >> v1:172.21.15.186:6789/0 conn(0x7fd8b0144d00 legacy=0x7fd8b015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.728+0000 7fd8af7fe640 1 -- 172.21.15.16:0/2550641108 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b0160390 con 0x7fd8b0144550 2024-04-25T10:06:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.729+0000 7fd8af7fe640 1 --2- 172.21.15.16:0/2550641108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8b0144550 0x7fd8b006d590 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00d0400 tx=0x7fd8a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e35580b877c054c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.729+0000 7fd8ae7fc640 1 -- 172.21.15.16:0/2550641108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4037070 con 0x7fd8b0144550 2024-04-25T10:06:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.729+0000 7fd8ae7fc640 1 -- 172.21.15.16:0/2550641108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a4032440 con 0x7fd8b0144550 2024-04-25T10:06:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.729+0000 7fd8ae7fc640 1 -- 172.21.15.16:0/2550641108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4032810 con 0x7fd8b0144550 2024-04-25T10:06:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.729+0000 7fd8b5f0a640 1 -- 172.21.15.16:0/2550641108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8b0144550 msgr2=0x7fd8b006d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.729+0000 7fd8b5f0a640 1 --2- 172.21.15.16:0/2550641108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8b0144550 0x7fd8b006d590 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00d0400 tx=0x7fd8a402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.729+0000 7fd8b5f0a640 1 -- 172.21.15.16:0/2550641108 shutdown_connections 2024-04-25T10:06:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.729+0000 7fd8b5f0a640 1 --2- 172.21.15.16:0/2550641108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8b00cf620 0x7fd8b00cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.729+0000 7fd8b5f0a640 1 --2- 172.21.15.16:0/2550641108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8b0144550 0x7fd8b006d590 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.729+0000 7fd8b5f0a640 1 -- 172.21.15.16:0/2550641108 >> 172.21.15.16:0/2550641108 conn(0x7fd8b006aba0 msgr2=0x7fd8b006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.729+0000 7fd8b5f0a640 1 -- 172.21.15.16:0/2550641108 shutdown_connections 2024-04-25T10:06:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.729+0000 7fd8b5f0a640 1 -- 172.21.15.16:0/2550641108 wait complete. 2024-04-25T10:06:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.731+0000 7fd8b5f0a640 1 Processor -- start 2024-04-25T10:06:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.731+0000 7fd8b5f0a640 1 -- start start 2024-04-25T10:06:59.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.731+0000 7fd8b5f0a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8b00cf620 0x7fd8b006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.731+0000 7fd8b5f0a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8b0144550 0x7fd8b00b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.731+0000 7fd8b5f0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8b0144d00 0x7fd8b0152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.731+0000 7fd8b5f0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0144930 con 0x7fd8b0144d00 2024-04-25T10:06:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.731+0000 7fd8b5f0a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b00cc190 con 0x7fd8b0144550 2024-04-25T10:06:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.731+0000 7fd8b5f0a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0144ab0 con 0x7fd8b00cf620 2024-04-25T10:06:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.732+0000 7fd8affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8b0144d00 0x7fd8b0152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.732+0000 7fd8affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8b0144d00 0x7fd8b0152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50546/0 (socket says 172.21.15.16:50546) 2024-04-25T10:06:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.732+0000 7fd8affff640 1 -- 172.21.15.16:0/3657220270 learned_addr learned my addr 172.21.15.16:0/3657220270 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:06:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.732+0000 7fd8aeffd640 1 --2- 172.21.15.16:0/3657220270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8b0144550 0x7fd8b00b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.732+0000 7fd8af7fe640 1 --2- 172.21.15.16:0/3657220270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8b00cf620 0x7fd8b006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.732+0000 7fd8affff640 1 -- 172.21.15.16:0/3657220270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8b00cf620 msgr2=0x7fd8b006ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.732+0000 7fd8affff640 1 --2- 172.21.15.16:0/3657220270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8b00cf620 0x7fd8b006ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.732+0000 7fd8affff640 1 -- 172.21.15.16:0/3657220270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8b0144550 msgr2=0x7fd8b00b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.732+0000 7fd8affff640 1 --2- 172.21.15.16:0/3657220270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8b0144550 0x7fd8b00b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.732+0000 7fd8affff640 1 -- 172.21.15.16:0/3657220270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b00afd30 con 0x7fd8b0144d00 2024-04-25T10:06:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.732+0000 7fd8af7fe640 1 --2- 172.21.15.16:0/3657220270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8b00cf620 0x7fd8b006ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.732+0000 7fd8affff640 1 --2- 172.21.15.16:0/3657220270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8b0144d00 0x7fd8b0152620 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800cc60 tx=0x7fd8a800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.732+0000 7fd897fff640 1 -- 172.21.15.16:0/3657220270 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8003990 con 0x7fd8b0144d00 2024-04-25T10:06:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.732+0000 7fd897fff640 1 -- 172.21.15.16:0/3657220270 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a8003b30 con 0x7fd8b0144d00 2024-04-25T10:06:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.732+0000 7fd8b5f0a640 1 -- 172.21.15.16:0/3657220270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b00b0010 con 0x7fd8b0144d00 2024-04-25T10:06:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.732+0000 7fd897fff640 1 -- 172.21.15.16:0/3657220270 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8004020 con 0x7fd8b0144d00 2024-04-25T10:06:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.732+0000 7fd8b5f0a640 1 -- 172.21.15.16:0/3657220270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b00b08b0 con 0x7fd8b0144d00 2024-04-25T10:06:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.733+0000 7fd8b5f0a640 1 -- 172.21.15.16:0/3657220270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b00749b0 con 0x7fd8b0144d00 2024-04-25T10:06:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.734+0000 7fd897fff640 1 -- 172.21.15.16:0/3657220270 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8a80041c0 con 0x7fd8b0144d00 2024-04-25T10:06:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.734+0000 7fd897fff640 1 --2- 172.21.15.16:0/3657220270 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd89c06b500 0x7fd89c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.734+0000 7fd897fff640 1 -- 172.21.15.16:0/3657220270 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8a808a7d0 con 0x7fd8b0144d00 2024-04-25T10:06:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.734+0000 7fd8af7fe640 1 --2- 172.21.15.16:0/3657220270 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd89c06b500 0x7fd89c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.735+0000 7fd8af7fe640 1 --2- 172.21.15.16:0/3657220270 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd89c06b500 0x7fd89c06d9c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4034000 tx=0x7fd8a402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.738+0000 7fd897fff640 1 -- 172.21.15.16:0/3657220270 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd8a8014030 con 0x7fd8b0144d00 2024-04-25T10:06:59.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.994+0000 7fd8b5f0a640 1 -- 172.21.15.16:0/3657220270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd8b00b0a80 con 0x7fd8b0144d00 2024-04-25T10:07:00.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.996+0000 7fd897fff640 1 -- 172.21.15.16:0/3657220270 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fd8a8059520 con 0x7fd8b0144d00 2024-04-25T10:07:00.001 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-04-25T10:07:00.001 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCZKipmAph/GhAAG3x1zC98yHQ13fjpmrlPkg== 2024-04-25T10:07:00.001 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-04-25T10:07:00.001 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-04-25T10:07:00.002 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:07:00.002 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-04-25T10:07:00.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.998+0000 7fd8b5f0a640 1 -- 172.21.15.16:0/3657220270 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd89c06b500 msgr2=0x7fd89c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.998+0000 7fd8b5f0a640 1 --2- 172.21.15.16:0/3657220270 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd89c06b500 0x7fd89c06d9c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4034000 tx=0x7fd8a402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:00.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.998+0000 7fd8b5f0a640 1 -- 172.21.15.16:0/3657220270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8b0144d00 msgr2=0x7fd8b0152620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.998+0000 7fd8b5f0a640 1 --2- 172.21.15.16:0/3657220270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8b0144d00 0x7fd8b0152620 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800cc60 tx=0x7fd8a800a5c0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.998+0000 7fd8b5f0a640 1 -- 172.21.15.16:0/3657220270 shutdown_connections 2024-04-25T10:07:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.998+0000 7fd8b5f0a640 1 --2- 172.21.15.16:0/3657220270 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd89c06b500 0x7fd89c06d9c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.998+0000 7fd8b5f0a640 1 --2- 172.21.15.16:0/3657220270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8b0144d00 0x7fd8b0152620 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.998+0000 7fd8b5f0a640 1 --2- 172.21.15.16:0/3657220270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8b0144550 0x7fd8b00b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.998+0000 7fd8b5f0a640 1 --2- 172.21.15.16:0/3657220270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8b00cf620 0x7fd8b006ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.998+0000 7fd8b5f0a640 1 -- 172.21.15.16:0/3657220270 >> 172.21.15.16:0/3657220270 conn(0x7fd8b006aba0 msgr2=0x7fd8b015f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.998+0000 7fd8b5f0a640 1 -- 172.21.15.16:0/3657220270 shutdown_connections 2024-04-25T10:07:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:06:59.998+0000 7fd8b5f0a640 1 -- 172.21.15.16:0/3657220270 wait complete. 2024-04-25T10:07:00.014 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:00.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.109+0000 7ff1c753b640 1 Processor -- start 2024-04-25T10:07:00.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.109+0000 7ff1c753b640 1 -- start start 2024-04-25T10:07:00.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.110+0000 7ff1c753b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1c015dce0 0x7ff1c00d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.110+0000 7ff1c753b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c015e0c0 con 0x7ff1c00b5b60 2024-04-25T10:07:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.110+0000 7ff1c753b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c00d2510 con 0x7ff1c00d35b0 2024-04-25T10:07:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.110+0000 7ff1c753b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c00d27f0 con 0x7ff1c015dce0 2024-04-25T10:07:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.110+0000 7ff1c5ad8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff1c00b5b60 0x7ff1c00b6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59530/0 (socket says 172.21.15.16:59530) 2024-04-25T10:07:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.110+0000 7ff1c5ad8640 1 -- 172.21.15.16:0/2932599709 learned_addr learned my addr 172.21.15.16:0/2932599709 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.110+0000 7ff1c52d7640 1 --2- 172.21.15.16:0/2932599709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1c015dce0 0x7ff1c00d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1b7fff640 1 -- 172.21.15.16:0/2932599709 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703783795 0 0) 0x7ff1c015e0c0 con 0x7ff1c00b5b60 2024-04-25T10:07:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1b7fff640 1 -- 172.21.15.16:0/2932599709 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1ac003520 con 0x7ff1c00b5b60 2024-04-25T10:07:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1b7fff640 1 -- 172.21.15.16:0/2932599709 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411608317 0 0) 0x7ff1c00d2510 con 0x7ff1c00d35b0 2024-04-25T10:07:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1b7fff640 1 -- 172.21.15.16:0/2932599709 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1c015e0c0 con 0x7ff1c00d35b0 2024-04-25T10:07:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1b7fff640 1 -- 172.21.15.16:0/2932599709 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2548511199 0 0) 0x7ff1ac003520 con 0x7ff1c00b5b60 2024-04-25T10:07:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1b7fff640 1 -- 172.21.15.16:0/2932599709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1c015dce0 msgr2=0x7ff1c00d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1b7fff640 1 --2- 172.21.15.16:0/2932599709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1c015dce0 0x7ff1c00d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1b7fff640 1 -- 172.21.15.16:0/2932599709 >> v1:172.21.15.186:6789/0 conn(0x7ff1c00d35b0 legacy=0x7ff1c00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1b7fff640 1 -- 172.21.15.16:0/2932599709 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c00d2ad0 con 0x7ff1c00b5b60 2024-04-25T10:07:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1b7fff640 1 -- 172.21.15.16:0/2932599709 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7ff1b80019f0 con 0x7ff1c00b5b60 2024-04-25T10:07:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1b7fff640 1 -- 172.21.15.16:0/2932599709 >> v1:172.21.15.16:6789/0 conn(0x7ff1c00b5b60 legacy=0x7ff1c00b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1b7fff640 1 --2- 172.21.15.16:0/2932599709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1c015dce0 0x7ff1ac003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1b7fff640 1 --2- 172.21.15.16:0/2932599709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1ac0040d0 0x7ff1ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1b7fff640 1 --2- 172.21.15.16:0/2932599709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1ac004d40 0x7ff1ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1b7fff640 1 -- 172.21.15.16:0/2932599709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c00d27f0 con 0x7ff1ac0040d0 2024-04-25T10:07:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1b7fff640 1 -- 172.21.15.16:0/2932599709 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac00baa0 con 0x7ff1ac004d40 2024-04-25T10:07:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1b7fff640 1 -- 172.21.15.16:0/2932599709 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac00bca0 con 0x7ff1c015dce0 2024-04-25T10:07:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1c4ad6640 1 --2- 172.21.15.16:0/2932599709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1ac0040d0 0x7ff1ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1c52d7640 1 --2- 172.21.15.16:0/2932599709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1c015dce0 0x7ff1ac003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1c5ad8640 1 --2- 172.21.15.16:0/2932599709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1ac004d40 0x7ff1ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1c4ad6640 1 -- 172.21.15.16:0/2932599709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1c015dce0 msgr2=0x7ff1ac003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1c4ad6640 1 --2- 172.21.15.16:0/2932599709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1c015dce0 0x7ff1ac003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1c4ad6640 1 -- 172.21.15.16:0/2932599709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1ac004d40 msgr2=0x7ff1ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1c4ad6640 1 --2- 172.21.15.16:0/2932599709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1ac004d40 0x7ff1ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.111+0000 7ff1c4ad6640 1 -- 172.21.15.16:0/2932599709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ac003520 con 0x7ff1ac0040d0 2024-04-25T10:07:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.112+0000 7ff1c4ad6640 1 --2- 172.21.15.16:0/2932599709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1ac0040d0 0x7ff1ac004800 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00018d0 tx=0x7ff1ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67f5acb4c49de3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.112+0000 7ff1b7fff640 1 -- 172.21.15.16:0/2932599709 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b00014f0 con 0x7ff1ac0040d0 2024-04-25T10:07:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.112+0000 7ff1b7fff640 1 -- 172.21.15.16:0/2932599709 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b00052d0 con 0x7ff1ac0040d0 2024-04-25T10:07:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.112+0000 7ff1b7fff640 1 -- 172.21.15.16:0/2932599709 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b002aca0 con 0x7ff1ac0040d0 2024-04-25T10:07:00.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.113+0000 7ff1c753b640 1 -- 172.21.15.16:0/2932599709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1ac0040d0 msgr2=0x7ff1ac004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.113+0000 7ff1c753b640 1 --2- 172.21.15.16:0/2932599709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1ac0040d0 0x7ff1ac004800 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00018d0 tx=0x7ff1ac005830 comp rx=0 tx=0).stop 2024-04-25T10:07:00.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.113+0000 7ff1c5ad8640 1 -- 172.21.15.16:0/2932599709 reap_dead start 2024-04-25T10:07:00.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.113+0000 7ff1c753b640 1 -- 172.21.15.16:0/2932599709 shutdown_connections 2024-04-25T10:07:00.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.113+0000 7ff1c753b640 1 -- 172.21.15.16:0/2932599709 >> 172.21.15.16:0/2932599709 conn(0x7ff1c0072900 msgr2=0x7ff1c00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.113+0000 7ff1c753b640 1 -- 172.21.15.16:0/2932599709 shutdown_connections 2024-04-25T10:07:00.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.113+0000 7ff1c753b640 1 -- 172.21.15.16:0/2932599709 wait complete. 2024-04-25T10:07:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.114+0000 7ff1c753b640 1 Processor -- start 2024-04-25T10:07:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.114+0000 7ff1c753b640 1 -- start start 2024-04-25T10:07:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.115+0000 7ff1c753b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1ac0040d0 0x7ff1c00cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.115+0000 7ff1c753b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c0153ad0 0x7ff1c0153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.115+0000 7ff1c753b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1c0143bb0 0x7ff1c0144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.115+0000 7ff1c753b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c00b9820 con 0x7ff1c0153ad0 2024-04-25T10:07:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.115+0000 7ff1c753b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c00d3090 con 0x7ff1ac0040d0 2024-04-25T10:07:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.115+0000 7ff1c753b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c00b99a0 con 0x7ff1c0143bb0 2024-04-25T10:07:00.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.115+0000 7ff1c4ad6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c0153ad0 0x7ff1c0153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.115+0000 7ff1c4ad6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c0153ad0 0x7ff1c0153f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50564/0 (socket says 172.21.15.16:50564) 2024-04-25T10:07:00.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.115+0000 7ff1c4ad6640 1 -- 172.21.15.16:0/3648077427 learned_addr learned my addr 172.21.15.16:0/3648077427 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:00.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.115+0000 7ff1c52d7640 1 --2- 172.21.15.16:0/3648077427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1ac0040d0 0x7ff1c00cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.115+0000 7ff1c5ad8640 1 --2- 172.21.15.16:0/3648077427 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1c0143bb0 0x7ff1c0144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.115+0000 7ff1c4ad6640 1 -- 172.21.15.16:0/3648077427 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1c0143bb0 msgr2=0x7ff1c0144030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.115+0000 7ff1c4ad6640 1 --2- 172.21.15.16:0/3648077427 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1c0143bb0 0x7ff1c0144030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.115+0000 7ff1c4ad6640 1 -- 172.21.15.16:0/3648077427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1ac0040d0 msgr2=0x7ff1c00cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.115+0000 7ff1c4ad6640 1 --2- 172.21.15.16:0/3648077427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1ac0040d0 0x7ff1c00cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.115+0000 7ff1c4ad6640 1 -- 172.21.15.16:0/3648077427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c00cb7b0 con 0x7ff1c0153ad0 2024-04-25T10:07:00.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.115+0000 7ff1c52d7640 1 --2- 172.21.15.16:0/3648077427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1ac0040d0 0x7ff1c00cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:00.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.115+0000 7ff1c4ad6640 1 --2- 172.21.15.16:0/3648077427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c0153ad0 0x7ff1c0153f50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00018d0 tx=0x7ff1b00012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.116+0000 7ff1b5ffb640 1 -- 172.21.15.16:0/3648077427 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b003f020 con 0x7ff1c0153ad0 2024-04-25T10:07:00.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.116+0000 7ff1b5ffb640 1 -- 172.21.15.16:0/3648077427 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b0006e50 con 0x7ff1c0153ad0 2024-04-25T10:07:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.116+0000 7ff1c753b640 1 -- 172.21.15.16:0/3648077427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c00cba90 con 0x7ff1c0153ad0 2024-04-25T10:07:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.116+0000 7ff1b5ffb640 1 -- 172.21.15.16:0/3648077427 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0037040 con 0x7ff1c0153ad0 2024-04-25T10:07:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.116+0000 7ff1c753b640 1 -- 172.21.15.16:0/3648077427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c00cc050 con 0x7ff1c0153ad0 2024-04-25T10:07:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.117+0000 7ff1c753b640 1 -- 172.21.15.16:0/3648077427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c00749b0 con 0x7ff1c0153ad0 2024-04-25T10:07:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.117+0000 7ff1b5ffb640 1 -- 172.21.15.16:0/3648077427 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff1b0042020 con 0x7ff1c0153ad0 2024-04-25T10:07:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.118+0000 7ff1b5ffb640 1 --2- 172.21.15.16:0/3648077427 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff1a406b4d0 0x7ff1a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.118+0000 7ff1b5ffb640 1 -- 172.21.15.16:0/3648077427 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1b00a9fa0 con 0x7ff1c0153ad0 2024-04-25T10:07:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.118+0000 7ff1c52d7640 1 --2- 172.21.15.16:0/3648077427 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff1a406b4d0 0x7ff1a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.118+0000 7ff1c52d7640 1 --2- 172.21.15.16:0/3648077427 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff1a406b4d0 0x7ff1a406d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0099e0 tx=0x7ff1bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.122+0000 7ff1b5ffb640 1 -- 172.21.15.16:0/3648077427 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff1b0049030 con 0x7ff1c0153ad0 2024-04-25T10:07:00.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.337+0000 7ff1c753b640 1 -- 172.21.15.16:0/3648077427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1c006cbf0 con 0x7ff1c0153ad0 2024-04-25T10:07:00.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.337+0000 7ff1b5ffb640 1 -- 172.21.15.16:0/3648077427 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4940 (secure 0 0 0) 0x7ff1b0078e20 con 0x7ff1c0153ad0 2024-04-25T10:07:00.343 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:00.343 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/791079479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":791079479},{"type":"v1","addr":"172.21.15.16:6835","nonce":791079479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1747921236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1747921236},{"type":"v1","addr":"172.21.15.16:6834","nonce":1747921236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":34880,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/3183779257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3183779257},{"type":"v1","addr":"172.21.15.186:6836","nonce":3183779257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:51.182691+0000","modified":"2024-04-25T10:06:53.179194+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34883},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34883":{"gid":34883,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/83423489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":83423489},{"type":"v1","addr":"172.21.15.186:6837","nonce":83423489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:00.343 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 21 2024-04-25T10:07:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.339+0000 7ff1c753b640 1 -- 172.21.15.16:0/3648077427 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff1a406b4d0 msgr2=0x7ff1a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.339+0000 7ff1c753b640 1 --2- 172.21.15.16:0/3648077427 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff1a406b4d0 0x7ff1a406d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0099e0 tx=0x7ff1bc008040 comp rx=0 tx=0).stop 2024-04-25T10:07:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.339+0000 7ff1c753b640 1 -- 172.21.15.16:0/3648077427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c0153ad0 msgr2=0x7ff1c0153f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.339+0000 7ff1c753b640 1 --2- 172.21.15.16:0/3648077427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c0153ad0 0x7ff1c0153f50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00018d0 tx=0x7ff1b00012d0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.339+0000 7ff1c753b640 1 -- 172.21.15.16:0/3648077427 shutdown_connections 2024-04-25T10:07:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.339+0000 7ff1c753b640 1 --2- 172.21.15.16:0/3648077427 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff1a406b4d0 0x7ff1a406d990 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.339+0000 7ff1c753b640 1 --2- 172.21.15.16:0/3648077427 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1c0143bb0 0x7ff1c0144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.339+0000 7ff1c753b640 1 --2- 172.21.15.16:0/3648077427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c0153ad0 0x7ff1c0153f50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.339+0000 7ff1c753b640 1 --2- 172.21.15.16:0/3648077427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1ac0040d0 0x7ff1c00cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.339+0000 7ff1c753b640 1 -- 172.21.15.16:0/3648077427 >> 172.21.15.16:0/3648077427 conn(0x7ff1c0072900 msgr2=0x7ff1c00b4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.339+0000 7ff1c753b640 1 -- 172.21.15.16:0/3648077427 shutdown_connections 2024-04-25T10:07:00.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.339+0000 7ff1c753b640 1 -- 172.21.15.16:0/3648077427 wait complete. 2024-04-25T10:07:00.759 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:06:51.182691+0000', 'modified': '2024-04-25T10:06:53.179194+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34883}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34883': {'gid': 34883, 'name': 'b', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.186:6837/83423489', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6835', 'nonce': 83423489}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 83423489}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:00.759 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:07:00.759 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:07:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.864+0000 7f33a95c8640 1 Processor -- start 2024-04-25T10:07:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.864+0000 7f33a95c8640 1 -- start start 2024-04-25T10:07:00.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.865+0000 7f33a95c8640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f33a40b9580 0x7f33a41618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.865+0000 7f33a95c8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a4162040 con 0x7f33a41610b0 2024-04-25T10:07:00.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.865+0000 7f33a95c8640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a4162320 con 0x7f33a40cef40 2024-04-25T10:07:00.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.865+0000 7f33a95c8640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a4161e10 con 0x7f33a40b9580 2024-04-25T10:07:00.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.865+0000 7f33a2ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f33a41610b0 0x7f33a40ce830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59540/0 (socket says 172.21.15.16:59540) 2024-04-25T10:07:00.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.865+0000 7f33a2ffd640 1 -- 172.21.15.16:0/1600615333 learned_addr learned my addr 172.21.15.16:0/1600615333 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:00.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.865+0000 7f33a37fe640 1 --2- 172.21.15.16:0/1600615333 >> v2:172.21.15.186:3300/0 conn(0x7f33a40b9580 0x7f33a41618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.865+0000 7f33a1ffb640 1 -- 172.21.15.16:0/1600615333 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 109254930 0 0) 0x7f33a4162040 con 0x7f33a41610b0 2024-04-25T10:07:00.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.865+0000 7f33a1ffb640 1 -- 172.21.15.16:0/1600615333 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f338c003520 con 0x7f33a41610b0 2024-04-25T10:07:00.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a1ffb640 1 -- 172.21.15.16:0/1600615333 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 674050848 0 0) 0x7f338c003520 con 0x7f33a41610b0 2024-04-25T10:07:00.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a1ffb640 1 -- 172.21.15.16:0/1600615333 >> v2:172.21.15.186:3300/0 conn(0x7f33a40b9580 msgr2=0x7f33a41618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a1ffb640 1 --2- 172.21.15.16:0/1600615333 >> v2:172.21.15.186:3300/0 conn(0x7f33a40b9580 0x7f33a41618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a1ffb640 1 -- 172.21.15.16:0/1600615333 >> v1:172.21.15.186:6789/0 conn(0x7f33a40cef40 legacy=0x7f33a40d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a1ffb640 1 -- 172.21.15.16:0/1600615333 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a4162600 con 0x7f33a41610b0 2024-04-25T10:07:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a1ffb640 1 -- 172.21.15.16:0/1600615333 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f33940019f0 con 0x7f33a41610b0 2024-04-25T10:07:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a1ffb640 1 -- 172.21.15.16:0/1600615333 >> v1:172.21.15.16:6789/0 conn(0x7f33a41610b0 legacy=0x7f33a40ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a1ffb640 1 --2- 172.21.15.16:0/1600615333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f338c0039b0 0x7f338c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a1ffb640 1 --2- 172.21.15.16:0/1600615333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338c004800 0x7f338c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a1ffb640 1 --2- 172.21.15.16:0/1600615333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f338c0051c0 0x7f338c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a1ffb640 1 -- 172.21.15.16:0/1600615333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c00c130 con 0x7f338c004800 2024-04-25T10:07:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a1ffb640 1 -- 172.21.15.16:0/1600615333 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c00c2e0 con 0x7f338c0039b0 2024-04-25T10:07:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a1ffb640 1 -- 172.21.15.16:0/1600615333 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338c00c4e0 con 0x7f338c0051c0 2024-04-25T10:07:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a27fc640 1 --2- 172.21.15.16:0/1600615333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338c004800 0x7f338c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a2ffd640 1 --2- 172.21.15.16:0/1600615333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f338c0039b0 0x7f338c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a37fe640 1 --2- 172.21.15.16:0/1600615333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f338c0051c0 0x7f338c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a27fc640 1 -- 172.21.15.16:0/1600615333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f338c0051c0 msgr2=0x7f338c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a27fc640 1 --2- 172.21.15.16:0/1600615333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f338c0051c0 0x7f338c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a27fc640 1 -- 172.21.15.16:0/1600615333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f338c0039b0 msgr2=0x7f338c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a27fc640 1 --2- 172.21.15.16:0/1600615333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f338c0039b0 0x7f338c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a37fe640 1 -- 172.21.15.16:0/1600615333 reap_dead start 2024-04-25T10:07:00.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.866+0000 7f33a27fc640 1 -- 172.21.15.16:0/1600615333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c003520 con 0x7f338c004800 2024-04-25T10:07:00.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.867+0000 7f33a27fc640 1 --2- 172.21.15.16:0/1600615333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338c004800 0x7f338c004c80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3398001750 tx=0x7f338c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481e0a3e37ee007e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.867+0000 7f33a1ffb640 1 -- 172.21.15.16:0/1600615333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398001370 con 0x7f338c004800 2024-04-25T10:07:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.867+0000 7f33a1ffb640 1 -- 172.21.15.16:0/1600615333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33980052d0 con 0x7f338c004800 2024-04-25T10:07:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.867+0000 7f33a1ffb640 1 -- 172.21.15.16:0/1600615333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339802abc0 con 0x7f338c004800 2024-04-25T10:07:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.868+0000 7f33a95c8640 1 -- 172.21.15.16:0/1600615333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338c004800 msgr2=0x7f338c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.868+0000 7f33a95c8640 1 --2- 172.21.15.16:0/1600615333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338c004800 0x7f338c004c80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3398001750 tx=0x7f338c005830 comp rx=0 tx=0).stop 2024-04-25T10:07:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.868+0000 7f33a95c8640 1 -- 172.21.15.16:0/1600615333 shutdown_connections 2024-04-25T10:07:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.868+0000 7f33a95c8640 1 --2- 172.21.15.16:0/1600615333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338c004800 0x7f338c004c80 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.868+0000 7f33a95c8640 1 -- 172.21.15.16:0/1600615333 >> 172.21.15.16:0/1600615333 conn(0x7f33a40b5570 msgr2=0x7f33a40b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.868+0000 7f33a95c8640 1 -- 172.21.15.16:0/1600615333 shutdown_connections 2024-04-25T10:07:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.868+0000 7f33a95c8640 1 -- 172.21.15.16:0/1600615333 wait complete. 2024-04-25T10:07:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.869+0000 7f33a95c8640 1 Processor -- start 2024-04-25T10:07:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.869+0000 7f33a95c8640 1 -- start start 2024-04-25T10:07:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.870+0000 7f33a95c8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f338c004800 0x7f33a40c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.870+0000 7f33a95c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33a414ea90 0x7f33a4154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.870+0000 7f33a95c8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33a415d390 0x7f33a415d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.870+0000 7f33a95c8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a40b99f0 con 0x7f33a414ea90 2024-04-25T10:07:00.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.870+0000 7f33a95c8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a40d1ea0 con 0x7f33a415d390 2024-04-25T10:07:00.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.870+0000 7f33a95c8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a40b6c00 con 0x7f338c004800 2024-04-25T10:07:00.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.870+0000 7f33a27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33a414ea90 0x7f33a4154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.870+0000 7f33a27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33a414ea90 0x7f33a4154de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50574/0 (socket says 172.21.15.16:50574) 2024-04-25T10:07:00.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.870+0000 7f33a27fc640 1 -- 172.21.15.16:0/2990582807 learned_addr learned my addr 172.21.15.16:0/2990582807 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:00.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.870+0000 7f33a2ffd640 1 --2- 172.21.15.16:0/2990582807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f338c004800 0x7f33a40c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.870+0000 7f33a37fe640 1 --2- 172.21.15.16:0/2990582807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33a415d390 0x7f33a415d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.870+0000 7f33a27fc640 1 -- 172.21.15.16:0/2990582807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f338c004800 msgr2=0x7f33a40c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.870+0000 7f33a27fc640 1 --2- 172.21.15.16:0/2990582807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f338c004800 0x7f33a40c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.870+0000 7f33a27fc640 1 -- 172.21.15.16:0/2990582807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33a415d390 msgr2=0x7f33a415d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.870+0000 7f33a27fc640 1 --2- 172.21.15.16:0/2990582807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33a415d390 0x7f33a415d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.870+0000 7f33a27fc640 1 -- 172.21.15.16:0/2990582807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a4155320 con 0x7f33a414ea90 2024-04-25T10:07:00.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.871+0000 7f33a27fc640 1 --2- 172.21.15.16:0/2990582807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33a414ea90 0x7f33a4154de0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3398000c00 tx=0x7f3398001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.871+0000 7f338b7fe640 1 -- 172.21.15.16:0/2990582807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398040030 con 0x7f33a414ea90 2024-04-25T10:07:00.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.871+0000 7f338b7fe640 1 -- 172.21.15.16:0/2990582807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339802be50 con 0x7f33a414ea90 2024-04-25T10:07:00.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.871+0000 7f33a95c8640 1 -- 172.21.15.16:0/2990582807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a414f200 con 0x7f33a414ea90 2024-04-25T10:07:00.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.871+0000 7f338b7fe640 1 -- 172.21.15.16:0/2990582807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398001370 con 0x7f33a414ea90 2024-04-25T10:07:00.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.871+0000 7f33a95c8640 1 -- 172.21.15.16:0/2990582807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a414f7c0 con 0x7f33a414ea90 2024-04-25T10:07:00.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.872+0000 7f338b7fe640 1 -- 172.21.15.16:0/2990582807 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3398043070 con 0x7f33a414ea90 2024-04-25T10:07:00.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.873+0000 7f338b7fe640 1 --2- 172.21.15.16:0/2990582807 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f339006b500 0x7f339006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.873+0000 7f338b7fe640 1 -- 172.21.15.16:0/2990582807 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33980a99e0 con 0x7f33a414ea90 2024-04-25T10:07:00.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.873+0000 7f33a2ffd640 1 --2- 172.21.15.16:0/2990582807 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f339006b500 0x7f339006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.874+0000 7f33a2ffd640 1 --2- 172.21.15.16:0/2990582807 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f339006b500 0x7f339006d9c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f33a40682b0 tx=0x7f3394006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.874+0000 7f33ac3a5740 1 Processor -- start 2024-04-25T10:07:00.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.874+0000 7f33ac3a5740 1 -- start start 2024-04-25T10:07:00.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.875+0000 7f33ac3a5740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a31943c230 0x55a31943c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.875+0000 7f33ac3a5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a319459ad0 0x55a31945a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.875+0000 7f33ac3a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319411f10 0x55a319458190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.875+0000 7f33ac3a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a319374a10 con 0x55a319411f10 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.875+0000 7f33ac3a5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31931db30 con 0x55a319459ad0 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.875+0000 7f33ac3a5740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a319370c50 con 0x55a31943c230 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.875+0000 7f33a37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319411f10 0x55a319458190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.875+0000 7f33a37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319411f10 0x55a319458190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50578/0 (socket says 172.21.15.16:50578) 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.875+0000 7f33a37fe640 1 -- 172.21.15.16:0/103867083 learned_addr learned my addr 172.21.15.16:0/103867083 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.875+0000 7f33a2ffd640 1 --2- 172.21.15.16:0/103867083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a31943c230 0x55a31943c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.875+0000 7f33a27fc640 1 --2- 172.21.15.16:0/103867083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a319459ad0 0x55a31945a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.875+0000 7f33a37fe640 1 -- 172.21.15.16:0/103867083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a31943c230 msgr2=0x55a31943c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.875+0000 7f33a37fe640 1 --2- 172.21.15.16:0/103867083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a31943c230 0x55a31943c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.875+0000 7f33a37fe640 1 -- 172.21.15.16:0/103867083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a319459ad0 msgr2=0x55a31945a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.875+0000 7f33a37fe640 1 --2- 172.21.15.16:0/103867083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a319459ad0 0x55a31945a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.875+0000 7f33a37fe640 1 -- 172.21.15.16:0/103867083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a319332e10 con 0x55a319411f10 2024-04-25T10:07:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.875+0000 7f33a2ffd640 1 --2- 172.21.15.16:0/103867083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a31943c230 0x55a31943c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.876+0000 7f33a37fe640 1 --2- 172.21.15.16:0/103867083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319411f10 0x55a319458190 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f33a40cfb90 tx=0x7f339c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e37efb67bdafeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.876+0000 7f3388ff9640 1 -- 172.21.15.16:0/103867083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c004240 con 0x55a319411f10 2024-04-25T10:07:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.876+0000 7f3388ff9640 1 -- 172.21.15.16:0/103867083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c0043e0 con 0x55a319411f10 2024-04-25T10:07:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.876+0000 7f3388ff9640 1 -- 172.21.15.16:0/103867083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c014b40 con 0x55a319411f10 2024-04-25T10:07:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.876+0000 7f33ac3a5740 1 -- 172.21.15.16:0/103867083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319411f10 msgr2=0x55a319458190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.876+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/103867083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319411f10 0x55a319458190 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f33a40cfb90 tx=0x7f339c009190 comp rx=0 tx=0).stop 2024-04-25T10:07:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.876+0000 7f33ac3a5740 1 -- 172.21.15.16:0/103867083 shutdown_connections 2024-04-25T10:07:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.876+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/103867083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319411f10 0x55a319458190 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.876+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/103867083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a319459ad0 0x55a31945a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.876+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/103867083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a31943c230 0x55a31943c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.876+0000 7f33ac3a5740 1 -- 172.21.15.16:0/103867083 >> 172.21.15.16:0/103867083 conn(0x55a31940e840 msgr2=0x55a319457040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.876+0000 7f33ac3a5740 1 -- 172.21.15.16:0/103867083 shutdown_connections 2024-04-25T10:07:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.876+0000 7f33ac3a5740 1 -- 172.21.15.16:0/103867083 wait complete. 2024-04-25T10:07:00.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.877+0000 7f33ac3a5740 1 -- start start 2024-04-25T10:07:00.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.877+0000 7f33ac3a5740 1 Processor -- start 2024-04-25T10:07:00.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f33ac3a5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a31943c230 0x55a31943ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f33ac3a5740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a319459ad0 0x55a31940b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f33ac3a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319435650 0x55a31940bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f33ac3a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a319412cb0 con 0x55a319435650 2024-04-25T10:07:00.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f33ac3a5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3193310c0 con 0x55a31943c230 2024-04-25T10:07:00.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f33ac3a5740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a31940c770 con 0x55a319459ad0 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f33ac3a5740 10 client.0 _ll_register_callbacks cb 0x55a319324ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f33a37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319435650 0x55a31940bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f33a37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319435650 0x55a31940bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50592/0 (socket says 172.21.15.16:50592) 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f33a37fe640 1 -- 172.21.15.16:0/2490265077 learned_addr learned my addr 172.21.15.16:0/2490265077 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f33a2ffd640 1 --2- 172.21.15.16:0/2490265077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a31943c230 0x55a31943ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f33a27fc640 1 --2- 172.21.15.16:0/2490265077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a319459ad0 0x55a31940b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f33a37fe640 1 -- 172.21.15.16:0/2490265077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a319459ad0 msgr2=0x55a31940b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f33a37fe640 1 --2- 172.21.15.16:0/2490265077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a319459ad0 0x55a31940b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f33a37fe640 1 -- 172.21.15.16:0/2490265077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a31943c230 msgr2=0x55a31943ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f33a37fe640 1 --2- 172.21.15.16:0/2490265077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a31943c230 0x55a31943ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f33a37fe640 1 -- 172.21.15.16:0/2490265077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3190892d0 con 0x55a319435650 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f33a37fe640 1 --2- 172.21.15.16:0/2490265077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319435650 0x55a31940bd10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f339c01a000 tx=0x7f339c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a747ef122a2409fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.878+0000 7f3381ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.879+0000 7f3381ffb640 1 -- 172.21.15.16:0/2490265077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c02e060 con 0x55a319435650 2024-04-25T10:07:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.879+0000 7f3381ffb640 1 -- 172.21.15.16:0/2490265077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f339c014b50 con 0x55a319435650 2024-04-25T10:07:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.879+0000 7f33ac3a5740 1 -- 172.21.15.16:0/2490265077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f339c0043e0 con 0x55a319435650 2024-04-25T10:07:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.879+0000 7f3381ffb640 1 -- 172.21.15.16:0/2490265077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f339c014d50 con 0x55a319435650 2024-04-25T10:07:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.879+0000 7f33ac3a5740 10 client.5769 fetch_fsmap learned FSMap version 21 2024-04-25T10:07:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.879+0000 7f33ac3a5740 1 -- 172.21.15.16:0/2490265077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55a3191536b0 con 0x55a319435650 2024-04-25T10:07:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.879+0000 7f3381ffb640 1 -- 172.21.15.16:0/2490265077 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c0043e0 con 0x55a319435650 2024-04-25T10:07:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.879+0000 7f3381ffb640 1 -- 172.21.15.16:0/2490265077 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f339c003610 con 0x55a319435650 2024-04-25T10:07:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.879+0000 7f33ac3a5740 10 client.5769 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T10:07:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.879+0000 7f33ac3a5740 10 client.5769 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:07:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.879+0000 7f33ac3a5740 20 client.5769 populate_metadata read hostname 'smithi016' 2024-04-25T10:07:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.879+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/2490265077 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55a31944eac0 0x55a31944eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.879+0000 7f33ac3a5740 4 client.5769 mds_command: new command op to 34883 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.879+0000 7f33ac3a5740 1 -- 172.21.15.16:0/2490265077 --> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3193d3920 con 0x55a31944eac0 2024-04-25T10:07:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.879+0000 7f33a2ffd640 1 --2- 172.21.15.16:0/2490265077 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55a31944eac0 0x55a31944eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.880+0000 7f33a2ffd640 1 --2- 172.21.15.16:0/2490265077 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55a31944eac0 0x55a31944eea0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9c21c3a6690d3bb server_cookie=a8d50df253bb9852 in_seq=0 out_seq=0 2024-04-25T10:07:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.880+0000 7f3381ffb640 10 client.5769 ms_handle_connect on v2:172.21.15.186:6835/83423489 2024-04-25T10:07:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.880+0000 7f3381ffb640 1 -- 172.21.15.16:0/2490265077 <== mds.0 v2:172.21.15.186:6835/83423489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a3193d3920 con 0x55a31944eac0 2024-04-25T10:07:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.881+0000 7f3381ffb640 10 client.5769 handle_command_reply: tid=0 2024-04-25T10:07:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.881+0000 7f33ac3a5740 1 client.5769 shutdown 2024-04-25T10:07:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.881+0000 7f33ac3a5740 1 -- 172.21.15.16:0/2490265077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319435650 msgr2=0x55a31940bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.881+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/2490265077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319435650 0x55a31940bd10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f339c01a000 tx=0x7f339c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.881+0000 7f33ac3a5740 1 -- 172.21.15.16:0/2490265077 shutdown_connections 2024-04-25T10:07:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.881+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/2490265077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319435650 0x55a31940bd10 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.881+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/2490265077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a319459ad0 0x55a31940b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.881+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/2490265077 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55a31944eac0 0x55a31944eea0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.881+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/2490265077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a31943c230 0x55a31943ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.881+0000 7f33ac3a5740 1 -- 172.21.15.16:0/2490265077 >> 172.21.15.16:0/2490265077 conn(0x55a319411f10 msgr2=0x55a31945a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.881+0000 7f3381ffb640 0 client.5769 ms_handle_reset on v2:172.21.15.186:6835/83423489 2024-04-25T10:07:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.881+0000 7f33ac3a5740 1 -- 172.21.15.16:0/2490265077 shutdown_connections 2024-04-25T10:07:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.881+0000 7f33ac3a5740 1 -- 172.21.15.16:0/2490265077 wait complete. 2024-04-25T10:07:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.881+0000 7f33ac3a5740 20 client.5769 trim_cache size 0 max 16384 2024-04-25T10:07:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.897+0000 7f33ac3a5740 1 Processor -- start 2024-04-25T10:07:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.897+0000 7f33ac3a5740 1 -- start start 2024-04-25T10:07:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.897+0000 7f33ac3a5740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a319435650 0x55a31940b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.897+0000 7f33ac3a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a31943c230 0x55a31940bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.897+0000 7f33ac3a5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a319459ad0 0x55a31943aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.898+0000 7f33ac3a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a4200660 con 0x55a31943c230 2024-04-25T10:07:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.898+0000 7f33ac3a5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a41ff890 con 0x55a319459ad0 2024-04-25T10:07:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.898+0000 7f33ac3a5740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a40cb800 con 0x55a319435650 2024-04-25T10:07:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.898+0000 7f33a27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a31943c230 0x55a31940bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.898+0000 7f33a27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a31943c230 0x55a31940bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50598/0 (socket says 172.21.15.16:50598) 2024-04-25T10:07:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.898+0000 7f33a27fc640 1 -- 172.21.15.16:0/223523026 learned_addr learned my addr 172.21.15.16:0/223523026 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.898+0000 7f33a27fc640 1 -- 172.21.15.16:0/223523026 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a319435650 msgr2=0x55a31940b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.898+0000 7f33a2ffd640 1 --2- 172.21.15.16:0/223523026 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a319435650 0x55a31940b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.898+0000 7f33a27fc640 1 --2- 172.21.15.16:0/223523026 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a319435650 0x55a31940b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.898+0000 7f33a37fe640 1 --2- 172.21.15.16:0/223523026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a319459ad0 0x55a31943aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.898+0000 7f33a27fc640 1 -- 172.21.15.16:0/223523026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a319459ad0 msgr2=0x55a31943aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.898+0000 7f33a27fc640 1 --2- 172.21.15.16:0/223523026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a319459ad0 0x55a31943aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.898+0000 7f33a27fc640 1 -- 172.21.15.16:0/223523026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a31943e8c0 con 0x55a31943c230 2024-04-25T10:07:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.898+0000 7f33a27fc640 1 --2- 172.21.15.16:0/223523026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a31943c230 0x55a31940bc50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3398001750 tx=0x7f3398009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cffbf0588ad428e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.898+0000 7f3383fff640 1 -- 172.21.15.16:0/223523026 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398049070 con 0x55a31943c230 2024-04-25T10:07:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.898+0000 7f3383fff640 1 -- 172.21.15.16:0/223523026 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33980780d0 con 0x55a31943c230 2024-04-25T10:07:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.898+0000 7f3383fff640 1 -- 172.21.15.16:0/223523026 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339807c220 con 0x55a31943c230 2024-04-25T10:07:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.899+0000 7f33ac3a5740 1 -- 172.21.15.16:0/223523026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a31943c230 msgr2=0x55a31940bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.899+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/223523026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a31943c230 0x55a31940bc50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3398001750 tx=0x7f3398009040 comp rx=0 tx=0).stop 2024-04-25T10:07:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.899+0000 7f33ac3a5740 1 -- 172.21.15.16:0/223523026 shutdown_connections 2024-04-25T10:07:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.899+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/223523026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a319459ad0 0x55a31943aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.899+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/223523026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a31943c230 0x55a31940bc50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.899+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/223523026 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a319435650 0x55a31940b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.899+0000 7f33ac3a5740 1 -- 172.21.15.16:0/223523026 >> 172.21.15.16:0/223523026 conn(0x55a319411f10 msgr2=0x55a319407db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.899+0000 7f33ac3a5740 1 -- 172.21.15.16:0/223523026 shutdown_connections 2024-04-25T10:07:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.899+0000 7f33ac3a5740 1 -- 172.21.15.16:0/223523026 wait complete. 2024-04-25T10:07:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.899+0000 7f33ac3a5740 1 -- start start 2024-04-25T10:07:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.899+0000 7f33ac3a5740 1 Processor -- start 2024-04-25T10:07:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.900+0000 7f33ac3a5740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a319435650 0x55a319457d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.900+0000 7f33ac3a5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a31943c230 0x55a3194101c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.900+0000 7f33ac3a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319459ad0 0x55a319410700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.900+0000 7f33ac3a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a31944f670 con 0x55a319459ad0 2024-04-25T10:07:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.900+0000 7f33ac3a5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a40cbe00 con 0x55a31943c230 2024-04-25T10:07:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.900+0000 7f33ac3a5740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a31946b540 con 0x55a319435650 2024-04-25T10:07:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.900+0000 7f33ac3a5740 10 client.0 _ll_register_callbacks cb 0x55a31940cc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.900+0000 7f33a37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319459ad0 0x55a319410700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.900+0000 7f33a37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319459ad0 0x55a319410700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50606/0 (socket says 172.21.15.16:50606) 2024-04-25T10:07:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.900+0000 7f33a37fe640 1 -- 172.21.15.16:0/2745108789 learned_addr learned my addr 172.21.15.16:0/2745108789 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.900+0000 7f33a2ffd640 1 --2- 172.21.15.16:0/2745108789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a319435650 0x55a319457d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.900+0000 7f33a27fc640 1 --2- 172.21.15.16:0/2745108789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a31943c230 0x55a3194101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f33a37fe640 1 -- 172.21.15.16:0/2745108789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a319435650 msgr2=0x55a319457d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f33a37fe640 1 --2- 172.21.15.16:0/2745108789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a319435650 0x55a319457d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f33a37fe640 1 -- 172.21.15.16:0/2745108789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a31943c230 msgr2=0x55a3194101c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f33a37fe640 1 --2- 172.21.15.16:0/2745108789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a31943c230 0x55a3194101c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f33a37fe640 1 -- 172.21.15.16:0/2745108789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a31946a3f0 con 0x55a319459ad0 2024-04-25T10:07:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f33a37fe640 1 --2- 172.21.15.16:0/2745108789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319459ad0 0x55a319410700 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f339c00b010 tx=0x7f339c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f036c52178476b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f3381ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:07:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f3381ffb640 1 -- 172.21.15.16:0/2745108789 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c014b50 con 0x55a319459ad0 2024-04-25T10:07:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f3381ffb640 1 -- 172.21.15.16:0/2745108789 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f339c004240 con 0x55a319459ad0 2024-04-25T10:07:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f33ac3a5740 1 -- 172.21.15.16:0/2745108789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33980780d0 con 0x55a319459ad0 2024-04-25T10:07:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f3381ffb640 1 -- 172.21.15.16:0/2745108789 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f339c02e030 con 0x55a319459ad0 2024-04-25T10:07:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f33ac3a5740 10 client.5775 fetch_fsmap learned FSMap version 21 2024-04-25T10:07:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f33ac3a5740 1 -- 172.21.15.16:0/2745108789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55a3194582e0 con 0x55a319459ad0 2024-04-25T10:07:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f3381ffb640 1 -- 172.21.15.16:0/2745108789 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33980780d0 con 0x55a319459ad0 2024-04-25T10:07:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f3381ffb640 1 -- 172.21.15.16:0/2745108789 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f339c038480 con 0x55a319459ad0 2024-04-25T10:07:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f33ac3a5740 10 client.5775 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T10:07:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f33ac3a5740 10 client.5775 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:07:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f33ac3a5740 20 client.5775 populate_metadata read hostname 'smithi016' 2024-04-25T10:07:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/2745108789 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55a319455ab0 0x55a3194544d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f33ac3a5740 4 client.5775 mds_command: new command op to 34883 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:07:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.901+0000 7f33ac3a5740 1 -- 172.21.15.16:0/2745108789 --> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a319381f50 con 0x55a319455ab0 2024-04-25T10:07:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.902+0000 7f33a2ffd640 1 --2- 172.21.15.16:0/2745108789 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55a319455ab0 0x55a3194544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.902+0000 7f3381ffb640 10 client.5775 ms_handle_connect on v2:172.21.15.186:6835/83423489 2024-04-25T10:07:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.902+0000 7f33a2ffd640 1 --2- 172.21.15.16:0/2745108789 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55a319455ab0 0x55a3194544d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5055d740a5f79325 server_cookie=7c662c923e59a1b2 in_seq=0 out_seq=0 2024-04-25T10:07:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.902+0000 7f3381ffb640 1 -- 172.21.15.16:0/2745108789 <== mds.0 v2:172.21.15.186:6835/83423489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55a319381f50 con 0x55a319455ab0 2024-04-25T10:07:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.902+0000 7f3381ffb640 10 client.5775 handle_command_reply: tid=0 2024-04-25T10:07:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.902+0000 7f33ac3a5740 1 client.5775 shutdown 2024-04-25T10:07:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.902+0000 7f33ac3a5740 1 -- 172.21.15.16:0/2745108789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319459ad0 msgr2=0x55a319410700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.902+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/2745108789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319459ad0 0x55a319410700 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f339c00b010 tx=0x7f339c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.902+0000 7f33ac3a5740 1 -- 172.21.15.16:0/2745108789 shutdown_connections 2024-04-25T10:07:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.902+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/2745108789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a319459ad0 0x55a319410700 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.902+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/2745108789 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55a319455ab0 0x55a3194544d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.902+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/2745108789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a31943c230 0x55a3194101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.902+0000 7f3381ffb640 0 client.5775 ms_handle_reset on v2:172.21.15.186:6835/83423489 2024-04-25T10:07:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.902+0000 7f33ac3a5740 1 --2- 172.21.15.16:0/2745108789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a319435650 0x55a319457d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.902+0000 7f33ac3a5740 1 -- 172.21.15.16:0/2745108789 >> 172.21.15.16:0/2745108789 conn(0x55a319411f10 msgr2=0x55a31940b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.902+0000 7f33ac3a5740 1 -- 172.21.15.16:0/2745108789 shutdown_connections 2024-04-25T10:07:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.902+0000 7f33ac3a5740 1 -- 172.21.15.16:0/2745108789 wait complete. 2024-04-25T10:07:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.903+0000 7f33ac3a5740 20 client.5775 trim_cache size 0 max 16384 2024-04-25T10:07:00.916 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:07:00.916 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "34bc5eca-3a21-42a6-a26f-07da0905e8bc", 2024-04-25T10:07:00.916 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-04-25T10:07:00.916 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 34883, 2024-04-25T10:07:00.916 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-04-25T10:07:00.916 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-04-25T10:07:00.916 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-04-25T10:07:00.916 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 9.0736541380000002, 2024-04-25T10:07:00.916 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 21, 2024-04-25T10:07:00.916 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 47, 2024-04-25T10:07:00.916 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 47, 2024-04-25T10:07:00.916 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 14.102860521 2024-04-25T10:07:00.917 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:07:00.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.903+0000 7f33a95c8640 1 -- 172.21.15.16:0/2990582807 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f339006b500 msgr2=0x7f339006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.903+0000 7f33a95c8640 1 --2- 172.21.15.16:0/2990582807 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f339006b500 0x7f339006d9c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f33a40682b0 tx=0x7f3394006040 comp rx=0 tx=0).stop 2024-04-25T10:07:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.903+0000 7f33a95c8640 1 -- 172.21.15.16:0/2990582807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33a414ea90 msgr2=0x7f33a4154de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.903+0000 7f33a95c8640 1 --2- 172.21.15.16:0/2990582807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33a414ea90 0x7f33a4154de0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3398000c00 tx=0x7f3398001120 comp rx=0 tx=0).stop 2024-04-25T10:07:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.903+0000 7f33a95c8640 1 -- 172.21.15.16:0/2990582807 shutdown_connections 2024-04-25T10:07:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.903+0000 7f33a95c8640 1 --2- 172.21.15.16:0/2990582807 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f339006b500 0x7f339006d9c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.903+0000 7f33a95c8640 1 --2- 172.21.15.16:0/2990582807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33a415d390 0x7f33a415d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.903+0000 7f33a95c8640 1 --2- 172.21.15.16:0/2990582807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33a414ea90 0x7f33a4154de0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.903+0000 7f33a95c8640 1 --2- 172.21.15.16:0/2990582807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f338c004800 0x7f33a40c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.903+0000 7f33a95c8640 1 -- 172.21.15.16:0/2990582807 >> 172.21.15.16:0/2990582807 conn(0x7f33a40b5570 msgr2=0x7f33a40b6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.903+0000 7f33a95c8640 1 -- 172.21.15.16:0/2990582807 shutdown_connections 2024-04-25T10:07:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:00.903+0000 7f33a95c8640 1 -- 172.21.15.16:0/2990582807 wait complete. 2024-04-25T10:07:00.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:07:01.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.014+0000 7f7db023f640 1 Processor -- start 2024-04-25T10:07:01.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.015+0000 7f7db023f640 1 -- start start 2024-04-25T10:07:01.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.015+0000 7f7db023f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7da815dc30 0x7f7da806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.015+0000 7f7db023f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7da80b1b30 0x7f7da80d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.015+0000 7f7db023f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da806c1f0 con 0x7f7da806cc00 2024-04-25T10:07:01.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.015+0000 7f7db023f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da80cfb40 con 0x7f7da815dc30 2024-04-25T10:07:01.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.015+0000 7f7db023f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da80b2060 con 0x7f7da80b1b30 2024-04-25T10:07:01.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.016+0000 7f7dadfdb640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7da815dc30 0x7f7da806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.016+0000 7f7dae7dc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7da80b1b30 0x7f7da80d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.016+0000 7f7dad7da640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f7da806cc00 0x7f7da80b1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:46296/0 (socket says 172.21.15.16:46296) 2024-04-25T10:07:01.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.016+0000 7f7dadfdb640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7da815dc30 0x7f7da806c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:59714/0 (socket says 172.21.15.16:59714) 2024-04-25T10:07:01.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.016+0000 7f7dad7da640 1 -- 172.21.15.16:0/899924000 learned_addr learned my addr 172.21.15.16:0/899924000 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:01.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.016+0000 7f7dae7dc640 1 -- 172.21.15.16:0/899924000 >> v2:172.21.15.186:3300/0 conn(0x7f7da815dc30 msgr2=0x7f7da806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.016+0000 7f7dae7dc640 1 --2- 172.21.15.16:0/899924000 >> v2:172.21.15.186:3300/0 conn(0x7f7da815dc30 0x7f7da806c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.016+0000 7f7dae7dc640 1 -- 172.21.15.16:0/899924000 >> v1:172.21.15.186:6789/0 conn(0x7f7da806cc00 legacy=0x7f7da80b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.016+0000 7f7dae7dc640 1 -- 172.21.15.16:0/899924000 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da80b2340 con 0x7f7da80b1b30 2024-04-25T10:07:01.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.017+0000 7f7dae7dc640 1 --2- 172.21.15.16:0/899924000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7da80b1b30 0x7f7da80d0300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009d90 tx=0x7f7da002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de5c9930292b8e19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.017+0000 7f7dacfd9640 1 -- 172.21.15.16:0/899924000 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da0007740 con 0x7f7da80b1b30 2024-04-25T10:07:01.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.017+0000 7f7dacfd9640 1 -- 172.21.15.16:0/899924000 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da00078e0 con 0x7f7da80b1b30 2024-04-25T10:07:01.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.017+0000 7f7dacfd9640 1 -- 172.21.15.16:0/899924000 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da0007cb0 con 0x7f7da80b1b30 2024-04-25T10:07:01.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.018+0000 7f7db023f640 1 -- 172.21.15.16:0/899924000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7da80b1b30 msgr2=0x7f7da80d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.018+0000 7f7db023f640 1 --2- 172.21.15.16:0/899924000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7da80b1b30 0x7f7da80d0300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009d90 tx=0x7f7da002a660 comp rx=0 tx=0).stop 2024-04-25T10:07:01.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.018+0000 7f7db023f640 1 -- 172.21.15.16:0/899924000 shutdown_connections 2024-04-25T10:07:01.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.018+0000 7f7db023f640 1 --2- 172.21.15.16:0/899924000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7da80b1b30 0x7f7da80d0300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.018+0000 7f7db023f640 1 --2- 172.21.15.16:0/899924000 >> v2:172.21.15.186:3300/0 conn(0x7f7da815dc30 0x7f7da806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.018+0000 7f7db023f640 1 -- 172.21.15.16:0/899924000 >> 172.21.15.16:0/899924000 conn(0x7f7da80d1900 msgr2=0x7f7da80d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.018+0000 7f7db023f640 1 -- 172.21.15.16:0/899924000 shutdown_connections 2024-04-25T10:07:01.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.018+0000 7f7db023f640 1 -- 172.21.15.16:0/899924000 wait complete. 2024-04-25T10:07:01.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.019+0000 7f7db023f640 1 Processor -- start 2024-04-25T10:07:01.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.019+0000 7f7db023f640 1 -- start start 2024-04-25T10:07:01.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7db023f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7da806cc00 0x7f7da80c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7db023f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da80b1b30 0x7f7da81f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7db023f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da815dc30 0x7f7da81f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7db023f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da815e010 con 0x7f7da815dc30 2024-04-25T10:07:01.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7db023f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da80cfde0 con 0x7f7da80b1b30 2024-04-25T10:07:01.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7db023f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da80d0840 con 0x7f7da806cc00 2024-04-25T10:07:01.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7dae7dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da815dc30 0x7f7da81f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7dae7dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da815dc30 0x7f7da81f7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50618/0 (socket says 172.21.15.16:50618) 2024-04-25T10:07:01.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7dae7dc640 1 -- 172.21.15.16:0/987482642 learned_addr learned my addr 172.21.15.16:0/987482642 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:01.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7dad7da640 1 --2- 172.21.15.16:0/987482642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da80b1b30 0x7f7da81f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7dadfdb640 1 --2- 172.21.15.16:0/987482642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7da806cc00 0x7f7da80c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7dae7dc640 1 -- 172.21.15.16:0/987482642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7da806cc00 msgr2=0x7f7da80c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7dae7dc640 1 --2- 172.21.15.16:0/987482642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7da806cc00 0x7f7da80c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7dae7dc640 1 -- 172.21.15.16:0/987482642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da80b1b30 msgr2=0x7f7da81f6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7dae7dc640 1 --2- 172.21.15.16:0/987482642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da80b1b30 0x7f7da81f6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7dae7dc640 1 -- 172.21.15.16:0/987482642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da80cb610 con 0x7f7da815dc30 2024-04-25T10:07:01.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7dadfdb640 1 --2- 172.21.15.16:0/987482642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7da806cc00 0x7f7da80c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:01.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7dad7da640 1 --2- 172.21.15.16:0/987482642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da80b1b30 0x7f7da81f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:01.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.020+0000 7f7dae7dc640 1 --2- 172.21.15.16:0/987482642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da815dc30 0x7f7da81f7220 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7da002ac50 tx=0x7f7da0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.021+0000 7f7d9e7fc640 1 -- 172.21.15.16:0/987482642 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da0040070 con 0x7f7da815dc30 2024-04-25T10:07:01.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.021+0000 7f7d9e7fc640 1 -- 172.21.15.16:0/987482642 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da0031c60 con 0x7f7da815dc30 2024-04-25T10:07:01.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.021+0000 7f7db023f640 1 -- 172.21.15.16:0/987482642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da80c6650 con 0x7f7da815dc30 2024-04-25T10:07:01.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.021+0000 7f7d9e7fc640 1 -- 172.21.15.16:0/987482642 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da0009450 con 0x7f7da815dc30 2024-04-25T10:07:01.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.021+0000 7f7db023f640 1 -- 172.21.15.16:0/987482642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da80c6c10 con 0x7f7da815dc30 2024-04-25T10:07:01.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.022+0000 7f7db023f640 1 -- 172.21.15.16:0/987482642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da80749b0 con 0x7f7da815dc30 2024-04-25T10:07:01.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.022+0000 7f7d9e7fc640 1 -- 172.21.15.16:0/987482642 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7da0039040 con 0x7f7da815dc30 2024-04-25T10:07:01.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.022+0000 7f7d9e7fc640 1 --2- 172.21.15.16:0/987482642 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7d8c06b4d0 0x7f7d8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.023+0000 7f7d9e7fc640 1 -- 172.21.15.16:0/987482642 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7da00ab400 con 0x7f7da815dc30 2024-04-25T10:07:01.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.023+0000 7f7dadfdb640 1 --2- 172.21.15.16:0/987482642 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7d8c06b4d0 0x7f7d8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.023+0000 7f7dadfdb640 1 --2- 172.21.15.16:0/987482642 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7d8c06b4d0 0x7f7d8c06d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009a20 tx=0x7f7da4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.027+0000 7f7d9e7fc640 1 -- 172.21.15.16:0/987482642 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7da0042310 con 0x7f7da815dc30 2024-04-25T10:07:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.243+0000 7f7db023f640 1 -- 172.21.15.16:0/987482642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7da80d1e60 con 0x7f7da815dc30 2024-04-25T10:07:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.243+0000 7f7d9e7fc640 1 -- 172.21.15.16:0/987482642 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7da007a160 con 0x7f7da815dc30 2024-04-25T10:07:01.249 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:07:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.245+0000 7f7db023f640 1 -- 172.21.15.16:0/987482642 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7d8c06b4d0 msgr2=0x7f7d8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.245+0000 7f7db023f640 1 --2- 172.21.15.16:0/987482642 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7d8c06b4d0 0x7f7d8c06d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009a20 tx=0x7f7da4008040 comp rx=0 tx=0).stop 2024-04-25T10:07:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.245+0000 7f7db023f640 1 -- 172.21.15.16:0/987482642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da815dc30 msgr2=0x7f7da81f7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.245+0000 7f7db023f640 1 --2- 172.21.15.16:0/987482642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da815dc30 0x7f7da81f7220 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7da002ac50 tx=0x7f7da0004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.245+0000 7f7db023f640 1 -- 172.21.15.16:0/987482642 shutdown_connections 2024-04-25T10:07:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.245+0000 7f7db023f640 1 --2- 172.21.15.16:0/987482642 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7d8c06b4d0 0x7f7d8c06d990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.245+0000 7f7db023f640 1 --2- 172.21.15.16:0/987482642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da815dc30 0x7f7da81f7220 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.245+0000 7f7db023f640 1 --2- 172.21.15.16:0/987482642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da80b1b30 0x7f7da81f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.245+0000 7f7db023f640 1 --2- 172.21.15.16:0/987482642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7da806cc00 0x7f7da80c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.245+0000 7f7db023f640 1 -- 172.21.15.16:0/987482642 >> 172.21.15.16:0/987482642 conn(0x7f7da80d1900 msgr2=0x7f7da80ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.245+0000 7f7db023f640 1 -- 172.21.15.16:0/987482642 shutdown_connections 2024-04-25T10:07:01.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:01.245+0000 7f7db023f640 1 -- 172.21.15.16:0/987482642 wait complete. 2024-04-25T10:07:01.261 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:07:01.261 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:07:01.261 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-04-25T10:07:01.261 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:07:01.262 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:01.262 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:07:01.262 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:07:01.262 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:07:01.262 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:01.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:01.262 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-04-25T10:07:01.278 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:07:01.278 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:07:01.278 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:07:01.278 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:01.278 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:07:01.278 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:01.278 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:01.278 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:01.278 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-04-25T10:07:01.279 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:01.279 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:01.279 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-04-25T10:07:01.279 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:07:01.279 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:01.279 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-04-25T10:07:01.279 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:07:01.279 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42539sec preferred_lft 42539sec 2024-04-25T10:07:01.279 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-25T10:07:01.279 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:01.279 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:01.279 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:01.279 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-04-25T10:07:01.279 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:07:01.280 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:01.280 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:07:01.280 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:01.280 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f06b:fdff:fed4:a335/64 scope link 2024-04-25T10:07:01.280 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:01.280 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:01.280 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:01.280 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::90ad:4fff:fe91:3c33/64 scope link 2024-04-25T10:07:01.280 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:01.280 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:01.280 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether be:14:4a:3c:16:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:01.280 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::bc14:4aff:fe3c:1605/64 scope link 2024-04-25T10:07:01.280 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:01.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:01.281 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:07:01.341 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:07:01.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:01.343 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-04-25T10:07:01.399 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-04-25T10:07:01.399 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:07:01.399 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:07:01.400 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:07:01.400 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:07:01.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:01.400 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:07:01.400 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:07:01.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:07:01.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:07:01.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:07:01.400 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:07:01.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:01.484 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:07:01.500 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:07:01.500 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:07:01.500 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:01.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:01.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-04-25T10:07:01.565 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:01.565 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:01.620 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:01.621 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:01.621 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:01.676 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:07:01.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:01.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:01.758 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:01.759 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:07:01.759 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:07:01.767 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:01.767 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:07:01.767 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:07:01.768 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:07:01.768 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:07:01.773 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:01.773 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:07:01.773 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:07:01.773 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:07:01.773 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:07:01.773 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:07:01.773 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:07:01.773 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:07:01.774 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:07:01.774 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:07:01.774 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:07:01.774 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:07:01.774 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:07:01.775 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:07:01.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:01.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:01.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:01.841 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:07:01.903 INFO:teuthology.orchestra.run.smithi016.stdout:client5398 2024-04-25T10:07:01.904 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:07:01.904 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:01.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:01.959 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-04-25T10:07:02.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:02.020 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-04-25T10:07:02.078 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:07:02.078 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:07:02.078 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:07:02.078 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:07:02.078 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-04-25T10:07:02.078 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:07:02.185 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627777 2024-04-25T10:07:02.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:07:02.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:02.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.330+0000 7f55f0b3b640 1 Processor -- start 2024-04-25T10:07:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.330+0000 7f55f0b3b640 1 -- start start 2024-04-25T10:07:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.330+0000 7f55f0b3b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55ec15ba10 0x7f55ec099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.331+0000 7f55f0b3b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f55ec06c3c0 0x7f55ec0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.331+0000 7f55f0b3b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55ec06c900 con 0x7f55ec099f80 2024-04-25T10:07:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.331+0000 7f55f0b3b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55ec0cbe50 con 0x7f55ec15ba10 2024-04-25T10:07:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.331+0000 7f55f0b3b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55ec09a3d0 con 0x7f55ec06c3c0 2024-04-25T10:07:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.331+0000 7f55ea59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55ec15ba10 0x7f55ec099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.331+0000 7f55e1d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f55ec099f80 0x7f55ec06bcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59554/0 (socket says 172.21.15.16:59554) 2024-04-25T10:07:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.331+0000 7f55e1d9b640 1 -- 172.21.15.16:0/591630344 learned_addr learned my addr 172.21.15.16:0/591630344 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.331+0000 7f55ead9d640 1 --2- 172.21.15.16:0/591630344 >> v2:172.21.15.186:3300/0 conn(0x7f55ec06c3c0 0x7f55ec0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.331+0000 7f55ea59c640 1 -- 172.21.15.16:0/591630344 >> v2:172.21.15.186:3300/0 conn(0x7f55ec06c3c0 msgr2=0x7f55ec0cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.331+0000 7f55ea59c640 1 --2- 172.21.15.16:0/591630344 >> v2:172.21.15.186:3300/0 conn(0x7f55ec06c3c0 0x7f55ec0cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.331+0000 7f55ea59c640 1 -- 172.21.15.16:0/591630344 >> v1:172.21.15.16:6789/0 conn(0x7f55ec099f80 legacy=0x7f55ec06bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.331+0000 7f55ea59c640 1 -- 172.21.15.16:0/591630344 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec06cbe0 con 0x7f55ec15ba10 2024-04-25T10:07:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.331+0000 7f55e9d9b640 1 -- 172.21.15.16:0/591630344 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3764494983 0 0) 0x7f55ec06c900 con 0x7f55ec099f80 2024-04-25T10:07:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.331+0000 7f55ead9d640 1 --2- 172.21.15.16:0/591630344 >> v2:172.21.15.186:3300/0 conn(0x7f55ec06c3c0 0x7f55ec0cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.332+0000 7f55ea59c640 1 --2- 172.21.15.16:0/591630344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec15ba10 0x7f55ec099a40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0b7850 tx=0x7f55dc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86aea3964c33610d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.332+0000 7f55e9d9b640 1 -- 172.21.15.16:0/591630344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc037070 con 0x7f55ec15ba10 2024-04-25T10:07:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.332+0000 7f55e9d9b640 1 -- 172.21.15.16:0/591630344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc032440 con 0x7f55ec15ba10 2024-04-25T10:07:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.332+0000 7f55e9d9b640 1 -- 172.21.15.16:0/591630344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc032810 con 0x7f55ec15ba10 2024-04-25T10:07:02.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.332+0000 7f55f0b3b640 1 -- 172.21.15.16:0/591630344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec15ba10 msgr2=0x7f55ec099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.332+0000 7f55f0b3b640 1 --2- 172.21.15.16:0/591630344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec15ba10 0x7f55ec099a40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0b7850 tx=0x7f55dc02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.332+0000 7f55f0b3b640 1 -- 172.21.15.16:0/591630344 shutdown_connections 2024-04-25T10:07:02.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.332+0000 7f55f0b3b640 1 --2- 172.21.15.16:0/591630344 >> v2:172.21.15.186:3300/0 conn(0x7f55ec06c3c0 0x7f55ec0cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.332+0000 7f55f0b3b640 1 --2- 172.21.15.16:0/591630344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec15ba10 0x7f55ec099a40 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.332+0000 7f55f0b3b640 1 -- 172.21.15.16:0/591630344 >> 172.21.15.16:0/591630344 conn(0x7f55ec0b32f0 msgr2=0x7f55ec0b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.333+0000 7f55f0b3b640 1 -- 172.21.15.16:0/591630344 shutdown_connections 2024-04-25T10:07:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.333+0000 7f55f0b3b640 1 -- 172.21.15.16:0/591630344 wait complete. 2024-04-25T10:07:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.334+0000 7f55f0b3b640 1 Processor -- start 2024-04-25T10:07:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.334+0000 7f55f0b3b640 1 -- start start 2024-04-25T10:07:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.335+0000 7f55f0b3b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55ec06c3c0 0x7f55ec1f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.335+0000 7f55f0b3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec099f80 0x7f55ec1f83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.335+0000 7f55f0b3b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55ec15ba10 0x7f55ec0af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.335+0000 7f55f0b3b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec0cd760 con 0x7f55ec099f80 2024-04-25T10:07:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.335+0000 7f55f0b3b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec0cfe80 con 0x7f55ec15ba10 2024-04-25T10:07:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.335+0000 7f55f0b3b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec0cd8e0 con 0x7f55ec06c3c0 2024-04-25T10:07:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.335+0000 7f55e1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec099f80 0x7f55ec1f83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.335+0000 7f55e1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec099f80 0x7f55ec1f83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50640/0 (socket says 172.21.15.16:50640) 2024-04-25T10:07:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.335+0000 7f55e1d9b640 1 -- 172.21.15.16:0/3026500483 learned_addr learned my addr 172.21.15.16:0/3026500483 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.335+0000 7f55ead9d640 1 --2- 172.21.15.16:0/3026500483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55ec15ba10 0x7f55ec0af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.335+0000 7f55ea59c640 1 --2- 172.21.15.16:0/3026500483 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55ec06c3c0 0x7f55ec1f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.335+0000 7f55e1d9b640 1 -- 172.21.15.16:0/3026500483 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55ec06c3c0 msgr2=0x7f55ec1f3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.335+0000 7f55e1d9b640 1 --2- 172.21.15.16:0/3026500483 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55ec06c3c0 0x7f55ec1f3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.335+0000 7f55e1d9b640 1 -- 172.21.15.16:0/3026500483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55ec15ba10 msgr2=0x7f55ec0af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.335+0000 7f55e1d9b640 1 --2- 172.21.15.16:0/3026500483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55ec15ba10 0x7f55ec0af500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.335+0000 7f55e1d9b640 1 -- 172.21.15.16:0/3026500483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec1f88e0 con 0x7f55ec099f80 2024-04-25T10:07:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.335+0000 7f55e1d9b640 1 --2- 172.21.15.16:0/3026500483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec099f80 0x7f55ec1f83a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c4d0 tx=0x7f55d400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.335+0000 7f55e37fe640 1 -- 172.21.15.16:0/3026500483 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4018070 con 0x7f55ec099f80 2024-04-25T10:07:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.336+0000 7f55e37fe640 1 -- 172.21.15.16:0/3026500483 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d4007500 con 0x7f55ec099f80 2024-04-25T10:07:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.336+0000 7f55f0b3b640 1 -- 172.21.15.16:0/3026500483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55ec0b4ae0 con 0x7f55ec099f80 2024-04-25T10:07:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.336+0000 7f55e37fe640 1 -- 172.21.15.16:0/3026500483 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4004a20 con 0x7f55ec099f80 2024-04-25T10:07:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.336+0000 7f55f0b3b640 1 -- 172.21.15.16:0/3026500483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55ec0b4da0 con 0x7f55ec099f80 2024-04-25T10:07:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.337+0000 7f55f0b3b640 1 -- 172.21.15.16:0/3026500483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55ec0cdfe0 con 0x7f55ec099f80 2024-04-25T10:07:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.337+0000 7f55e37fe640 1 -- 172.21.15.16:0/3026500483 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f55d401b070 con 0x7f55ec099f80 2024-04-25T10:07:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.337+0000 7f55e37fe640 1 --2- 172.21.15.16:0/3026500483 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f55cc06b4d0 0x7f55cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.338+0000 7f55e37fe640 1 -- 172.21.15.16:0/3026500483 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55d4089510 con 0x7f55ec099f80 2024-04-25T10:07:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.338+0000 7f55ea59c640 1 --2- 172.21.15.16:0/3026500483 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f55cc06b4d0 0x7f55cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.338+0000 7f55ea59c640 1 --2- 172.21.15.16:0/3026500483 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f55cc06b4d0 0x7f55cc06d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f55dc034000 tx=0x7f55dc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.342+0000 7f55e37fe640 1 -- 172.21.15.16:0/3026500483 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f55ec0cdfe0 con 0x7f55ec099f80 2024-04-25T10:07:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.562+0000 7f55f0b3b640 1 -- 172.21.15.16:0/3026500483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55ec0749b0 con 0x7f55ec099f80 2024-04-25T10:07:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.563+0000 7f55e37fe640 1 -- 172.21.15.16:0/3026500483 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f55ec1f88e0 con 0x7f55ec099f80 2024-04-25T10:07:02.569 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:02.569 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/791079479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":791079479},{"type":"v1","addr":"172.21.15.16:6835","nonce":791079479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1747921236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1747921236},{"type":"v1","addr":"172.21.15.16:6834","nonce":1747921236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":34880,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/3183779257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3183779257},{"type":"v1","addr":"172.21.15.186:6836","nonce":3183779257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:51.182691+0000","modified":"2024-04-25T10:06:53.179194+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34883},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34883":{"gid":34883,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/83423489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":83423489},{"type":"v1","addr":"172.21.15.186:6837","nonce":83423489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 21 2024-04-25T10:07:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.565+0000 7f55f0b3b640 1 -- 172.21.15.16:0/3026500483 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f55cc06b4d0 msgr2=0x7f55cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.565+0000 7f55f0b3b640 1 --2- 172.21.15.16:0/3026500483 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f55cc06b4d0 0x7f55cc06d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f55dc034000 tx=0x7f55dc02e000 comp rx=0 tx=0).stop 2024-04-25T10:07:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.565+0000 7f55f0b3b640 1 -- 172.21.15.16:0/3026500483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec099f80 msgr2=0x7f55ec1f83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.565+0000 7f55f0b3b640 1 --2- 172.21.15.16:0/3026500483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec099f80 0x7f55ec1f83a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c4d0 tx=0x7f55d400c990 comp rx=0 tx=0).stop 2024-04-25T10:07:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.565+0000 7f55f0b3b640 1 -- 172.21.15.16:0/3026500483 shutdown_connections 2024-04-25T10:07:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.565+0000 7f55f0b3b640 1 --2- 172.21.15.16:0/3026500483 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f55cc06b4d0 0x7f55cc06d990 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.565+0000 7f55f0b3b640 1 --2- 172.21.15.16:0/3026500483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55ec15ba10 0x7f55ec0af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.565+0000 7f55f0b3b640 1 --2- 172.21.15.16:0/3026500483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec099f80 0x7f55ec1f83a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.565+0000 7f55f0b3b640 1 --2- 172.21.15.16:0/3026500483 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55ec06c3c0 0x7f55ec1f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.565+0000 7f55f0b3b640 1 -- 172.21.15.16:0/3026500483 >> 172.21.15.16:0/3026500483 conn(0x7f55ec0b32f0 msgr2=0x7f55ec0d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.565+0000 7f55f0b3b640 1 -- 172.21.15.16:0/3026500483 shutdown_connections 2024-04-25T10:07:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.565+0000 7f55f0b3b640 1 -- 172.21.15.16:0/3026500483 wait complete. 2024-04-25T10:07:02.582 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-25T10:07:02.712 INFO:teuthology.orchestra.run.smithi186.stdout:{"message":"","return_code":0} 2024-04-25T10:07:02.712 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:07:02.713 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:07:02.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.751+0000 7f4bc9639f40 1 Processor -- start 2024-04-25T10:07:02.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.751+0000 7f4bc9639f40 1 -- start start 2024-04-25T10:07:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.751+0000 7f4bc9639f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d0f5f5b00 0x555d0f72c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.751+0000 7f4bc9639f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d0f5f5ee0 0x555d0f5f1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.751+0000 7f4bc9639f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d0f69ee50 0x555d0f5f1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.752+0000 7f4bc9639f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0f56cdf0 con 0x555d0f5f5ee0 2024-04-25T10:07:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.752+0000 7f4bc9639f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0f570d50 con 0x555d0f5f5b00 2024-04-25T10:07:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.752+0000 7f4bc9639f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d0f69b540 con 0x555d0f69ee50 2024-04-25T10:07:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.752+0000 7f4bc6939640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d0f5f5ee0 0x555d0f5f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.752+0000 7f4bc6939640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d0f5f5ee0 0x555d0f5f1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49718/0 (socket says 172.21.15.16:49718) 2024-04-25T10:07:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.752+0000 7f4bc6939640 1 -- 172.21.15.16:0/2022130680 learned_addr learned my addr 172.21.15.16:0/2022130680 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.752+0000 7f4bc713a640 1 --2- 172.21.15.16:0/2022130680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d0f5f5b00 0x555d0f72c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.752+0000 7f4bc793b640 1 --2- 172.21.15.16:0/2022130680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d0f69ee50 0x555d0f5f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.752+0000 7f4bc6939640 1 -- 172.21.15.16:0/2022130680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d0f69ee50 msgr2=0x555d0f5f1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.752+0000 7f4bc6939640 1 --2- 172.21.15.16:0/2022130680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d0f69ee50 0x555d0f5f1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.752+0000 7f4bc6939640 1 -- 172.21.15.16:0/2022130680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d0f5f5b00 msgr2=0x555d0f72c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.752+0000 7f4bc6939640 1 --2- 172.21.15.16:0/2022130680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d0f5f5b00 0x555d0f72c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.752+0000 7f4bc6939640 1 -- 172.21.15.16:0/2022130680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d0f69bff0 con 0x555d0f5f5ee0 2024-04-25T10:07:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.752+0000 7f4bc6939640 1 --2- 172.21.15.16:0/2022130680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d0f5f5ee0 0x555d0f5f1040 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00b3c0 tx=0x7f4bbc00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a846339490fc3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.752+0000 7f4bc5937640 1 -- 172.21.15.16:0/2022130680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc008070 con 0x555d0f5f5ee0 2024-04-25T10:07:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.752+0000 7f4bc5937640 1 -- 172.21.15.16:0/2022130680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc00d5c0 con 0x555d0f5f5ee0 2024-04-25T10:07:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.752+0000 7f4bc5937640 1 -- 172.21.15.16:0/2022130680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc00d990 con 0x555d0f5f5ee0 2024-04-25T10:07:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.753+0000 7f4bc9639f40 1 -- 172.21.15.16:0/2022130680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d0f5f5ee0 msgr2=0x555d0f5f1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.753+0000 7f4bc9639f40 1 --2- 172.21.15.16:0/2022130680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d0f5f5ee0 0x555d0f5f1040 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00b3c0 tx=0x7f4bbc00b880 comp rx=0 tx=0).stop 2024-04-25T10:07:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.753+0000 7f4bc9639f40 1 -- 172.21.15.16:0/2022130680 shutdown_connections 2024-04-25T10:07:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.753+0000 7f4bc9639f40 1 --2- 172.21.15.16:0/2022130680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d0f69ee50 0x555d0f5f1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.753+0000 7f4bc9639f40 1 --2- 172.21.15.16:0/2022130680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d0f5f5ee0 0x555d0f5f1040 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.753+0000 7f4bc9639f40 1 --2- 172.21.15.16:0/2022130680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d0f5f5b00 0x555d0f72c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.753+0000 7f4bc9639f40 1 -- 172.21.15.16:0/2022130680 >> 172.21.15.16:0/2022130680 conn(0x555d0f5f03d0 msgr2=0x555d0f60f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.753+0000 7f4bc9639f40 1 -- 172.21.15.16:0/2022130680 shutdown_connections 2024-04-25T10:07:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.753+0000 7f4bc9639f40 1 -- 172.21.15.16:0/2022130680 wait complete. 2024-04-25T10:07:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.753+0000 7f4bc9639f40 1 Processor -- start 2024-04-25T10:07:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.753+0000 7f4bc9639f40 1 -- start start 2024-04-25T10:07:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.754+0000 7f4bc9639f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d0f5f5b00 0x555d0f671770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.754+0000 7f4bc9639f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d0f5f5ee0 0x555d0f5ef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.754+0000 7f4bc9639f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d0f69ee50 0x555d0f699e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.754+0000 7f4bc9639f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0f69c750 con 0x555d0f69ee50 2024-04-25T10:07:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.754+0000 7f4bc9639f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0f687730 con 0x555d0f5f5b00 2024-04-25T10:07:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.754+0000 7f4bc9639f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d0f69ca30 con 0x555d0f5f5ee0 2024-04-25T10:07:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.754+0000 7f4bc793b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d0f69ee50 0x555d0f699e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.754+0000 7f4bc793b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d0f69ee50 0x555d0f699e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49732/0 (socket says 172.21.15.16:49732) 2024-04-25T10:07:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.754+0000 7f4bc793b640 1 -- 172.21.15.16:0/3460132738 learned_addr learned my addr 172.21.15.16:0/3460132738 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.754+0000 7f4bc6939640 1 --2- 172.21.15.16:0/3460132738 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d0f5f5ee0 0x555d0f5ef8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.754+0000 7f4bc713a640 1 --2- 172.21.15.16:0/3460132738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d0f5f5b00 0x555d0f671770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.754+0000 7f4bc793b640 1 -- 172.21.15.16:0/3460132738 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d0f5f5ee0 msgr2=0x555d0f5ef8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.754+0000 7f4bc793b640 1 --2- 172.21.15.16:0/3460132738 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d0f5f5ee0 0x555d0f5ef8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.754+0000 7f4bc793b640 1 -- 172.21.15.16:0/3460132738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d0f5f5b00 msgr2=0x555d0f671770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.754+0000 7f4bc793b640 1 --2- 172.21.15.16:0/3460132738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d0f5f5b00 0x555d0f671770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.754+0000 7f4bc793b640 1 -- 172.21.15.16:0/3460132738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d0f672070 con 0x555d0f69ee50 2024-04-25T10:07:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.755+0000 7f4bc793b640 1 --2- 172.21.15.16:0/3460132738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d0f69ee50 0x555d0f699e10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800d1e0 tx=0x7f4bb800d6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.755+0000 7f4bc4935640 1 -- 172.21.15.16:0/3460132738 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8018070 con 0x555d0f69ee50 2024-04-25T10:07:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.755+0000 7f4bc4935640 1 -- 172.21.15.16:0/3460132738 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb80032e0 con 0x555d0f69ee50 2024-04-25T10:07:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.755+0000 7f4bc9639f40 1 -- 172.21.15.16:0/3460132738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555d0f672350 con 0x555d0f69ee50 2024-04-25T10:07:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.755+0000 7f4bc4935640 1 -- 172.21.15.16:0/3460132738 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8003490 con 0x555d0f69ee50 2024-04-25T10:07:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.755+0000 7f4bc9639f40 1 -- 172.21.15.16:0/3460132738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d0f66fa60 con 0x555d0f69ee50 2024-04-25T10:07:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.756+0000 7f4bc4935640 1 -- 172.21.15.16:0/3460132738 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4bb8001830 con 0x555d0f69ee50 2024-04-25T10:07:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.757+0000 7f4bc4935640 1 --2- 172.21.15.16:0/3460132738 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4bb4069ee0 0x7f4bb406c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.757+0000 7f4bc4935640 1 -- 172.21.15.16:0/3460132738 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bb80883e0 con 0x555d0f69ee50 2024-04-25T10:07:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.757+0000 7f4bc713a640 1 --2- 172.21.15.16:0/3460132738 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4bb4069ee0 0x7f4bb406c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.757+0000 7f4bc9639f40 1 --2- 172.21.15.16:0/3460132738 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x555d0f66ff80 0x555d0f6703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.757+0000 7f4bc9639f40 1 -- 172.21.15.16:0/3460132738 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v9 -- 0x555d0f6708e0 con 0x555d0f66ff80 2024-04-25T10:07:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.757+0000 7f4bc6939640 1 --2- 172.21.15.16:0/3460132738 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x555d0f66ff80 0x555d0f6703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.757+0000 7f4bc713a640 1 --2- 172.21.15.16:0/3460132738 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4bb4069ee0 0x7f4bb406c3a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0056000 tx=0x7f4bc0050000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.757+0000 7f4bc6939640 1 --2- 172.21.15.16:0/3460132738 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x555d0f66ff80 0x555d0f6703a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.758+0000 7f4bc6939640 1 -- 172.21.15.16:0/3460132738 <== osd.3 v2:172.21.15.16:6824/256577261 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f4bbc00d9d0 con 0x555d0f66ff80 2024-04-25T10:07:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.758+0000 7f4bc9639f40 1 -- 172.21.15.16:0/3460132738 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x555d0f66ff80 msgr2=0x555d0f6703a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.758+0000 7f4bc9639f40 1 --2- 172.21.15.16:0/3460132738 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x555d0f66ff80 0x555d0f6703a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.758+0000 7f4bc9639f40 1 -- 172.21.15.16:0/3460132738 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4bb4069ee0 msgr2=0x7f4bb406c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.758+0000 7f4bc9639f40 1 --2- 172.21.15.16:0/3460132738 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4bb4069ee0 0x7f4bb406c3a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0056000 tx=0x7f4bc0050000 comp rx=0 tx=0).stop 2024-04-25T10:07:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.758+0000 7f4bc9639f40 1 -- 172.21.15.16:0/3460132738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d0f69ee50 msgr2=0x555d0f699e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.758+0000 7f4bc9639f40 1 --2- 172.21.15.16:0/3460132738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d0f69ee50 0x555d0f699e10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800d1e0 tx=0x7f4bb800d6a0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.758+0000 7f4bc793b640 1 -- 172.21.15.16:0/3460132738 reap_dead start 2024-04-25T10:07:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.758+0000 7f4bc9639f40 1 -- 172.21.15.16:0/3460132738 shutdown_connections 2024-04-25T10:07:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.758+0000 7f4bc9639f40 1 -- 172.21.15.16:0/3460132738 >> 172.21.15.16:0/3460132738 conn(0x555d0f5f03d0 msgr2=0x555d0f68f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.758+0000 7f4bc9639f40 1 -- 172.21.15.16:0/3460132738 shutdown_connections 2024-04-25T10:07:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.758+0000 7f4bc9639f40 1 -- 172.21.15.16:0/3460132738 wait complete. 2024-04-25T10:07:02.769 INFO:teuthology.orchestra.run.smithi016.stdout:Yalphaparent_a 2024-04-25T10:07:02.769 DEBUG:teuthology.orchestra.run.smithi016:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:07:02.871 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:07:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511627777, 2024-04-25T10:07:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "ancestors": [ 2024-04-25T10:07:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "dirino": 1099511627776, 2024-04-25T10:07:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "dname": "alpha", 2024-04-25T10:07:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 2 2024-04-25T10:07:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:02.871 INFO:teuthology.orchestra.run.smithi016.stdout: "dirino": 1, 2024-04-25T10:07:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: "dname": "parent_a", 2024-04-25T10:07:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 4 2024-04-25T10:07:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-25T10:07:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: "pool": 7, 2024-04-25T10:07:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: "old_pools": [] 2024-04-25T10:07:02.872 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:07:02.872 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:02.890 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-04-25T10:07:02.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.968+0000 7fe25fd21f40 1 Processor -- start 2024-04-25T10:07:02.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.968+0000 7fe25fd21f40 1 -- start start 2024-04-25T10:07:02.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.969+0000 7fe25fd21f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e8b92b00 0x55a6e8c2bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.969+0000 7fe25fd21f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e8b92ee0 0x55a6e8b648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.969+0000 7fe25fd21f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6e8c3be50 0x55a6e8c35270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.969+0000 7fe25fd21f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e8b09df0 con 0x55a6e8b92ee0 2024-04-25T10:07:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.969+0000 7fe25fd21f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e8b0dd50 con 0x55a6e8b92b00 2024-04-25T10:07:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.969+0000 7fe25fd21f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6e8c38540 con 0x55a6e8c3be50 2024-04-25T10:07:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.969+0000 7fe25d023640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e8b92ee0 0x55a6e8b648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.969+0000 7fe25d023640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e8b92ee0 0x55a6e8b648d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49736/0 (socket says 172.21.15.16:49736) 2024-04-25T10:07:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.969+0000 7fe25d023640 1 -- 172.21.15.16:0/4294619114 learned_addr learned my addr 172.21.15.16:0/4294619114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.969+0000 7fe25d023640 1 -- 172.21.15.16:0/4294619114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6e8c3be50 msgr2=0x55a6e8c35270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.969+0000 7fe25d023640 1 --2- 172.21.15.16:0/4294619114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6e8c3be50 0x55a6e8c35270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.969+0000 7fe25d023640 1 -- 172.21.15.16:0/4294619114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e8b92b00 msgr2=0x55a6e8c2bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.969+0000 7fe25d023640 1 --2- 172.21.15.16:0/4294619114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e8b92b00 0x55a6e8c2bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.969+0000 7fe25d023640 1 -- 172.21.15.16:0/4294619114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6e8c337f0 con 0x55a6e8b92ee0 2024-04-25T10:07:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.969+0000 7fe25d023640 1 --2- 172.21.15.16:0/4294619114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e8b92ee0 0x55a6e8b648d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe2480257d0 tx=0x7fe24804e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e300b3c77a36c927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.970+0000 7fe257fff640 1 -- 172.21.15.16:0/4294619114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe248059040 con 0x55a6e8b92ee0 2024-04-25T10:07:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.970+0000 7fe257fff640 1 -- 172.21.15.16:0/4294619114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe248064070 con 0x55a6e8b92ee0 2024-04-25T10:07:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.970+0000 7fe257fff640 1 -- 172.21.15.16:0/4294619114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258001670 con 0x55a6e8b92ee0 2024-04-25T10:07:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.970+0000 7fe25fd21f40 1 -- 172.21.15.16:0/4294619114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e8b92ee0 msgr2=0x55a6e8b648d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.970+0000 7fe25fd21f40 1 --2- 172.21.15.16:0/4294619114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e8b92ee0 0x55a6e8b648d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe2480257d0 tx=0x7fe24804e000 comp rx=0 tx=0).stop 2024-04-25T10:07:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.970+0000 7fe25fd21f40 1 -- 172.21.15.16:0/4294619114 shutdown_connections 2024-04-25T10:07:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.970+0000 7fe25fd21f40 1 --2- 172.21.15.16:0/4294619114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6e8c3be50 0x55a6e8c35270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.970+0000 7fe25fd21f40 1 --2- 172.21.15.16:0/4294619114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e8b92ee0 0x55a6e8b648d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.970+0000 7fe25fd21f40 1 --2- 172.21.15.16:0/4294619114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e8b92b00 0x55a6e8c2bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.970+0000 7fe25fd21f40 1 -- 172.21.15.16:0/4294619114 >> 172.21.15.16:0/4294619114 conn(0x55a6e8b8d3d0 msgr2=0x55a6e8b60f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.970+0000 7fe25fd21f40 1 -- 172.21.15.16:0/4294619114 shutdown_connections 2024-04-25T10:07:02.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.970+0000 7fe25fd21f40 1 -- 172.21.15.16:0/4294619114 wait complete. 2024-04-25T10:07:02.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.971+0000 7fe25fd21f40 1 Processor -- start 2024-04-25T10:07:02.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.971+0000 7fe25fd21f40 1 -- start start 2024-04-25T10:07:02.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.971+0000 7fe25fd21f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e8b92b00 0x55a6e8c357d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.971+0000 7fe25fd21f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e8b92ee0 0x55a6e8b64b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.971+0000 7fe25fd21f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6e8c3be50 0x55a6e8cc8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.971+0000 7fe25fd21f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e8b63c60 con 0x55a6e8b92b00 2024-04-25T10:07:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.971+0000 7fe25fd21f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e8c3bb80 con 0x55a6e8b92ee0 2024-04-25T10:07:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.971+0000 7fe25fd21f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6e8b64380 con 0x55a6e8c3be50 2024-04-25T10:07:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.972+0000 7fe25d824640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e8b92b00 0x55a6e8c357d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.972+0000 7fe25d824640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e8b92b00 0x55a6e8c357d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49742/0 (socket says 172.21.15.16:49742) 2024-04-25T10:07:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.972+0000 7fe25d824640 1 -- 172.21.15.16:0/2668974767 learned_addr learned my addr 172.21.15.16:0/2668974767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.972+0000 7fe25d824640 1 -- 172.21.15.16:0/2668974767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6e8c3be50 msgr2=0x55a6e8cc8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.972+0000 7fe25d824640 1 --2- 172.21.15.16:0/2668974767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a6e8c3be50 0x55a6e8cc8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.972+0000 7fe25d824640 1 -- 172.21.15.16:0/2668974767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e8b92ee0 msgr2=0x55a6e8b64b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.972+0000 7fe25d824640 1 --2- 172.21.15.16:0/2668974767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e8b92ee0 0x55a6e8b64b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.972+0000 7fe25d824640 1 -- 172.21.15.16:0/2668974767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6e8c36f30 con 0x55a6e8b92b00 2024-04-25T10:07:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.972+0000 7fe25d824640 1 --2- 172.21.15.16:0/2668974767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e8b92b00 0x55a6e8c357d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe25800bf30 tx=0x7fe258009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.972+0000 7fe256ffd640 1 -- 172.21.15.16:0/2668974767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258013070 con 0x55a6e8b92b00 2024-04-25T10:07:02.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.972+0000 7fe256ffd640 1 -- 172.21.15.16:0/2668974767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe258009e50 con 0x55a6e8b92b00 2024-04-25T10:07:02.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.972+0000 7fe25fd21f40 1 -- 172.21.15.16:0/2668974767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a6e8c2a8e0 con 0x55a6e8b92b00 2024-04-25T10:07:02.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.972+0000 7fe256ffd640 1 -- 172.21.15.16:0/2668974767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25800f820 con 0x55a6e8b92b00 2024-04-25T10:07:02.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.972+0000 7fe25fd21f40 1 -- 172.21.15.16:0/2668974767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6e8cc9130 con 0x55a6e8b92b00 2024-04-25T10:07:02.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.974+0000 7fe256ffd640 1 -- 172.21.15.16:0/2668974767 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe258016070 con 0x55a6e8b92b00 2024-04-25T10:07:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.974+0000 7fe256ffd640 1 --2- 172.21.15.16:0/2668974767 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe240069eb0 0x7fe24006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.974+0000 7fe256ffd640 1 -- 172.21.15.16:0/2668974767 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe258088a40 con 0x55a6e8b92b00 2024-04-25T10:07:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.974+0000 7fe25d023640 1 --2- 172.21.15.16:0/2668974767 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe240069eb0 0x7fe24006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.974+0000 7fe25fd21f40 1 --2- 172.21.15.16:0/2668974767 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55a6e8c2c8b0 0x55a6e8cd1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.974+0000 7fe25fd21f40 1 -- 172.21.15.16:0/2668974767 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v9 -- 0x55a6e8b63e10 con 0x55a6e8c2c8b0 2024-04-25T10:07:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.974+0000 7fe25e025640 1 --2- 172.21.15.16:0/2668974767 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55a6e8c2c8b0 0x55a6e8cd1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.975+0000 7fe25e025640 1 --2- 172.21.15.16:0/2668974767 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55a6e8c2c8b0 0x55a6e8cd1e10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.975+0000 7fe25d023640 1 --2- 172.21.15.16:0/2668974767 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe240069eb0 0x7fe24006c370 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe24800a470 tx=0x7fe248005a00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.976+0000 7fe25e025640 1 -- 172.21.15.16:0/2668974767 <== osd.3 v2:172.21.15.16:6824/256577261 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v47'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe250006560 con 0x55a6e8c2c8b0 2024-04-25T10:07:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.976+0000 7fe25fd21f40 1 -- 172.21.15.16:0/2668974767 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55a6e8c2c8b0 msgr2=0x55a6e8cd1e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.976+0000 7fe25fd21f40 1 --2- 172.21.15.16:0/2668974767 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55a6e8c2c8b0 0x55a6e8cd1e10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.976+0000 7fe25fd21f40 1 -- 172.21.15.16:0/2668974767 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe240069eb0 msgr2=0x7fe24006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.976+0000 7fe25fd21f40 1 --2- 172.21.15.16:0/2668974767 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe240069eb0 0x7fe24006c370 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe24800a470 tx=0x7fe248005a00 comp rx=0 tx=0).stop 2024-04-25T10:07:02.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.976+0000 7fe25fd21f40 1 -- 172.21.15.16:0/2668974767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e8b92b00 msgr2=0x55a6e8c357d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.976+0000 7fe25fd21f40 1 --2- 172.21.15.16:0/2668974767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e8b92b00 0x55a6e8c357d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe25800bf30 tx=0x7fe258009530 comp rx=0 tx=0).stop 2024-04-25T10:07:02.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.976+0000 7fe25e025640 1 -- 172.21.15.16:0/2668974767 reap_dead start 2024-04-25T10:07:02.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.976+0000 7fe25fd21f40 1 -- 172.21.15.16:0/2668974767 shutdown_connections 2024-04-25T10:07:02.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.976+0000 7fe25fd21f40 1 -- 172.21.15.16:0/2668974767 >> 172.21.15.16:0/2668974767 conn(0x55a6e8b8d3d0 msgr2=0x55a6e8c37470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.976+0000 7fe25fd21f40 1 -- 172.21.15.16:0/2668974767 shutdown_connections 2024-04-25T10:07:02.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:02.976+0000 7fe25fd21f40 1 -- 172.21.15.16:0/2668974767 wait complete. 2024-04-25T10:07:02.988 DEBUG:teuthology.orchestra.run.smithi016:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T10:07:02.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:02.990 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-04-25T10:07:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.089+0000 7f2bbaff4640 1 Processor -- start 2024-04-25T10:07:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.089+0000 7f2bbaff4640 1 -- start start 2024-04-25T10:07:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.090+0000 7f2bbaff4640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2bb406b040 0x7f2bb40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.090+0000 7f2bbaff4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2bb4161f10 0x7f2bb4162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.090+0000 7f2bbaff4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb4162860 con 0x7f2bb406b420 2024-04-25T10:07:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.090+0000 7f2bbaff4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb40cf980 con 0x7f2bb4161f10 2024-04-25T10:07:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.090+0000 7f2bbaff4640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb40cfc20 con 0x7f2bb406b040 2024-04-25T10:07:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.090+0000 7f2bb9591640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2bb4161f10 0x7f2bb4162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.090+0000 7f2bb3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2bb406b420 0x7f2bb40b29a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44088/0 (socket says 172.21.15.16:44088) 2024-04-25T10:07:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.090+0000 7f2bb9591640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2bb4161f10 0x7f2bb4162320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49744/0 (socket says 172.21.15.16:49744) 2024-04-25T10:07:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.090+0000 7f2bb3fff640 1 -- 172.21.15.16:0/1396655692 learned_addr learned my addr 172.21.15.16:0/1396655692 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.090+0000 7f2bb8d90640 1 --2- 172.21.15.16:0/1396655692 >> v2:172.21.15.186:3300/0 conn(0x7f2bb406b040 0x7f2bb40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.091+0000 7f2bb9591640 1 -- 172.21.15.16:0/1396655692 >> v2:172.21.15.186:3300/0 conn(0x7f2bb406b040 msgr2=0x7f2bb40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.091+0000 7f2bb9591640 1 --2- 172.21.15.16:0/1396655692 >> v2:172.21.15.186:3300/0 conn(0x7f2bb406b040 0x7f2bb40ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.091+0000 7f2bb9591640 1 -- 172.21.15.16:0/1396655692 >> v1:172.21.15.16:6789/0 conn(0x7f2bb406b420 legacy=0x7f2bb40b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.091+0000 7f2bb9591640 1 -- 172.21.15.16:0/1396655692 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb40cff00 con 0x7f2bb4161f10 2024-04-25T10:07:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.091+0000 7f2bb8d90640 1 --2- 172.21.15.16:0/1396655692 >> v2:172.21.15.186:3300/0 conn(0x7f2bb406b040 0x7f2bb40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.091+0000 7f2bb9591640 1 --2- 172.21.15.16:0/1396655692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb4161f10 0x7f2bb4162320 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2bb406e2b0 tx=0x7f2ba402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7317659c4074ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.092+0000 7f2bb37fe640 1 -- 172.21.15.16:0/1396655692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4037070 con 0x7f2bb4161f10 2024-04-25T10:07:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.092+0000 7f2bb37fe640 1 -- 172.21.15.16:0/1396655692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba4032440 con 0x7f2bb4161f10 2024-04-25T10:07:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.092+0000 7f2bb37fe640 1 -- 172.21.15.16:0/1396655692 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4032810 con 0x7f2bb4161f10 2024-04-25T10:07:03.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.092+0000 7f2bbaff4640 1 -- 172.21.15.16:0/1396655692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb4161f10 msgr2=0x7f2bb4162320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.092+0000 7f2bbaff4640 1 --2- 172.21.15.16:0/1396655692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb4161f10 0x7f2bb4162320 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2bb406e2b0 tx=0x7f2ba402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.092+0000 7f2bbaff4640 1 -- 172.21.15.16:0/1396655692 shutdown_connections 2024-04-25T10:07:03.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.092+0000 7f2bbaff4640 1 --2- 172.21.15.16:0/1396655692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb4161f10 0x7f2bb4162320 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.092+0000 7f2bbaff4640 1 --2- 172.21.15.16:0/1396655692 >> v2:172.21.15.186:3300/0 conn(0x7f2bb406b040 0x7f2bb40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.092+0000 7f2bbaff4640 1 -- 172.21.15.16:0/1396655692 >> 172.21.15.16:0/1396655692 conn(0x7f2bb4099a40 msgr2=0x7f2bb40b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.092+0000 7f2bbaff4640 1 -- 172.21.15.16:0/1396655692 shutdown_connections 2024-04-25T10:07:03.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.092+0000 7f2bbaff4640 1 -- 172.21.15.16:0/1396655692 wait complete. 2024-04-25T10:07:03.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.093+0000 7f2bbaff4640 1 Processor -- start 2024-04-25T10:07:03.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.093+0000 7f2bbaff4640 1 -- start start 2024-04-25T10:07:03.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.094+0000 7f2bbaff4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bb406b040 0x7f2bb40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.094+0000 7f2bbaff4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2bb406b420 0x7f2bb4152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.094+0000 7f2bbaff4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb4161f10 0x7f2bb415f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.094+0000 7f2bbaff4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb40ccba0 con 0x7f2bb4161f10 2024-04-25T10:07:03.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.094+0000 7f2bbaff4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb40b30b0 con 0x7f2bb406b040 2024-04-25T10:07:03.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.094+0000 7f2bbaff4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb40d04c0 con 0x7f2bb406b420 2024-04-25T10:07:03.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.094+0000 7f2bb9591640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb4161f10 0x7f2bb415f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.095+0000 7f2bb9591640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb4161f10 0x7f2bb415f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49752/0 (socket says 172.21.15.16:49752) 2024-04-25T10:07:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.095+0000 7f2bb9591640 1 -- 172.21.15.16:0/2944544722 learned_addr learned my addr 172.21.15.16:0/2944544722 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.095+0000 7f2bb3fff640 1 --2- 172.21.15.16:0/2944544722 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2bb406b420 0x7f2bb4152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.095+0000 7f2bb8d90640 1 --2- 172.21.15.16:0/2944544722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bb406b040 0x7f2bb40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.095+0000 7f2bb9591640 1 -- 172.21.15.16:0/2944544722 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2bb406b420 msgr2=0x7f2bb4152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.095+0000 7f2bb9591640 1 --2- 172.21.15.16:0/2944544722 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2bb406b420 0x7f2bb4152920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.095+0000 7f2bb9591640 1 -- 172.21.15.16:0/2944544722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bb406b040 msgr2=0x7f2bb40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.095+0000 7f2bb9591640 1 --2- 172.21.15.16:0/2944544722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bb406b040 0x7f2bb40b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.095+0000 7f2bb9591640 1 -- 172.21.15.16:0/2944544722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb4152e60 con 0x7f2bb4161f10 2024-04-25T10:07:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.095+0000 7f2bb8d90640 1 --2- 172.21.15.16:0/2944544722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bb406b040 0x7f2bb40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.095+0000 7f2bb3fff640 1 --2- 172.21.15.16:0/2944544722 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2bb406b420 0x7f2bb4152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.095+0000 7f2bb9591640 1 --2- 172.21.15.16:0/2944544722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb4161f10 0x7f2bb415f810 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4035040 tx=0x7f2ba40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.095+0000 7f2bb17fa640 1 -- 172.21.15.16:0/2944544722 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4037040 con 0x7f2bb4161f10 2024-04-25T10:07:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.095+0000 7f2bb17fa640 1 -- 172.21.15.16:0/2944544722 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba4042070 con 0x7f2bb4161f10 2024-04-25T10:07:03.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.095+0000 7f2bbaff4640 1 -- 172.21.15.16:0/2944544722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb415fd50 con 0x7f2bb4161f10 2024-04-25T10:07:03.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.095+0000 7f2bb17fa640 1 -- 172.21.15.16:0/2944544722 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4006e10 con 0x7f2bb4161f10 2024-04-25T10:07:03.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.095+0000 7f2bbaff4640 1 -- 172.21.15.16:0/2944544722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb40b5180 con 0x7f2bb4161f10 2024-04-25T10:07:03.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.097+0000 7f2bb17fa640 1 -- 172.21.15.16:0/2944544722 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ba4030bd0 con 0x7f2bb4161f10 2024-04-25T10:07:03.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.097+0000 7f2bb17fa640 1 --2- 172.21.15.16:0/2944544722 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2b9406b4d0 0x7f2b9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.098+0000 7f2bb17fa640 1 -- 172.21.15.16:0/2944544722 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ba40aae40 con 0x7f2bb4161f10 2024-04-25T10:07:03.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.098+0000 7f2bb8d90640 1 --2- 172.21.15.16:0/2944544722 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2b9406b4d0 0x7f2b9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.098+0000 7f2bb8d90640 1 --2- 172.21.15.16:0/2944544722 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2b9406b4d0 0x7f2b9406d990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0044f0 tx=0x7f2bac003a30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.098+0000 7f2bbddce740 1 Processor -- start 2024-04-25T10:07:03.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.098+0000 7f2bbddce740 1 -- start start 2024-04-25T10:07:03.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.099+0000 7f2bbddce740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb447d4bb0 0x55fb447d4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.099+0000 7f2bbddce740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb447cf830 0x55fb447d37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.099+0000 7f2bbddce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb447d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.099+0000 7f2bbddce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb44738ac0 con 0x55fb447d3cf0 2024-04-25T10:07:03.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.099+0000 7f2bbddce740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb446e1d80 con 0x55fb447d4bb0 2024-04-25T10:07:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.099+0000 7f2bbddce740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb44734b60 con 0x55fb447cf830 2024-04-25T10:07:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.099+0000 7f2bb9591640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb447d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.099+0000 7f2bb9591640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb447d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49758/0 (socket says 172.21.15.16:49758) 2024-04-25T10:07:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.099+0000 7f2bb9591640 1 -- 172.21.15.16:0/951241665 learned_addr learned my addr 172.21.15.16:0/951241665 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.099+0000 7f2bb8d90640 1 --2- 172.21.15.16:0/951241665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb447d4bb0 0x55fb447d4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.099+0000 7f2bb3fff640 1 --2- 172.21.15.16:0/951241665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb447cf830 0x55fb447d37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.100+0000 7f2bb9591640 1 -- 172.21.15.16:0/951241665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb447cf830 msgr2=0x55fb447d37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.100+0000 7f2bb9591640 1 --2- 172.21.15.16:0/951241665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb447cf830 0x55fb447d37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.100+0000 7f2bb9591640 1 -- 172.21.15.16:0/951241665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb447d4bb0 msgr2=0x55fb447d4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.100+0000 7f2bb9591640 1 --2- 172.21.15.16:0/951241665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb447d4bb0 0x55fb447d4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.100+0000 7f2bb9591640 1 -- 172.21.15.16:0/951241665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb445157e0 con 0x55fb447d3cf0 2024-04-25T10:07:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.100+0000 7f2bb9591640 1 --2- 172.21.15.16:0/951241665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb447d4130 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40759a0 tx=0x7f2ba4079440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a16928a5783fa6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.100+0000 7f2baa7fc640 1 -- 172.21.15.16:0/951241665 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4079540 con 0x55fb447d3cf0 2024-04-25T10:07:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.100+0000 7f2baa7fc640 1 -- 172.21.15.16:0/951241665 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba407d690 con 0x55fb447d3cf0 2024-04-25T10:07:03.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.100+0000 7f2bbddce740 1 -- 172.21.15.16:0/951241665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 msgr2=0x55fb447d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.100+0000 7f2bbddce740 1 --2- 172.21.15.16:0/951241665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb447d4130 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40759a0 tx=0x7f2ba4079440 comp rx=0 tx=0).stop 2024-04-25T10:07:03.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.101+0000 7f2bbddce740 1 -- 172.21.15.16:0/951241665 shutdown_connections 2024-04-25T10:07:03.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.101+0000 7f2bbddce740 1 --2- 172.21.15.16:0/951241665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb447d4130 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.101+0000 7f2bbddce740 1 --2- 172.21.15.16:0/951241665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb447cf830 0x55fb447d37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.101+0000 7f2bbddce740 1 --2- 172.21.15.16:0/951241665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb447d4bb0 0x55fb447d4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.101+0000 7f2bbddce740 1 -- 172.21.15.16:0/951241665 >> 172.21.15.16:0/951241665 conn(0x55fb447d1cb0 msgr2=0x55fb4481a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.101+0000 7f2bbddce740 1 -- 172.21.15.16:0/951241665 shutdown_connections 2024-04-25T10:07:03.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.101+0000 7f2bbddce740 1 -- 172.21.15.16:0/951241665 wait complete. 2024-04-25T10:07:03.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.101+0000 7f2bbddce740 1 -- start start 2024-04-25T10:07:03.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.101+0000 7f2bbddce740 1 Processor -- start 2024-04-25T10:07:03.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.102+0000 7f2bbddce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb447f15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.102+0000 7f2bbddce740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb447d4bb0 0x55fb447f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.102+0000 7f2bbddce740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb4481d210 0x55fb447f5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.102+0000 7f2bbddce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb44738ac0 con 0x55fb447d3cf0 2024-04-25T10:07:03.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.102+0000 7f2bbddce740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb446d7530 con 0x55fb447d4bb0 2024-04-25T10:07:03.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.102+0000 7f2bbddce740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb44800740 con 0x55fb4481d210 2024-04-25T10:07:03.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.102+0000 7f2bbddce740 10 client.0 _ll_register_callbacks cb 0x55fb4472f130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:03.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.102+0000 7f2bb8d90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb447f15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.102+0000 7f2bb8d90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb447f15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49772/0 (socket says 172.21.15.16:49772) 2024-04-25T10:07:03.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.102+0000 7f2bb8d90640 1 -- 172.21.15.16:0/454432538 learned_addr learned my addr 172.21.15.16:0/454432538 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:03.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.102+0000 7f2bb9591640 1 --2- 172.21.15.16:0/454432538 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb4481d210 0x55fb447f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.102+0000 7f2bb3fff640 1 --2- 172.21.15.16:0/454432538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb447d4bb0 0x55fb447f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.103+0000 7f2bb8d90640 1 -- 172.21.15.16:0/454432538 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb4481d210 msgr2=0x55fb447f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.103+0000 7f2bb8d90640 1 --2- 172.21.15.16:0/454432538 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb4481d210 0x55fb447f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.103+0000 7f2bb8d90640 1 -- 172.21.15.16:0/454432538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb447d4bb0 msgr2=0x55fb447f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.103+0000 7f2bb8d90640 1 --2- 172.21.15.16:0/454432538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb447d4bb0 0x55fb447f4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.103+0000 7f2bb8d90640 1 -- 172.21.15.16:0/454432538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb44575f70 con 0x55fb447d3cf0 2024-04-25T10:07:03.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.103+0000 7f2bb9591640 1 --2- 172.21.15.16:0/454432538 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb4481d210 0x55fb447f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:03.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.103+0000 7f2bb3fff640 1 --2- 172.21.15.16:0/454432538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb447d4bb0 0x55fb447f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:03.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.103+0000 7f2bb8d90640 1 --2- 172.21.15.16:0/454432538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb447f15f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0169d0 tx=0x7f2bac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44737b84e1cef11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.103+0000 7f2b937fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:07:03.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.103+0000 7f2b937fe640 1 -- 172.21.15.16:0/454432538 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac026020 con 0x55fb447d3cf0 2024-04-25T10:07:03.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.103+0000 7f2b937fe640 1 -- 172.21.15.16:0/454432538 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bac014b60 con 0x55fb447d3cf0 2024-04-25T10:07:03.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.103+0000 7f2bbddce740 1 -- 172.21.15.16:0/454432538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ba407d690 con 0x55fb447d3cf0 2024-04-25T10:07:03.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.103+0000 7f2b937fe640 1 -- 172.21.15.16:0/454432538 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bac014d60 con 0x55fb447d3cf0 2024-04-25T10:07:03.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.103+0000 7f2bbddce740 10 client.5820 fetch_fsmap learned FSMap version 21 2024-04-25T10:07:03.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.103+0000 7f2bbddce740 1 -- 172.21.15.16:0/454432538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55fb4440d170 con 0x55fb447d3cf0 2024-04-25T10:07:03.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.103+0000 7f2b937fe640 1 -- 172.21.15.16:0/454432538 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba407d690 con 0x55fb447d3cf0 2024-04-25T10:07:03.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.104+0000 7f2b937fe640 1 -- 172.21.15.16:0/454432538 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2bac033080 con 0x55fb447d3cf0 2024-04-25T10:07:03.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.104+0000 7f2bbddce740 10 client.5820 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T10:07:03.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.104+0000 7f2bbddce740 10 client.5820 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-04-25T10:07:03.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.104+0000 7f2bbddce740 20 client.5820 populate_metadata read hostname 'smithi016' 2024-04-25T10:07:03.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.104+0000 7f2bbddce740 1 --2- 172.21.15.16:0/454432538 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55fb4482e650 0x55fb4482ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.104+0000 7f2bbddce740 4 client.5820 mds_command: new command op to 34883 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:03.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.104+0000 7f2bbddce740 1 -- 172.21.15.16:0/454432538 --> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb44746000 con 0x55fb4482e650 2024-04-25T10:07:03.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.104+0000 7f2bb3fff640 1 --2- 172.21.15.16:0/454432538 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55fb4482e650 0x55fb4482ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.104+0000 7f2b937fe640 10 client.5820 ms_handle_connect on v2:172.21.15.186:6835/83423489 2024-04-25T10:07:03.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.104+0000 7f2bb3fff640 1 --2- 172.21.15.16:0/454432538 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55fb4482e650 0x55fb4482ea50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c34cfee95ab6432b server_cookie=a6253aa513c7971 in_seq=0 out_seq=0 2024-04-25T10:07:03.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.105+0000 7f2b937fe640 1 -- 172.21.15.16:0/454432538 <== mds.0 v2:172.21.15.186:6835/83423489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb44746000 con 0x55fb4482e650 2024-04-25T10:07:03.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.105+0000 7f2b937fe640 10 client.5820 handle_command_reply: tid=0 2024-04-25T10:07:03.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.105+0000 7f2bbddce740 1 client.5820 shutdown 2024-04-25T10:07:03.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.105+0000 7f2bbddce740 1 -- 172.21.15.16:0/454432538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 msgr2=0x55fb447f15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.105+0000 7f2bbddce740 1 --2- 172.21.15.16:0/454432538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb447f15f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0169d0 tx=0x7f2bac004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:03.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.105+0000 7f2bbddce740 1 -- 172.21.15.16:0/454432538 shutdown_connections 2024-04-25T10:07:03.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.105+0000 7f2bbddce740 1 --2- 172.21.15.16:0/454432538 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb4481d210 0x55fb447f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.105+0000 7f2bbddce740 1 --2- 172.21.15.16:0/454432538 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55fb4482e650 0x55fb4482ea50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.105+0000 7f2bbddce740 1 --2- 172.21.15.16:0/454432538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb447d4bb0 0x55fb447f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.105+0000 7f2bbddce740 1 --2- 172.21.15.16:0/454432538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb447f15f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.105+0000 7f2b937fe640 0 client.5820 ms_handle_reset on v2:172.21.15.186:6835/83423489 2024-04-25T10:07:03.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.105+0000 7f2bbddce740 1 -- 172.21.15.16:0/454432538 >> 172.21.15.16:0/454432538 conn(0x55fb447cf830 msgr2=0x55fb447cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.105+0000 7f2bbddce740 1 -- 172.21.15.16:0/454432538 shutdown_connections 2024-04-25T10:07:03.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.105+0000 7f2bbddce740 1 -- 172.21.15.16:0/454432538 wait complete. 2024-04-25T10:07:03.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.105+0000 7f2bbddce740 20 client.5820 trim_cache size 0 max 16384 2024-04-25T10:07:03.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.110+0000 7f75dcb70640 1 Processor -- start 2024-04-25T10:07:03.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.110+0000 7f75dcb70640 1 -- start start 2024-04-25T10:07:03.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.111+0000 7f75dcb70640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f75d815dca0 0x7f75d80cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.111+0000 7f75dcb70640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75d8162200 0x7f75d806cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.111+0000 7f75dcb70640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d815f090 con 0x7f75d80b54a0 2024-04-25T10:07:03.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.111+0000 7f75dcb70640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d806d1c0 con 0x7f75d815dca0 2024-04-25T10:07:03.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.111+0000 7f75dcb70640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d80cf340 con 0x7f75d8162200 2024-04-25T10:07:03.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.111+0000 7f75d6d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f75d80b54a0 0x7f75d80d1c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44102/0 (socket says 172.21.15.16:44102) 2024-04-25T10:07:03.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.111+0000 7f75d6d9d640 1 -- 172.21.15.16:0/3693415061 learned_addr learned my addr 172.21.15.16:0/3693415061 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:03.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.112+0000 7f75d5d9b640 1 --2- 172.21.15.16:0/3693415061 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75d8162200 0x7f75d806cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.112+0000 7f75d659c640 1 --2- 172.21.15.16:0/3693415061 >> v2:172.21.15.186:3300/0 conn(0x7f75d815dca0 0x7f75d80cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.112+0000 7f75d559a640 1 -- 172.21.15.16:0/3693415061 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 8348423 0 0) 0x7f75d815f090 con 0x7f75d80b54a0 2024-04-25T10:07:03.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.112+0000 7f75d559a640 1 -- 172.21.15.16:0/3693415061 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75c0003520 con 0x7f75d80b54a0 2024-04-25T10:07:03.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.112+0000 7f75d5d9b640 1 -- 172.21.15.16:0/3693415061 >> v2:172.21.15.186:3300/0 conn(0x7f75d815dca0 msgr2=0x7f75d80cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.112+0000 7f75d5d9b640 1 --2- 172.21.15.16:0/3693415061 >> v2:172.21.15.186:3300/0 conn(0x7f75d815dca0 0x7f75d80cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.112+0000 7f75d5d9b640 1 -- 172.21.15.16:0/3693415061 >> v1:172.21.15.16:6789/0 conn(0x7f75d80b54a0 legacy=0x7f75d80d1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.112+0000 7f75d5d9b640 1 -- 172.21.15.16:0/3693415061 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d80cf060 con 0x7f75d8162200 2024-04-25T10:07:03.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.112+0000 7f75d659c640 1 --2- 172.21.15.16:0/3693415061 >> v2:172.21.15.186:3300/0 conn(0x7f75d815dca0 0x7f75d80cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:03.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.113+0000 7f75d5d9b640 1 --2- 172.21.15.16:0/3693415061 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75d8162200 0x7f75d806cc80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f75c80097a0 tx=0x7f75c802a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96a1798eabe3b664 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.113+0000 7f75d559a640 1 -- 172.21.15.16:0/3693415061 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8037070 con 0x7f75d8162200 2024-04-25T10:07:03.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.113+0000 7f75d559a640 1 -- 172.21.15.16:0/3693415061 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c802ae40 con 0x7f75d8162200 2024-04-25T10:07:03.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.113+0000 7f75d559a640 1 -- 172.21.15.16:0/3693415061 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8032660 con 0x7f75d8162200 2024-04-25T10:07:03.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.113+0000 7f75dcb70640 1 -- 172.21.15.16:0/3693415061 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75d8162200 msgr2=0x7f75d806cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.113+0000 7f75dcb70640 1 --2- 172.21.15.16:0/3693415061 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75d8162200 0x7f75d806cc80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f75c80097a0 tx=0x7f75c802a0f0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.113+0000 7f75dcb70640 1 -- 172.21.15.16:0/3693415061 shutdown_connections 2024-04-25T10:07:03.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.113+0000 7f75dcb70640 1 --2- 172.21.15.16:0/3693415061 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75d8162200 0x7f75d806cc80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.113+0000 7f75dcb70640 1 --2- 172.21.15.16:0/3693415061 >> v2:172.21.15.186:3300/0 conn(0x7f75d815dca0 0x7f75d80cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.113+0000 7f75dcb70640 1 -- 172.21.15.16:0/3693415061 >> 172.21.15.16:0/3693415061 conn(0x7f75d80d04a0 msgr2=0x7f75d80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.113+0000 7f75dcb70640 1 -- 172.21.15.16:0/3693415061 shutdown_connections 2024-04-25T10:07:03.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.113+0000 7f75dcb70640 1 -- 172.21.15.16:0/3693415061 wait complete. 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.115+0000 7f75dcb70640 1 Processor -- start 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.115+0000 7f75dcb70640 1 -- start start 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.115+0000 7f75dcb70640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75d80b54a0 0x7f75d81f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.115+0000 7f75dcb70640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75d815dca0 0x7f75d814ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.115+0000 7f75dcb70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75d8162200 0x7f75d80cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.115+0000 7f75dcb70640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d806c480 con 0x7f75d8162200 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.115+0000 7f75dcb70640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d806c1a0 con 0x7f75d815dca0 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.116+0000 7f75dcb70640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d806c760 con 0x7f75d80b54a0 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.116+0000 7f75d6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75d8162200 0x7f75d80cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.116+0000 7f75d6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75d8162200 0x7f75d80cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49778/0 (socket says 172.21.15.16:49778) 2024-04-25T10:07:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.116+0000 7f75d6d9d640 1 -- 172.21.15.16:0/3411103386 learned_addr learned my addr 172.21.15.16:0/3411103386 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.116+0000 7f75d5d9b640 1 --2- 172.21.15.16:0/3411103386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75d815dca0 0x7f75d814ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.116+0000 7f75d659c640 1 --2- 172.21.15.16:0/3411103386 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75d80b54a0 0x7f75d81f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.116+0000 7f75d6d9d640 1 -- 172.21.15.16:0/3411103386 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75d80b54a0 msgr2=0x7f75d81f41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.116+0000 7f75d6d9d640 1 --2- 172.21.15.16:0/3411103386 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75d80b54a0 0x7f75d81f41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.116+0000 7f75d6d9d640 1 -- 172.21.15.16:0/3411103386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75d815dca0 msgr2=0x7f75d814ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.116+0000 7f75d6d9d640 1 --2- 172.21.15.16:0/3411103386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75d815dca0 0x7f75d814ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.116+0000 7f75d6d9d640 1 -- 172.21.15.16:0/3411103386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d81f4730 con 0x7f75d8162200 2024-04-25T10:07:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.116+0000 7f75d6d9d640 1 --2- 172.21.15.16:0/3411103386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75d8162200 0x7f75d80cd100 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f75d000cb40 tx=0x7f75d000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.116+0000 7f75c6ffd640 1 -- 172.21.15.16:0/3411103386 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0017070 con 0x7f75d8162200 2024-04-25T10:07:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.116+0000 7f75c6ffd640 1 -- 172.21.15.16:0/3411103386 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d000aca0 con 0x7f75d8162200 2024-04-25T10:07:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.116+0000 7f75dcb70640 1 -- 172.21.15.16:0/3411103386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d80b1000 con 0x7f75d8162200 2024-04-25T10:07:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.116+0000 7f75c6ffd640 1 -- 172.21.15.16:0/3411103386 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0012620 con 0x7f75d8162200 2024-04-25T10:07:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.116+0000 7f75dcb70640 1 -- 172.21.15.16:0/3411103386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d80b15c0 con 0x7f75d8162200 2024-04-25T10:07:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.117+0000 7f75dcb70640 1 -- 172.21.15.16:0/3411103386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f75d80749b0 con 0x7f75d8162200 2024-04-25T10:07:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.118+0000 7f75c6ffd640 1 -- 172.21.15.16:0/3411103386 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75d001a070 con 0x7f75d8162200 2024-04-25T10:07:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.118+0000 7f75c6ffd640 1 --2- 172.21.15.16:0/3411103386 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f75b806b4d0 0x7f75b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.118+0000 7f75c6ffd640 1 -- 172.21.15.16:0/3411103386 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75d008a690 con 0x7f75d8162200 2024-04-25T10:07:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.119+0000 7f75c6ffd640 1 -- 172.21.15.16:0/3411103386 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+501 (secure 0 0 0) 0x7f75d0021070 con 0x7f75d8162200 2024-04-25T10:07:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.119+0000 7f75d659c640 1 --2- 172.21.15.16:0/3411103386 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f75b806b4d0 0x7f75b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.119+0000 7f75dcb70640 1 -- 172.21.15.16:0/3411103386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f75d80d1810 con 0x7f75d8162200 2024-04-25T10:07:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.119+0000 7f75d659c640 1 --2- 172.21.15.16:0/3411103386 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f75b806b4d0 0x7f75b806d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f75d81f1360 tx=0x7f75cc009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.122+0000 7f2bbddce740 1 Processor -- start 2024-04-25T10:07:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.122+0000 7f2bbddce740 1 -- start start 2024-04-25T10:07:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.123+0000 7f2bbddce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb447d5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.123+0000 7f2bbddce740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb447d4bb0 0x55fb44819860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.123+0000 7f2bbddce740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb4481d210 0x55fb44819da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.123+0000 7f2bbddce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb41fff90 con 0x55fb447d3cf0 2024-04-25T10:07:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.123+0000 7f2bbddce740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb40b3ac0 con 0x55fb4481d210 2024-04-25T10:07:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.123+0000 7f2bbddce740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb4150ff0 con 0x55fb447d4bb0 2024-04-25T10:07:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.123+0000 7f2bb8d90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb447d5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.123+0000 7f2bb8d90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb447d5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49784/0 (socket says 172.21.15.16:49784) 2024-04-25T10:07:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.123+0000 7f2bb8d90640 1 -- 172.21.15.16:0/3822920408 learned_addr learned my addr 172.21.15.16:0/3822920408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.123+0000 7f2bb9591640 1 --2- 172.21.15.16:0/3822920408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb4481d210 0x55fb44819da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.123+0000 7f2bb3fff640 1 --2- 172.21.15.16:0/3822920408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb447d4bb0 0x55fb44819860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.124+0000 7f2bb8d90640 1 -- 172.21.15.16:0/3822920408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb447d4bb0 msgr2=0x55fb44819860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.124+0000 7f2bb8d90640 1 --2- 172.21.15.16:0/3822920408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb447d4bb0 0x55fb44819860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.124+0000 7f2bb8d90640 1 -- 172.21.15.16:0/3822920408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb4481d210 msgr2=0x55fb44819da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.124+0000 7f2bb8d90640 1 --2- 172.21.15.16:0/3822920408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb4481d210 0x55fb44819da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.124+0000 7f2bb8d90640 1 -- 172.21.15.16:0/3822920408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb447d49e0 con 0x55fb447d3cf0 2024-04-25T10:07:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.124+0000 7f2bb8d90640 1 --2- 172.21.15.16:0/3822920408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb447d5180 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2bac003ef0 tx=0x7f2bac022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f406dd620d48f0c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.124+0000 7f2ba9ffb640 1 -- 172.21.15.16:0/3822920408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac026070 con 0x55fb447d3cf0 2024-04-25T10:07:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.124+0000 7f2ba9ffb640 1 -- 172.21.15.16:0/3822920408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac014070 con 0x55fb447d3cf0 2024-04-25T10:07:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.124+0000 7f2ba9ffb640 1 -- 172.21.15.16:0/3822920408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac02b8a0 con 0x55fb447d3cf0 2024-04-25T10:07:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.125+0000 7f2bbddce740 1 -- 172.21.15.16:0/3822920408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 msgr2=0x55fb447d5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.125+0000 7f2bbddce740 1 --2- 172.21.15.16:0/3822920408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb447d5180 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2bac003ef0 tx=0x7f2bac022000 comp rx=0 tx=0).stop 2024-04-25T10:07:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.125+0000 7f2bbddce740 1 -- 172.21.15.16:0/3822920408 shutdown_connections 2024-04-25T10:07:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.125+0000 7f2bbddce740 1 --2- 172.21.15.16:0/3822920408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb4481d210 0x55fb44819da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.125+0000 7f2bbddce740 1 --2- 172.21.15.16:0/3822920408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb447d4bb0 0x55fb44819860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.125+0000 7f2bbddce740 1 --2- 172.21.15.16:0/3822920408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb447d5180 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.125+0000 7f2bbddce740 1 -- 172.21.15.16:0/3822920408 >> 172.21.15.16:0/3822920408 conn(0x55fb447cf830 msgr2=0x55fb447fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.125+0000 7f2bbddce740 1 -- 172.21.15.16:0/3822920408 shutdown_connections 2024-04-25T10:07:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.125+0000 7f2bbddce740 1 -- 172.21.15.16:0/3822920408 wait complete. 2024-04-25T10:07:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.125+0000 7f2bbddce740 1 -- start start 2024-04-25T10:07:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.125+0000 7f2bbddce740 1 Processor -- start 2024-04-25T10:07:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.126+0000 7f2bbddce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb4482eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.126+0000 7f2bbddce740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb447d4bb0 0x55fb4482f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.126+0000 7f2bbddce740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb4481d210 0x55fb447f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.126+0000 7f2bbddce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb447f00c0 con 0x55fb447d3cf0 2024-04-25T10:07:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.126+0000 7f2bbddce740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb40b34c0 con 0x55fb447d4bb0 2024-04-25T10:07:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.126+0000 7f2bbddce740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb4481ab90 con 0x55fb4481d210 2024-04-25T10:07:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.126+0000 7f2bbddce740 10 client.0 _ll_register_callbacks cb 0x7f2ba4009480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.126+0000 7f2bb8d90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb4482eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.126+0000 7f2bb8d90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb4482eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49790/0 (socket says 172.21.15.16:49790) 2024-04-25T10:07:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.126+0000 7f2bb8d90640 1 -- 172.21.15.16:0/694862631 learned_addr learned my addr 172.21.15.16:0/694862631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.126+0000 7f2bb9591640 1 --2- 172.21.15.16:0/694862631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb4481d210 0x55fb447f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.126+0000 7f2bb3fff640 1 --2- 172.21.15.16:0/694862631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb447d4bb0 0x55fb4482f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2bb8d90640 1 -- 172.21.15.16:0/694862631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb4481d210 msgr2=0x55fb447f3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2bb8d90640 1 --2- 172.21.15.16:0/694862631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb4481d210 0x55fb447f3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2bb8d90640 1 -- 172.21.15.16:0/694862631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb447d4bb0 msgr2=0x55fb4482f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2bb8d90640 1 --2- 172.21.15.16:0/694862631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb447d4bb0 0x55fb4482f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2bb8d90640 1 -- 172.21.15.16:0/694862631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb4481f780 con 0x55fb447d3cf0 2024-04-25T10:07:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2bb8d90640 1 --2- 172.21.15.16:0/694862631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb4482eff0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0042d0 tx=0x7f2bac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d71448158d0bd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2b937fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:07:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2b937fe640 1 -- 172.21.15.16:0/694862631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac026030 con 0x55fb447d3cf0 2024-04-25T10:07:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2b937fe640 1 -- 172.21.15.16:0/694862631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bac02ce10 con 0x55fb447d3cf0 2024-04-25T10:07:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2bbddce740 1 -- 172.21.15.16:0/694862631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bac014070 con 0x55fb447d3cf0 2024-04-25T10:07:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2b937fe640 1 -- 172.21.15.16:0/694862631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bac04c050 con 0x55fb447d3cf0 2024-04-25T10:07:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2bbddce740 10 client.5832 fetch_fsmap learned FSMap version 21 2024-04-25T10:07:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2bbddce740 1 -- 172.21.15.16:0/694862631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55fb447cf090 con 0x55fb447d3cf0 2024-04-25T10:07:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2b937fe640 1 -- 172.21.15.16:0/694862631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac014070 con 0x55fb447d3cf0 2024-04-25T10:07:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2b937fe640 1 -- 172.21.15.16:0/694862631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2bac04f020 con 0x55fb447d3cf0 2024-04-25T10:07:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2bbddce740 10 client.5832 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T10:07:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2bbddce740 10 client.5832 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2bbddce740 20 client.5832 populate_metadata read hostname 'smithi016' 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2bbddce740 1 --2- 172.21.15.16:0/694862631 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55fb44829d60 0x55fb447f4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2bbddce740 4 client.5832 mds_command: new command op to 34883 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.127+0000 7f2bbddce740 1 -- 172.21.15.16:0/694862631 --> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55fb44740350 con 0x55fb44829d60 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.128+0000 7f2bb3fff640 1 --2- 172.21.15.16:0/694862631 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55fb44829d60 0x55fb447f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.128+0000 7f2bb3fff640 1 --2- 172.21.15.16:0/694862631 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55fb44829d60 0x55fb447f4120 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae6ea0f18858da6c server_cookie=e4499ed8a9947062 in_seq=0 out_seq=0 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.128+0000 7f2b937fe640 10 client.5832 ms_handle_connect on v2:172.21.15.186:6835/83423489 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.129+0000 7f2b937fe640 1 -- 172.21.15.16:0/694862631 <== mds.0 v2:172.21.15.186:6835/83423489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fb44740350 con 0x55fb44829d60 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.129+0000 7f2b937fe640 10 client.5832 handle_command_reply: tid=0 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.129+0000 7f2bbddce740 1 client.5832 shutdown 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.129+0000 7f2bbddce740 1 -- 172.21.15.16:0/694862631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 msgr2=0x55fb4482eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.129+0000 7f2bbddce740 1 --2- 172.21.15.16:0/694862631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb4482eff0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0042d0 tx=0x7f2bac016000 comp rx=0 tx=0).stop 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.129+0000 7f2bbddce740 1 -- 172.21.15.16:0/694862631 shutdown_connections 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.129+0000 7f2bbddce740 1 --2- 172.21.15.16:0/694862631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fb4481d210 0x55fb447f3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.129+0000 7f2bbddce740 1 --2- 172.21.15.16:0/694862631 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55fb44829d60 0x55fb447f4120 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.129+0000 7f2bbddce740 1 --2- 172.21.15.16:0/694862631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fb447d4bb0 0x55fb4482f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.129+0000 7f2bbddce740 1 --2- 172.21.15.16:0/694862631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb447d3cf0 0x55fb4482eff0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.129+0000 7f2b937fe640 0 client.5832 ms_handle_reset on v2:172.21.15.186:6835/83423489 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.129+0000 7f2bbddce740 1 -- 172.21.15.16:0/694862631 >> 172.21.15.16:0/694862631 conn(0x55fb447cf830 msgr2=0x55fb4483d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.129+0000 7f2bbddce740 1 -- 172.21.15.16:0/694862631 shutdown_connections 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.129+0000 7f2bbddce740 1 -- 172.21.15.16:0/694862631 wait complete. 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.129+0000 7f2bbddce740 20 client.5832 trim_cache size 0 max 16384 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-04-25T10:07:03.141 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "3280e706-7f1d-4013-bc07-31c511ee9d09", 2024-04-25T10:07:03.141 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-04-25T10:07:03.141 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:07:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.130+0000 7f2bbaff4640 1 -- 172.21.15.16:0/2944544722 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2b9406b4d0 msgr2=0x7f2b9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.130+0000 7f2bbaff4640 1 --2- 172.21.15.16:0/2944544722 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2b9406b4d0 0x7f2b9406d990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0044f0 tx=0x7f2bac003a30 comp rx=0 tx=0).stop 2024-04-25T10:07:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.130+0000 7f2bbaff4640 1 -- 172.21.15.16:0/2944544722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb4161f10 msgr2=0x7f2bb415f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.130+0000 7f2bbaff4640 1 --2- 172.21.15.16:0/2944544722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb4161f10 0x7f2bb415f810 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4035040 tx=0x7f2ba40046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.130+0000 7f2bbaff4640 1 -- 172.21.15.16:0/2944544722 shutdown_connections 2024-04-25T10:07:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.130+0000 7f2bbaff4640 1 --2- 172.21.15.16:0/2944544722 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2b9406b4d0 0x7f2b9406d990 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.130+0000 7f2bbaff4640 1 --2- 172.21.15.16:0/2944544722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb4161f10 0x7f2bb415f810 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.130+0000 7f2bbaff4640 1 --2- 172.21.15.16:0/2944544722 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2bb406b420 0x7f2bb4152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.130+0000 7f2bbaff4640 1 --2- 172.21.15.16:0/2944544722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bb406b040 0x7f2bb40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.130+0000 7f2bbaff4640 1 -- 172.21.15.16:0/2944544722 >> 172.21.15.16:0/2944544722 conn(0x7f2bb4099a40 msgr2=0x7f2bb40adce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.130+0000 7f2bbaff4640 1 -- 172.21.15.16:0/2944544722 shutdown_connections 2024-04-25T10:07:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.130+0000 7f2bbaff4640 1 -- 172.21.15.16:0/2944544722 wait complete. 2024-04-25T10:07:03.143 INFO:tasks.ceph.mds.b.smithi186.stderr:2024-04-25T10:07:03.129+0000 7f25ba5b5640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5398=0-4194304@1} caps={5398=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5398 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x561185928100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:07:03.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:03.146 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:07:03.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.243+0000 7fbbcaefc640 1 Processor -- start 2024-04-25T10:07:03.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.243+0000 7fbbcaefc640 1 -- start start 2024-04-25T10:07:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.244+0000 7fbbcaefc640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbbc40d3b90 0x7fbbc40d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.244+0000 7fbbcaefc640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc40d4090 con 0x7fbbc415dca0 2024-04-25T10:07:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.244+0000 7fbbcaefc640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc40d4370 con 0x7fbbc40d0f60 2024-04-25T10:07:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.244+0000 7fbbcaefc640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc40b6b20 con 0x7fbbc40d3b90 2024-04-25T10:07:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.244+0000 7fbbc8c98640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbbc415dca0 0x7fbbc40ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44110/0 (socket says 172.21.15.16:44110) 2024-04-25T10:07:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.244+0000 7fbbc8c98640 1 -- 172.21.15.16:0/1613049571 learned_addr learned my addr 172.21.15.16:0/1613049571 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.244+0000 7fbbc3fff640 1 --2- 172.21.15.16:0/1613049571 >> v2:172.21.15.186:3300/0 conn(0x7fbbc40d3b90 0x7fbbc40d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.244+0000 7fbbc37fe640 1 -- 172.21.15.16:0/1613049571 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2347076008 0 0) 0x7fbbc40d4090 con 0x7fbbc415dca0 2024-04-25T10:07:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.244+0000 7fbbc37fe640 1 -- 172.21.15.16:0/1613049571 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbb4003520 con 0x7fbbc415dca0 2024-04-25T10:07:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.244+0000 7fbbc37fe640 1 -- 172.21.15.16:0/1613049571 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124097930 0 0) 0x7fbbc40d4370 con 0x7fbbc40d0f60 2024-04-25T10:07:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.244+0000 7fbbc37fe640 1 -- 172.21.15.16:0/1613049571 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbc40d4090 con 0x7fbbc40d0f60 2024-04-25T10:07:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.244+0000 7fbbc3fff640 1 -- 172.21.15.16:0/1613049571 >> v1:172.21.15.186:6789/0 conn(0x7fbbc40d0f60 legacy=0x7fbbc40b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.244+0000 7fbbc3fff640 1 -- 172.21.15.16:0/1613049571 >> v1:172.21.15.16:6789/0 conn(0x7fbbc415dca0 legacy=0x7fbbc40ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.244+0000 7fbbc37fe640 1 -- 172.21.15.16:0/1613049571 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1478517885 0 0) 0x7fbbb4003520 con 0x7fbbc415dca0 2024-04-25T10:07:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.244+0000 7fbbc3fff640 1 -- 172.21.15.16:0/1613049571 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc40d4650 con 0x7fbbc40d3b90 2024-04-25T10:07:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.245+0000 7fbbc3fff640 1 --2- 172.21.15.16:0/1613049571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc40d3b90 0x7fbbc40d0a20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00097a0 tx=0x7fbbb002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e93481a34787a376 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.245+0000 7fbbc37fe640 1 -- 172.21.15.16:0/1613049571 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0037070 con 0x7fbbc40d3b90 2024-04-25T10:07:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.245+0000 7fbbc37fe640 1 -- 172.21.15.16:0/1613049571 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb002ac50 con 0x7fbbc40d3b90 2024-04-25T10:07:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.245+0000 7fbbc37fe640 1 -- 172.21.15.16:0/1613049571 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8001670 con 0x7fbbc40d3b90 2024-04-25T10:07:03.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.246+0000 7fbbcaefc640 1 -- 172.21.15.16:0/1613049571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc40d3b90 msgr2=0x7fbbc40d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.246+0000 7fbbcaefc640 1 --2- 172.21.15.16:0/1613049571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc40d3b90 0x7fbbc40d0a20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00097a0 tx=0x7fbbb002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.246+0000 7fbbcaefc640 1 -- 172.21.15.16:0/1613049571 shutdown_connections 2024-04-25T10:07:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.246+0000 7fbbcaefc640 1 --2- 172.21.15.16:0/1613049571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc40d3b90 0x7fbbc40d0a20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.246+0000 7fbbcaefc640 1 -- 172.21.15.16:0/1613049571 >> 172.21.15.16:0/1613049571 conn(0x7fbbc40726c0 msgr2=0x7fbbc4072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.246+0000 7fbbcaefc640 1 -- 172.21.15.16:0/1613049571 shutdown_connections 2024-04-25T10:07:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.246+0000 7fbbcaefc640 1 -- 172.21.15.16:0/1613049571 wait complete. 2024-04-25T10:07:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.247+0000 7fbbcaefc640 1 Processor -- start 2024-04-25T10:07:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.247+0000 7fbbcaefc640 1 -- start start 2024-04-25T10:07:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.248+0000 7fbbcaefc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbc40d0f60 0x7fbbc4160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.248+0000 7fbbcaefc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbc40d3b90 0x7fbbc4157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.248+0000 7fbbcaefc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc415dca0 0x7fbbc4159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.248+0000 7fbbcaefc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc40cf050 con 0x7fbbc40d0f60 2024-04-25T10:07:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.248+0000 7fbbcaefc640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc40d1390 con 0x7fbbc415dca0 2024-04-25T10:07:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.248+0000 7fbbcaefc640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc415e080 con 0x7fbbc40d3b90 2024-04-25T10:07:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.248+0000 7fbbc8c98640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbc40d0f60 0x7fbbc4160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.248+0000 7fbbc8c98640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbc40d0f60 0x7fbbc4160830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49792/0 (socket says 172.21.15.16:49792) 2024-04-25T10:07:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.248+0000 7fbbc8c98640 1 -- 172.21.15.16:0/3618553419 learned_addr learned my addr 172.21.15.16:0/3618553419 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.248+0000 7fbbc9499640 1 --2- 172.21.15.16:0/3618553419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc415dca0 0x7fbbc4159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.248+0000 7fbbc3fff640 1 --2- 172.21.15.16:0/3618553419 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbc40d3b90 0x7fbbc4157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.249+0000 7fbbc8c98640 1 -- 172.21.15.16:0/3618553419 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbc40d3b90 msgr2=0x7fbbc4157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.249+0000 7fbbc8c98640 1 --2- 172.21.15.16:0/3618553419 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbc40d3b90 0x7fbbc4157310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.249+0000 7fbbc8c98640 1 -- 172.21.15.16:0/3618553419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc415dca0 msgr2=0x7fbbc4159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.249+0000 7fbbc8c98640 1 --2- 172.21.15.16:0/3618553419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc415dca0 0x7fbbc4159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.249+0000 7fbbc8c98640 1 -- 172.21.15.16:0/3618553419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc415f790 con 0x7fbbc40d0f60 2024-04-25T10:07:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.249+0000 7fbbc8c98640 1 --2- 172.21.15.16:0/3618553419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbc40d0f60 0x7fbbc4160830 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800c4f0 tx=0x7fbbb800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.249+0000 7fbbc17fa640 1 -- 172.21.15.16:0/3618553419 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8017070 con 0x7fbbc40d0f60 2024-04-25T10:07:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.249+0000 7fbbc17fa640 1 -- 172.21.15.16:0/3618553419 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb8005ce0 con 0x7fbbc40d0f60 2024-04-25T10:07:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.249+0000 7fbbcaefc640 1 -- 172.21.15.16:0/3618553419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc415fa70 con 0x7fbbc40d0f60 2024-04-25T10:07:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.249+0000 7fbbc17fa640 1 -- 172.21.15.16:0/3618553419 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8007290 con 0x7fbbc40d0f60 2024-04-25T10:07:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.249+0000 7fbbcaefc640 1 -- 172.21.15.16:0/3618553419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc4160030 con 0x7fbbc40d0f60 2024-04-25T10:07:03.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.250+0000 7fbbc17fa640 1 -- 172.21.15.16:0/3618553419 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbbb801a070 con 0x7fbbc40d0f60 2024-04-25T10:07:03.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.251+0000 7fbbc17fa640 1 --2- 172.21.15.16:0/3618553419 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbba806b4d0 0x7fbba806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.251+0000 7fbbc17fa640 1 -- 172.21.15.16:0/3618553419 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbb8089650 con 0x7fbbc40d0f60 2024-04-25T10:07:03.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.251+0000 7fbbc3fff640 1 --2- 172.21.15.16:0/3618553419 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbba806b4d0 0x7fbba806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.251+0000 7fbbc3fff640 1 --2- 172.21.15.16:0/3618553419 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbba806b4d0 0x7fbba806d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0034000 tx=0x7fbbb002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.252+0000 7fbbcdbf7740 1 Processor -- start 2024-04-25T10:07:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.252+0000 7fbbcdbf7740 1 -- start start 2024-04-25T10:07:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.252+0000 7fbbcdbf7740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b909b10 0x55cb7b90c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.252+0000 7fbbcdbf7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909ef0 0x55cb7b90c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.252+0000 7fbbcdbf7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b8dc770 0x55cb7b8dff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.253+0000 7fbbcdbf7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7b845da0 con 0x55cb7b909ef0 2024-04-25T10:07:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.253+0000 7fbbcdbf7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7b7eeb80 con 0x55cb7b8dc770 2024-04-25T10:07:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.253+0000 7fbbcdbf7740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb7b841e40 con 0x55cb7b909b10 2024-04-25T10:07:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.253+0000 7fbbc3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909ef0 0x55cb7b90c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.253+0000 7fbbc3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909ef0 0x55cb7b90c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49796/0 (socket says 172.21.15.16:49796) 2024-04-25T10:07:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.253+0000 7fbbc3fff640 1 -- 172.21.15.16:0/495907000 learned_addr learned my addr 172.21.15.16:0/495907000 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.253+0000 7fbbc8c98640 1 --2- 172.21.15.16:0/495907000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b909b10 0x55cb7b90c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.253+0000 7fbbc9499640 1 --2- 172.21.15.16:0/495907000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b8dc770 0x55cb7b8dff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.253+0000 7fbbc3fff640 1 -- 172.21.15.16:0/495907000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b909b10 msgr2=0x55cb7b90c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.253+0000 7fbbc3fff640 1 --2- 172.21.15.16:0/495907000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b909b10 0x55cb7b90c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.253+0000 7fbbc3fff640 1 -- 172.21.15.16:0/495907000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b8dc770 msgr2=0x55cb7b8dff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.253+0000 7fbbc3fff640 1 --2- 172.21.15.16:0/495907000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b8dc770 0x55cb7b8dff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.253+0000 7fbbc3fff640 1 -- 172.21.15.16:0/495907000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb7b7f8f10 con 0x55cb7b909ef0 2024-04-25T10:07:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.253+0000 7fbbc8c98640 1 --2- 172.21.15.16:0/495907000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b909b10 0x55cb7b90c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.253+0000 7fbbc3fff640 1 --2- 172.21.15.16:0/495907000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909ef0 0x55cb7b90c860 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbbb003e040 tx=0x7fbbb002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef116295007c856e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.253+0000 7fbbae7fc640 1 -- 172.21.15.16:0/495907000 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb004e070 con 0x55cb7b909ef0 2024-04-25T10:07:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.253+0000 7fbbae7fc640 1 -- 172.21.15.16:0/495907000 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb003c780 con 0x55cb7b909ef0 2024-04-25T10:07:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.253+0000 7fbbae7fc640 1 -- 172.21.15.16:0/495907000 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0037040 con 0x55cb7b909ef0 2024-04-25T10:07:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.254+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/495907000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909ef0 msgr2=0x55cb7b90c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.254+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/495907000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909ef0 0x55cb7b90c860 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbbb003e040 tx=0x7fbbb002c000 comp rx=0 tx=0).stop 2024-04-25T10:07:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.254+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/495907000 shutdown_connections 2024-04-25T10:07:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.254+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/495907000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b8dc770 0x55cb7b8dff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.254+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/495907000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909ef0 0x55cb7b90c860 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.254+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/495907000 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b909b10 0x55cb7b90c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.254+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/495907000 >> 172.21.15.16:0/495907000 conn(0x55cb7b8de9e0 msgr2=0x55cb7b927320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.254+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/495907000 shutdown_connections 2024-04-25T10:07:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.254+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/495907000 wait complete. 2024-04-25T10:07:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.255+0000 7fbbcdbf7740 1 -- start start 2024-04-25T10:07:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.255+0000 7fbbcdbf7740 1 Processor -- start 2024-04-25T10:07:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbcdbf7740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b909b10 0x55cb7b8feea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbcdbf7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909ef0 0x55cb7b908190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbcdbf7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b90fb00 0x55cb7b9086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbcdbf7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7b8e1b90 con 0x55cb7b909ef0 2024-04-25T10:07:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbcdbf7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7b7e44b0 con 0x55cb7b90fb00 2024-04-25T10:07:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbcdbf7740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb7b8e1e70 con 0x55cb7b909b10 2024-04-25T10:07:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbcdbf7740 10 client.0 _ll_register_callbacks cb 0x55cb7b824180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbc3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909ef0 0x55cb7b908190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbc3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909ef0 0x55cb7b908190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49806/0 (socket says 172.21.15.16:49806) 2024-04-25T10:07:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbc3fff640 1 -- 172.21.15.16:0/4252319151 learned_addr learned my addr 172.21.15.16:0/4252319151 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbc9499640 1 --2- 172.21.15.16:0/4252319151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b90fb00 0x55cb7b9086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbc3fff640 1 -- 172.21.15.16:0/4252319151 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b909b10 msgr2=0x55cb7b8feea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbc8c98640 1 --2- 172.21.15.16:0/4252319151 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b909b10 0x55cb7b8feea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbc3fff640 1 --2- 172.21.15.16:0/4252319151 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b909b10 0x55cb7b8feea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbc3fff640 1 -- 172.21.15.16:0/4252319151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b90fb00 msgr2=0x55cb7b9086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbc3fff640 1 --2- 172.21.15.16:0/4252319151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b90fb00 0x55cb7b9086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbc3fff640 1 -- 172.21.15.16:0/4252319151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb7b6247d0 con 0x55cb7b909ef0 2024-04-25T10:07:03.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbc9499640 1 --2- 172.21.15.16:0/4252319151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b90fb00 0x55cb7b9086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:03.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbc3fff640 1 --2- 172.21.15.16:0/4252319151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909ef0 0x55cb7b908190 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00098c0 tx=0x7fbbb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3739c9948c33bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbba37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:07:03.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbba37fe640 1 -- 172.21.15.16:0/4252319151 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0037030 con 0x55cb7b909ef0 2024-04-25T10:07:03.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbba37fe640 1 -- 172.21.15.16:0/4252319151 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbb0055cf0 con 0x55cb7b909ef0 2024-04-25T10:07:03.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.256+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/4252319151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbb003c780 con 0x55cb7b909ef0 2024-04-25T10:07:03.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.257+0000 7fbba37fe640 1 -- 172.21.15.16:0/4252319151 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbb004e040 con 0x55cb7b909ef0 2024-04-25T10:07:03.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.257+0000 7fbbcdbf7740 10 client.5844 fetch_fsmap learned FSMap version 21 2024-04-25T10:07:03.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.257+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/4252319151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55cb7b55a3f0 con 0x55cb7b909ef0 2024-04-25T10:07:03.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.257+0000 7fbba37fe640 1 -- 172.21.15.16:0/4252319151 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb003c780 con 0x55cb7b909ef0 2024-04-25T10:07:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.257+0000 7fbba37fe640 1 -- 172.21.15.16:0/4252319151 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbbb005c050 con 0x55cb7b909ef0 2024-04-25T10:07:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.257+0000 7fbbcdbf7740 10 client.5844 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T10:07:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.257+0000 7fbbcdbf7740 10 client.5844 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-04-25T10:07:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.257+0000 7fbbcdbf7740 20 client.5844 populate_metadata read hostname 'smithi016' 2024-04-25T10:07:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.257+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/4252319151 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55cb7b90e370 0x55cb7b90e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.257+0000 7fbbcdbf7740 4 client.5844 mds_command: new command op to 34883 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.257+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/4252319151 --> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb7b8532e0 con 0x55cb7b90e370 2024-04-25T10:07:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.257+0000 7fbbc8c98640 1 --2- 172.21.15.16:0/4252319151 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55cb7b90e370 0x55cb7b90e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.257+0000 7fbba37fe640 10 client.5844 ms_handle_connect on v2:172.21.15.186:6835/83423489 2024-04-25T10:07:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.257+0000 7fbbc8c98640 1 --2- 172.21.15.16:0/4252319151 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55cb7b90e370 0x55cb7b90e750 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db2a9673a5504ad5 server_cookie=b6559ed7850af903 in_seq=0 out_seq=0 2024-04-25T10:07:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.258+0000 7fbba37fe640 1 -- 172.21.15.16:0/4252319151 <== mds.0 v2:172.21.15.186:6835/83423489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cb7b8532e0 con 0x55cb7b90e370 2024-04-25T10:07:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.258+0000 7fbba37fe640 10 client.5844 handle_command_reply: tid=0 2024-04-25T10:07:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.258+0000 7fbbcdbf7740 1 client.5844 shutdown 2024-04-25T10:07:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.258+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/4252319151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909ef0 msgr2=0x55cb7b908190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.258+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/4252319151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909ef0 0x55cb7b908190 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00098c0 tx=0x7fbbb0004730 comp rx=0 tx=0).stop 2024-04-25T10:07:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.258+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/4252319151 shutdown_connections 2024-04-25T10:07:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.258+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/4252319151 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55cb7b90e370 0x55cb7b90e750 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.258+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/4252319151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b90fb00 0x55cb7b9086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.258+0000 7fbba37fe640 0 client.5844 ms_handle_reset on v2:172.21.15.186:6835/83423489 2024-04-25T10:07:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.258+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/4252319151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909ef0 0x55cb7b908190 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.258+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/4252319151 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b909b10 0x55cb7b8feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.258+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/4252319151 >> 172.21.15.16:0/4252319151 conn(0x55cb7b8dc770 msgr2=0x55cb7b8dd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.258+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/4252319151 shutdown_connections 2024-04-25T10:07:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.258+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/4252319151 wait complete. 2024-04-25T10:07:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.258+0000 7fbbcdbf7740 20 client.5844 trim_cache size 0 max 16384 2024-04-25T10:07:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.273+0000 7fbbcdbf7740 1 Processor -- start 2024-04-25T10:07:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.273+0000 7fbbcdbf7740 1 -- start start 2024-04-25T10:07:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.274+0000 7fbbcdbf7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909b10 0x55cb7b949bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.274+0000 7fbbcdbf7740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b909ef0 0x55cb7b9268f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.274+0000 7fbbcdbf7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b90fb00 0x55cb7b8db600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.274+0000 7fbbcdbf7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc41ffb40 con 0x55cb7b909b10 2024-04-25T10:07:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.274+0000 7fbbcdbf7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc41f7a00 con 0x55cb7b90fb00 2024-04-25T10:07:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.274+0000 7fbbcdbf7740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc4155210 con 0x55cb7b909ef0 2024-04-25T10:07:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.274+0000 7fbbc8c98640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909b10 0x55cb7b949bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.274+0000 7fbbc8c98640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909b10 0x55cb7b949bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49810/0 (socket says 172.21.15.16:49810) 2024-04-25T10:07:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.274+0000 7fbbc8c98640 1 -- 172.21.15.16:0/4081613041 learned_addr learned my addr 172.21.15.16:0/4081613041 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.274+0000 7fbbc9499640 1 --2- 172.21.15.16:0/4081613041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b90fb00 0x55cb7b8db600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.274+0000 7fbbc3fff640 1 --2- 172.21.15.16:0/4081613041 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b909ef0 0x55cb7b9268f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.275+0000 7fbbc8c98640 1 -- 172.21.15.16:0/4081613041 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b909ef0 msgr2=0x55cb7b9268f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.275+0000 7fbbc8c98640 1 --2- 172.21.15.16:0/4081613041 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b909ef0 0x55cb7b9268f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.275+0000 7fbbc8c98640 1 -- 172.21.15.16:0/4081613041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b90fb00 msgr2=0x55cb7b8db600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.275+0000 7fbbc8c98640 1 --2- 172.21.15.16:0/4081613041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b90fb00 0x55cb7b8db600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.275+0000 7fbbc8c98640 1 -- 172.21.15.16:0/4081613041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb7b51a0f0 con 0x55cb7b909b10 2024-04-25T10:07:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.275+0000 7fbbc9499640 1 --2- 172.21.15.16:0/4081613041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b90fb00 0x55cb7b8db600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.275+0000 7fbbc8c98640 1 --2- 172.21.15.16:0/4081613041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909b10 0x55cb7b949bb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8057ae0 tx=0x7fbbb8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03ba607773696a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.275+0000 7fbbadffb640 1 -- 172.21.15.16:0/4081613041 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8086070 con 0x55cb7b909b10 2024-04-25T10:07:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.275+0000 7fbbadffb640 1 -- 172.21.15.16:0/4081613041 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb8054450 con 0x55cb7b909b10 2024-04-25T10:07:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.275+0000 7fbbadffb640 1 -- 172.21.15.16:0/4081613041 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb805e170 con 0x55cb7b909b10 2024-04-25T10:07:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.275+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/4081613041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909b10 msgr2=0x55cb7b949bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.275+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/4081613041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909b10 0x55cb7b949bb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8057ae0 tx=0x7fbbb8000bc0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.275+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/4081613041 shutdown_connections 2024-04-25T10:07:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.275+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/4081613041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b90fb00 0x55cb7b8db600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.275+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/4081613041 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b909ef0 0x55cb7b9268f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.275+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/4081613041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909b10 0x55cb7b949bb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.275+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/4081613041 >> 172.21.15.16:0/4081613041 conn(0x55cb7b8dc770 msgr2=0x55cb7b927320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.276+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/4081613041 shutdown_connections 2024-04-25T10:07:03.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.276+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/4081613041 wait complete. 2024-04-25T10:07:03.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.276+0000 7fbbcdbf7740 1 -- start start 2024-04-25T10:07:03.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.276+0000 7fbbcdbf7740 1 Processor -- start 2024-04-25T10:07:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.277+0000 7fbbcdbf7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909b10 0x55cb7b949690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.277+0000 7fbbcdbf7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b909ef0 0x55cb7b9908c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.277+0000 7fbbcdbf7740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b90fb00 0x55cb7b8fecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.277+0000 7fbbcdbf7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7b8d8340 con 0x55cb7b909b10 2024-04-25T10:07:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.277+0000 7fbbcdbf7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc41f7400 con 0x55cb7b909ef0 2024-04-25T10:07:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.277+0000 7fbbcdbf7740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb7b90c6a0 con 0x55cb7b90fb00 2024-04-25T10:07:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.277+0000 7fbbcdbf7740 10 client.0 _ll_register_callbacks cb 0x7fbbb40036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.277+0000 7fbbc8c98640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909b10 0x55cb7b949690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.277+0000 7fbbc8c98640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909b10 0x55cb7b949690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49824/0 (socket says 172.21.15.16:49824) 2024-04-25T10:07:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.277+0000 7fbbc8c98640 1 -- 172.21.15.16:0/1403248286 learned_addr learned my addr 172.21.15.16:0/1403248286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.277+0000 7fbbc3fff640 1 --2- 172.21.15.16:0/1403248286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b909ef0 0x55cb7b9908c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.277+0000 7fbbc9499640 1 --2- 172.21.15.16:0/1403248286 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b90fb00 0x55cb7b8fecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.277+0000 7fbbc8c98640 1 -- 172.21.15.16:0/1403248286 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b90fb00 msgr2=0x55cb7b8fecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.277+0000 7fbbc8c98640 1 --2- 172.21.15.16:0/1403248286 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b90fb00 0x55cb7b8fecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.277+0000 7fbbc8c98640 1 -- 172.21.15.16:0/1403248286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b909ef0 msgr2=0x55cb7b9908c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.277+0000 7fbbc8c98640 1 --2- 172.21.15.16:0/1403248286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b909ef0 0x55cb7b9908c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.277+0000 7fbbc8c98640 1 -- 172.21.15.16:0/1403248286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb7b990e00 con 0x55cb7b909b10 2024-04-25T10:07:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.278+0000 7fbbc8c98640 1 --2- 172.21.15.16:0/1403248286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909b10 0x55cb7b949690 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8057ae0 tx=0x7fbbb80879b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46db00510c78d9b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.278+0000 7fbba37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:07:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.278+0000 7fbba37fe640 1 -- 172.21.15.16:0/1403248286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8086070 con 0x55cb7b909b10 2024-04-25T10:07:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.278+0000 7fbba37fe640 1 -- 172.21.15.16:0/1403248286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbb8007430 con 0x55cb7b909b10 2024-04-25T10:07:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.278+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/1403248286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbb8054450 con 0x55cb7b909b10 2024-04-25T10:07:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.278+0000 7fbba37fe640 1 -- 172.21.15.16:0/1403248286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbb805e170 con 0x55cb7b909b10 2024-04-25T10:07:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.278+0000 7fbbcdbf7740 10 client.5850 fetch_fsmap learned FSMap version 21 2024-04-25T10:07:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.278+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/1403248286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55cb7b8ff470 con 0x55cb7b909b10 2024-04-25T10:07:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.278+0000 7fbba37fe640 1 -- 172.21.15.16:0/1403248286 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb8054450 con 0x55cb7b909b10 2024-04-25T10:07:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.278+0000 7fbba37fe640 1 -- 172.21.15.16:0/1403248286 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbbb8057230 con 0x55cb7b909b10 2024-04-25T10:07:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.278+0000 7fbbcdbf7740 10 client.5850 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T10:07:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.278+0000 7fbbcdbf7740 10 client.5850 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-04-25T10:07:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.278+0000 7fbbcdbf7740 20 client.5850 populate_metadata read hostname 'smithi016' 2024-04-25T10:07:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.278+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/1403248286 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55cb7b9043c0 0x55cb7b994dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.278+0000 7fbbcdbf7740 4 client.5850 mds_command: new command op to 34883 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:07:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.278+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/1403248286 --> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cb7b84d630 con 0x55cb7b9043c0 2024-04-25T10:07:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.278+0000 7fbbc3fff640 1 --2- 172.21.15.16:0/1403248286 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55cb7b9043c0 0x55cb7b994dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.279+0000 7fbbc3fff640 1 --2- 172.21.15.16:0/1403248286 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55cb7b9043c0 0x55cb7b994dc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38251451341e50f3 server_cookie=fdfd7dcea2ab2181 in_seq=0 out_seq=0 2024-04-25T10:07:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.279+0000 7fbba37fe640 10 client.5850 ms_handle_connect on v2:172.21.15.186:6835/83423489 2024-04-25T10:07:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.279+0000 7fbba37fe640 1 -- 172.21.15.16:0/1403248286 <== mds.0 v2:172.21.15.186:6835/83423489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55cb7b84d630 con 0x55cb7b9043c0 2024-04-25T10:07:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.279+0000 7fbba37fe640 10 client.5850 handle_command_reply: tid=0 2024-04-25T10:07:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.279+0000 7fbbcdbf7740 1 client.5850 shutdown 2024-04-25T10:07:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.279+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/1403248286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909b10 msgr2=0x55cb7b949690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.279+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/1403248286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909b10 0x55cb7b949690 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8057ae0 tx=0x7fbbb80879b0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.279+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/1403248286 shutdown_connections 2024-04-25T10:07:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.279+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/1403248286 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cb7b90fb00 0x55cb7b8fecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.279+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/1403248286 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x55cb7b9043c0 0x55cb7b994dc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.279+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/1403248286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb7b909ef0 0x55cb7b9908c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.279+0000 7fbbcdbf7740 1 --2- 172.21.15.16:0/1403248286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb7b909b10 0x55cb7b949690 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.279+0000 7fbba37fe640 0 client.5850 ms_handle_reset on v2:172.21.15.186:6835/83423489 2024-04-25T10:07:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.279+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/1403248286 >> 172.21.15.16:0/1403248286 conn(0x55cb7b8dc770 msgr2=0x55cb7b8d7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.279+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/1403248286 shutdown_connections 2024-04-25T10:07:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.279+0000 7fbbcdbf7740 1 -- 172.21.15.16:0/1403248286 wait complete. 2024-04-25T10:07:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.279+0000 7fbbcdbf7740 20 client.5850 trim_cache size 0 max 16384 2024-04-25T10:07:03.293 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:07:03.293 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:07:03.293 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-04-25T10:07:03.293 INFO:teuthology.orchestra.run.smithi016.stdout: "3280e706-7f1d-4013-bc07-31c511ee9d09": { 2024-04-25T10:07:03.293 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-04-25T10:07:03.293 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "3280e706-7f1d-4013-bc07-31c511ee9d09", 2024-04-25T10:07:03.293 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,repair" 2024-04-25T10:07:03.293 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:03.293 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:03.293 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:07:03.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.280+0000 7fbbcaefc640 1 -- 172.21.15.16:0/3618553419 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbba806b4d0 msgr2=0x7fbba806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.280+0000 7fbbcaefc640 1 --2- 172.21.15.16:0/3618553419 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbba806b4d0 0x7fbba806d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0034000 tx=0x7fbbb002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:03.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.280+0000 7fbbcaefc640 1 -- 172.21.15.16:0/3618553419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbc40d0f60 msgr2=0x7fbbc4160830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.280+0000 7fbbcaefc640 1 --2- 172.21.15.16:0/3618553419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbc40d0f60 0x7fbbc4160830 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800c4f0 tx=0x7fbbb800c9b0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.280+0000 7fbbcaefc640 1 -- 172.21.15.16:0/3618553419 shutdown_connections 2024-04-25T10:07:03.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.280+0000 7fbbcaefc640 1 --2- 172.21.15.16:0/3618553419 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbba806b4d0 0x7fbba806d990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.280+0000 7fbbcaefc640 1 --2- 172.21.15.16:0/3618553419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc415dca0 0x7fbbc4159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.280+0000 7fbbcaefc640 1 --2- 172.21.15.16:0/3618553419 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbc40d3b90 0x7fbbc4157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.280+0000 7fbbcaefc640 1 --2- 172.21.15.16:0/3618553419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbc40d0f60 0x7fbbc4160830 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.280+0000 7fbbcaefc640 1 -- 172.21.15.16:0/3618553419 >> 172.21.15.16:0/3618553419 conn(0x7fbbc40726c0 msgr2=0x7fbbc40d1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.280+0000 7fbbcaefc640 1 -- 172.21.15.16:0/3618553419 shutdown_connections 2024-04-25T10:07:03.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.280+0000 7fbbcaefc640 1 -- 172.21.15.16:0/3618553419 wait complete. 2024-04-25T10:07:03.296 INFO:tasks.cephfs.filesystem:scrub status for tag:3280e706-7f1d-4013-bc07-31c511ee9d09 - {'path': '/', 'tag': '3280e706-7f1d-4013-bc07-31c511ee9d09', 'options': 'recursive,repair'} 2024-04-25T10:07:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:03.861+0000 7f75c6ffd640 1 -- 172.21.15.16:0/3411103386 <== mon.0 v2:172.21.15.16:3300/0 7 ==== log(6 entries from seq 1 at 2024-04-25T10:07:03.129602+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f75d0088cd0 con 0x7f75d8162200 2024-04-25T10:07:03.867 INFO:teuthology.orchestra.run.smithi016.stdout: cluster: 2024-04-25T10:07:03.867 INFO:teuthology.orchestra.run.smithi016.stdout: id: 34bc5eca-3a21-42a6-a26f-07da0905e8bc 2024-04-25T10:07:03.867 INFO:teuthology.orchestra.run.smithi016.stdout: health: HEALTH_OK 2024-04-25T10:07:03.867 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:03.868 INFO:teuthology.orchestra.run.smithi016.stdout: services: 2024-04-25T10:07:03.868 INFO:teuthology.orchestra.run.smithi016.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-04-25T10:07:03.868 INFO:teuthology.orchestra.run.smithi016.stdout: mgr: x(active, since 2m), standbys: y 2024-04-25T10:07:03.868 INFO:teuthology.orchestra.run.smithi016.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T10:07:03.868 INFO:teuthology.orchestra.run.smithi016.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-04-25T10:07:03.868 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:03.868 INFO:teuthology.orchestra.run.smithi016.stdout: data: 2024-04-25T10:07:03.868 INFO:teuthology.orchestra.run.smithi016.stdout: volumes: 1/1 healthy 2024-04-25T10:07:03.868 INFO:teuthology.orchestra.run.smithi016.stdout: pools: 3 pools, 97 pgs 2024-04-25T10:07:03.868 INFO:teuthology.orchestra.run.smithi016.stdout: objects: 24 objects, 579 KiB 2024-04-25T10:07:03.868 INFO:teuthology.orchestra.run.smithi016.stdout: usage: 237 MiB used, 720 GiB / 720 GiB avail 2024-04-25T10:07:03.868 INFO:teuthology.orchestra.run.smithi016.stdout: pgs: 97 active+clean 2024-04-25T10:07:03.868 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:03.868 INFO:teuthology.orchestra.run.smithi016.stdout: io: 2024-04-25T10:07:03.869 INFO:teuthology.orchestra.run.smithi016.stdout: client: 1.4 KiB/s wr, 0 op/s rd, 4 op/s wr 2024-04-25T10:07:03.869 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:03.869 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:03.869 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:07:03.129602+0000 mds.b [INF] scrub queued for path: / 2024-04-25T10:07:03.869 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:07:03.129608+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:07:03.869 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:07:03.129616+0000 mds.b [INF] scrub summary: active paths [/] 2024-04-25T10:07:03.869 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:07:03.130504+0000 mds.b [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-04-25T10:07:03.869 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:07:03.130514+0000 mds.b [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-04-25T10:07:03.869 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:07:03.130910+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:07:07.192 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:07.195 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:07.871+0000 7f75c6ffd640 1 -- 172.21.15.16:0/3411103386 <== mon.0 v2:172.21.15.16:3300/0 8 ==== log(1 entries from seq 7 at 2024-04-25T10:07:06.803423+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f75d0056f60 con 0x7f75d8162200 2024-04-25T10:07:07.877 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:07:06.803423+0000 mds.b [INF] scrub summary: idle 2024-04-25T10:07:33.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:33.298 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:07:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.409+0000 7f24bb63f640 1 Processor -- start 2024-04-25T10:07:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.409+0000 7f24bb63f640 1 -- start start 2024-04-25T10:07:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.410+0000 7f24bb63f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24b415dca0 0x7f24b40ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.410+0000 7f24bb63f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f24b40d3b90 0x7f24b40d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.410+0000 7f24bb63f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b40d4090 con 0x7f24b40cee80 2024-04-25T10:07:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.410+0000 7f24bb63f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b40b6b20 con 0x7f24b415dca0 2024-04-25T10:07:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.410+0000 7f24bb63f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b40d4370 con 0x7f24b40d3b90 2024-04-25T10:07:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.410+0000 7f24b93db640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24b415dca0 0x7f24b40ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.410+0000 7f24b93db640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24b415dca0 0x7f24b40ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47966/0 (socket says 172.21.15.16:47966) 2024-04-25T10:07:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.410+0000 7f24b93db640 1 -- 172.21.15.16:0/4294641369 learned_addr learned my addr 172.21.15.16:0/4294641369 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.410+0000 7f24b93db640 1 -- 172.21.15.16:0/4294641369 >> v2:172.21.15.186:3300/0 conn(0x7f24b40d3b90 msgr2=0x7f24b40d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.410+0000 7f24b93db640 1 --2- 172.21.15.16:0/4294641369 >> v2:172.21.15.186:3300/0 conn(0x7f24b40d3b90 0x7f24b40d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.410+0000 7f24b93db640 1 -- 172.21.15.16:0/4294641369 >> v1:172.21.15.186:6789/0 conn(0x7f24b40cee80 legacy=0x7f24b40b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.410+0000 7f24b93db640 1 -- 172.21.15.16:0/4294641369 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b40d4650 con 0x7f24b415dca0 2024-04-25T10:07:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.411+0000 7f24b93db640 1 --2- 172.21.15.16:0/4294641369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24b415dca0 0x7f24b40ce940 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f24b40b6560 tx=0x7f24b002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152d136baff83898 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.411+0000 7f24abfff640 1 -- 172.21.15.16:0/4294641369 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0037070 con 0x7f24b415dca0 2024-04-25T10:07:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.411+0000 7f24abfff640 1 -- 172.21.15.16:0/4294641369 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b0032440 con 0x7f24b415dca0 2024-04-25T10:07:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.411+0000 7f24abfff640 1 -- 172.21.15.16:0/4294641369 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0032810 con 0x7f24b415dca0 2024-04-25T10:07:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.412+0000 7f24bb63f640 1 -- 172.21.15.16:0/4294641369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24b415dca0 msgr2=0x7f24b40ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.412+0000 7f24bb63f640 1 --2- 172.21.15.16:0/4294641369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24b415dca0 0x7f24b40ce940 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f24b40b6560 tx=0x7f24b002a230 comp rx=0 tx=0).stop 2024-04-25T10:07:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.412+0000 7f24bb63f640 1 -- 172.21.15.16:0/4294641369 shutdown_connections 2024-04-25T10:07:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.412+0000 7f24bb63f640 1 --2- 172.21.15.16:0/4294641369 >> v2:172.21.15.186:3300/0 conn(0x7f24b40d3b90 0x7f24b40d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.412+0000 7f24bb63f640 1 --2- 172.21.15.16:0/4294641369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24b415dca0 0x7f24b40ce940 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.412+0000 7f24bb63f640 1 -- 172.21.15.16:0/4294641369 >> 172.21.15.16:0/4294641369 conn(0x7f24b40726c0 msgr2=0x7f24b4072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.412+0000 7f24bb63f640 1 -- 172.21.15.16:0/4294641369 shutdown_connections 2024-04-25T10:07:33.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.412+0000 7f24bb63f640 1 -- 172.21.15.16:0/4294641369 wait complete. 2024-04-25T10:07:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.413+0000 7f24bb63f640 1 Processor -- start 2024-04-25T10:07:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.413+0000 7f24bb63f640 1 -- start start 2024-04-25T10:07:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.414+0000 7f24bb63f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24b40cee80 0x7f24b40c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.414+0000 7f24bb63f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24b40d3b90 0x7f24b40c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.414+0000 7f24bb63f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24b415dca0 0x7f24b4160430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.414+0000 7f24bb63f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b415e200 con 0x7f24b40d3b90 2024-04-25T10:07:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.414+0000 7f24bb63f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b415e080 con 0x7f24b40cee80 2024-04-25T10:07:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.414+0000 7f24bb63f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b40d0f60 con 0x7f24b415dca0 2024-04-25T10:07:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.414+0000 7f24b8bda640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24b40d3b90 0x7f24b40c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.414+0000 7f24b8bda640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24b40d3b90 0x7f24b40c6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47978/0 (socket says 172.21.15.16:47978) 2024-04-25T10:07:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.414+0000 7f24b8bda640 1 -- 172.21.15.16:0/2756836 learned_addr learned my addr 172.21.15.16:0/2756836 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.414+0000 7f24b93db640 1 --2- 172.21.15.16:0/2756836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24b40cee80 0x7f24b40c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.414+0000 7f24b9bdc640 1 --2- 172.21.15.16:0/2756836 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24b415dca0 0x7f24b4160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.415+0000 7f24b8bda640 1 -- 172.21.15.16:0/2756836 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24b415dca0 msgr2=0x7f24b4160430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.415+0000 7f24b8bda640 1 --2- 172.21.15.16:0/2756836 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24b415dca0 0x7f24b4160430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.415+0000 7f24b8bda640 1 -- 172.21.15.16:0/2756836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24b40cee80 msgr2=0x7f24b40c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.415+0000 7f24b8bda640 1 --2- 172.21.15.16:0/2756836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24b40cee80 0x7f24b40c80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.415+0000 7f24b8bda640 1 -- 172.21.15.16:0/2756836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b40c85e0 con 0x7f24b40d3b90 2024-04-25T10:07:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.415+0000 7f24b8bda640 1 --2- 172.21.15.16:0/2756836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24b40d3b90 0x7f24b40c6c70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f24a400b450 tx=0x7f24a400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.415+0000 7f24a9ffb640 1 -- 172.21.15.16:0/2756836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a4015020 con 0x7f24b40d3b90 2024-04-25T10:07:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.415+0000 7f24a9ffb640 1 -- 172.21.15.16:0/2756836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a4011890 con 0x7f24b40d3b90 2024-04-25T10:07:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.415+0000 7f24bb63f640 1 -- 172.21.15.16:0/2756836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b40cb980 con 0x7f24b40d3b90 2024-04-25T10:07:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.415+0000 7f24a9ffb640 1 -- 172.21.15.16:0/2756836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a4011c80 con 0x7f24b40d3b90 2024-04-25T10:07:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.415+0000 7f24bb63f640 1 -- 172.21.15.16:0/2756836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b40c75c0 con 0x7f24b40d3b90 2024-04-25T10:07:33.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.417+0000 7f24a9ffb640 1 -- 172.21.15.16:0/2756836 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f24a4018020 con 0x7f24b40d3b90 2024-04-25T10:07:33.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.417+0000 7f24a9ffb640 1 --2- 172.21.15.16:0/2756836 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f249806b4d0 0x7f249806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.418+0000 7f24a9ffb640 1 -- 172.21.15.16:0/2756836 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24a408a180 con 0x7f24b40d3b90 2024-04-25T10:07:33.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.418+0000 7f24b93db640 1 --2- 172.21.15.16:0/2756836 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f249806b4d0 0x7f249806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.418+0000 7f24b93db640 1 --2- 172.21.15.16:0/2756836 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f249806b4d0 0x7f249806d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f24b0034000 tx=0x7f24b002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.418+0000 7f24be418740 1 Processor -- start 2024-04-25T10:07:33.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.419+0000 7f24be418740 1 -- start start 2024-04-25T10:07:33.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.419+0000 7f24be418740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b0f2b10 0x56045b0f5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.419+0000 7f24be418740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b0f2ef0 0x56045b0f5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.419+0000 7f24be418740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0c5770 0x56045b0c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.419+0000 7f24be418740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56045b02eda0 con 0x56045b0c5770 2024-04-25T10:07:33.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.419+0000 7f24be418740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56045afd7b80 con 0x56045b0f2ef0 2024-04-25T10:07:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.419+0000 7f24be418740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56045b02ae40 con 0x56045b0f2b10 2024-04-25T10:07:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.419+0000 7f24b9bdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0c5770 0x56045b0c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.419+0000 7f24b9bdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0c5770 0x56045b0c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47988/0 (socket says 172.21.15.16:47988) 2024-04-25T10:07:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.419+0000 7f24b9bdc640 1 -- 172.21.15.16:0/1699931738 learned_addr learned my addr 172.21.15.16:0/1699931738 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.419+0000 7f24b93db640 1 --2- 172.21.15.16:0/1699931738 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b0f2b10 0x56045b0f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.419+0000 7f24b8bda640 1 --2- 172.21.15.16:0/1699931738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b0f2ef0 0x56045b0f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.420+0000 7f24b9bdc640 1 -- 172.21.15.16:0/1699931738 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b0f2b10 msgr2=0x56045b0f5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.420+0000 7f24b9bdc640 1 --2- 172.21.15.16:0/1699931738 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b0f2b10 0x56045b0f5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.420+0000 7f24b9bdc640 1 -- 172.21.15.16:0/1699931738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b0f2ef0 msgr2=0x56045b0f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.420+0000 7f24b9bdc640 1 --2- 172.21.15.16:0/1699931738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b0f2ef0 0x56045b0f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.420+0000 7f24b9bdc640 1 -- 172.21.15.16:0/1699931738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56045afe1f10 con 0x56045b0c5770 2024-04-25T10:07:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.420+0000 7f24b9bdc640 1 --2- 172.21.15.16:0/1699931738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0c5770 0x56045b0c8f80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f24ac004670 tx=0x7f24ac00f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3b322533c77704 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.420+0000 7f249effd640 1 -- 172.21.15.16:0/1699931738 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac018020 con 0x56045b0c5770 2024-04-25T10:07:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.420+0000 7f249effd640 1 -- 172.21.15.16:0/1699931738 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ac00fe60 con 0x56045b0c5770 2024-04-25T10:07:33.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.420+0000 7f249effd640 1 -- 172.21.15.16:0/1699931738 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac014670 con 0x56045b0c5770 2024-04-25T10:07:33.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.421+0000 7f24be418740 1 -- 172.21.15.16:0/1699931738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0c5770 msgr2=0x56045b0c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.421+0000 7f24be418740 1 --2- 172.21.15.16:0/1699931738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0c5770 0x56045b0c8f80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f24ac004670 tx=0x7f24ac00f5a0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.421+0000 7f24be418740 1 -- 172.21.15.16:0/1699931738 shutdown_connections 2024-04-25T10:07:33.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.421+0000 7f24be418740 1 --2- 172.21.15.16:0/1699931738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0c5770 0x56045b0c8f80 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.421+0000 7f24be418740 1 --2- 172.21.15.16:0/1699931738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b0f2ef0 0x56045b0f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.421+0000 7f24be418740 1 --2- 172.21.15.16:0/1699931738 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b0f2b10 0x56045b0f5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.421+0000 7f24be418740 1 -- 172.21.15.16:0/1699931738 >> 172.21.15.16:0/1699931738 conn(0x56045b0c79e0 msgr2=0x56045b110320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.421+0000 7f24be418740 1 -- 172.21.15.16:0/1699931738 shutdown_connections 2024-04-25T10:07:33.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.421+0000 7f24be418740 1 -- 172.21.15.16:0/1699931738 wait complete. 2024-04-25T10:07:33.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.421+0000 7f24be418740 1 -- start start 2024-04-25T10:07:33.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.421+0000 7f24be418740 1 Processor -- start 2024-04-25T10:07:33.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.422+0000 7f24be418740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b0f2b10 0x56045b0ef6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.422+0000 7f24be418740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2ef0 0x56045b0eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.422+0000 7f24be418740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b124390 0x56045b111450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.422+0000 7f24be418740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56045b0cab90 con 0x56045b0f2ef0 2024-04-25T10:07:33.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.422+0000 7f24be418740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56045afcd4b0 con 0x56045b124390 2024-04-25T10:07:33.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.422+0000 7f24be418740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56045b0cae70 con 0x56045b0f2b10 2024-04-25T10:07:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.422+0000 7f24be418740 10 client.0 _ll_register_callbacks cb 0x56045b00d180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.422+0000 7f24b8bda640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2ef0 0x56045b0eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.422+0000 7f24b8bda640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2ef0 0x56045b0eb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47990/0 (socket says 172.21.15.16:47990) 2024-04-25T10:07:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.422+0000 7f24b8bda640 1 -- 172.21.15.16:0/192934581 learned_addr learned my addr 172.21.15.16:0/192934581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.423+0000 7f24b9bdc640 1 --2- 172.21.15.16:0/192934581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b124390 0x56045b111450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.423+0000 7f24b93db640 1 --2- 172.21.15.16:0/192934581 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b0f2b10 0x56045b0ef6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.423+0000 7f24b8bda640 1 -- 172.21.15.16:0/192934581 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b0f2b10 msgr2=0x56045b0ef6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.423+0000 7f24b8bda640 1 --2- 172.21.15.16:0/192934581 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b0f2b10 0x56045b0ef6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.423+0000 7f24b8bda640 1 -- 172.21.15.16:0/192934581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b124390 msgr2=0x56045b111450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.423+0000 7f24b8bda640 1 --2- 172.21.15.16:0/192934581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b124390 0x56045b111450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.423+0000 7f24b8bda640 1 -- 172.21.15.16:0/192934581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56045ae0d7d0 con 0x56045b0f2ef0 2024-04-25T10:07:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.423+0000 7f24b93db640 1 --2- 172.21.15.16:0/192934581 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b0f2b10 0x56045b0ef6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:33.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.423+0000 7f24b8bda640 1 --2- 172.21.15.16:0/192934581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2ef0 0x56045b0eb4b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f24a40589b0 tx=0x7f24a405cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f7b4a8e50f1a40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.423+0000 7f2493fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:07:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.423+0000 7f2493fff640 1 -- 172.21.15.16:0/192934581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a4056de0 con 0x56045b0f2ef0 2024-04-25T10:07:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.423+0000 7f2493fff640 1 -- 172.21.15.16:0/192934581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24a4058080 con 0x56045b0f2ef0 2024-04-25T10:07:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.423+0000 7f24be418740 1 -- 172.21.15.16:0/192934581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24ac00fe60 con 0x56045b0f2ef0 2024-04-25T10:07:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.423+0000 7f2493fff640 1 -- 172.21.15.16:0/192934581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24a4088300 con 0x56045b0f2ef0 2024-04-25T10:07:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.423+0000 7f24be418740 10 client.5862 fetch_fsmap learned FSMap version 21 2024-04-25T10:07:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.423+0000 7f24be418740 1 -- 172.21.15.16:0/192934581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x56045ad433f0 con 0x56045b0f2ef0 2024-04-25T10:07:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.423+0000 7f2493fff640 1 -- 172.21.15.16:0/192934581 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ac00fe60 con 0x56045b0f2ef0 2024-04-25T10:07:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.424+0000 7f2493fff640 1 -- 172.21.15.16:0/192934581 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f24a40772f0 con 0x56045b0f2ef0 2024-04-25T10:07:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.424+0000 7f24be418740 10 client.5862 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T10:07:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.424+0000 7f24be418740 10 client.5862 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-04-25T10:07:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.424+0000 7f24be418740 20 client.5862 populate_metadata read hostname 'smithi016' 2024-04-25T10:07:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.424+0000 7f24be418740 1 --2- 172.21.15.16:0/192934581 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x56045b113c50 0x56045b0edc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.424+0000 7f24be418740 4 client.5862 mds_command: new command op to 34883 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.424+0000 7f24be418740 1 -- 172.21.15.16:0/192934581 --> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56045b03c2e0 con 0x56045b113c50 2024-04-25T10:07:33.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.424+0000 7f24b93db640 1 --2- 172.21.15.16:0/192934581 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x56045b113c50 0x56045b0edc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.424+0000 7f2493fff640 10 client.5862 ms_handle_connect on v2:172.21.15.186:6835/83423489 2024-04-25T10:07:33.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.424+0000 7f24b93db640 1 --2- 172.21.15.16:0/192934581 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x56045b113c50 0x56045b0edc10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3df2d8d12d8dfd server_cookie=bf3c84e0bf805d64 in_seq=0 out_seq=0 2024-04-25T10:07:33.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.425+0000 7f2493fff640 1 -- 172.21.15.16:0/192934581 <== mds.0 v2:172.21.15.186:6835/83423489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56045b03c2e0 con 0x56045b113c50 2024-04-25T10:07:33.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.425+0000 7f2493fff640 10 client.5862 handle_command_reply: tid=0 2024-04-25T10:07:33.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.425+0000 7f24be418740 1 client.5862 shutdown 2024-04-25T10:07:33.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.425+0000 7f24be418740 1 -- 172.21.15.16:0/192934581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2ef0 msgr2=0x56045b0eb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.425+0000 7f24be418740 1 --2- 172.21.15.16:0/192934581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2ef0 0x56045b0eb4b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f24a40589b0 tx=0x7f24a405cac0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.425+0000 7f24be418740 1 -- 172.21.15.16:0/192934581 shutdown_connections 2024-04-25T10:07:33.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.425+0000 7f24be418740 1 --2- 172.21.15.16:0/192934581 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x56045b113c50 0x56045b0edc10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.425+0000 7f24be418740 1 --2- 172.21.15.16:0/192934581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b124390 0x56045b111450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.425+0000 7f2493fff640 0 client.5862 ms_handle_reset on v2:172.21.15.186:6835/83423489 2024-04-25T10:07:33.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.425+0000 7f24be418740 1 --2- 172.21.15.16:0/192934581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2ef0 0x56045b0eb4b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.425+0000 7f24be418740 1 --2- 172.21.15.16:0/192934581 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b0f2b10 0x56045b0ef6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.425+0000 7f24be418740 1 -- 172.21.15.16:0/192934581 >> 172.21.15.16:0/192934581 conn(0x56045b0c5770 msgr2=0x56045b0ed380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.425+0000 7f24be418740 1 -- 172.21.15.16:0/192934581 shutdown_connections 2024-04-25T10:07:33.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.425+0000 7f24be418740 1 -- 172.21.15.16:0/192934581 wait complete. 2024-04-25T10:07:33.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.425+0000 7f24be418740 20 client.5862 trim_cache size 0 max 16384 2024-04-25T10:07:33.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.440+0000 7f24be418740 1 Processor -- start 2024-04-25T10:07:33.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.441+0000 7f24be418740 1 -- start start 2024-04-25T10:07:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.441+0000 7f24be418740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2b10 0x56045b0c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.441+0000 7f24be418740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b0f2ef0 0x56045b110550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.441+0000 7f24be418740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b124390 0x56045b110a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.441+0000 7f24be418740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56045b0c55f0 con 0x56045b0f2b10 2024-04-25T10:07:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.441+0000 7f24be418740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b41fffd0 con 0x56045b0f2ef0 2024-04-25T10:07:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.441+0000 7f24be418740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b41ff210 con 0x56045b124390 2024-04-25T10:07:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.441+0000 7f24b93db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2b10 0x56045b0c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.441+0000 7f24b93db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2b10 0x56045b0c47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48002/0 (socket says 172.21.15.16:48002) 2024-04-25T10:07:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.441+0000 7f24b93db640 1 -- 172.21.15.16:0/3554321485 learned_addr learned my addr 172.21.15.16:0/3554321485 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f24b93db640 1 -- 172.21.15.16:0/3554321485 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b124390 msgr2=0x56045b110a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f24b93db640 1 --2- 172.21.15.16:0/3554321485 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b124390 0x56045b110a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f24b93db640 1 -- 172.21.15.16:0/3554321485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b0f2ef0 msgr2=0x56045b110550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f24b8bda640 1 --2- 172.21.15.16:0/3554321485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b0f2ef0 0x56045b110550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f24b93db640 1 --2- 172.21.15.16:0/3554321485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b0f2ef0 0x56045b110550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f24b93db640 1 -- 172.21.15.16:0/3554321485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56045ad030f0 con 0x56045b0f2b10 2024-04-25T10:07:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f24b8bda640 1 --2- 172.21.15.16:0/3554321485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b0f2ef0 0x56045b110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f24b93db640 1 --2- 172.21.15.16:0/3554321485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2b10 0x56045b0c47f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f24b003f040 tx=0x7f24b0008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6a4147f4201f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f249e7fc640 1 -- 172.21.15.16:0/3554321485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0037040 con 0x56045b0f2b10 2024-04-25T10:07:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f249e7fc640 1 -- 172.21.15.16:0/3554321485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b0058070 con 0x56045b0f2b10 2024-04-25T10:07:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f249e7fc640 1 -- 172.21.15.16:0/3554321485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b00535b0 con 0x56045b0f2b10 2024-04-25T10:07:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f24be418740 1 -- 172.21.15.16:0/3554321485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2b10 msgr2=0x56045b0c47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f24be418740 1 --2- 172.21.15.16:0/3554321485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2b10 0x56045b0c47f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f24b003f040 tx=0x7f24b0008030 comp rx=0 tx=0).stop 2024-04-25T10:07:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f24be418740 1 -- 172.21.15.16:0/3554321485 shutdown_connections 2024-04-25T10:07:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f24be418740 1 --2- 172.21.15.16:0/3554321485 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b124390 0x56045b110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f24be418740 1 --2- 172.21.15.16:0/3554321485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b0f2ef0 0x56045b110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f24be418740 1 --2- 172.21.15.16:0/3554321485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2b10 0x56045b0c47f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f24be418740 1 -- 172.21.15.16:0/3554321485 >> 172.21.15.16:0/3554321485 conn(0x56045b0c5770 msgr2=0x56045b0f07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f24be418740 1 -- 172.21.15.16:0/3554321485 shutdown_connections 2024-04-25T10:07:33.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.442+0000 7f24be418740 1 -- 172.21.15.16:0/3554321485 wait complete. 2024-04-25T10:07:33.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.443+0000 7f24be418740 1 -- start start 2024-04-25T10:07:33.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.443+0000 7f24be418740 1 Processor -- start 2024-04-25T10:07:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.444+0000 7f24be418740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2b10 0x56045b0e73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.444+0000 7f24be418740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b0f2ef0 0x56045b1230c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.444+0000 7f24be418740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b124390 0x56045b123600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.444+0000 7f24be418740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b40c94d0 con 0x56045b0f2b10 2024-04-25T10:07:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.444+0000 7f24be418740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b41f7a20 con 0x56045b124390 2024-04-25T10:07:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.444+0000 7f24be418740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56045b0eacb0 con 0x56045b0f2ef0 2024-04-25T10:07:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.444+0000 7f24be418740 10 client.0 _ll_register_callbacks cb 0x7f24ac0040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.444+0000 7f24b93db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2b10 0x56045b0e73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.444+0000 7f24b93db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2b10 0x56045b0e73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48010/0 (socket says 172.21.15.16:48010) 2024-04-25T10:07:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.444+0000 7f24b93db640 1 -- 172.21.15.16:0/745055103 learned_addr learned my addr 172.21.15.16:0/745055103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.444+0000 7f24b8bda640 1 --2- 172.21.15.16:0/745055103 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b0f2ef0 0x56045b1230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.444+0000 7f24b9bdc640 1 --2- 172.21.15.16:0/745055103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b124390 0x56045b123600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.444+0000 7f24b93db640 1 -- 172.21.15.16:0/745055103 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b0f2ef0 msgr2=0x56045b1230c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.444+0000 7f24b93db640 1 --2- 172.21.15.16:0/745055103 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b0f2ef0 0x56045b1230c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.444+0000 7f24b93db640 1 -- 172.21.15.16:0/745055103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b124390 msgr2=0x56045b123600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.444+0000 7f24b93db640 1 --2- 172.21.15.16:0/745055103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b124390 0x56045b123600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.444+0000 7f24b93db640 1 -- 172.21.15.16:0/745055103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56045b12a1e0 con 0x56045b0f2b10 2024-04-25T10:07:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.445+0000 7f24b93db640 1 --2- 172.21.15.16:0/745055103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2b10 0x56045b0e73a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f24b003f040 tx=0x7f24b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3c602804a9039f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.445+0000 7f2493fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:07:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.445+0000 7f2493fff640 1 -- 172.21.15.16:0/745055103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0060070 con 0x56045b0f2b10 2024-04-25T10:07:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.445+0000 7f2493fff640 1 -- 172.21.15.16:0/745055103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24b0004300 con 0x56045b0f2b10 2024-04-25T10:07:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.445+0000 7f24be418740 1 -- 172.21.15.16:0/745055103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24b0058070 con 0x56045b0f2b10 2024-04-25T10:07:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.445+0000 7f2493fff640 1 -- 172.21.15.16:0/745055103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24b006b050 con 0x56045b0f2b10 2024-04-25T10:07:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.445+0000 7f24be418740 10 client.5868 fetch_fsmap learned FSMap version 21 2024-04-25T10:07:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.445+0000 7f24be418740 1 -- 172.21.15.16:0/745055103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x56045b12a4e0 con 0x56045b0f2b10 2024-04-25T10:07:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.445+0000 7f2493fff640 1 -- 172.21.15.16:0/745055103 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b0058070 con 0x56045b0f2b10 2024-04-25T10:07:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.445+0000 7f2493fff640 1 -- 172.21.15.16:0/745055103 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f24b006e020 con 0x56045b0f2b10 2024-04-25T10:07:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.445+0000 7f24be418740 10 client.5868 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T10:07:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.445+0000 7f24be418740 10 client.5868 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-04-25T10:07:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.445+0000 7f24be418740 20 client.5868 populate_metadata read hostname 'smithi016' 2024-04-25T10:07:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.445+0000 7f24be418740 1 --2- 172.21.15.16:0/745055103 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x56045b0e8100 0x56045b0f8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.445+0000 7f24be418740 4 client.5868 mds_command: new command op to 34883 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:07:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.445+0000 7f24be418740 1 -- 172.21.15.16:0/745055103 --> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56045b036630 con 0x56045b0e8100 2024-04-25T10:07:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.446+0000 7f24b8bda640 1 --2- 172.21.15.16:0/745055103 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x56045b0e8100 0x56045b0f8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.446+0000 7f24b8bda640 1 --2- 172.21.15.16:0/745055103 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x56045b0e8100 0x56045b0f8930 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6ea63f850ebd826 server_cookie=89729d3561962a7f in_seq=0 out_seq=0 2024-04-25T10:07:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.446+0000 7f2493fff640 10 client.5868 ms_handle_connect on v2:172.21.15.186:6835/83423489 2024-04-25T10:07:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.446+0000 7f2493fff640 1 -- 172.21.15.16:0/745055103 <== mds.0 v2:172.21.15.186:6835/83423489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56045b036630 con 0x56045b0e8100 2024-04-25T10:07:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.446+0000 7f2493fff640 10 client.5868 handle_command_reply: tid=0 2024-04-25T10:07:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.446+0000 7f24be418740 1 client.5868 shutdown 2024-04-25T10:07:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.446+0000 7f24be418740 1 -- 172.21.15.16:0/745055103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2b10 msgr2=0x56045b0e73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.446+0000 7f24be418740 1 --2- 172.21.15.16:0/745055103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2b10 0x56045b0e73a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f24b003f040 tx=0x7f24b0004730 comp rx=0 tx=0).stop 2024-04-25T10:07:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.446+0000 7f24be418740 1 -- 172.21.15.16:0/745055103 shutdown_connections 2024-04-25T10:07:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.446+0000 7f24be418740 1 --2- 172.21.15.16:0/745055103 >> [v2:172.21.15.186:6835/83423489,v1:172.21.15.186:6837/83423489] conn(0x56045b0e8100 0x56045b0f8930 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.446+0000 7f24be418740 1 --2- 172.21.15.16:0/745055103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56045b124390 0x56045b123600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.446+0000 7f24be418740 1 --2- 172.21.15.16:0/745055103 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56045b0f2ef0 0x56045b1230c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.446+0000 7f24be418740 1 --2- 172.21.15.16:0/745055103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56045b0f2b10 0x56045b0e73a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.446+0000 7f24be418740 1 -- 172.21.15.16:0/745055103 >> 172.21.15.16:0/745055103 conn(0x56045b0c5770 msgr2=0x56045b0f52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.446+0000 7f2493fff640 0 client.5868 ms_handle_reset on v2:172.21.15.186:6835/83423489 2024-04-25T10:07:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.446+0000 7f24be418740 1 -- 172.21.15.16:0/745055103 shutdown_connections 2024-04-25T10:07:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.446+0000 7f24be418740 1 -- 172.21.15.16:0/745055103 wait complete. 2024-04-25T10:07:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.447+0000 7f24be418740 20 client.5868 trim_cache size 0 max 16384 2024-04-25T10:07:33.459 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:07:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-04-25T10:07:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-04-25T10:07:33.459 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:07:33.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.447+0000 7f24bb63f640 1 -- 172.21.15.16:0/2756836 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f249806b4d0 msgr2=0x7f249806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.447+0000 7f24bb63f640 1 --2- 172.21.15.16:0/2756836 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f249806b4d0 0x7f249806d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f24b0034000 tx=0x7f24b002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:33.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.447+0000 7f24bb63f640 1 -- 172.21.15.16:0/2756836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24b40d3b90 msgr2=0x7f24b40c6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.447+0000 7f24bb63f640 1 --2- 172.21.15.16:0/2756836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24b40d3b90 0x7f24b40c6c70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f24a400b450 tx=0x7f24a400b910 comp rx=0 tx=0).stop 2024-04-25T10:07:33.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.447+0000 7f24bb63f640 1 -- 172.21.15.16:0/2756836 shutdown_connections 2024-04-25T10:07:33.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.447+0000 7f24bb63f640 1 --2- 172.21.15.16:0/2756836 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f249806b4d0 0x7f249806d990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.447+0000 7f24bb63f640 1 --2- 172.21.15.16:0/2756836 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24b415dca0 0x7f24b4160430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.447+0000 7f24bb63f640 1 --2- 172.21.15.16:0/2756836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24b40d3b90 0x7f24b40c6c70 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.447+0000 7f24bb63f640 1 --2- 172.21.15.16:0/2756836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24b40cee80 0x7f24b40c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.447+0000 7f24bb63f640 1 -- 172.21.15.16:0/2756836 >> 172.21.15.16:0/2756836 conn(0x7f24b40726c0 msgr2=0x7f24b40d2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.447+0000 7f24bb63f640 1 -- 172.21.15.16:0/2756836 shutdown_connections 2024-04-25T10:07:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.447+0000 7f24bb63f640 1 -- 172.21.15.16:0/2756836 wait complete. 2024-04-25T10:07:33.463 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:07:33.565 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:33.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.661+0000 7fae334ac640 1 Processor -- start 2024-04-25T10:07:33.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.662+0000 7fae334ac640 1 -- start start 2024-04-25T10:07:33.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.662+0000 7fae334ac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fae2c06b2b0 0x7fae2c0ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.662+0000 7fae334ac640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2c161bf0 con 0x7fae2c160f00 2024-04-25T10:07:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.662+0000 7fae334ac640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2c161ed0 con 0x7fae2c1616f0 2024-04-25T10:07:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.662+0000 7fae334ac640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae2c06fab0 con 0x7fae2c06b2b0 2024-04-25T10:07:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.662+0000 7fae31a49640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fae2c06b2b0 0x7fae2c0ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.662+0000 7fae31248640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fae2c160f00 0x7fae2c072a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41792/0 (socket says 172.21.15.16:41792) 2024-04-25T10:07:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.663+0000 7fae31248640 1 -- 172.21.15.16:0/4027461659 learned_addr learned my addr 172.21.15.16:0/4027461659 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.663+0000 7fae31a49640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fae2c06b2b0 0x7fae2c0ce640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48020/0 (socket says 172.21.15.16:48020) 2024-04-25T10:07:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.663+0000 7fae31a49640 1 -- 172.21.15.16:0/4027461659 >> v1:172.21.15.186:6789/0 conn(0x7fae2c1616f0 legacy=0x7fae2c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.663+0000 7fae31a49640 1 -- 172.21.15.16:0/4027461659 >> v1:172.21.15.16:6789/0 conn(0x7fae2c160f00 legacy=0x7fae2c072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.663+0000 7fae31a49640 1 -- 172.21.15.16:0/4027461659 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c1621b0 con 0x7fae2c06b2b0 2024-04-25T10:07:33.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.663+0000 7fae31a49640 1 --2- 172.21.15.16:0/4027461659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae2c06b2b0 0x7fae2c0ce640 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fae240097a0 tx=0x7fae2402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91de3858e148449 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.664+0000 7fae2bfff640 1 -- 172.21.15.16:0/4027461659 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24037070 con 0x7fae2c06b2b0 2024-04-25T10:07:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.664+0000 7fae2bfff640 1 -- 172.21.15.16:0/4027461659 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae24032440 con 0x7fae2c06b2b0 2024-04-25T10:07:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.664+0000 7fae2bfff640 1 -- 172.21.15.16:0/4027461659 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24032810 con 0x7fae2c06b2b0 2024-04-25T10:07:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.664+0000 7fae334ac640 1 -- 172.21.15.16:0/4027461659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae2c06b2b0 msgr2=0x7fae2c0ce640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.664+0000 7fae334ac640 1 --2- 172.21.15.16:0/4027461659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae2c06b2b0 0x7fae2c0ce640 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fae240097a0 tx=0x7fae2402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.664+0000 7fae334ac640 1 -- 172.21.15.16:0/4027461659 shutdown_connections 2024-04-25T10:07:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.664+0000 7fae334ac640 1 --2- 172.21.15.16:0/4027461659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae2c06b2b0 0x7fae2c0ce640 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.664+0000 7fae334ac640 1 -- 172.21.15.16:0/4027461659 >> 172.21.15.16:0/4027461659 conn(0x7fae2c0b2370 msgr2=0x7fae2c0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.664+0000 7fae334ac640 1 -- 172.21.15.16:0/4027461659 shutdown_connections 2024-04-25T10:07:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.664+0000 7fae334ac640 1 -- 172.21.15.16:0/4027461659 wait complete. 2024-04-25T10:07:33.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.665+0000 7fae334ac640 1 Processor -- start 2024-04-25T10:07:33.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.665+0000 7fae334ac640 1 -- start start 2024-04-25T10:07:33.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.666+0000 7fae334ac640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae2c06b2b0 0x7fae2c0b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.666+0000 7fae334ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae2c160f00 0x7fae2c155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.666+0000 7fae334ac640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae2c1616f0 0x7fae2c0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.666+0000 7fae334ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c161460 con 0x7fae2c160f00 2024-04-25T10:07:33.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.666+0000 7fae334ac640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c1612e0 con 0x7fae2c1616f0 2024-04-25T10:07:33.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.666+0000 7fae334ac640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c06ff40 con 0x7fae2c06b2b0 2024-04-25T10:07:33.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.666+0000 7fae30a47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae2c160f00 0x7fae2c155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.666+0000 7fae30a47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae2c160f00 0x7fae2c155180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48032/0 (socket says 172.21.15.16:48032) 2024-04-25T10:07:33.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.666+0000 7fae30a47640 1 -- 172.21.15.16:0/402869633 learned_addr learned my addr 172.21.15.16:0/402869633 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:33.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.666+0000 7fae31a49640 1 --2- 172.21.15.16:0/402869633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae2c1616f0 0x7fae2c0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.666+0000 7fae31248640 1 --2- 172.21.15.16:0/402869633 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae2c06b2b0 0x7fae2c0b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.667+0000 7fae30a47640 1 -- 172.21.15.16:0/402869633 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae2c06b2b0 msgr2=0x7fae2c0b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.667+0000 7fae30a47640 1 --2- 172.21.15.16:0/402869633 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae2c06b2b0 0x7fae2c0b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.667+0000 7fae30a47640 1 -- 172.21.15.16:0/402869633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae2c1616f0 msgr2=0x7fae2c0b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.667+0000 7fae30a47640 1 --2- 172.21.15.16:0/402869633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae2c1616f0 0x7fae2c0b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.667+0000 7fae30a47640 1 -- 172.21.15.16:0/402869633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c0b3440 con 0x7fae2c160f00 2024-04-25T10:07:33.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.667+0000 7fae30a47640 1 --2- 172.21.15.16:0/402869633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae2c160f00 0x7fae2c155180 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fae2000b3f0 tx=0x7fae2000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.667+0000 7fae29ffb640 1 -- 172.21.15.16:0/402869633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20015020 con 0x7fae2c160f00 2024-04-25T10:07:33.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.667+0000 7fae29ffb640 1 -- 172.21.15.16:0/402869633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae20011830 con 0x7fae2c160f00 2024-04-25T10:07:33.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.667+0000 7fae334ac640 1 -- 172.21.15.16:0/402869633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae2c0b3720 con 0x7fae2c160f00 2024-04-25T10:07:33.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.667+0000 7fae29ffb640 1 -- 172.21.15.16:0/402869633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20011c00 con 0x7fae2c160f00 2024-04-25T10:07:33.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.667+0000 7fae334ac640 1 -- 172.21.15.16:0/402869633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae2c0b3fc0 con 0x7fae2c160f00 2024-04-25T10:07:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.668+0000 7fae334ac640 1 -- 172.21.15.16:0/402869633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae2c0749b0 con 0x7fae2c160f00 2024-04-25T10:07:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.669+0000 7fae29ffb640 1 -- 172.21.15.16:0/402869633 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae20018020 con 0x7fae2c160f00 2024-04-25T10:07:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.669+0000 7fae29ffb640 1 --2- 172.21.15.16:0/402869633 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae1806b4d0 0x7fae1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.669+0000 7fae29ffb640 1 -- 172.21.15.16:0/402869633 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae2008a000 con 0x7fae2c160f00 2024-04-25T10:07:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.669+0000 7fae31248640 1 --2- 172.21.15.16:0/402869633 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae1806b4d0 0x7fae1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.670+0000 7fae31248640 1 --2- 172.21.15.16:0/402869633 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae1806b4d0 0x7fae1806d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0cce40 tx=0x7fae1c001a30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.673+0000 7fae29ffb640 1 -- 172.21.15.16:0/402869633 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae2001f020 con 0x7fae2c160f00 2024-04-25T10:07:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.891+0000 7fae334ac640 1 -- 172.21.15.16:0/402869633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae2c0b6950 con 0x7fae2c160f00 2024-04-25T10:07:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.891+0000 7fae29ffb640 1 -- 172.21.15.16:0/402869633 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fae20058830 con 0x7fae2c160f00 2024-04-25T10:07:33.897 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:33.897 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/791079479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":791079479},{"type":"v1","addr":"172.21.15.16:6835","nonce":791079479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1747921236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1747921236},{"type":"v1","addr":"172.21.15.16:6834","nonce":1747921236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":34880,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/3183779257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3183779257},{"type":"v1","addr":"172.21.15.186:6836","nonce":3183779257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:51.182691+0000","modified":"2024-04-25T10:06:53.179194+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34883},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34883":{"gid":34883,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/83423489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":83423489},{"type":"v1","addr":"172.21.15.186:6837","nonce":83423489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:33.898 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 21 2024-04-25T10:07:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.893+0000 7fae334ac640 1 -- 172.21.15.16:0/402869633 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae1806b4d0 msgr2=0x7fae1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.893+0000 7fae334ac640 1 --2- 172.21.15.16:0/402869633 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae1806b4d0 0x7fae1806d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0cce40 tx=0x7fae1c001a30 comp rx=0 tx=0).stop 2024-04-25T10:07:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.893+0000 7fae334ac640 1 -- 172.21.15.16:0/402869633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae2c160f00 msgr2=0x7fae2c155180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.893+0000 7fae334ac640 1 --2- 172.21.15.16:0/402869633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae2c160f00 0x7fae2c155180 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fae2000b3f0 tx=0x7fae2000b8b0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.893+0000 7fae334ac640 1 -- 172.21.15.16:0/402869633 shutdown_connections 2024-04-25T10:07:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.893+0000 7fae334ac640 1 --2- 172.21.15.16:0/402869633 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae1806b4d0 0x7fae1806d990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.893+0000 7fae334ac640 1 --2- 172.21.15.16:0/402869633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae2c1616f0 0x7fae2c0b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.893+0000 7fae334ac640 1 --2- 172.21.15.16:0/402869633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae2c160f00 0x7fae2c155180 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.893+0000 7fae334ac640 1 --2- 172.21.15.16:0/402869633 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae2c06b2b0 0x7fae2c0b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.893+0000 7fae334ac640 1 -- 172.21.15.16:0/402869633 >> 172.21.15.16:0/402869633 conn(0x7fae2c0b2370 msgr2=0x7fae2c0cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.893+0000 7fae334ac640 1 -- 172.21.15.16:0/402869633 shutdown_connections 2024-04-25T10:07:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:33.893+0000 7fae334ac640 1 -- 172.21.15.16:0/402869633 wait complete. 2024-04-25T10:07:33.910 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-25T10:07:34.040 INFO:teuthology.orchestra.run.smithi186.stdout:{"message":"","return_code":0} 2024-04-25T10:07:34.040 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:07:34.040 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:07:34.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.078+0000 7fae8a71ef40 1 Processor -- start 2024-04-25T10:07:34.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.078+0000 7fae8a71ef40 1 -- start start 2024-04-25T10:07:34.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.078+0000 7fae8a71ef40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6d4c1a960 0x55c6d4c1b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.078+0000 7fae8a71ef40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c6d4c4cb00 0x55c6d4cf1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.078+0000 7fae8a71ef40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6d4c4cee0 0x55c6d4c5d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.078+0000 7fae8a71ef40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6d4cf2540 con 0x55c6d4c4cee0 2024-04-25T10:07:34.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.078+0000 7fae8a71ef40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6d4bc3df0 con 0x55c6d4c1a960 2024-04-25T10:07:34.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.078+0000 7fae8a71ef40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6d4c64320 con 0x55c6d4c4cb00 2024-04-25T10:07:34.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.079+0000 7fae87f52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6d4c4cee0 0x55c6d4c5d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.079+0000 7fae87f52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6d4c4cee0 0x55c6d4c5d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48042/0 (socket says 172.21.15.16:48042) 2024-04-25T10:07:34.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.079+0000 7fae87f52640 1 -- 172.21.15.16:0/2231469921 learned_addr learned my addr 172.21.15.16:0/2231469921 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:34.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.079+0000 7fae87f52640 1 -- 172.21.15.16:0/2231469921 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c6d4c4cb00 msgr2=0x55c6d4cf1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:34.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.079+0000 7fae87f52640 1 --2- 172.21.15.16:0/2231469921 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c6d4c4cb00 0x55c6d4cf1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.079+0000 7fae87f52640 1 -- 172.21.15.16:0/2231469921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6d4c1a960 msgr2=0x55c6d4c1b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:34.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.079+0000 7fae87f52640 1 --2- 172.21.15.16:0/2231469921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6d4c1a960 0x55c6d4c1b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.079+0000 7fae87f52640 1 -- 172.21.15.16:0/2231469921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6d4b49c50 con 0x55c6d4c4cee0 2024-04-25T10:07:34.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.079+0000 7fae87f52640 1 --2- 172.21.15.16:0/2231469921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6d4c4cee0 0x55c6d4c5d730 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fae780016b0 tx=0x7fae78001b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7c312967124ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.079+0000 7fae85f4e640 1 -- 172.21.15.16:0/2231469921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae780062e0 con 0x55c6d4c4cee0 2024-04-25T10:07:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.079+0000 7fae85f4e640 1 -- 172.21.15.16:0/2231469921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae78003650 con 0x55c6d4c4cee0 2024-04-25T10:07:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.079+0000 7fae85f4e640 1 -- 172.21.15.16:0/2231469921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae78003a60 con 0x55c6d4c4cee0 2024-04-25T10:07:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.079+0000 7fae8a71ef40 1 -- 172.21.15.16:0/2231469921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6d4c4cee0 msgr2=0x55c6d4c5d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.079+0000 7fae8a71ef40 1 --2- 172.21.15.16:0/2231469921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6d4c4cee0 0x55c6d4c5d730 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fae780016b0 tx=0x7fae78001b70 comp rx=0 tx=0).stop 2024-04-25T10:07:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.080+0000 7fae8a71ef40 1 -- 172.21.15.16:0/2231469921 shutdown_connections 2024-04-25T10:07:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.080+0000 7fae8a71ef40 1 --2- 172.21.15.16:0/2231469921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6d4c4cee0 0x55c6d4c5d730 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.080+0000 7fae8a71ef40 1 --2- 172.21.15.16:0/2231469921 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c6d4c4cb00 0x55c6d4cf1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.080+0000 7fae8a71ef40 1 --2- 172.21.15.16:0/2231469921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6d4c1a960 0x55c6d4c1b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.080+0000 7fae8a71ef40 1 -- 172.21.15.16:0/2231469921 >> 172.21.15.16:0/2231469921 conn(0x55c6d4c473d0 msgr2=0x55c6d4c46910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.080+0000 7fae8a71ef40 1 -- 172.21.15.16:0/2231469921 shutdown_connections 2024-04-25T10:07:34.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.080+0000 7fae8a71ef40 1 -- 172.21.15.16:0/2231469921 wait complete. 2024-04-25T10:07:34.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.080+0000 7fae8a71ef40 1 Processor -- start 2024-04-25T10:07:34.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.080+0000 7fae8a71ef40 1 -- start start 2024-04-25T10:07:34.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.081+0000 7fae8a71ef40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6d4c1a960 0x55c6d4cdcb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.081+0000 7fae8a71ef40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6d4c4cb00 0x55c6d4cdb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.081+0000 7fae8a71ef40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c6d4c4cee0 0x55c6d4cdbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.081+0000 7fae8a71ef40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6d4c62a70 con 0x55c6d4c4cb00 2024-04-25T10:07:34.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.081+0000 7fae8a71ef40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6d4cef930 con 0x55c6d4c1a960 2024-04-25T10:07:34.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.081+0000 7fae8a71ef40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6d4c62d50 con 0x55c6d4c4cee0 2024-04-25T10:07:34.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.081+0000 7fae86f50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6d4c4cb00 0x55c6d4cdb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.081+0000 7fae86f50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6d4c4cb00 0x55c6d4cdb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48046/0 (socket says 172.21.15.16:48046) 2024-04-25T10:07:34.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.081+0000 7fae86f50640 1 -- 172.21.15.16:0/2773798348 learned_addr learned my addr 172.21.15.16:0/2773798348 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:34.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.081+0000 7fae87f52640 1 --2- 172.21.15.16:0/2773798348 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c6d4c4cee0 0x55c6d4cdbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.081+0000 7fae87751640 1 --2- 172.21.15.16:0/2773798348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6d4c1a960 0x55c6d4cdcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.081+0000 7fae86f50640 1 -- 172.21.15.16:0/2773798348 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c6d4c4cee0 msgr2=0x55c6d4cdbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.081+0000 7fae86f50640 1 --2- 172.21.15.16:0/2773798348 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c6d4c4cee0 0x55c6d4cdbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.081+0000 7fae86f50640 1 -- 172.21.15.16:0/2773798348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6d4c1a960 msgr2=0x55c6d4cdcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.081+0000 7fae86f50640 1 --2- 172.21.15.16:0/2773798348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c6d4c1a960 0x55c6d4cdcb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.081+0000 7fae86f50640 1 -- 172.21.15.16:0/2773798348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6d4ce46a0 con 0x55c6d4c4cb00 2024-04-25T10:07:34.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.082+0000 7fae86f50640 1 --2- 172.21.15.16:0/2773798348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6d4c4cb00 0x55c6d4cdb680 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00af20 tx=0x7fae7c0087a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:34.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.082+0000 7fae84f4c640 1 -- 172.21.15.16:0/2773798348 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c00b810 con 0x55c6d4c4cb00 2024-04-25T10:07:34.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.082+0000 7fae84f4c640 1 -- 172.21.15.16:0/2773798348 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7c00f070 con 0x55c6d4c4cb00 2024-04-25T10:07:34.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.082+0000 7fae8a71ef40 1 -- 172.21.15.16:0/2773798348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c6d4ce4980 con 0x55c6d4c4cb00 2024-04-25T10:07:34.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.082+0000 7fae84f4c640 1 -- 172.21.15.16:0/2773798348 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c00bbf0 con 0x55c6d4c4cb00 2024-04-25T10:07:34.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.082+0000 7fae8a71ef40 1 -- 172.21.15.16:0/2773798348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6d4ce4f40 con 0x55c6d4c4cb00 2024-04-25T10:07:34.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.083+0000 7fae84f4c640 1 -- 172.21.15.16:0/2773798348 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae7c014470 con 0x55c6d4c4cb00 2024-04-25T10:07:34.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.083+0000 7fae84f4c640 1 --2- 172.21.15.16:0/2773798348 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae74069eb0 0x7fae7406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.084+0000 7fae84f4c640 1 -- 172.21.15.16:0/2773798348 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae7c088cb0 con 0x55c6d4c4cb00 2024-04-25T10:07:34.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.084+0000 7fae8a71ef40 1 --2- 172.21.15.16:0/2773798348 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55c6d4cdd060 0x55c6d4cdfce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.084+0000 7fae8a71ef40 1 -- 172.21.15.16:0/2773798348 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v9 -- 0x55c6d4cf1420 con 0x55c6d4cdd060 2024-04-25T10:07:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.084+0000 7fae87751640 1 --2- 172.21.15.16:0/2773798348 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae74069eb0 0x7fae7406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.084+0000 7fae87f52640 1 --2- 172.21.15.16:0/2773798348 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55c6d4cdd060 0x55c6d4cdfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.084+0000 7fae87f52640 1 --2- 172.21.15.16:0/2773798348 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55c6d4cdd060 0x55c6d4cdfce0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.084+0000 7fae87751640 1 --2- 172.21.15.16:0/2773798348 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae74069eb0 0x7fae7406c370 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55c6d4c490f0 tx=0x7fae8004e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.084+0000 7fae87f52640 1 -- 172.21.15.16:0/2773798348 <== osd.3 v2:172.21.15.16:6824/256577261 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7fae7800f030 con 0x55c6d4cdd060 2024-04-25T10:07:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.085+0000 7fae8a71ef40 1 -- 172.21.15.16:0/2773798348 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55c6d4cdd060 msgr2=0x55c6d4cdfce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.085+0000 7fae8a71ef40 1 --2- 172.21.15.16:0/2773798348 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55c6d4cdd060 0x55c6d4cdfce0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.085+0000 7fae8a71ef40 1 -- 172.21.15.16:0/2773798348 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae74069eb0 msgr2=0x7fae7406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.085+0000 7fae8a71ef40 1 --2- 172.21.15.16:0/2773798348 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae74069eb0 0x7fae7406c370 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55c6d4c490f0 tx=0x7fae8004e000 comp rx=0 tx=0).stop 2024-04-25T10:07:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.085+0000 7fae8a71ef40 1 -- 172.21.15.16:0/2773798348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6d4c4cb00 msgr2=0x55c6d4cdb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.085+0000 7fae8a71ef40 1 --2- 172.21.15.16:0/2773798348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6d4c4cb00 0x55c6d4cdb680 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00af20 tx=0x7fae7c0087a0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.085+0000 7fae87f52640 1 -- 172.21.15.16:0/2773798348 reap_dead start 2024-04-25T10:07:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.085+0000 7fae8a71ef40 1 -- 172.21.15.16:0/2773798348 shutdown_connections 2024-04-25T10:07:34.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.085+0000 7fae8a71ef40 1 -- 172.21.15.16:0/2773798348 >> 172.21.15.16:0/2773798348 conn(0x55c6d4c473d0 msgr2=0x55c6d4cf1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:34.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.085+0000 7fae8a71ef40 1 -- 172.21.15.16:0/2773798348 shutdown_connections 2024-04-25T10:07:34.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.085+0000 7fae8a71ef40 1 -- 172.21.15.16:0/2773798348 wait complete. 2024-04-25T10:07:34.098 INFO:teuthology.orchestra.run.smithi016.stdout:Yalphaparent_a 2024-04-25T10:07:34.099 DEBUG:teuthology.orchestra.run.smithi016:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:07:34.185 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:07:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511627777, 2024-04-25T10:07:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "ancestors": [ 2024-04-25T10:07:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "dirino": 1099511627776, 2024-04-25T10:07:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "dname": "alpha", 2024-04-25T10:07:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 2 2024-04-25T10:07:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "dirino": 1, 2024-04-25T10:07:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: "dname": "parent_a", 2024-04-25T10:07:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 4 2024-04-25T10:07:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-25T10:07:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: "pool": 7, 2024-04-25T10:07:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: "old_pools": [] 2024-04-25T10:07:34.187 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:07:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:34.189 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:07:34.189 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:34.268 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:34.305 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:07:34.306 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:07:34.306 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:34.324 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:07:34.324 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:34.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:34.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.515+0000 7f45d8807640 1 Processor -- start 2024-04-25T10:07:34.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.515+0000 7f45d8807640 1 -- start start 2024-04-25T10:07:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.516+0000 7f45d8807640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f45d415bf00 0x7f45d40cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.516+0000 7f45d8807640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d40ce6c0 con 0x7f45d406e8f0 2024-04-25T10:07:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.516+0000 7f45d8807640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d40cc7b0 con 0x7f45d40cc310 2024-04-25T10:07:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.516+0000 7f45d8807640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d41455e0 con 0x7f45d415bf00 2024-04-25T10:07:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.516+0000 7f45d2d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f45d406e8f0 0x7f45d41654b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41814/0 (socket says 172.21.15.16:41814) 2024-04-25T10:07:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.516+0000 7f45d2d9d640 1 -- 172.21.15.16:0/1519798450 learned_addr learned my addr 172.21.15.16:0/1519798450 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.516+0000 7f45d1d9b640 1 -- 172.21.15.16:0/1519798450 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402770252 0 0) 0x7f45cc001490 con 0x7f45d406e8f0 2024-04-25T10:07:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.516+0000 7f45d259c640 1 --2- 172.21.15.16:0/1519798450 >> v2:172.21.15.186:3300/0 conn(0x7f45d415bf00 0x7f45d40cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.516+0000 7f45d1d9b640 1 -- 172.21.15.16:0/1519798450 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45c0003520 con 0x7f45d406e8f0 2024-04-25T10:07:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.516+0000 7f45d1d9b640 1 -- 172.21.15.16:0/1519798450 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2195013406 0 0) 0x7f45c0003520 con 0x7f45d406e8f0 2024-04-25T10:07:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.516+0000 7f45d1d9b640 1 -- 172.21.15.16:0/1519798450 >> v2:172.21.15.186:3300/0 conn(0x7f45d415bf00 msgr2=0x7f45d40cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.516+0000 7f45d1d9b640 1 --2- 172.21.15.16:0/1519798450 >> v2:172.21.15.186:3300/0 conn(0x7f45d415bf00 0x7f45d40cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.516+0000 7f45d1d9b640 1 -- 172.21.15.16:0/1519798450 >> v1:172.21.15.186:6789/0 conn(0x7f45d40cc310 legacy=0x7f45d415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45d1d9b640 1 -- 172.21.15.16:0/1519798450 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d40cca90 con 0x7f45d406e8f0 2024-04-25T10:07:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45d1d9b640 1 -- 172.21.15.16:0/1519798450 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f45cc001bc0 con 0x7f45d406e8f0 2024-04-25T10:07:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45d1d9b640 1 -- 172.21.15.16:0/1519798450 >> v1:172.21.15.16:6789/0 conn(0x7f45d406e8f0 legacy=0x7f45d41654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45d1d9b640 1 --2- 172.21.15.16:0/1519798450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c00039b0 0x7f45c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45d1d9b640 1 --2- 172.21.15.16:0/1519798450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c00043b0 0x7f45c0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45d1d9b640 1 --2- 172.21.15.16:0/1519798450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45c0005080 0x7f45c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45d1d9b640 1 -- 172.21.15.16:0/1519798450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d41455e0 con 0x7f45c00039b0 2024-04-25T10:07:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45d1d9b640 1 -- 172.21.15.16:0/1519798450 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c000bfe0 con 0x7f45c00043b0 2024-04-25T10:07:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45d1d9b640 1 -- 172.21.15.16:0/1519798450 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c000c1e0 con 0x7f45c0005080 2024-04-25T10:07:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45d259c640 1 --2- 172.21.15.16:0/1519798450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c00039b0 0x7f45c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45cbfff640 1 --2- 172.21.15.16:0/1519798450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c00043b0 0x7f45c0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45d2d9d640 1 --2- 172.21.15.16:0/1519798450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45c0005080 0x7f45c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45d259c640 1 -- 172.21.15.16:0/1519798450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45c0005080 msgr2=0x7f45c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45d259c640 1 --2- 172.21.15.16:0/1519798450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45c0005080 0x7f45c000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45d259c640 1 -- 172.21.15.16:0/1519798450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c00043b0 msgr2=0x7f45c0004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45d259c640 1 --2- 172.21.15.16:0/1519798450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c00043b0 0x7f45c0004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45d259c640 1 -- 172.21.15.16:0/1519798450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c0003520 con 0x7f45c00039b0 2024-04-25T10:07:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45d2d9d640 1 --2- 172.21.15.16:0/1519798450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45c0005080 0x7f45c000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.517+0000 7f45d2d9d640 1 -- 172.21.15.16:0/1519798450 reap_dead start 2024-04-25T10:07:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.518+0000 7f45d259c640 1 --2- 172.21.15.16:0/1519798450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c00039b0 0x7f45c0003d90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009ac0 tx=0x7f45c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45914ee3778dea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.518+0000 7f45d1d9b640 1 -- 172.21.15.16:0/1519798450 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c4036070 con 0x7f45c00039b0 2024-04-25T10:07:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.518+0000 7f45d1d9b640 1 -- 172.21.15.16:0/1519798450 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c4031400 con 0x7f45c00039b0 2024-04-25T10:07:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.518+0000 7f45d1d9b640 1 -- 172.21.15.16:0/1519798450 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c4009690 con 0x7f45c00039b0 2024-04-25T10:07:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.518+0000 7f45d8807640 1 -- 172.21.15.16:0/1519798450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c00039b0 msgr2=0x7f45c0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.518+0000 7f45d8807640 1 --2- 172.21.15.16:0/1519798450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c00039b0 0x7f45c0003d90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009ac0 tx=0x7f45c0005830 comp rx=0 tx=0).stop 2024-04-25T10:07:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.519+0000 7f45d8807640 1 -- 172.21.15.16:0/1519798450 shutdown_connections 2024-04-25T10:07:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.519+0000 7f45d8807640 1 --2- 172.21.15.16:0/1519798450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c00039b0 0x7f45c0003d90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.519+0000 7f45d8807640 1 -- 172.21.15.16:0/1519798450 >> 172.21.15.16:0/1519798450 conn(0x7f45d406aba0 msgr2=0x7f45d406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.519+0000 7f45d8807640 1 -- 172.21.15.16:0/1519798450 shutdown_connections 2024-04-25T10:07:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.519+0000 7f45d8807640 1 -- 172.21.15.16:0/1519798450 wait complete. 2024-04-25T10:07:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.520+0000 7f45d8807640 1 Processor -- start 2024-04-25T10:07:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.520+0000 7f45d8807640 1 -- start start 2024-04-25T10:07:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45d8807640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c00039b0 0x7f45d40b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45d8807640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d413e140 0x7f45d40cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45d8807640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45d40c9300 0x7f45d41434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45d8807640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d40cd050 con 0x7f45d413e140 2024-04-25T10:07:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45d8807640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d406d450 con 0x7f45c00039b0 2024-04-25T10:07:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45d8807640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d406ee60 con 0x7f45d40c9300 2024-04-25T10:07:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d413e140 0x7f45d40cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d413e140 0x7f45d40cabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48060/0 (socket says 172.21.15.16:48060) 2024-04-25T10:07:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45cbfff640 1 -- 172.21.15.16:0/3663039068 learned_addr learned my addr 172.21.15.16:0/3663039068 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:34.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45d259c640 1 --2- 172.21.15.16:0/3663039068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c00039b0 0x7f45d40b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45d2d9d640 1 --2- 172.21.15.16:0/3663039068 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45d40c9300 0x7f45d41434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45cbfff640 1 -- 172.21.15.16:0/3663039068 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45d40c9300 msgr2=0x7f45d41434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45cbfff640 1 --2- 172.21.15.16:0/3663039068 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45d40c9300 0x7f45d41434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45cbfff640 1 -- 172.21.15.16:0/3663039068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c00039b0 msgr2=0x7f45d40b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45cbfff640 1 --2- 172.21.15.16:0/3663039068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c00039b0 0x7f45d40b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45cbfff640 1 -- 172.21.15.16:0/3663039068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d413e5c0 con 0x7f45d413e140 2024-04-25T10:07:34.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45d259c640 1 --2- 172.21.15.16:0/3663039068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c00039b0 0x7f45d40b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:34.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45d2d9d640 1 --2- 172.21.15.16:0/3663039068 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45d40c9300 0x7f45d41434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:34.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45cbfff640 1 --2- 172.21.15.16:0/3663039068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d413e140 0x7f45d40cabf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00f7a0 tx=0x7f45bc00fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:34.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45caffd640 1 -- 172.21.15.16:0/3663039068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc0052e0 con 0x7f45d413e140 2024-04-25T10:07:34.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45caffd640 1 -- 172.21.15.16:0/3663039068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45bc018070 con 0x7f45d413e140 2024-04-25T10:07:34.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45d8807640 1 -- 172.21.15.16:0/3663039068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d40c9b50 con 0x7f45d413e140 2024-04-25T10:07:34.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45caffd640 1 -- 172.21.15.16:0/3663039068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc00d970 con 0x7f45d413e140 2024-04-25T10:07:34.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.521+0000 7f45d8807640 1 -- 172.21.15.16:0/3663039068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d40ca110 con 0x7f45d413e140 2024-04-25T10:07:34.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.522+0000 7f45d8807640 1 -- 172.21.15.16:0/3663039068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d406bb30 con 0x7f45d413e140 2024-04-25T10:07:34.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.523+0000 7f45caffd640 1 -- 172.21.15.16:0/3663039068 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f45bc00db10 con 0x7f45d413e140 2024-04-25T10:07:34.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.523+0000 7f45caffd640 1 --2- 172.21.15.16:0/3663039068 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f45b406b500 0x7f45b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.524+0000 7f45caffd640 1 -- 172.21.15.16:0/3663039068 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45bc08a090 con 0x7f45d413e140 2024-04-25T10:07:34.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.524+0000 7f45d259c640 1 --2- 172.21.15.16:0/3663039068 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f45b406b500 0x7f45b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.524+0000 7f45d259c640 1 --2- 172.21.15.16:0/3663039068 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f45b406b500 0x7f45b406d9c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f45c4033000 tx=0x7f45c4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:34.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.528+0000 7f45caffd640 1 -- 172.21.15.16:0/3663039068 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f45bc01a310 con 0x7f45d413e140 2024-04-25T10:07:34.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.709+0000 7f45d8807640 1 -- 172.21.15.16:0/3663039068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45d415d7d0 con 0x7f45d413e140 2024-04-25T10:07:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.710+0000 7f45caffd640 1 -- 172.21.15.16:0/3663039068 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f45bc058fa0 con 0x7f45d413e140 2024-04-25T10:07:34.716 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:34.717 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":47,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:06:52.824898+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:49.462347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:50.153232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.713+0000 7f45d8807640 1 -- 172.21.15.16:0/3663039068 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f45b406b500 msgr2=0x7f45b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.713+0000 7f45d8807640 1 --2- 172.21.15.16:0/3663039068 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f45b406b500 0x7f45b406d9c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f45c4033000 tx=0x7f45c4008030 comp rx=0 tx=0).stop 2024-04-25T10:07:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.713+0000 7f45d8807640 1 -- 172.21.15.16:0/3663039068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d413e140 msgr2=0x7f45d40cabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.713+0000 7f45d8807640 1 --2- 172.21.15.16:0/3663039068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d413e140 0x7f45d40cabf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00f7a0 tx=0x7f45bc00fc60 comp rx=0 tx=0).stop 2024-04-25T10:07:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.713+0000 7f45d8807640 1 -- 172.21.15.16:0/3663039068 shutdown_connections 2024-04-25T10:07:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.713+0000 7f45d8807640 1 --2- 172.21.15.16:0/3663039068 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f45b406b500 0x7f45b406d9c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.713+0000 7f45d8807640 1 --2- 172.21.15.16:0/3663039068 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45d40c9300 0x7f45d41434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.713+0000 7f45d8807640 1 --2- 172.21.15.16:0/3663039068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d413e140 0x7f45d40cabf0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.713+0000 7f45d8807640 1 --2- 172.21.15.16:0/3663039068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45c00039b0 0x7f45d40b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.713+0000 7f45d8807640 1 -- 172.21.15.16:0/3663039068 >> 172.21.15.16:0/3663039068 conn(0x7f45d406aba0 msgr2=0x7f45d415dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:34.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.713+0000 7f45d8807640 1 -- 172.21.15.16:0/3663039068 shutdown_connections 2024-04-25T10:07:34.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.713+0000 7f45d8807640 1 -- 172.21.15.16:0/3663039068 wait complete. 2024-04-25T10:07:34.729 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:49.462347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:50.153232+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:34.730 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:34.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.829+0000 7fa21629a640 1 Processor -- start 2024-04-25T10:07:34.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.829+0000 7fa21629a640 1 -- start start 2024-04-25T10:07:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.829+0000 7fa21629a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa21015bbe0 0x7fa2100cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.829+0000 7fa21629a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2100cf510 0x7fa2100cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.829+0000 7fa21629a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2100cc830 con 0x7fa2100cff00 2024-04-25T10:07:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.829+0000 7fa21629a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2100ccb10 con 0x7fa21015bbe0 2024-04-25T10:07:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.829+0000 7fa21629a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100ccdf0 con 0x7fa2100cf510 2024-04-25T10:07:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.830+0000 7fa20f7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2100cf510 0x7fa2100cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.830+0000 7fa20ffff640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa21015bbe0 0x7fa2100cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.830+0000 7fa20f7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2100cf510 0x7fa2100cf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:34240/0 (socket says 172.21.15.16:34240) 2024-04-25T10:07:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.830+0000 7fa20f7fe640 1 -- 172.21.15.16:0/322294840 learned_addr learned my addr 172.21.15.16:0/322294840 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.830+0000 7fa214837640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fa2100cff00 0x7fa2100c5490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:37034/0 (socket says 172.21.15.16:37034) 2024-04-25T10:07:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.830+0000 7fa20ffff640 1 --2- 172.21.15.16:0/322294840 >> v2:172.21.15.186:3300/0 conn(0x7fa21015bbe0 0x7fa2100cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:33714/0 (socket says 172.21.15.16:33714) 2024-04-25T10:07:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.830+0000 7fa20f7fe640 1 -- 172.21.15.16:0/322294840 >> v2:172.21.15.186:3300/0 conn(0x7fa21015bbe0 msgr2=0x7fa2100cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:34.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.830+0000 7fa20f7fe640 1 --2- 172.21.15.16:0/322294840 >> v2:172.21.15.186:3300/0 conn(0x7fa21015bbe0 0x7fa2100cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.830+0000 7fa20f7fe640 1 -- 172.21.15.16:0/322294840 >> v1:172.21.15.186:6789/0 conn(0x7fa2100cff00 legacy=0x7fa2100c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.830+0000 7fa20f7fe640 1 -- 172.21.15.16:0/322294840 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2100cd0d0 con 0x7fa2100cf510 2024-04-25T10:07:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.831+0000 7fa20f7fe640 1 --2- 172.21.15.16:0/322294840 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2100cf510 0x7fa2100cf970 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa204009870 tx=0x7fa20402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92cda5282af797b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.831+0000 7fa20effd640 1 -- 172.21.15.16:0/322294840 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204037070 con 0x7fa2100cf510 2024-04-25T10:07:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.831+0000 7fa20effd640 1 -- 172.21.15.16:0/322294840 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa204032440 con 0x7fa2100cf510 2024-04-25T10:07:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.831+0000 7fa20effd640 1 -- 172.21.15.16:0/322294840 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204032810 con 0x7fa2100cf510 2024-04-25T10:07:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.832+0000 7fa21629a640 1 -- 172.21.15.16:0/322294840 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2100cf510 msgr2=0x7fa2100cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.832+0000 7fa21629a640 1 --2- 172.21.15.16:0/322294840 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2100cf510 0x7fa2100cf970 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa204009870 tx=0x7fa20402a270 comp rx=0 tx=0).stop 2024-04-25T10:07:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.832+0000 7fa21629a640 1 -- 172.21.15.16:0/322294840 shutdown_connections 2024-04-25T10:07:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.832+0000 7fa21629a640 1 --2- 172.21.15.16:0/322294840 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2100cf510 0x7fa2100cf970 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.832+0000 7fa21629a640 1 --2- 172.21.15.16:0/322294840 >> v2:172.21.15.186:3300/0 conn(0x7fa21015bbe0 0x7fa2100cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.832+0000 7fa21629a640 1 -- 172.21.15.16:0/322294840 >> 172.21.15.16:0/322294840 conn(0x7fa2100b3430 msgr2=0x7fa2100b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.832+0000 7fa21629a640 1 -- 172.21.15.16:0/322294840 shutdown_connections 2024-04-25T10:07:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.832+0000 7fa21629a640 1 -- 172.21.15.16:0/322294840 wait complete. 2024-04-25T10:07:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.833+0000 7fa21629a640 1 Processor -- start 2024-04-25T10:07:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.833+0000 7fa21629a640 1 -- start start 2024-04-25T10:07:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.834+0000 7fa21629a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2100cf510 0x7fa2100b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.834+0000 7fa21629a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2100cff00 0x7fa2100ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.834+0000 7fa21629a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa21015bbe0 0x7fa2100b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.834+0000 7fa21629a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100cde10 con 0x7fa21015bbe0 2024-04-25T10:07:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.834+0000 7fa21629a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21015bfc0 con 0x7fa2100cff00 2024-04-25T10:07:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.834+0000 7fa21629a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100cc450 con 0x7fa2100cf510 2024-04-25T10:07:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.834+0000 7fa214837640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa21015bbe0 0x7fa2100b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.834+0000 7fa214837640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa21015bbe0 0x7fa2100b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48068/0 (socket says 172.21.15.16:48068) 2024-04-25T10:07:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.834+0000 7fa214837640 1 -- 172.21.15.16:0/4002788108 learned_addr learned my addr 172.21.15.16:0/4002788108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:34.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.834+0000 7fa20ffff640 1 --2- 172.21.15.16:0/4002788108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2100cf510 0x7fa2100b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.834+0000 7fa20f7fe640 1 --2- 172.21.15.16:0/4002788108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2100cff00 0x7fa2100ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.834+0000 7fa214837640 1 -- 172.21.15.16:0/4002788108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2100cf510 msgr2=0x7fa2100b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.834+0000 7fa214837640 1 --2- 172.21.15.16:0/4002788108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2100cf510 0x7fa2100b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.834+0000 7fa214837640 1 -- 172.21.15.16:0/4002788108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2100cff00 msgr2=0x7fa2100ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.834+0000 7fa214837640 1 --2- 172.21.15.16:0/4002788108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2100cff00 0x7fa2100ae670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.834+0000 7fa214837640 1 -- 172.21.15.16:0/4002788108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21006da80 con 0x7fa21015bbe0 2024-04-25T10:07:34.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.834+0000 7fa20f7fe640 1 --2- 172.21.15.16:0/4002788108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2100cff00 0x7fa2100ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:34.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.834+0000 7fa214837640 1 --2- 172.21.15.16:0/4002788108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa21015bbe0 0x7fa2100b1450 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa20800c7b0 tx=0x7fa20800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:34.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.835+0000 7fa20cff9640 1 -- 172.21.15.16:0/4002788108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208017070 con 0x7fa21015bbe0 2024-04-25T10:07:34.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.835+0000 7fa20cff9640 1 -- 172.21.15.16:0/4002788108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20800a640 con 0x7fa21015bbe0 2024-04-25T10:07:34.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.835+0000 7fa21629a640 1 -- 172.21.15.16:0/4002788108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21006d7a0 con 0x7fa21015bbe0 2024-04-25T10:07:34.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.835+0000 7fa20cff9640 1 -- 172.21.15.16:0/4002788108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20800a9f0 con 0x7fa21015bbe0 2024-04-25T10:07:34.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.835+0000 7fa21629a640 1 -- 172.21.15.16:0/4002788108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21015d990 con 0x7fa21015bbe0 2024-04-25T10:07:34.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.836+0000 7fa21629a640 1 -- 172.21.15.16:0/4002788108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2100750b0 con 0x7fa21015bbe0 2024-04-25T10:07:34.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.836+0000 7fa20cff9640 1 -- 172.21.15.16:0/4002788108 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa20801a070 con 0x7fa21015bbe0 2024-04-25T10:07:34.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.836+0000 7fa20cff9640 1 --2- 172.21.15.16:0/4002788108 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa1fc06b4d0 0x7fa1fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.837+0000 7fa20cff9640 1 -- 172.21.15.16:0/4002788108 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2080896d0 con 0x7fa21015bbe0 2024-04-25T10:07:34.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.837+0000 7fa20ffff640 1 --2- 172.21.15.16:0/4002788108 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa1fc06b4d0 0x7fa1fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.837+0000 7fa20ffff640 1 --2- 172.21.15.16:0/4002788108 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa1fc06b4d0 0x7fa1fc06d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa200009b40 tx=0x7fa200009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:34.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:34.841+0000 7fa20cff9640 1 -- 172.21.15.16:0/4002788108 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa208057f90 con 0x7fa21015bbe0 2024-04-25T10:07:35.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.023+0000 7fa21629a640 1 -- 172.21.15.16:0/4002788108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2100b42d0 con 0x7fa21015bbe0 2024-04-25T10:07:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.023+0000 7fa20cff9640 1 -- 172.21.15.16:0/4002788108 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa20805c030 con 0x7fa21015bbe0 2024-04-25T10:07:35.029 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:07:35.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.025+0000 7fa21629a640 1 -- 172.21.15.16:0/4002788108 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa1fc06b4d0 msgr2=0x7fa1fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.025+0000 7fa21629a640 1 --2- 172.21.15.16:0/4002788108 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa1fc06b4d0 0x7fa1fc06d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa200009b40 tx=0x7fa200009190 comp rx=0 tx=0).stop 2024-04-25T10:07:35.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.025+0000 7fa21629a640 1 -- 172.21.15.16:0/4002788108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa21015bbe0 msgr2=0x7fa2100b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.025+0000 7fa21629a640 1 --2- 172.21.15.16:0/4002788108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa21015bbe0 0x7fa2100b1450 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa20800c7b0 tx=0x7fa20800cc70 comp rx=0 tx=0).stop 2024-04-25T10:07:35.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.025+0000 7fa21629a640 1 -- 172.21.15.16:0/4002788108 shutdown_connections 2024-04-25T10:07:35.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.025+0000 7fa21629a640 1 --2- 172.21.15.16:0/4002788108 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa1fc06b4d0 0x7fa1fc06d990 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.025+0000 7fa21629a640 1 --2- 172.21.15.16:0/4002788108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa21015bbe0 0x7fa2100b1450 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.025+0000 7fa21629a640 1 --2- 172.21.15.16:0/4002788108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2100cff00 0x7fa2100ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.025+0000 7fa21629a640 1 --2- 172.21.15.16:0/4002788108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2100cf510 0x7fa2100b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.025+0000 7fa21629a640 1 -- 172.21.15.16:0/4002788108 >> 172.21.15.16:0/4002788108 conn(0x7fa2100b3430 msgr2=0x7fa2100b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.025+0000 7fa21629a640 1 -- 172.21.15.16:0/4002788108 shutdown_connections 2024-04-25T10:07:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.025+0000 7fa21629a640 1 -- 172.21.15.16:0/4002788108 wait complete. 2024-04-25T10:07:35.041 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:35.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.139+0000 7febad130640 1 Processor -- start 2024-04-25T10:07:35.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.139+0000 7febad130640 1 -- start start 2024-04-25T10:07:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.140+0000 7febad130640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7feba80d0490 0x7feba815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.140+0000 7febad130640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feba80ce1a0 0x7feba80ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.140+0000 7febad130640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba806c150 con 0x7feba80b2740 2024-04-25T10:07:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.140+0000 7febad130640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feba80d2000 con 0x7feba80d0490 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.140+0000 7febad130640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba806cd60 con 0x7feba80ce1a0 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.140+0000 7feba759e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feba80ce1a0 0x7feba80ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.140+0000 7feba6d9d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7feba80d0490 0x7feba815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.140+0000 7feba659c640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7feba80b2740 0x7feba80d0870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:37040/0 (socket says 172.21.15.16:37040) 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.140+0000 7feba659c640 1 -- 172.21.15.16:0/575758264 learned_addr learned my addr 172.21.15.16:0/575758264 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.140+0000 7feba6d9d640 1 -- 172.21.15.16:0/575758264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feba80ce1a0 msgr2=0x7feba80ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.140+0000 7feba6d9d640 1 --2- 172.21.15.16:0/575758264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feba80ce1a0 0x7feba80ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.141+0000 7feba6d9d640 1 -- 172.21.15.16:0/575758264 >> v1:172.21.15.186:6789/0 conn(0x7feba80b2740 legacy=0x7feba80d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.141+0000 7feba6d9d640 1 -- 172.21.15.16:0/575758264 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba806c430 con 0x7feba80d0490 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.141+0000 7feba5d9b640 1 -- 172.21.15.16:0/575758264 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104347112 0 0) 0x7feba806c150 con 0x7feba80b2740 2024-04-25T10:07:35.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.141+0000 7feba6d9d640 1 --2- 172.21.15.16:0/575758264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feba80d0490 0x7feba815ec90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7feba00097d0 tx=0x7feba002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=befda1ca7f1e3522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.141+0000 7feba5d9b640 1 -- 172.21.15.16:0/575758264 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0037070 con 0x7feba80d0490 2024-04-25T10:07:35.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.141+0000 7feba5d9b640 1 -- 172.21.15.16:0/575758264 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba002ae40 con 0x7feba80d0490 2024-04-25T10:07:35.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.141+0000 7feba5d9b640 1 -- 172.21.15.16:0/575758264 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0032660 con 0x7feba80d0490 2024-04-25T10:07:35.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.142+0000 7febad130640 1 -- 172.21.15.16:0/575758264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feba80d0490 msgr2=0x7feba815ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.142+0000 7febad130640 1 --2- 172.21.15.16:0/575758264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feba80d0490 0x7feba815ec90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7feba00097d0 tx=0x7feba002a130 comp rx=0 tx=0).stop 2024-04-25T10:07:35.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.142+0000 7febad130640 1 -- 172.21.15.16:0/575758264 shutdown_connections 2024-04-25T10:07:35.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.142+0000 7febad130640 1 --2- 172.21.15.16:0/575758264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feba80ce1a0 0x7feba80ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.142+0000 7febad130640 1 --2- 172.21.15.16:0/575758264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feba80d0490 0x7feba815ec90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.142+0000 7febad130640 1 -- 172.21.15.16:0/575758264 >> 172.21.15.16:0/575758264 conn(0x7feba80b5660 msgr2=0x7feba80b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.142+0000 7febad130640 1 -- 172.21.15.16:0/575758264 shutdown_connections 2024-04-25T10:07:35.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.142+0000 7febad130640 1 -- 172.21.15.16:0/575758264 wait complete. 2024-04-25T10:07:35.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.144+0000 7febad130640 1 Processor -- start 2024-04-25T10:07:35.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.144+0000 7febad130640 1 -- start start 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.144+0000 7febad130640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feba80b2740 0x7feba80ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.144+0000 7febad130640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feba80ce1a0 0x7feba80ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.144+0000 7febad130640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba80d0490 0x7feba8158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.144+0000 7febad130640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba806c9f0 con 0x7feba80d0490 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.144+0000 7febad130640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba815f1d0 con 0x7feba80ce1a0 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.144+0000 7febad130640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba815e020 con 0x7feba80b2740 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.144+0000 7feba759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba80d0490 0x7feba8158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.144+0000 7feba759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba80d0490 0x7feba8158b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48080/0 (socket says 172.21.15.16:48080) 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.144+0000 7feba759e640 1 -- 172.21.15.16:0/2286361881 learned_addr learned my addr 172.21.15.16:0/2286361881 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.145+0000 7feba6d9d640 1 --2- 172.21.15.16:0/2286361881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feba80b2740 0x7feba80ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.145+0000 7feba659c640 1 --2- 172.21.15.16:0/2286361881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feba80ce1a0 0x7feba80ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.145+0000 7feba759e640 1 -- 172.21.15.16:0/2286361881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feba80b2740 msgr2=0x7feba80ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.145+0000 7feba759e640 1 --2- 172.21.15.16:0/2286361881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feba80b2740 0x7feba80ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.145+0000 7feba759e640 1 -- 172.21.15.16:0/2286361881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feba80ce1a0 msgr2=0x7feba80ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.145+0000 7feba759e640 1 --2- 172.21.15.16:0/2286361881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feba80ce1a0 0x7feba80ca5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.145+0000 7feba759e640 1 -- 172.21.15.16:0/2286361881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba8153460 con 0x7feba80d0490 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.145+0000 7feba759e640 1 --2- 172.21.15.16:0/2286361881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba80d0490 0x7feba8158b20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7feb9800efc0 tx=0x7feb9800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.145+0000 7feb9f7fe640 1 -- 172.21.15.16:0/2286361881 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb98018020 con 0x7feba80d0490 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.145+0000 7feb9f7fe640 1 -- 172.21.15.16:0/2286361881 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb98014480 con 0x7feba80d0490 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.145+0000 7febad130640 1 -- 172.21.15.16:0/2286361881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba8153a00 con 0x7feba80d0490 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.145+0000 7feb9f7fe640 1 -- 172.21.15.16:0/2286361881 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb98014870 con 0x7feba80d0490 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.145+0000 7febad130640 1 -- 172.21.15.16:0/2286361881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba8153fc0 con 0x7feba80d0490 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.146+0000 7febad130640 1 -- 172.21.15.16:0/2286361881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba80750b0 con 0x7feba80d0490 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.147+0000 7feb9f7fe640 1 -- 172.21.15.16:0/2286361881 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb98005d10 con 0x7feba80d0490 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.147+0000 7feb9f7fe640 1 --2- 172.21.15.16:0/2286361881 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7feb8806b4d0 0x7feb8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.147+0000 7feb9f7fe640 1 -- 172.21.15.16:0/2286361881 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb9808abb0 con 0x7feba80d0490 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.148+0000 7feba6d9d640 1 --2- 172.21.15.16:0/2286361881 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7feb8806b4d0 0x7feb8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.148+0000 7feba6d9d640 1 --2- 172.21.15.16:0/2286361881 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7feb8806b4d0 0x7feb8806d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feba0034000 tx=0x7feba002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.152+0000 7feb9f7fe640 1 -- 172.21.15.16:0/2286361881 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb98022070 con 0x7feba80d0490 2024-04-25T10:07:35.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.329+0000 7febad130640 1 -- 172.21.15.16:0/2286361881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feba80d18d0 con 0x7feba80d0490 2024-04-25T10:07:35.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.330+0000 7feb9f7fe640 1 -- 172.21.15.16:0/2286361881 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7feb98010040 con 0x7feba80d0490 2024-04-25T10:07:35.335 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:07:35.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.331+0000 7febad130640 1 -- 172.21.15.16:0/2286361881 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7feb8806b4d0 msgr2=0x7feb8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.331+0000 7febad130640 1 --2- 172.21.15.16:0/2286361881 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7feb8806b4d0 0x7feb8806d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feba0034000 tx=0x7feba002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:35.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.331+0000 7febad130640 1 -- 172.21.15.16:0/2286361881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba80d0490 msgr2=0x7feba8158b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.331+0000 7febad130640 1 --2- 172.21.15.16:0/2286361881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba80d0490 0x7feba8158b20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7feb9800efc0 tx=0x7feb9800c490 comp rx=0 tx=0).stop 2024-04-25T10:07:35.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.331+0000 7febad130640 1 -- 172.21.15.16:0/2286361881 shutdown_connections 2024-04-25T10:07:35.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.331+0000 7febad130640 1 --2- 172.21.15.16:0/2286361881 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7feb8806b4d0 0x7feb8806d990 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.331+0000 7febad130640 1 --2- 172.21.15.16:0/2286361881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba80d0490 0x7feba8158b20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.331+0000 7febad130640 1 --2- 172.21.15.16:0/2286361881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feba80ce1a0 0x7feba80ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.331+0000 7febad130640 1 --2- 172.21.15.16:0/2286361881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feba80b2740 0x7feba80ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.332+0000 7febad130640 1 -- 172.21.15.16:0/2286361881 >> 172.21.15.16:0/2286361881 conn(0x7feba80b5660 msgr2=0x7feba80d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.332+0000 7febad130640 1 -- 172.21.15.16:0/2286361881 shutdown_connections 2024-04-25T10:07:35.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.332+0000 7febad130640 1 -- 172.21.15.16:0/2286361881 wait complete. 2024-04-25T10:07:35.348 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.446+0000 7f281edf5640 1 Processor -- start 2024-04-25T10:07:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.446+0000 7f281edf5640 1 -- start start 2024-04-25T10:07:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.447+0000 7f281edf5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f281815dc80 0x7f2818161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.447+0000 7f281edf5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28180b97c0 0x7f28180b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.447+0000 7f281edf5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28181627a0 con 0x7f28180b5a30 2024-04-25T10:07:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.447+0000 7f281edf5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28180b5ea0 con 0x7f281815dc80 2024-04-25T10:07:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.447+0000 7f281edf5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28181621e0 con 0x7f28180b97c0 2024-04-25T10:07:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.447+0000 7f281cb91640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f281815dc80 0x7f2818161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.447+0000 7f281cb91640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f281815dc80 0x7f2818161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48086/0 (socket says 172.21.15.16:48086) 2024-04-25T10:07:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.447+0000 7f281cb91640 1 -- 172.21.15.16:0/2106950280 learned_addr learned my addr 172.21.15.16:0/2106950280 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.447+0000 7f2817fff640 1 --2- 172.21.15.16:0/2106950280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28180b97c0 0x7f28180b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.448+0000 7f281cb91640 1 -- 172.21.15.16:0/2106950280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28180b97c0 msgr2=0x7f28180b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.448+0000 7f281cb91640 1 --2- 172.21.15.16:0/2106950280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28180b97c0 0x7f28180b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.448+0000 7f281cb91640 1 -- 172.21.15.16:0/2106950280 >> v1:172.21.15.186:6789/0 conn(0x7f28180b5a30 legacy=0x7f281806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.448+0000 7f281cb91640 1 -- 172.21.15.16:0/2106950280 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28181624c0 con 0x7f281815dc80 2024-04-25T10:07:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.448+0000 7f281cb91640 1 --2- 172.21.15.16:0/2106950280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f281815dc80 0x7f2818161870 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2808009870 tx=0x7f280802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8733dc029ff927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.449+0000 7f28177fe640 1 -- 172.21.15.16:0/2106950280 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808037070 con 0x7f281815dc80 2024-04-25T10:07:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.449+0000 7f28177fe640 1 -- 172.21.15.16:0/2106950280 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2808032440 con 0x7f281815dc80 2024-04-25T10:07:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.449+0000 7f28177fe640 1 -- 172.21.15.16:0/2106950280 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808032810 con 0x7f281815dc80 2024-04-25T10:07:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.449+0000 7f281edf5640 1 -- 172.21.15.16:0/2106950280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f281815dc80 msgr2=0x7f2818161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.449+0000 7f281edf5640 1 --2- 172.21.15.16:0/2106950280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f281815dc80 0x7f2818161870 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2808009870 tx=0x7f280802a270 comp rx=0 tx=0).stop 2024-04-25T10:07:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.449+0000 7f281edf5640 1 -- 172.21.15.16:0/2106950280 shutdown_connections 2024-04-25T10:07:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.449+0000 7f281edf5640 1 --2- 172.21.15.16:0/2106950280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28180b97c0 0x7f28180b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.449+0000 7f281edf5640 1 --2- 172.21.15.16:0/2106950280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f281815dc80 0x7f2818161870 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.449+0000 7f281edf5640 1 -- 172.21.15.16:0/2106950280 >> 172.21.15.16:0/2106950280 conn(0x7f28180d04f0 msgr2=0x7f28180d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.449+0000 7f281edf5640 1 -- 172.21.15.16:0/2106950280 shutdown_connections 2024-04-25T10:07:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.449+0000 7f281edf5640 1 -- 172.21.15.16:0/2106950280 wait complete. 2024-04-25T10:07:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.450+0000 7f281edf5640 1 Processor -- start 2024-04-25T10:07:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.450+0000 7f281edf5640 1 -- start start 2024-04-25T10:07:35.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.451+0000 7f281edf5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28180b5a30 0x7f28180c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.451+0000 7f281edf5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28180b97c0 0x7f2818157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.451+0000 7f281edf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f281815dc80 0x7f28180b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.451+0000 7f281edf5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2818161db0 con 0x7f281815dc80 2024-04-25T10:07:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.451+0000 7f281edf5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281815e060 con 0x7f28180b5a30 2024-04-25T10:07:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.451+0000 7f281edf5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2818161f30 con 0x7f28180b97c0 2024-04-25T10:07:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.451+0000 7f281d392640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f281815dc80 0x7f28180b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.451+0000 7f281d392640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f281815dc80 0x7f28180b76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48090/0 (socket says 172.21.15.16:48090) 2024-04-25T10:07:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.451+0000 7f281d392640 1 -- 172.21.15.16:0/1791507405 learned_addr learned my addr 172.21.15.16:0/1791507405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.451+0000 7f2817fff640 1 --2- 172.21.15.16:0/1791507405 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28180b97c0 0x7f2818157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.451+0000 7f281cb91640 1 --2- 172.21.15.16:0/1791507405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28180b5a30 0x7f28180c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.452+0000 7f281d392640 1 -- 172.21.15.16:0/1791507405 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28180b97c0 msgr2=0x7f2818157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.452+0000 7f281d392640 1 --2- 172.21.15.16:0/1791507405 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28180b97c0 0x7f2818157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.452+0000 7f281d392640 1 -- 172.21.15.16:0/1791507405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28180b5a30 msgr2=0x7f28180c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.452+0000 7f281d392640 1 --2- 172.21.15.16:0/1791507405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28180b5a30 0x7f28180c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.452+0000 7f281d392640 1 -- 172.21.15.16:0/1791507405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281815ffd0 con 0x7f281815dc80 2024-04-25T10:07:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.452+0000 7f281d392640 1 --2- 172.21.15.16:0/1791507405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f281815dc80 0x7f28180b76d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f281000c7a0 tx=0x7f281000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.452+0000 7f28157fa640 1 -- 172.21.15.16:0/1791507405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2810003b70 con 0x7f281815dc80 2024-04-25T10:07:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.452+0000 7f28157fa640 1 -- 172.21.15.16:0/1791507405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2810003d30 con 0x7f281815dc80 2024-04-25T10:07:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.452+0000 7f281edf5640 1 -- 172.21.15.16:0/1791507405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281815fcf0 con 0x7f281815dc80 2024-04-25T10:07:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.452+0000 7f28157fa640 1 -- 172.21.15.16:0/1791507405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281000aa90 con 0x7f281815dc80 2024-04-25T10:07:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.452+0000 7f281edf5640 1 -- 172.21.15.16:0/1791507405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28180c7920 con 0x7f281815dc80 2024-04-25T10:07:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.453+0000 7f281edf5640 1 -- 172.21.15.16:0/1791507405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28180750b0 con 0x7f281815dc80 2024-04-25T10:07:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.454+0000 7f28157fa640 1 -- 172.21.15.16:0/1791507405 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f281000ac30 con 0x7f281815dc80 2024-04-25T10:07:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.454+0000 7f28157fa640 1 --2- 172.21.15.16:0/1791507405 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f280406b4d0 0x7f280406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.454+0000 7f28157fa640 1 -- 172.21.15.16:0/1791507405 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f281008a090 con 0x7f281815dc80 2024-04-25T10:07:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.454+0000 7f281cb91640 1 --2- 172.21.15.16:0/1791507405 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f280406b4d0 0x7f280406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.455+0000 7f281cb91640 1 --2- 172.21.15.16:0/1791507405 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f280406b4d0 0x7f280406d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2808034000 tx=0x7f280802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.458+0000 7f28157fa640 1 -- 172.21.15.16:0/1791507405 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2810058830 con 0x7f281815dc80 2024-04-25T10:07:35.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.640+0000 7f281edf5640 1 -- 172.21.15.16:0/1791507405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f28181619f0 con 0x7f281815dc80 2024-04-25T10:07:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.641+0000 7f28157fa640 1 -- 172.21.15.16:0/1791507405 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f281005c8d0 con 0x7f281815dc80 2024-04-25T10:07:35.650 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:07:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.643+0000 7f281edf5640 1 -- 172.21.15.16:0/1791507405 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f280406b4d0 msgr2=0x7f280406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.643+0000 7f281edf5640 1 --2- 172.21.15.16:0/1791507405 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f280406b4d0 0x7f280406d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2808034000 tx=0x7f280802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.643+0000 7f281edf5640 1 -- 172.21.15.16:0/1791507405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f281815dc80 msgr2=0x7f28180b76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.643+0000 7f281edf5640 1 --2- 172.21.15.16:0/1791507405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f281815dc80 0x7f28180b76d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f281000c7a0 tx=0x7f281000cc60 comp rx=0 tx=0).stop 2024-04-25T10:07:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.643+0000 7f281edf5640 1 -- 172.21.15.16:0/1791507405 shutdown_connections 2024-04-25T10:07:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.643+0000 7f281edf5640 1 --2- 172.21.15.16:0/1791507405 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f280406b4d0 0x7f280406d990 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.643+0000 7f281edf5640 1 --2- 172.21.15.16:0/1791507405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f281815dc80 0x7f28180b76d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.643+0000 7f281edf5640 1 --2- 172.21.15.16:0/1791507405 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28180b97c0 0x7f2818157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.643+0000 7f281edf5640 1 --2- 172.21.15.16:0/1791507405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28180b5a30 0x7f28180c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.643+0000 7f281edf5640 1 -- 172.21.15.16:0/1791507405 >> 172.21.15.16:0/1791507405 conn(0x7f28180d04f0 msgr2=0x7f28180cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.643+0000 7f281edf5640 1 -- 172.21.15.16:0/1791507405 shutdown_connections 2024-04-25T10:07:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.643+0000 7f281edf5640 1 -- 172.21.15.16:0/1791507405 wait complete. 2024-04-25T10:07:35.660 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:07:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.756+0000 7fae3cb22640 1 Processor -- start 2024-04-25T10:07:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.756+0000 7fae3cb22640 1 -- start start 2024-04-25T10:07:35.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.757+0000 7fae3cb22640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fae38161090 0x7fae38161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.757+0000 7fae3cb22640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae380ceeb0 0x7fae380d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.757+0000 7fae3cb22640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae38161f80 con 0x7fae380b9720 2024-04-25T10:07:35.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.757+0000 7fae3cb22640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae380d21a0 con 0x7fae38161090 2024-04-25T10:07:35.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.757+0000 7fae3cb22640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae38162260 con 0x7fae380ceeb0 2024-04-25T10:07:35.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.757+0000 7fae35d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fae380b9720 0x7fae380ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41828/0 (socket says 172.21.15.16:41828) 2024-04-25T10:07:35.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.757+0000 7fae35d9b640 1 -- 172.21.15.16:0/4278106603 learned_addr learned my addr 172.21.15.16:0/4278106603 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.757+0000 7fae3659c640 1 --2- 172.21.15.16:0/4278106603 >> v2:172.21.15.186:3300/0 conn(0x7fae38161090 0x7fae38161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.757+0000 7fae36d9d640 1 --2- 172.21.15.16:0/4278106603 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae380ceeb0 0x7fae380d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.757+0000 7fae3559a640 1 -- 172.21.15.16:0/4278106603 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42288896 0 0) 0x7fae38161f80 con 0x7fae380b9720 2024-04-25T10:07:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.757+0000 7fae3559a640 1 -- 172.21.15.16:0/4278106603 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae20003520 con 0x7fae380b9720 2024-04-25T10:07:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.757+0000 7fae3659c640 1 -- 172.21.15.16:0/4278106603 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae380ceeb0 msgr2=0x7fae380d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.757+0000 7fae3659c640 1 --2- 172.21.15.16:0/4278106603 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae380ceeb0 0x7fae380d2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.757+0000 7fae3659c640 1 -- 172.21.15.16:0/4278106603 >> v1:172.21.15.16:6789/0 conn(0x7fae380b9720 legacy=0x7fae380ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.757+0000 7fae3659c640 1 -- 172.21.15.16:0/4278106603 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae38162540 con 0x7fae38161090 2024-04-25T10:07:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.757+0000 7fae36d9d640 1 --2- 172.21.15.16:0/4278106603 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae380ceeb0 0x7fae380d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.758+0000 7fae3659c640 1 --2- 172.21.15.16:0/4278106603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae38161090 0x7fae38161730 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fae280097a0 tx=0x7fae2802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54fe4056ac70131 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.758+0000 7fae3559a640 1 -- 172.21.15.16:0/4278106603 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28037070 con 0x7fae38161090 2024-04-25T10:07:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.758+0000 7fae3559a640 1 -- 172.21.15.16:0/4278106603 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2802ae50 con 0x7fae38161090 2024-04-25T10:07:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.758+0000 7fae3559a640 1 -- 172.21.15.16:0/4278106603 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28032640 con 0x7fae38161090 2024-04-25T10:07:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.759+0000 7fae3cb22640 1 -- 172.21.15.16:0/4278106603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae38161090 msgr2=0x7fae38161730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.759+0000 7fae3cb22640 1 --2- 172.21.15.16:0/4278106603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae38161090 0x7fae38161730 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fae280097a0 tx=0x7fae2802a100 comp rx=0 tx=0).stop 2024-04-25T10:07:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.759+0000 7fae3cb22640 1 -- 172.21.15.16:0/4278106603 shutdown_connections 2024-04-25T10:07:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.759+0000 7fae3cb22640 1 --2- 172.21.15.16:0/4278106603 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae380ceeb0 0x7fae380d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.759+0000 7fae3cb22640 1 --2- 172.21.15.16:0/4278106603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae38161090 0x7fae38161730 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.759+0000 7fae3cb22640 1 -- 172.21.15.16:0/4278106603 >> 172.21.15.16:0/4278106603 conn(0x7fae380b55e0 msgr2=0x7fae380b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.759+0000 7fae3cb22640 1 -- 172.21.15.16:0/4278106603 shutdown_connections 2024-04-25T10:07:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.759+0000 7fae3cb22640 1 -- 172.21.15.16:0/4278106603 wait complete. 2024-04-25T10:07:35.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.760+0000 7fae3cb22640 1 Processor -- start 2024-04-25T10:07:35.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.760+0000 7fae3cb22640 1 -- start start 2024-04-25T10:07:35.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.761+0000 7fae3cb22640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae380b9720 0x7fae3814dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.761+0000 7fae3cb22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae380ceeb0 0x7fae3815cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.761+0000 7fae3cb22640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae38161090 0x7fae380cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.761+0000 7fae3cb22640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae38161c70 con 0x7fae380ceeb0 2024-04-25T10:07:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.761+0000 7fae3cb22640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae38161470 con 0x7fae38161090 2024-04-25T10:07:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.761+0000 7fae3cb22640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae380d2b70 con 0x7fae380b9720 2024-04-25T10:07:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.761+0000 7fae35d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae380ceeb0 0x7fae3815cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.761+0000 7fae35d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae380ceeb0 0x7fae3815cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48092/0 (socket says 172.21.15.16:48092) 2024-04-25T10:07:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.761+0000 7fae35d9b640 1 -- 172.21.15.16:0/2313377277 learned_addr learned my addr 172.21.15.16:0/2313377277 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.761+0000 7fae36d9d640 1 --2- 172.21.15.16:0/2313377277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae38161090 0x7fae380cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.761+0000 7fae3659c640 1 --2- 172.21.15.16:0/2313377277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae380b9720 0x7fae3814dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.762+0000 7fae35d9b640 1 -- 172.21.15.16:0/2313377277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae380b9720 msgr2=0x7fae3814dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.762+0000 7fae35d9b640 1 --2- 172.21.15.16:0/2313377277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae380b9720 0x7fae3814dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.762+0000 7fae35d9b640 1 -- 172.21.15.16:0/2313377277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae38161090 msgr2=0x7fae380cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.762+0000 7fae35d9b640 1 --2- 172.21.15.16:0/2313377277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae38161090 0x7fae380cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.762+0000 7fae35d9b640 1 -- 172.21.15.16:0/2313377277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3815d660 con 0x7fae380ceeb0 2024-04-25T10:07:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.762+0000 7fae36d9d640 1 --2- 172.21.15.16:0/2313377277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae38161090 0x7fae380cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.762+0000 7fae35d9b640 1 --2- 172.21.15.16:0/2313377277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae380ceeb0 0x7fae3815cd60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0010e0 tx=0x7fae2c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.762+0000 7fae26ffd640 1 -- 172.21.15.16:0/2313377277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c017070 con 0x7fae380ceeb0 2024-04-25T10:07:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.762+0000 7fae26ffd640 1 -- 172.21.15.16:0/2313377277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2c005d00 con 0x7fae380ceeb0 2024-04-25T10:07:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.762+0000 7fae3cb22640 1 -- 172.21.15.16:0/2313377277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3815d940 con 0x7fae380ceeb0 2024-04-25T10:07:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.762+0000 7fae26ffd640 1 -- 172.21.15.16:0/2313377277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c0072b0 con 0x7fae380ceeb0 2024-04-25T10:07:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.762+0000 7fae3cb22640 1 -- 172.21.15.16:0/2313377277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3815e880 con 0x7fae380ceeb0 2024-04-25T10:07:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.763+0000 7fae3cb22640 1 -- 172.21.15.16:0/2313377277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae380750b0 con 0x7fae380ceeb0 2024-04-25T10:07:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.763+0000 7fae26ffd640 1 -- 172.21.15.16:0/2313377277 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae2c01a070 con 0x7fae380ceeb0 2024-04-25T10:07:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.764+0000 7fae26ffd640 1 --2- 172.21.15.16:0/2313377277 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae1806b4d0 0x7fae1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.764+0000 7fae26ffd640 1 -- 172.21.15.16:0/2313377277 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae2c0895b0 con 0x7fae380ceeb0 2024-04-25T10:07:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.764+0000 7fae3659c640 1 --2- 172.21.15.16:0/2313377277 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae1806b4d0 0x7fae1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.764+0000 7fae3659c640 1 --2- 172.21.15.16:0/2313377277 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae1806b4d0 0x7fae1806d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fae28034000 tx=0x7fae2802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.768+0000 7fae26ffd640 1 -- 172.21.15.16:0/2313377277 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae2c057dd0 con 0x7fae380ceeb0 2024-04-25T10:07:35.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.961+0000 7fae3cb22640 1 -- 172.21.15.16:0/2313377277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fae380d1700 con 0x7fae380ceeb0 2024-04-25T10:07:35.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.961+0000 7fae26ffd640 1 -- 172.21.15.16:0/2313377277 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) v1 ==== 80+0+0 (secure 0 0 0) 0x7fae2c05be70 con 0x7fae380ceeb0 2024-04-25T10:07:35.967 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-04-25T10:07:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.963+0000 7fae3cb22640 1 -- 172.21.15.16:0/2313377277 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae1806b4d0 msgr2=0x7fae1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.963+0000 7fae3cb22640 1 --2- 172.21.15.16:0/2313377277 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae1806b4d0 0x7fae1806d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fae28034000 tx=0x7fae2802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.963+0000 7fae3cb22640 1 -- 172.21.15.16:0/2313377277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae380ceeb0 msgr2=0x7fae3815cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.963+0000 7fae3cb22640 1 --2- 172.21.15.16:0/2313377277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae380ceeb0 0x7fae3815cd60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0010e0 tx=0x7fae2c00c960 comp rx=0 tx=0).stop 2024-04-25T10:07:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.963+0000 7fae3cb22640 1 -- 172.21.15.16:0/2313377277 shutdown_connections 2024-04-25T10:07:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.963+0000 7fae3cb22640 1 --2- 172.21.15.16:0/2313377277 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae1806b4d0 0x7fae1806d990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.963+0000 7fae3cb22640 1 --2- 172.21.15.16:0/2313377277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae38161090 0x7fae380cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.963+0000 7fae3cb22640 1 --2- 172.21.15.16:0/2313377277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae380ceeb0 0x7fae3815cd60 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.963+0000 7fae3cb22640 1 --2- 172.21.15.16:0/2313377277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae380b9720 0x7fae3814dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.963+0000 7fae3cb22640 1 -- 172.21.15.16:0/2313377277 >> 172.21.15.16:0/2313377277 conn(0x7fae380b55e0 msgr2=0x7fae380b7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.963+0000 7fae3cb22640 1 -- 172.21.15.16:0/2313377277 shutdown_connections 2024-04-25T10:07:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:35.963+0000 7fae3cb22640 1 -- 172.21.15.16:0/2313377277 wait complete. 2024-04-25T10:07:35.980 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:07:35.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:35.981 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:36.024 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:07:36.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:36.024 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:07:36.080 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:07:36.080 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:36.097 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:36.097 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:07:36.097 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:36.136 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:07:36.136 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:07:36.136 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:36.153 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:36.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.271+0000 7ffb01d70640 1 Processor -- start 2024-04-25T10:07:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.271+0000 7ffb01d70640 1 -- start start 2024-04-25T10:07:36.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.272+0000 7ffb01d70640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffafc15dce0 0x7ffafc0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.272+0000 7ffb01d70640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ffafc0d35b0 0x7ffafc0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.272+0000 7ffb01d70640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffafc15e0c0 con 0x7ffafc0d1f90 2024-04-25T10:07:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.272+0000 7ffb01d70640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffafc0b7b10 con 0x7ffafc0d35b0 2024-04-25T10:07:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.272+0000 7ffb01d70640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc0d2510 con 0x7ffafc15dce0 2024-04-25T10:07:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.272+0000 7ffafaffd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ffafc0d35b0 0x7ffafc0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.273+0000 7ffafb7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffafc15dce0 0x7ffafc0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.273+0000 7ffafbfff640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7ffafc0d1f90 0x7ffafc06bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:37052/0 (socket says 172.21.15.16:37052) 2024-04-25T10:07:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.273+0000 7ffafbfff640 1 -- 172.21.15.16:0/3023483684 learned_addr learned my addr 172.21.15.16:0/3023483684 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.273+0000 7ffafaffd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ffafc0d35b0 0x7ffafc0b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:33800/0 (socket says 172.21.15.16:33800) 2024-04-25T10:07:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.273+0000 7ffafb7fe640 1 -- 172.21.15.16:0/3023483684 >> v2:172.21.15.186:3300/0 conn(0x7ffafc0d35b0 msgr2=0x7ffafc0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.273+0000 7ffafb7fe640 1 --2- 172.21.15.16:0/3023483684 >> v2:172.21.15.186:3300/0 conn(0x7ffafc0d35b0 0x7ffafc0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.273+0000 7ffafb7fe640 1 -- 172.21.15.16:0/3023483684 >> v1:172.21.15.186:6789/0 conn(0x7ffafc0d1f90 legacy=0x7ffafc06bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.273+0000 7ffafb7fe640 1 -- 172.21.15.16:0/3023483684 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc0d27f0 con 0x7ffafc15dce0 2024-04-25T10:07:36.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.273+0000 7ffafa7fc640 1 -- 172.21.15.16:0/3023483684 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 548790334 0 0) 0x7ffafc15e0c0 con 0x7ffafc0d1f90 2024-04-25T10:07:36.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.273+0000 7ffafaffd640 1 --2- 172.21.15.16:0/3023483684 >> v2:172.21.15.186:3300/0 conn(0x7ffafc0d35b0 0x7ffafc0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:36.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.274+0000 7ffafb7fe640 1 --2- 172.21.15.16:0/3023483684 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffafc15dce0 0x7ffafc0d1a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffaec0097a0 tx=0x7ffaec02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c44eb4bf05415610 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.274+0000 7ffafa7fc640 1 -- 172.21.15.16:0/3023483684 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec037070 con 0x7ffafc15dce0 2024-04-25T10:07:36.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.274+0000 7ffafa7fc640 1 -- 172.21.15.16:0/3023483684 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaec02ae50 con 0x7ffafc15dce0 2024-04-25T10:07:36.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.274+0000 7ffafa7fc640 1 -- 172.21.15.16:0/3023483684 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec032660 con 0x7ffafc15dce0 2024-04-25T10:07:36.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.274+0000 7ffb01d70640 1 -- 172.21.15.16:0/3023483684 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffafc15dce0 msgr2=0x7ffafc0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.274+0000 7ffb01d70640 1 --2- 172.21.15.16:0/3023483684 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffafc15dce0 0x7ffafc0d1a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffaec0097a0 tx=0x7ffaec02a100 comp rx=0 tx=0).stop 2024-04-25T10:07:36.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.275+0000 7ffb01d70640 1 -- 172.21.15.16:0/3023483684 shutdown_connections 2024-04-25T10:07:36.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.275+0000 7ffb01d70640 1 --2- 172.21.15.16:0/3023483684 >> v2:172.21.15.186:3300/0 conn(0x7ffafc0d35b0 0x7ffafc0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.275+0000 7ffb01d70640 1 --2- 172.21.15.16:0/3023483684 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffafc15dce0 0x7ffafc0d1a50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.275+0000 7ffb01d70640 1 -- 172.21.15.16:0/3023483684 >> 172.21.15.16:0/3023483684 conn(0x7ffafc072900 msgr2=0x7ffafc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:36.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.275+0000 7ffb01d70640 1 -- 172.21.15.16:0/3023483684 shutdown_connections 2024-04-25T10:07:36.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.275+0000 7ffb01d70640 1 -- 172.21.15.16:0/3023483684 wait complete. 2024-04-25T10:07:36.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.276+0000 7ffb01d70640 1 Processor -- start 2024-04-25T10:07:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.276+0000 7ffb01d70640 1 -- start start 2024-04-25T10:07:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffb01d70640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffafc0d1f90 0x7ffafc153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffb01d70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc0d35b0 0x7ffafc0b8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffb01d70640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffafc15dce0 0x7ffafc0c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffb01d70640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc0d3090 con 0x7ffafc0d35b0 2024-04-25T10:07:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffb01d70640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc0d2db0 con 0x7ffafc15dce0 2024-04-25T10:07:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffb01d70640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc0b9820 con 0x7ffafc0d1f90 2024-04-25T10:07:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffafaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc0d35b0 0x7ffafc0b8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffafaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc0d35b0 0x7ffafc0b8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48106/0 (socket says 172.21.15.16:48106) 2024-04-25T10:07:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffafaffd640 1 -- 172.21.15.16:0/1771692072 learned_addr learned my addr 172.21.15.16:0/1771692072 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffafb7fe640 1 --2- 172.21.15.16:0/1771692072 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffafc0d1f90 0x7ffafc153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffafbfff640 1 --2- 172.21.15.16:0/1771692072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffafc15dce0 0x7ffafc0c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffafaffd640 1 -- 172.21.15.16:0/1771692072 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffafc0d1f90 msgr2=0x7ffafc153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffafaffd640 1 --2- 172.21.15.16:0/1771692072 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffafc0d1f90 0x7ffafc153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffafaffd640 1 -- 172.21.15.16:0/1771692072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffafc15dce0 msgr2=0x7ffafc0c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffafaffd640 1 --2- 172.21.15.16:0/1771692072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffafc15dce0 0x7ffafc0c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffafaffd640 1 -- 172.21.15.16:0/1771692072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc153050 con 0x7ffafc0d35b0 2024-04-25T10:07:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffafb7fe640 1 --2- 172.21.15.16:0/1771692072 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffafc0d1f90 0x7ffafc153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffafaffd640 1 --2- 172.21.15.16:0/1771692072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc0d35b0 0x7ffafc0b8450 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000ebe0 tx=0x7ffaf000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffae3fff640 1 -- 172.21.15.16:0/1771692072 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0017070 con 0x7ffafc0d35b0 2024-04-25T10:07:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.277+0000 7ffae3fff640 1 -- 172.21.15.16:0/1771692072 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf000cc30 con 0x7ffafc0d35b0 2024-04-25T10:07:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.278+0000 7ffae3fff640 1 -- 172.21.15.16:0/1771692072 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0013470 con 0x7ffafc0d35b0 2024-04-25T10:07:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.278+0000 7ffb01d70640 1 -- 172.21.15.16:0/1771692072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffafc153330 con 0x7ffafc0d35b0 2024-04-25T10:07:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.278+0000 7ffb01d70640 1 -- 172.21.15.16:0/1771692072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffafc0b8990 con 0x7ffafc0d35b0 2024-04-25T10:07:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.278+0000 7ffb01d70640 1 -- 172.21.15.16:0/1771692072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffafc0749b0 con 0x7ffafc0d35b0 2024-04-25T10:07:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.279+0000 7ffae3fff640 1 -- 172.21.15.16:0/1771692072 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffaf001a070 con 0x7ffafc0d35b0 2024-04-25T10:07:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.279+0000 7ffae3fff640 1 --2- 172.21.15.16:0/1771692072 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffae806b4d0 0x7ffae806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.279+0000 7ffae3fff640 1 -- 172.21.15.16:0/1771692072 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffaf008a050 con 0x7ffafc0d35b0 2024-04-25T10:07:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.280+0000 7ffafb7fe640 1 --2- 172.21.15.16:0/1771692072 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffae806b4d0 0x7ffae806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.280+0000 7ffafb7fe640 1 --2- 172.21.15.16:0/1771692072 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffae806b4d0 0x7ffae806d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffaec034000 tx=0x7ffaec02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.284+0000 7ffae3fff640 1 -- 172.21.15.16:0/1771692072 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffaf0058910 con 0x7ffafc0d35b0 2024-04-25T10:07:36.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.499+0000 7ffb01d70640 1 -- 172.21.15.16:0/1771692072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffafc0d4750 con 0x7ffafc0d35b0 2024-04-25T10:07:36.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.499+0000 7ffae3fff640 1 -- 172.21.15.16:0/1771692072 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4940 (secure 0 0 0) 0x7ffaf005c9b0 con 0x7ffafc0d35b0 2024-04-25T10:07:36.505 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:36.505 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/791079479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":791079479},{"type":"v1","addr":"172.21.15.16:6835","nonce":791079479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1747921236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1747921236},{"type":"v1","addr":"172.21.15.16:6834","nonce":1747921236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":34880,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/3183779257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3183779257},{"type":"v1","addr":"172.21.15.186:6836","nonce":3183779257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:51.182691+0000","modified":"2024-04-25T10:06:53.179194+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34883},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34883":{"gid":34883,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/83423489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":83423489},{"type":"v1","addr":"172.21.15.186:6837","nonce":83423489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:36.506 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 21 2024-04-25T10:07:36.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.502+0000 7ffb01d70640 1 -- 172.21.15.16:0/1771692072 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffae806b4d0 msgr2=0x7ffae806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.502+0000 7ffb01d70640 1 --2- 172.21.15.16:0/1771692072 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffae806b4d0 0x7ffae806d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffaec034000 tx=0x7ffaec02e000 comp rx=0 tx=0).stop 2024-04-25T10:07:36.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.502+0000 7ffb01d70640 1 -- 172.21.15.16:0/1771692072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc0d35b0 msgr2=0x7ffafc0b8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.502+0000 7ffb01d70640 1 --2- 172.21.15.16:0/1771692072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc0d35b0 0x7ffafc0b8450 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000ebe0 tx=0x7ffaf000c520 comp rx=0 tx=0).stop 2024-04-25T10:07:36.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.502+0000 7ffb01d70640 1 -- 172.21.15.16:0/1771692072 shutdown_connections 2024-04-25T10:07:36.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.502+0000 7ffb01d70640 1 --2- 172.21.15.16:0/1771692072 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffae806b4d0 0x7ffae806d990 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.502+0000 7ffb01d70640 1 --2- 172.21.15.16:0/1771692072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffafc15dce0 0x7ffafc0c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.502+0000 7ffb01d70640 1 --2- 172.21.15.16:0/1771692072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc0d35b0 0x7ffafc0b8450 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.502+0000 7ffb01d70640 1 --2- 172.21.15.16:0/1771692072 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffafc0d1f90 0x7ffafc153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.502+0000 7ffb01d70640 1 -- 172.21.15.16:0/1771692072 >> 172.21.15.16:0/1771692072 conn(0x7ffafc072900 msgr2=0x7ffafc0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:36.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.502+0000 7ffb01d70640 1 -- 172.21.15.16:0/1771692072 shutdown_connections 2024-04-25T10:07:36.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.502+0000 7ffb01d70640 1 -- 172.21.15.16:0/1771692072 wait complete. 2024-04-25T10:07:36.520 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:36.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.614+0000 7f3877709640 1 Processor -- start 2024-04-25T10:07:36.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.615+0000 7f3877709640 1 -- start start 2024-04-25T10:07:36.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.615+0000 7f3877709640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f38700b9460 0x7f38700ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.615+0000 7f3877709640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f38700d34e0 0x7f38700cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.615+0000 7f3877709640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38700d2440 con 0x7f387015dd50 2024-04-25T10:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.615+0000 7f3877709640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38700d2720 con 0x7f38700d34e0 2024-04-25T10:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.615+0000 7f3877709640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38700d2a00 con 0x7f38700b9460 2024-04-25T10:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.615+0000 7f3875ca6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f38700d34e0 0x7f38700cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.615+0000 7f3875ca6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f38700d34e0 0x7f38700cece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48110/0 (socket says 172.21.15.16:48110) 2024-04-25T10:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.615+0000 7f3875ca6640 1 -- 172.21.15.16:0/1120403974 learned_addr learned my addr 172.21.15.16:0/1120403974 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.616+0000 7f3874ca4640 1 --2- 172.21.15.16:0/1120403974 >> v2:172.21.15.186:3300/0 conn(0x7f38700b9460 0x7f38700ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.616+0000 7f3875ca6640 1 -- 172.21.15.16:0/1120403974 >> v2:172.21.15.186:3300/0 conn(0x7f38700b9460 msgr2=0x7f38700ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.616+0000 7f3875ca6640 1 --2- 172.21.15.16:0/1120403974 >> v2:172.21.15.186:3300/0 conn(0x7f38700b9460 0x7f38700ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.616+0000 7f3875ca6640 1 -- 172.21.15.16:0/1120403974 >> v1:172.21.15.186:6789/0 conn(0x7f387015dd50 legacy=0x7f38700d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.616+0000 7f3875ca6640 1 -- 172.21.15.16:0/1120403974 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38700d2ce0 con 0x7f38700d34e0 2024-04-25T10:07:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.616+0000 7f3875ca6640 1 --2- 172.21.15.16:0/1120403974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38700d34e0 0x7f38700cece0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f387006ac90 tx=0x7f386802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22e58401890f60c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.617+0000 7f3867fff640 1 -- 172.21.15.16:0/1120403974 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3868036070 con 0x7f38700d34e0 2024-04-25T10:07:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.617+0000 7f3867fff640 1 -- 172.21.15.16:0/1120403974 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3868031600 con 0x7f38700d34e0 2024-04-25T10:07:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.617+0000 7f3867fff640 1 -- 172.21.15.16:0/1120403974 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38680319d0 con 0x7f38700d34e0 2024-04-25T10:07:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.617+0000 7f3877709640 1 -- 172.21.15.16:0/1120403974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38700d34e0 msgr2=0x7f38700cece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.617+0000 7f3877709640 1 --2- 172.21.15.16:0/1120403974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38700d34e0 0x7f38700cece0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f387006ac90 tx=0x7f386802a660 comp rx=0 tx=0).stop 2024-04-25T10:07:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.617+0000 7f3877709640 1 -- 172.21.15.16:0/1120403974 shutdown_connections 2024-04-25T10:07:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.617+0000 7f3877709640 1 --2- 172.21.15.16:0/1120403974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38700d34e0 0x7f38700cece0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.617+0000 7f3877709640 1 --2- 172.21.15.16:0/1120403974 >> v2:172.21.15.186:3300/0 conn(0x7f38700b9460 0x7f38700ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.617+0000 7f3877709640 1 -- 172.21.15.16:0/1120403974 >> 172.21.15.16:0/1120403974 conn(0x7f38700722d0 msgr2=0x7f38700726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.617+0000 7f3877709640 1 -- 172.21.15.16:0/1120403974 shutdown_connections 2024-04-25T10:07:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.617+0000 7f3877709640 1 -- 172.21.15.16:0/1120403974 wait complete. 2024-04-25T10:07:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.618+0000 7f3877709640 1 Processor -- start 2024-04-25T10:07:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.618+0000 7f3877709640 1 -- start start 2024-04-25T10:07:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.619+0000 7f3877709640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38700b9460 0x7f38700b4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.619+0000 7f3877709640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38700d34e0 0x7f387014ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.619+0000 7f3877709640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f387015dd50 0x7f387015aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.619+0000 7f3877709640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38700b6df0 con 0x7f387015dd50 2024-04-25T10:07:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.619+0000 7f3877709640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38700b6c70 con 0x7f38700b9460 2024-04-25T10:07:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.619+0000 7f3877709640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38700b6f70 con 0x7f38700d34e0 2024-04-25T10:07:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.619+0000 7f3875ca6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f387015dd50 0x7f387015aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.619+0000 7f3875ca6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f387015dd50 0x7f387015aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48120/0 (socket says 172.21.15.16:48120) 2024-04-25T10:07:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.619+0000 7f3875ca6640 1 -- 172.21.15.16:0/540424977 learned_addr learned my addr 172.21.15.16:0/540424977 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.619+0000 7f38754a5640 1 --2- 172.21.15.16:0/540424977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38700b9460 0x7f38700b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.619+0000 7f3874ca4640 1 --2- 172.21.15.16:0/540424977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38700d34e0 0x7f387014ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.620+0000 7f3875ca6640 1 -- 172.21.15.16:0/540424977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38700d34e0 msgr2=0x7f387014ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.620+0000 7f3875ca6640 1 --2- 172.21.15.16:0/540424977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38700d34e0 0x7f387014ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.620+0000 7f3875ca6640 1 -- 172.21.15.16:0/540424977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38700b9460 msgr2=0x7f38700b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.620+0000 7f3875ca6640 1 --2- 172.21.15.16:0/540424977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38700b9460 0x7f38700b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.620+0000 7f3875ca6640 1 -- 172.21.15.16:0/540424977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38700b5f00 con 0x7f387015dd50 2024-04-25T10:07:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.620+0000 7f38754a5640 1 --2- 172.21.15.16:0/540424977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38700b9460 0x7f38700b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.620+0000 7f3874ca4640 1 --2- 172.21.15.16:0/540424977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38700d34e0 0x7f387014ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.620+0000 7f3875ca6640 1 --2- 172.21.15.16:0/540424977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f387015dd50 0x7f387015aae0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3868034040 tx=0x7f3868004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.620+0000 7f3865ffb640 1 -- 172.21.15.16:0/540424977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3868036050 con 0x7f387015dd50 2024-04-25T10:07:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.620+0000 7f3865ffb640 1 -- 172.21.15.16:0/540424977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3868042070 con 0x7f387015dd50 2024-04-25T10:07:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.620+0000 7f3877709640 1 -- 172.21.15.16:0/540424977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38700b61e0 con 0x7f387015dd50 2024-04-25T10:07:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.620+0000 7f3865ffb640 1 -- 172.21.15.16:0/540424977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3868009510 con 0x7f387015dd50 2024-04-25T10:07:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.620+0000 7f3877709640 1 -- 172.21.15.16:0/540424977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387015f5c0 con 0x7f387015dd50 2024-04-25T10:07:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.621+0000 7f3877709640 1 -- 172.21.15.16:0/540424977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3870067850 con 0x7f387015dd50 2024-04-25T10:07:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.621+0000 7f3865ffb640 1 -- 172.21.15.16:0/540424977 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3868007bd0 con 0x7f387015dd50 2024-04-25T10:07:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.622+0000 7f3865ffb640 1 --2- 172.21.15.16:0/540424977 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f385406b500 0x7f385406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.622+0000 7f3865ffb640 1 -- 172.21.15.16:0/540424977 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38680ab000 con 0x7f387015dd50 2024-04-25T10:07:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.622+0000 7f38754a5640 1 --2- 172.21.15.16:0/540424977 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f385406b500 0x7f385406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.623+0000 7f38754a5640 1 --2- 172.21.15.16:0/540424977 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f385406b500 0x7f385406d9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f386c00a830 tx=0x7f386c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.626+0000 7f3865ffb640 1 -- 172.21.15.16:0/540424977 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3868045360 con 0x7f387015dd50 2024-04-25T10:07:36.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.808+0000 7f3877709640 1 -- 172.21.15.16:0/540424977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38700ca860 con 0x7f387015dd50 2024-04-25T10:07:36.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.809+0000 7f3865ffb640 1 -- 172.21.15.16:0/540424977 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f38700ca860 con 0x7f387015dd50 2024-04-25T10:07:36.815 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:36.815 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":47,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:06:52.824898+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:49.462347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:50.153232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.812+0000 7f3877709640 1 -- 172.21.15.16:0/540424977 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f385406b500 msgr2=0x7f385406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.812+0000 7f3877709640 1 --2- 172.21.15.16:0/540424977 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f385406b500 0x7f385406d9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f386c00a830 tx=0x7f386c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.812+0000 7f3877709640 1 -- 172.21.15.16:0/540424977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f387015dd50 msgr2=0x7f387015aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.812+0000 7f3877709640 1 --2- 172.21.15.16:0/540424977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f387015dd50 0x7f387015aae0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3868034040 tx=0x7f3868004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.812+0000 7f3877709640 1 -- 172.21.15.16:0/540424977 shutdown_connections 2024-04-25T10:07:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.812+0000 7f3877709640 1 --2- 172.21.15.16:0/540424977 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f385406b500 0x7f385406d9c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.812+0000 7f3877709640 1 --2- 172.21.15.16:0/540424977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f387015dd50 0x7f387015aae0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.812+0000 7f3877709640 1 --2- 172.21.15.16:0/540424977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38700d34e0 0x7f387014ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.812+0000 7f3877709640 1 --2- 172.21.15.16:0/540424977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38700b9460 0x7f38700b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.812+0000 7f3877709640 1 -- 172.21.15.16:0/540424977 >> 172.21.15.16:0/540424977 conn(0x7f38700722d0 msgr2=0x7f387015b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.812+0000 7f3877709640 1 -- 172.21.15.16:0/540424977 shutdown_connections 2024-04-25T10:07:36.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.812+0000 7f3877709640 1 -- 172.21.15.16:0/540424977 wait complete. 2024-04-25T10:07:36.828 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:49.462347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:50.153232+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:36.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:36.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.927+0000 7f9373fb6640 1 Processor -- start 2024-04-25T10:07:36.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.927+0000 7f9373fb6640 1 -- start start 2024-04-25T10:07:36.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.928+0000 7f9373fb6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f936c0b97c0 0x7f936c0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.928+0000 7f9373fb6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f936c0b5a30 0x7f936c0d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.928+0000 7f9373fb6640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936c0b5e40 con 0x7f936c15dc80 2024-04-25T10:07:36.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.928+0000 7f9373fb6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936c161f80 con 0x7f936c0b5a30 2024-04-25T10:07:36.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.928+0000 7f9373fb6640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936c1627a0 con 0x7f936c0b97c0 2024-04-25T10:07:36.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.928+0000 7f9372553640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f936c0b5a30 0x7f936c0d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.928+0000 7f9372553640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f936c0b5a30 0x7f936c0d2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48136/0 (socket says 172.21.15.16:48136) 2024-04-25T10:07:36.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.928+0000 7f9372553640 1 -- 172.21.15.16:0/1079365979 learned_addr learned my addr 172.21.15.16:0/1079365979 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:36.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.928+0000 7f9371551640 1 --2- 172.21.15.16:0/1079365979 >> v2:172.21.15.186:3300/0 conn(0x7f936c0b97c0 0x7f936c0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.928+0000 7f9372553640 1 -- 172.21.15.16:0/1079365979 >> v2:172.21.15.186:3300/0 conn(0x7f936c0b97c0 msgr2=0x7f936c0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:36.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.928+0000 7f9372553640 1 --2- 172.21.15.16:0/1079365979 >> v2:172.21.15.186:3300/0 conn(0x7f936c0b97c0 0x7f936c0b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.928+0000 7f9372553640 1 -- 172.21.15.16:0/1079365979 >> v1:172.21.15.186:6789/0 conn(0x7f936c15dc80 legacy=0x7f936c161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:36.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.928+0000 7f9372553640 1 -- 172.21.15.16:0/1079365979 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c1621e0 con 0x7f936c0b5a30 2024-04-25T10:07:36.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.929+0000 7f9372553640 1 --2- 172.21.15.16:0/1079365979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936c0b5a30 0x7f936c0d2470 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f936c06ba70 tx=0x7f936402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348f2c5af5161c84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.929+0000 7f9370d50640 1 -- 172.21.15.16:0/1079365979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364036070 con 0x7f936c0b5a30 2024-04-25T10:07:36.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.929+0000 7f9370d50640 1 -- 172.21.15.16:0/1079365979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9364031600 con 0x7f936c0b5a30 2024-04-25T10:07:36.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.929+0000 7f9370d50640 1 -- 172.21.15.16:0/1079365979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93640319d0 con 0x7f936c0b5a30 2024-04-25T10:07:36.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.929+0000 7f9373fb6640 1 -- 172.21.15.16:0/1079365979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936c0b5a30 msgr2=0x7f936c0d2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.929+0000 7f9373fb6640 1 --2- 172.21.15.16:0/1079365979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936c0b5a30 0x7f936c0d2470 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f936c06ba70 tx=0x7f936402a730 comp rx=0 tx=0).stop 2024-04-25T10:07:36.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.930+0000 7f9373fb6640 1 -- 172.21.15.16:0/1079365979 shutdown_connections 2024-04-25T10:07:36.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.930+0000 7f9373fb6640 1 --2- 172.21.15.16:0/1079365979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936c0b5a30 0x7f936c0d2470 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.930+0000 7f9373fb6640 1 --2- 172.21.15.16:0/1079365979 >> v2:172.21.15.186:3300/0 conn(0x7f936c0b97c0 0x7f936c0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.930+0000 7f9373fb6640 1 -- 172.21.15.16:0/1079365979 >> 172.21.15.16:0/1079365979 conn(0x7f936c0d04f0 msgr2=0x7f936c0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:36.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.930+0000 7f9373fb6640 1 -- 172.21.15.16:0/1079365979 shutdown_connections 2024-04-25T10:07:36.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.930+0000 7f9373fb6640 1 -- 172.21.15.16:0/1079365979 wait complete. 2024-04-25T10:07:36.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.931+0000 7f9373fb6640 1 Processor -- start 2024-04-25T10:07:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.931+0000 7f9373fb6640 1 -- start start 2024-04-25T10:07:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9373fb6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936c0b5a30 0x7f936c1e9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9373fb6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f936c0b97c0 0x7f936c158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9373fb6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f936c15dc80 0x7f936c0c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9373fb6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c15e060 con 0x7f936c0b5a30 2024-04-25T10:07:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9373fb6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c0d1c70 con 0x7f936c0b97c0 2024-04-25T10:07:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9373fb6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936c0d29b0 con 0x7f936c15dc80 2024-04-25T10:07:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9371d52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936c0b5a30 0x7f936c1e9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9371d52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936c0b5a30 0x7f936c1e9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48142/0 (socket says 172.21.15.16:48142) 2024-04-25T10:07:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9371d52640 1 -- 172.21.15.16:0/1755442951 learned_addr learned my addr 172.21.15.16:0/1755442951 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9372553640 1 --2- 172.21.15.16:0/1755442951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f936c15dc80 0x7f936c0c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9371551640 1 --2- 172.21.15.16:0/1755442951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f936c0b97c0 0x7f936c158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9371d52640 1 -- 172.21.15.16:0/1755442951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f936c15dc80 msgr2=0x7f936c0c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9371d52640 1 --2- 172.21.15.16:0/1755442951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f936c15dc80 0x7f936c0c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9371d52640 1 -- 172.21.15.16:0/1755442951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f936c0b97c0 msgr2=0x7f936c158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9371d52640 1 --2- 172.21.15.16:0/1755442951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f936c0b97c0 0x7f936c158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9371d52640 1 -- 172.21.15.16:0/1755442951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c0c7a60 con 0x7f936c0b5a30 2024-04-25T10:07:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9372553640 1 --2- 172.21.15.16:0/1755442951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f936c15dc80 0x7f936c0c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9371551640 1 --2- 172.21.15.16:0/1755442951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f936c0b97c0 0x7f936c158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9371d52640 1 --2- 172.21.15.16:0/1755442951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936c0b5a30 0x7f936c1e9ac0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f936800c3c0 tx=0x7f936800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f93627fc640 1 -- 172.21.15.16:0/1755442951 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93680052d0 con 0x7f936c0b5a30 2024-04-25T10:07:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f93627fc640 1 -- 172.21.15.16:0/1755442951 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9368005490 con 0x7f936c0b5a30 2024-04-25T10:07:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9373fb6640 1 -- 172.21.15.16:0/1755442951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936c0c7ce0 con 0x7f936c0b5a30 2024-04-25T10:07:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f93627fc640 1 -- 172.21.15.16:0/1755442951 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368007260 con 0x7f936c0b5a30 2024-04-25T10:07:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.932+0000 7f9373fb6640 1 -- 172.21.15.16:0/1755442951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936c0c82a0 con 0x7f936c0b5a30 2024-04-25T10:07:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.933+0000 7f9373fb6640 1 -- 172.21.15.16:0/1755442951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f936c0750b0 con 0x7f936c0b5a30 2024-04-25T10:07:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.934+0000 7f93627fc640 1 -- 172.21.15.16:0/1755442951 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9368007400 con 0x7f936c0b5a30 2024-04-25T10:07:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.934+0000 7f93627fc640 1 --2- 172.21.15.16:0/1755442951 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f935006b4d0 0x7f935006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.934+0000 7f93627fc640 1 -- 172.21.15.16:0/1755442951 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9368089700 con 0x7f936c0b5a30 2024-04-25T10:07:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.934+0000 7f9371551640 1 --2- 172.21.15.16:0/1755442951 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f935006b4d0 0x7f935006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.935+0000 7f9371551640 1 --2- 172.21.15.16:0/1755442951 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f935006b4d0 0x7f935006d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f936c1f6f00 tx=0x7f935c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:36.938+0000 7f93627fc640 1 -- 172.21.15.16:0/1755442951 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f936c0c7a60 con 0x7f936c0b5a30 2024-04-25T10:07:37.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.120+0000 7f9373fb6640 1 -- 172.21.15.16:0/1755442951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f936c0d1780 con 0x7f936c0b5a30 2024-04-25T10:07:37.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.121+0000 7f93627fc640 1 -- 172.21.15.16:0/1755442951 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f936c0b5e40 con 0x7f936c0b5a30 2024-04-25T10:07:37.130 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:07:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.123+0000 7f9373fb6640 1 -- 172.21.15.16:0/1755442951 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f935006b4d0 msgr2=0x7f935006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.123+0000 7f9373fb6640 1 --2- 172.21.15.16:0/1755442951 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f935006b4d0 0x7f935006d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f936c1f6f00 tx=0x7f935c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.123+0000 7f9373fb6640 1 -- 172.21.15.16:0/1755442951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936c0b5a30 msgr2=0x7f936c1e9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.123+0000 7f9373fb6640 1 --2- 172.21.15.16:0/1755442951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936c0b5a30 0x7f936c1e9ac0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f936800c3c0 tx=0x7f936800c880 comp rx=0 tx=0).stop 2024-04-25T10:07:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.123+0000 7f9373fb6640 1 -- 172.21.15.16:0/1755442951 shutdown_connections 2024-04-25T10:07:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.123+0000 7f9373fb6640 1 --2- 172.21.15.16:0/1755442951 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f935006b4d0 0x7f935006d990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.123+0000 7f9373fb6640 1 --2- 172.21.15.16:0/1755442951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f936c15dc80 0x7f936c0c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.123+0000 7f9373fb6640 1 --2- 172.21.15.16:0/1755442951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f936c0b97c0 0x7f936c158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.123+0000 7f9373fb6640 1 --2- 172.21.15.16:0/1755442951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f936c0b5a30 0x7f936c1e9ac0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.123+0000 7f9373fb6640 1 -- 172.21.15.16:0/1755442951 >> 172.21.15.16:0/1755442951 conn(0x7f936c0d04f0 msgr2=0x7f936c06bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.123+0000 7f9373fb6640 1 -- 172.21.15.16:0/1755442951 shutdown_connections 2024-04-25T10:07:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.123+0000 7f9373fb6640 1 -- 172.21.15.16:0/1755442951 wait complete. 2024-04-25T10:07:37.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:37.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.241+0000 7f7810bce640 1 Processor -- start 2024-04-25T10:07:37.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.241+0000 7f7810bce640 1 -- start start 2024-04-25T10:07:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.241+0000 7f7810bce640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f780c06b2b0 0x7f780c0b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.241+0000 7f7810bce640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f780c0ccee0 0x7f780c0b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.241+0000 7f7810bce640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f780c0cd450 con 0x7f780c15baf0 2024-04-25T10:07:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.241+0000 7f7810bce640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f780c0ce060 con 0x7f780c0ccee0 2024-04-25T10:07:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.241+0000 7f7810bce640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780c0cd730 con 0x7f780c06b2b0 2024-04-25T10:07:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.241+0000 7f780ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f780c0ccee0 0x7f780c0b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.242+0000 7f780ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f780c0ccee0 0x7f780c0b08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48158/0 (socket says 172.21.15.16:48158) 2024-04-25T10:07:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.242+0000 7f780ad9d640 1 -- 172.21.15.16:0/1845086333 learned_addr learned my addr 172.21.15.16:0/1845086333 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.242+0000 7f7803fff640 1 --2- 172.21.15.16:0/1845086333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f780c06b2b0 0x7f780c0b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.242+0000 7f780ad9d640 1 -- 172.21.15.16:0/1845086333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f780c06b2b0 msgr2=0x7f780c0b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.242+0000 7f780ad9d640 1 --2- 172.21.15.16:0/1845086333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f780c06b2b0 0x7f780c0b03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.242+0000 7f780ad9d640 1 -- 172.21.15.16:0/1845086333 >> v1:172.21.15.186:6789/0 conn(0x7f780c15baf0 legacy=0x7f780c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.242+0000 7f780ad9d640 1 -- 172.21.15.16:0/1845086333 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c0cda10 con 0x7f780c0ccee0 2024-04-25T10:07:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.242+0000 7f7803fff640 1 --2- 172.21.15.16:0/1845086333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f780c06b2b0 0x7f780c0b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.243+0000 7f780ad9d640 1 --2- 172.21.15.16:0/1845086333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c0ccee0 0x7f780c0b08f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7804009e50 tx=0x7f780402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508b980d3729f3f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.243+0000 7f7809d9b640 1 -- 172.21.15.16:0/1845086333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804036070 con 0x7f780c0ccee0 2024-04-25T10:07:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.243+0000 7f7809d9b640 1 -- 172.21.15.16:0/1845086333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7804031600 con 0x7f780c0ccee0 2024-04-25T10:07:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.243+0000 7f7809d9b640 1 -- 172.21.15.16:0/1845086333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78040319d0 con 0x7f780c0ccee0 2024-04-25T10:07:37.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.243+0000 7f7810bce640 1 -- 172.21.15.16:0/1845086333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c0ccee0 msgr2=0x7f780c0b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.243+0000 7f7810bce640 1 --2- 172.21.15.16:0/1845086333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c0ccee0 0x7f780c0b08f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7804009e50 tx=0x7f780402a710 comp rx=0 tx=0).stop 2024-04-25T10:07:37.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.243+0000 7f7810bce640 1 -- 172.21.15.16:0/1845086333 shutdown_connections 2024-04-25T10:07:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.243+0000 7f7810bce640 1 --2- 172.21.15.16:0/1845086333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c0ccee0 0x7f780c0b08f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.243+0000 7f7810bce640 1 --2- 172.21.15.16:0/1845086333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f780c06b2b0 0x7f780c0b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.243+0000 7f7810bce640 1 -- 172.21.15.16:0/1845086333 >> 172.21.15.16:0/1845086333 conn(0x7f780c0b2390 msgr2=0x7f780c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.243+0000 7f7810bce640 1 -- 172.21.15.16:0/1845086333 shutdown_connections 2024-04-25T10:07:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.243+0000 7f7810bce640 1 -- 172.21.15.16:0/1845086333 wait complete. 2024-04-25T10:07:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.245+0000 7f7810bce640 1 Processor -- start 2024-04-25T10:07:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.245+0000 7f7810bce640 1 -- start start 2024-04-25T10:07:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.245+0000 7f7810bce640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f780c06b2b0 0x7f780c0b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.245+0000 7f7810bce640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f780c0ccee0 0x7f780c0c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.245+0000 7f7810bce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c15baf0 0x7f780c0c4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.245+0000 7f7810bce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c0c41a0 con 0x7f780c15baf0 2024-04-25T10:07:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.245+0000 7f7810bce640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c0c4020 con 0x7f780c06b2b0 2024-04-25T10:07:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.245+0000 7f7810bce640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780c15bed0 con 0x7f780c0ccee0 2024-04-25T10:07:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.245+0000 7f780ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c15baf0 0x7f780c0c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.246+0000 7f780ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c15baf0 0x7f780c0c4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48166/0 (socket says 172.21.15.16:48166) 2024-04-25T10:07:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.246+0000 7f780ad9d640 1 -- 172.21.15.16:0/1792891394 learned_addr learned my addr 172.21.15.16:0/1792891394 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.246+0000 7f780a59c640 1 --2- 172.21.15.16:0/1792891394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f780c06b2b0 0x7f780c0b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.246+0000 7f7803fff640 1 --2- 172.21.15.16:0/1792891394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f780c0ccee0 0x7f780c0c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.246+0000 7f780ad9d640 1 -- 172.21.15.16:0/1792891394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f780c0ccee0 msgr2=0x7f780c0c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.246+0000 7f780ad9d640 1 --2- 172.21.15.16:0/1792891394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f780c0ccee0 0x7f780c0c5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.246+0000 7f780ad9d640 1 -- 172.21.15.16:0/1792891394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f780c06b2b0 msgr2=0x7f780c0b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.246+0000 7f780ad9d640 1 --2- 172.21.15.16:0/1792891394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f780c06b2b0 0x7f780c0b4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.246+0000 7f780ad9d640 1 -- 172.21.15.16:0/1792891394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c0b4cb0 con 0x7f780c15baf0 2024-04-25T10:07:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.246+0000 7f780ad9d640 1 --2- 172.21.15.16:0/1792891394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c15baf0 0x7f780c0c4760 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7804009f70 tx=0x7f7804004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.246+0000 7f7802ffd640 1 -- 172.21.15.16:0/1792891394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804036050 con 0x7f780c15baf0 2024-04-25T10:07:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.246+0000 7f7802ffd640 1 -- 172.21.15.16:0/1792891394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7804042070 con 0x7f780c15baf0 2024-04-25T10:07:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.246+0000 7f7802ffd640 1 -- 172.21.15.16:0/1792891394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804009510 con 0x7f780c15baf0 2024-04-25T10:07:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.246+0000 7f7810bce640 1 -- 172.21.15.16:0/1792891394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780c0b36d0 con 0x7f780c15baf0 2024-04-25T10:07:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.246+0000 7f7810bce640 1 -- 172.21.15.16:0/1792891394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780c0b3c90 con 0x7f780c15baf0 2024-04-25T10:07:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.247+0000 7f7810bce640 1 -- 172.21.15.16:0/1792891394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780c0750b0 con 0x7f780c15baf0 2024-04-25T10:07:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.248+0000 7f7802ffd640 1 -- 172.21.15.16:0/1792891394 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7804007bb0 con 0x7f780c15baf0 2024-04-25T10:07:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.248+0000 7f7802ffd640 1 --2- 172.21.15.16:0/1792891394 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f77ec06b500 0x7f77ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.248+0000 7f7802ffd640 1 -- 172.21.15.16:0/1792891394 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78040ab0a0 con 0x7f780c15baf0 2024-04-25T10:07:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.248+0000 7f780a59c640 1 --2- 172.21.15.16:0/1792891394 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f77ec06b500 0x7f77ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.249+0000 7f780a59c640 1 --2- 172.21.15.16:0/1792891394 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f77ec06b500 0x7f77ec06d9c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f77fc007fd0 tx=0x7f77fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.252+0000 7f7802ffd640 1 -- 172.21.15.16:0/1792891394 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7804045360 con 0x7f780c15baf0 2024-04-25T10:07:37.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:37.260 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:37.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.435+0000 7f7810bce640 1 -- 172.21.15.16:0/1792891394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f780c06f750 con 0x7f780c15baf0 2024-04-25T10:07:37.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.435+0000 7f7802ffd640 1 -- 172.21.15.16:0/1792891394 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7804079f20 con 0x7f780c15baf0 2024-04-25T10:07:37.441 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:07:37.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.437+0000 7f7810bce640 1 -- 172.21.15.16:0/1792891394 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f77ec06b500 msgr2=0x7f77ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.437+0000 7f7810bce640 1 --2- 172.21.15.16:0/1792891394 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f77ec06b500 0x7f77ec06d9c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f77fc007fd0 tx=0x7f77fc009040 comp rx=0 tx=0).stop 2024-04-25T10:07:37.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.437+0000 7f7810bce640 1 -- 172.21.15.16:0/1792891394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c15baf0 msgr2=0x7f780c0c4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.437+0000 7f7810bce640 1 --2- 172.21.15.16:0/1792891394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c15baf0 0x7f780c0c4760 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7804009f70 tx=0x7f7804004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:37.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.437+0000 7f7810bce640 1 -- 172.21.15.16:0/1792891394 shutdown_connections 2024-04-25T10:07:37.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.437+0000 7f7810bce640 1 --2- 172.21.15.16:0/1792891394 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f77ec06b500 0x7f77ec06d9c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.437+0000 7f7810bce640 1 --2- 172.21.15.16:0/1792891394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c15baf0 0x7f780c0c4760 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.437+0000 7f7810bce640 1 --2- 172.21.15.16:0/1792891394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f780c0ccee0 0x7f780c0c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.437+0000 7f7810bce640 1 --2- 172.21.15.16:0/1792891394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f780c06b2b0 0x7f780c0b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.437+0000 7f7810bce640 1 -- 172.21.15.16:0/1792891394 >> 172.21.15.16:0/1792891394 conn(0x7f780c0b2390 msgr2=0x7f780c06e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.437+0000 7f7810bce640 1 -- 172.21.15.16:0/1792891394 shutdown_connections 2024-04-25T10:07:37.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.437+0000 7f7810bce640 1 -- 172.21.15.16:0/1792891394 wait complete. 2024-04-25T10:07:37.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:37.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.551+0000 7fee13876640 1 Processor -- start 2024-04-25T10:07:37.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.552+0000 7fee13876640 1 -- start start 2024-04-25T10:07:37.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.552+0000 7fee13876640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fee0c0b98b0 0x7fee0c15eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.553+0000 7fee13876640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fee0c0b5580 0x7fee0c0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.553+0000 7fee13876640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee0c15f080 con 0x7fee0c0b24c0 2024-04-25T10:07:37.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.553+0000 7fee13876640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee0c15bb80 con 0x7fee0c0b5580 2024-04-25T10:07:37.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.553+0000 7fee13876640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee0c06c4b0 con 0x7fee0c0b98b0 2024-04-25T10:07:37.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.553+0000 7fee10e11640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fee0c0b5580 0x7fee0c0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.553+0000 7fee10e11640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fee0c0b5580 0x7fee0c0b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48178/0 (socket says 172.21.15.16:48178) 2024-04-25T10:07:37.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.553+0000 7fee10e11640 1 -- 172.21.15.16:0/2413863875 learned_addr learned my addr 172.21.15.16:0/2413863875 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:37.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.553+0000 7fee11612640 1 --2- 172.21.15.16:0/2413863875 >> v2:172.21.15.186:3300/0 conn(0x7fee0c0b98b0 0x7fee0c15eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.553+0000 7fee10e11640 1 -- 172.21.15.16:0/2413863875 >> v2:172.21.15.186:3300/0 conn(0x7fee0c0b98b0 msgr2=0x7fee0c15eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.553+0000 7fee10e11640 1 --2- 172.21.15.16:0/2413863875 >> v2:172.21.15.186:3300/0 conn(0x7fee0c0b98b0 0x7fee0c15eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.553+0000 7fee10e11640 1 -- 172.21.15.16:0/2413863875 >> v1:172.21.15.186:6789/0 conn(0x7fee0c0b24c0 legacy=0x7fee0c0b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.553+0000 7fee10e11640 1 -- 172.21.15.16:0/2413863875 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c06c790 con 0x7fee0c0b5580 2024-04-25T10:07:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.554+0000 7fee10e11640 1 --2- 172.21.15.16:0/2413863875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee0c0b5580 0x7fee0c0b5a30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fee0c0d4b10 tx=0x7fedfc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62cb122578fe1cec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.554+0000 7fee03fff640 1 -- 172.21.15.16:0/2413863875 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc037070 con 0x7fee0c0b5580 2024-04-25T10:07:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.554+0000 7fee03fff640 1 -- 172.21.15.16:0/2413863875 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedfc02ae20 con 0x7fee0c0b5580 2024-04-25T10:07:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.554+0000 7fee03fff640 1 -- 172.21.15.16:0/2413863875 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc032660 con 0x7fee0c0b5580 2024-04-25T10:07:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.554+0000 7fee13876640 1 -- 172.21.15.16:0/2413863875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee0c0b5580 msgr2=0x7fee0c0b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.554+0000 7fee13876640 1 --2- 172.21.15.16:0/2413863875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee0c0b5580 0x7fee0c0b5a30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fee0c0d4b10 tx=0x7fedfc02a110 comp rx=0 tx=0).stop 2024-04-25T10:07:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.554+0000 7fee13876640 1 -- 172.21.15.16:0/2413863875 shutdown_connections 2024-04-25T10:07:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.554+0000 7fee13876640 1 --2- 172.21.15.16:0/2413863875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee0c0b5580 0x7fee0c0b5a30 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.554+0000 7fee13876640 1 --2- 172.21.15.16:0/2413863875 >> v2:172.21.15.186:3300/0 conn(0x7fee0c0b98b0 0x7fee0c15eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.554+0000 7fee13876640 1 -- 172.21.15.16:0/2413863875 >> 172.21.15.16:0/2413863875 conn(0x7fee0c0d0580 msgr2=0x7fee0c0d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.555+0000 7fee13876640 1 -- 172.21.15.16:0/2413863875 shutdown_connections 2024-04-25T10:07:37.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.555+0000 7fee13876640 1 -- 172.21.15.16:0/2413863875 wait complete. 2024-04-25T10:07:37.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.556+0000 7fee13876640 1 Processor -- start 2024-04-25T10:07:37.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.556+0000 7fee13876640 1 -- start start 2024-04-25T10:07:37.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.556+0000 7fee13876640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee0c0b24c0 0x7fee0c1584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.556+0000 7fee13876640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee0c0b5580 0x7fee0c158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.556+0000 7fee13876640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee0c0b88e0 0x7fee0c1570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.556+0000 7fee13876640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c06c140 con 0x7fee0c0b88e0 2024-04-25T10:07:37.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.556+0000 7fee13876640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c06cd50 con 0x7fee0c0b5580 2024-04-25T10:07:37.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.556+0000 7fee13876640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c0cec40 con 0x7fee0c0b24c0 2024-04-25T10:07:37.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.557+0000 7fee11e13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee0c0b88e0 0x7fee0c1570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.557+0000 7fee11e13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee0c0b88e0 0x7fee0c1570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48186/0 (socket says 172.21.15.16:48186) 2024-04-25T10:07:37.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.557+0000 7fee11e13640 1 -- 172.21.15.16:0/962659349 learned_addr learned my addr 172.21.15.16:0/962659349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:37.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.557+0000 7fee11e13640 1 -- 172.21.15.16:0/962659349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee0c0b24c0 msgr2=0x7fee0c1584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.557+0000 7fee11e13640 1 --2- 172.21.15.16:0/962659349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee0c0b24c0 0x7fee0c1584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.557+0000 7fee11e13640 1 -- 172.21.15.16:0/962659349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee0c0b5580 msgr2=0x7fee0c158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.557+0000 7fee11e13640 1 --2- 172.21.15.16:0/962659349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee0c0b5580 0x7fee0c158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.557+0000 7fee11e13640 1 -- 172.21.15.16:0/962659349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c15f420 con 0x7fee0c0b88e0 2024-04-25T10:07:37.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.557+0000 7fee11e13640 1 --2- 172.21.15.16:0/962659349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee0c0b88e0 0x7fee0c1570f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fee0400c7b0 tx=0x7fee0400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.557+0000 7fee01ffb640 1 -- 172.21.15.16:0/962659349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee04004b60 con 0x7fee0c0b88e0 2024-04-25T10:07:37.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.557+0000 7fee01ffb640 1 -- 172.21.15.16:0/962659349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee04004d00 con 0x7fee0c0b88e0 2024-04-25T10:07:37.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.557+0000 7fee13876640 1 -- 172.21.15.16:0/962659349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee0c15f700 con 0x7fee0c0b88e0 2024-04-25T10:07:37.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.557+0000 7fee01ffb640 1 -- 172.21.15.16:0/962659349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0400aa30 con 0x7fee0c0b88e0 2024-04-25T10:07:37.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.557+0000 7fee13876640 1 -- 172.21.15.16:0/962659349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0c15fcc0 con 0x7fee0c0b88e0 2024-04-25T10:07:37.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.558+0000 7fee13876640 1 -- 172.21.15.16:0/962659349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0c0750b0 con 0x7fee0c0b88e0 2024-04-25T10:07:37.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.559+0000 7fee01ffb640 1 -- 172.21.15.16:0/962659349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee0400abd0 con 0x7fee0c0b88e0 2024-04-25T10:07:37.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.559+0000 7fee01ffb640 1 --2- 172.21.15.16:0/962659349 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fedf006b500 0x7fedf006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.559+0000 7fee01ffb640 1 -- 172.21.15.16:0/962659349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee0408a420 con 0x7fee0c0b88e0 2024-04-25T10:07:37.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.559+0000 7fee11612640 1 --2- 172.21.15.16:0/962659349 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fedf006b500 0x7fedf006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.560+0000 7fee11612640 1 --2- 172.21.15.16:0/962659349 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fedf006b500 0x7fedf006d9c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fee08006fd0 tx=0x7fee08008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.563+0000 7fee01ffb640 1 -- 172.21.15.16:0/962659349 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee04020070 con 0x7fee0c0b88e0 2024-04-25T10:07:37.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.742+0000 7fee13876640 1 -- 172.21.15.16:0/962659349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fee0c06b080 con 0x7fee0c0b88e0 2024-04-25T10:07:37.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.742+0000 7fee01ffb640 1 -- 172.21.15.16:0/962659349 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7fee04059330 con 0x7fee0c0b88e0 2024-04-25T10:07:37.748 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:07:37.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.744+0000 7fee13876640 1 -- 172.21.15.16:0/962659349 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fedf006b500 msgr2=0x7fedf006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.744+0000 7fee13876640 1 --2- 172.21.15.16:0/962659349 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fedf006b500 0x7fedf006d9c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fee08006fd0 tx=0x7fee08008040 comp rx=0 tx=0).stop 2024-04-25T10:07:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.744+0000 7fee13876640 1 -- 172.21.15.16:0/962659349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee0c0b88e0 msgr2=0x7fee0c1570f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.744+0000 7fee13876640 1 --2- 172.21.15.16:0/962659349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee0c0b88e0 0x7fee0c1570f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fee0400c7b0 tx=0x7fee0400cc70 comp rx=0 tx=0).stop 2024-04-25T10:07:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.744+0000 7fee13876640 1 -- 172.21.15.16:0/962659349 shutdown_connections 2024-04-25T10:07:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.744+0000 7fee13876640 1 --2- 172.21.15.16:0/962659349 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fedf006b500 0x7fedf006d9c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.744+0000 7fee13876640 1 --2- 172.21.15.16:0/962659349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee0c0b88e0 0x7fee0c1570f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.744+0000 7fee13876640 1 --2- 172.21.15.16:0/962659349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee0c0b5580 0x7fee0c158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.744+0000 7fee13876640 1 --2- 172.21.15.16:0/962659349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee0c0b24c0 0x7fee0c1584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.744+0000 7fee13876640 1 -- 172.21.15.16:0/962659349 >> 172.21.15.16:0/962659349 conn(0x7fee0c0d0580 msgr2=0x7fee0c0b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.745+0000 7fee13876640 1 -- 172.21.15.16:0/962659349 shutdown_connections 2024-04-25T10:07:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.745+0000 7fee13876640 1 -- 172.21.15.16:0/962659349 wait complete. 2024-04-25T10:07:37.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:37.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.856+0000 7f787e6e4640 1 Processor -- start 2024-04-25T10:07:37.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.856+0000 7f787e6e4640 1 -- start start 2024-04-25T10:07:37.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.856+0000 7f787e6e4640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f787815ba10 0x7f7878099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.856+0000 7f787e6e4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787806c900 con 0x7f787806c3c0 2024-04-25T10:07:37.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.856+0000 7f787e6e4640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787806cbe0 con 0x7f7878099f80 2024-04-25T10:07:37.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.856+0000 7f787e6e4640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78780cbe50 con 0x7f787815ba10 2024-04-25T10:07:37.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f787cc81640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f787806c3c0 0x7f78780cd220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41834/0 (socket says 172.21.15.16:41834) 2024-04-25T10:07:37.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f787cc81640 1 -- 172.21.15.16:0/2783947256 learned_addr learned my addr 172.21.15.16:0/2783947256 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:37.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f7877fff640 1 --2- 172.21.15.16:0/2783947256 >> v2:172.21.15.186:3300/0 conn(0x7f787815ba10 0x7f7878099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f7876ffd640 1 -- 172.21.15.16:0/2783947256 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323543616 0 0) 0x7f787806c900 con 0x7f787806c3c0 2024-04-25T10:07:37.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f7876ffd640 1 -- 172.21.15.16:0/2783947256 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7860003520 con 0x7f787806c3c0 2024-04-25T10:07:37.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f7876ffd640 1 -- 172.21.15.16:0/2783947256 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2129656221 0 0) 0x7f7860003520 con 0x7f787806c3c0 2024-04-25T10:07:37.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f7876ffd640 1 -- 172.21.15.16:0/2783947256 >> v2:172.21.15.186:3300/0 conn(0x7f787815ba10 msgr2=0x7f7878099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f7876ffd640 1 --2- 172.21.15.16:0/2783947256 >> v2:172.21.15.186:3300/0 conn(0x7f787815ba10 0x7f7878099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f7876ffd640 1 -- 172.21.15.16:0/2783947256 >> v1:172.21.15.186:6789/0 conn(0x7f7878099f80 legacy=0x7f787806bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f7876ffd640 1 -- 172.21.15.16:0/2783947256 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787806cec0 con 0x7f787806c3c0 2024-04-25T10:07:37.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f7876ffd640 1 -- 172.21.15.16:0/2783947256 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f78700019f0 con 0x7f787806c3c0 2024-04-25T10:07:37.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f7876ffd640 1 -- 172.21.15.16:0/2783947256 >> v1:172.21.15.16:6789/0 conn(0x7f787806c3c0 legacy=0x7f78780cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f7876ffd640 1 --2- 172.21.15.16:0/2783947256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78600039b0 0x7f7860003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f7876ffd640 1 --2- 172.21.15.16:0/2783947256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860004800 0x7f7860004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f7876ffd640 1 --2- 172.21.15.16:0/2783947256 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78600051c0 0x7f786000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f7876ffd640 1 -- 172.21.15.16:0/2783947256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786000c030 con 0x7f7860004800 2024-04-25T10:07:37.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f7876ffd640 1 -- 172.21.15.16:0/2783947256 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786000c1e0 con 0x7f78600039b0 2024-04-25T10:07:37.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f7876ffd640 1 -- 172.21.15.16:0/2783947256 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786000c3e0 con 0x7f78600051c0 2024-04-25T10:07:37.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f78777fe640 1 --2- 172.21.15.16:0/2783947256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860004800 0x7f7860004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.857+0000 7f7877fff640 1 --2- 172.21.15.16:0/2783947256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78600039b0 0x7f7860003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.858+0000 7f787cc81640 1 --2- 172.21.15.16:0/2783947256 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78600051c0 0x7f786000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.858+0000 7f78777fe640 1 -- 172.21.15.16:0/2783947256 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78600051c0 msgr2=0x7f786000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.858+0000 7f78777fe640 1 --2- 172.21.15.16:0/2783947256 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78600051c0 0x7f786000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.858+0000 7f78777fe640 1 -- 172.21.15.16:0/2783947256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78600039b0 msgr2=0x7f7860003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.858+0000 7f78777fe640 1 --2- 172.21.15.16:0/2783947256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78600039b0 0x7f7860003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.858+0000 7f78777fe640 1 -- 172.21.15.16:0/2783947256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7860003520 con 0x7f7860004800 2024-04-25T10:07:37.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.858+0000 7f787cc81640 1 --2- 172.21.15.16:0/2783947256 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78600051c0 0x7f786000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:37.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.858+0000 7f787cc81640 1 -- 172.21.15.16:0/2783947256 reap_dead start 2024-04-25T10:07:37.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.858+0000 7f78777fe640 1 --2- 172.21.15.16:0/2783947256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860004800 0x7f7860004c80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f786c001750 tx=0x7f7860005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2d0ad41d92489e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.858+0000 7f7876ffd640 1 -- 172.21.15.16:0/2783947256 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c001370 con 0x7f7860004800 2024-04-25T10:07:37.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.858+0000 7f7876ffd640 1 -- 172.21.15.16:0/2783947256 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c0052d0 con 0x7f7860004800 2024-04-25T10:07:37.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.859+0000 7f7876ffd640 1 -- 172.21.15.16:0/2783947256 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c02abc0 con 0x7f7860004800 2024-04-25T10:07:37.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.859+0000 7f787e6e4640 1 -- 172.21.15.16:0/2783947256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860004800 msgr2=0x7f7860004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.859+0000 7f787e6e4640 1 --2- 172.21.15.16:0/2783947256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860004800 0x7f7860004c80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f786c001750 tx=0x7f7860005830 comp rx=0 tx=0).stop 2024-04-25T10:07:37.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.859+0000 7f787e6e4640 1 -- 172.21.15.16:0/2783947256 shutdown_connections 2024-04-25T10:07:37.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.859+0000 7f787e6e4640 1 --2- 172.21.15.16:0/2783947256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860004800 0x7f7860004c80 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.859+0000 7f787e6e4640 1 -- 172.21.15.16:0/2783947256 >> 172.21.15.16:0/2783947256 conn(0x7f78780b32f0 msgr2=0x7f78780b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.859+0000 7f787e6e4640 1 -- 172.21.15.16:0/2783947256 shutdown_connections 2024-04-25T10:07:37.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.859+0000 7f787e6e4640 1 -- 172.21.15.16:0/2783947256 wait complete. 2024-04-25T10:07:37.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.860+0000 7f787e6e4640 1 Processor -- start 2024-04-25T10:07:37.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.860+0000 7f787e6e4640 1 -- start start 2024-04-25T10:07:37.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.861+0000 7f787e6e4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7860004800 0x7f78781f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.861+0000 7f787e6e4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78780c5330 0x7f78780c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.861+0000 7f787e6e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781f48b0 0x7f78781f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.861+0000 7f787e6e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7878099dd0 con 0x7f78781f48b0 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.861+0000 7f787e6e4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7878099c50 con 0x7f7860004800 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.861+0000 7f787e6e4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787809a3d0 con 0x7f78780c5330 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.861+0000 7f787cc81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781f48b0 0x7f78781f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.861+0000 7f787cc81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781f48b0 0x7f78781f4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48214/0 (socket says 172.21.15.16:48214) 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.861+0000 7f787cc81640 1 -- 172.21.15.16:0/690281489 learned_addr learned my addr 172.21.15.16:0/690281489 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.861+0000 7f7877fff640 1 --2- 172.21.15.16:0/690281489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7860004800 0x7f78781f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.861+0000 7f78777fe640 1 --2- 172.21.15.16:0/690281489 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78780c5330 0x7f78780c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.862+0000 7f787cc81640 1 -- 172.21.15.16:0/690281489 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78780c5330 msgr2=0x7f78780c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.862+0000 7f787cc81640 1 --2- 172.21.15.16:0/690281489 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78780c5330 0x7f78780c57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.862+0000 7f787cc81640 1 -- 172.21.15.16:0/690281489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7860004800 msgr2=0x7f78781f7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.862+0000 7f787cc81640 1 --2- 172.21.15.16:0/690281489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7860004800 0x7f78781f7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.862+0000 7f787cc81640 1 -- 172.21.15.16:0/690281489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78780c7130 con 0x7f78781f48b0 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.862+0000 7f787cc81640 1 --2- 172.21.15.16:0/690281489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781f48b0 0x7f78781f4d30 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f787806c830 tx=0x7f787000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.862+0000 7f7874ff9640 1 -- 172.21.15.16:0/690281489 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787000ebc0 con 0x7f78781f48b0 2024-04-25T10:07:37.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.862+0000 7f7874ff9640 1 -- 172.21.15.16:0/690281489 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787000ed60 con 0x7f78781f48b0 2024-04-25T10:07:37.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.862+0000 7f787e6e4640 1 -- 172.21.15.16:0/690281489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78780c73b0 con 0x7f78781f48b0 2024-04-25T10:07:37.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.862+0000 7f7874ff9640 1 -- 172.21.15.16:0/690281489 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870014650 con 0x7f78781f48b0 2024-04-25T10:07:37.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.862+0000 7f787e6e4640 1 -- 172.21.15.16:0/690281489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78780c7970 con 0x7f78781f48b0 2024-04-25T10:07:37.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.863+0000 7f787e6e4640 1 -- 172.21.15.16:0/690281489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78780749b0 con 0x7f78781f48b0 2024-04-25T10:07:37.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.863+0000 7f7874ff9640 1 -- 172.21.15.16:0/690281489 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78700147f0 con 0x7f78781f48b0 2024-04-25T10:07:37.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.864+0000 7f7874ff9640 1 --2- 172.21.15.16:0/690281489 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f786406b500 0x7f786406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.864+0000 7f7874ff9640 1 -- 172.21.15.16:0/690281489 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7870089fe0 con 0x7f78781f48b0 2024-04-25T10:07:37.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.864+0000 7f7877fff640 1 --2- 172.21.15.16:0/690281489 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f786406b500 0x7f786406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.865+0000 7f7877fff640 1 --2- 172.21.15.16:0/690281489 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f786406b500 0x7f786406d9c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7868009b60 tx=0x7f7868008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:37.868+0000 7f7874ff9640 1 -- 172.21.15.16:0/690281489 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78780c7130 con 0x7f78781f48b0 2024-04-25T10:07:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.087+0000 7f787e6e4640 1 -- 172.21.15.16:0/690281489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f787809a570 con 0x7f78781f48b0 2024-04-25T10:07:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.087+0000 7f7874ff9640 1 -- 172.21.15.16:0/690281489 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f7870058de0 con 0x7f78781f48b0 2024-04-25T10:07:38.093 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:38.093 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/791079479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":791079479},{"type":"v1","addr":"172.21.15.16:6835","nonce":791079479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1747921236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1747921236},{"type":"v1","addr":"172.21.15.16:6834","nonce":1747921236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":34880,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/3183779257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3183779257},{"type":"v1","addr":"172.21.15.186:6836","nonce":3183779257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:51.182691+0000","modified":"2024-04-25T10:06:53.179194+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34883},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34883":{"gid":34883,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/83423489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":83423489},{"type":"v1","addr":"172.21.15.186:6837","nonce":83423489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:38.094 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 21 2024-04-25T10:07:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.089+0000 7f787e6e4640 1 -- 172.21.15.16:0/690281489 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f786406b500 msgr2=0x7f786406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.089+0000 7f787e6e4640 1 --2- 172.21.15.16:0/690281489 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f786406b500 0x7f786406d9c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7868009b60 tx=0x7f7868008040 comp rx=0 tx=0).stop 2024-04-25T10:07:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.089+0000 7f787e6e4640 1 -- 172.21.15.16:0/690281489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781f48b0 msgr2=0x7f78781f4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.089+0000 7f787e6e4640 1 --2- 172.21.15.16:0/690281489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781f48b0 0x7f78781f4d30 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f787806c830 tx=0x7f787000e530 comp rx=0 tx=0).stop 2024-04-25T10:07:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.089+0000 7f787e6e4640 1 -- 172.21.15.16:0/690281489 shutdown_connections 2024-04-25T10:07:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.089+0000 7f787e6e4640 1 --2- 172.21.15.16:0/690281489 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f786406b500 0x7f786406d9c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.089+0000 7f787e6e4640 1 --2- 172.21.15.16:0/690281489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781f48b0 0x7f78781f4d30 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.089+0000 7f787e6e4640 1 --2- 172.21.15.16:0/690281489 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78780c5330 0x7f78780c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.089+0000 7f787e6e4640 1 --2- 172.21.15.16:0/690281489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7860004800 0x7f78781f7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.089+0000 7f787e6e4640 1 -- 172.21.15.16:0/690281489 >> 172.21.15.16:0/690281489 conn(0x7f78780b32f0 msgr2=0x7f78780afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.089+0000 7f787e6e4640 1 -- 172.21.15.16:0/690281489 shutdown_connections 2024-04-25T10:07:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.089+0000 7f787e6e4640 1 -- 172.21.15.16:0/690281489 wait complete. 2024-04-25T10:07:38.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:38.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.200+0000 7ff273c1b640 1 Processor -- start 2024-04-25T10:07:38.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.201+0000 7ff273c1b640 1 -- start start 2024-04-25T10:07:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.201+0000 7ff273c1b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff26c15dc30 0x7ff26c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.201+0000 7ff273c1b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26c06c1f0 con 0x7ff26c0b1b30 2024-04-25T10:07:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.201+0000 7ff273c1b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26c0cfb40 con 0x7ff26c06cc00 2024-04-25T10:07:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.201+0000 7ff273c1b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff26c0b2060 con 0x7ff26c15dc30 2024-04-25T10:07:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2721b8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff26c0b1b30 0x7ff26c0d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41842/0 (socket says 172.21.15.16:41842) 2024-04-25T10:07:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2721b8640 1 -- 172.21.15.16:0/1492681547 learned_addr learned my addr 172.21.15.16:0/1492681547 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2709b5640 1 -- 172.21.15.16:0/1492681547 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419858909 0 0) 0x7ff264001490 con 0x7ff26c0b1b30 2024-04-25T10:07:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2709b5640 1 -- 172.21.15.16:0/1492681547 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff258003520 con 0x7ff26c0b1b30 2024-04-25T10:07:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2719b7640 1 --2- 172.21.15.16:0/1492681547 >> v2:172.21.15.186:3300/0 conn(0x7ff26c15dc30 0x7ff26c06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2709b5640 1 -- 172.21.15.16:0/1492681547 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 383172507 0 0) 0x7ff258003520 con 0x7ff26c0b1b30 2024-04-25T10:07:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2709b5640 1 -- 172.21.15.16:0/1492681547 >> v2:172.21.15.186:3300/0 conn(0x7ff26c15dc30 msgr2=0x7ff26c06c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2709b5640 1 --2- 172.21.15.16:0/1492681547 >> v2:172.21.15.186:3300/0 conn(0x7ff26c15dc30 0x7ff26c06c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2709b5640 1 -- 172.21.15.16:0/1492681547 >> v1:172.21.15.186:6789/0 conn(0x7ff26c06cc00 legacy=0x7ff26c0b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2709b5640 1 -- 172.21.15.16:0/1492681547 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c0b2340 con 0x7ff26c0b1b30 2024-04-25T10:07:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2709b5640 1 -- 172.21.15.16:0/1492681547 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7ff264001bc0 con 0x7ff26c0b1b30 2024-04-25T10:07:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2719b7640 1 --2- 172.21.15.16:0/1492681547 >> v2:172.21.15.186:3300/0 conn(0x7ff26c15dc30 0x7ff26c06c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2709b5640 1 -- 172.21.15.16:0/1492681547 >> v1:172.21.15.16:6789/0 conn(0x7ff26c0b1b30 legacy=0x7ff26c0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2709b5640 1 --2- 172.21.15.16:0/1492681547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2580039b0 0x7ff258003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2709b5640 1 --2- 172.21.15.16:0/1492681547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff258004800 0x7ff258004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2709b5640 1 --2- 172.21.15.16:0/1492681547 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2580051c0 0x7ff25800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2709b5640 1 -- 172.21.15.16:0/1492681547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c0b2060 con 0x7ff258004800 2024-04-25T10:07:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2709b5640 1 -- 172.21.15.16:0/1492681547 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25800c0f0 con 0x7ff2580039b0 2024-04-25T10:07:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2709b5640 1 -- 172.21.15.16:0/1492681547 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25800c2f0 con 0x7ff2580051c0 2024-04-25T10:07:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.202+0000 7ff2711b6640 1 --2- 172.21.15.16:0/1492681547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff258004800 0x7ff258004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.203+0000 7ff2719b7640 1 --2- 172.21.15.16:0/1492681547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2580039b0 0x7ff258003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.203+0000 7ff2711b6640 1 -- 172.21.15.16:0/1492681547 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2580051c0 msgr2=0x7ff25800b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:07:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.203+0000 7ff2711b6640 1 --2- 172.21.15.16:0/1492681547 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2580051c0 0x7ff25800b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.203+0000 7ff2711b6640 1 -- 172.21.15.16:0/1492681547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2580039b0 msgr2=0x7ff258003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.203+0000 7ff2711b6640 1 --2- 172.21.15.16:0/1492681547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2580039b0 0x7ff258003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.203+0000 7ff2711b6640 1 -- 172.21.15.16:0/1492681547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff258003520 con 0x7ff258004800 2024-04-25T10:07:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.203+0000 7ff2721b8640 1 -- 172.21.15.16:0/1492681547 reap_dead start 2024-04-25T10:07:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.203+0000 7ff2711b6640 1 --2- 172.21.15.16:0/1492681547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff258004800 0x7ff258004c80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff25c001620 tx=0x7ff258005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13058f6d58393e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.204+0000 7ff2709b5640 1 -- 172.21.15.16:0/1492681547 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c001220 con 0x7ff258004800 2024-04-25T10:07:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.204+0000 7ff2709b5640 1 -- 172.21.15.16:0/1492681547 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c02e070 con 0x7ff258004800 2024-04-25T10:07:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.204+0000 7ff2709b5640 1 -- 172.21.15.16:0/1492681547 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c02a770 con 0x7ff258004800 2024-04-25T10:07:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.204+0000 7ff273c1b640 1 -- 172.21.15.16:0/1492681547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff258004800 msgr2=0x7ff258004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.204+0000 7ff273c1b640 1 --2- 172.21.15.16:0/1492681547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff258004800 0x7ff258004c80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff25c001620 tx=0x7ff258005830 comp rx=0 tx=0).stop 2024-04-25T10:07:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.204+0000 7ff273c1b640 1 -- 172.21.15.16:0/1492681547 shutdown_connections 2024-04-25T10:07:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.204+0000 7ff273c1b640 1 --2- 172.21.15.16:0/1492681547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff258004800 0x7ff258004c80 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.204+0000 7ff273c1b640 1 -- 172.21.15.16:0/1492681547 >> 172.21.15.16:0/1492681547 conn(0x7ff26c0d1900 msgr2=0x7ff26c0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.204+0000 7ff273c1b640 1 -- 172.21.15.16:0/1492681547 shutdown_connections 2024-04-25T10:07:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.204+0000 7ff273c1b640 1 -- 172.21.15.16:0/1492681547 wait complete. 2024-04-25T10:07:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.205+0000 7ff273c1b640 1 Processor -- start 2024-04-25T10:07:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.206+0000 7ff273c1b640 1 -- start start 2024-04-25T10:07:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.206+0000 7ff273c1b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff258004800 0x7ff26c0c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.206+0000 7ff273c1b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff26c1f45b0 0x7ff26c0c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.206+0000 7ff273c1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c15ef50 0x7ff26c1539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.206+0000 7ff273c1b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c15e010 con 0x7ff26c15ef50 2024-04-25T10:07:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.206+0000 7ff273c1b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c0cfde0 con 0x7ff26c1f45b0 2024-04-25T10:07:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.206+0000 7ff273c1b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c06c6c0 con 0x7ff258004800 2024-04-25T10:07:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.206+0000 7ff2721b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c15ef50 0x7ff26c1539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.206+0000 7ff2721b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c15ef50 0x7ff26c1539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48238/0 (socket says 172.21.15.16:48238) 2024-04-25T10:07:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.206+0000 7ff2721b8640 1 -- 172.21.15.16:0/3745866447 learned_addr learned my addr 172.21.15.16:0/3745866447 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.206+0000 7ff2711b6640 1 --2- 172.21.15.16:0/3745866447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff26c1f45b0 0x7ff26c0c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.206+0000 7ff2719b7640 1 --2- 172.21.15.16:0/3745866447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff258004800 0x7ff26c0c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.207+0000 7ff2721b8640 1 -- 172.21.15.16:0/3745866447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff258004800 msgr2=0x7ff26c0c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.207+0000 7ff2721b8640 1 --2- 172.21.15.16:0/3745866447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff258004800 0x7ff26c0c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.207+0000 7ff2721b8640 1 -- 172.21.15.16:0/3745866447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff26c1f45b0 msgr2=0x7ff26c0c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.207+0000 7ff2721b8640 1 --2- 172.21.15.16:0/3745866447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff26c1f45b0 0x7ff26c0c5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.207+0000 7ff2721b8640 1 -- 172.21.15.16:0/3745866447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c0c9ed0 con 0x7ff26c15ef50 2024-04-25T10:07:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.207+0000 7ff2719b7640 1 --2- 172.21.15.16:0/3745866447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff258004800 0x7ff26c0c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.207+0000 7ff2711b6640 1 --2- 172.21.15.16:0/3745866447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff26c1f45b0 0x7ff26c0c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.207+0000 7ff2721b8640 1 --2- 172.21.15.16:0/3745866447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c15ef50 0x7ff26c1539b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff26c06d210 tx=0x7ff26400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.207+0000 7ff2627fc640 1 -- 172.21.15.16:0/3745866447 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff264004d60 con 0x7ff26c15ef50 2024-04-25T10:07:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.207+0000 7ff2627fc640 1 -- 172.21.15.16:0/3745866447 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26400a5b0 con 0x7ff26c15ef50 2024-04-25T10:07:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.207+0000 7ff273c1b640 1 -- 172.21.15.16:0/3745866447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26c0ca1b0 con 0x7ff26c15ef50 2024-04-25T10:07:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.207+0000 7ff2627fc640 1 -- 172.21.15.16:0/3745866447 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26400a9a0 con 0x7ff26c15ef50 2024-04-25T10:07:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.207+0000 7ff273c1b640 1 -- 172.21.15.16:0/3745866447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26c0c7f50 con 0x7ff26c15ef50 2024-04-25T10:07:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.208+0000 7ff273c1b640 1 -- 172.21.15.16:0/3745866447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26c067850 con 0x7ff26c15ef50 2024-04-25T10:07:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.208+0000 7ff2627fc640 1 -- 172.21.15.16:0/3745866447 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff26400ab40 con 0x7ff26c15ef50 2024-04-25T10:07:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.209+0000 7ff2627fc640 1 --2- 172.21.15.16:0/3745866447 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff25006b4d0 0x7ff25006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.209+0000 7ff2627fc640 1 -- 172.21.15.16:0/3745866447 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff264089920 con 0x7ff26c15ef50 2024-04-25T10:07:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.209+0000 7ff2719b7640 1 --2- 172.21.15.16:0/3745866447 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff25006b4d0 0x7ff25006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.209+0000 7ff2719b7640 1 --2- 172.21.15.16:0/3745866447 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff25006b4d0 0x7ff25006d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff268006fd0 tx=0x7ff268008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.213+0000 7ff2627fc640 1 -- 172.21.15.16:0/3745866447 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff26c06c1f0 con 0x7ff26c15ef50 2024-04-25T10:07:38.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.393+0000 7ff273c1b640 1 -- 172.21.15.16:0/3745866447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff26c0cdf40 con 0x7ff26c15ef50 2024-04-25T10:07:38.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.394+0000 7ff2627fc640 1 -- 172.21.15.16:0/3745866447 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16609 (secure 0 0 0) 0x7ff2640587a0 con 0x7ff26c15ef50 2024-04-25T10:07:38.400 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:38.400 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":47,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:06:52.824898+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:49.462347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:50.153232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.396+0000 7ff273c1b640 1 -- 172.21.15.16:0/3745866447 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff25006b4d0 msgr2=0x7ff25006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.396+0000 7ff273c1b640 1 --2- 172.21.15.16:0/3745866447 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff25006b4d0 0x7ff25006d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff268006fd0 tx=0x7ff268008040 comp rx=0 tx=0).stop 2024-04-25T10:07:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.396+0000 7ff273c1b640 1 -- 172.21.15.16:0/3745866447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c15ef50 msgr2=0x7ff26c1539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.396+0000 7ff273c1b640 1 --2- 172.21.15.16:0/3745866447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c15ef50 0x7ff26c1539b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff26c06d210 tx=0x7ff26400cb80 comp rx=0 tx=0).stop 2024-04-25T10:07:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.396+0000 7ff273c1b640 1 -- 172.21.15.16:0/3745866447 shutdown_connections 2024-04-25T10:07:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.396+0000 7ff273c1b640 1 --2- 172.21.15.16:0/3745866447 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff25006b4d0 0x7ff25006d990 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.396+0000 7ff273c1b640 1 --2- 172.21.15.16:0/3745866447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c15ef50 0x7ff26c1539b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.396+0000 7ff273c1b640 1 --2- 172.21.15.16:0/3745866447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff26c1f45b0 0x7ff26c0c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.396+0000 7ff273c1b640 1 --2- 172.21.15.16:0/3745866447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff258004800 0x7ff26c0c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.396+0000 7ff273c1b640 1 -- 172.21.15.16:0/3745866447 >> 172.21.15.16:0/3745866447 conn(0x7ff26c0d1900 msgr2=0x7ff26c0ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.396+0000 7ff273c1b640 1 -- 172.21.15.16:0/3745866447 shutdown_connections 2024-04-25T10:07:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.396+0000 7ff273c1b640 1 -- 172.21.15.16:0/3745866447 wait complete. 2024-04-25T10:07:38.413 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:07:38.413 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.508+0000 7f1c0fb7e640 1 Processor -- start 2024-04-25T10:07:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.508+0000 7f1c0fb7e640 1 -- start start 2024-04-25T10:07:38.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0fb7e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1c0815dce0 0x7f1c080d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0fb7e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c080d35b0 0x7f1c080b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0fb7e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c0815e0c0 con 0x7f1c080d1f90 2024-04-25T10:07:38.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0fb7e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c080b7b10 con 0x7f1c0815dce0 2024-04-25T10:07:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0fb7e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c080d2510 con 0x7f1c080d35b0 2024-04-25T10:07:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0e11b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1c080d1f90 0x7f1c0806bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41854/0 (socket says 172.21.15.16:41854) 2024-04-25T10:07:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0e11b640 1 -- 172.21.15.16:0/3829804680 learned_addr learned my addr 172.21.15.16:0/3829804680 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0c918640 1 -- 172.21.15.16:0/3829804680 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463484077 0 0) 0x7f1c0815e0c0 con 0x7f1c080d1f90 2024-04-25T10:07:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0c918640 1 -- 172.21.15.16:0/3829804680 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bf4003520 con 0x7f1c080d1f90 2024-04-25T10:07:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0c918640 1 -- 172.21.15.16:0/3829804680 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4119960488 0 0) 0x7f1bf4003520 con 0x7f1c080d1f90 2024-04-25T10:07:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0c918640 1 -- 172.21.15.16:0/3829804680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c080d35b0 msgr2=0x7f1c080b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0c918640 1 --2- 172.21.15.16:0/3829804680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c080d35b0 0x7f1c080b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0c918640 1 -- 172.21.15.16:0/3829804680 >> v2:172.21.15.186:3300/0 conn(0x7f1c0815dce0 msgr2=0x7f1c080d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0c918640 1 --2- 172.21.15.16:0/3829804680 >> v2:172.21.15.186:3300/0 conn(0x7f1c0815dce0 0x7f1c080d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0c918640 1 -- 172.21.15.16:0/3829804680 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c080d27f0 con 0x7f1c080d1f90 2024-04-25T10:07:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0c918640 1 -- 172.21.15.16:0/3829804680 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f1c000019f0 con 0x7f1c080d1f90 2024-04-25T10:07:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0c918640 1 -- 172.21.15.16:0/3829804680 >> v1:172.21.15.16:6789/0 conn(0x7f1c080d1f90 legacy=0x7f1c0806bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0c918640 1 --2- 172.21.15.16:0/3829804680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bf40039b0 0x7f1bf4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0c918640 1 --2- 172.21.15.16:0/3829804680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bf4004800 0x7f1bf4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0c918640 1 --2- 172.21.15.16:0/3829804680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c080d35b0 0x7f1bf400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0c918640 1 -- 172.21.15.16:0/3829804680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c080b7b10 con 0x7f1bf4004800 2024-04-25T10:07:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0c918640 1 -- 172.21.15.16:0/3829804680 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c080d2510 con 0x7f1bf40039b0 2024-04-25T10:07:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.509+0000 7f1c0c918640 1 -- 172.21.15.16:0/3829804680 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf40055d0 con 0x7f1c080d35b0 2024-04-25T10:07:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.510+0000 7f1c0d119640 1 --2- 172.21.15.16:0/3829804680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bf4004800 0x7f1bf4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.510+0000 7f1c0e11b640 1 --2- 172.21.15.16:0/3829804680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c080d35b0 0x7f1bf400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.510+0000 7f1c0d91a640 1 --2- 172.21.15.16:0/3829804680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bf40039b0 0x7f1bf4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.510+0000 7f1c0d119640 1 -- 172.21.15.16:0/3829804680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c080d35b0 msgr2=0x7f1bf400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.510+0000 7f1c0d119640 1 --2- 172.21.15.16:0/3829804680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c080d35b0 0x7f1bf400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.510+0000 7f1c0d119640 1 -- 172.21.15.16:0/3829804680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bf40039b0 msgr2=0x7f1bf4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.510+0000 7f1c0d119640 1 --2- 172.21.15.16:0/3829804680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bf40039b0 0x7f1bf4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.510+0000 7f1c0d119640 1 -- 172.21.15.16:0/3829804680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf4003520 con 0x7f1bf4004800 2024-04-25T10:07:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.510+0000 7f1c0e11b640 1 --2- 172.21.15.16:0/3829804680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c080d35b0 0x7f1bf400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.510+0000 7f1c0d91a640 1 --2- 172.21.15.16:0/3829804680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bf40039b0 0x7f1bf4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.511+0000 7f1c0d119640 1 --2- 172.21.15.16:0/3829804680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bf4004800 0x7f1bf4004c80 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009860 tx=0x7f1bf4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8499fd9721efd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.511+0000 7f1c0c918640 1 -- 172.21.15.16:0/3829804680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8037070 con 0x7f1bf4004800 2024-04-25T10:07:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.511+0000 7f1c0c918640 1 -- 172.21.15.16:0/3829804680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf802aa70 con 0x7f1bf4004800 2024-04-25T10:07:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.511+0000 7f1c0c918640 1 -- 172.21.15.16:0/3829804680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8009460 con 0x7f1bf4004800 2024-04-25T10:07:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.511+0000 7f1c0fb7e640 1 -- 172.21.15.16:0/3829804680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bf4004800 msgr2=0x7f1bf4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.511+0000 7f1c0fb7e640 1 --2- 172.21.15.16:0/3829804680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bf4004800 0x7f1bf4004c80 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009860 tx=0x7f1bf4005830 comp rx=0 tx=0).stop 2024-04-25T10:07:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.511+0000 7f1c0e11b640 1 -- 172.21.15.16:0/3829804680 reap_dead start 2024-04-25T10:07:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.511+0000 7f1c0fb7e640 1 -- 172.21.15.16:0/3829804680 shutdown_connections 2024-04-25T10:07:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.511+0000 7f1c0fb7e640 1 -- 172.21.15.16:0/3829804680 >> 172.21.15.16:0/3829804680 conn(0x7f1c08072900 msgr2=0x7f1c080d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.511+0000 7f1c0fb7e640 1 -- 172.21.15.16:0/3829804680 shutdown_connections 2024-04-25T10:07:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.511+0000 7f1c0fb7e640 1 -- 172.21.15.16:0/3829804680 wait complete. 2024-04-25T10:07:38.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.513+0000 7f1c0fb7e640 1 Processor -- start 2024-04-25T10:07:38.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.513+0000 7f1c0fb7e640 1 -- start start 2024-04-25T10:07:38.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.513+0000 7f1c0fb7e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bf4004800 0x7f1c080aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.513+0000 7f1c0fb7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c080abc20 0x7f1c0815be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.513+0000 7f1c0fb7e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c081f70f0 0x7f1c081f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.513+0000 7f1c0fb7e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c080d3090 con 0x7f1c080abc20 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.513+0000 7f1c0fb7e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c080d2db0 con 0x7f1bf4004800 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.513+0000 7f1c0fb7e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c080b9820 con 0x7f1c081f70f0 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.513+0000 7f1c0d119640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c080abc20 0x7f1c0815be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.513+0000 7f1c0d119640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c080abc20 0x7f1c0815be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48260/0 (socket says 172.21.15.16:48260) 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.513+0000 7f1c0d119640 1 -- 172.21.15.16:0/1276311787 learned_addr learned my addr 172.21.15.16:0/1276311787 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.513+0000 7f1c0e11b640 1 --2- 172.21.15.16:0/1276311787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c081f70f0 0x7f1c081f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.513+0000 7f1c0d91a640 1 --2- 172.21.15.16:0/1276311787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bf4004800 0x7f1c080aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.514+0000 7f1c0d119640 1 -- 172.21.15.16:0/1276311787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c081f70f0 msgr2=0x7f1c081f7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.514+0000 7f1c0d119640 1 --2- 172.21.15.16:0/1276311787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c081f70f0 0x7f1c081f7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.514+0000 7f1c0d119640 1 -- 172.21.15.16:0/1276311787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bf4004800 msgr2=0x7f1c080aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.514+0000 7f1c0d119640 1 --2- 172.21.15.16:0/1276311787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bf4004800 0x7f1c080aa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.514+0000 7f1c0d119640 1 -- 172.21.15.16:0/1276311787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c080ac0a0 con 0x7f1c080abc20 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.514+0000 7f1c0d119640 1 --2- 172.21.15.16:0/1276311787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c080abc20 0x7f1c0815be60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8035040 tx=0x7f1bf80047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.514+0000 7f1bfe7fc640 1 -- 172.21.15.16:0/1276311787 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8042020 con 0x7f1c080abc20 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.514+0000 7f1bfe7fc640 1 -- 172.21.15.16:0/1276311787 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf8009440 con 0x7f1c080abc20 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.514+0000 7f1c0fb7e640 1 -- 172.21.15.16:0/1276311787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c080aaa90 con 0x7f1c080abc20 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.514+0000 7f1bfe7fc640 1 -- 172.21.15.16:0/1276311787 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8037040 con 0x7f1c080abc20 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.514+0000 7f1c0fb7e640 1 -- 172.21.15.16:0/1276311787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c080ab050 con 0x7f1c080abc20 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.515+0000 7f1c0fb7e640 1 -- 172.21.15.16:0/1276311787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c080ab290 con 0x7f1c080abc20 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.515+0000 7f1bfe7fc640 1 -- 172.21.15.16:0/1276311787 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1bf8045020 con 0x7f1c080abc20 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.516+0000 7f1bfe7fc640 1 --2- 172.21.15.16:0/1276311787 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1bec06b4d0 0x7f1bec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.516+0000 7f1bfe7fc640 1 -- 172.21.15.16:0/1276311787 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bf80aa820 con 0x7f1c080abc20 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.516+0000 7f1c0d91a640 1 --2- 172.21.15.16:0/1276311787 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1bec06b4d0 0x7f1bec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.516+0000 7f1c0d91a640 1 --2- 172.21.15.16:0/1276311787 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1bec06b4d0 0x7f1bec06d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1c04004650 tx=0x7f1c04008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.520+0000 7f1bfe7fc640 1 -- 172.21.15.16:0/1276311787 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c080ab290 con 0x7f1c080abc20 2024-04-25T10:07:38.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.738+0000 7f1c0fb7e640 1 -- 172.21.15.16:0/1276311787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c080749b0 con 0x7f1c080abc20 2024-04-25T10:07:38.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.738+0000 7f1bfe7fc640 1 -- 172.21.15.16:0/1276311787 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f1bf804c030 con 0x7f1c080abc20 2024-04-25T10:07:38.744 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:38.744 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/791079479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":791079479},{"type":"v1","addr":"172.21.15.16:6835","nonce":791079479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1747921236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1747921236},{"type":"v1","addr":"172.21.15.16:6834","nonce":1747921236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":34880,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/3183779257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3183779257},{"type":"v1","addr":"172.21.15.186:6836","nonce":3183779257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:51.182691+0000","modified":"2024-04-25T10:06:53.179194+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34883},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34883":{"gid":34883,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/83423489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":83423489},{"type":"v1","addr":"172.21.15.186:6837","nonce":83423489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:38.745 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 21 2024-04-25T10:07:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.740+0000 7f1c0fb7e640 1 -- 172.21.15.16:0/1276311787 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1bec06b4d0 msgr2=0x7f1bec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.740+0000 7f1c0fb7e640 1 --2- 172.21.15.16:0/1276311787 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1bec06b4d0 0x7f1bec06d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1c04004650 tx=0x7f1c04008040 comp rx=0 tx=0).stop 2024-04-25T10:07:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.740+0000 7f1c0fb7e640 1 -- 172.21.15.16:0/1276311787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c080abc20 msgr2=0x7f1c0815be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.740+0000 7f1c0fb7e640 1 --2- 172.21.15.16:0/1276311787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c080abc20 0x7f1c0815be60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8035040 tx=0x7f1bf80047d0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.740+0000 7f1c0fb7e640 1 -- 172.21.15.16:0/1276311787 shutdown_connections 2024-04-25T10:07:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.740+0000 7f1c0fb7e640 1 --2- 172.21.15.16:0/1276311787 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1bec06b4d0 0x7f1bec06d990 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.740+0000 7f1c0fb7e640 1 --2- 172.21.15.16:0/1276311787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c081f70f0 0x7f1c081f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.740+0000 7f1c0fb7e640 1 --2- 172.21.15.16:0/1276311787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c080abc20 0x7f1c0815be60 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.740+0000 7f1c0fb7e640 1 --2- 172.21.15.16:0/1276311787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bf4004800 0x7f1c080aa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.740+0000 7f1c0fb7e640 1 -- 172.21.15.16:0/1276311787 >> 172.21.15.16:0/1276311787 conn(0x7f1c08072900 msgr2=0x7f1c080b3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.740+0000 7f1c0fb7e640 1 -- 172.21.15.16:0/1276311787 shutdown_connections 2024-04-25T10:07:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.740+0000 7f1c0fb7e640 1 -- 172.21.15.16:0/1276311787 wait complete. 2024-04-25T10:07:38.757 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:38.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.853+0000 7fe87877e640 1 Processor -- start 2024-04-25T10:07:38.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.853+0000 7fe87877e640 1 -- start start 2024-04-25T10:07:38.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.854+0000 7fe87877e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe87015dce0 0x7fe8700d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.854+0000 7fe87877e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8700d35b0 0x7fe8700b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.854+0000 7fe87877e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe87015e0c0 con 0x7fe8700d1f90 2024-04-25T10:07:38.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.854+0000 7fe87877e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8700b7b10 con 0x7fe87015dce0 2024-04-25T10:07:38.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.854+0000 7fe87877e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8700d2510 con 0x7fe8700d35b0 2024-04-25T10:07:38.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.854+0000 7fe875d19640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8700d35b0 0x7fe8700b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.854+0000 7fe87651a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe87015dce0 0x7fe8700d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.854+0000 7fe87651a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe87015dce0 0x7fe8700d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:33972/0 (socket says 172.21.15.16:33972) 2024-04-25T10:07:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.854+0000 7fe87651a640 1 -- 172.21.15.16:0/1151768433 learned_addr learned my addr 172.21.15.16:0/1151768433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.854+0000 7fe876d1b640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fe8700d1f90 0x7fe87006bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:37126/0 (socket says 172.21.15.16:37126) 2024-04-25T10:07:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.854+0000 7fe875d19640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8700d35b0 0x7fe8700b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:34430/0 (socket says 172.21.15.16:34430) 2024-04-25T10:07:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.854+0000 7fe875d19640 1 -- 172.21.15.16:0/1151768433 >> v2:172.21.15.186:3300/0 conn(0x7fe87015dce0 msgr2=0x7fe8700d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.854+0000 7fe875d19640 1 --2- 172.21.15.16:0/1151768433 >> v2:172.21.15.186:3300/0 conn(0x7fe87015dce0 0x7fe8700d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.854+0000 7fe875d19640 1 -- 172.21.15.16:0/1151768433 >> v1:172.21.15.186:6789/0 conn(0x7fe8700d1f90 legacy=0x7fe87006bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.854+0000 7fe875d19640 1 -- 172.21.15.16:0/1151768433 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8700d27f0 con 0x7fe8700d35b0 2024-04-25T10:07:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.854+0000 7fe875518640 1 -- 172.21.15.16:0/1151768433 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350529456 0 0) 0x7fe87015e0c0 con 0x7fe8700d1f90 2024-04-25T10:07:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.854+0000 7fe87651a640 1 --2- 172.21.15.16:0/1151768433 >> v2:172.21.15.186:3300/0 conn(0x7fe87015dce0 0x7fe8700d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:38.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.855+0000 7fe875d19640 1 --2- 172.21.15.16:0/1151768433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8700d35b0 0x7fe8700b5450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe8600097a0 tx=0x7fe86002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33cecb5ccbd81853 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.855+0000 7fe875518640 1 -- 172.21.15.16:0/1151768433 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860037070 con 0x7fe8700d35b0 2024-04-25T10:07:38.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.855+0000 7fe875518640 1 -- 172.21.15.16:0/1151768433 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86002ae50 con 0x7fe8700d35b0 2024-04-25T10:07:38.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.855+0000 7fe875518640 1 -- 172.21.15.16:0/1151768433 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860032660 con 0x7fe8700d35b0 2024-04-25T10:07:38.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.856+0000 7fe87877e640 1 -- 172.21.15.16:0/1151768433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8700d35b0 msgr2=0x7fe8700b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.856+0000 7fe87877e640 1 --2- 172.21.15.16:0/1151768433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8700d35b0 0x7fe8700b5450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe8600097a0 tx=0x7fe86002a100 comp rx=0 tx=0).stop 2024-04-25T10:07:38.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.856+0000 7fe87877e640 1 -- 172.21.15.16:0/1151768433 shutdown_connections 2024-04-25T10:07:38.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.856+0000 7fe87877e640 1 --2- 172.21.15.16:0/1151768433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8700d35b0 0x7fe8700b5450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.856+0000 7fe87877e640 1 --2- 172.21.15.16:0/1151768433 >> v2:172.21.15.186:3300/0 conn(0x7fe87015dce0 0x7fe8700d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.856+0000 7fe87877e640 1 -- 172.21.15.16:0/1151768433 >> 172.21.15.16:0/1151768433 conn(0x7fe870072900 msgr2=0x7fe8700d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.856+0000 7fe87877e640 1 -- 172.21.15.16:0/1151768433 shutdown_connections 2024-04-25T10:07:38.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.856+0000 7fe87877e640 1 -- 172.21.15.16:0/1151768433 wait complete. 2024-04-25T10:07:38.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.857+0000 7fe87877e640 1 Processor -- start 2024-04-25T10:07:38.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.857+0000 7fe87877e640 1 -- start start 2024-04-25T10:07:38.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.858+0000 7fe87877e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8700d1f90 0x7fe870160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.858+0000 7fe87877e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8700d35b0 0x7fe8701569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.858+0000 7fe87877e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe87015dce0 0x7fe87015a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.858+0000 7fe87877e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8700d3090 con 0x7fe87015dce0 2024-04-25T10:07:38.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.858+0000 7fe87877e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8700d2db0 con 0x7fe8700d1f90 2024-04-25T10:07:38.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.858+0000 7fe87877e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8700b9820 con 0x7fe8700d35b0 2024-04-25T10:07:38.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.858+0000 7fe876d1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe87015dce0 0x7fe87015a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.858+0000 7fe876d1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe87015dce0 0x7fe87015a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48262/0 (socket says 172.21.15.16:48262) 2024-04-25T10:07:38.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.858+0000 7fe876d1b640 1 -- 172.21.15.16:0/285630216 learned_addr learned my addr 172.21.15.16:0/285630216 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:38.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.858+0000 7fe87651a640 1 --2- 172.21.15.16:0/285630216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8700d1f90 0x7fe870160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.858+0000 7fe875d19640 1 --2- 172.21.15.16:0/285630216 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8700d35b0 0x7fe8701569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.859+0000 7fe876d1b640 1 -- 172.21.15.16:0/285630216 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8700d35b0 msgr2=0x7fe8701569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.859+0000 7fe876d1b640 1 --2- 172.21.15.16:0/285630216 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8700d35b0 0x7fe8701569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.859+0000 7fe876d1b640 1 -- 172.21.15.16:0/285630216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8700d1f90 msgr2=0x7fe870160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.859+0000 7fe876d1b640 1 --2- 172.21.15.16:0/285630216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8700d1f90 0x7fe870160c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.859+0000 7fe876d1b640 1 -- 172.21.15.16:0/285630216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87015fa50 con 0x7fe87015dce0 2024-04-25T10:07:38.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.859+0000 7fe876d1b640 1 --2- 172.21.15.16:0/285630216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe87015dce0 0x7fe87015a410 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe86800c700 tx=0x7fe86800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.859+0000 7fe866ffd640 1 -- 172.21.15.16:0/285630216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868018070 con 0x7fe87015dce0 2024-04-25T10:07:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.859+0000 7fe866ffd640 1 -- 172.21.15.16:0/285630216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86800ce10 con 0x7fe87015dce0 2024-04-25T10:07:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.859+0000 7fe87877e640 1 -- 172.21.15.16:0/285630216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe870160130 con 0x7fe87015dce0 2024-04-25T10:07:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.859+0000 7fe866ffd640 1 -- 172.21.15.16:0/285630216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86800a860 con 0x7fe87015dce0 2024-04-25T10:07:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.859+0000 7fe87877e640 1 -- 172.21.15.16:0/285630216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8701606f0 con 0x7fe87015dce0 2024-04-25T10:07:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.860+0000 7fe87877e640 1 -- 172.21.15.16:0/285630216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8700749b0 con 0x7fe87015dce0 2024-04-25T10:07:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.860+0000 7fe866ffd640 1 -- 172.21.15.16:0/285630216 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe86801b070 con 0x7fe87015dce0 2024-04-25T10:07:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.861+0000 7fe866ffd640 1 --2- 172.21.15.16:0/285630216 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe85406b500 0x7fe85406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.861+0000 7fe866ffd640 1 -- 172.21.15.16:0/285630216 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe868089670 con 0x7fe87015dce0 2024-04-25T10:07:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.861+0000 7fe87651a640 1 --2- 172.21.15.16:0/285630216 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe85406b500 0x7fe85406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.862+0000 7fe87651a640 1 --2- 172.21.15.16:0/285630216 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe85406b500 0x7fe85406d9c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009630 tx=0x7fe86c009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:38.865+0000 7fe866ffd640 1 -- 172.21.15.16:0/285630216 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe868057f30 con 0x7fe87015dce0 2024-04-25T10:07:39.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.081+0000 7fe87877e640 1 -- 172.21.15.16:0/285630216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8700b59d0 con 0x7fe87015dce0 2024-04-25T10:07:39.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.082+0000 7fe866ffd640 1 -- 172.21.15.16:0/285630216 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fe86805b360 con 0x7fe87015dce0 2024-04-25T10:07:39.087 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:39.088 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/791079479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":791079479},{"type":"v1","addr":"172.21.15.16:6835","nonce":791079479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1747921236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1747921236},{"type":"v1","addr":"172.21.15.16:6834","nonce":1747921236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":34880,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/3183779257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3183779257},{"type":"v1","addr":"172.21.15.186:6836","nonce":3183779257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:51.182691+0000","modified":"2024-04-25T10:06:53.179194+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34883},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34883":{"gid":34883,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/83423489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":83423489},{"type":"v1","addr":"172.21.15.186:6837","nonce":83423489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:39.088 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 21 2024-04-25T10:07:39.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.083+0000 7fe87877e640 1 -- 172.21.15.16:0/285630216 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe85406b500 msgr2=0x7fe85406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.083+0000 7fe87877e640 1 --2- 172.21.15.16:0/285630216 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe85406b500 0x7fe85406d9c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009630 tx=0x7fe86c009240 comp rx=0 tx=0).stop 2024-04-25T10:07:39.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.083+0000 7fe87877e640 1 -- 172.21.15.16:0/285630216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe87015dce0 msgr2=0x7fe87015a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.083+0000 7fe87877e640 1 --2- 172.21.15.16:0/285630216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe87015dce0 0x7fe87015a410 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe86800c700 tx=0x7fe86800cbc0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.083+0000 7fe87877e640 1 -- 172.21.15.16:0/285630216 shutdown_connections 2024-04-25T10:07:39.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.083+0000 7fe87877e640 1 --2- 172.21.15.16:0/285630216 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe85406b500 0x7fe85406d9c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.083+0000 7fe87877e640 1 --2- 172.21.15.16:0/285630216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe87015dce0 0x7fe87015a410 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.083+0000 7fe87877e640 1 --2- 172.21.15.16:0/285630216 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8700d35b0 0x7fe8701569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.083+0000 7fe87877e640 1 --2- 172.21.15.16:0/285630216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8700d1f90 0x7fe870160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.083+0000 7fe87877e640 1 -- 172.21.15.16:0/285630216 >> 172.21.15.16:0/285630216 conn(0x7fe870072900 msgr2=0x7fe8700b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:39.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.083+0000 7fe87877e640 1 -- 172.21.15.16:0/285630216 shutdown_connections 2024-04-25T10:07:39.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.083+0000 7fe87877e640 1 -- 172.21.15.16:0/285630216 wait complete. 2024-04-25T10:07:39.100 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:39.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.194+0000 7ff83ed4f640 1 Processor -- start 2024-04-25T10:07:39.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.194+0000 7ff83ed4f640 1 -- start start 2024-04-25T10:07:39.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.195+0000 7ff83ed4f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff83806cdd0 0x7ff8380b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.195+0000 7ff83ed4f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff8380b1960 0x7ff8380d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.195+0000 7ff83ed4f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff83806c1f0 con 0x7ff83815dc30 2024-04-25T10:07:39.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.195+0000 7ff83ed4f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8380b1e10 con 0x7ff83806cdd0 2024-04-25T10:07:39.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.195+0000 7ff83ed4f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8380cfb40 con 0x7ff8380b1960 2024-04-25T10:07:39.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.195+0000 7ff83caeb640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff83815dc30 0x7ff83806c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41860/0 (socket says 172.21.15.16:41860) 2024-04-25T10:07:39.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.195+0000 7ff83caeb640 1 -- 172.21.15.16:0/3272190839 learned_addr learned my addr 172.21.15.16:0/3272190839 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:39.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.195+0000 7ff837fff640 1 --2- 172.21.15.16:0/3272190839 >> v2:172.21.15.16:3300/0 conn(0x7ff83806cdd0 0x7ff8380b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.195+0000 7ff837fff640 1 -- 172.21.15.16:0/3272190839 >> v2:172.21.15.186:3300/0 conn(0x7ff8380b1960 msgr2=0x7ff8380d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:39.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.195+0000 7ff837fff640 1 --2- 172.21.15.16:0/3272190839 >> v2:172.21.15.186:3300/0 conn(0x7ff8380b1960 0x7ff8380d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.196+0000 7ff837fff640 1 -- 172.21.15.16:0/3272190839 >> v1:172.21.15.16:6789/0 conn(0x7ff83815dc30 legacy=0x7ff83806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.196+0000 7ff837fff640 1 -- 172.21.15.16:0/3272190839 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8380b2060 con 0x7ff83806cdd0 2024-04-25T10:07:39.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.196+0000 7ff8377fe640 1 -- 172.21.15.16:0/3272190839 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3602772521 0 0) 0x7ff828001110 con 0x7ff83815dc30 2024-04-25T10:07:39.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.196+0000 7ff837fff640 1 --2- 172.21.15.16:0/3272190839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83806cdd0 0x7ff8380b1420 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff8380c8980 tx=0x7ff82c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b323ec65aa5d0ac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.196+0000 7ff8377fe640 1 -- 172.21.15.16:0/3272190839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c037070 con 0x7ff83806cdd0 2024-04-25T10:07:39.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.196+0000 7ff8377fe640 1 -- 172.21.15.16:0/3272190839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82c032440 con 0x7ff83806cdd0 2024-04-25T10:07:39.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.196+0000 7ff8377fe640 1 -- 172.21.15.16:0/3272190839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c032810 con 0x7ff83806cdd0 2024-04-25T10:07:39.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.197+0000 7ff83ed4f640 1 -- 172.21.15.16:0/3272190839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83806cdd0 msgr2=0x7ff8380b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.197+0000 7ff83ed4f640 1 --2- 172.21.15.16:0/3272190839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83806cdd0 0x7ff8380b1420 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff8380c8980 tx=0x7ff82c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.197+0000 7ff83ed4f640 1 -- 172.21.15.16:0/3272190839 shutdown_connections 2024-04-25T10:07:39.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.197+0000 7ff83ed4f640 1 --2- 172.21.15.16:0/3272190839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83806cdd0 0x7ff8380b1420 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.197+0000 7ff83ed4f640 1 --2- 172.21.15.16:0/3272190839 >> v2:172.21.15.186:3300/0 conn(0x7ff8380b1960 0x7ff8380d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.197+0000 7ff83ed4f640 1 -- 172.21.15.16:0/3272190839 >> 172.21.15.16:0/3272190839 conn(0x7ff8380d1900 msgr2=0x7ff8380d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:39.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.197+0000 7ff83ed4f640 1 -- 172.21.15.16:0/3272190839 shutdown_connections 2024-04-25T10:07:39.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.197+0000 7ff83ed4f640 1 -- 172.21.15.16:0/3272190839 wait complete. 2024-04-25T10:07:39.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.198+0000 7ff83ed4f640 1 Processor -- start 2024-04-25T10:07:39.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.198+0000 7ff83ed4f640 1 -- start start 2024-04-25T10:07:39.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.199+0000 7ff83ed4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83806cdd0 0x7ff83815fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.199+0000 7ff83ed4f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8380b1960 0x7ff8381606e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.199+0000 7ff83ed4f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff83815dc30 0x7ff838157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.199+0000 7ff83ed4f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8380cfde0 con 0x7ff83806cdd0 2024-04-25T10:07:39.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.199+0000 7ff83ed4f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8380b2620 con 0x7ff83815dc30 2024-04-25T10:07:39.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.199+0000 7ff83ed4f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83815e010 con 0x7ff8380b1960 2024-04-25T10:07:39.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.199+0000 7ff83caeb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83806cdd0 0x7ff83815fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.199+0000 7ff83caeb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83806cdd0 0x7ff83815fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48288/0 (socket says 172.21.15.16:48288) 2024-04-25T10:07:39.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.199+0000 7ff83caeb640 1 -- 172.21.15.16:0/1829980915 learned_addr learned my addr 172.21.15.16:0/1829980915 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:39.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.199+0000 7ff83d2ec640 1 --2- 172.21.15.16:0/1829980915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff83815dc30 0x7ff838157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.199+0000 7ff837fff640 1 --2- 172.21.15.16:0/1829980915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8380b1960 0x7ff8381606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.199+0000 7ff83caeb640 1 -- 172.21.15.16:0/1829980915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8380b1960 msgr2=0x7ff8381606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.199+0000 7ff83caeb640 1 --2- 172.21.15.16:0/1829980915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8380b1960 0x7ff8381606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.199+0000 7ff83caeb640 1 -- 172.21.15.16:0/1829980915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff83815dc30 msgr2=0x7ff838157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.199+0000 7ff83caeb640 1 --2- 172.21.15.16:0/1829980915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff83815dc30 0x7ff838157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.200+0000 7ff83caeb640 1 -- 172.21.15.16:0/1829980915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8380b61e0 con 0x7ff83806cdd0 2024-04-25T10:07:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.200+0000 7ff83d2ec640 1 --2- 172.21.15.16:0/1829980915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff83815dc30 0x7ff838157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.200+0000 7ff83caeb640 1 --2- 172.21.15.16:0/1829980915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83806cdd0 0x7ff83815fd70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff82800c7b0 tx=0x7ff82800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.200+0000 7ff8357fa640 1 -- 172.21.15.16:0/1829980915 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828018070 con 0x7ff83806cdd0 2024-04-25T10:07:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.200+0000 7ff8357fa640 1 -- 172.21.15.16:0/1829980915 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8280072d0 con 0x7ff83806cdd0 2024-04-25T10:07:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.200+0000 7ff83ed4f640 1 -- 172.21.15.16:0/1829980915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8380b64c0 con 0x7ff83806cdd0 2024-04-25T10:07:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.200+0000 7ff8357fa640 1 -- 172.21.15.16:0/1829980915 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8280047e0 con 0x7ff83806cdd0 2024-04-25T10:07:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.200+0000 7ff83ed4f640 1 -- 172.21.15.16:0/1829980915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8380b6a80 con 0x7ff83806cdd0 2024-04-25T10:07:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.201+0000 7ff83ed4f640 1 -- 172.21.15.16:0/1829980915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8380749b0 con 0x7ff83806cdd0 2024-04-25T10:07:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.201+0000 7ff8357fa640 1 -- 172.21.15.16:0/1829980915 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff82801b070 con 0x7ff83806cdd0 2024-04-25T10:07:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.202+0000 7ff8357fa640 1 --2- 172.21.15.16:0/1829980915 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff82406b4d0 0x7ff82406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.202+0000 7ff8357fa640 1 -- 172.21.15.16:0/1829980915 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff828089550 con 0x7ff83806cdd0 2024-04-25T10:07:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.202+0000 7ff837fff640 1 --2- 172.21.15.16:0/1829980915 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff82406b4d0 0x7ff82406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.203+0000 7ff837fff640 1 --2- 172.21.15.16:0/1829980915 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff82406b4d0 0x7ff82406d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff82c034000 tx=0x7ff82c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.206+0000 7ff8357fa640 1 -- 172.21.15.16:0/1829980915 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff83806c1f0 con 0x7ff83806cdd0 2024-04-25T10:07:39.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.390+0000 7ff83ed4f640 1 -- 172.21.15.16:0/1829980915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8380ce1a0 con 0x7ff83806cdd0 2024-04-25T10:07:39.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.391+0000 7ff8357fa640 1 -- 172.21.15.16:0/1829980915 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16609 (secure 0 0 0) 0x7ff8280583c0 con 0x7ff83806cdd0 2024-04-25T10:07:39.397 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:39.397 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":47,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:06:52.824898+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:49.462347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:50.153232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:39.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.393+0000 7ff83ed4f640 1 -- 172.21.15.16:0/1829980915 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff82406b4d0 msgr2=0x7ff82406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.393+0000 7ff83ed4f640 1 --2- 172.21.15.16:0/1829980915 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff82406b4d0 0x7ff82406d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff82c034000 tx=0x7ff82c02e000 comp rx=0 tx=0).stop 2024-04-25T10:07:39.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.393+0000 7ff83ed4f640 1 -- 172.21.15.16:0/1829980915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83806cdd0 msgr2=0x7ff83815fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.393+0000 7ff83ed4f640 1 --2- 172.21.15.16:0/1829980915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83806cdd0 0x7ff83815fd70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff82800c7b0 tx=0x7ff82800cc70 comp rx=0 tx=0).stop 2024-04-25T10:07:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.393+0000 7ff83ed4f640 1 -- 172.21.15.16:0/1829980915 shutdown_connections 2024-04-25T10:07:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.393+0000 7ff83ed4f640 1 --2- 172.21.15.16:0/1829980915 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff82406b4d0 0x7ff82406d990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.393+0000 7ff83ed4f640 1 --2- 172.21.15.16:0/1829980915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff83815dc30 0x7ff838157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.393+0000 7ff83ed4f640 1 --2- 172.21.15.16:0/1829980915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8380b1960 0x7ff8381606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.393+0000 7ff83ed4f640 1 --2- 172.21.15.16:0/1829980915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83806cdd0 0x7ff83815fd70 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.393+0000 7ff83ed4f640 1 -- 172.21.15.16:0/1829980915 >> 172.21.15.16:0/1829980915 conn(0x7ff8380d1900 msgr2=0x7ff8380d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.393+0000 7ff83ed4f640 1 -- 172.21.15.16:0/1829980915 shutdown_connections 2024-04-25T10:07:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.393+0000 7ff83ed4f640 1 -- 172.21.15.16:0/1829980915 wait complete. 2024-04-25T10:07:39.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:07:39.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.506+0000 7f10f5b90640 1 Processor -- start 2024-04-25T10:07:39.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.506+0000 7f10f5b90640 1 -- start start 2024-04-25T10:07:39.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.507+0000 7f10f5b90640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f10f01610b0 0x7f10f00b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.507+0000 7f10f5b90640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10f01618d0 0x7f10f00ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.507+0000 7f10f5b90640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f0162040 con 0x7f10f00ced70 2024-04-25T10:07:39.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.507+0000 7f10f5b90640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f00d1ea0 con 0x7f10f01610b0 2024-04-25T10:07:39.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.507+0000 7f10f5b90640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f0162320 con 0x7f10f01618d0 2024-04-25T10:07:39.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.507+0000 7f10ef7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f10f01610b0 0x7f10f00b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.507+0000 7f10effff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f10f00ced70 0x7f10f006c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41876/0 (socket says 172.21.15.16:41876) 2024-04-25T10:07:39.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.507+0000 7f10effff640 1 -- 172.21.15.16:0/1521059803 learned_addr learned my addr 172.21.15.16:0/1521059803 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:39.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.507+0000 7f10ef7fe640 1 --2- 172.21.15.16:0/1521059803 >> v2:172.21.15.16:3300/0 conn(0x7f10f01610b0 0x7f10f00b9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48300/0 (socket says 172.21.15.16:48300) 2024-04-25T10:07:39.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.507+0000 7f10eeffd640 1 --2- 172.21.15.16:0/1521059803 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10f01618d0 0x7f10f00ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.507+0000 7f10ef7fe640 1 -- 172.21.15.16:0/1521059803 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10f01618d0 msgr2=0x7f10f00ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:39.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.507+0000 7f10ef7fe640 1 --2- 172.21.15.16:0/1521059803 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10f01618d0 0x7f10f00ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.507+0000 7f10ef7fe640 1 -- 172.21.15.16:0/1521059803 >> v1:172.21.15.16:6789/0 conn(0x7f10f00ced70 legacy=0x7f10f006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.507+0000 7f10ef7fe640 1 -- 172.21.15.16:0/1521059803 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f0162600 con 0x7f10f01610b0 2024-04-25T10:07:39.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.507+0000 7f10ee7fc640 1 -- 172.21.15.16:0/1521059803 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980509017 0 0) 0x7f10f0162040 con 0x7f10f00ced70 2024-04-25T10:07:39.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.507+0000 7f10eeffd640 1 --2- 172.21.15.16:0/1521059803 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10f01618d0 0x7f10f00ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:39.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.508+0000 7f10ef7fe640 1 --2- 172.21.15.16:0/1521059803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f01610b0 0x7f10f00b9580 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f10e00097a0 tx=0x7f10e002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521842afa5247009 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.508+0000 7f10ee7fc640 1 -- 172.21.15.16:0/1521059803 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0037070 con 0x7f10f01610b0 2024-04-25T10:07:39.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.508+0000 7f10ee7fc640 1 -- 172.21.15.16:0/1521059803 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e002ae50 con 0x7f10f01610b0 2024-04-25T10:07:39.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.508+0000 7f10ee7fc640 1 -- 172.21.15.16:0/1521059803 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0032660 con 0x7f10f01610b0 2024-04-25T10:07:39.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.509+0000 7f10f5b90640 1 -- 172.21.15.16:0/1521059803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f01610b0 msgr2=0x7f10f00b9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.509+0000 7f10f5b90640 1 --2- 172.21.15.16:0/1521059803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f01610b0 0x7f10f00b9580 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f10e00097a0 tx=0x7f10e002a100 comp rx=0 tx=0).stop 2024-04-25T10:07:39.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.509+0000 7f10f5b90640 1 -- 172.21.15.16:0/1521059803 shutdown_connections 2024-04-25T10:07:39.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.509+0000 7f10f5b90640 1 --2- 172.21.15.16:0/1521059803 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10f01618d0 0x7f10f00ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.509+0000 7f10f5b90640 1 --2- 172.21.15.16:0/1521059803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f01610b0 0x7f10f00b9580 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.509+0000 7f10f5b90640 1 -- 172.21.15.16:0/1521059803 >> 172.21.15.16:0/1521059803 conn(0x7f10f00b5570 msgr2=0x7f10f00b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:39.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.509+0000 7f10f5b90640 1 -- 172.21.15.16:0/1521059803 shutdown_connections 2024-04-25T10:07:39.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.509+0000 7f10f5b90640 1 -- 172.21.15.16:0/1521059803 wait complete. 2024-04-25T10:07:39.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.510+0000 7f10f5b90640 1 Processor -- start 2024-04-25T10:07:39.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.510+0000 7f10f5b90640 1 -- start start 2024-04-25T10:07:39.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10f5b90640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10f00ced70 0x7f10f00c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10f5b90640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10f01610b0 0x7f10f015d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10f5b90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f01618d0 0x7f10f00cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10f5b90640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f0161d10 con 0x7f10f01618d0 2024-04-25T10:07:39.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10f5b90640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00b6c00 con 0x7f10f00ced70 2024-04-25T10:07:39.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10f5b90640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00d3620 con 0x7f10f01610b0 2024-04-25T10:07:39.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f01618d0 0x7f10f00cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f01618d0 0x7f10f00cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48302/0 (socket says 172.21.15.16:48302) 2024-04-25T10:07:39.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10effff640 1 -- 172.21.15.16:0/2699199689 learned_addr learned my addr 172.21.15.16:0/2699199689 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:39.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10eeffd640 1 --2- 172.21.15.16:0/2699199689 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10f01610b0 0x7f10f015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10effff640 1 -- 172.21.15.16:0/2699199689 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10f01610b0 msgr2=0x7f10f015d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10effff640 1 --2- 172.21.15.16:0/2699199689 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10f01610b0 0x7f10f015d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10effff640 1 -- 172.21.15.16:0/2699199689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10f00ced70 msgr2=0x7f10f00c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10effff640 1 --2- 172.21.15.16:0/2699199689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10f00ced70 0x7f10f00c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10effff640 1 -- 172.21.15.16:0/2699199689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f014e320 con 0x7f10f01618d0 2024-04-25T10:07:39.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10effff640 1 --2- 172.21.15.16:0/2699199689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f01618d0 0x7f10f00cced0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f10e800c7a0 tx=0x7f10e800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10d7fff640 1 -- 172.21.15.16:0/2699199689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8018070 con 0x7f10f01618d0 2024-04-25T10:07:39.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10d7fff640 1 -- 172.21.15.16:0/2699199689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e800a5e0 con 0x7f10f01618d0 2024-04-25T10:07:39.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10d7fff640 1 -- 172.21.15.16:0/2699199689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e800aa50 con 0x7f10f01618d0 2024-04-25T10:07:39.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.511+0000 7f10f5b90640 1 -- 172.21.15.16:0/2699199689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f014e600 con 0x7f10f01618d0 2024-04-25T10:07:39.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.512+0000 7f10f5b90640 1 -- 172.21.15.16:0/2699199689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f015d560 con 0x7f10f01618d0 2024-04-25T10:07:39.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.512+0000 7f10f5b90640 1 -- 172.21.15.16:0/2699199689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f00750b0 con 0x7f10f01618d0 2024-04-25T10:07:39.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.513+0000 7f10d7fff640 1 -- 172.21.15.16:0/2699199689 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f10e801b070 con 0x7f10f01618d0 2024-04-25T10:07:39.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.513+0000 7f10d7fff640 1 --2- 172.21.15.16:0/2699199689 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f10dc06b4d0 0x7f10dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.513+0000 7f10d7fff640 1 -- 172.21.15.16:0/2699199689 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10e8089730 con 0x7f10f01618d0 2024-04-25T10:07:39.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.514+0000 7f10ef7fe640 1 --2- 172.21.15.16:0/2699199689 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f10dc06b4d0 0x7f10dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.514+0000 7f10ef7fe640 1 --2- 172.21.15.16:0/2699199689 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f10dc06b4d0 0x7f10dc06d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f10e0034000 tx=0x7f10e002c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.518+0000 7f10d7fff640 1 -- 172.21.15.16:0/2699199689 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f10e80585b0 con 0x7f10f01618d0 2024-04-25T10:07:39.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.740+0000 7f10f5b90640 1 -- 172.21.15.16:0/2699199689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f10f00d1770 con 0x7f10f01618d0 2024-04-25T10:07:39.746 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-04-25T10:07:39.741+0000 7ff13f9bb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:07:39.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.742+0000 7f10d7fff640 1 -- 172.21.15.16:0/2699199689 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) v1 ==== 161+0+0 (secure 0 0 0) 0x7f10e8057eb0 con 0x7f10f01618d0 2024-04-25T10:07:39.748 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:07:39.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.745+0000 7f10f5b90640 1 -- 172.21.15.16:0/2699199689 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f10dc06b4d0 msgr2=0x7f10dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.745+0000 7f10f5b90640 1 --2- 172.21.15.16:0/2699199689 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f10dc06b4d0 0x7f10dc06d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f10e0034000 tx=0x7f10e002c000 comp rx=0 tx=0).stop 2024-04-25T10:07:39.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.745+0000 7f10f5b90640 1 -- 172.21.15.16:0/2699199689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f01618d0 msgr2=0x7f10f00cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.745+0000 7f10f5b90640 1 --2- 172.21.15.16:0/2699199689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f01618d0 0x7f10f00cced0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f10e800c7a0 tx=0x7f10e800cc60 comp rx=0 tx=0).stop 2024-04-25T10:07:39.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.745+0000 7f10f5b90640 1 -- 172.21.15.16:0/2699199689 shutdown_connections 2024-04-25T10:07:39.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.745+0000 7f10f5b90640 1 --2- 172.21.15.16:0/2699199689 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f10dc06b4d0 0x7f10dc06d990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.745+0000 7f10f5b90640 1 --2- 172.21.15.16:0/2699199689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f01618d0 0x7f10f00cced0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.745+0000 7f10f5b90640 1 --2- 172.21.15.16:0/2699199689 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10f01610b0 0x7f10f015d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.745+0000 7f10f5b90640 1 --2- 172.21.15.16:0/2699199689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10f00ced70 0x7f10f00c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.745+0000 7f10f5b90640 1 -- 172.21.15.16:0/2699199689 >> 172.21.15.16:0/2699199689 conn(0x7f10f00b5570 msgr2=0x7f10f0155cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:39.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.745+0000 7f10f5b90640 1 -- 172.21.15.16:0/2699199689 shutdown_connections 2024-04-25T10:07:39.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.745+0000 7f10f5b90640 1 -- 172.21.15.16:0/2699199689 wait complete. 2024-04-25T10:07:39.754 INFO:tasks.ceph.mds.b.smithi186.stderr: -5933> 2024-04-25T10:06:53.793+0000 7f25be5bd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:07:39.757 INFO:tasks.ceph.mds.b.smithi186.stderr: -3130> 2024-04-25T10:07:03.129+0000 7f25ba5b5640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5398=0-4194304@1} caps={5398=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5398 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x561185928100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:07:39.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:07:39.786 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2024-04-25T10:07:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.862+0000 7f11d5353640 1 Processor -- start 2024-04-25T10:07:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.862+0000 7f11d5353640 1 -- start start 2024-04-25T10:07:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.862+0000 7f11d5353640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f11d015b7b0 0x7f11d00ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.862+0000 7f11d5353640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d015f000 con 0x7f11d00cf760 2024-04-25T10:07:39.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.862+0000 7f11d5353640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d015ed20 con 0x7f11d015e820 2024-04-25T10:07:39.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.862+0000 7f11d5353640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d00ced00 con 0x7f11d015b7b0 2024-04-25T10:07:39.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.863+0000 7f11c7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f11d00cf760 0x7f11d015e110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41888/0 (socket says 172.21.15.16:41888) 2024-04-25T10:07:39.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.863+0000 7f11c7fff640 1 -- 172.21.15.16:0/347526317 learned_addr learned my addr 172.21.15.16:0/347526317 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:39.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.863+0000 7f11ceffd640 1 --2- 172.21.15.16:0/347526317 >> v2:172.21.15.186:3300/0 conn(0x7f11d015b7b0 0x7f11d00ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.863+0000 7f11ce7fc640 1 -- 172.21.15.16:0/347526317 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 671653705 0 0) 0x7f11d015f000 con 0x7f11d00cf760 2024-04-25T10:07:39.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.863+0000 7f11ce7fc640 1 -- 172.21.15.16:0/347526317 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11bc003520 con 0x7f11d00cf760 2024-04-25T10:07:39.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.863+0000 7f11ce7fc640 1 -- 172.21.15.16:0/347526317 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992924180 0 0) 0x7f11d015ed20 con 0x7f11d015e820 2024-04-25T10:07:39.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.863+0000 7f11ce7fc640 1 -- 172.21.15.16:0/347526317 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11d015f000 con 0x7f11d015e820 2024-04-25T10:07:39.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.863+0000 7f11ce7fc640 1 -- 172.21.15.16:0/347526317 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3160453013 0 0) 0x7f11bc003520 con 0x7f11d00cf760 2024-04-25T10:07:39.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.863+0000 7f11ceffd640 1 -- 172.21.15.16:0/347526317 >> v1:172.21.15.186:6789/0 conn(0x7f11d015e820 legacy=0x7f11d00cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.863+0000 7f11ceffd640 1 -- 172.21.15.16:0/347526317 >> v1:172.21.15.16:6789/0 conn(0x7f11d00cf760 legacy=0x7f11d015e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.863+0000 7f11ceffd640 1 -- 172.21.15.16:0/347526317 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d00cdce0 con 0x7f11d015b7b0 2024-04-25T10:07:39.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.864+0000 7f11ceffd640 1 --2- 172.21.15.16:0/347526317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11d015b7b0 0x7f11d00ce7c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f11c00097d0 tx=0x7f11c002a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45f546833bfcb939 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.864+0000 7f11ce7fc640 1 -- 172.21.15.16:0/347526317 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0037070 con 0x7f11d015b7b0 2024-04-25T10:07:39.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.864+0000 7f11ce7fc640 1 -- 172.21.15.16:0/347526317 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c002ac90 con 0x7f11d015b7b0 2024-04-25T10:07:39.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.864+0000 7f11ce7fc640 1 -- 172.21.15.16:0/347526317 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b8001670 con 0x7f11d015b7b0 2024-04-25T10:07:39.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.865+0000 7f11d5353640 1 -- 172.21.15.16:0/347526317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11d015b7b0 msgr2=0x7f11d00ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.865+0000 7f11d5353640 1 --2- 172.21.15.16:0/347526317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11d015b7b0 0x7f11d00ce7c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f11c00097d0 tx=0x7f11c002a200 comp rx=0 tx=0).stop 2024-04-25T10:07:39.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.865+0000 7f11d5353640 1 -- 172.21.15.16:0/347526317 shutdown_connections 2024-04-25T10:07:39.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.865+0000 7f11d5353640 1 --2- 172.21.15.16:0/347526317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11d015b7b0 0x7f11d00ce7c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.865+0000 7f11d5353640 1 -- 172.21.15.16:0/347526317 >> 172.21.15.16:0/347526317 conn(0x7f11d006f4a0 msgr2=0x7f11d006f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:39.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.865+0000 7f11d5353640 1 -- 172.21.15.16:0/347526317 shutdown_connections 2024-04-25T10:07:39.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.865+0000 7f11d5353640 1 -- 172.21.15.16:0/347526317 wait complete. 2024-04-25T10:07:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.866+0000 7f11d5353640 1 Processor -- start 2024-04-25T10:07:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.866+0000 7f11d5353640 1 -- start start 2024-04-25T10:07:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11d5353640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d00cf760 0x7f11d00b6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11d5353640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11d015b7b0 0x7f11d00c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11d5353640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11d015e820 0x7f11d0156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11d5353640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d01619f0 con 0x7f11d00cf760 2024-04-25T10:07:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11d5353640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d00ce330 con 0x7f11d015e820 2024-04-25T10:07:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11d5353640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d00cfbf0 con 0x7f11d015b7b0 2024-04-25T10:07:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d00cf760 0x7f11d00b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d00cf760 0x7f11d00b6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48304/0 (socket says 172.21.15.16:48304) 2024-04-25T10:07:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11ceffd640 1 -- 172.21.15.16:0/2516818703 learned_addr learned my addr 172.21.15.16:0/2516818703 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11cf7fe640 1 --2- 172.21.15.16:0/2516818703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11d015e820 0x7f11d0156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11ceffd640 1 -- 172.21.15.16:0/2516818703 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11d015b7b0 msgr2=0x7f11d00c93d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:07:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11ceffd640 1 --2- 172.21.15.16:0/2516818703 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11d015b7b0 0x7f11d00c93d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11ceffd640 1 -- 172.21.15.16:0/2516818703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11d015e820 msgr2=0x7f11d0156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11ceffd640 1 --2- 172.21.15.16:0/2516818703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11d015e820 0x7f11d0156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11ceffd640 1 -- 172.21.15.16:0/2516818703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d00c9910 con 0x7f11d00cf760 2024-04-25T10:07:39.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11ceffd640 1 --2- 172.21.15.16:0/2516818703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d00cf760 0x7f11d00b6850 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f11c0000c00 tx=0x7f11c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11c77fe640 1 -- 172.21.15.16:0/2516818703 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0041070 con 0x7f11d00cf760 2024-04-25T10:07:39.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11c77fe640 1 -- 172.21.15.16:0/2516818703 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c003c660 con 0x7f11d00cf760 2024-04-25T10:07:39.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11d5353640 1 -- 172.21.15.16:0/2516818703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d00c9bc0 con 0x7f11d00cf760 2024-04-25T10:07:39.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11c77fe640 1 -- 172.21.15.16:0/2516818703 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0037040 con 0x7f11d00cf760 2024-04-25T10:07:39.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.867+0000 7f11d5353640 1 -- 172.21.15.16:0/2516818703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d00ca180 con 0x7f11d00cf760 2024-04-25T10:07:39.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.868+0000 7f11d5353640 1 -- 172.21.15.16:0/2516818703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d015ed20 con 0x7f11d00cf760 2024-04-25T10:07:39.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.869+0000 7f11c77fe640 1 -- 172.21.15.16:0/2516818703 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11c0044070 con 0x7f11d00cf760 2024-04-25T10:07:39.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.869+0000 7f11c77fe640 1 --2- 172.21.15.16:0/2516818703 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f11b006b4d0 0x7f11b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.869+0000 7f11c77fe640 1 -- 172.21.15.16:0/2516818703 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f11c00aa9a0 con 0x7f11d00cf760 2024-04-25T10:07:39.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.869+0000 7f11c7fff640 1 --2- 172.21.15.16:0/2516818703 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f11b006b4d0 0x7f11b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.870+0000 7f11c7fff640 1 --2- 172.21.15.16:0/2516818703 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f11b006b4d0 0x7f11b006d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f11d00b3e50 tx=0x7f11b8001000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:39.874+0000 7f11c77fe640 1 -- 172.21.15.16:0/2516818703 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11d015ed20 con 0x7f11d00cf760 2024-04-25T10:07:40.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.093+0000 7f11d5353640 1 -- 172.21.15.16:0/2516818703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f11d015bd90 con 0x7f11d00cf760 2024-04-25T10:07:40.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.749+0000 7f11c77fe640 1 -- 172.21.15.16:0/2516818703 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f11c0079850 con 0x7f11d00cf760 2024-04-25T10:07:40.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.752+0000 7f11d5353640 1 -- 172.21.15.16:0/2516818703 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f11b006b4d0 msgr2=0x7f11b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.752+0000 7f11d5353640 1 --2- 172.21.15.16:0/2516818703 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f11b006b4d0 0x7f11b006d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f11d00b3e50 tx=0x7f11b8001000 comp rx=0 tx=0).stop 2024-04-25T10:07:40.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.752+0000 7f11d5353640 1 -- 172.21.15.16:0/2516818703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d00cf760 msgr2=0x7f11d00b6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.752+0000 7f11d5353640 1 --2- 172.21.15.16:0/2516818703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d00cf760 0x7f11d00b6850 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f11c0000c00 tx=0x7f11c00046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.752+0000 7f11d5353640 1 -- 172.21.15.16:0/2516818703 shutdown_connections 2024-04-25T10:07:40.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.752+0000 7f11d5353640 1 --2- 172.21.15.16:0/2516818703 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f11b006b4d0 0x7f11b006d990 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.752+0000 7f11d5353640 1 --2- 172.21.15.16:0/2516818703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11d015e820 0x7f11d0156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.752+0000 7f11d5353640 1 --2- 172.21.15.16:0/2516818703 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11d015b7b0 0x7f11d00c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.752+0000 7f11d5353640 1 --2- 172.21.15.16:0/2516818703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d00cf760 0x7f11d00b6850 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.752+0000 7f11d5353640 1 -- 172.21.15.16:0/2516818703 >> 172.21.15.16:0/2516818703 conn(0x7f11d006f4a0 msgr2=0x7f11d006f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:40.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.752+0000 7f11d5353640 1 -- 172.21.15.16:0/2516818703 shutdown_connections 2024-04-25T10:07:40.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.752+0000 7f11d5353640 1 -- 172.21.15.16:0/2516818703 wait complete. 2024-04-25T10:07:40.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:07:40.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.874+0000 7f0d4c44d640 1 Processor -- start 2024-04-25T10:07:40.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.874+0000 7f0d4c44d640 1 -- start start 2024-04-25T10:07:40.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.875+0000 7f0d4c44d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d441610b0 0x7f0d440b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.875+0000 7f0d4c44d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0d441618d0 0x7f0d440ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.875+0000 7f0d4c44d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0d440ced70 0x7f0d4406c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.875+0000 7f0d4c44d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d440d1ea0 con 0x7f0d440ced70 2024-04-25T10:07:40.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.875+0000 7f0d4c44d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d44162040 con 0x7f0d441618d0 2024-04-25T10:07:40.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.875+0000 7f0d4c44d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d44162320 con 0x7f0d441610b0 2024-04-25T10:07:40.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.875+0000 7f0d4a9ea640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0d440ced70 0x7f0d4406c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.875+0000 7f0d4a9ea640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0d440ced70 0x7f0d4406c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48314/0 (socket says 172.21.15.16:48314) 2024-04-25T10:07:40.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.875+0000 7f0d4a9ea640 1 -- 172.21.15.16:0/2256197635 learned_addr learned my addr 172.21.15.16:0/2256197635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:40.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.875+0000 7f0d4a1e9640 1 --2- 172.21.15.16:0/2256197635 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d441610b0 0x7f0d440b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.875+0000 7f0d499e8640 1 --2- 172.21.15.16:0/2256197635 >> v2:172.21.15.186:3300/0 conn(0x7f0d441618d0 0x7f0d440ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.875+0000 7f0d4a9ea640 1 -- 172.21.15.16:0/2256197635 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d441610b0 msgr2=0x7f0d440b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:40.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.875+0000 7f0d4a9ea640 1 --2- 172.21.15.16:0/2256197635 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d441610b0 0x7f0d440b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.875+0000 7f0d4a9ea640 1 -- 172.21.15.16:0/2256197635 >> v2:172.21.15.186:3300/0 conn(0x7f0d441618d0 msgr2=0x7f0d440ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:40.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.875+0000 7f0d4a9ea640 1 --2- 172.21.15.16:0/2256197635 >> v2:172.21.15.186:3300/0 conn(0x7f0d441618d0 0x7f0d440ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.875+0000 7f0d4a9ea640 1 -- 172.21.15.16:0/2256197635 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d44162600 con 0x7f0d440ced70 2024-04-25T10:07:40.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.876+0000 7f0d4a9ea640 1 --2- 172.21.15.16:0/2256197635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d440ced70 0x7f0d4406c250 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0d440d4b60 tx=0x7f0d3402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d95b37f2820815b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.876+0000 7f0d491e7640 1 -- 172.21.15.16:0/2256197635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34036070 con 0x7f0d440ced70 2024-04-25T10:07:40.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.876+0000 7f0d491e7640 1 -- 172.21.15.16:0/2256197635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d34031600 con 0x7f0d440ced70 2024-04-25T10:07:40.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.876+0000 7f0d491e7640 1 -- 172.21.15.16:0/2256197635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d340319d0 con 0x7f0d440ced70 2024-04-25T10:07:40.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.877+0000 7f0d4c44d640 1 -- 172.21.15.16:0/2256197635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d440ced70 msgr2=0x7f0d4406c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.877+0000 7f0d4c44d640 1 --2- 172.21.15.16:0/2256197635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d440ced70 0x7f0d4406c250 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0d440d4b60 tx=0x7f0d3402a610 comp rx=0 tx=0).stop 2024-04-25T10:07:40.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.877+0000 7f0d4c44d640 1 -- 172.21.15.16:0/2256197635 shutdown_connections 2024-04-25T10:07:40.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.877+0000 7f0d4c44d640 1 --2- 172.21.15.16:0/2256197635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d440ced70 0x7f0d4406c250 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.877+0000 7f0d4c44d640 1 --2- 172.21.15.16:0/2256197635 >> v2:172.21.15.186:3300/0 conn(0x7f0d441618d0 0x7f0d440ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.877+0000 7f0d4c44d640 1 --2- 172.21.15.16:0/2256197635 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d441610b0 0x7f0d440b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.877+0000 7f0d4c44d640 1 -- 172.21.15.16:0/2256197635 >> 172.21.15.16:0/2256197635 conn(0x7f0d440b5570 msgr2=0x7f0d440b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:40.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.877+0000 7f0d4c44d640 1 -- 172.21.15.16:0/2256197635 shutdown_connections 2024-04-25T10:07:40.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.877+0000 7f0d4c44d640 1 -- 172.21.15.16:0/2256197635 wait complete. 2024-04-25T10:07:40.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.878+0000 7f0d4c44d640 1 Processor -- start 2024-04-25T10:07:40.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.878+0000 7f0d4c44d640 1 -- start start 2024-04-25T10:07:40.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.879+0000 7f0d4c44d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d440ced70 0x7f0d4414fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.879+0000 7f0d4c44d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d441610b0 0x7f0d441537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.879+0000 7f0d4c44d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d441618d0 0x7f0d441f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.879+0000 7f0d4c44d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d44161d10 con 0x7f0d440ced70 2024-04-25T10:07:40.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.879+0000 7f0d4c44d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d440b6c00 con 0x7f0d441610b0 2024-04-25T10:07:40.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.879+0000 7f0d4c44d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d44161490 con 0x7f0d441618d0 2024-04-25T10:07:40.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.879+0000 7f0d4a1e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d440ced70 0x7f0d4414fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.879+0000 7f0d4a1e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d440ced70 0x7f0d4414fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48328/0 (socket says 172.21.15.16:48328) 2024-04-25T10:07:40.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.879+0000 7f0d4a1e9640 1 -- 172.21.15.16:0/3127848610 learned_addr learned my addr 172.21.15.16:0/3127848610 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:40.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.879+0000 7f0d499e8640 1 --2- 172.21.15.16:0/3127848610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d441610b0 0x7f0d441537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.879+0000 7f0d4a9ea640 1 --2- 172.21.15.16:0/3127848610 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d441618d0 0x7f0d441f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.879+0000 7f0d4a1e9640 1 -- 172.21.15.16:0/3127848610 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d441618d0 msgr2=0x7f0d441f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.879+0000 7f0d4a1e9640 1 --2- 172.21.15.16:0/3127848610 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d441618d0 0x7f0d441f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.879+0000 7f0d4a1e9640 1 -- 172.21.15.16:0/3127848610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d441610b0 msgr2=0x7f0d441537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.879+0000 7f0d4a1e9640 1 --2- 172.21.15.16:0/3127848610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d441610b0 0x7f0d441537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.879+0000 7f0d4a1e9640 1 -- 172.21.15.16:0/3127848610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d441500b0 con 0x7f0d440ced70 2024-04-25T10:07:40.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.879+0000 7f0d499e8640 1 --2- 172.21.15.16:0/3127848610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d441610b0 0x7f0d441537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:40.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.880+0000 7f0d4a1e9640 1 --2- 172.21.15.16:0/3127848610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d440ced70 0x7f0d4414fa20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00d800 tx=0x7f0d3c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.880+0000 7f0d42ffd640 1 -- 172.21.15.16:0/3127848610 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c00b6c0 con 0x7f0d440ced70 2024-04-25T10:07:40.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.880+0000 7f0d42ffd640 1 -- 172.21.15.16:0/3127848610 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3c019070 con 0x7f0d440ced70 2024-04-25T10:07:40.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.880+0000 7f0d4c44d640 1 -- 172.21.15.16:0/3127848610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d441521f0 con 0x7f0d440ced70 2024-04-25T10:07:40.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.880+0000 7f0d42ffd640 1 -- 172.21.15.16:0/3127848610 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c00ba80 con 0x7f0d440ced70 2024-04-25T10:07:40.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.880+0000 7f0d4c44d640 1 -- 172.21.15.16:0/3127848610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d441527b0 con 0x7f0d440ced70 2024-04-25T10:07:40.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.881+0000 7f0d4c44d640 1 -- 172.21.15.16:0/3127848610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d440750b0 con 0x7f0d440ced70 2024-04-25T10:07:40.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.881+0000 7f0d42ffd640 1 -- 172.21.15.16:0/3127848610 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d3c00bc20 con 0x7f0d440ced70 2024-04-25T10:07:40.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.882+0000 7f0d42ffd640 1 --2- 172.21.15.16:0/3127848610 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0d3006b4d0 0x7f0d3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.882+0000 7f0d42ffd640 1 -- 172.21.15.16:0/3127848610 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0d3c08a690 con 0x7f0d440ced70 2024-04-25T10:07:40.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.883+0000 7f0d499e8640 1 --2- 172.21.15.16:0/3127848610 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0d3006b4d0 0x7f0d3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.885+0000 7f0d499e8640 1 --2- 172.21.15.16:0/3127848610 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0d3006b4d0 0x7f0d3006d990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0d440b9a80 tx=0x7f0d38008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:40.886+0000 7f0d42ffd640 1 -- 172.21.15.16:0/3127848610 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d3c059540 con 0x7f0d440ced70 2024-04-25T10:07:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:41.066+0000 7f0d4c44d640 1 -- 172.21.15.16:0/3127848610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0d44152d60 con 0x7f0d440ced70 2024-04-25T10:07:41.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:41.939+0000 7f0d42ffd640 1 -- 172.21.15.16:0/3127848610 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0d3c058e40 con 0x7f0d440ced70 2024-04-25T10:07:41.945 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:07:41.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:41.943+0000 7f0d4c44d640 1 -- 172.21.15.16:0/3127848610 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0d3006b4d0 msgr2=0x7f0d3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:41.943+0000 7f0d4c44d640 1 --2- 172.21.15.16:0/3127848610 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0d3006b4d0 0x7f0d3006d990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0d440b9a80 tx=0x7f0d38008040 comp rx=0 tx=0).stop 2024-04-25T10:07:41.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:41.943+0000 7f0d4c44d640 1 -- 172.21.15.16:0/3127848610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d440ced70 msgr2=0x7f0d4414fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:41.943+0000 7f0d4c44d640 1 --2- 172.21.15.16:0/3127848610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d440ced70 0x7f0d4414fa20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00d800 tx=0x7f0d3c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:41.943+0000 7f0d4c44d640 1 -- 172.21.15.16:0/3127848610 shutdown_connections 2024-04-25T10:07:41.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:41.943+0000 7f0d4c44d640 1 --2- 172.21.15.16:0/3127848610 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0d3006b4d0 0x7f0d3006d990 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:41.943+0000 7f0d4c44d640 1 --2- 172.21.15.16:0/3127848610 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d441618d0 0x7f0d441f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:41.943+0000 7f0d4c44d640 1 --2- 172.21.15.16:0/3127848610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d441610b0 0x7f0d441537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:41.943+0000 7f0d4c44d640 1 --2- 172.21.15.16:0/3127848610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d440ced70 0x7f0d4414fa20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:41.943+0000 7f0d4c44d640 1 -- 172.21.15.16:0/3127848610 >> 172.21.15.16:0/3127848610 conn(0x7f0d440b5570 msgr2=0x7f0d44160380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:41.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:41.943+0000 7f0d4c44d640 1 -- 172.21.15.16:0/3127848610 shutdown_connections 2024-04-25T10:07:41.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:41.943+0000 7f0d4c44d640 1 -- 172.21.15.16:0/3127848610 wait complete. 2024-04-25T10:07:41.965 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:07:42.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.070+0000 7fa8812e1640 1 Processor -- start 2024-04-25T10:07:42.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.070+0000 7fa8812e1640 1 -- start start 2024-04-25T10:07:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.071+0000 7fa8812e1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa87c0aedc0 0x7fa87c15ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.071+0000 7fa8812e1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa87c0b4900 0x7fa87c15c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.071+0000 7fa8812e1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87c06da40 con 0x7fa87c06d260 2024-04-25T10:07:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.071+0000 7fa8812e1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa87c15c050 con 0x7fa87c0b4900 2024-04-25T10:07:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.071+0000 7fa8812e1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c0af1a0 con 0x7fa87c0aedc0 2024-04-25T10:07:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.071+0000 7fa87b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa87c0b4900 0x7fa87c15c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.071+0000 7fa87b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa87c0b4900 0x7fa87c15c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48338/0 (socket says 172.21.15.16:48338) 2024-04-25T10:07:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.071+0000 7fa87b7fe640 1 -- 172.21.15.16:0/2742093579 learned_addr learned my addr 172.21.15.16:0/2742093579 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.071+0000 7fa87a7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa87c06d260 0x7fa87c0b41f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41900/0 (socket says 172.21.15.16:41900) 2024-04-25T10:07:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.071+0000 7fa87b7fe640 1 -- 172.21.15.16:0/2742093579 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa87c0aedc0 msgr2=0x7fa87c15ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.071+0000 7fa87b7fe640 1 --2- 172.21.15.16:0/2742093579 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa87c0aedc0 0x7fa87c15ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.071+0000 7fa87b7fe640 1 -- 172.21.15.16:0/2742093579 >> v1:172.21.15.16:6789/0 conn(0x7fa87c06d260 legacy=0x7fa87c0b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.071+0000 7fa87b7fe640 1 -- 172.21.15.16:0/2742093579 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c06dd20 con 0x7fa87c0b4900 2024-04-25T10:07:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.071+0000 7fa879ffb640 1 -- 172.21.15.16:0/2742093579 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205740036 0 0) 0x7fa87c06da40 con 0x7fa87c06d260 2024-04-25T10:07:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.072+0000 7fa87b7fe640 1 --2- 172.21.15.16:0/2742093579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa87c0b4900 0x7fa87c15c7c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009e50 tx=0x7fa86c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee953a07d3d8a52d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.072+0000 7fa879ffb640 1 -- 172.21.15.16:0/2742093579 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c036070 con 0x7fa87c0b4900 2024-04-25T10:07:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.072+0000 7fa879ffb640 1 -- 172.21.15.16:0/2742093579 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86c031600 con 0x7fa87c0b4900 2024-04-25T10:07:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.072+0000 7fa879ffb640 1 -- 172.21.15.16:0/2742093579 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0319d0 con 0x7fa87c0b4900 2024-04-25T10:07:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.073+0000 7fa8812e1640 1 -- 172.21.15.16:0/2742093579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa87c0b4900 msgr2=0x7fa87c15c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.073+0000 7fa8812e1640 1 --2- 172.21.15.16:0/2742093579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa87c0b4900 0x7fa87c15c7c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009e50 tx=0x7fa86c02a680 comp rx=0 tx=0).stop 2024-04-25T10:07:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.073+0000 7fa8812e1640 1 -- 172.21.15.16:0/2742093579 shutdown_connections 2024-04-25T10:07:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.073+0000 7fa8812e1640 1 --2- 172.21.15.16:0/2742093579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa87c0b4900 0x7fa87c15c7c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.073+0000 7fa8812e1640 1 --2- 172.21.15.16:0/2742093579 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa87c0aedc0 0x7fa87c15ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.073+0000 7fa8812e1640 1 -- 172.21.15.16:0/2742093579 >> 172.21.15.16:0/2742093579 conn(0x7fa87c06aba0 msgr2=0x7fa87c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.073+0000 7fa8812e1640 1 -- 172.21.15.16:0/2742093579 shutdown_connections 2024-04-25T10:07:42.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.073+0000 7fa8812e1640 1 -- 172.21.15.16:0/2742093579 wait complete. 2024-04-25T10:07:42.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.074+0000 7fa8812e1640 1 Processor -- start 2024-04-25T10:07:42.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.074+0000 7fa8812e1640 1 -- start start 2024-04-25T10:07:42.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.075+0000 7fa8812e1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa87c06d260 0x7fa87c0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.075+0000 7fa8812e1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa87c0aedc0 0x7fa87c1402a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.075+0000 7fa8812e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa87c0b4900 0x7fa87c0a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.075+0000 7fa8812e1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c06d6c0 con 0x7fa87c0b4900 2024-04-25T10:07:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.075+0000 7fa8812e1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c15c2a0 con 0x7fa87c06d260 2024-04-25T10:07:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.075+0000 7fa8812e1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c15cd00 con 0x7fa87c0aedc0 2024-04-25T10:07:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.075+0000 7fa87b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa87c0b4900 0x7fa87c0a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.075+0000 7fa87b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa87c0b4900 0x7fa87c0a8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48352/0 (socket says 172.21.15.16:48352) 2024-04-25T10:07:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.075+0000 7fa87b7fe640 1 -- 172.21.15.16:0/1584877632 learned_addr learned my addr 172.21.15.16:0/1584877632 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.075+0000 7fa87b7fe640 1 -- 172.21.15.16:0/1584877632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa87c0aedc0 msgr2=0x7fa87c1402a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:07:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.075+0000 7fa87b7fe640 1 --2- 172.21.15.16:0/1584877632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa87c0aedc0 0x7fa87c1402a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.075+0000 7fa87b7fe640 1 -- 172.21.15.16:0/1584877632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa87c06d260 msgr2=0x7fa87c0cf2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:07:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.075+0000 7fa87b7fe640 1 --2- 172.21.15.16:0/1584877632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa87c06d260 0x7fa87c0cf2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.075+0000 7fa87b7fe640 1 -- 172.21.15.16:0/1584877632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c0a93f0 con 0x7fa87c0b4900 2024-04-25T10:07:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.075+0000 7fa87b7fe640 1 --2- 172.21.15.16:0/1584877632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa87c0b4900 0x7fa87c0a8eb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa86c034040 tx=0x7fa86c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.076+0000 7fa8637fe640 1 -- 172.21.15.16:0/1584877632 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c041070 con 0x7fa87c0b4900 2024-04-25T10:07:42.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.076+0000 7fa8637fe640 1 -- 172.21.15.16:0/1584877632 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86c03c650 con 0x7fa87c0b4900 2024-04-25T10:07:42.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.076+0000 7fa8812e1640 1 -- 172.21.15.16:0/1584877632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87c0a9670 con 0x7fa87c0b4900 2024-04-25T10:07:42.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.076+0000 7fa8637fe640 1 -- 172.21.15.16:0/1584877632 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c036040 con 0x7fa87c0b4900 2024-04-25T10:07:42.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.076+0000 7fa8812e1640 1 -- 172.21.15.16:0/1584877632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87c0a9c30 con 0x7fa87c0b4900 2024-04-25T10:07:42.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.077+0000 7fa8812e1640 1 -- 172.21.15.16:0/1584877632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa87c0750b0 con 0x7fa87c0b4900 2024-04-25T10:07:42.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.077+0000 7fa8637fe640 1 -- 172.21.15.16:0/1584877632 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa86c044070 con 0x7fa87c0b4900 2024-04-25T10:07:42.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.077+0000 7fa8637fe640 1 --2- 172.21.15.16:0/1584877632 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa86806b4d0 0x7fa86806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.078+0000 7fa8637fe640 1 -- 172.21.15.16:0/1584877632 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fa86c0aacb0 con 0x7fa87c0b4900 2024-04-25T10:07:42.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.078+0000 7fa87affd640 1 --2- 172.21.15.16:0/1584877632 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa86806b4d0 0x7fa86806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.078+0000 7fa87affd640 1 --2- 172.21.15.16:0/1584877632 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa86806b4d0 0x7fa86806d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa8740096b0 tx=0x7fa874006c20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:42.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.082+0000 7fa8637fe640 1 -- 172.21.15.16:0/1584877632 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa87c0a93f0 con 0x7fa87c0b4900 2024-04-25T10:07:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.264+0000 7fa8812e1640 1 -- 172.21.15.16:0/1584877632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa87c0cd330 con 0x7fa87c0b4900 2024-04-25T10:07:42.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.940+0000 7fa8637fe640 1 -- 172.21.15.16:0/1584877632 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa86c079c80 con 0x7fa87c0b4900 2024-04-25T10:07:42.946 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-04-25T10:07:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.944+0000 7fa8812e1640 1 -- 172.21.15.16:0/1584877632 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa86806b4d0 msgr2=0x7fa86806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.944+0000 7fa8812e1640 1 --2- 172.21.15.16:0/1584877632 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa86806b4d0 0x7fa86806d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa8740096b0 tx=0x7fa874006c20 comp rx=0 tx=0).stop 2024-04-25T10:07:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.944+0000 7fa8812e1640 1 -- 172.21.15.16:0/1584877632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa87c0b4900 msgr2=0x7fa87c0a8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.944+0000 7fa8812e1640 1 --2- 172.21.15.16:0/1584877632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa87c0b4900 0x7fa87c0a8eb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa86c034040 tx=0x7fa86c004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.945+0000 7fa8812e1640 1 -- 172.21.15.16:0/1584877632 shutdown_connections 2024-04-25T10:07:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.945+0000 7fa8812e1640 1 --2- 172.21.15.16:0/1584877632 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa86806b4d0 0x7fa86806d990 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.945+0000 7fa8812e1640 1 --2- 172.21.15.16:0/1584877632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa87c0b4900 0x7fa87c0a8eb0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.945+0000 7fa8812e1640 1 --2- 172.21.15.16:0/1584877632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa87c0aedc0 0x7fa87c1402a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.945+0000 7fa8812e1640 1 --2- 172.21.15.16:0/1584877632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa87c06d260 0x7fa87c0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.945+0000 7fa8812e1640 1 -- 172.21.15.16:0/1584877632 >> 172.21.15.16:0/1584877632 conn(0x7fa87c06aba0 msgr2=0x7fa87c0b1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.948+0000 7fa8812e1640 1 -- 172.21.15.16:0/1584877632 shutdown_connections 2024-04-25T10:07:42.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:42.948+0000 7fa8812e1640 1 -- 172.21.15.16:0/1584877632 wait complete. 2024-04-25T10:07:42.970 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:07:43.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.073+0000 7f965435c640 1 Processor -- start 2024-04-25T10:07:43.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.073+0000 7f965435c640 1 -- start start 2024-04-25T10:07:43.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.074+0000 7f965435c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f964c0b5550 0x7f964c0b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.074+0000 7f965435c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964c161500 con 0x7f964c0d2560 2024-04-25T10:07:43.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.074+0000 7f965435c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964c161eb0 con 0x7f964c161120 2024-04-25T10:07:43.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.074+0000 7f965435c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f964c0b5ef0 con 0x7f964c0b5550 2024-04-25T10:07:43.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.074+0000 7f96528f9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f964c0d2560 0x7f964c0d2980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57058/0 (socket says 172.21.15.16:57058) 2024-04-25T10:07:43.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.074+0000 7f96528f9640 1 -- 172.21.15.16:0/2173479122 learned_addr learned my addr 172.21.15.16:0/2173479122 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:43.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96510f6640 1 -- 172.21.15.16:0/2173479122 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943753597 0 0) 0x7f964c161500 con 0x7f964c0d2560 2024-04-25T10:07:43.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96518f7640 1 --2- 172.21.15.16:0/2173479122 >> v2:172.21.15.186:3300/0 conn(0x7f964c0b5550 0x7f964c0b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96510f6640 1 -- 172.21.15.16:0/2173479122 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9634003520 con 0x7f964c0d2560 2024-04-25T10:07:43.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96510f6640 1 -- 172.21.15.16:0/2173479122 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3429152440 0 0) 0x7f9634003520 con 0x7f964c0d2560 2024-04-25T10:07:43.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96510f6640 1 -- 172.21.15.16:0/2173479122 >> v2:172.21.15.186:3300/0 conn(0x7f964c0b5550 msgr2=0x7f964c0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:43.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96510f6640 1 --2- 172.21.15.16:0/2173479122 >> v2:172.21.15.186:3300/0 conn(0x7f964c0b5550 0x7f964c0b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96510f6640 1 -- 172.21.15.16:0/2173479122 >> v1:172.21.15.186:6789/0 conn(0x7f964c161120 legacy=0x7f964c1617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96510f6640 1 -- 172.21.15.16:0/2173479122 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964c162180 con 0x7f964c0d2560 2024-04-25T10:07:43.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96510f6640 1 -- 172.21.15.16:0/2173479122 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f9644001670 con 0x7f964c0d2560 2024-04-25T10:07:43.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96510f6640 1 -- 172.21.15.16:0/2173479122 >> v1:172.21.15.16:6789/0 conn(0x7f964c0d2560 legacy=0x7f964c0d2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96510f6640 1 --2- 172.21.15.16:0/2173479122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96340039b0 0x7f9634003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96510f6640 1 --2- 172.21.15.16:0/2173479122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9634004800 0x7f9634004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96510f6640 1 --2- 172.21.15.16:0/2173479122 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96340051c0 0x7f963400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96510f6640 1 -- 172.21.15.16:0/2173479122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963400c130 con 0x7f9634004800 2024-04-25T10:07:43.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96510f6640 1 -- 172.21.15.16:0/2173479122 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963400c2e0 con 0x7f96340039b0 2024-04-25T10:07:43.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96510f6640 1 -- 172.21.15.16:0/2173479122 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963400c4e0 con 0x7f96340051c0 2024-04-25T10:07:43.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96518f7640 1 --2- 172.21.15.16:0/2173479122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9634004800 0x7f9634004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96520f8640 1 --2- 172.21.15.16:0/2173479122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96340039b0 0x7f9634003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.075+0000 7f96528f9640 1 --2- 172.21.15.16:0/2173479122 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96340051c0 0x7f963400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.076+0000 7f96518f7640 1 -- 172.21.15.16:0/2173479122 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96340051c0 msgr2=0x7f963400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:43.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.076+0000 7f96518f7640 1 --2- 172.21.15.16:0/2173479122 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96340051c0 0x7f963400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.076+0000 7f96518f7640 1 -- 172.21.15.16:0/2173479122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96340039b0 msgr2=0x7f9634003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.076+0000 7f96518f7640 1 --2- 172.21.15.16:0/2173479122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96340039b0 0x7f9634003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.076+0000 7f96518f7640 1 -- 172.21.15.16:0/2173479122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9634003520 con 0x7f9634004800 2024-04-25T10:07:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.076+0000 7f96528f9640 1 --2- 172.21.15.16:0/2173479122 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96340051c0 0x7f963400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.076+0000 7f96528f9640 1 -- 172.21.15.16:0/2173479122 reap_dead start 2024-04-25T10:07:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.076+0000 7f96518f7640 1 --2- 172.21.15.16:0/2173479122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9634004800 0x7f9634004c80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9640009ac0 tx=0x7f9634005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d55db272a1c3f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.076+0000 7f96510f6640 1 -- 172.21.15.16:0/2173479122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640036070 con 0x7f9634004800 2024-04-25T10:07:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.076+0000 7f96510f6640 1 -- 172.21.15.16:0/2173479122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9640031400 con 0x7f9634004800 2024-04-25T10:07:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.076+0000 7f96510f6640 1 -- 172.21.15.16:0/2173479122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640009690 con 0x7f9634004800 2024-04-25T10:07:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.077+0000 7f965435c640 1 -- 172.21.15.16:0/2173479122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9634004800 msgr2=0x7f9634004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.077+0000 7f965435c640 1 --2- 172.21.15.16:0/2173479122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9634004800 0x7f9634004c80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9640009ac0 tx=0x7f9634005830 comp rx=0 tx=0).stop 2024-04-25T10:07:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.077+0000 7f965435c640 1 -- 172.21.15.16:0/2173479122 shutdown_connections 2024-04-25T10:07:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.077+0000 7f965435c640 1 --2- 172.21.15.16:0/2173479122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9634004800 0x7f9634004c80 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.077+0000 7f965435c640 1 -- 172.21.15.16:0/2173479122 >> 172.21.15.16:0/2173479122 conn(0x7f964c0d0550 msgr2=0x7f964c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.077+0000 7f965435c640 1 -- 172.21.15.16:0/2173479122 shutdown_connections 2024-04-25T10:07:43.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.077+0000 7f965435c640 1 -- 172.21.15.16:0/2173479122 wait complete. 2024-04-25T10:07:43.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.078+0000 7f965435c640 1 Processor -- start 2024-04-25T10:07:43.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.078+0000 7f965435c640 1 -- start start 2024-04-25T10:07:43.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.079+0000 7f965435c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9634004800 0x7f964c0cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.079+0000 7f965435c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c0c7fb0 0x7f964c0c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.079+0000 7f965435c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f964c0b7a30 0x7f964c0b7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.079+0000 7f965435c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c0d1d60 con 0x7f964c0c7fb0 2024-04-25T10:07:43.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.079+0000 7f965435c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c162740 con 0x7f9634004800 2024-04-25T10:07:43.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.079+0000 7f965435c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964c0d2040 con 0x7f964c0b7a30 2024-04-25T10:07:43.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.079+0000 7f96518f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c0c7fb0 0x7f964c0c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.079+0000 7f96518f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c0c7fb0 0x7f964c0c8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60062/0 (socket says 172.21.15.16:60062) 2024-04-25T10:07:43.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.079+0000 7f96518f7640 1 -- 172.21.15.16:0/4238597304 learned_addr learned my addr 172.21.15.16:0/4238597304 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:43.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.079+0000 7f96520f8640 1 --2- 172.21.15.16:0/4238597304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9634004800 0x7f964c0cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.079+0000 7f96528f9640 1 --2- 172.21.15.16:0/4238597304 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f964c0b7a30 0x7f964c0b7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.079+0000 7f96518f7640 1 -- 172.21.15.16:0/4238597304 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f964c0b7a30 msgr2=0x7f964c0b7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.079+0000 7f96518f7640 1 --2- 172.21.15.16:0/4238597304 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f964c0b7a30 0x7f964c0b7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.079+0000 7f96518f7640 1 -- 172.21.15.16:0/4238597304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9634004800 msgr2=0x7f964c0cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.079+0000 7f96518f7640 1 --2- 172.21.15.16:0/4238597304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9634004800 0x7f964c0cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.080+0000 7f96518f7640 1 -- 172.21.15.16:0/4238597304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964c0c8a30 con 0x7f964c0c7fb0 2024-04-25T10:07:43.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.080+0000 7f96528f9640 1 --2- 172.21.15.16:0/4238597304 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f964c0b7a30 0x7f964c0b7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:43.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.080+0000 7f96520f8640 1 --2- 172.21.15.16:0/4238597304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9634004800 0x7f964c0cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:43.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.080+0000 7f96518f7640 1 --2- 172.21.15.16:0/4238597304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c0c7fb0 0x7f964c0c8410 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9640009be0 tx=0x7f9640004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:43.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.080+0000 7f964affd640 1 -- 172.21.15.16:0/4238597304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964003f020 con 0x7f964c0c7fb0 2024-04-25T10:07:43.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.080+0000 7f964affd640 1 -- 172.21.15.16:0/4238597304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9640037040 con 0x7f964c0c7fb0 2024-04-25T10:07:43.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.080+0000 7f965435c640 1 -- 172.21.15.16:0/4238597304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964c0c8d10 con 0x7f964c0c7fb0 2024-04-25T10:07:43.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.080+0000 7f964affd640 1 -- 172.21.15.16:0/4238597304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640036070 con 0x7f964c0c7fb0 2024-04-25T10:07:43.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.080+0000 7f965435c640 1 -- 172.21.15.16:0/4238597304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964c0c92d0 con 0x7f964c0c7fb0 2024-04-25T10:07:43.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.081+0000 7f965435c640 1 -- 172.21.15.16:0/4238597304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964c0b83f0 con 0x7f964c0c7fb0 2024-04-25T10:07:43.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.081+0000 7f964affd640 1 -- 172.21.15.16:0/4238597304 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9640042020 con 0x7f964c0c7fb0 2024-04-25T10:07:43.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.082+0000 7f964affd640 1 --2- 172.21.15.16:0/4238597304 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f963806b4d0 0x7f963806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.082+0000 7f964affd640 1 -- 172.21.15.16:0/4238597304 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f96400a9ea0 con 0x7f964c0c7fb0 2024-04-25T10:07:43.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.082+0000 7f96520f8640 1 --2- 172.21.15.16:0/4238597304 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f963806b4d0 0x7f963806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.083+0000 7f96520f8640 1 --2- 172.21.15.16:0/4238597304 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f963806b4d0 0x7f963806d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f963c000f80 tx=0x7f963c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:43.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.086+0000 7f964affd640 1 -- 172.21.15.16:0/4238597304 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9640049030 con 0x7f964c0c7fb0 2024-04-25T10:07:43.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.271+0000 7f965435c640 1 -- 172.21.15.16:0/4238597304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f964c0750b0 con 0x7f964c0c7fb0 2024-04-25T10:07:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.274+0000 7f964affd640 1 -- 172.21.15.16:0/4238597304 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9640079170 con 0x7f964c0c7fb0 2024-04-25T10:07:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.276+0000 7f965435c640 1 -- 172.21.15.16:0/4238597304 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f963806b4d0 msgr2=0x7f963806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.276+0000 7f965435c640 1 --2- 172.21.15.16:0/4238597304 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f963806b4d0 0x7f963806d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f963c000f80 tx=0x7f963c009040 comp rx=0 tx=0).stop 2024-04-25T10:07:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.276+0000 7f965435c640 1 -- 172.21.15.16:0/4238597304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c0c7fb0 msgr2=0x7f964c0c8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.276+0000 7f965435c640 1 --2- 172.21.15.16:0/4238597304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c0c7fb0 0x7f964c0c8410 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9640009be0 tx=0x7f9640004730 comp rx=0 tx=0).stop 2024-04-25T10:07:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.276+0000 7f965435c640 1 -- 172.21.15.16:0/4238597304 shutdown_connections 2024-04-25T10:07:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.276+0000 7f965435c640 1 --2- 172.21.15.16:0/4238597304 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f963806b4d0 0x7f963806d990 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.276+0000 7f965435c640 1 --2- 172.21.15.16:0/4238597304 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f964c0b7a30 0x7f964c0b7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.276+0000 7f965435c640 1 --2- 172.21.15.16:0/4238597304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c0c7fb0 0x7f964c0c8410 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.276+0000 7f965435c640 1 --2- 172.21.15.16:0/4238597304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9634004800 0x7f964c0cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.276+0000 7f965435c640 1 -- 172.21.15.16:0/4238597304 >> 172.21.15.16:0/4238597304 conn(0x7f964c0d0550 msgr2=0x7f964c156630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.276+0000 7f965435c640 1 -- 172.21.15.16:0/4238597304 shutdown_connections 2024-04-25T10:07:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.276+0000 7f965435c640 1 -- 172.21.15.16:0/4238597304 wait complete. 2024-04-25T10:07:43.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-25T10:07:43.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.387+0000 7f09306b3640 1 Processor -- start 2024-04-25T10:07:43.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.387+0000 7f09306b3640 1 -- start start 2024-04-25T10:07:43.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.388+0000 7f09306b3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f092815dce0 0x7f09280d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.388+0000 7f09306b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09280d35b0 0x7f09280b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.388+0000 7f09306b3640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092815e0c0 con 0x7f09280d1f90 2024-04-25T10:07:43.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.388+0000 7f09306b3640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09280b7b10 con 0x7f092815dce0 2024-04-25T10:07:43.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.388+0000 7f09306b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09280d2510 con 0x7f09280d35b0 2024-04-25T10:07:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.389+0000 7f092dc4e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09280d35b0 0x7f09280b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.389+0000 7f092e44f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f092815dce0 0x7f09280d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.389+0000 7f092ec50640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f09280d1f90 0x7f092806bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:46000/0 (socket says 172.21.15.16:46000) 2024-04-25T10:07:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.389+0000 7f092e44f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f092815dce0 0x7f09280d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:34138/0 (socket says 172.21.15.16:34138) 2024-04-25T10:07:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.389+0000 7f092dc4e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09280d35b0 0x7f09280b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:42834/0 (socket says 172.21.15.16:42834) 2024-04-25T10:07:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.389+0000 7f092ec50640 1 -- 172.21.15.16:0/1646153778 learned_addr learned my addr 172.21.15.16:0/1646153778 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.389+0000 7f092dc4e640 1 -- 172.21.15.16:0/1646153778 >> v2:172.21.15.186:3300/0 conn(0x7f092815dce0 msgr2=0x7f09280d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.389+0000 7f092dc4e640 1 --2- 172.21.15.16:0/1646153778 >> v2:172.21.15.186:3300/0 conn(0x7f092815dce0 0x7f09280d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.389+0000 7f092dc4e640 1 -- 172.21.15.16:0/1646153778 >> v1:172.21.15.186:6789/0 conn(0x7f09280d1f90 legacy=0x7f092806bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.389+0000 7f092dc4e640 1 -- 172.21.15.16:0/1646153778 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09280d27f0 con 0x7f09280d35b0 2024-04-25T10:07:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.389+0000 7f092e44f640 1 --2- 172.21.15.16:0/1646153778 >> v2:172.21.15.186:3300/0 conn(0x7f092815dce0 0x7f09280d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.390+0000 7f092dc4e640 1 --2- 172.21.15.16:0/1646153778 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09280d35b0 0x7f09280b5450 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f091c0097a0 tx=0x7f091c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24d97f5573083850 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.390+0000 7f092d44d640 1 -- 172.21.15.16:0/1646153778 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c037070 con 0x7f09280d35b0 2024-04-25T10:07:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.390+0000 7f092d44d640 1 -- 172.21.15.16:0/1646153778 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c02ae50 con 0x7f09280d35b0 2024-04-25T10:07:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.390+0000 7f092d44d640 1 -- 172.21.15.16:0/1646153778 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c032660 con 0x7f09280d35b0 2024-04-25T10:07:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.391+0000 7f09306b3640 1 -- 172.21.15.16:0/1646153778 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09280d35b0 msgr2=0x7f09280b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.391+0000 7f09306b3640 1 --2- 172.21.15.16:0/1646153778 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09280d35b0 0x7f09280b5450 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f091c0097a0 tx=0x7f091c02a100 comp rx=0 tx=0).stop 2024-04-25T10:07:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.391+0000 7f09306b3640 1 -- 172.21.15.16:0/1646153778 shutdown_connections 2024-04-25T10:07:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.391+0000 7f09306b3640 1 --2- 172.21.15.16:0/1646153778 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09280d35b0 0x7f09280b5450 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.391+0000 7f09306b3640 1 --2- 172.21.15.16:0/1646153778 >> v2:172.21.15.186:3300/0 conn(0x7f092815dce0 0x7f09280d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.391+0000 7f09306b3640 1 -- 172.21.15.16:0/1646153778 >> 172.21.15.16:0/1646153778 conn(0x7f0928072900 msgr2=0x7f09280d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.391+0000 7f09306b3640 1 -- 172.21.15.16:0/1646153778 shutdown_connections 2024-04-25T10:07:43.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.391+0000 7f09306b3640 1 -- 172.21.15.16:0/1646153778 wait complete. 2024-04-25T10:07:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.392+0000 7f09306b3640 1 Processor -- start 2024-04-25T10:07:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.392+0000 7f09306b3640 1 -- start start 2024-04-25T10:07:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f09306b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09280d1f90 0x7f092814ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f09306b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09280d35b0 0x7f09280c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f09306b3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f092815dce0 0x7f09280b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f09306b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09280d3090 con 0x7f09280d35b0 2024-04-25T10:07:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f09306b3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09280d2db0 con 0x7f092815dce0 2024-04-25T10:07:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f09306b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09280b9820 con 0x7f09280d1f90 2024-04-25T10:07:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f092dc4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09280d35b0 0x7f09280c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f092dc4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09280d35b0 0x7f09280c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60078/0 (socket says 172.21.15.16:60078) 2024-04-25T10:07:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f092dc4e640 1 -- 172.21.15.16:0/2689096161 learned_addr learned my addr 172.21.15.16:0/2689096161 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f092ec50640 1 --2- 172.21.15.16:0/2689096161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f092815dce0 0x7f09280b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f092e44f640 1 --2- 172.21.15.16:0/2689096161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09280d1f90 0x7f092814ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f092dc4e640 1 -- 172.21.15.16:0/2689096161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09280d1f90 msgr2=0x7f092814ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f092dc4e640 1 --2- 172.21.15.16:0/2689096161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09280d1f90 0x7f092814ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f092dc4e640 1 -- 172.21.15.16:0/2689096161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f092815dce0 msgr2=0x7f09280b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f092dc4e640 1 --2- 172.21.15.16:0/2689096161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f092815dce0 0x7f09280b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f092dc4e640 1 -- 172.21.15.16:0/2689096161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092815e670 con 0x7f09280d35b0 2024-04-25T10:07:43.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f092dc4e640 1 --2- 172.21.15.16:0/2689096161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09280d35b0 0x7f09280c94e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f091c035040 tx=0x7f091c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:43.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f0926ffd640 1 -- 172.21.15.16:0/2689096161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c041070 con 0x7f09280d35b0 2024-04-25T10:07:43.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f0926ffd640 1 -- 172.21.15.16:0/2689096161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c03c460 con 0x7f09280d35b0 2024-04-25T10:07:43.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f09306b3640 1 -- 172.21.15.16:0/2689096161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092815e950 con 0x7f09280d35b0 2024-04-25T10:07:43.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.393+0000 7f0926ffd640 1 -- 172.21.15.16:0/2689096161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c037040 con 0x7f09280d35b0 2024-04-25T10:07:43.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.394+0000 7f09306b3640 1 -- 172.21.15.16:0/2689096161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092815ef10 con 0x7f09280d35b0 2024-04-25T10:07:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.395+0000 7f09306b3640 1 -- 172.21.15.16:0/2689096161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09280749b0 con 0x7f09280d35b0 2024-04-25T10:07:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.395+0000 7f0926ffd640 1 -- 172.21.15.16:0/2689096161 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f091c03a070 con 0x7f09280d35b0 2024-04-25T10:07:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.395+0000 7f0926ffd640 1 --2- 172.21.15.16:0/2689096161 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f091406b4d0 0x7f091406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.395+0000 7f0926ffd640 1 -- 172.21.15.16:0/2689096161 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f091c0aa390 con 0x7f09280d35b0 2024-04-25T10:07:43.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.395+0000 7f092e44f640 1 --2- 172.21.15.16:0/2689096161 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f091406b4d0 0x7f091406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.396+0000 7f092e44f640 1 --2- 172.21.15.16:0/2689096161 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f091406b4d0 0x7f091406d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0920009560 tx=0x7f0920008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:43.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.399+0000 7f0926ffd640 1 -- 172.21.15.16:0/2689096161 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f091c079510 con 0x7f09280d35b0 2024-04-25T10:07:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.656+0000 7f09306b3640 1 -- 172.21.15.16:0/2689096161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f09280b5b20 con 0x7f09280d35b0 2024-04-25T10:07:43.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.949+0000 7f0926ffd640 1 -- 172.21.15.16:0/2689096161 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v151) v1 ==== 149+0+0 (secure 0 0 0) 0x7f091c078e10 con 0x7f09280d35b0 2024-04-25T10:07:43.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.952+0000 7f09306b3640 1 -- 172.21.15.16:0/2689096161 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f091406b4d0 msgr2=0x7f091406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.952+0000 7f09306b3640 1 --2- 172.21.15.16:0/2689096161 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f091406b4d0 0x7f091406d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0920009560 tx=0x7f0920008040 comp rx=0 tx=0).stop 2024-04-25T10:07:43.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.952+0000 7f09306b3640 1 -- 172.21.15.16:0/2689096161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09280d35b0 msgr2=0x7f09280c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.952+0000 7f09306b3640 1 --2- 172.21.15.16:0/2689096161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09280d35b0 0x7f09280c94e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f091c035040 tx=0x7f091c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.952+0000 7f09306b3640 1 -- 172.21.15.16:0/2689096161 shutdown_connections 2024-04-25T10:07:43.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.952+0000 7f09306b3640 1 --2- 172.21.15.16:0/2689096161 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f091406b4d0 0x7f091406d990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.952+0000 7f09306b3640 1 --2- 172.21.15.16:0/2689096161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f092815dce0 0x7f09280b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.952+0000 7f09306b3640 1 --2- 172.21.15.16:0/2689096161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09280d35b0 0x7f09280c94e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.952+0000 7f09306b3640 1 --2- 172.21.15.16:0/2689096161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09280d1f90 0x7f092814ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.952+0000 7f09306b3640 1 -- 172.21.15.16:0/2689096161 >> 172.21.15.16:0/2689096161 conn(0x7f0928072900 msgr2=0x7f09280b7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:43.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.952+0000 7f09306b3640 1 -- 172.21.15.16:0/2689096161 shutdown_connections 2024-04-25T10:07:43.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:43.952+0000 7f09306b3640 1 -- 172.21.15.16:0/2689096161 wait complete. 2024-04-25T10:07:43.969 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-04-25T10:07:43.969 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T10:07:43.969 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:07:43.970 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.067+0000 7feb339ab640 1 Processor -- start 2024-04-25T10:07:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.067+0000 7feb339ab640 1 -- start start 2024-04-25T10:07:44.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.068+0000 7feb339ab640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb2c0b3040 0x7feb2c0b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.068+0000 7feb339ab640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feb2c06aba0 0x7feb2c06b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.068+0000 7feb339ab640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2c06e6c0 con 0x7feb2c15ec10 2024-04-25T10:07:44.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.068+0000 7feb339ab640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb2c06f2d0 con 0x7feb2c06aba0 2024-04-25T10:07:44.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.068+0000 7feb339ab640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c06e9a0 con 0x7feb2c0b3040 2024-04-25T10:07:44.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.068+0000 7feb31f48640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feb2c06aba0 0x7feb2c06b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.068+0000 7feb31f48640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feb2c06aba0 0x7feb2c06b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60080/0 (socket says 172.21.15.16:60080) 2024-04-25T10:07:44.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.068+0000 7feb31f48640 1 -- 172.21.15.16:0/3720056131 learned_addr learned my addr 172.21.15.16:0/3720056131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:44.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.068+0000 7feb30f46640 1 --2- 172.21.15.16:0/3720056131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb2c0b3040 0x7feb2c0b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.069+0000 7feb31f48640 1 -- 172.21.15.16:0/3720056131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb2c0b3040 msgr2=0x7feb2c0b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:44.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.069+0000 7feb31f48640 1 --2- 172.21.15.16:0/3720056131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb2c0b3040 0x7feb2c0b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.069+0000 7feb31f48640 1 -- 172.21.15.16:0/3720056131 >> v1:172.21.15.186:6789/0 conn(0x7feb2c15ec10 legacy=0x7feb2c06dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.069+0000 7feb31f48640 1 -- 172.21.15.16:0/3720056131 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c06ec80 con 0x7feb2c06aba0 2024-04-25T10:07:44.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.069+0000 7feb30f46640 1 --2- 172.21.15.16:0/3720056131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb2c0b3040 0x7feb2c0b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.069+0000 7feb31f48640 1 --2- 172.21.15.16:0/3720056131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb2c06aba0 0x7feb2c06b030 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7feb24009870 tx=0x7feb2402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d08858900f684e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.069+0000 7feb2bfff640 1 -- 172.21.15.16:0/3720056131 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24037070 con 0x7feb2c06aba0 2024-04-25T10:07:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.069+0000 7feb2bfff640 1 -- 172.21.15.16:0/3720056131 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb24032440 con 0x7feb2c06aba0 2024-04-25T10:07:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.069+0000 7feb2bfff640 1 -- 172.21.15.16:0/3720056131 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24032810 con 0x7feb2c06aba0 2024-04-25T10:07:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.070+0000 7feb339ab640 1 -- 172.21.15.16:0/3720056131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb2c06aba0 msgr2=0x7feb2c06b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.070+0000 7feb339ab640 1 --2- 172.21.15.16:0/3720056131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb2c06aba0 0x7feb2c06b030 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7feb24009870 tx=0x7feb2402a130 comp rx=0 tx=0).stop 2024-04-25T10:07:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.070+0000 7feb339ab640 1 -- 172.21.15.16:0/3720056131 shutdown_connections 2024-04-25T10:07:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.070+0000 7feb339ab640 1 --2- 172.21.15.16:0/3720056131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb2c06aba0 0x7feb2c06b030 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.070+0000 7feb339ab640 1 --2- 172.21.15.16:0/3720056131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb2c0b3040 0x7feb2c0b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.070+0000 7feb339ab640 1 -- 172.21.15.16:0/3720056131 >> 172.21.15.16:0/3720056131 conn(0x7feb2c0b2220 msgr2=0x7feb2c0cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.070+0000 7feb339ab640 1 -- 172.21.15.16:0/3720056131 shutdown_connections 2024-04-25T10:07:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.070+0000 7feb339ab640 1 -- 172.21.15.16:0/3720056131 wait complete. 2024-04-25T10:07:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.071+0000 7feb339ab640 1 Processor -- start 2024-04-25T10:07:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.071+0000 7feb339ab640 1 -- start start 2024-04-25T10:07:44.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.072+0000 7feb339ab640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb2c06aba0 0x7feb2c142b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.072+0000 7feb339ab640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb2c0b3040 0x7feb2c1f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.072+0000 7feb339ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb2c15ec10 0x7feb2c1f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.072+0000 7feb339ab640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c06e380 con 0x7feb2c15ec10 2024-04-25T10:07:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.072+0000 7feb339ab640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c15eff0 con 0x7feb2c06aba0 2024-04-25T10:07:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.072+0000 7feb339ab640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c15e2e0 con 0x7feb2c0b3040 2024-04-25T10:07:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.072+0000 7feb31f48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb2c15ec10 0x7feb2c1f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.072+0000 7feb31f48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb2c15ec10 0x7feb2c1f78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60088/0 (socket says 172.21.15.16:60088) 2024-04-25T10:07:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.072+0000 7feb31f48640 1 -- 172.21.15.16:0/3061114344 learned_addr learned my addr 172.21.15.16:0/3061114344 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.072+0000 7feb30f46640 1 --2- 172.21.15.16:0/3061114344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb2c0b3040 0x7feb2c1f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.072+0000 7feb31747640 1 --2- 172.21.15.16:0/3061114344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb2c06aba0 0x7feb2c142b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.073+0000 7feb31f48640 1 -- 172.21.15.16:0/3061114344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb2c0b3040 msgr2=0x7feb2c1f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.073+0000 7feb31f48640 1 --2- 172.21.15.16:0/3061114344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb2c0b3040 0x7feb2c1f3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.073+0000 7feb31f48640 1 -- 172.21.15.16:0/3061114344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb2c06aba0 msgr2=0x7feb2c142b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.073+0000 7feb31f48640 1 --2- 172.21.15.16:0/3061114344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb2c06aba0 0x7feb2c142b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.073+0000 7feb31f48640 1 -- 172.21.15.16:0/3061114344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c1f43e0 con 0x7feb2c15ec10 2024-04-25T10:07:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.073+0000 7feb31f48640 1 --2- 172.21.15.16:0/3061114344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb2c15ec10 0x7feb2c1f78d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7feb24009870 tx=0x7feb240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.073+0000 7feb29ffb640 1 -- 172.21.15.16:0/3061114344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24037040 con 0x7feb2c15ec10 2024-04-25T10:07:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.073+0000 7feb29ffb640 1 -- 172.21.15.16:0/3061114344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb24042070 con 0x7feb2c15ec10 2024-04-25T10:07:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.073+0000 7feb29ffb640 1 -- 172.21.15.16:0/3061114344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24030920 con 0x7feb2c15ec10 2024-04-25T10:07:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.073+0000 7feb339ab640 1 -- 172.21.15.16:0/3061114344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2c1f46c0 con 0x7feb2c15ec10 2024-04-25T10:07:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.073+0000 7feb339ab640 1 -- 172.21.15.16:0/3061114344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2c1f70d0 con 0x7feb2c15ec10 2024-04-25T10:07:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.074+0000 7feb339ab640 1 -- 172.21.15.16:0/3061114344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2c0749b0 con 0x7feb2c15ec10 2024-04-25T10:07:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.075+0000 7feb29ffb640 1 -- 172.21.15.16:0/3061114344 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb24030ac0 con 0x7feb2c15ec10 2024-04-25T10:07:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.075+0000 7feb29ffb640 1 --2- 172.21.15.16:0/3061114344 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7feb1806b4d0 0x7feb1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.075+0000 7feb29ffb640 1 -- 172.21.15.16:0/3061114344 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7feb240aa680 con 0x7feb2c15ec10 2024-04-25T10:07:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.075+0000 7feb31747640 1 --2- 172.21.15.16:0/3061114344 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7feb1806b4d0 0x7feb1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.076+0000 7feb31747640 1 --2- 172.21.15.16:0/3061114344 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7feb1806b4d0 0x7feb1806d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7feb1c001f10 tx=0x7feb1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.080+0000 7feb29ffb640 1 -- 172.21.15.16:0/3061114344 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb24079240 con 0x7feb2c15ec10 2024-04-25T10:07:44.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.261+0000 7feb339ab640 1 -- 172.21.15.16:0/3061114344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb2c0d0b00 con 0x7feb2c15ec10 2024-04-25T10:07:44.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.262+0000 7feb29ffb640 1 -- 172.21.15.16:0/3061114344 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+13018 (secure 0 0 0) 0x7feb2407d2e0 con 0x7feb2c15ec10 2024-04-25T10:07:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:44.268 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":50,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:07:42.940112+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.186:6837/83423489":"2024-04-26T10:07:39.741694+0000","172.21.15.186:6835/83423489":"2024-04-26T10:07:39.741694+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:44.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.264+0000 7feb339ab640 1 -- 172.21.15.16:0/3061114344 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7feb1806b4d0 msgr2=0x7feb1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.264+0000 7feb339ab640 1 --2- 172.21.15.16:0/3061114344 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7feb1806b4d0 0x7feb1806d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7feb1c001f10 tx=0x7feb1c009040 comp rx=0 tx=0).stop 2024-04-25T10:07:44.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.264+0000 7feb339ab640 1 -- 172.21.15.16:0/3061114344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb2c15ec10 msgr2=0x7feb2c1f78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.264+0000 7feb339ab640 1 --2- 172.21.15.16:0/3061114344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb2c15ec10 0x7feb2c1f78d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7feb24009870 tx=0x7feb240046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.264+0000 7feb339ab640 1 -- 172.21.15.16:0/3061114344 shutdown_connections 2024-04-25T10:07:44.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.264+0000 7feb339ab640 1 --2- 172.21.15.16:0/3061114344 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7feb1806b4d0 0x7feb1806d990 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.264+0000 7feb339ab640 1 --2- 172.21.15.16:0/3061114344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb2c15ec10 0x7feb2c1f78d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.264+0000 7feb339ab640 1 --2- 172.21.15.16:0/3061114344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb2c0b3040 0x7feb2c1f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.264+0000 7feb339ab640 1 --2- 172.21.15.16:0/3061114344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb2c06aba0 0x7feb2c142b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.264+0000 7feb339ab640 1 -- 172.21.15.16:0/3061114344 >> 172.21.15.16:0/3061114344 conn(0x7feb2c0b2220 msgr2=0x7feb2c06dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:44.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.264+0000 7feb339ab640 1 -- 172.21.15.16:0/3061114344 shutdown_connections 2024-04-25T10:07:44.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.264+0000 7feb339ab640 1 -- 172.21.15.16:0/3061114344 wait complete. 2024-04-25T10:07:44.281 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:44.281 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:44.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.379+0000 7f305a231640 1 Processor -- start 2024-04-25T10:07:44.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.379+0000 7f305a231640 1 -- start start 2024-04-25T10:07:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.380+0000 7f305a231640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30540cfe70 0x7f30540ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.380+0000 7f305a231640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f305415f000 con 0x7f305415b7b0 2024-04-25T10:07:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.380+0000 7f305a231640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f305415ed20 con 0x7f305415e110 2024-04-25T10:07:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.380+0000 7f305a231640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30540ced00 con 0x7f30540cfe70 2024-04-25T10:07:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.380+0000 7f30537fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f305415b7b0 0x7f30540cf760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57064/0 (socket says 172.21.15.16:57064) 2024-04-25T10:07:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.380+0000 7f30537fe640 1 -- 172.21.15.16:0/443339724 learned_addr learned my addr 172.21.15.16:0/443339724 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.380+0000 7f30527fc640 1 -- 172.21.15.16:0/443339724 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2654794279 0 0) 0x7f305415f000 con 0x7f305415b7b0 2024-04-25T10:07:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.380+0000 7f30527fc640 1 -- 172.21.15.16:0/443339724 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f303c003520 con 0x7f305415b7b0 2024-04-25T10:07:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.380+0000 7f3052ffd640 1 --2- 172.21.15.16:0/443339724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30540cfe70 0x7f30540ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.380+0000 7f30527fc640 1 -- 172.21.15.16:0/443339724 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1216686699 0 0) 0x7f303c003520 con 0x7f305415b7b0 2024-04-25T10:07:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.380+0000 7f30527fc640 1 -- 172.21.15.16:0/443339724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30540cfe70 msgr2=0x7f30540ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.380+0000 7f30527fc640 1 --2- 172.21.15.16:0/443339724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30540cfe70 0x7f30540ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.380+0000 7f30527fc640 1 -- 172.21.15.16:0/443339724 >> v1:172.21.15.186:6789/0 conn(0x7f305415e110 legacy=0x7f30540cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.380+0000 7f30527fc640 1 -- 172.21.15.16:0/443339724 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30540cdce0 con 0x7f305415b7b0 2024-04-25T10:07:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.380+0000 7f30527fc640 1 -- 172.21.15.16:0/443339724 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f3044001670 con 0x7f305415b7b0 2024-04-25T10:07:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.380+0000 7f30527fc640 1 -- 172.21.15.16:0/443339724 >> v1:172.21.15.16:6789/0 conn(0x7f305415b7b0 legacy=0x7f30540cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.380+0000 7f30527fc640 1 --2- 172.21.15.16:0/443339724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f303c0039e0 0x7f303c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.381+0000 7f30527fc640 1 --2- 172.21.15.16:0/443339724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f303c004800 0x7f303c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.381+0000 7f30527fc640 1 --2- 172.21.15.16:0/443339724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30540cfe70 0x7f303c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.381+0000 7f30527fc640 1 -- 172.21.15.16:0/443339724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30540ced00 con 0x7f303c004800 2024-04-25T10:07:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.381+0000 7f30527fc640 1 -- 172.21.15.16:0/443339724 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c00baa0 con 0x7f303c0039e0 2024-04-25T10:07:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.381+0000 7f30527fc640 1 -- 172.21.15.16:0/443339724 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303c00bca0 con 0x7f30540cfe70 2024-04-25T10:07:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.381+0000 7f3052ffd640 1 --2- 172.21.15.16:0/443339724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f303c004800 0x7f303c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.381+0000 7f30537fe640 1 --2- 172.21.15.16:0/443339724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f303c0039e0 0x7f303c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.381+0000 7f3053fff640 1 --2- 172.21.15.16:0/443339724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30540cfe70 0x7f303c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.381+0000 7f3052ffd640 1 -- 172.21.15.16:0/443339724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30540cfe70 msgr2=0x7f303c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.381+0000 7f3052ffd640 1 --2- 172.21.15.16:0/443339724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30540cfe70 0x7f303c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.381+0000 7f3052ffd640 1 -- 172.21.15.16:0/443339724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f303c0039e0 msgr2=0x7f303c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.381+0000 7f3052ffd640 1 --2- 172.21.15.16:0/443339724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f303c0039e0 0x7f303c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.381+0000 7f3052ffd640 1 -- 172.21.15.16:0/443339724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303c003520 con 0x7f303c004800 2024-04-25T10:07:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.382+0000 7f3052ffd640 1 --2- 172.21.15.16:0/443339724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f303c004800 0x7f303c004c80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3048009a90 tx=0x7f303c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da7db7117d1fa63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.382+0000 7f30527fc640 1 -- 172.21.15.16:0/443339724 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048036070 con 0x7f303c004800 2024-04-25T10:07:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.382+0000 7f30527fc640 1 -- 172.21.15.16:0/443339724 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3048031400 con 0x7f303c004800 2024-04-25T10:07:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.382+0000 7f30527fc640 1 -- 172.21.15.16:0/443339724 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048009690 con 0x7f303c004800 2024-04-25T10:07:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.382+0000 7f305a231640 1 -- 172.21.15.16:0/443339724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f303c004800 msgr2=0x7f303c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.382+0000 7f305a231640 1 --2- 172.21.15.16:0/443339724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f303c004800 0x7f303c004c80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3048009a90 tx=0x7f303c005830 comp rx=0 tx=0).stop 2024-04-25T10:07:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.382+0000 7f3053fff640 1 -- 172.21.15.16:0/443339724 reap_dead start 2024-04-25T10:07:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.382+0000 7f305a231640 1 -- 172.21.15.16:0/443339724 shutdown_connections 2024-04-25T10:07:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.382+0000 7f305a231640 1 -- 172.21.15.16:0/443339724 >> 172.21.15.16:0/443339724 conn(0x7f305406f4a0 msgr2=0x7f305406f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.382+0000 7f305a231640 1 -- 172.21.15.16:0/443339724 shutdown_connections 2024-04-25T10:07:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.382+0000 7f305a231640 1 -- 172.21.15.16:0/443339724 wait complete. 2024-04-25T10:07:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.384+0000 7f305a231640 1 Processor -- start 2024-04-25T10:07:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.384+0000 7f305a231640 1 -- start start 2024-04-25T10:07:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.384+0000 7f305a231640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f303c004800 0x7f30541e71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.384+0000 7f305a231640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3054156dd0 0x7f30541563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f305a231640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30540b1350 0x7f305406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f305a231640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30541619f0 con 0x7f3054156dd0 2024-04-25T10:07:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f305a231640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30540ce330 con 0x7f303c004800 2024-04-25T10:07:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f305a231640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30540ce7c0 con 0x7f30540b1350 2024-04-25T10:07:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f3052ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3054156dd0 0x7f30541563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f3052ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3054156dd0 0x7f30541563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60106/0 (socket says 172.21.15.16:60106) 2024-04-25T10:07:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f3052ffd640 1 -- 172.21.15.16:0/2565017197 learned_addr learned my addr 172.21.15.16:0/2565017197 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f3053fff640 1 --2- 172.21.15.16:0/2565017197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30540b1350 0x7f305406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f3052ffd640 1 -- 172.21.15.16:0/2565017197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30540b1350 msgr2=0x7f305406cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f30537fe640 1 --2- 172.21.15.16:0/2565017197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f303c004800 0x7f30541e71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f3052ffd640 1 --2- 172.21.15.16:0/2565017197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30540b1350 0x7f305406cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f3052ffd640 1 -- 172.21.15.16:0/2565017197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f303c004800 msgr2=0x7f30541e71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f3052ffd640 1 --2- 172.21.15.16:0/2565017197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f303c004800 0x7f30541e71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f3052ffd640 1 -- 172.21.15.16:0/2565017197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30540b1d30 con 0x7f3054156dd0 2024-04-25T10:07:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f3052ffd640 1 --2- 172.21.15.16:0/2565017197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3054156dd0 0x7f30541563b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3048034040 tx=0x7f3048004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f303bfff640 1 -- 172.21.15.16:0/2565017197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048041020 con 0x7f3054156dd0 2024-04-25T10:07:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f303bfff640 1 -- 172.21.15.16:0/2565017197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304803c470 con 0x7f3054156dd0 2024-04-25T10:07:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f303bfff640 1 -- 172.21.15.16:0/2565017197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048036040 con 0x7f3054156dd0 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f305a231640 1 -- 172.21.15.16:0/2565017197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30540b2010 con 0x7f3054156dd0 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.385+0000 7f305a231640 1 -- 172.21.15.16:0/2565017197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30540b2d40 con 0x7f3054156dd0 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.386+0000 7f305a231640 1 -- 172.21.15.16:0/2565017197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30540750b0 con 0x7f3054156dd0 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.387+0000 7f303bfff640 1 -- 172.21.15.16:0/2565017197 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3048044020 con 0x7f3054156dd0 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.387+0000 7f303bfff640 1 --2- 172.21.15.16:0/2565017197 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f304006b4d0 0x7f304006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.387+0000 7f303bfff640 1 -- 172.21.15.16:0/2565017197 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f30480aa370 con 0x7f3054156dd0 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.388+0000 7f30537fe640 1 --2- 172.21.15.16:0/2565017197 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f304006b4d0 0x7f304006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.388+0000 7f30537fe640 1 --2- 172.21.15.16:0/2565017197 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f304006b4d0 0x7f304006d990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f30440014f0 tx=0x7f3044009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.392+0000 7f303bfff640 1 -- 172.21.15.16:0/2565017197 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f304804b070 con 0x7f3054156dd0 2024-04-25T10:07:44.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.571+0000 7f305a231640 1 -- 172.21.15.16:0/2565017197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f30540ccd30 con 0x7f3054156dd0 2024-04-25T10:07:44.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.571+0000 7f303bfff640 1 -- 172.21.15.16:0/2565017197 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3048079580 con 0x7f3054156dd0 2024-04-25T10:07:44.577 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:07:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.573+0000 7f305a231640 1 -- 172.21.15.16:0/2565017197 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f304006b4d0 msgr2=0x7f304006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.573+0000 7f305a231640 1 --2- 172.21.15.16:0/2565017197 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f304006b4d0 0x7f304006d990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f30440014f0 tx=0x7f3044009040 comp rx=0 tx=0).stop 2024-04-25T10:07:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.573+0000 7f305a231640 1 -- 172.21.15.16:0/2565017197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3054156dd0 msgr2=0x7f30541563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.573+0000 7f305a231640 1 --2- 172.21.15.16:0/2565017197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3054156dd0 0x7f30541563b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3048034040 tx=0x7f3048004730 comp rx=0 tx=0).stop 2024-04-25T10:07:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.573+0000 7f305a231640 1 -- 172.21.15.16:0/2565017197 shutdown_connections 2024-04-25T10:07:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.573+0000 7f305a231640 1 --2- 172.21.15.16:0/2565017197 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f304006b4d0 0x7f304006d990 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.573+0000 7f305a231640 1 --2- 172.21.15.16:0/2565017197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f30540b1350 0x7f305406cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.573+0000 7f305a231640 1 --2- 172.21.15.16:0/2565017197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3054156dd0 0x7f30541563b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.573+0000 7f305a231640 1 --2- 172.21.15.16:0/2565017197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f303c004800 0x7f30541e71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.573+0000 7f305a231640 1 -- 172.21.15.16:0/2565017197 >> 172.21.15.16:0/2565017197 conn(0x7f305406f4a0 msgr2=0x7f30540cc040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.573+0000 7f305a231640 1 -- 172.21.15.16:0/2565017197 shutdown_connections 2024-04-25T10:07:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.573+0000 7f305a231640 1 -- 172.21.15.16:0/2565017197 wait complete. 2024-04-25T10:07:44.590 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-25T10:07:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.685+0000 7f08c7cf7640 1 Processor -- start 2024-04-25T10:07:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.685+0000 7f08c7cf7640 1 -- start start 2024-04-25T10:07:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.686+0000 7f08c7cf7640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f08c00d35b0 0x7f08c00b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.686+0000 7f08c7cf7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08c00b5990 0x7f08c00b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.686+0000 7f08c7cf7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c015e0c0 con 0x7f08c015dce0 2024-04-25T10:07:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.686+0000 7f08c7cf7640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c00d2510 con 0x7f08c00d35b0 2024-04-25T10:07:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.686+0000 7f08c7cf7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c00d27f0 con 0x7f08c00b5990 2024-04-25T10:07:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.686+0000 7f08c5a93640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f08c015dce0 0x7f08c00d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57066/0 (socket says 172.21.15.16:57066) 2024-04-25T10:07:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.686+0000 7f08c5a93640 1 -- 172.21.15.16:0/2378649655 learned_addr learned my addr 172.21.15.16:0/2378649655 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.686+0000 7f08c4a91640 1 -- 172.21.15.16:0/2378649655 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909495680 0 0) 0x7f08c015e0c0 con 0x7f08c015dce0 2024-04-25T10:07:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.686+0000 7f08c6294640 1 --2- 172.21.15.16:0/2378649655 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08c00b5990 0x7f08c00b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.686+0000 7f08c5292640 1 --2- 172.21.15.16:0/2378649655 >> v2:172.21.15.186:3300/0 conn(0x7f08c00d35b0 0x7f08c00b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.686+0000 7f08c4a91640 1 -- 172.21.15.16:0/2378649655 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08ac003520 con 0x7f08c015dce0 2024-04-25T10:07:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c4a91640 1 -- 172.21.15.16:0/2378649655 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3430853607 0 0) 0x7f08ac003520 con 0x7f08c015dce0 2024-04-25T10:07:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c4a91640 1 -- 172.21.15.16:0/2378649655 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08c00b5990 msgr2=0x7f08c00b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c4a91640 1 --2- 172.21.15.16:0/2378649655 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08c00b5990 0x7f08c00b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c4a91640 1 -- 172.21.15.16:0/2378649655 >> v2:172.21.15.186:3300/0 conn(0x7f08c00d35b0 msgr2=0x7f08c00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c4a91640 1 --2- 172.21.15.16:0/2378649655 >> v2:172.21.15.186:3300/0 conn(0x7f08c00d35b0 0x7f08c00b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c4a91640 1 -- 172.21.15.16:0/2378649655 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c00d2ad0 con 0x7f08c015dce0 2024-04-25T10:07:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c4a91640 1 -- 172.21.15.16:0/2378649655 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f08bc0019f0 con 0x7f08c015dce0 2024-04-25T10:07:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c4a91640 1 -- 172.21.15.16:0/2378649655 >> v1:172.21.15.16:6789/0 conn(0x7f08c015dce0 legacy=0x7f08c00d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c4a91640 1 --2- 172.21.15.16:0/2378649655 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08c00b5990 0x7f08ac0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c4a91640 1 --2- 172.21.15.16:0/2378649655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08ac003f30 0x7f08ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c4a91640 1 --2- 172.21.15.16:0/2378649655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08ac004d40 0x7f08ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c4a91640 1 -- 172.21.15.16:0/2378649655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c00d2510 con 0x7f08ac004d40 2024-04-25T10:07:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c4a91640 1 -- 172.21.15.16:0/2378649655 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c00d27f0 con 0x7f08ac003f30 2024-04-25T10:07:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c4a91640 1 -- 172.21.15.16:0/2378649655 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac00b9c0 con 0x7f08c00b5990 2024-04-25T10:07:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c6294640 1 --2- 172.21.15.16:0/2378649655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08ac004d40 0x7f08ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c5a93640 1 --2- 172.21.15.16:0/2378649655 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08c00b5990 0x7f08ac0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c5292640 1 --2- 172.21.15.16:0/2378649655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08ac003f30 0x7f08ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c6294640 1 -- 172.21.15.16:0/2378649655 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08c00b5990 msgr2=0x7f08ac0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c6294640 1 --2- 172.21.15.16:0/2378649655 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08c00b5990 0x7f08ac0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c6294640 1 -- 172.21.15.16:0/2378649655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08ac003f30 msgr2=0x7f08ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c6294640 1 --2- 172.21.15.16:0/2378649655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08ac003f30 0x7f08ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c6294640 1 -- 172.21.15.16:0/2378649655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac003520 con 0x7f08ac004d40 2024-04-25T10:07:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c5292640 1 --2- 172.21.15.16:0/2378649655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08ac003f30 0x7f08ac004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.687+0000 7f08c5a93640 1 --2- 172.21.15.16:0/2378649655 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08c00b5990 0x7f08ac0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.688+0000 7f08c6294640 1 --2- 172.21.15.16:0/2378649655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08ac004d40 0x7f08ac0051c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009ac0 tx=0x7f08ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c734909d7eee6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.688+0000 7f08c4a91640 1 -- 172.21.15.16:0/2378649655 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8036070 con 0x7f08ac004d40 2024-04-25T10:07:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.688+0000 7f08c4a91640 1 -- 172.21.15.16:0/2378649655 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b8031400 con 0x7f08ac004d40 2024-04-25T10:07:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.688+0000 7f08c4a91640 1 -- 172.21.15.16:0/2378649655 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8009690 con 0x7f08ac004d40 2024-04-25T10:07:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.689+0000 7f08c7cf7640 1 -- 172.21.15.16:0/2378649655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08ac004d40 msgr2=0x7f08ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.689+0000 7f08c7cf7640 1 --2- 172.21.15.16:0/2378649655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08ac004d40 0x7f08ac0051c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009ac0 tx=0x7f08ac005830 comp rx=0 tx=0).stop 2024-04-25T10:07:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.689+0000 7f08c6294640 1 -- 172.21.15.16:0/2378649655 reap_dead start 2024-04-25T10:07:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.689+0000 7f08c7cf7640 1 -- 172.21.15.16:0/2378649655 shutdown_connections 2024-04-25T10:07:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.689+0000 7f08c7cf7640 1 -- 172.21.15.16:0/2378649655 >> 172.21.15.16:0/2378649655 conn(0x7f08c0072900 msgr2=0x7f08c00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.689+0000 7f08c7cf7640 1 -- 172.21.15.16:0/2378649655 shutdown_connections 2024-04-25T10:07:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.689+0000 7f08c7cf7640 1 -- 172.21.15.16:0/2378649655 wait complete. 2024-04-25T10:07:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.690+0000 7f08c7cf7640 1 Processor -- start 2024-04-25T10:07:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.690+0000 7f08c7cf7640 1 -- start start 2024-04-25T10:07:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08c7cf7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c0160c80 0x7f08c0158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08c7cf7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08c0155750 0x7f08c0148100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08c7cf7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08c0148640 0x7f08c0157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08c7cf7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c00d2160 con 0x7f08c0160c80 2024-04-25T10:07:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08c7cf7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c00d3090 con 0x7f08c0148640 2024-04-25T10:07:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08c7cf7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c00cc550 con 0x7f08c0155750 2024-04-25T10:07:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08c5a93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c0160c80 0x7f08c0158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08c5a93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c0160c80 0x7f08c0158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60110/0 (socket says 172.21.15.16:60110) 2024-04-25T10:07:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08c5a93640 1 -- 172.21.15.16:0/1334231166 learned_addr learned my addr 172.21.15.16:0/1334231166 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08c5a93640 1 -- 172.21.15.16:0/1334231166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08c0155750 msgr2=0x7f08c0148100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08c5a93640 1 --2- 172.21.15.16:0/1334231166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08c0155750 0x7f08c0148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08c5a93640 1 -- 172.21.15.16:0/1334231166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08c0148640 msgr2=0x7f08c0157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08c5a93640 1 --2- 172.21.15.16:0/1334231166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08c0148640 0x7f08c0157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08c5a93640 1 -- 172.21.15.16:0/1334231166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c0148ac0 con 0x7f08c0160c80 2024-04-25T10:07:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08c5a93640 1 --2- 172.21.15.16:0/1334231166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c0160c80 0x7f08c0158630 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00e480 tx=0x7f08bc00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08b67fc640 1 -- 172.21.15.16:0/1334231166 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc013420 con 0x7f08c0160c80 2024-04-25T10:07:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08b67fc640 1 -- 172.21.15.16:0/1334231166 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08bc018070 con 0x7f08c0160c80 2024-04-25T10:07:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08b67fc640 1 -- 172.21.15.16:0/1334231166 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc0137c0 con 0x7f08c0160c80 2024-04-25T10:07:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08c7cf7640 1 -- 172.21.15.16:0/1334231166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c0155ec0 con 0x7f08c0160c80 2024-04-25T10:07:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.691+0000 7f08c7cf7640 1 -- 172.21.15.16:0/1334231166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c00cbf00 con 0x7f08c0160c80 2024-04-25T10:07:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.692+0000 7f08c7cf7640 1 -- 172.21.15.16:0/1334231166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c0074990 con 0x7f08c0160c80 2024-04-25T10:07:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.693+0000 7f08b67fc640 1 -- 172.21.15.16:0/1334231166 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08bc013960 con 0x7f08c0160c80 2024-04-25T10:07:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.693+0000 7f08b67fc640 1 --2- 172.21.15.16:0/1334231166 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f08a406b500 0x7f08a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.693+0000 7f08b67fc640 1 -- 172.21.15.16:0/1334231166 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f08bc08a620 con 0x7f08c0160c80 2024-04-25T10:07:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.694+0000 7f08c5292640 1 --2- 172.21.15.16:0/1334231166 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f08a406b500 0x7f08a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.694+0000 7f08c5292640 1 --2- 172.21.15.16:0/1334231166 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f08a406b500 0x7f08a406d9c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f08c015e750 tx=0x7f08b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.697+0000 7f08b67fc640 1 -- 172.21.15.16:0/1334231166 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f08bc059790 con 0x7f08c0160c80 2024-04-25T10:07:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.944+0000 7f08c7cf7640 1 -- 172.21.15.16:0/1334231166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f08c00cec20 con 0x7f08c0160c80 2024-04-25T10:07:44.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.950+0000 7f08b67fc640 1 -- 172.21.15.16:0/1334231166 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v152) v1 ==== 172+0+0 (secure 0 0 0) 0x7f08bc059090 con 0x7f08c0160c80 2024-04-25T10:07:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.952+0000 7f08c7cf7640 1 -- 172.21.15.16:0/1334231166 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f08a406b500 msgr2=0x7f08a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.952+0000 7f08c7cf7640 1 --2- 172.21.15.16:0/1334231166 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f08a406b500 0x7f08a406d9c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f08c015e750 tx=0x7f08b0008040 comp rx=0 tx=0).stop 2024-04-25T10:07:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.952+0000 7f08c7cf7640 1 -- 172.21.15.16:0/1334231166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c0160c80 msgr2=0x7f08c0158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.952+0000 7f08c7cf7640 1 --2- 172.21.15.16:0/1334231166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c0160c80 0x7f08c0158630 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00e480 tx=0x7f08bc00e570 comp rx=0 tx=0).stop 2024-04-25T10:07:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.952+0000 7f08c7cf7640 1 -- 172.21.15.16:0/1334231166 shutdown_connections 2024-04-25T10:07:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.952+0000 7f08c7cf7640 1 --2- 172.21.15.16:0/1334231166 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f08a406b500 0x7f08a406d9c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.952+0000 7f08c7cf7640 1 --2- 172.21.15.16:0/1334231166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08c0148640 0x7f08c0157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.952+0000 7f08c7cf7640 1 --2- 172.21.15.16:0/1334231166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08c0155750 0x7f08c0148100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.952+0000 7f08c7cf7640 1 --2- 172.21.15.16:0/1334231166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c0160c80 0x7f08c0158630 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.952+0000 7f08c7cf7640 1 -- 172.21.15.16:0/1334231166 >> 172.21.15.16:0/1334231166 conn(0x7f08c0072900 msgr2=0x7f08c00b7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.952+0000 7f08c7cf7640 1 -- 172.21.15.16:0/1334231166 shutdown_connections 2024-04-25T10:07:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:44.952+0000 7f08c7cf7640 1 -- 172.21.15.16:0/1334231166 wait complete. 2024-04-25T10:07:44.969 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:07:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.066+0000 7f3948722640 1 Processor -- start 2024-04-25T10:07:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.066+0000 7f3948722640 1 -- start start 2024-04-25T10:07:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.067+0000 7f3948722640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f394015db50 0x7f39400cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.067+0000 7f3948722640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f394015df30 0x7f394006cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.067+0000 7f3948722640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394006d0e0 con 0x7f39400d12a0 2024-04-25T10:07:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.067+0000 7f3948722640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394006c020 con 0x7f394015db50 2024-04-25T10:07:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.067+0000 7f3948722640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39400d0200 con 0x7f394015df30 2024-04-25T10:07:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.067+0000 7f39464be640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f394015db50 0x7f39400cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.067+0000 7f3946cbf640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f39400d12a0 0x7f3940099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57078/0 (socket says 172.21.15.16:57078) 2024-04-25T10:07:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.067+0000 7f39464be640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f394015db50 0x7f39400cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60118/0 (socket says 172.21.15.16:60118) 2024-04-25T10:07:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.067+0000 7f39464be640 1 -- 172.21.15.16:0/2164165975 learned_addr learned my addr 172.21.15.16:0/2164165975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.068+0000 7f3945cbd640 1 --2- 172.21.15.16:0/2164165975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f394015df30 0x7f394006cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.068+0000 7f39464be640 1 -- 172.21.15.16:0/2164165975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f394015df30 msgr2=0x7f394006cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.068+0000 7f39464be640 1 --2- 172.21.15.16:0/2164165975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f394015df30 0x7f394006cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.068+0000 7f39464be640 1 -- 172.21.15.16:0/2164165975 >> v1:172.21.15.16:6789/0 conn(0x7f39400d12a0 legacy=0x7f3940099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.068+0000 7f39464be640 1 -- 172.21.15.16:0/2164165975 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39400d04e0 con 0x7f394015db50 2024-04-25T10:07:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.068+0000 7f39454bc640 1 -- 172.21.15.16:0/2164165975 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072361320 0 0) 0x7f394006d0e0 con 0x7f39400d12a0 2024-04-25T10:07:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.068+0000 7f39464be640 1 --2- 172.21.15.16:0/2164165975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394015db50 0x7f39400cfb80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3938009870 tx=0x7f393802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a6b865dc8dd85f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.069+0000 7f39454bc640 1 -- 172.21.15.16:0/2164165975 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938037070 con 0x7f394015db50 2024-04-25T10:07:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.069+0000 7f39454bc640 1 -- 172.21.15.16:0/2164165975 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3938032440 con 0x7f394015db50 2024-04-25T10:07:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.069+0000 7f39454bc640 1 -- 172.21.15.16:0/2164165975 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938032810 con 0x7f394015db50 2024-04-25T10:07:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.069+0000 7f3948722640 1 -- 172.21.15.16:0/2164165975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394015db50 msgr2=0x7f39400cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.069+0000 7f3948722640 1 --2- 172.21.15.16:0/2164165975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394015db50 0x7f39400cfb80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3938009870 tx=0x7f393802a270 comp rx=0 tx=0).stop 2024-04-25T10:07:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.069+0000 7f3948722640 1 -- 172.21.15.16:0/2164165975 shutdown_connections 2024-04-25T10:07:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.069+0000 7f3948722640 1 --2- 172.21.15.16:0/2164165975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f394015df30 0x7f394006cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.069+0000 7f3948722640 1 --2- 172.21.15.16:0/2164165975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394015db50 0x7f39400cfb80 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.069+0000 7f3948722640 1 -- 172.21.15.16:0/2164165975 >> 172.21.15.16:0/2164165975 conn(0x7f39400b3430 msgr2=0x7f39400b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:45.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.069+0000 7f3948722640 1 -- 172.21.15.16:0/2164165975 shutdown_connections 2024-04-25T10:07:45.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.069+0000 7f3948722640 1 -- 172.21.15.16:0/2164165975 wait complete. 2024-04-25T10:07:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.070+0000 7f3948722640 1 Processor -- start 2024-04-25T10:07:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.070+0000 7f3948722640 1 -- start start 2024-04-25T10:07:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.071+0000 7f3948722640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39400d12a0 0x7f39401f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.071+0000 7f3948722640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f394015db50 0x7f39400c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.071+0000 7f3948722640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f394015df30 0x7f39401555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.071+0000 7f3948722640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39400d0d80 con 0x7f39400d12a0 2024-04-25T10:07:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.071+0000 7f3948722640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39400d0aa0 con 0x7f394015db50 2024-04-25T10:07:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.071+0000 7f3948722640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394009a150 con 0x7f394015df30 2024-04-25T10:07:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.071+0000 7f39464be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39400d12a0 0x7f39401f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.071+0000 7f39464be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39400d12a0 0x7f39401f3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60130/0 (socket says 172.21.15.16:60130) 2024-04-25T10:07:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.071+0000 7f39464be640 1 -- 172.21.15.16:0/3447914392 learned_addr learned my addr 172.21.15.16:0/3447914392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.071+0000 7f3945cbd640 1 --2- 172.21.15.16:0/3447914392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f394015db50 0x7f39400c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.071+0000 7f3946cbf640 1 --2- 172.21.15.16:0/3447914392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f394015df30 0x7f39401555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.071+0000 7f39464be640 1 -- 172.21.15.16:0/3447914392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f394015df30 msgr2=0x7f39401555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.071+0000 7f39464be640 1 --2- 172.21.15.16:0/3447914392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f394015df30 0x7f39401555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.071+0000 7f39464be640 1 -- 172.21.15.16:0/3447914392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f394015db50 msgr2=0x7f39400c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.071+0000 7f39464be640 1 --2- 172.21.15.16:0/3447914392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f394015db50 0x7f39400c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.071+0000 7f39464be640 1 -- 172.21.15.16:0/3447914392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39400c6e70 con 0x7f39400d12a0 2024-04-25T10:07:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.072+0000 7f3945cbd640 1 --2- 172.21.15.16:0/3447914392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f394015db50 0x7f39400c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.072+0000 7f3946cbf640 1 --2- 172.21.15.16:0/3447914392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f394015df30 0x7f39401555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.072+0000 7f39464be640 1 --2- 172.21.15.16:0/3447914392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39400d12a0 0x7f39401f3900 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3938000c00 tx=0x7f39380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.072+0000 7f3936ffd640 1 -- 172.21.15.16:0/3447914392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938041020 con 0x7f39400d12a0 2024-04-25T10:07:45.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.072+0000 7f3936ffd640 1 -- 172.21.15.16:0/3447914392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39380306c0 con 0x7f39400d12a0 2024-04-25T10:07:45.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.072+0000 7f3936ffd640 1 -- 172.21.15.16:0/3447914392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938037040 con 0x7f39400d12a0 2024-04-25T10:07:45.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.072+0000 7f3948722640 1 -- 172.21.15.16:0/3447914392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39400c7150 con 0x7f39400d12a0 2024-04-25T10:07:45.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.072+0000 7f3948722640 1 -- 172.21.15.16:0/3447914392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39400c7710 con 0x7f39400d12a0 2024-04-25T10:07:45.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.073+0000 7f3948722640 1 -- 172.21.15.16:0/3447914392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39400750b0 con 0x7f39400d12a0 2024-04-25T10:07:45.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.073+0000 7f3936ffd640 1 -- 172.21.15.16:0/3447914392 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3938044020 con 0x7f39400d12a0 2024-04-25T10:07:45.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.074+0000 7f3936ffd640 1 --2- 172.21.15.16:0/3447914392 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f392406b4d0 0x7f392406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.074+0000 7f3936ffd640 1 -- 172.21.15.16:0/3447914392 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f39380aa690 con 0x7f39400d12a0 2024-04-25T10:07:45.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.074+0000 7f3945cbd640 1 --2- 172.21.15.16:0/3447914392 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f392406b4d0 0x7f392406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.075+0000 7f3945cbd640 1 --2- 172.21.15.16:0/3447914392 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f392406b4d0 0x7f392406d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3930006fd0 tx=0x7f3930008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:45.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.078+0000 7f3936ffd640 1 -- 172.21.15.16:0/3447914392 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f393804b030 con 0x7f39400d12a0 2024-04-25T10:07:45.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.256+0000 7f3948722640 1 -- 172.21.15.16:0/3447914392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f39400cefa0 con 0x7f39400d12a0 2024-04-25T10:07:45.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.259+0000 7f3936ffd640 1 -- 172.21.15.16:0/3447914392 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f39380798a0 con 0x7f39400d12a0 2024-04-25T10:07:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.261+0000 7f3948722640 1 -- 172.21.15.16:0/3447914392 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f392406b4d0 msgr2=0x7f392406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.261+0000 7f3948722640 1 --2- 172.21.15.16:0/3447914392 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f392406b4d0 0x7f392406d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3930006fd0 tx=0x7f3930008040 comp rx=0 tx=0).stop 2024-04-25T10:07:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.261+0000 7f3948722640 1 -- 172.21.15.16:0/3447914392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39400d12a0 msgr2=0x7f39401f3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.261+0000 7f3948722640 1 --2- 172.21.15.16:0/3447914392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39400d12a0 0x7f39401f3900 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3938000c00 tx=0x7f39380046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.261+0000 7f3948722640 1 -- 172.21.15.16:0/3447914392 shutdown_connections 2024-04-25T10:07:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.261+0000 7f3948722640 1 --2- 172.21.15.16:0/3447914392 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f392406b4d0 0x7f392406d990 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.261+0000 7f3948722640 1 --2- 172.21.15.16:0/3447914392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f394015df30 0x7f39401555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.261+0000 7f3948722640 1 --2- 172.21.15.16:0/3447914392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f394015db50 0x7f39400c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.261+0000 7f3948722640 1 --2- 172.21.15.16:0/3447914392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39400d12a0 0x7f39401f3900 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.261+0000 7f3948722640 1 -- 172.21.15.16:0/3447914392 >> 172.21.15.16:0/3447914392 conn(0x7f39400b3430 msgr2=0x7f394006ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.261+0000 7f3948722640 1 -- 172.21.15.16:0/3447914392 shutdown_connections 2024-04-25T10:07:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.261+0000 7f3948722640 1 -- 172.21.15.16:0/3447914392 wait complete. 2024-04-25T10:07:45.278 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:07:45.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:45.327 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:45.365 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:07:45.365 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:07:45.365 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:45.381 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:07:45.381 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:07:45.381 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:45.422 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:45.423 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:07:45.423 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:45.438 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:07:45.439 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:07:45.439 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:45.480 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:45.480 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:07:45.480 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.575+0000 7f632c0ac640 1 Processor -- start 2024-04-25T10:07:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.575+0000 7f632c0ac640 1 -- start start 2024-04-25T10:07:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.576+0000 7f632c0ac640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63241616f0 0x7f63240ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.576+0000 7f632c0ac640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f632406aba0 0x7f632406b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.576+0000 7f632c0ac640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6324161bf0 con 0x7f6324160f00 2024-04-25T10:07:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.576+0000 7f632c0ac640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f632406fab0 con 0x7f632406aba0 2024-04-25T10:07:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.576+0000 7f632c0ac640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6324161ed0 con 0x7f63241616f0 2024-04-25T10:07:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.576+0000 7f6329e48640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6324160f00 0x7f6324072a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57084/0 (socket says 172.21.15.16:57084) 2024-04-25T10:07:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.576+0000 7f6329e48640 1 -- 172.21.15.16:0/2607005787 learned_addr learned my addr 172.21.15.16:0/2607005787 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6329647640 1 --2- 172.21.15.16:0/2607005787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63241616f0 0x7f63240ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f632a649640 1 --2- 172.21.15.16:0/2607005787 >> v2:172.21.15.186:3300/0 conn(0x7f632406aba0 0x7f632406b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6328e46640 1 -- 172.21.15.16:0/2607005787 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586302479 0 0) 0x7f6324161bf0 con 0x7f6324160f00 2024-04-25T10:07:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6328e46640 1 -- 172.21.15.16:0/2607005787 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f630c003520 con 0x7f6324160f00 2024-04-25T10:07:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6328e46640 1 -- 172.21.15.16:0/2607005787 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2265180857 0 0) 0x7f630c003520 con 0x7f6324160f00 2024-04-25T10:07:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6328e46640 1 -- 172.21.15.16:0/2607005787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63241616f0 msgr2=0x7f63240ce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6328e46640 1 --2- 172.21.15.16:0/2607005787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63241616f0 0x7f63240ce640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6328e46640 1 -- 172.21.15.16:0/2607005787 >> v2:172.21.15.186:3300/0 conn(0x7f632406aba0 msgr2=0x7f632406b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6328e46640 1 --2- 172.21.15.16:0/2607005787 >> v2:172.21.15.186:3300/0 conn(0x7f632406aba0 0x7f632406b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6328e46640 1 -- 172.21.15.16:0/2607005787 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63241621b0 con 0x7f6324160f00 2024-04-25T10:07:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6328e46640 1 -- 172.21.15.16:0/2607005787 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f63140019f0 con 0x7f6324160f00 2024-04-25T10:07:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6328e46640 1 -- 172.21.15.16:0/2607005787 >> v1:172.21.15.16:6789/0 conn(0x7f6324160f00 legacy=0x7f6324072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6328e46640 1 --2- 172.21.15.16:0/2607005787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f630c0039b0 0x7f630c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6328e46640 1 --2- 172.21.15.16:0/2607005787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63241616f0 0x7f630c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6328e46640 1 --2- 172.21.15.16:0/2607005787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f630c004d40 0x7f630c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6328e46640 1 -- 172.21.15.16:0/2607005787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632406fab0 con 0x7f630c0039b0 2024-04-25T10:07:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6328e46640 1 -- 172.21.15.16:0/2607005787 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6324161ed0 con 0x7f630c004d40 2024-04-25T10:07:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6328e46640 1 -- 172.21.15.16:0/2607005787 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630c00b9c0 con 0x7f63241616f0 2024-04-25T10:07:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6329e48640 1 --2- 172.21.15.16:0/2607005787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f630c0039b0 0x7f630c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6329647640 1 --2- 172.21.15.16:0/2607005787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63241616f0 0x7f630c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f632a649640 1 --2- 172.21.15.16:0/2607005787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f630c004d40 0x7f630c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6329e48640 1 -- 172.21.15.16:0/2607005787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63241616f0 msgr2=0x7f630c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6329e48640 1 --2- 172.21.15.16:0/2607005787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63241616f0 0x7f630c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6329e48640 1 -- 172.21.15.16:0/2607005787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f630c004d40 msgr2=0x7f630c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6329e48640 1 --2- 172.21.15.16:0/2607005787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f630c004d40 0x7f630c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f6329e48640 1 -- 172.21.15.16:0/2607005787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c003520 con 0x7f630c0039b0 2024-04-25T10:07:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.577+0000 7f632a649640 1 --2- 172.21.15.16:0/2607005787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f630c004d40 0x7f630c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.578+0000 7f6329e48640 1 --2- 172.21.15.16:0/2607005787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f630c0039b0 0x7f630c003dd0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f631400a8b0 tx=0x7f630c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb4e872734036ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.578+0000 7f6328e46640 1 -- 172.21.15.16:0/2607005787 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631400a4a0 con 0x7f630c0039b0 2024-04-25T10:07:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.578+0000 7f6328e46640 1 -- 172.21.15.16:0/2607005787 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631402f070 con 0x7f630c0039b0 2024-04-25T10:07:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.578+0000 7f6328e46640 1 -- 172.21.15.16:0/2607005787 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631402bcf0 con 0x7f630c0039b0 2024-04-25T10:07:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.579+0000 7f632c0ac640 1 -- 172.21.15.16:0/2607005787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f630c0039b0 msgr2=0x7f630c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.579+0000 7f632c0ac640 1 --2- 172.21.15.16:0/2607005787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f630c0039b0 0x7f630c003dd0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f631400a8b0 tx=0x7f630c005830 comp rx=0 tx=0).stop 2024-04-25T10:07:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.579+0000 7f632a649640 1 -- 172.21.15.16:0/2607005787 reap_dead start 2024-04-25T10:07:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.579+0000 7f632c0ac640 1 -- 172.21.15.16:0/2607005787 shutdown_connections 2024-04-25T10:07:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.579+0000 7f632c0ac640 1 -- 172.21.15.16:0/2607005787 >> 172.21.15.16:0/2607005787 conn(0x7f63240b2370 msgr2=0x7f63240b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.579+0000 7f632c0ac640 1 -- 172.21.15.16:0/2607005787 shutdown_connections 2024-04-25T10:07:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.579+0000 7f632c0ac640 1 -- 172.21.15.16:0/2607005787 wait complete. 2024-04-25T10:07:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.580+0000 7f632c0ac640 1 Processor -- start 2024-04-25T10:07:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.580+0000 7f632c0ac640 1 -- start start 2024-04-25T10:07:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.581+0000 7f632c0ac640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f630c0039b0 0x7f63241f0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.581+0000 7f632c0ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6324072e80 0x7f63240af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.581+0000 7f632c0ac640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63241f37d0 0x7f63240b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.581+0000 7f632c0ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6324161460 con 0x7f6324072e80 2024-04-25T10:07:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.581+0000 7f632c0ac640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63241612e0 con 0x7f63241f37d0 2024-04-25T10:07:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.581+0000 7f632c0ac640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63240ce640 con 0x7f630c0039b0 2024-04-25T10:07:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.581+0000 7f6329647640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6324072e80 0x7f63240af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.581+0000 7f6329647640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6324072e80 0x7f63240af1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60142/0 (socket says 172.21.15.16:60142) 2024-04-25T10:07:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.581+0000 7f6329647640 1 -- 172.21.15.16:0/1348683905 learned_addr learned my addr 172.21.15.16:0/1348683905 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.581+0000 7f6329647640 1 -- 172.21.15.16:0/1348683905 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f630c0039b0 msgr2=0x7f63241f0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.581+0000 7f6329e48640 1 --2- 172.21.15.16:0/1348683905 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f630c0039b0 0x7f63241f0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.581+0000 7f632a649640 1 --2- 172.21.15.16:0/1348683905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63241f37d0 0x7f63240b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.581+0000 7f6329647640 1 --2- 172.21.15.16:0/1348683905 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f630c0039b0 0x7f63241f0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.581+0000 7f6329647640 1 -- 172.21.15.16:0/1348683905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63241f37d0 msgr2=0x7f63240b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.581+0000 7f6329647640 1 --2- 172.21.15.16:0/1348683905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63241f37d0 0x7f63240b5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.581+0000 7f6329647640 1 -- 172.21.15.16:0/1348683905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63241f4750 con 0x7f6324072e80 2024-04-25T10:07:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.581+0000 7f6329647640 1 --2- 172.21.15.16:0/1348683905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6324072e80 0x7f63240af1d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f63180119a0 tx=0x7f6318011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.582+0000 7f63227fc640 1 -- 172.21.15.16:0/1348683905 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631800faf0 con 0x7f6324072e80 2024-04-25T10:07:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.582+0000 7f63227fc640 1 -- 172.21.15.16:0/1348683905 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631800fc90 con 0x7f6324072e80 2024-04-25T10:07:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.582+0000 7f632c0ac640 1 -- 172.21.15.16:0/1348683905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63241f4190 con 0x7f6324072e80 2024-04-25T10:07:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.582+0000 7f63227fc640 1 -- 172.21.15.16:0/1348683905 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318014440 con 0x7f6324072e80 2024-04-25T10:07:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.582+0000 7f632c0ac640 1 -- 172.21.15.16:0/1348683905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63240ae9d0 con 0x7f6324072e80 2024-04-25T10:07:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.583+0000 7f632c0ac640 1 -- 172.21.15.16:0/1348683905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63240749b0 con 0x7f6324072e80 2024-04-25T10:07:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.583+0000 7f63227fc640 1 -- 172.21.15.16:0/1348683905 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6318012040 con 0x7f6324072e80 2024-04-25T10:07:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.584+0000 7f63227fc640 1 --2- 172.21.15.16:0/1348683905 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f631006b4d0 0x7f631006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.584+0000 7f63227fc640 1 -- 172.21.15.16:0/1348683905 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f631808a610 con 0x7f6324072e80 2024-04-25T10:07:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.584+0000 7f6329e48640 1 --2- 172.21.15.16:0/1348683905 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f631006b4d0 0x7f631006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.584+0000 7f6329e48640 1 --2- 172.21.15.16:0/1348683905 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f631006b4d0 0x7f631006d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6314033000 tx=0x7f6314007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.588+0000 7f63227fc640 1 -- 172.21.15.16:0/1348683905 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6318019030 con 0x7f6324072e80 2024-04-25T10:07:45.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.809+0000 7f632c0ac640 1 -- 172.21.15.16:0/1348683905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63240c5960 con 0x7f6324072e80 2024-04-25T10:07:45.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.810+0000 7f63227fc640 1 -- 172.21.15.16:0/1348683905 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f63180590a0 con 0x7f6324072e80 2024-04-25T10:07:45.816 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:45.816 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":23,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/791079479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":791079479},{"type":"v1","addr":"172.21.15.16:6835","nonce":791079479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1747921236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1747921236},{"type":"v1","addr":"172.21.15.16:6834","nonce":1747921236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":34880,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/3183779257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3183779257},{"type":"v1","addr":"172.21.15.186:6836","nonce":3183779257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":35096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2933772732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":2933772732},{"type":"v1","addr":"172.21.15.186:6837","nonce":2933772732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[]} 2024-04-25T10:07:45.816 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 23 2024-04-25T10:07:45.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.812+0000 7f632c0ac640 1 -- 172.21.15.16:0/1348683905 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f631006b4d0 msgr2=0x7f631006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.812+0000 7f632c0ac640 1 --2- 172.21.15.16:0/1348683905 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f631006b4d0 0x7f631006d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6314033000 tx=0x7f6314007000 comp rx=0 tx=0).stop 2024-04-25T10:07:45.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.812+0000 7f632c0ac640 1 -- 172.21.15.16:0/1348683905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6324072e80 msgr2=0x7f63240af1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.812+0000 7f632c0ac640 1 --2- 172.21.15.16:0/1348683905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6324072e80 0x7f63240af1d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f63180119a0 tx=0x7f6318011e60 comp rx=0 tx=0).stop 2024-04-25T10:07:45.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.812+0000 7f632c0ac640 1 -- 172.21.15.16:0/1348683905 shutdown_connections 2024-04-25T10:07:45.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.812+0000 7f632c0ac640 1 --2- 172.21.15.16:0/1348683905 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f631006b4d0 0x7f631006d990 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.812+0000 7f632c0ac640 1 --2- 172.21.15.16:0/1348683905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63241f37d0 0x7f63240b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.812+0000 7f632c0ac640 1 --2- 172.21.15.16:0/1348683905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6324072e80 0x7f63240af1d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.812+0000 7f632c0ac640 1 --2- 172.21.15.16:0/1348683905 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f630c0039b0 0x7f63241f0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.812+0000 7f632c0ac640 1 -- 172.21.15.16:0/1348683905 >> 172.21.15.16:0/1348683905 conn(0x7f63240b2370 msgr2=0x7f632415b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:45.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.812+0000 7f632c0ac640 1 -- 172.21.15.16:0/1348683905 shutdown_connections 2024-04-25T10:07:45.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.812+0000 7f632c0ac640 1 -- 172.21.15.16:0/1348683905 wait complete. 2024-04-25T10:07:45.828 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:07:45.829 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:07:45.829 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:07:45.829 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:45.829 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:07:45.829 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:07:45.829 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:07:45.829 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:45.830 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:07:45.830 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:07:45.830 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:07:45.830 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:45.830 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:07:45.830 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:07:45.831 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:07:45.831 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:45.838 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:07:45.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:07:45.839 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:07:45.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:07:45.842 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:07:45.843 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:07:45.843 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:07:45.843 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:07:45.881 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:07:45.881 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:45.882 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:07:45.882 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:45.885 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:07:45.886 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:45.886 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:07:45.886 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:45.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:07:45.956 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-04-25T10:07:46.010 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-04-25T10:07:46.013 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2024-04-25T10:07:46.013 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2024-04-25T10:07:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.986+0000 7ff5ed1f5640 1 Processor -- start 2024-04-25T10:07:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.986+0000 7ff5ed1f5640 1 -- start start 2024-04-25T10:07:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.986+0000 7ff5ed1f5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff5e815ef60 0x7ff5e80ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.986+0000 7ff5ed1f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5e80cea00 0x7ff5e80b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.987+0000 7ff5ed1f5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e815f660 con 0x7ff5e80b0c80 2024-04-25T10:07:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.987+0000 7ff5ed1f5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e80b10d0 con 0x7ff5e815ef60 2024-04-25T10:07:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.987+0000 7ff5ed1f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e815f940 con 0x7ff5e80cea00 2024-04-25T10:07:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.987+0000 7ff5e6d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff5e815ef60 0x7ff5e80ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.987+0000 7ff5e6d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff5e815ef60 0x7ff5e80ce4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60176/0 (socket says 172.21.15.16:60176) 2024-04-25T10:07:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.987+0000 7ff5e6d9d640 1 -- 172.21.15.16:0/1262070057 learned_addr learned my addr 172.21.15.16:0/1262070057 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.987+0000 7ff5e659c640 1 --2- 172.21.15.16:0/1262070057 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5e80cea00 0x7ff5e80b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.987+0000 7ff5e6d9d640 1 -- 172.21.15.16:0/1262070057 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5e80cea00 msgr2=0x7ff5e80b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.987+0000 7ff5e6d9d640 1 --2- 172.21.15.16:0/1262070057 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5e80cea00 0x7ff5e80b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.987+0000 7ff5e6d9d640 1 -- 172.21.15.16:0/1262070057 >> v1:172.21.15.186:6789/0 conn(0x7ff5e80b0c80 legacy=0x7ff5e80b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.987+0000 7ff5e6d9d640 1 -- 172.21.15.16:0/1262070057 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e815fc20 con 0x7ff5e815ef60 2024-04-25T10:07:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.988+0000 7ff5e6d9d640 1 --2- 172.21.15.16:0/1262070057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5e815ef60 0x7ff5e80ce4c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009870 tx=0x7ff5d802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fe9dc346571ad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.988+0000 7ff5e5d9b640 1 -- 172.21.15.16:0/1262070057 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8037070 con 0x7ff5e815ef60 2024-04-25T10:07:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.988+0000 7ff5e5d9b640 1 -- 172.21.15.16:0/1262070057 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d802ae50 con 0x7ff5e815ef60 2024-04-25T10:07:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.988+0000 7ff5e5d9b640 1 -- 172.21.15.16:0/1262070057 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8032860 con 0x7ff5e815ef60 2024-04-25T10:07:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.988+0000 7ff5ed1f5640 1 -- 172.21.15.16:0/1262070057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5e815ef60 msgr2=0x7ff5e80ce4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.988+0000 7ff5ed1f5640 1 --2- 172.21.15.16:0/1262070057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5e815ef60 0x7ff5e80ce4c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009870 tx=0x7ff5d802a300 comp rx=0 tx=0).stop 2024-04-25T10:07:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.988+0000 7ff5ed1f5640 1 -- 172.21.15.16:0/1262070057 shutdown_connections 2024-04-25T10:07:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.988+0000 7ff5ed1f5640 1 --2- 172.21.15.16:0/1262070057 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5e80cea00 0x7ff5e80b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.988+0000 7ff5ed1f5640 1 --2- 172.21.15.16:0/1262070057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5e815ef60 0x7ff5e80ce4c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.988+0000 7ff5ed1f5640 1 -- 172.21.15.16:0/1262070057 >> 172.21.15.16:0/1262070057 conn(0x7ff5e806f4c0 msgr2=0x7ff5e806f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.988+0000 7ff5ed1f5640 1 -- 172.21.15.16:0/1262070057 shutdown_connections 2024-04-25T10:07:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.988+0000 7ff5ed1f5640 1 -- 172.21.15.16:0/1262070057 wait complete. 2024-04-25T10:07:46.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.990+0000 7ff5ed1f5640 1 Processor -- start 2024-04-25T10:07:46.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.990+0000 7ff5ed1f5640 1 -- start start 2024-04-25T10:07:46.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5ed1f5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e80b0c80 0x7ff5e8148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5ed1f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5e80cea00 0x7ff5e81f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5ed1f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5e815ef60 0x7ff5e80b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5ed1f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e81604c0 con 0x7ff5e815ef60 2024-04-25T10:07:46.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5ed1f5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e81601e0 con 0x7ff5e80b0c80 2024-04-25T10:07:46.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5ed1f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e815f340 con 0x7ff5e80cea00 2024-04-25T10:07:46.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5e759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5e815ef60 0x7ff5e80b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5e759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5e815ef60 0x7ff5e80b2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60188/0 (socket says 172.21.15.16:60188) 2024-04-25T10:07:46.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5e759e640 1 -- 172.21.15.16:0/1056359059 learned_addr learned my addr 172.21.15.16:0/1056359059 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:46.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5e659c640 1 --2- 172.21.15.16:0/1056359059 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5e80cea00 0x7ff5e81f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5e6d9d640 1 --2- 172.21.15.16:0/1056359059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e80b0c80 0x7ff5e8148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5e759e640 1 -- 172.21.15.16:0/1056359059 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5e80cea00 msgr2=0x7ff5e81f5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:46.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5e759e640 1 --2- 172.21.15.16:0/1056359059 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5e80cea00 0x7ff5e81f5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5e759e640 1 -- 172.21.15.16:0/1056359059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e80b0c80 msgr2=0x7ff5e8148e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:46.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5e759e640 1 --2- 172.21.15.16:0/1056359059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e80b0c80 0x7ff5e8148e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5e759e640 1 -- 172.21.15.16:0/1056359059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e80c69d0 con 0x7ff5e815ef60 2024-04-25T10:07:46.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5e6d9d640 1 --2- 172.21.15.16:0/1056359059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e80b0c80 0x7ff5e8148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:46.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5e759e640 1 --2- 172.21.15.16:0/1056359059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5e815ef60 0x7ff5e80b2aa0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000c7d0 tx=0x7ff5e000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:46.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.991+0000 7ff5cf7fe640 1 -- 172.21.15.16:0/1056359059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0003b70 con 0x7ff5e815ef60 2024-04-25T10:07:46.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.992+0000 7ff5cf7fe640 1 -- 172.21.15.16:0/1056359059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e0003d30 con 0x7ff5e815ef60 2024-04-25T10:07:46.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.992+0000 7ff5ed1f5640 1 -- 172.21.15.16:0/1056359059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e80c6cb0 con 0x7ff5e815ef60 2024-04-25T10:07:46.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.992+0000 7ff5cf7fe640 1 -- 172.21.15.16:0/1056359059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e000aa90 con 0x7ff5e815ef60 2024-04-25T10:07:46.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.992+0000 7ff5ed1f5640 1 -- 172.21.15.16:0/1056359059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e80c6f90 con 0x7ff5e815ef60 2024-04-25T10:07:46.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.993+0000 7ff5ed1f5640 1 -- 172.21.15.16:0/1056359059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e8067790 con 0x7ff5e815ef60 2024-04-25T10:07:46.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.993+0000 7ff5cf7fe640 1 -- 172.21.15.16:0/1056359059 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5e000ac30 con 0x7ff5e815ef60 2024-04-25T10:07:46.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.994+0000 7ff5cf7fe640 1 --2- 172.21.15.16:0/1056359059 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff5d406b500 0x7ff5d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.994+0000 7ff5cf7fe640 1 -- 172.21.15.16:0/1056359059 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff5e0089e40 con 0x7ff5e815ef60 2024-04-25T10:07:46.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.994+0000 7ff5e6d9d640 1 --2- 172.21.15.16:0/1056359059 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff5d406b500 0x7ff5d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.994+0000 7ff5e6d9d640 1 --2- 172.21.15.16:0/1056359059 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff5d406b500 0x7ff5d406d9c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8034000 tx=0x7ff5d802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:46.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:45.998+0000 7ff5cf7fe640 1 -- 172.21.15.16:0/1056359059 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5e8067790 con 0x7ff5e815ef60 2024-04-25T10:07:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:46.192+0000 7ff5ed1f5640 1 -- 172.21.15.16:0/1056359059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff5e815d950 con 0x7ff5e815ef60 2024-04-25T10:07:47.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.280+0000 7ff5cf7fe640 1 -- 172.21.15.16:0/1056359059 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v55) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff5e0059050 con 0x7ff5e815ef60 2024-04-25T10:07:47.286 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-04-25T10:07:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.283+0000 7ff5ed1f5640 1 -- 172.21.15.16:0/1056359059 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff5d406b500 msgr2=0x7ff5d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.283+0000 7ff5ed1f5640 1 --2- 172.21.15.16:0/1056359059 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff5d406b500 0x7ff5d406d9c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8034000 tx=0x7ff5d802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.283+0000 7ff5ed1f5640 1 -- 172.21.15.16:0/1056359059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5e815ef60 msgr2=0x7ff5e80b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.283+0000 7ff5ed1f5640 1 --2- 172.21.15.16:0/1056359059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5e815ef60 0x7ff5e80b2aa0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000c7d0 tx=0x7ff5e000cc90 comp rx=0 tx=0).stop 2024-04-25T10:07:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.283+0000 7ff5ed1f5640 1 -- 172.21.15.16:0/1056359059 shutdown_connections 2024-04-25T10:07:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.283+0000 7ff5ed1f5640 1 --2- 172.21.15.16:0/1056359059 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff5d406b500 0x7ff5d406d9c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.283+0000 7ff5ed1f5640 1 --2- 172.21.15.16:0/1056359059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5e815ef60 0x7ff5e80b2aa0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.283+0000 7ff5ed1f5640 1 --2- 172.21.15.16:0/1056359059 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5e80cea00 0x7ff5e81f5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.283+0000 7ff5ed1f5640 1 --2- 172.21.15.16:0/1056359059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e80b0c80 0x7ff5e8148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.283+0000 7ff5ed1f5640 1 -- 172.21.15.16:0/1056359059 >> 172.21.15.16:0/1056359059 conn(0x7ff5e806f4c0 msgr2=0x7ff5e815e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.283+0000 7ff5ed1f5640 1 -- 172.21.15.16:0/1056359059 shutdown_connections 2024-04-25T10:07:47.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.283+0000 7ff5ed1f5640 1 -- 172.21.15.16:0/1056359059 wait complete. 2024-04-25T10:07:47.303 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:07:47.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.404+0000 7fc33dd34640 1 Processor -- start 2024-04-25T10:07:47.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.404+0000 7fc33dd34640 1 -- start start 2024-04-25T10:07:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.405+0000 7fc33dd34640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc33809a150 0x7fc33806bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.405+0000 7fc33dd34640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc33806c660 con 0x7fc33806c1f0 2024-04-25T10:07:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.405+0000 7fc33dd34640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc33806c900 con 0x7fc33815ba10 2024-04-25T10:07:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.405+0000 7fc33dd34640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3380cbe50 con 0x7fc33809a150 2024-04-25T10:07:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.405+0000 7fc337fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc33806c1f0 0x7fc3380cd220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57090/0 (socket says 172.21.15.16:57090) 2024-04-25T10:07:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.405+0000 7fc337fff640 1 -- 172.21.15.16:0/3480456172 learned_addr learned my addr 172.21.15.16:0/3480456172 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.405+0000 7fc336ffd640 1 --2- 172.21.15.16:0/3480456172 >> v2:172.21.15.186:3300/0 conn(0x7fc33809a150 0x7fc33806bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.405+0000 7fc3367fc640 1 -- 172.21.15.16:0/3480456172 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4098161899 0 0) 0x7fc33806c660 con 0x7fc33806c1f0 2024-04-25T10:07:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.405+0000 7fc3367fc640 1 -- 172.21.15.16:0/3480456172 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc320003520 con 0x7fc33806c1f0 2024-04-25T10:07:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc3367fc640 1 -- 172.21.15.16:0/3480456172 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1053234712 0 0) 0x7fc320003520 con 0x7fc33806c1f0 2024-04-25T10:07:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc3367fc640 1 -- 172.21.15.16:0/3480456172 >> v2:172.21.15.186:3300/0 conn(0x7fc33809a150 msgr2=0x7fc33806bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc3367fc640 1 --2- 172.21.15.16:0/3480456172 >> v2:172.21.15.186:3300/0 conn(0x7fc33809a150 0x7fc33806bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc3367fc640 1 -- 172.21.15.16:0/3480456172 >> v1:172.21.15.186:6789/0 conn(0x7fc33815ba10 legacy=0x7fc338099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc3367fc640 1 -- 172.21.15.16:0/3480456172 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33806cbe0 con 0x7fc33806c1f0 2024-04-25T10:07:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc3367fc640 1 -- 172.21.15.16:0/3480456172 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fc3300019f0 con 0x7fc33806c1f0 2024-04-25T10:07:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc3367fc640 1 -- 172.21.15.16:0/3480456172 >> v1:172.21.15.16:6789/0 conn(0x7fc33806c1f0 legacy=0x7fc3380cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc3367fc640 1 --2- 172.21.15.16:0/3480456172 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3200039b0 0x7fc320003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc3367fc640 1 --2- 172.21.15.16:0/3480456172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc320004800 0x7fc320004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc3367fc640 1 --2- 172.21.15.16:0/3480456172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3200051c0 0x7fc32000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc3367fc640 1 -- 172.21.15.16:0/3480456172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32000c030 con 0x7fc3200051c0 2024-04-25T10:07:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc3367fc640 1 -- 172.21.15.16:0/3480456172 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32000c1e0 con 0x7fc320004800 2024-04-25T10:07:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc3367fc640 1 -- 172.21.15.16:0/3480456172 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32000c3e0 con 0x7fc3200039b0 2024-04-25T10:07:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc337fff640 1 --2- 172.21.15.16:0/3480456172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3200051c0 0x7fc32000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc336ffd640 1 --2- 172.21.15.16:0/3480456172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc320004800 0x7fc320004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc3377fe640 1 --2- 172.21.15.16:0/3480456172 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3200039b0 0x7fc320003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc337fff640 1 -- 172.21.15.16:0/3480456172 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3200039b0 msgr2=0x7fc320003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc337fff640 1 --2- 172.21.15.16:0/3480456172 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3200039b0 0x7fc320003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc337fff640 1 -- 172.21.15.16:0/3480456172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc320004800 msgr2=0x7fc320004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc337fff640 1 --2- 172.21.15.16:0/3480456172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc320004800 0x7fc320004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.406+0000 7fc337fff640 1 -- 172.21.15.16:0/3480456172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc320003520 con 0x7fc3200051c0 2024-04-25T10:07:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.407+0000 7fc337fff640 1 -- 172.21.15.16:0/3480456172 reap_dead start 2024-04-25T10:07:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.407+0000 7fc337fff640 1 --2- 172.21.15.16:0/3480456172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3200051c0 0x7fc32000b9c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc33000a850 tx=0x7fc320005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a8e0a8e5c87427 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.407+0000 7fc3367fc640 1 -- 172.21.15.16:0/3480456172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33000a4a0 con 0x7fc3200051c0 2024-04-25T10:07:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.407+0000 7fc3367fc640 1 -- 172.21.15.16:0/3480456172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33002f070 con 0x7fc3200051c0 2024-04-25T10:07:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.407+0000 7fc3367fc640 1 -- 172.21.15.16:0/3480456172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33002bce0 con 0x7fc3200051c0 2024-04-25T10:07:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.408+0000 7fc33dd34640 1 -- 172.21.15.16:0/3480456172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3200051c0 msgr2=0x7fc32000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.408+0000 7fc33dd34640 1 --2- 172.21.15.16:0/3480456172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3200051c0 0x7fc32000b9c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc33000a850 tx=0x7fc320005830 comp rx=0 tx=0).stop 2024-04-25T10:07:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.408+0000 7fc33dd34640 1 -- 172.21.15.16:0/3480456172 shutdown_connections 2024-04-25T10:07:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.408+0000 7fc33dd34640 1 --2- 172.21.15.16:0/3480456172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3200051c0 0x7fc32000b9c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.408+0000 7fc33dd34640 1 -- 172.21.15.16:0/3480456172 >> 172.21.15.16:0/3480456172 conn(0x7fc3380b32f0 msgr2=0x7fc3380b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.408+0000 7fc33dd34640 1 -- 172.21.15.16:0/3480456172 shutdown_connections 2024-04-25T10:07:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.408+0000 7fc33dd34640 1 -- 172.21.15.16:0/3480456172 wait complete. 2024-04-25T10:07:47.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.409+0000 7fc33dd34640 1 Processor -- start 2024-04-25T10:07:47.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.409+0000 7fc33dd34640 1 -- start start 2024-04-25T10:07:47.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.410+0000 7fc33dd34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3200051c0 0x7fc3380b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.410+0000 7fc33dd34640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc338141170 0x7fc3381f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.410+0000 7fc33dd34640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3381f3de0 0x7fc3380b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.410+0000 7fc33dd34640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33806bcb0 con 0x7fc3200051c0 2024-04-25T10:07:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.410+0000 7fc33dd34640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3380cfe80 con 0x7fc338141170 2024-04-25T10:07:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.410+0000 7fc33dd34640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33806be30 con 0x7fc3381f3de0 2024-04-25T10:07:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.410+0000 7fc3377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3200051c0 0x7fc3380b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.410+0000 7fc3377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3200051c0 0x7fc3380b5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60202/0 (socket says 172.21.15.16:60202) 2024-04-25T10:07:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.410+0000 7fc3377fe640 1 -- 172.21.15.16:0/300791228 learned_addr learned my addr 172.21.15.16:0/300791228 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.410+0000 7fc337fff640 1 --2- 172.21.15.16:0/300791228 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3381f3de0 0x7fc3380b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.410+0000 7fc336ffd640 1 --2- 172.21.15.16:0/300791228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc338141170 0x7fc3381f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.410+0000 7fc3377fe640 1 -- 172.21.15.16:0/300791228 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3381f3de0 msgr2=0x7fc3380b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.410+0000 7fc3377fe640 1 --2- 172.21.15.16:0/300791228 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3381f3de0 0x7fc3380b2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.410+0000 7fc3377fe640 1 -- 172.21.15.16:0/300791228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc338141170 msgr2=0x7fc3381f38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.410+0000 7fc3377fe640 1 --2- 172.21.15.16:0/300791228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc338141170 0x7fc3381f38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.410+0000 7fc3377fe640 1 -- 172.21.15.16:0/300791228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3380b61e0 con 0x7fc3200051c0 2024-04-25T10:07:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.410+0000 7fc3377fe640 1 --2- 172.21.15.16:0/300791228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3200051c0 0x7fc3380b5b00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc32800f620 tx=0x7fc32800fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.411+0000 7fc31ffff640 1 -- 172.21.15.16:0/300791228 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3280052e0 con 0x7fc3200051c0 2024-04-25T10:07:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.411+0000 7fc31ffff640 1 -- 172.21.15.16:0/300791228 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc328018070 con 0x7fc3200051c0 2024-04-25T10:07:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.411+0000 7fc33dd34640 1 -- 172.21.15.16:0/300791228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3380b64c0 con 0x7fc3200051c0 2024-04-25T10:07:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.411+0000 7fc31ffff640 1 -- 172.21.15.16:0/300791228 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328003810 con 0x7fc3200051c0 2024-04-25T10:07:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.411+0000 7fc33dd34640 1 -- 172.21.15.16:0/300791228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3380b6a80 con 0x7fc3200051c0 2024-04-25T10:07:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.412+0000 7fc33dd34640 1 -- 172.21.15.16:0/300791228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3380749b0 con 0x7fc3200051c0 2024-04-25T10:07:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.412+0000 7fc31ffff640 1 -- 172.21.15.16:0/300791228 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3280039b0 con 0x7fc3200051c0 2024-04-25T10:07:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.413+0000 7fc31ffff640 1 --2- 172.21.15.16:0/300791228 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc32406b4d0 0x7fc32406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.413+0000 7fc31ffff640 1 -- 172.21.15.16:0/300791228 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc328089c70 con 0x7fc3200051c0 2024-04-25T10:07:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.413+0000 7fc336ffd640 1 --2- 172.21.15.16:0/300791228 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc32406b4d0 0x7fc32406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.413+0000 7fc336ffd640 1 --2- 172.21.15.16:0/300791228 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc32406b4d0 0x7fc32406d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc33813e5a0 tx=0x7fc32c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.417+0000 7fc31ffff640 1 -- 172.21.15.16:0/300791228 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc328058db0 con 0x7fc3200051c0 2024-04-25T10:07:47.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.669+0000 7fc33dd34640 1 -- 172.21.15.16:0/300791228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc33809a570 con 0x7fc3200051c0 2024-04-25T10:07:47.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.669+0000 7fc31ffff640 1 -- 172.21.15.16:0/300791228 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc3280586b0 con 0x7fc3200051c0 2024-04-25T10:07:47.675 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:47.675 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:07:47.675 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-04-25T10:07:47.675 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.675 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-04-25T10:07:47.675 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmj+xmBBAAaWdHmoiYcXzFnMVQSioT2Q==", 2024-04-25T10:07:47.675 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.675 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:07:47.675 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:07:47.676 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:07:47.676 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:07:47.676 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:47.676 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.676 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-04-25T10:07:47.676 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmUES+DxAAu5It7dQmb7/BM/4x1fUYXA==", 2024-04-25T10:07:47.676 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.676 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:07:47.676 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:07:47.676 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:07:47.676 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:07:47.677 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:47.677 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.677 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-04-25T10:07:47.677 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipm+LtFCxAAGG79kv8eWzly8l5qzQwfww==", 2024-04-25T10:07:47.677 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.677 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:07:47.677 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:07:47.677 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:07:47.677 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:07:47.677 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:47.677 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.678 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-04-25T10:07:47.678 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmqbSSFhAAZtrLsAGSyaMGgv45Hy0WpA==", 2024-04-25T10:07:47.678 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.678 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:07:47.678 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:07:47.678 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:07:47.678 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:07:47.678 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:47.678 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.678 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-04-25T10:07:47.678 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCaKipm2+UdOxAAP8GBunvYsue8EILYV5Uq7A==", 2024-04-25T10:07:47.678 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.679 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:47.679 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:07:47.679 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:07:47.679 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:47.679 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.679 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-04-25T10:07:47.679 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCdKipmSv87ORAAigVU7nyzkcXqMEHKsSAsqg==", 2024-04-25T10:07:47.679 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.679 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:47.679 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:07:47.679 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:07:47.679 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:47.679 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.679 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-04-25T10:07:47.680 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCgKipm2VMLOBAAJETFfWgq7H/imQAfw93Ruw==", 2024-04-25T10:07:47.680 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.680 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:47.680 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:07:47.680 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:07:47.680 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:47.680 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.680 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-04-25T10:07:47.680 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCjKipmb1g0NhAADH2uHQPhW84KLYyWbOjQdw==", 2024-04-25T10:07:47.680 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.680 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:47.680 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:07:47.680 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:07:47.681 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:47.681 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.681 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-04-25T10:07:47.681 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCpKipmnH0ROBAAEb8i4nfswYQ/w5mEw48kIg==", 2024-04-25T10:07:47.681 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.681 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:47.681 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:07:47.681 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:07:47.681 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:47.681 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.681 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-04-25T10:07:47.681 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCsKipm8HCRNhAA3FhcMlUWxMAiFRXbBWWvNA==", 2024-04-25T10:07:47.681 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.681 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:47.682 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:07:47.682 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:07:47.682 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:47.682 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.682 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-04-25T10:07:47.682 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCvKipmfgqpNRAAtPhpSDU9q23Sfa+V7m+3NA==", 2024-04-25T10:07:47.682 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.682 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:47.682 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:07:47.682 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:07:47.682 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:47.683 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.683 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-04-25T10:07:47.683 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCyKipm3ndyMxAAeMNODGxNupb043CsA/SP1A==", 2024-04-25T10:07:47.683 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.683 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:47.683 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:07:47.683 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:07:47.683 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:47.683 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.683 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-04-25T10:07:47.683 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmAph/GhAAG3x1zC98yHQ13fjpmrlPkg==", 2024-04-25T10:07:47.683 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.683 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:07:47.683 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-04-25T10:07:47.684 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:07:47.684 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:47.684 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.684 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-04-25T10:07:47.684 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmrccTIxAAuDZamFXTe6J2d+HAcBe+DA==", 2024-04-25T10:07:47.684 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.684 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:07:47.684 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:07:47.684 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:07:47.684 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:07:47.684 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:47.684 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.684 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-04-25T10:07:47.685 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmsFACIBAAFuWsmuQlF58yzIcYUx/6Iw==", 2024-04-25T10:07:47.685 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.685 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:07:47.685 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:07:47.685 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:07:47.685 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:07:47.685 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:47.685 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.685 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-04-25T10:07:47.685 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmAe1/KBAAXzLkscuwpCwMnrIKQny9+Q==", 2024-04-25T10:07:47.685 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.685 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:07:47.685 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:07:47.685 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:07:47.686 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:07:47.686 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:47.686 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.686 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-04-25T10:07:47.686 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCYKipmVFQ2JBAAdqG2p/M746PZPMDcpQjwfw==", 2024-04-25T10:07:47.686 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.686 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:07:47.686 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:07:47.686 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:07:47.686 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:07:47.686 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:47.686 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.687 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-04-25T10:07:47.687 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmiPVHARAAGPbv3P3yVj3tlo0b2gmcyw==", 2024-04-25T10:07:47.687 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.687 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:07:47.687 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-04-25T10:07:47.687 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:07:47.687 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:07:47.687 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:47.687 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-04-25T10:07:47.687 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCYKipmghhyOBAAcCm0hQzU+Wlw45yZQETTTg==", 2024-04-25T10:07:47.687 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:07:47.687 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:07:47.687 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-04-25T10:07:47.687 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:07:47.687 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.688 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:47.688 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-04-25T10:07:47.688 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:07:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.671+0000 7fc33dd34640 1 -- 172.21.15.16:0/300791228 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc32406b4d0 msgr2=0x7fc32406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.671+0000 7fc33dd34640 1 --2- 172.21.15.16:0/300791228 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc32406b4d0 0x7fc32406d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc33813e5a0 tx=0x7fc32c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.671+0000 7fc33dd34640 1 -- 172.21.15.16:0/300791228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3200051c0 msgr2=0x7fc3380b5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.671+0000 7fc33dd34640 1 --2- 172.21.15.16:0/300791228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3200051c0 0x7fc3380b5b00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc32800f620 tx=0x7fc32800fae0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.671+0000 7fc33dd34640 1 -- 172.21.15.16:0/300791228 shutdown_connections 2024-04-25T10:07:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.671+0000 7fc33dd34640 1 --2- 172.21.15.16:0/300791228 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc32406b4d0 0x7fc32406d990 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.671+0000 7fc33dd34640 1 --2- 172.21.15.16:0/300791228 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3381f3de0 0x7fc3380b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.671+0000 7fc33dd34640 1 --2- 172.21.15.16:0/300791228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc338141170 0x7fc3381f38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.671+0000 7fc33dd34640 1 --2- 172.21.15.16:0/300791228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3200051c0 0x7fc3380b5b00 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.671+0000 7fc33dd34640 1 -- 172.21.15.16:0/300791228 >> 172.21.15.16:0/300791228 conn(0x7fc3380b32f0 msgr2=0x7fc338150860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.671+0000 7fc33dd34640 1 -- 172.21.15.16:0/300791228 shutdown_connections 2024-04-25T10:07:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.671+0000 7fc33dd34640 1 -- 172.21.15.16:0/300791228 wait complete. 2024-04-25T10:07:47.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.786+0000 7fd483310640 1 Processor -- start 2024-04-25T10:07:47.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.786+0000 7fd483310640 1 -- start start 2024-04-25T10:07:47.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.787+0000 7fd483310640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd47c06e8f0 0x7fd47c15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.787+0000 7fd483310640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd47c0ce6c0 con 0x7fd47c15c820 2024-04-25T10:07:47.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.787+0000 7fd483310640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd47c0cc4e0 con 0x7fd47c15bf00 2024-04-25T10:07:47.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.787+0000 7fd483310640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd47c1455e0 con 0x7fd47c06e8f0 2024-04-25T10:07:47.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.787+0000 7fd4808ab640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd47c06e8f0 0x7fd47c15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.787+0000 7fd4818ad640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd47c15c820 0x7fd47c1654b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57106/0 (socket says 172.21.15.16:57106) 2024-04-25T10:07:47.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.787+0000 7fd4818ad640 1 -- 172.21.15.16:0/1390754831 learned_addr learned my addr 172.21.15.16:0/1390754831 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:47.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.787+0000 7fd4808ab640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd47c06e8f0 0x7fd47c15c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60204/0 (socket says 172.21.15.16:60204) 2024-04-25T10:07:47.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.787+0000 7fd4808ab640 1 -- 172.21.15.16:0/1390754831 >> v1:172.21.15.186:6789/0 conn(0x7fd47c15bf00 legacy=0x7fd47c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.787+0000 7fd4808ab640 1 -- 172.21.15.16:0/1390754831 >> v1:172.21.15.16:6789/0 conn(0x7fd47c15c820 legacy=0x7fd47c1654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.787+0000 7fd473fff640 1 -- 172.21.15.16:0/1390754831 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387854833 0 0) 0x7fd474001490 con 0x7fd47c15c820 2024-04-25T10:07:47.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.787+0000 7fd4808ab640 1 -- 172.21.15.16:0/1390754831 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c0cc7b0 con 0x7fd47c06e8f0 2024-04-25T10:07:47.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.788+0000 7fd4808ab640 1 --2- 172.21.15.16:0/1390754831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd47c06e8f0 0x7fd47c15c2e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0097a0 tx=0x7fd46c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52bcf5f29ca4513 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.788+0000 7fd473fff640 1 -- 172.21.15.16:0/1390754831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c037070 con 0x7fd47c06e8f0 2024-04-25T10:07:47.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.788+0000 7fd473fff640 1 -- 172.21.15.16:0/1390754831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c032440 con 0x7fd47c06e8f0 2024-04-25T10:07:47.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.788+0000 7fd473fff640 1 -- 172.21.15.16:0/1390754831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c032810 con 0x7fd47c06e8f0 2024-04-25T10:07:47.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.789+0000 7fd483310640 1 -- 172.21.15.16:0/1390754831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd47c06e8f0 msgr2=0x7fd47c15c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.789+0000 7fd483310640 1 --2- 172.21.15.16:0/1390754831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd47c06e8f0 0x7fd47c15c2e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0097a0 tx=0x7fd46c02a230 comp rx=0 tx=0).stop 2024-04-25T10:07:47.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.789+0000 7fd483310640 1 -- 172.21.15.16:0/1390754831 shutdown_connections 2024-04-25T10:07:47.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.789+0000 7fd483310640 1 --2- 172.21.15.16:0/1390754831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd47c06e8f0 0x7fd47c15c2e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.789+0000 7fd483310640 1 -- 172.21.15.16:0/1390754831 >> 172.21.15.16:0/1390754831 conn(0x7fd47c06aba0 msgr2=0x7fd47c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:47.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.789+0000 7fd483310640 1 -- 172.21.15.16:0/1390754831 shutdown_connections 2024-04-25T10:07:47.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.789+0000 7fd483310640 1 -- 172.21.15.16:0/1390754831 wait complete. 2024-04-25T10:07:47.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.790+0000 7fd483310640 1 Processor -- start 2024-04-25T10:07:47.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.790+0000 7fd483310640 1 -- start start 2024-04-25T10:07:47.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.791+0000 7fd483310640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd47c06e8f0 0x7fd47c1f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.791+0000 7fd483310640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd47c15bf00 0x7fd47c138e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.791+0000 7fd483310640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd47c15c820 0x7fd47c141f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.791+0000 7fd483310640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c0ccd70 con 0x7fd47c15bf00 2024-04-25T10:07:47.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.791+0000 7fd483310640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c06d450 con 0x7fd47c06e8f0 2024-04-25T10:07:47.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.791+0000 7fd483310640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c0cd050 con 0x7fd47c15c820 2024-04-25T10:07:47.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.791+0000 7fd4808ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd47c15bf00 0x7fd47c138e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.791+0000 7fd4808ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd47c15bf00 0x7fd47c138e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60216/0 (socket says 172.21.15.16:60216) 2024-04-25T10:07:47.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.791+0000 7fd4808ab640 1 -- 172.21.15.16:0/2499593244 learned_addr learned my addr 172.21.15.16:0/2499593244 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:47.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.791+0000 7fd4818ad640 1 --2- 172.21.15.16:0/2499593244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd47c15c820 0x7fd47c141f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.791+0000 7fd4808ab640 1 -- 172.21.15.16:0/2499593244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd47c15c820 msgr2=0x7fd47c141f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.791+0000 7fd4808ab640 1 --2- 172.21.15.16:0/2499593244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd47c15c820 0x7fd47c141f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.791+0000 7fd4808ab640 1 -- 172.21.15.16:0/2499593244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd47c06e8f0 msgr2=0x7fd47c1f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.791+0000 7fd4810ac640 1 --2- 172.21.15.16:0/2499593244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd47c06e8f0 0x7fd47c1f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.791+0000 7fd4808ab640 1 --2- 172.21.15.16:0/2499593244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd47c06e8f0 0x7fd47c1f6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.791+0000 7fd4808ab640 1 -- 172.21.15.16:0/2499593244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c13a950 con 0x7fd47c15bf00 2024-04-25T10:07:47.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.791+0000 7fd4808ab640 1 --2- 172.21.15.16:0/2499593244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd47c15bf00 0x7fd47c138e00 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd46c000c00 tx=0x7fd46c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.791+0000 7fd471ffb640 1 -- 172.21.15.16:0/2499593244 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c037040 con 0x7fd47c15bf00 2024-04-25T10:07:47.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.792+0000 7fd471ffb640 1 -- 172.21.15.16:0/2499593244 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c042070 con 0x7fd47c15bf00 2024-04-25T10:07:47.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.792+0000 7fd471ffb640 1 -- 172.21.15.16:0/2499593244 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c030a70 con 0x7fd47c15bf00 2024-04-25T10:07:47.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.792+0000 7fd483310640 1 -- 172.21.15.16:0/2499593244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47c13abf0 con 0x7fd47c15bf00 2024-04-25T10:07:47.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.792+0000 7fd483310640 1 -- 172.21.15.16:0/2499593244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47c13b1b0 con 0x7fd47c15bf00 2024-04-25T10:07:47.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.793+0000 7fd483310640 1 -- 172.21.15.16:0/2499593244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd47c0749b0 con 0x7fd47c15bf00 2024-04-25T10:07:47.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.793+0000 7fd471ffb640 1 -- 172.21.15.16:0/2499593244 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd46c030c10 con 0x7fd47c15bf00 2024-04-25T10:07:47.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.794+0000 7fd471ffb640 1 --2- 172.21.15.16:0/2499593244 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd46006b4d0 0x7fd46006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.794+0000 7fd471ffb640 1 -- 172.21.15.16:0/2499593244 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd46c0aa930 con 0x7fd47c15bf00 2024-04-25T10:07:47.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.794+0000 7fd4810ac640 1 --2- 172.21.15.16:0/2499593244 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd46006b4d0 0x7fd46006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.794+0000 7fd4810ac640 1 --2- 172.21.15.16:0/2499593244 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd46006b4d0 0x7fd46006d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd47c14e050 tx=0x7fd478009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.798+0000 7fd471ffb640 1 -- 172.21.15.16:0/2499593244 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd46c045360 con 0x7fd47c15bf00 2024-04-25T10:07:47.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.981+0000 7fd483310640 1 -- 172.21.15.16:0/2499593244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd47c06aff0 con 0x7fd47c15bf00 2024-04-25T10:07:47.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.982+0000 7fd471ffb640 1 -- 172.21.15.16:0/2499593244 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+12891 (secure 0 0 0) 0x7fd46c079b90 con 0x7fd47c15bf00 2024-04-25T10:07:47.988 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:47.988 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":55,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:07:47.279648+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:47.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.984+0000 7fd483310640 1 -- 172.21.15.16:0/2499593244 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd46006b4d0 msgr2=0x7fd46006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.984+0000 7fd483310640 1 --2- 172.21.15.16:0/2499593244 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd46006b4d0 0x7fd46006d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd47c14e050 tx=0x7fd478009040 comp rx=0 tx=0).stop 2024-04-25T10:07:47.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.984+0000 7fd483310640 1 -- 172.21.15.16:0/2499593244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd47c15bf00 msgr2=0x7fd47c138e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.984+0000 7fd483310640 1 --2- 172.21.15.16:0/2499593244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd47c15bf00 0x7fd47c138e00 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd46c000c00 tx=0x7fd46c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.984+0000 7fd483310640 1 -- 172.21.15.16:0/2499593244 shutdown_connections 2024-04-25T10:07:47.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.984+0000 7fd483310640 1 --2- 172.21.15.16:0/2499593244 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd46006b4d0 0x7fd46006d990 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.984+0000 7fd483310640 1 --2- 172.21.15.16:0/2499593244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd47c15c820 0x7fd47c141f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.984+0000 7fd483310640 1 --2- 172.21.15.16:0/2499593244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd47c15bf00 0x7fd47c138e00 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.984+0000 7fd483310640 1 --2- 172.21.15.16:0/2499593244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd47c06e8f0 0x7fd47c1f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.984+0000 7fd483310640 1 -- 172.21.15.16:0/2499593244 >> 172.21.15.16:0/2499593244 conn(0x7fd47c06aba0 msgr2=0x7fd47c0b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:47.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.984+0000 7fd483310640 1 -- 172.21.15.16:0/2499593244 shutdown_connections 2024-04-25T10:07:47.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:47.984+0000 7fd483310640 1 -- 172.21.15.16:0/2499593244 wait complete. 2024-04-25T10:07:48.001 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:48.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:48.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.099+0000 7fb4d117e640 1 Processor -- start 2024-04-25T10:07:48.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.099+0000 7fb4d117e640 1 -- start start 2024-04-25T10:07:48.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.100+0000 7fb4d117e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb4cc06b2b0 0x7fb4cc0b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.100+0000 7fb4d117e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4cc0ccee0 0x7fb4cc0b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.100+0000 7fb4d117e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc0cd450 con 0x7fb4cc15baf0 2024-04-25T10:07:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.100+0000 7fb4d117e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4cc0ce060 con 0x7fb4cc0ccee0 2024-04-25T10:07:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.100+0000 7fb4d117e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4cc0cd730 con 0x7fb4cc06b2b0 2024-04-25T10:07:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.100+0000 7fb4cb59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4cc0ccee0 0x7fb4cc0b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.100+0000 7fb4cb59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4cc0ccee0 0x7fb4cc0b08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60226/0 (socket says 172.21.15.16:60226) 2024-04-25T10:07:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.100+0000 7fb4cb59e640 1 -- 172.21.15.16:0/979041412 learned_addr learned my addr 172.21.15.16:0/979041412 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.101+0000 7fb4ca59c640 1 --2- 172.21.15.16:0/979041412 >> v2:172.21.15.186:3300/0 conn(0x7fb4cc06b2b0 0x7fb4cc0b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.101+0000 7fb4cb59e640 1 -- 172.21.15.16:0/979041412 >> v2:172.21.15.186:3300/0 conn(0x7fb4cc06b2b0 msgr2=0x7fb4cc0b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.101+0000 7fb4cb59e640 1 --2- 172.21.15.16:0/979041412 >> v2:172.21.15.186:3300/0 conn(0x7fb4cc06b2b0 0x7fb4cc0b03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.101+0000 7fb4cb59e640 1 -- 172.21.15.16:0/979041412 >> v1:172.21.15.16:6789/0 conn(0x7fb4cc15baf0 legacy=0x7fb4cc06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.101+0000 7fb4c9d9b640 1 -- 172.21.15.16:0/979041412 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1777365652 0 0) 0x7fb4cc0cd450 con 0x7fb4cc15baf0 2024-04-25T10:07:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.101+0000 7fb4cb59e640 1 -- 172.21.15.16:0/979041412 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc0cda10 con 0x7fb4cc0ccee0 2024-04-25T10:07:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.101+0000 7fb4cb59e640 1 --2- 172.21.15.16:0/979041412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4cc0ccee0 0x7fb4cc0b08f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc071680 tx=0x7fb4bc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb6587befff98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.102+0000 7fb4c9d9b640 1 -- 172.21.15.16:0/979041412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc036070 con 0x7fb4cc0ccee0 2024-04-25T10:07:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.102+0000 7fb4c9d9b640 1 -- 172.21.15.16:0/979041412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc031600 con 0x7fb4cc0ccee0 2024-04-25T10:07:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.102+0000 7fb4c9d9b640 1 -- 172.21.15.16:0/979041412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0319d0 con 0x7fb4cc0ccee0 2024-04-25T10:07:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.102+0000 7fb4d117e640 1 -- 172.21.15.16:0/979041412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4cc0ccee0 msgr2=0x7fb4cc0b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.102+0000 7fb4d117e640 1 --2- 172.21.15.16:0/979041412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4cc0ccee0 0x7fb4cc0b08f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc071680 tx=0x7fb4bc02a730 comp rx=0 tx=0).stop 2024-04-25T10:07:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.102+0000 7fb4d117e640 1 -- 172.21.15.16:0/979041412 shutdown_connections 2024-04-25T10:07:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.102+0000 7fb4d117e640 1 --2- 172.21.15.16:0/979041412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4cc0ccee0 0x7fb4cc0b08f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.102+0000 7fb4d117e640 1 --2- 172.21.15.16:0/979041412 >> v2:172.21.15.186:3300/0 conn(0x7fb4cc06b2b0 0x7fb4cc0b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.102+0000 7fb4d117e640 1 -- 172.21.15.16:0/979041412 >> 172.21.15.16:0/979041412 conn(0x7fb4cc0b2390 msgr2=0x7fb4cc0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.102+0000 7fb4d117e640 1 -- 172.21.15.16:0/979041412 shutdown_connections 2024-04-25T10:07:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.102+0000 7fb4d117e640 1 -- 172.21.15.16:0/979041412 wait complete. 2024-04-25T10:07:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.104+0000 7fb4d117e640 1 Processor -- start 2024-04-25T10:07:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.104+0000 7fb4d117e640 1 -- start start 2024-04-25T10:07:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.104+0000 7fb4d117e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4cc06b2b0 0x7fb4cc159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.104+0000 7fb4d117e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4cc0ccee0 0x7fb4cc14d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.104+0000 7fb4d117e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4cc15baf0 0x7fb4cc14dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.104+0000 7fb4d117e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0c41a0 con 0x7fb4cc0ccee0 2024-04-25T10:07:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.104+0000 7fb4d117e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0c4020 con 0x7fb4cc15baf0 2024-04-25T10:07:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.104+0000 7fb4d117e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc15bed0 con 0x7fb4cc06b2b0 2024-04-25T10:07:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.105+0000 7fb4ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4cc0ccee0 0x7fb4cc14d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.105+0000 7fb4ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4cc0ccee0 0x7fb4cc14d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60232/0 (socket says 172.21.15.16:60232) 2024-04-25T10:07:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.105+0000 7fb4ca59c640 1 -- 172.21.15.16:0/97344644 learned_addr learned my addr 172.21.15.16:0/97344644 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.105+0000 7fb4cb59e640 1 --2- 172.21.15.16:0/97344644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4cc15baf0 0x7fb4cc14dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.105+0000 7fb4cad9d640 1 --2- 172.21.15.16:0/97344644 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4cc06b2b0 0x7fb4cc159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.105+0000 7fb4ca59c640 1 -- 172.21.15.16:0/97344644 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4cc06b2b0 msgr2=0x7fb4cc159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.105+0000 7fb4ca59c640 1 --2- 172.21.15.16:0/97344644 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4cc06b2b0 0x7fb4cc159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.105+0000 7fb4ca59c640 1 -- 172.21.15.16:0/97344644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4cc15baf0 msgr2=0x7fb4cc14dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.105+0000 7fb4ca59c640 1 --2- 172.21.15.16:0/97344644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4cc15baf0 0x7fb4cc14dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.105+0000 7fb4ca59c640 1 -- 172.21.15.16:0/97344644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc0b6020 con 0x7fb4cc0ccee0 2024-04-25T10:07:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.105+0000 7fb4ca59c640 1 --2- 172.21.15.16:0/97344644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4cc0ccee0 0x7fb4cc14d660 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400d830 tx=0x7fb4b400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.105+0000 7fb4c37fe640 1 -- 172.21.15.16:0/97344644 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400b6c0 con 0x7fb4cc0ccee0 2024-04-25T10:07:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.105+0000 7fb4c37fe640 1 -- 172.21.15.16:0/97344644 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b4019070 con 0x7fb4cc0ccee0 2024-04-25T10:07:48.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.105+0000 7fb4d117e640 1 -- 172.21.15.16:0/97344644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc0b6300 con 0x7fb4cc0ccee0 2024-04-25T10:07:48.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.105+0000 7fb4c37fe640 1 -- 172.21.15.16:0/97344644 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400ba80 con 0x7fb4cc0ccee0 2024-04-25T10:07:48.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.105+0000 7fb4d117e640 1 -- 172.21.15.16:0/97344644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc0ce820 con 0x7fb4cc0ccee0 2024-04-25T10:07:48.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.106+0000 7fb4d117e640 1 -- 172.21.15.16:0/97344644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc0c38c0 con 0x7fb4cc0ccee0 2024-04-25T10:07:48.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.107+0000 7fb4c37fe640 1 -- 172.21.15.16:0/97344644 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb4b401e460 con 0x7fb4cc0ccee0 2024-04-25T10:07:48.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.107+0000 7fb4c37fe640 1 --2- 172.21.15.16:0/97344644 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4ac06b4d0 0x7fb4ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.107+0000 7fb4c37fe640 1 -- 172.21.15.16:0/97344644 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb4b408a480 con 0x7fb4cc0ccee0 2024-04-25T10:07:48.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.107+0000 7fb4cad9d640 1 --2- 172.21.15.16:0/97344644 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4ac06b4d0 0x7fb4ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.108+0000 7fb4cad9d640 1 --2- 172.21.15.16:0/97344644 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4ac06b4d0 0x7fb4ac06d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4000f60 tx=0x7fb4c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.111+0000 7fb4c37fe640 1 -- 172.21.15.16:0/97344644 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb4b40596e0 con 0x7fb4cc0ccee0 2024-04-25T10:07:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.294+0000 7fb4d117e640 1 -- 172.21.15.16:0/97344644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4cc0750b0 con 0x7fb4cc0ccee0 2024-04-25T10:07:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.294+0000 7fb4c37fe640 1 -- 172.21.15.16:0/97344644 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4b4058fe0 con 0x7fb4cc0ccee0 2024-04-25T10:07:48.300 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:07:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.296+0000 7fb4d117e640 1 -- 172.21.15.16:0/97344644 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4ac06b4d0 msgr2=0x7fb4ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.296+0000 7fb4d117e640 1 --2- 172.21.15.16:0/97344644 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4ac06b4d0 0x7fb4ac06d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4000f60 tx=0x7fb4c4009040 comp rx=0 tx=0).stop 2024-04-25T10:07:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.296+0000 7fb4d117e640 1 -- 172.21.15.16:0/97344644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4cc0ccee0 msgr2=0x7fb4cc14d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.296+0000 7fb4d117e640 1 --2- 172.21.15.16:0/97344644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4cc0ccee0 0x7fb4cc14d660 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400d830 tx=0x7fb4b400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.296+0000 7fb4d117e640 1 -- 172.21.15.16:0/97344644 shutdown_connections 2024-04-25T10:07:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.296+0000 7fb4d117e640 1 --2- 172.21.15.16:0/97344644 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4ac06b4d0 0x7fb4ac06d990 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.296+0000 7fb4d117e640 1 --2- 172.21.15.16:0/97344644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4cc15baf0 0x7fb4cc14dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.296+0000 7fb4d117e640 1 --2- 172.21.15.16:0/97344644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4cc0ccee0 0x7fb4cc14d660 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.296+0000 7fb4d117e640 1 --2- 172.21.15.16:0/97344644 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4cc06b2b0 0x7fb4cc159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.296+0000 7fb4d117e640 1 -- 172.21.15.16:0/97344644 >> 172.21.15.16:0/97344644 conn(0x7fb4cc0b2390 msgr2=0x7fb4cc0b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.296+0000 7fb4d117e640 1 -- 172.21.15.16:0/97344644 shutdown_connections 2024-04-25T10:07:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.296+0000 7fb4d117e640 1 -- 172.21.15.16:0/97344644 wait complete. 2024-04-25T10:07:48.313 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:07:48.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.409+0000 7f47ec8d1640 1 Processor -- start 2024-04-25T10:07:48.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.409+0000 7f47ec8d1640 1 -- start start 2024-04-25T10:07:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.410+0000 7f47ec8d1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47e80d35b0 0x7f47e80b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.410+0000 7f47ec8d1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e815e0c0 con 0x7f47e815dce0 2024-04-25T10:07:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.410+0000 7f47ec8d1640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e80d2510 con 0x7f47e80b5990 2024-04-25T10:07:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.410+0000 7f47ec8d1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e80b5de0 con 0x7f47e80d35b0 2024-04-25T10:07:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.410+0000 7f47e659c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f47e815dce0 0x7f47e80d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57124/0 (socket says 172.21.15.16:57124) 2024-04-25T10:07:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.410+0000 7f47e659c640 1 -- 172.21.15.16:0/3030233443 learned_addr learned my addr 172.21.15.16:0/3030233443 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.410+0000 7f47dffff640 1 --2- 172.21.15.16:0/3030233443 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47e80d35b0 0x7f47e80b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.410+0000 7f47e5d9b640 1 -- 172.21.15.16:0/3030233443 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2606568892 0 0) 0x7f47e815e0c0 con 0x7f47e815dce0 2024-04-25T10:07:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.410+0000 7f47e5d9b640 1 -- 172.21.15.16:0/3030233443 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47d4003520 con 0x7f47e815dce0 2024-04-25T10:07:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47e5d9b640 1 -- 172.21.15.16:0/3030233443 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2173879063 0 0) 0x7f47d4003520 con 0x7f47e815dce0 2024-04-25T10:07:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47e5d9b640 1 -- 172.21.15.16:0/3030233443 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47e80d35b0 msgr2=0x7f47e80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47e5d9b640 1 --2- 172.21.15.16:0/3030233443 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47e80d35b0 0x7f47e80b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47e5d9b640 1 -- 172.21.15.16:0/3030233443 >> v1:172.21.15.186:6789/0 conn(0x7f47e80b5990 legacy=0x7f47e80b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47e5d9b640 1 -- 172.21.15.16:0/3030233443 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e80d27f0 con 0x7f47e815dce0 2024-04-25T10:07:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47e5d9b640 1 -- 172.21.15.16:0/3030233443 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f47d8001670 con 0x7f47e815dce0 2024-04-25T10:07:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47e5d9b640 1 -- 172.21.15.16:0/3030233443 >> v1:172.21.15.16:6789/0 conn(0x7f47e815dce0 legacy=0x7f47e80d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47e5d9b640 1 --2- 172.21.15.16:0/3030233443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47d40039e0 0x7f47d4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47e5d9b640 1 --2- 172.21.15.16:0/3030233443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d4004800 0x7f47d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47e5d9b640 1 --2- 172.21.15.16:0/3030233443 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47e80d35b0 0x7f47d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47e5d9b640 1 -- 172.21.15.16:0/3030233443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e80b5de0 con 0x7f47d4004800 2024-04-25T10:07:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47e5d9b640 1 -- 172.21.15.16:0/3030233443 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d400ba70 con 0x7f47d40039e0 2024-04-25T10:07:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47e5d9b640 1 -- 172.21.15.16:0/3030233443 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d400bc70 con 0x7f47e80d35b0 2024-04-25T10:07:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47dffff640 1 --2- 172.21.15.16:0/3030233443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d4004800 0x7f47d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47e659c640 1 --2- 172.21.15.16:0/3030233443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47d40039e0 0x7f47d4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47e6d9d640 1 --2- 172.21.15.16:0/3030233443 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47e80d35b0 0x7f47d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47dffff640 1 -- 172.21.15.16:0/3030233443 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47e80d35b0 msgr2=0x7f47d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47dffff640 1 --2- 172.21.15.16:0/3030233443 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47e80d35b0 0x7f47d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47dffff640 1 -- 172.21.15.16:0/3030233443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47d40039e0 msgr2=0x7f47d4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47dffff640 1 --2- 172.21.15.16:0/3030233443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47d40039e0 0x7f47d4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47dffff640 1 -- 172.21.15.16:0/3030233443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d4003520 con 0x7f47d4004800 2024-04-25T10:07:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.411+0000 7f47e6d9d640 1 --2- 172.21.15.16:0/3030233443 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47e80d35b0 0x7f47d40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.412+0000 7f47dffff640 1 --2- 172.21.15.16:0/3030233443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d4004800 0x7f47d4004c80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009a90 tx=0x7f47d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da937448a32468cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.412+0000 7f47e5d9b640 1 -- 172.21.15.16:0/3030233443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d0036070 con 0x7f47d4004800 2024-04-25T10:07:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.412+0000 7f47e5d9b640 1 -- 172.21.15.16:0/3030233443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d0031400 con 0x7f47d4004800 2024-04-25T10:07:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.412+0000 7f47e5d9b640 1 -- 172.21.15.16:0/3030233443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d0009690 con 0x7f47d4004800 2024-04-25T10:07:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.412+0000 7f47ec8d1640 1 -- 172.21.15.16:0/3030233443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d4004800 msgr2=0x7f47d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.412+0000 7f47ec8d1640 1 --2- 172.21.15.16:0/3030233443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d4004800 0x7f47d4004c80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009a90 tx=0x7f47d4005830 comp rx=0 tx=0).stop 2024-04-25T10:07:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.412+0000 7f47e6d9d640 1 -- 172.21.15.16:0/3030233443 reap_dead start 2024-04-25T10:07:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.412+0000 7f47ec8d1640 1 -- 172.21.15.16:0/3030233443 shutdown_connections 2024-04-25T10:07:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.412+0000 7f47ec8d1640 1 -- 172.21.15.16:0/3030233443 >> 172.21.15.16:0/3030233443 conn(0x7f47e8072900 msgr2=0x7f47e80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.413+0000 7f47ec8d1640 1 -- 172.21.15.16:0/3030233443 shutdown_connections 2024-04-25T10:07:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.413+0000 7f47ec8d1640 1 -- 172.21.15.16:0/3030233443 wait complete. 2024-04-25T10:07:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.414+0000 7f47ec8d1640 1 Processor -- start 2024-04-25T10:07:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.414+0000 7f47ec8d1640 1 -- start start 2024-04-25T10:07:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47ec8d1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47d4004800 0x7f47e8144c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47ec8d1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47e815fe70 0x7f47e81602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47ec8d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e80ccf80 0x7f47e80cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47ec8d1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e80d3090 con 0x7f47e80ccf80 2024-04-25T10:07:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47ec8d1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e80d2db0 con 0x7f47e815fe70 2024-04-25T10:07:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47ec8d1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e80d2160 con 0x7f47d4004800 2024-04-25T10:07:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47e6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e80ccf80 0x7f47e80cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47e6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e80ccf80 0x7f47e80cd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60256/0 (socket says 172.21.15.16:60256) 2024-04-25T10:07:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47e6d9d640 1 -- 172.21.15.16:0/1053443638 learned_addr learned my addr 172.21.15.16:0/1053443638 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47dffff640 1 --2- 172.21.15.16:0/1053443638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47e815fe70 0x7f47e81602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47e659c640 1 --2- 172.21.15.16:0/1053443638 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47d4004800 0x7f47e8144c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47e6d9d640 1 -- 172.21.15.16:0/1053443638 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47d4004800 msgr2=0x7f47e8144c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47e6d9d640 1 --2- 172.21.15.16:0/1053443638 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47d4004800 0x7f47e8144c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47e6d9d640 1 -- 172.21.15.16:0/1053443638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47e815fe70 msgr2=0x7f47e81602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47e6d9d640 1 --2- 172.21.15.16:0/1053443638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47e815fe70 0x7f47e81602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47e6d9d640 1 -- 172.21.15.16:0/1053443638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e8145190 con 0x7f47e80ccf80 2024-04-25T10:07:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47e659c640 1 --2- 172.21.15.16:0/1053443638 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47d4004800 0x7f47e8144c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47e6d9d640 1 --2- 172.21.15.16:0/1053443638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e80ccf80 0x7f47e80cd400 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f47e80b6420 tx=0x7f47e000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47deffd640 1 -- 172.21.15.16:0/1053443638 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0006b50 con 0x7f47e80ccf80 2024-04-25T10:07:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47deffd640 1 -- 172.21.15.16:0/1053443638 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e0006d10 con 0x7f47e80ccf80 2024-04-25T10:07:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47deffd640 1 -- 172.21.15.16:0/1053443638 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0003970 con 0x7f47e80ccf80 2024-04-25T10:07:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47ec8d1640 1 -- 172.21.15.16:0/1053443638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e80cb700 con 0x7f47e80ccf80 2024-04-25T10:07:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.415+0000 7f47ec8d1640 1 -- 172.21.15.16:0/1053443638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e80cbcc0 con 0x7f47e80ccf80 2024-04-25T10:07:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.416+0000 7f47ec8d1640 1 -- 172.21.15.16:0/1053443638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e80749b0 con 0x7f47e80ccf80 2024-04-25T10:07:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.417+0000 7f47deffd640 1 -- 172.21.15.16:0/1053443638 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47e001a020 con 0x7f47e80ccf80 2024-04-25T10:07:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.417+0000 7f47deffd640 1 --2- 172.21.15.16:0/1053443638 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f47c806b4d0 0x7f47c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.417+0000 7f47deffd640 1 -- 172.21.15.16:0/1053443638 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f47e0089540 con 0x7f47e80ccf80 2024-04-25T10:07:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.418+0000 7f47e659c640 1 --2- 172.21.15.16:0/1053443638 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f47c806b4d0 0x7f47c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.418+0000 7f47e659c640 1 --2- 172.21.15.16:0/1053443638 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f47c806b4d0 0x7f47c806d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f47d8000c00 tx=0x7f47d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.421+0000 7f47deffd640 1 -- 172.21.15.16:0/1053443638 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47e0058670 con 0x7f47e80ccf80 2024-04-25T10:07:48.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.603+0000 7f47ec8d1640 1 -- 172.21.15.16:0/1053443638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f47e8161490 con 0x7f47e80ccf80 2024-04-25T10:07:48.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.604+0000 7f47deffd640 1 -- 172.21.15.16:0/1053443638 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v55) v1 ==== 84+0+65 (secure 0 0 0) 0x7f47e0057f70 con 0x7f47e80ccf80 2024-04-25T10:07:48.610 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:48.610 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-04-25T10:07:48.610 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:07:48.610 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-04-25T10:07:48.610 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-04-25T10:07:48.610 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:07:48.610 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-04-25T10:07:48.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.606+0000 7f47ec8d1640 1 -- 172.21.15.16:0/1053443638 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f47c806b4d0 msgr2=0x7f47c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.606+0000 7f47ec8d1640 1 --2- 172.21.15.16:0/1053443638 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f47c806b4d0 0x7f47c806d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f47d8000c00 tx=0x7f47d8009040 comp rx=0 tx=0).stop 2024-04-25T10:07:48.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.606+0000 7f47ec8d1640 1 -- 172.21.15.16:0/1053443638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e80ccf80 msgr2=0x7f47e80cd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.606+0000 7f47ec8d1640 1 --2- 172.21.15.16:0/1053443638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e80ccf80 0x7f47e80cd400 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f47e80b6420 tx=0x7f47e000eb90 comp rx=0 tx=0).stop 2024-04-25T10:07:48.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.606+0000 7f47ec8d1640 1 -- 172.21.15.16:0/1053443638 shutdown_connections 2024-04-25T10:07:48.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.606+0000 7f47ec8d1640 1 --2- 172.21.15.16:0/1053443638 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f47c806b4d0 0x7f47c806d990 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.606+0000 7f47ec8d1640 1 --2- 172.21.15.16:0/1053443638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e80ccf80 0x7f47e80cd400 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.606+0000 7f47ec8d1640 1 --2- 172.21.15.16:0/1053443638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47e815fe70 0x7f47e81602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.606+0000 7f47ec8d1640 1 --2- 172.21.15.16:0/1053443638 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47d4004800 0x7f47e8144c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.606+0000 7f47ec8d1640 1 -- 172.21.15.16:0/1053443638 >> 172.21.15.16:0/1053443638 conn(0x7f47e8072900 msgr2=0x7f47e80d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:48.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.606+0000 7f47ec8d1640 1 -- 172.21.15.16:0/1053443638 shutdown_connections 2024-04-25T10:07:48.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.606+0000 7f47ec8d1640 1 -- 172.21.15.16:0/1053443638 wait complete. 2024-04-25T10:07:48.622 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:07:48.623 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:07:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.722+0000 7f09d4b53640 1 Processor -- start 2024-04-25T10:07:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.722+0000 7f09d4b53640 1 -- start start 2024-04-25T10:07:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09d4b53640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09d0159f60 0x7f09d00b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09d4b53640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f09d0071200 0x7f09d0071680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09d4b53640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d015b5d0 con 0x7f09d006d260 2024-04-25T10:07:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09d4b53640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d00b3820 con 0x7f09d0071200 2024-04-25T10:07:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09d4b53640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d015b2f0 con 0x7f09d0159f60 2024-04-25T10:07:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09ced9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f09d006d260 0x7f09d006d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57130/0 (socket says 172.21.15.16:57130) 2024-04-25T10:07:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09ced9d640 1 -- 172.21.15.16:0/609242705 learned_addr learned my addr 172.21.15.16:0/609242705 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09cdd9b640 1 --2- 172.21.15.16:0/609242705 >> v2:172.21.15.186:3300/0 conn(0x7f09d0071200 0x7f09d0071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09cd59a640 1 -- 172.21.15.16:0/609242705 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1046998154 0 0) 0x7f09d015b5d0 con 0x7f09d006d260 2024-04-25T10:07:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09ce59c640 1 --2- 172.21.15.16:0/609242705 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09d0159f60 0x7f09d00b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09cd59a640 1 -- 172.21.15.16:0/609242705 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09b8004630 con 0x7f09d006d260 2024-04-25T10:07:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09cd59a640 1 -- 172.21.15.16:0/609242705 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 560086089 0 0) 0x7f09b8004630 con 0x7f09d006d260 2024-04-25T10:07:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09cd59a640 1 -- 172.21.15.16:0/609242705 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09d0159f60 msgr2=0x7f09d00b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09cd59a640 1 --2- 172.21.15.16:0/609242705 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09d0159f60 0x7f09d00b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09cd59a640 1 -- 172.21.15.16:0/609242705 >> v2:172.21.15.186:3300/0 conn(0x7f09d0071200 msgr2=0x7f09d0071680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09cd59a640 1 --2- 172.21.15.16:0/609242705 >> v2:172.21.15.186:3300/0 conn(0x7f09d0071200 0x7f09d0071680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09cd59a640 1 -- 172.21.15.16:0/609242705 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d00b2800 con 0x7f09d006d260 2024-04-25T10:07:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09cd59a640 1 -- 172.21.15.16:0/609242705 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f09c00019f0 con 0x7f09d006d260 2024-04-25T10:07:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09cd59a640 1 -- 172.21.15.16:0/609242705 >> v1:172.21.15.16:6789/0 conn(0x7f09d006d260 legacy=0x7f09d006d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09cd59a640 1 --2- 172.21.15.16:0/609242705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09b8004ac0 0x7f09b8004ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09cd59a640 1 --2- 172.21.15.16:0/609242705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09b8005910 0x7f09b8005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09cd59a640 1 --2- 172.21.15.16:0/609242705 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09d0159f60 0x7f09b80062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09cd59a640 1 -- 172.21.15.16:0/609242705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00b3820 con 0x7f09b8005910 2024-04-25T10:07:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.723+0000 7f09cd59a640 1 -- 172.21.15.16:0/609242705 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d015b2f0 con 0x7f09b8004ac0 2024-04-25T10:07:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.724+0000 7f09cd59a640 1 -- 172.21.15.16:0/609242705 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b800cad0 con 0x7f09d0159f60 2024-04-25T10:07:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.724+0000 7f09cdd9b640 1 --2- 172.21.15.16:0/609242705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09b8005910 0x7f09b8005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.724+0000 7f09ce59c640 1 --2- 172.21.15.16:0/609242705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09b8004ac0 0x7f09b8004ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.724+0000 7f09ced9d640 1 --2- 172.21.15.16:0/609242705 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09d0159f60 0x7f09b80062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.724+0000 7f09cdd9b640 1 -- 172.21.15.16:0/609242705 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09d0159f60 msgr2=0x7f09b80062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.724+0000 7f09cdd9b640 1 --2- 172.21.15.16:0/609242705 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09d0159f60 0x7f09b80062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.724+0000 7f09cdd9b640 1 -- 172.21.15.16:0/609242705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09b8004ac0 msgr2=0x7f09b8004ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.724+0000 7f09cdd9b640 1 --2- 172.21.15.16:0/609242705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09b8004ac0 0x7f09b8004ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.724+0000 7f09cdd9b640 1 -- 172.21.15.16:0/609242705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b8004630 con 0x7f09b8005910 2024-04-25T10:07:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.724+0000 7f09ced9d640 1 --2- 172.21.15.16:0/609242705 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09d0159f60 0x7f09b80062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.725+0000 7f09cdd9b640 1 --2- 172.21.15.16:0/609242705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09b8005910 0x7f09b8005d90 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009ac0 tx=0x7f09b8006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667abf976f8799a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.725+0000 7f09cd59a640 1 -- 172.21.15.16:0/609242705 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c4036070 con 0x7f09b8005910 2024-04-25T10:07:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.725+0000 7f09cd59a640 1 -- 172.21.15.16:0/609242705 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c4031400 con 0x7f09b8005910 2024-04-25T10:07:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.725+0000 7f09cd59a640 1 -- 172.21.15.16:0/609242705 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c4009690 con 0x7f09b8005910 2024-04-25T10:07:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.725+0000 7f09d4b53640 1 -- 172.21.15.16:0/609242705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09b8005910 msgr2=0x7f09b8005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.725+0000 7f09d4b53640 1 --2- 172.21.15.16:0/609242705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09b8005910 0x7f09b8005d90 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009ac0 tx=0x7f09b8006940 comp rx=0 tx=0).stop 2024-04-25T10:07:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.725+0000 7f09ced9d640 1 -- 172.21.15.16:0/609242705 reap_dead start 2024-04-25T10:07:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.725+0000 7f09d4b53640 1 -- 172.21.15.16:0/609242705 shutdown_connections 2024-04-25T10:07:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.725+0000 7f09d4b53640 1 -- 172.21.15.16:0/609242705 >> 172.21.15.16:0/609242705 conn(0x7f09d006aba0 msgr2=0x7f09d006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.725+0000 7f09d4b53640 1 -- 172.21.15.16:0/609242705 shutdown_connections 2024-04-25T10:07:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.725+0000 7f09d4b53640 1 -- 172.21.15.16:0/609242705 wait complete. 2024-04-25T10:07:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.727+0000 7f09d4b53640 1 Processor -- start 2024-04-25T10:07:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.727+0000 7f09d4b53640 1 -- start start 2024-04-25T10:07:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.727+0000 7f09d4b53640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09b8005910 0x7f09d014d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.727+0000 7f09d4b53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09d0150e90 0x7f09d01565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.727+0000 7f09d4b53640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09d01f17d0 0x7f09d0153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.727+0000 7f09d4b53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d006dda0 con 0x7f09d0150e90 2024-04-25T10:07:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.727+0000 7f09d4b53640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00b2dc0 con 0x7f09d01f17d0 2024-04-25T10:07:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.727+0000 7f09d4b53640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00b32e0 con 0x7f09b8005910 2024-04-25T10:07:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.728+0000 7f09cdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09d0150e90 0x7f09d01565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.728+0000 7f09cdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09d0150e90 0x7f09d01565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60274/0 (socket says 172.21.15.16:60274) 2024-04-25T10:07:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.728+0000 7f09cdd9b640 1 -- 172.21.15.16:0/1826317511 learned_addr learned my addr 172.21.15.16:0/1826317511 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.728+0000 7f09ce59c640 1 --2- 172.21.15.16:0/1826317511 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09b8005910 0x7f09d014d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.728+0000 7f09ced9d640 1 --2- 172.21.15.16:0/1826317511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09d01f17d0 0x7f09d0153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.728+0000 7f09cdd9b640 1 -- 172.21.15.16:0/1826317511 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09b8005910 msgr2=0x7f09d014d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.728+0000 7f09cdd9b640 1 --2- 172.21.15.16:0/1826317511 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09b8005910 0x7f09d014d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.728+0000 7f09cdd9b640 1 -- 172.21.15.16:0/1826317511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09d01f17d0 msgr2=0x7f09d0153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.728+0000 7f09cdd9b640 1 --2- 172.21.15.16:0/1826317511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09d01f17d0 0x7f09d0153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.728+0000 7f09cdd9b640 1 -- 172.21.15.16:0/1826317511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d0156e30 con 0x7f09d0150e90 2024-04-25T10:07:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.728+0000 7f09ce59c640 1 --2- 172.21.15.16:0/1826317511 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09b8005910 0x7f09d014d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.728+0000 7f09cdd9b640 1 --2- 172.21.15.16:0/1826317511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09d0150e90 0x7f09d01565e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f09c4034040 tx=0x7f09c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.728+0000 7f09beffd640 1 -- 172.21.15.16:0/1826317511 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c4040020 con 0x7f09d0150e90 2024-04-25T10:07:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.728+0000 7f09beffd640 1 -- 172.21.15.16:0/1826317511 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c4003950 con 0x7f09d0150e90 2024-04-25T10:07:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.728+0000 7f09d4b53640 1 -- 172.21.15.16:0/1826317511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d0157110 con 0x7f09d0150e90 2024-04-25T10:07:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.728+0000 7f09beffd640 1 -- 172.21.15.16:0/1826317511 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c4036040 con 0x7f09d0150e90 2024-04-25T10:07:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.728+0000 7f09d4b53640 1 -- 172.21.15.16:0/1826317511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d01576d0 con 0x7f09d0150e90 2024-04-25T10:07:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.729+0000 7f09d4b53640 1 -- 172.21.15.16:0/1826317511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d00750b0 con 0x7f09d0150e90 2024-04-25T10:07:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.730+0000 7f09beffd640 1 -- 172.21.15.16:0/1826317511 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f09c4043020 con 0x7f09d0150e90 2024-04-25T10:07:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.730+0000 7f09beffd640 1 --2- 172.21.15.16:0/1826317511 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f09b006b500 0x7f09b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.730+0000 7f09beffd640 1 -- 172.21.15.16:0/1826317511 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f09c40aa050 con 0x7f09d0150e90 2024-04-25T10:07:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.730+0000 7f09ce59c640 1 --2- 172.21.15.16:0/1826317511 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f09b006b500 0x7f09b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.731+0000 7f09ce59c640 1 --2- 172.21.15.16:0/1826317511 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f09b006b500 0x7f09b006d9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009ae0 tx=0x7f09c8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.734+0000 7f09beffd640 1 -- 172.21.15.16:0/1826317511 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f09c404a070 con 0x7f09d0150e90 2024-04-25T10:07:48.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:48.917+0000 7f09d4b53640 1 -- 172.21.15.16:0/1826317511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f09d00cf4c0 con 0x7f09d0150e90 2024-04-25T10:07:49.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.283+0000 7f09beffd640 1 -- 172.21.15.16:0/1826317511 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v56) v1 ==== 138+0+0 (secure 0 0 0) 0x7f09c4079210 con 0x7f09d0150e90 2024-04-25T10:07:49.289 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-04-25T10:07:49.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.286+0000 7f09d4b53640 1 -- 172.21.15.16:0/1826317511 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f09b006b500 msgr2=0x7f09b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.286+0000 7f09d4b53640 1 --2- 172.21.15.16:0/1826317511 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f09b006b500 0x7f09b006d9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009ae0 tx=0x7f09c8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:49.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.286+0000 7f09d4b53640 1 -- 172.21.15.16:0/1826317511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09d0150e90 msgr2=0x7f09d01565e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.286+0000 7f09d4b53640 1 --2- 172.21.15.16:0/1826317511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09d0150e90 0x7f09d01565e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f09c4034040 tx=0x7f09c4004730 comp rx=0 tx=0).stop 2024-04-25T10:07:49.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.286+0000 7f09d4b53640 1 -- 172.21.15.16:0/1826317511 shutdown_connections 2024-04-25T10:07:49.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.286+0000 7f09d4b53640 1 --2- 172.21.15.16:0/1826317511 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f09b006b500 0x7f09b006d9c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.286+0000 7f09d4b53640 1 --2- 172.21.15.16:0/1826317511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09d01f17d0 0x7f09d0153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.286+0000 7f09d4b53640 1 --2- 172.21.15.16:0/1826317511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09d0150e90 0x7f09d01565e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.286+0000 7f09d4b53640 1 --2- 172.21.15.16:0/1826317511 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09b8005910 0x7f09d014d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.286+0000 7f09d4b53640 1 -- 172.21.15.16:0/1826317511 >> 172.21.15.16:0/1826317511 conn(0x7f09d006aba0 msgr2=0x7f09d01549f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:49.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.287+0000 7f09d4b53640 1 -- 172.21.15.16:0/1826317511 shutdown_connections 2024-04-25T10:07:49.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.288+0000 7f09d4b53640 1 -- 172.21.15.16:0/1826317511 wait complete. 2024-04-25T10:07:49.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:07:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.410+0000 7fc688db0640 1 Processor -- start 2024-04-25T10:07:49.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.411+0000 7fc688db0640 1 -- start start 2024-04-25T10:07:49.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.411+0000 7fc688db0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6840b9290 0x7fc6840d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.411+0000 7fc688db0640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc6840b9700 0x7fc6840d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.411+0000 7fc688db0640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc68415e070 con 0x7fc68415dc90 2024-04-25T10:07:49.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.411+0000 7fc688db0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6840d1b10 con 0x7fc6840b9290 2024-04-25T10:07:49.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.411+0000 7fc688db0640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6840d1df0 con 0x7fc6840b9700 2024-04-25T10:07:49.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.411+0000 7fc67bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6840b9290 0x7fc6840d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:49.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.412+0000 7fc67bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6840b9290 0x7fc6840d0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60286/0 (socket says 172.21.15.16:60286) 2024-04-25T10:07:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.412+0000 7fc67bfff640 1 -- 172.21.15.16:0/2825595573 learned_addr learned my addr 172.21.15.16:0/2825595573 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.412+0000 7fc682d9d640 1 --2- 172.21.15.16:0/2825595573 >> v2:172.21.15.186:3300/0 conn(0x7fc6840b9700 0x7fc6840d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.412+0000 7fc67bfff640 1 -- 172.21.15.16:0/2825595573 >> v2:172.21.15.186:3300/0 conn(0x7fc6840b9700 msgr2=0x7fc6840d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.412+0000 7fc67bfff640 1 --2- 172.21.15.16:0/2825595573 >> v2:172.21.15.186:3300/0 conn(0x7fc6840b9700 0x7fc6840d0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.412+0000 7fc67bfff640 1 -- 172.21.15.16:0/2825595573 >> v1:172.21.15.186:6789/0 conn(0x7fc68415dc90 legacy=0x7fc6840d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.412+0000 7fc67bfff640 1 -- 172.21.15.16:0/2825595573 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6840d20d0 con 0x7fc6840b9290 2024-04-25T10:07:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.413+0000 7fc67bfff640 1 --2- 172.21.15.16:0/2825595573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6840b9290 0x7fc6840d0420 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc6840b65a0 tx=0x7fc66c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d504261732b47449 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.413+0000 7fc681d9b640 1 -- 172.21.15.16:0/2825595573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c037070 con 0x7fc6840b9290 2024-04-25T10:07:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.413+0000 7fc681d9b640 1 -- 172.21.15.16:0/2825595573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66c032440 con 0x7fc6840b9290 2024-04-25T10:07:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.413+0000 7fc681d9b640 1 -- 172.21.15.16:0/2825595573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c032810 con 0x7fc6840b9290 2024-04-25T10:07:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.413+0000 7fc688db0640 1 -- 172.21.15.16:0/2825595573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6840b9290 msgr2=0x7fc6840d0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.413+0000 7fc688db0640 1 --2- 172.21.15.16:0/2825595573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6840b9290 0x7fc6840d0420 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc6840b65a0 tx=0x7fc66c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.413+0000 7fc688db0640 1 -- 172.21.15.16:0/2825595573 shutdown_connections 2024-04-25T10:07:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.413+0000 7fc688db0640 1 --2- 172.21.15.16:0/2825595573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6840b9290 0x7fc6840d0420 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.413+0000 7fc688db0640 1 --2- 172.21.15.16:0/2825595573 >> v2:172.21.15.186:3300/0 conn(0x7fc6840b9700 0x7fc6840d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.413+0000 7fc688db0640 1 -- 172.21.15.16:0/2825595573 >> 172.21.15.16:0/2825595573 conn(0x7fc684072900 msgr2=0x7fc68406c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.413+0000 7fc688db0640 1 -- 172.21.15.16:0/2825595573 shutdown_connections 2024-04-25T10:07:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.413+0000 7fc688db0640 1 -- 172.21.15.16:0/2825595573 wait complete. 2024-04-25T10:07:49.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.415+0000 7fc688db0640 1 Processor -- start 2024-04-25T10:07:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.415+0000 7fc688db0640 1 -- start start 2024-04-25T10:07:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.415+0000 7fc688db0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6840b9290 0x7fc684152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.415+0000 7fc688db0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6840b9700 0x7fc684155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc688db0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc68415dc90 0x7fc68415b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc688db0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840d3480 con 0x7fc68415dc90 2024-04-25T10:07:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc688db0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840d2690 con 0x7fc6840b9700 2024-04-25T10:07:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc688db0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840d3600 con 0x7fc6840b9290 2024-04-25T10:07:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc682d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc68415dc90 0x7fc68415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc682d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc68415dc90 0x7fc68415b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60290/0 (socket says 172.21.15.16:60290) 2024-04-25T10:07:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc682d9d640 1 -- 172.21.15.16:0/308363529 learned_addr learned my addr 172.21.15.16:0/308363529 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc68259c640 1 --2- 172.21.15.16:0/308363529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6840b9290 0x7fc684152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc67bfff640 1 --2- 172.21.15.16:0/308363529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6840b9700 0x7fc684155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc682d9d640 1 -- 172.21.15.16:0/308363529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6840b9290 msgr2=0x7fc684152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc682d9d640 1 --2- 172.21.15.16:0/308363529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6840b9290 0x7fc684152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc682d9d640 1 -- 172.21.15.16:0/308363529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6840b9700 msgr2=0x7fc684155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc682d9d640 1 --2- 172.21.15.16:0/308363529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6840b9700 0x7fc684155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc682d9d640 1 -- 172.21.15.16:0/308363529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6840b63f0 con 0x7fc68415dc90 2024-04-25T10:07:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc682d9d640 1 --2- 172.21.15.16:0/308363529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc68415dc90 0x7fc68415b380 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc67c007e40 tx=0x7fc67c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc67affd640 1 -- 172.21.15.16:0/308363529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c014620 con 0x7fc68415dc90 2024-04-25T10:07:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc67affd640 1 -- 172.21.15.16:0/308363529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67c019070 con 0x7fc68415dc90 2024-04-25T10:07:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc688db0640 1 -- 172.21.15.16:0/308363529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc684154510 con 0x7fc68415dc90 2024-04-25T10:07:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc67affd640 1 -- 172.21.15.16:0/308363529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c014a20 con 0x7fc68415dc90 2024-04-25T10:07:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.416+0000 7fc688db0640 1 -- 172.21.15.16:0/308363529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc684154f80 con 0x7fc68415dc90 2024-04-25T10:07:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.417+0000 7fc688db0640 1 -- 172.21.15.16:0/308363529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6840749b0 con 0x7fc68415dc90 2024-04-25T10:07:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.418+0000 7fc67affd640 1 -- 172.21.15.16:0/308363529 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc67c014bc0 con 0x7fc68415dc90 2024-04-25T10:07:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.418+0000 7fc67affd640 1 --2- 172.21.15.16:0/308363529 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc66406b4d0 0x7fc66406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.419+0000 7fc67affd640 1 -- 172.21.15.16:0/308363529 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fc67c08b5d0 con 0x7fc68415dc90 2024-04-25T10:07:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.419+0000 7fc68259c640 1 --2- 172.21.15.16:0/308363529 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc66406b4d0 0x7fc66406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.419+0000 7fc68259c640 1 --2- 172.21.15.16:0/308363529 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc66406b4d0 0x7fc66406d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc6740016e0 tx=0x7fc674009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.422+0000 7fc67affd640 1 -- 172.21.15.16:0/308363529 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc67c05a050 con 0x7fc68415dc90 2024-04-25T10:07:49.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:49.607+0000 7fc688db0640 1 -- 172.21.15.16:0/308363529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc6840b8cf0 con 0x7fc68415dc90 2024-04-25T10:07:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.288+0000 7fc67affd640 1 -- 172.21.15.16:0/308363529 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v57) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc67c05e0f0 con 0x7fc68415dc90 2024-04-25T10:07:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-04-25T10:07:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.291+0000 7fc688db0640 1 -- 172.21.15.16:0/308363529 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc66406b4d0 msgr2=0x7fc66406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.291+0000 7fc688db0640 1 --2- 172.21.15.16:0/308363529 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc66406b4d0 0x7fc66406d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc6740016e0 tx=0x7fc674009040 comp rx=0 tx=0).stop 2024-04-25T10:07:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.291+0000 7fc688db0640 1 -- 172.21.15.16:0/308363529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc68415dc90 msgr2=0x7fc68415b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.291+0000 7fc688db0640 1 --2- 172.21.15.16:0/308363529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc68415dc90 0x7fc68415b380 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc67c007e40 tx=0x7fc67c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.293+0000 7fc688db0640 1 -- 172.21.15.16:0/308363529 shutdown_connections 2024-04-25T10:07:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.293+0000 7fc688db0640 1 --2- 172.21.15.16:0/308363529 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc66406b4d0 0x7fc66406d990 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.293+0000 7fc688db0640 1 --2- 172.21.15.16:0/308363529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc68415dc90 0x7fc68415b380 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.293+0000 7fc688db0640 1 --2- 172.21.15.16:0/308363529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6840b9700 0x7fc684155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.293+0000 7fc688db0640 1 --2- 172.21.15.16:0/308363529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6840b9290 0x7fc684152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.293+0000 7fc688db0640 1 -- 172.21.15.16:0/308363529 >> 172.21.15.16:0/308363529 conn(0x7fc684072900 msgr2=0x7fc6840b8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.293+0000 7fc688db0640 1 -- 172.21.15.16:0/308363529 shutdown_connections 2024-04-25T10:07:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.293+0000 7fc688db0640 1 -- 172.21.15.16:0/308363529 wait complete. 2024-04-25T10:07:50.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:07:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.434+0000 7fbb1ec8f640 1 Processor -- start 2024-04-25T10:07:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.434+0000 7fbb1ec8f640 1 -- start start 2024-04-25T10:07:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.435+0000 7fbb1ec8f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb1815dc80 0x7fbb18161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.435+0000 7fbb1ec8f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb180b5c00 0x7fbb180d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.435+0000 7fbb1ec8f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb181627a0 con 0x7fbb180b97c0 2024-04-25T10:07:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.435+0000 7fbb1ec8f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb181621e0 con 0x7fbb1815dc80 2024-04-25T10:07:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.435+0000 7fbb1ec8f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb181624c0 con 0x7fbb180b5c00 2024-04-25T10:07:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.435+0000 7fbb1ca2b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb1815dc80 0x7fbb18161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.435+0000 7fbb17fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbb180b97c0 0x7fbb180b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57134/0 (socket says 172.21.15.16:57134) 2024-04-25T10:07:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.435+0000 7fbb17fff640 1 -- 172.21.15.16:0/3258749456 learned_addr learned my addr 172.21.15.16:0/3258749456 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.435+0000 7fbb1d22c640 1 --2- 172.21.15.16:0/3258749456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb180b5c00 0x7fbb180d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.435+0000 7fbb1ca2b640 1 -- 172.21.15.16:0/3258749456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb180b5c00 msgr2=0x7fbb180d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.435+0000 7fbb1ca2b640 1 --2- 172.21.15.16:0/3258749456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb180b5c00 0x7fbb180d2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.435+0000 7fbb1ca2b640 1 -- 172.21.15.16:0/3258749456 >> v1:172.21.15.16:6789/0 conn(0x7fbb180b97c0 legacy=0x7fbb180b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.435+0000 7fbb177fe640 1 -- 172.21.15.16:0/3258749456 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636938745 0 0) 0x7fbb181627a0 con 0x7fbb180b97c0 2024-04-25T10:07:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.435+0000 7fbb1ca2b640 1 -- 172.21.15.16:0/3258749456 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180d1c70 con 0x7fbb1815dc80 2024-04-25T10:07:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.436+0000 7fbb1ca2b640 1 --2- 172.21.15.16:0/3258749456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1815dc80 0x7fbb18161870 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0097a0 tx=0x7fbb0c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264dbd64da63d723 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.436+0000 7fbb177fe640 1 -- 172.21.15.16:0/3258749456 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c037070 con 0x7fbb1815dc80 2024-04-25T10:07:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.436+0000 7fbb177fe640 1 -- 172.21.15.16:0/3258749456 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c032440 con 0x7fbb1815dc80 2024-04-25T10:07:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.436+0000 7fbb177fe640 1 -- 172.21.15.16:0/3258749456 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c032810 con 0x7fbb1815dc80 2024-04-25T10:07:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.437+0000 7fbb1ec8f640 1 -- 172.21.15.16:0/3258749456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1815dc80 msgr2=0x7fbb18161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.437+0000 7fbb1ec8f640 1 --2- 172.21.15.16:0/3258749456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1815dc80 0x7fbb18161870 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0097a0 tx=0x7fbb0c02a180 comp rx=0 tx=0).stop 2024-04-25T10:07:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.437+0000 7fbb1ec8f640 1 -- 172.21.15.16:0/3258749456 shutdown_connections 2024-04-25T10:07:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.437+0000 7fbb1ec8f640 1 --2- 172.21.15.16:0/3258749456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb180b5c00 0x7fbb180d2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.437+0000 7fbb1ec8f640 1 --2- 172.21.15.16:0/3258749456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1815dc80 0x7fbb18161870 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.437+0000 7fbb1ec8f640 1 -- 172.21.15.16:0/3258749456 >> 172.21.15.16:0/3258749456 conn(0x7fbb180d04f0 msgr2=0x7fbb180d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.437+0000 7fbb1ec8f640 1 -- 172.21.15.16:0/3258749456 shutdown_connections 2024-04-25T10:07:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.437+0000 7fbb1ec8f640 1 -- 172.21.15.16:0/3258749456 wait complete. 2024-04-25T10:07:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.438+0000 7fbb1ec8f640 1 Processor -- start 2024-04-25T10:07:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.438+0000 7fbb1ec8f640 1 -- start start 2024-04-25T10:07:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.439+0000 7fbb1ec8f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb180b5c00 0x7fbb181f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.439+0000 7fbb1ec8f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb180b97c0 0x7fbb1815e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.439+0000 7fbb1ec8f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb1815dc80 0x7fbb1815e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.439+0000 7fbb1ec8f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18161f30 con 0x7fbb180b5c00 2024-04-25T10:07:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.439+0000 7fbb1ec8f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18161db0 con 0x7fbb180b97c0 2024-04-25T10:07:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.439+0000 7fbb1ec8f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180d29b0 con 0x7fbb1815dc80 2024-04-25T10:07:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.439+0000 7fbb1ca2b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb180b5c00 0x7fbb181f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.439+0000 7fbb1ca2b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb180b5c00 0x7fbb181f7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60314/0 (socket says 172.21.15.16:60314) 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.439+0000 7fbb1ca2b640 1 -- 172.21.15.16:0/2834011460 learned_addr learned my addr 172.21.15.16:0/2834011460 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.439+0000 7fbb1d22c640 1 --2- 172.21.15.16:0/2834011460 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb1815dc80 0x7fbb1815e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.439+0000 7fbb1ca2b640 1 -- 172.21.15.16:0/2834011460 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb1815dc80 msgr2=0x7fbb1815e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.439+0000 7fbb1ca2b640 1 --2- 172.21.15.16:0/2834011460 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb1815dc80 0x7fbb1815e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.439+0000 7fbb1ca2b640 1 -- 172.21.15.16:0/2834011460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb180b97c0 msgr2=0x7fbb1815e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.439+0000 7fbb17fff640 1 --2- 172.21.15.16:0/2834011460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb180b97c0 0x7fbb1815e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.439+0000 7fbb1ca2b640 1 --2- 172.21.15.16:0/2834011460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb180b97c0 0x7fbb1815e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.439+0000 7fbb1ca2b640 1 -- 172.21.15.16:0/2834011460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb181f47c0 con 0x7fbb180b5c00 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.439+0000 7fbb1d22c640 1 --2- 172.21.15.16:0/2834011460 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb1815dc80 0x7fbb1815e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.439+0000 7fbb17fff640 1 --2- 172.21.15.16:0/2834011460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb180b97c0 0x7fbb1815e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.440+0000 7fbb1ca2b640 1 --2- 172.21.15.16:0/2834011460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb180b5c00 0x7fbb181f7620 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c035040 tx=0x7fbb0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.440+0000 7fbb157fa640 1 -- 172.21.15.16:0/2834011460 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c037040 con 0x7fbb180b5c00 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.440+0000 7fbb157fa640 1 -- 172.21.15.16:0/2834011460 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c030770 con 0x7fbb180b5c00 2024-04-25T10:07:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.440+0000 7fbb157fa640 1 -- 172.21.15.16:0/2834011460 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c030b80 con 0x7fbb180b5c00 2024-04-25T10:07:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.440+0000 7fbb1ec8f640 1 -- 172.21.15.16:0/2834011460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb180c7380 con 0x7fbb180b5c00 2024-04-25T10:07:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.440+0000 7fbb1ec8f640 1 -- 172.21.15.16:0/2834011460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb181606a0 con 0x7fbb180b5c00 2024-04-25T10:07:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.441+0000 7fbb1ec8f640 1 -- 172.21.15.16:0/2834011460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb180750b0 con 0x7fbb180b5c00 2024-04-25T10:07:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.441+0000 7fbb157fa640 1 -- 172.21.15.16:0/2834011460 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb0c044020 con 0x7fbb180b5c00 2024-04-25T10:07:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.442+0000 7fbb157fa640 1 --2- 172.21.15.16:0/2834011460 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbafc06b4d0 0x7fbafc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.442+0000 7fbb157fa640 1 -- 172.21.15.16:0/2834011460 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fbb0c0aa710 con 0x7fbb180b5c00 2024-04-25T10:07:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.442+0000 7fbb17fff640 1 --2- 172.21.15.16:0/2834011460 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbafc06b4d0 0x7fbafc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.443+0000 7fbb17fff640 1 --2- 172.21.15.16:0/2834011460 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbafc06b4d0 0x7fbafc06d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbb18157730 tx=0x7fbb04009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.446+0000 7fbb157fa640 1 -- 172.21.15.16:0/2834011460 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb180c7380 con 0x7fbb180b5c00 2024-04-25T10:07:50.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:50.663+0000 7fbb1ec8f640 1 -- 172.21.15.16:0/2834011460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbb180d1780 con 0x7fbb180b5c00 2024-04-25T10:07:51.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.290+0000 7fbb157fa640 1 -- 172.21.15.16:0/2834011460 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v25) v1 ==== 172+0+0 (secure 0 0 0) 0x7fbb0c032e20 con 0x7fbb180b5c00 2024-04-25T10:07:51.296 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 8 and data pool 9 2024-04-25T10:07:51.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.302+0000 7fbb1ec8f640 1 -- 172.21.15.16:0/2834011460 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbafc06b4d0 msgr2=0x7fbafc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.302+0000 7fbb1ec8f640 1 --2- 172.21.15.16:0/2834011460 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbafc06b4d0 0x7fbafc06d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbb18157730 tx=0x7fbb04009040 comp rx=0 tx=0).stop 2024-04-25T10:07:51.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.302+0000 7fbb1ec8f640 1 -- 172.21.15.16:0/2834011460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb180b5c00 msgr2=0x7fbb181f7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.302+0000 7fbb1ec8f640 1 --2- 172.21.15.16:0/2834011460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb180b5c00 0x7fbb181f7620 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c035040 tx=0x7fbb0c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.306+0000 7fbb1ec8f640 1 -- 172.21.15.16:0/2834011460 shutdown_connections 2024-04-25T10:07:51.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.306+0000 7fbb1ec8f640 1 --2- 172.21.15.16:0/2834011460 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbafc06b4d0 0x7fbafc06d990 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.306+0000 7fbb1ec8f640 1 --2- 172.21.15.16:0/2834011460 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb1815dc80 0x7fbb1815e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.306+0000 7fbb1ec8f640 1 --2- 172.21.15.16:0/2834011460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb180b97c0 0x7fbb1815e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.306+0000 7fbb1ec8f640 1 --2- 172.21.15.16:0/2834011460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb180b5c00 0x7fbb181f7620 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.306+0000 7fbb1ec8f640 1 -- 172.21.15.16:0/2834011460 >> 172.21.15.16:0/2834011460 conn(0x7fbb180d04f0 msgr2=0x7fbb180b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:51.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.307+0000 7fbb1ec8f640 1 -- 172.21.15.16:0/2834011460 shutdown_connections 2024-04-25T10:07:51.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.309+0000 7fbb1ec8f640 1 -- 172.21.15.16:0/2834011460 wait complete. 2024-04-25T10:07:51.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.436+0000 7f34da7d5640 1 Processor -- start 2024-04-25T10:07:51.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.436+0000 7f34da7d5640 1 -- start start 2024-04-25T10:07:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.436+0000 7f34da7d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d415dd50 0x7f34d40d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34da7d5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f34d40ceeb0 0x7f34d40b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34da7d5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34d40d2440 con 0x7f34d40b9460 2024-04-25T10:07:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34da7d5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d40d2720 con 0x7f34d40ceeb0 2024-04-25T10:07:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34da7d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d40d2a00 con 0x7f34d415dd50 2024-04-25T10:07:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d37fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f34d40b9460 0x7f34d40ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57146/0 (socket says 172.21.15.16:57146) 2024-04-25T10:07:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d37fe640 1 -- 172.21.15.16:0/2550446433 learned_addr learned my addr 172.21.15.16:0/2550446433 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d3fff640 1 --2- 172.21.15.16:0/2550446433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d415dd50 0x7f34d40d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d8d72640 1 --2- 172.21.15.16:0/2550446433 >> v2:172.21.15.186:3300/0 conn(0x7f34d40ceeb0 0x7f34d40b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d2ffd640 1 -- 172.21.15.16:0/2550446433 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461245916 0 0) 0x7f34d40d2440 con 0x7f34d40b9460 2024-04-25T10:07:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d2ffd640 1 -- 172.21.15.16:0/2550446433 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34c0003520 con 0x7f34d40b9460 2024-04-25T10:07:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d2ffd640 1 -- 172.21.15.16:0/2550446433 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4263135253 0 0) 0x7f34c0003520 con 0x7f34d40b9460 2024-04-25T10:07:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d2ffd640 1 -- 172.21.15.16:0/2550446433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d415dd50 msgr2=0x7f34d40d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d2ffd640 1 --2- 172.21.15.16:0/2550446433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d415dd50 0x7f34d40d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d2ffd640 1 -- 172.21.15.16:0/2550446433 >> v2:172.21.15.186:3300/0 conn(0x7f34d40ceeb0 msgr2=0x7f34d40b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d2ffd640 1 --2- 172.21.15.16:0/2550446433 >> v2:172.21.15.186:3300/0 conn(0x7f34d40ceeb0 0x7f34d40b6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d2ffd640 1 -- 172.21.15.16:0/2550446433 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d40d2ce0 con 0x7f34d40b9460 2024-04-25T10:07:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d2ffd640 1 -- 172.21.15.16:0/2550446433 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f34bc001670 con 0x7f34d40b9460 2024-04-25T10:07:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d2ffd640 1 -- 172.21.15.16:0/2550446433 >> v1:172.21.15.16:6789/0 conn(0x7f34d40b9460 legacy=0x7f34d40ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d2ffd640 1 --2- 172.21.15.16:0/2550446433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c00039b0 0x7f34c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d2ffd640 1 --2- 172.21.15.16:0/2550446433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d415dd50 0x7f34c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d2ffd640 1 --2- 172.21.15.16:0/2550446433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34c0004d40 0x7f34c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d2ffd640 1 -- 172.21.15.16:0/2550446433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d40d2720 con 0x7f34c00039b0 2024-04-25T10:07:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d2ffd640 1 -- 172.21.15.16:0/2550446433 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d40d2a00 con 0x7f34c0004d40 2024-04-25T10:07:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.437+0000 7f34d2ffd640 1 -- 172.21.15.16:0/2550446433 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c000b9c0 con 0x7f34d415dd50 2024-04-25T10:07:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.438+0000 7f34d3fff640 1 --2- 172.21.15.16:0/2550446433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c00039b0 0x7f34c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.438+0000 7f34d8d72640 1 --2- 172.21.15.16:0/2550446433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34c0004d40 0x7f34c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.438+0000 7f34d37fe640 1 --2- 172.21.15.16:0/2550446433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d415dd50 0x7f34c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.438+0000 7f34d3fff640 1 -- 172.21.15.16:0/2550446433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d415dd50 msgr2=0x7f34c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.438+0000 7f34d3fff640 1 --2- 172.21.15.16:0/2550446433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d415dd50 0x7f34c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.438+0000 7f34d3fff640 1 -- 172.21.15.16:0/2550446433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34c0004d40 msgr2=0x7f34c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.438+0000 7f34d3fff640 1 --2- 172.21.15.16:0/2550446433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34c0004d40 0x7f34c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.438+0000 7f34d3fff640 1 -- 172.21.15.16:0/2550446433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c0003520 con 0x7f34c00039b0 2024-04-25T10:07:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.438+0000 7f34d37fe640 1 --2- 172.21.15.16:0/2550446433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d415dd50 0x7f34c0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.438+0000 7f34d3fff640 1 --2- 172.21.15.16:0/2550446433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c00039b0 0x7f34c0003dd0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009ac0 tx=0x7f34c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd4e076c26888f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.439+0000 7f34d2ffd640 1 -- 172.21.15.16:0/2550446433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4036070 con 0x7f34c00039b0 2024-04-25T10:07:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.439+0000 7f34d2ffd640 1 -- 172.21.15.16:0/2550446433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c4031400 con 0x7f34c00039b0 2024-04-25T10:07:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.439+0000 7f34d2ffd640 1 -- 172.21.15.16:0/2550446433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4009690 con 0x7f34c00039b0 2024-04-25T10:07:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.439+0000 7f34da7d5640 1 -- 172.21.15.16:0/2550446433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c00039b0 msgr2=0x7f34c0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.439+0000 7f34da7d5640 1 --2- 172.21.15.16:0/2550446433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c00039b0 0x7f34c0003dd0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009ac0 tx=0x7f34c0005830 comp rx=0 tx=0).stop 2024-04-25T10:07:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.439+0000 7f34d8d72640 1 -- 172.21.15.16:0/2550446433 reap_dead start 2024-04-25T10:07:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.439+0000 7f34da7d5640 1 -- 172.21.15.16:0/2550446433 shutdown_connections 2024-04-25T10:07:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.439+0000 7f34da7d5640 1 -- 172.21.15.16:0/2550446433 >> 172.21.15.16:0/2550446433 conn(0x7f34d40722d0 msgr2=0x7f34d40726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.439+0000 7f34da7d5640 1 -- 172.21.15.16:0/2550446433 shutdown_connections 2024-04-25T10:07:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.439+0000 7f34da7d5640 1 -- 172.21.15.16:0/2550446433 wait complete. 2024-04-25T10:07:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.440+0000 7f34da7d5640 1 Processor -- start 2024-04-25T10:07:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.441+0000 7f34da7d5640 1 -- start start 2024-04-25T10:07:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.441+0000 7f34da7d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c00039b0 0x7f34d41f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.441+0000 7f34da7d5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34d40ccd70 0x7f34d40cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.441+0000 7f34da7d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d41f6ff0 0x7f34d41f7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.441+0000 7f34da7d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d40d3660 con 0x7f34c00039b0 2024-04-25T10:07:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.441+0000 7f34da7d5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d40d34e0 con 0x7f34d40ccd70 2024-04-25T10:07:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.441+0000 7f34da7d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d40d37e0 con 0x7f34d41f6ff0 2024-04-25T10:07:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.441+0000 7f34d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c00039b0 0x7f34d41f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.441+0000 7f34d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c00039b0 0x7f34d41f32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60322/0 (socket says 172.21.15.16:60322) 2024-04-25T10:07:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.441+0000 7f34d3fff640 1 -- 172.21.15.16:0/4186430827 learned_addr learned my addr 172.21.15.16:0/4186430827 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.442+0000 7f34d8d72640 1 --2- 172.21.15.16:0/4186430827 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d41f6ff0 0x7f34d41f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.442+0000 7f34d37fe640 1 --2- 172.21.15.16:0/4186430827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34d40ccd70 0x7f34d40cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.442+0000 7f34d3fff640 1 -- 172.21.15.16:0/4186430827 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d41f6ff0 msgr2=0x7f34d41f7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.442+0000 7f34d3fff640 1 --2- 172.21.15.16:0/4186430827 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d41f6ff0 0x7f34d41f7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.442+0000 7f34d3fff640 1 -- 172.21.15.16:0/4186430827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34d40ccd70 msgr2=0x7f34d40cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.442+0000 7f34d3fff640 1 --2- 172.21.15.16:0/4186430827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34d40ccd70 0x7f34d40cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.442+0000 7f34d3fff640 1 -- 172.21.15.16:0/4186430827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d41f37e0 con 0x7f34c00039b0 2024-04-25T10:07:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.442+0000 7f34d3fff640 1 --2- 172.21.15.16:0/4186430827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c00039b0 0x7f34d41f32a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009be0 tx=0x7f34c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.442+0000 7f34d0ff9640 1 -- 172.21.15.16:0/4186430827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4040020 con 0x7f34c00039b0 2024-04-25T10:07:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.442+0000 7f34d0ff9640 1 -- 172.21.15.16:0/4186430827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c4003bc0 con 0x7f34c00039b0 2024-04-25T10:07:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.442+0000 7f34da7d5640 1 -- 172.21.15.16:0/4186430827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d41f7d20 con 0x7f34c00039b0 2024-04-25T10:07:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.442+0000 7f34d0ff9640 1 -- 172.21.15.16:0/4186430827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4036040 con 0x7f34c00039b0 2024-04-25T10:07:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.442+0000 7f34da7d5640 1 -- 172.21.15.16:0/4186430827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d41600f0 con 0x7f34c00039b0 2024-04-25T10:07:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.443+0000 7f34da7d5640 1 -- 172.21.15.16:0/4186430827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d4067850 con 0x7f34c00039b0 2024-04-25T10:07:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.444+0000 7f34d0ff9640 1 -- 172.21.15.16:0/4186430827 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34c4043020 con 0x7f34c00039b0 2024-04-25T10:07:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.444+0000 7f34d0ff9640 1 --2- 172.21.15.16:0/4186430827 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f34b406b4d0 0x7f34b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.444+0000 7f34d0ff9640 1 -- 172.21.15.16:0/4186430827 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34c40aa340 con 0x7f34c00039b0 2024-04-25T10:07:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.444+0000 7f34d37fe640 1 --2- 172.21.15.16:0/4186430827 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f34b406b4d0 0x7f34b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.445+0000 7f34d37fe640 1 --2- 172.21.15.16:0/4186430827 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f34b406b4d0 0x7f34b406d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f34d41f6e70 tx=0x7f34bc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.448+0000 7f34d0ff9640 1 -- 172.21.15.16:0/4186430827 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34c404a070 con 0x7f34c00039b0 2024-04-25T10:07:51.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.628+0000 7f34da7d5640 1 -- 172.21.15.16:0/4186430827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34d40749b0 con 0x7f34c00039b0 2024-04-25T10:07:51.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.629+0000 7f34d0ff9640 1 -- 172.21.15.16:0/4186430827 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16678 (secure 0 0 0) 0x7f34c40044e0 con 0x7f34c00039b0 2024-04-25T10:07:51.635 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:51.635 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":58,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:07:50.665140+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:48.948980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:49.638457+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.631+0000 7f34da7d5640 1 -- 172.21.15.16:0/4186430827 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f34b406b4d0 msgr2=0x7f34b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.631+0000 7f34da7d5640 1 --2- 172.21.15.16:0/4186430827 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f34b406b4d0 0x7f34b406d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f34d41f6e70 tx=0x7f34bc009040 comp rx=0 tx=0).stop 2024-04-25T10:07:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.631+0000 7f34da7d5640 1 -- 172.21.15.16:0/4186430827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c00039b0 msgr2=0x7f34d41f32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.631+0000 7f34da7d5640 1 --2- 172.21.15.16:0/4186430827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c00039b0 0x7f34d41f32a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009be0 tx=0x7f34c4004730 comp rx=0 tx=0).stop 2024-04-25T10:07:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.631+0000 7f34da7d5640 1 -- 172.21.15.16:0/4186430827 shutdown_connections 2024-04-25T10:07:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.631+0000 7f34da7d5640 1 --2- 172.21.15.16:0/4186430827 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f34b406b4d0 0x7f34b406d990 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.631+0000 7f34da7d5640 1 --2- 172.21.15.16:0/4186430827 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d41f6ff0 0x7f34d41f7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.631+0000 7f34da7d5640 1 --2- 172.21.15.16:0/4186430827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34d40ccd70 0x7f34d40cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.631+0000 7f34da7d5640 1 --2- 172.21.15.16:0/4186430827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c00039b0 0x7f34d41f32a0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.631+0000 7f34da7d5640 1 -- 172.21.15.16:0/4186430827 >> 172.21.15.16:0/4186430827 conn(0x7f34d40722d0 msgr2=0x7f34d415aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.631+0000 7f34da7d5640 1 -- 172.21.15.16:0/4186430827 shutdown_connections 2024-04-25T10:07:51.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.631+0000 7f34da7d5640 1 -- 172.21.15.16:0/4186430827 wait complete. 2024-04-25T10:07:51.648 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.743+0000 7f974e4f5640 1 Processor -- start 2024-04-25T10:07:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.743+0000 7f974e4f5640 1 -- start start 2024-04-25T10:07:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.744+0000 7f974e4f5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f974815dc30 0x7f974806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.744+0000 7f974e4f5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f974806cc00 0x7f97480b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.744+0000 7f974e4f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97480b1960 0x7f9748099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.744+0000 7f974e4f5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f974806c1f0 con 0x7f974806cc00 2024-04-25T10:07:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.744+0000 7f974e4f5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97480cfb40 con 0x7f974815dc30 2024-04-25T10:07:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.744+0000 7f974e4f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97480b1db0 con 0x7f97480b1960 2024-04-25T10:07:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.744+0000 7f97477fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f974806cc00 0x7f97480b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.744+0000 7f97477fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f974806cc00 0x7f97480b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60326/0 (socket says 172.21.15.16:60326) 2024-04-25T10:07:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.744+0000 7f97477fe640 1 -- 172.21.15.16:0/1028137423 learned_addr learned my addr 172.21.15.16:0/1028137423 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.745+0000 7f974ca92640 1 --2- 172.21.15.16:0/1028137423 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97480b1960 0x7f9748099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.745+0000 7f9747fff640 1 --2- 172.21.15.16:0/1028137423 >> v2:172.21.15.186:3300/0 conn(0x7f974815dc30 0x7f974806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.745+0000 7f97477fe640 1 -- 172.21.15.16:0/1028137423 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97480b1960 msgr2=0x7f9748099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.745+0000 7f97477fe640 1 --2- 172.21.15.16:0/1028137423 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97480b1960 0x7f9748099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.745+0000 7f97477fe640 1 -- 172.21.15.16:0/1028137423 >> v2:172.21.15.186:3300/0 conn(0x7f974815dc30 msgr2=0x7f974806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.745+0000 7f97477fe640 1 --2- 172.21.15.16:0/1028137423 >> v2:172.21.15.186:3300/0 conn(0x7f974815dc30 0x7f974806c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.745+0000 7f97477fe640 1 -- 172.21.15.16:0/1028137423 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97480b2060 con 0x7f974806cc00 2024-04-25T10:07:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.745+0000 7f97477fe640 1 --2- 172.21.15.16:0/1028137423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f974806cc00 0x7f97480b1420 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f974809ab70 tx=0x7f973c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f557f7bcefa9c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.746+0000 7f9746ffd640 1 -- 172.21.15.16:0/1028137423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c037070 con 0x7f974806cc00 2024-04-25T10:07:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.746+0000 7f9746ffd640 1 -- 172.21.15.16:0/1028137423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973c032440 con 0x7f974806cc00 2024-04-25T10:07:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.746+0000 7f9746ffd640 1 -- 172.21.15.16:0/1028137423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c032810 con 0x7f974806cc00 2024-04-25T10:07:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.746+0000 7f974e4f5640 1 -- 172.21.15.16:0/1028137423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f974806cc00 msgr2=0x7f97480b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.746+0000 7f974e4f5640 1 --2- 172.21.15.16:0/1028137423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f974806cc00 0x7f97480b1420 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f974809ab70 tx=0x7f973c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.746+0000 7f974e4f5640 1 -- 172.21.15.16:0/1028137423 shutdown_connections 2024-04-25T10:07:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.746+0000 7f974e4f5640 1 --2- 172.21.15.16:0/1028137423 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97480b1960 0x7f9748099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.746+0000 7f974e4f5640 1 --2- 172.21.15.16:0/1028137423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f974806cc00 0x7f97480b1420 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.746+0000 7f974e4f5640 1 --2- 172.21.15.16:0/1028137423 >> v2:172.21.15.186:3300/0 conn(0x7f974815dc30 0x7f974806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.746+0000 7f974e4f5640 1 -- 172.21.15.16:0/1028137423 >> 172.21.15.16:0/1028137423 conn(0x7f97480d1900 msgr2=0x7f97480d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.746+0000 7f974e4f5640 1 -- 172.21.15.16:0/1028137423 shutdown_connections 2024-04-25T10:07:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.746+0000 7f974e4f5640 1 -- 172.21.15.16:0/1028137423 wait complete. 2024-04-25T10:07:51.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.747+0000 7f974e4f5640 1 Processor -- start 2024-04-25T10:07:51.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.748+0000 7f974e4f5640 1 -- start start 2024-04-25T10:07:51.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.748+0000 7f974e4f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f974806cc00 0x7f9748154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.748+0000 7f974e4f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97480b1960 0x7f97480b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.748+0000 7f974e4f5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f974815dc30 0x7f97480b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.748+0000 7f974e4f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97480cfde0 con 0x7f974806cc00 2024-04-25T10:07:51.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.748+0000 7f974e4f5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97480b2620 con 0x7f974815dc30 2024-04-25T10:07:51.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.748+0000 7f974e4f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974815e010 con 0x7f97480b1960 2024-04-25T10:07:51.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.749+0000 7f9747fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f974806cc00 0x7f9748154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.749+0000 7f9747fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f974806cc00 0x7f9748154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60342/0 (socket says 172.21.15.16:60342) 2024-04-25T10:07:51.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.749+0000 7f9747fff640 1 -- 172.21.15.16:0/977164383 learned_addr learned my addr 172.21.15.16:0/977164383 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:51.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.749+0000 7f97477fe640 1 --2- 172.21.15.16:0/977164383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97480b1960 0x7f97480b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.749+0000 7f974ca92640 1 --2- 172.21.15.16:0/977164383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f974815dc30 0x7f97480b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.749+0000 7f9747fff640 1 -- 172.21.15.16:0/977164383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97480b1960 msgr2=0x7f97480b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.749+0000 7f9747fff640 1 --2- 172.21.15.16:0/977164383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97480b1960 0x7f97480b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.749+0000 7f9747fff640 1 -- 172.21.15.16:0/977164383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f974815dc30 msgr2=0x7f97480b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.749+0000 7f9747fff640 1 --2- 172.21.15.16:0/977164383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f974815dc30 0x7f97480b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.749+0000 7f9747fff640 1 -- 172.21.15.16:0/977164383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97481553a0 con 0x7f974806cc00 2024-04-25T10:07:51.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.749+0000 7f974ca92640 1 --2- 172.21.15.16:0/977164383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f974815dc30 0x7f97480b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:51.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.749+0000 7f9747fff640 1 --2- 172.21.15.16:0/977164383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f974806cc00 0x7f9748154e60 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f973800e9b0 tx=0x7f973800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:51.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.749+0000 7f9744ff9640 1 -- 172.21.15.16:0/977164383 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738017070 con 0x7f974806cc00 2024-04-25T10:07:51.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.749+0000 7f9744ff9640 1 -- 172.21.15.16:0/977164383 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973800c9a0 con 0x7f974806cc00 2024-04-25T10:07:51.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.749+0000 7f974e4f5640 1 -- 172.21.15.16:0/977164383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97481e7ae0 con 0x7f974806cc00 2024-04-25T10:07:51.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.749+0000 7f9744ff9640 1 -- 172.21.15.16:0/977164383 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973800cd90 con 0x7f974806cc00 2024-04-25T10:07:51.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.749+0000 7f974e4f5640 1 -- 172.21.15.16:0/977164383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97481e80a0 con 0x7f974806cc00 2024-04-25T10:07:51.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.750+0000 7f974e4f5640 1 -- 172.21.15.16:0/977164383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9748067850 con 0x7f974806cc00 2024-04-25T10:07:51.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.751+0000 7f9744ff9640 1 -- 172.21.15.16:0/977164383 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f973801a070 con 0x7f974806cc00 2024-04-25T10:07:51.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.751+0000 7f9744ff9640 1 --2- 172.21.15.16:0/977164383 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f973406b4d0 0x7f973406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.751+0000 7f9744ff9640 1 -- 172.21.15.16:0/977164383 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f973808a230 con 0x7f974806cc00 2024-04-25T10:07:51.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.751+0000 7f97477fe640 1 --2- 172.21.15.16:0/977164383 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f973406b4d0 0x7f973406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.752+0000 7f97477fe640 1 --2- 172.21.15.16:0/977164383 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f973406b4d0 0x7f973406d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f973c034000 tx=0x7f973c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:51.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.755+0000 7f9744ff9640 1 -- 172.21.15.16:0/977164383 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97380589c0 con 0x7f974806cc00 2024-04-25T10:07:51.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.938+0000 7f974e4f5640 1 -- 172.21.15.16:0/977164383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97481e82e0 con 0x7f974806cc00 2024-04-25T10:07:51.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.939+0000 7f9744ff9640 1 -- 172.21.15.16:0/977164383 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16678 (secure 0 0 0) 0x7f97481e82e0 con 0x7f974806cc00 2024-04-25T10:07:51.945 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:51.945 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":58,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:07:50.665140+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:48.948980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:49.638457+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:51.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.941+0000 7f974e4f5640 1 -- 172.21.15.16:0/977164383 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f973406b4d0 msgr2=0x7f973406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.941+0000 7f974e4f5640 1 --2- 172.21.15.16:0/977164383 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f973406b4d0 0x7f973406d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f973c034000 tx=0x7f973c02e000 comp rx=0 tx=0).stop 2024-04-25T10:07:51.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.941+0000 7f974e4f5640 1 -- 172.21.15.16:0/977164383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f974806cc00 msgr2=0x7f9748154e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.941+0000 7f974e4f5640 1 --2- 172.21.15.16:0/977164383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f974806cc00 0x7f9748154e60 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f973800e9b0 tx=0x7f973800ee70 comp rx=0 tx=0).stop 2024-04-25T10:07:51.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.941+0000 7f974e4f5640 1 -- 172.21.15.16:0/977164383 shutdown_connections 2024-04-25T10:07:51.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.941+0000 7f974e4f5640 1 --2- 172.21.15.16:0/977164383 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f973406b4d0 0x7f973406d990 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.941+0000 7f974e4f5640 1 --2- 172.21.15.16:0/977164383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f974815dc30 0x7f97480b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.941+0000 7f974e4f5640 1 --2- 172.21.15.16:0/977164383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97480b1960 0x7f97480b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.941+0000 7f974e4f5640 1 --2- 172.21.15.16:0/977164383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f974806cc00 0x7f9748154e60 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.941+0000 7f974e4f5640 1 -- 172.21.15.16:0/977164383 >> 172.21.15.16:0/977164383 conn(0x7f97480d1900 msgr2=0x7f97480d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:51.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.941+0000 7f974e4f5640 1 -- 172.21.15.16:0/977164383 shutdown_connections 2024-04-25T10:07:51.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:51.941+0000 7f974e4f5640 1 -- 172.21.15.16:0/977164383 wait complete. 2024-04-25T10:07:51.958 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:07:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.056+0000 7ff9abb89640 1 Processor -- start 2024-04-25T10:07:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.056+0000 7ff9abb89640 1 -- start start 2024-04-25T10:07:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.057+0000 7ff9abb89640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9a415dca0 0x7ff9a40ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.057+0000 7ff9abb89640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff9a40d3b90 0x7ff9a40d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.057+0000 7ff9abb89640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a40d4090 con 0x7ff9a40cee80 2024-04-25T10:07:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.057+0000 7ff9abb89640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a40b6b20 con 0x7ff9a40d3b90 2024-04-25T10:07:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.057+0000 7ff9abb89640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a40d4370 con 0x7ff9a415dca0 2024-04-25T10:07:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.057+0000 7ff9a9124640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff9a40d3b90 0x7ff9a40d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.057+0000 7ff9a9124640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff9a40d3b90 0x7ff9a40d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60348/0 (socket says 172.21.15.16:60348) 2024-04-25T10:07:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.057+0000 7ff9a9124640 1 -- 172.21.15.16:0/1860402225 learned_addr learned my addr 172.21.15.16:0/1860402225 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.057+0000 7ff9a9925640 1 --2- 172.21.15.16:0/1860402225 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9a415dca0 0x7ff9a40ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.057+0000 7ff9a9124640 1 -- 172.21.15.16:0/1860402225 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9a415dca0 msgr2=0x7ff9a40ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.057+0000 7ff9a9124640 1 --2- 172.21.15.16:0/1860402225 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9a415dca0 0x7ff9a40ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.057+0000 7ff9a9124640 1 -- 172.21.15.16:0/1860402225 >> v1:172.21.15.186:6789/0 conn(0x7ff9a40cee80 legacy=0x7ff9a40b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.058+0000 7ff9a9124640 1 -- 172.21.15.16:0/1860402225 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a40d4650 con 0x7ff9a40d3b90 2024-04-25T10:07:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.058+0000 7ff9a9124640 1 --2- 172.21.15.16:0/1860402225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a40d3b90 0x7ff9a40d0a20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff998009870 tx=0x7ff99802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7b9749a6286c76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.058+0000 7ff9a8923640 1 -- 172.21.15.16:0/1860402225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998037070 con 0x7ff9a40d3b90 2024-04-25T10:07:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.058+0000 7ff9a8923640 1 -- 172.21.15.16:0/1860402225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff998032440 con 0x7ff9a40d3b90 2024-04-25T10:07:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.058+0000 7ff9a8923640 1 -- 172.21.15.16:0/1860402225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998032810 con 0x7ff9a40d3b90 2024-04-25T10:07:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.059+0000 7ff9abb89640 1 -- 172.21.15.16:0/1860402225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a40d3b90 msgr2=0x7ff9a40d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.059+0000 7ff9abb89640 1 --2- 172.21.15.16:0/1860402225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a40d3b90 0x7ff9a40d0a20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff998009870 tx=0x7ff99802a210 comp rx=0 tx=0).stop 2024-04-25T10:07:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.059+0000 7ff9abb89640 1 -- 172.21.15.16:0/1860402225 shutdown_connections 2024-04-25T10:07:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.059+0000 7ff9abb89640 1 --2- 172.21.15.16:0/1860402225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a40d3b90 0x7ff9a40d0a20 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.059+0000 7ff9abb89640 1 --2- 172.21.15.16:0/1860402225 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9a415dca0 0x7ff9a40ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.059+0000 7ff9abb89640 1 -- 172.21.15.16:0/1860402225 >> 172.21.15.16:0/1860402225 conn(0x7ff9a40726c0 msgr2=0x7ff9a4072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.059+0000 7ff9abb89640 1 -- 172.21.15.16:0/1860402225 shutdown_connections 2024-04-25T10:07:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.059+0000 7ff9abb89640 1 -- 172.21.15.16:0/1860402225 wait complete. 2024-04-25T10:07:52.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.060+0000 7ff9abb89640 1 Processor -- start 2024-04-25T10:07:52.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.060+0000 7ff9abb89640 1 -- start start 2024-04-25T10:07:52.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.061+0000 7ff9abb89640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9a40cee80 0x7ff9a40cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.061+0000 7ff9abb89640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9a40d3b90 0x7ff9a4152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.061+0000 7ff9abb89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a415dca0 0x7ff9a4153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.061+0000 7ff9abb89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a415e200 con 0x7ff9a415dca0 2024-04-25T10:07:52.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.061+0000 7ff9abb89640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a415e080 con 0x7ff9a40cee80 2024-04-25T10:07:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.061+0000 7ff9abb89640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a40d0f60 con 0x7ff9a40d3b90 2024-04-25T10:07:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.061+0000 7ff9aa126640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a415dca0 0x7ff9a4153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.061+0000 7ff9aa126640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a415dca0 0x7ff9a4153c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60358/0 (socket says 172.21.15.16:60358) 2024-04-25T10:07:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.061+0000 7ff9aa126640 1 -- 172.21.15.16:0/2886742969 learned_addr learned my addr 172.21.15.16:0/2886742969 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.061+0000 7ff9a9124640 1 --2- 172.21.15.16:0/2886742969 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9a40d3b90 0x7ff9a4152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.061+0000 7ff9a9925640 1 --2- 172.21.15.16:0/2886742969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9a40cee80 0x7ff9a40cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.061+0000 7ff9aa126640 1 -- 172.21.15.16:0/2886742969 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9a40d3b90 msgr2=0x7ff9a4152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.061+0000 7ff9aa126640 1 --2- 172.21.15.16:0/2886742969 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9a40d3b90 0x7ff9a4152b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.061+0000 7ff9aa126640 1 -- 172.21.15.16:0/2886742969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9a40cee80 msgr2=0x7ff9a40cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.061+0000 7ff9aa126640 1 --2- 172.21.15.16:0/2886742969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9a40cee80 0x7ff9a40cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.061+0000 7ff9aa126640 1 -- 172.21.15.16:0/2886742969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a40cce40 con 0x7ff9a415dca0 2024-04-25T10:07:52.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.061+0000 7ff9aa126640 1 --2- 172.21.15.16:0/2886742969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a415dca0 0x7ff9a4153c20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff99400c780 tx=0x7ff99400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.061+0000 7ff9a27fc640 1 -- 172.21.15.16:0/2886742969 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994004b60 con 0x7ff9a415dca0 2024-04-25T10:07:52.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.062+0000 7ff9a27fc640 1 -- 172.21.15.16:0/2886742969 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff994004d00 con 0x7ff9a415dca0 2024-04-25T10:07:52.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.062+0000 7ff9abb89640 1 -- 172.21.15.16:0/2886742969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a41530a0 con 0x7ff9a415dca0 2024-04-25T10:07:52.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.062+0000 7ff9a27fc640 1 -- 172.21.15.16:0/2886742969 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99400aa90 con 0x7ff9a415dca0 2024-04-25T10:07:52.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.062+0000 7ff9abb89640 1 -- 172.21.15.16:0/2886742969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a41556a0 con 0x7ff9a415dca0 2024-04-25T10:07:52.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.062+0000 7ff9abb89640 1 -- 172.21.15.16:0/2886742969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a41f7560 con 0x7ff9a415dca0 2024-04-25T10:07:52.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.063+0000 7ff9a27fc640 1 -- 172.21.15.16:0/2886742969 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff99400ac30 con 0x7ff9a415dca0 2024-04-25T10:07:52.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.063+0000 7ff9a27fc640 1 --2- 172.21.15.16:0/2886742969 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff99006b4d0 0x7ff99006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.064+0000 7ff9a27fc640 1 -- 172.21.15.16:0/2886742969 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff99408a6c0 con 0x7ff9a415dca0 2024-04-25T10:07:52.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.064+0000 7ff9a9925640 1 --2- 172.21.15.16:0/2886742969 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff99006b4d0 0x7ff99006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.064+0000 7ff9a9925640 1 --2- 172.21.15.16:0/2886742969 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff99006b4d0 0x7ff99006d990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff99c006fd0 tx=0x7ff99c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.068+0000 7ff9a27fc640 1 -- 172.21.15.16:0/2886742969 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff9a40cce40 con 0x7ff9a415dca0 2024-04-25T10:07:52.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.284+0000 7ff9abb89640 1 -- 172.21.15.16:0/2886742969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff9a40750b0 con 0x7ff9a415dca0 2024-04-25T10:07:52.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.316+0000 7ff9a27fc640 1 -- 172.21.15.16:0/2886742969 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff9940595b0 con 0x7ff9a415dca0 2024-04-25T10:07:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.318+0000 7ff9abb89640 1 -- 172.21.15.16:0/2886742969 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff99006b4d0 msgr2=0x7ff99006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.318+0000 7ff9abb89640 1 --2- 172.21.15.16:0/2886742969 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff99006b4d0 0x7ff99006d990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff99c006fd0 tx=0x7ff99c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.318+0000 7ff9abb89640 1 -- 172.21.15.16:0/2886742969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a415dca0 msgr2=0x7ff9a4153c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.318+0000 7ff9abb89640 1 --2- 172.21.15.16:0/2886742969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a415dca0 0x7ff9a4153c20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff99400c780 tx=0x7ff99400cc40 comp rx=0 tx=0).stop 2024-04-25T10:07:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.318+0000 7ff9abb89640 1 -- 172.21.15.16:0/2886742969 shutdown_connections 2024-04-25T10:07:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.318+0000 7ff9abb89640 1 --2- 172.21.15.16:0/2886742969 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff99006b4d0 0x7ff99006d990 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.318+0000 7ff9abb89640 1 --2- 172.21.15.16:0/2886742969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a415dca0 0x7ff9a4153c20 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.318+0000 7ff9abb89640 1 --2- 172.21.15.16:0/2886742969 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9a40d3b90 0x7ff9a4152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.318+0000 7ff9abb89640 1 --2- 172.21.15.16:0/2886742969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9a40cee80 0x7ff9a40cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.318+0000 7ff9abb89640 1 -- 172.21.15.16:0/2886742969 >> 172.21.15.16:0/2886742969 conn(0x7ff9a40726c0 msgr2=0x7ff9a415a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.318+0000 7ff9abb89640 1 -- 172.21.15.16:0/2886742969 shutdown_connections 2024-04-25T10:07:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.318+0000 7ff9abb89640 1 -- 172.21.15.16:0/2886742969 wait complete. 2024-04-25T10:07:52.335 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:07:52.335 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:07:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.433+0000 7ff21f01f640 1 Processor -- start 2024-04-25T10:07:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.433+0000 7ff21f01f640 1 -- start start 2024-04-25T10:07:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.434+0000 7ff21f01f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff21815dc80 0x7ff218161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.434+0000 7ff21f01f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff2180b5c00 0x7ff2180d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.434+0000 7ff21f01f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2181627a0 con 0x7ff2180b97c0 2024-04-25T10:07:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.434+0000 7ff21f01f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2181621e0 con 0x7ff21815dc80 2024-04-25T10:07:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.434+0000 7ff21f01f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2181624c0 con 0x7ff2180b5c00 2024-04-25T10:07:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.434+0000 7ff21cdbb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff21815dc80 0x7ff218161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.434+0000 7ff217fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff2180b97c0 0x7ff2180b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57160/0 (socket says 172.21.15.16:57160) 2024-04-25T10:07:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.434+0000 7ff21cdbb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff21815dc80 0x7ff218161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60360/0 (socket says 172.21.15.16:60360) 2024-04-25T10:07:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.434+0000 7ff217fff640 1 -- 172.21.15.16:0/383706998 learned_addr learned my addr 172.21.15.16:0/383706998 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.434+0000 7ff21d5bc640 1 --2- 172.21.15.16:0/383706998 >> v2:172.21.15.186:3300/0 conn(0x7ff2180b5c00 0x7ff2180d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.434+0000 7ff21cdbb640 1 -- 172.21.15.16:0/383706998 >> v2:172.21.15.186:3300/0 conn(0x7ff2180b5c00 msgr2=0x7ff2180d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.434+0000 7ff21cdbb640 1 --2- 172.21.15.16:0/383706998 >> v2:172.21.15.186:3300/0 conn(0x7ff2180b5c00 0x7ff2180d2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.434+0000 7ff2177fe640 1 -- 172.21.15.16:0/383706998 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440551685 0 0) 0x7ff2181627a0 con 0x7ff2180b97c0 2024-04-25T10:07:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.434+0000 7ff21cdbb640 1 -- 172.21.15.16:0/383706998 >> v1:172.21.15.16:6789/0 conn(0x7ff2180b97c0 legacy=0x7ff2180b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.434+0000 7ff21cdbb640 1 -- 172.21.15.16:0/383706998 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2180d1c70 con 0x7ff21815dc80 2024-04-25T10:07:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.435+0000 7ff21cdbb640 1 --2- 172.21.15.16:0/383706998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21815dc80 0x7ff218161870 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff2180b6300 tx=0x7ff20c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8b53dd4c3157ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.435+0000 7ff2177fe640 1 -- 172.21.15.16:0/383706998 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c037070 con 0x7ff21815dc80 2024-04-25T10:07:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.435+0000 7ff2177fe640 1 -- 172.21.15.16:0/383706998 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20c032440 con 0x7ff21815dc80 2024-04-25T10:07:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.435+0000 7ff2177fe640 1 -- 172.21.15.16:0/383706998 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c032810 con 0x7ff21815dc80 2024-04-25T10:07:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.436+0000 7ff21f01f640 1 -- 172.21.15.16:0/383706998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21815dc80 msgr2=0x7ff218161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.436+0000 7ff21f01f640 1 --2- 172.21.15.16:0/383706998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21815dc80 0x7ff218161870 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff2180b6300 tx=0x7ff20c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.436+0000 7ff21f01f640 1 -- 172.21.15.16:0/383706998 shutdown_connections 2024-04-25T10:07:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.436+0000 7ff21f01f640 1 --2- 172.21.15.16:0/383706998 >> v2:172.21.15.186:3300/0 conn(0x7ff2180b5c00 0x7ff2180d2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.436+0000 7ff21f01f640 1 --2- 172.21.15.16:0/383706998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21815dc80 0x7ff218161870 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.436+0000 7ff21f01f640 1 -- 172.21.15.16:0/383706998 >> 172.21.15.16:0/383706998 conn(0x7ff2180d04f0 msgr2=0x7ff2180d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.436+0000 7ff21f01f640 1 -- 172.21.15.16:0/383706998 shutdown_connections 2024-04-25T10:07:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.436+0000 7ff21f01f640 1 -- 172.21.15.16:0/383706998 wait complete. 2024-04-25T10:07:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.437+0000 7ff21f01f640 1 Processor -- start 2024-04-25T10:07:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.437+0000 7ff21f01f640 1 -- start start 2024-04-25T10:07:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff21f01f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2180b5c00 0x7ff218156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff21f01f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2180b97c0 0x7ff2180ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff21f01f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff21815dc80 0x7ff2180cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff21f01f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff218161f30 con 0x7ff2180b5c00 2024-04-25T10:07:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff21f01f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff218161db0 con 0x7ff21815dc80 2024-04-25T10:07:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff21f01f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180d29b0 con 0x7ff2180b97c0 2024-04-25T10:07:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff21cdbb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2180b5c00 0x7ff218156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff21cdbb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2180b5c00 0x7ff218156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60362/0 (socket says 172.21.15.16:60362) 2024-04-25T10:07:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff21cdbb640 1 -- 172.21.15.16:0/2306939376 learned_addr learned my addr 172.21.15.16:0/2306939376 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff21d5bc640 1 --2- 172.21.15.16:0/2306939376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff21815dc80 0x7ff2180cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff217fff640 1 --2- 172.21.15.16:0/2306939376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2180b97c0 0x7ff2180ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff21cdbb640 1 -- 172.21.15.16:0/2306939376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2180b97c0 msgr2=0x7ff2180ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff21cdbb640 1 --2- 172.21.15.16:0/2306939376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2180b97c0 0x7ff2180ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff21cdbb640 1 -- 172.21.15.16:0/2306939376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff21815dc80 msgr2=0x7ff2180cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff21cdbb640 1 --2- 172.21.15.16:0/2306939376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff21815dc80 0x7ff2180cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff21cdbb640 1 -- 172.21.15.16:0/2306939376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2180c9210 con 0x7ff2180b5c00 2024-04-25T10:07:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff21d5bc640 1 --2- 172.21.15.16:0/2306939376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff21815dc80 0x7ff2180cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff217fff640 1 --2- 172.21.15.16:0/2306939376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2180b97c0 0x7ff2180ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.438+0000 7ff21cdbb640 1 --2- 172.21.15.16:0/2306939376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2180b5c00 0x7ff218156af0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff20c000c00 tx=0x7ff20c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.439+0000 7ff2157fa640 1 -- 172.21.15.16:0/2306939376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c037050 con 0x7ff2180b5c00 2024-04-25T10:07:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.439+0000 7ff2157fa640 1 -- 172.21.15.16:0/2306939376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20c038040 con 0x7ff2180b5c00 2024-04-25T10:07:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.439+0000 7ff2157fa640 1 -- 172.21.15.16:0/2306939376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c004450 con 0x7ff2180b5c00 2024-04-25T10:07:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.439+0000 7ff21f01f640 1 -- 172.21.15.16:0/2306939376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2180c94f0 con 0x7ff2180b5c00 2024-04-25T10:07:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.439+0000 7ff21f01f640 1 -- 172.21.15.16:0/2306939376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2180c9ab0 con 0x7ff2180b5c00 2024-04-25T10:07:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.440+0000 7ff21f01f640 1 -- 172.21.15.16:0/2306939376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2180750b0 con 0x7ff2180b5c00 2024-04-25T10:07:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.440+0000 7ff2157fa640 1 -- 172.21.15.16:0/2306939376 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff20c030980 con 0x7ff2180b5c00 2024-04-25T10:07:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.441+0000 7ff2157fa640 1 --2- 172.21.15.16:0/2306939376 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff1fc06b500 0x7ff1fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.441+0000 7ff2157fa640 1 -- 172.21.15.16:0/2306939376 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff20c0aae20 con 0x7ff2180b5c00 2024-04-25T10:07:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.441+0000 7ff217fff640 1 --2- 172.21.15.16:0/2306939376 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff1fc06b500 0x7ff1fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.441+0000 7ff217fff640 1 --2- 172.21.15.16:0/2306939376 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff1fc06b500 0x7ff1fc06d9c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff2180b7130 tx=0x7ff204009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.445+0000 7ff2157fa640 1 -- 172.21.15.16:0/2306939376 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff20c079d40 con 0x7ff2180b5c00 2024-04-25T10:07:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:52.662+0000 7ff21f01f640 1 -- 172.21.15.16:0/2306939376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff2180d1780 con 0x7ff2180b5c00 2024-04-25T10:07:53.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.318+0000 7ff2157fa640 1 -- 172.21.15.16:0/2306939376 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff20c079640 con 0x7ff2180b5c00 2024-04-25T10:07:53.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.320+0000 7ff21f01f640 1 -- 172.21.15.16:0/2306939376 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff1fc06b500 msgr2=0x7ff1fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.320+0000 7ff21f01f640 1 --2- 172.21.15.16:0/2306939376 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff1fc06b500 0x7ff1fc06d9c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff2180b7130 tx=0x7ff204009040 comp rx=0 tx=0).stop 2024-04-25T10:07:53.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.320+0000 7ff21f01f640 1 -- 172.21.15.16:0/2306939376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2180b5c00 msgr2=0x7ff218156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.320+0000 7ff21f01f640 1 --2- 172.21.15.16:0/2306939376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2180b5c00 0x7ff218156af0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff20c000c00 tx=0x7ff20c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.320+0000 7ff21f01f640 1 -- 172.21.15.16:0/2306939376 shutdown_connections 2024-04-25T10:07:53.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.320+0000 7ff21f01f640 1 --2- 172.21.15.16:0/2306939376 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff1fc06b500 0x7ff1fc06d9c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.320+0000 7ff21f01f640 1 --2- 172.21.15.16:0/2306939376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff21815dc80 0x7ff2180cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.320+0000 7ff21f01f640 1 --2- 172.21.15.16:0/2306939376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2180b97c0 0x7ff2180ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.320+0000 7ff21f01f640 1 --2- 172.21.15.16:0/2306939376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2180b5c00 0x7ff218156af0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.320+0000 7ff21f01f640 1 -- 172.21.15.16:0/2306939376 >> 172.21.15.16:0/2306939376 conn(0x7ff2180d04f0 msgr2=0x7ff2180b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:53.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.320+0000 7ff21f01f640 1 -- 172.21.15.16:0/2306939376 shutdown_connections 2024-04-25T10:07:53.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.320+0000 7ff21f01f640 1 -- 172.21.15.16:0/2306939376 wait complete. 2024-04-25T10:07:53.339 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:07:53.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:53.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.438+0000 7f706763f640 1 Processor -- start 2024-04-25T10:07:53.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.438+0000 7f706763f640 1 -- start start 2024-04-25T10:07:53.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.439+0000 7f706763f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f706015bf00 0x7f70600cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.439+0000 7f706763f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70600cc310 0x7f706015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.439+0000 7f706763f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f706015c820 0x7f706006d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.439+0000 7f706763f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70601455e0 con 0x7f706015c820 2024-04-25T10:07:53.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.439+0000 7f706763f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f706006d450 con 0x7f706015bf00 2024-04-25T10:07:53.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.439+0000 7f706763f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70600ce6c0 con 0x7f70600cc310 2024-04-25T10:07:53.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.439+0000 7f7065bdc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f706015c820 0x7f706006d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.439+0000 7f7065bdc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f706015c820 0x7f706006d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53074/0 (socket says 172.21.15.16:53074) 2024-04-25T10:07:53.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.439+0000 7f7065bdc640 1 -- 172.21.15.16:0/1269614089 learned_addr learned my addr 172.21.15.16:0/1269614089 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:53.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.439+0000 7f7065bdc640 1 -- 172.21.15.16:0/1269614089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70600cc310 msgr2=0x7f706015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:53.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.439+0000 7f7065bdc640 1 --2- 172.21.15.16:0/1269614089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70600cc310 0x7f706015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.439+0000 7f7065bdc640 1 -- 172.21.15.16:0/1269614089 >> v2:172.21.15.186:3300/0 conn(0x7f706015bf00 msgr2=0x7f70600cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:53.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.439+0000 7f70653db640 1 --2- 172.21.15.16:0/1269614089 >> v2:172.21.15.186:3300/0 conn(0x7f706015bf00 0x7f70600cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.439+0000 7f7065bdc640 1 --2- 172.21.15.16:0/1269614089 >> v2:172.21.15.186:3300/0 conn(0x7f706015bf00 0x7f70600cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.440+0000 7f7065bdc640 1 -- 172.21.15.16:0/1269614089 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70600cc7b0 con 0x7f706015c820 2024-04-25T10:07:53.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.440+0000 7f7065bdc640 1 --2- 172.21.15.16:0/1269614089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f706015c820 0x7f706006d8e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f706006ba10 tx=0x7f705c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab1c0d8cd13445c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.440+0000 7f7057fff640 1 -- 172.21.15.16:0/1269614089 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c036070 con 0x7f706015c820 2024-04-25T10:07:53.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.440+0000 7f7057fff640 1 -- 172.21.15.16:0/1269614089 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c031600 con 0x7f706015c820 2024-04-25T10:07:53.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.440+0000 7f7057fff640 1 -- 172.21.15.16:0/1269614089 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c0319d0 con 0x7f706015c820 2024-04-25T10:07:53.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.441+0000 7f706763f640 1 -- 172.21.15.16:0/1269614089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f706015c820 msgr2=0x7f706006d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.441+0000 7f706763f640 1 --2- 172.21.15.16:0/1269614089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f706015c820 0x7f706006d8e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f706006ba10 tx=0x7f705c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.441+0000 7f706763f640 1 -- 172.21.15.16:0/1269614089 shutdown_connections 2024-04-25T10:07:53.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.441+0000 7f706763f640 1 --2- 172.21.15.16:0/1269614089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f706015c820 0x7f706006d8e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.441+0000 7f706763f640 1 --2- 172.21.15.16:0/1269614089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70600cc310 0x7f706015c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.441+0000 7f706763f640 1 --2- 172.21.15.16:0/1269614089 >> v2:172.21.15.186:3300/0 conn(0x7f706015bf00 0x7f70600cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.441+0000 7f706763f640 1 -- 172.21.15.16:0/1269614089 >> 172.21.15.16:0/1269614089 conn(0x7f706006aba0 msgr2=0x7f706006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:53.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.441+0000 7f706763f640 1 -- 172.21.15.16:0/1269614089 shutdown_connections 2024-04-25T10:07:53.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.441+0000 7f706763f640 1 -- 172.21.15.16:0/1269614089 wait complete. 2024-04-25T10:07:53.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.442+0000 7f706763f640 1 Processor -- start 2024-04-25T10:07:53.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.442+0000 7f706763f640 1 -- start start 2024-04-25T10:07:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.443+0000 7f706763f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70600cc310 0x7f70601f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.443+0000 7f706763f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f706015bf00 0x7f70601e4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.443+0000 7f706763f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f706015c820 0x7f7060157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.443+0000 7f706763f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70600cd050 con 0x7f706015c820 2024-04-25T10:07:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.443+0000 7f706763f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70600ccd70 con 0x7f706015bf00 2024-04-25T10:07:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.443+0000 7f706763f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706015cca0 con 0x7f70600cc310 2024-04-25T10:07:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.443+0000 7f7065bdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f706015c820 0x7f7060157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.443+0000 7f7065bdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f706015c820 0x7f7060157d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53076/0 (socket says 172.21.15.16:53076) 2024-04-25T10:07:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.443+0000 7f7065bdc640 1 -- 172.21.15.16:0/2633127140 learned_addr learned my addr 172.21.15.16:0/2633127140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.443+0000 7f7065bdc640 1 -- 172.21.15.16:0/2633127140 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70600cc310 msgr2=0x7f70601f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.443+0000 7f70653db640 1 --2- 172.21.15.16:0/2633127140 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70600cc310 0x7f70601f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.443+0000 7f7064bda640 1 --2- 172.21.15.16:0/2633127140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f706015bf00 0x7f70601e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.443+0000 7f7065bdc640 1 --2- 172.21.15.16:0/2633127140 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70600cc310 0x7f70601f2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.443+0000 7f7065bdc640 1 -- 172.21.15.16:0/2633127140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f706015bf00 msgr2=0x7f70601e4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.443+0000 7f7065bdc640 1 --2- 172.21.15.16:0/2633127140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f706015bf00 0x7f70601e4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.443+0000 7f7065bdc640 1 -- 172.21.15.16:0/2633127140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70601e4970 con 0x7f706015c820 2024-04-25T10:07:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.443+0000 7f7064bda640 1 --2- 172.21.15.16:0/2633127140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f706015bf00 0x7f70601e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.444+0000 7f7065bdc640 1 --2- 172.21.15.16:0/2633127140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f706015c820 0x7f7060157d20 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f705c009f40 tx=0x7f705c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.444+0000 7f7055ffb640 1 -- 172.21.15.16:0/2633127140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c036050 con 0x7f706015c820 2024-04-25T10:07:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.444+0000 7f7055ffb640 1 -- 172.21.15.16:0/2633127140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c042070 con 0x7f706015c820 2024-04-25T10:07:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.444+0000 7f706763f640 1 -- 172.21.15.16:0/2633127140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70601e4c50 con 0x7f706015c820 2024-04-25T10:07:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.444+0000 7f7055ffb640 1 -- 172.21.15.16:0/2633127140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c009510 con 0x7f706015c820 2024-04-25T10:07:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.444+0000 7f706763f640 1 -- 172.21.15.16:0/2633127140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70601e5210 con 0x7f706015c820 2024-04-25T10:07:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.445+0000 7f706763f640 1 -- 172.21.15.16:0/2633127140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f706006bb30 con 0x7f706015c820 2024-04-25T10:07:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.445+0000 7f7055ffb640 1 -- 172.21.15.16:0/2633127140 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f705c007bd0 con 0x7f706015c820 2024-04-25T10:07:53.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.446+0000 7f7055ffb640 1 --2- 172.21.15.16:0/2633127140 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f704406b4d0 0x7f704406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.446+0000 7f7055ffb640 1 -- 172.21.15.16:0/2633127140 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f705c0ab2a0 con 0x7f706015c820 2024-04-25T10:07:53.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.446+0000 7f70653db640 1 --2- 172.21.15.16:0/2633127140 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f704406b4d0 0x7f704406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.447+0000 7f70653db640 1 --2- 172.21.15.16:0/2633127140 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f704406b4d0 0x7f704406d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7058006fd0 tx=0x7f7058008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.450+0000 7f7055ffb640 1 -- 172.21.15.16:0/2633127140 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f705c07a080 con 0x7f706015c820 2024-04-25T10:07:53.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.666+0000 7f706763f640 1 -- 172.21.15.16:0/2633127140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70600749b0 con 0x7f706015c820 2024-04-25T10:07:53.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.667+0000 7f7055ffb640 1 -- 172.21.15.16:0/2633127140 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f705c079980 con 0x7f706015c820 2024-04-25T10:07:53.673 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:53.673 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2270839865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2270839865},{"type":"v1","addr":"172.21.15.186:6836","nonce":2270839865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5647,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/925282579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":925282579},{"type":"v1","addr":"172.21.15.186:6837","nonce":925282579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2471714328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2471714328},{"type":"v1","addr":"172.21.15.16:6835","nonce":2471714328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:50.665436+0000","modified":"2024-04-25T10:07:52.663229+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35141":{"gid":35141,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6834/3311584521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3311584521},{"type":"v1","addr":"172.21.15.16:6834","nonce":3311584521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:07:53.673 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-04-25T10:07:53.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.669+0000 7f706763f640 1 -- 172.21.15.16:0/2633127140 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f704406b4d0 msgr2=0x7f704406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.669+0000 7f706763f640 1 --2- 172.21.15.16:0/2633127140 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f704406b4d0 0x7f704406d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7058006fd0 tx=0x7f7058008040 comp rx=0 tx=0).stop 2024-04-25T10:07:53.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.669+0000 7f706763f640 1 -- 172.21.15.16:0/2633127140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f706015c820 msgr2=0x7f7060157d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.669+0000 7f706763f640 1 --2- 172.21.15.16:0/2633127140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f706015c820 0x7f7060157d20 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f705c009f40 tx=0x7f705c004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:53.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.669+0000 7f706763f640 1 -- 172.21.15.16:0/2633127140 shutdown_connections 2024-04-25T10:07:53.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.669+0000 7f706763f640 1 --2- 172.21.15.16:0/2633127140 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f704406b4d0 0x7f704406d990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.669+0000 7f706763f640 1 --2- 172.21.15.16:0/2633127140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f706015c820 0x7f7060157d20 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.669+0000 7f706763f640 1 --2- 172.21.15.16:0/2633127140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f706015bf00 0x7f70601e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.669+0000 7f706763f640 1 --2- 172.21.15.16:0/2633127140 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f70600cc310 0x7f70601f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.669+0000 7f706763f640 1 -- 172.21.15.16:0/2633127140 >> 172.21.15.16:0/2633127140 conn(0x7f706006aba0 msgr2=0x7f706015df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:53.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.669+0000 7f706763f640 1 -- 172.21.15.16:0/2633127140 shutdown_connections 2024-04-25T10:07:53.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.669+0000 7f706763f640 1 -- 172.21.15.16:0/2633127140 wait complete. 2024-04-25T10:07:53.685 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.781+0000 7ff3d8b5c640 1 Processor -- start 2024-04-25T10:07:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.781+0000 7ff3d8b5c640 1 -- start start 2024-04-25T10:07:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.782+0000 7ff3d8b5c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3d40d35b0 0x7ff3d40b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.782+0000 7ff3d8b5c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff3d40b5990 0x7ff3d40b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.782+0000 7ff3d8b5c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d415e0c0 con 0x7ff3d415dce0 2024-04-25T10:07:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.782+0000 7ff3d8b5c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d40b5de0 con 0x7ff3d40b5990 2024-04-25T10:07:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.782+0000 7ff3d8b5c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d40d2510 con 0x7ff3d40d35b0 2024-04-25T10:07:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.782+0000 7ff3d2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff3d40b5990 0x7ff3d40b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.782+0000 7ff3d259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff3d415dce0 0x7ff3d40d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54424/0 (socket says 172.21.15.16:54424) 2024-04-25T10:07:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.782+0000 7ff3d259c640 1 -- 172.21.15.16:0/708673442 learned_addr learned my addr 172.21.15.16:0/708673442 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.782+0000 7ff3d2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff3d40b5990 0x7ff3d40b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53078/0 (socket says 172.21.15.16:53078) 2024-04-25T10:07:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.782+0000 7ff3d1d9b640 1 --2- 172.21.15.16:0/708673442 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3d40d35b0 0x7ff3d40b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.782+0000 7ff3d2d9d640 1 -- 172.21.15.16:0/708673442 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3d40d35b0 msgr2=0x7ff3d40b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.782+0000 7ff3d2d9d640 1 --2- 172.21.15.16:0/708673442 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3d40d35b0 0x7ff3d40b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.782+0000 7ff3d2d9d640 1 -- 172.21.15.16:0/708673442 >> v1:172.21.15.16:6789/0 conn(0x7ff3d415dce0 legacy=0x7ff3d40d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.782+0000 7ff3d2d9d640 1 -- 172.21.15.16:0/708673442 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d40d27f0 con 0x7ff3d40b5990 2024-04-25T10:07:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.783+0000 7ff3d2d9d640 1 --2- 172.21.15.16:0/708673442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d40b5990 0x7ff3d40b6b00 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40097a0 tx=0x7ff3c402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89f46089ee4484b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.783+0000 7ff3d159a640 1 -- 172.21.15.16:0/708673442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4037070 con 0x7ff3d40b5990 2024-04-25T10:07:53.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.783+0000 7ff3d159a640 1 -- 172.21.15.16:0/708673442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c4032440 con 0x7ff3d40b5990 2024-04-25T10:07:53.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.783+0000 7ff3d159a640 1 -- 172.21.15.16:0/708673442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4032810 con 0x7ff3d40b5990 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.784+0000 7ff3d8b5c640 1 -- 172.21.15.16:0/708673442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d40b5990 msgr2=0x7ff3d40b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.784+0000 7ff3d8b5c640 1 --2- 172.21.15.16:0/708673442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d40b5990 0x7ff3d40b6b00 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40097a0 tx=0x7ff3c402a180 comp rx=0 tx=0).stop 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.784+0000 7ff3d8b5c640 1 -- 172.21.15.16:0/708673442 shutdown_connections 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.784+0000 7ff3d8b5c640 1 --2- 172.21.15.16:0/708673442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d40b5990 0x7ff3d40b6b00 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.784+0000 7ff3d8b5c640 1 --2- 172.21.15.16:0/708673442 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3d40d35b0 0x7ff3d40b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.784+0000 7ff3d8b5c640 1 -- 172.21.15.16:0/708673442 >> 172.21.15.16:0/708673442 conn(0x7ff3d4072900 msgr2=0x7ff3d40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.784+0000 7ff3d8b5c640 1 -- 172.21.15.16:0/708673442 shutdown_connections 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.784+0000 7ff3d8b5c640 1 -- 172.21.15.16:0/708673442 wait complete. 2024-04-25T10:07:53.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.785+0000 7ff3d8b5c640 1 Processor -- start 2024-04-25T10:07:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.785+0000 7ff3d8b5c640 1 -- start start 2024-04-25T10:07:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d8b5c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3d40b5990 0x7ff3d4152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d8b5c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3d40d35b0 0x7ff3d4153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d8b5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d415dce0 0x7ff3d4153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d8b5c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d40d3090 con 0x7ff3d415dce0 2024-04-25T10:07:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d8b5c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d40d2db0 con 0x7ff3d40d35b0 2024-04-25T10:07:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d8b5c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d40d2160 con 0x7ff3d40b5990 2024-04-25T10:07:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d415dce0 0x7ff3d4153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d415dce0 0x7ff3d4153230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53092/0 (socket says 172.21.15.16:53092) 2024-04-25T10:07:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d2d9d640 1 -- 172.21.15.16:0/395643676 learned_addr learned my addr 172.21.15.16:0/395643676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d1d9b640 1 --2- 172.21.15.16:0/395643676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3d40d35b0 0x7ff3d4153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d259c640 1 --2- 172.21.15.16:0/395643676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3d40b5990 0x7ff3d4152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d2d9d640 1 -- 172.21.15.16:0/395643676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3d40b5990 msgr2=0x7ff3d4152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d2d9d640 1 --2- 172.21.15.16:0/395643676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3d40b5990 0x7ff3d4152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d2d9d640 1 -- 172.21.15.16:0/395643676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3d40d35b0 msgr2=0x7ff3d4153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d2d9d640 1 --2- 172.21.15.16:0/395643676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3d40d35b0 0x7ff3d4153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d2d9d640 1 -- 172.21.15.16:0/395643676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d40c8950 con 0x7ff3d415dce0 2024-04-25T10:07:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d1d9b640 1 --2- 172.21.15.16:0/395643676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3d40d35b0 0x7ff3d4153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d259c640 1 --2- 172.21.15.16:0/395643676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3d40b5990 0x7ff3d4152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3d2d9d640 1 --2- 172.21.15.16:0/395643676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d415dce0 0x7ff3d4153230 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4035040 tx=0x7ff3c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.786+0000 7ff3caffd640 1 -- 172.21.15.16:0/395643676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4037040 con 0x7ff3d415dce0 2024-04-25T10:07:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.787+0000 7ff3caffd640 1 -- 172.21.15.16:0/395643676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c4042070 con 0x7ff3d415dce0 2024-04-25T10:07:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.787+0000 7ff3d8b5c640 1 -- 172.21.15.16:0/395643676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d40c8bf0 con 0x7ff3d415dce0 2024-04-25T10:07:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.787+0000 7ff3caffd640 1 -- 172.21.15.16:0/395643676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4030b50 con 0x7ff3d415dce0 2024-04-25T10:07:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.787+0000 7ff3d8b5c640 1 -- 172.21.15.16:0/395643676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d40c91b0 con 0x7ff3d415dce0 2024-04-25T10:07:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.787+0000 7ff3d8b5c640 1 -- 172.21.15.16:0/395643676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d40cf5f0 con 0x7ff3d415dce0 2024-04-25T10:07:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.788+0000 7ff3caffd640 1 -- 172.21.15.16:0/395643676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3c4030cf0 con 0x7ff3d415dce0 2024-04-25T10:07:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.789+0000 7ff3caffd640 1 --2- 172.21.15.16:0/395643676 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff3b406b4d0 0x7ff3b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.789+0000 7ff3caffd640 1 -- 172.21.15.16:0/395643676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3c40aad00 con 0x7ff3d415dce0 2024-04-25T10:07:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.789+0000 7ff3d259c640 1 --2- 172.21.15.16:0/395643676 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff3b406b4d0 0x7ff3b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.789+0000 7ff3d259c640 1 --2- 172.21.15.16:0/395643676 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff3b406b4d0 0x7ff3b406d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc0012e0 tx=0x7ff3cc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.793+0000 7ff3caffd640 1 -- 172.21.15.16:0/395643676 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3d40cf5f0 con 0x7ff3d415dce0 2024-04-25T10:07:53.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.978+0000 7ff3d8b5c640 1 -- 172.21.15.16:0/395643676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3d40b9390 con 0x7ff3d415dce0 2024-04-25T10:07:53.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.979+0000 7ff3caffd640 1 -- 172.21.15.16:0/395643676 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16665 (secure 0 0 0) 0x7ff3c4032e20 con 0x7ff3d415dce0 2024-04-25T10:07:53.985 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:53.985 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":59,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:07:52.291250+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:48.948980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:49.638457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:53.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.981+0000 7ff3d8b5c640 1 -- 172.21.15.16:0/395643676 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff3b406b4d0 msgr2=0x7ff3b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.981+0000 7ff3d8b5c640 1 --2- 172.21.15.16:0/395643676 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff3b406b4d0 0x7ff3b406d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc0012e0 tx=0x7ff3cc009040 comp rx=0 tx=0).stop 2024-04-25T10:07:53.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.981+0000 7ff3d8b5c640 1 -- 172.21.15.16:0/395643676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d415dce0 msgr2=0x7ff3d4153230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.981+0000 7ff3d8b5c640 1 --2- 172.21.15.16:0/395643676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d415dce0 0x7ff3d4153230 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4035040 tx=0x7ff3c40046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.981+0000 7ff3d8b5c640 1 -- 172.21.15.16:0/395643676 shutdown_connections 2024-04-25T10:07:53.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.981+0000 7ff3d8b5c640 1 --2- 172.21.15.16:0/395643676 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff3b406b4d0 0x7ff3b406d990 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.981+0000 7ff3d8b5c640 1 --2- 172.21.15.16:0/395643676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d415dce0 0x7ff3d4153230 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.981+0000 7ff3d8b5c640 1 --2- 172.21.15.16:0/395643676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3d40d35b0 0x7ff3d4153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.981+0000 7ff3d8b5c640 1 --2- 172.21.15.16:0/395643676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3d40b5990 0x7ff3d4152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.981+0000 7ff3d8b5c640 1 -- 172.21.15.16:0/395643676 >> 172.21.15.16:0/395643676 conn(0x7ff3d4072900 msgr2=0x7ff3d40b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:53.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.981+0000 7ff3d8b5c640 1 -- 172.21.15.16:0/395643676 shutdown_connections 2024-04-25T10:07:53.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:53.981+0000 7ff3d8b5c640 1 -- 172.21.15.16:0/395643676 wait complete. 2024-04-25T10:07:53.998 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:07:53.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:54.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.093+0000 7f0d433f9640 1 Processor -- start 2024-04-25T10:07:54.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.093+0000 7f0d433f9640 1 -- start start 2024-04-25T10:07:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.094+0000 7f0d433f9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0d3c15dce0 0x7f0d3c0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.094+0000 7f0d433f9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d3c15e0c0 con 0x7f0d3c0d35b0 2024-04-25T10:07:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.094+0000 7f0d433f9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d3c0d2510 con 0x7f0d3c0b5b60 2024-04-25T10:07:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.094+0000 7f0d433f9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d3c0d27f0 con 0x7f0d3c15dce0 2024-04-25T10:07:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.094+0000 7f0d40994640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0d3c0d35b0 0x7f0d3c0b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54430/0 (socket says 172.21.15.16:54430) 2024-04-25T10:07:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.094+0000 7f0d40994640 1 -- 172.21.15.16:0/3033007757 learned_addr learned my addr 172.21.15.16:0/3033007757 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.094+0000 7f0d3bfff640 1 -- 172.21.15.16:0/3033007757 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802519834 0 0) 0x7f0d3c15e0c0 con 0x7f0d3c0d35b0 2024-04-25T10:07:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.094+0000 7f0d3bfff640 1 -- 172.21.15.16:0/3033007757 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d24003520 con 0x7f0d3c0d35b0 2024-04-25T10:07:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.094+0000 7f0d41195640 1 --2- 172.21.15.16:0/3033007757 >> v2:172.21.15.186:3300/0 conn(0x7f0d3c15dce0 0x7f0d3c0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d3bfff640 1 -- 172.21.15.16:0/3033007757 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2245405995 0 0) 0x7f0d24003520 con 0x7f0d3c0d35b0 2024-04-25T10:07:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d3bfff640 1 -- 172.21.15.16:0/3033007757 >> v2:172.21.15.186:3300/0 conn(0x7f0d3c15dce0 msgr2=0x7f0d3c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d3bfff640 1 --2- 172.21.15.16:0/3033007757 >> v2:172.21.15.186:3300/0 conn(0x7f0d3c15dce0 0x7f0d3c0d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d3bfff640 1 -- 172.21.15.16:0/3033007757 >> v1:172.21.15.186:6789/0 conn(0x7f0d3c0b5b60 legacy=0x7f0d3c0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d3bfff640 1 -- 172.21.15.16:0/3033007757 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3c0d2ad0 con 0x7f0d3c0d35b0 2024-04-25T10:07:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d3bfff640 1 -- 172.21.15.16:0/3033007757 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f0d30001670 con 0x7f0d3c0d35b0 2024-04-25T10:07:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d3bfff640 1 -- 172.21.15.16:0/3033007757 >> v1:172.21.15.16:6789/0 conn(0x7f0d3c0d35b0 legacy=0x7f0d3c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d3bfff640 1 --2- 172.21.15.16:0/3033007757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d240039b0 0x7f0d24003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d3bfff640 1 --2- 172.21.15.16:0/3033007757 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d24004800 0x7f0d24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d3bfff640 1 --2- 172.21.15.16:0/3033007757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d240051c0 0x7f0d2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d3bfff640 1 -- 172.21.15.16:0/3033007757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c0d27f0 con 0x7f0d240039b0 2024-04-25T10:07:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d3bfff640 1 -- 172.21.15.16:0/3033007757 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2400c060 con 0x7f0d240051c0 2024-04-25T10:07:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d3bfff640 1 -- 172.21.15.16:0/3033007757 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2400c260 con 0x7f0d24004800 2024-04-25T10:07:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d41195640 1 --2- 172.21.15.16:0/3033007757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d240039b0 0x7f0d24003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d41996640 1 --2- 172.21.15.16:0/3033007757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d240051c0 0x7f0d2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d40994640 1 --2- 172.21.15.16:0/3033007757 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d24004800 0x7f0d24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d41195640 1 -- 172.21.15.16:0/3033007757 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d24004800 msgr2=0x7f0d24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d41195640 1 --2- 172.21.15.16:0/3033007757 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d24004800 0x7f0d24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d41195640 1 -- 172.21.15.16:0/3033007757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d240051c0 msgr2=0x7f0d2400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d41195640 1 --2- 172.21.15.16:0/3033007757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d240051c0 0x7f0d2400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d41195640 1 -- 172.21.15.16:0/3033007757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d24003520 con 0x7f0d240039b0 2024-04-25T10:07:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d41996640 1 --2- 172.21.15.16:0/3033007757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d240051c0 0x7f0d2400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.095+0000 7f0d41996640 1 -- 172.21.15.16:0/3033007757 reap_dead start 2024-04-25T10:07:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.096+0000 7f0d41195640 1 --2- 172.21.15.16:0/3033007757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d240039b0 0x7f0d24003dd0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009fc0 tx=0x7f0d24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715e1e566ebcc509 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.096+0000 7f0d3bfff640 1 -- 172.21.15.16:0/3033007757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34009c10 con 0x7f0d240039b0 2024-04-25T10:07:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.096+0000 7f0d3bfff640 1 -- 172.21.15.16:0/3033007757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d34036070 con 0x7f0d240039b0 2024-04-25T10:07:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.096+0000 7f0d3bfff640 1 -- 172.21.15.16:0/3033007757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34031880 con 0x7f0d240039b0 2024-04-25T10:07:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.096+0000 7f0d433f9640 1 -- 172.21.15.16:0/3033007757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d240039b0 msgr2=0x7f0d24003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.096+0000 7f0d433f9640 1 --2- 172.21.15.16:0/3033007757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d240039b0 0x7f0d24003dd0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009fc0 tx=0x7f0d24005830 comp rx=0 tx=0).stop 2024-04-25T10:07:54.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.097+0000 7f0d433f9640 1 -- 172.21.15.16:0/3033007757 shutdown_connections 2024-04-25T10:07:54.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.097+0000 7f0d433f9640 1 --2- 172.21.15.16:0/3033007757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d240039b0 0x7f0d24003dd0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.097+0000 7f0d433f9640 1 -- 172.21.15.16:0/3033007757 >> 172.21.15.16:0/3033007757 conn(0x7f0d3c072900 msgr2=0x7f0d3c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:54.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.097+0000 7f0d433f9640 1 -- 172.21.15.16:0/3033007757 shutdown_connections 2024-04-25T10:07:54.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.097+0000 7f0d433f9640 1 -- 172.21.15.16:0/3033007757 wait complete. 2024-04-25T10:07:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.098+0000 7f0d433f9640 1 Processor -- start 2024-04-25T10:07:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.098+0000 7f0d433f9640 1 -- start start 2024-04-25T10:07:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.098+0000 7f0d433f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d240039b0 0x7f0d3c160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d433f9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d3c153a40 0x7f0d3c153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d433f9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d3c145e70 0x7f0d3c1462f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d433f9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c0b9820 con 0x7f0d240039b0 2024-04-25T10:07:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d433f9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c0d3090 con 0x7f0d3c153a40 2024-04-25T10:07:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d433f9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c0b99a0 con 0x7f0d3c145e70 2024-04-25T10:07:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d41195640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d240039b0 0x7f0d3c160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d41195640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d240039b0 0x7f0d3c160fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53118/0 (socket says 172.21.15.16:53118) 2024-04-25T10:07:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d41195640 1 -- 172.21.15.16:0/2440317755 learned_addr learned my addr 172.21.15.16:0/2440317755 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d40994640 1 --2- 172.21.15.16:0/2440317755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d3c153a40 0x7f0d3c153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d41996640 1 --2- 172.21.15.16:0/2440317755 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d3c145e70 0x7f0d3c1462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d41195640 1 -- 172.21.15.16:0/2440317755 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d3c145e70 msgr2=0x7f0d3c1462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d41195640 1 --2- 172.21.15.16:0/2440317755 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d3c145e70 0x7f0d3c1462f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d41195640 1 -- 172.21.15.16:0/2440317755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d3c153a40 msgr2=0x7f0d3c153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d41195640 1 --2- 172.21.15.16:0/2440317755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d3c153a40 0x7f0d3c153ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d41195640 1 -- 172.21.15.16:0/2440317755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3c146830 con 0x7f0d240039b0 2024-04-25T10:07:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d41996640 1 --2- 172.21.15.16:0/2440317755 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d3c145e70 0x7f0d3c1462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d41195640 1 --2- 172.21.15.16:0/2440317755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d240039b0 0x7f0d3c160fb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400e230 tx=0x7f0d34009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d39ffb640 1 -- 172.21.15.16:0/2440317755 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34009c10 con 0x7f0d240039b0 2024-04-25T10:07:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d39ffb640 1 -- 172.21.15.16:0/2440317755 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d34003990 con 0x7f0d240039b0 2024-04-25T10:07:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d39ffb640 1 -- 172.21.15.16:0/2440317755 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34003d60 con 0x7f0d240039b0 2024-04-25T10:07:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.099+0000 7f0d433f9640 1 -- 172.21.15.16:0/2440317755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d3c0cd480 con 0x7f0d240039b0 2024-04-25T10:07:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.100+0000 7f0d433f9640 1 -- 172.21.15.16:0/2440317755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d3c0ccb50 con 0x7f0d240039b0 2024-04-25T10:07:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.100+0000 7f0d433f9640 1 -- 172.21.15.16:0/2440317755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3c0749b0 con 0x7f0d240039b0 2024-04-25T10:07:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.101+0000 7f0d39ffb640 1 -- 172.21.15.16:0/2440317755 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d34005d10 con 0x7f0d240039b0 2024-04-25T10:07:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.101+0000 7f0d39ffb640 1 --2- 172.21.15.16:0/2440317755 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0d2806b4d0 0x7f0d2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.101+0000 7f0d39ffb640 1 -- 172.21.15.16:0/2440317755 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d340ab5b0 con 0x7f0d240039b0 2024-04-25T10:07:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.102+0000 7f0d40994640 1 --2- 172.21.15.16:0/2440317755 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0d2806b4d0 0x7f0d2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.102+0000 7f0d40994640 1 --2- 172.21.15.16:0/2440317755 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0d2806b4d0 0x7f0d2806d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0d30001cc0 tx=0x7f0d30007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.105+0000 7f0d39ffb640 1 -- 172.21.15.16:0/2440317755 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d3407a300 con 0x7f0d240039b0 2024-04-25T10:07:54.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.287+0000 7f0d433f9640 1 -- 172.21.15.16:0/2440317755 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d3c0b4280 con 0x7f0d2806b4d0 2024-04-25T10:07:54.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.294+0000 7f0d39ffb640 1 -- 172.21.15.16:0/2440317755 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249962 (secure 0 0 0) 0x7f0d3c0b4280 con 0x7f0d2806b4d0 2024-04-25T10:07:54.302 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:54.304 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:07:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.297+0000 7f0d433f9640 1 -- 172.21.15.16:0/2440317755 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0d2806b4d0 msgr2=0x7f0d2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.297+0000 7f0d433f9640 1 --2- 172.21.15.16:0/2440317755 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0d2806b4d0 0x7f0d2806d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0d30001cc0 tx=0x7f0d30007040 comp rx=0 tx=0).stop 2024-04-25T10:07:54.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.297+0000 7f0d433f9640 1 -- 172.21.15.16:0/2440317755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d240039b0 msgr2=0x7f0d3c160fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.297+0000 7f0d433f9640 1 --2- 172.21.15.16:0/2440317755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d240039b0 0x7f0d3c160fb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400e230 tx=0x7f0d34009750 comp rx=0 tx=0).stop 2024-04-25T10:07:54.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.297+0000 7f0d433f9640 1 -- 172.21.15.16:0/2440317755 shutdown_connections 2024-04-25T10:07:54.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.297+0000 7f0d433f9640 1 --2- 172.21.15.16:0/2440317755 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0d2806b4d0 0x7f0d2806d990 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.297+0000 7f0d433f9640 1 --2- 172.21.15.16:0/2440317755 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d3c145e70 0x7f0d3c1462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.297+0000 7f0d433f9640 1 --2- 172.21.15.16:0/2440317755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d3c153a40 0x7f0d3c153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.297+0000 7f0d433f9640 1 --2- 172.21.15.16:0/2440317755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d240039b0 0x7f0d3c160fb0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.297+0000 7f0d433f9640 1 -- 172.21.15.16:0/2440317755 >> 172.21.15.16:0/2440317755 conn(0x7f0d3c072900 msgr2=0x7f0d3c0b48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:54.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.297+0000 7f0d433f9640 1 -- 172.21.15.16:0/2440317755 shutdown_connections 2024-04-25T10:07:54.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.297+0000 7f0d433f9640 1 -- 172.21.15.16:0/2440317755 wait complete. 2024-04-25T10:07:54.313 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-04-25T10:07:53.818447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":197,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243604,"kb_used_data":6608,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731116,"statfs":{"total":773094113280,"available":772844662784,"internally_reserved":0,"allocated":6766592,"data_stored":3451228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.533513"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.472064+0000","last_change":"2024-04-25T10:07:51.311604+0000","last_active":"2024-04-25T10:07:52.472064+0000","last_peered":"2024-04-25T10:07:52.472064+0000","last_clean":"2024-04-25T10:07:52.472064+0000","last_became_active":"2024-04-25T10:07:51.310395+0000","last_became_peered":"2024-04-25T10:07:51.310395+0000","last_unstale":"2024-04-25T10:07:52.472064+0000","last_undegraded":"2024-04-25T10:07:52.472064+0000","last_fullsized":"2024-04-25T10:07:52.472064+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471748+0000","last_change":"2024-04-25T10:07:51.315131+0000","last_active":"2024-04-25T10:07:52.471748+0000","last_peered":"2024-04-25T10:07:52.471748+0000","last_clean":"2024-04-25T10:07:52.471748+0000","last_became_active":"2024-04-25T10:07:51.314869+0000","last_became_peered":"2024-04-25T10:07:51.314869+0000","last_unstale":"2024-04-25T10:07:52.471748+0000","last_undegraded":"2024-04-25T10:07:52.471748+0000","last_fullsized":"2024-04-25T10:07:52.471748+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:38.125900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470577+0000","last_change":"2024-04-25T10:07:51.303339+0000","last_active":"2024-04-25T10:07:52.470577+0000","last_peered":"2024-04-25T10:07:52.470577+0000","last_clean":"2024-04-25T10:07:52.470577+0000","last_became_active":"2024-04-25T10:07:51.303082+0000","last_became_peered":"2024-04-25T10:07:51.303082+0000","last_unstale":"2024-04-25T10:07:52.470577+0000","last_undegraded":"2024-04-25T10:07:52.470577+0000","last_fullsized":"2024-04-25T10:07:52.470577+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:16.208395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470265+0000","last_change":"2024-04-25T10:07:51.318977+0000","last_active":"2024-04-25T10:07:52.470265+0000","last_peered":"2024-04-25T10:07:52.470265+0000","last_clean":"2024-04-25T10:07:52.470265+0000","last_became_active":"2024-04-25T10:07:51.318704+0000","last_became_peered":"2024-04-25T10:07:51.318704+0000","last_unstale":"2024-04-25T10:07:52.470265+0000","last_undegraded":"2024-04-25T10:07:52.470265+0000","last_fullsized":"2024-04-25T10:07:52.470265+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:01.127872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471710+0000","last_change":"2024-04-25T10:07:51.309203+0000","last_active":"2024-04-25T10:07:52.471710+0000","last_peered":"2024-04-25T10:07:52.471710+0000","last_clean":"2024-04-25T10:07:52.471710+0000","last_became_active":"2024-04-25T10:07:51.308918+0000","last_became_peered":"2024-04-25T10:07:51.308918+0000","last_unstale":"2024-04-25T10:07:52.471710+0000","last_undegraded":"2024-04-25T10:07:52.471710+0000","last_fullsized":"2024-04-25T10:07:52.471710+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470720+0000","last_change":"2024-04-25T10:07:51.317240+0000","last_active":"2024-04-25T10:07:52.470720+0000","last_peered":"2024-04-25T10:07:52.470720+0000","last_clean":"2024-04-25T10:07:52.470720+0000","last_became_active":"2024-04-25T10:07:51.316735+0000","last_became_peered":"2024-04-25T10:07:51.316735+0000","last_unstale":"2024-04-25T10:07:52.470720+0000","last_undegraded":"2024-04-25T10:07:52.470720+0000","last_fullsized":"2024-04-25T10:07:52.470720+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:49.018249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470088+0000","last_change":"2024-04-25T10:07:51.310589+0000","last_active":"2024-04-25T10:07:52.470088+0000","last_peered":"2024-04-25T10:07:52.470088+0000","last_clean":"2024-04-25T10:07:52.470088+0000","last_became_active":"2024-04-25T10:07:51.310211+0000","last_became_peered":"2024-04-25T10:07:51.310211+0000","last_unstale":"2024-04-25T10:07:52.470088+0000","last_undegraded":"2024-04-25T10:07:52.470088+0000","last_fullsized":"2024-04-25T10:07:52.470088+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:16.223783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296172+0000","last_change":"2024-04-25T10:07:51.312254+0000","last_active":"2024-04-25T10:07:52.296172+0000","last_peered":"2024-04-25T10:07:52.296172+0000","last_clean":"2024-04-25T10:07:52.296172+0000","last_became_active":"2024-04-25T10:07:51.311901+0000","last_became_peered":"2024-04-25T10:07:51.311901+0000","last_unstale":"2024-04-25T10:07:52.296172+0000","last_undegraded":"2024-04-25T10:07:52.296172+0000","last_fullsized":"2024-04-25T10:07:52.296172+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:42.400180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470490+0000","last_change":"2024-04-25T10:07:51.317615+0000","last_active":"2024-04-25T10:07:52.470490+0000","last_peered":"2024-04-25T10:07:52.470490+0000","last_clean":"2024-04-25T10:07:52.470490+0000","last_became_active":"2024-04-25T10:07:51.317131+0000","last_became_peered":"2024-04-25T10:07:51.317131+0000","last_unstale":"2024-04-25T10:07:52.470490+0000","last_undegraded":"2024-04-25T10:07:52.470490+0000","last_fullsized":"2024-04-25T10:07:52.470490+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:19.853622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296651+0000","last_change":"2024-04-25T10:07:51.323605+0000","last_active":"2024-04-25T10:07:52.296651+0000","last_peered":"2024-04-25T10:07:52.296651+0000","last_clean":"2024-04-25T10:07:52.296651+0000","last_became_active":"2024-04-25T10:07:51.323306+0000","last_became_peered":"2024-04-25T10:07:51.323306+0000","last_unstale":"2024-04-25T10:07:52.296651+0000","last_undegraded":"2024-04-25T10:07:52.296651+0000","last_fullsized":"2024-04-25T10:07:52.296651+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.168043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296163+0000","last_change":"2024-04-25T10:07:51.312425+0000","last_active":"2024-04-25T10:07:52.296163+0000","last_peered":"2024-04-25T10:07:52.296163+0000","last_clean":"2024-04-25T10:07:52.296163+0000","last_became_active":"2024-04-25T10:07:51.312133+0000","last_became_peered":"2024-04-25T10:07:51.312133+0000","last_unstale":"2024-04-25T10:07:52.296163+0000","last_undegraded":"2024-04-25T10:07:52.296163+0000","last_fullsized":"2024-04-25T10:07:52.296163+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.404296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294383+0000","last_change":"2024-04-25T10:07:51.317166+0000","last_active":"2024-04-25T10:07:52.294383+0000","last_peered":"2024-04-25T10:07:52.294383+0000","last_clean":"2024-04-25T10:07:52.294383+0000","last_became_active":"2024-04-25T10:07:51.315513+0000","last_became_peered":"2024-04-25T10:07:51.315513+0000","last_unstale":"2024-04-25T10:07:52.294383+0000","last_undegraded":"2024-04-25T10:07:52.294383+0000","last_fullsized":"2024-04-25T10:07:52.294383+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:51.296503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470214+0000","last_change":"2024-04-25T10:07:51.313310+0000","last_active":"2024-04-25T10:07:52.470214+0000","last_peered":"2024-04-25T10:07:52.470214+0000","last_clean":"2024-04-25T10:07:52.470214+0000","last_became_active":"2024-04-25T10:07:51.313001+0000","last_became_peered":"2024-04-25T10:07:51.313001+0000","last_unstale":"2024-04-25T10:07:52.470214+0000","last_undegraded":"2024-04-25T10:07:52.470214+0000","last_fullsized":"2024-04-25T10:07:52.470214+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:13.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294437+0000","last_change":"2024-04-25T10:07:51.317685+0000","last_active":"2024-04-25T10:07:52.294437+0000","last_peered":"2024-04-25T10:07:52.294437+0000","last_clean":"2024-04-25T10:07:52.294437+0000","last_became_active":"2024-04-25T10:07:51.317386+0000","last_became_peered":"2024-04-25T10:07:51.317386+0000","last_unstale":"2024-04-25T10:07:52.294437+0000","last_undegraded":"2024-04-25T10:07:52.294437+0000","last_fullsized":"2024-04-25T10:07:52.294437+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:16.208395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470284+0000","last_change":"2024-04-25T10:07:51.306911+0000","last_active":"2024-04-25T10:07:52.470284+0000","last_peered":"2024-04-25T10:07:52.470284+0000","last_clean":"2024-04-25T10:07:52.470284+0000","last_became_active":"2024-04-25T10:07:51.306636+0000","last_became_peered":"2024-04-25T10:07:51.306636+0000","last_unstale":"2024-04-25T10:07:52.470284+0000","last_undegraded":"2024-04-25T10:07:52.470284+0000","last_fullsized":"2024-04-25T10:07:52.470284+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:57.699968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.472107+0000","last_change":"2024-04-25T10:07:51.304905+0000","last_active":"2024-04-25T10:07:52.472107+0000","last_peered":"2024-04-25T10:07:52.472107+0000","last_clean":"2024-04-25T10:07:52.472107+0000","last_became_active":"2024-04-25T10:07:51.304584+0000","last_became_peered":"2024-04-25T10:07:51.304584+0000","last_unstale":"2024-04-25T10:07:52.472107+0000","last_undegraded":"2024-04-25T10:07:52.472107+0000","last_fullsized":"2024-04-25T10:07:52.472107+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470893+0000","last_change":"2024-04-25T10:07:51.313605+0000","last_active":"2024-04-25T10:07:52.470893+0000","last_peered":"2024-04-25T10:07:52.470893+0000","last_clean":"2024-04-25T10:07:52.470893+0000","last_became_active":"2024-04-25T10:07:51.313265+0000","last_became_peered":"2024-04-25T10:07:51.313265+0000","last_unstale":"2024-04-25T10:07:52.470893+0000","last_undegraded":"2024-04-25T10:07:52.470893+0000","last_fullsized":"2024-04-25T10:07:52.470893+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471365+0000","last_change":"2024-04-25T10:07:51.312675+0000","last_active":"2024-04-25T10:07:52.471365+0000","last_peered":"2024-04-25T10:07:52.471365+0000","last_clean":"2024-04-25T10:07:52.471365+0000","last_became_active":"2024-04-25T10:07:51.311781+0000","last_became_peered":"2024-04-25T10:07:51.311781+0000","last_unstale":"2024-04-25T10:07:52.471365+0000","last_undegraded":"2024-04-25T10:07:52.471365+0000","last_fullsized":"2024-04-25T10:07:52.471365+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.404296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296412+0000","last_change":"2024-04-25T10:07:51.322910+0000","last_active":"2024-04-25T10:07:52.296412+0000","last_peered":"2024-04-25T10:07:52.296412+0000","last_clean":"2024-04-25T10:07:52.296412+0000","last_became_active":"2024-04-25T10:07:51.322604+0000","last_became_peered":"2024-04-25T10:07:51.322604+0000","last_unstale":"2024-04-25T10:07:52.296412+0000","last_undegraded":"2024-04-25T10:07:52.296412+0000","last_fullsized":"2024-04-25T10:07:52.296412+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294454+0000","last_change":"2024-04-25T10:07:51.316521+0000","last_active":"2024-04-25T10:07:52.294454+0000","last_peered":"2024-04-25T10:07:52.294454+0000","last_clean":"2024-04-25T10:07:52.294454+0000","last_became_active":"2024-04-25T10:07:51.316238+0000","last_became_peered":"2024-04-25T10:07:51.316238+0000","last_unstale":"2024-04-25T10:07:52.294454+0000","last_undegraded":"2024-04-25T10:07:52.294454+0000","last_fullsized":"2024-04-25T10:07:52.294454+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296471+0000","last_change":"2024-04-25T10:07:51.312907+0000","last_active":"2024-04-25T10:07:52.296471+0000","last_peered":"2024-04-25T10:07:52.296471+0000","last_clean":"2024-04-25T10:07:52.296471+0000","last_became_active":"2024-04-25T10:07:51.312595+0000","last_became_peered":"2024-04-25T10:07:51.312595+0000","last_unstale":"2024-04-25T10:07:52.296471+0000","last_undegraded":"2024-04-25T10:07:52.296471+0000","last_fullsized":"2024-04-25T10:07:52.296471+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470232+0000","last_change":"2024-04-25T10:07:51.309738+0000","last_active":"2024-04-25T10:07:52.470232+0000","last_peered":"2024-04-25T10:07:52.470232+0000","last_clean":"2024-04-25T10:07:52.470232+0000","last_became_active":"2024-04-25T10:07:51.309394+0000","last_became_peered":"2024-04-25T10:07:51.309394+0000","last_unstale":"2024-04-25T10:07:52.470232+0000","last_undegraded":"2024-04-25T10:07:52.470232+0000","last_fullsized":"2024-04-25T10:07:52.470232+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:51.800823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295906+0000","last_change":"2024-04-25T10:07:51.311322+0000","last_active":"2024-04-25T10:07:52.295906+0000","last_peered":"2024-04-25T10:07:52.295906+0000","last_clean":"2024-04-25T10:07:52.295906+0000","last_became_active":"2024-04-25T10:07:51.311041+0000","last_became_peered":"2024-04-25T10:07:51.311041+0000","last_unstale":"2024-04-25T10:07:52.295906+0000","last_undegraded":"2024-04-25T10:07:52.295906+0000","last_fullsized":"2024-04-25T10:07:52.295906+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:05.899651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471407+0000","last_change":"2024-04-25T10:07:51.317497+0000","last_active":"2024-04-25T10:07:52.471407+0000","last_peered":"2024-04-25T10:07:52.471407+0000","last_clean":"2024-04-25T10:07:52.471407+0000","last_became_active":"2024-04-25T10:07:51.317196+0000","last_became_peered":"2024-04-25T10:07:51.317196+0000","last_unstale":"2024-04-25T10:07:52.471407+0000","last_undegraded":"2024-04-25T10:07:52.471407+0000","last_fullsized":"2024-04-25T10:07:52.471407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:18.847819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294675+0000","last_change":"2024-04-25T10:07:51.324223+0000","last_active":"2024-04-25T10:07:52.294675+0000","last_peered":"2024-04-25T10:07:52.294675+0000","last_clean":"2024-04-25T10:07:52.294675+0000","last_became_active":"2024-04-25T10:07:51.323857+0000","last_became_peered":"2024-04-25T10:07:51.323857+0000","last_unstale":"2024-04-25T10:07:52.294675+0000","last_undegraded":"2024-04-25T10:07:52.294675+0000","last_fullsized":"2024-04-25T10:07:52.294675+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471513+0000","last_change":"2024-04-25T10:07:51.319543+0000","last_active":"2024-04-25T10:07:52.471513+0000","last_peered":"2024-04-25T10:07:52.471513+0000","last_clean":"2024-04-25T10:07:52.471513+0000","last_became_active":"2024-04-25T10:07:51.319265+0000","last_became_peered":"2024-04-25T10:07:51.319265+0000","last_unstale":"2024-04-25T10:07:52.471513+0000","last_undegraded":"2024-04-25T10:07:52.471513+0000","last_fullsized":"2024-04-25T10:07:52.471513+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:42.400180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470296+0000","last_change":"2024-04-25T10:07:51.318244+0000","last_active":"2024-04-25T10:07:52.470296+0000","last_peered":"2024-04-25T10:07:52.470296+0000","last_clean":"2024-04-25T10:07:52.470296+0000","last_became_active":"2024-04-25T10:07:51.317253+0000","last_became_peered":"2024-04-25T10:07:51.317253+0000","last_unstale":"2024-04-25T10:07:52.470296+0000","last_undegraded":"2024-04-25T10:07:52.470296+0000","last_fullsized":"2024-04-25T10:07:52.470296+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:17.235813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470996+0000","last_change":"2024-04-25T10:07:51.313123+0000","last_active":"2024-04-25T10:07:52.470996+0000","last_peered":"2024-04-25T10:07:52.470996+0000","last_clean":"2024-04-25T10:07:52.470996+0000","last_became_active":"2024-04-25T10:07:51.312765+0000","last_became_peered":"2024-04-25T10:07:51.312765+0000","last_unstale":"2024-04-25T10:07:52.470996+0000","last_undegraded":"2024-04-25T10:07:52.470996+0000","last_fullsized":"2024-04-25T10:07:52.470996+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:21.727353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296245+0000","last_change":"2024-04-25T10:07:51.312907+0000","last_active":"2024-04-25T10:07:52.296245+0000","last_peered":"2024-04-25T10:07:52.296245+0000","last_clean":"2024-04-25T10:07:52.296245+0000","last_became_active":"2024-04-25T10:07:51.312558+0000","last_became_peered":"2024-04-25T10:07:51.312558+0000","last_unstale":"2024-04-25T10:07:52.296245+0000","last_undegraded":"2024-04-25T10:07:52.296245+0000","last_fullsized":"2024-04-25T10:07:52.296245+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471064+0000","last_change":"2024-04-25T10:07:51.298328+0000","last_active":"2024-04-25T10:07:52.471064+0000","last_peered":"2024-04-25T10:07:52.471064+0000","last_clean":"2024-04-25T10:07:52.471064+0000","last_became_active":"2024-04-25T10:07:51.297878+0000","last_became_peered":"2024-04-25T10:07:51.297878+0000","last_unstale":"2024-04-25T10:07:52.471064+0000","last_undegraded":"2024-04-25T10:07:52.471064+0000","last_fullsized":"2024-04-25T10:07:52.471064+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:51.296503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296580+0000","last_change":"2024-04-25T10:07:50.306577+0000","last_active":"2024-04-25T10:07:52.296580+0000","last_peered":"2024-04-25T10:07:52.296580+0000","last_clean":"2024-04-25T10:07:52.296580+0000","last_became_active":"2024-04-25T10:07:50.306291+0000","last_became_peered":"2024-04-25T10:07:50.306291+0000","last_unstale":"2024-04-25T10:07:52.296580+0000","last_undegraded":"2024-04-25T10:07:52.296580+0000","last_fullsized":"2024-04-25T10:07:52.296580+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:23.427143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.472178+0000","last_change":"2024-04-25T10:07:51.316840+0000","last_active":"2024-04-25T10:07:52.472178+0000","last_peered":"2024-04-25T10:07:52.472178+0000","last_clean":"2024-04-25T10:07:52.472178+0000","last_became_active":"2024-04-25T10:07:51.316519+0000","last_became_peered":"2024-04-25T10:07:51.316519+0000","last_unstale":"2024-04-25T10:07:52.472178+0000","last_undegraded":"2024-04-25T10:07:52.472178+0000","last_fullsized":"2024-04-25T10:07:52.472178+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.168043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471882+0000","last_change":"2024-04-25T10:07:50.306264+0000","last_active":"2024-04-25T10:07:52.471882+0000","last_peered":"2024-04-25T10:07:52.471882+0000","last_clean":"2024-04-25T10:07:52.471882+0000","last_became_active":"2024-04-25T10:07:50.305960+0000","last_became_peered":"2024-04-25T10:07:50.305960+0000","last_unstale":"2024-04-25T10:07:52.471882+0000","last_undegraded":"2024-04-25T10:07:52.471882+0000","last_fullsized":"2024-04-25T10:07:52.471882+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:02.311727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471542+0000","last_change":"2024-04-25T10:07:50.311222+0000","last_active":"2024-04-25T10:07:52.471542+0000","last_peered":"2024-04-25T10:07:52.471542+0000","last_clean":"2024-04-25T10:07:52.471542+0000","last_became_active":"2024-04-25T10:07:50.310954+0000","last_became_peered":"2024-04-25T10:07:50.310954+0000","last_unstale":"2024-04-25T10:07:52.471542+0000","last_undegraded":"2024-04-25T10:07:52.471542+0000","last_fullsized":"2024-04-25T10:07:52.471542+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:44.898508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471321+0000","last_change":"2024-04-25T10:07:51.312039+0000","last_active":"2024-04-25T10:07:52.471321+0000","last_peered":"2024-04-25T10:07:52.471321+0000","last_clean":"2024-04-25T10:07:52.471321+0000","last_became_active":"2024-04-25T10:07:51.305475+0000","last_became_peered":"2024-04-25T10:07:51.305475+0000","last_unstale":"2024-04-25T10:07:52.471321+0000","last_undegraded":"2024-04-25T10:07:52.471321+0000","last_fullsized":"2024-04-25T10:07:52.471321+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470435+0000","last_change":"2024-04-25T10:07:50.309547+0000","last_active":"2024-04-25T10:07:52.470435+0000","last_peered":"2024-04-25T10:07:52.470435+0000","last_clean":"2024-04-25T10:07:52.470435+0000","last_became_active":"2024-04-25T10:07:50.309240+0000","last_became_peered":"2024-04-25T10:07:50.309240+0000","last_unstale":"2024-04-25T10:07:52.470435+0000","last_undegraded":"2024-04-25T10:07:52.470435+0000","last_fullsized":"2024-04-25T10:07:52.470435+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:23.723920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470369+0000","last_change":"2024-04-25T10:07:50.306966+0000","last_active":"2024-04-25T10:07:52.470369+0000","last_peered":"2024-04-25T10:07:52.470369+0000","last_clean":"2024-04-25T10:07:52.470369+0000","last_became_active":"2024-04-25T10:07:50.306613+0000","last_became_peered":"2024-04-25T10:07:50.306613+0000","last_unstale":"2024-04-25T10:07:52.470369+0000","last_undegraded":"2024-04-25T10:07:52.470369+0000","last_fullsized":"2024-04-25T10:07:52.470369+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:16.044211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296418+0000","last_change":"2024-04-25T10:07:51.319075+0000","last_active":"2024-04-25T10:07:52.296418+0000","last_peered":"2024-04-25T10:07:52.296418+0000","last_clean":"2024-04-25T10:07:52.296418+0000","last_became_active":"2024-04-25T10:07:51.318802+0000","last_became_peered":"2024-04-25T10:07:51.318802+0000","last_unstale":"2024-04-25T10:07:52.296418+0000","last_undegraded":"2024-04-25T10:07:52.296418+0000","last_fullsized":"2024-04-25T10:07:52.296418+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470993+0000","last_change":"2024-04-25T10:07:50.311086+0000","last_active":"2024-04-25T10:07:52.470993+0000","last_peered":"2024-04-25T10:07:52.470993+0000","last_clean":"2024-04-25T10:07:52.470993+0000","last_became_active":"2024-04-25T10:07:50.310798+0000","last_became_peered":"2024-04-25T10:07:50.310798+0000","last_unstale":"2024-04-25T10:07:52.470993+0000","last_undegraded":"2024-04-25T10:07:52.470993+0000","last_fullsized":"2024-04-25T10:07:52.470993+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:46.162970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.469999+0000","last_change":"2024-04-25T10:07:51.305304+0000","last_active":"2024-04-25T10:07:52.469999+0000","last_peered":"2024-04-25T10:07:52.469999+0000","last_clean":"2024-04-25T10:07:52.469999+0000","last_became_active":"2024-04-25T10:07:51.304968+0000","last_became_peered":"2024-04-25T10:07:51.304968+0000","last_unstale":"2024-04-25T10:07:52.469999+0000","last_undegraded":"2024-04-25T10:07:52.469999+0000","last_fullsized":"2024-04-25T10:07:52.469999+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:49.018249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295971+0000","last_change":"2024-04-25T10:07:50.305140+0000","last_active":"2024-04-25T10:07:52.295971+0000","last_peered":"2024-04-25T10:07:52.295971+0000","last_clean":"2024-04-25T10:07:52.295971+0000","last_became_active":"2024-04-25T10:07:50.304795+0000","last_became_peered":"2024-04-25T10:07:50.304795+0000","last_unstale":"2024-04-25T10:07:52.295971+0000","last_undegraded":"2024-04-25T10:07:52.295971+0000","last_fullsized":"2024-04-25T10:07:52.295971+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:22.138626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471236+0000","last_change":"2024-04-25T10:07:51.317275+0000","last_active":"2024-04-25T10:07:52.471236+0000","last_peered":"2024-04-25T10:07:52.471236+0000","last_clean":"2024-04-25T10:07:52.471236+0000","last_became_active":"2024-04-25T10:07:51.316910+0000","last_became_peered":"2024-04-25T10:07:51.316910+0000","last_unstale":"2024-04-25T10:07:52.471236+0000","last_undegraded":"2024-04-25T10:07:52.471236+0000","last_fullsized":"2024-04-25T10:07:52.471236+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:16.208395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294270+0000","last_change":"2024-04-25T10:07:50.307893+0000","last_active":"2024-04-25T10:07:52.294270+0000","last_peered":"2024-04-25T10:07:52.294270+0000","last_clean":"2024-04-25T10:07:52.294270+0000","last_became_active":"2024-04-25T10:07:50.307514+0000","last_became_peered":"2024-04-25T10:07:50.307514+0000","last_unstale":"2024-04-25T10:07:52.294270+0000","last_undegraded":"2024-04-25T10:07:52.294270+0000","last_fullsized":"2024-04-25T10:07:52.294270+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:33.584068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294636+0000","last_change":"2024-04-25T10:07:51.317446+0000","last_active":"2024-04-25T10:07:52.294636+0000","last_peered":"2024-04-25T10:07:52.294636+0000","last_clean":"2024-04-25T10:07:52.294636+0000","last_became_active":"2024-04-25T10:07:51.315123+0000","last_became_peered":"2024-04-25T10:07:51.315123+0000","last_unstale":"2024-04-25T10:07:52.294636+0000","last_undegraded":"2024-04-25T10:07:52.294636+0000","last_fullsized":"2024-04-25T10:07:52.294636+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296218+0000","last_change":"2024-04-25T10:07:50.306714+0000","last_active":"2024-04-25T10:07:52.296218+0000","last_peered":"2024-04-25T10:07:52.296218+0000","last_clean":"2024-04-25T10:07:52.296218+0000","last_became_active":"2024-04-25T10:07:50.306439+0000","last_became_peered":"2024-04-25T10:07:50.306439+0000","last_unstale":"2024-04-25T10:07:52.296218+0000","last_undegraded":"2024-04-25T10:07:52.296218+0000","last_fullsized":"2024-04-25T10:07:52.296218+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:48.651672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470854+0000","last_change":"2024-04-25T10:07:51.309229+0000","last_active":"2024-04-25T10:07:52.470854+0000","last_peered":"2024-04-25T10:07:52.470854+0000","last_clean":"2024-04-25T10:07:52.470854+0000","last_became_active":"2024-04-25T10:07:51.308925+0000","last_became_peered":"2024-04-25T10:07:51.308925+0000","last_unstale":"2024-04-25T10:07:52.470854+0000","last_undegraded":"2024-04-25T10:07:52.470854+0000","last_fullsized":"2024-04-25T10:07:52.470854+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.293101+0000","last_change":"2024-04-25T10:07:49.293101+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.293101+0000","last_undegraded":"2024-04-25T10:07:49.293101+0000","last_fullsized":"2024-04-25T10:07:49.293101+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470639+0000","last_change":"2024-04-25T10:07:50.309095+0000","last_active":"2024-04-25T10:07:52.470639+0000","last_peered":"2024-04-25T10:07:52.470639+0000","last_clean":"2024-04-25T10:07:52.470639+0000","last_became_active":"2024-04-25T10:07:50.308756+0000","last_became_peered":"2024-04-25T10:07:50.308756+0000","last_unstale":"2024-04-25T10:07:52.470639+0000","last_undegraded":"2024-04-25T10:07:52.470639+0000","last_fullsized":"2024-04-25T10:07:52.470639+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:04.298496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470541+0000","last_change":"2024-04-25T10:07:51.311605+0000","last_active":"2024-04-25T10:07:52.470541+0000","last_peered":"2024-04-25T10:07:52.470541+0000","last_clean":"2024-04-25T10:07:52.470541+0000","last_became_active":"2024-04-25T10:07:51.311307+0000","last_became_peered":"2024-04-25T10:07:51.311307+0000","last_unstale":"2024-04-25T10:07:52.470541+0000","last_undegraded":"2024-04-25T10:07:52.470541+0000","last_fullsized":"2024-04-25T10:07:52.470541+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:00.278450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294600+0000","last_change":"2024-04-25T10:07:50.306790+0000","last_active":"2024-04-25T10:07:52.294600+0000","last_peered":"2024-04-25T10:07:52.294600+0000","last_clean":"2024-04-25T10:07:52.294600+0000","last_became_active":"2024-04-25T10:07:50.306491+0000","last_became_peered":"2024-04-25T10:07:50.306491+0000","last_unstale":"2024-04-25T10:07:52.294600+0000","last_undegraded":"2024-04-25T10:07:52.294600+0000","last_fullsized":"2024-04-25T10:07:52.294600+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:43.478815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295998+0000","last_change":"2024-04-25T10:07:51.306158+0000","last_active":"2024-04-25T10:07:52.295998+0000","last_peered":"2024-04-25T10:07:52.295998+0000","last_clean":"2024-04-25T10:07:52.295998+0000","last_became_active":"2024-04-25T10:07:51.305824+0000","last_became_peered":"2024-04-25T10:07:51.305824+0000","last_unstale":"2024-04-25T10:07:52.295998+0000","last_undegraded":"2024-04-25T10:07:52.295998+0000","last_fullsized":"2024-04-25T10:07:52.295998+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:17.474618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292461+0000","last_change":"2024-04-25T10:07:49.292461+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292461+0000","last_undegraded":"2024-04-25T10:07:49.292461+0000","last_fullsized":"2024-04-25T10:07:49.292461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470269+0000","last_change":"2024-04-25T10:07:51.307313+0000","last_active":"2024-04-25T10:07:52.470269+0000","last_peered":"2024-04-25T10:07:52.470269+0000","last_clean":"2024-04-25T10:07:52.470269+0000","last_became_active":"2024-04-25T10:07:51.307032+0000","last_became_peered":"2024-04-25T10:07:51.307032+0000","last_unstale":"2024-04-25T10:07:52.470269+0000","last_undegraded":"2024-04-25T10:07:52.470269+0000","last_fullsized":"2024-04-25T10:07:52.470269+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:51.296503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470393+0000","last_change":"2024-04-25T10:07:50.304944+0000","last_active":"2024-04-25T10:07:52.470393+0000","last_peered":"2024-04-25T10:07:52.470393+0000","last_clean":"2024-04-25T10:07:52.470393+0000","last_became_active":"2024-04-25T10:07:50.304542+0000","last_became_peered":"2024-04-25T10:07:50.304542+0000","last_unstale":"2024-04-25T10:07:52.470393+0000","last_undegraded":"2024-04-25T10:07:52.470393+0000","last_fullsized":"2024-04-25T10:07:52.470393+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:39.934850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295938+0000","last_change":"2024-04-25T10:07:51.320164+0000","last_active":"2024-04-25T10:07:52.295938+0000","last_peered":"2024-04-25T10:07:52.295938+0000","last_clean":"2024-04-25T10:07:52.295938+0000","last_became_active":"2024-04-25T10:07:51.319859+0000","last_became_peered":"2024-04-25T10:07:51.319859+0000","last_unstale":"2024-04-25T10:07:52.295938+0000","last_undegraded":"2024-04-25T10:07:52.295938+0000","last_fullsized":"2024-04-25T10:07:52.295938+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:21.727353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470461+0000","last_change":"2024-04-25T10:07:50.309933+0000","last_active":"2024-04-25T10:07:52.470461+0000","last_peered":"2024-04-25T10:07:52.470461+0000","last_clean":"2024-04-25T10:07:52.470461+0000","last_became_active":"2024-04-25T10:07:50.309637+0000","last_became_peered":"2024-04-25T10:07:50.309637+0000","last_unstale":"2024-04-25T10:07:52.470461+0000","last_undegraded":"2024-04-25T10:07:52.470461+0000","last_fullsized":"2024-04-25T10:07:52.470461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:09.460489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470368+0000","last_change":"2024-04-25T10:07:51.308930+0000","last_active":"2024-04-25T10:07:52.470368+0000","last_peered":"2024-04-25T10:07:52.470368+0000","last_clean":"2024-04-25T10:07:52.470368+0000","last_became_active":"2024-04-25T10:07:51.308603+0000","last_became_peered":"2024-04-25T10:07:51.308603+0000","last_unstale":"2024-04-25T10:07:52.470368+0000","last_undegraded":"2024-04-25T10:07:52.470368+0000","last_fullsized":"2024-04-25T10:07:52.470368+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:17.724785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470020+0000","last_change":"2024-04-25T10:07:50.304956+0000","last_active":"2024-04-25T10:07:52.470020+0000","last_peered":"2024-04-25T10:07:52.470020+0000","last_clean":"2024-04-25T10:07:52.470020+0000","last_became_active":"2024-04-25T10:07:50.304634+0000","last_became_peered":"2024-04-25T10:07:50.304634+0000","last_unstale":"2024-04-25T10:07:52.470020+0000","last_undegraded":"2024-04-25T10:07:52.470020+0000","last_fullsized":"2024-04-25T10:07:52.470020+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:28.896776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470917+0000","last_change":"2024-04-25T10:07:51.314008+0000","last_active":"2024-04-25T10:07:52.470917+0000","last_peered":"2024-04-25T10:07:52.470917+0000","last_clean":"2024-04-25T10:07:52.470917+0000","last_became_active":"2024-04-25T10:07:51.313611+0000","last_became_peered":"2024-04-25T10:07:51.313611+0000","last_unstale":"2024-04-25T10:07:52.470917+0000","last_undegraded":"2024-04-25T10:07:52.470917+0000","last_fullsized":"2024-04-25T10:07:52.470917+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471515+0000","last_change":"2024-04-25T10:07:50.313909+0000","last_active":"2024-04-25T10:07:52.471515+0000","last_peered":"2024-04-25T10:07:52.471515+0000","last_clean":"2024-04-25T10:07:52.471515+0000","last_became_active":"2024-04-25T10:07:50.313400+0000","last_became_peered":"2024-04-25T10:07:50.313400+0000","last_unstale":"2024-04-25T10:07:52.471515+0000","last_undegraded":"2024-04-25T10:07:52.471515+0000","last_fullsized":"2024-04-25T10:07:52.471515+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:09.642875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471017+0000","last_change":"2024-04-25T10:07:51.310494+0000","last_active":"2024-04-25T10:07:52.471017+0000","last_peered":"2024-04-25T10:07:52.471017+0000","last_clean":"2024-04-25T10:07:52.471017+0000","last_became_active":"2024-04-25T10:07:51.310224+0000","last_became_peered":"2024-04-25T10:07:51.310224+0000","last_unstale":"2024-04-25T10:07:52.471017+0000","last_undegraded":"2024-04-25T10:07:52.471017+0000","last_fullsized":"2024-04-25T10:07:52.471017+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292480+0000","last_change":"2024-04-25T10:07:49.292480+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292480+0000","last_undegraded":"2024-04-25T10:07:49.292480+0000","last_fullsized":"2024-04-25T10:07:49.292480+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471787+0000","last_change":"2024-04-25T10:07:51.318600+0000","last_active":"2024-04-25T10:07:52.471787+0000","last_peered":"2024-04-25T10:07:52.471787+0000","last_clean":"2024-04-25T10:07:52.471787+0000","last_became_active":"2024-04-25T10:07:51.318304+0000","last_became_peered":"2024-04-25T10:07:51.318304+0000","last_unstale":"2024-04-25T10:07:52.471787+0000","last_undegraded":"2024-04-25T10:07:52.471787+0000","last_fullsized":"2024-04-25T10:07:52.471787+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:53.592626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470762+0000","last_change":"2024-04-25T10:07:50.304446+0000","last_active":"2024-04-25T10:07:52.470762+0000","last_peered":"2024-04-25T10:07:52.470762+0000","last_clean":"2024-04-25T10:07:52.470762+0000","last_became_active":"2024-04-25T10:07:50.303898+0000","last_became_peered":"2024-04-25T10:07:50.303898+0000","last_unstale":"2024-04-25T10:07:52.470762+0000","last_undegraded":"2024-04-25T10:07:52.470762+0000","last_fullsized":"2024-04-25T10:07:52.470762+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:23.749111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470626+0000","last_change":"2024-04-25T10:07:51.319204+0000","last_active":"2024-04-25T10:07:52.470626+0000","last_peered":"2024-04-25T10:07:52.470626+0000","last_clean":"2024-04-25T10:07:52.470626+0000","last_became_active":"2024-04-25T10:07:51.318367+0000","last_became_peered":"2024-04-25T10:07:51.318367+0000","last_unstale":"2024-04-25T10:07:52.470626+0000","last_undegraded":"2024-04-25T10:07:52.470626+0000","last_fullsized":"2024-04-25T10:07:52.470626+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:46.811230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292499+0000","last_change":"2024-04-25T10:07:49.292499+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292499+0000","last_undegraded":"2024-04-25T10:07:49.292499+0000","last_fullsized":"2024-04-25T10:07:49.292499+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296169+0000","last_change":"2024-04-25T10:07:51.312843+0000","last_active":"2024-04-25T10:07:52.296169+0000","last_peered":"2024-04-25T10:07:52.296169+0000","last_clean":"2024-04-25T10:07:52.296169+0000","last_became_active":"2024-04-25T10:07:51.311808+0000","last_became_peered":"2024-04-25T10:07:51.311808+0000","last_unstale":"2024-04-25T10:07:52.296169+0000","last_undegraded":"2024-04-25T10:07:52.296169+0000","last_fullsized":"2024-04-25T10:07:52.296169+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:18.847819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296623+0000","last_change":"2024-04-25T10:07:50.306664+0000","last_active":"2024-04-25T10:07:52.296623+0000","last_peered":"2024-04-25T10:07:52.296623+0000","last_clean":"2024-04-25T10:07:52.296623+0000","last_became_active":"2024-04-25T10:07:50.306383+0000","last_became_peered":"2024-04-25T10:07:50.306383+0000","last_unstale":"2024-04-25T10:07:52.296623+0000","last_undegraded":"2024-04-25T10:07:52.296623+0000","last_fullsized":"2024-04-25T10:07:52.296623+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:10.698506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292457+0000","last_change":"2024-04-25T10:07:49.292457+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292457+0000","last_undegraded":"2024-04-25T10:07:49.292457+0000","last_fullsized":"2024-04-25T10:07:49.292457+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296086+0000","last_change":"2024-04-25T10:07:50.311607+0000","last_active":"2024-04-25T10:07:52.296086+0000","last_peered":"2024-04-25T10:07:52.296086+0000","last_clean":"2024-04-25T10:07:52.296086+0000","last_became_active":"2024-04-25T10:07:50.311300+0000","last_became_peered":"2024-04-25T10:07:50.311300+0000","last_unstale":"2024-04-25T10:07:52.296086+0000","last_undegraded":"2024-04-25T10:07:52.296086+0000","last_fullsized":"2024-04-25T10:07:52.296086+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.109707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471043+0000","last_change":"2024-04-25T10:07:51.314396+0000","last_active":"2024-04-25T10:07:52.471043+0000","last_peered":"2024-04-25T10:07:52.471043+0000","last_clean":"2024-04-25T10:07:52.471043+0000","last_became_active":"2024-04-25T10:07:51.314116+0000","last_became_peered":"2024-04-25T10:07:51.314116+0000","last_unstale":"2024-04-25T10:07:52.471043+0000","last_undegraded":"2024-04-25T10:07:52.471043+0000","last_fullsized":"2024-04-25T10:07:52.471043+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:19.021120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471632+0000","last_change":"2024-04-25T10:07:50.307927+0000","last_active":"2024-04-25T10:07:52.471632+0000","last_peered":"2024-04-25T10:07:52.471632+0000","last_clean":"2024-04-25T10:07:52.471632+0000","last_became_active":"2024-04-25T10:07:50.307641+0000","last_became_peered":"2024-04-25T10:07:50.307641+0000","last_unstale":"2024-04-25T10:07:52.471632+0000","last_undegraded":"2024-04-25T10:07:52.471632+0000","last_fullsized":"2024-04-25T10:07:52.471632+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:22.138626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471021+0000","last_change":"2024-04-25T10:07:51.311790+0000","last_active":"2024-04-25T10:07:52.471021+0000","last_peered":"2024-04-25T10:07:52.471021+0000","last_clean":"2024-04-25T10:07:52.471021+0000","last_became_active":"2024-04-25T10:07:51.311524+0000","last_became_peered":"2024-04-25T10:07:51.311524+0000","last_unstale":"2024-04-25T10:07:52.471021+0000","last_undegraded":"2024-04-25T10:07:52.471021+0000","last_fullsized":"2024-04-25T10:07:52.471021+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.168043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470815+0000","last_change":"2024-04-25T10:07:50.313145+0000","last_active":"2024-04-25T10:07:52.470815+0000","last_peered":"2024-04-25T10:07:52.470815+0000","last_clean":"2024-04-25T10:07:52.470815+0000","last_became_active":"2024-04-25T10:07:50.312860+0000","last_became_peered":"2024-04-25T10:07:50.312860+0000","last_unstale":"2024-04-25T10:07:52.470815+0000","last_undegraded":"2024-04-25T10:07:52.470815+0000","last_fullsized":"2024-04-25T10:07:52.470815+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:16.044211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471897+0000","last_change":"2024-04-25T10:07:51.322690+0000","last_active":"2024-04-25T10:07:52.471897+0000","last_peered":"2024-04-25T10:07:52.471897+0000","last_clean":"2024-04-25T10:07:52.471897+0000","last_became_active":"2024-04-25T10:07:51.322404+0000","last_became_peered":"2024-04-25T10:07:51.322404+0000","last_unstale":"2024-04-25T10:07:52.471897+0000","last_undegraded":"2024-04-25T10:07:52.471897+0000","last_fullsized":"2024-04-25T10:07:52.471897+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:00.981450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296163+0000","last_change":"2024-04-25T10:07:50.309078+0000","last_active":"2024-04-25T10:07:52.296163+0000","last_peered":"2024-04-25T10:07:52.296163+0000","last_clean":"2024-04-25T10:07:52.296163+0000","last_became_active":"2024-04-25T10:07:50.308794+0000","last_became_peered":"2024-04-25T10:07:50.308794+0000","last_unstale":"2024-04-25T10:07:52.296163+0000","last_undegraded":"2024-04-25T10:07:52.296163+0000","last_fullsized":"2024-04-25T10:07:52.296163+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:09.642875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470834+0000","last_change":"2024-04-25T10:07:50.307963+0000","last_active":"2024-04-25T10:07:52.470834+0000","last_peered":"2024-04-25T10:07:52.470834+0000","last_clean":"2024-04-25T10:07:52.470834+0000","last_became_active":"2024-04-25T10:07:50.307611+0000","last_became_peered":"2024-04-25T10:07:50.307611+0000","last_unstale":"2024-04-25T10:07:52.470834+0000","last_undegraded":"2024-04-25T10:07:52.470834+0000","last_fullsized":"2024-04-25T10:07:52.470834+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:13.364471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470886+0000","last_change":"2024-04-25T10:07:51.316433+0000","last_active":"2024-04-25T10:07:52.470886+0000","last_peered":"2024-04-25T10:07:52.470886+0000","last_clean":"2024-04-25T10:07:52.470886+0000","last_became_active":"2024-04-25T10:07:51.315952+0000","last_became_peered":"2024-04-25T10:07:51.315952+0000","last_unstale":"2024-04-25T10:07:52.470886+0000","last_undegraded":"2024-04-25T10:07:52.470886+0000","last_fullsized":"2024-04-25T10:07:52.470886+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:13.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471022+0000","last_change":"2024-04-25T10:07:50.313990+0000","last_active":"2024-04-25T10:07:52.471022+0000","last_peered":"2024-04-25T10:07:52.471022+0000","last_clean":"2024-04-25T10:07:52.471022+0000","last_became_active":"2024-04-25T10:07:50.313700+0000","last_became_peered":"2024-04-25T10:07:50.313700+0000","last_unstale":"2024-04-25T10:07:52.471022+0000","last_undegraded":"2024-04-25T10:07:52.471022+0000","last_fullsized":"2024-04-25T10:07:52.471022+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:46.602681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294542+0000","last_change":"2024-04-25T10:07:51.315938+0000","last_active":"2024-04-25T10:07:52.294542+0000","last_peered":"2024-04-25T10:07:52.294542+0000","last_clean":"2024-04-25T10:07:52.294542+0000","last_became_active":"2024-04-25T10:07:51.315335+0000","last_became_peered":"2024-04-25T10:07:51.315335+0000","last_unstale":"2024-04-25T10:07:52.294542+0000","last_undegraded":"2024-04-25T10:07:52.294542+0000","last_fullsized":"2024-04-25T10:07:52.294542+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470304+0000","last_change":"2024-04-25T10:07:50.306917+0000","last_active":"2024-04-25T10:07:52.470304+0000","last_peered":"2024-04-25T10:07:52.470304+0000","last_clean":"2024-04-25T10:07:52.470304+0000","last_became_active":"2024-04-25T10:07:50.306549+0000","last_became_peered":"2024-04-25T10:07:50.306549+0000","last_unstale":"2024-04-25T10:07:52.470304+0000","last_undegraded":"2024-04-25T10:07:52.470304+0000","last_fullsized":"2024-04-25T10:07:52.470304+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:06.727495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471599+0000","last_change":"2024-04-25T10:07:50.311119+0000","last_active":"2024-04-25T10:07:52.471599+0000","last_peered":"2024-04-25T10:07:52.471599+0000","last_clean":"2024-04-25T10:07:52.471599+0000","last_became_active":"2024-04-25T10:07:50.310843+0000","last_became_peered":"2024-04-25T10:07:50.310843+0000","last_unstale":"2024-04-25T10:07:52.471599+0000","last_undegraded":"2024-04-25T10:07:52.471599+0000","last_fullsized":"2024-04-25T10:07:52.471599+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:26.803283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471140+0000","last_change":"2024-04-25T10:07:51.316546+0000","last_active":"2024-04-25T10:07:52.471140+0000","last_peered":"2024-04-25T10:07:52.471140+0000","last_clean":"2024-04-25T10:07:52.471140+0000","last_became_active":"2024-04-25T10:07:51.313979+0000","last_became_peered":"2024-04-25T10:07:51.313979+0000","last_unstale":"2024-04-25T10:07:52.471140+0000","last_undegraded":"2024-04-25T10:07:52.471140+0000","last_fullsized":"2024-04-25T10:07:52.471140+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:17.474618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292461+0000","last_change":"2024-04-25T10:07:49.292461+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292461+0000","last_undegraded":"2024-04-25T10:07:49.292461+0000","last_fullsized":"2024-04-25T10:07:49.292461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294591+0000","last_change":"2024-04-25T10:07:50.309026+0000","last_active":"2024-04-25T10:07:52.294591+0000","last_peered":"2024-04-25T10:07:52.294591+0000","last_clean":"2024-04-25T10:07:52.294591+0000","last_became_active":"2024-04-25T10:07:50.308706+0000","last_became_peered":"2024-04-25T10:07:50.308706+0000","last_unstale":"2024-04-25T10:07:52.294591+0000","last_undegraded":"2024-04-25T10:07:52.294591+0000","last_fullsized":"2024-04-25T10:07:52.294591+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:17.725300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":245,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295995+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:07:52.295995+0000","last_peered":"2024-04-25T10:07:52.295995+0000","last_clean":"2024-04-25T10:07:52.295995+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:07:52.295995+0000","last_undegraded":"2024-04-25T10:07:52.295995+0000","last_fullsized":"2024-04-25T10:07:52.295995+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":7},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":288874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":692224,"data_stored":282504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":289398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":692224,"data_stored":282572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":284254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509478,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1298432,"data_stored":879732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":700416,"data_stored":284094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":859800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:54.318 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:54.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.449+0000 7fad91549640 1 Processor -- start 2024-04-25T10:07:54.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.449+0000 7fad91549640 1 -- start start 2024-04-25T10:07:54.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.450+0000 7fad91549640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad8c0d35b0 0x7fad8c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.450+0000 7fad91549640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fad8c0b5990 0x7fad8c0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.450+0000 7fad91549640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8c15e0c0 con 0x7fad8c15dce0 2024-04-25T10:07:54.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.450+0000 7fad91549640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8c0b5de0 con 0x7fad8c0b5990 2024-04-25T10:07:54.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.450+0000 7fad91549640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0d2510 con 0x7fad8c0d35b0 2024-04-25T10:07:54.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.450+0000 7fad8b7fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fad8c0b5990 0x7fad8c0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.450+0000 7fad8a7fc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad8c0d35b0 0x7fad8c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.450+0000 7fad8b7fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fad8c0b5990 0x7fad8c0b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:42452/0 (socket says 172.21.15.16:42452) 2024-04-25T10:07:54.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.450+0000 7fad8b7fe640 1 -- 172.21.15.16:0/2042362765 learned_addr learned my addr 172.21.15.16:0/2042362765 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:54.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.450+0000 7fad8affd640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fad8c15dce0 0x7fad8c0d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:34994/0 (socket says 172.21.15.16:34994) 2024-04-25T10:07:54.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.450+0000 7fad8a7fc640 1 -- 172.21.15.16:0/2042362765 >> v2:172.21.15.186:3300/0 conn(0x7fad8c0b5990 msgr2=0x7fad8c0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:54.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.450+0000 7fad8a7fc640 1 --2- 172.21.15.16:0/2042362765 >> v2:172.21.15.186:3300/0 conn(0x7fad8c0b5990 0x7fad8c0b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.450+0000 7fad8a7fc640 1 -- 172.21.15.16:0/2042362765 >> v1:172.21.15.186:6789/0 conn(0x7fad8c15dce0 legacy=0x7fad8c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.450+0000 7fad8a7fc640 1 -- 172.21.15.16:0/2042362765 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c0d27f0 con 0x7fad8c0d35b0 2024-04-25T10:07:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.451+0000 7fad8a7fc640 1 --2- 172.21.15.16:0/2042362765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad8c0d35b0 0x7fad8c0b5450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fad800097a0 tx=0x7fad8002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c4adb761bf182d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.451+0000 7fad89ffb640 1 -- 172.21.15.16:0/2042362765 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80037070 con 0x7fad8c0d35b0 2024-04-25T10:07:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.451+0000 7fad89ffb640 1 -- 172.21.15.16:0/2042362765 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad80032440 con 0x7fad8c0d35b0 2024-04-25T10:07:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.451+0000 7fad89ffb640 1 -- 172.21.15.16:0/2042362765 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80032810 con 0x7fad8c0d35b0 2024-04-25T10:07:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.452+0000 7fad91549640 1 -- 172.21.15.16:0/2042362765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad8c0d35b0 msgr2=0x7fad8c0b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.452+0000 7fad91549640 1 --2- 172.21.15.16:0/2042362765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad8c0d35b0 0x7fad8c0b5450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fad800097a0 tx=0x7fad8002a180 comp rx=0 tx=0).stop 2024-04-25T10:07:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.452+0000 7fad91549640 1 -- 172.21.15.16:0/2042362765 shutdown_connections 2024-04-25T10:07:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.452+0000 7fad91549640 1 --2- 172.21.15.16:0/2042362765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad8c0d35b0 0x7fad8c0b5450 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.452+0000 7fad91549640 1 --2- 172.21.15.16:0/2042362765 >> v2:172.21.15.186:3300/0 conn(0x7fad8c0b5990 0x7fad8c0b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.452+0000 7fad91549640 1 -- 172.21.15.16:0/2042362765 >> 172.21.15.16:0/2042362765 conn(0x7fad8c072900 msgr2=0x7fad8c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.452+0000 7fad91549640 1 -- 172.21.15.16:0/2042362765 shutdown_connections 2024-04-25T10:07:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.452+0000 7fad91549640 1 -- 172.21.15.16:0/2042362765 wait complete. 2024-04-25T10:07:54.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.453+0000 7fad91549640 1 Processor -- start 2024-04-25T10:07:54.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.453+0000 7fad91549640 1 -- start start 2024-04-25T10:07:54.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.454+0000 7fad91549640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad8c0b5990 0x7fad8c14d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.454+0000 7fad91549640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad8c0d35b0 0x7fad8c15fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.454+0000 7fad91549640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad8c15dce0 0x7fad8c149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.454+0000 7fad91549640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0d3090 con 0x7fad8c0b5990 2024-04-25T10:07:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.454+0000 7fad91549640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0d2db0 con 0x7fad8c0d35b0 2024-04-25T10:07:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.454+0000 7fad91549640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0d2160 con 0x7fad8c15dce0 2024-04-25T10:07:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.454+0000 7fad8affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad8c0b5990 0x7fad8c14d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.454+0000 7fad8affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad8c0b5990 0x7fad8c14d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53122/0 (socket says 172.21.15.16:53122) 2024-04-25T10:07:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.454+0000 7fad8affd640 1 -- 172.21.15.16:0/3908737993 learned_addr learned my addr 172.21.15.16:0/3908737993 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.454+0000 7fad8b7fe640 1 --2- 172.21.15.16:0/3908737993 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad8c15dce0 0x7fad8c149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.454+0000 7fad8a7fc640 1 --2- 172.21.15.16:0/3908737993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad8c0d35b0 0x7fad8c15fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.454+0000 7fad8affd640 1 -- 172.21.15.16:0/3908737993 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad8c15dce0 msgr2=0x7fad8c149d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.454+0000 7fad8affd640 1 --2- 172.21.15.16:0/3908737993 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad8c15dce0 0x7fad8c149d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.454+0000 7fad8affd640 1 -- 172.21.15.16:0/3908737993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad8c0d35b0 msgr2=0x7fad8c15fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.454+0000 7fad8affd640 1 --2- 172.21.15.16:0/3908737993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad8c0d35b0 0x7fad8c15fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.454+0000 7fad8affd640 1 -- 172.21.15.16:0/3908737993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c1611c0 con 0x7fad8c0b5990 2024-04-25T10:07:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.455+0000 7fad8affd640 1 --2- 172.21.15.16:0/3908737993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad8c0b5990 0x7fad8c14d240 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00c3c0 tx=0x7fad7c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.455+0000 7fad737fe640 1 -- 172.21.15.16:0/3908737993 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0074e0 con 0x7fad8c0b5990 2024-04-25T10:07:54.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.455+0000 7fad737fe640 1 -- 172.21.15.16:0/3908737993 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c007680 con 0x7fad8c0b5990 2024-04-25T10:07:54.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.455+0000 7fad91549640 1 -- 172.21.15.16:0/3908737993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8c160640 con 0x7fad8c0b5990 2024-04-25T10:07:54.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.455+0000 7fad737fe640 1 -- 172.21.15.16:0/3908737993 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c004a00 con 0x7fad8c0b5990 2024-04-25T10:07:54.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.455+0000 7fad91549640 1 -- 172.21.15.16:0/3908737993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8c160c00 con 0x7fad8c0b5990 2024-04-25T10:07:54.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.456+0000 7fad91549640 1 -- 172.21.15.16:0/3908737993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8c067490 con 0x7fad8c0b5990 2024-04-25T10:07:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.456+0000 7fad737fe640 1 -- 172.21.15.16:0/3908737993 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad7c01a020 con 0x7fad8c0b5990 2024-04-25T10:07:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.457+0000 7fad737fe640 1 --2- 172.21.15.16:0/3908737993 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fad7806b4d0 0x7fad7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.457+0000 7fad737fe640 1 -- 172.21.15.16:0/3908737993 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad7c0899a0 con 0x7fad8c0b5990 2024-04-25T10:07:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.457+0000 7fad8a7fc640 1 --2- 172.21.15.16:0/3908737993 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fad7806b4d0 0x7fad7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.457+0000 7fad8a7fc640 1 --2- 172.21.15.16:0/3908737993 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fad7806b4d0 0x7fad7806d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fad80034000 tx=0x7fad8002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.461+0000 7fad737fe640 1 -- 172.21.15.16:0/3908737993 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad8c067490 con 0x7fad8c0b5990 2024-04-25T10:07:54.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.637+0000 7fad91549640 1 -- 172.21.15.16:0/3908737993 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad8c15e290 con 0x7fad7806b4d0 2024-04-25T10:07:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.644+0000 7fad737fe640 1 -- 172.21.15.16:0/3908737993 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249962 (secure 0 0 0) 0x7fad8003c460 con 0x7fad7806b4d0 2024-04-25T10:07:54.652 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:54.653 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:07:54.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.646+0000 7fad91549640 1 -- 172.21.15.16:0/3908737993 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fad7806b4d0 msgr2=0x7fad7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.646+0000 7fad91549640 1 --2- 172.21.15.16:0/3908737993 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fad7806b4d0 0x7fad7806d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fad80034000 tx=0x7fad8002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:54.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.646+0000 7fad91549640 1 -- 172.21.15.16:0/3908737993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad8c0b5990 msgr2=0x7fad8c14d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.646+0000 7fad91549640 1 --2- 172.21.15.16:0/3908737993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad8c0b5990 0x7fad8c14d240 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00c3c0 tx=0x7fad7c00c880 comp rx=0 tx=0).stop 2024-04-25T10:07:54.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.646+0000 7fad91549640 1 -- 172.21.15.16:0/3908737993 shutdown_connections 2024-04-25T10:07:54.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.646+0000 7fad91549640 1 --2- 172.21.15.16:0/3908737993 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fad7806b4d0 0x7fad7806d990 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.646+0000 7fad91549640 1 --2- 172.21.15.16:0/3908737993 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad8c15dce0 0x7fad8c149d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.646+0000 7fad91549640 1 --2- 172.21.15.16:0/3908737993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad8c0d35b0 0x7fad8c15fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.646+0000 7fad91549640 1 --2- 172.21.15.16:0/3908737993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad8c0b5990 0x7fad8c14d240 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.646+0000 7fad91549640 1 -- 172.21.15.16:0/3908737993 >> 172.21.15.16:0/3908737993 conn(0x7fad8c072900 msgr2=0x7fad8c0d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:54.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.647+0000 7fad91549640 1 -- 172.21.15.16:0/3908737993 shutdown_connections 2024-04-25T10:07:54.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.647+0000 7fad91549640 1 -- 172.21.15.16:0/3908737993 wait complete. 2024-04-25T10:07:54.663 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-04-25T10:07:53.818447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":197,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243604,"kb_used_data":6608,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731116,"statfs":{"total":773094113280,"available":772844662784,"internally_reserved":0,"allocated":6766592,"data_stored":3451228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.533513"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.472064+0000","last_change":"2024-04-25T10:07:51.311604+0000","last_active":"2024-04-25T10:07:52.472064+0000","last_peered":"2024-04-25T10:07:52.472064+0000","last_clean":"2024-04-25T10:07:52.472064+0000","last_became_active":"2024-04-25T10:07:51.310395+0000","last_became_peered":"2024-04-25T10:07:51.310395+0000","last_unstale":"2024-04-25T10:07:52.472064+0000","last_undegraded":"2024-04-25T10:07:52.472064+0000","last_fullsized":"2024-04-25T10:07:52.472064+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471748+0000","last_change":"2024-04-25T10:07:51.315131+0000","last_active":"2024-04-25T10:07:52.471748+0000","last_peered":"2024-04-25T10:07:52.471748+0000","last_clean":"2024-04-25T10:07:52.471748+0000","last_became_active":"2024-04-25T10:07:51.314869+0000","last_became_peered":"2024-04-25T10:07:51.314869+0000","last_unstale":"2024-04-25T10:07:52.471748+0000","last_undegraded":"2024-04-25T10:07:52.471748+0000","last_fullsized":"2024-04-25T10:07:52.471748+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:38.125900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470577+0000","last_change":"2024-04-25T10:07:51.303339+0000","last_active":"2024-04-25T10:07:52.470577+0000","last_peered":"2024-04-25T10:07:52.470577+0000","last_clean":"2024-04-25T10:07:52.470577+0000","last_became_active":"2024-04-25T10:07:51.303082+0000","last_became_peered":"2024-04-25T10:07:51.303082+0000","last_unstale":"2024-04-25T10:07:52.470577+0000","last_undegraded":"2024-04-25T10:07:52.470577+0000","last_fullsized":"2024-04-25T10:07:52.470577+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:16.208395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470265+0000","last_change":"2024-04-25T10:07:51.318977+0000","last_active":"2024-04-25T10:07:52.470265+0000","last_peered":"2024-04-25T10:07:52.470265+0000","last_clean":"2024-04-25T10:07:52.470265+0000","last_became_active":"2024-04-25T10:07:51.318704+0000","last_became_peered":"2024-04-25T10:07:51.318704+0000","last_unstale":"2024-04-25T10:07:52.470265+0000","last_undegraded":"2024-04-25T10:07:52.470265+0000","last_fullsized":"2024-04-25T10:07:52.470265+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:01.127872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471710+0000","last_change":"2024-04-25T10:07:51.309203+0000","last_active":"2024-04-25T10:07:52.471710+0000","last_peered":"2024-04-25T10:07:52.471710+0000","last_clean":"2024-04-25T10:07:52.471710+0000","last_became_active":"2024-04-25T10:07:51.308918+0000","last_became_peered":"2024-04-25T10:07:51.308918+0000","last_unstale":"2024-04-25T10:07:52.471710+0000","last_undegraded":"2024-04-25T10:07:52.471710+0000","last_fullsized":"2024-04-25T10:07:52.471710+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470720+0000","last_change":"2024-04-25T10:07:51.317240+0000","last_active":"2024-04-25T10:07:52.470720+0000","last_peered":"2024-04-25T10:07:52.470720+0000","last_clean":"2024-04-25T10:07:52.470720+0000","last_became_active":"2024-04-25T10:07:51.316735+0000","last_became_peered":"2024-04-25T10:07:51.316735+0000","last_unstale":"2024-04-25T10:07:52.470720+0000","last_undegraded":"2024-04-25T10:07:52.470720+0000","last_fullsized":"2024-04-25T10:07:52.470720+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:49.018249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470088+0000","last_change":"2024-04-25T10:07:51.310589+0000","last_active":"2024-04-25T10:07:52.470088+0000","last_peered":"2024-04-25T10:07:52.470088+0000","last_clean":"2024-04-25T10:07:52.470088+0000","last_became_active":"2024-04-25T10:07:51.310211+0000","last_became_peered":"2024-04-25T10:07:51.310211+0000","last_unstale":"2024-04-25T10:07:52.470088+0000","last_undegraded":"2024-04-25T10:07:52.470088+0000","last_fullsized":"2024-04-25T10:07:52.470088+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:16.223783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296172+0000","last_change":"2024-04-25T10:07:51.312254+0000","last_active":"2024-04-25T10:07:52.296172+0000","last_peered":"2024-04-25T10:07:52.296172+0000","last_clean":"2024-04-25T10:07:52.296172+0000","last_became_active":"2024-04-25T10:07:51.311901+0000","last_became_peered":"2024-04-25T10:07:51.311901+0000","last_unstale":"2024-04-25T10:07:52.296172+0000","last_undegraded":"2024-04-25T10:07:52.296172+0000","last_fullsized":"2024-04-25T10:07:52.296172+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:42.400180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470490+0000","last_change":"2024-04-25T10:07:51.317615+0000","last_active":"2024-04-25T10:07:52.470490+0000","last_peered":"2024-04-25T10:07:52.470490+0000","last_clean":"2024-04-25T10:07:52.470490+0000","last_became_active":"2024-04-25T10:07:51.317131+0000","last_became_peered":"2024-04-25T10:07:51.317131+0000","last_unstale":"2024-04-25T10:07:52.470490+0000","last_undegraded":"2024-04-25T10:07:52.470490+0000","last_fullsized":"2024-04-25T10:07:52.470490+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:19.853622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296651+0000","last_change":"2024-04-25T10:07:51.323605+0000","last_active":"2024-04-25T10:07:52.296651+0000","last_peered":"2024-04-25T10:07:52.296651+0000","last_clean":"2024-04-25T10:07:52.296651+0000","last_became_active":"2024-04-25T10:07:51.323306+0000","last_became_peered":"2024-04-25T10:07:51.323306+0000","last_unstale":"2024-04-25T10:07:52.296651+0000","last_undegraded":"2024-04-25T10:07:52.296651+0000","last_fullsized":"2024-04-25T10:07:52.296651+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.168043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296163+0000","last_change":"2024-04-25T10:07:51.312425+0000","last_active":"2024-04-25T10:07:52.296163+0000","last_peered":"2024-04-25T10:07:52.296163+0000","last_clean":"2024-04-25T10:07:52.296163+0000","last_became_active":"2024-04-25T10:07:51.312133+0000","last_became_peered":"2024-04-25T10:07:51.312133+0000","last_unstale":"2024-04-25T10:07:52.296163+0000","last_undegraded":"2024-04-25T10:07:52.296163+0000","last_fullsized":"2024-04-25T10:07:52.296163+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.404296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294383+0000","last_change":"2024-04-25T10:07:51.317166+0000","last_active":"2024-04-25T10:07:52.294383+0000","last_peered":"2024-04-25T10:07:52.294383+0000","last_clean":"2024-04-25T10:07:52.294383+0000","last_became_active":"2024-04-25T10:07:51.315513+0000","last_became_peered":"2024-04-25T10:07:51.315513+0000","last_unstale":"2024-04-25T10:07:52.294383+0000","last_undegraded":"2024-04-25T10:07:52.294383+0000","last_fullsized":"2024-04-25T10:07:52.294383+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:51.296503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470214+0000","last_change":"2024-04-25T10:07:51.313310+0000","last_active":"2024-04-25T10:07:52.470214+0000","last_peered":"2024-04-25T10:07:52.470214+0000","last_clean":"2024-04-25T10:07:52.470214+0000","last_became_active":"2024-04-25T10:07:51.313001+0000","last_became_peered":"2024-04-25T10:07:51.313001+0000","last_unstale":"2024-04-25T10:07:52.470214+0000","last_undegraded":"2024-04-25T10:07:52.470214+0000","last_fullsized":"2024-04-25T10:07:52.470214+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:13.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294437+0000","last_change":"2024-04-25T10:07:51.317685+0000","last_active":"2024-04-25T10:07:52.294437+0000","last_peered":"2024-04-25T10:07:52.294437+0000","last_clean":"2024-04-25T10:07:52.294437+0000","last_became_active":"2024-04-25T10:07:51.317386+0000","last_became_peered":"2024-04-25T10:07:51.317386+0000","last_unstale":"2024-04-25T10:07:52.294437+0000","last_undegraded":"2024-04-25T10:07:52.294437+0000","last_fullsized":"2024-04-25T10:07:52.294437+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:16.208395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470284+0000","last_change":"2024-04-25T10:07:51.306911+0000","last_active":"2024-04-25T10:07:52.470284+0000","last_peered":"2024-04-25T10:07:52.470284+0000","last_clean":"2024-04-25T10:07:52.470284+0000","last_became_active":"2024-04-25T10:07:51.306636+0000","last_became_peered":"2024-04-25T10:07:51.306636+0000","last_unstale":"2024-04-25T10:07:52.470284+0000","last_undegraded":"2024-04-25T10:07:52.470284+0000","last_fullsized":"2024-04-25T10:07:52.470284+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:57.699968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.472107+0000","last_change":"2024-04-25T10:07:51.304905+0000","last_active":"2024-04-25T10:07:52.472107+0000","last_peered":"2024-04-25T10:07:52.472107+0000","last_clean":"2024-04-25T10:07:52.472107+0000","last_became_active":"2024-04-25T10:07:51.304584+0000","last_became_peered":"2024-04-25T10:07:51.304584+0000","last_unstale":"2024-04-25T10:07:52.472107+0000","last_undegraded":"2024-04-25T10:07:52.472107+0000","last_fullsized":"2024-04-25T10:07:52.472107+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470893+0000","last_change":"2024-04-25T10:07:51.313605+0000","last_active":"2024-04-25T10:07:52.470893+0000","last_peered":"2024-04-25T10:07:52.470893+0000","last_clean":"2024-04-25T10:07:52.470893+0000","last_became_active":"2024-04-25T10:07:51.313265+0000","last_became_peered":"2024-04-25T10:07:51.313265+0000","last_unstale":"2024-04-25T10:07:52.470893+0000","last_undegraded":"2024-04-25T10:07:52.470893+0000","last_fullsized":"2024-04-25T10:07:52.470893+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471365+0000","last_change":"2024-04-25T10:07:51.312675+0000","last_active":"2024-04-25T10:07:52.471365+0000","last_peered":"2024-04-25T10:07:52.471365+0000","last_clean":"2024-04-25T10:07:52.471365+0000","last_became_active":"2024-04-25T10:07:51.311781+0000","last_became_peered":"2024-04-25T10:07:51.311781+0000","last_unstale":"2024-04-25T10:07:52.471365+0000","last_undegraded":"2024-04-25T10:07:52.471365+0000","last_fullsized":"2024-04-25T10:07:52.471365+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.404296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296412+0000","last_change":"2024-04-25T10:07:51.322910+0000","last_active":"2024-04-25T10:07:52.296412+0000","last_peered":"2024-04-25T10:07:52.296412+0000","last_clean":"2024-04-25T10:07:52.296412+0000","last_became_active":"2024-04-25T10:07:51.322604+0000","last_became_peered":"2024-04-25T10:07:51.322604+0000","last_unstale":"2024-04-25T10:07:52.296412+0000","last_undegraded":"2024-04-25T10:07:52.296412+0000","last_fullsized":"2024-04-25T10:07:52.296412+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294454+0000","last_change":"2024-04-25T10:07:51.316521+0000","last_active":"2024-04-25T10:07:52.294454+0000","last_peered":"2024-04-25T10:07:52.294454+0000","last_clean":"2024-04-25T10:07:52.294454+0000","last_became_active":"2024-04-25T10:07:51.316238+0000","last_became_peered":"2024-04-25T10:07:51.316238+0000","last_unstale":"2024-04-25T10:07:52.294454+0000","last_undegraded":"2024-04-25T10:07:52.294454+0000","last_fullsized":"2024-04-25T10:07:52.294454+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296471+0000","last_change":"2024-04-25T10:07:51.312907+0000","last_active":"2024-04-25T10:07:52.296471+0000","last_peered":"2024-04-25T10:07:52.296471+0000","last_clean":"2024-04-25T10:07:52.296471+0000","last_became_active":"2024-04-25T10:07:51.312595+0000","last_became_peered":"2024-04-25T10:07:51.312595+0000","last_unstale":"2024-04-25T10:07:52.296471+0000","last_undegraded":"2024-04-25T10:07:52.296471+0000","last_fullsized":"2024-04-25T10:07:52.296471+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470232+0000","last_change":"2024-04-25T10:07:51.309738+0000","last_active":"2024-04-25T10:07:52.470232+0000","last_peered":"2024-04-25T10:07:52.470232+0000","last_clean":"2024-04-25T10:07:52.470232+0000","last_became_active":"2024-04-25T10:07:51.309394+0000","last_became_peered":"2024-04-25T10:07:51.309394+0000","last_unstale":"2024-04-25T10:07:52.470232+0000","last_undegraded":"2024-04-25T10:07:52.470232+0000","last_fullsized":"2024-04-25T10:07:52.470232+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:51.800823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295906+0000","last_change":"2024-04-25T10:07:51.311322+0000","last_active":"2024-04-25T10:07:52.295906+0000","last_peered":"2024-04-25T10:07:52.295906+0000","last_clean":"2024-04-25T10:07:52.295906+0000","last_became_active":"2024-04-25T10:07:51.311041+0000","last_became_peered":"2024-04-25T10:07:51.311041+0000","last_unstale":"2024-04-25T10:07:52.295906+0000","last_undegraded":"2024-04-25T10:07:52.295906+0000","last_fullsized":"2024-04-25T10:07:52.295906+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:05.899651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471407+0000","last_change":"2024-04-25T10:07:51.317497+0000","last_active":"2024-04-25T10:07:52.471407+0000","last_peered":"2024-04-25T10:07:52.471407+0000","last_clean":"2024-04-25T10:07:52.471407+0000","last_became_active":"2024-04-25T10:07:51.317196+0000","last_became_peered":"2024-04-25T10:07:51.317196+0000","last_unstale":"2024-04-25T10:07:52.471407+0000","last_undegraded":"2024-04-25T10:07:52.471407+0000","last_fullsized":"2024-04-25T10:07:52.471407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:18.847819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294675+0000","last_change":"2024-04-25T10:07:51.324223+0000","last_active":"2024-04-25T10:07:52.294675+0000","last_peered":"2024-04-25T10:07:52.294675+0000","last_clean":"2024-04-25T10:07:52.294675+0000","last_became_active":"2024-04-25T10:07:51.323857+0000","last_became_peered":"2024-04-25T10:07:51.323857+0000","last_unstale":"2024-04-25T10:07:52.294675+0000","last_undegraded":"2024-04-25T10:07:52.294675+0000","last_fullsized":"2024-04-25T10:07:52.294675+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471513+0000","last_change":"2024-04-25T10:07:51.319543+0000","last_active":"2024-04-25T10:07:52.471513+0000","last_peered":"2024-04-25T10:07:52.471513+0000","last_clean":"2024-04-25T10:07:52.471513+0000","last_became_active":"2024-04-25T10:07:51.319265+0000","last_became_peered":"2024-04-25T10:07:51.319265+0000","last_unstale":"2024-04-25T10:07:52.471513+0000","last_undegraded":"2024-04-25T10:07:52.471513+0000","last_fullsized":"2024-04-25T10:07:52.471513+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:42.400180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470296+0000","last_change":"2024-04-25T10:07:51.318244+0000","last_active":"2024-04-25T10:07:52.470296+0000","last_peered":"2024-04-25T10:07:52.470296+0000","last_clean":"2024-04-25T10:07:52.470296+0000","last_became_active":"2024-04-25T10:07:51.317253+0000","last_became_peered":"2024-04-25T10:07:51.317253+0000","last_unstale":"2024-04-25T10:07:52.470296+0000","last_undegraded":"2024-04-25T10:07:52.470296+0000","last_fullsized":"2024-04-25T10:07:52.470296+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:17.235813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470996+0000","last_change":"2024-04-25T10:07:51.313123+0000","last_active":"2024-04-25T10:07:52.470996+0000","last_peered":"2024-04-25T10:07:52.470996+0000","last_clean":"2024-04-25T10:07:52.470996+0000","last_became_active":"2024-04-25T10:07:51.312765+0000","last_became_peered":"2024-04-25T10:07:51.312765+0000","last_unstale":"2024-04-25T10:07:52.470996+0000","last_undegraded":"2024-04-25T10:07:52.470996+0000","last_fullsized":"2024-04-25T10:07:52.470996+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:21.727353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296245+0000","last_change":"2024-04-25T10:07:51.312907+0000","last_active":"2024-04-25T10:07:52.296245+0000","last_peered":"2024-04-25T10:07:52.296245+0000","last_clean":"2024-04-25T10:07:52.296245+0000","last_became_active":"2024-04-25T10:07:51.312558+0000","last_became_peered":"2024-04-25T10:07:51.312558+0000","last_unstale":"2024-04-25T10:07:52.296245+0000","last_undegraded":"2024-04-25T10:07:52.296245+0000","last_fullsized":"2024-04-25T10:07:52.296245+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471064+0000","last_change":"2024-04-25T10:07:51.298328+0000","last_active":"2024-04-25T10:07:52.471064+0000","last_peered":"2024-04-25T10:07:52.471064+0000","last_clean":"2024-04-25T10:07:52.471064+0000","last_became_active":"2024-04-25T10:07:51.297878+0000","last_became_peered":"2024-04-25T10:07:51.297878+0000","last_unstale":"2024-04-25T10:07:52.471064+0000","last_undegraded":"2024-04-25T10:07:52.471064+0000","last_fullsized":"2024-04-25T10:07:52.471064+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:51.296503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296580+0000","last_change":"2024-04-25T10:07:50.306577+0000","last_active":"2024-04-25T10:07:52.296580+0000","last_peered":"2024-04-25T10:07:52.296580+0000","last_clean":"2024-04-25T10:07:52.296580+0000","last_became_active":"2024-04-25T10:07:50.306291+0000","last_became_peered":"2024-04-25T10:07:50.306291+0000","last_unstale":"2024-04-25T10:07:52.296580+0000","last_undegraded":"2024-04-25T10:07:52.296580+0000","last_fullsized":"2024-04-25T10:07:52.296580+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:23.427143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.472178+0000","last_change":"2024-04-25T10:07:51.316840+0000","last_active":"2024-04-25T10:07:52.472178+0000","last_peered":"2024-04-25T10:07:52.472178+0000","last_clean":"2024-04-25T10:07:52.472178+0000","last_became_active":"2024-04-25T10:07:51.316519+0000","last_became_peered":"2024-04-25T10:07:51.316519+0000","last_unstale":"2024-04-25T10:07:52.472178+0000","last_undegraded":"2024-04-25T10:07:52.472178+0000","last_fullsized":"2024-04-25T10:07:52.472178+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.168043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471882+0000","last_change":"2024-04-25T10:07:50.306264+0000","last_active":"2024-04-25T10:07:52.471882+0000","last_peered":"2024-04-25T10:07:52.471882+0000","last_clean":"2024-04-25T10:07:52.471882+0000","last_became_active":"2024-04-25T10:07:50.305960+0000","last_became_peered":"2024-04-25T10:07:50.305960+0000","last_unstale":"2024-04-25T10:07:52.471882+0000","last_undegraded":"2024-04-25T10:07:52.471882+0000","last_fullsized":"2024-04-25T10:07:52.471882+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:02.311727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471542+0000","last_change":"2024-04-25T10:07:50.311222+0000","last_active":"2024-04-25T10:07:52.471542+0000","last_peered":"2024-04-25T10:07:52.471542+0000","last_clean":"2024-04-25T10:07:52.471542+0000","last_became_active":"2024-04-25T10:07:50.310954+0000","last_became_peered":"2024-04-25T10:07:50.310954+0000","last_unstale":"2024-04-25T10:07:52.471542+0000","last_undegraded":"2024-04-25T10:07:52.471542+0000","last_fullsized":"2024-04-25T10:07:52.471542+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:44.898508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471321+0000","last_change":"2024-04-25T10:07:51.312039+0000","last_active":"2024-04-25T10:07:52.471321+0000","last_peered":"2024-04-25T10:07:52.471321+0000","last_clean":"2024-04-25T10:07:52.471321+0000","last_became_active":"2024-04-25T10:07:51.305475+0000","last_became_peered":"2024-04-25T10:07:51.305475+0000","last_unstale":"2024-04-25T10:07:52.471321+0000","last_undegraded":"2024-04-25T10:07:52.471321+0000","last_fullsized":"2024-04-25T10:07:52.471321+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470435+0000","last_change":"2024-04-25T10:07:50.309547+0000","last_active":"2024-04-25T10:07:52.470435+0000","last_peered":"2024-04-25T10:07:52.470435+0000","last_clean":"2024-04-25T10:07:52.470435+0000","last_became_active":"2024-04-25T10:07:50.309240+0000","last_became_peered":"2024-04-25T10:07:50.309240+0000","last_unstale":"2024-04-25T10:07:52.470435+0000","last_undegraded":"2024-04-25T10:07:52.470435+0000","last_fullsized":"2024-04-25T10:07:52.470435+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:23.723920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470369+0000","last_change":"2024-04-25T10:07:50.306966+0000","last_active":"2024-04-25T10:07:52.470369+0000","last_peered":"2024-04-25T10:07:52.470369+0000","last_clean":"2024-04-25T10:07:52.470369+0000","last_became_active":"2024-04-25T10:07:50.306613+0000","last_became_peered":"2024-04-25T10:07:50.306613+0000","last_unstale":"2024-04-25T10:07:52.470369+0000","last_undegraded":"2024-04-25T10:07:52.470369+0000","last_fullsized":"2024-04-25T10:07:52.470369+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:16.044211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296418+0000","last_change":"2024-04-25T10:07:51.319075+0000","last_active":"2024-04-25T10:07:52.296418+0000","last_peered":"2024-04-25T10:07:52.296418+0000","last_clean":"2024-04-25T10:07:52.296418+0000","last_became_active":"2024-04-25T10:07:51.318802+0000","last_became_peered":"2024-04-25T10:07:51.318802+0000","last_unstale":"2024-04-25T10:07:52.296418+0000","last_undegraded":"2024-04-25T10:07:52.296418+0000","last_fullsized":"2024-04-25T10:07:52.296418+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470993+0000","last_change":"2024-04-25T10:07:50.311086+0000","last_active":"2024-04-25T10:07:52.470993+0000","last_peered":"2024-04-25T10:07:52.470993+0000","last_clean":"2024-04-25T10:07:52.470993+0000","last_became_active":"2024-04-25T10:07:50.310798+0000","last_became_peered":"2024-04-25T10:07:50.310798+0000","last_unstale":"2024-04-25T10:07:52.470993+0000","last_undegraded":"2024-04-25T10:07:52.470993+0000","last_fullsized":"2024-04-25T10:07:52.470993+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:46.162970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.469999+0000","last_change":"2024-04-25T10:07:51.305304+0000","last_active":"2024-04-25T10:07:52.469999+0000","last_peered":"2024-04-25T10:07:52.469999+0000","last_clean":"2024-04-25T10:07:52.469999+0000","last_became_active":"2024-04-25T10:07:51.304968+0000","last_became_peered":"2024-04-25T10:07:51.304968+0000","last_unstale":"2024-04-25T10:07:52.469999+0000","last_undegraded":"2024-04-25T10:07:52.469999+0000","last_fullsized":"2024-04-25T10:07:52.469999+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:49.018249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295971+0000","last_change":"2024-04-25T10:07:50.305140+0000","last_active":"2024-04-25T10:07:52.295971+0000","last_peered":"2024-04-25T10:07:52.295971+0000","last_clean":"2024-04-25T10:07:52.295971+0000","last_became_active":"2024-04-25T10:07:50.304795+0000","last_became_peered":"2024-04-25T10:07:50.304795+0000","last_unstale":"2024-04-25T10:07:52.295971+0000","last_undegraded":"2024-04-25T10:07:52.295971+0000","last_fullsized":"2024-04-25T10:07:52.295971+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:22.138626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471236+0000","last_change":"2024-04-25T10:07:51.317275+0000","last_active":"2024-04-25T10:07:52.471236+0000","last_peered":"2024-04-25T10:07:52.471236+0000","last_clean":"2024-04-25T10:07:52.471236+0000","last_became_active":"2024-04-25T10:07:51.316910+0000","last_became_peered":"2024-04-25T10:07:51.316910+0000","last_unstale":"2024-04-25T10:07:52.471236+0000","last_undegraded":"2024-04-25T10:07:52.471236+0000","last_fullsized":"2024-04-25T10:07:52.471236+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:16.208395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294270+0000","last_change":"2024-04-25T10:07:50.307893+0000","last_active":"2024-04-25T10:07:52.294270+0000","last_peered":"2024-04-25T10:07:52.294270+0000","last_clean":"2024-04-25T10:07:52.294270+0000","last_became_active":"2024-04-25T10:07:50.307514+0000","last_became_peered":"2024-04-25T10:07:50.307514+0000","last_unstale":"2024-04-25T10:07:52.294270+0000","last_undegraded":"2024-04-25T10:07:52.294270+0000","last_fullsized":"2024-04-25T10:07:52.294270+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:33.584068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294636+0000","last_change":"2024-04-25T10:07:51.317446+0000","last_active":"2024-04-25T10:07:52.294636+0000","last_peered":"2024-04-25T10:07:52.294636+0000","last_clean":"2024-04-25T10:07:52.294636+0000","last_became_active":"2024-04-25T10:07:51.315123+0000","last_became_peered":"2024-04-25T10:07:51.315123+0000","last_unstale":"2024-04-25T10:07:52.294636+0000","last_undegraded":"2024-04-25T10:07:52.294636+0000","last_fullsized":"2024-04-25T10:07:52.294636+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296218+0000","last_change":"2024-04-25T10:07:50.306714+0000","last_active":"2024-04-25T10:07:52.296218+0000","last_peered":"2024-04-25T10:07:52.296218+0000","last_clean":"2024-04-25T10:07:52.296218+0000","last_became_active":"2024-04-25T10:07:50.306439+0000","last_became_peered":"2024-04-25T10:07:50.306439+0000","last_unstale":"2024-04-25T10:07:52.296218+0000","last_undegraded":"2024-04-25T10:07:52.296218+0000","last_fullsized":"2024-04-25T10:07:52.296218+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:48.651672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470854+0000","last_change":"2024-04-25T10:07:51.309229+0000","last_active":"2024-04-25T10:07:52.470854+0000","last_peered":"2024-04-25T10:07:52.470854+0000","last_clean":"2024-04-25T10:07:52.470854+0000","last_became_active":"2024-04-25T10:07:51.308925+0000","last_became_peered":"2024-04-25T10:07:51.308925+0000","last_unstale":"2024-04-25T10:07:52.470854+0000","last_undegraded":"2024-04-25T10:07:52.470854+0000","last_fullsized":"2024-04-25T10:07:52.470854+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.293101+0000","last_change":"2024-04-25T10:07:49.293101+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.293101+0000","last_undegraded":"2024-04-25T10:07:49.293101+0000","last_fullsized":"2024-04-25T10:07:49.293101+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470639+0000","last_change":"2024-04-25T10:07:50.309095+0000","last_active":"2024-04-25T10:07:52.470639+0000","last_peered":"2024-04-25T10:07:52.470639+0000","last_clean":"2024-04-25T10:07:52.470639+0000","last_became_active":"2024-04-25T10:07:50.308756+0000","last_became_peered":"2024-04-25T10:07:50.308756+0000","last_unstale":"2024-04-25T10:07:52.470639+0000","last_undegraded":"2024-04-25T10:07:52.470639+0000","last_fullsized":"2024-04-25T10:07:52.470639+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:04.298496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470541+0000","last_change":"2024-04-25T10:07:51.311605+0000","last_active":"2024-04-25T10:07:52.470541+0000","last_peered":"2024-04-25T10:07:52.470541+0000","last_clean":"2024-04-25T10:07:52.470541+0000","last_became_active":"2024-04-25T10:07:51.311307+0000","last_became_peered":"2024-04-25T10:07:51.311307+0000","last_unstale":"2024-04-25T10:07:52.470541+0000","last_undegraded":"2024-04-25T10:07:52.470541+0000","last_fullsized":"2024-04-25T10:07:52.470541+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:00.278450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294600+0000","last_change":"2024-04-25T10:07:50.306790+0000","last_active":"2024-04-25T10:07:52.294600+0000","last_peered":"2024-04-25T10:07:52.294600+0000","last_clean":"2024-04-25T10:07:52.294600+0000","last_became_active":"2024-04-25T10:07:50.306491+0000","last_became_peered":"2024-04-25T10:07:50.306491+0000","last_unstale":"2024-04-25T10:07:52.294600+0000","last_undegraded":"2024-04-25T10:07:52.294600+0000","last_fullsized":"2024-04-25T10:07:52.294600+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:43.478815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295998+0000","last_change":"2024-04-25T10:07:51.306158+0000","last_active":"2024-04-25T10:07:52.295998+0000","last_peered":"2024-04-25T10:07:52.295998+0000","last_clean":"2024-04-25T10:07:52.295998+0000","last_became_active":"2024-04-25T10:07:51.305824+0000","last_became_peered":"2024-04-25T10:07:51.305824+0000","last_unstale":"2024-04-25T10:07:52.295998+0000","last_undegraded":"2024-04-25T10:07:52.295998+0000","last_fullsized":"2024-04-25T10:07:52.295998+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:17.474618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292461+0000","last_change":"2024-04-25T10:07:49.292461+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292461+0000","last_undegraded":"2024-04-25T10:07:49.292461+0000","last_fullsized":"2024-04-25T10:07:49.292461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470269+0000","last_change":"2024-04-25T10:07:51.307313+0000","last_active":"2024-04-25T10:07:52.470269+0000","last_peered":"2024-04-25T10:07:52.470269+0000","last_clean":"2024-04-25T10:07:52.470269+0000","last_became_active":"2024-04-25T10:07:51.307032+0000","last_became_peered":"2024-04-25T10:07:51.307032+0000","last_unstale":"2024-04-25T10:07:52.470269+0000","last_undegraded":"2024-04-25T10:07:52.470269+0000","last_fullsized":"2024-04-25T10:07:52.470269+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:51.296503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470393+0000","last_change":"2024-04-25T10:07:50.304944+0000","last_active":"2024-04-25T10:07:52.470393+0000","last_peered":"2024-04-25T10:07:52.470393+0000","last_clean":"2024-04-25T10:07:52.470393+0000","last_became_active":"2024-04-25T10:07:50.304542+0000","last_became_peered":"2024-04-25T10:07:50.304542+0000","last_unstale":"2024-04-25T10:07:52.470393+0000","last_undegraded":"2024-04-25T10:07:52.470393+0000","last_fullsized":"2024-04-25T10:07:52.470393+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:39.934850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295938+0000","last_change":"2024-04-25T10:07:51.320164+0000","last_active":"2024-04-25T10:07:52.295938+0000","last_peered":"2024-04-25T10:07:52.295938+0000","last_clean":"2024-04-25T10:07:52.295938+0000","last_became_active":"2024-04-25T10:07:51.319859+0000","last_became_peered":"2024-04-25T10:07:51.319859+0000","last_unstale":"2024-04-25T10:07:52.295938+0000","last_undegraded":"2024-04-25T10:07:52.295938+0000","last_fullsized":"2024-04-25T10:07:52.295938+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:21.727353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470461+0000","last_change":"2024-04-25T10:07:50.309933+0000","last_active":"2024-04-25T10:07:52.470461+0000","last_peered":"2024-04-25T10:07:52.470461+0000","last_clean":"2024-04-25T10:07:52.470461+0000","last_became_active":"2024-04-25T10:07:50.309637+0000","last_became_peered":"2024-04-25T10:07:50.309637+0000","last_unstale":"2024-04-25T10:07:52.470461+0000","last_undegraded":"2024-04-25T10:07:52.470461+0000","last_fullsized":"2024-04-25T10:07:52.470461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:09.460489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470368+0000","last_change":"2024-04-25T10:07:51.308930+0000","last_active":"2024-04-25T10:07:52.470368+0000","last_peered":"2024-04-25T10:07:52.470368+0000","last_clean":"2024-04-25T10:07:52.470368+0000","last_became_active":"2024-04-25T10:07:51.308603+0000","last_became_peered":"2024-04-25T10:07:51.308603+0000","last_unstale":"2024-04-25T10:07:52.470368+0000","last_undegraded":"2024-04-25T10:07:52.470368+0000","last_fullsized":"2024-04-25T10:07:52.470368+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:17.724785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470020+0000","last_change":"2024-04-25T10:07:50.304956+0000","last_active":"2024-04-25T10:07:52.470020+0000","last_peered":"2024-04-25T10:07:52.470020+0000","last_clean":"2024-04-25T10:07:52.470020+0000","last_became_active":"2024-04-25T10:07:50.304634+0000","last_became_peered":"2024-04-25T10:07:50.304634+0000","last_unstale":"2024-04-25T10:07:52.470020+0000","last_undegraded":"2024-04-25T10:07:52.470020+0000","last_fullsized":"2024-04-25T10:07:52.470020+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:28.896776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470917+0000","last_change":"2024-04-25T10:07:51.314008+0000","last_active":"2024-04-25T10:07:52.470917+0000","last_peered":"2024-04-25T10:07:52.470917+0000","last_clean":"2024-04-25T10:07:52.470917+0000","last_became_active":"2024-04-25T10:07:51.313611+0000","last_became_peered":"2024-04-25T10:07:51.313611+0000","last_unstale":"2024-04-25T10:07:52.470917+0000","last_undegraded":"2024-04-25T10:07:52.470917+0000","last_fullsized":"2024-04-25T10:07:52.470917+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471515+0000","last_change":"2024-04-25T10:07:50.313909+0000","last_active":"2024-04-25T10:07:52.471515+0000","last_peered":"2024-04-25T10:07:52.471515+0000","last_clean":"2024-04-25T10:07:52.471515+0000","last_became_active":"2024-04-25T10:07:50.313400+0000","last_became_peered":"2024-04-25T10:07:50.313400+0000","last_unstale":"2024-04-25T10:07:52.471515+0000","last_undegraded":"2024-04-25T10:07:52.471515+0000","last_fullsized":"2024-04-25T10:07:52.471515+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:09.642875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471017+0000","last_change":"2024-04-25T10:07:51.310494+0000","last_active":"2024-04-25T10:07:52.471017+0000","last_peered":"2024-04-25T10:07:52.471017+0000","last_clean":"2024-04-25T10:07:52.471017+0000","last_became_active":"2024-04-25T10:07:51.310224+0000","last_became_peered":"2024-04-25T10:07:51.310224+0000","last_unstale":"2024-04-25T10:07:52.471017+0000","last_undegraded":"2024-04-25T10:07:52.471017+0000","last_fullsized":"2024-04-25T10:07:52.471017+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292480+0000","last_change":"2024-04-25T10:07:49.292480+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292480+0000","last_undegraded":"2024-04-25T10:07:49.292480+0000","last_fullsized":"2024-04-25T10:07:49.292480+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471787+0000","last_change":"2024-04-25T10:07:51.318600+0000","last_active":"2024-04-25T10:07:52.471787+0000","last_peered":"2024-04-25T10:07:52.471787+0000","last_clean":"2024-04-25T10:07:52.471787+0000","last_became_active":"2024-04-25T10:07:51.318304+0000","last_became_peered":"2024-04-25T10:07:51.318304+0000","last_unstale":"2024-04-25T10:07:52.471787+0000","last_undegraded":"2024-04-25T10:07:52.471787+0000","last_fullsized":"2024-04-25T10:07:52.471787+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:53.592626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470762+0000","last_change":"2024-04-25T10:07:50.304446+0000","last_active":"2024-04-25T10:07:52.470762+0000","last_peered":"2024-04-25T10:07:52.470762+0000","last_clean":"2024-04-25T10:07:52.470762+0000","last_became_active":"2024-04-25T10:07:50.303898+0000","last_became_peered":"2024-04-25T10:07:50.303898+0000","last_unstale":"2024-04-25T10:07:52.470762+0000","last_undegraded":"2024-04-25T10:07:52.470762+0000","last_fullsized":"2024-04-25T10:07:52.470762+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:23.749111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470626+0000","last_change":"2024-04-25T10:07:51.319204+0000","last_active":"2024-04-25T10:07:52.470626+0000","last_peered":"2024-04-25T10:07:52.470626+0000","last_clean":"2024-04-25T10:07:52.470626+0000","last_became_active":"2024-04-25T10:07:51.318367+0000","last_became_peered":"2024-04-25T10:07:51.318367+0000","last_unstale":"2024-04-25T10:07:52.470626+0000","last_undegraded":"2024-04-25T10:07:52.470626+0000","last_fullsized":"2024-04-25T10:07:52.470626+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:46.811230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292499+0000","last_change":"2024-04-25T10:07:49.292499+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292499+0000","last_undegraded":"2024-04-25T10:07:49.292499+0000","last_fullsized":"2024-04-25T10:07:49.292499+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296169+0000","last_change":"2024-04-25T10:07:51.312843+0000","last_active":"2024-04-25T10:07:52.296169+0000","last_peered":"2024-04-25T10:07:52.296169+0000","last_clean":"2024-04-25T10:07:52.296169+0000","last_became_active":"2024-04-25T10:07:51.311808+0000","last_became_peered":"2024-04-25T10:07:51.311808+0000","last_unstale":"2024-04-25T10:07:52.296169+0000","last_undegraded":"2024-04-25T10:07:52.296169+0000","last_fullsized":"2024-04-25T10:07:52.296169+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:18.847819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296623+0000","last_change":"2024-04-25T10:07:50.306664+0000","last_active":"2024-04-25T10:07:52.296623+0000","last_peered":"2024-04-25T10:07:52.296623+0000","last_clean":"2024-04-25T10:07:52.296623+0000","last_became_active":"2024-04-25T10:07:50.306383+0000","last_became_peered":"2024-04-25T10:07:50.306383+0000","last_unstale":"2024-04-25T10:07:52.296623+0000","last_undegraded":"2024-04-25T10:07:52.296623+0000","last_fullsized":"2024-04-25T10:07:52.296623+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:10.698506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292457+0000","last_change":"2024-04-25T10:07:49.292457+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292457+0000","last_undegraded":"2024-04-25T10:07:49.292457+0000","last_fullsized":"2024-04-25T10:07:49.292457+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296086+0000","last_change":"2024-04-25T10:07:50.311607+0000","last_active":"2024-04-25T10:07:52.296086+0000","last_peered":"2024-04-25T10:07:52.296086+0000","last_clean":"2024-04-25T10:07:52.296086+0000","last_became_active":"2024-04-25T10:07:50.311300+0000","last_became_peered":"2024-04-25T10:07:50.311300+0000","last_unstale":"2024-04-25T10:07:52.296086+0000","last_undegraded":"2024-04-25T10:07:52.296086+0000","last_fullsized":"2024-04-25T10:07:52.296086+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.109707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471043+0000","last_change":"2024-04-25T10:07:51.314396+0000","last_active":"2024-04-25T10:07:52.471043+0000","last_peered":"2024-04-25T10:07:52.471043+0000","last_clean":"2024-04-25T10:07:52.471043+0000","last_became_active":"2024-04-25T10:07:51.314116+0000","last_became_peered":"2024-04-25T10:07:51.314116+0000","last_unstale":"2024-04-25T10:07:52.471043+0000","last_undegraded":"2024-04-25T10:07:52.471043+0000","last_fullsized":"2024-04-25T10:07:52.471043+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:19.021120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471632+0000","last_change":"2024-04-25T10:07:50.307927+0000","last_active":"2024-04-25T10:07:52.471632+0000","last_peered":"2024-04-25T10:07:52.471632+0000","last_clean":"2024-04-25T10:07:52.471632+0000","last_became_active":"2024-04-25T10:07:50.307641+0000","last_became_peered":"2024-04-25T10:07:50.307641+0000","last_unstale":"2024-04-25T10:07:52.471632+0000","last_undegraded":"2024-04-25T10:07:52.471632+0000","last_fullsized":"2024-04-25T10:07:52.471632+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:22.138626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471021+0000","last_change":"2024-04-25T10:07:51.311790+0000","last_active":"2024-04-25T10:07:52.471021+0000","last_peered":"2024-04-25T10:07:52.471021+0000","last_clean":"2024-04-25T10:07:52.471021+0000","last_became_active":"2024-04-25T10:07:51.311524+0000","last_became_peered":"2024-04-25T10:07:51.311524+0000","last_unstale":"2024-04-25T10:07:52.471021+0000","last_undegraded":"2024-04-25T10:07:52.471021+0000","last_fullsized":"2024-04-25T10:07:52.471021+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.168043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470815+0000","last_change":"2024-04-25T10:07:50.313145+0000","last_active":"2024-04-25T10:07:52.470815+0000","last_peered":"2024-04-25T10:07:52.470815+0000","last_clean":"2024-04-25T10:07:52.470815+0000","last_became_active":"2024-04-25T10:07:50.312860+0000","last_became_peered":"2024-04-25T10:07:50.312860+0000","last_unstale":"2024-04-25T10:07:52.470815+0000","last_undegraded":"2024-04-25T10:07:52.470815+0000","last_fullsized":"2024-04-25T10:07:52.470815+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:16.044211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471897+0000","last_change":"2024-04-25T10:07:51.322690+0000","last_active":"2024-04-25T10:07:52.471897+0000","last_peered":"2024-04-25T10:07:52.471897+0000","last_clean":"2024-04-25T10:07:52.471897+0000","last_became_active":"2024-04-25T10:07:51.322404+0000","last_became_peered":"2024-04-25T10:07:51.322404+0000","last_unstale":"2024-04-25T10:07:52.471897+0000","last_undegraded":"2024-04-25T10:07:52.471897+0000","last_fullsized":"2024-04-25T10:07:52.471897+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:00.981450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296163+0000","last_change":"2024-04-25T10:07:50.309078+0000","last_active":"2024-04-25T10:07:52.296163+0000","last_peered":"2024-04-25T10:07:52.296163+0000","last_clean":"2024-04-25T10:07:52.296163+0000","last_became_active":"2024-04-25T10:07:50.308794+0000","last_became_peered":"2024-04-25T10:07:50.308794+0000","last_unstale":"2024-04-25T10:07:52.296163+0000","last_undegraded":"2024-04-25T10:07:52.296163+0000","last_fullsized":"2024-04-25T10:07:52.296163+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:09.642875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470834+0000","last_change":"2024-04-25T10:07:50.307963+0000","last_active":"2024-04-25T10:07:52.470834+0000","last_peered":"2024-04-25T10:07:52.470834+0000","last_clean":"2024-04-25T10:07:52.470834+0000","last_became_active":"2024-04-25T10:07:50.307611+0000","last_became_peered":"2024-04-25T10:07:50.307611+0000","last_unstale":"2024-04-25T10:07:52.470834+0000","last_undegraded":"2024-04-25T10:07:52.470834+0000","last_fullsized":"2024-04-25T10:07:52.470834+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:13.364471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470886+0000","last_change":"2024-04-25T10:07:51.316433+0000","last_active":"2024-04-25T10:07:52.470886+0000","last_peered":"2024-04-25T10:07:52.470886+0000","last_clean":"2024-04-25T10:07:52.470886+0000","last_became_active":"2024-04-25T10:07:51.315952+0000","last_became_peered":"2024-04-25T10:07:51.315952+0000","last_unstale":"2024-04-25T10:07:52.470886+0000","last_undegraded":"2024-04-25T10:07:52.470886+0000","last_fullsized":"2024-04-25T10:07:52.470886+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:13.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471022+0000","last_change":"2024-04-25T10:07:50.313990+0000","last_active":"2024-04-25T10:07:52.471022+0000","last_peered":"2024-04-25T10:07:52.471022+0000","last_clean":"2024-04-25T10:07:52.471022+0000","last_became_active":"2024-04-25T10:07:50.313700+0000","last_became_peered":"2024-04-25T10:07:50.313700+0000","last_unstale":"2024-04-25T10:07:52.471022+0000","last_undegraded":"2024-04-25T10:07:52.471022+0000","last_fullsized":"2024-04-25T10:07:52.471022+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:46.602681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294542+0000","last_change":"2024-04-25T10:07:51.315938+0000","last_active":"2024-04-25T10:07:52.294542+0000","last_peered":"2024-04-25T10:07:52.294542+0000","last_clean":"2024-04-25T10:07:52.294542+0000","last_became_active":"2024-04-25T10:07:51.315335+0000","last_became_peered":"2024-04-25T10:07:51.315335+0000","last_unstale":"2024-04-25T10:07:52.294542+0000","last_undegraded":"2024-04-25T10:07:52.294542+0000","last_fullsized":"2024-04-25T10:07:52.294542+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470304+0000","last_change":"2024-04-25T10:07:50.306917+0000","last_active":"2024-04-25T10:07:52.470304+0000","last_peered":"2024-04-25T10:07:52.470304+0000","last_clean":"2024-04-25T10:07:52.470304+0000","last_became_active":"2024-04-25T10:07:50.306549+0000","last_became_peered":"2024-04-25T10:07:50.306549+0000","last_unstale":"2024-04-25T10:07:52.470304+0000","last_undegraded":"2024-04-25T10:07:52.470304+0000","last_fullsized":"2024-04-25T10:07:52.470304+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:06.727495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471599+0000","last_change":"2024-04-25T10:07:50.311119+0000","last_active":"2024-04-25T10:07:52.471599+0000","last_peered":"2024-04-25T10:07:52.471599+0000","last_clean":"2024-04-25T10:07:52.471599+0000","last_became_active":"2024-04-25T10:07:50.310843+0000","last_became_peered":"2024-04-25T10:07:50.310843+0000","last_unstale":"2024-04-25T10:07:52.471599+0000","last_undegraded":"2024-04-25T10:07:52.471599+0000","last_fullsized":"2024-04-25T10:07:52.471599+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:26.803283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471140+0000","last_change":"2024-04-25T10:07:51.316546+0000","last_active":"2024-04-25T10:07:52.471140+0000","last_peered":"2024-04-25T10:07:52.471140+0000","last_clean":"2024-04-25T10:07:52.471140+0000","last_became_active":"2024-04-25T10:07:51.313979+0000","last_became_peered":"2024-04-25T10:07:51.313979+0000","last_unstale":"2024-04-25T10:07:52.471140+0000","last_undegraded":"2024-04-25T10:07:52.471140+0000","last_fullsized":"2024-04-25T10:07:52.471140+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:17.474618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292461+0000","last_change":"2024-04-25T10:07:49.292461+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292461+0000","last_undegraded":"2024-04-25T10:07:49.292461+0000","last_fullsized":"2024-04-25T10:07:49.292461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294591+0000","last_change":"2024-04-25T10:07:50.309026+0000","last_active":"2024-04-25T10:07:52.294591+0000","last_peered":"2024-04-25T10:07:52.294591+0000","last_clean":"2024-04-25T10:07:52.294591+0000","last_became_active":"2024-04-25T10:07:50.308706+0000","last_became_peered":"2024-04-25T10:07:50.308706+0000","last_unstale":"2024-04-25T10:07:52.294591+0000","last_undegraded":"2024-04-25T10:07:52.294591+0000","last_fullsized":"2024-04-25T10:07:52.294591+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:17.725300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":245,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295995+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:07:52.295995+0000","last_peered":"2024-04-25T10:07:52.295995+0000","last_clean":"2024-04-25T10:07:52.295995+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:07:52.295995+0000","last_undegraded":"2024-04-25T10:07:52.295995+0000","last_fullsized":"2024-04-25T10:07:52.295995+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":7},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":288874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":692224,"data_stored":282504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":289398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":692224,"data_stored":282572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":284254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509478,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1298432,"data_stored":879732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":700416,"data_stored":284094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":859800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:54.667 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.799+0000 7ff911132640 1 Processor -- start 2024-04-25T10:07:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.799+0000 7ff911132640 1 -- start start 2024-04-25T10:07:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.800+0000 7ff911132640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff90c0d35b0 0x7ff90c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.800+0000 7ff911132640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90c15e0c0 con 0x7ff90c15dce0 2024-04-25T10:07:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.800+0000 7ff911132640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90c0d2510 con 0x7ff90c0b5990 2024-04-25T10:07:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.800+0000 7ff911132640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff90c0d27f0 con 0x7ff90c0d35b0 2024-04-25T10:07:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.800+0000 7ff90ad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff90c15dce0 0x7ff90c0d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54432/0 (socket says 172.21.15.16:54432) 2024-04-25T10:07:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.800+0000 7ff90ad9d640 1 -- 172.21.15.16:0/3415714447 learned_addr learned my addr 172.21.15.16:0/3415714447 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.800+0000 7ff909d9b640 1 -- 172.21.15.16:0/3415714447 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370250528 0 0) 0x7ff90c15e0c0 con 0x7ff90c15dce0 2024-04-25T10:07:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.800+0000 7ff90a59c640 1 --2- 172.21.15.16:0/3415714447 >> v2:172.21.15.186:3300/0 conn(0x7ff90c0d35b0 0x7ff90c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.800+0000 7ff909d9b640 1 -- 172.21.15.16:0/3415714447 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8f8003520 con 0x7ff90c15dce0 2024-04-25T10:07:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff909d9b640 1 -- 172.21.15.16:0/3415714447 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3974971360 0 0) 0x7ff8f8003520 con 0x7ff90c15dce0 2024-04-25T10:07:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff909d9b640 1 -- 172.21.15.16:0/3415714447 >> v2:172.21.15.186:3300/0 conn(0x7ff90c0d35b0 msgr2=0x7ff90c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff909d9b640 1 --2- 172.21.15.16:0/3415714447 >> v2:172.21.15.186:3300/0 conn(0x7ff90c0d35b0 0x7ff90c0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff909d9b640 1 -- 172.21.15.16:0/3415714447 >> v1:172.21.15.186:6789/0 conn(0x7ff90c0b5990 legacy=0x7ff90c0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff909d9b640 1 -- 172.21.15.16:0/3415714447 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c0d2ad0 con 0x7ff90c15dce0 2024-04-25T10:07:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff909d9b640 1 -- 172.21.15.16:0/3415714447 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7ff904001670 con 0x7ff90c15dce0 2024-04-25T10:07:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff909d9b640 1 -- 172.21.15.16:0/3415714447 >> v1:172.21.15.16:6789/0 conn(0x7ff90c15dce0 legacy=0x7ff90c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff909d9b640 1 --2- 172.21.15.16:0/3415714447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8f80039b0 0x7ff8f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff909d9b640 1 --2- 172.21.15.16:0/3415714447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8f8004800 0x7ff8f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff909d9b640 1 --2- 172.21.15.16:0/3415714447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f80051c0 0x7ff8f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff909d9b640 1 -- 172.21.15.16:0/3415714447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c0d27f0 con 0x7ff8f80039b0 2024-04-25T10:07:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff909d9b640 1 -- 172.21.15.16:0/3415714447 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f800c060 con 0x7ff8f80051c0 2024-04-25T10:07:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff909d9b640 1 -- 172.21.15.16:0/3415714447 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f800c260 con 0x7ff8f8004800 2024-04-25T10:07:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff90ad9d640 1 --2- 172.21.15.16:0/3415714447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8f80039b0 0x7ff8f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff90b59e640 1 --2- 172.21.15.16:0/3415714447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f80051c0 0x7ff8f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff90a59c640 1 --2- 172.21.15.16:0/3415714447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8f8004800 0x7ff8f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff90ad9d640 1 -- 172.21.15.16:0/3415714447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8f8004800 msgr2=0x7ff8f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff90ad9d640 1 --2- 172.21.15.16:0/3415714447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8f8004800 0x7ff8f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff90ad9d640 1 -- 172.21.15.16:0/3415714447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f80051c0 msgr2=0x7ff8f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff90ad9d640 1 --2- 172.21.15.16:0/3415714447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f80051c0 0x7ff8f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff90b59e640 1 -- 172.21.15.16:0/3415714447 reap_dead start 2024-04-25T10:07:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff90ad9d640 1 -- 172.21.15.16:0/3415714447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f8003520 con 0x7ff8f80039b0 2024-04-25T10:07:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.801+0000 7ff90a59c640 1 --2- 172.21.15.16:0/3415714447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8f8004800 0x7ff8f8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.802+0000 7ff90ad9d640 1 --2- 172.21.15.16:0/3415714447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8f80039b0 0x7ff8f8003dd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff90400c240 tx=0x7ff8f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1382cf9e140e3356 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.802+0000 7ff909d9b640 1 -- 172.21.15.16:0/3415714447 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff904007d10 con 0x7ff8f80039b0 2024-04-25T10:07:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.802+0000 7ff909d9b640 1 -- 172.21.15.16:0/3415714447 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff904005ad0 con 0x7ff8f80039b0 2024-04-25T10:07:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.802+0000 7ff909d9b640 1 -- 172.21.15.16:0/3415714447 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff904007070 con 0x7ff8f80039b0 2024-04-25T10:07:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.802+0000 7ff911132640 1 -- 172.21.15.16:0/3415714447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8f80039b0 msgr2=0x7ff8f8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.802+0000 7ff911132640 1 --2- 172.21.15.16:0/3415714447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8f80039b0 0x7ff8f8003dd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff90400c240 tx=0x7ff8f8005830 comp rx=0 tx=0).stop 2024-04-25T10:07:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.803+0000 7ff911132640 1 -- 172.21.15.16:0/3415714447 shutdown_connections 2024-04-25T10:07:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.803+0000 7ff911132640 1 --2- 172.21.15.16:0/3415714447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8f80039b0 0x7ff8f8003dd0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.803+0000 7ff911132640 1 -- 172.21.15.16:0/3415714447 >> 172.21.15.16:0/3415714447 conn(0x7ff90c072900 msgr2=0x7ff90c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.803+0000 7ff911132640 1 -- 172.21.15.16:0/3415714447 shutdown_connections 2024-04-25T10:07:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.803+0000 7ff911132640 1 -- 172.21.15.16:0/3415714447 wait complete. 2024-04-25T10:07:54.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.804+0000 7ff911132640 1 Processor -- start 2024-04-25T10:07:54.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.804+0000 7ff911132640 1 -- start start 2024-04-25T10:07:54.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.805+0000 7ff911132640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f80039b0 0x7ff90c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.805+0000 7ff911132640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff90c0c9570 0x7ff90c0caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.805+0000 7ff911132640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff90c0c99f0 0x7ff90c15b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.805+0000 7ff911132640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c0b9820 con 0x7ff90c0c99f0 2024-04-25T10:07:54.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.805+0000 7ff911132640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c0d3090 con 0x7ff8f80039b0 2024-04-25T10:07:54.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.805+0000 7ff911132640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c0b99a0 con 0x7ff90c0c9570 2024-04-25T10:07:54.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.805+0000 7ff90b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff90c0c99f0 0x7ff90c15b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.805+0000 7ff90b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff90c0c99f0 0x7ff90c15b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53138/0 (socket says 172.21.15.16:53138) 2024-04-25T10:07:54.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.805+0000 7ff90b59e640 1 -- 172.21.15.16:0/4262430299 learned_addr learned my addr 172.21.15.16:0/4262430299 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:54.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.805+0000 7ff90ad9d640 1 --2- 172.21.15.16:0/4262430299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f80039b0 0x7ff90c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.805+0000 7ff90a59c640 1 --2- 172.21.15.16:0/4262430299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff90c0c9570 0x7ff90c0caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.805+0000 7ff90b59e640 1 -- 172.21.15.16:0/4262430299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff90c0c9570 msgr2=0x7ff90c0caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.805+0000 7ff90b59e640 1 --2- 172.21.15.16:0/4262430299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff90c0c9570 0x7ff90c0caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.805+0000 7ff90b59e640 1 -- 172.21.15.16:0/4262430299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f80039b0 msgr2=0x7ff90c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.805+0000 7ff90b59e640 1 --2- 172.21.15.16:0/4262430299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f80039b0 0x7ff90c15ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.805+0000 7ff90b59e640 1 -- 172.21.15.16:0/4262430299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c0cb560 con 0x7ff90c0c99f0 2024-04-25T10:07:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.805+0000 7ff90a59c640 1 --2- 172.21.15.16:0/4262430299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff90c0c9570 0x7ff90c0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.806+0000 7ff90b59e640 1 --2- 172.21.15.16:0/4262430299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff90c0c99f0 0x7ff90c15b520 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff90c0cfb10 tx=0x7ff8fc00fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.806+0000 7ff9037fe640 1 -- 172.21.15.16:0/4262430299 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc01a070 con 0x7ff90c0c99f0 2024-04-25T10:07:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.806+0000 7ff9037fe640 1 -- 172.21.15.16:0/4262430299 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8fc006d60 con 0x7ff90c0c99f0 2024-04-25T10:07:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.806+0000 7ff911132640 1 -- 172.21.15.16:0/4262430299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90c0cb840 con 0x7ff90c0c99f0 2024-04-25T10:07:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.806+0000 7ff9037fe640 1 -- 172.21.15.16:0/4262430299 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc003bb0 con 0x7ff90c0c99f0 2024-04-25T10:07:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.806+0000 7ff911132640 1 -- 172.21.15.16:0/4262430299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff90c15e7a0 con 0x7ff90c0c99f0 2024-04-25T10:07:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.807+0000 7ff911132640 1 -- 172.21.15.16:0/4262430299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff90c15e2b0 con 0x7ff90c0c99f0 2024-04-25T10:07:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.807+0000 7ff9037fe640 1 -- 172.21.15.16:0/4262430299 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff8fc01d070 con 0x7ff90c0c99f0 2024-04-25T10:07:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.808+0000 7ff9037fe640 1 --2- 172.21.15.16:0/4262430299 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff8ec06b4d0 0x7ff8ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.808+0000 7ff9037fe640 1 -- 172.21.15.16:0/4262430299 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8fc0897d0 con 0x7ff90c0c99f0 2024-04-25T10:07:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.808+0000 7ff90ad9d640 1 --2- 172.21.15.16:0/4262430299 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff8ec06b4d0 0x7ff8ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.809+0000 7ff90ad9d640 1 --2- 172.21.15.16:0/4262430299 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff8ec06b4d0 0x7ff8ec06d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff904033000 tx=0x7ff90402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.812+0000 7ff9037fe640 1 -- 172.21.15.16:0/4262430299 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff8fc058650 con 0x7ff90c0c99f0 2024-04-25T10:07:54.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.988+0000 7ff911132640 1 -- 172.21.15.16:0/4262430299 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff90c15e970 con 0x7ff8ec06b4d0 2024-04-25T10:07:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.994+0000 7ff9037fe640 1 -- 172.21.15.16:0/4262430299 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249962 (secure 0 0 0) 0x7ff90c15e970 con 0x7ff8ec06b4d0 2024-04-25T10:07:55.003 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:07:55.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.997+0000 7ff911132640 1 -- 172.21.15.16:0/4262430299 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff8ec06b4d0 msgr2=0x7ff8ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.997+0000 7ff911132640 1 --2- 172.21.15.16:0/4262430299 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff8ec06b4d0 0x7ff8ec06d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff904033000 tx=0x7ff90402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.997+0000 7ff911132640 1 -- 172.21.15.16:0/4262430299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff90c0c99f0 msgr2=0x7ff90c15b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.997+0000 7ff911132640 1 --2- 172.21.15.16:0/4262430299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff90c0c99f0 0x7ff90c15b520 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff90c0cfb10 tx=0x7ff8fc00fc00 comp rx=0 tx=0).stop 2024-04-25T10:07:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.997+0000 7ff911132640 1 -- 172.21.15.16:0/4262430299 shutdown_connections 2024-04-25T10:07:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.997+0000 7ff911132640 1 --2- 172.21.15.16:0/4262430299 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff8ec06b4d0 0x7ff8ec06d990 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.997+0000 7ff911132640 1 --2- 172.21.15.16:0/4262430299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff90c0c99f0 0x7ff90c15b520 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.997+0000 7ff911132640 1 --2- 172.21.15.16:0/4262430299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff90c0c9570 0x7ff90c0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.997+0000 7ff911132640 1 --2- 172.21.15.16:0/4262430299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f80039b0 0x7ff90c15ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.997+0000 7ff911132640 1 -- 172.21.15.16:0/4262430299 >> 172.21.15.16:0/4262430299 conn(0x7ff90c072900 msgr2=0x7ff90c0b47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.997+0000 7ff911132640 1 -- 172.21.15.16:0/4262430299 shutdown_connections 2024-04-25T10:07:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:54.997+0000 7ff911132640 1 -- 172.21.15.16:0/4262430299 wait complete. 2024-04-25T10:07:55.014 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-04-25T10:07:53.818447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":197,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243604,"kb_used_data":6608,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731116,"statfs":{"total":773094113280,"available":772844662784,"internally_reserved":0,"allocated":6766592,"data_stored":3451228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.533513"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.472064+0000","last_change":"2024-04-25T10:07:51.311604+0000","last_active":"2024-04-25T10:07:52.472064+0000","last_peered":"2024-04-25T10:07:52.472064+0000","last_clean":"2024-04-25T10:07:52.472064+0000","last_became_active":"2024-04-25T10:07:51.310395+0000","last_became_peered":"2024-04-25T10:07:51.310395+0000","last_unstale":"2024-04-25T10:07:52.472064+0000","last_undegraded":"2024-04-25T10:07:52.472064+0000","last_fullsized":"2024-04-25T10:07:52.472064+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471748+0000","last_change":"2024-04-25T10:07:51.315131+0000","last_active":"2024-04-25T10:07:52.471748+0000","last_peered":"2024-04-25T10:07:52.471748+0000","last_clean":"2024-04-25T10:07:52.471748+0000","last_became_active":"2024-04-25T10:07:51.314869+0000","last_became_peered":"2024-04-25T10:07:51.314869+0000","last_unstale":"2024-04-25T10:07:52.471748+0000","last_undegraded":"2024-04-25T10:07:52.471748+0000","last_fullsized":"2024-04-25T10:07:52.471748+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:38.125900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470577+0000","last_change":"2024-04-25T10:07:51.303339+0000","last_active":"2024-04-25T10:07:52.470577+0000","last_peered":"2024-04-25T10:07:52.470577+0000","last_clean":"2024-04-25T10:07:52.470577+0000","last_became_active":"2024-04-25T10:07:51.303082+0000","last_became_peered":"2024-04-25T10:07:51.303082+0000","last_unstale":"2024-04-25T10:07:52.470577+0000","last_undegraded":"2024-04-25T10:07:52.470577+0000","last_fullsized":"2024-04-25T10:07:52.470577+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:16.208395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470265+0000","last_change":"2024-04-25T10:07:51.318977+0000","last_active":"2024-04-25T10:07:52.470265+0000","last_peered":"2024-04-25T10:07:52.470265+0000","last_clean":"2024-04-25T10:07:52.470265+0000","last_became_active":"2024-04-25T10:07:51.318704+0000","last_became_peered":"2024-04-25T10:07:51.318704+0000","last_unstale":"2024-04-25T10:07:52.470265+0000","last_undegraded":"2024-04-25T10:07:52.470265+0000","last_fullsized":"2024-04-25T10:07:52.470265+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:01.127872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471710+0000","last_change":"2024-04-25T10:07:51.309203+0000","last_active":"2024-04-25T10:07:52.471710+0000","last_peered":"2024-04-25T10:07:52.471710+0000","last_clean":"2024-04-25T10:07:52.471710+0000","last_became_active":"2024-04-25T10:07:51.308918+0000","last_became_peered":"2024-04-25T10:07:51.308918+0000","last_unstale":"2024-04-25T10:07:52.471710+0000","last_undegraded":"2024-04-25T10:07:52.471710+0000","last_fullsized":"2024-04-25T10:07:52.471710+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470720+0000","last_change":"2024-04-25T10:07:51.317240+0000","last_active":"2024-04-25T10:07:52.470720+0000","last_peered":"2024-04-25T10:07:52.470720+0000","last_clean":"2024-04-25T10:07:52.470720+0000","last_became_active":"2024-04-25T10:07:51.316735+0000","last_became_peered":"2024-04-25T10:07:51.316735+0000","last_unstale":"2024-04-25T10:07:52.470720+0000","last_undegraded":"2024-04-25T10:07:52.470720+0000","last_fullsized":"2024-04-25T10:07:52.470720+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:49.018249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470088+0000","last_change":"2024-04-25T10:07:51.310589+0000","last_active":"2024-04-25T10:07:52.470088+0000","last_peered":"2024-04-25T10:07:52.470088+0000","last_clean":"2024-04-25T10:07:52.470088+0000","last_became_active":"2024-04-25T10:07:51.310211+0000","last_became_peered":"2024-04-25T10:07:51.310211+0000","last_unstale":"2024-04-25T10:07:52.470088+0000","last_undegraded":"2024-04-25T10:07:52.470088+0000","last_fullsized":"2024-04-25T10:07:52.470088+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:16.223783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296172+0000","last_change":"2024-04-25T10:07:51.312254+0000","last_active":"2024-04-25T10:07:52.296172+0000","last_peered":"2024-04-25T10:07:52.296172+0000","last_clean":"2024-04-25T10:07:52.296172+0000","last_became_active":"2024-04-25T10:07:51.311901+0000","last_became_peered":"2024-04-25T10:07:51.311901+0000","last_unstale":"2024-04-25T10:07:52.296172+0000","last_undegraded":"2024-04-25T10:07:52.296172+0000","last_fullsized":"2024-04-25T10:07:52.296172+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:42.400180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470490+0000","last_change":"2024-04-25T10:07:51.317615+0000","last_active":"2024-04-25T10:07:52.470490+0000","last_peered":"2024-04-25T10:07:52.470490+0000","last_clean":"2024-04-25T10:07:52.470490+0000","last_became_active":"2024-04-25T10:07:51.317131+0000","last_became_peered":"2024-04-25T10:07:51.317131+0000","last_unstale":"2024-04-25T10:07:52.470490+0000","last_undegraded":"2024-04-25T10:07:52.470490+0000","last_fullsized":"2024-04-25T10:07:52.470490+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:19.853622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296651+0000","last_change":"2024-04-25T10:07:51.323605+0000","last_active":"2024-04-25T10:07:52.296651+0000","last_peered":"2024-04-25T10:07:52.296651+0000","last_clean":"2024-04-25T10:07:52.296651+0000","last_became_active":"2024-04-25T10:07:51.323306+0000","last_became_peered":"2024-04-25T10:07:51.323306+0000","last_unstale":"2024-04-25T10:07:52.296651+0000","last_undegraded":"2024-04-25T10:07:52.296651+0000","last_fullsized":"2024-04-25T10:07:52.296651+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.168043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296163+0000","last_change":"2024-04-25T10:07:51.312425+0000","last_active":"2024-04-25T10:07:52.296163+0000","last_peered":"2024-04-25T10:07:52.296163+0000","last_clean":"2024-04-25T10:07:52.296163+0000","last_became_active":"2024-04-25T10:07:51.312133+0000","last_became_peered":"2024-04-25T10:07:51.312133+0000","last_unstale":"2024-04-25T10:07:52.296163+0000","last_undegraded":"2024-04-25T10:07:52.296163+0000","last_fullsized":"2024-04-25T10:07:52.296163+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.404296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294383+0000","last_change":"2024-04-25T10:07:51.317166+0000","last_active":"2024-04-25T10:07:52.294383+0000","last_peered":"2024-04-25T10:07:52.294383+0000","last_clean":"2024-04-25T10:07:52.294383+0000","last_became_active":"2024-04-25T10:07:51.315513+0000","last_became_peered":"2024-04-25T10:07:51.315513+0000","last_unstale":"2024-04-25T10:07:52.294383+0000","last_undegraded":"2024-04-25T10:07:52.294383+0000","last_fullsized":"2024-04-25T10:07:52.294383+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:51.296503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470214+0000","last_change":"2024-04-25T10:07:51.313310+0000","last_active":"2024-04-25T10:07:52.470214+0000","last_peered":"2024-04-25T10:07:52.470214+0000","last_clean":"2024-04-25T10:07:52.470214+0000","last_became_active":"2024-04-25T10:07:51.313001+0000","last_became_peered":"2024-04-25T10:07:51.313001+0000","last_unstale":"2024-04-25T10:07:52.470214+0000","last_undegraded":"2024-04-25T10:07:52.470214+0000","last_fullsized":"2024-04-25T10:07:52.470214+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:13.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294437+0000","last_change":"2024-04-25T10:07:51.317685+0000","last_active":"2024-04-25T10:07:52.294437+0000","last_peered":"2024-04-25T10:07:52.294437+0000","last_clean":"2024-04-25T10:07:52.294437+0000","last_became_active":"2024-04-25T10:07:51.317386+0000","last_became_peered":"2024-04-25T10:07:51.317386+0000","last_unstale":"2024-04-25T10:07:52.294437+0000","last_undegraded":"2024-04-25T10:07:52.294437+0000","last_fullsized":"2024-04-25T10:07:52.294437+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:16.208395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470284+0000","last_change":"2024-04-25T10:07:51.306911+0000","last_active":"2024-04-25T10:07:52.470284+0000","last_peered":"2024-04-25T10:07:52.470284+0000","last_clean":"2024-04-25T10:07:52.470284+0000","last_became_active":"2024-04-25T10:07:51.306636+0000","last_became_peered":"2024-04-25T10:07:51.306636+0000","last_unstale":"2024-04-25T10:07:52.470284+0000","last_undegraded":"2024-04-25T10:07:52.470284+0000","last_fullsized":"2024-04-25T10:07:52.470284+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:57.699968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.472107+0000","last_change":"2024-04-25T10:07:51.304905+0000","last_active":"2024-04-25T10:07:52.472107+0000","last_peered":"2024-04-25T10:07:52.472107+0000","last_clean":"2024-04-25T10:07:52.472107+0000","last_became_active":"2024-04-25T10:07:51.304584+0000","last_became_peered":"2024-04-25T10:07:51.304584+0000","last_unstale":"2024-04-25T10:07:52.472107+0000","last_undegraded":"2024-04-25T10:07:52.472107+0000","last_fullsized":"2024-04-25T10:07:52.472107+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470893+0000","last_change":"2024-04-25T10:07:51.313605+0000","last_active":"2024-04-25T10:07:52.470893+0000","last_peered":"2024-04-25T10:07:52.470893+0000","last_clean":"2024-04-25T10:07:52.470893+0000","last_became_active":"2024-04-25T10:07:51.313265+0000","last_became_peered":"2024-04-25T10:07:51.313265+0000","last_unstale":"2024-04-25T10:07:52.470893+0000","last_undegraded":"2024-04-25T10:07:52.470893+0000","last_fullsized":"2024-04-25T10:07:52.470893+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471365+0000","last_change":"2024-04-25T10:07:51.312675+0000","last_active":"2024-04-25T10:07:52.471365+0000","last_peered":"2024-04-25T10:07:52.471365+0000","last_clean":"2024-04-25T10:07:52.471365+0000","last_became_active":"2024-04-25T10:07:51.311781+0000","last_became_peered":"2024-04-25T10:07:51.311781+0000","last_unstale":"2024-04-25T10:07:52.471365+0000","last_undegraded":"2024-04-25T10:07:52.471365+0000","last_fullsized":"2024-04-25T10:07:52.471365+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.404296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296412+0000","last_change":"2024-04-25T10:07:51.322910+0000","last_active":"2024-04-25T10:07:52.296412+0000","last_peered":"2024-04-25T10:07:52.296412+0000","last_clean":"2024-04-25T10:07:52.296412+0000","last_became_active":"2024-04-25T10:07:51.322604+0000","last_became_peered":"2024-04-25T10:07:51.322604+0000","last_unstale":"2024-04-25T10:07:52.296412+0000","last_undegraded":"2024-04-25T10:07:52.296412+0000","last_fullsized":"2024-04-25T10:07:52.296412+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294454+0000","last_change":"2024-04-25T10:07:51.316521+0000","last_active":"2024-04-25T10:07:52.294454+0000","last_peered":"2024-04-25T10:07:52.294454+0000","last_clean":"2024-04-25T10:07:52.294454+0000","last_became_active":"2024-04-25T10:07:51.316238+0000","last_became_peered":"2024-04-25T10:07:51.316238+0000","last_unstale":"2024-04-25T10:07:52.294454+0000","last_undegraded":"2024-04-25T10:07:52.294454+0000","last_fullsized":"2024-04-25T10:07:52.294454+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296471+0000","last_change":"2024-04-25T10:07:51.312907+0000","last_active":"2024-04-25T10:07:52.296471+0000","last_peered":"2024-04-25T10:07:52.296471+0000","last_clean":"2024-04-25T10:07:52.296471+0000","last_became_active":"2024-04-25T10:07:51.312595+0000","last_became_peered":"2024-04-25T10:07:51.312595+0000","last_unstale":"2024-04-25T10:07:52.296471+0000","last_undegraded":"2024-04-25T10:07:52.296471+0000","last_fullsized":"2024-04-25T10:07:52.296471+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470232+0000","last_change":"2024-04-25T10:07:51.309738+0000","last_active":"2024-04-25T10:07:52.470232+0000","last_peered":"2024-04-25T10:07:52.470232+0000","last_clean":"2024-04-25T10:07:52.470232+0000","last_became_active":"2024-04-25T10:07:51.309394+0000","last_became_peered":"2024-04-25T10:07:51.309394+0000","last_unstale":"2024-04-25T10:07:52.470232+0000","last_undegraded":"2024-04-25T10:07:52.470232+0000","last_fullsized":"2024-04-25T10:07:52.470232+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:51.800823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295906+0000","last_change":"2024-04-25T10:07:51.311322+0000","last_active":"2024-04-25T10:07:52.295906+0000","last_peered":"2024-04-25T10:07:52.295906+0000","last_clean":"2024-04-25T10:07:52.295906+0000","last_became_active":"2024-04-25T10:07:51.311041+0000","last_became_peered":"2024-04-25T10:07:51.311041+0000","last_unstale":"2024-04-25T10:07:52.295906+0000","last_undegraded":"2024-04-25T10:07:52.295906+0000","last_fullsized":"2024-04-25T10:07:52.295906+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:05.899651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471407+0000","last_change":"2024-04-25T10:07:51.317497+0000","last_active":"2024-04-25T10:07:52.471407+0000","last_peered":"2024-04-25T10:07:52.471407+0000","last_clean":"2024-04-25T10:07:52.471407+0000","last_became_active":"2024-04-25T10:07:51.317196+0000","last_became_peered":"2024-04-25T10:07:51.317196+0000","last_unstale":"2024-04-25T10:07:52.471407+0000","last_undegraded":"2024-04-25T10:07:52.471407+0000","last_fullsized":"2024-04-25T10:07:52.471407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:18.847819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294675+0000","last_change":"2024-04-25T10:07:51.324223+0000","last_active":"2024-04-25T10:07:52.294675+0000","last_peered":"2024-04-25T10:07:52.294675+0000","last_clean":"2024-04-25T10:07:52.294675+0000","last_became_active":"2024-04-25T10:07:51.323857+0000","last_became_peered":"2024-04-25T10:07:51.323857+0000","last_unstale":"2024-04-25T10:07:52.294675+0000","last_undegraded":"2024-04-25T10:07:52.294675+0000","last_fullsized":"2024-04-25T10:07:52.294675+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471513+0000","last_change":"2024-04-25T10:07:51.319543+0000","last_active":"2024-04-25T10:07:52.471513+0000","last_peered":"2024-04-25T10:07:52.471513+0000","last_clean":"2024-04-25T10:07:52.471513+0000","last_became_active":"2024-04-25T10:07:51.319265+0000","last_became_peered":"2024-04-25T10:07:51.319265+0000","last_unstale":"2024-04-25T10:07:52.471513+0000","last_undegraded":"2024-04-25T10:07:52.471513+0000","last_fullsized":"2024-04-25T10:07:52.471513+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:42.400180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470296+0000","last_change":"2024-04-25T10:07:51.318244+0000","last_active":"2024-04-25T10:07:52.470296+0000","last_peered":"2024-04-25T10:07:52.470296+0000","last_clean":"2024-04-25T10:07:52.470296+0000","last_became_active":"2024-04-25T10:07:51.317253+0000","last_became_peered":"2024-04-25T10:07:51.317253+0000","last_unstale":"2024-04-25T10:07:52.470296+0000","last_undegraded":"2024-04-25T10:07:52.470296+0000","last_fullsized":"2024-04-25T10:07:52.470296+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:17.235813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470996+0000","last_change":"2024-04-25T10:07:51.313123+0000","last_active":"2024-04-25T10:07:52.470996+0000","last_peered":"2024-04-25T10:07:52.470996+0000","last_clean":"2024-04-25T10:07:52.470996+0000","last_became_active":"2024-04-25T10:07:51.312765+0000","last_became_peered":"2024-04-25T10:07:51.312765+0000","last_unstale":"2024-04-25T10:07:52.470996+0000","last_undegraded":"2024-04-25T10:07:52.470996+0000","last_fullsized":"2024-04-25T10:07:52.470996+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:21.727353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296245+0000","last_change":"2024-04-25T10:07:51.312907+0000","last_active":"2024-04-25T10:07:52.296245+0000","last_peered":"2024-04-25T10:07:52.296245+0000","last_clean":"2024-04-25T10:07:52.296245+0000","last_became_active":"2024-04-25T10:07:51.312558+0000","last_became_peered":"2024-04-25T10:07:51.312558+0000","last_unstale":"2024-04-25T10:07:52.296245+0000","last_undegraded":"2024-04-25T10:07:52.296245+0000","last_fullsized":"2024-04-25T10:07:52.296245+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471064+0000","last_change":"2024-04-25T10:07:51.298328+0000","last_active":"2024-04-25T10:07:52.471064+0000","last_peered":"2024-04-25T10:07:52.471064+0000","last_clean":"2024-04-25T10:07:52.471064+0000","last_became_active":"2024-04-25T10:07:51.297878+0000","last_became_peered":"2024-04-25T10:07:51.297878+0000","last_unstale":"2024-04-25T10:07:52.471064+0000","last_undegraded":"2024-04-25T10:07:52.471064+0000","last_fullsized":"2024-04-25T10:07:52.471064+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:51.296503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296580+0000","last_change":"2024-04-25T10:07:50.306577+0000","last_active":"2024-04-25T10:07:52.296580+0000","last_peered":"2024-04-25T10:07:52.296580+0000","last_clean":"2024-04-25T10:07:52.296580+0000","last_became_active":"2024-04-25T10:07:50.306291+0000","last_became_peered":"2024-04-25T10:07:50.306291+0000","last_unstale":"2024-04-25T10:07:52.296580+0000","last_undegraded":"2024-04-25T10:07:52.296580+0000","last_fullsized":"2024-04-25T10:07:52.296580+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:23.427143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.472178+0000","last_change":"2024-04-25T10:07:51.316840+0000","last_active":"2024-04-25T10:07:52.472178+0000","last_peered":"2024-04-25T10:07:52.472178+0000","last_clean":"2024-04-25T10:07:52.472178+0000","last_became_active":"2024-04-25T10:07:51.316519+0000","last_became_peered":"2024-04-25T10:07:51.316519+0000","last_unstale":"2024-04-25T10:07:52.472178+0000","last_undegraded":"2024-04-25T10:07:52.472178+0000","last_fullsized":"2024-04-25T10:07:52.472178+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.168043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471882+0000","last_change":"2024-04-25T10:07:50.306264+0000","last_active":"2024-04-25T10:07:52.471882+0000","last_peered":"2024-04-25T10:07:52.471882+0000","last_clean":"2024-04-25T10:07:52.471882+0000","last_became_active":"2024-04-25T10:07:50.305960+0000","last_became_peered":"2024-04-25T10:07:50.305960+0000","last_unstale":"2024-04-25T10:07:52.471882+0000","last_undegraded":"2024-04-25T10:07:52.471882+0000","last_fullsized":"2024-04-25T10:07:52.471882+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:02.311727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471542+0000","last_change":"2024-04-25T10:07:50.311222+0000","last_active":"2024-04-25T10:07:52.471542+0000","last_peered":"2024-04-25T10:07:52.471542+0000","last_clean":"2024-04-25T10:07:52.471542+0000","last_became_active":"2024-04-25T10:07:50.310954+0000","last_became_peered":"2024-04-25T10:07:50.310954+0000","last_unstale":"2024-04-25T10:07:52.471542+0000","last_undegraded":"2024-04-25T10:07:52.471542+0000","last_fullsized":"2024-04-25T10:07:52.471542+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:44.898508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471321+0000","last_change":"2024-04-25T10:07:51.312039+0000","last_active":"2024-04-25T10:07:52.471321+0000","last_peered":"2024-04-25T10:07:52.471321+0000","last_clean":"2024-04-25T10:07:52.471321+0000","last_became_active":"2024-04-25T10:07:51.305475+0000","last_became_peered":"2024-04-25T10:07:51.305475+0000","last_unstale":"2024-04-25T10:07:52.471321+0000","last_undegraded":"2024-04-25T10:07:52.471321+0000","last_fullsized":"2024-04-25T10:07:52.471321+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470435+0000","last_change":"2024-04-25T10:07:50.309547+0000","last_active":"2024-04-25T10:07:52.470435+0000","last_peered":"2024-04-25T10:07:52.470435+0000","last_clean":"2024-04-25T10:07:52.470435+0000","last_became_active":"2024-04-25T10:07:50.309240+0000","last_became_peered":"2024-04-25T10:07:50.309240+0000","last_unstale":"2024-04-25T10:07:52.470435+0000","last_undegraded":"2024-04-25T10:07:52.470435+0000","last_fullsized":"2024-04-25T10:07:52.470435+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:23.723920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470369+0000","last_change":"2024-04-25T10:07:50.306966+0000","last_active":"2024-04-25T10:07:52.470369+0000","last_peered":"2024-04-25T10:07:52.470369+0000","last_clean":"2024-04-25T10:07:52.470369+0000","last_became_active":"2024-04-25T10:07:50.306613+0000","last_became_peered":"2024-04-25T10:07:50.306613+0000","last_unstale":"2024-04-25T10:07:52.470369+0000","last_undegraded":"2024-04-25T10:07:52.470369+0000","last_fullsized":"2024-04-25T10:07:52.470369+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:16.044211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296418+0000","last_change":"2024-04-25T10:07:51.319075+0000","last_active":"2024-04-25T10:07:52.296418+0000","last_peered":"2024-04-25T10:07:52.296418+0000","last_clean":"2024-04-25T10:07:52.296418+0000","last_became_active":"2024-04-25T10:07:51.318802+0000","last_became_peered":"2024-04-25T10:07:51.318802+0000","last_unstale":"2024-04-25T10:07:52.296418+0000","last_undegraded":"2024-04-25T10:07:52.296418+0000","last_fullsized":"2024-04-25T10:07:52.296418+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470993+0000","last_change":"2024-04-25T10:07:50.311086+0000","last_active":"2024-04-25T10:07:52.470993+0000","last_peered":"2024-04-25T10:07:52.470993+0000","last_clean":"2024-04-25T10:07:52.470993+0000","last_became_active":"2024-04-25T10:07:50.310798+0000","last_became_peered":"2024-04-25T10:07:50.310798+0000","last_unstale":"2024-04-25T10:07:52.470993+0000","last_undegraded":"2024-04-25T10:07:52.470993+0000","last_fullsized":"2024-04-25T10:07:52.470993+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:46.162970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.469999+0000","last_change":"2024-04-25T10:07:51.305304+0000","last_active":"2024-04-25T10:07:52.469999+0000","last_peered":"2024-04-25T10:07:52.469999+0000","last_clean":"2024-04-25T10:07:52.469999+0000","last_became_active":"2024-04-25T10:07:51.304968+0000","last_became_peered":"2024-04-25T10:07:51.304968+0000","last_unstale":"2024-04-25T10:07:52.469999+0000","last_undegraded":"2024-04-25T10:07:52.469999+0000","last_fullsized":"2024-04-25T10:07:52.469999+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:49.018249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295971+0000","last_change":"2024-04-25T10:07:50.305140+0000","last_active":"2024-04-25T10:07:52.295971+0000","last_peered":"2024-04-25T10:07:52.295971+0000","last_clean":"2024-04-25T10:07:52.295971+0000","last_became_active":"2024-04-25T10:07:50.304795+0000","last_became_peered":"2024-04-25T10:07:50.304795+0000","last_unstale":"2024-04-25T10:07:52.295971+0000","last_undegraded":"2024-04-25T10:07:52.295971+0000","last_fullsized":"2024-04-25T10:07:52.295971+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:22.138626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471236+0000","last_change":"2024-04-25T10:07:51.317275+0000","last_active":"2024-04-25T10:07:52.471236+0000","last_peered":"2024-04-25T10:07:52.471236+0000","last_clean":"2024-04-25T10:07:52.471236+0000","last_became_active":"2024-04-25T10:07:51.316910+0000","last_became_peered":"2024-04-25T10:07:51.316910+0000","last_unstale":"2024-04-25T10:07:52.471236+0000","last_undegraded":"2024-04-25T10:07:52.471236+0000","last_fullsized":"2024-04-25T10:07:52.471236+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:16.208395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294270+0000","last_change":"2024-04-25T10:07:50.307893+0000","last_active":"2024-04-25T10:07:52.294270+0000","last_peered":"2024-04-25T10:07:52.294270+0000","last_clean":"2024-04-25T10:07:52.294270+0000","last_became_active":"2024-04-25T10:07:50.307514+0000","last_became_peered":"2024-04-25T10:07:50.307514+0000","last_unstale":"2024-04-25T10:07:52.294270+0000","last_undegraded":"2024-04-25T10:07:52.294270+0000","last_fullsized":"2024-04-25T10:07:52.294270+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:33.584068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294636+0000","last_change":"2024-04-25T10:07:51.317446+0000","last_active":"2024-04-25T10:07:52.294636+0000","last_peered":"2024-04-25T10:07:52.294636+0000","last_clean":"2024-04-25T10:07:52.294636+0000","last_became_active":"2024-04-25T10:07:51.315123+0000","last_became_peered":"2024-04-25T10:07:51.315123+0000","last_unstale":"2024-04-25T10:07:52.294636+0000","last_undegraded":"2024-04-25T10:07:52.294636+0000","last_fullsized":"2024-04-25T10:07:52.294636+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296218+0000","last_change":"2024-04-25T10:07:50.306714+0000","last_active":"2024-04-25T10:07:52.296218+0000","last_peered":"2024-04-25T10:07:52.296218+0000","last_clean":"2024-04-25T10:07:52.296218+0000","last_became_active":"2024-04-25T10:07:50.306439+0000","last_became_peered":"2024-04-25T10:07:50.306439+0000","last_unstale":"2024-04-25T10:07:52.296218+0000","last_undegraded":"2024-04-25T10:07:52.296218+0000","last_fullsized":"2024-04-25T10:07:52.296218+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:48.651672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470854+0000","last_change":"2024-04-25T10:07:51.309229+0000","last_active":"2024-04-25T10:07:52.470854+0000","last_peered":"2024-04-25T10:07:52.470854+0000","last_clean":"2024-04-25T10:07:52.470854+0000","last_became_active":"2024-04-25T10:07:51.308925+0000","last_became_peered":"2024-04-25T10:07:51.308925+0000","last_unstale":"2024-04-25T10:07:52.470854+0000","last_undegraded":"2024-04-25T10:07:52.470854+0000","last_fullsized":"2024-04-25T10:07:52.470854+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.293101+0000","last_change":"2024-04-25T10:07:49.293101+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.293101+0000","last_undegraded":"2024-04-25T10:07:49.293101+0000","last_fullsized":"2024-04-25T10:07:49.293101+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470639+0000","last_change":"2024-04-25T10:07:50.309095+0000","last_active":"2024-04-25T10:07:52.470639+0000","last_peered":"2024-04-25T10:07:52.470639+0000","last_clean":"2024-04-25T10:07:52.470639+0000","last_became_active":"2024-04-25T10:07:50.308756+0000","last_became_peered":"2024-04-25T10:07:50.308756+0000","last_unstale":"2024-04-25T10:07:52.470639+0000","last_undegraded":"2024-04-25T10:07:52.470639+0000","last_fullsized":"2024-04-25T10:07:52.470639+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:04.298496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470541+0000","last_change":"2024-04-25T10:07:51.311605+0000","last_active":"2024-04-25T10:07:52.470541+0000","last_peered":"2024-04-25T10:07:52.470541+0000","last_clean":"2024-04-25T10:07:52.470541+0000","last_became_active":"2024-04-25T10:07:51.311307+0000","last_became_peered":"2024-04-25T10:07:51.311307+0000","last_unstale":"2024-04-25T10:07:52.470541+0000","last_undegraded":"2024-04-25T10:07:52.470541+0000","last_fullsized":"2024-04-25T10:07:52.470541+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:00.278450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294600+0000","last_change":"2024-04-25T10:07:50.306790+0000","last_active":"2024-04-25T10:07:52.294600+0000","last_peered":"2024-04-25T10:07:52.294600+0000","last_clean":"2024-04-25T10:07:52.294600+0000","last_became_active":"2024-04-25T10:07:50.306491+0000","last_became_peered":"2024-04-25T10:07:50.306491+0000","last_unstale":"2024-04-25T10:07:52.294600+0000","last_undegraded":"2024-04-25T10:07:52.294600+0000","last_fullsized":"2024-04-25T10:07:52.294600+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:43.478815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295998+0000","last_change":"2024-04-25T10:07:51.306158+0000","last_active":"2024-04-25T10:07:52.295998+0000","last_peered":"2024-04-25T10:07:52.295998+0000","last_clean":"2024-04-25T10:07:52.295998+0000","last_became_active":"2024-04-25T10:07:51.305824+0000","last_became_peered":"2024-04-25T10:07:51.305824+0000","last_unstale":"2024-04-25T10:07:52.295998+0000","last_undegraded":"2024-04-25T10:07:52.295998+0000","last_fullsized":"2024-04-25T10:07:52.295998+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:17.474618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292461+0000","last_change":"2024-04-25T10:07:49.292461+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292461+0000","last_undegraded":"2024-04-25T10:07:49.292461+0000","last_fullsized":"2024-04-25T10:07:49.292461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470269+0000","last_change":"2024-04-25T10:07:51.307313+0000","last_active":"2024-04-25T10:07:52.470269+0000","last_peered":"2024-04-25T10:07:52.470269+0000","last_clean":"2024-04-25T10:07:52.470269+0000","last_became_active":"2024-04-25T10:07:51.307032+0000","last_became_peered":"2024-04-25T10:07:51.307032+0000","last_unstale":"2024-04-25T10:07:52.470269+0000","last_undegraded":"2024-04-25T10:07:52.470269+0000","last_fullsized":"2024-04-25T10:07:52.470269+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:51.296503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470393+0000","last_change":"2024-04-25T10:07:50.304944+0000","last_active":"2024-04-25T10:07:52.470393+0000","last_peered":"2024-04-25T10:07:52.470393+0000","last_clean":"2024-04-25T10:07:52.470393+0000","last_became_active":"2024-04-25T10:07:50.304542+0000","last_became_peered":"2024-04-25T10:07:50.304542+0000","last_unstale":"2024-04-25T10:07:52.470393+0000","last_undegraded":"2024-04-25T10:07:52.470393+0000","last_fullsized":"2024-04-25T10:07:52.470393+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:39.934850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295938+0000","last_change":"2024-04-25T10:07:51.320164+0000","last_active":"2024-04-25T10:07:52.295938+0000","last_peered":"2024-04-25T10:07:52.295938+0000","last_clean":"2024-04-25T10:07:52.295938+0000","last_became_active":"2024-04-25T10:07:51.319859+0000","last_became_peered":"2024-04-25T10:07:51.319859+0000","last_unstale":"2024-04-25T10:07:52.295938+0000","last_undegraded":"2024-04-25T10:07:52.295938+0000","last_fullsized":"2024-04-25T10:07:52.295938+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:21.727353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470461+0000","last_change":"2024-04-25T10:07:50.309933+0000","last_active":"2024-04-25T10:07:52.470461+0000","last_peered":"2024-04-25T10:07:52.470461+0000","last_clean":"2024-04-25T10:07:52.470461+0000","last_became_active":"2024-04-25T10:07:50.309637+0000","last_became_peered":"2024-04-25T10:07:50.309637+0000","last_unstale":"2024-04-25T10:07:52.470461+0000","last_undegraded":"2024-04-25T10:07:52.470461+0000","last_fullsized":"2024-04-25T10:07:52.470461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:09.460489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470368+0000","last_change":"2024-04-25T10:07:51.308930+0000","last_active":"2024-04-25T10:07:52.470368+0000","last_peered":"2024-04-25T10:07:52.470368+0000","last_clean":"2024-04-25T10:07:52.470368+0000","last_became_active":"2024-04-25T10:07:51.308603+0000","last_became_peered":"2024-04-25T10:07:51.308603+0000","last_unstale":"2024-04-25T10:07:52.470368+0000","last_undegraded":"2024-04-25T10:07:52.470368+0000","last_fullsized":"2024-04-25T10:07:52.470368+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:17.724785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470020+0000","last_change":"2024-04-25T10:07:50.304956+0000","last_active":"2024-04-25T10:07:52.470020+0000","last_peered":"2024-04-25T10:07:52.470020+0000","last_clean":"2024-04-25T10:07:52.470020+0000","last_became_active":"2024-04-25T10:07:50.304634+0000","last_became_peered":"2024-04-25T10:07:50.304634+0000","last_unstale":"2024-04-25T10:07:52.470020+0000","last_undegraded":"2024-04-25T10:07:52.470020+0000","last_fullsized":"2024-04-25T10:07:52.470020+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:28.896776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470917+0000","last_change":"2024-04-25T10:07:51.314008+0000","last_active":"2024-04-25T10:07:52.470917+0000","last_peered":"2024-04-25T10:07:52.470917+0000","last_clean":"2024-04-25T10:07:52.470917+0000","last_became_active":"2024-04-25T10:07:51.313611+0000","last_became_peered":"2024-04-25T10:07:51.313611+0000","last_unstale":"2024-04-25T10:07:52.470917+0000","last_undegraded":"2024-04-25T10:07:52.470917+0000","last_fullsized":"2024-04-25T10:07:52.470917+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471515+0000","last_change":"2024-04-25T10:07:50.313909+0000","last_active":"2024-04-25T10:07:52.471515+0000","last_peered":"2024-04-25T10:07:52.471515+0000","last_clean":"2024-04-25T10:07:52.471515+0000","last_became_active":"2024-04-25T10:07:50.313400+0000","last_became_peered":"2024-04-25T10:07:50.313400+0000","last_unstale":"2024-04-25T10:07:52.471515+0000","last_undegraded":"2024-04-25T10:07:52.471515+0000","last_fullsized":"2024-04-25T10:07:52.471515+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:09.642875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471017+0000","last_change":"2024-04-25T10:07:51.310494+0000","last_active":"2024-04-25T10:07:52.471017+0000","last_peered":"2024-04-25T10:07:52.471017+0000","last_clean":"2024-04-25T10:07:52.471017+0000","last_became_active":"2024-04-25T10:07:51.310224+0000","last_became_peered":"2024-04-25T10:07:51.310224+0000","last_unstale":"2024-04-25T10:07:52.471017+0000","last_undegraded":"2024-04-25T10:07:52.471017+0000","last_fullsized":"2024-04-25T10:07:52.471017+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292480+0000","last_change":"2024-04-25T10:07:49.292480+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292480+0000","last_undegraded":"2024-04-25T10:07:49.292480+0000","last_fullsized":"2024-04-25T10:07:49.292480+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471787+0000","last_change":"2024-04-25T10:07:51.318600+0000","last_active":"2024-04-25T10:07:52.471787+0000","last_peered":"2024-04-25T10:07:52.471787+0000","last_clean":"2024-04-25T10:07:52.471787+0000","last_became_active":"2024-04-25T10:07:51.318304+0000","last_became_peered":"2024-04-25T10:07:51.318304+0000","last_unstale":"2024-04-25T10:07:52.471787+0000","last_undegraded":"2024-04-25T10:07:52.471787+0000","last_fullsized":"2024-04-25T10:07:52.471787+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:53.592626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470762+0000","last_change":"2024-04-25T10:07:50.304446+0000","last_active":"2024-04-25T10:07:52.470762+0000","last_peered":"2024-04-25T10:07:52.470762+0000","last_clean":"2024-04-25T10:07:52.470762+0000","last_became_active":"2024-04-25T10:07:50.303898+0000","last_became_peered":"2024-04-25T10:07:50.303898+0000","last_unstale":"2024-04-25T10:07:52.470762+0000","last_undegraded":"2024-04-25T10:07:52.470762+0000","last_fullsized":"2024-04-25T10:07:52.470762+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:23.749111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470626+0000","last_change":"2024-04-25T10:07:51.319204+0000","last_active":"2024-04-25T10:07:52.470626+0000","last_peered":"2024-04-25T10:07:52.470626+0000","last_clean":"2024-04-25T10:07:52.470626+0000","last_became_active":"2024-04-25T10:07:51.318367+0000","last_became_peered":"2024-04-25T10:07:51.318367+0000","last_unstale":"2024-04-25T10:07:52.470626+0000","last_undegraded":"2024-04-25T10:07:52.470626+0000","last_fullsized":"2024-04-25T10:07:52.470626+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:46.811230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292499+0000","last_change":"2024-04-25T10:07:49.292499+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292499+0000","last_undegraded":"2024-04-25T10:07:49.292499+0000","last_fullsized":"2024-04-25T10:07:49.292499+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296169+0000","last_change":"2024-04-25T10:07:51.312843+0000","last_active":"2024-04-25T10:07:52.296169+0000","last_peered":"2024-04-25T10:07:52.296169+0000","last_clean":"2024-04-25T10:07:52.296169+0000","last_became_active":"2024-04-25T10:07:51.311808+0000","last_became_peered":"2024-04-25T10:07:51.311808+0000","last_unstale":"2024-04-25T10:07:52.296169+0000","last_undegraded":"2024-04-25T10:07:52.296169+0000","last_fullsized":"2024-04-25T10:07:52.296169+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:18.847819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296623+0000","last_change":"2024-04-25T10:07:50.306664+0000","last_active":"2024-04-25T10:07:52.296623+0000","last_peered":"2024-04-25T10:07:52.296623+0000","last_clean":"2024-04-25T10:07:52.296623+0000","last_became_active":"2024-04-25T10:07:50.306383+0000","last_became_peered":"2024-04-25T10:07:50.306383+0000","last_unstale":"2024-04-25T10:07:52.296623+0000","last_undegraded":"2024-04-25T10:07:52.296623+0000","last_fullsized":"2024-04-25T10:07:52.296623+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:10.698506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292457+0000","last_change":"2024-04-25T10:07:49.292457+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292457+0000","last_undegraded":"2024-04-25T10:07:49.292457+0000","last_fullsized":"2024-04-25T10:07:49.292457+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296086+0000","last_change":"2024-04-25T10:07:50.311607+0000","last_active":"2024-04-25T10:07:52.296086+0000","last_peered":"2024-04-25T10:07:52.296086+0000","last_clean":"2024-04-25T10:07:52.296086+0000","last_became_active":"2024-04-25T10:07:50.311300+0000","last_became_peered":"2024-04-25T10:07:50.311300+0000","last_unstale":"2024-04-25T10:07:52.296086+0000","last_undegraded":"2024-04-25T10:07:52.296086+0000","last_fullsized":"2024-04-25T10:07:52.296086+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.109707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471043+0000","last_change":"2024-04-25T10:07:51.314396+0000","last_active":"2024-04-25T10:07:52.471043+0000","last_peered":"2024-04-25T10:07:52.471043+0000","last_clean":"2024-04-25T10:07:52.471043+0000","last_became_active":"2024-04-25T10:07:51.314116+0000","last_became_peered":"2024-04-25T10:07:51.314116+0000","last_unstale":"2024-04-25T10:07:52.471043+0000","last_undegraded":"2024-04-25T10:07:52.471043+0000","last_fullsized":"2024-04-25T10:07:52.471043+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:19.021120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471632+0000","last_change":"2024-04-25T10:07:50.307927+0000","last_active":"2024-04-25T10:07:52.471632+0000","last_peered":"2024-04-25T10:07:52.471632+0000","last_clean":"2024-04-25T10:07:52.471632+0000","last_became_active":"2024-04-25T10:07:50.307641+0000","last_became_peered":"2024-04-25T10:07:50.307641+0000","last_unstale":"2024-04-25T10:07:52.471632+0000","last_undegraded":"2024-04-25T10:07:52.471632+0000","last_fullsized":"2024-04-25T10:07:52.471632+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:22.138626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471021+0000","last_change":"2024-04-25T10:07:51.311790+0000","last_active":"2024-04-25T10:07:52.471021+0000","last_peered":"2024-04-25T10:07:52.471021+0000","last_clean":"2024-04-25T10:07:52.471021+0000","last_became_active":"2024-04-25T10:07:51.311524+0000","last_became_peered":"2024-04-25T10:07:51.311524+0000","last_unstale":"2024-04-25T10:07:52.471021+0000","last_undegraded":"2024-04-25T10:07:52.471021+0000","last_fullsized":"2024-04-25T10:07:52.471021+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.168043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470815+0000","last_change":"2024-04-25T10:07:50.313145+0000","last_active":"2024-04-25T10:07:52.470815+0000","last_peered":"2024-04-25T10:07:52.470815+0000","last_clean":"2024-04-25T10:07:52.470815+0000","last_became_active":"2024-04-25T10:07:50.312860+0000","last_became_peered":"2024-04-25T10:07:50.312860+0000","last_unstale":"2024-04-25T10:07:52.470815+0000","last_undegraded":"2024-04-25T10:07:52.470815+0000","last_fullsized":"2024-04-25T10:07:52.470815+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:16.044211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471897+0000","last_change":"2024-04-25T10:07:51.322690+0000","last_active":"2024-04-25T10:07:52.471897+0000","last_peered":"2024-04-25T10:07:52.471897+0000","last_clean":"2024-04-25T10:07:52.471897+0000","last_became_active":"2024-04-25T10:07:51.322404+0000","last_became_peered":"2024-04-25T10:07:51.322404+0000","last_unstale":"2024-04-25T10:07:52.471897+0000","last_undegraded":"2024-04-25T10:07:52.471897+0000","last_fullsized":"2024-04-25T10:07:52.471897+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:00.981450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296163+0000","last_change":"2024-04-25T10:07:50.309078+0000","last_active":"2024-04-25T10:07:52.296163+0000","last_peered":"2024-04-25T10:07:52.296163+0000","last_clean":"2024-04-25T10:07:52.296163+0000","last_became_active":"2024-04-25T10:07:50.308794+0000","last_became_peered":"2024-04-25T10:07:50.308794+0000","last_unstale":"2024-04-25T10:07:52.296163+0000","last_undegraded":"2024-04-25T10:07:52.296163+0000","last_fullsized":"2024-04-25T10:07:52.296163+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:09.642875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470834+0000","last_change":"2024-04-25T10:07:50.307963+0000","last_active":"2024-04-25T10:07:52.470834+0000","last_peered":"2024-04-25T10:07:52.470834+0000","last_clean":"2024-04-25T10:07:52.470834+0000","last_became_active":"2024-04-25T10:07:50.307611+0000","last_became_peered":"2024-04-25T10:07:50.307611+0000","last_unstale":"2024-04-25T10:07:52.470834+0000","last_undegraded":"2024-04-25T10:07:52.470834+0000","last_fullsized":"2024-04-25T10:07:52.470834+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:13.364471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470886+0000","last_change":"2024-04-25T10:07:51.316433+0000","last_active":"2024-04-25T10:07:52.470886+0000","last_peered":"2024-04-25T10:07:52.470886+0000","last_clean":"2024-04-25T10:07:52.470886+0000","last_became_active":"2024-04-25T10:07:51.315952+0000","last_became_peered":"2024-04-25T10:07:51.315952+0000","last_unstale":"2024-04-25T10:07:52.470886+0000","last_undegraded":"2024-04-25T10:07:52.470886+0000","last_fullsized":"2024-04-25T10:07:52.470886+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:13.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471022+0000","last_change":"2024-04-25T10:07:50.313990+0000","last_active":"2024-04-25T10:07:52.471022+0000","last_peered":"2024-04-25T10:07:52.471022+0000","last_clean":"2024-04-25T10:07:52.471022+0000","last_became_active":"2024-04-25T10:07:50.313700+0000","last_became_peered":"2024-04-25T10:07:50.313700+0000","last_unstale":"2024-04-25T10:07:52.471022+0000","last_undegraded":"2024-04-25T10:07:52.471022+0000","last_fullsized":"2024-04-25T10:07:52.471022+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:46.602681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294542+0000","last_change":"2024-04-25T10:07:51.315938+0000","last_active":"2024-04-25T10:07:52.294542+0000","last_peered":"2024-04-25T10:07:52.294542+0000","last_clean":"2024-04-25T10:07:52.294542+0000","last_became_active":"2024-04-25T10:07:51.315335+0000","last_became_peered":"2024-04-25T10:07:51.315335+0000","last_unstale":"2024-04-25T10:07:52.294542+0000","last_undegraded":"2024-04-25T10:07:52.294542+0000","last_fullsized":"2024-04-25T10:07:52.294542+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470304+0000","last_change":"2024-04-25T10:07:50.306917+0000","last_active":"2024-04-25T10:07:52.470304+0000","last_peered":"2024-04-25T10:07:52.470304+0000","last_clean":"2024-04-25T10:07:52.470304+0000","last_became_active":"2024-04-25T10:07:50.306549+0000","last_became_peered":"2024-04-25T10:07:50.306549+0000","last_unstale":"2024-04-25T10:07:52.470304+0000","last_undegraded":"2024-04-25T10:07:52.470304+0000","last_fullsized":"2024-04-25T10:07:52.470304+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:06.727495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471599+0000","last_change":"2024-04-25T10:07:50.311119+0000","last_active":"2024-04-25T10:07:52.471599+0000","last_peered":"2024-04-25T10:07:52.471599+0000","last_clean":"2024-04-25T10:07:52.471599+0000","last_became_active":"2024-04-25T10:07:50.310843+0000","last_became_peered":"2024-04-25T10:07:50.310843+0000","last_unstale":"2024-04-25T10:07:52.471599+0000","last_undegraded":"2024-04-25T10:07:52.471599+0000","last_fullsized":"2024-04-25T10:07:52.471599+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:26.803283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471140+0000","last_change":"2024-04-25T10:07:51.316546+0000","last_active":"2024-04-25T10:07:52.471140+0000","last_peered":"2024-04-25T10:07:52.471140+0000","last_clean":"2024-04-25T10:07:52.471140+0000","last_became_active":"2024-04-25T10:07:51.313979+0000","last_became_peered":"2024-04-25T10:07:51.313979+0000","last_unstale":"2024-04-25T10:07:52.471140+0000","last_undegraded":"2024-04-25T10:07:52.471140+0000","last_fullsized":"2024-04-25T10:07:52.471140+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:17.474618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292461+0000","last_change":"2024-04-25T10:07:49.292461+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292461+0000","last_undegraded":"2024-04-25T10:07:49.292461+0000","last_fullsized":"2024-04-25T10:07:49.292461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294591+0000","last_change":"2024-04-25T10:07:50.309026+0000","last_active":"2024-04-25T10:07:52.294591+0000","last_peered":"2024-04-25T10:07:52.294591+0000","last_clean":"2024-04-25T10:07:52.294591+0000","last_became_active":"2024-04-25T10:07:50.308706+0000","last_became_peered":"2024-04-25T10:07:50.308706+0000","last_unstale":"2024-04-25T10:07:52.294591+0000","last_undegraded":"2024-04-25T10:07:52.294591+0000","last_fullsized":"2024-04-25T10:07:52.294591+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:17.725300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":245,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295995+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:07:52.295995+0000","last_peered":"2024-04-25T10:07:52.295995+0000","last_clean":"2024-04-25T10:07:52.295995+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:07:52.295995+0000","last_undegraded":"2024-04-25T10:07:52.295995+0000","last_fullsized":"2024-04-25T10:07:52.295995+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":7},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":288874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":692224,"data_stored":282504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":289398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":692224,"data_stored":282572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":284254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509478,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1298432,"data_stored":879732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":700416,"data_stored":284094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":859800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:55.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.38 is not active+clean 2024-04-25T10:07:55.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:50.284453+0000', 'last_change': '2024-04-25T10:07:50.284453+0000', 'last_active': '2024-04-25T10:07:50.284453+0000', 'last_peered': '2024-04-25T10:07:50.284453+0000', 'last_clean': '2024-04-25T10:07:50.284453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:50.284453+0000', 'last_undegraded': '2024-04-25T10:07:50.284453+0000', 'last_fullsized': '2024-04-25T10:07:50.284453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:55.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.24 is not active+clean 2024-04-25T10:07:55.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:50.284453+0000', 'last_change': '2024-04-25T10:07:50.284453+0000', 'last_active': '2024-04-25T10:07:50.284453+0000', 'last_peered': '2024-04-25T10:07:50.284453+0000', 'last_clean': '2024-04-25T10:07:50.284453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:50.284453+0000', 'last_undegraded': '2024-04-25T10:07:50.284453+0000', 'last_fullsized': '2024-04-25T10:07:50.284453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:55.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.23 is not active+clean 2024-04-25T10:07:55.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:50.284453+0000', 'last_change': '2024-04-25T10:07:50.284453+0000', 'last_active': '2024-04-25T10:07:50.284453+0000', 'last_peered': '2024-04-25T10:07:50.284453+0000', 'last_clean': '2024-04-25T10:07:50.284453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:50.284453+0000', 'last_undegraded': '2024-04-25T10:07:50.284453+0000', 'last_fullsized': '2024-04-25T10:07:50.284453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:55.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2024-04-25T10:07:55.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:50.284453+0000', 'last_change': '2024-04-25T10:07:50.284453+0000', 'last_active': '2024-04-25T10:07:50.284453+0000', 'last_peered': '2024-04-25T10:07:50.284453+0000', 'last_clean': '2024-04-25T10:07:50.284453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:50.284453+0000', 'last_undegraded': '2024-04-25T10:07:50.284453+0000', 'last_fullsized': '2024-04-25T10:07:50.284453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:55.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2024-04-25T10:07:55.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:50.284453+0000', 'last_change': '2024-04-25T10:07:50.284453+0000', 'last_active': '2024-04-25T10:07:50.284453+0000', 'last_peered': '2024-04-25T10:07:50.284453+0000', 'last_clean': '2024-04-25T10:07:50.284453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:50.284453+0000', 'last_undegraded': '2024-04-25T10:07:50.284453+0000', 'last_fullsized': '2024-04-25T10:07:50.284453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:55.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2024-04-25T10:07:55.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:49.293101+0000', 'last_change': '2024-04-25T10:07:49.293101+0000', 'last_active': '2024-04-25T10:07:49.282421+0000', 'last_peered': '2024-04-25T10:07:49.282421+0000', 'last_clean': '2024-04-25T10:07:49.282421+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:49.293101+0000', 'last_undegraded': '2024-04-25T10:07:49.293101+0000', 'last_fullsized': '2024-04-25T10:07:49.293101+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:49.282421+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:49.282421+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:49.282421+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:07:55.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2024-04-25T10:07:55.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:50.284453+0000', 'last_change': '2024-04-25T10:07:50.284453+0000', 'last_active': '2024-04-25T10:07:50.284453+0000', 'last_peered': '2024-04-25T10:07:50.284453+0000', 'last_clean': '2024-04-25T10:07:50.284453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:50.284453+0000', 'last_undegraded': '2024-04-25T10:07:50.284453+0000', 'last_fullsized': '2024-04-25T10:07:50.284453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:55.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2024-04-25T10:07:55.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:49.292461+0000', 'last_change': '2024-04-25T10:07:49.292461+0000', 'last_active': '2024-04-25T10:07:49.282421+0000', 'last_peered': '2024-04-25T10:07:49.282421+0000', 'last_clean': '2024-04-25T10:07:49.282421+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:49.292461+0000', 'last_undegraded': '2024-04-25T10:07:49.292461+0000', 'last_fullsized': '2024-04-25T10:07:49.292461+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:49.282421+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:49.282421+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:49.282421+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:07:55.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2024-04-25T10:07:55.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:49.292480+0000', 'last_change': '2024-04-25T10:07:49.292480+0000', 'last_active': '2024-04-25T10:07:49.282421+0000', 'last_peered': '2024-04-25T10:07:49.282421+0000', 'last_clean': '2024-04-25T10:07:49.282421+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:49.292480+0000', 'last_undegraded': '2024-04-25T10:07:49.292480+0000', 'last_fullsized': '2024-04-25T10:07:49.292480+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:49.282421+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:49.282421+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:49.282421+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:07:55.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2024-04-25T10:07:55.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:49.292499+0000', 'last_change': '2024-04-25T10:07:49.292499+0000', 'last_active': '2024-04-25T10:07:49.282421+0000', 'last_peered': '2024-04-25T10:07:49.282421+0000', 'last_clean': '2024-04-25T10:07:49.282421+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:49.292499+0000', 'last_undegraded': '2024-04-25T10:07:49.292499+0000', 'last_fullsized': '2024-04-25T10:07:49.292499+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:49.282421+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:49.282421+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:49.282421+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:07:55.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2024-04-25T10:07:55.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:50.284453+0000', 'last_change': '2024-04-25T10:07:50.284453+0000', 'last_active': '2024-04-25T10:07:50.284453+0000', 'last_peered': '2024-04-25T10:07:50.284453+0000', 'last_clean': '2024-04-25T10:07:50.284453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:50.284453+0000', 'last_undegraded': '2024-04-25T10:07:50.284453+0000', 'last_fullsized': '2024-04-25T10:07:50.284453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:55.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2024-04-25T10:07:55.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:49.292457+0000', 'last_change': '2024-04-25T10:07:49.292457+0000', 'last_active': '2024-04-25T10:07:49.282421+0000', 'last_peered': '2024-04-25T10:07:49.282421+0000', 'last_clean': '2024-04-25T10:07:49.282421+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:49.292457+0000', 'last_undegraded': '2024-04-25T10:07:49.292457+0000', 'last_fullsized': '2024-04-25T10:07:49.292457+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:49.282421+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:49.282421+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:49.282421+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:07:55.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2024-04-25T10:07:55.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:50.284453+0000', 'last_change': '2024-04-25T10:07:50.284453+0000', 'last_active': '2024-04-25T10:07:50.284453+0000', 'last_peered': '2024-04-25T10:07:50.284453+0000', 'last_clean': '2024-04-25T10:07:50.284453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:50.284453+0000', 'last_undegraded': '2024-04-25T10:07:50.284453+0000', 'last_fullsized': '2024-04-25T10:07:50.284453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:55.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2024-04-25T10:07:55.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:50.284453+0000', 'last_change': '2024-04-25T10:07:50.284453+0000', 'last_active': '2024-04-25T10:07:50.284453+0000', 'last_peered': '2024-04-25T10:07:50.284453+0000', 'last_clean': '2024-04-25T10:07:50.284453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:50.284453+0000', 'last_undegraded': '2024-04-25T10:07:50.284453+0000', 'last_fullsized': '2024-04-25T10:07:50.284453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:55.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2024-04-25T10:07:55.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:50.284453+0000', 'last_change': '2024-04-25T10:07:50.284453+0000', 'last_active': '2024-04-25T10:07:50.284453+0000', 'last_peered': '2024-04-25T10:07:50.284453+0000', 'last_clean': '2024-04-25T10:07:50.284453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:50.284453+0000', 'last_undegraded': '2024-04-25T10:07:50.284453+0000', 'last_fullsized': '2024-04-25T10:07:50.284453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:55.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2024-04-25T10:07:55.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:49.292461+0000', 'last_change': '2024-04-25T10:07:49.292461+0000', 'last_active': '2024-04-25T10:07:49.282421+0000', 'last_peered': '2024-04-25T10:07:49.282421+0000', 'last_clean': '2024-04-25T10:07:49.282421+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:49.292461+0000', 'last_undegraded': '2024-04-25T10:07:49.292461+0000', 'last_fullsized': '2024-04-25T10:07:49.292461+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:49.282421+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:49.282421+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:49.282421+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:07:55.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2024-04-25T10:07:55.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:50.284453+0000', 'last_change': '2024-04-25T10:07:50.284453+0000', 'last_active': '2024-04-25T10:07:50.284453+0000', 'last_peered': '2024-04-25T10:07:50.284453+0000', 'last_clean': '2024-04-25T10:07:50.284453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:50.284453+0000', 'last_undegraded': '2024-04-25T10:07:50.284453+0000', 'last_fullsized': '2024-04-25T10:07:50.284453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:50.284453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:55.021 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:07:55.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.149+0000 7f4a9d3e0640 1 Processor -- start 2024-04-25T10:07:55.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.149+0000 7f4a9d3e0640 1 -- start start 2024-04-25T10:07:55.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.150+0000 7f4a9d3e0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a980d35b0 0x7f4a980b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.150+0000 7f4a9d3e0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4a980b5990 0x7f4a980b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.150+0000 7f4a9d3e0640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a9815e0c0 con 0x7f4a9815dce0 2024-04-25T10:07:55.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.150+0000 7f4a9d3e0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a980d2510 con 0x7f4a980d35b0 2024-04-25T10:07:55.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.150+0000 7f4a9d3e0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a980d27f0 con 0x7f4a980b5990 2024-04-25T10:07:55.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.150+0000 7f4a8ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a980d35b0 0x7f4a980b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.150+0000 7f4a8ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a980d35b0 0x7f4a980b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53150/0 (socket says 172.21.15.16:53150) 2024-04-25T10:07:55.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.150+0000 7f4a8ffff640 1 -- 172.21.15.16:0/3853454881 learned_addr learned my addr 172.21.15.16:0/3853454881 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:55.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.150+0000 7f4a977fe640 1 --2- 172.21.15.16:0/3853454881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4a980b5990 0x7f4a980b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.150+0000 7f4a8ffff640 1 -- 172.21.15.16:0/3853454881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4a980b5990 msgr2=0x7f4a980b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:55.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.150+0000 7f4a8ffff640 1 --2- 172.21.15.16:0/3853454881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4a980b5990 0x7f4a980b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.150+0000 7f4a8ffff640 1 -- 172.21.15.16:0/3853454881 >> v1:172.21.15.186:6789/0 conn(0x7f4a9815dce0 legacy=0x7f4a980d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.150+0000 7f4a8ffff640 1 -- 172.21.15.16:0/3853454881 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a980d2ad0 con 0x7f4a980d35b0 2024-04-25T10:07:55.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.150+0000 7f4a977fe640 1 --2- 172.21.15.16:0/3853454881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4a980b5990 0x7f4a980b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:55.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.151+0000 7f4a8ffff640 1 --2- 172.21.15.16:0/3853454881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a980d35b0 0x7f4a980b5450 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009870 tx=0x7f4a8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d146f5e7ce6801 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:55.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.151+0000 7f4a967fc640 1 -- 172.21.15.16:0/3853454881 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a80037070 con 0x7f4a980d35b0 2024-04-25T10:07:55.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.151+0000 7f4a967fc640 1 -- 172.21.15.16:0/3853454881 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a80032440 con 0x7f4a980d35b0 2024-04-25T10:07:55.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.151+0000 7f4a967fc640 1 -- 172.21.15.16:0/3853454881 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a80032810 con 0x7f4a980d35b0 2024-04-25T10:07:55.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.151+0000 7f4a9d3e0640 1 -- 172.21.15.16:0/3853454881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a980d35b0 msgr2=0x7f4a980b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.151+0000 7f4a9d3e0640 1 --2- 172.21.15.16:0/3853454881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a980d35b0 0x7f4a980b5450 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009870 tx=0x7f4a8002a270 comp rx=0 tx=0).stop 2024-04-25T10:07:55.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.152+0000 7f4a9d3e0640 1 -- 172.21.15.16:0/3853454881 shutdown_connections 2024-04-25T10:07:55.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.152+0000 7f4a9d3e0640 1 --2- 172.21.15.16:0/3853454881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4a980b5990 0x7f4a980b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.152+0000 7f4a9d3e0640 1 --2- 172.21.15.16:0/3853454881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a980d35b0 0x7f4a980b5450 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.152+0000 7f4a9d3e0640 1 -- 172.21.15.16:0/3853454881 >> 172.21.15.16:0/3853454881 conn(0x7f4a98072900 msgr2=0x7f4a980d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.152+0000 7f4a9d3e0640 1 -- 172.21.15.16:0/3853454881 shutdown_connections 2024-04-25T10:07:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.152+0000 7f4a9d3e0640 1 -- 172.21.15.16:0/3853454881 wait complete. 2024-04-25T10:07:55.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.153+0000 7f4a9d3e0640 1 Processor -- start 2024-04-25T10:07:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.153+0000 7f4a9d3e0640 1 -- start start 2024-04-25T10:07:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.153+0000 7f4a9d3e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a980b5990 0x7f4a980c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a9d3e0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4a980d35b0 0x7f4a980cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a9d3e0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a9815dce0 0x7f4a980cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a9d3e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a980d2160 con 0x7f4a980b5990 2024-04-25T10:07:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a9d3e0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a980d3090 con 0x7f4a9815dce0 2024-04-25T10:07:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a9d3e0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a980cd3e0 con 0x7f4a980d35b0 2024-04-25T10:07:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a96ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a980b5990 0x7f4a980c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a96ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a980b5990 0x7f4a980c7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53154/0 (socket says 172.21.15.16:53154) 2024-04-25T10:07:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a96ffd640 1 -- 172.21.15.16:0/1407058046 learned_addr learned my addr 172.21.15.16:0/1407058046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a977fe640 1 --2- 172.21.15.16:0/1407058046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a9815dce0 0x7f4a980cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a8ffff640 1 --2- 172.21.15.16:0/1407058046 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4a980d35b0 0x7f4a980cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a96ffd640 1 -- 172.21.15.16:0/1407058046 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4a980d35b0 msgr2=0x7f4a980cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a96ffd640 1 --2- 172.21.15.16:0/1407058046 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4a980d35b0 0x7f4a980cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a96ffd640 1 -- 172.21.15.16:0/1407058046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a9815dce0 msgr2=0x7f4a980cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a96ffd640 1 --2- 172.21.15.16:0/1407058046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a9815dce0 0x7f4a980cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a96ffd640 1 -- 172.21.15.16:0/1407058046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a980cc860 con 0x7f4a980b5990 2024-04-25T10:07:55.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a977fe640 1 --2- 172.21.15.16:0/1407058046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a9815dce0 0x7f4a980cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:55.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a96ffd640 1 --2- 172.21.15.16:0/1407058046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a980b5990 0x7f4a980c7770 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800c3c0 tx=0x7f4a8800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:55.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a8f7fe640 1 -- 172.21.15.16:0/1407058046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a880052d0 con 0x7f4a980b5990 2024-04-25T10:07:55.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a8f7fe640 1 -- 172.21.15.16:0/1407058046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a88005490 con 0x7f4a980b5990 2024-04-25T10:07:55.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a9d3e0640 1 -- 172.21.15.16:0/1407058046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a980ccba0 con 0x7f4a980b5990 2024-04-25T10:07:55.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a8f7fe640 1 -- 172.21.15.16:0/1407058046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a88007260 con 0x7f4a980b5990 2024-04-25T10:07:55.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.154+0000 7f4a9d3e0640 1 -- 172.21.15.16:0/1407058046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a980cc570 con 0x7f4a980b5990 2024-04-25T10:07:55.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.155+0000 7f4a9d3e0640 1 -- 172.21.15.16:0/1407058046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a980749b0 con 0x7f4a980b5990 2024-04-25T10:07:55.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.156+0000 7f4a8f7fe640 1 -- 172.21.15.16:0/1407058046 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a88007400 con 0x7f4a980b5990 2024-04-25T10:07:55.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.156+0000 7f4a8f7fe640 1 --2- 172.21.15.16:0/1407058046 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4a7806b4d0 0x7f4a7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.156+0000 7f4a8f7fe640 1 -- 172.21.15.16:0/1407058046 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a880896e0 con 0x7f4a980b5990 2024-04-25T10:07:55.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.157+0000 7f4a8ffff640 1 --2- 172.21.15.16:0/1407058046 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4a7806b4d0 0x7f4a7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.157+0000 7f4a8ffff640 1 --2- 172.21.15.16:0/1407058046 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4a7806b4d0 0x7f4a7806d990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4a80034000 tx=0x7f4a8002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:55.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.160+0000 7f4a8f7fe640 1 -- 172.21.15.16:0/1407058046 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a980ccba0 con 0x7f4a980b5990 2024-04-25T10:07:55.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.413+0000 7f4a9d3e0640 1 -- 172.21.15.16:0/1407058046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f4a9806b080 con 0x7f4a980b5990 2024-04-25T10:07:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.413+0000 7f4a8f7fe640 1 -- 172.21.15.16:0/1407058046 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1595 (secure 0 0 0) 0x7f4a88058560 con 0x7f4a980b5990 2024-04-25T10:07:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:55.419 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":188,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039497,"num_in_osds":8,"osd_in_since":1714039488,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":80},{"state_name":"unknown","count":11},{"state_name":"creating+peering","count":6}],"num_pgs":97,"num_pools":3,"num_objects":20,"data_bytes":592744,"bytes_used":249450496,"bytes_avail":772844662784,"bytes_total":773094113280,"unknown_pgs_ratio":0.11340206116437912,"inactive_pgs_ratio":0.061855670064687729,"write_bytes_sec":1581,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":27,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":35141}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-04-25T10:07:39.815355+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:07:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.415+0000 7f4a9d3e0640 1 -- 172.21.15.16:0/1407058046 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4a7806b4d0 msgr2=0x7f4a7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.415+0000 7f4a9d3e0640 1 --2- 172.21.15.16:0/1407058046 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4a7806b4d0 0x7f4a7806d990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4a80034000 tx=0x7f4a8002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.415+0000 7f4a9d3e0640 1 -- 172.21.15.16:0/1407058046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a980b5990 msgr2=0x7f4a980c7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.415+0000 7f4a9d3e0640 1 --2- 172.21.15.16:0/1407058046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a980b5990 0x7f4a980c7770 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800c3c0 tx=0x7f4a8800c880 comp rx=0 tx=0).stop 2024-04-25T10:07:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.415+0000 7f4a9d3e0640 1 -- 172.21.15.16:0/1407058046 shutdown_connections 2024-04-25T10:07:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.415+0000 7f4a9d3e0640 1 --2- 172.21.15.16:0/1407058046 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4a7806b4d0 0x7f4a7806d990 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.415+0000 7f4a9d3e0640 1 --2- 172.21.15.16:0/1407058046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a9815dce0 0x7f4a980cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.415+0000 7f4a9d3e0640 1 --2- 172.21.15.16:0/1407058046 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4a980d35b0 0x7f4a980cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.415+0000 7f4a9d3e0640 1 --2- 172.21.15.16:0/1407058046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a980b5990 0x7f4a980c7770 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.415+0000 7f4a9d3e0640 1 -- 172.21.15.16:0/1407058046 >> 172.21.15.16:0/1407058046 conn(0x7f4a98072900 msgr2=0x7f4a980ce250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.415+0000 7f4a9d3e0640 1 -- 172.21.15.16:0/1407058046 shutdown_connections 2024-04-25T10:07:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.415+0000 7f4a9d3e0640 1 -- 172.21.15.16:0/1407058046 wait complete. 2024-04-25T10:07:55.432 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:07:55.432 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:55.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.526+0000 7feffcd7e640 1 Processor -- start 2024-04-25T10:07:55.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.526+0000 7feffcd7e640 1 -- start start 2024-04-25T10:07:55.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.527+0000 7feffcd7e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feff815a870 0x7feff80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.527+0000 7feffcd7e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feff80cfc40 0x7feff815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.527+0000 7feffcd7e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff80cdb10 con 0x7feff806e2e0 2024-04-25T10:07:55.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.527+0000 7feffcd7e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff80ce720 con 0x7feff80cfc40 2024-04-25T10:07:55.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.527+0000 7feffcd7e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff80cddf0 con 0x7feff815a870 2024-04-25T10:07:55.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.527+0000 7feff6d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feff80cfc40 0x7feff815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.527+0000 7feff5d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7feff806e2e0 0x7feff80cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54438/0 (socket says 172.21.15.16:54438) 2024-04-25T10:07:55.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.527+0000 7feff5d9b640 1 -- 172.21.15.16:0/441429522 learned_addr learned my addr 172.21.15.16:0/441429522 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:55.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.527+0000 7feff6d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feff80cfc40 0x7feff815ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53170/0 (socket says 172.21.15.16:53170) 2024-04-25T10:07:55.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.527+0000 7feff6d9d640 1 -- 172.21.15.16:0/441429522 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feff815a870 msgr2=0x7feff80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:55.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.527+0000 7feff659c640 1 --2- 172.21.15.16:0/441429522 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feff815a870 0x7feff80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.528+0000 7feff6d9d640 1 --2- 172.21.15.16:0/441429522 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feff815a870 0x7feff80cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.528+0000 7feff6d9d640 1 -- 172.21.15.16:0/441429522 >> v1:172.21.15.16:6789/0 conn(0x7feff806e2e0 legacy=0x7feff80cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.528+0000 7feff6d9d640 1 -- 172.21.15.16:0/441429522 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff80ce0d0 con 0x7feff80cfc40 2024-04-25T10:07:55.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.528+0000 7feff559a640 1 -- 172.21.15.16:0/441429522 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2465061922 0 0) 0x7feff80cdb10 con 0x7feff806e2e0 2024-04-25T10:07:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.528+0000 7feff6d9d640 1 --2- 172.21.15.16:0/441429522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff80cfc40 0x7feff815ac50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009d90 tx=0x7fefe802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9854cf416e94d53b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.528+0000 7feff559a640 1 -- 172.21.15.16:0/441429522 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe8007740 con 0x7feff80cfc40 2024-04-25T10:07:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.528+0000 7feff559a640 1 -- 172.21.15.16:0/441429522 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe80078e0 con 0x7feff80cfc40 2024-04-25T10:07:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.528+0000 7feff559a640 1 -- 172.21.15.16:0/441429522 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe8007cb0 con 0x7feff80cfc40 2024-04-25T10:07:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.529+0000 7feffcd7e640 1 -- 172.21.15.16:0/441429522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff80cfc40 msgr2=0x7feff815ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.529+0000 7feffcd7e640 1 --2- 172.21.15.16:0/441429522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff80cfc40 0x7feff815ac50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009d90 tx=0x7fefe802a660 comp rx=0 tx=0).stop 2024-04-25T10:07:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.529+0000 7feffcd7e640 1 -- 172.21.15.16:0/441429522 shutdown_connections 2024-04-25T10:07:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.529+0000 7feffcd7e640 1 --2- 172.21.15.16:0/441429522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff80cfc40 0x7feff815ac50 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.529+0000 7feffcd7e640 1 --2- 172.21.15.16:0/441429522 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feff815a870 0x7feff80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.529+0000 7feffcd7e640 1 -- 172.21.15.16:0/441429522 >> 172.21.15.16:0/441429522 conn(0x7feff806aba0 msgr2=0x7feff806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.529+0000 7feffcd7e640 1 -- 172.21.15.16:0/441429522 shutdown_connections 2024-04-25T10:07:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.529+0000 7feffcd7e640 1 -- 172.21.15.16:0/441429522 wait complete. 2024-04-25T10:07:55.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.530+0000 7feffcd7e640 1 Processor -- start 2024-04-25T10:07:55.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.530+0000 7feffcd7e640 1 -- start start 2024-04-25T10:07:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.531+0000 7feffcd7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff806e2e0 0x7feff8146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.531+0000 7feffcd7e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feff80cfc40 0x7feff80b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.531+0000 7feffcd7e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feff815a870 0x7feff80b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.531+0000 7feffcd7e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff80cd700 con 0x7feff806e2e0 2024-04-25T10:07:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.531+0000 7feffcd7e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff80cd580 con 0x7feff815a870 2024-04-25T10:07:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.531+0000 7feffcd7e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff80cd880 con 0x7feff80cfc40 2024-04-25T10:07:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.531+0000 7feff659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff806e2e0 0x7feff8146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.531+0000 7feff659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff806e2e0 0x7feff8146b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53172/0 (socket says 172.21.15.16:53172) 2024-04-25T10:07:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.531+0000 7feff659c640 1 -- 172.21.15.16:0/948337774 learned_addr learned my addr 172.21.15.16:0/948337774 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.531+0000 7feff6d9d640 1 --2- 172.21.15.16:0/948337774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feff815a870 0x7feff80b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.531+0000 7feff5d9b640 1 --2- 172.21.15.16:0/948337774 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feff80cfc40 0x7feff80b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.532+0000 7feff659c640 1 -- 172.21.15.16:0/948337774 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feff80cfc40 msgr2=0x7feff80b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.532+0000 7feff659c640 1 --2- 172.21.15.16:0/948337774 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feff80cfc40 0x7feff80b34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.532+0000 7feff659c640 1 -- 172.21.15.16:0/948337774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feff815a870 msgr2=0x7feff80b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.532+0000 7feff659c640 1 --2- 172.21.15.16:0/948337774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feff815a870 0x7feff80b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.532+0000 7feff659c640 1 -- 172.21.15.16:0/948337774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff8147610 con 0x7feff806e2e0 2024-04-25T10:07:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.532+0000 7feff659c640 1 --2- 172.21.15.16:0/948337774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff806e2e0 0x7feff8146b60 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7feff000c7f0 tx=0x7feff000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.532+0000 7fefe6ffd640 1 -- 172.21.15.16:0/948337774 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0015070 con 0x7feff806e2e0 2024-04-25T10:07:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.532+0000 7fefe6ffd640 1 -- 172.21.15.16:0/948337774 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff0004220 con 0x7feff806e2e0 2024-04-25T10:07:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.532+0000 7feffcd7e640 1 -- 172.21.15.16:0/948337774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff81478f0 con 0x7feff806e2e0 2024-04-25T10:07:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.532+0000 7fefe6ffd640 1 -- 172.21.15.16:0/948337774 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0007620 con 0x7feff806e2e0 2024-04-25T10:07:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.532+0000 7feffcd7e640 1 -- 172.21.15.16:0/948337774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff81470a0 con 0x7feff806e2e0 2024-04-25T10:07:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.533+0000 7feffcd7e640 1 -- 172.21.15.16:0/948337774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff80749b0 con 0x7feff806e2e0 2024-04-25T10:07:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.534+0000 7fefe6ffd640 1 -- 172.21.15.16:0/948337774 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feff00079d0 con 0x7feff806e2e0 2024-04-25T10:07:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.534+0000 7fefe6ffd640 1 --2- 172.21.15.16:0/948337774 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fefd806b500 0x7fefd806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.534+0000 7fefe6ffd640 1 -- 172.21.15.16:0/948337774 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feff008a140 con 0x7feff806e2e0 2024-04-25T10:07:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.534+0000 7feff5d9b640 1 --2- 172.21.15.16:0/948337774 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fefd806b500 0x7fefd806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.535+0000 7feff5d9b640 1 --2- 172.21.15.16:0/948337774 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fefd806b500 0x7fefd806d9c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fefec007f70 tx=0x7fefec007f00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:55.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.539+0000 7fefe6ffd640 1 -- 172.21.15.16:0/948337774 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feff0058970 con 0x7feff806e2e0 2024-04-25T10:07:55.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.715+0000 7feffcd7e640 1 -- 172.21.15.16:0/948337774 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feff815ba10 con 0x7fefd806b500 2024-04-25T10:07:55.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.721+0000 7fefe6ffd640 1 -- 172.21.15.16:0/948337774 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249962 (secure 0 0 0) 0x7feff815ba10 con 0x7fefd806b500 2024-04-25T10:07:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:55.731 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:07:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.724+0000 7feffcd7e640 1 -- 172.21.15.16:0/948337774 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fefd806b500 msgr2=0x7fefd806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.724+0000 7feffcd7e640 1 --2- 172.21.15.16:0/948337774 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fefd806b500 0x7fefd806d9c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fefec007f70 tx=0x7fefec007f00 comp rx=0 tx=0).stop 2024-04-25T10:07:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.724+0000 7feffcd7e640 1 -- 172.21.15.16:0/948337774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff806e2e0 msgr2=0x7feff8146b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.724+0000 7feffcd7e640 1 --2- 172.21.15.16:0/948337774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff806e2e0 0x7feff8146b60 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7feff000c7f0 tx=0x7feff000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.724+0000 7feffcd7e640 1 -- 172.21.15.16:0/948337774 shutdown_connections 2024-04-25T10:07:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.724+0000 7feffcd7e640 1 --2- 172.21.15.16:0/948337774 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fefd806b500 0x7fefd806d9c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.724+0000 7feffcd7e640 1 --2- 172.21.15.16:0/948337774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feff815a870 0x7feff80b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.724+0000 7feffcd7e640 1 --2- 172.21.15.16:0/948337774 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feff80cfc40 0x7feff80b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.724+0000 7feffcd7e640 1 --2- 172.21.15.16:0/948337774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff806e2e0 0x7feff8146b60 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.724+0000 7feffcd7e640 1 -- 172.21.15.16:0/948337774 >> 172.21.15.16:0/948337774 conn(0x7feff806aba0 msgr2=0x7feff80b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.724+0000 7feffcd7e640 1 -- 172.21.15.16:0/948337774 shutdown_connections 2024-04-25T10:07:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:55.724+0000 7feffcd7e640 1 -- 172.21.15.16:0/948337774 wait complete. 2024-04-25T10:07:55.741 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-04-25T10:07:53.818447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":197,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243604,"kb_used_data":6608,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731116,"statfs":{"total":773094113280,"available":772844662784,"internally_reserved":0,"allocated":6766592,"data_stored":3451228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.533513"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.472064+0000","last_change":"2024-04-25T10:07:51.311604+0000","last_active":"2024-04-25T10:07:52.472064+0000","last_peered":"2024-04-25T10:07:52.472064+0000","last_clean":"2024-04-25T10:07:52.472064+0000","last_became_active":"2024-04-25T10:07:51.310395+0000","last_became_peered":"2024-04-25T10:07:51.310395+0000","last_unstale":"2024-04-25T10:07:52.472064+0000","last_undegraded":"2024-04-25T10:07:52.472064+0000","last_fullsized":"2024-04-25T10:07:52.472064+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471748+0000","last_change":"2024-04-25T10:07:51.315131+0000","last_active":"2024-04-25T10:07:52.471748+0000","last_peered":"2024-04-25T10:07:52.471748+0000","last_clean":"2024-04-25T10:07:52.471748+0000","last_became_active":"2024-04-25T10:07:51.314869+0000","last_became_peered":"2024-04-25T10:07:51.314869+0000","last_unstale":"2024-04-25T10:07:52.471748+0000","last_undegraded":"2024-04-25T10:07:52.471748+0000","last_fullsized":"2024-04-25T10:07:52.471748+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:38.125900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470577+0000","last_change":"2024-04-25T10:07:51.303339+0000","last_active":"2024-04-25T10:07:52.470577+0000","last_peered":"2024-04-25T10:07:52.470577+0000","last_clean":"2024-04-25T10:07:52.470577+0000","last_became_active":"2024-04-25T10:07:51.303082+0000","last_became_peered":"2024-04-25T10:07:51.303082+0000","last_unstale":"2024-04-25T10:07:52.470577+0000","last_undegraded":"2024-04-25T10:07:52.470577+0000","last_fullsized":"2024-04-25T10:07:52.470577+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:16.208395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470265+0000","last_change":"2024-04-25T10:07:51.318977+0000","last_active":"2024-04-25T10:07:52.470265+0000","last_peered":"2024-04-25T10:07:52.470265+0000","last_clean":"2024-04-25T10:07:52.470265+0000","last_became_active":"2024-04-25T10:07:51.318704+0000","last_became_peered":"2024-04-25T10:07:51.318704+0000","last_unstale":"2024-04-25T10:07:52.470265+0000","last_undegraded":"2024-04-25T10:07:52.470265+0000","last_fullsized":"2024-04-25T10:07:52.470265+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:01.127872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471710+0000","last_change":"2024-04-25T10:07:51.309203+0000","last_active":"2024-04-25T10:07:52.471710+0000","last_peered":"2024-04-25T10:07:52.471710+0000","last_clean":"2024-04-25T10:07:52.471710+0000","last_became_active":"2024-04-25T10:07:51.308918+0000","last_became_peered":"2024-04-25T10:07:51.308918+0000","last_unstale":"2024-04-25T10:07:52.471710+0000","last_undegraded":"2024-04-25T10:07:52.471710+0000","last_fullsized":"2024-04-25T10:07:52.471710+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470720+0000","last_change":"2024-04-25T10:07:51.317240+0000","last_active":"2024-04-25T10:07:52.470720+0000","last_peered":"2024-04-25T10:07:52.470720+0000","last_clean":"2024-04-25T10:07:52.470720+0000","last_became_active":"2024-04-25T10:07:51.316735+0000","last_became_peered":"2024-04-25T10:07:51.316735+0000","last_unstale":"2024-04-25T10:07:52.470720+0000","last_undegraded":"2024-04-25T10:07:52.470720+0000","last_fullsized":"2024-04-25T10:07:52.470720+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:49.018249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470088+0000","last_change":"2024-04-25T10:07:51.310589+0000","last_active":"2024-04-25T10:07:52.470088+0000","last_peered":"2024-04-25T10:07:52.470088+0000","last_clean":"2024-04-25T10:07:52.470088+0000","last_became_active":"2024-04-25T10:07:51.310211+0000","last_became_peered":"2024-04-25T10:07:51.310211+0000","last_unstale":"2024-04-25T10:07:52.470088+0000","last_undegraded":"2024-04-25T10:07:52.470088+0000","last_fullsized":"2024-04-25T10:07:52.470088+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:16.223783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296172+0000","last_change":"2024-04-25T10:07:51.312254+0000","last_active":"2024-04-25T10:07:52.296172+0000","last_peered":"2024-04-25T10:07:52.296172+0000","last_clean":"2024-04-25T10:07:52.296172+0000","last_became_active":"2024-04-25T10:07:51.311901+0000","last_became_peered":"2024-04-25T10:07:51.311901+0000","last_unstale":"2024-04-25T10:07:52.296172+0000","last_undegraded":"2024-04-25T10:07:52.296172+0000","last_fullsized":"2024-04-25T10:07:52.296172+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:42.400180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470490+0000","last_change":"2024-04-25T10:07:51.317615+0000","last_active":"2024-04-25T10:07:52.470490+0000","last_peered":"2024-04-25T10:07:52.470490+0000","last_clean":"2024-04-25T10:07:52.470490+0000","last_became_active":"2024-04-25T10:07:51.317131+0000","last_became_peered":"2024-04-25T10:07:51.317131+0000","last_unstale":"2024-04-25T10:07:52.470490+0000","last_undegraded":"2024-04-25T10:07:52.470490+0000","last_fullsized":"2024-04-25T10:07:52.470490+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:19.853622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296651+0000","last_change":"2024-04-25T10:07:51.323605+0000","last_active":"2024-04-25T10:07:52.296651+0000","last_peered":"2024-04-25T10:07:52.296651+0000","last_clean":"2024-04-25T10:07:52.296651+0000","last_became_active":"2024-04-25T10:07:51.323306+0000","last_became_peered":"2024-04-25T10:07:51.323306+0000","last_unstale":"2024-04-25T10:07:52.296651+0000","last_undegraded":"2024-04-25T10:07:52.296651+0000","last_fullsized":"2024-04-25T10:07:52.296651+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.168043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296163+0000","last_change":"2024-04-25T10:07:51.312425+0000","last_active":"2024-04-25T10:07:52.296163+0000","last_peered":"2024-04-25T10:07:52.296163+0000","last_clean":"2024-04-25T10:07:52.296163+0000","last_became_active":"2024-04-25T10:07:51.312133+0000","last_became_peered":"2024-04-25T10:07:51.312133+0000","last_unstale":"2024-04-25T10:07:52.296163+0000","last_undegraded":"2024-04-25T10:07:52.296163+0000","last_fullsized":"2024-04-25T10:07:52.296163+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.404296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294383+0000","last_change":"2024-04-25T10:07:51.317166+0000","last_active":"2024-04-25T10:07:52.294383+0000","last_peered":"2024-04-25T10:07:52.294383+0000","last_clean":"2024-04-25T10:07:52.294383+0000","last_became_active":"2024-04-25T10:07:51.315513+0000","last_became_peered":"2024-04-25T10:07:51.315513+0000","last_unstale":"2024-04-25T10:07:52.294383+0000","last_undegraded":"2024-04-25T10:07:52.294383+0000","last_fullsized":"2024-04-25T10:07:52.294383+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:51.296503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470214+0000","last_change":"2024-04-25T10:07:51.313310+0000","last_active":"2024-04-25T10:07:52.470214+0000","last_peered":"2024-04-25T10:07:52.470214+0000","last_clean":"2024-04-25T10:07:52.470214+0000","last_became_active":"2024-04-25T10:07:51.313001+0000","last_became_peered":"2024-04-25T10:07:51.313001+0000","last_unstale":"2024-04-25T10:07:52.470214+0000","last_undegraded":"2024-04-25T10:07:52.470214+0000","last_fullsized":"2024-04-25T10:07:52.470214+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:13.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294437+0000","last_change":"2024-04-25T10:07:51.317685+0000","last_active":"2024-04-25T10:07:52.294437+0000","last_peered":"2024-04-25T10:07:52.294437+0000","last_clean":"2024-04-25T10:07:52.294437+0000","last_became_active":"2024-04-25T10:07:51.317386+0000","last_became_peered":"2024-04-25T10:07:51.317386+0000","last_unstale":"2024-04-25T10:07:52.294437+0000","last_undegraded":"2024-04-25T10:07:52.294437+0000","last_fullsized":"2024-04-25T10:07:52.294437+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:16.208395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470284+0000","last_change":"2024-04-25T10:07:51.306911+0000","last_active":"2024-04-25T10:07:52.470284+0000","last_peered":"2024-04-25T10:07:52.470284+0000","last_clean":"2024-04-25T10:07:52.470284+0000","last_became_active":"2024-04-25T10:07:51.306636+0000","last_became_peered":"2024-04-25T10:07:51.306636+0000","last_unstale":"2024-04-25T10:07:52.470284+0000","last_undegraded":"2024-04-25T10:07:52.470284+0000","last_fullsized":"2024-04-25T10:07:52.470284+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:57.699968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.472107+0000","last_change":"2024-04-25T10:07:51.304905+0000","last_active":"2024-04-25T10:07:52.472107+0000","last_peered":"2024-04-25T10:07:52.472107+0000","last_clean":"2024-04-25T10:07:52.472107+0000","last_became_active":"2024-04-25T10:07:51.304584+0000","last_became_peered":"2024-04-25T10:07:51.304584+0000","last_unstale":"2024-04-25T10:07:52.472107+0000","last_undegraded":"2024-04-25T10:07:52.472107+0000","last_fullsized":"2024-04-25T10:07:52.472107+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470893+0000","last_change":"2024-04-25T10:07:51.313605+0000","last_active":"2024-04-25T10:07:52.470893+0000","last_peered":"2024-04-25T10:07:52.470893+0000","last_clean":"2024-04-25T10:07:52.470893+0000","last_became_active":"2024-04-25T10:07:51.313265+0000","last_became_peered":"2024-04-25T10:07:51.313265+0000","last_unstale":"2024-04-25T10:07:52.470893+0000","last_undegraded":"2024-04-25T10:07:52.470893+0000","last_fullsized":"2024-04-25T10:07:52.470893+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471365+0000","last_change":"2024-04-25T10:07:51.312675+0000","last_active":"2024-04-25T10:07:52.471365+0000","last_peered":"2024-04-25T10:07:52.471365+0000","last_clean":"2024-04-25T10:07:52.471365+0000","last_became_active":"2024-04-25T10:07:51.311781+0000","last_became_peered":"2024-04-25T10:07:51.311781+0000","last_unstale":"2024-04-25T10:07:52.471365+0000","last_undegraded":"2024-04-25T10:07:52.471365+0000","last_fullsized":"2024-04-25T10:07:52.471365+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.404296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296412+0000","last_change":"2024-04-25T10:07:51.322910+0000","last_active":"2024-04-25T10:07:52.296412+0000","last_peered":"2024-04-25T10:07:52.296412+0000","last_clean":"2024-04-25T10:07:52.296412+0000","last_became_active":"2024-04-25T10:07:51.322604+0000","last_became_peered":"2024-04-25T10:07:51.322604+0000","last_unstale":"2024-04-25T10:07:52.296412+0000","last_undegraded":"2024-04-25T10:07:52.296412+0000","last_fullsized":"2024-04-25T10:07:52.296412+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294454+0000","last_change":"2024-04-25T10:07:51.316521+0000","last_active":"2024-04-25T10:07:52.294454+0000","last_peered":"2024-04-25T10:07:52.294454+0000","last_clean":"2024-04-25T10:07:52.294454+0000","last_became_active":"2024-04-25T10:07:51.316238+0000","last_became_peered":"2024-04-25T10:07:51.316238+0000","last_unstale":"2024-04-25T10:07:52.294454+0000","last_undegraded":"2024-04-25T10:07:52.294454+0000","last_fullsized":"2024-04-25T10:07:52.294454+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296471+0000","last_change":"2024-04-25T10:07:51.312907+0000","last_active":"2024-04-25T10:07:52.296471+0000","last_peered":"2024-04-25T10:07:52.296471+0000","last_clean":"2024-04-25T10:07:52.296471+0000","last_became_active":"2024-04-25T10:07:51.312595+0000","last_became_peered":"2024-04-25T10:07:51.312595+0000","last_unstale":"2024-04-25T10:07:52.296471+0000","last_undegraded":"2024-04-25T10:07:52.296471+0000","last_fullsized":"2024-04-25T10:07:52.296471+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470232+0000","last_change":"2024-04-25T10:07:51.309738+0000","last_active":"2024-04-25T10:07:52.470232+0000","last_peered":"2024-04-25T10:07:52.470232+0000","last_clean":"2024-04-25T10:07:52.470232+0000","last_became_active":"2024-04-25T10:07:51.309394+0000","last_became_peered":"2024-04-25T10:07:51.309394+0000","last_unstale":"2024-04-25T10:07:52.470232+0000","last_undegraded":"2024-04-25T10:07:52.470232+0000","last_fullsized":"2024-04-25T10:07:52.470232+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:51.800823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295906+0000","last_change":"2024-04-25T10:07:51.311322+0000","last_active":"2024-04-25T10:07:52.295906+0000","last_peered":"2024-04-25T10:07:52.295906+0000","last_clean":"2024-04-25T10:07:52.295906+0000","last_became_active":"2024-04-25T10:07:51.311041+0000","last_became_peered":"2024-04-25T10:07:51.311041+0000","last_unstale":"2024-04-25T10:07:52.295906+0000","last_undegraded":"2024-04-25T10:07:52.295906+0000","last_fullsized":"2024-04-25T10:07:52.295906+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:05.899651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471407+0000","last_change":"2024-04-25T10:07:51.317497+0000","last_active":"2024-04-25T10:07:52.471407+0000","last_peered":"2024-04-25T10:07:52.471407+0000","last_clean":"2024-04-25T10:07:52.471407+0000","last_became_active":"2024-04-25T10:07:51.317196+0000","last_became_peered":"2024-04-25T10:07:51.317196+0000","last_unstale":"2024-04-25T10:07:52.471407+0000","last_undegraded":"2024-04-25T10:07:52.471407+0000","last_fullsized":"2024-04-25T10:07:52.471407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:18.847819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294675+0000","last_change":"2024-04-25T10:07:51.324223+0000","last_active":"2024-04-25T10:07:52.294675+0000","last_peered":"2024-04-25T10:07:52.294675+0000","last_clean":"2024-04-25T10:07:52.294675+0000","last_became_active":"2024-04-25T10:07:51.323857+0000","last_became_peered":"2024-04-25T10:07:51.323857+0000","last_unstale":"2024-04-25T10:07:52.294675+0000","last_undegraded":"2024-04-25T10:07:52.294675+0000","last_fullsized":"2024-04-25T10:07:52.294675+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471513+0000","last_change":"2024-04-25T10:07:51.319543+0000","last_active":"2024-04-25T10:07:52.471513+0000","last_peered":"2024-04-25T10:07:52.471513+0000","last_clean":"2024-04-25T10:07:52.471513+0000","last_became_active":"2024-04-25T10:07:51.319265+0000","last_became_peered":"2024-04-25T10:07:51.319265+0000","last_unstale":"2024-04-25T10:07:52.471513+0000","last_undegraded":"2024-04-25T10:07:52.471513+0000","last_fullsized":"2024-04-25T10:07:52.471513+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:42.400180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470296+0000","last_change":"2024-04-25T10:07:51.318244+0000","last_active":"2024-04-25T10:07:52.470296+0000","last_peered":"2024-04-25T10:07:52.470296+0000","last_clean":"2024-04-25T10:07:52.470296+0000","last_became_active":"2024-04-25T10:07:51.317253+0000","last_became_peered":"2024-04-25T10:07:51.317253+0000","last_unstale":"2024-04-25T10:07:52.470296+0000","last_undegraded":"2024-04-25T10:07:52.470296+0000","last_fullsized":"2024-04-25T10:07:52.470296+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:17.235813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470996+0000","last_change":"2024-04-25T10:07:51.313123+0000","last_active":"2024-04-25T10:07:52.470996+0000","last_peered":"2024-04-25T10:07:52.470996+0000","last_clean":"2024-04-25T10:07:52.470996+0000","last_became_active":"2024-04-25T10:07:51.312765+0000","last_became_peered":"2024-04-25T10:07:51.312765+0000","last_unstale":"2024-04-25T10:07:52.470996+0000","last_undegraded":"2024-04-25T10:07:52.470996+0000","last_fullsized":"2024-04-25T10:07:52.470996+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:21.727353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296245+0000","last_change":"2024-04-25T10:07:51.312907+0000","last_active":"2024-04-25T10:07:52.296245+0000","last_peered":"2024-04-25T10:07:52.296245+0000","last_clean":"2024-04-25T10:07:52.296245+0000","last_became_active":"2024-04-25T10:07:51.312558+0000","last_became_peered":"2024-04-25T10:07:51.312558+0000","last_unstale":"2024-04-25T10:07:52.296245+0000","last_undegraded":"2024-04-25T10:07:52.296245+0000","last_fullsized":"2024-04-25T10:07:52.296245+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471064+0000","last_change":"2024-04-25T10:07:51.298328+0000","last_active":"2024-04-25T10:07:52.471064+0000","last_peered":"2024-04-25T10:07:52.471064+0000","last_clean":"2024-04-25T10:07:52.471064+0000","last_became_active":"2024-04-25T10:07:51.297878+0000","last_became_peered":"2024-04-25T10:07:51.297878+0000","last_unstale":"2024-04-25T10:07:52.471064+0000","last_undegraded":"2024-04-25T10:07:52.471064+0000","last_fullsized":"2024-04-25T10:07:52.471064+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:51.296503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296580+0000","last_change":"2024-04-25T10:07:50.306577+0000","last_active":"2024-04-25T10:07:52.296580+0000","last_peered":"2024-04-25T10:07:52.296580+0000","last_clean":"2024-04-25T10:07:52.296580+0000","last_became_active":"2024-04-25T10:07:50.306291+0000","last_became_peered":"2024-04-25T10:07:50.306291+0000","last_unstale":"2024-04-25T10:07:52.296580+0000","last_undegraded":"2024-04-25T10:07:52.296580+0000","last_fullsized":"2024-04-25T10:07:52.296580+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:23.427143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.472178+0000","last_change":"2024-04-25T10:07:51.316840+0000","last_active":"2024-04-25T10:07:52.472178+0000","last_peered":"2024-04-25T10:07:52.472178+0000","last_clean":"2024-04-25T10:07:52.472178+0000","last_became_active":"2024-04-25T10:07:51.316519+0000","last_became_peered":"2024-04-25T10:07:51.316519+0000","last_unstale":"2024-04-25T10:07:52.472178+0000","last_undegraded":"2024-04-25T10:07:52.472178+0000","last_fullsized":"2024-04-25T10:07:52.472178+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.168043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471882+0000","last_change":"2024-04-25T10:07:50.306264+0000","last_active":"2024-04-25T10:07:52.471882+0000","last_peered":"2024-04-25T10:07:52.471882+0000","last_clean":"2024-04-25T10:07:52.471882+0000","last_became_active":"2024-04-25T10:07:50.305960+0000","last_became_peered":"2024-04-25T10:07:50.305960+0000","last_unstale":"2024-04-25T10:07:52.471882+0000","last_undegraded":"2024-04-25T10:07:52.471882+0000","last_fullsized":"2024-04-25T10:07:52.471882+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:02.311727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471542+0000","last_change":"2024-04-25T10:07:50.311222+0000","last_active":"2024-04-25T10:07:52.471542+0000","last_peered":"2024-04-25T10:07:52.471542+0000","last_clean":"2024-04-25T10:07:52.471542+0000","last_became_active":"2024-04-25T10:07:50.310954+0000","last_became_peered":"2024-04-25T10:07:50.310954+0000","last_unstale":"2024-04-25T10:07:52.471542+0000","last_undegraded":"2024-04-25T10:07:52.471542+0000","last_fullsized":"2024-04-25T10:07:52.471542+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:44.898508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471321+0000","last_change":"2024-04-25T10:07:51.312039+0000","last_active":"2024-04-25T10:07:52.471321+0000","last_peered":"2024-04-25T10:07:52.471321+0000","last_clean":"2024-04-25T10:07:52.471321+0000","last_became_active":"2024-04-25T10:07:51.305475+0000","last_became_peered":"2024-04-25T10:07:51.305475+0000","last_unstale":"2024-04-25T10:07:52.471321+0000","last_undegraded":"2024-04-25T10:07:52.471321+0000","last_fullsized":"2024-04-25T10:07:52.471321+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470435+0000","last_change":"2024-04-25T10:07:50.309547+0000","last_active":"2024-04-25T10:07:52.470435+0000","last_peered":"2024-04-25T10:07:52.470435+0000","last_clean":"2024-04-25T10:07:52.470435+0000","last_became_active":"2024-04-25T10:07:50.309240+0000","last_became_peered":"2024-04-25T10:07:50.309240+0000","last_unstale":"2024-04-25T10:07:52.470435+0000","last_undegraded":"2024-04-25T10:07:52.470435+0000","last_fullsized":"2024-04-25T10:07:52.470435+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:23.723920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470369+0000","last_change":"2024-04-25T10:07:50.306966+0000","last_active":"2024-04-25T10:07:52.470369+0000","last_peered":"2024-04-25T10:07:52.470369+0000","last_clean":"2024-04-25T10:07:52.470369+0000","last_became_active":"2024-04-25T10:07:50.306613+0000","last_became_peered":"2024-04-25T10:07:50.306613+0000","last_unstale":"2024-04-25T10:07:52.470369+0000","last_undegraded":"2024-04-25T10:07:52.470369+0000","last_fullsized":"2024-04-25T10:07:52.470369+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:16.044211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296418+0000","last_change":"2024-04-25T10:07:51.319075+0000","last_active":"2024-04-25T10:07:52.296418+0000","last_peered":"2024-04-25T10:07:52.296418+0000","last_clean":"2024-04-25T10:07:52.296418+0000","last_became_active":"2024-04-25T10:07:51.318802+0000","last_became_peered":"2024-04-25T10:07:51.318802+0000","last_unstale":"2024-04-25T10:07:52.296418+0000","last_undegraded":"2024-04-25T10:07:52.296418+0000","last_fullsized":"2024-04-25T10:07:52.296418+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470993+0000","last_change":"2024-04-25T10:07:50.311086+0000","last_active":"2024-04-25T10:07:52.470993+0000","last_peered":"2024-04-25T10:07:52.470993+0000","last_clean":"2024-04-25T10:07:52.470993+0000","last_became_active":"2024-04-25T10:07:50.310798+0000","last_became_peered":"2024-04-25T10:07:50.310798+0000","last_unstale":"2024-04-25T10:07:52.470993+0000","last_undegraded":"2024-04-25T10:07:52.470993+0000","last_fullsized":"2024-04-25T10:07:52.470993+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:46.162970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.469999+0000","last_change":"2024-04-25T10:07:51.305304+0000","last_active":"2024-04-25T10:07:52.469999+0000","last_peered":"2024-04-25T10:07:52.469999+0000","last_clean":"2024-04-25T10:07:52.469999+0000","last_became_active":"2024-04-25T10:07:51.304968+0000","last_became_peered":"2024-04-25T10:07:51.304968+0000","last_unstale":"2024-04-25T10:07:52.469999+0000","last_undegraded":"2024-04-25T10:07:52.469999+0000","last_fullsized":"2024-04-25T10:07:52.469999+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:49.018249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295971+0000","last_change":"2024-04-25T10:07:50.305140+0000","last_active":"2024-04-25T10:07:52.295971+0000","last_peered":"2024-04-25T10:07:52.295971+0000","last_clean":"2024-04-25T10:07:52.295971+0000","last_became_active":"2024-04-25T10:07:50.304795+0000","last_became_peered":"2024-04-25T10:07:50.304795+0000","last_unstale":"2024-04-25T10:07:52.295971+0000","last_undegraded":"2024-04-25T10:07:52.295971+0000","last_fullsized":"2024-04-25T10:07:52.295971+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:22.138626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471236+0000","last_change":"2024-04-25T10:07:51.317275+0000","last_active":"2024-04-25T10:07:52.471236+0000","last_peered":"2024-04-25T10:07:52.471236+0000","last_clean":"2024-04-25T10:07:52.471236+0000","last_became_active":"2024-04-25T10:07:51.316910+0000","last_became_peered":"2024-04-25T10:07:51.316910+0000","last_unstale":"2024-04-25T10:07:52.471236+0000","last_undegraded":"2024-04-25T10:07:52.471236+0000","last_fullsized":"2024-04-25T10:07:52.471236+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:16.208395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294270+0000","last_change":"2024-04-25T10:07:50.307893+0000","last_active":"2024-04-25T10:07:52.294270+0000","last_peered":"2024-04-25T10:07:52.294270+0000","last_clean":"2024-04-25T10:07:52.294270+0000","last_became_active":"2024-04-25T10:07:50.307514+0000","last_became_peered":"2024-04-25T10:07:50.307514+0000","last_unstale":"2024-04-25T10:07:52.294270+0000","last_undegraded":"2024-04-25T10:07:52.294270+0000","last_fullsized":"2024-04-25T10:07:52.294270+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:33.584068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294636+0000","last_change":"2024-04-25T10:07:51.317446+0000","last_active":"2024-04-25T10:07:52.294636+0000","last_peered":"2024-04-25T10:07:52.294636+0000","last_clean":"2024-04-25T10:07:52.294636+0000","last_became_active":"2024-04-25T10:07:51.315123+0000","last_became_peered":"2024-04-25T10:07:51.315123+0000","last_unstale":"2024-04-25T10:07:52.294636+0000","last_undegraded":"2024-04-25T10:07:52.294636+0000","last_fullsized":"2024-04-25T10:07:52.294636+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296218+0000","last_change":"2024-04-25T10:07:50.306714+0000","last_active":"2024-04-25T10:07:52.296218+0000","last_peered":"2024-04-25T10:07:52.296218+0000","last_clean":"2024-04-25T10:07:52.296218+0000","last_became_active":"2024-04-25T10:07:50.306439+0000","last_became_peered":"2024-04-25T10:07:50.306439+0000","last_unstale":"2024-04-25T10:07:52.296218+0000","last_undegraded":"2024-04-25T10:07:52.296218+0000","last_fullsized":"2024-04-25T10:07:52.296218+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:48.651672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470854+0000","last_change":"2024-04-25T10:07:51.309229+0000","last_active":"2024-04-25T10:07:52.470854+0000","last_peered":"2024-04-25T10:07:52.470854+0000","last_clean":"2024-04-25T10:07:52.470854+0000","last_became_active":"2024-04-25T10:07:51.308925+0000","last_became_peered":"2024-04-25T10:07:51.308925+0000","last_unstale":"2024-04-25T10:07:52.470854+0000","last_undegraded":"2024-04-25T10:07:52.470854+0000","last_fullsized":"2024-04-25T10:07:52.470854+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.293101+0000","last_change":"2024-04-25T10:07:49.293101+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.293101+0000","last_undegraded":"2024-04-25T10:07:49.293101+0000","last_fullsized":"2024-04-25T10:07:49.293101+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470639+0000","last_change":"2024-04-25T10:07:50.309095+0000","last_active":"2024-04-25T10:07:52.470639+0000","last_peered":"2024-04-25T10:07:52.470639+0000","last_clean":"2024-04-25T10:07:52.470639+0000","last_became_active":"2024-04-25T10:07:50.308756+0000","last_became_peered":"2024-04-25T10:07:50.308756+0000","last_unstale":"2024-04-25T10:07:52.470639+0000","last_undegraded":"2024-04-25T10:07:52.470639+0000","last_fullsized":"2024-04-25T10:07:52.470639+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:04.298496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470541+0000","last_change":"2024-04-25T10:07:51.311605+0000","last_active":"2024-04-25T10:07:52.470541+0000","last_peered":"2024-04-25T10:07:52.470541+0000","last_clean":"2024-04-25T10:07:52.470541+0000","last_became_active":"2024-04-25T10:07:51.311307+0000","last_became_peered":"2024-04-25T10:07:51.311307+0000","last_unstale":"2024-04-25T10:07:52.470541+0000","last_undegraded":"2024-04-25T10:07:52.470541+0000","last_fullsized":"2024-04-25T10:07:52.470541+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:00.278450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294600+0000","last_change":"2024-04-25T10:07:50.306790+0000","last_active":"2024-04-25T10:07:52.294600+0000","last_peered":"2024-04-25T10:07:52.294600+0000","last_clean":"2024-04-25T10:07:52.294600+0000","last_became_active":"2024-04-25T10:07:50.306491+0000","last_became_peered":"2024-04-25T10:07:50.306491+0000","last_unstale":"2024-04-25T10:07:52.294600+0000","last_undegraded":"2024-04-25T10:07:52.294600+0000","last_fullsized":"2024-04-25T10:07:52.294600+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:43.478815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295998+0000","last_change":"2024-04-25T10:07:51.306158+0000","last_active":"2024-04-25T10:07:52.295998+0000","last_peered":"2024-04-25T10:07:52.295998+0000","last_clean":"2024-04-25T10:07:52.295998+0000","last_became_active":"2024-04-25T10:07:51.305824+0000","last_became_peered":"2024-04-25T10:07:51.305824+0000","last_unstale":"2024-04-25T10:07:52.295998+0000","last_undegraded":"2024-04-25T10:07:52.295998+0000","last_fullsized":"2024-04-25T10:07:52.295998+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:17.474618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292461+0000","last_change":"2024-04-25T10:07:49.292461+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292461+0000","last_undegraded":"2024-04-25T10:07:49.292461+0000","last_fullsized":"2024-04-25T10:07:49.292461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470269+0000","last_change":"2024-04-25T10:07:51.307313+0000","last_active":"2024-04-25T10:07:52.470269+0000","last_peered":"2024-04-25T10:07:52.470269+0000","last_clean":"2024-04-25T10:07:52.470269+0000","last_became_active":"2024-04-25T10:07:51.307032+0000","last_became_peered":"2024-04-25T10:07:51.307032+0000","last_unstale":"2024-04-25T10:07:52.470269+0000","last_undegraded":"2024-04-25T10:07:52.470269+0000","last_fullsized":"2024-04-25T10:07:52.470269+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:51.296503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470393+0000","last_change":"2024-04-25T10:07:50.304944+0000","last_active":"2024-04-25T10:07:52.470393+0000","last_peered":"2024-04-25T10:07:52.470393+0000","last_clean":"2024-04-25T10:07:52.470393+0000","last_became_active":"2024-04-25T10:07:50.304542+0000","last_became_peered":"2024-04-25T10:07:50.304542+0000","last_unstale":"2024-04-25T10:07:52.470393+0000","last_undegraded":"2024-04-25T10:07:52.470393+0000","last_fullsized":"2024-04-25T10:07:52.470393+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:39.934850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295938+0000","last_change":"2024-04-25T10:07:51.320164+0000","last_active":"2024-04-25T10:07:52.295938+0000","last_peered":"2024-04-25T10:07:52.295938+0000","last_clean":"2024-04-25T10:07:52.295938+0000","last_became_active":"2024-04-25T10:07:51.319859+0000","last_became_peered":"2024-04-25T10:07:51.319859+0000","last_unstale":"2024-04-25T10:07:52.295938+0000","last_undegraded":"2024-04-25T10:07:52.295938+0000","last_fullsized":"2024-04-25T10:07:52.295938+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:21.727353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470461+0000","last_change":"2024-04-25T10:07:50.309933+0000","last_active":"2024-04-25T10:07:52.470461+0000","last_peered":"2024-04-25T10:07:52.470461+0000","last_clean":"2024-04-25T10:07:52.470461+0000","last_became_active":"2024-04-25T10:07:50.309637+0000","last_became_peered":"2024-04-25T10:07:50.309637+0000","last_unstale":"2024-04-25T10:07:52.470461+0000","last_undegraded":"2024-04-25T10:07:52.470461+0000","last_fullsized":"2024-04-25T10:07:52.470461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:09.460489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470368+0000","last_change":"2024-04-25T10:07:51.308930+0000","last_active":"2024-04-25T10:07:52.470368+0000","last_peered":"2024-04-25T10:07:52.470368+0000","last_clean":"2024-04-25T10:07:52.470368+0000","last_became_active":"2024-04-25T10:07:51.308603+0000","last_became_peered":"2024-04-25T10:07:51.308603+0000","last_unstale":"2024-04-25T10:07:52.470368+0000","last_undegraded":"2024-04-25T10:07:52.470368+0000","last_fullsized":"2024-04-25T10:07:52.470368+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:17.724785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470020+0000","last_change":"2024-04-25T10:07:50.304956+0000","last_active":"2024-04-25T10:07:52.470020+0000","last_peered":"2024-04-25T10:07:52.470020+0000","last_clean":"2024-04-25T10:07:52.470020+0000","last_became_active":"2024-04-25T10:07:50.304634+0000","last_became_peered":"2024-04-25T10:07:50.304634+0000","last_unstale":"2024-04-25T10:07:52.470020+0000","last_undegraded":"2024-04-25T10:07:52.470020+0000","last_fullsized":"2024-04-25T10:07:52.470020+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:28.896776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470917+0000","last_change":"2024-04-25T10:07:51.314008+0000","last_active":"2024-04-25T10:07:52.470917+0000","last_peered":"2024-04-25T10:07:52.470917+0000","last_clean":"2024-04-25T10:07:52.470917+0000","last_became_active":"2024-04-25T10:07:51.313611+0000","last_became_peered":"2024-04-25T10:07:51.313611+0000","last_unstale":"2024-04-25T10:07:52.470917+0000","last_undegraded":"2024-04-25T10:07:52.470917+0000","last_fullsized":"2024-04-25T10:07:52.470917+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471515+0000","last_change":"2024-04-25T10:07:50.313909+0000","last_active":"2024-04-25T10:07:52.471515+0000","last_peered":"2024-04-25T10:07:52.471515+0000","last_clean":"2024-04-25T10:07:52.471515+0000","last_became_active":"2024-04-25T10:07:50.313400+0000","last_became_peered":"2024-04-25T10:07:50.313400+0000","last_unstale":"2024-04-25T10:07:52.471515+0000","last_undegraded":"2024-04-25T10:07:52.471515+0000","last_fullsized":"2024-04-25T10:07:52.471515+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:09.642875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471017+0000","last_change":"2024-04-25T10:07:51.310494+0000","last_active":"2024-04-25T10:07:52.471017+0000","last_peered":"2024-04-25T10:07:52.471017+0000","last_clean":"2024-04-25T10:07:52.471017+0000","last_became_active":"2024-04-25T10:07:51.310224+0000","last_became_peered":"2024-04-25T10:07:51.310224+0000","last_unstale":"2024-04-25T10:07:52.471017+0000","last_undegraded":"2024-04-25T10:07:52.471017+0000","last_fullsized":"2024-04-25T10:07:52.471017+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292480+0000","last_change":"2024-04-25T10:07:49.292480+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292480+0000","last_undegraded":"2024-04-25T10:07:49.292480+0000","last_fullsized":"2024-04-25T10:07:49.292480+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471787+0000","last_change":"2024-04-25T10:07:51.318600+0000","last_active":"2024-04-25T10:07:52.471787+0000","last_peered":"2024-04-25T10:07:52.471787+0000","last_clean":"2024-04-25T10:07:52.471787+0000","last_became_active":"2024-04-25T10:07:51.318304+0000","last_became_peered":"2024-04-25T10:07:51.318304+0000","last_unstale":"2024-04-25T10:07:52.471787+0000","last_undegraded":"2024-04-25T10:07:52.471787+0000","last_fullsized":"2024-04-25T10:07:52.471787+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:53.592626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470762+0000","last_change":"2024-04-25T10:07:50.304446+0000","last_active":"2024-04-25T10:07:52.470762+0000","last_peered":"2024-04-25T10:07:52.470762+0000","last_clean":"2024-04-25T10:07:52.470762+0000","last_became_active":"2024-04-25T10:07:50.303898+0000","last_became_peered":"2024-04-25T10:07:50.303898+0000","last_unstale":"2024-04-25T10:07:52.470762+0000","last_undegraded":"2024-04-25T10:07:52.470762+0000","last_fullsized":"2024-04-25T10:07:52.470762+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:23.749111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470626+0000","last_change":"2024-04-25T10:07:51.319204+0000","last_active":"2024-04-25T10:07:52.470626+0000","last_peered":"2024-04-25T10:07:52.470626+0000","last_clean":"2024-04-25T10:07:52.470626+0000","last_became_active":"2024-04-25T10:07:51.318367+0000","last_became_peered":"2024-04-25T10:07:51.318367+0000","last_unstale":"2024-04-25T10:07:52.470626+0000","last_undegraded":"2024-04-25T10:07:52.470626+0000","last_fullsized":"2024-04-25T10:07:52.470626+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:46.811230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292499+0000","last_change":"2024-04-25T10:07:49.292499+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292499+0000","last_undegraded":"2024-04-25T10:07:49.292499+0000","last_fullsized":"2024-04-25T10:07:49.292499+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296169+0000","last_change":"2024-04-25T10:07:51.312843+0000","last_active":"2024-04-25T10:07:52.296169+0000","last_peered":"2024-04-25T10:07:52.296169+0000","last_clean":"2024-04-25T10:07:52.296169+0000","last_became_active":"2024-04-25T10:07:51.311808+0000","last_became_peered":"2024-04-25T10:07:51.311808+0000","last_unstale":"2024-04-25T10:07:52.296169+0000","last_undegraded":"2024-04-25T10:07:52.296169+0000","last_fullsized":"2024-04-25T10:07:52.296169+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:18.847819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296623+0000","last_change":"2024-04-25T10:07:50.306664+0000","last_active":"2024-04-25T10:07:52.296623+0000","last_peered":"2024-04-25T10:07:52.296623+0000","last_clean":"2024-04-25T10:07:52.296623+0000","last_became_active":"2024-04-25T10:07:50.306383+0000","last_became_peered":"2024-04-25T10:07:50.306383+0000","last_unstale":"2024-04-25T10:07:52.296623+0000","last_undegraded":"2024-04-25T10:07:52.296623+0000","last_fullsized":"2024-04-25T10:07:52.296623+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:10.698506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292457+0000","last_change":"2024-04-25T10:07:49.292457+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292457+0000","last_undegraded":"2024-04-25T10:07:49.292457+0000","last_fullsized":"2024-04-25T10:07:49.292457+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296086+0000","last_change":"2024-04-25T10:07:50.311607+0000","last_active":"2024-04-25T10:07:52.296086+0000","last_peered":"2024-04-25T10:07:52.296086+0000","last_clean":"2024-04-25T10:07:52.296086+0000","last_became_active":"2024-04-25T10:07:50.311300+0000","last_became_peered":"2024-04-25T10:07:50.311300+0000","last_unstale":"2024-04-25T10:07:52.296086+0000","last_undegraded":"2024-04-25T10:07:52.296086+0000","last_fullsized":"2024-04-25T10:07:52.296086+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.109707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471043+0000","last_change":"2024-04-25T10:07:51.314396+0000","last_active":"2024-04-25T10:07:52.471043+0000","last_peered":"2024-04-25T10:07:52.471043+0000","last_clean":"2024-04-25T10:07:52.471043+0000","last_became_active":"2024-04-25T10:07:51.314116+0000","last_became_peered":"2024-04-25T10:07:51.314116+0000","last_unstale":"2024-04-25T10:07:52.471043+0000","last_undegraded":"2024-04-25T10:07:52.471043+0000","last_fullsized":"2024-04-25T10:07:52.471043+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:19.021120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471632+0000","last_change":"2024-04-25T10:07:50.307927+0000","last_active":"2024-04-25T10:07:52.471632+0000","last_peered":"2024-04-25T10:07:52.471632+0000","last_clean":"2024-04-25T10:07:52.471632+0000","last_became_active":"2024-04-25T10:07:50.307641+0000","last_became_peered":"2024-04-25T10:07:50.307641+0000","last_unstale":"2024-04-25T10:07:52.471632+0000","last_undegraded":"2024-04-25T10:07:52.471632+0000","last_fullsized":"2024-04-25T10:07:52.471632+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:22.138626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471021+0000","last_change":"2024-04-25T10:07:51.311790+0000","last_active":"2024-04-25T10:07:52.471021+0000","last_peered":"2024-04-25T10:07:52.471021+0000","last_clean":"2024-04-25T10:07:52.471021+0000","last_became_active":"2024-04-25T10:07:51.311524+0000","last_became_peered":"2024-04-25T10:07:51.311524+0000","last_unstale":"2024-04-25T10:07:52.471021+0000","last_undegraded":"2024-04-25T10:07:52.471021+0000","last_fullsized":"2024-04-25T10:07:52.471021+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.168043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470815+0000","last_change":"2024-04-25T10:07:50.313145+0000","last_active":"2024-04-25T10:07:52.470815+0000","last_peered":"2024-04-25T10:07:52.470815+0000","last_clean":"2024-04-25T10:07:52.470815+0000","last_became_active":"2024-04-25T10:07:50.312860+0000","last_became_peered":"2024-04-25T10:07:50.312860+0000","last_unstale":"2024-04-25T10:07:52.470815+0000","last_undegraded":"2024-04-25T10:07:52.470815+0000","last_fullsized":"2024-04-25T10:07:52.470815+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:16.044211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471897+0000","last_change":"2024-04-25T10:07:51.322690+0000","last_active":"2024-04-25T10:07:52.471897+0000","last_peered":"2024-04-25T10:07:52.471897+0000","last_clean":"2024-04-25T10:07:52.471897+0000","last_became_active":"2024-04-25T10:07:51.322404+0000","last_became_peered":"2024-04-25T10:07:51.322404+0000","last_unstale":"2024-04-25T10:07:52.471897+0000","last_undegraded":"2024-04-25T10:07:52.471897+0000","last_fullsized":"2024-04-25T10:07:52.471897+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:00.981450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296163+0000","last_change":"2024-04-25T10:07:50.309078+0000","last_active":"2024-04-25T10:07:52.296163+0000","last_peered":"2024-04-25T10:07:52.296163+0000","last_clean":"2024-04-25T10:07:52.296163+0000","last_became_active":"2024-04-25T10:07:50.308794+0000","last_became_peered":"2024-04-25T10:07:50.308794+0000","last_unstale":"2024-04-25T10:07:52.296163+0000","last_undegraded":"2024-04-25T10:07:52.296163+0000","last_fullsized":"2024-04-25T10:07:52.296163+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:09.642875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470834+0000","last_change":"2024-04-25T10:07:50.307963+0000","last_active":"2024-04-25T10:07:52.470834+0000","last_peered":"2024-04-25T10:07:52.470834+0000","last_clean":"2024-04-25T10:07:52.470834+0000","last_became_active":"2024-04-25T10:07:50.307611+0000","last_became_peered":"2024-04-25T10:07:50.307611+0000","last_unstale":"2024-04-25T10:07:52.470834+0000","last_undegraded":"2024-04-25T10:07:52.470834+0000","last_fullsized":"2024-04-25T10:07:52.470834+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:13.364471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470886+0000","last_change":"2024-04-25T10:07:51.316433+0000","last_active":"2024-04-25T10:07:52.470886+0000","last_peered":"2024-04-25T10:07:52.470886+0000","last_clean":"2024-04-25T10:07:52.470886+0000","last_became_active":"2024-04-25T10:07:51.315952+0000","last_became_peered":"2024-04-25T10:07:51.315952+0000","last_unstale":"2024-04-25T10:07:52.470886+0000","last_undegraded":"2024-04-25T10:07:52.470886+0000","last_fullsized":"2024-04-25T10:07:52.470886+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:13.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471022+0000","last_change":"2024-04-25T10:07:50.313990+0000","last_active":"2024-04-25T10:07:52.471022+0000","last_peered":"2024-04-25T10:07:52.471022+0000","last_clean":"2024-04-25T10:07:52.471022+0000","last_became_active":"2024-04-25T10:07:50.313700+0000","last_became_peered":"2024-04-25T10:07:50.313700+0000","last_unstale":"2024-04-25T10:07:52.471022+0000","last_undegraded":"2024-04-25T10:07:52.471022+0000","last_fullsized":"2024-04-25T10:07:52.471022+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:46.602681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294542+0000","last_change":"2024-04-25T10:07:51.315938+0000","last_active":"2024-04-25T10:07:52.294542+0000","last_peered":"2024-04-25T10:07:52.294542+0000","last_clean":"2024-04-25T10:07:52.294542+0000","last_became_active":"2024-04-25T10:07:51.315335+0000","last_became_peered":"2024-04-25T10:07:51.315335+0000","last_unstale":"2024-04-25T10:07:52.294542+0000","last_undegraded":"2024-04-25T10:07:52.294542+0000","last_fullsized":"2024-04-25T10:07:52.294542+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470304+0000","last_change":"2024-04-25T10:07:50.306917+0000","last_active":"2024-04-25T10:07:52.470304+0000","last_peered":"2024-04-25T10:07:52.470304+0000","last_clean":"2024-04-25T10:07:52.470304+0000","last_became_active":"2024-04-25T10:07:50.306549+0000","last_became_peered":"2024-04-25T10:07:50.306549+0000","last_unstale":"2024-04-25T10:07:52.470304+0000","last_undegraded":"2024-04-25T10:07:52.470304+0000","last_fullsized":"2024-04-25T10:07:52.470304+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:06.727495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471599+0000","last_change":"2024-04-25T10:07:50.311119+0000","last_active":"2024-04-25T10:07:52.471599+0000","last_peered":"2024-04-25T10:07:52.471599+0000","last_clean":"2024-04-25T10:07:52.471599+0000","last_became_active":"2024-04-25T10:07:50.310843+0000","last_became_peered":"2024-04-25T10:07:50.310843+0000","last_unstale":"2024-04-25T10:07:52.471599+0000","last_undegraded":"2024-04-25T10:07:52.471599+0000","last_fullsized":"2024-04-25T10:07:52.471599+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:26.803283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471140+0000","last_change":"2024-04-25T10:07:51.316546+0000","last_active":"2024-04-25T10:07:52.471140+0000","last_peered":"2024-04-25T10:07:52.471140+0000","last_clean":"2024-04-25T10:07:52.471140+0000","last_became_active":"2024-04-25T10:07:51.313979+0000","last_became_peered":"2024-04-25T10:07:51.313979+0000","last_unstale":"2024-04-25T10:07:52.471140+0000","last_undegraded":"2024-04-25T10:07:52.471140+0000","last_fullsized":"2024-04-25T10:07:52.471140+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:17.474618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-25T10:07:49.292461+0000","last_change":"2024-04-25T10:07:49.292461+0000","last_active":"2024-04-25T10:07:49.282421+0000","last_peered":"2024-04-25T10:07:49.282421+0000","last_clean":"2024-04-25T10:07:49.282421+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:49.292461+0000","last_undegraded":"2024-04-25T10:07:49.292461+0000","last_fullsized":"2024-04-25T10:07:49.292461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:50.284453+0000","last_change":"2024-04-25T10:07:50.284453+0000","last_active":"2024-04-25T10:07:50.284453+0000","last_peered":"2024-04-25T10:07:50.284453+0000","last_clean":"2024-04-25T10:07:50.284453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:50.284453+0000","last_undegraded":"2024-04-25T10:07:50.284453+0000","last_fullsized":"2024-04-25T10:07:50.284453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294591+0000","last_change":"2024-04-25T10:07:50.309026+0000","last_active":"2024-04-25T10:07:52.294591+0000","last_peered":"2024-04-25T10:07:52.294591+0000","last_clean":"2024-04-25T10:07:52.294591+0000","last_became_active":"2024-04-25T10:07:50.308706+0000","last_became_peered":"2024-04-25T10:07:50.308706+0000","last_unstale":"2024-04-25T10:07:52.294591+0000","last_undegraded":"2024-04-25T10:07:52.294591+0000","last_fullsized":"2024-04-25T10:07:52.294591+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:17.725300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":245,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295995+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:07:52.295995+0000","last_peered":"2024-04-25T10:07:52.295995+0000","last_clean":"2024-04-25T10:07:52.295995+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:07:52.295995+0000","last_undegraded":"2024-04-25T10:07:52.295995+0000","last_fullsized":"2024-04-25T10:07:52.295995+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":7},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":288874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":692224,"data_stored":282504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":289398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":692224,"data_stored":282572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":284254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509478,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1298432,"data_stored":879732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":700416,"data_stored":284094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":859800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:56.977 INFO:tasks.ceph.mds.a.smithi016.stderr:2024-04-25T10:07:56.972+0000 7fce9e140640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:07:58.745 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.855+0000 7f0d9604d640 1 Processor -- start 2024-04-25T10:07:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.855+0000 7f0d9604d640 1 -- start start 2024-04-25T10:07:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.856+0000 7f0d9604d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d9015a730 0x7f0d900ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.856+0000 7f0d9604d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0d900b4020 0x7f0d900b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.856+0000 7f0d9604d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0d9006d1b0 0x7f0d9006d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.856+0000 7f0d9604d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d900b49a0 con 0x7f0d900b4020 2024-04-25T10:07:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.856+0000 7f0d9604d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d900cc020 con 0x7f0d9006d1b0 2024-04-25T10:07:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.856+0000 7f0d9604d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900b32b0 con 0x7f0d9015a730 2024-04-25T10:07:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.856+0000 7f0d8effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0d900b4020 0x7f0d900b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.856+0000 7f0d8effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0d900b4020 0x7f0d900b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53184/0 (socket says 172.21.15.16:53184) 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.856+0000 7f0d8effd640 1 -- 172.21.15.16:0/2865605799 learned_addr learned my addr 172.21.15.16:0/2865605799 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.856+0000 7f0d8f7fe640 1 --2- 172.21.15.16:0/2865605799 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d9015a730 0x7f0d900ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.856+0000 7f0d8ffff640 1 --2- 172.21.15.16:0/2865605799 >> v2:172.21.15.186:3300/0 conn(0x7f0d9006d1b0 0x7f0d9006d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.856+0000 7f0d8effd640 1 -- 172.21.15.16:0/2865605799 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d9015a730 msgr2=0x7f0d900ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.856+0000 7f0d8effd640 1 --2- 172.21.15.16:0/2865605799 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d9015a730 0x7f0d900ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.856+0000 7f0d8effd640 1 -- 172.21.15.16:0/2865605799 >> v2:172.21.15.186:3300/0 conn(0x7f0d9006d1b0 msgr2=0x7f0d9006d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.856+0000 7f0d8effd640 1 --2- 172.21.15.16:0/2865605799 >> v2:172.21.15.186:3300/0 conn(0x7f0d9006d1b0 0x7f0d9006d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.856+0000 7f0d8effd640 1 -- 172.21.15.16:0/2865605799 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d900b3590 con 0x7f0d900b4020 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.856+0000 7f0d8ffff640 1 --2- 172.21.15.16:0/2865605799 >> v2:172.21.15.186:3300/0 conn(0x7f0d9006d1b0 0x7f0d9006d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.856+0000 7f0d8f7fe640 1 --2- 172.21.15.16:0/2865605799 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d9015a730 0x7f0d900ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.857+0000 7f0d8effd640 1 --2- 172.21.15.16:0/2865605799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d900b4020 0x7f0d900b4460 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0d9006ba90 tx=0x7f0d8402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6875249d239a9fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.857+0000 7f0d8e7fc640 1 -- 172.21.15.16:0/2865605799 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84037070 con 0x7f0d900b4020 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.857+0000 7f0d8e7fc640 1 -- 172.21.15.16:0/2865605799 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d84032440 con 0x7f0d900b4020 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.857+0000 7f0d8e7fc640 1 -- 172.21.15.16:0/2865605799 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84032810 con 0x7f0d900b4020 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.858+0000 7f0d9604d640 1 -- 172.21.15.16:0/2865605799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d900b4020 msgr2=0x7f0d900b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.858+0000 7f0d9604d640 1 --2- 172.21.15.16:0/2865605799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d900b4020 0x7f0d900b4460 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0d9006ba90 tx=0x7f0d8402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.858+0000 7f0d9604d640 1 -- 172.21.15.16:0/2865605799 shutdown_connections 2024-04-25T10:07:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.858+0000 7f0d9604d640 1 --2- 172.21.15.16:0/2865605799 >> v2:172.21.15.186:3300/0 conn(0x7f0d9006d1b0 0x7f0d9006d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.858+0000 7f0d9604d640 1 --2- 172.21.15.16:0/2865605799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d900b4020 0x7f0d900b4460 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.858+0000 7f0d9604d640 1 --2- 172.21.15.16:0/2865605799 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d9015a730 0x7f0d900ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.858+0000 7f0d9604d640 1 -- 172.21.15.16:0/2865605799 >> 172.21.15.16:0/2865605799 conn(0x7f0d9006aba0 msgr2=0x7f0d9006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.858+0000 7f0d9604d640 1 -- 172.21.15.16:0/2865605799 shutdown_connections 2024-04-25T10:07:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.858+0000 7f0d9604d640 1 -- 172.21.15.16:0/2865605799 wait complete. 2024-04-25T10:07:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.859+0000 7f0d9604d640 1 Processor -- start 2024-04-25T10:07:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.859+0000 7f0d9604d640 1 -- start start 2024-04-25T10:07:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.860+0000 7f0d9604d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9006d1b0 0x7f0d900cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.860+0000 7f0d9604d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d900b4020 0x7f0d9014f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.860+0000 7f0d9604d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d9015a730 0x7f0d901e3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.860+0000 7f0d9604d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900cd620 con 0x7f0d9006d1b0 2024-04-25T10:07:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.860+0000 7f0d9604d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900b3b50 con 0x7f0d900b4020 2024-04-25T10:07:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.860+0000 7f0d9604d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900aeef0 con 0x7f0d9015a730 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.860+0000 7f0d8f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9006d1b0 0x7f0d900cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.860+0000 7f0d8f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9006d1b0 0x7f0d900cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53196/0 (socket says 172.21.15.16:53196) 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.860+0000 7f0d8f7fe640 1 -- 172.21.15.16:0/3657746179 learned_addr learned my addr 172.21.15.16:0/3657746179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.860+0000 7f0d8ffff640 1 --2- 172.21.15.16:0/3657746179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d9015a730 0x7f0d901e3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.860+0000 7f0d8effd640 1 --2- 172.21.15.16:0/3657746179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d900b4020 0x7f0d9014f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.861+0000 7f0d8f7fe640 1 -- 172.21.15.16:0/3657746179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d9015a730 msgr2=0x7f0d901e3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.861+0000 7f0d8f7fe640 1 --2- 172.21.15.16:0/3657746179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d9015a730 0x7f0d901e3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.861+0000 7f0d8f7fe640 1 -- 172.21.15.16:0/3657746179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d900b4020 msgr2=0x7f0d9014f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.861+0000 7f0d8f7fe640 1 --2- 172.21.15.16:0/3657746179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d900b4020 0x7f0d9014f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.861+0000 7f0d8f7fe640 1 -- 172.21.15.16:0/3657746179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d900cf5f0 con 0x7f0d9006d1b0 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.861+0000 7f0d8f7fe640 1 --2- 172.21.15.16:0/3657746179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9006d1b0 0x7f0d900cee80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000d7d0 tx=0x7f0d8000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.861+0000 7f0d77fff640 1 -- 172.21.15.16:0/3657746179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80004030 con 0x7f0d9006d1b0 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.861+0000 7f0d77fff640 1 -- 172.21.15.16:0/3657746179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d800041d0 con 0x7f0d9006d1b0 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.861+0000 7f0d9604d640 1 -- 172.21.15.16:0/3657746179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d900cf8d0 con 0x7f0d9006d1b0 2024-04-25T10:07:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.861+0000 7f0d77fff640 1 -- 172.21.15.16:0/3657746179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8000b6c0 con 0x7f0d9006d1b0 2024-04-25T10:07:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.861+0000 7f0d9604d640 1 -- 172.21.15.16:0/3657746179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d900acd30 con 0x7f0d9006d1b0 2024-04-25T10:07:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.862+0000 7f0d9604d640 1 -- 172.21.15.16:0/3657746179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d900749b0 con 0x7f0d9006d1b0 2024-04-25T10:07:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.863+0000 7f0d77fff640 1 -- 172.21.15.16:0/3657746179 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d8000b860 con 0x7f0d9006d1b0 2024-04-25T10:07:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.863+0000 7f0d77fff640 1 --2- 172.21.15.16:0/3657746179 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0d7c06b4d0 0x7f0d7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.863+0000 7f0d77fff640 1 -- 172.21.15.16:0/3657746179 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d8008a2f0 con 0x7f0d9006d1b0 2024-04-25T10:07:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.863+0000 7f0d8effd640 1 --2- 172.21.15.16:0/3657746179 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0d7c06b4d0 0x7f0d7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.864+0000 7f0d8effd640 1 --2- 172.21.15.16:0/3657746179 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0d7c06b4d0 0x7f0d7c06d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0d84034000 tx=0x7f0d8402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:58.867+0000 7f0d77fff640 1 -- 172.21.15.16:0/3657746179 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d80058b20 con 0x7f0d9006d1b0 2024-04-25T10:07:59.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.044+0000 7f0d9604d640 1 -- 172.21.15.16:0/3657746179 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d900b4580 con 0x7f0d7c06b4d0 2024-04-25T10:07:59.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.050+0000 7f0d77fff640 1 -- 172.21.15.16:0/3657746179 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251871 (secure 0 0 0) 0x7f0d900b4580 con 0x7f0d7c06b4d0 2024-04-25T10:07:59.059 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:59.060 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:07:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.053+0000 7f0d9604d640 1 -- 172.21.15.16:0/3657746179 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0d7c06b4d0 msgr2=0x7f0d7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.053+0000 7f0d9604d640 1 --2- 172.21.15.16:0/3657746179 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0d7c06b4d0 0x7f0d7c06d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0d84034000 tx=0x7f0d8402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.053+0000 7f0d9604d640 1 -- 172.21.15.16:0/3657746179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9006d1b0 msgr2=0x7f0d900cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.053+0000 7f0d9604d640 1 --2- 172.21.15.16:0/3657746179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9006d1b0 0x7f0d900cee80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000d7d0 tx=0x7f0d8000dc90 comp rx=0 tx=0).stop 2024-04-25T10:07:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.053+0000 7f0d9604d640 1 -- 172.21.15.16:0/3657746179 shutdown_connections 2024-04-25T10:07:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.053+0000 7f0d9604d640 1 --2- 172.21.15.16:0/3657746179 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0d7c06b4d0 0x7f0d7c06d990 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.053+0000 7f0d9604d640 1 --2- 172.21.15.16:0/3657746179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d9015a730 0x7f0d901e3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.053+0000 7f0d9604d640 1 --2- 172.21.15.16:0/3657746179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d900b4020 0x7f0d9014f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.053+0000 7f0d9604d640 1 --2- 172.21.15.16:0/3657746179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9006d1b0 0x7f0d900cee80 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.053+0000 7f0d9604d640 1 -- 172.21.15.16:0/3657746179 >> 172.21.15.16:0/3657746179 conn(0x7f0d9006aba0 msgr2=0x7f0d9006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.053+0000 7f0d9604d640 1 -- 172.21.15.16:0/3657746179 shutdown_connections 2024-04-25T10:07:59.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.053+0000 7f0d9604d640 1 -- 172.21.15.16:0/3657746179 wait complete. 2024-04-25T10:07:59.070 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-04-25T10:07:57.819478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243712,"kb_used_data":6696,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731008,"statfs":{"total":773094113280,"available":772844552192,"internally_reserved":0,"allocated":6856704,"data_stored":3495812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.524631"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.472064+0000","last_change":"2024-04-25T10:07:51.311604+0000","last_active":"2024-04-25T10:07:52.472064+0000","last_peered":"2024-04-25T10:07:52.472064+0000","last_clean":"2024-04-25T10:07:52.472064+0000","last_became_active":"2024-04-25T10:07:51.310395+0000","last_became_peered":"2024-04-25T10:07:51.310395+0000","last_unstale":"2024-04-25T10:07:52.472064+0000","last_undegraded":"2024-04-25T10:07:52.472064+0000","last_fullsized":"2024-04-25T10:07:52.472064+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471748+0000","last_change":"2024-04-25T10:07:51.315131+0000","last_active":"2024-04-25T10:07:52.471748+0000","last_peered":"2024-04-25T10:07:52.471748+0000","last_clean":"2024-04-25T10:07:52.471748+0000","last_became_active":"2024-04-25T10:07:51.314869+0000","last_became_peered":"2024-04-25T10:07:51.314869+0000","last_unstale":"2024-04-25T10:07:52.471748+0000","last_undegraded":"2024-04-25T10:07:52.471748+0000","last_fullsized":"2024-04-25T10:07:52.471748+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:38.125900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470577+0000","last_change":"2024-04-25T10:07:51.303339+0000","last_active":"2024-04-25T10:07:52.470577+0000","last_peered":"2024-04-25T10:07:52.470577+0000","last_clean":"2024-04-25T10:07:52.470577+0000","last_became_active":"2024-04-25T10:07:51.303082+0000","last_became_peered":"2024-04-25T10:07:51.303082+0000","last_unstale":"2024-04-25T10:07:52.470577+0000","last_undegraded":"2024-04-25T10:07:52.470577+0000","last_fullsized":"2024-04-25T10:07:52.470577+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:16.208395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470265+0000","last_change":"2024-04-25T10:07:51.318977+0000","last_active":"2024-04-25T10:07:52.470265+0000","last_peered":"2024-04-25T10:07:52.470265+0000","last_clean":"2024-04-25T10:07:52.470265+0000","last_became_active":"2024-04-25T10:07:51.318704+0000","last_became_peered":"2024-04-25T10:07:51.318704+0000","last_unstale":"2024-04-25T10:07:52.470265+0000","last_undegraded":"2024-04-25T10:07:52.470265+0000","last_fullsized":"2024-04-25T10:07:52.470265+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:01.127872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471710+0000","last_change":"2024-04-25T10:07:51.309203+0000","last_active":"2024-04-25T10:07:52.471710+0000","last_peered":"2024-04-25T10:07:52.471710+0000","last_clean":"2024-04-25T10:07:52.471710+0000","last_became_active":"2024-04-25T10:07:51.308918+0000","last_became_peered":"2024-04-25T10:07:51.308918+0000","last_unstale":"2024-04-25T10:07:52.471710+0000","last_undegraded":"2024-04-25T10:07:52.471710+0000","last_fullsized":"2024-04-25T10:07:52.471710+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470720+0000","last_change":"2024-04-25T10:07:51.317240+0000","last_active":"2024-04-25T10:07:52.470720+0000","last_peered":"2024-04-25T10:07:52.470720+0000","last_clean":"2024-04-25T10:07:52.470720+0000","last_became_active":"2024-04-25T10:07:51.316735+0000","last_became_peered":"2024-04-25T10:07:51.316735+0000","last_unstale":"2024-04-25T10:07:52.470720+0000","last_undegraded":"2024-04-25T10:07:52.470720+0000","last_fullsized":"2024-04-25T10:07:52.470720+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:49.018249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470088+0000","last_change":"2024-04-25T10:07:51.310589+0000","last_active":"2024-04-25T10:07:52.470088+0000","last_peered":"2024-04-25T10:07:52.470088+0000","last_clean":"2024-04-25T10:07:52.470088+0000","last_became_active":"2024-04-25T10:07:51.310211+0000","last_became_peered":"2024-04-25T10:07:51.310211+0000","last_unstale":"2024-04-25T10:07:52.470088+0000","last_undegraded":"2024-04-25T10:07:52.470088+0000","last_fullsized":"2024-04-25T10:07:52.470088+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:16.223783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471105+0000","last_change":"2024-04-25T10:07:51.308827+0000","last_active":"2024-04-25T10:07:52.471105+0000","last_peered":"2024-04-25T10:07:52.471105+0000","last_clean":"2024-04-25T10:07:52.471105+0000","last_became_active":"2024-04-25T10:07:51.308515+0000","last_became_peered":"2024-04-25T10:07:51.308515+0000","last_unstale":"2024-04-25T10:07:52.471105+0000","last_undegraded":"2024-04-25T10:07:52.471105+0000","last_fullsized":"2024-04-25T10:07:52.471105+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:13.878575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296172+0000","last_change":"2024-04-25T10:07:51.312254+0000","last_active":"2024-04-25T10:07:52.296172+0000","last_peered":"2024-04-25T10:07:52.296172+0000","last_clean":"2024-04-25T10:07:52.296172+0000","last_became_active":"2024-04-25T10:07:51.311901+0000","last_became_peered":"2024-04-25T10:07:51.311901+0000","last_unstale":"2024-04-25T10:07:52.296172+0000","last_undegraded":"2024-04-25T10:07:52.296172+0000","last_fullsized":"2024-04-25T10:07:52.296172+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:42.400180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470490+0000","last_change":"2024-04-25T10:07:51.317615+0000","last_active":"2024-04-25T10:07:52.470490+0000","last_peered":"2024-04-25T10:07:52.470490+0000","last_clean":"2024-04-25T10:07:52.470490+0000","last_became_active":"2024-04-25T10:07:51.317131+0000","last_became_peered":"2024-04-25T10:07:51.317131+0000","last_unstale":"2024-04-25T10:07:52.470490+0000","last_undegraded":"2024-04-25T10:07:52.470490+0000","last_fullsized":"2024-04-25T10:07:52.470490+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:19.853622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296651+0000","last_change":"2024-04-25T10:07:51.323605+0000","last_active":"2024-04-25T10:07:52.296651+0000","last_peered":"2024-04-25T10:07:52.296651+0000","last_clean":"2024-04-25T10:07:52.296651+0000","last_became_active":"2024-04-25T10:07:51.323306+0000","last_became_peered":"2024-04-25T10:07:51.323306+0000","last_unstale":"2024-04-25T10:07:52.296651+0000","last_undegraded":"2024-04-25T10:07:52.296651+0000","last_fullsized":"2024-04-25T10:07:52.296651+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.168043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296163+0000","last_change":"2024-04-25T10:07:51.312425+0000","last_active":"2024-04-25T10:07:52.296163+0000","last_peered":"2024-04-25T10:07:52.296163+0000","last_clean":"2024-04-25T10:07:52.296163+0000","last_became_active":"2024-04-25T10:07:51.312133+0000","last_became_peered":"2024-04-25T10:07:51.312133+0000","last_unstale":"2024-04-25T10:07:52.296163+0000","last_undegraded":"2024-04-25T10:07:52.296163+0000","last_fullsized":"2024-04-25T10:07:52.296163+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.404296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294383+0000","last_change":"2024-04-25T10:07:51.317166+0000","last_active":"2024-04-25T10:07:52.294383+0000","last_peered":"2024-04-25T10:07:52.294383+0000","last_clean":"2024-04-25T10:07:52.294383+0000","last_became_active":"2024-04-25T10:07:51.315513+0000","last_became_peered":"2024-04-25T10:07:51.315513+0000","last_unstale":"2024-04-25T10:07:52.294383+0000","last_undegraded":"2024-04-25T10:07:52.294383+0000","last_fullsized":"2024-04-25T10:07:52.294383+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:51.296503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470214+0000","last_change":"2024-04-25T10:07:51.313310+0000","last_active":"2024-04-25T10:07:52.470214+0000","last_peered":"2024-04-25T10:07:52.470214+0000","last_clean":"2024-04-25T10:07:52.470214+0000","last_became_active":"2024-04-25T10:07:51.313001+0000","last_became_peered":"2024-04-25T10:07:51.313001+0000","last_unstale":"2024-04-25T10:07:52.470214+0000","last_undegraded":"2024-04-25T10:07:52.470214+0000","last_fullsized":"2024-04-25T10:07:52.470214+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:13.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294437+0000","last_change":"2024-04-25T10:07:51.317685+0000","last_active":"2024-04-25T10:07:52.294437+0000","last_peered":"2024-04-25T10:07:52.294437+0000","last_clean":"2024-04-25T10:07:52.294437+0000","last_became_active":"2024-04-25T10:07:51.317386+0000","last_became_peered":"2024-04-25T10:07:51.317386+0000","last_unstale":"2024-04-25T10:07:52.294437+0000","last_undegraded":"2024-04-25T10:07:52.294437+0000","last_fullsized":"2024-04-25T10:07:52.294437+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:16.208395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470284+0000","last_change":"2024-04-25T10:07:51.306911+0000","last_active":"2024-04-25T10:07:52.470284+0000","last_peered":"2024-04-25T10:07:52.470284+0000","last_clean":"2024-04-25T10:07:52.470284+0000","last_became_active":"2024-04-25T10:07:51.306636+0000","last_became_peered":"2024-04-25T10:07:51.306636+0000","last_unstale":"2024-04-25T10:07:52.470284+0000","last_undegraded":"2024-04-25T10:07:52.470284+0000","last_fullsized":"2024-04-25T10:07:52.470284+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:57.699968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.472107+0000","last_change":"2024-04-25T10:07:51.304905+0000","last_active":"2024-04-25T10:07:52.472107+0000","last_peered":"2024-04-25T10:07:52.472107+0000","last_clean":"2024-04-25T10:07:52.472107+0000","last_became_active":"2024-04-25T10:07:51.304584+0000","last_became_peered":"2024-04-25T10:07:51.304584+0000","last_unstale":"2024-04-25T10:07:52.472107+0000","last_undegraded":"2024-04-25T10:07:52.472107+0000","last_fullsized":"2024-04-25T10:07:52.472107+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470893+0000","last_change":"2024-04-25T10:07:51.313605+0000","last_active":"2024-04-25T10:07:52.470893+0000","last_peered":"2024-04-25T10:07:52.470893+0000","last_clean":"2024-04-25T10:07:52.470893+0000","last_became_active":"2024-04-25T10:07:51.313265+0000","last_became_peered":"2024-04-25T10:07:51.313265+0000","last_unstale":"2024-04-25T10:07:52.470893+0000","last_undegraded":"2024-04-25T10:07:52.470893+0000","last_fullsized":"2024-04-25T10:07:52.470893+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471365+0000","last_change":"2024-04-25T10:07:51.312675+0000","last_active":"2024-04-25T10:07:52.471365+0000","last_peered":"2024-04-25T10:07:52.471365+0000","last_clean":"2024-04-25T10:07:52.471365+0000","last_became_active":"2024-04-25T10:07:51.311781+0000","last_became_peered":"2024-04-25T10:07:51.311781+0000","last_unstale":"2024-04-25T10:07:52.471365+0000","last_undegraded":"2024-04-25T10:07:52.471365+0000","last_fullsized":"2024-04-25T10:07:52.471365+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.404296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296412+0000","last_change":"2024-04-25T10:07:51.322910+0000","last_active":"2024-04-25T10:07:52.296412+0000","last_peered":"2024-04-25T10:07:52.296412+0000","last_clean":"2024-04-25T10:07:52.296412+0000","last_became_active":"2024-04-25T10:07:51.322604+0000","last_became_peered":"2024-04-25T10:07:51.322604+0000","last_unstale":"2024-04-25T10:07:52.296412+0000","last_undegraded":"2024-04-25T10:07:52.296412+0000","last_fullsized":"2024-04-25T10:07:52.296412+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294454+0000","last_change":"2024-04-25T10:07:51.316521+0000","last_active":"2024-04-25T10:07:52.294454+0000","last_peered":"2024-04-25T10:07:52.294454+0000","last_clean":"2024-04-25T10:07:52.294454+0000","last_became_active":"2024-04-25T10:07:51.316238+0000","last_became_peered":"2024-04-25T10:07:51.316238+0000","last_unstale":"2024-04-25T10:07:52.294454+0000","last_undegraded":"2024-04-25T10:07:52.294454+0000","last_fullsized":"2024-04-25T10:07:52.294454+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296471+0000","last_change":"2024-04-25T10:07:51.312907+0000","last_active":"2024-04-25T10:07:52.296471+0000","last_peered":"2024-04-25T10:07:52.296471+0000","last_clean":"2024-04-25T10:07:52.296471+0000","last_became_active":"2024-04-25T10:07:51.312595+0000","last_became_peered":"2024-04-25T10:07:51.312595+0000","last_unstale":"2024-04-25T10:07:52.296471+0000","last_undegraded":"2024-04-25T10:07:52.296471+0000","last_fullsized":"2024-04-25T10:07:52.296471+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470232+0000","last_change":"2024-04-25T10:07:51.309738+0000","last_active":"2024-04-25T10:07:52.470232+0000","last_peered":"2024-04-25T10:07:52.470232+0000","last_clean":"2024-04-25T10:07:52.470232+0000","last_became_active":"2024-04-25T10:07:51.309394+0000","last_became_peered":"2024-04-25T10:07:51.309394+0000","last_unstale":"2024-04-25T10:07:52.470232+0000","last_undegraded":"2024-04-25T10:07:52.470232+0000","last_fullsized":"2024-04-25T10:07:52.470232+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:51.800823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295906+0000","last_change":"2024-04-25T10:07:51.311322+0000","last_active":"2024-04-25T10:07:52.295906+0000","last_peered":"2024-04-25T10:07:52.295906+0000","last_clean":"2024-04-25T10:07:52.295906+0000","last_became_active":"2024-04-25T10:07:51.311041+0000","last_became_peered":"2024-04-25T10:07:51.311041+0000","last_unstale":"2024-04-25T10:07:52.295906+0000","last_undegraded":"2024-04-25T10:07:52.295906+0000","last_fullsized":"2024-04-25T10:07:52.295906+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:05.899651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471407+0000","last_change":"2024-04-25T10:07:51.317497+0000","last_active":"2024-04-25T10:07:52.471407+0000","last_peered":"2024-04-25T10:07:52.471407+0000","last_clean":"2024-04-25T10:07:52.471407+0000","last_became_active":"2024-04-25T10:07:51.317196+0000","last_became_peered":"2024-04-25T10:07:51.317196+0000","last_unstale":"2024-04-25T10:07:52.471407+0000","last_undegraded":"2024-04-25T10:07:52.471407+0000","last_fullsized":"2024-04-25T10:07:52.471407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:18.847819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294675+0000","last_change":"2024-04-25T10:07:51.324223+0000","last_active":"2024-04-25T10:07:52.294675+0000","last_peered":"2024-04-25T10:07:52.294675+0000","last_clean":"2024-04-25T10:07:52.294675+0000","last_became_active":"2024-04-25T10:07:51.323857+0000","last_became_peered":"2024-04-25T10:07:51.323857+0000","last_unstale":"2024-04-25T10:07:52.294675+0000","last_undegraded":"2024-04-25T10:07:52.294675+0000","last_fullsized":"2024-04-25T10:07:52.294675+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471513+0000","last_change":"2024-04-25T10:07:51.319543+0000","last_active":"2024-04-25T10:07:52.471513+0000","last_peered":"2024-04-25T10:07:52.471513+0000","last_clean":"2024-04-25T10:07:52.471513+0000","last_became_active":"2024-04-25T10:07:51.319265+0000","last_became_peered":"2024-04-25T10:07:51.319265+0000","last_unstale":"2024-04-25T10:07:52.471513+0000","last_undegraded":"2024-04-25T10:07:52.471513+0000","last_fullsized":"2024-04-25T10:07:52.471513+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:42.400180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470498+0000","last_change":"2024-04-25T10:07:51.319809+0000","last_active":"2024-04-25T10:07:52.470498+0000","last_peered":"2024-04-25T10:07:52.470498+0000","last_clean":"2024-04-25T10:07:52.470498+0000","last_became_active":"2024-04-25T10:07:51.319540+0000","last_became_peered":"2024-04-25T10:07:51.319540+0000","last_unstale":"2024-04-25T10:07:52.470498+0000","last_undegraded":"2024-04-25T10:07:52.470498+0000","last_fullsized":"2024-04-25T10:07:52.470498+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:53.867684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470796+0000","last_change":"2024-04-25T10:07:51.311163+0000","last_active":"2024-04-25T10:07:52.470796+0000","last_peered":"2024-04-25T10:07:52.470796+0000","last_clean":"2024-04-25T10:07:52.470796+0000","last_became_active":"2024-04-25T10:07:51.310892+0000","last_became_peered":"2024-04-25T10:07:51.310892+0000","last_unstale":"2024-04-25T10:07:52.470796+0000","last_undegraded":"2024-04-25T10:07:52.470796+0000","last_fullsized":"2024-04-25T10:07:52.470796+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:14.283525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470296+0000","last_change":"2024-04-25T10:07:51.318244+0000","last_active":"2024-04-25T10:07:52.470296+0000","last_peered":"2024-04-25T10:07:52.470296+0000","last_clean":"2024-04-25T10:07:52.470296+0000","last_became_active":"2024-04-25T10:07:51.317253+0000","last_became_peered":"2024-04-25T10:07:51.317253+0000","last_unstale":"2024-04-25T10:07:52.470296+0000","last_undegraded":"2024-04-25T10:07:52.470296+0000","last_fullsized":"2024-04-25T10:07:52.470296+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:17.235813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470996+0000","last_change":"2024-04-25T10:07:51.313123+0000","last_active":"2024-04-25T10:07:52.470996+0000","last_peered":"2024-04-25T10:07:52.470996+0000","last_clean":"2024-04-25T10:07:52.470996+0000","last_became_active":"2024-04-25T10:07:51.312765+0000","last_became_peered":"2024-04-25T10:07:51.312765+0000","last_unstale":"2024-04-25T10:07:52.470996+0000","last_undegraded":"2024-04-25T10:07:52.470996+0000","last_fullsized":"2024-04-25T10:07:52.470996+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:21.727353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296245+0000","last_change":"2024-04-25T10:07:51.312907+0000","last_active":"2024-04-25T10:07:52.296245+0000","last_peered":"2024-04-25T10:07:52.296245+0000","last_clean":"2024-04-25T10:07:52.296245+0000","last_became_active":"2024-04-25T10:07:51.312558+0000","last_became_peered":"2024-04-25T10:07:51.312558+0000","last_unstale":"2024-04-25T10:07:52.296245+0000","last_undegraded":"2024-04-25T10:07:52.296245+0000","last_fullsized":"2024-04-25T10:07:52.296245+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471064+0000","last_change":"2024-04-25T10:07:51.298328+0000","last_active":"2024-04-25T10:07:52.471064+0000","last_peered":"2024-04-25T10:07:52.471064+0000","last_clean":"2024-04-25T10:07:52.471064+0000","last_became_active":"2024-04-25T10:07:51.297878+0000","last_became_peered":"2024-04-25T10:07:51.297878+0000","last_unstale":"2024-04-25T10:07:52.471064+0000","last_undegraded":"2024-04-25T10:07:52.471064+0000","last_fullsized":"2024-04-25T10:07:52.471064+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:51.296503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296580+0000","last_change":"2024-04-25T10:07:50.306577+0000","last_active":"2024-04-25T10:07:52.296580+0000","last_peered":"2024-04-25T10:07:52.296580+0000","last_clean":"2024-04-25T10:07:52.296580+0000","last_became_active":"2024-04-25T10:07:50.306291+0000","last_became_peered":"2024-04-25T10:07:50.306291+0000","last_unstale":"2024-04-25T10:07:52.296580+0000","last_undegraded":"2024-04-25T10:07:52.296580+0000","last_fullsized":"2024-04-25T10:07:52.296580+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:23.427143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.472178+0000","last_change":"2024-04-25T10:07:51.316840+0000","last_active":"2024-04-25T10:07:52.472178+0000","last_peered":"2024-04-25T10:07:52.472178+0000","last_clean":"2024-04-25T10:07:52.472178+0000","last_became_active":"2024-04-25T10:07:51.316519+0000","last_became_peered":"2024-04-25T10:07:51.316519+0000","last_unstale":"2024-04-25T10:07:52.472178+0000","last_undegraded":"2024-04-25T10:07:52.472178+0000","last_fullsized":"2024-04-25T10:07:52.472178+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.168043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471882+0000","last_change":"2024-04-25T10:07:50.306264+0000","last_active":"2024-04-25T10:07:52.471882+0000","last_peered":"2024-04-25T10:07:52.471882+0000","last_clean":"2024-04-25T10:07:52.471882+0000","last_became_active":"2024-04-25T10:07:50.305960+0000","last_became_peered":"2024-04-25T10:07:50.305960+0000","last_unstale":"2024-04-25T10:07:52.471882+0000","last_undegraded":"2024-04-25T10:07:52.471882+0000","last_fullsized":"2024-04-25T10:07:52.471882+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:02.311727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470673+0000","last_change":"2024-04-25T10:07:51.319534+0000","last_active":"2024-04-25T10:07:52.470673+0000","last_peered":"2024-04-25T10:07:52.470673+0000","last_clean":"2024-04-25T10:07:52.470673+0000","last_became_active":"2024-04-25T10:07:51.319127+0000","last_became_peered":"2024-04-25T10:07:51.319127+0000","last_unstale":"2024-04-25T10:07:52.470673+0000","last_undegraded":"2024-04-25T10:07:52.470673+0000","last_fullsized":"2024-04-25T10:07:52.470673+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:10.191590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471542+0000","last_change":"2024-04-25T10:07:50.311222+0000","last_active":"2024-04-25T10:07:52.471542+0000","last_peered":"2024-04-25T10:07:52.471542+0000","last_clean":"2024-04-25T10:07:52.471542+0000","last_became_active":"2024-04-25T10:07:50.310954+0000","last_became_peered":"2024-04-25T10:07:50.310954+0000","last_unstale":"2024-04-25T10:07:52.471542+0000","last_undegraded":"2024-04-25T10:07:52.471542+0000","last_fullsized":"2024-04-25T10:07:52.471542+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:44.898508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471321+0000","last_change":"2024-04-25T10:07:51.312039+0000","last_active":"2024-04-25T10:07:52.471321+0000","last_peered":"2024-04-25T10:07:52.471321+0000","last_clean":"2024-04-25T10:07:52.471321+0000","last_became_active":"2024-04-25T10:07:51.305475+0000","last_became_peered":"2024-04-25T10:07:51.305475+0000","last_unstale":"2024-04-25T10:07:52.471321+0000","last_undegraded":"2024-04-25T10:07:52.471321+0000","last_fullsized":"2024-04-25T10:07:52.471321+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470435+0000","last_change":"2024-04-25T10:07:50.309547+0000","last_active":"2024-04-25T10:07:52.470435+0000","last_peered":"2024-04-25T10:07:52.470435+0000","last_clean":"2024-04-25T10:07:52.470435+0000","last_became_active":"2024-04-25T10:07:50.309240+0000","last_became_peered":"2024-04-25T10:07:50.309240+0000","last_unstale":"2024-04-25T10:07:52.470435+0000","last_undegraded":"2024-04-25T10:07:52.470435+0000","last_fullsized":"2024-04-25T10:07:52.470435+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:23.723920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470541+0000","last_change":"2024-04-25T10:07:51.309022+0000","last_active":"2024-04-25T10:07:52.470541+0000","last_peered":"2024-04-25T10:07:52.470541+0000","last_clean":"2024-04-25T10:07:52.470541+0000","last_became_active":"2024-04-25T10:07:51.308686+0000","last_became_peered":"2024-04-25T10:07:51.308686+0000","last_unstale":"2024-04-25T10:07:52.470541+0000","last_undegraded":"2024-04-25T10:07:52.470541+0000","last_fullsized":"2024-04-25T10:07:52.470541+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:13.951988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470369+0000","last_change":"2024-04-25T10:07:50.306966+0000","last_active":"2024-04-25T10:07:52.470369+0000","last_peered":"2024-04-25T10:07:52.470369+0000","last_clean":"2024-04-25T10:07:52.470369+0000","last_became_active":"2024-04-25T10:07:50.306613+0000","last_became_peered":"2024-04-25T10:07:50.306613+0000","last_unstale":"2024-04-25T10:07:52.470369+0000","last_undegraded":"2024-04-25T10:07:52.470369+0000","last_fullsized":"2024-04-25T10:07:52.470369+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:16.044211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296418+0000","last_change":"2024-04-25T10:07:51.319075+0000","last_active":"2024-04-25T10:07:52.296418+0000","last_peered":"2024-04-25T10:07:52.296418+0000","last_clean":"2024-04-25T10:07:52.296418+0000","last_became_active":"2024-04-25T10:07:51.318802+0000","last_became_peered":"2024-04-25T10:07:51.318802+0000","last_unstale":"2024-04-25T10:07:52.296418+0000","last_undegraded":"2024-04-25T10:07:52.296418+0000","last_fullsized":"2024-04-25T10:07:52.296418+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470993+0000","last_change":"2024-04-25T10:07:50.311086+0000","last_active":"2024-04-25T10:07:52.470993+0000","last_peered":"2024-04-25T10:07:52.470993+0000","last_clean":"2024-04-25T10:07:52.470993+0000","last_became_active":"2024-04-25T10:07:50.310798+0000","last_became_peered":"2024-04-25T10:07:50.310798+0000","last_unstale":"2024-04-25T10:07:52.470993+0000","last_undegraded":"2024-04-25T10:07:52.470993+0000","last_fullsized":"2024-04-25T10:07:52.470993+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:46.162970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.469999+0000","last_change":"2024-04-25T10:07:51.305304+0000","last_active":"2024-04-25T10:07:52.469999+0000","last_peered":"2024-04-25T10:07:52.469999+0000","last_clean":"2024-04-25T10:07:52.469999+0000","last_became_active":"2024-04-25T10:07:51.304968+0000","last_became_peered":"2024-04-25T10:07:51.304968+0000","last_unstale":"2024-04-25T10:07:52.469999+0000","last_undegraded":"2024-04-25T10:07:52.469999+0000","last_fullsized":"2024-04-25T10:07:52.469999+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:49.018249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295971+0000","last_change":"2024-04-25T10:07:50.305140+0000","last_active":"2024-04-25T10:07:52.295971+0000","last_peered":"2024-04-25T10:07:52.295971+0000","last_clean":"2024-04-25T10:07:52.295971+0000","last_became_active":"2024-04-25T10:07:50.304795+0000","last_became_peered":"2024-04-25T10:07:50.304795+0000","last_unstale":"2024-04-25T10:07:52.295971+0000","last_undegraded":"2024-04-25T10:07:52.295971+0000","last_fullsized":"2024-04-25T10:07:52.295971+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:22.138626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471236+0000","last_change":"2024-04-25T10:07:51.317275+0000","last_active":"2024-04-25T10:07:52.471236+0000","last_peered":"2024-04-25T10:07:52.471236+0000","last_clean":"2024-04-25T10:07:52.471236+0000","last_became_active":"2024-04-25T10:07:51.316910+0000","last_became_peered":"2024-04-25T10:07:51.316910+0000","last_unstale":"2024-04-25T10:07:52.471236+0000","last_undegraded":"2024-04-25T10:07:52.471236+0000","last_fullsized":"2024-04-25T10:07:52.471236+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:16.208395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294270+0000","last_change":"2024-04-25T10:07:50.307893+0000","last_active":"2024-04-25T10:07:52.294270+0000","last_peered":"2024-04-25T10:07:52.294270+0000","last_clean":"2024-04-25T10:07:52.294270+0000","last_became_active":"2024-04-25T10:07:50.307514+0000","last_became_peered":"2024-04-25T10:07:50.307514+0000","last_unstale":"2024-04-25T10:07:52.294270+0000","last_undegraded":"2024-04-25T10:07:52.294270+0000","last_fullsized":"2024-04-25T10:07:52.294270+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:33.584068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294636+0000","last_change":"2024-04-25T10:07:51.317446+0000","last_active":"2024-04-25T10:07:52.294636+0000","last_peered":"2024-04-25T10:07:52.294636+0000","last_clean":"2024-04-25T10:07:52.294636+0000","last_became_active":"2024-04-25T10:07:51.315123+0000","last_became_peered":"2024-04-25T10:07:51.315123+0000","last_unstale":"2024-04-25T10:07:52.294636+0000","last_undegraded":"2024-04-25T10:07:52.294636+0000","last_fullsized":"2024-04-25T10:07:52.294636+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296218+0000","last_change":"2024-04-25T10:07:50.306714+0000","last_active":"2024-04-25T10:07:52.296218+0000","last_peered":"2024-04-25T10:07:52.296218+0000","last_clean":"2024-04-25T10:07:52.296218+0000","last_became_active":"2024-04-25T10:07:50.306439+0000","last_became_peered":"2024-04-25T10:07:50.306439+0000","last_unstale":"2024-04-25T10:07:52.296218+0000","last_undegraded":"2024-04-25T10:07:52.296218+0000","last_fullsized":"2024-04-25T10:07:52.296218+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:48.651672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470854+0000","last_change":"2024-04-25T10:07:51.309229+0000","last_active":"2024-04-25T10:07:52.470854+0000","last_peered":"2024-04-25T10:07:52.470854+0000","last_clean":"2024-04-25T10:07:52.470854+0000","last_became_active":"2024-04-25T10:07:51.308925+0000","last_became_peered":"2024-04-25T10:07:51.308925+0000","last_unstale":"2024-04-25T10:07:52.470854+0000","last_undegraded":"2024-04-25T10:07:52.470854+0000","last_fullsized":"2024-04-25T10:07:52.470854+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:49.435031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470725+0000","last_change":"2024-04-25T10:07:50.309931+0000","last_active":"2024-04-25T10:07:52.470725+0000","last_peered":"2024-04-25T10:07:52.470725+0000","last_clean":"2024-04-25T10:07:52.470725+0000","last_became_active":"2024-04-25T10:07:50.309501+0000","last_became_peered":"2024-04-25T10:07:50.309501+0000","last_unstale":"2024-04-25T10:07:52.470725+0000","last_undegraded":"2024-04-25T10:07:52.470725+0000","last_fullsized":"2024-04-25T10:07:52.470725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:35.696486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470948+0000","last_change":"2024-04-25T10:07:51.318666+0000","last_active":"2024-04-25T10:07:52.470948+0000","last_peered":"2024-04-25T10:07:52.470948+0000","last_clean":"2024-04-25T10:07:52.470948+0000","last_became_active":"2024-04-25T10:07:51.318373+0000","last_became_peered":"2024-04-25T10:07:51.318373+0000","last_unstale":"2024-04-25T10:07:52.470948+0000","last_undegraded":"2024-04-25T10:07:52.470948+0000","last_fullsized":"2024-04-25T10:07:52.470948+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:44.011482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470639+0000","last_change":"2024-04-25T10:07:50.309095+0000","last_active":"2024-04-25T10:07:52.470639+0000","last_peered":"2024-04-25T10:07:52.470639+0000","last_clean":"2024-04-25T10:07:52.470639+0000","last_became_active":"2024-04-25T10:07:50.308756+0000","last_became_peered":"2024-04-25T10:07:50.308756+0000","last_unstale":"2024-04-25T10:07:52.470639+0000","last_undegraded":"2024-04-25T10:07:52.470639+0000","last_fullsized":"2024-04-25T10:07:52.470639+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:04.298496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470541+0000","last_change":"2024-04-25T10:07:51.311605+0000","last_active":"2024-04-25T10:07:52.470541+0000","last_peered":"2024-04-25T10:07:52.470541+0000","last_clean":"2024-04-25T10:07:52.470541+0000","last_became_active":"2024-04-25T10:07:51.311307+0000","last_became_peered":"2024-04-25T10:07:51.311307+0000","last_unstale":"2024-04-25T10:07:52.470541+0000","last_undegraded":"2024-04-25T10:07:52.470541+0000","last_fullsized":"2024-04-25T10:07:52.470541+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:00.278450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294600+0000","last_change":"2024-04-25T10:07:50.306790+0000","last_active":"2024-04-25T10:07:52.294600+0000","last_peered":"2024-04-25T10:07:52.294600+0000","last_clean":"2024-04-25T10:07:52.294600+0000","last_became_active":"2024-04-25T10:07:50.306491+0000","last_became_peered":"2024-04-25T10:07:50.306491+0000","last_unstale":"2024-04-25T10:07:52.294600+0000","last_undegraded":"2024-04-25T10:07:52.294600+0000","last_fullsized":"2024-04-25T10:07:52.294600+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:43.478815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295998+0000","last_change":"2024-04-25T10:07:51.306158+0000","last_active":"2024-04-25T10:07:52.295998+0000","last_peered":"2024-04-25T10:07:52.295998+0000","last_clean":"2024-04-25T10:07:52.295998+0000","last_became_active":"2024-04-25T10:07:51.305824+0000","last_became_peered":"2024-04-25T10:07:51.305824+0000","last_unstale":"2024-04-25T10:07:52.295998+0000","last_undegraded":"2024-04-25T10:07:52.295998+0000","last_fullsized":"2024-04-25T10:07:52.295998+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:17.474618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471114+0000","last_change":"2024-04-25T10:07:50.309508+0000","last_active":"2024-04-25T10:07:52.471114+0000","last_peered":"2024-04-25T10:07:52.471114+0000","last_clean":"2024-04-25T10:07:52.471114+0000","last_became_active":"2024-04-25T10:07:50.309244+0000","last_became_peered":"2024-04-25T10:07:50.309244+0000","last_unstale":"2024-04-25T10:07:52.471114+0000","last_undegraded":"2024-04-25T10:07:52.471114+0000","last_fullsized":"2024-04-25T10:07:52.471114+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:13.036175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470269+0000","last_change":"2024-04-25T10:07:51.307313+0000","last_active":"2024-04-25T10:07:52.470269+0000","last_peered":"2024-04-25T10:07:52.470269+0000","last_clean":"2024-04-25T10:07:52.470269+0000","last_became_active":"2024-04-25T10:07:51.307032+0000","last_became_peered":"2024-04-25T10:07:51.307032+0000","last_unstale":"2024-04-25T10:07:52.470269+0000","last_undegraded":"2024-04-25T10:07:52.470269+0000","last_fullsized":"2024-04-25T10:07:52.470269+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:51.296503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470393+0000","last_change":"2024-04-25T10:07:50.304944+0000","last_active":"2024-04-25T10:07:52.470393+0000","last_peered":"2024-04-25T10:07:52.470393+0000","last_clean":"2024-04-25T10:07:52.470393+0000","last_became_active":"2024-04-25T10:07:50.304542+0000","last_became_peered":"2024-04-25T10:07:50.304542+0000","last_unstale":"2024-04-25T10:07:52.470393+0000","last_undegraded":"2024-04-25T10:07:52.470393+0000","last_fullsized":"2024-04-25T10:07:52.470393+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:39.934850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295938+0000","last_change":"2024-04-25T10:07:51.320164+0000","last_active":"2024-04-25T10:07:52.295938+0000","last_peered":"2024-04-25T10:07:52.295938+0000","last_clean":"2024-04-25T10:07:52.295938+0000","last_became_active":"2024-04-25T10:07:51.319859+0000","last_became_peered":"2024-04-25T10:07:51.319859+0000","last_unstale":"2024-04-25T10:07:52.295938+0000","last_undegraded":"2024-04-25T10:07:52.295938+0000","last_fullsized":"2024-04-25T10:07:52.295938+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:21.727353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470461+0000","last_change":"2024-04-25T10:07:50.309933+0000","last_active":"2024-04-25T10:07:52.470461+0000","last_peered":"2024-04-25T10:07:52.470461+0000","last_clean":"2024-04-25T10:07:52.470461+0000","last_became_active":"2024-04-25T10:07:50.309637+0000","last_became_peered":"2024-04-25T10:07:50.309637+0000","last_unstale":"2024-04-25T10:07:52.470461+0000","last_undegraded":"2024-04-25T10:07:52.470461+0000","last_fullsized":"2024-04-25T10:07:52.470461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:09.460489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470368+0000","last_change":"2024-04-25T10:07:51.308930+0000","last_active":"2024-04-25T10:07:52.470368+0000","last_peered":"2024-04-25T10:07:52.470368+0000","last_clean":"2024-04-25T10:07:52.470368+0000","last_became_active":"2024-04-25T10:07:51.308603+0000","last_became_peered":"2024-04-25T10:07:51.308603+0000","last_unstale":"2024-04-25T10:07:52.470368+0000","last_undegraded":"2024-04-25T10:07:52.470368+0000","last_fullsized":"2024-04-25T10:07:52.470368+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:17.724785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470020+0000","last_change":"2024-04-25T10:07:50.304956+0000","last_active":"2024-04-25T10:07:52.470020+0000","last_peered":"2024-04-25T10:07:52.470020+0000","last_clean":"2024-04-25T10:07:52.470020+0000","last_became_active":"2024-04-25T10:07:50.304634+0000","last_became_peered":"2024-04-25T10:07:50.304634+0000","last_unstale":"2024-04-25T10:07:52.470020+0000","last_undegraded":"2024-04-25T10:07:52.470020+0000","last_fullsized":"2024-04-25T10:07:52.470020+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:28.896776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470917+0000","last_change":"2024-04-25T10:07:51.314008+0000","last_active":"2024-04-25T10:07:52.470917+0000","last_peered":"2024-04-25T10:07:52.470917+0000","last_clean":"2024-04-25T10:07:52.470917+0000","last_became_active":"2024-04-25T10:07:51.313611+0000","last_became_peered":"2024-04-25T10:07:51.313611+0000","last_unstale":"2024-04-25T10:07:52.470917+0000","last_undegraded":"2024-04-25T10:07:52.470917+0000","last_fullsized":"2024-04-25T10:07:52.470917+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:08.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471515+0000","last_change":"2024-04-25T10:07:50.313909+0000","last_active":"2024-04-25T10:07:52.471515+0000","last_peered":"2024-04-25T10:07:52.471515+0000","last_clean":"2024-04-25T10:07:52.471515+0000","last_became_active":"2024-04-25T10:07:50.313400+0000","last_became_peered":"2024-04-25T10:07:50.313400+0000","last_unstale":"2024-04-25T10:07:52.471515+0000","last_undegraded":"2024-04-25T10:07:52.471515+0000","last_fullsized":"2024-04-25T10:07:52.471515+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:09.642875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471017+0000","last_change":"2024-04-25T10:07:51.310494+0000","last_active":"2024-04-25T10:07:52.471017+0000","last_peered":"2024-04-25T10:07:52.471017+0000","last_clean":"2024-04-25T10:07:52.471017+0000","last_became_active":"2024-04-25T10:07:51.310224+0000","last_became_peered":"2024-04-25T10:07:51.310224+0000","last_unstale":"2024-04-25T10:07:52.471017+0000","last_undegraded":"2024-04-25T10:07:52.471017+0000","last_fullsized":"2024-04-25T10:07:52.471017+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471102+0000","last_change":"2024-04-25T10:07:50.309494+0000","last_active":"2024-04-25T10:07:52.471102+0000","last_peered":"2024-04-25T10:07:52.471102+0000","last_clean":"2024-04-25T10:07:52.471102+0000","last_became_active":"2024-04-25T10:07:50.309218+0000","last_became_peered":"2024-04-25T10:07:50.309218+0000","last_unstale":"2024-04-25T10:07:52.471102+0000","last_undegraded":"2024-04-25T10:07:52.471102+0000","last_fullsized":"2024-04-25T10:07:52.471102+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:55.139546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471787+0000","last_change":"2024-04-25T10:07:51.318600+0000","last_active":"2024-04-25T10:07:52.471787+0000","last_peered":"2024-04-25T10:07:52.471787+0000","last_clean":"2024-04-25T10:07:52.471787+0000","last_became_active":"2024-04-25T10:07:51.318304+0000","last_became_peered":"2024-04-25T10:07:51.318304+0000","last_unstale":"2024-04-25T10:07:52.471787+0000","last_undegraded":"2024-04-25T10:07:52.471787+0000","last_fullsized":"2024-04-25T10:07:52.471787+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:53.592626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470762+0000","last_change":"2024-04-25T10:07:50.304446+0000","last_active":"2024-04-25T10:07:52.470762+0000","last_peered":"2024-04-25T10:07:52.470762+0000","last_clean":"2024-04-25T10:07:52.470762+0000","last_became_active":"2024-04-25T10:07:50.303898+0000","last_became_peered":"2024-04-25T10:07:50.303898+0000","last_unstale":"2024-04-25T10:07:52.470762+0000","last_undegraded":"2024-04-25T10:07:52.470762+0000","last_fullsized":"2024-04-25T10:07:52.470762+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:23.749111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470626+0000","last_change":"2024-04-25T10:07:51.319204+0000","last_active":"2024-04-25T10:07:52.470626+0000","last_peered":"2024-04-25T10:07:52.470626+0000","last_clean":"2024-04-25T10:07:52.470626+0000","last_became_active":"2024-04-25T10:07:51.318367+0000","last_became_peered":"2024-04-25T10:07:51.318367+0000","last_unstale":"2024-04-25T10:07:52.470626+0000","last_undegraded":"2024-04-25T10:07:52.470626+0000","last_fullsized":"2024-04-25T10:07:52.470626+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:46.811230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470891+0000","last_change":"2024-04-25T10:07:50.313706+0000","last_active":"2024-04-25T10:07:52.470891+0000","last_peered":"2024-04-25T10:07:52.470891+0000","last_clean":"2024-04-25T10:07:52.470891+0000","last_became_active":"2024-04-25T10:07:50.313408+0000","last_became_peered":"2024-04-25T10:07:50.313408+0000","last_unstale":"2024-04-25T10:07:52.470891+0000","last_undegraded":"2024-04-25T10:07:52.470891+0000","last_fullsized":"2024-04-25T10:07:52.470891+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:48.190278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296169+0000","last_change":"2024-04-25T10:07:51.312843+0000","last_active":"2024-04-25T10:07:52.296169+0000","last_peered":"2024-04-25T10:07:52.296169+0000","last_clean":"2024-04-25T10:07:52.296169+0000","last_became_active":"2024-04-25T10:07:51.311808+0000","last_became_peered":"2024-04-25T10:07:51.311808+0000","last_unstale":"2024-04-25T10:07:52.296169+0000","last_undegraded":"2024-04-25T10:07:52.296169+0000","last_fullsized":"2024-04-25T10:07:52.296169+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:18.847819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296623+0000","last_change":"2024-04-25T10:07:50.306664+0000","last_active":"2024-04-25T10:07:52.296623+0000","last_peered":"2024-04-25T10:07:52.296623+0000","last_clean":"2024-04-25T10:07:52.296623+0000","last_became_active":"2024-04-25T10:07:50.306383+0000","last_became_peered":"2024-04-25T10:07:50.306383+0000","last_unstale":"2024-04-25T10:07:52.296623+0000","last_undegraded":"2024-04-25T10:07:52.296623+0000","last_fullsized":"2024-04-25T10:07:52.296623+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:10.698506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471265+0000","last_change":"2024-04-25T10:07:51.321773+0000","last_active":"2024-04-25T10:07:52.471265+0000","last_peered":"2024-04-25T10:07:52.471265+0000","last_clean":"2024-04-25T10:07:52.471265+0000","last_became_active":"2024-04-25T10:07:51.321510+0000","last_became_peered":"2024-04-25T10:07:51.321510+0000","last_unstale":"2024-04-25T10:07:52.471265+0000","last_undegraded":"2024-04-25T10:07:52.471265+0000","last_fullsized":"2024-04-25T10:07:52.471265+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:18.453438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470603+0000","last_change":"2024-04-25T10:07:50.308847+0000","last_active":"2024-04-25T10:07:52.470603+0000","last_peered":"2024-04-25T10:07:52.470603+0000","last_clean":"2024-04-25T10:07:52.470603+0000","last_became_active":"2024-04-25T10:07:50.308577+0000","last_became_peered":"2024-04-25T10:07:50.308577+0000","last_unstale":"2024-04-25T10:07:52.470603+0000","last_undegraded":"2024-04-25T10:07:52.470603+0000","last_fullsized":"2024-04-25T10:07:52.470603+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:46.568515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470898+0000","last_change":"2024-04-25T10:07:51.319559+0000","last_active":"2024-04-25T10:07:52.470898+0000","last_peered":"2024-04-25T10:07:52.470898+0000","last_clean":"2024-04-25T10:07:52.470898+0000","last_became_active":"2024-04-25T10:07:51.319210+0000","last_became_peered":"2024-04-25T10:07:51.319210+0000","last_unstale":"2024-04-25T10:07:52.470898+0000","last_undegraded":"2024-04-25T10:07:52.470898+0000","last_fullsized":"2024-04-25T10:07:52.470898+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:53.462753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296086+0000","last_change":"2024-04-25T10:07:50.311607+0000","last_active":"2024-04-25T10:07:52.296086+0000","last_peered":"2024-04-25T10:07:52.296086+0000","last_clean":"2024-04-25T10:07:52.296086+0000","last_became_active":"2024-04-25T10:07:50.311300+0000","last_became_peered":"2024-04-25T10:07:50.311300+0000","last_unstale":"2024-04-25T10:07:52.296086+0000","last_undegraded":"2024-04-25T10:07:52.296086+0000","last_fullsized":"2024-04-25T10:07:52.296086+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.109707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471043+0000","last_change":"2024-04-25T10:07:51.314396+0000","last_active":"2024-04-25T10:07:52.471043+0000","last_peered":"2024-04-25T10:07:52.471043+0000","last_clean":"2024-04-25T10:07:52.471043+0000","last_became_active":"2024-04-25T10:07:51.314116+0000","last_became_peered":"2024-04-25T10:07:51.314116+0000","last_unstale":"2024-04-25T10:07:52.471043+0000","last_undegraded":"2024-04-25T10:07:52.471043+0000","last_fullsized":"2024-04-25T10:07:52.471043+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:19.021120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471632+0000","last_change":"2024-04-25T10:07:50.307927+0000","last_active":"2024-04-25T10:07:52.471632+0000","last_peered":"2024-04-25T10:07:52.471632+0000","last_clean":"2024-04-25T10:07:52.471632+0000","last_became_active":"2024-04-25T10:07:50.307641+0000","last_became_peered":"2024-04-25T10:07:50.307641+0000","last_unstale":"2024-04-25T10:07:52.471632+0000","last_undegraded":"2024-04-25T10:07:52.471632+0000","last_fullsized":"2024-04-25T10:07:52.471632+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:22.138626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471021+0000","last_change":"2024-04-25T10:07:51.311790+0000","last_active":"2024-04-25T10:07:52.471021+0000","last_peered":"2024-04-25T10:07:52.471021+0000","last_clean":"2024-04-25T10:07:52.471021+0000","last_became_active":"2024-04-25T10:07:51.311524+0000","last_became_peered":"2024-04-25T10:07:51.311524+0000","last_unstale":"2024-04-25T10:07:52.471021+0000","last_undegraded":"2024-04-25T10:07:52.471021+0000","last_fullsized":"2024-04-25T10:07:52.471021+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.168043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"59'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.811112+0000","last_change":"2024-04-25T10:07:50.313145+0000","last_active":"2024-04-25T10:07:52.811112+0000","last_peered":"2024-04-25T10:07:52.811112+0000","last_clean":"2024-04-25T10:07:52.811112+0000","last_became_active":"2024-04-25T10:07:50.312860+0000","last_became_peered":"2024-04-25T10:07:50.312860+0000","last_unstale":"2024-04-25T10:07:52.811112+0000","last_undegraded":"2024-04-25T10:07:52.811112+0000","last_fullsized":"2024-04-25T10:07:52.811112+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:16.044211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471897+0000","last_change":"2024-04-25T10:07:51.322690+0000","last_active":"2024-04-25T10:07:52.471897+0000","last_peered":"2024-04-25T10:07:52.471897+0000","last_clean":"2024-04-25T10:07:52.471897+0000","last_became_active":"2024-04-25T10:07:51.322404+0000","last_became_peered":"2024-04-25T10:07:51.322404+0000","last_unstale":"2024-04-25T10:07:52.471897+0000","last_undegraded":"2024-04-25T10:07:52.471897+0000","last_fullsized":"2024-04-25T10:07:52.471897+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:00.981450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.296163+0000","last_change":"2024-04-25T10:07:50.309078+0000","last_active":"2024-04-25T10:07:52.296163+0000","last_peered":"2024-04-25T10:07:52.296163+0000","last_clean":"2024-04-25T10:07:52.296163+0000","last_became_active":"2024-04-25T10:07:50.308794+0000","last_became_peered":"2024-04-25T10:07:50.308794+0000","last_unstale":"2024-04-25T10:07:52.296163+0000","last_undegraded":"2024-04-25T10:07:52.296163+0000","last_fullsized":"2024-04-25T10:07:52.296163+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:09.642875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.472043+0000","last_change":"2024-04-25T10:07:51.318555+0000","last_active":"2024-04-25T10:07:52.472043+0000","last_peered":"2024-04-25T10:07:52.472043+0000","last_clean":"2024-04-25T10:07:52.472043+0000","last_became_active":"2024-04-25T10:07:51.318243+0000","last_became_peered":"2024-04-25T10:07:51.318243+0000","last_unstale":"2024-04-25T10:07:52.472043+0000","last_undegraded":"2024-04-25T10:07:52.472043+0000","last_fullsized":"2024-04-25T10:07:52.472043+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:46.249801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470834+0000","last_change":"2024-04-25T10:07:50.307963+0000","last_active":"2024-04-25T10:07:52.470834+0000","last_peered":"2024-04-25T10:07:52.470834+0000","last_clean":"2024-04-25T10:07:52.470834+0000","last_became_active":"2024-04-25T10:07:50.307611+0000","last_became_peered":"2024-04-25T10:07:50.307611+0000","last_unstale":"2024-04-25T10:07:52.470834+0000","last_undegraded":"2024-04-25T10:07:52.470834+0000","last_fullsized":"2024-04-25T10:07:52.470834+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:13.364471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470886+0000","last_change":"2024-04-25T10:07:51.316433+0000","last_active":"2024-04-25T10:07:52.470886+0000","last_peered":"2024-04-25T10:07:52.470886+0000","last_clean":"2024-04-25T10:07:52.470886+0000","last_became_active":"2024-04-25T10:07:51.315952+0000","last_became_peered":"2024-04-25T10:07:51.315952+0000","last_unstale":"2024-04-25T10:07:52.470886+0000","last_undegraded":"2024-04-25T10:07:52.470886+0000","last_fullsized":"2024-04-25T10:07:52.470886+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:13.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471022+0000","last_change":"2024-04-25T10:07:50.313990+0000","last_active":"2024-04-25T10:07:52.471022+0000","last_peered":"2024-04-25T10:07:52.471022+0000","last_clean":"2024-04-25T10:07:52.471022+0000","last_became_active":"2024-04-25T10:07:50.313700+0000","last_became_peered":"2024-04-25T10:07:50.313700+0000","last_unstale":"2024-04-25T10:07:52.471022+0000","last_undegraded":"2024-04-25T10:07:52.471022+0000","last_fullsized":"2024-04-25T10:07:52.471022+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:46.602681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294542+0000","last_change":"2024-04-25T10:07:51.315938+0000","last_active":"2024-04-25T10:07:52.294542+0000","last_peered":"2024-04-25T10:07:52.294542+0000","last_clean":"2024-04-25T10:07:52.294542+0000","last_became_active":"2024-04-25T10:07:51.315335+0000","last_became_peered":"2024-04-25T10:07:51.315335+0000","last_unstale":"2024-04-25T10:07:52.294542+0000","last_undegraded":"2024-04-25T10:07:52.294542+0000","last_fullsized":"2024-04-25T10:07:52.294542+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:19.859870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470304+0000","last_change":"2024-04-25T10:07:50.306917+0000","last_active":"2024-04-25T10:07:52.470304+0000","last_peered":"2024-04-25T10:07:52.470304+0000","last_clean":"2024-04-25T10:07:52.470304+0000","last_became_active":"2024-04-25T10:07:50.306549+0000","last_became_peered":"2024-04-25T10:07:50.306549+0000","last_unstale":"2024-04-25T10:07:52.470304+0000","last_undegraded":"2024-04-25T10:07:52.470304+0000","last_fullsized":"2024-04-25T10:07:52.470304+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:06.727495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470599+0000","last_change":"2024-04-25T10:07:51.308187+0000","last_active":"2024-04-25T10:07:52.470599+0000","last_peered":"2024-04-25T10:07:52.470599+0000","last_clean":"2024-04-25T10:07:52.470599+0000","last_became_active":"2024-04-25T10:07:51.307731+0000","last_became_peered":"2024-04-25T10:07:51.307731+0000","last_unstale":"2024-04-25T10:07:52.470599+0000","last_undegraded":"2024-04-25T10:07:52.470599+0000","last_fullsized":"2024-04-25T10:07:52.470599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:04.155153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471599+0000","last_change":"2024-04-25T10:07:50.311119+0000","last_active":"2024-04-25T10:07:52.471599+0000","last_peered":"2024-04-25T10:07:52.471599+0000","last_clean":"2024-04-25T10:07:52.471599+0000","last_became_active":"2024-04-25T10:07:50.310843+0000","last_became_peered":"2024-04-25T10:07:50.310843+0000","last_unstale":"2024-04-25T10:07:52.471599+0000","last_undegraded":"2024-04-25T10:07:52.471599+0000","last_fullsized":"2024-04-25T10:07:52.471599+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:26.803283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471140+0000","last_change":"2024-04-25T10:07:51.316546+0000","last_active":"2024-04-25T10:07:52.471140+0000","last_peered":"2024-04-25T10:07:52.471140+0000","last_clean":"2024-04-25T10:07:52.471140+0000","last_became_active":"2024-04-25T10:07:51.313979+0000","last_became_peered":"2024-04-25T10:07:51.313979+0000","last_unstale":"2024-04-25T10:07:52.471140+0000","last_undegraded":"2024-04-25T10:07:52.471140+0000","last_fullsized":"2024-04-25T10:07:52.471140+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:17.474618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.470666+0000","last_change":"2024-04-25T10:07:50.310973+0000","last_active":"2024-04-25T10:07:52.470666+0000","last_peered":"2024-04-25T10:07:52.470666+0000","last_clean":"2024-04-25T10:07:52.470666+0000","last_became_active":"2024-04-25T10:07:50.310701+0000","last_became_peered":"2024-04-25T10:07:50.310701+0000","last_unstale":"2024-04-25T10:07:52.470666+0000","last_undegraded":"2024-04-25T10:07:52.470666+0000","last_fullsized":"2024-04-25T10:07:52.470666+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:04.171725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.471131+0000","last_change":"2024-04-25T10:07:51.309454+0000","last_active":"2024-04-25T10:07:52.471131+0000","last_peered":"2024-04-25T10:07:52.471131+0000","last_clean":"2024-04-25T10:07:52.471131+0000","last_became_active":"2024-04-25T10:07:51.309105+0000","last_became_peered":"2024-04-25T10:07:51.309105+0000","last_unstale":"2024-04-25T10:07:52.471131+0000","last_undegraded":"2024-04-25T10:07:52.471131+0000","last_fullsized":"2024-04-25T10:07:52.471131+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:50.284453+0000","last_clean_scrub_stamp":"2024-04-25T10:07:50.284453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:20.919759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.294591+0000","last_change":"2024-04-25T10:07:50.309026+0000","last_active":"2024-04-25T10:07:52.294591+0000","last_peered":"2024-04-25T10:07:52.294591+0000","last_clean":"2024-04-25T10:07:52.294591+0000","last_became_active":"2024-04-25T10:07:50.308706+0000","last_became_peered":"2024-04-25T10:07:50.308706+0000","last_unstale":"2024-04-25T10:07:52.294591+0000","last_undegraded":"2024-04-25T10:07:52.294591+0000","last_fullsized":"2024-04-25T10:07:52.294591+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:49.282421+0000","last_clean_scrub_stamp":"2024-04-25T10:07:49.282421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:17.725300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":245,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T10:07:52.295995+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:07:52.295995+0000","last_peered":"2024-04-25T10:07:52.295995+0000","last_clean":"2024-04-25T10:07:52.295995+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:07:52.295995+0000","last_undegraded":"2024-04-25T10:07:52.295995+0000","last_fullsized":"2024-04-25T10:07:52.295995+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":288874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":704512,"data_stored":288804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":289398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":704512,"data_stored":288872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":720896,"data_stored":290554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1298432,"data_stored":879732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":712704,"data_stored":290394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1298432,"data_stored":879184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:59.074 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:07:59.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:07:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.213+0000 7fc17d412640 1 Processor -- start 2024-04-25T10:07:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.213+0000 7fc17d412640 1 -- start start 2024-04-25T10:07:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.214+0000 7fc17d412640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc1780b9290 0x7fc1780d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.214+0000 7fc17d412640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1780b9700 0x7fc1780d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.214+0000 7fc17d412640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17815e070 con 0x7fc17815dc90 2024-04-25T10:07:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.214+0000 7fc17d412640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1780d1b10 con 0x7fc1780b9290 2024-04-25T10:07:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.214+0000 7fc17d412640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780d1df0 con 0x7fc1780b9700 2024-04-25T10:07:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.214+0000 7fc16ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc1780b9290 0x7fc1780d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.214+0000 7fc16ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc1780b9290 0x7fc1780d0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53212/0 (socket says 172.21.15.16:53212) 2024-04-25T10:07:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.214+0000 7fc16ffff640 1 -- 172.21.15.16:0/2627608204 learned_addr learned my addr 172.21.15.16:0/2627608204 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.214+0000 7fc1777fe640 1 --2- 172.21.15.16:0/2627608204 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1780b9700 0x7fc1780d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.214+0000 7fc16ffff640 1 -- 172.21.15.16:0/2627608204 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1780b9700 msgr2=0x7fc1780d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.214+0000 7fc16ffff640 1 --2- 172.21.15.16:0/2627608204 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1780b9700 0x7fc1780d0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.214+0000 7fc16ffff640 1 -- 172.21.15.16:0/2627608204 >> v1:172.21.15.186:6789/0 conn(0x7fc17815dc90 legacy=0x7fc1780d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.214+0000 7fc16ffff640 1 -- 172.21.15.16:0/2627608204 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1780d20d0 con 0x7fc1780b9290 2024-04-25T10:07:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.215+0000 7fc16ffff640 1 --2- 172.21.15.16:0/2627608204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1780b9290 0x7fc1780d0420 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc160009870 tx=0x7fc16002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdad60df8cb37081 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.215+0000 7fc1767fc640 1 -- 172.21.15.16:0/2627608204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160037070 con 0x7fc1780b9290 2024-04-25T10:07:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.215+0000 7fc1767fc640 1 -- 172.21.15.16:0/2627608204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc160032440 con 0x7fc1780b9290 2024-04-25T10:07:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.215+0000 7fc1767fc640 1 -- 172.21.15.16:0/2627608204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160032810 con 0x7fc1780b9290 2024-04-25T10:07:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.216+0000 7fc17d412640 1 -- 172.21.15.16:0/2627608204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1780b9290 msgr2=0x7fc1780d0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.216+0000 7fc17d412640 1 --2- 172.21.15.16:0/2627608204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1780b9290 0x7fc1780d0420 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc160009870 tx=0x7fc16002a270 comp rx=0 tx=0).stop 2024-04-25T10:07:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.216+0000 7fc17d412640 1 -- 172.21.15.16:0/2627608204 shutdown_connections 2024-04-25T10:07:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.216+0000 7fc17d412640 1 --2- 172.21.15.16:0/2627608204 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1780b9700 0x7fc1780d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.216+0000 7fc17d412640 1 --2- 172.21.15.16:0/2627608204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1780b9290 0x7fc1780d0420 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.216+0000 7fc17d412640 1 -- 172.21.15.16:0/2627608204 >> 172.21.15.16:0/2627608204 conn(0x7fc178072900 msgr2=0x7fc17806c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.216+0000 7fc17d412640 1 -- 172.21.15.16:0/2627608204 shutdown_connections 2024-04-25T10:07:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.216+0000 7fc17d412640 1 -- 172.21.15.16:0/2627608204 wait complete. 2024-04-25T10:07:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.217+0000 7fc17d412640 1 Processor -- start 2024-04-25T10:07:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.217+0000 7fc17d412640 1 -- start start 2024-04-25T10:07:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc17d412640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1780b9290 0x7fc1780cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc17d412640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1780b9700 0x7fc1780b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc17d412640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc17815dc90 0x7fc1780b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc17d412640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780d3480 con 0x7fc1780b9700 2024-04-25T10:07:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc17d412640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780d2690 con 0x7fc1780b9290 2024-04-25T10:07:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc17d412640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780d3600 con 0x7fc17815dc90 2024-04-25T10:07:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc16ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1780b9700 0x7fc1780b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc16ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1780b9700 0x7fc1780b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53226/0 (socket says 172.21.15.16:53226) 2024-04-25T10:07:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc16ffff640 1 -- 172.21.15.16:0/684868249 learned_addr learned my addr 172.21.15.16:0/684868249 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc176ffd640 1 --2- 172.21.15.16:0/684868249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1780b9290 0x7fc1780cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc1777fe640 1 --2- 172.21.15.16:0/684868249 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc17815dc90 0x7fc1780b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc16ffff640 1 -- 172.21.15.16:0/684868249 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc17815dc90 msgr2=0x7fc1780b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc16ffff640 1 --2- 172.21.15.16:0/684868249 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc17815dc90 0x7fc1780b4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc16ffff640 1 -- 172.21.15.16:0/684868249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1780b9290 msgr2=0x7fc1780cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc16ffff640 1 --2- 172.21.15.16:0/684868249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1780b9290 0x7fc1780cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc16ffff640 1 -- 172.21.15.16:0/684868249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1780b4b80 con 0x7fc1780b9700 2024-04-25T10:07:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc176ffd640 1 --2- 172.21.15.16:0/684868249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1780b9290 0x7fc1780cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc16ffff640 1 --2- 172.21.15.16:0/684868249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1780b9700 0x7fc1780b4100 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc160000c00 tx=0x7fc1600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc16f7fe640 1 -- 172.21.15.16:0/684868249 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160037040 con 0x7fc1780b9700 2024-04-25T10:07:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc16f7fe640 1 -- 172.21.15.16:0/684868249 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc160042070 con 0x7fc1780b9700 2024-04-25T10:07:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc16f7fe640 1 -- 172.21.15.16:0/684868249 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160030880 con 0x7fc1780b9700 2024-04-25T10:07:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.218+0000 7fc17d412640 1 -- 172.21.15.16:0/684868249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1780b4e30 con 0x7fc1780b9700 2024-04-25T10:07:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.219+0000 7fc17d412640 1 -- 172.21.15.16:0/684868249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17814de30 con 0x7fc1780b9700 2024-04-25T10:07:59.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.219+0000 7fc17d412640 1 -- 172.21.15.16:0/684868249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1780749b0 con 0x7fc1780b9700 2024-04-25T10:07:59.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.220+0000 7fc16f7fe640 1 -- 172.21.15.16:0/684868249 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc160030a20 con 0x7fc1780b9700 2024-04-25T10:07:59.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.221+0000 7fc16f7fe640 1 --2- 172.21.15.16:0/684868249 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc15806b500 0x7fc15806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.221+0000 7fc16f7fe640 1 -- 172.21.15.16:0/684868249 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1600aa960 con 0x7fc1780b9700 2024-04-25T10:07:59.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.221+0000 7fc176ffd640 1 --2- 172.21.15.16:0/684868249 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc15806b500 0x7fc15806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.221+0000 7fc176ffd640 1 --2- 172.21.15.16:0/684868249 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc15806b500 0x7fc15806d9c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc1680015e0 tx=0x7fc168009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.225+0000 7fc16f7fe640 1 -- 172.21.15.16:0/684868249 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc160079870 con 0x7fc1780b9700 2024-04-25T10:07:59.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.483+0000 7fc17d412640 1 -- 172.21.15.16:0/684868249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc1780b80d0 con 0x7fc1780b9700 2024-04-25T10:07:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.485+0000 7fc16f7fe640 1 -- 172.21.15.16:0/684868249 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc160079170 con 0x7fc1780b9700 2024-04-25T10:07:59.491 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-04-25T10:07:59.491 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCZKipmAph/GhAAG3x1zC98yHQ13fjpmrlPkg== 2024-04-25T10:07:59.491 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-04-25T10:07:59.491 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-04-25T10:07:59.491 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:07:59.492 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-04-25T10:07:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.487+0000 7fc17d412640 1 -- 172.21.15.16:0/684868249 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc15806b500 msgr2=0x7fc15806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.487+0000 7fc17d412640 1 --2- 172.21.15.16:0/684868249 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc15806b500 0x7fc15806d9c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc1680015e0 tx=0x7fc168009040 comp rx=0 tx=0).stop 2024-04-25T10:07:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.487+0000 7fc17d412640 1 -- 172.21.15.16:0/684868249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1780b9700 msgr2=0x7fc1780b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.487+0000 7fc17d412640 1 --2- 172.21.15.16:0/684868249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1780b9700 0x7fc1780b4100 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc160000c00 tx=0x7fc1600046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.487+0000 7fc17d412640 1 -- 172.21.15.16:0/684868249 shutdown_connections 2024-04-25T10:07:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.487+0000 7fc17d412640 1 --2- 172.21.15.16:0/684868249 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc15806b500 0x7fc15806d9c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.487+0000 7fc17d412640 1 --2- 172.21.15.16:0/684868249 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc17815dc90 0x7fc1780b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.487+0000 7fc17d412640 1 --2- 172.21.15.16:0/684868249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1780b9700 0x7fc1780b4100 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.487+0000 7fc17d412640 1 --2- 172.21.15.16:0/684868249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1780b9290 0x7fc1780cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.487+0000 7fc17d412640 1 -- 172.21.15.16:0/684868249 >> 172.21.15.16:0/684868249 conn(0x7fc178072900 msgr2=0x7fc1780b8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.487+0000 7fc17d412640 1 -- 172.21.15.16:0/684868249 shutdown_connections 2024-04-25T10:07:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.487+0000 7fc17d412640 1 -- 172.21.15.16:0/684868249 wait complete. 2024-04-25T10:07:59.504 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:59.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.600+0000 7f311fe88640 1 Processor -- start 2024-04-25T10:07:59.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.600+0000 7f311fe88640 1 -- start start 2024-04-25T10:07:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.601+0000 7f311fe88640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f311815ba10 0x7f3118099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.601+0000 7f311fe88640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f311806c3c0 0x7f31180cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.601+0000 7f311fe88640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311806c900 con 0x7f3118099f80 2024-04-25T10:07:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.601+0000 7f311fe88640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31180cbe50 con 0x7f311815ba10 2024-04-25T10:07:59.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.601+0000 7f311fe88640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311809a3d0 con 0x7f311806c3c0 2024-04-25T10:07:59.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.601+0000 7f311d423640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3118099f80 0x7f311806bcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54450/0 (socket says 172.21.15.16:54450) 2024-04-25T10:07:59.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.601+0000 7f311d423640 1 -- 172.21.15.16:0/3155105923 learned_addr learned my addr 172.21.15.16:0/3155105923 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:07:59.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.601+0000 7f311dc24640 1 --2- 172.21.15.16:0/3155105923 >> v2:172.21.15.186:3300/0 conn(0x7f311815ba10 0x7f3118099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.601+0000 7f311e425640 1 --2- 172.21.15.16:0/3155105923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f311806c3c0 0x7f31180cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.601+0000 7f311cc22640 1 -- 172.21.15.16:0/3155105923 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 797280905 0 0) 0x7f311806c900 con 0x7f3118099f80 2024-04-25T10:07:59.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.601+0000 7f311cc22640 1 -- 172.21.15.16:0/3155105923 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3104003520 con 0x7f3118099f80 2024-04-25T10:07:59.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.601+0000 7f311e425640 1 -- 172.21.15.16:0/3155105923 >> v2:172.21.15.186:3300/0 conn(0x7f311815ba10 msgr2=0x7f3118099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.601+0000 7f311e425640 1 --2- 172.21.15.16:0/3155105923 >> v2:172.21.15.186:3300/0 conn(0x7f311815ba10 0x7f3118099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.601+0000 7f311e425640 1 -- 172.21.15.16:0/3155105923 >> v1:172.21.15.16:6789/0 conn(0x7f3118099f80 legacy=0x7f311806bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.601+0000 7f311e425640 1 -- 172.21.15.16:0/3155105923 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311806cbe0 con 0x7f311806c3c0 2024-04-25T10:07:59.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.601+0000 7f311dc24640 1 --2- 172.21.15.16:0/3155105923 >> v2:172.21.15.186:3300/0 conn(0x7f311815ba10 0x7f3118099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:59.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.602+0000 7f311e425640 1 --2- 172.21.15.16:0/3155105923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f311806c3c0 0x7f31180cd220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3110009d90 tx=0x7f311002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b2ec5474a7a014c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.602+0000 7f311cc22640 1 -- 172.21.15.16:0/3155105923 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3110007740 con 0x7f311806c3c0 2024-04-25T10:07:59.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.603+0000 7f311cc22640 1 -- 172.21.15.16:0/3155105923 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31100078e0 con 0x7f311806c3c0 2024-04-25T10:07:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.603+0000 7f311cc22640 1 -- 172.21.15.16:0/3155105923 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3110007c90 con 0x7f311806c3c0 2024-04-25T10:07:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.603+0000 7f311fe88640 1 -- 172.21.15.16:0/3155105923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f311806c3c0 msgr2=0x7f31180cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.603+0000 7f311fe88640 1 --2- 172.21.15.16:0/3155105923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f311806c3c0 0x7f31180cd220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3110009d90 tx=0x7f311002a660 comp rx=0 tx=0).stop 2024-04-25T10:07:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.603+0000 7f311fe88640 1 -- 172.21.15.16:0/3155105923 shutdown_connections 2024-04-25T10:07:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.603+0000 7f311fe88640 1 --2- 172.21.15.16:0/3155105923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f311806c3c0 0x7f31180cd220 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.603+0000 7f311fe88640 1 --2- 172.21.15.16:0/3155105923 >> v2:172.21.15.186:3300/0 conn(0x7f311815ba10 0x7f3118099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.603+0000 7f311fe88640 1 -- 172.21.15.16:0/3155105923 >> 172.21.15.16:0/3155105923 conn(0x7f31180b32f0 msgr2=0x7f31180b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.603+0000 7f311fe88640 1 -- 172.21.15.16:0/3155105923 shutdown_connections 2024-04-25T10:07:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.603+0000 7f311fe88640 1 -- 172.21.15.16:0/3155105923 wait complete. 2024-04-25T10:07:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.604+0000 7f311fe88640 1 Processor -- start 2024-04-25T10:07:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.604+0000 7f311fe88640 1 -- start start 2024-04-25T10:07:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.605+0000 7f311fe88640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f311806c3c0 0x7f311815a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.605+0000 7f311fe88640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3118099f80 0x7f31180af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.605+0000 7f311fe88640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f311815ba10 0x7f31181602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.605+0000 7f311fe88640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31180cd760 con 0x7f3118099f80 2024-04-25T10:07:59.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.605+0000 7f311fe88640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31180cfe80 con 0x7f311806c3c0 2024-04-25T10:07:59.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.605+0000 7f311fe88640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31180cd8e0 con 0x7f311815ba10 2024-04-25T10:07:59.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.605+0000 7f311d423640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3118099f80 0x7f31180af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.605+0000 7f311d423640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3118099f80 0x7f31180af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53236/0 (socket says 172.21.15.16:53236) 2024-04-25T10:07:59.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.605+0000 7f311d423640 1 -- 172.21.15.16:0/2140315468 learned_addr learned my addr 172.21.15.16:0/2140315468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:07:59.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.605+0000 7f311dc24640 1 --2- 172.21.15.16:0/2140315468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f311806c3c0 0x7f311815a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.605+0000 7f311e425640 1 --2- 172.21.15.16:0/2140315468 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f311815ba10 0x7f31181602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.606+0000 7f311d423640 1 -- 172.21.15.16:0/2140315468 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f311815ba10 msgr2=0x7f31181602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.606+0000 7f311d423640 1 --2- 172.21.15.16:0/2140315468 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f311815ba10 0x7f31181602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.606+0000 7f311d423640 1 -- 172.21.15.16:0/2140315468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f311806c3c0 msgr2=0x7f311815a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.606+0000 7f311d423640 1 --2- 172.21.15.16:0/2140315468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f311806c3c0 0x7f311815a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.606+0000 7f311d423640 1 -- 172.21.15.16:0/2140315468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31180b4800 con 0x7f3118099f80 2024-04-25T10:07:59.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.606+0000 7f311e425640 1 --2- 172.21.15.16:0/2140315468 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f311815ba10 0x7f31181602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:59.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.606+0000 7f311d423640 1 --2- 172.21.15.16:0/2140315468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3118099f80 0x7f31180af7d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f310800c430 tx=0x7f310800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.606+0000 7f310e7fc640 1 -- 172.21.15.16:0/2140315468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108017070 con 0x7f3118099f80 2024-04-25T10:07:59.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.606+0000 7f310e7fc640 1 -- 172.21.15.16:0/2140315468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3108005ce0 con 0x7f3118099f80 2024-04-25T10:07:59.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.606+0000 7f311fe88640 1 -- 172.21.15.16:0/2140315468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31180b4ae0 con 0x7f3118099f80 2024-04-25T10:07:59.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.606+0000 7f310e7fc640 1 -- 172.21.15.16:0/2140315468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31080072f0 con 0x7f3118099f80 2024-04-25T10:07:59.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.606+0000 7f311fe88640 1 -- 172.21.15.16:0/2140315468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31180b50a0 con 0x7f3118099f80 2024-04-25T10:07:59.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.607+0000 7f311fe88640 1 -- 172.21.15.16:0/2140315468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31180749b0 con 0x7f3118099f80 2024-04-25T10:07:59.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.607+0000 7f310e7fc640 1 -- 172.21.15.16:0/2140315468 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f310801a070 con 0x7f3118099f80 2024-04-25T10:07:59.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.608+0000 7f310e7fc640 1 --2- 172.21.15.16:0/2140315468 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f30fc06b500 0x7f30fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.608+0000 7f310e7fc640 1 -- 172.21.15.16:0/2140315468 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31080895e0 con 0x7f3118099f80 2024-04-25T10:07:59.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.608+0000 7f311dc24640 1 --2- 172.21.15.16:0/2140315468 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f30fc06b500 0x7f30fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.609+0000 7f311dc24640 1 --2- 172.21.15.16:0/2140315468 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f30fc06b500 0x7f30fc06d9c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3114009560 tx=0x7f3114009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.612+0000 7f310e7fc640 1 -- 172.21.15.16:0/2140315468 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3108057e20 con 0x7f3118099f80 2024-04-25T10:07:59.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.830+0000 7f311fe88640 1 -- 172.21.15.16:0/2140315468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31180b4120 con 0x7f3118099f80 2024-04-25T10:07:59.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.831+0000 7f310e7fc640 1 -- 172.21.15.16:0/2140315468 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f310805bec0 con 0x7f3118099f80 2024-04-25T10:07:59.837 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:07:59.837 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2270839865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2270839865},{"type":"v1","addr":"172.21.15.186:6836","nonce":2270839865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5647,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/925282579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":925282579},{"type":"v1","addr":"172.21.15.186:6837","nonce":925282579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2471714328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2471714328},{"type":"v1","addr":"172.21.15.16:6835","nonce":2471714328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:50.665436+0000","modified":"2024-04-25T10:07:52.663229+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35141":{"gid":35141,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6834/3311584521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3311584521},{"type":"v1","addr":"172.21.15.16:6834","nonce":3311584521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:07:59.837 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-04-25T10:07:59.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.833+0000 7f311fe88640 1 -- 172.21.15.16:0/2140315468 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f30fc06b500 msgr2=0x7f30fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.833+0000 7f311fe88640 1 --2- 172.21.15.16:0/2140315468 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f30fc06b500 0x7f30fc06d9c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3114009560 tx=0x7f3114009190 comp rx=0 tx=0).stop 2024-04-25T10:07:59.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.833+0000 7f311fe88640 1 -- 172.21.15.16:0/2140315468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3118099f80 msgr2=0x7f31180af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.833+0000 7f311fe88640 1 --2- 172.21.15.16:0/2140315468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3118099f80 0x7f31180af7d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f310800c430 tx=0x7f310800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.833+0000 7f311fe88640 1 -- 172.21.15.16:0/2140315468 shutdown_connections 2024-04-25T10:07:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.833+0000 7f311fe88640 1 --2- 172.21.15.16:0/2140315468 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f30fc06b500 0x7f30fc06d9c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.833+0000 7f311fe88640 1 --2- 172.21.15.16:0/2140315468 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f311815ba10 0x7f31181602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.833+0000 7f311fe88640 1 --2- 172.21.15.16:0/2140315468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3118099f80 0x7f31180af7d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.833+0000 7f311fe88640 1 --2- 172.21.15.16:0/2140315468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f311806c3c0 0x7f311815a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.833+0000 7f311fe88640 1 -- 172.21.15.16:0/2140315468 >> 172.21.15.16:0/2140315468 conn(0x7f31180b32f0 msgr2=0x7f31180ccac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.833+0000 7f311fe88640 1 -- 172.21.15.16:0/2140315468 shutdown_connections 2024-04-25T10:07:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:07:59.833+0000 7f311fe88640 1 -- 172.21.15.16:0/2140315468 wait complete. 2024-04-25T10:08:00.252 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:07:50.665436+0000', 'modified': '2024-04-25T10:07:52.663229+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 35141}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_35141': {'gid': 35141, 'name': 'a', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6834/3311584521', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6832', 'nonce': 3311584521}, {'type': 'v1', 'addr': '172.21.15.16:6834', 'nonce': 3311584521}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:08:00.253 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:08:00.253 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:08:00.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.355+0000 7fd8cff10640 1 Processor -- start 2024-04-25T10:08:00.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.356+0000 7fd8cff10640 1 -- start start 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.356+0000 7fd8cff10640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8c8161090 0x7fd8c8161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.356+0000 7fd8cff10640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd8c80b9720 0x7fd8c80ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.356+0000 7fd8cff10640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c80cf0f0 con 0x7fd8c80cece0 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.357+0000 7fd8cff10640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c80d21a0 con 0x7fd8c8161090 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.357+0000 7fd8cff10640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c8161f80 con 0x7fd8c80b9720 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.357+0000 7fd8cdcac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8c8161090 0x7fd8c8161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.357+0000 7fd8ce4ad640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd8c80cece0 0x7fd8c806c0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54458/0 (socket says 172.21.15.16:54458) 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.357+0000 7fd8ce4ad640 1 -- 172.21.15.16:0/2053574134 learned_addr learned my addr 172.21.15.16:0/2053574134 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.357+0000 7fd8cd4ab640 1 --2- 172.21.15.16:0/2053574134 >> v2:172.21.15.186:3300/0 conn(0x7fd8c80b9720 0x7fd8c80ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.357+0000 7fd8cdcac640 1 -- 172.21.15.16:0/2053574134 >> v2:172.21.15.186:3300/0 conn(0x7fd8c80b9720 msgr2=0x7fd8c80ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:00.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.357+0000 7fd8cdcac640 1 --2- 172.21.15.16:0/2053574134 >> v2:172.21.15.186:3300/0 conn(0x7fd8c80b9720 0x7fd8c80ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.357+0000 7fd8cdcac640 1 -- 172.21.15.16:0/2053574134 >> v1:172.21.15.16:6789/0 conn(0x7fd8c80cece0 legacy=0x7fd8c806c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.357+0000 7fd8cdcac640 1 -- 172.21.15.16:0/2053574134 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c8162260 con 0x7fd8c8161090 2024-04-25T10:08:00.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.357+0000 7fd8cccaa640 1 -- 172.21.15.16:0/2053574134 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129412023 0 0) 0x7fd8b8001490 con 0x7fd8c80cece0 2024-04-25T10:08:00.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.357+0000 7fd8cd4ab640 1 --2- 172.21.15.16:0/2053574134 >> v2:172.21.15.186:3300/0 conn(0x7fd8c80b9720 0x7fd8c80ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:00.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.358+0000 7fd8cdcac640 1 --2- 172.21.15.16:0/2053574134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c8161090 0x7fd8c8161730 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd8c808cf10 tx=0x7fd8c002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe40bcf39f6b5178 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.358+0000 7fd8cccaa640 1 -- 172.21.15.16:0/2053574134 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0037070 con 0x7fd8c8161090 2024-04-25T10:08:00.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.358+0000 7fd8cccaa640 1 -- 172.21.15.16:0/2053574134 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c0032440 con 0x7fd8c8161090 2024-04-25T10:08:00.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.358+0000 7fd8cccaa640 1 -- 172.21.15.16:0/2053574134 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c002ae60 con 0x7fd8c8161090 2024-04-25T10:08:00.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.358+0000 7fd8cff10640 1 -- 172.21.15.16:0/2053574134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c8161090 msgr2=0x7fd8c8161730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.358+0000 7fd8cff10640 1 --2- 172.21.15.16:0/2053574134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c8161090 0x7fd8c8161730 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd8c808cf10 tx=0x7fd8c002a110 comp rx=0 tx=0).stop 2024-04-25T10:08:00.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.358+0000 7fd8cff10640 1 -- 172.21.15.16:0/2053574134 shutdown_connections 2024-04-25T10:08:00.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.358+0000 7fd8cff10640 1 --2- 172.21.15.16:0/2053574134 >> v2:172.21.15.186:3300/0 conn(0x7fd8c80b9720 0x7fd8c80ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.358+0000 7fd8cff10640 1 --2- 172.21.15.16:0/2053574134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c8161090 0x7fd8c8161730 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.358+0000 7fd8cff10640 1 -- 172.21.15.16:0/2053574134 >> 172.21.15.16:0/2053574134 conn(0x7fd8c80b55e0 msgr2=0x7fd8c80b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.359+0000 7fd8cff10640 1 -- 172.21.15.16:0/2053574134 shutdown_connections 2024-04-25T10:08:00.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.359+0000 7fd8cff10640 1 -- 172.21.15.16:0/2053574134 wait complete. 2024-04-25T10:08:00.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.360+0000 7fd8cff10640 1 Processor -- start 2024-04-25T10:08:00.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.360+0000 7fd8cff10640 1 -- start start 2024-04-25T10:08:00.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.360+0000 7fd8cff10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c80b9720 0x7fd8c814e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.360+0000 7fd8cff10640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8c80cece0 0x7fd8c80cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.360+0000 7fd8cff10640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8c8161090 0x7fd8c81e91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.360+0000 7fd8cff10640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c8161470 con 0x7fd8c80b9720 2024-04-25T10:08:00.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.360+0000 7fd8cff10640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c80d1e30 con 0x7fd8c80cece0 2024-04-25T10:08:00.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.360+0000 7fd8cff10640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c8161c70 con 0x7fd8c8161090 2024-04-25T10:08:00.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.361+0000 7fd8cdcac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c80b9720 0x7fd8c814e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.361+0000 7fd8cdcac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c80b9720 0x7fd8c814e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53242/0 (socket says 172.21.15.16:53242) 2024-04-25T10:08:00.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.361+0000 7fd8cdcac640 1 -- 172.21.15.16:0/3947883494 learned_addr learned my addr 172.21.15.16:0/3947883494 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:00.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.361+0000 7fd8cdcac640 1 -- 172.21.15.16:0/3947883494 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8c8161090 msgr2=0x7fd8c81e91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.361+0000 7fd8cdcac640 1 --2- 172.21.15.16:0/3947883494 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8c8161090 0x7fd8c81e91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.361+0000 7fd8cdcac640 1 -- 172.21.15.16:0/3947883494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8c80cece0 msgr2=0x7fd8c80cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.361+0000 7fd8cdcac640 1 --2- 172.21.15.16:0/3947883494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8c80cece0 0x7fd8c80cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.361+0000 7fd8cdcac640 1 -- 172.21.15.16:0/3947883494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c815b540 con 0x7fd8c80b9720 2024-04-25T10:08:00.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.361+0000 7fd8cdcac640 1 --2- 172.21.15.16:0/3947883494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c80b9720 0x7fd8c814e110 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0000c00 tx=0x7fd8c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.361+0000 7fd8c67fc640 1 -- 172.21.15.16:0/3947883494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0037040 con 0x7fd8c80b9720 2024-04-25T10:08:00.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.361+0000 7fd8c67fc640 1 -- 172.21.15.16:0/3947883494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c0042070 con 0x7fd8c80b9720 2024-04-25T10:08:00.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.361+0000 7fd8cff10640 1 -- 172.21.15.16:0/3947883494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c814e650 con 0x7fd8c80b9720 2024-04-25T10:08:00.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.361+0000 7fd8c67fc640 1 -- 172.21.15.16:0/3947883494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0030a50 con 0x7fd8c80b9720 2024-04-25T10:08:00.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.361+0000 7fd8cff10640 1 -- 172.21.15.16:0/3947883494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c80ccb20 con 0x7fd8c80b9720 2024-04-25T10:08:00.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.363+0000 7fd8c67fc640 1 -- 172.21.15.16:0/3947883494 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8c0030bf0 con 0x7fd8c80b9720 2024-04-25T10:08:00.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.363+0000 7fd8c67fc640 1 --2- 172.21.15.16:0/3947883494 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd8b406b4d0 0x7fd8b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.363+0000 7fd8c67fc640 1 -- 172.21.15.16:0/3947883494 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8c00aac80 con 0x7fd8c80b9720 2024-04-25T10:08:00.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.363+0000 7fd8cd4ab640 1 --2- 172.21.15.16:0/3947883494 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd8b406b4d0 0x7fd8b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.364+0000 7fd8cd4ab640 1 --2- 172.21.15.16:0/3947883494 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd8b406b4d0 0x7fd8b406d990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc006fd0 tx=0x7fd8bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.364+0000 7fd8d2c10740 1 Processor -- start 2024-04-25T10:08:00.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.364+0000 7fd8d2c10740 1 -- start start 2024-04-25T10:08:00.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.365+0000 7fd8d2c10740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d116230 0x55c89d116610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.365+0000 7fd8d2c10740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d133ad0 0x55c89d134020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.365+0000 7fd8d2c10740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c89d0ebf10 0x55c89d132190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.365+0000 7fd8d2c10740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89d04ea10 con 0x55c89d116230 2024-04-25T10:08:00.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.365+0000 7fd8d2c10740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89cff7b30 con 0x55c89d133ad0 2024-04-25T10:08:00.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.365+0000 7fd8d2c10740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c89d04ac50 con 0x55c89d0ebf10 2024-04-25T10:08:00.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.365+0000 7fd8cdcac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d116230 0x55c89d116610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.365+0000 7fd8cdcac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d116230 0x55c89d116610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53244/0 (socket says 172.21.15.16:53244) 2024-04-25T10:08:00.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.365+0000 7fd8cdcac640 1 -- 172.21.15.16:0/874106148 learned_addr learned my addr 172.21.15.16:0/874106148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:00.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.365+0000 7fd8cd4ab640 1 --2- 172.21.15.16:0/874106148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d133ad0 0x55c89d134020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.365+0000 7fd8ce4ad640 1 --2- 172.21.15.16:0/874106148 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c89d0ebf10 0x55c89d132190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.366+0000 7fd8cdcac640 1 -- 172.21.15.16:0/874106148 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c89d0ebf10 msgr2=0x55c89d132190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:00.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.366+0000 7fd8cdcac640 1 --2- 172.21.15.16:0/874106148 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c89d0ebf10 0x55c89d132190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.366+0000 7fd8cdcac640 1 -- 172.21.15.16:0/874106148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d133ad0 msgr2=0x55c89d134020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:00.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.366+0000 7fd8cdcac640 1 --2- 172.21.15.16:0/874106148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d133ad0 0x55c89d134020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.366+0000 7fd8cdcac640 1 -- 172.21.15.16:0/874106148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c89d00ce10 con 0x55c89d116230 2024-04-25T10:08:00.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.366+0000 7fd8cdcac640 1 --2- 172.21.15.16:0/874106148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d116230 0x55c89d116610 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0081670 tx=0x7fd8c0075860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4624cc241b1abcf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.366+0000 7fd8af7fe640 1 -- 172.21.15.16:0/874106148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0079b90 con 0x55c89d116230 2024-04-25T10:08:00.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.366+0000 7fd8af7fe640 1 -- 172.21.15.16:0/874106148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c0079490 con 0x55c89d116230 2024-04-25T10:08:00.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.366+0000 7fd8af7fe640 1 -- 172.21.15.16:0/874106148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c007d5e0 con 0x55c89d116230 2024-04-25T10:08:00.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.366+0000 7fd8d2c10740 1 -- 172.21.15.16:0/874106148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d116230 msgr2=0x55c89d116610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.366+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/874106148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d116230 0x55c89d116610 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0081670 tx=0x7fd8c0075860 comp rx=0 tx=0).stop 2024-04-25T10:08:00.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.366+0000 7fd8d2c10740 1 -- 172.21.15.16:0/874106148 shutdown_connections 2024-04-25T10:08:00.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.366+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/874106148 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c89d0ebf10 0x55c89d132190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.366+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/874106148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d133ad0 0x55c89d134020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.367+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/874106148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d116230 0x55c89d116610 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.367+0000 7fd8d2c10740 1 -- 172.21.15.16:0/874106148 >> 172.21.15.16:0/874106148 conn(0x55c89d0e8840 msgr2=0x55c89d131040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.367+0000 7fd8d2c10740 1 -- 172.21.15.16:0/874106148 shutdown_connections 2024-04-25T10:08:00.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.367+0000 7fd8d2c10740 1 -- 172.21.15.16:0/874106148 wait complete. 2024-04-25T10:08:00.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.367+0000 7fd8d2c10740 1 -- start start 2024-04-25T10:08:00.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.367+0000 7fd8d2c10740 1 Processor -- start 2024-04-25T10:08:00.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.368+0000 7fd8d2c10740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c89d116230 0x55c89d118450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.368+0000 7fd8d2c10740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d133ad0 0x55c89d1193c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.368+0000 7fd8d2c10740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d116ef0 0x55c89d1415d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.368+0000 7fd8d2c10740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89d04ea10 con 0x55c89d133ad0 2024-04-25T10:08:00.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.368+0000 7fd8d2c10740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89d00b0c0 con 0x55c89d116ef0 2024-04-25T10:08:00.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.368+0000 7fd8d2c10740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c89d0eccb0 con 0x55c89d116230 2024-04-25T10:08:00.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.368+0000 7fd8d2c10740 10 client.0 _ll_register_callbacks cb 0x55c89cffeec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:00.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.368+0000 7fd8cd4ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d133ad0 0x55c89d1193c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.368+0000 7fd8cd4ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d133ad0 0x55c89d1193c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53254/0 (socket says 172.21.15.16:53254) 2024-04-25T10:08:00.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.368+0000 7fd8cd4ab640 1 -- 172.21.15.16:0/3896637250 learned_addr learned my addr 172.21.15.16:0/3896637250 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:00.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.368+0000 7fd8cdcac640 1 --2- 172.21.15.16:0/3896637250 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c89d116230 0x55c89d118450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.368+0000 7fd8ce4ad640 1 --2- 172.21.15.16:0/3896637250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d116ef0 0x55c89d1415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.368+0000 7fd8cd4ab640 1 -- 172.21.15.16:0/3896637250 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c89d116230 msgr2=0x55c89d118450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:00.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.368+0000 7fd8cd4ab640 1 --2- 172.21.15.16:0/3896637250 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c89d116230 0x55c89d118450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.368+0000 7fd8cd4ab640 1 -- 172.21.15.16:0/3896637250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d116ef0 msgr2=0x55c89d1415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:00.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.368+0000 7fd8cd4ab640 1 --2- 172.21.15.16:0/3896637250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d116ef0 0x55c89d1415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.368+0000 7fd8cd4ab640 1 -- 172.21.15.16:0/3896637250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c89cd632d0 con 0x55c89d133ad0 2024-04-25T10:08:00.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.369+0000 7fd8cd4ab640 1 --2- 172.21.15.16:0/3896637250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d133ad0 0x55c89d1193c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00ce40 tx=0x7fd8bc006c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f9614cfee6fac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.369+0000 7fd8acff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:08:00.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.369+0000 7fd8acff9640 1 -- 172.21.15.16:0/3896637250 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc023020 con 0x55c89d133ad0 2024-04-25T10:08:00.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.369+0000 7fd8d2c10740 1 -- 172.21.15.16:0/3896637250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8c0079490 con 0x55c89d133ad0 2024-04-25T10:08:00.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.369+0000 7fd8acff9640 1 -- 172.21.15.16:0/3896637250 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8bc01f670 con 0x55c89d133ad0 2024-04-25T10:08:00.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.369+0000 7fd8acff9640 1 -- 172.21.15.16:0/3896637250 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8bc02d050 con 0x55c89d133ad0 2024-04-25T10:08:00.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.369+0000 7fd8d2c10740 10 client.6231 fetch_fsmap learned FSMap version 27 2024-04-25T10:08:00.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.369+0000 7fd8d2c10740 1 -- 172.21.15.16:0/3896637250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55c89ce2d6b0 con 0x55c89d133ad0 2024-04-25T10:08:00.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.369+0000 7fd8acff9640 1 -- 172.21.15.16:0/3896637250 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c0079490 con 0x55c89d133ad0 2024-04-25T10:08:00.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.369+0000 7fd8acff9640 1 -- 172.21.15.16:0/3896637250 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd8bc030020 con 0x55c89d133ad0 2024-04-25T10:08:00.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.369+0000 7fd8d2c10740 10 client.6231 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:08:00.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.369+0000 7fd8d2c10740 10 client.6231 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:08:00.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.369+0000 7fd8d2c10740 20 client.6231 populate_metadata read hostname 'smithi016' 2024-04-25T10:08:00.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.369+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/3896637250 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c89d144ce0 0x55c89d1450c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.369+0000 7fd8d2c10740 4 client.6231 mds_command: new command op to 35141 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:00.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.369+0000 7fd8d2c10740 1 -- 172.21.15.16:0/3896637250 --> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c89d0ad920 con 0x55c89d144ce0 2024-04-25T10:08:00.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.369+0000 7fd8cdcac640 1 --2- 172.21.15.16:0/3896637250 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c89d144ce0 0x55c89d1450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.370+0000 7fd8cdcac640 1 --2- 172.21.15.16:0/3896637250 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c89d144ce0 0x55c89d1450c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=602a16b7d5cc226d server_cookie=411004bb810e9a09 in_seq=0 out_seq=0 2024-04-25T10:08:00.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.370+0000 7fd8acff9640 10 client.6231 ms_handle_connect on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:00.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.370+0000 7fd8acff9640 1 -- 172.21.15.16:0/3896637250 <== mds.0 v2:172.21.15.16:6832/3311584521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c89d0ad920 con 0x55c89d144ce0 2024-04-25T10:08:00.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.370+0000 7fd8acff9640 10 client.6231 handle_command_reply: tid=0 2024-04-25T10:08:00.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.370+0000 7fd8d2c10740 1 client.6231 shutdown 2024-04-25T10:08:00.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.370+0000 7fd8d2c10740 1 -- 172.21.15.16:0/3896637250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d133ad0 msgr2=0x55c89d1193c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.370+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/3896637250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d133ad0 0x55c89d1193c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00ce40 tx=0x7fd8bc006c90 comp rx=0 tx=0).stop 2024-04-25T10:08:00.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.370+0000 7fd8d2c10740 1 -- 172.21.15.16:0/3896637250 shutdown_connections 2024-04-25T10:08:00.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.370+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/3896637250 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c89d144ce0 0x55c89d1450c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.370+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/3896637250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d116ef0 0x55c89d1415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.371+0000 7fd8acff9640 0 client.6231 ms_handle_reset on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:00.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.371+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/3896637250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d133ad0 0x55c89d1193c0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.371+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/3896637250 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c89d116230 0x55c89d118450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.371+0000 7fd8d2c10740 1 -- 172.21.15.16:0/3896637250 >> 172.21.15.16:0/3896637250 conn(0x55c89d0ebf10 msgr2=0x55c89d0eb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.371+0000 7fd8d2c10740 1 -- 172.21.15.16:0/3896637250 shutdown_connections 2024-04-25T10:08:00.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.371+0000 7fd8d2c10740 1 -- 172.21.15.16:0/3896637250 wait complete. 2024-04-25T10:08:00.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.371+0000 7fd8d2c10740 20 client.6231 trim_cache size 0 max 16384 2024-04-25T10:08:00.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.386+0000 7fd8d2c10740 1 Processor -- start 2024-04-25T10:08:00.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.387+0000 7fd8d2c10740 1 -- start start 2024-04-25T10:08:00.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.387+0000 7fd8d2c10740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d116230 0x55c89d119a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.387+0000 7fd8d2c10740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d116ef0 0x55c89d119f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.387+0000 7fd8d2c10740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c89d133ad0 0x55c89d0e54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.387+0000 7fd8d2c10740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c81ff0c0 con 0x55c89d116ef0 2024-04-25T10:08:00.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.387+0000 7fd8d2c10740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c81fe540 con 0x55c89d116230 2024-04-25T10:08:00.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.387+0000 7fd8d2c10740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c81feaf0 con 0x55c89d133ad0 2024-04-25T10:08:00.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.387+0000 7fd8cd4ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d116ef0 0x55c89d119f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.387+0000 7fd8cd4ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d116ef0 0x55c89d119f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53260/0 (socket says 172.21.15.16:53260) 2024-04-25T10:08:00.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.387+0000 7fd8cd4ab640 1 -- 172.21.15.16:0/445312987 learned_addr learned my addr 172.21.15.16:0/445312987 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:00.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.388+0000 7fd8cd4ab640 1 -- 172.21.15.16:0/445312987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c89d133ad0 msgr2=0x55c89d0e54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:00.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.388+0000 7fd8cd4ab640 1 --2- 172.21.15.16:0/445312987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c89d133ad0 0x55c89d0e54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.388+0000 7fd8cd4ab640 1 -- 172.21.15.16:0/445312987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d116230 msgr2=0x55c89d119a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:00.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.388+0000 7fd8cd4ab640 1 --2- 172.21.15.16:0/445312987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d116230 0x55c89d119a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.388+0000 7fd8cd4ab640 1 -- 172.21.15.16:0/445312987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c89d130e40 con 0x55c89d116ef0 2024-04-25T10:08:00.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.388+0000 7fd8cd4ab640 1 --2- 172.21.15.16:0/445312987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d116ef0 0x55c89d119f60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00aec0 tx=0x7fd8bc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56afa250eeff490 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.388+0000 7fd8aeffd640 1 -- 172.21.15.16:0/445312987 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc023070 con 0x55c89d116ef0 2024-04-25T10:08:00.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.388+0000 7fd8aeffd640 1 -- 172.21.15.16:0/445312987 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc013c60 con 0x55c89d116ef0 2024-04-25T10:08:00.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.388+0000 7fd8aeffd640 1 -- 172.21.15.16:0/445312987 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc019040 con 0x55c89d116ef0 2024-04-25T10:08:00.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.388+0000 7fd8d2c10740 1 -- 172.21.15.16:0/445312987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d116ef0 msgr2=0x55c89d119f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.388+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/445312987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d116ef0 0x55c89d119f60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00aec0 tx=0x7fd8bc01d000 comp rx=0 tx=0).stop 2024-04-25T10:08:00.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.388+0000 7fd8d2c10740 1 -- 172.21.15.16:0/445312987 shutdown_connections 2024-04-25T10:08:00.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.388+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/445312987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c89d133ad0 0x55c89d0e54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.388+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/445312987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d116ef0 0x55c89d119f60 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.388+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/445312987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d116230 0x55c89d119a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.388+0000 7fd8d2c10740 1 -- 172.21.15.16:0/445312987 >> 172.21.15.16:0/445312987 conn(0x55c89d0ebf10 msgr2=0x55c89d1413e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.388+0000 7fd8d2c10740 1 -- 172.21.15.16:0/445312987 shutdown_connections 2024-04-25T10:08:00.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.389+0000 7fd8d2c10740 1 -- 172.21.15.16:0/445312987 wait complete. 2024-04-25T10:08:00.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.389+0000 7fd8d2c10740 1 -- start start 2024-04-25T10:08:00.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.389+0000 7fd8d2c10740 1 Processor -- start 2024-04-25T10:08:00.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.390+0000 7fd8d2c10740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c89d116230 0x55c89d0e5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.390+0000 7fd8d2c10740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d133ad0 0x55c89d102000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.390+0000 7fd8d2c10740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d12dec0 0x55c89d102540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.390+0000 7fd8d2c10740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89d117a10 con 0x55c89d133ad0 2024-04-25T10:08:00.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.390+0000 7fd8d2c10740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89d141b10 con 0x55c89d12dec0 2024-04-25T10:08:00.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.390+0000 7fd8d2c10740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c89d103ad0 con 0x55c89d116230 2024-04-25T10:08:00.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.390+0000 7fd8d2c10740 10 client.0 _ll_register_callbacks cb 0x55c89d0e6c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:00.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.390+0000 7fd8cd4ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d133ad0 0x55c89d102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.390+0000 7fd8cd4ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d133ad0 0x55c89d102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53264/0 (socket says 172.21.15.16:53264) 2024-04-25T10:08:00.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.390+0000 7fd8cd4ab640 1 -- 172.21.15.16:0/1043968500 learned_addr learned my addr 172.21.15.16:0/1043968500 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:00.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.390+0000 7fd8cd4ab640 1 -- 172.21.15.16:0/1043968500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c89d116230 msgr2=0x55c89d0e5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:00.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.390+0000 7fd8cd4ab640 1 --2- 172.21.15.16:0/1043968500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c89d116230 0x55c89d0e5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.390+0000 7fd8ce4ad640 1 --2- 172.21.15.16:0/1043968500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d12dec0 0x55c89d102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.390+0000 7fd8cd4ab640 1 -- 172.21.15.16:0/1043968500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d12dec0 msgr2=0x55c89d102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:00.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.390+0000 7fd8cd4ab640 1 --2- 172.21.15.16:0/1043968500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d12dec0 0x55c89d102540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.390+0000 7fd8cd4ab640 1 -- 172.21.15.16:0/1043968500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c89d0e6d60 con 0x55c89d133ad0 2024-04-25T10:08:00.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.391+0000 7fd8cd4ab640 1 --2- 172.21.15.16:0/1043968500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d133ad0 0x55c89d102000 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc033040 tx=0x7fd8bc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe8840eb4ca3bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.391+0000 7fd8acff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:08:00.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.391+0000 7fd8acff9640 1 -- 172.21.15.16:0/1043968500 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc023040 con 0x55c89d133ad0 2024-04-25T10:08:00.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.391+0000 7fd8acff9640 1 -- 172.21.15.16:0/1043968500 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8bc03aaa0 con 0x55c89d133ad0 2024-04-25T10:08:00.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.391+0000 7fd8d2c10740 1 -- 172.21.15.16:0/1043968500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8bc013c60 con 0x55c89d133ad0 2024-04-25T10:08:00.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.391+0000 7fd8acff9640 1 -- 172.21.15.16:0/1043968500 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8bc03acc0 con 0x55c89d133ad0 2024-04-25T10:08:00.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.391+0000 7fd8d2c10740 10 client.6237 fetch_fsmap learned FSMap version 27 2024-04-25T10:08:00.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.391+0000 7fd8d2c10740 1 -- 172.21.15.16:0/1043968500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55c89d153df0 con 0x55c89d133ad0 2024-04-25T10:08:00.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.391+0000 7fd8acff9640 1 -- 172.21.15.16:0/1043968500 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc013c60 con 0x55c89d133ad0 2024-04-25T10:08:00.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.391+0000 7fd8acff9640 1 -- 172.21.15.16:0/1043968500 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd8bc030040 con 0x55c89d133ad0 2024-04-25T10:08:00.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.391+0000 7fd8d2c10740 10 client.6237 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:08:00.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.391+0000 7fd8d2c10740 10 client.6237 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:08:00.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.391+0000 7fd8d2c10740 20 client.6237 populate_metadata read hostname 'smithi016' 2024-04-25T10:08:00.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.391+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/1043968500 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c89d1269d0 0x55c89d126db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.391+0000 7fd8d2c10740 4 client.6237 mds_command: new command op to 35141 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:08:00.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.391+0000 7fd8d2c10740 1 -- 172.21.15.16:0/1043968500 --> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c89d05bf50 con 0x55c89d1269d0 2024-04-25T10:08:00.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.391+0000 7fd8cdcac640 1 --2- 172.21.15.16:0/1043968500 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c89d1269d0 0x55c89d126db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.392+0000 7fd8acff9640 10 client.6237 ms_handle_connect on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:00.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.392+0000 7fd8cdcac640 1 --2- 172.21.15.16:0/1043968500 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c89d1269d0 0x55c89d126db0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a074367cd97c2743 server_cookie=5e7526a5c0a58579 in_seq=0 out_seq=0 2024-04-25T10:08:00.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.392+0000 7fd8acff9640 1 -- 172.21.15.16:0/1043968500 <== mds.0 v2:172.21.15.16:6832/3311584521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55c89d05bf50 con 0x55c89d1269d0 2024-04-25T10:08:00.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.392+0000 7fd8acff9640 10 client.6237 handle_command_reply: tid=0 2024-04-25T10:08:00.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.392+0000 7fd8d2c10740 1 client.6237 shutdown 2024-04-25T10:08:00.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.392+0000 7fd8d2c10740 1 -- 172.21.15.16:0/1043968500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d133ad0 msgr2=0x55c89d102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.392+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/1043968500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d133ad0 0x55c89d102000 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc033040 tx=0x7fd8bc011000 comp rx=0 tx=0).stop 2024-04-25T10:08:00.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.392+0000 7fd8d2c10740 1 -- 172.21.15.16:0/1043968500 shutdown_connections 2024-04-25T10:08:00.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.392+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/1043968500 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c89d1269d0 0x55c89d126db0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.392+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/1043968500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c89d12dec0 0x55c89d102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.392+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/1043968500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c89d133ad0 0x55c89d102000 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.392+0000 7fd8d2c10740 1 --2- 172.21.15.16:0/1043968500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c89d116230 0x55c89d0e5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.392+0000 7fd8acff9640 0 client.6237 ms_handle_reset on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:00.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.392+0000 7fd8d2c10740 1 -- 172.21.15.16:0/1043968500 >> 172.21.15.16:0/1043968500 conn(0x55c89d0ebf10 msgr2=0x55c89d1199c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.392+0000 7fd8d2c10740 1 -- 172.21.15.16:0/1043968500 shutdown_connections 2024-04-25T10:08:00.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.392+0000 7fd8d2c10740 1 -- 172.21.15.16:0/1043968500 wait complete. 2024-04-25T10:08:00.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.392+0000 7fd8d2c10740 20 client.6237 trim_cache size 0 max 16384 2024-04-25T10:08:00.404 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:08:00.404 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "34bc5eca-3a21-42a6-a26f-07da0905e8bc", 2024-04-25T10:08:00.404 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-04-25T10:08:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 35141, 2024-04-25T10:08:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-04-25T10:08:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-04-25T10:08:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-04-25T10:08:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 9.0998843829999991, 2024-04-25T10:08:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-04-25T10:08:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 59, 2024-04-25T10:08:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 59, 2024-04-25T10:08:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 14.439554679 2024-04-25T10:08:00.405 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:08:00.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.393+0000 7fd8cff10640 1 -- 172.21.15.16:0/3947883494 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd8b406b4d0 msgr2=0x7fd8b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.393+0000 7fd8cff10640 1 --2- 172.21.15.16:0/3947883494 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd8b406b4d0 0x7fd8b406d990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc006fd0 tx=0x7fd8bc008040 comp rx=0 tx=0).stop 2024-04-25T10:08:00.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.393+0000 7fd8cff10640 1 -- 172.21.15.16:0/3947883494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c80b9720 msgr2=0x7fd8c814e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.393+0000 7fd8cff10640 1 --2- 172.21.15.16:0/3947883494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c80b9720 0x7fd8c814e110 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0000c00 tx=0x7fd8c00046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.393+0000 7fd8cff10640 1 -- 172.21.15.16:0/3947883494 shutdown_connections 2024-04-25T10:08:00.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.393+0000 7fd8cff10640 1 --2- 172.21.15.16:0/3947883494 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd8b406b4d0 0x7fd8b406d990 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.393+0000 7fd8cff10640 1 --2- 172.21.15.16:0/3947883494 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8c8161090 0x7fd8c81e91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.393+0000 7fd8cff10640 1 --2- 172.21.15.16:0/3947883494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8c80cece0 0x7fd8c80cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.393+0000 7fd8cff10640 1 --2- 172.21.15.16:0/3947883494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c80b9720 0x7fd8c814e110 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.393+0000 7fd8cff10640 1 -- 172.21.15.16:0/3947883494 >> 172.21.15.16:0/3947883494 conn(0x7fd8c80b55e0 msgr2=0x7fd8c80c9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.393+0000 7fd8cff10640 1 -- 172.21.15.16:0/3947883494 shutdown_connections 2024-04-25T10:08:00.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.393+0000 7fd8cff10640 1 -- 172.21.15.16:0/3947883494 wait complete. 2024-04-25T10:08:00.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:08:00.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.504+0000 7fdaa9db0640 1 Processor -- start 2024-04-25T10:08:00.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.504+0000 7fdaa9db0640 1 -- start start 2024-04-25T10:08:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.505+0000 7fdaa9db0640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fdaa40b5b60 0x7fdaa41687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.505+0000 7fdaa9db0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa415e0c0 con 0x7fdaa415dce0 2024-04-25T10:08:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.505+0000 7fdaa9db0640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa40d2510 con 0x7fdaa40d35b0 2024-04-25T10:08:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.505+0000 7fdaa9db0640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa40b7b10 con 0x7fdaa40b5b60 2024-04-25T10:08:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.505+0000 7fdaa37fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdaa415dce0 0x7fdaa40d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54474/0 (socket says 172.21.15.16:54474) 2024-04-25T10:08:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.505+0000 7fdaa37fe640 1 -- 172.21.15.16:0/2591185219 learned_addr learned my addr 172.21.15.16:0/2591185219 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:08:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.505+0000 7fdaa3fff640 1 --2- 172.21.15.16:0/2591185219 >> v2:172.21.15.186:3300/0 conn(0x7fdaa40b5b60 0x7fdaa41687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.506+0000 7fdaa27fc640 1 -- 172.21.15.16:0/2591185219 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000871743 0 0) 0x7fdaa415e0c0 con 0x7fdaa415dce0 2024-04-25T10:08:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.506+0000 7fdaa27fc640 1 -- 172.21.15.16:0/2591185219 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda8c003520 con 0x7fdaa415dce0 2024-04-25T10:08:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.506+0000 7fdaa27fc640 1 -- 172.21.15.16:0/2591185219 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2692379647 0 0) 0x7fda8c003520 con 0x7fdaa415dce0 2024-04-25T10:08:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.506+0000 7fdaa3fff640 1 -- 172.21.15.16:0/2591185219 >> v1:172.21.15.186:6789/0 conn(0x7fdaa40d35b0 legacy=0x7fdaa40b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.506+0000 7fdaa3fff640 1 -- 172.21.15.16:0/2591185219 >> v1:172.21.15.16:6789/0 conn(0x7fdaa415dce0 legacy=0x7fdaa40d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.506+0000 7fdaa3fff640 1 -- 172.21.15.16:0/2591185219 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa40d27f0 con 0x7fdaa40b5b60 2024-04-25T10:08:00.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.507+0000 7fdaa3fff640 1 --2- 172.21.15.16:0/2591185219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaa40b5b60 0x7fdaa41687f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009d90 tx=0x7fda9c02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62bf05ed4b41ab0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.507+0000 7fdaa27fc640 1 -- 172.21.15.16:0/2591185219 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda94001670 con 0x7fdaa40b5b60 2024-04-25T10:08:00.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.507+0000 7fdaa27fc640 1 -- 172.21.15.16:0/2591185219 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9c007540 con 0x7fdaa40b5b60 2024-04-25T10:08:00.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.507+0000 7fdaa27fc640 1 -- 172.21.15.16:0/2591185219 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c0078f0 con 0x7fdaa40b5b60 2024-04-25T10:08:00.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.507+0000 7fdaa9db0640 1 -- 172.21.15.16:0/2591185219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaa40b5b60 msgr2=0x7fdaa41687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.507+0000 7fdaa9db0640 1 --2- 172.21.15.16:0/2591185219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaa40b5b60 0x7fdaa41687f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009d90 tx=0x7fda9c02a680 comp rx=0 tx=0).stop 2024-04-25T10:08:00.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.507+0000 7fdaa9db0640 1 -- 172.21.15.16:0/2591185219 shutdown_connections 2024-04-25T10:08:00.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.507+0000 7fdaa9db0640 1 --2- 172.21.15.16:0/2591185219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaa40b5b60 0x7fdaa41687f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.507+0000 7fdaa9db0640 1 -- 172.21.15.16:0/2591185219 >> 172.21.15.16:0/2591185219 conn(0x7fdaa4072900 msgr2=0x7fdaa40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.507+0000 7fdaa9db0640 1 -- 172.21.15.16:0/2591185219 shutdown_connections 2024-04-25T10:08:00.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.507+0000 7fdaa9db0640 1 -- 172.21.15.16:0/2591185219 wait complete. 2024-04-25T10:08:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.509+0000 7fdaa9db0640 1 Processor -- start 2024-04-25T10:08:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.509+0000 7fdaa9db0640 1 -- start start 2024-04-25T10:08:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa9db0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa40b5b60 0x7fdaa40ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa9db0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaa40d35b0 0x7fdaa41565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa9db0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdaa415dce0 0x7fdaa4152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa9db0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa40d3090 con 0x7fdaa40b5b60 2024-04-25T10:08:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa9db0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa40d2db0 con 0x7fdaa40d35b0 2024-04-25T10:08:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa9db0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa40d2160 con 0x7fdaa415dce0 2024-04-25T10:08:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa40b5b60 0x7fdaa40ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa40b5b60 0x7fdaa40ccc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53278/0 (socket says 172.21.15.16:53278) 2024-04-25T10:08:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa37fe640 1 -- 172.21.15.16:0/2008805397 learned_addr learned my addr 172.21.15.16:0/2008805397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa2ffd640 1 --2- 172.21.15.16:0/2008805397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaa40d35b0 0x7fdaa41565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa37fe640 1 -- 172.21.15.16:0/2008805397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdaa415dce0 msgr2=0x7fdaa4152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa37fe640 1 --2- 172.21.15.16:0/2008805397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdaa415dce0 0x7fdaa4152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa37fe640 1 -- 172.21.15.16:0/2008805397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaa40d35b0 msgr2=0x7fdaa41565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa37fe640 1 --2- 172.21.15.16:0/2008805397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaa40d35b0 0x7fdaa41565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa37fe640 1 -- 172.21.15.16:0/2008805397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa4153480 con 0x7fdaa40b5b60 2024-04-25T10:08:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa2ffd640 1 --2- 172.21.15.16:0/2008805397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaa40d35b0 0x7fdaa41565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa37fe640 1 --2- 172.21.15.16:0/2008805397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa40b5b60 0x7fdaa40ccc70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fda9400c580 tx=0x7fda9400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fda8bfff640 1 -- 172.21.15.16:0/2008805397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda94017070 con 0x7fdaa40b5b60 2024-04-25T10:08:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa9db0640 1 -- 172.21.15.16:0/2008805397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa40cd370 con 0x7fdaa40b5b60 2024-04-25T10:08:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fda8bfff640 1 -- 172.21.15.16:0/2008805397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda94005ce0 con 0x7fdaa40b5b60 2024-04-25T10:08:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fda8bfff640 1 -- 172.21.15.16:0/2008805397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda94007330 con 0x7fdaa40b5b60 2024-04-25T10:08:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.510+0000 7fdaa9db0640 1 -- 172.21.15.16:0/2008805397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa415e450 con 0x7fdaa40b5b60 2024-04-25T10:08:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.511+0000 7fdaa9db0640 1 -- 172.21.15.16:0/2008805397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa40749b0 con 0x7fdaa40b5b60 2024-04-25T10:08:00.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.512+0000 7fda8bfff640 1 -- 172.21.15.16:0/2008805397 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda9401a070 con 0x7fdaa40b5b60 2024-04-25T10:08:00.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.512+0000 7fda8bfff640 1 --2- 172.21.15.16:0/2008805397 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fda9006b4d0 0x7fda9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.513+0000 7fda8bfff640 1 -- 172.21.15.16:0/2008805397 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda940897d0 con 0x7fdaa40b5b60 2024-04-25T10:08:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.513+0000 7fdaa2ffd640 1 --2- 172.21.15.16:0/2008805397 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fda9006b4d0 0x7fda9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.513+0000 7fdaa2ffd640 1 --2- 172.21.15.16:0/2008805397 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fda9006b4d0 0x7fda9006d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40b2d50 tx=0x7fda98009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.517+0000 7fda8bfff640 1 -- 172.21.15.16:0/2008805397 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fda94058660 con 0x7fdaa40b5b60 2024-04-25T10:08:00.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.740+0000 7fdaa9db0640 1 -- 172.21.15.16:0/2008805397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdaa40cd890 con 0x7fdaa40b5b60 2024-04-25T10:08:00.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.740+0000 7fda8bfff640 1 -- 172.21.15.16:0/2008805397 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7fda94057f60 con 0x7fdaa40b5b60 2024-04-25T10:08:00.746 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:08:00.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.742+0000 7fdaa9db0640 1 -- 172.21.15.16:0/2008805397 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fda9006b4d0 msgr2=0x7fda9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.742+0000 7fdaa9db0640 1 --2- 172.21.15.16:0/2008805397 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fda9006b4d0 0x7fda9006d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40b2d50 tx=0x7fda98009040 comp rx=0 tx=0).stop 2024-04-25T10:08:00.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.742+0000 7fdaa9db0640 1 -- 172.21.15.16:0/2008805397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa40b5b60 msgr2=0x7fdaa40ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.742+0000 7fdaa9db0640 1 --2- 172.21.15.16:0/2008805397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa40b5b60 0x7fdaa40ccc70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fda9400c580 tx=0x7fda9400ca40 comp rx=0 tx=0).stop 2024-04-25T10:08:00.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.742+0000 7fdaa9db0640 1 -- 172.21.15.16:0/2008805397 shutdown_connections 2024-04-25T10:08:00.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.742+0000 7fdaa9db0640 1 --2- 172.21.15.16:0/2008805397 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fda9006b4d0 0x7fda9006d990 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.742+0000 7fdaa9db0640 1 --2- 172.21.15.16:0/2008805397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdaa415dce0 0x7fdaa4152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.742+0000 7fdaa9db0640 1 --2- 172.21.15.16:0/2008805397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaa40d35b0 0x7fdaa41565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.742+0000 7fdaa9db0640 1 --2- 172.21.15.16:0/2008805397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa40b5b60 0x7fdaa40ccc70 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.742+0000 7fdaa9db0640 1 -- 172.21.15.16:0/2008805397 >> 172.21.15.16:0/2008805397 conn(0x7fdaa4072900 msgr2=0x7fdaa40b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.743+0000 7fdaa9db0640 1 -- 172.21.15.16:0/2008805397 shutdown_connections 2024-04-25T10:08:00.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:00.743+0000 7fdaa9db0640 1 -- 172.21.15.16:0/2008805397 wait complete. 2024-04-25T10:08:00.759 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:08:00.759 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:08:00.759 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-04-25T10:08:00.759 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:08:00.760 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:00.760 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:08:00.760 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:08:00.760 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:08:00.760 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:00.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:00.760 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-04-25T10:08:00.776 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:08:00.777 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:08:00.777 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:08:00.777 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:00.777 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:08:00.777 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:00.778 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:08:00.778 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:00.778 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-04-25T10:08:00.778 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:08:00.778 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:00.778 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-04-25T10:08:00.778 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:08:00.778 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:00.778 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-04-25T10:08:00.779 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:08:00.779 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42480sec preferred_lft 42480sec 2024-04-25T10:08:00.779 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-25T10:08:00.779 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:00.779 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:08:00.779 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:00.779 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-04-25T10:08:00.779 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:08:00.779 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:00.779 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:08:00.779 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:00.779 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f06b:fdff:fed4:a335/64 scope link 2024-04-25T10:08:00.779 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:00.779 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:08:00.780 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:00.780 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::90ad:4fff:fe91:3c33/64 scope link 2024-04-25T10:08:00.780 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:00.780 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:08:00.780 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether be:14:4a:3c:16:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:00.780 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::bc14:4aff:fe3c:1605/64 scope link 2024-04-25T10:08:00.780 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:00.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:00.780 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:08:00.840 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:08:00.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:00.842 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-04-25T10:08:00.899 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-04-25T10:08:00.899 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:08:00.899 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:08:00.899 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:08:00.899 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:08:00.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:00.900 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:08:00.900 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:08:00.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:08:00.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:08:00.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:08:00.900 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:08:00.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:00.986 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:08:01.001 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:08:01.001 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:08:01.001 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:01.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:01.002 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-04-25T10:08:01.067 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:08:01.067 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:01.122 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:01.123 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:08:01.123 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:01.178 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:08:01.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:01.178 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:08:01.260 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:08:01.260 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:08:01.260 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:08:01.270 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:08:01.270 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:08:01.270 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:08:01.271 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:08:01.271 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:08:01.274 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:08:01.274 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:08:01.274 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:08:01.274 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:08:01.274 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:08:01.274 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:08:01.274 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:08:01.274 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:08:01.275 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:08:01.275 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:08:01.275 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:08:01.275 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:08:01.275 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:08:01.276 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:08:01.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:01.276 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:01.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:01.356 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:08:01.401 INFO:teuthology.orchestra.run.smithi016.stdout:client35240 2024-04-25T10:08:01.401 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:08:01.401 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:01.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:01.458 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-04-25T10:08:01.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:01.518 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-04-25T10:08:01.574 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:08:01.574 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:08:01.574 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:08:01.574 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:08:01.574 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-04-25T10:08:01.574 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:08:01.682 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627777 2024-04-25T10:08:01.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:08:01.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:01.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.825+0000 7f904fdce640 1 Processor -- start 2024-04-25T10:08:01.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.825+0000 7f904fdce640 1 -- start start 2024-04-25T10:08:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.826+0000 7f904fdce640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f904815dce0 0x7f90480d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.826+0000 7f904fdce640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90480d35b0 0x7f90480b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.826+0000 7f904fdce640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f90480d1f90 0x7f90480b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.826+0000 7f904fdce640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f904815e0c0 con 0x7f90480d35b0 2024-04-25T10:08:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.826+0000 7f904fdce640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90480d2510 con 0x7f90480d1f90 2024-04-25T10:08:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.826+0000 7f904fdce640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90480d27f0 con 0x7f904815dce0 2024-04-25T10:08:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.826+0000 7f904d369640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90480d35b0 0x7f90480b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.826+0000 7f904d369640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90480d35b0 0x7f90480b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53294/0 (socket says 172.21.15.16:53294) 2024-04-25T10:08:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.826+0000 7f904d369640 1 -- 172.21.15.16:0/3972732728 learned_addr learned my addr 172.21.15.16:0/3972732728 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.826+0000 7f904e36b640 1 --2- 172.21.15.16:0/3972732728 >> v2:172.21.15.186:3300/0 conn(0x7f90480d1f90 0x7f90480b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.826+0000 7f904db6a640 1 --2- 172.21.15.16:0/3972732728 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f904815dce0 0x7f90480d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.826+0000 7f904d369640 1 -- 172.21.15.16:0/3972732728 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f904815dce0 msgr2=0x7f90480d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.826+0000 7f904d369640 1 --2- 172.21.15.16:0/3972732728 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f904815dce0 0x7f90480d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.826+0000 7f904d369640 1 -- 172.21.15.16:0/3972732728 >> v2:172.21.15.186:3300/0 conn(0x7f90480d1f90 msgr2=0x7f90480b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.826+0000 7f904d369640 1 --2- 172.21.15.16:0/3972732728 >> v2:172.21.15.186:3300/0 conn(0x7f90480d1f90 0x7f90480b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.826+0000 7f904d369640 1 -- 172.21.15.16:0/3972732728 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90480d2ad0 con 0x7f90480d35b0 2024-04-25T10:08:01.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.827+0000 7f904d369640 1 --2- 172.21.15.16:0/3972732728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90480d35b0 0x7f90480b5450 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9048161710 tx=0x7f903802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e683fa3e2e71cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.827+0000 7f904cb68640 1 -- 172.21.15.16:0/3972732728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038037070 con 0x7f90480d35b0 2024-04-25T10:08:01.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.827+0000 7f904cb68640 1 -- 172.21.15.16:0/3972732728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9038032440 con 0x7f90480d35b0 2024-04-25T10:08:01.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.827+0000 7f904cb68640 1 -- 172.21.15.16:0/3972732728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038032810 con 0x7f90480d35b0 2024-04-25T10:08:01.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.827+0000 7f904fdce640 1 -- 172.21.15.16:0/3972732728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90480d35b0 msgr2=0x7f90480b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.827+0000 7f904fdce640 1 --2- 172.21.15.16:0/3972732728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90480d35b0 0x7f90480b5450 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9048161710 tx=0x7f903802a140 comp rx=0 tx=0).stop 2024-04-25T10:08:01.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.828+0000 7f904fdce640 1 -- 172.21.15.16:0/3972732728 shutdown_connections 2024-04-25T10:08:01.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.828+0000 7f904fdce640 1 --2- 172.21.15.16:0/3972732728 >> v2:172.21.15.186:3300/0 conn(0x7f90480d1f90 0x7f90480b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.828+0000 7f904fdce640 1 --2- 172.21.15.16:0/3972732728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90480d35b0 0x7f90480b5450 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.828+0000 7f904fdce640 1 --2- 172.21.15.16:0/3972732728 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f904815dce0 0x7f90480d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.828+0000 7f904fdce640 1 -- 172.21.15.16:0/3972732728 >> 172.21.15.16:0/3972732728 conn(0x7f9048072900 msgr2=0x7f90480d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:01.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.828+0000 7f904fdce640 1 -- 172.21.15.16:0/3972732728 shutdown_connections 2024-04-25T10:08:01.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.828+0000 7f904fdce640 1 -- 172.21.15.16:0/3972732728 wait complete. 2024-04-25T10:08:01.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.829+0000 7f904fdce640 1 Processor -- start 2024-04-25T10:08:01.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.829+0000 7f904fdce640 1 -- start start 2024-04-25T10:08:01.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.829+0000 7f904fdce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90480d1f90 0x7f90480d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.829+0000 7f904fdce640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90480d35b0 0x7f90480d4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.829+0000 7f904fdce640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f904815dce0 0x7f904815f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.829+0000 7f904fdce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90480b9820 con 0x7f90480d1f90 2024-04-25T10:08:01.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.829+0000 7f904fdce640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90480d3090 con 0x7f904815dce0 2024-04-25T10:08:01.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.829+0000 7f904fdce640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90480b99a0 con 0x7f90480d35b0 2024-04-25T10:08:01.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.830+0000 7f904db6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90480d1f90 0x7f90480d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.830+0000 7f904db6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90480d1f90 0x7f90480d3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53296/0 (socket says 172.21.15.16:53296) 2024-04-25T10:08:01.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.830+0000 7f904db6a640 1 -- 172.21.15.16:0/4059497269 learned_addr learned my addr 172.21.15.16:0/4059497269 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:01.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.830+0000 7f904e36b640 1 --2- 172.21.15.16:0/4059497269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f904815dce0 0x7f904815f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.830+0000 7f904d369640 1 --2- 172.21.15.16:0/4059497269 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90480d35b0 0x7f90480d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.830+0000 7f904db6a640 1 -- 172.21.15.16:0/4059497269 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90480d35b0 msgr2=0x7f90480d4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.830+0000 7f904db6a640 1 --2- 172.21.15.16:0/4059497269 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90480d35b0 0x7f90480d4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.830+0000 7f904db6a640 1 -- 172.21.15.16:0/4059497269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f904815dce0 msgr2=0x7f904815f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.830+0000 7f904db6a640 1 --2- 172.21.15.16:0/4059497269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f904815dce0 0x7f904815f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.830+0000 7f904db6a640 1 -- 172.21.15.16:0/4059497269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90481f35f0 con 0x7f90480d1f90 2024-04-25T10:08:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.830+0000 7f904db6a640 1 --2- 172.21.15.16:0/4059497269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90480d1f90 0x7f90480d3ee0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f904400e9e0 tx=0x7f904400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.830+0000 7f903e7fc640 1 -- 172.21.15.16:0/4059497269 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9044017070 con 0x7f90480d1f90 2024-04-25T10:08:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.830+0000 7f903e7fc640 1 -- 172.21.15.16:0/4059497269 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904400c9d0 con 0x7f90480d1f90 2024-04-25T10:08:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.830+0000 7f904fdce640 1 -- 172.21.15.16:0/4059497269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90481f38d0 con 0x7f90480d1f90 2024-04-25T10:08:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.830+0000 7f903e7fc640 1 -- 172.21.15.16:0/4059497269 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904400cdc0 con 0x7f90480d1f90 2024-04-25T10:08:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.830+0000 7f904fdce640 1 -- 172.21.15.16:0/4059497269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90481f3e90 con 0x7f90480d1f90 2024-04-25T10:08:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.831+0000 7f904fdce640 1 -- 172.21.15.16:0/4059497269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90480749b0 con 0x7f90480d1f90 2024-04-25T10:08:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.832+0000 7f903e7fc640 1 -- 172.21.15.16:0/4059497269 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f904401a070 con 0x7f90480d1f90 2024-04-25T10:08:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.832+0000 7f903e7fc640 1 --2- 172.21.15.16:0/4059497269 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f902c06b4d0 0x7f902c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.832+0000 7f903e7fc640 1 -- 172.21.15.16:0/4059497269 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f904408a5b0 con 0x7f90480d1f90 2024-04-25T10:08:01.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.832+0000 7f904d369640 1 --2- 172.21.15.16:0/4059497269 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f902c06b4d0 0x7f902c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.833+0000 7f904d369640 1 --2- 172.21.15.16:0/4059497269 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f902c06b4d0 0x7f902c06d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9038034000 tx=0x7f903802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:01.836+0000 7f903e7fc640 1 -- 172.21.15.16:0/4059497269 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90440594c0 con 0x7f90480d1f90 2024-04-25T10:08:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.053+0000 7f904fdce640 1 -- 172.21.15.16:0/4059497269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90481f4270 con 0x7f90480d1f90 2024-04-25T10:08:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.054+0000 7f903e7fc640 1 -- 172.21.15.16:0/4059497269 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f9044058dc0 con 0x7f90480d1f90 2024-04-25T10:08:02.060 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:08:02.060 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2270839865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2270839865},{"type":"v1","addr":"172.21.15.186:6836","nonce":2270839865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5647,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/925282579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":925282579},{"type":"v1","addr":"172.21.15.186:6837","nonce":925282579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2471714328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2471714328},{"type":"v1","addr":"172.21.15.16:6835","nonce":2471714328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:50.665436+0000","modified":"2024-04-25T10:07:52.663229+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35141":{"gid":35141,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6834/3311584521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3311584521},{"type":"v1","addr":"172.21.15.16:6834","nonce":3311584521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:08:02.060 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-04-25T10:08:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.056+0000 7f904fdce640 1 -- 172.21.15.16:0/4059497269 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f902c06b4d0 msgr2=0x7f902c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.056+0000 7f904fdce640 1 --2- 172.21.15.16:0/4059497269 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f902c06b4d0 0x7f902c06d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9038034000 tx=0x7f903802e000 comp rx=0 tx=0).stop 2024-04-25T10:08:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.056+0000 7f904fdce640 1 -- 172.21.15.16:0/4059497269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90480d1f90 msgr2=0x7f90480d3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.056+0000 7f904fdce640 1 --2- 172.21.15.16:0/4059497269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90480d1f90 0x7f90480d3ee0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f904400e9e0 tx=0x7f904400eea0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.056+0000 7f904fdce640 1 -- 172.21.15.16:0/4059497269 shutdown_connections 2024-04-25T10:08:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.056+0000 7f904fdce640 1 --2- 172.21.15.16:0/4059497269 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f902c06b4d0 0x7f902c06d990 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.056+0000 7f904fdce640 1 --2- 172.21.15.16:0/4059497269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f904815dce0 0x7f904815f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.056+0000 7f904fdce640 1 --2- 172.21.15.16:0/4059497269 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90480d35b0 0x7f90480d4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.056+0000 7f904fdce640 1 --2- 172.21.15.16:0/4059497269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90480d1f90 0x7f90480d3ee0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.056+0000 7f904fdce640 1 -- 172.21.15.16:0/4059497269 >> 172.21.15.16:0/4059497269 conn(0x7f9048072900 msgr2=0x7f90480b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.056+0000 7f904fdce640 1 -- 172.21.15.16:0/4059497269 shutdown_connections 2024-04-25T10:08:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.056+0000 7f904fdce640 1 -- 172.21.15.16:0/4059497269 wait complete. 2024-04-25T10:08:02.072 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:08:02.195 INFO:teuthology.orchestra.run.smithi016.stdout:{"message":"","return_code":0} 2024-04-25T10:08:02.195 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:08:02.195 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:08:02.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.232+0000 7f12739cdf40 1 Processor -- start 2024-04-25T10:08:02.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.232+0000 7f12739cdf40 1 -- start start 2024-04-25T10:08:02.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.233+0000 7f12739cdf40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b8ea45880 0x560b8eadfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.233+0000 7f12739cdf40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560b8eae8b60 0x560b8eae7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.233+0000 7f12739cdf40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b8eaeb5e0 0x560b8eae7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.233+0000 7f12739cdf40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8e9bcdf0 con 0x560b8ea45880 2024-04-25T10:08:02.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.233+0000 7f12739cdf40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8ea461d0 con 0x560b8eaeb5e0 2024-04-25T10:08:02.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.233+0000 7f12739cdf40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8eae3990 con 0x560b8eae8b60 2024-04-25T10:08:02.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.233+0000 7f12714ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b8ea45880 0x560b8eadfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.233+0000 7f12714ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b8ea45880 0x560b8eadfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53322/0 (socket says 172.21.15.16:53322) 2024-04-25T10:08:02.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.233+0000 7f12714ce640 1 -- 172.21.15.16:0/1353935664 learned_addr learned my addr 172.21.15.16:0/1353935664 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:02.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.233+0000 7f12714ce640 1 -- 172.21.15.16:0/1353935664 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560b8eae8b60 msgr2=0x560b8eae7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.233+0000 7f12714ce640 1 --2- 172.21.15.16:0/1353935664 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560b8eae8b60 0x560b8eae7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.233+0000 7f12714ce640 1 -- 172.21.15.16:0/1353935664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b8eaeb5e0 msgr2=0x560b8eae7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.233+0000 7f12714ce640 1 --2- 172.21.15.16:0/1353935664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b8eaeb5e0 0x560b8eae7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.233+0000 7f12714ce640 1 -- 172.21.15.16:0/1353935664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b8e942c50 con 0x560b8ea45880 2024-04-25T10:08:02.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.234+0000 7f12714ce640 1 --2- 172.21.15.16:0/1353935664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b8ea45880 0x560b8eadfe20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f126400b2e0 tx=0x7f126400b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1701e5c29fa007 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.234+0000 7f12637fe640 1 -- 172.21.15.16:0/1353935664 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1264008070 con 0x560b8ea45880 2024-04-25T10:08:02.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.234+0000 7f12637fe640 1 -- 172.21.15.16:0/1353935664 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126400d500 con 0x560b8ea45880 2024-04-25T10:08:02.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.234+0000 7f12637fe640 1 -- 172.21.15.16:0/1353935664 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126400d8b0 con 0x560b8ea45880 2024-04-25T10:08:02.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.234+0000 7f12739cdf40 1 -- 172.21.15.16:0/1353935664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b8ea45880 msgr2=0x560b8eadfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.234+0000 7f12739cdf40 1 --2- 172.21.15.16:0/1353935664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b8ea45880 0x560b8eadfe20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f126400b2e0 tx=0x7f126400b7a0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.234+0000 7f12739cdf40 1 -- 172.21.15.16:0/1353935664 shutdown_connections 2024-04-25T10:08:02.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.234+0000 7f12739cdf40 1 --2- 172.21.15.16:0/1353935664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b8eaeb5e0 0x560b8eae7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.234+0000 7f12739cdf40 1 --2- 172.21.15.16:0/1353935664 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560b8eae8b60 0x560b8eae7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.234+0000 7f12739cdf40 1 --2- 172.21.15.16:0/1353935664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b8ea45880 0x560b8eadfe20 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.234+0000 7f12739cdf40 1 -- 172.21.15.16:0/1353935664 >> 172.21.15.16:0/1353935664 conn(0x560b8ea403d0 msgr2=0x560b8ea141f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.234+0000 7f12739cdf40 1 -- 172.21.15.16:0/1353935664 shutdown_connections 2024-04-25T10:08:02.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.234+0000 7f12739cdf40 1 -- 172.21.15.16:0/1353935664 wait complete. 2024-04-25T10:08:02.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.235+0000 7f12739cdf40 1 Processor -- start 2024-04-25T10:08:02.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.235+0000 7f12739cdf40 1 -- start start 2024-04-25T10:08:02.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f12739cdf40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560b8ea45880 0x560b8ea18370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f12739cdf40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b8eae8b60 0x560b8eae6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f12739cdf40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b8eaeb5e0 0x560b8eae73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f12739cdf40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8eaeaee0 con 0x560b8eae8b60 2024-04-25T10:08:02.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f12739cdf40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8eaeac40 con 0x560b8eaeb5e0 2024-04-25T10:08:02.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f12739cdf40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8eae9070 con 0x560b8ea45880 2024-04-25T10:08:02.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f1270ccd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b8eae8b60 0x560b8eae6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f1270ccd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b8eae8b60 0x560b8eae6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53336/0 (socket says 172.21.15.16:53336) 2024-04-25T10:08:02.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f1270ccd640 1 -- 172.21.15.16:0/106497451 learned_addr learned my addr 172.21.15.16:0/106497451 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:02.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f1271ccf640 1 --2- 172.21.15.16:0/106497451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b8eaeb5e0 0x560b8eae73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f12714ce640 1 --2- 172.21.15.16:0/106497451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560b8ea45880 0x560b8ea18370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f1270ccd640 1 -- 172.21.15.16:0/106497451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560b8ea45880 msgr2=0x560b8ea18370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f1270ccd640 1 --2- 172.21.15.16:0/106497451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560b8ea45880 0x560b8ea18370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f1270ccd640 1 -- 172.21.15.16:0/106497451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b8eaeb5e0 msgr2=0x560b8eae73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f1270ccd640 1 --2- 172.21.15.16:0/106497451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560b8eaeb5e0 0x560b8eae73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f1270ccd640 1 -- 172.21.15.16:0/106497451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b8eb81970 con 0x560b8eae8b60 2024-04-25T10:08:02.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f12714ce640 1 --2- 172.21.15.16:0/106497451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560b8ea45880 0x560b8ea18370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:02.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f1270ccd640 1 --2- 172.21.15.16:0/106497451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b8eae8b60 0x560b8eae6eb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1268009d30 tx=0x7f1268006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f12627fc640 1 -- 172.21.15.16:0/106497451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1268017070 con 0x560b8eae8b60 2024-04-25T10:08:02.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f12627fc640 1 -- 172.21.15.16:0/106497451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1268009e20 con 0x560b8eae8b60 2024-04-25T10:08:02.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f12627fc640 1 -- 172.21.15.16:0/106497451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1268012650 con 0x560b8eae8b60 2024-04-25T10:08:02.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f12739cdf40 1 -- 172.21.15.16:0/106497451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560b8eb81c50 con 0x560b8eae8b60 2024-04-25T10:08:02.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.236+0000 7f12739cdf40 1 -- 172.21.15.16:0/106497451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b8eb82210 con 0x560b8eae8b60 2024-04-25T10:08:02.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.238+0000 7f12627fc640 1 -- 172.21.15.16:0/106497451 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f126801a070 con 0x560b8eae8b60 2024-04-25T10:08:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.238+0000 7f12627fc640 1 --2- 172.21.15.16:0/106497451 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1254069eb0 0x7f125406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.238+0000 7f12627fc640 1 -- 172.21.15.16:0/106497451 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1268087f50 con 0x560b8eae8b60 2024-04-25T10:08:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.238+0000 7f12739cdf40 1 --2- 172.21.15.16:0/106497451 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x560b8ead7250 0x560b8eb83450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.238+0000 7f12739cdf40 1 -- 172.21.15.16:0/106497451 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59) v9 -- 0x560b8eaebad0 con 0x560b8ead7250 2024-04-25T10:08:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.238+0000 7f12714ce640 1 --2- 172.21.15.16:0/106497451 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1254069eb0 0x7f125406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.238+0000 7f1271ccf640 1 --2- 172.21.15.16:0/106497451 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x560b8ead7250 0x560b8eb83450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.239+0000 7f1271ccf640 1 --2- 172.21.15.16:0/106497451 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x560b8ead7250 0x560b8eb83450 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.239+0000 7f12714ce640 1 --2- 172.21.15.16:0/106497451 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1254069eb0 0x7f125406c370 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f126400c010 tx=0x7f1264008000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.239+0000 7f1271ccf640 1 -- 172.21.15.16:0/106497451 <== osd.2 v2:172.21.15.16:6800/2094356376 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7f126c04e040 con 0x560b8ead7250 2024-04-25T10:08:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.239+0000 7f12739cdf40 1 -- 172.21.15.16:0/106497451 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x560b8ead7250 msgr2=0x560b8eb83450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.239+0000 7f12739cdf40 1 --2- 172.21.15.16:0/106497451 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x560b8ead7250 0x560b8eb83450 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.239+0000 7f12739cdf40 1 -- 172.21.15.16:0/106497451 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1254069eb0 msgr2=0x7f125406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.239+0000 7f12739cdf40 1 --2- 172.21.15.16:0/106497451 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1254069eb0 0x7f125406c370 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f126400c010 tx=0x7f1264008000 comp rx=0 tx=0).stop 2024-04-25T10:08:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.239+0000 7f12739cdf40 1 -- 172.21.15.16:0/106497451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b8eae8b60 msgr2=0x560b8eae6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.239+0000 7f12739cdf40 1 --2- 172.21.15.16:0/106497451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b8eae8b60 0x560b8eae6eb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1268009d30 tx=0x7f1268006750 comp rx=0 tx=0).stop 2024-04-25T10:08:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.239+0000 7f1271ccf640 1 -- 172.21.15.16:0/106497451 reap_dead start 2024-04-25T10:08:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.239+0000 7f12739cdf40 1 -- 172.21.15.16:0/106497451 shutdown_connections 2024-04-25T10:08:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.239+0000 7f12739cdf40 1 -- 172.21.15.16:0/106497451 >> 172.21.15.16:0/106497451 conn(0x560b8ea403d0 msgr2=0x560b8eadfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.239+0000 7f12739cdf40 1 -- 172.21.15.16:0/106497451 shutdown_connections 2024-04-25T10:08:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.239+0000 7f12739cdf40 1 -- 172.21.15.16:0/106497451 wait complete. 2024-04-25T10:08:02.253 INFO:teuthology.orchestra.run.smithi016.stdout:Xfiledir_test 2024-04-25T10:08:02.254 DEBUG:teuthology.orchestra.run.smithi016:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:08:02.343 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:08:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511627777, 2024-04-25T10:08:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: "ancestors": [ 2024-04-25T10:08:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:08:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: "dirino": 1099511627776, 2024-04-25T10:08:02.343 INFO:teuthology.orchestra.run.smithi016.stdout: "dname": "file", 2024-04-25T10:08:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 2 2024-04-25T10:08:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:08:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:08:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "dirino": 1, 2024-04-25T10:08:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "dname": "dir_test", 2024-04-25T10:08:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 4 2024-04-25T10:08:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:08:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-25T10:08:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "pool": 9, 2024-04-25T10:08:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: "old_pools": [] 2024-04-25T10:08:02.344 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:08:02.344 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:08:02.345 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-04-25T10:08:02.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.425+0000 7f014b9e2f40 1 Processor -- start 2024-04-25T10:08:02.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.425+0000 7f014b9e2f40 1 -- start start 2024-04-25T10:08:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.426+0000 7f014b9e2f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd54117b00 0x55dd541bc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.426+0000 7f014b9e2f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd54117ee0 0x55dd541127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.426+0000 7f014b9e2f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55dd541be9b0 0x55dd54112cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.426+0000 7f014b9e2f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd5408edf0 con 0x55dd54117b00 2024-04-25T10:08:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.426+0000 7f014b9e2f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd54092d50 con 0x55dd54117ee0 2024-04-25T10:08:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.426+0000 7f014b9e2f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd541bd540 con 0x55dd541be9b0 2024-04-25T10:08:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.426+0000 7f01494e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd54117b00 0x55dd541bc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.426+0000 7f01494e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd54117b00 0x55dd541bc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53352/0 (socket says 172.21.15.16:53352) 2024-04-25T10:08:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.426+0000 7f01494e3640 1 -- 172.21.15.16:0/4288388622 learned_addr learned my addr 172.21.15.16:0/4288388622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.426+0000 7f01494e3640 1 -- 172.21.15.16:0/4288388622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55dd541be9b0 msgr2=0x55dd54112cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.426+0000 7f01494e3640 1 --2- 172.21.15.16:0/4288388622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55dd541be9b0 0x55dd54112cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.426+0000 7f01494e3640 1 -- 172.21.15.16:0/4288388622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd54117ee0 msgr2=0x55dd541127b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.426+0000 7f01494e3640 1 --2- 172.21.15.16:0/4288388622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd54117ee0 0x55dd541127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.426+0000 7f01494e3640 1 -- 172.21.15.16:0/4288388622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd54111660 con 0x55dd54117b00 2024-04-25T10:08:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.426+0000 7f01494e3640 1 --2- 172.21.15.16:0/4288388622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd54117b00 0x55dd541bc1e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f013c001700 tx=0x7f013c001bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53cab7f4f4e4664 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.426+0000 7f01437fe640 1 -- 172.21.15.16:0/4288388622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c016070 con 0x55dd54117b00 2024-04-25T10:08:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.426+0000 7f01437fe640 1 -- 172.21.15.16:0/4288388622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c010790 con 0x55dd54117b00 2024-04-25T10:08:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.426+0000 7f01437fe640 1 -- 172.21.15.16:0/4288388622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c010b60 con 0x55dd54117b00 2024-04-25T10:08:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.427+0000 7f014b9e2f40 1 -- 172.21.15.16:0/4288388622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd54117b00 msgr2=0x55dd541bc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.427+0000 7f014b9e2f40 1 --2- 172.21.15.16:0/4288388622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd54117b00 0x55dd541bc1e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f013c001700 tx=0x7f013c001bc0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.427+0000 7f014b9e2f40 1 -- 172.21.15.16:0/4288388622 shutdown_connections 2024-04-25T10:08:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.427+0000 7f014b9e2f40 1 --2- 172.21.15.16:0/4288388622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55dd541be9b0 0x55dd54112cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.427+0000 7f014b9e2f40 1 --2- 172.21.15.16:0/4288388622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd54117ee0 0x55dd541127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.427+0000 7f014b9e2f40 1 --2- 172.21.15.16:0/4288388622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd54117b00 0x55dd541bc1e0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.427+0000 7f014b9e2f40 1 -- 172.21.15.16:0/4288388622 >> 172.21.15.16:0/4288388622 conn(0x55dd541123d0 msgr2=0x55dd540e88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.427+0000 7f014b9e2f40 1 -- 172.21.15.16:0/4288388622 shutdown_connections 2024-04-25T10:08:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.427+0000 7f014b9e2f40 1 -- 172.21.15.16:0/4288388622 wait complete. 2024-04-25T10:08:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.427+0000 7f014b9e2f40 1 Processor -- start 2024-04-25T10:08:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.427+0000 7f014b9e2f40 1 -- start start 2024-04-25T10:08:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.428+0000 7f014b9e2f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd54117b00 0x55dd541a7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.428+0000 7f014b9e2f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55dd54117ee0 0x55dd541b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.428+0000 7f014b9e2f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd541be9b0 0x55dd541b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.428+0000 7f014b9e2f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd54111900 con 0x55dd541be9b0 2024-04-25T10:08:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.428+0000 7f014b9e2f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd540e9a30 con 0x55dd54117b00 2024-04-25T10:08:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.428+0000 7f014b9e2f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd54113860 con 0x55dd54117ee0 2024-04-25T10:08:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.428+0000 7f0149ce4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd541be9b0 0x55dd541b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.428+0000 7f0149ce4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd541be9b0 0x55dd541b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53358/0 (socket says 172.21.15.16:53358) 2024-04-25T10:08:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.428+0000 7f0149ce4640 1 -- 172.21.15.16:0/1220722932 learned_addr learned my addr 172.21.15.16:0/1220722932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.428+0000 7f01494e3640 1 --2- 172.21.15.16:0/1220722932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd54117b00 0x55dd541a7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.428+0000 7f0148ce2640 1 --2- 172.21.15.16:0/1220722932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55dd54117ee0 0x55dd541b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.429+0000 7f0149ce4640 1 -- 172.21.15.16:0/1220722932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55dd54117ee0 msgr2=0x55dd541b89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.429+0000 7f0149ce4640 1 --2- 172.21.15.16:0/1220722932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55dd54117ee0 0x55dd541b89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.429+0000 7f0149ce4640 1 -- 172.21.15.16:0/1220722932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd54117b00 msgr2=0x55dd541a7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.429+0000 7f0149ce4640 1 --2- 172.21.15.16:0/1220722932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dd54117b00 0x55dd541a7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.429+0000 7f0149ce4640 1 -- 172.21.15.16:0/1220722932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd542549d0 con 0x55dd541be9b0 2024-04-25T10:08:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.429+0000 7f0149ce4640 1 --2- 172.21.15.16:0/1220722932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd541be9b0 0x55dd541b30a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0144057040 tx=0x7f014400a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.429+0000 7f01427fc640 1 -- 172.21.15.16:0/1220722932 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144063020 con 0x55dd541be9b0 2024-04-25T10:08:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.429+0000 7f01427fc640 1 -- 172.21.15.16:0/1220722932 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01440548b0 con 0x55dd541be9b0 2024-04-25T10:08:02.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.429+0000 7f014b9e2f40 1 -- 172.21.15.16:0/1220722932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dd54254cb0 con 0x55dd541be9b0 2024-04-25T10:08:02.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.429+0000 7f01427fc640 1 -- 172.21.15.16:0/1220722932 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144059040 con 0x55dd541be9b0 2024-04-25T10:08:02.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.429+0000 7f014b9e2f40 1 -- 172.21.15.16:0/1220722932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd54255270 con 0x55dd541be9b0 2024-04-25T10:08:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.430+0000 7f01427fc640 1 -- 172.21.15.16:0/1220722932 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0144066020 con 0x55dd541be9b0 2024-04-25T10:08:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.431+0000 7f01427fc640 1 --2- 172.21.15.16:0/1220722932 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f012c069ee0 0x7f012c06c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.431+0000 7f01427fc640 1 -- 172.21.15.16:0/1220722932 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01440cc670 con 0x55dd541be9b0 2024-04-25T10:08:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.431+0000 7f014b9e2f40 1 --2- 172.21.15.16:0/1220722932 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55dd54254540 0x55dd54255a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.431+0000 7f014b9e2f40 1 -- 172.21.15.16:0/1220722932 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59) v9 -- 0x55dd54113490 con 0x55dd54254540 2024-04-25T10:08:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.431+0000 7f01494e3640 1 --2- 172.21.15.16:0/1220722932 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f012c069ee0 0x7f012c06c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.431+0000 7f0148ce2640 1 --2- 172.21.15.16:0/1220722932 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55dd54254540 0x55dd54255a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.431+0000 7f0148ce2640 1 --2- 172.21.15.16:0/1220722932 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55dd54254540 0x55dd54255a70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.432+0000 7f01494e3640 1 --2- 172.21.15.16:0/1220722932 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f012c069ee0 0x7f012c06c3a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f013c001700 tx=0x7f013c010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.433+0000 7f0148ce2640 1 -- 172.21.15.16:0/1220722932 <== osd.2 v2:172.21.15.16:6800/2094356376 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v59'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f0134009040 con 0x55dd54254540 2024-04-25T10:08:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.433+0000 7f014b9e2f40 1 -- 172.21.15.16:0/1220722932 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55dd54254540 msgr2=0x55dd54255a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.433+0000 7f014b9e2f40 1 --2- 172.21.15.16:0/1220722932 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55dd54254540 0x55dd54255a70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.433+0000 7f014b9e2f40 1 -- 172.21.15.16:0/1220722932 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f012c069ee0 msgr2=0x7f012c06c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.433+0000 7f014b9e2f40 1 --2- 172.21.15.16:0/1220722932 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f012c069ee0 0x7f012c06c3a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f013c001700 tx=0x7f013c010000 comp rx=0 tx=0).stop 2024-04-25T10:08:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.433+0000 7f014b9e2f40 1 -- 172.21.15.16:0/1220722932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd541be9b0 msgr2=0x55dd541b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.433+0000 7f014b9e2f40 1 --2- 172.21.15.16:0/1220722932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd541be9b0 0x55dd541b30a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0144057040 tx=0x7f014400a420 comp rx=0 tx=0).stop 2024-04-25T10:08:02.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.433+0000 7f0149ce4640 1 -- 172.21.15.16:0/1220722932 reap_dead start 2024-04-25T10:08:02.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.433+0000 7f014b9e2f40 1 -- 172.21.15.16:0/1220722932 shutdown_connections 2024-04-25T10:08:02.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.433+0000 7f014b9e2f40 1 -- 172.21.15.16:0/1220722932 >> 172.21.15.16:0/1220722932 conn(0x55dd541123d0 msgr2=0x55dd541bcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.433+0000 7f014b9e2f40 1 -- 172.21.15.16:0/1220722932 shutdown_connections 2024-04-25T10:08:02.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.433+0000 7f014b9e2f40 1 -- 172.21.15.16:0/1220722932 wait complete. 2024-04-25T10:08:02.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:02.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-04-25T10:08:02.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.542+0000 7f28aeadb640 1 Processor -- start 2024-04-25T10:08:02.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.542+0000 7f28aeadb640 1 -- start start 2024-04-25T10:08:02.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.543+0000 7f28aeadb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f28a80d3580 0x7f28a815ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.543+0000 7f28aeadb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a80b9980 con 0x7f28a80b5c80 2024-04-25T10:08:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.543+0000 7f28aeadb640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a80ce140 con 0x7f28a815dbe0 2024-04-25T10:08:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.543+0000 7f28aeadb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a815f210 con 0x7f28a80d3580 2024-04-25T10:08:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.543+0000 7f28ad078640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f28a80d3580 0x7f28a815ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.543+0000 7f28a7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f28a80b5c80 0x7f28a8161800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54504/0 (socket says 172.21.15.16:54504) 2024-04-25T10:08:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.543+0000 7f28a7fff640 1 -- 172.21.15.16:0/3169914628 learned_addr learned my addr 172.21.15.16:0/3169914628 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:08:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.544+0000 7f28ad078640 1 -- 172.21.15.16:0/3169914628 >> v1:172.21.15.186:6789/0 conn(0x7f28a815dbe0 legacy=0x7f28a80b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.544+0000 7f28ad078640 1 -- 172.21.15.16:0/3169914628 >> v1:172.21.15.16:6789/0 conn(0x7f28a80b5c80 legacy=0x7f28a8161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.544+0000 7f28a77fe640 1 -- 172.21.15.16:0/3169914628 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992300755 0 0) 0x7f28a80b9980 con 0x7f28a80b5c80 2024-04-25T10:08:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.544+0000 7f28ad078640 1 -- 172.21.15.16:0/3169914628 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a80ce420 con 0x7f28a80d3580 2024-04-25T10:08:02.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.545+0000 7f28ad078640 1 --2- 172.21.15.16:0/3169914628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a80d3580 0x7f28a815ecd0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f28900097a0 tx=0x7f289002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f6b3515716587e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.545+0000 7f28a77fe640 1 -- 172.21.15.16:0/3169914628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890037070 con 0x7f28a80d3580 2024-04-25T10:08:02.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.545+0000 7f28a77fe640 1 -- 172.21.15.16:0/3169914628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2890032440 con 0x7f28a80d3580 2024-04-25T10:08:02.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.545+0000 7f28a77fe640 1 -- 172.21.15.16:0/3169914628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890032810 con 0x7f28a80d3580 2024-04-25T10:08:02.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.545+0000 7f28aeadb640 1 -- 172.21.15.16:0/3169914628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a80d3580 msgr2=0x7f28a815ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.545+0000 7f28aeadb640 1 --2- 172.21.15.16:0/3169914628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a80d3580 0x7f28a815ecd0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f28900097a0 tx=0x7f289002a230 comp rx=0 tx=0).stop 2024-04-25T10:08:02.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.545+0000 7f28aeadb640 1 -- 172.21.15.16:0/3169914628 shutdown_connections 2024-04-25T10:08:02.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.545+0000 7f28aeadb640 1 --2- 172.21.15.16:0/3169914628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a80d3580 0x7f28a815ecd0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.545+0000 7f28aeadb640 1 -- 172.21.15.16:0/3169914628 >> 172.21.15.16:0/3169914628 conn(0x7f28a8072900 msgr2=0x7f28a80d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.545+0000 7f28aeadb640 1 -- 172.21.15.16:0/3169914628 shutdown_connections 2024-04-25T10:08:02.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.545+0000 7f28aeadb640 1 -- 172.21.15.16:0/3169914628 wait complete. 2024-04-25T10:08:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.546+0000 7f28aeadb640 1 Processor -- start 2024-04-25T10:08:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.546+0000 7f28aeadb640 1 -- start start 2024-04-25T10:08:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.547+0000 7f28aeadb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28a80b5c80 0x7f28a814edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.547+0000 7f28aeadb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28a80d3580 0x7f28a80c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.547+0000 7f28aeadb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a815dbe0 0x7f28a80c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.547+0000 7f28aeadb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a815dfc0 con 0x7f28a815dbe0 2024-04-25T10:08:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.547+0000 7f28aeadb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a80d3060 con 0x7f28a80b5c80 2024-04-25T10:08:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.547+0000 7f28aeadb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a815e140 con 0x7f28a80d3580 2024-04-25T10:08:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.547+0000 7f28ad078640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a815dbe0 0x7f28a80c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.547+0000 7f28ad078640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a815dbe0 0x7f28a80c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53374/0 (socket says 172.21.15.16:53374) 2024-04-25T10:08:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.547+0000 7f28ad078640 1 -- 172.21.15.16:0/4285047338 learned_addr learned my addr 172.21.15.16:0/4285047338 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.548+0000 7f28ac877640 1 --2- 172.21.15.16:0/4285047338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28a80b5c80 0x7f28a814edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.548+0000 7f28a7fff640 1 --2- 172.21.15.16:0/4285047338 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28a80d3580 0x7f28a80c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.548+0000 7f28ad078640 1 -- 172.21.15.16:0/4285047338 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28a80d3580 msgr2=0x7f28a80c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.548+0000 7f28ad078640 1 --2- 172.21.15.16:0/4285047338 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28a80d3580 0x7f28a80c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.548+0000 7f28ad078640 1 -- 172.21.15.16:0/4285047338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28a80b5c80 msgr2=0x7f28a814edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.548+0000 7f28ad078640 1 --2- 172.21.15.16:0/4285047338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28a80b5c80 0x7f28a814edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.548+0000 7f28ad078640 1 -- 172.21.15.16:0/4285047338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a80c83e0 con 0x7f28a815dbe0 2024-04-25T10:08:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.548+0000 7f28a7fff640 1 --2- 172.21.15.16:0/4285047338 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28a80d3580 0x7f28a80c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.548+0000 7f28ad078640 1 --2- 172.21.15.16:0/4285047338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a815dbe0 0x7f28a80c9350 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2890035040 tx=0x7f28900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.548+0000 7f28a57fa640 1 -- 172.21.15.16:0/4285047338 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890037050 con 0x7f28a815dbe0 2024-04-25T10:08:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.548+0000 7f28a57fa640 1 -- 172.21.15.16:0/4285047338 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2890038040 con 0x7f28a815dbe0 2024-04-25T10:08:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.548+0000 7f28a57fa640 1 -- 172.21.15.16:0/4285047338 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890004490 con 0x7f28a815dbe0 2024-04-25T10:08:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.548+0000 7f28aeadb640 1 -- 172.21.15.16:0/4285047338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a80c86c0 con 0x7f28a815dbe0 2024-04-25T10:08:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.548+0000 7f28aeadb640 1 -- 172.21.15.16:0/4285047338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a80cad40 con 0x7f28a815dbe0 2024-04-25T10:08:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.550+0000 7f28a57fa640 1 -- 172.21.15.16:0/4285047338 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2890030980 con 0x7f28a815dbe0 2024-04-25T10:08:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.550+0000 7f28a57fa640 1 --2- 172.21.15.16:0/4285047338 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f288806b4d0 0x7f288806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.551+0000 7f28a57fa640 1 -- 172.21.15.16:0/4285047338 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28900aac90 con 0x7f28a815dbe0 2024-04-25T10:08:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.551+0000 7f28ac877640 1 --2- 172.21.15.16:0/4285047338 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f288806b4d0 0x7f288806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.551+0000 7f28b17de740 1 Processor -- start 2024-04-25T10:08:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.551+0000 7f28b17de740 1 -- start start 2024-04-25T10:08:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.551+0000 7f28ac877640 1 --2- 172.21.15.16:0/4285047338 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f288806b4d0 0x7f288806d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f28a0000f60 tx=0x7f28a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.552+0000 7f28b17de740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2080 0x55715cbf4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.552+0000 7f28b17de740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbf2460 0x55715cbf4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.552+0000 7f28b17de740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbc4ce0 0x55715cbc84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.552+0000 7f28b17de740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715cb2daf0 con 0x55715cbf2080 2024-04-25T10:08:02.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.552+0000 7f28b17de740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715cad6de0 con 0x55715cbf2460 2024-04-25T10:08:02.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.552+0000 7f28b17de740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55715cb29b90 con 0x55715cbc4ce0 2024-04-25T10:08:02.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.552+0000 7f28ac877640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2080 0x55715cbf4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.552+0000 7f28ac877640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2080 0x55715cbf4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53384/0 (socket says 172.21.15.16:53384) 2024-04-25T10:08:02.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.552+0000 7f28ac877640 1 -- 172.21.15.16:0/1304278095 learned_addr learned my addr 172.21.15.16:0/1304278095 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:02.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.552+0000 7f28ad078640 1 --2- 172.21.15.16:0/1304278095 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbc4ce0 0x55715cbc84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.552+0000 7f28a7fff640 1 --2- 172.21.15.16:0/1304278095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbf2460 0x55715cbf4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.552+0000 7f28ac877640 1 -- 172.21.15.16:0/1304278095 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbc4ce0 msgr2=0x55715cbc84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.552+0000 7f28ac877640 1 --2- 172.21.15.16:0/1304278095 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbc4ce0 0x55715cbc84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.552+0000 7f28ac877640 1 -- 172.21.15.16:0/1304278095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbf2460 msgr2=0x55715cbf4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.553+0000 7f28ac877640 1 --2- 172.21.15.16:0/1304278095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbf2460 0x55715cbf4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.553+0000 7f28ac877640 1 -- 172.21.15.16:0/1304278095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55715c90e7b0 con 0x55715cbf2080 2024-04-25T10:08:02.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.553+0000 7f28a7fff640 1 --2- 172.21.15.16:0/1304278095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbf2460 0x55715cbf4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:02.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.553+0000 7f28ac877640 1 --2- 172.21.15.16:0/1304278095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2080 0x55715cbf4890 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f28a000f9e0 tx=0x7f28a0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85aa7f0be24df079 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.553+0000 7f289e7fc640 1 -- 172.21.15.16:0/1304278095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a0025070 con 0x55715cbf2080 2024-04-25T10:08:02.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.553+0000 7f289e7fc640 1 -- 172.21.15.16:0/1304278095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a001b050 con 0x55715cbf2080 2024-04-25T10:08:02.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.553+0000 7f289e7fc640 1 -- 172.21.15.16:0/1304278095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a0014de0 con 0x55715cbf2080 2024-04-25T10:08:02.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.553+0000 7f28b17de740 1 -- 172.21.15.16:0/1304278095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2080 msgr2=0x55715cbf4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.553+0000 7f28b17de740 1 --2- 172.21.15.16:0/1304278095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2080 0x55715cbf4890 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f28a000f9e0 tx=0x7f28a0012000 comp rx=0 tx=0).stop 2024-04-25T10:08:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.553+0000 7f28b17de740 1 -- 172.21.15.16:0/1304278095 shutdown_connections 2024-04-25T10:08:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.553+0000 7f28b17de740 1 --2- 172.21.15.16:0/1304278095 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbc4ce0 0x55715cbc84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.553+0000 7f28b17de740 1 --2- 172.21.15.16:0/1304278095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbf2460 0x55715cbf4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.553+0000 7f28b17de740 1 --2- 172.21.15.16:0/1304278095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2080 0x55715cbf4890 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.553+0000 7f28b17de740 1 -- 172.21.15.16:0/1304278095 >> 172.21.15.16:0/1304278095 conn(0x55715cbc6f50 msgr2=0x55715cc0f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.554+0000 7f28b17de740 1 -- 172.21.15.16:0/1304278095 shutdown_connections 2024-04-25T10:08:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.554+0000 7f28b17de740 1 -- 172.21.15.16:0/1304278095 wait complete. 2024-04-25T10:08:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.554+0000 7f28b17de740 1 -- start start 2024-04-25T10:08:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.554+0000 7f28b17de740 1 Processor -- start 2024-04-25T10:08:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.555+0000 7f28b17de740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbf2080 0x55715cbf7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.555+0000 7f28b17de740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbf2460 0x55715cbf81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.555+0000 7f28b17de740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbe8ef0 0x55715cbf86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.555+0000 7f28b17de740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715cbca100 con 0x55715cbe8ef0 2024-04-25T10:08:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.555+0000 7f28b17de740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715cacc3e0 con 0x55715cbf2460 2024-04-25T10:08:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.555+0000 7f28b17de740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55715cbca3e0 con 0x55715cbf2080 2024-04-25T10:08:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.555+0000 7f28b17de740 10 client.0 _ll_register_callbacks cb 0x55715cae9d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.555+0000 7f28ad078640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbe8ef0 0x55715cbf86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.555+0000 7f28ad078640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbe8ef0 0x55715cbf86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53390/0 (socket says 172.21.15.16:53390) 2024-04-25T10:08:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.555+0000 7f28ad078640 1 -- 172.21.15.16:0/3587945301 learned_addr learned my addr 172.21.15.16:0/3587945301 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.555+0000 7f28ac877640 1 --2- 172.21.15.16:0/3587945301 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbf2080 0x55715cbf7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.555+0000 7f28a7fff640 1 --2- 172.21.15.16:0/3587945301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbf2460 0x55715cbf81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.555+0000 7f28ad078640 1 -- 172.21.15.16:0/3587945301 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbf2080 msgr2=0x55715cbf7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.555+0000 7f28ad078640 1 --2- 172.21.15.16:0/3587945301 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbf2080 0x55715cbf7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.555+0000 7f28ad078640 1 -- 172.21.15.16:0/3587945301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbf2460 msgr2=0x55715cbf81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.555+0000 7f28ad078640 1 --2- 172.21.15.16:0/3587945301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbf2460 0x55715cbf81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.555+0000 7f28ad078640 1 -- 172.21.15.16:0/3587945301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55715c96ef40 con 0x55715cbe8ef0 2024-04-25T10:08:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.556+0000 7f28ad078640 1 --2- 172.21.15.16:0/3587945301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbe8ef0 0x55715cbf86f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f28900a91f0 tx=0x7f2890009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148456fb26c3ec40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.556+0000 7f28877fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:08:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.556+0000 7f28877fe640 1 -- 172.21.15.16:0/3587945301 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890079cc0 con 0x55715cbe8ef0 2024-04-25T10:08:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.556+0000 7f28877fe640 1 -- 172.21.15.16:0/3587945301 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2890081840 con 0x55715cbe8ef0 2024-04-25T10:08:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.556+0000 7f28b17de740 1 -- 172.21.15.16:0/3587945301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28a001b050 con 0x55715cbe8ef0 2024-04-25T10:08:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.556+0000 7f28877fe640 1 -- 172.21.15.16:0/3587945301 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28900795c0 con 0x55715cbe8ef0 2024-04-25T10:08:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.556+0000 7f28b17de740 10 client.6285 fetch_fsmap learned FSMap version 27 2024-04-25T10:08:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.556+0000 7f28b17de740 1 -- 172.21.15.16:0/3587945301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55715c8020a0 con 0x55715cbe8ef0 2024-04-25T10:08:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.556+0000 7f28877fe640 1 -- 172.21.15.16:0/3587945301 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a001b050 con 0x55715cbe8ef0 2024-04-25T10:08:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.556+0000 7f28877fe640 1 -- 172.21.15.16:0/3587945301 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f28900a2bb0 con 0x55715cbe8ef0 2024-04-25T10:08:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.556+0000 7f28b17de740 10 client.6285 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:08:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.556+0000 7f28b17de740 10 client.6285 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.556+0000 7f28b17de740 20 client.6285 populate_metadata read hostname 'smithi016' 2024-04-25T10:08:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.556+0000 7f28b17de740 1 --2- 172.21.15.16:0/3587945301 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55715cc0b040 0x55715cc0b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.556+0000 7f28b17de740 4 client.6285 mds_command: new command op to 35141 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.556+0000 7f28b17de740 1 -- 172.21.15.16:0/3587945301 --> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55715cb3b030 con 0x55715cc0b040 2024-04-25T10:08:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.556+0000 7f28ac877640 1 --2- 172.21.15.16:0/3587945301 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55715cc0b040 0x55715cc0b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.557+0000 7f28877fe640 10 client.6285 ms_handle_connect on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.557+0000 7f28ac877640 1 --2- 172.21.15.16:0/3587945301 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55715cc0b040 0x55715cc0b450 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59047df607c2ad04 server_cookie=138b6d367f30caf4 in_seq=0 out_seq=0 2024-04-25T10:08:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.557+0000 7f28877fe640 1 -- 172.21.15.16:0/3587945301 <== mds.0 v2:172.21.15.16:6832/3311584521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55715cb3b030 con 0x55715cc0b040 2024-04-25T10:08:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.557+0000 7f28877fe640 10 client.6285 handle_command_reply: tid=0 2024-04-25T10:08:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.557+0000 7f28b17de740 1 client.6285 shutdown 2024-04-25T10:08:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.557+0000 7f28b17de740 1 -- 172.21.15.16:0/3587945301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbe8ef0 msgr2=0x55715cbf86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.557+0000 7f28b17de740 1 --2- 172.21.15.16:0/3587945301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbe8ef0 0x55715cbf86f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f28900a91f0 tx=0x7f2890009520 comp rx=0 tx=0).stop 2024-04-25T10:08:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.558+0000 7f28b17de740 1 -- 172.21.15.16:0/3587945301 shutdown_connections 2024-04-25T10:08:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.558+0000 7f28b17de740 1 --2- 172.21.15.16:0/3587945301 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55715cc0b040 0x55715cc0b450 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.558+0000 7f28b17de740 1 --2- 172.21.15.16:0/3587945301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbe8ef0 0x55715cbf86f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.558+0000 7f28b17de740 1 --2- 172.21.15.16:0/3587945301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbf2460 0x55715cbf81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.558+0000 7f28877fe640 0 client.6285 ms_handle_reset on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.558+0000 7f28b17de740 1 --2- 172.21.15.16:0/3587945301 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbf2080 0x55715cbf7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.558+0000 7f28b17de740 1 -- 172.21.15.16:0/3587945301 >> 172.21.15.16:0/3587945301 conn(0x55715cbc4ce0 msgr2=0x55715cbc3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.558+0000 7f28b17de740 1 -- 172.21.15.16:0/3587945301 shutdown_connections 2024-04-25T10:08:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.558+0000 7f28b17de740 1 -- 172.21.15.16:0/3587945301 wait complete. 2024-04-25T10:08:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.558+0000 7f28b17de740 20 client.6285 trim_cache size 0 max 16384 2024-04-25T10:08:02.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.573+0000 7f28b17de740 1 Processor -- start 2024-04-25T10:08:02.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.573+0000 7f28b17de740 1 -- start start 2024-04-25T10:08:02.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f28b17de740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbe8ef0 0x55715cc05f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f28b17de740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbf2080 0x55715cbc9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f28b17de740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2460 0x55715cbc3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f28b17de740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a81ffd60 con 0x55715cbf2460 2024-04-25T10:08:02.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f28b17de740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a81fefb0 con 0x55715cbf2080 2024-04-25T10:08:02.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f28b17de740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a80cbb70 con 0x55715cbe8ef0 2024-04-25T10:08:02.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f28ad078640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2460 0x55715cbc3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f28ad078640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2460 0x55715cbc3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53400/0 (socket says 172.21.15.16:53400) 2024-04-25T10:08:02.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f28ad078640 1 -- 172.21.15.16:0/242658371 learned_addr learned my addr 172.21.15.16:0/242658371 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:02.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f28ad078640 1 -- 172.21.15.16:0/242658371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbe8ef0 msgr2=0x55715cc05f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f28ad078640 1 --2- 172.21.15.16:0/242658371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbe8ef0 0x55715cc05f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f28ad078640 1 -- 172.21.15.16:0/242658371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbf2080 msgr2=0x55715cbc9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f28a7fff640 1 --2- 172.21.15.16:0/242658371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbf2080 0x55715cbc9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f28ad078640 1 --2- 172.21.15.16:0/242658371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbf2080 0x55715cbc9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f28ad078640 1 -- 172.21.15.16:0/242658371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55715cbe96d0 con 0x55715cbf2460 2024-04-25T10:08:02.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f28ad078640 1 --2- 172.21.15.16:0/242658371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2460 0x55715cbc3b00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2890077870 tx=0x7f28900aa000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4445d49b7993d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f289dffb640 1 -- 172.21.15.16:0/242658371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890079100 con 0x55715cbf2460 2024-04-25T10:08:02.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f289dffb640 1 -- 172.21.15.16:0/242658371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2890079cc0 con 0x55715cbf2460 2024-04-25T10:08:02.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.574+0000 7f289dffb640 1 -- 172.21.15.16:0/242658371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890081cd0 con 0x55715cbf2460 2024-04-25T10:08:02.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.575+0000 7f28b17de740 1 -- 172.21.15.16:0/242658371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2460 msgr2=0x55715cbc3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.575+0000 7f28b17de740 1 --2- 172.21.15.16:0/242658371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2460 0x55715cbc3b00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2890077870 tx=0x7f28900aa000 comp rx=0 tx=0).stop 2024-04-25T10:08:02.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.575+0000 7f28b17de740 1 -- 172.21.15.16:0/242658371 shutdown_connections 2024-04-25T10:08:02.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.575+0000 7f28b17de740 1 --2- 172.21.15.16:0/242658371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2460 0x55715cbc3b00 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.575+0000 7f28b17de740 1 --2- 172.21.15.16:0/242658371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbf2080 0x55715cbc9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.575+0000 7f28b17de740 1 --2- 172.21.15.16:0/242658371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbe8ef0 0x55715cc05f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.575+0000 7f28b17de740 1 -- 172.21.15.16:0/242658371 >> 172.21.15.16:0/242658371 conn(0x55715cbc4ce0 msgr2=0x55715cc0c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.575+0000 7f28b17de740 1 -- 172.21.15.16:0/242658371 shutdown_connections 2024-04-25T10:08:02.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.575+0000 7f28b17de740 1 -- 172.21.15.16:0/242658371 wait complete. 2024-04-25T10:08:02.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.575+0000 7f28b17de740 1 -- start start 2024-04-25T10:08:02.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.576+0000 7f28b17de740 1 Processor -- start 2024-04-25T10:08:02.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.576+0000 7f28b17de740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbe8ef0 0x55715cc0f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.576+0000 7f28b17de740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbf2080 0x55715cc0fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.576+0000 7f28b17de740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2460 0x55715cc10c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.576+0000 7f28b17de740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715cc332e0 con 0x55715cbf2460 2024-04-25T10:08:02.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.576+0000 7f28b17de740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a80cc170 con 0x55715cbe8ef0 2024-04-25T10:08:02.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.576+0000 7f28b17de740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55715cc0e1d0 con 0x55715cbf2080 2024-04-25T10:08:02.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.576+0000 7f28b17de740 10 client.0 _ll_register_callbacks cb 0x7f2890006c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:02.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28ad078640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2460 0x55715cc10c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28ad078640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2460 0x55715cc10c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53416/0 (socket says 172.21.15.16:53416) 2024-04-25T10:08:02.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28ad078640 1 -- 172.21.15.16:0/148487711 learned_addr learned my addr 172.21.15.16:0/148487711 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:02.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28ac877640 1 --2- 172.21.15.16:0/148487711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbe8ef0 0x55715cc0f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28a7fff640 1 --2- 172.21.15.16:0/148487711 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbf2080 0x55715cc0fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28ad078640 1 -- 172.21.15.16:0/148487711 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbf2080 msgr2=0x55715cc0fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28ad078640 1 --2- 172.21.15.16:0/148487711 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbf2080 0x55715cc0fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28ad078640 1 -- 172.21.15.16:0/148487711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbe8ef0 msgr2=0x55715cc0f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28ad078640 1 --2- 172.21.15.16:0/148487711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbe8ef0 0x55715cc0f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28ad078640 1 -- 172.21.15.16:0/148487711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55715cc17f60 con 0x55715cbf2460 2024-04-25T10:08:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28ad078640 1 --2- 172.21.15.16:0/148487711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2460 0x55715cc10c00 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f28900792c0 tx=0x7f289007c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740d8623778eb00c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28877fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:08:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28877fe640 1 -- 172.21.15.16:0/148487711 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890079100 con 0x55715cbf2460 2024-04-25T10:08:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28877fe640 1 -- 172.21.15.16:0/148487711 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f289008d020 con 0x55715cbf2460 2024-04-25T10:08:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28b17de740 1 -- 172.21.15.16:0/148487711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2890079cc0 con 0x55715cbf2460 2024-04-25T10:08:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28877fe640 1 -- 172.21.15.16:0/148487711 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f289007d710 con 0x55715cbf2460 2024-04-25T10:08:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28b17de740 10 client.6291 fetch_fsmap learned FSMap version 27 2024-04-25T10:08:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28b17de740 1 -- 172.21.15.16:0/148487711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55715cc17f60 con 0x55715cbf2460 2024-04-25T10:08:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.577+0000 7f28877fe640 1 -- 172.21.15.16:0/148487711 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2890079cc0 con 0x55715cbf2460 2024-04-25T10:08:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.578+0000 7f28877fe640 1 -- 172.21.15.16:0/148487711 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2890078a10 con 0x55715cbf2460 2024-04-25T10:08:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.578+0000 7f28b17de740 10 client.6291 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:08:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.578+0000 7f28b17de740 10 client.6291 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.578+0000 7f28b17de740 20 client.6291 populate_metadata read hostname 'smithi016' 2024-04-25T10:08:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.578+0000 7f28b17de740 1 --2- 172.21.15.16:0/148487711 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55715cc05220 0x55715cc11140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.578+0000 7f28b17de740 4 client.6291 mds_command: new command op to 35141 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-04-25T10:08:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.578+0000 7f28b17de740 1 -- 172.21.15.16:0/148487711 --> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x55715cb35380 con 0x55715cc05220 2024-04-25T10:08:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.578+0000 7f28ac877640 1 --2- 172.21.15.16:0/148487711 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55715cc05220 0x55715cc11140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.578+0000 7f28877fe640 10 client.6291 ms_handle_connect on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.578+0000 7f28ac877640 1 --2- 172.21.15.16:0/148487711 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55715cc05220 0x55715cc11140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8fc824d4fae6fe8 server_cookie=41370ac6046db8ba in_seq=0 out_seq=0 2024-04-25T10:08:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.579+0000 7f28877fe640 1 -- 172.21.15.16:0/148487711 <== mds.0 v2:172.21.15.16:6832/3311584521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55715cb35380 con 0x55715cc05220 2024-04-25T10:08:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.579+0000 7f28877fe640 10 client.6291 handle_command_reply: tid=0 2024-04-25T10:08:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.579+0000 7f28b17de740 1 client.6291 shutdown 2024-04-25T10:08:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.579+0000 7f28b17de740 1 -- 172.21.15.16:0/148487711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2460 msgr2=0x55715cc10c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.579+0000 7f28b17de740 1 --2- 172.21.15.16:0/148487711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2460 0x55715cc10c00 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f28900792c0 tx=0x7f289007c070 comp rx=0 tx=0).stop 2024-04-25T10:08:02.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.579+0000 7f28b17de740 1 -- 172.21.15.16:0/148487711 shutdown_connections 2024-04-25T10:08:02.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.579+0000 7f28b17de740 1 --2- 172.21.15.16:0/148487711 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55715cc05220 0x55715cc11140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.579+0000 7f28b17de740 1 --2- 172.21.15.16:0/148487711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55715cbf2460 0x55715cc10c00 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.579+0000 7f28877fe640 0 client.6291 ms_handle_reset on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:02.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.579+0000 7f28b17de740 1 --2- 172.21.15.16:0/148487711 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55715cbf2080 0x55715cc0fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.579+0000 7f28b17de740 1 --2- 172.21.15.16:0/148487711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55715cbe8ef0 0x55715cc0f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.579+0000 7f28b17de740 1 -- 172.21.15.16:0/148487711 >> 172.21.15.16:0/148487711 conn(0x55715cbc4ce0 msgr2=0x55715cc0c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.590 INFO:tasks.ceph.mds.a.smithi016.stderr:2024-04-25T10:08:02.579+0000 7fce9a138640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={35240=0-4194304@1} caps={35240=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=35240 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55cf29494100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:08:02.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.579+0000 7f28b17de740 1 -- 172.21.15.16:0/148487711 shutdown_connections 2024-04-25T10:08:02.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.579+0000 7f28b17de740 1 -- 172.21.15.16:0/148487711 wait complete. 2024-04-25T10:08:02.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.579+0000 7f28b17de740 20 client.6291 trim_cache size 0 max 16384 2024-04-25T10:08:02.591 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:08:02.591 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-04-25T10:08:02.591 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "5fbaf134-e0e2-4f7c-861d-bb7f7ccb0973", 2024-04-25T10:08:02.591 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-04-25T10:08:02.592 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:08:02.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.580+0000 7f28aeadb640 1 -- 172.21.15.16:0/4285047338 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f288806b4d0 msgr2=0x7f288806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.580+0000 7f28aeadb640 1 --2- 172.21.15.16:0/4285047338 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f288806b4d0 0x7f288806d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f28a0000f60 tx=0x7f28a0009040 comp rx=0 tx=0).stop 2024-04-25T10:08:02.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.580+0000 7f28aeadb640 1 -- 172.21.15.16:0/4285047338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a815dbe0 msgr2=0x7f28a80c9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.580+0000 7f28aeadb640 1 --2- 172.21.15.16:0/4285047338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a815dbe0 0x7f28a80c9350 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2890035040 tx=0x7f28900046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.580+0000 7f28aeadb640 1 -- 172.21.15.16:0/4285047338 shutdown_connections 2024-04-25T10:08:02.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.580+0000 7f28aeadb640 1 --2- 172.21.15.16:0/4285047338 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f288806b4d0 0x7f288806d990 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.580+0000 7f28aeadb640 1 --2- 172.21.15.16:0/4285047338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28a815dbe0 0x7f28a80c9350 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.580+0000 7f28aeadb640 1 --2- 172.21.15.16:0/4285047338 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28a80d3580 0x7f28a80c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.580+0000 7f28aeadb640 1 --2- 172.21.15.16:0/4285047338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28a80b5c80 0x7f28a814edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.580+0000 7f28aeadb640 1 -- 172.21.15.16:0/4285047338 >> 172.21.15.16:0/4285047338 conn(0x7f28a8072900 msgr2=0x7f28a815fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.580+0000 7f28aeadb640 1 -- 172.21.15.16:0/4285047338 shutdown_connections 2024-04-25T10:08:02.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.580+0000 7f28aeadb640 1 -- 172.21.15.16:0/4285047338 wait complete. 2024-04-25T10:08:02.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:02.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:08:02.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.693+0000 7fbdb73b7640 1 Processor -- start 2024-04-25T10:08:02.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.694+0000 7fbdb73b7640 1 -- start start 2024-04-25T10:08:02.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.694+0000 7fbdb73b7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdb015dca0 0x7fbdb00ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.694+0000 7fbdb73b7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbdb00d1130 0x7fbdb00b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.694+0000 7fbdb73b7640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb00d4090 con 0x7fbdb00d3b90 2024-04-25T10:08:02.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.694+0000 7fbdb73b7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb00b6b20 con 0x7fbdb00d1130 2024-04-25T10:08:02.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.694+0000 7fbdb73b7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb00d4370 con 0x7fbdb015dca0 2024-04-25T10:08:02.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.695+0000 7fbdb5954640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbdb00d1130 0x7fbdb00b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.695+0000 7fbdb5954640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbdb00d1130 0x7fbdb00b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53422/0 (socket says 172.21.15.16:53422) 2024-04-25T10:08:02.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.695+0000 7fbdb5954640 1 -- 172.21.15.16:0/3281308675 learned_addr learned my addr 172.21.15.16:0/3281308675 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:02.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.695+0000 7fbdb5153640 1 --2- 172.21.15.16:0/3281308675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdb015dca0 0x7fbdb00ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.695+0000 7fbdb5954640 1 -- 172.21.15.16:0/3281308675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdb015dca0 msgr2=0x7fbdb00ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.695+0000 7fbdb5954640 1 --2- 172.21.15.16:0/3281308675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdb015dca0 0x7fbdb00ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.695+0000 7fbdb5954640 1 -- 172.21.15.16:0/3281308675 >> v1:172.21.15.186:6789/0 conn(0x7fbdb00d3b90 legacy=0x7fbdb00d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.695+0000 7fbdb5954640 1 -- 172.21.15.16:0/3281308675 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb00d4650 con 0x7fbdb00d1130 2024-04-25T10:08:02.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.696+0000 7fbdb5954640 1 --2- 172.21.15.16:0/3281308675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdb00d1130 0x7fbdb00b2500 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009e50 tx=0x7fbda002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27fea1265c11b34e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.696+0000 7fbdaffff640 1 -- 172.21.15.16:0/3281308675 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda0036070 con 0x7fbdb00d1130 2024-04-25T10:08:02.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.696+0000 7fbdaffff640 1 -- 172.21.15.16:0/3281308675 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda0031600 con 0x7fbdb00d1130 2024-04-25T10:08:02.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.696+0000 7fbdaffff640 1 -- 172.21.15.16:0/3281308675 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda00319d0 con 0x7fbdb00d1130 2024-04-25T10:08:02.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.696+0000 7fbdb73b7640 1 -- 172.21.15.16:0/3281308675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdb00d1130 msgr2=0x7fbdb00b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.696+0000 7fbdb73b7640 1 --2- 172.21.15.16:0/3281308675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdb00d1130 0x7fbdb00b2500 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009e50 tx=0x7fbda002a710 comp rx=0 tx=0).stop 2024-04-25T10:08:02.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.696+0000 7fbdb73b7640 1 -- 172.21.15.16:0/3281308675 shutdown_connections 2024-04-25T10:08:02.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.696+0000 7fbdb73b7640 1 --2- 172.21.15.16:0/3281308675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdb00d1130 0x7fbdb00b2500 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.696+0000 7fbdb73b7640 1 --2- 172.21.15.16:0/3281308675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdb015dca0 0x7fbdb00ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.696+0000 7fbdb73b7640 1 -- 172.21.15.16:0/3281308675 >> 172.21.15.16:0/3281308675 conn(0x7fbdb00726c0 msgr2=0x7fbdb0072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.696+0000 7fbdb73b7640 1 -- 172.21.15.16:0/3281308675 shutdown_connections 2024-04-25T10:08:02.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.696+0000 7fbdb73b7640 1 -- 172.21.15.16:0/3281308675 wait complete. 2024-04-25T10:08:02.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.698+0000 7fbdb73b7640 1 Processor -- start 2024-04-25T10:08:02.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.698+0000 7fbdb73b7640 1 -- start start 2024-04-25T10:08:02.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.698+0000 7fbdb73b7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdb00d1130 0x7fbdb01570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.698+0000 7fbdb73b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdb00d3b90 0x7fbdb00ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.698+0000 7fbdb73b7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb015dca0 0x7fbdb0159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.698+0000 7fbdb73b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb015e200 con 0x7fbdb00d3b90 2024-04-25T10:08:02.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.698+0000 7fbdb73b7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb015e080 con 0x7fbdb015dca0 2024-04-25T10:08:02.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.698+0000 7fbdb73b7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb00cee80 con 0x7fbdb00d1130 2024-04-25T10:08:02.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.698+0000 7fbdb4952640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdb00d3b90 0x7fbdb00ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.698+0000 7fbdb4952640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdb00d3b90 0x7fbdb00ccaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40052/0 (socket says 172.21.15.16:40052) 2024-04-25T10:08:02.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.698+0000 7fbdb4952640 1 -- 172.21.15.16:0/3280214074 learned_addr learned my addr 172.21.15.16:0/3280214074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:02.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.699+0000 7fbdb5153640 1 --2- 172.21.15.16:0/3280214074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdb00d1130 0x7fbdb01570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.699+0000 7fbdb5954640 1 --2- 172.21.15.16:0/3280214074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb015dca0 0x7fbdb0159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.699+0000 7fbdb4952640 1 -- 172.21.15.16:0/3280214074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdb00d1130 msgr2=0x7fbdb01570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.699+0000 7fbdb4952640 1 --2- 172.21.15.16:0/3280214074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdb00d1130 0x7fbdb01570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.699+0000 7fbdb4952640 1 -- 172.21.15.16:0/3280214074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb015dca0 msgr2=0x7fbdb0159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.699+0000 7fbdb4952640 1 --2- 172.21.15.16:0/3280214074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb015dca0 0x7fbdb0159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.699+0000 7fbdb4952640 1 -- 172.21.15.16:0/3280214074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb00cbce0 con 0x7fbdb00d3b90 2024-04-25T10:08:02.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.699+0000 7fbdb4952640 1 --2- 172.21.15.16:0/3280214074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdb00d3b90 0x7fbdb00ccaa0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbda400c430 tx=0x7fbda400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.699+0000 7fbdadffb640 1 -- 172.21.15.16:0/3280214074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda40052d0 con 0x7fbdb00d3b90 2024-04-25T10:08:02.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.699+0000 7fbdadffb640 1 -- 172.21.15.16:0/3280214074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda4005490 con 0x7fbdb00d3b90 2024-04-25T10:08:02.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.699+0000 7fbdb73b7640 1 -- 172.21.15.16:0/3280214074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb00cbfc0 con 0x7fbdb00d3b90 2024-04-25T10:08:02.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.699+0000 7fbdadffb640 1 -- 172.21.15.16:0/3280214074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda4007260 con 0x7fbdb00d3b90 2024-04-25T10:08:02.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.699+0000 7fbdb73b7640 1 -- 172.21.15.16:0/3280214074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb0159850 con 0x7fbdb00d3b90 2024-04-25T10:08:02.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.701+0000 7fbdadffb640 1 -- 172.21.15.16:0/3280214074 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbda4007400 con 0x7fbdb00d3b90 2024-04-25T10:08:02.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.701+0000 7fbdadffb640 1 --2- 172.21.15.16:0/3280214074 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbd9c06b4d0 0x7fbd9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.701+0000 7fbdadffb640 1 -- 172.21.15.16:0/3280214074 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbda40895f0 con 0x7fbdb00d3b90 2024-04-25T10:08:02.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.701+0000 7fbdb5153640 1 --2- 172.21.15.16:0/3280214074 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbd9c06b4d0 0x7fbd9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.702+0000 7fbdb5153640 1 --2- 172.21.15.16:0/3280214074 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbd9c06b4d0 0x7fbd9c06d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009aa0 tx=0x7fbda8009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.702+0000 7fbdba19a740 1 Processor -- start 2024-04-25T10:08:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.702+0000 7fbdba19a740 1 -- start start 2024-04-25T10:08:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.703+0000 7fbdba19a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cb10 0x55c79b28f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.703+0000 7fbdba19a740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b28cef0 0x55c79b28f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.703+0000 7fbdba19a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b25f770 0x55c79b262f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.703+0000 7fbdba19a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c79b1c8da0 con 0x55c79b28cb10 2024-04-25T10:08:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.703+0000 7fbdba19a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c79b171b80 con 0x55c79b25f770 2024-04-25T10:08:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.703+0000 7fbdba19a740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c79b1c4e40 con 0x55c79b28cef0 2024-04-25T10:08:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.703+0000 7fbdb5153640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cb10 0x55c79b28f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.703+0000 7fbdb5153640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cb10 0x55c79b28f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40058/0 (socket says 172.21.15.16:40058) 2024-04-25T10:08:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.703+0000 7fbdb5153640 1 -- 172.21.15.16:0/1900616331 learned_addr learned my addr 172.21.15.16:0/1900616331 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.703+0000 7fbdb4952640 1 --2- 172.21.15.16:0/1900616331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b28cef0 0x55c79b28f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.703+0000 7fbdb5954640 1 --2- 172.21.15.16:0/1900616331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b25f770 0x55c79b262f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.703+0000 7fbdb5153640 1 -- 172.21.15.16:0/1900616331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b28cef0 msgr2=0x55c79b28f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.703+0000 7fbdb5153640 1 --2- 172.21.15.16:0/1900616331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b28cef0 0x55c79b28f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.703+0000 7fbdb5153640 1 -- 172.21.15.16:0/1900616331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b25f770 msgr2=0x55c79b262f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.703+0000 7fbdb5153640 1 --2- 172.21.15.16:0/1900616331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b25f770 0x55c79b262f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.703+0000 7fbdb5153640 1 -- 172.21.15.16:0/1900616331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c79b17bf10 con 0x55c79b28cb10 2024-04-25T10:08:02.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.703+0000 7fbdb5954640 1 --2- 172.21.15.16:0/1900616331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b25f770 0x55c79b262f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:02.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.704+0000 7fbdb5153640 1 --2- 172.21.15.16:0/1900616331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cb10 0x55c79b28f320 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fbda8004440 tx=0x7fbda8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b6b0ee815c8ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.704+0000 7fbd96ffd640 1 -- 172.21.15.16:0/1900616331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda801b070 con 0x55c79b28cb10 2024-04-25T10:08:02.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.704+0000 7fbd96ffd640 1 -- 172.21.15.16:0/1900616331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda8027070 con 0x55c79b28cb10 2024-04-25T10:08:02.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.704+0000 7fbd96ffd640 1 -- 172.21.15.16:0/1900616331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8003a20 con 0x55c79b28cb10 2024-04-25T10:08:02.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.704+0000 7fbdba19a740 1 -- 172.21.15.16:0/1900616331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cb10 msgr2=0x55c79b28f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.704+0000 7fbdba19a740 1 --2- 172.21.15.16:0/1900616331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cb10 0x55c79b28f320 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fbda8004440 tx=0x7fbda8014000 comp rx=0 tx=0).stop 2024-04-25T10:08:02.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.704+0000 7fbdba19a740 1 -- 172.21.15.16:0/1900616331 shutdown_connections 2024-04-25T10:08:02.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.704+0000 7fbdba19a740 1 --2- 172.21.15.16:0/1900616331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b25f770 0x55c79b262f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.704+0000 7fbdba19a740 1 --2- 172.21.15.16:0/1900616331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b28cef0 0x55c79b28f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.704+0000 7fbdba19a740 1 --2- 172.21.15.16:0/1900616331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cb10 0x55c79b28f320 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.704+0000 7fbdba19a740 1 -- 172.21.15.16:0/1900616331 >> 172.21.15.16:0/1900616331 conn(0x55c79b2619e0 msgr2=0x55c79b2aa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.704+0000 7fbdba19a740 1 -- 172.21.15.16:0/1900616331 shutdown_connections 2024-04-25T10:08:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.704+0000 7fbdba19a740 1 -- 172.21.15.16:0/1900616331 wait complete. 2024-04-25T10:08:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.705+0000 7fbdba19a740 1 -- start start 2024-04-25T10:08:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.705+0000 7fbdba19a740 1 Processor -- start 2024-04-25T10:08:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.706+0000 7fbdba19a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b28cb10 0x55c79b290ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.706+0000 7fbdba19a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cef0 0x55c79b2ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.706+0000 7fbdba19a740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b27cd30 0x55c79b2aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.706+0000 7fbdba19a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c79b264b90 con 0x55c79b28cef0 2024-04-25T10:08:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.706+0000 7fbdba19a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c79b1674b0 con 0x55c79b28cb10 2024-04-25T10:08:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.706+0000 7fbdba19a740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c79b264e70 con 0x55c79b27cd30 2024-04-25T10:08:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.706+0000 7fbdba19a740 10 client.0 _ll_register_callbacks cb 0x55c79b1a7180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.706+0000 7fbdb4952640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cef0 0x55c79b2ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.706+0000 7fbdb4952640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cef0 0x55c79b2ae9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40070/0 (socket says 172.21.15.16:40070) 2024-04-25T10:08:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.706+0000 7fbdb4952640 1 -- 172.21.15.16:0/964265876 learned_addr learned my addr 172.21.15.16:0/964265876 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.706+0000 7fbdb5954640 1 --2- 172.21.15.16:0/964265876 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b27cd30 0x55c79b2aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.706+0000 7fbdb4952640 1 -- 172.21.15.16:0/964265876 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b27cd30 msgr2=0x55c79b2aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.706+0000 7fbdb5153640 1 --2- 172.21.15.16:0/964265876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b28cb10 0x55c79b290ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.706+0000 7fbdb4952640 1 --2- 172.21.15.16:0/964265876 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b27cd30 0x55c79b2aef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.706+0000 7fbdb4952640 1 -- 172.21.15.16:0/964265876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b28cb10 msgr2=0x55c79b290ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.706+0000 7fbdb4952640 1 --2- 172.21.15.16:0/964265876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b28cb10 0x55c79b290ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.706+0000 7fbdb4952640 1 -- 172.21.15.16:0/964265876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c79afa77d0 con 0x55c79b28cef0 2024-04-25T10:08:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.706+0000 7fbdb5153640 1 --2- 172.21.15.16:0/964265876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b28cb10 0x55c79b290ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.707+0000 7fbdb4952640 1 --2- 172.21.15.16:0/964265876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cef0 0x55c79b2ae9f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fbda4087b20 tx=0x7fbda4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776e713309db947e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.707+0000 7fbd8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:08:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.707+0000 7fbd8ffff640 1 -- 172.21.15.16:0/964265876 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda4060510 con 0x55c79b28cef0 2024-04-25T10:08:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.707+0000 7fbd8ffff640 1 -- 172.21.15.16:0/964265876 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbda405e070 con 0x55c79b28cef0 2024-04-25T10:08:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.707+0000 7fbdba19a740 1 -- 172.21.15.16:0/964265876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbda8027070 con 0x55c79b28cef0 2024-04-25T10:08:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.707+0000 7fbd8ffff640 1 -- 172.21.15.16:0/964265876 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbda406fb90 con 0x55c79b28cef0 2024-04-25T10:08:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.707+0000 7fbdba19a740 10 client.6303 fetch_fsmap learned FSMap version 27 2024-04-25T10:08:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.707+0000 7fbdba19a740 1 -- 172.21.15.16:0/964265876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55c79aedd3f0 con 0x55c79b28cef0 2024-04-25T10:08:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.707+0000 7fbd8ffff640 1 -- 172.21.15.16:0/964265876 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda8027070 con 0x55c79b28cef0 2024-04-25T10:08:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.707+0000 7fbd8ffff640 1 -- 172.21.15.16:0/964265876 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbda4057300 con 0x55c79b28cef0 2024-04-25T10:08:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.707+0000 7fbdba19a740 10 client.6303 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:08:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.707+0000 7fbdba19a740 10 client.6303 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.707+0000 7fbdba19a740 20 client.6303 populate_metadata read hostname 'smithi016' 2024-04-25T10:08:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.707+0000 7fbdba19a740 1 --2- 172.21.15.16:0/964265876 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c79b2ad4a0 0x55c79b2ad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.707+0000 7fbdba19a740 4 client.6303 mds_command: new command op to 35141 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.707+0000 7fbdba19a740 1 -- 172.21.15.16:0/964265876 --> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c79b1d62e0 con 0x55c79b2ad4a0 2024-04-25T10:08:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.707+0000 7fbdb5153640 1 --2- 172.21.15.16:0/964265876 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c79b2ad4a0 0x55c79b2ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.708+0000 7fbd8ffff640 10 client.6303 ms_handle_connect on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.708+0000 7fbdb5153640 1 --2- 172.21.15.16:0/964265876 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c79b2ad4a0 0x55c79b2ad880 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47792157a514b9b4 server_cookie=e9256059051f2603 in_seq=0 out_seq=0 2024-04-25T10:08:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.708+0000 7fbd8ffff640 1 -- 172.21.15.16:0/964265876 <== mds.0 v2:172.21.15.16:6832/3311584521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c79b1d62e0 con 0x55c79b2ad4a0 2024-04-25T10:08:02.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.708+0000 7fbd8ffff640 10 client.6303 handle_command_reply: tid=0 2024-04-25T10:08:02.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.708+0000 7fbdba19a740 1 client.6303 shutdown 2024-04-25T10:08:02.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.708+0000 7fbdba19a740 1 -- 172.21.15.16:0/964265876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cef0 msgr2=0x55c79b2ae9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.708+0000 7fbdba19a740 1 --2- 172.21.15.16:0/964265876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cef0 0x55c79b2ae9f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fbda4087b20 tx=0x7fbda4000bc0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.708+0000 7fbdba19a740 1 -- 172.21.15.16:0/964265876 shutdown_connections 2024-04-25T10:08:02.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.708+0000 7fbdba19a740 1 --2- 172.21.15.16:0/964265876 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c79b2ad4a0 0x55c79b2ad880 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.708+0000 7fbdba19a740 1 --2- 172.21.15.16:0/964265876 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b27cd30 0x55c79b2aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.708+0000 7fbd8ffff640 0 client.6303 ms_handle_reset on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:02.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.708+0000 7fbdba19a740 1 --2- 172.21.15.16:0/964265876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cef0 0x55c79b2ae9f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.708+0000 7fbdba19a740 1 --2- 172.21.15.16:0/964265876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b28cb10 0x55c79b290ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.708+0000 7fbdba19a740 1 -- 172.21.15.16:0/964265876 >> 172.21.15.16:0/964265876 conn(0x55c79b25f770 msgr2=0x55c79b28f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.708+0000 7fbdba19a740 1 -- 172.21.15.16:0/964265876 shutdown_connections 2024-04-25T10:08:02.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.709+0000 7fbdba19a740 1 -- 172.21.15.16:0/964265876 wait complete. 2024-04-25T10:08:02.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.709+0000 7fbdba19a740 20 client.6303 trim_cache size 0 max 16384 2024-04-25T10:08:02.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.724+0000 7fbdba19a740 1 Processor -- start 2024-04-25T10:08:02.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.724+0000 7fbdba19a740 1 -- start start 2024-04-25T10:08:02.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.724+0000 7fbdba19a740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b27cd30 0x55c79b2ad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.724+0000 7fbdba19a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b28cb10 0x55c79b263f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.724+0000 7fbdba19a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cef0 0x55c79b2644d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.724+0000 7fbdba19a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb01ffe10 con 0x55c79b28cef0 2024-04-25T10:08:02.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.724+0000 7fbdba19a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb01ff050 con 0x55c79b28cb10 2024-04-25T10:08:02.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.724+0000 7fbdba19a740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb0158500 con 0x55c79b27cd30 2024-04-25T10:08:02.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.725+0000 7fbdb5954640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cef0 0x55c79b2644d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.725+0000 7fbdb5954640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cef0 0x55c79b2644d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40084/0 (socket says 172.21.15.16:40084) 2024-04-25T10:08:02.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.725+0000 7fbdb5954640 1 -- 172.21.15.16:0/2970021479 learned_addr learned my addr 172.21.15.16:0/2970021479 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:02.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.725+0000 7fbdb4952640 1 --2- 172.21.15.16:0/2970021479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b28cb10 0x55c79b263f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.725+0000 7fbdb5954640 1 -- 172.21.15.16:0/2970021479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b27cd30 msgr2=0x55c79b2ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.725+0000 7fbdb5153640 1 --2- 172.21.15.16:0/2970021479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b27cd30 0x55c79b2ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.725+0000 7fbdb5954640 1 --2- 172.21.15.16:0/2970021479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b27cd30 0x55c79b2ad880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.725+0000 7fbdb5954640 1 -- 172.21.15.16:0/2970021479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b28cb10 msgr2=0x55c79b263f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.725+0000 7fbdb5954640 1 --2- 172.21.15.16:0/2970021479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b28cb10 0x55c79b263f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.725+0000 7fbdb5954640 1 -- 172.21.15.16:0/2970021479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c79ae9d0f0 con 0x55c79b28cef0 2024-04-25T10:08:02.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.725+0000 7fbdb4952640 1 --2- 172.21.15.16:0/2970021479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b28cb10 0x55c79b263f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:02.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.725+0000 7fbdb5954640 1 --2- 172.21.15.16:0/2970021479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cef0 0x55c79b2644d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009e50 tx=0x7fbda002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f604a13b803fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.725+0000 7fbd967fc640 1 -- 172.21.15.16:0/2970021479 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda0043070 con 0x55c79b28cef0 2024-04-25T10:08:02.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.725+0000 7fbd967fc640 1 -- 172.21.15.16:0/2970021479 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda0037040 con 0x55c79b28cef0 2024-04-25T10:08:02.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.725+0000 7fbd967fc640 1 -- 172.21.15.16:0/2970021479 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda0036040 con 0x55c79b28cef0 2024-04-25T10:08:02.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.726+0000 7fbdba19a740 1 -- 172.21.15.16:0/2970021479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cef0 msgr2=0x55c79b2644d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.726+0000 7fbdba19a740 1 --2- 172.21.15.16:0/2970021479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cef0 0x55c79b2644d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009e50 tx=0x7fbda002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:02.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.726+0000 7fbdba19a740 1 -- 172.21.15.16:0/2970021479 shutdown_connections 2024-04-25T10:08:02.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.726+0000 7fbdba19a740 1 --2- 172.21.15.16:0/2970021479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b28cef0 0x55c79b2644d0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.726+0000 7fbdba19a740 1 --2- 172.21.15.16:0/2970021479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b28cb10 0x55c79b263f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.726+0000 7fbdba19a740 1 --2- 172.21.15.16:0/2970021479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b27cd30 0x55c79b2ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.726+0000 7fbdba19a740 1 -- 172.21.15.16:0/2970021479 >> 172.21.15.16:0/2970021479 conn(0x55c79b25f770 msgr2=0x55c79b28b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.726+0000 7fbdba19a740 1 -- 172.21.15.16:0/2970021479 shutdown_connections 2024-04-25T10:08:02.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.726+0000 7fbdba19a740 1 -- 172.21.15.16:0/2970021479 wait complete. 2024-04-25T10:08:02.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.726+0000 7fbdba19a740 1 -- start start 2024-04-25T10:08:02.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.727+0000 7fbdba19a740 1 Processor -- start 2024-04-25T10:08:02.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.727+0000 7fbdba19a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b27cd30 0x55c79b313f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.727+0000 7fbdba19a740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b28cb10 0x55c79b262f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.727+0000 7fbdba19a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b28cef0 0x55c79b2ad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.727+0000 7fbdba19a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb01f41d0 con 0x55c79b27cd30 2024-04-25T10:08:02.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.727+0000 7fbdba19a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c79b2a9f10 con 0x55c79b28cef0 2024-04-25T10:08:02.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.727+0000 7fbdba19a740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c79b2637f0 con 0x55c79b28cb10 2024-04-25T10:08:02.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.727+0000 7fbdba19a740 10 client.0 _ll_register_callbacks cb 0x7fbda8009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:02.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbdb5153640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b27cd30 0x55c79b313f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbdb5153640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b27cd30 0x55c79b313f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40096/0 (socket says 172.21.15.16:40096) 2024-04-25T10:08:02.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbdb5153640 1 -- 172.21.15.16:0/2789899399 learned_addr learned my addr 172.21.15.16:0/2789899399 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:02.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbdb4952640 1 --2- 172.21.15.16:0/2789899399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b28cb10 0x55c79b262f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbdb5954640 1 --2- 172.21.15.16:0/2789899399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b28cef0 0x55c79b2ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbdb5153640 1 -- 172.21.15.16:0/2789899399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b28cb10 msgr2=0x55c79b262f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbdb5153640 1 --2- 172.21.15.16:0/2789899399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b28cb10 0x55c79b262f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbdb5153640 1 -- 172.21.15.16:0/2789899399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b28cef0 msgr2=0x55c79b2ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbdb5153640 1 --2- 172.21.15.16:0/2789899399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b28cef0 0x55c79b2ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbdb5153640 1 -- 172.21.15.16:0/2789899399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c79b290100 con 0x55c79b27cd30 2024-04-25T10:08:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbdb5153640 1 --2- 172.21.15.16:0/2789899399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b27cd30 0x55c79b313f00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fbda8024040 tx=0x7fbda8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f002ff2ae99f60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbd8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:08:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbd8ffff640 1 -- 172.21.15.16:0/2789899399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda801b040 con 0x55c79b27cd30 2024-04-25T10:08:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbd8ffff640 1 -- 172.21.15.16:0/2789899399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbda80038a0 con 0x55c79b27cd30 2024-04-25T10:08:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbdba19a740 1 -- 172.21.15.16:0/2789899399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbda0037040 con 0x55c79b27cd30 2024-04-25T10:08:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbd8ffff640 1 -- 172.21.15.16:0/2789899399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbda8044050 con 0x55c79b27cd30 2024-04-25T10:08:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbdba19a740 10 client.6309 fetch_fsmap learned FSMap version 27 2024-04-25T10:08:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbdba19a740 1 -- 172.21.15.16:0/2789899399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55c79b2634c0 con 0x55c79b27cd30 2024-04-25T10:08:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.728+0000 7fbd8ffff640 1 -- 172.21.15.16:0/2789899399 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda0037040 con 0x55c79b27cd30 2024-04-25T10:08:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbd8ffff640 1 -- 172.21.15.16:0/2789899399 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbda8047020 con 0x55c79b27cd30 2024-04-25T10:08:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbdba19a740 10 client.6309 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:08:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbdba19a740 10 client.6309 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbdba19a740 20 client.6309 populate_metadata read hostname 'smithi016' 2024-04-25T10:08:02.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbdba19a740 1 --2- 172.21.15.16:0/2789899399 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c79b2bbea0 0x55c79b25e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbdba19a740 4 client.6309 mds_command: new command op to 35141 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:08:02.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbdba19a740 1 -- 172.21.15.16:0/2789899399 --> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c79b1d0630 con 0x55c79b2bbea0 2024-04-25T10:08:02.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbdb4952640 1 --2- 172.21.15.16:0/2789899399 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c79b2bbea0 0x55c79b25e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbd8ffff640 10 client.6309 ms_handle_connect on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:02.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbdb4952640 1 --2- 172.21.15.16:0/2789899399 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c79b2bbea0 0x55c79b25e600 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=307e9d8d4f39a872 server_cookie=12644a7ea31d1bf in_seq=0 out_seq=0 2024-04-25T10:08:02.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbd8ffff640 1 -- 172.21.15.16:0/2789899399 <== mds.0 v2:172.21.15.16:6832/3311584521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x55c79b1d0630 con 0x55c79b2bbea0 2024-04-25T10:08:02.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbd8ffff640 10 client.6309 handle_command_reply: tid=0 2024-04-25T10:08:02.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbdba19a740 1 client.6309 shutdown 2024-04-25T10:08:02.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbdba19a740 1 -- 172.21.15.16:0/2789899399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b27cd30 msgr2=0x55c79b313f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbdba19a740 1 --2- 172.21.15.16:0/2789899399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b27cd30 0x55c79b313f00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fbda8024040 tx=0x7fbda8004730 comp rx=0 tx=0).stop 2024-04-25T10:08:02.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbdba19a740 1 -- 172.21.15.16:0/2789899399 shutdown_connections 2024-04-25T10:08:02.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbdba19a740 1 --2- 172.21.15.16:0/2789899399 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c79b2bbea0 0x55c79b25e600 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbdba19a740 1 --2- 172.21.15.16:0/2789899399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c79b28cef0 0x55c79b2ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbdba19a740 1 --2- 172.21.15.16:0/2789899399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c79b28cb10 0x55c79b262f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbd8ffff640 0 client.6309 ms_handle_reset on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:02.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbdba19a740 1 --2- 172.21.15.16:0/2789899399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c79b27cd30 0x55c79b313f00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.729+0000 7fbdba19a740 1 -- 172.21.15.16:0/2789899399 >> 172.21.15.16:0/2789899399 conn(0x55c79b25f770 msgr2=0x55c79b2657a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.730+0000 7fbdba19a740 1 -- 172.21.15.16:0/2789899399 shutdown_connections 2024-04-25T10:08:02.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.730+0000 7fbdba19a740 1 -- 172.21.15.16:0/2789899399 wait complete. 2024-04-25T10:08:02.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.730+0000 7fbdba19a740 20 client.6309 trim_cache size 0 max 16384 2024-04-25T10:08:02.742 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:08:02.742 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:08:02.742 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-04-25T10:08:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: "5fbaf134-e0e2-4f7c-861d-bb7f7ccb0973": { 2024-04-25T10:08:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-04-25T10:08:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "5fbaf134-e0e2-4f7c-861d-bb7f7ccb0973", 2024-04-25T10:08:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive" 2024-04-25T10:08:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:08:02.743 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:08:02.743 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:08:02.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.730+0000 7fbdb73b7640 1 -- 172.21.15.16:0/3280214074 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbd9c06b4d0 msgr2=0x7fbd9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.730+0000 7fbdb73b7640 1 --2- 172.21.15.16:0/3280214074 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbd9c06b4d0 0x7fbd9c06d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009aa0 tx=0x7fbda8009190 comp rx=0 tx=0).stop 2024-04-25T10:08:02.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.730+0000 7fbdb73b7640 1 -- 172.21.15.16:0/3280214074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdb00d3b90 msgr2=0x7fbdb00ccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.730+0000 7fbdb73b7640 1 --2- 172.21.15.16:0/3280214074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdb00d3b90 0x7fbdb00ccaa0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbda400c430 tx=0x7fbda400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.730+0000 7fbdb73b7640 1 -- 172.21.15.16:0/3280214074 shutdown_connections 2024-04-25T10:08:02.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.730+0000 7fbdb73b7640 1 --2- 172.21.15.16:0/3280214074 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbd9c06b4d0 0x7fbd9c06d990 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.730+0000 7fbdb73b7640 1 --2- 172.21.15.16:0/3280214074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb015dca0 0x7fbdb0159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.730+0000 7fbdb73b7640 1 --2- 172.21.15.16:0/3280214074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdb00d3b90 0x7fbdb00ccaa0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.730+0000 7fbdb73b7640 1 --2- 172.21.15.16:0/3280214074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdb00d1130 0x7fbdb01570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.730+0000 7fbdb73b7640 1 -- 172.21.15.16:0/3280214074 >> 172.21.15.16:0/3280214074 conn(0x7fbdb00726c0 msgr2=0x7fbdb015cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.730+0000 7fbdb73b7640 1 -- 172.21.15.16:0/3280214074 shutdown_connections 2024-04-25T10:08:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:02.730+0000 7fbdb73b7640 1 -- 172.21.15.16:0/3280214074 wait complete. 2024-04-25T10:08:02.747 INFO:tasks.cephfs.filesystem:scrub status for tag:5fbaf134-e0e2-4f7c-861d-bb7f7ccb0973 - {'path': '/', 'tag': '5fbaf134-e0e2-4f7c-861d-bb7f7ccb0973', 'options': 'recursive'} 2024-04-25T10:08:07.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:07.461 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:07.504 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-04-25T10:08:07.498+0000 7ff1421c0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:08:32.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:32.749 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:08:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.860+0000 7f81cade0640 1 Processor -- start 2024-04-25T10:08:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.860+0000 7f81cade0640 1 -- start start 2024-04-25T10:08:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.860+0000 7f81cade0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81c40cfe70 0x7f81c40ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.860+0000 7f81cade0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c415f000 con 0x7f81c415e110 2024-04-25T10:08:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.861+0000 7f81cade0640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c415ed20 con 0x7f81c415b7b0 2024-04-25T10:08:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.861+0000 7f81cade0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81c40ced00 con 0x7f81c40cfe70 2024-04-25T10:08:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.861+0000 7f81c3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81c40cfe70 0x7f81c40ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.861+0000 7f81c3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81c40cfe70 0x7f81c40ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37532/0 (socket says 172.21.15.16:37532) 2024-04-25T10:08:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.861+0000 7f81c3fff640 1 -- 172.21.15.16:0/409197974 learned_addr learned my addr 172.21.15.16:0/409197974 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.861+0000 7f81c937d640 1 --1- 172.21.15.16:0/409197974 >> v1:172.21.15.16:6789/0 conn(0x7f81c415e110 0x7f81c415e520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36758/0 (socket says 172.21.15.16:36758) 2024-04-25T10:08:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.861+0000 7f81c3fff640 1 -- 172.21.15.16:0/409197974 >> v1:172.21.15.186:6789/0 conn(0x7f81c415b7b0 legacy=0x7f81c40cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.861+0000 7f81c3fff640 1 -- 172.21.15.16:0/409197974 >> v1:172.21.15.16:6789/0 conn(0x7f81c415e110 legacy=0x7f81c415e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.861+0000 7f81c3fff640 1 -- 172.21.15.16:0/409197974 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c40cdce0 con 0x7f81c40cfe70 2024-04-25T10:08:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.861+0000 7f81c37fe640 1 -- 172.21.15.16:0/409197974 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963688261 0 0) 0x7f81c415f000 con 0x7f81c415e110 2024-04-25T10:08:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.862+0000 7f81c3fff640 1 --2- 172.21.15.16:0/409197974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c40cfe70 0x7f81c40ce7c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009870 tx=0x7f81ac02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba11ee3af64678f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.862+0000 7f81c37fe640 1 -- 172.21.15.16:0/409197974 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac037070 con 0x7f81c40cfe70 2024-04-25T10:08:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.862+0000 7f81c37fe640 1 -- 172.21.15.16:0/409197974 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac02ae50 con 0x7f81c40cfe70 2024-04-25T10:08:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.862+0000 7f81c37fe640 1 -- 172.21.15.16:0/409197974 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac032860 con 0x7f81c40cfe70 2024-04-25T10:08:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.862+0000 7f81cade0640 1 -- 172.21.15.16:0/409197974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c40cfe70 msgr2=0x7f81c40ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.862+0000 7f81cade0640 1 --2- 172.21.15.16:0/409197974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c40cfe70 0x7f81c40ce7c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009870 tx=0x7f81ac02a300 comp rx=0 tx=0).stop 2024-04-25T10:08:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.863+0000 7f81cade0640 1 -- 172.21.15.16:0/409197974 shutdown_connections 2024-04-25T10:08:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.863+0000 7f81cade0640 1 --2- 172.21.15.16:0/409197974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c40cfe70 0x7f81c40ce7c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.863+0000 7f81cade0640 1 -- 172.21.15.16:0/409197974 >> 172.21.15.16:0/409197974 conn(0x7f81c406f4a0 msgr2=0x7f81c406f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.863+0000 7f81cade0640 1 -- 172.21.15.16:0/409197974 shutdown_connections 2024-04-25T10:08:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.863+0000 7f81cade0640 1 -- 172.21.15.16:0/409197974 wait complete. 2024-04-25T10:08:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.864+0000 7f81cade0640 1 Processor -- start 2024-04-25T10:08:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.864+0000 7f81cade0640 1 -- start start 2024-04-25T10:08:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81cade0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c40cfe70 0x7f81c41f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81cade0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81c415b7b0 0x7f81c41f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81cade0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81c415e110 0x7f81c41e7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81cade0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c41619f0 con 0x7f81c40cfe70 2024-04-25T10:08:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81cade0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c40ce330 con 0x7f81c415e110 2024-04-25T10:08:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81cade0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c40cc080 con 0x7f81c415b7b0 2024-04-25T10:08:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81c8b7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c40cfe70 0x7f81c41f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81c8b7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c40cfe70 0x7f81c41f69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37542/0 (socket says 172.21.15.16:37542) 2024-04-25T10:08:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81c8b7c640 1 -- 172.21.15.16:0/3118560513 learned_addr learned my addr 172.21.15.16:0/3118560513 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81c3fff640 1 --2- 172.21.15.16:0/3118560513 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81c415b7b0 0x7f81c41f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81c937d640 1 --2- 172.21.15.16:0/3118560513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81c415e110 0x7f81c41e7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81c8b7c640 1 -- 172.21.15.16:0/3118560513 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81c415b7b0 msgr2=0x7f81c41f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81c8b7c640 1 --2- 172.21.15.16:0/3118560513 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81c415b7b0 0x7f81c41f3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81c8b7c640 1 -- 172.21.15.16:0/3118560513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81c415e110 msgr2=0x7f81c41e7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81c8b7c640 1 --2- 172.21.15.16:0/3118560513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81c415e110 0x7f81c41e7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81c8b7c640 1 -- 172.21.15.16:0/3118560513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c41e85b0 con 0x7f81c40cfe70 2024-04-25T10:08:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81c3fff640 1 --2- 172.21.15.16:0/3118560513 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81c415b7b0 0x7f81c41f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81c8b7c640 1 --2- 172.21.15.16:0/3118560513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c40cfe70 0x7f81c41f69f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00c870 tx=0x7f81bc00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.865+0000 7f81c17fa640 1 -- 172.21.15.16:0/3118560513 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81bc003b70 con 0x7f81c40cfe70 2024-04-25T10:08:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.866+0000 7f81c17fa640 1 -- 172.21.15.16:0/3118560513 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81bc003d30 con 0x7f81c40cfe70 2024-04-25T10:08:32.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.866+0000 7f81c17fa640 1 -- 172.21.15.16:0/3118560513 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81bc00aa90 con 0x7f81c40cfe70 2024-04-25T10:08:32.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.866+0000 7f81cade0640 1 -- 172.21.15.16:0/3118560513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c40b4620 con 0x7f81c40cfe70 2024-04-25T10:08:32.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.866+0000 7f81cade0640 1 -- 172.21.15.16:0/3118560513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c40b4be0 con 0x7f81c40cfe70 2024-04-25T10:08:32.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.867+0000 7f81c17fa640 1 -- 172.21.15.16:0/3118560513 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81bc00ac30 con 0x7f81c40cfe70 2024-04-25T10:08:32.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.868+0000 7f81c17fa640 1 --2- 172.21.15.16:0/3118560513 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f81a406b4d0 0x7f81a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.868+0000 7f81c17fa640 1 -- 172.21.15.16:0/3118560513 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81bc08a210 con 0x7f81c40cfe70 2024-04-25T10:08:32.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.868+0000 7f81c3fff640 1 --2- 172.21.15.16:0/3118560513 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f81a406b4d0 0x7f81a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.869+0000 7f81c3fff640 1 --2- 172.21.15.16:0/3118560513 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f81a406b4d0 0x7f81a406d990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f81ac034000 tx=0x7f81ac02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:32.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.869+0000 7f81cdbc3740 1 Processor -- start 2024-04-25T10:08:32.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.869+0000 7f81cdbc3740 1 -- start start 2024-04-25T10:08:32.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81cdbc3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3347fbb10 0x55c3347fe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81cdbc3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c3347fe860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81cdbc3740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3347ce770 0x55c3347d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81cdbc3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c334737da0 con 0x55c3347fbef0 2024-04-25T10:08:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81cdbc3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3346e0b80 con 0x55c3347fbb10 2024-04-25T10:08:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81cdbc3740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c334733e40 con 0x55c3347ce770 2024-04-25T10:08:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c3347fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c3347fe860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37558/0 (socket says 172.21.15.16:37558) 2024-04-25T10:08:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81c3fff640 1 -- 172.21.15.16:0/3079249932 learned_addr learned my addr 172.21.15.16:0/3079249932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81c937d640 1 --2- 172.21.15.16:0/3079249932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3347ce770 0x55c3347d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81c8b7c640 1 --2- 172.21.15.16:0/3079249932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3347fbb10 0x55c3347fe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81c3fff640 1 -- 172.21.15.16:0/3079249932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3347ce770 msgr2=0x55c3347d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81c3fff640 1 --2- 172.21.15.16:0/3079249932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3347ce770 0x55c3347d1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81c3fff640 1 -- 172.21.15.16:0/3079249932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3347fbb10 msgr2=0x55c3347fe320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81c3fff640 1 --2- 172.21.15.16:0/3079249932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3347fbb10 0x55c3347fe320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81c3fff640 1 -- 172.21.15.16:0/3079249932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3346eaf10 con 0x55c3347fbef0 2024-04-25T10:08:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81c937d640 1 --2- 172.21.15.16:0/3079249932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3347ce770 0x55c3347d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81c3fff640 1 --2- 172.21.15.16:0/3079249932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c3347fe860 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f81ac02a8f0 tx=0x7f81ac02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb7003f5208ecbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81ba7fc640 1 -- 172.21.15.16:0/3079249932 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac04e070 con 0x55c3347fbef0 2024-04-25T10:08:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.870+0000 7f81ba7fc640 1 -- 172.21.15.16:0/3079249932 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac02ae50 con 0x55c3347fbef0 2024-04-25T10:08:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.871+0000 7f81ba7fc640 1 -- 172.21.15.16:0/3079249932 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac037040 con 0x55c3347fbef0 2024-04-25T10:08:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.871+0000 7f81cdbc3740 1 -- 172.21.15.16:0/3079249932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 msgr2=0x55c3347fe860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.871+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/3079249932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c3347fe860 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f81ac02a8f0 tx=0x7f81ac02c000 comp rx=0 tx=0).stop 2024-04-25T10:08:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.871+0000 7f81cdbc3740 1 -- 172.21.15.16:0/3079249932 shutdown_connections 2024-04-25T10:08:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.871+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/3079249932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3347ce770 0x55c3347d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.871+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/3079249932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c3347fe860 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.871+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/3079249932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3347fbb10 0x55c3347fe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.871+0000 7f81cdbc3740 1 -- 172.21.15.16:0/3079249932 >> 172.21.15.16:0/3079249932 conn(0x55c3347d09e0 msgr2=0x55c334819320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.871+0000 7f81cdbc3740 1 -- 172.21.15.16:0/3079249932 shutdown_connections 2024-04-25T10:08:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.871+0000 7f81cdbc3740 1 -- 172.21.15.16:0/3079249932 wait complete. 2024-04-25T10:08:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.872+0000 7f81cdbc3740 1 -- start start 2024-04-25T10:08:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.872+0000 7f81cdbc3740 1 Processor -- start 2024-04-25T10:08:32.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81cdbc3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3347fbb10 0x55c33481bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81cdbc3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c33481d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81cdbc3740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c33481a450 0x55c334815650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81cdbc3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3347d3b90 con 0x55c3347fbef0 2024-04-25T10:08:32.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81cdbc3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3346d64b0 con 0x55c3347fbb10 2024-04-25T10:08:32.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81cdbc3740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3347d3e70 con 0x55c33481a450 2024-04-25T10:08:32.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81cdbc3740 10 client.0 _ll_register_callbacks cb 0x55c334716180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:32.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c33481d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c33481d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37574/0 (socket says 172.21.15.16:37574) 2024-04-25T10:08:32.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81c3fff640 1 -- 172.21.15.16:0/1548503625 learned_addr learned my addr 172.21.15.16:0/1548503625 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:32.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81c8b7c640 1 --2- 172.21.15.16:0/1548503625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3347fbb10 0x55c33481bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81c937d640 1 --2- 172.21.15.16:0/1548503625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c33481a450 0x55c334815650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81c3fff640 1 -- 172.21.15.16:0/1548503625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c33481a450 msgr2=0x55c334815650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:32.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81c3fff640 1 --2- 172.21.15.16:0/1548503625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c33481a450 0x55c334815650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81c3fff640 1 -- 172.21.15.16:0/1548503625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3347fbb10 msgr2=0x55c33481bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:32.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81c3fff640 1 --2- 172.21.15.16:0/1548503625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3347fbb10 0x55c33481bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81c3fff640 1 -- 172.21.15.16:0/1548503625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3345167d0 con 0x55c3347fbef0 2024-04-25T10:08:32.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81c8b7c640 1 --2- 172.21.15.16:0/1548503625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3347fbb10 0x55c33481bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:32.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81c3fff640 1 --2- 172.21.15.16:0/1548503625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c33481d070 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f81ac000c00 tx=0x7f81ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f992d376c6aad789 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:32.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.873+0000 7f81a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:08:32.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.874+0000 7f81a37fe640 1 -- 172.21.15.16:0/1548503625 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac04e070 con 0x55c3347fbef0 2024-04-25T10:08:32.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.874+0000 7f81a37fe640 1 -- 172.21.15.16:0/1548503625 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81ac03ed40 con 0x55c3347fbef0 2024-04-25T10:08:32.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.874+0000 7f81cdbc3740 1 -- 172.21.15.16:0/1548503625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81ac02ae50 con 0x55c3347fbef0 2024-04-25T10:08:32.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.874+0000 7f81a37fe640 1 -- 172.21.15.16:0/1548503625 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81ac037040 con 0x55c3347fbef0 2024-04-25T10:08:32.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.874+0000 7f81cdbc3740 10 client.6324 fetch_fsmap learned FSMap version 28 2024-04-25T10:08:32.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.874+0000 7f81cdbc3740 1 -- 172.21.15.16:0/1548503625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55c33444c3f0 con 0x55c3347fbef0 2024-04-25T10:08:32.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.874+0000 7f81a37fe640 1 -- 172.21.15.16:0/1548503625 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac02ae50 con 0x55c3347fbef0 2024-04-25T10:08:32.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.874+0000 7f81a37fe640 1 -- 172.21.15.16:0/1548503625 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f81ac05e080 con 0x55c3347fbef0 2024-04-25T10:08:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.874+0000 7f81cdbc3740 10 client.6324 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:08:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.874+0000 7f81cdbc3740 10 client.6324 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.874+0000 7f81cdbc3740 20 client.6324 populate_metadata read hostname 'smithi016' 2024-04-25T10:08:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.874+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/1548503625 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c3347f95d0 0x55c334813900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.874+0000 7f81cdbc3740 4 client.6324 mds_command: new command op to 35141 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.874+0000 7f81cdbc3740 1 -- 172.21.15.16:0/1548503625 --> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3347452e0 con 0x55c3347f95d0 2024-04-25T10:08:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.874+0000 7f81c8b7c640 1 --2- 172.21.15.16:0/1548503625 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c3347f95d0 0x55c334813900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.875+0000 7f81a37fe640 10 client.6324 ms_handle_connect on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.875+0000 7f81c8b7c640 1 --2- 172.21.15.16:0/1548503625 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c3347f95d0 0x55c334813900 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e06ca5fbf85b6589 server_cookie=84876c251b7e852e in_seq=0 out_seq=0 2024-04-25T10:08:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.875+0000 7f81a37fe640 1 -- 172.21.15.16:0/1548503625 <== mds.0 v2:172.21.15.16:6832/3311584521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c3347452e0 con 0x55c3347f95d0 2024-04-25T10:08:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.875+0000 7f81a37fe640 10 client.6324 handle_command_reply: tid=0 2024-04-25T10:08:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.875+0000 7f81cdbc3740 1 client.6324 shutdown 2024-04-25T10:08:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.875+0000 7f81cdbc3740 1 -- 172.21.15.16:0/1548503625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 msgr2=0x55c33481d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.875+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/1548503625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c33481d070 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f81ac000c00 tx=0x7f81ac004730 comp rx=0 tx=0).stop 2024-04-25T10:08:32.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.875+0000 7f81cdbc3740 1 -- 172.21.15.16:0/1548503625 shutdown_connections 2024-04-25T10:08:32.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.875+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/1548503625 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c3347f95d0 0x55c334813900 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.875+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/1548503625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c33481a450 0x55c334815650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.875+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/1548503625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c33481d070 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.875+0000 7f81a37fe640 0 client.6324 ms_handle_reset on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:32.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.875+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/1548503625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3347fbb10 0x55c33481bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.875+0000 7f81cdbc3740 1 -- 172.21.15.16:0/1548503625 >> 172.21.15.16:0/1548503625 conn(0x55c3347ce770 msgr2=0x55c3347cf810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:32.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.876+0000 7f81cdbc3740 1 -- 172.21.15.16:0/1548503625 shutdown_connections 2024-04-25T10:08:32.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.876+0000 7f81cdbc3740 1 -- 172.21.15.16:0/1548503625 wait complete. 2024-04-25T10:08:32.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.876+0000 7f81cdbc3740 20 client.6324 trim_cache size 0 max 16384 2024-04-25T10:08:32.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.891+0000 7f81cdbc3740 1 Processor -- start 2024-04-25T10:08:32.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.891+0000 7f81cdbc3740 1 -- start start 2024-04-25T10:08:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81cdbc3740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3347fbb10 0x55c33483bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81cdbc3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c3348329a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81cdbc3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c33481a450 0x55c334801e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81cdbc3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c41ffca0 con 0x55c3347fbef0 2024-04-25T10:08:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81cdbc3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c41feeb0 con 0x55c33481a450 2024-04-25T10:08:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81cdbc3740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c41fe380 con 0x55c3347fbb10 2024-04-25T10:08:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c3348329a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c3348329a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37584/0 (socket says 172.21.15.16:37584) 2024-04-25T10:08:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81c3fff640 1 -- 172.21.15.16:0/4197621824 learned_addr learned my addr 172.21.15.16:0/4197621824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81c937d640 1 --2- 172.21.15.16:0/4197621824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c33481a450 0x55c334801e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81c8b7c640 1 --2- 172.21.15.16:0/4197621824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3347fbb10 0x55c33483bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81c3fff640 1 -- 172.21.15.16:0/4197621824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3347fbb10 msgr2=0x55c33483bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81c3fff640 1 --2- 172.21.15.16:0/4197621824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3347fbb10 0x55c33483bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81c3fff640 1 -- 172.21.15.16:0/4197621824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c33481a450 msgr2=0x55c334801e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81c3fff640 1 --2- 172.21.15.16:0/4197621824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c33481a450 0x55c334801e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81c3fff640 1 -- 172.21.15.16:0/4197621824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c33440c0f0 con 0x55c3347fbef0 2024-04-25T10:08:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81c8b7c640 1 --2- 172.21.15.16:0/4197621824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3347fbb10 0x55c33483bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81c3fff640 1 --2- 172.21.15.16:0/4197621824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c3348329a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f81ac032d30 tx=0x7f81ac04a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5db9fd09448878 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81b9ffb640 1 -- 172.21.15.16:0/4197621824 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac04e040 con 0x55c3347fbef0 2024-04-25T10:08:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81b9ffb640 1 -- 172.21.15.16:0/4197621824 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac02ae50 con 0x55c3347fbef0 2024-04-25T10:08:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.892+0000 7f81b9ffb640 1 -- 172.21.15.16:0/4197621824 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac03eb40 con 0x55c3347fbef0 2024-04-25T10:08:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.893+0000 7f81cdbc3740 1 -- 172.21.15.16:0/4197621824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 msgr2=0x55c3348329a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.893+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/4197621824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c3348329a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f81ac032d30 tx=0x7f81ac04a000 comp rx=0 tx=0).stop 2024-04-25T10:08:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.893+0000 7f81cdbc3740 1 -- 172.21.15.16:0/4197621824 shutdown_connections 2024-04-25T10:08:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.893+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/4197621824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c33481a450 0x55c334801e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.893+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/4197621824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c3348329a0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.893+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/4197621824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3347fbb10 0x55c33483bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.893+0000 7f81cdbc3740 1 -- 172.21.15.16:0/4197621824 >> 172.21.15.16:0/4197621824 conn(0x55c3347ce770 msgr2=0x55c334814ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.893+0000 7f81cdbc3740 1 -- 172.21.15.16:0/4197621824 shutdown_connections 2024-04-25T10:08:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.893+0000 7f81cdbc3740 1 -- 172.21.15.16:0/4197621824 wait complete. 2024-04-25T10:08:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.893+0000 7f81cdbc3740 1 -- start start 2024-04-25T10:08:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.894+0000 7f81cdbc3740 1 Processor -- start 2024-04-25T10:08:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.894+0000 7f81cdbc3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3347fbb10 0x55c334883720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.894+0000 7f81cdbc3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c334813900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.894+0000 7f81cdbc3740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c33481a450 0x55c33481c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.894+0000 7f81cdbc3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c334832ee0 con 0x55c3347fbef0 2024-04-25T10:08:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.894+0000 7f81cdbc3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c41fe8b0 con 0x55c3347fbb10 2024-04-25T10:08:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.894+0000 7f81cdbc3740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3347d3900 con 0x55c33481a450 2024-04-25T10:08:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.895+0000 7f81cdbc3740 10 client.0 _ll_register_callbacks cb 0x7f81ac035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.895+0000 7f81c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c334813900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.895+0000 7f81c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c334813900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37598/0 (socket says 172.21.15.16:37598) 2024-04-25T10:08:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.895+0000 7f81c3fff640 1 -- 172.21.15.16:0/1894085403 learned_addr learned my addr 172.21.15.16:0/1894085403 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.895+0000 7f81c8b7c640 1 --2- 172.21.15.16:0/1894085403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3347fbb10 0x55c334883720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.895+0000 7f81c937d640 1 --2- 172.21.15.16:0/1894085403 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c33481a450 0x55c33481c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.895+0000 7f81c3fff640 1 -- 172.21.15.16:0/1894085403 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c33481a450 msgr2=0x55c33481c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.895+0000 7f81c3fff640 1 --2- 172.21.15.16:0/1894085403 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c33481a450 0x55c33481c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.895+0000 7f81c3fff640 1 -- 172.21.15.16:0/1894085403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3347fbb10 msgr2=0x55c334883720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.895+0000 7f81c3fff640 1 --2- 172.21.15.16:0/1894085403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3347fbb10 0x55c334883720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.895+0000 7f81c3fff640 1 -- 172.21.15.16:0/1894085403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c334801ae0 con 0x55c3347fbef0 2024-04-25T10:08:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.895+0000 7f81c3fff640 1 --2- 172.21.15.16:0/1894085403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c334813900 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f81ac032d30 tx=0x7f81ac008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78de2a22a007a8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.895+0000 7f81a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:08:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.895+0000 7f81a37fe640 1 -- 172.21.15.16:0/1894085403 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac068020 con 0x55c3347fbef0 2024-04-25T10:08:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.895+0000 7f81a37fe640 1 -- 172.21.15.16:0/1894085403 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81ac053b80 con 0x55c3347fbef0 2024-04-25T10:08:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.895+0000 7f81cdbc3740 1 -- 172.21.15.16:0/1894085403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81ac02ae50 con 0x55c3347fbef0 2024-04-25T10:08:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.895+0000 7f81a37fe640 1 -- 172.21.15.16:0/1894085403 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81ac06f050 con 0x55c3347fbef0 2024-04-25T10:08:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.896+0000 7f81cdbc3740 10 client.6330 fetch_fsmap learned FSMap version 28 2024-04-25T10:08:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.896+0000 7f81cdbc3740 1 -- 172.21.15.16:0/1894085403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55c33481c350 con 0x55c3347fbef0 2024-04-25T10:08:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.896+0000 7f81a37fe640 1 -- 172.21.15.16:0/1894085403 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac02ae50 con 0x55c3347fbef0 2024-04-25T10:08:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.896+0000 7f81a37fe640 1 -- 172.21.15.16:0/1894085403 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f81ac072020 con 0x55c3347fbef0 2024-04-25T10:08:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.896+0000 7f81cdbc3740 10 client.6330 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:08:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.896+0000 7f81cdbc3740 10 client.6330 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.896+0000 7f81cdbc3740 20 client.6330 populate_metadata read hostname 'smithi016' 2024-04-25T10:08:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.896+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/1894085403 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c33481aba0 0x55c33482af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.896+0000 7f81cdbc3740 4 client.6330 mds_command: new command op to 35141 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:08:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.896+0000 7f81cdbc3740 1 -- 172.21.15.16:0/1894085403 --> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c33473f630 con 0x55c33481aba0 2024-04-25T10:08:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.896+0000 7f81c8b7c640 1 --2- 172.21.15.16:0/1894085403 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c33481aba0 0x55c33482af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.896+0000 7f81a37fe640 10 client.6330 ms_handle_connect on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.896+0000 7f81c8b7c640 1 --2- 172.21.15.16:0/1894085403 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c33481aba0 0x55c33482af90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b6853c3af079b8d server_cookie=c6485367d3996e4b in_seq=0 out_seq=0 2024-04-25T10:08:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.896+0000 7f81a37fe640 1 -- 172.21.15.16:0/1894085403 <== mds.0 v2:172.21.15.16:6832/3311584521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c33473f630 con 0x55c33481aba0 2024-04-25T10:08:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.896+0000 7f81a37fe640 10 client.6330 handle_command_reply: tid=0 2024-04-25T10:08:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.896+0000 7f81cdbc3740 1 client.6330 shutdown 2024-04-25T10:08:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cdbc3740 1 -- 172.21.15.16:0/1894085403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 msgr2=0x55c334813900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/1894085403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c334813900 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f81ac032d30 tx=0x7f81ac008030 comp rx=0 tx=0).stop 2024-04-25T10:08:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cdbc3740 1 -- 172.21.15.16:0/1894085403 shutdown_connections 2024-04-25T10:08:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/1894085403 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55c33481aba0 0x55c33482af90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/1894085403 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c33481a450 0x55c33481c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/1894085403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3347fbef0 0x55c334813900 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81a37fe640 0 client.6330 ms_handle_reset on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cdbc3740 1 --2- 172.21.15.16:0/1894085403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3347fbb10 0x55c334883720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cdbc3740 1 -- 172.21.15.16:0/1894085403 >> 172.21.15.16:0/1894085403 conn(0x55c3347ce770 msgr2=0x55c334814ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cdbc3740 1 -- 172.21.15.16:0/1894085403 shutdown_connections 2024-04-25T10:08:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cdbc3740 1 -- 172.21.15.16:0/1894085403 wait complete. 2024-04-25T10:08:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cdbc3740 20 client.6330 trim_cache size 0 max 16384 2024-04-25T10:08:32.911 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:08:32.911 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-04-25T10:08:32.911 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-04-25T10:08:32.911 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:08:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cade0640 1 -- 172.21.15.16:0/3118560513 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f81a406b4d0 msgr2=0x7f81a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cade0640 1 --2- 172.21.15.16:0/3118560513 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f81a406b4d0 0x7f81a406d990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f81ac034000 tx=0x7f81ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:08:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cade0640 1 -- 172.21.15.16:0/3118560513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c40cfe70 msgr2=0x7f81c41f69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cade0640 1 --2- 172.21.15.16:0/3118560513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c40cfe70 0x7f81c41f69f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00c870 tx=0x7f81bc00cd30 comp rx=0 tx=0).stop 2024-04-25T10:08:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cade0640 1 -- 172.21.15.16:0/3118560513 shutdown_connections 2024-04-25T10:08:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cade0640 1 --2- 172.21.15.16:0/3118560513 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f81a406b4d0 0x7f81a406d990 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cade0640 1 --2- 172.21.15.16:0/3118560513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81c415e110 0x7f81c41e7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cade0640 1 --2- 172.21.15.16:0/3118560513 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81c415b7b0 0x7f81c41f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cade0640 1 --2- 172.21.15.16:0/3118560513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c40cfe70 0x7f81c41f69f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.897+0000 7f81cade0640 1 -- 172.21.15.16:0/3118560513 >> 172.21.15.16:0/3118560513 conn(0x7f81c406f4a0 msgr2=0x7f81c406bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.898+0000 7f81cade0640 1 -- 172.21.15.16:0/3118560513 shutdown_connections 2024-04-25T10:08:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:32.898+0000 7f81cade0640 1 -- 172.21.15.16:0/3118560513 wait complete. 2024-04-25T10:08:32.914 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:08:32.914 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:33.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.009+0000 7fcaff6e1640 1 Processor -- start 2024-04-25T10:08:33.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.009+0000 7fcaff6e1640 1 -- start start 2024-04-25T10:08:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.010+0000 7fcaff6e1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaf806e8f0 0x7fcaf815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.010+0000 7fcaff6e1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf80ce6c0 con 0x7fcaf815c820 2024-04-25T10:08:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.010+0000 7fcaff6e1640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf80cc4e0 con 0x7fcaf815bf00 2024-04-25T10:08:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.010+0000 7fcaff6e1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf81455e0 con 0x7fcaf806e8f0 2024-04-25T10:08:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.010+0000 7fcafdc7e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcaf815c820 0x7fcaf81654b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36760/0 (socket says 172.21.15.16:36760) 2024-04-25T10:08:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.010+0000 7fcafdc7e640 1 -- 172.21.15.16:0/2825865331 learned_addr learned my addr 172.21.15.16:0/2825865331 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:08:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.010+0000 7fcafcc7c640 1 --2- 172.21.15.16:0/2825865331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaf806e8f0 0x7fcaf815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.011+0000 7fcaf7fff640 1 -- 172.21.15.16:0/2825865331 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387872314 0 0) 0x7fcae8001490 con 0x7fcaf815c820 2024-04-25T10:08:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.011+0000 7fcaf7fff640 1 -- 172.21.15.16:0/2825865331 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcae0003520 con 0x7fcaf815c820 2024-04-25T10:08:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.011+0000 7fcaf7fff640 1 -- 172.21.15.16:0/2825865331 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4260790525 0 0) 0x7fcaf80cc4e0 con 0x7fcaf815bf00 2024-04-25T10:08:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.011+0000 7fcafcc7c640 1 -- 172.21.15.16:0/2825865331 >> v1:172.21.15.186:6789/0 conn(0x7fcaf815bf00 legacy=0x7fcaf80cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.011+0000 7fcafcc7c640 1 -- 172.21.15.16:0/2825865331 >> v1:172.21.15.16:6789/0 conn(0x7fcaf815c820 legacy=0x7fcaf81654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.011+0000 7fcafcc7c640 1 -- 172.21.15.16:0/2825865331 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf80cc7b0 con 0x7fcaf806e8f0 2024-04-25T10:08:33.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.011+0000 7fcafcc7c640 1 --2- 172.21.15.16:0/2825865331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaf806e8f0 0x7fcaf815c2e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0097d0 tx=0x7fcaec02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47fa550de7e4107a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.012+0000 7fcaf7fff640 1 -- 172.21.15.16:0/2825865331 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec037070 con 0x7fcaf806e8f0 2024-04-25T10:08:33.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.012+0000 7fcaf7fff640 1 -- 172.21.15.16:0/2825865331 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaec032440 con 0x7fcaf806e8f0 2024-04-25T10:08:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.012+0000 7fcaf7fff640 1 -- 172.21.15.16:0/2825865331 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec032810 con 0x7fcaf806e8f0 2024-04-25T10:08:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.012+0000 7fcaff6e1640 1 -- 172.21.15.16:0/2825865331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaf806e8f0 msgr2=0x7fcaf815c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.012+0000 7fcaff6e1640 1 --2- 172.21.15.16:0/2825865331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaf806e8f0 0x7fcaf815c2e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0097d0 tx=0x7fcaec02a210 comp rx=0 tx=0).stop 2024-04-25T10:08:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.012+0000 7fcaff6e1640 1 -- 172.21.15.16:0/2825865331 shutdown_connections 2024-04-25T10:08:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.012+0000 7fcaff6e1640 1 --2- 172.21.15.16:0/2825865331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaf806e8f0 0x7fcaf815c2e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.012+0000 7fcaff6e1640 1 -- 172.21.15.16:0/2825865331 >> 172.21.15.16:0/2825865331 conn(0x7fcaf806aba0 msgr2=0x7fcaf806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.012+0000 7fcaff6e1640 1 -- 172.21.15.16:0/2825865331 shutdown_connections 2024-04-25T10:08:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.012+0000 7fcaff6e1640 1 -- 172.21.15.16:0/2825865331 wait complete. 2024-04-25T10:08:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.013+0000 7fcaff6e1640 1 Processor -- start 2024-04-25T10:08:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.014+0000 7fcaff6e1640 1 -- start start 2024-04-25T10:08:33.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.014+0000 7fcaff6e1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcaf806e8f0 0x7fcaf813cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.014+0000 7fcaff6e1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaf815bf00 0x7fcaf815e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.014+0000 7fcaff6e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf815c820 0x7fcaf814ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.014+0000 7fcaff6e1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf80ccd70 con 0x7fcaf815c820 2024-04-25T10:08:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.014+0000 7fcaff6e1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf806d450 con 0x7fcaf806e8f0 2024-04-25T10:08:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.014+0000 7fcaff6e1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf80cd050 con 0x7fcaf815bf00 2024-04-25T10:08:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.014+0000 7fcafdc7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf815c820 0x7fcaf814ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.014+0000 7fcafdc7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf815c820 0x7fcaf814ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37612/0 (socket says 172.21.15.16:37612) 2024-04-25T10:08:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.014+0000 7fcafdc7e640 1 -- 172.21.15.16:0/569185303 learned_addr learned my addr 172.21.15.16:0/569185303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.015+0000 7fcafcc7c640 1 --2- 172.21.15.16:0/569185303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaf815bf00 0x7fcaf815e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.015+0000 7fcafd47d640 1 --2- 172.21.15.16:0/569185303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcaf806e8f0 0x7fcaf813cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.015+0000 7fcafdc7e640 1 -- 172.21.15.16:0/569185303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaf815bf00 msgr2=0x7fcaf815e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.015+0000 7fcafdc7e640 1 --2- 172.21.15.16:0/569185303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaf815bf00 0x7fcaf815e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.015+0000 7fcafdc7e640 1 -- 172.21.15.16:0/569185303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcaf806e8f0 msgr2=0x7fcaf813cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.015+0000 7fcafdc7e640 1 --2- 172.21.15.16:0/569185303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcaf806e8f0 0x7fcaf813cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.015+0000 7fcafdc7e640 1 -- 172.21.15.16:0/569185303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf813d000 con 0x7fcaf815c820 2024-04-25T10:08:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.015+0000 7fcafdc7e640 1 --2- 172.21.15.16:0/569185303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf815c820 0x7fcaf814ebb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcae800cb10 tx=0x7fcae800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.015+0000 7fcaf5ffb640 1 -- 172.21.15.16:0/569185303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8017070 con 0x7fcaf815c820 2024-04-25T10:08:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.015+0000 7fcaf5ffb640 1 -- 172.21.15.16:0/569185303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae800ac10 con 0x7fcaf815c820 2024-04-25T10:08:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.015+0000 7fcaff6e1640 1 -- 172.21.15.16:0/569185303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf8140640 con 0x7fcaf815c820 2024-04-25T10:08:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.015+0000 7fcaf5ffb640 1 -- 172.21.15.16:0/569185303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8012400 con 0x7fcaf815c820 2024-04-25T10:08:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.015+0000 7fcaff6e1640 1 -- 172.21.15.16:0/569185303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf8140ee0 con 0x7fcaf815c820 2024-04-25T10:08:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.016+0000 7fcaff6e1640 1 -- 172.21.15.16:0/569185303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf806bb30 con 0x7fcaf815c820 2024-04-25T10:08:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.017+0000 7fcaf5ffb640 1 -- 172.21.15.16:0/569185303 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcae801a070 con 0x7fcaf815c820 2024-04-25T10:08:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.017+0000 7fcaf5ffb640 1 --2- 172.21.15.16:0/569185303 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcae406b4d0 0x7fcae406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.017+0000 7fcaf5ffb640 1 -- 172.21.15.16:0/569185303 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcae808a070 con 0x7fcaf815c820 2024-04-25T10:08:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.017+0000 7fcafd47d640 1 --2- 172.21.15.16:0/569185303 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcae406b4d0 0x7fcae406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.018+0000 7fcafd47d640 1 --2- 172.21.15.16:0/569185303 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcae406b4d0 0x7fcae406d990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0000f60 tx=0x7fcaf0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.021+0000 7fcaf5ffb640 1 -- 172.21.15.16:0/569185303 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcaf80ce6c0 con 0x7fcaf815c820 2024-04-25T10:08:33.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.237+0000 7fcaff6e1640 1 -- 172.21.15.16:0/569185303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcaf80cff00 con 0x7fcaf815c820 2024-04-25T10:08:33.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.237+0000 7fcaf5ffb640 1 -- 172.21.15.16:0/569185303 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fcae8058e60 con 0x7fcaf815c820 2024-04-25T10:08:33.243 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:08:33.243 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2270839865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2270839865},{"type":"v1","addr":"172.21.15.186:6836","nonce":2270839865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5647,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/925282579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":925282579},{"type":"v1","addr":"172.21.15.186:6837","nonce":925282579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2471714328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2471714328},{"type":"v1","addr":"172.21.15.16:6835","nonce":2471714328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:50.665436+0000","modified":"2024-04-25T10:08:07.326031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35141":{"gid":35141,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.16:6834/3311584521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3311584521},{"type":"v1","addr":"172.21.15.16:6834","nonce":3311584521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:08:33.244 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-04-25T10:08:33.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.239+0000 7fcaff6e1640 1 -- 172.21.15.16:0/569185303 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcae406b4d0 msgr2=0x7fcae406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.239+0000 7fcaff6e1640 1 --2- 172.21.15.16:0/569185303 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcae406b4d0 0x7fcae406d990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0000f60 tx=0x7fcaf0009040 comp rx=0 tx=0).stop 2024-04-25T10:08:33.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.239+0000 7fcaff6e1640 1 -- 172.21.15.16:0/569185303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf815c820 msgr2=0x7fcaf814ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.239+0000 7fcaff6e1640 1 --2- 172.21.15.16:0/569185303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf815c820 0x7fcaf814ebb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcae800cb10 tx=0x7fcae800a060 comp rx=0 tx=0).stop 2024-04-25T10:08:33.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.239+0000 7fcaff6e1640 1 -- 172.21.15.16:0/569185303 shutdown_connections 2024-04-25T10:08:33.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.239+0000 7fcaff6e1640 1 --2- 172.21.15.16:0/569185303 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcae406b4d0 0x7fcae406d990 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.239+0000 7fcaff6e1640 1 --2- 172.21.15.16:0/569185303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf815c820 0x7fcaf814ebb0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.239+0000 7fcaff6e1640 1 --2- 172.21.15.16:0/569185303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaf815bf00 0x7fcaf815e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.239+0000 7fcaff6e1640 1 --2- 172.21.15.16:0/569185303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcaf806e8f0 0x7fcaf813cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.239+0000 7fcaff6e1640 1 -- 172.21.15.16:0/569185303 >> 172.21.15.16:0/569185303 conn(0x7fcaf806aba0 msgr2=0x7fcaf80ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.239+0000 7fcaff6e1640 1 -- 172.21.15.16:0/569185303 shutdown_connections 2024-04-25T10:08:33.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.239+0000 7fcaff6e1640 1 -- 172.21.15.16:0/569185303 wait complete. 2024-04-25T10:08:33.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:08:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.353+0000 7f8eaa4ee640 1 Processor -- start 2024-04-25T10:08:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.353+0000 7f8eaa4ee640 1 -- start start 2024-04-25T10:08:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.354+0000 7f8eaa4ee640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ea415dca0 0x7f8ea40ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.354+0000 7f8eaa4ee640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8ea40d3b90 0x7f8ea40d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.354+0000 7f8eaa4ee640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8ea40cee80 0x7f8ea40d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.354+0000 7f8eaa4ee640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea40b6b20 con 0x7f8ea40cee80 2024-04-25T10:08:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.354+0000 7f8eaa4ee640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea40d4090 con 0x7f8ea40d3b90 2024-04-25T10:08:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.354+0000 7f8eaa4ee640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea40d4370 con 0x7f8ea415dca0 2024-04-25T10:08:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.354+0000 7f8ea8a8b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8ea40cee80 0x7f8ea40d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.354+0000 7f8ea8a8b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8ea40cee80 0x7f8ea40d0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37620/0 (socket says 172.21.15.16:37620) 2024-04-25T10:08:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.354+0000 7f8ea8a8b640 1 -- 172.21.15.16:0/3964542234 learned_addr learned my addr 172.21.15.16:0/3964542234 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.354+0000 7f8ea3fff640 1 --2- 172.21.15.16:0/3964542234 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ea415dca0 0x7f8ea40ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.354+0000 7f8ea37fe640 1 --2- 172.21.15.16:0/3964542234 >> v2:172.21.15.186:3300/0 conn(0x7f8ea40d3b90 0x7f8ea40d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.354+0000 7f8ea8a8b640 1 -- 172.21.15.16:0/3964542234 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ea415dca0 msgr2=0x7f8ea40ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.354+0000 7f8ea8a8b640 1 --2- 172.21.15.16:0/3964542234 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ea415dca0 0x7f8ea40ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.354+0000 7f8ea8a8b640 1 -- 172.21.15.16:0/3964542234 >> v2:172.21.15.186:3300/0 conn(0x7f8ea40d3b90 msgr2=0x7f8ea40d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.354+0000 7f8ea8a8b640 1 --2- 172.21.15.16:0/3964542234 >> v2:172.21.15.186:3300/0 conn(0x7f8ea40d3b90 0x7f8ea40d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.354+0000 7f8ea8a8b640 1 -- 172.21.15.16:0/3964542234 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea40d4650 con 0x7f8ea40cee80 2024-04-25T10:08:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.355+0000 7f8ea8a8b640 1 --2- 172.21.15.16:0/3964542234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea40cee80 0x7f8ea40d0f60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f8ea406d210 tx=0x7f8e9c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195b3d6d04cee42b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.355+0000 7f8ea2ffd640 1 -- 172.21.15.16:0/3964542234 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c036070 con 0x7f8ea40cee80 2024-04-25T10:08:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.355+0000 7f8ea2ffd640 1 -- 172.21.15.16:0/3964542234 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9c031600 con 0x7f8ea40cee80 2024-04-25T10:08:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.355+0000 7f8ea2ffd640 1 -- 172.21.15.16:0/3964542234 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c0319d0 con 0x7f8ea40cee80 2024-04-25T10:08:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.356+0000 7f8eaa4ee640 1 -- 172.21.15.16:0/3964542234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea40cee80 msgr2=0x7f8ea40d0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.356+0000 7f8eaa4ee640 1 --2- 172.21.15.16:0/3964542234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea40cee80 0x7f8ea40d0f60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f8ea406d210 tx=0x7f8e9c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.356+0000 7f8eaa4ee640 1 -- 172.21.15.16:0/3964542234 shutdown_connections 2024-04-25T10:08:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.356+0000 7f8eaa4ee640 1 --2- 172.21.15.16:0/3964542234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea40cee80 0x7f8ea40d0f60 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.356+0000 7f8eaa4ee640 1 --2- 172.21.15.16:0/3964542234 >> v2:172.21.15.186:3300/0 conn(0x7f8ea40d3b90 0x7f8ea40d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.356+0000 7f8eaa4ee640 1 --2- 172.21.15.16:0/3964542234 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ea415dca0 0x7f8ea40ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.356+0000 7f8eaa4ee640 1 -- 172.21.15.16:0/3964542234 >> 172.21.15.16:0/3964542234 conn(0x7f8ea40726c0 msgr2=0x7f8ea4072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.356+0000 7f8eaa4ee640 1 -- 172.21.15.16:0/3964542234 shutdown_connections 2024-04-25T10:08:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.356+0000 7f8eaa4ee640 1 -- 172.21.15.16:0/3964542234 wait complete. 2024-04-25T10:08:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.357+0000 7f8eaa4ee640 1 Processor -- start 2024-04-25T10:08:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.357+0000 7f8eaa4ee640 1 -- start start 2024-04-25T10:08:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.358+0000 7f8eaa4ee640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ea40cee80 0x7f8ea414df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.358+0000 7f8eaa4ee640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ea40d3b90 0x7f8ea40cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.358+0000 7f8eaa4ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea415dca0 0x7f8ea414f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.358+0000 7f8eaa4ee640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea415e200 con 0x7f8ea415dca0 2024-04-25T10:08:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.358+0000 7f8eaa4ee640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea415e080 con 0x7f8ea40cee80 2024-04-25T10:08:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.358+0000 7f8eaa4ee640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea40b2500 con 0x7f8ea40d3b90 2024-04-25T10:08:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.358+0000 7f8ea8a8b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea415dca0 0x7f8ea414f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.358+0000 7f8ea8a8b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea415dca0 0x7f8ea414f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37622/0 (socket says 172.21.15.16:37622) 2024-04-25T10:08:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.358+0000 7f8ea8a8b640 1 -- 172.21.15.16:0/3888171290 learned_addr learned my addr 172.21.15.16:0/3888171290 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.358+0000 7f8ea3fff640 1 --2- 172.21.15.16:0/3888171290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ea40cee80 0x7f8ea414df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.358+0000 7f8ea37fe640 1 --2- 172.21.15.16:0/3888171290 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ea40d3b90 0x7f8ea40cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.358+0000 7f8ea8a8b640 1 -- 172.21.15.16:0/3888171290 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ea40d3b90 msgr2=0x7f8ea40cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.358+0000 7f8ea8a8b640 1 --2- 172.21.15.16:0/3888171290 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ea40d3b90 0x7f8ea40cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.358+0000 7f8ea8a8b640 1 -- 172.21.15.16:0/3888171290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ea40cee80 msgr2=0x7f8ea414df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.358+0000 7f8ea8a8b640 1 --2- 172.21.15.16:0/3888171290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ea40cee80 0x7f8ea414df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.358+0000 7f8ea8a8b640 1 -- 172.21.15.16:0/3888171290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea4160410 con 0x7f8ea415dca0 2024-04-25T10:08:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.358+0000 7f8ea8a8b640 1 --2- 172.21.15.16:0/3888171290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea415dca0 0x7f8ea414f540 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c02ac90 tx=0x7f8e9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.359+0000 7f8ea0ff9640 1 -- 172.21.15.16:0/3888171290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c03f020 con 0x7f8ea415dca0 2024-04-25T10:08:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.359+0000 7f8ea0ff9640 1 -- 172.21.15.16:0/3888171290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9c037040 con 0x7f8ea415dca0 2024-04-25T10:08:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.359+0000 7f8ea0ff9640 1 -- 172.21.15.16:0/3888171290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c036070 con 0x7f8ea415dca0 2024-04-25T10:08:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.359+0000 7f8eaa4ee640 1 -- 172.21.15.16:0/3888171290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea415e4b0 con 0x7f8ea415dca0 2024-04-25T10:08:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.359+0000 7f8eaa4ee640 1 -- 172.21.15.16:0/3888171290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea415ea70 con 0x7f8ea415dca0 2024-04-25T10:08:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.360+0000 7f8ea0ff9640 1 -- 172.21.15.16:0/3888171290 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e9c042020 con 0x7f8ea415dca0 2024-04-25T10:08:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.360+0000 7f8ea0ff9640 1 --2- 172.21.15.16:0/3888171290 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8e9006b500 0x7f8e9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.361+0000 7f8ea0ff9640 1 -- 172.21.15.16:0/3888171290 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e9c0aa9a0 con 0x7f8ea415dca0 2024-04-25T10:08:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.361+0000 7f8ea3fff640 1 --2- 172.21.15.16:0/3888171290 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8e9006b500 0x7f8e9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.361+0000 7f8ea3fff640 1 --2- 172.21.15.16:0/3888171290 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8e9006b500 0x7f8e9006d9c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8e94006fd0 tx=0x7f8e94008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.361+0000 7f8ead1ef740 1 Processor -- start 2024-04-25T10:08:33.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.361+0000 7f8ead1ef740 1 -- start start 2024-04-25T10:08:33.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.362+0000 7f8ead1ef740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e6fe590 0x55e40e7006f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.362+0000 7f8ead1ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e73ac60 0x55e40e6fe970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.362+0000 7f8ead1ef740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e729050 0x55e40e7294d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.362+0000 7f8ead1ef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40e661c60 con 0x55e40e73ac60 2024-04-25T10:08:33.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.362+0000 7f8ead1ef740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40e600d80 con 0x55e40e6fe590 2024-04-25T10:08:33.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.362+0000 7f8ead1ef740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e40e65dd00 con 0x55e40e729050 2024-04-25T10:08:33.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.362+0000 7f8ea37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e73ac60 0x55e40e6fe970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.362+0000 7f8ea37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e73ac60 0x55e40e6fe970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37634/0 (socket says 172.21.15.16:37634) 2024-04-25T10:08:33.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.362+0000 7f8ea37fe640 1 -- 172.21.15.16:0/2361185025 learned_addr learned my addr 172.21.15.16:0/2361185025 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:33.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.362+0000 7f8ea3fff640 1 --2- 172.21.15.16:0/2361185025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e6fe590 0x55e40e7006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.362+0000 7f8ea8a8b640 1 --2- 172.21.15.16:0/2361185025 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e729050 0x55e40e7294d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.363+0000 7f8ea37fe640 1 -- 172.21.15.16:0/2361185025 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e729050 msgr2=0x55e40e7294d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.363+0000 7f8ea37fe640 1 --2- 172.21.15.16:0/2361185025 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e729050 0x55e40e7294d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.363+0000 7f8ea37fe640 1 -- 172.21.15.16:0/2361185025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e6fe590 msgr2=0x55e40e7006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.363+0000 7f8ea37fe640 1 --2- 172.21.15.16:0/2361185025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e6fe590 0x55e40e7006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.363+0000 7f8ea37fe640 1 -- 172.21.15.16:0/2361185025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e40e4427c0 con 0x55e40e73ac60 2024-04-25T10:08:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.363+0000 7f8ea37fe640 1 --2- 172.21.15.16:0/2361185025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e73ac60 0x55e40e6fe970 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800c9f0 tx=0x7f8e9800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb8f90b92eaecba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.363+0000 7f8e89ffb640 1 -- 172.21.15.16:0/2361185025 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9800ea80 con 0x55e40e73ac60 2024-04-25T10:08:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.363+0000 7f8e89ffb640 1 -- 172.21.15.16:0/2361185025 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9800ec20 con 0x55e40e73ac60 2024-04-25T10:08:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.363+0000 7f8e89ffb640 1 -- 172.21.15.16:0/2361185025 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98004030 con 0x55e40e73ac60 2024-04-25T10:08:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.363+0000 7f8ead1ef740 1 -- 172.21.15.16:0/2361185025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e73ac60 msgr2=0x55e40e6fe970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.363+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/2361185025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e73ac60 0x55e40e6fe970 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800c9f0 tx=0x7f8e9800ceb0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.364+0000 7f8ead1ef740 1 -- 172.21.15.16:0/2361185025 shutdown_connections 2024-04-25T10:08:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.364+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/2361185025 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e729050 0x55e40e7294d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.364+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/2361185025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e73ac60 0x55e40e6fe970 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.364+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/2361185025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e6fe590 0x55e40e7006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.364+0000 7f8ead1ef740 1 -- 172.21.15.16:0/2361185025 >> 172.21.15.16:0/2361185025 conn(0x55e40e6c66e0 msgr2=0x55e40e744050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.364+0000 7f8ead1ef740 1 -- 172.21.15.16:0/2361185025 shutdown_connections 2024-04-25T10:08:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.364+0000 7f8ead1ef740 1 -- 172.21.15.16:0/2361185025 wait complete. 2024-04-25T10:08:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.364+0000 7f8ead1ef740 1 -- start start 2024-04-25T10:08:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.364+0000 7f8ead1ef740 1 Processor -- start 2024-04-25T10:08:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.365+0000 7f8ead1ef740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e6fe590 0x55e40e712160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.365+0000 7f8ead1ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e729050 0x55e40e73ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.365+0000 7f8ead1ef740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e73ac60 0x55e40e727480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.365+0000 7f8ead1ef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40e5fe840 con 0x55e40e729050 2024-04-25T10:08:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.365+0000 7f8ead1ef740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40e726ba0 con 0x55e40e6fe590 2024-04-25T10:08:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.365+0000 7f8ead1ef740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e40e7126a0 con 0x55e40e73ac60 2024-04-25T10:08:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.365+0000 7f8ead1ef740 10 client.0 _ll_register_callbacks cb 0x55e40e618980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.365+0000 7f8ea37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e729050 0x55e40e73ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.365+0000 7f8ea37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e729050 0x55e40e73ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37638/0 (socket says 172.21.15.16:37638) 2024-04-25T10:08:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.365+0000 7f8ea37fe640 1 -- 172.21.15.16:0/618422317 learned_addr learned my addr 172.21.15.16:0/618422317 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.365+0000 7f8ea3fff640 1 --2- 172.21.15.16:0/618422317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e6fe590 0x55e40e712160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.365+0000 7f8ea8a8b640 1 --2- 172.21.15.16:0/618422317 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e73ac60 0x55e40e727480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8ea37fe640 1 -- 172.21.15.16:0/618422317 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e73ac60 msgr2=0x55e40e727480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8ea37fe640 1 --2- 172.21.15.16:0/618422317 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e73ac60 0x55e40e727480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8ea37fe640 1 -- 172.21.15.16:0/618422317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e6fe590 msgr2=0x55e40e712160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8ea37fe640 1 --2- 172.21.15.16:0/618422317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e6fe590 0x55e40e712160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8ea37fe640 1 -- 172.21.15.16:0/618422317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e40e4a2f50 con 0x55e40e729050 2024-04-25T10:08:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8ea37fe640 1 --2- 172.21.15.16:0/618422317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e729050 0x55e40e73ec70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8e98004000 tx=0x7f8e98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e739c4dd68e63a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8e82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:08:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8e82ffd640 1 -- 172.21.15.16:0/618422317 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98003a20 con 0x55e40e729050 2024-04-25T10:08:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8e82ffd640 1 -- 172.21.15.16:0/618422317 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e98003bc0 con 0x55e40e729050 2024-04-25T10:08:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8ead1ef740 1 -- 172.21.15.16:0/618422317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e9800ec20 con 0x55e40e729050 2024-04-25T10:08:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8e82ffd640 1 -- 172.21.15.16:0/618422317 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e9802e050 con 0x55e40e729050 2024-04-25T10:08:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8ead1ef740 10 client.6348 fetch_fsmap learned FSMap version 28 2024-04-25T10:08:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8ead1ef740 1 -- 172.21.15.16:0/618422317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55e40e3360b0 con 0x55e40e729050 2024-04-25T10:08:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8e82ffd640 1 -- 172.21.15.16:0/618422317 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9800ec20 con 0x55e40e729050 2024-04-25T10:08:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8e82ffd640 1 -- 172.21.15.16:0/618422317 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8e98031020 con 0x55e40e729050 2024-04-25T10:08:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8ead1ef740 10 client.6348 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:08:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8ead1ef740 10 client.6348 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:08:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8ead1ef740 20 client.6348 populate_metadata read hostname 'smithi016' 2024-04-25T10:08:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/618422317 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55e40e75f460 0x55e40e761850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8ead1ef740 4 client.6348 mds_command: new command op to 35141 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.366+0000 7f8ead1ef740 1 -- 172.21.15.16:0/618422317 --> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e40e621e60 con 0x55e40e75f460 2024-04-25T10:08:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.367+0000 7f8ea3fff640 1 --2- 172.21.15.16:0/618422317 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55e40e75f460 0x55e40e761850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.367+0000 7f8ea3fff640 1 --2- 172.21.15.16:0/618422317 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55e40e75f460 0x55e40e761850 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b413974194130356 server_cookie=d3bde2acdfe576b1 in_seq=0 out_seq=0 2024-04-25T10:08:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.367+0000 7f8e82ffd640 10 client.6348 ms_handle_connect on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.367+0000 7f8e82ffd640 1 -- 172.21.15.16:0/618422317 <== mds.0 v2:172.21.15.16:6832/3311584521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e40e621e60 con 0x55e40e75f460 2024-04-25T10:08:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.367+0000 7f8e82ffd640 10 client.6348 handle_command_reply: tid=0 2024-04-25T10:08:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.367+0000 7f8ead1ef740 1 client.6348 shutdown 2024-04-25T10:08:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.368+0000 7f8ead1ef740 1 -- 172.21.15.16:0/618422317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e729050 msgr2=0x55e40e73ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.368+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/618422317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e729050 0x55e40e73ec70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8e98004000 tx=0x7f8e98004730 comp rx=0 tx=0).stop 2024-04-25T10:08:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.368+0000 7f8ead1ef740 1 -- 172.21.15.16:0/618422317 shutdown_connections 2024-04-25T10:08:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.368+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/618422317 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55e40e75f460 0x55e40e761850 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.368+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/618422317 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e73ac60 0x55e40e727480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.368+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/618422317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e729050 0x55e40e73ec70 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.368+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/618422317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e6fe590 0x55e40e712160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.368+0000 7f8e82ffd640 0 client.6348 ms_handle_reset on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.368+0000 7f8ead1ef740 1 -- 172.21.15.16:0/618422317 >> 172.21.15.16:0/618422317 conn(0x55e40e6c66e0 msgr2=0x55e40e7241c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.368+0000 7f8ead1ef740 1 -- 172.21.15.16:0/618422317 shutdown_connections 2024-04-25T10:08:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.368+0000 7f8ead1ef740 1 -- 172.21.15.16:0/618422317 wait complete. 2024-04-25T10:08:33.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.368+0000 7f8ead1ef740 20 client.6348 trim_cache size 0 max 16384 2024-04-25T10:08:33.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.379+0000 7f8ead1ef740 1 Processor -- start 2024-04-25T10:08:33.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.380+0000 7f8ead1ef740 1 -- start start 2024-04-25T10:08:33.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.380+0000 7f8ead1ef740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e6fe590 0x55e40e724510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.380+0000 7f8ead1ef740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e729050 0x55e40e726c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.380+0000 7f8ead1ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e73ac60 0x55e40e765aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.380+0000 7f8ead1ef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea41ff1b0 con 0x55e40e73ac60 2024-04-25T10:08:33.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.380+0000 7f8ead1ef740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea41fe650 con 0x55e40e729050 2024-04-25T10:08:33.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.380+0000 7f8ead1ef740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea41febb0 con 0x55e40e6fe590 2024-04-25T10:08:33.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.380+0000 7f8ea8a8b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e73ac60 0x55e40e765aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.380+0000 7f8ea8a8b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e73ac60 0x55e40e765aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37654/0 (socket says 172.21.15.16:37654) 2024-04-25T10:08:33.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.380+0000 7f8ea8a8b640 1 -- 172.21.15.16:0/3689992352 learned_addr learned my addr 172.21.15.16:0/3689992352 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:33.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8ea37fe640 1 --2- 172.21.15.16:0/3689992352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e729050 0x55e40e726c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.380+0000 7f8ea3fff640 1 --2- 172.21.15.16:0/3689992352 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e6fe590 0x55e40e724510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8ea8a8b640 1 -- 172.21.15.16:0/3689992352 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e6fe590 msgr2=0x55e40e724510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8ea8a8b640 1 --2- 172.21.15.16:0/3689992352 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e6fe590 0x55e40e724510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8ea8a8b640 1 -- 172.21.15.16:0/3689992352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e729050 msgr2=0x55e40e726c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8ea8a8b640 1 --2- 172.21.15.16:0/3689992352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e729050 0x55e40e726c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8ea8a8b640 1 -- 172.21.15.16:0/3689992352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e40e75da00 con 0x55e40e73ac60 2024-04-25T10:08:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8ea37fe640 1 --2- 172.21.15.16:0/3689992352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e729050 0x55e40e726c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8ea3fff640 1 --2- 172.21.15.16:0/3689992352 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e6fe590 0x55e40e724510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8ea8a8b640 1 --2- 172.21.15.16:0/3689992352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e73ac60 0x55e40e765aa0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c02a6e0 tx=0x7f8e9c03d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c50394f3c85a4c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8e897fa640 1 -- 172.21.15.16:0/3689992352 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c0a7070 con 0x55e40e73ac60 2024-04-25T10:08:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8e897fa640 1 -- 172.21.15.16:0/3689992352 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9c031600 con 0x55e40e73ac60 2024-04-25T10:08:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8e897fa640 1 -- 172.21.15.16:0/3689992352 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c08e070 con 0x55e40e73ac60 2024-04-25T10:08:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8ead1ef740 1 -- 172.21.15.16:0/3689992352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e73ac60 msgr2=0x55e40e765aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/3689992352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e73ac60 0x55e40e765aa0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c02a6e0 tx=0x7f8e9c03d040 comp rx=0 tx=0).stop 2024-04-25T10:08:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8ead1ef740 1 -- 172.21.15.16:0/3689992352 shutdown_connections 2024-04-25T10:08:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/3689992352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e73ac60 0x55e40e765aa0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/3689992352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e729050 0x55e40e726c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/3689992352 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e6fe590 0x55e40e724510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8ead1ef740 1 -- 172.21.15.16:0/3689992352 >> 172.21.15.16:0/3689992352 conn(0x55e40e6c66e0 msgr2=0x55e40e73f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8ead1ef740 1 -- 172.21.15.16:0/3689992352 shutdown_connections 2024-04-25T10:08:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.381+0000 7f8ead1ef740 1 -- 172.21.15.16:0/3689992352 wait complete. 2024-04-25T10:08:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.382+0000 7f8ead1ef740 1 -- start start 2024-04-25T10:08:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.382+0000 7f8ead1ef740 1 Processor -- start 2024-04-25T10:08:33.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ead1ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e6fe590 0x55e40e714730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ead1ef740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e729050 0x55e40e7242b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ead1ef740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e73ac60 0x55e40e7247f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ead1ef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40e715890 con 0x55e40e6fe590 2024-04-25T10:08:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ead1ef740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40e713b30 con 0x55e40e73ac60 2024-04-25T10:08:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ead1ef740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e40e75d730 con 0x55e40e729050 2024-04-25T10:08:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ead1ef740 10 client.0 _ll_register_callbacks cb 0x7f8e9800c1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ea3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e6fe590 0x55e40e714730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ea3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e6fe590 0x55e40e714730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37658/0 (socket says 172.21.15.16:37658) 2024-04-25T10:08:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ea3fff640 1 -- 172.21.15.16:0/1591137652 learned_addr learned my addr 172.21.15.16:0/1591137652 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ea37fe640 1 --2- 172.21.15.16:0/1591137652 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e729050 0x55e40e7242b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ea8a8b640 1 --2- 172.21.15.16:0/1591137652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e73ac60 0x55e40e7247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ea3fff640 1 -- 172.21.15.16:0/1591137652 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e729050 msgr2=0x55e40e7242b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ea3fff640 1 --2- 172.21.15.16:0/1591137652 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e729050 0x55e40e7242b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ea3fff640 1 -- 172.21.15.16:0/1591137652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e73ac60 msgr2=0x55e40e7247f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ea3fff640 1 --2- 172.21.15.16:0/1591137652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e73ac60 0x55e40e7247f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ea3fff640 1 -- 172.21.15.16:0/1591137652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e40e725470 con 0x55e40e6fe590 2024-04-25T10:08:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ea8a8b640 1 --2- 172.21.15.16:0/1591137652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e73ac60 0x55e40e7247f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8ea3fff640 1 --2- 172.21.15.16:0/1591137652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e6fe590 0x55e40e714730 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400b1d0 tx=0x7f8e94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02c6b2dc7b29943 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.383+0000 7f8e82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:08:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.384+0000 7f8e82ffd640 1 -- 172.21.15.16:0/1591137652 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9401b040 con 0x55e40e6fe590 2024-04-25T10:08:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.384+0000 7f8e82ffd640 1 -- 172.21.15.16:0/1591137652 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e94014ac0 con 0x55e40e6fe590 2024-04-25T10:08:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.384+0000 7f8ead1ef740 1 -- 172.21.15.16:0/1591137652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e9c031600 con 0x55e40e6fe590 2024-04-25T10:08:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.384+0000 7f8e82ffd640 1 -- 172.21.15.16:0/1591137652 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e94033050 con 0x55e40e6fe590 2024-04-25T10:08:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.384+0000 7f8ead1ef740 10 client.6354 fetch_fsmap learned FSMap version 28 2024-04-25T10:08:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.384+0000 7f8ead1ef740 1 -- 172.21.15.16:0/1591137652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55e40e725710 con 0x55e40e6fe590 2024-04-25T10:08:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.384+0000 7f8e82ffd640 1 -- 172.21.15.16:0/1591137652 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9c031600 con 0x55e40e6fe590 2024-04-25T10:08:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.384+0000 7f8e82ffd640 1 -- 172.21.15.16:0/1591137652 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8e94036020 con 0x55e40e6fe590 2024-04-25T10:08:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.384+0000 7f8ead1ef740 10 client.6354 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:08:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.384+0000 7f8ead1ef740 10 client.6354 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:08:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.384+0000 7f8ead1ef740 20 client.6354 populate_metadata read hostname 'smithi016' 2024-04-25T10:08:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.384+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/1591137652 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55e40e7aea00 0x55e40e7230f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.384+0000 7f8ead1ef740 4 client.6354 mds_command: new command op to 35141 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:08:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.384+0000 7f8ead1ef740 1 -- 172.21.15.16:0/1591137652 --> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55e40e66f1a0 con 0x55e40e7aea00 2024-04-25T10:08:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.384+0000 7f8ea37fe640 1 --2- 172.21.15.16:0/1591137652 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55e40e7aea00 0x55e40e7230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.384+0000 7f8ea37fe640 1 --2- 172.21.15.16:0/1591137652 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55e40e7aea00 0x55e40e7230f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee3ad8ecfa0a943 server_cookie=7d81024d4b7a1be2 in_seq=0 out_seq=0 2024-04-25T10:08:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.384+0000 7f8e82ffd640 10 client.6354 ms_handle_connect on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8e82ffd640 1 -- 172.21.15.16:0/1591137652 <== mds.0 v2:172.21.15.16:6832/3311584521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0x55e40e66f1a0 con 0x55e40e7aea00 2024-04-25T10:08:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8e82ffd640 10 client.6354 handle_command_reply: tid=0 2024-04-25T10:08:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8ead1ef740 1 client.6354 shutdown 2024-04-25T10:08:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8ead1ef740 1 -- 172.21.15.16:0/1591137652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e6fe590 msgr2=0x55e40e714730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/1591137652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e6fe590 0x55e40e714730 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400b1d0 tx=0x7f8e94004730 comp rx=0 tx=0).stop 2024-04-25T10:08:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8ead1ef740 1 -- 172.21.15.16:0/1591137652 shutdown_connections 2024-04-25T10:08:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/1591137652 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55e40e7aea00 0x55e40e7230f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/1591137652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e40e73ac60 0x55e40e7247f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/1591137652 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e40e729050 0x55e40e7242b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8ead1ef740 1 --2- 172.21.15.16:0/1591137652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40e6fe590 0x55e40e714730 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8e82ffd640 0 client.6354 ms_handle_reset on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:33.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8ead1ef740 1 -- 172.21.15.16:0/1591137652 >> 172.21.15.16:0/1591137652 conn(0x55e40e6c66e0 msgr2=0x55e40e6f8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8ead1ef740 1 -- 172.21.15.16:0/1591137652 shutdown_connections 2024-04-25T10:08:33.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8ead1ef740 1 -- 172.21.15.16:0/1591137652 wait complete. 2024-04-25T10:08:33.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8ead1ef740 20 client.6354 trim_cache size 0 max 16384 2024-04-25T10:08:33.397 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:08:33.397 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-04-25T10:08:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:08:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-04-25T10:08:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4125907959, 2024-04-25T10:08:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511627777, 2024-04-25T10:08:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/dir_test/file" 2024-04-25T10:08:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:08:33.398 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-04-25T10:08:33.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8eaa4ee640 1 -- 172.21.15.16:0/3888171290 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8e9006b500 msgr2=0x7f8e9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8eaa4ee640 1 --2- 172.21.15.16:0/3888171290 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8e9006b500 0x7f8e9006d9c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8e94006fd0 tx=0x7f8e94008040 comp rx=0 tx=0).stop 2024-04-25T10:08:33.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8eaa4ee640 1 -- 172.21.15.16:0/3888171290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea415dca0 msgr2=0x7f8ea414f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.385+0000 7f8eaa4ee640 1 --2- 172.21.15.16:0/3888171290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea415dca0 0x7f8ea414f540 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c02ac90 tx=0x7f8e9c004a40 comp rx=0 tx=0).stop 2024-04-25T10:08:33.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.386+0000 7f8eaa4ee640 1 -- 172.21.15.16:0/3888171290 shutdown_connections 2024-04-25T10:08:33.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.386+0000 7f8eaa4ee640 1 --2- 172.21.15.16:0/3888171290 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8e9006b500 0x7f8e9006d9c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.386+0000 7f8eaa4ee640 1 --2- 172.21.15.16:0/3888171290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea415dca0 0x7f8ea414f540 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.386+0000 7f8eaa4ee640 1 --2- 172.21.15.16:0/3888171290 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ea40d3b90 0x7f8ea40cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.386+0000 7f8eaa4ee640 1 --2- 172.21.15.16:0/3888171290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ea40cee80 0x7f8ea414df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.386+0000 7f8eaa4ee640 1 -- 172.21.15.16:0/3888171290 >> 172.21.15.16:0/3888171290 conn(0x7f8ea40726c0 msgr2=0x7f8ea40b8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.386+0000 7f8eaa4ee640 1 -- 172.21.15.16:0/3888171290 shutdown_connections 2024-04-25T10:08:33.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.386+0000 7f8eaa4ee640 1 -- 172.21.15.16:0/3888171290 wait complete. 2024-04-25T10:08:33.402 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:08:33.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.495+0000 7f0ae6cfc640 1 Processor -- start 2024-04-25T10:08:33.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.495+0000 7f0ae6cfc640 1 -- start start 2024-04-25T10:08:33.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.496+0000 7f0ae6cfc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ae006bef0 0x7f0ae006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.496+0000 7f0ae6cfc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ae006c2d0 0x7f0ae015c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.496+0000 7f0ae6cfc640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0ae00acd80 0x7f0ae00ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.496+0000 7f0ae6cfc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ae006eb20 con 0x7f0ae006bef0 2024-04-25T10:08:33.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.496+0000 7f0ae6cfc640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ae015ce50 con 0x7f0ae00acd80 2024-04-25T10:08:33.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.496+0000 7f0ae6cfc640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae006dc70 con 0x7f0ae006c2d0 2024-04-25T10:08:33.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.496+0000 7f0ae4a98640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ae006bef0 0x7f0ae006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.496+0000 7f0ae4a98640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ae006bef0 0x7f0ae006efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37664/0 (socket says 172.21.15.16:37664) 2024-04-25T10:08:33.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.496+0000 7f0ae4a98640 1 -- 172.21.15.16:0/997980408 learned_addr learned my addr 172.21.15.16:0/997980408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:33.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.496+0000 7f0adffff640 1 --2- 172.21.15.16:0/997980408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ae006c2d0 0x7f0ae015c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.496+0000 7f0ae5299640 1 --2- 172.21.15.16:0/997980408 >> v2:172.21.15.186:3300/0 conn(0x7f0ae00acd80 0x7f0ae00ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.496+0000 7f0ae4a98640 1 -- 172.21.15.16:0/997980408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ae006c2d0 msgr2=0x7f0ae015c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.496+0000 7f0ae4a98640 1 --2- 172.21.15.16:0/997980408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ae006c2d0 0x7f0ae015c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.496+0000 7f0ae4a98640 1 -- 172.21.15.16:0/997980408 >> v2:172.21.15.186:3300/0 conn(0x7f0ae00acd80 msgr2=0x7f0ae00ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.496+0000 7f0ae4a98640 1 --2- 172.21.15.16:0/997980408 >> v2:172.21.15.186:3300/0 conn(0x7f0ae00acd80 0x7f0ae00ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.496+0000 7f0ae4a98640 1 -- 172.21.15.16:0/997980408 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae006df10 con 0x7f0ae006bef0 2024-04-25T10:08:33.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.496+0000 7f0ae5299640 1 --2- 172.21.15.16:0/997980408 >> v2:172.21.15.186:3300/0 conn(0x7f0ae00acd80 0x7f0ae00ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:33.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.497+0000 7f0ae4a98640 1 --2- 172.21.15.16:0/997980408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae006bef0 0x7f0ae006efb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00d0b20 tx=0x7f0ad402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadf104e09d55d02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.497+0000 7f0adf7fe640 1 -- 172.21.15.16:0/997980408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4037070 con 0x7f0ae006bef0 2024-04-25T10:08:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.497+0000 7f0adf7fe640 1 -- 172.21.15.16:0/997980408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad4032440 con 0x7f0ae006bef0 2024-04-25T10:08:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.497+0000 7f0adf7fe640 1 -- 172.21.15.16:0/997980408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4032810 con 0x7f0ae006bef0 2024-04-25T10:08:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.498+0000 7f0ae6cfc640 1 -- 172.21.15.16:0/997980408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae006bef0 msgr2=0x7f0ae006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.498+0000 7f0ae6cfc640 1 --2- 172.21.15.16:0/997980408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae006bef0 0x7f0ae006efb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00d0b20 tx=0x7f0ad402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.498+0000 7f0ae6cfc640 1 -- 172.21.15.16:0/997980408 shutdown_connections 2024-04-25T10:08:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.498+0000 7f0ae6cfc640 1 --2- 172.21.15.16:0/997980408 >> v2:172.21.15.186:3300/0 conn(0x7f0ae00acd80 0x7f0ae00ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.498+0000 7f0ae6cfc640 1 --2- 172.21.15.16:0/997980408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ae006c2d0 0x7f0ae015c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.498+0000 7f0ae6cfc640 1 --2- 172.21.15.16:0/997980408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae006bef0 0x7f0ae006efb0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.498+0000 7f0ae6cfc640 1 -- 172.21.15.16:0/997980408 >> 172.21.15.16:0/997980408 conn(0x7f0ae00b2220 msgr2=0x7f0ae00cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.498+0000 7f0ae6cfc640 1 -- 172.21.15.16:0/997980408 shutdown_connections 2024-04-25T10:08:33.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.498+0000 7f0ae6cfc640 1 -- 172.21.15.16:0/997980408 wait complete. 2024-04-25T10:08:33.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.499+0000 7f0ae6cfc640 1 Processor -- start 2024-04-25T10:08:33.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.499+0000 7f0ae6cfc640 1 -- start start 2024-04-25T10:08:33.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.500+0000 7f0ae6cfc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae006bef0 0x7f0ae01f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.500+0000 7f0ae6cfc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ae006c2d0 0x7f0ae0158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.500+0000 7f0ae6cfc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ae00acd80 0x7f0ae0146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.500+0000 7f0ae6cfc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae006e7b0 con 0x7f0ae006bef0 2024-04-25T10:08:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.500+0000 7f0ae6cfc640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae006e4d0 con 0x7f0ae006c2d0 2024-04-25T10:08:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.500+0000 7f0ae6cfc640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae00ada00 con 0x7f0ae00acd80 2024-04-25T10:08:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.500+0000 7f0ae4a98640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae006bef0 0x7f0ae01f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.500+0000 7f0ae4a98640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae006bef0 0x7f0ae01f55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37666/0 (socket says 172.21.15.16:37666) 2024-04-25T10:08:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.500+0000 7f0ae4a98640 1 -- 172.21.15.16:0/2224053077 learned_addr learned my addr 172.21.15.16:0/2224053077 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.500+0000 7f0ae4a98640 1 -- 172.21.15.16:0/2224053077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ae00acd80 msgr2=0x7f0ae0146300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.500+0000 7f0ae4a98640 1 --2- 172.21.15.16:0/2224053077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ae00acd80 0x7f0ae0146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.500+0000 7f0ae4a98640 1 -- 172.21.15.16:0/2224053077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ae006c2d0 msgr2=0x7f0ae0158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.500+0000 7f0ae4a98640 1 --2- 172.21.15.16:0/2224053077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ae006c2d0 0x7f0ae0158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.500+0000 7f0ae4a98640 1 -- 172.21.15.16:0/2224053077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae0146a50 con 0x7f0ae006bef0 2024-04-25T10:08:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.500+0000 7f0ae4a98640 1 --2- 172.21.15.16:0/2224053077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae006bef0 0x7f0ae01f55d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4000c00 tx=0x7f0ad40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.501+0000 7f0add7fa640 1 -- 172.21.15.16:0/2224053077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4041070 con 0x7f0ae006bef0 2024-04-25T10:08:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.501+0000 7f0add7fa640 1 -- 172.21.15.16:0/2224053077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad403c870 con 0x7f0ae006bef0 2024-04-25T10:08:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.501+0000 7f0ae6cfc640 1 -- 172.21.15.16:0/2224053077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae0146d30 con 0x7f0ae006bef0 2024-04-25T10:08:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.501+0000 7f0add7fa640 1 -- 172.21.15.16:0/2224053077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4037040 con 0x7f0ae006bef0 2024-04-25T10:08:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.501+0000 7f0ae6cfc640 1 -- 172.21.15.16:0/2224053077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae01f4dd0 con 0x7f0ae006bef0 2024-04-25T10:08:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.502+0000 7f0ae6cfc640 1 -- 172.21.15.16:0/2224053077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae00749b0 con 0x7f0ae006bef0 2024-04-25T10:08:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.502+0000 7f0add7fa640 1 -- 172.21.15.16:0/2224053077 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ad4044070 con 0x7f0ae006bef0 2024-04-25T10:08:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.503+0000 7f0add7fa640 1 --2- 172.21.15.16:0/2224053077 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0ac406b4d0 0x7f0ac406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.503+0000 7f0add7fa640 1 -- 172.21.15.16:0/2224053077 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ad40aab30 con 0x7f0ae006bef0 2024-04-25T10:08:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.503+0000 7f0adffff640 1 --2- 172.21.15.16:0/2224053077 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0ac406b4d0 0x7f0ac406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.504+0000 7f0adffff640 1 --2- 172.21.15.16:0/2224053077 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0ac406b4d0 0x7f0ac406d990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0acc006a90 tx=0x7f0acc006a20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.507+0000 7f0add7fa640 1 -- 172.21.15.16:0/2224053077 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ad4079a40 con 0x7f0ae006bef0 2024-04-25T10:08:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.725+0000 7f0ae6cfc640 1 -- 172.21.15.16:0/2224053077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0ae00d0b70 con 0x7f0ae006bef0 2024-04-25T10:08:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.726+0000 7f0add7fa640 1 -- 172.21.15.16:0/2224053077 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f0ad4079340 con 0x7f0ae006bef0 2024-04-25T10:08:33.732 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:08:33.732 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:08:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.728+0000 7f0ae6cfc640 1 -- 172.21.15.16:0/2224053077 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0ac406b4d0 msgr2=0x7f0ac406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.728+0000 7f0ae6cfc640 1 --2- 172.21.15.16:0/2224053077 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0ac406b4d0 0x7f0ac406d990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0acc006a90 tx=0x7f0acc006a20 comp rx=0 tx=0).stop 2024-04-25T10:08:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.728+0000 7f0ae6cfc640 1 -- 172.21.15.16:0/2224053077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae006bef0 msgr2=0x7f0ae01f55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.728+0000 7f0ae6cfc640 1 --2- 172.21.15.16:0/2224053077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae006bef0 0x7f0ae01f55d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4000c00 tx=0x7f0ad40046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.728+0000 7f0ae6cfc640 1 -- 172.21.15.16:0/2224053077 shutdown_connections 2024-04-25T10:08:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.728+0000 7f0ae6cfc640 1 --2- 172.21.15.16:0/2224053077 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0ac406b4d0 0x7f0ac406d990 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.728+0000 7f0ae6cfc640 1 --2- 172.21.15.16:0/2224053077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ae00acd80 0x7f0ae0146300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.728+0000 7f0ae6cfc640 1 --2- 172.21.15.16:0/2224053077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ae006c2d0 0x7f0ae0158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.728+0000 7f0ae6cfc640 1 --2- 172.21.15.16:0/2224053077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ae006bef0 0x7f0ae01f55d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.728+0000 7f0ae6cfc640 1 -- 172.21.15.16:0/2224053077 >> 172.21.15.16:0/2224053077 conn(0x7f0ae00b2220 msgr2=0x7f0ae015fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.728+0000 7f0ae6cfc640 1 -- 172.21.15.16:0/2224053077 shutdown_connections 2024-04-25T10:08:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.728+0000 7f0ae6cfc640 1 -- 172.21.15.16:0/2224053077 wait complete. 2024-04-25T10:08:33.744 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:08:33.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:33.745 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-04-25T10:08:33.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.844+0000 7f1b189ab640 1 Processor -- start 2024-04-25T10:08:33.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.844+0000 7f1b189ab640 1 -- start start 2024-04-25T10:08:33.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.845+0000 7f1b189ab640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b141610b0 0x7f1b140b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.845+0000 7f1b189ab640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b14162040 con 0x7f1b141618d0 2024-04-25T10:08:33.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.845+0000 7f1b189ab640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b14162320 con 0x7f1b140cef40 2024-04-25T10:08:33.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.845+0000 7f1b189ab640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b140d1ea0 con 0x7f1b141610b0 2024-04-25T10:08:33.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.845+0000 7f1b1259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b141610b0 0x7f1b140b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.845+0000 7f1b0bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1b141618d0 0x7f1b140ce830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36762/0 (socket says 172.21.15.16:36762) 2024-04-25T10:08:33.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.845+0000 7f1b1259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b141610b0 0x7f1b140b9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37668/0 (socket says 172.21.15.16:37668) 2024-04-25T10:08:33.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.845+0000 7f1b0bfff640 1 -- 172.21.15.16:0/2825631601 learned_addr learned my addr 172.21.15.16:0/2825631601 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:08:33.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.845+0000 7f1b1259c640 1 -- 172.21.15.16:0/2825631601 >> v1:172.21.15.186:6789/0 conn(0x7f1b140cef40 legacy=0x7f1b140d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.845+0000 7f1b1259c640 1 -- 172.21.15.16:0/2825631601 >> v1:172.21.15.16:6789/0 conn(0x7f1b141618d0 legacy=0x7f1b140ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.845+0000 7f1b1259c640 1 -- 172.21.15.16:0/2825631601 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b14162600 con 0x7f1b141610b0 2024-04-25T10:08:33.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.845+0000 7f1b11d9b640 1 -- 172.21.15.16:0/2825631601 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3357927892 0 0) 0x7f1b14162040 con 0x7f1b141618d0 2024-04-25T10:08:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.846+0000 7f1b1259c640 1 --2- 172.21.15.16:0/2825631601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b141610b0 0x7f1b140b9580 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f1b040097a0 tx=0x7f1b0402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a92ac93dcceaa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.846+0000 7f1b11d9b640 1 -- 172.21.15.16:0/2825631601 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04037070 con 0x7f1b141610b0 2024-04-25T10:08:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.846+0000 7f1b11d9b640 1 -- 172.21.15.16:0/2825631601 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0402ae50 con 0x7f1b141610b0 2024-04-25T10:08:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.846+0000 7f1b11d9b640 1 -- 172.21.15.16:0/2825631601 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04032660 con 0x7f1b141610b0 2024-04-25T10:08:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.846+0000 7f1b189ab640 1 -- 172.21.15.16:0/2825631601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b141610b0 msgr2=0x7f1b140b9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.846+0000 7f1b189ab640 1 --2- 172.21.15.16:0/2825631601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b141610b0 0x7f1b140b9580 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f1b040097a0 tx=0x7f1b0402a140 comp rx=0 tx=0).stop 2024-04-25T10:08:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.847+0000 7f1b189ab640 1 -- 172.21.15.16:0/2825631601 shutdown_connections 2024-04-25T10:08:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.847+0000 7f1b189ab640 1 --2- 172.21.15.16:0/2825631601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b141610b0 0x7f1b140b9580 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.847+0000 7f1b189ab640 1 -- 172.21.15.16:0/2825631601 >> 172.21.15.16:0/2825631601 conn(0x7f1b140b5570 msgr2=0x7f1b140b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.847+0000 7f1b189ab640 1 -- 172.21.15.16:0/2825631601 shutdown_connections 2024-04-25T10:08:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.847+0000 7f1b189ab640 1 -- 172.21.15.16:0/2825631601 wait complete. 2024-04-25T10:08:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.848+0000 7f1b189ab640 1 Processor -- start 2024-04-25T10:08:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.848+0000 7f1b189ab640 1 -- start start 2024-04-25T10:08:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b189ab640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b140cef40 0x7f1b141f0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b189ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b141610b0 0x7f1b141f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b189ab640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b141618d0 0x7f1b14157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b189ab640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b14161d10 con 0x7f1b141610b0 2024-04-25T10:08:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b189ab640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b140b6c00 con 0x7f1b141618d0 2024-04-25T10:08:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b189ab640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b14161490 con 0x7f1b140cef40 2024-04-25T10:08:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b141610b0 0x7f1b141f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b141610b0 0x7f1b141f43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37678/0 (socket says 172.21.15.16:37678) 2024-04-25T10:08:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b0bfff640 1 -- 172.21.15.16:0/3329788271 learned_addr learned my addr 172.21.15.16:0/3329788271 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b12d9d640 1 --2- 172.21.15.16:0/3329788271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b141618d0 0x7f1b14157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b1259c640 1 --2- 172.21.15.16:0/3329788271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b140cef40 0x7f1b141f0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b0bfff640 1 -- 172.21.15.16:0/3329788271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b140cef40 msgr2=0x7f1b141f0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b0bfff640 1 --2- 172.21.15.16:0/3329788271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b140cef40 0x7f1b141f0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b0bfff640 1 -- 172.21.15.16:0/3329788271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b141618d0 msgr2=0x7f1b14157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b0bfff640 1 --2- 172.21.15.16:0/3329788271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b141618d0 0x7f1b14157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b0bfff640 1 -- 172.21.15.16:0/3329788271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b140cd1c0 con 0x7f1b141610b0 2024-04-25T10:08:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b12d9d640 1 --2- 172.21.15.16:0/3329788271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b141618d0 0x7f1b14157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b0bfff640 1 --2- 172.21.15.16:0/3329788271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b141610b0 0x7f1b141f43e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00c430 tx=0x7f1afc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b0affd640 1 -- 172.21.15.16:0/3329788271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc0074e0 con 0x7f1b141610b0 2024-04-25T10:08:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b0affd640 1 -- 172.21.15.16:0/3329788271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1afc0047e0 con 0x7f1b141610b0 2024-04-25T10:08:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b189ab640 1 -- 172.21.15.16:0/3329788271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b141f4d70 con 0x7f1b141610b0 2024-04-25T10:08:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b0affd640 1 -- 172.21.15.16:0/3329788271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc004bd0 con 0x7f1b141610b0 2024-04-25T10:08:33.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.849+0000 7f1b189ab640 1 -- 172.21.15.16:0/3329788271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b141f5330 con 0x7f1b141610b0 2024-04-25T10:08:33.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.851+0000 7f1b0affd640 1 -- 172.21.15.16:0/3329788271 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1afc011070 con 0x7f1b141610b0 2024-04-25T10:08:33.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.851+0000 7f1b0affd640 1 --2- 172.21.15.16:0/3329788271 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1af406b4d0 0x7f1af406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.851+0000 7f1b0affd640 1 -- 172.21.15.16:0/3329788271 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1afc089d50 con 0x7f1b141610b0 2024-04-25T10:08:33.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.852+0000 7f1b1259c640 1 --2- 172.21.15.16:0/3329788271 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1af406b4d0 0x7f1af406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.852+0000 7f1b1b781740 1 Processor -- start 2024-04-25T10:08:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.852+0000 7f1b1259c640 1 --2- 172.21.15.16:0/3329788271 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1af406b4d0 0x7f1af406d990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1b04034000 tx=0x7f1b0402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.853+0000 7f1b1b781740 1 -- start start 2024-04-25T10:08:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.853+0000 7f1b1b781740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc5710 0x55f578fc7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.853+0000 7f1b1b781740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578fc8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.853+0000 7f1b1b781740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578f98370 0x55f578f9bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.853+0000 7f1b1b781740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f578f00bc0 con 0x55f578fc5af0 2024-04-25T10:08:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.853+0000 7f1b1b781740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f578ea9e70 con 0x55f578f98370 2024-04-25T10:08:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.853+0000 7f1b1b781740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f578efcc60 con 0x55f578fc5710 2024-04-25T10:08:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.853+0000 7f1b0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578fc8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.853+0000 7f1b0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578fc8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37682/0 (socket says 172.21.15.16:37682) 2024-04-25T10:08:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.853+0000 7f1b0bfff640 1 -- 172.21.15.16:0/1787734372 learned_addr learned my addr 172.21.15.16:0/1787734372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.854+0000 7f1b1259c640 1 --2- 172.21.15.16:0/1787734372 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc5710 0x55f578fc7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.854+0000 7f1b12d9d640 1 --2- 172.21.15.16:0/1787734372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578f98370 0x55f578f9bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.854+0000 7f1b0bfff640 1 -- 172.21.15.16:0/1787734372 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc5710 msgr2=0x55f578fc7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.854+0000 7f1b0bfff640 1 --2- 172.21.15.16:0/1787734372 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc5710 0x55f578fc7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.854+0000 7f1b0bfff640 1 -- 172.21.15.16:0/1787734372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578f98370 msgr2=0x55f578f9bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.854+0000 7f1b0bfff640 1 --2- 172.21.15.16:0/1787734372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578f98370 0x55f578f9bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.854+0000 7f1b0bfff640 1 -- 172.21.15.16:0/1787734372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f578ecaad0 con 0x55f578fc5af0 2024-04-25T10:08:33.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.854+0000 7f1b0bfff640 1 --2- 172.21.15.16:0/1787734372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578fc8460 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0882b0 tx=0x7f1afc057f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cb664eda266427 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.854+0000 7f1af3fff640 1 -- 172.21.15.16:0/1787734372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc05c6b0 con 0x55f578fc5af0 2024-04-25T10:08:33.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.854+0000 7f1af3fff640 1 -- 172.21.15.16:0/1787734372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1afc060c70 con 0x55f578fc5af0 2024-04-25T10:08:33.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.854+0000 7f1af3fff640 1 -- 172.21.15.16:0/1787734372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc004d70 con 0x55f578fc5af0 2024-04-25T10:08:33.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.855+0000 7f1b1b781740 1 -- 172.21.15.16:0/1787734372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 msgr2=0x55f578fc8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.855+0000 7f1b1b781740 1 --2- 172.21.15.16:0/1787734372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578fc8460 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0882b0 tx=0x7f1afc057f80 comp rx=0 tx=0).stop 2024-04-25T10:08:33.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.855+0000 7f1b1b781740 1 -- 172.21.15.16:0/1787734372 shutdown_connections 2024-04-25T10:08:33.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.855+0000 7f1b1b781740 1 --2- 172.21.15.16:0/1787734372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578f98370 0x55f578f9bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.855+0000 7f1b1b781740 1 --2- 172.21.15.16:0/1787734372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578fc8460 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.855+0000 7f1b1b781740 1 --2- 172.21.15.16:0/1787734372 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc5710 0x55f578fc7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.855+0000 7f1b1b781740 1 -- 172.21.15.16:0/1787734372 >> 172.21.15.16:0/1787734372 conn(0x55f578f9a5e0 msgr2=0x55f578fe2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.855+0000 7f1b1b781740 1 -- 172.21.15.16:0/1787734372 shutdown_connections 2024-04-25T10:08:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.855+0000 7f1b1b781740 1 -- 172.21.15.16:0/1787734372 wait complete. 2024-04-25T10:08:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.855+0000 7f1b1b781740 1 -- start start 2024-04-25T10:08:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.855+0000 7f1b1b781740 1 Processor -- start 2024-04-25T10:08:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.856+0000 7f1b1b781740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578fc5710 0x55f578fca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.856+0000 7f1b1b781740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578feaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.856+0000 7f1b1b781740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc4610 0x55f578fe5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.856+0000 7f1b1b781740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f578f9d790 con 0x55f578fc5af0 2024-04-25T10:08:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.856+0000 7f1b1b781740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f578e9f4e0 con 0x55f578fc5710 2024-04-25T10:08:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.856+0000 7f1b1b781740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f578f9da70 con 0x55f578fc4610 2024-04-25T10:08:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.856+0000 7f1b1b781740 10 client.0 _ll_register_callbacks cb 0x55f578ebcde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.856+0000 7f1b0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578feaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.856+0000 7f1b0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578feaf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37698/0 (socket says 172.21.15.16:37698) 2024-04-25T10:08:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.856+0000 7f1b0bfff640 1 -- 172.21.15.16:0/3431984173 learned_addr learned my addr 172.21.15.16:0/3431984173 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1b12d9d640 1 --2- 172.21.15.16:0/3431984173 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc4610 0x55f578fe5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1b0bfff640 1 -- 172.21.15.16:0/3431984173 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc4610 msgr2=0x55f578fe5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1b0bfff640 1 --2- 172.21.15.16:0/3431984173 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc4610 0x55f578fe5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1b1259c640 1 --2- 172.21.15.16:0/3431984173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578fc5710 0x55f578fca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1b0bfff640 1 -- 172.21.15.16:0/3431984173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578fc5710 msgr2=0x55f578fca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1b0bfff640 1 --2- 172.21.15.16:0/3431984173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578fc5710 0x55f578fca0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1b0bfff640 1 -- 172.21.15.16:0/3431984173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f578ce1940 con 0x55f578fc5af0 2024-04-25T10:08:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1b0bfff640 1 --2- 172.21.15.16:0/3431984173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578feaf00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0584a0 tx=0x7f1afc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4aeec5fbda1df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1af17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:08:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1af17fa640 1 -- 172.21.15.16:0/3431984173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc0579b0 con 0x55f578fc5af0 2024-04-25T10:08:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1af17fa640 1 -- 172.21.15.16:0/3431984173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1afc057b50 con 0x55f578fc5af0 2024-04-25T10:08:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1b1b781740 1 -- 172.21.15.16:0/3431984173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1afc060c70 con 0x55f578fc5af0 2024-04-25T10:08:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1af17fa640 1 -- 172.21.15.16:0/3431984173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1afc077120 con 0x55f578fc5af0 2024-04-25T10:08:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1b1b781740 10 client.6375 fetch_fsmap learned FSMap version 28 2024-04-25T10:08:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1b1b781740 1 -- 172.21.15.16:0/3431984173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55f578d420d0 con 0x55f578fc5af0 2024-04-25T10:08:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1af17fa640 1 -- 172.21.15.16:0/3431984173 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1afc060c70 con 0x55f578fc5af0 2024-04-25T10:08:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1af17fa640 1 -- 172.21.15.16:0/3431984173 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1afc071070 con 0x55f578fc5af0 2024-04-25T10:08:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1b1b781740 10 client.6375 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:08:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1b1b781740 10 client.6375 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1b1b781740 20 client.6375 populate_metadata read hostname 'smithi016' 2024-04-25T10:08:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.857+0000 7f1b1b781740 1 --2- 172.21.15.16:0/3431984173 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55f578fc8a50 0x55f578fc8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.858+0000 7f1b1b781740 4 client.6375 mds_command: new command op to 35141 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.858+0000 7f1b1b781740 1 -- 172.21.15.16:0/3431984173 --> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f578f0e100 con 0x55f578fc8a50 2024-04-25T10:08:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.858+0000 7f1b1259c640 1 --2- 172.21.15.16:0/3431984173 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55f578fc8a50 0x55f578fc8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.858+0000 7f1af17fa640 10 client.6375 ms_handle_connect on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.858+0000 7f1b1259c640 1 --2- 172.21.15.16:0/3431984173 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55f578fc8a50 0x55f578fc8e30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bda3abe3924d4ca server_cookie=95ed8f736715f738 in_seq=0 out_seq=0 2024-04-25T10:08:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.858+0000 7f1af17fa640 1 -- 172.21.15.16:0/3431984173 <== mds.0 v2:172.21.15.16:6832/3311584521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f578f0e100 con 0x55f578fc8a50 2024-04-25T10:08:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.858+0000 7f1af17fa640 10 client.6375 handle_command_reply: tid=0 2024-04-25T10:08:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.858+0000 7f1b1b781740 1 client.6375 shutdown 2024-04-25T10:08:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.859+0000 7f1b1b781740 1 -- 172.21.15.16:0/3431984173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 msgr2=0x55f578feaf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.859+0000 7f1b1b781740 1 --2- 172.21.15.16:0/3431984173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578feaf00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0584a0 tx=0x7f1afc000ed0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.859+0000 7f1b1b781740 1 -- 172.21.15.16:0/3431984173 shutdown_connections 2024-04-25T10:08:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.859+0000 7f1b1b781740 1 --2- 172.21.15.16:0/3431984173 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55f578fc8a50 0x55f578fc8e30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.859+0000 7f1b1b781740 1 --2- 172.21.15.16:0/3431984173 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc4610 0x55f578fe5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.859+0000 7f1af17fa640 0 client.6375 ms_handle_reset on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.859+0000 7f1b1b781740 1 --2- 172.21.15.16:0/3431984173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578feaf00 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.859+0000 7f1b1b781740 1 --2- 172.21.15.16:0/3431984173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578fc5710 0x55f578fca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.859+0000 7f1b1b781740 1 -- 172.21.15.16:0/3431984173 >> 172.21.15.16:0/3431984173 conn(0x55f578f98370 msgr2=0x55f578fc7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.859+0000 7f1b1b781740 1 -- 172.21.15.16:0/3431984173 shutdown_connections 2024-04-25T10:08:33.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.859+0000 7f1b1b781740 1 -- 172.21.15.16:0/3431984173 wait complete. 2024-04-25T10:08:33.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.859+0000 7f1b1b781740 20 client.6375 trim_cache size 0 max 16384 2024-04-25T10:08:33.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.874+0000 7f1b1b781740 1 Processor -- start 2024-04-25T10:08:33.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.874+0000 7f1b1b781740 1 -- start start 2024-04-25T10:08:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1b1b781740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578fc4610 0x55f578fb7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1b1b781740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc5710 0x55f578fc9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1b1b781740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578fca380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1b1b781740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b141ffc40 con 0x55f578fc5af0 2024-04-25T10:08:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1b1b781740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b141533c0 con 0x55f578fc4610 2024-04-25T10:08:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1b1b781740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b141f79d0 con 0x55f578fc5710 2024-04-25T10:08:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1b12d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578fca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1b12d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578fca380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37710/0 (socket says 172.21.15.16:37710) 2024-04-25T10:08:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1b12d9d640 1 -- 172.21.15.16:0/3210262183 learned_addr learned my addr 172.21.15.16:0/3210262183 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1b1259c640 1 --2- 172.21.15.16:0/3210262183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578fc4610 0x55f578fb7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1b0bfff640 1 --2- 172.21.15.16:0/3210262183 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc5710 0x55f578fc9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1b12d9d640 1 -- 172.21.15.16:0/3210262183 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc5710 msgr2=0x55f578fc9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1b12d9d640 1 --2- 172.21.15.16:0/3210262183 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc5710 0x55f578fc9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1b12d9d640 1 -- 172.21.15.16:0/3210262183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578fc4610 msgr2=0x55f578fb7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1b12d9d640 1 --2- 172.21.15.16:0/3210262183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578fc4610 0x55f578fb7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1b12d9d640 1 -- 172.21.15.16:0/3210262183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f578bd5230 con 0x55f578fc5af0 2024-04-25T10:08:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1b1259c640 1 --2- 172.21.15.16:0/3210262183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578fc4610 0x55f578fb7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1b12d9d640 1 --2- 172.21.15.16:0/3210262183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578fca380 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0054c0 tx=0x7f1b0c0111f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a84ee39b055ee37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1af37fe640 1 -- 172.21.15.16:0/3210262183 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c010040 con 0x55f578fc5af0 2024-04-25T10:08:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1af37fe640 1 -- 172.21.15.16:0/3210262183 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0c011e50 con 0x55f578fc5af0 2024-04-25T10:08:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.875+0000 7f1af37fe640 1 -- 172.21.15.16:0/3210262183 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c006d90 con 0x55f578fc5af0 2024-04-25T10:08:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.876+0000 7f1b1b781740 1 -- 172.21.15.16:0/3210262183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 msgr2=0x55f578fca380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.876+0000 7f1b1b781740 1 --2- 172.21.15.16:0/3210262183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578fca380 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0054c0 tx=0x7f1b0c0111f0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.876+0000 7f1b1b781740 1 -- 172.21.15.16:0/3210262183 shutdown_connections 2024-04-25T10:08:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.876+0000 7f1b1b781740 1 --2- 172.21.15.16:0/3210262183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578fca380 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.876+0000 7f1b1b781740 1 --2- 172.21.15.16:0/3210262183 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc5710 0x55f578fc9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.876+0000 7f1b1b781740 1 --2- 172.21.15.16:0/3210262183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578fc4610 0x55f578fb7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.876+0000 7f1b1b781740 1 -- 172.21.15.16:0/3210262183 >> 172.21.15.16:0/3210262183 conn(0x55f578f98370 msgr2=0x55f578f9e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.876+0000 7f1b1b781740 1 -- 172.21.15.16:0/3210262183 shutdown_connections 2024-04-25T10:08:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.876+0000 7f1b1b781740 1 -- 172.21.15.16:0/3210262183 wait complete. 2024-04-25T10:08:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.877+0000 7f1b1b781740 1 -- start start 2024-04-25T10:08:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.877+0000 7f1b1b781740 1 Processor -- start 2024-04-25T10:08:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b1b781740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578fc4610 0x55f578fc1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b1b781740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc5710 0x55f578fc1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b1b781740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578f9c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b1b781740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f578fc4ec0 con 0x55f578fc5af0 2024-04-25T10:08:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b1b781740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b14152dc0 con 0x55f578fc4610 2024-04-25T10:08:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b1b781740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f578f942f0 con 0x55f578fc5710 2024-04-25T10:08:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b1b781740 10 client.0 _ll_register_callbacks cb 0x7f1afc0580a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b12d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578f9c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b12d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578f9c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37722/0 (socket says 172.21.15.16:37722) 2024-04-25T10:08:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b12d9d640 1 -- 172.21.15.16:0/3923470811 learned_addr learned my addr 172.21.15.16:0/3923470811 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:33.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b0bfff640 1 --2- 172.21.15.16:0/3923470811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc5710 0x55f578fc1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b1259c640 1 --2- 172.21.15.16:0/3923470811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578fc4610 0x55f578fc1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b12d9d640 1 -- 172.21.15.16:0/3923470811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc5710 msgr2=0x55f578fc1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b12d9d640 1 --2- 172.21.15.16:0/3923470811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc5710 0x55f578fc1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b12d9d640 1 -- 172.21.15.16:0/3923470811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578fc4610 msgr2=0x55f578fc1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b12d9d640 1 --2- 172.21.15.16:0/3923470811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578fc4610 0x55f578fc1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b12d9d640 1 -- 172.21.15.16:0/3923470811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f578fba7d0 con 0x55f578fc5af0 2024-04-25T10:08:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b0bfff640 1 --2- 172.21.15.16:0/3923470811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc5710 0x55f578fc1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b12d9d640 1 --2- 172.21.15.16:0/3923470811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578f9c140 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x55f579006850 tx=0x7f1b0c003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd63c4744c623d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1af17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:08:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1af17fa640 1 -- 172.21.15.16:0/3923470811 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c01d800 con 0x55f578fc5af0 2024-04-25T10:08:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1af17fa640 1 -- 172.21.15.16:0/3923470811 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b0c01d9a0 con 0x55f578fc5af0 2024-04-25T10:08:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1b1b781740 1 -- 172.21.15.16:0/3923470811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b0c011e50 con 0x55f578fc5af0 2024-04-25T10:08:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.878+0000 7f1af17fa640 1 -- 172.21.15.16:0/3923470811 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b0c02f050 con 0x55f578fc5af0 2024-04-25T10:08:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.879+0000 7f1b1b781740 10 client.6381 fetch_fsmap learned FSMap version 28 2024-04-25T10:08:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.879+0000 7f1b1b781740 1 -- 172.21.15.16:0/3923470811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55f578f9bc20 con 0x55f578fc5af0 2024-04-25T10:08:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.879+0000 7f1af17fa640 1 -- 172.21.15.16:0/3923470811 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0c011e50 con 0x55f578fc5af0 2024-04-25T10:08:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.879+0000 7f1af17fa640 1 -- 172.21.15.16:0/3923470811 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1b0c032020 con 0x55f578fc5af0 2024-04-25T10:08:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.879+0000 7f1b1b781740 10 client.6381 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:08:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.879+0000 7f1b1b781740 10 client.6381 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.879+0000 7f1b1b781740 20 client.6381 populate_metadata read hostname 'smithi016' 2024-04-25T10:08:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.879+0000 7f1b1b781740 1 --2- 172.21.15.16:0/3923470811 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55f578fe1650 0x55f578f9c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.879+0000 7f1b1b781740 4 client.6381 mds_command: new command op to 35141 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-04-25T10:08:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.879+0000 7f1b1b781740 1 -- 172.21.15.16:0/3923470811 --> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) v1 -- 0x55f578f08450 con 0x55f578fe1650 2024-04-25T10:08:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.879+0000 7f1b1259c640 1 --2- 172.21.15.16:0/3923470811 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55f578fe1650 0x55f578f9c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.879+0000 7f1b1259c640 1 --2- 172.21.15.16:0/3923470811 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55f578fe1650 0x55f578f9c680 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee361ea50d57112d server_cookie=d21cdf8b02c02731 in_seq=0 out_seq=0 2024-04-25T10:08:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.879+0000 7f1af17fa640 10 client.6381 ms_handle_connect on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.880+0000 7f1af17fa640 1 -- 172.21.15.16:0/3923470811 <== mds.0 v2:172.21.15.16:6832/3311584521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f578f08450 con 0x55f578fe1650 2024-04-25T10:08:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.880+0000 7f1af17fa640 10 client.6381 handle_command_reply: tid=0 2024-04-25T10:08:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.880+0000 7f1b1b781740 1 client.6381 shutdown 2024-04-25T10:08:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.880+0000 7f1b1b781740 1 -- 172.21.15.16:0/3923470811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 msgr2=0x55f578f9c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.880+0000 7f1b1b781740 1 --2- 172.21.15.16:0/3923470811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578f9c140 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x55f579006850 tx=0x7f1b0c003a70 comp rx=0 tx=0).stop 2024-04-25T10:08:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.880+0000 7f1b1b781740 1 -- 172.21.15.16:0/3923470811 shutdown_connections 2024-04-25T10:08:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.880+0000 7f1b1b781740 1 --2- 172.21.15.16:0/3923470811 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x55f578fe1650 0x55f578f9c680 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.880+0000 7f1b1b781740 1 --2- 172.21.15.16:0/3923470811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f578fc5af0 0x55f578f9c140 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.880+0000 7f1b1b781740 1 --2- 172.21.15.16:0/3923470811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f578fc5710 0x55f578fc1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.880+0000 7f1af17fa640 0 client.6381 ms_handle_reset on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.880+0000 7f1b1b781740 1 --2- 172.21.15.16:0/3923470811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f578fc4610 0x55f578fc1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.880+0000 7f1b1b781740 1 -- 172.21.15.16:0/3923470811 >> 172.21.15.16:0/3923470811 conn(0x55f578f98370 msgr2=0x55f578f9cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.880+0000 7f1b1b781740 1 -- 172.21.15.16:0/3923470811 shutdown_connections 2024-04-25T10:08:33.893 INFO:tasks.ceph.mds.a.smithi016.stderr:2024-04-25T10:08:33.880+0000 7fce9a138640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={35240=0-4194304@1} caps={35240=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=35240 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55cf29494100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:08:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.880+0000 7f1b1b781740 1 -- 172.21.15.16:0/3923470811 wait complete. 2024-04-25T10:08:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.880+0000 7f1b1b781740 20 client.6381 trim_cache size 0 max 16384 2024-04-25T10:08:33.893 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:08:33.894 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-04-25T10:08:33.894 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "8dceffcd-ddcb-4374-8d46-b41ea9e2b324", 2024-04-25T10:08:33.894 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-04-25T10:08:33.894 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:08:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.881+0000 7f1b189ab640 1 -- 172.21.15.16:0/3329788271 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1af406b4d0 msgr2=0x7f1af406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.881+0000 7f1b189ab640 1 --2- 172.21.15.16:0/3329788271 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1af406b4d0 0x7f1af406d990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1b04034000 tx=0x7f1b0402e000 comp rx=0 tx=0).stop 2024-04-25T10:08:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.881+0000 7f1b189ab640 1 -- 172.21.15.16:0/3329788271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b141610b0 msgr2=0x7f1b141f43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.881+0000 7f1b189ab640 1 --2- 172.21.15.16:0/3329788271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b141610b0 0x7f1b141f43e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00c430 tx=0x7f1afc00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.881+0000 7f1b189ab640 1 -- 172.21.15.16:0/3329788271 shutdown_connections 2024-04-25T10:08:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.881+0000 7f1b189ab640 1 --2- 172.21.15.16:0/3329788271 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1af406b4d0 0x7f1af406d990 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.881+0000 7f1b189ab640 1 --2- 172.21.15.16:0/3329788271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b141618d0 0x7f1b14157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.881+0000 7f1b189ab640 1 --2- 172.21.15.16:0/3329788271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b141610b0 0x7f1b141f43e0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.881+0000 7f1b189ab640 1 --2- 172.21.15.16:0/3329788271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b140cef40 0x7f1b141f0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.881+0000 7f1b189ab640 1 -- 172.21.15.16:0/3329788271 >> 172.21.15.16:0/3329788271 conn(0x7f1b140b5570 msgr2=0x7f1b140d2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.881+0000 7f1b189ab640 1 -- 172.21.15.16:0/3329788271 shutdown_connections 2024-04-25T10:08:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.881+0000 7f1b189ab640 1 -- 172.21.15.16:0/3329788271 wait complete. 2024-04-25T10:08:33.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:33.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:08:34.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.994+0000 7f6cf46f5640 1 Processor -- start 2024-04-25T10:08:34.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.994+0000 7f6cf46f5640 1 -- start start 2024-04-25T10:08:34.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.995+0000 7f6cf46f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6cec15deb0 0x7f6cec0b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.995+0000 7f6cf46f5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6cec0b1d30 0x7f6cec06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.995+0000 7f6cf46f5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6cec06b0e0 0x7f6cec161b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.995+0000 7f6cf46f5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cec06b530 con 0x7f6cec06b0e0 2024-04-25T10:08:34.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.995+0000 7f6cf46f5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cec0b21d0 con 0x7f6cec0b1d30 2024-04-25T10:08:34.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.995+0000 7f6cf46f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0cd150 con 0x7f6cec15deb0 2024-04-25T10:08:34.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.995+0000 7f6cf2c92640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6cec06b0e0 0x7f6cec161b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.995+0000 7f6cf2c92640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6cec06b0e0 0x7f6cec161b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37730/0 (socket says 172.21.15.16:37730) 2024-04-25T10:08:34.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.995+0000 7f6cf2c92640 1 -- 172.21.15.16:0/2825451143 learned_addr learned my addr 172.21.15.16:0/2825451143 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:34.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.995+0000 7f6cf2491640 1 --2- 172.21.15.16:0/2825451143 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6cec15deb0 0x7f6cec0b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.996+0000 7f6cf1c90640 1 --2- 172.21.15.16:0/2825451143 >> v2:172.21.15.186:3300/0 conn(0x7f6cec0b1d30 0x7f6cec06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.996+0000 7f6cf2c92640 1 -- 172.21.15.16:0/2825451143 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6cec15deb0 msgr2=0x7f6cec0b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:34.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.996+0000 7f6cf2c92640 1 --2- 172.21.15.16:0/2825451143 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6cec15deb0 0x7f6cec0b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.996+0000 7f6cf2c92640 1 -- 172.21.15.16:0/2825451143 >> v2:172.21.15.186:3300/0 conn(0x7f6cec0b1d30 msgr2=0x7f6cec06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:34.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.996+0000 7f6cf2c92640 1 --2- 172.21.15.16:0/2825451143 >> v2:172.21.15.186:3300/0 conn(0x7f6cec0b1d30 0x7f6cec06aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.996+0000 7f6cf2c92640 1 -- 172.21.15.16:0/2825451143 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec0cd430 con 0x7f6cec06b0e0 2024-04-25T10:08:34.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.996+0000 7f6cf2491640 1 --2- 172.21.15.16:0/2825451143 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6cec15deb0 0x7f6cec0b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:34.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.996+0000 7f6cf2c92640 1 --2- 172.21.15.16:0/2825451143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cec06b0e0 0x7f6cec161b20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6cec0d1940 tx=0x7f6cdc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c829a9113170e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:34.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.997+0000 7f6cf148f640 1 -- 172.21.15.16:0/2825451143 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc036070 con 0x7f6cec06b0e0 2024-04-25T10:08:34.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.997+0000 7f6cf148f640 1 -- 172.21.15.16:0/2825451143 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc031600 con 0x7f6cec06b0e0 2024-04-25T10:08:34.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.997+0000 7f6cf148f640 1 -- 172.21.15.16:0/2825451143 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc0319d0 con 0x7f6cec06b0e0 2024-04-25T10:08:34.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.997+0000 7f6cf46f5640 1 -- 172.21.15.16:0/2825451143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cec06b0e0 msgr2=0x7f6cec161b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.997+0000 7f6cf46f5640 1 --2- 172.21.15.16:0/2825451143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cec06b0e0 0x7f6cec161b20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6cec0d1940 tx=0x7f6cdc02a610 comp rx=0 tx=0).stop 2024-04-25T10:08:34.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.997+0000 7f6cf46f5640 1 -- 172.21.15.16:0/2825451143 shutdown_connections 2024-04-25T10:08:34.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.997+0000 7f6cf46f5640 1 --2- 172.21.15.16:0/2825451143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cec06b0e0 0x7f6cec161b20 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.997+0000 7f6cf46f5640 1 --2- 172.21.15.16:0/2825451143 >> v2:172.21.15.186:3300/0 conn(0x7f6cec0b1d30 0x7f6cec06aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.997+0000 7f6cf46f5640 1 --2- 172.21.15.16:0/2825451143 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6cec15deb0 0x7f6cec0b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.997+0000 7f6cf46f5640 1 -- 172.21.15.16:0/2825451143 >> 172.21.15.16:0/2825451143 conn(0x7f6cec0b2390 msgr2=0x7f6cec0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:34.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.997+0000 7f6cf46f5640 1 -- 172.21.15.16:0/2825451143 shutdown_connections 2024-04-25T10:08:34.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.997+0000 7f6cf46f5640 1 -- 172.21.15.16:0/2825451143 wait complete. 2024-04-25T10:08:34.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.998+0000 7f6cf46f5640 1 Processor -- start 2024-04-25T10:08:34.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.998+0000 7f6cf46f5640 1 -- start start 2024-04-25T10:08:34.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.999+0000 7f6cf46f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cec06b0e0 0x7f6cec158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.999+0000 7f6cf46f5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cec0b1d30 0x7f6cec1f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.999+0000 7f6cf46f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6cec15deb0 0x7f6cec1f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.999+0000 7f6cf46f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec15e290 con 0x7f6cec06b0e0 2024-04-25T10:08:34.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.999+0000 7f6cf46f5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0cd9f0 con 0x7f6cec0b1d30 2024-04-25T10:08:34.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.999+0000 7f6cf46f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec15e410 con 0x7f6cec15deb0 2024-04-25T10:08:34.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.999+0000 7f6cf2491640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cec06b0e0 0x7f6cec158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.999+0000 7f6cf2491640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cec06b0e0 0x7f6cec158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37732/0 (socket says 172.21.15.16:37732) 2024-04-25T10:08:34.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.999+0000 7f6cf2491640 1 -- 172.21.15.16:0/2069320934 learned_addr learned my addr 172.21.15.16:0/2069320934 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:34.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.999+0000 7f6cf1c90640 1 --2- 172.21.15.16:0/2069320934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cec0b1d30 0x7f6cec1f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:33.999+0000 7f6cf2c92640 1 --2- 172.21.15.16:0/2069320934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6cec15deb0 0x7f6cec1f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.000+0000 7f6cf2491640 1 -- 172.21.15.16:0/2069320934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6cec15deb0 msgr2=0x7f6cec1f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.000+0000 7f6cf2491640 1 --2- 172.21.15.16:0/2069320934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6cec15deb0 0x7f6cec1f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.000+0000 7f6cf2491640 1 -- 172.21.15.16:0/2069320934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cec0b1d30 msgr2=0x7f6cec1f6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.000+0000 7f6cf2491640 1 --2- 172.21.15.16:0/2069320934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cec0b1d30 0x7f6cec1f6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.000+0000 7f6cf2491640 1 -- 172.21.15.16:0/2069320934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec0c5cd0 con 0x7f6cec06b0e0 2024-04-25T10:08:34.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.000+0000 7f6cf2491640 1 --2- 172.21.15.16:0/2069320934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cec06b0e0 0x7f6cec158630 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400d7d0 tx=0x7f6ce400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:34.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.000+0000 7f6ceaffd640 1 -- 172.21.15.16:0/2069320934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4004030 con 0x7f6cec06b0e0 2024-04-25T10:08:34.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.000+0000 7f6ceaffd640 1 -- 172.21.15.16:0/2069320934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce40041d0 con 0x7f6cec06b0e0 2024-04-25T10:08:34.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.000+0000 7f6ceaffd640 1 -- 172.21.15.16:0/2069320934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4009640 con 0x7f6cec06b0e0 2024-04-25T10:08:34.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.000+0000 7f6cf46f5640 1 -- 172.21.15.16:0/2069320934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cec0c7af0 con 0x7f6cec06b0e0 2024-04-25T10:08:34.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.000+0000 7f6cf46f5640 1 -- 172.21.15.16:0/2069320934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec158b70 con 0x7f6cec06b0e0 2024-04-25T10:08:34.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.002+0000 7f6ceaffd640 1 -- 172.21.15.16:0/2069320934 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ce40097e0 con 0x7f6cec06b0e0 2024-04-25T10:08:34.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.002+0000 7f6ceaffd640 1 --2- 172.21.15.16:0/2069320934 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6cd806b4d0 0x7f6cd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.003+0000 7f6ceaffd640 1 -- 172.21.15.16:0/2069320934 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ce408a290 con 0x7f6cec06b0e0 2024-04-25T10:08:34.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.003+0000 7f6cf1c90640 1 --2- 172.21.15.16:0/2069320934 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6cd806b4d0 0x7f6cd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.003+0000 7f6cf73fa740 1 Processor -- start 2024-04-25T10:08:34.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.003+0000 7f6cf1c90640 1 --2- 172.21.15.16:0/2069320934 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6cd806b4d0 0x7f6cd806d990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6cec06aeb0 tx=0x7f6ce0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:34.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.003+0000 7f6cf73fa740 1 -- start start 2024-04-25T10:08:34.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.004+0000 7f6cf73fa740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0360b10 0x5558b0363320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.004+0000 7f6cf73fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b0363860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.004+0000 7f6cf73fa740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0333770 0x5558b0336f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.004+0000 7f6cf73fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b029cda0 con 0x5558b0360ef0 2024-04-25T10:08:34.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.004+0000 7f6cf73fa740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b0245b80 con 0x5558b0360b10 2024-04-25T10:08:34.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.004+0000 7f6cf73fa740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558b0298e40 con 0x5558b0333770 2024-04-25T10:08:34.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.004+0000 7f6cf1c90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b0363860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.004+0000 7f6cf1c90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b0363860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37742/0 (socket says 172.21.15.16:37742) 2024-04-25T10:08:34.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.004+0000 7f6cf1c90640 1 -- 172.21.15.16:0/2930562109 learned_addr learned my addr 172.21.15.16:0/2930562109 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:34.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.004+0000 7f6cf2c92640 1 --2- 172.21.15.16:0/2930562109 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0333770 0x5558b0336f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.004+0000 7f6cf2491640 1 --2- 172.21.15.16:0/2930562109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0360b10 0x5558b0363320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.004+0000 7f6cf1c90640 1 -- 172.21.15.16:0/2930562109 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0333770 msgr2=0x5558b0336f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.004+0000 7f6cf1c90640 1 --2- 172.21.15.16:0/2930562109 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0333770 0x5558b0336f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.004+0000 7f6cf1c90640 1 -- 172.21.15.16:0/2930562109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0360b10 msgr2=0x5558b0363320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.004+0000 7f6cf1c90640 1 --2- 172.21.15.16:0/2930562109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0360b10 0x5558b0363320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.004+0000 7f6cf1c90640 1 -- 172.21.15.16:0/2930562109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558b024ff10 con 0x5558b0360ef0 2024-04-25T10:08:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.005+0000 7f6cf1c90640 1 --2- 172.21.15.16:0/2930562109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b0363860 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0004400 tx=0x7f6ce0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28c54188b9c30a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.005+0000 7f6cd3fff640 1 -- 172.21.15.16:0/2930562109 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce001b070 con 0x5558b0360ef0 2024-04-25T10:08:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.005+0000 7f6cd3fff640 1 -- 172.21.15.16:0/2930562109 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce0027070 con 0x5558b0360ef0 2024-04-25T10:08:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.005+0000 7f6cd3fff640 1 -- 172.21.15.16:0/2930562109 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0003a20 con 0x5558b0360ef0 2024-04-25T10:08:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.005+0000 7f6cf73fa740 1 -- 172.21.15.16:0/2930562109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 msgr2=0x5558b0363860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.005+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/2930562109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b0363860 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0004400 tx=0x7f6ce0014000 comp rx=0 tx=0).stop 2024-04-25T10:08:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.006+0000 7f6cf73fa740 1 -- 172.21.15.16:0/2930562109 shutdown_connections 2024-04-25T10:08:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.006+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/2930562109 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0333770 0x5558b0336f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.006+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/2930562109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b0363860 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.006+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/2930562109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0360b10 0x5558b0363320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.006+0000 7f6cf73fa740 1 -- 172.21.15.16:0/2930562109 >> 172.21.15.16:0/2930562109 conn(0x5558b03359e0 msgr2=0x5558b037e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.006+0000 7f6cf73fa740 1 -- 172.21.15.16:0/2930562109 shutdown_connections 2024-04-25T10:08:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.006+0000 7f6cf73fa740 1 -- 172.21.15.16:0/2930562109 wait complete. 2024-04-25T10:08:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.006+0000 7f6cf73fa740 1 -- start start 2024-04-25T10:08:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.006+0000 7f6cf73fa740 1 Processor -- start 2024-04-25T10:08:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.007+0000 7f6cf73fa740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0360b10 0x5558b0363e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.007+0000 7f6cf73fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b0334400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.007+0000 7f6cf73fa740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0334940 0x5558b03570e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.007+0000 7f6cf73fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b0338b90 con 0x5558b0360ef0 2024-04-25T10:08:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.007+0000 7f6cf73fa740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b023b4b0 con 0x5558b0360b10 2024-04-25T10:08:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.007+0000 7f6cf73fa740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558b0338e70 con 0x5558b0334940 2024-04-25T10:08:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.007+0000 7f6cf73fa740 10 client.0 _ll_register_callbacks cb 0x5558b027b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.007+0000 7f6cf1c90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b0334400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.007+0000 7f6cf1c90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b0334400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37750/0 (socket says 172.21.15.16:37750) 2024-04-25T10:08:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.007+0000 7f6cf1c90640 1 -- 172.21.15.16:0/2827559763 learned_addr learned my addr 172.21.15.16:0/2827559763 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.007+0000 7f6cf2c92640 1 --2- 172.21.15.16:0/2827559763 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0334940 0x5558b03570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.007+0000 7f6cf2491640 1 --2- 172.21.15.16:0/2827559763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0360b10 0x5558b0363e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.008+0000 7f6cf1c90640 1 -- 172.21.15.16:0/2827559763 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0334940 msgr2=0x5558b03570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.008+0000 7f6cf1c90640 1 --2- 172.21.15.16:0/2827559763 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0334940 0x5558b03570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.008+0000 7f6cf1c90640 1 -- 172.21.15.16:0/2827559763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0360b10 msgr2=0x5558b0363e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.008+0000 7f6cf1c90640 1 --2- 172.21.15.16:0/2827559763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0360b10 0x5558b0363e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.008+0000 7f6cf1c90640 1 -- 172.21.15.16:0/2827559763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558b007b7d0 con 0x5558b0360ef0 2024-04-25T10:08:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.008+0000 7f6cf1c90640 1 --2- 172.21.15.16:0/2827559763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b0334400 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x5558b033b070 tx=0x7f6ce0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a66583600e13de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.008+0000 7f6cd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:08:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.008+0000 7f6cd17fa640 1 -- 172.21.15.16:0/2827559763 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce001b040 con 0x5558b0360ef0 2024-04-25T10:08:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.008+0000 7f6cd17fa640 1 -- 172.21.15.16:0/2827559763 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ce0036ae0 con 0x5558b0360ef0 2024-04-25T10:08:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.008+0000 7f6cf73fa740 1 -- 172.21.15.16:0/2827559763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ce0027070 con 0x5558b0360ef0 2024-04-25T10:08:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.008+0000 7f6cd17fa640 1 -- 172.21.15.16:0/2827559763 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ce003d050 con 0x5558b0360ef0 2024-04-25T10:08:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.008+0000 7f6cf73fa740 10 client.6393 fetch_fsmap learned FSMap version 28 2024-04-25T10:08:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.008+0000 7f6cf73fa740 1 -- 172.21.15.16:0/2827559763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5558affb13f0 con 0x5558b0360ef0 2024-04-25T10:08:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.008+0000 7f6cd17fa640 1 -- 172.21.15.16:0/2827559763 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce0027070 con 0x5558b0360ef0 2024-04-25T10:08:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.008+0000 7f6cd17fa640 1 -- 172.21.15.16:0/2827559763 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6ce0040020 con 0x5558b0360ef0 2024-04-25T10:08:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.009+0000 7f6cf73fa740 10 client.6393 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:08:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.009+0000 7f6cf73fa740 10 client.6393 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.009+0000 7f6cf73fa740 20 client.6393 populate_metadata read hostname 'smithi016' 2024-04-25T10:08:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.009+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/2827559763 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x5558b0378e20 0x5558b0379230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.009+0000 7f6cf73fa740 4 client.6393 mds_command: new command op to 35141 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.009+0000 7f6cf73fa740 1 -- 172.21.15.16:0/2827559763 --> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558b02aa2e0 con 0x5558b0378e20 2024-04-25T10:08:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.009+0000 7f6cf2491640 1 --2- 172.21.15.16:0/2827559763 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x5558b0378e20 0x5558b0379230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.009+0000 7f6cd17fa640 10 client.6393 ms_handle_connect on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.009+0000 7f6cf2491640 1 --2- 172.21.15.16:0/2827559763 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x5558b0378e20 0x5558b0379230 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56e70fa95bd1f22 server_cookie=6f6169bc8b45c07b in_seq=0 out_seq=0 2024-04-25T10:08:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.009+0000 7f6cd17fa640 1 -- 172.21.15.16:0/2827559763 <== mds.0 v2:172.21.15.16:6832/3311584521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5558b02aa2e0 con 0x5558b0378e20 2024-04-25T10:08:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.009+0000 7f6cd17fa640 10 client.6393 handle_command_reply: tid=0 2024-04-25T10:08:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.009+0000 7f6cf73fa740 1 client.6393 shutdown 2024-04-25T10:08:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.010+0000 7f6cf73fa740 1 -- 172.21.15.16:0/2827559763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 msgr2=0x5558b0334400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.010+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/2827559763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b0334400 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x5558b033b070 tx=0x7f6ce0004730 comp rx=0 tx=0).stop 2024-04-25T10:08:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.010+0000 7f6cf73fa740 1 -- 172.21.15.16:0/2827559763 shutdown_connections 2024-04-25T10:08:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.010+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/2827559763 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x5558b0378e20 0x5558b0379230 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.010+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/2827559763 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0334940 0x5558b03570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.010+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/2827559763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b0334400 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.010+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/2827559763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0360b10 0x5558b0363e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.010+0000 7f6cd17fa640 0 client.6393 ms_handle_reset on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.010+0000 7f6cf73fa740 1 -- 172.21.15.16:0/2827559763 >> 172.21.15.16:0/2827559763 conn(0x5558b0333770 msgr2=0x5558b035fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.010+0000 7f6cf73fa740 1 -- 172.21.15.16:0/2827559763 shutdown_connections 2024-04-25T10:08:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.010+0000 7f6cf73fa740 1 -- 172.21.15.16:0/2827559763 wait complete. 2024-04-25T10:08:34.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.010+0000 7f6cf73fa740 20 client.6393 trim_cache size 0 max 16384 2024-04-25T10:08:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.025+0000 7f6cf73fa740 1 Processor -- start 2024-04-25T10:08:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.025+0000 7f6cf73fa740 1 -- start start 2024-04-25T10:08:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cf73fa740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0334940 0x5558b03570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cf73fa740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0360b10 0x5558b03399c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cf73fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b0338da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cf73fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec200030 con 0x5558b0360ef0 2024-04-25T10:08:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cf73fa740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec1ff390 con 0x5558b0334940 2024-04-25T10:08:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cf73fa740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0c5090 con 0x5558b0360b10 2024-04-25T10:08:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cf2c92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b0338da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cf2c92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b0338da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37760/0 (socket says 172.21.15.16:37760) 2024-04-25T10:08:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cf2c92640 1 -- 172.21.15.16:0/1270431257 learned_addr learned my addr 172.21.15.16:0/1270431257 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cf1c90640 1 --2- 172.21.15.16:0/1270431257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0360b10 0x5558b03399c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cf2491640 1 --2- 172.21.15.16:0/1270431257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0334940 0x5558b03570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cf2c92640 1 -- 172.21.15.16:0/1270431257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0360b10 msgr2=0x5558b03399c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cf2c92640 1 --2- 172.21.15.16:0/1270431257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0360b10 0x5558b03399c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cf2c92640 1 -- 172.21.15.16:0/1270431257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0334940 msgr2=0x5558b03570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cf2c92640 1 --2- 172.21.15.16:0/1270431257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0334940 0x5558b03570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cf2c92640 1 -- 172.21.15.16:0/1270431257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558b0365d30 con 0x5558b0360ef0 2024-04-25T10:08:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cf2491640 1 --2- 172.21.15.16:0/1270431257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0334940 0x5558b03570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cf2c92640 1 --2- 172.21.15.16:0/1270431257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b0338da0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0093e0 tx=0x7f6cdc02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34cba436f7e6565a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.026+0000 7f6cd37fe640 1 -- 172.21.15.16:0/1270431257 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc036040 con 0x5558b0360ef0 2024-04-25T10:08:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.027+0000 7f6cd37fe640 1 -- 172.21.15.16:0/1270431257 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc037040 con 0x5558b0360ef0 2024-04-25T10:08:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.027+0000 7f6cd37fe640 1 -- 172.21.15.16:0/1270431257 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc031940 con 0x5558b0360ef0 2024-04-25T10:08:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.027+0000 7f6cf73fa740 1 -- 172.21.15.16:0/1270431257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 msgr2=0x5558b0338da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.027+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/1270431257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b0338da0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0093e0 tx=0x7f6cdc02e000 comp rx=0 tx=0).stop 2024-04-25T10:08:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.027+0000 7f6cf73fa740 1 -- 172.21.15.16:0/1270431257 shutdown_connections 2024-04-25T10:08:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.027+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/1270431257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b0338da0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.027+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/1270431257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0360b10 0x5558b03399c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.027+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/1270431257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0334940 0x5558b03570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.027+0000 7f6cf73fa740 1 -- 172.21.15.16:0/1270431257 >> 172.21.15.16:0/1270431257 conn(0x5558b0333770 msgr2=0x5558b03632e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.027+0000 7f6cf73fa740 1 -- 172.21.15.16:0/1270431257 shutdown_connections 2024-04-25T10:08:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.027+0000 7f6cf73fa740 1 -- 172.21.15.16:0/1270431257 wait complete. 2024-04-25T10:08:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.028+0000 7f6cf73fa740 1 -- start start 2024-04-25T10:08:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.028+0000 7f6cf73fa740 1 Processor -- start 2024-04-25T10:08:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf73fa740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0334940 0x5558b0381e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf73fa740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0360b10 0x5558b035a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf73fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b035aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf73fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b0386a40 con 0x5558b0360ef0 2024-04-25T10:08:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf73fa740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0c5690 con 0x5558b0334940 2024-04-25T10:08:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf73fa740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558b035b3f0 con 0x5558b0360b10 2024-04-25T10:08:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf73fa740 10 client.0 _ll_register_callbacks cb 0x7f6ce0009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf2c92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b035aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf2c92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b035aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37774/0 (socket says 172.21.15.16:37774) 2024-04-25T10:08:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf2c92640 1 -- 172.21.15.16:0/3685238364 learned_addr learned my addr 172.21.15.16:0/3685238364 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:08:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf1c90640 1 --2- 172.21.15.16:0/3685238364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0360b10 0x5558b035a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf2491640 1 --2- 172.21.15.16:0/3685238364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0334940 0x5558b0381e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf2c92640 1 -- 172.21.15.16:0/3685238364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0360b10 msgr2=0x5558b035a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf2c92640 1 --2- 172.21.15.16:0/3685238364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0360b10 0x5558b035a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf2c92640 1 -- 172.21.15.16:0/3685238364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0334940 msgr2=0x5558b0381e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:34.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf2c92640 1 --2- 172.21.15.16:0/3685238364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0334940 0x5558b0381e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf2c92640 1 -- 172.21.15.16:0/3685238364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558aff710f0 con 0x5558b0360ef0 2024-04-25T10:08:34.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf2491640 1 --2- 172.21.15.16:0/3685238364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0334940 0x5558b0381e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:34.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf1c90640 1 --2- 172.21.15.16:0/3685238364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0360b10 0x5558b035a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:34.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf2c92640 1 --2- 172.21.15.16:0/3685238364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b035aba0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc02ac00 tx=0x7f6cdc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcf5b73d9bd7dd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:34.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:08:34.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cd17fa640 1 -- 172.21.15.16:0/3685238364 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc050020 con 0x5558b0360ef0 2024-04-25T10:08:34.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cd17fa640 1 -- 172.21.15.16:0/3685238364 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6cdc031cc0 con 0x5558b0360ef0 2024-04-25T10:08:34.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf73fa740 1 -- 172.21.15.16:0/3685238364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cdc037040 con 0x5558b0360ef0 2024-04-25T10:08:34.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cd17fa640 1 -- 172.21.15.16:0/3685238364 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cdc05a050 con 0x5558b0360ef0 2024-04-25T10:08:34.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.029+0000 7f6cf73fa740 10 client.6399 fetch_fsmap learned FSMap version 28 2024-04-25T10:08:34.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.030+0000 7f6cf73fa740 1 -- 172.21.15.16:0/3685238364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5558b0357dd0 con 0x5558b0360ef0 2024-04-25T10:08:34.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.030+0000 7f6cd17fa640 1 -- 172.21.15.16:0/3685238364 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc037040 con 0x5558b0360ef0 2024-04-25T10:08:34.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.030+0000 7f6cd17fa640 1 -- 172.21.15.16:0/3685238364 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6cdc05d020 con 0x5558b0360ef0 2024-04-25T10:08:34.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.030+0000 7f6cf73fa740 10 client.6399 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:08:34.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.030+0000 7f6cf73fa740 10 client.6399 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:34.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.030+0000 7f6cf73fa740 20 client.6399 populate_metadata read hostname 'smithi016' 2024-04-25T10:08:34.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.030+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/3685238364 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x5558b037a2f0 0x5558b0332600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.030+0000 7f6cf73fa740 4 client.6399 mds_command: new command op to 35141 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:08:34.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.030+0000 7f6cf73fa740 1 -- 172.21.15.16:0/3685238364 --> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5558b02a4630 con 0x5558b037a2f0 2024-04-25T10:08:34.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.030+0000 7f6cf2491640 1 --2- 172.21.15.16:0/3685238364 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x5558b037a2f0 0x5558b0332600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.030+0000 7f6cd17fa640 10 client.6399 ms_handle_connect on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:34.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.030+0000 7f6cf2491640 1 --2- 172.21.15.16:0/3685238364 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x5558b037a2f0 0x5558b0332600 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=360e08b8ce85eb39 server_cookie=6850d9e086de8fa4 in_seq=0 out_seq=0 2024-04-25T10:08:34.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.030+0000 7f6cd17fa640 1 -- 172.21.15.16:0/3685238364 <== mds.0 v2:172.21.15.16:6832/3311584521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x5558b02a4630 con 0x5558b037a2f0 2024-04-25T10:08:34.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.030+0000 7f6cd17fa640 10 client.6399 handle_command_reply: tid=0 2024-04-25T10:08:34.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.030+0000 7f6cf73fa740 1 client.6399 shutdown 2024-04-25T10:08:34.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf73fa740 1 -- 172.21.15.16:0/3685238364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 msgr2=0x5558b035aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/3685238364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b035aba0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc02ac00 tx=0x7f6cdc004ab0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf73fa740 1 -- 172.21.15.16:0/3685238364 shutdown_connections 2024-04-25T10:08:34.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/3685238364 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x5558b037a2f0 0x5558b0332600 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/3685238364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558b0360ef0 0x5558b035aba0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/3685238364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5558b0360b10 0x5558b035a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cd17fa640 0 client.6399 ms_handle_reset on v2:172.21.15.16:6832/3311584521 2024-04-25T10:08:34.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf73fa740 1 --2- 172.21.15.16:0/3685238364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5558b0334940 0x5558b0381e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf73fa740 1 -- 172.21.15.16:0/3685238364 >> 172.21.15.16:0/3685238364 conn(0x5558b0333770 msgr2=0x5558b033a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:34.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf73fa740 1 -- 172.21.15.16:0/3685238364 shutdown_connections 2024-04-25T10:08:34.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf73fa740 1 -- 172.21.15.16:0/3685238364 wait complete. 2024-04-25T10:08:34.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf73fa740 20 client.6399 trim_cache size 0 max 16384 2024-04-25T10:08:34.046 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:08:34.046 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:08:34.046 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-04-25T10:08:34.046 INFO:teuthology.orchestra.run.smithi016.stdout: "8dceffcd-ddcb-4374-8d46-b41ea9e2b324": { 2024-04-25T10:08:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-04-25T10:08:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "8dceffcd-ddcb-4374-8d46-b41ea9e2b324", 2024-04-25T10:08:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,repair,force" 2024-04-25T10:08:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:08:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:08:34.047 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:08:34.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf46f5640 1 -- 172.21.15.16:0/2069320934 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6cd806b4d0 msgr2=0x7f6cd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf46f5640 1 --2- 172.21.15.16:0/2069320934 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6cd806b4d0 0x7f6cd806d990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6cec06aeb0 tx=0x7f6ce0009190 comp rx=0 tx=0).stop 2024-04-25T10:08:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf46f5640 1 -- 172.21.15.16:0/2069320934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cec06b0e0 msgr2=0x7f6cec158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf46f5640 1 --2- 172.21.15.16:0/2069320934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cec06b0e0 0x7f6cec158630 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400d7d0 tx=0x7f6ce400dc90 comp rx=0 tx=0).stop 2024-04-25T10:08:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf46f5640 1 -- 172.21.15.16:0/2069320934 shutdown_connections 2024-04-25T10:08:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf46f5640 1 --2- 172.21.15.16:0/2069320934 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6cd806b4d0 0x7f6cd806d990 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf46f5640 1 --2- 172.21.15.16:0/2069320934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6cec15deb0 0x7f6cec1f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf46f5640 1 --2- 172.21.15.16:0/2069320934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cec0b1d30 0x7f6cec1f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf46f5640 1 --2- 172.21.15.16:0/2069320934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cec06b0e0 0x7f6cec158630 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf46f5640 1 -- 172.21.15.16:0/2069320934 >> 172.21.15.16:0/2069320934 conn(0x7f6cec0b2390 msgr2=0x7f6cec15c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf46f5640 1 -- 172.21.15.16:0/2069320934 shutdown_connections 2024-04-25T10:08:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:08:34.031+0000 7f6cf46f5640 1 -- 172.21.15.16:0/2069320934 wait complete. 2024-04-25T10:08:34.049 INFO:tasks.cephfs.filesystem:scrub status for tag:8dceffcd-ddcb-4374-8d46-b41ea9e2b324 - {'path': '/', 'tag': '8dceffcd-ddcb-4374-8d46-b41ea9e2b324', 'options': 'recursive,repair,force'} 2024-04-25T10:08:37.591 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:37.594 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:04.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:04.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:09:04.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.161+0000 7f7367934640 1 Processor -- start 2024-04-25T10:09:04.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.161+0000 7f7367934640 1 -- start start 2024-04-25T10:09:04.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.162+0000 7f7367934640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f736015dca0 0x7f73600ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.162+0000 7f7367934640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f73600d3b90 0x7f73600d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.162+0000 7f7367934640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73600d4090 con 0x7f73600cee80 2024-04-25T10:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.162+0000 7f7367934640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73600b6b20 con 0x7f73600d3b90 2024-04-25T10:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.162+0000 7f7367934640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73600d4370 con 0x7f736015dca0 2024-04-25T10:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.162+0000 7f7364ecf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f73600d3b90 0x7f73600d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.162+0000 7f7365ed1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f73600cee80 0x7f73600b2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32806/0 (socket says 172.21.15.16:32806) 2024-04-25T10:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.162+0000 7f7365ed1640 1 -- 172.21.15.16:0/4236579022 learned_addr learned my addr 172.21.15.16:0/4236579022 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.162+0000 7f73656d0640 1 --2- 172.21.15.16:0/4236579022 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f736015dca0 0x7f73600ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.162+0000 7f7364ecf640 1 -- 172.21.15.16:0/4236579022 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f736015dca0 msgr2=0x7f73600ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.162+0000 7f7364ecf640 1 --2- 172.21.15.16:0/4236579022 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f736015dca0 0x7f73600ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.162+0000 7f7364ecf640 1 -- 172.21.15.16:0/4236579022 >> v1:172.21.15.16:6789/0 conn(0x7f73600cee80 legacy=0x7f73600b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.162+0000 7f7364ecf640 1 -- 172.21.15.16:0/4236579022 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73600d4650 con 0x7f73600d3b90 2024-04-25T10:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.162+0000 7f7357fff640 1 -- 172.21.15.16:0/4236579022 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244737834 0 0) 0x7f73600d4090 con 0x7f73600cee80 2024-04-25T10:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.163+0000 7f7364ecf640 1 --2- 172.21.15.16:0/4236579022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73600d3b90 0x7f73600d0a20 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f73500097a0 tx=0x7f735002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e176e67d8454922f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.163+0000 7f7357fff640 1 -- 172.21.15.16:0/4236579022 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350037070 con 0x7f73600d3b90 2024-04-25T10:09:04.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.163+0000 7f7357fff640 1 -- 172.21.15.16:0/4236579022 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7350032440 con 0x7f73600d3b90 2024-04-25T10:09:04.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.163+0000 7f7357fff640 1 -- 172.21.15.16:0/4236579022 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350032810 con 0x7f73600d3b90 2024-04-25T10:09:04.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.164+0000 7f7367934640 1 -- 172.21.15.16:0/4236579022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73600d3b90 msgr2=0x7f73600d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.164+0000 7f7367934640 1 --2- 172.21.15.16:0/4236579022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73600d3b90 0x7f73600d0a20 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f73500097a0 tx=0x7f735002a180 comp rx=0 tx=0).stop 2024-04-25T10:09:04.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.164+0000 7f7367934640 1 -- 172.21.15.16:0/4236579022 shutdown_connections 2024-04-25T10:09:04.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.164+0000 7f7367934640 1 --2- 172.21.15.16:0/4236579022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73600d3b90 0x7f73600d0a20 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.164+0000 7f7367934640 1 --2- 172.21.15.16:0/4236579022 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f736015dca0 0x7f73600ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.164+0000 7f7367934640 1 -- 172.21.15.16:0/4236579022 >> 172.21.15.16:0/4236579022 conn(0x7f73600726c0 msgr2=0x7f7360072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.164+0000 7f7367934640 1 -- 172.21.15.16:0/4236579022 shutdown_connections 2024-04-25T10:09:04.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.164+0000 7f7367934640 1 -- 172.21.15.16:0/4236579022 wait complete. 2024-04-25T10:09:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.165+0000 7f7367934640 1 Processor -- start 2024-04-25T10:09:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.165+0000 7f7367934640 1 -- start start 2024-04-25T10:09:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.166+0000 7f7367934640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73600cee80 0x7f7360157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.166+0000 7f7367934640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73600d3b90 0x7f73600cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.166+0000 7f7367934640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f736015dca0 0x7f7360159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.166+0000 7f7367934640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736015e200 con 0x7f736015dca0 2024-04-25T10:09:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.166+0000 7f7367934640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736015e080 con 0x7f73600cee80 2024-04-25T10:09:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.166+0000 7f7367934640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73600d0f60 con 0x7f73600d3b90 2024-04-25T10:09:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.166+0000 7f7365ed1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f736015dca0 0x7f7360159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.166+0000 7f7365ed1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f736015dca0 0x7f7360159bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40878/0 (socket says 172.21.15.16:40878) 2024-04-25T10:09:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.166+0000 7f7365ed1640 1 -- 172.21.15.16:0/1262285700 learned_addr learned my addr 172.21.15.16:0/1262285700 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.166+0000 7f7364ecf640 1 --2- 172.21.15.16:0/1262285700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73600d3b90 0x7f73600cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.166+0000 7f73656d0640 1 --2- 172.21.15.16:0/1262285700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73600cee80 0x7f7360157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.166+0000 7f7365ed1640 1 -- 172.21.15.16:0/1262285700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73600d3b90 msgr2=0x7f73600cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.166+0000 7f7365ed1640 1 --2- 172.21.15.16:0/1262285700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73600d3b90 0x7f73600cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.166+0000 7f7365ed1640 1 -- 172.21.15.16:0/1262285700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73600cee80 msgr2=0x7f7360157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.166+0000 7f7365ed1640 1 --2- 172.21.15.16:0/1262285700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73600cee80 0x7f7360157170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.166+0000 7f7365ed1640 1 -- 172.21.15.16:0/1262285700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73601576b0 con 0x7f736015dca0 2024-04-25T10:09:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.167+0000 7f7365ed1640 1 --2- 172.21.15.16:0/1262285700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f736015dca0 0x7f7360159bc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f735800c7a0 tx=0x7f735800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.167+0000 7f7355ffb640 1 -- 172.21.15.16:0/1262285700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358018070 con 0x7f736015dca0 2024-04-25T10:09:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.167+0000 7f7355ffb640 1 -- 172.21.15.16:0/1262285700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735800a640 con 0x7f736015dca0 2024-04-25T10:09:04.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.167+0000 7f7367934640 1 -- 172.21.15.16:0/1262285700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73600cdb20 con 0x7f736015dca0 2024-04-25T10:09:04.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.167+0000 7f7355ffb640 1 -- 172.21.15.16:0/1262285700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735800aab0 con 0x7f736015dca0 2024-04-25T10:09:04.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.167+0000 7f7367934640 1 -- 172.21.15.16:0/1262285700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73600cbce0 con 0x7f736015dca0 2024-04-25T10:09:04.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.169+0000 7f7355ffb640 1 -- 172.21.15.16:0/1262285700 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f735801b070 con 0x7f736015dca0 2024-04-25T10:09:04.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.169+0000 7f7355ffb640 1 --2- 172.21.15.16:0/1262285700 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f734406b500 0x7f734406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.169+0000 7f7355ffb640 1 -- 172.21.15.16:0/1262285700 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7358089740 con 0x7f736015dca0 2024-04-25T10:09:04.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.169+0000 7f73656d0640 1 --2- 172.21.15.16:0/1262285700 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f734406b500 0x7f734406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.170+0000 7f73656d0640 1 --2- 172.21.15.16:0/1262285700 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f734406b500 0x7f734406d9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f735c0079a0 tx=0x7f735c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.170+0000 7f736a63b740 1 Processor -- start 2024-04-25T10:09:04.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.170+0000 7f736a63b740 1 -- start start 2024-04-25T10:09:04.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.171+0000 7f736a63b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607b10 0x558c6e60a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.171+0000 7f736a63b740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e607ef0 0x558c6e60a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.171+0000 7f736a63b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e5da770 0x558c6e5ddf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.171+0000 7f736a63b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c6e543da0 con 0x558c6e607b10 2024-04-25T10:09:04.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.171+0000 7f736a63b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c6e4ecb80 con 0x558c6e5da770 2024-04-25T10:09:04.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.171+0000 7f736a63b740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c6e53fe40 con 0x558c6e607ef0 2024-04-25T10:09:04.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.171+0000 7f73656d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607b10 0x558c6e60a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.171+0000 7f73656d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607b10 0x558c6e60a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40886/0 (socket says 172.21.15.16:40886) 2024-04-25T10:09:04.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.171+0000 7f73656d0640 1 -- 172.21.15.16:0/1328403184 learned_addr learned my addr 172.21.15.16:0/1328403184 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:04.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.171+0000 7f7364ecf640 1 --2- 172.21.15.16:0/1328403184 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e607ef0 0x558c6e60a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.171+0000 7f7365ed1640 1 --2- 172.21.15.16:0/1328403184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e5da770 0x558c6e5ddf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.171+0000 7f73656d0640 1 -- 172.21.15.16:0/1328403184 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e607ef0 msgr2=0x558c6e60a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.171+0000 7f73656d0640 1 --2- 172.21.15.16:0/1328403184 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e607ef0 0x558c6e60a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.171+0000 7f73656d0640 1 -- 172.21.15.16:0/1328403184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e5da770 msgr2=0x558c6e5ddf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.171+0000 7f73656d0640 1 --2- 172.21.15.16:0/1328403184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e5da770 0x558c6e5ddf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.171+0000 7f73656d0640 1 -- 172.21.15.16:0/1328403184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c6e4f6f10 con 0x558c6e607b10 2024-04-25T10:09:04.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.171+0000 7f7364ecf640 1 --2- 172.21.15.16:0/1328403184 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e607ef0 0x558c6e60a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:04.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.171+0000 7f7365ed1640 1 --2- 172.21.15.16:0/1328403184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e5da770 0x558c6e5ddf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:04.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.172+0000 7f73656d0640 1 --2- 172.21.15.16:0/1328403184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607b10 0x558c6e60a320 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f735c010ac0 tx=0x7f735c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe0d4fc23fd675 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.172+0000 7f734affd640 1 -- 172.21.15.16:0/1328403184 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c01c050 con 0x558c6e607b10 2024-04-25T10:09:04.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.172+0000 7f734affd640 1 -- 172.21.15.16:0/1328403184 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735c026070 con 0x558c6e607b10 2024-04-25T10:09:04.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.172+0000 7f734affd640 1 -- 172.21.15.16:0/1328403184 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c016e10 con 0x558c6e607b10 2024-04-25T10:09:04.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.172+0000 7f736a63b740 1 -- 172.21.15.16:0/1328403184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607b10 msgr2=0x558c6e60a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.172+0000 7f736a63b740 1 --2- 172.21.15.16:0/1328403184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607b10 0x558c6e60a320 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f735c010ac0 tx=0x7f735c013000 comp rx=0 tx=0).stop 2024-04-25T10:09:04.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.172+0000 7f736a63b740 1 -- 172.21.15.16:0/1328403184 shutdown_connections 2024-04-25T10:09:04.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.172+0000 7f736a63b740 1 --2- 172.21.15.16:0/1328403184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e5da770 0x558c6e5ddf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.172+0000 7f736a63b740 1 --2- 172.21.15.16:0/1328403184 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e607ef0 0x558c6e60a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.172+0000 7f736a63b740 1 --2- 172.21.15.16:0/1328403184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607b10 0x558c6e60a320 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.172+0000 7f736a63b740 1 -- 172.21.15.16:0/1328403184 >> 172.21.15.16:0/1328403184 conn(0x558c6e5dc9e0 msgr2=0x558c6e625320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.172+0000 7f736a63b740 1 -- 172.21.15.16:0/1328403184 shutdown_connections 2024-04-25T10:09:04.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.172+0000 7f736a63b740 1 -- 172.21.15.16:0/1328403184 wait complete. 2024-04-25T10:09:04.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.173+0000 7f736a63b740 1 -- start start 2024-04-25T10:09:04.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.173+0000 7f736a63b740 1 Processor -- start 2024-04-25T10:09:04.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.174+0000 7f736a63b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607b10 0x558c6e60e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.174+0000 7f736a63b740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e607ef0 0x558c6e5d9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.174+0000 7f736a63b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e5fd360 0x558c6e5d9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.174+0000 7f736a63b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c6e5dfb90 con 0x558c6e607b10 2024-04-25T10:09:04.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.174+0000 7f736a63b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c6e4e24b0 con 0x558c6e5fd360 2024-04-25T10:09:04.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.174+0000 7f736a63b740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c6e5dfe70 con 0x558c6e607ef0 2024-04-25T10:09:04.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.174+0000 7f736a63b740 10 client.0 _ll_register_callbacks cb 0x558c6e522180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:04.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.174+0000 7f73656d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607b10 0x558c6e60e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.174+0000 7f73656d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607b10 0x558c6e60e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40890/0 (socket says 172.21.15.16:40890) 2024-04-25T10:09:04.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.174+0000 7f73656d0640 1 -- 172.21.15.16:0/2537331691 learned_addr learned my addr 172.21.15.16:0/2537331691 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:04.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.174+0000 7f7364ecf640 1 --2- 172.21.15.16:0/2537331691 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e607ef0 0x558c6e5d9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.174+0000 7f7365ed1640 1 --2- 172.21.15.16:0/2537331691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e5fd360 0x558c6e5d9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.174+0000 7f73656d0640 1 -- 172.21.15.16:0/2537331691 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e607ef0 msgr2=0x558c6e5d9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.174+0000 7f73656d0640 1 --2- 172.21.15.16:0/2537331691 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e607ef0 0x558c6e5d9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.174+0000 7f73656d0640 1 -- 172.21.15.16:0/2537331691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e5fd360 msgr2=0x558c6e5d9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.174+0000 7f73656d0640 1 --2- 172.21.15.16:0/2537331691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e5fd360 0x558c6e5d9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.174+0000 7f73656d0640 1 -- 172.21.15.16:0/2537331691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c6e3227d0 con 0x558c6e607b10 2024-04-25T10:09:04.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.174+0000 7f7365ed1640 1 --2- 172.21.15.16:0/2537331691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e5fd360 0x558c6e5d9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:04.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.175+0000 7f73656d0640 1 --2- 172.21.15.16:0/2537331691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607b10 0x558c6e60e150 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x558c6e5e2070 tx=0x7f735c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1088cb8647b0fafe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.175+0000 7f733ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:09:04.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.175+0000 7f733ffff640 1 -- 172.21.15.16:0/2537331691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c021850 con 0x558c6e607b10 2024-04-25T10:09:04.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.175+0000 7f733ffff640 1 -- 172.21.15.16:0/2537331691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f735c035b60 con 0x558c6e607b10 2024-04-25T10:09:04.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.175+0000 7f736a63b740 1 -- 172.21.15.16:0/2537331691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f735c026070 con 0x558c6e607b10 2024-04-25T10:09:04.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.175+0000 7f733ffff640 1 -- 172.21.15.16:0/2537331691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f735c035d60 con 0x558c6e607b10 2024-04-25T10:09:04.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.175+0000 7f736a63b740 10 client.6414 fetch_fsmap learned FSMap version 29 2024-04-25T10:09:04.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.175+0000 7f736a63b740 1 -- 172.21.15.16:0/2537331691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x558c6e2583f0 con 0x558c6e607b10 2024-04-25T10:09:04.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.175+0000 7f733ffff640 1 -- 172.21.15.16:0/2537331691 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735c026070 con 0x558c6e607b10 2024-04-25T10:09:04.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.175+0000 7f733ffff640 1 -- 172.21.15.16:0/2537331691 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f735c0219f0 con 0x558c6e607b10 2024-04-25T10:09:04.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.175+0000 7f736a63b740 10 client.6414 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:09:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.175+0000 7f736a63b740 10 client.6414 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:09:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.175+0000 7f736a63b740 20 client.6414 populate_metadata read hostname 'smithi016' 2024-04-25T10:09:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.175+0000 7f736a63b740 1 --2- 172.21.15.16:0/2537331691 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x558c6e627d70 0x558c6e628170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.175+0000 7f736a63b740 4 client.6414 mds_command: new command op to 35141 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.175+0000 7f736a63b740 1 -- 172.21.15.16:0/2537331691 --> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c6e5512e0 con 0x558c6e627d70 2024-04-25T10:09:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.175+0000 7f7364ecf640 1 --2- 172.21.15.16:0/2537331691 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x558c6e627d70 0x558c6e628170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.176+0000 7f733ffff640 10 client.6414 ms_handle_connect on v2:172.21.15.16:6832/3311584521 2024-04-25T10:09:04.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.176+0000 7f7364ecf640 1 --2- 172.21.15.16:0/2537331691 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x558c6e627d70 0x558c6e628170 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fff19f1ba0472ac4 server_cookie=ef0489436990ff52 in_seq=0 out_seq=0 2024-04-25T10:09:04.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.176+0000 7f733ffff640 1 -- 172.21.15.16:0/2537331691 <== mds.0 v2:172.21.15.16:6832/3311584521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558c6e5512e0 con 0x558c6e627d70 2024-04-25T10:09:04.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.176+0000 7f733ffff640 10 client.6414 handle_command_reply: tid=0 2024-04-25T10:09:04.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.176+0000 7f736a63b740 1 client.6414 shutdown 2024-04-25T10:09:04.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.177+0000 7f736a63b740 1 -- 172.21.15.16:0/2537331691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607b10 msgr2=0x558c6e60e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.177+0000 7f736a63b740 1 --2- 172.21.15.16:0/2537331691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607b10 0x558c6e60e150 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x558c6e5e2070 tx=0x7f735c004730 comp rx=0 tx=0).stop 2024-04-25T10:09:04.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.177+0000 7f736a63b740 1 -- 172.21.15.16:0/2537331691 shutdown_connections 2024-04-25T10:09:04.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.177+0000 7f736a63b740 1 --2- 172.21.15.16:0/2537331691 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x558c6e627d70 0x558c6e628170 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.177+0000 7f736a63b740 1 --2- 172.21.15.16:0/2537331691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e5fd360 0x558c6e5d9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.177+0000 7f736a63b740 1 --2- 172.21.15.16:0/2537331691 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e607ef0 0x558c6e5d9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.177+0000 7f733ffff640 0 client.6414 ms_handle_reset on v2:172.21.15.16:6832/3311584521 2024-04-25T10:09:04.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.177+0000 7f736a63b740 1 --2- 172.21.15.16:0/2537331691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607b10 0x558c6e60e150 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.177+0000 7f736a63b740 1 -- 172.21.15.16:0/2537331691 >> 172.21.15.16:0/2537331691 conn(0x558c6e5da770 msgr2=0x558c6e605990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.177+0000 7f736a63b740 1 -- 172.21.15.16:0/2537331691 shutdown_connections 2024-04-25T10:09:04.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.177+0000 7f736a63b740 1 -- 172.21.15.16:0/2537331691 wait complete. 2024-04-25T10:09:04.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.177+0000 7f736a63b740 20 client.6414 trim_cache size 0 max 16384 2024-04-25T10:09:04.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.192+0000 7f736a63b740 1 Processor -- start 2024-04-25T10:09:04.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.193+0000 7f736a63b740 1 -- start start 2024-04-25T10:09:04.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.193+0000 7f736a63b740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e5fd360 0x558c6e624bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.193+0000 7f736a63b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e607b10 0x558c6e5de280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.193+0000 7f736a63b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607ef0 0x558c6e605930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.193+0000 7f736a63b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c6e63f460 con 0x558c6e607ef0 2024-04-25T10:09:04.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.193+0000 7f736a63b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73601ffe60 con 0x558c6e607b10 2024-04-25T10:09:04.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.193+0000 7f736a63b740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73601f51e0 con 0x558c6e5fd360 2024-04-25T10:09:04.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.193+0000 7f7365ed1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607ef0 0x558c6e605930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.193+0000 7f7365ed1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607ef0 0x558c6e605930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40894/0 (socket says 172.21.15.16:40894) 2024-04-25T10:09:04.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.193+0000 7f7365ed1640 1 -- 172.21.15.16:0/1298124314 learned_addr learned my addr 172.21.15.16:0/1298124314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:04.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.194+0000 7f73656d0640 1 --2- 172.21.15.16:0/1298124314 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e5fd360 0x558c6e624bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.194+0000 7f7364ecf640 1 --2- 172.21.15.16:0/1298124314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e607b10 0x558c6e5de280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.194+0000 7f7365ed1640 1 -- 172.21.15.16:0/1298124314 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e5fd360 msgr2=0x558c6e624bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.194+0000 7f7365ed1640 1 --2- 172.21.15.16:0/1298124314 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e5fd360 0x558c6e624bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.194+0000 7f7365ed1640 1 -- 172.21.15.16:0/1298124314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e607b10 msgr2=0x558c6e5de280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.194+0000 7f7365ed1640 1 --2- 172.21.15.16:0/1298124314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e607b10 0x558c6e5de280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.194+0000 7f7365ed1640 1 -- 172.21.15.16:0/1298124314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c6e5db6c0 con 0x558c6e607ef0 2024-04-25T10:09:04.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.194+0000 7f7364ecf640 1 --2- 172.21.15.16:0/1298124314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e607b10 0x558c6e5de280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:04.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.194+0000 7f73656d0640 1 --2- 172.21.15.16:0/1298124314 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e5fd360 0x558c6e624bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:04.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.194+0000 7f7365ed1640 1 --2- 172.21.15.16:0/1298124314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607ef0 0x558c6e605930 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7358001330 tx=0x7f7358057890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610813532e3c031 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.194+0000 7f734a7fc640 1 -- 172.21.15.16:0/1298124314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358060590 con 0x558c6e607ef0 2024-04-25T10:09:04.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.194+0000 7f734a7fc640 1 -- 172.21.15.16:0/1298124314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7358056230 con 0x558c6e607ef0 2024-04-25T10:09:04.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.194+0000 7f734a7fc640 1 -- 172.21.15.16:0/1298124314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735800ac50 con 0x558c6e607ef0 2024-04-25T10:09:04.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.195+0000 7f736a63b740 1 -- 172.21.15.16:0/1298124314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607ef0 msgr2=0x558c6e605930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.195+0000 7f736a63b740 1 --2- 172.21.15.16:0/1298124314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607ef0 0x558c6e605930 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7358001330 tx=0x7f7358057890 comp rx=0 tx=0).stop 2024-04-25T10:09:04.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.195+0000 7f736a63b740 1 -- 172.21.15.16:0/1298124314 shutdown_connections 2024-04-25T10:09:04.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.195+0000 7f736a63b740 1 --2- 172.21.15.16:0/1298124314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607ef0 0x558c6e605930 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.195+0000 7f736a63b740 1 --2- 172.21.15.16:0/1298124314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e607b10 0x558c6e5de280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.195+0000 7f736a63b740 1 --2- 172.21.15.16:0/1298124314 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e5fd360 0x558c6e624bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.195+0000 7f736a63b740 1 -- 172.21.15.16:0/1298124314 >> 172.21.15.16:0/1298124314 conn(0x558c6e5da770 msgr2=0x558c6e5dfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.195+0000 7f736a63b740 1 -- 172.21.15.16:0/1298124314 shutdown_connections 2024-04-25T10:09:04.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.195+0000 7f736a63b740 1 -- 172.21.15.16:0/1298124314 wait complete. 2024-04-25T10:09:04.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.195+0000 7f736a63b740 1 -- start start 2024-04-25T10:09:04.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.195+0000 7f736a63b740 1 Processor -- start 2024-04-25T10:09:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.196+0000 7f736a63b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e5fd360 0x558c6e678000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.196+0000 7f736a63b740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e607b10 0x558c6e5d9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.196+0000 7f736a63b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607ef0 0x558c6e5d9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.196+0000 7f736a63b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73601f4be0 con 0x558c6e607ef0 2024-04-25T10:09:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.196+0000 7f736a63b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7360158970 con 0x558c6e5fd360 2024-04-25T10:09:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.196+0000 7f736a63b740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c6e627010 con 0x558c6e607b10 2024-04-25T10:09:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.196+0000 7f736a63b740 10 client.0 _ll_register_callbacks cb 0x7f735c0109c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.196+0000 7f7365ed1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607ef0 0x558c6e5d9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.196+0000 7f7365ed1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607ef0 0x558c6e5d9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40904/0 (socket says 172.21.15.16:40904) 2024-04-25T10:09:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.196+0000 7f7365ed1640 1 -- 172.21.15.16:0/3608581966 learned_addr learned my addr 172.21.15.16:0/3608581966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.196+0000 7f7364ecf640 1 --2- 172.21.15.16:0/3608581966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e607b10 0x558c6e5d9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.196+0000 7f73656d0640 1 --2- 172.21.15.16:0/3608581966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e5fd360 0x558c6e678000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f7365ed1640 1 -- 172.21.15.16:0/3608581966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e607b10 msgr2=0x558c6e5d9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f7365ed1640 1 --2- 172.21.15.16:0/3608581966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e607b10 0x558c6e5d9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f7365ed1640 1 -- 172.21.15.16:0/3608581966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e5fd360 msgr2=0x558c6e678000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f7365ed1640 1 --2- 172.21.15.16:0/3608581966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e5fd360 0x558c6e678000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f7365ed1640 1 -- 172.21.15.16:0/3608581966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c6e2180f0 con 0x558c6e607ef0 2024-04-25T10:09:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f7365ed1640 1 --2- 172.21.15.16:0/3608581966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607ef0 0x558c6e5d9b40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f73580012e0 tx=0x7f7358014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d81702fe3d4b672 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f733ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:09:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f733ffff640 1 -- 172.21.15.16:0/3608581966 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735800ae60 con 0x558c6e607ef0 2024-04-25T10:09:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f733ffff640 1 -- 172.21.15.16:0/3608581966 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7358060300 con 0x558c6e607ef0 2024-04-25T10:09:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f736a63b740 1 -- 172.21.15.16:0/3608581966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7358056230 con 0x558c6e607ef0 2024-04-25T10:09:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f733ffff640 1 -- 172.21.15.16:0/3608581966 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7358057380 con 0x558c6e607ef0 2024-04-25T10:09:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f736a63b740 10 client.6420 fetch_fsmap learned FSMap version 29 2024-04-25T10:09:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f736a63b740 1 -- 172.21.15.16:0/3608581966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x558c6e60d0e0 con 0x558c6e607ef0 2024-04-25T10:09:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f733ffff640 1 -- 172.21.15.16:0/3608581966 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7358056230 con 0x558c6e607ef0 2024-04-25T10:09:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f733ffff640 1 -- 172.21.15.16:0/3608581966 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7358057520 con 0x558c6e607ef0 2024-04-25T10:09:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f736a63b740 10 client.6420 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:09:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f736a63b740 10 client.6420 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:09:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f736a63b740 20 client.6420 populate_metadata read hostname 'smithi016' 2024-04-25T10:09:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f736a63b740 1 --2- 172.21.15.16:0/3608581966 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x558c6e6224a0 0x558c6e690050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f736a63b740 4 client.6420 mds_command: new command op to 35141 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:09:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f736a63b740 1 -- 172.21.15.16:0/3608581966 --> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558c6e5fe830 con 0x558c6e6224a0 2024-04-25T10:09:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.197+0000 7f73656d0640 1 --2- 172.21.15.16:0/3608581966 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x558c6e6224a0 0x558c6e690050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.198+0000 7f733ffff640 10 client.6420 ms_handle_connect on v2:172.21.15.16:6832/3311584521 2024-04-25T10:09:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.198+0000 7f73656d0640 1 --2- 172.21.15.16:0/3608581966 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x558c6e6224a0 0x558c6e690050 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34f8c6a89d925fd7 server_cookie=1fa5eaf00d2f5d23 in_seq=0 out_seq=0 2024-04-25T10:09:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.198+0000 7f733ffff640 1 -- 172.21.15.16:0/3608581966 <== mds.0 v2:172.21.15.16:6832/3311584521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558c6e5fe830 con 0x558c6e6224a0 2024-04-25T10:09:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.198+0000 7f733ffff640 10 client.6420 handle_command_reply: tid=0 2024-04-25T10:09:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.198+0000 7f736a63b740 1 client.6420 shutdown 2024-04-25T10:09:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.198+0000 7f736a63b740 1 -- 172.21.15.16:0/3608581966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607ef0 msgr2=0x558c6e5d9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.198+0000 7f736a63b740 1 --2- 172.21.15.16:0/3608581966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607ef0 0x558c6e5d9b40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f73580012e0 tx=0x7f7358014040 comp rx=0 tx=0).stop 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.198+0000 7f736a63b740 1 -- 172.21.15.16:0/3608581966 shutdown_connections 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.198+0000 7f736a63b740 1 --2- 172.21.15.16:0/3608581966 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x558c6e6224a0 0x558c6e690050 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.198+0000 7f736a63b740 1 --2- 172.21.15.16:0/3608581966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c6e607ef0 0x558c6e5d9b40 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.198+0000 7f736a63b740 1 --2- 172.21.15.16:0/3608581966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c6e607b10 0x558c6e5d9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.198+0000 7f733ffff640 0 client.6420 ms_handle_reset on v2:172.21.15.16:6832/3311584521 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.198+0000 7f736a63b740 1 --2- 172.21.15.16:0/3608581966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c6e5fd360 0x558c6e678000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.198+0000 7f736a63b740 1 -- 172.21.15.16:0/3608581966 >> 172.21.15.16:0/3608581966 conn(0x558c6e5da770 msgr2=0x558c6e5e1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.198+0000 7f736a63b740 1 -- 172.21.15.16:0/3608581966 shutdown_connections 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.198+0000 7f736a63b740 1 -- 172.21.15.16:0/3608581966 wait complete. 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.198+0000 7f736a63b740 20 client.6420 trim_cache size 0 max 16384 2024-04-25T10:09:04.212 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:09:04.212 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-04-25T10:09:04.212 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-04-25T10:09:04.212 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:09:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.199+0000 7f7367934640 1 -- 172.21.15.16:0/1262285700 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f734406b500 msgr2=0x7f734406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.199+0000 7f7367934640 1 --2- 172.21.15.16:0/1262285700 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f734406b500 0x7f734406d9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f735c0079a0 tx=0x7f735c008040 comp rx=0 tx=0).stop 2024-04-25T10:09:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.199+0000 7f7367934640 1 -- 172.21.15.16:0/1262285700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f736015dca0 msgr2=0x7f7360159bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.199+0000 7f7367934640 1 --2- 172.21.15.16:0/1262285700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f736015dca0 0x7f7360159bc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f735800c7a0 tx=0x7f735800cc60 comp rx=0 tx=0).stop 2024-04-25T10:09:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.199+0000 7f7367934640 1 -- 172.21.15.16:0/1262285700 shutdown_connections 2024-04-25T10:09:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.199+0000 7f7367934640 1 --2- 172.21.15.16:0/1262285700 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f734406b500 0x7f734406d9c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.199+0000 7f7367934640 1 --2- 172.21.15.16:0/1262285700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f736015dca0 0x7f7360159bc0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.199+0000 7f7367934640 1 --2- 172.21.15.16:0/1262285700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f73600d3b90 0x7f73600cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.199+0000 7f7367934640 1 --2- 172.21.15.16:0/1262285700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73600cee80 0x7f7360157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.199+0000 7f7367934640 1 -- 172.21.15.16:0/1262285700 >> 172.21.15.16:0/1262285700 conn(0x7f73600726c0 msgr2=0x7f736015cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.199+0000 7f7367934640 1 -- 172.21.15.16:0/1262285700 shutdown_connections 2024-04-25T10:09:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.199+0000 7f7367934640 1 -- 172.21.15.16:0/1262285700 wait complete. 2024-04-25T10:09:04.216 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:09:04.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.311+0000 7f3f2061b640 1 Processor -- start 2024-04-25T10:09:04.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.312+0000 7f3f2061b640 1 -- start start 2024-04-25T10:09:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.312+0000 7f3f2061b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f180d35b0 0x7f3f180b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.312+0000 7f3f2061b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f180b5990 0x7f3f180b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.312+0000 7f3f2061b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1815e0c0 con 0x7f3f1815dce0 2024-04-25T10:09:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.312+0000 7f3f2061b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f180d2510 con 0x7f3f180d35b0 2024-04-25T10:09:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.312+0000 7f3f2061b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f180d27f0 con 0x7f3f180b5990 2024-04-25T10:09:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.313+0000 7f3f1dbb6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f180d35b0 0x7f3f180b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.313+0000 7f3f1e3b7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3f1815dce0 0x7f3f180d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32812/0 (socket says 172.21.15.16:32812) 2024-04-25T10:09:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.313+0000 7f3f1e3b7640 1 -- 172.21.15.16:0/3212614834 learned_addr learned my addr 172.21.15.16:0/3212614834 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.313+0000 7f3f1ebb8640 1 --2- 172.21.15.16:0/3212614834 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f180b5990 0x7f3f180b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.313+0000 7f3f1dbb6640 1 -- 172.21.15.16:0/3212614834 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f180b5990 msgr2=0x7f3f180b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.313+0000 7f3f1dbb6640 1 --2- 172.21.15.16:0/3212614834 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f180b5990 0x7f3f180b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.313+0000 7f3f1dbb6640 1 -- 172.21.15.16:0/3212614834 >> v1:172.21.15.16:6789/0 conn(0x7f3f1815dce0 legacy=0x7f3f180d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.313+0000 7f3f1d3b5640 1 -- 172.21.15.16:0/3212614834 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516412323 0 0) 0x7f3f1815e0c0 con 0x7f3f1815dce0 2024-04-25T10:09:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.313+0000 7f3f1dbb6640 1 -- 172.21.15.16:0/3212614834 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f180d2ad0 con 0x7f3f180d35b0 2024-04-25T10:09:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.314+0000 7f3f1dbb6640 1 --2- 172.21.15.16:0/3212614834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f180d35b0 0x7f3f180b5450 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3f080097a0 tx=0x7f3f0802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efeea82d7b0fade6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.314+0000 7f3f1d3b5640 1 -- 172.21.15.16:0/3212614834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f08037070 con 0x7f3f180d35b0 2024-04-25T10:09:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.314+0000 7f3f1d3b5640 1 -- 172.21.15.16:0/3212614834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f08032440 con 0x7f3f180d35b0 2024-04-25T10:09:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.314+0000 7f3f1d3b5640 1 -- 172.21.15.16:0/3212614834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f08032810 con 0x7f3f180d35b0 2024-04-25T10:09:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.314+0000 7f3f2061b640 1 -- 172.21.15.16:0/3212614834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f180d35b0 msgr2=0x7f3f180b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.314+0000 7f3f2061b640 1 --2- 172.21.15.16:0/3212614834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f180d35b0 0x7f3f180b5450 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3f080097a0 tx=0x7f3f0802a180 comp rx=0 tx=0).stop 2024-04-25T10:09:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.314+0000 7f3f2061b640 1 -- 172.21.15.16:0/3212614834 shutdown_connections 2024-04-25T10:09:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.314+0000 7f3f2061b640 1 --2- 172.21.15.16:0/3212614834 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f180b5990 0x7f3f180b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.314+0000 7f3f2061b640 1 --2- 172.21.15.16:0/3212614834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f180d35b0 0x7f3f180b5450 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.314+0000 7f3f2061b640 1 -- 172.21.15.16:0/3212614834 >> 172.21.15.16:0/3212614834 conn(0x7f3f18072900 msgr2=0x7f3f180d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.315+0000 7f3f2061b640 1 -- 172.21.15.16:0/3212614834 shutdown_connections 2024-04-25T10:09:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.315+0000 7f3f2061b640 1 -- 172.21.15.16:0/3212614834 wait complete. 2024-04-25T10:09:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.316+0000 7f3f2061b640 1 Processor -- start 2024-04-25T10:09:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.316+0000 7f3f2061b640 1 -- start start 2024-04-25T10:09:04.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.316+0000 7f3f2061b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f180b5990 0x7f3f1815a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.316+0000 7f3f2061b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f180d35b0 0x7f3f18156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.316+0000 7f3f2061b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f1815dce0 0x7f3f18156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.316+0000 7f3f2061b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f180d2160 con 0x7f3f1815dce0 2024-04-25T10:09:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.316+0000 7f3f2061b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f180d3090 con 0x7f3f180b5990 2024-04-25T10:09:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.316+0000 7f3f2061b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f181f77c0 con 0x7f3f180d35b0 2024-04-25T10:09:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.317+0000 7f3f1ebb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f1815dce0 0x7f3f18156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.317+0000 7f3f1ebb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f1815dce0 0x7f3f18156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40914/0 (socket says 172.21.15.16:40914) 2024-04-25T10:09:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.317+0000 7f3f1ebb8640 1 -- 172.21.15.16:0/2479567681 learned_addr learned my addr 172.21.15.16:0/2479567681 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.317+0000 7f3f1dbb6640 1 --2- 172.21.15.16:0/2479567681 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f180d35b0 0x7f3f18156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.317+0000 7f3f1e3b7640 1 --2- 172.21.15.16:0/2479567681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f180b5990 0x7f3f1815a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.317+0000 7f3f1ebb8640 1 -- 172.21.15.16:0/2479567681 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f180d35b0 msgr2=0x7f3f18156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.317+0000 7f3f1ebb8640 1 --2- 172.21.15.16:0/2479567681 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f180d35b0 0x7f3f18156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.317+0000 7f3f1ebb8640 1 -- 172.21.15.16:0/2479567681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f180b5990 msgr2=0x7f3f1815a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.317+0000 7f3f1ebb8640 1 --2- 172.21.15.16:0/2479567681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f180b5990 0x7f3f1815a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.317+0000 7f3f1ebb8640 1 -- 172.21.15.16:0/2479567681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1815e2b0 con 0x7f3f1815dce0 2024-04-25T10:09:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.317+0000 7f3f1dbb6640 1 --2- 172.21.15.16:0/2479567681 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f180d35b0 0x7f3f18156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.317+0000 7f3f1e3b7640 1 --2- 172.21.15.16:0/2479567681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f180b5990 0x7f3f1815a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.317+0000 7f3f1ebb8640 1 --2- 172.21.15.16:0/2479567681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f1815dce0 0x7f3f18156f40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000cd70 tx=0x7f3f1000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.317+0000 7f3f0effd640 1 -- 172.21.15.16:0/2479567681 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10003b90 con 0x7f3f1815dce0 2024-04-25T10:09:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.317+0000 7f3f0effd640 1 -- 172.21.15.16:0/2479567681 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f10003d30 con 0x7f3f1815dce0 2024-04-25T10:09:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.317+0000 7f3f2061b640 1 -- 172.21.15.16:0/2479567681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1815e590 con 0x7f3f1815dce0 2024-04-25T10:09:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.317+0000 7f3f0effd640 1 -- 172.21.15.16:0/2479567681 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10004240 con 0x7f3f1815dce0 2024-04-25T10:09:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.317+0000 7f3f2061b640 1 -- 172.21.15.16:0/2479567681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1815eb50 con 0x7f3f1815dce0 2024-04-25T10:09:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.318+0000 7f3f2061b640 1 -- 172.21.15.16:0/2479567681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f180749b0 con 0x7f3f1815dce0 2024-04-25T10:09:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.319+0000 7f3f0effd640 1 -- 172.21.15.16:0/2479567681 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f100043e0 con 0x7f3f1815dce0 2024-04-25T10:09:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.319+0000 7f3f0effd640 1 --2- 172.21.15.16:0/2479567681 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3efc06b4d0 0x7f3efc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.319+0000 7f3f0effd640 1 -- 172.21.15.16:0/2479567681 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3f1008acd0 con 0x7f3f1815dce0 2024-04-25T10:09:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.319+0000 7f3f1e3b7640 1 --2- 172.21.15.16:0/2479567681 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3efc06b4d0 0x7f3efc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.320+0000 7f3f1e3b7640 1 --2- 172.21.15.16:0/2479567681 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3efc06b4d0 0x7f3efc06d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3f14000f60 tx=0x7f3f14009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.323+0000 7f3f0effd640 1 -- 172.21.15.16:0/2479567681 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f10059ba0 con 0x7f3f1815dce0 2024-04-25T10:09:04.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.541+0000 7f3f2061b640 1 -- 172.21.15.16:0/2479567681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f180b7650 con 0x7f3f1815dce0 2024-04-25T10:09:04.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.541+0000 7f3f0effd640 1 -- 172.21.15.16:0/2479567681 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f3f100594a0 con 0x7f3f1815dce0 2024-04-25T10:09:04.547 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:04.547 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2270839865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2270839865},{"type":"v1","addr":"172.21.15.186:6836","nonce":2270839865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5647,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/925282579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":925282579},{"type":"v1","addr":"172.21.15.186:6837","nonce":925282579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2471714328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2471714328},{"type":"v1","addr":"172.21.15.16:6835","nonce":2471714328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:50.665436+0000","modified":"2024-04-25T10:08:39.328915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35141":{"gid":35141,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.16:6834/3311584521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3311584521},{"type":"v1","addr":"172.21.15.16:6834","nonce":3311584521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:09:04.548 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-04-25T10:09:04.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.543+0000 7f3f2061b640 1 -- 172.21.15.16:0/2479567681 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3efc06b4d0 msgr2=0x7f3efc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.543+0000 7f3f2061b640 1 --2- 172.21.15.16:0/2479567681 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3efc06b4d0 0x7f3efc06d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3f14000f60 tx=0x7f3f14009040 comp rx=0 tx=0).stop 2024-04-25T10:09:04.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.543+0000 7f3f2061b640 1 -- 172.21.15.16:0/2479567681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f1815dce0 msgr2=0x7f3f18156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.543+0000 7f3f2061b640 1 --2- 172.21.15.16:0/2479567681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f1815dce0 0x7f3f18156f40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000cd70 tx=0x7f3f1000a660 comp rx=0 tx=0).stop 2024-04-25T10:09:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.543+0000 7f3f2061b640 1 -- 172.21.15.16:0/2479567681 shutdown_connections 2024-04-25T10:09:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.543+0000 7f3f2061b640 1 --2- 172.21.15.16:0/2479567681 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3efc06b4d0 0x7f3efc06d990 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.543+0000 7f3f2061b640 1 --2- 172.21.15.16:0/2479567681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f1815dce0 0x7f3f18156f40 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.543+0000 7f3f2061b640 1 --2- 172.21.15.16:0/2479567681 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f180d35b0 0x7f3f18156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.543+0000 7f3f2061b640 1 --2- 172.21.15.16:0/2479567681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f180b5990 0x7f3f1815a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.543+0000 7f3f2061b640 1 -- 172.21.15.16:0/2479567681 >> 172.21.15.16:0/2479567681 conn(0x7f3f18072900 msgr2=0x7f3f180b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.543+0000 7f3f2061b640 1 -- 172.21.15.16:0/2479567681 shutdown_connections 2024-04-25T10:09:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.543+0000 7f3f2061b640 1 -- 172.21.15.16:0/2479567681 wait complete. 2024-04-25T10:09:04.560 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:09:04.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.656+0000 7fed74f70640 1 Processor -- start 2024-04-25T10:09:04.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.656+0000 7fed74f70640 1 -- start start 2024-04-25T10:09:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.657+0000 7fed74f70640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fed7015dca0 0x7fed700ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.657+0000 7fed74f70640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed700d4090 con 0x7fed700d3b90 2024-04-25T10:09:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.657+0000 7fed74f70640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed700d4370 con 0x7fed700d1130 2024-04-25T10:09:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.657+0000 7fed74f70640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed700b6b20 con 0x7fed7015dca0 2024-04-25T10:09:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.657+0000 7fed65d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fed700d3b90 0x7fed700d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32816/0 (socket says 172.21.15.16:32816) 2024-04-25T10:09:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.657+0000 7fed65d9b640 1 -- 172.21.15.16:0/3215770818 learned_addr learned my addr 172.21.15.16:0/3215770818 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.657+0000 7fed6e59c640 1 --2- 172.21.15.16:0/3215770818 >> v2:172.21.15.186:3300/0 conn(0x7fed7015dca0 0x7fed700ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.657+0000 7fed6dd9b640 1 -- 172.21.15.16:0/3215770818 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3234113874 0 0) 0x7fed700d4090 con 0x7fed700d3b90 2024-04-25T10:09:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.657+0000 7fed6dd9b640 1 -- 172.21.15.16:0/3215770818 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed5c003520 con 0x7fed700d3b90 2024-04-25T10:09:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.658+0000 7fed6e59c640 1 -- 172.21.15.16:0/3215770818 >> v1:172.21.15.186:6789/0 conn(0x7fed700d1130 legacy=0x7fed700b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.658+0000 7fed6dd9b640 1 -- 172.21.15.16:0/3215770818 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995012787 0 0) 0x7fed700d4370 con 0x7fed700d1130 2024-04-25T10:09:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.658+0000 7fed6e59c640 1 -- 172.21.15.16:0/3215770818 >> v1:172.21.15.16:6789/0 conn(0x7fed700d3b90 legacy=0x7fed700d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.658+0000 7fed6e59c640 1 -- 172.21.15.16:0/3215770818 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed700d4650 con 0x7fed7015dca0 2024-04-25T10:09:04.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.658+0000 7fed6e59c640 1 --2- 172.21.15.16:0/3215770818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed7015dca0 0x7fed700ce940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fed600097a0 tx=0x7fed6002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a904099fec4eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.659+0000 7fed6dd9b640 1 -- 172.21.15.16:0/3215770818 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed60037070 con 0x7fed7015dca0 2024-04-25T10:09:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.659+0000 7fed6dd9b640 1 -- 172.21.15.16:0/3215770818 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6002ae50 con 0x7fed7015dca0 2024-04-25T10:09:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.659+0000 7fed6dd9b640 1 -- 172.21.15.16:0/3215770818 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed60032660 con 0x7fed7015dca0 2024-04-25T10:09:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.659+0000 7fed74f70640 1 -- 172.21.15.16:0/3215770818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed7015dca0 msgr2=0x7fed700ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.659+0000 7fed74f70640 1 --2- 172.21.15.16:0/3215770818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed7015dca0 0x7fed700ce940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fed600097a0 tx=0x7fed6002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.659+0000 7fed74f70640 1 -- 172.21.15.16:0/3215770818 shutdown_connections 2024-04-25T10:09:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.659+0000 7fed74f70640 1 --2- 172.21.15.16:0/3215770818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed7015dca0 0x7fed700ce940 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.659+0000 7fed74f70640 1 -- 172.21.15.16:0/3215770818 >> 172.21.15.16:0/3215770818 conn(0x7fed700726c0 msgr2=0x7fed70072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.659+0000 7fed74f70640 1 -- 172.21.15.16:0/3215770818 shutdown_connections 2024-04-25T10:09:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.659+0000 7fed74f70640 1 -- 172.21.15.16:0/3215770818 wait complete. 2024-04-25T10:09:04.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.661+0000 7fed74f70640 1 Processor -- start 2024-04-25T10:09:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.661+0000 7fed74f70640 1 -- start start 2024-04-25T10:09:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.661+0000 7fed74f70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed700d1130 0x7fed70151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.661+0000 7fed74f70640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed700d3b90 0x7fed7015fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.661+0000 7fed74f70640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed7015dca0 0x7fed7015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.661+0000 7fed74f70640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7015e200 con 0x7fed700d1130 2024-04-25T10:09:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.661+0000 7fed74f70640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7015e080 con 0x7fed7015dca0 2024-04-25T10:09:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.661+0000 7fed74f70640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed700cee80 con 0x7fed700d3b90 2024-04-25T10:09:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.661+0000 7fed6e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed700d1130 0x7fed70151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.661+0000 7fed6e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed700d1130 0x7fed70151ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40918/0 (socket says 172.21.15.16:40918) 2024-04-25T10:09:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.661+0000 7fed6e59c640 1 -- 172.21.15.16:0/2137676047 learned_addr learned my addr 172.21.15.16:0/2137676047 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.662+0000 7fed65d9b640 1 --2- 172.21.15.16:0/2137676047 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed700d3b90 0x7fed7015fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.662+0000 7fed6ed9d640 1 --2- 172.21.15.16:0/2137676047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed7015dca0 0x7fed7015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.662+0000 7fed6e59c640 1 -- 172.21.15.16:0/2137676047 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed700d3b90 msgr2=0x7fed7015fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.662+0000 7fed6e59c640 1 --2- 172.21.15.16:0/2137676047 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed700d3b90 0x7fed7015fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.662+0000 7fed6e59c640 1 -- 172.21.15.16:0/2137676047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed7015dca0 msgr2=0x7fed7015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.662+0000 7fed6e59c640 1 --2- 172.21.15.16:0/2137676047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed7015dca0 0x7fed7015ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.662+0000 7fed6e59c640 1 -- 172.21.15.16:0/2137676047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed70160360 con 0x7fed700d1130 2024-04-25T10:09:04.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.662+0000 7fed6e59c640 1 --2- 172.21.15.16:0/2137676047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed700d1130 0x7fed70151ff0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fed60035040 tx=0x7fed600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.662+0000 7fed677fe640 1 -- 172.21.15.16:0/2137676047 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed60042070 con 0x7fed700d1130 2024-04-25T10:09:04.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.662+0000 7fed677fe640 1 -- 172.21.15.16:0/2137676047 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6003e670 con 0x7fed700d1130 2024-04-25T10:09:04.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.662+0000 7fed74f70640 1 -- 172.21.15.16:0/2137676047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed70160640 con 0x7fed700d1130 2024-04-25T10:09:04.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.662+0000 7fed677fe640 1 -- 172.21.15.16:0/2137676047 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed60037040 con 0x7fed700d1130 2024-04-25T10:09:04.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.662+0000 7fed74f70640 1 -- 172.21.15.16:0/2137676047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed70160920 con 0x7fed700d1130 2024-04-25T10:09:04.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.664+0000 7fed677fe640 1 -- 172.21.15.16:0/2137676047 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed60045070 con 0x7fed700d1130 2024-04-25T10:09:04.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.664+0000 7fed677fe640 1 --2- 172.21.15.16:0/2137676047 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed5006b500 0x7fed5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.664+0000 7fed677fe640 1 -- 172.21.15.16:0/2137676047 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fed600aa7f0 con 0x7fed700d1130 2024-04-25T10:09:04.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.664+0000 7fed65d9b640 1 --2- 172.21.15.16:0/2137676047 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed5006b500 0x7fed5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.665+0000 7fed65d9b640 1 --2- 172.21.15.16:0/2137676047 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed5006b500 0x7fed5006d9c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fed701f7110 tx=0x7fed58007cf0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.665+0000 7fed77c77740 1 Processor -- start 2024-04-25T10:09:04.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.665+0000 7fed77c77740 1 -- start start 2024-04-25T10:09:04.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.666+0000 7fed77c77740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca074590 0x5650ca0766f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.666+0000 7fed77c77740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca0b0c60 0x5650ca074970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.666+0000 7fed77c77740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca09f050 0x5650ca09f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.666+0000 7fed77c77740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c9fd7c60 con 0x5650ca0b0c60 2024-04-25T10:09:04.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.666+0000 7fed77c77740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c9f76d80 con 0x5650ca074590 2024-04-25T10:09:04.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.666+0000 7fed77c77740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650c9fd3d00 con 0x5650ca09f050 2024-04-25T10:09:04.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.666+0000 7fed65d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca0b0c60 0x5650ca074970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.666+0000 7fed65d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca0b0c60 0x5650ca074970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40926/0 (socket says 172.21.15.16:40926) 2024-04-25T10:09:04.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.666+0000 7fed65d9b640 1 -- 172.21.15.16:0/62505659 learned_addr learned my addr 172.21.15.16:0/62505659 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:04.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.666+0000 7fed6e59c640 1 --2- 172.21.15.16:0/62505659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca074590 0x5650ca0766f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.666+0000 7fed6ed9d640 1 --2- 172.21.15.16:0/62505659 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca09f050 0x5650ca09f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.666+0000 7fed65d9b640 1 -- 172.21.15.16:0/62505659 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca09f050 msgr2=0x5650ca09f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.666+0000 7fed65d9b640 1 --2- 172.21.15.16:0/62505659 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca09f050 0x5650ca09f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.666+0000 7fed65d9b640 1 -- 172.21.15.16:0/62505659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca074590 msgr2=0x5650ca0766f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.666+0000 7fed65d9b640 1 --2- 172.21.15.16:0/62505659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca074590 0x5650ca0766f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.666+0000 7fed65d9b640 1 -- 172.21.15.16:0/62505659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650c9db87c0 con 0x5650ca0b0c60 2024-04-25T10:09:04.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.667+0000 7fed65d9b640 1 --2- 172.21.15.16:0/62505659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca0b0c60 0x5650ca074970 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fed580019f0 tx=0x7fed58012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5b73521e72fc80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.667+0000 7fed4ffff640 1 -- 172.21.15.16:0/62505659 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58025070 con 0x5650ca0b0c60 2024-04-25T10:09:04.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.667+0000 7fed4ffff640 1 -- 172.21.15.16:0/62505659 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5801b050 con 0x5650ca0b0c60 2024-04-25T10:09:04.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.667+0000 7fed4ffff640 1 -- 172.21.15.16:0/62505659 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58014db0 con 0x5650ca0b0c60 2024-04-25T10:09:04.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.667+0000 7fed77c77740 1 -- 172.21.15.16:0/62505659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca0b0c60 msgr2=0x5650ca074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.667+0000 7fed77c77740 1 --2- 172.21.15.16:0/62505659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca0b0c60 0x5650ca074970 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fed580019f0 tx=0x7fed58012000 comp rx=0 tx=0).stop 2024-04-25T10:09:04.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.667+0000 7fed77c77740 1 -- 172.21.15.16:0/62505659 shutdown_connections 2024-04-25T10:09:04.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.667+0000 7fed77c77740 1 --2- 172.21.15.16:0/62505659 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca09f050 0x5650ca09f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.667+0000 7fed77c77740 1 --2- 172.21.15.16:0/62505659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca0b0c60 0x5650ca074970 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.667+0000 7fed77c77740 1 --2- 172.21.15.16:0/62505659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca074590 0x5650ca0766f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.667+0000 7fed77c77740 1 -- 172.21.15.16:0/62505659 >> 172.21.15.16:0/62505659 conn(0x5650ca03c6e0 msgr2=0x5650ca0ba050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.668+0000 7fed77c77740 1 -- 172.21.15.16:0/62505659 shutdown_connections 2024-04-25T10:09:04.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.668+0000 7fed77c77740 1 -- 172.21.15.16:0/62505659 wait complete. 2024-04-25T10:09:04.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.668+0000 7fed77c77740 1 -- start start 2024-04-25T10:09:04.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.668+0000 7fed77c77740 1 Processor -- start 2024-04-25T10:09:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.669+0000 7fed77c77740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca074590 0x5650ca088560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.669+0000 7fed77c77740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca09f050 0x5650ca0cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.669+0000 7fed77c77740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca0b0c60 0x5650ca0cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.669+0000 7fed77c77740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c9f74840 con 0x5650ca09f050 2024-04-25T10:09:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.669+0000 7fed77c77740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ca09cba0 con 0x5650ca074590 2024-04-25T10:09:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.669+0000 7fed77c77740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650ca08bc80 con 0x5650ca0b0c60 2024-04-25T10:09:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.669+0000 7fed77c77740 10 client.0 _ll_register_callbacks cb 0x5650c9f8e980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.669+0000 7fed65d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca09f050 0x5650ca0cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.669+0000 7fed65d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca09f050 0x5650ca0cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40934/0 (socket says 172.21.15.16:40934) 2024-04-25T10:09:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.669+0000 7fed65d9b640 1 -- 172.21.15.16:0/110927540 learned_addr learned my addr 172.21.15.16:0/110927540 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.669+0000 7fed6ed9d640 1 --2- 172.21.15.16:0/110927540 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca0b0c60 0x5650ca0cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.669+0000 7fed6e59c640 1 --2- 172.21.15.16:0/110927540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca074590 0x5650ca088560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.670+0000 7fed65d9b640 1 -- 172.21.15.16:0/110927540 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca0b0c60 msgr2=0x5650ca0cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.670+0000 7fed65d9b640 1 --2- 172.21.15.16:0/110927540 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca0b0c60 0x5650ca0cdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.670+0000 7fed65d9b640 1 -- 172.21.15.16:0/110927540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca074590 msgr2=0x5650ca088560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.670+0000 7fed65d9b640 1 --2- 172.21.15.16:0/110927540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca074590 0x5650ca088560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.670+0000 7fed65d9b640 1 -- 172.21.15.16:0/110927540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650c9e18f50 con 0x5650ca09f050 2024-04-25T10:09:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.670+0000 7fed65d9b640 1 --2- 172.21.15.16:0/110927540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca09f050 0x5650ca0cd830 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fed58023040 tx=0x7fed58000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994f2827d2707c5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.670+0000 7fed4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:09:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.670+0000 7fed4d7fa640 1 -- 172.21.15.16:0/110927540 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58032060 con 0x5650ca09f050 2024-04-25T10:09:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.670+0000 7fed4d7fa640 1 -- 172.21.15.16:0/110927540 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fed5802dae0 con 0x5650ca09f050 2024-04-25T10:09:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.670+0000 7fed77c77740 1 -- 172.21.15.16:0/110927540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed5801b050 con 0x5650ca09f050 2024-04-25T10:09:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.670+0000 7fed4d7fa640 1 -- 172.21.15.16:0/110927540 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed58025070 con 0x5650ca09f050 2024-04-25T10:09:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.670+0000 7fed77c77740 10 client.6441 fetch_fsmap learned FSMap version 29 2024-04-25T10:09:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.670+0000 7fed77c77740 1 -- 172.21.15.16:0/110927540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5650c9cac0b0 con 0x5650ca09f050 2024-04-25T10:09:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.670+0000 7fed4d7fa640 1 -- 172.21.15.16:0/110927540 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5801b050 con 0x5650ca09f050 2024-04-25T10:09:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.670+0000 7fed4d7fa640 1 -- 172.21.15.16:0/110927540 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fed5802b070 con 0x5650ca09f050 2024-04-25T10:09:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.670+0000 7fed77c77740 10 client.6441 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:09:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.670+0000 7fed77c77740 10 client.6441 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:09:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.670+0000 7fed77c77740 20 client.6441 populate_metadata read hostname 'smithi016' 2024-04-25T10:09:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.671+0000 7fed77c77740 1 --2- 172.21.15.16:0/110927540 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x5650ca09c230 0x5650ca09f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.671+0000 7fed77c77740 4 client.6441 mds_command: new command op to 35141 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.671+0000 7fed77c77740 1 -- 172.21.15.16:0/110927540 --> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650c9f97e60 con 0x5650ca09c230 2024-04-25T10:09:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.671+0000 7fed6e59c640 1 --2- 172.21.15.16:0/110927540 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x5650ca09c230 0x5650ca09f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.671+0000 7fed6e59c640 1 --2- 172.21.15.16:0/110927540 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x5650ca09c230 0x5650ca09f4d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2143743522247de server_cookie=b7aeec6b1b8de188 in_seq=0 out_seq=0 2024-04-25T10:09:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.671+0000 7fed4d7fa640 10 client.6441 ms_handle_connect on v2:172.21.15.16:6832/3311584521 2024-04-25T10:09:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.671+0000 7fed4d7fa640 1 -- 172.21.15.16:0/110927540 <== mds.0 v2:172.21.15.16:6832/3311584521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5650c9f97e60 con 0x5650ca09c230 2024-04-25T10:09:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.671+0000 7fed4d7fa640 10 client.6441 handle_command_reply: tid=0 2024-04-25T10:09:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.671+0000 7fed77c77740 1 client.6441 shutdown 2024-04-25T10:09:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.672+0000 7fed77c77740 1 -- 172.21.15.16:0/110927540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca09f050 msgr2=0x5650ca0cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.672+0000 7fed77c77740 1 --2- 172.21.15.16:0/110927540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca09f050 0x5650ca0cd830 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fed58023040 tx=0x7fed58000bc0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.672+0000 7fed77c77740 1 -- 172.21.15.16:0/110927540 shutdown_connections 2024-04-25T10:09:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.672+0000 7fed77c77740 1 --2- 172.21.15.16:0/110927540 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x5650ca09c230 0x5650ca09f4d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.672+0000 7fed77c77740 1 --2- 172.21.15.16:0/110927540 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca0b0c60 0x5650ca0cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.672+0000 7fed77c77740 1 --2- 172.21.15.16:0/110927540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca09f050 0x5650ca0cd830 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.672+0000 7fed77c77740 1 --2- 172.21.15.16:0/110927540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca074590 0x5650ca088560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.672+0000 7fed4d7fa640 0 client.6441 ms_handle_reset on v2:172.21.15.16:6832/3311584521 2024-04-25T10:09:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.672+0000 7fed77c77740 1 -- 172.21.15.16:0/110927540 >> 172.21.15.16:0/110927540 conn(0x5650ca03c6e0 msgr2=0x5650ca09cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.672+0000 7fed77c77740 1 -- 172.21.15.16:0/110927540 shutdown_connections 2024-04-25T10:09:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.672+0000 7fed77c77740 1 -- 172.21.15.16:0/110927540 wait complete. 2024-04-25T10:09:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.672+0000 7fed77c77740 20 client.6441 trim_cache size 0 max 16384 2024-04-25T10:09:04.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.684+0000 7fed77c77740 1 Processor -- start 2024-04-25T10:09:04.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.684+0000 7fed77c77740 1 -- start start 2024-04-25T10:09:04.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.684+0000 7fed77c77740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca074590 0x5650ca0bd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.684+0000 7fed77c77740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca09f050 0x5650ca0bdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.684+0000 7fed77c77740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca0b0c60 0x5650ca086ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.684+0000 7fed77c77740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed701543d0 con 0x5650ca09f050 2024-04-25T10:09:04.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.684+0000 7fed77c77740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed701537d0 con 0x5650ca0b0c60 2024-04-25T10:09:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.684+0000 7fed77c77740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed70153dd0 con 0x5650ca074590 2024-04-25T10:09:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.684+0000 7fed65d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca09f050 0x5650ca0bdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.684+0000 7fed65d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca09f050 0x5650ca0bdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40944/0 (socket says 172.21.15.16:40944) 2024-04-25T10:09:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.684+0000 7fed65d9b640 1 -- 172.21.15.16:0/98675196 learned_addr learned my addr 172.21.15.16:0/98675196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.685+0000 7fed6ed9d640 1 --2- 172.21.15.16:0/98675196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca0b0c60 0x5650ca086ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.685+0000 7fed6e59c640 1 --2- 172.21.15.16:0/98675196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca074590 0x5650ca0bd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.685+0000 7fed65d9b640 1 -- 172.21.15.16:0/98675196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca074590 msgr2=0x5650ca0bd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.685+0000 7fed65d9b640 1 --2- 172.21.15.16:0/98675196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca074590 0x5650ca0bd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.685+0000 7fed65d9b640 1 -- 172.21.15.16:0/98675196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca0b0c60 msgr2=0x5650ca086ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.685+0000 7fed65d9b640 1 --2- 172.21.15.16:0/98675196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca0b0c60 0x5650ca086ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.685+0000 7fed65d9b640 1 -- 172.21.15.16:0/98675196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650ca0dcb80 con 0x5650ca09f050 2024-04-25T10:09:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.685+0000 7fed6e59c640 1 --2- 172.21.15.16:0/98675196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca074590 0x5650ca0bd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.685+0000 7fed65d9b640 1 --2- 172.21.15.16:0/98675196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca09f050 0x5650ca0bdc00 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fed58006340 tx=0x7fed5802b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f06539b94b5298 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.685+0000 7fed4f7fe640 1 -- 172.21.15.16:0/98675196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58006440 con 0x5650ca09f050 2024-04-25T10:09:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.685+0000 7fed4f7fe640 1 -- 172.21.15.16:0/98675196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5801b070 con 0x5650ca09f050 2024-04-25T10:09:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.685+0000 7fed4f7fe640 1 -- 172.21.15.16:0/98675196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed580065e0 con 0x5650ca09f050 2024-04-25T10:09:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.685+0000 7fed77c77740 1 -- 172.21.15.16:0/98675196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca09f050 msgr2=0x5650ca0bdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.686+0000 7fed77c77740 1 --2- 172.21.15.16:0/98675196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca09f050 0x5650ca0bdc00 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fed58006340 tx=0x7fed5802b000 comp rx=0 tx=0).stop 2024-04-25T10:09:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.686+0000 7fed77c77740 1 -- 172.21.15.16:0/98675196 shutdown_connections 2024-04-25T10:09:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.686+0000 7fed77c77740 1 --2- 172.21.15.16:0/98675196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca0b0c60 0x5650ca086ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.686+0000 7fed77c77740 1 --2- 172.21.15.16:0/98675196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca09f050 0x5650ca0bdc00 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.686+0000 7fed77c77740 1 --2- 172.21.15.16:0/98675196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca074590 0x5650ca0bd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.686+0000 7fed77c77740 1 -- 172.21.15.16:0/98675196 >> 172.21.15.16:0/98675196 conn(0x5650ca03c6e0 msgr2=0x5650ca09b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.686+0000 7fed77c77740 1 -- 172.21.15.16:0/98675196 shutdown_connections 2024-04-25T10:09:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.686+0000 7fed77c77740 1 -- 172.21.15.16:0/98675196 wait complete. 2024-04-25T10:09:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.686+0000 7fed77c77740 1 -- start start 2024-04-25T10:09:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.686+0000 7fed77c77740 1 Processor -- start 2024-04-25T10:09:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.687+0000 7fed77c77740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca074590 0x5650ca09a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.687+0000 7fed77c77740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca09f050 0x5650ca09a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.687+0000 7fed77c77740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca0b0c60 0x5650ca0cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.687+0000 7fed77c77740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ca09c0b0 con 0x5650ca074590 2024-04-25T10:09:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.687+0000 7fed77c77740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c9f74840 con 0x5650ca0b0c60 2024-04-25T10:09:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.687+0000 7fed77c77740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650ca0a1250 con 0x5650ca09f050 2024-04-25T10:09:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.687+0000 7fed77c77740 10 client.0 _ll_register_callbacks cb 0x7fed58017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.687+0000 7fed6e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca074590 0x5650ca09a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.687+0000 7fed6e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca074590 0x5650ca09a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40946/0 (socket says 172.21.15.16:40946) 2024-04-25T10:09:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.687+0000 7fed6e59c640 1 -- 172.21.15.16:0/1525110316 learned_addr learned my addr 172.21.15.16:0/1525110316 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.687+0000 7fed6ed9d640 1 --2- 172.21.15.16:0/1525110316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca0b0c60 0x5650ca0cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.687+0000 7fed65d9b640 1 --2- 172.21.15.16:0/1525110316 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca09f050 0x5650ca09a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed6e59c640 1 -- 172.21.15.16:0/1525110316 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca09f050 msgr2=0x5650ca09a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed6e59c640 1 --2- 172.21.15.16:0/1525110316 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca09f050 0x5650ca09a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed6e59c640 1 -- 172.21.15.16:0/1525110316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca0b0c60 msgr2=0x5650ca0cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed6e59c640 1 --2- 172.21.15.16:0/1525110316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca0b0c60 0x5650ca0cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed6e59c640 1 -- 172.21.15.16:0/1525110316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650ca124c50 con 0x5650ca074590 2024-04-25T10:09:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed6ed9d640 1 --2- 172.21.15.16:0/1525110316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca0b0c60 0x5650ca0cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed6e59c640 1 --2- 172.21.15.16:0/1525110316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca074590 0x5650ca09a0f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fed60075450 tx=0x7fed60009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16da17a6b525a45e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed4d7fa640 1 -- 172.21.15.16:0/1525110316 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6004c040 con 0x5650ca074590 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed4d7fa640 1 -- 172.21.15.16:0/1525110316 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fed60075660 con 0x5650ca074590 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed77c77740 1 -- 172.21.15.16:0/1525110316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed5801b070 con 0x5650ca074590 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed4d7fa640 1 -- 172.21.15.16:0/1525110316 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed600811c0 con 0x5650ca074590 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed77c77740 10 client.6447 fetch_fsmap learned FSMap version 29 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed77c77740 1 -- 172.21.15.16:0/1525110316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5650ca072e50 con 0x5650ca074590 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed4d7fa640 1 -- 172.21.15.16:0/1525110316 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5801b070 con 0x5650ca074590 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed4d7fa640 1 -- 172.21.15.16:0/1525110316 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fed6007f0b0 con 0x5650ca074590 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed77c77740 10 client.6447 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed77c77740 10 client.6447 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed77c77740 20 client.6447 populate_metadata read hostname 'smithi016' 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed77c77740 1 --2- 172.21.15.16:0/1525110316 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x5650ca0af9e0 0x5650ca0afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed77c77740 4 client.6447 mds_command: new command op to 35141 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:09:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.688+0000 7fed77c77740 1 -- 172.21.15.16:0/1525110316 --> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5650c9fe51a0 con 0x5650ca0af9e0 2024-04-25T10:09:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.689+0000 7fed65d9b640 1 --2- 172.21.15.16:0/1525110316 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x5650ca0af9e0 0x5650ca0afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.689+0000 7fed4d7fa640 10 client.6447 ms_handle_connect on v2:172.21.15.16:6832/3311584521 2024-04-25T10:09:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.689+0000 7fed65d9b640 1 --2- 172.21.15.16:0/1525110316 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x5650ca0af9e0 0x5650ca0afdc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88e20b8fbd441dee server_cookie=438c3c8c4986705a in_seq=0 out_seq=0 2024-04-25T10:09:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.689+0000 7fed4d7fa640 1 -- 172.21.15.16:0/1525110316 <== mds.0 v2:172.21.15.16:6832/3311584521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5650c9fe51a0 con 0x5650ca0af9e0 2024-04-25T10:09:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.689+0000 7fed4d7fa640 10 client.6447 handle_command_reply: tid=0 2024-04-25T10:09:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.689+0000 7fed77c77740 1 client.6447 shutdown 2024-04-25T10:09:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.689+0000 7fed77c77740 1 -- 172.21.15.16:0/1525110316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca074590 msgr2=0x5650ca09a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.689+0000 7fed77c77740 1 --2- 172.21.15.16:0/1525110316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca074590 0x5650ca09a0f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fed60075450 tx=0x7fed60009190 comp rx=0 tx=0).stop 2024-04-25T10:09:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.689+0000 7fed77c77740 1 -- 172.21.15.16:0/1525110316 shutdown_connections 2024-04-25T10:09:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.689+0000 7fed77c77740 1 --2- 172.21.15.16:0/1525110316 >> [v2:172.21.15.16:6832/3311584521,v1:172.21.15.16:6834/3311584521] conn(0x5650ca0af9e0 0x5650ca0afdc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.689+0000 7fed77c77740 1 --2- 172.21.15.16:0/1525110316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650ca0b0c60 0x5650ca0cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.689+0000 7fed77c77740 1 --2- 172.21.15.16:0/1525110316 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5650ca09f050 0x5650ca09a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.689+0000 7fed4d7fa640 0 client.6447 ms_handle_reset on v2:172.21.15.16:6832/3311584521 2024-04-25T10:09:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.689+0000 7fed77c77740 1 --2- 172.21.15.16:0/1525110316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650ca074590 0x5650ca09a0f0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.689+0000 7fed77c77740 1 -- 172.21.15.16:0/1525110316 >> 172.21.15.16:0/1525110316 conn(0x5650ca03c6e0 msgr2=0x5650ca0bd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.689+0000 7fed77c77740 1 -- 172.21.15.16:0/1525110316 shutdown_connections 2024-04-25T10:09:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.689+0000 7fed77c77740 1 -- 172.21.15.16:0/1525110316 wait complete. 2024-04-25T10:09:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.689+0000 7fed77c77740 20 client.6447 trim_cache size 0 max 16384 2024-04-25T10:09:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:04.702 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-04-25T10:09:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.690+0000 7fed74f70640 1 -- 172.21.15.16:0/2137676047 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed5006b500 msgr2=0x7fed5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.690+0000 7fed74f70640 1 --2- 172.21.15.16:0/2137676047 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed5006b500 0x7fed5006d9c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fed701f7110 tx=0x7fed58007cf0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.690+0000 7fed74f70640 1 -- 172.21.15.16:0/2137676047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed700d1130 msgr2=0x7fed70151ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.690+0000 7fed74f70640 1 --2- 172.21.15.16:0/2137676047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed700d1130 0x7fed70151ff0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fed60035040 tx=0x7fed600046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.690+0000 7fed74f70640 1 -- 172.21.15.16:0/2137676047 shutdown_connections 2024-04-25T10:09:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.690+0000 7fed74f70640 1 --2- 172.21.15.16:0/2137676047 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed5006b500 0x7fed5006d9c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.690+0000 7fed74f70640 1 --2- 172.21.15.16:0/2137676047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed7015dca0 0x7fed7015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.690+0000 7fed74f70640 1 --2- 172.21.15.16:0/2137676047 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed700d3b90 0x7fed7015fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.690+0000 7fed74f70640 1 --2- 172.21.15.16:0/2137676047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed700d1130 0x7fed70151ff0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.690+0000 7fed74f70640 1 -- 172.21.15.16:0/2137676047 >> 172.21.15.16:0/2137676047 conn(0x7fed700726c0 msgr2=0x7fed700b4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.690+0000 7fed74f70640 1 -- 172.21.15.16:0/2137676047 shutdown_connections 2024-04-25T10:09:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.690+0000 7fed74f70640 1 -- 172.21.15.16:0/2137676047 wait complete. 2024-04-25T10:09:04.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:09:04.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.800+0000 7f616fc07640 1 Processor -- start 2024-04-25T10:09:04.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.800+0000 7f616fc07640 1 -- start start 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.801+0000 7f616fc07640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f61680d35b0 0x7f61680b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.801+0000 7f616fc07640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61680b5990 0x7f61680b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.801+0000 7f616fc07640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f616815e0c0 con 0x7f616815dce0 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.801+0000 7f616fc07640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61680d2510 con 0x7f61680b5990 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.801+0000 7f616fc07640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61680d27f0 con 0x7f61680d35b0 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.801+0000 7f616e1a4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61680b5990 0x7f61680b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.801+0000 7f616e1a4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61680b5990 0x7f61680b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40962/0 (socket says 172.21.15.16:40962) 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.801+0000 7f616e1a4640 1 -- 172.21.15.16:0/1662844696 learned_addr learned my addr 172.21.15.16:0/1662844696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.801+0000 7f616d1a2640 1 --2- 172.21.15.16:0/1662844696 >> v2:172.21.15.186:3300/0 conn(0x7f61680d35b0 0x7f61680b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.801+0000 7f616e1a4640 1 -- 172.21.15.16:0/1662844696 >> v2:172.21.15.186:3300/0 conn(0x7f61680d35b0 msgr2=0x7f61680b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.801+0000 7f616e1a4640 1 --2- 172.21.15.16:0/1662844696 >> v2:172.21.15.186:3300/0 conn(0x7f61680d35b0 0x7f61680b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.801+0000 7f616e1a4640 1 -- 172.21.15.16:0/1662844696 >> v1:172.21.15.186:6789/0 conn(0x7f616815dce0 legacy=0x7f61680d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.801+0000 7f616e1a4640 1 -- 172.21.15.16:0/1662844696 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61680d2ad0 con 0x7f61680b5990 2024-04-25T10:09:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.802+0000 7f616e1a4640 1 --2- 172.21.15.16:0/1662844696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61680b5990 0x7f61680b6b00 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f61680cdfa0 tx=0x7f616002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f25c05f925b7fd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.802+0000 7f616c9a1640 1 -- 172.21.15.16:0/1662844696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6160036070 con 0x7f61680b5990 2024-04-25T10:09:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.802+0000 7f616c9a1640 1 -- 172.21.15.16:0/1662844696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6160031600 con 0x7f61680b5990 2024-04-25T10:09:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.802+0000 7f616c9a1640 1 -- 172.21.15.16:0/1662844696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61600319d0 con 0x7f61680b5990 2024-04-25T10:09:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.802+0000 7f616fc07640 1 -- 172.21.15.16:0/1662844696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61680b5990 msgr2=0x7f61680b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.802+0000 7f616fc07640 1 --2- 172.21.15.16:0/1662844696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61680b5990 0x7f61680b6b00 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f61680cdfa0 tx=0x7f616002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.803+0000 7f616fc07640 1 -- 172.21.15.16:0/1662844696 shutdown_connections 2024-04-25T10:09:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.803+0000 7f616fc07640 1 --2- 172.21.15.16:0/1662844696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61680b5990 0x7f61680b6b00 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.803+0000 7f616fc07640 1 --2- 172.21.15.16:0/1662844696 >> v2:172.21.15.186:3300/0 conn(0x7f61680d35b0 0x7f61680b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.803+0000 7f616fc07640 1 -- 172.21.15.16:0/1662844696 >> 172.21.15.16:0/1662844696 conn(0x7f6168072900 msgr2=0x7f61680d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.803+0000 7f616fc07640 1 -- 172.21.15.16:0/1662844696 shutdown_connections 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.803+0000 7f616fc07640 1 -- 172.21.15.16:0/1662844696 wait complete. 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.804+0000 7f616fc07640 1 Processor -- start 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.804+0000 7f616fc07640 1 -- start start 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.804+0000 7f616fc07640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61680b5990 0x7f61680c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.804+0000 7f616fc07640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61680d35b0 0x7f6168159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.804+0000 7f616fc07640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f616815dce0 0x7f616815e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.804+0000 7f616fc07640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61680d2160 con 0x7f61680b5990 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.804+0000 7f616fc07640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61680d3090 con 0x7f61680d35b0 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.804+0000 7f616fc07640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61680c9970 con 0x7f616815dce0 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.805+0000 7f616d9a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61680b5990 0x7f61680c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.805+0000 7f616d9a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61680b5990 0x7f61680c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40974/0 (socket says 172.21.15.16:40974) 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.805+0000 7f616d9a3640 1 -- 172.21.15.16:0/1509482519 learned_addr learned my addr 172.21.15.16:0/1509482519 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.805+0000 7f616e1a4640 1 --2- 172.21.15.16:0/1509482519 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f616815dce0 0x7f616815e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.805+0000 7f616d9a3640 1 -- 172.21.15.16:0/1509482519 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f616815dce0 msgr2=0x7f616815e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.805+0000 7f616d9a3640 1 --2- 172.21.15.16:0/1509482519 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f616815dce0 0x7f616815e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.805+0000 7f616d1a2640 1 --2- 172.21.15.16:0/1509482519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61680d35b0 0x7f6168159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.805+0000 7f616d9a3640 1 -- 172.21.15.16:0/1509482519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61680d35b0 msgr2=0x7f6168159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.805+0000 7f616d9a3640 1 --2- 172.21.15.16:0/1509482519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61680d35b0 0x7f6168159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.805+0000 7f616d9a3640 1 -- 172.21.15.16:0/1509482519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61680ca340 con 0x7f61680b5990 2024-04-25T10:09:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.805+0000 7f616d9a3640 1 --2- 172.21.15.16:0/1509482519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61680b5990 0x7f61680c9e00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f616400b420 tx=0x7f616400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.805+0000 7f615e7fc640 1 -- 172.21.15.16:0/1509482519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164011830 con 0x7f61680b5990 2024-04-25T10:09:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.805+0000 7f615e7fc640 1 -- 172.21.15.16:0/1509482519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6164016070 con 0x7f61680b5990 2024-04-25T10:09:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.805+0000 7f616fc07640 1 -- 172.21.15.16:0/1509482519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61680c8d60 con 0x7f61680b5990 2024-04-25T10:09:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.805+0000 7f615e7fc640 1 -- 172.21.15.16:0/1509482519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164011c10 con 0x7f61680b5990 2024-04-25T10:09:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.805+0000 7f616fc07640 1 -- 172.21.15.16:0/1509482519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61680c9320 con 0x7f61680b5990 2024-04-25T10:09:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.806+0000 7f616fc07640 1 -- 172.21.15.16:0/1509482519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61680b28e0 con 0x7f61680b5990 2024-04-25T10:09:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.807+0000 7f615e7fc640 1 -- 172.21.15.16:0/1509482519 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f616401b460 con 0x7f61680b5990 2024-04-25T10:09:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.807+0000 7f615e7fc640 1 --2- 172.21.15.16:0/1509482519 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f614c06b500 0x7f614c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.807+0000 7f615e7fc640 1 -- 172.21.15.16:0/1509482519 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6164089f90 con 0x7f61680b5990 2024-04-25T10:09:04.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.808+0000 7f616d1a2640 1 --2- 172.21.15.16:0/1509482519 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f614c06b500 0x7f614c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.808+0000 7f616d1a2640 1 --2- 172.21.15.16:0/1509482519 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f614c06b500 0x7f614c06d9c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f61680b4aa0 tx=0x7f6158008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:04.812+0000 7f615e7fc640 1 -- 172.21.15.16:0/1509482519 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f61680b28e0 con 0x7f61680b5990 2024-04-25T10:09:05.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.028+0000 7f616fc07640 1 -- 172.21.15.16:0/1509482519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f61680749b0 con 0x7f61680b5990 2024-04-25T10:09:05.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.029+0000 7f615e7fc640 1 -- 172.21.15.16:0/1509482519 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f6164058d40 con 0x7f61680b5990 2024-04-25T10:09:05.035 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:05.035 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:09:05.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.031+0000 7f616fc07640 1 -- 172.21.15.16:0/1509482519 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f614c06b500 msgr2=0x7f614c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.031+0000 7f616fc07640 1 --2- 172.21.15.16:0/1509482519 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f614c06b500 0x7f614c06d9c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f61680b4aa0 tx=0x7f6158008040 comp rx=0 tx=0).stop 2024-04-25T10:09:05.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.031+0000 7f616fc07640 1 -- 172.21.15.16:0/1509482519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61680b5990 msgr2=0x7f61680c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.031+0000 7f616fc07640 1 --2- 172.21.15.16:0/1509482519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61680b5990 0x7f61680c9e00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f616400b420 tx=0x7f616400b8e0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.031+0000 7f616fc07640 1 -- 172.21.15.16:0/1509482519 shutdown_connections 2024-04-25T10:09:05.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.031+0000 7f616fc07640 1 --2- 172.21.15.16:0/1509482519 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f614c06b500 0x7f614c06d9c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.031+0000 7f616fc07640 1 --2- 172.21.15.16:0/1509482519 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f616815dce0 0x7f616815e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.031+0000 7f616fc07640 1 --2- 172.21.15.16:0/1509482519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61680d35b0 0x7f6168159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.031+0000 7f616fc07640 1 --2- 172.21.15.16:0/1509482519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61680b5990 0x7f61680c9e00 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.031+0000 7f616fc07640 1 -- 172.21.15.16:0/1509482519 >> 172.21.15.16:0/1509482519 conn(0x7f6168072900 msgr2=0x7f61680b81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.031+0000 7f616fc07640 1 -- 172.21.15.16:0/1509482519 shutdown_connections 2024-04-25T10:09:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.031+0000 7f616fc07640 1 -- 172.21.15.16:0/1509482519 wait complete. 2024-04-25T10:09:05.047 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:09:05.048 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:09:05.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:09:05.089 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:09:05.134 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:09:05.135 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:09:05.135 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:05.151 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:09:05.151 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:05.206 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:05.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.341+0000 7f719decd640 1 Processor -- start 2024-04-25T10:09:05.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.341+0000 7f719decd640 1 -- start start 2024-04-25T10:09:05.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.342+0000 7f719decd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f719815dce0 0x7f71980d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.342+0000 7f719decd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71980b5b60 0x7f71980b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.342+0000 7f719decd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719815e0c0 con 0x7f71980d35b0 2024-04-25T10:09:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.342+0000 7f719decd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71980d2510 con 0x7f719815dce0 2024-04-25T10:09:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.342+0000 7f719decd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71980d27f0 con 0x7f71980b5b60 2024-04-25T10:09:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.342+0000 7f71977fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f719815dce0 0x7f71980d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.342+0000 7f71977fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f719815dce0 0x7f71980d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40978/0 (socket says 172.21.15.16:40978) 2024-04-25T10:09:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.342+0000 7f71977fe640 1 -- 172.21.15.16:0/2915541737 learned_addr learned my addr 172.21.15.16:0/2915541737 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.342+0000 7f7197fff640 1 --2- 172.21.15.16:0/2915541737 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71980b5b60 0x7f71980b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.343+0000 7f71977fe640 1 -- 172.21.15.16:0/2915541737 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71980b5b60 msgr2=0x7f71980b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.343+0000 7f71977fe640 1 --2- 172.21.15.16:0/2915541737 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71980b5b60 0x7f71980b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.343+0000 7f71977fe640 1 -- 172.21.15.16:0/2915541737 >> v1:172.21.15.186:6789/0 conn(0x7f71980d35b0 legacy=0x7f71980b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.343+0000 7f71977fe640 1 -- 172.21.15.16:0/2915541737 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71980d2ad0 con 0x7f719815dce0 2024-04-25T10:09:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.343+0000 7f71977fe640 1 --2- 172.21.15.16:0/2915541737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f719815dce0 0x7f71980d1a50 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7188009870 tx=0x7f718802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac38f422ed98f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.343+0000 7f71967fc640 1 -- 172.21.15.16:0/2915541737 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188037070 con 0x7f719815dce0 2024-04-25T10:09:05.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.343+0000 7f71967fc640 1 -- 172.21.15.16:0/2915541737 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7188032440 con 0x7f719815dce0 2024-04-25T10:09:05.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.343+0000 7f71967fc640 1 -- 172.21.15.16:0/2915541737 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188032810 con 0x7f719815dce0 2024-04-25T10:09:05.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.344+0000 7f719decd640 1 -- 172.21.15.16:0/2915541737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f719815dce0 msgr2=0x7f71980d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.344+0000 7f719decd640 1 --2- 172.21.15.16:0/2915541737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f719815dce0 0x7f71980d1a50 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7188009870 tx=0x7f718802a210 comp rx=0 tx=0).stop 2024-04-25T10:09:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.344+0000 7f719decd640 1 -- 172.21.15.16:0/2915541737 shutdown_connections 2024-04-25T10:09:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.344+0000 7f719decd640 1 --2- 172.21.15.16:0/2915541737 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71980b5b60 0x7f71980b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.344+0000 7f719decd640 1 --2- 172.21.15.16:0/2915541737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f719815dce0 0x7f71980d1a50 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.344+0000 7f719decd640 1 -- 172.21.15.16:0/2915541737 >> 172.21.15.16:0/2915541737 conn(0x7f7198072900 msgr2=0x7f71980d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.344+0000 7f719decd640 1 -- 172.21.15.16:0/2915541737 shutdown_connections 2024-04-25T10:09:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.344+0000 7f719decd640 1 -- 172.21.15.16:0/2915541737 wait complete. 2024-04-25T10:09:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.345+0000 7f719decd640 1 Processor -- start 2024-04-25T10:09:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.345+0000 7f719decd640 1 -- start start 2024-04-25T10:09:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.346+0000 7f719decd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71980b5b60 0x7f7198159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.346+0000 7f719decd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71980d35b0 0x7f71980ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.346+0000 7f719decd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f719815dce0 0x7f719815e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.346+0000 7f719decd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71980b9820 con 0x7f71980b5b60 2024-04-25T10:09:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.346+0000 7f719decd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71980d3090 con 0x7f71980d35b0 2024-04-25T10:09:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.346+0000 7f719decd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71980b99a0 con 0x7f719815dce0 2024-04-25T10:09:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.346+0000 7f71977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71980b5b60 0x7f7198159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.346+0000 7f71977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71980b5b60 0x7f7198159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40984/0 (socket says 172.21.15.16:40984) 2024-04-25T10:09:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.346+0000 7f71977fe640 1 -- 172.21.15.16:0/2774840636 learned_addr learned my addr 172.21.15.16:0/2774840636 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.347+0000 7f7197fff640 1 --2- 172.21.15.16:0/2774840636 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f719815dce0 0x7f719815e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.347+0000 7f7196ffd640 1 --2- 172.21.15.16:0/2774840636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71980d35b0 0x7f71980ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.347+0000 7f71977fe640 1 -- 172.21.15.16:0/2774840636 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f719815dce0 msgr2=0x7f719815e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.347+0000 7f71977fe640 1 --2- 172.21.15.16:0/2774840636 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f719815dce0 0x7f719815e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.347+0000 7f71977fe640 1 -- 172.21.15.16:0/2774840636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71980d35b0 msgr2=0x7f71980ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.347+0000 7f71977fe640 1 --2- 172.21.15.16:0/2774840636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71980d35b0 0x7f71980ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.347+0000 7f71977fe640 1 -- 172.21.15.16:0/2774840636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71980cbc00 con 0x7f71980b5b60 2024-04-25T10:09:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.347+0000 7f7197fff640 1 --2- 172.21.15.16:0/2774840636 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f719815dce0 0x7f719815e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:05.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.347+0000 7f7196ffd640 1 --2- 172.21.15.16:0/2774840636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71980d35b0 0x7f71980ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:05.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.347+0000 7f71977fe640 1 --2- 172.21.15.16:0/2774840636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71980b5b60 0x7f7198159a80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7188009870 tx=0x7f71880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:05.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.347+0000 7f717ffff640 1 -- 172.21.15.16:0/2774840636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188041020 con 0x7f71980b5b60 2024-04-25T10:09:05.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.347+0000 7f717ffff640 1 -- 172.21.15.16:0/2774840636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71880306c0 con 0x7f71980b5b60 2024-04-25T10:09:05.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.347+0000 7f719decd640 1 -- 172.21.15.16:0/2774840636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71980cbee0 con 0x7f71980b5b60 2024-04-25T10:09:05.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.347+0000 7f717ffff640 1 -- 172.21.15.16:0/2774840636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188037040 con 0x7f71980b5b60 2024-04-25T10:09:05.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.347+0000 7f719decd640 1 -- 172.21.15.16:0/2774840636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71980cc4a0 con 0x7f71980b5b60 2024-04-25T10:09:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.348+0000 7f719decd640 1 -- 172.21.15.16:0/2774840636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71980749b0 con 0x7f71980b5b60 2024-04-25T10:09:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.349+0000 7f717ffff640 1 -- 172.21.15.16:0/2774840636 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7188044020 con 0x7f71980b5b60 2024-04-25T10:09:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.349+0000 7f717ffff640 1 --2- 172.21.15.16:0/2774840636 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f718406b4d0 0x7f718406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.349+0000 7f717ffff640 1 -- 172.21.15.16:0/2774840636 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f71880aa9f0 con 0x7f71980b5b60 2024-04-25T10:09:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.349+0000 7f7196ffd640 1 --2- 172.21.15.16:0/2774840636 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f718406b4d0 0x7f718406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.350+0000 7f7196ffd640 1 --2- 172.21.15.16:0/2774840636 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f718406b4d0 0x7f718406d990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f71980d1660 tx=0x7f718c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:05.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.353+0000 7f717ffff640 1 -- 172.21.15.16:0/2774840636 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f718804b030 con 0x7f71980b5b60 2024-04-25T10:09:05.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.535+0000 7f719decd640 1 -- 172.21.15.16:0/2774840636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71980b72f0 con 0x7f71980b5b60 2024-04-25T10:09:05.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.537+0000 7f717ffff640 1 -- 172.21.15.16:0/2774840636 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16761 (secure 0 0 0) 0x7f7188079830 con 0x7f71980b5b60 2024-04-25T10:09:05.543 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:05.544 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":61,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:08:52.605846+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:48.948980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:49.638457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.19","mappings":[{"from":6,"to":1}]},{"pgid":"9.28","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:05.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.538+0000 7f719decd640 1 -- 172.21.15.16:0/2774840636 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f718406b4d0 msgr2=0x7f718406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.538+0000 7f719decd640 1 --2- 172.21.15.16:0/2774840636 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f718406b4d0 0x7f718406d990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f71980d1660 tx=0x7f718c009040 comp rx=0 tx=0).stop 2024-04-25T10:09:05.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.538+0000 7f719decd640 1 -- 172.21.15.16:0/2774840636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71980b5b60 msgr2=0x7f7198159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.538+0000 7f719decd640 1 --2- 172.21.15.16:0/2774840636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71980b5b60 0x7f7198159a80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7188009870 tx=0x7f71880046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.538+0000 7f719decd640 1 -- 172.21.15.16:0/2774840636 shutdown_connections 2024-04-25T10:09:05.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.538+0000 7f719decd640 1 --2- 172.21.15.16:0/2774840636 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f718406b4d0 0x7f718406d990 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.538+0000 7f719decd640 1 --2- 172.21.15.16:0/2774840636 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f719815dce0 0x7f719815e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.538+0000 7f719decd640 1 --2- 172.21.15.16:0/2774840636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71980d35b0 0x7f71980ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.538+0000 7f719decd640 1 --2- 172.21.15.16:0/2774840636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71980b5b60 0x7f7198159a80 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.538+0000 7f719decd640 1 -- 172.21.15.16:0/2774840636 >> 172.21.15.16:0/2774840636 conn(0x7f7198072900 msgr2=0x7f71980ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:05.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.539+0000 7f719decd640 1 -- 172.21.15.16:0/2774840636 shutdown_connections 2024-04-25T10:09:05.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.539+0000 7f719decd640 1 -- 172.21.15.16:0/2774840636 wait complete. 2024-04-25T10:09:05.555 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:48.948980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:49.638457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:05.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.654+0000 7ff46da38640 1 Processor -- start 2024-04-25T10:09:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.654+0000 7ff46da38640 1 -- start start 2024-04-25T10:09:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.655+0000 7ff46da38640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff46815deb0 0x7ff4680b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.655+0000 7ff46da38640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff4680b1d30 0x7ff46806aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.655+0000 7ff46da38640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4680cd150 con 0x7ff46806b0e0 2024-04-25T10:09:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.655+0000 7ff46da38640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff46806b530 con 0x7ff4680b1d30 2024-04-25T10:09:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.655+0000 7ff46da38640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4680b21d0 con 0x7ff46815deb0 2024-04-25T10:09:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.655+0000 7ff4677fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff46806b0e0 0x7ff468161b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32830/0 (socket says 172.21.15.16:32830) 2024-04-25T10:09:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.655+0000 7ff4677fe640 1 -- 172.21.15.16:0/2271050111 learned_addr learned my addr 172.21.15.16:0/2271050111 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.656+0000 7ff466ffd640 1 --2- 172.21.15.16:0/2271050111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff46815deb0 0x7ff4680b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.656+0000 7ff4667fc640 1 --2- 172.21.15.16:0/2271050111 >> v2:172.21.15.186:3300/0 conn(0x7ff4680b1d30 0x7ff46806aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.656+0000 7ff465ffb640 1 -- 172.21.15.16:0/2271050111 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038364295 0 0) 0x7ff4680cd150 con 0x7ff46806b0e0 2024-04-25T10:09:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.656+0000 7ff465ffb640 1 -- 172.21.15.16:0/2271050111 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff454003520 con 0x7ff46806b0e0 2024-04-25T10:09:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.656+0000 7ff466ffd640 1 -- 172.21.15.16:0/2271050111 >> v2:172.21.15.186:3300/0 conn(0x7ff4680b1d30 msgr2=0x7ff46806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.656+0000 7ff466ffd640 1 --2- 172.21.15.16:0/2271050111 >> v2:172.21.15.186:3300/0 conn(0x7ff4680b1d30 0x7ff46806aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.656+0000 7ff466ffd640 1 -- 172.21.15.16:0/2271050111 >> v1:172.21.15.16:6789/0 conn(0x7ff46806b0e0 legacy=0x7ff468161b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.656+0000 7ff466ffd640 1 -- 172.21.15.16:0/2271050111 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4680cd430 con 0x7ff46815deb0 2024-04-25T10:09:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.656+0000 7ff4667fc640 1 --2- 172.21.15.16:0/2271050111 >> v2:172.21.15.186:3300/0 conn(0x7ff4680b1d30 0x7ff46806aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.657+0000 7ff466ffd640 1 --2- 172.21.15.16:0/2271050111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff46815deb0 0x7ff4680b17f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff458009d90 tx=0x7ff45802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c995d981f3a2483 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.657+0000 7ff465ffb640 1 -- 172.21.15.16:0/2271050111 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458031600 con 0x7ff46815deb0 2024-04-25T10:09:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.657+0000 7ff465ffb640 1 -- 172.21.15.16:0/2271050111 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4580317a0 con 0x7ff46815deb0 2024-04-25T10:09:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.657+0000 7ff465ffb640 1 -- 172.21.15.16:0/2271050111 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458031b70 con 0x7ff46815deb0 2024-04-25T10:09:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.657+0000 7ff46da38640 1 -- 172.21.15.16:0/2271050111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff46815deb0 msgr2=0x7ff4680b17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.657+0000 7ff46da38640 1 --2- 172.21.15.16:0/2271050111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff46815deb0 0x7ff4680b17f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff458009d90 tx=0x7ff45802a5d0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.657+0000 7ff46da38640 1 -- 172.21.15.16:0/2271050111 shutdown_connections 2024-04-25T10:09:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.657+0000 7ff46da38640 1 --2- 172.21.15.16:0/2271050111 >> v2:172.21.15.186:3300/0 conn(0x7ff4680b1d30 0x7ff46806aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.657+0000 7ff46da38640 1 --2- 172.21.15.16:0/2271050111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff46815deb0 0x7ff4680b17f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.657+0000 7ff46da38640 1 -- 172.21.15.16:0/2271050111 >> 172.21.15.16:0/2271050111 conn(0x7ff4680b2390 msgr2=0x7ff4680b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.658+0000 7ff46da38640 1 -- 172.21.15.16:0/2271050111 shutdown_connections 2024-04-25T10:09:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.658+0000 7ff46da38640 1 -- 172.21.15.16:0/2271050111 wait complete. 2024-04-25T10:09:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.659+0000 7ff46da38640 1 Processor -- start 2024-04-25T10:09:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.659+0000 7ff46da38640 1 -- start start 2024-04-25T10:09:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff46da38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff46806b0e0 0x7ff4680b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff46da38640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4680b1d30 0x7ff46814b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff46da38640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff46815deb0 0x7ff46814bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff46da38640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46815e290 con 0x7ff46806b0e0 2024-04-25T10:09:05.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff46da38640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4680cd9f0 con 0x7ff4680b1d30 2024-04-25T10:09:05.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff46da38640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff46815e410 con 0x7ff46815deb0 2024-04-25T10:09:05.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff466ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff46806b0e0 0x7ff4680b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff466ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff46806b0e0 0x7ff4680b5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40990/0 (socket says 172.21.15.16:40990) 2024-04-25T10:09:05.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff466ffd640 1 -- 172.21.15.16:0/3289005283 learned_addr learned my addr 172.21.15.16:0/3289005283 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:05.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff466ffd640 1 -- 172.21.15.16:0/3289005283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff46815deb0 msgr2=0x7ff46814bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff4667fc640 1 --2- 172.21.15.16:0/3289005283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4680b1d30 0x7ff46814b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff4677fe640 1 --2- 172.21.15.16:0/3289005283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff46815deb0 0x7ff46814bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff466ffd640 1 --2- 172.21.15.16:0/3289005283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff46815deb0 0x7ff46814bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff466ffd640 1 -- 172.21.15.16:0/3289005283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4680b1d30 msgr2=0x7ff46814b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff466ffd640 1 --2- 172.21.15.16:0/3289005283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4680b1d30 0x7ff46814b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff466ffd640 1 -- 172.21.15.16:0/3289005283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4680c5be0 con 0x7ff46806b0e0 2024-04-25T10:09:05.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff4677fe640 1 --2- 172.21.15.16:0/3289005283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff46815deb0 0x7ff46814bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:05.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff466ffd640 1 --2- 172.21.15.16:0/3289005283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff46806b0e0 0x7ff4680b5690 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff458009d90 tx=0x7ff458004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:05.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff45ffff640 1 -- 172.21.15.16:0/3289005283 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45803f070 con 0x7ff46806b0e0 2024-04-25T10:09:05.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff45ffff640 1 -- 172.21.15.16:0/3289005283 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff458030830 con 0x7ff46806b0e0 2024-04-25T10:09:05.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff46da38640 1 -- 172.21.15.16:0/3289005283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4680c5060 con 0x7ff46806b0e0 2024-04-25T10:09:05.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.660+0000 7ff45ffff640 1 -- 172.21.15.16:0/3289005283 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458030c20 con 0x7ff46806b0e0 2024-04-25T10:09:05.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.661+0000 7ff46da38640 1 -- 172.21.15.16:0/3289005283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4680c5620 con 0x7ff46806b0e0 2024-04-25T10:09:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.661+0000 7ff46da38640 1 -- 172.21.15.16:0/3289005283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4680750b0 con 0x7ff46806b0e0 2024-04-25T10:09:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.662+0000 7ff45ffff640 1 -- 172.21.15.16:0/3289005283 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff458044460 con 0x7ff46806b0e0 2024-04-25T10:09:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.662+0000 7ff45ffff640 1 --2- 172.21.15.16:0/3289005283 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff44806b4d0 0x7ff44806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.662+0000 7ff45ffff640 1 -- 172.21.15.16:0/3289005283 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff4580ab680 con 0x7ff46806b0e0 2024-04-25T10:09:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.663+0000 7ff4667fc640 1 --2- 172.21.15.16:0/3289005283 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff44806b4d0 0x7ff44806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.663+0000 7ff4667fc640 1 --2- 172.21.15.16:0/3289005283 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff44806b4d0 0x7ff44806d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff450009560 tx=0x7ff450008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:05.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.667+0000 7ff45ffff640 1 -- 172.21.15.16:0/3289005283 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff45807a670 con 0x7ff46806b0e0 2024-04-25T10:09:05.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.844+0000 7ff46da38640 1 -- 172.21.15.16:0/3289005283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff4680b67c0 con 0x7ff46806b0e0 2024-04-25T10:09:05.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.845+0000 7ff45ffff640 1 -- 172.21.15.16:0/3289005283 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff458079f70 con 0x7ff46806b0e0 2024-04-25T10:09:05.851 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:09:05.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.847+0000 7ff46da38640 1 -- 172.21.15.16:0/3289005283 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff44806b4d0 msgr2=0x7ff44806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.847+0000 7ff46da38640 1 --2- 172.21.15.16:0/3289005283 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff44806b4d0 0x7ff44806d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff450009560 tx=0x7ff450008040 comp rx=0 tx=0).stop 2024-04-25T10:09:05.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.847+0000 7ff46da38640 1 -- 172.21.15.16:0/3289005283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff46806b0e0 msgr2=0x7ff4680b5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.847+0000 7ff46da38640 1 --2- 172.21.15.16:0/3289005283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff46806b0e0 0x7ff4680b5690 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff458009d90 tx=0x7ff458004a40 comp rx=0 tx=0).stop 2024-04-25T10:09:05.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.847+0000 7ff46da38640 1 -- 172.21.15.16:0/3289005283 shutdown_connections 2024-04-25T10:09:05.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.847+0000 7ff46da38640 1 --2- 172.21.15.16:0/3289005283 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff44806b4d0 0x7ff44806d990 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.847+0000 7ff46da38640 1 --2- 172.21.15.16:0/3289005283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff46815deb0 0x7ff46814bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.847+0000 7ff46da38640 1 --2- 172.21.15.16:0/3289005283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4680b1d30 0x7ff46814b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.847+0000 7ff46da38640 1 --2- 172.21.15.16:0/3289005283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff46806b0e0 0x7ff4680b5690 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.847+0000 7ff46da38640 1 -- 172.21.15.16:0/3289005283 >> 172.21.15.16:0/3289005283 conn(0x7ff4680b2390 msgr2=0x7ff4680b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:05.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.847+0000 7ff46da38640 1 -- 172.21.15.16:0/3289005283 shutdown_connections 2024-04-25T10:09:05.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.847+0000 7ff46da38640 1 -- 172.21.15.16:0/3289005283 wait complete. 2024-04-25T10:09:05.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:09:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.961+0000 7f96770b3640 1 Processor -- start 2024-04-25T10:09:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.961+0000 7f96770b3640 1 -- start start 2024-04-25T10:09:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.961+0000 7f96770b3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f967015db50 0x7f96700cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.961+0000 7f96770b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f967015df30 0x7f967006cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.961+0000 7f96770b3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f96700d12a0 0x7f9670099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.961+0000 7f96770b3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f967006c020 con 0x7f96700d12a0 2024-04-25T10:09:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.961+0000 7f96770b3640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f967006d0e0 con 0x7f967015db50 2024-04-25T10:09:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.961+0000 7f96770b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96700d0200 con 0x7f967015df30 2024-04-25T10:09:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.961+0000 7f9675650640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f96700d12a0 0x7f9670099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.962+0000 7f9675650640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f96700d12a0 0x7f9670099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41006/0 (socket says 172.21.15.16:41006) 2024-04-25T10:09:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.962+0000 7f9675650640 1 -- 172.21.15.16:0/2681792227 learned_addr learned my addr 172.21.15.16:0/2681792227 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.962+0000 7f9674e4f640 1 --2- 172.21.15.16:0/2681792227 >> v2:172.21.15.186:3300/0 conn(0x7f967015db50 0x7f96700cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.962+0000 7f966ffff640 1 --2- 172.21.15.16:0/2681792227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f967015df30 0x7f967006cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.962+0000 7f9675650640 1 -- 172.21.15.16:0/2681792227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f967015df30 msgr2=0x7f967006cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.962+0000 7f9675650640 1 --2- 172.21.15.16:0/2681792227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f967015df30 0x7f967006cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.962+0000 7f9675650640 1 -- 172.21.15.16:0/2681792227 >> v2:172.21.15.186:3300/0 conn(0x7f967015db50 msgr2=0x7f96700cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.962+0000 7f9675650640 1 --2- 172.21.15.16:0/2681792227 >> v2:172.21.15.186:3300/0 conn(0x7f967015db50 0x7f96700cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.962+0000 7f9675650640 1 -- 172.21.15.16:0/2681792227 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96700d04e0 con 0x7f96700d12a0 2024-04-25T10:09:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.962+0000 7f9675650640 1 --2- 172.21.15.16:0/2681792227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96700d12a0 0x7f9670099a40 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f967009a700 tx=0x7f966802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667b37fd2d46577f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.963+0000 7f966f7fe640 1 -- 172.21.15.16:0/2681792227 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668036070 con 0x7f96700d12a0 2024-04-25T10:09:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.963+0000 7f966f7fe640 1 -- 172.21.15.16:0/2681792227 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9668031600 con 0x7f96700d12a0 2024-04-25T10:09:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.963+0000 7f966f7fe640 1 -- 172.21.15.16:0/2681792227 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96680319d0 con 0x7f96700d12a0 2024-04-25T10:09:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.963+0000 7f96770b3640 1 -- 172.21.15.16:0/2681792227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96700d12a0 msgr2=0x7f9670099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.963+0000 7f96770b3640 1 --2- 172.21.15.16:0/2681792227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96700d12a0 0x7f9670099a40 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f967009a700 tx=0x7f966802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.963+0000 7f96770b3640 1 -- 172.21.15.16:0/2681792227 shutdown_connections 2024-04-25T10:09:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.963+0000 7f96770b3640 1 --2- 172.21.15.16:0/2681792227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96700d12a0 0x7f9670099a40 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.963+0000 7f96770b3640 1 --2- 172.21.15.16:0/2681792227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f967015df30 0x7f967006cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.963+0000 7f96770b3640 1 --2- 172.21.15.16:0/2681792227 >> v2:172.21.15.186:3300/0 conn(0x7f967015db50 0x7f96700cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.963+0000 7f96770b3640 1 -- 172.21.15.16:0/2681792227 >> 172.21.15.16:0/2681792227 conn(0x7f96700b3430 msgr2=0x7f96700b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.963+0000 7f96770b3640 1 -- 172.21.15.16:0/2681792227 shutdown_connections 2024-04-25T10:09:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.963+0000 7f96770b3640 1 -- 172.21.15.16:0/2681792227 wait complete. 2024-04-25T10:09:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.964+0000 7f96770b3640 1 Processor -- start 2024-04-25T10:09:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.964+0000 7f96770b3640 1 -- start start 2024-04-25T10:09:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.965+0000 7f96770b3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96700d12a0 0x7f9670151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.965+0000 7f96770b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f967015db50 0x7f96700b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.965+0000 7f96770b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f967015df30 0x7f96701583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.965+0000 7f96770b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96700d0d80 con 0x7f967015df30 2024-04-25T10:09:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.965+0000 7f96770b3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96700d0aa0 con 0x7f96700d12a0 2024-04-25T10:09:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.965+0000 7f96770b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9670099f80 con 0x7f967015db50 2024-04-25T10:09:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.965+0000 7f9675650640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f967015df30 0x7f96701583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.965+0000 7f9675650640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f967015df30 0x7f96701583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41018/0 (socket says 172.21.15.16:41018) 2024-04-25T10:09:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.965+0000 7f9675650640 1 -- 172.21.15.16:0/448477814 learned_addr learned my addr 172.21.15.16:0/448477814 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.965+0000 7f9674e4f640 1 --2- 172.21.15.16:0/448477814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96700d12a0 0x7f9670151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.966+0000 7f966ffff640 1 --2- 172.21.15.16:0/448477814 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f967015db50 0x7f96700b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.966+0000 7f9675650640 1 -- 172.21.15.16:0/448477814 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f967015db50 msgr2=0x7f96700b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.966+0000 7f9675650640 1 --2- 172.21.15.16:0/448477814 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f967015db50 0x7f96700b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.966+0000 7f9675650640 1 -- 172.21.15.16:0/448477814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96700d12a0 msgr2=0x7f9670151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.966+0000 7f9675650640 1 --2- 172.21.15.16:0/448477814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96700d12a0 0x7f9670151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.966+0000 7f9675650640 1 -- 172.21.15.16:0/448477814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9670152400 con 0x7f967015df30 2024-04-25T10:09:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.966+0000 7f9674e4f640 1 --2- 172.21.15.16:0/448477814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96700d12a0 0x7f9670151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.966+0000 7f9675650640 1 --2- 172.21.15.16:0/448477814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f967015df30 0x7f96701583b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9668009f40 tx=0x7f9668004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:05.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.966+0000 7f966d7fa640 1 -- 172.21.15.16:0/448477814 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966803f020 con 0x7f967015df30 2024-04-25T10:09:05.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.966+0000 7f966d7fa640 1 -- 172.21.15.16:0/448477814 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9668037040 con 0x7f967015df30 2024-04-25T10:09:05.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.966+0000 7f96770b3640 1 -- 172.21.15.16:0/448477814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96700b6440 con 0x7f967015df30 2024-04-25T10:09:05.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.966+0000 7f966d7fa640 1 -- 172.21.15.16:0/448477814 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668036070 con 0x7f967015df30 2024-04-25T10:09:05.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.966+0000 7f96770b3640 1 -- 172.21.15.16:0/448477814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96700b6a00 con 0x7f967015df30 2024-04-25T10:09:05.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.967+0000 7f96770b3640 1 -- 172.21.15.16:0/448477814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96700750b0 con 0x7f967015df30 2024-04-25T10:09:05.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.968+0000 7f966d7fa640 1 -- 172.21.15.16:0/448477814 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9668042020 con 0x7f967015df30 2024-04-25T10:09:05.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.968+0000 7f966d7fa640 1 --2- 172.21.15.16:0/448477814 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f965406b4d0 0x7f965406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.968+0000 7f966d7fa640 1 -- 172.21.15.16:0/448477814 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f96680aa530 con 0x7f967015df30 2024-04-25T10:09:05.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.968+0000 7f9674e4f640 1 --2- 172.21.15.16:0/448477814 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f965406b4d0 0x7f965406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.969+0000 7f9674e4f640 1 --2- 172.21.15.16:0/448477814 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f965406b4d0 0x7f965406d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9664006fd0 tx=0x7f9664008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:05.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:05.972+0000 7f966d7fa640 1 -- 172.21.15.16:0/448477814 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9668049030 con 0x7f967015df30 2024-04-25T10:09:06.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.156+0000 7f96770b3640 1 -- 172.21.15.16:0/448477814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f967009a980 con 0x7f967015df30 2024-04-25T10:09:06.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.156+0000 7f966d7fa640 1 -- 172.21.15.16:0/448477814 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v61) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9668078db0 con 0x7f967015df30 2024-04-25T10:09:06.163 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:09:06.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.158+0000 7f96770b3640 1 -- 172.21.15.16:0/448477814 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f965406b4d0 msgr2=0x7f965406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.158+0000 7f96770b3640 1 --2- 172.21.15.16:0/448477814 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f965406b4d0 0x7f965406d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9664006fd0 tx=0x7f9664008040 comp rx=0 tx=0).stop 2024-04-25T10:09:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.158+0000 7f96770b3640 1 -- 172.21.15.16:0/448477814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f967015df30 msgr2=0x7f96701583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.158+0000 7f96770b3640 1 --2- 172.21.15.16:0/448477814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f967015df30 0x7f96701583b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9668009f40 tx=0x7f9668004a40 comp rx=0 tx=0).stop 2024-04-25T10:09:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.158+0000 7f96770b3640 1 -- 172.21.15.16:0/448477814 shutdown_connections 2024-04-25T10:09:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.158+0000 7f96770b3640 1 --2- 172.21.15.16:0/448477814 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f965406b4d0 0x7f965406d990 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.158+0000 7f96770b3640 1 --2- 172.21.15.16:0/448477814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f967015df30 0x7f96701583b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.158+0000 7f96770b3640 1 --2- 172.21.15.16:0/448477814 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f967015db50 0x7f96700b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.158+0000 7f96770b3640 1 --2- 172.21.15.16:0/448477814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96700d12a0 0x7f9670151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.158+0000 7f96770b3640 1 -- 172.21.15.16:0/448477814 >> 172.21.15.16:0/448477814 conn(0x7f96700b3430 msgr2=0x7f96701e6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.158+0000 7f96770b3640 1 -- 172.21.15.16:0/448477814 shutdown_connections 2024-04-25T10:09:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.158+0000 7f96770b3640 1 -- 172.21.15.16:0/448477814 wait complete. 2024-04-25T10:09:06.176 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:09:06.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.274+0000 7f9961fed640 1 Processor -- start 2024-04-25T10:09:06.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.274+0000 7f9961fed640 1 -- start start 2024-04-25T10:09:06.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.274+0000 7f9961fed640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f995c0b97c0 0x7f995c0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.274+0000 7f9961fed640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995c1627a0 con 0x7f995c0b5a30 2024-04-25T10:09:06.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f9961fed640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995c1621e0 con 0x7f995c15dc80 2024-04-25T10:09:06.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f9961fed640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995c0b5ea0 con 0x7f995c0b97c0 2024-04-25T10:09:06.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f995c0b5a30 0x7f995c0d2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32846/0 (socket says 172.21.15.16:32846) 2024-04-25T10:09:06.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995bfff640 1 -- 172.21.15.16:0/31458768 learned_addr learned my addr 172.21.15.16:0/31458768 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:06.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995affd640 1 --2- 172.21.15.16:0/31458768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f995c0b97c0 0x7f995c0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995a7fc640 1 -- 172.21.15.16:0/31458768 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010878678 0 0) 0x7f995c1627a0 con 0x7f995c0b5a30 2024-04-25T10:09:06.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995a7fc640 1 -- 172.21.15.16:0/31458768 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9944003520 con 0x7f995c0b5a30 2024-04-25T10:09:06.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995a7fc640 1 -- 172.21.15.16:0/31458768 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3937987491 0 0) 0x7f9944003520 con 0x7f995c0b5a30 2024-04-25T10:09:06.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995a7fc640 1 -- 172.21.15.16:0/31458768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f995c0b97c0 msgr2=0x7f995c0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:06.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995a7fc640 1 --2- 172.21.15.16:0/31458768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f995c0b97c0 0x7f995c0b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995a7fc640 1 -- 172.21.15.16:0/31458768 >> v1:172.21.15.186:6789/0 conn(0x7f995c15dc80 legacy=0x7f995c161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995a7fc640 1 -- 172.21.15.16:0/31458768 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995c1624c0 con 0x7f995c0b5a30 2024-04-25T10:09:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995a7fc640 1 -- 172.21.15.16:0/31458768 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f99540019f0 con 0x7f995c0b5a30 2024-04-25T10:09:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995a7fc640 1 -- 172.21.15.16:0/31458768 >> v1:172.21.15.16:6789/0 conn(0x7f995c0b5a30 legacy=0x7f995c0d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995a7fc640 1 --2- 172.21.15.16:0/31458768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99440039e0 0x7f9944003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995a7fc640 1 --2- 172.21.15.16:0/31458768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9944004800 0x7f9944004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995a7fc640 1 --2- 172.21.15.16:0/31458768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f995c0b97c0 0x7f99440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995a7fc640 1 -- 172.21.15.16:0/31458768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c0b5ea0 con 0x7f99440039e0 2024-04-25T10:09:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995a7fc640 1 -- 172.21.15.16:0/31458768 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994400ba70 con 0x7f9944004800 2024-04-25T10:09:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995a7fc640 1 -- 172.21.15.16:0/31458768 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f994400bc70 con 0x7f995c0b97c0 2024-04-25T10:09:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995b7fe640 1 --2- 172.21.15.16:0/31458768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99440039e0 0x7f9944003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.275+0000 7f995affd640 1 --2- 172.21.15.16:0/31458768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9944004800 0x7f9944004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.276+0000 7f995bfff640 1 --2- 172.21.15.16:0/31458768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f995c0b97c0 0x7f99440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.276+0000 7f995b7fe640 1 -- 172.21.15.16:0/31458768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f995c0b97c0 msgr2=0x7f99440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:06.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.276+0000 7f995b7fe640 1 --2- 172.21.15.16:0/31458768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f995c0b97c0 0x7f99440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.276+0000 7f995b7fe640 1 -- 172.21.15.16:0/31458768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9944004800 msgr2=0x7f9944004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:06.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.276+0000 7f995b7fe640 1 --2- 172.21.15.16:0/31458768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9944004800 0x7f9944004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.276+0000 7f995b7fe640 1 -- 172.21.15.16:0/31458768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9944003520 con 0x7f99440039e0 2024-04-25T10:09:06.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.276+0000 7f995b7fe640 1 --2- 172.21.15.16:0/31458768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99440039e0 0x7f9944003e00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f994c001720 tx=0x7f9944005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6268eb042298a858 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.277+0000 7f995a7fc640 1 -- 172.21.15.16:0/31458768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c001370 con 0x7f99440039e0 2024-04-25T10:09:06.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.277+0000 7f995a7fc640 1 -- 172.21.15.16:0/31458768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994c0052d0 con 0x7f99440039e0 2024-04-25T10:09:06.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.277+0000 7f995a7fc640 1 -- 172.21.15.16:0/31458768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c02ab90 con 0x7f99440039e0 2024-04-25T10:09:06.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.277+0000 7f9961fed640 1 -- 172.21.15.16:0/31458768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99440039e0 msgr2=0x7f9944003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.277+0000 7f9961fed640 1 --2- 172.21.15.16:0/31458768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99440039e0 0x7f9944003e00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f994c001720 tx=0x7f9944005830 comp rx=0 tx=0).stop 2024-04-25T10:09:06.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.277+0000 7f995bfff640 1 -- 172.21.15.16:0/31458768 reap_dead start 2024-04-25T10:09:06.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.277+0000 7f9961fed640 1 -- 172.21.15.16:0/31458768 shutdown_connections 2024-04-25T10:09:06.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.277+0000 7f9961fed640 1 -- 172.21.15.16:0/31458768 >> 172.21.15.16:0/31458768 conn(0x7f995c0d04f0 msgr2=0x7f995c0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:06.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.277+0000 7f9961fed640 1 -- 172.21.15.16:0/31458768 shutdown_connections 2024-04-25T10:09:06.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.277+0000 7f9961fed640 1 -- 172.21.15.16:0/31458768 wait complete. 2024-04-25T10:09:06.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.278+0000 7f9961fed640 1 Processor -- start 2024-04-25T10:09:06.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.278+0000 7f9961fed640 1 -- start start 2024-04-25T10:09:06.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.279+0000 7f9961fed640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99440039e0 0x7f995c156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.279+0000 7f9961fed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c0cd010 0x7f995c157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.279+0000 7f9961fed640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f995c158400 0x7f995c0b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.279+0000 7f9961fed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c15e060 con 0x7f995c0cd010 2024-04-25T10:09:06.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.279+0000 7f9961fed640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c161f80 con 0x7f995c158400 2024-04-25T10:09:06.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.279+0000 7f9961fed640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995c0b54f0 con 0x7f99440039e0 2024-04-25T10:09:06.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.279+0000 7f995affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c0cd010 0x7f995c157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.279+0000 7f995affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c0cd010 0x7f995c157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41030/0 (socket says 172.21.15.16:41030) 2024-04-25T10:09:06.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.279+0000 7f995affd640 1 -- 172.21.15.16:0/1567772377 learned_addr learned my addr 172.21.15.16:0/1567772377 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:06.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.279+0000 7f995b7fe640 1 --2- 172.21.15.16:0/1567772377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99440039e0 0x7f995c156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.279+0000 7f995bfff640 1 --2- 172.21.15.16:0/1567772377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f995c158400 0x7f995c0b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.280+0000 7f995affd640 1 -- 172.21.15.16:0/1567772377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99440039e0 msgr2=0x7f995c156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.280+0000 7f995affd640 1 --2- 172.21.15.16:0/1567772377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99440039e0 0x7f995c156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.280+0000 7f995affd640 1 -- 172.21.15.16:0/1567772377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f995c158400 msgr2=0x7f995c0b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.280+0000 7f995affd640 1 --2- 172.21.15.16:0/1567772377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f995c158400 0x7f995c0b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.280+0000 7f995affd640 1 -- 172.21.15.16:0/1567772377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995c1572c0 con 0x7f995c0cd010 2024-04-25T10:09:06.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.280+0000 7f995affd640 1 --2- 172.21.15.16:0/1567772377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c0cd010 0x7f995c157ec0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9950011960 tx=0x7f9950011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.280+0000 7f9943fff640 1 -- 172.21.15.16:0/1567772377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950019070 con 0x7f995c0cd010 2024-04-25T10:09:06.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.280+0000 7f9943fff640 1 -- 172.21.15.16:0/1567772377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9950004240 con 0x7f995c0cd010 2024-04-25T10:09:06.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.280+0000 7f9961fed640 1 -- 172.21.15.16:0/1567772377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995c0cbf70 con 0x7f995c0cd010 2024-04-25T10:09:06.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.280+0000 7f9943fff640 1 -- 172.21.15.16:0/1567772377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995000f7c0 con 0x7f995c0cd010 2024-04-25T10:09:06.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.280+0000 7f9961fed640 1 -- 172.21.15.16:0/1567772377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995c0cc530 con 0x7f995c0cd010 2024-04-25T10:09:06.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.281+0000 7f9961fed640 1 -- 172.21.15.16:0/1567772377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f995c0750b0 con 0x7f995c0cd010 2024-04-25T10:09:06.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.281+0000 7f9943fff640 1 -- 172.21.15.16:0/1567772377 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f995001c070 con 0x7f995c0cd010 2024-04-25T10:09:06.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.282+0000 7f9943fff640 1 --2- 172.21.15.16:0/1567772377 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f994806b500 0x7f994806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.282+0000 7f9943fff640 1 -- 172.21.15.16:0/1567772377 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f995008a530 con 0x7f995c0cd010 2024-04-25T10:09:06.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.282+0000 7f995b7fe640 1 --2- 172.21.15.16:0/1567772377 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f994806b500 0x7f994806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.283+0000 7f995b7fe640 1 --2- 172.21.15.16:0/1567772377 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f994806b500 0x7f994806d9c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f994c032000 tx=0x7f994c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.286+0000 7f9943fff640 1 -- 172.21.15.16:0/1567772377 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9950058c80 con 0x7f995c0cd010 2024-04-25T10:09:06.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.465+0000 7f9961fed640 1 -- 172.21.15.16:0/1567772377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f995c0b6b60 con 0x7f995c0cd010 2024-04-25T10:09:06.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.466+0000 7f9943fff640 1 -- 172.21.15.16:0/1567772377 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) v1 ==== 100+0+11 (secure 0 0 0) 0x7f995005cd20 con 0x7f995c0cd010 2024-04-25T10:09:06.472 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:09:06.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.467+0000 7f9961fed640 1 -- 172.21.15.16:0/1567772377 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f994806b500 msgr2=0x7f994806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.467+0000 7f9961fed640 1 --2- 172.21.15.16:0/1567772377 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f994806b500 0x7f994806d9c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f994c032000 tx=0x7f994c02e000 comp rx=0 tx=0).stop 2024-04-25T10:09:06.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.467+0000 7f9961fed640 1 -- 172.21.15.16:0/1567772377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c0cd010 msgr2=0x7f995c157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.467+0000 7f9961fed640 1 --2- 172.21.15.16:0/1567772377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c0cd010 0x7f995c157ec0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9950011960 tx=0x7f9950011e20 comp rx=0 tx=0).stop 2024-04-25T10:09:06.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.467+0000 7f9961fed640 1 -- 172.21.15.16:0/1567772377 shutdown_connections 2024-04-25T10:09:06.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.467+0000 7f9961fed640 1 --2- 172.21.15.16:0/1567772377 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f994806b500 0x7f994806d9c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.467+0000 7f9961fed640 1 --2- 172.21.15.16:0/1567772377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f995c158400 0x7f995c0b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.467+0000 7f9961fed640 1 --2- 172.21.15.16:0/1567772377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c0cd010 0x7f995c157ec0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.467+0000 7f9961fed640 1 --2- 172.21.15.16:0/1567772377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99440039e0 0x7f995c156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.468+0000 7f9961fed640 1 -- 172.21.15.16:0/1567772377 >> 172.21.15.16:0/1567772377 conn(0x7f995c0d04f0 msgr2=0x7f995c0d22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:06.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.468+0000 7f9961fed640 1 -- 172.21.15.16:0/1567772377 shutdown_connections 2024-04-25T10:09:06.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.468+0000 7f9961fed640 1 -- 172.21.15.16:0/1567772377 wait complete. 2024-04-25T10:09:06.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:09:06.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.580+0000 7f6f99a53640 1 Processor -- start 2024-04-25T10:09:06.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.580+0000 7f6f99a53640 1 -- start start 2024-04-25T10:09:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.581+0000 7f6f99a53640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f940ceeb0 0x7f6f94168a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.581+0000 7f6f99a53640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f94161f80 con 0x7f6f940b9720 2024-04-25T10:09:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.581+0000 7f6f99a53640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f94162260 con 0x7f6f94161090 2024-04-25T10:09:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.581+0000 7f6f99a53640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f940d21a0 con 0x7f6f940ceeb0 2024-04-25T10:09:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.581+0000 7f6f8bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6f940b9720 0x7f6f940ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32848/0 (socket says 172.21.15.16:32848) 2024-04-25T10:09:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.581+0000 7f6f8bfff640 1 -- 172.21.15.16:0/4280231323 learned_addr learned my addr 172.21.15.16:0/4280231323 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.581+0000 7f6f937fe640 1 --2- 172.21.15.16:0/4280231323 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f940ceeb0 0x7f6f94168a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.581+0000 7f6f927fc640 1 -- 172.21.15.16:0/4280231323 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185816737 0 0) 0x7f6f94161f80 con 0x7f6f940b9720 2024-04-25T10:09:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.581+0000 7f6f927fc640 1 -- 172.21.15.16:0/4280231323 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f80003520 con 0x7f6f940b9720 2024-04-25T10:09:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.581+0000 7f6f927fc640 1 -- 172.21.15.16:0/4280231323 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767476277 0 0) 0x7f6f94162260 con 0x7f6f94161090 2024-04-25T10:09:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.581+0000 7f6f937fe640 1 -- 172.21.15.16:0/4280231323 >> v1:172.21.15.186:6789/0 conn(0x7f6f94161090 legacy=0x7f6f94161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.582+0000 7f6f937fe640 1 -- 172.21.15.16:0/4280231323 >> v1:172.21.15.16:6789/0 conn(0x7f6f940b9720 legacy=0x7f6f940ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.582+0000 7f6f937fe640 1 -- 172.21.15.16:0/4280231323 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f94162540 con 0x7f6f940ceeb0 2024-04-25T10:09:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.582+0000 7f6f937fe640 1 --2- 172.21.15.16:0/4280231323 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f940ceeb0 0x7f6f94168a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009dc0 tx=0x7f6f8c02a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47fc408a63f2d7f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.583+0000 7f6f927fc640 1 -- 172.21.15.16:0/4280231323 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c035020 con 0x7f6f940ceeb0 2024-04-25T10:09:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.583+0000 7f6f927fc640 1 -- 172.21.15.16:0/4280231323 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8c007540 con 0x7f6f940ceeb0 2024-04-25T10:09:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.583+0000 7f6f927fc640 1 -- 172.21.15.16:0/4280231323 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c001690 con 0x7f6f940ceeb0 2024-04-25T10:09:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.583+0000 7f6f99a53640 1 -- 172.21.15.16:0/4280231323 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f940ceeb0 msgr2=0x7f6f94168a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.583+0000 7f6f99a53640 1 --2- 172.21.15.16:0/4280231323 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f940ceeb0 0x7f6f94168a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009dc0 tx=0x7f6f8c02a5b0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.583+0000 7f6f99a53640 1 -- 172.21.15.16:0/4280231323 shutdown_connections 2024-04-25T10:09:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.583+0000 7f6f99a53640 1 --2- 172.21.15.16:0/4280231323 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f940ceeb0 0x7f6f94168a60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.583+0000 7f6f99a53640 1 -- 172.21.15.16:0/4280231323 >> 172.21.15.16:0/4280231323 conn(0x7f6f940b55e0 msgr2=0x7f6f940b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.583+0000 7f6f99a53640 1 -- 172.21.15.16:0/4280231323 shutdown_connections 2024-04-25T10:09:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.583+0000 7f6f99a53640 1 -- 172.21.15.16:0/4280231323 wait complete. 2024-04-25T10:09:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.584+0000 7f6f99a53640 1 Processor -- start 2024-04-25T10:09:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.584+0000 7f6f99a53640 1 -- start start 2024-04-25T10:09:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.585+0000 7f6f99a53640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f940b9720 0x7f6f94151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.585+0000 7f6f99a53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f940ceeb0 0x7f6f940caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.585+0000 7f6f99a53640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f94161090 0x7f6f940cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.585+0000 7f6f99a53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f940d3640 con 0x7f6f940ceeb0 2024-04-25T10:09:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.585+0000 7f6f99a53640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f94161470 con 0x7f6f940b9720 2024-04-25T10:09:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.585+0000 7f6f99a53640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f940d37c0 con 0x7f6f94161090 2024-04-25T10:09:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.585+0000 7f6f8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f940ceeb0 0x7f6f940caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.585+0000 7f6f8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f940ceeb0 0x7f6f940caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41032/0 (socket says 172.21.15.16:41032) 2024-04-25T10:09:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.585+0000 7f6f92ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f940b9720 0x7f6f94151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.585+0000 7f6f8bfff640 1 -- 172.21.15.16:0/1326870807 learned_addr learned my addr 172.21.15.16:0/1326870807 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.585+0000 7f6f937fe640 1 --2- 172.21.15.16:0/1326870807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f94161090 0x7f6f940cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.586+0000 7f6f8bfff640 1 -- 172.21.15.16:0/1326870807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f94161090 msgr2=0x7f6f940cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.586+0000 7f6f8bfff640 1 --2- 172.21.15.16:0/1326870807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f94161090 0x7f6f940cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.586+0000 7f6f8bfff640 1 -- 172.21.15.16:0/1326870807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f940b9720 msgr2=0x7f6f94151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.586+0000 7f6f8bfff640 1 --2- 172.21.15.16:0/1326870807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f940b9720 0x7f6f94151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.586+0000 7f6f8bfff640 1 -- 172.21.15.16:0/1326870807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f94154fb0 con 0x7f6f940ceeb0 2024-04-25T10:09:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.586+0000 7f6f8bfff640 1 --2- 172.21.15.16:0/1326870807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f940ceeb0 0x7f6f940caba0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00c750 tx=0x7f6f7c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.586+0000 7f6f90ff9640 1 -- 172.21.15.16:0/1326870807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c018070 con 0x7f6f940ceeb0 2024-04-25T10:09:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.586+0000 7f6f90ff9640 1 -- 172.21.15.16:0/1326870807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7c007500 con 0x7f6f940ceeb0 2024-04-25T10:09:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.586+0000 7f6f99a53640 1 -- 172.21.15.16:0/1326870807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f940c8220 con 0x7f6f940ceeb0 2024-04-25T10:09:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.586+0000 7f6f90ff9640 1 -- 172.21.15.16:0/1326870807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c00ce60 con 0x7f6f940ceeb0 2024-04-25T10:09:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.586+0000 7f6f99a53640 1 -- 172.21.15.16:0/1326870807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f940c87e0 con 0x7f6f940ceeb0 2024-04-25T10:09:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.587+0000 7f6f99a53640 1 -- 172.21.15.16:0/1326870807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f940cfbe0 con 0x7f6f940ceeb0 2024-04-25T10:09:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.587+0000 7f6f90ff9640 1 -- 172.21.15.16:0/1326870807 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f7c01b070 con 0x7f6f940ceeb0 2024-04-25T10:09:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.588+0000 7f6f90ff9640 1 --2- 172.21.15.16:0/1326870807 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f7406b4d0 0x7f6f7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.588+0000 7f6f90ff9640 1 -- 172.21.15.16:0/1326870807 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6f7c089730 con 0x7f6f940ceeb0 2024-04-25T10:09:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.588+0000 7f6f92ffd640 1 --2- 172.21.15.16:0/1326870807 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f7406b4d0 0x7f6f7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.589+0000 7f6f92ffd640 1 --2- 172.21.15.16:0/1326870807 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f7406b4d0 0x7f6f7406d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6f84000f60 tx=0x7f6f84009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.592+0000 7f6f90ff9640 1 -- 172.21.15.16:0/1326870807 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6f940cfbe0 con 0x7f6f940ceeb0 2024-04-25T10:09:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.784+0000 7f6f99a53640 1 -- 172.21.15.16:0/1326870807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6f940c89e0 con 0x7f6f940ceeb0 2024-04-25T10:09:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.785+0000 7f6f90ff9640 1 -- 172.21.15.16:0/1326870807 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v61) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6f940c89e0 con 0x7f6f940ceeb0 2024-04-25T10:09:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-04-25T10:09:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.786+0000 7f6f99a53640 1 -- 172.21.15.16:0/1326870807 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f7406b4d0 msgr2=0x7f6f7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.786+0000 7f6f99a53640 1 --2- 172.21.15.16:0/1326870807 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f7406b4d0 0x7f6f7406d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6f84000f60 tx=0x7f6f84009040 comp rx=0 tx=0).stop 2024-04-25T10:09:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.787+0000 7f6f99a53640 1 -- 172.21.15.16:0/1326870807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f940ceeb0 msgr2=0x7f6f940caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.787+0000 7f6f99a53640 1 --2- 172.21.15.16:0/1326870807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f940ceeb0 0x7f6f940caba0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00c750 tx=0x7f6f7c00cc10 comp rx=0 tx=0).stop 2024-04-25T10:09:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.787+0000 7f6f99a53640 1 -- 172.21.15.16:0/1326870807 shutdown_connections 2024-04-25T10:09:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.787+0000 7f6f99a53640 1 --2- 172.21.15.16:0/1326870807 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f7406b4d0 0x7f6f7406d990 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.787+0000 7f6f99a53640 1 --2- 172.21.15.16:0/1326870807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f94161090 0x7f6f940cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.787+0000 7f6f99a53640 1 --2- 172.21.15.16:0/1326870807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f940ceeb0 0x7f6f940caba0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.787+0000 7f6f99a53640 1 --2- 172.21.15.16:0/1326870807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f940b9720 0x7f6f94151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.787+0000 7f6f99a53640 1 -- 172.21.15.16:0/1326870807 >> 172.21.15.16:0/1326870807 conn(0x7f6f940b55e0 msgr2=0x7f6f940cdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.787+0000 7f6f99a53640 1 -- 172.21.15.16:0/1326870807 shutdown_connections 2024-04-25T10:09:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:06.787+0000 7f6f99a53640 1 -- 172.21.15.16:0/1326870807 wait complete. 2024-04-25T10:09:06.803 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:06.804 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:06.842 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:09:06.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:06.842 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:09:06.898 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:09:06.898 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:06.916 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:09:06.916 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:09:06.916 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:06.954 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:09:06.955 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:09:06.955 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:06.972 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:09:06.973 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:07.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.089+0000 7fb37041f640 1 Processor -- start 2024-04-25T10:09:07.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.089+0000 7fb37041f640 1 -- start start 2024-04-25T10:09:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.090+0000 7fb37041f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb36815dce0 0x7fb3680d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.090+0000 7fb37041f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb3680b5b60 0x7fb3680b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.090+0000 7fb37041f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36815e0c0 con 0x7fb3680d35b0 2024-04-25T10:09:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.090+0000 7fb37041f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3680d2510 con 0x7fb3680b5b60 2024-04-25T10:09:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.090+0000 7fb37041f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3680d27f0 con 0x7fb36815dce0 2024-04-25T10:09:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.090+0000 7fb36e9bc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb3680b5b60 0x7fb3680b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.090+0000 7fb36e9bc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb3680b5b60 0x7fb3680b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41048/0 (socket says 172.21.15.16:41048) 2024-04-25T10:09:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.090+0000 7fb36e9bc640 1 -- 172.21.15.16:0/2219079029 learned_addr learned my addr 172.21.15.16:0/2219079029 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:07.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.090+0000 7fb36e9bc640 1 -- 172.21.15.16:0/2219079029 >> v2:172.21.15.186:3300/0 conn(0x7fb36815dce0 msgr2=0x7fb3680d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.090+0000 7fb36e9bc640 1 --2- 172.21.15.16:0/2219079029 >> v2:172.21.15.186:3300/0 conn(0x7fb36815dce0 0x7fb3680d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.090+0000 7fb36e9bc640 1 -- 172.21.15.16:0/2219079029 >> v1:172.21.15.186:6789/0 conn(0x7fb3680d35b0 legacy=0x7fb3680b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.090+0000 7fb36e9bc640 1 -- 172.21.15.16:0/2219079029 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3680d2ad0 con 0x7fb3680b5b60 2024-04-25T10:09:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.091+0000 7fb36e9bc640 1 --2- 172.21.15.16:0/2219079029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3680b5b60 0x7fb3680b6b00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb3680ce220 tx=0x7fb36002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f054e1ea682815f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.091+0000 7fb36d1b9640 1 -- 172.21.15.16:0/2219079029 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb360036070 con 0x7fb3680b5b60 2024-04-25T10:09:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.091+0000 7fb36d1b9640 1 -- 172.21.15.16:0/2219079029 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb360031600 con 0x7fb3680b5b60 2024-04-25T10:09:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.091+0000 7fb36d1b9640 1 -- 172.21.15.16:0/2219079029 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3600319d0 con 0x7fb3680b5b60 2024-04-25T10:09:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.092+0000 7fb37041f640 1 -- 172.21.15.16:0/2219079029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3680b5b60 msgr2=0x7fb3680b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.092+0000 7fb37041f640 1 --2- 172.21.15.16:0/2219079029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3680b5b60 0x7fb3680b6b00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb3680ce220 tx=0x7fb36002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.092+0000 7fb37041f640 1 -- 172.21.15.16:0/2219079029 shutdown_connections 2024-04-25T10:09:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.092+0000 7fb37041f640 1 --2- 172.21.15.16:0/2219079029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3680b5b60 0x7fb3680b6b00 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.092+0000 7fb37041f640 1 --2- 172.21.15.16:0/2219079029 >> v2:172.21.15.186:3300/0 conn(0x7fb36815dce0 0x7fb3680d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.092+0000 7fb37041f640 1 -- 172.21.15.16:0/2219079029 >> 172.21.15.16:0/2219079029 conn(0x7fb368072900 msgr2=0x7fb3680d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.092+0000 7fb37041f640 1 -- 172.21.15.16:0/2219079029 shutdown_connections 2024-04-25T10:09:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.092+0000 7fb37041f640 1 -- 172.21.15.16:0/2219079029 wait complete. 2024-04-25T10:09:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.093+0000 7fb37041f640 1 Processor -- start 2024-04-25T10:09:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.093+0000 7fb37041f640 1 -- start start 2024-04-25T10:09:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.094+0000 7fb37041f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3680b5b60 0x7fb368153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.094+0000 7fb37041f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3680d35b0 0x7fb36815e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.094+0000 7fb37041f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb36815dce0 0x7fb36815ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.094+0000 7fb37041f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3680b9820 con 0x7fb36815dce0 2024-04-25T10:09:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.094+0000 7fb37041f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3680d3090 con 0x7fb3680d35b0 2024-04-25T10:09:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.094+0000 7fb37041f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3680b99a0 con 0x7fb3680b5b60 2024-04-25T10:09:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.094+0000 7fb36e9bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb36815dce0 0x7fb36815ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.094+0000 7fb36e9bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb36815dce0 0x7fb36815ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41062/0 (socket says 172.21.15.16:41062) 2024-04-25T10:09:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.094+0000 7fb36e9bc640 1 -- 172.21.15.16:0/41165840 learned_addr learned my addr 172.21.15.16:0/41165840 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.094+0000 7fb36d9ba640 1 --2- 172.21.15.16:0/41165840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3680d35b0 0x7fb36815e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.094+0000 7fb36e1bb640 1 --2- 172.21.15.16:0/41165840 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3680b5b60 0x7fb368153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.094+0000 7fb36e9bc640 1 -- 172.21.15.16:0/41165840 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3680b5b60 msgr2=0x7fb368153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.094+0000 7fb36e9bc640 1 --2- 172.21.15.16:0/41165840 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3680b5b60 0x7fb368153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.094+0000 7fb36e9bc640 1 -- 172.21.15.16:0/41165840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3680d35b0 msgr2=0x7fb36815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.094+0000 7fb36e9bc640 1 --2- 172.21.15.16:0/41165840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3680d35b0 0x7fb36815e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.094+0000 7fb36e9bc640 1 -- 172.21.15.16:0/41165840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3680cd340 con 0x7fb36815dce0 2024-04-25T10:09:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.094+0000 7fb36e9bc640 1 --2- 172.21.15.16:0/41165840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb36815dce0 0x7fb36815ef30 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb360034040 tx=0x7fb360004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.094+0000 7fb35effd640 1 -- 172.21.15.16:0/41165840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb360041020 con 0x7fb36815dce0 2024-04-25T10:09:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.095+0000 7fb35effd640 1 -- 172.21.15.16:0/41165840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb360007780 con 0x7fb36815dce0 2024-04-25T10:09:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.095+0000 7fb35effd640 1 -- 172.21.15.16:0/41165840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb360036040 con 0x7fb36815dce0 2024-04-25T10:09:07.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.095+0000 7fb37041f640 1 -- 172.21.15.16:0/41165840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3680cd620 con 0x7fb36815dce0 2024-04-25T10:09:07.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.095+0000 7fb37041f640 1 -- 172.21.15.16:0/41165840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3680c8780 con 0x7fb36815dce0 2024-04-25T10:09:07.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.096+0000 7fb37041f640 1 -- 172.21.15.16:0/41165840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3680749b0 con 0x7fb36815dce0 2024-04-25T10:09:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.096+0000 7fb35effd640 1 -- 172.21.15.16:0/41165840 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb360044020 con 0x7fb36815dce0 2024-04-25T10:09:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.096+0000 7fb35effd640 1 --2- 172.21.15.16:0/41165840 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb34c06b4d0 0x7fb34c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.097+0000 7fb35effd640 1 -- 172.21.15.16:0/41165840 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb3600aaf00 con 0x7fb36815dce0 2024-04-25T10:09:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.097+0000 7fb36e1bb640 1 --2- 172.21.15.16:0/41165840 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb34c06b4d0 0x7fb34c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.097+0000 7fb36e1bb640 1 --2- 172.21.15.16:0/41165840 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb34c06b4d0 0x7fb34c06d990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb36400b810 tx=0x7fb364008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.101+0000 7fb35effd640 1 -- 172.21.15.16:0/41165840 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb36004b030 con 0x7fb36815dce0 2024-04-25T10:09:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.316+0000 7fb37041f640 1 -- 172.21.15.16:0/41165840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3680c6fe0 con 0x7fb36815dce0 2024-04-25T10:09:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.316+0000 7fb35effd640 1 -- 172.21.15.16:0/41165840 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4945 (secure 0 0 0) 0x7fb360079e90 con 0x7fb36815dce0 2024-04-25T10:09:07.322 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:07.322 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2270839865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2270839865},{"type":"v1","addr":"172.21.15.186:6836","nonce":2270839865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5647,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/925282579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":925282579},{"type":"v1","addr":"172.21.15.186:6837","nonce":925282579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2471714328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2471714328},{"type":"v1","addr":"172.21.15.16:6835","nonce":2471714328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:50.665436+0000","modified":"2024-04-25T10:08:39.328915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35141":{"gid":35141,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.16:6834/3311584521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3311584521},{"type":"v1","addr":"172.21.15.16:6834","nonce":3311584521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:09:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-04-25T10:09:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.318+0000 7fb37041f640 1 -- 172.21.15.16:0/41165840 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb34c06b4d0 msgr2=0x7fb34c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.318+0000 7fb37041f640 1 --2- 172.21.15.16:0/41165840 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb34c06b4d0 0x7fb34c06d990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb36400b810 tx=0x7fb364008040 comp rx=0 tx=0).stop 2024-04-25T10:09:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.318+0000 7fb37041f640 1 -- 172.21.15.16:0/41165840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb36815dce0 msgr2=0x7fb36815ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.318+0000 7fb37041f640 1 --2- 172.21.15.16:0/41165840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb36815dce0 0x7fb36815ef30 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb360034040 tx=0x7fb360004a40 comp rx=0 tx=0).stop 2024-04-25T10:09:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.318+0000 7fb37041f640 1 -- 172.21.15.16:0/41165840 shutdown_connections 2024-04-25T10:09:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.318+0000 7fb37041f640 1 --2- 172.21.15.16:0/41165840 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb34c06b4d0 0x7fb34c06d990 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.318+0000 7fb37041f640 1 --2- 172.21.15.16:0/41165840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb36815dce0 0x7fb36815ef30 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.318+0000 7fb37041f640 1 --2- 172.21.15.16:0/41165840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3680d35b0 0x7fb36815e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.318+0000 7fb37041f640 1 --2- 172.21.15.16:0/41165840 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3680b5b60 0x7fb368153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.318+0000 7fb37041f640 1 -- 172.21.15.16:0/41165840 >> 172.21.15.16:0/41165840 conn(0x7fb368072900 msgr2=0x7fb36815cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.318+0000 7fb37041f640 1 -- 172.21.15.16:0/41165840 shutdown_connections 2024-04-25T10:09:07.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.318+0000 7fb37041f640 1 -- 172.21.15.16:0/41165840 wait complete. 2024-04-25T10:09:07.336 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:07.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.432+0000 7fea20ecb640 1 Processor -- start 2024-04-25T10:09:07.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.432+0000 7fea20ecb640 1 -- start start 2024-04-25T10:09:07.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.433+0000 7fea20ecb640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fea1c0aecf0 0x7fea1c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.433+0000 7fea20ecb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea1c15ed70 0x7fea1c0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.433+0000 7fea20ecb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1c0712c0 con 0x7fea1c15d980 2024-04-25T10:09:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.433+0000 7fea20ecb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1c071ed0 con 0x7fea1c15ed70 2024-04-25T10:09:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.433+0000 7fea20ecb640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1c0715a0 con 0x7fea1c0aecf0 2024-04-25T10:09:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.433+0000 7fea1ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea1c15ed70 0x7fea1c0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.433+0000 7fea1ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea1c15ed70 0x7fea1c0cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41070/0 (socket says 172.21.15.16:41070) 2024-04-25T10:09:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.433+0000 7fea1a59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fea1c15d980 0x7fea1c0ae5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32856/0 (socket says 172.21.15.16:32856) 2024-04-25T10:09:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.433+0000 7fea1ad9d640 1 -- 172.21.15.16:0/2381890215 learned_addr learned my addr 172.21.15.16:0/2381890215 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.433+0000 7fea13fff640 1 --2- 172.21.15.16:0/2381890215 >> v2:172.21.15.186:3300/0 conn(0x7fea1c0aecf0 0x7fea1c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.433+0000 7fea1ad9d640 1 -- 172.21.15.16:0/2381890215 >> v2:172.21.15.186:3300/0 conn(0x7fea1c0aecf0 msgr2=0x7fea1c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.433+0000 7fea1ad9d640 1 --2- 172.21.15.16:0/2381890215 >> v2:172.21.15.186:3300/0 conn(0x7fea1c0aecf0 0x7fea1c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.433+0000 7fea1ad9d640 1 -- 172.21.15.16:0/2381890215 >> v1:172.21.15.16:6789/0 conn(0x7fea1c15d980 legacy=0x7fea1c0ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.433+0000 7fea19d9b640 1 -- 172.21.15.16:0/2381890215 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925148173 0 0) 0x7fea1c0712c0 con 0x7fea1c15d980 2024-04-25T10:09:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.433+0000 7fea1ad9d640 1 -- 172.21.15.16:0/2381890215 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c071880 con 0x7fea1c15ed70 2024-04-25T10:09:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.434+0000 7fea1ad9d640 1 --2- 172.21.15.16:0/2381890215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1c15ed70 0x7fea1c0cf7c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fea1c072050 tx=0x7fea1402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f487a45018ca9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.434+0000 7fea19d9b640 1 -- 172.21.15.16:0/2381890215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea14036070 con 0x7fea1c15ed70 2024-04-25T10:09:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.434+0000 7fea19d9b640 1 -- 172.21.15.16:0/2381890215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea14031600 con 0x7fea1c15ed70 2024-04-25T10:09:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.434+0000 7fea19d9b640 1 -- 172.21.15.16:0/2381890215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea140319d0 con 0x7fea1c15ed70 2024-04-25T10:09:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.434+0000 7fea20ecb640 1 -- 172.21.15.16:0/2381890215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1c15ed70 msgr2=0x7fea1c0cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.434+0000 7fea20ecb640 1 --2- 172.21.15.16:0/2381890215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1c15ed70 0x7fea1c0cf7c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fea1c072050 tx=0x7fea1402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.434+0000 7fea20ecb640 1 -- 172.21.15.16:0/2381890215 shutdown_connections 2024-04-25T10:09:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.434+0000 7fea20ecb640 1 --2- 172.21.15.16:0/2381890215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1c15ed70 0x7fea1c0cf7c0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.434+0000 7fea20ecb640 1 --2- 172.21.15.16:0/2381890215 >> v2:172.21.15.186:3300/0 conn(0x7fea1c0aecf0 0x7fea1c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.435+0000 7fea20ecb640 1 -- 172.21.15.16:0/2381890215 >> 172.21.15.16:0/2381890215 conn(0x7fea1c06aba0 msgr2=0x7fea1c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.435+0000 7fea20ecb640 1 -- 172.21.15.16:0/2381890215 shutdown_connections 2024-04-25T10:09:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.435+0000 7fea20ecb640 1 -- 172.21.15.16:0/2381890215 wait complete. 2024-04-25T10:09:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.436+0000 7fea20ecb640 1 Processor -- start 2024-04-25T10:09:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.436+0000 7fea20ecb640 1 -- start start 2024-04-25T10:09:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea20ecb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea1c15d980 0x7fea1c14ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea20ecb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea1c153860 0x7fea1c0b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea20ecb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1c0b3880 0x7fea1c151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea20ecb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c0ad0f0 con 0x7fea1c0b3880 2024-04-25T10:09:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea20ecb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c0acf70 con 0x7fea1c15d980 2024-04-25T10:09:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea20ecb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c0ad270 con 0x7fea1c153860 2024-04-25T10:09:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea1ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1c0b3880 0x7fea1c151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea1ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1c0b3880 0x7fea1c151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41080/0 (socket says 172.21.15.16:41080) 2024-04-25T10:09:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea1ad9d640 1 -- 172.21.15.16:0/3875434965 learned_addr learned my addr 172.21.15.16:0/3875434965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea1a59c640 1 --2- 172.21.15.16:0/3875434965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea1c15d980 0x7fea1c14ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea13fff640 1 --2- 172.21.15.16:0/3875434965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea1c153860 0x7fea1c0b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea1ad9d640 1 -- 172.21.15.16:0/3875434965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea1c153860 msgr2=0x7fea1c0b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea1ad9d640 1 --2- 172.21.15.16:0/3875434965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea1c153860 0x7fea1c0b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea1ad9d640 1 -- 172.21.15.16:0/3875434965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea1c15d980 msgr2=0x7fea1c14ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea1ad9d640 1 --2- 172.21.15.16:0/3875434965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea1c15d980 0x7fea1c14ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea1ad9d640 1 -- 172.21.15.16:0/3875434965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c159890 con 0x7fea1c0b3880 2024-04-25T10:09:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea1ad9d640 1 --2- 172.21.15.16:0/3875434965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1c0b3880 0x7fea1c151d80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fea1402ac90 tx=0x7fea14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea12ffd640 1 -- 172.21.15.16:0/3875434965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea14036050 con 0x7fea1c0b3880 2024-04-25T10:09:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea12ffd640 1 -- 172.21.15.16:0/3875434965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea14042070 con 0x7fea1c0b3880 2024-04-25T10:09:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea20ecb640 1 -- 172.21.15.16:0/3875434965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1c159b70 con 0x7fea1c0b3880 2024-04-25T10:09:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea12ffd640 1 -- 172.21.15.16:0/3875434965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea14009510 con 0x7fea1c0b3880 2024-04-25T10:09:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.437+0000 7fea20ecb640 1 -- 172.21.15.16:0/3875434965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1c15a130 con 0x7fea1c0b3880 2024-04-25T10:09:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.438+0000 7fea20ecb640 1 -- 172.21.15.16:0/3875434965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea1c0749b0 con 0x7fea1c0b3880 2024-04-25T10:09:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.439+0000 7fea12ffd640 1 -- 172.21.15.16:0/3875434965 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea14007bd0 con 0x7fea1c0b3880 2024-04-25T10:09:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.439+0000 7fea12ffd640 1 --2- 172.21.15.16:0/3875434965 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe9fc06b4d0 0x7fe9fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.440+0000 7fea12ffd640 1 -- 172.21.15.16:0/3875434965 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fea140ab080 con 0x7fea1c0b3880 2024-04-25T10:09:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.440+0000 7fea1a59c640 1 --2- 172.21.15.16:0/3875434965 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe9fc06b4d0 0x7fe9fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.440+0000 7fea1a59c640 1 --2- 172.21.15.16:0/3875434965 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe9fc06b4d0 0x7fe9fc06d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fea0c000f60 tx=0x7fea0c007b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.444+0000 7fea12ffd640 1 -- 172.21.15.16:0/3875434965 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea14079f50 con 0x7fea1c0b3880 2024-04-25T10:09:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.624+0000 7fea20ecb640 1 -- 172.21.15.16:0/3875434965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea1c06d4c0 con 0x7fea1c0b3880 2024-04-25T10:09:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.625+0000 7fea12ffd640 1 -- 172.21.15.16:0/3875434965 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16761 (secure 0 0 0) 0x7fea14079850 con 0x7fea1c0b3880 2024-04-25T10:09:07.631 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:07.632 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":61,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:08:52.605846+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:48.948980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:49.638457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.19","mappings":[{"from":6,"to":1}]},{"pgid":"9.28","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.627+0000 7fea20ecb640 1 -- 172.21.15.16:0/3875434965 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe9fc06b4d0 msgr2=0x7fe9fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.627+0000 7fea20ecb640 1 --2- 172.21.15.16:0/3875434965 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe9fc06b4d0 0x7fe9fc06d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fea0c000f60 tx=0x7fea0c007b50 comp rx=0 tx=0).stop 2024-04-25T10:09:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.627+0000 7fea20ecb640 1 -- 172.21.15.16:0/3875434965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1c0b3880 msgr2=0x7fea1c151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.627+0000 7fea20ecb640 1 --2- 172.21.15.16:0/3875434965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1c0b3880 0x7fea1c151d80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fea1402ac90 tx=0x7fea14004a40 comp rx=0 tx=0).stop 2024-04-25T10:09:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.627+0000 7fea20ecb640 1 -- 172.21.15.16:0/3875434965 shutdown_connections 2024-04-25T10:09:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.627+0000 7fea20ecb640 1 --2- 172.21.15.16:0/3875434965 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe9fc06b4d0 0x7fe9fc06d990 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.627+0000 7fea20ecb640 1 --2- 172.21.15.16:0/3875434965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1c0b3880 0x7fea1c151d80 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.627+0000 7fea20ecb640 1 --2- 172.21.15.16:0/3875434965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fea1c153860 0x7fea1c0b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.627+0000 7fea20ecb640 1 --2- 172.21.15.16:0/3875434965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea1c15d980 0x7fea1c14ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.627+0000 7fea20ecb640 1 -- 172.21.15.16:0/3875434965 >> 172.21.15.16:0/3875434965 conn(0x7fea1c06aba0 msgr2=0x7fea1c15dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.627+0000 7fea20ecb640 1 -- 172.21.15.16:0/3875434965 shutdown_connections 2024-04-25T10:09:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.627+0000 7fea20ecb640 1 -- 172.21.15.16:0/3875434965 wait complete. 2024-04-25T10:09:07.645 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:48.948980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:49.638457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:07.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:07.713 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:07.715 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:07.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.744+0000 7f9261212640 1 Processor -- start 2024-04-25T10:09:07.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.744+0000 7f9261212640 1 -- start start 2024-04-25T10:09:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.745+0000 7f9261212640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f925c15bdb0 0x7f925c09a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.745+0000 7f9261212640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f925c09a990 0x7f925c165890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.745+0000 7f9261212640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925c09ae30 con 0x7f925c0cf830 2024-04-25T10:09:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.745+0000 7f9261212640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f925c09b110 con 0x7f925c15bdb0 2024-04-25T10:09:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.745+0000 7f9261212640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925c0c5bd0 con 0x7f925c09a990 2024-04-25T10:09:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.745+0000 7f925b59e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f925c09a990 0x7f925c165890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.745+0000 7f925ad9d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f925c15bdb0 0x7f925c09a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.745+0000 7f925ad9d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f925c15bdb0 0x7f925c09a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:56644/0 (socket says 172.21.15.16:56644) 2024-04-25T10:09:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.745+0000 7f925a59c640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f925c0cf830 0x7f925c0cfee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:47394/0 (socket says 172.21.15.16:47394) 2024-04-25T10:09:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.745+0000 7f925ad9d640 1 -- 172.21.15.16:0/2349924064 learned_addr learned my addr 172.21.15.16:0/2349924064 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.745+0000 7f925b59e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f925c09a990 0x7f925c165890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:55652/0 (socket says 172.21.15.16:55652) 2024-04-25T10:09:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.746+0000 7f925ad9d640 1 -- 172.21.15.16:0/2349924064 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f925c09a990 msgr2=0x7f925c165890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.746+0000 7f925ad9d640 1 --2- 172.21.15.16:0/2349924064 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f925c09a990 0x7f925c165890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.746+0000 7f925ad9d640 1 -- 172.21.15.16:0/2349924064 >> v1:172.21.15.186:6789/0 conn(0x7f925c0cf830 legacy=0x7f925c0cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.746+0000 7f925ad9d640 1 -- 172.21.15.16:0/2349924064 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c0c5580 con 0x7f925c15bdb0 2024-04-25T10:09:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.746+0000 7f925b59e640 1 --2- 172.21.15.16:0/2349924064 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f925c09a990 0x7f925c165890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.747+0000 7f925ad9d640 1 --2- 172.21.15.16:0/2349924064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f925c15bdb0 0x7f925c09a450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9244009870 tx=0x7f924402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=579f45ca54c22c83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.747+0000 7f9259d9b640 1 -- 172.21.15.16:0/2349924064 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244037070 con 0x7f925c15bdb0 2024-04-25T10:09:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.747+0000 7f9259d9b640 1 -- 172.21.15.16:0/2349924064 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9244032440 con 0x7f925c15bdb0 2024-04-25T10:09:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.747+0000 7f9259d9b640 1 -- 172.21.15.16:0/2349924064 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244032810 con 0x7f925c15bdb0 2024-04-25T10:09:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.747+0000 7f9261212640 1 -- 172.21.15.16:0/2349924064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f925c15bdb0 msgr2=0x7f925c09a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.747+0000 7f9261212640 1 --2- 172.21.15.16:0/2349924064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f925c15bdb0 0x7f925c09a450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9244009870 tx=0x7f924402a250 comp rx=0 tx=0).stop 2024-04-25T10:09:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.747+0000 7f9261212640 1 -- 172.21.15.16:0/2349924064 shutdown_connections 2024-04-25T10:09:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.747+0000 7f9261212640 1 --2- 172.21.15.16:0/2349924064 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f925c09a990 0x7f925c165890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.747+0000 7f9261212640 1 --2- 172.21.15.16:0/2349924064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f925c15bdb0 0x7f925c09a450 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.747+0000 7f9261212640 1 -- 172.21.15.16:0/2349924064 >> 172.21.15.16:0/2349924064 conn(0x7f925c0b33f0 msgr2=0x7f925c0b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.747+0000 7f9261212640 1 -- 172.21.15.16:0/2349924064 shutdown_connections 2024-04-25T10:09:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.747+0000 7f9261212640 1 -- 172.21.15.16:0/2349924064 wait complete. 2024-04-25T10:09:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.749+0000 7f9261212640 1 Processor -- start 2024-04-25T10:09:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.749+0000 7f9261212640 1 -- start start 2024-04-25T10:09:07.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f9261212640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f925c09a990 0x7f925c0c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f9261212640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f925c0cf830 0x7f925c1f0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f9261212640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f925c15bdb0 0x7f925c1f1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f9261212640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c0c8070 con 0x7f925c0cf830 2024-04-25T10:09:07.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f9261212640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c15c190 con 0x7f925c15bdb0 2024-04-25T10:09:07.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f9261212640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925c0c81f0 con 0x7f925c09a990 2024-04-25T10:09:07.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f925a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f925c0cf830 0x7f925c1f0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f925a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f925c0cf830 0x7f925c1f0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41094/0 (socket says 172.21.15.16:41094) 2024-04-25T10:09:07.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f925a59c640 1 -- 172.21.15.16:0/2627112586 learned_addr learned my addr 172.21.15.16:0/2627112586 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:07.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f925ad9d640 1 --2- 172.21.15.16:0/2627112586 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f925c09a990 0x7f925c0c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f925b59e640 1 --2- 172.21.15.16:0/2627112586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f925c15bdb0 0x7f925c1f1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f925a59c640 1 -- 172.21.15.16:0/2627112586 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f925c09a990 msgr2=0x7f925c0c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f925a59c640 1 --2- 172.21.15.16:0/2627112586 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f925c09a990 0x7f925c0c6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f925a59c640 1 -- 172.21.15.16:0/2627112586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f925c15bdb0 msgr2=0x7f925c1f1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f925a59c640 1 --2- 172.21.15.16:0/2627112586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f925c15bdb0 0x7f925c1f1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f925a59c640 1 -- 172.21.15.16:0/2627112586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c1f1550 con 0x7f925c0cf830 2024-04-25T10:09:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f925a59c640 1 --2- 172.21.15.16:0/2627112586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f925c0cf830 0x7f925c1f0ad0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f924c00c3c0 tx=0x7f924c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f92537fe640 1 -- 172.21.15.16:0/2627112586 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c0074e0 con 0x7f925c0cf830 2024-04-25T10:09:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f92537fe640 1 -- 172.21.15.16:0/2627112586 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924c007680 con 0x7f925c0cf830 2024-04-25T10:09:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f9261212640 1 -- 172.21.15.16:0/2627112586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925c15c520 con 0x7f925c0cf830 2024-04-25T10:09:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.750+0000 7f92537fe640 1 -- 172.21.15.16:0/2627112586 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c004a00 con 0x7f925c0cf830 2024-04-25T10:09:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.751+0000 7f9261212640 1 -- 172.21.15.16:0/2627112586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925c15cae0 con 0x7f925c0cf830 2024-04-25T10:09:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.751+0000 7f9261212640 1 -- 172.21.15.16:0/2627112586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f925c0750b0 con 0x7f925c0cf830 2024-04-25T10:09:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.752+0000 7f92537fe640 1 -- 172.21.15.16:0/2627112586 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f924c01a020 con 0x7f925c0cf830 2024-04-25T10:09:07.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.752+0000 7f92537fe640 1 --2- 172.21.15.16:0/2627112586 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f923c06b4d0 0x7f923c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.752+0000 7f92537fe640 1 -- 172.21.15.16:0/2627112586 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f924c0897d0 con 0x7f925c0cf830 2024-04-25T10:09:07.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.752+0000 7f925ad9d640 1 --2- 172.21.15.16:0/2627112586 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f923c06b4d0 0x7f923c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.753+0000 7f925ad9d640 1 --2- 172.21.15.16:0/2627112586 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f923c06b4d0 0x7f923c06d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9244034000 tx=0x7f924402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.757+0000 7f92537fe640 1 -- 172.21.15.16:0/2627112586 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f924c058050 con 0x7f925c0cf830 2024-04-25T10:09:07.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.939+0000 7f9261212640 1 -- 172.21.15.16:0/2627112586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f925c0c4d80 con 0x7f925c0cf830 2024-04-25T10:09:07.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.940+0000 7f92537fe640 1 -- 172.21.15.16:0/2627112586 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7f924c05c0f0 con 0x7f925c0cf830 2024-04-25T10:09:07.946 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:09:07.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.942+0000 7f9261212640 1 -- 172.21.15.16:0/2627112586 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f923c06b4d0 msgr2=0x7f923c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.942+0000 7f9261212640 1 --2- 172.21.15.16:0/2627112586 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f923c06b4d0 0x7f923c06d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9244034000 tx=0x7f924402e000 comp rx=0 tx=0).stop 2024-04-25T10:09:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.942+0000 7f9261212640 1 -- 172.21.15.16:0/2627112586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f925c0cf830 msgr2=0x7f925c1f0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.942+0000 7f9261212640 1 --2- 172.21.15.16:0/2627112586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f925c0cf830 0x7f925c1f0ad0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f924c00c3c0 tx=0x7f924c00c880 comp rx=0 tx=0).stop 2024-04-25T10:09:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.942+0000 7f9261212640 1 -- 172.21.15.16:0/2627112586 shutdown_connections 2024-04-25T10:09:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.942+0000 7f9261212640 1 --2- 172.21.15.16:0/2627112586 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f923c06b4d0 0x7f923c06d990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.942+0000 7f9261212640 1 --2- 172.21.15.16:0/2627112586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f925c15bdb0 0x7f925c1f1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.942+0000 7f9261212640 1 --2- 172.21.15.16:0/2627112586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f925c0cf830 0x7f925c1f0ad0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.942+0000 7f9261212640 1 --2- 172.21.15.16:0/2627112586 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f925c09a990 0x7f925c0c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.942+0000 7f9261212640 1 -- 172.21.15.16:0/2627112586 >> 172.21.15.16:0/2627112586 conn(0x7f925c0b33f0 msgr2=0x7f925c06f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.942+0000 7f9261212640 1 -- 172.21.15.16:0/2627112586 shutdown_connections 2024-04-25T10:09:07.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:07.942+0000 7f9261212640 1 -- 172.21.15.16:0/2627112586 wait complete. 2024-04-25T10:09:07.959 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:09:08.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.056+0000 7f5ffe9d5640 1 Processor -- start 2024-04-25T10:09:08.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.056+0000 7f5ffe9d5640 1 -- start start 2024-04-25T10:09:08.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.057+0000 7f5ffe9d5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5ff815deb0 0x7f5ff80b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.057+0000 7f5ffe9d5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5ff80b1d30 0x7f5ff806aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.057+0000 7f5ffe9d5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff806b4e0 con 0x7f5ff806b0e0 2024-04-25T10:09:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.057+0000 7f5ffe9d5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff80b21d0 con 0x7f5ff815deb0 2024-04-25T10:09:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.057+0000 7f5ffe9d5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff80cd150 con 0x7f5ff80b1d30 2024-04-25T10:09:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.057+0000 7f5ff7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5ff815deb0 0x7f5ff80b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.057+0000 7f5ffcf72640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5ff806b0e0 0x7f5ff8161b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32860/0 (socket says 172.21.15.16:32860) 2024-04-25T10:09:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.057+0000 7f5ffcf72640 1 -- 172.21.15.16:0/3575373185 learned_addr learned my addr 172.21.15.16:0/3575373185 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.058+0000 7f5ff77fe640 1 --2- 172.21.15.16:0/3575373185 >> v2:172.21.15.186:3300/0 conn(0x7f5ff80b1d30 0x7f5ff806aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.058+0000 7f5ff7fff640 1 -- 172.21.15.16:0/3575373185 >> v2:172.21.15.186:3300/0 conn(0x7f5ff80b1d30 msgr2=0x7f5ff806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.058+0000 7f5ff7fff640 1 --2- 172.21.15.16:0/3575373185 >> v2:172.21.15.186:3300/0 conn(0x7f5ff80b1d30 0x7f5ff806aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.058+0000 7f5ff7fff640 1 -- 172.21.15.16:0/3575373185 >> v1:172.21.15.16:6789/0 conn(0x7f5ff806b0e0 legacy=0x7f5ff8161b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.058+0000 7f5ff7fff640 1 -- 172.21.15.16:0/3575373185 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff80cd430 con 0x7f5ff815deb0 2024-04-25T10:09:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.058+0000 7f5ff6ffd640 1 -- 172.21.15.16:0/3575373185 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3847991016 0 0) 0x7f5ff806b4e0 con 0x7f5ff806b0e0 2024-04-25T10:09:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.058+0000 7f5ff7fff640 1 --2- 172.21.15.16:0/3575373185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff815deb0 0x7f5ff80b17f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80727c0 tx=0x7f5fe802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd969dd88052cd74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.059+0000 7f5ff6ffd640 1 -- 172.21.15.16:0/3575373185 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8037070 con 0x7f5ff815deb0 2024-04-25T10:09:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.059+0000 7f5ff6ffd640 1 -- 172.21.15.16:0/3575373185 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe8032440 con 0x7f5ff815deb0 2024-04-25T10:09:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.059+0000 7f5ff6ffd640 1 -- 172.21.15.16:0/3575373185 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe802ae60 con 0x7f5ff815deb0 2024-04-25T10:09:08.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.059+0000 7f5ffe9d5640 1 -- 172.21.15.16:0/3575373185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff815deb0 msgr2=0x7f5ff80b17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:08.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.059+0000 7f5ffe9d5640 1 --2- 172.21.15.16:0/3575373185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff815deb0 0x7f5ff80b17f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80727c0 tx=0x7f5fe802a110 comp rx=0 tx=0).stop 2024-04-25T10:09:08.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.059+0000 7f5ffe9d5640 1 -- 172.21.15.16:0/3575373185 shutdown_connections 2024-04-25T10:09:08.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.059+0000 7f5ffe9d5640 1 --2- 172.21.15.16:0/3575373185 >> v2:172.21.15.186:3300/0 conn(0x7f5ff80b1d30 0x7f5ff806aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.059+0000 7f5ffe9d5640 1 --2- 172.21.15.16:0/3575373185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff815deb0 0x7f5ff80b17f0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.059+0000 7f5ffe9d5640 1 -- 172.21.15.16:0/3575373185 >> 172.21.15.16:0/3575373185 conn(0x7f5ff80b2390 msgr2=0x7f5ff80b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.059+0000 7f5ffe9d5640 1 -- 172.21.15.16:0/3575373185 shutdown_connections 2024-04-25T10:09:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.059+0000 7f5ffe9d5640 1 -- 172.21.15.16:0/3575373185 wait complete. 2024-04-25T10:09:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.060+0000 7f5ffe9d5640 1 Processor -- start 2024-04-25T10:09:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.061+0000 7f5ffe9d5640 1 -- start start 2024-04-25T10:09:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.061+0000 7f5ffe9d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ff806b0e0 0x7f5ff8155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.061+0000 7f5ffe9d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff80b1d30 0x7f5ff80c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.061+0000 7f5ffe9d5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ff815deb0 0x7f5ff8153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.061+0000 7f5ffe9d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff815e290 con 0x7f5ff80b1d30 2024-04-25T10:09:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.061+0000 7f5ffe9d5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff80cd9f0 con 0x7f5ff815deb0 2024-04-25T10:09:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.061+0000 7f5ffe9d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff815e410 con 0x7f5ff806b0e0 2024-04-25T10:09:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.061+0000 7f5ff77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff80b1d30 0x7f5ff80c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.061+0000 7f5ff77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff80b1d30 0x7f5ff80c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41106/0 (socket says 172.21.15.16:41106) 2024-04-25T10:09:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.061+0000 7f5ff77fe640 1 -- 172.21.15.16:0/2427021349 learned_addr learned my addr 172.21.15.16:0/2427021349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.062+0000 7f5ffcf72640 1 --2- 172.21.15.16:0/2427021349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ff815deb0 0x7f5ff8153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.062+0000 7f5ff7fff640 1 --2- 172.21.15.16:0/2427021349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ff806b0e0 0x7f5ff8155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.062+0000 7f5ff77fe640 1 -- 172.21.15.16:0/2427021349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ff806b0e0 msgr2=0x7f5ff8155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.062+0000 7f5ff77fe640 1 --2- 172.21.15.16:0/2427021349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ff806b0e0 0x7f5ff8155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.062+0000 7f5ff77fe640 1 -- 172.21.15.16:0/2427021349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ff815deb0 msgr2=0x7f5ff8153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.062+0000 7f5ff77fe640 1 --2- 172.21.15.16:0/2427021349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ff815deb0 0x7f5ff8153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.062+0000 7f5ff77fe640 1 -- 172.21.15.16:0/2427021349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff80c49f0 con 0x7f5ff80b1d30 2024-04-25T10:09:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.062+0000 7f5ffcf72640 1 --2- 172.21.15.16:0/2427021349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ff815deb0 0x7f5ff8153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.062+0000 7f5ff77fe640 1 --2- 172.21.15.16:0/2427021349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff80b1d30 0x7f5ff80c42a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00d7d0 tx=0x7f5fec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.062+0000 7f5ff4ff9640 1 -- 172.21.15.16:0/2427021349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec004020 con 0x7f5ff80b1d30 2024-04-25T10:09:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.062+0000 7f5ff4ff9640 1 -- 172.21.15.16:0/2427021349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec0041e0 con 0x7f5ff80b1d30 2024-04-25T10:09:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.062+0000 7f5ffe9d5640 1 -- 172.21.15.16:0/2427021349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff80c4cd0 con 0x7f5ff80b1d30 2024-04-25T10:09:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.062+0000 7f5ff4ff9640 1 -- 172.21.15.16:0/2427021349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec0043d0 con 0x7f5ff80b1d30 2024-04-25T10:09:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.062+0000 7f5ffe9d5640 1 -- 172.21.15.16:0/2427021349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff80c5290 con 0x7f5ff80b1d30 2024-04-25T10:09:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.063+0000 7f5ffe9d5640 1 -- 172.21.15.16:0/2427021349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff806af00 con 0x7f5ff80b1d30 2024-04-25T10:09:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.063+0000 7f5ff4ff9640 1 -- 172.21.15.16:0/2427021349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5fec01b020 con 0x7f5ff80b1d30 2024-04-25T10:09:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.064+0000 7f5ff4ff9640 1 --2- 172.21.15.16:0/2427021349 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5fe406b4d0 0x7f5fe406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.064+0000 7f5ff4ff9640 1 -- 172.21.15.16:0/2427021349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5fec08a0f0 con 0x7f5ff80b1d30 2024-04-25T10:09:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.064+0000 7f5ff7fff640 1 --2- 172.21.15.16:0/2427021349 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5fe406b4d0 0x7f5fe406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.065+0000 7f5ff7fff640 1 --2- 172.21.15.16:0/2427021349 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5fe406b4d0 0x7f5fe406d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8034000 tx=0x7f5fe802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:08.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.068+0000 7f5ff4ff9640 1 -- 172.21.15.16:0/2427021349 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ff806af00 con 0x7f5ff80b1d30 2024-04-25T10:09:08.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.246+0000 7f5ffe9d5640 1 -- 172.21.15.16:0/2427021349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5ff80750b0 con 0x7f5ff80b1d30 2024-04-25T10:09:08.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.247+0000 7f5ff4ff9640 1 -- 172.21.15.16:0/2427021349 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v61) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5fec058e90 con 0x7f5ff80b1d30 2024-04-25T10:09:08.253 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:09:08.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.249+0000 7f5ffe9d5640 1 -- 172.21.15.16:0/2427021349 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5fe406b4d0 msgr2=0x7f5fe406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:08.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.249+0000 7f5ffe9d5640 1 --2- 172.21.15.16:0/2427021349 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5fe406b4d0 0x7f5fe406d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8034000 tx=0x7f5fe802e000 comp rx=0 tx=0).stop 2024-04-25T10:09:08.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.249+0000 7f5ffe9d5640 1 -- 172.21.15.16:0/2427021349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff80b1d30 msgr2=0x7f5ff80c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:08.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.249+0000 7f5ffe9d5640 1 --2- 172.21.15.16:0/2427021349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff80b1d30 0x7f5ff80c42a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00d7d0 tx=0x7f5fec00dc90 comp rx=0 tx=0).stop 2024-04-25T10:09:08.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.249+0000 7f5ffe9d5640 1 -- 172.21.15.16:0/2427021349 shutdown_connections 2024-04-25T10:09:08.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.249+0000 7f5ffe9d5640 1 --2- 172.21.15.16:0/2427021349 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5fe406b4d0 0x7f5fe406d990 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.249+0000 7f5ffe9d5640 1 --2- 172.21.15.16:0/2427021349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ff815deb0 0x7f5ff8153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.249+0000 7f5ffe9d5640 1 --2- 172.21.15.16:0/2427021349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff80b1d30 0x7f5ff80c42a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.249+0000 7f5ffe9d5640 1 --2- 172.21.15.16:0/2427021349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5ff806b0e0 0x7f5ff8155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.249+0000 7f5ffe9d5640 1 -- 172.21.15.16:0/2427021349 >> 172.21.15.16:0/2427021349 conn(0x7f5ff80b2390 msgr2=0x7f5ff8160b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:08.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.249+0000 7f5ffe9d5640 1 -- 172.21.15.16:0/2427021349 shutdown_connections 2024-04-25T10:09:08.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.249+0000 7f5ffe9d5640 1 -- 172.21.15.16:0/2427021349 wait complete. 2024-04-25T10:09:08.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:09:08.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.363+0000 7fa2da86b640 1 Processor -- start 2024-04-25T10:09:08.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.363+0000 7fa2da86b640 1 -- start start 2024-04-25T10:09:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.364+0000 7fa2da86b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d415dca0 0x7fa2d40ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.364+0000 7fa2da86b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa2d40d3b90 0x7fa2d40d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.364+0000 7fa2da86b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d40d4090 con 0x7fa2d40cee80 2024-04-25T10:09:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.364+0000 7fa2da86b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d40b6b20 con 0x7fa2d40d3b90 2024-04-25T10:09:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.364+0000 7fa2da86b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d40d4370 con 0x7fa2d415dca0 2024-04-25T10:09:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.364+0000 7fa2d37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa2d40d3b90 0x7fa2d40d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.364+0000 7fa2d37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa2d40d3b90 0x7fa2d40d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41120/0 (socket says 172.21.15.16:41120) 2024-04-25T10:09:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.364+0000 7fa2d37fe640 1 -- 172.21.15.16:0/1730095984 learned_addr learned my addr 172.21.15.16:0/1730095984 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.364+0000 7fa2d3fff640 1 --2- 172.21.15.16:0/1730095984 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d415dca0 0x7fa2d40ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.364+0000 7fa2d37fe640 1 -- 172.21.15.16:0/1730095984 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d415dca0 msgr2=0x7fa2d40ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.364+0000 7fa2d37fe640 1 --2- 172.21.15.16:0/1730095984 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d415dca0 0x7fa2d40ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.364+0000 7fa2d37fe640 1 -- 172.21.15.16:0/1730095984 >> v1:172.21.15.186:6789/0 conn(0x7fa2d40cee80 legacy=0x7fa2d40b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.364+0000 7fa2d37fe640 1 -- 172.21.15.16:0/1730095984 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d40d4650 con 0x7fa2d40d3b90 2024-04-25T10:09:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.365+0000 7fa2d37fe640 1 --2- 172.21.15.16:0/1730095984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d40d3b90 0x7fa2d40d0a20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4009870 tx=0x7fa2c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cdd054671d37e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.365+0000 7fa2d2ffd640 1 -- 172.21.15.16:0/1730095984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4037070 con 0x7fa2d40d3b90 2024-04-25T10:09:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.365+0000 7fa2d2ffd640 1 -- 172.21.15.16:0/1730095984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c4032440 con 0x7fa2d40d3b90 2024-04-25T10:09:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.365+0000 7fa2d2ffd640 1 -- 172.21.15.16:0/1730095984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4032810 con 0x7fa2d40d3b90 2024-04-25T10:09:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.366+0000 7fa2da86b640 1 -- 172.21.15.16:0/1730095984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d40d3b90 msgr2=0x7fa2d40d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.366+0000 7fa2da86b640 1 --2- 172.21.15.16:0/1730095984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d40d3b90 0x7fa2d40d0a20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4009870 tx=0x7fa2c402a270 comp rx=0 tx=0).stop 2024-04-25T10:09:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.366+0000 7fa2da86b640 1 -- 172.21.15.16:0/1730095984 shutdown_connections 2024-04-25T10:09:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.366+0000 7fa2da86b640 1 --2- 172.21.15.16:0/1730095984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d40d3b90 0x7fa2d40d0a20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.366+0000 7fa2da86b640 1 --2- 172.21.15.16:0/1730095984 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d415dca0 0x7fa2d40ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.366+0000 7fa2da86b640 1 -- 172.21.15.16:0/1730095984 >> 172.21.15.16:0/1730095984 conn(0x7fa2d40726c0 msgr2=0x7fa2d4072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.366+0000 7fa2da86b640 1 -- 172.21.15.16:0/1730095984 shutdown_connections 2024-04-25T10:09:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.366+0000 7fa2da86b640 1 -- 172.21.15.16:0/1730095984 wait complete. 2024-04-25T10:09:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.367+0000 7fa2da86b640 1 Processor -- start 2024-04-25T10:09:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.367+0000 7fa2da86b640 1 -- start start 2024-04-25T10:09:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.368+0000 7fa2da86b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2d40cee80 0x7fa2d40b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.368+0000 7fa2da86b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d40d3b90 0x7fa2d41e65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.368+0000 7fa2da86b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d415dca0 0x7fa2d40ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.368+0000 7fa2da86b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d415e200 con 0x7fa2d40d3b90 2024-04-25T10:09:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.368+0000 7fa2da86b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d415e080 con 0x7fa2d40cee80 2024-04-25T10:09:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.368+0000 7fa2da86b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d40d0f60 con 0x7fa2d415dca0 2024-04-25T10:09:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.368+0000 7fa2d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d40d3b90 0x7fa2d41e65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.368+0000 7fa2d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d40d3b90 0x7fa2d41e65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41128/0 (socket says 172.21.15.16:41128) 2024-04-25T10:09:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.368+0000 7fa2d37fe640 1 -- 172.21.15.16:0/4048052036 learned_addr learned my addr 172.21.15.16:0/4048052036 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.368+0000 7fa2d3fff640 1 --2- 172.21.15.16:0/4048052036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2d40cee80 0x7fa2d40b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.368+0000 7fa2d8e08640 1 --2- 172.21.15.16:0/4048052036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d415dca0 0x7fa2d40ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.368+0000 7fa2d37fe640 1 -- 172.21.15.16:0/4048052036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d415dca0 msgr2=0x7fa2d40ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.368+0000 7fa2d37fe640 1 --2- 172.21.15.16:0/4048052036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d415dca0 0x7fa2d40ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.368+0000 7fa2d37fe640 1 -- 172.21.15.16:0/4048052036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2d40cee80 msgr2=0x7fa2d40b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.368+0000 7fa2d37fe640 1 --2- 172.21.15.16:0/4048052036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2d40cee80 0x7fa2d40b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.368+0000 7fa2d37fe640 1 -- 172.21.15.16:0/4048052036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d40b32c0 con 0x7fa2d40d3b90 2024-04-25T10:09:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.368+0000 7fa2d3fff640 1 --2- 172.21.15.16:0/4048052036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2d40cee80 0x7fa2d40b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.369+0000 7fa2d37fe640 1 --2- 172.21.15.16:0/4048052036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d40d3b90 0x7fa2d41e65a0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4000c00 tx=0x7fa2c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.369+0000 7fa2d0ff9640 1 -- 172.21.15.16:0/4048052036 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4037040 con 0x7fa2d40d3b90 2024-04-25T10:09:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.369+0000 7fa2d0ff9640 1 -- 172.21.15.16:0/4048052036 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c4042070 con 0x7fa2d40d3b90 2024-04-25T10:09:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.369+0000 7fa2da86b640 1 -- 172.21.15.16:0/4048052036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d41e7060 con 0x7fa2d40d3b90 2024-04-25T10:09:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.369+0000 7fa2d0ff9640 1 -- 172.21.15.16:0/4048052036 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4030880 con 0x7fa2d40d3b90 2024-04-25T10:09:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.369+0000 7fa2da86b640 1 -- 172.21.15.16:0/4048052036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d414ad70 con 0x7fa2d40d3b90 2024-04-25T10:09:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.370+0000 7fa2da86b640 1 -- 172.21.15.16:0/4048052036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d40cbb00 con 0x7fa2d40d3b90 2024-04-25T10:09:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.370+0000 7fa2d0ff9640 1 -- 172.21.15.16:0/4048052036 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa2c4030a20 con 0x7fa2d40d3b90 2024-04-25T10:09:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.371+0000 7fa2d0ff9640 1 --2- 172.21.15.16:0/4048052036 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa2b406b4d0 0x7fa2b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.371+0000 7fa2d0ff9640 1 -- 172.21.15.16:0/4048052036 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa2c40aaaa0 con 0x7fa2d40d3b90 2024-04-25T10:09:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.371+0000 7fa2d3fff640 1 --2- 172.21.15.16:0/4048052036 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa2b406b4d0 0x7fa2b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.372+0000 7fa2d3fff640 1 --2- 172.21.15.16:0/4048052036 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa2b406b4d0 0x7fa2b406d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009af0 tx=0x7fa2bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.375+0000 7fa2d0ff9640 1 -- 172.21.15.16:0/4048052036 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa2d40cbb00 con 0x7fa2d40d3b90 2024-04-25T10:09:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.555+0000 7fa2da86b640 1 -- 172.21.15.16:0/4048052036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa2d40750b0 con 0x7fa2d40d3b90 2024-04-25T10:09:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.555+0000 7fa2d0ff9640 1 -- 172.21.15.16:0/4048052036 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa2c40798d0 con 0x7fa2d40d3b90 2024-04-25T10:09:08.561 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:09:08.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.557+0000 7fa2da86b640 1 -- 172.21.15.16:0/4048052036 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa2b406b4d0 msgr2=0x7fa2b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.557+0000 7fa2da86b640 1 --2- 172.21.15.16:0/4048052036 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa2b406b4d0 0x7fa2b406d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009af0 tx=0x7fa2bc008040 comp rx=0 tx=0).stop 2024-04-25T10:09:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.557+0000 7fa2da86b640 1 -- 172.21.15.16:0/4048052036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d40d3b90 msgr2=0x7fa2d41e65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.557+0000 7fa2da86b640 1 --2- 172.21.15.16:0/4048052036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d40d3b90 0x7fa2d41e65a0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4000c00 tx=0x7fa2c40046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.557+0000 7fa2da86b640 1 -- 172.21.15.16:0/4048052036 shutdown_connections 2024-04-25T10:09:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.557+0000 7fa2da86b640 1 --2- 172.21.15.16:0/4048052036 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa2b406b4d0 0x7fa2b406d990 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.557+0000 7fa2da86b640 1 --2- 172.21.15.16:0/4048052036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d415dca0 0x7fa2d40ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.557+0000 7fa2da86b640 1 --2- 172.21.15.16:0/4048052036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d40d3b90 0x7fa2d41e65a0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.557+0000 7fa2da86b640 1 --2- 172.21.15.16:0/4048052036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2d40cee80 0x7fa2d40b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.557+0000 7fa2da86b640 1 -- 172.21.15.16:0/4048052036 >> 172.21.15.16:0/4048052036 conn(0x7fa2d40726c0 msgr2=0x7fa2d4160df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.557+0000 7fa2da86b640 1 -- 172.21.15.16:0/4048052036 shutdown_connections 2024-04-25T10:09:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.557+0000 7fa2da86b640 1 -- 172.21.15.16:0/4048052036 wait complete. 2024-04-25T10:09:08.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:08.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.669+0000 7fb20cba5640 1 Processor -- start 2024-04-25T10:09:08.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.669+0000 7fb20cba5640 1 -- start start 2024-04-25T10:09:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.670+0000 7fb20cba5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb20815dce0 0x7fb2080d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.670+0000 7fb20cba5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb2080d35b0 0x7fb2080b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.670+0000 7fb20cba5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb20815e0c0 con 0x7fb2080d1f90 2024-04-25T10:09:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.670+0000 7fb20cba5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2080b7b10 con 0x7fb2080d35b0 2024-04-25T10:09:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.670+0000 7fb20cba5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2080d2510 con 0x7fb20815dce0 2024-04-25T10:09:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.670+0000 7fb206d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb2080d1f90 0x7fb20806bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32866/0 (socket says 172.21.15.16:32866) 2024-04-25T10:09:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.670+0000 7fb206d9d640 1 -- 172.21.15.16:0/2814061724 learned_addr learned my addr 172.21.15.16:0/2814061724 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.670+0000 7fb205d9b640 1 --2- 172.21.15.16:0/2814061724 >> v2:172.21.15.186:3300/0 conn(0x7fb2080d35b0 0x7fb2080b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.670+0000 7fb20659c640 1 --2- 172.21.15.16:0/2814061724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb20815dce0 0x7fb2080d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.670+0000 7fb20559a640 1 -- 172.21.15.16:0/2814061724 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995689989 0 0) 0x7fb20815e0c0 con 0x7fb2080d1f90 2024-04-25T10:09:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.670+0000 7fb20559a640 1 -- 172.21.15.16:0/2814061724 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1f4003520 con 0x7fb2080d1f90 2024-04-25T10:09:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.670+0000 7fb20559a640 1 -- 172.21.15.16:0/2814061724 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2379194418 0 0) 0x7fb1f4003520 con 0x7fb2080d1f90 2024-04-25T10:09:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20559a640 1 -- 172.21.15.16:0/2814061724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb20815dce0 msgr2=0x7fb2080d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20559a640 1 --2- 172.21.15.16:0/2814061724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb20815dce0 0x7fb2080d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20559a640 1 -- 172.21.15.16:0/2814061724 >> v2:172.21.15.186:3300/0 conn(0x7fb2080d35b0 msgr2=0x7fb2080b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20559a640 1 --2- 172.21.15.16:0/2814061724 >> v2:172.21.15.186:3300/0 conn(0x7fb2080d35b0 0x7fb2080b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20559a640 1 -- 172.21.15.16:0/2814061724 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2080d27f0 con 0x7fb2080d1f90 2024-04-25T10:09:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20559a640 1 -- 172.21.15.16:0/2814061724 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fb2000019f0 con 0x7fb2080d1f90 2024-04-25T10:09:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb205d9b640 1 --2- 172.21.15.16:0/2814061724 >> v2:172.21.15.186:3300/0 conn(0x7fb2080d35b0 0x7fb2080b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20559a640 1 -- 172.21.15.16:0/2814061724 >> v1:172.21.15.16:6789/0 conn(0x7fb2080d1f90 legacy=0x7fb20806bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20559a640 1 --2- 172.21.15.16:0/2814061724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f40039b0 0x7fb1f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20559a640 1 --2- 172.21.15.16:0/2814061724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb20815dce0 0x7fb1f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20559a640 1 --2- 172.21.15.16:0/2814061724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1f4004d40 0x7fb1f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20559a640 1 -- 172.21.15.16:0/2814061724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2080b7b10 con 0x7fb1f40039b0 2024-04-25T10:09:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20559a640 1 -- 172.21.15.16:0/2814061724 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2080d2510 con 0x7fb1f4004d40 2024-04-25T10:09:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20559a640 1 -- 172.21.15.16:0/2814061724 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f400b9c0 con 0x7fb20815dce0 2024-04-25T10:09:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20659c640 1 --2- 172.21.15.16:0/2814061724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f40039b0 0x7fb1f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb206d9d640 1 --2- 172.21.15.16:0/2814061724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1f4004d40 0x7fb1f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb205d9b640 1 --2- 172.21.15.16:0/2814061724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb20815dce0 0x7fb1f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20659c640 1 -- 172.21.15.16:0/2814061724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb20815dce0 msgr2=0x7fb1f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20659c640 1 --2- 172.21.15.16:0/2814061724 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb20815dce0 0x7fb1f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20659c640 1 -- 172.21.15.16:0/2814061724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1f4004d40 msgr2=0x7fb1f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20659c640 1 --2- 172.21.15.16:0/2814061724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1f4004d40 0x7fb1f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb20659c640 1 -- 172.21.15.16:0/2814061724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f4003520 con 0x7fb1f40039b0 2024-04-25T10:09:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.671+0000 7fb206d9d640 1 --2- 172.21.15.16:0/2814061724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1f4004d40 0x7fb1f40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.672+0000 7fb20659c640 1 --2- 172.21.15.16:0/2814061724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f40039b0 0x7fb1f4003dd0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009ac0 tx=0x7fb1f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e2eccd5cc812a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.672+0000 7fb20559a640 1 -- 172.21.15.16:0/2814061724 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8036070 con 0x7fb1f40039b0 2024-04-25T10:09:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.672+0000 7fb20559a640 1 -- 172.21.15.16:0/2814061724 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f8031400 con 0x7fb1f40039b0 2024-04-25T10:09:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.672+0000 7fb20559a640 1 -- 172.21.15.16:0/2814061724 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8009690 con 0x7fb1f40039b0 2024-04-25T10:09:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.672+0000 7fb20cba5640 1 -- 172.21.15.16:0/2814061724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f40039b0 msgr2=0x7fb1f4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.672+0000 7fb20cba5640 1 --2- 172.21.15.16:0/2814061724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f40039b0 0x7fb1f4003dd0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009ac0 tx=0x7fb1f4005830 comp rx=0 tx=0).stop 2024-04-25T10:09:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.672+0000 7fb206d9d640 1 -- 172.21.15.16:0/2814061724 reap_dead start 2024-04-25T10:09:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.672+0000 7fb20cba5640 1 -- 172.21.15.16:0/2814061724 shutdown_connections 2024-04-25T10:09:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.672+0000 7fb20cba5640 1 -- 172.21.15.16:0/2814061724 >> 172.21.15.16:0/2814061724 conn(0x7fb208072900 msgr2=0x7fb2080d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.673+0000 7fb20cba5640 1 -- 172.21.15.16:0/2814061724 shutdown_connections 2024-04-25T10:09:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.673+0000 7fb20cba5640 1 -- 172.21.15.16:0/2814061724 wait complete. 2024-04-25T10:09:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.674+0000 7fb20cba5640 1 Processor -- start 2024-04-25T10:09:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.674+0000 7fb20cba5640 1 -- start start 2024-04-25T10:09:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb20cba5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f40039b0 0x7fb2081f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb20cba5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb20815f070 0x7fb20815fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb20cba5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb208160090 0x7fb2080ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb20cba5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2080d3090 con 0x7fb1f40039b0 2024-04-25T10:09:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb20cba5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2080d2db0 con 0x7fb20815f070 2024-04-25T10:09:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb20cba5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2080b9820 con 0x7fb208160090 2024-04-25T10:09:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb20659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f40039b0 0x7fb2081f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb20659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f40039b0 0x7fb2081f6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41156/0 (socket says 172.21.15.16:41156) 2024-04-25T10:09:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb20659c640 1 -- 172.21.15.16:0/752678872 learned_addr learned my addr 172.21.15.16:0/752678872 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb205d9b640 1 --2- 172.21.15.16:0/752678872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb20815f070 0x7fb20815fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb206d9d640 1 --2- 172.21.15.16:0/752678872 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb208160090 0x7fb2080ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb20659c640 1 -- 172.21.15.16:0/752678872 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb208160090 msgr2=0x7fb2080ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb20659c640 1 --2- 172.21.15.16:0/752678872 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb208160090 0x7fb2080ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb20659c640 1 -- 172.21.15.16:0/752678872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb20815f070 msgr2=0x7fb20815fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb20659c640 1 --2- 172.21.15.16:0/752678872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb20815f070 0x7fb20815fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb20659c640 1 -- 172.21.15.16:0/752678872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb208160530 con 0x7fb1f40039b0 2024-04-25T10:09:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb205d9b640 1 --2- 172.21.15.16:0/752678872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb20815f070 0x7fb20815fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb20659c640 1 --2- 172.21.15.16:0/752678872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f40039b0 0x7fb2081f6190 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009ac0 tx=0x7fb1f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb1feffd640 1 -- 172.21.15.16:0/752678872 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8040020 con 0x7fb1f40039b0 2024-04-25T10:09:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb1feffd640 1 -- 172.21.15.16:0/752678872 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f8003bc0 con 0x7fb1f40039b0 2024-04-25T10:09:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb20cba5640 1 -- 172.21.15.16:0/752678872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb208157880 con 0x7fb1f40039b0 2024-04-25T10:09:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb1feffd640 1 -- 172.21.15.16:0/752678872 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8036040 con 0x7fb1f40039b0 2024-04-25T10:09:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.675+0000 7fb20cba5640 1 -- 172.21.15.16:0/752678872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb208157e40 con 0x7fb1f40039b0 2024-04-25T10:09:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.677+0000 7fb20cba5640 1 -- 172.21.15.16:0/752678872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2080749b0 con 0x7fb1f40039b0 2024-04-25T10:09:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.677+0000 7fb1feffd640 1 -- 172.21.15.16:0/752678872 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb1f8043020 con 0x7fb1f40039b0 2024-04-25T10:09:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.677+0000 7fb1feffd640 1 --2- 172.21.15.16:0/752678872 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb1e806b4d0 0x7fb1e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.677+0000 7fb1feffd640 1 -- 172.21.15.16:0/752678872 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb1f80aa3f0 con 0x7fb1f40039b0 2024-04-25T10:09:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.678+0000 7fb205d9b640 1 --2- 172.21.15.16:0/752678872 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb1e806b4d0 0x7fb1e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.678+0000 7fb205d9b640 1 --2- 172.21.15.16:0/752678872 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb1e806b4d0 0x7fb1e806d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb20815a7c0 tx=0x7fb1f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.681+0000 7fb1feffd640 1 -- 172.21.15.16:0/752678872 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1f804a070 con 0x7fb1f40039b0 2024-04-25T10:09:08.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.896+0000 7fb20cba5640 1 -- 172.21.15.16:0/752678872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2081f66d0 con 0x7fb1f40039b0 2024-04-25T10:09:08.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.896+0000 7fb1feffd640 1 -- 172.21.15.16:0/752678872 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4945 (secure 0 0 0) 0x7fb1f80044e0 con 0x7fb1f40039b0 2024-04-25T10:09:08.903 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:08.903 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2270839865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2270839865},{"type":"v1","addr":"172.21.15.186:6836","nonce":2270839865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5647,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/925282579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":925282579},{"type":"v1","addr":"172.21.15.186:6837","nonce":925282579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2471714328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2471714328},{"type":"v1","addr":"172.21.15.16:6835","nonce":2471714328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:50.665436+0000","modified":"2024-04-25T10:08:39.328915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35141":{"gid":35141,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.16:6834/3311584521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3311584521},{"type":"v1","addr":"172.21.15.16:6834","nonce":3311584521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:09:08.903 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-04-25T10:09:08.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.898+0000 7fb20cba5640 1 -- 172.21.15.16:0/752678872 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb1e806b4d0 msgr2=0x7fb1e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:08.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.898+0000 7fb20cba5640 1 --2- 172.21.15.16:0/752678872 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb1e806b4d0 0x7fb1e806d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb20815a7c0 tx=0x7fb1f0008040 comp rx=0 tx=0).stop 2024-04-25T10:09:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.898+0000 7fb20cba5640 1 -- 172.21.15.16:0/752678872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f40039b0 msgr2=0x7fb2081f6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.898+0000 7fb20cba5640 1 --2- 172.21.15.16:0/752678872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f40039b0 0x7fb2081f6190 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009ac0 tx=0x7fb1f8004730 comp rx=0 tx=0).stop 2024-04-25T10:09:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.898+0000 7fb20cba5640 1 -- 172.21.15.16:0/752678872 shutdown_connections 2024-04-25T10:09:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.898+0000 7fb20cba5640 1 --2- 172.21.15.16:0/752678872 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb1e806b4d0 0x7fb1e806d990 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.898+0000 7fb20cba5640 1 --2- 172.21.15.16:0/752678872 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb208160090 0x7fb2080ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.898+0000 7fb20cba5640 1 --2- 172.21.15.16:0/752678872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb20815f070 0x7fb20815fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.898+0000 7fb20cba5640 1 --2- 172.21.15.16:0/752678872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f40039b0 0x7fb2081f6190 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.898+0000 7fb20cba5640 1 -- 172.21.15.16:0/752678872 >> 172.21.15.16:0/752678872 conn(0x7fb208072900 msgr2=0x7fb2080d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.898+0000 7fb20cba5640 1 -- 172.21.15.16:0/752678872 shutdown_connections 2024-04-25T10:09:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:08.898+0000 7fb20cba5640 1 -- 172.21.15.16:0/752678872 wait complete. 2024-04-25T10:09:08.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:09.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.013+0000 7feddd6b3640 1 Processor -- start 2024-04-25T10:09:09.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.013+0000 7feddd6b3640 1 -- start start 2024-04-25T10:09:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.014+0000 7feddd6b3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedd815ded0 0x7fedd80cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.014+0000 7feddd6b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedd806fab0 0x7fedd8070320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.014+0000 7feddd6b3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fedd806ff10 0x7fedd8070860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.014+0000 7feddd6b3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd8072bc0 con 0x7fedd815ded0 2024-04-25T10:09:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.014+0000 7feddd6b3640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd80ccfc0 con 0x7fedd806ff10 2024-04-25T10:09:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.014+0000 7feddd6b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd80c6540 con 0x7fedd806fab0 2024-04-25T10:09:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.014+0000 7fedd6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedd815ded0 0x7fedd80cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.014+0000 7fedd6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedd815ded0 0x7fedd80cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41158/0 (socket says 172.21.15.16:41158) 2024-04-25T10:09:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.014+0000 7fedd6ffd640 1 -- 172.21.15.16:0/636700214 learned_addr learned my addr 172.21.15.16:0/636700214 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.014+0000 7fedd77fe640 1 --2- 172.21.15.16:0/636700214 >> v2:172.21.15.186:3300/0 conn(0x7fedd806ff10 0x7fedd8070860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.015+0000 7fedd67fc640 1 --2- 172.21.15.16:0/636700214 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedd806fab0 0x7fedd8070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.015+0000 7fedd6ffd640 1 -- 172.21.15.16:0/636700214 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedd806fab0 msgr2=0x7fedd8070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.015+0000 7fedd6ffd640 1 --2- 172.21.15.16:0/636700214 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedd806fab0 0x7fedd8070320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.015+0000 7fedd6ffd640 1 -- 172.21.15.16:0/636700214 >> v2:172.21.15.186:3300/0 conn(0x7fedd806ff10 msgr2=0x7fedd8070860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.015+0000 7fedd6ffd640 1 --2- 172.21.15.16:0/636700214 >> v2:172.21.15.186:3300/0 conn(0x7fedd806ff10 0x7fedd8070860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.015+0000 7fedd6ffd640 1 -- 172.21.15.16:0/636700214 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd8072e10 con 0x7fedd815ded0 2024-04-25T10:09:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.015+0000 7fedd67fc640 1 --2- 172.21.15.16:0/636700214 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedd806fab0 0x7fedd8070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.015+0000 7fedd6ffd640 1 --2- 172.21.15.16:0/636700214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd815ded0 0x7fedd80cda10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fedd80c5e80 tx=0x7fedc802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c681457f07f16fd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.016+0000 7fedd5ffb640 1 -- 172.21.15.16:0/636700214 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8037070 con 0x7fedd815ded0 2024-04-25T10:09:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.016+0000 7fedd5ffb640 1 -- 172.21.15.16:0/636700214 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc8032440 con 0x7fedd815ded0 2024-04-25T10:09:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.016+0000 7fedd5ffb640 1 -- 172.21.15.16:0/636700214 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8032810 con 0x7fedd815ded0 2024-04-25T10:09:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.016+0000 7feddd6b3640 1 -- 172.21.15.16:0/636700214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd815ded0 msgr2=0x7fedd80cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.016+0000 7feddd6b3640 1 --2- 172.21.15.16:0/636700214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd815ded0 0x7fedd80cda10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fedd80c5e80 tx=0x7fedc802a270 comp rx=0 tx=0).stop 2024-04-25T10:09:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.016+0000 7feddd6b3640 1 -- 172.21.15.16:0/636700214 shutdown_connections 2024-04-25T10:09:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.016+0000 7feddd6b3640 1 --2- 172.21.15.16:0/636700214 >> v2:172.21.15.186:3300/0 conn(0x7fedd806ff10 0x7fedd8070860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.016+0000 7feddd6b3640 1 --2- 172.21.15.16:0/636700214 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedd806fab0 0x7fedd8070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.016+0000 7feddd6b3640 1 --2- 172.21.15.16:0/636700214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd815ded0 0x7fedd80cda10 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.016+0000 7feddd6b3640 1 -- 172.21.15.16:0/636700214 >> 172.21.15.16:0/636700214 conn(0x7fedd80b2370 msgr2=0x7fedd80b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.016+0000 7feddd6b3640 1 -- 172.21.15.16:0/636700214 shutdown_connections 2024-04-25T10:09:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.016+0000 7feddd6b3640 1 -- 172.21.15.16:0/636700214 wait complete. 2024-04-25T10:09:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.018+0000 7feddd6b3640 1 Processor -- start 2024-04-25T10:09:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.018+0000 7feddd6b3640 1 -- start start 2024-04-25T10:09:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.018+0000 7feddd6b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd806fab0 0x7fedd80b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.018+0000 7feddd6b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedd815ded0 0x7fedd815f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.018+0000 7feddd6b3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedd815fcc0 0x7fedd814fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.018+0000 7feddd6b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd80cd4f0 con 0x7fedd806fab0 2024-04-25T10:09:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.018+0000 7feddd6b3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd80cd210 con 0x7fedd815fcc0 2024-04-25T10:09:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.018+0000 7feddd6b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd80cdf50 con 0x7fedd815ded0 2024-04-25T10:09:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.019+0000 7fedd6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd806fab0 0x7fedd80b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.019+0000 7fedd6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd806fab0 0x7fedd80b5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41166/0 (socket says 172.21.15.16:41166) 2024-04-25T10:09:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.019+0000 7fedd6ffd640 1 -- 172.21.15.16:0/2210744358 learned_addr learned my addr 172.21.15.16:0/2210744358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.019+0000 7fedd67fc640 1 --2- 172.21.15.16:0/2210744358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedd815ded0 0x7fedd815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.019+0000 7fedd77fe640 1 --2- 172.21.15.16:0/2210744358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedd815fcc0 0x7fedd814fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.019+0000 7fedd6ffd640 1 -- 172.21.15.16:0/2210744358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedd815ded0 msgr2=0x7fedd815f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.019+0000 7fedd6ffd640 1 --2- 172.21.15.16:0/2210744358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedd815ded0 0x7fedd815f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.019+0000 7fedd6ffd640 1 -- 172.21.15.16:0/2210744358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedd815fcc0 msgr2=0x7fedd814fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.019+0000 7fedd6ffd640 1 --2- 172.21.15.16:0/2210744358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedd815fcc0 0x7fedd814fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.019+0000 7fedd6ffd640 1 -- 172.21.15.16:0/2210744358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd80ca600 con 0x7fedd806fab0 2024-04-25T10:09:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.019+0000 7fedd6ffd640 1 --2- 172.21.15.16:0/2210744358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd806fab0 0x7fedd80b5790 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fedc802a2b0 tx=0x7fedc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.019+0000 7fedcf7fe640 1 -- 172.21.15.16:0/2210744358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8041030 con 0x7fedd806fab0 2024-04-25T10:09:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.019+0000 7fedcf7fe640 1 -- 172.21.15.16:0/2210744358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc8030440 con 0x7fedd806fab0 2024-04-25T10:09:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.019+0000 7feddd6b3640 1 -- 172.21.15.16:0/2210744358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd81603b0 con 0x7fedd806fab0 2024-04-25T10:09:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.019+0000 7fedcf7fe640 1 -- 172.21.15.16:0/2210744358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8037040 con 0x7fedd806fab0 2024-04-25T10:09:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.019+0000 7feddd6b3640 1 -- 172.21.15.16:0/2210744358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd8160970 con 0x7fedd806fab0 2024-04-25T10:09:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.020+0000 7feddd6b3640 1 -- 172.21.15.16:0/2210744358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd80749b0 con 0x7fedd806fab0 2024-04-25T10:09:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.021+0000 7fedcf7fe640 1 -- 172.21.15.16:0/2210744358 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fedc8044070 con 0x7fedd806fab0 2024-04-25T10:09:09.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.021+0000 7fedcf7fe640 1 --2- 172.21.15.16:0/2210744358 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fedb806b4d0 0x7fedb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.021+0000 7fedcf7fe640 1 -- 172.21.15.16:0/2210744358 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fedc80aa790 con 0x7fedd806fab0 2024-04-25T10:09:09.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.021+0000 7fedd67fc640 1 --2- 172.21.15.16:0/2210744358 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fedb806b4d0 0x7fedb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.022+0000 7fedd67fc640 1 --2- 172.21.15.16:0/2210744358 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fedb806b4d0 0x7fedb806d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fedd80c9b80 tx=0x7fedc0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:09.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.025+0000 7fedcf7fe640 1 -- 172.21.15.16:0/2210744358 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fedc80790a0 con 0x7fedd806fab0 2024-04-25T10:09:09.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.206+0000 7feddd6b3640 1 -- 172.21.15.16:0/2210744358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fedd80b2a30 con 0x7fedd806fab0 2024-04-25T10:09:09.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.207+0000 7fedcf7fe640 1 -- 172.21.15.16:0/2210744358 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16761 (secure 0 0 0) 0x7fedc807d140 con 0x7fedd806fab0 2024-04-25T10:09:09.213 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:09.213 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":61,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:08:52.605846+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:48.948980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:49.638457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.19","mappings":[{"from":6,"to":1}]},{"pgid":"9.28","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.209+0000 7feddd6b3640 1 -- 172.21.15.16:0/2210744358 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fedb806b4d0 msgr2=0x7fedb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.209+0000 7feddd6b3640 1 --2- 172.21.15.16:0/2210744358 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fedb806b4d0 0x7fedb806d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fedd80c9b80 tx=0x7fedc0008040 comp rx=0 tx=0).stop 2024-04-25T10:09:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.209+0000 7feddd6b3640 1 -- 172.21.15.16:0/2210744358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd806fab0 msgr2=0x7fedd80b5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.209+0000 7feddd6b3640 1 --2- 172.21.15.16:0/2210744358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd806fab0 0x7fedd80b5790 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fedc802a2b0 tx=0x7fedc80046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.209+0000 7feddd6b3640 1 -- 172.21.15.16:0/2210744358 shutdown_connections 2024-04-25T10:09:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.209+0000 7feddd6b3640 1 --2- 172.21.15.16:0/2210744358 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fedb806b4d0 0x7fedb806d990 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.209+0000 7feddd6b3640 1 --2- 172.21.15.16:0/2210744358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedd815fcc0 0x7fedd814fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.209+0000 7feddd6b3640 1 --2- 172.21.15.16:0/2210744358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedd815ded0 0x7fedd815f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.209+0000 7feddd6b3640 1 --2- 172.21.15.16:0/2210744358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd806fab0 0x7fedd80b5790 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.209+0000 7feddd6b3640 1 -- 172.21.15.16:0/2210744358 >> 172.21.15.16:0/2210744358 conn(0x7fedd80b2370 msgr2=0x7fedd815b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.209+0000 7feddd6b3640 1 -- 172.21.15.16:0/2210744358 shutdown_connections 2024-04-25T10:09:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.209+0000 7feddd6b3640 1 -- 172.21.15.16:0/2210744358 wait complete. 2024-04-25T10:09:09.226 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:09:09.226 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:09.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.321+0000 7ff8276b3640 1 Processor -- start 2024-04-25T10:09:09.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.321+0000 7ff8276b3640 1 -- start start 2024-04-25T10:09:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.322+0000 7ff8276b3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff82015dce0 0x7ff8200d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.322+0000 7ff8276b3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff8200d35b0 0x7ff8200b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.322+0000 7ff8276b3640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff82015e0c0 con 0x7ff8200d1f90 2024-04-25T10:09:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.322+0000 7ff8276b3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8200b7b10 con 0x7ff8200d35b0 2024-04-25T10:09:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.322+0000 7ff8276b3640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8200d2510 con 0x7ff82015dce0 2024-04-25T10:09:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.322+0000 7ff824c4e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff8200d35b0 0x7ff8200b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.322+0000 7ff824c4e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff8200d35b0 0x7ff8200b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41176/0 (socket says 172.21.15.16:41176) 2024-04-25T10:09:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.322+0000 7ff824c4e640 1 -- 172.21.15.16:0/3069978842 learned_addr learned my addr 172.21.15.16:0/3069978842 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.322+0000 7ff82544f640 1 --2- 172.21.15.16:0/3069978842 >> v2:172.21.15.186:3300/0 conn(0x7ff82015dce0 0x7ff8200d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.322+0000 7ff824c4e640 1 -- 172.21.15.16:0/3069978842 >> v2:172.21.15.186:3300/0 conn(0x7ff82015dce0 msgr2=0x7ff8200d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.322+0000 7ff824c4e640 1 --2- 172.21.15.16:0/3069978842 >> v2:172.21.15.186:3300/0 conn(0x7ff82015dce0 0x7ff8200d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.322+0000 7ff824c4e640 1 -- 172.21.15.16:0/3069978842 >> v1:172.21.15.186:6789/0 conn(0x7ff8200d1f90 legacy=0x7ff82006bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.322+0000 7ff824c4e640 1 -- 172.21.15.16:0/3069978842 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8200d27f0 con 0x7ff8200d35b0 2024-04-25T10:09:09.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.323+0000 7ff824c4e640 1 --2- 172.21.15.16:0/3069978842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8200d35b0 0x7ff8200b5450 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff8200ce220 tx=0x7ff81002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e122e13beff16e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:09.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.323+0000 7ff817fff640 1 -- 172.21.15.16:0/3069978842 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810037070 con 0x7ff8200d35b0 2024-04-25T10:09:09.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.323+0000 7ff817fff640 1 -- 172.21.15.16:0/3069978842 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff810032440 con 0x7ff8200d35b0 2024-04-25T10:09:09.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.323+0000 7ff817fff640 1 -- 172.21.15.16:0/3069978842 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810032810 con 0x7ff8200d35b0 2024-04-25T10:09:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.324+0000 7ff8276b3640 1 -- 172.21.15.16:0/3069978842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8200d35b0 msgr2=0x7ff8200b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.324+0000 7ff8276b3640 1 --2- 172.21.15.16:0/3069978842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8200d35b0 0x7ff8200b5450 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff8200ce220 tx=0x7ff81002a230 comp rx=0 tx=0).stop 2024-04-25T10:09:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.324+0000 7ff8276b3640 1 -- 172.21.15.16:0/3069978842 shutdown_connections 2024-04-25T10:09:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.324+0000 7ff8276b3640 1 --2- 172.21.15.16:0/3069978842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8200d35b0 0x7ff8200b5450 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.324+0000 7ff8276b3640 1 --2- 172.21.15.16:0/3069978842 >> v2:172.21.15.186:3300/0 conn(0x7ff82015dce0 0x7ff8200d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.324+0000 7ff8276b3640 1 -- 172.21.15.16:0/3069978842 >> 172.21.15.16:0/3069978842 conn(0x7ff820072900 msgr2=0x7ff8200d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.324+0000 7ff8276b3640 1 -- 172.21.15.16:0/3069978842 shutdown_connections 2024-04-25T10:09:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.324+0000 7ff8276b3640 1 -- 172.21.15.16:0/3069978842 wait complete. 2024-04-25T10:09:09.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.325+0000 7ff8276b3640 1 Processor -- start 2024-04-25T10:09:09.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.325+0000 7ff8276b3640 1 -- start start 2024-04-25T10:09:09.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.326+0000 7ff8276b3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8200d1f90 0x7ff82015ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.326+0000 7ff8276b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8200d35b0 0x7ff8200cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.326+0000 7ff8276b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff82015dce0 0x7ff820152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.326+0000 7ff8276b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200d3090 con 0x7ff8200d35b0 2024-04-25T10:09:09.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.326+0000 7ff8276b3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200d2db0 con 0x7ff8200d1f90 2024-04-25T10:09:09.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.326+0000 7ff8276b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200b9820 con 0x7ff82015dce0 2024-04-25T10:09:09.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.326+0000 7ff824c4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8200d35b0 0x7ff8200cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.326+0000 7ff824c4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8200d35b0 0x7ff8200cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41192/0 (socket says 172.21.15.16:41192) 2024-04-25T10:09:09.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.326+0000 7ff824c4e640 1 -- 172.21.15.16:0/1515502256 learned_addr learned my addr 172.21.15.16:0/1515502256 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:09.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.326+0000 7ff824c4e640 1 -- 172.21.15.16:0/1515502256 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff82015dce0 msgr2=0x7ff820152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.326+0000 7ff824c4e640 1 --2- 172.21.15.16:0/1515502256 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff82015dce0 0x7ff820152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.326+0000 7ff824c4e640 1 -- 172.21.15.16:0/1515502256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8200d1f90 msgr2=0x7ff82015ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.326+0000 7ff824c4e640 1 --2- 172.21.15.16:0/1515502256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8200d1f90 0x7ff82015ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.326+0000 7ff824c4e640 1 -- 172.21.15.16:0/1515502256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8201609e0 con 0x7ff8200d35b0 2024-04-25T10:09:09.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.326+0000 7ff824c4e640 1 --2- 172.21.15.16:0/1515502256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8200d35b0 0x7ff8200cc300 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff810000c00 tx=0x7ff8100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:09.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.326+0000 7ff815ffb640 1 -- 172.21.15.16:0/1515502256 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810041020 con 0x7ff8200d35b0 2024-04-25T10:09:09.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.327+0000 7ff815ffb640 1 -- 172.21.15.16:0/1515502256 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8100306c0 con 0x7ff8200d35b0 2024-04-25T10:09:09.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.327+0000 7ff8276b3640 1 -- 172.21.15.16:0/1515502256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff820160cc0 con 0x7ff8200d35b0 2024-04-25T10:09:09.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.327+0000 7ff815ffb640 1 -- 172.21.15.16:0/1515502256 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810037040 con 0x7ff8200d35b0 2024-04-25T10:09:09.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.327+0000 7ff8276b3640 1 -- 172.21.15.16:0/1515502256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82015ef70 con 0x7ff8200d35b0 2024-04-25T10:09:09.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.328+0000 7ff8276b3640 1 -- 172.21.15.16:0/1515502256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8200749b0 con 0x7ff8200d35b0 2024-04-25T10:09:09.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.328+0000 7ff815ffb640 1 -- 172.21.15.16:0/1515502256 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff810044020 con 0x7ff8200d35b0 2024-04-25T10:09:09.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.329+0000 7ff815ffb640 1 --2- 172.21.15.16:0/1515502256 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff80406b4d0 0x7ff80406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.329+0000 7ff815ffb640 1 -- 172.21.15.16:0/1515502256 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff8100aab10 con 0x7ff8200d35b0 2024-04-25T10:09:09.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.329+0000 7ff82544f640 1 --2- 172.21.15.16:0/1515502256 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff80406b4d0 0x7ff80406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.330+0000 7ff82544f640 1 --2- 172.21.15.16:0/1515502256 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff80406b4d0 0x7ff80406d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff81c006fd0 tx=0x7ff81c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:09.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.333+0000 7ff815ffb640 1 -- 172.21.15.16:0/1515502256 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff81004b030 con 0x7ff8200d35b0 2024-04-25T10:09:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.553+0000 7ff8276b3640 1 -- 172.21.15.16:0/1515502256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff82006b080 con 0x7ff8200d35b0 2024-04-25T10:09:09.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.554+0000 7ff815ffb640 1 -- 172.21.15.16:0/1515502256 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4945 (secure 0 0 0) 0x7ff8100798b0 con 0x7ff8200d35b0 2024-04-25T10:09:09.560 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:09.560 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2270839865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2270839865},{"type":"v1","addr":"172.21.15.186:6836","nonce":2270839865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5647,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/925282579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":925282579},{"type":"v1","addr":"172.21.15.186:6837","nonce":925282579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2471714328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2471714328},{"type":"v1","addr":"172.21.15.16:6835","nonce":2471714328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:50.665436+0000","modified":"2024-04-25T10:08:39.328915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35141":{"gid":35141,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.16:6834/3311584521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3311584521},{"type":"v1","addr":"172.21.15.16:6834","nonce":3311584521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:09:09.560 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-04-25T10:09:09.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.556+0000 7ff8276b3640 1 -- 172.21.15.16:0/1515502256 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff80406b4d0 msgr2=0x7ff80406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.556+0000 7ff8276b3640 1 --2- 172.21.15.16:0/1515502256 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff80406b4d0 0x7ff80406d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff81c006fd0 tx=0x7ff81c008040 comp rx=0 tx=0).stop 2024-04-25T10:09:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.556+0000 7ff8276b3640 1 -- 172.21.15.16:0/1515502256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8200d35b0 msgr2=0x7ff8200cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.556+0000 7ff8276b3640 1 --2- 172.21.15.16:0/1515502256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8200d35b0 0x7ff8200cc300 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff810000c00 tx=0x7ff8100046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.556+0000 7ff8276b3640 1 -- 172.21.15.16:0/1515502256 shutdown_connections 2024-04-25T10:09:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.556+0000 7ff8276b3640 1 --2- 172.21.15.16:0/1515502256 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff80406b4d0 0x7ff80406d990 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.556+0000 7ff8276b3640 1 --2- 172.21.15.16:0/1515502256 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff82015dce0 0x7ff820152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.556+0000 7ff8276b3640 1 --2- 172.21.15.16:0/1515502256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8200d35b0 0x7ff8200cc300 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.556+0000 7ff8276b3640 1 --2- 172.21.15.16:0/1515502256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8200d1f90 0x7ff82015ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.556+0000 7ff8276b3640 1 -- 172.21.15.16:0/1515502256 >> 172.21.15.16:0/1515502256 conn(0x7ff820072900 msgr2=0x7ff8200d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.556+0000 7ff8276b3640 1 -- 172.21.15.16:0/1515502256 shutdown_connections 2024-04-25T10:09:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.556+0000 7ff8276b3640 1 -- 172.21.15.16:0/1515502256 wait complete. 2024-04-25T10:09:09.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:09.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.668+0000 7f4830fa5640 1 Processor -- start 2024-04-25T10:09:09.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.668+0000 7f4830fa5640 1 -- start start 2024-04-25T10:09:09.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.669+0000 7f4830fa5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0cea00 0x7f482c0cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.669+0000 7f4830fa5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482c0d0430 con 0x7f482c0cfbe0 2024-04-25T10:09:09.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.669+0000 7f4830fa5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482c0d0710 con 0x7f482c15f120 2024-04-25T10:09:09.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.669+0000 7f4830fa5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0d09f0 con 0x7f482c0cea00 2024-04-25T10:09:09.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.669+0000 7f482ad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f482c0cfbe0 0x7f482c0ccfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32872/0 (socket says 172.21.15.16:32872) 2024-04-25T10:09:09.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.669+0000 7f482ad9d640 1 -- 172.21.15.16:0/3029316889 learned_addr learned my addr 172.21.15.16:0/3029316889 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f4829d9b640 1 --2- 172.21.15.16:0/3029316889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0cea00 0x7f482c0cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f482959a640 1 -- 172.21.15.16:0/3029316889 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1650355367 0 0) 0x7f482c0d0430 con 0x7f482c0cfbe0 2024-04-25T10:09:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f482959a640 1 -- 172.21.15.16:0/3029316889 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4818003520 con 0x7f482c0cfbe0 2024-04-25T10:09:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f482959a640 1 -- 172.21.15.16:0/3029316889 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1393017560 0 0) 0x7f4818003520 con 0x7f482c0cfbe0 2024-04-25T10:09:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f482959a640 1 -- 172.21.15.16:0/3029316889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0cea00 msgr2=0x7f482c0cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f482959a640 1 --2- 172.21.15.16:0/3029316889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0cea00 0x7f482c0cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f482959a640 1 -- 172.21.15.16:0/3029316889 >> v1:172.21.15.186:6789/0 conn(0x7f482c15f120 legacy=0x7f482c0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f482959a640 1 -- 172.21.15.16:0/3029316889 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c0cc7d0 con 0x7f482c0cfbe0 2024-04-25T10:09:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f482959a640 1 -- 172.21.15.16:0/3029316889 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f481c0019f0 con 0x7f482c0cfbe0 2024-04-25T10:09:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f482959a640 1 -- 172.21.15.16:0/3029316889 >> v1:172.21.15.16:6789/0 conn(0x7f482c0cfbe0 legacy=0x7f482c0ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f482959a640 1 --2- 172.21.15.16:0/3029316889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0cea00 0x7f48180039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f482959a640 1 --2- 172.21.15.16:0/3029316889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4818003f60 0x7f4818004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f482959a640 1 --2- 172.21.15.16:0/3029316889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4818004d40 0x7f48180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f482959a640 1 -- 172.21.15.16:0/3029316889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0d09f0 con 0x7f4818003f60 2024-04-25T10:09:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f482959a640 1 -- 172.21.15.16:0/3029316889 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481800bb30 con 0x7f4818004d40 2024-04-25T10:09:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f482959a640 1 -- 172.21.15.16:0/3029316889 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481800bd30 con 0x7f482c0cea00 2024-04-25T10:09:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f4829d9b640 1 --2- 172.21.15.16:0/3029316889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4818003f60 0x7f4818004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f482a59c640 1 --2- 172.21.15.16:0/3029316889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0cea00 0x7f48180039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f482ad9d640 1 --2- 172.21.15.16:0/3029316889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4818004d40 0x7f48180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f4829d9b640 1 -- 172.21.15.16:0/3029316889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0cea00 msgr2=0x7f48180039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f4829d9b640 1 --2- 172.21.15.16:0/3029316889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0cea00 0x7f48180039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f4829d9b640 1 -- 172.21.15.16:0/3029316889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4818004d40 msgr2=0x7f48180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f4829d9b640 1 --2- 172.21.15.16:0/3029316889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4818004d40 0x7f48180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f4829d9b640 1 -- 172.21.15.16:0/3029316889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4818003520 con 0x7f4818003f60 2024-04-25T10:09:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.670+0000 7f482ad9d640 1 --2- 172.21.15.16:0/3029316889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4818004d40 0x7f48180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.671+0000 7f4829d9b640 1 --2- 172.21.15.16:0/3029316889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4818003f60 0x7f4818004800 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4814009ac0 tx=0x7f4818005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4535751e5c94bed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.671+0000 7f482959a640 1 -- 172.21.15.16:0/3029316889 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4814036070 con 0x7f4818003f60 2024-04-25T10:09:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.671+0000 7f482959a640 1 -- 172.21.15.16:0/3029316889 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4814031400 con 0x7f4818003f60 2024-04-25T10:09:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.671+0000 7f482959a640 1 -- 172.21.15.16:0/3029316889 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4814009690 con 0x7f4818003f60 2024-04-25T10:09:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.672+0000 7f4830fa5640 1 -- 172.21.15.16:0/3029316889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4818003f60 msgr2=0x7f4818004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.672+0000 7f4830fa5640 1 --2- 172.21.15.16:0/3029316889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4818003f60 0x7f4818004800 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4814009ac0 tx=0x7f4818005830 comp rx=0 tx=0).stop 2024-04-25T10:09:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.672+0000 7f482ad9d640 1 -- 172.21.15.16:0/3029316889 reap_dead start 2024-04-25T10:09:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.672+0000 7f4830fa5640 1 -- 172.21.15.16:0/3029316889 shutdown_connections 2024-04-25T10:09:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.672+0000 7f4830fa5640 1 -- 172.21.15.16:0/3029316889 >> 172.21.15.16:0/3029316889 conn(0x7f482c0d1900 msgr2=0x7f482c0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.672+0000 7f4830fa5640 1 -- 172.21.15.16:0/3029316889 shutdown_connections 2024-04-25T10:09:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.672+0000 7f4830fa5640 1 -- 172.21.15.16:0/3029316889 wait complete. 2024-04-25T10:09:09.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.673+0000 7f4830fa5640 1 Processor -- start 2024-04-25T10:09:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.673+0000 7f4830fa5640 1 -- start start 2024-04-25T10:09:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.674+0000 7f4830fa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4818003f60 0x7f482c157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.674+0000 7f4830fa5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0cb1a0 0x7f482c0b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.674+0000 7f4830fa5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f482c0cb620 0x7f482c0b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.674+0000 7f4830fa5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0c5450 con 0x7f4818003f60 2024-04-25T10:09:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.674+0000 7f4830fa5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0cc480 con 0x7f482c0cb620 2024-04-25T10:09:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.674+0000 7f4830fa5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0d0090 con 0x7f482c0cb1a0 2024-04-25T10:09:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.674+0000 7f482a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4818003f60 0x7f482c157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.674+0000 7f482a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4818003f60 0x7f482c157010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41200/0 (socket says 172.21.15.16:41200) 2024-04-25T10:09:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.674+0000 7f482a59c640 1 -- 172.21.15.16:0/832466649 learned_addr learned my addr 172.21.15.16:0/832466649 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.674+0000 7f482ad9d640 1 --2- 172.21.15.16:0/832466649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f482c0cb620 0x7f482c0b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.674+0000 7f4829d9b640 1 --2- 172.21.15.16:0/832466649 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0cb1a0 0x7f482c0b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.675+0000 7f482a59c640 1 -- 172.21.15.16:0/832466649 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0cb1a0 msgr2=0x7f482c0b66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.675+0000 7f482a59c640 1 --2- 172.21.15.16:0/832466649 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0cb1a0 0x7f482c0b66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.675+0000 7f482a59c640 1 -- 172.21.15.16:0/832466649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f482c0cb620 msgr2=0x7f482c0b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.675+0000 7f482a59c640 1 --2- 172.21.15.16:0/832466649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f482c0cb620 0x7f482c0b6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.675+0000 7f482a59c640 1 -- 172.21.15.16:0/832466649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c0b4590 con 0x7f4818003f60 2024-04-25T10:09:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.675+0000 7f4829d9b640 1 --2- 172.21.15.16:0/832466649 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0cb1a0 0x7f482c0b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.675+0000 7f482a59c640 1 --2- 172.21.15.16:0/832466649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4818003f60 0x7f482c157010 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f482400f850 tx=0x7f482400fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.675+0000 7f4822ffd640 1 -- 172.21.15.16:0/832466649 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482400fe40 con 0x7f4818003f60 2024-04-25T10:09:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.675+0000 7f4822ffd640 1 -- 172.21.15.16:0/832466649 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4824018070 con 0x7f4818003f60 2024-04-25T10:09:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.675+0000 7f4830fa5640 1 -- 172.21.15.16:0/832466649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482c0b0320 con 0x7f4818003f60 2024-04-25T10:09:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.675+0000 7f4822ffd640 1 -- 172.21.15.16:0/832466649 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482400d730 con 0x7f4818003f60 2024-04-25T10:09:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.675+0000 7f4830fa5640 1 -- 172.21.15.16:0/832466649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f482c0b4800 con 0x7f4818003f60 2024-04-25T10:09:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.676+0000 7f4830fa5640 1 -- 172.21.15.16:0/832466649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482c067850 con 0x7f4818003f60 2024-04-25T10:09:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.676+0000 7f4822ffd640 1 -- 172.21.15.16:0/832466649 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f482400d8d0 con 0x7f4818003f60 2024-04-25T10:09:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.677+0000 7f4822ffd640 1 --2- 172.21.15.16:0/832466649 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f480c06b4d0 0x7f480c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.677+0000 7f4822ffd640 1 -- 172.21.15.16:0/832466649 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f482408a070 con 0x7f4818003f60 2024-04-25T10:09:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.677+0000 7f4829d9b640 1 --2- 172.21.15.16:0/832466649 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f480c06b4d0 0x7f480c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.678+0000 7f4829d9b640 1 --2- 172.21.15.16:0/832466649 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f480c06b4d0 0x7f480c06d990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4814033000 tx=0x7f4814008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:09.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.681+0000 7f4822ffd640 1 -- 172.21.15.16:0/832466649 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4824058f40 con 0x7f4818003f60 2024-04-25T10:09:09.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.898+0000 7f4830fa5640 1 -- 172.21.15.16:0/832466649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f482c0749b0 con 0x7f4818003f60 2024-04-25T10:09:09.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.898+0000 7f4822ffd640 1 -- 172.21.15.16:0/832466649 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f4824058840 con 0x7f4818003f60 2024-04-25T10:09:09.905 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:09.905 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2270839865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2270839865},{"type":"v1","addr":"172.21.15.186:6836","nonce":2270839865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5647,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/925282579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":925282579},{"type":"v1","addr":"172.21.15.186:6837","nonce":925282579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2471714328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2471714328},{"type":"v1","addr":"172.21.15.16:6835","nonce":2471714328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:50.665436+0000","modified":"2024-04-25T10:08:39.328915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35141":{"gid":35141,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.16:6834/3311584521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3311584521},{"type":"v1","addr":"172.21.15.16:6834","nonce":3311584521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:09:09.905 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-04-25T10:09:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.900+0000 7f4830fa5640 1 -- 172.21.15.16:0/832466649 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f480c06b4d0 msgr2=0x7f480c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.900+0000 7f4830fa5640 1 --2- 172.21.15.16:0/832466649 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f480c06b4d0 0x7f480c06d990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4814033000 tx=0x7f4814008030 comp rx=0 tx=0).stop 2024-04-25T10:09:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.900+0000 7f4830fa5640 1 -- 172.21.15.16:0/832466649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4818003f60 msgr2=0x7f482c157010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.900+0000 7f4830fa5640 1 --2- 172.21.15.16:0/832466649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4818003f60 0x7f482c157010 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f482400f850 tx=0x7f482400fd10 comp rx=0 tx=0).stop 2024-04-25T10:09:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.900+0000 7f4830fa5640 1 -- 172.21.15.16:0/832466649 shutdown_connections 2024-04-25T10:09:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.900+0000 7f4830fa5640 1 --2- 172.21.15.16:0/832466649 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f480c06b4d0 0x7f480c06d990 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.900+0000 7f4830fa5640 1 --2- 172.21.15.16:0/832466649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f482c0cb620 0x7f482c0b6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.900+0000 7f4830fa5640 1 --2- 172.21.15.16:0/832466649 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482c0cb1a0 0x7f482c0b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.900+0000 7f4830fa5640 1 --2- 172.21.15.16:0/832466649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4818003f60 0x7f482c157010 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.900+0000 7f4830fa5640 1 -- 172.21.15.16:0/832466649 >> 172.21.15.16:0/832466649 conn(0x7f482c0d1900 msgr2=0x7f482c0ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.900+0000 7f4830fa5640 1 -- 172.21.15.16:0/832466649 shutdown_connections 2024-04-25T10:09:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:09.900+0000 7f4830fa5640 1 -- 172.21.15.16:0/832466649 wait complete. 2024-04-25T10:09:09.917 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.012+0000 7fb7a0b48640 1 Processor -- start 2024-04-25T10:09:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.012+0000 7fb7a0b48640 1 -- start start 2024-04-25T10:09:10.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.013+0000 7fb7a0b48640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb79c15dce0 0x7fb79c0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.013+0000 7fb7a0b48640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb79c0b5b60 0x7fb79c0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.013+0000 7fb7a0b48640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb79c15e0c0 con 0x7fb79c0d35b0 2024-04-25T10:09:10.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.013+0000 7fb7a0b48640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb79c0d2510 con 0x7fb79c15dce0 2024-04-25T10:09:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.013+0000 7fb7a0b48640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c0d27f0 con 0x7fb79c0b5b60 2024-04-25T10:09:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.013+0000 7fb79ad9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb79c0b5b60 0x7fb79c0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.013+0000 7fb79a59c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb79c15dce0 0x7fb79c0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.013+0000 7fb793fff640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fb79c0d35b0 0x7fb79c0b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:47428/0 (socket says 172.21.15.16:47428) 2024-04-25T10:09:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.013+0000 7fb793fff640 1 -- 172.21.15.16:0/3615338392 learned_addr learned my addr 172.21.15.16:0/3615338392 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.013+0000 7fb79a59c640 1 -- 172.21.15.16:0/3615338392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb79c0b5b60 msgr2=0x7fb79c0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.013+0000 7fb79a59c640 1 --2- 172.21.15.16:0/3615338392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb79c0b5b60 0x7fb79c0b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.013+0000 7fb79a59c640 1 -- 172.21.15.16:0/3615338392 >> v1:172.21.15.186:6789/0 conn(0x7fb79c0d35b0 legacy=0x7fb79c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.013+0000 7fb79a59c640 1 -- 172.21.15.16:0/3615338392 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79c0d2ad0 con 0x7fb79c15dce0 2024-04-25T10:09:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.013+0000 7fb799d9b640 1 -- 172.21.15.16:0/3615338392 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052930097 0 0) 0x7fb79c15e0c0 con 0x7fb79c0d35b0 2024-04-25T10:09:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.013+0000 7fb79ad9d640 1 --2- 172.21.15.16:0/3615338392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb79c0b5b60 0x7fb79c0b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.014+0000 7fb79a59c640 1 --2- 172.21.15.16:0/3615338392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb79c15dce0 0x7fb79c0d1a50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0097a0 tx=0x7fb78c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe5a9b4ce3ad224c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.014+0000 7fb799d9b640 1 -- 172.21.15.16:0/3615338392 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c037070 con 0x7fb79c15dce0 2024-04-25T10:09:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.014+0000 7fb799d9b640 1 -- 172.21.15.16:0/3615338392 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78c032440 con 0x7fb79c15dce0 2024-04-25T10:09:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.014+0000 7fb799d9b640 1 -- 172.21.15.16:0/3615338392 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c032810 con 0x7fb79c15dce0 2024-04-25T10:09:10.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.015+0000 7fb7a0b48640 1 -- 172.21.15.16:0/3615338392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb79c15dce0 msgr2=0x7fb79c0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.015+0000 7fb7a0b48640 1 --2- 172.21.15.16:0/3615338392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb79c15dce0 0x7fb79c0d1a50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0097a0 tx=0x7fb78c02a180 comp rx=0 tx=0).stop 2024-04-25T10:09:10.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.015+0000 7fb7a0b48640 1 -- 172.21.15.16:0/3615338392 shutdown_connections 2024-04-25T10:09:10.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.015+0000 7fb7a0b48640 1 --2- 172.21.15.16:0/3615338392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb79c0b5b60 0x7fb79c0b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.015+0000 7fb7a0b48640 1 --2- 172.21.15.16:0/3615338392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb79c15dce0 0x7fb79c0d1a50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.015+0000 7fb7a0b48640 1 -- 172.21.15.16:0/3615338392 >> 172.21.15.16:0/3615338392 conn(0x7fb79c072900 msgr2=0x7fb79c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.015+0000 7fb7a0b48640 1 -- 172.21.15.16:0/3615338392 shutdown_connections 2024-04-25T10:09:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.015+0000 7fb7a0b48640 1 -- 172.21.15.16:0/3615338392 wait complete. 2024-04-25T10:09:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.016+0000 7fb7a0b48640 1 Processor -- start 2024-04-25T10:09:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.016+0000 7fb7a0b48640 1 -- start start 2024-04-25T10:09:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.017+0000 7fb7a0b48640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb79c0b5b60 0x7fb79c157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.017+0000 7fb7a0b48640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb79c0d35b0 0x7fb79c158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.017+0000 7fb7a0b48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb79c15dce0 0x7fb79c152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.017+0000 7fb7a0b48640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c0b9820 con 0x7fb79c15dce0 2024-04-25T10:09:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.017+0000 7fb7a0b48640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c0d3090 con 0x7fb79c0d35b0 2024-04-25T10:09:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.017+0000 7fb7a0b48640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c0b99a0 con 0x7fb79c0b5b60 2024-04-25T10:09:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.017+0000 7fb79ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb79c15dce0 0x7fb79c152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.017+0000 7fb79ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb79c15dce0 0x7fb79c152730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41216/0 (socket says 172.21.15.16:41216) 2024-04-25T10:09:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.017+0000 7fb79ad9d640 1 -- 172.21.15.16:0/1044516206 learned_addr learned my addr 172.21.15.16:0/1044516206 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:10.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.017+0000 7fb793fff640 1 --2- 172.21.15.16:0/1044516206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb79c0d35b0 0x7fb79c158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.017+0000 7fb79a59c640 1 --2- 172.21.15.16:0/1044516206 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb79c0b5b60 0x7fb79c157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.018+0000 7fb79ad9d640 1 -- 172.21.15.16:0/1044516206 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb79c0b5b60 msgr2=0x7fb79c157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.018+0000 7fb79ad9d640 1 --2- 172.21.15.16:0/1044516206 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb79c0b5b60 0x7fb79c157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.018+0000 7fb79ad9d640 1 -- 172.21.15.16:0/1044516206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb79c0d35b0 msgr2=0x7fb79c158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.018+0000 7fb79ad9d640 1 --2- 172.21.15.16:0/1044516206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb79c0d35b0 0x7fb79c158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.018+0000 7fb79ad9d640 1 -- 172.21.15.16:0/1044516206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79c15b610 con 0x7fb79c15dce0 2024-04-25T10:09:10.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.018+0000 7fb79ad9d640 1 --2- 172.21.15.16:0/1044516206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb79c15dce0 0x7fb79c152730 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb79400efc0 tx=0x7fb794009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:10.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.018+0000 7fb792ffd640 1 -- 172.21.15.16:0/1044516206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794018030 con 0x7fb79c15dce0 2024-04-25T10:09:10.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.018+0000 7fb792ffd640 1 -- 172.21.15.16:0/1044516206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79400cde0 con 0x7fb79c15dce0 2024-04-25T10:09:10.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.018+0000 7fb792ffd640 1 -- 172.21.15.16:0/1044516206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794003bb0 con 0x7fb79c15dce0 2024-04-25T10:09:10.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.018+0000 7fb7a0b48640 1 -- 172.21.15.16:0/1044516206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79c15aa90 con 0x7fb79c15dce0 2024-04-25T10:09:10.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.018+0000 7fb7a0b48640 1 -- 172.21.15.16:0/1044516206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79c15b050 con 0x7fb79c15dce0 2024-04-25T10:09:10.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.019+0000 7fb7a0b48640 1 -- 172.21.15.16:0/1044516206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb79c0749b0 con 0x7fb79c15dce0 2024-04-25T10:09:10.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.019+0000 7fb792ffd640 1 -- 172.21.15.16:0/1044516206 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb794005d10 con 0x7fb79c15dce0 2024-04-25T10:09:10.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.020+0000 7fb792ffd640 1 --2- 172.21.15.16:0/1044516206 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb77c06b4d0 0x7fb77c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.020+0000 7fb792ffd640 1 -- 172.21.15.16:0/1044516206 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb79408ab20 con 0x7fb79c15dce0 2024-04-25T10:09:10.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.020+0000 7fb79a59c640 1 --2- 172.21.15.16:0/1044516206 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb77c06b4d0 0x7fb77c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.021+0000 7fb79a59c640 1 --2- 172.21.15.16:0/1044516206 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb77c06b4d0 0x7fb77c06d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb78c034000 tx=0x7fb78c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:10.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.024+0000 7fb792ffd640 1 -- 172.21.15.16:0/1044516206 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb794022070 con 0x7fb79c15dce0 2024-04-25T10:09:10.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.206+0000 7fb7a0b48640 1 -- 172.21.15.16:0/1044516206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb79c0cdce0 con 0x7fb79c15dce0 2024-04-25T10:09:10.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.207+0000 7fb792ffd640 1 -- 172.21.15.16:0/1044516206 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16761 (secure 0 0 0) 0x7fb794010040 con 0x7fb79c15dce0 2024-04-25T10:09:10.214 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:10.214 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":61,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:08:52.605846+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:48.948980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:49.638457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.19","mappings":[{"from":6,"to":1}]},{"pgid":"9.28","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.209+0000 7fb7a0b48640 1 -- 172.21.15.16:0/1044516206 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb77c06b4d0 msgr2=0x7fb77c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.209+0000 7fb7a0b48640 1 --2- 172.21.15.16:0/1044516206 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb77c06b4d0 0x7fb77c06d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb78c034000 tx=0x7fb78c02e000 comp rx=0 tx=0).stop 2024-04-25T10:09:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.209+0000 7fb7a0b48640 1 -- 172.21.15.16:0/1044516206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb79c15dce0 msgr2=0x7fb79c152730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.209+0000 7fb7a0b48640 1 --2- 172.21.15.16:0/1044516206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb79c15dce0 0x7fb79c152730 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb79400efc0 tx=0x7fb794009f90 comp rx=0 tx=0).stop 2024-04-25T10:09:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.210+0000 7fb7a0b48640 1 -- 172.21.15.16:0/1044516206 shutdown_connections 2024-04-25T10:09:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.210+0000 7fb7a0b48640 1 --2- 172.21.15.16:0/1044516206 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb77c06b4d0 0x7fb77c06d990 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.210+0000 7fb7a0b48640 1 --2- 172.21.15.16:0/1044516206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb79c15dce0 0x7fb79c152730 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.210+0000 7fb7a0b48640 1 --2- 172.21.15.16:0/1044516206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb79c0d35b0 0x7fb79c158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.210+0000 7fb7a0b48640 1 --2- 172.21.15.16:0/1044516206 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb79c0b5b60 0x7fb79c157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.210+0000 7fb7a0b48640 1 -- 172.21.15.16:0/1044516206 >> 172.21.15.16:0/1044516206 conn(0x7fb79c072900 msgr2=0x7fb79c0b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.210+0000 7fb7a0b48640 1 -- 172.21.15.16:0/1044516206 shutdown_connections 2024-04-25T10:09:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.210+0000 7fb7a0b48640 1 -- 172.21.15.16:0/1044516206 wait complete. 2024-04-25T10:09:10.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:09:10.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.324+0000 7f21df59e640 1 Processor -- start 2024-04-25T10:09:10.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.324+0000 7f21df59e640 1 -- start start 2024-04-25T10:09:10.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.325+0000 7f21df59e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f21d80d07b0 0x7f21d80cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.325+0000 7f21df59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d80b5710 con 0x7f21d80ce860 2024-04-25T10:09:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.325+0000 7f21df59e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d80b59f0 con 0x7f21d815b7e0 2024-04-25T10:09:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.325+0000 7f21df59e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d80b5cd0 con 0x7f21d80d07b0 2024-04-25T10:09:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.325+0000 7f21ddd9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f21d80ce860 0x7f21d80d00a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32878/0 (socket says 172.21.15.16:32878) 2024-04-25T10:09:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.325+0000 7f21ddd9b640 1 -- 172.21.15.16:0/365091412 learned_addr learned my addr 172.21.15.16:0/365091412 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.325+0000 7f21ded9d640 1 --2- 172.21.15.16:0/365091412 >> v2:172.21.15.186:3300/0 conn(0x7f21d80d07b0 0x7f21d80cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.325+0000 7f21dd59a640 1 -- 172.21.15.16:0/365091412 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 189394242 0 0) 0x7f21d80b5710 con 0x7f21d80ce860 2024-04-25T10:09:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.325+0000 7f21dd59a640 1 -- 172.21.15.16:0/365091412 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21c4003520 con 0x7f21d80ce860 2024-04-25T10:09:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.325+0000 7f21dd59a640 1 -- 172.21.15.16:0/365091412 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021537657 0 0) 0x7f21d80b59f0 con 0x7f21d815b7e0 2024-04-25T10:09:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.325+0000 7f21dd59a640 1 -- 172.21.15.16:0/365091412 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21d80b5710 con 0x7f21d815b7e0 2024-04-25T10:09:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.325+0000 7f21ded9d640 1 -- 172.21.15.16:0/365091412 >> v1:172.21.15.186:6789/0 conn(0x7f21d815b7e0 legacy=0x7f21d815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.325+0000 7f21ded9d640 1 -- 172.21.15.16:0/365091412 >> v1:172.21.15.16:6789/0 conn(0x7f21d80ce860 legacy=0x7f21d80d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.325+0000 7f21ded9d640 1 -- 172.21.15.16:0/365091412 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d80b5fb0 con 0x7f21d80d07b0 2024-04-25T10:09:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.326+0000 7f21ded9d640 1 --2- 172.21.15.16:0/365091412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21d80d07b0 0x7f21d80cecf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009d90 tx=0x7f21d402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0a17a47b042b189 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.326+0000 7f21dd59a640 1 -- 172.21.15.16:0/365091412 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d4031600 con 0x7f21d80d07b0 2024-04-25T10:09:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.326+0000 7f21dd59a640 1 -- 172.21.15.16:0/365091412 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d40317a0 con 0x7f21d80d07b0 2024-04-25T10:09:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.326+0000 7f21dd59a640 1 -- 172.21.15.16:0/365091412 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d4031b70 con 0x7f21d80d07b0 2024-04-25T10:09:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.327+0000 7f21df59e640 1 -- 172.21.15.16:0/365091412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21d80d07b0 msgr2=0x7f21d80cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.327+0000 7f21df59e640 1 --2- 172.21.15.16:0/365091412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21d80d07b0 0x7f21d80cecf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009d90 tx=0x7f21d402a5d0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.327+0000 7f21df59e640 1 -- 172.21.15.16:0/365091412 shutdown_connections 2024-04-25T10:09:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.327+0000 7f21df59e640 1 --2- 172.21.15.16:0/365091412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21d80d07b0 0x7f21d80cecf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.327+0000 7f21df59e640 1 -- 172.21.15.16:0/365091412 >> 172.21.15.16:0/365091412 conn(0x7f21d80722d0 msgr2=0x7f21d80726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.327+0000 7f21df59e640 1 -- 172.21.15.16:0/365091412 shutdown_connections 2024-04-25T10:09:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.327+0000 7f21df59e640 1 -- 172.21.15.16:0/365091412 wait complete. 2024-04-25T10:09:10.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.328+0000 7f21df59e640 1 Processor -- start 2024-04-25T10:09:10.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.328+0000 7f21df59e640 1 -- start start 2024-04-25T10:09:10.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.329+0000 7f21df59e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21d80ce860 0x7f21d80ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.329+0000 7f21df59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d80d07b0 0x7f21d815bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.329+0000 7f21df59e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21d815b7e0 0x7f21d8146750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.329+0000 7f21df59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d8161ae0 con 0x7f21d80d07b0 2024-04-25T10:09:10.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.329+0000 7f21df59e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d8161960 con 0x7f21d815b7e0 2024-04-25T10:09:10.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.329+0000 7f21df59e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d8161c60 con 0x7f21d80ce860 2024-04-25T10:09:10.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.329+0000 7f21ddd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d80d07b0 0x7f21d815bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.329+0000 7f21ddd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d80d07b0 0x7f21d815bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41224/0 (socket says 172.21.15.16:41224) 2024-04-25T10:09:10.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.329+0000 7f21ddd9b640 1 -- 172.21.15.16:0/3142335727 learned_addr learned my addr 172.21.15.16:0/3142335727 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.329+0000 7f21ded9d640 1 --2- 172.21.15.16:0/3142335727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21d815b7e0 0x7f21d8146750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.329+0000 7f21de59c640 1 --2- 172.21.15.16:0/3142335727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21d80ce860 0x7f21d80ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.330+0000 7f21ddd9b640 1 -- 172.21.15.16:0/3142335727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21d80ce860 msgr2=0x7f21d80ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.330+0000 7f21ddd9b640 1 --2- 172.21.15.16:0/3142335727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21d80ce860 0x7f21d80ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.330+0000 7f21ddd9b640 1 -- 172.21.15.16:0/3142335727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21d815b7e0 msgr2=0x7f21d8146750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.330+0000 7f21ddd9b640 1 --2- 172.21.15.16:0/3142335727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21d815b7e0 0x7f21d8146750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.330+0000 7f21ddd9b640 1 -- 172.21.15.16:0/3142335727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d80cbda0 con 0x7f21d80d07b0 2024-04-25T10:09:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.330+0000 7f21de59c640 1 --2- 172.21.15.16:0/3142335727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21d80ce860 0x7f21d80ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.330+0000 7f21ddd9b640 1 --2- 172.21.15.16:0/3142335727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d80d07b0 0x7f21d815bc20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f21d000c530 tx=0x7f21d000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.330+0000 7f21caffd640 1 -- 172.21.15.16:0/3142335727 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d0017070 con 0x7f21d80d07b0 2024-04-25T10:09:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.330+0000 7f21caffd640 1 -- 172.21.15.16:0/3142335727 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d0005ce0 con 0x7f21d80d07b0 2024-04-25T10:09:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.330+0000 7f21df59e640 1 -- 172.21.15.16:0/3142335727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d80cc080 con 0x7f21d80d07b0 2024-04-25T10:09:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.330+0000 7f21caffd640 1 -- 172.21.15.16:0/3142335727 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d0007270 con 0x7f21d80d07b0 2024-04-25T10:09:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.330+0000 7f21df59e640 1 -- 172.21.15.16:0/3142335727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d80cc640 con 0x7f21d80d07b0 2024-04-25T10:09:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.331+0000 7f21df59e640 1 -- 172.21.15.16:0/3142335727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d80749b0 con 0x7f21d80d07b0 2024-04-25T10:09:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.331+0000 7f21caffd640 1 -- 172.21.15.16:0/3142335727 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21d001a070 con 0x7f21d80d07b0 2024-04-25T10:09:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.332+0000 7f21caffd640 1 --2- 172.21.15.16:0/3142335727 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f21bc06b4d0 0x7f21bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.332+0000 7f21caffd640 1 -- 172.21.15.16:0/3142335727 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f21d0089520 con 0x7f21d80d07b0 2024-04-25T10:09:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.332+0000 7f21de59c640 1 --2- 172.21.15.16:0/3142335727 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f21bc06b4d0 0x7f21bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.333+0000 7f21de59c640 1 --2- 172.21.15.16:0/3142335727 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f21bc06b4d0 0x7f21bc06d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f21cc001670 tx=0x7f21cc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:10.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.336+0000 7f21caffd640 1 -- 172.21.15.16:0/3142335727 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21d0057d10 con 0x7f21d80d07b0 2024-04-25T10:09:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.556+0000 7f21df59e640 1 -- 172.21.15.16:0/3142335727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f21d80b6eb0 con 0x7f21d80d07b0 2024-04-25T10:09:10.562 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-04-25T10:09:10.557+0000 7ff13f9bb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:09:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.558+0000 7f21caffd640 1 -- 172.21.15.16:0/3142335727 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) v1 ==== 161+0+0 (secure 0 0 0) 0x7f21d005bdb0 con 0x7f21d80d07b0 2024-04-25T10:09:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:09:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.560+0000 7f21df59e640 1 -- 172.21.15.16:0/3142335727 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f21bc06b4d0 msgr2=0x7f21bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.560+0000 7f21df59e640 1 --2- 172.21.15.16:0/3142335727 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f21bc06b4d0 0x7f21bc06d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f21cc001670 tx=0x7f21cc009040 comp rx=0 tx=0).stop 2024-04-25T10:09:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.561+0000 7f21df59e640 1 -- 172.21.15.16:0/3142335727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d80d07b0 msgr2=0x7f21d815bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.561+0000 7f21df59e640 1 --2- 172.21.15.16:0/3142335727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d80d07b0 0x7f21d815bc20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f21d000c530 tx=0x7f21d000c9f0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.561+0000 7f21df59e640 1 -- 172.21.15.16:0/3142335727 shutdown_connections 2024-04-25T10:09:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.561+0000 7f21df59e640 1 --2- 172.21.15.16:0/3142335727 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f21bc06b4d0 0x7f21bc06d990 secure :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f21cc001670 tx=0x7f21cc009040 comp rx=0 tx=0).stop 2024-04-25T10:09:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.561+0000 7f21df59e640 1 --2- 172.21.15.16:0/3142335727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21d815b7e0 0x7f21d8146750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.561+0000 7f21df59e640 1 --2- 172.21.15.16:0/3142335727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d80d07b0 0x7f21d815bc20 secure :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f21d000c530 tx=0x7f21d000c9f0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.561+0000 7f21df59e640 1 --2- 172.21.15.16:0/3142335727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21d80ce860 0x7f21d80ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.561+0000 7f21df59e640 1 -- 172.21.15.16:0/3142335727 >> 172.21.15.16:0/3142335727 conn(0x7f21d80722d0 msgr2=0x7f21d8153e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.561+0000 7f21df59e640 1 -- 172.21.15.16:0/3142335727 shutdown_connections 2024-04-25T10:09:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.561+0000 7f21df59e640 1 -- 172.21.15.16:0/3142335727 wait complete. 2024-04-25T10:09:10.571 INFO:tasks.ceph.mds.a.smithi016.stderr: -7491> 2024-04-25T10:07:56.972+0000 7fce9e140640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:09:10.573 INFO:tasks.ceph.mds.a.smithi016.stderr: -4837> 2024-04-25T10:08:02.579+0000 7fce9a138640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={35240=0-4194304@1} caps={35240=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=35240 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55cf29494100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:09:10.574 INFO:tasks.ceph.mds.a.smithi016.stderr: -3083> 2024-04-25T10:08:33.880+0000 7fce9a138640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={35240=0-4194304@1} caps={35240=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=35240 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55cf29494100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:09:10.579 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:09:10.603 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-04-25T10:09:10.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.679+0000 7fd8f3322640 1 Processor -- start 2024-04-25T10:09:10.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.679+0000 7fd8f3322640 1 -- start start 2024-04-25T10:09:10.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.680+0000 7fd8f3322640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8ec0b5550 0x7fd8ec0b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.680+0000 7fd8f3322640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ec161500 con 0x7fd8ec0d2560 2024-04-25T10:09:10.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.680+0000 7fd8f3322640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ec161eb0 con 0x7fd8ec161120 2024-04-25T10:09:10.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.680+0000 7fd8f3322640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8ec0b5ef0 con 0x7fd8ec0b5550 2024-04-25T10:09:10.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.680+0000 7fd8f08bd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8ec0b5550 0x7fd8ec0b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.680+0000 7fd8f18bf640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd8ec0d2560 0x7fd8ec0d2980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32884/0 (socket says 172.21.15.16:32884) 2024-04-25T10:09:10.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.680+0000 7fd8f18bf640 1 -- 172.21.15.16:0/3727157817 learned_addr learned my addr 172.21.15.16:0/3727157817 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:10.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.680+0000 7fd8f08bd640 1 -- 172.21.15.16:0/3727157817 >> v1:172.21.15.186:6789/0 conn(0x7fd8ec161120 legacy=0x7fd8ec1617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:10.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.680+0000 7fd8f08bd640 1 -- 172.21.15.16:0/3727157817 >> v1:172.21.15.16:6789/0 conn(0x7fd8ec0d2560 legacy=0x7fd8ec0d2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.680+0000 7fd8f08bd640 1 -- 172.21.15.16:0/3727157817 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ec162180 con 0x7fd8ec0b5550 2024-04-25T10:09:10.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.680+0000 7fd8e3fff640 1 -- 172.21.15.16:0/3727157817 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936485021 0 0) 0x7fd8ec161500 con 0x7fd8ec0d2560 2024-04-25T10:09:10.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.681+0000 7fd8f08bd640 1 --2- 172.21.15.16:0/3727157817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ec0b5550 0x7fd8ec0b59b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0097a0 tx=0x7fd8dc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63da551b96ab5b1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:10.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.681+0000 7fd8e3fff640 1 -- 172.21.15.16:0/3727157817 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc037070 con 0x7fd8ec0b5550 2024-04-25T10:09:10.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.681+0000 7fd8e3fff640 1 -- 172.21.15.16:0/3727157817 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8dc02ae50 con 0x7fd8ec0b5550 2024-04-25T10:09:10.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.681+0000 7fd8e3fff640 1 -- 172.21.15.16:0/3727157817 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc032660 con 0x7fd8ec0b5550 2024-04-25T10:09:10.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.682+0000 7fd8f3322640 1 -- 172.21.15.16:0/3727157817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ec0b5550 msgr2=0x7fd8ec0b59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.682+0000 7fd8f3322640 1 --2- 172.21.15.16:0/3727157817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ec0b5550 0x7fd8ec0b59b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0097a0 tx=0x7fd8dc02a140 comp rx=0 tx=0).stop 2024-04-25T10:09:10.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.682+0000 7fd8f3322640 1 -- 172.21.15.16:0/3727157817 shutdown_connections 2024-04-25T10:09:10.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.682+0000 7fd8f3322640 1 --2- 172.21.15.16:0/3727157817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ec0b5550 0x7fd8ec0b59b0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.682+0000 7fd8f3322640 1 -- 172.21.15.16:0/3727157817 >> 172.21.15.16:0/3727157817 conn(0x7fd8ec0d0550 msgr2=0x7fd8ec0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:10.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.682+0000 7fd8f3322640 1 -- 172.21.15.16:0/3727157817 shutdown_connections 2024-04-25T10:09:10.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.682+0000 7fd8f3322640 1 -- 172.21.15.16:0/3727157817 wait complete. 2024-04-25T10:09:10.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.683+0000 7fd8f3322640 1 Processor -- start 2024-04-25T10:09:10.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.683+0000 7fd8f3322640 1 -- start start 2024-04-25T10:09:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8f3322640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8ec0b5550 0x7fd8ec14c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8f3322640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8ec0d2560 0x7fd8ec15f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8f3322640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ec161120 0x7fd8ec157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8f3322640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0d1d60 con 0x7fd8ec161120 2024-04-25T10:09:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8f3322640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec162740 con 0x7fd8ec0d2560 2024-04-25T10:09:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8f3322640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0d2040 con 0x7fd8ec0b5550 2024-04-25T10:09:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8f18bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ec161120 0x7fd8ec157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8f18bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ec161120 0x7fd8ec157c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41242/0 (socket says 172.21.15.16:41242) 2024-04-25T10:09:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8f18bf640 1 -- 172.21.15.16:0/1925829926 learned_addr learned my addr 172.21.15.16:0/1925829926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8f08bd640 1 --2- 172.21.15.16:0/1925829926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8ec0d2560 0x7fd8ec15f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8f10be640 1 --2- 172.21.15.16:0/1925829926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8ec0b5550 0x7fd8ec14c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8f18bf640 1 -- 172.21.15.16:0/1925829926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8ec0b5550 msgr2=0x7fd8ec14c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8f18bf640 1 --2- 172.21.15.16:0/1925829926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8ec0b5550 0x7fd8ec14c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8f18bf640 1 -- 172.21.15.16:0/1925829926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8ec0d2560 msgr2=0x7fd8ec15f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8f18bf640 1 --2- 172.21.15.16:0/1925829926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8ec0d2560 0x7fd8ec15f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8f18bf640 1 -- 172.21.15.16:0/1925829926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ec14cb50 con 0x7fd8ec161120 2024-04-25T10:09:10.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8f18bf640 1 --2- 172.21.15.16:0/1925829926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ec161120 0x7fd8ec157c10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400c7a0 tx=0x7fd8e400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:10.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8e1ffb640 1 -- 172.21.15.16:0/1925829926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4018070 con 0x7fd8ec161120 2024-04-25T10:09:10.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8e1ffb640 1 -- 172.21.15.16:0/1925829926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e400a640 con 0x7fd8ec161120 2024-04-25T10:09:10.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8f3322640 1 -- 172.21.15.16:0/1925829926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ec0cac40 con 0x7fd8ec161120 2024-04-25T10:09:10.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.684+0000 7fd8e1ffb640 1 -- 172.21.15.16:0/1925829926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e400aab0 con 0x7fd8ec161120 2024-04-25T10:09:10.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.685+0000 7fd8f3322640 1 -- 172.21.15.16:0/1925829926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ec160340 con 0x7fd8ec161120 2024-04-25T10:09:10.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.686+0000 7fd8f3322640 1 -- 172.21.15.16:0/1925829926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ec0cae10 con 0x7fd8ec161120 2024-04-25T10:09:10.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.686+0000 7fd8e1ffb640 1 -- 172.21.15.16:0/1925829926 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8e401b070 con 0x7fd8ec161120 2024-04-25T10:09:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.686+0000 7fd8e1ffb640 1 --2- 172.21.15.16:0/1925829926 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd8d006b4d0 0x7fd8d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.686+0000 7fd8e1ffb640 1 -- 172.21.15.16:0/1925829926 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd8e4089820 con 0x7fd8ec161120 2024-04-25T10:09:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.687+0000 7fd8f10be640 1 --2- 172.21.15.16:0/1925829926 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd8d006b4d0 0x7fd8d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.687+0000 7fd8f10be640 1 --2- 172.21.15.16:0/1925829926 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd8d006b4d0 0x7fd8d006d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8006fd0 tx=0x7fd8e8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:10.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.691+0000 7fd8e1ffb640 1 -- 172.21.15.16:0/1925829926 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd8e40586a0 con 0x7fd8ec161120 2024-04-25T10:09:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:10.914+0000 7fd8f3322640 1 -- 172.21.15.16:0/1925829926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd8ec0750b0 con 0x7fd8ec161120 2024-04-25T10:09:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.559+0000 7fd8e1ffb640 1 -- 172.21.15.16:0/1925829926 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd8e4057fa0 con 0x7fd8ec161120 2024-04-25T10:09:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.562+0000 7fd8f3322640 1 -- 172.21.15.16:0/1925829926 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd8d006b4d0 msgr2=0x7fd8d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.562+0000 7fd8f3322640 1 --2- 172.21.15.16:0/1925829926 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd8d006b4d0 0x7fd8d006d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8006fd0 tx=0x7fd8e8008040 comp rx=0 tx=0).stop 2024-04-25T10:09:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.562+0000 7fd8f3322640 1 -- 172.21.15.16:0/1925829926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ec161120 msgr2=0x7fd8ec157c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.562+0000 7fd8f3322640 1 --2- 172.21.15.16:0/1925829926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ec161120 0x7fd8ec157c10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400c7a0 tx=0x7fd8e400cc60 comp rx=0 tx=0).stop 2024-04-25T10:09:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.562+0000 7fd8f3322640 1 -- 172.21.15.16:0/1925829926 shutdown_connections 2024-04-25T10:09:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.562+0000 7fd8f3322640 1 --2- 172.21.15.16:0/1925829926 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd8d006b4d0 0x7fd8d006d990 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.562+0000 7fd8f3322640 1 --2- 172.21.15.16:0/1925829926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ec161120 0x7fd8ec157c10 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.562+0000 7fd8f3322640 1 --2- 172.21.15.16:0/1925829926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8ec0d2560 0x7fd8ec15f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.562+0000 7fd8f3322640 1 --2- 172.21.15.16:0/1925829926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8ec0b5550 0x7fd8ec14c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.562+0000 7fd8f3322640 1 -- 172.21.15.16:0/1925829926 >> 172.21.15.16:0/1925829926 conn(0x7fd8ec0d0550 msgr2=0x7fd8ec06c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.562+0000 7fd8f3322640 1 -- 172.21.15.16:0/1925829926 shutdown_connections 2024-04-25T10:09:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.562+0000 7fd8f3322640 1 -- 172.21.15.16:0/1925829926 wait complete. 2024-04-25T10:09:11.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:09:11.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.733+0000 7fab3ce88640 1 Processor -- start 2024-04-25T10:09:11.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.733+0000 7fab3ce88640 1 -- start start 2024-04-25T10:09:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.734+0000 7fab3ce88640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab3806f980 0x7fab380cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.734+0000 7fab3ce88640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab380cc8c0 0x7fab380cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.734+0000 7fab3ce88640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab38070440 con 0x7fab3806aba0 2024-04-25T10:09:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.734+0000 7fab3ce88640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab3815e280 con 0x7fab3806f980 2024-04-25T10:09:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.734+0000 7fab3ce88640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab380caed0 con 0x7fab380cc8c0 2024-04-25T10:09:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.734+0000 7fab35d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab3806f980 0x7fab380cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.734+0000 7fab35d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab3806f980 0x7fab380cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41252/0 (socket says 172.21.15.16:41252) 2024-04-25T10:09:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.734+0000 7fab35d9b640 1 -- 172.21.15.16:0/1531739234 learned_addr learned my addr 172.21.15.16:0/1531739234 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.734+0000 7fab36d9d640 1 --2- 172.21.15.16:0/1531739234 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab380cc8c0 0x7fab380cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.734+0000 7fab35d9b640 1 -- 172.21.15.16:0/1531739234 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab380cc8c0 msgr2=0x7fab380cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.734+0000 7fab35d9b640 1 --2- 172.21.15.16:0/1531739234 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab380cc8c0 0x7fab380cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.734+0000 7fab35d9b640 1 -- 172.21.15.16:0/1531739234 >> v1:172.21.15.186:6789/0 conn(0x7fab3806aba0 legacy=0x7fab3806af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.734+0000 7fab35d9b640 1 -- 172.21.15.16:0/1531739234 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3806fe80 con 0x7fab3806f980 2024-04-25T10:09:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.735+0000 7fab35d9b640 1 --2- 172.21.15.16:0/1531739234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3806f980 0x7fab380cb690 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009870 tx=0x7fab2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee67395bbc33aa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.735+0000 7fab3559a640 1 -- 172.21.15.16:0/1531739234 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c037070 con 0x7fab3806f980 2024-04-25T10:09:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.735+0000 7fab3559a640 1 -- 172.21.15.16:0/1531739234 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2c032440 con 0x7fab3806f980 2024-04-25T10:09:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.735+0000 7fab3559a640 1 -- 172.21.15.16:0/1531739234 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c032810 con 0x7fab3806f980 2024-04-25T10:09:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.736+0000 7fab3ce88640 1 -- 172.21.15.16:0/1531739234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3806f980 msgr2=0x7fab380cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.736+0000 7fab3ce88640 1 --2- 172.21.15.16:0/1531739234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3806f980 0x7fab380cb690 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009870 tx=0x7fab2c02a270 comp rx=0 tx=0).stop 2024-04-25T10:09:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.736+0000 7fab3ce88640 1 -- 172.21.15.16:0/1531739234 shutdown_connections 2024-04-25T10:09:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.736+0000 7fab3ce88640 1 --2- 172.21.15.16:0/1531739234 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab380cc8c0 0x7fab380cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.736+0000 7fab3ce88640 1 --2- 172.21.15.16:0/1531739234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3806f980 0x7fab380cb690 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.736+0000 7fab3ce88640 1 -- 172.21.15.16:0/1531739234 >> 172.21.15.16:0/1531739234 conn(0x7fab380d0750 msgr2=0x7fab380d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.736+0000 7fab3ce88640 1 -- 172.21.15.16:0/1531739234 shutdown_connections 2024-04-25T10:09:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.736+0000 7fab3ce88640 1 -- 172.21.15.16:0/1531739234 wait complete. 2024-04-25T10:09:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.737+0000 7fab3ce88640 1 Processor -- start 2024-04-25T10:09:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.737+0000 7fab3ce88640 1 -- start start 2024-04-25T10:09:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.738+0000 7fab3ce88640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab3806aba0 0x7fab3814f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.738+0000 7fab3ce88640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3806f980 0x7fab380c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.738+0000 7fab3ce88640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab380cc8c0 0x7fab380c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.738+0000 7fab3ce88640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3806b950 con 0x7fab3806f980 2024-04-25T10:09:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.738+0000 7fab3ce88640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab380cb170 con 0x7fab3806aba0 2024-04-25T10:09:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.738+0000 7fab3ce88640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3806bad0 con 0x7fab380cc8c0 2024-04-25T10:09:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.738+0000 7fab35d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3806f980 0x7fab380c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.738+0000 7fab35d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3806f980 0x7fab380c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41258/0 (socket says 172.21.15.16:41258) 2024-04-25T10:09:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.738+0000 7fab35d9b640 1 -- 172.21.15.16:0/2439769358 learned_addr learned my addr 172.21.15.16:0/2439769358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:11.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.738+0000 7fab3659c640 1 --2- 172.21.15.16:0/2439769358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab3806aba0 0x7fab3814f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.738+0000 7fab36d9d640 1 --2- 172.21.15.16:0/2439769358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab380cc8c0 0x7fab380c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.738+0000 7fab35d9b640 1 -- 172.21.15.16:0/2439769358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab380cc8c0 msgr2=0x7fab380c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.738+0000 7fab35d9b640 1 --2- 172.21.15.16:0/2439769358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab380cc8c0 0x7fab380c6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.738+0000 7fab35d9b640 1 -- 172.21.15.16:0/2439769358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab3806aba0 msgr2=0x7fab3814f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.738+0000 7fab35d9b640 1 --2- 172.21.15.16:0/2439769358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab3806aba0 0x7fab3814f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.738+0000 7fab35d9b640 1 -- 172.21.15.16:0/2439769358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab380ca220 con 0x7fab3806f980 2024-04-25T10:09:11.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.739+0000 7fab36d9d640 1 --2- 172.21.15.16:0/2439769358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab380cc8c0 0x7fab380c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:11.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.739+0000 7fab35d9b640 1 --2- 172.21.15.16:0/2439769358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3806f980 0x7fab380c9230 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fab2c000c00 tx=0x7fab2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:11.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.739+0000 7fab26ffd640 1 -- 172.21.15.16:0/2439769358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c041020 con 0x7fab3806f980 2024-04-25T10:09:11.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.739+0000 7fab26ffd640 1 -- 172.21.15.16:0/2439769358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2c0306c0 con 0x7fab3806f980 2024-04-25T10:09:11.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.739+0000 7fab3ce88640 1 -- 172.21.15.16:0/2439769358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab380c9980 con 0x7fab3806f980 2024-04-25T10:09:11.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.739+0000 7fab26ffd640 1 -- 172.21.15.16:0/2439769358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c037040 con 0x7fab3806f980 2024-04-25T10:09:11.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.739+0000 7fab3ce88640 1 -- 172.21.15.16:0/2439769358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab380c9f40 con 0x7fab3806f980 2024-04-25T10:09:11.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.740+0000 7fab3ce88640 1 -- 172.21.15.16:0/2439769358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab380750b0 con 0x7fab3806f980 2024-04-25T10:09:11.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.740+0000 7fab26ffd640 1 -- 172.21.15.16:0/2439769358 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab2c044020 con 0x7fab3806f980 2024-04-25T10:09:11.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.741+0000 7fab26ffd640 1 --2- 172.21.15.16:0/2439769358 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fab1806b4d0 0x7fab1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.741+0000 7fab26ffd640 1 -- 172.21.15.16:0/2439769358 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fab2c0aa9a0 con 0x7fab3806f980 2024-04-25T10:09:11.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.741+0000 7fab3659c640 1 --2- 172.21.15.16:0/2439769358 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fab1806b4d0 0x7fab1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.742+0000 7fab3659c640 1 --2- 172.21.15.16:0/2439769358 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fab1806b4d0 0x7fab1806d990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fab28001780 tx=0x7fab28009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.745+0000 7fab26ffd640 1 -- 172.21.15.16:0/2439769358 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab2c04b030 con 0x7fab3806f980 2024-04-25T10:09:11.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:11.925+0000 7fab3ce88640 1 -- 172.21.15.16:0/2439769358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fab380d14a0 con 0x7fab3806f980 2024-04-25T10:09:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.660+0000 7fab26ffd640 1 -- 172.21.15.16:0/2439769358 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v63) v1 ==== 181+0+0 (secure 0 0 0) 0x7fab2c079790 con 0x7fab3806f980 2024-04-25T10:09:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:09:12.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.664+0000 7fab3ce88640 1 -- 172.21.15.16:0/2439769358 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fab1806b4d0 msgr2=0x7fab1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.664+0000 7fab3ce88640 1 --2- 172.21.15.16:0/2439769358 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fab1806b4d0 0x7fab1806d990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fab28001780 tx=0x7fab28009040 comp rx=0 tx=0).stop 2024-04-25T10:09:12.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.664+0000 7fab3ce88640 1 -- 172.21.15.16:0/2439769358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3806f980 msgr2=0x7fab380c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.664+0000 7fab3ce88640 1 --2- 172.21.15.16:0/2439769358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3806f980 0x7fab380c9230 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fab2c000c00 tx=0x7fab2c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.664+0000 7fab3ce88640 1 -- 172.21.15.16:0/2439769358 shutdown_connections 2024-04-25T10:09:12.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.664+0000 7fab3ce88640 1 --2- 172.21.15.16:0/2439769358 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fab1806b4d0 0x7fab1806d990 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.664+0000 7fab3ce88640 1 --2- 172.21.15.16:0/2439769358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab380cc8c0 0x7fab380c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.664+0000 7fab3ce88640 1 --2- 172.21.15.16:0/2439769358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3806f980 0x7fab380c9230 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.664+0000 7fab3ce88640 1 --2- 172.21.15.16:0/2439769358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab3806aba0 0x7fab3814f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.664+0000 7fab3ce88640 1 -- 172.21.15.16:0/2439769358 >> 172.21.15.16:0/2439769358 conn(0x7fab380d0750 msgr2=0x7fab380ae7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.664+0000 7fab3ce88640 1 -- 172.21.15.16:0/2439769358 shutdown_connections 2024-04-25T10:09:12.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.664+0000 7fab3ce88640 1 -- 172.21.15.16:0/2439769358 wait complete. 2024-04-25T10:09:12.687 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:09:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.791+0000 7f5292370640 1 Processor -- start 2024-04-25T10:09:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.791+0000 7f5292370640 1 -- start start 2024-04-25T10:09:12.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.792+0000 7f5292370640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f528c0b9720 0x7f528c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.792+0000 7f5292370640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0cece0 0x7f528c0d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.792+0000 7f5292370640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f528c161f80 con 0x7f528c161090 2024-04-25T10:09:12.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.792+0000 7f5292370640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f528c0cf130 con 0x7f528c0b9720 2024-04-25T10:09:12.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.792+0000 7f5292370640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528c0d21a0 con 0x7f528c0cece0 2024-04-25T10:09:12.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.792+0000 7f529090d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0cece0 0x7f528c0d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.792+0000 7f528b7fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f528c0b9720 0x7f528c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.792+0000 7f528b7fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f528c0b9720 0x7f528c0ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:47712/0 (socket says 172.21.15.16:47712) 2024-04-25T10:09:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.792+0000 7f528b7fe640 1 -- 172.21.15.16:0/943593923 learned_addr learned my addr 172.21.15.16:0/943593923 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.792+0000 7f528bfff640 1 --1- 172.21.15.16:0/943593923 >> v1:172.21.15.186:6789/0 conn(0x7f528c161090 0x7f528c161730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:39334/0 (socket says 172.21.15.16:39334) 2024-04-25T10:09:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.793+0000 7f528b7fe640 1 -- 172.21.15.16:0/943593923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0cece0 msgr2=0x7f528c0d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.793+0000 7f528b7fe640 1 --2- 172.21.15.16:0/943593923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0cece0 0x7f528c0d2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.793+0000 7f528b7fe640 1 -- 172.21.15.16:0/943593923 >> v1:172.21.15.186:6789/0 conn(0x7f528c161090 legacy=0x7f528c161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.793+0000 7f528b7fe640 1 -- 172.21.15.16:0/943593923 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c162260 con 0x7f528c0b9720 2024-04-25T10:09:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.793+0000 7f528affd640 1 -- 172.21.15.16:0/943593923 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462061890 0 0) 0x7f528c161f80 con 0x7f528c161090 2024-04-25T10:09:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.793+0000 7f529090d640 1 --2- 172.21.15.16:0/943593923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0cece0 0x7f528c0d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:12.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.793+0000 7f528b7fe640 1 --2- 172.21.15.16:0/943593923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f528c0b9720 0x7f528c0ce7a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f527c009870 tx=0x7f527c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f73f61bb67590273 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.794+0000 7f528affd640 1 -- 172.21.15.16:0/943593923 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c037070 con 0x7f528c0b9720 2024-04-25T10:09:12.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.794+0000 7f528affd640 1 -- 172.21.15.16:0/943593923 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527c032440 con 0x7f528c0b9720 2024-04-25T10:09:12.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.794+0000 7f528affd640 1 -- 172.21.15.16:0/943593923 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c032810 con 0x7f528c0b9720 2024-04-25T10:09:12.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.794+0000 7f5292370640 1 -- 172.21.15.16:0/943593923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f528c0b9720 msgr2=0x7f528c0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.794+0000 7f5292370640 1 --2- 172.21.15.16:0/943593923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f528c0b9720 0x7f528c0ce7a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f527c009870 tx=0x7f527c02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.794+0000 7f5292370640 1 -- 172.21.15.16:0/943593923 shutdown_connections 2024-04-25T10:09:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.794+0000 7f5292370640 1 --2- 172.21.15.16:0/943593923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0cece0 0x7f528c0d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.794+0000 7f5292370640 1 --2- 172.21.15.16:0/943593923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f528c0b9720 0x7f528c0ce7a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.794+0000 7f5292370640 1 -- 172.21.15.16:0/943593923 >> 172.21.15.16:0/943593923 conn(0x7f528c0b55e0 msgr2=0x7f528c0b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.794+0000 7f5292370640 1 -- 172.21.15.16:0/943593923 shutdown_connections 2024-04-25T10:09:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.794+0000 7f5292370640 1 -- 172.21.15.16:0/943593923 wait complete. 2024-04-25T10:09:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.795+0000 7f5292370640 1 Processor -- start 2024-04-25T10:09:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.796+0000 7f5292370640 1 -- start start 2024-04-25T10:09:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.796+0000 7f5292370640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0b9720 0x7f528c0aa440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.796+0000 7f5292370640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f528c0cece0 0x7f528c0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.796+0000 7f5292370640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f528c161090 0x7f528c0c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.796+0000 7f5292370640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c161470 con 0x7f528c161090 2024-04-25T10:09:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.796+0000 7f5292370640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c0d1e30 con 0x7f528c0cece0 2024-04-25T10:09:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.796+0000 7f5292370640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528c0d2b70 con 0x7f528c0b9720 2024-04-25T10:09:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.797+0000 7f529090d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f528c161090 0x7f528c0c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.797+0000 7f529090d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f528c161090 0x7f528c0c8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39616/0 (socket says 172.21.15.16:39616) 2024-04-25T10:09:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.797+0000 7f529090d640 1 -- 172.21.15.16:0/1403675105 learned_addr learned my addr 172.21.15.16:0/1403675105 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.797+0000 7f528bfff640 1 --2- 172.21.15.16:0/1403675105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0b9720 0x7f528c0aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.797+0000 7f528b7fe640 1 --2- 172.21.15.16:0/1403675105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f528c0cece0 0x7f528c0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.797+0000 7f529090d640 1 -- 172.21.15.16:0/1403675105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0b9720 msgr2=0x7f528c0aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.797+0000 7f529090d640 1 --2- 172.21.15.16:0/1403675105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0b9720 0x7f528c0aa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.797+0000 7f529090d640 1 -- 172.21.15.16:0/1403675105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f528c0cece0 msgr2=0x7f528c0c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.797+0000 7f529090d640 1 --2- 172.21.15.16:0/1403675105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f528c0cece0 0x7f528c0c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.797+0000 7f529090d640 1 -- 172.21.15.16:0/1403675105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c0aa980 con 0x7f528c161090 2024-04-25T10:09:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.797+0000 7f529090d640 1 --2- 172.21.15.16:0/1403675105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f528c161090 0x7f528c0c8910 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f528400efc0 tx=0x7f5284009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.797+0000 7f5288ff9640 1 -- 172.21.15.16:0/1403675105 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284018030 con 0x7f528c161090 2024-04-25T10:09:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.797+0000 7f5288ff9640 1 -- 172.21.15.16:0/1403675105 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528400cde0 con 0x7f528c161090 2024-04-25T10:09:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.797+0000 7f5292370640 1 -- 172.21.15.16:0/1403675105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528c1f37c0 con 0x7f528c161090 2024-04-25T10:09:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.797+0000 7f5288ff9640 1 -- 172.21.15.16:0/1403675105 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284003bb0 con 0x7f528c161090 2024-04-25T10:09:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.797+0000 7f5292370640 1 -- 172.21.15.16:0/1403675105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528c1f3d80 con 0x7f528c161090 2024-04-25T10:09:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.798+0000 7f5292370640 1 -- 172.21.15.16:0/1403675105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528c0750b0 con 0x7f528c161090 2024-04-25T10:09:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.799+0000 7f5288ff9640 1 -- 172.21.15.16:0/1403675105 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5284005d10 con 0x7f528c161090 2024-04-25T10:09:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.799+0000 7f5288ff9640 1 --2- 172.21.15.16:0/1403675105 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f527806b4d0 0x7f527806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.799+0000 7f5288ff9640 1 -- 172.21.15.16:0/1403675105 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f528408a930 con 0x7f528c161090 2024-04-25T10:09:12.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.800+0000 7f528bfff640 1 --2- 172.21.15.16:0/1403675105 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f527806b4d0 0x7f527806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.800+0000 7f528bfff640 1 --2- 172.21.15.16:0/1403675105 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f527806b4d0 0x7f527806d990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5280007fd0 tx=0x7f5280007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.803+0000 7f5288ff9640 1 -- 172.21.15.16:0/1403675105 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5284022070 con 0x7f528c161090 2024-04-25T10:09:12.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:12.984+0000 7f5292370640 1 -- 172.21.15.16:0/1403675105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f528c0aadf0 con 0x7f528c161090 2024-04-25T10:09:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.660+0000 7f5288ff9640 1 -- 172.21.15.16:0/1403675105 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v64) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5284010040 con 0x7f528c161090 2024-04-25T10:09:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-04-25T10:09:13.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.665+0000 7f5292370640 1 -- 172.21.15.16:0/1403675105 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f527806b4d0 msgr2=0x7f527806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.665+0000 7f5292370640 1 --2- 172.21.15.16:0/1403675105 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f527806b4d0 0x7f527806d990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5280007fd0 tx=0x7f5280007de0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.665+0000 7f5292370640 1 -- 172.21.15.16:0/1403675105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f528c161090 msgr2=0x7f528c0c8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.665+0000 7f5292370640 1 --2- 172.21.15.16:0/1403675105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f528c161090 0x7f528c0c8910 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f528400efc0 tx=0x7f5284009f90 comp rx=0 tx=0).stop 2024-04-25T10:09:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.665+0000 7f5292370640 1 -- 172.21.15.16:0/1403675105 shutdown_connections 2024-04-25T10:09:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.665+0000 7f5292370640 1 --2- 172.21.15.16:0/1403675105 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f527806b4d0 0x7f527806d990 secure :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5280007fd0 tx=0x7f5280007de0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.665+0000 7f5292370640 1 --2- 172.21.15.16:0/1403675105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f528c161090 0x7f528c0c8910 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.665+0000 7f5292370640 1 --2- 172.21.15.16:0/1403675105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f528c0cece0 0x7f528c0c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.665+0000 7f5292370640 1 --2- 172.21.15.16:0/1403675105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0b9720 0x7f528c0aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.665+0000 7f5292370640 1 -- 172.21.15.16:0/1403675105 >> 172.21.15.16:0/1403675105 conn(0x7f528c0b55e0 msgr2=0x7f528c15ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:13.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.667+0000 7f5292370640 1 -- 172.21.15.16:0/1403675105 shutdown_connections 2024-04-25T10:09:13.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.667+0000 7f5292370640 1 -- 172.21.15.16:0/1403675105 wait complete. 2024-04-25T10:09:13.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:09:13.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.794+0000 7f2fadabd640 1 Processor -- start 2024-04-25T10:09:13.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.794+0000 7f2fadabd640 1 -- start start 2024-04-25T10:09:13.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.794+0000 7f2fadabd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2fa80b5550 0x7f2fa80b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.794+0000 7f2fadabd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa8161500 con 0x7f2fa80d2560 2024-04-25T10:09:13.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.794+0000 7f2fadabd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa8161eb0 con 0x7f2fa8161120 2024-04-25T10:09:13.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.794+0000 7f2fadabd640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa80b5ef0 con 0x7f2fa80b5550 2024-04-25T10:09:13.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2fa80d2560 0x7f2fa80d2980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55990/0 (socket says 172.21.15.16:55990) 2024-04-25T10:09:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa7fff640 1 -- 172.21.15.16:0/2480338284 learned_addr learned my addr 172.21.15.16:0/2480338284 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa67fc640 1 -- 172.21.15.16:0/2480338284 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3888684644 0 0) 0x7f2fa8161500 con 0x7f2fa80d2560 2024-04-25T10:09:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa67fc640 1 -- 172.21.15.16:0/2480338284 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f90003520 con 0x7f2fa80d2560 2024-04-25T10:09:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa6ffd640 1 --2- 172.21.15.16:0/2480338284 >> v2:172.21.15.186:3300/0 conn(0x7f2fa80b5550 0x7f2fa80b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa67fc640 1 -- 172.21.15.16:0/2480338284 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 510810630 0 0) 0x7f2f90003520 con 0x7f2fa80d2560 2024-04-25T10:09:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa67fc640 1 -- 172.21.15.16:0/2480338284 >> v2:172.21.15.186:3300/0 conn(0x7f2fa80b5550 msgr2=0x7f2fa80b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa67fc640 1 --2- 172.21.15.16:0/2480338284 >> v2:172.21.15.186:3300/0 conn(0x7f2fa80b5550 0x7f2fa80b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa67fc640 1 -- 172.21.15.16:0/2480338284 >> v1:172.21.15.186:6789/0 conn(0x7f2fa8161120 legacy=0x7f2fa81617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa67fc640 1 -- 172.21.15.16:0/2480338284 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa8162180 con 0x7f2fa80d2560 2024-04-25T10:09:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa67fc640 1 -- 172.21.15.16:0/2480338284 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f2fa00019f0 con 0x7f2fa80d2560 2024-04-25T10:09:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa67fc640 1 -- 172.21.15.16:0/2480338284 >> v1:172.21.15.16:6789/0 conn(0x7f2fa80d2560 legacy=0x7f2fa80d2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa67fc640 1 --2- 172.21.15.16:0/2480338284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f90003a00 0x7f2f90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa67fc640 1 --2- 172.21.15.16:0/2480338284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f90004800 0x7f2f90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa67fc640 1 --2- 172.21.15.16:0/2480338284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f90005210 0x7f2f9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa67fc640 1 -- 172.21.15.16:0/2480338284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9000c130 con 0x7f2f90004800 2024-04-25T10:09:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa67fc640 1 -- 172.21.15.16:0/2480338284 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9000c2e0 con 0x7f2f90005210 2024-04-25T10:09:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa67fc640 1 -- 172.21.15.16:0/2480338284 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9000c4e0 con 0x7f2f90003a00 2024-04-25T10:09:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa6ffd640 1 --2- 172.21.15.16:0/2480338284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f90004800 0x7f2f90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa7fff640 1 --2- 172.21.15.16:0/2480338284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f90005210 0x7f2f9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.795+0000 7f2fa77fe640 1 --2- 172.21.15.16:0/2480338284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f90003a00 0x7f2f90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.796+0000 7f2fa6ffd640 1 -- 172.21.15.16:0/2480338284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f90003a00 msgr2=0x7f2f90003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.796+0000 7f2fa6ffd640 1 --2- 172.21.15.16:0/2480338284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f90003a00 0x7f2f90003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.796+0000 7f2fa6ffd640 1 -- 172.21.15.16:0/2480338284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f90005210 msgr2=0x7f2f9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.796+0000 7f2fa6ffd640 1 --2- 172.21.15.16:0/2480338284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f90005210 0x7f2f9000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.796+0000 7f2fa6ffd640 1 -- 172.21.15.16:0/2480338284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f90003520 con 0x7f2f90004800 2024-04-25T10:09:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.796+0000 7f2fa7fff640 1 -- 172.21.15.16:0/2480338284 reap_dead start 2024-04-25T10:09:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.796+0000 7f2fa6ffd640 1 --2- 172.21.15.16:0/2480338284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f90004800 0x7f2f90004c80 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c009a90 tx=0x7f2f90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc79aea0bf8cb2a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.796+0000 7f2fa67fc640 1 -- 172.21.15.16:0/2480338284 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c036070 con 0x7f2f90004800 2024-04-25T10:09:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.796+0000 7f2fa67fc640 1 -- 172.21.15.16:0/2480338284 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9c031400 con 0x7f2f90004800 2024-04-25T10:09:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.796+0000 7f2fa67fc640 1 -- 172.21.15.16:0/2480338284 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c009690 con 0x7f2f90004800 2024-04-25T10:09:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.797+0000 7f2fadabd640 1 -- 172.21.15.16:0/2480338284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f90004800 msgr2=0x7f2f90004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.797+0000 7f2fadabd640 1 --2- 172.21.15.16:0/2480338284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f90004800 0x7f2f90004c80 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c009a90 tx=0x7f2f90005830 comp rx=0 tx=0).stop 2024-04-25T10:09:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.797+0000 7f2fadabd640 1 -- 172.21.15.16:0/2480338284 shutdown_connections 2024-04-25T10:09:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.797+0000 7f2fadabd640 1 --2- 172.21.15.16:0/2480338284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f90004800 0x7f2f90004c80 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.797+0000 7f2fadabd640 1 -- 172.21.15.16:0/2480338284 >> 172.21.15.16:0/2480338284 conn(0x7f2fa80d0550 msgr2=0x7f2fa80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.797+0000 7f2fadabd640 1 -- 172.21.15.16:0/2480338284 shutdown_connections 2024-04-25T10:09:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.797+0000 7f2fadabd640 1 -- 172.21.15.16:0/2480338284 wait complete. 2024-04-25T10:09:13.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.798+0000 7f2fadabd640 1 Processor -- start 2024-04-25T10:09:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.798+0000 7f2fadabd640 1 -- start start 2024-04-25T10:09:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.799+0000 7f2fadabd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f90004800 0x7f2fa81f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.799+0000 7f2fadabd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fa81532f0 0x7f2fa80cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.799+0000 7f2fadabd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fa80cc4b0 0x7f2fa80cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.799+0000 7f2fadabd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa80d1d60 con 0x7f2f90004800 2024-04-25T10:09:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.799+0000 7f2fadabd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa8162740 con 0x7f2fa80cc4b0 2024-04-25T10:09:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.799+0000 7f2fadabd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa80d2040 con 0x7f2fa81532f0 2024-04-25T10:09:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.799+0000 7f2fa77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f90004800 0x7f2fa81f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.799+0000 7f2fa77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f90004800 0x7f2fa81f43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39628/0 (socket says 172.21.15.16:39628) 2024-04-25T10:09:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.799+0000 7f2fa77fe640 1 -- 172.21.15.16:0/1933628636 learned_addr learned my addr 172.21.15.16:0/1933628636 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.799+0000 7f2fa6ffd640 1 --2- 172.21.15.16:0/1933628636 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fa81532f0 0x7f2fa80cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.799+0000 7f2fa7fff640 1 --2- 172.21.15.16:0/1933628636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fa80cc4b0 0x7f2fa80cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.800+0000 7f2fa77fe640 1 -- 172.21.15.16:0/1933628636 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fa81532f0 msgr2=0x7f2fa80cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.800+0000 7f2fa77fe640 1 --2- 172.21.15.16:0/1933628636 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fa81532f0 0x7f2fa80cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.800+0000 7f2fa77fe640 1 -- 172.21.15.16:0/1933628636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fa80cc4b0 msgr2=0x7f2fa80cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.800+0000 7f2fa77fe640 1 --2- 172.21.15.16:0/1933628636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fa80cc4b0 0x7f2fa80cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.800+0000 7f2fa77fe640 1 -- 172.21.15.16:0/1933628636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa80cbaa0 con 0x7f2f90004800 2024-04-25T10:09:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.800+0000 7f2fa77fe640 1 --2- 172.21.15.16:0/1933628636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f90004800 0x7f2fa81f43a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800f620 tx=0x7f2f9800fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.800+0000 7f2f8ffff640 1 -- 172.21.15.16:0/1933628636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f980052e0 con 0x7f2f90004800 2024-04-25T10:09:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.800+0000 7f2f8ffff640 1 -- 172.21.15.16:0/1933628636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f98018070 con 0x7f2f90004800 2024-04-25T10:09:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.800+0000 7f2fadabd640 1 -- 172.21.15.16:0/1933628636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa80cbd20 con 0x7f2f90004800 2024-04-25T10:09:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.800+0000 7f2f8ffff640 1 -- 172.21.15.16:0/1933628636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f98003810 con 0x7f2f90004800 2024-04-25T10:09:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.800+0000 7f2fadabd640 1 -- 172.21.15.16:0/1933628636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa80c7600 con 0x7f2f90004800 2024-04-25T10:09:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.801+0000 7f2fadabd640 1 -- 172.21.15.16:0/1933628636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa80750b0 con 0x7f2f90004800 2024-04-25T10:09:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.801+0000 7f2f8ffff640 1 -- 172.21.15.16:0/1933628636 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f980039b0 con 0x7f2f90004800 2024-04-25T10:09:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.802+0000 7f2f8ffff640 1 --2- 172.21.15.16:0/1933628636 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2f9406b500 0x7f2f9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.802+0000 7f2f8ffff640 1 -- 172.21.15.16:0/1933628636 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2f98089d80 con 0x7f2f90004800 2024-04-25T10:09:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.802+0000 7f2fa6ffd640 1 --2- 172.21.15.16:0/1933628636 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2f9406b500 0x7f2f9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.802+0000 7f2fa6ffd640 1 --2- 172.21.15.16:0/1933628636 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2f9406b500 0x7f2f9406d9c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c033000 tx=0x7f2f9c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.806+0000 7f2f8ffff640 1 -- 172.21.15.16:0/1933628636 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2fa80cbaa0 con 0x7f2f90004800 2024-04-25T10:09:13.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.985+0000 7f2fadabd640 1 -- 172.21.15.16:0/1933628636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2fa80d17e0 con 0x7f2f90004800 2024-04-25T10:09:13.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.988+0000 7f2f8ffff640 1 -- 172.21.15.16:0/1933628636 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2f98058de0 con 0x7f2f90004800 2024-04-25T10:09:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.990+0000 7f2fadabd640 1 -- 172.21.15.16:0/1933628636 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2f9406b500 msgr2=0x7f2f9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.990+0000 7f2fadabd640 1 --2- 172.21.15.16:0/1933628636 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2f9406b500 0x7f2f9406d9c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c033000 tx=0x7f2f9c008030 comp rx=0 tx=0).stop 2024-04-25T10:09:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.990+0000 7f2fadabd640 1 -- 172.21.15.16:0/1933628636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f90004800 msgr2=0x7f2fa81f43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.990+0000 7f2fadabd640 1 --2- 172.21.15.16:0/1933628636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f90004800 0x7f2fa81f43a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800f620 tx=0x7f2f9800fae0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.990+0000 7f2fadabd640 1 -- 172.21.15.16:0/1933628636 shutdown_connections 2024-04-25T10:09:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.990+0000 7f2fadabd640 1 --2- 172.21.15.16:0/1933628636 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2f9406b500 0x7f2f9406d9c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.990+0000 7f2fadabd640 1 --2- 172.21.15.16:0/1933628636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fa80cc4b0 0x7f2fa80cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.990+0000 7f2fadabd640 1 --2- 172.21.15.16:0/1933628636 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fa81532f0 0x7f2fa80cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.990+0000 7f2fadabd640 1 --2- 172.21.15.16:0/1933628636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f90004800 0x7f2fa81f43a0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.990+0000 7f2fadabd640 1 -- 172.21.15.16:0/1933628636 >> 172.21.15.16:0/1933628636 conn(0x7f2fa80d0550 msgr2=0x7f2fa80b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.990+0000 7f2fadabd640 1 -- 172.21.15.16:0/1933628636 shutdown_connections 2024-04-25T10:09:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:13.990+0000 7f2fadabd640 1 -- 172.21.15.16:0/1933628636 wait complete. 2024-04-25T10:09:14.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-25T10:09:14.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.101+0000 7f5ad28b6640 1 Processor -- start 2024-04-25T10:09:14.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.102+0000 7f5ad28b6640 1 -- start start 2024-04-25T10:09:14.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.102+0000 7f5ad28b6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5acc06cdd0 0x7f5acc0b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.102+0000 7f5ad28b6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5acc06c1f0 con 0x7f5acc15dc30 2024-04-25T10:09:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.102+0000 7f5ad28b6640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5acc0cfb40 con 0x7f5acc0b1960 2024-04-25T10:09:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.102+0000 7f5ad28b6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5acc0b1e10 con 0x7f5acc06cdd0 2024-04-25T10:09:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.102+0000 7f5acb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5acc06cdd0 0x7f5acc0b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.103+0000 7f5acbfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5acc15dc30 0x7f5acc06c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56002/0 (socket says 172.21.15.16:56002) 2024-04-25T10:09:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.103+0000 7f5acbfff640 1 -- 172.21.15.16:0/687010886 learned_addr learned my addr 172.21.15.16:0/687010886 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.103+0000 7f5acb7fe640 1 -- 172.21.15.16:0/687010886 >> v1:172.21.15.186:6789/0 conn(0x7f5acc0b1960 legacy=0x7f5acc0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.103+0000 7f5acb7fe640 1 -- 172.21.15.16:0/687010886 >> v1:172.21.15.16:6789/0 conn(0x7f5acc15dc30 legacy=0x7f5acc06c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.103+0000 7f5acb7fe640 1 -- 172.21.15.16:0/687010886 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc0b2060 con 0x7f5acc06cdd0 2024-04-25T10:09:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.103+0000 7f5acaffd640 1 -- 172.21.15.16:0/687010886 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3215435277 0 0) 0x7f5abc001110 con 0x7f5acc15dc30 2024-04-25T10:09:14.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.104+0000 7f5acb7fe640 1 --2- 172.21.15.16:0/687010886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5acc06cdd0 0x7f5acc0b1420 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0b1f90 tx=0x7f5ac002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d373de94ea67ff75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.104+0000 7f5acaffd640 1 -- 172.21.15.16:0/687010886 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0037070 con 0x7f5acc06cdd0 2024-04-25T10:09:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.104+0000 7f5acaffd640 1 -- 172.21.15.16:0/687010886 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac0032440 con 0x7f5acc06cdd0 2024-04-25T10:09:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.104+0000 7f5acaffd640 1 -- 172.21.15.16:0/687010886 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0032810 con 0x7f5acc06cdd0 2024-04-25T10:09:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.104+0000 7f5ad28b6640 1 -- 172.21.15.16:0/687010886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5acc06cdd0 msgr2=0x7f5acc0b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.104+0000 7f5ad28b6640 1 --2- 172.21.15.16:0/687010886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5acc06cdd0 0x7f5acc0b1420 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0b1f90 tx=0x7f5ac002a210 comp rx=0 tx=0).stop 2024-04-25T10:09:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.104+0000 7f5ad28b6640 1 -- 172.21.15.16:0/687010886 shutdown_connections 2024-04-25T10:09:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.104+0000 7f5ad28b6640 1 --2- 172.21.15.16:0/687010886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5acc06cdd0 0x7f5acc0b1420 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.104+0000 7f5ad28b6640 1 -- 172.21.15.16:0/687010886 >> 172.21.15.16:0/687010886 conn(0x7f5acc0d1900 msgr2=0x7f5acc0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.104+0000 7f5ad28b6640 1 -- 172.21.15.16:0/687010886 shutdown_connections 2024-04-25T10:09:14.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.104+0000 7f5ad28b6640 1 -- 172.21.15.16:0/687010886 wait complete. 2024-04-25T10:09:14.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.105+0000 7f5ad28b6640 1 Processor -- start 2024-04-25T10:09:14.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.106+0000 7f5ad28b6640 1 -- start start 2024-04-25T10:09:14.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.106+0000 7f5ad28b6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5acc06cdd0 0x7f5acc0af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:14.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.106+0000 7f5ad28b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5acc0b1960 0x7f5acc14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:14.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.106+0000 7f5ad28b6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5acc15dc30 0x7f5acc14ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:14.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.106+0000 7f5ad28b6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0cfde0 con 0x7f5acc0b1960 2024-04-25T10:09:14.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.106+0000 7f5ad28b6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0b2620 con 0x7f5acc15dc30 2024-04-25T10:09:14.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.106+0000 7f5ad28b6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc15e010 con 0x7f5acc06cdd0 2024-04-25T10:09:14.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.107+0000 7f5acb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5acc0b1960 0x7f5acc14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:14.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.107+0000 7f5acb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5acc0b1960 0x7f5acc14b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39640/0 (socket says 172.21.15.16:39640) 2024-04-25T10:09:14.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.107+0000 7f5acb7fe640 1 -- 172.21.15.16:0/2864514074 learned_addr learned my addr 172.21.15.16:0/2864514074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.107+0000 7f5ad0e53640 1 --2- 172.21.15.16:0/2864514074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5acc15dc30 0x7f5acc14ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.107+0000 7f5acb7fe640 1 -- 172.21.15.16:0/2864514074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5acc06cdd0 msgr2=0x7f5acc0af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.107+0000 7f5acb7fe640 1 --2- 172.21.15.16:0/2864514074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5acc06cdd0 0x7f5acc0af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.107+0000 7f5acb7fe640 1 -- 172.21.15.16:0/2864514074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5acc15dc30 msgr2=0x7f5acc14ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.107+0000 7f5acb7fe640 1 --2- 172.21.15.16:0/2864514074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5acc15dc30 0x7f5acc14ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.107+0000 7f5acb7fe640 1 -- 172.21.15.16:0/2864514074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc0afa50 con 0x7f5acc0b1960 2024-04-25T10:09:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.107+0000 7f5acb7fe640 1 --2- 172.21.15.16:0/2864514074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5acc0b1960 0x7f5acc14b4c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0000c00 tx=0x7f5ac00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.107+0000 7f5ac8ff9640 1 -- 172.21.15.16:0/2864514074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0037040 con 0x7f5acc0b1960 2024-04-25T10:09:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.107+0000 7f5ac8ff9640 1 -- 172.21.15.16:0/2864514074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac0042070 con 0x7f5acc0b1960 2024-04-25T10:09:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.107+0000 7f5ad28b6640 1 -- 172.21.15.16:0/2864514074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5acc14f800 con 0x7f5acc0b1960 2024-04-25T10:09:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.107+0000 7f5ac8ff9640 1 -- 172.21.15.16:0/2864514074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0030bd0 con 0x7f5acc0b1960 2024-04-25T10:09:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.107+0000 7f5ad28b6640 1 -- 172.21.15.16:0/2864514074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5acc0ca020 con 0x7f5acc0b1960 2024-04-25T10:09:14.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.108+0000 7f5ad28b6640 1 -- 172.21.15.16:0/2864514074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5acc067850 con 0x7f5acc0b1960 2024-04-25T10:09:14.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.109+0000 7f5ac8ff9640 1 -- 172.21.15.16:0/2864514074 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ac0047460 con 0x7f5acc0b1960 2024-04-25T10:09:14.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.109+0000 7f5ac8ff9640 1 --2- 172.21.15.16:0/2864514074 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5ab806b4d0 0x7f5ab806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:14.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.109+0000 7f5ac8ff9640 1 -- 172.21.15.16:0/2864514074 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5ac00aaca0 con 0x7f5acc0b1960 2024-04-25T10:09:14.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.110+0000 7f5acbfff640 1 --2- 172.21.15.16:0/2864514074 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5ab806b4d0 0x7f5ab806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:14.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.110+0000 7f5acbfff640 1 --2- 172.21.15.16:0/2864514074 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5ab806b4d0 0x7f5ab806d990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0013b0 tx=0x7f5abc001300 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:14.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.113+0000 7f5ac8ff9640 1 -- 172.21.15.16:0/2864514074 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5acc14f800 con 0x7f5acc0b1960 2024-04-25T10:09:14.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.362+0000 7f5ad28b6640 1 -- 172.21.15.16:0/2864514074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f5acc0cdf40 con 0x7f5acc0b1960 2024-04-25T10:09:14.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.674+0000 7f5ac8ff9640 1 -- 172.21.15.16:0/2864514074 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v215) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5ac0032e20 con 0x7f5acc0b1960 2024-04-25T10:09:14.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.676+0000 7f5ad28b6640 1 -- 172.21.15.16:0/2864514074 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5ab806b4d0 msgr2=0x7f5ab806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:14.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.676+0000 7f5ad28b6640 1 --2- 172.21.15.16:0/2864514074 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5ab806b4d0 0x7f5ab806d990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0013b0 tx=0x7f5abc001300 comp rx=0 tx=0).stop 2024-04-25T10:09:14.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.676+0000 7f5ad28b6640 1 -- 172.21.15.16:0/2864514074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5acc0b1960 msgr2=0x7f5acc14b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:14.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.676+0000 7f5ad28b6640 1 --2- 172.21.15.16:0/2864514074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5acc0b1960 0x7f5acc14b4c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0000c00 tx=0x7f5ac00046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:14.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.676+0000 7f5ad28b6640 1 -- 172.21.15.16:0/2864514074 shutdown_connections 2024-04-25T10:09:14.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.676+0000 7f5ad28b6640 1 --2- 172.21.15.16:0/2864514074 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5ab806b4d0 0x7f5ab806d990 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:14.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.676+0000 7f5ad28b6640 1 --2- 172.21.15.16:0/2864514074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5acc15dc30 0x7f5acc14ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:14.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.676+0000 7f5ad28b6640 1 --2- 172.21.15.16:0/2864514074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5acc0b1960 0x7f5acc14b4c0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:14.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.676+0000 7f5ad28b6640 1 --2- 172.21.15.16:0/2864514074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5acc06cdd0 0x7f5acc0af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:14.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.676+0000 7f5ad28b6640 1 -- 172.21.15.16:0/2864514074 >> 172.21.15.16:0/2864514074 conn(0x7f5acc0d1900 msgr2=0x7f5acc0705f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:14.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.677+0000 7f5ad28b6640 1 -- 172.21.15.16:0/2864514074 shutdown_connections 2024-04-25T10:09:14.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.677+0000 7f5ad28b6640 1 -- 172.21.15.16:0/2864514074 wait complete. 2024-04-25T10:09:14.695 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T10:09:14.696 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T10:09:14.696 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:09:14.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.791+0000 7fadb6a38640 1 Processor -- start 2024-04-25T10:09:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.791+0000 7fadb6a38640 1 -- start start 2024-04-25T10:09:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.792+0000 7fadb6a38640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fadb015dce0 0x7fadb00d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.792+0000 7fadb6a38640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fadb00b5b60 0x7fadb00b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.792+0000 7fadb6a38640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb015e0c0 con 0x7fadb00d35b0 2024-04-25T10:09:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.792+0000 7fadb6a38640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb00d2510 con 0x7fadb015dce0 2024-04-25T10:09:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.792+0000 7fadb6a38640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb00d27f0 con 0x7fadb00b5b60 2024-04-25T10:09:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.792+0000 7fadaffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fadb015dce0 0x7fadb00d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.792+0000 7fadaf7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fadb00d35b0 0x7fadb00b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56010/0 (socket says 172.21.15.16:56010) 2024-04-25T10:09:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.792+0000 7fadaf7fe640 1 -- 172.21.15.16:0/4010386721 learned_addr learned my addr 172.21.15.16:0/4010386721 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.792+0000 7fadb4fd5640 1 --2- 172.21.15.16:0/4010386721 >> v2:172.21.15.186:3300/0 conn(0x7fadb00b5b60 0x7fadb00b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.792+0000 7fadaffff640 1 -- 172.21.15.16:0/4010386721 >> v2:172.21.15.186:3300/0 conn(0x7fadb00b5b60 msgr2=0x7fadb00b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.792+0000 7fadaffff640 1 --2- 172.21.15.16:0/4010386721 >> v2:172.21.15.186:3300/0 conn(0x7fadb00b5b60 0x7fadb00b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.792+0000 7fadaffff640 1 -- 172.21.15.16:0/4010386721 >> v1:172.21.15.16:6789/0 conn(0x7fadb00d35b0 legacy=0x7fadb00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.792+0000 7fadaffff640 1 -- 172.21.15.16:0/4010386721 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb00d2ad0 con 0x7fadb015dce0 2024-04-25T10:09:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.792+0000 7fadaeffd640 1 -- 172.21.15.16:0/4010386721 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889525912 0 0) 0x7fadb015e0c0 con 0x7fadb00d35b0 2024-04-25T10:09:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.793+0000 7fadaffff640 1 --2- 172.21.15.16:0/4010386721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadb015dce0 0x7fadb00d1a50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fadb00b3390 tx=0x7fada002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce6154d176a36cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.793+0000 7fadaeffd640 1 -- 172.21.15.16:0/4010386721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0037070 con 0x7fadb015dce0 2024-04-25T10:09:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.793+0000 7fadaeffd640 1 -- 172.21.15.16:0/4010386721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada0032440 con 0x7fadb015dce0 2024-04-25T10:09:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.793+0000 7fadaeffd640 1 -- 172.21.15.16:0/4010386721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0032810 con 0x7fadb015dce0 2024-04-25T10:09:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.794+0000 7fadb6a38640 1 -- 172.21.15.16:0/4010386721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadb015dce0 msgr2=0x7fadb00d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.794+0000 7fadb6a38640 1 --2- 172.21.15.16:0/4010386721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadb015dce0 0x7fadb00d1a50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fadb00b3390 tx=0x7fada002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:09:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.794+0000 7fadb6a38640 1 -- 172.21.15.16:0/4010386721 shutdown_connections 2024-04-25T10:09:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.794+0000 7fadb6a38640 1 --2- 172.21.15.16:0/4010386721 >> v2:172.21.15.186:3300/0 conn(0x7fadb00b5b60 0x7fadb00b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.794+0000 7fadb6a38640 1 --2- 172.21.15.16:0/4010386721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadb015dce0 0x7fadb00d1a50 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.794+0000 7fadb6a38640 1 -- 172.21.15.16:0/4010386721 >> 172.21.15.16:0/4010386721 conn(0x7fadb0072900 msgr2=0x7fadb00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.794+0000 7fadb6a38640 1 -- 172.21.15.16:0/4010386721 shutdown_connections 2024-04-25T10:09:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.794+0000 7fadb6a38640 1 -- 172.21.15.16:0/4010386721 wait complete. 2024-04-25T10:09:14.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.795+0000 7fadb6a38640 1 Processor -- start 2024-04-25T10:09:14.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.795+0000 7fadb6a38640 1 -- start start 2024-04-25T10:09:14.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadb6a38640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadb00b5b60 0x7fadb01467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadb6a38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadb00d35b0 0x7fadb00ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadb6a38640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb015dce0 0x7fadb01f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadb6a38640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb00b9820 con 0x7fadb00d35b0 2024-04-25T10:09:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadb6a38640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb00d3090 con 0x7fadb015dce0 2024-04-25T10:09:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadb6a38640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb00b99a0 con 0x7fadb00b5b60 2024-04-25T10:09:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadb00d35b0 0x7fadb00ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadb00d35b0 0x7fadb00ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39654/0 (socket says 172.21.15.16:39654) 2024-04-25T10:09:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadaf7fe640 1 -- 172.21.15.16:0/3125549078 learned_addr learned my addr 172.21.15.16:0/3125549078 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadb4fd5640 1 --2- 172.21.15.16:0/3125549078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb015dce0 0x7fadb01f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadaffff640 1 --2- 172.21.15.16:0/3125549078 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadb00b5b60 0x7fadb01467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadaf7fe640 1 -- 172.21.15.16:0/3125549078 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadb00b5b60 msgr2=0x7fadb01467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadaf7fe640 1 --2- 172.21.15.16:0/3125549078 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadb00b5b60 0x7fadb01467c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadaf7fe640 1 -- 172.21.15.16:0/3125549078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb015dce0 msgr2=0x7fadb01f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadaf7fe640 1 --2- 172.21.15.16:0/3125549078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb015dce0 0x7fadb01f34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadaf7fe640 1 -- 172.21.15.16:0/3125549078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb01f3ec0 con 0x7fadb00d35b0 2024-04-25T10:09:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadaffff640 1 --2- 172.21.15.16:0/3125549078 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadb00b5b60 0x7fadb01467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadb4fd5640 1 --2- 172.21.15.16:0/3125549078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb015dce0 0x7fadb01f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadaf7fe640 1 --2- 172.21.15.16:0/3125549078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadb00d35b0 0x7fadb00ccd20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fad9800c4b0 tx=0x7fad9800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.796+0000 7fadacff9640 1 -- 172.21.15.16:0/3125549078 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad98018070 con 0x7fadb00d35b0 2024-04-25T10:09:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.797+0000 7fadacff9640 1 -- 172.21.15.16:0/3125549078 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad98007500 con 0x7fadb00d35b0 2024-04-25T10:09:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.797+0000 7fadb6a38640 1 -- 172.21.15.16:0/3125549078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb01f41a0 con 0x7fadb00d35b0 2024-04-25T10:09:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.797+0000 7fadacff9640 1 -- 172.21.15.16:0/3125549078 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad98004a20 con 0x7fadb00d35b0 2024-04-25T10:09:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.797+0000 7fadb6a38640 1 -- 172.21.15.16:0/3125549078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb01462a0 con 0x7fadb00d35b0 2024-04-25T10:09:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.798+0000 7fadb6a38640 1 -- 172.21.15.16:0/3125549078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb00749b0 con 0x7fadb00d35b0 2024-04-25T10:09:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.798+0000 7fadacff9640 1 -- 172.21.15.16:0/3125549078 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad9801b070 con 0x7fadb00d35b0 2024-04-25T10:09:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.798+0000 7fadacff9640 1 --2- 172.21.15.16:0/3125549078 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fad9006b500 0x7fad9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.798+0000 7fadacff9640 1 -- 172.21.15.16:0/3125549078 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fad98089170 con 0x7fadb00d35b0 2024-04-25T10:09:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.799+0000 7fadaffff640 1 --2- 172.21.15.16:0/3125549078 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fad9006b500 0x7fad9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.799+0000 7fadaffff640 1 --2- 172.21.15.16:0/3125549078 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fad9006b500 0x7fad9006d9c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fada0034000 tx=0x7fada002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.803+0000 7fadacff9640 1 -- 172.21.15.16:0/3125549078 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad98058440 con 0x7fadb00d35b0 2024-04-25T10:09:14.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.986+0000 7fadb6a38640 1 -- 172.21.15.16:0/3125549078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadb00d08d0 con 0x7fadb00d35b0 2024-04-25T10:09:14.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.987+0000 7fadacff9640 1 -- 172.21.15.16:0/3125549078 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+13020 (secure 0 0 0) 0x7fad98057d40 con 0x7fadb00d35b0 2024-04-25T10:09:14.994 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:14.994 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":64,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:09:13.661285+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6834/3311584521":"2024-04-26T10:09:10.557638+0000","172.21.15.16:6832/3311584521":"2024-04-26T10:09:10.557638+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.989+0000 7fadb6a38640 1 -- 172.21.15.16:0/3125549078 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fad9006b500 msgr2=0x7fad9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.989+0000 7fadb6a38640 1 --2- 172.21.15.16:0/3125549078 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fad9006b500 0x7fad9006d9c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fada0034000 tx=0x7fada002e000 comp rx=0 tx=0).stop 2024-04-25T10:09:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.989+0000 7fadb6a38640 1 -- 172.21.15.16:0/3125549078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadb00d35b0 msgr2=0x7fadb00ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.989+0000 7fadb6a38640 1 --2- 172.21.15.16:0/3125549078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadb00d35b0 0x7fadb00ccd20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fad9800c4b0 tx=0x7fad9800c970 comp rx=0 tx=0).stop 2024-04-25T10:09:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.989+0000 7fadb6a38640 1 -- 172.21.15.16:0/3125549078 shutdown_connections 2024-04-25T10:09:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.989+0000 7fadb6a38640 1 --2- 172.21.15.16:0/3125549078 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fad9006b500 0x7fad9006d9c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.989+0000 7fadb6a38640 1 --2- 172.21.15.16:0/3125549078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb015dce0 0x7fadb01f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.989+0000 7fadb6a38640 1 --2- 172.21.15.16:0/3125549078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadb00d35b0 0x7fadb00ccd20 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.989+0000 7fadb6a38640 1 --2- 172.21.15.16:0/3125549078 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadb00b5b60 0x7fadb01467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.989+0000 7fadb6a38640 1 -- 172.21.15.16:0/3125549078 >> 172.21.15.16:0/3125549078 conn(0x7fadb0072900 msgr2=0x7fadb00b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.989+0000 7fadb6a38640 1 -- 172.21.15.16:0/3125549078 shutdown_connections 2024-04-25T10:09:15.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:14.989+0000 7fadb6a38640 1 -- 172.21.15.16:0/3125549078 wait complete. 2024-04-25T10:09:15.007 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:15.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.104+0000 7f39a3246640 1 Processor -- start 2024-04-25T10:09:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.104+0000 7f39a3246640 1 -- start start 2024-04-25T10:09:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.104+0000 7f39a3246640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f399c0b97c0 0x7f399c0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.104+0000 7f39a3246640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f399c0b5a30 0x7f399c0d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.104+0000 7f39a3246640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399c1627a0 con 0x7f399c15dc80 2024-04-25T10:09:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.104+0000 7f39a3246640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f399c161f80 con 0x7f399c0b97c0 2024-04-25T10:09:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.104+0000 7f39a3246640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0b5e60 con 0x7f399c0b5a30 2024-04-25T10:09:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f39a0fe2640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f399c15dc80 0x7f399c161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56022/0 (socket says 172.21.15.16:56022) 2024-04-25T10:09:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f39a0fe2640 1 -- 172.21.15.16:0/3746876278 learned_addr learned my addr 172.21.15.16:0/3746876278 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399bfff640 1 --2- 172.21.15.16:0/3746876278 >> v2:172.21.15.186:3300/0 conn(0x7f399c0b97c0 0x7f399c0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f39a17e3640 1 --2- 172.21.15.16:0/3746876278 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f399c0b5a30 0x7f399c0d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399b7fe640 1 -- 172.21.15.16:0/3746876278 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1484185491 0 0) 0x7f399c1627a0 con 0x7f399c15dc80 2024-04-25T10:09:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399b7fe640 1 -- 172.21.15.16:0/3746876278 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f398c003520 con 0x7f399c15dc80 2024-04-25T10:09:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399b7fe640 1 -- 172.21.15.16:0/3746876278 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2648648118 0 0) 0x7f398c003520 con 0x7f399c15dc80 2024-04-25T10:09:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399b7fe640 1 -- 172.21.15.16:0/3746876278 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f399c0b5a30 msgr2=0x7f399c0d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399b7fe640 1 --2- 172.21.15.16:0/3746876278 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f399c0b5a30 0x7f399c0d2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399b7fe640 1 -- 172.21.15.16:0/3746876278 >> v2:172.21.15.186:3300/0 conn(0x7f399c0b97c0 msgr2=0x7f399c0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399b7fe640 1 --2- 172.21.15.16:0/3746876278 >> v2:172.21.15.186:3300/0 conn(0x7f399c0b97c0 0x7f399c0b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399b7fe640 1 -- 172.21.15.16:0/3746876278 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c1621e0 con 0x7f399c15dc80 2024-04-25T10:09:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399b7fe640 1 -- 172.21.15.16:0/3746876278 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f3990001670 con 0x7f399c15dc80 2024-04-25T10:09:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399bfff640 1 --2- 172.21.15.16:0/3746876278 >> v2:172.21.15.186:3300/0 conn(0x7f399c0b97c0 0x7f399c0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399b7fe640 1 -- 172.21.15.16:0/3746876278 >> v1:172.21.15.16:6789/0 conn(0x7f399c15dc80 legacy=0x7f399c161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399b7fe640 1 --2- 172.21.15.16:0/3746876278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f398c0039e0 0x7f398c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399b7fe640 1 --2- 172.21.15.16:0/3746876278 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f399c0b5a30 0x7f398c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399b7fe640 1 --2- 172.21.15.16:0/3746876278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f398c004d40 0x7f398c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399b7fe640 1 -- 172.21.15.16:0/3746876278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c161f80 con 0x7f398c0039e0 2024-04-25T10:09:15.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399b7fe640 1 -- 172.21.15.16:0/3746876278 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0b5e60 con 0x7f398c004d40 2024-04-25T10:09:15.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399b7fe640 1 -- 172.21.15.16:0/3746876278 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c00b9c0 con 0x7f399c0b5a30 2024-04-25T10:09:15.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f39a0fe2640 1 --2- 172.21.15.16:0/3746876278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f398c0039e0 0x7f398c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f399bfff640 1 --2- 172.21.15.16:0/3746876278 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f399c0b5a30 0x7f398c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.105+0000 7f39a17e3640 1 --2- 172.21.15.16:0/3746876278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f398c004d40 0x7f398c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.106+0000 7f39a0fe2640 1 -- 172.21.15.16:0/3746876278 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f399c0b5a30 msgr2=0x7f398c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:15.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.106+0000 7f39a0fe2640 1 --2- 172.21.15.16:0/3746876278 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f399c0b5a30 0x7f398c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.106+0000 7f39a0fe2640 1 -- 172.21.15.16:0/3746876278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f398c004d40 msgr2=0x7f398c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:15.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.106+0000 7f39a0fe2640 1 --2- 172.21.15.16:0/3746876278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f398c004d40 0x7f398c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.106+0000 7f39a0fe2640 1 -- 172.21.15.16:0/3746876278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c003520 con 0x7f398c0039e0 2024-04-25T10:09:15.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.106+0000 7f39a17e3640 1 --2- 172.21.15.16:0/3746876278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f398c004d40 0x7f398c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:15.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.106+0000 7f39a0fe2640 1 --2- 172.21.15.16:0/3746876278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f398c0039e0 0x7f398c003e00 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f399000c270 tx=0x7f398c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5880b487734701 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:15.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.106+0000 7f399b7fe640 1 -- 172.21.15.16:0/3746876278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990007d10 con 0x7f398c0039e0 2024-04-25T10:09:15.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.106+0000 7f399b7fe640 1 -- 172.21.15.16:0/3746876278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3990005ad0 con 0x7f398c0039e0 2024-04-25T10:09:15.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.106+0000 7f399b7fe640 1 -- 172.21.15.16:0/3746876278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990007070 con 0x7f398c0039e0 2024-04-25T10:09:15.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.107+0000 7f39a3246640 1 -- 172.21.15.16:0/3746876278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f398c0039e0 msgr2=0x7f398c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.107+0000 7f39a3246640 1 --2- 172.21.15.16:0/3746876278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f398c0039e0 0x7f398c003e00 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f399000c270 tx=0x7f398c005830 comp rx=0 tx=0).stop 2024-04-25T10:09:15.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.107+0000 7f39a17e3640 1 -- 172.21.15.16:0/3746876278 reap_dead start 2024-04-25T10:09:15.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.107+0000 7f39a3246640 1 -- 172.21.15.16:0/3746876278 shutdown_connections 2024-04-25T10:09:15.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.107+0000 7f39a3246640 1 -- 172.21.15.16:0/3746876278 >> 172.21.15.16:0/3746876278 conn(0x7f399c0d04f0 msgr2=0x7f399c0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:15.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.107+0000 7f39a3246640 1 -- 172.21.15.16:0/3746876278 shutdown_connections 2024-04-25T10:09:15.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.107+0000 7f39a3246640 1 -- 172.21.15.16:0/3746876278 wait complete. 2024-04-25T10:09:15.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.108+0000 7f39a3246640 1 Processor -- start 2024-04-25T10:09:15.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.108+0000 7f39a3246640 1 -- start start 2024-04-25T10:09:15.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.109+0000 7f39a3246640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f398c0039e0 0x7f399c157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.109+0000 7f39a3246640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f399c0b44d0 0x7f399c156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.109+0000 7f39a3246640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f399c15e480 0x7f399c15e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.109+0000 7f39a3246640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c15e060 con 0x7f399c15e480 2024-04-25T10:09:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.109+0000 7f39a3246640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0d1c70 con 0x7f398c0039e0 2024-04-25T10:09:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.109+0000 7f39a3246640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0d2470 con 0x7f399c0b44d0 2024-04-25T10:09:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.109+0000 7f39a17e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f399c15e480 0x7f399c15e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.109+0000 7f39a17e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f399c15e480 0x7f399c15e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39678/0 (socket says 172.21.15.16:39678) 2024-04-25T10:09:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.109+0000 7f39a17e3640 1 -- 172.21.15.16:0/3617741965 learned_addr learned my addr 172.21.15.16:0/3617741965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.109+0000 7f39a0fe2640 1 --2- 172.21.15.16:0/3617741965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f398c0039e0 0x7f399c157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.109+0000 7f399bfff640 1 --2- 172.21.15.16:0/3617741965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f399c0b44d0 0x7f399c156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.109+0000 7f39a17e3640 1 -- 172.21.15.16:0/3617741965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f399c0b44d0 msgr2=0x7f399c156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.109+0000 7f39a17e3640 1 --2- 172.21.15.16:0/3617741965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f399c0b44d0 0x7f399c156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.109+0000 7f39a17e3640 1 -- 172.21.15.16:0/3617741965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f398c0039e0 msgr2=0x7f399c157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.109+0000 7f39a17e3640 1 --2- 172.21.15.16:0/3617741965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f398c0039e0 0x7f399c157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.109+0000 7f39a17e3640 1 -- 172.21.15.16:0/3617741965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c157750 con 0x7f399c15e480 2024-04-25T10:09:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.110+0000 7f39a17e3640 1 --2- 172.21.15.16:0/3617741965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f399c15e480 0x7f399c15e930 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f399c0d15f0 tx=0x7f39940079f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.110+0000 7f39997fa640 1 -- 172.21.15.16:0/3617741965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994003990 con 0x7f399c15e480 2024-04-25T10:09:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.110+0000 7f39997fa640 1 -- 172.21.15.16:0/3617741965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3994003b50 con 0x7f399c15e480 2024-04-25T10:09:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.110+0000 7f39a3246640 1 -- 172.21.15.16:0/3617741965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399c15f1c0 con 0x7f399c15e480 2024-04-25T10:09:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.110+0000 7f39997fa640 1 -- 172.21.15.16:0/3617741965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399401d690 con 0x7f399c15e480 2024-04-25T10:09:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.110+0000 7f39a3246640 1 -- 172.21.15.16:0/3617741965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399c157350 con 0x7f399c15e480 2024-04-25T10:09:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.111+0000 7f39a3246640 1 -- 172.21.15.16:0/3617741965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399c0750b0 con 0x7f399c15e480 2024-04-25T10:09:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.111+0000 7f39997fa640 1 -- 172.21.15.16:0/3617741965 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3994005d10 con 0x7f399c15e480 2024-04-25T10:09:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.112+0000 7f39997fa640 1 --2- 172.21.15.16:0/3617741965 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f398006b500 0x7f398006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.112+0000 7f39997fa640 1 -- 172.21.15.16:0/3617741965 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3994092cb0 con 0x7f399c15e480 2024-04-25T10:09:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.112+0000 7f39a0fe2640 1 --2- 172.21.15.16:0/3617741965 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f398006b500 0x7f398006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.112+0000 7f39a0fe2640 1 --2- 172.21.15.16:0/3617741965 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f398006b500 0x7f398006d9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3990033000 tx=0x7f3990009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.116+0000 7f39997fa640 1 -- 172.21.15.16:0/3617741965 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f399402d020 con 0x7f399c15e480 2024-04-25T10:09:15.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.299+0000 7f39a3246640 1 -- 172.21.15.16:0/3617741965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f399c0d1780 con 0x7f399c15e480 2024-04-25T10:09:15.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.299+0000 7f39997fa640 1 -- 172.21.15.16:0/3617741965 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7f399401a040 con 0x7f399c15e480 2024-04-25T10:09:15.306 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:09:15.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.301+0000 7f39a3246640 1 -- 172.21.15.16:0/3617741965 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f398006b500 msgr2=0x7f398006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.301+0000 7f39a3246640 1 --2- 172.21.15.16:0/3617741965 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f398006b500 0x7f398006d9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3990033000 tx=0x7f3990009030 comp rx=0 tx=0).stop 2024-04-25T10:09:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.301+0000 7f39a3246640 1 -- 172.21.15.16:0/3617741965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f399c15e480 msgr2=0x7f399c15e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.301+0000 7f39a3246640 1 --2- 172.21.15.16:0/3617741965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f399c15e480 0x7f399c15e930 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f399c0d15f0 tx=0x7f39940079f0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.301+0000 7f39a3246640 1 -- 172.21.15.16:0/3617741965 shutdown_connections 2024-04-25T10:09:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.301+0000 7f39a3246640 1 --2- 172.21.15.16:0/3617741965 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f398006b500 0x7f398006d9c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.301+0000 7f39a3246640 1 --2- 172.21.15.16:0/3617741965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f399c15e480 0x7f399c15e930 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.301+0000 7f39a3246640 1 --2- 172.21.15.16:0/3617741965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f399c0b44d0 0x7f399c156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.301+0000 7f39a3246640 1 --2- 172.21.15.16:0/3617741965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f398c0039e0 0x7f399c157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.301+0000 7f39a3246640 1 -- 172.21.15.16:0/3617741965 >> 172.21.15.16:0/3617741965 conn(0x7f399c0d04f0 msgr2=0x7f399c06bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.301+0000 7f39a3246640 1 -- 172.21.15.16:0/3617741965 shutdown_connections 2024-04-25T10:09:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.301+0000 7f39a3246640 1 -- 172.21.15.16:0/3617741965 wait complete. 2024-04-25T10:09:15.320 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-25T10:09:15.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.414+0000 7fa5e0fce640 1 Processor -- start 2024-04-25T10:09:15.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.414+0000 7fa5e0fce640 1 -- start start 2024-04-25T10:09:15.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.415+0000 7fa5e0fce640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa5dc0d35b0 0x7fa5dc0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.415+0000 7fa5e0fce640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa5dc0b5990 0x7fa5dc0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.415+0000 7fa5e0fce640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5dc15e0c0 con 0x7fa5dc15dce0 2024-04-25T10:09:15.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.415+0000 7fa5e0fce640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5dc0d2510 con 0x7fa5dc0d35b0 2024-04-25T10:09:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.415+0000 7fa5e0fce640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5dc0d27f0 con 0x7fa5dc0b5990 2024-04-25T10:09:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.415+0000 7fa5d9d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa5dc0d35b0 0x7fa5dc0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.415+0000 7fa5da59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa5dc15dce0 0x7fa5dc0d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56024/0 (socket says 172.21.15.16:56024) 2024-04-25T10:09:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.415+0000 7fa5da59c640 1 -- 172.21.15.16:0/2087579621 learned_addr learned my addr 172.21.15.16:0/2087579621 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.415+0000 7fa5dad9d640 1 --2- 172.21.15.16:0/2087579621 >> v2:172.21.15.186:3300/0 conn(0x7fa5dc0b5990 0x7fa5dc0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.415+0000 7fa5d9d9b640 1 -- 172.21.15.16:0/2087579621 >> v2:172.21.15.186:3300/0 conn(0x7fa5dc0b5990 msgr2=0x7fa5dc0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.415+0000 7fa5d9d9b640 1 --2- 172.21.15.16:0/2087579621 >> v2:172.21.15.186:3300/0 conn(0x7fa5dc0b5990 0x7fa5dc0b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.415+0000 7fa5d9d9b640 1 -- 172.21.15.16:0/2087579621 >> v1:172.21.15.16:6789/0 conn(0x7fa5dc15dce0 legacy=0x7fa5dc0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.415+0000 7fa5d9d9b640 1 -- 172.21.15.16:0/2087579621 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc0d2ad0 con 0x7fa5dc0d35b0 2024-04-25T10:09:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.416+0000 7fa5d9d9b640 1 --2- 172.21.15.16:0/2087579621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc0d35b0 0x7fa5dc0b5450 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0b3390 tx=0x7fa5d002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a09610c138dd76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.416+0000 7fa5d959a640 1 -- 172.21.15.16:0/2087579621 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0037070 con 0x7fa5dc0d35b0 2024-04-25T10:09:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.416+0000 7fa5d959a640 1 -- 172.21.15.16:0/2087579621 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d0032440 con 0x7fa5dc0d35b0 2024-04-25T10:09:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.416+0000 7fa5d959a640 1 -- 172.21.15.16:0/2087579621 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0032810 con 0x7fa5dc0d35b0 2024-04-25T10:09:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.416+0000 7fa5e0fce640 1 -- 172.21.15.16:0/2087579621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc0d35b0 msgr2=0x7fa5dc0b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.416+0000 7fa5e0fce640 1 --2- 172.21.15.16:0/2087579621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc0d35b0 0x7fa5dc0b5450 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0b3390 tx=0x7fa5d002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.417+0000 7fa5e0fce640 1 -- 172.21.15.16:0/2087579621 shutdown_connections 2024-04-25T10:09:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.417+0000 7fa5e0fce640 1 --2- 172.21.15.16:0/2087579621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc0d35b0 0x7fa5dc0b5450 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.417+0000 7fa5e0fce640 1 --2- 172.21.15.16:0/2087579621 >> v2:172.21.15.186:3300/0 conn(0x7fa5dc0b5990 0x7fa5dc0b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.417+0000 7fa5e0fce640 1 -- 172.21.15.16:0/2087579621 >> 172.21.15.16:0/2087579621 conn(0x7fa5dc072900 msgr2=0x7fa5dc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.417+0000 7fa5e0fce640 1 -- 172.21.15.16:0/2087579621 shutdown_connections 2024-04-25T10:09:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.417+0000 7fa5e0fce640 1 -- 172.21.15.16:0/2087579621 wait complete. 2024-04-25T10:09:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.418+0000 7fa5e0fce640 1 Processor -- start 2024-04-25T10:09:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.418+0000 7fa5e0fce640 1 -- start start 2024-04-25T10:09:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.419+0000 7fa5e0fce640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5dc0b5990 0x7fa5dc1f8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.419+0000 7fa5e0fce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc0d35b0 0x7fa5dc156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.419+0000 7fa5e0fce640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5dc15dce0 0x7fa5dc0b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.419+0000 7fa5e0fce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc0d2160 con 0x7fa5dc0d35b0 2024-04-25T10:09:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.419+0000 7fa5e0fce640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc0d3090 con 0x7fa5dc0b5990 2024-04-25T10:09:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.419+0000 7fa5e0fce640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc1f8590 con 0x7fa5dc15dce0 2024-04-25T10:09:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.419+0000 7fa5d9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc0d35b0 0x7fa5dc156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.419+0000 7fa5d9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc0d35b0 0x7fa5dc156a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39696/0 (socket says 172.21.15.16:39696) 2024-04-25T10:09:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.419+0000 7fa5d9d9b640 1 -- 172.21.15.16:0/3912354027 learned_addr learned my addr 172.21.15.16:0/3912354027 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.419+0000 7fa5dad9d640 1 --2- 172.21.15.16:0/3912354027 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5dc15dce0 0x7fa5dc0b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.419+0000 7fa5da59c640 1 --2- 172.21.15.16:0/3912354027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5dc0b5990 0x7fa5dc1f8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.419+0000 7fa5d9d9b640 1 -- 172.21.15.16:0/3912354027 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5dc15dce0 msgr2=0x7fa5dc0b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.419+0000 7fa5d9d9b640 1 --2- 172.21.15.16:0/3912354027 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5dc15dce0 0x7fa5dc0b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.419+0000 7fa5d9d9b640 1 -- 172.21.15.16:0/3912354027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5dc0b5990 msgr2=0x7fa5dc1f8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.419+0000 7fa5d9d9b640 1 --2- 172.21.15.16:0/3912354027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5dc0b5990 0x7fa5dc1f8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.419+0000 7fa5d9d9b640 1 -- 172.21.15.16:0/3912354027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc0c7630 con 0x7fa5dc0d35b0 2024-04-25T10:09:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.419+0000 7fa5dad9d640 1 --2- 172.21.15.16:0/3912354027 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5dc15dce0 0x7fa5dc0b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.419+0000 7fa5d9d9b640 1 --2- 172.21.15.16:0/3912354027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc0d35b0 0x7fa5dc156a50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009890 tx=0x7fa5d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.420+0000 7fa5caffd640 1 -- 172.21.15.16:0/3912354027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0037040 con 0x7fa5dc0d35b0 2024-04-25T10:09:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.420+0000 7fa5caffd640 1 -- 172.21.15.16:0/3912354027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d0042070 con 0x7fa5dc0d35b0 2024-04-25T10:09:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.420+0000 7fa5e0fce640 1 -- 172.21.15.16:0/3912354027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5dc0c7910 con 0x7fa5dc0d35b0 2024-04-25T10:09:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.420+0000 7fa5caffd640 1 -- 172.21.15.16:0/3912354027 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0030bd0 con 0x7fa5dc0d35b0 2024-04-25T10:09:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.420+0000 7fa5e0fce640 1 -- 172.21.15.16:0/3912354027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5dc160cf0 con 0x7fa5dc0d35b0 2024-04-25T10:09:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.421+0000 7fa5e0fce640 1 -- 172.21.15.16:0/3912354027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5dc0749b0 con 0x7fa5dc0d35b0 2024-04-25T10:09:15.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.421+0000 7fa5caffd640 1 -- 172.21.15.16:0/3912354027 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa5d0047460 con 0x7fa5dc0d35b0 2024-04-25T10:09:15.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.421+0000 7fa5caffd640 1 --2- 172.21.15.16:0/3912354027 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa5bc06b4d0 0x7fa5bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.421+0000 7fa5caffd640 1 -- 172.21.15.16:0/3912354027 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa5d00aa960 con 0x7fa5dc0d35b0 2024-04-25T10:09:15.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.422+0000 7fa5da59c640 1 --2- 172.21.15.16:0/3912354027 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa5bc06b4d0 0x7fa5bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.422+0000 7fa5da59c640 1 --2- 172.21.15.16:0/3912354027 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa5bc06b4d0 0x7fa5bc06d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc007fd0 tx=0x7fa5cc007e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:15.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.426+0000 7fa5caffd640 1 -- 172.21.15.16:0/3912354027 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa5d0032e20 con 0x7fa5dc0d35b0 2024-04-25T10:09:15.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:15.680+0000 7fa5e0fce640 1 -- 172.21.15.16:0/3912354027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7fa5dc0c72a0 con 0x7fa5dc0d35b0 2024-04-25T10:09:16.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.679+0000 7fa5caffd640 1 -- 172.21.15.16:0/3912354027 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v217) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa5d0079b70 con 0x7fa5dc0d35b0 2024-04-25T10:09:16.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.682+0000 7fa5e0fce640 1 -- 172.21.15.16:0/3912354027 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa5bc06b4d0 msgr2=0x7fa5bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:16.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.682+0000 7fa5e0fce640 1 --2- 172.21.15.16:0/3912354027 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa5bc06b4d0 0x7fa5bc06d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc007fd0 tx=0x7fa5cc007e80 comp rx=0 tx=0).stop 2024-04-25T10:09:16.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.682+0000 7fa5e0fce640 1 -- 172.21.15.16:0/3912354027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc0d35b0 msgr2=0x7fa5dc156a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:16.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.682+0000 7fa5e0fce640 1 --2- 172.21.15.16:0/3912354027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc0d35b0 0x7fa5dc156a50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009890 tx=0x7fa5d00046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:16.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.682+0000 7fa5e0fce640 1 -- 172.21.15.16:0/3912354027 shutdown_connections 2024-04-25T10:09:16.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.682+0000 7fa5e0fce640 1 --2- 172.21.15.16:0/3912354027 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa5bc06b4d0 0x7fa5bc06d990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:16.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.682+0000 7fa5e0fce640 1 --2- 172.21.15.16:0/3912354027 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5dc15dce0 0x7fa5dc0b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:16.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.682+0000 7fa5e0fce640 1 --2- 172.21.15.16:0/3912354027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc0d35b0 0x7fa5dc156a50 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:16.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.682+0000 7fa5e0fce640 1 --2- 172.21.15.16:0/3912354027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5dc0b5990 0x7fa5dc1f8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:16.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.682+0000 7fa5e0fce640 1 -- 172.21.15.16:0/3912354027 >> 172.21.15.16:0/3912354027 conn(0x7fa5dc072900 msgr2=0x7fa5dc0d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:16.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.682+0000 7fa5e0fce640 1 -- 172.21.15.16:0/3912354027 shutdown_connections 2024-04-25T10:09:16.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.682+0000 7fa5e0fce640 1 -- 172.21.15.16:0/3912354027 wait complete. 2024-04-25T10:09:16.703 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:09:16.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.805+0000 7fd1f0709640 1 Processor -- start 2024-04-25T10:09:16.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.805+0000 7fd1f0709640 1 -- start start 2024-04-25T10:09:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.806+0000 7fd1f0709640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1e80ccee0 0x7fd1e80b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.806+0000 7fd1f0709640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e80cd450 con 0x7fd1e806b2b0 2024-04-25T10:09:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.806+0000 7fd1f0709640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e80cd730 con 0x7fd1e815baf0 2024-04-25T10:09:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.806+0000 7fd1f0709640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1e80ce060 con 0x7fd1e80ccee0 2024-04-25T10:09:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.807+0000 7fd1eeca6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1e80ccee0 0x7fd1e80b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.807+0000 7fd1eeca6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1e80ccee0 0x7fd1e80b0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39700/0 (socket says 172.21.15.16:39700) 2024-04-25T10:09:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.807+0000 7fd1eeca6640 1 -- 172.21.15.16:0/1710638290 learned_addr learned my addr 172.21.15.16:0/1710638290 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.807+0000 7fd1edca4640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd1e806b2b0 0x7fd1e80b03b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56040/0 (socket says 172.21.15.16:56040) 2024-04-25T10:09:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.807+0000 7fd1eeca6640 1 -- 172.21.15.16:0/1710638290 >> v1:172.21.15.186:6789/0 conn(0x7fd1e815baf0 legacy=0x7fd1e806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.807+0000 7fd1eeca6640 1 -- 172.21.15.16:0/1710638290 >> v1:172.21.15.16:6789/0 conn(0x7fd1e806b2b0 legacy=0x7fd1e80b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:16.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.807+0000 7fd1eeca6640 1 -- 172.21.15.16:0/1710638290 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e80cda10 con 0x7fd1e80ccee0 2024-04-25T10:09:16.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.808+0000 7fd1eeca6640 1 --2- 172.21.15.16:0/1710638290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1e80ccee0 0x7fd1e80b0ac0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0009870 tx=0x7fd1e002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb171878e64eeca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:16.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.808+0000 7fd1ed4a3640 1 -- 172.21.15.16:0/1710638290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0037070 con 0x7fd1e80ccee0 2024-04-25T10:09:16.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.808+0000 7fd1ed4a3640 1 -- 172.21.15.16:0/1710638290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e0032440 con 0x7fd1e80ccee0 2024-04-25T10:09:16.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.808+0000 7fd1ed4a3640 1 -- 172.21.15.16:0/1710638290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0032810 con 0x7fd1e80ccee0 2024-04-25T10:09:16.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.808+0000 7fd1f0709640 1 -- 172.21.15.16:0/1710638290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1e80ccee0 msgr2=0x7fd1e80b0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:16.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.808+0000 7fd1f0709640 1 --2- 172.21.15.16:0/1710638290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1e80ccee0 0x7fd1e80b0ac0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0009870 tx=0x7fd1e002a2a0 comp rx=0 tx=0).stop 2024-04-25T10:09:16.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.808+0000 7fd1f0709640 1 -- 172.21.15.16:0/1710638290 shutdown_connections 2024-04-25T10:09:16.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.808+0000 7fd1f0709640 1 --2- 172.21.15.16:0/1710638290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1e80ccee0 0x7fd1e80b0ac0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:16.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.808+0000 7fd1f0709640 1 -- 172.21.15.16:0/1710638290 >> 172.21.15.16:0/1710638290 conn(0x7fd1e80b2390 msgr2=0x7fd1e80b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:16.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.808+0000 7fd1f0709640 1 -- 172.21.15.16:0/1710638290 shutdown_connections 2024-04-25T10:09:16.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.808+0000 7fd1f0709640 1 -- 172.21.15.16:0/1710638290 wait complete. 2024-04-25T10:09:16.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.810+0000 7fd1f0709640 1 Processor -- start 2024-04-25T10:09:16.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.810+0000 7fd1f0709640 1 -- start start 2024-04-25T10:09:16.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.810+0000 7fd1f0709640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1e806b2b0 0x7fd1e8152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:16.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.810+0000 7fd1f0709640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1e80ccee0 0x7fd1e80c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:16.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.810+0000 7fd1f0709640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1e815baf0 0x7fd1e81522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:16.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.810+0000 7fd1f0709640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e80c41a0 con 0x7fd1e80ccee0 2024-04-25T10:09:16.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.810+0000 7fd1f0709640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e80c4020 con 0x7fd1e815baf0 2024-04-25T10:09:16.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.810+0000 7fd1f0709640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e815bed0 con 0x7fd1e806b2b0 2024-04-25T10:09:16.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.810+0000 7fd1edca4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1e80ccee0 0x7fd1e80c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:16.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.810+0000 7fd1edca4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1e80ccee0 0x7fd1e80c8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39710/0 (socket says 172.21.15.16:39710) 2024-04-25T10:09:16.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.810+0000 7fd1edca4640 1 -- 172.21.15.16:0/4100067503 learned_addr learned my addr 172.21.15.16:0/4100067503 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.811+0000 7fd1eeca6640 1 --2- 172.21.15.16:0/4100067503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1e815baf0 0x7fd1e81522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.811+0000 7fd1ee4a5640 1 --2- 172.21.15.16:0/4100067503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1e806b2b0 0x7fd1e8152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.811+0000 7fd1edca4640 1 -- 172.21.15.16:0/4100067503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1e806b2b0 msgr2=0x7fd1e8152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.811+0000 7fd1edca4640 1 --2- 172.21.15.16:0/4100067503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1e806b2b0 0x7fd1e8152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.811+0000 7fd1edca4640 1 -- 172.21.15.16:0/4100067503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1e815baf0 msgr2=0x7fd1e81522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.811+0000 7fd1edca4640 1 --2- 172.21.15.16:0/4100067503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1e815baf0 0x7fd1e81522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.811+0000 7fd1edca4640 1 -- 172.21.15.16:0/4100067503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e80c5480 con 0x7fd1e80ccee0 2024-04-25T10:09:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.811+0000 7fd1edca4640 1 --2- 172.21.15.16:0/4100067503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1e80ccee0 0x7fd1e80c8c30 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c3c0 tx=0x7fd1dc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.811+0000 7fd1e6ffd640 1 -- 172.21.15.16:0/4100067503 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0074e0 con 0x7fd1e80ccee0 2024-04-25T10:09:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.811+0000 7fd1e6ffd640 1 -- 172.21.15.16:0/4100067503 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1dc007680 con 0x7fd1e80ccee0 2024-04-25T10:09:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.811+0000 7fd1e6ffd640 1 -- 172.21.15.16:0/4100067503 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc004a00 con 0x7fd1e80ccee0 2024-04-25T10:09:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.811+0000 7fd1f0709640 1 -- 172.21.15.16:0/4100067503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e80c5760 con 0x7fd1e80ccee0 2024-04-25T10:09:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.811+0000 7fd1f0709640 1 -- 172.21.15.16:0/4100067503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e80c6210 con 0x7fd1e80ccee0 2024-04-25T10:09:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.812+0000 7fd1f0709640 1 -- 172.21.15.16:0/4100067503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e80750b0 con 0x7fd1e80ccee0 2024-04-25T10:09:16.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.813+0000 7fd1e6ffd640 1 -- 172.21.15.16:0/4100067503 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1dc01a020 con 0x7fd1e80ccee0 2024-04-25T10:09:16.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.813+0000 7fd1e6ffd640 1 --2- 172.21.15.16:0/4100067503 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd1d406b500 0x7fd1d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:16.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.813+0000 7fd1e6ffd640 1 -- 172.21.15.16:0/4100067503 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd1dc089580 con 0x7fd1e80ccee0 2024-04-25T10:09:16.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.813+0000 7fd1ee4a5640 1 --2- 172.21.15.16:0/4100067503 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd1d406b500 0x7fd1d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:16.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.814+0000 7fd1ee4a5640 1 --2- 172.21.15.16:0/4100067503 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd1d406b500 0x7fd1d406d9c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8005520 tx=0x7fd1d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:16.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.818+0000 7fd1e6ffd640 1 -- 172.21.15.16:0/4100067503 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd1dc058700 con 0x7fd1e80ccee0 2024-04-25T10:09:17.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:16.998+0000 7fd1f0709640 1 -- 172.21.15.16:0/4100067503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd1e80b67c0 con 0x7fd1e80ccee0 2024-04-25T10:09:17.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.001+0000 7fd1e6ffd640 1 -- 172.21.15.16:0/4100067503 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd1dc058000 con 0x7fd1e80ccee0 2024-04-25T10:09:17.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.003+0000 7fd1f0709640 1 -- 172.21.15.16:0/4100067503 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd1d406b500 msgr2=0x7fd1d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.003+0000 7fd1f0709640 1 --2- 172.21.15.16:0/4100067503 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd1d406b500 0x7fd1d406d9c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8005520 tx=0x7fd1d8009040 comp rx=0 tx=0).stop 2024-04-25T10:09:17.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.003+0000 7fd1f0709640 1 -- 172.21.15.16:0/4100067503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1e80ccee0 msgr2=0x7fd1e80c8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.003+0000 7fd1f0709640 1 --2- 172.21.15.16:0/4100067503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1e80ccee0 0x7fd1e80c8c30 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c3c0 tx=0x7fd1dc00c880 comp rx=0 tx=0).stop 2024-04-25T10:09:17.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.003+0000 7fd1f0709640 1 -- 172.21.15.16:0/4100067503 shutdown_connections 2024-04-25T10:09:17.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.003+0000 7fd1f0709640 1 --2- 172.21.15.16:0/4100067503 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd1d406b500 0x7fd1d406d9c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.003+0000 7fd1f0709640 1 --2- 172.21.15.16:0/4100067503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1e815baf0 0x7fd1e81522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.003+0000 7fd1f0709640 1 --2- 172.21.15.16:0/4100067503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1e80ccee0 0x7fd1e80c8c30 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.003+0000 7fd1f0709640 1 --2- 172.21.15.16:0/4100067503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1e806b2b0 0x7fd1e8152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.003+0000 7fd1f0709640 1 -- 172.21.15.16:0/4100067503 >> 172.21.15.16:0/4100067503 conn(0x7fd1e80b2390 msgr2=0x7fd1e80b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:17.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.003+0000 7fd1f0709640 1 -- 172.21.15.16:0/4100067503 shutdown_connections 2024-04-25T10:09:17.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.003+0000 7fd1f0709640 1 -- 172.21.15.16:0/4100067503 wait complete. 2024-04-25T10:09:17.021 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:09:17.021 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:09:17.058 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:09:17.107 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:09:17.107 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:09:17.107 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:17.123 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:09:17.123 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:09:17.123 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:17.141 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:09:17.141 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:09:17.141 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:17.178 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:09:17.178 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:09:17.179 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:17.199 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:09:17.199 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:09:17.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.312+0000 7fb17d3f5640 1 Processor -- start 2024-04-25T10:09:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.312+0000 7fb17d3f5640 1 -- start start 2024-04-25T10:09:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.313+0000 7fb17d3f5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb178144450 0x7fb1780c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.313+0000 7fb17d3f5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb1780ccdc0 0x7fb1780722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.313+0000 7fb17d3f5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1780c75c0 con 0x7fb1780727e0 2024-04-25T10:09:17.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.313+0000 7fb17d3f5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1780cdb60 con 0x7fb178144450 2024-04-25T10:09:17.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.313+0000 7fb17d3f5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1780cd2c0 con 0x7fb1780ccdc0 2024-04-25T10:09:17.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.313+0000 7fb176ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb178144450 0x7fb1780c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.313+0000 7fb1777fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb1780727e0 0x7fb17815eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56048/0 (socket says 172.21.15.16:56048) 2024-04-25T10:09:17.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.313+0000 7fb1777fe640 1 -- 172.21.15.16:0/2176899848 learned_addr learned my addr 172.21.15.16:0/2176899848 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:17.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.313+0000 7fb1767fc640 1 --2- 172.21.15.16:0/2176899848 >> v2:172.21.15.186:3300/0 conn(0x7fb1780ccdc0 0x7fb1780722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.314+0000 7fb176ffd640 1 -- 172.21.15.16:0/2176899848 >> v2:172.21.15.186:3300/0 conn(0x7fb1780ccdc0 msgr2=0x7fb1780722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:17.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.314+0000 7fb176ffd640 1 --2- 172.21.15.16:0/2176899848 >> v2:172.21.15.186:3300/0 conn(0x7fb1780ccdc0 0x7fb1780722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.314+0000 7fb176ffd640 1 -- 172.21.15.16:0/2176899848 >> v1:172.21.15.16:6789/0 conn(0x7fb1780727e0 legacy=0x7fb17815eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.314+0000 7fb175ffb640 1 -- 172.21.15.16:0/2176899848 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983084429 0 0) 0x7fb1780c75c0 con 0x7fb1780727e0 2024-04-25T10:09:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.314+0000 7fb176ffd640 1 -- 172.21.15.16:0/2176899848 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1780cd5a0 con 0x7fb178144450 2024-04-25T10:09:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.314+0000 7fb176ffd640 1 --2- 172.21.15.16:0/2176899848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb178144450 0x7fb1780c7080 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb17806f3b0 tx=0x7fb17002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3019532ede6fbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.314+0000 7fb175ffb640 1 -- 172.21.15.16:0/2176899848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170037070 con 0x7fb178144450 2024-04-25T10:09:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.314+0000 7fb175ffb640 1 -- 172.21.15.16:0/2176899848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb170032440 con 0x7fb178144450 2024-04-25T10:09:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.314+0000 7fb175ffb640 1 -- 172.21.15.16:0/2176899848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17002ae60 con 0x7fb178144450 2024-04-25T10:09:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.315+0000 7fb17d3f5640 1 -- 172.21.15.16:0/2176899848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb178144450 msgr2=0x7fb1780c7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.315+0000 7fb17d3f5640 1 --2- 172.21.15.16:0/2176899848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb178144450 0x7fb1780c7080 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb17806f3b0 tx=0x7fb17002a110 comp rx=0 tx=0).stop 2024-04-25T10:09:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.315+0000 7fb17d3f5640 1 -- 172.21.15.16:0/2176899848 shutdown_connections 2024-04-25T10:09:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.315+0000 7fb17d3f5640 1 --2- 172.21.15.16:0/2176899848 >> v2:172.21.15.186:3300/0 conn(0x7fb1780ccdc0 0x7fb1780722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.315+0000 7fb17d3f5640 1 --2- 172.21.15.16:0/2176899848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb178144450 0x7fb1780c7080 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.315+0000 7fb17d3f5640 1 -- 172.21.15.16:0/2176899848 >> 172.21.15.16:0/2176899848 conn(0x7fb1780b2370 msgr2=0x7fb1780b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:17.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.315+0000 7fb17d3f5640 1 -- 172.21.15.16:0/2176899848 shutdown_connections 2024-04-25T10:09:17.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.315+0000 7fb17d3f5640 1 -- 172.21.15.16:0/2176899848 wait complete. 2024-04-25T10:09:17.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.316+0000 7fb17d3f5640 1 Processor -- start 2024-04-25T10:09:17.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.316+0000 7fb17d3f5640 1 -- start start 2024-04-25T10:09:17.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.317+0000 7fb17d3f5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1780727e0 0x7fb17813b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.317+0000 7fb17d3f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1780ccdc0 0x7fb178159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.317+0000 7fb17d3f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb178144450 0x7fb17815a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.317+0000 7fb17d3f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17815f1b0 con 0x7fb178144450 2024-04-25T10:09:17.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.317+0000 7fb17d3f5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17815e580 con 0x7fb1780727e0 2024-04-25T10:09:17.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.317+0000 7fb17d3f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17815f330 con 0x7fb1780ccdc0 2024-04-25T10:09:17.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.317+0000 7fb1777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb178144450 0x7fb17815a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.317+0000 7fb1777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb178144450 0x7fb17815a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39740/0 (socket says 172.21.15.16:39740) 2024-04-25T10:09:17.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.317+0000 7fb1777fe640 1 -- 172.21.15.16:0/3222786546 learned_addr learned my addr 172.21.15.16:0/3222786546 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:17.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.317+0000 7fb176ffd640 1 --2- 172.21.15.16:0/3222786546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1780727e0 0x7fb17813b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.317+0000 7fb1767fc640 1 --2- 172.21.15.16:0/3222786546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1780ccdc0 0x7fb178159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.317+0000 7fb1777fe640 1 -- 172.21.15.16:0/3222786546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1780ccdc0 msgr2=0x7fb178159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.317+0000 7fb1777fe640 1 --2- 172.21.15.16:0/3222786546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1780ccdc0 0x7fb178159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.317+0000 7fb1777fe640 1 -- 172.21.15.16:0/3222786546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1780727e0 msgr2=0x7fb17813b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.317+0000 7fb1777fe640 1 --2- 172.21.15.16:0/3222786546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1780727e0 0x7fb17813b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.318+0000 7fb1777fe640 1 -- 172.21.15.16:0/3222786546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1780b4cc0 con 0x7fb178144450 2024-04-25T10:09:17.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.318+0000 7fb176ffd640 1 --2- 172.21.15.16:0/3222786546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1780727e0 0x7fb17813b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:17.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.318+0000 7fb1777fe640 1 --2- 172.21.15.16:0/3222786546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb178144450 0x7fb17815a310 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb16800c7d0 tx=0x7fb16800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.318+0000 7fb16f7fe640 1 -- 172.21.15.16:0/3222786546 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168018070 con 0x7fb178144450 2024-04-25T10:09:17.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.318+0000 7fb16f7fe640 1 -- 172.21.15.16:0/3222786546 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16800a640 con 0x7fb178144450 2024-04-25T10:09:17.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.318+0000 7fb17d3f5640 1 -- 172.21.15.16:0/3222786546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1780c7990 con 0x7fb178144450 2024-04-25T10:09:17.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.318+0000 7fb16f7fe640 1 -- 172.21.15.16:0/3222786546 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16800aa70 con 0x7fb178144450 2024-04-25T10:09:17.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.318+0000 7fb17d3f5640 1 -- 172.21.15.16:0/3222786546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1780c7f50 con 0x7fb178144450 2024-04-25T10:09:17.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.319+0000 7fb17d3f5640 1 -- 172.21.15.16:0/3222786546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1780c8150 con 0x7fb178144450 2024-04-25T10:09:17.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.319+0000 7fb16f7fe640 1 -- 172.21.15.16:0/3222786546 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb16801b070 con 0x7fb178144450 2024-04-25T10:09:17.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.320+0000 7fb16f7fe640 1 --2- 172.21.15.16:0/3222786546 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb15806b4d0 0x7fb15806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.320+0000 7fb16f7fe640 1 -- 172.21.15.16:0/3222786546 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb1680895c0 con 0x7fb178144450 2024-04-25T10:09:17.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.320+0000 7fb176ffd640 1 --2- 172.21.15.16:0/3222786546 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb15806b4d0 0x7fb15806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.321+0000 7fb176ffd640 1 --2- 172.21.15.16:0/3222786546 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb15806b4d0 0x7fb15806d990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb170034000 tx=0x7fb17002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.324+0000 7fb16f7fe640 1 -- 172.21.15.16:0/3222786546 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1780c8150 con 0x7fb178144450 2024-04-25T10:09:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.541+0000 7fb17d3f5640 1 -- 172.21.15.16:0/3222786546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1780749b0 con 0x7fb178144450 2024-04-25T10:09:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.542+0000 7fb16f7fe640 1 -- 172.21.15.16:0/3222786546 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fb1680586a0 con 0x7fb178144450 2024-04-25T10:09:17.549 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:17.549 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2270839865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2270839865},{"type":"v1","addr":"172.21.15.186:6836","nonce":2270839865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5647,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/925282579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":925282579},{"type":"v1","addr":"172.21.15.186:6837","nonce":925282579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2471714328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2471714328},{"type":"v1","addr":"172.21.15.16:6835","nonce":2471714328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6552,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3911308615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3911308615},{"type":"v1","addr":"172.21.15.16:6834","nonce":3911308615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[]} 2024-04-25T10:09:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-04-25T10:09:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.544+0000 7fb17d3f5640 1 -- 172.21.15.16:0/3222786546 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb15806b4d0 msgr2=0x7fb15806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.544+0000 7fb17d3f5640 1 --2- 172.21.15.16:0/3222786546 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb15806b4d0 0x7fb15806d990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb170034000 tx=0x7fb17002e000 comp rx=0 tx=0).stop 2024-04-25T10:09:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.544+0000 7fb17d3f5640 1 -- 172.21.15.16:0/3222786546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb178144450 msgr2=0x7fb17815a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.544+0000 7fb17d3f5640 1 --2- 172.21.15.16:0/3222786546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb178144450 0x7fb17815a310 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb16800c7d0 tx=0x7fb16800cc90 comp rx=0 tx=0).stop 2024-04-25T10:09:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.544+0000 7fb17d3f5640 1 -- 172.21.15.16:0/3222786546 shutdown_connections 2024-04-25T10:09:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.544+0000 7fb17d3f5640 1 --2- 172.21.15.16:0/3222786546 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb15806b4d0 0x7fb15806d990 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.544+0000 7fb17d3f5640 1 --2- 172.21.15.16:0/3222786546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb178144450 0x7fb17815a310 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.544+0000 7fb17d3f5640 1 --2- 172.21.15.16:0/3222786546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1780ccdc0 0x7fb178159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.544+0000 7fb17d3f5640 1 --2- 172.21.15.16:0/3222786546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1780727e0 0x7fb17813b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.544+0000 7fb17d3f5640 1 -- 172.21.15.16:0/3222786546 >> 172.21.15.16:0/3222786546 conn(0x7fb1780b2370 msgr2=0x7fb1780b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.544+0000 7fb17d3f5640 1 -- 172.21.15.16:0/3222786546 shutdown_connections 2024-04-25T10:09:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.544+0000 7fb17d3f5640 1 -- 172.21.15.16:0/3222786546 wait complete. 2024-04-25T10:09:17.562 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:09:17.562 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:09:17.563 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:09:17.563 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:09:17.563 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:09:17.563 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:09:17.563 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:09:17.563 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:09:17.564 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:09:17.564 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:09:17.564 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:09:17.564 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:09:17.564 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:09:17.565 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:09:17.565 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:09:17.565 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:09:17.572 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:09:17.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:09:17.573 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:09:17.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:09:17.576 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:09:17.577 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:09:17.577 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:09:17.578 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:09:17.615 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:09:17.616 DEBUG:teuthology.parallel:result is None 2024-04-25T10:09:17.616 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:09:17.616 DEBUG:teuthology.parallel:result is None 2024-04-25T10:09:17.619 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:09:17.619 DEBUG:teuthology.parallel:result is None 2024-04-25T10:09:17.619 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:09:17.620 DEBUG:teuthology.parallel:result is None 2024-04-25T10:09:17.620 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:09:17.691 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-04-25T10:09:17.695 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-04-25T10:09:17.702 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2024-04-25T10:09:17.703 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2024-04-25T10:09:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.719+0000 7f0f657f4640 1 Processor -- start 2024-04-25T10:09:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.719+0000 7f0f657f4640 1 -- start start 2024-04-25T10:09:17.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.720+0000 7f0f657f4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f6015dbe0 0x7f0f6015ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.720+0000 7f0f657f4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f600d3580 0x7f0f600b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.720+0000 7f0f657f4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f600b9980 con 0x7f0f600b9410 2024-04-25T10:09:17.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.720+0000 7f0f657f4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f6015f210 con 0x7f0f6015dbe0 2024-04-25T10:09:17.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.720+0000 7f0f657f4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f600ce140 con 0x7f0f600d3580 2024-04-25T10:09:17.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.720+0000 7f0f5effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f6015dbe0 0x7f0f6015ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.720+0000 7f0f5f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0f600b9410 0x7f0f6006c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56066/0 (socket says 172.21.15.16:56066) 2024-04-25T10:09:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.720+0000 7f0f5f7fe640 1 -- 172.21.15.16:0/1005939286 learned_addr learned my addr 172.21.15.16:0/1005939286 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.720+0000 7f0f57fff640 1 --2- 172.21.15.16:0/1005939286 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f600d3580 0x7f0f600b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.720+0000 7f0f5effd640 1 -- 172.21.15.16:0/1005939286 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f600d3580 msgr2=0x7f0f600b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.720+0000 7f0f5effd640 1 --2- 172.21.15.16:0/1005939286 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f600d3580 0x7f0f600b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.720+0000 7f0f5effd640 1 -- 172.21.15.16:0/1005939286 >> v1:172.21.15.16:6789/0 conn(0x7f0f600b9410 legacy=0x7f0f6006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.720+0000 7f0f5e7fc640 1 -- 172.21.15.16:0/1005939286 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238016158 0 0) 0x7f0f600b9980 con 0x7f0f600b9410 2024-04-25T10:09:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.720+0000 7f0f5effd640 1 -- 172.21.15.16:0/1005939286 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f600ce420 con 0x7f0f6015dbe0 2024-04-25T10:09:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.721+0000 7f0f5effd640 1 --2- 172.21.15.16:0/1005939286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f6015dbe0 0x7f0f6015ecd0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f0f500097a0 tx=0x7f0f5002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1275ec0db09afd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.721+0000 7f0f5e7fc640 1 -- 172.21.15.16:0/1005939286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f50037070 con 0x7f0f6015dbe0 2024-04-25T10:09:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.721+0000 7f0f5e7fc640 1 -- 172.21.15.16:0/1005939286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f50032440 con 0x7f0f6015dbe0 2024-04-25T10:09:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.721+0000 7f0f5e7fc640 1 -- 172.21.15.16:0/1005939286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f50032810 con 0x7f0f6015dbe0 2024-04-25T10:09:17.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.722+0000 7f0f657f4640 1 -- 172.21.15.16:0/1005939286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f6015dbe0 msgr2=0x7f0f6015ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.722+0000 7f0f657f4640 1 --2- 172.21.15.16:0/1005939286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f6015dbe0 0x7f0f6015ecd0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f0f500097a0 tx=0x7f0f5002a180 comp rx=0 tx=0).stop 2024-04-25T10:09:17.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.722+0000 7f0f657f4640 1 -- 172.21.15.16:0/1005939286 shutdown_connections 2024-04-25T10:09:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.722+0000 7f0f657f4640 1 --2- 172.21.15.16:0/1005939286 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f600d3580 0x7f0f600b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.722+0000 7f0f657f4640 1 --2- 172.21.15.16:0/1005939286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f6015dbe0 0x7f0f6015ecd0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.722+0000 7f0f657f4640 1 -- 172.21.15.16:0/1005939286 >> 172.21.15.16:0/1005939286 conn(0x7f0f60072900 msgr2=0x7f0f600d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.722+0000 7f0f657f4640 1 -- 172.21.15.16:0/1005939286 shutdown_connections 2024-04-25T10:09:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.722+0000 7f0f657f4640 1 -- 172.21.15.16:0/1005939286 wait complete. 2024-04-25T10:09:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.723+0000 7f0f657f4640 1 Processor -- start 2024-04-25T10:09:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.723+0000 7f0f657f4640 1 -- start start 2024-04-25T10:09:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.724+0000 7f0f657f4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f600b9410 0x7f0f60156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.724+0000 7f0f657f4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f600d3580 0x7f0f600c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.724+0000 7f0f657f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f6015dbe0 0x7f0f60161970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.724+0000 7f0f657f4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6015dfc0 con 0x7f0f6015dbe0 2024-04-25T10:09:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.724+0000 7f0f657f4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f600d3060 con 0x7f0f600b9410 2024-04-25T10:09:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.724+0000 7f0f657f4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6015e140 con 0x7f0f600d3580 2024-04-25T10:09:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.724+0000 7f0f5f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f6015dbe0 0x7f0f60161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.724+0000 7f0f5f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f6015dbe0 0x7f0f60161970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39788/0 (socket says 172.21.15.16:39788) 2024-04-25T10:09:17.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.724+0000 7f0f5f7fe640 1 -- 172.21.15.16:0/1038957939 learned_addr learned my addr 172.21.15.16:0/1038957939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:17.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.724+0000 7f0f5effd640 1 --2- 172.21.15.16:0/1038957939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f600b9410 0x7f0f60156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.724+0000 7f0f57fff640 1 --2- 172.21.15.16:0/1038957939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f600d3580 0x7f0f600c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.724+0000 7f0f5f7fe640 1 -- 172.21.15.16:0/1038957939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f600d3580 msgr2=0x7f0f600c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.724+0000 7f0f5f7fe640 1 --2- 172.21.15.16:0/1038957939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f600d3580 0x7f0f600c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.724+0000 7f0f5f7fe640 1 -- 172.21.15.16:0/1038957939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f600b9410 msgr2=0x7f0f60156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.724+0000 7f0f5f7fe640 1 --2- 172.21.15.16:0/1038957939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f600b9410 0x7f0f60156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.724+0000 7f0f5f7fe640 1 -- 172.21.15.16:0/1038957939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f60156680 con 0x7f0f6015dbe0 2024-04-25T10:09:17.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.724+0000 7f0f5f7fe640 1 --2- 172.21.15.16:0/1038957939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f6015dbe0 0x7f0f60161970 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800c700 tx=0x7f0f5800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.724+0000 7f0f577fe640 1 -- 172.21.15.16:0/1038957939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f58018070 con 0x7f0f6015dbe0 2024-04-25T10:09:17.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.725+0000 7f0f577fe640 1 -- 172.21.15.16:0/1038957939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5800ce10 con 0x7f0f6015dbe0 2024-04-25T10:09:17.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.725+0000 7f0f657f4640 1 -- 172.21.15.16:0/1038957939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f600b7890 con 0x7f0f6015dbe0 2024-04-25T10:09:17.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.725+0000 7f0f577fe640 1 -- 172.21.15.16:0/1038957939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5800a800 con 0x7f0f6015dbe0 2024-04-25T10:09:17.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.725+0000 7f0f657f4640 1 -- 172.21.15.16:0/1038957939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f600cc1c0 con 0x7f0f6015dbe0 2024-04-25T10:09:17.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.726+0000 7f0f657f4640 1 -- 172.21.15.16:0/1038957939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f600750b0 con 0x7f0f6015dbe0 2024-04-25T10:09:17.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.726+0000 7f0f577fe640 1 -- 172.21.15.16:0/1038957939 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f5801b070 con 0x7f0f6015dbe0 2024-04-25T10:09:17.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.726+0000 7f0f577fe640 1 --2- 172.21.15.16:0/1038957939 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0f4006b4d0 0x7f0f4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.726+0000 7f0f577fe640 1 -- 172.21.15.16:0/1038957939 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0f580892d0 con 0x7f0f6015dbe0 2024-04-25T10:09:17.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.727+0000 7f0f5effd640 1 --2- 172.21.15.16:0/1038957939 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0f4006b4d0 0x7f0f4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.727+0000 7f0f5effd640 1 --2- 172.21.15.16:0/1038957939 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0f4006b4d0 0x7f0f4006d990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0f50034000 tx=0x7f0f5002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.731+0000 7f0f577fe640 1 -- 172.21.15.16:0/1038957939 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f60156680 con 0x7f0f6015dbe0 2024-04-25T10:09:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:17.925+0000 7f0f657f4640 1 -- 172.21.15.16:0/1038957939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0f600ca6c0 con 0x7f0f6015dbe0 2024-04-25T10:09:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.691+0000 7f0f577fe640 1 -- 172.21.15.16:0/1038957939 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v66) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0f58057e90 con 0x7f0f6015dbe0 2024-04-25T10:09:18.699 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-04-25T10:09:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.695+0000 7f0f657f4640 1 -- 172.21.15.16:0/1038957939 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0f4006b4d0 msgr2=0x7f0f4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.695+0000 7f0f657f4640 1 --2- 172.21.15.16:0/1038957939 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0f4006b4d0 0x7f0f4006d990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0f50034000 tx=0x7f0f5002e000 comp rx=0 tx=0).stop 2024-04-25T10:09:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.695+0000 7f0f657f4640 1 -- 172.21.15.16:0/1038957939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f6015dbe0 msgr2=0x7f0f60161970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.695+0000 7f0f657f4640 1 --2- 172.21.15.16:0/1038957939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f6015dbe0 0x7f0f60161970 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800c700 tx=0x7f0f5800cbc0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.695+0000 7f0f657f4640 1 -- 172.21.15.16:0/1038957939 shutdown_connections 2024-04-25T10:09:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.695+0000 7f0f657f4640 1 --2- 172.21.15.16:0/1038957939 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0f4006b4d0 0x7f0f4006d990 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.695+0000 7f0f657f4640 1 --2- 172.21.15.16:0/1038957939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f6015dbe0 0x7f0f60161970 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.695+0000 7f0f657f4640 1 --2- 172.21.15.16:0/1038957939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0f600d3580 0x7f0f600c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.695+0000 7f0f657f4640 1 --2- 172.21.15.16:0/1038957939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f600b9410 0x7f0f60156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.695+0000 7f0f657f4640 1 -- 172.21.15.16:0/1038957939 >> 172.21.15.16:0/1038957939 conn(0x7f0f60072900 msgr2=0x7f0f60160210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.695+0000 7f0f657f4640 1 -- 172.21.15.16:0/1038957939 shutdown_connections 2024-04-25T10:09:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.695+0000 7f0f657f4640 1 -- 172.21.15.16:0/1038957939 wait complete. 2024-04-25T10:09:18.718 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:09:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.820+0000 7f82243ce640 1 Processor -- start 2024-04-25T10:09:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.820+0000 7f82243ce640 1 -- start start 2024-04-25T10:09:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.821+0000 7f82243ce640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f821c15dce0 0x7f821c0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.821+0000 7f82243ce640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f821c0b5b60 0x7f821c0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.821+0000 7f82243ce640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821c15e0c0 con 0x7f821c0d35b0 2024-04-25T10:09:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.821+0000 7f82243ce640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f821c0d2510 con 0x7f821c15dce0 2024-04-25T10:09:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.821+0000 7f82243ce640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0d27f0 con 0x7f821c0b5b60 2024-04-25T10:09:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.821+0000 7f822216a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f821c15dce0 0x7f821c0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.821+0000 7f822216a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f821c15dce0 0x7f821c0d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39792/0 (socket says 172.21.15.16:39792) 2024-04-25T10:09:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.821+0000 7f822216a640 1 -- 172.21.15.16:0/1626959452 learned_addr learned my addr 172.21.15.16:0/1626959452 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.821+0000 7f822216a640 1 -- 172.21.15.16:0/1626959452 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f821c0b5b60 msgr2=0x7f821c0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.821+0000 7f822216a640 1 --2- 172.21.15.16:0/1626959452 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f821c0b5b60 0x7f821c0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.821+0000 7f822216a640 1 -- 172.21.15.16:0/1626959452 >> v1:172.21.15.186:6789/0 conn(0x7f821c0d35b0 legacy=0x7f821c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.821+0000 7f822216a640 1 -- 172.21.15.16:0/1626959452 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c0d2ad0 con 0x7f821c15dce0 2024-04-25T10:09:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.822+0000 7f822216a640 1 --2- 172.21.15.16:0/1626959452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821c15dce0 0x7f821c0d1a50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8218009870 tx=0x7f821802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3012161526b4f7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.822+0000 7f8221168640 1 -- 172.21.15.16:0/1626959452 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8218037070 con 0x7f821c15dce0 2024-04-25T10:09:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.822+0000 7f8221168640 1 -- 172.21.15.16:0/1626959452 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8218032440 con 0x7f821c15dce0 2024-04-25T10:09:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.822+0000 7f8221168640 1 -- 172.21.15.16:0/1626959452 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8218032810 con 0x7f821c15dce0 2024-04-25T10:09:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.823+0000 7f82243ce640 1 -- 172.21.15.16:0/1626959452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821c15dce0 msgr2=0x7f821c0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.823+0000 7f82243ce640 1 --2- 172.21.15.16:0/1626959452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821c15dce0 0x7f821c0d1a50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8218009870 tx=0x7f821802a210 comp rx=0 tx=0).stop 2024-04-25T10:09:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.823+0000 7f82243ce640 1 -- 172.21.15.16:0/1626959452 shutdown_connections 2024-04-25T10:09:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.823+0000 7f82243ce640 1 --2- 172.21.15.16:0/1626959452 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f821c0b5b60 0x7f821c0b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.823+0000 7f82243ce640 1 --2- 172.21.15.16:0/1626959452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821c15dce0 0x7f821c0d1a50 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.823+0000 7f82243ce640 1 -- 172.21.15.16:0/1626959452 >> 172.21.15.16:0/1626959452 conn(0x7f821c072900 msgr2=0x7f821c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.823+0000 7f82243ce640 1 -- 172.21.15.16:0/1626959452 shutdown_connections 2024-04-25T10:09:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.823+0000 7f82243ce640 1 -- 172.21.15.16:0/1626959452 wait complete. 2024-04-25T10:09:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.824+0000 7f82243ce640 1 Processor -- start 2024-04-25T10:09:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.824+0000 7f82243ce640 1 -- start start 2024-04-25T10:09:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f82243ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821c0b5b60 0x7f821c0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f82243ce640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f821c0d35b0 0x7f821c157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f82243ce640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f821c15dce0 0x7f821c0c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f82243ce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0b9820 con 0x7f821c0b5b60 2024-04-25T10:09:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f82243ce640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0d3090 con 0x7f821c15dce0 2024-04-25T10:09:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f82243ce640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0b99a0 con 0x7f821c0d35b0 2024-04-25T10:09:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f822216a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821c0b5b60 0x7f821c0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f822216a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821c0b5b60 0x7f821c0cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39800/0 (socket says 172.21.15.16:39800) 2024-04-25T10:09:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f822216a640 1 -- 172.21.15.16:0/2943473117 learned_addr learned my addr 172.21.15.16:0/2943473117 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f8221969640 1 --2- 172.21.15.16:0/2943473117 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f821c0d35b0 0x7f821c157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f822216a640 1 -- 172.21.15.16:0/2943473117 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f821c0d35b0 msgr2=0x7f821c157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f822296b640 1 --2- 172.21.15.16:0/2943473117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f821c15dce0 0x7f821c0c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f822216a640 1 --2- 172.21.15.16:0/2943473117 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f821c0d35b0 0x7f821c157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f822216a640 1 -- 172.21.15.16:0/2943473117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f821c15dce0 msgr2=0x7f821c0c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f822216a640 1 --2- 172.21.15.16:0/2943473117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f821c15dce0 0x7f821c0c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f822216a640 1 -- 172.21.15.16:0/2943473117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c0c7740 con 0x7f821c0b5b60 2024-04-25T10:09:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f8221969640 1 --2- 172.21.15.16:0/2943473117 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f821c0d35b0 0x7f821c157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f822296b640 1 --2- 172.21.15.16:0/2943473117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f821c15dce0 0x7f821c0c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.825+0000 7f822216a640 1 --2- 172.21.15.16:0/2943473117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821c0b5b60 0x7f821c0cd430 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8218009870 tx=0x7f82180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.826+0000 7f8212ffd640 1 -- 172.21.15.16:0/2943473117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8218041070 con 0x7f821c0b5b60 2024-04-25T10:09:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.826+0000 7f8212ffd640 1 -- 172.21.15.16:0/2943473117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821803c870 con 0x7f821c0b5b60 2024-04-25T10:09:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.826+0000 7f82243ce640 1 -- 172.21.15.16:0/2943473117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821c0c7a30 con 0x7f821c0b5b60 2024-04-25T10:09:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.826+0000 7f8212ffd640 1 -- 172.21.15.16:0/2943473117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8218037040 con 0x7f821c0b5b60 2024-04-25T10:09:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.826+0000 7f82243ce640 1 -- 172.21.15.16:0/2943473117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821c0c7fb0 con 0x7f821c0b5b60 2024-04-25T10:09:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.827+0000 7f82243ce640 1 -- 172.21.15.16:0/2943473117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f821c0749b0 con 0x7f821c0b5b60 2024-04-25T10:09:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.827+0000 7f8212ffd640 1 -- 172.21.15.16:0/2943473117 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8218044070 con 0x7f821c0b5b60 2024-04-25T10:09:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.827+0000 7f8212ffd640 1 --2- 172.21.15.16:0/2943473117 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f820006b4d0 0x7f820006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.828+0000 7f8212ffd640 1 -- 172.21.15.16:0/2943473117 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f82180aabc0 con 0x7f821c0b5b60 2024-04-25T10:09:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.828+0000 7f8221969640 1 --2- 172.21.15.16:0/2943473117 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f820006b4d0 0x7f820006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.828+0000 7f8221969640 1 --2- 172.21.15.16:0/2943473117 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f820006b4d0 0x7f820006d990 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f820c007fd0 tx=0x7f820c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:18.832+0000 7f8212ffd640 1 -- 172.21.15.16:0/2943473117 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f821c0c7740 con 0x7f821c0b5b60 2024-04-25T10:09:19.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.077+0000 7f82243ce640 1 -- 172.21.15.16:0/2943473117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f821c0b73c0 con 0x7f821c0b5b60 2024-04-25T10:09:19.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.078+0000 7f8212ffd640 1 -- 172.21.15.16:0/2943473117 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8218079eb0 con 0x7f821c0b5b60 2024-04-25T10:09:19.085 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:19.085 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:09:19.085 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-04-25T10:09:19.085 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.085 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-04-25T10:09:19.086 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmj+xmBBAAaWdHmoiYcXzFnMVQSioT2Q==", 2024-04-25T10:09:19.086 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.086 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:09:19.086 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:09:19.086 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:09:19.086 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:09:19.086 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:09:19.086 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.086 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-04-25T10:09:19.086 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmUES+DxAAu5It7dQmb7/BM/4x1fUYXA==", 2024-04-25T10:09:19.087 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.087 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:09:19.087 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:09:19.087 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:09:19.087 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:09:19.087 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:09:19.087 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.087 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-04-25T10:09:19.087 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipm+LtFCxAAGG79kv8eWzly8l5qzQwfww==", 2024-04-25T10:09:19.087 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.087 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:09:19.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:09:19.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:09:19.088 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:09:19.088 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:09:19.088 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.088 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-04-25T10:09:19.088 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmqbSSFhAAZtrLsAGSyaMGgv45Hy0WpA==", 2024-04-25T10:09:19.088 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:09:19.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:09:19.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:09:19.088 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:09:19.088 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:09:19.089 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.089 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-04-25T10:09:19.089 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCaKipm2+UdOxAAP8GBunvYsue8EILYV5Uq7A==", 2024-04-25T10:09:19.089 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.089 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:19.089 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:09:19.089 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:09:19.089 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:09:19.089 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.089 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-04-25T10:09:19.089 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCdKipmSv87ORAAigVU7nyzkcXqMEHKsSAsqg==", 2024-04-25T10:09:19.089 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.089 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:19.090 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:09:19.090 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:09:19.090 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:09:19.090 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.090 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-04-25T10:09:19.090 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCgKipm2VMLOBAAJETFfWgq7H/imQAfw93Ruw==", 2024-04-25T10:09:19.090 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.090 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:19.090 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:09:19.090 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:09:19.090 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:09:19.090 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.091 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-04-25T10:09:19.091 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCjKipmb1g0NhAADH2uHQPhW84KLYyWbOjQdw==", 2024-04-25T10:09:19.091 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.091 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:19.091 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:09:19.091 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:09:19.091 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:09:19.091 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.091 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-04-25T10:09:19.091 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCpKipmnH0ROBAAEb8i4nfswYQ/w5mEw48kIg==", 2024-04-25T10:09:19.091 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.091 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:19.091 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:09:19.091 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:09:19.092 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:09:19.092 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.092 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-04-25T10:09:19.092 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCsKipm8HCRNhAA3FhcMlUWxMAiFRXbBWWvNA==", 2024-04-25T10:09:19.092 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.092 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:19.092 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:09:19.092 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:09:19.092 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:09:19.092 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.092 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-04-25T10:09:19.092 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCvKipmfgqpNRAAtPhpSDU9q23Sfa+V7m+3NA==", 2024-04-25T10:09:19.093 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.093 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:19.093 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:09:19.093 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:09:19.093 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:09:19.093 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.093 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-04-25T10:09:19.093 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCyKipm3ndyMxAAeMNODGxNupb043CsA/SP1A==", 2024-04-25T10:09:19.093 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.093 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:19.093 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:09:19.093 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:09:19.093 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:09:19.094 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.094 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-04-25T10:09:19.094 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmAph/GhAAG3x1zC98yHQ13fjpmrlPkg==", 2024-04-25T10:09:19.094 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.094 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:09:19.094 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-04-25T10:09:19.094 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:09:19.094 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:09:19.094 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.094 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-04-25T10:09:19.094 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmrccTIxAAuDZamFXTe6J2d+HAcBe+DA==", 2024-04-25T10:09:19.095 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.095 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:09:19.095 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:09:19.095 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:09:19.095 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:09:19.095 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:09:19.095 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.095 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-04-25T10:09:19.095 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmsFACIBAAFuWsmuQlF58yzIcYUx/6Iw==", 2024-04-25T10:09:19.095 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.095 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:09:19.095 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:09:19.095 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:09:19.095 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:09:19.096 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:09:19.096 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.096 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-04-25T10:09:19.096 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmAe1/KBAAXzLkscuwpCwMnrIKQny9+Q==", 2024-04-25T10:09:19.096 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.096 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:09:19.096 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:09:19.096 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:09:19.096 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:09:19.096 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:09:19.096 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.096 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-04-25T10:09:19.097 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCYKipmVFQ2JBAAdqG2p/M746PZPMDcpQjwfw==", 2024-04-25T10:09:19.097 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.097 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:09:19.097 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:09:19.097 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:09:19.097 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:09:19.097 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:09:19.097 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.097 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-04-25T10:09:19.097 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmiPVHARAAGPbv3P3yVj3tlo0b2gmcyw==", 2024-04-25T10:09:19.097 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.097 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:09:19.097 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-04-25T10:09:19.097 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:09:19.098 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:09:19.098 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:19.098 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-04-25T10:09:19.098 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCYKipmghhyOBAAcCm0hQzU+Wlw45yZQETTTg==", 2024-04-25T10:09:19.098 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:09:19.098 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:09:19.098 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-04-25T10:09:19.098 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:09:19.098 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.098 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:19.098 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-04-25T10:09:19.098 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:09:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.080+0000 7f82243ce640 1 -- 172.21.15.16:0/2943473117 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f820006b4d0 msgr2=0x7f820006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.080+0000 7f82243ce640 1 --2- 172.21.15.16:0/2943473117 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f820006b4d0 0x7f820006d990 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f820c007fd0 tx=0x7f820c009040 comp rx=0 tx=0).stop 2024-04-25T10:09:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.080+0000 7f82243ce640 1 -- 172.21.15.16:0/2943473117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821c0b5b60 msgr2=0x7f821c0cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.080+0000 7f82243ce640 1 --2- 172.21.15.16:0/2943473117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821c0b5b60 0x7f821c0cd430 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8218009870 tx=0x7f82180046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.080+0000 7f82243ce640 1 -- 172.21.15.16:0/2943473117 shutdown_connections 2024-04-25T10:09:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.080+0000 7f82243ce640 1 --2- 172.21.15.16:0/2943473117 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f820006b4d0 0x7f820006d990 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.080+0000 7f82243ce640 1 --2- 172.21.15.16:0/2943473117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f821c15dce0 0x7f821c0c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.080+0000 7f82243ce640 1 --2- 172.21.15.16:0/2943473117 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f821c0d35b0 0x7f821c157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.080+0000 7f82243ce640 1 --2- 172.21.15.16:0/2943473117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821c0b5b60 0x7f821c0cd430 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.080+0000 7f82243ce640 1 -- 172.21.15.16:0/2943473117 >> 172.21.15.16:0/2943473117 conn(0x7f821c072900 msgr2=0x7f821c0b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.080+0000 7f82243ce640 1 -- 172.21.15.16:0/2943473117 shutdown_connections 2024-04-25T10:09:19.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.080+0000 7f82243ce640 1 -- 172.21.15.16:0/2943473117 wait complete. 2024-04-25T10:09:19.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:19.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.199+0000 7f000e8b3640 1 Processor -- start 2024-04-25T10:09:19.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.199+0000 7f000e8b3640 1 -- start start 2024-04-25T10:09:19.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.200+0000 7f000e8b3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f000809a150 0x7f000806bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.200+0000 7f000e8b3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000806c660 con 0x7f000806c1f0 2024-04-25T10:09:19.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.200+0000 7f000e8b3640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000806c900 con 0x7f000815ba10 2024-04-25T10:09:19.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.200+0000 7f000e8b3640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00080cbe50 con 0x7f000809a150 2024-04-25T10:09:19.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.200+0000 7f000ce50640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f000806c1f0 0x7f00080cd220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56072/0 (socket says 172.21.15.16:56072) 2024-04-25T10:09:19.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.200+0000 7f000ce50640 1 -- 172.21.15.16:0/2504943145 learned_addr learned my addr 172.21.15.16:0/2504943145 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:19.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.200+0000 7f00077fe640 1 --2- 172.21.15.16:0/2504943145 >> v2:172.21.15.186:3300/0 conn(0x7f000809a150 0x7f000806bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f0006ffd640 1 -- 172.21.15.16:0/2504943145 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585489128 0 0) 0x7f000806c660 con 0x7f000806c1f0 2024-04-25T10:09:19.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f0006ffd640 1 -- 172.21.15.16:0/2504943145 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efff4003520 con 0x7f000806c1f0 2024-04-25T10:09:19.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f0006ffd640 1 -- 172.21.15.16:0/2504943145 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3773539910 0 0) 0x7efff4003520 con 0x7f000806c1f0 2024-04-25T10:09:19.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f0006ffd640 1 -- 172.21.15.16:0/2504943145 >> v2:172.21.15.186:3300/0 conn(0x7f000809a150 msgr2=0x7f000806bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:19.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f0006ffd640 1 --2- 172.21.15.16:0/2504943145 >> v2:172.21.15.186:3300/0 conn(0x7f000809a150 0x7f000806bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f0006ffd640 1 -- 172.21.15.16:0/2504943145 >> v1:172.21.15.186:6789/0 conn(0x7f000815ba10 legacy=0x7f0008099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f0006ffd640 1 -- 172.21.15.16:0/2504943145 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000806cbe0 con 0x7f000806c1f0 2024-04-25T10:09:19.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f0006ffd640 1 -- 172.21.15.16:0/2504943145 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f00000019f0 con 0x7f000806c1f0 2024-04-25T10:09:19.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f0006ffd640 1 -- 172.21.15.16:0/2504943145 >> v1:172.21.15.16:6789/0 conn(0x7f000806c1f0 legacy=0x7f00080cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f0006ffd640 1 --2- 172.21.15.16:0/2504943145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efff40039b0 0x7efff4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f0006ffd640 1 --2- 172.21.15.16:0/2504943145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff4004800 0x7efff4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f0006ffd640 1 --2- 172.21.15.16:0/2504943145 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efff40051c0 0x7efff400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f0006ffd640 1 -- 172.21.15.16:0/2504943145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff400c030 con 0x7efff4004800 2024-04-25T10:09:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f0006ffd640 1 -- 172.21.15.16:0/2504943145 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff400c1e0 con 0x7efff40039b0 2024-04-25T10:09:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f0006ffd640 1 -- 172.21.15.16:0/2504943145 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff400c3e0 con 0x7efff40051c0 2024-04-25T10:09:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f00077fe640 1 --2- 172.21.15.16:0/2504943145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff4004800 0x7efff4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f0007fff640 1 --2- 172.21.15.16:0/2504943145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efff40039b0 0x7efff4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f000ce50640 1 --2- 172.21.15.16:0/2504943145 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efff40051c0 0x7efff400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f00077fe640 1 -- 172.21.15.16:0/2504943145 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efff40051c0 msgr2=0x7efff400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f00077fe640 1 --2- 172.21.15.16:0/2504943145 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efff40051c0 0x7efff400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f00077fe640 1 -- 172.21.15.16:0/2504943145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efff40039b0 msgr2=0x7efff4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f00077fe640 1 --2- 172.21.15.16:0/2504943145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efff40039b0 0x7efff4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f00077fe640 1 -- 172.21.15.16:0/2504943145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff4003520 con 0x7efff4004800 2024-04-25T10:09:19.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.201+0000 7f000ce50640 1 -- 172.21.15.16:0/2504943145 reap_dead start 2024-04-25T10:09:19.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.202+0000 7f00077fe640 1 --2- 172.21.15.16:0/2504943145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff4004800 0x7efff4004c80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7efff0009ac0 tx=0x7efff4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b441da81cddc75ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:19.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.202+0000 7f0006ffd640 1 -- 172.21.15.16:0/2504943145 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff0036070 con 0x7efff4004800 2024-04-25T10:09:19.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.202+0000 7f0006ffd640 1 -- 172.21.15.16:0/2504943145 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff0031400 con 0x7efff4004800 2024-04-25T10:09:19.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.202+0000 7f0006ffd640 1 -- 172.21.15.16:0/2504943145 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff0009690 con 0x7efff4004800 2024-04-25T10:09:19.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.203+0000 7f000e8b3640 1 -- 172.21.15.16:0/2504943145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff4004800 msgr2=0x7efff4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.203+0000 7f000e8b3640 1 --2- 172.21.15.16:0/2504943145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff4004800 0x7efff4004c80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7efff0009ac0 tx=0x7efff4005830 comp rx=0 tx=0).stop 2024-04-25T10:09:19.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.203+0000 7f000e8b3640 1 -- 172.21.15.16:0/2504943145 shutdown_connections 2024-04-25T10:09:19.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.203+0000 7f000e8b3640 1 --2- 172.21.15.16:0/2504943145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff4004800 0x7efff4004c80 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.203+0000 7f000e8b3640 1 -- 172.21.15.16:0/2504943145 >> 172.21.15.16:0/2504943145 conn(0x7f00080b32f0 msgr2=0x7f00080b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:19.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.203+0000 7f000e8b3640 1 -- 172.21.15.16:0/2504943145 shutdown_connections 2024-04-25T10:09:19.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.203+0000 7f000e8b3640 1 -- 172.21.15.16:0/2504943145 wait complete. 2024-04-25T10:09:19.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.204+0000 7f000e8b3640 1 Processor -- start 2024-04-25T10:09:19.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.204+0000 7f000e8b3640 1 -- start start 2024-04-25T10:09:19.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.205+0000 7f000e8b3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efff4004800 0x7f00080c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.205+0000 7f000e8b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00081487a0 0x7f0008147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.205+0000 7f000e8b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008147fe0 0x7f0008145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.205+0000 7f000e8b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000806bcb0 con 0x7f0008147fe0 2024-04-25T10:09:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.205+0000 7f000e8b3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00080cfe80 con 0x7efff4004800 2024-04-25T10:09:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.205+0000 7f000e8b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000806be30 con 0x7f00081487a0 2024-04-25T10:09:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.205+0000 7f000ce50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008147fe0 0x7f0008145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.205+0000 7f000ce50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008147fe0 0x7f0008145a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39818/0 (socket says 172.21.15.16:39818) 2024-04-25T10:09:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.205+0000 7f000ce50640 1 -- 172.21.15.16:0/3536377699 learned_addr learned my addr 172.21.15.16:0/3536377699 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.205+0000 7f00077fe640 1 --2- 172.21.15.16:0/3536377699 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00081487a0 0x7f0008147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.205+0000 7f0007fff640 1 --2- 172.21.15.16:0/3536377699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efff4004800 0x7f00080c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.205+0000 7f000ce50640 1 -- 172.21.15.16:0/3536377699 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00081487a0 msgr2=0x7f0008147aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.205+0000 7f000ce50640 1 --2- 172.21.15.16:0/3536377699 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00081487a0 0x7f0008147aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.205+0000 7f000ce50640 1 -- 172.21.15.16:0/3536377699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efff4004800 msgr2=0x7f00080c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.205+0000 7f000ce50640 1 --2- 172.21.15.16:0/3536377699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efff4004800 0x7f00080c7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.205+0000 7f000ce50640 1 -- 172.21.15.16:0/3536377699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0008148460 con 0x7f0008147fe0 2024-04-25T10:09:19.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.206+0000 7f000ce50640 1 --2- 172.21.15.16:0/3536377699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008147fe0 0x7f0008145a90 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0008072530 tx=0x7f0000007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:19.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.206+0000 7f0004ff9640 1 -- 172.21.15.16:0/3536377699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0000017020 con 0x7f0008147fe0 2024-04-25T10:09:19.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.206+0000 7f0004ff9640 1 -- 172.21.15.16:0/3536377699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000000abc0 con 0x7f0008147fe0 2024-04-25T10:09:19.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.206+0000 7f000e8b3640 1 -- 172.21.15.16:0/3536377699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00081557b0 con 0x7f0008147fe0 2024-04-25T10:09:19.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.206+0000 7f0004ff9640 1 -- 172.21.15.16:0/3536377699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00000133f0 con 0x7f0008147fe0 2024-04-25T10:09:19.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.206+0000 7f000e8b3640 1 -- 172.21.15.16:0/3536377699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00080c6e30 con 0x7f0008147fe0 2024-04-25T10:09:19.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.207+0000 7f000e8b3640 1 -- 172.21.15.16:0/3536377699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00080749b0 con 0x7f0008147fe0 2024-04-25T10:09:19.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.207+0000 7f0004ff9640 1 -- 172.21.15.16:0/3536377699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f000001a020 con 0x7f0008147fe0 2024-04-25T10:09:19.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.208+0000 7f0004ff9640 1 --2- 172.21.15.16:0/3536377699 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7effe806b500 0x7effe806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.208+0000 7f0004ff9640 1 -- 172.21.15.16:0/3536377699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f000008a6b0 con 0x7f0008147fe0 2024-04-25T10:09:19.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.208+0000 7f0007fff640 1 --2- 172.21.15.16:0/3536377699 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7effe806b500 0x7effe806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.208+0000 7f0007fff640 1 --2- 172.21.15.16:0/3536377699 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7effe806b500 0x7effe806d9c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efff8000c00 tx=0x7efff8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:19.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.212+0000 7f0004ff9640 1 -- 172.21.15.16:0/3536377699 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0000021020 con 0x7f0008147fe0 2024-04-25T10:09:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.394+0000 7f000e8b3640 1 -- 172.21.15.16:0/3536377699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f000809a570 con 0x7f0008147fe0 2024-04-25T10:09:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.395+0000 7f0004ff9640 1 -- 172.21.15.16:0/3536377699 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+12891 (secure 0 0 0) 0x7f0000059910 con 0x7f0008147fe0 2024-04-25T10:09:19.402 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:19.402 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":68,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:09:18.700907+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.397+0000 7f000e8b3640 1 -- 172.21.15.16:0/3536377699 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7effe806b500 msgr2=0x7effe806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.397+0000 7f000e8b3640 1 --2- 172.21.15.16:0/3536377699 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7effe806b500 0x7effe806d9c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efff8000c00 tx=0x7efff8009040 comp rx=0 tx=0).stop 2024-04-25T10:09:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.397+0000 7f000e8b3640 1 -- 172.21.15.16:0/3536377699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008147fe0 msgr2=0x7f0008145a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.397+0000 7f000e8b3640 1 --2- 172.21.15.16:0/3536377699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008147fe0 0x7f0008145a90 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0008072530 tx=0x7f0000007b20 comp rx=0 tx=0).stop 2024-04-25T10:09:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.397+0000 7f000e8b3640 1 -- 172.21.15.16:0/3536377699 shutdown_connections 2024-04-25T10:09:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.397+0000 7f000e8b3640 1 --2- 172.21.15.16:0/3536377699 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7effe806b500 0x7effe806d9c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.397+0000 7f000e8b3640 1 --2- 172.21.15.16:0/3536377699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008147fe0 0x7f0008145a90 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.397+0000 7f000e8b3640 1 --2- 172.21.15.16:0/3536377699 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00081487a0 0x7f0008147aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.397+0000 7f000e8b3640 1 --2- 172.21.15.16:0/3536377699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efff4004800 0x7f00080c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.397+0000 7f000e8b3640 1 -- 172.21.15.16:0/3536377699 >> 172.21.15.16:0/3536377699 conn(0x7f00080b32f0 msgr2=0x7f0008154c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.397+0000 7f000e8b3640 1 -- 172.21.15.16:0/3536377699 shutdown_connections 2024-04-25T10:09:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.397+0000 7f000e8b3640 1 -- 172.21.15.16:0/3536377699 wait complete. 2024-04-25T10:09:19.415 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:19.415 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:19.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.511+0000 7f128419e640 1 Processor -- start 2024-04-25T10:09:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.512+0000 7f128419e640 1 -- start start 2024-04-25T10:09:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.512+0000 7f128419e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f127c15deb0 0x7f127c0b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.512+0000 7f128419e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f127c06b2b0 0x7f127c0cae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.512+0000 7f128419e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f127c0cd150 con 0x7f127c0b1d30 2024-04-25T10:09:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.512+0000 7f128419e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f127c0b21d0 con 0x7f127c15deb0 2024-04-25T10:09:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.512+0000 7f128419e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127c0cd430 con 0x7f127c06b2b0 2024-04-25T10:09:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.513+0000 7f1281f3a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f127c15deb0 0x7f127c0b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.513+0000 7f1281739640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f127c0b1d30 0x7f127c06aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56076/0 (socket says 172.21.15.16:56076) 2024-04-25T10:09:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.513+0000 7f1281739640 1 -- 172.21.15.16:0/1938584194 learned_addr learned my addr 172.21.15.16:0/1938584194 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.513+0000 7f128273b640 1 --2- 172.21.15.16:0/1938584194 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f127c06b2b0 0x7f127c0cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.513+0000 7f1281f3a640 1 -- 172.21.15.16:0/1938584194 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f127c06b2b0 msgr2=0x7f127c0cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.513+0000 7f1281f3a640 1 --2- 172.21.15.16:0/1938584194 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f127c06b2b0 0x7f127c0cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.513+0000 7f1281f3a640 1 -- 172.21.15.16:0/1938584194 >> v1:172.21.15.16:6789/0 conn(0x7f127c0b1d30 legacy=0x7f127c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.513+0000 7f1281f3a640 1 -- 172.21.15.16:0/1938584194 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127c0cd710 con 0x7f127c15deb0 2024-04-25T10:09:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.513+0000 7f1280f38640 1 -- 172.21.15.16:0/1938584194 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318026122 0 0) 0x7f127c0cd150 con 0x7f127c0b1d30 2024-04-25T10:09:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.514+0000 7f1281f3a640 1 --2- 172.21.15.16:0/1938584194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c15deb0 0x7f127c0b17f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f12740097a0 tx=0x7f127402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffa7fbf6dc954e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.514+0000 7f1280f38640 1 -- 172.21.15.16:0/1938584194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274037070 con 0x7f127c15deb0 2024-04-25T10:09:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.514+0000 7f1280f38640 1 -- 172.21.15.16:0/1938584194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127402ae50 con 0x7f127c15deb0 2024-04-25T10:09:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.514+0000 7f1280f38640 1 -- 172.21.15.16:0/1938584194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274032660 con 0x7f127c15deb0 2024-04-25T10:09:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.514+0000 7f128419e640 1 -- 172.21.15.16:0/1938584194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c15deb0 msgr2=0x7f127c0b17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.514+0000 7f128419e640 1 --2- 172.21.15.16:0/1938584194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c15deb0 0x7f127c0b17f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f12740097a0 tx=0x7f127402a100 comp rx=0 tx=0).stop 2024-04-25T10:09:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.514+0000 7f128419e640 1 -- 172.21.15.16:0/1938584194 shutdown_connections 2024-04-25T10:09:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.514+0000 7f128419e640 1 --2- 172.21.15.16:0/1938584194 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f127c06b2b0 0x7f127c0cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.514+0000 7f128419e640 1 --2- 172.21.15.16:0/1938584194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c15deb0 0x7f127c0b17f0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.514+0000 7f128419e640 1 -- 172.21.15.16:0/1938584194 >> 172.21.15.16:0/1938584194 conn(0x7f127c0b2390 msgr2=0x7f127c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.514+0000 7f128419e640 1 -- 172.21.15.16:0/1938584194 shutdown_connections 2024-04-25T10:09:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.514+0000 7f128419e640 1 -- 172.21.15.16:0/1938584194 wait complete. 2024-04-25T10:09:19.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.516+0000 7f128419e640 1 Processor -- start 2024-04-25T10:09:19.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.516+0000 7f128419e640 1 -- start start 2024-04-25T10:09:19.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.516+0000 7f128419e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f127c06b2b0 0x7f127c0b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.516+0000 7f128419e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c0b1d30 0x7f127c0c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.516+0000 7f128419e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f127c15deb0 0x7f127c0c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.516+0000 7f128419e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c15e410 con 0x7f127c0b1d30 2024-04-25T10:09:19.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.516+0000 7f128419e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c15e290 con 0x7f127c15deb0 2024-04-25T10:09:19.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.516+0000 7f128419e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127c0ccc40 con 0x7f127c06b2b0 2024-04-25T10:09:19.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.516+0000 7f1281739640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c0b1d30 0x7f127c0c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.516+0000 7f1281739640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c0b1d30 0x7f127c0c7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39838/0 (socket says 172.21.15.16:39838) 2024-04-25T10:09:19.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.516+0000 7f1281739640 1 -- 172.21.15.16:0/4243292119 learned_addr learned my addr 172.21.15.16:0/4243292119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:19.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.517+0000 7f128273b640 1 --2- 172.21.15.16:0/4243292119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f127c15deb0 0x7f127c0c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.517+0000 7f1281f3a640 1 --2- 172.21.15.16:0/4243292119 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f127c06b2b0 0x7f127c0b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.517+0000 7f1281739640 1 -- 172.21.15.16:0/4243292119 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f127c06b2b0 msgr2=0x7f127c0b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.517+0000 7f1281739640 1 --2- 172.21.15.16:0/4243292119 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f127c06b2b0 0x7f127c0b5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.517+0000 7f1281739640 1 -- 172.21.15.16:0/4243292119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f127c15deb0 msgr2=0x7f127c0c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.517+0000 7f1281739640 1 --2- 172.21.15.16:0/4243292119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f127c15deb0 0x7f127c0c7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.517+0000 7f1281739640 1 -- 172.21.15.16:0/4243292119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127c0b47b0 con 0x7f127c0b1d30 2024-04-25T10:09:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.517+0000 7f1281739640 1 --2- 172.21.15.16:0/4243292119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c0b1d30 0x7f127c0c7910 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f127000c470 tx=0x7f127000c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.517+0000 7f127a7fc640 1 -- 172.21.15.16:0/4243292119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1270017070 con 0x7f127c0b1d30 2024-04-25T10:09:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.517+0000 7f127a7fc640 1 -- 172.21.15.16:0/4243292119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1270005ce0 con 0x7f127c0b1d30 2024-04-25T10:09:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.517+0000 7f128419e640 1 -- 172.21.15.16:0/4243292119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f127c0b4a90 con 0x7f127c0b1d30 2024-04-25T10:09:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.517+0000 7f127a7fc640 1 -- 172.21.15.16:0/4243292119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12700072b0 con 0x7f127c0b1d30 2024-04-25T10:09:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.517+0000 7f128419e640 1 -- 172.21.15.16:0/4243292119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f127c0b5050 con 0x7f127c0b1d30 2024-04-25T10:09:19.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.518+0000 7f128419e640 1 -- 172.21.15.16:0/4243292119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f127c0b5d90 con 0x7f127c0b1d30 2024-04-25T10:09:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.518+0000 7f127a7fc640 1 -- 172.21.15.16:0/4243292119 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f127001a070 con 0x7f127c0b1d30 2024-04-25T10:09:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.519+0000 7f127a7fc640 1 --2- 172.21.15.16:0/4243292119 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f126806b500 0x7f126806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.519+0000 7f127a7fc640 1 -- 172.21.15.16:0/4243292119 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1270089210 con 0x7f127c0b1d30 2024-04-25T10:09:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.519+0000 7f1281f3a640 1 --2- 172.21.15.16:0/4243292119 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f126806b500 0x7f126806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.520+0000 7f1281f3a640 1 --2- 172.21.15.16:0/4243292119 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f126806b500 0x7f126806d9c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1274034000 tx=0x7f127402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:19.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.523+0000 7f127a7fc640 1 -- 172.21.15.16:0/4243292119 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f12700583e0 con 0x7f127c0b1d30 2024-04-25T10:09:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.707+0000 7f128419e640 1 -- 172.21.15.16:0/4243292119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f127c0750b0 con 0x7f127c0b1d30 2024-04-25T10:09:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.707+0000 7f127a7fc640 1 -- 172.21.15.16:0/4243292119 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1270057ce0 con 0x7f127c0b1d30 2024-04-25T10:09:19.719 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:09:19.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.709+0000 7f128419e640 1 -- 172.21.15.16:0/4243292119 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f126806b500 msgr2=0x7f126806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.709+0000 7f128419e640 1 --2- 172.21.15.16:0/4243292119 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f126806b500 0x7f126806d9c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1274034000 tx=0x7f127402e000 comp rx=0 tx=0).stop 2024-04-25T10:09:19.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.709+0000 7f128419e640 1 -- 172.21.15.16:0/4243292119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c0b1d30 msgr2=0x7f127c0c7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.709+0000 7f128419e640 1 --2- 172.21.15.16:0/4243292119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c0b1d30 0x7f127c0c7910 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f127000c470 tx=0x7f127000c930 comp rx=0 tx=0).stop 2024-04-25T10:09:19.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.709+0000 7f128419e640 1 -- 172.21.15.16:0/4243292119 shutdown_connections 2024-04-25T10:09:19.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.709+0000 7f128419e640 1 --2- 172.21.15.16:0/4243292119 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f126806b500 0x7f126806d9c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.709+0000 7f128419e640 1 --2- 172.21.15.16:0/4243292119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f127c15deb0 0x7f127c0c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.709+0000 7f128419e640 1 --2- 172.21.15.16:0/4243292119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c0b1d30 0x7f127c0c7910 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.709+0000 7f128419e640 1 --2- 172.21.15.16:0/4243292119 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f127c06b2b0 0x7f127c0b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.709+0000 7f128419e640 1 -- 172.21.15.16:0/4243292119 >> 172.21.15.16:0/4243292119 conn(0x7f127c0b2390 msgr2=0x7f127c06e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:19.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.709+0000 7f128419e640 1 -- 172.21.15.16:0/4243292119 shutdown_connections 2024-04-25T10:09:19.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.709+0000 7f128419e640 1 -- 172.21.15.16:0/4243292119 wait complete. 2024-04-25T10:09:19.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:09:19.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.822+0000 7f2360fd5640 1 Processor -- start 2024-04-25T10:09:19.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.822+0000 7f2360fd5640 1 -- start start 2024-04-25T10:09:19.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.823+0000 7f2360fd5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f235c09a9c0 0x7f235c166910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.823+0000 7f2360fd5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235c0cfb90 con 0x7f235c099a40 2024-04-25T10:09:19.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.823+0000 7f2360fd5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235c15c210 con 0x7f235c15bd40 2024-04-25T10:09:19.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.823+0000 7f2360fd5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c15cfd0 con 0x7f235c09a9c0 2024-04-25T10:09:19.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.823+0000 7f235a59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f235c099a40 0x7f235c0cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56082/0 (socket says 172.21.15.16:56082) 2024-04-25T10:09:19.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.823+0000 7f235a59c640 1 -- 172.21.15.16:0/1891109650 learned_addr learned my addr 172.21.15.16:0/1891109650 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:19.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.823+0000 7f235ad9d640 1 --2- 172.21.15.16:0/1891109650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f235c09a9c0 0x7f235c166910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.823+0000 7f2359d9b640 1 -- 172.21.15.16:0/1891109650 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701298426 0 0) 0x7f235c0cfb90 con 0x7f235c099a40 2024-04-25T10:09:19.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.823+0000 7f2359d9b640 1 -- 172.21.15.16:0/1891109650 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2348003520 con 0x7f235c099a40 2024-04-25T10:09:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.823+0000 7f2359d9b640 1 -- 172.21.15.16:0/1891109650 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1416875396 0 0) 0x7f235c15c210 con 0x7f235c15bd40 2024-04-25T10:09:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.823+0000 7f2359d9b640 1 -- 172.21.15.16:0/1891109650 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f235c0cfb90 con 0x7f235c15bd40 2024-04-25T10:09:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.823+0000 7f2359d9b640 1 -- 172.21.15.16:0/1891109650 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1791625938 0 0) 0x7f2348003520 con 0x7f235c099a40 2024-04-25T10:09:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.823+0000 7f235ad9d640 1 -- 172.21.15.16:0/1891109650 >> v1:172.21.15.186:6789/0 conn(0x7f235c15bd40 legacy=0x7f235c099e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.823+0000 7f235ad9d640 1 -- 172.21.15.16:0/1891109650 >> v1:172.21.15.16:6789/0 conn(0x7f235c099a40 legacy=0x7f235c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.823+0000 7f235ad9d640 1 -- 172.21.15.16:0/1891109650 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c15c4f0 con 0x7f235c09a9c0 2024-04-25T10:09:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.824+0000 7f235ad9d640 1 --2- 172.21.15.16:0/1891109650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f235c09a9c0 0x7f235c166910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2354009dc0 tx=0x7f235402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b29a4194902e9cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.824+0000 7f2359d9b640 1 -- 172.21.15.16:0/1891109650 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c001690 con 0x7f235c09a9c0 2024-04-25T10:09:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.824+0000 7f2359d9b640 1 -- 172.21.15.16:0/1891109650 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2354007540 con 0x7f235c09a9c0 2024-04-25T10:09:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.824+0000 7f2359d9b640 1 -- 172.21.15.16:0/1891109650 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23540078f0 con 0x7f235c09a9c0 2024-04-25T10:09:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.825+0000 7f2360fd5640 1 -- 172.21.15.16:0/1891109650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f235c09a9c0 msgr2=0x7f235c166910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.825+0000 7f2360fd5640 1 --2- 172.21.15.16:0/1891109650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f235c09a9c0 0x7f235c166910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2354009dc0 tx=0x7f235402a660 comp rx=0 tx=0).stop 2024-04-25T10:09:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.825+0000 7f2360fd5640 1 -- 172.21.15.16:0/1891109650 shutdown_connections 2024-04-25T10:09:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.825+0000 7f2360fd5640 1 --2- 172.21.15.16:0/1891109650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f235c09a9c0 0x7f235c166910 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.825+0000 7f2360fd5640 1 -- 172.21.15.16:0/1891109650 >> 172.21.15.16:0/1891109650 conn(0x7f235c0b32f0 msgr2=0x7f235c0b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:19.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.825+0000 7f2360fd5640 1 -- 172.21.15.16:0/1891109650 shutdown_connections 2024-04-25T10:09:19.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.825+0000 7f2360fd5640 1 -- 172.21.15.16:0/1891109650 wait complete. 2024-04-25T10:09:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.826+0000 7f2360fd5640 1 Processor -- start 2024-04-25T10:09:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.826+0000 7f2360fd5640 1 -- start start 2024-04-25T10:09:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.827+0000 7f2360fd5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f235c099a40 0x7f235c0c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.827+0000 7f2360fd5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f235c09a9c0 0x7f235c153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.827+0000 7f2360fd5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f235c15bd40 0x7f235c15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.827+0000 7f2360fd5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c15cab0 con 0x7f235c099a40 2024-04-25T10:09:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.827+0000 7f2360fd5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c09ae10 con 0x7f235c09a9c0 2024-04-25T10:09:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.827+0000 7f2360fd5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c0cd400 con 0x7f235c15bd40 2024-04-25T10:09:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.827+0000 7f235a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f235c099a40 0x7f235c0c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.827+0000 7f235a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f235c099a40 0x7f235c0c6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39840/0 (socket says 172.21.15.16:39840) 2024-04-25T10:09:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.827+0000 7f235a59c640 1 -- 172.21.15.16:0/3995940058 learned_addr learned my addr 172.21.15.16:0/3995940058 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.827+0000 7f235ad9d640 1 --2- 172.21.15.16:0/3995940058 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f235c15bd40 0x7f235c15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.827+0000 7f2353fff640 1 --2- 172.21.15.16:0/3995940058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f235c09a9c0 0x7f235c153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.827+0000 7f235a59c640 1 -- 172.21.15.16:0/3995940058 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f235c15bd40 msgr2=0x7f235c15aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.827+0000 7f235a59c640 1 --2- 172.21.15.16:0/3995940058 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f235c15bd40 0x7f235c15aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.827+0000 7f235a59c640 1 -- 172.21.15.16:0/3995940058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f235c09a9c0 msgr2=0x7f235c153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.827+0000 7f235a59c640 1 --2- 172.21.15.16:0/3995940058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f235c09a9c0 0x7f235c153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.827+0000 7f235a59c640 1 -- 172.21.15.16:0/3995940058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c0c5580 con 0x7f235c099a40 2024-04-25T10:09:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.827+0000 7f235ad9d640 1 --2- 172.21.15.16:0/3995940058 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f235c15bd40 0x7f235c15aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.828+0000 7f235a59c640 1 --2- 172.21.15.16:0/3995940058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f235c099a40 0x7f235c0c6620 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f234c001200 tx=0x7f234c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.828+0000 7f2352ffd640 1 -- 172.21.15.16:0/3995940058 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c018070 con 0x7f235c099a40 2024-04-25T10:09:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.828+0000 7f2352ffd640 1 -- 172.21.15.16:0/3995940058 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234c0074e0 con 0x7f235c099a40 2024-04-25T10:09:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.828+0000 7f2360fd5640 1 -- 172.21.15.16:0/3995940058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235c0c5860 con 0x7f235c099a40 2024-04-25T10:09:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.828+0000 7f2352ffd640 1 -- 172.21.15.16:0/3995940058 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c004a20 con 0x7f235c099a40 2024-04-25T10:09:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.828+0000 7f2360fd5640 1 -- 172.21.15.16:0/3995940058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235c0c5e20 con 0x7f235c099a40 2024-04-25T10:09:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.829+0000 7f2360fd5640 1 -- 172.21.15.16:0/3995940058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235c0749b0 con 0x7f235c099a40 2024-04-25T10:09:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.829+0000 7f2352ffd640 1 -- 172.21.15.16:0/3995940058 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f234c01b070 con 0x7f235c099a40 2024-04-25T10:09:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.830+0000 7f2352ffd640 1 --2- 172.21.15.16:0/3995940058 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f233c06b4d0 0x7f233c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.830+0000 7f2352ffd640 1 -- 172.21.15.16:0/3995940058 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f234c0891d0 con 0x7f235c099a40 2024-04-25T10:09:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.830+0000 7f2353fff640 1 --2- 172.21.15.16:0/3995940058 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f233c06b4d0 0x7f233c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.830+0000 7f2353fff640 1 --2- 172.21.15.16:0/3995940058 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f233c06b4d0 0x7f233c06d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f235c15b560 tx=0x7f2344009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:19.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:19.834+0000 7f2352ffd640 1 -- 172.21.15.16:0/3995940058 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f234c058430 con 0x7f235c099a40 2024-04-25T10:09:20.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.019+0000 7f2360fd5640 1 -- 172.21.15.16:0/3995940058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f235c0b4120 con 0x7f235c099a40 2024-04-25T10:09:20.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.020+0000 7f2352ffd640 1 -- 172.21.15.16:0/3995940058 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) v1 ==== 84+0+65 (secure 0 0 0) 0x7f234c057d30 con 0x7f235c099a40 2024-04-25T10:09:20.027 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:20.027 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-04-25T10:09:20.027 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:09:20.027 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-04-25T10:09:20.027 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-04-25T10:09:20.028 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:20.028 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-04-25T10:09:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.022+0000 7f2360fd5640 1 -- 172.21.15.16:0/3995940058 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f233c06b4d0 msgr2=0x7f233c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.022+0000 7f2360fd5640 1 --2- 172.21.15.16:0/3995940058 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f233c06b4d0 0x7f233c06d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f235c15b560 tx=0x7f2344009040 comp rx=0 tx=0).stop 2024-04-25T10:09:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.022+0000 7f2360fd5640 1 -- 172.21.15.16:0/3995940058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f235c099a40 msgr2=0x7f235c0c6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.022+0000 7f2360fd5640 1 --2- 172.21.15.16:0/3995940058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f235c099a40 0x7f235c0c6620 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f234c001200 tx=0x7f234c00c9b0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.022+0000 7f2360fd5640 1 -- 172.21.15.16:0/3995940058 shutdown_connections 2024-04-25T10:09:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.022+0000 7f2360fd5640 1 --2- 172.21.15.16:0/3995940058 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f233c06b4d0 0x7f233c06d990 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.022+0000 7f2360fd5640 1 --2- 172.21.15.16:0/3995940058 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f235c15bd40 0x7f235c15aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.022+0000 7f2360fd5640 1 --2- 172.21.15.16:0/3995940058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f235c09a9c0 0x7f235c153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.022+0000 7f2360fd5640 1 --2- 172.21.15.16:0/3995940058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f235c099a40 0x7f235c0c6620 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.022+0000 7f2360fd5640 1 -- 172.21.15.16:0/3995940058 >> 172.21.15.16:0/3995940058 conn(0x7f235c0b32f0 msgr2=0x7f235c15f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.022+0000 7f2360fd5640 1 -- 172.21.15.16:0/3995940058 shutdown_connections 2024-04-25T10:09:20.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.022+0000 7f2360fd5640 1 -- 172.21.15.16:0/3995940058 wait complete. 2024-04-25T10:09:20.040 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:09:20.040 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:09:20.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.137+0000 7f4b56de7640 1 Processor -- start 2024-04-25T10:09:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.137+0000 7f4b56de7640 1 -- start start 2024-04-25T10:09:20.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.138+0000 7f4b56de7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b50161090 0x7f4b50161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.138+0000 7f4b56de7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4b500b9720 0x7f4b500ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.138+0000 7f4b56de7640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b50161f80 con 0x7f4b500cece0 2024-04-25T10:09:20.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.138+0000 7f4b56de7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b500cf130 con 0x7f4b500b9720 2024-04-25T10:09:20.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.138+0000 7f4b56de7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500d21a0 con 0x7f4b50161090 2024-04-25T10:09:20.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.138+0000 7f4b4ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4b500b9720 0x7f4b500ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.138+0000 7f4b4ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4b500b9720 0x7f4b500ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39854/0 (socket says 172.21.15.16:39854) 2024-04-25T10:09:20.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.138+0000 7f4b4ffff640 1 -- 172.21.15.16:0/3726771999 learned_addr learned my addr 172.21.15.16:0/3726771999 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:20.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.139+0000 7f4b54b83640 1 --2- 172.21.15.16:0/3726771999 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b50161090 0x7f4b50161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.139+0000 7f4b4ffff640 1 -- 172.21.15.16:0/3726771999 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b50161090 msgr2=0x7f4b50161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.139+0000 7f4b4ffff640 1 --2- 172.21.15.16:0/3726771999 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b50161090 0x7f4b50161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.139+0000 7f4b4ffff640 1 -- 172.21.15.16:0/3726771999 >> v1:172.21.15.186:6789/0 conn(0x7f4b500cece0 legacy=0x7f4b5006c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.139+0000 7f4b4ffff640 1 -- 172.21.15.16:0/3726771999 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b50162260 con 0x7f4b500b9720 2024-04-25T10:09:20.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.139+0000 7f4b4ffff640 1 --2- 172.21.15.16:0/3726771999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b500b9720 0x7f4b500ce7a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009870 tx=0x7f4b4402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b13e3b84147fcbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.140+0000 7f4b4f7fe640 1 -- 172.21.15.16:0/3726771999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44037070 con 0x7f4b500b9720 2024-04-25T10:09:20.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.140+0000 7f4b4f7fe640 1 -- 172.21.15.16:0/3726771999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b44032440 con 0x7f4b500b9720 2024-04-25T10:09:20.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.140+0000 7f4b4f7fe640 1 -- 172.21.15.16:0/3726771999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44032810 con 0x7f4b500b9720 2024-04-25T10:09:20.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.140+0000 7f4b56de7640 1 -- 172.21.15.16:0/3726771999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b500b9720 msgr2=0x7f4b500ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.140+0000 7f4b56de7640 1 --2- 172.21.15.16:0/3726771999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b500b9720 0x7f4b500ce7a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009870 tx=0x7f4b4402a270 comp rx=0 tx=0).stop 2024-04-25T10:09:20.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.140+0000 7f4b56de7640 1 -- 172.21.15.16:0/3726771999 shutdown_connections 2024-04-25T10:09:20.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.140+0000 7f4b56de7640 1 --2- 172.21.15.16:0/3726771999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b500b9720 0x7f4b500ce7a0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.140+0000 7f4b56de7640 1 --2- 172.21.15.16:0/3726771999 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b50161090 0x7f4b50161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.140+0000 7f4b56de7640 1 -- 172.21.15.16:0/3726771999 >> 172.21.15.16:0/3726771999 conn(0x7f4b500b55e0 msgr2=0x7f4b500b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.140+0000 7f4b56de7640 1 -- 172.21.15.16:0/3726771999 shutdown_connections 2024-04-25T10:09:20.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.140+0000 7f4b56de7640 1 -- 172.21.15.16:0/3726771999 wait complete. 2024-04-25T10:09:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.141+0000 7f4b56de7640 1 Processor -- start 2024-04-25T10:09:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.141+0000 7f4b56de7640 1 -- start start 2024-04-25T10:09:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.142+0000 7f4b56de7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b500b9720 0x7f4b501f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.142+0000 7f4b56de7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b500cece0 0x7f4b501f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.142+0000 7f4b56de7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b50161090 0x7f4b500c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.142+0000 7f4b56de7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b50161470 con 0x7f4b500cece0 2024-04-25T10:09:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.142+0000 7f4b56de7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500d1e30 con 0x7f4b50161090 2024-04-25T10:09:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.142+0000 7f4b56de7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b50161c70 con 0x7f4b500b9720 2024-04-25T10:09:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.142+0000 7f4b4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b500cece0 0x7f4b501f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.142+0000 7f4b4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b500cece0 0x7f4b501f3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39862/0 (socket says 172.21.15.16:39862) 2024-04-25T10:09:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.142+0000 7f4b4ffff640 1 -- 172.21.15.16:0/63139806 learned_addr learned my addr 172.21.15.16:0/63139806 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.143+0000 7f4b54b83640 1 --2- 172.21.15.16:0/63139806 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b500b9720 0x7f4b501f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.143+0000 7f4b55384640 1 --2- 172.21.15.16:0/63139806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b50161090 0x7f4b500c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.143+0000 7f4b4ffff640 1 -- 172.21.15.16:0/63139806 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b500b9720 msgr2=0x7f4b501f80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.143+0000 7f4b4ffff640 1 --2- 172.21.15.16:0/63139806 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b500b9720 0x7f4b501f80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.143+0000 7f4b4ffff640 1 -- 172.21.15.16:0/63139806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b50161090 msgr2=0x7f4b500c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.143+0000 7f4b4ffff640 1 --2- 172.21.15.16:0/63139806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b50161090 0x7f4b500c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.143+0000 7f4b4ffff640 1 -- 172.21.15.16:0/63139806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b501f51e0 con 0x7f4b500cece0 2024-04-25T10:09:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.143+0000 7f4b55384640 1 --2- 172.21.15.16:0/63139806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b50161090 0x7f4b500c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.143+0000 7f4b4ffff640 1 --2- 172.21.15.16:0/63139806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b500cece0 0x7f4b501f3b40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4b4402a2b0 tx=0x7f4b440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.143+0000 7f4b4d7fa640 1 -- 172.21.15.16:0/63139806 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44041020 con 0x7f4b500cece0 2024-04-25T10:09:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.143+0000 7f4b4d7fa640 1 -- 172.21.15.16:0/63139806 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b440306c0 con 0x7f4b500cece0 2024-04-25T10:09:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.143+0000 7f4b56de7640 1 -- 172.21.15.16:0/63139806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b501f45d0 con 0x7f4b500cece0 2024-04-25T10:09:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.143+0000 7f4b4d7fa640 1 -- 172.21.15.16:0/63139806 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44037040 con 0x7f4b500cece0 2024-04-25T10:09:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.143+0000 7f4b56de7640 1 -- 172.21.15.16:0/63139806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b501f4b90 con 0x7f4b500cece0 2024-04-25T10:09:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.144+0000 7f4b56de7640 1 -- 172.21.15.16:0/63139806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b500750b0 con 0x7f4b500cece0 2024-04-25T10:09:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.145+0000 7f4b4d7fa640 1 -- 172.21.15.16:0/63139806 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b44044020 con 0x7f4b500cece0 2024-04-25T10:09:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.145+0000 7f4b4d7fa640 1 --2- 172.21.15.16:0/63139806 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4b3c06b500 0x7f4b3c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.145+0000 7f4b4d7fa640 1 -- 172.21.15.16:0/63139806 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4b440aa5e0 con 0x7f4b500cece0 2024-04-25T10:09:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.145+0000 7f4b54b83640 1 --2- 172.21.15.16:0/63139806 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4b3c06b500 0x7f4b3c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.146+0000 7f4b54b83640 1 --2- 172.21.15.16:0/63139806 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4b3c06b500 0x7f4b3c06d9c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4b50067d80 tx=0x7f4b48009800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.149+0000 7f4b4d7fa640 1 -- 172.21.15.16:0/63139806 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4b4404b030 con 0x7f4b500cece0 2024-04-25T10:09:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.328+0000 7f4b56de7640 1 -- 172.21.15.16:0/63139806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4b501607b0 con 0x7f4b500cece0 2024-04-25T10:09:20.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.693+0000 7f4b4d7fa640 1 -- 172.21.15.16:0/63139806 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v69) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4b440797b0 con 0x7f4b500cece0 2024-04-25T10:09:20.700 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-04-25T10:09:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.696+0000 7f4b56de7640 1 -- 172.21.15.16:0/63139806 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4b3c06b500 msgr2=0x7f4b3c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.696+0000 7f4b56de7640 1 --2- 172.21.15.16:0/63139806 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4b3c06b500 0x7f4b3c06d9c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4b50067d80 tx=0x7f4b48009800 comp rx=0 tx=0).stop 2024-04-25T10:09:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.696+0000 7f4b56de7640 1 -- 172.21.15.16:0/63139806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b500cece0 msgr2=0x7f4b501f3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.696+0000 7f4b56de7640 1 --2- 172.21.15.16:0/63139806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b500cece0 0x7f4b501f3b40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4b4402a2b0 tx=0x7f4b440046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.696+0000 7f4b56de7640 1 -- 172.21.15.16:0/63139806 shutdown_connections 2024-04-25T10:09:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.696+0000 7f4b56de7640 1 --2- 172.21.15.16:0/63139806 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4b3c06b500 0x7f4b3c06d9c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.696+0000 7f4b56de7640 1 --2- 172.21.15.16:0/63139806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b50161090 0x7f4b500c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.696+0000 7f4b56de7640 1 --2- 172.21.15.16:0/63139806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b500cece0 0x7f4b501f3b40 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.696+0000 7f4b56de7640 1 --2- 172.21.15.16:0/63139806 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b500b9720 0x7f4b501f80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.696+0000 7f4b56de7640 1 -- 172.21.15.16:0/63139806 >> 172.21.15.16:0/63139806 conn(0x7f4b500b55e0 msgr2=0x7f4b500d3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.696+0000 7f4b56de7640 1 -- 172.21.15.16:0/63139806 shutdown_connections 2024-04-25T10:09:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.696+0000 7f4b56de7640 1 -- 172.21.15.16:0/63139806 wait complete. 2024-04-25T10:09:20.718 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:09:20.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.819+0000 7f04e85d5640 1 Processor -- start 2024-04-25T10:09:20.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.819+0000 7f04e85d5640 1 -- start start 2024-04-25T10:09:20.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.820+0000 7f04e85d5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04e015dd90 0x7f04e00d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.820+0000 7f04e85d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04e00d1a30 0x7f04e00b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.820+0000 7f04e85d5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04e00d2410 con 0x7f04e00d03e0 2024-04-25T10:09:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.820+0000 7f04e85d5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04e00d26f0 con 0x7f04e015dd90 2024-04-25T10:09:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.820+0000 7f04e85d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00d29d0 con 0x7f04e00d1a30 2024-04-25T10:09:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.820+0000 7f04e5b70640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04e015dd90 0x7f04e00d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.820+0000 7f04e5b70640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04e015dd90 0x7f04e00d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39870/0 (socket says 172.21.15.16:39870) 2024-04-25T10:09:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.820+0000 7f04e5b70640 1 -- 172.21.15.16:0/2731171297 learned_addr learned my addr 172.21.15.16:0/2731171297 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.820+0000 7f04e5b70640 1 -- 172.21.15.16:0/2731171297 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04e00d1a30 msgr2=0x7f04e00b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.820+0000 7f04e5b70640 1 --2- 172.21.15.16:0/2731171297 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04e00d1a30 0x7f04e00b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.820+0000 7f04e5b70640 1 -- 172.21.15.16:0/2731171297 >> v1:172.21.15.186:6789/0 conn(0x7f04e00d03e0 legacy=0x7f04e00d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.820+0000 7f04e5b70640 1 -- 172.21.15.16:0/2731171297 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e00d2cb0 con 0x7f04e015dd90 2024-04-25T10:09:20.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.821+0000 7f04e5b70640 1 --2- 172.21.15.16:0/2731171297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e015dd90 0x7f04e00d34b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009850 tx=0x7f04d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67773c4eec558e36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.821+0000 7f04e536f640 1 -- 172.21.15.16:0/2731171297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d4004030 con 0x7f04e015dd90 2024-04-25T10:09:20.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.821+0000 7f04e536f640 1 -- 172.21.15.16:0/2731171297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d40041d0 con 0x7f04e015dd90 2024-04-25T10:09:20.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.821+0000 7f04e536f640 1 -- 172.21.15.16:0/2731171297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d4003610 con 0x7f04e015dd90 2024-04-25T10:09:20.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.822+0000 7f04e85d5640 1 -- 172.21.15.16:0/2731171297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e015dd90 msgr2=0x7f04e00d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.822+0000 7f04e85d5640 1 --2- 172.21.15.16:0/2731171297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e015dd90 0x7f04e00d34b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009850 tx=0x7f04d402a270 comp rx=0 tx=0).stop 2024-04-25T10:09:20.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.822+0000 7f04e85d5640 1 -- 172.21.15.16:0/2731171297 shutdown_connections 2024-04-25T10:09:20.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.822+0000 7f04e85d5640 1 --2- 172.21.15.16:0/2731171297 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04e00d1a30 0x7f04e00b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.822+0000 7f04e85d5640 1 --2- 172.21.15.16:0/2731171297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e015dd90 0x7f04e00d34b0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.822+0000 7f04e85d5640 1 -- 172.21.15.16:0/2731171297 >> 172.21.15.16:0/2731171297 conn(0x7f04e0072900 msgr2=0x7f04e006c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.822+0000 7f04e85d5640 1 -- 172.21.15.16:0/2731171297 shutdown_connections 2024-04-25T10:09:20.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.822+0000 7f04e85d5640 1 -- 172.21.15.16:0/2731171297 wait complete. 2024-04-25T10:09:20.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.823+0000 7f04e85d5640 1 Processor -- start 2024-04-25T10:09:20.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.823+0000 7f04e85d5640 1 -- start start 2024-04-25T10:09:20.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.824+0000 7f04e85d5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04e00d03e0 0x7f04e00c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.824+0000 7f04e85d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e00d1a30 0x7f04e0152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.824+0000 7f04e85d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04e015dd90 0x7f04e01f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.824+0000 7f04e85d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00b99a0 con 0x7f04e00d1a30 2024-04-25T10:09:20.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.824+0000 7f04e85d5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00b9820 con 0x7f04e00d03e0 2024-04-25T10:09:20.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.824+0000 7f04e85d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00d1ee0 con 0x7f04e015dd90 2024-04-25T10:09:20.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.824+0000 7f04e5b70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e00d1a30 0x7f04e0152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.824+0000 7f04e5b70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e00d1a30 0x7f04e0152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39880/0 (socket says 172.21.15.16:39880) 2024-04-25T10:09:20.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.824+0000 7f04e5b70640 1 -- 172.21.15.16:0/560573282 learned_addr learned my addr 172.21.15.16:0/560573282 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:20.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.824+0000 7f04e6b72640 1 --2- 172.21.15.16:0/560573282 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04e015dd90 0x7f04e01f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.824+0000 7f04e6371640 1 --2- 172.21.15.16:0/560573282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04e00d03e0 0x7f04e00c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.824+0000 7f04e5b70640 1 -- 172.21.15.16:0/560573282 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04e015dd90 msgr2=0x7f04e01f6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.824+0000 7f04e5b70640 1 --2- 172.21.15.16:0/560573282 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04e015dd90 0x7f04e01f6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.824+0000 7f04e5b70640 1 -- 172.21.15.16:0/560573282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04e00d03e0 msgr2=0x7f04e00c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.824+0000 7f04e5b70640 1 --2- 172.21.15.16:0/560573282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04e00d03e0 0x7f04e00c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.824+0000 7f04e5b70640 1 -- 172.21.15.16:0/560573282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e0152760 con 0x7f04e00d1a30 2024-04-25T10:09:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.824+0000 7f04e6371640 1 --2- 172.21.15.16:0/560573282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04e00d03e0 0x7f04e00c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.824+0000 7f04e5b70640 1 --2- 172.21.15.16:0/560573282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e00d1a30 0x7f04e0152220 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f04d402a2b0 tx=0x7f04d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.825+0000 7f04deffd640 1 -- 172.21.15.16:0/560573282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d4042020 con 0x7f04e00d1a30 2024-04-25T10:09:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.825+0000 7f04deffd640 1 -- 172.21.15.16:0/560573282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d4003b40 con 0x7f04e00d1a30 2024-04-25T10:09:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.825+0000 7f04e85d5640 1 -- 172.21.15.16:0/560573282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e00b5290 con 0x7f04e00d1a30 2024-04-25T10:09:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.825+0000 7f04deffd640 1 -- 172.21.15.16:0/560573282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d4006040 con 0x7f04e00d1a30 2024-04-25T10:09:20.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.825+0000 7f04e85d5640 1 -- 172.21.15.16:0/560573282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e00b5850 con 0x7f04e00d1a30 2024-04-25T10:09:20.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.826+0000 7f04e85d5640 1 -- 172.21.15.16:0/560573282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e00749b0 con 0x7f04e00d1a30 2024-04-25T10:09:20.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.826+0000 7f04deffd640 1 -- 172.21.15.16:0/560573282 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f04d4045020 con 0x7f04e00d1a30 2024-04-25T10:09:20.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.827+0000 7f04deffd640 1 --2- 172.21.15.16:0/560573282 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f04cc06b4d0 0x7f04cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.827+0000 7f04deffd640 1 -- 172.21.15.16:0/560573282 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f04d40aad30 con 0x7f04e00d1a30 2024-04-25T10:09:20.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.827+0000 7f04e6371640 1 --2- 172.21.15.16:0/560573282 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f04cc06b4d0 0x7f04cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.828+0000 7f04e6371640 1 --2- 172.21.15.16:0/560573282 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f04cc06b4d0 0x7f04cc06d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f04d0001dc0 tx=0x7f04d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:20.831+0000 7f04deffd640 1 -- 172.21.15.16:0/560573282 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f04d404c030 con 0x7f04e00d1a30 2024-04-25T10:09:21.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.016+0000 7f04e85d5640 1 -- 172.21.15.16:0/560573282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f04e00cbe10 con 0x7f04e00d1a30 2024-04-25T10:09:21.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.695+0000 7f04deffd640 1 -- 172.21.15.16:0/560573282 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v70) v1 ==== 170+0+0 (secure 0 0 0) 0x7f04d4079d90 con 0x7f04e00d1a30 2024-04-25T10:09:21.703 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-04-25T10:09:21.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.699+0000 7f04e85d5640 1 -- 172.21.15.16:0/560573282 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f04cc06b4d0 msgr2=0x7f04cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:21.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.699+0000 7f04e85d5640 1 --2- 172.21.15.16:0/560573282 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f04cc06b4d0 0x7f04cc06d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f04d0001dc0 tx=0x7f04d0009040 comp rx=0 tx=0).stop 2024-04-25T10:09:21.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.699+0000 7f04e85d5640 1 -- 172.21.15.16:0/560573282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e00d1a30 msgr2=0x7f04e0152220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:21.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.699+0000 7f04e85d5640 1 --2- 172.21.15.16:0/560573282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e00d1a30 0x7f04e0152220 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f04d402a2b0 tx=0x7f04d40046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:21.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.700+0000 7f04e85d5640 1 -- 172.21.15.16:0/560573282 shutdown_connections 2024-04-25T10:09:21.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.700+0000 7f04e85d5640 1 --2- 172.21.15.16:0/560573282 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f04cc06b4d0 0x7f04cc06d990 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:21.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.700+0000 7f04e85d5640 1 --2- 172.21.15.16:0/560573282 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04e015dd90 0x7f04e01f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:21.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.700+0000 7f04e85d5640 1 --2- 172.21.15.16:0/560573282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04e00d1a30 0x7f04e0152220 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:21.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.700+0000 7f04e85d5640 1 --2- 172.21.15.16:0/560573282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04e00d03e0 0x7f04e00c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:21.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.700+0000 7f04e85d5640 1 -- 172.21.15.16:0/560573282 >> 172.21.15.16:0/560573282 conn(0x7f04e0072900 msgr2=0x7f04e00ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:21.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.700+0000 7f04e85d5640 1 -- 172.21.15.16:0/560573282 shutdown_connections 2024-04-25T10:09:21.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.701+0000 7f04e85d5640 1 -- 172.21.15.16:0/560573282 wait complete. 2024-04-25T10:09:21.736 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:09:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.839+0000 7f23bf9e2640 1 Processor -- start 2024-04-25T10:09:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.839+0000 7f23bf9e2640 1 -- start start 2024-04-25T10:09:21.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.839+0000 7f23bf9e2640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23b8144e70 0x7f23b806d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:21.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.839+0000 7f23bf9e2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23b80cf0e0 0x7f23b80cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:21.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.839+0000 7f23bf9e2640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b806dcb0 con 0x7f23b806d7a0 2024-04-25T10:09:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.839+0000 7f23bf9e2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b815bfe0 con 0x7f23b80cf0e0 2024-04-25T10:09:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.839+0000 7f23bf9e2640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b806df90 con 0x7f23b8144e70 2024-04-25T10:09:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.839+0000 7f23bdf7f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23b80cf0e0 0x7f23b80cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.839+0000 7f23bdf7f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23b80cf0e0 0x7f23b80cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39886/0 (socket says 172.21.15.16:39886) 2024-04-25T10:09:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.839+0000 7f23bdf7f640 1 -- 172.21.15.16:0/2810748471 learned_addr learned my addr 172.21.15.16:0/2810748471 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.839+0000 7f23bd77e640 1 --2- 172.21.15.16:0/2810748471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23b8144e70 0x7f23b806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.839+0000 7f23bdf7f640 1 -- 172.21.15.16:0/2810748471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23b8144e70 msgr2=0x7f23b806d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.839+0000 7f23bdf7f640 1 --2- 172.21.15.16:0/2810748471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23b8144e70 0x7f23b806d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.839+0000 7f23bdf7f640 1 -- 172.21.15.16:0/2810748471 >> v1:172.21.15.186:6789/0 conn(0x7f23b806d7a0 legacy=0x7f23b80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.839+0000 7f23bdf7f640 1 -- 172.21.15.16:0/2810748471 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b806e270 con 0x7f23b80cf0e0 2024-04-25T10:09:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.839+0000 7f23bd77e640 1 --2- 172.21.15.16:0/2810748471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23b8144e70 0x7f23b806d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.840+0000 7f23bdf7f640 1 --2- 172.21.15.16:0/2810748471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b80cf0e0 0x7f23b80cc300 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009e50 tx=0x7f23b002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a43707fda11caa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.840+0000 7f23affff640 1 -- 172.21.15.16:0/2810748471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0036070 con 0x7f23b80cf0e0 2024-04-25T10:09:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.840+0000 7f23affff640 1 -- 172.21.15.16:0/2810748471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b0031600 con 0x7f23b80cf0e0 2024-04-25T10:09:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.840+0000 7f23affff640 1 -- 172.21.15.16:0/2810748471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b00319d0 con 0x7f23b80cf0e0 2024-04-25T10:09:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.841+0000 7f23bf9e2640 1 -- 172.21.15.16:0/2810748471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b80cf0e0 msgr2=0x7f23b80cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.841+0000 7f23bf9e2640 1 --2- 172.21.15.16:0/2810748471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b80cf0e0 0x7f23b80cc300 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009e50 tx=0x7f23b002a680 comp rx=0 tx=0).stop 2024-04-25T10:09:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.841+0000 7f23bf9e2640 1 -- 172.21.15.16:0/2810748471 shutdown_connections 2024-04-25T10:09:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.841+0000 7f23bf9e2640 1 --2- 172.21.15.16:0/2810748471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b80cf0e0 0x7f23b80cc300 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.841+0000 7f23bf9e2640 1 --2- 172.21.15.16:0/2810748471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23b8144e70 0x7f23b806d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.841+0000 7f23bf9e2640 1 -- 172.21.15.16:0/2810748471 >> 172.21.15.16:0/2810748471 conn(0x7f23b806aba0 msgr2=0x7f23b806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.841+0000 7f23bf9e2640 1 -- 172.21.15.16:0/2810748471 shutdown_connections 2024-04-25T10:09:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.841+0000 7f23bf9e2640 1 -- 172.21.15.16:0/2810748471 wait complete. 2024-04-25T10:09:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.842+0000 7f23bf9e2640 1 Processor -- start 2024-04-25T10:09:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.842+0000 7f23bf9e2640 1 -- start start 2024-04-25T10:09:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23bf9e2640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23b806d7a0 0x7f23b815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23bf9e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b80cf0e0 0x7f23b813c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23bf9e2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23b8144e70 0x7f23b813f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23bf9e2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b8145630 con 0x7f23b80cf0e0 2024-04-25T10:09:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23bf9e2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b81454b0 con 0x7f23b8144e70 2024-04-25T10:09:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23bf9e2640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b80cc840 con 0x7f23b806d7a0 2024-04-25T10:09:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23bcf7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b80cf0e0 0x7f23b813c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23bcf7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b80cf0e0 0x7f23b813c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39898/0 (socket says 172.21.15.16:39898) 2024-04-25T10:09:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23bcf7d640 1 -- 172.21.15.16:0/3709763978 learned_addr learned my addr 172.21.15.16:0/3709763978 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23bd77e640 1 --2- 172.21.15.16:0/3709763978 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23b806d7a0 0x7f23b815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23bdf7f640 1 --2- 172.21.15.16:0/3709763978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23b8144e70 0x7f23b813f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23bcf7d640 1 -- 172.21.15.16:0/3709763978 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23b806d7a0 msgr2=0x7f23b815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23bcf7d640 1 --2- 172.21.15.16:0/3709763978 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23b806d7a0 0x7f23b815eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23bcf7d640 1 -- 172.21.15.16:0/3709763978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23b8144e70 msgr2=0x7f23b813f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23bcf7d640 1 --2- 172.21.15.16:0/3709763978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23b8144e70 0x7f23b813f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23bcf7d640 1 -- 172.21.15.16:0/3709763978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b813d0c0 con 0x7f23b80cf0e0 2024-04-25T10:09:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23bdf7f640 1 --2- 172.21.15.16:0/3709763978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23b8144e70 0x7f23b813f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23bcf7d640 1 --2- 172.21.15.16:0/3709763978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b80cf0e0 0x7f23b813c970 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f23a800c3c0 tx=0x7f23a800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23adffb640 1 -- 172.21.15.16:0/3709763978 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8017070 con 0x7f23b80cf0e0 2024-04-25T10:09:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23adffb640 1 -- 172.21.15.16:0/3709763978 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a80052f0 con 0x7f23b80cf0e0 2024-04-25T10:09:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.843+0000 7f23adffb640 1 -- 172.21.15.16:0/3709763978 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8003850 con 0x7f23b80cf0e0 2024-04-25T10:09:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.844+0000 7f23bf9e2640 1 -- 172.21.15.16:0/3709763978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b813d3a0 con 0x7f23b80cf0e0 2024-04-25T10:09:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.844+0000 7f23bf9e2640 1 -- 172.21.15.16:0/3709763978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b813d960 con 0x7f23b80cf0e0 2024-04-25T10:09:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.845+0000 7f23bf9e2640 1 -- 172.21.15.16:0/3709763978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b80750b0 con 0x7f23b80cf0e0 2024-04-25T10:09:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.845+0000 7f23adffb640 1 -- 172.21.15.16:0/3709763978 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f23a801a070 con 0x7f23b80cf0e0 2024-04-25T10:09:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.846+0000 7f23adffb640 1 --2- 172.21.15.16:0/3709763978 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f239c06b4d0 0x7f239c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.846+0000 7f23adffb640 1 -- 172.21.15.16:0/3709763978 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f23a8089420 con 0x7f23b80cf0e0 2024-04-25T10:09:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.846+0000 7f23bd77e640 1 --2- 172.21.15.16:0/3709763978 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f239c06b4d0 0x7f239c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.847+0000 7f23bd77e640 1 --2- 172.21.15.16:0/3709763978 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f239c06b4d0 0x7f239c06d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f23b4006fd0 tx=0x7f23b4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:21.850+0000 7f23adffb640 1 -- 172.21.15.16:0/3709763978 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f23a8057ca0 con 0x7f23b80cf0e0 2024-04-25T10:09:22.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.075+0000 7f23bf9e2640 1 -- 172.21.15.16:0/3709763978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f23b806b260 con 0x7f23b80cf0e0 2024-04-25T10:09:22.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.706+0000 7f23adffb640 1 -- 172.21.15.16:0/3709763978 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v34) v1 ==== 174+0+0 (secure 0 0 0) 0x7f23a805bd40 con 0x7f23b80cf0e0 2024-04-25T10:09:22.712 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 10 and data pool 11 2024-04-25T10:09:22.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.709+0000 7f23bf9e2640 1 -- 172.21.15.16:0/3709763978 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f239c06b4d0 msgr2=0x7f239c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:22.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.709+0000 7f23bf9e2640 1 --2- 172.21.15.16:0/3709763978 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f239c06b4d0 0x7f239c06d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f23b4006fd0 tx=0x7f23b4008040 comp rx=0 tx=0).stop 2024-04-25T10:09:22.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.709+0000 7f23bf9e2640 1 -- 172.21.15.16:0/3709763978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b80cf0e0 msgr2=0x7f23b813c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:22.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.709+0000 7f23bf9e2640 1 --2- 172.21.15.16:0/3709763978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b80cf0e0 0x7f23b813c970 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f23a800c3c0 tx=0x7f23a800c880 comp rx=0 tx=0).stop 2024-04-25T10:09:22.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.709+0000 7f23bf9e2640 1 -- 172.21.15.16:0/3709763978 shutdown_connections 2024-04-25T10:09:22.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.709+0000 7f23bf9e2640 1 --2- 172.21.15.16:0/3709763978 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f239c06b4d0 0x7f239c06d990 secure :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f23b4006fd0 tx=0x7f23b4008040 comp rx=0 tx=0).stop 2024-04-25T10:09:22.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.709+0000 7f23bf9e2640 1 --2- 172.21.15.16:0/3709763978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23b8144e70 0x7f23b813f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:22.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.709+0000 7f23bf9e2640 1 --2- 172.21.15.16:0/3709763978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b80cf0e0 0x7f23b813c970 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:22.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.709+0000 7f23bf9e2640 1 --2- 172.21.15.16:0/3709763978 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23b806d7a0 0x7f23b815eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:22.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.709+0000 7f23bf9e2640 1 -- 172.21.15.16:0/3709763978 >> 172.21.15.16:0/3709763978 conn(0x7f23b806aba0 msgr2=0x7f23b80b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:22.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.710+0000 7f23bf9e2640 1 -- 172.21.15.16:0/3709763978 shutdown_connections 2024-04-25T10:09:22.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.710+0000 7f23bf9e2640 1 -- 172.21.15.16:0/3709763978 wait complete. 2024-04-25T10:09:22.747 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:22.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.847+0000 7f194cbf9640 1 Processor -- start 2024-04-25T10:09:22.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.847+0000 7f194cbf9640 1 -- start start 2024-04-25T10:09:22.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.848+0000 7f194cbf9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f194815dce0 0x7f19480d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:22.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.848+0000 7f194cbf9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f19480d35b0 0x7f19480b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:22.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.848+0000 7f194cbf9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19480d1f90 0x7f19480b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:22.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.848+0000 7f194cbf9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194815e0c0 con 0x7f19480d1f90 2024-04-25T10:09:22.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.848+0000 7f194cbf9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19480d2510 con 0x7f19480d35b0 2024-04-25T10:09:22.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.848+0000 7f194cbf9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19480d27f0 con 0x7f194815dce0 2024-04-25T10:09:22.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.848+0000 7f1946d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19480d1f90 0x7f19480b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.848+0000 7f1946d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19480d1f90 0x7f19480b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52276/0 (socket says 172.21.15.16:52276) 2024-04-25T10:09:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.848+0000 7f1946d9d640 1 -- 172.21.15.16:0/111717644 learned_addr learned my addr 172.21.15.16:0/111717644 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.848+0000 7f193ffff640 1 --2- 172.21.15.16:0/111717644 >> v2:172.21.15.186:3300/0 conn(0x7f19480d35b0 0x7f19480b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.848+0000 7f194659c640 1 --2- 172.21.15.16:0/111717644 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f194815dce0 0x7f19480d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.848+0000 7f1946d9d640 1 -- 172.21.15.16:0/111717644 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f194815dce0 msgr2=0x7f19480d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.848+0000 7f1946d9d640 1 --2- 172.21.15.16:0/111717644 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f194815dce0 0x7f19480d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.848+0000 7f1946d9d640 1 -- 172.21.15.16:0/111717644 >> v2:172.21.15.186:3300/0 conn(0x7f19480d35b0 msgr2=0x7f19480b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.848+0000 7f1946d9d640 1 --2- 172.21.15.16:0/111717644 >> v2:172.21.15.186:3300/0 conn(0x7f19480d35b0 0x7f19480b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.848+0000 7f1946d9d640 1 -- 172.21.15.16:0/111717644 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19480d2ad0 con 0x7f19480d1f90 2024-04-25T10:09:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.848+0000 7f193ffff640 1 --2- 172.21.15.16:0/111717644 >> v2:172.21.15.186:3300/0 conn(0x7f19480d35b0 0x7f19480b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.848+0000 7f194659c640 1 --2- 172.21.15.16:0/111717644 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f194815dce0 0x7f19480d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.849+0000 7f1946d9d640 1 --2- 172.21.15.16:0/111717644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19480d1f90 0x7f19480b5990 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f194806d0d0 tx=0x7f194002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8295b60e5b9d722 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.849+0000 7f1945d9b640 1 -- 172.21.15.16:0/111717644 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940036070 con 0x7f19480d1f90 2024-04-25T10:09:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.849+0000 7f1945d9b640 1 -- 172.21.15.16:0/111717644 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1940031600 con 0x7f19480d1f90 2024-04-25T10:09:22.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.849+0000 7f1945d9b640 1 -- 172.21.15.16:0/111717644 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19400319d0 con 0x7f19480d1f90 2024-04-25T10:09:22.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.850+0000 7f194cbf9640 1 -- 172.21.15.16:0/111717644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19480d1f90 msgr2=0x7f19480b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:22.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.850+0000 7f194cbf9640 1 --2- 172.21.15.16:0/111717644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19480d1f90 0x7f19480b5990 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f194806d0d0 tx=0x7f194002a610 comp rx=0 tx=0).stop 2024-04-25T10:09:22.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.850+0000 7f194cbf9640 1 -- 172.21.15.16:0/111717644 shutdown_connections 2024-04-25T10:09:22.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.850+0000 7f194cbf9640 1 --2- 172.21.15.16:0/111717644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19480d1f90 0x7f19480b5990 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:22.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.850+0000 7f194cbf9640 1 --2- 172.21.15.16:0/111717644 >> v2:172.21.15.186:3300/0 conn(0x7f19480d35b0 0x7f19480b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:22.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.850+0000 7f194cbf9640 1 --2- 172.21.15.16:0/111717644 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f194815dce0 0x7f19480d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:22.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.850+0000 7f194cbf9640 1 -- 172.21.15.16:0/111717644 >> 172.21.15.16:0/111717644 conn(0x7f1948072900 msgr2=0x7f19480d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:22.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.850+0000 7f194cbf9640 1 -- 172.21.15.16:0/111717644 shutdown_connections 2024-04-25T10:09:22.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.850+0000 7f194cbf9640 1 -- 172.21.15.16:0/111717644 wait complete. 2024-04-25T10:09:22.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.851+0000 7f194cbf9640 1 Processor -- start 2024-04-25T10:09:22.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.851+0000 7f194cbf9640 1 -- start start 2024-04-25T10:09:22.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.852+0000 7f194cbf9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19480d1f90 0x7f1948160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:22.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.852+0000 7f194cbf9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f19480d35b0 0x7f1948152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:22.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.852+0000 7f194cbf9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194815dce0 0x7f1948153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:22.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.852+0000 7f194cbf9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19480b9820 con 0x7f194815dce0 2024-04-25T10:09:22.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.852+0000 7f194cbf9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19480d3090 con 0x7f19480d1f90 2024-04-25T10:09:22.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.852+0000 7f194cbf9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19480b99a0 con 0x7f19480d35b0 2024-04-25T10:09:22.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.852+0000 7f1946d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194815dce0 0x7f1948153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:22.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.852+0000 7f1946d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194815dce0 0x7f1948153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52292/0 (socket says 172.21.15.16:52292) 2024-04-25T10:09:22.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.852+0000 7f1946d9d640 1 -- 172.21.15.16:0/3985424901 learned_addr learned my addr 172.21.15.16:0/3985424901 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:22.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.852+0000 7f193ffff640 1 --2- 172.21.15.16:0/3985424901 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f19480d35b0 0x7f1948152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:22.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.852+0000 7f194659c640 1 --2- 172.21.15.16:0/3985424901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19480d1f90 0x7f1948160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:22.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.852+0000 7f1946d9d640 1 -- 172.21.15.16:0/3985424901 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f19480d35b0 msgr2=0x7f1948152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:22.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.852+0000 7f1946d9d640 1 --2- 172.21.15.16:0/3985424901 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f19480d35b0 0x7f1948152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:22.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.852+0000 7f1946d9d640 1 -- 172.21.15.16:0/3985424901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19480d1f90 msgr2=0x7f1948160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:22.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.852+0000 7f1946d9d640 1 --2- 172.21.15.16:0/3985424901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19480d1f90 0x7f1948160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.852+0000 7f1946d9d640 1 -- 172.21.15.16:0/3985424901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19480cc5a0 con 0x7f194815dce0 2024-04-25T10:09:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.852+0000 7f194659c640 1 --2- 172.21.15.16:0/3985424901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19480d1f90 0x7f1948160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.852+0000 7f193ffff640 1 --2- 172.21.15.16:0/3985424901 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f19480d35b0 0x7f1948152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.853+0000 7f1946d9d640 1 --2- 172.21.15.16:0/3985424901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194815dce0 0x7f1948153280 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f1940009f40 tx=0x7f1940004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.853+0000 7f193effd640 1 -- 172.21.15.16:0/3985424901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940036050 con 0x7f194815dce0 2024-04-25T10:09:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.853+0000 7f193effd640 1 -- 172.21.15.16:0/3985424901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1940042070 con 0x7f194815dce0 2024-04-25T10:09:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.853+0000 7f194cbf9640 1 -- 172.21.15.16:0/3985424901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19480cc840 con 0x7f194815dce0 2024-04-25T10:09:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.853+0000 7f193effd640 1 -- 172.21.15.16:0/3985424901 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940009510 con 0x7f194815dce0 2024-04-25T10:09:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.853+0000 7f194cbf9640 1 -- 172.21.15.16:0/3985424901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1948161000 con 0x7f194815dce0 2024-04-25T10:09:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.854+0000 7f194cbf9640 1 -- 172.21.15.16:0/3985424901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19480ccc30 con 0x7f194815dce0 2024-04-25T10:09:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.854+0000 7f193effd640 1 -- 172.21.15.16:0/3985424901 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1940007bd0 con 0x7f194815dce0 2024-04-25T10:09:22.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.854+0000 7f193effd640 1 --2- 172.21.15.16:0/3985424901 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f192806b4d0 0x7f192806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:22.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.855+0000 7f193effd640 1 -- 172.21.15.16:0/3985424901 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19400ab090 con 0x7f194815dce0 2024-04-25T10:09:22.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.855+0000 7f194659c640 1 --2- 172.21.15.16:0/3985424901 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f192806b4d0 0x7f192806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:22.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.855+0000 7f194659c640 1 --2- 172.21.15.16:0/3985424901 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f192806b4d0 0x7f192806d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1938006fd0 tx=0x7f1938008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:22.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:22.859+0000 7f193effd640 1 -- 172.21.15.16:0/3985424901 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1940079f10 con 0x7f194815dce0 2024-04-25T10:09:23.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.044+0000 7f194cbf9640 1 -- 172.21.15.16:0/3985424901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19480cca10 con 0x7f194815dce0 2024-04-25T10:09:23.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.045+0000 7f193effd640 1 -- 172.21.15.16:0/3985424901 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16621 (secure 0 0 0) 0x7f1940079810 con 0x7f194815dce0 2024-04-25T10:09:23.051 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:23.051 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":71,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:09:22.076706+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:20.361009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:21.048892+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:23.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.047+0000 7f194cbf9640 1 -- 172.21.15.16:0/3985424901 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f192806b4d0 msgr2=0x7f192806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:23.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.047+0000 7f194cbf9640 1 --2- 172.21.15.16:0/3985424901 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f192806b4d0 0x7f192806d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1938006fd0 tx=0x7f1938008040 comp rx=0 tx=0).stop 2024-04-25T10:09:23.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.047+0000 7f194cbf9640 1 -- 172.21.15.16:0/3985424901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194815dce0 msgr2=0x7f1948153280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:23.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.047+0000 7f194cbf9640 1 --2- 172.21.15.16:0/3985424901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194815dce0 0x7f1948153280 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f1940009f40 tx=0x7f1940004a40 comp rx=0 tx=0).stop 2024-04-25T10:09:23.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.047+0000 7f194cbf9640 1 -- 172.21.15.16:0/3985424901 shutdown_connections 2024-04-25T10:09:23.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.047+0000 7f194cbf9640 1 --2- 172.21.15.16:0/3985424901 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f192806b4d0 0x7f192806d990 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.047+0000 7f194cbf9640 1 --2- 172.21.15.16:0/3985424901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194815dce0 0x7f1948153280 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.047+0000 7f194cbf9640 1 --2- 172.21.15.16:0/3985424901 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f19480d35b0 0x7f1948152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.047+0000 7f194cbf9640 1 --2- 172.21.15.16:0/3985424901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19480d1f90 0x7f1948160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.047+0000 7f194cbf9640 1 -- 172.21.15.16:0/3985424901 >> 172.21.15.16:0/3985424901 conn(0x7f1948072900 msgr2=0x7f19480b7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:23.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.047+0000 7f194cbf9640 1 -- 172.21.15.16:0/3985424901 shutdown_connections 2024-04-25T10:09:23.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.047+0000 7f194cbf9640 1 -- 172.21.15.16:0/3985424901 wait complete. 2024-04-25T10:09:23.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:23.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.159+0000 7f966bcac640 1 Processor -- start 2024-04-25T10:09:23.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.159+0000 7f966bcac640 1 -- start start 2024-04-25T10:09:23.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.160+0000 7f966bcac640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f966409a150 0x7f966406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:23.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.160+0000 7f966bcac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f966406c1f0 0x7f96640cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:23.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.160+0000 7f966bcac640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f966406c660 con 0x7f966415ba10 2024-04-25T10:09:23.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.160+0000 7f966bcac640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96640cbe50 con 0x7f966406c1f0 2024-04-25T10:09:23.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.160+0000 7f966bcac640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96640cfe80 con 0x7f966409a150 2024-04-25T10:09:23.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.160+0000 7f9669a48640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f966415ba10 0x7f9664099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50422/0 (socket says 172.21.15.16:50422) 2024-04-25T10:09:23.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.160+0000 7f9669a48640 1 -- 172.21.15.16:0/3251177683 learned_addr learned my addr 172.21.15.16:0/3251177683 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:23.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.160+0000 7f966a249640 1 --2- 172.21.15.16:0/3251177683 >> v2:172.21.15.16:3300/0 conn(0x7f966406c1f0 0x7f96640cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:23.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.160+0000 7f966a249640 1 -- 172.21.15.16:0/3251177683 >> v2:172.21.15.186:3300/0 conn(0x7f966409a150 msgr2=0x7f966406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:23.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.160+0000 7f966a249640 1 --2- 172.21.15.16:0/3251177683 >> v2:172.21.15.186:3300/0 conn(0x7f966409a150 0x7f966406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.160+0000 7f966a249640 1 -- 172.21.15.16:0/3251177683 >> v1:172.21.15.16:6789/0 conn(0x7f966415ba10 legacy=0x7f9664099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:23.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.160+0000 7f966a249640 1 -- 172.21.15.16:0/3251177683 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966406c900 con 0x7f966406c1f0 2024-04-25T10:09:23.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.161+0000 7f966a249640 1 --2- 172.21.15.16:0/3251177683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f966406c1f0 0x7f96640cd220 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f966409a9c0 tx=0x7f965c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d792fab8896f85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:23.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.161+0000 7f9668a46640 1 -- 172.21.15.16:0/3251177683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c007740 con 0x7f966406c1f0 2024-04-25T10:09:23.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.161+0000 7f9668a46640 1 -- 172.21.15.16:0/3251177683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965c0078e0 con 0x7f966406c1f0 2024-04-25T10:09:23.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.161+0000 7f9668a46640 1 -- 172.21.15.16:0/3251177683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c007cb0 con 0x7f966406c1f0 2024-04-25T10:09:23.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.161+0000 7f966bcac640 1 -- 172.21.15.16:0/3251177683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f966406c1f0 msgr2=0x7f96640cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:23.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.161+0000 7f966bcac640 1 --2- 172.21.15.16:0/3251177683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f966406c1f0 0x7f96640cd220 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f966409a9c0 tx=0x7f965c02a680 comp rx=0 tx=0).stop 2024-04-25T10:09:23.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.162+0000 7f966bcac640 1 -- 172.21.15.16:0/3251177683 shutdown_connections 2024-04-25T10:09:23.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.162+0000 7f966bcac640 1 --2- 172.21.15.16:0/3251177683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f966406c1f0 0x7f96640cd220 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.162+0000 7f966bcac640 1 --2- 172.21.15.16:0/3251177683 >> v2:172.21.15.186:3300/0 conn(0x7f966409a150 0x7f966406bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.162+0000 7f966bcac640 1 -- 172.21.15.16:0/3251177683 >> 172.21.15.16:0/3251177683 conn(0x7f96640b32f0 msgr2=0x7f96640b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.162+0000 7f966bcac640 1 -- 172.21.15.16:0/3251177683 shutdown_connections 2024-04-25T10:09:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.162+0000 7f966bcac640 1 -- 172.21.15.16:0/3251177683 wait complete. 2024-04-25T10:09:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.163+0000 7f966bcac640 1 Processor -- start 2024-04-25T10:09:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.163+0000 7f966bcac640 1 -- start start 2024-04-25T10:09:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.163+0000 7f966bcac640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f966406c1f0 0x7f96641600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.163+0000 7f966bcac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f966409a150 0x7f96640b0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.163+0000 7f966bcac640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f966415ba10 0x7f966415a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.163+0000 7f966bcac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96640cd760 con 0x7f966409a150 2024-04-25T10:09:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.163+0000 7f966bcac640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966406cec0 con 0x7f966415ba10 2024-04-25T10:09:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.163+0000 7f966bcac640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96640cd8e0 con 0x7f966406c1f0 2024-04-25T10:09:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.164+0000 7f9669247640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f966409a150 0x7f96640b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.164+0000 7f9669247640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f966409a150 0x7f96640b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52320/0 (socket says 172.21.15.16:52320) 2024-04-25T10:09:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.164+0000 7f9669247640 1 -- 172.21.15.16:0/210543789 learned_addr learned my addr 172.21.15.16:0/210543789 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.164+0000 7f966a249640 1 --2- 172.21.15.16:0/210543789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f966415ba10 0x7f966415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.164+0000 7f9669a48640 1 --2- 172.21.15.16:0/210543789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f966406c1f0 0x7f96641600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.164+0000 7f9669247640 1 -- 172.21.15.16:0/210543789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f966406c1f0 msgr2=0x7f96641600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.164+0000 7f9669247640 1 --2- 172.21.15.16:0/210543789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f966406c1f0 0x7f96641600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.164+0000 7f9669247640 1 -- 172.21.15.16:0/210543789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f966415ba10 msgr2=0x7f966415a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.164+0000 7f9669247640 1 --2- 172.21.15.16:0/210543789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f966415ba10 0x7f966415a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.164+0000 7f9669247640 1 -- 172.21.15.16:0/210543789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9664160780 con 0x7f966409a150 2024-04-25T10:09:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.164+0000 7f966a249640 1 --2- 172.21.15.16:0/210543789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f966415ba10 0x7f966415a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:23.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.164+0000 7f9669a48640 1 --2- 172.21.15.16:0/210543789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f966406c1f0 0x7f96641600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:23.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.164+0000 7f9669247640 1 --2- 172.21.15.16:0/210543789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f966409a150 0x7f96640b0720 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f965400b4a0 tx=0x7f965400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:23.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.164+0000 7f965a7fc640 1 -- 172.21.15.16:0/210543789 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9654014070 con 0x7f966409a150 2024-04-25T10:09:23.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.164+0000 7f965a7fc640 1 -- 172.21.15.16:0/210543789 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9654010610 con 0x7f966409a150 2024-04-25T10:09:23.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.164+0000 7f966bcac640 1 -- 172.21.15.16:0/210543789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9664160a60 con 0x7f966409a150 2024-04-25T10:09:23.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.164+0000 7f965a7fc640 1 -- 172.21.15.16:0/210543789 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9654010a00 con 0x7f966409a150 2024-04-25T10:09:23.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.164+0000 7f966bcac640 1 -- 172.21.15.16:0/210543789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9664161020 con 0x7f966409a150 2024-04-25T10:09:23.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.165+0000 7f966bcac640 1 -- 172.21.15.16:0/210543789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96640c8690 con 0x7f966409a150 2024-04-25T10:09:23.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.166+0000 7f965a7fc640 1 -- 172.21.15.16:0/210543789 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9654017070 con 0x7f966409a150 2024-04-25T10:09:23.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.166+0000 7f965a7fc640 1 --2- 172.21.15.16:0/210543789 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f964806b500 0x7f964806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:23.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.166+0000 7f965a7fc640 1 -- 172.21.15.16:0/210543789 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9654089c20 con 0x7f966409a150 2024-04-25T10:09:23.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.167+0000 7f9669a48640 1 --2- 172.21.15.16:0/210543789 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f964806b500 0x7f964806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:23.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.167+0000 7f9669a48640 1 --2- 172.21.15.16:0/210543789 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f964806b500 0x7f964806d9c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9664071210 tx=0x7f9660009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.171+0000 7f965a7fc640 1 -- 172.21.15.16:0/210543789 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96640c8690 con 0x7f966409a150 2024-04-25T10:09:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.354+0000 7f966bcac640 1 -- 172.21.15.16:0/210543789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96640d1620 con 0x7f966409a150 2024-04-25T10:09:23.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.355+0000 7f965a7fc640 1 -- 172.21.15.16:0/210543789 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16621 (secure 0 0 0) 0x7f96640d1620 con 0x7f966409a150 2024-04-25T10:09:23.361 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:23.361 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":71,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:09:22.076706+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:20.361009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:21.048892+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.357+0000 7f966bcac640 1 -- 172.21.15.16:0/210543789 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f964806b500 msgr2=0x7f964806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.357+0000 7f966bcac640 1 --2- 172.21.15.16:0/210543789 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f964806b500 0x7f964806d9c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9664071210 tx=0x7f9660009040 comp rx=0 tx=0).stop 2024-04-25T10:09:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.357+0000 7f966bcac640 1 -- 172.21.15.16:0/210543789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f966409a150 msgr2=0x7f96640b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.357+0000 7f966bcac640 1 --2- 172.21.15.16:0/210543789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f966409a150 0x7f96640b0720 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f965400b4a0 tx=0x7f965400b960 comp rx=0 tx=0).stop 2024-04-25T10:09:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.357+0000 7f966bcac640 1 -- 172.21.15.16:0/210543789 shutdown_connections 2024-04-25T10:09:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.357+0000 7f966bcac640 1 --2- 172.21.15.16:0/210543789 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f964806b500 0x7f964806d9c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.357+0000 7f966bcac640 1 --2- 172.21.15.16:0/210543789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f966415ba10 0x7f966415a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.357+0000 7f966bcac640 1 --2- 172.21.15.16:0/210543789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f966409a150 0x7f96640b0720 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.357+0000 7f966bcac640 1 --2- 172.21.15.16:0/210543789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f966406c1f0 0x7f96641600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.357+0000 7f966bcac640 1 -- 172.21.15.16:0/210543789 >> 172.21.15.16:0/210543789 conn(0x7f96640b32f0 msgr2=0x7f9664099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.357+0000 7f966bcac640 1 -- 172.21.15.16:0/210543789 shutdown_connections 2024-04-25T10:09:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.357+0000 7f966bcac640 1 -- 172.21.15.16:0/210543789 wait complete. 2024-04-25T10:09:23.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:09:23.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.472+0000 7fd4c5fb7640 1 Processor -- start 2024-04-25T10:09:23.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.472+0000 7fd4c5fb7640 1 -- start start 2024-04-25T10:09:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.473+0000 7fd4c5fb7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4c015bbe0 0x7fd4c00cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.473+0000 7fd4c5fb7640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd4c00cf510 0x7fd4c00cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.473+0000 7fd4c5fb7640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c00cc830 con 0x7fd4c00cff00 2024-04-25T10:09:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.473+0000 7fd4c5fb7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4c00ccb10 con 0x7fd4c015bbe0 2024-04-25T10:09:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.473+0000 7fd4c5fb7640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4c00ccdf0 con 0x7fd4c00cf510 2024-04-25T10:09:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.473+0000 7fd4bf7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4c015bbe0 0x7fd4c00cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.473+0000 7fd4bf7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4c015bbe0 0x7fd4c00cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52328/0 (socket says 172.21.15.16:52328) 2024-04-25T10:09:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.473+0000 7fd4bf7fe640 1 -- 172.21.15.16:0/1265863631 learned_addr learned my addr 172.21.15.16:0/1265863631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.473+0000 7fd4beffd640 1 --2- 172.21.15.16:0/1265863631 >> v2:172.21.15.186:3300/0 conn(0x7fd4c00cf510 0x7fd4c00cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.473+0000 7fd4bf7fe640 1 -- 172.21.15.16:0/1265863631 >> v2:172.21.15.186:3300/0 conn(0x7fd4c00cf510 msgr2=0x7fd4c00cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.473+0000 7fd4bf7fe640 1 --2- 172.21.15.16:0/1265863631 >> v2:172.21.15.186:3300/0 conn(0x7fd4c00cf510 0x7fd4c00cf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.473+0000 7fd4bf7fe640 1 -- 172.21.15.16:0/1265863631 >> v1:172.21.15.186:6789/0 conn(0x7fd4c00cff00 legacy=0x7fd4c00c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.473+0000 7fd4bf7fe640 1 -- 172.21.15.16:0/1265863631 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c00cd0d0 con 0x7fd4c015bbe0 2024-04-25T10:09:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.474+0000 7fd4bf7fe640 1 --2- 172.21.15.16:0/1265863631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4c015bbe0 0x7fd4c00cd8d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00b42a0 tx=0x7fd4b002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736f5819d846842b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.474+0000 7fd4be7fc640 1 -- 172.21.15.16:0/1265863631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0037070 con 0x7fd4c015bbe0 2024-04-25T10:09:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.474+0000 7fd4be7fc640 1 -- 172.21.15.16:0/1265863631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b0032440 con 0x7fd4c015bbe0 2024-04-25T10:09:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.474+0000 7fd4be7fc640 1 -- 172.21.15.16:0/1265863631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0032810 con 0x7fd4c015bbe0 2024-04-25T10:09:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.474+0000 7fd4c5fb7640 1 -- 172.21.15.16:0/1265863631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4c015bbe0 msgr2=0x7fd4c00cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.474+0000 7fd4c5fb7640 1 --2- 172.21.15.16:0/1265863631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4c015bbe0 0x7fd4c00cd8d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00b42a0 tx=0x7fd4b002a230 comp rx=0 tx=0).stop 2024-04-25T10:09:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.474+0000 7fd4c5fb7640 1 -- 172.21.15.16:0/1265863631 shutdown_connections 2024-04-25T10:09:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.474+0000 7fd4c5fb7640 1 --2- 172.21.15.16:0/1265863631 >> v2:172.21.15.186:3300/0 conn(0x7fd4c00cf510 0x7fd4c00cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.474+0000 7fd4c5fb7640 1 --2- 172.21.15.16:0/1265863631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4c015bbe0 0x7fd4c00cd8d0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.474+0000 7fd4c5fb7640 1 -- 172.21.15.16:0/1265863631 >> 172.21.15.16:0/1265863631 conn(0x7fd4c00b3430 msgr2=0x7fd4c00b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.475+0000 7fd4c5fb7640 1 -- 172.21.15.16:0/1265863631 shutdown_connections 2024-04-25T10:09:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.475+0000 7fd4c5fb7640 1 -- 172.21.15.16:0/1265863631 wait complete. 2024-04-25T10:09:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.476+0000 7fd4c5fb7640 1 Processor -- start 2024-04-25T10:09:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.476+0000 7fd4c5fb7640 1 -- start start 2024-04-25T10:09:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.476+0000 7fd4c5fb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4c00cf510 0x7fd4c009e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.476+0000 7fd4c5fb7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4c00cff00 0x7fd4c0151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4c5fb7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4c015bbe0 0x7fd4c015db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4c5fb7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c00cde10 con 0x7fd4c00cf510 2024-04-25T10:09:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4c5fb7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c015bfc0 con 0x7fd4c015bbe0 2024-04-25T10:09:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4c5fb7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c00cc450 con 0x7fd4c00cff00 2024-04-25T10:09:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4c00cf510 0x7fd4c009e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4c00cf510 0x7fd4c009e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52344/0 (socket says 172.21.15.16:52344) 2024-04-25T10:09:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4bf7fe640 1 -- 172.21.15.16:0/2530283656 learned_addr learned my addr 172.21.15.16:0/2530283656 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4bffff640 1 --2- 172.21.15.16:0/2530283656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4c015bbe0 0x7fd4c015db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4beffd640 1 --2- 172.21.15.16:0/2530283656 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4c00cff00 0x7fd4c0151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4bf7fe640 1 -- 172.21.15.16:0/2530283656 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4c00cff00 msgr2=0x7fd4c0151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4bf7fe640 1 --2- 172.21.15.16:0/2530283656 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4c00cff00 0x7fd4c0151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4bf7fe640 1 -- 172.21.15.16:0/2530283656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4c015bbe0 msgr2=0x7fd4c015db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4bf7fe640 1 --2- 172.21.15.16:0/2530283656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4c015bbe0 0x7fd4c015db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4bf7fe640 1 -- 172.21.15.16:0/2530283656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c009eb60 con 0x7fd4c00cf510 2024-04-25T10:09:23.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4beffd640 1 --2- 172.21.15.16:0/2530283656 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4c00cff00 0x7fd4c0151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:23.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4bf7fe640 1 --2- 172.21.15.16:0/2530283656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4c00cf510 0x7fd4c009e620 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0000c00 tx=0x7fd4b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:23.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4a7fff640 1 -- 172.21.15.16:0/2530283656 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0041070 con 0x7fd4c00cf510 2024-04-25T10:09:23.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4a7fff640 1 -- 172.21.15.16:0/2530283656 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b003c870 con 0x7fd4c00cf510 2024-04-25T10:09:23.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4a7fff640 1 -- 172.21.15.16:0/2530283656 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0037040 con 0x7fd4c00cf510 2024-04-25T10:09:23.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4c5fb7640 1 -- 172.21.15.16:0/2530283656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c009ee20 con 0x7fd4c00cf510 2024-04-25T10:09:23.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.477+0000 7fd4c5fb7640 1 -- 172.21.15.16:0/2530283656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c009f3e0 con 0x7fd4c00cf510 2024-04-25T10:09:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.478+0000 7fd4c5fb7640 1 -- 172.21.15.16:0/2530283656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4c00750b0 con 0x7fd4c00cf510 2024-04-25T10:09:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.479+0000 7fd4a7fff640 1 -- 172.21.15.16:0/2530283656 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4b0044070 con 0x7fd4c00cf510 2024-04-25T10:09:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.479+0000 7fd4a7fff640 1 --2- 172.21.15.16:0/2530283656 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd4ac06b4d0 0x7fd4ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.479+0000 7fd4a7fff640 1 -- 172.21.15.16:0/2530283656 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4b00aaee0 con 0x7fd4c00cf510 2024-04-25T10:09:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.480+0000 7fd4beffd640 1 --2- 172.21.15.16:0/2530283656 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd4ac06b4d0 0x7fd4ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.480+0000 7fd4beffd640 1 --2- 172.21.15.16:0/2530283656 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd4ac06b4d0 0x7fd4ac06d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd4c015b5d0 tx=0x7fd4b4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:23.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.483+0000 7fd4a7fff640 1 -- 172.21.15.16:0/2530283656 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4b0079cb0 con 0x7fd4c00cf510 2024-04-25T10:09:23.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.702+0000 7fd4c5fb7640 1 -- 172.21.15.16:0/2530283656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd4c015f490 con 0x7fd4c00cf510 2024-04-25T10:09:23.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.718+0000 7fd4a7fff640 1 -- 172.21.15.16:0/2530283656 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd4b00795b0 con 0x7fd4c00cf510 2024-04-25T10:09:23.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.720+0000 7fd4c5fb7640 1 -- 172.21.15.16:0/2530283656 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd4ac06b4d0 msgr2=0x7fd4ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:23.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.720+0000 7fd4c5fb7640 1 --2- 172.21.15.16:0/2530283656 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd4ac06b4d0 0x7fd4ac06d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd4c015b5d0 tx=0x7fd4b4008040 comp rx=0 tx=0).stop 2024-04-25T10:09:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.720+0000 7fd4c5fb7640 1 -- 172.21.15.16:0/2530283656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4c00cf510 msgr2=0x7fd4c009e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.720+0000 7fd4c5fb7640 1 --2- 172.21.15.16:0/2530283656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4c00cf510 0x7fd4c009e620 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0000c00 tx=0x7fd4b00046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.720+0000 7fd4c5fb7640 1 -- 172.21.15.16:0/2530283656 shutdown_connections 2024-04-25T10:09:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.720+0000 7fd4c5fb7640 1 --2- 172.21.15.16:0/2530283656 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd4ac06b4d0 0x7fd4ac06d990 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.720+0000 7fd4c5fb7640 1 --2- 172.21.15.16:0/2530283656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4c015bbe0 0x7fd4c015db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.720+0000 7fd4c5fb7640 1 --2- 172.21.15.16:0/2530283656 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4c00cff00 0x7fd4c0151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.720+0000 7fd4c5fb7640 1 --2- 172.21.15.16:0/2530283656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4c00cf510 0x7fd4c009e620 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.720+0000 7fd4c5fb7640 1 -- 172.21.15.16:0/2530283656 >> 172.21.15.16:0/2530283656 conn(0x7fd4c00b3430 msgr2=0x7fd4c00c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.720+0000 7fd4c5fb7640 1 -- 172.21.15.16:0/2530283656 shutdown_connections 2024-04-25T10:09:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.720+0000 7fd4c5fb7640 1 -- 172.21.15.16:0/2530283656 wait complete. 2024-04-25T10:09:23.739 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:09:23.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:09:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.836+0000 7f88c6a07640 1 Processor -- start 2024-04-25T10:09:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.836+0000 7f88c6a07640 1 -- start start 2024-04-25T10:09:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.837+0000 7f88c6a07640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f88c015dc80 0x7f88c0161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.837+0000 7f88c6a07640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88c00b97c0 0x7f88c00b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.837+0000 7f88c6a07640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c00b5e40 con 0x7f88c00b5a30 2024-04-25T10:09:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.837+0000 7f88c6a07640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c01627a0 con 0x7f88c00b97c0 2024-04-25T10:09:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.837+0000 7f88c6a07640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c01621e0 con 0x7f88c015dc80 2024-04-25T10:09:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.837+0000 7f88bf7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88c00b97c0 0x7f88c00b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.837+0000 7f88c4fa4640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f88c00b5a30 0x7f88c006bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50432/0 (socket says 172.21.15.16:50432) 2024-04-25T10:09:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.837+0000 7f88bf7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88c00b97c0 0x7f88c00b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52352/0 (socket says 172.21.15.16:52352) 2024-04-25T10:09:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.837+0000 7f88c4fa4640 1 -- 172.21.15.16:0/125379729 learned_addr learned my addr 172.21.15.16:0/125379729 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.838+0000 7f88bf7fe640 1 -- 172.21.15.16:0/125379729 >> v2:172.21.15.186:3300/0 conn(0x7f88c015dc80 msgr2=0x7f88c0161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.838+0000 7f88bffff640 1 --2- 172.21.15.16:0/125379729 >> v2:172.21.15.186:3300/0 conn(0x7f88c015dc80 0x7f88c0161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.838+0000 7f88bf7fe640 1 --2- 172.21.15.16:0/125379729 >> v2:172.21.15.186:3300/0 conn(0x7f88c015dc80 0x7f88c0161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.838+0000 7f88beffd640 1 -- 172.21.15.16:0/125379729 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714889272 0 0) 0x7f88b8001490 con 0x7f88c00b5a30 2024-04-25T10:09:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.838+0000 7f88bf7fe640 1 -- 172.21.15.16:0/125379729 >> v1:172.21.15.16:6789/0 conn(0x7f88c00b5a30 legacy=0x7f88c006bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.838+0000 7f88bf7fe640 1 -- 172.21.15.16:0/125379729 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c01624c0 con 0x7f88c00b97c0 2024-04-25T10:09:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.838+0000 7f88bffff640 1 --2- 172.21.15.16:0/125379729 >> v2:172.21.15.186:3300/0 conn(0x7f88c015dc80 0x7f88c0161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.838+0000 7f88bf7fe640 1 --2- 172.21.15.16:0/125379729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c00b97c0 0x7f88c00b54f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f88c006dab0 tx=0x7f88b402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2495ba2731c1d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.839+0000 7f88beffd640 1 -- 172.21.15.16:0/125379729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4037070 con 0x7f88c00b97c0 2024-04-25T10:09:23.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.839+0000 7f88beffd640 1 -- 172.21.15.16:0/125379729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b4032440 con 0x7f88c00b97c0 2024-04-25T10:09:23.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.839+0000 7f88beffd640 1 -- 172.21.15.16:0/125379729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4032810 con 0x7f88c00b97c0 2024-04-25T10:09:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.839+0000 7f88c6a07640 1 -- 172.21.15.16:0/125379729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c00b97c0 msgr2=0x7f88c00b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.839+0000 7f88c6a07640 1 --2- 172.21.15.16:0/125379729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c00b97c0 0x7f88c00b54f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f88c006dab0 tx=0x7f88b402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.839+0000 7f88c6a07640 1 -- 172.21.15.16:0/125379729 shutdown_connections 2024-04-25T10:09:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.839+0000 7f88c6a07640 1 --2- 172.21.15.16:0/125379729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c00b97c0 0x7f88c00b54f0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.839+0000 7f88c6a07640 1 --2- 172.21.15.16:0/125379729 >> v2:172.21.15.186:3300/0 conn(0x7f88c015dc80 0x7f88c0161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.839+0000 7f88c6a07640 1 -- 172.21.15.16:0/125379729 >> 172.21.15.16:0/125379729 conn(0x7f88c00d04f0 msgr2=0x7f88c00d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.839+0000 7f88c6a07640 1 -- 172.21.15.16:0/125379729 shutdown_connections 2024-04-25T10:09:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.839+0000 7f88c6a07640 1 -- 172.21.15.16:0/125379729 wait complete. 2024-04-25T10:09:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.840+0000 7f88c6a07640 1 Processor -- start 2024-04-25T10:09:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.841+0000 7f88c6a07640 1 -- start start 2024-04-25T10:09:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.841+0000 7f88c6a07640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88c00b5a30 0x7f88c014c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.841+0000 7f88c6a07640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c00b97c0 0x7f88c014c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.841+0000 7f88c6a07640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88c015dc80 0x7f88c00ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.841+0000 7f88c6a07640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c0161db0 con 0x7f88c00b97c0 2024-04-25T10:09:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.841+0000 7f88c6a07640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c015e060 con 0x7f88c00b5a30 2024-04-25T10:09:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.841+0000 7f88c6a07640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c0161f30 con 0x7f88c015dc80 2024-04-25T10:09:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.841+0000 7f88bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c00b97c0 0x7f88c014c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.841+0000 7f88bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c00b97c0 0x7f88c014c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52358/0 (socket says 172.21.15.16:52358) 2024-04-25T10:09:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.841+0000 7f88bf7fe640 1 -- 172.21.15.16:0/1561914373 learned_addr learned my addr 172.21.15.16:0/1561914373 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.842+0000 7f88bffff640 1 --2- 172.21.15.16:0/1561914373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88c00b5a30 0x7f88c014c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.842+0000 7f88c4fa4640 1 --2- 172.21.15.16:0/1561914373 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88c015dc80 0x7f88c00ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.842+0000 7f88bf7fe640 1 -- 172.21.15.16:0/1561914373 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88c015dc80 msgr2=0x7f88c00ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.842+0000 7f88bf7fe640 1 --2- 172.21.15.16:0/1561914373 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88c015dc80 0x7f88c00ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.842+0000 7f88bf7fe640 1 -- 172.21.15.16:0/1561914373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88c00b5a30 msgr2=0x7f88c014c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.842+0000 7f88bf7fe640 1 --2- 172.21.15.16:0/1561914373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88c00b5a30 0x7f88c014c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.842+0000 7f88bf7fe640 1 -- 172.21.15.16:0/1561914373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c00cd4a0 con 0x7f88c00b97c0 2024-04-25T10:09:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.842+0000 7f88bffff640 1 --2- 172.21.15.16:0/1561914373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88c00b5a30 0x7f88c014c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.842+0000 7f88bf7fe640 1 --2- 172.21.15.16:0/1561914373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c00b97c0 0x7f88c014c1b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f88b4000c00 tx=0x7f88b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.842+0000 7f88bcff9640 1 -- 172.21.15.16:0/1561914373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4037050 con 0x7f88c00b97c0 2024-04-25T10:09:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.842+0000 7f88bcff9640 1 -- 172.21.15.16:0/1561914373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b4038040 con 0x7f88c00b97c0 2024-04-25T10:09:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.842+0000 7f88bcff9640 1 -- 172.21.15.16:0/1561914373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4004450 con 0x7f88c00b97c0 2024-04-25T10:09:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.842+0000 7f88c6a07640 1 -- 172.21.15.16:0/1561914373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c014d000 con 0x7f88c00b97c0 2024-04-25T10:09:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.842+0000 7f88c6a07640 1 -- 172.21.15.16:0/1561914373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c014d5c0 con 0x7f88c00b97c0 2024-04-25T10:09:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.843+0000 7f88c6a07640 1 -- 172.21.15.16:0/1561914373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c00750b0 con 0x7f88c00b97c0 2024-04-25T10:09:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.843+0000 7f88bcff9640 1 -- 172.21.15.16:0/1561914373 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88b4030980 con 0x7f88c00b97c0 2024-04-25T10:09:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.844+0000 7f88bcff9640 1 --2- 172.21.15.16:0/1561914373 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f88ac06b4d0 0x7f88ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.844+0000 7f88bcff9640 1 -- 172.21.15.16:0/1561914373 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88b40aada0 con 0x7f88c00b97c0 2024-04-25T10:09:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.844+0000 7f88bffff640 1 --2- 172.21.15.16:0/1561914373 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f88ac06b4d0 0x7f88ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.845+0000 7f88bffff640 1 --2- 172.21.15.16:0/1561914373 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f88ac06b4d0 0x7f88ac06d990 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f88b000a990 tx=0x7f88b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:23.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:23.848+0000 7f88bcff9640 1 -- 172.21.15.16:0/1561914373 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88b4079d40 con 0x7f88c00b97c0 2024-04-25T10:09:24.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.067+0000 7f88c6a07640 1 -- 172.21.15.16:0/1561914373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f88c0161310 con 0x7f88c00b97c0 2024-04-25T10:09:24.702 INFO:tasks.ceph.mds.d.smithi186.stderr:2024-04-25T10:09:24.695+0000 7fcee278a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:09:24.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.732+0000 7f88bcff9640 1 -- 172.21.15.16:0/1561914373 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) v1 ==== 122+0+0 (secure 0 0 0) 0x7f88b4079640 con 0x7f88c00b97c0 2024-04-25T10:09:24.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.735+0000 7f88c6a07640 1 -- 172.21.15.16:0/1561914373 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f88ac06b4d0 msgr2=0x7f88ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:24.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.735+0000 7f88c6a07640 1 --2- 172.21.15.16:0/1561914373 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f88ac06b4d0 0x7f88ac06d990 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f88b000a990 tx=0x7f88b0008040 comp rx=0 tx=0).stop 2024-04-25T10:09:24.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.735+0000 7f88c6a07640 1 -- 172.21.15.16:0/1561914373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c00b97c0 msgr2=0x7f88c014c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:24.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.735+0000 7f88c6a07640 1 --2- 172.21.15.16:0/1561914373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c00b97c0 0x7f88c014c1b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f88b4000c00 tx=0x7f88b40046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.735+0000 7f88c6a07640 1 -- 172.21.15.16:0/1561914373 shutdown_connections 2024-04-25T10:09:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.735+0000 7f88c6a07640 1 --2- 172.21.15.16:0/1561914373 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f88ac06b4d0 0x7f88ac06d990 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.735+0000 7f88c6a07640 1 --2- 172.21.15.16:0/1561914373 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88c015dc80 0x7f88c00ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.735+0000 7f88c6a07640 1 --2- 172.21.15.16:0/1561914373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c00b97c0 0x7f88c014c1b0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.735+0000 7f88c6a07640 1 --2- 172.21.15.16:0/1561914373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88c00b5a30 0x7f88c014c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.735+0000 7f88c6a07640 1 -- 172.21.15.16:0/1561914373 >> 172.21.15.16:0/1561914373 conn(0x7f88c00d04f0 msgr2=0x7f88c01601f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.735+0000 7f88c6a07640 1 -- 172.21.15.16:0/1561914373 shutdown_connections 2024-04-25T10:09:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.735+0000 7f88c6a07640 1 -- 172.21.15.16:0/1561914373 wait complete. 2024-04-25T10:09:24.755 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:09:24.755 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:24.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.857+0000 7fe9d851b640 1 Processor -- start 2024-04-25T10:09:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.857+0000 7fe9d851b640 1 -- start start 2024-04-25T10:09:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.857+0000 7fe9d851b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe9d015ba10 0x7fe9d0099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.858+0000 7fe9d851b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d006c900 con 0x7fe9d006c3c0 2024-04-25T10:09:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.858+0000 7fe9d851b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d006cbe0 con 0x7fe9d0099f80 2024-04-25T10:09:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.858+0000 7fe9d851b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d00cbe50 con 0x7fe9d015ba10 2024-04-25T10:09:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.858+0000 7fe9d62b7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe9d015ba10 0x7fe9d0099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.858+0000 7fe9d6ab8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe9d006c3c0 0x7fe9d00cd220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50436/0 (socket says 172.21.15.16:50436) 2024-04-25T10:09:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.858+0000 7fe9d6ab8640 1 -- 172.21.15.16:0/3316535592 learned_addr learned my addr 172.21.15.16:0/3316535592 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.858+0000 7fe9d62b7640 1 -- 172.21.15.16:0/3316535592 >> v1:172.21.15.186:6789/0 conn(0x7fe9d0099f80 legacy=0x7fe9d006bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.858+0000 7fe9d62b7640 1 -- 172.21.15.16:0/3316535592 >> v1:172.21.15.16:6789/0 conn(0x7fe9d006c3c0 legacy=0x7fe9d00cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.858+0000 7fe9d62b7640 1 -- 172.21.15.16:0/3316535592 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d006cec0 con 0x7fe9d015ba10 2024-04-25T10:09:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.859+0000 7fe9d62b7640 1 --2- 172.21.15.16:0/3316535592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d015ba10 0x7fe9d0099a40 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe9c80097a0 tx=0x7fe9c802a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31c60a04ba9db67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.859+0000 7fe9d52b5640 1 -- 172.21.15.16:0/3316535592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8037070 con 0x7fe9d015ba10 2024-04-25T10:09:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.859+0000 7fe9d52b5640 1 -- 172.21.15.16:0/3316535592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c802ad60 con 0x7fe9d015ba10 2024-04-25T10:09:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.859+0000 7fe9d52b5640 1 -- 172.21.15.16:0/3316535592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8032660 con 0x7fe9d015ba10 2024-04-25T10:09:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.859+0000 7fe9d851b640 1 -- 172.21.15.16:0/3316535592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d015ba10 msgr2=0x7fe9d0099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.859+0000 7fe9d851b640 1 --2- 172.21.15.16:0/3316535592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d015ba10 0x7fe9d0099a40 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe9c80097a0 tx=0x7fe9c802a0d0 comp rx=0 tx=0).stop 2024-04-25T10:09:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.860+0000 7fe9d851b640 1 -- 172.21.15.16:0/3316535592 shutdown_connections 2024-04-25T10:09:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.860+0000 7fe9d851b640 1 --2- 172.21.15.16:0/3316535592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d015ba10 0x7fe9d0099a40 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.860+0000 7fe9d851b640 1 -- 172.21.15.16:0/3316535592 >> 172.21.15.16:0/3316535592 conn(0x7fe9d00b32f0 msgr2=0x7fe9d00b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.860+0000 7fe9d851b640 1 -- 172.21.15.16:0/3316535592 shutdown_connections 2024-04-25T10:09:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.860+0000 7fe9d851b640 1 -- 172.21.15.16:0/3316535592 wait complete. 2024-04-25T10:09:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.861+0000 7fe9d851b640 1 Processor -- start 2024-04-25T10:09:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.861+0000 7fe9d851b640 1 -- start start 2024-04-25T10:09:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.861+0000 7fe9d851b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9d006c3c0 0x7fe9d00c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d851b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9d0099f80 0x7fe9d00ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d851b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d015ba10 0x7fe9d00b6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d851b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d009a3d0 con 0x7fe9d015ba10 2024-04-25T10:09:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d851b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d00d1530 con 0x7fe9d006c3c0 2024-04-25T10:09:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d851b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d00cfe80 con 0x7fe9d0099f80 2024-04-25T10:09:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d6ab8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d015ba10 0x7fe9d00b6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d6ab8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d015ba10 0x7fe9d00b6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52374/0 (socket says 172.21.15.16:52374) 2024-04-25T10:09:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d6ab8640 1 -- 172.21.15.16:0/2117377558 learned_addr learned my addr 172.21.15.16:0/2117377558 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d62b7640 1 --2- 172.21.15.16:0/2117377558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9d006c3c0 0x7fe9d00c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d5ab6640 1 --2- 172.21.15.16:0/2117377558 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9d0099f80 0x7fe9d00ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d6ab8640 1 -- 172.21.15.16:0/2117377558 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9d0099f80 msgr2=0x7fe9d00ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d6ab8640 1 --2- 172.21.15.16:0/2117377558 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9d0099f80 0x7fe9d00ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d6ab8640 1 -- 172.21.15.16:0/2117377558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9d006c3c0 msgr2=0x7fe9d00c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d6ab8640 1 --2- 172.21.15.16:0/2117377558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9d006c3c0 0x7fe9d00c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d6ab8640 1 -- 172.21.15.16:0/2117377558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d00b5d10 con 0x7fe9d015ba10 2024-04-25T10:09:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d62b7640 1 --2- 172.21.15.16:0/2117377558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9d006c3c0 0x7fe9d00c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d5ab6640 1 --2- 172.21.15.16:0/2117377558 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9d0099f80 0x7fe9d00ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d6ab8640 1 --2- 172.21.15.16:0/2117377558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d015ba10 0x7fe9d00b6a90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000cb20 tx=0x7fe9c0002080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9ceffd640 1 -- 172.21.15.16:0/2117377558 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0018070 con 0x7fe9d015ba10 2024-04-25T10:09:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9ceffd640 1 -- 172.21.15.16:0/2117377558 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c000a700 con 0x7fe9d015ba10 2024-04-25T10:09:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d851b640 1 -- 172.21.15.16:0/2117377558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d0151530 con 0x7fe9d015ba10 2024-04-25T10:09:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9ceffd640 1 -- 172.21.15.16:0/2117377558 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c000aad0 con 0x7fe9d015ba10 2024-04-25T10:09:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.862+0000 7fe9d851b640 1 -- 172.21.15.16:0/2117377558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d00b6290 con 0x7fe9d015ba10 2024-04-25T10:09:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.863+0000 7fe9d851b640 1 -- 172.21.15.16:0/2117377558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d00749b0 con 0x7fe9d015ba10 2024-04-25T10:09:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.864+0000 7fe9ceffd640 1 -- 172.21.15.16:0/2117377558 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9c001b070 con 0x7fe9d015ba10 2024-04-25T10:09:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.864+0000 7fe9ceffd640 1 --2- 172.21.15.16:0/2117377558 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe9bc06b4d0 0x7fe9bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.864+0000 7fe9ceffd640 1 -- 172.21.15.16:0/2117377558 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9c0089950 con 0x7fe9d015ba10 2024-04-25T10:09:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.865+0000 7fe9d62b7640 1 --2- 172.21.15.16:0/2117377558 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe9bc06b4d0 0x7fe9bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.865+0000 7fe9d62b7640 1 --2- 172.21.15.16:0/2117377558 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe9bc06b4d0 0x7fe9bc06d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8034000 tx=0x7fe9c802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:24.869+0000 7fe9ceffd640 1 -- 172.21.15.16:0/2117377558 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9c000ac70 con 0x7fe9d015ba10 2024-04-25T10:09:25.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.086+0000 7fe9d851b640 1 -- 172.21.15.16:0/2117377558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9d00c8360 con 0x7fe9d015ba10 2024-04-25T10:09:25.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.087+0000 7fe9ceffd640 1 -- 172.21.15.16:0/2117377558 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fe9c00587d0 con 0x7fe9d015ba10 2024-04-25T10:09:25.093 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:25.093 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6070,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2805428272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2805428272},{"type":"v1","addr":"172.21.15.186:6835","nonce":2805428272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2897154125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2897154125},{"type":"v1","addr":"172.21.15.16:6833","nonce":2897154125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6636,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1735811292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1735811292},{"type":"v1","addr":"172.21.15.16:6835","nonce":1735811292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:22.077032+0000","modified":"2024-04-25T10:09:24.068248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35489":{"gid":35489,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/3292077067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3292077067},{"type":"v1","addr":"172.21.15.186:6837","nonce":3292077067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:09:25.093 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 36 2024-04-25T10:09:25.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.089+0000 7fe9d851b640 1 -- 172.21.15.16:0/2117377558 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe9bc06b4d0 msgr2=0x7fe9bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.089+0000 7fe9d851b640 1 --2- 172.21.15.16:0/2117377558 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe9bc06b4d0 0x7fe9bc06d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8034000 tx=0x7fe9c802e000 comp rx=0 tx=0).stop 2024-04-25T10:09:25.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.089+0000 7fe9d851b640 1 -- 172.21.15.16:0/2117377558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d015ba10 msgr2=0x7fe9d00b6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.089+0000 7fe9d851b640 1 --2- 172.21.15.16:0/2117377558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d015ba10 0x7fe9d00b6a90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000cb20 tx=0x7fe9c0002080 comp rx=0 tx=0).stop 2024-04-25T10:09:25.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.089+0000 7fe9d851b640 1 -- 172.21.15.16:0/2117377558 shutdown_connections 2024-04-25T10:09:25.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.089+0000 7fe9d851b640 1 --2- 172.21.15.16:0/2117377558 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe9bc06b4d0 0x7fe9bc06d990 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.089+0000 7fe9d851b640 1 --2- 172.21.15.16:0/2117377558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d015ba10 0x7fe9d00b6a90 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.089+0000 7fe9d851b640 1 --2- 172.21.15.16:0/2117377558 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9d0099f80 0x7fe9d00ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.089+0000 7fe9d851b640 1 --2- 172.21.15.16:0/2117377558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9d006c3c0 0x7fe9d00c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.089+0000 7fe9d851b640 1 -- 172.21.15.16:0/2117377558 >> 172.21.15.16:0/2117377558 conn(0x7fe9d00b32f0 msgr2=0x7fe9d015ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.089+0000 7fe9d851b640 1 -- 172.21.15.16:0/2117377558 shutdown_connections 2024-04-25T10:09:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.089+0000 7fe9d851b640 1 -- 172.21.15.16:0/2117377558 wait complete. 2024-04-25T10:09:25.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:25.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.201+0000 7f86b6cdd640 1 Processor -- start 2024-04-25T10:09:25.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.201+0000 7f86b6cdd640 1 -- start start 2024-04-25T10:09:25.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.202+0000 7f86b6cdd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86b015a870 0x7f86b00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.202+0000 7f86b6cdd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f86b00cfc40 0x7f86b015ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.202+0000 7f86b6cdd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b00cdb10 con 0x7f86b006e2e0 2024-04-25T10:09:25.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.202+0000 7f86b6cdd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b00ce720 con 0x7f86b00cfc40 2024-04-25T10:09:25.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.202+0000 7f86b6cdd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b00cddf0 con 0x7f86b015a870 2024-04-25T10:09:25.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.202+0000 7f86b527a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f86b00cfc40 0x7f86b015ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.202+0000 7f86b527a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f86b00cfc40 0x7f86b015ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52388/0 (socket says 172.21.15.16:52388) 2024-04-25T10:09:25.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.202+0000 7f86b527a640 1 -- 172.21.15.16:0/157003916 learned_addr learned my addr 172.21.15.16:0/157003916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:25.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.202+0000 7f86b4a79640 1 --2- 172.21.15.16:0/157003916 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86b015a870 0x7f86b00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.202+0000 7f86b527a640 1 -- 172.21.15.16:0/157003916 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86b015a870 msgr2=0x7f86b00cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.202+0000 7f86b527a640 1 --2- 172.21.15.16:0/157003916 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86b015a870 0x7f86b00cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.202+0000 7f86b527a640 1 -- 172.21.15.16:0/157003916 >> v1:172.21.15.186:6789/0 conn(0x7f86b006e2e0 legacy=0x7f86b00cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.202+0000 7f86b527a640 1 -- 172.21.15.16:0/157003916 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b00ce0d0 con 0x7f86b00cfc40 2024-04-25T10:09:25.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.202+0000 7f86b4a79640 1 --2- 172.21.15.16:0/157003916 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86b015a870 0x7f86b00cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:25.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.203+0000 7f86b527a640 1 --2- 172.21.15.16:0/157003916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b00cfc40 0x7f86b015ac50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009e50 tx=0x7f86a802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99707b4ae99d6d9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.203+0000 7f86af7fe640 1 -- 172.21.15.16:0/157003916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a8036070 con 0x7f86b00cfc40 2024-04-25T10:09:25.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.203+0000 7f86af7fe640 1 -- 172.21.15.16:0/157003916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a8031600 con 0x7f86b00cfc40 2024-04-25T10:09:25.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.203+0000 7f86af7fe640 1 -- 172.21.15.16:0/157003916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a80319d0 con 0x7f86b00cfc40 2024-04-25T10:09:25.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.203+0000 7f86b6cdd640 1 -- 172.21.15.16:0/157003916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b00cfc40 msgr2=0x7f86b015ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.203+0000 7f86b6cdd640 1 --2- 172.21.15.16:0/157003916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b00cfc40 0x7f86b015ac50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009e50 tx=0x7f86a802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.204+0000 7f86b6cdd640 1 -- 172.21.15.16:0/157003916 shutdown_connections 2024-04-25T10:09:25.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.204+0000 7f86b6cdd640 1 --2- 172.21.15.16:0/157003916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b00cfc40 0x7f86b015ac50 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.204+0000 7f86b6cdd640 1 --2- 172.21.15.16:0/157003916 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86b015a870 0x7f86b00cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.204+0000 7f86b6cdd640 1 -- 172.21.15.16:0/157003916 >> 172.21.15.16:0/157003916 conn(0x7f86b006aba0 msgr2=0x7f86b006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:25.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.204+0000 7f86b6cdd640 1 -- 172.21.15.16:0/157003916 shutdown_connections 2024-04-25T10:09:25.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.204+0000 7f86b6cdd640 1 -- 172.21.15.16:0/157003916 wait complete. 2024-04-25T10:09:25.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.205+0000 7f86b6cdd640 1 Processor -- start 2024-04-25T10:09:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.205+0000 7f86b6cdd640 1 -- start start 2024-04-25T10:09:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86b6cdd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86b006e2e0 0x7f86b014fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86b6cdd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86b00cfc40 0x7f86b0156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86b6cdd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b015a870 0x7f86b01e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86b6cdd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b00cd700 con 0x7f86b015a870 2024-04-25T10:09:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86b6cdd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b00cd580 con 0x7f86b00cfc40 2024-04-25T10:09:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86b6cdd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b00cd880 con 0x7f86b006e2e0 2024-04-25T10:09:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86b527a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b015a870 0x7f86b01e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86b527a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b015a870 0x7f86b01e73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52402/0 (socket says 172.21.15.16:52402) 2024-04-25T10:09:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86b527a640 1 -- 172.21.15.16:0/4201739507 learned_addr learned my addr 172.21.15.16:0/4201739507 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86b4a79640 1 --2- 172.21.15.16:0/4201739507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86b006e2e0 0x7f86b014fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86affff640 1 --2- 172.21.15.16:0/4201739507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86b00cfc40 0x7f86b0156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86b527a640 1 -- 172.21.15.16:0/4201739507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86b006e2e0 msgr2=0x7f86b014fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86b527a640 1 --2- 172.21.15.16:0/4201739507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86b006e2e0 0x7f86b014fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86b527a640 1 -- 172.21.15.16:0/4201739507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86b00cfc40 msgr2=0x7f86b0156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86b527a640 1 --2- 172.21.15.16:0/4201739507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86b00cfc40 0x7f86b0156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86b527a640 1 -- 172.21.15.16:0/4201739507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b0157ee0 con 0x7f86b015a870 2024-04-25T10:09:25.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86affff640 1 --2- 172.21.15.16:0/4201739507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86b00cfc40 0x7f86b0156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:25.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86b527a640 1 --2- 172.21.15.16:0/4201739507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b015a870 0x7f86b01e73f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f86a8034040 tx=0x7f86a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86ad7fa640 1 -- 172.21.15.16:0/4201739507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a8036050 con 0x7f86b015a870 2024-04-25T10:09:25.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.206+0000 7f86ad7fa640 1 -- 172.21.15.16:0/4201739507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a8042070 con 0x7f86b015a870 2024-04-25T10:09:25.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.207+0000 7f86b6cdd640 1 -- 172.21.15.16:0/4201739507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b01581c0 con 0x7f86b015a870 2024-04-25T10:09:25.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.207+0000 7f86ad7fa640 1 -- 172.21.15.16:0/4201739507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a8009510 con 0x7f86b015a870 2024-04-25T10:09:25.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.207+0000 7f86b6cdd640 1 -- 172.21.15.16:0/4201739507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b0158780 con 0x7f86b015a870 2024-04-25T10:09:25.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.208+0000 7f86b6cdd640 1 -- 172.21.15.16:0/4201739507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b00749b0 con 0x7f86b015a870 2024-04-25T10:09:25.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.208+0000 7f86ad7fa640 1 -- 172.21.15.16:0/4201739507 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f86a8007bd0 con 0x7f86b015a870 2024-04-25T10:09:25.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.208+0000 7f86ad7fa640 1 --2- 172.21.15.16:0/4201739507 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f869406b4d0 0x7f869406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.208+0000 7f86ad7fa640 1 -- 172.21.15.16:0/4201739507 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86a80ab1f0 con 0x7f86b015a870 2024-04-25T10:09:25.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.209+0000 7f86b4a79640 1 --2- 172.21.15.16:0/4201739507 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f869406b4d0 0x7f869406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.209+0000 7f86b4a79640 1 --2- 172.21.15.16:0/4201739507 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f869406b4d0 0x7f869406d990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f86a400b840 tx=0x7f86a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.213+0000 7f86ad7fa640 1 -- 172.21.15.16:0/4201739507 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f86a807a070 con 0x7f86b015a870 2024-04-25T10:09:25.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.392+0000 7f86b6cdd640 1 -- 172.21.15.16:0/4201739507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86b015bbc0 con 0x7f86b015a870 2024-04-25T10:09:25.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.393+0000 7f86ad7fa640 1 -- 172.21.15.16:0/4201739507 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16608 (secure 0 0 0) 0x7f86a8079970 con 0x7f86b015a870 2024-04-25T10:09:25.400 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:25.400 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":72,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:09:23.718608+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:20.361009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:21.048892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:25.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.395+0000 7f86b6cdd640 1 -- 172.21.15.16:0/4201739507 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f869406b4d0 msgr2=0x7f869406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.395+0000 7f86b6cdd640 1 --2- 172.21.15.16:0/4201739507 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f869406b4d0 0x7f869406d990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f86a400b840 tx=0x7f86a4008040 comp rx=0 tx=0).stop 2024-04-25T10:09:25.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.395+0000 7f86b6cdd640 1 -- 172.21.15.16:0/4201739507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b015a870 msgr2=0x7f86b01e73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.395+0000 7f86b6cdd640 1 --2- 172.21.15.16:0/4201739507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b015a870 0x7f86b01e73f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f86a8034040 tx=0x7f86a8004a40 comp rx=0 tx=0).stop 2024-04-25T10:09:25.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.395+0000 7f86b6cdd640 1 -- 172.21.15.16:0/4201739507 shutdown_connections 2024-04-25T10:09:25.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.395+0000 7f86b6cdd640 1 --2- 172.21.15.16:0/4201739507 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f869406b4d0 0x7f869406d990 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.395+0000 7f86b6cdd640 1 --2- 172.21.15.16:0/4201739507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b015a870 0x7f86b01e73f0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.395+0000 7f86b6cdd640 1 --2- 172.21.15.16:0/4201739507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86b00cfc40 0x7f86b0156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.395+0000 7f86b6cdd640 1 --2- 172.21.15.16:0/4201739507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86b006e2e0 0x7f86b014fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.395+0000 7f86b6cdd640 1 -- 172.21.15.16:0/4201739507 >> 172.21.15.16:0/4201739507 conn(0x7f86b006aba0 msgr2=0x7f86b006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:25.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.395+0000 7f86b6cdd640 1 -- 172.21.15.16:0/4201739507 shutdown_connections 2024-04-25T10:09:25.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.395+0000 7f86b6cdd640 1 -- 172.21.15.16:0/4201739507 wait complete. 2024-04-25T10:09:25.412 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:09:25.413 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:09:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.508+0000 7fc6b73b0640 1 Processor -- start 2024-04-25T10:09:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.508+0000 7fc6b73b0640 1 -- start start 2024-04-25T10:09:25.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.509+0000 7fc6b73b0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6b015dce0 0x7fc6b00d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.509+0000 7fc6b73b0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6b00d35b0 0x7fc6b00b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.509+0000 7fc6b73b0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b015e0c0 con 0x7fc6b00d1f90 2024-04-25T10:09:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.509+0000 7fc6b73b0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b00b7b10 con 0x7fc6b015dce0 2024-04-25T10:09:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.509+0000 7fc6b73b0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b00d2510 con 0x7fc6b00d35b0 2024-04-25T10:09:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.509+0000 7fc6b514c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6b015dce0 0x7fc6b00d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.509+0000 7fc6b594d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc6b00d1f90 0x7fc6b006bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50438/0 (socket says 172.21.15.16:50438) 2024-04-25T10:09:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.509+0000 7fc6b594d640 1 -- 172.21.15.16:0/1587271111 learned_addr learned my addr 172.21.15.16:0/1587271111 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.509+0000 7fc6b514c640 1 -- 172.21.15.16:0/1587271111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6b00d35b0 msgr2=0x7fc6b00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.509+0000 7fc6b494b640 1 --2- 172.21.15.16:0/1587271111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6b00d35b0 0x7fc6b00b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.509+0000 7fc6b514c640 1 --2- 172.21.15.16:0/1587271111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6b00d35b0 0x7fc6b00b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.509+0000 7fc6b514c640 1 -- 172.21.15.16:0/1587271111 >> v1:172.21.15.16:6789/0 conn(0x7fc6b00d1f90 legacy=0x7fc6b006bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.509+0000 7fc6a7fff640 1 -- 172.21.15.16:0/1587271111 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136569359 0 0) 0x7fc6b015e0c0 con 0x7fc6b00d1f90 2024-04-25T10:09:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.509+0000 7fc6b514c640 1 -- 172.21.15.16:0/1587271111 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b00d27f0 con 0x7fc6b015dce0 2024-04-25T10:09:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.509+0000 7fc6b494b640 1 --2- 172.21.15.16:0/1587271111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6b00d35b0 0x7fc6b00b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.510+0000 7fc6b514c640 1 --2- 172.21.15.16:0/1587271111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b015dce0 0x7fc6b00d1a50 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac0097a0 tx=0x7fc6ac02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3c81a4b3d64ff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.510+0000 7fc6a7fff640 1 -- 172.21.15.16:0/1587271111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac037070 con 0x7fc6b015dce0 2024-04-25T10:09:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.510+0000 7fc6a7fff640 1 -- 172.21.15.16:0/1587271111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac02ae50 con 0x7fc6b015dce0 2024-04-25T10:09:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.510+0000 7fc6a7fff640 1 -- 172.21.15.16:0/1587271111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac032660 con 0x7fc6b015dce0 2024-04-25T10:09:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.510+0000 7fc6b73b0640 1 -- 172.21.15.16:0/1587271111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b015dce0 msgr2=0x7fc6b00d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.510+0000 7fc6b73b0640 1 --2- 172.21.15.16:0/1587271111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b015dce0 0x7fc6b00d1a50 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac0097a0 tx=0x7fc6ac02a100 comp rx=0 tx=0).stop 2024-04-25T10:09:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.510+0000 7fc6b73b0640 1 -- 172.21.15.16:0/1587271111 shutdown_connections 2024-04-25T10:09:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.510+0000 7fc6b73b0640 1 --2- 172.21.15.16:0/1587271111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6b00d35b0 0x7fc6b00b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.510+0000 7fc6b73b0640 1 --2- 172.21.15.16:0/1587271111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b015dce0 0x7fc6b00d1a50 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.510+0000 7fc6b73b0640 1 -- 172.21.15.16:0/1587271111 >> 172.21.15.16:0/1587271111 conn(0x7fc6b0072900 msgr2=0x7fc6b00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.511+0000 7fc6b73b0640 1 -- 172.21.15.16:0/1587271111 shutdown_connections 2024-04-25T10:09:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.511+0000 7fc6b73b0640 1 -- 172.21.15.16:0/1587271111 wait complete. 2024-04-25T10:09:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.512+0000 7fc6b73b0640 1 Processor -- start 2024-04-25T10:09:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.512+0000 7fc6b73b0640 1 -- start start 2024-04-25T10:09:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.512+0000 7fc6b73b0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6b00d1f90 0x7fc6b01f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.512+0000 7fc6b73b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b00d35b0 0x7fc6b014c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.512+0000 7fc6b73b0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6b015dce0 0x7fc6b014bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.512+0000 7fc6b73b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b00d3090 con 0x7fc6b00d35b0 2024-04-25T10:09:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.512+0000 7fc6b73b0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b00d2db0 con 0x7fc6b015dce0 2024-04-25T10:09:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.512+0000 7fc6b73b0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b00b9820 con 0x7fc6b00d1f90 2024-04-25T10:09:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.512+0000 7fc6b494b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b00d35b0 0x7fc6b014c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.513+0000 7fc6b494b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b00d35b0 0x7fc6b014c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52420/0 (socket says 172.21.15.16:52420) 2024-04-25T10:09:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.513+0000 7fc6b494b640 1 -- 172.21.15.16:0/3183468478 learned_addr learned my addr 172.21.15.16:0/3183468478 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.513+0000 7fc6b514c640 1 --2- 172.21.15.16:0/3183468478 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6b00d1f90 0x7fc6b01f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.513+0000 7fc6b594d640 1 --2- 172.21.15.16:0/3183468478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6b015dce0 0x7fc6b014bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.513+0000 7fc6b494b640 1 -- 172.21.15.16:0/3183468478 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6b00d1f90 msgr2=0x7fc6b01f3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.513+0000 7fc6b494b640 1 --2- 172.21.15.16:0/3183468478 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6b00d1f90 0x7fc6b01f3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.513+0000 7fc6b494b640 1 -- 172.21.15.16:0/3183468478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6b015dce0 msgr2=0x7fc6b014bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.513+0000 7fc6b494b640 1 --2- 172.21.15.16:0/3183468478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6b015dce0 0x7fc6b014bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.513+0000 7fc6b494b640 1 -- 172.21.15.16:0/3183468478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b00c7710 con 0x7fc6b00d35b0 2024-04-25T10:09:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.513+0000 7fc6b594d640 1 --2- 172.21.15.16:0/3183468478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6b015dce0 0x7fc6b014bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.513+0000 7fc6b494b640 1 --2- 172.21.15.16:0/3183468478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b00d35b0 0x7fc6b014c610 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000c7f0 tx=0x7fc6a000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.513+0000 7fc6a5ffb640 1 -- 172.21.15.16:0/3183468478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0015070 con 0x7fc6b00d35b0 2024-04-25T10:09:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.513+0000 7fc6a5ffb640 1 -- 172.21.15.16:0/3183468478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a0007a30 con 0x7fc6b00d35b0 2024-04-25T10:09:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.513+0000 7fc6b73b0640 1 -- 172.21.15.16:0/3183468478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b00c79f0 con 0x7fc6b00d35b0 2024-04-25T10:09:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.513+0000 7fc6a5ffb640 1 -- 172.21.15.16:0/3183468478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0007e00 con 0x7fc6b00d35b0 2024-04-25T10:09:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.513+0000 7fc6b73b0640 1 -- 172.21.15.16:0/3183468478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b00c7fb0 con 0x7fc6b00d35b0 2024-04-25T10:09:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.514+0000 7fc6b73b0640 1 -- 172.21.15.16:0/3183468478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b00749b0 con 0x7fc6b00d35b0 2024-04-25T10:09:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.515+0000 7fc6a5ffb640 1 -- 172.21.15.16:0/3183468478 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc6a0018070 con 0x7fc6b00d35b0 2024-04-25T10:09:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.515+0000 7fc6a5ffb640 1 --2- 172.21.15.16:0/3183468478 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc69406b4d0 0x7fc69406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.515+0000 7fc6a5ffb640 1 -- 172.21.15.16:0/3183468478 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6a008a100 con 0x7fc6b00d35b0 2024-04-25T10:09:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.515+0000 7fc6b514c640 1 --2- 172.21.15.16:0/3183468478 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc69406b4d0 0x7fc69406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.516+0000 7fc6b514c640 1 --2- 172.21.15.16:0/3183468478 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc69406b4d0 0x7fc69406d990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac034000 tx=0x7fc6ac02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.519+0000 7fc6a5ffb640 1 -- 172.21.15.16:0/3183468478 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc6a0058930 con 0x7fc6b00d35b0 2024-04-25T10:09:25.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.696+0000 7fc6b73b0640 1 -- 172.21.15.16:0/3183468478 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6b006b080 con 0x7fc69406b4d0 2024-04-25T10:09:25.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.703+0000 7fc6a5ffb640 1 -- 172.21.15.16:0/3183468478 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250058 (secure 0 0 0) 0x7fc6b006b080 con 0x7fc69406b4d0 2024-04-25T10:09:25.712 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:09:25.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.706+0000 7fc6b73b0640 1 -- 172.21.15.16:0/3183468478 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc69406b4d0 msgr2=0x7fc69406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.706+0000 7fc6b73b0640 1 --2- 172.21.15.16:0/3183468478 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc69406b4d0 0x7fc69406d990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac034000 tx=0x7fc6ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:09:25.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.706+0000 7fc6b73b0640 1 -- 172.21.15.16:0/3183468478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b00d35b0 msgr2=0x7fc6b014c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.706+0000 7fc6b73b0640 1 --2- 172.21.15.16:0/3183468478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b00d35b0 0x7fc6b014c610 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000c7f0 tx=0x7fc6a000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.706+0000 7fc6b73b0640 1 -- 172.21.15.16:0/3183468478 shutdown_connections 2024-04-25T10:09:25.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.706+0000 7fc6b73b0640 1 --2- 172.21.15.16:0/3183468478 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc69406b4d0 0x7fc69406d990 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.706+0000 7fc6b73b0640 1 --2- 172.21.15.16:0/3183468478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6b015dce0 0x7fc6b014bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.706+0000 7fc6b73b0640 1 --2- 172.21.15.16:0/3183468478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b00d35b0 0x7fc6b014c610 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.706+0000 7fc6b73b0640 1 --2- 172.21.15.16:0/3183468478 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6b00d1f90 0x7fc6b01f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.706+0000 7fc6b73b0640 1 -- 172.21.15.16:0/3183468478 >> 172.21.15.16:0/3183468478 conn(0x7fc6b0072900 msgr2=0x7fc6b00b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:25.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.706+0000 7fc6b73b0640 1 -- 172.21.15.16:0/3183468478 shutdown_connections 2024-04-25T10:09:25.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.706+0000 7fc6b73b0640 1 -- 172.21.15.16:0/3183468478 wait complete. 2024-04-25T10:09:25.723 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2024-04-25T10:09:23.840575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":168,"acting":168,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244716,"kb_used_data":7728,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730004,"statfs":{"total":773094113280,"available":772843524096,"internally_reserved":0,"allocated":7913472,"data_stored":3977044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000987"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.742094+0000","last_change":"2024-04-25T10:09:22.740700+0000","last_active":"2024-04-25T10:09:22.742094+0000","last_peered":"2024-04-25T10:09:22.742094+0000","last_clean":"2024-04-25T10:09:22.742094+0000","last_became_active":"2024-04-25T10:09:22.740086+0000","last_became_peered":"2024-04-25T10:09:22.740086+0000","last_unstale":"2024-04-25T10:09:22.742094+0000","last_undegraded":"2024-04-25T10:09:22.742094+0000","last_fullsized":"2024-04-25T10:09:22.742094+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:08.877974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.730366+0000","last_change":"2024-04-25T10:09:22.727919+0000","last_active":"2024-04-25T10:09:22.730366+0000","last_peered":"2024-04-25T10:09:22.730366+0000","last_clean":"2024-04-25T10:09:22.730366+0000","last_became_active":"2024-04-25T10:09:22.727518+0000","last_became_peered":"2024-04-25T10:09:22.727518+0000","last_unstale":"2024-04-25T10:09:22.730366+0000","last_undegraded":"2024-04-25T10:09:22.730366+0000","last_fullsized":"2024-04-25T10:09:22.730366+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:52.862877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741321+0000","last_change":"2024-04-25T10:09:22.737653+0000","last_active":"2024-04-25T10:09:22.741321+0000","last_peered":"2024-04-25T10:09:22.741321+0000","last_clean":"2024-04-25T10:09:22.741321+0000","last_became_active":"2024-04-25T10:09:22.737017+0000","last_became_peered":"2024-04-25T10:09:22.737017+0000","last_unstale":"2024-04-25T10:09:22.741321+0000","last_undegraded":"2024-04-25T10:09:22.741321+0000","last_fullsized":"2024-04-25T10:09:22.741321+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:08.346248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741243+0000","last_change":"2024-04-25T10:09:22.735283+0000","last_active":"2024-04-25T10:09:22.741243+0000","last_peered":"2024-04-25T10:09:22.741243+0000","last_clean":"2024-04-25T10:09:22.741243+0000","last_became_active":"2024-04-25T10:09:22.734973+0000","last_became_peered":"2024-04-25T10:09:22.734973+0000","last_unstale":"2024-04-25T10:09:22.741243+0000","last_undegraded":"2024-04-25T10:09:22.741243+0000","last_fullsized":"2024-04-25T10:09:22.741243+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:50.431542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:21.694875+0000","last_change":"2024-04-25T10:09:21.694875+0000","last_active":"2024-04-25T10:09:21.694875+0000","last_peered":"2024-04-25T10:09:21.694875+0000","last_clean":"2024-04-25T10:09:21.694875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:21.694875+0000","last_undegraded":"2024-04-25T10:09:21.694875+0000","last_fullsized":"2024-04-25T10:09:21.694875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.729024+0000","last_change":"2024-04-25T10:09:22.724555+0000","last_active":"2024-04-25T10:09:22.729024+0000","last_peered":"2024-04-25T10:09:22.729024+0000","last_clean":"2024-04-25T10:09:22.729024+0000","last_became_active":"2024-04-25T10:09:22.724224+0000","last_became_peered":"2024-04-25T10:09:22.724224+0000","last_unstale":"2024-04-25T10:09:22.729024+0000","last_undegraded":"2024-04-25T10:09:22.729024+0000","last_fullsized":"2024-04-25T10:09:22.729024+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:46.677718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.740277+0000","last_change":"2024-04-25T10:09:22.735978+0000","last_active":"2024-04-25T10:09:22.740277+0000","last_peered":"2024-04-25T10:09:22.740277+0000","last_clean":"2024-04-25T10:09:22.740277+0000","last_became_active":"2024-04-25T10:09:22.735712+0000","last_became_peered":"2024-04-25T10:09:22.735712+0000","last_unstale":"2024-04-25T10:09:22.740277+0000","last_undegraded":"2024-04-25T10:09:22.740277+0000","last_fullsized":"2024-04-25T10:09:22.740277+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:41.872942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735429+0000","last_change":"2024-04-25T10:09:22.722595+0000","last_active":"2024-04-25T10:09:22.735429+0000","last_peered":"2024-04-25T10:09:22.735429+0000","last_clean":"2024-04-25T10:09:22.735429+0000","last_became_active":"2024-04-25T10:09:22.721683+0000","last_became_peered":"2024-04-25T10:09:22.721683+0000","last_unstale":"2024-04-25T10:09:22.735429+0000","last_undegraded":"2024-04-25T10:09:22.735429+0000","last_fullsized":"2024-04-25T10:09:22.735429+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:00.988197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741848+0000","last_change":"2024-04-25T10:09:22.740092+0000","last_active":"2024-04-25T10:09:22.741848+0000","last_peered":"2024-04-25T10:09:22.741848+0000","last_clean":"2024-04-25T10:09:22.741848+0000","last_became_active":"2024-04-25T10:09:22.739795+0000","last_became_peered":"2024-04-25T10:09:22.739795+0000","last_unstale":"2024-04-25T10:09:22.741848+0000","last_undegraded":"2024-04-25T10:09:22.741848+0000","last_fullsized":"2024-04-25T10:09:22.741848+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:45.867130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741920+0000","last_change":"2024-04-25T10:09:22.740023+0000","last_active":"2024-04-25T10:09:22.741920+0000","last_peered":"2024-04-25T10:09:22.741920+0000","last_clean":"2024-04-25T10:09:22.741920+0000","last_became_active":"2024-04-25T10:09:22.739751+0000","last_became_peered":"2024-04-25T10:09:22.739751+0000","last_unstale":"2024-04-25T10:09:22.741920+0000","last_undegraded":"2024-04-25T10:09:22.741920+0000","last_fullsized":"2024-04-25T10:09:22.741920+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:12.347303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.736162+0000","last_change":"2024-04-25T10:09:22.731846+0000","last_active":"2024-04-25T10:09:22.736162+0000","last_peered":"2024-04-25T10:09:22.736162+0000","last_clean":"2024-04-25T10:09:22.736162+0000","last_became_active":"2024-04-25T10:09:22.731589+0000","last_became_peered":"2024-04-25T10:09:22.731589+0000","last_unstale":"2024-04-25T10:09:22.736162+0000","last_undegraded":"2024-04-25T10:09:22.736162+0000","last_fullsized":"2024-04-25T10:09:22.736162+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:59.127307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.734270+0000","last_change":"2024-04-25T10:09:22.726514+0000","last_active":"2024-04-25T10:09:22.734270+0000","last_peered":"2024-04-25T10:09:22.734270+0000","last_clean":"2024-04-25T10:09:22.734270+0000","last_became_active":"2024-04-25T10:09:22.725590+0000","last_became_peered":"2024-04-25T10:09:22.725590+0000","last_unstale":"2024-04-25T10:09:22.734270+0000","last_undegraded":"2024-04-25T10:09:22.734270+0000","last_fullsized":"2024-04-25T10:09:22.734270+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:44.497003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741488+0000","last_change":"2024-04-25T10:09:22.737785+0000","last_active":"2024-04-25T10:09:22.741488+0000","last_peered":"2024-04-25T10:09:22.741488+0000","last_clean":"2024-04-25T10:09:22.741488+0000","last_became_active":"2024-04-25T10:09:22.737513+0000","last_became_peered":"2024-04-25T10:09:22.737513+0000","last_unstale":"2024-04-25T10:09:22.741488+0000","last_undegraded":"2024-04-25T10:09:22.741488+0000","last_fullsized":"2024-04-25T10:09:22.741488+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:27.547340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.732846+0000","last_change":"2024-04-25T10:09:22.726657+0000","last_active":"2024-04-25T10:09:22.732846+0000","last_peered":"2024-04-25T10:09:22.732846+0000","last_clean":"2024-04-25T10:09:22.732846+0000","last_became_active":"2024-04-25T10:09:22.725689+0000","last_became_peered":"2024-04-25T10:09:22.725689+0000","last_unstale":"2024-04-25T10:09:22.732846+0000","last_undegraded":"2024-04-25T10:09:22.732846+0000","last_fullsized":"2024-04-25T10:09:22.732846+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:46.677718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.733132+0000","last_change":"2024-04-25T10:09:22.726406+0000","last_active":"2024-04-25T10:09:22.733132+0000","last_peered":"2024-04-25T10:09:22.733132+0000","last_clean":"2024-04-25T10:09:22.733132+0000","last_became_active":"2024-04-25T10:09:22.726122+0000","last_became_peered":"2024-04-25T10:09:22.726122+0000","last_unstale":"2024-04-25T10:09:22.733132+0000","last_undegraded":"2024-04-25T10:09:22.733132+0000","last_fullsized":"2024-04-25T10:09:22.733132+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:01.928851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735879+0000","last_change":"2024-04-25T10:09:22.727997+0000","last_active":"2024-04-25T10:09:22.735879+0000","last_peered":"2024-04-25T10:09:22.735879+0000","last_clean":"2024-04-25T10:09:22.735879+0000","last_became_active":"2024-04-25T10:09:22.727704+0000","last_became_peered":"2024-04-25T10:09:22.727704+0000","last_unstale":"2024-04-25T10:09:22.735879+0000","last_undegraded":"2024-04-25T10:09:22.735879+0000","last_fullsized":"2024-04-25T10:09:22.735879+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:26.658710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:21.694875+0000","last_change":"2024-04-25T10:09:21.694875+0000","last_active":"2024-04-25T10:09:21.694875+0000","last_peered":"2024-04-25T10:09:21.694875+0000","last_clean":"2024-04-25T10:09:21.694875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:21.694875+0000","last_undegraded":"2024-04-25T10:09:21.694875+0000","last_fullsized":"2024-04-25T10:09:21.694875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:21.694875+0000","last_change":"2024-04-25T10:09:21.694875+0000","last_active":"2024-04-25T10:09:21.694875+0000","last_peered":"2024-04-25T10:09:21.694875+0000","last_clean":"2024-04-25T10:09:21.694875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:21.694875+0000","last_undegraded":"2024-04-25T10:09:21.694875+0000","last_fullsized":"2024-04-25T10:09:21.694875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735919+0000","last_change":"2024-04-25T10:09:22.721738+0000","last_active":"2024-04-25T10:09:22.735919+0000","last_peered":"2024-04-25T10:09:22.735919+0000","last_clean":"2024-04-25T10:09:22.735919+0000","last_became_active":"2024-04-25T10:09:22.721357+0000","last_became_peered":"2024-04-25T10:09:22.721357+0000","last_unstale":"2024-04-25T10:09:22.735919+0000","last_undegraded":"2024-04-25T10:09:22.735919+0000","last_fullsized":"2024-04-25T10:09:22.735919+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:41.775227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:21.694875+0000","last_change":"2024-04-25T10:09:21.694875+0000","last_active":"2024-04-25T10:09:21.694875+0000","last_peered":"2024-04-25T10:09:21.694875+0000","last_clean":"2024-04-25T10:09:21.694875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:21.694875+0000","last_undegraded":"2024-04-25T10:09:21.694875+0000","last_fullsized":"2024-04-25T10:09:21.694875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.730567+0000","last_change":"2024-04-25T10:09:22.729031+0000","last_active":"2024-04-25T10:09:22.730567+0000","last_peered":"2024-04-25T10:09:22.730567+0000","last_clean":"2024-04-25T10:09:22.730567+0000","last_became_active":"2024-04-25T10:09:22.728748+0000","last_became_peered":"2024-04-25T10:09:22.728748+0000","last_unstale":"2024-04-25T10:09:22.730567+0000","last_undegraded":"2024-04-25T10:09:22.730567+0000","last_fullsized":"2024-04-25T10:09:22.730567+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:04.675050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741959+0000","last_change":"2024-04-25T10:09:22.740317+0000","last_active":"2024-04-25T10:09:22.741959+0000","last_peered":"2024-04-25T10:09:22.741959+0000","last_clean":"2024-04-25T10:09:22.741959+0000","last_became_active":"2024-04-25T10:09:22.740008+0000","last_became_peered":"2024-04-25T10:09:22.740008+0000","last_unstale":"2024-04-25T10:09:22.741959+0000","last_undegraded":"2024-04-25T10:09:22.741959+0000","last_fullsized":"2024-04-25T10:09:22.741959+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:04.675050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735407+0000","last_change":"2024-04-25T10:09:22.729817+0000","last_active":"2024-04-25T10:09:22.735407+0000","last_peered":"2024-04-25T10:09:22.735407+0000","last_clean":"2024-04-25T10:09:22.735407+0000","last_became_active":"2024-04-25T10:09:22.729498+0000","last_became_peered":"2024-04-25T10:09:22.729498+0000","last_unstale":"2024-04-25T10:09:22.735407+0000","last_undegraded":"2024-04-25T10:09:22.735407+0000","last_fullsized":"2024-04-25T10:09:22.735407+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:24.427888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741092+0000","last_change":"2024-04-25T10:09:22.736028+0000","last_active":"2024-04-25T10:09:22.741092+0000","last_peered":"2024-04-25T10:09:22.741092+0000","last_clean":"2024-04-25T10:09:22.741092+0000","last_became_active":"2024-04-25T10:09:22.735640+0000","last_became_peered":"2024-04-25T10:09:22.735640+0000","last_unstale":"2024-04-25T10:09:22.741092+0000","last_undegraded":"2024-04-25T10:09:22.741092+0000","last_fullsized":"2024-04-25T10:09:22.741092+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:44.624012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735373+0000","last_change":"2024-04-25T10:09:22.731701+0000","last_active":"2024-04-25T10:09:22.735373+0000","last_peered":"2024-04-25T10:09:22.735373+0000","last_clean":"2024-04-25T10:09:22.735373+0000","last_became_active":"2024-04-25T10:09:22.731421+0000","last_became_peered":"2024-04-25T10:09:22.731421+0000","last_unstale":"2024-04-25T10:09:22.735373+0000","last_undegraded":"2024-04-25T10:09:22.735373+0000","last_fullsized":"2024-04-25T10:09:22.735373+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:26.842520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.738443+0000","last_change":"2024-04-25T10:09:22.734761+0000","last_active":"2024-04-25T10:09:22.738443+0000","last_peered":"2024-04-25T10:09:22.738443+0000","last_clean":"2024-04-25T10:09:22.738443+0000","last_became_active":"2024-04-25T10:09:22.734460+0000","last_became_peered":"2024-04-25T10:09:22.734460+0000","last_unstale":"2024-04-25T10:09:22.738443+0000","last_undegraded":"2024-04-25T10:09:22.738443+0000","last_fullsized":"2024-04-25T10:09:22.738443+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:00.291315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:21.694875+0000","last_change":"2024-04-25T10:09:21.694875+0000","last_active":"2024-04-25T10:09:21.694875+0000","last_peered":"2024-04-25T10:09:21.694875+0000","last_clean":"2024-04-25T10:09:21.694875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:21.694875+0000","last_undegraded":"2024-04-25T10:09:21.694875+0000","last_fullsized":"2024-04-25T10:09:21.694875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.729520+0000","last_change":"2024-04-25T10:09:22.726315+0000","last_active":"2024-04-25T10:09:22.729520+0000","last_peered":"2024-04-25T10:09:22.729520+0000","last_clean":"2024-04-25T10:09:22.729520+0000","last_became_active":"2024-04-25T10:09:22.725776+0000","last_became_peered":"2024-04-25T10:09:22.725776+0000","last_unstale":"2024-04-25T10:09:22.729520+0000","last_undegraded":"2024-04-25T10:09:22.729520+0000","last_fullsized":"2024-04-25T10:09:22.729520+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:01.928851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741458+0000","last_change":"2024-04-25T10:09:22.736699+0000","last_active":"2024-04-25T10:09:22.741458+0000","last_peered":"2024-04-25T10:09:22.741458+0000","last_clean":"2024-04-25T10:09:22.741458+0000","last_became_active":"2024-04-25T10:09:22.736407+0000","last_became_peered":"2024-04-25T10:09:22.736407+0000","last_unstale":"2024-04-25T10:09:22.741458+0000","last_undegraded":"2024-04-25T10:09:22.741458+0000","last_fullsized":"2024-04-25T10:09:22.741458+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:35.012470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735398+0000","last_change":"2024-04-25T10:09:22.722355+0000","last_active":"2024-04-25T10:09:22.735398+0000","last_peered":"2024-04-25T10:09:22.735398+0000","last_clean":"2024-04-25T10:09:22.735398+0000","last_became_active":"2024-04-25T10:09:22.722064+0000","last_became_peered":"2024-04-25T10:09:22.722064+0000","last_unstale":"2024-04-25T10:09:22.735398+0000","last_undegraded":"2024-04-25T10:09:22.735398+0000","last_fullsized":"2024-04-25T10:09:22.735398+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:43.678534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.730338+0000","last_change":"2024-04-25T10:09:22.727553+0000","last_active":"2024-04-25T10:09:22.730338+0000","last_peered":"2024-04-25T10:09:22.730338+0000","last_clean":"2024-04-25T10:09:22.730338+0000","last_became_active":"2024-04-25T10:09:22.727223+0000","last_became_peered":"2024-04-25T10:09:22.727223+0000","last_unstale":"2024-04-25T10:09:22.730338+0000","last_undegraded":"2024-04-25T10:09:22.730338+0000","last_fullsized":"2024-04-25T10:09:22.730338+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:46.677718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.726296+0000","last_change":"2024-04-25T10:09:22.722740+0000","last_active":"2024-04-25T10:09:22.726296+0000","last_peered":"2024-04-25T10:09:22.726296+0000","last_clean":"2024-04-25T10:09:22.726296+0000","last_became_active":"2024-04-25T10:09:22.722477+0000","last_became_peered":"2024-04-25T10:09:22.722477+0000","last_unstale":"2024-04-25T10:09:22.726296+0000","last_undegraded":"2024-04-25T10:09:22.726296+0000","last_fullsized":"2024-04-25T10:09:22.726296+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:41.763989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.736685+0000","last_change":"2024-04-25T10:09:22.728881+0000","last_active":"2024-04-25T10:09:22.736685+0000","last_peered":"2024-04-25T10:09:22.736685+0000","last_clean":"2024-04-25T10:09:22.736685+0000","last_became_active":"2024-04-25T10:09:22.728568+0000","last_became_peered":"2024-04-25T10:09:22.728568+0000","last_unstale":"2024-04-25T10:09:22.736685+0000","last_undegraded":"2024-04-25T10:09:22.736685+0000","last_fullsized":"2024-04-25T10:09:22.736685+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:05.585375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"71'4","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.742536+0000","last_change":"2024-04-25T10:09:21.713002+0000","last_active":"2024-04-25T10:09:22.742536+0000","last_peered":"2024-04-25T10:09:22.742536+0000","last_clean":"2024-04-25T10:09:22.742536+0000","last_became_active":"2024-04-25T10:09:21.712704+0000","last_became_peered":"2024-04-25T10:09:21.712704+0000","last_unstale":"2024-04-25T10:09:22.742536+0000","last_undegraded":"2024-04-25T10:09:22.742536+0000","last_fullsized":"2024-04-25T10:09:22.742536+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:30.994528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741275+0000","last_change":"2024-04-25T10:09:22.733619+0000","last_active":"2024-04-25T10:09:22.741275+0000","last_peered":"2024-04-25T10:09:22.741275+0000","last_clean":"2024-04-25T10:09:22.741275+0000","last_became_active":"2024-04-25T10:09:22.733216+0000","last_became_peered":"2024-04-25T10:09:22.733216+0000","last_unstale":"2024-04-25T10:09:22.741275+0000","last_undegraded":"2024-04-25T10:09:22.741275+0000","last_fullsized":"2024-04-25T10:09:22.741275+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:39.734544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:20.690908+0000","last_change":"2024-04-25T10:09:20.690908+0000","last_active":"2024-04-25T10:09:20.690908+0000","last_peered":"2024-04-25T10:09:20.690908+0000","last_clean":"2024-04-25T10:09:20.690908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:20.690908+0000","last_undegraded":"2024-04-25T10:09:20.690908+0000","last_fullsized":"2024-04-25T10:09:20.690908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.728687+0000","last_change":"2024-04-25T10:09:22.724533+0000","last_active":"2024-04-25T10:09:22.728687+0000","last_peered":"2024-04-25T10:09:22.728687+0000","last_clean":"2024-04-25T10:09:22.728687+0000","last_became_active":"2024-04-25T10:09:22.724269+0000","last_became_peered":"2024-04-25T10:09:22.724269+0000","last_unstale":"2024-04-25T10:09:22.728687+0000","last_undegraded":"2024-04-25T10:09:22.728687+0000","last_fullsized":"2024-04-25T10:09:22.728687+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:44.497003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.736112+0000","last_change":"2024-04-25T10:09:21.721186+0000","last_active":"2024-04-25T10:09:22.736112+0000","last_peered":"2024-04-25T10:09:22.736112+0000","last_clean":"2024-04-25T10:09:22.736112+0000","last_became_active":"2024-04-25T10:09:21.720525+0000","last_became_peered":"2024-04-25T10:09:21.720525+0000","last_unstale":"2024-04-25T10:09:22.736112+0000","last_undegraded":"2024-04-25T10:09:22.736112+0000","last_fullsized":"2024-04-25T10:09:22.736112+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:00.537304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741112+0000","last_change":"2024-04-25T10:09:22.738338+0000","last_active":"2024-04-25T10:09:22.741112+0000","last_peered":"2024-04-25T10:09:22.741112+0000","last_clean":"2024-04-25T10:09:22.741112+0000","last_became_active":"2024-04-25T10:09:22.738084+0000","last_became_peered":"2024-04-25T10:09:22.738084+0000","last_unstale":"2024-04-25T10:09:22.741112+0000","last_undegraded":"2024-04-25T10:09:22.741112+0000","last_fullsized":"2024-04-25T10:09:22.741112+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:01.309230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.742333+0000","last_change":"2024-04-25T10:09:21.715338+0000","last_active":"2024-04-25T10:09:22.742333+0000","last_peered":"2024-04-25T10:09:22.742333+0000","last_clean":"2024-04-25T10:09:22.742333+0000","last_became_active":"2024-04-25T10:09:21.715072+0000","last_became_peered":"2024-04-25T10:09:21.715072+0000","last_unstale":"2024-04-25T10:09:22.742333+0000","last_undegraded":"2024-04-25T10:09:22.742333+0000","last_fullsized":"2024-04-25T10:09:22.742333+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:36.752649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741199+0000","last_change":"2024-04-25T10:09:22.736877+0000","last_active":"2024-04-25T10:09:22.741199+0000","last_peered":"2024-04-25T10:09:22.741199+0000","last_clean":"2024-04-25T10:09:22.741199+0000","last_became_active":"2024-04-25T10:09:22.736597+0000","last_became_peered":"2024-04-25T10:09:22.736597+0000","last_unstale":"2024-04-25T10:09:22.741199+0000","last_undegraded":"2024-04-25T10:09:22.741199+0000","last_fullsized":"2024-04-25T10:09:22.741199+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:57.265654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.734913+0000","last_change":"2024-04-25T10:09:21.719141+0000","last_active":"2024-04-25T10:09:22.734913+0000","last_peered":"2024-04-25T10:09:22.734913+0000","last_clean":"2024-04-25T10:09:22.734913+0000","last_became_active":"2024-04-25T10:09:21.718859+0000","last_became_peered":"2024-04-25T10:09:21.718859+0000","last_unstale":"2024-04-25T10:09:22.734913+0000","last_undegraded":"2024-04-25T10:09:22.734913+0000","last_fullsized":"2024-04-25T10:09:22.734913+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.907558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.725510+0000","last_change":"2024-04-25T10:09:22.720830+0000","last_active":"2024-04-25T10:09:22.725510+0000","last_peered":"2024-04-25T10:09:22.725510+0000","last_clean":"2024-04-25T10:09:22.725510+0000","last_became_active":"2024-04-25T10:09:22.720541+0000","last_became_peered":"2024-04-25T10:09:22.720541+0000","last_unstale":"2024-04-25T10:09:22.725510+0000","last_undegraded":"2024-04-25T10:09:22.725510+0000","last_fullsized":"2024-04-25T10:09:22.725510+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:01.928851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.732780+0000","last_change":"2024-04-25T10:09:21.718524+0000","last_active":"2024-04-25T10:09:22.732780+0000","last_peered":"2024-04-25T10:09:22.732780+0000","last_clean":"2024-04-25T10:09:22.732780+0000","last_became_active":"2024-04-25T10:09:21.718244+0000","last_became_peered":"2024-04-25T10:09:21.718244+0000","last_unstale":"2024-04-25T10:09:22.732780+0000","last_undegraded":"2024-04-25T10:09:22.732780+0000","last_fullsized":"2024-04-25T10:09:22.732780+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:07.342282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741569+0000","last_change":"2024-04-25T10:09:22.735544+0000","last_active":"2024-04-25T10:09:22.741569+0000","last_peered":"2024-04-25T10:09:22.741569+0000","last_clean":"2024-04-25T10:09:22.741569+0000","last_became_active":"2024-04-25T10:09:22.735219+0000","last_became_peered":"2024-04-25T10:09:22.735219+0000","last_unstale":"2024-04-25T10:09:22.741569+0000","last_undegraded":"2024-04-25T10:09:22.741569+0000","last_fullsized":"2024-04-25T10:09:22.741569+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:31.998494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.721005+0000","last_change":"2024-04-25T10:09:21.718431+0000","last_active":"2024-04-25T10:09:22.721005+0000","last_peered":"2024-04-25T10:09:22.721005+0000","last_clean":"2024-04-25T10:09:22.721005+0000","last_became_active":"2024-04-25T10:09:21.718200+0000","last_became_peered":"2024-04-25T10:09:21.718200+0000","last_unstale":"2024-04-25T10:09:22.721005+0000","last_undegraded":"2024-04-25T10:09:22.721005+0000","last_fullsized":"2024-04-25T10:09:22.721005+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:17.676932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741386+0000","last_change":"2024-04-25T10:09:22.733006+0000","last_active":"2024-04-25T10:09:22.741386+0000","last_peered":"2024-04-25T10:09:22.741386+0000","last_clean":"2024-04-25T10:09:22.741386+0000","last_became_active":"2024-04-25T10:09:22.732611+0000","last_became_peered":"2024-04-25T10:09:22.732611+0000","last_unstale":"2024-04-25T10:09:22.741386+0000","last_undegraded":"2024-04-25T10:09:22.741386+0000","last_fullsized":"2024-04-25T10:09:22.741386+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:04.675050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.718277+0000","last_change":"2024-04-25T10:09:21.712492+0000","last_active":"2024-04-25T10:09:22.718277+0000","last_peered":"2024-04-25T10:09:22.718277+0000","last_clean":"2024-04-25T10:09:22.718277+0000","last_became_active":"2024-04-25T10:09:21.712144+0000","last_became_peered":"2024-04-25T10:09:21.712144+0000","last_unstale":"2024-04-25T10:09:22.718277+0000","last_undegraded":"2024-04-25T10:09:22.718277+0000","last_fullsized":"2024-04-25T10:09:22.718277+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:50.101256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.737045+0000","last_change":"2024-04-25T10:09:22.732594+0000","last_active":"2024-04-25T10:09:22.737045+0000","last_peered":"2024-04-25T10:09:22.737045+0000","last_clean":"2024-04-25T10:09:22.737045+0000","last_became_active":"2024-04-25T10:09:22.732206+0000","last_became_peered":"2024-04-25T10:09:22.732206+0000","last_unstale":"2024-04-25T10:09:22.737045+0000","last_undegraded":"2024-04-25T10:09:22.737045+0000","last_fullsized":"2024-04-25T10:09:22.737045+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:12.628900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.714576+0000","last_change":"2024-04-25T10:09:21.716690+0000","last_active":"2024-04-25T10:09:22.714576+0000","last_peered":"2024-04-25T10:09:22.714576+0000","last_clean":"2024-04-25T10:09:22.714576+0000","last_became_active":"2024-04-25T10:09:21.716399+0000","last_became_peered":"2024-04-25T10:09:21.716399+0000","last_unstale":"2024-04-25T10:09:22.714576+0000","last_undegraded":"2024-04-25T10:09:22.714576+0000","last_fullsized":"2024-04-25T10:09:22.714576+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:44.475563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.739376+0000","last_change":"2024-04-25T10:09:22.735326+0000","last_active":"2024-04-25T10:09:22.739376+0000","last_peered":"2024-04-25T10:09:22.739376+0000","last_clean":"2024-04-25T10:09:22.739376+0000","last_became_active":"2024-04-25T10:09:22.734995+0000","last_became_peered":"2024-04-25T10:09:22.734995+0000","last_unstale":"2024-04-25T10:09:22.739376+0000","last_undegraded":"2024-04-25T10:09:22.739376+0000","last_fullsized":"2024-04-25T10:09:22.739376+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:07.878932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.715845+0000","last_change":"2024-04-25T10:09:21.715138+0000","last_active":"2024-04-25T10:09:22.715845+0000","last_peered":"2024-04-25T10:09:22.715845+0000","last_clean":"2024-04-25T10:09:22.715845+0000","last_became_active":"2024-04-25T10:09:21.714766+0000","last_became_peered":"2024-04-25T10:09:21.714766+0000","last_unstale":"2024-04-25T10:09:22.715845+0000","last_undegraded":"2024-04-25T10:09:22.715845+0000","last_fullsized":"2024-04-25T10:09:22.715845+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:30.994528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735372+0000","last_change":"2024-04-25T10:09:22.721662+0000","last_active":"2024-04-25T10:09:22.735372+0000","last_peered":"2024-04-25T10:09:22.735372+0000","last_clean":"2024-04-25T10:09:22.735372+0000","last_became_active":"2024-04-25T10:09:22.721339+0000","last_became_peered":"2024-04-25T10:09:22.721339+0000","last_unstale":"2024-04-25T10:09:22.735372+0000","last_undegraded":"2024-04-25T10:09:22.735372+0000","last_fullsized":"2024-04-25T10:09:22.735372+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:36.005374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"71'3","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.740562+0000","last_change":"2024-04-25T10:09:21.714788+0000","last_active":"2024-04-25T10:09:22.740562+0000","last_peered":"2024-04-25T10:09:22.740562+0000","last_clean":"2024-04-25T10:09:22.740562+0000","last_became_active":"2024-04-25T10:09:21.714533+0000","last_became_peered":"2024-04-25T10:09:21.714533+0000","last_unstale":"2024-04-25T10:09:22.740562+0000","last_undegraded":"2024-04-25T10:09:22.740562+0000","last_fullsized":"2024-04-25T10:09:22.740562+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:36.752649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:21.694875+0000","last_change":"2024-04-25T10:09:21.694875+0000","last_active":"2024-04-25T10:09:21.694875+0000","last_peered":"2024-04-25T10:09:21.694875+0000","last_clean":"2024-04-25T10:09:21.694875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:21.694875+0000","last_undegraded":"2024-04-25T10:09:21.694875+0000","last_fullsized":"2024-04-25T10:09:21.694875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"71'6","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.743850+0000","last_change":"2024-04-25T10:09:21.712915+0000","last_active":"2024-04-25T10:09:22.743850+0000","last_peered":"2024-04-25T10:09:22.743850+0000","last_clean":"2024-04-25T10:09:22.743850+0000","last_became_active":"2024-04-25T10:09:21.712644+0000","last_became_peered":"2024-04-25T10:09:21.712644+0000","last_unstale":"2024-04-25T10:09:22.743850+0000","last_undegraded":"2024-04-25T10:09:22.743850+0000","last_fullsized":"2024-04-25T10:09:22.743850+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:06.170911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.737377+0000","last_change":"2024-04-25T10:09:22.733894+0000","last_active":"2024-04-25T10:09:22.737377+0000","last_peered":"2024-04-25T10:09:22.737377+0000","last_clean":"2024-04-25T10:09:22.737377+0000","last_became_active":"2024-04-25T10:09:22.733596+0000","last_became_peered":"2024-04-25T10:09:22.733596+0000","last_unstale":"2024-04-25T10:09:22.737377+0000","last_undegraded":"2024-04-25T10:09:22.737377+0000","last_fullsized":"2024-04-25T10:09:22.737377+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:52.862877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.736865+0000","last_change":"2024-04-25T10:09:21.716006+0000","last_active":"2024-04-25T10:09:22.736865+0000","last_peered":"2024-04-25T10:09:22.736865+0000","last_clean":"2024-04-25T10:09:22.736865+0000","last_became_active":"2024-04-25T10:09:21.715740+0000","last_became_peered":"2024-04-25T10:09:21.715740+0000","last_unstale":"2024-04-25T10:09:22.736865+0000","last_undegraded":"2024-04-25T10:09:22.736865+0000","last_fullsized":"2024-04-25T10:09:22.736865+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:17.217685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741155+0000","last_change":"2024-04-25T10:09:22.738577+0000","last_active":"2024-04-25T10:09:22.741155+0000","last_peered":"2024-04-25T10:09:22.741155+0000","last_clean":"2024-04-25T10:09:22.741155+0000","last_became_active":"2024-04-25T10:09:22.738329+0000","last_became_peered":"2024-04-25T10:09:22.738329+0000","last_unstale":"2024-04-25T10:09:22.741155+0000","last_undegraded":"2024-04-25T10:09:22.741155+0000","last_fullsized":"2024-04-25T10:09:22.741155+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:36.710950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.720776+0000","last_change":"2024-04-25T10:09:21.716512+0000","last_active":"2024-04-25T10:09:22.720776+0000","last_peered":"2024-04-25T10:09:22.720776+0000","last_clean":"2024-04-25T10:09:22.720776+0000","last_became_active":"2024-04-25T10:09:21.716233+0000","last_became_peered":"2024-04-25T10:09:21.716233+0000","last_unstale":"2024-04-25T10:09:22.720776+0000","last_undegraded":"2024-04-25T10:09:22.720776+0000","last_fullsized":"2024-04-25T10:09:22.720776+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:26.543373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:21.694875+0000","last_change":"2024-04-25T10:09:21.694875+0000","last_active":"2024-04-25T10:09:21.694875+0000","last_peered":"2024-04-25T10:09:21.694875+0000","last_clean":"2024-04-25T10:09:21.694875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:21.694875+0000","last_undegraded":"2024-04-25T10:09:21.694875+0000","last_fullsized":"2024-04-25T10:09:21.694875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.720039+0000","last_change":"2024-04-25T10:09:21.713643+0000","last_active":"2024-04-25T10:09:22.720039+0000","last_peered":"2024-04-25T10:09:22.720039+0000","last_clean":"2024-04-25T10:09:22.720039+0000","last_became_active":"2024-04-25T10:09:21.713359+0000","last_became_peered":"2024-04-25T10:09:21.713359+0000","last_unstale":"2024-04-25T10:09:22.720039+0000","last_undegraded":"2024-04-25T10:09:22.720039+0000","last_fullsized":"2024-04-25T10:09:22.720039+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:49.427576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741100+0000","last_change":"2024-04-25T10:09:22.733905+0000","last_active":"2024-04-25T10:09:22.741100+0000","last_peered":"2024-04-25T10:09:22.741100+0000","last_clean":"2024-04-25T10:09:22.741100+0000","last_became_active":"2024-04-25T10:09:22.733630+0000","last_became_peered":"2024-04-25T10:09:22.733630+0000","last_unstale":"2024-04-25T10:09:22.741100+0000","last_undegraded":"2024-04-25T10:09:22.741100+0000","last_fullsized":"2024-04-25T10:09:22.741100+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:08.686900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"71'8","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735236+0000","last_change":"2024-04-25T10:09:21.719246+0000","last_active":"2024-04-25T10:09:22.735236+0000","last_peered":"2024-04-25T10:09:22.735236+0000","last_clean":"2024-04-25T10:09:22.735236+0000","last_became_active":"2024-04-25T10:09:21.718931+0000","last_became_peered":"2024-04-25T10:09:21.718931+0000","last_unstale":"2024-04-25T10:09:22.735236+0000","last_undegraded":"2024-04-25T10:09:22.735236+0000","last_fullsized":"2024-04-25T10:09:22.735236+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:53.574499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.742293+0000","last_change":"2024-04-25T10:09:22.737752+0000","last_active":"2024-04-25T10:09:22.742293+0000","last_peered":"2024-04-25T10:09:22.742293+0000","last_clean":"2024-04-25T10:09:22.742293+0000","last_became_active":"2024-04-25T10:09:22.737400+0000","last_became_peered":"2024-04-25T10:09:22.737400+0000","last_unstale":"2024-04-25T10:09:22.742293+0000","last_undegraded":"2024-04-25T10:09:22.742293+0000","last_fullsized":"2024-04-25T10:09:22.742293+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:55.327479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741686+0000","last_change":"2024-04-25T10:09:21.718854+0000","last_active":"2024-04-25T10:09:22.741686+0000","last_peered":"2024-04-25T10:09:22.741686+0000","last_clean":"2024-04-25T10:09:22.741686+0000","last_became_active":"2024-04-25T10:09:21.718471+0000","last_became_peered":"2024-04-25T10:09:21.718471+0000","last_unstale":"2024-04-25T10:09:22.741686+0000","last_undegraded":"2024-04-25T10:09:22.741686+0000","last_fullsized":"2024-04-25T10:09:22.741686+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:59.803346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.730320+0000","last_change":"2024-04-25T10:09:22.727103+0000","last_active":"2024-04-25T10:09:22.730320+0000","last_peered":"2024-04-25T10:09:22.730320+0000","last_clean":"2024-04-25T10:09:22.730320+0000","last_became_active":"2024-04-25T10:09:22.726504+0000","last_became_peered":"2024-04-25T10:09:22.726504+0000","last_unstale":"2024-04-25T10:09:22.730320+0000","last_undegraded":"2024-04-25T10:09:22.730320+0000","last_fullsized":"2024-04-25T10:09:22.730320+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:44.497003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.742808+0000","last_change":"2024-04-25T10:09:21.714571+0000","last_active":"2024-04-25T10:09:22.742808+0000","last_peered":"2024-04-25T10:09:22.742808+0000","last_clean":"2024-04-25T10:09:22.742808+0000","last_became_active":"2024-04-25T10:09:21.713769+0000","last_became_peered":"2024-04-25T10:09:21.713769+0000","last_unstale":"2024-04-25T10:09:22.742808+0000","last_undegraded":"2024-04-25T10:09:22.742808+0000","last_fullsized":"2024-04-25T10:09:22.742808+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:36.752649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741220+0000","last_change":"2024-04-25T10:09:22.735039+0000","last_active":"2024-04-25T10:09:22.741220+0000","last_peered":"2024-04-25T10:09:22.741220+0000","last_clean":"2024-04-25T10:09:22.741220+0000","last_became_active":"2024-04-25T10:09:22.734774+0000","last_became_peered":"2024-04-25T10:09:22.734774+0000","last_unstale":"2024-04-25T10:09:22.741220+0000","last_undegraded":"2024-04-25T10:09:22.741220+0000","last_fullsized":"2024-04-25T10:09:22.741220+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:54.578465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.743203+0000","last_change":"2024-04-25T10:09:21.716949+0000","last_active":"2024-04-25T10:09:22.743203+0000","last_peered":"2024-04-25T10:09:22.743203+0000","last_clean":"2024-04-25T10:09:22.743203+0000","last_became_active":"2024-04-25T10:09:21.716630+0000","last_became_peered":"2024-04-25T10:09:21.716630+0000","last_unstale":"2024-04-25T10:09:22.743203+0000","last_undegraded":"2024-04-25T10:09:22.743203+0000","last_fullsized":"2024-04-25T10:09:22.743203+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:17.676932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741354+0000","last_change":"2024-04-25T10:09:22.737902+0000","last_active":"2024-04-25T10:09:22.741354+0000","last_peered":"2024-04-25T10:09:22.741354+0000","last_clean":"2024-04-25T10:09:22.741354+0000","last_became_active":"2024-04-25T10:09:22.737380+0000","last_became_peered":"2024-04-25T10:09:22.737380+0000","last_unstale":"2024-04-25T10:09:22.741354+0000","last_undegraded":"2024-04-25T10:09:22.741354+0000","last_fullsized":"2024-04-25T10:09:22.741354+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:31.226453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.710382+0000","last_change":"2024-04-25T10:09:21.716162+0000","last_active":"2024-04-25T10:09:22.710382+0000","last_peered":"2024-04-25T10:09:22.710382+0000","last_clean":"2024-04-25T10:09:22.710382+0000","last_became_active":"2024-04-25T10:09:21.715886+0000","last_became_peered":"2024-04-25T10:09:21.715886+0000","last_unstale":"2024-04-25T10:09:22.710382+0000","last_undegraded":"2024-04-25T10:09:22.710382+0000","last_fullsized":"2024-04-25T10:09:22.710382+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.760023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741392+0000","last_change":"2024-04-25T10:09:22.733689+0000","last_active":"2024-04-25T10:09:22.741392+0000","last_peered":"2024-04-25T10:09:22.741392+0000","last_clean":"2024-04-25T10:09:22.741392+0000","last_became_active":"2024-04-25T10:09:22.733420+0000","last_became_peered":"2024-04-25T10:09:22.733420+0000","last_unstale":"2024-04-25T10:09:22.741392+0000","last_undegraded":"2024-04-25T10:09:22.741392+0000","last_fullsized":"2024-04-25T10:09:22.741392+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:35.911524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.710603+0000","last_change":"2024-04-25T10:09:21.713011+0000","last_active":"2024-04-25T10:09:22.710603+0000","last_peered":"2024-04-25T10:09:22.710603+0000","last_clean":"2024-04-25T10:09:22.710603+0000","last_became_active":"2024-04-25T10:09:21.712653+0000","last_became_peered":"2024-04-25T10:09:21.712653+0000","last_unstale":"2024-04-25T10:09:22.710603+0000","last_undegraded":"2024-04-25T10:09:22.710603+0000","last_fullsized":"2024-04-25T10:09:22.710603+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:40.526086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741658+0000","last_change":"2024-04-25T10:09:22.738516+0000","last_active":"2024-04-25T10:09:22.741658+0000","last_peered":"2024-04-25T10:09:22.741658+0000","last_clean":"2024-04-25T10:09:22.741658+0000","last_became_active":"2024-04-25T10:09:22.738264+0000","last_became_peered":"2024-04-25T10:09:22.738264+0000","last_unstale":"2024-04-25T10:09:22.741658+0000","last_undegraded":"2024-04-25T10:09:22.741658+0000","last_fullsized":"2024-04-25T10:09:22.741658+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:37.756616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.715613+0000","last_change":"2024-04-25T10:09:21.707102+0000","last_active":"2024-04-25T10:09:22.715613+0000","last_peered":"2024-04-25T10:09:22.715613+0000","last_clean":"2024-04-25T10:09:22.715613+0000","last_became_active":"2024-04-25T10:09:21.706788+0000","last_became_peered":"2024-04-25T10:09:21.706788+0000","last_unstale":"2024-04-25T10:09:22.715613+0000","last_undegraded":"2024-04-25T10:09:22.715613+0000","last_fullsized":"2024-04-25T10:09:22.715613+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.907558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.738481+0000","last_change":"2024-04-25T10:09:22.730156+0000","last_active":"2024-04-25T10:09:22.738481+0000","last_peered":"2024-04-25T10:09:22.738481+0000","last_clean":"2024-04-25T10:09:22.738481+0000","last_became_active":"2024-04-25T10:09:22.729878+0000","last_became_peered":"2024-04-25T10:09:22.729878+0000","last_unstale":"2024-04-25T10:09:22.738481+0000","last_undegraded":"2024-04-25T10:09:22.738481+0000","last_fullsized":"2024-04-25T10:09:22.738481+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:52.862877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:20.690908+0000","last_change":"2024-04-25T10:09:20.690908+0000","last_active":"2024-04-25T10:09:20.690908+0000","last_peered":"2024-04-25T10:09:20.690908+0000","last_clean":"2024-04-25T10:09:20.690908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:20.690908+0000","last_undegraded":"2024-04-25T10:09:20.690908+0000","last_fullsized":"2024-04-25T10:09:20.690908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:21.694875+0000","last_change":"2024-04-25T10:09:21.694875+0000","last_active":"2024-04-25T10:09:21.694875+0000","last_peered":"2024-04-25T10:09:21.694875+0000","last_clean":"2024-04-25T10:09:21.694875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:21.694875+0000","last_undegraded":"2024-04-25T10:09:21.694875+0000","last_fullsized":"2024-04-25T10:09:21.694875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741460+0000","last_change":"2024-04-25T10:09:21.712999+0000","last_active":"2024-04-25T10:09:22.741460+0000","last_peered":"2024-04-25T10:09:22.741460+0000","last_clean":"2024-04-25T10:09:22.741460+0000","last_became_active":"2024-04-25T10:09:21.712715+0000","last_became_peered":"2024-04-25T10:09:21.712715+0000","last_unstale":"2024-04-25T10:09:22.741460+0000","last_undegraded":"2024-04-25T10:09:22.741460+0000","last_fullsized":"2024-04-25T10:09:22.741460+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:19.872440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.732689+0000","last_change":"2024-04-25T10:09:22.724675+0000","last_active":"2024-04-25T10:09:22.732689+0000","last_peered":"2024-04-25T10:09:22.732689+0000","last_clean":"2024-04-25T10:09:22.732689+0000","last_became_active":"2024-04-25T10:09:22.723506+0000","last_became_peered":"2024-04-25T10:09:22.723506+0000","last_unstale":"2024-04-25T10:09:22.732689+0000","last_undegraded":"2024-04-25T10:09:22.732689+0000","last_fullsized":"2024-04-25T10:09:22.732689+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:31.226453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.714801+0000","last_change":"2024-04-25T10:09:21.721913+0000","last_active":"2024-04-25T10:09:22.714801+0000","last_peered":"2024-04-25T10:09:22.714801+0000","last_clean":"2024-04-25T10:09:22.714801+0000","last_became_active":"2024-04-25T10:09:21.721616+0000","last_became_peered":"2024-04-25T10:09:21.721616+0000","last_unstale":"2024-04-25T10:09:22.714801+0000","last_undegraded":"2024-04-25T10:09:22.714801+0000","last_fullsized":"2024-04-25T10:09:22.714801+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:15.166230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.728083+0000","last_change":"2024-04-25T10:09:22.726050+0000","last_active":"2024-04-25T10:09:22.728083+0000","last_peered":"2024-04-25T10:09:22.728083+0000","last_clean":"2024-04-25T10:09:22.728083+0000","last_became_active":"2024-04-25T10:09:22.725480+0000","last_became_peered":"2024-04-25T10:09:22.725480+0000","last_unstale":"2024-04-25T10:09:22.728083+0000","last_undegraded":"2024-04-25T10:09:22.728083+0000","last_fullsized":"2024-04-25T10:09:22.728083+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:41.530052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.725342+0000","last_change":"2024-04-25T10:09:21.710120+0000","last_active":"2024-04-25T10:09:22.725342+0000","last_peered":"2024-04-25T10:09:22.725342+0000","last_clean":"2024-04-25T10:09:22.725342+0000","last_became_active":"2024-04-25T10:09:21.709858+0000","last_became_peered":"2024-04-25T10:09:21.709858+0000","last_unstale":"2024-04-25T10:09:22.725342+0000","last_undegraded":"2024-04-25T10:09:22.725342+0000","last_fullsized":"2024-04-25T10:09:22.725342+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.760023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735167+0000","last_change":"2024-04-25T10:09:22.727860+0000","last_active":"2024-04-25T10:09:22.735167+0000","last_peered":"2024-04-25T10:09:22.735167+0000","last_clean":"2024-04-25T10:09:22.735167+0000","last_became_active":"2024-04-25T10:09:22.727568+0000","last_became_peered":"2024-04-25T10:09:22.727568+0000","last_unstale":"2024-04-25T10:09:22.735167+0000","last_undegraded":"2024-04-25T10:09:22.735167+0000","last_fullsized":"2024-04-25T10:09:22.735167+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:45.867130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.725415+0000","last_change":"2024-04-25T10:09:21.714194+0000","last_active":"2024-04-25T10:09:22.725415+0000","last_peered":"2024-04-25T10:09:22.725415+0000","last_clean":"2024-04-25T10:09:22.725415+0000","last_became_active":"2024-04-25T10:09:21.713930+0000","last_became_peered":"2024-04-25T10:09:21.713930+0000","last_unstale":"2024-04-25T10:09:22.725415+0000","last_undegraded":"2024-04-25T10:09:22.725415+0000","last_fullsized":"2024-04-25T10:09:22.725415+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:40.526086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:21.694875+0000","last_change":"2024-04-25T10:09:21.694875+0000","last_active":"2024-04-25T10:09:21.694875+0000","last_peered":"2024-04-25T10:09:21.694875+0000","last_clean":"2024-04-25T10:09:21.694875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:21.694875+0000","last_undegraded":"2024-04-25T10:09:21.694875+0000","last_fullsized":"2024-04-25T10:09:21.694875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.724853+0000","last_change":"2024-04-25T10:09:21.719021+0000","last_active":"2024-04-25T10:09:22.724853+0000","last_peered":"2024-04-25T10:09:22.724853+0000","last_clean":"2024-04-25T10:09:22.724853+0000","last_became_active":"2024-04-25T10:09:21.718771+0000","last_became_peered":"2024-04-25T10:09:21.718771+0000","last_unstale":"2024-04-25T10:09:22.724853+0000","last_undegraded":"2024-04-25T10:09:22.724853+0000","last_fullsized":"2024-04-25T10:09:22.724853+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:30.994528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735627+0000","last_change":"2024-04-25T10:09:22.727538+0000","last_active":"2024-04-25T10:09:22.735627+0000","last_peered":"2024-04-25T10:09:22.735627+0000","last_clean":"2024-04-25T10:09:22.735627+0000","last_became_active":"2024-04-25T10:09:22.727237+0000","last_became_peered":"2024-04-25T10:09:22.727237+0000","last_unstale":"2024-04-25T10:09:22.735627+0000","last_undegraded":"2024-04-25T10:09:22.735627+0000","last_fullsized":"2024-04-25T10:09:22.735627+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:12.943229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:20.690908+0000","last_change":"2024-04-25T10:09:20.690908+0000","last_active":"2024-04-25T10:09:20.690908+0000","last_peered":"2024-04-25T10:09:20.690908+0000","last_clean":"2024-04-25T10:09:20.690908+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:20.690908+0000","last_undegraded":"2024-04-25T10:09:20.690908+0000","last_fullsized":"2024-04-25T10:09:20.690908+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735963+0000","last_change":"2024-04-25T10:09:22.732403+0000","last_active":"2024-04-25T10:09:22.735963+0000","last_peered":"2024-04-25T10:09:22.735963+0000","last_clean":"2024-04-25T10:09:22.735963+0000","last_became_active":"2024-04-25T10:09:22.732058+0000","last_became_peered":"2024-04-25T10:09:22.732058+0000","last_unstale":"2024-04-25T10:09:22.735963+0000","last_undegraded":"2024-04-25T10:09:22.735963+0000","last_fullsized":"2024-04-25T10:09:22.735963+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:51.105222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.714988+0000","last_change":"2024-04-25T10:09:21.708836+0000","last_active":"2024-04-25T10:09:22.714988+0000","last_peered":"2024-04-25T10:09:22.714988+0000","last_clean":"2024-04-25T10:09:22.714988+0000","last_became_active":"2024-04-25T10:09:21.708407+0000","last_became_peered":"2024-04-25T10:09:21.708407+0000","last_unstale":"2024-04-25T10:09:22.714988+0000","last_undegraded":"2024-04-25T10:09:22.714988+0000","last_fullsized":"2024-04-25T10:09:22.714988+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:50.101256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.734632+0000","last_change":"2024-04-25T10:09:22.732235+0000","last_active":"2024-04-25T10:09:22.734632+0000","last_peered":"2024-04-25T10:09:22.734632+0000","last_clean":"2024-04-25T10:09:22.734632+0000","last_became_active":"2024-04-25T10:09:22.731603+0000","last_became_peered":"2024-04-25T10:09:22.731603+0000","last_unstale":"2024-04-25T10:09:22.734632+0000","last_undegraded":"2024-04-25T10:09:22.734632+0000","last_fullsized":"2024-04-25T10:09:22.734632+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:26.292053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.709727+0000","last_change":"2024-04-25T10:09:21.716256+0000","last_active":"2024-04-25T10:09:22.709727+0000","last_peered":"2024-04-25T10:09:22.709727+0000","last_clean":"2024-04-25T10:09:22.709727+0000","last_became_active":"2024-04-25T10:09:21.715933+0000","last_became_peered":"2024-04-25T10:09:21.715933+0000","last_unstale":"2024-04-25T10:09:22.709727+0000","last_undegraded":"2024-04-25T10:09:22.709727+0000","last_fullsized":"2024-04-25T10:09:22.709727+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.907558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:21.694875+0000","last_change":"2024-04-25T10:09:21.694875+0000","last_active":"2024-04-25T10:09:21.694875+0000","last_peered":"2024-04-25T10:09:21.694875+0000","last_clean":"2024-04-25T10:09:21.694875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:21.694875+0000","last_undegraded":"2024-04-25T10:09:21.694875+0000","last_fullsized":"2024-04-25T10:09:21.694875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.739567+0000","last_change":"2024-04-25T10:09:21.712598+0000","last_active":"2024-04-25T10:09:22.739567+0000","last_peered":"2024-04-25T10:09:22.739567+0000","last_clean":"2024-04-25T10:09:22.739567+0000","last_became_active":"2024-04-25T10:09:21.712297+0000","last_became_peered":"2024-04-25T10:09:21.712297+0000","last_unstale":"2024-04-25T10:09:22.739567+0000","last_undegraded":"2024-04-25T10:09:22.739567+0000","last_fullsized":"2024-04-25T10:09:22.739567+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:34.008504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":270,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.720040+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:09:22.720040+0000","last_peered":"2024-04-25T10:09:22.720040+0000","last_clean":"2024-04-25T10:09:22.720040+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:09:22.720040+0000","last_undegraded":"2024-04-25T10:09:22.720040+0000","last_fullsized":"2024-04-25T10:09:22.720040+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":108,"acting":108,"num_store_stats":7},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":58,"acting":58,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509496,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":843776,"data_stored":351049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509496,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":843776,"data_stored":351049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509497,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":843776,"data_stored":351049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509497,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":843776,"data_stored":351049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509498,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":843776,"data_stored":351049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509496,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1437696,"data_stored":941417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509498,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":843776,"data_stored":351049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1413120,"data_stored":929333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:09:25.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:09:25.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.858+0000 7f9215f6a640 1 Processor -- start 2024-04-25T10:09:25.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.858+0000 7f9215f6a640 1 -- start start 2024-04-25T10:09:25.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.859+0000 7f9215f6a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f921015dce0 0x7f92100d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.859+0000 7f9215f6a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f921015e0c0 con 0x7f92100d35b0 2024-04-25T10:09:25.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.859+0000 7f9215f6a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92100d2510 con 0x7f92100b5b60 2024-04-25T10:09:25.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.859+0000 7f9215f6a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92100d27f0 con 0x7f921015dce0 2024-04-25T10:09:25.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.859+0000 7f920effd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f92100d35b0 0x7f92100b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50442/0 (socket says 172.21.15.16:50442) 2024-04-25T10:09:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.859+0000 7f920effd640 1 -- 172.21.15.16:0/2108756416 learned_addr learned my addr 172.21.15.16:0/2108756416 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920e7fc640 1 -- 172.21.15.16:0/2108756416 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 757970385 0 0) 0x7f921015e0c0 con 0x7f92100d35b0 2024-04-25T10:09:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920e7fc640 1 -- 172.21.15.16:0/2108756416 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91f8003520 con 0x7f92100d35b0 2024-04-25T10:09:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920f7fe640 1 --2- 172.21.15.16:0/2108756416 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f921015dce0 0x7f92100d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920e7fc640 1 -- 172.21.15.16:0/2108756416 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1712279464 0 0) 0x7f91f8003520 con 0x7f92100d35b0 2024-04-25T10:09:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920e7fc640 1 -- 172.21.15.16:0/2108756416 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f921015dce0 msgr2=0x7f92100d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920e7fc640 1 --2- 172.21.15.16:0/2108756416 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f921015dce0 0x7f92100d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920e7fc640 1 -- 172.21.15.16:0/2108756416 >> v1:172.21.15.186:6789/0 conn(0x7f92100b5b60 legacy=0x7f92100b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920e7fc640 1 -- 172.21.15.16:0/2108756416 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92100d2ad0 con 0x7f92100d35b0 2024-04-25T10:09:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920e7fc640 1 -- 172.21.15.16:0/2108756416 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f9204001670 con 0x7f92100d35b0 2024-04-25T10:09:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920f7fe640 1 --2- 172.21.15.16:0/2108756416 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f921015dce0 0x7f92100d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920e7fc640 1 -- 172.21.15.16:0/2108756416 >> v1:172.21.15.16:6789/0 conn(0x7f92100d35b0 legacy=0x7f92100b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920e7fc640 1 --2- 172.21.15.16:0/2108756416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91f80039e0 0x7f91f8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920e7fc640 1 --2- 172.21.15.16:0/2108756416 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f921015dce0 0x7f91f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920e7fc640 1 --2- 172.21.15.16:0/2108756416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f8004d40 0x7f91f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920e7fc640 1 -- 172.21.15.16:0/2108756416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92100d27f0 con 0x7f91f8004d40 2024-04-25T10:09:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920e7fc640 1 -- 172.21.15.16:0/2108756416 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f800bb50 con 0x7f91f80039e0 2024-04-25T10:09:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920e7fc640 1 -- 172.21.15.16:0/2108756416 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f800bd20 con 0x7f921015dce0 2024-04-25T10:09:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920ffff640 1 --2- 172.21.15.16:0/2108756416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f8004d40 0x7f91f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920f7fe640 1 --2- 172.21.15.16:0/2108756416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91f80039e0 0x7f91f8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920effd640 1 --2- 172.21.15.16:0/2108756416 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f921015dce0 0x7f91f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920ffff640 1 -- 172.21.15.16:0/2108756416 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f921015dce0 msgr2=0x7f91f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920ffff640 1 --2- 172.21.15.16:0/2108756416 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f921015dce0 0x7f91f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920ffff640 1 -- 172.21.15.16:0/2108756416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91f80039e0 msgr2=0x7f91f8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920ffff640 1 --2- 172.21.15.16:0/2108756416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91f80039e0 0x7f91f8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.860+0000 7f920ffff640 1 -- 172.21.15.16:0/2108756416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f8003520 con 0x7f91f8004d40 2024-04-25T10:09:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.861+0000 7f920ffff640 1 --2- 172.21.15.16:0/2108756416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f8004d40 0x7f91f80051c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f9208001e50 tx=0x7f91f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177afee0b9fb29c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.861+0000 7f920e7fc640 1 -- 172.21.15.16:0/2108756416 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208001a70 con 0x7f91f8004d40 2024-04-25T10:09:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.861+0000 7f920e7fc640 1 -- 172.21.15.16:0/2108756416 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920802ae30 con 0x7f91f8004d40 2024-04-25T10:09:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.861+0000 7f920e7fc640 1 -- 172.21.15.16:0/2108756416 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208003b90 con 0x7f91f8004d40 2024-04-25T10:09:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.862+0000 7f9215f6a640 1 -- 172.21.15.16:0/2108756416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f8004d40 msgr2=0x7f91f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.862+0000 7f9215f6a640 1 --2- 172.21.15.16:0/2108756416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f8004d40 0x7f91f80051c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f9208001e50 tx=0x7f91f8005830 comp rx=0 tx=0).stop 2024-04-25T10:09:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.862+0000 7f920ffff640 1 -- 172.21.15.16:0/2108756416 reap_dead start 2024-04-25T10:09:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.862+0000 7f9215f6a640 1 -- 172.21.15.16:0/2108756416 shutdown_connections 2024-04-25T10:09:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.862+0000 7f9215f6a640 1 -- 172.21.15.16:0/2108756416 >> 172.21.15.16:0/2108756416 conn(0x7f9210072900 msgr2=0x7f92100d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.862+0000 7f9215f6a640 1 -- 172.21.15.16:0/2108756416 shutdown_connections 2024-04-25T10:09:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.862+0000 7f9215f6a640 1 -- 172.21.15.16:0/2108756416 wait complete. 2024-04-25T10:09:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.863+0000 7f9215f6a640 1 Processor -- start 2024-04-25T10:09:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.863+0000 7f9215f6a640 1 -- start start 2024-04-25T10:09:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.864+0000 7f9215f6a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91f8004d40 0x7f9210151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.864+0000 7f9215f6a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92100c7500 0x7f92100c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.864+0000 7f9215f6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92100c7ec0 0x7f92100caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.864+0000 7f9215f6a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92100b9820 con 0x7f92100c7ec0 2024-04-25T10:09:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.864+0000 7f9215f6a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92100d3090 con 0x7f91f8004d40 2024-04-25T10:09:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.864+0000 7f9215f6a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92100b99a0 con 0x7f92100c7500 2024-04-25T10:09:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.864+0000 7f920ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92100c7ec0 0x7f92100caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.864+0000 7f920ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92100c7ec0 0x7f92100caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52450/0 (socket says 172.21.15.16:52450) 2024-04-25T10:09:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.864+0000 7f920ffff640 1 -- 172.21.15.16:0/1832418316 learned_addr learned my addr 172.21.15.16:0/1832418316 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.864+0000 7f920effd640 1 --2- 172.21.15.16:0/1832418316 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92100c7500 0x7f92100c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.864+0000 7f920f7fe640 1 --2- 172.21.15.16:0/1832418316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91f8004d40 0x7f9210151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.864+0000 7f920ffff640 1 -- 172.21.15.16:0/1832418316 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92100c7500 msgr2=0x7f92100c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.864+0000 7f920ffff640 1 --2- 172.21.15.16:0/1832418316 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92100c7500 0x7f92100c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.864+0000 7f920ffff640 1 -- 172.21.15.16:0/1832418316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91f8004d40 msgr2=0x7f9210151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.864+0000 7f920ffff640 1 --2- 172.21.15.16:0/1832418316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91f8004d40 0x7f9210151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.864+0000 7f920ffff640 1 -- 172.21.15.16:0/1832418316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92100c8730 con 0x7f92100c7ec0 2024-04-25T10:09:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.864+0000 7f920f7fe640 1 --2- 172.21.15.16:0/1832418316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91f8004d40 0x7f9210151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.864+0000 7f920effd640 1 --2- 172.21.15.16:0/1832418316 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92100c7500 0x7f92100c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.865+0000 7f920ffff640 1 --2- 172.21.15.16:0/1832418316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92100c7ec0 0x7f92100caaa0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9208001f70 tx=0x7f92080014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.865+0000 7f91f7fff640 1 -- 172.21.15.16:0/1832418316 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208001a70 con 0x7f92100c7ec0 2024-04-25T10:09:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.865+0000 7f91f7fff640 1 -- 172.21.15.16:0/1832418316 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9208041070 con 0x7f92100c7ec0 2024-04-25T10:09:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.865+0000 7f91f7fff640 1 -- 172.21.15.16:0/1832418316 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208007db0 con 0x7f92100c7ec0 2024-04-25T10:09:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.865+0000 7f9215f6a640 1 -- 172.21.15.16:0/1832418316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921015fe90 con 0x7f92100c7ec0 2024-04-25T10:09:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.865+0000 7f9215f6a640 1 -- 172.21.15.16:0/1832418316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9210160450 con 0x7f92100c7ec0 2024-04-25T10:09:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.866+0000 7f9215f6a640 1 -- 172.21.15.16:0/1832418316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92100749b0 con 0x7f92100c7ec0 2024-04-25T10:09:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.866+0000 7f91f7fff640 1 -- 172.21.15.16:0/1832418316 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f920803a070 con 0x7f92100c7ec0 2024-04-25T10:09:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.867+0000 7f91f7fff640 1 --2- 172.21.15.16:0/1832418316 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f91fc06b500 0x7f91fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.867+0000 7f91f7fff640 1 -- 172.21.15.16:0/1832418316 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92080a9bc0 con 0x7f92100c7ec0 2024-04-25T10:09:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.867+0000 7f920f7fe640 1 --2- 172.21.15.16:0/1832418316 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f91fc06b500 0x7f91fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.868+0000 7f920f7fe640 1 --2- 172.21.15.16:0/1832418316 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f91fc06b500 0x7f91fc06d9c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9200006fd0 tx=0x7f9200008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:25.871+0000 7f91f7fff640 1 -- 172.21.15.16:0/1832418316 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9208078b60 con 0x7f92100c7ec0 2024-04-25T10:09:26.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.047+0000 7f9215f6a640 1 -- 172.21.15.16:0/1832418316 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f92100b4090 con 0x7f91fc06b500 2024-04-25T10:09:26.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.053+0000 7f91f7fff640 1 -- 172.21.15.16:0/1832418316 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252012 (secure 0 0 0) 0x7f92100b4090 con 0x7f91fc06b500 2024-04-25T10:09:26.062 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:26.064 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:09:26.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.056+0000 7f9215f6a640 1 -- 172.21.15.16:0/1832418316 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f91fc06b500 msgr2=0x7f91fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.056+0000 7f9215f6a640 1 --2- 172.21.15.16:0/1832418316 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f91fc06b500 0x7f91fc06d9c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9200006fd0 tx=0x7f9200008040 comp rx=0 tx=0).stop 2024-04-25T10:09:26.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.056+0000 7f9215f6a640 1 -- 172.21.15.16:0/1832418316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92100c7ec0 msgr2=0x7f92100caaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.056+0000 7f9215f6a640 1 --2- 172.21.15.16:0/1832418316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92100c7ec0 0x7f92100caaa0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9208001f70 tx=0x7f92080014a0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.056+0000 7f9215f6a640 1 -- 172.21.15.16:0/1832418316 shutdown_connections 2024-04-25T10:09:26.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.056+0000 7f9215f6a640 1 --2- 172.21.15.16:0/1832418316 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f91fc06b500 0x7f91fc06d9c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.056+0000 7f9215f6a640 1 --2- 172.21.15.16:0/1832418316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92100c7ec0 0x7f92100caaa0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.056+0000 7f9215f6a640 1 --2- 172.21.15.16:0/1832418316 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92100c7500 0x7f92100c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.056+0000 7f9215f6a640 1 --2- 172.21.15.16:0/1832418316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91f8004d40 0x7f9210151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.056+0000 7f9215f6a640 1 -- 172.21.15.16:0/1832418316 >> 172.21.15.16:0/1832418316 conn(0x7f9210072900 msgr2=0x7f92100d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:26.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.057+0000 7f9215f6a640 1 -- 172.21.15.16:0/1832418316 shutdown_connections 2024-04-25T10:09:26.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.057+0000 7f9215f6a640 1 -- 172.21.15.16:0/1832418316 wait complete. 2024-04-25T10:09:26.073 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-04-25T10:09:25.840975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244764,"kb_used_data":7768,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729956,"statfs":{"total":773094113280,"available":772843474944,"internally_reserved":0,"allocated":7954432,"data_stored":3995993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.145269"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.742094+0000","last_change":"2024-04-25T10:09:22.740700+0000","last_active":"2024-04-25T10:09:22.742094+0000","last_peered":"2024-04-25T10:09:22.742094+0000","last_clean":"2024-04-25T10:09:22.742094+0000","last_became_active":"2024-04-25T10:09:22.740086+0000","last_became_peered":"2024-04-25T10:09:22.740086+0000","last_unstale":"2024-04-25T10:09:22.742094+0000","last_undegraded":"2024-04-25T10:09:22.742094+0000","last_fullsized":"2024-04-25T10:09:22.742094+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:08.877974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.730366+0000","last_change":"2024-04-25T10:09:22.727919+0000","last_active":"2024-04-25T10:09:22.730366+0000","last_peered":"2024-04-25T10:09:22.730366+0000","last_clean":"2024-04-25T10:09:22.730366+0000","last_became_active":"2024-04-25T10:09:22.727518+0000","last_became_peered":"2024-04-25T10:09:22.727518+0000","last_unstale":"2024-04-25T10:09:22.730366+0000","last_undegraded":"2024-04-25T10:09:22.730366+0000","last_fullsized":"2024-04-25T10:09:22.730366+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:52.862877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741321+0000","last_change":"2024-04-25T10:09:22.737653+0000","last_active":"2024-04-25T10:09:22.741321+0000","last_peered":"2024-04-25T10:09:22.741321+0000","last_clean":"2024-04-25T10:09:22.741321+0000","last_became_active":"2024-04-25T10:09:22.737017+0000","last_became_peered":"2024-04-25T10:09:22.737017+0000","last_unstale":"2024-04-25T10:09:22.741321+0000","last_undegraded":"2024-04-25T10:09:22.741321+0000","last_fullsized":"2024-04-25T10:09:22.741321+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:08.346248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741243+0000","last_change":"2024-04-25T10:09:22.735283+0000","last_active":"2024-04-25T10:09:22.741243+0000","last_peered":"2024-04-25T10:09:22.741243+0000","last_clean":"2024-04-25T10:09:22.741243+0000","last_became_active":"2024-04-25T10:09:22.734973+0000","last_became_peered":"2024-04-25T10:09:22.734973+0000","last_unstale":"2024-04-25T10:09:22.741243+0000","last_undegraded":"2024-04-25T10:09:22.741243+0000","last_fullsized":"2024-04-25T10:09:22.741243+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:50.431542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:09:24.390739+0000","last_change":"2024-04-25T10:09:22.739091+0000","last_active":"2024-04-25T10:09:24.390739+0000","last_peered":"2024-04-25T10:09:24.390739+0000","last_clean":"2024-04-25T10:09:24.390739+0000","last_became_active":"2024-04-25T10:09:22.738752+0000","last_became_peered":"2024-04-25T10:09:22.738752+0000","last_unstale":"2024-04-25T10:09:24.390739+0000","last_undegraded":"2024-04-25T10:09:24.390739+0000","last_fullsized":"2024-04-25T10:09:24.390739+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:51.022615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.729024+0000","last_change":"2024-04-25T10:09:22.724555+0000","last_active":"2024-04-25T10:09:22.729024+0000","last_peered":"2024-04-25T10:09:22.729024+0000","last_clean":"2024-04-25T10:09:22.729024+0000","last_became_active":"2024-04-25T10:09:22.724224+0000","last_became_peered":"2024-04-25T10:09:22.724224+0000","last_unstale":"2024-04-25T10:09:22.729024+0000","last_undegraded":"2024-04-25T10:09:22.729024+0000","last_fullsized":"2024-04-25T10:09:22.729024+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:46.677718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.740277+0000","last_change":"2024-04-25T10:09:22.735978+0000","last_active":"2024-04-25T10:09:22.740277+0000","last_peered":"2024-04-25T10:09:22.740277+0000","last_clean":"2024-04-25T10:09:22.740277+0000","last_became_active":"2024-04-25T10:09:22.735712+0000","last_became_peered":"2024-04-25T10:09:22.735712+0000","last_unstale":"2024-04-25T10:09:22.740277+0000","last_undegraded":"2024-04-25T10:09:22.740277+0000","last_fullsized":"2024-04-25T10:09:22.740277+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:41.872942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735429+0000","last_change":"2024-04-25T10:09:22.722595+0000","last_active":"2024-04-25T10:09:22.735429+0000","last_peered":"2024-04-25T10:09:22.735429+0000","last_clean":"2024-04-25T10:09:22.735429+0000","last_became_active":"2024-04-25T10:09:22.721683+0000","last_became_peered":"2024-04-25T10:09:22.721683+0000","last_unstale":"2024-04-25T10:09:22.735429+0000","last_undegraded":"2024-04-25T10:09:22.735429+0000","last_fullsized":"2024-04-25T10:09:22.735429+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:00.988197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741848+0000","last_change":"2024-04-25T10:09:22.740092+0000","last_active":"2024-04-25T10:09:22.741848+0000","last_peered":"2024-04-25T10:09:22.741848+0000","last_clean":"2024-04-25T10:09:22.741848+0000","last_became_active":"2024-04-25T10:09:22.739795+0000","last_became_peered":"2024-04-25T10:09:22.739795+0000","last_unstale":"2024-04-25T10:09:22.741848+0000","last_undegraded":"2024-04-25T10:09:22.741848+0000","last_fullsized":"2024-04-25T10:09:22.741848+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:45.867130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741920+0000","last_change":"2024-04-25T10:09:22.740023+0000","last_active":"2024-04-25T10:09:22.741920+0000","last_peered":"2024-04-25T10:09:22.741920+0000","last_clean":"2024-04-25T10:09:22.741920+0000","last_became_active":"2024-04-25T10:09:22.739751+0000","last_became_peered":"2024-04-25T10:09:22.739751+0000","last_unstale":"2024-04-25T10:09:22.741920+0000","last_undegraded":"2024-04-25T10:09:22.741920+0000","last_fullsized":"2024-04-25T10:09:22.741920+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:12.347303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.736162+0000","last_change":"2024-04-25T10:09:22.731846+0000","last_active":"2024-04-25T10:09:22.736162+0000","last_peered":"2024-04-25T10:09:22.736162+0000","last_clean":"2024-04-25T10:09:22.736162+0000","last_became_active":"2024-04-25T10:09:22.731589+0000","last_became_peered":"2024-04-25T10:09:22.731589+0000","last_unstale":"2024-04-25T10:09:22.736162+0000","last_undegraded":"2024-04-25T10:09:22.736162+0000","last_fullsized":"2024-04-25T10:09:22.736162+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:59.127307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.734270+0000","last_change":"2024-04-25T10:09:22.726514+0000","last_active":"2024-04-25T10:09:22.734270+0000","last_peered":"2024-04-25T10:09:22.734270+0000","last_clean":"2024-04-25T10:09:22.734270+0000","last_became_active":"2024-04-25T10:09:22.725590+0000","last_became_peered":"2024-04-25T10:09:22.725590+0000","last_unstale":"2024-04-25T10:09:22.734270+0000","last_undegraded":"2024-04-25T10:09:22.734270+0000","last_fullsized":"2024-04-25T10:09:22.734270+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:44.497003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741488+0000","last_change":"2024-04-25T10:09:22.737785+0000","last_active":"2024-04-25T10:09:22.741488+0000","last_peered":"2024-04-25T10:09:22.741488+0000","last_clean":"2024-04-25T10:09:22.741488+0000","last_became_active":"2024-04-25T10:09:22.737513+0000","last_became_peered":"2024-04-25T10:09:22.737513+0000","last_unstale":"2024-04-25T10:09:22.741488+0000","last_undegraded":"2024-04-25T10:09:22.741488+0000","last_fullsized":"2024-04-25T10:09:22.741488+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:27.547340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.732846+0000","last_change":"2024-04-25T10:09:22.726657+0000","last_active":"2024-04-25T10:09:22.732846+0000","last_peered":"2024-04-25T10:09:22.732846+0000","last_clean":"2024-04-25T10:09:22.732846+0000","last_became_active":"2024-04-25T10:09:22.725689+0000","last_became_peered":"2024-04-25T10:09:22.725689+0000","last_unstale":"2024-04-25T10:09:22.732846+0000","last_undegraded":"2024-04-25T10:09:22.732846+0000","last_fullsized":"2024-04-25T10:09:22.732846+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:46.677718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.733132+0000","last_change":"2024-04-25T10:09:22.726406+0000","last_active":"2024-04-25T10:09:22.733132+0000","last_peered":"2024-04-25T10:09:22.733132+0000","last_clean":"2024-04-25T10:09:22.733132+0000","last_became_active":"2024-04-25T10:09:22.726122+0000","last_became_peered":"2024-04-25T10:09:22.726122+0000","last_unstale":"2024-04-25T10:09:22.733132+0000","last_undegraded":"2024-04-25T10:09:22.733132+0000","last_fullsized":"2024-04-25T10:09:22.733132+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:01.928851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735879+0000","last_change":"2024-04-25T10:09:22.727997+0000","last_active":"2024-04-25T10:09:22.735879+0000","last_peered":"2024-04-25T10:09:22.735879+0000","last_clean":"2024-04-25T10:09:22.735879+0000","last_became_active":"2024-04-25T10:09:22.727704+0000","last_became_peered":"2024-04-25T10:09:22.727704+0000","last_unstale":"2024-04-25T10:09:22.735879+0000","last_undegraded":"2024-04-25T10:09:22.735879+0000","last_fullsized":"2024-04-25T10:09:22.735879+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:26.658710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:09:24.391084+0000","last_change":"2024-04-25T10:09:22.734460+0000","last_active":"2024-04-25T10:09:24.391084+0000","last_peered":"2024-04-25T10:09:24.391084+0000","last_clean":"2024-04-25T10:09:24.391084+0000","last_became_active":"2024-04-25T10:09:22.734213+0000","last_became_peered":"2024-04-25T10:09:22.734213+0000","last_unstale":"2024-04-25T10:09:24.391084+0000","last_undegraded":"2024-04-25T10:09:24.391084+0000","last_fullsized":"2024-04-25T10:09:24.391084+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:50.727574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:09:24.391043+0000","last_change":"2024-04-25T10:09:22.736913+0000","last_active":"2024-04-25T10:09:24.391043+0000","last_peered":"2024-04-25T10:09:24.391043+0000","last_clean":"2024-04-25T10:09:24.391043+0000","last_became_active":"2024-04-25T10:09:22.736631+0000","last_became_peered":"2024-04-25T10:09:22.736631+0000","last_unstale":"2024-04-25T10:09:24.391043+0000","last_undegraded":"2024-04-25T10:09:24.391043+0000","last_fullsized":"2024-04-25T10:09:24.391043+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:10.446850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735919+0000","last_change":"2024-04-25T10:09:22.721738+0000","last_active":"2024-04-25T10:09:22.735919+0000","last_peered":"2024-04-25T10:09:22.735919+0000","last_clean":"2024-04-25T10:09:22.735919+0000","last_became_active":"2024-04-25T10:09:22.721357+0000","last_became_peered":"2024-04-25T10:09:22.721357+0000","last_unstale":"2024-04-25T10:09:22.735919+0000","last_undegraded":"2024-04-25T10:09:22.735919+0000","last_fullsized":"2024-04-25T10:09:22.735919+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:41.775227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:09:24.390779+0000","last_change":"2024-04-25T10:09:22.733984+0000","last_active":"2024-04-25T10:09:24.390779+0000","last_peered":"2024-04-25T10:09:24.390779+0000","last_clean":"2024-04-25T10:09:24.390779+0000","last_became_active":"2024-04-25T10:09:22.733679+0000","last_became_peered":"2024-04-25T10:09:22.733679+0000","last_unstale":"2024-04-25T10:09:24.390779+0000","last_undegraded":"2024-04-25T10:09:24.390779+0000","last_fullsized":"2024-04-25T10:09:24.390779+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:13.573895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.730567+0000","last_change":"2024-04-25T10:09:22.729031+0000","last_active":"2024-04-25T10:09:22.730567+0000","last_peered":"2024-04-25T10:09:22.730567+0000","last_clean":"2024-04-25T10:09:22.730567+0000","last_became_active":"2024-04-25T10:09:22.728748+0000","last_became_peered":"2024-04-25T10:09:22.728748+0000","last_unstale":"2024-04-25T10:09:22.730567+0000","last_undegraded":"2024-04-25T10:09:22.730567+0000","last_fullsized":"2024-04-25T10:09:22.730567+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:04.675050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741959+0000","last_change":"2024-04-25T10:09:22.740317+0000","last_active":"2024-04-25T10:09:22.741959+0000","last_peered":"2024-04-25T10:09:22.741959+0000","last_clean":"2024-04-25T10:09:22.741959+0000","last_became_active":"2024-04-25T10:09:22.740008+0000","last_became_peered":"2024-04-25T10:09:22.740008+0000","last_unstale":"2024-04-25T10:09:22.741959+0000","last_undegraded":"2024-04-25T10:09:22.741959+0000","last_fullsized":"2024-04-25T10:09:22.741959+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:04.675050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735407+0000","last_change":"2024-04-25T10:09:22.729817+0000","last_active":"2024-04-25T10:09:22.735407+0000","last_peered":"2024-04-25T10:09:22.735407+0000","last_clean":"2024-04-25T10:09:22.735407+0000","last_became_active":"2024-04-25T10:09:22.729498+0000","last_became_peered":"2024-04-25T10:09:22.729498+0000","last_unstale":"2024-04-25T10:09:22.735407+0000","last_undegraded":"2024-04-25T10:09:22.735407+0000","last_fullsized":"2024-04-25T10:09:22.735407+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:24.427888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741092+0000","last_change":"2024-04-25T10:09:22.736028+0000","last_active":"2024-04-25T10:09:22.741092+0000","last_peered":"2024-04-25T10:09:22.741092+0000","last_clean":"2024-04-25T10:09:22.741092+0000","last_became_active":"2024-04-25T10:09:22.735640+0000","last_became_peered":"2024-04-25T10:09:22.735640+0000","last_unstale":"2024-04-25T10:09:22.741092+0000","last_undegraded":"2024-04-25T10:09:22.741092+0000","last_fullsized":"2024-04-25T10:09:22.741092+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:44.624012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735373+0000","last_change":"2024-04-25T10:09:22.731701+0000","last_active":"2024-04-25T10:09:22.735373+0000","last_peered":"2024-04-25T10:09:22.735373+0000","last_clean":"2024-04-25T10:09:22.735373+0000","last_became_active":"2024-04-25T10:09:22.731421+0000","last_became_peered":"2024-04-25T10:09:22.731421+0000","last_unstale":"2024-04-25T10:09:22.735373+0000","last_undegraded":"2024-04-25T10:09:22.735373+0000","last_fullsized":"2024-04-25T10:09:22.735373+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:26.842520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.738443+0000","last_change":"2024-04-25T10:09:22.734761+0000","last_active":"2024-04-25T10:09:22.738443+0000","last_peered":"2024-04-25T10:09:22.738443+0000","last_clean":"2024-04-25T10:09:22.738443+0000","last_became_active":"2024-04-25T10:09:22.734460+0000","last_became_peered":"2024-04-25T10:09:22.734460+0000","last_unstale":"2024-04-25T10:09:22.738443+0000","last_undegraded":"2024-04-25T10:09:22.738443+0000","last_fullsized":"2024-04-25T10:09:22.738443+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:00.291315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:09:24.390891+0000","last_change":"2024-04-25T10:09:22.730291+0000","last_active":"2024-04-25T10:09:24.390891+0000","last_peered":"2024-04-25T10:09:24.390891+0000","last_clean":"2024-04-25T10:09:24.390891+0000","last_became_active":"2024-04-25T10:09:22.729801+0000","last_became_peered":"2024-04-25T10:09:22.729801+0000","last_unstale":"2024-04-25T10:09:24.390891+0000","last_undegraded":"2024-04-25T10:09:24.390891+0000","last_fullsized":"2024-04-25T10:09:24.390891+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:33.233295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.729520+0000","last_change":"2024-04-25T10:09:22.726315+0000","last_active":"2024-04-25T10:09:22.729520+0000","last_peered":"2024-04-25T10:09:22.729520+0000","last_clean":"2024-04-25T10:09:22.729520+0000","last_became_active":"2024-04-25T10:09:22.725776+0000","last_became_peered":"2024-04-25T10:09:22.725776+0000","last_unstale":"2024-04-25T10:09:22.729520+0000","last_undegraded":"2024-04-25T10:09:22.729520+0000","last_fullsized":"2024-04-25T10:09:22.729520+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:01.928851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741458+0000","last_change":"2024-04-25T10:09:22.736699+0000","last_active":"2024-04-25T10:09:22.741458+0000","last_peered":"2024-04-25T10:09:22.741458+0000","last_clean":"2024-04-25T10:09:22.741458+0000","last_became_active":"2024-04-25T10:09:22.736407+0000","last_became_peered":"2024-04-25T10:09:22.736407+0000","last_unstale":"2024-04-25T10:09:22.741458+0000","last_undegraded":"2024-04-25T10:09:22.741458+0000","last_fullsized":"2024-04-25T10:09:22.741458+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:35.012470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735398+0000","last_change":"2024-04-25T10:09:22.722355+0000","last_active":"2024-04-25T10:09:22.735398+0000","last_peered":"2024-04-25T10:09:22.735398+0000","last_clean":"2024-04-25T10:09:22.735398+0000","last_became_active":"2024-04-25T10:09:22.722064+0000","last_became_peered":"2024-04-25T10:09:22.722064+0000","last_unstale":"2024-04-25T10:09:22.735398+0000","last_undegraded":"2024-04-25T10:09:22.735398+0000","last_fullsized":"2024-04-25T10:09:22.735398+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:43.678534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.730338+0000","last_change":"2024-04-25T10:09:22.727553+0000","last_active":"2024-04-25T10:09:22.730338+0000","last_peered":"2024-04-25T10:09:22.730338+0000","last_clean":"2024-04-25T10:09:22.730338+0000","last_became_active":"2024-04-25T10:09:22.727223+0000","last_became_peered":"2024-04-25T10:09:22.727223+0000","last_unstale":"2024-04-25T10:09:22.730338+0000","last_undegraded":"2024-04-25T10:09:22.730338+0000","last_fullsized":"2024-04-25T10:09:22.730338+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:46.677718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.726296+0000","last_change":"2024-04-25T10:09:22.722740+0000","last_active":"2024-04-25T10:09:22.726296+0000","last_peered":"2024-04-25T10:09:22.726296+0000","last_clean":"2024-04-25T10:09:22.726296+0000","last_became_active":"2024-04-25T10:09:22.722477+0000","last_became_peered":"2024-04-25T10:09:22.722477+0000","last_unstale":"2024-04-25T10:09:22.726296+0000","last_undegraded":"2024-04-25T10:09:22.726296+0000","last_fullsized":"2024-04-25T10:09:22.726296+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:41.763989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.736685+0000","last_change":"2024-04-25T10:09:22.728881+0000","last_active":"2024-04-25T10:09:22.736685+0000","last_peered":"2024-04-25T10:09:22.736685+0000","last_clean":"2024-04-25T10:09:22.736685+0000","last_became_active":"2024-04-25T10:09:22.728568+0000","last_became_peered":"2024-04-25T10:09:22.728568+0000","last_unstale":"2024-04-25T10:09:22.736685+0000","last_undegraded":"2024-04-25T10:09:22.736685+0000","last_fullsized":"2024-04-25T10:09:22.736685+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:05.585375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"71'4","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.742536+0000","last_change":"2024-04-25T10:09:21.713002+0000","last_active":"2024-04-25T10:09:22.742536+0000","last_peered":"2024-04-25T10:09:22.742536+0000","last_clean":"2024-04-25T10:09:22.742536+0000","last_became_active":"2024-04-25T10:09:21.712704+0000","last_became_peered":"2024-04-25T10:09:21.712704+0000","last_unstale":"2024-04-25T10:09:22.742536+0000","last_undegraded":"2024-04-25T10:09:22.742536+0000","last_fullsized":"2024-04-25T10:09:22.742536+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:30.994528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741275+0000","last_change":"2024-04-25T10:09:22.733619+0000","last_active":"2024-04-25T10:09:22.741275+0000","last_peered":"2024-04-25T10:09:22.741275+0000","last_clean":"2024-04-25T10:09:22.741275+0000","last_became_active":"2024-04-25T10:09:22.733216+0000","last_became_peered":"2024-04-25T10:09:22.733216+0000","last_unstale":"2024-04-25T10:09:22.741275+0000","last_undegraded":"2024-04-25T10:09:22.741275+0000","last_fullsized":"2024-04-25T10:09:22.741275+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:39.734544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"71'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:09:24.391138+0000","last_change":"2024-04-25T10:09:21.712781+0000","last_active":"2024-04-25T10:09:24.391138+0000","last_peered":"2024-04-25T10:09:24.391138+0000","last_clean":"2024-04-25T10:09:24.391138+0000","last_became_active":"2024-04-25T10:09:21.712497+0000","last_became_peered":"2024-04-25T10:09:21.712497+0000","last_unstale":"2024-04-25T10:09:24.391138+0000","last_undegraded":"2024-04-25T10:09:24.391138+0000","last_fullsized":"2024-04-25T10:09:24.391138+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:05.226452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.728687+0000","last_change":"2024-04-25T10:09:22.724533+0000","last_active":"2024-04-25T10:09:22.728687+0000","last_peered":"2024-04-25T10:09:22.728687+0000","last_clean":"2024-04-25T10:09:22.728687+0000","last_became_active":"2024-04-25T10:09:22.724269+0000","last_became_peered":"2024-04-25T10:09:22.724269+0000","last_unstale":"2024-04-25T10:09:22.728687+0000","last_undegraded":"2024-04-25T10:09:22.728687+0000","last_fullsized":"2024-04-25T10:09:22.728687+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:44.497003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.736112+0000","last_change":"2024-04-25T10:09:21.721186+0000","last_active":"2024-04-25T10:09:22.736112+0000","last_peered":"2024-04-25T10:09:22.736112+0000","last_clean":"2024-04-25T10:09:22.736112+0000","last_became_active":"2024-04-25T10:09:21.720525+0000","last_became_peered":"2024-04-25T10:09:21.720525+0000","last_unstale":"2024-04-25T10:09:22.736112+0000","last_undegraded":"2024-04-25T10:09:22.736112+0000","last_fullsized":"2024-04-25T10:09:22.736112+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:00.537304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741112+0000","last_change":"2024-04-25T10:09:22.738338+0000","last_active":"2024-04-25T10:09:22.741112+0000","last_peered":"2024-04-25T10:09:22.741112+0000","last_clean":"2024-04-25T10:09:22.741112+0000","last_became_active":"2024-04-25T10:09:22.738084+0000","last_became_peered":"2024-04-25T10:09:22.738084+0000","last_unstale":"2024-04-25T10:09:22.741112+0000","last_undegraded":"2024-04-25T10:09:22.741112+0000","last_fullsized":"2024-04-25T10:09:22.741112+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:01.309230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.742333+0000","last_change":"2024-04-25T10:09:21.715338+0000","last_active":"2024-04-25T10:09:22.742333+0000","last_peered":"2024-04-25T10:09:22.742333+0000","last_clean":"2024-04-25T10:09:22.742333+0000","last_became_active":"2024-04-25T10:09:21.715072+0000","last_became_peered":"2024-04-25T10:09:21.715072+0000","last_unstale":"2024-04-25T10:09:22.742333+0000","last_undegraded":"2024-04-25T10:09:22.742333+0000","last_fullsized":"2024-04-25T10:09:22.742333+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:36.752649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741199+0000","last_change":"2024-04-25T10:09:22.736877+0000","last_active":"2024-04-25T10:09:22.741199+0000","last_peered":"2024-04-25T10:09:22.741199+0000","last_clean":"2024-04-25T10:09:22.741199+0000","last_became_active":"2024-04-25T10:09:22.736597+0000","last_became_peered":"2024-04-25T10:09:22.736597+0000","last_unstale":"2024-04-25T10:09:22.741199+0000","last_undegraded":"2024-04-25T10:09:22.741199+0000","last_fullsized":"2024-04-25T10:09:22.741199+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:57.265654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.734913+0000","last_change":"2024-04-25T10:09:21.719141+0000","last_active":"2024-04-25T10:09:22.734913+0000","last_peered":"2024-04-25T10:09:22.734913+0000","last_clean":"2024-04-25T10:09:22.734913+0000","last_became_active":"2024-04-25T10:09:21.718859+0000","last_became_peered":"2024-04-25T10:09:21.718859+0000","last_unstale":"2024-04-25T10:09:22.734913+0000","last_undegraded":"2024-04-25T10:09:22.734913+0000","last_fullsized":"2024-04-25T10:09:22.734913+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.907558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.725510+0000","last_change":"2024-04-25T10:09:22.720830+0000","last_active":"2024-04-25T10:09:22.725510+0000","last_peered":"2024-04-25T10:09:22.725510+0000","last_clean":"2024-04-25T10:09:22.725510+0000","last_became_active":"2024-04-25T10:09:22.720541+0000","last_became_peered":"2024-04-25T10:09:22.720541+0000","last_unstale":"2024-04-25T10:09:22.725510+0000","last_undegraded":"2024-04-25T10:09:22.725510+0000","last_fullsized":"2024-04-25T10:09:22.725510+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:01.928851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.732780+0000","last_change":"2024-04-25T10:09:21.718524+0000","last_active":"2024-04-25T10:09:22.732780+0000","last_peered":"2024-04-25T10:09:22.732780+0000","last_clean":"2024-04-25T10:09:22.732780+0000","last_became_active":"2024-04-25T10:09:21.718244+0000","last_became_peered":"2024-04-25T10:09:21.718244+0000","last_unstale":"2024-04-25T10:09:22.732780+0000","last_undegraded":"2024-04-25T10:09:22.732780+0000","last_fullsized":"2024-04-25T10:09:22.732780+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:07.342282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741569+0000","last_change":"2024-04-25T10:09:22.735544+0000","last_active":"2024-04-25T10:09:22.741569+0000","last_peered":"2024-04-25T10:09:22.741569+0000","last_clean":"2024-04-25T10:09:22.741569+0000","last_became_active":"2024-04-25T10:09:22.735219+0000","last_became_peered":"2024-04-25T10:09:22.735219+0000","last_unstale":"2024-04-25T10:09:22.741569+0000","last_undegraded":"2024-04-25T10:09:22.741569+0000","last_fullsized":"2024-04-25T10:09:22.741569+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:31.998494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.721005+0000","last_change":"2024-04-25T10:09:21.718431+0000","last_active":"2024-04-25T10:09:22.721005+0000","last_peered":"2024-04-25T10:09:22.721005+0000","last_clean":"2024-04-25T10:09:22.721005+0000","last_became_active":"2024-04-25T10:09:21.718200+0000","last_became_peered":"2024-04-25T10:09:21.718200+0000","last_unstale":"2024-04-25T10:09:22.721005+0000","last_undegraded":"2024-04-25T10:09:22.721005+0000","last_fullsized":"2024-04-25T10:09:22.721005+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:17.676932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741386+0000","last_change":"2024-04-25T10:09:22.733006+0000","last_active":"2024-04-25T10:09:22.741386+0000","last_peered":"2024-04-25T10:09:22.741386+0000","last_clean":"2024-04-25T10:09:22.741386+0000","last_became_active":"2024-04-25T10:09:22.732611+0000","last_became_peered":"2024-04-25T10:09:22.732611+0000","last_unstale":"2024-04-25T10:09:22.741386+0000","last_undegraded":"2024-04-25T10:09:22.741386+0000","last_fullsized":"2024-04-25T10:09:22.741386+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:04.675050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.718277+0000","last_change":"2024-04-25T10:09:21.712492+0000","last_active":"2024-04-25T10:09:22.718277+0000","last_peered":"2024-04-25T10:09:22.718277+0000","last_clean":"2024-04-25T10:09:22.718277+0000","last_became_active":"2024-04-25T10:09:21.712144+0000","last_became_peered":"2024-04-25T10:09:21.712144+0000","last_unstale":"2024-04-25T10:09:22.718277+0000","last_undegraded":"2024-04-25T10:09:22.718277+0000","last_fullsized":"2024-04-25T10:09:22.718277+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:50.101256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.737045+0000","last_change":"2024-04-25T10:09:22.732594+0000","last_active":"2024-04-25T10:09:22.737045+0000","last_peered":"2024-04-25T10:09:22.737045+0000","last_clean":"2024-04-25T10:09:22.737045+0000","last_became_active":"2024-04-25T10:09:22.732206+0000","last_became_peered":"2024-04-25T10:09:22.732206+0000","last_unstale":"2024-04-25T10:09:22.737045+0000","last_undegraded":"2024-04-25T10:09:22.737045+0000","last_fullsized":"2024-04-25T10:09:22.737045+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:12.628900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.714576+0000","last_change":"2024-04-25T10:09:21.716690+0000","last_active":"2024-04-25T10:09:22.714576+0000","last_peered":"2024-04-25T10:09:22.714576+0000","last_clean":"2024-04-25T10:09:22.714576+0000","last_became_active":"2024-04-25T10:09:21.716399+0000","last_became_peered":"2024-04-25T10:09:21.716399+0000","last_unstale":"2024-04-25T10:09:22.714576+0000","last_undegraded":"2024-04-25T10:09:22.714576+0000","last_fullsized":"2024-04-25T10:09:22.714576+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:44.475563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.739376+0000","last_change":"2024-04-25T10:09:22.735326+0000","last_active":"2024-04-25T10:09:22.739376+0000","last_peered":"2024-04-25T10:09:22.739376+0000","last_clean":"2024-04-25T10:09:22.739376+0000","last_became_active":"2024-04-25T10:09:22.734995+0000","last_became_peered":"2024-04-25T10:09:22.734995+0000","last_unstale":"2024-04-25T10:09:22.739376+0000","last_undegraded":"2024-04-25T10:09:22.739376+0000","last_fullsized":"2024-04-25T10:09:22.739376+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:07.878932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.715845+0000","last_change":"2024-04-25T10:09:21.715138+0000","last_active":"2024-04-25T10:09:22.715845+0000","last_peered":"2024-04-25T10:09:22.715845+0000","last_clean":"2024-04-25T10:09:22.715845+0000","last_became_active":"2024-04-25T10:09:21.714766+0000","last_became_peered":"2024-04-25T10:09:21.714766+0000","last_unstale":"2024-04-25T10:09:22.715845+0000","last_undegraded":"2024-04-25T10:09:22.715845+0000","last_fullsized":"2024-04-25T10:09:22.715845+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:30.994528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735372+0000","last_change":"2024-04-25T10:09:22.721662+0000","last_active":"2024-04-25T10:09:22.735372+0000","last_peered":"2024-04-25T10:09:22.735372+0000","last_clean":"2024-04-25T10:09:22.735372+0000","last_became_active":"2024-04-25T10:09:22.721339+0000","last_became_peered":"2024-04-25T10:09:22.721339+0000","last_unstale":"2024-04-25T10:09:22.735372+0000","last_undegraded":"2024-04-25T10:09:22.735372+0000","last_fullsized":"2024-04-25T10:09:22.735372+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:36.005374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"71'3","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.740562+0000","last_change":"2024-04-25T10:09:21.714788+0000","last_active":"2024-04-25T10:09:22.740562+0000","last_peered":"2024-04-25T10:09:22.740562+0000","last_clean":"2024-04-25T10:09:22.740562+0000","last_became_active":"2024-04-25T10:09:21.714533+0000","last_became_peered":"2024-04-25T10:09:21.714533+0000","last_unstale":"2024-04-25T10:09:22.740562+0000","last_undegraded":"2024-04-25T10:09:22.740562+0000","last_fullsized":"2024-04-25T10:09:22.740562+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:36.752649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:09:24.390902+0000","last_change":"2024-04-25T10:09:22.730064+0000","last_active":"2024-04-25T10:09:24.390902+0000","last_peered":"2024-04-25T10:09:24.390902+0000","last_clean":"2024-04-25T10:09:24.390902+0000","last_became_active":"2024-04-25T10:09:22.729454+0000","last_became_peered":"2024-04-25T10:09:22.729454+0000","last_unstale":"2024-04-25T10:09:24.390902+0000","last_undegraded":"2024-04-25T10:09:24.390902+0000","last_fullsized":"2024-04-25T10:09:24.390902+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:22.401538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"71'6","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.743850+0000","last_change":"2024-04-25T10:09:21.712915+0000","last_active":"2024-04-25T10:09:22.743850+0000","last_peered":"2024-04-25T10:09:22.743850+0000","last_clean":"2024-04-25T10:09:22.743850+0000","last_became_active":"2024-04-25T10:09:21.712644+0000","last_became_peered":"2024-04-25T10:09:21.712644+0000","last_unstale":"2024-04-25T10:09:22.743850+0000","last_undegraded":"2024-04-25T10:09:22.743850+0000","last_fullsized":"2024-04-25T10:09:22.743850+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:06.170911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.737377+0000","last_change":"2024-04-25T10:09:22.733894+0000","last_active":"2024-04-25T10:09:22.737377+0000","last_peered":"2024-04-25T10:09:22.737377+0000","last_clean":"2024-04-25T10:09:22.737377+0000","last_became_active":"2024-04-25T10:09:22.733596+0000","last_became_peered":"2024-04-25T10:09:22.733596+0000","last_unstale":"2024-04-25T10:09:22.737377+0000","last_undegraded":"2024-04-25T10:09:22.737377+0000","last_fullsized":"2024-04-25T10:09:22.737377+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:52.862877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.736865+0000","last_change":"2024-04-25T10:09:21.716006+0000","last_active":"2024-04-25T10:09:22.736865+0000","last_peered":"2024-04-25T10:09:22.736865+0000","last_clean":"2024-04-25T10:09:22.736865+0000","last_became_active":"2024-04-25T10:09:21.715740+0000","last_became_peered":"2024-04-25T10:09:21.715740+0000","last_unstale":"2024-04-25T10:09:22.736865+0000","last_undegraded":"2024-04-25T10:09:22.736865+0000","last_fullsized":"2024-04-25T10:09:22.736865+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:17.217685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741155+0000","last_change":"2024-04-25T10:09:22.738577+0000","last_active":"2024-04-25T10:09:22.741155+0000","last_peered":"2024-04-25T10:09:22.741155+0000","last_clean":"2024-04-25T10:09:22.741155+0000","last_became_active":"2024-04-25T10:09:22.738329+0000","last_became_peered":"2024-04-25T10:09:22.738329+0000","last_unstale":"2024-04-25T10:09:22.741155+0000","last_undegraded":"2024-04-25T10:09:22.741155+0000","last_fullsized":"2024-04-25T10:09:22.741155+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:36.710950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.720776+0000","last_change":"2024-04-25T10:09:21.716512+0000","last_active":"2024-04-25T10:09:22.720776+0000","last_peered":"2024-04-25T10:09:22.720776+0000","last_clean":"2024-04-25T10:09:22.720776+0000","last_became_active":"2024-04-25T10:09:21.716233+0000","last_became_peered":"2024-04-25T10:09:21.716233+0000","last_unstale":"2024-04-25T10:09:22.720776+0000","last_undegraded":"2024-04-25T10:09:22.720776+0000","last_fullsized":"2024-04-25T10:09:22.720776+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:26.543373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:09:24.390615+0000","last_change":"2024-04-25T10:09:22.739690+0000","last_active":"2024-04-25T10:09:24.390615+0000","last_peered":"2024-04-25T10:09:24.390615+0000","last_clean":"2024-04-25T10:09:24.390615+0000","last_became_active":"2024-04-25T10:09:22.739242+0000","last_became_peered":"2024-04-25T10:09:22.739242+0000","last_unstale":"2024-04-25T10:09:24.390615+0000","last_undegraded":"2024-04-25T10:09:24.390615+0000","last_fullsized":"2024-04-25T10:09:24.390615+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:50.263962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.720039+0000","last_change":"2024-04-25T10:09:21.713643+0000","last_active":"2024-04-25T10:09:22.720039+0000","last_peered":"2024-04-25T10:09:22.720039+0000","last_clean":"2024-04-25T10:09:22.720039+0000","last_became_active":"2024-04-25T10:09:21.713359+0000","last_became_peered":"2024-04-25T10:09:21.713359+0000","last_unstale":"2024-04-25T10:09:22.720039+0000","last_undegraded":"2024-04-25T10:09:22.720039+0000","last_fullsized":"2024-04-25T10:09:22.720039+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:49.427576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741100+0000","last_change":"2024-04-25T10:09:22.733905+0000","last_active":"2024-04-25T10:09:22.741100+0000","last_peered":"2024-04-25T10:09:22.741100+0000","last_clean":"2024-04-25T10:09:22.741100+0000","last_became_active":"2024-04-25T10:09:22.733630+0000","last_became_peered":"2024-04-25T10:09:22.733630+0000","last_unstale":"2024-04-25T10:09:22.741100+0000","last_undegraded":"2024-04-25T10:09:22.741100+0000","last_fullsized":"2024-04-25T10:09:22.741100+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:08.686900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"71'8","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735236+0000","last_change":"2024-04-25T10:09:21.719246+0000","last_active":"2024-04-25T10:09:22.735236+0000","last_peered":"2024-04-25T10:09:22.735236+0000","last_clean":"2024-04-25T10:09:22.735236+0000","last_became_active":"2024-04-25T10:09:21.718931+0000","last_became_peered":"2024-04-25T10:09:21.718931+0000","last_unstale":"2024-04-25T10:09:22.735236+0000","last_undegraded":"2024-04-25T10:09:22.735236+0000","last_fullsized":"2024-04-25T10:09:22.735236+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:53.574499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.742293+0000","last_change":"2024-04-25T10:09:22.737752+0000","last_active":"2024-04-25T10:09:22.742293+0000","last_peered":"2024-04-25T10:09:22.742293+0000","last_clean":"2024-04-25T10:09:22.742293+0000","last_became_active":"2024-04-25T10:09:22.737400+0000","last_became_peered":"2024-04-25T10:09:22.737400+0000","last_unstale":"2024-04-25T10:09:22.742293+0000","last_undegraded":"2024-04-25T10:09:22.742293+0000","last_fullsized":"2024-04-25T10:09:22.742293+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:55.327479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741686+0000","last_change":"2024-04-25T10:09:21.718854+0000","last_active":"2024-04-25T10:09:22.741686+0000","last_peered":"2024-04-25T10:09:22.741686+0000","last_clean":"2024-04-25T10:09:22.741686+0000","last_became_active":"2024-04-25T10:09:21.718471+0000","last_became_peered":"2024-04-25T10:09:21.718471+0000","last_unstale":"2024-04-25T10:09:22.741686+0000","last_undegraded":"2024-04-25T10:09:22.741686+0000","last_fullsized":"2024-04-25T10:09:22.741686+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:59.803346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.730320+0000","last_change":"2024-04-25T10:09:22.727103+0000","last_active":"2024-04-25T10:09:22.730320+0000","last_peered":"2024-04-25T10:09:22.730320+0000","last_clean":"2024-04-25T10:09:22.730320+0000","last_became_active":"2024-04-25T10:09:22.726504+0000","last_became_peered":"2024-04-25T10:09:22.726504+0000","last_unstale":"2024-04-25T10:09:22.730320+0000","last_undegraded":"2024-04-25T10:09:22.730320+0000","last_fullsized":"2024-04-25T10:09:22.730320+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:44.497003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.742808+0000","last_change":"2024-04-25T10:09:21.714571+0000","last_active":"2024-04-25T10:09:22.742808+0000","last_peered":"2024-04-25T10:09:22.742808+0000","last_clean":"2024-04-25T10:09:22.742808+0000","last_became_active":"2024-04-25T10:09:21.713769+0000","last_became_peered":"2024-04-25T10:09:21.713769+0000","last_unstale":"2024-04-25T10:09:22.742808+0000","last_undegraded":"2024-04-25T10:09:22.742808+0000","last_fullsized":"2024-04-25T10:09:22.742808+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:36.752649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741220+0000","last_change":"2024-04-25T10:09:22.735039+0000","last_active":"2024-04-25T10:09:22.741220+0000","last_peered":"2024-04-25T10:09:22.741220+0000","last_clean":"2024-04-25T10:09:22.741220+0000","last_became_active":"2024-04-25T10:09:22.734774+0000","last_became_peered":"2024-04-25T10:09:22.734774+0000","last_unstale":"2024-04-25T10:09:22.741220+0000","last_undegraded":"2024-04-25T10:09:22.741220+0000","last_fullsized":"2024-04-25T10:09:22.741220+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:54.578465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.743203+0000","last_change":"2024-04-25T10:09:21.716949+0000","last_active":"2024-04-25T10:09:22.743203+0000","last_peered":"2024-04-25T10:09:22.743203+0000","last_clean":"2024-04-25T10:09:22.743203+0000","last_became_active":"2024-04-25T10:09:21.716630+0000","last_became_peered":"2024-04-25T10:09:21.716630+0000","last_unstale":"2024-04-25T10:09:22.743203+0000","last_undegraded":"2024-04-25T10:09:22.743203+0000","last_fullsized":"2024-04-25T10:09:22.743203+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:17.676932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741354+0000","last_change":"2024-04-25T10:09:22.737902+0000","last_active":"2024-04-25T10:09:22.741354+0000","last_peered":"2024-04-25T10:09:22.741354+0000","last_clean":"2024-04-25T10:09:22.741354+0000","last_became_active":"2024-04-25T10:09:22.737380+0000","last_became_peered":"2024-04-25T10:09:22.737380+0000","last_unstale":"2024-04-25T10:09:22.741354+0000","last_undegraded":"2024-04-25T10:09:22.741354+0000","last_fullsized":"2024-04-25T10:09:22.741354+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:31.226453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.710382+0000","last_change":"2024-04-25T10:09:21.716162+0000","last_active":"2024-04-25T10:09:22.710382+0000","last_peered":"2024-04-25T10:09:22.710382+0000","last_clean":"2024-04-25T10:09:22.710382+0000","last_became_active":"2024-04-25T10:09:21.715886+0000","last_became_peered":"2024-04-25T10:09:21.715886+0000","last_unstale":"2024-04-25T10:09:22.710382+0000","last_undegraded":"2024-04-25T10:09:22.710382+0000","last_fullsized":"2024-04-25T10:09:22.710382+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.760023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741392+0000","last_change":"2024-04-25T10:09:22.733689+0000","last_active":"2024-04-25T10:09:22.741392+0000","last_peered":"2024-04-25T10:09:22.741392+0000","last_clean":"2024-04-25T10:09:22.741392+0000","last_became_active":"2024-04-25T10:09:22.733420+0000","last_became_peered":"2024-04-25T10:09:22.733420+0000","last_unstale":"2024-04-25T10:09:22.741392+0000","last_undegraded":"2024-04-25T10:09:22.741392+0000","last_fullsized":"2024-04-25T10:09:22.741392+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:35.911524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.710603+0000","last_change":"2024-04-25T10:09:21.713011+0000","last_active":"2024-04-25T10:09:22.710603+0000","last_peered":"2024-04-25T10:09:22.710603+0000","last_clean":"2024-04-25T10:09:22.710603+0000","last_became_active":"2024-04-25T10:09:21.712653+0000","last_became_peered":"2024-04-25T10:09:21.712653+0000","last_unstale":"2024-04-25T10:09:22.710603+0000","last_undegraded":"2024-04-25T10:09:22.710603+0000","last_fullsized":"2024-04-25T10:09:22.710603+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:40.526086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741658+0000","last_change":"2024-04-25T10:09:22.738516+0000","last_active":"2024-04-25T10:09:22.741658+0000","last_peered":"2024-04-25T10:09:22.741658+0000","last_clean":"2024-04-25T10:09:22.741658+0000","last_became_active":"2024-04-25T10:09:22.738264+0000","last_became_peered":"2024-04-25T10:09:22.738264+0000","last_unstale":"2024-04-25T10:09:22.741658+0000","last_undegraded":"2024-04-25T10:09:22.741658+0000","last_fullsized":"2024-04-25T10:09:22.741658+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:37.756616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.715613+0000","last_change":"2024-04-25T10:09:21.707102+0000","last_active":"2024-04-25T10:09:22.715613+0000","last_peered":"2024-04-25T10:09:22.715613+0000","last_clean":"2024-04-25T10:09:22.715613+0000","last_became_active":"2024-04-25T10:09:21.706788+0000","last_became_peered":"2024-04-25T10:09:21.706788+0000","last_unstale":"2024-04-25T10:09:22.715613+0000","last_undegraded":"2024-04-25T10:09:22.715613+0000","last_fullsized":"2024-04-25T10:09:22.715613+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.907558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.738481+0000","last_change":"2024-04-25T10:09:22.730156+0000","last_active":"2024-04-25T10:09:22.738481+0000","last_peered":"2024-04-25T10:09:22.738481+0000","last_clean":"2024-04-25T10:09:22.738481+0000","last_became_active":"2024-04-25T10:09:22.729878+0000","last_became_peered":"2024-04-25T10:09:22.729878+0000","last_unstale":"2024-04-25T10:09:22.738481+0000","last_undegraded":"2024-04-25T10:09:22.738481+0000","last_fullsized":"2024-04-25T10:09:22.738481+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:52.862877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:09:24.390515+0000","last_change":"2024-04-25T10:09:21.714259+0000","last_active":"2024-04-25T10:09:24.390515+0000","last_peered":"2024-04-25T10:09:24.390515+0000","last_clean":"2024-04-25T10:09:24.390515+0000","last_became_active":"2024-04-25T10:09:21.713986+0000","last_became_peered":"2024-04-25T10:09:21.713986+0000","last_unstale":"2024-04-25T10:09:24.390515+0000","last_undegraded":"2024-04-25T10:09:24.390515+0000","last_fullsized":"2024-04-25T10:09:24.390515+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:33.022843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:09:24.391091+0000","last_change":"2024-04-25T10:09:22.736073+0000","last_active":"2024-04-25T10:09:24.391091+0000","last_peered":"2024-04-25T10:09:24.391091+0000","last_clean":"2024-04-25T10:09:24.391091+0000","last_became_active":"2024-04-25T10:09:22.734393+0000","last_became_peered":"2024-04-25T10:09:22.734393+0000","last_unstale":"2024-04-25T10:09:24.391091+0000","last_undegraded":"2024-04-25T10:09:24.391091+0000","last_fullsized":"2024-04-25T10:09:24.391091+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:16.758933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.741460+0000","last_change":"2024-04-25T10:09:21.712999+0000","last_active":"2024-04-25T10:09:22.741460+0000","last_peered":"2024-04-25T10:09:22.741460+0000","last_clean":"2024-04-25T10:09:22.741460+0000","last_became_active":"2024-04-25T10:09:21.712715+0000","last_became_peered":"2024-04-25T10:09:21.712715+0000","last_unstale":"2024-04-25T10:09:22.741460+0000","last_undegraded":"2024-04-25T10:09:22.741460+0000","last_fullsized":"2024-04-25T10:09:22.741460+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:19.872440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.732689+0000","last_change":"2024-04-25T10:09:22.724675+0000","last_active":"2024-04-25T10:09:22.732689+0000","last_peered":"2024-04-25T10:09:22.732689+0000","last_clean":"2024-04-25T10:09:22.732689+0000","last_became_active":"2024-04-25T10:09:22.723506+0000","last_became_peered":"2024-04-25T10:09:22.723506+0000","last_unstale":"2024-04-25T10:09:22.732689+0000","last_undegraded":"2024-04-25T10:09:22.732689+0000","last_fullsized":"2024-04-25T10:09:22.732689+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:31.226453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.714801+0000","last_change":"2024-04-25T10:09:21.721913+0000","last_active":"2024-04-25T10:09:22.714801+0000","last_peered":"2024-04-25T10:09:22.714801+0000","last_clean":"2024-04-25T10:09:22.714801+0000","last_became_active":"2024-04-25T10:09:21.721616+0000","last_became_peered":"2024-04-25T10:09:21.721616+0000","last_unstale":"2024-04-25T10:09:22.714801+0000","last_undegraded":"2024-04-25T10:09:22.714801+0000","last_fullsized":"2024-04-25T10:09:22.714801+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:15.166230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.728083+0000","last_change":"2024-04-25T10:09:22.726050+0000","last_active":"2024-04-25T10:09:22.728083+0000","last_peered":"2024-04-25T10:09:22.728083+0000","last_clean":"2024-04-25T10:09:22.728083+0000","last_became_active":"2024-04-25T10:09:22.725480+0000","last_became_peered":"2024-04-25T10:09:22.725480+0000","last_unstale":"2024-04-25T10:09:22.728083+0000","last_undegraded":"2024-04-25T10:09:22.728083+0000","last_fullsized":"2024-04-25T10:09:22.728083+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:41.530052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.725342+0000","last_change":"2024-04-25T10:09:21.710120+0000","last_active":"2024-04-25T10:09:22.725342+0000","last_peered":"2024-04-25T10:09:22.725342+0000","last_clean":"2024-04-25T10:09:22.725342+0000","last_became_active":"2024-04-25T10:09:21.709858+0000","last_became_peered":"2024-04-25T10:09:21.709858+0000","last_unstale":"2024-04-25T10:09:22.725342+0000","last_undegraded":"2024-04-25T10:09:22.725342+0000","last_fullsized":"2024-04-25T10:09:22.725342+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.760023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735167+0000","last_change":"2024-04-25T10:09:22.727860+0000","last_active":"2024-04-25T10:09:22.735167+0000","last_peered":"2024-04-25T10:09:22.735167+0000","last_clean":"2024-04-25T10:09:22.735167+0000","last_became_active":"2024-04-25T10:09:22.727568+0000","last_became_peered":"2024-04-25T10:09:22.727568+0000","last_unstale":"2024-04-25T10:09:22.735167+0000","last_undegraded":"2024-04-25T10:09:22.735167+0000","last_fullsized":"2024-04-25T10:09:22.735167+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:45.867130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.725415+0000","last_change":"2024-04-25T10:09:21.714194+0000","last_active":"2024-04-25T10:09:22.725415+0000","last_peered":"2024-04-25T10:09:22.725415+0000","last_clean":"2024-04-25T10:09:22.725415+0000","last_became_active":"2024-04-25T10:09:21.713930+0000","last_became_peered":"2024-04-25T10:09:21.713930+0000","last_unstale":"2024-04-25T10:09:22.725415+0000","last_undegraded":"2024-04-25T10:09:22.725415+0000","last_fullsized":"2024-04-25T10:09:22.725415+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:40.526086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:09:24.390776+0000","last_change":"2024-04-25T10:09:22.733789+0000","last_active":"2024-04-25T10:09:24.390776+0000","last_peered":"2024-04-25T10:09:24.390776+0000","last_clean":"2024-04-25T10:09:24.390776+0000","last_became_active":"2024-04-25T10:09:22.733516+0000","last_became_peered":"2024-04-25T10:09:22.733516+0000","last_unstale":"2024-04-25T10:09:24.390776+0000","last_undegraded":"2024-04-25T10:09:24.390776+0000","last_fullsized":"2024-04-25T10:09:24.390776+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:42.464848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.724853+0000","last_change":"2024-04-25T10:09:21.719021+0000","last_active":"2024-04-25T10:09:22.724853+0000","last_peered":"2024-04-25T10:09:22.724853+0000","last_clean":"2024-04-25T10:09:22.724853+0000","last_became_active":"2024-04-25T10:09:21.718771+0000","last_became_peered":"2024-04-25T10:09:21.718771+0000","last_unstale":"2024-04-25T10:09:22.724853+0000","last_undegraded":"2024-04-25T10:09:22.724853+0000","last_fullsized":"2024-04-25T10:09:22.724853+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:30.994528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735627+0000","last_change":"2024-04-25T10:09:22.727538+0000","last_active":"2024-04-25T10:09:22.735627+0000","last_peered":"2024-04-25T10:09:22.735627+0000","last_clean":"2024-04-25T10:09:22.735627+0000","last_became_active":"2024-04-25T10:09:22.727237+0000","last_became_peered":"2024-04-25T10:09:22.727237+0000","last_unstale":"2024-04-25T10:09:22.735627+0000","last_undegraded":"2024-04-25T10:09:22.735627+0000","last_fullsized":"2024-04-25T10:09:22.735627+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:12.943229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:09:24.390831+0000","last_change":"2024-04-25T10:09:21.720653+0000","last_active":"2024-04-25T10:09:24.390831+0000","last_peered":"2024-04-25T10:09:24.390831+0000","last_clean":"2024-04-25T10:09:24.390831+0000","last_became_active":"2024-04-25T10:09:21.720349+0000","last_became_peered":"2024-04-25T10:09:21.720349+0000","last_unstale":"2024-04-25T10:09:24.390831+0000","last_undegraded":"2024-04-25T10:09:24.390831+0000","last_fullsized":"2024-04-25T10:09:24.390831+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:26.308994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.735963+0000","last_change":"2024-04-25T10:09:22.732403+0000","last_active":"2024-04-25T10:09:22.735963+0000","last_peered":"2024-04-25T10:09:22.735963+0000","last_clean":"2024-04-25T10:09:22.735963+0000","last_became_active":"2024-04-25T10:09:22.732058+0000","last_became_peered":"2024-04-25T10:09:22.732058+0000","last_unstale":"2024-04-25T10:09:22.735963+0000","last_undegraded":"2024-04-25T10:09:22.735963+0000","last_fullsized":"2024-04-25T10:09:22.735963+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:51.105222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.714988+0000","last_change":"2024-04-25T10:09:21.708836+0000","last_active":"2024-04-25T10:09:22.714988+0000","last_peered":"2024-04-25T10:09:22.714988+0000","last_clean":"2024-04-25T10:09:22.714988+0000","last_became_active":"2024-04-25T10:09:21.708407+0000","last_became_peered":"2024-04-25T10:09:21.708407+0000","last_unstale":"2024-04-25T10:09:22.714988+0000","last_undegraded":"2024-04-25T10:09:22.714988+0000","last_fullsized":"2024-04-25T10:09:22.714988+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:50.101256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.734632+0000","last_change":"2024-04-25T10:09:22.732235+0000","last_active":"2024-04-25T10:09:22.734632+0000","last_peered":"2024-04-25T10:09:22.734632+0000","last_clean":"2024-04-25T10:09:22.734632+0000","last_became_active":"2024-04-25T10:09:22.731603+0000","last_became_peered":"2024-04-25T10:09:22.731603+0000","last_unstale":"2024-04-25T10:09:22.734632+0000","last_undegraded":"2024-04-25T10:09:22.734632+0000","last_fullsized":"2024-04-25T10:09:22.734632+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:26.292053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.709727+0000","last_change":"2024-04-25T10:09:21.716256+0000","last_active":"2024-04-25T10:09:22.709727+0000","last_peered":"2024-04-25T10:09:22.709727+0000","last_clean":"2024-04-25T10:09:22.709727+0000","last_became_active":"2024-04-25T10:09:21.715933+0000","last_became_peered":"2024-04-25T10:09:21.715933+0000","last_unstale":"2024-04-25T10:09:22.709727+0000","last_undegraded":"2024-04-25T10:09:22.709727+0000","last_fullsized":"2024-04-25T10:09:22.709727+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.907558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T10:09:24.390704+0000","last_change":"2024-04-25T10:09:22.738057+0000","last_active":"2024-04-25T10:09:24.390704+0000","last_peered":"2024-04-25T10:09:24.390704+0000","last_clean":"2024-04-25T10:09:24.390704+0000","last_became_active":"2024-04-25T10:09:22.737775+0000","last_became_peered":"2024-04-25T10:09:22.737775+0000","last_unstale":"2024-04-25T10:09:24.390704+0000","last_undegraded":"2024-04-25T10:09:24.390704+0000","last_fullsized":"2024-04-25T10:09:24.390704+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:21.694875+0000","last_clean_scrub_stamp":"2024-04-25T10:09:21.694875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:54.705160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.739567+0000","last_change":"2024-04-25T10:09:21.712598+0000","last_active":"2024-04-25T10:09:22.739567+0000","last_peered":"2024-04-25T10:09:22.739567+0000","last_clean":"2024-04-25T10:09:22.739567+0000","last_became_active":"2024-04-25T10:09:21.712297+0000","last_became_peered":"2024-04-25T10:09:21.712297+0000","last_unstale":"2024-04-25T10:09:22.739567+0000","last_undegraded":"2024-04-25T10:09:22.739567+0000","last_fullsized":"2024-04-25T10:09:22.739567+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:20.690908+0000","last_clean_scrub_stamp":"2024-04-25T10:09:20.690908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:34.008504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":270,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-25T10:09:22.720040+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:09:22.720040+0000","last_peered":"2024-04-25T10:09:22.720040+0000","last_clean":"2024-04-25T10:09:22.720040+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:09:22.720040+0000","last_undegraded":"2024-04-25T10:09:22.720040+0000","last_fullsized":"2024-04-25T10:09:22.720040+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509496,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":843776,"data_stored":351049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509496,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":843776,"data_stored":351049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509497,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":843776,"data_stored":351049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509497,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":843776,"data_stored":351049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509498,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":843776,"data_stored":351049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509496,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1437696,"data_stored":941417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509498,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":843776,"data_stored":351049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1454080,"data_stored":948282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:09:26.077 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:09:26.077 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:09:26.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.216+0000 7fbb31b63640 1 Processor -- start 2024-04-25T10:09:26.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.216+0000 7fbb31b63640 1 -- start start 2024-04-25T10:09:26.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.216+0000 7fbb31b63640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb2c0cd880 0x7fbb2c0ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.216+0000 7fbb31b63640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2c15be80 con 0x7fbb2c0cb2d0 2024-04-25T10:09:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.216+0000 7fbb31b63640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2c15b0c0 con 0x7fbb2c0ceda0 2024-04-25T10:09:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.216+0000 7fbb31b63640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c15ae90 con 0x7fbb2c0cd880 2024-04-25T10:09:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbb2c0cb2d0 0x7fbb2c0cb760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50450/0 (socket says 172.21.15.16:50450) 2024-04-25T10:09:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2bfff640 1 -- 172.21.15.16:0/1684437796 learned_addr learned my addr 172.21.15.16:0/1684437796 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2affd640 1 --2- 172.21.15.16:0/1684437796 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb2c0cd880 0x7fbb2c0ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/1684437796 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3875742933 0 0) 0x7fbb2c15be80 con 0x7fbb2c0cb2d0 2024-04-25T10:09:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/1684437796 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb14003520 con 0x7fbb2c0cb2d0 2024-04-25T10:09:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/1684437796 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1450061098 0 0) 0x7fbb14003520 con 0x7fbb2c0cb2d0 2024-04-25T10:09:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/1684437796 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb2c0cd880 msgr2=0x7fbb2c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2a7fc640 1 --2- 172.21.15.16:0/1684437796 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb2c0cd880 0x7fbb2c0ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/1684437796 >> v1:172.21.15.186:6789/0 conn(0x7fbb2c0ceda0 legacy=0x7fbb2c0cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/1684437796 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c15b3a0 con 0x7fbb2c0cb2d0 2024-04-25T10:09:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/1684437796 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fbb240019f0 con 0x7fbb2c0cb2d0 2024-04-25T10:09:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/1684437796 >> v1:172.21.15.16:6789/0 conn(0x7fbb2c0cb2d0 legacy=0x7fbb2c0cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2a7fc640 1 --2- 172.21.15.16:0/1684437796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb140039e0 0x7fbb14003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2a7fc640 1 --2- 172.21.15.16:0/1684437796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb14004390 0x7fbb14004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2a7fc640 1 --2- 172.21.15.16:0/1684437796 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb2c0cd880 0x7fbb140050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/1684437796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1400b9c0 con 0x7fbb14004390 2024-04-25T10:09:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/1684437796 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1400bb40 con 0x7fbb140039e0 2024-04-25T10:09:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/1684437796 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1400df50 con 0x7fbb2c0cd880 2024-04-25T10:09:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2affd640 1 --2- 172.21.15.16:0/1684437796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb14004390 0x7fbb14004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.217+0000 7fbb2b7fe640 1 --2- 172.21.15.16:0/1684437796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb140039e0 0x7fbb14003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.218+0000 7fbb2bfff640 1 --2- 172.21.15.16:0/1684437796 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb2c0cd880 0x7fbb140050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.218+0000 7fbb2affd640 1 -- 172.21.15.16:0/1684437796 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb2c0cd880 msgr2=0x7fbb140050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.218+0000 7fbb2affd640 1 --2- 172.21.15.16:0/1684437796 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb2c0cd880 0x7fbb140050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.218+0000 7fbb2affd640 1 -- 172.21.15.16:0/1684437796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb140039e0 msgr2=0x7fbb14003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.218+0000 7fbb2affd640 1 --2- 172.21.15.16:0/1684437796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb140039e0 0x7fbb14003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.218+0000 7fbb2affd640 1 -- 172.21.15.16:0/1684437796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb14003520 con 0x7fbb14004390 2024-04-25T10:09:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.218+0000 7fbb2affd640 1 --2- 172.21.15.16:0/1684437796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb14004390 0x7fbb14004860 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009ac0 tx=0x7fbb14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482df4c9618d2f62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.219+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/1684437796 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20036070 con 0x7fbb14004390 2024-04-25T10:09:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.219+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/1684437796 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb20031400 con 0x7fbb14004390 2024-04-25T10:09:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.219+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/1684437796 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20009690 con 0x7fbb14004390 2024-04-25T10:09:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.219+0000 7fbb31b63640 1 -- 172.21.15.16:0/1684437796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb14004390 msgr2=0x7fbb14004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.219+0000 7fbb31b63640 1 --2- 172.21.15.16:0/1684437796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb14004390 0x7fbb14004860 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009ac0 tx=0x7fbb14005830 comp rx=0 tx=0).stop 2024-04-25T10:09:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.219+0000 7fbb2bfff640 1 -- 172.21.15.16:0/1684437796 reap_dead start 2024-04-25T10:09:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.219+0000 7fbb31b63640 1 -- 172.21.15.16:0/1684437796 shutdown_connections 2024-04-25T10:09:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.219+0000 7fbb31b63640 1 -- 172.21.15.16:0/1684437796 >> 172.21.15.16:0/1684437796 conn(0x7fbb2c099a40 msgr2=0x7fbb2c0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.219+0000 7fbb31b63640 1 -- 172.21.15.16:0/1684437796 shutdown_connections 2024-04-25T10:09:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.219+0000 7fbb31b63640 1 -- 172.21.15.16:0/1684437796 wait complete. 2024-04-25T10:09:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.220+0000 7fbb31b63640 1 Processor -- start 2024-04-25T10:09:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.221+0000 7fbb31b63640 1 -- start start 2024-04-25T10:09:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.221+0000 7fbb31b63640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb14004390 0x7fbb2c155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.221+0000 7fbb31b63640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c0c7ff0 0x7fbb2c0c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.221+0000 7fbb31b63640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb2c15c850 0x7fbb2c15ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.221+0000 7fbb31b63640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c15b960 con 0x7fbb2c0c7ff0 2024-04-25T10:09:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.221+0000 7fbb31b63640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0ceae0 con 0x7fbb14004390 2024-04-25T10:09:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.221+0000 7fbb31b63640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0ce390 con 0x7fbb2c15c850 2024-04-25T10:09:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.221+0000 7fbb2affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c0c7ff0 0x7fbb2c0c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.221+0000 7fbb2affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c0c7ff0 0x7fbb2c0c8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52474/0 (socket says 172.21.15.16:52474) 2024-04-25T10:09:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.221+0000 7fbb2affd640 1 -- 172.21.15.16:0/17219451 learned_addr learned my addr 172.21.15.16:0/17219451 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.222+0000 7fbb2bfff640 1 --2- 172.21.15.16:0/17219451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb2c15c850 0x7fbb2c15ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.222+0000 7fbb2b7fe640 1 --2- 172.21.15.16:0/17219451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb14004390 0x7fbb2c155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.222+0000 7fbb2affd640 1 -- 172.21.15.16:0/17219451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb2c15c850 msgr2=0x7fbb2c15ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.222+0000 7fbb2affd640 1 --2- 172.21.15.16:0/17219451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb2c15c850 0x7fbb2c15ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.222+0000 7fbb2affd640 1 -- 172.21.15.16:0/17219451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb14004390 msgr2=0x7fbb2c155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.222+0000 7fbb2affd640 1 --2- 172.21.15.16:0/17219451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb14004390 0x7fbb2c155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.222+0000 7fbb2affd640 1 -- 172.21.15.16:0/17219451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c15d580 con 0x7fbb2c0c7ff0 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.222+0000 7fbb2affd640 1 --2- 172.21.15.16:0/17219451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c0c7ff0 0x7fbb2c0c8470 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fbb20034040 tx=0x7fbb20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.222+0000 7fbb13fff640 1 -- 172.21.15.16:0/17219451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20040020 con 0x7fbb2c0c7ff0 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.222+0000 7fbb13fff640 1 -- 172.21.15.16:0/17219451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb20003bc0 con 0x7fbb2c0c7ff0 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.222+0000 7fbb31b63640 1 -- 172.21.15.16:0/17219451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2c15d860 con 0x7fbb2c0c7ff0 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.222+0000 7fbb13fff640 1 -- 172.21.15.16:0/17219451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20036040 con 0x7fbb2c0c7ff0 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.222+0000 7fbb31b63640 1 -- 172.21.15.16:0/17219451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2c15de20 con 0x7fbb2c0c7ff0 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.223+0000 7fbb31b63640 1 -- 172.21.15.16:0/17219451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb2c0749b0 con 0x7fbb2c0c7ff0 2024-04-25T10:09:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.224+0000 7fbb13fff640 1 -- 172.21.15.16:0/17219451 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb20043020 con 0x7fbb2c0c7ff0 2024-04-25T10:09:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.224+0000 7fbb13fff640 1 --2- 172.21.15.16:0/17219451 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbb1806b500 0x7fbb1806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.224+0000 7fbb13fff640 1 -- 172.21.15.16:0/17219451 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb200aa340 con 0x7fbb2c0c7ff0 2024-04-25T10:09:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.224+0000 7fbb2b7fe640 1 --2- 172.21.15.16:0/17219451 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbb1806b500 0x7fbb1806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.225+0000 7fbb2b7fe640 1 --2- 172.21.15.16:0/17219451 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbb1806b500 0x7fbb1806d9c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c005620 tx=0x7fbb1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.228+0000 7fbb13fff640 1 -- 172.21.15.16:0/17219451 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb2004a070 con 0x7fbb2c0c7ff0 2024-04-25T10:09:26.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.479+0000 7fbb31b63640 1 -- 172.21.15.16:0/17219451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fbb2c072fd0 con 0x7fbb2c0c7ff0 2024-04-25T10:09:26.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.481+0000 7fbb13fff640 1 -- 172.21.15.16:0/17219451 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7fbb200044e0 con 0x7fbb2c0c7ff0 2024-04-25T10:09:26.488 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-04-25T10:09:26.488 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCZKipmAph/GhAAG3x1zC98yHQ13fjpmrlPkg== 2024-04-25T10:09:26.488 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-04-25T10:09:26.488 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-04-25T10:09:26.488 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:09:26.488 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-04-25T10:09:26.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.483+0000 7fbb31b63640 1 -- 172.21.15.16:0/17219451 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbb1806b500 msgr2=0x7fbb1806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.483+0000 7fbb31b63640 1 --2- 172.21.15.16:0/17219451 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbb1806b500 0x7fbb1806d9c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c005620 tx=0x7fbb1c009040 comp rx=0 tx=0).stop 2024-04-25T10:09:26.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.483+0000 7fbb31b63640 1 -- 172.21.15.16:0/17219451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c0c7ff0 msgr2=0x7fbb2c0c8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.483+0000 7fbb31b63640 1 --2- 172.21.15.16:0/17219451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c0c7ff0 0x7fbb2c0c8470 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fbb20034040 tx=0x7fbb20004730 comp rx=0 tx=0).stop 2024-04-25T10:09:26.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.483+0000 7fbb31b63640 1 -- 172.21.15.16:0/17219451 shutdown_connections 2024-04-25T10:09:26.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.483+0000 7fbb31b63640 1 --2- 172.21.15.16:0/17219451 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbb1806b500 0x7fbb1806d9c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.483+0000 7fbb31b63640 1 --2- 172.21.15.16:0/17219451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb2c15c850 0x7fbb2c15ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.483+0000 7fbb31b63640 1 --2- 172.21.15.16:0/17219451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c0c7ff0 0x7fbb2c0c8470 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.483+0000 7fbb31b63640 1 --2- 172.21.15.16:0/17219451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb14004390 0x7fbb2c155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.484+0000 7fbb31b63640 1 -- 172.21.15.16:0/17219451 >> 172.21.15.16:0/17219451 conn(0x7fbb2c099a40 msgr2=0x7fbb2c0ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:26.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.484+0000 7fbb31b63640 1 -- 172.21.15.16:0/17219451 shutdown_connections 2024-04-25T10:09:26.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.484+0000 7fbb31b63640 1 -- 172.21.15.16:0/17219451 wait complete. 2024-04-25T10:09:26.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:26.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.596+0000 7fc8a8cb3640 1 Processor -- start 2024-04-25T10:09:26.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.596+0000 7fc8a8cb3640 1 -- start start 2024-04-25T10:09:26.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.596+0000 7fc8a8cb3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc8a40d35b0 0x7fc8a40b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.596+0000 7fc8a8cb3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc8a40b5990 0x7fc8a40b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.596+0000 7fc8a8cb3640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a415e0c0 con 0x7fc8a415dce0 2024-04-25T10:09:26.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.596+0000 7fc8a8cb3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a40d2510 con 0x7fc8a40d35b0 2024-04-25T10:09:26.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.596+0000 7fc8a8cb3640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a40d27f0 con 0x7fc8a40b5990 2024-04-25T10:09:26.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.596+0000 7fc89bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc8a40d35b0 0x7fc8a40b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.597+0000 7fc89bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc8a40d35b0 0x7fc8a40b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52482/0 (socket says 172.21.15.16:52482) 2024-04-25T10:09:26.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.597+0000 7fc89bfff640 1 -- 172.21.15.16:0/25433258 learned_addr learned my addr 172.21.15.16:0/25433258 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:26.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.597+0000 7fc8a2d9d640 1 --2- 172.21.15.16:0/25433258 >> v2:172.21.15.186:3300/0 conn(0x7fc8a40b5990 0x7fc8a40b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.597+0000 7fc89bfff640 1 -- 172.21.15.16:0/25433258 >> v2:172.21.15.186:3300/0 conn(0x7fc8a40b5990 msgr2=0x7fc8a40b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.597+0000 7fc89bfff640 1 --2- 172.21.15.16:0/25433258 >> v2:172.21.15.186:3300/0 conn(0x7fc8a40b5990 0x7fc8a40b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.597+0000 7fc89bfff640 1 -- 172.21.15.16:0/25433258 >> v1:172.21.15.186:6789/0 conn(0x7fc8a415dce0 legacy=0x7fc8a40d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.597+0000 7fc89bfff640 1 -- 172.21.15.16:0/25433258 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a40d2ad0 con 0x7fc8a40d35b0 2024-04-25T10:09:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.597+0000 7fc89bfff640 1 --2- 172.21.15.16:0/25433258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8a40d35b0 0x7fc8a40b5450 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40b3390 tx=0x7fc88c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a730614295c3dd68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.598+0000 7fc8a1d9b640 1 -- 172.21.15.16:0/25433258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c037070 con 0x7fc8a40d35b0 2024-04-25T10:09:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.598+0000 7fc8a1d9b640 1 -- 172.21.15.16:0/25433258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c032440 con 0x7fc8a40d35b0 2024-04-25T10:09:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.598+0000 7fc8a1d9b640 1 -- 172.21.15.16:0/25433258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c032810 con 0x7fc8a40d35b0 2024-04-25T10:09:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.598+0000 7fc8a8cb3640 1 -- 172.21.15.16:0/25433258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8a40d35b0 msgr2=0x7fc8a40b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.598+0000 7fc8a8cb3640 1 --2- 172.21.15.16:0/25433258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8a40d35b0 0x7fc8a40b5450 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40b3390 tx=0x7fc88c02a230 comp rx=0 tx=0).stop 2024-04-25T10:09:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.598+0000 7fc8a8cb3640 1 -- 172.21.15.16:0/25433258 shutdown_connections 2024-04-25T10:09:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.598+0000 7fc8a8cb3640 1 --2- 172.21.15.16:0/25433258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8a40d35b0 0x7fc8a40b5450 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.598+0000 7fc8a8cb3640 1 --2- 172.21.15.16:0/25433258 >> v2:172.21.15.186:3300/0 conn(0x7fc8a40b5990 0x7fc8a40b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.598+0000 7fc8a8cb3640 1 -- 172.21.15.16:0/25433258 >> 172.21.15.16:0/25433258 conn(0x7fc8a4072900 msgr2=0x7fc8a40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.598+0000 7fc8a8cb3640 1 -- 172.21.15.16:0/25433258 shutdown_connections 2024-04-25T10:09:26.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.598+0000 7fc8a8cb3640 1 -- 172.21.15.16:0/25433258 wait complete. 2024-04-25T10:09:26.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.600+0000 7fc8a8cb3640 1 Processor -- start 2024-04-25T10:09:26.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.600+0000 7fc8a8cb3640 1 -- start start 2024-04-25T10:09:26.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.600+0000 7fc8a8cb3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8a40b5990 0x7fc8a415c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.600+0000 7fc8a8cb3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8a40d35b0 0x7fc8a415f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.600+0000 7fc8a8cb3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc8a415dce0 0x7fc8a40aa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.600+0000 7fc8a8cb3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a40d2160 con 0x7fc8a40b5990 2024-04-25T10:09:26.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.600+0000 7fc8a8cb3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a40d3090 con 0x7fc8a40d35b0 2024-04-25T10:09:26.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.600+0000 7fc8a8cb3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a40c84c0 con 0x7fc8a415dce0 2024-04-25T10:09:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.601+0000 7fc8a259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8a40b5990 0x7fc8a415c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.601+0000 7fc8a259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8a40b5990 0x7fc8a415c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52492/0 (socket says 172.21.15.16:52492) 2024-04-25T10:09:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.601+0000 7fc8a259c640 1 -- 172.21.15.16:0/3327539195 learned_addr learned my addr 172.21.15.16:0/3327539195 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.601+0000 7fc8a259c640 1 -- 172.21.15.16:0/3327539195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc8a415dce0 msgr2=0x7fc8a40aa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.601+0000 7fc8a259c640 1 --2- 172.21.15.16:0/3327539195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc8a415dce0 0x7fc8a40aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.601+0000 7fc8a259c640 1 -- 172.21.15.16:0/3327539195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8a40d35b0 msgr2=0x7fc8a415f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.601+0000 7fc8a259c640 1 --2- 172.21.15.16:0/3327539195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8a40d35b0 0x7fc8a415f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.601+0000 7fc8a259c640 1 -- 172.21.15.16:0/3327539195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a40c8170 con 0x7fc8a40b5990 2024-04-25T10:09:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.601+0000 7fc8a259c640 1 --2- 172.21.15.16:0/3327539195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8a40b5990 0x7fc8a415c210 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc89400b3f0 tx=0x7fc89400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.601+0000 7fc89affd640 1 -- 172.21.15.16:0/3327539195 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894015020 con 0x7fc8a40b5990 2024-04-25T10:09:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.601+0000 7fc89affd640 1 -- 172.21.15.16:0/3327539195 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc894011830 con 0x7fc8a40b5990 2024-04-25T10:09:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.601+0000 7fc8a8cb3640 1 -- 172.21.15.16:0/3327539195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a415fee0 con 0x7fc8a40b5990 2024-04-25T10:09:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.601+0000 7fc89affd640 1 -- 172.21.15.16:0/3327539195 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894011c20 con 0x7fc8a40b5990 2024-04-25T10:09:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.601+0000 7fc8a8cb3640 1 -- 172.21.15.16:0/3327539195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a4160a60 con 0x7fc8a40b5990 2024-04-25T10:09:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.602+0000 7fc8a8cb3640 1 -- 172.21.15.16:0/3327539195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a40749b0 con 0x7fc8a40b5990 2024-04-25T10:09:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.603+0000 7fc89affd640 1 -- 172.21.15.16:0/3327539195 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc894018020 con 0x7fc8a40b5990 2024-04-25T10:09:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.603+0000 7fc89affd640 1 --2- 172.21.15.16:0/3327539195 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc88406b4d0 0x7fc88406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.603+0000 7fc89affd640 1 -- 172.21.15.16:0/3327539195 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc894089d90 con 0x7fc8a40b5990 2024-04-25T10:09:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.603+0000 7fc89bfff640 1 --2- 172.21.15.16:0/3327539195 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc88406b4d0 0x7fc88406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.604+0000 7fc89bfff640 1 --2- 172.21.15.16:0/3327539195 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc88406b4d0 0x7fc88406d990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc88c034000 tx=0x7fc88c02c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.607+0000 7fc89affd640 1 -- 172.21.15.16:0/3327539195 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc89401f030 con 0x7fc8a40b5990 2024-04-25T10:09:26.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.828+0000 7fc8a8cb3640 1 -- 172.21.15.16:0/3327539195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8a40b5770 con 0x7fc8a40b5990 2024-04-25T10:09:26.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.828+0000 7fc89affd640 1 -- 172.21.15.16:0/3327539195 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fc8940172c0 con 0x7fc8a40b5990 2024-04-25T10:09:26.835 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:26.835 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6070,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2805428272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2805428272},{"type":"v1","addr":"172.21.15.186:6835","nonce":2805428272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2897154125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2897154125},{"type":"v1","addr":"172.21.15.16:6833","nonce":2897154125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6636,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1735811292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1735811292},{"type":"v1","addr":"172.21.15.16:6835","nonce":1735811292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:22.077032+0000","modified":"2024-04-25T10:09:24.068248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35489":{"gid":35489,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/3292077067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3292077067},{"type":"v1","addr":"172.21.15.186:6837","nonce":3292077067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:09:26.835 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 36 2024-04-25T10:09:26.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.830+0000 7fc8a8cb3640 1 -- 172.21.15.16:0/3327539195 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc88406b4d0 msgr2=0x7fc88406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.830+0000 7fc8a8cb3640 1 --2- 172.21.15.16:0/3327539195 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc88406b4d0 0x7fc88406d990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc88c034000 tx=0x7fc88c02c000 comp rx=0 tx=0).stop 2024-04-25T10:09:26.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.830+0000 7fc8a8cb3640 1 -- 172.21.15.16:0/3327539195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8a40b5990 msgr2=0x7fc8a415c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.830+0000 7fc8a8cb3640 1 --2- 172.21.15.16:0/3327539195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8a40b5990 0x7fc8a415c210 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc89400b3f0 tx=0x7fc89400b8b0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.830+0000 7fc8a8cb3640 1 -- 172.21.15.16:0/3327539195 shutdown_connections 2024-04-25T10:09:26.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.831+0000 7fc8a8cb3640 1 --2- 172.21.15.16:0/3327539195 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc88406b4d0 0x7fc88406d990 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.831+0000 7fc8a8cb3640 1 --2- 172.21.15.16:0/3327539195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc8a415dce0 0x7fc8a40aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.831+0000 7fc8a8cb3640 1 --2- 172.21.15.16:0/3327539195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8a40d35b0 0x7fc8a415f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.831+0000 7fc8a8cb3640 1 --2- 172.21.15.16:0/3327539195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8a40b5990 0x7fc8a415c210 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.831+0000 7fc8a8cb3640 1 -- 172.21.15.16:0/3327539195 >> 172.21.15.16:0/3327539195 conn(0x7fc8a4072900 msgr2=0x7fc8a40b8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:26.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.831+0000 7fc8a8cb3640 1 -- 172.21.15.16:0/3327539195 shutdown_connections 2024-04-25T10:09:26.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:26.831+0000 7fc8a8cb3640 1 -- 172.21.15.16:0/3327539195 wait complete. 2024-04-25T10:09:27.250 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:09:22.077032+0000', 'modified': '2024-04-25T10:09:24.068248+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 35489}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_35489': {'gid': 35489, 'name': 'd', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.186:6837/3292077067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 3292077067}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 3292077067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:09:27.251 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:09:27.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:09:27.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.349+0000 7fd891609640 1 Processor -- start 2024-04-25T10:09:27.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.349+0000 7fd891609640 1 -- start start 2024-04-25T10:09:27.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.350+0000 7fd891609640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd88c0ceeb0 0x7fd88c168a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.350+0000 7fd891609640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd88c161f80 con 0x7fd88c161090 2024-04-25T10:09:27.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.350+0000 7fd891609640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd88c162260 con 0x7fd88c0b9720 2024-04-25T10:09:27.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.350+0000 7fd891609640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd88c0d21a0 con 0x7fd88c0ceeb0 2024-04-25T10:09:27.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.350+0000 7fd88affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd88c161090 0x7fd88c161730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50460/0 (socket says 172.21.15.16:50460) 2024-04-25T10:09:27.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.350+0000 7fd88affd640 1 -- 172.21.15.16:0/1999412344 learned_addr learned my addr 172.21.15.16:0/1999412344 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:27.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.350+0000 7fd889ffb640 1 -- 172.21.15.16:0/1999412344 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903453547 0 0) 0x7fd88c161f80 con 0x7fd88c161090 2024-04-25T10:09:27.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.350+0000 7fd889ffb640 1 -- 172.21.15.16:0/1999412344 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd878003520 con 0x7fd88c161090 2024-04-25T10:09:27.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.350+0000 7fd88b7fe640 1 --2- 172.21.15.16:0/1999412344 >> v2:172.21.15.186:3300/0 conn(0x7fd88c0ceeb0 0x7fd88c168a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd889ffb640 1 -- 172.21.15.16:0/1999412344 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2527700853 0 0) 0x7fd878003520 con 0x7fd88c161090 2024-04-25T10:09:27.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd889ffb640 1 -- 172.21.15.16:0/1999412344 >> v2:172.21.15.186:3300/0 conn(0x7fd88c0ceeb0 msgr2=0x7fd88c168a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:27.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd889ffb640 1 --2- 172.21.15.16:0/1999412344 >> v2:172.21.15.186:3300/0 conn(0x7fd88c0ceeb0 0x7fd88c168a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd889ffb640 1 -- 172.21.15.16:0/1999412344 >> v1:172.21.15.186:6789/0 conn(0x7fd88c0b9720 legacy=0x7fd88c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd889ffb640 1 -- 172.21.15.16:0/1999412344 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c162540 con 0x7fd88c161090 2024-04-25T10:09:27.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd889ffb640 1 -- 172.21.15.16:0/1999412344 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fd884001670 con 0x7fd88c161090 2024-04-25T10:09:27.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd889ffb640 1 -- 172.21.15.16:0/1999412344 >> v1:172.21.15.16:6789/0 conn(0x7fd88c161090 legacy=0x7fd88c161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd889ffb640 1 --2- 172.21.15.16:0/1999412344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8780039b0 0x7fd878003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd889ffb640 1 --2- 172.21.15.16:0/1999412344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd878004800 0x7fd878004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd889ffb640 1 --2- 172.21.15.16:0/1999412344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd878005210 0x7fd87800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd889ffb640 1 -- 172.21.15.16:0/1999412344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c0d21a0 con 0x7fd878005210 2024-04-25T10:09:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd889ffb640 1 -- 172.21.15.16:0/1999412344 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87800c140 con 0x7fd8780039b0 2024-04-25T10:09:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd889ffb640 1 -- 172.21.15.16:0/1999412344 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd87800c340 con 0x7fd878004800 2024-04-25T10:09:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd88affd640 1 --2- 172.21.15.16:0/1999412344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8780039b0 0x7fd878003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd88b7fe640 1 --2- 172.21.15.16:0/1999412344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd878005210 0x7fd87800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd88a7fc640 1 --2- 172.21.15.16:0/1999412344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd878004800 0x7fd878004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd88b7fe640 1 -- 172.21.15.16:0/1999412344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd878004800 msgr2=0x7fd878004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd88b7fe640 1 --2- 172.21.15.16:0/1999412344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd878004800 0x7fd878004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd88b7fe640 1 -- 172.21.15.16:0/1999412344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8780039b0 msgr2=0x7fd878003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd88b7fe640 1 --2- 172.21.15.16:0/1999412344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8780039b0 0x7fd878003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.351+0000 7fd88b7fe640 1 -- 172.21.15.16:0/1999412344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd878003520 con 0x7fd878005210 2024-04-25T10:09:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.352+0000 7fd88b7fe640 1 -- 172.21.15.16:0/1999412344 reap_dead start 2024-04-25T10:09:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.352+0000 7fd88b7fe640 1 --2- 172.21.15.16:0/1999412344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd878005210 0x7fd87800b9c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009fc0 tx=0x7fd878005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10dddad3671bce97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.352+0000 7fd889ffb640 1 -- 172.21.15.16:0/1999412344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c035020 con 0x7fd878005210 2024-04-25T10:09:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.352+0000 7fd889ffb640 1 -- 172.21.15.16:0/1999412344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c003bd0 con 0x7fd878005210 2024-04-25T10:09:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.352+0000 7fd889ffb640 1 -- 172.21.15.16:0/1999412344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c009c10 con 0x7fd878005210 2024-04-25T10:09:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.353+0000 7fd891609640 1 -- 172.21.15.16:0/1999412344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd878005210 msgr2=0x7fd87800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.353+0000 7fd891609640 1 --2- 172.21.15.16:0/1999412344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd878005210 0x7fd87800b9c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009fc0 tx=0x7fd878005830 comp rx=0 tx=0).stop 2024-04-25T10:09:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.353+0000 7fd891609640 1 -- 172.21.15.16:0/1999412344 shutdown_connections 2024-04-25T10:09:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.353+0000 7fd891609640 1 --2- 172.21.15.16:0/1999412344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd878005210 0x7fd87800b9c0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.353+0000 7fd891609640 1 -- 172.21.15.16:0/1999412344 >> 172.21.15.16:0/1999412344 conn(0x7fd88c0b55e0 msgr2=0x7fd88c0b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.353+0000 7fd891609640 1 -- 172.21.15.16:0/1999412344 shutdown_connections 2024-04-25T10:09:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.353+0000 7fd891609640 1 -- 172.21.15.16:0/1999412344 wait complete. 2024-04-25T10:09:27.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.354+0000 7fd891609640 1 Processor -- start 2024-04-25T10:09:27.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.354+0000 7fd891609640 1 -- start start 2024-04-25T10:09:27.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.355+0000 7fd891609640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd878005210 0x7fd88c15d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.355+0000 7fd891609640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c14b300 0x7fd88c0cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.355+0000 7fd891609640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd88c0cd380 0x7fd88c15a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.355+0000 7fd891609640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c0ce7a0 con 0x7fd878005210 2024-04-25T10:09:27.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.355+0000 7fd891609640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c161470 con 0x7fd88c14b300 2024-04-25T10:09:27.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.355+0000 7fd891609640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c0ce920 con 0x7fd88c0cd380 2024-04-25T10:09:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.355+0000 7fd88affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd878005210 0x7fd88c15d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.355+0000 7fd88affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd878005210 0x7fd88c15d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52518/0 (socket says 172.21.15.16:52518) 2024-04-25T10:09:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.355+0000 7fd88affd640 1 -- 172.21.15.16:0/3935893844 learned_addr learned my addr 172.21.15.16:0/3935893844 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.355+0000 7fd88a7fc640 1 --2- 172.21.15.16:0/3935893844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c14b300 0x7fd88c0cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.355+0000 7fd88b7fe640 1 --2- 172.21.15.16:0/3935893844 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd88c0cd380 0x7fd88c15a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.355+0000 7fd88affd640 1 -- 172.21.15.16:0/3935893844 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd88c0cd380 msgr2=0x7fd88c15a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.355+0000 7fd88affd640 1 --2- 172.21.15.16:0/3935893844 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd88c0cd380 0x7fd88c15a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.355+0000 7fd88affd640 1 -- 172.21.15.16:0/3935893844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c14b300 msgr2=0x7fd88c0cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.355+0000 7fd88affd640 1 --2- 172.21.15.16:0/3935893844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c14b300 0x7fd88c0cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.355+0000 7fd88affd640 1 -- 172.21.15.16:0/3935893844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c0cbe40 con 0x7fd878005210 2024-04-25T10:09:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.356+0000 7fd88affd640 1 --2- 172.21.15.16:0/3935893844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd878005210 0x7fd88c15d7e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd884004f30 tx=0x7fd8840074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.356+0000 7fd8837fe640 1 -- 172.21.15.16:0/3935893844 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88401a070 con 0x7fd878005210 2024-04-25T10:09:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.356+0000 7fd8837fe640 1 -- 172.21.15.16:0/3935893844 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88400f750 con 0x7fd878005210 2024-04-25T10:09:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.356+0000 7fd891609640 1 -- 172.21.15.16:0/3935893844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88c0cc120 con 0x7fd878005210 2024-04-25T10:09:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.356+0000 7fd8837fe640 1 -- 172.21.15.16:0/3935893844 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88400fb40 con 0x7fd878005210 2024-04-25T10:09:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.356+0000 7fd891609640 1 -- 172.21.15.16:0/3935893844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88c0cc6e0 con 0x7fd878005210 2024-04-25T10:09:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.357+0000 7fd8837fe640 1 -- 172.21.15.16:0/3935893844 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd88401d070 con 0x7fd878005210 2024-04-25T10:09:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.358+0000 7fd8837fe640 1 --2- 172.21.15.16:0/3935893844 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd86c06b4d0 0x7fd86c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.358+0000 7fd8837fe640 1 -- 172.21.15.16:0/3935893844 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd884089a50 con 0x7fd878005210 2024-04-25T10:09:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.358+0000 7fd88a7fc640 1 --2- 172.21.15.16:0/3935893844 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd86c06b4d0 0x7fd86c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.359+0000 7fd88a7fc640 1 --2- 172.21.15.16:0/3935893844 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd86c06b4d0 0x7fd86c06d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd874000c00 tx=0x7fd874009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.359+0000 7fd8943e0740 1 Processor -- start 2024-04-25T10:09:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.359+0000 7fd8943e0740 1 -- start start 2024-04-25T10:09:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd8943e0740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6eb9230 0x558fd6eb9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd8943e0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6ed6ad0 0x558fd6ed7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd8943e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6e8ef10 0x558fd6ed5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd8943e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd6df1a10 con 0x558fd6e8ef10 2024-04-25T10:09:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd8943e0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd6d9ab30 con 0x558fd6ed6ad0 2024-04-25T10:09:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd8943e0740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x558fd6dedc50 con 0x558fd6eb9230 2024-04-25T10:09:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd88b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6e8ef10 0x558fd6ed5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd88b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6e8ef10 0x558fd6ed5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52534/0 (socket says 172.21.15.16:52534) 2024-04-25T10:09:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd88b7fe640 1 -- 172.21.15.16:0/3336749609 learned_addr learned my addr 172.21.15.16:0/3336749609 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd88a7fc640 1 --2- 172.21.15.16:0/3336749609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6ed6ad0 0x558fd6ed7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd88affd640 1 --2- 172.21.15.16:0/3336749609 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6eb9230 0x558fd6eb9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd88b7fe640 1 -- 172.21.15.16:0/3336749609 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6eb9230 msgr2=0x558fd6eb9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd88b7fe640 1 --2- 172.21.15.16:0/3336749609 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6eb9230 0x558fd6eb9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd88b7fe640 1 -- 172.21.15.16:0/3336749609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6ed6ad0 msgr2=0x558fd6ed7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd88b7fe640 1 --2- 172.21.15.16:0/3336749609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6ed6ad0 0x558fd6ed7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd88b7fe640 1 -- 172.21.15.16:0/3336749609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fd6dafe10 con 0x558fd6e8ef10 2024-04-25T10:09:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd88b7fe640 1 --2- 172.21.15.16:0/3336749609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6e8ef10 0x558fd6ed5190 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00e230 tx=0x7fd87c02d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae54ecb359af0666 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd880ff9640 1 -- 172.21.15.16:0/3336749609 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c040070 con 0x558fd6e8ef10 2024-04-25T10:09:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd880ff9640 1 -- 172.21.15.16:0/3336749609 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c03c690 con 0x558fd6e8ef10 2024-04-25T10:09:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.360+0000 7fd880ff9640 1 -- 172.21.15.16:0/3336749609 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c02d070 con 0x558fd6e8ef10 2024-04-25T10:09:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.361+0000 7fd8943e0740 1 -- 172.21.15.16:0/3336749609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6e8ef10 msgr2=0x558fd6ed5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.361+0000 7fd8943e0740 1 --2- 172.21.15.16:0/3336749609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6e8ef10 0x558fd6ed5190 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00e230 tx=0x7fd87c02d000 comp rx=0 tx=0).stop 2024-04-25T10:09:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.361+0000 7fd8943e0740 1 -- 172.21.15.16:0/3336749609 shutdown_connections 2024-04-25T10:09:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.361+0000 7fd8943e0740 1 --2- 172.21.15.16:0/3336749609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6e8ef10 0x558fd6ed5190 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.361+0000 7fd8943e0740 1 --2- 172.21.15.16:0/3336749609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6ed6ad0 0x558fd6ed7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.361+0000 7fd8943e0740 1 --2- 172.21.15.16:0/3336749609 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6eb9230 0x558fd6eb9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.361+0000 7fd8943e0740 1 -- 172.21.15.16:0/3336749609 >> 172.21.15.16:0/3336749609 conn(0x558fd6e8b840 msgr2=0x558fd6ed4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.361+0000 7fd8943e0740 1 -- 172.21.15.16:0/3336749609 shutdown_connections 2024-04-25T10:09:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.361+0000 7fd8943e0740 1 -- 172.21.15.16:0/3336749609 wait complete. 2024-04-25T10:09:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.362+0000 7fd8943e0740 1 -- start start 2024-04-25T10:09:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.362+0000 7fd8943e0740 1 Processor -- start 2024-04-25T10:09:27.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd8943e0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6eb9230 0x558fd6ee4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd8943e0740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6ed6ad0 0x558fd6ecc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd8943e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6eccec0 0x558fd6ecb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd8943e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd6e8fcb0 con 0x558fd6eccec0 2024-04-25T10:09:27.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd8943e0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd6dae0c0 con 0x558fd6eb9230 2024-04-25T10:09:27.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd8943e0740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x558fd6e89770 con 0x558fd6ed6ad0 2024-04-25T10:09:27.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd8943e0740 10 client.0 _ll_register_callbacks cb 0x558fd6da1ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:27.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd88b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6eccec0 0x558fd6ecb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd88b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6eccec0 0x558fd6ecb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52548/0 (socket says 172.21.15.16:52548) 2024-04-25T10:09:27.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd88b7fe640 1 -- 172.21.15.16:0/115045858 learned_addr learned my addr 172.21.15.16:0/115045858 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:27.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd88affd640 1 --2- 172.21.15.16:0/115045858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6eb9230 0x558fd6ee4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd88a7fc640 1 --2- 172.21.15.16:0/115045858 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6ed6ad0 0x558fd6ecc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd88b7fe640 1 -- 172.21.15.16:0/115045858 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6ed6ad0 msgr2=0x558fd6ecc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:27.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd88b7fe640 1 --2- 172.21.15.16:0/115045858 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6ed6ad0 0x558fd6ecc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd88b7fe640 1 -- 172.21.15.16:0/115045858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6eb9230 msgr2=0x558fd6ee4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:27.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd88b7fe640 1 --2- 172.21.15.16:0/115045858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6eb9230 0x558fd6ee4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd88b7fe640 1 -- 172.21.15.16:0/115045858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558fd6b062d0 con 0x558fd6eccec0 2024-04-25T10:09:27.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd88b7fe640 1 --2- 172.21.15.16:0/115045858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6eccec0 0x558fd6ecb8f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00e200 tx=0x7fd87c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88661c142de4b13c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd869ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:09:27.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd869ffb640 1 -- 172.21.15.16:0/115045858 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c040070 con 0x558fd6eccec0 2024-04-25T10:09:27.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd869ffb640 1 -- 172.21.15.16:0/115045858 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd87c03cc50 con 0x558fd6eccec0 2024-04-25T10:09:27.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.363+0000 7fd8943e0740 1 -- 172.21.15.16:0/115045858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd87c03c690 con 0x558fd6eccec0 2024-04-25T10:09:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.364+0000 7fd869ffb640 1 -- 172.21.15.16:0/115045858 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd87c055050 con 0x558fd6eccec0 2024-04-25T10:09:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.364+0000 7fd8943e0740 10 client.6783 fetch_fsmap learned FSMap version 36 2024-04-25T10:09:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.364+0000 7fd8943e0740 1 -- 172.21.15.16:0/115045858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x558fd6bd06b0 con 0x558fd6eccec0 2024-04-25T10:09:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.364+0000 7fd869ffb640 1 -- 172.21.15.16:0/115045858 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c03c690 con 0x558fd6eccec0 2024-04-25T10:09:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.364+0000 7fd869ffb640 1 -- 172.21.15.16:0/115045858 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd87c058020 con 0x558fd6eccec0 2024-04-25T10:09:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.364+0000 7fd8943e0740 10 client.6783 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T10:09:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.364+0000 7fd8943e0740 10 client.6783 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:09:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.364+0000 7fd8943e0740 20 client.6783 populate_metadata read hostname 'smithi016' 2024-04-25T10:09:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.364+0000 7fd8943e0740 1 --2- 172.21.15.16:0/115045858 >> [v2:172.21.15.186:6836/3292077067,v1:172.21.15.186:6837/3292077067] conn(0x558fd6eb9f20 0x558fd6ee7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.364+0000 7fd8943e0740 4 client.6783 mds_command: new command op to 35489 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.364+0000 7fd8943e0740 1 -- 172.21.15.16:0/115045858 --> [v2:172.21.15.186:6836/3292077067,v1:172.21.15.186:6837/3292077067] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fd6e50920 con 0x558fd6eb9f20 2024-04-25T10:09:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.364+0000 7fd88affd640 1 --2- 172.21.15.16:0/115045858 >> [v2:172.21.15.186:6836/3292077067,v1:172.21.15.186:6837/3292077067] conn(0x558fd6eb9f20 0x558fd6ee7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.365+0000 7fd869ffb640 10 client.6783 ms_handle_connect on v2:172.21.15.186:6836/3292077067 2024-04-25T10:09:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.365+0000 7fd88affd640 1 --2- 172.21.15.16:0/115045858 >> [v2:172.21.15.186:6836/3292077067,v1:172.21.15.186:6837/3292077067] conn(0x558fd6eb9f20 0x558fd6ee7c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68bf74653fe907e2 server_cookie=673cd8ce7a8a0955 in_seq=0 out_seq=0 2024-04-25T10:09:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.365+0000 7fd869ffb640 1 -- 172.21.15.16:0/115045858 <== mds.0 v2:172.21.15.186:6836/3292077067 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558fd6e50920 con 0x558fd6eb9f20 2024-04-25T10:09:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.365+0000 7fd869ffb640 10 client.6783 handle_command_reply: tid=0 2024-04-25T10:09:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.365+0000 7fd8943e0740 1 client.6783 shutdown 2024-04-25T10:09:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.365+0000 7fd8943e0740 1 -- 172.21.15.16:0/115045858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6eccec0 msgr2=0x558fd6ecb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.365+0000 7fd8943e0740 1 --2- 172.21.15.16:0/115045858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6eccec0 0x558fd6ecb8f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00e200 tx=0x7fd87c004ab0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.365+0000 7fd8943e0740 1 -- 172.21.15.16:0/115045858 shutdown_connections 2024-04-25T10:09:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.365+0000 7fd8943e0740 1 --2- 172.21.15.16:0/115045858 >> [v2:172.21.15.186:6836/3292077067,v1:172.21.15.186:6837/3292077067] conn(0x558fd6eb9f20 0x558fd6ee7c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.365+0000 7fd8943e0740 1 --2- 172.21.15.16:0/115045858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6eccec0 0x558fd6ecb8f0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.365+0000 7fd8943e0740 1 --2- 172.21.15.16:0/115045858 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6ed6ad0 0x558fd6ecc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.365+0000 7fd869ffb640 0 client.6783 ms_handle_reset on v2:172.21.15.186:6836/3292077067 2024-04-25T10:09:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.365+0000 7fd8943e0740 1 --2- 172.21.15.16:0/115045858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6eb9230 0x558fd6ee4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.365+0000 7fd8943e0740 1 -- 172.21.15.16:0/115045858 >> 172.21.15.16:0/115045858 conn(0x558fd6e8ef10 msgr2=0x558fd6ed7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.365+0000 7fd8943e0740 1 -- 172.21.15.16:0/115045858 shutdown_connections 2024-04-25T10:09:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.365+0000 7fd8943e0740 1 -- 172.21.15.16:0/115045858 wait complete. 2024-04-25T10:09:27.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.366+0000 7fd8943e0740 20 client.6783 trim_cache size 0 max 16384 2024-04-25T10:09:27.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.381+0000 7fd8943e0740 1 Processor -- start 2024-04-25T10:09:27.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.381+0000 7fd8943e0740 1 -- start start 2024-04-25T10:09:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd8943e0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6eb9230 0x558fd6eede70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd8943e0740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6eccec0 0x558fd6e88510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd8943e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6ed6ad0 0x558fd6e88a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd8943e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c1ff530 con 0x558fd6ed6ad0 2024-04-25T10:09:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd8943e0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c0c7e80 con 0x558fd6eb9230 2024-04-25T10:09:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd8943e0740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c1ff0b0 con 0x558fd6eccec0 2024-04-25T10:09:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd88b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6ed6ad0 0x558fd6e88a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd88b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6ed6ad0 0x558fd6e88a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52550/0 (socket says 172.21.15.16:52550) 2024-04-25T10:09:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd88b7fe640 1 -- 172.21.15.16:0/97153925 learned_addr learned my addr 172.21.15.16:0/97153925 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd88b7fe640 1 -- 172.21.15.16:0/97153925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6eccec0 msgr2=0x558fd6e88510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd88a7fc640 1 --2- 172.21.15.16:0/97153925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6eccec0 0x558fd6e88510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd88affd640 1 --2- 172.21.15.16:0/97153925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6eb9230 0x558fd6eede70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd88b7fe640 1 --2- 172.21.15.16:0/97153925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6eccec0 0x558fd6e88510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd88b7fe640 1 -- 172.21.15.16:0/97153925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6eb9230 msgr2=0x558fd6eede70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd88b7fe640 1 --2- 172.21.15.16:0/97153925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6eb9230 0x558fd6eede70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd88b7fe640 1 -- 172.21.15.16:0/97153925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fd6ed6eb0 con 0x558fd6ed6ad0 2024-04-25T10:09:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd88a7fc640 1 --2- 172.21.15.16:0/97153925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6eccec0 0x558fd6e88510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd88affd640 1 --2- 172.21.15.16:0/97153925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6eb9230 0x558fd6eede70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.382+0000 7fd88b7fe640 1 --2- 172.21.15.16:0/97153925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6ed6ad0 0x558fd6e88a50 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fd87c02ae40 tx=0x7fd87c047040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1c9966c2c77f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.383+0000 7fd86bfff640 1 -- 172.21.15.16:0/97153925 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c040070 con 0x558fd6ed6ad0 2024-04-25T10:09:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.383+0000 7fd86bfff640 1 -- 172.21.15.16:0/97153925 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c04d070 con 0x558fd6ed6ad0 2024-04-25T10:09:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.383+0000 7fd86bfff640 1 -- 172.21.15.16:0/97153925 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c031470 con 0x558fd6ed6ad0 2024-04-25T10:09:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.383+0000 7fd8943e0740 1 -- 172.21.15.16:0/97153925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6ed6ad0 msgr2=0x558fd6e88a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.383+0000 7fd8943e0740 1 --2- 172.21.15.16:0/97153925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6ed6ad0 0x558fd6e88a50 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fd87c02ae40 tx=0x7fd87c047040 comp rx=0 tx=0).stop 2024-04-25T10:09:27.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.383+0000 7fd8943e0740 1 -- 172.21.15.16:0/97153925 shutdown_connections 2024-04-25T10:09:27.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.383+0000 7fd8943e0740 1 --2- 172.21.15.16:0/97153925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6ed6ad0 0x558fd6e88a50 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.383+0000 7fd8943e0740 1 --2- 172.21.15.16:0/97153925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6eccec0 0x558fd6e88510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.383+0000 7fd8943e0740 1 --2- 172.21.15.16:0/97153925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6eb9230 0x558fd6eede70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.383+0000 7fd8943e0740 1 -- 172.21.15.16:0/97153925 >> 172.21.15.16:0/97153925 conn(0x558fd6e8ef10 msgr2=0x558fd6ecdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:27.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.383+0000 7fd8943e0740 1 -- 172.21.15.16:0/97153925 shutdown_connections 2024-04-25T10:09:27.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.383+0000 7fd8943e0740 1 -- 172.21.15.16:0/97153925 wait complete. 2024-04-25T10:09:27.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.384+0000 7fd8943e0740 1 -- start start 2024-04-25T10:09:27.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.384+0000 7fd8943e0740 1 Processor -- start 2024-04-25T10:09:27.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.384+0000 7fd8943e0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6eb9230 0x558fd6e8cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.384+0000 7fd8943e0740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6eccec0 0x558fd6e8df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.384+0000 7fd8943e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6ed6ad0 0x558fd6ee3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.384+0000 7fd8943e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd6e8fcb0 con 0x558fd6ed6ad0 2024-04-25T10:09:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.384+0000 7fd8943e0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd6dae0c0 con 0x558fd6eb9230 2024-04-25T10:09:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.384+0000 7fd8943e0740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x558fd6eba330 con 0x558fd6eccec0 2024-04-25T10:09:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd8943e0740 10 client.0 _ll_register_callbacks cb 0x558fd6e89c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd88b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6ed6ad0 0x558fd6ee3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd88b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6ed6ad0 0x558fd6ee3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52554/0 (socket says 172.21.15.16:52554) 2024-04-25T10:09:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd88b7fe640 1 -- 172.21.15.16:0/576610443 learned_addr learned my addr 172.21.15.16:0/576610443 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd88affd640 1 --2- 172.21.15.16:0/576610443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6eb9230 0x558fd6e8cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd88a7fc640 1 --2- 172.21.15.16:0/576610443 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6eccec0 0x558fd6e8df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd88b7fe640 1 -- 172.21.15.16:0/576610443 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6eccec0 msgr2=0x558fd6e8df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd88b7fe640 1 --2- 172.21.15.16:0/576610443 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6eccec0 0x558fd6e8df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd88b7fe640 1 -- 172.21.15.16:0/576610443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6eb9230 msgr2=0x558fd6e8cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd88b7fe640 1 --2- 172.21.15.16:0/576610443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6eb9230 0x558fd6e8cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd88b7fe640 1 -- 172.21.15.16:0/576610443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558fd6ed7540 con 0x558fd6ed6ad0 2024-04-25T10:09:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd88b7fe640 1 --2- 172.21.15.16:0/576610443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6ed6ad0 0x558fd6ee3a30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd87c02af10 tx=0x7fd87c02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ce1890b2286187 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd869ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:09:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd869ffb640 1 -- 172.21.15.16:0/576610443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c040070 con 0x558fd6ed6ad0 2024-04-25T10:09:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd869ffb640 1 -- 172.21.15.16:0/576610443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd87c0319c0 con 0x558fd6ed6ad0 2024-04-25T10:09:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd8943e0740 1 -- 172.21.15.16:0/576610443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd87c04d070 con 0x558fd6ed6ad0 2024-04-25T10:09:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd869ffb640 1 -- 172.21.15.16:0/576610443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd87c064050 con 0x558fd6ed6ad0 2024-04-25T10:09:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd8943e0740 10 client.6789 fetch_fsmap learned FSMap version 36 2024-04-25T10:09:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.385+0000 7fd8943e0740 1 -- 172.21.15.16:0/576610443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x558fd6ecba90 con 0x558fd6ed6ad0 2024-04-25T10:09:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.386+0000 7fd869ffb640 1 -- 172.21.15.16:0/576610443 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c04d070 con 0x558fd6ed6ad0 2024-04-25T10:09:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.386+0000 7fd869ffb640 1 -- 172.21.15.16:0/576610443 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd87c067020 con 0x558fd6ed6ad0 2024-04-25T10:09:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.386+0000 7fd8943e0740 10 client.6789 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T10:09:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.386+0000 7fd8943e0740 10 client.6789 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:09:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.386+0000 7fd8943e0740 20 client.6789 populate_metadata read hostname 'smithi016' 2024-04-25T10:09:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.386+0000 7fd8943e0740 1 --2- 172.21.15.16:0/576610443 >> [v2:172.21.15.186:6836/3292077067,v1:172.21.15.186:6837/3292077067] conn(0x558fd6ed7f30 0x558fd6ee3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.386+0000 7fd8943e0740 4 client.6789 mds_command: new command op to 35489 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:09:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.386+0000 7fd8943e0740 1 -- 172.21.15.16:0/576610443 --> [v2:172.21.15.186:6836/3292077067,v1:172.21.15.186:6837/3292077067] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558fd6dfef50 con 0x558fd6ed7f30 2024-04-25T10:09:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.386+0000 7fd88affd640 1 --2- 172.21.15.16:0/576610443 >> [v2:172.21.15.186:6836/3292077067,v1:172.21.15.186:6837/3292077067] conn(0x558fd6ed7f30 0x558fd6ee3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.386+0000 7fd869ffb640 10 client.6789 ms_handle_connect on v2:172.21.15.186:6836/3292077067 2024-04-25T10:09:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.386+0000 7fd88affd640 1 --2- 172.21.15.16:0/576610443 >> [v2:172.21.15.186:6836/3292077067,v1:172.21.15.186:6837/3292077067] conn(0x558fd6ed7f30 0x558fd6ee3f70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20f91558fdf06fb server_cookie=4fa8fc2a7fe3fc18 in_seq=0 out_seq=0 2024-04-25T10:09:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.386+0000 7fd869ffb640 1 -- 172.21.15.16:0/576610443 <== mds.0 v2:172.21.15.186:6836/3292077067 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558fd6dfef50 con 0x558fd6ed7f30 2024-04-25T10:09:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.386+0000 7fd869ffb640 10 client.6789 handle_command_reply: tid=0 2024-04-25T10:09:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.386+0000 7fd8943e0740 1 client.6789 shutdown 2024-04-25T10:09:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd8943e0740 1 -- 172.21.15.16:0/576610443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6ed6ad0 msgr2=0x558fd6ee3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd8943e0740 1 --2- 172.21.15.16:0/576610443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6ed6ad0 0x558fd6ee3a30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd87c02af10 tx=0x7fd87c02f000 comp rx=0 tx=0).stop 2024-04-25T10:09:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd8943e0740 1 -- 172.21.15.16:0/576610443 shutdown_connections 2024-04-25T10:09:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd8943e0740 1 --2- 172.21.15.16:0/576610443 >> [v2:172.21.15.186:6836/3292077067,v1:172.21.15.186:6837/3292077067] conn(0x558fd6ed7f30 0x558fd6ee3f70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd8943e0740 1 --2- 172.21.15.16:0/576610443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd6ed6ad0 0x558fd6ee3a30 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd8943e0740 1 --2- 172.21.15.16:0/576610443 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558fd6eccec0 0x558fd6e8df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd869ffb640 0 client.6789 ms_handle_reset on v2:172.21.15.186:6836/3292077067 2024-04-25T10:09:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd8943e0740 1 --2- 172.21.15.16:0/576610443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd6eb9230 0x558fd6e8cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd8943e0740 1 -- 172.21.15.16:0/576610443 >> 172.21.15.16:0/576610443 conn(0x558fd6e8ef10 msgr2=0x558fd6e90660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd8943e0740 1 -- 172.21.15.16:0/576610443 shutdown_connections 2024-04-25T10:09:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd8943e0740 1 -- 172.21.15.16:0/576610443 wait complete. 2024-04-25T10:09:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd8943e0740 20 client.6789 trim_cache size 0 max 16384 2024-04-25T10:09:27.400 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:09:27.400 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "34bc5eca-3a21-42a6-a26f-07da0905e8bc", 2024-04-25T10:09:27.400 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-04-25T10:09:27.400 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 35489, 2024-04-25T10:09:27.400 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-04-25T10:09:27.401 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-04-25T10:09:27.401 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-04-25T10:09:27.401 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 4.6746811490000004, 2024-04-25T10:09:27.401 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 36, 2024-04-25T10:09:27.401 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 72, 2024-04-25T10:09:27.401 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 72, 2024-04-25T10:09:27.401 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 9.6874308550000006 2024-04-25T10:09:27.401 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:09:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd891609640 1 -- 172.21.15.16:0/3935893844 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd86c06b4d0 msgr2=0x7fd86c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd891609640 1 --2- 172.21.15.16:0/3935893844 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd86c06b4d0 0x7fd86c06d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd874000c00 tx=0x7fd874009040 comp rx=0 tx=0).stop 2024-04-25T10:09:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd891609640 1 -- 172.21.15.16:0/3935893844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd878005210 msgr2=0x7fd88c15d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd891609640 1 --2- 172.21.15.16:0/3935893844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd878005210 0x7fd88c15d7e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd884004f30 tx=0x7fd8840074f0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd891609640 1 -- 172.21.15.16:0/3935893844 shutdown_connections 2024-04-25T10:09:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd891609640 1 --2- 172.21.15.16:0/3935893844 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd86c06b4d0 0x7fd86c06d990 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd891609640 1 --2- 172.21.15.16:0/3935893844 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd88c0cd380 0x7fd88c15a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd891609640 1 --2- 172.21.15.16:0/3935893844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c14b300 0x7fd88c0cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd891609640 1 --2- 172.21.15.16:0/3935893844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd878005210 0x7fd88c15d7e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd891609640 1 -- 172.21.15.16:0/3935893844 >> 172.21.15.16:0/3935893844 conn(0x7fd88c0b55e0 msgr2=0x7fd88c0b8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd891609640 1 -- 172.21.15.16:0/3935893844 shutdown_connections 2024-04-25T10:09:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.387+0000 7fd891609640 1 -- 172.21.15.16:0/3935893844 wait complete. 2024-04-25T10:09:27.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:09:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.498+0000 7f3eee95c640 1 Processor -- start 2024-04-25T10:09:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.498+0000 7f3eee95c640 1 -- start start 2024-04-25T10:09:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.499+0000 7f3eee95c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3ee815a870 0x7f3ee80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.499+0000 7f3eee95c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ee80cfc40 0x7f3ee815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.499+0000 7f3eee95c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee80cdb10 con 0x7f3ee806e2e0 2024-04-25T10:09:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.499+0000 7f3eee95c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee80ce720 con 0x7f3ee815a870 2024-04-25T10:09:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.499+0000 7f3eee95c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80cddf0 con 0x7f3ee80cfc40 2024-04-25T10:09:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.499+0000 7f3ee7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3ee815a870 0x7f3ee80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.499+0000 7f3ee7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3ee815a870 0x7f3ee80cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52564/0 (socket says 172.21.15.16:52564) 2024-04-25T10:09:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.499+0000 7f3ee7fff640 1 -- 172.21.15.16:0/2686216625 learned_addr learned my addr 172.21.15.16:0/2686216625 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.499+0000 7f3eecef9640 1 --2- 172.21.15.16:0/2686216625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ee80cfc40 0x7f3ee815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.499+0000 7f3ee7fff640 1 -- 172.21.15.16:0/2686216625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ee80cfc40 msgr2=0x7f3ee815ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.499+0000 7f3ee7fff640 1 --2- 172.21.15.16:0/2686216625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ee80cfc40 0x7f3ee815ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.499+0000 7f3ee7fff640 1 -- 172.21.15.16:0/2686216625 >> v1:172.21.15.186:6789/0 conn(0x7f3ee806e2e0 legacy=0x7f3ee80cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.499+0000 7f3ee7fff640 1 -- 172.21.15.16:0/2686216625 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee80ce0d0 con 0x7f3ee815a870 2024-04-25T10:09:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.499+0000 7f3eecef9640 1 --2- 172.21.15.16:0/2686216625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ee80cfc40 0x7f3ee815ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.500+0000 7f3ee7fff640 1 --2- 172.21.15.16:0/2686216625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ee815a870 0x7f3ee80cebb0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0009870 tx=0x7f3ed002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cc13b879166e1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.500+0000 7f3ee6ffd640 1 -- 172.21.15.16:0/2686216625 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0037070 con 0x7f3ee815a870 2024-04-25T10:09:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.500+0000 7f3ee6ffd640 1 -- 172.21.15.16:0/2686216625 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed0032440 con 0x7f3ee815a870 2024-04-25T10:09:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.500+0000 7f3ee6ffd640 1 -- 172.21.15.16:0/2686216625 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0032810 con 0x7f3ee815a870 2024-04-25T10:09:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.501+0000 7f3eee95c640 1 -- 172.21.15.16:0/2686216625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ee815a870 msgr2=0x7f3ee80cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.501+0000 7f3eee95c640 1 --2- 172.21.15.16:0/2686216625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ee815a870 0x7f3ee80cebb0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0009870 tx=0x7f3ed002a270 comp rx=0 tx=0).stop 2024-04-25T10:09:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.501+0000 7f3eee95c640 1 -- 172.21.15.16:0/2686216625 shutdown_connections 2024-04-25T10:09:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.501+0000 7f3eee95c640 1 --2- 172.21.15.16:0/2686216625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ee80cfc40 0x7f3ee815ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.501+0000 7f3eee95c640 1 --2- 172.21.15.16:0/2686216625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ee815a870 0x7f3ee80cebb0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.501+0000 7f3eee95c640 1 -- 172.21.15.16:0/2686216625 >> 172.21.15.16:0/2686216625 conn(0x7f3ee806aba0 msgr2=0x7f3ee806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.501+0000 7f3eee95c640 1 -- 172.21.15.16:0/2686216625 shutdown_connections 2024-04-25T10:09:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.501+0000 7f3eee95c640 1 -- 172.21.15.16:0/2686216625 wait complete. 2024-04-25T10:09:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.502+0000 7f3eee95c640 1 Processor -- start 2024-04-25T10:09:27.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.502+0000 7f3eee95c640 1 -- start start 2024-04-25T10:09:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.503+0000 7f3eee95c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ee806e2e0 0x7f3ee8150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.503+0000 7f3eee95c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ee80cfc40 0x7f3ee813c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.503+0000 7f3eee95c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ee815a870 0x7f3ee80b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.503+0000 7f3eee95c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80cd700 con 0x7f3ee815a870 2024-04-25T10:09:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.503+0000 7f3eee95c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80cd580 con 0x7f3ee806e2e0 2024-04-25T10:09:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.503+0000 7f3eee95c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80cd880 con 0x7f3ee80cfc40 2024-04-25T10:09:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.503+0000 7f3eecef9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ee815a870 0x7f3ee80b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.503+0000 7f3eecef9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ee815a870 0x7f3ee80b34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52566/0 (socket says 172.21.15.16:52566) 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.503+0000 7f3eecef9640 1 -- 172.21.15.16:0/2285391973 learned_addr learned my addr 172.21.15.16:0/2285391973 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.503+0000 7f3ee7fff640 1 --2- 172.21.15.16:0/2285391973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ee806e2e0 0x7f3ee8150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.503+0000 7f3ee77fe640 1 --2- 172.21.15.16:0/2285391973 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ee80cfc40 0x7f3ee813c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.504+0000 7f3eecef9640 1 -- 172.21.15.16:0/2285391973 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ee80cfc40 msgr2=0x7f3ee813c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.504+0000 7f3eecef9640 1 --2- 172.21.15.16:0/2285391973 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ee80cfc40 0x7f3ee813c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.504+0000 7f3eecef9640 1 -- 172.21.15.16:0/2285391973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ee806e2e0 msgr2=0x7f3ee8150b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.504+0000 7f3eecef9640 1 --2- 172.21.15.16:0/2285391973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ee806e2e0 0x7f3ee8150b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.504+0000 7f3eecef9640 1 -- 172.21.15.16:0/2285391973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee813cb30 con 0x7f3ee815a870 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.504+0000 7f3eecef9640 1 --2- 172.21.15.16:0/2285391973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ee815a870 0x7f3ee80b34e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009f70 tx=0x7f3ee000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.504+0000 7f3ee4ff9640 1 -- 172.21.15.16:0/2285391973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee001a070 con 0x7f3ee815a870 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.504+0000 7f3ee4ff9640 1 -- 172.21.15.16:0/2285391973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee0003bb0 con 0x7f3ee815a870 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.504+0000 7f3eee95c640 1 -- 172.21.15.16:0/2285391973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee81517d0 con 0x7f3ee815a870 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.504+0000 7f3ee4ff9640 1 -- 172.21.15.16:0/2285391973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0004030 con 0x7f3ee815a870 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.504+0000 7f3eee95c640 1 -- 172.21.15.16:0/2285391973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee8151d90 con 0x7f3ee815a870 2024-04-25T10:09:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.505+0000 7f3eee95c640 1 -- 172.21.15.16:0/2285391973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee80749b0 con 0x7f3ee815a870 2024-04-25T10:09:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.505+0000 7f3ee4ff9640 1 -- 172.21.15.16:0/2285391973 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ee001d070 con 0x7f3ee815a870 2024-04-25T10:09:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.506+0000 7f3ee4ff9640 1 --2- 172.21.15.16:0/2285391973 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3ec806b4d0 0x7f3ec806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.506+0000 7f3ee4ff9640 1 -- 172.21.15.16:0/2285391973 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ee008a4a0 con 0x7f3ee815a870 2024-04-25T10:09:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.506+0000 7f3ee7fff640 1 --2- 172.21.15.16:0/2285391973 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3ec806b4d0 0x7f3ec806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.507+0000 7f3ee7fff640 1 --2- 172.21.15.16:0/2285391973 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3ec806b4d0 0x7f3ec806d990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0034000 tx=0x7f3ed002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.510+0000 7f3ee4ff9640 1 -- 172.21.15.16:0/2285391973 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ee0010070 con 0x7f3ee815a870 2024-04-25T10:09:27.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.728+0000 7f3eee95c640 1 -- 172.21.15.16:0/2285391973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3ee80cec50 con 0x7f3ee815a870 2024-04-25T10:09:27.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.729+0000 7f3ee4ff9640 1 -- 172.21.15.16:0/2285391973 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3ee0059320 con 0x7f3ee815a870 2024-04-25T10:09:27.735 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:09:27.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.731+0000 7f3eee95c640 1 -- 172.21.15.16:0/2285391973 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3ec806b4d0 msgr2=0x7f3ec806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.731+0000 7f3eee95c640 1 --2- 172.21.15.16:0/2285391973 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3ec806b4d0 0x7f3ec806d990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0034000 tx=0x7f3ed002e000 comp rx=0 tx=0).stop 2024-04-25T10:09:27.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.731+0000 7f3eee95c640 1 -- 172.21.15.16:0/2285391973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ee815a870 msgr2=0x7f3ee80b34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.731+0000 7f3eee95c640 1 --2- 172.21.15.16:0/2285391973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ee815a870 0x7f3ee80b34e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009f70 tx=0x7f3ee000d560 comp rx=0 tx=0).stop 2024-04-25T10:09:27.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.731+0000 7f3eee95c640 1 -- 172.21.15.16:0/2285391973 shutdown_connections 2024-04-25T10:09:27.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.731+0000 7f3eee95c640 1 --2- 172.21.15.16:0/2285391973 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3ec806b4d0 0x7f3ec806d990 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.731+0000 7f3eee95c640 1 --2- 172.21.15.16:0/2285391973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ee815a870 0x7f3ee80b34e0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.731+0000 7f3eee95c640 1 --2- 172.21.15.16:0/2285391973 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ee80cfc40 0x7f3ee813c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.731+0000 7f3eee95c640 1 --2- 172.21.15.16:0/2285391973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ee806e2e0 0x7f3ee8150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.731+0000 7f3eee95c640 1 -- 172.21.15.16:0/2285391973 >> 172.21.15.16:0/2285391973 conn(0x7f3ee806aba0 msgr2=0x7f3ee80af150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:27.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.731+0000 7f3eee95c640 1 -- 172.21.15.16:0/2285391973 shutdown_connections 2024-04-25T10:09:27.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:27.731+0000 7f3eee95c640 1 -- 172.21.15.16:0/2285391973 wait complete. 2024-04-25T10:09:27.748 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:09:27.748 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:09:27.748 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-04-25T10:09:27.748 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:09:27.748 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:27.748 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:09:27.748 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:09:27.748 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:09:27.748 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:27.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:27.749 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-04-25T10:09:27.764 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:09:27.764 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:09:27.764 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:09:27.764 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:27.764 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:09:27.765 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:27.765 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:09:27.765 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:27.765 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-04-25T10:09:27.765 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:09:27.765 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:27.765 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-04-25T10:09:27.765 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:09:27.765 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:27.765 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-04-25T10:09:27.765 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:09:27.765 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42393sec preferred_lft 42393sec 2024-04-25T10:09:27.765 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-25T10:09:27.765 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:27.766 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:09:27.766 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:27.766 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-04-25T10:09:27.766 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:09:27.766 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:27.766 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:09:27.766 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:27.766 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f06b:fdff:fed4:a335/64 scope link 2024-04-25T10:09:27.766 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:27.766 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:09:27.766 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:27.766 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::90ad:4fff:fe91:3c33/64 scope link 2024-04-25T10:09:27.766 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:27.767 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:09:27.767 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether be:14:4a:3c:16:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:27.767 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::bc14:4aff:fe3c:1605/64 scope link 2024-04-25T10:09:27.767 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:27.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:27.767 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:09:27.827 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:09:27.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:27.829 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-04-25T10:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-04-25T10:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:09:27.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:27.887 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:09:27.887 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:09:27.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:09:27.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:09:27.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:09:27.887 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:09:27.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:27.971 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:09:27.986 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:09:27.986 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:09:27.986 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:27.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:27.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-04-25T10:09:28.051 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:09:28.051 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:28.107 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:28.107 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:09:28.107 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:28.161 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:09:28.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:28.161 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:28.244 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:28.244 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:09:28.244 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:09:28.252 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:28.252 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:09:28.252 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:09:28.252 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:09:28.253 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:09:28.257 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:28.257 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:09:28.258 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:09:28.258 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:09:28.258 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:09:28.258 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:09:28.258 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:09:28.258 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:09:28.258 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:09:28.258 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:09:28.258 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:09:28.258 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:09:28.259 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:09:28.260 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:09:28.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:28.260 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:28.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:28.326 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:09:28.389 INFO:teuthology.orchestra.run.smithi016.stdout:client6145 2024-04-25T10:09:28.390 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:09:28.390 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:28.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:28.446 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-04-25T10:09:28.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:28.506 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-04-25T10:09:28.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:28.563 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-04-25T10:09:28.618 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:09:28.618 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:09:28.618 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:09:28.618 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:09:28.619 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-04-25T10:09:28.619 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:09:28.726 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627776 2024-04-25T10:09:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:09:28.734 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:09:28.734 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:28.790 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:09:28.790 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:28.845 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:28.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.979+0000 7f92181ed640 1 Processor -- start 2024-04-25T10:09:28.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.979+0000 7f92181ed640 1 -- start start 2024-04-25T10:09:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.980+0000 7f92181ed640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f921015bf00 0x7f92100cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.980+0000 7f92181ed640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f92100cc310 0x7f921015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.980+0000 7f92181ed640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92100ce6c0 con 0x7f921015c820 2024-04-25T10:09:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.980+0000 7f92181ed640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92101455e0 con 0x7f921015bf00 2024-04-25T10:09:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.980+0000 7f92181ed640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f921006d450 con 0x7f92100cc310 2024-04-25T10:09:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.980+0000 7f9215f89640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f921015bf00 0x7f92100cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.980+0000 7f921678a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f921015c820 0x7f921006d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50462/0 (socket says 172.21.15.16:50462) 2024-04-25T10:09:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.980+0000 7f921678a640 1 -- 172.21.15.16:0/4091915115 learned_addr learned my addr 172.21.15.16:0/4091915115 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.980+0000 7f9215788640 1 --2- 172.21.15.16:0/4091915115 >> v2:172.21.15.186:3300/0 conn(0x7f92100cc310 0x7f921015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.981+0000 7f9215f89640 1 -- 172.21.15.16:0/4091915115 >> v2:172.21.15.186:3300/0 conn(0x7f92100cc310 msgr2=0x7f921015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.981+0000 7f9215f89640 1 --2- 172.21.15.16:0/4091915115 >> v2:172.21.15.186:3300/0 conn(0x7f92100cc310 0x7f921015c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.981+0000 7f9215f89640 1 -- 172.21.15.16:0/4091915115 >> v1:172.21.15.16:6789/0 conn(0x7f921015c820 legacy=0x7f921006d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.981+0000 7f9215f89640 1 -- 172.21.15.16:0/4091915115 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92100cc7b0 con 0x7f921015bf00 2024-04-25T10:09:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.981+0000 7f9214f87640 1 -- 172.21.15.16:0/4091915115 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064724323 0 0) 0x7f920c001490 con 0x7f921015c820 2024-04-25T10:09:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.981+0000 7f9215f89640 1 --2- 172.21.15.16:0/4091915115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921015bf00 0x7f92100cbdd0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f921006b900 tx=0x7f920802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c77a78495786f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.981+0000 7f9214f87640 1 -- 172.21.15.16:0/4091915115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208037070 con 0x7f921015bf00 2024-04-25T10:09:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.981+0000 7f9214f87640 1 -- 172.21.15.16:0/4091915115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9208032440 con 0x7f921015bf00 2024-04-25T10:09:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.982+0000 7f9214f87640 1 -- 172.21.15.16:0/4091915115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208032810 con 0x7f921015bf00 2024-04-25T10:09:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.982+0000 7f92181ed640 1 -- 172.21.15.16:0/4091915115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921015bf00 msgr2=0x7f92100cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.982+0000 7f92181ed640 1 --2- 172.21.15.16:0/4091915115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921015bf00 0x7f92100cbdd0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f921006b900 tx=0x7f920802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.982+0000 7f92181ed640 1 -- 172.21.15.16:0/4091915115 shutdown_connections 2024-04-25T10:09:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.982+0000 7f92181ed640 1 --2- 172.21.15.16:0/4091915115 >> v2:172.21.15.186:3300/0 conn(0x7f92100cc310 0x7f921015c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.982+0000 7f92181ed640 1 --2- 172.21.15.16:0/4091915115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921015bf00 0x7f92100cbdd0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.982+0000 7f92181ed640 1 -- 172.21.15.16:0/4091915115 >> 172.21.15.16:0/4091915115 conn(0x7f921006aba0 msgr2=0x7f921006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.982+0000 7f92181ed640 1 -- 172.21.15.16:0/4091915115 shutdown_connections 2024-04-25T10:09:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.982+0000 7f92181ed640 1 -- 172.21.15.16:0/4091915115 wait complete. 2024-04-25T10:09:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.983+0000 7f92181ed640 1 Processor -- start 2024-04-25T10:09:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.983+0000 7f92181ed640 1 -- start start 2024-04-25T10:09:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.984+0000 7f92181ed640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92100cc310 0x7f921015a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.984+0000 7f92181ed640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f921015bf00 0x7f921013f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.984+0000 7f92181ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921015c820 0x7f921013feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.984+0000 7f92181ed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92100cd050 con 0x7f921015c820 2024-04-25T10:09:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.984+0000 7f92181ed640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92100ccd70 con 0x7f921015bf00 2024-04-25T10:09:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.984+0000 7f92181ed640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921015ccd0 con 0x7f92100cc310 2024-04-25T10:09:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.984+0000 7f921678a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921015c820 0x7f921013feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.984+0000 7f921678a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921015c820 0x7f921013feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52598/0 (socket says 172.21.15.16:52598) 2024-04-25T10:09:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.984+0000 7f921678a640 1 -- 172.21.15.16:0/1138642244 learned_addr learned my addr 172.21.15.16:0/1138642244 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.984+0000 7f9215788640 1 --2- 172.21.15.16:0/1138642244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f921015bf00 0x7f921013f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.984+0000 7f9215f89640 1 --2- 172.21.15.16:0/1138642244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92100cc310 0x7f921015a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.985+0000 7f921678a640 1 -- 172.21.15.16:0/1138642244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92100cc310 msgr2=0x7f921015a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.985+0000 7f921678a640 1 --2- 172.21.15.16:0/1138642244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92100cc310 0x7f921015a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.985+0000 7f921678a640 1 -- 172.21.15.16:0/1138642244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f921015bf00 msgr2=0x7f921013f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.985+0000 7f921678a640 1 --2- 172.21.15.16:0/1138642244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f921015bf00 0x7f921013f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.985+0000 7f921678a640 1 -- 172.21.15.16:0/1138642244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92101403f0 con 0x7f921015c820 2024-04-25T10:09:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.985+0000 7f9215f89640 1 --2- 172.21.15.16:0/1138642244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92100cc310 0x7f921015a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.985+0000 7f921678a640 1 --2- 172.21.15.16:0/1138642244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921015c820 0x7f921013feb0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f920c00c730 tx=0x7f920c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.985+0000 7f92067fc640 1 -- 172.21.15.16:0/1138642244 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c018070 con 0x7f921015c820 2024-04-25T10:09:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.985+0000 7f92067fc640 1 -- 172.21.15.16:0/1138642244 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920c00ce40 con 0x7f921015c820 2024-04-25T10:09:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.985+0000 7f92181ed640 1 -- 172.21.15.16:0/1138642244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92101406a0 con 0x7f921015c820 2024-04-25T10:09:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.985+0000 7f92067fc640 1 -- 172.21.15.16:0/1138642244 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c00a860 con 0x7f921015c820 2024-04-25T10:09:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.985+0000 7f92181ed640 1 -- 172.21.15.16:0/1138642244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9210140c60 con 0x7f921015c820 2024-04-25T10:09:28.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.986+0000 7f92181ed640 1 -- 172.21.15.16:0/1138642244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f921006bb30 con 0x7f921015c820 2024-04-25T10:09:28.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.986+0000 7f92067fc640 1 -- 172.21.15.16:0/1138642244 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f920c01b070 con 0x7f921015c820 2024-04-25T10:09:28.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.987+0000 7f92067fc640 1 --2- 172.21.15.16:0/1138642244 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f91f406b4d0 0x7f91f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:28.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.987+0000 7f92067fc640 1 -- 172.21.15.16:0/1138642244 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f920c089720 con 0x7f921015c820 2024-04-25T10:09:28.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.987+0000 7f9215f89640 1 --2- 172.21.15.16:0/1138642244 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f91f406b4d0 0x7f91f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:28.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.988+0000 7f9215f89640 1 --2- 172.21.15.16:0/1138642244 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f91f406b4d0 0x7f91f406d990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9208034000 tx=0x7f920802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:28.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:28.991+0000 7f92067fc640 1 -- 172.21.15.16:0/1138642244 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f92100ce6c0 con 0x7f921015c820 2024-04-25T10:09:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.171+0000 7f92181ed640 1 -- 172.21.15.16:0/1138642244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f921006b300 con 0x7f921015c820 2024-04-25T10:09:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.172+0000 7f92067fc640 1 -- 172.21.15.16:0/1138642244 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16608 (secure 0 0 0) 0x7f920c0585a0 con 0x7f921015c820 2024-04-25T10:09:29.178 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:29.179 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":72,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:09:23.718608+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:20.361009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:21.048892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:29.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.174+0000 7f92181ed640 1 -- 172.21.15.16:0/1138642244 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f91f406b4d0 msgr2=0x7f91f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.174+0000 7f92181ed640 1 --2- 172.21.15.16:0/1138642244 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f91f406b4d0 0x7f91f406d990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9208034000 tx=0x7f920802e000 comp rx=0 tx=0).stop 2024-04-25T10:09:29.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.174+0000 7f92181ed640 1 -- 172.21.15.16:0/1138642244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921015c820 msgr2=0x7f921013feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.174+0000 7f92181ed640 1 --2- 172.21.15.16:0/1138642244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921015c820 0x7f921013feb0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f920c00c730 tx=0x7f920c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.174+0000 7f92181ed640 1 -- 172.21.15.16:0/1138642244 shutdown_connections 2024-04-25T10:09:29.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.174+0000 7f92181ed640 1 --2- 172.21.15.16:0/1138642244 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f91f406b4d0 0x7f91f406d990 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.174+0000 7f92181ed640 1 --2- 172.21.15.16:0/1138642244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921015c820 0x7f921013feb0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.174+0000 7f92181ed640 1 --2- 172.21.15.16:0/1138642244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f921015bf00 0x7f921013f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.174+0000 7f92181ed640 1 --2- 172.21.15.16:0/1138642244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92100cc310 0x7f921015a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.174+0000 7f92181ed640 1 -- 172.21.15.16:0/1138642244 >> 172.21.15.16:0/1138642244 conn(0x7f921006aba0 msgr2=0x7f92100cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:29.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.174+0000 7f92181ed640 1 -- 172.21.15.16:0/1138642244 shutdown_connections 2024-04-25T10:09:29.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.174+0000 7f92181ed640 1 -- 172.21.15.16:0/1138642244 wait complete. 2024-04-25T10:09:29.191 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:09:20.361009+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:09:21.048892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:29.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:29.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.288+0000 7fe2c5b22640 1 Processor -- start 2024-04-25T10:09:29.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.289+0000 7fe2c5b22640 1 -- start start 2024-04-25T10:09:29.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.289+0000 7fe2c5b22640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2c00d3b90 0x7fe2c00d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.289+0000 7fe2c5b22640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c00d4090 con 0x7fe2c015dca0 2024-04-25T10:09:29.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.289+0000 7fe2c5b22640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c00d4370 con 0x7fe2c00d0f60 2024-04-25T10:09:29.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.289+0000 7fe2c5b22640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c00b6b20 con 0x7fe2c00d3b90 2024-04-25T10:09:29.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.289+0000 7fe2bf7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe2c015dca0 0x7fe2c00ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50478/0 (socket says 172.21.15.16:50478) 2024-04-25T10:09:29.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.290+0000 7fe2bf7fe640 1 -- 172.21.15.16:0/1139102725 learned_addr learned my addr 172.21.15.16:0/1139102725 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:29.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.290+0000 7fe2beffd640 1 --2- 172.21.15.16:0/1139102725 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2c00d3b90 0x7fe2c00d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.290+0000 7fe2be7fc640 1 -- 172.21.15.16:0/1139102725 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317280136 0 0) 0x7fe2c00d4090 con 0x7fe2c015dca0 2024-04-25T10:09:29.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.290+0000 7fe2be7fc640 1 -- 172.21.15.16:0/1139102725 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2a8003520 con 0x7fe2c015dca0 2024-04-25T10:09:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.290+0000 7fe2be7fc640 1 -- 172.21.15.16:0/1139102725 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956614151 0 0) 0x7fe2c00d4370 con 0x7fe2c00d0f60 2024-04-25T10:09:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.290+0000 7fe2be7fc640 1 -- 172.21.15.16:0/1139102725 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2c00d4090 con 0x7fe2c00d0f60 2024-04-25T10:09:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.290+0000 7fe2beffd640 1 -- 172.21.15.16:0/1139102725 >> v1:172.21.15.186:6789/0 conn(0x7fe2c00d0f60 legacy=0x7fe2c00b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.290+0000 7fe2beffd640 1 -- 172.21.15.16:0/1139102725 >> v1:172.21.15.16:6789/0 conn(0x7fe2c015dca0 legacy=0x7fe2c00ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.290+0000 7fe2be7fc640 1 -- 172.21.15.16:0/1139102725 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 690436420 0 0) 0x7fe2a8003520 con 0x7fe2c015dca0 2024-04-25T10:09:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.290+0000 7fe2beffd640 1 -- 172.21.15.16:0/1139102725 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c00d4650 con 0x7fe2c00d3b90 2024-04-25T10:09:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.291+0000 7fe2beffd640 1 --2- 172.21.15.16:0/1139102725 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2c00d3b90 0x7fe2c00d0a20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40097d0 tx=0x7fe2b402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1df928bf2a1162d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.291+0000 7fe2be7fc640 1 -- 172.21.15.16:0/1139102725 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4037070 con 0x7fe2c00d3b90 2024-04-25T10:09:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.291+0000 7fe2be7fc640 1 -- 172.21.15.16:0/1139102725 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b402ac30 con 0x7fe2c00d3b90 2024-04-25T10:09:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.291+0000 7fe2be7fc640 1 -- 172.21.15.16:0/1139102725 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0001670 con 0x7fe2c00d3b90 2024-04-25T10:09:29.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.291+0000 7fe2c5b22640 1 -- 172.21.15.16:0/1139102725 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2c00d3b90 msgr2=0x7fe2c00d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.291+0000 7fe2c5b22640 1 --2- 172.21.15.16:0/1139102725 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2c00d3b90 0x7fe2c00d0a20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40097d0 tx=0x7fe2b402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.292+0000 7fe2c5b22640 1 -- 172.21.15.16:0/1139102725 shutdown_connections 2024-04-25T10:09:29.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.292+0000 7fe2c5b22640 1 --2- 172.21.15.16:0/1139102725 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2c00d3b90 0x7fe2c00d0a20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.292+0000 7fe2c5b22640 1 -- 172.21.15.16:0/1139102725 >> 172.21.15.16:0/1139102725 conn(0x7fe2c00726c0 msgr2=0x7fe2c0072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.292+0000 7fe2c5b22640 1 -- 172.21.15.16:0/1139102725 shutdown_connections 2024-04-25T10:09:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.292+0000 7fe2c5b22640 1 -- 172.21.15.16:0/1139102725 wait complete. 2024-04-25T10:09:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.293+0000 7fe2c5b22640 1 Processor -- start 2024-04-25T10:09:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.293+0000 7fe2c5b22640 1 -- start start 2024-04-25T10:09:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.294+0000 7fe2c5b22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2c00d0f60 0x7fe2c00ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.294+0000 7fe2c5b22640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2c00d3b90 0x7fe2c00cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.294+0000 7fe2c5b22640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2c015dca0 0x7fe2c0155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.294+0000 7fe2c5b22640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c015e080 con 0x7fe2c00d0f60 2024-04-25T10:09:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.294+0000 7fe2c5b22640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c00cf050 con 0x7fe2c00d3b90 2024-04-25T10:09:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.294+0000 7fe2c5b22640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c015e200 con 0x7fe2c015dca0 2024-04-25T10:09:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.294+0000 7fe2bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2c00d0f60 0x7fe2c00ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.294+0000 7fe2bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2c00d0f60 0x7fe2c00ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52608/0 (socket says 172.21.15.16:52608) 2024-04-25T10:09:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.294+0000 7fe2bf7fe640 1 -- 172.21.15.16:0/579926595 learned_addr learned my addr 172.21.15.16:0/579926595 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:29.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.294+0000 7fe2beffd640 1 --2- 172.21.15.16:0/579926595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2c00d3b90 0x7fe2c00cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.294+0000 7fe2bffff640 1 --2- 172.21.15.16:0/579926595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2c015dca0 0x7fe2c0155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.294+0000 7fe2bf7fe640 1 -- 172.21.15.16:0/579926595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2c015dca0 msgr2=0x7fe2c0155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.294+0000 7fe2bf7fe640 1 --2- 172.21.15.16:0/579926595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2c015dca0 0x7fe2c0155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.294+0000 7fe2bf7fe640 1 -- 172.21.15.16:0/579926595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2c00d3b90 msgr2=0x7fe2c00cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.294+0000 7fe2bf7fe640 1 --2- 172.21.15.16:0/579926595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2c00d3b90 0x7fe2c00cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.294+0000 7fe2bf7fe640 1 -- 172.21.15.16:0/579926595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c0154980 con 0x7fe2c00d0f60 2024-04-25T10:09:29.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.294+0000 7fe2bf7fe640 1 --2- 172.21.15.16:0/579926595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2c00d0f60 0x7fe2c00ca1f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000c4f0 tx=0x7fe2b000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:29.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.294+0000 7fe2a7fff640 1 -- 172.21.15.16:0/579926595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0017070 con 0x7fe2c00d0f60 2024-04-25T10:09:29.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.295+0000 7fe2a7fff640 1 -- 172.21.15.16:0/579926595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b0005ce0 con 0x7fe2c00d0f60 2024-04-25T10:09:29.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.295+0000 7fe2a7fff640 1 -- 172.21.15.16:0/579926595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0007290 con 0x7fe2c00d0f60 2024-04-25T10:09:29.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.295+0000 7fe2c5b22640 1 -- 172.21.15.16:0/579926595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c0154c60 con 0x7fe2c00d0f60 2024-04-25T10:09:29.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.295+0000 7fe2c5b22640 1 -- 172.21.15.16:0/579926595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c00cad00 con 0x7fe2c00d0f60 2024-04-25T10:09:29.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.295+0000 7fe2c5b22640 1 -- 172.21.15.16:0/579926595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c0155180 con 0x7fe2c00d0f60 2024-04-25T10:09:29.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.296+0000 7fe2a7fff640 1 -- 172.21.15.16:0/579926595 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2b001a070 con 0x7fe2c00d0f60 2024-04-25T10:09:29.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.296+0000 7fe2a7fff640 1 --2- 172.21.15.16:0/579926595 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe2ac06b4d0 0x7fe2ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.297+0000 7fe2a7fff640 1 -- 172.21.15.16:0/579926595 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2b0089680 con 0x7fe2c00d0f60 2024-04-25T10:09:29.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.297+0000 7fe2beffd640 1 --2- 172.21.15.16:0/579926595 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe2ac06b4d0 0x7fe2ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.297+0000 7fe2beffd640 1 --2- 172.21.15.16:0/579926595 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe2ac06b4d0 0x7fe2ac06d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4034000 tx=0x7fe2b402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:29.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.301+0000 7fe2a7fff640 1 -- 172.21.15.16:0/579926595 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2c0155180 con 0x7fe2c00d0f60 2024-04-25T10:09:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.481+0000 7fe2c5b22640 1 -- 172.21.15.16:0/579926595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2c00750b0 con 0x7fe2c00d0f60 2024-04-25T10:09:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.481+0000 7fe2a7fff640 1 -- 172.21.15.16:0/579926595 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2b0057e30 con 0x7fe2c00d0f60 2024-04-25T10:09:29.488 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:09:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.483+0000 7fe2c5b22640 1 -- 172.21.15.16:0/579926595 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe2ac06b4d0 msgr2=0x7fe2ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.483+0000 7fe2c5b22640 1 --2- 172.21.15.16:0/579926595 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe2ac06b4d0 0x7fe2ac06d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4034000 tx=0x7fe2b402e000 comp rx=0 tx=0).stop 2024-04-25T10:09:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.483+0000 7fe2c5b22640 1 -- 172.21.15.16:0/579926595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2c00d0f60 msgr2=0x7fe2c00ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.483+0000 7fe2c5b22640 1 --2- 172.21.15.16:0/579926595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2c00d0f60 0x7fe2c00ca1f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000c4f0 tx=0x7fe2b000c9b0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.483+0000 7fe2c5b22640 1 -- 172.21.15.16:0/579926595 shutdown_connections 2024-04-25T10:09:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.483+0000 7fe2c5b22640 1 --2- 172.21.15.16:0/579926595 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe2ac06b4d0 0x7fe2ac06d990 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.483+0000 7fe2c5b22640 1 --2- 172.21.15.16:0/579926595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2c015dca0 0x7fe2c0155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.483+0000 7fe2c5b22640 1 --2- 172.21.15.16:0/579926595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2c00d3b90 0x7fe2c00cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.483+0000 7fe2c5b22640 1 --2- 172.21.15.16:0/579926595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2c00d0f60 0x7fe2c00ca1f0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.483+0000 7fe2c5b22640 1 -- 172.21.15.16:0/579926595 >> 172.21.15.16:0/579926595 conn(0x7fe2c00726c0 msgr2=0x7fe2c00b84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.484+0000 7fe2c5b22640 1 -- 172.21.15.16:0/579926595 shutdown_connections 2024-04-25T10:09:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.484+0000 7fe2c5b22640 1 -- 172.21.15.16:0/579926595 wait complete. 2024-04-25T10:09:29.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:09:29.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.599+0000 7f085a2b3640 1 Processor -- start 2024-04-25T10:09:29.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.599+0000 7f085a2b3640 1 -- start start 2024-04-25T10:09:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.600+0000 7f085a2b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f085415dc80 0x7f0854161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.600+0000 7f085a2b3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f08540b97c0 0x7f08540b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.600+0000 7f085a2b3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f08540b5a30 0x7f085406bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.600+0000 7f085a2b3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08540b5ea0 con 0x7f08540b97c0 2024-04-25T10:09:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.600+0000 7f085a2b3640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08541627a0 con 0x7f08540b5a30 2024-04-25T10:09:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.600+0000 7f085a2b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08541621e0 con 0x7f085415dc80 2024-04-25T10:09:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.600+0000 7f08537fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f08540b97c0 0x7f08540b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.600+0000 7f08537fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f08540b97c0 0x7f08540b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52614/0 (socket says 172.21.15.16:52614) 2024-04-25T10:09:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.600+0000 7f08537fe640 1 -- 172.21.15.16:0/1523468350 learned_addr learned my addr 172.21.15.16:0/1523468350 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.600+0000 7f0858850640 1 --2- 172.21.15.16:0/1523468350 >> v2:172.21.15.186:3300/0 conn(0x7f08540b5a30 0x7f085406bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.600+0000 7f08537fe640 1 -- 172.21.15.16:0/1523468350 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f085415dc80 msgr2=0x7f0854161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.600+0000 7f0853fff640 1 --2- 172.21.15.16:0/1523468350 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f085415dc80 0x7f0854161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.600+0000 7f08537fe640 1 --2- 172.21.15.16:0/1523468350 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f085415dc80 0x7f0854161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.600+0000 7f08537fe640 1 -- 172.21.15.16:0/1523468350 >> v2:172.21.15.186:3300/0 conn(0x7f08540b5a30 msgr2=0x7f085406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.600+0000 7f08537fe640 1 --2- 172.21.15.16:0/1523468350 >> v2:172.21.15.186:3300/0 conn(0x7f08540b5a30 0x7f085406bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.600+0000 7f08537fe640 1 -- 172.21.15.16:0/1523468350 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08541624c0 con 0x7f08540b97c0 2024-04-25T10:09:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.600+0000 7f0858850640 1 --2- 172.21.15.16:0/1523468350 >> v2:172.21.15.186:3300/0 conn(0x7f08540b5a30 0x7f085406bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.601+0000 7f08537fe640 1 --2- 172.21.15.16:0/1523468350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08540b97c0 0x7f08540b54f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f08540d16d0 tx=0x7f084802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745d54366513fadc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.601+0000 7f0852ffd640 1 -- 172.21.15.16:0/1523468350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0848037070 con 0x7f08540b97c0 2024-04-25T10:09:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.601+0000 7f0852ffd640 1 -- 172.21.15.16:0/1523468350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0848032440 con 0x7f08540b97c0 2024-04-25T10:09:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.601+0000 7f0852ffd640 1 -- 172.21.15.16:0/1523468350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0848032810 con 0x7f08540b97c0 2024-04-25T10:09:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.601+0000 7f085a2b3640 1 -- 172.21.15.16:0/1523468350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08540b97c0 msgr2=0x7f08540b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.601+0000 7f085a2b3640 1 --2- 172.21.15.16:0/1523468350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08540b97c0 0x7f08540b54f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f08540d16d0 tx=0x7f084802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.602+0000 7f085a2b3640 1 -- 172.21.15.16:0/1523468350 shutdown_connections 2024-04-25T10:09:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.602+0000 7f085a2b3640 1 --2- 172.21.15.16:0/1523468350 >> v2:172.21.15.186:3300/0 conn(0x7f08540b5a30 0x7f085406bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.602+0000 7f085a2b3640 1 --2- 172.21.15.16:0/1523468350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08540b97c0 0x7f08540b54f0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.602+0000 7f085a2b3640 1 --2- 172.21.15.16:0/1523468350 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f085415dc80 0x7f0854161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.602+0000 7f085a2b3640 1 -- 172.21.15.16:0/1523468350 >> 172.21.15.16:0/1523468350 conn(0x7f08540d04f0 msgr2=0x7f08540d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.602+0000 7f085a2b3640 1 -- 172.21.15.16:0/1523468350 shutdown_connections 2024-04-25T10:09:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.602+0000 7f085a2b3640 1 -- 172.21.15.16:0/1523468350 wait complete. 2024-04-25T10:09:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.603+0000 7f085a2b3640 1 Processor -- start 2024-04-25T10:09:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.603+0000 7f085a2b3640 1 -- start start 2024-04-25T10:09:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f085a2b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08540b5a30 0x7f08540c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f085a2b3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08540b97c0 0x7f08540cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f085a2b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f085415dc80 0x7f085415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f085a2b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0854161db0 con 0x7f085415dc80 2024-04-25T10:09:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f085a2b3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085415e060 con 0x7f08540b97c0 2024-04-25T10:09:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f085a2b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0854161f30 con 0x7f08540b5a30 2024-04-25T10:09:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f0858850640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f085415dc80 0x7f085415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f0858850640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f085415dc80 0x7f085415a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52624/0 (socket says 172.21.15.16:52624) 2024-04-25T10:09:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f0858850640 1 -- 172.21.15.16:0/4118384631 learned_addr learned my addr 172.21.15.16:0/4118384631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f08537fe640 1 --2- 172.21.15.16:0/4118384631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08540b97c0 0x7f08540cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f0858850640 1 -- 172.21.15.16:0/4118384631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08540b5a30 msgr2=0x7f08540c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f0853fff640 1 --2- 172.21.15.16:0/4118384631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08540b5a30 0x7f08540c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f0858850640 1 --2- 172.21.15.16:0/4118384631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08540b5a30 0x7f08540c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f0858850640 1 -- 172.21.15.16:0/4118384631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08540b97c0 msgr2=0x7f08540cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f0858850640 1 --2- 172.21.15.16:0/4118384631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08540b97c0 0x7f08540cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f0858850640 1 -- 172.21.15.16:0/4118384631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08540c8f30 con 0x7f085415dc80 2024-04-25T10:09:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f08537fe640 1 --2- 172.21.15.16:0/4118384631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08540b97c0 0x7f08540cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f0858850640 1 --2- 172.21.15.16:0/4118384631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f085415dc80 0x7f085415a020 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f084c00cd10 tx=0x7f084c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f0850ff9640 1 -- 172.21.15.16:0/4118384631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c00f680 con 0x7f085415dc80 2024-04-25T10:09:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f0850ff9640 1 -- 172.21.15.16:0/4118384631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084c00f840 con 0x7f085415dc80 2024-04-25T10:09:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f0850ff9640 1 -- 172.21.15.16:0/4118384631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c00fc30 con 0x7f085415dc80 2024-04-25T10:09:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f085a2b3640 1 -- 172.21.15.16:0/4118384631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08540c9210 con 0x7f085415dc80 2024-04-25T10:09:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.604+0000 7f085a2b3640 1 -- 172.21.15.16:0/4118384631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08540c97d0 con 0x7f085415dc80 2024-04-25T10:09:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.605+0000 7f085a2b3640 1 -- 172.21.15.16:0/4118384631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08540750b0 con 0x7f085415dc80 2024-04-25T10:09:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.606+0000 7f0850ff9640 1 -- 172.21.15.16:0/4118384631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f084c00e3e0 con 0x7f085415dc80 2024-04-25T10:09:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.606+0000 7f0850ff9640 1 --2- 172.21.15.16:0/4118384631 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f084006b4d0 0x7f084006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.606+0000 7f0850ff9640 1 -- 172.21.15.16:0/4118384631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f084c08aff0 con 0x7f085415dc80 2024-04-25T10:09:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.607+0000 7f0853fff640 1 --2- 172.21.15.16:0/4118384631 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f084006b4d0 0x7f084006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.607+0000 7f0853fff640 1 --2- 172.21.15.16:0/4118384631 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f084006b4d0 0x7f084006d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0844006fd0 tx=0x7f0844008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.611+0000 7f0850ff9640 1 -- 172.21.15.16:0/4118384631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f084c014030 con 0x7f085415dc80 2024-04-25T10:09:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.793+0000 7f085a2b3640 1 -- 172.21.15.16:0/4118384631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f08540cee90 con 0x7f085415dc80 2024-04-25T10:09:29.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.794+0000 7f0850ff9640 1 -- 172.21.15.16:0/4118384631 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) v1 ==== 104+0+11 (secure 0 0 0) 0x7f084c059de0 con 0x7f085415dc80 2024-04-25T10:09:29.800 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:09:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.796+0000 7f085a2b3640 1 -- 172.21.15.16:0/4118384631 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f084006b4d0 msgr2=0x7f084006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.796+0000 7f085a2b3640 1 --2- 172.21.15.16:0/4118384631 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f084006b4d0 0x7f084006d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0844006fd0 tx=0x7f0844008040 comp rx=0 tx=0).stop 2024-04-25T10:09:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.796+0000 7f085a2b3640 1 -- 172.21.15.16:0/4118384631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f085415dc80 msgr2=0x7f085415a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.796+0000 7f085a2b3640 1 --2- 172.21.15.16:0/4118384631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f085415dc80 0x7f085415a020 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f084c00cd10 tx=0x7f084c00a650 comp rx=0 tx=0).stop 2024-04-25T10:09:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.796+0000 7f085a2b3640 1 -- 172.21.15.16:0/4118384631 shutdown_connections 2024-04-25T10:09:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.796+0000 7f085a2b3640 1 --2- 172.21.15.16:0/4118384631 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f084006b4d0 0x7f084006d990 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.796+0000 7f085a2b3640 1 --2- 172.21.15.16:0/4118384631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f085415dc80 0x7f085415a020 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.796+0000 7f085a2b3640 1 --2- 172.21.15.16:0/4118384631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08540b97c0 0x7f08540cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.796+0000 7f085a2b3640 1 --2- 172.21.15.16:0/4118384631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08540b5a30 0x7f08540c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.796+0000 7f085a2b3640 1 -- 172.21.15.16:0/4118384631 >> 172.21.15.16:0/4118384631 conn(0x7f08540d04f0 msgr2=0x7f08540b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:29.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.796+0000 7f085a2b3640 1 -- 172.21.15.16:0/4118384631 shutdown_connections 2024-04-25T10:09:29.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.796+0000 7f085a2b3640 1 -- 172.21.15.16:0/4118384631 wait complete. 2024-04-25T10:09:29.813 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:09:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.910+0000 7f62e87d5640 1 Processor -- start 2024-04-25T10:09:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.910+0000 7f62e87d5640 1 -- start start 2024-04-25T10:09:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.911+0000 7f62e87d5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62e015aef0 0x7f62e015ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.911+0000 7f62e87d5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f62e00b46a0 0x7f62e01549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.911+0000 7f62e87d5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e015b370 con 0x7f62e00b42c0 2024-04-25T10:09:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.911+0000 7f62e87d5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e00d05d0 con 0x7f62e015aef0 2024-04-25T10:09:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.911+0000 7f62e87d5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e015b650 con 0x7f62e00b46a0 2024-04-25T10:09:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.911+0000 7f62e5d70640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62e015aef0 0x7f62e015ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.911+0000 7f62e6571640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f62e00b42c0 0x7f62e00cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50494/0 (socket says 172.21.15.16:50494) 2024-04-25T10:09:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.911+0000 7f62e6571640 1 -- 172.21.15.16:0/1297688164 learned_addr learned my addr 172.21.15.16:0/1297688164 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.911+0000 7f62e5d70640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62e015aef0 0x7f62e015ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52634/0 (socket says 172.21.15.16:52634) 2024-04-25T10:09:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.911+0000 7f62e5d70640 1 -- 172.21.15.16:0/1297688164 >> v2:172.21.15.186:3300/0 conn(0x7f62e00b46a0 msgr2=0x7f62e01549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.911+0000 7f62e6d72640 1 --2- 172.21.15.16:0/1297688164 >> v2:172.21.15.186:3300/0 conn(0x7f62e00b46a0 0x7f62e01549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.912+0000 7f62e5d70640 1 --2- 172.21.15.16:0/1297688164 >> v2:172.21.15.186:3300/0 conn(0x7f62e00b46a0 0x7f62e01549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.912+0000 7f62e5d70640 1 -- 172.21.15.16:0/1297688164 >> v1:172.21.15.16:6789/0 conn(0x7f62e00b42c0 legacy=0x7f62e00cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.912+0000 7f62e556f640 1 -- 172.21.15.16:0/1297688164 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194866259 0 0) 0x7f62e015b370 con 0x7f62e00b42c0 2024-04-25T10:09:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.912+0000 7f62e5d70640 1 -- 172.21.15.16:0/1297688164 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e015b930 con 0x7f62e015aef0 2024-04-25T10:09:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.912+0000 7f62e6d72640 1 --2- 172.21.15.16:0/1297688164 >> v2:172.21.15.186:3300/0 conn(0x7f62e00b46a0 0x7f62e01549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.912+0000 7f62e5d70640 1 --2- 172.21.15.16:0/1297688164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e015aef0 0x7f62e015ef40 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f62e00cd110 tx=0x7f62d402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a630bcc2fb86f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.912+0000 7f62e556f640 1 -- 172.21.15.16:0/1297688164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4037070 con 0x7f62e015aef0 2024-04-25T10:09:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.912+0000 7f62e556f640 1 -- 172.21.15.16:0/1297688164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d402add0 con 0x7f62e015aef0 2024-04-25T10:09:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.912+0000 7f62e556f640 1 -- 172.21.15.16:0/1297688164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4032660 con 0x7f62e015aef0 2024-04-25T10:09:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.913+0000 7f62e87d5640 1 -- 172.21.15.16:0/1297688164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e015aef0 msgr2=0x7f62e015ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.913+0000 7f62e87d5640 1 --2- 172.21.15.16:0/1297688164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e015aef0 0x7f62e015ef40 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f62e00cd110 tx=0x7f62d402a080 comp rx=0 tx=0).stop 2024-04-25T10:09:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.913+0000 7f62e87d5640 1 -- 172.21.15.16:0/1297688164 shutdown_connections 2024-04-25T10:09:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.913+0000 7f62e87d5640 1 --2- 172.21.15.16:0/1297688164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e015aef0 0x7f62e015ef40 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.913+0000 7f62e87d5640 1 --2- 172.21.15.16:0/1297688164 >> v2:172.21.15.186:3300/0 conn(0x7f62e00b46a0 0x7f62e01549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.913+0000 7f62e87d5640 1 -- 172.21.15.16:0/1297688164 >> 172.21.15.16:0/1297688164 conn(0x7f62e006aba0 msgr2=0x7f62e006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.913+0000 7f62e87d5640 1 -- 172.21.15.16:0/1297688164 shutdown_connections 2024-04-25T10:09:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.913+0000 7f62e87d5640 1 -- 172.21.15.16:0/1297688164 wait complete. 2024-04-25T10:09:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.914+0000 7f62e87d5640 1 Processor -- start 2024-04-25T10:09:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.914+0000 7f62e87d5640 1 -- start start 2024-04-25T10:09:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.915+0000 7f62e87d5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62e00b42c0 0x7f62e0143900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.915+0000 7f62e87d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e00b46a0 0x7f62e0148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.915+0000 7f62e87d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e015aef0 0x7f62e0149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.915+0000 7f62e87d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e006ec60 con 0x7f62e00b46a0 2024-04-25T10:09:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.915+0000 7f62e87d5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e006eae0 con 0x7f62e00b42c0 2024-04-25T10:09:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.915+0000 7f62e87d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e0154f10 con 0x7f62e015aef0 2024-04-25T10:09:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.915+0000 7f62e5d70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e00b46a0 0x7f62e0148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.915+0000 7f62e5d70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e00b46a0 0x7f62e0148e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52648/0 (socket says 172.21.15.16:52648) 2024-04-25T10:09:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.915+0000 7f62e5d70640 1 -- 172.21.15.16:0/1339992762 learned_addr learned my addr 172.21.15.16:0/1339992762 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.915+0000 7f62e6571640 1 --2- 172.21.15.16:0/1339992762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62e00b42c0 0x7f62e0143900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.915+0000 7f62e5d70640 1 -- 172.21.15.16:0/1339992762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e015aef0 msgr2=0x7f62e0149340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.915+0000 7f62e5d70640 1 --2- 172.21.15.16:0/1339992762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e015aef0 0x7f62e0149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.915+0000 7f62e5d70640 1 -- 172.21.15.16:0/1339992762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62e00b42c0 msgr2=0x7f62e0143900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.915+0000 7f62e5d70640 1 --2- 172.21.15.16:0/1339992762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62e00b42c0 0x7f62e0143900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.915+0000 7f62e5d70640 1 -- 172.21.15.16:0/1339992762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e012e620 con 0x7f62e00b46a0 2024-04-25T10:09:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.916+0000 7f62e5d70640 1 --2- 172.21.15.16:0/1339992762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e00b46a0 0x7f62e0148e00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f62d4000c00 tx=0x7f62d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.916+0000 7f62deffd640 1 -- 172.21.15.16:0/1339992762 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4037050 con 0x7f62e00b46a0 2024-04-25T10:09:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.916+0000 7f62deffd640 1 -- 172.21.15.16:0/1339992762 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d4038040 con 0x7f62e00b46a0 2024-04-25T10:09:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.916+0000 7f62e87d5640 1 -- 172.21.15.16:0/1339992762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e012e900 con 0x7f62e00b46a0 2024-04-25T10:09:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.916+0000 7f62deffd640 1 -- 172.21.15.16:0/1339992762 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4004470 con 0x7f62e00b46a0 2024-04-25T10:09:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.916+0000 7f62e87d5640 1 -- 172.21.15.16:0/1339992762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e012eec0 con 0x7f62e00b46a0 2024-04-25T10:09:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.917+0000 7f62e87d5640 1 -- 172.21.15.16:0/1339992762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e0067640 con 0x7f62e00b46a0 2024-04-25T10:09:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.917+0000 7f62deffd640 1 -- 172.21.15.16:0/1339992762 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62d402ac90 con 0x7f62e00b46a0 2024-04-25T10:09:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.917+0000 7f62deffd640 1 --2- 172.21.15.16:0/1339992762 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f62cc06b4d0 0x7f62cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.918+0000 7f62deffd640 1 -- 172.21.15.16:0/1339992762 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62d40aaab0 con 0x7f62e00b46a0 2024-04-25T10:09:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.918+0000 7f62e6571640 1 --2- 172.21.15.16:0/1339992762 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f62cc06b4d0 0x7f62cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.919+0000 7f62e6571640 1 --2- 172.21.15.16:0/1339992762 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f62cc06b4d0 0x7f62cc06d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f62d8000f60 tx=0x7f62d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:29.922+0000 7f62deffd640 1 -- 172.21.15.16:0/1339992762 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62e0067640 con 0x7f62e00b46a0 2024-04-25T10:09:30.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.099+0000 7f62e87d5640 1 -- 172.21.15.16:0/1339992762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f62e006e700 con 0x7f62e00b46a0 2024-04-25T10:09:30.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.100+0000 7f62deffd640 1 -- 172.21.15.16:0/1339992762 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) v1 ==== 100+0+11 (secure 0 0 0) 0x7f62d4079a50 con 0x7f62e00b46a0 2024-04-25T10:09:30.106 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:09:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.101+0000 7f62e87d5640 1 -- 172.21.15.16:0/1339992762 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f62cc06b4d0 msgr2=0x7f62cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.102+0000 7f62e87d5640 1 --2- 172.21.15.16:0/1339992762 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f62cc06b4d0 0x7f62cc06d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f62d8000f60 tx=0x7f62d8009040 comp rx=0 tx=0).stop 2024-04-25T10:09:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.102+0000 7f62e87d5640 1 -- 172.21.15.16:0/1339992762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e00b46a0 msgr2=0x7f62e0148e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.102+0000 7f62e87d5640 1 --2- 172.21.15.16:0/1339992762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e00b46a0 0x7f62e0148e00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f62d4000c00 tx=0x7f62d40046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.102+0000 7f62e87d5640 1 -- 172.21.15.16:0/1339992762 shutdown_connections 2024-04-25T10:09:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.102+0000 7f62e87d5640 1 --2- 172.21.15.16:0/1339992762 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f62cc06b4d0 0x7f62cc06d990 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.102+0000 7f62e87d5640 1 --2- 172.21.15.16:0/1339992762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62e015aef0 0x7f62e0149340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.102+0000 7f62e87d5640 1 --2- 172.21.15.16:0/1339992762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e00b46a0 0x7f62e0148e00 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.102+0000 7f62e87d5640 1 --2- 172.21.15.16:0/1339992762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62e00b42c0 0x7f62e0143900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.102+0000 7f62e87d5640 1 -- 172.21.15.16:0/1339992762 >> 172.21.15.16:0/1339992762 conn(0x7f62e006aba0 msgr2=0x7f62e00cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.102+0000 7f62e87d5640 1 -- 172.21.15.16:0/1339992762 shutdown_connections 2024-04-25T10:09:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.102+0000 7f62e87d5640 1 -- 172.21.15.16:0/1339992762 wait complete. 2024-04-25T10:09:30.118 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:09:30.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.214+0000 7f7574e1f640 1 Processor -- start 2024-04-25T10:09:30.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.214+0000 7f7574e1f640 1 -- start start 2024-04-25T10:09:30.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.215+0000 7f7574e1f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7570071590 0x7f75700cc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.215+0000 7f7574e1f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75700cd940 0x7f75700ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.215+0000 7f7574e1f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757006c220 con 0x7f757015dc50 2024-04-25T10:09:30.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.215+0000 7f7574e1f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75700cfe00 con 0x7f7570071590 2024-04-25T10:09:30.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.215+0000 7f7574e1f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75700cec10 con 0x7f75700cd940 2024-04-25T10:09:30.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.215+0000 7f7567fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7570071590 0x7f75700cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.215+0000 7f756e59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f757015dc50 0x7f7570070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50496/0 (socket says 172.21.15.16:50496) 2024-04-25T10:09:30.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.215+0000 7f756e59c640 1 -- 172.21.15.16:0/4078576104 learned_addr learned my addr 172.21.15.16:0/4078576104 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:30.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.215+0000 7f756ed9d640 1 --2- 172.21.15.16:0/4078576104 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75700cd940 0x7f75700ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.215+0000 7f7567fff640 1 -- 172.21.15.16:0/4078576104 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75700cd940 msgr2=0x7f75700ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:30.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.215+0000 7f7567fff640 1 --2- 172.21.15.16:0/4078576104 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75700cd940 0x7f75700ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.215+0000 7f7567fff640 1 -- 172.21.15.16:0/4078576104 >> v1:172.21.15.16:6789/0 conn(0x7f757015dc50 legacy=0x7f7570070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.215+0000 7f7567fff640 1 -- 172.21.15.16:0/4078576104 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75700d0050 con 0x7f7570071590 2024-04-25T10:09:30.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.215+0000 7f756dd9b640 1 -- 172.21.15.16:0/4078576104 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158061517 0 0) 0x7f757006c220 con 0x7f757015dc50 2024-04-25T10:09:30.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.216+0000 7f7567fff640 1 --2- 172.21.15.16:0/4078576104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570071590 0x7f75700cc610 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f75580097a0 tx=0x7f755802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884a5c5d72c45fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:30.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.216+0000 7f756dd9b640 1 -- 172.21.15.16:0/4078576104 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558037070 con 0x7f7570071590 2024-04-25T10:09:30.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.216+0000 7f756dd9b640 1 -- 172.21.15.16:0/4078576104 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755802ae50 con 0x7f7570071590 2024-04-25T10:09:30.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.216+0000 7f756dd9b640 1 -- 172.21.15.16:0/4078576104 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558032660 con 0x7f7570071590 2024-04-25T10:09:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.216+0000 7f7574e1f640 1 -- 172.21.15.16:0/4078576104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570071590 msgr2=0x7f75700cc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.216+0000 7f7574e1f640 1 --2- 172.21.15.16:0/4078576104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570071590 0x7f75700cc610 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f75580097a0 tx=0x7f755802a100 comp rx=0 tx=0).stop 2024-04-25T10:09:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.217+0000 7f7574e1f640 1 -- 172.21.15.16:0/4078576104 shutdown_connections 2024-04-25T10:09:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.217+0000 7f7574e1f640 1 --2- 172.21.15.16:0/4078576104 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75700cd940 0x7f75700ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.217+0000 7f7574e1f640 1 --2- 172.21.15.16:0/4078576104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570071590 0x7f75700cc610 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.217+0000 7f7574e1f640 1 -- 172.21.15.16:0/4078576104 >> 172.21.15.16:0/4078576104 conn(0x7f757006f430 msgr2=0x7f757006f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.217+0000 7f7574e1f640 1 -- 172.21.15.16:0/4078576104 shutdown_connections 2024-04-25T10:09:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.217+0000 7f7574e1f640 1 -- 172.21.15.16:0/4078576104 wait complete. 2024-04-25T10:09:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.218+0000 7f7574e1f640 1 Processor -- start 2024-04-25T10:09:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.218+0000 7f7574e1f640 1 -- start start 2024-04-25T10:09:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f7574e1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570071590 0x7f757014fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f7574e1f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75700cd940 0x7f75700b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f7574e1f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f757015dc50 0x7f75700c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f7574e1f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75700ce8a0 con 0x7f7570071590 2024-04-25T10:09:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f7574e1f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75700cc0c0 con 0x7f75700cd940 2024-04-25T10:09:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f7574e1f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757015e030 con 0x7f757015dc50 2024-04-25T10:09:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f756e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570071590 0x7f757014fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f756e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570071590 0x7f757014fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52652/0 (socket says 172.21.15.16:52652) 2024-04-25T10:09:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f756e59c640 1 -- 172.21.15.16:0/4065331255 learned_addr learned my addr 172.21.15.16:0/4065331255 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f7567fff640 1 --2- 172.21.15.16:0/4065331255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75700cd940 0x7f75700b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f756ed9d640 1 --2- 172.21.15.16:0/4065331255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f757015dc50 0x7f75700c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f756e59c640 1 -- 172.21.15.16:0/4065331255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f757015dc50 msgr2=0x7f75700c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f756e59c640 1 --2- 172.21.15.16:0/4065331255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f757015dc50 0x7f75700c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f756e59c640 1 -- 172.21.15.16:0/4065331255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75700cd940 msgr2=0x7f75700b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f756e59c640 1 --2- 172.21.15.16:0/4065331255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75700cd940 0x7f75700b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f756e59c640 1 -- 172.21.15.16:0/4065331255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75700b6320 con 0x7f7570071590 2024-04-25T10:09:30.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f756ed9d640 1 --2- 172.21.15.16:0/4065331255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f757015dc50 0x7f75700c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:30.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f756e59c640 1 --2- 172.21.15.16:0/4065331255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570071590 0x7f757014fca0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f756000c430 tx=0x7f756000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:30.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.219+0000 7f7566ffd640 1 -- 172.21.15.16:0/4065331255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75600074e0 con 0x7f7570071590 2024-04-25T10:09:30.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.220+0000 7f7566ffd640 1 -- 172.21.15.16:0/4065331255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75600047e0 con 0x7f7570071590 2024-04-25T10:09:30.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.220+0000 7f7574e1f640 1 -- 172.21.15.16:0/4065331255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75700b6600 con 0x7f7570071590 2024-04-25T10:09:30.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.220+0000 7f7566ffd640 1 -- 172.21.15.16:0/4065331255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560004bd0 con 0x7f7570071590 2024-04-25T10:09:30.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.220+0000 7f7574e1f640 1 -- 172.21.15.16:0/4065331255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75700b6ea0 con 0x7f7570071590 2024-04-25T10:09:30.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.220+0000 7f7574e1f640 1 -- 172.21.15.16:0/4065331255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75700b67d0 con 0x7f7570071590 2024-04-25T10:09:30.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.221+0000 7f7566ffd640 1 -- 172.21.15.16:0/4065331255 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7560011070 con 0x7f7570071590 2024-04-25T10:09:30.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.221+0000 7f7566ffd640 1 --2- 172.21.15.16:0/4065331255 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f755006b4d0 0x7f755006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.222+0000 7f7566ffd640 1 -- 172.21.15.16:0/4065331255 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7560089f60 con 0x7f7570071590 2024-04-25T10:09:30.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.222+0000 7f7567fff640 1 --2- 172.21.15.16:0/4065331255 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f755006b4d0 0x7f755006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.222+0000 7f7567fff640 1 --2- 172.21.15.16:0/4065331255 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f755006b4d0 0x7f755006d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7558034000 tx=0x7f755802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:30.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.226+0000 7f7566ffd640 1 -- 172.21.15.16:0/4065331255 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7560058d40 con 0x7f7570071590 2024-04-25T10:09:30.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.417+0000 7f7574e1f640 1 -- 172.21.15.16:0/4065331255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7570067c00 con 0x7f7570071590 2024-04-25T10:09:30.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.417+0000 7f7566ffd640 1 -- 172.21.15.16:0/4065331255 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v72) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7570067c00 con 0x7f7570071590 2024-04-25T10:09:30.423 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-04-25T10:09:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.419+0000 7f7574e1f640 1 -- 172.21.15.16:0/4065331255 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f755006b4d0 msgr2=0x7f755006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.419+0000 7f7574e1f640 1 --2- 172.21.15.16:0/4065331255 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f755006b4d0 0x7f755006d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7558034000 tx=0x7f755802e000 comp rx=0 tx=0).stop 2024-04-25T10:09:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.419+0000 7f7574e1f640 1 -- 172.21.15.16:0/4065331255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570071590 msgr2=0x7f757014fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.419+0000 7f7574e1f640 1 --2- 172.21.15.16:0/4065331255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570071590 0x7f757014fca0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f756000c430 tx=0x7f756000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.419+0000 7f7574e1f640 1 -- 172.21.15.16:0/4065331255 shutdown_connections 2024-04-25T10:09:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.419+0000 7f7574e1f640 1 --2- 172.21.15.16:0/4065331255 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f755006b4d0 0x7f755006d990 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.419+0000 7f7574e1f640 1 --2- 172.21.15.16:0/4065331255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f757015dc50 0x7f75700c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.419+0000 7f7574e1f640 1 --2- 172.21.15.16:0/4065331255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75700cd940 0x7f75700b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.419+0000 7f7574e1f640 1 --2- 172.21.15.16:0/4065331255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570071590 0x7f757014fca0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.419+0000 7f7574e1f640 1 -- 172.21.15.16:0/4065331255 >> 172.21.15.16:0/4065331255 conn(0x7f757006f430 msgr2=0x7f75700b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.419+0000 7f7574e1f640 1 -- 172.21.15.16:0/4065331255 shutdown_connections 2024-04-25T10:09:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.419+0000 7f7574e1f640 1 -- 172.21.15.16:0/4065331255 wait complete. 2024-04-25T10:09:30.436 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:09:30.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:30.436 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:30.496 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:09:30.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:30.496 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:09:30.518 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.653+0000 7fbaec6ac640 1 Processor -- start 2024-04-25T10:09:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.653+0000 7fbaec6ac640 1 -- start start 2024-04-25T10:09:30.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.654+0000 7fbaec6ac640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbae409a150 0x7fbae406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.654+0000 7fbaec6ac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbae406c1f0 0x7fbae40cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.654+0000 7fbaec6ac640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae406c680 con 0x7fbae415ba10 2024-04-25T10:09:30.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.654+0000 7fbaec6ac640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae40cbe50 con 0x7fbae406c1f0 2024-04-25T10:09:30.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.654+0000 7fbaec6ac640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40cfe80 con 0x7fbae409a150 2024-04-25T10:09:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.654+0000 7fbaeac49640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbae406c1f0 0x7fbae40cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.654+0000 7fbaea448640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbae415ba10 0x7fbae4099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50508/0 (socket says 172.21.15.16:50508) 2024-04-25T10:09:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.654+0000 7fbaea448640 1 -- 172.21.15.16:0/2990765847 learned_addr learned my addr 172.21.15.16:0/2990765847 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.654+0000 7fbaeac49640 1 -- 172.21.15.16:0/2990765847 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbae409a150 msgr2=0x7fbae406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.654+0000 7fbae9c47640 1 --2- 172.21.15.16:0/2990765847 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbae409a150 0x7fbae406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.654+0000 7fbaeac49640 1 --2- 172.21.15.16:0/2990765847 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbae409a150 0x7fbae406bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.654+0000 7fbae9446640 1 -- 172.21.15.16:0/2990765847 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326470445 0 0) 0x7fbae0001110 con 0x7fbae415ba10 2024-04-25T10:09:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.654+0000 7fbaeac49640 1 -- 172.21.15.16:0/2990765847 >> v1:172.21.15.16:6789/0 conn(0x7fbae415ba10 legacy=0x7fbae4099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.654+0000 7fbaeac49640 1 -- 172.21.15.16:0/2990765847 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae406c900 con 0x7fbae406c1f0 2024-04-25T10:09:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.654+0000 7fbae9c47640 1 --2- 172.21.15.16:0/2990765847 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbae409a150 0x7fbae406bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.655+0000 7fbaeac49640 1 --2- 172.21.15.16:0/2990765847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae406c1f0 0x7fbae40cd220 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009d90 tx=0x7fbadc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345ecaaf251edc9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.655+0000 7fbae9446640 1 -- 172.21.15.16:0/2990765847 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc007740 con 0x7fbae406c1f0 2024-04-25T10:09:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.655+0000 7fbae9446640 1 -- 172.21.15.16:0/2990765847 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbadc0078e0 con 0x7fbae406c1f0 2024-04-25T10:09:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.655+0000 7fbae9446640 1 -- 172.21.15.16:0/2990765847 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc007cb0 con 0x7fbae406c1f0 2024-04-25T10:09:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.655+0000 7fbaec6ac640 1 -- 172.21.15.16:0/2990765847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae406c1f0 msgr2=0x7fbae40cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.655+0000 7fbaec6ac640 1 --2- 172.21.15.16:0/2990765847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae406c1f0 0x7fbae40cd220 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009d90 tx=0x7fbadc02a660 comp rx=0 tx=0).stop 2024-04-25T10:09:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.656+0000 7fbaec6ac640 1 -- 172.21.15.16:0/2990765847 shutdown_connections 2024-04-25T10:09:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.656+0000 7fbaec6ac640 1 --2- 172.21.15.16:0/2990765847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae406c1f0 0x7fbae40cd220 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.656+0000 7fbaec6ac640 1 --2- 172.21.15.16:0/2990765847 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbae409a150 0x7fbae406bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.656+0000 7fbaec6ac640 1 -- 172.21.15.16:0/2990765847 >> 172.21.15.16:0/2990765847 conn(0x7fbae40b32f0 msgr2=0x7fbae40b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.656+0000 7fbaec6ac640 1 -- 172.21.15.16:0/2990765847 shutdown_connections 2024-04-25T10:09:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.656+0000 7fbaec6ac640 1 -- 172.21.15.16:0/2990765847 wait complete. 2024-04-25T10:09:30.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.657+0000 7fbaec6ac640 1 Processor -- start 2024-04-25T10:09:30.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.657+0000 7fbaec6ac640 1 -- start start 2024-04-25T10:09:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.657+0000 7fbaec6ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae406c1f0 0x7fbae41f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.657+0000 7fbaec6ac640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbae409a150 0x7fbae41500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.657+0000 7fbaec6ac640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbae415ba10 0x7fbae40b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.657+0000 7fbaec6ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40cd760 con 0x7fbae406c1f0 2024-04-25T10:09:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.657+0000 7fbaec6ac640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae406cec0 con 0x7fbae415ba10 2024-04-25T10:09:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.657+0000 7fbaec6ac640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40cd8e0 con 0x7fbae409a150 2024-04-25T10:09:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.657+0000 7fbaea448640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae406c1f0 0x7fbae41f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.658+0000 7fbaea448640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae406c1f0 0x7fbae41f48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52680/0 (socket says 172.21.15.16:52680) 2024-04-25T10:09:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.658+0000 7fbaea448640 1 -- 172.21.15.16:0/2502936178 learned_addr learned my addr 172.21.15.16:0/2502936178 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.658+0000 7fbae9c47640 1 --2- 172.21.15.16:0/2502936178 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbae409a150 0x7fbae41500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.658+0000 7fbaeac49640 1 --2- 172.21.15.16:0/2502936178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbae415ba10 0x7fbae40b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.658+0000 7fbaea448640 1 -- 172.21.15.16:0/2502936178 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbae409a150 msgr2=0x7fbae41500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.658+0000 7fbaea448640 1 --2- 172.21.15.16:0/2502936178 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbae409a150 0x7fbae41500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.658+0000 7fbaea448640 1 -- 172.21.15.16:0/2502936178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbae415ba10 msgr2=0x7fbae40b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.658+0000 7fbaea448640 1 --2- 172.21.15.16:0/2502936178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbae415ba10 0x7fbae40b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.658+0000 7fbaea448640 1 -- 172.21.15.16:0/2502936178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae41f7620 con 0x7fbae406c1f0 2024-04-25T10:09:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.658+0000 7fbaea448640 1 --2- 172.21.15.16:0/2502936178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae406c1f0 0x7fbae41f48a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fbae000c430 tx=0x7fbae000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.658+0000 7fbadaffd640 1 -- 172.21.15.16:0/2502936178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae00074e0 con 0x7fbae406c1f0 2024-04-25T10:09:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.658+0000 7fbadaffd640 1 -- 172.21.15.16:0/2502936178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae00047e0 con 0x7fbae406c1f0 2024-04-25T10:09:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.658+0000 7fbaec6ac640 1 -- 172.21.15.16:0/2502936178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae41f7900 con 0x7fbae406c1f0 2024-04-25T10:09:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.658+0000 7fbadaffd640 1 -- 172.21.15.16:0/2502936178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0004bd0 con 0x7fbae406c1f0 2024-04-25T10:09:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.658+0000 7fbaec6ac640 1 -- 172.21.15.16:0/2502936178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae41f7ec0 con 0x7fbae406c1f0 2024-04-25T10:09:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.659+0000 7fbaec6ac640 1 -- 172.21.15.16:0/2502936178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae40749b0 con 0x7fbae406c1f0 2024-04-25T10:09:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.660+0000 7fbadaffd640 1 -- 172.21.15.16:0/2502936178 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbae0011070 con 0x7fbae406c1f0 2024-04-25T10:09:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.660+0000 7fbadaffd640 1 --2- 172.21.15.16:0/2502936178 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbac806b500 0x7fbac806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.660+0000 7fbadaffd640 1 -- 172.21.15.16:0/2502936178 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbae0089e70 con 0x7fbae406c1f0 2024-04-25T10:09:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.660+0000 7fbae9c47640 1 --2- 172.21.15.16:0/2502936178 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbac806b500 0x7fbac806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.661+0000 7fbae9c47640 1 --2- 172.21.15.16:0/2502936178 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbac806b500 0x7fbac806d9c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbae40cf190 tx=0x7fbad4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.664+0000 7fbadaffd640 1 -- 172.21.15.16:0/2502936178 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbae406c680 con 0x7fbae406c1f0 2024-04-25T10:09:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.887+0000 7fbaec6ac640 1 -- 172.21.15.16:0/2502936178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbae409a570 con 0x7fbae406c1f0 2024-04-25T10:09:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.887+0000 7fbadaffd640 1 -- 172.21.15.16:0/2502936178 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fbae0058cf0 con 0x7fbae406c1f0 2024-04-25T10:09:30.893 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:30.893 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6070,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2805428272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2805428272},{"type":"v1","addr":"172.21.15.186:6835","nonce":2805428272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2897154125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2897154125},{"type":"v1","addr":"172.21.15.16:6833","nonce":2897154125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6636,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1735811292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1735811292},{"type":"v1","addr":"172.21.15.16:6835","nonce":1735811292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:22.077032+0000","modified":"2024-04-25T10:09:24.068248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35489":{"gid":35489,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/3292077067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3292077067},{"type":"v1","addr":"172.21.15.186:6837","nonce":3292077067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:09:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 36 2024-04-25T10:09:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.889+0000 7fbaec6ac640 1 -- 172.21.15.16:0/2502936178 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbac806b500 msgr2=0x7fbac806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.889+0000 7fbaec6ac640 1 --2- 172.21.15.16:0/2502936178 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbac806b500 0x7fbac806d9c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbae40cf190 tx=0x7fbad4008040 comp rx=0 tx=0).stop 2024-04-25T10:09:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.889+0000 7fbaec6ac640 1 -- 172.21.15.16:0/2502936178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae406c1f0 msgr2=0x7fbae41f48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.889+0000 7fbaec6ac640 1 --2- 172.21.15.16:0/2502936178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae406c1f0 0x7fbae41f48a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fbae000c430 tx=0x7fbae000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.889+0000 7fbaec6ac640 1 -- 172.21.15.16:0/2502936178 shutdown_connections 2024-04-25T10:09:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.889+0000 7fbaec6ac640 1 --2- 172.21.15.16:0/2502936178 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbac806b500 0x7fbac806d9c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.889+0000 7fbaec6ac640 1 --2- 172.21.15.16:0/2502936178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbae415ba10 0x7fbae40b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.889+0000 7fbaec6ac640 1 --2- 172.21.15.16:0/2502936178 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbae409a150 0x7fbae41500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.889+0000 7fbaec6ac640 1 --2- 172.21.15.16:0/2502936178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae406c1f0 0x7fbae41f48a0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.889+0000 7fbaec6ac640 1 -- 172.21.15.16:0/2502936178 >> 172.21.15.16:0/2502936178 conn(0x7fbae40b32f0 msgr2=0x7fbae415ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.889+0000 7fbaec6ac640 1 -- 172.21.15.16:0/2502936178 shutdown_connections 2024-04-25T10:09:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:30.889+0000 7fbaec6ac640 1 -- 172.21.15.16:0/2502936178 wait complete. 2024-04-25T10:09:30.906 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-25T10:09:31.043 INFO:teuthology.orchestra.run.smithi186.stdout:{"message":"","return_code":0} 2024-04-25T10:09:31.043 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:09:31.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:09:31.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.139+0000 7ff27f75c640 1 Processor -- start 2024-04-25T10:09:31.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.140+0000 7ff27f75c640 1 -- start start 2024-04-25T10:09:31.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.140+0000 7ff27f75c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2780b9720 0x7ff2780ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.140+0000 7ff27f75c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff2780cece0 0x7ff2780d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.140+0000 7ff27f75c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff278161f80 con 0x7ff278161090 2024-04-25T10:09:31.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.140+0000 7ff27f75c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2780cf130 con 0x7ff2780cece0 2024-04-25T10:09:31.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.140+0000 7ff27f75c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2780d21a0 con 0x7ff2780b9720 2024-04-25T10:09:31.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.141+0000 7ff27d4f8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff278161090 0x7ff278161730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50520/0 (socket says 172.21.15.16:50520) 2024-04-25T10:09:31.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.141+0000 7ff27d4f8640 1 -- 172.21.15.16:0/3687005698 learned_addr learned my addr 172.21.15.16:0/3687005698 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:31.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.141+0000 7ff27ccf7640 1 --2- 172.21.15.16:0/3687005698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2780b9720 0x7ff2780ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.141+0000 7ff27dcf9640 1 --2- 172.21.15.16:0/3687005698 >> v2:172.21.15.186:3300/0 conn(0x7ff2780cece0 0x7ff2780d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.141+0000 7ff277fff640 1 -- 172.21.15.16:0/3687005698 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2423430259 0 0) 0x7ff278161f80 con 0x7ff278161090 2024-04-25T10:09:31.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.141+0000 7ff277fff640 1 -- 172.21.15.16:0/3687005698 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff260003520 con 0x7ff278161090 2024-04-25T10:09:31.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.141+0000 7ff27ccf7640 1 -- 172.21.15.16:0/3687005698 >> v2:172.21.15.186:3300/0 conn(0x7ff2780cece0 msgr2=0x7ff2780d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:31.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.141+0000 7ff27ccf7640 1 --2- 172.21.15.16:0/3687005698 >> v2:172.21.15.186:3300/0 conn(0x7ff2780cece0 0x7ff2780d2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.141+0000 7ff27ccf7640 1 -- 172.21.15.16:0/3687005698 >> v1:172.21.15.16:6789/0 conn(0x7ff278161090 legacy=0x7ff278161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.141+0000 7ff277fff640 1 -- 172.21.15.16:0/3687005698 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1113945508 0 0) 0x7ff260003520 con 0x7ff278161090 2024-04-25T10:09:31.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.141+0000 7ff27ccf7640 1 -- 172.21.15.16:0/3687005698 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff278162260 con 0x7ff2780b9720 2024-04-25T10:09:31.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.142+0000 7ff27ccf7640 1 --2- 172.21.15.16:0/3687005698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2780b9720 0x7ff2780ce7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0097a0 tx=0x7ff26c02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e805161f2bce0217 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.142+0000 7ff277fff640 1 -- 172.21.15.16:0/3687005698 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c037070 con 0x7ff2780b9720 2024-04-25T10:09:31.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.142+0000 7ff277fff640 1 -- 172.21.15.16:0/3687005698 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26c02aba0 con 0x7ff2780b9720 2024-04-25T10:09:31.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.142+0000 7ff277fff640 1 -- 172.21.15.16:0/3687005698 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2700019f0 con 0x7ff2780b9720 2024-04-25T10:09:31.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.142+0000 7ff27f75c640 1 -- 172.21.15.16:0/3687005698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2780b9720 msgr2=0x7ff2780ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.142+0000 7ff27f75c640 1 --2- 172.21.15.16:0/3687005698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2780b9720 0x7ff2780ce7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0097a0 tx=0x7ff26c02a110 comp rx=0 tx=0).stop 2024-04-25T10:09:31.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.143+0000 7ff27f75c640 1 -- 172.21.15.16:0/3687005698 shutdown_connections 2024-04-25T10:09:31.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.143+0000 7ff27f75c640 1 --2- 172.21.15.16:0/3687005698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2780b9720 0x7ff2780ce7a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.143+0000 7ff27f75c640 1 --2- 172.21.15.16:0/3687005698 >> v2:172.21.15.186:3300/0 conn(0x7ff2780cece0 0x7ff2780d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.143+0000 7ff27f75c640 1 -- 172.21.15.16:0/3687005698 >> 172.21.15.16:0/3687005698 conn(0x7ff2780b55e0 msgr2=0x7ff2780b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:31.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.143+0000 7ff27f75c640 1 -- 172.21.15.16:0/3687005698 shutdown_connections 2024-04-25T10:09:31.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.143+0000 7ff27f75c640 1 -- 172.21.15.16:0/3687005698 wait complete. 2024-04-25T10:09:31.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.144+0000 7ff27f75c640 1 Processor -- start 2024-04-25T10:09:31.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.144+0000 7ff27f75c640 1 -- start start 2024-04-25T10:09:31.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.144+0000 7ff27f75c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2780b9720 0x7ff278157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.144+0000 7ff27f75c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2780cece0 0x7ff2781f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.144+0000 7ff27f75c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff278161090 0x7ff2780ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.144+0000 7ff27f75c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff278161470 con 0x7ff2780cece0 2024-04-25T10:09:31.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.144+0000 7ff27f75c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2780d1e30 con 0x7ff2780b9720 2024-04-25T10:09:31.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.145+0000 7ff27f75c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2780d2b70 con 0x7ff278161090 2024-04-25T10:09:31.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.145+0000 7ff27ccf7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2780cece0 0x7ff2781f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.145+0000 7ff27ccf7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2780cece0 0x7ff2781f3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52682/0 (socket says 172.21.15.16:52682) 2024-04-25T10:09:31.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.145+0000 7ff27ccf7640 1 -- 172.21.15.16:0/81891170 learned_addr learned my addr 172.21.15.16:0/81891170 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:31.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.145+0000 7ff27d4f8640 1 --2- 172.21.15.16:0/81891170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2780b9720 0x7ff278157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.145+0000 7ff27dcf9640 1 --2- 172.21.15.16:0/81891170 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff278161090 0x7ff2780ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.145+0000 7ff27ccf7640 1 -- 172.21.15.16:0/81891170 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff278161090 msgr2=0x7ff2780ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.145+0000 7ff27ccf7640 1 --2- 172.21.15.16:0/81891170 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff278161090 0x7ff2780ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.145+0000 7ff27ccf7640 1 -- 172.21.15.16:0/81891170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2780b9720 msgr2=0x7ff278157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.145+0000 7ff27ccf7640 1 --2- 172.21.15.16:0/81891170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2780b9720 0x7ff278157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.145+0000 7ff27ccf7640 1 -- 172.21.15.16:0/81891170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27815d9c0 con 0x7ff2780cece0 2024-04-25T10:09:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.145+0000 7ff27ccf7640 1 --2- 172.21.15.16:0/81891170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2780cece0 0x7ff2781f3cd0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0098c0 tx=0x7ff26c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.145+0000 7ff275ffb640 1 -- 172.21.15.16:0/81891170 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c041020 con 0x7ff2780cece0 2024-04-25T10:09:31.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.145+0000 7ff275ffb640 1 -- 172.21.15.16:0/81891170 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26c03d620 con 0x7ff2780cece0 2024-04-25T10:09:31.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.145+0000 7ff27f75c640 1 -- 172.21.15.16:0/81891170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2780b5030 con 0x7ff2780cece0 2024-04-25T10:09:31.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.145+0000 7ff275ffb640 1 -- 172.21.15.16:0/81891170 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c037040 con 0x7ff2780cece0 2024-04-25T10:09:31.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.146+0000 7ff27f75c640 1 -- 172.21.15.16:0/81891170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27815df40 con 0x7ff2780cece0 2024-04-25T10:09:31.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.146+0000 7ff27f75c640 1 -- 172.21.15.16:0/81891170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2780ce2c0 con 0x7ff2780cece0 2024-04-25T10:09:31.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.147+0000 7ff275ffb640 1 -- 172.21.15.16:0/81891170 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff26c044020 con 0x7ff2780cece0 2024-04-25T10:09:31.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.147+0000 7ff275ffb640 1 --2- 172.21.15.16:0/81891170 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff26406b4d0 0x7ff26406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.148+0000 7ff275ffb640 1 -- 172.21.15.16:0/81891170 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff26c0aa6b0 con 0x7ff2780cece0 2024-04-25T10:09:31.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.148+0000 7ff27d4f8640 1 --2- 172.21.15.16:0/81891170 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff26406b4d0 0x7ff26406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.148+0000 7ff27d4f8640 1 --2- 172.21.15.16:0/81891170 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff26406b4d0 0x7ff26406d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff270005c00 tx=0x7ff270009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.152+0000 7ff275ffb640 1 -- 172.21.15.16:0/81891170 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff2780ce2c0 con 0x7ff2780cece0 2024-04-25T10:09:31.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.373+0000 7ff27f75c640 1 -- 172.21.15.16:0/81891170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff27815b5e0 con 0x7ff2780cece0 2024-04-25T10:09:31.379 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-04-25T10:09:31.373+0000 7ff13f9bb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:09:31.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.374+0000 7ff275ffb640 1 -- 172.21.15.16:0/81891170 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff26c04b030 con 0x7ff2780cece0 2024-04-25T10:09:31.380 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:09:31.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.377+0000 7ff27f75c640 1 -- 172.21.15.16:0/81891170 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff26406b4d0 msgr2=0x7ff26406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.377+0000 7ff27f75c640 1 --2- 172.21.15.16:0/81891170 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff26406b4d0 0x7ff26406d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff270005c00 tx=0x7ff270009040 comp rx=0 tx=0).stop 2024-04-25T10:09:31.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.377+0000 7ff27f75c640 1 -- 172.21.15.16:0/81891170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2780cece0 msgr2=0x7ff2781f3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.377+0000 7ff27f75c640 1 --2- 172.21.15.16:0/81891170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2780cece0 0x7ff2781f3cd0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0098c0 tx=0x7ff26c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.377+0000 7ff27f75c640 1 -- 172.21.15.16:0/81891170 shutdown_connections 2024-04-25T10:09:31.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.377+0000 7ff27f75c640 1 --2- 172.21.15.16:0/81891170 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff26406b4d0 0x7ff26406d990 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.377+0000 7ff27f75c640 1 --2- 172.21.15.16:0/81891170 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff278161090 0x7ff2780ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.377+0000 7ff27f75c640 1 --2- 172.21.15.16:0/81891170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2780cece0 0x7ff2781f3cd0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.377+0000 7ff27f75c640 1 --2- 172.21.15.16:0/81891170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2780b9720 0x7ff278157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.377+0000 7ff27f75c640 1 -- 172.21.15.16:0/81891170 >> 172.21.15.16:0/81891170 conn(0x7ff2780b55e0 msgr2=0x7ff2780ce500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.378+0000 7ff27f75c640 1 -- 172.21.15.16:0/81891170 shutdown_connections 2024-04-25T10:09:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.378+0000 7ff27f75c640 1 -- 172.21.15.16:0/81891170 wait complete. 2024-04-25T10:09:31.385 INFO:tasks.ceph.mds.d.smithi186.stderr: -4416> 2024-04-25T10:09:24.695+0000 7fcee278a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:09:31.395 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T10:09:31.417 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2024-04-25T10:09:31.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.433+0000 7f9db49b7f40 1 Processor -- start 2024-04-25T10:09:31.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.433+0000 7f9db49b7f40 1 -- start start 2024-04-25T10:09:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.434+0000 7f9db49b7f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9da0003ab0 0x55ad72059b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.434+0000 7f9db49b7f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ad721052d0 0x55ad72058760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.434+0000 7f9db49b7f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad720e87d0 0x55ad720e7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.434+0000 7f9db49b7f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad71fd5df0 con 0x7f9da0003ab0 2024-04-25T10:09:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.434+0000 7f9db49b7f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad71fd9d50 con 0x55ad720e87d0 2024-04-25T10:09:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.434+0000 7f9db49b7f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad72078ea0 con 0x55ad721052d0 2024-04-25T10:09:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.434+0000 7f9db24ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9da0003ab0 0x55ad72059b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.434+0000 7f9db24ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9da0003ab0 0x55ad72059b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52692/0 (socket says 172.21.15.16:52692) 2024-04-25T10:09:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.434+0000 7f9db24ba640 1 -- 172.21.15.16:0/262894261 learned_addr learned my addr 172.21.15.16:0/262894261 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.434+0000 7f9db1cb9640 1 --2- 172.21.15.16:0/262894261 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ad721052d0 0x55ad72058760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.434+0000 7f9db2cbb640 1 --2- 172.21.15.16:0/262894261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad720e87d0 0x55ad720e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.434+0000 7f9db24ba640 1 -- 172.21.15.16:0/262894261 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ad721052d0 msgr2=0x55ad72058760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.434+0000 7f9db24ba640 1 --2- 172.21.15.16:0/262894261 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ad721052d0 0x55ad72058760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.434+0000 7f9db24ba640 1 -- 172.21.15.16:0/262894261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad720e87d0 msgr2=0x55ad720e7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.434+0000 7f9db24ba640 1 --2- 172.21.15.16:0/262894261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad720e87d0 0x55ad720e7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.434+0000 7f9db24ba640 1 -- 172.21.15.16:0/262894261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad7202f9f0 con 0x7f9da0003ab0 2024-04-25T10:09:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.434+0000 7f9db2cbb640 1 --2- 172.21.15.16:0/262894261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad720e87d0 0x55ad720e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.434+0000 7f9db24ba640 1 --2- 172.21.15.16:0/262894261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9da0003ab0 0x55ad72059b90 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9da400a7b0 tx=0x7f9da4005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26dc456df365c6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.435+0000 7f9db0cb7640 1 -- 172.21.15.16:0/262894261 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4017030 con 0x7f9da0003ab0 2024-04-25T10:09:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.435+0000 7f9db0cb7640 1 -- 172.21.15.16:0/262894261 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da400ae50 con 0x7f9da0003ab0 2024-04-25T10:09:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.435+0000 7f9db0cb7640 1 -- 172.21.15.16:0/262894261 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4010a50 con 0x7f9da0003ab0 2024-04-25T10:09:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.435+0000 7f9db49b7f40 1 -- 172.21.15.16:0/262894261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9da0003ab0 msgr2=0x55ad72059b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.435+0000 7f9db49b7f40 1 --2- 172.21.15.16:0/262894261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9da0003ab0 0x55ad72059b90 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9da400a7b0 tx=0x7f9da4005d80 comp rx=0 tx=0).stop 2024-04-25T10:09:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.435+0000 7f9db49b7f40 1 -- 172.21.15.16:0/262894261 shutdown_connections 2024-04-25T10:09:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.435+0000 7f9db49b7f40 1 --2- 172.21.15.16:0/262894261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad720e87d0 0x55ad720e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.435+0000 7f9db49b7f40 1 --2- 172.21.15.16:0/262894261 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ad721052d0 0x55ad72058760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.435+0000 7f9db49b7f40 1 --2- 172.21.15.16:0/262894261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9da0003ab0 0x55ad72059b90 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.435+0000 7f9db49b7f40 1 -- 172.21.15.16:0/262894261 >> 172.21.15.16:0/262894261 conn(0x55ad720593d0 msgr2=0x55ad720eab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.435+0000 7f9db49b7f40 1 -- 172.21.15.16:0/262894261 shutdown_connections 2024-04-25T10:09:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.435+0000 7f9db49b7f40 1 -- 172.21.15.16:0/262894261 wait complete. 2024-04-25T10:09:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.436+0000 7f9db49b7f40 1 Processor -- start 2024-04-25T10:09:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.436+0000 7f9db49b7f40 1 -- start start 2024-04-25T10:09:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.436+0000 7f9db49b7f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad720e87d0 0x55ad7205e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.436+0000 7f9db49b7f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ad721052d0 0x55ad720e5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.436+0000 7f9db49b7f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9da0003ab0 0x55ad720ee550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.436+0000 7f9db49b7f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad72031460 con 0x7f9da0003ab0 2024-04-25T10:09:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.436+0000 7f9db49b7f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7205a0d0 con 0x55ad720e87d0 2024-04-25T10:09:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.436+0000 7f9db49b7f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad720316b0 con 0x55ad721052d0 2024-04-25T10:09:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.436+0000 7f9db2cbb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9da0003ab0 0x55ad720ee550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.437+0000 7f9db2cbb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9da0003ab0 0x55ad720ee550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52698/0 (socket says 172.21.15.16:52698) 2024-04-25T10:09:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.437+0000 7f9db2cbb640 1 -- 172.21.15.16:0/2078507249 learned_addr learned my addr 172.21.15.16:0/2078507249 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.437+0000 7f9db24ba640 1 --2- 172.21.15.16:0/2078507249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad720e87d0 0x55ad7205e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.437+0000 7f9db1cb9640 1 --2- 172.21.15.16:0/2078507249 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ad721052d0 0x55ad720e5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.437+0000 7f9db2cbb640 1 -- 172.21.15.16:0/2078507249 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ad721052d0 msgr2=0x55ad720e5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.437+0000 7f9db2cbb640 1 --2- 172.21.15.16:0/2078507249 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ad721052d0 0x55ad720e5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.437+0000 7f9db2cbb640 1 -- 172.21.15.16:0/2078507249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad720e87d0 msgr2=0x55ad7205e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.437+0000 7f9db2cbb640 1 --2- 172.21.15.16:0/2078507249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ad720e87d0 0x55ad7205e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.437+0000 7f9db2cbb640 1 -- 172.21.15.16:0/2078507249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad72030e10 con 0x7f9da0003ab0 2024-04-25T10:09:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.437+0000 7f9db2cbb640 1 --2- 172.21.15.16:0/2078507249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9da0003ab0 0x55ad720ee550 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0258a0 tx=0x7f9dac001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.437+0000 7f9dab7fe640 1 -- 172.21.15.16:0/2078507249 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac068070 con 0x7f9da0003ab0 2024-04-25T10:09:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.437+0000 7f9dab7fe640 1 -- 172.21.15.16:0/2078507249 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac055440 con 0x7f9da0003ab0 2024-04-25T10:09:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.437+0000 7f9db49b7f40 1 -- 172.21.15.16:0/2078507249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ad720e8150 con 0x7f9da0003ab0 2024-04-25T10:09:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.437+0000 7f9dab7fe640 1 -- 172.21.15.16:0/2078507249 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac05a040 con 0x7f9da0003ab0 2024-04-25T10:09:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.437+0000 7f9db49b7f40 1 -- 172.21.15.16:0/2078507249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad720e6180 con 0x7f9da0003ab0 2024-04-25T10:09:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.439+0000 7f9dab7fe640 1 -- 172.21.15.16:0/2078507249 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9dac06b070 con 0x7f9da0003ab0 2024-04-25T10:09:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.439+0000 7f9dab7fe640 1 --2- 172.21.15.16:0/2078507249 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9d94069eb0 0x7f9d9406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.439+0000 7f9dab7fe640 1 -- 172.21.15.16:0/2078507249 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9dac0cb790 con 0x7f9da0003ab0 2024-04-25T10:09:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.439+0000 7f9db24ba640 1 --2- 172.21.15.16:0/2078507249 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9d94069eb0 0x7f9d9406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.439+0000 7f9db49b7f40 1 --2- 172.21.15.16:0/2078507249 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55ad720eea90 0x55ad72107780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.439+0000 7f9db49b7f40 1 -- 172.21.15.16:0/2078507249 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73) v9 -- 0x55ad72107cc0 con 0x55ad720eea90 2024-04-25T10:09:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.439+0000 7f9db1cb9640 1 --2- 172.21.15.16:0/2078507249 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55ad720eea90 0x55ad72107780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.440+0000 7f9db24ba640 1 --2- 172.21.15.16:0/2078507249 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9d94069eb0 0x7f9d9406c370 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9da4000f80 tx=0x7f9da4015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.440+0000 7f9db1cb9640 1 --2- 172.21.15.16:0/2078507249 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55ad720eea90 0x55ad72107780 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.441+0000 7f9db1cb9640 1 -- 172.21.15.16:0/2078507249 <== osd.1 v2:172.21.15.16:6808/1916704159 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v73'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9d9c009980 con 0x55ad720eea90 2024-04-25T10:09:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.441+0000 7f9db49b7f40 1 -- 172.21.15.16:0/2078507249 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55ad720eea90 msgr2=0x55ad72107780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.441+0000 7f9db49b7f40 1 --2- 172.21.15.16:0/2078507249 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55ad720eea90 0x55ad72107780 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.441+0000 7f9db49b7f40 1 -- 172.21.15.16:0/2078507249 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9d94069eb0 msgr2=0x7f9d9406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.441+0000 7f9db49b7f40 1 --2- 172.21.15.16:0/2078507249 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9d94069eb0 0x7f9d9406c370 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9da4000f80 tx=0x7f9da4015000 comp rx=0 tx=0).stop 2024-04-25T10:09:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.441+0000 7f9db49b7f40 1 -- 172.21.15.16:0/2078507249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9da0003ab0 msgr2=0x55ad720ee550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.441+0000 7f9db49b7f40 1 --2- 172.21.15.16:0/2078507249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9da0003ab0 0x55ad720ee550 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0258a0 tx=0x7f9dac001120 comp rx=0 tx=0).stop 2024-04-25T10:09:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.441+0000 7f9db2cbb640 1 -- 172.21.15.16:0/2078507249 reap_dead start 2024-04-25T10:09:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.441+0000 7f9db49b7f40 1 -- 172.21.15.16:0/2078507249 shutdown_connections 2024-04-25T10:09:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.441+0000 7f9db49b7f40 1 -- 172.21.15.16:0/2078507249 >> 172.21.15.16:0/2078507249 conn(0x55ad720593d0 msgr2=0x55ad72030560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.441+0000 7f9db49b7f40 1 -- 172.21.15.16:0/2078507249 shutdown_connections 2024-04-25T10:09:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.441+0000 7f9db49b7f40 1 -- 172.21.15.16:0/2078507249 wait complete. 2024-04-25T10:09:31.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:09:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.550+0000 7f15b1cf5640 1 Processor -- start 2024-04-25T10:09:31.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.550+0000 7f15b1cf5640 1 -- start start 2024-04-25T10:09:31.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.551+0000 7f15b1cf5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15ac15e040 0x7f15ac0cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.551+0000 7f15b1cf5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f15ac0cee20 0x7f15ac0cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.551+0000 7f15b1cf5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15ac0cc7e0 con 0x7f15ac0ce410 2024-04-25T10:09:31.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.551+0000 7f15b1cf5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ac0cb350 con 0x7f15ac0cee20 2024-04-25T10:09:31.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.551+0000 7f15b1cf5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac0ce9d0 con 0x7f15ac15e040 2024-04-25T10:09:31.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.551+0000 7f15aaffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f15ac0ce410 0x7f15ac0cc0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50548/0 (socket says 172.21.15.16:50548) 2024-04-25T10:09:31.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.551+0000 7f15aaffd640 1 -- 172.21.15.16:0/3130952273 learned_addr learned my addr 172.21.15.16:0/3130952273 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:31.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15abfff640 1 --2- 172.21.15.16:0/3130952273 >> v2:172.21.15.186:3300/0 conn(0x7f15ac0cee20 0x7f15ac0cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aa7fc640 1 -- 172.21.15.16:0/3130952273 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 509771100 0 0) 0x7f15ac0cc7e0 con 0x7f15ac0ce410 2024-04-25T10:09:31.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15ab7fe640 1 --2- 172.21.15.16:0/3130952273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15ac15e040 0x7f15ac0cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aa7fc640 1 -- 172.21.15.16:0/3130952273 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1594005730 con 0x7f15ac0ce410 2024-04-25T10:09:31.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aa7fc640 1 -- 172.21.15.16:0/3130952273 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1913834629 0 0) 0x7f1594005730 con 0x7f15ac0ce410 2024-04-25T10:09:31.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aa7fc640 1 -- 172.21.15.16:0/3130952273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15ac15e040 msgr2=0x7f15ac0cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:31.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aa7fc640 1 --2- 172.21.15.16:0/3130952273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15ac15e040 0x7f15ac0cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aa7fc640 1 -- 172.21.15.16:0/3130952273 >> v2:172.21.15.186:3300/0 conn(0x7f15ac0cee20 msgr2=0x7f15ac0cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:31.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aa7fc640 1 --2- 172.21.15.16:0/3130952273 >> v2:172.21.15.186:3300/0 conn(0x7f15ac0cee20 0x7f15ac0cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aa7fc640 1 -- 172.21.15.16:0/3130952273 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac0cb5f0 con 0x7f15ac0ce410 2024-04-25T10:09:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aa7fc640 1 -- 172.21.15.16:0/3130952273 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f15a0001670 con 0x7f15ac0ce410 2024-04-25T10:09:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aa7fc640 1 -- 172.21.15.16:0/3130952273 >> v1:172.21.15.16:6789/0 conn(0x7f15ac0ce410 legacy=0x7f15ac0cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aa7fc640 1 --2- 172.21.15.16:0/3130952273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15ac15e040 0x7f1594005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aa7fc640 1 --2- 172.21.15.16:0/3130952273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1594006140 0x7f1594006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aa7fc640 1 --2- 172.21.15.16:0/3130952273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1594006f50 0x7f15940073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aa7fc640 1 -- 172.21.15.16:0/3130952273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac0cb350 con 0x7f1594006140 2024-04-25T10:09:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aa7fc640 1 -- 172.21.15.16:0/3130952273 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac0ce9d0 con 0x7f1594006f50 2024-04-25T10:09:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aa7fc640 1 -- 172.21.15.16:0/3130952273 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159400dbd0 con 0x7f15ac15e040 2024-04-25T10:09:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aaffd640 1 --2- 172.21.15.16:0/3130952273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1594006140 0x7f1594006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15abfff640 1 --2- 172.21.15.16:0/3130952273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1594006f50 0x7f15940073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15ab7fe640 1 --2- 172.21.15.16:0/3130952273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15ac15e040 0x7f1594005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aaffd640 1 -- 172.21.15.16:0/3130952273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15ac15e040 msgr2=0x7f1594005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aaffd640 1 --2- 172.21.15.16:0/3130952273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15ac15e040 0x7f1594005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aaffd640 1 -- 172.21.15.16:0/3130952273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1594006f50 msgr2=0x7f15940073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aaffd640 1 --2- 172.21.15.16:0/3130952273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1594006f50 0x7f15940073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15aaffd640 1 -- 172.21.15.16:0/3130952273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1594005730 con 0x7f1594006140 2024-04-25T10:09:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.552+0000 7f15ab7fe640 1 --2- 172.21.15.16:0/3130952273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15ac15e040 0x7f1594005bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.553+0000 7f15aaffd640 1 --2- 172.21.15.16:0/3130952273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1594006140 0x7f1594006a10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f15a0007fd0 tx=0x7f1594007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe295ff02411a3dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.553+0000 7f15aa7fc640 1 -- 172.21.15.16:0/3130952273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a0007d50 con 0x7f1594006140 2024-04-25T10:09:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.553+0000 7f15aa7fc640 1 -- 172.21.15.16:0/3130952273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a0005ad0 con 0x7f1594006140 2024-04-25T10:09:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.553+0000 7f15aa7fc640 1 -- 172.21.15.16:0/3130952273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a0007070 con 0x7f1594006140 2024-04-25T10:09:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.554+0000 7f15b1cf5640 1 -- 172.21.15.16:0/3130952273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1594006140 msgr2=0x7f1594006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.554+0000 7f15b1cf5640 1 --2- 172.21.15.16:0/3130952273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1594006140 0x7f1594006a10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f15a0007fd0 tx=0x7f1594007a40 comp rx=0 tx=0).stop 2024-04-25T10:09:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.554+0000 7f15abfff640 1 -- 172.21.15.16:0/3130952273 reap_dead start 2024-04-25T10:09:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.554+0000 7f15b1cf5640 1 -- 172.21.15.16:0/3130952273 shutdown_connections 2024-04-25T10:09:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.554+0000 7f15b1cf5640 1 -- 172.21.15.16:0/3130952273 >> 172.21.15.16:0/3130952273 conn(0x7f15ac0b2390 msgr2=0x7f15ac0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.554+0000 7f15b1cf5640 1 -- 172.21.15.16:0/3130952273 shutdown_connections 2024-04-25T10:09:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.554+0000 7f15b1cf5640 1 -- 172.21.15.16:0/3130952273 wait complete. 2024-04-25T10:09:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.555+0000 7f15b1cf5640 1 Processor -- start 2024-04-25T10:09:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.555+0000 7f15b1cf5640 1 -- start start 2024-04-25T10:09:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.556+0000 7f15b1cf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1594006140 0x7f15ac0b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.556+0000 7f15b1cf5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15ac0c9ca0 0x7f15ac0c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.556+0000 7f15b1cf5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15ac06f590 0x7f15ac06fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.556+0000 7f15b1cf5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac15e420 con 0x7f1594006140 2024-04-25T10:09:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.556+0000 7f15b1cf5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac0cbbb0 con 0x7f15ac0c9ca0 2024-04-25T10:09:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.556+0000 7f15b1cf5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac0cdbb0 con 0x7f15ac06f590 2024-04-25T10:09:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.556+0000 7f15ab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1594006140 0x7f15ac0b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.556+0000 7f15ab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1594006140 0x7f15ac0b5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52720/0 (socket says 172.21.15.16:52720) 2024-04-25T10:09:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.556+0000 7f15ab7fe640 1 -- 172.21.15.16:0/4058802911 learned_addr learned my addr 172.21.15.16:0/4058802911 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.556+0000 7f15aaffd640 1 --2- 172.21.15.16:0/4058802911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15ac0c9ca0 0x7f15ac0c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.556+0000 7f15abfff640 1 --2- 172.21.15.16:0/4058802911 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15ac06f590 0x7f15ac06fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.556+0000 7f15ab7fe640 1 -- 172.21.15.16:0/4058802911 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15ac06f590 msgr2=0x7f15ac06fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.556+0000 7f15ab7fe640 1 --2- 172.21.15.16:0/4058802911 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15ac06f590 0x7f15ac06fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.556+0000 7f15ab7fe640 1 -- 172.21.15.16:0/4058802911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15ac0c9ca0 msgr2=0x7f15ac0c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.556+0000 7f15ab7fe640 1 --2- 172.21.15.16:0/4058802911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15ac0c9ca0 0x7f15ac0c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.556+0000 7f15ab7fe640 1 -- 172.21.15.16:0/4058802911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac0c4db0 con 0x7f1594006140 2024-04-25T10:09:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.556+0000 7f15ab7fe640 1 --2- 172.21.15.16:0/4058802911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1594006140 0x7f15ac0b5fb0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f159c011600 tx=0x7f159c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.557+0000 7f1593fff640 1 -- 172.21.15.16:0/4058802911 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c00f920 con 0x7f1594006140 2024-04-25T10:09:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.557+0000 7f1593fff640 1 -- 172.21.15.16:0/4058802911 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159c00fac0 con 0x7f1594006140 2024-04-25T10:09:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.557+0000 7f1593fff640 1 -- 172.21.15.16:0/4058802911 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c00fe90 con 0x7f1594006140 2024-04-25T10:09:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.557+0000 7f15b1cf5640 1 -- 172.21.15.16:0/4058802911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ac0c5090 con 0x7f1594006140 2024-04-25T10:09:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.557+0000 7f15b1cf5640 1 -- 172.21.15.16:0/4058802911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ac0c4ac0 con 0x7f1594006140 2024-04-25T10:09:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.558+0000 7f15b1cf5640 1 -- 172.21.15.16:0/4058802911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ac0750b0 con 0x7f1594006140 2024-04-25T10:09:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.558+0000 7f1593fff640 1 -- 172.21.15.16:0/4058802911 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f159c015460 con 0x7f1594006140 2024-04-25T10:09:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.559+0000 7f1593fff640 1 --2- 172.21.15.16:0/4058802911 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f159806b670 0x7f159806db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.559+0000 7f1593fff640 1 -- 172.21.15.16:0/4058802911 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f159c08aa80 con 0x7f1594006140 2024-04-25T10:09:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.559+0000 7f15aaffd640 1 --2- 172.21.15.16:0/4058802911 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f159806b670 0x7f159806db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.559+0000 7f15aaffd640 1 --2- 172.21.15.16:0/4058802911 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f159806b670 0x7f159806db30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f15a0033000 tx=0x7f15a0009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.563+0000 7f1593fff640 1 -- 172.21.15.16:0/4058802911 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f159c019030 con 0x7f1594006140 2024-04-25T10:09:31.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:31.780+0000 7f15b1cf5640 1 -- 172.21.15.16:0/4058802911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f15ac0d0620 con 0x7f1594006140 2024-04-25T10:09:32.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.376+0000 7f1593fff640 1 -- 172.21.15.16:0/4058802911 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v38) v1 ==== 162+0+0 (secure 0 0 0) 0x7f159c059930 con 0x7f1594006140 2024-04-25T10:09:32.382 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:09:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.378+0000 7f15b1cf5640 1 -- 172.21.15.16:0/4058802911 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f159806b670 msgr2=0x7f159806db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.378+0000 7f15b1cf5640 1 --2- 172.21.15.16:0/4058802911 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f159806b670 0x7f159806db30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f15a0033000 tx=0x7f15a0009030 comp rx=0 tx=0).stop 2024-04-25T10:09:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.378+0000 7f15b1cf5640 1 -- 172.21.15.16:0/4058802911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1594006140 msgr2=0x7f15ac0b5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.378+0000 7f15b1cf5640 1 --2- 172.21.15.16:0/4058802911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1594006140 0x7f15ac0b5fb0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f159c011600 tx=0x7f159c011ac0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.378+0000 7f15b1cf5640 1 -- 172.21.15.16:0/4058802911 shutdown_connections 2024-04-25T10:09:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.378+0000 7f15b1cf5640 1 --2- 172.21.15.16:0/4058802911 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f159806b670 0x7f159806db30 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.378+0000 7f15b1cf5640 1 --2- 172.21.15.16:0/4058802911 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15ac06f590 0x7f15ac06fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.378+0000 7f15b1cf5640 1 --2- 172.21.15.16:0/4058802911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15ac0c9ca0 0x7f15ac0c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.378+0000 7f15b1cf5640 1 --2- 172.21.15.16:0/4058802911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1594006140 0x7f15ac0b5fb0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.378+0000 7f15b1cf5640 1 -- 172.21.15.16:0/4058802911 >> 172.21.15.16:0/4058802911 conn(0x7f15ac0b2390 msgr2=0x7f15ac0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:32.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.379+0000 7f15b1cf5640 1 -- 172.21.15.16:0/4058802911 shutdown_connections 2024-04-25T10:09:32.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.379+0000 7f15b1cf5640 1 -- 172.21.15.16:0/4058802911 wait complete. 2024-04-25T10:09:32.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:32.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.501+0000 7f1c04be0640 1 Processor -- start 2024-04-25T10:09:32.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.501+0000 7f1c04be0640 1 -- start start 2024-04-25T10:09:32.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.502+0000 7f1c04be0640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1c0015bb90 0x7f1c00160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.502+0000 7f1c04be0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c000cc950 0x7f1c000b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.502+0000 7f1c04be0640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c0006b310 con 0x7f1c0006aba0 2024-04-25T10:09:32.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.502+0000 7f1c04be0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c000b0010 con 0x7f1c000cc950 2024-04-25T10:09:32.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.502+0000 7f1c04be0640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c0006b5f0 con 0x7f1c0015bb90 2024-04-25T10:09:32.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.502+0000 7f1bfed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c000cc950 0x7f1c000b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.502+0000 7f1bfed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c000cc950 0x7f1c000b0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52736/0 (socket says 172.21.15.16:52736) 2024-04-25T10:09:32.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.502+0000 7f1bfed9d640 1 -- 172.21.15.16:0/2231905097 learned_addr learned my addr 172.21.15.16:0/2231905097 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:32.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.502+0000 7f1bfe59c640 1 --2- 172.21.15.16:0/2231905097 >> v2:172.21.15.186:3300/0 conn(0x7f1c0015bb90 0x7f1c00160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.502+0000 7f1bfed9d640 1 -- 172.21.15.16:0/2231905097 >> v2:172.21.15.186:3300/0 conn(0x7f1c0015bb90 msgr2=0x7f1c00160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:32.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.502+0000 7f1bfed9d640 1 --2- 172.21.15.16:0/2231905097 >> v2:172.21.15.186:3300/0 conn(0x7f1c0015bb90 0x7f1c00160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.502+0000 7f1bfed9d640 1 -- 172.21.15.16:0/2231905097 >> v1:172.21.15.186:6789/0 conn(0x7f1c0006aba0 legacy=0x7f1c000cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:32.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.502+0000 7f1bfed9d640 1 -- 172.21.15.16:0/2231905097 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c000b0260 con 0x7f1c000cc950 2024-04-25T10:09:32.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.502+0000 7f1bfe59c640 1 --2- 172.21.15.16:0/2231905097 >> v2:172.21.15.186:3300/0 conn(0x7f1c0015bb90 0x7f1c00160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:32.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.503+0000 7f1bfed9d640 1 --2- 172.21.15.16:0/2231905097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c000cc950 0x7f1c000b0a60 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f1c000ce160 tx=0x7f1bf802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcdd4219d0e7f11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:32.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.503+0000 7f1bfdd9b640 1 -- 172.21.15.16:0/2231905097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8036070 con 0x7f1c000cc950 2024-04-25T10:09:32.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.503+0000 7f1bfdd9b640 1 -- 172.21.15.16:0/2231905097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf8031600 con 0x7f1c000cc950 2024-04-25T10:09:32.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.503+0000 7f1bfdd9b640 1 -- 172.21.15.16:0/2231905097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80319d0 con 0x7f1c000cc950 2024-04-25T10:09:32.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.504+0000 7f1c04be0640 1 -- 172.21.15.16:0/2231905097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c000cc950 msgr2=0x7f1c000b0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:32.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.504+0000 7f1c04be0640 1 --2- 172.21.15.16:0/2231905097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c000cc950 0x7f1c000b0a60 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f1c000ce160 tx=0x7f1bf802a610 comp rx=0 tx=0).stop 2024-04-25T10:09:32.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.504+0000 7f1c04be0640 1 -- 172.21.15.16:0/2231905097 shutdown_connections 2024-04-25T10:09:32.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.504+0000 7f1c04be0640 1 --2- 172.21.15.16:0/2231905097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c000cc950 0x7f1c000b0a60 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.504+0000 7f1c04be0640 1 --2- 172.21.15.16:0/2231905097 >> v2:172.21.15.186:3300/0 conn(0x7f1c0015bb90 0x7f1c00160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.504+0000 7f1c04be0640 1 -- 172.21.15.16:0/2231905097 >> 172.21.15.16:0/2231905097 conn(0x7f1c000d06b0 msgr2=0x7f1c000d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:32.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.504+0000 7f1c04be0640 1 -- 172.21.15.16:0/2231905097 shutdown_connections 2024-04-25T10:09:32.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.504+0000 7f1c04be0640 1 -- 172.21.15.16:0/2231905097 wait complete. 2024-04-25T10:09:32.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.505+0000 7f1c04be0640 1 Processor -- start 2024-04-25T10:09:32.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.505+0000 7f1c04be0640 1 -- start start 2024-04-25T10:09:32.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.506+0000 7f1c04be0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c0006aba0 0x7f1c000b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.506+0000 7f1c04be0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c000cc950 0x7f1c001f1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.506+0000 7f1c04be0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c0015bb90 0x7f1c0006f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.506+0000 7f1c04be0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c000b0fa0 con 0x7f1c000cc950 2024-04-25T10:09:32.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.506+0000 7f1c04be0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0006b030 con 0x7f1c0015bb90 2024-04-25T10:09:32.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.506+0000 7f1c04be0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c000b1120 con 0x7f1c0006aba0 2024-04-25T10:09:32.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.506+0000 7f1bf7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c000cc950 0x7f1c001f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.506+0000 7f1bf7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c000cc950 0x7f1c001f1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52748/0 (socket says 172.21.15.16:52748) 2024-04-25T10:09:32.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.506+0000 7f1bf7fff640 1 -- 172.21.15.16:0/749358787 learned_addr learned my addr 172.21.15.16:0/749358787 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:32.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.506+0000 7f1bfed9d640 1 --2- 172.21.15.16:0/749358787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c0015bb90 0x7f1c0006f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.506+0000 7f1bfe59c640 1 --2- 172.21.15.16:0/749358787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c0006aba0 0x7f1c000b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.506+0000 7f1bf7fff640 1 -- 172.21.15.16:0/749358787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c0006aba0 msgr2=0x7f1c000b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.506+0000 7f1bf7fff640 1 --2- 172.21.15.16:0/749358787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c0006aba0 0x7f1c000b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.506+0000 7f1bf7fff640 1 -- 172.21.15.16:0/749358787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c0015bb90 msgr2=0x7f1c0006f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.506+0000 7f1bf7fff640 1 --2- 172.21.15.16:0/749358787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c0015bb90 0x7f1c0006f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.506+0000 7f1bf7fff640 1 -- 172.21.15.16:0/749358787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c001f1fa0 con 0x7f1c000cc950 2024-04-25T10:09:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.506+0000 7f1bfe59c640 1 --2- 172.21.15.16:0/749358787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c0006aba0 0x7f1c000b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.506+0000 7f1bf7fff640 1 --2- 172.21.15.16:0/749358787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c000cc950 0x7f1c001f1830 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f1be800c400 tx=0x7f1be800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.507+0000 7f1bf6ffd640 1 -- 172.21.15.16:0/749358787 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be80052d0 con 0x7f1c000cc950 2024-04-25T10:09:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.507+0000 7f1bf6ffd640 1 -- 172.21.15.16:0/749358787 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be8005490 con 0x7f1c000cc950 2024-04-25T10:09:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.507+0000 7f1c04be0640 1 -- 172.21.15.16:0/749358787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c001f2280 con 0x7f1c000cc950 2024-04-25T10:09:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.507+0000 7f1bf6ffd640 1 -- 172.21.15.16:0/749358787 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be8007260 con 0x7f1c000cc950 2024-04-25T10:09:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.507+0000 7f1c04be0640 1 -- 172.21.15.16:0/749358787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0006fb40 con 0x7f1c000cc950 2024-04-25T10:09:32.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.508+0000 7f1c04be0640 1 -- 172.21.15.16:0/749358787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c000749b0 con 0x7f1c000cc950 2024-04-25T10:09:32.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.508+0000 7f1bf6ffd640 1 -- 172.21.15.16:0/749358787 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1be8007400 con 0x7f1c000cc950 2024-04-25T10:09:32.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.508+0000 7f1bf6ffd640 1 --2- 172.21.15.16:0/749358787 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1be006b500 0x7f1be006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.509+0000 7f1bf6ffd640 1 -- 172.21.15.16:0/749358787 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1be8089740 con 0x7f1c000cc950 2024-04-25T10:09:32.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.509+0000 7f1bfe59c640 1 --2- 172.21.15.16:0/749358787 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1be006b500 0x7f1be006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.509+0000 7f1bfe59c640 1 --2- 172.21.15.16:0/749358787 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1be006b500 0x7f1be006d9c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0006fd0 tx=0x7f1bf0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:32.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.513+0000 7f1bf6ffd640 1 -- 172.21.15.16:0/749358787 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1be8058560 con 0x7f1c000cc950 2024-04-25T10:09:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.731+0000 7f1c04be0640 1 -- 172.21.15.16:0/749358787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c000724d0 con 0x7f1c000cc950 2024-04-25T10:09:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.732+0000 7f1bf6ffd640 1 -- 172.21.15.16:0/749358787 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f1be8057e60 con 0x7f1c000cc950 2024-04-25T10:09:32.738 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:32.738 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6070,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2805428272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2805428272},{"type":"v1","addr":"172.21.15.186:6835","nonce":2805428272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1052667029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1052667029},{"type":"v1","addr":"172.21.15.186:6837","nonce":1052667029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2897154125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2897154125},{"type":"v1","addr":"172.21.15.16:6833","nonce":2897154125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:22.077032+0000","modified":"2024-04-25T10:09:32.379459+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6636":{"gid":6636,"name":"a","rank":0,"incarnation":39,"state":"up:replay","state_seq":1,"addr":"172.21.15.16:6835/1735811292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1735811292},{"type":"v1","addr":"172.21.15.16:6835","nonce":1735811292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:09:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 39 2024-04-25T10:09:32.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.733+0000 7f1c04be0640 1 -- 172.21.15.16:0/749358787 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1be006b500 msgr2=0x7f1be006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:32.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.733+0000 7f1c04be0640 1 --2- 172.21.15.16:0/749358787 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1be006b500 0x7f1be006d9c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0006fd0 tx=0x7f1bf0008040 comp rx=0 tx=0).stop 2024-04-25T10:09:32.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.733+0000 7f1c04be0640 1 -- 172.21.15.16:0/749358787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c000cc950 msgr2=0x7f1c001f1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:32.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.733+0000 7f1c04be0640 1 --2- 172.21.15.16:0/749358787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c000cc950 0x7f1c001f1830 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f1be800c400 tx=0x7f1be800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.734+0000 7f1c04be0640 1 -- 172.21.15.16:0/749358787 shutdown_connections 2024-04-25T10:09:32.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.734+0000 7f1c04be0640 1 --2- 172.21.15.16:0/749358787 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1be006b500 0x7f1be006d9c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.734+0000 7f1c04be0640 1 --2- 172.21.15.16:0/749358787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c0015bb90 0x7f1c0006f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.734+0000 7f1c04be0640 1 --2- 172.21.15.16:0/749358787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c000cc950 0x7f1c001f1830 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.734+0000 7f1c04be0640 1 --2- 172.21.15.16:0/749358787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c0006aba0 0x7f1c000b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.734+0000 7f1c04be0640 1 -- 172.21.15.16:0/749358787 >> 172.21.15.16:0/749358787 conn(0x7f1c000d06b0 msgr2=0x7f1c00157ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.734+0000 7f1c04be0640 1 -- 172.21.15.16:0/749358787 shutdown_connections 2024-04-25T10:09:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:32.734+0000 7f1c04be0640 1 -- 172.21.15.16:0/749358787 wait complete. 2024-04-25T10:09:33.153 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:09:22.077032+0000', 'modified': '2024-04-25T10:09:32.379459+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 73, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6636}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6636': {'gid': 6636, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.16:6835/1735811292', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 1735811292}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 1735811292}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:09:33.153 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6636:up:replay 2024-04-25T10:09:34.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:34.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.261+0000 7f6fabd7e640 1 Processor -- start 2024-04-25T10:09:34.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.262+0000 7f6fabd7e640 1 -- start start 2024-04-25T10:09:34.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.262+0000 7f6fabd7e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fa40b5b60 0x7f6fa41687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:34.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.262+0000 7f6fabd7e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa415e0c0 con 0x7f6fa415dce0 2024-04-25T10:09:34.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.262+0000 7f6fabd7e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa40d2510 con 0x7f6fa40d35b0 2024-04-25T10:09:34.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.262+0000 7f6fabd7e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa40b7b10 con 0x7f6fa40b5b60 2024-04-25T10:09:34.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.263+0000 7f6faa31b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fa40b5b60 0x7f6fa41687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:34.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.263+0000 7f6fa9b1a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6fa415dce0 0x7f6fa40d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38900/0 (socket says 172.21.15.16:38900) 2024-04-25T10:09:34.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.263+0000 7f6fa9b1a640 1 -- 172.21.15.16:0/2042377529 learned_addr learned my addr 172.21.15.16:0/2042377529 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:34.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.263+0000 7f6faa31b640 1 -- 172.21.15.16:0/2042377529 >> v1:172.21.15.186:6789/0 conn(0x7f6fa40d35b0 legacy=0x7f6fa40b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:34.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.263+0000 7f6faa31b640 1 -- 172.21.15.16:0/2042377529 >> v1:172.21.15.16:6789/0 conn(0x7f6fa415dce0 legacy=0x7f6fa40d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:34.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.263+0000 7f6faa31b640 1 -- 172.21.15.16:0/2042377529 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa40d27f0 con 0x7f6fa40b5b60 2024-04-25T10:09:34.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.263+0000 7f6fa8b18640 1 -- 172.21.15.16:0/2042377529 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476541187 0 0) 0x7f6fa415e0c0 con 0x7f6fa415dce0 2024-04-25T10:09:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.264+0000 7f6faa31b640 1 --2- 172.21.15.16:0/2042377529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa40b5b60 0x7f6fa41687f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0097a0 tx=0x7f6f9c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8942d230e186ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.264+0000 7f6fa8b18640 1 -- 172.21.15.16:0/2042377529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c037070 con 0x7f6fa40b5b60 2024-04-25T10:09:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.264+0000 7f6fa8b18640 1 -- 172.21.15.16:0/2042377529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9c032440 con 0x7f6fa40b5b60 2024-04-25T10:09:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.264+0000 7f6fa8b18640 1 -- 172.21.15.16:0/2042377529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c032810 con 0x7f6fa40b5b60 2024-04-25T10:09:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.264+0000 7f6fabd7e640 1 -- 172.21.15.16:0/2042377529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa40b5b60 msgr2=0x7f6fa41687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.264+0000 7f6fabd7e640 1 --2- 172.21.15.16:0/2042377529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa40b5b60 0x7f6fa41687f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0097a0 tx=0x7f6f9c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.264+0000 7f6fabd7e640 1 -- 172.21.15.16:0/2042377529 shutdown_connections 2024-04-25T10:09:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.264+0000 7f6fabd7e640 1 --2- 172.21.15.16:0/2042377529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa40b5b60 0x7f6fa41687f0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.264+0000 7f6fabd7e640 1 -- 172.21.15.16:0/2042377529 >> 172.21.15.16:0/2042377529 conn(0x7f6fa4072900 msgr2=0x7f6fa40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.265+0000 7f6fabd7e640 1 -- 172.21.15.16:0/2042377529 shutdown_connections 2024-04-25T10:09:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.265+0000 7f6fabd7e640 1 -- 172.21.15.16:0/2042377529 wait complete. 2024-04-25T10:09:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.266+0000 7f6fabd7e640 1 Processor -- start 2024-04-25T10:09:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.266+0000 7f6fabd7e640 1 -- start start 2024-04-25T10:09:34.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.266+0000 7f6fabd7e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fa40b5b60 0x7f6fa40cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:34.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fabd7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa40d35b0 0x7f6fa40cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:34.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fabd7e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fa415dce0 0x7f6fa40cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fabd7e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa40d3090 con 0x7f6fa40d35b0 2024-04-25T10:09:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fabd7e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa40d2db0 con 0x7f6fa40b5b60 2024-04-25T10:09:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fabd7e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa40d2160 con 0x7f6fa415dce0 2024-04-25T10:09:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fa9319640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa40d35b0 0x7f6fa40cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fa9319640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa40d35b0 0x7f6fa40cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40548/0 (socket says 172.21.15.16:40548) 2024-04-25T10:09:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fa9319640 1 -- 172.21.15.16:0/1143067494 learned_addr learned my addr 172.21.15.16:0/1143067494 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fa9b1a640 1 --2- 172.21.15.16:0/1143067494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fa40b5b60 0x7f6fa40cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6faa31b640 1 --2- 172.21.15.16:0/1143067494 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fa415dce0 0x7f6fa40cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fa9319640 1 -- 172.21.15.16:0/1143067494 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fa415dce0 msgr2=0x7f6fa40cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fa9319640 1 --2- 172.21.15.16:0/1143067494 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fa415dce0 0x7f6fa40cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fa9319640 1 -- 172.21.15.16:0/1143067494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fa40b5b60 msgr2=0x7f6fa40cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fa9319640 1 --2- 172.21.15.16:0/1143067494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fa40b5b60 0x7f6fa40cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fa9319640 1 -- 172.21.15.16:0/1143067494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa4160050 con 0x7f6fa40d35b0 2024-04-25T10:09:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fa9b1a640 1 --2- 172.21.15.16:0/1143067494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fa40b5b60 0x7f6fa40cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fa9319640 1 --2- 172.21.15.16:0/1143067494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa40d35b0 0x7f6fa40cb590 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800c430 tx=0x7f6f9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fa27fc640 1 -- 172.21.15.16:0/1143067494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f98011790 con 0x7f6fa40d35b0 2024-04-25T10:09:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fa27fc640 1 -- 172.21.15.16:0/1143067494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f98017070 con 0x7f6fa40d35b0 2024-04-25T10:09:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fabd7e640 1 -- 172.21.15.16:0/1143067494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa41533b0 con 0x7f6fa40d35b0 2024-04-25T10:09:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.267+0000 7f6fa27fc640 1 -- 172.21.15.16:0/1143067494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f98011b70 con 0x7f6fa40d35b0 2024-04-25T10:09:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.268+0000 7f6fabd7e640 1 -- 172.21.15.16:0/1143067494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa415f570 con 0x7f6fa40d35b0 2024-04-25T10:09:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.269+0000 7f6fabd7e640 1 -- 172.21.15.16:0/1143067494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa40749b0 con 0x7f6fa40d35b0 2024-04-25T10:09:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.269+0000 7f6fa27fc640 1 -- 172.21.15.16:0/1143067494 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f98011d10 con 0x7f6fa40d35b0 2024-04-25T10:09:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.269+0000 7f6fa27fc640 1 --2- 172.21.15.16:0/1143067494 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f9006b500 0x7f6f9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.270+0000 7f6fa27fc640 1 -- 172.21.15.16:0/1143067494 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6f9808a010 con 0x7f6fa40d35b0 2024-04-25T10:09:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.270+0000 7f6fa9b1a640 1 --2- 172.21.15.16:0/1143067494 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f9006b500 0x7f6f9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.270+0000 7f6fa9b1a640 1 --2- 172.21.15.16:0/1143067494 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f9006b500 0x7f6f9006d9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6f940012e0 tx=0x7f6f94005b90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:34.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.274+0000 7f6fa27fc640 1 -- 172.21.15.16:0/1143067494 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6f98058ec0 con 0x7f6fa40d35b0 2024-04-25T10:09:34.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.495+0000 7f6fabd7e640 1 -- 172.21.15.16:0/1143067494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fa40b9260 con 0x7f6fa40d35b0 2024-04-25T10:09:34.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.495+0000 7f6fa27fc640 1 -- 172.21.15.16:0/1143067494 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f6f980587c0 con 0x7f6fa40d35b0 2024-04-25T10:09:34.501 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:34.502 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6070,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2805428272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2805428272},{"type":"v1","addr":"172.21.15.186:6835","nonce":2805428272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1052667029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1052667029},{"type":"v1","addr":"172.21.15.186:6837","nonce":1052667029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2897154125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2897154125},{"type":"v1","addr":"172.21.15.16:6833","nonce":2897154125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:22.077032+0000","modified":"2024-04-25T10:09:33.384844+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6636":{"gid":6636,"name":"a","rank":0,"incarnation":39,"state":"up:rejoin","state_seq":6,"addr":"172.21.15.16:6835/1735811292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1735811292},{"type":"v1","addr":"172.21.15.16:6835","nonce":1735811292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:09:34.502 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2024-04-25T10:09:34.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.497+0000 7f6fabd7e640 1 -- 172.21.15.16:0/1143067494 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f9006b500 msgr2=0x7f6f9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.497+0000 7f6fabd7e640 1 --2- 172.21.15.16:0/1143067494 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f9006b500 0x7f6f9006d9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6f940012e0 tx=0x7f6f94005b90 comp rx=0 tx=0).stop 2024-04-25T10:09:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.497+0000 7f6fabd7e640 1 -- 172.21.15.16:0/1143067494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa40d35b0 msgr2=0x7f6fa40cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.497+0000 7f6fabd7e640 1 --2- 172.21.15.16:0/1143067494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa40d35b0 0x7f6fa40cb590 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800c430 tx=0x7f6f9800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.497+0000 7f6fabd7e640 1 -- 172.21.15.16:0/1143067494 shutdown_connections 2024-04-25T10:09:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.497+0000 7f6fabd7e640 1 --2- 172.21.15.16:0/1143067494 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f9006b500 0x7f6f9006d9c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.497+0000 7f6fabd7e640 1 --2- 172.21.15.16:0/1143067494 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fa415dce0 0x7f6fa40cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.497+0000 7f6fabd7e640 1 --2- 172.21.15.16:0/1143067494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa40d35b0 0x7f6fa40cb590 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.497+0000 7f6fabd7e640 1 --2- 172.21.15.16:0/1143067494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fa40b5b60 0x7f6fa40cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.497+0000 7f6fabd7e640 1 -- 172.21.15.16:0/1143067494 >> 172.21.15.16:0/1143067494 conn(0x7f6fa4072900 msgr2=0x7f6fa40ce250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.497+0000 7f6fabd7e640 1 -- 172.21.15.16:0/1143067494 shutdown_connections 2024-04-25T10:09:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:34.497+0000 7f6fabd7e640 1 -- 172.21.15.16:0/1143067494 wait complete. 2024-04-25T10:09:34.917 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:09:22.077032+0000', 'modified': '2024-04-25T10:09:33.384844+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 73, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6636}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6636': {'gid': 6636, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:rejoin', 'state_seq': 6, 'addr': '172.21.15.16:6835/1735811292', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 1735811292}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 1735811292}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:09:34.917 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6636:up:rejoin 2024-04-25T10:09:35.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.026+0000 7f113b377640 1 Processor -- start 2024-04-25T10:09:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.026+0000 7f113b377640 1 -- start start 2024-04-25T10:09:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.027+0000 7f113b377640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f113409a150 0x7f113406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.027+0000 7f113b377640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f113406c1f0 0x7f11340cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.027+0000 7f113b377640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f113406c680 con 0x7f113415ba10 2024-04-25T10:09:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.027+0000 7f113b377640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11340cbe50 con 0x7f113409a150 2024-04-25T10:09:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.027+0000 7f113b377640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11340cfe80 con 0x7f113406c1f0 2024-04-25T10:09:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.027+0000 7f1138912640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f113409a150 0x7f113406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.027+0000 7f1138912640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f113409a150 0x7f113406bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40558/0 (socket says 172.21.15.16:40558) 2024-04-25T10:09:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.027+0000 7f1138912640 1 -- 172.21.15.16:0/3787423482 learned_addr learned my addr 172.21.15.16:0/3787423482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.027+0000 7f1139914640 1 --2- 172.21.15.16:0/3787423482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f113406c1f0 0x7f11340cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.027+0000 7f1138912640 1 -- 172.21.15.16:0/3787423482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f113406c1f0 msgr2=0x7f11340cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.027+0000 7f1138912640 1 --2- 172.21.15.16:0/3787423482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f113406c1f0 0x7f11340cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.027+0000 7f1138912640 1 -- 172.21.15.16:0/3787423482 >> v1:172.21.15.186:6789/0 conn(0x7f113415ba10 legacy=0x7f1134099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.027+0000 7f1138912640 1 -- 172.21.15.16:0/3787423482 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113406c900 con 0x7f113409a150 2024-04-25T10:09:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.027+0000 7f1139914640 1 --2- 172.21.15.16:0/3787423482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f113406c1f0 0x7f11340cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.028+0000 7f1138912640 1 --2- 172.21.15.16:0/3787423482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409a150 0x7f113406bcb0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1124009870 tx=0x7f112402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbca22c6383c6eb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.028+0000 7f112bfff640 1 -- 172.21.15.16:0/3787423482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1124037070 con 0x7f113409a150 2024-04-25T10:09:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.028+0000 7f112bfff640 1 -- 172.21.15.16:0/3787423482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1124032440 con 0x7f113409a150 2024-04-25T10:09:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.028+0000 7f112bfff640 1 -- 172.21.15.16:0/3787423482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1124032810 con 0x7f113409a150 2024-04-25T10:09:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.029+0000 7f113b377640 1 -- 172.21.15.16:0/3787423482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409a150 msgr2=0x7f113406bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.029+0000 7f113b377640 1 --2- 172.21.15.16:0/3787423482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409a150 0x7f113406bcb0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1124009870 tx=0x7f112402a200 comp rx=0 tx=0).stop 2024-04-25T10:09:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.029+0000 7f113b377640 1 -- 172.21.15.16:0/3787423482 shutdown_connections 2024-04-25T10:09:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.029+0000 7f113b377640 1 --2- 172.21.15.16:0/3787423482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f113406c1f0 0x7f11340cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.029+0000 7f113b377640 1 --2- 172.21.15.16:0/3787423482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409a150 0x7f113406bcb0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.029+0000 7f113b377640 1 -- 172.21.15.16:0/3787423482 >> 172.21.15.16:0/3787423482 conn(0x7f11340b32f0 msgr2=0x7f11340b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.029+0000 7f113b377640 1 -- 172.21.15.16:0/3787423482 shutdown_connections 2024-04-25T10:09:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.029+0000 7f113b377640 1 -- 172.21.15.16:0/3787423482 wait complete. 2024-04-25T10:09:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.030+0000 7f113b377640 1 Processor -- start 2024-04-25T10:09:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.030+0000 7f113b377640 1 -- start start 2024-04-25T10:09:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.031+0000 7f113b377640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f113406c1f0 0x7f11340b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.031+0000 7f113b377640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409a150 0x7f11340b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.031+0000 7f113b377640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f113415ba10 0x7f11340b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.031+0000 7f113b377640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11340cd760 con 0x7f113409a150 2024-04-25T10:09:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.031+0000 7f113b377640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113406cec0 con 0x7f113406c1f0 2024-04-25T10:09:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.031+0000 7f113b377640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11340cd8e0 con 0x7f113415ba10 2024-04-25T10:09:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.031+0000 7f1138912640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409a150 0x7f11340b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.031+0000 7f1138912640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409a150 0x7f11340b16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40566/0 (socket says 172.21.15.16:40566) 2024-04-25T10:09:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.031+0000 7f1138912640 1 -- 172.21.15.16:0/4060554043 learned_addr learned my addr 172.21.15.16:0/4060554043 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.031+0000 7f1139914640 1 --2- 172.21.15.16:0/4060554043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f113415ba10 0x7f11340b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.031+0000 7f1139113640 1 --2- 172.21.15.16:0/4060554043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f113406c1f0 0x7f11340b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.031+0000 7f1138912640 1 -- 172.21.15.16:0/4060554043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f113415ba10 msgr2=0x7f11340b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.031+0000 7f1138912640 1 --2- 172.21.15.16:0/4060554043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f113415ba10 0x7f11340b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.031+0000 7f1138912640 1 -- 172.21.15.16:0/4060554043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f113406c1f0 msgr2=0x7f11340b68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.031+0000 7f1138912640 1 --2- 172.21.15.16:0/4060554043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f113406c1f0 0x7f11340b68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.031+0000 7f1138912640 1 -- 172.21.15.16:0/4060554043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1134160240 con 0x7f113409a150 2024-04-25T10:09:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.032+0000 7f1138912640 1 --2- 172.21.15.16:0/4060554043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409a150 0x7f11340b16a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f1124000c00 tx=0x7f11240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.032+0000 7f1129ffb640 1 -- 172.21.15.16:0/4060554043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1124037040 con 0x7f113409a150 2024-04-25T10:09:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.032+0000 7f1129ffb640 1 -- 172.21.15.16:0/4060554043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1124042070 con 0x7f113409a150 2024-04-25T10:09:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.032+0000 7f113b377640 1 -- 172.21.15.16:0/4060554043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11340ae550 con 0x7f113409a150 2024-04-25T10:09:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.032+0000 7f1129ffb640 1 -- 172.21.15.16:0/4060554043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1124030bb0 con 0x7f113409a150 2024-04-25T10:09:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.032+0000 7f113b377640 1 -- 172.21.15.16:0/4060554043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11340aeb10 con 0x7f113409a150 2024-04-25T10:09:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.033+0000 7f113b377640 1 -- 172.21.15.16:0/4060554043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11340749b0 con 0x7f113409a150 2024-04-25T10:09:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.034+0000 7f1129ffb640 1 -- 172.21.15.16:0/4060554043 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f112402ad10 con 0x7f113409a150 2024-04-25T10:09:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.034+0000 7f1129ffb640 1 --2- 172.21.15.16:0/4060554043 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f111806b500 0x7f111806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.034+0000 7f1129ffb640 1 -- 172.21.15.16:0/4060554043 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f11240aab50 con 0x7f113409a150 2024-04-25T10:09:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.034+0000 7f1139113640 1 --2- 172.21.15.16:0/4060554043 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f111806b500 0x7f111806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.035+0000 7f1139113640 1 --2- 172.21.15.16:0/4060554043 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f111806b500 0x7f111806d9c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1130006fd0 tx=0x7f1130008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.038+0000 7f1129ffb640 1 -- 172.21.15.16:0/4060554043 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f113406c680 con 0x7f113409a150 2024-04-25T10:09:36.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.253+0000 7f113b377640 1 -- 172.21.15.16:0/4060554043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f113409a570 con 0x7f113409a150 2024-04-25T10:09:36.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.254+0000 7f1129ffb640 1 -- 172.21.15.16:0/4060554043 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f1124079a00 con 0x7f113409a150 2024-04-25T10:09:36.260 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:36.260 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6070,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2805428272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2805428272},{"type":"v1","addr":"172.21.15.186:6835","nonce":2805428272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1052667029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1052667029},{"type":"v1","addr":"172.21.15.186:6837","nonce":1052667029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2897154125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2897154125},{"type":"v1","addr":"172.21.15.16:6833","nonce":2897154125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:22.077032+0000","modified":"2024-04-25T10:09:34.388564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6636":{"gid":6636,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.16:6835/1735811292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1735811292},{"type":"v1","addr":"172.21.15.16:6835","nonce":1735811292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:09:36.260 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 42 2024-04-25T10:09:36.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.256+0000 7f113b377640 1 -- 172.21.15.16:0/4060554043 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f111806b500 msgr2=0x7f111806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.256+0000 7f113b377640 1 --2- 172.21.15.16:0/4060554043 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f111806b500 0x7f111806d9c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1130006fd0 tx=0x7f1130008040 comp rx=0 tx=0).stop 2024-04-25T10:09:36.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.256+0000 7f113b377640 1 -- 172.21.15.16:0/4060554043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409a150 msgr2=0x7f11340b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.256+0000 7f113b377640 1 --2- 172.21.15.16:0/4060554043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409a150 0x7f11340b16a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f1124000c00 tx=0x7f11240046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.256+0000 7f113b377640 1 -- 172.21.15.16:0/4060554043 shutdown_connections 2024-04-25T10:09:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.256+0000 7f113b377640 1 --2- 172.21.15.16:0/4060554043 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f111806b500 0x7f111806d9c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.256+0000 7f113b377640 1 --2- 172.21.15.16:0/4060554043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f113415ba10 0x7f11340b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.256+0000 7f113b377640 1 --2- 172.21.15.16:0/4060554043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409a150 0x7f11340b16a0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.256+0000 7f113b377640 1 --2- 172.21.15.16:0/4060554043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f113406c1f0 0x7f11340b68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.256+0000 7f113b377640 1 -- 172.21.15.16:0/4060554043 >> 172.21.15.16:0/4060554043 conn(0x7f11340b32f0 msgr2=0x7f1134099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.256+0000 7f113b377640 1 -- 172.21.15.16:0/4060554043 shutdown_connections 2024-04-25T10:09:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.256+0000 7f113b377640 1 -- 172.21.15.16:0/4060554043 wait complete. 2024-04-25T10:09:36.675 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:09:22.077032+0000', 'modified': '2024-04-25T10:09:34.388564+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 73, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6636}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6636': {'gid': 6636, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.16:6835/1735811292', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 1735811292}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 1735811292}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:09:36.676 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:09:36.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:09:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.776+0000 7fd3ff3ce640 1 Processor -- start 2024-04-25T10:09:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.776+0000 7fd3ff3ce640 1 -- start start 2024-04-25T10:09:36.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.777+0000 7fd3ff3ce640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd3f81610b0 0x7fd3f80b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.777+0000 7fd3ff3ce640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3f80cef40 0x7fd3f80d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.777+0000 7fd3ff3ce640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f8162040 con 0x7fd3f81618d0 2024-04-25T10:09:36.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.777+0000 7fd3ff3ce640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f80d1ea0 con 0x7fd3f81610b0 2024-04-25T10:09:36.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.777+0000 7fd3ff3ce640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f8162320 con 0x7fd3f80cef40 2024-04-25T10:09:36.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.777+0000 7fd3fd96b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3f80cef40 0x7fd3f80d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.777+0000 7fd3fd16a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd3f81610b0 0x7fd3f80b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.777+0000 7fd3fd16a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd3f81610b0 0x7fd3f80b9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:45250/0 (socket says 172.21.15.16:45250) 2024-04-25T10:09:36.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.777+0000 7fd3fc969640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fd3f81618d0 0x7fd3f80ce830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:45498/0 (socket says 172.21.15.16:45498) 2024-04-25T10:09:36.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.777+0000 7fd3fd96b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3f80cef40 0x7fd3f80d2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:43720/0 (socket says 172.21.15.16:43720) 2024-04-25T10:09:36.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.777+0000 7fd3fd16a640 1 -- 172.21.15.16:0/1028267888 learned_addr learned my addr 172.21.15.16:0/1028267888 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:36.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.778+0000 7fd3fd16a640 1 -- 172.21.15.16:0/1028267888 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3f80cef40 msgr2=0x7fd3f80d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:36.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.778+0000 7fd3fd16a640 1 --2- 172.21.15.16:0/1028267888 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3f80cef40 0x7fd3f80d2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.778+0000 7fd3fd16a640 1 -- 172.21.15.16:0/1028267888 >> v1:172.21.15.186:6789/0 conn(0x7fd3f81618d0 legacy=0x7fd3f80ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.778+0000 7fd3fd16a640 1 -- 172.21.15.16:0/1028267888 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f8162600 con 0x7fd3f81610b0 2024-04-25T10:09:36.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.778+0000 7fd3effff640 1 -- 172.21.15.16:0/1028267888 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133932356 0 0) 0x7fd3f8162040 con 0x7fd3f81618d0 2024-04-25T10:09:36.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.778+0000 7fd3fd96b640 1 --2- 172.21.15.16:0/1028267888 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3f80cef40 0x7fd3f80d2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:36.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.778+0000 7fd3fd16a640 1 --2- 172.21.15.16:0/1028267888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3f81610b0 0x7fd3f80b9580 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009870 tx=0x7fd3f402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da1ca2ae3389c229 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.778+0000 7fd3effff640 1 -- 172.21.15.16:0/1028267888 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4037070 con 0x7fd3f81610b0 2024-04-25T10:09:36.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.778+0000 7fd3effff640 1 -- 172.21.15.16:0/1028267888 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f4032440 con 0x7fd3f81610b0 2024-04-25T10:09:36.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.779+0000 7fd3effff640 1 -- 172.21.15.16:0/1028267888 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4032810 con 0x7fd3f81610b0 2024-04-25T10:09:36.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.779+0000 7fd3ff3ce640 1 -- 172.21.15.16:0/1028267888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3f81610b0 msgr2=0x7fd3f80b9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.779+0000 7fd3ff3ce640 1 --2- 172.21.15.16:0/1028267888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3f81610b0 0x7fd3f80b9580 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009870 tx=0x7fd3f402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.779+0000 7fd3ff3ce640 1 -- 172.21.15.16:0/1028267888 shutdown_connections 2024-04-25T10:09:36.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.779+0000 7fd3ff3ce640 1 --2- 172.21.15.16:0/1028267888 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3f80cef40 0x7fd3f80d2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.779+0000 7fd3ff3ce640 1 --2- 172.21.15.16:0/1028267888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3f81610b0 0x7fd3f80b9580 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.779+0000 7fd3ff3ce640 1 -- 172.21.15.16:0/1028267888 >> 172.21.15.16:0/1028267888 conn(0x7fd3f80b5570 msgr2=0x7fd3f80b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:36.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.779+0000 7fd3ff3ce640 1 -- 172.21.15.16:0/1028267888 shutdown_connections 2024-04-25T10:09:36.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.779+0000 7fd3ff3ce640 1 -- 172.21.15.16:0/1028267888 wait complete. 2024-04-25T10:09:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.780+0000 7fd3ff3ce640 1 Processor -- start 2024-04-25T10:09:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.780+0000 7fd3ff3ce640 1 -- start start 2024-04-25T10:09:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.781+0000 7fd3ff3ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f80cef40 0x7fd3f8144340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.781+0000 7fd3ff3ce640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3f81610b0 0x7fd3f8155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.781+0000 7fd3ff3ce640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3f81618d0 0x7fd3f8156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.781+0000 7fd3ff3ce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f8161d10 con 0x7fd3f80cef40 2024-04-25T10:09:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.781+0000 7fd3ff3ce640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f80b6c00 con 0x7fd3f81610b0 2024-04-25T10:09:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.781+0000 7fd3ff3ce640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f8161490 con 0x7fd3f81618d0 2024-04-25T10:09:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.781+0000 7fd3fd16a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f80cef40 0x7fd3f8144340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.781+0000 7fd3fd16a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f80cef40 0x7fd3f8144340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40578/0 (socket says 172.21.15.16:40578) 2024-04-25T10:09:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.781+0000 7fd3fd16a640 1 -- 172.21.15.16:0/2703667142 learned_addr learned my addr 172.21.15.16:0/2703667142 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.781+0000 7fd3fc969640 1 --2- 172.21.15.16:0/2703667142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3f81610b0 0x7fd3f8155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.781+0000 7fd3fd96b640 1 --2- 172.21.15.16:0/2703667142 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3f81618d0 0x7fd3f8156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.781+0000 7fd3fd16a640 1 -- 172.21.15.16:0/2703667142 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3f81618d0 msgr2=0x7fd3f8156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.781+0000 7fd3fd16a640 1 --2- 172.21.15.16:0/2703667142 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3f81618d0 0x7fd3f8156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.781+0000 7fd3fd16a640 1 -- 172.21.15.16:0/2703667142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3f81610b0 msgr2=0x7fd3f8155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.781+0000 7fd3fd16a640 1 --2- 172.21.15.16:0/2703667142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3f81610b0 0x7fd3f8155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.781+0000 7fd3fd16a640 1 -- 172.21.15.16:0/2703667142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f81f7e40 con 0x7fd3f80cef40 2024-04-25T10:09:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.782+0000 7fd3fd16a640 1 --2- 172.21.15.16:0/2703667142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f80cef40 0x7fd3f8144340 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4000c00 tx=0x7fd3f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.782+0000 7fd3edffb640 1 -- 172.21.15.16:0/2703667142 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4037040 con 0x7fd3f80cef40 2024-04-25T10:09:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.782+0000 7fd3edffb640 1 -- 172.21.15.16:0/2703667142 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f4042070 con 0x7fd3f80cef40 2024-04-25T10:09:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.782+0000 7fd3ff3ce640 1 -- 172.21.15.16:0/2703667142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f8144dc0 con 0x7fd3f80cef40 2024-04-25T10:09:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.782+0000 7fd3edffb640 1 -- 172.21.15.16:0/2703667142 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4030880 con 0x7fd3f80cef40 2024-04-25T10:09:36.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.782+0000 7fd3ff3ce640 1 -- 172.21.15.16:0/2703667142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f8145380 con 0x7fd3f80cef40 2024-04-25T10:09:36.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.783+0000 7fd3edffb640 1 -- 172.21.15.16:0/2703667142 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd3f402ad10 con 0x7fd3f80cef40 2024-04-25T10:09:36.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.784+0000 7fd3edffb640 1 --2- 172.21.15.16:0/2703667142 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd3dc06b4d0 0x7fd3dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.784+0000 7fd3edffb640 1 -- 172.21.15.16:0/2703667142 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd3f40aa890 con 0x7fd3f80cef40 2024-04-25T10:09:36.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.784+0000 7fd3fc969640 1 --2- 172.21.15.16:0/2703667142 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd3dc06b4d0 0x7fd3dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.785+0000 7fd3fc969640 1 --2- 172.21.15.16:0/2703667142 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd3dc06b4d0 0x7fd3dc06d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8007fd0 tx=0x7fd3e8007c20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.785+0000 7fd4021af740 1 Processor -- start 2024-04-25T10:09:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.785+0000 7fd4021af740 1 -- start start 2024-04-25T10:09:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd4021af740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f6fb230 0x55cf8f6fb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd4021af740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f718ad0 0x55cf8f719020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd4021af740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f6d0f10 0x55cf8f717190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd4021af740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8f633a10 con 0x55cf8f718ad0 2024-04-25T10:09:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd4021af740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8f5dcb30 con 0x55cf8f6fb230 2024-04-25T10:09:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd4021af740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cf8f62fc50 con 0x55cf8f6d0f10 2024-04-25T10:09:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd3fc969640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f718ad0 0x55cf8f719020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd3fc969640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f718ad0 0x55cf8f719020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40590/0 (socket says 172.21.15.16:40590) 2024-04-25T10:09:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd3fc969640 1 -- 172.21.15.16:0/2522656268 learned_addr learned my addr 172.21.15.16:0/2522656268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd3fd96b640 1 --2- 172.21.15.16:0/2522656268 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f6d0f10 0x55cf8f717190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd3fd16a640 1 --2- 172.21.15.16:0/2522656268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f6fb230 0x55cf8f6fb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd3fc969640 1 -- 172.21.15.16:0/2522656268 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f6d0f10 msgr2=0x55cf8f717190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd3fc969640 1 --2- 172.21.15.16:0/2522656268 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f6d0f10 0x55cf8f717190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd3fc969640 1 -- 172.21.15.16:0/2522656268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f6fb230 msgr2=0x55cf8f6fb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd3fc969640 1 --2- 172.21.15.16:0/2522656268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f6fb230 0x55cf8f6fb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd3fc969640 1 -- 172.21.15.16:0/2522656268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf8f5f1e10 con 0x55cf8f718ad0 2024-04-25T10:09:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd3fc969640 1 --2- 172.21.15.16:0/2522656268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f718ad0 0x55cf8f719020 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8005d80 tx=0x7fd3e8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4aacce1a0396431 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd3e2ffd640 1 -- 172.21.15.16:0/2522656268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8015110 con 0x55cf8f718ad0 2024-04-25T10:09:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd3e2ffd640 1 -- 172.21.15.16:0/2522656268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e80159c0 con 0x55cf8f718ad0 2024-04-25T10:09:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.786+0000 7fd3e2ffd640 1 -- 172.21.15.16:0/2522656268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8015d70 con 0x55cf8f718ad0 2024-04-25T10:09:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.787+0000 7fd4021af740 1 -- 172.21.15.16:0/2522656268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f718ad0 msgr2=0x55cf8f719020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.787+0000 7fd4021af740 1 --2- 172.21.15.16:0/2522656268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f718ad0 0x55cf8f719020 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8005d80 tx=0x7fd3e8012000 comp rx=0 tx=0).stop 2024-04-25T10:09:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.787+0000 7fd4021af740 1 -- 172.21.15.16:0/2522656268 shutdown_connections 2024-04-25T10:09:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.787+0000 7fd4021af740 1 --2- 172.21.15.16:0/2522656268 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f6d0f10 0x55cf8f717190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.787+0000 7fd4021af740 1 --2- 172.21.15.16:0/2522656268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f718ad0 0x55cf8f719020 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.787+0000 7fd4021af740 1 --2- 172.21.15.16:0/2522656268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f6fb230 0x55cf8f6fb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.787+0000 7fd4021af740 1 -- 172.21.15.16:0/2522656268 >> 172.21.15.16:0/2522656268 conn(0x55cf8f6cd840 msgr2=0x55cf8f716040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.787+0000 7fd4021af740 1 -- 172.21.15.16:0/2522656268 shutdown_connections 2024-04-25T10:09:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.787+0000 7fd4021af740 1 -- 172.21.15.16:0/2522656268 wait complete. 2024-04-25T10:09:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.787+0000 7fd4021af740 1 -- start start 2024-04-25T10:09:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.788+0000 7fd4021af740 1 Processor -- start 2024-04-25T10:09:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.788+0000 7fd4021af740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f6fb230 0x55cf8f6f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.788+0000 7fd4021af740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f718ad0 0x55cf8f70d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.788+0000 7fd4021af740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f7262f0 0x55cf8f726770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.788+0000 7fd4021af740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8f6d1cb0 con 0x55cf8f7262f0 2024-04-25T10:09:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.788+0000 7fd4021af740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8f5f00c0 con 0x55cf8f718ad0 2024-04-25T10:09:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.788+0000 7fd4021af740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cf8f6cb770 con 0x55cf8f6fb230 2024-04-25T10:09:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.788+0000 7fd4021af740 10 client.0 _ll_register_callbacks cb 0x55cf8f5e3ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.789+0000 7fd3fd96b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f7262f0 0x55cf8f726770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.789+0000 7fd3fd96b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f7262f0 0x55cf8f726770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40594/0 (socket says 172.21.15.16:40594) 2024-04-25T10:09:36.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.789+0000 7fd3fd96b640 1 -- 172.21.15.16:0/4037747803 learned_addr learned my addr 172.21.15.16:0/4037747803 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:36.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.789+0000 7fd3fd16a640 1 --2- 172.21.15.16:0/4037747803 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f6fb230 0x55cf8f6f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.789+0000 7fd3fc969640 1 --2- 172.21.15.16:0/4037747803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f718ad0 0x55cf8f70d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.789+0000 7fd3fd96b640 1 -- 172.21.15.16:0/4037747803 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f6fb230 msgr2=0x55cf8f6f7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:36.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.789+0000 7fd3fd96b640 1 --2- 172.21.15.16:0/4037747803 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f6fb230 0x55cf8f6f7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.789+0000 7fd3fd96b640 1 -- 172.21.15.16:0/4037747803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f718ad0 msgr2=0x55cf8f70d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:36.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.789+0000 7fd3fd96b640 1 --2- 172.21.15.16:0/4037747803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f718ad0 0x55cf8f70d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.789+0000 7fd3fd96b640 1 -- 172.21.15.16:0/4037747803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf8f3482d0 con 0x55cf8f7262f0 2024-04-25T10:09:36.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.789+0000 7fd3fd96b640 1 --2- 172.21.15.16:0/4037747803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f7262f0 0x55cf8f726770 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0007da0 tx=0x7fd3f0013680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85050f560a8d84f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.789+0000 7fd3d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:09:36.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.789+0000 7fd3d7fff640 1 -- 172.21.15.16:0/4037747803 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f001c070 con 0x55cf8f7262f0 2024-04-25T10:09:36.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.789+0000 7fd3d7fff640 1 -- 172.21.15.16:0/4037747803 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd3f0016630 con 0x55cf8f7262f0 2024-04-25T10:09:36.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.789+0000 7fd4021af740 1 -- 172.21.15.16:0/4037747803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3e80159c0 con 0x55cf8f7262f0 2024-04-25T10:09:36.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.789+0000 7fd3d7fff640 1 -- 172.21.15.16:0/4037747803 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3f0007e90 con 0x55cf8f7262f0 2024-04-25T10:09:36.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.789+0000 7fd4021af740 10 client.6900 fetch_fsmap learned FSMap version 42 2024-04-25T10:09:36.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.789+0000 7fd4021af740 1 -- 172.21.15.16:0/4037747803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x55cf8f4126b0 con 0x55cf8f7262f0 2024-04-25T10:09:36.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.790+0000 7fd3d7fff640 1 -- 172.21.15.16:0/4037747803 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e80159c0 con 0x55cf8f7262f0 2024-04-25T10:09:36.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.790+0000 7fd3d7fff640 1 -- 172.21.15.16:0/4037747803 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd3f002a080 con 0x55cf8f7262f0 2024-04-25T10:09:36.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.790+0000 7fd4021af740 10 client.6900 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T10:09:36.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.790+0000 7fd4021af740 10 client.6900 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:09:36.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.790+0000 7fd4021af740 20 client.6900 populate_metadata read hostname 'smithi016' 2024-04-25T10:09:36.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.790+0000 7fd4021af740 1 --2- 172.21.15.16:0/4037747803 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55cf8f7320c0 0x55cf8f7344b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.790+0000 7fd4021af740 4 client.6900 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:36.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.790+0000 7fd4021af740 1 -- 172.21.15.16:0/4037747803 --> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf8f692920 con 0x55cf8f7320c0 2024-04-25T10:09:36.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.790+0000 7fd3fd16a640 1 --2- 172.21.15.16:0/4037747803 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55cf8f7320c0 0x55cf8f7344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.790+0000 7fd3fd16a640 1 --2- 172.21.15.16:0/4037747803 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55cf8f7320c0 0x55cf8f7344b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e08e03b9989885 server_cookie=f89a854b6ee3fb77 in_seq=0 out_seq=0 2024-04-25T10:09:36.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.790+0000 7fd3d7fff640 10 client.6900 ms_handle_connect on v2:172.21.15.16:6834/1735811292 2024-04-25T10:09:36.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.791+0000 7fd3d7fff640 1 -- 172.21.15.16:0/4037747803 <== mds.0 v2:172.21.15.16:6834/1735811292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cf8f692920 con 0x55cf8f7320c0 2024-04-25T10:09:36.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.791+0000 7fd3d7fff640 10 client.6900 handle_command_reply: tid=0 2024-04-25T10:09:36.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.791+0000 7fd4021af740 1 client.6900 shutdown 2024-04-25T10:09:36.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.791+0000 7fd4021af740 1 -- 172.21.15.16:0/4037747803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f7262f0 msgr2=0x55cf8f726770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.791+0000 7fd4021af740 1 --2- 172.21.15.16:0/4037747803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f7262f0 0x55cf8f726770 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0007da0 tx=0x7fd3f0013680 comp rx=0 tx=0).stop 2024-04-25T10:09:36.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.791+0000 7fd4021af740 1 -- 172.21.15.16:0/4037747803 shutdown_connections 2024-04-25T10:09:36.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.791+0000 7fd4021af740 1 --2- 172.21.15.16:0/4037747803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f7262f0 0x55cf8f726770 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.791+0000 7fd4021af740 1 --2- 172.21.15.16:0/4037747803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f718ad0 0x55cf8f70d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.791+0000 7fd4021af740 1 --2- 172.21.15.16:0/4037747803 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55cf8f7320c0 0x55cf8f7344b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.791+0000 7fd4021af740 1 --2- 172.21.15.16:0/4037747803 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f6fb230 0x55cf8f6f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.791+0000 7fd3d7fff640 0 client.6900 ms_handle_reset on v2:172.21.15.16:6834/1735811292 2024-04-25T10:09:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.791+0000 7fd4021af740 1 -- 172.21.15.16:0/4037747803 >> 172.21.15.16:0/4037747803 conn(0x55cf8f6d0f10 msgr2=0x55cf8f719e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.791+0000 7fd4021af740 1 -- 172.21.15.16:0/4037747803 shutdown_connections 2024-04-25T10:09:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.791+0000 7fd4021af740 1 -- 172.21.15.16:0/4037747803 wait complete. 2024-04-25T10:09:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.791+0000 7fd4021af740 20 client.6900 trim_cache size 0 max 16384 2024-04-25T10:09:36.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.807+0000 7fd4021af740 1 Processor -- start 2024-04-25T10:09:36.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.807+0000 7fd4021af740 1 -- start start 2024-04-25T10:09:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd4021af740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f6fb230 0x55cf8f6d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd4021af740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f718ad0 0x55cf8f70b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd4021af740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f7262f0 0x55cf8f70bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd4021af740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f80cb880 con 0x55cf8f6fb230 2024-04-25T10:09:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd4021af740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f80cac80 con 0x55cf8f718ad0 2024-04-25T10:09:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd4021af740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f80cb280 con 0x55cf8f7262f0 2024-04-25T10:09:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd3fd16a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f6fb230 0x55cf8f6d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd3fd16a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f6fb230 0x55cf8f6d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40604/0 (socket says 172.21.15.16:40604) 2024-04-25T10:09:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd3fd16a640 1 -- 172.21.15.16:0/631408979 learned_addr learned my addr 172.21.15.16:0/631408979 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd3fc969640 1 --2- 172.21.15.16:0/631408979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f718ad0 0x55cf8f70b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd3fd96b640 1 --2- 172.21.15.16:0/631408979 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f7262f0 0x55cf8f70bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd3fd16a640 1 -- 172.21.15.16:0/631408979 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f7262f0 msgr2=0x55cf8f70bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd3fd16a640 1 --2- 172.21.15.16:0/631408979 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f7262f0 0x55cf8f70bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd3fd16a640 1 -- 172.21.15.16:0/631408979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f718ad0 msgr2=0x55cf8f70b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd3fd16a640 1 --2- 172.21.15.16:0/631408979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f718ad0 0x55cf8f70b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd3fd16a640 1 -- 172.21.15.16:0/631408979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf8f6f80b0 con 0x55cf8f6fb230 2024-04-25T10:09:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd3fd16a640 1 --2- 172.21.15.16:0/631408979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f6fb230 0x55cf8f6d05c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4075370 tx=0x7fd3f4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9638a7f86c0f2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd3e27fc640 1 -- 172.21.15.16:0/631408979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f40a6070 con 0x55cf8f6fb230 2024-04-25T10:09:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.808+0000 7fd3e27fc640 1 -- 172.21.15.16:0/631408979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f4081620 con 0x55cf8f6fb230 2024-04-25T10:09:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.809+0000 7fd3e27fc640 1 -- 172.21.15.16:0/631408979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f40a8dd0 con 0x55cf8f6fb230 2024-04-25T10:09:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.809+0000 7fd4021af740 1 -- 172.21.15.16:0/631408979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f6fb230 msgr2=0x55cf8f6d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.809+0000 7fd4021af740 1 --2- 172.21.15.16:0/631408979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f6fb230 0x55cf8f6d05c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4075370 tx=0x7fd3f4008040 comp rx=0 tx=0).stop 2024-04-25T10:09:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.809+0000 7fd4021af740 1 -- 172.21.15.16:0/631408979 shutdown_connections 2024-04-25T10:09:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.809+0000 7fd4021af740 1 --2- 172.21.15.16:0/631408979 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f7262f0 0x55cf8f70bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.809+0000 7fd4021af740 1 --2- 172.21.15.16:0/631408979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f718ad0 0x55cf8f70b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.809+0000 7fd4021af740 1 --2- 172.21.15.16:0/631408979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f6fb230 0x55cf8f6d05c0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.809+0000 7fd4021af740 1 -- 172.21.15.16:0/631408979 >> 172.21.15.16:0/631408979 conn(0x55cf8f6d0f10 msgr2=0x55cf8f6fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.809+0000 7fd4021af740 1 -- 172.21.15.16:0/631408979 shutdown_connections 2024-04-25T10:09:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.809+0000 7fd4021af740 1 -- 172.21.15.16:0/631408979 wait complete. 2024-04-25T10:09:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.810+0000 7fd4021af740 1 -- start start 2024-04-25T10:09:36.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.810+0000 7fd4021af740 1 Processor -- start 2024-04-25T10:09:36.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.810+0000 7fd4021af740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f6fb230 0x55cf8f72b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.810+0000 7fd4021af740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f718ad0 0x55cf8f6cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.810+0000 7fd4021af740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f7262f0 0x55cf8f6fe230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.810+0000 7fd4021af740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8f6fcdd0 con 0x55cf8f7262f0 2024-04-25T10:09:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd4021af740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8f6fc980 con 0x55cf8f6fb230 2024-04-25T10:09:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd4021af740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cf8f6fd310 con 0x55cf8f718ad0 2024-04-25T10:09:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd4021af740 10 client.0 _ll_register_callbacks cb 0x55cf8f6cbc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd3fd96b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f7262f0 0x55cf8f6fe230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd3fd96b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f7262f0 0x55cf8f6fe230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40614/0 (socket says 172.21.15.16:40614) 2024-04-25T10:09:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd3fd96b640 1 -- 172.21.15.16:0/95328771 learned_addr learned my addr 172.21.15.16:0/95328771 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd3fc969640 1 --2- 172.21.15.16:0/95328771 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f718ad0 0x55cf8f6cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd3fd16a640 1 --2- 172.21.15.16:0/95328771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f6fb230 0x55cf8f72b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd3fd96b640 1 -- 172.21.15.16:0/95328771 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f718ad0 msgr2=0x55cf8f6cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd3fd96b640 1 --2- 172.21.15.16:0/95328771 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f718ad0 0x55cf8f6cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd3fd96b640 1 -- 172.21.15.16:0/95328771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f6fb230 msgr2=0x55cf8f72b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd3fd96b640 1 --2- 172.21.15.16:0/95328771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f6fb230 0x55cf8f72b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd3fd96b640 1 -- 172.21.15.16:0/95328771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf8f70b9d0 con 0x55cf8f7262f0 2024-04-25T10:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd3fd96b640 1 --2- 172.21.15.16:0/95328771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f7262f0 0x55cf8f6fe230 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0009fd0 tx=0x7fd3f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da27584331a2c72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd3d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd3d7fff640 1 -- 172.21.15.16:0/95328771 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0036060 con 0x55cf8f7262f0 2024-04-25T10:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd3d7fff640 1 -- 172.21.15.16:0/95328771 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd3f0021650 con 0x55cf8f7262f0 2024-04-25T10:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd4021af740 1 -- 172.21.15.16:0/95328771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3f4081620 con 0x55cf8f7262f0 2024-04-25T10:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd3d7fff640 1 -- 172.21.15.16:0/95328771 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3f003c050 con 0x55cf8f7262f0 2024-04-25T10:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd4021af740 10 client.6906 fetch_fsmap learned FSMap version 42 2024-04-25T10:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.811+0000 7fd4021af740 1 -- 172.21.15.16:0/95328771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x55cf8f6fdd10 con 0x55cf8f7262f0 2024-04-25T10:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.812+0000 7fd3d7fff640 1 -- 172.21.15.16:0/95328771 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f4081620 con 0x55cf8f7262f0 2024-04-25T10:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.812+0000 7fd3d7fff640 1 -- 172.21.15.16:0/95328771 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd3f003f020 con 0x55cf8f7262f0 2024-04-25T10:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.812+0000 7fd4021af740 10 client.6906 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T10:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.812+0000 7fd4021af740 10 client.6906 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:09:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.812+0000 7fd4021af740 20 client.6906 populate_metadata read hostname 'smithi016' 2024-04-25T10:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.812+0000 7fd4021af740 1 --2- 172.21.15.16:0/95328771 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55cf8f738bc0 0x55cf8f724760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.812+0000 7fd4021af740 4 client.6906 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.812+0000 7fd4021af740 1 -- 172.21.15.16:0/95328771 --> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cf8f640f50 con 0x55cf8f738bc0 2024-04-25T10:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.812+0000 7fd3fd16a640 1 --2- 172.21.15.16:0/95328771 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55cf8f738bc0 0x55cf8f724760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.812+0000 7fd3d7fff640 10 client.6906 ms_handle_connect on v2:172.21.15.16:6834/1735811292 2024-04-25T10:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.812+0000 7fd3fd16a640 1 --2- 172.21.15.16:0/95328771 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55cf8f738bc0 0x55cf8f724760 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b2b1583c678dcce server_cookie=f54d0e3f56946a9c in_seq=0 out_seq=0 2024-04-25T10:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.812+0000 7fd3d7fff640 1 -- 172.21.15.16:0/95328771 <== mds.0 v2:172.21.15.16:6834/1735811292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55cf8f640f50 con 0x55cf8f738bc0 2024-04-25T10:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.812+0000 7fd3d7fff640 10 client.6906 handle_command_reply: tid=0 2024-04-25T10:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.812+0000 7fd4021af740 1 client.6906 shutdown 2024-04-25T10:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.812+0000 7fd4021af740 1 -- 172.21.15.16:0/95328771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f7262f0 msgr2=0x55cf8f6fe230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.812+0000 7fd4021af740 1 --2- 172.21.15.16:0/95328771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f7262f0 0x55cf8f6fe230 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0009fd0 tx=0x7fd3f0004a40 comp rx=0 tx=0).stop 2024-04-25T10:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd4021af740 1 -- 172.21.15.16:0/95328771 shutdown_connections 2024-04-25T10:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd4021af740 1 --2- 172.21.15.16:0/95328771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf8f7262f0 0x55cf8f6fe230 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd4021af740 1 --2- 172.21.15.16:0/95328771 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55cf8f738bc0 0x55cf8f724760 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd4021af740 1 --2- 172.21.15.16:0/95328771 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cf8f718ad0 0x55cf8f6cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd4021af740 1 --2- 172.21.15.16:0/95328771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf8f6fb230 0x55cf8f72b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd3d7fff640 0 client.6906 ms_handle_reset on v2:172.21.15.16:6834/1735811292 2024-04-25T10:09:36.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd4021af740 1 -- 172.21.15.16:0/95328771 >> 172.21.15.16:0/95328771 conn(0x55cf8f6d0f10 msgr2=0x55cf8f6d12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:36.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd4021af740 1 -- 172.21.15.16:0/95328771 shutdown_connections 2024-04-25T10:09:36.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd4021af740 1 -- 172.21.15.16:0/95328771 wait complete. 2024-04-25T10:09:36.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd4021af740 20 client.6906 trim_cache size 0 max 16384 2024-04-25T10:09:36.825 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:09:36.825 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "34bc5eca-3a21-42a6-a26f-07da0905e8bc", 2024-04-25T10:09:36.825 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-04-25T10:09:36.825 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6636, 2024-04-25T10:09:36.826 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-04-25T10:09:36.826 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-04-25T10:09:36.826 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-04-25T10:09:36.826 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 4.4321197540000004, 2024-04-25T10:09:36.826 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 42, 2024-04-25T10:09:36.826 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 73, 2024-04-25T10:09:36.826 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 73, 2024-04-25T10:09:36.826 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 19.121987551 2024-04-25T10:09:36.826 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:09:36.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd3ff3ce640 1 -- 172.21.15.16:0/2703667142 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd3dc06b4d0 msgr2=0x7fd3dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd3ff3ce640 1 --2- 172.21.15.16:0/2703667142 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd3dc06b4d0 0x7fd3dc06d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8007fd0 tx=0x7fd3e8007c20 comp rx=0 tx=0).stop 2024-04-25T10:09:36.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd3ff3ce640 1 -- 172.21.15.16:0/2703667142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f80cef40 msgr2=0x7fd3f8144340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd3ff3ce640 1 --2- 172.21.15.16:0/2703667142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f80cef40 0x7fd3f8144340 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4000c00 tx=0x7fd3f40046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd3ff3ce640 1 -- 172.21.15.16:0/2703667142 shutdown_connections 2024-04-25T10:09:36.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd3ff3ce640 1 --2- 172.21.15.16:0/2703667142 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd3dc06b4d0 0x7fd3dc06d990 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd3ff3ce640 1 --2- 172.21.15.16:0/2703667142 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3f81618d0 0x7fd3f8156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd3ff3ce640 1 --2- 172.21.15.16:0/2703667142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3f81610b0 0x7fd3f8155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd3ff3ce640 1 --2- 172.21.15.16:0/2703667142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f80cef40 0x7fd3f8144340 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd3ff3ce640 1 -- 172.21.15.16:0/2703667142 >> 172.21.15.16:0/2703667142 conn(0x7fd3f80b5570 msgr2=0x7fd3f80d2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd3ff3ce640 1 -- 172.21.15.16:0/2703667142 shutdown_connections 2024-04-25T10:09:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.813+0000 7fd3ff3ce640 1 -- 172.21.15.16:0/2703667142 wait complete. 2024-04-25T10:09:36.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:09:36.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.924+0000 7fe3bb969640 1 Processor -- start 2024-04-25T10:09:36.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.924+0000 7fe3bb969640 1 -- start start 2024-04-25T10:09:36.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.925+0000 7fe3bb969640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe3b4144450 0x7fe3b40c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.925+0000 7fe3bb969640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b40c75c0 con 0x7fe3b40ccdc0 2024-04-25T10:09:36.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.925+0000 7fe3bb969640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b40cd2c0 con 0x7fe3b415eaa0 2024-04-25T10:09:36.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.925+0000 7fe3bb969640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b40cdb60 con 0x7fe3b4144450 2024-04-25T10:09:36.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.925+0000 7fe3b8f04640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe3b40ccdc0 0x7fe3b40722a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38904/0 (socket says 172.21.15.16:38904) 2024-04-25T10:09:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.925+0000 7fe3b8f04640 1 -- 172.21.15.16:0/370268047 learned_addr learned my addr 172.21.15.16:0/370268047 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b3fff640 1 -- 172.21.15.16:0/370268047 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034141292 0 0) 0x7fe3b40c75c0 con 0x7fe3b40ccdc0 2024-04-25T10:09:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b9705640 1 --2- 172.21.15.16:0/370268047 >> v2:172.21.15.186:3300/0 conn(0x7fe3b4144450 0x7fe3b40c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b3fff640 1 -- 172.21.15.16:0/370268047 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe39c003520 con 0x7fe3b40ccdc0 2024-04-25T10:09:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b3fff640 1 -- 172.21.15.16:0/370268047 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3084111106 0 0) 0x7fe39c003520 con 0x7fe3b40ccdc0 2024-04-25T10:09:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b3fff640 1 -- 172.21.15.16:0/370268047 >> v2:172.21.15.186:3300/0 conn(0x7fe3b4144450 msgr2=0x7fe3b40c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b3fff640 1 --2- 172.21.15.16:0/370268047 >> v2:172.21.15.186:3300/0 conn(0x7fe3b4144450 0x7fe3b40c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b3fff640 1 -- 172.21.15.16:0/370268047 >> v1:172.21.15.186:6789/0 conn(0x7fe3b415eaa0 legacy=0x7fe3b40af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b3fff640 1 -- 172.21.15.16:0/370268047 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b40cd5a0 con 0x7fe3b40ccdc0 2024-04-25T10:09:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b3fff640 1 -- 172.21.15.16:0/370268047 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fe3a8001670 con 0x7fe3b40ccdc0 2024-04-25T10:09:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b3fff640 1 -- 172.21.15.16:0/370268047 >> v1:172.21.15.16:6789/0 conn(0x7fe3b40ccdc0 legacy=0x7fe3b40722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b3fff640 1 --2- 172.21.15.16:0/370268047 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe39c003a00 0x7fe39c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b3fff640 1 --2- 172.21.15.16:0/370268047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe39c004800 0x7fe39c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b3fff640 1 --2- 172.21.15.16:0/370268047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe39c005210 0x7fe39c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b3fff640 1 -- 172.21.15.16:0/370268047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c00c130 con 0x7fe39c005210 2024-04-25T10:09:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b3fff640 1 -- 172.21.15.16:0/370268047 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c00c2e0 con 0x7fe39c004800 2024-04-25T10:09:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b3fff640 1 -- 172.21.15.16:0/370268047 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c00c4e0 con 0x7fe39c003a00 2024-04-25T10:09:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b9f06640 1 --2- 172.21.15.16:0/370268047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe39c005210 0x7fe39c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b9705640 1 --2- 172.21.15.16:0/370268047 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe39c003a00 0x7fe39c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b8f04640 1 --2- 172.21.15.16:0/370268047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe39c004800 0x7fe39c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b9f06640 1 -- 172.21.15.16:0/370268047 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe39c003a00 msgr2=0x7fe39c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b9f06640 1 --2- 172.21.15.16:0/370268047 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe39c003a00 0x7fe39c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b9f06640 1 -- 172.21.15.16:0/370268047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe39c004800 msgr2=0x7fe39c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b9f06640 1 --2- 172.21.15.16:0/370268047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe39c004800 0x7fe39c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.926+0000 7fe3b9f06640 1 -- 172.21.15.16:0/370268047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c003520 con 0x7fe39c005210 2024-04-25T10:09:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.927+0000 7fe3b9f06640 1 -- 172.21.15.16:0/370268047 reap_dead start 2024-04-25T10:09:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.927+0000 7fe3b9f06640 1 --2- 172.21.15.16:0/370268047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe39c005210 0x7fe39c00b9c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4001e20 tx=0x7fe39c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0049861bb575d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.927+0000 7fe3b3fff640 1 -- 172.21.15.16:0/370268047 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4001a70 con 0x7fe39c005210 2024-04-25T10:09:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.927+0000 7fe3b3fff640 1 -- 172.21.15.16:0/370268047 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a402ae40 con 0x7fe39c005210 2024-04-25T10:09:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.927+0000 7fe3b3fff640 1 -- 172.21.15.16:0/370268047 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4003b90 con 0x7fe39c005210 2024-04-25T10:09:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.928+0000 7fe3bb969640 1 -- 172.21.15.16:0/370268047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe39c005210 msgr2=0x7fe39c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.928+0000 7fe3bb969640 1 --2- 172.21.15.16:0/370268047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe39c005210 0x7fe39c00b9c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4001e20 tx=0x7fe39c005830 comp rx=0 tx=0).stop 2024-04-25T10:09:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.928+0000 7fe3bb969640 1 -- 172.21.15.16:0/370268047 shutdown_connections 2024-04-25T10:09:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.928+0000 7fe3bb969640 1 --2- 172.21.15.16:0/370268047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe39c005210 0x7fe39c00b9c0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.928+0000 7fe3bb969640 1 -- 172.21.15.16:0/370268047 >> 172.21.15.16:0/370268047 conn(0x7fe3b40b2370 msgr2=0x7fe3b40b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.928+0000 7fe3bb969640 1 -- 172.21.15.16:0/370268047 shutdown_connections 2024-04-25T10:09:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.928+0000 7fe3bb969640 1 -- 172.21.15.16:0/370268047 wait complete. 2024-04-25T10:09:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.929+0000 7fe3bb969640 1 Processor -- start 2024-04-25T10:09:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.929+0000 7fe3bb969640 1 -- start start 2024-04-25T10:09:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.930+0000 7fe3bb969640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe39c005210 0x7fe3b41521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.930+0000 7fe3bb969640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3b41434f0 0x7fe3b415d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.930+0000 7fe3bb969640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3b4143970 0x7fe3b415d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.930+0000 7fe3bb969640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b40c7080 con 0x7fe39c005210 2024-04-25T10:09:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.930+0000 7fe3bb969640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b415e580 con 0x7fe3b4143970 2024-04-25T10:09:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.930+0000 7fe3bb969640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b40c7200 con 0x7fe3b41434f0 2024-04-25T10:09:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.930+0000 7fe3b9705640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe39c005210 0x7fe3b41521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.930+0000 7fe3b9705640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe39c005210 0x7fe3b41521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40640/0 (socket says 172.21.15.16:40640) 2024-04-25T10:09:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.930+0000 7fe3b9705640 1 -- 172.21.15.16:0/2472854925 learned_addr learned my addr 172.21.15.16:0/2472854925 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.930+0000 7fe3b9f06640 1 --2- 172.21.15.16:0/2472854925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3b4143970 0x7fe3b415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.930+0000 7fe3b8f04640 1 --2- 172.21.15.16:0/2472854925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3b41434f0 0x7fe3b415d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.930+0000 7fe3b9705640 1 -- 172.21.15.16:0/2472854925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3b41434f0 msgr2=0x7fe3b415d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.930+0000 7fe3b9705640 1 --2- 172.21.15.16:0/2472854925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3b41434f0 0x7fe3b415d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.930+0000 7fe3b9705640 1 -- 172.21.15.16:0/2472854925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3b4143970 msgr2=0x7fe3b415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.930+0000 7fe3b9705640 1 --2- 172.21.15.16:0/2472854925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3b4143970 0x7fe3b415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.930+0000 7fe3b9705640 1 -- 172.21.15.16:0/2472854925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b40b36b0 con 0x7fe39c005210 2024-04-25T10:09:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.930+0000 7fe3b9705640 1 --2- 172.21.15.16:0/2472854925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe39c005210 0x7fe3b41521f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0118e0 tx=0x7fe3ac011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.931+0000 7fe3b1ffb640 1 -- 172.21.15.16:0/2472854925 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac00faf0 con 0x7fe39c005210 2024-04-25T10:09:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.931+0000 7fe3b1ffb640 1 -- 172.21.15.16:0/2472854925 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac00fc90 con 0x7fe39c005210 2024-04-25T10:09:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.931+0000 7fe3bb969640 1 -- 172.21.15.16:0/2472854925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b40b3990 con 0x7fe39c005210 2024-04-25T10:09:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.931+0000 7fe3b1ffb640 1 -- 172.21.15.16:0/2472854925 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac014440 con 0x7fe39c005210 2024-04-25T10:09:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.931+0000 7fe3bb969640 1 -- 172.21.15.16:0/2472854925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b40b3f50 con 0x7fe39c005210 2024-04-25T10:09:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.932+0000 7fe3bb969640 1 -- 172.21.15.16:0/2472854925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b40749b0 con 0x7fe39c005210 2024-04-25T10:09:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.932+0000 7fe3b1ffb640 1 -- 172.21.15.16:0/2472854925 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3ac012040 con 0x7fe39c005210 2024-04-25T10:09:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.933+0000 7fe3b1ffb640 1 --2- 172.21.15.16:0/2472854925 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe3a006b500 0x7fe3a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.933+0000 7fe3b1ffb640 1 -- 172.21.15.16:0/2472854925 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe3ac08a9a0 con 0x7fe39c005210 2024-04-25T10:09:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.933+0000 7fe3b8f04640 1 --2- 172.21.15.16:0/2472854925 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe3a006b500 0x7fe3a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.933+0000 7fe3b8f04640 1 --2- 172.21.15.16:0/2472854925 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe3a006b500 0x7fe3a006d9c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe3b415dd50 tx=0x7fe3a8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:36.937+0000 7fe3b1ffb640 1 -- 172.21.15.16:0/2472854925 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe3ac019030 con 0x7fe39c005210 2024-04-25T10:09:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:37.154+0000 7fe3bb969640 1 -- 172.21.15.16:0/2472854925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe3b406fb80 con 0x7fe39c005210 2024-04-25T10:09:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:37.154+0000 7fe3b1ffb640 1 -- 172.21.15.16:0/2472854925 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v42) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe3ac059170 con 0x7fe39c005210 2024-04-25T10:09:37.161 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:09:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:37.156+0000 7fe3bb969640 1 -- 172.21.15.16:0/2472854925 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe3a006b500 msgr2=0x7fe3a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:37.156+0000 7fe3bb969640 1 --2- 172.21.15.16:0/2472854925 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe3a006b500 0x7fe3a006d9c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe3b415dd50 tx=0x7fe3a8009040 comp rx=0 tx=0).stop 2024-04-25T10:09:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:37.156+0000 7fe3bb969640 1 -- 172.21.15.16:0/2472854925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe39c005210 msgr2=0x7fe3b41521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:37.156+0000 7fe3bb969640 1 --2- 172.21.15.16:0/2472854925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe39c005210 0x7fe3b41521f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0118e0 tx=0x7fe3ac011da0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:37.156+0000 7fe3bb969640 1 -- 172.21.15.16:0/2472854925 shutdown_connections 2024-04-25T10:09:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:37.156+0000 7fe3bb969640 1 --2- 172.21.15.16:0/2472854925 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe3a006b500 0x7fe3a006d9c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:37.156+0000 7fe3bb969640 1 --2- 172.21.15.16:0/2472854925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3b4143970 0x7fe3b415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:37.156+0000 7fe3bb969640 1 --2- 172.21.15.16:0/2472854925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3b41434f0 0x7fe3b415d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:37.156+0000 7fe3bb969640 1 --2- 172.21.15.16:0/2472854925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe39c005210 0x7fe3b41521f0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:37.156+0000 7fe3bb969640 1 -- 172.21.15.16:0/2472854925 >> 172.21.15.16:0/2472854925 conn(0x7fe3b40b2370 msgr2=0x7fe3b415b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:37.157+0000 7fe3bb969640 1 -- 172.21.15.16:0/2472854925 shutdown_connections 2024-04-25T10:09:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:37.157+0000 7fe3bb969640 1 -- 172.21.15.16:0/2472854925 wait complete. 2024-04-25T10:09:37.173 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:09:37.173 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:09:37.173 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-04-25T10:09:37.173 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:09:37.173 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:37.173 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:09:37.173 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:09:37.173 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:09:37.174 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:37.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:37.174 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-04-25T10:09:37.189 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:09:37.190 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:09:37.190 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:09:37.190 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:37.190 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:09:37.190 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:37.190 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:09:37.190 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:37.190 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-04-25T10:09:37.190 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:09:37.190 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:37.190 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-04-25T10:09:37.190 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:09:37.191 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:37.191 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-04-25T10:09:37.191 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:09:37.191 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42383sec preferred_lft 42383sec 2024-04-25T10:09:37.191 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-25T10:09:37.191 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:37.191 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:09:37.191 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:37.191 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-04-25T10:09:37.191 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:09:37.191 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:37.191 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:09:37.191 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:37.191 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f06b:fdff:fed4:a335/64 scope link 2024-04-25T10:09:37.191 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:37.192 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:09:37.192 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:37.192 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::90ad:4fff:fe91:3c33/64 scope link 2024-04-25T10:09:37.192 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:37.192 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:09:37.192 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether be:14:4a:3c:16:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:37.192 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::bc14:4aff:fe3c:1605/64 scope link 2024-04-25T10:09:37.192 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:37.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:37.192 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:09:37.252 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:09:37.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:37.254 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-04-25T10:09:37.309 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-04-25T10:09:37.309 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:09:37.309 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:09:37.310 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:09:37.310 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:09:37.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:37.310 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:09:37.310 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:09:37.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:09:37.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:09:37.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:09:37.311 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:09:37.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:37.395 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:09:37.410 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:09:37.411 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:09:37.411 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:37.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:37.411 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-04-25T10:09:37.475 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:09:37.476 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:37.531 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:37.531 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:09:37.531 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:37.586 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:09:37.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:37.586 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:37.670 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:37.670 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:09:37.670 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:09:37.678 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:37.679 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:09:37.679 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:09:37.679 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:09:37.679 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:09:37.683 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:37.684 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:09:37.684 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:09:37.684 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:09:37.684 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:09:37.684 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:09:37.684 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:09:37.684 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:09:37.684 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:09:37.684 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:09:37.685 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:09:37.685 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:09:37.685 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:09:37.686 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:09:37.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:37.686 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:37.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:37.751 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:09:37.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:37.808 INFO:teuthology.orchestra.run.smithi016.stdout:client6915 2024-04-25T10:09:37.808 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:09:37.808 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:37.809 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:37.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:37.865 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-04-25T10:09:37.921 INFO:teuthology.orchestra.run.smithi016.stdout:file_undamaged 2024-04-25T10:09:37.922 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:09:37.922 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:09:37.922 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:09:37.923 DEBUG:teuthology.orchestra.run.smithi016:> import json 2024-04-25T10:09:37.923 DEBUG:teuthology.orchestra.run.smithi016:> import sys 2024-04-25T10:09:37.923 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:09:37.923 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-04-25T10:09:37.923 DEBUG:teuthology.orchestra.run.smithi016:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-25T10:09:37.923 DEBUG:teuthology.orchestra.run.smithi016:> except OSError as e: 2024-04-25T10:09:37.923 DEBUG:teuthology.orchestra.run.smithi016:> sys.exit(e.errno) 2024-04-25T10:09:37.923 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:09:37.923 DEBUG:teuthology.orchestra.run.smithi016:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:09:37.923 DEBUG:teuthology.orchestra.run.smithi016:> print(json.dumps( 2024-04-25T10:09:37.923 DEBUG:teuthology.orchestra.run.smithi016:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:09:37.923 DEBUG:teuthology.orchestra.run.smithi016:> indent=2)) 2024-04-25T10:09:37.923 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:09:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T10:09:38.038 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T10:09:38.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:38.039 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-25T10:09:38.095 INFO:teuthology.orchestra.run.smithi016.stdout:2 2024-04-25T10:09:38.096 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:09:38.096 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:38.151 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:09:38.151 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:38.207 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:38.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.342+0000 7f220d770640 1 Processor -- start 2024-04-25T10:09:38.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.342+0000 7f220d770640 1 -- start start 2024-04-25T10:09:38.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.343+0000 7f220d770640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f220815a870 0x7f22080cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.343+0000 7f220d770640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22080cfc40 0x7f220815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.343+0000 7f220d770640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22080cdb10 con 0x7f220806e2e0 2024-04-25T10:09:38.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.343+0000 7f220d770640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22080ce720 con 0x7f220815a870 2024-04-25T10:09:38.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.343+0000 7f220d770640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22080cddf0 con 0x7f22080cfc40 2024-04-25T10:09:38.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.343+0000 7f22077fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22080cfc40 0x7f220815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.343+0000 7f22077fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22080cfc40 0x7f220815ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:43774/0 (socket says 172.21.15.16:43774) 2024-04-25T10:09:38.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.343+0000 7f22077fe640 1 -- 172.21.15.16:0/1328398869 learned_addr learned my addr 172.21.15.16:0/1328398869 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:38.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.343+0000 7f2206ffd640 1 --2- 172.21.15.16:0/1328398869 >> v2:172.21.15.186:3300/0 conn(0x7f220815a870 0x7f22080cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.344+0000 7f22077fe640 1 -- 172.21.15.16:0/1328398869 >> v2:172.21.15.186:3300/0 conn(0x7f220815a870 msgr2=0x7f22080cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:38.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.344+0000 7f22077fe640 1 --2- 172.21.15.16:0/1328398869 >> v2:172.21.15.186:3300/0 conn(0x7f220815a870 0x7f22080cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.344+0000 7f22077fe640 1 -- 172.21.15.16:0/1328398869 >> v1:172.21.15.186:6789/0 conn(0x7f220806e2e0 legacy=0x7f22080cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.344+0000 7f22067fc640 1 -- 172.21.15.16:0/1328398869 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693329619 0 0) 0x7f22080cdb10 con 0x7f220806e2e0 2024-04-25T10:09:38.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.344+0000 7f22077fe640 1 -- 172.21.15.16:0/1328398869 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22080ce0d0 con 0x7f22080cfc40 2024-04-25T10:09:38.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.345+0000 7f22077fe640 1 --2- 172.21.15.16:0/1328398869 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22080cfc40 0x7f220815ac50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2200009e50 tx=0x7f220002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=642b014876182773 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.345+0000 7f22067fc640 1 -- 172.21.15.16:0/1328398869 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200036070 con 0x7f22080cfc40 2024-04-25T10:09:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.345+0000 7f22067fc640 1 -- 172.21.15.16:0/1328398869 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2200031600 con 0x7f22080cfc40 2024-04-25T10:09:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.345+0000 7f22067fc640 1 -- 172.21.15.16:0/1328398869 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22000319d0 con 0x7f22080cfc40 2024-04-25T10:09:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.345+0000 7f220d770640 1 -- 172.21.15.16:0/1328398869 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22080cfc40 msgr2=0x7f220815ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.345+0000 7f220d770640 1 --2- 172.21.15.16:0/1328398869 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22080cfc40 0x7f220815ac50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2200009e50 tx=0x7f220002a710 comp rx=0 tx=0).stop 2024-04-25T10:09:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.345+0000 7f220d770640 1 -- 172.21.15.16:0/1328398869 shutdown_connections 2024-04-25T10:09:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.345+0000 7f220d770640 1 --2- 172.21.15.16:0/1328398869 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22080cfc40 0x7f220815ac50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.345+0000 7f220d770640 1 --2- 172.21.15.16:0/1328398869 >> v2:172.21.15.186:3300/0 conn(0x7f220815a870 0x7f22080cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.345+0000 7f220d770640 1 -- 172.21.15.16:0/1328398869 >> 172.21.15.16:0/1328398869 conn(0x7f220806aba0 msgr2=0x7f220806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.345+0000 7f220d770640 1 -- 172.21.15.16:0/1328398869 shutdown_connections 2024-04-25T10:09:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.345+0000 7f220d770640 1 -- 172.21.15.16:0/1328398869 wait complete. 2024-04-25T10:09:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.346+0000 7f220d770640 1 Processor -- start 2024-04-25T10:09:38.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.347+0000 7f220d770640 1 -- start start 2024-04-25T10:09:38.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.347+0000 7f220d770640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f220806e2e0 0x7f22080adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.347+0000 7f220d770640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22080cfc40 0x7f2208144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.347+0000 7f220d770640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f220815a870 0x7f22080a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.347+0000 7f220d770640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22080cd700 con 0x7f220815a870 2024-04-25T10:09:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.347+0000 7f220d770640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22080cd580 con 0x7f220806e2e0 2024-04-25T10:09:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.347+0000 7f220d770640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22080cd880 con 0x7f22080cfc40 2024-04-25T10:09:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.347+0000 7f22077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f220815a870 0x7f22080a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.347+0000 7f22077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f220815a870 0x7f22080a99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40654/0 (socket says 172.21.15.16:40654) 2024-04-25T10:09:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.347+0000 7f22077fe640 1 -- 172.21.15.16:0/1891140259 learned_addr learned my addr 172.21.15.16:0/1891140259 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.347+0000 7f2206ffd640 1 --2- 172.21.15.16:0/1891140259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f220806e2e0 0x7f22080adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.348+0000 7f21fffff640 1 --2- 172.21.15.16:0/1891140259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22080cfc40 0x7f2208144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.348+0000 7f22077fe640 1 -- 172.21.15.16:0/1891140259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22080cfc40 msgr2=0x7f2208144850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.348+0000 7f22077fe640 1 --2- 172.21.15.16:0/1891140259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22080cfc40 0x7f2208144850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.348+0000 7f22077fe640 1 -- 172.21.15.16:0/1891140259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f220806e2e0 msgr2=0x7f22080adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.348+0000 7f22077fe640 1 --2- 172.21.15.16:0/1891140259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f220806e2e0 0x7f22080adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.348+0000 7f22077fe640 1 -- 172.21.15.16:0/1891140259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22080ae520 con 0x7f220815a870 2024-04-25T10:09:38.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.348+0000 7f22077fe640 1 --2- 172.21.15.16:0/1891140259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f220815a870 0x7f22080a99d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2200034040 tx=0x7f2200004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.348+0000 7f21ff7fe640 1 -- 172.21.15.16:0/1891140259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200036050 con 0x7f220815a870 2024-04-25T10:09:38.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.348+0000 7f21ff7fe640 1 -- 172.21.15.16:0/1891140259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2200042070 con 0x7f220815a870 2024-04-25T10:09:38.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.348+0000 7f220d770640 1 -- 172.21.15.16:0/1891140259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22081e3e60 con 0x7f220815a870 2024-04-25T10:09:38.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.348+0000 7f21ff7fe640 1 -- 172.21.15.16:0/1891140259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200009510 con 0x7f220815a870 2024-04-25T10:09:38.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.348+0000 7f220d770640 1 -- 172.21.15.16:0/1891140259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22081e46c0 con 0x7f220815a870 2024-04-25T10:09:38.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.349+0000 7f220d770640 1 -- 172.21.15.16:0/1891140259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22081e4210 con 0x7f220815a870 2024-04-25T10:09:38.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.350+0000 7f21ff7fe640 1 -- 172.21.15.16:0/1891140259 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2200007bf0 con 0x7f220815a870 2024-04-25T10:09:38.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.350+0000 7f21ff7fe640 1 --2- 172.21.15.16:0/1891140259 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f21e806b4d0 0x7f21e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.350+0000 7f21ff7fe640 1 -- 172.21.15.16:0/1891140259 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f22000ab040 con 0x7f220815a870 2024-04-25T10:09:38.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.350+0000 7f2206ffd640 1 --2- 172.21.15.16:0/1891140259 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f21e806b4d0 0x7f21e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.351+0000 7f2206ffd640 1 --2- 172.21.15.16:0/1891140259 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f21e806b4d0 0x7f21e806d990 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f22081f4810 tx=0x7f21f8009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.354+0000 7f21ff7fe640 1 -- 172.21.15.16:0/1891140259 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f22080ae520 con 0x7f220815a870 2024-04-25T10:09:38.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.535+0000 7f220d770640 1 -- 172.21.15.16:0/1891140259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f220815adf0 con 0x7f220815a870 2024-04-25T10:09:38.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.536+0000 7f21ff7fe640 1 -- 172.21.15.16:0/1891140259 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16739 (secure 0 0 0) 0x7f2200079770 con 0x7f220815a870 2024-04-25T10:09:38.543 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:09:38.543 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:09:31.374274+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:20.361009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:21.048892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.186:6837/3292077067":"2024-04-26T10:09:31.374252+0000","172.21.15.186:6836/3292077067":"2024-04-26T10:09:31.374252+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.538+0000 7f220d770640 1 -- 172.21.15.16:0/1891140259 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f21e806b4d0 msgr2=0x7f21e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.538+0000 7f220d770640 1 --2- 172.21.15.16:0/1891140259 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f21e806b4d0 0x7f21e806d990 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f22081f4810 tx=0x7f21f8009190 comp rx=0 tx=0).stop 2024-04-25T10:09:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.538+0000 7f220d770640 1 -- 172.21.15.16:0/1891140259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f220815a870 msgr2=0x7f22080a99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.538+0000 7f220d770640 1 --2- 172.21.15.16:0/1891140259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f220815a870 0x7f22080a99d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2200034040 tx=0x7f2200004a40 comp rx=0 tx=0).stop 2024-04-25T10:09:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.538+0000 7f220d770640 1 -- 172.21.15.16:0/1891140259 shutdown_connections 2024-04-25T10:09:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.538+0000 7f220d770640 1 --2- 172.21.15.16:0/1891140259 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f21e806b4d0 0x7f21e806d990 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.538+0000 7f220d770640 1 --2- 172.21.15.16:0/1891140259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f220815a870 0x7f22080a99d0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.538+0000 7f220d770640 1 --2- 172.21.15.16:0/1891140259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22080cfc40 0x7f2208144850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.538+0000 7f220d770640 1 --2- 172.21.15.16:0/1891140259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f220806e2e0 0x7f22080adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.538+0000 7f220d770640 1 -- 172.21.15.16:0/1891140259 >> 172.21.15.16:0/1891140259 conn(0x7f220806aba0 msgr2=0x7f220815cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.539+0000 7f220d770640 1 -- 172.21.15.16:0/1891140259 shutdown_connections 2024-04-25T10:09:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.539+0000 7f220d770640 1 -- 172.21.15.16:0/1891140259 wait complete. 2024-04-25T10:09:38.555 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:09:20.361009+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:09:21.048892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:38.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:38.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.654+0000 7f58a8ce2640 1 Processor -- start 2024-04-25T10:09:38.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.654+0000 7f58a8ce2640 1 -- start start 2024-04-25T10:09:38.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.655+0000 7f58a8ce2640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58a415dca0 0x7f58a40ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.655+0000 7f58a8ce2640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f58a40d3b90 0x7f58a40d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.655+0000 7f58a8ce2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58a40cee80 0x7f58a40d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.655+0000 7f58a8ce2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a40b6b20 con 0x7f58a40cee80 2024-04-25T10:09:38.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.655+0000 7f58a8ce2640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a40d4090 con 0x7f58a40d3b90 2024-04-25T10:09:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.655+0000 7f58a8ce2640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40d4370 con 0x7f58a415dca0 2024-04-25T10:09:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.655+0000 7f58a2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58a40cee80 0x7f58a40d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.655+0000 7f58a2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58a40cee80 0x7f58a40d0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40670/0 (socket says 172.21.15.16:40670) 2024-04-25T10:09:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.655+0000 7f58a2d9d640 1 -- 172.21.15.16:0/3468466094 learned_addr learned my addr 172.21.15.16:0/3468466094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.655+0000 7f58a2d9d640 1 -- 172.21.15.16:0/3468466094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58a415dca0 msgr2=0x7f58a40ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.655+0000 7f58a2d9d640 1 --2- 172.21.15.16:0/3468466094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58a415dca0 0x7f58a40ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.655+0000 7f58a2d9d640 1 -- 172.21.15.16:0/3468466094 >> v2:172.21.15.186:3300/0 conn(0x7f58a40d3b90 msgr2=0x7f58a40d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.655+0000 7f58a2d9d640 1 --2- 172.21.15.16:0/3468466094 >> v2:172.21.15.186:3300/0 conn(0x7f58a40d3b90 0x7f58a40d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.655+0000 7f58a2d9d640 1 -- 172.21.15.16:0/3468466094 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a40d4650 con 0x7f58a40cee80 2024-04-25T10:09:38.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.656+0000 7f58a2d9d640 1 --2- 172.21.15.16:0/3468466094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a40cee80 0x7f58a40d0f60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f58a406d210 tx=0x7f589c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a95b5500b13f7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.656+0000 7f58a1d9b640 1 -- 172.21.15.16:0/3468466094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c036070 con 0x7f58a40cee80 2024-04-25T10:09:38.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.656+0000 7f58a1d9b640 1 -- 172.21.15.16:0/3468466094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c031600 con 0x7f58a40cee80 2024-04-25T10:09:38.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.656+0000 7f58a1d9b640 1 -- 172.21.15.16:0/3468466094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0319d0 con 0x7f58a40cee80 2024-04-25T10:09:38.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.656+0000 7f58a8ce2640 1 -- 172.21.15.16:0/3468466094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a40cee80 msgr2=0x7f58a40d0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.656+0000 7f58a8ce2640 1 --2- 172.21.15.16:0/3468466094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a40cee80 0x7f58a40d0f60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f58a406d210 tx=0x7f589c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.656+0000 7f58a8ce2640 1 -- 172.21.15.16:0/3468466094 shutdown_connections 2024-04-25T10:09:38.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.656+0000 7f58a8ce2640 1 --2- 172.21.15.16:0/3468466094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a40cee80 0x7f58a40d0f60 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.656+0000 7f58a8ce2640 1 --2- 172.21.15.16:0/3468466094 >> v2:172.21.15.186:3300/0 conn(0x7f58a40d3b90 0x7f58a40d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.657+0000 7f58a8ce2640 1 --2- 172.21.15.16:0/3468466094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58a415dca0 0x7f58a40ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.657+0000 7f58a8ce2640 1 -- 172.21.15.16:0/3468466094 >> 172.21.15.16:0/3468466094 conn(0x7f58a40726c0 msgr2=0x7f58a4072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:38.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.657+0000 7f58a8ce2640 1 -- 172.21.15.16:0/3468466094 shutdown_connections 2024-04-25T10:09:38.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.657+0000 7f58a8ce2640 1 -- 172.21.15.16:0/3468466094 wait complete. 2024-04-25T10:09:38.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.658+0000 7f58a8ce2640 1 Processor -- start 2024-04-25T10:09:38.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.658+0000 7f58a8ce2640 1 -- start start 2024-04-25T10:09:38.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f58a8ce2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58a40cee80 0x7f58a40ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f58a8ce2640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58a40d3b90 0x7f58a41e8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f58a8ce2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a415dca0 0x7f58a41e88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f58a8ce2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a415e200 con 0x7f58a415dca0 2024-04-25T10:09:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f58a8ce2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a415e080 con 0x7f58a40cee80 2024-04-25T10:09:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f58a8ce2640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40b2500 con 0x7f58a40d3b90 2024-04-25T10:09:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f58a2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a415dca0 0x7f58a41e88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f58a2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a415dca0 0x7f58a41e88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40678/0 (socket says 172.21.15.16:40678) 2024-04-25T10:09:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f58a2d9d640 1 -- 172.21.15.16:0/360261561 learned_addr learned my addr 172.21.15.16:0/360261561 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f58a259c640 1 --2- 172.21.15.16:0/360261561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58a40cee80 0x7f58a40ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f589bfff640 1 --2- 172.21.15.16:0/360261561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58a40d3b90 0x7f58a41e8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f58a2d9d640 1 -- 172.21.15.16:0/360261561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58a40d3b90 msgr2=0x7f58a41e8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f58a2d9d640 1 --2- 172.21.15.16:0/360261561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58a40d3b90 0x7f58a41e8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f58a2d9d640 1 -- 172.21.15.16:0/360261561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58a40cee80 msgr2=0x7f58a40ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f58a2d9d640 1 --2- 172.21.15.16:0/360261561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58a40cee80 0x7f58a40ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f58a2d9d640 1 -- 172.21.15.16:0/360261561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a41e72d0 con 0x7f58a415dca0 2024-04-25T10:09:38.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f58a2d9d640 1 --2- 172.21.15.16:0/360261561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a415dca0 0x7f58a41e88b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f589c034040 tx=0x7f589c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f589affd640 1 -- 172.21.15.16:0/360261561 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c041020 con 0x7f58a415dca0 2024-04-25T10:09:38.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f589affd640 1 -- 172.21.15.16:0/360261561 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c007780 con 0x7f58a415dca0 2024-04-25T10:09:38.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f58a8ce2640 1 -- 172.21.15.16:0/360261561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a41e75b0 con 0x7f58a415dca0 2024-04-25T10:09:38.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.659+0000 7f589affd640 1 -- 172.21.15.16:0/360261561 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c036040 con 0x7f58a415dca0 2024-04-25T10:09:38.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.660+0000 7f58a8ce2640 1 -- 172.21.15.16:0/360261561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a41e7b70 con 0x7f58a415dca0 2024-04-25T10:09:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.661+0000 7f58a8ce2640 1 -- 172.21.15.16:0/360261561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a41e8060 con 0x7f58a415dca0 2024-04-25T10:09:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.661+0000 7f589affd640 1 -- 172.21.15.16:0/360261561 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f589c044020 con 0x7f58a415dca0 2024-04-25T10:09:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.661+0000 7f589affd640 1 --2- 172.21.15.16:0/360261561 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f588406b4d0 0x7f588406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.662+0000 7f589affd640 1 -- 172.21.15.16:0/360261561 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f589c0ab0d0 con 0x7f58a415dca0 2024-04-25T10:09:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.662+0000 7f58a259c640 1 --2- 172.21.15.16:0/360261561 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f588406b4d0 0x7f588406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.662+0000 7f58a259c640 1 --2- 172.21.15.16:0/360261561 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f588406b4d0 0x7f588406d990 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f589400b810 tx=0x7f5894008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.666+0000 7f589affd640 1 -- 172.21.15.16:0/360261561 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f589c04b030 con 0x7f58a415dca0 2024-04-25T10:09:38.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.848+0000 7f58a8ce2640 1 -- 172.21.15.16:0/360261561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f58a40750b0 con 0x7f58a415dca0 2024-04-25T10:09:38.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.849+0000 7f589affd640 1 -- 172.21.15.16:0/360261561 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f589c079fb0 con 0x7f58a415dca0 2024-04-25T10:09:38.855 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:09:38.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.851+0000 7f58a8ce2640 1 -- 172.21.15.16:0/360261561 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f588406b4d0 msgr2=0x7f588406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.851+0000 7f58a8ce2640 1 --2- 172.21.15.16:0/360261561 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f588406b4d0 0x7f588406d990 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f589400b810 tx=0x7f5894008040 comp rx=0 tx=0).stop 2024-04-25T10:09:38.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.851+0000 7f58a8ce2640 1 -- 172.21.15.16:0/360261561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a415dca0 msgr2=0x7f58a41e88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.851+0000 7f58a8ce2640 1 --2- 172.21.15.16:0/360261561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a415dca0 0x7f58a41e88b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f589c034040 tx=0x7f589c004a40 comp rx=0 tx=0).stop 2024-04-25T10:09:38.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.851+0000 7f58a8ce2640 1 -- 172.21.15.16:0/360261561 shutdown_connections 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.851+0000 7f58a8ce2640 1 --2- 172.21.15.16:0/360261561 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f588406b4d0 0x7f588406d990 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.851+0000 7f58a8ce2640 1 --2- 172.21.15.16:0/360261561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a415dca0 0x7f58a41e88b0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.851+0000 7f58a8ce2640 1 --2- 172.21.15.16:0/360261561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58a40d3b90 0x7f58a41e8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.851+0000 7f58a8ce2640 1 --2- 172.21.15.16:0/360261561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58a40cee80 0x7f58a40ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.851+0000 7f58a8ce2640 1 -- 172.21.15.16:0/360261561 >> 172.21.15.16:0/360261561 conn(0x7f58a40726c0 msgr2=0x7f58a40b8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.851+0000 7f58a8ce2640 1 -- 172.21.15.16:0/360261561 shutdown_connections 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.851+0000 7f58a8ce2640 1 -- 172.21.15.16:0/360261561 wait complete. 2024-04-25T10:09:38.868 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:09:38.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.967+0000 7ff468bab640 1 Processor -- start 2024-04-25T10:09:38.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.967+0000 7ff468bab640 1 -- start start 2024-04-25T10:09:38.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.968+0000 7ff468bab640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff46415dca0 0x7ff4640ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.968+0000 7ff468bab640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4640d1130 0x7ff4640b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.968+0000 7ff468bab640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4640d4090 con 0x7ff4640d3b90 2024-04-25T10:09:38.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.968+0000 7ff468bab640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4640b6b20 con 0x7ff4640d1130 2024-04-25T10:09:38.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.968+0000 7ff468bab640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4640d4370 con 0x7ff46415dca0 2024-04-25T10:09:38.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.968+0000 7ff462d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4640d1130 0x7ff4640b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.968+0000 7ff45bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff4640d3b90 0x7ff4640d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38912/0 (socket says 172.21.15.16:38912) 2024-04-25T10:09:38.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.968+0000 7ff45bfff640 1 -- 172.21.15.16:0/4250732530 learned_addr learned my addr 172.21.15.16:0/4250732530 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:38.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.968+0000 7ff46259c640 1 --2- 172.21.15.16:0/4250732530 >> v2:172.21.15.186:3300/0 conn(0x7ff46415dca0 0x7ff4640ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.968+0000 7ff461d9b640 1 -- 172.21.15.16:0/4250732530 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1823532584 0 0) 0x7ff4640d4090 con 0x7ff4640d3b90 2024-04-25T10:09:38.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.968+0000 7ff461d9b640 1 -- 172.21.15.16:0/4250732530 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff450003520 con 0x7ff4640d3b90 2024-04-25T10:09:38.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.968+0000 7ff462d9d640 1 -- 172.21.15.16:0/4250732530 >> v2:172.21.15.186:3300/0 conn(0x7ff46415dca0 msgr2=0x7ff4640ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:38.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.968+0000 7ff462d9d640 1 --2- 172.21.15.16:0/4250732530 >> v2:172.21.15.186:3300/0 conn(0x7ff46415dca0 0x7ff4640ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.968+0000 7ff462d9d640 1 -- 172.21.15.16:0/4250732530 >> v1:172.21.15.16:6789/0 conn(0x7ff4640d3b90 legacy=0x7ff4640d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.968+0000 7ff462d9d640 1 -- 172.21.15.16:0/4250732530 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4640d4650 con 0x7ff4640d1130 2024-04-25T10:09:38.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.969+0000 7ff462d9d640 1 --2- 172.21.15.16:0/4250732530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4640d1130 0x7ff4640b2500 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff4640b6340 tx=0x7ff45c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394bf72fef73c1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.969+0000 7ff461d9b640 1 -- 172.21.15.16:0/4250732530 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c007740 con 0x7ff4640d1130 2024-04-25T10:09:38.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.969+0000 7ff461d9b640 1 -- 172.21.15.16:0/4250732530 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45c0078e0 con 0x7ff4640d1130 2024-04-25T10:09:38.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.969+0000 7ff461d9b640 1 -- 172.21.15.16:0/4250732530 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c007c90 con 0x7ff4640d1130 2024-04-25T10:09:38.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.970+0000 7ff468bab640 1 -- 172.21.15.16:0/4250732530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4640d1130 msgr2=0x7ff4640b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.970+0000 7ff468bab640 1 --2- 172.21.15.16:0/4250732530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4640d1130 0x7ff4640b2500 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff4640b6340 tx=0x7ff45c02a680 comp rx=0 tx=0).stop 2024-04-25T10:09:38.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.970+0000 7ff468bab640 1 -- 172.21.15.16:0/4250732530 shutdown_connections 2024-04-25T10:09:38.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.970+0000 7ff468bab640 1 --2- 172.21.15.16:0/4250732530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4640d1130 0x7ff4640b2500 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.970+0000 7ff468bab640 1 --2- 172.21.15.16:0/4250732530 >> v2:172.21.15.186:3300/0 conn(0x7ff46415dca0 0x7ff4640ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.970+0000 7ff468bab640 1 -- 172.21.15.16:0/4250732530 >> 172.21.15.16:0/4250732530 conn(0x7ff4640726c0 msgr2=0x7ff464072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:38.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.970+0000 7ff468bab640 1 -- 172.21.15.16:0/4250732530 shutdown_connections 2024-04-25T10:09:38.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.970+0000 7ff468bab640 1 -- 172.21.15.16:0/4250732530 wait complete. 2024-04-25T10:09:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.971+0000 7ff468bab640 1 Processor -- start 2024-04-25T10:09:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.971+0000 7ff468bab640 1 -- start start 2024-04-25T10:09:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff468bab640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4640d1130 0x7ff4640cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff468bab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4640d3b90 0x7ff4641f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff468bab640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff46415dca0 0x7ff46414ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff468bab640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46415e200 con 0x7ff4640d3b90 2024-04-25T10:09:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff468bab640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46415e080 con 0x7ff46415dca0 2024-04-25T10:09:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff468bab640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4640cee80 con 0x7ff4640d1130 2024-04-25T10:09:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff45bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4640d3b90 0x7ff4641f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff45bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4640d3b90 0x7ff4641f2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40686/0 (socket says 172.21.15.16:40686) 2024-04-25T10:09:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff45bfff640 1 -- 172.21.15.16:0/3353250440 learned_addr learned my addr 172.21.15.16:0/3353250440 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff462d9d640 1 --2- 172.21.15.16:0/3353250440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff46415dca0 0x7ff46414ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff46259c640 1 --2- 172.21.15.16:0/3353250440 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4640d1130 0x7ff4640cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff45bfff640 1 -- 172.21.15.16:0/3353250440 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4640d1130 msgr2=0x7ff4640cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff45bfff640 1 --2- 172.21.15.16:0/3353250440 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4640d1130 0x7ff4640cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff45bfff640 1 -- 172.21.15.16:0/3353250440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff46415dca0 msgr2=0x7ff46414ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff45bfff640 1 --2- 172.21.15.16:0/3353250440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff46415dca0 0x7ff46414ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff45bfff640 1 -- 172.21.15.16:0/3353250440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff464150a70 con 0x7ff4640d3b90 2024-04-25T10:09:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff462d9d640 1 --2- 172.21.15.16:0/3353250440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff46415dca0 0x7ff46414ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff46259c640 1 --2- 172.21.15.16:0/3353250440 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4640d1130 0x7ff4640cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff45bfff640 1 --2- 172.21.15.16:0/3353250440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4640d3b90 0x7ff4641f2d90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00c400 tx=0x7ff44c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff45affd640 1 -- 172.21.15.16:0/3353250440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c017070 con 0x7ff4640d3b90 2024-04-25T10:09:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff45affd640 1 -- 172.21.15.16:0/3353250440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44c005d00 con 0x7ff4640d3b90 2024-04-25T10:09:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff468bab640 1 -- 172.21.15.16:0/3353250440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46414d820 con 0x7ff4640d3b90 2024-04-25T10:09:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.972+0000 7ff45affd640 1 -- 172.21.15.16:0/3353250440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c007310 con 0x7ff4640d3b90 2024-04-25T10:09:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.973+0000 7ff468bab640 1 -- 172.21.15.16:0/3353250440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4640cab60 con 0x7ff4640d3b90 2024-04-25T10:09:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.973+0000 7ff468bab640 1 -- 172.21.15.16:0/3353250440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4640750b0 con 0x7ff4640d3b90 2024-04-25T10:09:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.974+0000 7ff45affd640 1 -- 172.21.15.16:0/3353250440 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff44c01a070 con 0x7ff4640d3b90 2024-04-25T10:09:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.974+0000 7ff45affd640 1 --2- 172.21.15.16:0/3353250440 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff44406b4d0 0x7ff44406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.974+0000 7ff45affd640 1 -- 172.21.15.16:0/3353250440 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff44c089700 con 0x7ff4640d3b90 2024-04-25T10:09:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.975+0000 7ff46259c640 1 --2- 172.21.15.16:0/3353250440 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff44406b4d0 0x7ff44406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.975+0000 7ff46259c640 1 --2- 172.21.15.16:0/3353250440 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff44406b4d0 0x7ff44406d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff454006fd0 tx=0x7ff454008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:38.979+0000 7ff45affd640 1 -- 172.21.15.16:0/3353250440 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff44c058510 con 0x7ff4640d3b90 2024-04-25T10:09:39.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.159+0000 7ff468bab640 1 -- 172.21.15.16:0/3353250440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff4640b8640 con 0x7ff4640d3b90 2024-04-25T10:09:39.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.159+0000 7ff45affd640 1 -- 172.21.15.16:0/3353250440 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff44c057e10 con 0x7ff4640d3b90 2024-04-25T10:09:39.166 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:09:39.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.161+0000 7ff468bab640 1 -- 172.21.15.16:0/3353250440 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff44406b4d0 msgr2=0x7ff44406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.161+0000 7ff468bab640 1 --2- 172.21.15.16:0/3353250440 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff44406b4d0 0x7ff44406d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff454006fd0 tx=0x7ff454008040 comp rx=0 tx=0).stop 2024-04-25T10:09:39.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.161+0000 7ff468bab640 1 -- 172.21.15.16:0/3353250440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4640d3b90 msgr2=0x7ff4641f2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.161+0000 7ff468bab640 1 --2- 172.21.15.16:0/3353250440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4640d3b90 0x7ff4641f2d90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00c400 tx=0x7ff44c00c8c0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.161+0000 7ff468bab640 1 -- 172.21.15.16:0/3353250440 shutdown_connections 2024-04-25T10:09:39.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.161+0000 7ff468bab640 1 --2- 172.21.15.16:0/3353250440 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff44406b4d0 0x7ff44406d990 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.161+0000 7ff468bab640 1 --2- 172.21.15.16:0/3353250440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff46415dca0 0x7ff46414ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.161+0000 7ff468bab640 1 --2- 172.21.15.16:0/3353250440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4640d3b90 0x7ff4641f2d90 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.161+0000 7ff468bab640 1 --2- 172.21.15.16:0/3353250440 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4640d1130 0x7ff4640cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.161+0000 7ff468bab640 1 -- 172.21.15.16:0/3353250440 >> 172.21.15.16:0/3353250440 conn(0x7ff4640726c0 msgr2=0x7ff464160390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:39.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.161+0000 7ff468bab640 1 -- 172.21.15.16:0/3353250440 shutdown_connections 2024-04-25T10:09:39.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.161+0000 7ff468bab640 1 -- 172.21.15.16:0/3353250440 wait complete. 2024-04-25T10:09:39.178 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:09:39.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.276+0000 7f391acdb640 1 Processor -- start 2024-04-25T10:09:39.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.276+0000 7f391acdb640 1 -- start start 2024-04-25T10:09:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.277+0000 7f391acdb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f391415dc80 0x7f3914161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.277+0000 7f391acdb640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f39140b5c00 0x7f39140d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.277+0000 7f391acdb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39141627a0 con 0x7f39140b97c0 2024-04-25T10:09:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.277+0000 7f391acdb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39141621e0 con 0x7f391415dc80 2024-04-25T10:09:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.277+0000 7f391acdb640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39141624c0 con 0x7f39140b5c00 2024-04-25T10:09:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.277+0000 7f3918a77640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f391415dc80 0x7f3914161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.277+0000 7f3913fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f39140b97c0 0x7f39140b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38916/0 (socket says 172.21.15.16:38916) 2024-04-25T10:09:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.277+0000 7f3913fff640 1 -- 172.21.15.16:0/3795565866 learned_addr learned my addr 172.21.15.16:0/3795565866 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.277+0000 7f3918a77640 1 -- 172.21.15.16:0/3795565866 >> v2:172.21.15.186:3300/0 conn(0x7f39140b5c00 msgr2=0x7f39140d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.277+0000 7f3919278640 1 --2- 172.21.15.16:0/3795565866 >> v2:172.21.15.186:3300/0 conn(0x7f39140b5c00 0x7f39140d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.277+0000 7f3918a77640 1 --2- 172.21.15.16:0/3795565866 >> v2:172.21.15.186:3300/0 conn(0x7f39140b5c00 0x7f39140d2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.277+0000 7f3918a77640 1 -- 172.21.15.16:0/3795565866 >> v1:172.21.15.16:6789/0 conn(0x7f39140b97c0 legacy=0x7f39140b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.277+0000 7f3918a77640 1 -- 172.21.15.16:0/3795565866 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39140d1c70 con 0x7f391415dc80 2024-04-25T10:09:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.277+0000 7f39137fe640 1 -- 172.21.15.16:0/3795565866 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167540249 0 0) 0x7f39141627a0 con 0x7f39140b97c0 2024-04-25T10:09:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.278+0000 7f3918a77640 1 --2- 172.21.15.16:0/3795565866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f391415dc80 0x7f3914161870 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f39140b6300 tx=0x7f390802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c78ccc38bb4a076 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.278+0000 7f39137fe640 1 -- 172.21.15.16:0/3795565866 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908037070 con 0x7f391415dc80 2024-04-25T10:09:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.278+0000 7f39137fe640 1 -- 172.21.15.16:0/3795565866 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3908032440 con 0x7f391415dc80 2024-04-25T10:09:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.278+0000 7f39137fe640 1 -- 172.21.15.16:0/3795565866 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908032810 con 0x7f391415dc80 2024-04-25T10:09:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.279+0000 7f391acdb640 1 -- 172.21.15.16:0/3795565866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f391415dc80 msgr2=0x7f3914161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.279+0000 7f391acdb640 1 --2- 172.21.15.16:0/3795565866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f391415dc80 0x7f3914161870 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f39140b6300 tx=0x7f390802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.279+0000 7f391acdb640 1 -- 172.21.15.16:0/3795565866 shutdown_connections 2024-04-25T10:09:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.279+0000 7f391acdb640 1 --2- 172.21.15.16:0/3795565866 >> v2:172.21.15.186:3300/0 conn(0x7f39140b5c00 0x7f39140d2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.279+0000 7f391acdb640 1 --2- 172.21.15.16:0/3795565866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f391415dc80 0x7f3914161870 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.279+0000 7f391acdb640 1 -- 172.21.15.16:0/3795565866 >> 172.21.15.16:0/3795565866 conn(0x7f39140d04f0 msgr2=0x7f39140d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.279+0000 7f391acdb640 1 -- 172.21.15.16:0/3795565866 shutdown_connections 2024-04-25T10:09:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.279+0000 7f391acdb640 1 -- 172.21.15.16:0/3795565866 wait complete. 2024-04-25T10:09:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.280+0000 7f391acdb640 1 Processor -- start 2024-04-25T10:09:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.280+0000 7f391acdb640 1 -- start start 2024-04-25T10:09:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f391acdb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39140b5c00 0x7f39141f7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f391acdb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39140b97c0 0x7f39141f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f391acdb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f391415dc80 0x7f39140b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f391acdb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3914161f30 con 0x7f39140b5c00 2024-04-25T10:09:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f391acdb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3914161db0 con 0x7f391415dc80 2024-04-25T10:09:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f391acdb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39140d29b0 con 0x7f39140b97c0 2024-04-25T10:09:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f3918a77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39140b5c00 0x7f39141f7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f3918a77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39140b5c00 0x7f39141f7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40700/0 (socket says 172.21.15.16:40700) 2024-04-25T10:09:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f3918a77640 1 -- 172.21.15.16:0/3715275173 learned_addr learned my addr 172.21.15.16:0/3715275173 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f3913fff640 1 --2- 172.21.15.16:0/3715275173 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39140b97c0 0x7f39141f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f3919278640 1 --2- 172.21.15.16:0/3715275173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f391415dc80 0x7f39140b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f3918a77640 1 -- 172.21.15.16:0/3715275173 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39140b97c0 msgr2=0x7f39141f0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f3918a77640 1 --2- 172.21.15.16:0/3715275173 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39140b97c0 0x7f39141f0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f3918a77640 1 -- 172.21.15.16:0/3715275173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f391415dc80 msgr2=0x7f39140b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f3918a77640 1 --2- 172.21.15.16:0/3715275173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f391415dc80 0x7f39140b71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f3918a77640 1 -- 172.21.15.16:0/3715275173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39141f7c70 con 0x7f39140b5c00 2024-04-25T10:09:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f3918a77640 1 --2- 172.21.15.16:0/3715275173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39140b5c00 0x7f39141f7730 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3908035040 tx=0x7f39080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f39117fa640 1 -- 172.21.15.16:0/3715275173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908037050 con 0x7f39140b5c00 2024-04-25T10:09:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f39117fa640 1 -- 172.21.15.16:0/3715275173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3908038040 con 0x7f39140b5c00 2024-04-25T10:09:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f39117fa640 1 -- 172.21.15.16:0/3715275173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908004450 con 0x7f39140b5c00 2024-04-25T10:09:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.281+0000 7f391acdb640 1 -- 172.21.15.16:0/3715275173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39141f3b40 con 0x7f39140b5c00 2024-04-25T10:09:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.282+0000 7f391acdb640 1 -- 172.21.15.16:0/3715275173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39141f4100 con 0x7f39140b5c00 2024-04-25T10:09:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.282+0000 7f391acdb640 1 -- 172.21.15.16:0/3715275173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39140750b0 con 0x7f39140b5c00 2024-04-25T10:09:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.283+0000 7f39117fa640 1 -- 172.21.15.16:0/3715275173 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3908030980 con 0x7f39140b5c00 2024-04-25T10:09:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.283+0000 7f39117fa640 1 --2- 172.21.15.16:0/3715275173 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f38f806b4d0 0x7f38f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.283+0000 7f39117fa640 1 -- 172.21.15.16:0/3715275173 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f39080aafe0 con 0x7f39140b5c00 2024-04-25T10:09:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.284+0000 7f3913fff640 1 --2- 172.21.15.16:0/3715275173 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f38f806b4d0 0x7f38f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.284+0000 7f3913fff640 1 --2- 172.21.15.16:0/3715275173 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f38f806b4d0 0x7f38f806d990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3900007fd0 tx=0x7f3900009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.288+0000 7f39117fa640 1 -- 172.21.15.16:0/3715275173 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f39141f7c70 con 0x7f39140b5c00 2024-04-25T10:09:39.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.468+0000 7f391acdb640 1 -- 172.21.15.16:0/3715275173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f39140d1780 con 0x7f39140b5c00 2024-04-25T10:09:39.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.468+0000 7f39117fa640 1 -- 172.21.15.16:0/3715275173 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3908079fb0 con 0x7f39140b5c00 2024-04-25T10:09:39.474 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:09:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.470+0000 7f391acdb640 1 -- 172.21.15.16:0/3715275173 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f38f806b4d0 msgr2=0x7f38f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.470+0000 7f391acdb640 1 --2- 172.21.15.16:0/3715275173 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f38f806b4d0 0x7f38f806d990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3900007fd0 tx=0x7f3900009040 comp rx=0 tx=0).stop 2024-04-25T10:09:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.470+0000 7f391acdb640 1 -- 172.21.15.16:0/3715275173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39140b5c00 msgr2=0x7f39141f7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.470+0000 7f391acdb640 1 --2- 172.21.15.16:0/3715275173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39140b5c00 0x7f39141f7730 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3908035040 tx=0x7f39080046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.470+0000 7f391acdb640 1 -- 172.21.15.16:0/3715275173 shutdown_connections 2024-04-25T10:09:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.470+0000 7f391acdb640 1 --2- 172.21.15.16:0/3715275173 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f38f806b4d0 0x7f38f806d990 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.470+0000 7f391acdb640 1 --2- 172.21.15.16:0/3715275173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f391415dc80 0x7f39140b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.470+0000 7f391acdb640 1 --2- 172.21.15.16:0/3715275173 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39140b97c0 0x7f39141f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.470+0000 7f391acdb640 1 --2- 172.21.15.16:0/3715275173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39140b5c00 0x7f39141f7730 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.470+0000 7f391acdb640 1 -- 172.21.15.16:0/3715275173 >> 172.21.15.16:0/3715275173 conn(0x7f39140d04f0 msgr2=0x7f39140b8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.470+0000 7f391acdb640 1 -- 172.21.15.16:0/3715275173 shutdown_connections 2024-04-25T10:09:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.470+0000 7f391acdb640 1 -- 172.21.15.16:0/3715275173 wait complete. 2024-04-25T10:09:39.487 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:09:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.583+0000 7f8aa7293640 1 Processor -- start 2024-04-25T10:09:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.583+0000 7f8aa7293640 1 -- start start 2024-04-25T10:09:39.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.584+0000 7f8aa7293640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8aa015ae20 0x7f8aa00b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.584+0000 7f8aa7293640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8aa00b4730 0x7f8aa015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.584+0000 7f8aa7293640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aa006da40 con 0x7f8aa00aedc0 2024-04-25T10:09:39.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.584+0000 7f8aa7293640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa015b280 con 0x7f8aa00b4730 2024-04-25T10:09:39.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.584+0000 7f8aa7293640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa015c050 con 0x7f8aa015ae20 2024-04-25T10:09:39.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.584+0000 7f8aa482e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8aa015ae20 0x7f8aa00b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.584+0000 7f8aa5830640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8aa00b4730 0x7f8aa015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.584+0000 7f8aa482e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8aa015ae20 0x7f8aa00b41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:43828/0 (socket says 172.21.15.16:43828) 2024-04-25T10:09:39.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.584+0000 7f8aa502f640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f8aa00aedc0 0x7f8aa006d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:45518/0 (socket says 172.21.15.16:45518) 2024-04-25T10:09:39.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.584+0000 7f8aa5830640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8aa00b4730 0x7f8aa015c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:45408/0 (socket says 172.21.15.16:45408) 2024-04-25T10:09:39.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.584+0000 7f8aa482e640 1 -- 172.21.15.16:0/1580677062 learned_addr learned my addr 172.21.15.16:0/1580677062 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:39.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.585+0000 7f8a97fff640 1 -- 172.21.15.16:0/1580677062 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2531687923 0 0) 0x7f8aa006da40 con 0x7f8aa00aedc0 2024-04-25T10:09:39.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.585+0000 7f8a97fff640 1 -- 172.21.15.16:0/1580677062 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a8c003520 con 0x7f8aa00aedc0 2024-04-25T10:09:39.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.585+0000 7f8aa5830640 1 -- 172.21.15.16:0/1580677062 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8aa015ae20 msgr2=0x7f8aa00b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:39.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.585+0000 7f8aa5830640 1 --2- 172.21.15.16:0/1580677062 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8aa015ae20 0x7f8aa00b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.585+0000 7f8aa5830640 1 -- 172.21.15.16:0/1580677062 >> v1:172.21.15.186:6789/0 conn(0x7f8aa00aedc0 legacy=0x7f8aa006d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.585+0000 7f8aa5830640 1 -- 172.21.15.16:0/1580677062 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa006dd20 con 0x7f8aa00b4730 2024-04-25T10:09:39.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.585+0000 7f8aa482e640 1 --2- 172.21.15.16:0/1580677062 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8aa015ae20 0x7f8aa00b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.585+0000 7f8aa5830640 1 --2- 172.21.15.16:0/1580677062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8aa00b4730 0x7f8aa015c7c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009d90 tx=0x7f8a9802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdc86e396c5b1f93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.586+0000 7f8a97fff640 1 -- 172.21.15.16:0/1580677062 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98031600 con 0x7f8aa00b4730 2024-04-25T10:09:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.586+0000 7f8a97fff640 1 -- 172.21.15.16:0/1580677062 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a980317a0 con 0x7f8aa00b4730 2024-04-25T10:09:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.586+0000 7f8a97fff640 1 -- 172.21.15.16:0/1580677062 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98031b70 con 0x7f8aa00b4730 2024-04-25T10:09:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.586+0000 7f8aa7293640 1 -- 172.21.15.16:0/1580677062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8aa00b4730 msgr2=0x7f8aa015c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.586+0000 7f8aa7293640 1 --2- 172.21.15.16:0/1580677062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8aa00b4730 0x7f8aa015c7c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009d90 tx=0x7f8a9802a5d0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.586+0000 7f8aa7293640 1 -- 172.21.15.16:0/1580677062 shutdown_connections 2024-04-25T10:09:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.586+0000 7f8aa7293640 1 --2- 172.21.15.16:0/1580677062 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8aa015ae20 0x7f8aa00b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.586+0000 7f8aa7293640 1 --2- 172.21.15.16:0/1580677062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8aa00b4730 0x7f8aa015c7c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.586+0000 7f8aa7293640 1 -- 172.21.15.16:0/1580677062 >> 172.21.15.16:0/1580677062 conn(0x7f8aa006aba0 msgr2=0x7f8aa006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.586+0000 7f8aa7293640 1 -- 172.21.15.16:0/1580677062 shutdown_connections 2024-04-25T10:09:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.586+0000 7f8aa7293640 1 -- 172.21.15.16:0/1580677062 wait complete. 2024-04-25T10:09:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.587+0000 7f8aa7293640 1 Processor -- start 2024-04-25T10:09:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.587+0000 7f8aa7293640 1 -- start start 2024-04-25T10:09:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.588+0000 7f8aa7293640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8aa00aedc0 0x7f8aa014c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.588+0000 7f8aa7293640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8aa00b4730 0x7f8aa0153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.588+0000 7f8aa7293640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa015ae20 0x7f8aa0071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.588+0000 7f8aa7293640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa015c2a0 con 0x7f8aa015ae20 2024-04-25T10:09:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.588+0000 7f8aa7293640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa00af1a0 con 0x7f8aa00aedc0 2024-04-25T10:09:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.588+0000 7f8aa7293640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa015cd00 con 0x7f8aa00b4730 2024-04-25T10:09:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.588+0000 7f8aa5830640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa015ae20 0x7f8aa0071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.588+0000 7f8aa5830640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa015ae20 0x7f8aa0071200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40710/0 (socket says 172.21.15.16:40710) 2024-04-25T10:09:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.588+0000 7f8aa5830640 1 -- 172.21.15.16:0/2593296676 learned_addr learned my addr 172.21.15.16:0/2593296676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.588+0000 7f8aa482e640 1 --2- 172.21.15.16:0/2593296676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8aa00b4730 0x7f8aa0153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.588+0000 7f8aa502f640 1 --2- 172.21.15.16:0/2593296676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8aa00aedc0 0x7f8aa014c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.589+0000 7f8aa5830640 1 -- 172.21.15.16:0/2593296676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8aa00b4730 msgr2=0x7f8aa0153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.589+0000 7f8aa5830640 1 --2- 172.21.15.16:0/2593296676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8aa00b4730 0x7f8aa0153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.589+0000 7f8aa5830640 1 -- 172.21.15.16:0/2593296676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8aa00aedc0 msgr2=0x7f8aa014c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.589+0000 7f8aa5830640 1 --2- 172.21.15.16:0/2593296676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8aa00aedc0 0x7f8aa014c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.589+0000 7f8aa5830640 1 -- 172.21.15.16:0/2593296676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa01548f0 con 0x7f8aa015ae20 2024-04-25T10:09:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.589+0000 7f8aa482e640 1 --2- 172.21.15.16:0/2593296676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8aa00b4730 0x7f8aa0153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.589+0000 7f8aa5830640 1 --2- 172.21.15.16:0/2593296676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa015ae20 0x7f8aa0071200 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009eb0 tx=0x7f8a98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.589+0000 7f8a95ffb640 1 -- 172.21.15.16:0/2593296676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98040070 con 0x7f8aa015ae20 2024-04-25T10:09:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.589+0000 7f8a95ffb640 1 -- 172.21.15.16:0/2593296676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9803c830 con 0x7f8aa015ae20 2024-04-25T10:09:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.589+0000 7f8aa7293640 1 -- 172.21.15.16:0/2593296676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa0154050 con 0x7f8aa015ae20 2024-04-25T10:09:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.589+0000 7f8a95ffb640 1 -- 172.21.15.16:0/2593296676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9803cbe0 con 0x7f8aa015ae20 2024-04-25T10:09:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.589+0000 7f8aa7293640 1 -- 172.21.15.16:0/2593296676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa00a97d0 con 0x7f8aa015ae20 2024-04-25T10:09:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.590+0000 7f8aa7293640 1 -- 172.21.15.16:0/2593296676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa00a99d0 con 0x7f8aa015ae20 2024-04-25T10:09:39.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.591+0000 7f8a95ffb640 1 -- 172.21.15.16:0/2593296676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a98039040 con 0x7f8aa015ae20 2024-04-25T10:09:39.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.591+0000 7f8a95ffb640 1 --2- 172.21.15.16:0/2593296676 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8a8406b500 0x7f8a8406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.591+0000 7f8a95ffb640 1 -- 172.21.15.16:0/2593296676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8a980ab770 con 0x7f8aa015ae20 2024-04-25T10:09:39.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.591+0000 7f8aa502f640 1 --2- 172.21.15.16:0/2593296676 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8a8406b500 0x7f8a8406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.592+0000 7f8aa502f640 1 --2- 172.21.15.16:0/2593296676 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8a8406b500 0x7f8a8406d9c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c000f60 tx=0x7f8a9c007cf0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:39.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.595+0000 7f8a95ffb640 1 -- 172.21.15.16:0/2593296676 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8aa00a99d0 con 0x7f8aa015ae20 2024-04-25T10:09:39.701 INFO:tasks.ceph.mds.a.smithi016.stderr:2024-04-25T10:09:39.695+0000 7f81c84b3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:09:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.789+0000 7f8aa7293640 1 -- 172.21.15.16:0/2593296676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8aa015a2f0 con 0x7f8aa015ae20 2024-04-25T10:09:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.789+0000 7f8a95ffb640 1 -- 172.21.15.16:0/2593296676 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v73) v1 ==== 80+0+124 (secure 0 0 0) 0x7f8a98079fc0 con 0x7f8aa015ae20 2024-04-25T10:09:39.798 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.186:6837/3292077067 2024-04-26T10:09:31.374252+0000 2024-04-25T10:09:39.798 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.186:6836/3292077067 2024-04-26T10:09:31.374252+0000 2024-04-25T10:09:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:listed 2 entries 2024-04-25T10:09:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.791+0000 7f8aa7293640 1 -- 172.21.15.16:0/2593296676 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8a8406b500 msgr2=0x7f8a8406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.791+0000 7f8aa7293640 1 --2- 172.21.15.16:0/2593296676 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8a8406b500 0x7f8a8406d9c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c000f60 tx=0x7f8a9c007cf0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.791+0000 7f8aa7293640 1 -- 172.21.15.16:0/2593296676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa015ae20 msgr2=0x7f8aa0071200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.791+0000 7f8aa7293640 1 --2- 172.21.15.16:0/2593296676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa015ae20 0x7f8aa0071200 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009eb0 tx=0x7f8a98004a40 comp rx=0 tx=0).stop 2024-04-25T10:09:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.791+0000 7f8aa7293640 1 -- 172.21.15.16:0/2593296676 shutdown_connections 2024-04-25T10:09:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.791+0000 7f8aa7293640 1 --2- 172.21.15.16:0/2593296676 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8a8406b500 0x7f8a8406d9c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.791+0000 7f8aa7293640 1 --2- 172.21.15.16:0/2593296676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa015ae20 0x7f8aa0071200 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.791+0000 7f8aa7293640 1 --2- 172.21.15.16:0/2593296676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8aa00b4730 0x7f8aa0153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.791+0000 7f8aa7293640 1 --2- 172.21.15.16:0/2593296676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8aa00aedc0 0x7f8aa014c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.791+0000 7f8aa7293640 1 -- 172.21.15.16:0/2593296676 >> 172.21.15.16:0/2593296676 conn(0x7f8aa006aba0 msgr2=0x7f8aa00b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:39.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.791+0000 7f8aa7293640 1 -- 172.21.15.16:0/2593296676 shutdown_connections 2024-04-25T10:09:39.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:39.791+0000 7f8aa7293640 1 -- 172.21.15.16:0/2593296676 wait complete. 2024-04-25T10:09:39.808 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:09:39.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:39.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:39.836 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:09:39.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:39.836 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:09:39.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:39.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-04-25T10:09:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.031+0000 7f11e5909640 1 Processor -- start 2024-04-25T10:09:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.031+0000 7f11e5909640 1 -- start start 2024-04-25T10:09:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.032+0000 7f11e5909640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f11e00b9720 0x7f11e00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.032+0000 7f11e5909640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f11e00cece0 0x7f11e00d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.032+0000 7f11e5909640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e00cf0f0 con 0x7f11e0161090 2024-04-25T10:09:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.032+0000 7f11e5909640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e00d21a0 con 0x7f11e00cece0 2024-04-25T10:09:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.032+0000 7f11e5909640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e0161f80 con 0x7f11e00b9720 2024-04-25T10:09:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.032+0000 7f11df7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f11e00cece0 0x7f11e00d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.032+0000 7f11deffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f11e0161090 0x7f11e0161730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38920/0 (socket says 172.21.15.16:38920) 2024-04-25T10:09:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.032+0000 7f11deffd640 1 -- 172.21.15.16:0/3678629505 learned_addr learned my addr 172.21.15.16:0/3678629505 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.032+0000 7f11df7fe640 1 -- 172.21.15.16:0/3678629505 >> v2:172.21.15.186:3300/0 conn(0x7f11e00b9720 msgr2=0x7f11e00ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.032+0000 7f11df7fe640 1 --2- 172.21.15.16:0/3678629505 >> v2:172.21.15.186:3300/0 conn(0x7f11e00b9720 0x7f11e00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.032+0000 7f11df7fe640 1 -- 172.21.15.16:0/3678629505 >> v1:172.21.15.16:6789/0 conn(0x7f11e0161090 legacy=0x7f11e0161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.032+0000 7f11df7fe640 1 -- 172.21.15.16:0/3678629505 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e0162260 con 0x7f11e00cece0 2024-04-25T10:09:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.033+0000 7f11df7fe640 1 --2- 172.21.15.16:0/3678629505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11e00cece0 0x7f11e00d2630 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f11e008cf10 tx=0x7f11d002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a9d86782d9411b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.033+0000 7f11ddffb640 1 -- 172.21.15.16:0/3678629505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d0007740 con 0x7f11e00cece0 2024-04-25T10:09:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.033+0000 7f11ddffb640 1 -- 172.21.15.16:0/3678629505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d00078e0 con 0x7f11e00cece0 2024-04-25T10:09:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.033+0000 7f11ddffb640 1 -- 172.21.15.16:0/3678629505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d0007cb0 con 0x7f11e00cece0 2024-04-25T10:09:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.034+0000 7f11e5909640 1 -- 172.21.15.16:0/3678629505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11e00cece0 msgr2=0x7f11e00d2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.034+0000 7f11e5909640 1 --2- 172.21.15.16:0/3678629505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11e00cece0 0x7f11e00d2630 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f11e008cf10 tx=0x7f11d002a5f0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.034+0000 7f11e5909640 1 -- 172.21.15.16:0/3678629505 shutdown_connections 2024-04-25T10:09:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.034+0000 7f11e5909640 1 --2- 172.21.15.16:0/3678629505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11e00cece0 0x7f11e00d2630 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.034+0000 7f11e5909640 1 --2- 172.21.15.16:0/3678629505 >> v2:172.21.15.186:3300/0 conn(0x7f11e00b9720 0x7f11e00ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.034+0000 7f11e5909640 1 -- 172.21.15.16:0/3678629505 >> 172.21.15.16:0/3678629505 conn(0x7f11e00b55e0 msgr2=0x7f11e00b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.034+0000 7f11e5909640 1 -- 172.21.15.16:0/3678629505 shutdown_connections 2024-04-25T10:09:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.034+0000 7f11e5909640 1 -- 172.21.15.16:0/3678629505 wait complete. 2024-04-25T10:09:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.035+0000 7f11e5909640 1 Processor -- start 2024-04-25T10:09:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.035+0000 7f11e5909640 1 -- start start 2024-04-25T10:09:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.036+0000 7f11e5909640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11e00b9720 0x7f11e014adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.036+0000 7f11e5909640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11e00cece0 0x7f11e01e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.036+0000 7f11e5909640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11e0161090 0x7f11e01e9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.036+0000 7f11e5909640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e0161470 con 0x7f11e00cece0 2024-04-25T10:09:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.036+0000 7f11e5909640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e00d1e30 con 0x7f11e00b9720 2024-04-25T10:09:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.036+0000 7f11e5909640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e00d2b70 con 0x7f11e0161090 2024-04-25T10:09:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.036+0000 7f11de7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11e00cece0 0x7f11e01e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.036+0000 7f11de7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11e00cece0 0x7f11e01e7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40720/0 (socket says 172.21.15.16:40720) 2024-04-25T10:09:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.036+0000 7f11de7fc640 1 -- 172.21.15.16:0/3392181590 learned_addr learned my addr 172.21.15.16:0/3392181590 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.036+0000 7f11df7fe640 1 --2- 172.21.15.16:0/3392181590 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11e0161090 0x7f11e01e9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.036+0000 7f11deffd640 1 --2- 172.21.15.16:0/3392181590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11e00b9720 0x7f11e014adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.036+0000 7f11de7fc640 1 -- 172.21.15.16:0/3392181590 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11e0161090 msgr2=0x7f11e01e9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.036+0000 7f11de7fc640 1 --2- 172.21.15.16:0/3392181590 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11e0161090 0x7f11e01e9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.036+0000 7f11de7fc640 1 -- 172.21.15.16:0/3392181590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11e00b9720 msgr2=0x7f11e014adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.036+0000 7f11de7fc640 1 --2- 172.21.15.16:0/3392181590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11e00b9720 0x7f11e014adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.036+0000 7f11de7fc640 1 -- 172.21.15.16:0/3392181590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e01e8a00 con 0x7f11e00cece0 2024-04-25T10:09:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.036+0000 7f11de7fc640 1 --2- 172.21.15.16:0/3392181590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11e00cece0 0x7f11e01e7f40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f11d400b4a0 tx=0x7f11d400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.037+0000 7f11c7fff640 1 -- 172.21.15.16:0/3392181590 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4014070 con 0x7f11e00cece0 2024-04-25T10:09:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.037+0000 7f11c7fff640 1 -- 172.21.15.16:0/3392181590 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d4010610 con 0x7f11e00cece0 2024-04-25T10:09:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.037+0000 7f11e5909640 1 -- 172.21.15.16:0/3392181590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e01e8ce0 con 0x7f11e00cece0 2024-04-25T10:09:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.037+0000 7f11c7fff640 1 -- 172.21.15.16:0/3392181590 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4010a00 con 0x7f11e00cece0 2024-04-25T10:09:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.037+0000 7f11e5909640 1 -- 172.21.15.16:0/3392181590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e00c7ab0 con 0x7f11e00cece0 2024-04-25T10:09:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.038+0000 7f11c7fff640 1 -- 172.21.15.16:0/3392181590 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11d4017070 con 0x7f11e00cece0 2024-04-25T10:09:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.038+0000 7f11c7fff640 1 --2- 172.21.15.16:0/3392181590 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f11cc06b4d0 0x7f11cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.039+0000 7f11c7fff640 1 -- 172.21.15.16:0/3392181590 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f11d4089cb0 con 0x7f11e00cece0 2024-04-25T10:09:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.039+0000 7f11deffd640 1 --2- 172.21.15.16:0/3392181590 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f11cc06b4d0 0x7f11cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.039+0000 7f11deffd640 1 --2- 172.21.15.16:0/3392181590 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f11cc06b4d0 0x7f11cc06d990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f11d8000f60 tx=0x7f11d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.039+0000 7f11e8612740 1 Processor -- start 2024-04-25T10:09:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.039+0000 7f11e8612740 1 -- start start 2024-04-25T10:09:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.040+0000 7f11e8612740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a4ca080 0x56361a4ca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.040+0000 7f11e8612740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4caaa0 0x56361a49a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.040+0000 7f11e8612740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a49a740 0x56361a4c2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.040+0000 7f11e8612740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361a4030a0 con 0x56361a4caaa0 2024-04-25T10:09:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.040+0000 7f11e8612740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361a3abd40 con 0x56361a4ca080 2024-04-25T10:09:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.040+0000 7f11e8612740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56361a3ff140 con 0x56361a49a740 2024-04-25T10:09:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.040+0000 7f11de7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4caaa0 0x56361a49a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.040+0000 7f11de7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4caaa0 0x56361a49a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40726/0 (socket says 172.21.15.16:40726) 2024-04-25T10:09:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.040+0000 7f11de7fc640 1 -- 172.21.15.16:0/2246114525 learned_addr learned my addr 172.21.15.16:0/2246114525 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.040+0000 7f11deffd640 1 --2- 172.21.15.16:0/2246114525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a4ca080 0x56361a4ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.040+0000 7f11df7fe640 1 --2- 172.21.15.16:0/2246114525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a49a740 0x56361a4c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.041+0000 7f11de7fc640 1 -- 172.21.15.16:0/2246114525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a49a740 msgr2=0x56361a4c2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.041+0000 7f11de7fc640 1 --2- 172.21.15.16:0/2246114525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a49a740 0x56361a4c2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.041+0000 7f11de7fc640 1 -- 172.21.15.16:0/2246114525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a4ca080 msgr2=0x56361a4ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.041+0000 7f11de7fc640 1 --2- 172.21.15.16:0/2246114525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a4ca080 0x56361a4ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.041+0000 7f11de7fc640 1 -- 172.21.15.16:0/2246114525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56361a1e3a20 con 0x56361a4caaa0 2024-04-25T10:09:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.041+0000 7f11df7fe640 1 --2- 172.21.15.16:0/2246114525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a49a740 0x56361a4c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.041+0000 7f11de7fc640 1 --2- 172.21.15.16:0/2246114525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4caaa0 0x56361a49a200 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f11d40880b0 tx=0x7f11d400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd310ebacabbc8e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.041+0000 7f11c57fa640 1 -- 172.21.15.16:0/2246114525 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d401e040 con 0x56361a4caaa0 2024-04-25T10:09:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.041+0000 7f11c57fa640 1 -- 172.21.15.16:0/2246114525 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d4054a30 con 0x56361a4caaa0 2024-04-25T10:09:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.041+0000 7f11c57fa640 1 -- 172.21.15.16:0/2246114525 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4010ba0 con 0x56361a4caaa0 2024-04-25T10:09:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.042+0000 7f11e8612740 1 -- 172.21.15.16:0/2246114525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4caaa0 msgr2=0x56361a49a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.042+0000 7f11e8612740 1 --2- 172.21.15.16:0/2246114525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4caaa0 0x56361a49a200 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f11d40880b0 tx=0x7f11d400bf90 comp rx=0 tx=0).stop 2024-04-25T10:09:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.042+0000 7f11e8612740 1 -- 172.21.15.16:0/2246114525 shutdown_connections 2024-04-25T10:09:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.042+0000 7f11e8612740 1 --2- 172.21.15.16:0/2246114525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a49a740 0x56361a4c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.042+0000 7f11e8612740 1 --2- 172.21.15.16:0/2246114525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4caaa0 0x56361a49a200 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.042+0000 7f11e8612740 1 --2- 172.21.15.16:0/2246114525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a4ca080 0x56361a4ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.042+0000 7f11e8612740 1 -- 172.21.15.16:0/2246114525 >> 172.21.15.16:0/2246114525 conn(0x56361a49c470 msgr2=0x56361a4e4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.042+0000 7f11e8612740 1 -- 172.21.15.16:0/2246114525 shutdown_connections 2024-04-25T10:09:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.042+0000 7f11e8612740 1 -- 172.21.15.16:0/2246114525 wait complete. 2024-04-25T10:09:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.042+0000 7f11e8612740 1 -- start start 2024-04-25T10:09:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.042+0000 7f11e8612740 1 Processor -- start 2024-04-25T10:09:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.043+0000 7f11e8612740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a4ca080 0x56361a4bbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.043+0000 7f11e8612740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a4caaa0 0x56361a49f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.043+0000 7f11e8612740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a49f7f0 0x56361a4bd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.043+0000 7f11e8612740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361a4c67d0 con 0x56361a49f7f0 2024-04-25T10:09:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.043+0000 7f11e8612740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361a3a13d0 con 0x56361a4caaa0 2024-04-25T10:09:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.043+0000 7f11e8612740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56361a4c6a70 con 0x56361a4ca080 2024-04-25T10:09:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.043+0000 7f11e8612740 10 client.0 _ll_register_callbacks cb 0x56361a116e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.043+0000 7f11df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a49f7f0 0x56361a4bd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.043+0000 7f11df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a49f7f0 0x56361a4bd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40740/0 (socket says 172.21.15.16:40740) 2024-04-25T10:09:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.043+0000 7f11df7fe640 1 -- 172.21.15.16:0/606245692 learned_addr learned my addr 172.21.15.16:0/606245692 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.043+0000 7f11de7fc640 1 --2- 172.21.15.16:0/606245692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a4caaa0 0x56361a49f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.043+0000 7f11deffd640 1 --2- 172.21.15.16:0/606245692 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a4ca080 0x56361a4bbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.044+0000 7f11de7fc640 1 -- 172.21.15.16:0/606245692 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a4ca080 msgr2=0x56361a4bbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.044+0000 7f11de7fc640 1 --2- 172.21.15.16:0/606245692 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a4ca080 0x56361a4bbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.044+0000 7f11de7fc640 1 -- 172.21.15.16:0/606245692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a49f7f0 msgr2=0x56361a4bd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.044+0000 7f11de7fc640 1 --2- 172.21.15.16:0/606245692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a49f7f0 0x56361a4bd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.044+0000 7f11de7fc640 1 -- 172.21.15.16:0/606245692 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56361a0d7310 con 0x56361a4caaa0 2024-04-25T10:09:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.044+0000 7f11df7fe640 1 --2- 172.21.15.16:0/606245692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a49f7f0 0x56361a4bd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.044+0000 7f11de7fc640 1 --2- 172.21.15.16:0/606245692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a4caaa0 0x56361a49f2b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f11d40580c0 tx=0x7f11d4088040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f945eaea556fb91d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.044+0000 7f11be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-25T10:09:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.044+0000 7f11be7fc640 1 -- 172.21.15.16:0/606245692 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4074020 con 0x56361a4caaa0 2024-04-25T10:09:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.044+0000 7f11e8612740 1 -- 172.21.15.16:0/606245692 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11d4054a30 con 0x56361a4caaa0 2024-04-25T10:09:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.044+0000 7f11be7fc640 1 -- 172.21.15.16:0/606245692 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f11d4010da0 con 0x56361a4caaa0 2024-04-25T10:09:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.044+0000 7f11be7fc640 1 -- 172.21.15.16:0/606245692 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11d407f050 con 0x56361a4caaa0 2024-04-25T10:09:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.044+0000 7f11e8612740 10 client.6271 fetch_fsmap learned FSMap version 42 2024-04-25T10:09:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.044+0000 7f11e8612740 1 -- 172.21.15.16:0/606245692 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x56361a2441b0 con 0x56361a4caaa0 2024-04-25T10:09:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.044+0000 7f11be7fc640 1 -- 172.21.15.16:0/606245692 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d4054a30 con 0x56361a4caaa0 2024-04-25T10:09:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.044+0000 7f11be7fc640 1 -- 172.21.15.16:0/606245692 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f11d4057810 con 0x56361a4caaa0 2024-04-25T10:09:40.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.044+0000 7f11e8612740 10 client.6271 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T10:09:40.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.044+0000 7f11e8612740 10 client.6271 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:09:40.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.045+0000 7f11e8612740 20 client.6271 populate_metadata read hostname 'smithi016' 2024-04-25T10:09:40.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.045+0000 7f11e8612740 1 --2- 172.21.15.16:0/606245692 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x56361a4bdee0 0x56361a4f8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.045+0000 7f11e8612740 4 client.6271 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:40.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.045+0000 7f11e8612740 1 -- 172.21.15.16:0/606245692 --> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56361a4105e0 con 0x56361a4bdee0 2024-04-25T10:09:40.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.045+0000 7f11deffd640 1 --2- 172.21.15.16:0/606245692 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x56361a4bdee0 0x56361a4f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.045+0000 7f11deffd640 1 --2- 172.21.15.16:0/606245692 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x56361a4bdee0 0x56361a4f8680 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d482ffb976bd4298 server_cookie=943e9b86e3edb2 in_seq=0 out_seq=0 2024-04-25T10:09:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.045+0000 7f11be7fc640 10 client.6271 ms_handle_connect on v2:172.21.15.16:6834/1735811292 2024-04-25T10:09:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.045+0000 7f11be7fc640 1 -- 172.21.15.16:0/606245692 <== mds.0 v2:172.21.15.16:6834/1735811292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56361a4105e0 con 0x56361a4bdee0 2024-04-25T10:09:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.045+0000 7f11be7fc640 10 client.6271 handle_command_reply: tid=0 2024-04-25T10:09:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.045+0000 7f11e8612740 1 client.6271 shutdown 2024-04-25T10:09:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.046+0000 7f11e8612740 1 -- 172.21.15.16:0/606245692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a4caaa0 msgr2=0x56361a49f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.046+0000 7f11e8612740 1 --2- 172.21.15.16:0/606245692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a4caaa0 0x56361a49f2b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f11d40580c0 tx=0x7f11d4088040 comp rx=0 tx=0).stop 2024-04-25T10:09:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.046+0000 7f11e8612740 1 -- 172.21.15.16:0/606245692 shutdown_connections 2024-04-25T10:09:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.046+0000 7f11e8612740 1 --2- 172.21.15.16:0/606245692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a49f7f0 0x56361a4bd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.046+0000 7f11e8612740 1 --2- 172.21.15.16:0/606245692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a4caaa0 0x56361a49f2b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.046+0000 7f11e8612740 1 --2- 172.21.15.16:0/606245692 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x56361a4bdee0 0x56361a4f8680 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.046+0000 7f11e8612740 1 --2- 172.21.15.16:0/606245692 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a4ca080 0x56361a4bbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.046+0000 7f11be7fc640 0 client.6271 ms_handle_reset on v2:172.21.15.16:6834/1735811292 2024-04-25T10:09:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.046+0000 7f11e8612740 1 -- 172.21.15.16:0/606245692 >> 172.21.15.16:0/606245692 conn(0x56361a49a740 msgr2=0x56361a4e4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.046+0000 7f11e8612740 1 -- 172.21.15.16:0/606245692 shutdown_connections 2024-04-25T10:09:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.046+0000 7f11e8612740 1 -- 172.21.15.16:0/606245692 wait complete. 2024-04-25T10:09:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.046+0000 7f11e8612740 20 client.6271 trim_cache size 0 max 16384 2024-04-25T10:09:40.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.061+0000 7f11e8612740 1 Processor -- start 2024-04-25T10:09:40.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.061+0000 7f11e8612740 1 -- start start 2024-04-25T10:09:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.062+0000 7f11e8612740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a49f7f0 0x56361a4c8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.062+0000 7f11e8612740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4ca080 0x56361a4f4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.062+0000 7f11e8612740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a4caaa0 0x56361a4e4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.062+0000 7f11e8612740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e01fff50 con 0x56361a4ca080 2024-04-25T10:09:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.062+0000 7f11e8612740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e01ff150 con 0x56361a49f7f0 2024-04-25T10:09:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.062+0000 7f11e8612740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e01f8450 con 0x56361a4caaa0 2024-04-25T10:09:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.062+0000 7f11de7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4ca080 0x56361a4f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.062+0000 7f11de7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4ca080 0x56361a4f4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40754/0 (socket says 172.21.15.16:40754) 2024-04-25T10:09:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.062+0000 7f11de7fc640 1 -- 172.21.15.16:0/2504384503 learned_addr learned my addr 172.21.15.16:0/2504384503 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.062+0000 7f11de7fc640 1 -- 172.21.15.16:0/2504384503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a4caaa0 msgr2=0x56361a4e4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.062+0000 7f11de7fc640 1 --2- 172.21.15.16:0/2504384503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a4caaa0 0x56361a4e4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.062+0000 7f11de7fc640 1 -- 172.21.15.16:0/2504384503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a49f7f0 msgr2=0x56361a4c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.062+0000 7f11deffd640 1 --2- 172.21.15.16:0/2504384503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a49f7f0 0x56361a4c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.062+0000 7f11de7fc640 1 --2- 172.21.15.16:0/2504384503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a49f7f0 0x56361a4c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.062+0000 7f11de7fc640 1 -- 172.21.15.16:0/2504384503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56361a0b0490 con 0x56361a4ca080 2024-04-25T10:09:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.063+0000 7f11de7fc640 1 --2- 172.21.15.16:0/2504384503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4ca080 0x56361a4f4350 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f11d4000c00 tx=0x7f11d401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e4903a37e0a7ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.063+0000 7f11c4ff9640 1 -- 172.21.15.16:0/2504384503 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4074070 con 0x56361a4ca080 2024-04-25T10:09:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.063+0000 7f11c4ff9640 1 -- 172.21.15.16:0/2504384503 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d4054a30 con 0x56361a4ca080 2024-04-25T10:09:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.063+0000 7f11c4ff9640 1 -- 172.21.15.16:0/2504384503 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d401e070 con 0x56361a4ca080 2024-04-25T10:09:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.063+0000 7f11e8612740 1 -- 172.21.15.16:0/2504384503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4ca080 msgr2=0x56361a4f4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.063+0000 7f11e8612740 1 --2- 172.21.15.16:0/2504384503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4ca080 0x56361a4f4350 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f11d4000c00 tx=0x7f11d401e000 comp rx=0 tx=0).stop 2024-04-25T10:09:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.063+0000 7f11e8612740 1 -- 172.21.15.16:0/2504384503 shutdown_connections 2024-04-25T10:09:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.063+0000 7f11e8612740 1 --2- 172.21.15.16:0/2504384503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a4caaa0 0x56361a4e4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.063+0000 7f11e8612740 1 --2- 172.21.15.16:0/2504384503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4ca080 0x56361a4f4350 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.063+0000 7f11e8612740 1 --2- 172.21.15.16:0/2504384503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a49f7f0 0x56361a4c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.063+0000 7f11e8612740 1 -- 172.21.15.16:0/2504384503 >> 172.21.15.16:0/2504384503 conn(0x56361a49a740 msgr2=0x56361a49ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.063+0000 7f11e8612740 1 -- 172.21.15.16:0/2504384503 shutdown_connections 2024-04-25T10:09:40.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.063+0000 7f11e8612740 1 -- 172.21.15.16:0/2504384503 wait complete. 2024-04-25T10:09:40.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.064+0000 7f11e8612740 1 -- start start 2024-04-25T10:09:40.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.064+0000 7f11e8612740 1 Processor -- start 2024-04-25T10:09:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11e8612740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a49f7f0 0x56361a4fe640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11e8612740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4ca080 0x56361a4feb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11e8612740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a4caaa0 0x56361a4b9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11e8612740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361a4c96e0 con 0x56361a4ca080 2024-04-25T10:09:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11e8612740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e01feb80 con 0x56361a4caaa0 2024-04-25T10:09:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11e8612740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56361a4c63b0 con 0x56361a49f7f0 2024-04-25T10:09:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11e8612740 10 client.0 _ll_register_callbacks cb 0x7f11d4058210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11de7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4ca080 0x56361a4feb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11de7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4ca080 0x56361a4feb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40758/0 (socket says 172.21.15.16:40758) 2024-04-25T10:09:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11de7fc640 1 -- 172.21.15.16:0/1671499305 learned_addr learned my addr 172.21.15.16:0/1671499305 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11deffd640 1 --2- 172.21.15.16:0/1671499305 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a49f7f0 0x56361a4fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11df7fe640 1 --2- 172.21.15.16:0/1671499305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a4caaa0 0x56361a4b9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11de7fc640 1 -- 172.21.15.16:0/1671499305 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a49f7f0 msgr2=0x56361a4fe640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11de7fc640 1 --2- 172.21.15.16:0/1671499305 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a49f7f0 0x56361a4fe640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11de7fc640 1 -- 172.21.15.16:0/1671499305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a4caaa0 msgr2=0x56361a4b9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11de7fc640 1 --2- 172.21.15.16:0/1671499305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a4caaa0 0x56361a4b9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11de7fc640 1 -- 172.21.15.16:0/1671499305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56361a507f30 con 0x56361a4ca080 2024-04-25T10:09:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11df7fe640 1 --2- 172.21.15.16:0/1671499305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a4caaa0 0x56361a4b9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11deffd640 1 --2- 172.21.15.16:0/1671499305 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a49f7f0 0x56361a4fe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11de7fc640 1 --2- 172.21.15.16:0/1671499305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4ca080 0x56361a4feb80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f11d40584d0 tx=0x7f11d40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592e66b485fa34c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:09:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11be7fc640 1 -- 172.21.15.16:0/1671499305 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4065060 con 0x56361a4ca080 2024-04-25T10:09:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11be7fc640 1 -- 172.21.15.16:0/1671499305 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f11d4054110 con 0x56361a4ca080 2024-04-25T10:09:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.065+0000 7f11e8612740 1 -- 172.21.15.16:0/1671499305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11d4054a30 con 0x56361a4ca080 2024-04-25T10:09:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.066+0000 7f11be7fc640 1 -- 172.21.15.16:0/1671499305 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11d4074070 con 0x56361a4ca080 2024-04-25T10:09:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.066+0000 7f11e8612740 10 client.6966 fetch_fsmap learned FSMap version 42 2024-04-25T10:09:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.066+0000 7f11e8612740 1 -- 172.21.15.16:0/1671499305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x56361a507f30 con 0x56361a4ca080 2024-04-25T10:09:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.066+0000 7f11be7fc640 1 -- 172.21.15.16:0/1671499305 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d4054a30 con 0x56361a4ca080 2024-04-25T10:09:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.066+0000 7f11be7fc640 1 -- 172.21.15.16:0/1671499305 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f11d4081ad0 con 0x56361a4ca080 2024-04-25T10:09:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.066+0000 7f11e8612740 10 client.6966 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T10:09:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.066+0000 7f11e8612740 10 client.6966 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:09:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.066+0000 7f11e8612740 20 client.6966 populate_metadata read hostname 'smithi016' 2024-04-25T10:09:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.066+0000 7f11e8612740 1 --2- 172.21.15.16:0/1671499305 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x56361a4ff0c0 0x56361a4ba470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.066+0000 7f11e8612740 4 client.6966 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-04-25T10:09:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.066+0000 7f11e8612740 1 -- 172.21.15.16:0/1671499305 --> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) v1 -- 0x56361a4bf300 con 0x56361a4ff0c0 2024-04-25T10:09:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.066+0000 7f11deffd640 1 --2- 172.21.15.16:0/1671499305 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x56361a4ff0c0 0x56361a4ba470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.066+0000 7f11be7fc640 10 client.6966 ms_handle_connect on v2:172.21.15.16:6834/1735811292 2024-04-25T10:09:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.066+0000 7f11deffd640 1 --2- 172.21.15.16:0/1671499305 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x56361a4ff0c0 0x56361a4ba470 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89f788c18c8c6f7 server_cookie=f0344a08310346d6 in_seq=0 out_seq=0 2024-04-25T10:09:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.067+0000 7f11be7fc640 1 -- 172.21.15.16:0/1671499305 <== mds.0 v2:172.21.15.16:6834/1735811292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56361a4bf300 con 0x56361a4ff0c0 2024-04-25T10:09:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.067+0000 7f11be7fc640 10 client.6966 handle_command_reply: tid=0 2024-04-25T10:09:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.067+0000 7f11e8612740 1 client.6966 shutdown 2024-04-25T10:09:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.067+0000 7f11e8612740 1 -- 172.21.15.16:0/1671499305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4ca080 msgr2=0x56361a4feb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.067+0000 7f11e8612740 1 --2- 172.21.15.16:0/1671499305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4ca080 0x56361a4feb80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f11d40584d0 tx=0x7f11d40047d0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.067+0000 7f11e8612740 1 -- 172.21.15.16:0/1671499305 shutdown_connections 2024-04-25T10:09:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.067+0000 7f11e8612740 1 --2- 172.21.15.16:0/1671499305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361a4caaa0 0x56361a4b9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.067+0000 7f11e8612740 1 --2- 172.21.15.16:0/1671499305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361a4ca080 0x56361a4feb80 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.067+0000 7f11e8612740 1 --2- 172.21.15.16:0/1671499305 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x56361a4ff0c0 0x56361a4ba470 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.067+0000 7f11e8612740 1 --2- 172.21.15.16:0/1671499305 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56361a49f7f0 0x56361a4fe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.067+0000 7f11e8612740 1 -- 172.21.15.16:0/1671499305 >> 172.21.15.16:0/1671499305 conn(0x56361a49a740 msgr2=0x56361a4e4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.067+0000 7f11be7fc640 0 client.6966 ms_handle_reset on v2:172.21.15.16:6834/1735811292 2024-04-25T10:09:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.067+0000 7f11e8612740 1 -- 172.21.15.16:0/1671499305 shutdown_connections 2024-04-25T10:09:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.067+0000 7f11e8612740 1 -- 172.21.15.16:0/1671499305 wait complete. 2024-04-25T10:09:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.067+0000 7f11e8612740 20 client.6966 trim_cache size 0 max 16384 2024-04-25T10:09:40.081 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:09:40.081 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-04-25T10:09:40.082 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "c713549c-8958-45b1-9500-5d76f27cafab", 2024-04-25T10:09:40.082 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-04-25T10:09:40.082 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:09:40.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.068+0000 7f11e5909640 1 -- 172.21.15.16:0/3392181590 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f11cc06b4d0 msgr2=0x7f11cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.068+0000 7f11e5909640 1 --2- 172.21.15.16:0/3392181590 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f11cc06b4d0 0x7f11cc06d990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f11d8000f60 tx=0x7f11d8009040 comp rx=0 tx=0).stop 2024-04-25T10:09:40.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.068+0000 7f11e5909640 1 -- 172.21.15.16:0/3392181590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11e00cece0 msgr2=0x7f11e01e7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.068+0000 7f11e5909640 1 --2- 172.21.15.16:0/3392181590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11e00cece0 0x7f11e01e7f40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f11d400b4a0 tx=0x7f11d400b960 comp rx=0 tx=0).stop 2024-04-25T10:09:40.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.068+0000 7f11e5909640 1 -- 172.21.15.16:0/3392181590 shutdown_connections 2024-04-25T10:09:40.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.068+0000 7f11e5909640 1 --2- 172.21.15.16:0/3392181590 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f11cc06b4d0 0x7f11cc06d990 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.068+0000 7f11e5909640 1 --2- 172.21.15.16:0/3392181590 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11e0161090 0x7f11e01e9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.068+0000 7f11e5909640 1 --2- 172.21.15.16:0/3392181590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11e00cece0 0x7f11e01e7f40 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.068+0000 7f11e5909640 1 --2- 172.21.15.16:0/3392181590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11e00b9720 0x7f11e014adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.068+0000 7f11e5909640 1 -- 172.21.15.16:0/3392181590 >> 172.21.15.16:0/3392181590 conn(0x7f11e00b55e0 msgr2=0x7f11e00b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.068+0000 7f11e5909640 1 -- 172.21.15.16:0/3392181590 shutdown_connections 2024-04-25T10:09:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.068+0000 7f11e5909640 1 -- 172.21.15.16:0/3392181590 wait complete. 2024-04-25T10:09:40.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:40.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:09:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.181+0000 7fc0a2fe0640 1 Processor -- start 2024-04-25T10:09:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.181+0000 7fc0a2fe0640 1 -- start start 2024-04-25T10:09:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.182+0000 7fc0a2fe0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc09c0b5550 0x7fc09c0b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.182+0000 7fc0a2fe0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc09c161500 con 0x7fc09c0d2560 2024-04-25T10:09:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.182+0000 7fc0a2fe0640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc09c161eb0 con 0x7fc09c161120 2024-04-25T10:09:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.182+0000 7fc0a2fe0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c0b5ef0 con 0x7fc09c0b5550 2024-04-25T10:09:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.182+0000 7fc0a157d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc09c0d2560 0x7fc09c0d29f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38924/0 (socket says 172.21.15.16:38924) 2024-04-25T10:09:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.182+0000 7fc0a157d640 1 -- 172.21.15.16:0/1016366499 learned_addr learned my addr 172.21.15.16:0/1016366499 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:09:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.182+0000 7fc09bfff640 1 --2- 172.21.15.16:0/1016366499 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc09c0b5550 0x7fc09c0b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.182+0000 7fc09b7fe640 1 -- 172.21.15.16:0/1016366499 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3208327864 0 0) 0x7fc09c161500 con 0x7fc09c0d2560 2024-04-25T10:09:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.182+0000 7fc09b7fe640 1 -- 172.21.15.16:0/1016366499 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc084003520 con 0x7fc09c0d2560 2024-04-25T10:09:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.182+0000 7fc09bfff640 1 -- 172.21.15.16:0/1016366499 >> v1:172.21.15.186:6789/0 conn(0x7fc09c161120 legacy=0x7fc09c1617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.182+0000 7fc09bfff640 1 -- 172.21.15.16:0/1016366499 >> v1:172.21.15.16:6789/0 conn(0x7fc09c0d2560 legacy=0x7fc09c0d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.182+0000 7fc09b7fe640 1 -- 172.21.15.16:0/1016366499 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3949613271 0 0) 0x7fc09c161eb0 con 0x7fc09c161120 2024-04-25T10:09:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.183+0000 7fc09bfff640 1 -- 172.21.15.16:0/1016366499 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09c162180 con 0x7fc09c0b5550 2024-04-25T10:09:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.183+0000 7fc09bfff640 1 --2- 172.21.15.16:0/1016366499 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc09c0b5550 0x7fc09c0b59b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc0900097d0 tx=0x7fc09002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8994dd69f257974f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.183+0000 7fc09b7fe640 1 -- 172.21.15.16:0/1016366499 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090037070 con 0x7fc09c0b5550 2024-04-25T10:09:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.184+0000 7fc09b7fe640 1 -- 172.21.15.16:0/1016366499 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc090032440 con 0x7fc09c0b5550 2024-04-25T10:09:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.184+0000 7fc09b7fe640 1 -- 172.21.15.16:0/1016366499 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0900327f0 con 0x7fc09c0b5550 2024-04-25T10:09:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.184+0000 7fc0a2fe0640 1 -- 172.21.15.16:0/1016366499 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc09c0b5550 msgr2=0x7fc09c0b59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.184+0000 7fc0a2fe0640 1 --2- 172.21.15.16:0/1016366499 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc09c0b5550 0x7fc09c0b59b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc0900097d0 tx=0x7fc09002a180 comp rx=0 tx=0).stop 2024-04-25T10:09:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.184+0000 7fc0a2fe0640 1 -- 172.21.15.16:0/1016366499 shutdown_connections 2024-04-25T10:09:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.184+0000 7fc0a2fe0640 1 --2- 172.21.15.16:0/1016366499 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc09c0b5550 0x7fc09c0b59b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.184+0000 7fc0a2fe0640 1 -- 172.21.15.16:0/1016366499 >> 172.21.15.16:0/1016366499 conn(0x7fc09c0d0550 msgr2=0x7fc09c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.184+0000 7fc0a2fe0640 1 -- 172.21.15.16:0/1016366499 shutdown_connections 2024-04-25T10:09:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.184+0000 7fc0a2fe0640 1 -- 172.21.15.16:0/1016366499 wait complete. 2024-04-25T10:09:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.185+0000 7fc0a2fe0640 1 Processor -- start 2024-04-25T10:09:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.185+0000 7fc0a2fe0640 1 -- start start 2024-04-25T10:09:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.186+0000 7fc0a2fe0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc09c0b5550 0x7fc09c0d4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.186+0000 7fc0a2fe0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc09c0d2560 0x7fc09c14adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.186+0000 7fc0a2fe0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc09c161120 0x7fc09c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.186+0000 7fc0a2fe0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c0d1d60 con 0x7fc09c0b5550 2024-04-25T10:09:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.186+0000 7fc0a2fe0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c162740 con 0x7fc09c0d2560 2024-04-25T10:09:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.186+0000 7fc0a2fe0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c0d2040 con 0x7fc09c161120 2024-04-25T10:09:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.186+0000 7fc0a0d7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc09c0b5550 0x7fc09c0d4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.186+0000 7fc0a0d7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc09c0b5550 0x7fc09c0d4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40768/0 (socket says 172.21.15.16:40768) 2024-04-25T10:09:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.186+0000 7fc0a0d7c640 1 -- 172.21.15.16:0/584559100 learned_addr learned my addr 172.21.15.16:0/584559100 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.186+0000 7fc09bfff640 1 --2- 172.21.15.16:0/584559100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc09c0d2560 0x7fc09c14adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.186+0000 7fc0a157d640 1 --2- 172.21.15.16:0/584559100 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc09c161120 0x7fc09c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.187+0000 7fc0a0d7c640 1 -- 172.21.15.16:0/584559100 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc09c161120 msgr2=0x7fc09c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.187+0000 7fc0a0d7c640 1 --2- 172.21.15.16:0/584559100 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc09c161120 0x7fc09c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.187+0000 7fc0a0d7c640 1 -- 172.21.15.16:0/584559100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc09c0d2560 msgr2=0x7fc09c14adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.187+0000 7fc0a0d7c640 1 --2- 172.21.15.16:0/584559100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc09c0d2560 0x7fc09c14adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.187+0000 7fc0a0d7c640 1 -- 172.21.15.16:0/584559100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09c0cd200 con 0x7fc09c0b5550 2024-04-25T10:09:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.187+0000 7fc0a0d7c640 1 --2- 172.21.15.16:0/584559100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc09c0b5550 0x7fc09c0d4350 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00c4e0 tx=0x7fc08c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.187+0000 7fc0997fa640 1 -- 172.21.15.16:0/584559100 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c017070 con 0x7fc09c0b5550 2024-04-25T10:09:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.187+0000 7fc0997fa640 1 -- 172.21.15.16:0/584559100 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08c0035f0 con 0x7fc09c0b5550 2024-04-25T10:09:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.187+0000 7fc0a2fe0640 1 -- 172.21.15.16:0/584559100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09c0cc160 con 0x7fc09c0b5550 2024-04-25T10:09:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.187+0000 7fc0997fa640 1 -- 172.21.15.16:0/584559100 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c0039e0 con 0x7fc09c0b5550 2024-04-25T10:09:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.187+0000 7fc0a2fe0640 1 -- 172.21.15.16:0/584559100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc09c0cc720 con 0x7fc09c0b5550 2024-04-25T10:09:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.188+0000 7fc0997fa640 1 -- 172.21.15.16:0/584559100 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc08c01a070 con 0x7fc09c0b5550 2024-04-25T10:09:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.189+0000 7fc0997fa640 1 --2- 172.21.15.16:0/584559100 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc08806b4d0 0x7fc08806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.189+0000 7fc0997fa640 1 -- 172.21.15.16:0/584559100 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc08c0894e0 con 0x7fc09c0b5550 2024-04-25T10:09:40.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.189+0000 7fc09bfff640 1 --2- 172.21.15.16:0/584559100 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc08806b4d0 0x7fc08806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.190+0000 7fc09bfff640 1 --2- 172.21.15.16:0/584559100 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc08806b4d0 0x7fc08806d990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc090034000 tx=0x7fc09002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.190+0000 7fc0a5dc3740 1 Processor -- start 2024-04-25T10:09:40.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.190+0000 7fc0a5dc3740 1 -- start start 2024-04-25T10:09:40.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc0a5dc3740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e1b10 0x55b5f37e4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc0a5dc3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1ef0 0x55b5f37e4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc0a5dc3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37b4770 0x55b5f37b7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc0a5dc3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5f371dda0 con 0x55b5f37e1ef0 2024-04-25T10:09:40.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc0a5dc3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5f36c6b80 con 0x55b5f37b4770 2024-04-25T10:09:40.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc0a5dc3740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5f3719e40 con 0x55b5f37e1b10 2024-04-25T10:09:40.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc09bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1ef0 0x55b5f37e4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc09bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1ef0 0x55b5f37e4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40770/0 (socket says 172.21.15.16:40770) 2024-04-25T10:09:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc09bfff640 1 -- 172.21.15.16:0/1542917538 learned_addr learned my addr 172.21.15.16:0/1542917538 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc0a0d7c640 1 --2- 172.21.15.16:0/1542917538 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e1b10 0x55b5f37e4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc0a157d640 1 --2- 172.21.15.16:0/1542917538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37b4770 0x55b5f37b7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc09bfff640 1 -- 172.21.15.16:0/1542917538 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e1b10 msgr2=0x55b5f37e4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc09bfff640 1 --2- 172.21.15.16:0/1542917538 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e1b10 0x55b5f37e4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc09bfff640 1 -- 172.21.15.16:0/1542917538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37b4770 msgr2=0x55b5f37b7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc09bfff640 1 --2- 172.21.15.16:0/1542917538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37b4770 0x55b5f37b7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc09bfff640 1 -- 172.21.15.16:0/1542917538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5f36d0f10 con 0x55b5f37e1ef0 2024-04-25T10:09:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc0a0d7c640 1 --2- 172.21.15.16:0/1542917538 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e1b10 0x55b5f37e4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc0a157d640 1 --2- 172.21.15.16:0/1542917538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37b4770 0x55b5f37b7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc09bfff640 1 --2- 172.21.15.16:0/1542917538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1ef0 0x55b5f37e4860 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc09002a770 tx=0x7fc09002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f159c56ab412eb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc0827fc640 1 -- 172.21.15.16:0/1542917538 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090037040 con 0x55b5f37e1ef0 2024-04-25T10:09:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc0827fc640 1 -- 172.21.15.16:0/1542917538 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09004c070 con 0x55b5f37e1ef0 2024-04-25T10:09:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.191+0000 7fc0827fc640 1 -- 172.21.15.16:0/1542917538 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090032440 con 0x55b5f37e1ef0 2024-04-25T10:09:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.192+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/1542917538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1ef0 msgr2=0x55b5f37e4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.192+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/1542917538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1ef0 0x55b5f37e4860 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc09002a770 tx=0x7fc09002c000 comp rx=0 tx=0).stop 2024-04-25T10:09:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.192+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/1542917538 shutdown_connections 2024-04-25T10:09:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.192+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/1542917538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37b4770 0x55b5f37b7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.192+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/1542917538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1ef0 0x55b5f37e4860 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.192+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/1542917538 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e1b10 0x55b5f37e4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.192+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/1542917538 >> 172.21.15.16:0/1542917538 conn(0x55b5f37b69e0 msgr2=0x55b5f37ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.192+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/1542917538 shutdown_connections 2024-04-25T10:09:40.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.192+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/1542917538 wait complete. 2024-04-25T10:09:40.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.193+0000 7fc0a5dc3740 1 -- start start 2024-04-25T10:09:40.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.193+0000 7fc0a5dc3740 1 Processor -- start 2024-04-25T10:09:40.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a5dc3740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e1b10 0x55b5f37d8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a5dc3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37e1ef0 0x55b5f3802120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a5dc3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e5860 0x55b5f37b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a5dc3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5f37b9b90 con 0x55b5f37e5860 2024-04-25T10:09:40.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a5dc3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5f36bc4b0 con 0x55b5f37e1ef0 2024-04-25T10:09:40.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a5dc3740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5f37b9e70 con 0x55b5f37e1b10 2024-04-25T10:09:40.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a5dc3740 10 client.0 _ll_register_callbacks cb 0x55b5f36fc180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:40.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a157d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e5860 0x55b5f37b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a157d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e5860 0x55b5f37b5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40784/0 (socket says 172.21.15.16:40784) 2024-04-25T10:09:40.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a157d640 1 -- 172.21.15.16:0/4004636533 learned_addr learned my addr 172.21.15.16:0/4004636533 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:40.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc09bfff640 1 --2- 172.21.15.16:0/4004636533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37e1ef0 0x55b5f3802120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a0d7c640 1 --2- 172.21.15.16:0/4004636533 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e1b10 0x55b5f37d8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a157d640 1 -- 172.21.15.16:0/4004636533 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e1b10 msgr2=0x55b5f37d8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a157d640 1 --2- 172.21.15.16:0/4004636533 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e1b10 0x55b5f37d8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a157d640 1 -- 172.21.15.16:0/4004636533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37e1ef0 msgr2=0x55b5f3802120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a157d640 1 --2- 172.21.15.16:0/4004636533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37e1ef0 0x55b5f3802120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a157d640 1 -- 172.21.15.16:0/4004636533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5f34fc7d0 con 0x55b5f37e5860 2024-04-25T10:09:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a157d640 1 --2- 172.21.15.16:0/4004636533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e5860 0x55b5f37b5420 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fc094007690 tx=0x7fc094007b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ac7ad5d2b914d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc07b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:09:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc07b7fe640 1 -- 172.21.15.16:0/4004636533 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094019020 con 0x55b5f37e5860 2024-04-25T10:09:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc07b7fe640 1 -- 172.21.15.16:0/4004636533 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc094011a40 con 0x55b5f37e5860 2024-04-25T10:09:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/4004636533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc09004c070 con 0x55b5f37e5860 2024-04-25T10:09:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc07b7fe640 1 -- 172.21.15.16:0/4004636533 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc094023050 con 0x55b5f37e5860 2024-04-25T10:09:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.194+0000 7fc0a5dc3740 10 client.6978 fetch_fsmap learned FSMap version 42 2024-04-25T10:09:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.195+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/4004636533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x55b5f34323f0 con 0x55b5f37e5860 2024-04-25T10:09:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.195+0000 7fc07b7fe640 1 -- 172.21.15.16:0/4004636533 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09004c070 con 0x55b5f37e5860 2024-04-25T10:09:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.195+0000 7fc07b7fe640 1 -- 172.21.15.16:0/4004636533 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc094026020 con 0x55b5f37e5860 2024-04-25T10:09:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.195+0000 7fc0a5dc3740 10 client.6978 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T10:09:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.195+0000 7fc0a5dc3740 10 client.6978 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:09:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.195+0000 7fc0a5dc3740 20 client.6978 populate_metadata read hostname 'smithi016' 2024-04-25T10:09:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.195+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/4004636533 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55b5f3800920 0x55b5f3800d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.195+0000 7fc0a5dc3740 4 client.6978 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.195+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/4004636533 --> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5f372b2e0 con 0x55b5f3800920 2024-04-25T10:09:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.195+0000 7fc0a0d7c640 1 --2- 172.21.15.16:0/4004636533 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55b5f3800920 0x55b5f3800d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.195+0000 7fc07b7fe640 10 client.6978 ms_handle_connect on v2:172.21.15.16:6834/1735811292 2024-04-25T10:09:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.195+0000 7fc0a0d7c640 1 --2- 172.21.15.16:0/4004636533 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55b5f3800920 0x55b5f3800d30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2473c07cc4f6022d server_cookie=733a3e191c9e7caa in_seq=0 out_seq=0 2024-04-25T10:09:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.196+0000 7fc07b7fe640 1 -- 172.21.15.16:0/4004636533 <== mds.0 v2:172.21.15.16:6834/1735811292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b5f372b2e0 con 0x55b5f3800920 2024-04-25T10:09:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.196+0000 7fc07b7fe640 10 client.6978 handle_command_reply: tid=0 2024-04-25T10:09:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.196+0000 7fc0a5dc3740 1 client.6978 shutdown 2024-04-25T10:09:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.196+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/4004636533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e5860 msgr2=0x55b5f37b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.196+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/4004636533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e5860 0x55b5f37b5420 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fc094007690 tx=0x7fc094007b50 comp rx=0 tx=0).stop 2024-04-25T10:09:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.196+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/4004636533 shutdown_connections 2024-04-25T10:09:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.196+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/4004636533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e5860 0x55b5f37b5420 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.196+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/4004636533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37e1ef0 0x55b5f3802120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.196+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/4004636533 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55b5f3800920 0x55b5f3800d30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.196+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/4004636533 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e1b10 0x55b5f37d8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.196+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/4004636533 >> 172.21.15.16:0/4004636533 conn(0x55b5f37b4770 msgr2=0x55b5f37e0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.196+0000 7fc07b7fe640 0 client.6978 ms_handle_reset on v2:172.21.15.16:6834/1735811292 2024-04-25T10:09:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.196+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/4004636533 shutdown_connections 2024-04-25T10:09:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.196+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/4004636533 wait complete. 2024-04-25T10:09:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.196+0000 7fc0a5dc3740 20 client.6978 trim_cache size 0 max 16384 2024-04-25T10:09:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.211+0000 7fc0a5dc3740 1 Processor -- start 2024-04-25T10:09:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.211+0000 7fc0a5dc3740 1 -- start start 2024-04-25T10:09:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.212+0000 7fc0a5dc3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37e1b10 0x55b5f37bb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.212+0000 7fc0a5dc3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1ef0 0x55b5f37ff320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.212+0000 7fc0a5dc3740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e5860 0x55b5f37ff860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.212+0000 7fc0a5dc3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c1ff910 con 0x55b5f37e1ef0 2024-04-25T10:09:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.212+0000 7fc0a5dc3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c1feae0 con 0x55b5f37e1b10 2024-04-25T10:09:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.212+0000 7fc0a5dc3740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c0c8160 con 0x55b5f37e5860 2024-04-25T10:09:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.212+0000 7fc09bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1ef0 0x55b5f37ff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.212+0000 7fc09bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1ef0 0x55b5f37ff320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40794/0 (socket says 172.21.15.16:40794) 2024-04-25T10:09:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.212+0000 7fc09bfff640 1 -- 172.21.15.16:0/1078527615 learned_addr learned my addr 172.21.15.16:0/1078527615 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.212+0000 7fc0a0d7c640 1 --2- 172.21.15.16:0/1078527615 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37e1b10 0x55b5f37bb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.212+0000 7fc0a157d640 1 --2- 172.21.15.16:0/1078527615 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e5860 0x55b5f37ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.212+0000 7fc09bfff640 1 -- 172.21.15.16:0/1078527615 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e5860 msgr2=0x55b5f37ff860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.212+0000 7fc09bfff640 1 --2- 172.21.15.16:0/1078527615 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e5860 0x55b5f37ff860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.212+0000 7fc09bfff640 1 -- 172.21.15.16:0/1078527615 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37e1b10 msgr2=0x55b5f37bb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.212+0000 7fc09bfff640 1 --2- 172.21.15.16:0/1078527615 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37e1b10 0x55b5f37bb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.212+0000 7fc09bfff640 1 -- 172.21.15.16:0/1078527615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5f33f20f0 con 0x55b5f37e1ef0 2024-04-25T10:09:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.212+0000 7fc09bfff640 1 --2- 172.21.15.16:0/1078527615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1ef0 0x55b5f37ff320 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fc0900043a0 tx=0x7fc090009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6e9c96061dcc11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.213+0000 7fc081ffb640 1 -- 172.21.15.16:0/1078527615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090056030 con 0x55b5f37e1ef0 2024-04-25T10:09:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.213+0000 7fc081ffb640 1 -- 172.21.15.16:0/1078527615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09004c040 con 0x55b5f37e1ef0 2024-04-25T10:09:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.213+0000 7fc081ffb640 1 -- 172.21.15.16:0/1078527615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090003ce0 con 0x55b5f37e1ef0 2024-04-25T10:09:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.213+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/1078527615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1ef0 msgr2=0x55b5f37ff320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.213+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/1078527615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1ef0 0x55b5f37ff320 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fc0900043a0 tx=0x7fc090009190 comp rx=0 tx=0).stop 2024-04-25T10:09:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.213+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/1078527615 shutdown_connections 2024-04-25T10:09:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.213+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/1078527615 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e5860 0x55b5f37ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.213+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/1078527615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1ef0 0x55b5f37ff320 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.213+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/1078527615 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37e1b10 0x55b5f37bb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.213+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/1078527615 >> 172.21.15.16:0/1078527615 conn(0x55b5f37b4770 msgr2=0x55b5f37ba7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.213+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/1078527615 shutdown_connections 2024-04-25T10:09:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.213+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/1078527615 wait complete. 2024-04-25T10:09:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.214+0000 7fc0a5dc3740 1 -- start start 2024-04-25T10:09:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.214+0000 7fc0a5dc3740 1 Processor -- start 2024-04-25T10:09:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc0a5dc3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1b10 0x55b5f37ff320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc0a5dc3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37e1ef0 0x55b5f37ff860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc0a5dc3740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e5860 0x55b5f386a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc0a5dc3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5f37af1a0 con 0x55b5f37e1b10 2024-04-25T10:09:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc0a5dc3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c1fe4e0 con 0x55b5f37e1ef0 2024-04-25T10:09:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc0a5dc3740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5f37d81d0 con 0x55b5f37e5860 2024-04-25T10:09:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc0a5dc3740 10 client.0 _ll_register_callbacks cb 0x7fc0840036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc0a0d7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1b10 0x55b5f37ff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc0a0d7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1b10 0x55b5f37ff320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40800/0 (socket says 172.21.15.16:40800) 2024-04-25T10:09:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc0a0d7c640 1 -- 172.21.15.16:0/2733124848 learned_addr learned my addr 172.21.15.16:0/2733124848 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:09:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc09bfff640 1 --2- 172.21.15.16:0/2733124848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37e1ef0 0x55b5f37ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc0a157d640 1 --2- 172.21.15.16:0/2733124848 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e5860 0x55b5f386a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc0a0d7c640 1 -- 172.21.15.16:0/2733124848 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e5860 msgr2=0x55b5f386a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc0a0d7c640 1 --2- 172.21.15.16:0/2733124848 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e5860 0x55b5f386a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc0a0d7c640 1 -- 172.21.15.16:0/2733124848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37e1ef0 msgr2=0x55b5f37ff860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc0a0d7c640 1 --2- 172.21.15.16:0/2733124848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37e1ef0 0x55b5f37ff860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc0a0d7c640 1 -- 172.21.15.16:0/2733124848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5f3869fe0 con 0x55b5f37e1b10 2024-04-25T10:09:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc0a0d7c640 1 --2- 172.21.15.16:0/2733124848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1b10 0x55b5f37ff320 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0579b0 tx=0x7fc08c089040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114586e2121ef6b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc07b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:09:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc07b7fe640 1 -- 172.21.15.16:0/2733124848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c074060 con 0x55b5f37e1b10 2024-04-25T10:09:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc07b7fe640 1 -- 172.21.15.16:0/2733124848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc08c054320 con 0x55b5f37e1b10 2024-04-25T10:09:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.215+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/2733124848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc09004c040 con 0x55b5f37e1b10 2024-04-25T10:09:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.216+0000 7fc07b7fe640 1 -- 172.21.15.16:0/2733124848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc08c060310 con 0x55b5f37e1b10 2024-04-25T10:09:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.216+0000 7fc0a5dc3740 10 client.6984 fetch_fsmap learned FSMap version 42 2024-04-25T10:09:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.216+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/2733124848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x55b5f37b3930 con 0x55b5f37e1b10 2024-04-25T10:09:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.216+0000 7fc07b7fe640 1 -- 172.21.15.16:0/2733124848 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09004c040 con 0x55b5f37e1b10 2024-04-25T10:09:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.216+0000 7fc07b7fe640 1 -- 172.21.15.16:0/2733124848 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc08c0890b0 con 0x55b5f37e1b10 2024-04-25T10:09:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.216+0000 7fc0a5dc3740 10 client.6984 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T10:09:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.216+0000 7fc0a5dc3740 10 client.6984 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:09:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.216+0000 7fc0a5dc3740 20 client.6984 populate_metadata read hostname 'smithi016' 2024-04-25T10:09:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.216+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/2733124848 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55b5f37e1160 0x55b5f386aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.216+0000 7fc0a5dc3740 4 client.6984 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:09:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.216+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/2733124848 --> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b5f3725630 con 0x55b5f37e1160 2024-04-25T10:09:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.216+0000 7fc09bfff640 1 --2- 172.21.15.16:0/2733124848 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55b5f37e1160 0x55b5f386aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.216+0000 7fc07b7fe640 10 client.6984 ms_handle_connect on v2:172.21.15.16:6834/1735811292 2024-04-25T10:09:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.216+0000 7fc09bfff640 1 --2- 172.21.15.16:0/2733124848 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55b5f37e1160 0x55b5f386aa00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a0bb6fa863ee19 server_cookie=9ad2a6be8c3a18eb in_seq=0 out_seq=0 2024-04-25T10:09:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.216+0000 7fc07b7fe640 1 -- 172.21.15.16:0/2733124848 <== mds.0 v2:172.21.15.16:6834/1735811292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b5f3725630 con 0x55b5f37e1160 2024-04-25T10:09:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.216+0000 7fc07b7fe640 10 client.6984 handle_command_reply: tid=0 2024-04-25T10:09:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.216+0000 7fc0a5dc3740 1 client.6984 shutdown 2024-04-25T10:09:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/2733124848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1b10 msgr2=0x55b5f37ff320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/2733124848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1b10 0x55b5f37ff320 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0579b0 tx=0x7fc08c089040 comp rx=0 tx=0).stop 2024-04-25T10:09:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/2733124848 shutdown_connections 2024-04-25T10:09:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/2733124848 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55b5f37e1160 0x55b5f386aa00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/2733124848 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b5f37e5860 0x55b5f386a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/2733124848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5f37e1ef0 0x55b5f37ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc07b7fe640 0 client.6984 ms_handle_reset on v2:172.21.15.16:6834/1735811292 2024-04-25T10:09:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a5dc3740 1 --2- 172.21.15.16:0/2733124848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5f37e1b10 0x55b5f37ff320 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/2733124848 >> 172.21.15.16:0/2733124848 conn(0x55b5f37b4770 msgr2=0x55b5f37e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/2733124848 shutdown_connections 2024-04-25T10:09:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a5dc3740 1 -- 172.21.15.16:0/2733124848 wait complete. 2024-04-25T10:09:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a5dc3740 20 client.6984 trim_cache size 0 max 16384 2024-04-25T10:09:40.237 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:09:40.237 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:09:40.238 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-04-25T10:09:40.238 INFO:teuthology.orchestra.run.smithi016.stdout: "c713549c-8958-45b1-9500-5d76f27cafab": { 2024-04-25T10:09:40.238 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/subdir", 2024-04-25T10:09:40.238 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "c713549c-8958-45b1-9500-5d76f27cafab", 2024-04-25T10:09:40.238 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive" 2024-04-25T10:09:40.238 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:40.238 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:09:40.238 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:09:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a2fe0640 1 -- 172.21.15.16:0/584559100 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc08806b4d0 msgr2=0x7fc08806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a2fe0640 1 --2- 172.21.15.16:0/584559100 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc08806b4d0 0x7fc08806d990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc090034000 tx=0x7fc09002e000 comp rx=0 tx=0).stop 2024-04-25T10:09:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a2fe0640 1 -- 172.21.15.16:0/584559100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc09c0b5550 msgr2=0x7fc09c0d4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a2fe0640 1 --2- 172.21.15.16:0/584559100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc09c0b5550 0x7fc09c0d4350 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00c4e0 tx=0x7fc08c00c9a0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a2fe0640 1 -- 172.21.15.16:0/584559100 shutdown_connections 2024-04-25T10:09:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a2fe0640 1 --2- 172.21.15.16:0/584559100 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc08806b4d0 0x7fc08806d990 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a2fe0640 1 --2- 172.21.15.16:0/584559100 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc09c161120 0x7fc09c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a2fe0640 1 --2- 172.21.15.16:0/584559100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc09c0d2560 0x7fc09c14adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a2fe0640 1 --2- 172.21.15.16:0/584559100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc09c0b5550 0x7fc09c0d4350 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.217+0000 7fc0a2fe0640 1 -- 172.21.15.16:0/584559100 >> 172.21.15.16:0/584559100 conn(0x7fc09c0d0550 msgr2=0x7fc09c0b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.218+0000 7fc0a2fe0640 1 -- 172.21.15.16:0/584559100 shutdown_connections 2024-04-25T10:09:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:09:40.218+0000 7fc0a2fe0640 1 -- 172.21.15.16:0/584559100 wait complete. 2024-04-25T10:09:40.241 INFO:tasks.cephfs.filesystem:scrub status for tag:c713549c-8958-45b1-9500-5d76f27cafab - {'path': '/subdir', 'tag': 'c713549c-8958-45b1-9500-5d76f27cafab', 'options': 'recursive'} 2024-04-25T10:09:46.420 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-04-25T10:09:46.414+0000 7ff1421c0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:10:00.005 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-04-25T10:09:59.999+0000 7ff1421c0640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-04-25T10:10:07.858 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:07.861 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:10.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:10.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:10:10.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.353+0000 7f53f5f7e640 1 Processor -- start 2024-04-25T10:10:10.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.353+0000 7f53f5f7e640 1 -- start start 2024-04-25T10:10:10.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.354+0000 7f53f5f7e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f53f015ba70 0x7f53f00d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.354+0000 7f53f5f7e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53f00b7030 0x7f53f00b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.354+0000 7f53f5f7e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f00cfcf0 con 0x7f53f00ce230 2024-04-25T10:10:10.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.354+0000 7f53f5f7e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f00cffd0 con 0x7f53f00b7030 2024-04-25T10:10:10.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.354+0000 7f53f5f7e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f00d02b0 con 0x7f53f015ba70 2024-04-25T10:10:10.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.354+0000 7f53eeffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53f00b7030 0x7f53f00b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.354+0000 7f53effff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f53f00ce230 0x7f53f00ce650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58332/0 (socket says 172.21.15.16:58332) 2024-04-25T10:10:10.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.354+0000 7f53effff640 1 -- 172.21.15.16:0/3413380559 learned_addr learned my addr 172.21.15.16:0/3413380559 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:10.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.354+0000 7f53ef7fe640 1 --2- 172.21.15.16:0/3413380559 >> v2:172.21.15.186:3300/0 conn(0x7f53f015ba70 0x7f53f00d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.354+0000 7f53eeffd640 1 -- 172.21.15.16:0/3413380559 >> v2:172.21.15.186:3300/0 conn(0x7f53f015ba70 msgr2=0x7f53f00d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.354+0000 7f53eeffd640 1 --2- 172.21.15.16:0/3413380559 >> v2:172.21.15.186:3300/0 conn(0x7f53f015ba70 0x7f53f00d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.354+0000 7f53eeffd640 1 -- 172.21.15.16:0/3413380559 >> v1:172.21.15.16:6789/0 conn(0x7f53f00ce230 legacy=0x7f53f00ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.354+0000 7f53eeffd640 1 -- 172.21.15.16:0/3413380559 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f00d0590 con 0x7f53f00b7030 2024-04-25T10:10:10.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.355+0000 7f53eeffd640 1 --2- 172.21.15.16:0/3413380559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f00b7030 0x7f53f00b74c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f53f00700e0 tx=0x7f53e402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741771098ddbc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:10.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.355+0000 7f53ee7fc640 1 -- 172.21.15.16:0/3413380559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4037070 con 0x7f53f00b7030 2024-04-25T10:10:10.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.355+0000 7f53ee7fc640 1 -- 172.21.15.16:0/3413380559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e4032440 con 0x7f53f00b7030 2024-04-25T10:10:10.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.355+0000 7f53ee7fc640 1 -- 172.21.15.16:0/3413380559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e402ae60 con 0x7f53f00b7030 2024-04-25T10:10:10.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.356+0000 7f53f5f7e640 1 -- 172.21.15.16:0/3413380559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f00b7030 msgr2=0x7f53f00b74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.356+0000 7f53f5f7e640 1 --2- 172.21.15.16:0/3413380559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f00b7030 0x7f53f00b74c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f53f00700e0 tx=0x7f53e402a110 comp rx=0 tx=0).stop 2024-04-25T10:10:10.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.356+0000 7f53f5f7e640 1 -- 172.21.15.16:0/3413380559 shutdown_connections 2024-04-25T10:10:10.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.356+0000 7f53f5f7e640 1 --2- 172.21.15.16:0/3413380559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f00b7030 0x7f53f00b74c0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.356+0000 7f53f5f7e640 1 --2- 172.21.15.16:0/3413380559 >> v2:172.21.15.186:3300/0 conn(0x7f53f015ba70 0x7f53f00d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.356+0000 7f53f5f7e640 1 -- 172.21.15.16:0/3413380559 >> 172.21.15.16:0/3413380559 conn(0x7f53f006f4a0 msgr2=0x7f53f006f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:10.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.356+0000 7f53f5f7e640 1 -- 172.21.15.16:0/3413380559 shutdown_connections 2024-04-25T10:10:10.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.356+0000 7f53f5f7e640 1 -- 172.21.15.16:0/3413380559 wait complete. 2024-04-25T10:10:10.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.357+0000 7f53f5f7e640 1 Processor -- start 2024-04-25T10:10:10.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.357+0000 7f53f5f7e640 1 -- start start 2024-04-25T10:10:10.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.358+0000 7f53f5f7e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53f00b7030 0x7f53f01f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.358+0000 7f53f5f7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f00ce230 0x7f53f00b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.358+0000 7f53f5f7e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53f015ba70 0x7f53f0157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.358+0000 7f53f5f7e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f015be50 con 0x7f53f00ce230 2024-04-25T10:10:10.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.358+0000 7f53f5f7e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f00cf980 con 0x7f53f00b7030 2024-04-25T10:10:10.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.358+0000 7f53f5f7e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f00b1870 con 0x7f53f015ba70 2024-04-25T10:10:10.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.358+0000 7f53eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f00ce230 0x7f53f00b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.358+0000 7f53eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f00ce230 0x7f53f00b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47938/0 (socket says 172.21.15.16:47938) 2024-04-25T10:10:10.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.358+0000 7f53eeffd640 1 -- 172.21.15.16:0/2078275554 learned_addr learned my addr 172.21.15.16:0/2078275554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:10.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.358+0000 7f53ef7fe640 1 --2- 172.21.15.16:0/2078275554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53f00b7030 0x7f53f01f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.358+0000 7f53effff640 1 --2- 172.21.15.16:0/2078275554 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53f015ba70 0x7f53f0157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.359+0000 7f53eeffd640 1 -- 172.21.15.16:0/2078275554 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53f015ba70 msgr2=0x7f53f0157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.359+0000 7f53eeffd640 1 --2- 172.21.15.16:0/2078275554 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53f015ba70 0x7f53f0157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.359+0000 7f53eeffd640 1 -- 172.21.15.16:0/2078275554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53f00b7030 msgr2=0x7f53f01f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.359+0000 7f53eeffd640 1 --2- 172.21.15.16:0/2078275554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53f00b7030 0x7f53f01f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.359+0000 7f53eeffd640 1 -- 172.21.15.16:0/2078275554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f01f4fb0 con 0x7f53f00ce230 2024-04-25T10:10:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.359+0000 7f53eeffd640 1 --2- 172.21.15.16:0/2078275554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f00ce230 0x7f53f00b1330 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009890 tx=0x7f53e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.359+0000 7f53d7fff640 1 -- 172.21.15.16:0/2078275554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4037040 con 0x7f53f00ce230 2024-04-25T10:10:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.359+0000 7f53d7fff640 1 -- 172.21.15.16:0/2078275554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e4042070 con 0x7f53f00ce230 2024-04-25T10:10:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.359+0000 7f53f5f7e640 1 -- 172.21.15.16:0/2078275554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f01f5290 con 0x7f53f00ce230 2024-04-25T10:10:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.359+0000 7f53d7fff640 1 -- 172.21.15.16:0/2078275554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4030bd0 con 0x7f53f00ce230 2024-04-25T10:10:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.359+0000 7f53f5f7e640 1 -- 172.21.15.16:0/2078275554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f01f5850 con 0x7f53f00ce230 2024-04-25T10:10:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.361+0000 7f53d7fff640 1 -- 172.21.15.16:0/2078275554 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53e4047460 con 0x7f53f00ce230 2024-04-25T10:10:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.361+0000 7f53d7fff640 1 --2- 172.21.15.16:0/2078275554 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f53dc06b4d0 0x7f53dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.361+0000 7f53d7fff640 1 -- 172.21.15.16:0/2078275554 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f53e40aac30 con 0x7f53f00ce230 2024-04-25T10:10:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.361+0000 7f53ef7fe640 1 --2- 172.21.15.16:0/2078275554 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f53dc06b4d0 0x7f53dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.362+0000 7f53ef7fe640 1 --2- 172.21.15.16:0/2078275554 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f53dc06b4d0 0x7f53dc06d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f53e00079a0 tx=0x7f53e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.362+0000 7f53f8c82740 1 Processor -- start 2024-04-25T10:10:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.362+0000 7f53f8c82740 1 -- start start 2024-04-25T10:10:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.363+0000 7f53f8c82740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fa3b10 0x561bf3fa6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.363+0000 7f53f8c82740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3ef0 0x561bf3fa6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.363+0000 7f53f8c82740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3f76770 0x561bf3f79f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.363+0000 7f53f8c82740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf3edfda0 con 0x561bf3fa3ef0 2024-04-25T10:10:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.363+0000 7f53f8c82740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf3e88b80 con 0x561bf3fa3b10 2024-04-25T10:10:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.363+0000 7f53f8c82740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x561bf3edbe40 con 0x561bf3f76770 2024-04-25T10:10:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.363+0000 7f53eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3ef0 0x561bf3fa6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.363+0000 7f53eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3ef0 0x561bf3fa6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47942/0 (socket says 172.21.15.16:47942) 2024-04-25T10:10:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.363+0000 7f53eeffd640 1 -- 172.21.15.16:0/3015378622 learned_addr learned my addr 172.21.15.16:0/3015378622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.364+0000 7f53ef7fe640 1 --2- 172.21.15.16:0/3015378622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fa3b10 0x561bf3fa6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.364+0000 7f53effff640 1 --2- 172.21.15.16:0/3015378622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3f76770 0x561bf3f79f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.364+0000 7f53eeffd640 1 -- 172.21.15.16:0/3015378622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3f76770 msgr2=0x561bf3f79f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.364+0000 7f53eeffd640 1 --2- 172.21.15.16:0/3015378622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3f76770 0x561bf3f79f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.364+0000 7f53eeffd640 1 -- 172.21.15.16:0/3015378622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fa3b10 msgr2=0x561bf3fa6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.364+0000 7f53eeffd640 1 --2- 172.21.15.16:0/3015378622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fa3b10 0x561bf3fa6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.364+0000 7f53eeffd640 1 -- 172.21.15.16:0/3015378622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bf3e92f10 con 0x561bf3fa3ef0 2024-04-25T10:10:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.364+0000 7f53eeffd640 1 --2- 172.21.15.16:0/3015378622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3ef0 0x561bf3fa6860 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f53e40758a0 tx=0x7f53e402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bb557fb2e9555b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.364+0000 7f53d57fa640 1 -- 172.21.15.16:0/3015378622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e407d4e0 con 0x561bf3fa3ef0 2024-04-25T10:10:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.364+0000 7f53d57fa640 1 -- 172.21.15.16:0/3015378622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e4081aa0 con 0x561bf3fa3ef0 2024-04-25T10:10:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.364+0000 7f53d57fa640 1 -- 172.21.15.16:0/3015378622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4030d70 con 0x561bf3fa3ef0 2024-04-25T10:10:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.365+0000 7f53f8c82740 1 -- 172.21.15.16:0/3015378622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3ef0 msgr2=0x561bf3fa6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.365+0000 7f53f8c82740 1 --2- 172.21.15.16:0/3015378622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3ef0 0x561bf3fa6860 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f53e40758a0 tx=0x7f53e402a6c0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.365+0000 7f53f8c82740 1 -- 172.21.15.16:0/3015378622 shutdown_connections 2024-04-25T10:10:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.365+0000 7f53f8c82740 1 --2- 172.21.15.16:0/3015378622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3f76770 0x561bf3f79f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.365+0000 7f53f8c82740 1 --2- 172.21.15.16:0/3015378622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3ef0 0x561bf3fa6860 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.365+0000 7f53f8c82740 1 --2- 172.21.15.16:0/3015378622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fa3b10 0x561bf3fa6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.365+0000 7f53f8c82740 1 -- 172.21.15.16:0/3015378622 >> 172.21.15.16:0/3015378622 conn(0x561bf3f789e0 msgr2=0x561bf3fc1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.365+0000 7f53f8c82740 1 -- 172.21.15.16:0/3015378622 shutdown_connections 2024-04-25T10:10:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.365+0000 7f53f8c82740 1 -- 172.21.15.16:0/3015378622 wait complete. 2024-04-25T10:10:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.365+0000 7f53f8c82740 1 -- start start 2024-04-25T10:10:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.365+0000 7f53f8c82740 1 Processor -- start 2024-04-25T10:10:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.366+0000 7f53f8c82740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3b10 0x561bf3f99a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.366+0000 7f53f8c82740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3fa3ef0 0x561bf3fbd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.366+0000 7f53f8c82740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fbddd0 0x561bf3f7af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.366+0000 7f53f8c82740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf3edfda0 con 0x561bf3fa3b10 2024-04-25T10:10:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.366+0000 7f53f8c82740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf3e7e4b0 con 0x561bf3fbddd0 2024-04-25T10:10:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.366+0000 7f53f8c82740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x561bf3f7bb90 con 0x561bf3fa3ef0 2024-04-25T10:10:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.366+0000 7f53f8c82740 10 client.0 _ll_register_callbacks cb 0x561bf3ebe180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.366+0000 7f53ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3b10 0x561bf3f99a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.366+0000 7f53ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3b10 0x561bf3f99a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47946/0 (socket says 172.21.15.16:47946) 2024-04-25T10:10:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.366+0000 7f53ef7fe640 1 -- 172.21.15.16:0/4138390090 learned_addr learned my addr 172.21.15.16:0/4138390090 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.366+0000 7f53eeffd640 1 --2- 172.21.15.16:0/4138390090 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3fa3ef0 0x561bf3fbd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.366+0000 7f53effff640 1 --2- 172.21.15.16:0/4138390090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fbddd0 0x561bf3f7af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.367+0000 7f53ef7fe640 1 -- 172.21.15.16:0/4138390090 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3fa3ef0 msgr2=0x561bf3fbd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.367+0000 7f53ef7fe640 1 --2- 172.21.15.16:0/4138390090 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3fa3ef0 0x561bf3fbd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.367+0000 7f53ef7fe640 1 -- 172.21.15.16:0/4138390090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fbddd0 msgr2=0x561bf3f7af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.367+0000 7f53ef7fe640 1 --2- 172.21.15.16:0/4138390090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fbddd0 0x561bf3f7af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.367+0000 7f53ef7fe640 1 -- 172.21.15.16:0/4138390090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bf3cbe7d0 con 0x561bf3fa3b10 2024-04-25T10:10:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.367+0000 7f53ef7fe640 1 --2- 172.21.15.16:0/4138390090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3b10 0x561bf3f99a80 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f53e0016830 tx=0x7f53e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a281b11757213a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.367+0000 7f53ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:10:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.367+0000 7f53ce7fc640 1 -- 172.21.15.16:0/4138390090 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0026020 con 0x561bf3fa3b10 2024-04-25T10:10:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.367+0000 7f53ce7fc640 1 -- 172.21.15.16:0/4138390090 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f53e00149b0 con 0x561bf3fa3b10 2024-04-25T10:10:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.367+0000 7f53f8c82740 1 -- 172.21.15.16:0/4138390090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53e4081aa0 con 0x561bf3fa3b10 2024-04-25T10:10:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.367+0000 7f53ce7fc640 1 -- 172.21.15.16:0/4138390090 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53e0030050 con 0x561bf3fa3b10 2024-04-25T10:10:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.367+0000 7f53f8c82740 10 client.6999 fetch_fsmap learned FSMap version 43 2024-04-25T10:10:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.367+0000 7f53f8c82740 1 -- 172.21.15.16:0/4138390090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x561bf3bf43f0 con 0x561bf3fa3b10 2024-04-25T10:10:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.367+0000 7f53ce7fc640 1 -- 172.21.15.16:0/4138390090 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e4081aa0 con 0x561bf3fa3b10 2024-04-25T10:10:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.367+0000 7f53ce7fc640 1 -- 172.21.15.16:0/4138390090 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f53e0033020 con 0x561bf3fa3b10 2024-04-25T10:10:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.367+0000 7f53f8c82740 10 client.6999 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T10:10:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.367+0000 7f53f8c82740 10 client.6999 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:10:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.367+0000 7f53f8c82740 20 client.6999 populate_metadata read hostname 'smithi016' 2024-04-25T10:10:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.368+0000 7f53f8c82740 1 --2- 172.21.15.16:0/4138390090 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x561bf3fbbe20 0x561bf3fbc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.368+0000 7f53f8c82740 4 client.6999 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.368+0000 7f53f8c82740 1 -- 172.21.15.16:0/4138390090 --> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bf3eed2e0 con 0x561bf3fbbe20 2024-04-25T10:10:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.368+0000 7f53eeffd640 1 --2- 172.21.15.16:0/4138390090 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x561bf3fbbe20 0x561bf3fbc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.368+0000 7f53eeffd640 1 --2- 172.21.15.16:0/4138390090 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x561bf3fbbe20 0x561bf3fbc200 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8def17315ffd65f1 server_cookie=6e43d9bd180a60fb in_seq=0 out_seq=0 2024-04-25T10:10:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.368+0000 7f53ce7fc640 10 client.6999 ms_handle_connect on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.368+0000 7f53ce7fc640 1 -- 172.21.15.16:0/4138390090 <== mds.0 v2:172.21.15.16:6834/1735811292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561bf3eed2e0 con 0x561bf3fbbe20 2024-04-25T10:10:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.368+0000 7f53ce7fc640 10 client.6999 handle_command_reply: tid=0 2024-04-25T10:10:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.369+0000 7f53f8c82740 1 client.6999 shutdown 2024-04-25T10:10:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.369+0000 7f53f8c82740 1 -- 172.21.15.16:0/4138390090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3b10 msgr2=0x561bf3f99a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.369+0000 7f53f8c82740 1 --2- 172.21.15.16:0/4138390090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3b10 0x561bf3f99a80 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f53e0016830 tx=0x7f53e0004730 comp rx=0 tx=0).stop 2024-04-25T10:10:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.369+0000 7f53f8c82740 1 -- 172.21.15.16:0/4138390090 shutdown_connections 2024-04-25T10:10:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.369+0000 7f53f8c82740 1 --2- 172.21.15.16:0/4138390090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fbddd0 0x561bf3f7af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.369+0000 7f53f8c82740 1 --2- 172.21.15.16:0/4138390090 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x561bf3fbbe20 0x561bf3fbc200 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.369+0000 7f53f8c82740 1 --2- 172.21.15.16:0/4138390090 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3fa3ef0 0x561bf3fbd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.369+0000 7f53f8c82740 1 --2- 172.21.15.16:0/4138390090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3b10 0x561bf3f99a80 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.369+0000 7f53f8c82740 1 -- 172.21.15.16:0/4138390090 >> 172.21.15.16:0/4138390090 conn(0x561bf3f76770 msgr2=0x561bf3fa2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.369+0000 7f53ce7fc640 0 client.6999 ms_handle_reset on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.369+0000 7f53f8c82740 1 -- 172.21.15.16:0/4138390090 shutdown_connections 2024-04-25T10:10:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.369+0000 7f53f8c82740 1 -- 172.21.15.16:0/4138390090 wait complete. 2024-04-25T10:10:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.369+0000 7f53f8c82740 20 client.6999 trim_cache size 0 max 16384 2024-04-25T10:10:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.385+0000 7f53f8c82740 1 Processor -- start 2024-04-25T10:10:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.385+0000 7f53f8c82740 1 -- start start 2024-04-25T10:10:10.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.386+0000 7f53f8c82740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3b10 0x561bf3f7baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.386+0000 7f53f8c82740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fa3ef0 0x561bf3f7c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.386+0000 7f53f8c82740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3fbddd0 0x561bf3fa21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.386+0000 7f53f8c82740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f01fe040 con 0x561bf3fa3b10 2024-04-25T10:10:10.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.386+0000 7f53f8c82740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f01fd240 con 0x561bf3fa3ef0 2024-04-25T10:10:10.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.386+0000 7f53f8c82740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f01599a0 con 0x561bf3fbddd0 2024-04-25T10:10:10.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.386+0000 7f53ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3b10 0x561bf3f7baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.386+0000 7f53ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3b10 0x561bf3f7baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47950/0 (socket says 172.21.15.16:47950) 2024-04-25T10:10:10.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.386+0000 7f53ef7fe640 1 -- 172.21.15.16:0/1552112573 learned_addr learned my addr 172.21.15.16:0/1552112573 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:10.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.386+0000 7f53eeffd640 1 --2- 172.21.15.16:0/1552112573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fa3ef0 0x561bf3f7c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.386+0000 7f53effff640 1 --2- 172.21.15.16:0/1552112573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3fbddd0 0x561bf3fa21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.386+0000 7f53ef7fe640 1 -- 172.21.15.16:0/1552112573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3fbddd0 msgr2=0x561bf3fa21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.386+0000 7f53ef7fe640 1 --2- 172.21.15.16:0/1552112573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3fbddd0 0x561bf3fa21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.386+0000 7f53ef7fe640 1 -- 172.21.15.16:0/1552112573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fa3ef0 msgr2=0x561bf3f7c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.386+0000 7f53ef7fe640 1 --2- 172.21.15.16:0/1552112573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fa3ef0 0x561bf3f7c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.386+0000 7f53ef7fe640 1 -- 172.21.15.16:0/1552112573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bf3bb40f0 con 0x561bf3fa3b10 2024-04-25T10:10:10.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.386+0000 7f53ef7fe640 1 --2- 172.21.15.16:0/1552112573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3b10 0x561bf3f7baf0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f53e0016480 tx=0x7f53e0021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7307cb35bd3f1dd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:10.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.387+0000 7f53d4ff9640 1 -- 172.21.15.16:0/1552112573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0038070 con 0x561bf3fa3b10 2024-04-25T10:10:10.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.387+0000 7f53d4ff9640 1 -- 172.21.15.16:0/1552112573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e001d040 con 0x561bf3fa3b10 2024-04-25T10:10:10.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.387+0000 7f53d4ff9640 1 -- 172.21.15.16:0/1552112573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0026040 con 0x561bf3fa3b10 2024-04-25T10:10:10.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.387+0000 7f53f8c82740 1 -- 172.21.15.16:0/1552112573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3b10 msgr2=0x561bf3f7baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.387+0000 7f53f8c82740 1 --2- 172.21.15.16:0/1552112573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3b10 0x561bf3f7baf0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f53e0016480 tx=0x7f53e0021000 comp rx=0 tx=0).stop 2024-04-25T10:10:10.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.387+0000 7f53f8c82740 1 -- 172.21.15.16:0/1552112573 shutdown_connections 2024-04-25T10:10:10.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.387+0000 7f53f8c82740 1 --2- 172.21.15.16:0/1552112573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3fbddd0 0x561bf3fa21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.387+0000 7f53f8c82740 1 --2- 172.21.15.16:0/1552112573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fa3ef0 0x561bf3f7c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.387+0000 7f53f8c82740 1 --2- 172.21.15.16:0/1552112573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fa3b10 0x561bf3f7baf0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.387+0000 7f53f8c82740 1 -- 172.21.15.16:0/1552112573 >> 172.21.15.16:0/1552112573 conn(0x561bf3f76770 msgr2=0x561bf3fc4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:10.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.387+0000 7f53f8c82740 1 -- 172.21.15.16:0/1552112573 shutdown_connections 2024-04-25T10:10:10.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.387+0000 7f53f8c82740 1 -- 172.21.15.16:0/1552112573 wait complete. 2024-04-25T10:10:10.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.388+0000 7f53f8c82740 1 -- start start 2024-04-25T10:10:10.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.388+0000 7f53f8c82740 1 Processor -- start 2024-04-25T10:10:10.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53f8c82740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3fa3b10 0x561bf402f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53f8c82740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fa3ef0 0x561bf3f7ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53f8c82740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fbddd0 0x561bf3f7bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53f8c82740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf3fc3eb0 con 0x561bf3fbddd0 2024-04-25T10:10:10.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53f8c82740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f01fcd40 con 0x561bf3fa3ef0 2024-04-25T10:10:10.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53f8c82740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x561bf3f797b0 con 0x561bf3fa3b10 2024-04-25T10:10:10.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53f8c82740 10 client.0 _ll_register_callbacks cb 0x7f53e402a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:10.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fbddd0 0x561bf3f7bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fbddd0 0x561bf3f7bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47966/0 (socket says 172.21.15.16:47966) 2024-04-25T10:10:10.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53effff640 1 -- 172.21.15.16:0/2729441200 learned_addr learned my addr 172.21.15.16:0/2729441200 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:10.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53eeffd640 1 --2- 172.21.15.16:0/2729441200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fa3ef0 0x561bf3f7ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53ef7fe640 1 --2- 172.21.15.16:0/2729441200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3fa3b10 0x561bf402f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53effff640 1 -- 172.21.15.16:0/2729441200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3fa3b10 msgr2=0x561bf402f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53effff640 1 --2- 172.21.15.16:0/2729441200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3fa3b10 0x561bf402f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53effff640 1 -- 172.21.15.16:0/2729441200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fa3ef0 msgr2=0x561bf3f7ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53effff640 1 --2- 172.21.15.16:0/2729441200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fa3ef0 0x561bf3f7ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53effff640 1 -- 172.21.15.16:0/2729441200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bf3fa19d0 con 0x561bf3fbddd0 2024-04-25T10:10:10.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53eeffd640 1 --2- 172.21.15.16:0/2729441200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fa3ef0 0x561bf3f7ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:10.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53effff640 1 --2- 172.21.15.16:0/2729441200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fbddd0 0x561bf3f7bfd0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f53e80012e0 tx=0x7f53e80078b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222b3636fd3c773f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:10.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.389+0000 7f53ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:10:10.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.390+0000 7f53ce7fc640 1 -- 172.21.15.16:0/2729441200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8015850 con 0x561bf3fbddd0 2024-04-25T10:10:10.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.390+0000 7f53ce7fc640 1 -- 172.21.15.16:0/2729441200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f53e80159f0 con 0x561bf3fbddd0 2024-04-25T10:10:10.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.390+0000 7f53f8c82740 1 -- 172.21.15.16:0/2729441200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53e001d040 con 0x561bf3fbddd0 2024-04-25T10:10:10.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.390+0000 7f53ce7fc640 1 -- 172.21.15.16:0/2729441200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53e8027050 con 0x561bf3fbddd0 2024-04-25T10:10:10.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.390+0000 7f53f8c82740 10 client.7005 fetch_fsmap learned FSMap version 43 2024-04-25T10:10:10.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.390+0000 7f53f8c82740 1 -- 172.21.15.16:0/2729441200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x561bf3f7a330 con 0x561bf3fbddd0 2024-04-25T10:10:10.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.390+0000 7f53ce7fc640 1 -- 172.21.15.16:0/2729441200 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e001d040 con 0x561bf3fbddd0 2024-04-25T10:10:10.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.390+0000 7f53ce7fc640 1 -- 172.21.15.16:0/2729441200 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f53e802a020 con 0x561bf3fbddd0 2024-04-25T10:10:10.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.390+0000 7f53f8c82740 10 client.7005 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T10:10:10.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.390+0000 7f53f8c82740 10 client.7005 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:10:10.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.390+0000 7f53f8c82740 20 client.7005 populate_metadata read hostname 'smithi016' 2024-04-25T10:10:10.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.390+0000 7f53f8c82740 1 --2- 172.21.15.16:0/2729441200 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x561bf3fc1320 0x561bf3fb71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.390+0000 7f53f8c82740 4 client.7005 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:10:10.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.390+0000 7f53f8c82740 1 -- 172.21.15.16:0/2729441200 --> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561bf3f9bb10 con 0x561bf3fc1320 2024-04-25T10:10:10.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.390+0000 7f53ef7fe640 1 --2- 172.21.15.16:0/2729441200 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x561bf3fc1320 0x561bf3fb71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.391+0000 7f53ce7fc640 10 client.7005 ms_handle_connect on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:10.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.391+0000 7f53ef7fe640 1 --2- 172.21.15.16:0/2729441200 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x561bf3fc1320 0x561bf3fb71d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73a1c93136eb79 server_cookie=4582d7790c1681e0 in_seq=0 out_seq=0 2024-04-25T10:10:10.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.391+0000 7f53ce7fc640 1 -- 172.21.15.16:0/2729441200 <== mds.0 v2:172.21.15.16:6834/1735811292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561bf3f9bb10 con 0x561bf3fc1320 2024-04-25T10:10:10.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.391+0000 7f53ce7fc640 10 client.7005 handle_command_reply: tid=0 2024-04-25T10:10:10.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.391+0000 7f53f8c82740 1 client.7005 shutdown 2024-04-25T10:10:10.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.391+0000 7f53f8c82740 1 -- 172.21.15.16:0/2729441200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fbddd0 msgr2=0x561bf3f7bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.391+0000 7f53f8c82740 1 --2- 172.21.15.16:0/2729441200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fbddd0 0x561bf3f7bfd0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f53e80012e0 tx=0x7f53e80078b0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.391+0000 7f53f8c82740 1 -- 172.21.15.16:0/2729441200 shutdown_connections 2024-04-25T10:10:10.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.391+0000 7f53f8c82740 1 --2- 172.21.15.16:0/2729441200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bf3fbddd0 0x561bf3f7bfd0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.391+0000 7f53f8c82740 1 --2- 172.21.15.16:0/2729441200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561bf3fa3ef0 0x561bf3f7ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.391+0000 7f53f8c82740 1 --2- 172.21.15.16:0/2729441200 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x561bf3fc1320 0x561bf3fb71d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.391+0000 7f53f8c82740 1 --2- 172.21.15.16:0/2729441200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561bf3fa3b10 0x561bf402f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.391+0000 7f53ce7fc640 0 client.7005 ms_handle_reset on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:10.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.391+0000 7f53f8c82740 1 -- 172.21.15.16:0/2729441200 >> 172.21.15.16:0/2729441200 conn(0x561bf3f76770 msgr2=0x561bf3fa21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:10.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.391+0000 7f53f8c82740 1 -- 172.21.15.16:0/2729441200 shutdown_connections 2024-04-25T10:10:10.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.391+0000 7f53f8c82740 1 -- 172.21.15.16:0/2729441200 wait complete. 2024-04-25T10:10:10.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.391+0000 7f53f8c82740 20 client.7005 trim_cache size 0 max 16384 2024-04-25T10:10:10.403 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:10:10.403 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-04-25T10:10:10.404 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-04-25T10:10:10.404 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:10:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.392+0000 7f53f5f7e640 1 -- 172.21.15.16:0/2078275554 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f53dc06b4d0 msgr2=0x7f53dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.392+0000 7f53f5f7e640 1 --2- 172.21.15.16:0/2078275554 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f53dc06b4d0 0x7f53dc06d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f53e00079a0 tx=0x7f53e0008040 comp rx=0 tx=0).stop 2024-04-25T10:10:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.392+0000 7f53f5f7e640 1 -- 172.21.15.16:0/2078275554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f00ce230 msgr2=0x7f53f00b1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.392+0000 7f53f5f7e640 1 --2- 172.21.15.16:0/2078275554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f00ce230 0x7f53f00b1330 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009890 tx=0x7f53e40046c0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.392+0000 7f53f5f7e640 1 -- 172.21.15.16:0/2078275554 shutdown_connections 2024-04-25T10:10:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.392+0000 7f53f5f7e640 1 --2- 172.21.15.16:0/2078275554 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f53dc06b4d0 0x7f53dc06d990 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.392+0000 7f53f5f7e640 1 --2- 172.21.15.16:0/2078275554 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53f015ba70 0x7f53f0157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.392+0000 7f53f5f7e640 1 --2- 172.21.15.16:0/2078275554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f00ce230 0x7f53f00b1330 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.392+0000 7f53f5f7e640 1 --2- 172.21.15.16:0/2078275554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53f00b7030 0x7f53f01f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.392+0000 7f53f5f7e640 1 -- 172.21.15.16:0/2078275554 >> 172.21.15.16:0/2078275554 conn(0x7f53f006f4a0 msgr2=0x7f53f006f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.392+0000 7f53f5f7e640 1 -- 172.21.15.16:0/2078275554 shutdown_connections 2024-04-25T10:10:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.392+0000 7f53f5f7e640 1 -- 172.21.15.16:0/2078275554 wait complete. 2024-04-25T10:10:10.408 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:10:10.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:10.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.503+0000 7f015b534640 1 Processor -- start 2024-04-25T10:10:10.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.504+0000 7f015b534640 1 -- start start 2024-04-25T10:10:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.504+0000 7f015b534640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f01540ae9a0 0x7f01540b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.504+0000 7f015b534640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f01540b2c60 0x7f01540b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.504+0000 7f015b534640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f015406d550 con 0x7f0154071190 2024-04-25T10:10:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.504+0000 7f015b534640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015406d830 con 0x7f01540ae9a0 2024-04-25T10:10:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.504+0000 7f015b534640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015406db10 con 0x7f01540b2c60 2024-04-25T10:10:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.505+0000 7f0158acf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f01540ae9a0 0x7f01540b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.505+0000 7f0158acf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f01540ae9a0 0x7f01540b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47968/0 (socket says 172.21.15.16:47968) 2024-04-25T10:10:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.505+0000 7f0158acf640 1 -- 172.21.15.16:0/629773106 learned_addr learned my addr 172.21.15.16:0/629773106 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.505+0000 7f0159ad1640 1 --2- 172.21.15.16:0/629773106 >> v2:172.21.15.186:3300/0 conn(0x7f01540b2c60 0x7f01540b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.505+0000 7f0158acf640 1 -- 172.21.15.16:0/629773106 >> v2:172.21.15.186:3300/0 conn(0x7f01540b2c60 msgr2=0x7f01540b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.505+0000 7f0158acf640 1 --2- 172.21.15.16:0/629773106 >> v2:172.21.15.186:3300/0 conn(0x7f01540b2c60 0x7f01540b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.505+0000 7f0158acf640 1 -- 172.21.15.16:0/629773106 >> v1:172.21.15.186:6789/0 conn(0x7f0154071190 legacy=0x7f0154071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.505+0000 7f0158acf640 1 -- 172.21.15.16:0/629773106 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015406ddf0 con 0x7f01540ae9a0 2024-04-25T10:10:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.505+0000 7f0159ad1640 1 --2- 172.21.15.16:0/629773106 >> v2:172.21.15.186:3300/0 conn(0x7f01540b2c60 0x7f01540b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.506+0000 7f0158acf640 1 --2- 172.21.15.16:0/629773106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01540ae9a0 0x7f01540b2720 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f0144009870 tx=0x7f014402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6864d5d1c693c3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.506+0000 7f014bfff640 1 -- 172.21.15.16:0/629773106 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144037070 con 0x7f01540ae9a0 2024-04-25T10:10:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.506+0000 7f014bfff640 1 -- 172.21.15.16:0/629773106 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0144032440 con 0x7f01540ae9a0 2024-04-25T10:10:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.506+0000 7f014bfff640 1 -- 172.21.15.16:0/629773106 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144032810 con 0x7f01540ae9a0 2024-04-25T10:10:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.506+0000 7f015b534640 1 -- 172.21.15.16:0/629773106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01540ae9a0 msgr2=0x7f01540b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.506+0000 7f015b534640 1 --2- 172.21.15.16:0/629773106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01540ae9a0 0x7f01540b2720 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f0144009870 tx=0x7f014402a270 comp rx=0 tx=0).stop 2024-04-25T10:10:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.506+0000 7f015b534640 1 -- 172.21.15.16:0/629773106 shutdown_connections 2024-04-25T10:10:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.506+0000 7f015b534640 1 --2- 172.21.15.16:0/629773106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01540ae9a0 0x7f01540b2720 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.506+0000 7f015b534640 1 --2- 172.21.15.16:0/629773106 >> v2:172.21.15.186:3300/0 conn(0x7f01540b2c60 0x7f01540b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.506+0000 7f015b534640 1 -- 172.21.15.16:0/629773106 >> 172.21.15.16:0/629773106 conn(0x7f015406aba0 msgr2=0x7f015406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.507+0000 7f015b534640 1 -- 172.21.15.16:0/629773106 shutdown_connections 2024-04-25T10:10:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.507+0000 7f015b534640 1 -- 172.21.15.16:0/629773106 wait complete. 2024-04-25T10:10:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.508+0000 7f015b534640 1 Processor -- start 2024-04-25T10:10:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.508+0000 7f015b534640 1 -- start start 2024-04-25T10:10:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.508+0000 7f015b534640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0154071190 0x7f0154139080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.508+0000 7f015b534640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01540ae9a0 0x7f015414cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.508+0000 7f015b534640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01540b2c60 0x7f0154149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.508+0000 7f015b534640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015406e8b0 con 0x7f01540ae9a0 2024-04-25T10:10:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.508+0000 7f015b534640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01540aee30 con 0x7f0154071190 2024-04-25T10:10:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.508+0000 7f015b534640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015406d1b0 con 0x7f01540b2c60 2024-04-25T10:10:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.508+0000 7f0158acf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01540ae9a0 0x7f015414cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.508+0000 7f0158acf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01540ae9a0 0x7f015414cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47974/0 (socket says 172.21.15.16:47974) 2024-04-25T10:10:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.508+0000 7f0158acf640 1 -- 172.21.15.16:0/4060612853 learned_addr learned my addr 172.21.15.16:0/4060612853 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.509+0000 7f0158acf640 1 -- 172.21.15.16:0/4060612853 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01540b2c60 msgr2=0x7f0154149070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.509+0000 7f0158acf640 1 --2- 172.21.15.16:0/4060612853 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01540b2c60 0x7f0154149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.509+0000 7f0158acf640 1 -- 172.21.15.16:0/4060612853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0154071190 msgr2=0x7f0154139080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.509+0000 7f0158acf640 1 --2- 172.21.15.16:0/4060612853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0154071190 0x7f0154139080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.509+0000 7f0158acf640 1 -- 172.21.15.16:0/4060612853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015414d400 con 0x7f01540ae9a0 2024-04-25T10:10:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.509+0000 7f0158acf640 1 --2- 172.21.15.16:0/4060612853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01540ae9a0 0x7f015414cec0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0144009870 tx=0x7f01440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.509+0000 7f0149ffb640 1 -- 172.21.15.16:0/4060612853 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144037040 con 0x7f01540ae9a0 2024-04-25T10:10:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.509+0000 7f0149ffb640 1 -- 172.21.15.16:0/4060612853 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0144004230 con 0x7f01540ae9a0 2024-04-25T10:10:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.509+0000 7f015b534640 1 -- 172.21.15.16:0/4060612853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0154147fd0 con 0x7f01540ae9a0 2024-04-25T10:10:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.509+0000 7f0149ffb640 1 -- 172.21.15.16:0/4060612853 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014403c730 con 0x7f01540ae9a0 2024-04-25T10:10:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.509+0000 7f015b534640 1 -- 172.21.15.16:0/4060612853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0154148590 con 0x7f01540ae9a0 2024-04-25T10:10:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.510+0000 7f015b534640 1 -- 172.21.15.16:0/4060612853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01540749b0 con 0x7f01540ae9a0 2024-04-25T10:10:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.511+0000 7f0149ffb640 1 -- 172.21.15.16:0/4060612853 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f014403a070 con 0x7f01540ae9a0 2024-04-25T10:10:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.511+0000 7f0149ffb640 1 --2- 172.21.15.16:0/4060612853 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f013806b4d0 0x7f013806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.511+0000 7f0149ffb640 1 -- 172.21.15.16:0/4060612853 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f01440aacc0 con 0x7f01540ae9a0 2024-04-25T10:10:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.512+0000 7f01592d0640 1 --2- 172.21.15.16:0/4060612853 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f013806b4d0 0x7f013806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.512+0000 7f01592d0640 1 --2- 172.21.15.16:0/4060612853 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f013806b4d0 0x7f013806d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f014c003cc0 tx=0x7f014c0079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:10.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.516+0000 7f0149ffb640 1 -- 172.21.15.16:0/4060612853 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f01440b1080 con 0x7f01540ae9a0 2024-04-25T10:10:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.734+0000 7f015b534640 1 -- 172.21.15.16:0/4060612853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01540cbdc0 con 0x7f01540ae9a0 2024-04-25T10:10:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.735+0000 7f0149ffb640 1 -- 172.21.15.16:0/4060612853 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f0144041020 con 0x7f01540ae9a0 2024-04-25T10:10:10.740 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:10.741 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6070,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2805428272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2805428272},{"type":"v1","addr":"172.21.15.186:6835","nonce":2805428272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1052667029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1052667029},{"type":"v1","addr":"172.21.15.186:6837","nonce":1052667029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2897154125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2897154125},{"type":"v1","addr":"172.21.15.16:6833","nonce":2897154125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:22.077032+0000","modified":"2024-04-25T10:09:46.389046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6636":{"gid":6636,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":10,"addr":"172.21.15.16:6835/1735811292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1735811292},{"type":"v1","addr":"172.21.15.16:6835","nonce":1735811292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:10:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 43 2024-04-25T10:10:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.737+0000 7f015b534640 1 -- 172.21.15.16:0/4060612853 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f013806b4d0 msgr2=0x7f013806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.737+0000 7f015b534640 1 --2- 172.21.15.16:0/4060612853 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f013806b4d0 0x7f013806d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f014c003cc0 tx=0x7f014c0079b0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.737+0000 7f015b534640 1 -- 172.21.15.16:0/4060612853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01540ae9a0 msgr2=0x7f015414cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.737+0000 7f015b534640 1 --2- 172.21.15.16:0/4060612853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01540ae9a0 0x7f015414cec0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0144009870 tx=0x7f01440046c0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.737+0000 7f015b534640 1 -- 172.21.15.16:0/4060612853 shutdown_connections 2024-04-25T10:10:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.737+0000 7f015b534640 1 --2- 172.21.15.16:0/4060612853 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f013806b4d0 0x7f013806d990 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.737+0000 7f015b534640 1 --2- 172.21.15.16:0/4060612853 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01540b2c60 0x7f0154149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.737+0000 7f015b534640 1 --2- 172.21.15.16:0/4060612853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01540ae9a0 0x7f015414cec0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.737+0000 7f015b534640 1 --2- 172.21.15.16:0/4060612853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0154071190 0x7f0154139080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.737+0000 7f015b534640 1 -- 172.21.15.16:0/4060612853 >> 172.21.15.16:0/4060612853 conn(0x7f015406aba0 msgr2=0x7f0154159550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.737+0000 7f015b534640 1 -- 172.21.15.16:0/4060612853 shutdown_connections 2024-04-25T10:10:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.737+0000 7f015b534640 1 -- 172.21.15.16:0/4060612853 wait complete. 2024-04-25T10:10:10.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:10:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.851+0000 7fc1093e7640 1 Processor -- start 2024-04-25T10:10:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.851+0000 7fc1093e7640 1 -- start start 2024-04-25T10:10:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.852+0000 7fc1093e7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc10415b7b0 0x7fc1040ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.852+0000 7fc1093e7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10415f000 con 0x7fc10415e820 2024-04-25T10:10:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.852+0000 7fc1093e7640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10415ed20 con 0x7fc1040cf760 2024-04-25T10:10:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.852+0000 7fc1093e7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1040ced00 con 0x7fc10415b7b0 2024-04-25T10:10:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.852+0000 7fc1037fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc10415e820 0x7fc1040cc080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58338/0 (socket says 172.21.15.16:58338) 2024-04-25T10:10:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.852+0000 7fc1037fe640 1 -- 172.21.15.16:0/2561980937 learned_addr learned my addr 172.21.15.16:0/2561980937 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.852+0000 7fc102ffd640 1 --2- 172.21.15.16:0/2561980937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc10415b7b0 0x7fc1040ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.852+0000 7fc1027fc640 1 -- 172.21.15.16:0/2561980937 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064470156 0 0) 0x7fc10415f000 con 0x7fc10415e820 2024-04-25T10:10:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.852+0000 7fc1027fc640 1 -- 172.21.15.16:0/2561980937 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0f0003520 con 0x7fc10415e820 2024-04-25T10:10:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.852+0000 7fc1027fc640 1 -- 172.21.15.16:0/2561980937 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091811639 0 0) 0x7fc10415ed20 con 0x7fc1040cf760 2024-04-25T10:10:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.852+0000 7fc1027fc640 1 -- 172.21.15.16:0/2561980937 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc10415f000 con 0x7fc1040cf760 2024-04-25T10:10:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.852+0000 7fc1027fc640 1 -- 172.21.15.16:0/2561980937 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1509062847 0 0) 0x7fc0f0003520 con 0x7fc10415e820 2024-04-25T10:10:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.852+0000 7fc1027fc640 1 -- 172.21.15.16:0/2561980937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc10415b7b0 msgr2=0x7fc1040ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.852+0000 7fc1027fc640 1 --2- 172.21.15.16:0/2561980937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc10415b7b0 0x7fc1040ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.852+0000 7fc1027fc640 1 -- 172.21.15.16:0/2561980937 >> v1:172.21.15.186:6789/0 conn(0x7fc1040cf760 legacy=0x7fc10415e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.852+0000 7fc1027fc640 1 -- 172.21.15.16:0/2561980937 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1040cdce0 con 0x7fc10415e820 2024-04-25T10:10:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.852+0000 7fc1027fc640 1 -- 172.21.15.16:0/2561980937 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fc0fc0019f0 con 0x7fc10415e820 2024-04-25T10:10:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.853+0000 7fc1027fc640 1 -- 172.21.15.16:0/2561980937 >> v1:172.21.15.16:6789/0 conn(0x7fc10415e820 legacy=0x7fc1040cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.853+0000 7fc1027fc640 1 --2- 172.21.15.16:0/2561980937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f0003b70 0x7fc0f0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.853+0000 7fc1027fc640 1 --2- 172.21.15.16:0/2561980937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc10415b7b0 0x7fc0f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.853+0000 7fc1027fc640 1 --2- 172.21.15.16:0/2561980937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0f0004d40 0x7fc0f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.853+0000 7fc1027fc640 1 -- 172.21.15.16:0/2561980937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1040ced00 con 0x7fc0f0003b70 2024-04-25T10:10:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.853+0000 7fc1027fc640 1 -- 172.21.15.16:0/2561980937 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f000ba70 con 0x7fc0f0004d40 2024-04-25T10:10:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.853+0000 7fc1027fc640 1 -- 172.21.15.16:0/2561980937 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f000bc70 con 0x7fc10415b7b0 2024-04-25T10:10:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.853+0000 7fc102ffd640 1 --2- 172.21.15.16:0/2561980937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f0003b70 0x7fc0f0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.853+0000 7fc1037fe640 1 --2- 172.21.15.16:0/2561980937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0f0004d40 0x7fc0f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.853+0000 7fc0fbfff640 1 --2- 172.21.15.16:0/2561980937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc10415b7b0 0x7fc0f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.853+0000 7fc102ffd640 1 -- 172.21.15.16:0/2561980937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc10415b7b0 msgr2=0x7fc0f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.853+0000 7fc102ffd640 1 --2- 172.21.15.16:0/2561980937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc10415b7b0 0x7fc0f0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.853+0000 7fc102ffd640 1 -- 172.21.15.16:0/2561980937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0f0004d40 msgr2=0x7fc0f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.853+0000 7fc102ffd640 1 --2- 172.21.15.16:0/2561980937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0f0004d40 0x7fc0f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.853+0000 7fc102ffd640 1 -- 172.21.15.16:0/2561980937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f0003520 con 0x7fc0f0003b70 2024-04-25T10:10:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.853+0000 7fc0fbfff640 1 --2- 172.21.15.16:0/2561980937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc10415b7b0 0x7fc0f0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.854+0000 7fc102ffd640 1 --2- 172.21.15.16:0/2561980937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f0003b70 0x7fc0f0003f70 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009ac0 tx=0x7fc0f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84c86e58f29ccc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.854+0000 7fc1027fc640 1 -- 172.21.15.16:0/2561980937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4036070 con 0x7fc0f0003b70 2024-04-25T10:10:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.854+0000 7fc1027fc640 1 -- 172.21.15.16:0/2561980937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f4031400 con 0x7fc0f0003b70 2024-04-25T10:10:10.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.854+0000 7fc1027fc640 1 -- 172.21.15.16:0/2561980937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4009690 con 0x7fc0f0003b70 2024-04-25T10:10:10.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.854+0000 7fc1093e7640 1 -- 172.21.15.16:0/2561980937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f0003b70 msgr2=0x7fc0f0003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.854+0000 7fc1093e7640 1 --2- 172.21.15.16:0/2561980937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f0003b70 0x7fc0f0003f70 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009ac0 tx=0x7fc0f0005830 comp rx=0 tx=0).stop 2024-04-25T10:10:10.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.854+0000 7fc1037fe640 1 -- 172.21.15.16:0/2561980937 reap_dead start 2024-04-25T10:10:10.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.854+0000 7fc1093e7640 1 -- 172.21.15.16:0/2561980937 shutdown_connections 2024-04-25T10:10:10.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.854+0000 7fc1093e7640 1 -- 172.21.15.16:0/2561980937 >> 172.21.15.16:0/2561980937 conn(0x7fc10406f4a0 msgr2=0x7fc10406f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:10.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.854+0000 7fc1093e7640 1 -- 172.21.15.16:0/2561980937 shutdown_connections 2024-04-25T10:10:10.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.854+0000 7fc1093e7640 1 -- 172.21.15.16:0/2561980937 wait complete. 2024-04-25T10:10:10.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.856+0000 7fc1093e7640 1 Processor -- start 2024-04-25T10:10:10.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.856+0000 7fc1093e7640 1 -- start start 2024-04-25T10:10:10.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.856+0000 7fc1093e7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0f0003b70 0x7fc1040b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.856+0000 7fc1093e7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1040b60b0 0x7fc1040cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.856+0000 7fc1093e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1041f3cb0 0x7fc1040cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.856+0000 7fc1093e7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1041619f0 con 0x7fc1041f3cb0 2024-04-25T10:10:10.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.856+0000 7fc1093e7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1040ce330 con 0x7fc1040b60b0 2024-04-25T10:10:10.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.856+0000 7fc1093e7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1040ce7c0 con 0x7fc0f0003b70 2024-04-25T10:10:10.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.856+0000 7fc1037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1041f3cb0 0x7fc1040cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.856+0000 7fc1037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1041f3cb0 0x7fc1040cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47996/0 (socket says 172.21.15.16:47996) 2024-04-25T10:10:10.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.857+0000 7fc1037fe640 1 -- 172.21.15.16:0/3242333182 learned_addr learned my addr 172.21.15.16:0/3242333182 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:10.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.857+0000 7fc0fbfff640 1 --2- 172.21.15.16:0/3242333182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1040b60b0 0x7fc1040cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.857+0000 7fc102ffd640 1 --2- 172.21.15.16:0/3242333182 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0f0003b70 0x7fc1040b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.857+0000 7fc1037fe640 1 -- 172.21.15.16:0/3242333182 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0f0003b70 msgr2=0x7fc1040b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.857+0000 7fc1037fe640 1 --2- 172.21.15.16:0/3242333182 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0f0003b70 0x7fc1040b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.857+0000 7fc1037fe640 1 -- 172.21.15.16:0/3242333182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1040b60b0 msgr2=0x7fc1040cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.857+0000 7fc1037fe640 1 --2- 172.21.15.16:0/3242333182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1040b60b0 0x7fc1040cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.857+0000 7fc1037fe640 1 -- 172.21.15.16:0/3242333182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1040b1480 con 0x7fc1041f3cb0 2024-04-25T10:10:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.857+0000 7fc102ffd640 1 --2- 172.21.15.16:0/3242333182 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0f0003b70 0x7fc1040b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.857+0000 7fc1037fe640 1 --2- 172.21.15.16:0/3242333182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1041f3cb0 0x7fc1040cb250 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc001560 tx=0x7fc0fc007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.857+0000 7fc0fb7fe640 1 -- 172.21.15.16:0/3242333182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc017020 con 0x7fc1041f3cb0 2024-04-25T10:10:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.857+0000 7fc0fb7fe640 1 -- 172.21.15.16:0/3242333182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc00ab10 con 0x7fc1041f3cb0 2024-04-25T10:10:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.857+0000 7fc1093e7640 1 -- 172.21.15.16:0/3242333182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1040b1720 con 0x7fc1041f3cb0 2024-04-25T10:10:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.857+0000 7fc0fb7fe640 1 -- 172.21.15.16:0/3242333182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc0133f0 con 0x7fc1041f3cb0 2024-04-25T10:10:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.857+0000 7fc1093e7640 1 -- 172.21.15.16:0/3242333182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1040b1ce0 con 0x7fc1041f3cb0 2024-04-25T10:10:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.859+0000 7fc0fb7fe640 1 -- 172.21.15.16:0/3242333182 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc0fc01a020 con 0x7fc1041f3cb0 2024-04-25T10:10:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.859+0000 7fc0fb7fe640 1 --2- 172.21.15.16:0/3242333182 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc0e406b500 0x7fc0e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.859+0000 7fc0fb7fe640 1 -- 172.21.15.16:0/3242333182 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fc0fc08a890 con 0x7fc1041f3cb0 2024-04-25T10:10:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.859+0000 7fc102ffd640 1 --2- 172.21.15.16:0/3242333182 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc0e406b500 0x7fc0e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.860+0000 7fc10c1c8740 1 Processor -- start 2024-04-25T10:10:10.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.860+0000 7fc102ffd640 1 --2- 172.21.15.16:0/3242333182 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc0e406b500 0x7fc0e406d9c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4033000 tx=0x7fc0f4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:10.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.860+0000 7fc10c1c8740 1 -- start start 2024-04-25T10:10:10.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.861+0000 7fc10c1c8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa00590 0x560c1fa026f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.861+0000 7fc10c1c8740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa3cc60 0x560c1fa00970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.861+0000 7fc10c1c8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa2b050 0x560c1fa2b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.861+0000 7fc10c1c8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1f963c60 con 0x560c1fa2b050 2024-04-25T10:10:10.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.861+0000 7fc10c1c8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1f902d80 con 0x560c1fa00590 2024-04-25T10:10:10.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.861+0000 7fc10c1c8740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c1f95fd00 con 0x560c1fa3cc60 2024-04-25T10:10:10.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.861+0000 7fc1037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa2b050 0x560c1fa2b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.861+0000 7fc1037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa2b050 0x560c1fa2b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48010/0 (socket says 172.21.15.16:48010) 2024-04-25T10:10:10.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.861+0000 7fc1037fe640 1 -- 172.21.15.16:0/750905194 learned_addr learned my addr 172.21.15.16:0/750905194 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:10.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.861+0000 7fc102ffd640 1 --2- 172.21.15.16:0/750905194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa00590 0x560c1fa026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.861+0000 7fc0fbfff640 1 --2- 172.21.15.16:0/750905194 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa3cc60 0x560c1fa00970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.861+0000 7fc1037fe640 1 -- 172.21.15.16:0/750905194 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa3cc60 msgr2=0x560c1fa00970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.861+0000 7fc1037fe640 1 --2- 172.21.15.16:0/750905194 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa3cc60 0x560c1fa00970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.861+0000 7fc1037fe640 1 -- 172.21.15.16:0/750905194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa00590 msgr2=0x560c1fa026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.861+0000 7fc1037fe640 1 --2- 172.21.15.16:0/750905194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa00590 0x560c1fa026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.861+0000 7fc1037fe640 1 -- 172.21.15.16:0/750905194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c1f7447c0 con 0x560c1fa2b050 2024-04-25T10:10:10.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.861+0000 7fc1037fe640 1 --2- 172.21.15.16:0/750905194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa2b050 0x560c1fa2b4d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc088c70 tx=0x7fc0fc058f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ee11d4d1f4969a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:10.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.862+0000 7fc0f8ff9640 1 -- 172.21.15.16:0/750905194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc087070 con 0x560c1fa2b050 2024-04-25T10:10:10.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.862+0000 7fc0f8ff9640 1 -- 172.21.15.16:0/750905194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc061660 con 0x560c1fa2b050 2024-04-25T10:10:10.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.862+0000 7fc0f8ff9640 1 -- 172.21.15.16:0/750905194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc0555c0 con 0x560c1fa2b050 2024-04-25T10:10:10.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.862+0000 7fc10c1c8740 1 -- 172.21.15.16:0/750905194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa2b050 msgr2=0x560c1fa2b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.862+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/750905194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa2b050 0x560c1fa2b4d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc088c70 tx=0x7fc0fc058f50 comp rx=0 tx=0).stop 2024-04-25T10:10:10.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.862+0000 7fc10c1c8740 1 -- 172.21.15.16:0/750905194 shutdown_connections 2024-04-25T10:10:10.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.862+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/750905194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa2b050 0x560c1fa2b4d0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.862+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/750905194 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa3cc60 0x560c1fa00970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.862+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/750905194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa00590 0x560c1fa026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.862+0000 7fc10c1c8740 1 -- 172.21.15.16:0/750905194 >> 172.21.15.16:0/750905194 conn(0x560c1f9c86e0 msgr2=0x560c1fa46050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:10.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.862+0000 7fc10c1c8740 1 -- 172.21.15.16:0/750905194 shutdown_connections 2024-04-25T10:10:10.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.862+0000 7fc10c1c8740 1 -- 172.21.15.16:0/750905194 wait complete. 2024-04-25T10:10:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.863+0000 7fc10c1c8740 1 -- start start 2024-04-25T10:10:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.863+0000 7fc10c1c8740 1 Processor -- start 2024-04-25T10:10:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc10c1c8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa00590 0x560c1fa26db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc10c1c8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa2b050 0x560c1fa56b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc10c1c8740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa3cc60 0x560c1fa41d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc10c1c8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1f900840 con 0x560c1fa00590 2024-04-25T10:10:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc10c1c8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1fa28ba0 con 0x560c1fa2b050 2024-04-25T10:10:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc10c1c8740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c1fa2c680 con 0x560c1fa3cc60 2024-04-25T10:10:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc10c1c8740 10 client.0 _ll_register_callbacks cb 0x560c1f91a980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:10.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc102ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa00590 0x560c1fa26db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc102ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa00590 0x560c1fa26db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48020/0 (socket says 172.21.15.16:48020) 2024-04-25T10:10:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc102ffd640 1 -- 172.21.15.16:0/1538058071 learned_addr learned my addr 172.21.15.16:0/1538058071 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc0fbfff640 1 --2- 172.21.15.16:0/1538058071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa2b050 0x560c1fa56b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc1037fe640 1 --2- 172.21.15.16:0/1538058071 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa3cc60 0x560c1fa41d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc102ffd640 1 -- 172.21.15.16:0/1538058071 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa3cc60 msgr2=0x560c1fa41d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc102ffd640 1 --2- 172.21.15.16:0/1538058071 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa3cc60 0x560c1fa41d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc102ffd640 1 -- 172.21.15.16:0/1538058071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa2b050 msgr2=0x560c1fa56b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc102ffd640 1 --2- 172.21.15.16:0/1538058071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa2b050 0x560c1fa56b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc102ffd640 1 -- 172.21.15.16:0/1538058071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c1f7a4f50 con 0x560c1fa00590 2024-04-25T10:10:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc102ffd640 1 --2- 172.21.15.16:0/1538058071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa00590 0x560c1fa26db0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fc0f402abb0 tx=0x7fc0f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d873d2bc5654ce81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc0e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:10:10.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc0e1ffb640 1 -- 172.21.15.16:0/1538058071 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4050060 con 0x560c1fa00590 2024-04-25T10:10:10.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc0e1ffb640 1 -- 172.21.15.16:0/1538058071 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fc0f4003cf0 con 0x560c1fa00590 2024-04-25T10:10:10.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.864+0000 7fc10c1c8740 1 -- 172.21.15.16:0/1538058071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0fc061660 con 0x560c1fa00590 2024-04-25T10:10:10.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.865+0000 7fc0e1ffb640 1 -- 172.21.15.16:0/1538058071 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0f4036050 con 0x560c1fa00590 2024-04-25T10:10:10.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.865+0000 7fc10c1c8740 10 client.7023 fetch_fsmap learned FSMap version 43 2024-04-25T10:10:10.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.865+0000 7fc10c1c8740 1 -- 172.21.15.16:0/1538058071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x560c1f6380b0 con 0x560c1fa00590 2024-04-25T10:10:10.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.865+0000 7fc0e1ffb640 1 -- 172.21.15.16:0/1538058071 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc061660 con 0x560c1fa00590 2024-04-25T10:10:10.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.865+0000 7fc0e1ffb640 1 -- 172.21.15.16:0/1538058071 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc0f4059080 con 0x560c1fa00590 2024-04-25T10:10:10.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.865+0000 7fc10c1c8740 10 client.7023 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T10:10:10.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.865+0000 7fc10c1c8740 10 client.7023 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:10:10.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.865+0000 7fc10c1c8740 20 client.7023 populate_metadata read hostname 'smithi016' 2024-04-25T10:10:10.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.865+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/1538058071 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x560c1fa619d0 0x560c1fa63dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.865+0000 7fc10c1c8740 4 client.7023 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:10.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.865+0000 7fc10c1c8740 1 -- 172.21.15.16:0/1538058071 --> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c1f923e60 con 0x560c1fa619d0 2024-04-25T10:10:10.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.865+0000 7fc0fbfff640 1 --2- 172.21.15.16:0/1538058071 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x560c1fa619d0 0x560c1fa63dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.865+0000 7fc0e1ffb640 10 client.7023 ms_handle_connect on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:10.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.865+0000 7fc0fbfff640 1 --2- 172.21.15.16:0/1538058071 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x560c1fa619d0 0x560c1fa63dc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daf1f4a51a24d96a server_cookie=f2222bc5fa4ac3bd in_seq=0 out_seq=0 2024-04-25T10:10:10.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.866+0000 7fc0e1ffb640 1 -- 172.21.15.16:0/1538058071 <== mds.0 v2:172.21.15.16:6834/1735811292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560c1f923e60 con 0x560c1fa619d0 2024-04-25T10:10:10.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.866+0000 7fc0e1ffb640 10 client.7023 handle_command_reply: tid=0 2024-04-25T10:10:10.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.866+0000 7fc10c1c8740 1 client.7023 shutdown 2024-04-25T10:10:10.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.866+0000 7fc10c1c8740 1 -- 172.21.15.16:0/1538058071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa00590 msgr2=0x560c1fa26db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.866+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/1538058071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa00590 0x560c1fa26db0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fc0f402abb0 tx=0x7fc0f40046c0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.866+0000 7fc10c1c8740 1 -- 172.21.15.16:0/1538058071 shutdown_connections 2024-04-25T10:10:10.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.866+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/1538058071 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x560c1fa619d0 0x560c1fa63dc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.866+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/1538058071 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa3cc60 0x560c1fa41d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.866+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/1538058071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa2b050 0x560c1fa56b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.866+0000 7fc0e1ffb640 0 client.7023 ms_handle_reset on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:10.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.866+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/1538058071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa00590 0x560c1fa26db0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.866+0000 7fc10c1c8740 1 -- 172.21.15.16:0/1538058071 >> 172.21.15.16:0/1538058071 conn(0x560c1f9c86e0 msgr2=0x560c1fa3f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:10.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.866+0000 7fc10c1c8740 1 -- 172.21.15.16:0/1538058071 shutdown_connections 2024-04-25T10:10:10.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.866+0000 7fc10c1c8740 1 -- 172.21.15.16:0/1538058071 wait complete. 2024-04-25T10:10:10.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.866+0000 7fc10c1c8740 20 client.7023 trim_cache size 0 max 16384 2024-04-25T10:10:10.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.878+0000 7fc10c1c8740 1 Processor -- start 2024-04-25T10:10:10.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.878+0000 7fc10c1c8740 1 -- start start 2024-04-25T10:10:10.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc10c1c8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa00590 0x560c1fa561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc10c1c8740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa2b050 0x560c1fa279f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc10c1c8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa3cc60 0x560c1fa431f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc10c1c8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1041f79a0 con 0x560c1fa3cc60 2024-04-25T10:10:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc10c1c8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1040b4d80 con 0x560c1fa00590 2024-04-25T10:10:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc10c1c8740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1041f73a0 con 0x560c1fa2b050 2024-04-25T10:10:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc1037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa3cc60 0x560c1fa431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc1037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa3cc60 0x560c1fa431f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48026/0 (socket says 172.21.15.16:48026) 2024-04-25T10:10:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc1037fe640 1 -- 172.21.15.16:0/7277912 learned_addr learned my addr 172.21.15.16:0/7277912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc102ffd640 1 --2- 172.21.15.16:0/7277912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa00590 0x560c1fa561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc0fbfff640 1 --2- 172.21.15.16:0/7277912 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa2b050 0x560c1fa279f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc1037fe640 1 -- 172.21.15.16:0/7277912 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa2b050 msgr2=0x560c1fa279f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc1037fe640 1 --2- 172.21.15.16:0/7277912 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa2b050 0x560c1fa279f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc1037fe640 1 -- 172.21.15.16:0/7277912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa00590 msgr2=0x560c1fa561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc1037fe640 1 --2- 172.21.15.16:0/7277912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa00590 0x560c1fa561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc1037fe640 1 -- 172.21.15.16:0/7277912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c1f9fe8f0 con 0x560c1fa3cc60 2024-04-25T10:10:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc102ffd640 1 --2- 172.21.15.16:0/7277912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa00590 0x560c1fa561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc1037fe640 1 --2- 172.21.15.16:0/7277912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa3cc60 0x560c1fa431f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc05d260 tx=0x7fc0fc01e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00555d99c7bdf1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc0e3fff640 1 -- 172.21.15.16:0/7277912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc087040 con 0x560c1fa3cc60 2024-04-25T10:10:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc0e3fff640 1 -- 172.21.15.16:0/7277912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc074070 con 0x560c1fa3cc60 2024-04-25T10:10:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.879+0000 7fc0e3fff640 1 -- 172.21.15.16:0/7277912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc0555c0 con 0x560c1fa3cc60 2024-04-25T10:10:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.880+0000 7fc10c1c8740 1 -- 172.21.15.16:0/7277912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa3cc60 msgr2=0x560c1fa431f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.880+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/7277912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa3cc60 0x560c1fa431f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc05d260 tx=0x7fc0fc01e030 comp rx=0 tx=0).stop 2024-04-25T10:10:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.880+0000 7fc10c1c8740 1 -- 172.21.15.16:0/7277912 shutdown_connections 2024-04-25T10:10:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.880+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/7277912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa3cc60 0x560c1fa431f0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.880+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/7277912 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa2b050 0x560c1fa279f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.880+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/7277912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa00590 0x560c1fa561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.880+0000 7fc10c1c8740 1 -- 172.21.15.16:0/7277912 >> 172.21.15.16:0/7277912 conn(0x560c1f9c86e0 msgr2=0x560c1fa58ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.880+0000 7fc10c1c8740 1 -- 172.21.15.16:0/7277912 shutdown_connections 2024-04-25T10:10:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.880+0000 7fc10c1c8740 1 -- 172.21.15.16:0/7277912 wait complete. 2024-04-25T10:10:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.881+0000 7fc10c1c8740 1 -- start start 2024-04-25T10:10:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.881+0000 7fc10c1c8740 1 Processor -- start 2024-04-25T10:10:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.881+0000 7fc10c1c8740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa00590 0x560c1faa71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.881+0000 7fc10c1c8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa2b050 0x560c1fa45200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.881+0000 7fc10c1c8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa3cc60 0x560c1fab3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.881+0000 7fc10c1c8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1fa2dea0 con 0x560c1fa2b050 2024-04-25T10:10:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.881+0000 7fc10c1c8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1fa412e0 con 0x560c1fa3cc60 2024-04-25T10:10:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.881+0000 7fc10c1c8740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c1fa2e3e0 con 0x560c1fa00590 2024-04-25T10:10:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.881+0000 7fc10c1c8740 10 client.0 _ll_register_callbacks cb 0x7fc0f00036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.881+0000 7fc0fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa2b050 0x560c1fa45200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.882+0000 7fc0fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa2b050 0x560c1fa45200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48036/0 (socket says 172.21.15.16:48036) 2024-04-25T10:10:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.882+0000 7fc0fbfff640 1 -- 172.21.15.16:0/2617348587 learned_addr learned my addr 172.21.15.16:0/2617348587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.882+0000 7fc1037fe640 1 --2- 172.21.15.16:0/2617348587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa3cc60 0x560c1fab3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.882+0000 7fc102ffd640 1 --2- 172.21.15.16:0/2617348587 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa00590 0x560c1faa71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.882+0000 7fc0fbfff640 1 -- 172.21.15.16:0/2617348587 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa00590 msgr2=0x560c1faa71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.882+0000 7fc0fbfff640 1 --2- 172.21.15.16:0/2617348587 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa00590 0x560c1faa71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.882+0000 7fc0fbfff640 1 -- 172.21.15.16:0/2617348587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa3cc60 msgr2=0x560c1fab3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.882+0000 7fc0fbfff640 1 --2- 172.21.15.16:0/2617348587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa3cc60 0x560c1fab3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.882+0000 7fc0fbfff640 1 -- 172.21.15.16:0/2617348587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c1fa3ca50 con 0x560c1fa2b050 2024-04-25T10:10:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.882+0000 7fc0fbfff640 1 --2- 172.21.15.16:0/2617348587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa2b050 0x560c1fa45200 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fc1040b7470 tx=0x7fc0ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d196044e5a654e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.882+0000 7fc0e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:10:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.882+0000 7fc0e1ffb640 1 -- 172.21.15.16:0/2617348587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec007260 con 0x560c1fa2b050 2024-04-25T10:10:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.882+0000 7fc0e1ffb640 1 -- 172.21.15.16:0/2617348587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fc0ec007400 con 0x560c1fa2b050 2024-04-25T10:10:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.882+0000 7fc10c1c8740 1 -- 172.21.15.16:0/2617348587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0fc074070 con 0x560c1fa2b050 2024-04-25T10:10:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.882+0000 7fc0e1ffb640 1 -- 172.21.15.16:0/2617348587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0ec02d050 con 0x560c1fa2b050 2024-04-25T10:10:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.882+0000 7fc10c1c8740 10 client.7029 fetch_fsmap learned FSMap version 43 2024-04-25T10:10:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.882+0000 7fc10c1c8740 1 -- 172.21.15.16:0/2617348587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x560c1fa68b10 con 0x560c1fa2b050 2024-04-25T10:10:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.882+0000 7fc0e1ffb640 1 -- 172.21.15.16:0/2617348587 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc074070 con 0x560c1fa2b050 2024-04-25T10:10:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc0e1ffb640 1 -- 172.21.15.16:0/2617348587 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc0ec030020 con 0x560c1fa2b050 2024-04-25T10:10:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc10c1c8740 10 client.7029 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T10:10:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc10c1c8740 10 client.7029 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:10:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc10c1c8740 20 client.7029 populate_metadata read hostname 'smithi016' 2024-04-25T10:10:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/2617348587 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x560c1fab44a0 0x560c1fab07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc10c1c8740 4 client.7029 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:10:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc10c1c8740 1 -- 172.21.15.16:0/2617348587 --> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x560c1f9711a0 con 0x560c1fab44a0 2024-04-25T10:10:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc102ffd640 1 --2- 172.21.15.16:0/2617348587 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x560c1fab44a0 0x560c1fab07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc0e1ffb640 10 client.7029 ms_handle_connect on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc102ffd640 1 --2- 172.21.15.16:0/2617348587 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x560c1fab44a0 0x560c1fab07e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5e35ba58ba189d0 server_cookie=89d16e2801eaa879 in_seq=0 out_seq=0 2024-04-25T10:10:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc0e1ffb640 1 -- 172.21.15.16:0/2617348587 <== mds.0 v2:172.21.15.16:6834/1735811292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+236 (crc 0 0 0) 0x560c1f9711a0 con 0x560c1fab44a0 2024-04-25T10:10:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc0e1ffb640 10 client.7029 handle_command_reply: tid=0 2024-04-25T10:10:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc10c1c8740 1 client.7029 shutdown 2024-04-25T10:10:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc10c1c8740 1 -- 172.21.15.16:0/2617348587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa2b050 msgr2=0x560c1fa45200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/2617348587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa2b050 0x560c1fa45200 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fc1040b7470 tx=0x7fc0ec0046c0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc10c1c8740 1 -- 172.21.15.16:0/2617348587 shutdown_connections 2024-04-25T10:10:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/2617348587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1fa3cc60 0x560c1fab3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/2617348587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1fa2b050 0x560c1fa45200 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/2617348587 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x560c1fab44a0 0x560c1fab07e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc10c1c8740 1 --2- 172.21.15.16:0/2617348587 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560c1fa00590 0x560c1faa71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc10c1c8740 1 -- 172.21.15.16:0/2617348587 >> 172.21.15.16:0/2617348587 conn(0x560c1f9c86e0 msgr2=0x560c1fa279d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:10.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc0e1ffb640 0 client.7029 ms_handle_reset on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:10.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc10c1c8740 1 -- 172.21.15.16:0/2617348587 shutdown_connections 2024-04-25T10:10:10.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.883+0000 7fc10c1c8740 1 -- 172.21.15.16:0/2617348587 wait complete. 2024-04-25T10:10:10.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.884+0000 7fc10c1c8740 20 client.7029 trim_cache size 0 max 16384 2024-04-25T10:10:10.896 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:10.896 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-04-25T10:10:10.897 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:10.897 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "dentry", 2024-04-25T10:10:10.897 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 831869697, 2024-04-25T10:10:10.897 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511627776, 2024-04-25T10:10:10.897 INFO:teuthology.orchestra.run.smithi016.stdout: "frag": "*", 2024-04-25T10:10:10.897 INFO:teuthology.orchestra.run.smithi016.stdout: "dname": "file_to_be_damaged", 2024-04-25T10:10:10.897 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_id": "head", 2024-04-25T10:10:10.897 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/subdir/file_to_be_damaged" 2024-04-25T10:10:10.897 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:10.897 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-04-25T10:10:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.884+0000 7fc1093e7640 1 -- 172.21.15.16:0/3242333182 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc0e406b500 msgr2=0x7fc0e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.884+0000 7fc1093e7640 1 --2- 172.21.15.16:0/3242333182 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc0e406b500 0x7fc0e406d9c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4033000 tx=0x7fc0f4008030 comp rx=0 tx=0).stop 2024-04-25T10:10:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.884+0000 7fc1093e7640 1 -- 172.21.15.16:0/3242333182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1041f3cb0 msgr2=0x7fc1040cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.884+0000 7fc1093e7640 1 --2- 172.21.15.16:0/3242333182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1041f3cb0 0x7fc1040cb250 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc001560 tx=0x7fc0fc007bd0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.884+0000 7fc1093e7640 1 -- 172.21.15.16:0/3242333182 shutdown_connections 2024-04-25T10:10:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.884+0000 7fc1093e7640 1 --2- 172.21.15.16:0/3242333182 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc0e406b500 0x7fc0e406d9c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.884+0000 7fc1093e7640 1 --2- 172.21.15.16:0/3242333182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1041f3cb0 0x7fc1040cb250 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.884+0000 7fc1093e7640 1 --2- 172.21.15.16:0/3242333182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1040b60b0 0x7fc1040cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.884+0000 7fc1093e7640 1 --2- 172.21.15.16:0/3242333182 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0f0003b70 0x7fc1040b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.884+0000 7fc1093e7640 1 -- 172.21.15.16:0/3242333182 >> 172.21.15.16:0/3242333182 conn(0x7fc10406f4a0 msgr2=0x7fc10415bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.884+0000 7fc1093e7640 1 -- 172.21.15.16:0/3242333182 shutdown_connections 2024-04-25T10:10:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.884+0000 7fc1093e7640 1 -- 172.21.15.16:0/3242333182 wait complete. 2024-04-25T10:10:10.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:10:11.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.994+0000 7fcbbb3b7640 1 Processor -- start 2024-04-25T10:10:11.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.994+0000 7fcbbb3b7640 1 -- start start 2024-04-25T10:10:11.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.995+0000 7fcbbb3b7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcbb40b5b60 0x7fcbb41687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.995+0000 7fcbbb3b7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb415e0c0 con 0x7fcbb40d35b0 2024-04-25T10:10:11.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.995+0000 7fcbbb3b7640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb40d2510 con 0x7fcbb415dce0 2024-04-25T10:10:11.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.995+0000 7fcbbb3b7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb40b7b10 con 0x7fcbb40b5b60 2024-04-25T10:10:11.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.995+0000 7fcbb8952640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcbb40d35b0 0x7fcbb40b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58352/0 (socket says 172.21.15.16:58352) 2024-04-25T10:10:11.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.995+0000 7fcbb8952640 1 -- 172.21.15.16:0/2484234844 learned_addr learned my addr 172.21.15.16:0/2484234844 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:11.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.995+0000 7fcbb9954640 1 --2- 172.21.15.16:0/2484234844 >> v2:172.21.15.16:3300/0 conn(0x7fcbb40b5b60 0x7fcbb41687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.995+0000 7fcbb9954640 1 -- 172.21.15.16:0/2484234844 >> v1:172.21.15.186:6789/0 conn(0x7fcbb415dce0 legacy=0x7fcbb40d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.995+0000 7fcbb9954640 1 -- 172.21.15.16:0/2484234844 >> v1:172.21.15.16:6789/0 conn(0x7fcbb40d35b0 legacy=0x7fcbb40b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.995+0000 7fcbb9954640 1 -- 172.21.15.16:0/2484234844 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb40d27f0 con 0x7fcbb40b5b60 2024-04-25T10:10:11.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.996+0000 7fcbb9954640 1 --2- 172.21.15.16:0/2484234844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbb40b5b60 0x7fcbb41687f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009d90 tx=0x7fcbac02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f3156a923b6014 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.996+0000 7fcbabfff640 1 -- 172.21.15.16:0/2484234844 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac007740 con 0x7fcbb40b5b60 2024-04-25T10:10:11.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.996+0000 7fcbabfff640 1 -- 172.21.15.16:0/2484234844 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbac0078e0 con 0x7fcbb40b5b60 2024-04-25T10:10:11.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.996+0000 7fcbabfff640 1 -- 172.21.15.16:0/2484234844 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac007cb0 con 0x7fcbb40b5b60 2024-04-25T10:10:11.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.997+0000 7fcbbb3b7640 1 -- 172.21.15.16:0/2484234844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbb40b5b60 msgr2=0x7fcbb41687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.997+0000 7fcbbb3b7640 1 --2- 172.21.15.16:0/2484234844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbb40b5b60 0x7fcbb41687f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009d90 tx=0x7fcbac02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.997+0000 7fcbbb3b7640 1 -- 172.21.15.16:0/2484234844 shutdown_connections 2024-04-25T10:10:11.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.997+0000 7fcbbb3b7640 1 --2- 172.21.15.16:0/2484234844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbb40b5b60 0x7fcbb41687f0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.997+0000 7fcbbb3b7640 1 -- 172.21.15.16:0/2484234844 >> 172.21.15.16:0/2484234844 conn(0x7fcbb4072900 msgr2=0x7fcbb40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.997+0000 7fcbbb3b7640 1 -- 172.21.15.16:0/2484234844 shutdown_connections 2024-04-25T10:10:11.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.997+0000 7fcbbb3b7640 1 -- 172.21.15.16:0/2484234844 wait complete. 2024-04-25T10:10:11.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.998+0000 7fcbbb3b7640 1 Processor -- start 2024-04-25T10:10:11.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.998+0000 7fcbbb3b7640 1 -- start start 2024-04-25T10:10:11.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.999+0000 7fcbbb3b7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbb40b5b60 0x7fcbb415e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.999+0000 7fcbbb3b7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb40d35b0 0x7fcbb41525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.999+0000 7fcbbb3b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbb415dce0 0x7fcbb41604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.999+0000 7fcbbb3b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb40d3090 con 0x7fcbb415dce0 2024-04-25T10:10:11.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.999+0000 7fcbbb3b7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb40d2db0 con 0x7fcbb40d35b0 2024-04-25T10:10:11.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.999+0000 7fcbbb3b7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb40d2160 con 0x7fcbb40b5b60 2024-04-25T10:10:11.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.999+0000 7fcbb9954640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbb415dce0 0x7fcbb41604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.999+0000 7fcbb9954640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbb415dce0 0x7fcbb41604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48048/0 (socket says 172.21.15.16:48048) 2024-04-25T10:10:11.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.999+0000 7fcbb9954640 1 -- 172.21.15.16:0/2427244078 learned_addr learned my addr 172.21.15.16:0/2427244078 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:11.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.999+0000 7fcbb8952640 1 --2- 172.21.15.16:0/2427244078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb40d35b0 0x7fcbb41525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.999+0000 7fcbb9153640 1 --2- 172.21.15.16:0/2427244078 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbb40b5b60 0x7fcbb415e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.999+0000 7fcbb9954640 1 -- 172.21.15.16:0/2427244078 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbb40b5b60 msgr2=0x7fcbb415e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.999+0000 7fcbb9954640 1 --2- 172.21.15.16:0/2427244078 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbb40b5b60 0x7fcbb415e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.999+0000 7fcbb9954640 1 -- 172.21.15.16:0/2427244078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb40d35b0 msgr2=0x7fcbb41525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.999+0000 7fcbb9954640 1 --2- 172.21.15.16:0/2427244078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb40d35b0 0x7fcbb41525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.999+0000 7fcbb9954640 1 -- 172.21.15.16:0/2427244078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb40cb610 con 0x7fcbb415dce0 2024-04-25T10:10:11.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.999+0000 7fcbb9153640 1 --2- 172.21.15.16:0/2427244078 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbb40b5b60 0x7fcbb415e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:11.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:10.999+0000 7fcbb9954640 1 --2- 172.21.15.16:0/2427244078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbb415dce0 0x7fcbb41604b0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009eb0 tx=0x7fcbac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.000+0000 7fcba9ffb640 1 -- 172.21.15.16:0/2427244078 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac040070 con 0x7fcbb415dce0 2024-04-25T10:10:11.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.000+0000 7fcba9ffb640 1 -- 172.21.15.16:0/2427244078 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbac031c90 con 0x7fcbb415dce0 2024-04-25T10:10:11.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.000+0000 7fcbbb3b7640 1 -- 172.21.15.16:0/2427244078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb415e920 con 0x7fcbb415dce0 2024-04-25T10:10:11.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.000+0000 7fcba9ffb640 1 -- 172.21.15.16:0/2427244078 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac03c640 con 0x7fcbb415dce0 2024-04-25T10:10:11.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.000+0000 7fcbbb3b7640 1 -- 172.21.15.16:0/2427244078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb40cb880 con 0x7fcbb415dce0 2024-04-25T10:10:11.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.001+0000 7fcbbb3b7640 1 -- 172.21.15.16:0/2427244078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb40749b0 con 0x7fcbb415dce0 2024-04-25T10:10:11.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.001+0000 7fcba9ffb640 1 -- 172.21.15.16:0/2427244078 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcbac039040 con 0x7fcbb415dce0 2024-04-25T10:10:11.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.002+0000 7fcba9ffb640 1 --2- 172.21.15.16:0/2427244078 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb9806b4d0 0x7fcb9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.002+0000 7fcba9ffb640 1 -- 172.21.15.16:0/2427244078 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fcbac0ab8d0 con 0x7fcbb415dce0 2024-04-25T10:10:11.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.002+0000 7fcbb9153640 1 --2- 172.21.15.16:0/2427244078 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb9806b4d0 0x7fcb9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.002+0000 7fcbb9153640 1 --2- 172.21.15.16:0/2427244078 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb9806b4d0 0x7fcb9806d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000a830 tx=0x7fcbb0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.006+0000 7fcba9ffb640 1 -- 172.21.15.16:0/2427244078 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcbac07a760 con 0x7fcbb415dce0 2024-04-25T10:10:11.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.229+0000 7fcbbb3b7640 1 -- 172.21.15.16:0/2427244078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcbb40cba50 con 0x7fcbb415dce0 2024-04-25T10:10:11.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.229+0000 7fcba9ffb640 1 -- 172.21.15.16:0/2427244078 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fcbac07a060 con 0x7fcbb415dce0 2024-04-25T10:10:11.235 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:11.235 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:10:11.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.231+0000 7fcbbb3b7640 1 -- 172.21.15.16:0/2427244078 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb9806b4d0 msgr2=0x7fcb9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.231+0000 7fcbbb3b7640 1 --2- 172.21.15.16:0/2427244078 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb9806b4d0 0x7fcb9806d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000a830 tx=0x7fcbb0008040 comp rx=0 tx=0).stop 2024-04-25T10:10:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.231+0000 7fcbbb3b7640 1 -- 172.21.15.16:0/2427244078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbb415dce0 msgr2=0x7fcbb41604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.231+0000 7fcbbb3b7640 1 --2- 172.21.15.16:0/2427244078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbb415dce0 0x7fcbb41604b0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009eb0 tx=0x7fcbac004a40 comp rx=0 tx=0).stop 2024-04-25T10:10:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.231+0000 7fcbbb3b7640 1 -- 172.21.15.16:0/2427244078 shutdown_connections 2024-04-25T10:10:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.231+0000 7fcbbb3b7640 1 --2- 172.21.15.16:0/2427244078 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb9806b4d0 0x7fcb9806d990 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.231+0000 7fcbbb3b7640 1 --2- 172.21.15.16:0/2427244078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbb415dce0 0x7fcbb41604b0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.231+0000 7fcbbb3b7640 1 --2- 172.21.15.16:0/2427244078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb40d35b0 0x7fcbb41525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.231+0000 7fcbbb3b7640 1 --2- 172.21.15.16:0/2427244078 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbb40b5b60 0x7fcbb415e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.231+0000 7fcbbb3b7640 1 -- 172.21.15.16:0/2427244078 >> 172.21.15.16:0/2427244078 conn(0x7fcbb4072900 msgr2=0x7fcbb40b6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.231+0000 7fcbbb3b7640 1 -- 172.21.15.16:0/2427244078 shutdown_connections 2024-04-25T10:10:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.231+0000 7fcbbb3b7640 1 -- 172.21.15.16:0/2427244078 wait complete. 2024-04-25T10:10:11.248 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:10:11.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:11.248 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-04-25T10:10:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.347+0000 7fc02824a640 1 Processor -- start 2024-04-25T10:10:11.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.348+0000 7fc02824a640 1 -- start start 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.348+0000 7fc02824a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc0200b0740 0x7fc0200b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.349+0000 7fc02824a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc02015f660 con 0x7fc0200cfd60 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.349+0000 7fc0257e5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc0200b0740 0x7fc0200b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.349+0000 7fc02824a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc02015f940 con 0x7fc02015ef60 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.349+0000 7fc0267e7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc0200cfd60 0x7fc0200d0140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58364/0 (socket says 172.21.15.16:58364) 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.349+0000 7fc02824a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02015fc20 con 0x7fc0200b0740 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.349+0000 7fc0267e7640 1 -- 172.21.15.16:0/3985243064 learned_addr learned my addr 172.21.15.16:0/3985243064 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:11.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.349+0000 7fc0257e5640 1 -- 172.21.15.16:0/3985243064 >> v1:172.21.15.186:6789/0 conn(0x7fc02015ef60 legacy=0x7fc0200ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.349+0000 7fc0257e5640 1 -- 172.21.15.16:0/3985243064 >> v1:172.21.15.16:6789/0 conn(0x7fc0200cfd60 legacy=0x7fc0200d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.349+0000 7fc0257e5640 1 -- 172.21.15.16:0/3985243064 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02015ff00 con 0x7fc0200b0740 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.350+0000 7fc0257e5640 1 --2- 172.21.15.16:0/3985243064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0200b0740 0x7fc0200b0bd0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fc0100097a0 tx=0x7fc01002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7818e25b73d83f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.350+0000 7fc024fe4640 1 -- 172.21.15.16:0/3985243064 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010037070 con 0x7fc0200b0740 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.350+0000 7fc024fe4640 1 -- 172.21.15.16:0/3985243064 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc010032640 con 0x7fc0200b0740 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.350+0000 7fc024fe4640 1 -- 172.21.15.16:0/3985243064 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010032a10 con 0x7fc0200b0740 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.350+0000 7fc02824a640 1 -- 172.21.15.16:0/3985243064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0200b0740 msgr2=0x7fc0200b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.350+0000 7fc02824a640 1 --2- 172.21.15.16:0/3985243064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0200b0740 0x7fc0200b0bd0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fc0100097a0 tx=0x7fc01002a2c0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.350+0000 7fc02824a640 1 -- 172.21.15.16:0/3985243064 shutdown_connections 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.350+0000 7fc02824a640 1 --2- 172.21.15.16:0/3985243064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0200b0740 0x7fc0200b0bd0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.350+0000 7fc02824a640 1 -- 172.21.15.16:0/3985243064 >> 172.21.15.16:0/3985243064 conn(0x7fc02006f4c0 msgr2=0x7fc02006f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.351+0000 7fc02824a640 1 -- 172.21.15.16:0/3985243064 shutdown_connections 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.351+0000 7fc02824a640 1 -- 172.21.15.16:0/3985243064 wait complete. 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.352+0000 7fc02824a640 1 Processor -- start 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.352+0000 7fc02824a640 1 -- start start 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.352+0000 7fc02824a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0200b0740 0x7fc0200b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.352+0000 7fc02824a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0200cfd60 0x7fc02006c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.352+0000 7fc02824a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc02015ef60 0x7fc0200b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.352+0000 7fc02824a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02015f340 con 0x7fc0200b0740 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.352+0000 7fc02824a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0201604c0 con 0x7fc0200cfd60 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.352+0000 7fc02824a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200cebd0 con 0x7fc02015ef60 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.352+0000 7fc025fe6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0200b0740 0x7fc0200b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.353+0000 7fc025fe6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0200b0740 0x7fc0200b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48064/0 (socket says 172.21.15.16:48064) 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.353+0000 7fc025fe6640 1 -- 172.21.15.16:0/236232166 learned_addr learned my addr 172.21.15.16:0/236232166 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.353+0000 7fc0257e5640 1 --2- 172.21.15.16:0/236232166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0200cfd60 0x7fc02006c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.353+0000 7fc025fe6640 1 -- 172.21.15.16:0/236232166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc02015ef60 msgr2=0x7fc0200b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.353+0000 7fc025fe6640 1 --2- 172.21.15.16:0/236232166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc02015ef60 0x7fc0200b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.353+0000 7fc025fe6640 1 -- 172.21.15.16:0/236232166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0200cfd60 msgr2=0x7fc02006c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.353+0000 7fc025fe6640 1 --2- 172.21.15.16:0/236232166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0200cfd60 0x7fc02006c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.353+0000 7fc025fe6640 1 -- 172.21.15.16:0/236232166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02006cd30 con 0x7fc0200b0740 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.353+0000 7fc0257e5640 1 --2- 172.21.15.16:0/236232166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0200cfd60 0x7fc02006c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.353+0000 7fc025fe6640 1 --2- 172.21.15.16:0/236232166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0200b0740 0x7fc0200b0bd0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00c430 tx=0x7fc01c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.353+0000 7fc0167fc640 1 -- 172.21.15.16:0/236232166 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c017070 con 0x7fc0200b0740 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.353+0000 7fc0167fc640 1 -- 172.21.15.16:0/236232166 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01c0052d0 con 0x7fc0200b0740 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.353+0000 7fc02824a640 1 -- 172.21.15.16:0/236232166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0200b11b0 con 0x7fc0200b0740 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.353+0000 7fc0167fc640 1 -- 172.21.15.16:0/236232166 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c003830 con 0x7fc0200b0740 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.353+0000 7fc02824a640 1 -- 172.21.15.16:0/236232166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0200b1a50 con 0x7fc0200b0740 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.355+0000 7fc0167fc640 1 -- 172.21.15.16:0/236232166 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc01c01a070 con 0x7fc0200b0740 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.355+0000 7fc0167fc640 1 --2- 172.21.15.16:0/236232166 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc00406b4d0 0x7fc00406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.355+0000 7fc0167fc640 1 -- 172.21.15.16:0/236232166 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fc01c0895a0 con 0x7fc0200b0740 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.355+0000 7fc0257e5640 1 --2- 172.21.15.16:0/236232166 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc00406b4d0 0x7fc00406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.356+0000 7fc0257e5640 1 --2- 172.21.15.16:0/236232166 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc00406b4d0 0x7fc00406d990 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc010034000 tx=0x7fc01002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.356+0000 7fc02b027740 1 Processor -- start 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.356+0000 7fc02b027740 1 -- start start 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.357+0000 7fc02b027740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e9861ff70 0x564e98620350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.357+0000 7fc02b027740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98620990 0x564e985efee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.357+0000 7fc02b027740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e985f0420 0x564e986183f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.357+0000 7fc02b027740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e98558c90 con 0x564e98620990 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.357+0000 7fc02b027740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e98501f50 con 0x564e985f0420 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.357+0000 7fc02b027740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e98554d30 con 0x564e9861ff70 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.357+0000 7fc0257e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98620990 0x564e985efee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.357+0000 7fc0257e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98620990 0x564e985efee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48076/0 (socket says 172.21.15.16:48076) 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.357+0000 7fc0257e5640 1 -- 172.21.15.16:0/3495952673 learned_addr learned my addr 172.21.15.16:0/3495952673 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.357+0000 7fc025fe6640 1 --2- 172.21.15.16:0/3495952673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e9861ff70 0x564e98620350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.357+0000 7fc0267e7640 1 --2- 172.21.15.16:0/3495952673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e985f0420 0x564e986183f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.357+0000 7fc0257e5640 1 -- 172.21.15.16:0/3495952673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e9861ff70 msgr2=0x564e98620350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.357+0000 7fc0257e5640 1 --2- 172.21.15.16:0/3495952673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e9861ff70 0x564e98620350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.357+0000 7fc0257e5640 1 -- 172.21.15.16:0/3495952673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e985f0420 msgr2=0x564e986183f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.357+0000 7fc0257e5640 1 --2- 172.21.15.16:0/3495952673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e985f0420 0x564e986183f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.358+0000 7fc0257e5640 1 -- 172.21.15.16:0/3495952673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e98335a50 con 0x564e98620990 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.358+0000 7fc0257e5640 1 --2- 172.21.15.16:0/3495952673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98620990 0x564e985efee0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc01002a8b0 tx=0x7fc01002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72793b9320b692f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.358+0000 7fc0037fe640 1 -- 172.21.15.16:0/3495952673 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01004d070 con 0x564e98620990 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.358+0000 7fc0037fe640 1 -- 172.21.15.16:0/3495952673 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc010048470 con 0x564e98620990 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.358+0000 7fc0037fe640 1 -- 172.21.15.16:0/3495952673 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010037040 con 0x564e98620990 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.358+0000 7fc02b027740 1 -- 172.21.15.16:0/3495952673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98620990 msgr2=0x564e985efee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.358+0000 7fc02b027740 1 --2- 172.21.15.16:0/3495952673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98620990 0x564e985efee0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc01002a8b0 tx=0x7fc01002c000 comp rx=0 tx=0).stop 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.358+0000 7fc02b027740 1 -- 172.21.15.16:0/3495952673 shutdown_connections 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.358+0000 7fc02b027740 1 --2- 172.21.15.16:0/3495952673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e985f0420 0x564e986183f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.358+0000 7fc02b027740 1 --2- 172.21.15.16:0/3495952673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98620990 0x564e985efee0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.358+0000 7fc02b027740 1 --2- 172.21.15.16:0/3495952673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e9861ff70 0x564e98620350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.359+0000 7fc02b027740 1 -- 172.21.15.16:0/3495952673 >> 172.21.15.16:0/3495952673 conn(0x564e985f2360 msgr2=0x564e9863ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.359+0000 7fc02b027740 1 -- 172.21.15.16:0/3495952673 shutdown_connections 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.359+0000 7fc02b027740 1 -- 172.21.15.16:0/3495952673 wait complete. 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.359+0000 7fc02b027740 1 -- start start 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.359+0000 7fc02b027740 1 Processor -- start 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc02b027740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e9861ff70 0x564e98613a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc02b027740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e98620990 0x564e9861f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc02b027740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e98614850 0x564e98620d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc02b027740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9861c6c0 con 0x564e9861ff70 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc02b027740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e984f7700 con 0x564e98620990 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc02b027740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e9861c960 con 0x564e98614850 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc02b027740 10 client.0 _ll_register_callbacks cb 0x564e9854f150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc025fe6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e9861ff70 0x564e98613a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc025fe6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e9861ff70 0x564e98613a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48090/0 (socket says 172.21.15.16:48090) 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc025fe6640 1 -- 172.21.15.16:0/3422952948 learned_addr learned my addr 172.21.15.16:0/3422952948 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc0267e7640 1 --2- 172.21.15.16:0/3422952948 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e98614850 0x564e98620d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc0257e5640 1 --2- 172.21.15.16:0/3422952948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e98620990 0x564e9861f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc025fe6640 1 -- 172.21.15.16:0/3422952948 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e98614850 msgr2=0x564e98620d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc025fe6640 1 --2- 172.21.15.16:0/3422952948 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e98614850 0x564e98620d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc025fe6640 1 -- 172.21.15.16:0/3422952948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e98620990 msgr2=0x564e9861f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc025fe6640 1 --2- 172.21.15.16:0/3422952948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e98620990 0x564e9861f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc025fe6640 1 -- 172.21.15.16:0/3422952948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e9822d3e0 con 0x564e9861ff70 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc0267e7640 1 --2- 172.21.15.16:0/3422952948 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e98614850 0x564e98620d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.360+0000 7fc0257e5640 1 --2- 172.21.15.16:0/3422952948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e98620990 0x564e9861f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.361+0000 7fc025fe6640 1 --2- 172.21.15.16:0/3422952948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e9861ff70 0x564e98613a90 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc01c000f30 tx=0x7fc01c057e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7287d7189e9168e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.361+0000 7fc000ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.361+0000 7fc000ff9640 1 -- 172.21.15.16:0/3422952948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c06f020 con 0x564e9861ff70 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.361+0000 7fc000ff9640 1 -- 172.21.15.16:0/3422952948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fc01c057150 con 0x564e9861ff70 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.361+0000 7fc02b027740 1 -- 172.21.15.16:0/3422952948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc010048470 con 0x564e9861ff70 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.361+0000 7fc000ff9640 1 -- 172.21.15.16:0/3422952948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc01c057350 con 0x564e9861ff70 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.361+0000 7fc02b027740 10 client.7053 fetch_fsmap learned FSMap version 43 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.361+0000 7fc02b027740 1 -- 172.21.15.16:0/3422952948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x564e983961e0 con 0x564e9861ff70 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.361+0000 7fc000ff9640 1 -- 172.21.15.16:0/3422952948 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc010048470 con 0x564e9861ff70 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.361+0000 7fc000ff9640 1 -- 172.21.15.16:0/3422952948 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc01c0890b0 con 0x564e9861ff70 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.361+0000 7fc02b027740 10 client.7053 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.361+0000 7fc02b027740 10 client.7053 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.361+0000 7fc02b027740 20 client.7053 populate_metadata read hostname 'smithi016' 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.361+0000 7fc02b027740 1 --2- 172.21.15.16:0/3422952948 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x564e9864b0c0 0x564e9864b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.361+0000 7fc02b027740 4 client.7053 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.361+0000 7fc02b027740 1 -- 172.21.15.16:0/3422952948 --> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e985661d0 con 0x564e9864b0c0 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.361+0000 7fc0257e5640 1 --2- 172.21.15.16:0/3422952948 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x564e9864b0c0 0x564e9864b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.362+0000 7fc000ff9640 10 client.7053 ms_handle_connect on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.362+0000 7fc0257e5640 1 --2- 172.21.15.16:0/3422952948 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x564e9864b0c0 0x564e9864b4f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eaa8e5704250170 server_cookie=41c6f30551df1883 in_seq=0 out_seq=0 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.362+0000 7fc000ff9640 1 -- 172.21.15.16:0/3422952948 <== mds.0 v2:172.21.15.16:6834/1735811292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564e985661d0 con 0x564e9864b0c0 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.362+0000 7fc000ff9640 10 client.7053 handle_command_reply: tid=0 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.362+0000 7fc02b027740 1 client.7053 shutdown 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.362+0000 7fc02b027740 1 -- 172.21.15.16:0/3422952948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e9861ff70 msgr2=0x564e98613a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.362+0000 7fc02b027740 1 --2- 172.21.15.16:0/3422952948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e9861ff70 0x564e98613a90 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc01c000f30 tx=0x7fc01c057e40 comp rx=0 tx=0).stop 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.362+0000 7fc02b027740 1 -- 172.21.15.16:0/3422952948 shutdown_connections 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.362+0000 7fc02b027740 1 --2- 172.21.15.16:0/3422952948 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x564e9864b0c0 0x564e9864b4f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.362+0000 7fc02b027740 1 --2- 172.21.15.16:0/3422952948 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e98614850 0x564e98620d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.362+0000 7fc02b027740 1 --2- 172.21.15.16:0/3422952948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e98620990 0x564e9861f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.362+0000 7fc000ff9640 0 client.7053 ms_handle_reset on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.362+0000 7fc02b027740 1 --2- 172.21.15.16:0/3422952948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e9861ff70 0x564e98613a90 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.362+0000 7fc02b027740 1 -- 172.21.15.16:0/3422952948 >> 172.21.15.16:0/3422952948 conn(0x564e985f0420 msgr2=0x564e9863ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.362+0000 7fc02b027740 1 -- 172.21.15.16:0/3422952948 shutdown_connections 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.362+0000 7fc02b027740 1 -- 172.21.15.16:0/3422952948 wait complete. 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.363+0000 7fc02b027740 20 client.7053 trim_cache size 0 max 16384 2024-04-25T10:10:11.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.378+0000 7fc02b027740 1 Processor -- start 2024-04-25T10:10:11.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.378+0000 7fc02b027740 1 -- start start 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.379+0000 7fc02b027740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98614850 0x564e985ef800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.379+0000 7fc02b027740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9861ff70 0x564e9863e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.379+0000 7fc02b027740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e98620990 0x564e985f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.379+0000 7fc02b027740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0202001e0 con 0x564e98614850 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.379+0000 7fc02b027740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0201ff380 con 0x564e9861ff70 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.379+0000 7fc02b027740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0201e7110 con 0x564e98620990 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.379+0000 7fc025fe6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98614850 0x564e985ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.379+0000 7fc025fe6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98614850 0x564e985ef800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48098/0 (socket says 172.21.15.16:48098) 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.379+0000 7fc025fe6640 1 -- 172.21.15.16:0/138974302 learned_addr learned my addr 172.21.15.16:0/138974302 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.379+0000 7fc025fe6640 1 -- 172.21.15.16:0/138974302 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e98620990 msgr2=0x564e985f4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.379+0000 7fc025fe6640 1 --2- 172.21.15.16:0/138974302 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e98620990 0x564e985f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.379+0000 7fc025fe6640 1 -- 172.21.15.16:0/138974302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9861ff70 msgr2=0x564e9863e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.379+0000 7fc025fe6640 1 --2- 172.21.15.16:0/138974302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9861ff70 0x564e9863e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.379+0000 7fc025fe6640 1 -- 172.21.15.16:0/138974302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e98621100 con 0x564e98614850 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.379+0000 7fc025fe6640 1 --2- 172.21.15.16:0/138974302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98614850 0x564e985ef800 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc01c05fee0 tx=0x7fc01c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f57fda57641fea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.379+0000 7fc002ffd640 1 -- 172.21.15.16:0/138974302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c06f040 con 0x564e98614850 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.379+0000 7fc002ffd640 1 -- 172.21.15.16:0/138974302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01c058400 con 0x564e98614850 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.379+0000 7fc002ffd640 1 -- 172.21.15.16:0/138974302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c054370 con 0x564e98614850 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.380+0000 7fc02b027740 1 -- 172.21.15.16:0/138974302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98614850 msgr2=0x564e985ef800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.380+0000 7fc02b027740 1 --2- 172.21.15.16:0/138974302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98614850 0x564e985ef800 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc01c05fee0 tx=0x7fc01c014000 comp rx=0 tx=0).stop 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.380+0000 7fc02b027740 1 -- 172.21.15.16:0/138974302 shutdown_connections 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.380+0000 7fc02b027740 1 --2- 172.21.15.16:0/138974302 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e98620990 0x564e985f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.380+0000 7fc02b027740 1 --2- 172.21.15.16:0/138974302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e9861ff70 0x564e9863e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.380+0000 7fc02b027740 1 --2- 172.21.15.16:0/138974302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98614850 0x564e985ef800 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.380+0000 7fc02b027740 1 -- 172.21.15.16:0/138974302 >> 172.21.15.16:0/138974302 conn(0x564e985f0420 msgr2=0x564e986138a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.380+0000 7fc02b027740 1 -- 172.21.15.16:0/138974302 shutdown_connections 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.380+0000 7fc02b027740 1 -- 172.21.15.16:0/138974302 wait complete. 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.380+0000 7fc02b027740 1 -- start start 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.381+0000 7fc02b027740 1 Processor -- start 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.381+0000 7fc02b027740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98614850 0x564e9864a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.381+0000 7fc02b027740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e9861ff70 0x564e9863a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.381+0000 7fc02b027740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e98620990 0x564e9863aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.381+0000 7fc02b027740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200ad210 con 0x564e98614850 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.381+0000 7fc02b027740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e986a68c0 con 0x564e98620990 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.381+0000 7fc02b027740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e98649e10 con 0x564e9861ff70 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.381+0000 7fc02b027740 10 client.0 _ll_register_callbacks cb 0x7fc010035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.381+0000 7fc025fe6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98614850 0x564e9864a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.382+0000 7fc025fe6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98614850 0x564e9864a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48104/0 (socket says 172.21.15.16:48104) 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.382+0000 7fc025fe6640 1 -- 172.21.15.16:0/3663409359 learned_addr learned my addr 172.21.15.16:0/3663409359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.382+0000 7fc0267e7640 1 --2- 172.21.15.16:0/3663409359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e98620990 0x564e9863aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.382+0000 7fc0257e5640 1 --2- 172.21.15.16:0/3663409359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e9861ff70 0x564e9863a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.382+0000 7fc025fe6640 1 -- 172.21.15.16:0/3663409359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e9861ff70 msgr2=0x564e9863a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.382+0000 7fc025fe6640 1 --2- 172.21.15.16:0/3663409359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e9861ff70 0x564e9863a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.382+0000 7fc025fe6640 1 -- 172.21.15.16:0/3663409359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e98620990 msgr2=0x564e9863aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.382+0000 7fc025fe6640 1 --2- 172.21.15.16:0/3663409359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e98620990 0x564e9863aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.382+0000 7fc025fe6640 1 -- 172.21.15.16:0/3663409359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e982064a0 con 0x564e98614850 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.382+0000 7fc025fe6640 1 --2- 172.21.15.16:0/3663409359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98614850 0x564e9864a2f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00cdf0 tx=0x7fc01c087980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87456b369ff0378d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.382+0000 7fc000ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.382+0000 7fc000ff9640 1 -- 172.21.15.16:0/3663409359 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c06f040 con 0x564e98614850 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.382+0000 7fc000ff9640 1 -- 172.21.15.16:0/3663409359 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fc01c057150 con 0x564e98614850 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.382+0000 7fc02b027740 1 -- 172.21.15.16:0/3663409359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc01c058400 con 0x564e98614850 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.382+0000 7fc000ff9640 1 -- 172.21.15.16:0/3663409359 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc01c057660 con 0x564e98614850 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.382+0000 7fc02b027740 10 client.7059 fetch_fsmap learned FSMap version 43 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.382+0000 7fc02b027740 1 -- 172.21.15.16:0/3663409359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x564e982064a0 con 0x564e98614850 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.382+0000 7fc000ff9640 1 -- 172.21.15.16:0/3663409359 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01c058400 con 0x564e98614850 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.383+0000 7fc000ff9640 1 -- 172.21.15.16:0/3663409359 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc01c054370 con 0x564e98614850 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.383+0000 7fc02b027740 10 client.7059 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.383+0000 7fc02b027740 10 client.7059 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.383+0000 7fc02b027740 20 client.7059 populate_metadata read hostname 'smithi016' 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.383+0000 7fc02b027740 1 --2- 172.21.15.16:0/3663409359 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x564e98624050 0x564e986a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.383+0000 7fc02b027740 4 client.7059 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.383+0000 7fc02b027740 1 -- 172.21.15.16:0/3663409359 --> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) v1 -- 0x564e98560520 con 0x564e98624050 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.383+0000 7fc0257e5640 1 --2- 172.21.15.16:0/3663409359 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x564e98624050 0x564e986a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.383+0000 7fc000ff9640 10 client.7059 ms_handle_connect on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.383+0000 7fc0257e5640 1 --2- 172.21.15.16:0/3663409359 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x564e98624050 0x564e986a5a00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2346424526391e6d server_cookie=b8a14e7169c78be2 in_seq=0 out_seq=0 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc000ff9640 1 -- 172.21.15.16:0/3663409359 <== mds.0 v2:172.21.15.16:6834/1735811292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564e98560520 con 0x564e98624050 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc000ff9640 10 client.7059 handle_command_reply: tid=0 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc02b027740 1 client.7059 shutdown 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc02b027740 1 -- 172.21.15.16:0/3663409359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98614850 msgr2=0x564e9864a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc02b027740 1 --2- 172.21.15.16:0/3663409359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98614850 0x564e9864a2f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00cdf0 tx=0x7fc01c087980 comp rx=0 tx=0).stop 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc02b027740 1 -- 172.21.15.16:0/3663409359 shutdown_connections 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc02b027740 1 --2- 172.21.15.16:0/3663409359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564e98620990 0x564e9863aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc02b027740 1 --2- 172.21.15.16:0/3663409359 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x564e98624050 0x564e986a5a00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc02b027740 1 --2- 172.21.15.16:0/3663409359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564e9861ff70 0x564e9863a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc02b027740 1 --2- 172.21.15.16:0/3663409359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e98614850 0x564e9864a2f0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc02b027740 1 -- 172.21.15.16:0/3663409359 >> 172.21.15.16:0/3663409359 conn(0x564e985f0420 msgr2=0x564e9863e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc000ff9640 0 client.7059 ms_handle_reset on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc02b027740 1 -- 172.21.15.16:0/3663409359 shutdown_connections 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc02b027740 1 -- 172.21.15.16:0/3663409359 wait complete. 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc02b027740 20 client.7059 trim_cache size 0 max 16384 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "7f5897fa-e235-4587-b48b-3cdad5206572", 2024-04-25T10:10:11.400 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-04-25T10:10:11.400 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc02824a640 1 -- 172.21.15.16:0/236232166 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc00406b4d0 msgr2=0x7fc00406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc02824a640 1 --2- 172.21.15.16:0/236232166 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc00406b4d0 0x7fc00406d990 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc010034000 tx=0x7fc01002e000 comp rx=0 tx=0).stop 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc02824a640 1 -- 172.21.15.16:0/236232166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0200b0740 msgr2=0x7fc0200b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.384+0000 7fc02824a640 1 --2- 172.21.15.16:0/236232166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0200b0740 0x7fc0200b0bd0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00c430 tx=0x7fc01c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.385+0000 7fc02824a640 1 -- 172.21.15.16:0/236232166 shutdown_connections 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.385+0000 7fc02824a640 1 --2- 172.21.15.16:0/236232166 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc00406b4d0 0x7fc00406d990 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.385+0000 7fc02824a640 1 --2- 172.21.15.16:0/236232166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc02015ef60 0x7fc0200b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.385+0000 7fc02824a640 1 --2- 172.21.15.16:0/236232166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0200cfd60 0x7fc02006c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.385+0000 7fc02824a640 1 --2- 172.21.15.16:0/236232166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0200b0740 0x7fc0200b0bd0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.385+0000 7fc02824a640 1 -- 172.21.15.16:0/236232166 >> 172.21.15.16:0/236232166 conn(0x7fc02006f4c0 msgr2=0x7fc0200c8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.385+0000 7fc02824a640 1 -- 172.21.15.16:0/236232166 shutdown_connections 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.385+0000 7fc02824a640 1 -- 172.21.15.16:0/236232166 wait complete. 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:11.402 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:10:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.499+0000 7fa9c47b0640 1 Processor -- start 2024-04-25T10:10:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.499+0000 7fa9c47b0640 1 -- start start 2024-04-25T10:10:11.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.500+0000 7fa9c47b0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9bc15dca0 0x7fa9bc0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.500+0000 7fa9c47b0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9bc0d3b90 0x7fa9bc0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.500+0000 7fa9c47b0640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa9bc0cee80 0x7fa9bc0d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.500+0000 7fa9c47b0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9bc0b6b20 con 0x7fa9bc0d3b90 2024-04-25T10:10:11.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.500+0000 7fa9c47b0640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9bc0d4090 con 0x7fa9bc0cee80 2024-04-25T10:10:11.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.500+0000 7fa9c47b0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0d4370 con 0x7fa9bc15dca0 2024-04-25T10:10:11.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.500+0000 7fa9c1d4b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9bc0d3b90 0x7fa9bc0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.500+0000 7fa9c1d4b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9bc0d3b90 0x7fa9bc0d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48110/0 (socket says 172.21.15.16:48110) 2024-04-25T10:10:11.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.500+0000 7fa9c1d4b640 1 -- 172.21.15.16:0/3866574742 learned_addr learned my addr 172.21.15.16:0/3866574742 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:11.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.500+0000 7fa9c2d4d640 1 --2- 172.21.15.16:0/3866574742 >> v2:172.21.15.186:3300/0 conn(0x7fa9bc0cee80 0x7fa9bc0d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.500+0000 7fa9c254c640 1 --2- 172.21.15.16:0/3866574742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9bc15dca0 0x7fa9bc0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.500+0000 7fa9c1d4b640 1 -- 172.21.15.16:0/3866574742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9bc15dca0 msgr2=0x7fa9bc0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.500+0000 7fa9c1d4b640 1 --2- 172.21.15.16:0/3866574742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9bc15dca0 0x7fa9bc0ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.500+0000 7fa9c1d4b640 1 -- 172.21.15.16:0/3866574742 >> v2:172.21.15.186:3300/0 conn(0x7fa9bc0cee80 msgr2=0x7fa9bc0d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.500+0000 7fa9c1d4b640 1 --2- 172.21.15.16:0/3866574742 >> v2:172.21.15.186:3300/0 conn(0x7fa9bc0cee80 0x7fa9bc0d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.500+0000 7fa9c1d4b640 1 -- 172.21.15.16:0/3866574742 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc0d4650 con 0x7fa9bc0d3b90 2024-04-25T10:10:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.500+0000 7fa9c2d4d640 1 --2- 172.21.15.16:0/3866574742 >> v2:172.21.15.186:3300/0 conn(0x7fa9bc0cee80 0x7fa9bc0d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.500+0000 7fa9c254c640 1 --2- 172.21.15.16:0/3866574742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9bc15dca0 0x7fa9bc0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.501+0000 7fa9c1d4b640 1 --2- 172.21.15.16:0/3866574742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0d3b90 0x7fa9bc0d0a20 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc0d04c0 tx=0x7fa9b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a900c3aa3e6796d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.501+0000 7fa9c154a640 1 -- 172.21.15.16:0/3866574742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8037070 con 0x7fa9bc0d3b90 2024-04-25T10:10:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.501+0000 7fa9c154a640 1 -- 172.21.15.16:0/3866574742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b8032440 con 0x7fa9bc0d3b90 2024-04-25T10:10:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.501+0000 7fa9c154a640 1 -- 172.21.15.16:0/3866574742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8032810 con 0x7fa9bc0d3b90 2024-04-25T10:10:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.502+0000 7fa9c47b0640 1 -- 172.21.15.16:0/3866574742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0d3b90 msgr2=0x7fa9bc0d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.502+0000 7fa9c47b0640 1 --2- 172.21.15.16:0/3866574742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0d3b90 0x7fa9bc0d0a20 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc0d04c0 tx=0x7fa9b802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.502+0000 7fa9c47b0640 1 -- 172.21.15.16:0/3866574742 shutdown_connections 2024-04-25T10:10:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.502+0000 7fa9c47b0640 1 --2- 172.21.15.16:0/3866574742 >> v2:172.21.15.186:3300/0 conn(0x7fa9bc0cee80 0x7fa9bc0d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.502+0000 7fa9c47b0640 1 --2- 172.21.15.16:0/3866574742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0d3b90 0x7fa9bc0d0a20 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.502+0000 7fa9c47b0640 1 --2- 172.21.15.16:0/3866574742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9bc15dca0 0x7fa9bc0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.502+0000 7fa9c47b0640 1 -- 172.21.15.16:0/3866574742 >> 172.21.15.16:0/3866574742 conn(0x7fa9bc0726c0 msgr2=0x7fa9bc072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.502+0000 7fa9c47b0640 1 -- 172.21.15.16:0/3866574742 shutdown_connections 2024-04-25T10:10:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.502+0000 7fa9c47b0640 1 -- 172.21.15.16:0/3866574742 wait complete. 2024-04-25T10:10:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.503+0000 7fa9c47b0640 1 Processor -- start 2024-04-25T10:10:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.503+0000 7fa9c47b0640 1 -- start start 2024-04-25T10:10:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.504+0000 7fa9c47b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0cee80 0x7fa9bc155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.504+0000 7fa9c47b0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9bc0d3b90 0x7fa9bc1564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.504+0000 7fa9c47b0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9bc15dca0 0x7fa9bc152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.504+0000 7fa9c47b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc15e200 con 0x7fa9bc0cee80 2024-04-25T10:10:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.504+0000 7fa9c47b0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc15e080 con 0x7fa9bc0d3b90 2024-04-25T10:10:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.504+0000 7fa9c47b0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0b2500 con 0x7fa9bc15dca0 2024-04-25T10:10:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.504+0000 7fa9c254c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0cee80 0x7fa9bc155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.504+0000 7fa9c254c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0cee80 0x7fa9bc155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48120/0 (socket says 172.21.15.16:48120) 2024-04-25T10:10:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.504+0000 7fa9c254c640 1 -- 172.21.15.16:0/2028747645 learned_addr learned my addr 172.21.15.16:0/2028747645 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.504+0000 7fa9c1d4b640 1 --2- 172.21.15.16:0/2028747645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9bc0d3b90 0x7fa9bc1564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.504+0000 7fa9c2d4d640 1 --2- 172.21.15.16:0/2028747645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9bc15dca0 0x7fa9bc152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.504+0000 7fa9c254c640 1 -- 172.21.15.16:0/2028747645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9bc15dca0 msgr2=0x7fa9bc152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.504+0000 7fa9c254c640 1 --2- 172.21.15.16:0/2028747645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9bc15dca0 0x7fa9bc152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.504+0000 7fa9c254c640 1 -- 172.21.15.16:0/2028747645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9bc0d3b90 msgr2=0x7fa9bc1564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.504+0000 7fa9c254c640 1 --2- 172.21.15.16:0/2028747645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9bc0d3b90 0x7fa9bc1564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.504+0000 7fa9c254c640 1 -- 172.21.15.16:0/2028747645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc153a90 con 0x7fa9bc0cee80 2024-04-25T10:10:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.504+0000 7fa9c254c640 1 --2- 172.21.15.16:0/2028747645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0cee80 0x7fa9bc155f70 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00c7f0 tx=0x7fa9ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.504+0000 7fa9b2ffd640 1 -- 172.21.15.16:0/2028747645 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac015070 con 0x7fa9bc0cee80 2024-04-25T10:10:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.505+0000 7fa9b2ffd640 1 -- 172.21.15.16:0/2028747645 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ac007820 con 0x7fa9bc0cee80 2024-04-25T10:10:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.505+0000 7fa9c47b0640 1 -- 172.21.15.16:0/2028747645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9bc153d70 con 0x7fa9bc0cee80 2024-04-25T10:10:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.505+0000 7fa9b2ffd640 1 -- 172.21.15.16:0/2028747645 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac007bd0 con 0x7fa9bc0cee80 2024-04-25T10:10:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.505+0000 7fa9c47b0640 1 -- 172.21.15.16:0/2028747645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9bc154760 con 0x7fa9bc0cee80 2024-04-25T10:10:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.506+0000 7fa9b2ffd640 1 -- 172.21.15.16:0/2028747645 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9ac018070 con 0x7fa9bc0cee80 2024-04-25T10:10:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.506+0000 7fa9b2ffd640 1 --2- 172.21.15.16:0/2028747645 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa9a006b4d0 0x7fa9a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.506+0000 7fa9b2ffd640 1 -- 172.21.15.16:0/2028747645 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fa9ac08a430 con 0x7fa9bc0cee80 2024-04-25T10:10:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.507+0000 7fa9c1d4b640 1 --2- 172.21.15.16:0/2028747645 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa9a006b4d0 0x7fa9a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.507+0000 7fa9c1d4b640 1 --2- 172.21.15.16:0/2028747645 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa9a006b4d0 0x7fa9a006d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8034000 tx=0x7fa9b802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.507+0000 7fa9c758f740 1 Processor -- start 2024-04-25T10:10:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.507+0000 7fa9c758f740 1 -- start start 2024-04-25T10:10:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.508+0000 7fa9c758f740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a07b10 0x564cf3a0a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.508+0000 7fa9c758f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07ef0 0x564cf3a0a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.508+0000 7fa9c758f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf39da770 0x564cf39ddf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.508+0000 7fa9c758f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf3943da0 con 0x564cf3a07ef0 2024-04-25T10:10:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.508+0000 7fa9c758f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf38ecb80 con 0x564cf39da770 2024-04-25T10:10:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.508+0000 7fa9c758f740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x564cf393fe40 con 0x564cf3a07b10 2024-04-25T10:10:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.508+0000 7fa9c1d4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07ef0 0x564cf3a0a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.508+0000 7fa9c1d4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07ef0 0x564cf3a0a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48134/0 (socket says 172.21.15.16:48134) 2024-04-25T10:10:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.508+0000 7fa9c1d4b640 1 -- 172.21.15.16:0/282237356 learned_addr learned my addr 172.21.15.16:0/282237356 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.508+0000 7fa9c2d4d640 1 --2- 172.21.15.16:0/282237356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf39da770 0x564cf39ddf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.508+0000 7fa9c254c640 1 --2- 172.21.15.16:0/282237356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a07b10 0x564cf3a0a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.508+0000 7fa9c1d4b640 1 -- 172.21.15.16:0/282237356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a07b10 msgr2=0x564cf3a0a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.508+0000 7fa9c1d4b640 1 --2- 172.21.15.16:0/282237356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a07b10 0x564cf3a0a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.508+0000 7fa9c1d4b640 1 -- 172.21.15.16:0/282237356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf39da770 msgr2=0x564cf39ddf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.508+0000 7fa9c1d4b640 1 --2- 172.21.15.16:0/282237356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf39da770 0x564cf39ddf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.508+0000 7fa9c1d4b640 1 -- 172.21.15.16:0/282237356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cf38f6f10 con 0x564cf3a07ef0 2024-04-25T10:10:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.508+0000 7fa9c2d4d640 1 --2- 172.21.15.16:0/282237356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf39da770 0x564cf39ddf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.509+0000 7fa9c1d4b640 1 --2- 172.21.15.16:0/282237356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07ef0 0x564cf3a0a860 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8000c00 tx=0x7fa9b802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912f4c622bc49acb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.509+0000 7fa99ffff640 1 -- 172.21.15.16:0/282237356 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b804e070 con 0x564cf3a07ef0 2024-04-25T10:10:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.509+0000 7fa99ffff640 1 -- 172.21.15.16:0/282237356 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b803e460 con 0x564cf3a07ef0 2024-04-25T10:10:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.509+0000 7fa99ffff640 1 -- 172.21.15.16:0/282237356 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8037040 con 0x564cf3a07ef0 2024-04-25T10:10:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.509+0000 7fa9c758f740 1 -- 172.21.15.16:0/282237356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07ef0 msgr2=0x564cf3a0a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.509+0000 7fa9c758f740 1 --2- 172.21.15.16:0/282237356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07ef0 0x564cf3a0a860 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8000c00 tx=0x7fa9b802c000 comp rx=0 tx=0).stop 2024-04-25T10:10:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.509+0000 7fa9c758f740 1 -- 172.21.15.16:0/282237356 shutdown_connections 2024-04-25T10:10:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.509+0000 7fa9c758f740 1 --2- 172.21.15.16:0/282237356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf39da770 0x564cf39ddf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.509+0000 7fa9c758f740 1 --2- 172.21.15.16:0/282237356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07ef0 0x564cf3a0a860 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.509+0000 7fa9c758f740 1 --2- 172.21.15.16:0/282237356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a07b10 0x564cf3a0a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.510+0000 7fa9c758f740 1 -- 172.21.15.16:0/282237356 >> 172.21.15.16:0/282237356 conn(0x564cf39dc9e0 msgr2=0x564cf3a25320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.510+0000 7fa9c758f740 1 -- 172.21.15.16:0/282237356 shutdown_connections 2024-04-25T10:10:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.510+0000 7fa9c758f740 1 -- 172.21.15.16:0/282237356 wait complete. 2024-04-25T10:10:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.510+0000 7fa9c758f740 1 -- start start 2024-04-25T10:10:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.510+0000 7fa9c758f740 1 Processor -- start 2024-04-25T10:10:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.511+0000 7fa9c758f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf3a07b10 0x564cf3a04de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.511+0000 7fa9c758f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07ef0 0x564cf3a34ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.511+0000 7fa9c758f740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a0ae50 0x564cf3a0ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.511+0000 7fa9c758f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf39dfb90 con 0x564cf3a07ef0 2024-04-25T10:10:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.511+0000 7fa9c758f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf38e24b0 con 0x564cf3a07b10 2024-04-25T10:10:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.511+0000 7fa9c758f740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x564cf39dfe70 con 0x564cf3a0ae50 2024-04-25T10:10:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.511+0000 7fa9c758f740 10 client.0 _ll_register_callbacks cb 0x564cf3922180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.511+0000 7fa9c1d4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07ef0 0x564cf3a34ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.511+0000 7fa9c1d4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07ef0 0x564cf3a34ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48138/0 (socket says 172.21.15.16:48138) 2024-04-25T10:10:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.511+0000 7fa9c1d4b640 1 -- 172.21.15.16:0/3875442730 learned_addr learned my addr 172.21.15.16:0/3875442730 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.511+0000 7fa9c254c640 1 --2- 172.21.15.16:0/3875442730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf3a07b10 0x564cf3a04de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.511+0000 7fa9c2d4d640 1 --2- 172.21.15.16:0/3875442730 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a0ae50 0x564cf3a0ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.511+0000 7fa9c1d4b640 1 -- 172.21.15.16:0/3875442730 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a0ae50 msgr2=0x564cf3a0ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.511+0000 7fa9c1d4b640 1 --2- 172.21.15.16:0/3875442730 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a0ae50 0x564cf3a0ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa9c1d4b640 1 -- 172.21.15.16:0/3875442730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf3a07b10 msgr2=0x564cf3a04de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa9c1d4b640 1 --2- 172.21.15.16:0/3875442730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf3a07b10 0x564cf3a04de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa9c1d4b640 1 -- 172.21.15.16:0/3875442730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564cf37227d0 con 0x564cf3a07ef0 2024-04-25T10:10:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa9c254c640 1 --2- 172.21.15.16:0/3875442730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf3a07b10 0x564cf3a04de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa9c2d4d640 1 --2- 172.21.15.16:0/3875442730 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a0ae50 0x564cf3a0ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa9c1d4b640 1 --2- 172.21.15.16:0/3875442730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07ef0 0x564cf3a34ee0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa9b802a7d0 tx=0x7fa9b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82d3d59e37c9ab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa99d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:10:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa99d7fa640 1 -- 172.21.15.16:0/3875442730 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b804e070 con 0x564cf3a07ef0 2024-04-25T10:10:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa99d7fa640 1 -- 172.21.15.16:0/3875442730 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fa9b803ed00 con 0x564cf3a07ef0 2024-04-25T10:10:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa9c758f740 1 -- 172.21.15.16:0/3875442730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9b803e460 con 0x564cf3a07ef0 2024-04-25T10:10:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa99d7fa640 1 -- 172.21.15.16:0/3875442730 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9b8059050 con 0x564cf3a07ef0 2024-04-25T10:10:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa9c758f740 10 client.7071 fetch_fsmap learned FSMap version 43 2024-04-25T10:10:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa9c758f740 1 -- 172.21.15.16:0/3875442730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x564cf36583f0 con 0x564cf3a07ef0 2024-04-25T10:10:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa99d7fa640 1 -- 172.21.15.16:0/3875442730 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b803e460 con 0x564cf3a07ef0 2024-04-25T10:10:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa99d7fa640 1 -- 172.21.15.16:0/3875442730 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa9b805c020 con 0x564cf3a07ef0 2024-04-25T10:10:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa9c758f740 10 client.7071 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T10:10:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa9c758f740 10 client.7071 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:10:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa9c758f740 20 client.7071 populate_metadata read hostname 'smithi016' 2024-04-25T10:10:11.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa9c758f740 1 --2- 172.21.15.16:0/3875442730 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x564cf3a0b2d0 0x564cf39de930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa9c758f740 4 client.7071 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:11.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.512+0000 7fa9c758f740 1 -- 172.21.15.16:0/3875442730 --> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564cf39512e0 con 0x564cf3a0b2d0 2024-04-25T10:10:11.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.513+0000 7fa9c254c640 1 --2- 172.21.15.16:0/3875442730 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x564cf3a0b2d0 0x564cf39de930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.513+0000 7fa9c254c640 1 --2- 172.21.15.16:0/3875442730 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x564cf3a0b2d0 0x564cf39de930 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b89fea50b323f8fc server_cookie=92f2de5dc200c373 in_seq=0 out_seq=0 2024-04-25T10:10:11.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.513+0000 7fa99d7fa640 10 client.7071 ms_handle_connect on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:11.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.513+0000 7fa99d7fa640 1 -- 172.21.15.16:0/3875442730 <== mds.0 v2:172.21.15.16:6834/1735811292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564cf39512e0 con 0x564cf3a0b2d0 2024-04-25T10:10:11.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.513+0000 7fa99d7fa640 10 client.7071 handle_command_reply: tid=0 2024-04-25T10:10:11.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.513+0000 7fa9c758f740 1 client.7071 shutdown 2024-04-25T10:10:11.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.513+0000 7fa9c758f740 1 -- 172.21.15.16:0/3875442730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07ef0 msgr2=0x564cf3a34ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.513+0000 7fa9c758f740 1 --2- 172.21.15.16:0/3875442730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07ef0 0x564cf3a34ee0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa9b802a7d0 tx=0x7fa9b8004730 comp rx=0 tx=0).stop 2024-04-25T10:10:11.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.514+0000 7fa9c758f740 1 -- 172.21.15.16:0/3875442730 shutdown_connections 2024-04-25T10:10:11.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.514+0000 7fa9c758f740 1 --2- 172.21.15.16:0/3875442730 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x564cf3a0b2d0 0x564cf39de930 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.514+0000 7fa9c758f740 1 --2- 172.21.15.16:0/3875442730 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a0ae50 0x564cf3a0ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.514+0000 7fa9c758f740 1 --2- 172.21.15.16:0/3875442730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07ef0 0x564cf3a34ee0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.514+0000 7fa99d7fa640 0 client.7071 ms_handle_reset on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:11.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.514+0000 7fa9c758f740 1 --2- 172.21.15.16:0/3875442730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf3a07b10 0x564cf3a04de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.514+0000 7fa9c758f740 1 -- 172.21.15.16:0/3875442730 >> 172.21.15.16:0/3875442730 conn(0x564cf39da770 msgr2=0x564cf3a0a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.514+0000 7fa9c758f740 1 -- 172.21.15.16:0/3875442730 shutdown_connections 2024-04-25T10:10:11.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.514+0000 7fa9c758f740 1 -- 172.21.15.16:0/3875442730 wait complete. 2024-04-25T10:10:11.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.514+0000 7fa9c758f740 20 client.7071 trim_cache size 0 max 16384 2024-04-25T10:10:11.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.529+0000 7fa9c758f740 1 Processor -- start 2024-04-25T10:10:11.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.529+0000 7fa9c758f740 1 -- start start 2024-04-25T10:10:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa9c758f740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a07b10 0x564cf3a22220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa9c758f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf3a07ef0 0x564cf3a86270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa9c758f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a0ae50 0x564cf3a27e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa9c758f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc1ffdb0 con 0x564cf3a0ae50 2024-04-25T10:10:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa9c758f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc1fefe0 con 0x564cf3a07ef0 2024-04-25T10:10:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa9c758f740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc1f4610 con 0x564cf3a07b10 2024-04-25T10:10:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa9c2d4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a0ae50 0x564cf3a27e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa9c2d4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a0ae50 0x564cf3a27e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48140/0 (socket says 172.21.15.16:48140) 2024-04-25T10:10:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa9c2d4d640 1 -- 172.21.15.16:0/2970470678 learned_addr learned my addr 172.21.15.16:0/2970470678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa9c2d4d640 1 -- 172.21.15.16:0/2970470678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a07b10 msgr2=0x564cf3a22220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa9c2d4d640 1 --2- 172.21.15.16:0/2970470678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a07b10 0x564cf3a22220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa9c1d4b640 1 --2- 172.21.15.16:0/2970470678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf3a07ef0 0x564cf3a86270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa9c2d4d640 1 -- 172.21.15.16:0/2970470678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf3a07ef0 msgr2=0x564cf3a86270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa9c2d4d640 1 --2- 172.21.15.16:0/2970470678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf3a07ef0 0x564cf3a86270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa9c2d4d640 1 -- 172.21.15.16:0/2970470678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cf36180f0 con 0x564cf3a0ae50 2024-04-25T10:10:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa9c1d4b640 1 --2- 172.21.15.16:0/2970470678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf3a07ef0 0x564cf3a86270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa9c2d4d640 1 --2- 172.21.15.16:0/2970470678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a0ae50 0x564cf3a27e70 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4007f10 tx=0x7fa9b400f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e617bf23a9b028ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa99f7fe640 1 -- 172.21.15.16:0/2970470678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b400a4c0 con 0x564cf3a0ae50 2024-04-25T10:10:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.530+0000 7fa99f7fe640 1 -- 172.21.15.16:0/2970470678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b400a660 con 0x564cf3a0ae50 2024-04-25T10:10:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.531+0000 7fa99f7fe640 1 -- 172.21.15.16:0/2970470678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b400aa50 con 0x564cf3a0ae50 2024-04-25T10:10:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.531+0000 7fa9c758f740 1 -- 172.21.15.16:0/2970470678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a0ae50 msgr2=0x564cf3a27e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.531+0000 7fa9c758f740 1 --2- 172.21.15.16:0/2970470678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a0ae50 0x564cf3a27e70 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4007f10 tx=0x7fa9b400f820 comp rx=0 tx=0).stop 2024-04-25T10:10:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.531+0000 7fa9c758f740 1 -- 172.21.15.16:0/2970470678 shutdown_connections 2024-04-25T10:10:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.531+0000 7fa9c758f740 1 --2- 172.21.15.16:0/2970470678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a0ae50 0x564cf3a27e70 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.531+0000 7fa9c758f740 1 --2- 172.21.15.16:0/2970470678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf3a07ef0 0x564cf3a86270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.531+0000 7fa9c758f740 1 --2- 172.21.15.16:0/2970470678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a07b10 0x564cf3a22220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.531+0000 7fa9c758f740 1 -- 172.21.15.16:0/2970470678 >> 172.21.15.16:0/2970470678 conn(0x564cf39da770 msgr2=0x564cf39e07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.531+0000 7fa9c758f740 1 -- 172.21.15.16:0/2970470678 shutdown_connections 2024-04-25T10:10:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.531+0000 7fa9c758f740 1 -- 172.21.15.16:0/2970470678 wait complete. 2024-04-25T10:10:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.532+0000 7fa9c758f740 1 -- start start 2024-04-25T10:10:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.532+0000 7fa9c758f740 1 Processor -- start 2024-04-25T10:10:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c758f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07b10 0x564cf3a28c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c758f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf3a07ef0 0x564cf3a1b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c758f740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a0ae50 0x564cf3a1b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c758f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf39dea50 con 0x564cf3a07b10 2024-04-25T10:10:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c758f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc1f7990 con 0x564cf3a07ef0 2024-04-25T10:10:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c758f740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x564cf3a86930 con 0x564cf3a0ae50 2024-04-25T10:10:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c758f740 10 client.0 _ll_register_callbacks cb 0x7fa9b8035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c254c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07b10 0x564cf3a28c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c254c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07b10 0x564cf3a28c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48148/0 (socket says 172.21.15.16:48148) 2024-04-25T10:10:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c254c640 1 -- 172.21.15.16:0/1297597449 learned_addr learned my addr 172.21.15.16:0/1297597449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:11.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c2d4d640 1 --2- 172.21.15.16:0/1297597449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a0ae50 0x564cf3a1b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c1d4b640 1 --2- 172.21.15.16:0/1297597449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf3a07ef0 0x564cf3a1b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c254c640 1 -- 172.21.15.16:0/1297597449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a0ae50 msgr2=0x564cf3a1b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c254c640 1 --2- 172.21.15.16:0/1297597449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a0ae50 0x564cf3a1b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c254c640 1 -- 172.21.15.16:0/1297597449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf3a07ef0 msgr2=0x564cf3a1b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c254c640 1 --2- 172.21.15.16:0/1297597449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf3a07ef0 0x564cf3a1b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c254c640 1 -- 172.21.15.16:0/1297597449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564cf3a291d0 con 0x564cf3a07b10 2024-04-25T10:10:11.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c1d4b640 1 --2- 172.21.15.16:0/1297597449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf3a07ef0 0x564cf3a1b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:11.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c2d4d640 1 --2- 172.21.15.16:0/1297597449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a0ae50 0x564cf3a1b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:11.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c254c640 1 --2- 172.21.15.16:0/1297597449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07b10 0x564cf3a28c90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0587f0 tx=0x7fa9ac004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce652b5b6e97718 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa99d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:10:11.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa99d7fa640 1 -- 172.21.15.16:0/1297597449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac01f040 con 0x564cf3a07b10 2024-04-25T10:10:11.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa99d7fa640 1 -- 172.21.15.16:0/1297597449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fa9ac08a0b0 con 0x564cf3a07b10 2024-04-25T10:10:11.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.533+0000 7fa9c758f740 1 -- 172.21.15.16:0/1297597449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9b400a660 con 0x564cf3a07b10 2024-04-25T10:10:11.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.534+0000 7fa99d7fa640 1 -- 172.21.15.16:0/1297597449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9ac076050 con 0x564cf3a07b10 2024-04-25T10:10:11.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.534+0000 7fa9c758f740 10 client.7077 fetch_fsmap learned FSMap version 43 2024-04-25T10:10:11.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.534+0000 7fa9c758f740 1 -- 172.21.15.16:0/1297597449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x564cf39dbab0 con 0x564cf3a07b10 2024-04-25T10:10:11.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.534+0000 7fa99d7fa640 1 -- 172.21.15.16:0/1297597449 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b400a660 con 0x564cf3a07b10 2024-04-25T10:10:11.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.534+0000 7fa99d7fa640 1 -- 172.21.15.16:0/1297597449 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa9ac055160 con 0x564cf3a07b10 2024-04-25T10:10:11.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.534+0000 7fa9c758f740 10 client.7077 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T10:10:11.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.534+0000 7fa9c758f740 10 client.7077 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:10:11.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.534+0000 7fa9c758f740 20 client.7077 populate_metadata read hostname 'smithi016' 2024-04-25T10:10:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.534+0000 7fa9c758f740 1 --2- 172.21.15.16:0/1297597449 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x564cf3a83330 0x564cf39d9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.534+0000 7fa9c758f740 4 client.7077 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:10:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.534+0000 7fa9c758f740 1 -- 172.21.15.16:0/1297597449 --> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564cf394b630 con 0x564cf3a83330 2024-04-25T10:10:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.534+0000 7fa9c1d4b640 1 --2- 172.21.15.16:0/1297597449 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x564cf3a83330 0x564cf39d9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.534+0000 7fa99d7fa640 10 client.7077 ms_handle_connect on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.534+0000 7fa9c1d4b640 1 --2- 172.21.15.16:0/1297597449 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x564cf3a83330 0x564cf39d9600 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e5fbdca65cc2ee7 server_cookie=f40387115726cbfe in_seq=0 out_seq=0 2024-04-25T10:10:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa99d7fa640 1 -- 172.21.15.16:0/1297597449 <== mds.0 v2:172.21.15.16:6834/1735811292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x564cf394b630 con 0x564cf3a83330 2024-04-25T10:10:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa99d7fa640 10 client.7077 handle_command_reply: tid=0 2024-04-25T10:10:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c758f740 1 client.7077 shutdown 2024-04-25T10:10:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c758f740 1 -- 172.21.15.16:0/1297597449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07b10 msgr2=0x564cf3a28c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c758f740 1 --2- 172.21.15.16:0/1297597449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07b10 0x564cf3a28c90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0587f0 tx=0x7fa9ac004740 comp rx=0 tx=0).stop 2024-04-25T10:10:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c758f740 1 -- 172.21.15.16:0/1297597449 shutdown_connections 2024-04-25T10:10:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c758f740 1 --2- 172.21.15.16:0/1297597449 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x564cf3a83330 0x564cf39d9600 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c758f740 1 --2- 172.21.15.16:0/1297597449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x564cf3a0ae50 0x564cf3a1b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c758f740 1 --2- 172.21.15.16:0/1297597449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cf3a07ef0 0x564cf3a1b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c758f740 1 --2- 172.21.15.16:0/1297597449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cf3a07b10 0x564cf3a28c90 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa99d7fa640 0 client.7077 ms_handle_reset on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c758f740 1 -- 172.21.15.16:0/1297597449 >> 172.21.15.16:0/1297597449 conn(0x564cf39da770 msgr2=0x564cf39d5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c758f740 1 -- 172.21.15.16:0/1297597449 shutdown_connections 2024-04-25T10:10:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c758f740 1 -- 172.21.15.16:0/1297597449 wait complete. 2024-04-25T10:10:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c758f740 20 client.7077 trim_cache size 0 max 16384 2024-04-25T10:10:11.549 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:10:11.550 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:10:11.550 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-04-25T10:10:11.550 INFO:teuthology.orchestra.run.smithi016.stdout: "7f5897fa-e235-4587-b48b-3cdad5206572": { 2024-04-25T10:10:11.550 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/subdir", 2024-04-25T10:10:11.550 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "7f5897fa-e235-4587-b48b-3cdad5206572", 2024-04-25T10:10:11.550 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,repair" 2024-04-25T10:10:11.550 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:11.550 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:11.550 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:10:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c47b0640 1 -- 172.21.15.16:0/2028747645 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa9a006b4d0 msgr2=0x7fa9a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c47b0640 1 --2- 172.21.15.16:0/2028747645 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa9a006b4d0 0x7fa9a006d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8034000 tx=0x7fa9b802e000 comp rx=0 tx=0).stop 2024-04-25T10:10:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c47b0640 1 -- 172.21.15.16:0/2028747645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0cee80 msgr2=0x7fa9bc155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c47b0640 1 --2- 172.21.15.16:0/2028747645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0cee80 0x7fa9bc155f70 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00c7f0 tx=0x7fa9ac00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c47b0640 1 -- 172.21.15.16:0/2028747645 shutdown_connections 2024-04-25T10:10:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c47b0640 1 --2- 172.21.15.16:0/2028747645 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa9a006b4d0 0x7fa9a006d990 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c47b0640 1 --2- 172.21.15.16:0/2028747645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa9bc15dca0 0x7fa9bc152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c47b0640 1 --2- 172.21.15.16:0/2028747645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9bc0d3b90 0x7fa9bc1564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c47b0640 1 --2- 172.21.15.16:0/2028747645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9bc0cee80 0x7fa9bc155f70 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.535+0000 7fa9c47b0640 1 -- 172.21.15.16:0/2028747645 >> 172.21.15.16:0/2028747645 conn(0x7fa9bc0726c0 msgr2=0x7fa9bc0ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.536+0000 7fa9c47b0640 1 -- 172.21.15.16:0/2028747645 shutdown_connections 2024-04-25T10:10:11.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:11.536+0000 7fa9c47b0640 1 -- 172.21.15.16:0/2028747645 wait complete. 2024-04-25T10:10:11.553 INFO:tasks.cephfs.filesystem:scrub status for tag:7f5897fa-e235-4587-b48b-3cdad5206572 - {'path': '/subdir', 'tag': '7f5897fa-e235-4587-b48b-3cdad5206572', 'options': 'recursive,repair'} 2024-04-25T10:10:38.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:38.017 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:41.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:41.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:10:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.667+0000 7f98caf9b640 1 Processor -- start 2024-04-25T10:10:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.667+0000 7f98caf9b640 1 -- start start 2024-04-25T10:10:41.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.668+0000 7f98caf9b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f98c40b42c0 0x7f98c415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.668+0000 7f98caf9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98c415aef0 0x7f98c40cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.668+0000 7f98caf9b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c415b370 con 0x7f98c40b46a0 2024-04-25T10:10:41.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.668+0000 7f98caf9b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c40d05d0 con 0x7f98c415aef0 2024-04-25T10:10:41.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.668+0000 7f98caf9b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c415b650 con 0x7f98c40b42c0 2024-04-25T10:10:41.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.668+0000 7f98c3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98c415aef0 0x7f98c40cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.668+0000 7f98c9538640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f98c40b46a0 0x7f98c41549d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52386/0 (socket says 172.21.15.16:52386) 2024-04-25T10:10:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.668+0000 7f98c9538640 1 -- 172.21.15.16:0/3339605410 learned_addr learned my addr 172.21.15.16:0/3339605410 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.668+0000 7f98c8d37640 1 --2- 172.21.15.16:0/3339605410 >> v2:172.21.15.186:3300/0 conn(0x7f98c40b42c0 0x7f98c415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.668+0000 7f98c3fff640 1 -- 172.21.15.16:0/3339605410 >> v2:172.21.15.186:3300/0 conn(0x7f98c40b42c0 msgr2=0x7f98c415ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.668+0000 7f98c3fff640 1 --2- 172.21.15.16:0/3339605410 >> v2:172.21.15.186:3300/0 conn(0x7f98c40b42c0 0x7f98c415ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.668+0000 7f98c3fff640 1 -- 172.21.15.16:0/3339605410 >> v1:172.21.15.16:6789/0 conn(0x7f98c40b46a0 legacy=0x7f98c41549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.668+0000 7f98c3fff640 1 -- 172.21.15.16:0/3339605410 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c415b930 con 0x7f98c415aef0 2024-04-25T10:10:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.668+0000 7f98c37fe640 1 -- 172.21.15.16:0/3339605410 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059038162 0 0) 0x7f98c415b370 con 0x7f98c40b46a0 2024-04-25T10:10:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.669+0000 7f98c3fff640 1 --2- 172.21.15.16:0/3339605410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c415aef0 0x7f98c40cf440 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f98c40d07c0 tx=0x7f98b002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519b77a7a52ca739 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.669+0000 7f98c37fe640 1 -- 172.21.15.16:0/3339605410 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0037070 con 0x7f98c415aef0 2024-04-25T10:10:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.669+0000 7f98c37fe640 1 -- 172.21.15.16:0/3339605410 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b0032440 con 0x7f98c415aef0 2024-04-25T10:10:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.669+0000 7f98c37fe640 1 -- 172.21.15.16:0/3339605410 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b002ae60 con 0x7f98c415aef0 2024-04-25T10:10:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.670+0000 7f98caf9b640 1 -- 172.21.15.16:0/3339605410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c415aef0 msgr2=0x7f98c40cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.670+0000 7f98caf9b640 1 --2- 172.21.15.16:0/3339605410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c415aef0 0x7f98c40cf440 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f98c40d07c0 tx=0x7f98b002a110 comp rx=0 tx=0).stop 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.670+0000 7f98caf9b640 1 -- 172.21.15.16:0/3339605410 shutdown_connections 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.670+0000 7f98caf9b640 1 --2- 172.21.15.16:0/3339605410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c415aef0 0x7f98c40cf440 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.670+0000 7f98caf9b640 1 --2- 172.21.15.16:0/3339605410 >> v2:172.21.15.186:3300/0 conn(0x7f98c40b42c0 0x7f98c415ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.670+0000 7f98caf9b640 1 -- 172.21.15.16:0/3339605410 >> 172.21.15.16:0/3339605410 conn(0x7f98c406aba0 msgr2=0x7f98c406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.670+0000 7f98caf9b640 1 -- 172.21.15.16:0/3339605410 shutdown_connections 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.670+0000 7f98caf9b640 1 -- 172.21.15.16:0/3339605410 wait complete. 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.671+0000 7f98caf9b640 1 Processor -- start 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.671+0000 7f98caf9b640 1 -- start start 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.672+0000 7f98caf9b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98c40b42c0 0x7f98c41e3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.672+0000 7f98caf9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c40b46a0 0x7f98c41e4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.672+0000 7f98caf9b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98c415aef0 0x7f98c41e4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.672+0000 7f98caf9b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c406eae0 con 0x7f98c40b46a0 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.672+0000 7f98caf9b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c40cf980 con 0x7f98c40b42c0 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.672+0000 7f98caf9b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c406ec60 con 0x7f98c415aef0 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.672+0000 7f98c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c40b46a0 0x7f98c41e4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.672+0000 7f98c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c40b46a0 0x7f98c41e4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57188/0 (socket says 172.21.15.16:57188) 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.672+0000 7f98c3fff640 1 -- 172.21.15.16:0/3351169871 learned_addr learned my addr 172.21.15.16:0/3351169871 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.673+0000 7f98c8d37640 1 --2- 172.21.15.16:0/3351169871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98c40b42c0 0x7f98c41e3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.673+0000 7f98c9538640 1 --2- 172.21.15.16:0/3351169871 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98c415aef0 0x7f98c41e4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.673+0000 7f98c3fff640 1 -- 172.21.15.16:0/3351169871 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98c415aef0 msgr2=0x7f98c41e4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.673+0000 7f98c3fff640 1 --2- 172.21.15.16:0/3351169871 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98c415aef0 0x7f98c41e4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.673+0000 7f98c3fff640 1 -- 172.21.15.16:0/3351169871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98c40b42c0 msgr2=0x7f98c41e3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.673+0000 7f98c3fff640 1 --2- 172.21.15.16:0/3351169871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98c40b42c0 0x7f98c41e3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.673+0000 7f98c3fff640 1 -- 172.21.15.16:0/3351169871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c4155ec0 con 0x7f98c40b46a0 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.673+0000 7f98c8d37640 1 --2- 172.21.15.16:0/3351169871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98c40b42c0 0x7f98c41e3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.673+0000 7f98c9538640 1 --2- 172.21.15.16:0/3351169871 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98c415aef0 0x7f98c41e4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.673+0000 7f98c3fff640 1 --2- 172.21.15.16:0/3351169871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c40b46a0 0x7f98c41e4130 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f98b0000c00 tx=0x7f98b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.673+0000 7f98c17fa640 1 -- 172.21.15.16:0/3351169871 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0037050 con 0x7f98c40b46a0 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.673+0000 7f98c17fa640 1 -- 172.21.15.16:0/3351169871 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b0038040 con 0x7f98c40b46a0 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.673+0000 7f98c17fa640 1 -- 172.21.15.16:0/3351169871 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0006e10 con 0x7f98c40b46a0 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.673+0000 7f98caf9b640 1 -- 172.21.15.16:0/3351169871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c41561a0 con 0x7f98c40b46a0 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.673+0000 7f98caf9b640 1 -- 172.21.15.16:0/3351169871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c4151820 con 0x7f98c40b46a0 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.675+0000 7f98c17fa640 1 -- 172.21.15.16:0/3351169871 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f98b0030980 con 0x7f98c40b46a0 2024-04-25T10:10:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.675+0000 7f98c17fa640 1 --2- 172.21.15.16:0/3351169871 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f98a806b4d0 0x7f98a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.676+0000 7f98c17fa640 1 -- 172.21.15.16:0/3351169871 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f98b00aafa0 con 0x7f98c40b46a0 2024-04-25T10:10:41.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.676+0000 7f98c8d37640 1 --2- 172.21.15.16:0/3351169871 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f98a806b4d0 0x7f98a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.676+0000 7f98c8d37640 1 --2- 172.21.15.16:0/3351169871 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f98a806b4d0 0x7f98a806d990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f98b8006fd0 tx=0x7f98b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.677+0000 7f98cdd78740 1 Processor -- start 2024-04-25T10:10:41.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.677+0000 7f98cdd78740 1 -- start start 2024-04-25T10:10:41.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.677+0000 7f98cdd78740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee7c510 0x56431ee7c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.677+0000 7f98cdd78740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee4c770 0x56431ee527a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.677+0000 7f98cdd78740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee98470 0x56431ee988f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.677+0000 7f98cdd78740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431edb5da0 con 0x56431ee4c770 2024-04-25T10:10:41.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.677+0000 7f98cdd78740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431ed5eb80 con 0x56431ee98470 2024-04-25T10:10:41.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.677+0000 7f98cdd78740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56431edb1e40 con 0x56431ee7c510 2024-04-25T10:10:41.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.678+0000 7f98c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee4c770 0x56431ee527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.678+0000 7f98c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee4c770 0x56431ee527a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57196/0 (socket says 172.21.15.16:57196) 2024-04-25T10:10:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.678+0000 7f98c3fff640 1 -- 172.21.15.16:0/1033915683 learned_addr learned my addr 172.21.15.16:0/1033915683 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.678+0000 7f98c8d37640 1 --2- 172.21.15.16:0/1033915683 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee7c510 0x56431ee7c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.678+0000 7f98c9538640 1 --2- 172.21.15.16:0/1033915683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee98470 0x56431ee988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.678+0000 7f98c3fff640 1 -- 172.21.15.16:0/1033915683 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee7c510 msgr2=0x56431ee7c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.678+0000 7f98c3fff640 1 --2- 172.21.15.16:0/1033915683 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee7c510 0x56431ee7c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.678+0000 7f98c3fff640 1 -- 172.21.15.16:0/1033915683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee98470 msgr2=0x56431ee988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.678+0000 7f98c3fff640 1 --2- 172.21.15.16:0/1033915683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee98470 0x56431ee988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.678+0000 7f98c3fff640 1 -- 172.21.15.16:0/1033915683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56431ed68f10 con 0x56431ee4c770 2024-04-25T10:10:41.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.678+0000 7f98c3fff640 1 --2- 172.21.15.16:0/1033915683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee4c770 0x56431ee527a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f98b00a9390 tx=0x7f98b0009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad2f18d7298083a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.678+0000 7f98ae7fc640 1 -- 172.21.15.16:0/1033915683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0079e90 con 0x56431ee4c770 2024-04-25T10:10:41.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.678+0000 7f98ae7fc640 1 -- 172.21.15.16:0/1033915683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b00a7070 con 0x56431ee4c770 2024-04-25T10:10:41.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.678+0000 7f98ae7fc640 1 -- 172.21.15.16:0/1033915683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0079790 con 0x56431ee4c770 2024-04-25T10:10:41.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.679+0000 7f98cdd78740 1 -- 172.21.15.16:0/1033915683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee4c770 msgr2=0x56431ee527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.679+0000 7f98cdd78740 1 --2- 172.21.15.16:0/1033915683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee4c770 0x56431ee527a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f98b00a9390 tx=0x7f98b0009520 comp rx=0 tx=0).stop 2024-04-25T10:10:41.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.679+0000 7f98cdd78740 1 -- 172.21.15.16:0/1033915683 shutdown_connections 2024-04-25T10:10:41.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.679+0000 7f98cdd78740 1 --2- 172.21.15.16:0/1033915683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee98470 0x56431ee988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.679+0000 7f98cdd78740 1 --2- 172.21.15.16:0/1033915683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee4c770 0x56431ee527a0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.679+0000 7f98cdd78740 1 --2- 172.21.15.16:0/1033915683 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee7c510 0x56431ee7c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.679+0000 7f98cdd78740 1 -- 172.21.15.16:0/1033915683 >> 172.21.15.16:0/1033915683 conn(0x56431ee4e9e0 msgr2=0x56431ee97320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.679+0000 7f98cdd78740 1 -- 172.21.15.16:0/1033915683 shutdown_connections 2024-04-25T10:10:41.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.679+0000 7f98cdd78740 1 -- 172.21.15.16:0/1033915683 wait complete. 2024-04-25T10:10:41.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.679+0000 7f98cdd78740 1 -- start start 2024-04-25T10:10:41.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.680+0000 7f98cdd78740 1 Processor -- start 2024-04-25T10:10:41.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.680+0000 7f98cdd78740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee7c510 0x56431ee8eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.680+0000 7f98cdd78740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee98470 0x56431ee965a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.680+0000 7f98cdd78740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee6dee0 0x56431ee96ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.680+0000 7f98cdd78740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431edb5da0 con 0x56431ee7c510 2024-04-25T10:10:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.680+0000 7f98cdd78740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431ed544b0 con 0x56431ee6dee0 2024-04-25T10:10:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.680+0000 7f98cdd78740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56431ee52ce0 con 0x56431ee98470 2024-04-25T10:10:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.680+0000 7f98cdd78740 10 client.0 _ll_register_callbacks cb 0x56431ed94180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:41.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98c8d37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee7c510 0x56431ee8eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98c8d37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee7c510 0x56431ee8eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57210/0 (socket says 172.21.15.16:57210) 2024-04-25T10:10:41.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98c8d37640 1 -- 172.21.15.16:0/2142514868 learned_addr learned my addr 172.21.15.16:0/2142514868 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:41.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98c9538640 1 --2- 172.21.15.16:0/2142514868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee6dee0 0x56431ee96ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98c3fff640 1 --2- 172.21.15.16:0/2142514868 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee98470 0x56431ee965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98c8d37640 1 -- 172.21.15.16:0/2142514868 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee98470 msgr2=0x56431ee965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98c8d37640 1 --2- 172.21.15.16:0/2142514868 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee98470 0x56431ee965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98c8d37640 1 -- 172.21.15.16:0/2142514868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee6dee0 msgr2=0x56431ee96ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98c8d37640 1 --2- 172.21.15.16:0/2142514868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee6dee0 0x56431ee96ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98c8d37640 1 -- 172.21.15.16:0/2142514868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56431eb947d0 con 0x56431ee7c510 2024-04-25T10:10:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98c9538640 1 --2- 172.21.15.16:0/2142514868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee6dee0 0x56431ee96ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98c8d37640 1 --2- 172.21.15.16:0/2142514868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee7c510 0x56431ee8eca0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f98b8006c00 tx=0x7f98b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13920ed465cf5ca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:10:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98a37fe640 1 -- 172.21.15.16:0/2142514868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b8027060 con 0x56431ee7c510 2024-04-25T10:10:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98cdd78740 1 -- 172.21.15.16:0/2142514868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98b00a7070 con 0x56431ee7c510 2024-04-25T10:10:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98a37fe640 1 -- 172.21.15.16:0/2142514868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f98b8013430 con 0x56431ee7c510 2024-04-25T10:10:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98a37fe640 1 -- 172.21.15.16:0/2142514868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98b802d050 con 0x56431ee7c510 2024-04-25T10:10:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98cdd78740 10 client.7092 fetch_fsmap learned FSMap version 44 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.681+0000 7f98cdd78740 1 -- 172.21.15.16:0/2142514868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x56431eaca3f0 con 0x56431ee7c510 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.682+0000 7f98a37fe640 1 -- 172.21.15.16:0/2142514868 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b00a7070 con 0x56431ee7c510 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.682+0000 7f98a37fe640 1 -- 172.21.15.16:0/2142514868 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f98b8030020 con 0x56431ee7c510 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.682+0000 7f98cdd78740 10 client.7092 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.682+0000 7f98cdd78740 10 client.7092 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.682+0000 7f98cdd78740 20 client.7092 populate_metadata read hostname 'smithi016' 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.682+0000 7f98cdd78740 1 --2- 172.21.15.16:0/2142514868 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x56431ee7d420 0x56431ee7d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.682+0000 7f98cdd78740 4 client.7092 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.682+0000 7f98cdd78740 1 -- 172.21.15.16:0/2142514868 --> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56431edc32e0 con 0x56431ee7d420 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.682+0000 7f98c3fff640 1 --2- 172.21.15.16:0/2142514868 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x56431ee7d420 0x56431ee7d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.682+0000 7f98a37fe640 10 client.7092 ms_handle_connect on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.682+0000 7f98c3fff640 1 --2- 172.21.15.16:0/2142514868 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x56431ee7d420 0x56431ee7d830 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2349516037d64d3e server_cookie=1f92c25e39e2ecdb in_seq=0 out_seq=0 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.683+0000 7f98a37fe640 1 -- 172.21.15.16:0/2142514868 <== mds.0 v2:172.21.15.16:6834/1735811292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56431edc32e0 con 0x56431ee7d420 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.683+0000 7f98a37fe640 10 client.7092 handle_command_reply: tid=0 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.683+0000 7f98cdd78740 1 client.7092 shutdown 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.683+0000 7f98cdd78740 1 -- 172.21.15.16:0/2142514868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee7c510 msgr2=0x56431ee8eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.683+0000 7f98cdd78740 1 --2- 172.21.15.16:0/2142514868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee7c510 0x56431ee8eca0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f98b8006c00 tx=0x7f98b80046c0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.683+0000 7f98cdd78740 1 -- 172.21.15.16:0/2142514868 shutdown_connections 2024-04-25T10:10:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.683+0000 7f98cdd78740 1 --2- 172.21.15.16:0/2142514868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee6dee0 0x56431ee96ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.683+0000 7f98cdd78740 1 --2- 172.21.15.16:0/2142514868 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x56431ee7d420 0x56431ee7d830 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.683+0000 7f98cdd78740 1 --2- 172.21.15.16:0/2142514868 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee98470 0x56431ee965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.683+0000 7f98cdd78740 1 --2- 172.21.15.16:0/2142514868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee7c510 0x56431ee8eca0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.683+0000 7f98a37fe640 0 client.7092 ms_handle_reset on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.683+0000 7f98cdd78740 1 -- 172.21.15.16:0/2142514868 >> 172.21.15.16:0/2142514868 conn(0x56431ee4c770 msgr2=0x56431ee802b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.683+0000 7f98cdd78740 1 -- 172.21.15.16:0/2142514868 shutdown_connections 2024-04-25T10:10:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.683+0000 7f98cdd78740 1 -- 172.21.15.16:0/2142514868 wait complete. 2024-04-25T10:10:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.683+0000 7f98cdd78740 20 client.7092 trim_cache size 0 max 16384 2024-04-25T10:10:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.699+0000 7f98cdd78740 1 Processor -- start 2024-04-25T10:10:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.699+0000 7f98cdd78740 1 -- start start 2024-04-25T10:10:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.699+0000 7f98cdd78740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee6dee0 0x56431ee8e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.699+0000 7f98cdd78740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee7c510 0x56431ee8e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.699+0000 7f98cdd78740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee98470 0x56431ee6f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.699+0000 7f98cdd78740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c4140cb0 con 0x56431ee7c510 2024-04-25T10:10:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.699+0000 7f98cdd78740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c413fe70 con 0x56431ee98470 2024-04-25T10:10:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.699+0000 7f98cdd78740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c40c5b90 con 0x56431ee6dee0 2024-04-25T10:10:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.699+0000 7f98c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee7c510 0x56431ee8e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.700+0000 7f98c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee7c510 0x56431ee8e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57214/0 (socket says 172.21.15.16:57214) 2024-04-25T10:10:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.700+0000 7f98c3fff640 1 -- 172.21.15.16:0/1260005674 learned_addr learned my addr 172.21.15.16:0/1260005674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.700+0000 7f98c8d37640 1 --2- 172.21.15.16:0/1260005674 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee6dee0 0x56431ee8e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.700+0000 7f98c9538640 1 --2- 172.21.15.16:0/1260005674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee98470 0x56431ee6f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.700+0000 7f98c3fff640 1 -- 172.21.15.16:0/1260005674 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee6dee0 msgr2=0x56431ee8e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.700+0000 7f98c3fff640 1 --2- 172.21.15.16:0/1260005674 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee6dee0 0x56431ee8e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.700+0000 7f98c3fff640 1 -- 172.21.15.16:0/1260005674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee98470 msgr2=0x56431ee6f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.700+0000 7f98c3fff640 1 --2- 172.21.15.16:0/1260005674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee98470 0x56431ee6f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.700+0000 7f98c3fff640 1 -- 172.21.15.16:0/1260005674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56431ea8a0f0 con 0x56431ee7c510 2024-04-25T10:10:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.700+0000 7f98c9538640 1 --2- 172.21.15.16:0/1260005674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee98470 0x56431ee6f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.700+0000 7f98c3fff640 1 --2- 172.21.15.16:0/1260005674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee7c510 0x56431ee8e840 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f98b0081970 tx=0x7f98b007d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50548dd57d069936 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.700+0000 7f98adffb640 1 -- 172.21.15.16:0/1260005674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b007d8e0 con 0x56431ee7c510 2024-04-25T10:10:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.700+0000 7f98adffb640 1 -- 172.21.15.16:0/1260005674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b00a7070 con 0x56431ee7c510 2024-04-25T10:10:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.700+0000 7f98adffb640 1 -- 172.21.15.16:0/1260005674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0081ea0 con 0x56431ee7c510 2024-04-25T10:10:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.701+0000 7f98cdd78740 1 -- 172.21.15.16:0/1260005674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee7c510 msgr2=0x56431ee8e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.701+0000 7f98cdd78740 1 --2- 172.21.15.16:0/1260005674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee7c510 0x56431ee8e840 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f98b0081970 tx=0x7f98b007d000 comp rx=0 tx=0).stop 2024-04-25T10:10:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.701+0000 7f98cdd78740 1 -- 172.21.15.16:0/1260005674 shutdown_connections 2024-04-25T10:10:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.701+0000 7f98cdd78740 1 --2- 172.21.15.16:0/1260005674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee98470 0x56431ee6f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.701+0000 7f98cdd78740 1 --2- 172.21.15.16:0/1260005674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee7c510 0x56431ee8e840 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.701+0000 7f98cdd78740 1 --2- 172.21.15.16:0/1260005674 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee6dee0 0x56431ee8e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.701+0000 7f98cdd78740 1 -- 172.21.15.16:0/1260005674 >> 172.21.15.16:0/1260005674 conn(0x56431ee4c770 msgr2=0x56431ee47b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.701+0000 7f98cdd78740 1 -- 172.21.15.16:0/1260005674 shutdown_connections 2024-04-25T10:10:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.701+0000 7f98cdd78740 1 -- 172.21.15.16:0/1260005674 wait complete. 2024-04-25T10:10:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.701+0000 7f98cdd78740 1 -- start start 2024-04-25T10:10:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.702+0000 7f98cdd78740 1 Processor -- start 2024-04-25T10:10:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.702+0000 7f98cdd78740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee6dee0 0x56431ee8f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.702+0000 7f98cdd78740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee7c510 0x56431ee78c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.702+0000 7f98cdd78740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee98470 0x56431ee96500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.702+0000 7f98cdd78740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431ee4e6c0 con 0x56431ee6dee0 2024-04-25T10:10:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.702+0000 7f98cdd78740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c40c6190 con 0x56431ee98470 2024-04-25T10:10:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.702+0000 7f98cdd78740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56431ee53180 con 0x56431ee7c510 2024-04-25T10:10:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.702+0000 7f98cdd78740 10 client.0 _ll_register_callbacks cb 0x7f98b0075ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98c8d37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee6dee0 0x56431ee8f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98c8d37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee6dee0 0x56431ee8f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57216/0 (socket says 172.21.15.16:57216) 2024-04-25T10:10:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98c8d37640 1 -- 172.21.15.16:0/7567812 learned_addr learned my addr 172.21.15.16:0/7567812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98c3fff640 1 --2- 172.21.15.16:0/7567812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee7c510 0x56431ee78c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98c9538640 1 --2- 172.21.15.16:0/7567812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee98470 0x56431ee96500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98c8d37640 1 -- 172.21.15.16:0/7567812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee7c510 msgr2=0x56431ee78c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98c8d37640 1 --2- 172.21.15.16:0/7567812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee7c510 0x56431ee78c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98c8d37640 1 -- 172.21.15.16:0/7567812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee98470 msgr2=0x56431ee96500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98c8d37640 1 --2- 172.21.15.16:0/7567812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee98470 0x56431ee96500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98c8d37640 1 -- 172.21.15.16:0/7567812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56431eeadb20 con 0x56431ee6dee0 2024-04-25T10:10:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98c9538640 1 --2- 172.21.15.16:0/7567812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee98470 0x56431ee96500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98c8d37640 1 --2- 172.21.15.16:0/7567812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee6dee0 0x56431ee8f9c0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f98b800f010 tx=0x7f98b8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a56a4939343bbdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:10:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98a37fe640 1 -- 172.21.15.16:0/7567812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b800f040 con 0x56431ee6dee0 2024-04-25T10:10:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98a37fe640 1 -- 172.21.15.16:0/7567812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f98b803a440 con 0x56431ee6dee0 2024-04-25T10:10:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98cdd78740 1 -- 172.21.15.16:0/7567812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98b00a7070 con 0x56431ee6dee0 2024-04-25T10:10:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98a37fe640 1 -- 172.21.15.16:0/7567812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98b8042050 con 0x56431ee6dee0 2024-04-25T10:10:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98cdd78740 10 client.7098 fetch_fsmap learned FSMap version 44 2024-04-25T10:10:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.703+0000 7f98cdd78740 1 -- 172.21.15.16:0/7567812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x56431ee791a0 con 0x56431ee6dee0 2024-04-25T10:10:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98a37fe640 1 -- 172.21.15.16:0/7567812 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b00a7070 con 0x56431ee6dee0 2024-04-25T10:10:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98a37fe640 1 -- 172.21.15.16:0/7567812 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f98b8045020 con 0x56431ee6dee0 2024-04-25T10:10:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98cdd78740 10 client.7098 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T10:10:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98cdd78740 10 client.7098 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:10:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98cdd78740 20 client.7098 populate_metadata read hostname 'smithi016' 2024-04-25T10:10:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98cdd78740 1 --2- 172.21.15.16:0/7567812 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x56431ee96a40 0x56431ef092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98cdd78740 4 client.7098 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:10:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98cdd78740 1 -- 172.21.15.16:0/7567812 --> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56431edbd630 con 0x56431ee96a40 2024-04-25T10:10:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98c3fff640 1 --2- 172.21.15.16:0/7567812 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x56431ee96a40 0x56431ef092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98a37fe640 10 client.7098 ms_handle_connect on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98c3fff640 1 --2- 172.21.15.16:0/7567812 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x56431ee96a40 0x56431ef092c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=397c1fa5a213d640 server_cookie=c7c2f6bc82a12562 in_seq=0 out_seq=0 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98a37fe640 1 -- 172.21.15.16:0/7567812 <== mds.0 v2:172.21.15.16:6834/1735811292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56431edbd630 con 0x56431ee96a40 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98a37fe640 10 client.7098 handle_command_reply: tid=0 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98cdd78740 1 client.7098 shutdown 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98cdd78740 1 -- 172.21.15.16:0/7567812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee6dee0 msgr2=0x56431ee8f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98cdd78740 1 --2- 172.21.15.16:0/7567812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee6dee0 0x56431ee8f9c0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f98b800f010 tx=0x7f98b8013000 comp rx=0 tx=0).stop 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98cdd78740 1 -- 172.21.15.16:0/7567812 shutdown_connections 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98cdd78740 1 --2- 172.21.15.16:0/7567812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431ee98470 0x56431ee96500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98cdd78740 1 --2- 172.21.15.16:0/7567812 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x56431ee96a40 0x56431ef092c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98cdd78740 1 --2- 172.21.15.16:0/7567812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56431ee7c510 0x56431ee78c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98cdd78740 1 --2- 172.21.15.16:0/7567812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431ee6dee0 0x56431ee8f9c0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98a37fe640 0 client.7098 ms_handle_reset on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:41.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.704+0000 7f98cdd78740 1 -- 172.21.15.16:0/7567812 >> 172.21.15.16:0/7567812 conn(0x56431ee4c770 msgr2=0x56431ee8e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.705+0000 7f98cdd78740 1 -- 172.21.15.16:0/7567812 shutdown_connections 2024-04-25T10:10:41.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.705+0000 7f98cdd78740 1 -- 172.21.15.16:0/7567812 wait complete. 2024-04-25T10:10:41.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.705+0000 7f98cdd78740 20 client.7098 trim_cache size 0 max 16384 2024-04-25T10:10:41.727 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:10:41.727 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-04-25T10:10:41.727 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-04-25T10:10:41.727 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:10:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.705+0000 7f98caf9b640 1 -- 172.21.15.16:0/3351169871 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f98a806b4d0 msgr2=0x7f98a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.705+0000 7f98caf9b640 1 --2- 172.21.15.16:0/3351169871 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f98a806b4d0 0x7f98a806d990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f98b8006fd0 tx=0x7f98b8008040 comp rx=0 tx=0).stop 2024-04-25T10:10:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.705+0000 7f98caf9b640 1 -- 172.21.15.16:0/3351169871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c40b46a0 msgr2=0x7f98c41e4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.705+0000 7f98caf9b640 1 --2- 172.21.15.16:0/3351169871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c40b46a0 0x7f98c41e4130 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f98b0000c00 tx=0x7f98b00046c0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.705+0000 7f98caf9b640 1 -- 172.21.15.16:0/3351169871 shutdown_connections 2024-04-25T10:10:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.705+0000 7f98caf9b640 1 --2- 172.21.15.16:0/3351169871 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f98a806b4d0 0x7f98a806d990 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.705+0000 7f98caf9b640 1 --2- 172.21.15.16:0/3351169871 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98c415aef0 0x7f98c41e4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.705+0000 7f98caf9b640 1 --2- 172.21.15.16:0/3351169871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c40b46a0 0x7f98c41e4130 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.705+0000 7f98caf9b640 1 --2- 172.21.15.16:0/3351169871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98c40b42c0 0x7f98c41e3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.705+0000 7f98caf9b640 1 -- 172.21.15.16:0/3351169871 >> 172.21.15.16:0/3351169871 conn(0x7f98c406aba0 msgr2=0x7f98c415a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.705+0000 7f98caf9b640 1 -- 172.21.15.16:0/3351169871 shutdown_connections 2024-04-25T10:10:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.705+0000 7f98caf9b640 1 -- 172.21.15.16:0/3351169871 wait complete. 2024-04-25T10:10:41.730 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:10:41.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.827+0000 7fb0357f9640 1 Processor -- start 2024-04-25T10:10:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.827+0000 7fb0357f9640 1 -- start start 2024-04-25T10:10:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.828+0000 7fb0357f9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb030099a40 0x7fb03009a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.828+0000 7fb0357f9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb03015bd40 0x7fb0300cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.828+0000 7fb0357f9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0300cf910 con 0x7fb030099e20 2024-04-25T10:10:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.828+0000 7fb0357f9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb03015cfd0 con 0x7fb030099a40 2024-04-25T10:10:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.828+0000 7fb0357f9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0300cfb90 con 0x7fb03015bd40 2024-04-25T10:10:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.828+0000 7fb02effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb030099a40 0x7fb03009a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.828+0000 7fb02effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb030099a40 0x7fb03009a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57222/0 (socket says 172.21.15.16:57222) 2024-04-25T10:10:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.828+0000 7fb02effd640 1 -- 172.21.15.16:0/795367084 learned_addr learned my addr 172.21.15.16:0/795367084 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.828+0000 7fb02effd640 1 -- 172.21.15.16:0/795367084 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb03015bd40 msgr2=0x7fb0300cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.828+0000 7fb02e7fc640 1 --2- 172.21.15.16:0/795367084 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb03015bd40 0x7fb0300cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.828+0000 7fb02effd640 1 --2- 172.21.15.16:0/795367084 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb03015bd40 0x7fb0300cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.829+0000 7fb02effd640 1 -- 172.21.15.16:0/795367084 >> v1:172.21.15.186:6789/0 conn(0x7fb030099e20 legacy=0x7fb03015f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.829+0000 7fb02effd640 1 -- 172.21.15.16:0/795367084 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03015c210 con 0x7fb030099a40 2024-04-25T10:10:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.829+0000 7fb02effd640 1 --2- 172.21.15.16:0/795367084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb030099a40 0x7fb03009a9c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb028009870 tx=0x7fb02802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b8ef4255e693cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.829+0000 7fb02dffb640 1 -- 172.21.15.16:0/795367084 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb028037070 con 0x7fb030099a40 2024-04-25T10:10:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.829+0000 7fb02dffb640 1 -- 172.21.15.16:0/795367084 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb028032440 con 0x7fb030099a40 2024-04-25T10:10:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.829+0000 7fb02dffb640 1 -- 172.21.15.16:0/795367084 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb028032810 con 0x7fb030099a40 2024-04-25T10:10:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.830+0000 7fb0357f9640 1 -- 172.21.15.16:0/795367084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb030099a40 msgr2=0x7fb03009a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.830+0000 7fb0357f9640 1 --2- 172.21.15.16:0/795367084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb030099a40 0x7fb03009a9c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb028009870 tx=0x7fb02802a180 comp rx=0 tx=0).stop 2024-04-25T10:10:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.830+0000 7fb0357f9640 1 -- 172.21.15.16:0/795367084 shutdown_connections 2024-04-25T10:10:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.830+0000 7fb0357f9640 1 --2- 172.21.15.16:0/795367084 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb03015bd40 0x7fb0300cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.830+0000 7fb0357f9640 1 --2- 172.21.15.16:0/795367084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb030099a40 0x7fb03009a9c0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.830+0000 7fb0357f9640 1 -- 172.21.15.16:0/795367084 >> 172.21.15.16:0/795367084 conn(0x7fb0300b32f0 msgr2=0x7fb0300b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.830+0000 7fb0357f9640 1 -- 172.21.15.16:0/795367084 shutdown_connections 2024-04-25T10:10:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.830+0000 7fb0357f9640 1 -- 172.21.15.16:0/795367084 wait complete. 2024-04-25T10:10:41.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.831+0000 7fb0357f9640 1 Processor -- start 2024-04-25T10:10:41.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.831+0000 7fb0357f9640 1 -- start start 2024-04-25T10:10:41.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.832+0000 7fb0357f9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb030099a40 0x7fb0301e6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.832+0000 7fb0357f9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb030099e20 0x7fb0300c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.832+0000 7fb0357f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03015bd40 0x7fb0300c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.832+0000 7fb0357f9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03015cab0 con 0x7fb03015bd40 2024-04-25T10:10:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.832+0000 7fb0357f9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03015c7d0 con 0x7fb030099a40 2024-04-25T10:10:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.832+0000 7fb0357f9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03009a2b0 con 0x7fb030099e20 2024-04-25T10:10:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.832+0000 7fb02f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03015bd40 0x7fb0300c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.832+0000 7fb02f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03015bd40 0x7fb0300c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57230/0 (socket says 172.21.15.16:57230) 2024-04-25T10:10:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.832+0000 7fb02f7fe640 1 -- 172.21.15.16:0/1878295110 learned_addr learned my addr 172.21.15.16:0/1878295110 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.832+0000 7fb02effd640 1 --2- 172.21.15.16:0/1878295110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb030099a40 0x7fb0301e6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.832+0000 7fb02e7fc640 1 --2- 172.21.15.16:0/1878295110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb030099e20 0x7fb0300c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.833+0000 7fb02f7fe640 1 -- 172.21.15.16:0/1878295110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb030099e20 msgr2=0x7fb0300c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.833+0000 7fb02f7fe640 1 --2- 172.21.15.16:0/1878295110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb030099e20 0x7fb0300c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.833+0000 7fb02f7fe640 1 -- 172.21.15.16:0/1878295110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb030099a40 msgr2=0x7fb0301e6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.833+0000 7fb02f7fe640 1 --2- 172.21.15.16:0/1878295110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb030099a40 0x7fb0301e6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.833+0000 7fb02f7fe640 1 -- 172.21.15.16:0/1878295110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0301e5480 con 0x7fb03015bd40 2024-04-25T10:10:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.833+0000 7fb02f7fe640 1 --2- 172.21.15.16:0/1878295110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03015bd40 0x7fb0300c6050 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb02000b970 tx=0x7fb02000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.833+0000 7fb0277fe640 1 -- 172.21.15.16:0/1878295110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020019070 con 0x7fb03015bd40 2024-04-25T10:10:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.833+0000 7fb0277fe640 1 -- 172.21.15.16:0/1878295110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02000cb40 con 0x7fb03015bd40 2024-04-25T10:10:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.833+0000 7fb0357f9640 1 -- 172.21.15.16:0/1878295110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0300c6590 con 0x7fb03015bd40 2024-04-25T10:10:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.833+0000 7fb0277fe640 1 -- 172.21.15.16:0/1878295110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020014440 con 0x7fb03015bd40 2024-04-25T10:10:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.833+0000 7fb0357f9640 1 -- 172.21.15.16:0/1878295110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0301e7670 con 0x7fb03015bd40 2024-04-25T10:10:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.834+0000 7fb0357f9640 1 -- 172.21.15.16:0/1878295110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0300749b0 con 0x7fb03015bd40 2024-04-25T10:10:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.834+0000 7fb0277fe640 1 -- 172.21.15.16:0/1878295110 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb02001c070 con 0x7fb03015bd40 2024-04-25T10:10:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.835+0000 7fb0277fe640 1 --2- 172.21.15.16:0/1878295110 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb01006b4d0 0x7fb01006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.835+0000 7fb0277fe640 1 -- 172.21.15.16:0/1878295110 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fb020089e10 con 0x7fb03015bd40 2024-04-25T10:10:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.835+0000 7fb02effd640 1 --2- 172.21.15.16:0/1878295110 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb01006b4d0 0x7fb01006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.836+0000 7fb02effd640 1 --2- 172.21.15.16:0/1878295110 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb01006b4d0 0x7fb01006d990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb028034000 tx=0x7fb02802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:41.839+0000 7fb0277fe640 1 -- 172.21.15.16:0/1878295110 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb020058590 con 0x7fb03015bd40 2024-04-25T10:10:42.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.063+0000 7fb0357f9640 1 -- 172.21.15.16:0/1878295110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0301e5650 con 0x7fb03015bd40 2024-04-25T10:10:42.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.063+0000 7fb0277fe640 1 -- 172.21.15.16:0/1878295110 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fb02005c630 con 0x7fb03015bd40 2024-04-25T10:10:42.069 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:42.069 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6070,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2805428272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2805428272},{"type":"v1","addr":"172.21.15.186:6835","nonce":2805428272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1052667029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1052667029},{"type":"v1","addr":"172.21.15.186:6837","nonce":1052667029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2897154125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2897154125},{"type":"v1","addr":"172.21.15.16:6833","nonce":2897154125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:22.077032+0000","modified":"2024-04-25T10:10:14.390498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6636":{"gid":6636,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":17,"addr":"172.21.15.16:6835/1735811292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1735811292},{"type":"v1","addr":"172.21.15.16:6835","nonce":1735811292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:10:42.070 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 44 2024-04-25T10:10:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.065+0000 7fb0357f9640 1 -- 172.21.15.16:0/1878295110 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb01006b4d0 msgr2=0x7fb01006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.065+0000 7fb0357f9640 1 --2- 172.21.15.16:0/1878295110 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb01006b4d0 0x7fb01006d990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb028034000 tx=0x7fb02802e000 comp rx=0 tx=0).stop 2024-04-25T10:10:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.065+0000 7fb0357f9640 1 -- 172.21.15.16:0/1878295110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03015bd40 msgr2=0x7fb0300c6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.065+0000 7fb0357f9640 1 --2- 172.21.15.16:0/1878295110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03015bd40 0x7fb0300c6050 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb02000b970 tx=0x7fb02000be30 comp rx=0 tx=0).stop 2024-04-25T10:10:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.066+0000 7fb0357f9640 1 -- 172.21.15.16:0/1878295110 shutdown_connections 2024-04-25T10:10:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.066+0000 7fb0357f9640 1 --2- 172.21.15.16:0/1878295110 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb01006b4d0 0x7fb01006d990 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.066+0000 7fb0357f9640 1 --2- 172.21.15.16:0/1878295110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03015bd40 0x7fb0300c6050 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.066+0000 7fb0357f9640 1 --2- 172.21.15.16:0/1878295110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb030099e20 0x7fb0300c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.066+0000 7fb0357f9640 1 --2- 172.21.15.16:0/1878295110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb030099a40 0x7fb0301e6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.066+0000 7fb0357f9640 1 -- 172.21.15.16:0/1878295110 >> 172.21.15.16:0/1878295110 conn(0x7fb0300b32f0 msgr2=0x7fb03006fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.066+0000 7fb0357f9640 1 -- 172.21.15.16:0/1878295110 shutdown_connections 2024-04-25T10:10:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.066+0000 7fb0357f9640 1 -- 172.21.15.16:0/1878295110 wait complete. 2024-04-25T10:10:42.082 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:10:42.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.179+0000 7fd9e857e640 1 Processor -- start 2024-04-25T10:10:42.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.179+0000 7fd9e857e640 1 -- start start 2024-04-25T10:10:42.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.180+0000 7fd9e857e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd9e00d3b90 0x7fd9e00d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.180+0000 7fd9e857e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e00d0f60 0x7fd9e00b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.180+0000 7fd9e857e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e00d4090 con 0x7fd9e015dca0 2024-04-25T10:10:42.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.180+0000 7fd9e857e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e00b6b20 con 0x7fd9e00d3b90 2024-04-25T10:10:42.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.180+0000 7fd9e857e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00d1390 con 0x7fd9e00d0f60 2024-04-25T10:10:42.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.180+0000 7fd9e5b19640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd9e00d3b90 0x7fd9e00d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.180+0000 7fd9e5b19640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd9e00d3b90 0x7fd9e00d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57244/0 (socket says 172.21.15.16:57244) 2024-04-25T10:10:42.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.180+0000 7fd9e5b19640 1 -- 172.21.15.16:0/3771137271 learned_addr learned my addr 172.21.15.16:0/3771137271 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:42.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.181+0000 7fd9e6b1b640 1 --2- 172.21.15.16:0/3771137271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e00d0f60 0x7fd9e00b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.181+0000 7fd9e5b19640 1 -- 172.21.15.16:0/3771137271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e00d0f60 msgr2=0x7fd9e00b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.181+0000 7fd9e5b19640 1 --2- 172.21.15.16:0/3771137271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e00d0f60 0x7fd9e00b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.181+0000 7fd9e5b19640 1 -- 172.21.15.16:0/3771137271 >> v1:172.21.15.186:6789/0 conn(0x7fd9e015dca0 legacy=0x7fd9e00ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.181+0000 7fd9e5b19640 1 -- 172.21.15.16:0/3771137271 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e00d4370 con 0x7fd9e00d3b90 2024-04-25T10:10:42.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.181+0000 7fd9e5b19640 1 --2- 172.21.15.16:0/3771137271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e00d3b90 0x7fd9e00d0a20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40098a0 tx=0x7fd9d402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1fa05c103577ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.181+0000 7fd9e5318640 1 -- 172.21.15.16:0/3771137271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4037070 con 0x7fd9e00d3b90 2024-04-25T10:10:42.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.182+0000 7fd9e5318640 1 -- 172.21.15.16:0/3771137271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d4032440 con 0x7fd9e00d3b90 2024-04-25T10:10:42.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.182+0000 7fd9e5318640 1 -- 172.21.15.16:0/3771137271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4032810 con 0x7fd9e00d3b90 2024-04-25T10:10:42.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.182+0000 7fd9e857e640 1 -- 172.21.15.16:0/3771137271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e00d3b90 msgr2=0x7fd9e00d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.182+0000 7fd9e857e640 1 --2- 172.21.15.16:0/3771137271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e00d3b90 0x7fd9e00d0a20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40098a0 tx=0x7fd9d402a290 comp rx=0 tx=0).stop 2024-04-25T10:10:42.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.182+0000 7fd9e857e640 1 -- 172.21.15.16:0/3771137271 shutdown_connections 2024-04-25T10:10:42.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.182+0000 7fd9e857e640 1 --2- 172.21.15.16:0/3771137271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e00d0f60 0x7fd9e00b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.182+0000 7fd9e857e640 1 --2- 172.21.15.16:0/3771137271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e00d3b90 0x7fd9e00d0a20 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.182+0000 7fd9e857e640 1 -- 172.21.15.16:0/3771137271 >> 172.21.15.16:0/3771137271 conn(0x7fd9e00726c0 msgr2=0x7fd9e0072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:42.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.182+0000 7fd9e857e640 1 -- 172.21.15.16:0/3771137271 shutdown_connections 2024-04-25T10:10:42.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.182+0000 7fd9e857e640 1 -- 172.21.15.16:0/3771137271 wait complete. 2024-04-25T10:10:42.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.183+0000 7fd9e857e640 1 Processor -- start 2024-04-25T10:10:42.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.183+0000 7fd9e857e640 1 -- start start 2024-04-25T10:10:42.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.184+0000 7fd9e857e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e00d0f60 0x7fd9e00b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.184+0000 7fd9e857e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e00d3b90 0x7fd9e01f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.184+0000 7fd9e857e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e015dca0 0x7fd9e01f4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.184+0000 7fd9e857e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00cf050 con 0x7fd9e00d3b90 2024-04-25T10:10:42.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.184+0000 7fd9e857e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00b6d50 con 0x7fd9e00d0f60 2024-04-25T10:10:42.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.184+0000 7fd9e857e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e015e080 con 0x7fd9e015dca0 2024-04-25T10:10:42.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.184+0000 7fd9e5b19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e00d3b90 0x7fd9e01f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.184+0000 7fd9e5b19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e00d3b90 0x7fd9e01f3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57256/0 (socket says 172.21.15.16:57256) 2024-04-25T10:10:42.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.184+0000 7fd9e5b19640 1 -- 172.21.15.16:0/776473191 learned_addr learned my addr 172.21.15.16:0/776473191 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.184+0000 7fd9e631a640 1 --2- 172.21.15.16:0/776473191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e00d0f60 0x7fd9e00b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.184+0000 7fd9e6b1b640 1 --2- 172.21.15.16:0/776473191 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e015dca0 0x7fd9e01f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.184+0000 7fd9e5b19640 1 -- 172.21.15.16:0/776473191 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e015dca0 msgr2=0x7fd9e01f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.184+0000 7fd9e5b19640 1 --2- 172.21.15.16:0/776473191 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e015dca0 0x7fd9e01f4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.184+0000 7fd9e5b19640 1 -- 172.21.15.16:0/776473191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e00d0f60 msgr2=0x7fd9e00b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.184+0000 7fd9e5b19640 1 --2- 172.21.15.16:0/776473191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e00d0f60 0x7fd9e00b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.184+0000 7fd9e5b19640 1 -- 172.21.15.16:0/776473191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e00c9670 con 0x7fd9e00d3b90 2024-04-25T10:10:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.184+0000 7fd9e631a640 1 --2- 172.21.15.16:0/776473191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e00d0f60 0x7fd9e00b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.185+0000 7fd9e5b19640 1 --2- 172.21.15.16:0/776473191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e00d3b90 0x7fd9e01f3ce0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40098a0 tx=0x7fd9d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.185+0000 7fd9deffd640 1 -- 172.21.15.16:0/776473191 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4041020 con 0x7fd9e00d3b90 2024-04-25T10:10:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.185+0000 7fd9deffd640 1 -- 172.21.15.16:0/776473191 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d40306c0 con 0x7fd9e00d3b90 2024-04-25T10:10:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.185+0000 7fd9deffd640 1 -- 172.21.15.16:0/776473191 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4037040 con 0x7fd9e00d3b90 2024-04-25T10:10:42.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.185+0000 7fd9e857e640 1 -- 172.21.15.16:0/776473191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e00c8610 con 0x7fd9e00d3b90 2024-04-25T10:10:42.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.185+0000 7fd9e857e640 1 -- 172.21.15.16:0/776473191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e00c8e70 con 0x7fd9e00d3b90 2024-04-25T10:10:42.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.186+0000 7fd9deffd640 1 -- 172.21.15.16:0/776473191 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9d4044020 con 0x7fd9e00d3b90 2024-04-25T10:10:42.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.187+0000 7fd9deffd640 1 --2- 172.21.15.16:0/776473191 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd9cc06b4d0 0x7fd9cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.187+0000 7fd9deffd640 1 -- 172.21.15.16:0/776473191 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fd9d40aaae0 con 0x7fd9e00d3b90 2024-04-25T10:10:42.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.187+0000 7fd9e631a640 1 --2- 172.21.15.16:0/776473191 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd9cc06b4d0 0x7fd9cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.187+0000 7fd9e631a640 1 --2- 172.21.15.16:0/776473191 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd9cc06b4d0 0x7fd9cc06d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00ca170 tx=0x7fd9d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.188+0000 7fd9eb280740 1 Processor -- start 2024-04-25T10:10:42.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.188+0000 7fd9eb280740 1 -- start start 2024-04-25T10:10:42.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.189+0000 7fd9eb280740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a543e590 0x55f4a54406f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.189+0000 7fd9eb280740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a547ac60 0x55f4a543e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.189+0000 7fd9eb280740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a5469050 0x55f4a54694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.189+0000 7fd9eb280740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4a53a1c60 con 0x55f4a5469050 2024-04-25T10:10:42.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.189+0000 7fd9eb280740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4a5340d80 con 0x55f4a543e590 2024-04-25T10:10:42.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.189+0000 7fd9eb280740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4a539dd00 con 0x55f4a547ac60 2024-04-25T10:10:42.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.189+0000 7fd9e6b1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a5469050 0x55f4a54694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.189+0000 7fd9e6b1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a5469050 0x55f4a54694d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57266/0 (socket says 172.21.15.16:57266) 2024-04-25T10:10:42.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.189+0000 7fd9e6b1b640 1 -- 172.21.15.16:0/3622210305 learned_addr learned my addr 172.21.15.16:0/3622210305 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:42.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.189+0000 7fd9e631a640 1 --2- 172.21.15.16:0/3622210305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a543e590 0x55f4a54406f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.189+0000 7fd9e5b19640 1 --2- 172.21.15.16:0/3622210305 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a547ac60 0x55f4a543e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.189+0000 7fd9e6b1b640 1 -- 172.21.15.16:0/3622210305 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a547ac60 msgr2=0x55f4a543e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.189+0000 7fd9e6b1b640 1 --2- 172.21.15.16:0/3622210305 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a547ac60 0x55f4a543e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.189+0000 7fd9e6b1b640 1 -- 172.21.15.16:0/3622210305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a543e590 msgr2=0x55f4a54406f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.189+0000 7fd9e6b1b640 1 --2- 172.21.15.16:0/3622210305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a543e590 0x55f4a54406f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.189+0000 7fd9e6b1b640 1 -- 172.21.15.16:0/3622210305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4a51827c0 con 0x55f4a5469050 2024-04-25T10:10:42.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.190+0000 7fd9e6b1b640 1 --2- 172.21.15.16:0/3622210305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a5469050 0x55f4a54694d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00073e0 tx=0x7fd9d00078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d81d048751bae60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.190+0000 7fd9c7fff640 1 -- 172.21.15.16:0/3622210305 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d00147e0 con 0x55f4a5469050 2024-04-25T10:10:42.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.190+0000 7fd9c7fff640 1 -- 172.21.15.16:0/3622210305 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d001a070 con 0x55f4a5469050 2024-04-25T10:10:42.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.190+0000 7fd9c7fff640 1 -- 172.21.15.16:0/3622210305 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0014b80 con 0x55f4a5469050 2024-04-25T10:10:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.190+0000 7fd9eb280740 1 -- 172.21.15.16:0/3622210305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a5469050 msgr2=0x55f4a54694d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.190+0000 7fd9eb280740 1 --2- 172.21.15.16:0/3622210305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a5469050 0x55f4a54694d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00073e0 tx=0x7fd9d00078a0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.190+0000 7fd9eb280740 1 -- 172.21.15.16:0/3622210305 shutdown_connections 2024-04-25T10:10:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.190+0000 7fd9eb280740 1 --2- 172.21.15.16:0/3622210305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a5469050 0x55f4a54694d0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.190+0000 7fd9eb280740 1 --2- 172.21.15.16:0/3622210305 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a547ac60 0x55f4a543e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.190+0000 7fd9eb280740 1 --2- 172.21.15.16:0/3622210305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a543e590 0x55f4a54406f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.190+0000 7fd9eb280740 1 -- 172.21.15.16:0/3622210305 >> 172.21.15.16:0/3622210305 conn(0x55f4a54066e0 msgr2=0x55f4a5484050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.191+0000 7fd9eb280740 1 -- 172.21.15.16:0/3622210305 shutdown_connections 2024-04-25T10:10:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.191+0000 7fd9eb280740 1 -- 172.21.15.16:0/3622210305 wait complete. 2024-04-25T10:10:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.191+0000 7fd9eb280740 1 -- start start 2024-04-25T10:10:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.191+0000 7fd9eb280740 1 Processor -- start 2024-04-25T10:10:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.192+0000 7fd9eb280740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a543e590 0x55f4a5469430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.192+0000 7fd9eb280740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a5469050 0x55f4a546ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.192+0000 7fd9eb280740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a547ac60 0x55f4a547d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.192+0000 7fd9eb280740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4a533e840 con 0x55f4a5469050 2024-04-25T10:10:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.192+0000 7fd9eb280740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4a5466ba0 con 0x55f4a547ac60 2024-04-25T10:10:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.192+0000 7fd9eb280740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4a547c400 con 0x55f4a543e590 2024-04-25T10:10:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.192+0000 7fd9eb280740 10 client.0 _ll_register_callbacks cb 0x55f4a5358980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.192+0000 7fd9e5b19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a5469050 0x55f4a546ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.192+0000 7fd9e5b19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a5469050 0x55f4a546ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57282/0 (socket says 172.21.15.16:57282) 2024-04-25T10:10:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.192+0000 7fd9e5b19640 1 -- 172.21.15.16:0/646131899 learned_addr learned my addr 172.21.15.16:0/646131899 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.192+0000 7fd9e6b1b640 1 --2- 172.21.15.16:0/646131899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a547ac60 0x55f4a547d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.192+0000 7fd9e631a640 1 --2- 172.21.15.16:0/646131899 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a543e590 0x55f4a5469430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9e5b19640 1 -- 172.21.15.16:0/646131899 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a543e590 msgr2=0x55f4a5469430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9e5b19640 1 --2- 172.21.15.16:0/646131899 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a543e590 0x55f4a5469430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9e5b19640 1 -- 172.21.15.16:0/646131899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a547ac60 msgr2=0x55f4a547d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9e5b19640 1 --2- 172.21.15.16:0/646131899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a547ac60 0x55f4a547d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9e5b19640 1 -- 172.21.15.16:0/646131899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4a51e2f50 con 0x55f4a5469050 2024-04-25T10:10:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9e5b19640 1 --2- 172.21.15.16:0/646131899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a5469050 0x55f4a546ba50 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4075570 tx=0x7fd9d4079110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7b65c52cc19389 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:10:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9c57fa640 1 -- 172.21.15.16:0/646131899 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4081820 con 0x55f4a5469050 2024-04-25T10:10:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9c57fa640 1 -- 172.21.15.16:0/646131899 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fd9d4077410 con 0x55f4a5469050 2024-04-25T10:10:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9eb280740 1 -- 172.21.15.16:0/646131899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9d001a070 con 0x55f4a5469050 2024-04-25T10:10:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9c57fa640 1 -- 172.21.15.16:0/646131899 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9d4081390 con 0x55f4a5469050 2024-04-25T10:10:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9eb280740 10 client.7116 fetch_fsmap learned FSMap version 44 2024-04-25T10:10:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9eb280740 1 -- 172.21.15.16:0/646131899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55f4a50760b0 con 0x55f4a5469050 2024-04-25T10:10:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9c57fa640 1 -- 172.21.15.16:0/646131899 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d001a070 con 0x55f4a5469050 2024-04-25T10:10:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9c57fa640 1 -- 172.21.15.16:0/646131899 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd9d403a030 con 0x55f4a5469050 2024-04-25T10:10:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9eb280740 10 client.7116 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T10:10:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9eb280740 10 client.7116 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:10:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9eb280740 20 client.7116 populate_metadata read hostname 'smithi016' 2024-04-25T10:10:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9eb280740 1 --2- 172.21.15.16:0/646131899 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55f4a547b0c0 0x55f4a547b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9eb280740 4 client.7116 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.193+0000 7fd9eb280740 1 -- 172.21.15.16:0/646131899 --> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4a5361e60 con 0x55f4a547b0c0 2024-04-25T10:10:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.194+0000 7fd9e631a640 1 --2- 172.21.15.16:0/646131899 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55f4a547b0c0 0x55f4a547b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.194+0000 7fd9e631a640 1 --2- 172.21.15.16:0/646131899 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55f4a547b0c0 0x55f4a547b4a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b23d74f0f8afd82 server_cookie=dd508b8e30b5b632 in_seq=0 out_seq=0 2024-04-25T10:10:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.194+0000 7fd9c57fa640 10 client.7116 ms_handle_connect on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.194+0000 7fd9c57fa640 1 -- 172.21.15.16:0/646131899 <== mds.0 v2:172.21.15.16:6834/1735811292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f4a5361e60 con 0x55f4a547b0c0 2024-04-25T10:10:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.194+0000 7fd9c57fa640 10 client.7116 handle_command_reply: tid=0 2024-04-25T10:10:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.194+0000 7fd9eb280740 1 client.7116 shutdown 2024-04-25T10:10:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.195+0000 7fd9eb280740 1 -- 172.21.15.16:0/646131899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a5469050 msgr2=0x55f4a546ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.195+0000 7fd9eb280740 1 --2- 172.21.15.16:0/646131899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a5469050 0x55f4a546ba50 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4075570 tx=0x7fd9d4079110 comp rx=0 tx=0).stop 2024-04-25T10:10:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.195+0000 7fd9eb280740 1 -- 172.21.15.16:0/646131899 shutdown_connections 2024-04-25T10:10:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.195+0000 7fd9eb280740 1 --2- 172.21.15.16:0/646131899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a547ac60 0x55f4a547d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.195+0000 7fd9eb280740 1 --2- 172.21.15.16:0/646131899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a5469050 0x55f4a546ba50 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.195+0000 7fd9eb280740 1 --2- 172.21.15.16:0/646131899 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55f4a547b0c0 0x55f4a547b4a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.195+0000 7fd9eb280740 1 --2- 172.21.15.16:0/646131899 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a543e590 0x55f4a5469430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.195+0000 7fd9c57fa640 0 client.7116 ms_handle_reset on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:42.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.195+0000 7fd9eb280740 1 -- 172.21.15.16:0/646131899 >> 172.21.15.16:0/646131899 conn(0x55f4a54066e0 msgr2=0x55f4a5466fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:42.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.195+0000 7fd9eb280740 1 -- 172.21.15.16:0/646131899 shutdown_connections 2024-04-25T10:10:42.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.195+0000 7fd9eb280740 1 -- 172.21.15.16:0/646131899 wait complete. 2024-04-25T10:10:42.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.195+0000 7fd9eb280740 20 client.7116 trim_cache size 0 max 16384 2024-04-25T10:10:42.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.207+0000 7fd9eb280740 1 Processor -- start 2024-04-25T10:10:42.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.207+0000 7fd9eb280740 1 -- start start 2024-04-25T10:10:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.207+0000 7fd9eb280740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a543e590 0x55f4a543e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.207+0000 7fd9eb280740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a5469050 0x55f4a5487190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.207+0000 7fd9eb280740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a547ac60 0x55f4a5484a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.207+0000 7fd9eb280740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e01ff410 con 0x55f4a543e590 2024-04-25T10:10:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.207+0000 7fd9eb280740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00cb450 con 0x55f4a5469050 2024-04-25T10:10:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.207+0000 7fd9eb280740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00cba50 con 0x55f4a547ac60 2024-04-25T10:10:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.207+0000 7fd9e631a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a543e590 0x55f4a543e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.207+0000 7fd9e631a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a543e590 0x55f4a543e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57296/0 (socket says 172.21.15.16:57296) 2024-04-25T10:10:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.207+0000 7fd9e631a640 1 -- 172.21.15.16:0/3081953453 learned_addr learned my addr 172.21.15.16:0/3081953453 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.208+0000 7fd9e631a640 1 -- 172.21.15.16:0/3081953453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a547ac60 msgr2=0x55f4a5484a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.208+0000 7fd9e631a640 1 --2- 172.21.15.16:0/3081953453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a547ac60 0x55f4a5484a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.208+0000 7fd9e631a640 1 -- 172.21.15.16:0/3081953453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a5469050 msgr2=0x55f4a5487190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.208+0000 7fd9e631a640 1 --2- 172.21.15.16:0/3081953453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a5469050 0x55f4a5487190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.208+0000 7fd9e631a640 1 -- 172.21.15.16:0/3081953453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4a54341c0 con 0x55f4a543e590 2024-04-25T10:10:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.208+0000 7fd9e631a640 1 --2- 172.21.15.16:0/3081953453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a543e590 0x55f4a543e9d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800e160 tx=0x7fd9d8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b50e7704c0dd77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.208+0000 7fd9c77fe640 1 -- 172.21.15.16:0/3081953453 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d802c070 con 0x55f4a543e590 2024-04-25T10:10:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.208+0000 7fd9c77fe640 1 -- 172.21.15.16:0/3081953453 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d8017030 con 0x55f4a543e590 2024-04-25T10:10:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.208+0000 7fd9c77fe640 1 -- 172.21.15.16:0/3081953453 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8027e60 con 0x55f4a543e590 2024-04-25T10:10:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.209+0000 7fd9eb280740 1 -- 172.21.15.16:0/3081953453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a543e590 msgr2=0x55f4a543e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.209+0000 7fd9eb280740 1 --2- 172.21.15.16:0/3081953453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a543e590 0x55f4a543e9d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800e160 tx=0x7fd9d8011000 comp rx=0 tx=0).stop 2024-04-25T10:10:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.209+0000 7fd9eb280740 1 -- 172.21.15.16:0/3081953453 shutdown_connections 2024-04-25T10:10:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.209+0000 7fd9eb280740 1 --2- 172.21.15.16:0/3081953453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a547ac60 0x55f4a5484a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.209+0000 7fd9eb280740 1 --2- 172.21.15.16:0/3081953453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a5469050 0x55f4a5487190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.209+0000 7fd9eb280740 1 --2- 172.21.15.16:0/3081953453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a543e590 0x55f4a543e9d0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.209+0000 7fd9eb280740 1 -- 172.21.15.16:0/3081953453 >> 172.21.15.16:0/3081953453 conn(0x55f4a54066e0 msgr2=0x55f4a5469430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:42.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.209+0000 7fd9eb280740 1 -- 172.21.15.16:0/3081953453 shutdown_connections 2024-04-25T10:10:42.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.209+0000 7fd9eb280740 1 -- 172.21.15.16:0/3081953453 wait complete. 2024-04-25T10:10:42.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.209+0000 7fd9eb280740 1 -- start start 2024-04-25T10:10:42.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.209+0000 7fd9eb280740 1 Processor -- start 2024-04-25T10:10:42.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.210+0000 7fd9eb280740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a543e590 0x55f4a54799e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.210+0000 7fd9eb280740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a5469050 0x55f4a549e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.210+0000 7fd9eb280740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a547ac60 0x55f4a5467db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.210+0000 7fd9eb280740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4a5458f10 con 0x55f4a547ac60 2024-04-25T10:10:42.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.210+0000 7fd9eb280740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4a547c140 con 0x55f4a5469050 2024-04-25T10:10:42.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.210+0000 7fd9eb280740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4a5497910 con 0x55f4a543e590 2024-04-25T10:10:42.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.210+0000 7fd9eb280740 10 client.0 _ll_register_callbacks cb 0x7fd9d00098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:42.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.210+0000 7fd9e6b1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a547ac60 0x55f4a5467db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.210+0000 7fd9e6b1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a547ac60 0x55f4a5467db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57306/0 (socket says 172.21.15.16:57306) 2024-04-25T10:10:42.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.210+0000 7fd9e6b1b640 1 -- 172.21.15.16:0/2862487370 learned_addr learned my addr 172.21.15.16:0/2862487370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:42.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.210+0000 7fd9e5b19640 1 --2- 172.21.15.16:0/2862487370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a5469050 0x55f4a549e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.210+0000 7fd9e631a640 1 --2- 172.21.15.16:0/2862487370 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a543e590 0x55f4a54799e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.210+0000 7fd9e6b1b640 1 -- 172.21.15.16:0/2862487370 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a543e590 msgr2=0x55f4a54799e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.210+0000 7fd9e6b1b640 1 --2- 172.21.15.16:0/2862487370 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a543e590 0x55f4a54799e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9e6b1b640 1 -- 172.21.15.16:0/2862487370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a5469050 msgr2=0x55f4a549e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9e6b1b640 1 --2- 172.21.15.16:0/2862487370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a5469050 0x55f4a549e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9e6b1b640 1 -- 172.21.15.16:0/2862487370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4a54682f0 con 0x55f4a547ac60 2024-04-25T10:10:42.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9e6b1b640 1 --2- 172.21.15.16:0/2862487370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a547ac60 0x55f4a5467db0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00078e0 tx=0x7fd9d0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8d3161cb15cc4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:10:42.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9c57fa640 1 -- 172.21.15.16:0/2862487370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0014ca0 con 0x55f4a547ac60 2024-04-25T10:10:42.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9c57fa640 1 -- 172.21.15.16:0/2862487370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fd9d0014e40 con 0x55f4a547ac60 2024-04-25T10:10:42.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9eb280740 1 -- 172.21.15.16:0/2862487370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9d8017030 con 0x55f4a547ac60 2024-04-25T10:10:42.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9c57fa640 1 -- 172.21.15.16:0/2862487370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9d0015470 con 0x55f4a547ac60 2024-04-25T10:10:42.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9eb280740 10 client.7122 fetch_fsmap learned FSMap version 44 2024-04-25T10:10:42.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9eb280740 1 -- 172.21.15.16:0/2862487370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55f4a54a6d00 con 0x55f4a547ac60 2024-04-25T10:10:42.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9c57fa640 1 -- 172.21.15.16:0/2862487370 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d8017030 con 0x55f4a547ac60 2024-04-25T10:10:42.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9c57fa640 1 -- 172.21.15.16:0/2862487370 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd9d0015610 con 0x55f4a547ac60 2024-04-25T10:10:42.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9eb280740 10 client.7122 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T10:10:42.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9eb280740 10 client.7122 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:10:42.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9eb280740 20 client.7122 populate_metadata read hostname 'smithi016' 2024-04-25T10:10:42.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9eb280740 1 --2- 172.21.15.16:0/2862487370 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55f4a546bef0 0x55f4a546c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9eb280740 4 client.7122 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:10:42.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.211+0000 7fd9eb280740 1 -- 172.21.15.16:0/2862487370 --> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55f4a53af1a0 con 0x55f4a546bef0 2024-04-25T10:10:42.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.212+0000 7fd9e631a640 1 --2- 172.21.15.16:0/2862487370 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55f4a546bef0 0x55f4a546c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.212+0000 7fd9e631a640 1 --2- 172.21.15.16:0/2862487370 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55f4a546bef0 0x55f4a546c2d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49d70fa4652d0564 server_cookie=b4b7e6b4d83a15e2 in_seq=0 out_seq=0 2024-04-25T10:10:42.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.212+0000 7fd9c57fa640 10 client.7122 ms_handle_connect on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:42.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.212+0000 7fd9c57fa640 1 -- 172.21.15.16:0/2862487370 <== mds.0 v2:172.21.15.16:6834/1735811292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f4a53af1a0 con 0x55f4a546bef0 2024-04-25T10:10:42.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.212+0000 7fd9c57fa640 10 client.7122 handle_command_reply: tid=0 2024-04-25T10:10:42.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.212+0000 7fd9eb280740 1 client.7122 shutdown 2024-04-25T10:10:42.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.212+0000 7fd9eb280740 1 -- 172.21.15.16:0/2862487370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a547ac60 msgr2=0x55f4a5467db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.212+0000 7fd9eb280740 1 --2- 172.21.15.16:0/2862487370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a547ac60 0x55f4a5467db0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00078e0 tx=0x7fd9d0004ab0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.212+0000 7fd9eb280740 1 -- 172.21.15.16:0/2862487370 shutdown_connections 2024-04-25T10:10:42.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.212+0000 7fd9eb280740 1 --2- 172.21.15.16:0/2862487370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4a547ac60 0x55f4a5467db0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.212+0000 7fd9eb280740 1 --2- 172.21.15.16:0/2862487370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4a5469050 0x55f4a549e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.212+0000 7fd9eb280740 1 --2- 172.21.15.16:0/2862487370 >> [v2:172.21.15.16:6834/1735811292,v1:172.21.15.16:6835/1735811292] conn(0x55f4a546bef0 0x55f4a546c2d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.212+0000 7fd9eb280740 1 --2- 172.21.15.16:0/2862487370 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f4a543e590 0x55f4a54799e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.212+0000 7fd9c57fa640 0 client.7122 ms_handle_reset on v2:172.21.15.16:6834/1735811292 2024-04-25T10:10:42.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.212+0000 7fd9eb280740 1 -- 172.21.15.16:0/2862487370 >> 172.21.15.16:0/2862487370 conn(0x55f4a54066e0 msgr2=0x55f4a5469430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:42.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.212+0000 7fd9eb280740 1 -- 172.21.15.16:0/2862487370 shutdown_connections 2024-04-25T10:10:42.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.212+0000 7fd9eb280740 1 -- 172.21.15.16:0/2862487370 wait complete. 2024-04-25T10:10:42.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.212+0000 7fd9eb280740 20 client.7122 trim_cache size 0 max 16384 2024-04-25T10:10:42.224 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:42.224 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-04-25T10:10:42.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.213+0000 7fd9e857e640 1 -- 172.21.15.16:0/776473191 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd9cc06b4d0 msgr2=0x7fd9cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.213+0000 7fd9e857e640 1 --2- 172.21.15.16:0/776473191 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd9cc06b4d0 0x7fd9cc06d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00ca170 tx=0x7fd9d8009040 comp rx=0 tx=0).stop 2024-04-25T10:10:42.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.213+0000 7fd9e857e640 1 -- 172.21.15.16:0/776473191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e00d3b90 msgr2=0x7fd9e01f3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.213+0000 7fd9e857e640 1 --2- 172.21.15.16:0/776473191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e00d3b90 0x7fd9e01f3ce0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40098a0 tx=0x7fd9d40046c0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.213+0000 7fd9e857e640 1 -- 172.21.15.16:0/776473191 shutdown_connections 2024-04-25T10:10:42.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.213+0000 7fd9e857e640 1 --2- 172.21.15.16:0/776473191 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd9cc06b4d0 0x7fd9cc06d990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.213+0000 7fd9e857e640 1 --2- 172.21.15.16:0/776473191 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e015dca0 0x7fd9e01f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.213+0000 7fd9e857e640 1 --2- 172.21.15.16:0/776473191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e00d3b90 0x7fd9e01f3ce0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.213+0000 7fd9e857e640 1 --2- 172.21.15.16:0/776473191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e00d0f60 0x7fd9e00b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.213+0000 7fd9e857e640 1 -- 172.21.15.16:0/776473191 >> 172.21.15.16:0/776473191 conn(0x7fd9e00726c0 msgr2=0x7fd9e015e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:42.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.213+0000 7fd9e857e640 1 -- 172.21.15.16:0/776473191 shutdown_connections 2024-04-25T10:10:42.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.213+0000 7fd9e857e640 1 -- 172.21.15.16:0/776473191 wait complete. 2024-04-25T10:10:42.229 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:10:42.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.323+0000 7fca830db640 1 Processor -- start 2024-04-25T10:10:42.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.323+0000 7fca830db640 1 -- start start 2024-04-25T10:10:42.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.324+0000 7fca830db640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fca7c0b5b60 0x7fca7c1687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.324+0000 7fca830db640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca7c15e0c0 con 0x7fca7c15dce0 2024-04-25T10:10:42.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.324+0000 7fca830db640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca7c0d2510 con 0x7fca7c0d35b0 2024-04-25T10:10:42.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.324+0000 7fca830db640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca7c0b7b10 con 0x7fca7c0b5b60 2024-04-25T10:10:42.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.324+0000 7fca80e77640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fca7c15dce0 0x7fca7c0d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52388/0 (socket says 172.21.15.16:52388) 2024-04-25T10:10:42.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.324+0000 7fca80e77640 1 -- 172.21.15.16:0/2709522455 learned_addr learned my addr 172.21.15.16:0/2709522455 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:42.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.324+0000 7fca7b7fe640 1 -- 172.21.15.16:0/2709522455 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285344889 0 0) 0x7fca7c15e0c0 con 0x7fca7c15dce0 2024-04-25T10:10:42.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.324+0000 7fca81678640 1 --2- 172.21.15.16:0/2709522455 >> v2:172.21.15.186:3300/0 conn(0x7fca7c0b5b60 0x7fca7c1687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.324+0000 7fca7b7fe640 1 -- 172.21.15.16:0/2709522455 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca6c003520 con 0x7fca7c15dce0 2024-04-25T10:10:42.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.324+0000 7fca7b7fe640 1 -- 172.21.15.16:0/2709522455 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 619724646 0 0) 0x7fca6c003520 con 0x7fca7c15dce0 2024-04-25T10:10:42.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.324+0000 7fca7b7fe640 1 -- 172.21.15.16:0/2709522455 >> v2:172.21.15.186:3300/0 conn(0x7fca7c0b5b60 msgr2=0x7fca7c1687f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.324+0000 7fca7b7fe640 1 --2- 172.21.15.16:0/2709522455 >> v2:172.21.15.186:3300/0 conn(0x7fca7c0b5b60 0x7fca7c1687f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.324+0000 7fca7b7fe640 1 -- 172.21.15.16:0/2709522455 >> v1:172.21.15.186:6789/0 conn(0x7fca7c0d35b0 legacy=0x7fca7c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.324+0000 7fca7b7fe640 1 -- 172.21.15.16:0/2709522455 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7c0d27f0 con 0x7fca7c15dce0 2024-04-25T10:10:42.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.324+0000 7fca7b7fe640 1 -- 172.21.15.16:0/2709522455 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fca70001670 con 0x7fca7c15dce0 2024-04-25T10:10:42.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.324+0000 7fca7b7fe640 1 -- 172.21.15.16:0/2709522455 >> v1:172.21.15.16:6789/0 conn(0x7fca7c15dce0 legacy=0x7fca7c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.325+0000 7fca7b7fe640 1 --2- 172.21.15.16:0/2709522455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c0039b0 0x7fca6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.325+0000 7fca7b7fe640 1 --2- 172.21.15.16:0/2709522455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca6c004800 0x7fca6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.325+0000 7fca7b7fe640 1 --2- 172.21.15.16:0/2709522455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca6c0051c0 0x7fca6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.325+0000 7fca7b7fe640 1 -- 172.21.15.16:0/2709522455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c0b7b10 con 0x7fca6c0051c0 2024-04-25T10:10:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.325+0000 7fca7b7fe640 1 -- 172.21.15.16:0/2709522455 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c00c060 con 0x7fca6c004800 2024-04-25T10:10:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.325+0000 7fca7b7fe640 1 -- 172.21.15.16:0/2709522455 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c00c260 con 0x7fca6c0039b0 2024-04-25T10:10:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.325+0000 7fca81678640 1 --2- 172.21.15.16:0/2709522455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca6c0051c0 0x7fca6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.325+0000 7fca80e77640 1 --2- 172.21.15.16:0/2709522455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c0039b0 0x7fca6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.325+0000 7fca7bfff640 1 --2- 172.21.15.16:0/2709522455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca6c004800 0x7fca6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.325+0000 7fca81678640 1 -- 172.21.15.16:0/2709522455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c0039b0 msgr2=0x7fca6c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.325+0000 7fca81678640 1 --2- 172.21.15.16:0/2709522455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c0039b0 0x7fca6c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.325+0000 7fca81678640 1 -- 172.21.15.16:0/2709522455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca6c004800 msgr2=0x7fca6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.325+0000 7fca81678640 1 --2- 172.21.15.16:0/2709522455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca6c004800 0x7fca6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.325+0000 7fca81678640 1 -- 172.21.15.16:0/2709522455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6c003520 con 0x7fca6c0051c0 2024-04-25T10:10:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.326+0000 7fca81678640 1 -- 172.21.15.16:0/2709522455 reap_dead start 2024-04-25T10:10:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.326+0000 7fca81678640 1 --2- 172.21.15.16:0/2709522455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca6c0051c0 0x7fca6c00b9c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fca74009fc0 tx=0x7fca6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d12ec6d76807c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.326+0000 7fca7b7fe640 1 -- 172.21.15.16:0/2709522455 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca74035020 con 0x7fca6c0051c0 2024-04-25T10:10:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.326+0000 7fca7b7fe640 1 -- 172.21.15.16:0/2709522455 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca74003bd0 con 0x7fca6c0051c0 2024-04-25T10:10:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.326+0000 7fca7b7fe640 1 -- 172.21.15.16:0/2709522455 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca74009c10 con 0x7fca6c0051c0 2024-04-25T10:10:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.326+0000 7fca830db640 1 -- 172.21.15.16:0/2709522455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca6c0051c0 msgr2=0x7fca6c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.326+0000 7fca830db640 1 --2- 172.21.15.16:0/2709522455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca6c0051c0 0x7fca6c00b9c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fca74009fc0 tx=0x7fca6c005830 comp rx=0 tx=0).stop 2024-04-25T10:10:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.326+0000 7fca830db640 1 -- 172.21.15.16:0/2709522455 shutdown_connections 2024-04-25T10:10:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.326+0000 7fca830db640 1 --2- 172.21.15.16:0/2709522455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca6c0051c0 0x7fca6c00b9c0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.326+0000 7fca830db640 1 -- 172.21.15.16:0/2709522455 >> 172.21.15.16:0/2709522455 conn(0x7fca7c072900 msgr2=0x7fca7c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.326+0000 7fca830db640 1 -- 172.21.15.16:0/2709522455 shutdown_connections 2024-04-25T10:10:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.327+0000 7fca830db640 1 -- 172.21.15.16:0/2709522455 wait complete. 2024-04-25T10:10:42.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.328+0000 7fca830db640 1 Processor -- start 2024-04-25T10:10:42.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.328+0000 7fca830db640 1 -- start start 2024-04-25T10:10:42.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca830db640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c0051c0 0x7fca7c153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca830db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7c0c9570 0x7fca7c153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca830db640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca7c154510 0x7fca7c0cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca830db640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c0d3090 con 0x7fca7c0c9570 2024-04-25T10:10:42.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca830db640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c0d2db0 con 0x7fca7c154510 2024-04-25T10:10:42.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca830db640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c0b9820 con 0x7fca6c0051c0 2024-04-25T10:10:42.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7c0c9570 0x7fca7c153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7c0c9570 0x7fca7c153fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57326/0 (socket says 172.21.15.16:57326) 2024-04-25T10:10:42.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca7bfff640 1 -- 172.21.15.16:0/201308941 learned_addr learned my addr 172.21.15.16:0/201308941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:42.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca80e77640 1 --2- 172.21.15.16:0/201308941 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c0051c0 0x7fca7c153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca81678640 1 --2- 172.21.15.16:0/201308941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca7c154510 0x7fca7c0cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca7bfff640 1 -- 172.21.15.16:0/201308941 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c0051c0 msgr2=0x7fca7c153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca7bfff640 1 --2- 172.21.15.16:0/201308941 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c0051c0 0x7fca7c153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca7bfff640 1 -- 172.21.15.16:0/201308941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca7c154510 msgr2=0x7fca7c0cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca7bfff640 1 --2- 172.21.15.16:0/201308941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca7c154510 0x7fca7c0cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca7bfff640 1 -- 172.21.15.16:0/201308941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7c15e350 con 0x7fca7c0c9570 2024-04-25T10:10:42.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca7bfff640 1 --2- 172.21.15.16:0/201308941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7c0c9570 0x7fca7c153fd0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fca6800f5c0 tx=0x7fca6800fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca797fa640 1 -- 172.21.15.16:0/201308941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca680052e0 con 0x7fca7c0c9570 2024-04-25T10:10:42.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca797fa640 1 -- 172.21.15.16:0/201308941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca68018070 con 0x7fca7c0c9570 2024-04-25T10:10:42.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca830db640 1 -- 172.21.15.16:0/201308941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7c15e630 con 0x7fca7c0c9570 2024-04-25T10:10:42.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca797fa640 1 -- 172.21.15.16:0/201308941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68003830 con 0x7fca7c0c9570 2024-04-25T10:10:42.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.329+0000 7fca830db640 1 -- 172.21.15.16:0/201308941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7c0cba00 con 0x7fca7c0c9570 2024-04-25T10:10:42.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.330+0000 7fca830db640 1 -- 172.21.15.16:0/201308941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7c0c9a90 con 0x7fca7c0c9570 2024-04-25T10:10:42.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.331+0000 7fca797fa640 1 -- 172.21.15.16:0/201308941 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca680039d0 con 0x7fca7c0c9570 2024-04-25T10:10:42.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.331+0000 7fca797fa640 1 --2- 172.21.15.16:0/201308941 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fca6006b4d0 0x7fca6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.332+0000 7fca797fa640 1 -- 172.21.15.16:0/201308941 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fca6808a150 con 0x7fca7c0c9570 2024-04-25T10:10:42.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.332+0000 7fca80e77640 1 --2- 172.21.15.16:0/201308941 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fca6006b4d0 0x7fca6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.332+0000 7fca80e77640 1 --2- 172.21.15.16:0/201308941 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fca6006b4d0 0x7fca6006d990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fca70001050 tx=0x7fca70009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.336+0000 7fca797fa640 1 -- 172.21.15.16:0/201308941 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca68058e80 con 0x7fca7c0c9570 2024-04-25T10:10:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.553+0000 7fca830db640 1 -- 172.21.15.16:0/201308941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fca7c0749b0 con 0x7fca7c0c9570 2024-04-25T10:10:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.553+0000 7fca797fa640 1 -- 172.21.15.16:0/201308941 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fca68058780 con 0x7fca7c0c9570 2024-04-25T10:10:42.559 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:42.559 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:10:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.555+0000 7fca830db640 1 -- 172.21.15.16:0/201308941 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fca6006b4d0 msgr2=0x7fca6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.555+0000 7fca830db640 1 --2- 172.21.15.16:0/201308941 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fca6006b4d0 0x7fca6006d990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fca70001050 tx=0x7fca70009040 comp rx=0 tx=0).stop 2024-04-25T10:10:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.555+0000 7fca830db640 1 -- 172.21.15.16:0/201308941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7c0c9570 msgr2=0x7fca7c153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.555+0000 7fca830db640 1 --2- 172.21.15.16:0/201308941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7c0c9570 0x7fca7c153fd0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fca6800f5c0 tx=0x7fca6800fa80 comp rx=0 tx=0).stop 2024-04-25T10:10:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.555+0000 7fca830db640 1 -- 172.21.15.16:0/201308941 shutdown_connections 2024-04-25T10:10:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.555+0000 7fca830db640 1 --2- 172.21.15.16:0/201308941 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fca6006b4d0 0x7fca6006d990 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.555+0000 7fca830db640 1 --2- 172.21.15.16:0/201308941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca7c154510 0x7fca7c0cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.555+0000 7fca830db640 1 --2- 172.21.15.16:0/201308941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7c0c9570 0x7fca7c153fd0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.555+0000 7fca830db640 1 --2- 172.21.15.16:0/201308941 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c0051c0 0x7fca7c153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.555+0000 7fca830db640 1 -- 172.21.15.16:0/201308941 >> 172.21.15.16:0/201308941 conn(0x7fca7c072900 msgr2=0x7fca7c0b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.555+0000 7fca830db640 1 -- 172.21.15.16:0/201308941 shutdown_connections 2024-04-25T10:10:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.555+0000 7fca830db640 1 -- 172.21.15.16:0/201308941 wait complete. 2024-04-25T10:10:42.571 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:10:42.571 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:10:42.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.666+0000 7f29c9407640 1 Processor -- start 2024-04-25T10:10:42.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.666+0000 7f29c9407640 1 -- start start 2024-04-25T10:10:42.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.667+0000 7f29c9407640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29c415dce0 0x7f29c40d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.667+0000 7f29c9407640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f29c40d35b0 0x7f29c40b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.667+0000 7f29c9407640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c415e0c0 con 0x7f29c40d1f90 2024-04-25T10:10:42.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.667+0000 7f29c9407640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c40b7b10 con 0x7f29c40d35b0 2024-04-25T10:10:42.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.667+0000 7f29c9407640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40d2510 con 0x7f29c415dce0 2024-04-25T10:10:42.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.667+0000 7f29c27fc640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f29c40d35b0 0x7f29c40b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.667+0000 7f29c2ffd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29c415dce0 0x7f29c40d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.667+0000 7f29c37fe640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f29c40d1f90 0x7f29c406bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:59270/0 (socket says 172.21.15.16:59270) 2024-04-25T10:10:42.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.667+0000 7f29c37fe640 1 -- 172.21.15.16:0/3749135339 learned_addr learned my addr 172.21.15.16:0/3749135339 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:42.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.668+0000 7f29c1ffb640 1 -- 172.21.15.16:0/3749135339 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3979237686 0 0) 0x7f29c415e0c0 con 0x7f29c40d1f90 2024-04-25T10:10:42.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.668+0000 7f29c2ffd640 1 -- 172.21.15.16:0/3749135339 >> v2:172.21.15.186:3300/0 conn(0x7f29c40d35b0 msgr2=0x7f29c40b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.668+0000 7f29c2ffd640 1 --2- 172.21.15.16:0/3749135339 >> v2:172.21.15.186:3300/0 conn(0x7f29c40d35b0 0x7f29c40b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.668+0000 7f29c2ffd640 1 -- 172.21.15.16:0/3749135339 >> v1:172.21.15.186:6789/0 conn(0x7f29c40d1f90 legacy=0x7f29c406bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.668+0000 7f29c2ffd640 1 -- 172.21.15.16:0/3749135339 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c40d27f0 con 0x7f29c415dce0 2024-04-25T10:10:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.668+0000 7f29c2ffd640 1 --2- 172.21.15.16:0/3749135339 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29c415dce0 0x7f29c40d1a50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009d90 tx=0x7f29b402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bbddbc62c5878fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.669+0000 7f29c1ffb640 1 -- 172.21.15.16:0/3749135339 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4031600 con 0x7f29c415dce0 2024-04-25T10:10:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.669+0000 7f29c1ffb640 1 -- 172.21.15.16:0/3749135339 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b40317a0 con 0x7f29c415dce0 2024-04-25T10:10:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.669+0000 7f29c1ffb640 1 -- 172.21.15.16:0/3749135339 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4031b70 con 0x7f29c415dce0 2024-04-25T10:10:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.669+0000 7f29c9407640 1 -- 172.21.15.16:0/3749135339 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29c415dce0 msgr2=0x7f29c40d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.669+0000 7f29c9407640 1 --2- 172.21.15.16:0/3749135339 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29c415dce0 0x7f29c40d1a50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009d90 tx=0x7f29b402a5d0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.669+0000 7f29c9407640 1 -- 172.21.15.16:0/3749135339 shutdown_connections 2024-04-25T10:10:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.669+0000 7f29c9407640 1 --2- 172.21.15.16:0/3749135339 >> v2:172.21.15.186:3300/0 conn(0x7f29c40d35b0 0x7f29c40b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.669+0000 7f29c9407640 1 --2- 172.21.15.16:0/3749135339 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29c415dce0 0x7f29c40d1a50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.669+0000 7f29c9407640 1 -- 172.21.15.16:0/3749135339 >> 172.21.15.16:0/3749135339 conn(0x7f29c4072900 msgr2=0x7f29c40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:42.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.669+0000 7f29c9407640 1 -- 172.21.15.16:0/3749135339 shutdown_connections 2024-04-25T10:10:42.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.669+0000 7f29c9407640 1 -- 172.21.15.16:0/3749135339 wait complete. 2024-04-25T10:10:42.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.671+0000 7f29c9407640 1 Processor -- start 2024-04-25T10:10:42.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.671+0000 7f29c9407640 1 -- start start 2024-04-25T10:10:42.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.671+0000 7f29c9407640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c40d1f90 0x7f29c40c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.671+0000 7f29c9407640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29c40d35b0 0x7f29c40b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.671+0000 7f29c9407640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29c415dce0 0x7f29c415e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.671+0000 7f29c9407640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40d3090 con 0x7f29c40d1f90 2024-04-25T10:10:42.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.671+0000 7f29c9407640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40d2db0 con 0x7f29c40d35b0 2024-04-25T10:10:42.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.671+0000 7f29c9407640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40b9820 con 0x7f29c415dce0 2024-04-25T10:10:42.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.671+0000 7f29c2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c40d1f90 0x7f29c40c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.671+0000 7f29c2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c40d1f90 0x7f29c40c8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57330/0 (socket says 172.21.15.16:57330) 2024-04-25T10:10:42.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.671+0000 7f29c2ffd640 1 -- 172.21.15.16:0/1477887439 learned_addr learned my addr 172.21.15.16:0/1477887439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:42.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.672+0000 7f29c37fe640 1 --2- 172.21.15.16:0/1477887439 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29c415dce0 0x7f29c415e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.672+0000 7f29c27fc640 1 --2- 172.21.15.16:0/1477887439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29c40d35b0 0x7f29c40b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.672+0000 7f29c2ffd640 1 -- 172.21.15.16:0/1477887439 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29c415dce0 msgr2=0x7f29c415e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.672+0000 7f29c2ffd640 1 --2- 172.21.15.16:0/1477887439 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29c415dce0 0x7f29c415e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.672+0000 7f29c2ffd640 1 -- 172.21.15.16:0/1477887439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29c40d35b0 msgr2=0x7f29c40b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.672+0000 7f29c2ffd640 1 --2- 172.21.15.16:0/1477887439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29c40d35b0 0x7f29c40b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.672+0000 7f29c2ffd640 1 -- 172.21.15.16:0/1477887439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c40c8e80 con 0x7f29c40d1f90 2024-04-25T10:10:42.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.672+0000 7f29c2ffd640 1 --2- 172.21.15.16:0/1477887439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c40d1f90 0x7f29c40c8630 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009d90 tx=0x7f29b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.672+0000 7f29ab7fe640 1 -- 172.21.15.16:0/1477887439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b403f070 con 0x7f29c40d1f90 2024-04-25T10:10:42.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.672+0000 7f29ab7fe640 1 -- 172.21.15.16:0/1477887439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b4030be0 con 0x7f29c40d1f90 2024-04-25T10:10:42.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.672+0000 7f29c9407640 1 -- 172.21.15.16:0/1477887439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c40c9160 con 0x7f29c40d1f90 2024-04-25T10:10:42.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.672+0000 7f29ab7fe640 1 -- 172.21.15.16:0/1477887439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4009450 con 0x7f29c40d1f90 2024-04-25T10:10:42.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.672+0000 7f29c9407640 1 -- 172.21.15.16:0/1477887439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c40c9720 con 0x7f29c40d1f90 2024-04-25T10:10:42.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.673+0000 7f29c9407640 1 -- 172.21.15.16:0/1477887439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c40749b0 con 0x7f29c40d1f90 2024-04-25T10:10:42.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.674+0000 7f29ab7fe640 1 -- 172.21.15.16:0/1477887439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29b4043410 con 0x7f29c40d1f90 2024-04-25T10:10:42.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.674+0000 7f29ab7fe640 1 --2- 172.21.15.16:0/1477887439 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f29b006b4d0 0x7f29b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.674+0000 7f29ab7fe640 1 -- 172.21.15.16:0/1477887439 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f29b40ab360 con 0x7f29c40d1f90 2024-04-25T10:10:42.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.674+0000 7f29c27fc640 1 --2- 172.21.15.16:0/1477887439 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f29b006b4d0 0x7f29b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.675+0000 7f29c27fc640 1 --2- 172.21.15.16:0/1477887439 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f29b006b4d0 0x7f29b006d990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f29c41616e0 tx=0x7f29b8009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.678+0000 7f29ab7fe640 1 -- 172.21.15.16:0/1477887439 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f29b4079ae0 con 0x7f29c40d1f90 2024-04-25T10:10:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.900+0000 7f29c9407640 1 -- 172.21.15.16:0/1477887439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f29c40d1160 con 0x7f29c40d1f90 2024-04-25T10:10:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.900+0000 7f29ab7fe640 1 -- 172.21.15.16:0/1477887439 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7f29b407db80 con 0x7f29c40d1f90 2024-04-25T10:10:42.906 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:10:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.902+0000 7f29c9407640 1 -- 172.21.15.16:0/1477887439 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f29b006b4d0 msgr2=0x7f29b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.902+0000 7f29c9407640 1 --2- 172.21.15.16:0/1477887439 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f29b006b4d0 0x7f29b006d990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f29c41616e0 tx=0x7f29b8009190 comp rx=0 tx=0).stop 2024-04-25T10:10:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.902+0000 7f29c9407640 1 -- 172.21.15.16:0/1477887439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c40d1f90 msgr2=0x7f29c40c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.902+0000 7f29c9407640 1 --2- 172.21.15.16:0/1477887439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c40d1f90 0x7f29c40c8630 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009d90 tx=0x7f29b4004a40 comp rx=0 tx=0).stop 2024-04-25T10:10:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.902+0000 7f29c9407640 1 -- 172.21.15.16:0/1477887439 shutdown_connections 2024-04-25T10:10:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.902+0000 7f29c9407640 1 --2- 172.21.15.16:0/1477887439 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f29b006b4d0 0x7f29b006d990 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.902+0000 7f29c9407640 1 --2- 172.21.15.16:0/1477887439 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29c415dce0 0x7f29c415e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.902+0000 7f29c9407640 1 --2- 172.21.15.16:0/1477887439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29c40d35b0 0x7f29c40b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.902+0000 7f29c9407640 1 --2- 172.21.15.16:0/1477887439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c40d1f90 0x7f29c40c8630 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.902+0000 7f29c9407640 1 -- 172.21.15.16:0/1477887439 >> 172.21.15.16:0/1477887439 conn(0x7f29c4072900 msgr2=0x7f29c415cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.902+0000 7f29c9407640 1 -- 172.21.15.16:0/1477887439 shutdown_connections 2024-04-25T10:10:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:42.902+0000 7f29c9407640 1 -- 172.21.15.16:0/1477887439 wait complete. 2024-04-25T10:10:42.918 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:10:42.918 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:10:42.918 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-04-25T10:10:42.918 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:10:42.918 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:42.919 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:10:42.919 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:10:42.919 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:10:42.919 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:10:42.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:42.919 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-04-25T10:10:42.935 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:10:42.935 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:10:42.935 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:10:42.936 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:42.936 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:10:42.936 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:42.936 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:10:42.936 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:42.936 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-04-25T10:10:42.936 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:10:42.936 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:42.936 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-04-25T10:10:42.936 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:10:42.936 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:42.936 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-04-25T10:10:42.937 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:10:42.937 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42317sec preferred_lft 42317sec 2024-04-25T10:10:42.937 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-25T10:10:42.937 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:42.937 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:10:42.937 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:42.937 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-04-25T10:10:42.937 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:10:42.937 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:42.937 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:10:42.937 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:42.937 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f06b:fdff:fed4:a335/64 scope link 2024-04-25T10:10:42.937 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:42.937 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:10:42.937 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:42.938 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::90ad:4fff:fe91:3c33/64 scope link 2024-04-25T10:10:42.938 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:42.938 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:10:42.938 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether be:14:4a:3c:16:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:42.938 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::bc14:4aff:fe3c:1605/64 scope link 2024-04-25T10:10:42.938 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:42.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:42.938 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:10:42.998 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:10:42.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:43.000 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-04-25T10:10:43.056 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-04-25T10:10:43.056 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:10:43.056 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:10:43.056 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:10:43.056 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:10:43.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:43.057 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:10:43.057 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:10:43.057 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:10:43.057 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:10:43.057 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:10:43.057 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:10:43.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:43.142 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:10:43.158 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:10:43.158 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:10:43.158 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:10:43.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:43.158 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-04-25T10:10:43.223 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:10:43.223 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:43.279 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:10:43.279 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:10:43.279 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:43.334 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:10:43.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:43.335 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:10:43.416 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:10:43.417 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:10:43.417 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:10:43.425 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:10:43.425 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:10:43.425 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:10:43.425 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:10:43.425 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:10:43.430 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:10:43.430 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:10:43.430 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:10:43.430 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:10:43.431 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:10:43.431 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:10:43.431 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:10:43.431 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:10:43.431 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:10:43.431 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:10:43.431 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:10:43.431 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:10:43.431 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:10:43.433 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:10:43.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:43.433 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:43.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:43.502 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:10:43.558 INFO:teuthology.orchestra.run.smithi016.stdout:client6397 2024-04-25T10:10:43.558 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:10:43.558 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:43.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:43.614 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-25T10:10:43.670 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:10:43.670 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T10:10:43.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.749+0000 7fe99bed1f40 1 Processor -- start 2024-04-25T10:10:43.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.749+0000 7fe99bed1f40 1 -- start start 2024-04-25T10:10:43.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.750+0000 7fe99bed1f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556efe652390 0x556efe776c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.750+0000 7fe99bed1f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556efe652770 0x556efe777150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.750+0000 7fe99bed1f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556efe6558e0 0x556efe6e12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.750+0000 7fe99bed1f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556efe656650 con 0x556efe6558e0 2024-04-25T10:10:43.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.750+0000 7fe99bed1f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556efe5b3df0 con 0x556efe652770 2024-04-25T10:10:43.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.750+0000 7fe99bed1f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x556efe6568f0 con 0x556efe652390 2024-04-25T10:10:43.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.750+0000 7fe99a1d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556efe6558e0 0x556efe6e12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.750+0000 7fe99a1d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556efe6558e0 0x556efe6e12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57356/0 (socket says 172.21.15.16:57356) 2024-04-25T10:10:43.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.750+0000 7fe99a1d3640 1 -- 172.21.15.16:0/3600224214 learned_addr learned my addr 172.21.15.16:0/3600224214 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:43.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.750+0000 7fe9999d2640 1 --2- 172.21.15.16:0/3600224214 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556efe652390 0x556efe776c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.750+0000 7fe9991d1640 1 --2- 172.21.15.16:0/3600224214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556efe652770 0x556efe777150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.750+0000 7fe99a1d3640 1 -- 172.21.15.16:0/3600224214 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556efe652390 msgr2=0x556efe776c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:43.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.750+0000 7fe99a1d3640 1 --2- 172.21.15.16:0/3600224214 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556efe652390 0x556efe776c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.750+0000 7fe99a1d3640 1 -- 172.21.15.16:0/3600224214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556efe652770 msgr2=0x556efe777150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:43.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.750+0000 7fe99a1d3640 1 --2- 172.21.15.16:0/3600224214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556efe652770 0x556efe777150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.750+0000 7fe99a1d3640 1 -- 172.21.15.16:0/3600224214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556efe539c50 con 0x556efe6558e0 2024-04-25T10:10:43.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.750+0000 7fe99a1d3640 1 --2- 172.21.15.16:0/3600224214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556efe6558e0 0x556efe6e12c0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe99400b560 tx=0x7fe99400ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf1787e0476a2eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:43.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.751+0000 7fe98bfff640 1 -- 172.21.15.16:0/3600224214 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994014070 con 0x556efe6558e0 2024-04-25T10:10:43.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.751+0000 7fe98bfff640 1 -- 172.21.15.16:0/3600224214 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe994010800 con 0x556efe6558e0 2024-04-25T10:10:43.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.751+0000 7fe98bfff640 1 -- 172.21.15.16:0/3600224214 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994010c30 con 0x556efe6558e0 2024-04-25T10:10:43.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.751+0000 7fe99bed1f40 1 -- 172.21.15.16:0/3600224214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556efe6558e0 msgr2=0x556efe6e12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:43.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.751+0000 7fe99bed1f40 1 --2- 172.21.15.16:0/3600224214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556efe6558e0 0x556efe6e12c0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe99400b560 tx=0x7fe99400ba20 comp rx=0 tx=0).stop 2024-04-25T10:10:43.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.751+0000 7fe99bed1f40 1 -- 172.21.15.16:0/3600224214 shutdown_connections 2024-04-25T10:10:43.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.751+0000 7fe99bed1f40 1 --2- 172.21.15.16:0/3600224214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556efe6558e0 0x556efe6e12c0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.751+0000 7fe99bed1f40 1 --2- 172.21.15.16:0/3600224214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556efe652770 0x556efe777150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.751+0000 7fe99bed1f40 1 --2- 172.21.15.16:0/3600224214 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556efe652390 0x556efe776c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.751+0000 7fe99bed1f40 1 -- 172.21.15.16:0/3600224214 >> 172.21.15.16:0/3600224214 conn(0x556efe638480 msgr2=0x556efe6390f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:43.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.751+0000 7fe99bed1f40 1 -- 172.21.15.16:0/3600224214 shutdown_connections 2024-04-25T10:10:43.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.751+0000 7fe99bed1f40 1 -- 172.21.15.16:0/3600224214 wait complete. 2024-04-25T10:10:43.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.752+0000 7fe99bed1f40 1 Processor -- start 2024-04-25T10:10:43.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.752+0000 7fe99bed1f40 1 -- start start 2024-04-25T10:10:43.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.752+0000 7fe99bed1f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556efe652390 0x556efe657b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.752+0000 7fe99bed1f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556efe652770 0x556efe60c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.752+0000 7fe99bed1f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556efe6558e0 0x556efe77adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.752+0000 7fe99bed1f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556efe6d35a0 con 0x556efe652390 2024-04-25T10:10:43.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.752+0000 7fe99bed1f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556efe6e7410 con 0x556efe6558e0 2024-04-25T10:10:43.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.752+0000 7fe99bed1f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x556efe6e0c30 con 0x556efe652770 2024-04-25T10:10:43.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.753+0000 7fe9999d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556efe652390 0x556efe657b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.753+0000 7fe9999d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556efe652390 0x556efe657b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57370/0 (socket says 172.21.15.16:57370) 2024-04-25T10:10:43.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.753+0000 7fe9999d2640 1 -- 172.21.15.16:0/2218141297 learned_addr learned my addr 172.21.15.16:0/2218141297 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:43.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.753+0000 7fe99a1d3640 1 --2- 172.21.15.16:0/2218141297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556efe6558e0 0x556efe77adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.753+0000 7fe9991d1640 1 --2- 172.21.15.16:0/2218141297 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556efe652770 0x556efe60c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.753+0000 7fe9999d2640 1 -- 172.21.15.16:0/2218141297 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556efe652770 msgr2=0x556efe60c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:43.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.753+0000 7fe9999d2640 1 --2- 172.21.15.16:0/2218141297 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556efe652770 0x556efe60c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.753+0000 7fe9999d2640 1 -- 172.21.15.16:0/2218141297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556efe6558e0 msgr2=0x556efe77adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:43.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.753+0000 7fe9999d2640 1 --2- 172.21.15.16:0/2218141297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556efe6558e0 0x556efe77adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.753+0000 7fe9999d2640 1 -- 172.21.15.16:0/2218141297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556efe60cac0 con 0x556efe652390 2024-04-25T10:10:43.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.753+0000 7fe9999d2640 1 --2- 172.21.15.16:0/2218141297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556efe652390 0x556efe657b40 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe98c000c00 tx=0x7fe98c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:43.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.753+0000 7fe98affd640 1 -- 172.21.15.16:0/2218141297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c06b070 con 0x556efe652390 2024-04-25T10:10:43.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.753+0000 7fe98affd640 1 -- 172.21.15.16:0/2218141297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98c052910 con 0x556efe652390 2024-04-25T10:10:43.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.753+0000 7fe99bed1f40 1 -- 172.21.15.16:0/2218141297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556efe60cda0 con 0x556efe652390 2024-04-25T10:10:43.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.753+0000 7fe98affd640 1 -- 172.21.15.16:0/2218141297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c056070 con 0x556efe652390 2024-04-25T10:10:43.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.753+0000 7fe99bed1f40 1 -- 172.21.15.16:0/2218141297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556efe60d360 con 0x556efe652390 2024-04-25T10:10:43.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.755+0000 7fe98affd640 1 -- 172.21.15.16:0/2218141297 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe98c06e070 con 0x556efe652390 2024-04-25T10:10:43.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.755+0000 7fe98affd640 1 --2- 172.21.15.16:0/2218141297 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe97c069ee0 0x7fe97c06c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.755+0000 7fe98affd640 1 -- 172.21.15.16:0/2218141297 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fe98c0ccb10 con 0x556efe652390 2024-04-25T10:10:43.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.755+0000 7fe99bed1f40 1 --2- 172.21.15.16:0/2218141297 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x556efe60f5f0 0x556efe77dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.755+0000 7fe9991d1640 1 --2- 172.21.15.16:0/2218141297 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe97c069ee0 0x7fe97c06c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.755+0000 7fe99bed1f40 1 -- 172.21.15.16:0/2218141297 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75) v9 -- 0x556efe60db60 con 0x556efe60f5f0 2024-04-25T10:10:43.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.755+0000 7fe99a1d3640 1 --2- 172.21.15.16:0/2218141297 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x556efe60f5f0 0x556efe77dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.756+0000 7fe9991d1640 1 --2- 172.21.15.16:0/2218141297 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe97c069ee0 0x7fe97c06c3a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x556efe6d2f70 tx=0x7fe990003910 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:43.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.756+0000 7fe99a1d3640 1 --2- 172.21.15.16:0/2218141297 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x556efe60f5f0 0x556efe77dbd0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:43.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.757+0000 7fe99a1d3640 1 -- 172.21.15.16:0/2218141297 <== osd.1 v2:172.21.15.16:6808/1916704159 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v75'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe994010ad0 con 0x556efe60f5f0 2024-04-25T10:10:43.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.757+0000 7fe99bed1f40 1 -- 172.21.15.16:0/2218141297 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x556efe60f5f0 msgr2=0x556efe77dbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:43.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.757+0000 7fe99bed1f40 1 --2- 172.21.15.16:0/2218141297 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x556efe60f5f0 0x556efe77dbd0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.757+0000 7fe99bed1f40 1 -- 172.21.15.16:0/2218141297 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe97c069ee0 msgr2=0x7fe97c06c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:43.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.757+0000 7fe99bed1f40 1 --2- 172.21.15.16:0/2218141297 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe97c069ee0 0x7fe97c06c3a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x556efe6d2f70 tx=0x7fe990003910 comp rx=0 tx=0).stop 2024-04-25T10:10:43.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.757+0000 7fe99bed1f40 1 -- 172.21.15.16:0/2218141297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556efe652390 msgr2=0x556efe657b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:43.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.757+0000 7fe99bed1f40 1 --2- 172.21.15.16:0/2218141297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556efe652390 0x556efe657b40 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe98c000c00 tx=0x7fe98c002fb0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.757+0000 7fe99a1d3640 1 -- 172.21.15.16:0/2218141297 reap_dead start 2024-04-25T10:10:43.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.757+0000 7fe99bed1f40 1 -- 172.21.15.16:0/2218141297 shutdown_connections 2024-04-25T10:10:43.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.757+0000 7fe99bed1f40 1 -- 172.21.15.16:0/2218141297 >> 172.21.15.16:0/2218141297 conn(0x556efe638480 msgr2=0x556efe6e5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:43.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.758+0000 7fe99bed1f40 1 -- 172.21.15.16:0/2218141297 shutdown_connections 2024-04-25T10:10:43.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:43.758+0000 7fe99bed1f40 1 -- 172.21.15.16:0/2218141297 wait complete. 2024-04-25T10:10:43.768 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:10:43.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:10:43.806 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:10:43.855 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:10:43.855 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:10:43.855 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:43.871 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:10:43.871 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:43.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:44.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.063+0000 7fab5e1b0640 1 Processor -- start 2024-04-25T10:10:44.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.063+0000 7fab5e1b0640 1 -- start start 2024-04-25T10:10:44.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.064+0000 7fab5e1b0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab5815a870 0x7fab580cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.064+0000 7fab5e1b0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab580cfc40 0x7fab5815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.064+0000 7fab5e1b0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab580cdb10 con 0x7fab5806e2e0 2024-04-25T10:10:44.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.064+0000 7fab5e1b0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab580ce720 con 0x7fab580cfc40 2024-04-25T10:10:44.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.064+0000 7fab5e1b0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab580cddf0 con 0x7fab5815a870 2024-04-25T10:10:44.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.064+0000 7fab57fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab580cfc40 0x7fab5815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.064+0000 7fab56ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fab5806e2e0 0x7fab580cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56680/0 (socket says 172.21.15.16:56680) 2024-04-25T10:10:44.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.064+0000 7fab56ffd640 1 -- 172.21.15.16:0/22389041 learned_addr learned my addr 172.21.15.16:0/22389041 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:44.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.064+0000 7fab577fe640 1 --2- 172.21.15.16:0/22389041 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab5815a870 0x7fab580cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.064+0000 7fab57fff640 1 -- 172.21.15.16:0/22389041 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab5815a870 msgr2=0x7fab580cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:44.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.064+0000 7fab57fff640 1 --2- 172.21.15.16:0/22389041 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab5815a870 0x7fab580cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.065+0000 7fab57fff640 1 -- 172.21.15.16:0/22389041 >> v1:172.21.15.16:6789/0 conn(0x7fab5806e2e0 legacy=0x7fab580cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.065+0000 7fab57fff640 1 -- 172.21.15.16:0/22389041 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab580ce0d0 con 0x7fab580cfc40 2024-04-25T10:10:44.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.065+0000 7fab567fc640 1 -- 172.21.15.16:0/22389041 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 415935574 0 0) 0x7fab580cdb10 con 0x7fab5806e2e0 2024-04-25T10:10:44.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.065+0000 7fab57fff640 1 --2- 172.21.15.16:0/22389041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab580cfc40 0x7fab5815ac50 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fab50009d90 tx=0x7fab5002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a489399fcbe32c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:44.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.065+0000 7fab567fc640 1 -- 172.21.15.16:0/22389041 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50007740 con 0x7fab580cfc40 2024-04-25T10:10:44.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.065+0000 7fab567fc640 1 -- 172.21.15.16:0/22389041 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab500078e0 con 0x7fab580cfc40 2024-04-25T10:10:44.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.065+0000 7fab567fc640 1 -- 172.21.15.16:0/22389041 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50007cb0 con 0x7fab580cfc40 2024-04-25T10:10:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.066+0000 7fab5e1b0640 1 -- 172.21.15.16:0/22389041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab580cfc40 msgr2=0x7fab5815ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.066+0000 7fab5e1b0640 1 --2- 172.21.15.16:0/22389041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab580cfc40 0x7fab5815ac50 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fab50009d90 tx=0x7fab5002a660 comp rx=0 tx=0).stop 2024-04-25T10:10:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.066+0000 7fab5e1b0640 1 -- 172.21.15.16:0/22389041 shutdown_connections 2024-04-25T10:10:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.066+0000 7fab5e1b0640 1 --2- 172.21.15.16:0/22389041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab580cfc40 0x7fab5815ac50 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.066+0000 7fab5e1b0640 1 --2- 172.21.15.16:0/22389041 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab5815a870 0x7fab580cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.066+0000 7fab5e1b0640 1 -- 172.21.15.16:0/22389041 >> 172.21.15.16:0/22389041 conn(0x7fab5806aba0 msgr2=0x7fab5806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.066+0000 7fab5e1b0640 1 -- 172.21.15.16:0/22389041 shutdown_connections 2024-04-25T10:10:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.066+0000 7fab5e1b0640 1 -- 172.21.15.16:0/22389041 wait complete. 2024-04-25T10:10:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.067+0000 7fab5e1b0640 1 Processor -- start 2024-04-25T10:10:44.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.067+0000 7fab5e1b0640 1 -- start start 2024-04-25T10:10:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.068+0000 7fab5e1b0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab5806e2e0 0x7fab5814b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.068+0000 7fab5e1b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab580cfc40 0x7fab580b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.068+0000 7fab5e1b0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab5815a870 0x7fab580b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.068+0000 7fab5e1b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab580cd700 con 0x7fab580cfc40 2024-04-25T10:10:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.068+0000 7fab5e1b0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab580cd580 con 0x7fab5815a870 2024-04-25T10:10:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.068+0000 7fab5e1b0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab580cd880 con 0x7fab5806e2e0 2024-04-25T10:10:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.068+0000 7fab56ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab580cfc40 0x7fab580b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.068+0000 7fab56ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab580cfc40 0x7fab580b0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57380/0 (socket says 172.21.15.16:57380) 2024-04-25T10:10:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.068+0000 7fab56ffd640 1 -- 172.21.15.16:0/3011055852 learned_addr learned my addr 172.21.15.16:0/3011055852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.068+0000 7fab577fe640 1 --2- 172.21.15.16:0/3011055852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab5806e2e0 0x7fab5814b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.068+0000 7fab57fff640 1 --2- 172.21.15.16:0/3011055852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab5815a870 0x7fab580b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.069+0000 7fab56ffd640 1 -- 172.21.15.16:0/3011055852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab5806e2e0 msgr2=0x7fab5814b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.069+0000 7fab56ffd640 1 --2- 172.21.15.16:0/3011055852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab5806e2e0 0x7fab5814b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.069+0000 7fab56ffd640 1 -- 172.21.15.16:0/3011055852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab5815a870 msgr2=0x7fab580b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.069+0000 7fab56ffd640 1 --2- 172.21.15.16:0/3011055852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab5815a870 0x7fab580b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.069+0000 7fab56ffd640 1 -- 172.21.15.16:0/3011055852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab58155ac0 con 0x7fab580cfc40 2024-04-25T10:10:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.069+0000 7fab57fff640 1 --2- 172.21.15.16:0/3011055852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab5815a870 0x7fab580b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.069+0000 7fab56ffd640 1 --2- 172.21.15.16:0/3011055852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab580cfc40 0x7fab580b0390 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00c4d0 tx=0x7fab4c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.069+0000 7fab3ffff640 1 -- 172.21.15.16:0/3011055852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c018070 con 0x7fab580cfc40 2024-04-25T10:10:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.069+0000 7fab3ffff640 1 -- 172.21.15.16:0/3011055852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4c007500 con 0x7fab580cfc40 2024-04-25T10:10:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.069+0000 7fab5e1b0640 1 -- 172.21.15.16:0/3011055852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5814b6e0 con 0x7fab580cfc40 2024-04-25T10:10:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.069+0000 7fab3ffff640 1 -- 172.21.15.16:0/3011055852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c004a20 con 0x7fab580cfc40 2024-04-25T10:10:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.069+0000 7fab5e1b0640 1 -- 172.21.15.16:0/3011055852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab580b1510 con 0x7fab580cfc40 2024-04-25T10:10:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.070+0000 7fab5e1b0640 1 -- 172.21.15.16:0/3011055852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab580749b0 con 0x7fab580cfc40 2024-04-25T10:10:44.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.071+0000 7fab3ffff640 1 -- 172.21.15.16:0/3011055852 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab4c01b070 con 0x7fab580cfc40 2024-04-25T10:10:44.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.071+0000 7fab3ffff640 1 --2- 172.21.15.16:0/3011055852 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fab4406b4d0 0x7fab4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.071+0000 7fab3ffff640 1 -- 172.21.15.16:0/3011055852 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fab4c089510 con 0x7fab580cfc40 2024-04-25T10:10:44.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.071+0000 7fab577fe640 1 --2- 172.21.15.16:0/3011055852 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fab4406b4d0 0x7fab4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.072+0000 7fab577fe640 1 --2- 172.21.15.16:0/3011055852 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fab4406b4d0 0x7fab4406d990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fab580af010 tx=0x7fab48008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.075+0000 7fab3ffff640 1 -- 172.21.15.16:0/3011055852 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab58155ac0 con 0x7fab580cfc40 2024-04-25T10:10:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.258+0000 7fab5e1b0640 1 -- 172.21.15.16:0/3011055852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab580b0f50 con 0x7fab580cfc40 2024-04-25T10:10:44.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.259+0000 7fab3ffff640 1 -- 172.21.15.16:0/3011055852 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16885 (secure 0 0 0) 0x7fab4c057c90 con 0x7fab580cfc40 2024-04-25T10:10:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:44.265 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:09:53.479005+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:20.361009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:21.048892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.19","mappings":[{"from":2,"to":7}]},{"pgid":"11.26","mappings":[{"from":1,"to":4}]},{"pgid":"11.27","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.186:6837/3292077067":"2024-04-26T10:09:31.374252+0000","172.21.15.186:6836/3292077067":"2024-04-26T10:09:31.374252+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:44.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.261+0000 7fab5e1b0640 1 -- 172.21.15.16:0/3011055852 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fab4406b4d0 msgr2=0x7fab4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.261+0000 7fab5e1b0640 1 --2- 172.21.15.16:0/3011055852 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fab4406b4d0 0x7fab4406d990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fab580af010 tx=0x7fab48008040 comp rx=0 tx=0).stop 2024-04-25T10:10:44.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.261+0000 7fab5e1b0640 1 -- 172.21.15.16:0/3011055852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab580cfc40 msgr2=0x7fab580b0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.261+0000 7fab5e1b0640 1 --2- 172.21.15.16:0/3011055852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab580cfc40 0x7fab580b0390 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00c4d0 tx=0x7fab4c00c990 comp rx=0 tx=0).stop 2024-04-25T10:10:44.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.261+0000 7fab5e1b0640 1 -- 172.21.15.16:0/3011055852 shutdown_connections 2024-04-25T10:10:44.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.261+0000 7fab5e1b0640 1 --2- 172.21.15.16:0/3011055852 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fab4406b4d0 0x7fab4406d990 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.261+0000 7fab5e1b0640 1 --2- 172.21.15.16:0/3011055852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab5815a870 0x7fab580b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.261+0000 7fab5e1b0640 1 --2- 172.21.15.16:0/3011055852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab580cfc40 0x7fab580b0390 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.261+0000 7fab5e1b0640 1 --2- 172.21.15.16:0/3011055852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab5806e2e0 0x7fab5814b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.261+0000 7fab5e1b0640 1 -- 172.21.15.16:0/3011055852 >> 172.21.15.16:0/3011055852 conn(0x7fab5806aba0 msgr2=0x7fab580b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:44.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.261+0000 7fab5e1b0640 1 -- 172.21.15.16:0/3011055852 shutdown_connections 2024-04-25T10:10:44.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.261+0000 7fab5e1b0640 1 -- 172.21.15.16:0/3011055852 wait complete. 2024-04-25T10:10:44.278 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:09:20.361009+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:09:21.048892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:10:44.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:10:44.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.377+0000 7fd889a1f640 1 Processor -- start 2024-04-25T10:10:44.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.377+0000 7fd889a1f640 1 -- start start 2024-04-25T10:10:44.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.378+0000 7fd889a1f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd88415dc80 0x7fd884161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.378+0000 7fd889a1f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd8840b97c0 0x7fd8840b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.378+0000 7fd889a1f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8840b5a30 0x7fd88406bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.378+0000 7fd889a1f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8840b5e60 con 0x7fd88415dc80 2024-04-25T10:10:44.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.378+0000 7fd889a1f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8841627a0 con 0x7fd8840b97c0 2024-04-25T10:10:44.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.378+0000 7fd889a1f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8841621e0 con 0x7fd8840b5a30 2024-04-25T10:10:44.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.378+0000 7fd882ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd88415dc80 0x7fd884161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.378+0000 7fd882ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd88415dc80 0x7fd884161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57390/0 (socket says 172.21.15.16:57390) 2024-04-25T10:10:44.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.378+0000 7fd882ffd640 1 -- 172.21.15.16:0/2101480891 learned_addr learned my addr 172.21.15.16:0/2101480891 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:44.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.378+0000 7fd8837fe640 1 --2- 172.21.15.16:0/2101480891 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8840b5a30 0x7fd88406bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.378+0000 7fd87a7fc640 1 --2- 172.21.15.16:0/2101480891 >> v2:172.21.15.186:3300/0 conn(0x7fd8840b97c0 0x7fd8840b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.378+0000 7fd882ffd640 1 -- 172.21.15.16:0/2101480891 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8840b5a30 msgr2=0x7fd88406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:44.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.378+0000 7fd882ffd640 1 --2- 172.21.15.16:0/2101480891 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8840b5a30 0x7fd88406bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.378+0000 7fd882ffd640 1 -- 172.21.15.16:0/2101480891 >> v2:172.21.15.186:3300/0 conn(0x7fd8840b97c0 msgr2=0x7fd8840b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:44.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.378+0000 7fd882ffd640 1 --2- 172.21.15.16:0/2101480891 >> v2:172.21.15.186:3300/0 conn(0x7fd8840b97c0 0x7fd8840b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.378+0000 7fd882ffd640 1 -- 172.21.15.16:0/2101480891 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8841624c0 con 0x7fd88415dc80 2024-04-25T10:10:44.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.379+0000 7fd882ffd640 1 --2- 172.21.15.16:0/2101480891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88415dc80 0x7fd884161870 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fd88406ba70 tx=0x7fd87402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1714e534dd54b9a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:44.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.379+0000 7fd8827fc640 1 -- 172.21.15.16:0/2101480891 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874037070 con 0x7fd88415dc80 2024-04-25T10:10:44.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.379+0000 7fd8827fc640 1 -- 172.21.15.16:0/2101480891 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd874032440 con 0x7fd88415dc80 2024-04-25T10:10:44.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.379+0000 7fd8827fc640 1 -- 172.21.15.16:0/2101480891 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874032810 con 0x7fd88415dc80 2024-04-25T10:10:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.380+0000 7fd889a1f640 1 -- 172.21.15.16:0/2101480891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88415dc80 msgr2=0x7fd884161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.380+0000 7fd889a1f640 1 --2- 172.21.15.16:0/2101480891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88415dc80 0x7fd884161870 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fd88406ba70 tx=0x7fd87402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.380+0000 7fd889a1f640 1 -- 172.21.15.16:0/2101480891 shutdown_connections 2024-04-25T10:10:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.380+0000 7fd889a1f640 1 --2- 172.21.15.16:0/2101480891 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8840b5a30 0x7fd88406bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.380+0000 7fd889a1f640 1 --2- 172.21.15.16:0/2101480891 >> v2:172.21.15.186:3300/0 conn(0x7fd8840b97c0 0x7fd8840b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.380+0000 7fd889a1f640 1 --2- 172.21.15.16:0/2101480891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88415dc80 0x7fd884161870 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.380+0000 7fd889a1f640 1 -- 172.21.15.16:0/2101480891 >> 172.21.15.16:0/2101480891 conn(0x7fd8840d04f0 msgr2=0x7fd8840d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.380+0000 7fd889a1f640 1 -- 172.21.15.16:0/2101480891 shutdown_connections 2024-04-25T10:10:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.380+0000 7fd889a1f640 1 -- 172.21.15.16:0/2101480891 wait complete. 2024-04-25T10:10:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.382+0000 7fd889a1f640 1 Processor -- start 2024-04-25T10:10:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.382+0000 7fd889a1f640 1 -- start start 2024-04-25T10:10:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.383+0000 7fd889a1f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8840b5a30 0x7fd884158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.383+0000 7fd889a1f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8840b97c0 0x7fd884156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.383+0000 7fd889a1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88415dc80 0x7fd88415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.383+0000 7fd889a1f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd884161db0 con 0x7fd88415dc80 2024-04-25T10:10:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.383+0000 7fd889a1f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88415e060 con 0x7fd8840b5a30 2024-04-25T10:10:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.383+0000 7fd889a1f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd884161f30 con 0x7fd8840b97c0 2024-04-25T10:10:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.383+0000 7fd8837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88415dc80 0x7fd88415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.383+0000 7fd8837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88415dc80 0x7fd88415a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57394/0 (socket says 172.21.15.16:57394) 2024-04-25T10:10:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.383+0000 7fd8837fe640 1 -- 172.21.15.16:0/327031690 learned_addr learned my addr 172.21.15.16:0/327031690 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.383+0000 7fd882ffd640 1 --2- 172.21.15.16:0/327031690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8840b5a30 0x7fd884158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.383+0000 7fd87a7fc640 1 --2- 172.21.15.16:0/327031690 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8840b97c0 0x7fd884156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.383+0000 7fd8837fe640 1 -- 172.21.15.16:0/327031690 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8840b97c0 msgr2=0x7fd884156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.383+0000 7fd8837fe640 1 --2- 172.21.15.16:0/327031690 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8840b97c0 0x7fd884156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.383+0000 7fd8837fe640 1 -- 172.21.15.16:0/327031690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8840b5a30 msgr2=0x7fd884158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.383+0000 7fd8837fe640 1 --2- 172.21.15.16:0/327031690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8840b5a30 0x7fd884158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.383+0000 7fd8837fe640 1 -- 172.21.15.16:0/327031690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8841586a0 con 0x7fd88415dc80 2024-04-25T10:10:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.384+0000 7fd882ffd640 1 --2- 172.21.15.16:0/327031690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8840b5a30 0x7fd884158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.384+0000 7fd8837fe640 1 --2- 172.21.15.16:0/327031690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88415dc80 0x7fd88415a070 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fd87c007e60 tx=0x7fd87c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.384+0000 7fd880ff9640 1 -- 172.21.15.16:0/327031690 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c014620 con 0x7fd88415dc80 2024-04-25T10:10:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.384+0000 7fd880ff9640 1 -- 172.21.15.16:0/327031690 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c019070 con 0x7fd88415dc80 2024-04-25T10:10:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.384+0000 7fd889a1f640 1 -- 172.21.15.16:0/327031690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd884158980 con 0x7fd88415dc80 2024-04-25T10:10:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.384+0000 7fd880ff9640 1 -- 172.21.15.16:0/327031690 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c014a20 con 0x7fd88415dc80 2024-04-25T10:10:44.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.384+0000 7fd889a1f640 1 -- 172.21.15.16:0/327031690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88415fa00 con 0x7fd88415dc80 2024-04-25T10:10:44.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.385+0000 7fd889a1f640 1 -- 172.21.15.16:0/327031690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8840750b0 con 0x7fd88415dc80 2024-04-25T10:10:44.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.385+0000 7fd880ff9640 1 -- 172.21.15.16:0/327031690 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd87c014bc0 con 0x7fd88415dc80 2024-04-25T10:10:44.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.386+0000 7fd880ff9640 1 --2- 172.21.15.16:0/327031690 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd86406b4d0 0x7fd86406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.386+0000 7fd880ff9640 1 -- 172.21.15.16:0/327031690 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fd87c08abe0 con 0x7fd88415dc80 2024-04-25T10:10:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.386+0000 7fd882ffd640 1 --2- 172.21.15.16:0/327031690 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd86406b4d0 0x7fd86406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.387+0000 7fd882ffd640 1 --2- 172.21.15.16:0/327031690 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd86406b4d0 0x7fd86406d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd874034000 tx=0x7fd87402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.390+0000 7fd880ff9640 1 -- 172.21.15.16:0/327031690 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd87c059910 con 0x7fd88415dc80 2024-04-25T10:10:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.569+0000 7fd889a1f640 1 -- 172.21.15.16:0/327031690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8840d1780 con 0x7fd88415dc80 2024-04-25T10:10:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.570+0000 7fd880ff9640 1 -- 172.21.15.16:0/327031690 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd87c059210 con 0x7fd88415dc80 2024-04-25T10:10:44.575 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:10:44.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.572+0000 7fd889a1f640 1 -- 172.21.15.16:0/327031690 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd86406b4d0 msgr2=0x7fd86406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.572+0000 7fd889a1f640 1 --2- 172.21.15.16:0/327031690 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd86406b4d0 0x7fd86406d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd874034000 tx=0x7fd87402e000 comp rx=0 tx=0).stop 2024-04-25T10:10:44.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.572+0000 7fd889a1f640 1 -- 172.21.15.16:0/327031690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88415dc80 msgr2=0x7fd88415a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.572+0000 7fd889a1f640 1 --2- 172.21.15.16:0/327031690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88415dc80 0x7fd88415a070 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fd87c007e60 tx=0x7fd87c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.572+0000 7fd889a1f640 1 -- 172.21.15.16:0/327031690 shutdown_connections 2024-04-25T10:10:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.572+0000 7fd889a1f640 1 --2- 172.21.15.16:0/327031690 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd86406b4d0 0x7fd86406d990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.572+0000 7fd889a1f640 1 --2- 172.21.15.16:0/327031690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88415dc80 0x7fd88415a070 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.572+0000 7fd889a1f640 1 --2- 172.21.15.16:0/327031690 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8840b97c0 0x7fd884156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.572+0000 7fd889a1f640 1 --2- 172.21.15.16:0/327031690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8840b5a30 0x7fd884158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.572+0000 7fd889a1f640 1 -- 172.21.15.16:0/327031690 >> 172.21.15.16:0/327031690 conn(0x7fd8840d04f0 msgr2=0x7fd8840b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.572+0000 7fd889a1f640 1 -- 172.21.15.16:0/327031690 shutdown_connections 2024-04-25T10:10:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.572+0000 7fd889a1f640 1 -- 172.21.15.16:0/327031690 wait complete. 2024-04-25T10:10:44.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:10:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.687+0000 7f786b66a640 1 Processor -- start 2024-04-25T10:10:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.688+0000 7f786b66a640 1 -- start start 2024-04-25T10:10:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.688+0000 7f786b66a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f78640cf510 0x7f78640cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.688+0000 7f786b66a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78640cff00 0x7f78640d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.688+0000 7f786b66a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78640cc830 con 0x7f786415bbe0 2024-04-25T10:10:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.688+0000 7f786b66a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78640ccb10 con 0x7f78640cff00 2024-04-25T10:10:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.688+0000 7f786b66a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78640ccdf0 con 0x7f78640cf510 2024-04-25T10:10:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.688+0000 7f7869c07640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78640cff00 0x7f78640d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.688+0000 7f7869406640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f786415bbe0 0x7f78640cd8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56696/0 (socket says 172.21.15.16:56696) 2024-04-25T10:10:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.688+0000 7f7869c07640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78640cff00 0x7f78640d03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57410/0 (socket says 172.21.15.16:57410) 2024-04-25T10:10:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.688+0000 7f7869406640 1 -- 172.21.15.16:0/1861823874 learned_addr learned my addr 172.21.15.16:0/1861823874 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.689+0000 7f7869c07640 1 -- 172.21.15.16:0/1861823874 >> v2:172.21.15.186:3300/0 conn(0x7f78640cf510 msgr2=0x7f78640cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.689+0000 7f7868c05640 1 --2- 172.21.15.16:0/1861823874 >> v2:172.21.15.186:3300/0 conn(0x7f78640cf510 0x7f78640cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.689+0000 7f7869c07640 1 --2- 172.21.15.16:0/1861823874 >> v2:172.21.15.186:3300/0 conn(0x7f78640cf510 0x7f78640cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.689+0000 7f7869c07640 1 -- 172.21.15.16:0/1861823874 >> v1:172.21.15.16:6789/0 conn(0x7f786415bbe0 legacy=0x7f78640cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.689+0000 7f785bfff640 1 -- 172.21.15.16:0/1861823874 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 500427153 0 0) 0x7f78640cc830 con 0x7f786415bbe0 2024-04-25T10:10:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.689+0000 7f7869c07640 1 -- 172.21.15.16:0/1861823874 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78640cd0d0 con 0x7f78640cff00 2024-04-25T10:10:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.689+0000 7f7868c05640 1 --2- 172.21.15.16:0/1861823874 >> v2:172.21.15.186:3300/0 conn(0x7f78640cf510 0x7f78640cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.690+0000 7f7869c07640 1 --2- 172.21.15.16:0/1861823874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78640cff00 0x7f78640d03b0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f78640b42a0 tx=0x7f785c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d89a6807a22ee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.690+0000 7f785bfff640 1 -- 172.21.15.16:0/1861823874 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c007740 con 0x7f78640cff00 2024-04-25T10:10:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.690+0000 7f785bfff640 1 -- 172.21.15.16:0/1861823874 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785c0078e0 con 0x7f78640cff00 2024-04-25T10:10:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.690+0000 7f785bfff640 1 -- 172.21.15.16:0/1861823874 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c007cb0 con 0x7f78640cff00 2024-04-25T10:10:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.690+0000 7f786b66a640 1 -- 172.21.15.16:0/1861823874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78640cff00 msgr2=0x7f78640d03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.690+0000 7f786b66a640 1 --2- 172.21.15.16:0/1861823874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78640cff00 0x7f78640d03b0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f78640b42a0 tx=0x7f785c02a680 comp rx=0 tx=0).stop 2024-04-25T10:10:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.690+0000 7f786b66a640 1 -- 172.21.15.16:0/1861823874 shutdown_connections 2024-04-25T10:10:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.690+0000 7f786b66a640 1 --2- 172.21.15.16:0/1861823874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78640cff00 0x7f78640d03b0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.690+0000 7f786b66a640 1 --2- 172.21.15.16:0/1861823874 >> v2:172.21.15.186:3300/0 conn(0x7f78640cf510 0x7f78640cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.690+0000 7f786b66a640 1 -- 172.21.15.16:0/1861823874 >> 172.21.15.16:0/1861823874 conn(0x7f78640b3430 msgr2=0x7f78640b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.690+0000 7f786b66a640 1 -- 172.21.15.16:0/1861823874 shutdown_connections 2024-04-25T10:10:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.690+0000 7f786b66a640 1 -- 172.21.15.16:0/1861823874 wait complete. 2024-04-25T10:10:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.692+0000 7f786b66a640 1 Processor -- start 2024-04-25T10:10:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.692+0000 7f786b66a640 1 -- start start 2024-04-25T10:10:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.693+0000 7f786b66a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78640cf510 0x7f786415eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.693+0000 7f786b66a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78640cff00 0x7f78641f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.693+0000 7f786b66a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f786415bbe0 0x7f78641f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.693+0000 7f786b66a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78640cc450 con 0x7f78640cff00 2024-04-25T10:10:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.693+0000 7f786b66a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786415bfc0 con 0x7f786415bbe0 2024-04-25T10:10:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.693+0000 7f786b66a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78640cc5d0 con 0x7f78640cf510 2024-04-25T10:10:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.694+0000 7f7868c05640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78640cff00 0x7f78641f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.694+0000 7f7868c05640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78640cff00 0x7f78641f4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57420/0 (socket says 172.21.15.16:57420) 2024-04-25T10:10:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.694+0000 7f7868c05640 1 -- 172.21.15.16:0/4117497058 learned_addr learned my addr 172.21.15.16:0/4117497058 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.694+0000 7f7868c05640 1 -- 172.21.15.16:0/4117497058 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78640cf510 msgr2=0x7f786415eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.694+0000 7f7869c07640 1 --2- 172.21.15.16:0/4117497058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f786415bbe0 0x7f78641f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.694+0000 7f7869406640 1 --2- 172.21.15.16:0/4117497058 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78640cf510 0x7f786415eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.694+0000 7f7868c05640 1 --2- 172.21.15.16:0/4117497058 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78640cf510 0x7f786415eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.694+0000 7f7868c05640 1 -- 172.21.15.16:0/4117497058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f786415bbe0 msgr2=0x7f78641f54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.694+0000 7f7868c05640 1 --2- 172.21.15.16:0/4117497058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f786415bbe0 0x7f78641f54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.694+0000 7f7868c05640 1 -- 172.21.15.16:0/4117497058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78640b1460 con 0x7f78640cff00 2024-04-25T10:10:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.694+0000 7f7868c05640 1 --2- 172.21.15.16:0/4117497058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78640cff00 0x7f78641f4fb0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f785400c820 tx=0x7f785400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.694+0000 7f7859ffb640 1 -- 172.21.15.16:0/4117497058 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854004230 con 0x7f78640cff00 2024-04-25T10:10:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.694+0000 7f7859ffb640 1 -- 172.21.15.16:0/4117497058 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78540043d0 con 0x7f78640cff00 2024-04-25T10:10:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.694+0000 7f786b66a640 1 -- 172.21.15.16:0/4117497058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78640b1740 con 0x7f78640cff00 2024-04-25T10:10:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.694+0000 7f7859ffb640 1 -- 172.21.15.16:0/4117497058 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854007a70 con 0x7f78640cff00 2024-04-25T10:10:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.695+0000 7f786b66a640 1 -- 172.21.15.16:0/4117497058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786415f410 con 0x7f78640cff00 2024-04-25T10:10:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.695+0000 7f786b66a640 1 -- 172.21.15.16:0/4117497058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78640b1c60 con 0x7f78640cff00 2024-04-25T10:10:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.696+0000 7f7859ffb640 1 -- 172.21.15.16:0/4117497058 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7854007c10 con 0x7f78640cff00 2024-04-25T10:10:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.696+0000 7f7859ffb640 1 --2- 172.21.15.16:0/4117497058 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f784806b4d0 0x7f784806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.696+0000 7f7859ffb640 1 -- 172.21.15.16:0/4117497058 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f785408a330 con 0x7f78640cff00 2024-04-25T10:10:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.697+0000 7f7869406640 1 --2- 172.21.15.16:0/4117497058 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f784806b4d0 0x7f784806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.697+0000 7f7869406640 1 --2- 172.21.15.16:0/4117497058 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f784806b4d0 0x7f784806d990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7860000f60 tx=0x7f7860009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.701+0000 7f7859ffb640 1 -- 172.21.15.16:0/4117497058 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78640b1c60 con 0x7f78640cff00 2024-04-25T10:10:44.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.881+0000 7f786b66a640 1 -- 172.21.15.16:0/4117497058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78640750b0 con 0x7f78640cff00 2024-04-25T10:10:44.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.882+0000 7f7859ffb640 1 -- 172.21.15.16:0/4117497058 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7854015020 con 0x7f78640cff00 2024-04-25T10:10:44.887 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:10:44.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.884+0000 7f786b66a640 1 -- 172.21.15.16:0/4117497058 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f784806b4d0 msgr2=0x7f784806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.884+0000 7f786b66a640 1 --2- 172.21.15.16:0/4117497058 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f784806b4d0 0x7f784806d990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7860000f60 tx=0x7f7860009040 comp rx=0 tx=0).stop 2024-04-25T10:10:44.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.884+0000 7f786b66a640 1 -- 172.21.15.16:0/4117497058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78640cff00 msgr2=0x7f78641f4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.884+0000 7f786b66a640 1 --2- 172.21.15.16:0/4117497058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78640cff00 0x7f78641f4fb0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f785400c820 tx=0x7f785400cce0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.884+0000 7f786b66a640 1 -- 172.21.15.16:0/4117497058 shutdown_connections 2024-04-25T10:10:44.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.884+0000 7f786b66a640 1 --2- 172.21.15.16:0/4117497058 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f784806b4d0 0x7f784806d990 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.884+0000 7f786b66a640 1 --2- 172.21.15.16:0/4117497058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f786415bbe0 0x7f78641f54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.884+0000 7f786b66a640 1 --2- 172.21.15.16:0/4117497058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78640cff00 0x7f78641f4fb0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.884+0000 7f786b66a640 1 --2- 172.21.15.16:0/4117497058 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78640cf510 0x7f786415eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.884+0000 7f786b66a640 1 -- 172.21.15.16:0/4117497058 >> 172.21.15.16:0/4117497058 conn(0x7f78640b3430 msgr2=0x7f78640c6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:44.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.884+0000 7f786b66a640 1 -- 172.21.15.16:0/4117497058 shutdown_connections 2024-04-25T10:10:44.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.884+0000 7f786b66a640 1 -- 172.21.15.16:0/4117497058 wait complete. 2024-04-25T10:10:44.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:10:45.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.998+0000 7fa451fe7640 1 Processor -- start 2024-04-25T10:10:45.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.998+0000 7fa451fe7640 1 -- start start 2024-04-25T10:10:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.998+0000 7fa451fe7640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa44c06aba0 0x7fa44c0b1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.998+0000 7fa451fe7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44c0b1f60 con 0x7fa44c0ce3a0 2024-04-25T10:10:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.998+0000 7fa451fe7640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44c0b0380 con 0x7fa44c15d450 2024-04-25T10:10:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.998+0000 7fa451fe7640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa44c0b0130 con 0x7fa44c06aba0 2024-04-25T10:10:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa44c0ce3a0 0x7fa44c06af80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56698/0 (socket says 172.21.15.16:56698) 2024-04-25T10:10:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44affd640 1 -- 172.21.15.16:0/1975490288 learned_addr learned my addr 172.21.15.16:0/1975490288 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44a7fc640 1 -- 172.21.15.16:0/1975490288 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499415830 0 0) 0x7fa44c0b1f60 con 0x7fa44c0ce3a0 2024-04-25T10:10:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44b7fe640 1 --2- 172.21.15.16:0/1975490288 >> v2:172.21.15.186:3300/0 conn(0x7fa44c06aba0 0x7fa44c0b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44a7fc640 1 -- 172.21.15.16:0/1975490288 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa434003520 con 0x7fa44c0ce3a0 2024-04-25T10:10:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44a7fc640 1 -- 172.21.15.16:0/1975490288 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3636065577 0 0) 0x7fa434003520 con 0x7fa44c0ce3a0 2024-04-25T10:10:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44a7fc640 1 -- 172.21.15.16:0/1975490288 >> v2:172.21.15.186:3300/0 conn(0x7fa44c06aba0 msgr2=0x7fa44c0b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44a7fc640 1 --2- 172.21.15.16:0/1975490288 >> v2:172.21.15.186:3300/0 conn(0x7fa44c06aba0 0x7fa44c0b1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44a7fc640 1 -- 172.21.15.16:0/1975490288 >> v1:172.21.15.186:6789/0 conn(0x7fa44c15d450 legacy=0x7fa44c15d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44a7fc640 1 -- 172.21.15.16:0/1975490288 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c0b0660 con 0x7fa44c0ce3a0 2024-04-25T10:10:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44a7fc640 1 -- 172.21.15.16:0/1975490288 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fa440001670 con 0x7fa44c0ce3a0 2024-04-25T10:10:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44a7fc640 1 -- 172.21.15.16:0/1975490288 >> v1:172.21.15.16:6789/0 conn(0x7fa44c0ce3a0 legacy=0x7fa44c06af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44a7fc640 1 --2- 172.21.15.16:0/1975490288 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4340039b0 0x7fa434003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44a7fc640 1 --2- 172.21.15.16:0/1975490288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa434004800 0x7fa434004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44a7fc640 1 --2- 172.21.15.16:0/1975490288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4340051c0 0x7fa43400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44a7fc640 1 -- 172.21.15.16:0/1975490288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0b0130 con 0x7fa4340051c0 2024-04-25T10:10:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44a7fc640 1 -- 172.21.15.16:0/1975490288 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43400c090 con 0x7fa434004800 2024-04-25T10:10:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44a7fc640 1 -- 172.21.15.16:0/1975490288 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43400c290 con 0x7fa4340039b0 2024-04-25T10:10:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44bfff640 1 --2- 172.21.15.16:0/1975490288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4340051c0 0x7fa43400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44b7fe640 1 --2- 172.21.15.16:0/1975490288 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4340039b0 0x7fa434003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:44.999+0000 7fa44affd640 1 --2- 172.21.15.16:0/1975490288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa434004800 0x7fa434004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.000+0000 7fa44bfff640 1 -- 172.21.15.16:0/1975490288 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4340039b0 msgr2=0x7fa434003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:45.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.000+0000 7fa44bfff640 1 --2- 172.21.15.16:0/1975490288 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4340039b0 0x7fa434003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.000+0000 7fa44bfff640 1 -- 172.21.15.16:0/1975490288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa434004800 msgr2=0x7fa434004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:45.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.000+0000 7fa44bfff640 1 --2- 172.21.15.16:0/1975490288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa434004800 0x7fa434004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.000+0000 7fa44bfff640 1 -- 172.21.15.16:0/1975490288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa434003520 con 0x7fa4340051c0 2024-04-25T10:10:45.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.000+0000 7fa44bfff640 1 -- 172.21.15.16:0/1975490288 reap_dead start 2024-04-25T10:10:45.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.000+0000 7fa44bfff640 1 --2- 172.21.15.16:0/1975490288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4340051c0 0x7fa43400b9c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fa444001e50 tx=0x7fa434005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82fe0793db50fd0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:45.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.000+0000 7fa44a7fc640 1 -- 172.21.15.16:0/1975490288 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444001a70 con 0x7fa4340051c0 2024-04-25T10:10:45.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.001+0000 7fa44a7fc640 1 -- 172.21.15.16:0/1975490288 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa444003970 con 0x7fa4340051c0 2024-04-25T10:10:45.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.001+0000 7fa44a7fc640 1 -- 172.21.15.16:0/1975490288 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444003d60 con 0x7fa4340051c0 2024-04-25T10:10:45.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.001+0000 7fa451fe7640 1 -- 172.21.15.16:0/1975490288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4340051c0 msgr2=0x7fa43400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.001+0000 7fa451fe7640 1 --2- 172.21.15.16:0/1975490288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4340051c0 0x7fa43400b9c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fa444001e50 tx=0x7fa434005830 comp rx=0 tx=0).stop 2024-04-25T10:10:45.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.001+0000 7fa451fe7640 1 -- 172.21.15.16:0/1975490288 shutdown_connections 2024-04-25T10:10:45.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.001+0000 7fa451fe7640 1 --2- 172.21.15.16:0/1975490288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4340051c0 0x7fa43400b9c0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.001+0000 7fa451fe7640 1 -- 172.21.15.16:0/1975490288 >> 172.21.15.16:0/1975490288 conn(0x7fa44c0d07d0 msgr2=0x7fa44c0d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:45.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.001+0000 7fa451fe7640 1 -- 172.21.15.16:0/1975490288 shutdown_connections 2024-04-25T10:10:45.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.001+0000 7fa451fe7640 1 -- 172.21.15.16:0/1975490288 wait complete. 2024-04-25T10:10:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.002+0000 7fa451fe7640 1 Processor -- start 2024-04-25T10:10:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.003+0000 7fa451fe7640 1 -- start start 2024-04-25T10:10:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.003+0000 7fa451fe7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4340051c0 0x7fa44c0b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.003+0000 7fa451fe7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa44c152030 0x7fa44c15a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.003+0000 7fa451fe7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0b5c10 0x7fa44c06e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.003+0000 7fa451fe7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0b0e60 con 0x7fa44c0b5c10 2024-04-25T10:10:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.003+0000 7fa451fe7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0b09d0 con 0x7fa4340051c0 2024-04-25T10:10:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.003+0000 7fa451fe7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0b0fe0 con 0x7fa44c152030 2024-04-25T10:10:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.003+0000 7fa44bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0b5c10 0x7fa44c06e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.003+0000 7fa44bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0b5c10 0x7fa44c06e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57450/0 (socket says 172.21.15.16:57450) 2024-04-25T10:10:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.003+0000 7fa44bfff640 1 -- 172.21.15.16:0/3869322435 learned_addr learned my addr 172.21.15.16:0/3869322435 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.004+0000 7fa44affd640 1 --2- 172.21.15.16:0/3869322435 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa44c152030 0x7fa44c15a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.004+0000 7fa44b7fe640 1 --2- 172.21.15.16:0/3869322435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4340051c0 0x7fa44c0b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.004+0000 7fa44bfff640 1 -- 172.21.15.16:0/3869322435 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa44c152030 msgr2=0x7fa44c15a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.004+0000 7fa44bfff640 1 --2- 172.21.15.16:0/3869322435 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa44c152030 0x7fa44c15a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.004+0000 7fa44bfff640 1 -- 172.21.15.16:0/3869322435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4340051c0 msgr2=0x7fa44c0b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.004+0000 7fa44bfff640 1 --2- 172.21.15.16:0/3869322435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4340051c0 0x7fa44c0b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.004+0000 7fa44bfff640 1 -- 172.21.15.16:0/3869322435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c0b4b70 con 0x7fa44c0b5c10 2024-04-25T10:10:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.004+0000 7fa44b7fe640 1 --2- 172.21.15.16:0/3869322435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4340051c0 0x7fa44c0b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.004+0000 7fa44bfff640 1 --2- 172.21.15.16:0/3869322435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0b5c10 0x7fa44c06e970 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa444001f70 tx=0x7fa4440014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.004+0000 7fa433fff640 1 -- 172.21.15.16:0/3869322435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444007db0 con 0x7fa44c0b5c10 2024-04-25T10:10:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.004+0000 7fa433fff640 1 -- 172.21.15.16:0/3869322435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa444006b50 con 0x7fa44c0b5c10 2024-04-25T10:10:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.004+0000 7fa451fe7640 1 -- 172.21.15.16:0/3869322435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44c0b4e50 con 0x7fa44c0b5c10 2024-04-25T10:10:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.004+0000 7fa433fff640 1 -- 172.21.15.16:0/3869322435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444001a70 con 0x7fa44c0b5c10 2024-04-25T10:10:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.004+0000 7fa451fe7640 1 -- 172.21.15.16:0/3869322435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44c0b5410 con 0x7fa44c0b5c10 2024-04-25T10:10:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.005+0000 7fa451fe7640 1 -- 172.21.15.16:0/3869322435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44c0750b0 con 0x7fa44c0b5c10 2024-04-25T10:10:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.006+0000 7fa433fff640 1 -- 172.21.15.16:0/3869322435 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa444007030 con 0x7fa44c0b5c10 2024-04-25T10:10:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.006+0000 7fa433fff640 1 --2- 172.21.15.16:0/3869322435 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa43806b500 0x7fa43806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.006+0000 7fa433fff640 1 -- 172.21.15.16:0/3869322435 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fa4440a9c20 con 0x7fa44c0b5c10 2024-04-25T10:10:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.006+0000 7fa44b7fe640 1 --2- 172.21.15.16:0/3869322435 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa43806b500 0x7fa43806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.007+0000 7fa44b7fe640 1 --2- 172.21.15.16:0/3869322435 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa43806b500 0x7fa43806d9c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0cf140 tx=0x7fa43c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:45.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.010+0000 7fa433fff640 1 -- 172.21.15.16:0/3869322435 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa444043060 con 0x7fa44c0b5c10 2024-04-25T10:10:45.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.188+0000 7fa451fe7640 1 -- 172.21.15.16:0/3869322435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa44c0d0430 con 0x7fa44c0b5c10 2024-04-25T10:10:45.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.189+0000 7fa433fff640 1 -- 172.21.15.16:0/3869322435 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa4440789f0 con 0x7fa44c0b5c10 2024-04-25T10:10:45.221 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:10:45.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.191+0000 7fa451fe7640 1 -- 172.21.15.16:0/3869322435 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa43806b500 msgr2=0x7fa43806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.191+0000 7fa451fe7640 1 --2- 172.21.15.16:0/3869322435 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa43806b500 0x7fa43806d9c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0cf140 tx=0x7fa43c008040 comp rx=0 tx=0).stop 2024-04-25T10:10:45.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.191+0000 7fa451fe7640 1 -- 172.21.15.16:0/3869322435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0b5c10 msgr2=0x7fa44c06e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.191+0000 7fa451fe7640 1 --2- 172.21.15.16:0/3869322435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0b5c10 0x7fa44c06e970 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa444001f70 tx=0x7fa4440014a0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.191+0000 7fa451fe7640 1 -- 172.21.15.16:0/3869322435 shutdown_connections 2024-04-25T10:10:45.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.191+0000 7fa451fe7640 1 --2- 172.21.15.16:0/3869322435 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa43806b500 0x7fa43806d9c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.191+0000 7fa451fe7640 1 --2- 172.21.15.16:0/3869322435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0b5c10 0x7fa44c06e970 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.191+0000 7fa451fe7640 1 --2- 172.21.15.16:0/3869322435 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa44c152030 0x7fa44c15a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.191+0000 7fa451fe7640 1 --2- 172.21.15.16:0/3869322435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4340051c0 0x7fa44c0b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.191+0000 7fa451fe7640 1 -- 172.21.15.16:0/3869322435 >> 172.21.15.16:0/3869322435 conn(0x7fa44c0d07d0 msgr2=0x7fa44c0ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:45.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.191+0000 7fa451fe7640 1 -- 172.21.15.16:0/3869322435 shutdown_connections 2024-04-25T10:10:45.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.191+0000 7fa451fe7640 1 -- 172.21.15.16:0/3869322435 wait complete. 2024-04-25T10:10:45.223 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:10:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.319+0000 7f5a2f24d640 1 Processor -- start 2024-04-25T10:10:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.319+0000 7f5a2f24d640 1 -- start start 2024-04-25T10:10:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.320+0000 7f5a2f24d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a280b0bd0 0x7f5a280d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.320+0000 7f5a2f24d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a2815f660 con 0x7f5a2815ef60 2024-04-25T10:10:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.320+0000 7f5a2f24d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a2815f940 con 0x7f5a280b0740 2024-04-25T10:10:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.320+0000 7f5a2f24d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a2815fc20 con 0x7f5a280b0bd0 2024-04-25T10:10:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.320+0000 7f5a2d7ea640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a280b0bd0 0x7f5a280d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.320+0000 7f5a2cfe9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5a2815ef60 0x7f5a280ce4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56710/0 (socket says 172.21.15.16:56710) 2024-04-25T10:10:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.320+0000 7f5a2cfe9640 1 -- 172.21.15.16:0/1707691117 learned_addr learned my addr 172.21.15.16:0/1707691117 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.321+0000 7f5a2d7ea640 1 -- 172.21.15.16:0/1707691117 >> v1:172.21.15.186:6789/0 conn(0x7f5a280b0740 legacy=0x7f5a280cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:45.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.321+0000 7f5a2d7ea640 1 -- 172.21.15.16:0/1707691117 >> v1:172.21.15.16:6789/0 conn(0x7f5a2815ef60 legacy=0x7f5a280ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.321+0000 7f5a2d7ea640 1 -- 172.21.15.16:0/1707691117 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2815ff00 con 0x7f5a280b0bd0 2024-04-25T10:10:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.321+0000 7f5a2d7ea640 1 --2- 172.21.15.16:0/1707691117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a280b0bd0 0x7f5a280d0470 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f5a180097a0 tx=0x7f5a1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6646c07bc4f4875a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.322+0000 7f5a277fe640 1 -- 172.21.15.16:0/1707691117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a18037070 con 0x7f5a280b0bd0 2024-04-25T10:10:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.322+0000 7f5a277fe640 1 -- 172.21.15.16:0/1707691117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a18032440 con 0x7f5a280b0bd0 2024-04-25T10:10:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.322+0000 7f5a277fe640 1 -- 172.21.15.16:0/1707691117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a18032810 con 0x7f5a280b0bd0 2024-04-25T10:10:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.322+0000 7f5a2f24d640 1 -- 172.21.15.16:0/1707691117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a280b0bd0 msgr2=0x7f5a280d0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.322+0000 7f5a2f24d640 1 --2- 172.21.15.16:0/1707691117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a280b0bd0 0x7f5a280d0470 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f5a180097a0 tx=0x7f5a1802a270 comp rx=0 tx=0).stop 2024-04-25T10:10:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.322+0000 7f5a2f24d640 1 -- 172.21.15.16:0/1707691117 shutdown_connections 2024-04-25T10:10:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.322+0000 7f5a2f24d640 1 --2- 172.21.15.16:0/1707691117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a280b0bd0 0x7f5a280d0470 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.322+0000 7f5a2f24d640 1 -- 172.21.15.16:0/1707691117 >> 172.21.15.16:0/1707691117 conn(0x7f5a2806f4c0 msgr2=0x7f5a2806f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.322+0000 7f5a2f24d640 1 -- 172.21.15.16:0/1707691117 shutdown_connections 2024-04-25T10:10:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.322+0000 7f5a2f24d640 1 -- 172.21.15.16:0/1707691117 wait complete. 2024-04-25T10:10:45.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.323+0000 7f5a2f24d640 1 Processor -- start 2024-04-25T10:10:45.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.323+0000 7f5a2f24d640 1 -- start start 2024-04-25T10:10:45.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.324+0000 7f5a2f24d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a280b0740 0x7f5a2806bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.324+0000 7f5a2f24d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a280b0bd0 0x7f5a2815a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.324+0000 7f5a2f24d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a2815ef60 0x7f5a280c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.324+0000 7f5a2f24d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2815f340 con 0x7f5a280b0bd0 2024-04-25T10:10:45.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.324+0000 7f5a2f24d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a281604c0 con 0x7f5a280b0740 2024-04-25T10:10:45.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.324+0000 7f5a2f24d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a280b0fb0 con 0x7f5a2815ef60 2024-04-25T10:10:45.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.324+0000 7f5a27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a280b0bd0 0x7f5a2815a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.324+0000 7f5a27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a280b0bd0 0x7f5a2815a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57464/0 (socket says 172.21.15.16:57464) 2024-04-25T10:10:45.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.324+0000 7f5a27fff640 1 -- 172.21.15.16:0/293666781 learned_addr learned my addr 172.21.15.16:0/293666781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:45.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.325+0000 7f5a2d7ea640 1 --2- 172.21.15.16:0/293666781 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a2815ef60 0x7f5a280c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.325+0000 7f5a2cfe9640 1 --2- 172.21.15.16:0/293666781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a280b0740 0x7f5a2806bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.325+0000 7f5a27fff640 1 -- 172.21.15.16:0/293666781 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a2815ef60 msgr2=0x7f5a280c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.325+0000 7f5a27fff640 1 --2- 172.21.15.16:0/293666781 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a2815ef60 0x7f5a280c5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.325+0000 7f5a27fff640 1 -- 172.21.15.16:0/293666781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a280b0740 msgr2=0x7f5a2806bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.325+0000 7f5a27fff640 1 --2- 172.21.15.16:0/293666781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a280b0740 0x7f5a2806bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.325+0000 7f5a27fff640 1 -- 172.21.15.16:0/293666781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a280b24a0 con 0x7f5a280b0bd0 2024-04-25T10:10:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.325+0000 7f5a2cfe9640 1 --2- 172.21.15.16:0/293666781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a280b0740 0x7f5a2806bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.325+0000 7f5a27fff640 1 --2- 172.21.15.16:0/293666781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a280b0bd0 0x7f5a2815a230 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00c430 tx=0x7f5a1c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.325+0000 7f5a257fa640 1 -- 172.21.15.16:0/293666781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c011790 con 0x7f5a280b0bd0 2024-04-25T10:10:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.325+0000 7f5a257fa640 1 -- 172.21.15.16:0/293666781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1c017070 con 0x7f5a280b0bd0 2024-04-25T10:10:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.325+0000 7f5a2f24d640 1 -- 172.21.15.16:0/293666781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a280b2740 con 0x7f5a280b0bd0 2024-04-25T10:10:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.325+0000 7f5a257fa640 1 -- 172.21.15.16:0/293666781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c011b70 con 0x7f5a280b0bd0 2024-04-25T10:10:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.325+0000 7f5a2f24d640 1 -- 172.21.15.16:0/293666781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a280b2d00 con 0x7f5a280b0bd0 2024-04-25T10:10:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.326+0000 7f5a2f24d640 1 -- 172.21.15.16:0/293666781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a280750b0 con 0x7f5a280b0bd0 2024-04-25T10:10:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.327+0000 7f5a257fa640 1 -- 172.21.15.16:0/293666781 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a1c011d10 con 0x7f5a280b0bd0 2024-04-25T10:10:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.327+0000 7f5a257fa640 1 --2- 172.21.15.16:0/293666781 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5a1406b4d0 0x7f5a1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.327+0000 7f5a257fa640 1 -- 172.21.15.16:0/293666781 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f5a1c08a120 con 0x7f5a280b0bd0 2024-04-25T10:10:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.327+0000 7f5a2cfe9640 1 --2- 172.21.15.16:0/293666781 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5a1406b4d0 0x7f5a1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.328+0000 7f5a2cfe9640 1 --2- 172.21.15.16:0/293666781 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5a1406b4d0 0x7f5a1406d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5a200012e0 tx=0x7f5a20009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:45.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.331+0000 7f5a257fa640 1 -- 172.21.15.16:0/293666781 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a1c058ef0 con 0x7f5a280b0bd0 2024-04-25T10:10:45.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.523+0000 7f5a2f24d640 1 -- 172.21.15.16:0/293666781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5a2815d9a0 con 0x7f5a280b0bd0 2024-04-25T10:10:45.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.524+0000 7f5a257fa640 1 -- 172.21.15.16:0/293666781 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v75) v1 ==== 80+0+124 (secure 0 0 0) 0x7f5a1c0587f0 con 0x7f5a280b0bd0 2024-04-25T10:10:45.529 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.186:6837/3292077067 2024-04-26T10:09:31.374252+0000 2024-04-25T10:10:45.530 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.186:6836/3292077067 2024-04-26T10:09:31.374252+0000 2024-04-25T10:10:45.530 INFO:teuthology.orchestra.run.smithi016.stderr:listed 2 entries 2024-04-25T10:10:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.526+0000 7f5a2f24d640 1 -- 172.21.15.16:0/293666781 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5a1406b4d0 msgr2=0x7f5a1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.526+0000 7f5a2f24d640 1 --2- 172.21.15.16:0/293666781 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5a1406b4d0 0x7f5a1406d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5a200012e0 tx=0x7f5a20009040 comp rx=0 tx=0).stop 2024-04-25T10:10:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.526+0000 7f5a2f24d640 1 -- 172.21.15.16:0/293666781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a280b0bd0 msgr2=0x7f5a2815a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.526+0000 7f5a2f24d640 1 --2- 172.21.15.16:0/293666781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a280b0bd0 0x7f5a2815a230 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00c430 tx=0x7f5a1c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.526+0000 7f5a2f24d640 1 -- 172.21.15.16:0/293666781 shutdown_connections 2024-04-25T10:10:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.526+0000 7f5a2f24d640 1 --2- 172.21.15.16:0/293666781 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5a1406b4d0 0x7f5a1406d990 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.526+0000 7f5a2f24d640 1 --2- 172.21.15.16:0/293666781 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a2815ef60 0x7f5a280c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.526+0000 7f5a2f24d640 1 --2- 172.21.15.16:0/293666781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a280b0bd0 0x7f5a2815a230 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.526+0000 7f5a2f24d640 1 --2- 172.21.15.16:0/293666781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a280b0740 0x7f5a2806bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.526+0000 7f5a2f24d640 1 -- 172.21.15.16:0/293666781 >> 172.21.15.16:0/293666781 conn(0x7f5a2806f4c0 msgr2=0x7f5a280d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.526+0000 7f5a2f24d640 1 -- 172.21.15.16:0/293666781 shutdown_connections 2024-04-25T10:10:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.526+0000 7f5a2f24d640 1 -- 172.21.15.16:0/293666781 wait complete. 2024-04-25T10:10:45.542 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:10:45.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:45.542 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:45.570 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:10:45.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:45.570 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:10:45.626 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:10:45.626 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:45.645 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:10:45.645 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:10:45.645 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:45.681 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:10:45.682 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:10:45.682 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:45.705 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:10:45.705 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:45.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.817+0000 7f086db61640 1 Processor -- start 2024-04-25T10:10:45.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.817+0000 7f086db61640 1 -- start start 2024-04-25T10:10:45.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.818+0000 7f086db61640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f086815dce0 0x7f08680d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.818+0000 7f086db61640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08680d35b0 0x7f08680b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.818+0000 7f086db61640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f086815e0c0 con 0x7f08680d1f90 2024-04-25T10:10:45.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.818+0000 7f086db61640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08680b7b10 con 0x7f086815dce0 2024-04-25T10:10:45.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.818+0000 7f086db61640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08680d2510 con 0x7f08680d35b0 2024-04-25T10:10:45.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.818+0000 7f0867fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f08680d1f90 0x7f086806bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56720/0 (socket says 172.21.15.16:56720) 2024-04-25T10:10:45.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.818+0000 7f0867fff640 1 -- 172.21.15.16:0/2479793331 learned_addr learned my addr 172.21.15.16:0/2479793331 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:45.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08677fe640 1 --2- 172.21.15.16:0/2479793331 >> v2:172.21.15.186:3300/0 conn(0x7f086815dce0 0x7f08680d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f0866ffd640 1 --2- 172.21.15.16:0/2479793331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08680d35b0 0x7f08680b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08667fc640 1 -- 172.21.15.16:0/2479793331 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828326578 0 0) 0x7f086815e0c0 con 0x7f08680d1f90 2024-04-25T10:10:45.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08667fc640 1 -- 172.21.15.16:0/2479793331 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0850003520 con 0x7f08680d1f90 2024-04-25T10:10:45.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08667fc640 1 -- 172.21.15.16:0/2479793331 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 367428092 0 0) 0x7f0850003520 con 0x7f08680d1f90 2024-04-25T10:10:45.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08667fc640 1 -- 172.21.15.16:0/2479793331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08680d35b0 msgr2=0x7f08680b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:45.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08667fc640 1 --2- 172.21.15.16:0/2479793331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08680d35b0 0x7f08680b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08667fc640 1 -- 172.21.15.16:0/2479793331 >> v2:172.21.15.186:3300/0 conn(0x7f086815dce0 msgr2=0x7f08680d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:45.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08667fc640 1 --2- 172.21.15.16:0/2479793331 >> v2:172.21.15.186:3300/0 conn(0x7f086815dce0 0x7f08680d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08667fc640 1 -- 172.21.15.16:0/2479793331 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08680d27f0 con 0x7f08680d1f90 2024-04-25T10:10:45.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08667fc640 1 -- 172.21.15.16:0/2479793331 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f08600019f0 con 0x7f08680d1f90 2024-04-25T10:10:45.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08677fe640 1 --2- 172.21.15.16:0/2479793331 >> v2:172.21.15.186:3300/0 conn(0x7f086815dce0 0x7f08680d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:10:45.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08667fc640 1 -- 172.21.15.16:0/2479793331 >> v1:172.21.15.16:6789/0 conn(0x7f08680d1f90 legacy=0x7f086806bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08667fc640 1 --2- 172.21.15.16:0/2479793331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08500039b0 0x7f0850003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08667fc640 1 --2- 172.21.15.16:0/2479793331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08680d35b0 0x7f0850004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08667fc640 1 --2- 172.21.15.16:0/2479793331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0850004d40 0x7f08500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08667fc640 1 -- 172.21.15.16:0/2479793331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680b7b10 con 0x7f0850004d40 2024-04-25T10:10:45.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08667fc640 1 -- 172.21.15.16:0/2479793331 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680d2510 con 0x7f08500039b0 2024-04-25T10:10:45.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08667fc640 1 -- 172.21.15.16:0/2479793331 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085000b9c0 con 0x7f08680d35b0 2024-04-25T10:10:45.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f0867fff640 1 --2- 172.21.15.16:0/2479793331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0850004d40 0x7f08500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f0866ffd640 1 --2- 172.21.15.16:0/2479793331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08680d35b0 0x7f0850004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.819+0000 7f08677fe640 1 --2- 172.21.15.16:0/2479793331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08500039b0 0x7f0850003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.820+0000 7f0867fff640 1 -- 172.21.15.16:0/2479793331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08680d35b0 msgr2=0x7f0850004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:45.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.820+0000 7f0867fff640 1 --2- 172.21.15.16:0/2479793331 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08680d35b0 0x7f0850004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.820+0000 7f0867fff640 1 -- 172.21.15.16:0/2479793331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08500039b0 msgr2=0x7f0850003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:45.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.820+0000 7f0867fff640 1 --2- 172.21.15.16:0/2479793331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08500039b0 0x7f0850003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.820+0000 7f0867fff640 1 -- 172.21.15.16:0/2479793331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0850003520 con 0x7f0850004d40 2024-04-25T10:10:45.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.820+0000 7f0867fff640 1 --2- 172.21.15.16:0/2479793331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0850004d40 0x7f08500051c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f086000a870 tx=0x7f0850005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc38e418c52e2c9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:45.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.820+0000 7f08667fc640 1 -- 172.21.15.16:0/2479793331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086000a4c0 con 0x7f0850004d40 2024-04-25T10:10:45.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.820+0000 7f08667fc640 1 -- 172.21.15.16:0/2479793331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086002f070 con 0x7f0850004d40 2024-04-25T10:10:45.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.820+0000 7f08667fc640 1 -- 172.21.15.16:0/2479793331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086002bcc0 con 0x7f0850004d40 2024-04-25T10:10:45.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.821+0000 7f086db61640 1 -- 172.21.15.16:0/2479793331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0850004d40 msgr2=0x7f08500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.821+0000 7f086db61640 1 --2- 172.21.15.16:0/2479793331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0850004d40 0x7f08500051c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f086000a870 tx=0x7f0850005830 comp rx=0 tx=0).stop 2024-04-25T10:10:45.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.821+0000 7f0867fff640 1 -- 172.21.15.16:0/2479793331 reap_dead start 2024-04-25T10:10:45.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.821+0000 7f086db61640 1 -- 172.21.15.16:0/2479793331 shutdown_connections 2024-04-25T10:10:45.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.821+0000 7f086db61640 1 -- 172.21.15.16:0/2479793331 >> 172.21.15.16:0/2479793331 conn(0x7f0868072900 msgr2=0x7f08680d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:45.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.821+0000 7f086db61640 1 -- 172.21.15.16:0/2479793331 shutdown_connections 2024-04-25T10:10:45.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.821+0000 7f086db61640 1 -- 172.21.15.16:0/2479793331 wait complete. 2024-04-25T10:10:45.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.822+0000 7f086db61640 1 Processor -- start 2024-04-25T10:10:45.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.822+0000 7f086db61640 1 -- start start 2024-04-25T10:10:45.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.823+0000 7f086db61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0868160790 0x7f086815e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.823+0000 7f086db61640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08680cbaf0 0x7f086815e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.823+0000 7f086db61640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08680cd130 0x7f0868160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.823+0000 7f086db61640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680d3090 con 0x7f0868160790 2024-04-25T10:10:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.823+0000 7f086db61640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680d2db0 con 0x7f08680cbaf0 2024-04-25T10:10:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.823+0000 7f086db61640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08680b9820 con 0x7f08680cd130 2024-04-25T10:10:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.823+0000 7f08677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0868160790 0x7f086815e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.823+0000 7f08677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0868160790 0x7f086815e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57476/0 (socket says 172.21.15.16:57476) 2024-04-25T10:10:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.823+0000 7f08677fe640 1 -- 172.21.15.16:0/721853083 learned_addr learned my addr 172.21.15.16:0/721853083 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.823+0000 7f0866ffd640 1 --2- 172.21.15.16:0/721853083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08680cbaf0 0x7f086815e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.823+0000 7f0867fff640 1 --2- 172.21.15.16:0/721853083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08680cd130 0x7f0868160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.823+0000 7f08677fe640 1 -- 172.21.15.16:0/721853083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08680cd130 msgr2=0x7f0868160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.823+0000 7f08677fe640 1 --2- 172.21.15.16:0/721853083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08680cd130 0x7f0868160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.823+0000 7f08677fe640 1 -- 172.21.15.16:0/721853083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08680cbaf0 msgr2=0x7f086815e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.823+0000 7f08677fe640 1 --2- 172.21.15.16:0/721853083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08680cbaf0 0x7f086815e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.824+0000 7f08677fe640 1 -- 172.21.15.16:0/721853083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08681f3940 con 0x7f0868160790 2024-04-25T10:10:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.824+0000 7f0866ffd640 1 --2- 172.21.15.16:0/721853083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08680cbaf0 0x7f086815e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.824+0000 7f0867fff640 1 --2- 172.21.15.16:0/721853083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08680cd130 0x7f0868160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.824+0000 7f08677fe640 1 --2- 172.21.15.16:0/721853083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0868160790 0x7f086815e2b0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0858011d00 tx=0x7f085800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.824+0000 7f084ffff640 1 -- 172.21.15.16:0/721853083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085800fd20 con 0x7f0868160790 2024-04-25T10:10:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.824+0000 7f084ffff640 1 -- 172.21.15.16:0/721853083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0858004030 con 0x7f0868160790 2024-04-25T10:10:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.824+0000 7f086db61640 1 -- 172.21.15.16:0/721853083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08680cc090 con 0x7f0868160790 2024-04-25T10:10:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.824+0000 7f084ffff640 1 -- 172.21.15.16:0/721853083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858004420 con 0x7f0868160790 2024-04-25T10:10:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.824+0000 7f086db61640 1 -- 172.21.15.16:0/721853083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08680cc650 con 0x7f0868160790 2024-04-25T10:10:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.825+0000 7f086db61640 1 -- 172.21.15.16:0/721853083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08680749b0 con 0x7f0868160790 2024-04-25T10:10:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.825+0000 7f084ffff640 1 -- 172.21.15.16:0/721853083 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0858012040 con 0x7f0868160790 2024-04-25T10:10:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.826+0000 7f084ffff640 1 --2- 172.21.15.16:0/721853083 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f085406b4d0 0x7f085406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.826+0000 7f084ffff640 1 -- 172.21.15.16:0/721853083 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f085808a890 con 0x7f0868160790 2024-04-25T10:10:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.826+0000 7f0866ffd640 1 --2- 172.21.15.16:0/721853083 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f085406b4d0 0x7f085406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.827+0000 7f0866ffd640 1 --2- 172.21.15.16:0/721853083 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f085406b4d0 0x7f085406d990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f08681f6f90 tx=0x7f085c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:45.830+0000 7f084ffff640 1 -- 172.21.15.16:0/721853083 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0858019070 con 0x7f0868160790 2024-04-25T10:10:46.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.046+0000 7f086db61640 1 -- 172.21.15.16:0/721853083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08680ceaf0 con 0x7f0868160790 2024-04-25T10:10:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.047+0000 7f084ffff640 1 -- 172.21.15.16:0/721853083 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f08580595e0 con 0x7f0868160790 2024-04-25T10:10:46.053 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:46.053 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6070,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2805428272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2805428272},{"type":"v1","addr":"172.21.15.186:6835","nonce":2805428272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1052667029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1052667029},{"type":"v1","addr":"172.21.15.186:6837","nonce":1052667029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2897154125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2897154125},{"type":"v1","addr":"172.21.15.16:6833","nonce":2897154125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:22.077032+0000","modified":"2024-04-25T10:10:14.390498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6636":{"gid":6636,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":17,"addr":"172.21.15.16:6835/1735811292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1735811292},{"type":"v1","addr":"172.21.15.16:6835","nonce":1735811292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:10:46.055 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 44 2024-04-25T10:10:46.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.049+0000 7f086db61640 1 -- 172.21.15.16:0/721853083 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f085406b4d0 msgr2=0x7f085406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.049+0000 7f086db61640 1 --2- 172.21.15.16:0/721853083 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f085406b4d0 0x7f085406d990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f08681f6f90 tx=0x7f085c008040 comp rx=0 tx=0).stop 2024-04-25T10:10:46.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.049+0000 7f086db61640 1 -- 172.21.15.16:0/721853083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0868160790 msgr2=0x7f086815e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.049+0000 7f086db61640 1 --2- 172.21.15.16:0/721853083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0868160790 0x7f086815e2b0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0858011d00 tx=0x7f085800f610 comp rx=0 tx=0).stop 2024-04-25T10:10:46.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.049+0000 7f086db61640 1 -- 172.21.15.16:0/721853083 shutdown_connections 2024-04-25T10:10:46.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.049+0000 7f086db61640 1 --2- 172.21.15.16:0/721853083 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f085406b4d0 0x7f085406d990 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.049+0000 7f086db61640 1 --2- 172.21.15.16:0/721853083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08680cd130 0x7f0868160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.049+0000 7f086db61640 1 --2- 172.21.15.16:0/721853083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08680cbaf0 0x7f086815e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.049+0000 7f086db61640 1 --2- 172.21.15.16:0/721853083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0868160790 0x7f086815e2b0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.049+0000 7f086db61640 1 -- 172.21.15.16:0/721853083 >> 172.21.15.16:0/721853083 conn(0x7f0868072900 msgr2=0x7f08680b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.049+0000 7f086db61640 1 -- 172.21.15.16:0/721853083 shutdown_connections 2024-04-25T10:10:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.049+0000 7f086db61640 1 -- 172.21.15.16:0/721853083 wait complete. 2024-04-25T10:10:46.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.162+0000 7fb9ed42a640 1 Processor -- start 2024-04-25T10:10:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.162+0000 7fb9ed42a640 1 -- start start 2024-04-25T10:10:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.162+0000 7fb9ed42a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9e815dd50 0x7fb9e80d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.162+0000 7fb9ed42a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e80d2440 con 0x7fb9e80ceeb0 2024-04-25T10:10:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.162+0000 7fb9ed42a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e80d2720 con 0x7fb9e80b9460 2024-04-25T10:10:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.162+0000 7fb9ed42a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80d2a00 con 0x7fb9e815dd50 2024-04-25T10:10:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.163+0000 7fb9e77fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb9e80ceeb0 0x7fb9e80b6c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56726/0 (socket says 172.21.15.16:56726) 2024-04-25T10:10:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.163+0000 7fb9e77fe640 1 -- 172.21.15.16:0/3786610905 learned_addr learned my addr 172.21.15.16:0/3786610905 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.163+0000 7fb9e6ffd640 1 --2- 172.21.15.16:0/3786610905 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9e815dd50 0x7fb9e80d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.163+0000 7fb9e67fc640 1 -- 172.21.15.16:0/3786610905 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538653010 0 0) 0x7fb9e80d2440 con 0x7fb9e80ceeb0 2024-04-25T10:10:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.163+0000 7fb9e67fc640 1 -- 172.21.15.16:0/3786610905 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9d4003520 con 0x7fb9e80ceeb0 2024-04-25T10:10:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.163+0000 7fb9e67fc640 1 -- 172.21.15.16:0/3786610905 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819364101 0 0) 0x7fb9e80d2720 con 0x7fb9e80b9460 2024-04-25T10:10:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.163+0000 7fb9e67fc640 1 -- 172.21.15.16:0/3786610905 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9e80d2440 con 0x7fb9e80b9460 2024-04-25T10:10:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.163+0000 7fb9e6ffd640 1 -- 172.21.15.16:0/3786610905 >> v1:172.21.15.186:6789/0 conn(0x7fb9e80b9460 legacy=0x7fb9e80ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.163+0000 7fb9e6ffd640 1 -- 172.21.15.16:0/3786610905 >> v1:172.21.15.16:6789/0 conn(0x7fb9e80ceeb0 legacy=0x7fb9e80b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.163+0000 7fb9e67fc640 1 -- 172.21.15.16:0/3786610905 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3032360414 0 0) 0x7fb9d4003520 con 0x7fb9e80ceeb0 2024-04-25T10:10:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.163+0000 7fb9e6ffd640 1 -- 172.21.15.16:0/3786610905 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e80d2ce0 con 0x7fb9e815dd50 2024-04-25T10:10:46.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.164+0000 7fb9e6ffd640 1 --2- 172.21.15.16:0/3786610905 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9e815dd50 0x7fb9e80d1c10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80097d0 tx=0x7fb9d802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75d23971222b2ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:46.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.164+0000 7fb9e67fc640 1 -- 172.21.15.16:0/3786610905 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8037070 con 0x7fb9e815dd50 2024-04-25T10:10:46.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.164+0000 7fb9e67fc640 1 -- 172.21.15.16:0/3786610905 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d802ab90 con 0x7fb9e815dd50 2024-04-25T10:10:46.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.164+0000 7fb9e67fc640 1 -- 172.21.15.16:0/3786610905 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e00019f0 con 0x7fb9e815dd50 2024-04-25T10:10:46.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.165+0000 7fb9ed42a640 1 -- 172.21.15.16:0/3786610905 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9e815dd50 msgr2=0x7fb9e80d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.165+0000 7fb9ed42a640 1 --2- 172.21.15.16:0/3786610905 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9e815dd50 0x7fb9e80d1c10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80097d0 tx=0x7fb9d802a100 comp rx=0 tx=0).stop 2024-04-25T10:10:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.165+0000 7fb9ed42a640 1 -- 172.21.15.16:0/3786610905 shutdown_connections 2024-04-25T10:10:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.165+0000 7fb9ed42a640 1 --2- 172.21.15.16:0/3786610905 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9e815dd50 0x7fb9e80d1c10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.165+0000 7fb9ed42a640 1 -- 172.21.15.16:0/3786610905 >> 172.21.15.16:0/3786610905 conn(0x7fb9e80722d0 msgr2=0x7fb9e80726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.165+0000 7fb9ed42a640 1 -- 172.21.15.16:0/3786610905 shutdown_connections 2024-04-25T10:10:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.165+0000 7fb9ed42a640 1 -- 172.21.15.16:0/3786610905 wait complete. 2024-04-25T10:10:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.166+0000 7fb9ed42a640 1 Processor -- start 2024-04-25T10:10:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.166+0000 7fb9ed42a640 1 -- start start 2024-04-25T10:10:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.167+0000 7fb9ed42a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e80b9460 0x7fb9e80b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.167+0000 7fb9ed42a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9e80ceeb0 0x7fb9e81f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.167+0000 7fb9ed42a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9e815dd50 0x7fb9e8150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.167+0000 7fb9ed42a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80d34e0 con 0x7fb9e80b9460 2024-04-25T10:10:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.167+0000 7fb9ed42a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80d2150 con 0x7fb9e80ceeb0 2024-04-25T10:10:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.167+0000 7fb9ed42a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80d3660 con 0x7fb9e815dd50 2024-04-25T10:10:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.167+0000 7fb9e6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e80b9460 0x7fb9e80b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.167+0000 7fb9e6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e80b9460 0x7fb9e80b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57480/0 (socket says 172.21.15.16:57480) 2024-04-25T10:10:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.167+0000 7fb9e6ffd640 1 -- 172.21.15.16:0/1359218398 learned_addr learned my addr 172.21.15.16:0/1359218398 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.167+0000 7fb9dffff640 1 --2- 172.21.15.16:0/1359218398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9e80ceeb0 0x7fb9e81f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.167+0000 7fb9e6ffd640 1 -- 172.21.15.16:0/1359218398 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9e815dd50 msgr2=0x7fb9e8150da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.167+0000 7fb9e6ffd640 1 --2- 172.21.15.16:0/1359218398 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9e815dd50 0x7fb9e8150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.167+0000 7fb9e6ffd640 1 -- 172.21.15.16:0/1359218398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9e80ceeb0 msgr2=0x7fb9e81f47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.167+0000 7fb9e6ffd640 1 --2- 172.21.15.16:0/1359218398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9e80ceeb0 0x7fb9e81f47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.167+0000 7fb9e6ffd640 1 -- 172.21.15.16:0/1359218398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e81f3610 con 0x7fb9e80b9460 2024-04-25T10:10:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.167+0000 7fb9dffff640 1 --2- 172.21.15.16:0/1359218398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9e80ceeb0 0x7fb9e81f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.167+0000 7fb9e6ffd640 1 --2- 172.21.15.16:0/1359218398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e80b9460 0x7fb9e80b5d80 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80097d0 tx=0x7fb9d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.168+0000 7fb9df7fe640 1 -- 172.21.15.16:0/1359218398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8037050 con 0x7fb9e80b9460 2024-04-25T10:10:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.168+0000 7fb9df7fe640 1 -- 172.21.15.16:0/1359218398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d8038040 con 0x7fb9e80b9460 2024-04-25T10:10:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.168+0000 7fb9ed42a640 1 -- 172.21.15.16:0/1359218398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e80b5950 con 0x7fb9e80b9460 2024-04-25T10:10:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.168+0000 7fb9df7fe640 1 -- 172.21.15.16:0/1359218398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8006e10 con 0x7fb9e80b9460 2024-04-25T10:10:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.168+0000 7fb9ed42a640 1 -- 172.21.15.16:0/1359218398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e815e910 con 0x7fb9e80b9460 2024-04-25T10:10:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.169+0000 7fb9ed42a640 1 -- 172.21.15.16:0/1359218398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e8067850 con 0x7fb9e80b9460 2024-04-25T10:10:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.169+0000 7fb9df7fe640 1 -- 172.21.15.16:0/1359218398 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb9d8045020 con 0x7fb9e80b9460 2024-04-25T10:10:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.170+0000 7fb9df7fe640 1 --2- 172.21.15.16:0/1359218398 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb9c806b4d0 0x7fb9c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.170+0000 7fb9df7fe640 1 -- 172.21.15.16:0/1359218398 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fb9d80aa2e0 con 0x7fb9e80b9460 2024-04-25T10:10:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.170+0000 7fb9dffff640 1 --2- 172.21.15.16:0/1359218398 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb9c806b4d0 0x7fb9c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.170+0000 7fb9dffff640 1 --2- 172.21.15.16:0/1359218398 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb9c806b4d0 0x7fb9c806d990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb9d00011e0 tx=0x7fb9d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.174+0000 7fb9df7fe640 1 -- 172.21.15.16:0/1359218398 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb9d804c020 con 0x7fb9e80b9460 2024-04-25T10:10:46.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.356+0000 7fb9ed42a640 1 -- 172.21.15.16:0/1359218398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9e815ae80 con 0x7fb9e80b9460 2024-04-25T10:10:46.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.357+0000 7fb9df7fe640 1 -- 172.21.15.16:0/1359218398 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16885 (secure 0 0 0) 0x7fb9e81f3610 con 0x7fb9e80b9460 2024-04-25T10:10:46.363 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:46.364 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:09:53.479005+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:20.361009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:21.048892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.19","mappings":[{"from":2,"to":7}]},{"pgid":"11.26","mappings":[{"from":1,"to":4}]},{"pgid":"11.27","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.186:6837/3292077067":"2024-04-26T10:09:31.374252+0000","172.21.15.186:6836/3292077067":"2024-04-26T10:09:31.374252+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.360+0000 7fb9ed42a640 1 -- 172.21.15.16:0/1359218398 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb9c806b4d0 msgr2=0x7fb9c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.360+0000 7fb9ed42a640 1 --2- 172.21.15.16:0/1359218398 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb9c806b4d0 0x7fb9c806d990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb9d00011e0 tx=0x7fb9d0009040 comp rx=0 tx=0).stop 2024-04-25T10:10:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.360+0000 7fb9ed42a640 1 -- 172.21.15.16:0/1359218398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e80b9460 msgr2=0x7fb9e80b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.360+0000 7fb9ed42a640 1 --2- 172.21.15.16:0/1359218398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e80b9460 0x7fb9e80b5d80 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80097d0 tx=0x7fb9d80046c0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.360+0000 7fb9ed42a640 1 -- 172.21.15.16:0/1359218398 shutdown_connections 2024-04-25T10:10:46.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.360+0000 7fb9ed42a640 1 --2- 172.21.15.16:0/1359218398 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb9c806b4d0 0x7fb9c806d990 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.360+0000 7fb9ed42a640 1 --2- 172.21.15.16:0/1359218398 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9e815dd50 0x7fb9e8150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.360+0000 7fb9ed42a640 1 --2- 172.21.15.16:0/1359218398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9e80ceeb0 0x7fb9e81f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.360+0000 7fb9ed42a640 1 --2- 172.21.15.16:0/1359218398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e80b9460 0x7fb9e80b5d80 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.360+0000 7fb9ed42a640 1 -- 172.21.15.16:0/1359218398 >> 172.21.15.16:0/1359218398 conn(0x7fb9e80722d0 msgr2=0x7fb9e80ce450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:46.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.360+0000 7fb9ed42a640 1 -- 172.21.15.16:0/1359218398 shutdown_connections 2024-04-25T10:10:46.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.360+0000 7fb9ed42a640 1 -- 172.21.15.16:0/1359218398 wait complete. 2024-04-25T10:10:46.379 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:09:20.361009+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:09:21.048892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:10:46.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:10:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.478+0000 7f833c5f5640 1 Processor -- start 2024-04-25T10:10:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.478+0000 7f833c5f5640 1 -- start start 2024-04-25T10:10:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.479+0000 7f833c5f5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f833415b7b0 0x7f83340ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.479+0000 7f833c5f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f833415e820 0x7f83340cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.479+0000 7f833c5f5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f833415f000 con 0x7f83340cf760 2024-04-25T10:10:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.479+0000 7f833c5f5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83340ced00 con 0x7f833415b7b0 2024-04-25T10:10:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.479+0000 7f833c5f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83340ce330 con 0x7f833415e820 2024-04-25T10:10:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.479+0000 7f833a391640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f833415b7b0 0x7f83340ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.479+0000 7f833a391640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f833415b7b0 0x7f83340ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57482/0 (socket says 172.21.15.16:57482) 2024-04-25T10:10:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.479+0000 7f833a391640 1 -- 172.21.15.16:0/302425707 learned_addr learned my addr 172.21.15.16:0/302425707 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.479+0000 7f833ab92640 1 --2- 172.21.15.16:0/302425707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f833415e820 0x7f83340cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.479+0000 7f833a391640 1 -- 172.21.15.16:0/302425707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f833415e820 msgr2=0x7f83340cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.479+0000 7f833a391640 1 --2- 172.21.15.16:0/302425707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f833415e820 0x7f83340cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.479+0000 7f833a391640 1 -- 172.21.15.16:0/302425707 >> v1:172.21.15.186:6789/0 conn(0x7f83340cf760 legacy=0x7f833415e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.479+0000 7f833a391640 1 -- 172.21.15.16:0/302425707 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833415ed20 con 0x7f833415b7b0 2024-04-25T10:10:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.479+0000 7f833ab92640 1 --2- 172.21.15.16:0/302425707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f833415e820 0x7f83340cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.480+0000 7f833a391640 1 --2- 172.21.15.16:0/302425707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f833415b7b0 0x7f83340ce7c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f83300098a0 tx=0x7f833002a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1de3217fe039ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.480+0000 7f833938f640 1 -- 172.21.15.16:0/302425707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330037070 con 0x7f833415b7b0 2024-04-25T10:10:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.480+0000 7f833938f640 1 -- 172.21.15.16:0/302425707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8330032440 con 0x7f833415b7b0 2024-04-25T10:10:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.480+0000 7f833938f640 1 -- 172.21.15.16:0/302425707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330032810 con 0x7f833415b7b0 2024-04-25T10:10:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.481+0000 7f833c5f5640 1 -- 172.21.15.16:0/302425707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f833415b7b0 msgr2=0x7f83340ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.481+0000 7f833c5f5640 1 --2- 172.21.15.16:0/302425707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f833415b7b0 0x7f83340ce7c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f83300098a0 tx=0x7f833002a290 comp rx=0 tx=0).stop 2024-04-25T10:10:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.481+0000 7f833c5f5640 1 -- 172.21.15.16:0/302425707 shutdown_connections 2024-04-25T10:10:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.481+0000 7f833c5f5640 1 --2- 172.21.15.16:0/302425707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f833415e820 0x7f83340cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.481+0000 7f833c5f5640 1 --2- 172.21.15.16:0/302425707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f833415b7b0 0x7f83340ce7c0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.481+0000 7f833c5f5640 1 -- 172.21.15.16:0/302425707 >> 172.21.15.16:0/302425707 conn(0x7f833406f4a0 msgr2=0x7f833406f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.481+0000 7f833c5f5640 1 -- 172.21.15.16:0/302425707 shutdown_connections 2024-04-25T10:10:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.481+0000 7f833c5f5640 1 -- 172.21.15.16:0/302425707 wait complete. 2024-04-25T10:10:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.482+0000 7f833c5f5640 1 Processor -- start 2024-04-25T10:10:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.482+0000 7f833c5f5640 1 -- start start 2024-04-25T10:10:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.483+0000 7f833c5f5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83340cf760 0x7f83340b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.483+0000 7f833c5f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f833415b7b0 0x7f8334152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.483+0000 7f833c5f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f833415e820 0x7f83340b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.483+0000 7f833c5f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83341619f0 con 0x7f833415e820 2024-04-25T10:10:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.483+0000 7f833c5f5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83340cdfc0 con 0x7f83340cf760 2024-04-25T10:10:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.483+0000 7f833c5f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83340cc080 con 0x7f833415b7b0 2024-04-25T10:10:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.483+0000 7f833ab92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f833415e820 0x7f83340b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.483+0000 7f833ab92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f833415e820 0x7f83340b53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57490/0 (socket says 172.21.15.16:57490) 2024-04-25T10:10:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.483+0000 7f833ab92640 1 -- 172.21.15.16:0/2671821388 learned_addr learned my addr 172.21.15.16:0/2671821388 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.483+0000 7f833a391640 1 --2- 172.21.15.16:0/2671821388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83340cf760 0x7f83340b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.483+0000 7f8339b90640 1 --2- 172.21.15.16:0/2671821388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f833415b7b0 0x7f8334152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.483+0000 7f833ab92640 1 -- 172.21.15.16:0/2671821388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f833415b7b0 msgr2=0x7f8334152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.483+0000 7f833ab92640 1 --2- 172.21.15.16:0/2671821388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f833415b7b0 0x7f8334152c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.483+0000 7f833ab92640 1 -- 172.21.15.16:0/2671821388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83340cf760 msgr2=0x7f83340b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.483+0000 7f833ab92640 1 --2- 172.21.15.16:0/2671821388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83340cf760 0x7f83340b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.483+0000 7f833ab92640 1 -- 172.21.15.16:0/2671821388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83340b27f0 con 0x7f833415e820 2024-04-25T10:10:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.484+0000 7f833ab92640 1 --2- 172.21.15.16:0/2671821388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f833415e820 0x7f83340b53a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f832c00ccf0 tx=0x7f832c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.484+0000 7f832affd640 1 -- 172.21.15.16:0/2671821388 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c00f680 con 0x7f833415e820 2024-04-25T10:10:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.484+0000 7f832affd640 1 -- 172.21.15.16:0/2671821388 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c00f840 con 0x7f833415e820 2024-04-25T10:10:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.484+0000 7f833c5f5640 1 -- 172.21.15.16:0/2671821388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83340b48c0 con 0x7f833415e820 2024-04-25T10:10:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.484+0000 7f832affd640 1 -- 172.21.15.16:0/2671821388 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c00fc30 con 0x7f833415e820 2024-04-25T10:10:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.484+0000 7f833c5f5640 1 -- 172.21.15.16:0/2671821388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83340b2480 con 0x7f833415e820 2024-04-25T10:10:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.485+0000 7f833c5f5640 1 -- 172.21.15.16:0/2671821388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83340750b0 con 0x7f833415e820 2024-04-25T10:10:46.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.485+0000 7f832affd640 1 -- 172.21.15.16:0/2671821388 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f832c0213f0 con 0x7f833415e820 2024-04-25T10:10:46.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.486+0000 7f832affd640 1 --2- 172.21.15.16:0/2671821388 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f831806b500 0x7f831806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.486+0000 7f832affd640 1 -- 172.21.15.16:0/2671821388 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f832c08b030 con 0x7f833415e820 2024-04-25T10:10:46.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.486+0000 7f833a391640 1 --2- 172.21.15.16:0/2671821388 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f831806b500 0x7f831806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.486+0000 7f833a391640 1 --2- 172.21.15.16:0/2671821388 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f831806b500 0x7f831806d9c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8330034000 tx=0x7f833002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:46.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.490+0000 7f832affd640 1 -- 172.21.15.16:0/2671821388 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f832c014030 con 0x7f833415e820 2024-04-25T10:10:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.668+0000 7f833c5f5640 1 -- 172.21.15.16:0/2671821388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83340b7870 con 0x7f833415e820 2024-04-25T10:10:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.668+0000 7f832affd640 1 -- 172.21.15.16:0/2671821388 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f832c0597b0 con 0x7f833415e820 2024-04-25T10:10:46.675 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:10:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.670+0000 7f833c5f5640 1 -- 172.21.15.16:0/2671821388 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f831806b500 msgr2=0x7f831806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.670+0000 7f833c5f5640 1 --2- 172.21.15.16:0/2671821388 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f831806b500 0x7f831806d9c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8330034000 tx=0x7f833002e000 comp rx=0 tx=0).stop 2024-04-25T10:10:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.670+0000 7f833c5f5640 1 -- 172.21.15.16:0/2671821388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f833415e820 msgr2=0x7f83340b53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.670+0000 7f833c5f5640 1 --2- 172.21.15.16:0/2671821388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f833415e820 0x7f83340b53a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f832c00ccf0 tx=0x7f832c00a650 comp rx=0 tx=0).stop 2024-04-25T10:10:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.670+0000 7f833c5f5640 1 -- 172.21.15.16:0/2671821388 shutdown_connections 2024-04-25T10:10:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.670+0000 7f833c5f5640 1 --2- 172.21.15.16:0/2671821388 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f831806b500 0x7f831806d9c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.670+0000 7f833c5f5640 1 --2- 172.21.15.16:0/2671821388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f833415e820 0x7f83340b53a0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.670+0000 7f833c5f5640 1 --2- 172.21.15.16:0/2671821388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f833415b7b0 0x7f8334152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.670+0000 7f833c5f5640 1 --2- 172.21.15.16:0/2671821388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83340cf760 0x7f83340b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.670+0000 7f833c5f5640 1 -- 172.21.15.16:0/2671821388 >> 172.21.15.16:0/2671821388 conn(0x7f833406f4a0 msgr2=0x7f833415c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.670+0000 7f833c5f5640 1 -- 172.21.15.16:0/2671821388 shutdown_connections 2024-04-25T10:10:46.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.670+0000 7f833c5f5640 1 -- 172.21.15.16:0/2671821388 wait complete. 2024-04-25T10:10:46.687 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:10:46.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.785+0000 7f6797ed2640 1 Processor -- start 2024-04-25T10:10:46.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.785+0000 7f6797ed2640 1 -- start start 2024-04-25T10:10:46.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.786+0000 7f6797ed2640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f679015dbb0 0x7f67900b9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.786+0000 7f6797ed2640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f67900b5c70 0x7f67900b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.786+0000 7f6797ed2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67900d2670 con 0x7f67900d1e20 2024-04-25T10:10:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.786+0000 7f6797ed2640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67900d2950 con 0x7f67900b5c70 2024-04-25T10:10:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.786+0000 7f6797ed2640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67900d2c30 con 0x7f679015dbb0 2024-04-25T10:10:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.786+0000 7f679546d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f67900d1e20 0x7f67900b5560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56740/0 (socket says 172.21.15.16:56740) 2024-04-25T10:10:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.786+0000 7f679546d640 1 -- 172.21.15.16:0/1348092472 learned_addr learned my addr 172.21.15.16:0/1348092472 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.786+0000 7f6795c6e640 1 --2- 172.21.15.16:0/1348092472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f679015dbb0 0x7f67900b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.786+0000 7f679646f640 1 --2- 172.21.15.16:0/1348092472 >> v2:172.21.15.186:3300/0 conn(0x7f67900b5c70 0x7f67900b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.787+0000 7f6794c6c640 1 -- 172.21.15.16:0/1348092472 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222736136 0 0) 0x7f67900d2670 con 0x7f67900d1e20 2024-04-25T10:10:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.787+0000 7f6794c6c640 1 -- 172.21.15.16:0/1348092472 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6778003520 con 0x7f67900d1e20 2024-04-25T10:10:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.787+0000 7f6795c6e640 1 -- 172.21.15.16:0/1348092472 >> v2:172.21.15.186:3300/0 conn(0x7f67900b5c70 msgr2=0x7f67900b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:46.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.787+0000 7f6795c6e640 1 --2- 172.21.15.16:0/1348092472 >> v2:172.21.15.186:3300/0 conn(0x7f67900b5c70 0x7f67900b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.787+0000 7f6795c6e640 1 -- 172.21.15.16:0/1348092472 >> v1:172.21.15.16:6789/0 conn(0x7f67900d1e20 legacy=0x7f67900b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.787+0000 7f6794c6c640 1 -- 172.21.15.16:0/1348092472 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2057896959 0 0) 0x7f6778003520 con 0x7f67900d1e20 2024-04-25T10:10:46.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.787+0000 7f6795c6e640 1 -- 172.21.15.16:0/1348092472 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67900d2f10 con 0x7f679015dbb0 2024-04-25T10:10:46.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.787+0000 7f679646f640 1 --2- 172.21.15.16:0/1348092472 >> v2:172.21.15.186:3300/0 conn(0x7f67900b5c70 0x7f67900b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:10:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.788+0000 7f6795c6e640 1 --2- 172.21.15.16:0/1348092472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f679015dbb0 0x7f67900b9720 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f67880097a0 tx=0x7f678802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1142280d579ecfc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.788+0000 7f6794c6c640 1 -- 172.21.15.16:0/1348092472 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6788037070 con 0x7f679015dbb0 2024-04-25T10:10:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.788+0000 7f6794c6c640 1 -- 172.21.15.16:0/1348092472 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678802ab10 con 0x7f679015dbb0 2024-04-25T10:10:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.788+0000 7f6794c6c640 1 -- 172.21.15.16:0/1348092472 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6784001670 con 0x7f679015dbb0 2024-04-25T10:10:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.788+0000 7f6797ed2640 1 -- 172.21.15.16:0/1348092472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f679015dbb0 msgr2=0x7f67900b9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.788+0000 7f6797ed2640 1 --2- 172.21.15.16:0/1348092472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f679015dbb0 0x7f67900b9720 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f67880097a0 tx=0x7f678802a080 comp rx=0 tx=0).stop 2024-04-25T10:10:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.788+0000 7f6797ed2640 1 -- 172.21.15.16:0/1348092472 shutdown_connections 2024-04-25T10:10:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.788+0000 7f6797ed2640 1 --2- 172.21.15.16:0/1348092472 >> v2:172.21.15.186:3300/0 conn(0x7f67900b5c70 0x7f67900b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.788+0000 7f6797ed2640 1 --2- 172.21.15.16:0/1348092472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f679015dbb0 0x7f67900b9720 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.788+0000 7f6797ed2640 1 -- 172.21.15.16:0/1348092472 >> 172.21.15.16:0/1348092472 conn(0x7f67900d0560 msgr2=0x7f67900d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:46.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.788+0000 7f6797ed2640 1 -- 172.21.15.16:0/1348092472 shutdown_connections 2024-04-25T10:10:46.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.788+0000 7f6797ed2640 1 -- 172.21.15.16:0/1348092472 wait complete. 2024-04-25T10:10:46.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.790+0000 7f6797ed2640 1 Processor -- start 2024-04-25T10:10:46.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.790+0000 7f6797ed2640 1 -- start start 2024-04-25T10:10:46.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.790+0000 7f6797ed2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67900b5c70 0x7f67901f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.790+0000 7f6797ed2640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67900d1e20 0x7f6790158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.790+0000 7f6797ed2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f679015dbb0 0x7f6790159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.790+0000 7f6797ed2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679015e110 con 0x7f67900b5c70 2024-04-25T10:10:46.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.790+0000 7f6797ed2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679015df90 con 0x7f679015dbb0 2024-04-25T10:10:46.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.790+0000 7f6797ed2640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67900d2260 con 0x7f67900d1e20 2024-04-25T10:10:46.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.790+0000 7f6795c6e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67900b5c70 0x7f67901f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.790+0000 7f6795c6e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67900b5c70 0x7f67901f65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57500/0 (socket says 172.21.15.16:57500) 2024-04-25T10:10:46.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.790+0000 7f6795c6e640 1 -- 172.21.15.16:0/4052255824 learned_addr learned my addr 172.21.15.16:0/4052255824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:46.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.791+0000 7f679546d640 1 --2- 172.21.15.16:0/4052255824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67900d1e20 0x7f6790158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.791+0000 7f679646f640 1 --2- 172.21.15.16:0/4052255824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f679015dbb0 0x7f6790159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.791+0000 7f6795c6e640 1 -- 172.21.15.16:0/4052255824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67900d1e20 msgr2=0x7f6790158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.791+0000 7f6795c6e640 1 --2- 172.21.15.16:0/4052255824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67900d1e20 0x7f6790158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.791+0000 7f6795c6e640 1 -- 172.21.15.16:0/4052255824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f679015dbb0 msgr2=0x7f6790159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.791+0000 7f6795c6e640 1 --2- 172.21.15.16:0/4052255824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f679015dbb0 0x7f6790159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.791+0000 7f6795c6e640 1 -- 172.21.15.16:0/4052255824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6790156c90 con 0x7f67900b5c70 2024-04-25T10:10:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.791+0000 7f6795c6e640 1 --2- 172.21.15.16:0/4052255824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67900b5c70 0x7f67901f65f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6788035040 tx=0x7f67880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.791+0000 7f678e7fc640 1 -- 172.21.15.16:0/4052255824 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6788041020 con 0x7f67900b5c70 2024-04-25T10:10:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.791+0000 7f678e7fc640 1 -- 172.21.15.16:0/4052255824 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678803d620 con 0x7f67900b5c70 2024-04-25T10:10:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.791+0000 7f6797ed2640 1 -- 172.21.15.16:0/4052255824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6790156f70 con 0x7f67900b5c70 2024-04-25T10:10:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.791+0000 7f678e7fc640 1 -- 172.21.15.16:0/4052255824 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6788037040 con 0x7f67900b5c70 2024-04-25T10:10:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.791+0000 7f6797ed2640 1 -- 172.21.15.16:0/4052255824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6790157530 con 0x7f67900b5c70 2024-04-25T10:10:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.792+0000 7f6797ed2640 1 -- 172.21.15.16:0/4052255824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67900750b0 con 0x7f67900b5c70 2024-04-25T10:10:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.793+0000 7f678e7fc640 1 -- 172.21.15.16:0/4052255824 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6788044020 con 0x7f67900b5c70 2024-04-25T10:10:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.793+0000 7f678e7fc640 1 --2- 172.21.15.16:0/4052255824 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f677c06b500 0x7f677c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.793+0000 7f678e7fc640 1 -- 172.21.15.16:0/4052255824 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f67880aa840 con 0x7f67900b5c70 2024-04-25T10:10:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.793+0000 7f679546d640 1 --2- 172.21.15.16:0/4052255824 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f677c06b500 0x7f677c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.794+0000 7f679546d640 1 --2- 172.21.15.16:0/4052255824 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f677c06b500 0x7f677c06d9c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f679015a260 tx=0x7f6784009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.797+0000 7f678e7fc640 1 -- 172.21.15.16:0/4052255824 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f678804b030 con 0x7f67900b5c70 2024-04-25T10:10:46.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.979+0000 7f6797ed2640 1 -- 172.21.15.16:0/4052255824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f67900b7020 con 0x7f67900b5c70 2024-04-25T10:10:46.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.980+0000 7f678e7fc640 1 -- 172.21.15.16:0/4052255824 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6788079580 con 0x7f67900b5c70 2024-04-25T10:10:46.985 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:10:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.981+0000 7f6797ed2640 1 -- 172.21.15.16:0/4052255824 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f677c06b500 msgr2=0x7f677c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.981+0000 7f6797ed2640 1 --2- 172.21.15.16:0/4052255824 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f677c06b500 0x7f677c06d9c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f679015a260 tx=0x7f6784009040 comp rx=0 tx=0).stop 2024-04-25T10:10:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.981+0000 7f6797ed2640 1 -- 172.21.15.16:0/4052255824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67900b5c70 msgr2=0x7f67901f65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.981+0000 7f6797ed2640 1 --2- 172.21.15.16:0/4052255824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67900b5c70 0x7f67901f65f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6788035040 tx=0x7f67880046c0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.981+0000 7f6797ed2640 1 -- 172.21.15.16:0/4052255824 shutdown_connections 2024-04-25T10:10:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.981+0000 7f6797ed2640 1 --2- 172.21.15.16:0/4052255824 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f677c06b500 0x7f677c06d9c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.981+0000 7f6797ed2640 1 --2- 172.21.15.16:0/4052255824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f679015dbb0 0x7f6790159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.981+0000 7f6797ed2640 1 --2- 172.21.15.16:0/4052255824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67900d1e20 0x7f6790158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.981+0000 7f6797ed2640 1 --2- 172.21.15.16:0/4052255824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67900b5c70 0x7f67901f65f0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.981+0000 7f6797ed2640 1 -- 172.21.15.16:0/4052255824 >> 172.21.15.16:0/4052255824 conn(0x7f67900d0560 msgr2=0x7f679006c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.981+0000 7f6797ed2640 1 -- 172.21.15.16:0/4052255824 shutdown_connections 2024-04-25T10:10:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:46.981+0000 7f6797ed2640 1 -- 172.21.15.16:0/4052255824 wait complete. 2024-04-25T10:10:46.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:10:47.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.095+0000 7fc73b529640 1 Processor -- start 2024-04-25T10:10:47.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.095+0000 7fc73b529640 1 -- start start 2024-04-25T10:10:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.096+0000 7fc73b529640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc7340b97c0 0x7fc7340b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.096+0000 7fc73b529640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc7340b5a30 0x7fc7340d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.096+0000 7fc73b529640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7340b5e40 con 0x7fc73415dc80 2024-04-25T10:10:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.096+0000 7fc73b529640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc734161f80 con 0x7fc7340b5a30 2024-04-25T10:10:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.096+0000 7fc73b529640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7341627a0 con 0x7fc7340b97c0 2024-04-25T10:10:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.096+0000 7fc7392c5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc73415dc80 0x7fc734161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56742/0 (socket says 172.21.15.16:56742) 2024-04-25T10:10:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.096+0000 7fc7392c5640 1 -- 172.21.15.16:0/1563875182 learned_addr learned my addr 172.21.15.16:0/1563875182 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.096+0000 7fc739ac6640 1 --2- 172.21.15.16:0/1563875182 >> v2:172.21.15.16:3300/0 conn(0x7fc7340b5a30 0x7fc7340d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.096+0000 7fc738ac4640 1 --2- 172.21.15.16:0/1563875182 >> v2:172.21.15.186:3300/0 conn(0x7fc7340b97c0 0x7fc7340b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.096+0000 7fc733fff640 1 -- 172.21.15.16:0/1563875182 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803756616 0 0) 0x7fc72c001110 con 0x7fc73415dc80 2024-04-25T10:10:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.097+0000 7fc733fff640 1 -- 172.21.15.16:0/1563875182 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc71c003520 con 0x7fc73415dc80 2024-04-25T10:10:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.097+0000 7fc739ac6640 1 -- 172.21.15.16:0/1563875182 >> v2:172.21.15.186:3300/0 conn(0x7fc7340b97c0 msgr2=0x7fc7340b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.097+0000 7fc739ac6640 1 --2- 172.21.15.16:0/1563875182 >> v2:172.21.15.186:3300/0 conn(0x7fc7340b97c0 0x7fc7340b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.097+0000 7fc739ac6640 1 -- 172.21.15.16:0/1563875182 >> v1:172.21.15.16:6789/0 conn(0x7fc73415dc80 legacy=0x7fc734161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.097+0000 7fc739ac6640 1 -- 172.21.15.16:0/1563875182 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7341621e0 con 0x7fc7340b5a30 2024-04-25T10:10:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.097+0000 7fc738ac4640 1 --2- 172.21.15.16:0/1563875182 >> v2:172.21.15.186:3300/0 conn(0x7fc7340b97c0 0x7fc7340b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.097+0000 7fc739ac6640 1 --2- 172.21.15.16:0/1563875182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340b5a30 0x7fc7340d2470 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc73406ba70 tx=0x7fc72402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86973454477e71e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.097+0000 7fc733fff640 1 -- 172.21.15.16:0/1563875182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc724007740 con 0x7fc7340b5a30 2024-04-25T10:10:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.097+0000 7fc733fff640 1 -- 172.21.15.16:0/1563875182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7240078e0 con 0x7fc7340b5a30 2024-04-25T10:10:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.097+0000 7fc733fff640 1 -- 172.21.15.16:0/1563875182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc724007c90 con 0x7fc7340b5a30 2024-04-25T10:10:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.098+0000 7fc73b529640 1 -- 172.21.15.16:0/1563875182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340b5a30 msgr2=0x7fc7340d2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.098+0000 7fc73b529640 1 --2- 172.21.15.16:0/1563875182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340b5a30 0x7fc7340d2470 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc73406ba70 tx=0x7fc72402a680 comp rx=0 tx=0).stop 2024-04-25T10:10:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.098+0000 7fc73b529640 1 -- 172.21.15.16:0/1563875182 shutdown_connections 2024-04-25T10:10:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.098+0000 7fc73b529640 1 --2- 172.21.15.16:0/1563875182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340b5a30 0x7fc7340d2470 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.098+0000 7fc73b529640 1 --2- 172.21.15.16:0/1563875182 >> v2:172.21.15.186:3300/0 conn(0x7fc7340b97c0 0x7fc7340b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.098+0000 7fc73b529640 1 -- 172.21.15.16:0/1563875182 >> 172.21.15.16:0/1563875182 conn(0x7fc7340d04f0 msgr2=0x7fc7340d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.098+0000 7fc73b529640 1 -- 172.21.15.16:0/1563875182 shutdown_connections 2024-04-25T10:10:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.098+0000 7fc73b529640 1 -- 172.21.15.16:0/1563875182 wait complete. 2024-04-25T10:10:47.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.099+0000 7fc73b529640 1 Processor -- start 2024-04-25T10:10:47.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.099+0000 7fc73b529640 1 -- start start 2024-04-25T10:10:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.100+0000 7fc73b529640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340b5a30 0x7fc73415eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.100+0000 7fc73b529640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7340b97c0 0x7fc73415f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.100+0000 7fc73b529640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc73415dc80 0x7fc7341f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.100+0000 7fc73b529640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73415e060 con 0x7fc7340b5a30 2024-04-25T10:10:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.100+0000 7fc73b529640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7340d1c70 con 0x7fc73415dc80 2024-04-25T10:10:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.100+0000 7fc73b529640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7340d29b0 con 0x7fc7340b97c0 2024-04-25T10:10:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.100+0000 7fc7392c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340b5a30 0x7fc73415eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.100+0000 7fc7392c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340b5a30 0x7fc73415eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57528/0 (socket says 172.21.15.16:57528) 2024-04-25T10:10:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.100+0000 7fc7392c5640 1 -- 172.21.15.16:0/1027976742 learned_addr learned my addr 172.21.15.16:0/1027976742 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.100+0000 7fc739ac6640 1 --2- 172.21.15.16:0/1027976742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc73415dc80 0x7fc7341f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.100+0000 7fc738ac4640 1 --2- 172.21.15.16:0/1027976742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7340b97c0 0x7fc73415f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.100+0000 7fc7392c5640 1 -- 172.21.15.16:0/1027976742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7340b97c0 msgr2=0x7fc73415f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.100+0000 7fc7392c5640 1 --2- 172.21.15.16:0/1027976742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7340b97c0 0x7fc73415f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.100+0000 7fc7392c5640 1 -- 172.21.15.16:0/1027976742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc73415dc80 msgr2=0x7fc7341f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.100+0000 7fc7392c5640 1 --2- 172.21.15.16:0/1027976742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc73415dc80 0x7fc7341f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.100+0000 7fc7392c5640 1 -- 172.21.15.16:0/1027976742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7341f4140 con 0x7fc7340b5a30 2024-04-25T10:10:47.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.100+0000 7fc739ac6640 1 --2- 172.21.15.16:0/1027976742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc73415dc80 0x7fc7341f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:47.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.101+0000 7fc7392c5640 1 --2- 172.21.15.16:0/1027976742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340b5a30 0x7fc73415eaf0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00c3d0 tx=0x7fc72c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:47.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.101+0000 7fc731ffb640 1 -- 172.21.15.16:0/1027976742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c017070 con 0x7fc7340b5a30 2024-04-25T10:10:47.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.101+0000 7fc731ffb640 1 -- 172.21.15.16:0/1027976742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c005ce0 con 0x7fc7340b5a30 2024-04-25T10:10:47.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.101+0000 7fc73b529640 1 -- 172.21.15.16:0/1027976742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7341f4420 con 0x7fc7340b5a30 2024-04-25T10:10:47.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.101+0000 7fc731ffb640 1 -- 172.21.15.16:0/1027976742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c007310 con 0x7fc7340b5a30 2024-04-25T10:10:47.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.101+0000 7fc73b529640 1 -- 172.21.15.16:0/1027976742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73415e5f0 con 0x7fc7340b5a30 2024-04-25T10:10:47.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.102+0000 7fc73b529640 1 -- 172.21.15.16:0/1027976742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7340750b0 con 0x7fc7340b5a30 2024-04-25T10:10:47.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.102+0000 7fc731ffb640 1 -- 172.21.15.16:0/1027976742 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc72c01a070 con 0x7fc7340b5a30 2024-04-25T10:10:47.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.103+0000 7fc731ffb640 1 --2- 172.21.15.16:0/1027976742 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc72006b4d0 0x7fc72006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.103+0000 7fc731ffb640 1 -- 172.21.15.16:0/1027976742 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fc72c0896f0 con 0x7fc7340b5a30 2024-04-25T10:10:47.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.103+0000 7fc738ac4640 1 --2- 172.21.15.16:0/1027976742 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc72006b4d0 0x7fc72006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.104+0000 7fc738ac4640 1 --2- 172.21.15.16:0/1027976742 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc72006b4d0 0x7fc72006d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc73415af00 tx=0x7fc728008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.107+0000 7fc731ffb640 1 -- 172.21.15.16:0/1027976742 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7340b5e40 con 0x7fc7340b5a30 2024-04-25T10:10:47.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.289+0000 7fc73b529640 1 -- 172.21.15.16:0/1027976742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc73415d390 con 0x7fc7340b5a30 2024-04-25T10:10:47.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.290+0000 7fc731ffb640 1 -- 172.21.15.16:0/1027976742 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc72c05ce20 con 0x7fc7340b5a30 2024-04-25T10:10:47.295 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:10:47.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.292+0000 7fc73b529640 1 -- 172.21.15.16:0/1027976742 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc72006b4d0 msgr2=0x7fc72006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.292+0000 7fc73b529640 1 --2- 172.21.15.16:0/1027976742 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc72006b4d0 0x7fc72006d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc73415af00 tx=0x7fc728008040 comp rx=0 tx=0).stop 2024-04-25T10:10:47.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.292+0000 7fc73b529640 1 -- 172.21.15.16:0/1027976742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340b5a30 msgr2=0x7fc73415eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.292+0000 7fc73b529640 1 --2- 172.21.15.16:0/1027976742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340b5a30 0x7fc73415eaf0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00c3d0 tx=0x7fc72c00c890 comp rx=0 tx=0).stop 2024-04-25T10:10:47.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.292+0000 7fc73b529640 1 -- 172.21.15.16:0/1027976742 shutdown_connections 2024-04-25T10:10:47.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.292+0000 7fc73b529640 1 --2- 172.21.15.16:0/1027976742 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc72006b4d0 0x7fc72006d990 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.292+0000 7fc73b529640 1 --2- 172.21.15.16:0/1027976742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc73415dc80 0x7fc7341f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.292+0000 7fc73b529640 1 --2- 172.21.15.16:0/1027976742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7340b97c0 0x7fc73415f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.292+0000 7fc73b529640 1 --2- 172.21.15.16:0/1027976742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340b5a30 0x7fc73415eaf0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.292+0000 7fc73b529640 1 -- 172.21.15.16:0/1027976742 >> 172.21.15.16:0/1027976742 conn(0x7fc7340d04f0 msgr2=0x7fc73406bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:47.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.292+0000 7fc73b529640 1 -- 172.21.15.16:0/1027976742 shutdown_connections 2024-04-25T10:10:47.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.292+0000 7fc73b529640 1 -- 172.21.15.16:0/1027976742 wait complete. 2024-04-25T10:10:47.309 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:47.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.404+0000 7fb0f43cf640 1 Processor -- start 2024-04-25T10:10:47.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.404+0000 7fb0f43cf640 1 -- start start 2024-04-25T10:10:47.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.405+0000 7fb0f43cf640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb0ec15a730 0x7fb0ec0ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.405+0000 7fb0f43cf640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0ec0b4020 0x7fb0ec0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.405+0000 7fb0f43cf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb0ec06d1b0 0x7fb0ec06d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.405+0000 7fb0f43cf640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0ec0b49a0 con 0x7fb0ec06d1b0 2024-04-25T10:10:47.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.405+0000 7fb0f43cf640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0ec0cc020 con 0x7fb0ec15a730 2024-04-25T10:10:47.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.405+0000 7fb0f43cf640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0b32b0 con 0x7fb0ec0b4020 2024-04-25T10:10:47.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.405+0000 7fb0f296c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb0ec06d1b0 0x7fb0ec06d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.405+0000 7fb0f296c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb0ec06d1b0 0x7fb0ec06d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57532/0 (socket says 172.21.15.16:57532) 2024-04-25T10:10:47.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.405+0000 7fb0f296c640 1 -- 172.21.15.16:0/1839423702 learned_addr learned my addr 172.21.15.16:0/1839423702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:47.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.405+0000 7fb0f216b640 1 --2- 172.21.15.16:0/1839423702 >> v2:172.21.15.186:3300/0 conn(0x7fb0ec15a730 0x7fb0ec0ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.405+0000 7fb0f196a640 1 --2- 172.21.15.16:0/1839423702 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0ec0b4020 0x7fb0ec0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.406+0000 7fb0f296c640 1 -- 172.21.15.16:0/1839423702 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0ec0b4020 msgr2=0x7fb0ec0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.406+0000 7fb0f296c640 1 --2- 172.21.15.16:0/1839423702 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0ec0b4020 0x7fb0ec0b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.406+0000 7fb0f296c640 1 -- 172.21.15.16:0/1839423702 >> v2:172.21.15.186:3300/0 conn(0x7fb0ec15a730 msgr2=0x7fb0ec0ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.406+0000 7fb0f296c640 1 --2- 172.21.15.16:0/1839423702 >> v2:172.21.15.186:3300/0 conn(0x7fb0ec15a730 0x7fb0ec0ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.406+0000 7fb0f296c640 1 -- 172.21.15.16:0/1839423702 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec0b3590 con 0x7fb0ec06d1b0 2024-04-25T10:10:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.406+0000 7fb0f296c640 1 --2- 172.21.15.16:0/1839423702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0ec06d1b0 0x7fb0ec06d620 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec06b9b0 tx=0x7fb0dc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781d2864a41b1fa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.406+0000 7fb0f1169640 1 -- 172.21.15.16:0/1839423702 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc036070 con 0x7fb0ec06d1b0 2024-04-25T10:10:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.407+0000 7fb0f1169640 1 -- 172.21.15.16:0/1839423702 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0dc031600 con 0x7fb0ec06d1b0 2024-04-25T10:10:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.407+0000 7fb0f1169640 1 -- 172.21.15.16:0/1839423702 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc0319d0 con 0x7fb0ec06d1b0 2024-04-25T10:10:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.407+0000 7fb0f43cf640 1 -- 172.21.15.16:0/1839423702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0ec06d1b0 msgr2=0x7fb0ec06d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.407+0000 7fb0f43cf640 1 --2- 172.21.15.16:0/1839423702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0ec06d1b0 0x7fb0ec06d620 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec06b9b0 tx=0x7fb0dc02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.407+0000 7fb0f43cf640 1 -- 172.21.15.16:0/1839423702 shutdown_connections 2024-04-25T10:10:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.407+0000 7fb0f43cf640 1 --2- 172.21.15.16:0/1839423702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0ec06d1b0 0x7fb0ec06d620 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.407+0000 7fb0f43cf640 1 --2- 172.21.15.16:0/1839423702 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0ec0b4020 0x7fb0ec0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.407+0000 7fb0f43cf640 1 --2- 172.21.15.16:0/1839423702 >> v2:172.21.15.186:3300/0 conn(0x7fb0ec15a730 0x7fb0ec0ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.407+0000 7fb0f43cf640 1 -- 172.21.15.16:0/1839423702 >> 172.21.15.16:0/1839423702 conn(0x7fb0ec06aba0 msgr2=0x7fb0ec06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.407+0000 7fb0f43cf640 1 -- 172.21.15.16:0/1839423702 shutdown_connections 2024-04-25T10:10:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.407+0000 7fb0f43cf640 1 -- 172.21.15.16:0/1839423702 wait complete. 2024-04-25T10:10:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.408+0000 7fb0f43cf640 1 Processor -- start 2024-04-25T10:10:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.408+0000 7fb0f43cf640 1 -- start start 2024-04-25T10:10:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.409+0000 7fb0f43cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0ec06d1b0 0x7fb0ec1482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.409+0000 7fb0f43cf640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0ec0b4020 0x7fb0ec15cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.409+0000 7fb0f43cf640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0ec15a730 0x7fb0ec145320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.409+0000 7fb0f43cf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0cd620 con 0x7fb0ec06d1b0 2024-04-25T10:10:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.409+0000 7fb0f43cf640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0b3b50 con 0x7fb0ec0b4020 2024-04-25T10:10:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.409+0000 7fb0f43cf640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0aeef0 con 0x7fb0ec15a730 2024-04-25T10:10:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.409+0000 7fb0f216b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0ec06d1b0 0x7fb0ec1482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.409+0000 7fb0f216b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0ec06d1b0 0x7fb0ec1482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57544/0 (socket says 172.21.15.16:57544) 2024-04-25T10:10:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.409+0000 7fb0f216b640 1 -- 172.21.15.16:0/1055875075 learned_addr learned my addr 172.21.15.16:0/1055875075 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.409+0000 7fb0f296c640 1 --2- 172.21.15.16:0/1055875075 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0ec15a730 0x7fb0ec145320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.409+0000 7fb0f196a640 1 --2- 172.21.15.16:0/1055875075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0ec0b4020 0x7fb0ec15cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.409+0000 7fb0f216b640 1 -- 172.21.15.16:0/1055875075 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0ec15a730 msgr2=0x7fb0ec145320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.409+0000 7fb0f216b640 1 --2- 172.21.15.16:0/1055875075 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0ec15a730 0x7fb0ec145320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.409+0000 7fb0f216b640 1 -- 172.21.15.16:0/1055875075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0ec0b4020 msgr2=0x7fb0ec15cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.409+0000 7fb0f216b640 1 --2- 172.21.15.16:0/1055875075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0ec0b4020 0x7fb0ec15cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.410+0000 7fb0f216b640 1 -- 172.21.15.16:0/1055875075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec1658e0 con 0x7fb0ec06d1b0 2024-04-25T10:10:47.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.410+0000 7fb0f216b640 1 --2- 172.21.15.16:0/1055875075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0ec06d1b0 0x7fb0ec1482f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400e870 tx=0x7fb0e400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:47.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.410+0000 7fb0eaffd640 1 -- 172.21.15.16:0/1055875075 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4017070 con 0x7fb0ec06d1b0 2024-04-25T10:10:47.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.410+0000 7fb0eaffd640 1 -- 172.21.15.16:0/1055875075 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e400c940 con 0x7fb0ec06d1b0 2024-04-25T10:10:47.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.410+0000 7fb0f43cf640 1 -- 172.21.15.16:0/1055875075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ec165bc0 con 0x7fb0ec06d1b0 2024-04-25T10:10:47.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.410+0000 7fb0eaffd640 1 -- 172.21.15.16:0/1055875075 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e400cd30 con 0x7fb0ec06d1b0 2024-04-25T10:10:47.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.410+0000 7fb0f43cf640 1 -- 172.21.15.16:0/1055875075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ec148830 con 0x7fb0ec06d1b0 2024-04-25T10:10:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.411+0000 7fb0f43cf640 1 -- 172.21.15.16:0/1055875075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0ec165fb0 con 0x7fb0ec06d1b0 2024-04-25T10:10:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.412+0000 7fb0eaffd640 1 -- 172.21.15.16:0/1055875075 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0e401a070 con 0x7fb0ec06d1b0 2024-04-25T10:10:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.412+0000 7fb0eaffd640 1 --2- 172.21.15.16:0/1055875075 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb0d806b500 0x7fb0d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.412+0000 7fb0eaffd640 1 -- 172.21.15.16:0/1055875075 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fb0e408a400 con 0x7fb0ec06d1b0 2024-04-25T10:10:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.412+0000 7fb0f196a640 1 --2- 172.21.15.16:0/1055875075 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb0d806b500 0x7fb0d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.413+0000 7fb0f196a640 1 --2- 172.21.15.16:0/1055875075 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb0d806b500 0x7fb0d806d9c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec166930 tx=0x7fb0e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.416+0000 7fb0eaffd640 1 -- 172.21.15.16:0/1055875075 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb0e4059260 con 0x7fb0ec06d1b0 2024-04-25T10:10:47.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.641+0000 7fb0f43cf640 1 -- 172.21.15.16:0/1055875075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0ec0749b0 con 0x7fb0ec06d1b0 2024-04-25T10:10:47.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.642+0000 7fb0eaffd640 1 -- 172.21.15.16:0/1055875075 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fb0e4058b60 con 0x7fb0ec06d1b0 2024-04-25T10:10:47.647 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:47.647 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6070,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2805428272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2805428272},{"type":"v1","addr":"172.21.15.186:6835","nonce":2805428272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1052667029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1052667029},{"type":"v1","addr":"172.21.15.186:6837","nonce":1052667029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2897154125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2897154125},{"type":"v1","addr":"172.21.15.16:6833","nonce":2897154125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:22.077032+0000","modified":"2024-04-25T10:10:14.390498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6636":{"gid":6636,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":17,"addr":"172.21.15.16:6835/1735811292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1735811292},{"type":"v1","addr":"172.21.15.16:6835","nonce":1735811292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:10:47.648 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 44 2024-04-25T10:10:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.643+0000 7fb0f43cf640 1 -- 172.21.15.16:0/1055875075 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb0d806b500 msgr2=0x7fb0d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.643+0000 7fb0f43cf640 1 --2- 172.21.15.16:0/1055875075 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb0d806b500 0x7fb0d806d9c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec166930 tx=0x7fb0e0008040 comp rx=0 tx=0).stop 2024-04-25T10:10:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.643+0000 7fb0f43cf640 1 -- 172.21.15.16:0/1055875075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0ec06d1b0 msgr2=0x7fb0ec1482f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.643+0000 7fb0f43cf640 1 --2- 172.21.15.16:0/1055875075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0ec06d1b0 0x7fb0ec1482f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400e870 tx=0x7fb0e400ed30 comp rx=0 tx=0).stop 2024-04-25T10:10:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.644+0000 7fb0f43cf640 1 -- 172.21.15.16:0/1055875075 shutdown_connections 2024-04-25T10:10:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.644+0000 7fb0f43cf640 1 --2- 172.21.15.16:0/1055875075 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb0d806b500 0x7fb0d806d9c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.644+0000 7fb0f43cf640 1 --2- 172.21.15.16:0/1055875075 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0ec15a730 0x7fb0ec145320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.644+0000 7fb0f43cf640 1 --2- 172.21.15.16:0/1055875075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0ec0b4020 0x7fb0ec15cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.644+0000 7fb0f43cf640 1 --2- 172.21.15.16:0/1055875075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0ec06d1b0 0x7fb0ec1482f0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.644+0000 7fb0f43cf640 1 -- 172.21.15.16:0/1055875075 >> 172.21.15.16:0/1055875075 conn(0x7fb0ec06aba0 msgr2=0x7fb0ec149df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.644+0000 7fb0f43cf640 1 -- 172.21.15.16:0/1055875075 shutdown_connections 2024-04-25T10:10:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.644+0000 7fb0f43cf640 1 -- 172.21.15.16:0/1055875075 wait complete. 2024-04-25T10:10:47.660 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:47.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.754+0000 7fb22d0cb640 1 Processor -- start 2024-04-25T10:10:47.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.754+0000 7fb22d0cb640 1 -- start start 2024-04-25T10:10:47.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.755+0000 7fb22d0cb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb22815dd50 0x7fb2280d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.755+0000 7fb22d0cb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2280ceeb0 0x7fb2280b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.755+0000 7fb22d0cb640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2280d2440 con 0x7fb2280b9460 2024-04-25T10:10:47.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.755+0000 7fb22d0cb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2280d2720 con 0x7fb22815dd50 2024-04-25T10:10:47.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.755+0000 7fb22d0cb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280d2a00 con 0x7fb2280ceeb0 2024-04-25T10:10:47.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.755+0000 7fb226d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb22815dd50 0x7fb2280d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.755+0000 7fb226d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb22815dd50 0x7fb2280d1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57556/0 (socket says 172.21.15.16:57556) 2024-04-25T10:10:47.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.755+0000 7fb226d9d640 1 -- 172.21.15.16:0/1173953176 learned_addr learned my addr 172.21.15.16:0/1173953176 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:47.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.755+0000 7fb22759e640 1 --2- 172.21.15.16:0/1173953176 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2280ceeb0 0x7fb2280b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.756+0000 7fb226d9d640 1 -- 172.21.15.16:0/1173953176 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2280ceeb0 msgr2=0x7fb2280b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:47.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.756+0000 7fb226d9d640 1 --2- 172.21.15.16:0/1173953176 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2280ceeb0 0x7fb2280b6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.756+0000 7fb226d9d640 1 -- 172.21.15.16:0/1173953176 >> v1:172.21.15.186:6789/0 conn(0x7fb2280b9460 legacy=0x7fb2280ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.756+0000 7fb226d9d640 1 -- 172.21.15.16:0/1173953176 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2280d2ce0 con 0x7fb22815dd50 2024-04-25T10:10:47.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.756+0000 7fb226d9d640 1 --2- 172.21.15.16:0/1173953176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb22815dd50 0x7fb2280d1c10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb220009870 tx=0x7fb22002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b511c42e6e8d20a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:47.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.756+0000 7fb225d9b640 1 -- 172.21.15.16:0/1173953176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220037070 con 0x7fb22815dd50 2024-04-25T10:10:47.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.757+0000 7fb225d9b640 1 -- 172.21.15.16:0/1173953176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb220032440 con 0x7fb22815dd50 2024-04-25T10:10:47.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.757+0000 7fb225d9b640 1 -- 172.21.15.16:0/1173953176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220032810 con 0x7fb22815dd50 2024-04-25T10:10:47.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.757+0000 7fb22d0cb640 1 -- 172.21.15.16:0/1173953176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb22815dd50 msgr2=0x7fb2280d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.757+0000 7fb22d0cb640 1 --2- 172.21.15.16:0/1173953176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb22815dd50 0x7fb2280d1c10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb220009870 tx=0x7fb22002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.757+0000 7fb22d0cb640 1 -- 172.21.15.16:0/1173953176 shutdown_connections 2024-04-25T10:10:47.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.757+0000 7fb22d0cb640 1 --2- 172.21.15.16:0/1173953176 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2280ceeb0 0x7fb2280b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.757+0000 7fb22d0cb640 1 --2- 172.21.15.16:0/1173953176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb22815dd50 0x7fb2280d1c10 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.757+0000 7fb22d0cb640 1 -- 172.21.15.16:0/1173953176 >> 172.21.15.16:0/1173953176 conn(0x7fb2280722d0 msgr2=0x7fb2280726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:47.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.757+0000 7fb22d0cb640 1 -- 172.21.15.16:0/1173953176 shutdown_connections 2024-04-25T10:10:47.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.757+0000 7fb22d0cb640 1 -- 172.21.15.16:0/1173953176 wait complete. 2024-04-25T10:10:47.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.758+0000 7fb22d0cb640 1 Processor -- start 2024-04-25T10:10:47.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.759+0000 7fb22d0cb640 1 -- start start 2024-04-25T10:10:47.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.759+0000 7fb22d0cb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2280b9460 0x7fb2280c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.759+0000 7fb22d0cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2280ceeb0 0x7fb228145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.759+0000 7fb22d0cb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb22815dd50 0x7fb228146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.759+0000 7fb22d0cb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280d34e0 con 0x7fb2280ceeb0 2024-04-25T10:10:47.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.759+0000 7fb22d0cb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280d2150 con 0x7fb22815dd50 2024-04-25T10:10:47.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.759+0000 7fb22d0cb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280d3660 con 0x7fb2280b9460 2024-04-25T10:10:47.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.759+0000 7fb22659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2280ceeb0 0x7fb228145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.759+0000 7fb22659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2280ceeb0 0x7fb228145bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57568/0 (socket says 172.21.15.16:57568) 2024-04-25T10:10:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.759+0000 7fb22659c640 1 -- 172.21.15.16:0/509617909 learned_addr learned my addr 172.21.15.16:0/509617909 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.759+0000 7fb22759e640 1 --2- 172.21.15.16:0/509617909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb22815dd50 0x7fb228146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.759+0000 7fb226d9d640 1 --2- 172.21.15.16:0/509617909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2280b9460 0x7fb2280c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.760+0000 7fb22659c640 1 -- 172.21.15.16:0/509617909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2280b9460 msgr2=0x7fb2280c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.760+0000 7fb22659c640 1 --2- 172.21.15.16:0/509617909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2280b9460 0x7fb2280c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.760+0000 7fb22659c640 1 -- 172.21.15.16:0/509617909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb22815dd50 msgr2=0x7fb228146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.760+0000 7fb22659c640 1 --2- 172.21.15.16:0/509617909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb22815dd50 0x7fb228146110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.760+0000 7fb22659c640 1 -- 172.21.15.16:0/509617909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb228147660 con 0x7fb2280ceeb0 2024-04-25T10:10:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.760+0000 7fb22659c640 1 --2- 172.21.15.16:0/509617909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2280ceeb0 0x7fb228145bd0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb21000c3c0 tx=0x7fb21000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.760+0000 7fb21f7fe640 1 -- 172.21.15.16:0/509617909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2100052d0 con 0x7fb2280ceeb0 2024-04-25T10:10:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.760+0000 7fb21f7fe640 1 -- 172.21.15.16:0/509617909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb210005490 con 0x7fb2280ceeb0 2024-04-25T10:10:47.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.760+0000 7fb22d0cb640 1 -- 172.21.15.16:0/509617909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2281468a0 con 0x7fb2280ceeb0 2024-04-25T10:10:47.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.760+0000 7fb21f7fe640 1 -- 172.21.15.16:0/509617909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210007260 con 0x7fb2280ceeb0 2024-04-25T10:10:47.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.760+0000 7fb22d0cb640 1 -- 172.21.15.16:0/509617909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb228146e60 con 0x7fb2280ceeb0 2024-04-25T10:10:47.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.761+0000 7fb22d0cb640 1 -- 172.21.15.16:0/509617909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb228067850 con 0x7fb2280ceeb0 2024-04-25T10:10:47.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.762+0000 7fb21f7fe640 1 -- 172.21.15.16:0/509617909 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb210007400 con 0x7fb2280ceeb0 2024-04-25T10:10:47.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.762+0000 7fb21f7fe640 1 --2- 172.21.15.16:0/509617909 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb20806b4d0 0x7fb20806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.762+0000 7fb21f7fe640 1 -- 172.21.15.16:0/509617909 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fb210089690 con 0x7fb2280ceeb0 2024-04-25T10:10:47.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.762+0000 7fb226d9d640 1 --2- 172.21.15.16:0/509617909 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb20806b4d0 0x7fb20806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.763+0000 7fb226d9d640 1 --2- 172.21.15.16:0/509617909 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb20806b4d0 0x7fb20806d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb220034000 tx=0x7fb22002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:47.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.766+0000 7fb21f7fe640 1 -- 172.21.15.16:0/509617909 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb210057e10 con 0x7fb2280ceeb0 2024-04-25T10:10:47.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.947+0000 7fb22d0cb640 1 -- 172.21.15.16:0/509617909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2280cfa70 con 0x7fb2280ceeb0 2024-04-25T10:10:47.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.948+0000 7fb21f7fe640 1 -- 172.21.15.16:0/509617909 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16885 (secure 0 0 0) 0x7fb2280cfa70 con 0x7fb2280ceeb0 2024-04-25T10:10:47.953 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:47.953 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:09:53.479005+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:20.361009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:21.048892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.19","mappings":[{"from":2,"to":7}]},{"pgid":"11.26","mappings":[{"from":1,"to":4}]},{"pgid":"11.27","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.186:6837/3292077067":"2024-04-26T10:09:31.374252+0000","172.21.15.186:6836/3292077067":"2024-04-26T10:09:31.374252+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:47.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.950+0000 7fb22d0cb640 1 -- 172.21.15.16:0/509617909 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb20806b4d0 msgr2=0x7fb20806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.950+0000 7fb22d0cb640 1 --2- 172.21.15.16:0/509617909 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb20806b4d0 0x7fb20806d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb220034000 tx=0x7fb22002e000 comp rx=0 tx=0).stop 2024-04-25T10:10:47.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.950+0000 7fb22d0cb640 1 -- 172.21.15.16:0/509617909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2280ceeb0 msgr2=0x7fb228145bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.950+0000 7fb22d0cb640 1 --2- 172.21.15.16:0/509617909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2280ceeb0 0x7fb228145bd0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb21000c3c0 tx=0x7fb21000c880 comp rx=0 tx=0).stop 2024-04-25T10:10:47.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.950+0000 7fb22d0cb640 1 -- 172.21.15.16:0/509617909 shutdown_connections 2024-04-25T10:10:47.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.950+0000 7fb22d0cb640 1 --2- 172.21.15.16:0/509617909 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb20806b4d0 0x7fb20806d990 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.950+0000 7fb22d0cb640 1 --2- 172.21.15.16:0/509617909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb22815dd50 0x7fb228146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.950+0000 7fb22d0cb640 1 --2- 172.21.15.16:0/509617909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2280ceeb0 0x7fb228145bd0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.950+0000 7fb22d0cb640 1 --2- 172.21.15.16:0/509617909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2280b9460 0x7fb2280c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.950+0000 7fb22d0cb640 1 -- 172.21.15.16:0/509617909 >> 172.21.15.16:0/509617909 conn(0x7fb2280722d0 msgr2=0x7fb2280b7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:47.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.950+0000 7fb22d0cb640 1 -- 172.21.15.16:0/509617909 shutdown_connections 2024-04-25T10:10:47.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:47.950+0000 7fb22d0cb640 1 -- 172.21.15.16:0/509617909 wait complete. 2024-04-25T10:10:47.966 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:10:47.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:48.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.062+0000 7f8565a93640 1 Processor -- start 2024-04-25T10:10:48.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.062+0000 7f8565a93640 1 -- start start 2024-04-25T10:10:48.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.062+0000 7f8565a93640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f856006e2e0 0x7f85600cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.062+0000 7f8565a93640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85600cdb10 con 0x7f856015a870 2024-04-25T10:10:48.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.062+0000 7f8565a93640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85600cddf0 con 0x7f85600cfc40 2024-04-25T10:10:48.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.062+0000 7f8565a93640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85600ce720 con 0x7f856006e2e0 2024-04-25T10:10:48.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f856015a870 0x7f85600cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56752/0 (socket says 172.21.15.16:56752) 2024-04-25T10:10:48.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855f7fe640 1 -- 172.21.15.16:0/294090972 learned_addr learned my addr 172.21.15.16:0/294090972 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:48.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855effd640 1 --2- 172.21.15.16:0/294090972 >> v2:172.21.15.186:3300/0 conn(0x7f856006e2e0 0x7f85600cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855e7fc640 1 -- 172.21.15.16:0/294090972 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4040806720 0 0) 0x7f85600cdb10 con 0x7f856015a870 2024-04-25T10:10:48.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855e7fc640 1 -- 172.21.15.16:0/294090972 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8548003520 con 0x7f856015a870 2024-04-25T10:10:48.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855e7fc640 1 -- 172.21.15.16:0/294090972 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2364356680 0 0) 0x7f8548003520 con 0x7f856015a870 2024-04-25T10:10:48.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855e7fc640 1 -- 172.21.15.16:0/294090972 >> v2:172.21.15.186:3300/0 conn(0x7f856006e2e0 msgr2=0x7f85600cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:48.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855e7fc640 1 --2- 172.21.15.16:0/294090972 >> v2:172.21.15.186:3300/0 conn(0x7f856006e2e0 0x7f85600cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855e7fc640 1 -- 172.21.15.16:0/294090972 >> v1:172.21.15.186:6789/0 conn(0x7f85600cfc40 legacy=0x7f856015bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855e7fc640 1 -- 172.21.15.16:0/294090972 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85600ce0d0 con 0x7f856015a870 2024-04-25T10:10:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855e7fc640 1 -- 172.21.15.16:0/294090972 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f8550001670 con 0x7f856015a870 2024-04-25T10:10:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855e7fc640 1 -- 172.21.15.16:0/294090972 >> v1:172.21.15.16:6789/0 conn(0x7f856015a870 legacy=0x7f85600cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855e7fc640 1 --2- 172.21.15.16:0/294090972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85480039b0 0x7f8548003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855e7fc640 1 --2- 172.21.15.16:0/294090972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8548004800 0x7f8548004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855e7fc640 1 --2- 172.21.15.16:0/294090972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85480051c0 0x7f854800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855e7fc640 1 -- 172.21.15.16:0/294090972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85600ce720 con 0x7f85480039b0 2024-04-25T10:10:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855e7fc640 1 -- 172.21.15.16:0/294090972 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854800c060 con 0x7f8548004800 2024-04-25T10:10:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855e7fc640 1 -- 172.21.15.16:0/294090972 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f854800c260 con 0x7f85480051c0 2024-04-25T10:10:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855f7fe640 1 --2- 172.21.15.16:0/294090972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85480039b0 0x7f8548003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855effd640 1 --2- 172.21.15.16:0/294090972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8548004800 0x7f8548004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.063+0000 7f855ffff640 1 --2- 172.21.15.16:0/294090972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85480051c0 0x7f854800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.064+0000 7f855f7fe640 1 -- 172.21.15.16:0/294090972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85480051c0 msgr2=0x7f854800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:48.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.064+0000 7f855f7fe640 1 --2- 172.21.15.16:0/294090972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85480051c0 0x7f854800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.064+0000 7f855f7fe640 1 -- 172.21.15.16:0/294090972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8548004800 msgr2=0x7f8548004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:48.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.064+0000 7f855f7fe640 1 --2- 172.21.15.16:0/294090972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8548004800 0x7f8548004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.064+0000 7f855ffff640 1 -- 172.21.15.16:0/294090972 reap_dead start 2024-04-25T10:10:48.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.064+0000 7f855f7fe640 1 -- 172.21.15.16:0/294090972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8548003520 con 0x7f85480039b0 2024-04-25T10:10:48.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.064+0000 7f855f7fe640 1 --2- 172.21.15.16:0/294090972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85480039b0 0x7f8548003dd0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f855000c240 tx=0x7f8548005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b630b6af985ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:48.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.064+0000 7f855e7fc640 1 -- 172.21.15.16:0/294090972 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8550007cf0 con 0x7f85480039b0 2024-04-25T10:10:48.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.064+0000 7f855e7fc640 1 -- 172.21.15.16:0/294090972 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8550005af0 con 0x7f85480039b0 2024-04-25T10:10:48.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.064+0000 7f855e7fc640 1 -- 172.21.15.16:0/294090972 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8550007070 con 0x7f85480039b0 2024-04-25T10:10:48.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.065+0000 7f8565a93640 1 -- 172.21.15.16:0/294090972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85480039b0 msgr2=0x7f8548003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.065+0000 7f8565a93640 1 --2- 172.21.15.16:0/294090972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85480039b0 0x7f8548003dd0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f855000c240 tx=0x7f8548005830 comp rx=0 tx=0).stop 2024-04-25T10:10:48.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.065+0000 7f8565a93640 1 -- 172.21.15.16:0/294090972 shutdown_connections 2024-04-25T10:10:48.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.065+0000 7f8565a93640 1 --2- 172.21.15.16:0/294090972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85480039b0 0x7f8548003dd0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.065+0000 7f8565a93640 1 -- 172.21.15.16:0/294090972 >> 172.21.15.16:0/294090972 conn(0x7f856006aba0 msgr2=0x7f856006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:48.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.065+0000 7f8565a93640 1 -- 172.21.15.16:0/294090972 shutdown_connections 2024-04-25T10:10:48.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.065+0000 7f8565a93640 1 -- 172.21.15.16:0/294090972 wait complete. 2024-04-25T10:10:48.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.066+0000 7f8565a93640 1 Processor -- start 2024-04-25T10:10:48.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.066+0000 7f8565a93640 1 -- start start 2024-04-25T10:10:48.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.067+0000 7f8565a93640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85480039b0 0x7f8560145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.067+0000 7f8565a93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85601563c0 0x7f8560156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.067+0000 7f8565a93640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85600a84a0 0x7f85600a8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.067+0000 7f8565a93640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85600cd700 con 0x7f85601563c0 2024-04-25T10:10:48.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.067+0000 7f8565a93640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85600cd580 con 0x7f85600a84a0 2024-04-25T10:10:48.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.067+0000 7f8565a93640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85600cd880 con 0x7f85480039b0 2024-04-25T10:10:48.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.067+0000 7f855effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85601563c0 0x7f8560156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.067+0000 7f855effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85601563c0 0x7f8560156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57594/0 (socket says 172.21.15.16:57594) 2024-04-25T10:10:48.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.067+0000 7f855effd640 1 -- 172.21.15.16:0/1926518696 learned_addr learned my addr 172.21.15.16:0/1926518696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:48.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.067+0000 7f855f7fe640 1 --2- 172.21.15.16:0/1926518696 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85480039b0 0x7f8560145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.067+0000 7f855ffff640 1 --2- 172.21.15.16:0/1926518696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85600a84a0 0x7f85600a8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.068+0000 7f855effd640 1 -- 172.21.15.16:0/1926518696 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85480039b0 msgr2=0x7f8560145e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.068+0000 7f855effd640 1 --2- 172.21.15.16:0/1926518696 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85480039b0 0x7f8560145e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.068+0000 7f855effd640 1 -- 172.21.15.16:0/1926518696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85600a84a0 msgr2=0x7f85600a8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.068+0000 7f855effd640 1 --2- 172.21.15.16:0/1926518696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85600a84a0 0x7f85600a8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.068+0000 7f855effd640 1 -- 172.21.15.16:0/1926518696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8560146880 con 0x7f85601563c0 2024-04-25T10:10:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.068+0000 7f855effd640 1 --2- 172.21.15.16:0/1926518696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85601563c0 0x7f8560156840 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f8554011a20 tx=0x7f8554011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.068+0000 7f8547fff640 1 -- 172.21.15.16:0/1926518696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554004240 con 0x7f85601563c0 2024-04-25T10:10:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.068+0000 7f8547fff640 1 -- 172.21.15.16:0/1926518696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85540043e0 con 0x7f85601563c0 2024-04-25T10:10:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.068+0000 7f8565a93640 1 -- 172.21.15.16:0/1926518696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85600a9080 con 0x7f85601563c0 2024-04-25T10:10:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.068+0000 7f8547fff640 1 -- 172.21.15.16:0/1926518696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855400e910 con 0x7f85601563c0 2024-04-25T10:10:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.068+0000 7f8565a93640 1 -- 172.21.15.16:0/1926518696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8560146360 con 0x7f85601563c0 2024-04-25T10:10:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.069+0000 7f8565a93640 1 -- 172.21.15.16:0/1926518696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85600749b0 con 0x7f85601563c0 2024-04-25T10:10:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.069+0000 7f8547fff640 1 -- 172.21.15.16:0/1926518696 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f855400eab0 con 0x7f85601563c0 2024-04-25T10:10:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.070+0000 7f8547fff640 1 --2- 172.21.15.16:0/1926518696 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f854c06b500 0x7f854c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.070+0000 7f8547fff640 1 -- 172.21.15.16:0/1926518696 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f855408c1f0 con 0x7f85601563c0 2024-04-25T10:10:48.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.070+0000 7f855f7fe640 1 --2- 172.21.15.16:0/1926518696 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f854c06b500 0x7f854c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.071+0000 7f855f7fe640 1 --2- 172.21.15.16:0/1926518696 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f854c06b500 0x7f854c06d9c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8550032000 tx=0x7f8550009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:48.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.074+0000 7f8547fff640 1 -- 172.21.15.16:0/1926518696 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f855405afc0 con 0x7f85601563c0 2024-04-25T10:10:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.295+0000 7f8565a93640 1 -- 172.21.15.16:0/1926518696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85600a95a0 con 0x7f85601563c0 2024-04-25T10:10:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.296+0000 7f8547fff640 1 -- 172.21.15.16:0/1926518696 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f855405a8c0 con 0x7f85601563c0 2024-04-25T10:10:48.302 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:48.302 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6070,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2805428272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2805428272},{"type":"v1","addr":"172.21.15.186:6835","nonce":2805428272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1052667029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1052667029},{"type":"v1","addr":"172.21.15.186:6837","nonce":1052667029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2897154125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2897154125},{"type":"v1","addr":"172.21.15.16:6833","nonce":2897154125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:22.077032+0000","modified":"2024-04-25T10:10:14.390498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6636":{"gid":6636,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":17,"addr":"172.21.15.16:6835/1735811292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1735811292},{"type":"v1","addr":"172.21.15.16:6835","nonce":1735811292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:10:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 44 2024-04-25T10:10:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.298+0000 7f8565a93640 1 -- 172.21.15.16:0/1926518696 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f854c06b500 msgr2=0x7f854c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.298+0000 7f8565a93640 1 --2- 172.21.15.16:0/1926518696 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f854c06b500 0x7f854c06d9c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8550032000 tx=0x7f8550009030 comp rx=0 tx=0).stop 2024-04-25T10:10:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.298+0000 7f8565a93640 1 -- 172.21.15.16:0/1926518696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85601563c0 msgr2=0x7f8560156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.298+0000 7f8565a93640 1 --2- 172.21.15.16:0/1926518696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85601563c0 0x7f8560156840 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f8554011a20 tx=0x7f8554011ee0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.298+0000 7f8565a93640 1 -- 172.21.15.16:0/1926518696 shutdown_connections 2024-04-25T10:10:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.298+0000 7f8565a93640 1 --2- 172.21.15.16:0/1926518696 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f854c06b500 0x7f854c06d9c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.298+0000 7f8565a93640 1 --2- 172.21.15.16:0/1926518696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85600a84a0 0x7f85600a8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.298+0000 7f8565a93640 1 --2- 172.21.15.16:0/1926518696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85601563c0 0x7f8560156840 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.298+0000 7f8565a93640 1 --2- 172.21.15.16:0/1926518696 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85480039b0 0x7f8560145e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.298+0000 7f8565a93640 1 -- 172.21.15.16:0/1926518696 >> 172.21.15.16:0/1926518696 conn(0x7f856006aba0 msgr2=0x7f8560157a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.298+0000 7f8565a93640 1 -- 172.21.15.16:0/1926518696 shutdown_connections 2024-04-25T10:10:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.298+0000 7f8565a93640 1 -- 172.21.15.16:0/1926518696 wait complete. 2024-04-25T10:10:48.314 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:48.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.410+0000 7f63a57b7640 1 Processor -- start 2024-04-25T10:10:48.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.410+0000 7f63a57b7640 1 -- start start 2024-04-25T10:10:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.410+0000 7f63a57b7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a00b5b60 0x7f63a01687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.410+0000 7f63a57b7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a015e0c0 con 0x7f63a015dce0 2024-04-25T10:10:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.410+0000 7f63a57b7640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a00d2510 con 0x7f63a00d35b0 2024-04-25T10:10:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.410+0000 7f63a57b7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a00b7b10 con 0x7f63a00b5b60 2024-04-25T10:10:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639effd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f63a015dce0 0x7f63a00d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56754/0 (socket says 172.21.15.16:56754) 2024-04-25T10:10:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639effd640 1 -- 172.21.15.16:0/2780451321 learned_addr learned my addr 172.21.15.16:0/2780451321 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639f7fe640 1 --2- 172.21.15.16:0/2780451321 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a00b5b60 0x7f63a01687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639e7fc640 1 -- 172.21.15.16:0/2780451321 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3795237853 0 0) 0x7f63a015e0c0 con 0x7f63a015dce0 2024-04-25T10:10:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639e7fc640 1 -- 172.21.15.16:0/2780451321 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f638c003520 con 0x7f63a015dce0 2024-04-25T10:10:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639e7fc640 1 -- 172.21.15.16:0/2780451321 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2116782525 0 0) 0x7f638c003520 con 0x7f63a015dce0 2024-04-25T10:10:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639e7fc640 1 -- 172.21.15.16:0/2780451321 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a00b5b60 msgr2=0x7f63a01687f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639e7fc640 1 --2- 172.21.15.16:0/2780451321 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a00b5b60 0x7f63a01687f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639e7fc640 1 -- 172.21.15.16:0/2780451321 >> v1:172.21.15.186:6789/0 conn(0x7f63a00d35b0 legacy=0x7f63a00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639e7fc640 1 -- 172.21.15.16:0/2780451321 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a00d27f0 con 0x7f63a015dce0 2024-04-25T10:10:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639e7fc640 1 -- 172.21.15.16:0/2780451321 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f6390001670 con 0x7f63a015dce0 2024-04-25T10:10:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639e7fc640 1 -- 172.21.15.16:0/2780451321 >> v1:172.21.15.16:6789/0 conn(0x7f63a015dce0 legacy=0x7f63a00d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639e7fc640 1 --2- 172.21.15.16:0/2780451321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f638c0039e0 0x7f638c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639e7fc640 1 --2- 172.21.15.16:0/2780451321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f638c004390 0x7f638c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639e7fc640 1 --2- 172.21.15.16:0/2780451321 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a00b5b60 0x7f638c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639e7fc640 1 -- 172.21.15.16:0/2780451321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a00b7b10 con 0x7f638c0039e0 2024-04-25T10:10:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639e7fc640 1 -- 172.21.15.16:0/2780451321 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c00b9c0 con 0x7f638c004390 2024-04-25T10:10:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639e7fc640 1 -- 172.21.15.16:0/2780451321 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638c00bbc0 con 0x7f63a00b5b60 2024-04-25T10:10:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f639effd640 1 --2- 172.21.15.16:0/2780451321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f638c0039e0 0x7f638c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.411+0000 7f6397fff640 1 --2- 172.21.15.16:0/2780451321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f638c004390 0x7f638c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.412+0000 7f639f7fe640 1 --2- 172.21.15.16:0/2780451321 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a00b5b60 0x7f638c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.412+0000 7f639effd640 1 -- 172.21.15.16:0/2780451321 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a00b5b60 msgr2=0x7f638c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.412+0000 7f639effd640 1 --2- 172.21.15.16:0/2780451321 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a00b5b60 0x7f638c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.412+0000 7f639effd640 1 -- 172.21.15.16:0/2780451321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f638c004390 msgr2=0x7f638c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.412+0000 7f639effd640 1 --2- 172.21.15.16:0/2780451321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f638c004390 0x7f638c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.412+0000 7f639effd640 1 -- 172.21.15.16:0/2780451321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c003520 con 0x7f638c0039e0 2024-04-25T10:10:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.412+0000 7f639f7fe640 1 --2- 172.21.15.16:0/2780451321 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a00b5b60 0x7f638c0050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.412+0000 7f639effd640 1 --2- 172.21.15.16:0/2780451321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f638c0039e0 0x7f638c003dc0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f639000c240 tx=0x7f638c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b1961fed2a8081 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.413+0000 7f639e7fc640 1 -- 172.21.15.16:0/2780451321 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639002e070 con 0x7f638c0039e0 2024-04-25T10:10:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.413+0000 7f639e7fc640 1 -- 172.21.15.16:0/2780451321 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6390005af0 con 0x7f638c0039e0 2024-04-25T10:10:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.413+0000 7f639e7fc640 1 -- 172.21.15.16:0/2780451321 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390007d30 con 0x7f638c0039e0 2024-04-25T10:10:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.413+0000 7f63a57b7640 1 -- 172.21.15.16:0/2780451321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f638c0039e0 msgr2=0x7f638c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.413+0000 7f63a57b7640 1 --2- 172.21.15.16:0/2780451321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f638c0039e0 0x7f638c003dc0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f639000c240 tx=0x7f638c005830 comp rx=0 tx=0).stop 2024-04-25T10:10:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.413+0000 7f639f7fe640 1 -- 172.21.15.16:0/2780451321 reap_dead start 2024-04-25T10:10:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.413+0000 7f63a57b7640 1 -- 172.21.15.16:0/2780451321 shutdown_connections 2024-04-25T10:10:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.413+0000 7f63a57b7640 1 -- 172.21.15.16:0/2780451321 >> 172.21.15.16:0/2780451321 conn(0x7f63a0072900 msgr2=0x7f63a00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.413+0000 7f63a57b7640 1 -- 172.21.15.16:0/2780451321 shutdown_connections 2024-04-25T10:10:48.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.413+0000 7f63a57b7640 1 -- 172.21.15.16:0/2780451321 wait complete. 2024-04-25T10:10:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.414+0000 7f63a57b7640 1 Processor -- start 2024-04-25T10:10:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.414+0000 7f63a57b7640 1 -- start start 2024-04-25T10:10:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.415+0000 7f63a57b7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f638c0039e0 0x7f63a0157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.415+0000 7f63a57b7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a0158b00 0x7f63a01f6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.415+0000 7f63a57b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63a01f6b40 0x7f63a015e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.415+0000 7f63a57b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a00d3090 con 0x7f63a01f6b40 2024-04-25T10:10:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.415+0000 7f63a57b7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a00d2db0 con 0x7f638c0039e0 2024-04-25T10:10:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.415+0000 7f63a57b7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a00d2160 con 0x7f63a0158b00 2024-04-25T10:10:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.415+0000 7f639f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63a01f6b40 0x7f63a015e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.415+0000 7f639f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63a01f6b40 0x7f63a015e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57620/0 (socket says 172.21.15.16:57620) 2024-04-25T10:10:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.415+0000 7f639f7fe640 1 -- 172.21.15.16:0/684651868 learned_addr learned my addr 172.21.15.16:0/684651868 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.415+0000 7f6397fff640 1 --2- 172.21.15.16:0/684651868 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a0158b00 0x7f63a01f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.415+0000 7f639effd640 1 --2- 172.21.15.16:0/684651868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f638c0039e0 0x7f63a0157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.416+0000 7f639f7fe640 1 -- 172.21.15.16:0/684651868 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a0158b00 msgr2=0x7f63a01f6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.416+0000 7f639f7fe640 1 --2- 172.21.15.16:0/684651868 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a0158b00 0x7f63a01f6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.416+0000 7f639f7fe640 1 -- 172.21.15.16:0/684651868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f638c0039e0 msgr2=0x7f63a0157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.416+0000 7f639f7fe640 1 --2- 172.21.15.16:0/684651868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f638c0039e0 0x7f63a0157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.416+0000 7f639f7fe640 1 -- 172.21.15.16:0/684651868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a00cbb90 con 0x7f63a01f6b40 2024-04-25T10:10:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.416+0000 7f639f7fe640 1 --2- 172.21.15.16:0/684651868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63a01f6b40 0x7f63a015e0c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f63a008ce80 tx=0x7f6398009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.416+0000 7f63977fe640 1 -- 172.21.15.16:0/684651868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639801a030 con 0x7f63a01f6b40 2024-04-25T10:10:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.416+0000 7f63977fe640 1 -- 172.21.15.16:0/684651868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6398010e50 con 0x7f63a01f6b40 2024-04-25T10:10:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.416+0000 7f63a57b7640 1 -- 172.21.15.16:0/684651868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a0158fe0 con 0x7f63a01f6b40 2024-04-25T10:10:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.416+0000 7f63977fe640 1 -- 172.21.15.16:0/684651868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6398003bd0 con 0x7f63a01f6b40 2024-04-25T10:10:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.416+0000 7f63a57b7640 1 -- 172.21.15.16:0/684651868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a01592c0 con 0x7f63a01f6b40 2024-04-25T10:10:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.417+0000 7f63a57b7640 1 -- 172.21.15.16:0/684651868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a00749b0 con 0x7f63a01f6b40 2024-04-25T10:10:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.417+0000 7f63977fe640 1 -- 172.21.15.16:0/684651868 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6398005d10 con 0x7f63a01f6b40 2024-04-25T10:10:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.418+0000 7f63977fe640 1 --2- 172.21.15.16:0/684651868 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f638006b4d0 0x7f638006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.418+0000 7f63977fe640 1 -- 172.21.15.16:0/684651868 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f639808a890 con 0x7f63a01f6b40 2024-04-25T10:10:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.418+0000 7f639effd640 1 --2- 172.21.15.16:0/684651868 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f638006b4d0 0x7f638006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.419+0000 7f639effd640 1 --2- 172.21.15.16:0/684651868 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f638006b4d0 0x7f638006d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f639000c240 tx=0x7f639002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:48.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.422+0000 7f63977fe640 1 -- 172.21.15.16:0/684651868 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6398025020 con 0x7f63a01f6b40 2024-04-25T10:10:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.642+0000 7f63a57b7640 1 -- 172.21.15.16:0/684651868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63a015c3b0 con 0x7f63a01f6b40 2024-04-25T10:10:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.642+0000 7f63977fe640 1 -- 172.21.15.16:0/684651868 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f6398059000 con 0x7f63a01f6b40 2024-04-25T10:10:48.648 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:48.648 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6070,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2805428272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2805428272},{"type":"v1","addr":"172.21.15.186:6835","nonce":2805428272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1052667029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1052667029},{"type":"v1","addr":"172.21.15.186:6837","nonce":1052667029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2897154125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2897154125},{"type":"v1","addr":"172.21.15.16:6833","nonce":2897154125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:22.077032+0000","modified":"2024-04-25T10:10:14.390498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6636":{"gid":6636,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":17,"addr":"172.21.15.16:6835/1735811292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1735811292},{"type":"v1","addr":"172.21.15.16:6835","nonce":1735811292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:10:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 44 2024-04-25T10:10:48.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.644+0000 7f63a57b7640 1 -- 172.21.15.16:0/684651868 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f638006b4d0 msgr2=0x7f638006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.644+0000 7f63a57b7640 1 --2- 172.21.15.16:0/684651868 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f638006b4d0 0x7f638006d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f639000c240 tx=0x7f639002e000 comp rx=0 tx=0).stop 2024-04-25T10:10:48.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.644+0000 7f63a57b7640 1 -- 172.21.15.16:0/684651868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63a01f6b40 msgr2=0x7f63a015e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.644+0000 7f63a57b7640 1 --2- 172.21.15.16:0/684651868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63a01f6b40 0x7f63a015e0c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f63a008ce80 tx=0x7f6398009f90 comp rx=0 tx=0).stop 2024-04-25T10:10:48.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.645+0000 7f63a57b7640 1 -- 172.21.15.16:0/684651868 shutdown_connections 2024-04-25T10:10:48.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.645+0000 7f63a57b7640 1 --2- 172.21.15.16:0/684651868 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f638006b4d0 0x7f638006d990 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.645+0000 7f63a57b7640 1 --2- 172.21.15.16:0/684651868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63a01f6b40 0x7f63a015e0c0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.645+0000 7f63a57b7640 1 --2- 172.21.15.16:0/684651868 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a0158b00 0x7f63a01f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.645+0000 7f63a57b7640 1 --2- 172.21.15.16:0/684651868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f638c0039e0 0x7f63a0157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.645+0000 7f63a57b7640 1 -- 172.21.15.16:0/684651868 >> 172.21.15.16:0/684651868 conn(0x7f63a0072900 msgr2=0x7f63a00d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:48.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.645+0000 7f63a57b7640 1 -- 172.21.15.16:0/684651868 shutdown_connections 2024-04-25T10:10:48.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.645+0000 7f63a57b7640 1 -- 172.21.15.16:0/684651868 wait complete. 2024-04-25T10:10:48.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:48.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.757+0000 7ff6d454c640 1 Processor -- start 2024-04-25T10:10:48.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.757+0000 7ff6d454c640 1 -- start start 2024-04-25T10:10:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.757+0000 7ff6d454c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff6cc15ba10 0x7ff6cc099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.757+0000 7ff6d454c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6cc06c3c0 0x7ff6cc0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.757+0000 7ff6d454c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6cc06c900 con 0x7ff6cc099f80 2024-04-25T10:10:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.758+0000 7ff6d454c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6cc0cbe50 con 0x7ff6cc15ba10 2024-04-25T10:10:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.758+0000 7ff6d454c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc09a3d0 con 0x7ff6cc06c3c0 2024-04-25T10:10:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.758+0000 7ff6d2ae9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6cc06c3c0 0x7ff6cc0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.758+0000 7ff6d22e8640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff6cc15ba10 0x7ff6cc099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.758+0000 7ff6d1ae7640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7ff6cc099f80 0x7ff6cc06bcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:59894/0 (socket says 172.21.15.16:59894) 2024-04-25T10:10:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.758+0000 7ff6d2ae9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6cc06c3c0 0x7ff6cc0cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:57778/0 (socket says 172.21.15.16:57778) 2024-04-25T10:10:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.758+0000 7ff6d1ae7640 1 -- 172.21.15.16:0/1142212650 learned_addr learned my addr 172.21.15.16:0/1142212650 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.758+0000 7ff6d12e6640 1 -- 172.21.15.16:0/1142212650 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334290282 0 0) 0x7ff6cc06c900 con 0x7ff6cc099f80 2024-04-25T10:10:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.758+0000 7ff6d12e6640 1 -- 172.21.15.16:0/1142212650 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6b8003520 con 0x7ff6cc099f80 2024-04-25T10:10:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.758+0000 7ff6d2ae9640 1 -- 172.21.15.16:0/1142212650 >> v2:172.21.15.186:3300/0 conn(0x7ff6cc15ba10 msgr2=0x7ff6cc099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.758+0000 7ff6d2ae9640 1 --2- 172.21.15.16:0/1142212650 >> v2:172.21.15.186:3300/0 conn(0x7ff6cc15ba10 0x7ff6cc099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.758+0000 7ff6d2ae9640 1 -- 172.21.15.16:0/1142212650 >> v1:172.21.15.186:6789/0 conn(0x7ff6cc099f80 legacy=0x7ff6cc06bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.758+0000 7ff6d2ae9640 1 -- 172.21.15.16:0/1142212650 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc06cbe0 con 0x7ff6cc06c3c0 2024-04-25T10:10:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.758+0000 7ff6d22e8640 1 --2- 172.21.15.16:0/1142212650 >> v2:172.21.15.186:3300/0 conn(0x7ff6cc15ba10 0x7ff6cc099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:10:48.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.759+0000 7ff6d2ae9640 1 --2- 172.21.15.16:0/1142212650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6cc06c3c0 0x7ff6cc0cd220 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009d90 tx=0x7ff6c402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d7be8ab6de163f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:48.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.759+0000 7ff6d12e6640 1 -- 172.21.15.16:0/1142212650 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4007740 con 0x7ff6cc06c3c0 2024-04-25T10:10:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.759+0000 7ff6d12e6640 1 -- 172.21.15.16:0/1142212650 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c40078e0 con 0x7ff6cc06c3c0 2024-04-25T10:10:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.759+0000 7ff6d12e6640 1 -- 172.21.15.16:0/1142212650 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4007c90 con 0x7ff6cc06c3c0 2024-04-25T10:10:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.760+0000 7ff6d454c640 1 -- 172.21.15.16:0/1142212650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6cc06c3c0 msgr2=0x7ff6cc0cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.760+0000 7ff6d454c640 1 --2- 172.21.15.16:0/1142212650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6cc06c3c0 0x7ff6cc0cd220 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009d90 tx=0x7ff6c402a660 comp rx=0 tx=0).stop 2024-04-25T10:10:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.760+0000 7ff6d454c640 1 -- 172.21.15.16:0/1142212650 shutdown_connections 2024-04-25T10:10:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.760+0000 7ff6d454c640 1 --2- 172.21.15.16:0/1142212650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6cc06c3c0 0x7ff6cc0cd220 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.760+0000 7ff6d454c640 1 --2- 172.21.15.16:0/1142212650 >> v2:172.21.15.186:3300/0 conn(0x7ff6cc15ba10 0x7ff6cc099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.760+0000 7ff6d454c640 1 -- 172.21.15.16:0/1142212650 >> 172.21.15.16:0/1142212650 conn(0x7ff6cc0b32f0 msgr2=0x7ff6cc0b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.760+0000 7ff6d454c640 1 -- 172.21.15.16:0/1142212650 shutdown_connections 2024-04-25T10:10:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.760+0000 7ff6d454c640 1 -- 172.21.15.16:0/1142212650 wait complete. 2024-04-25T10:10:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.761+0000 7ff6d454c640 1 Processor -- start 2024-04-25T10:10:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.761+0000 7ff6d454c640 1 -- start start 2024-04-25T10:10:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.762+0000 7ff6d454c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6cc06c3c0 0x7ff6cc15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.762+0000 7ff6d454c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6cc099f80 0x7ff6cc1616d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.762+0000 7ff6d454c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6cc15ba10 0x7ff6cc161c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.762+0000 7ff6d454c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0cd760 con 0x7ff6cc099f80 2024-04-25T10:10:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.762+0000 7ff6d454c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0cfe80 con 0x7ff6cc15ba10 2024-04-25T10:10:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.762+0000 7ff6d454c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0cd8e0 con 0x7ff6cc06c3c0 2024-04-25T10:10:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.762+0000 7ff6d1ae7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6cc099f80 0x7ff6cc1616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.762+0000 7ff6d1ae7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6cc099f80 0x7ff6cc1616d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57632/0 (socket says 172.21.15.16:57632) 2024-04-25T10:10:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.762+0000 7ff6d1ae7640 1 -- 172.21.15.16:0/1900224079 learned_addr learned my addr 172.21.15.16:0/1900224079 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.762+0000 7ff6d2ae9640 1 --2- 172.21.15.16:0/1900224079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6cc15ba10 0x7ff6cc161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.762+0000 7ff6d22e8640 1 --2- 172.21.15.16:0/1900224079 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6cc06c3c0 0x7ff6cc15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.762+0000 7ff6d1ae7640 1 -- 172.21.15.16:0/1900224079 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6cc06c3c0 msgr2=0x7ff6cc15a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.762+0000 7ff6d1ae7640 1 --2- 172.21.15.16:0/1900224079 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6cc06c3c0 0x7ff6cc15a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.762+0000 7ff6d1ae7640 1 -- 172.21.15.16:0/1900224079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6cc15ba10 msgr2=0x7ff6cc161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.762+0000 7ff6d1ae7640 1 --2- 172.21.15.16:0/1900224079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6cc15ba10 0x7ff6cc161c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.762+0000 7ff6d1ae7640 1 -- 172.21.15.16:0/1900224079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc1f57f0 con 0x7ff6cc099f80 2024-04-25T10:10:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.763+0000 7ff6d1ae7640 1 --2- 172.21.15.16:0/1900224079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6cc099f80 0x7ff6cc1616d0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00c430 tx=0x7ff6bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.763+0000 7ff6c2ffd640 1 -- 172.21.15.16:0/1900224079 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc017070 con 0x7ff6cc099f80 2024-04-25T10:10:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.763+0000 7ff6c2ffd640 1 -- 172.21.15.16:0/1900224079 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6bc005ce0 con 0x7ff6cc099f80 2024-04-25T10:10:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.763+0000 7ff6d454c640 1 -- 172.21.15.16:0/1900224079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6cc1f5c10 con 0x7ff6cc099f80 2024-04-25T10:10:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.763+0000 7ff6c2ffd640 1 -- 172.21.15.16:0/1900224079 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc0072f0 con 0x7ff6cc099f80 2024-04-25T10:10:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.763+0000 7ff6d454c640 1 -- 172.21.15.16:0/1900224079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6cc1f5eb0 con 0x7ff6cc099f80 2024-04-25T10:10:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.764+0000 7ff6d454c640 1 -- 172.21.15.16:0/1900224079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6cc0cede0 con 0x7ff6cc099f80 2024-04-25T10:10:48.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.764+0000 7ff6c2ffd640 1 -- 172.21.15.16:0/1900224079 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff6bc01a070 con 0x7ff6cc099f80 2024-04-25T10:10:48.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.765+0000 7ff6c2ffd640 1 --2- 172.21.15.16:0/1900224079 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff6b006b4d0 0x7ff6b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.765+0000 7ff6c2ffd640 1 -- 172.21.15.16:0/1900224079 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7ff6bc0896a0 con 0x7ff6cc099f80 2024-04-25T10:10:48.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.765+0000 7ff6d22e8640 1 --2- 172.21.15.16:0/1900224079 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff6b006b4d0 0x7ff6b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.765+0000 7ff6d22e8640 1 --2- 172.21.15.16:0/1900224079 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff6b006b4d0 0x7ff6b006d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009560 tx=0x7ff6c8009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:48.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.769+0000 7ff6c2ffd640 1 -- 172.21.15.16:0/1900224079 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff6cc0cede0 con 0x7ff6cc099f80 2024-04-25T10:10:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.953+0000 7ff6d454c640 1 -- 172.21.15.16:0/1900224079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6cc09a570 con 0x7ff6cc099f80 2024-04-25T10:10:48.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.955+0000 7ff6c2ffd640 1 -- 172.21.15.16:0/1900224079 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16885 (secure 0 0 0) 0x7ff6bc057e30 con 0x7ff6cc099f80 2024-04-25T10:10:48.960 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:48.960 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:09:53.479005+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:20.361009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:21.048892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.19","mappings":[{"from":2,"to":7}]},{"pgid":"11.26","mappings":[{"from":1,"to":4}]},{"pgid":"11.27","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.186:6837/3292077067":"2024-04-26T10:09:31.374252+0000","172.21.15.186:6836/3292077067":"2024-04-26T10:09:31.374252+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.956+0000 7ff6d454c640 1 -- 172.21.15.16:0/1900224079 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff6b006b4d0 msgr2=0x7ff6b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.956+0000 7ff6d454c640 1 --2- 172.21.15.16:0/1900224079 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff6b006b4d0 0x7ff6b006d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009560 tx=0x7ff6c8009190 comp rx=0 tx=0).stop 2024-04-25T10:10:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.956+0000 7ff6d454c640 1 -- 172.21.15.16:0/1900224079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6cc099f80 msgr2=0x7ff6cc1616d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.956+0000 7ff6d454c640 1 --2- 172.21.15.16:0/1900224079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6cc099f80 0x7ff6cc1616d0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00c430 tx=0x7ff6bc00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.957+0000 7ff6d454c640 1 -- 172.21.15.16:0/1900224079 shutdown_connections 2024-04-25T10:10:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.957+0000 7ff6d454c640 1 --2- 172.21.15.16:0/1900224079 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff6b006b4d0 0x7ff6b006d990 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.957+0000 7ff6d454c640 1 --2- 172.21.15.16:0/1900224079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6cc15ba10 0x7ff6cc161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.957+0000 7ff6d454c640 1 --2- 172.21.15.16:0/1900224079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6cc099f80 0x7ff6cc1616d0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.957+0000 7ff6d454c640 1 --2- 172.21.15.16:0/1900224079 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff6cc06c3c0 0x7ff6cc15a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.957+0000 7ff6d454c640 1 -- 172.21.15.16:0/1900224079 >> 172.21.15.16:0/1900224079 conn(0x7ff6cc0b32f0 msgr2=0x7ff6cc15ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.957+0000 7ff6d454c640 1 -- 172.21.15.16:0/1900224079 shutdown_connections 2024-04-25T10:10:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:48.957+0000 7ff6d454c640 1 -- 172.21.15.16:0/1900224079 wait complete. 2024-04-25T10:10:48.973 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:10:49.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.070+0000 7f7a807f4640 1 Processor -- start 2024-04-25T10:10:49.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.070+0000 7f7a807f4640 1 -- start start 2024-04-25T10:10:49.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.071+0000 7f7a807f4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a781610b0 0x7f7a780b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.071+0000 7f7a807f4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a781618d0 0x7f7a780ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.071+0000 7f7a807f4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a78162040 con 0x7f7a780ced70 2024-04-25T10:10:49.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.071+0000 7f7a807f4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a780d1ea0 con 0x7f7a781618d0 2024-04-25T10:10:49.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.071+0000 7f7a807f4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a78162320 con 0x7f7a781610b0 2024-04-25T10:10:49.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.071+0000 7f7a7dd8f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a781618d0 0x7f7a780ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.071+0000 7f7a7ed91640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7a780ced70 0x7f7a7806c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56760/0 (socket says 172.21.15.16:56760) 2024-04-25T10:10:49.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.071+0000 7f7a7ed91640 1 -- 172.21.15.16:0/4029037176 learned_addr learned my addr 172.21.15.16:0/4029037176 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:49.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.071+0000 7f7a7dd8f640 1 -- 172.21.15.16:0/4029037176 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a781610b0 msgr2=0x7f7a780b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:49.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.071+0000 7f7a7dd8f640 1 --2- 172.21.15.16:0/4029037176 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a781610b0 0x7f7a780b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.071+0000 7f7a7dd8f640 1 -- 172.21.15.16:0/4029037176 >> v1:172.21.15.16:6789/0 conn(0x7f7a780ced70 legacy=0x7f7a7806c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.071+0000 7f7a7dd8f640 1 -- 172.21.15.16:0/4029037176 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a78162600 con 0x7f7a781618d0 2024-04-25T10:10:49.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.072+0000 7f7a7dd8f640 1 --2- 172.21.15.16:0/4029037176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a781618d0 0x7f7a780ce830 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f7a680097a0 tx=0x7f7a6802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9f3ecc8f2d2659 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.072+0000 7f7a7d58e640 1 -- 172.21.15.16:0/4029037176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68037070 con 0x7f7a781618d0 2024-04-25T10:10:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.072+0000 7f7a7d58e640 1 -- 172.21.15.16:0/4029037176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6802ae50 con 0x7f7a781618d0 2024-04-25T10:10:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.072+0000 7f7a7d58e640 1 -- 172.21.15.16:0/4029037176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68032660 con 0x7f7a781618d0 2024-04-25T10:10:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.072+0000 7f7a807f4640 1 -- 172.21.15.16:0/4029037176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a781618d0 msgr2=0x7f7a780ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.072+0000 7f7a807f4640 1 --2- 172.21.15.16:0/4029037176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a781618d0 0x7f7a780ce830 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f7a680097a0 tx=0x7f7a6802a100 comp rx=0 tx=0).stop 2024-04-25T10:10:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.073+0000 7f7a807f4640 1 -- 172.21.15.16:0/4029037176 shutdown_connections 2024-04-25T10:10:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.073+0000 7f7a807f4640 1 --2- 172.21.15.16:0/4029037176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a781618d0 0x7f7a780ce830 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.073+0000 7f7a807f4640 1 --2- 172.21.15.16:0/4029037176 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a781610b0 0x7f7a780b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.073+0000 7f7a807f4640 1 -- 172.21.15.16:0/4029037176 >> 172.21.15.16:0/4029037176 conn(0x7f7a780b5570 msgr2=0x7f7a780b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.073+0000 7f7a807f4640 1 -- 172.21.15.16:0/4029037176 shutdown_connections 2024-04-25T10:10:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.073+0000 7f7a807f4640 1 -- 172.21.15.16:0/4029037176 wait complete. 2024-04-25T10:10:49.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.074+0000 7f7a807f4640 1 Processor -- start 2024-04-25T10:10:49.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.074+0000 7f7a807f4640 1 -- start start 2024-04-25T10:10:49.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.074+0000 7f7a807f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a780ced70 0x7f7a781f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.074+0000 7f7a807f4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a781610b0 0x7f7a78152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.074+0000 7f7a807f4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a781618d0 0x7f7a781596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.074+0000 7f7a807f4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a78161d10 con 0x7f7a780ced70 2024-04-25T10:10:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.074+0000 7f7a807f4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780b6c00 con 0x7f7a781618d0 2024-04-25T10:10:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.074+0000 7f7a807f4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780d3620 con 0x7f7a781610b0 2024-04-25T10:10:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.075+0000 7f7a7e590640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a780ced70 0x7f7a781f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.075+0000 7f7a7e590640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a780ced70 0x7f7a781f3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57654/0 (socket says 172.21.15.16:57654) 2024-04-25T10:10:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.075+0000 7f7a7e590640 1 -- 172.21.15.16:0/352556742 learned_addr learned my addr 172.21.15.16:0/352556742 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.075+0000 7f7a7dd8f640 1 --2- 172.21.15.16:0/352556742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a781610b0 0x7f7a78152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.075+0000 7f7a7ed91640 1 --2- 172.21.15.16:0/352556742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a781618d0 0x7f7a781596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.075+0000 7f7a7e590640 1 -- 172.21.15.16:0/352556742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a781610b0 msgr2=0x7f7a78152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.075+0000 7f7a7e590640 1 --2- 172.21.15.16:0/352556742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a781610b0 0x7f7a78152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.075+0000 7f7a7e590640 1 -- 172.21.15.16:0/352556742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a781618d0 msgr2=0x7f7a781596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.075+0000 7f7a7e590640 1 --2- 172.21.15.16:0/352556742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a781618d0 0x7f7a781596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.075+0000 7f7a7e590640 1 -- 172.21.15.16:0/352556742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a781f4240 con 0x7f7a780ced70 2024-04-25T10:10:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.075+0000 7f7a7ed91640 1 --2- 172.21.15.16:0/352556742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a781618d0 0x7f7a781596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.075+0000 7f7a7dd8f640 1 --2- 172.21.15.16:0/352556742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a781610b0 0x7f7a78152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.075+0000 7f7a7e590640 1 --2- 172.21.15.16:0/352556742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a780ced70 0x7f7a781f3d00 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400b4a0 tx=0x7f7a7400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.075+0000 7f7a6effd640 1 -- 172.21.15.16:0/352556742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74014070 con 0x7f7a780ced70 2024-04-25T10:10:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.075+0000 7f7a6effd640 1 -- 172.21.15.16:0/352556742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a74010610 con 0x7f7a780ced70 2024-04-25T10:10:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.075+0000 7f7a6effd640 1 -- 172.21.15.16:0/352556742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74010a00 con 0x7f7a780ced70 2024-04-25T10:10:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.075+0000 7f7a807f4640 1 -- 172.21.15.16:0/352556742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7815a560 con 0x7f7a780ced70 2024-04-25T10:10:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.075+0000 7f7a807f4640 1 -- 172.21.15.16:0/352556742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7815ab20 con 0x7f7a780ced70 2024-04-25T10:10:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.076+0000 7f7a807f4640 1 -- 172.21.15.16:0/352556742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a780750b0 con 0x7f7a780ced70 2024-04-25T10:10:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.077+0000 7f7a6effd640 1 -- 172.21.15.16:0/352556742 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a74017070 con 0x7f7a780ced70 2024-04-25T10:10:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.077+0000 7f7a6effd640 1 --2- 172.21.15.16:0/352556742 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a5c06b4d0 0x7f7a5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.077+0000 7f7a6effd640 1 -- 172.21.15.16:0/352556742 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f7a74089d00 con 0x7f7a780ced70 2024-04-25T10:10:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.077+0000 7f7a7dd8f640 1 --2- 172.21.15.16:0/352556742 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a5c06b4d0 0x7f7a5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.078+0000 7f7a7dd8f640 1 --2- 172.21.15.16:0/352556742 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a5c06b4d0 0x7f7a5c06d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7a68034000 tx=0x7f7a6802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.081+0000 7f7a6effd640 1 -- 172.21.15.16:0/352556742 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a74058ae0 con 0x7f7a780ced70 2024-04-25T10:10:49.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.299+0000 7f7a807f4640 1 -- 172.21.15.16:0/352556742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7a780d1770 con 0x7f7a780ced70 2024-04-25T10:10:49.305 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-04-25T10:10:49.299+0000 7ff13f9bb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:10:49.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.301+0000 7f7a6effd640 1 -- 172.21.15.16:0/352556742 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7a740583e0 con 0x7f7a780ced70 2024-04-25T10:10:49.306 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:10:49.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.303+0000 7f7a807f4640 1 -- 172.21.15.16:0/352556742 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a5c06b4d0 msgr2=0x7f7a5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.303+0000 7f7a807f4640 1 --2- 172.21.15.16:0/352556742 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a5c06b4d0 0x7f7a5c06d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7a68034000 tx=0x7f7a6802e000 comp rx=0 tx=0).stop 2024-04-25T10:10:49.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.303+0000 7f7a807f4640 1 -- 172.21.15.16:0/352556742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a780ced70 msgr2=0x7f7a781f3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.303+0000 7f7a807f4640 1 --2- 172.21.15.16:0/352556742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a780ced70 0x7f7a781f3d00 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400b4a0 tx=0x7f7a7400b960 comp rx=0 tx=0).stop 2024-04-25T10:10:49.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.304+0000 7f7a807f4640 1 -- 172.21.15.16:0/352556742 shutdown_connections 2024-04-25T10:10:49.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.304+0000 7f7a807f4640 1 --2- 172.21.15.16:0/352556742 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a5c06b4d0 0x7f7a5c06d990 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.304+0000 7f7a807f4640 1 --2- 172.21.15.16:0/352556742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a781618d0 0x7f7a781596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.304+0000 7f7a807f4640 1 --2- 172.21.15.16:0/352556742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a781610b0 0x7f7a78152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.304+0000 7f7a807f4640 1 --2- 172.21.15.16:0/352556742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a780ced70 0x7f7a781f3d00 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.304+0000 7f7a807f4640 1 -- 172.21.15.16:0/352556742 >> 172.21.15.16:0/352556742 conn(0x7f7a780b5570 msgr2=0x7f7a780b6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:49.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.304+0000 7f7a807f4640 1 -- 172.21.15.16:0/352556742 shutdown_connections 2024-04-25T10:10:49.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.304+0000 7f7a807f4640 1 -- 172.21.15.16:0/352556742 wait complete. 2024-04-25T10:10:49.313 INFO:tasks.ceph.mds.a.smithi016.stderr: -3394> 2024-04-25T10:09:39.695+0000 7f81c84b3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:10:49.321 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:10:49.341 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-04-25T10:10:49.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.420+0000 7f318fb89640 1 Processor -- start 2024-04-25T10:10:49.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.420+0000 7f318fb89640 1 -- start start 2024-04-25T10:10:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.421+0000 7f318fb89640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31880d3b90 0x7f31880d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.421+0000 7f318fb89640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f31880d0f60 0x7f31880b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.421+0000 7f318fb89640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31880d4090 con 0x7f318815dca0 2024-04-25T10:10:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.421+0000 7f318fb89640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31880b6b20 con 0x7f31880d0f60 2024-04-25T10:10:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.421+0000 7f318fb89640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31880d4370 con 0x7f31880d3b90 2024-04-25T10:10:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.421+0000 7f318e126640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f31880d0f60 0x7f31880b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.421+0000 7f318d124640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31880d3b90 0x7f31880d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.421+0000 7f318e126640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f31880d0f60 0x7f31880b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:38116/0 (socket says 172.21.15.16:38116) 2024-04-25T10:10:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.421+0000 7f318d925640 1 --1- 172.21.15.16:0/3020874287 >> v1:172.21.15.186:6789/0 conn(0x7f318815dca0 0x7f31880ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:59916/0 (socket says 172.21.15.16:59916) 2024-04-25T10:10:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.421+0000 7f318e126640 1 -- 172.21.15.16:0/3020874287 learned_addr learned my addr 172.21.15.16:0/3020874287 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.421+0000 7f318d124640 1 -- 172.21.15.16:0/3020874287 >> v2:172.21.15.186:3300/0 conn(0x7f31880d0f60 msgr2=0x7f31880b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.421+0000 7f318d124640 1 --2- 172.21.15.16:0/3020874287 >> v2:172.21.15.186:3300/0 conn(0x7f31880d0f60 0x7f31880b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.421+0000 7f318d124640 1 -- 172.21.15.16:0/3020874287 >> v1:172.21.15.186:6789/0 conn(0x7f318815dca0 legacy=0x7f31880ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.421+0000 7f318d124640 1 -- 172.21.15.16:0/3020874287 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31880d4650 con 0x7f31880d3b90 2024-04-25T10:10:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.421+0000 7f318e126640 1 --2- 172.21.15.16:0/3020874287 >> v2:172.21.15.186:3300/0 conn(0x7f31880d0f60 0x7f31880b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:10:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.422+0000 7f318d124640 1 --2- 172.21.15.16:0/3020874287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31880d3b90 0x7f31880d0a20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f31780097a0 tx=0x7f317802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a31a5dd7a3a71b49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.422+0000 7f318c923640 1 -- 172.21.15.16:0/3020874287 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178037070 con 0x7f31880d3b90 2024-04-25T10:10:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.422+0000 7f318c923640 1 -- 172.21.15.16:0/3020874287 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3178032440 con 0x7f31880d3b90 2024-04-25T10:10:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.422+0000 7f318c923640 1 -- 172.21.15.16:0/3020874287 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178032810 con 0x7f31880d3b90 2024-04-25T10:10:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.423+0000 7f318fb89640 1 -- 172.21.15.16:0/3020874287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31880d3b90 msgr2=0x7f31880d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.423+0000 7f318fb89640 1 --2- 172.21.15.16:0/3020874287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31880d3b90 0x7f31880d0a20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f31780097a0 tx=0x7f317802a180 comp rx=0 tx=0).stop 2024-04-25T10:10:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.423+0000 7f318fb89640 1 -- 172.21.15.16:0/3020874287 shutdown_connections 2024-04-25T10:10:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.423+0000 7f318fb89640 1 --2- 172.21.15.16:0/3020874287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31880d3b90 0x7f31880d0a20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.423+0000 7f318fb89640 1 --2- 172.21.15.16:0/3020874287 >> v2:172.21.15.186:3300/0 conn(0x7f31880d0f60 0x7f31880b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.423+0000 7f318fb89640 1 -- 172.21.15.16:0/3020874287 >> 172.21.15.16:0/3020874287 conn(0x7f31880726c0 msgr2=0x7f3188072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.423+0000 7f318fb89640 1 -- 172.21.15.16:0/3020874287 shutdown_connections 2024-04-25T10:10:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.423+0000 7f318fb89640 1 -- 172.21.15.16:0/3020874287 wait complete. 2024-04-25T10:10:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.424+0000 7f318fb89640 1 Processor -- start 2024-04-25T10:10:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.424+0000 7f318fb89640 1 -- start start 2024-04-25T10:10:49.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.425+0000 7f318fb89640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31880d0f60 0x7f31881f4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.425+0000 7f318fb89640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31880d3b90 0x7f31880cce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.425+0000 7f318fb89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f318815dca0 0x7f31881f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.425+0000 7f318fb89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318815e080 con 0x7f318815dca0 2024-04-25T10:10:49.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.425+0000 7f318fb89640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31880cf050 con 0x7f31880d0f60 2024-04-25T10:10:49.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.425+0000 7f318fb89640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318815e200 con 0x7f31880d3b90 2024-04-25T10:10:49.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.425+0000 7f318e126640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f318815dca0 0x7f31881f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.425+0000 7f318e126640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f318815dca0 0x7f31881f4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57686/0 (socket says 172.21.15.16:57686) 2024-04-25T10:10:49.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.425+0000 7f318e126640 1 -- 172.21.15.16:0/3495389842 learned_addr learned my addr 172.21.15.16:0/3495389842 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:49.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.425+0000 7f318d925640 1 --2- 172.21.15.16:0/3495389842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31880d0f60 0x7f31881f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.425+0000 7f318d124640 1 --2- 172.21.15.16:0/3495389842 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31880d3b90 0x7f31880cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.425+0000 7f318e126640 1 -- 172.21.15.16:0/3495389842 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31880d3b90 msgr2=0x7f31880cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.425+0000 7f318e126640 1 --2- 172.21.15.16:0/3495389842 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31880d3b90 0x7f31880cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.425+0000 7f318e126640 1 -- 172.21.15.16:0/3495389842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31880d0f60 msgr2=0x7f31881f4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.425+0000 7f318e126640 1 --2- 172.21.15.16:0/3495389842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31880d0f60 0x7f31881f4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.425+0000 7f318e126640 1 -- 172.21.15.16:0/3495389842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31880cbdf0 con 0x7f318815dca0 2024-04-25T10:10:49.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.426+0000 7f318e126640 1 --2- 172.21.15.16:0/3495389842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f318815dca0 0x7f31881f4bd0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f318000efc0 tx=0x7f318000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:49.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.426+0000 7f317e7fc640 1 -- 172.21.15.16:0/3495389842 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318000cbe0 con 0x7f318815dca0 2024-04-25T10:10:49.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.426+0000 7f317e7fc640 1 -- 172.21.15.16:0/3495389842 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318000cd80 con 0x7f318815dca0 2024-04-25T10:10:49.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.426+0000 7f318fb89640 1 -- 172.21.15.16:0/3495389842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31880cc0d0 con 0x7f318815dca0 2024-04-25T10:10:49.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.426+0000 7f317e7fc640 1 -- 172.21.15.16:0/3495389842 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180003bb0 con 0x7f318815dca0 2024-04-25T10:10:49.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.426+0000 7f318fb89640 1 -- 172.21.15.16:0/3495389842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31880cc690 con 0x7f318815dca0 2024-04-25T10:10:49.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.427+0000 7f318fb89640 1 -- 172.21.15.16:0/3495389842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31880750b0 con 0x7f318815dca0 2024-04-25T10:10:49.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.427+0000 7f317e7fc640 1 -- 172.21.15.16:0/3495389842 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f318001b030 con 0x7f318815dca0 2024-04-25T10:10:49.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.428+0000 7f317e7fc640 1 --2- 172.21.15.16:0/3495389842 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f316c06b4d0 0x7f316c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.428+0000 7f317e7fc640 1 -- 172.21.15.16:0/3495389842 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5968+0+0 (secure 0 0 0) 0x7f318008a6e0 con 0x7f318815dca0 2024-04-25T10:10:49.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.428+0000 7f318d925640 1 --2- 172.21.15.16:0/3495389842 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f316c06b4d0 0x7f316c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.428+0000 7f318d925640 1 --2- 172.21.15.16:0/3495389842 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f316c06b4d0 0x7f316c06d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3184000f60 tx=0x7f3184007ce0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:49.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.432+0000 7f317e7fc640 1 -- 172.21.15.16:0/3495389842 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3180022020 con 0x7f318815dca0 2024-04-25T10:10:49.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:49.652+0000 7f318fb89640 1 -- 172.21.15.16:0/3495389842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f318806cab0 con 0x7f318815dca0 2024-04-25T10:10:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.302+0000 7f317e7fc640 1 -- 172.21.15.16:0/3495389842 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3180010040 con 0x7f318815dca0 2024-04-25T10:10:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.305+0000 7f318fb89640 1 -- 172.21.15.16:0/3495389842 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f316c06b4d0 msgr2=0x7f316c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.305+0000 7f318fb89640 1 --2- 172.21.15.16:0/3495389842 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f316c06b4d0 0x7f316c06d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3184000f60 tx=0x7f3184007ce0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.305+0000 7f318fb89640 1 -- 172.21.15.16:0/3495389842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f318815dca0 msgr2=0x7f31881f4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.305+0000 7f318fb89640 1 --2- 172.21.15.16:0/3495389842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f318815dca0 0x7f31881f4bd0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f318000efc0 tx=0x7f318000c490 comp rx=0 tx=0).stop 2024-04-25T10:10:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.305+0000 7f318fb89640 1 -- 172.21.15.16:0/3495389842 shutdown_connections 2024-04-25T10:10:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.305+0000 7f318fb89640 1 --2- 172.21.15.16:0/3495389842 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f316c06b4d0 0x7f316c06d990 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.305+0000 7f318fb89640 1 --2- 172.21.15.16:0/3495389842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f318815dca0 0x7f31881f4bd0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.305+0000 7f318fb89640 1 --2- 172.21.15.16:0/3495389842 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31880d3b90 0x7f31880cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.305+0000 7f318fb89640 1 --2- 172.21.15.16:0/3495389842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31880d0f60 0x7f31881f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.305+0000 7f318fb89640 1 -- 172.21.15.16:0/3495389842 >> 172.21.15.16:0/3495389842 conn(0x7f31880726c0 msgr2=0x7f31880b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.305+0000 7f318fb89640 1 -- 172.21.15.16:0/3495389842 shutdown_connections 2024-04-25T10:10:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.305+0000 7f318fb89640 1 -- 172.21.15.16:0/3495389842 wait complete. 2024-04-25T10:10:50.324 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:10:50.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.473+0000 7fcb5ab68640 1 Processor -- start 2024-04-25T10:10:50.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.473+0000 7fcb5ab68640 1 -- start start 2024-04-25T10:10:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.474+0000 7fcb5ab68640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb54161090 0x7fcb54161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.474+0000 7fcb5ab68640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb54161f80 con 0x7fcb540ceeb0 2024-04-25T10:10:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.474+0000 7fcb5ab68640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb54162260 con 0x7fcb540b9720 2024-04-25T10:10:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.474+0000 7fcb5ab68640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb540d21a0 con 0x7fcb54161090 2024-04-25T10:10:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.474+0000 7fcb59105640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcb540ceeb0 0x7fcb540d2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56782/0 (socket says 172.21.15.16:56782) 2024-04-25T10:10:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.474+0000 7fcb59105640 1 -- 172.21.15.16:0/911154814 learned_addr learned my addr 172.21.15.16:0/911154814 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.474+0000 7fcb58904640 1 --2- 172.21.15.16:0/911154814 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb54161090 0x7fcb54161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.474+0000 7fcb537fe640 1 -- 172.21.15.16:0/911154814 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982628916 0 0) 0x7fcb54161f80 con 0x7fcb540ceeb0 2024-04-25T10:10:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.474+0000 7fcb537fe640 1 -- 172.21.15.16:0/911154814 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb44003520 con 0x7fcb540ceeb0 2024-04-25T10:10:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.474+0000 7fcb537fe640 1 -- 172.21.15.16:0/911154814 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3976841424 0 0) 0x7fcb54162260 con 0x7fcb540b9720 2024-04-25T10:10:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.474+0000 7fcb537fe640 1 -- 172.21.15.16:0/911154814 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb54161f80 con 0x7fcb540b9720 2024-04-25T10:10:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.474+0000 7fcb537fe640 1 -- 172.21.15.16:0/911154814 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 332680123 0 0) 0x7fcb44003520 con 0x7fcb540ceeb0 2024-04-25T10:10:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.474+0000 7fcb537fe640 1 -- 172.21.15.16:0/911154814 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb54161090 msgr2=0x7fcb54161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.474+0000 7fcb537fe640 1 --2- 172.21.15.16:0/911154814 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb54161090 0x7fcb54161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb537fe640 1 -- 172.21.15.16:0/911154814 >> v1:172.21.15.186:6789/0 conn(0x7fcb540b9720 legacy=0x7fcb540ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb537fe640 1 -- 172.21.15.16:0/911154814 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb54162540 con 0x7fcb540ceeb0 2024-04-25T10:10:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb537fe640 1 -- 172.21.15.16:0/911154814 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fcb4c0019f0 con 0x7fcb540ceeb0 2024-04-25T10:10:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb58904640 1 --2- 172.21.15.16:0/911154814 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb54161090 0x7fcb54161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:10:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb537fe640 1 -- 172.21.15.16:0/911154814 >> v1:172.21.15.16:6789/0 conn(0x7fcb540ceeb0 legacy=0x7fcb540d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb537fe640 1 --2- 172.21.15.16:0/911154814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb44003b70 0x7fcb44003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb537fe640 1 --2- 172.21.15.16:0/911154814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb44004800 0x7fcb44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb537fe640 1 --2- 172.21.15.16:0/911154814 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb54161090 0x7fcb440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb537fe640 1 -- 172.21.15.16:0/911154814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb540d21a0 con 0x7fcb44004800 2024-04-25T10:10:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb537fe640 1 -- 172.21.15.16:0/911154814 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4400ba70 con 0x7fcb44003b70 2024-04-25T10:10:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb537fe640 1 -- 172.21.15.16:0/911154814 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4400bc20 con 0x7fcb54161090 2024-04-25T10:10:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb53fff640 1 --2- 172.21.15.16:0/911154814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb44004800 0x7fcb44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb58904640 1 --2- 172.21.15.16:0/911154814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb44003b70 0x7fcb44003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb59105640 1 --2- 172.21.15.16:0/911154814 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb54161090 0x7fcb440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb53fff640 1 -- 172.21.15.16:0/911154814 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb54161090 msgr2=0x7fcb440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb53fff640 1 --2- 172.21.15.16:0/911154814 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb54161090 0x7fcb440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb53fff640 1 -- 172.21.15.16:0/911154814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb44003b70 msgr2=0x7fcb44003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb53fff640 1 --2- 172.21.15.16:0/911154814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb44003b70 0x7fcb44003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.475+0000 7fcb53fff640 1 -- 172.21.15.16:0/911154814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb44003520 con 0x7fcb44004800 2024-04-25T10:10:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.476+0000 7fcb53fff640 1 --2- 172.21.15.16:0/911154814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb44004800 0x7fcb44004c80 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fcb400018a0 tx=0x7fcb44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2863a4dc2a2ae852 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.476+0000 7fcb537fe640 1 -- 172.21.15.16:0/911154814 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb400014f0 con 0x7fcb44004800 2024-04-25T10:10:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.476+0000 7fcb537fe640 1 -- 172.21.15.16:0/911154814 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb400052d0 con 0x7fcb44004800 2024-04-25T10:10:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.476+0000 7fcb537fe640 1 -- 172.21.15.16:0/911154814 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4002ac40 con 0x7fcb44004800 2024-04-25T10:10:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.476+0000 7fcb5ab68640 1 -- 172.21.15.16:0/911154814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb44004800 msgr2=0x7fcb44004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.476+0000 7fcb5ab68640 1 --2- 172.21.15.16:0/911154814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb44004800 0x7fcb44004c80 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fcb400018a0 tx=0x7fcb44005830 comp rx=0 tx=0).stop 2024-04-25T10:10:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.476+0000 7fcb59105640 1 -- 172.21.15.16:0/911154814 reap_dead start 2024-04-25T10:10:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.477+0000 7fcb5ab68640 1 -- 172.21.15.16:0/911154814 shutdown_connections 2024-04-25T10:10:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.477+0000 7fcb5ab68640 1 -- 172.21.15.16:0/911154814 >> 172.21.15.16:0/911154814 conn(0x7fcb540b55e0 msgr2=0x7fcb540b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.477+0000 7fcb5ab68640 1 -- 172.21.15.16:0/911154814 shutdown_connections 2024-04-25T10:10:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.477+0000 7fcb5ab68640 1 -- 172.21.15.16:0/911154814 wait complete. 2024-04-25T10:10:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.478+0000 7fcb5ab68640 1 Processor -- start 2024-04-25T10:10:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.478+0000 7fcb5ab68640 1 -- start start 2024-04-25T10:10:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.479+0000 7fcb5ab68640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb44004800 0x7fcb540c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.479+0000 7fcb5ab68640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb5415be50 0x7fcb540c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.479+0000 7fcb5ab68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5415efa0 0x7fcb5415f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.479+0000 7fcb5ab68640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb54161730 con 0x7fcb5415efa0 2024-04-25T10:10:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.479+0000 7fcb5ab68640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb54161470 con 0x7fcb5415be50 2024-04-25T10:10:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.479+0000 7fcb5ab68640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb541618b0 con 0x7fcb44004800 2024-04-25T10:10:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.479+0000 7fcb59105640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5415efa0 0x7fcb5415f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.479+0000 7fcb59105640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5415efa0 0x7fcb5415f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57700/0 (socket says 172.21.15.16:57700) 2024-04-25T10:10:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.479+0000 7fcb59105640 1 -- 172.21.15.16:0/528221609 learned_addr learned my addr 172.21.15.16:0/528221609 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.479+0000 7fcb53fff640 1 --2- 172.21.15.16:0/528221609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb5415be50 0x7fcb540c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.479+0000 7fcb58904640 1 --2- 172.21.15.16:0/528221609 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb44004800 0x7fcb540c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.479+0000 7fcb59105640 1 -- 172.21.15.16:0/528221609 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb44004800 msgr2=0x7fcb540c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.479+0000 7fcb59105640 1 --2- 172.21.15.16:0/528221609 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb44004800 0x7fcb540c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.479+0000 7fcb59105640 1 -- 172.21.15.16:0/528221609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb5415be50 msgr2=0x7fcb540c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.479+0000 7fcb59105640 1 --2- 172.21.15.16:0/528221609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb5415be50 0x7fcb540c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.479+0000 7fcb59105640 1 -- 172.21.15.16:0/528221609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb541f3bd0 con 0x7fcb5415efa0 2024-04-25T10:10:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.479+0000 7fcb53fff640 1 --2- 172.21.15.16:0/528221609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb5415be50 0x7fcb540c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.479+0000 7fcb59105640 1 --2- 172.21.15.16:0/528221609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5415efa0 0x7fcb5415f420 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fcb540d1210 tx=0x7fcb4c007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.480+0000 7fcb517fa640 1 -- 172.21.15.16:0/528221609 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c00ab90 con 0x7fcb5415efa0 2024-04-25T10:10:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.480+0000 7fcb517fa640 1 -- 172.21.15.16:0/528221609 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c019070 con 0x7fcb5415efa0 2024-04-25T10:10:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.480+0000 7fcb5ab68640 1 -- 172.21.15.16:0/528221609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb540c8440 con 0x7fcb5415efa0 2024-04-25T10:10:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.480+0000 7fcb517fa640 1 -- 172.21.15.16:0/528221609 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c0133f0 con 0x7fcb5415efa0 2024-04-25T10:10:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.480+0000 7fcb5ab68640 1 -- 172.21.15.16:0/528221609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb540c8a00 con 0x7fcb5415efa0 2024-04-25T10:10:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.481+0000 7fcb5ab68640 1 -- 172.21.15.16:0/528221609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb540750b0 con 0x7fcb5415efa0 2024-04-25T10:10:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.481+0000 7fcb517fa640 1 -- 172.21.15.16:0/528221609 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb4c013590 con 0x7fcb5415efa0 2024-04-25T10:10:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.482+0000 7fcb517fa640 1 --2- 172.21.15.16:0/528221609 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb3806b500 0x7fcb3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.482+0000 7fcb517fa640 1 -- 172.21.15.16:0/528221609 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5968+0+0 (secure 0 0 0) 0x7fcb4c08a9b0 con 0x7fcb5415efa0 2024-04-25T10:10:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.482+0000 7fcb58904640 1 --2- 172.21.15.16:0/528221609 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb3806b500 0x7fcb3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.482+0000 7fcb58904640 1 --2- 172.21.15.16:0/528221609 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb3806b500 0x7fcb3806d9c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009560 tx=0x7fcb48008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.486+0000 7fcb517fa640 1 -- 172.21.15.16:0/528221609 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb4c0596a0 con 0x7fcb5415efa0 2024-04-25T10:10:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:50.670+0000 7fcb5ab68640 1 -- 172.21.15.16:0/528221609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcb540d1700 con 0x7fcb5415efa0 2024-04-25T10:10:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.604+0000 7fcb517fa640 1 -- 172.21.15.16:0/528221609 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v77) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcb4c058fa0 con 0x7fcb5415efa0 2024-04-25T10:10:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:10:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.609+0000 7fcb5ab68640 1 -- 172.21.15.16:0/528221609 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb3806b500 msgr2=0x7fcb3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.609+0000 7fcb5ab68640 1 --2- 172.21.15.16:0/528221609 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb3806b500 0x7fcb3806d9c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009560 tx=0x7fcb48008040 comp rx=0 tx=0).stop 2024-04-25T10:10:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.609+0000 7fcb5ab68640 1 -- 172.21.15.16:0/528221609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5415efa0 msgr2=0x7fcb5415f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.609+0000 7fcb5ab68640 1 --2- 172.21.15.16:0/528221609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5415efa0 0x7fcb5415f420 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fcb540d1210 tx=0x7fcb4c007bd0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.612+0000 7fcb5ab68640 1 -- 172.21.15.16:0/528221609 shutdown_connections 2024-04-25T10:10:51.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.612+0000 7fcb5ab68640 1 --2- 172.21.15.16:0/528221609 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb3806b500 0x7fcb3806d9c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.612+0000 7fcb5ab68640 1 --2- 172.21.15.16:0/528221609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5415efa0 0x7fcb5415f420 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.612+0000 7fcb5ab68640 1 --2- 172.21.15.16:0/528221609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb5415be50 0x7fcb540c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.612+0000 7fcb5ab68640 1 --2- 172.21.15.16:0/528221609 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcb44004800 0x7fcb540c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.612+0000 7fcb5ab68640 1 -- 172.21.15.16:0/528221609 >> 172.21.15.16:0/528221609 conn(0x7fcb540b55e0 msgr2=0x7fcb540b8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:51.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.612+0000 7fcb5ab68640 1 -- 172.21.15.16:0/528221609 shutdown_connections 2024-04-25T10:10:51.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.612+0000 7fcb5ab68640 1 -- 172.21.15.16:0/528221609 wait complete. 2024-04-25T10:10:51.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:10:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.737+0000 7f3f20d22640 1 Processor -- start 2024-04-25T10:10:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.738+0000 7f3f20d22640 1 -- start start 2024-04-25T10:10:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.738+0000 7f3f20d22640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3f1c15dc50 0x7f3f1c070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.738+0000 7f3f20d22640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1c162690 con 0x7f3f1c0cf600 2024-04-25T10:10:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.738+0000 7f3f20d22640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1c06c220 con 0x7f3f1c0cd940 2024-04-25T10:10:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.738+0000 7f3f20d22640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1c0cfe00 con 0x7f3f1c15dc50 2024-04-25T10:10:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.739+0000 7f3f1ad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3f1c0cf600 0x7f3f1c161f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56788/0 (socket says 172.21.15.16:56788) 2024-04-25T10:10:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.739+0000 7f3f1ad9d640 1 -- 172.21.15.16:0/3815172453 learned_addr learned my addr 172.21.15.16:0/3815172453 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:51.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.739+0000 7f3f1a59c640 1 --2- 172.21.15.16:0/3815172453 >> v2:172.21.15.186:3300/0 conn(0x7f3f1c15dc50 0x7f3f1c070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.739+0000 7f3f1959a640 1 -- 172.21.15.16:0/3815172453 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4243177294 0 0) 0x7f3f1c162690 con 0x7f3f1c0cf600 2024-04-25T10:10:51.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.739+0000 7f3f1959a640 1 -- 172.21.15.16:0/3815172453 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f08003520 con 0x7f3f1c0cf600 2024-04-25T10:10:51.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.739+0000 7f3f1959a640 1 -- 172.21.15.16:0/3815172453 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2412570349 0 0) 0x7f3f08003520 con 0x7f3f1c0cf600 2024-04-25T10:10:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.739+0000 7f3f1959a640 1 -- 172.21.15.16:0/3815172453 >> v2:172.21.15.186:3300/0 conn(0x7f3f1c15dc50 msgr2=0x7f3f1c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.739+0000 7f3f1959a640 1 --2- 172.21.15.16:0/3815172453 >> v2:172.21.15.186:3300/0 conn(0x7f3f1c15dc50 0x7f3f1c070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.739+0000 7f3f1959a640 1 -- 172.21.15.16:0/3815172453 >> v1:172.21.15.186:6789/0 conn(0x7f3f1c0cd940 legacy=0x7f3f1c161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.739+0000 7f3f1959a640 1 -- 172.21.15.16:0/3815172453 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c0cec10 con 0x7f3f1c0cf600 2024-04-25T10:10:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.739+0000 7f3f1959a640 1 -- 172.21.15.16:0/3815172453 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f3f0c0019f0 con 0x7f3f1c0cf600 2024-04-25T10:10:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.739+0000 7f3f1959a640 1 -- 172.21.15.16:0/3815172453 >> v1:172.21.15.16:6789/0 conn(0x7f3f1c0cf600 legacy=0x7f3f1c161f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.739+0000 7f3f1959a640 1 --2- 172.21.15.16:0/3815172453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f080039b0 0x7f3f08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.739+0000 7f3f1959a640 1 --2- 172.21.15.16:0/3815172453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f08004800 0x7f3f08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.739+0000 7f3f1959a640 1 --2- 172.21.15.16:0/3815172453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f080051c0 0x7f3f0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.739+0000 7f3f1959a640 1 -- 172.21.15.16:0/3815172453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0cfe00 con 0x7f3f08004800 2024-04-25T10:10:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.739+0000 7f3f1959a640 1 -- 172.21.15.16:0/3815172453 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0800c160 con 0x7f3f080039b0 2024-04-25T10:10:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.739+0000 7f3f1959a640 1 -- 172.21.15.16:0/3815172453 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0800c360 con 0x7f3f080051c0 2024-04-25T10:10:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.740+0000 7f3f19d9b640 1 --2- 172.21.15.16:0/3815172453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f08004800 0x7f3f08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.740+0000 7f3f1a59c640 1 --2- 172.21.15.16:0/3815172453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f080039b0 0x7f3f08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.740+0000 7f3f1ad9d640 1 --2- 172.21.15.16:0/3815172453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f080051c0 0x7f3f0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.740+0000 7f3f19d9b640 1 -- 172.21.15.16:0/3815172453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f080051c0 msgr2=0x7f3f0800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.740+0000 7f3f19d9b640 1 --2- 172.21.15.16:0/3815172453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f080051c0 0x7f3f0800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.740+0000 7f3f19d9b640 1 -- 172.21.15.16:0/3815172453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f080039b0 msgr2=0x7f3f08003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.740+0000 7f3f19d9b640 1 --2- 172.21.15.16:0/3815172453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f080039b0 0x7f3f08003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.740+0000 7f3f1ad9d640 1 -- 172.21.15.16:0/3815172453 reap_dead start 2024-04-25T10:10:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.740+0000 7f3f19d9b640 1 -- 172.21.15.16:0/3815172453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f08003520 con 0x7f3f08004800 2024-04-25T10:10:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.741+0000 7f3f19d9b640 1 --2- 172.21.15.16:0/3815172453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f08004800 0x7f3f08004c80 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3f04001750 tx=0x7f3f08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdf8dec9441c95a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.741+0000 7f3f1959a640 1 -- 172.21.15.16:0/3815172453 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04001370 con 0x7f3f08004800 2024-04-25T10:10:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.741+0000 7f3f1959a640 1 -- 172.21.15.16:0/3815172453 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f040052d0 con 0x7f3f08004800 2024-04-25T10:10:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.741+0000 7f3f1959a640 1 -- 172.21.15.16:0/3815172453 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0402abc0 con 0x7f3f08004800 2024-04-25T10:10:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.741+0000 7f3f20d22640 1 -- 172.21.15.16:0/3815172453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f08004800 msgr2=0x7f3f08004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.741+0000 7f3f20d22640 1 --2- 172.21.15.16:0/3815172453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f08004800 0x7f3f08004c80 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3f04001750 tx=0x7f3f08005830 comp rx=0 tx=0).stop 2024-04-25T10:10:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.741+0000 7f3f20d22640 1 -- 172.21.15.16:0/3815172453 shutdown_connections 2024-04-25T10:10:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.741+0000 7f3f20d22640 1 --2- 172.21.15.16:0/3815172453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f08004800 0x7f3f08004c80 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.741+0000 7f3f20d22640 1 -- 172.21.15.16:0/3815172453 >> 172.21.15.16:0/3815172453 conn(0x7f3f1c06f430 msgr2=0x7f3f1c06f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.741+0000 7f3f20d22640 1 -- 172.21.15.16:0/3815172453 shutdown_connections 2024-04-25T10:10:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.741+0000 7f3f20d22640 1 -- 172.21.15.16:0/3815172453 wait complete. 2024-04-25T10:10:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.743+0000 7f3f20d22640 1 Processor -- start 2024-04-25T10:10:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.743+0000 7f3f20d22640 1 -- start start 2024-04-25T10:10:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f20d22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f08004800 0x7f3f1c150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f20d22640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f1c160c70 0x7f3f1c0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f20d22640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f1c0b1f60 0x7f3f1c0b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f20d22640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0cc0c0 con 0x7f3f08004800 2024-04-25T10:10:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f20d22640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0d0330 con 0x7f3f1c160c70 2024-04-25T10:10:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f20d22640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0ce8a0 con 0x7f3f1c0b1f60 2024-04-25T10:10:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f1a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f08004800 0x7f3f1c150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f1a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f08004800 0x7f3f1c150cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57722/0 (socket says 172.21.15.16:57722) 2024-04-25T10:10:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f1a59c640 1 -- 172.21.15.16:0/2769057777 learned_addr learned my addr 172.21.15.16:0/2769057777 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f19d9b640 1 --2- 172.21.15.16:0/2769057777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f1c160c70 0x7f3f1c0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f1ad9d640 1 --2- 172.21.15.16:0/2769057777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f1c0b1f60 0x7f3f1c0b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f1a59c640 1 -- 172.21.15.16:0/2769057777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f1c0b1f60 msgr2=0x7f3f1c0b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f1a59c640 1 --2- 172.21.15.16:0/2769057777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f1c0b1f60 0x7f3f1c0b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f1a59c640 1 -- 172.21.15.16:0/2769057777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f1c160c70 msgr2=0x7f3f1c0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f1a59c640 1 --2- 172.21.15.16:0/2769057777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f1c160c70 0x7f3f1c0b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f1a59c640 1 -- 172.21.15.16:0/2769057777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c0b4d60 con 0x7f3f08004800 2024-04-25T10:10:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f1ad9d640 1 --2- 172.21.15.16:0/2769057777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f1c0b1f60 0x7f3f1c0b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f1a59c640 1 --2- 172.21.15.16:0/2769057777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f08004800 0x7f3f1c150cb0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f3f140118c0 tx=0x7f3f14011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f12ffd640 1 -- 172.21.15.16:0/2769057777 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f14019070 con 0x7f3f08004800 2024-04-25T10:10:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f12ffd640 1 -- 172.21.15.16:0/2769057777 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f14004240 con 0x7f3f08004800 2024-04-25T10:10:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f12ffd640 1 -- 172.21.15.16:0/2769057777 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1400f820 con 0x7f3f08004800 2024-04-25T10:10:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f20d22640 1 -- 172.21.15.16:0/2769057777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1c0b5040 con 0x7f3f08004800 2024-04-25T10:10:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.744+0000 7f3f20d22640 1 -- 172.21.15.16:0/2769057777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1c0b5600 con 0x7f3f08004800 2024-04-25T10:10:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.745+0000 7f3f20d22640 1 -- 172.21.15.16:0/2769057777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f1c0750b0 con 0x7f3f08004800 2024-04-25T10:10:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.746+0000 7f3f12ffd640 1 -- 172.21.15.16:0/2769057777 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f1401c070 con 0x7f3f08004800 2024-04-25T10:10:51.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.746+0000 7f3f12ffd640 1 --2- 172.21.15.16:0/2769057777 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3efc06b500 0x7f3efc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.746+0000 7f3f12ffd640 1 -- 172.21.15.16:0/2769057777 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f3f1408a4d0 con 0x7f3f08004800 2024-04-25T10:10:51.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.746+0000 7f3f19d9b640 1 --2- 172.21.15.16:0/2769057777 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3efc06b500 0x7f3efc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.747+0000 7f3f19d9b640 1 --2- 172.21.15.16:0/2769057777 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3efc06b500 0x7f3efc06d9c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3f04032000 tx=0x7f3f0402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:51.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.750+0000 7f3f12ffd640 1 -- 172.21.15.16:0/2769057777 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f14059420 con 0x7f3f08004800 2024-04-25T10:10:51.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:51.934+0000 7f3f20d22640 1 -- 172.21.15.16:0/2769057777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3f1c0b77a0 con 0x7f3f08004800 2024-04-25T10:10:52.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.610+0000 7f3f12ffd640 1 -- 172.21.15.16:0/2769057777 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v78) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3f14058d20 con 0x7f3f08004800 2024-04-25T10:10:52.615 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-04-25T10:10:52.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.613+0000 7f3f20d22640 1 -- 172.21.15.16:0/2769057777 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3efc06b500 msgr2=0x7f3efc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.614+0000 7f3f20d22640 1 --2- 172.21.15.16:0/2769057777 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3efc06b500 0x7f3efc06d9c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3f04032000 tx=0x7f3f0402e000 comp rx=0 tx=0).stop 2024-04-25T10:10:52.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.614+0000 7f3f20d22640 1 -- 172.21.15.16:0/2769057777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f08004800 msgr2=0x7f3f1c150cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.614+0000 7f3f20d22640 1 --2- 172.21.15.16:0/2769057777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f08004800 0x7f3f1c150cb0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f3f140118c0 tx=0x7f3f14011d80 comp rx=0 tx=0).stop 2024-04-25T10:10:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.615+0000 7f3f20d22640 1 -- 172.21.15.16:0/2769057777 shutdown_connections 2024-04-25T10:10:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.615+0000 7f3f20d22640 1 --2- 172.21.15.16:0/2769057777 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3efc06b500 0x7f3efc06d9c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.615+0000 7f3f20d22640 1 --2- 172.21.15.16:0/2769057777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f1c0b1f60 0x7f3f1c0b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.615+0000 7f3f20d22640 1 --2- 172.21.15.16:0/2769057777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f1c160c70 0x7f3f1c0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.616+0000 7f3f20d22640 1 --2- 172.21.15.16:0/2769057777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f08004800 0x7f3f1c150cb0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.616+0000 7f3f20d22640 1 -- 172.21.15.16:0/2769057777 >> 172.21.15.16:0/2769057777 conn(0x7f3f1c06f430 msgr2=0x7f3f1c0b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.616+0000 7f3f20d22640 1 -- 172.21.15.16:0/2769057777 shutdown_connections 2024-04-25T10:10:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.617+0000 7f3f20d22640 1 -- 172.21.15.16:0/2769057777 wait complete. 2024-04-25T10:10:52.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:10:52.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.741+0000 7ff467f09640 1 Processor -- start 2024-04-25T10:10:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.741+0000 7ff467f09640 1 -- start start 2024-04-25T10:10:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.742+0000 7ff467f09640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4600d3b90 0x7ff4600d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.742+0000 7ff467f09640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff4600d0f60 0x7ff4600b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.742+0000 7ff467f09640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4600d4090 con 0x7ff46015dca0 2024-04-25T10:10:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.742+0000 7ff467f09640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4600b6b20 con 0x7ff4600d3b90 2024-04-25T10:10:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.742+0000 7ff467f09640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4600d1390 con 0x7ff4600d0f60 2024-04-25T10:10:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.742+0000 7ff4654a4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4600d3b90 0x7ff4600d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.742+0000 7ff465ca5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff46015dca0 0x7ff4600ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46210/0 (socket says 172.21.15.16:46210) 2024-04-25T10:10:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.742+0000 7ff465ca5640 1 -- 172.21.15.16:0/3822338834 learned_addr learned my addr 172.21.15.16:0/3822338834 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.742+0000 7ff4654a4640 1 --2- 172.21.15.16:0/3822338834 >> v2:172.21.15.16:3300/0 conn(0x7ff4600d3b90 0x7ff4600d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55198/0 (socket says 172.21.15.16:55198) 2024-04-25T10:10:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.742+0000 7ff4664a6640 1 --2- 172.21.15.16:0/3822338834 >> v2:172.21.15.186:3300/0 conn(0x7ff4600d0f60 0x7ff4600b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.742+0000 7ff4654a4640 1 -- 172.21.15.16:0/3822338834 >> v2:172.21.15.186:3300/0 conn(0x7ff4600d0f60 msgr2=0x7ff4600b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.742+0000 7ff4654a4640 1 --2- 172.21.15.16:0/3822338834 >> v2:172.21.15.186:3300/0 conn(0x7ff4600d0f60 0x7ff4600b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.742+0000 7ff4654a4640 1 -- 172.21.15.16:0/3822338834 >> v1:172.21.15.16:6789/0 conn(0x7ff46015dca0 legacy=0x7ff4600ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.742+0000 7ff4654a4640 1 -- 172.21.15.16:0/3822338834 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4600d4370 con 0x7ff4600d3b90 2024-04-25T10:10:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.742+0000 7ff464ca3640 1 -- 172.21.15.16:0/3822338834 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510912108 0 0) 0x7ff4600d4090 con 0x7ff46015dca0 2024-04-25T10:10:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.743+0000 7ff4654a4640 1 --2- 172.21.15.16:0/3822338834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4600d3b90 0x7ff4600d0a20 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ff4600b9640 tx=0x7ff45002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62c3a224a8b3d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.743+0000 7ff464ca3640 1 -- 172.21.15.16:0/3822338834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff450037070 con 0x7ff4600d3b90 2024-04-25T10:10:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.743+0000 7ff464ca3640 1 -- 172.21.15.16:0/3822338834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff450032440 con 0x7ff4600d3b90 2024-04-25T10:10:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.743+0000 7ff464ca3640 1 -- 172.21.15.16:0/3822338834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff450032810 con 0x7ff4600d3b90 2024-04-25T10:10:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.744+0000 7ff467f09640 1 -- 172.21.15.16:0/3822338834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4600d3b90 msgr2=0x7ff4600d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.744+0000 7ff467f09640 1 --2- 172.21.15.16:0/3822338834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4600d3b90 0x7ff4600d0a20 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ff4600b9640 tx=0x7ff45002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.744+0000 7ff467f09640 1 -- 172.21.15.16:0/3822338834 shutdown_connections 2024-04-25T10:10:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.744+0000 7ff467f09640 1 --2- 172.21.15.16:0/3822338834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4600d3b90 0x7ff4600d0a20 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.744+0000 7ff467f09640 1 --2- 172.21.15.16:0/3822338834 >> v2:172.21.15.186:3300/0 conn(0x7ff4600d0f60 0x7ff4600b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.744+0000 7ff467f09640 1 -- 172.21.15.16:0/3822338834 >> 172.21.15.16:0/3822338834 conn(0x7ff4600726c0 msgr2=0x7ff460072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.744+0000 7ff467f09640 1 -- 172.21.15.16:0/3822338834 shutdown_connections 2024-04-25T10:10:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.744+0000 7ff467f09640 1 -- 172.21.15.16:0/3822338834 wait complete. 2024-04-25T10:10:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.745+0000 7ff467f09640 1 Processor -- start 2024-04-25T10:10:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.745+0000 7ff467f09640 1 -- start start 2024-04-25T10:10:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.746+0000 7ff467f09640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4600d0f60 0x7ff4601f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.746+0000 7ff467f09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4600d3b90 0x7ff4600ac740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.746+0000 7ff467f09640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff46015dca0 0x7ff460160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.746+0000 7ff467f09640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600cf050 con 0x7ff4600d3b90 2024-04-25T10:10:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.746+0000 7ff467f09640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600b6d50 con 0x7ff46015dca0 2024-04-25T10:10:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.746+0000 7ff467f09640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff46015e080 con 0x7ff4600d0f60 2024-04-25T10:10:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.746+0000 7ff4654a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4600d3b90 0x7ff4600ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.746+0000 7ff4654a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4600d3b90 0x7ff4600ac740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55202/0 (socket says 172.21.15.16:55202) 2024-04-25T10:10:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.746+0000 7ff4654a4640 1 -- 172.21.15.16:0/4110112706 learned_addr learned my addr 172.21.15.16:0/4110112706 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.746+0000 7ff4664a6640 1 --2- 172.21.15.16:0/4110112706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff46015dca0 0x7ff460160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.746+0000 7ff465ca5640 1 --2- 172.21.15.16:0/4110112706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4600d0f60 0x7ff4601f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.746+0000 7ff4654a4640 1 -- 172.21.15.16:0/4110112706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4600d0f60 msgr2=0x7ff4601f4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.746+0000 7ff4654a4640 1 --2- 172.21.15.16:0/4110112706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4600d0f60 0x7ff4601f4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.746+0000 7ff4654a4640 1 -- 172.21.15.16:0/4110112706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff46015dca0 msgr2=0x7ff460160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.746+0000 7ff4654a4640 1 --2- 172.21.15.16:0/4110112706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff46015dca0 0x7ff460160600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.746+0000 7ff4654a4640 1 -- 172.21.15.16:0/4110112706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4600cd320 con 0x7ff4600d3b90 2024-04-25T10:10:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.747+0000 7ff4654a4640 1 --2- 172.21.15.16:0/4110112706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4600d3b90 0x7ff4600ac740 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff450035040 tx=0x7ff4500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.747+0000 7ff4567fc640 1 -- 172.21.15.16:0/4110112706 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff450037040 con 0x7ff4600d3b90 2024-04-25T10:10:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.747+0000 7ff4567fc640 1 -- 172.21.15.16:0/4110112706 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff450030770 con 0x7ff4600d3b90 2024-04-25T10:10:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.747+0000 7ff467f09640 1 -- 172.21.15.16:0/4110112706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4601f3d20 con 0x7ff4600d3b90 2024-04-25T10:10:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.747+0000 7ff4567fc640 1 -- 172.21.15.16:0/4110112706 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff450030b80 con 0x7ff4600d3b90 2024-04-25T10:10:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.747+0000 7ff467f09640 1 -- 172.21.15.16:0/4110112706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4601f42e0 con 0x7ff4600d3b90 2024-04-25T10:10:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.748+0000 7ff467f09640 1 -- 172.21.15.16:0/4110112706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4600750b0 con 0x7ff4600d3b90 2024-04-25T10:10:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.748+0000 7ff4567fc640 1 -- 172.21.15.16:0/4110112706 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff450044020 con 0x7ff4600d3b90 2024-04-25T10:10:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.749+0000 7ff4567fc640 1 --2- 172.21.15.16:0/4110112706 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff44406b4d0 0x7ff44406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.749+0000 7ff4567fc640 1 -- 172.21.15.16:0/4110112706 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5029+0+0 (secure 0 0 0) 0x7ff4500aa490 con 0x7ff4600d3b90 2024-04-25T10:10:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.749+0000 7ff465ca5640 1 --2- 172.21.15.16:0/4110112706 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff44406b4d0 0x7ff44406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.749+0000 7ff465ca5640 1 --2- 172.21.15.16:0/4110112706 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff44406b4d0 0x7ff44406d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff45c000f60 tx=0x7ff45c007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.753+0000 7ff4567fc640 1 -- 172.21.15.16:0/4110112706 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff450032e20 con 0x7ff4600d3b90 2024-04-25T10:10:52.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.934+0000 7ff467f09640 1 -- 172.21.15.16:0/4110112706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff4600d2910 con 0x7ff4600d3b90 2024-04-25T10:10:52.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.937+0000 7ff4567fc640 1 -- 172.21.15.16:0/4110112706 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff4500795a0 con 0x7ff4600d3b90 2024-04-25T10:10:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.939+0000 7ff467f09640 1 -- 172.21.15.16:0/4110112706 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff44406b4d0 msgr2=0x7ff44406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.939+0000 7ff467f09640 1 --2- 172.21.15.16:0/4110112706 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff44406b4d0 0x7ff44406d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff45c000f60 tx=0x7ff45c007c00 comp rx=0 tx=0).stop 2024-04-25T10:10:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.939+0000 7ff467f09640 1 -- 172.21.15.16:0/4110112706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4600d3b90 msgr2=0x7ff4600ac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.939+0000 7ff467f09640 1 --2- 172.21.15.16:0/4110112706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4600d3b90 0x7ff4600ac740 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff450035040 tx=0x7ff4500046c0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.939+0000 7ff467f09640 1 -- 172.21.15.16:0/4110112706 shutdown_connections 2024-04-25T10:10:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.939+0000 7ff467f09640 1 --2- 172.21.15.16:0/4110112706 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff44406b4d0 0x7ff44406d990 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.939+0000 7ff467f09640 1 --2- 172.21.15.16:0/4110112706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff46015dca0 0x7ff460160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.939+0000 7ff467f09640 1 --2- 172.21.15.16:0/4110112706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4600d3b90 0x7ff4600ac740 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.939+0000 7ff467f09640 1 --2- 172.21.15.16:0/4110112706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4600d0f60 0x7ff4601f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.939+0000 7ff467f09640 1 -- 172.21.15.16:0/4110112706 >> 172.21.15.16:0/4110112706 conn(0x7ff4600726c0 msgr2=0x7ff4600b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.939+0000 7ff467f09640 1 -- 172.21.15.16:0/4110112706 shutdown_connections 2024-04-25T10:10:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:52.939+0000 7ff467f09640 1 -- 172.21.15.16:0/4110112706 wait complete. 2024-04-25T10:10:52.955 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-25T10:10:53.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.051+0000 7fb276edb640 1 Processor -- start 2024-04-25T10:10:53.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.051+0000 7fb276edb640 1 -- start start 2024-04-25T10:10:53.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.051+0000 7fb276edb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb2700cd320 0x7fb2700cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.051+0000 7fb276edb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2700cfe80 0x7fb2700cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.051+0000 7fb276edb640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2700cd930 con 0x7fb27015df40 2024-04-25T10:10:53.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.051+0000 7fb276edb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2700cdc10 con 0x7fb2700cd320 2024-04-25T10:10:53.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.051+0000 7fb276edb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2700cdef0 con 0x7fb2700cfe80 2024-04-25T10:10:53.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.052+0000 7fb26ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb2700cd320 0x7fb2700cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.052+0000 7fb26ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb2700cd320 0x7fb2700cf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55214/0 (socket says 172.21.15.16:55214) 2024-04-25T10:10:53.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.052+0000 7fb26ffff640 1 -- 172.21.15.16:0/1689161824 learned_addr learned my addr 172.21.15.16:0/1689161824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:53.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.052+0000 7fb275478640 1 --2- 172.21.15.16:0/1689161824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2700cfe80 0x7fb2700cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.052+0000 7fb26ffff640 1 -- 172.21.15.16:0/1689161824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2700cfe80 msgr2=0x7fb2700cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.052+0000 7fb26ffff640 1 --2- 172.21.15.16:0/1689161824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2700cfe80 0x7fb2700cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.052+0000 7fb26ffff640 1 -- 172.21.15.16:0/1689161824 >> v1:172.21.15.186:6789/0 conn(0x7fb27015df40 legacy=0x7fb2700b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.052+0000 7fb26ffff640 1 -- 172.21.15.16:0/1689161824 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2700cada0 con 0x7fb2700cd320 2024-04-25T10:10:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.052+0000 7fb275478640 1 --2- 172.21.15.16:0/1689161824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2700cfe80 0x7fb2700cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.053+0000 7fb26ffff640 1 --2- 172.21.15.16:0/1689161824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2700cd320 0x7fb2700cf940 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fb264009870 tx=0x7fb26402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d3a12ec6ab70d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.053+0000 7fb26f7fe640 1 -- 172.21.15.16:0/1689161824 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb264037070 con 0x7fb2700cd320 2024-04-25T10:10:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.053+0000 7fb26f7fe640 1 -- 172.21.15.16:0/1689161824 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb264032440 con 0x7fb2700cd320 2024-04-25T10:10:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.053+0000 7fb26f7fe640 1 -- 172.21.15.16:0/1689161824 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb264032810 con 0x7fb2700cd320 2024-04-25T10:10:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.053+0000 7fb276edb640 1 -- 172.21.15.16:0/1689161824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2700cd320 msgr2=0x7fb2700cf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.053+0000 7fb276edb640 1 --2- 172.21.15.16:0/1689161824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2700cd320 0x7fb2700cf940 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fb264009870 tx=0x7fb26402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.053+0000 7fb276edb640 1 -- 172.21.15.16:0/1689161824 shutdown_connections 2024-04-25T10:10:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.053+0000 7fb276edb640 1 --2- 172.21.15.16:0/1689161824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2700cfe80 0x7fb2700cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.053+0000 7fb276edb640 1 --2- 172.21.15.16:0/1689161824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2700cd320 0x7fb2700cf940 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.053+0000 7fb276edb640 1 -- 172.21.15.16:0/1689161824 >> 172.21.15.16:0/1689161824 conn(0x7fb2700d06b0 msgr2=0x7fb2700d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.053+0000 7fb276edb640 1 -- 172.21.15.16:0/1689161824 shutdown_connections 2024-04-25T10:10:53.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.054+0000 7fb276edb640 1 -- 172.21.15.16:0/1689161824 wait complete. 2024-04-25T10:10:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.055+0000 7fb276edb640 1 Processor -- start 2024-04-25T10:10:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.055+0000 7fb276edb640 1 -- start start 2024-04-25T10:10:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb276edb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2700cd320 0x7fb2700c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb276edb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2700cfe80 0x7fb2700b5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb276edb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb27015df40 0x7fb27015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb276edb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2700c4200 con 0x7fb2700cd320 2024-04-25T10:10:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb276edb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2700d02b0 con 0x7fb2700cfe80 2024-04-25T10:10:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb276edb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2700c4380 con 0x7fb27015df40 2024-04-25T10:10:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb274c77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2700cd320 0x7fb2700c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb274c77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2700cd320 0x7fb2700c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55226/0 (socket says 172.21.15.16:55226) 2024-04-25T10:10:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb274c77640 1 -- 172.21.15.16:0/2139439277 learned_addr learned my addr 172.21.15.16:0/2139439277 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb26ffff640 1 --2- 172.21.15.16:0/2139439277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2700cfe80 0x7fb2700b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb274c77640 1 -- 172.21.15.16:0/2139439277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb27015df40 msgr2=0x7fb27015ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb275478640 1 --2- 172.21.15.16:0/2139439277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb27015df40 0x7fb27015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb274c77640 1 --2- 172.21.15.16:0/2139439277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb27015df40 0x7fb27015ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb274c77640 1 -- 172.21.15.16:0/2139439277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2700cfe80 msgr2=0x7fb2700b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb274c77640 1 --2- 172.21.15.16:0/2139439277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2700cfe80 0x7fb2700b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb274c77640 1 -- 172.21.15.16:0/2139439277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2700ca610 con 0x7fb2700cd320 2024-04-25T10:10:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb26ffff640 1 --2- 172.21.15.16:0/2139439277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2700cfe80 0x7fb2700b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb274c77640 1 --2- 172.21.15.16:0/2139439277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2700cd320 0x7fb2700c8d10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb26800c760 tx=0x7fb26800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb26d7fa640 1 -- 172.21.15.16:0/2139439277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268016020 con 0x7fb2700cd320 2024-04-25T10:10:53.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb26d7fa640 1 -- 172.21.15.16:0/2139439277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb268006d50 con 0x7fb2700cd320 2024-04-25T10:10:53.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb276edb640 1 -- 172.21.15.16:0/2139439277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2700ca330 con 0x7fb2700cd320 2024-04-25T10:10:53.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb26d7fa640 1 -- 172.21.15.16:0/2139439277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268003bb0 con 0x7fb2700cd320 2024-04-25T10:10:53.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.056+0000 7fb276edb640 1 -- 172.21.15.16:0/2139439277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2700c8510 con 0x7fb2700cd320 2024-04-25T10:10:53.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.058+0000 7fb276edb640 1 -- 172.21.15.16:0/2139439277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2700c8840 con 0x7fb2700cd320 2024-04-25T10:10:53.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.058+0000 7fb26d7fa640 1 -- 172.21.15.16:0/2139439277 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb26800a850 con 0x7fb2700cd320 2024-04-25T10:10:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.058+0000 7fb26d7fa640 1 --2- 172.21.15.16:0/2139439277 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb25c06b4d0 0x7fb25c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.058+0000 7fb26d7fa640 1 -- 172.21.15.16:0/2139439277 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fb268089bc0 con 0x7fb2700cd320 2024-04-25T10:10:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.059+0000 7fb26ffff640 1 --2- 172.21.15.16:0/2139439277 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb25c06b4d0 0x7fb25c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.059+0000 7fb26ffff640 1 --2- 172.21.15.16:0/2139439277 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb25c06b4d0 0x7fb25c06d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb264034000 tx=0x7fb26402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.062+0000 7fb26d7fa640 1 -- 172.21.15.16:0/2139439277 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb268020020 con 0x7fb2700cd320 2024-04-25T10:10:53.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.316+0000 7fb276edb640 1 -- 172.21.15.16:0/2139439277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7fb2700749b0 con 0x7fb2700cd320 2024-04-25T10:10:53.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.611+0000 7fb26d7fa640 1 -- 172.21.15.16:0/2139439277 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v289) v1 ==== 166+0+0 (secure 0 0 0) 0x7fb268019310 con 0x7fb2700cd320 2024-04-25T10:10:53.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.614+0000 7fb276edb640 1 -- 172.21.15.16:0/2139439277 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb25c06b4d0 msgr2=0x7fb25c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.614+0000 7fb276edb640 1 --2- 172.21.15.16:0/2139439277 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb25c06b4d0 0x7fb25c06d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb264034000 tx=0x7fb26402e000 comp rx=0 tx=0).stop 2024-04-25T10:10:53.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.614+0000 7fb276edb640 1 -- 172.21.15.16:0/2139439277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2700cd320 msgr2=0x7fb2700c8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.614+0000 7fb276edb640 1 --2- 172.21.15.16:0/2139439277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2700cd320 0x7fb2700c8d10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb26800c760 tx=0x7fb26800cc20 comp rx=0 tx=0).stop 2024-04-25T10:10:53.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.614+0000 7fb276edb640 1 -- 172.21.15.16:0/2139439277 shutdown_connections 2024-04-25T10:10:53.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.614+0000 7fb276edb640 1 --2- 172.21.15.16:0/2139439277 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb25c06b4d0 0x7fb25c06d990 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.614+0000 7fb276edb640 1 --2- 172.21.15.16:0/2139439277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb27015df40 0x7fb27015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.614+0000 7fb276edb640 1 --2- 172.21.15.16:0/2139439277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2700cfe80 0x7fb2700b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.614+0000 7fb276edb640 1 --2- 172.21.15.16:0/2139439277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2700cd320 0x7fb2700c8d10 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.614+0000 7fb276edb640 1 -- 172.21.15.16:0/2139439277 >> 172.21.15.16:0/2139439277 conn(0x7fb2700d06b0 msgr2=0x7fb2700b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.614+0000 7fb276edb640 1 -- 172.21.15.16:0/2139439277 shutdown_connections 2024-04-25T10:10:53.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.614+0000 7fb276edb640 1 -- 172.21.15.16:0/2139439277 wait complete. 2024-04-25T10:10:53.631 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T10:10:53.631 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T10:10:53.631 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:10:53.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:53.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.728+0000 7f2e45e1f640 1 Processor -- start 2024-04-25T10:10:53.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.728+0000 7f2e45e1f640 1 -- start start 2024-04-25T10:10:53.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.729+0000 7f2e45e1f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2e400cc950 0x7f2e40160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.729+0000 7f2e45e1f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e4006aba0 0x7f2e400b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.729+0000 7f2e45e1f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4006b310 con 0x7f2e4015bb90 2024-04-25T10:10:53.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.729+0000 7f2e45e1f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e400b0010 con 0x7f2e400cc950 2024-04-25T10:10:53.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.729+0000 7f2e45e1f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4006b5f0 con 0x7f2e4006aba0 2024-04-25T10:10:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.729+0000 7f2e3effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2e400cc950 0x7f2e40160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.729+0000 7f2e3f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2e4015bb90 0x7f2e400cc240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46222/0 (socket says 172.21.15.16:46222) 2024-04-25T10:10:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.729+0000 7f2e3f7fe640 1 -- 172.21.15.16:0/12748448 learned_addr learned my addr 172.21.15.16:0/12748448 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.729+0000 7f2e3effd640 1 -- 172.21.15.16:0/12748448 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e4006aba0 msgr2=0x7f2e400b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.729+0000 7f2e3effd640 1 --2- 172.21.15.16:0/12748448 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e4006aba0 0x7f2e400b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.729+0000 7f2e3effd640 1 -- 172.21.15.16:0/12748448 >> v1:172.21.15.16:6789/0 conn(0x7f2e4015bb90 legacy=0x7f2e400cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.729+0000 7f2e3effd640 1 -- 172.21.15.16:0/12748448 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e400b0260 con 0x7f2e400cc950 2024-04-25T10:10:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.729+0000 7f2e3e7fc640 1 -- 172.21.15.16:0/12748448 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1484073701 0 0) 0x7f2e4006b310 con 0x7f2e4015bb90 2024-04-25T10:10:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.730+0000 7f2e3effd640 1 --2- 172.21.15.16:0/12748448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e400cc950 0x7f2e40160070 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2e340097a0 tx=0x7f2e3402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790e75da647ea514 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.730+0000 7f2e3e7fc640 1 -- 172.21.15.16:0/12748448 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34037070 con 0x7f2e400cc950 2024-04-25T10:10:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.730+0000 7f2e3e7fc640 1 -- 172.21.15.16:0/12748448 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3402adb0 con 0x7f2e400cc950 2024-04-25T10:10:53.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.730+0000 7f2e3e7fc640 1 -- 172.21.15.16:0/12748448 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34032660 con 0x7f2e400cc950 2024-04-25T10:10:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.730+0000 7f2e45e1f640 1 -- 172.21.15.16:0/12748448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e400cc950 msgr2=0x7f2e40160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.730+0000 7f2e45e1f640 1 --2- 172.21.15.16:0/12748448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e400cc950 0x7f2e40160070 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2e340097a0 tx=0x7f2e3402a060 comp rx=0 tx=0).stop 2024-04-25T10:10:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.731+0000 7f2e45e1f640 1 -- 172.21.15.16:0/12748448 shutdown_connections 2024-04-25T10:10:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.731+0000 7f2e45e1f640 1 --2- 172.21.15.16:0/12748448 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e4006aba0 0x7f2e400b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.731+0000 7f2e45e1f640 1 --2- 172.21.15.16:0/12748448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e400cc950 0x7f2e40160070 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.731+0000 7f2e45e1f640 1 -- 172.21.15.16:0/12748448 >> 172.21.15.16:0/12748448 conn(0x7f2e400d06b0 msgr2=0x7f2e400d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.731+0000 7f2e45e1f640 1 -- 172.21.15.16:0/12748448 shutdown_connections 2024-04-25T10:10:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.731+0000 7f2e45e1f640 1 -- 172.21.15.16:0/12748448 wait complete. 2024-04-25T10:10:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.732+0000 7f2e45e1f640 1 Processor -- start 2024-04-25T10:10:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.732+0000 7f2e45e1f640 1 -- start start 2024-04-25T10:10:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e45e1f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e4006aba0 0x7f2e4015e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e45e1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e400cc950 0x7f2e4014d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e45e1f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e4015bb90 0x7f2e400b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e45e1f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400b0fa0 con 0x7f2e400cc950 2024-04-25T10:10:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e45e1f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4006b030 con 0x7f2e4006aba0 2024-04-25T10:10:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e45e1f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400b1120 con 0x7f2e4015bb90 2024-04-25T10:10:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e3effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e400cc950 0x7f2e4014d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e3effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e400cc950 0x7f2e4014d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55236/0 (socket says 172.21.15.16:55236) 2024-04-25T10:10:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e3effd640 1 -- 172.21.15.16:0/2868302509 learned_addr learned my addr 172.21.15.16:0/2868302509 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e3effd640 1 -- 172.21.15.16:0/2868302509 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e4015bb90 msgr2=0x7f2e400b52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e3effd640 1 --2- 172.21.15.16:0/2868302509 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e4015bb90 0x7f2e400b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e3f7fe640 1 --2- 172.21.15.16:0/2868302509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e4006aba0 0x7f2e4015e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e3effd640 1 -- 172.21.15.16:0/2868302509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e4006aba0 msgr2=0x7f2e4015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e3effd640 1 --2- 172.21.15.16:0/2868302509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e4006aba0 0x7f2e4015e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e3effd640 1 -- 172.21.15.16:0/2868302509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e401f1850 con 0x7f2e400cc950 2024-04-25T10:10:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e3effd640 1 --2- 172.21.15.16:0/2868302509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e400cc950 0x7f2e4014d230 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2e340097a0 tx=0x7f2e340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e27fff640 1 -- 172.21.15.16:0/2868302509 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34042070 con 0x7f2e400cc950 2024-04-25T10:10:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e27fff640 1 -- 172.21.15.16:0/2868302509 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3403e670 con 0x7f2e400cc950 2024-04-25T10:10:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e45e1f640 1 -- 172.21.15.16:0/2868302509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e400c8b40 con 0x7f2e400cc950 2024-04-25T10:10:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.733+0000 7f2e27fff640 1 -- 172.21.15.16:0/2868302509 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34037040 con 0x7f2e400cc950 2024-04-25T10:10:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.734+0000 7f2e45e1f640 1 -- 172.21.15.16:0/2868302509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e400c9100 con 0x7f2e400cc950 2024-04-25T10:10:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.734+0000 7f2e45e1f640 1 -- 172.21.15.16:0/2868302509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e400749b0 con 0x7f2e400cc950 2024-04-25T10:10:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.735+0000 7f2e27fff640 1 -- 172.21.15.16:0/2868302509 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e34045070 con 0x7f2e400cc950 2024-04-25T10:10:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.735+0000 7f2e27fff640 1 --2- 172.21.15.16:0/2868302509 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2e2c06b4d0 0x7f2e2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.735+0000 7f2e27fff640 1 -- 172.21.15.16:0/2868302509 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f2e340aa8d0 con 0x7f2e400cc950 2024-04-25T10:10:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.736+0000 7f2e3f7fe640 1 --2- 172.21.15.16:0/2868302509 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2e2c06b4d0 0x7f2e2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.736+0000 7f2e3f7fe640 1 --2- 172.21.15.16:0/2868302509 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2e2c06b4d0 0x7f2e2c06d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2e30000f60 tx=0x7f2e30009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.740+0000 7f2e27fff640 1 -- 172.21.15.16:0/2868302509 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e34079ba0 con 0x7f2e400cc950 2024-04-25T10:10:53.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.920+0000 7f2e45e1f640 1 -- 172.21.15.16:0/2868302509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e400adcc0 con 0x7f2e400cc950 2024-04-25T10:10:53.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.921+0000 7f2e27fff640 1 -- 172.21.15.16:0/2868302509 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+13153 (secure 0 0 0) 0x7f2e340794a0 con 0x7f2e400cc950 2024-04-25T10:10:53.926 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:53.926 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":78,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:10:52.605983+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/1735811292":"2024-04-26T10:10:49.300510+0000","172.21.15.16:6834/1735811292":"2024-04-26T10:10:49.300510+0000","172.21.15.186:6837/3292077067":"2024-04-26T10:09:31.374252+0000","172.21.15.186:6836/3292077067":"2024-04-26T10:09:31.374252+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:53.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.923+0000 7f2e45e1f640 1 -- 172.21.15.16:0/2868302509 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2e2c06b4d0 msgr2=0x7f2e2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.923+0000 7f2e45e1f640 1 --2- 172.21.15.16:0/2868302509 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2e2c06b4d0 0x7f2e2c06d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2e30000f60 tx=0x7f2e30009040 comp rx=0 tx=0).stop 2024-04-25T10:10:53.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.923+0000 7f2e45e1f640 1 -- 172.21.15.16:0/2868302509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e400cc950 msgr2=0x7f2e4014d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.923+0000 7f2e45e1f640 1 --2- 172.21.15.16:0/2868302509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e400cc950 0x7f2e4014d230 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2e340097a0 tx=0x7f2e340046c0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.923+0000 7f2e45e1f640 1 -- 172.21.15.16:0/2868302509 shutdown_connections 2024-04-25T10:10:53.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.923+0000 7f2e45e1f640 1 --2- 172.21.15.16:0/2868302509 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2e2c06b4d0 0x7f2e2c06d990 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.923+0000 7f2e45e1f640 1 --2- 172.21.15.16:0/2868302509 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e4015bb90 0x7f2e400b52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.923+0000 7f2e45e1f640 1 --2- 172.21.15.16:0/2868302509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e400cc950 0x7f2e4014d230 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.923+0000 7f2e45e1f640 1 --2- 172.21.15.16:0/2868302509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e4006aba0 0x7f2e4015e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.923+0000 7f2e45e1f640 1 -- 172.21.15.16:0/2868302509 >> 172.21.15.16:0/2868302509 conn(0x7f2e400d06b0 msgr2=0x7f2e4015abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.923+0000 7f2e45e1f640 1 -- 172.21.15.16:0/2868302509 shutdown_connections 2024-04-25T10:10:53.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:53.923+0000 7f2e45e1f640 1 -- 172.21.15.16:0/2868302509 wait complete. 2024-04-25T10:10:53.939 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:10:53.940 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:10:54.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.041+0000 7efd34e6b640 1 Processor -- start 2024-04-25T10:10:54.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.041+0000 7efd34e6b640 1 -- start start 2024-04-25T10:10:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd34e6b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7efd3015dc80 0x7efd30161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd34e6b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd300b97c0 0x7efd300b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd34e6b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd301627a0 con 0x7efd300b5a30 2024-04-25T10:10:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd34e6b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd300b5ea0 con 0x7efd3015dc80 2024-04-25T10:10:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd34e6b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd301621e0 con 0x7efd300b97c0 2024-04-25T10:10:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd2ed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efd300b5a30 0x7efd3006bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46228/0 (socket says 172.21.15.16:46228) 2024-04-25T10:10:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd2ed9d640 1 -- 172.21.15.16:0/490226008 learned_addr learned my addr 172.21.15.16:0/490226008 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd2e59c640 1 --2- 172.21.15.16:0/490226008 >> v2:172.21.15.186:3300/0 conn(0x7efd3015dc80 0x7efd30161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd2d59a640 1 -- 172.21.15.16:0/490226008 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022697765 0 0) 0x7efd301627a0 con 0x7efd300b5a30 2024-04-25T10:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd2d59a640 1 -- 172.21.15.16:0/490226008 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd18003520 con 0x7efd300b5a30 2024-04-25T10:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd2dd9b640 1 --2- 172.21.15.16:0/490226008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd300b97c0 0x7efd300b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd2d59a640 1 -- 172.21.15.16:0/490226008 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2387787311 0 0) 0x7efd18003520 con 0x7efd300b5a30 2024-04-25T10:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd2d59a640 1 -- 172.21.15.16:0/490226008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd300b97c0 msgr2=0x7efd300b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd2d59a640 1 --2- 172.21.15.16:0/490226008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd300b97c0 0x7efd300b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd2d59a640 1 -- 172.21.15.16:0/490226008 >> v2:172.21.15.186:3300/0 conn(0x7efd3015dc80 msgr2=0x7efd30161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd2d59a640 1 --2- 172.21.15.16:0/490226008 >> v2:172.21.15.186:3300/0 conn(0x7efd3015dc80 0x7efd30161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd2d59a640 1 -- 172.21.15.16:0/490226008 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd301624c0 con 0x7efd300b5a30 2024-04-25T10:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd2d59a640 1 -- 172.21.15.16:0/490226008 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7efd20001670 con 0x7efd300b5a30 2024-04-25T10:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.042+0000 7efd2d59a640 1 -- 172.21.15.16:0/490226008 >> v1:172.21.15.16:6789/0 conn(0x7efd300b5a30 legacy=0x7efd3006bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.043+0000 7efd2d59a640 1 --2- 172.21.15.16:0/490226008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd300b97c0 0x7efd180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.043+0000 7efd2d59a640 1 --2- 172.21.15.16:0/490226008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd18003f30 0x7efd18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.043+0000 7efd2d59a640 1 --2- 172.21.15.16:0/490226008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd18004d40 0x7efd180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.043+0000 7efd2d59a640 1 -- 172.21.15.16:0/490226008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd300b5ea0 con 0x7efd18003f30 2024-04-25T10:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.043+0000 7efd2d59a640 1 -- 172.21.15.16:0/490226008 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd301621e0 con 0x7efd18004d40 2024-04-25T10:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.043+0000 7efd2d59a640 1 -- 172.21.15.16:0/490226008 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1800ba10 con 0x7efd300b97c0 2024-04-25T10:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.043+0000 7efd2dd9b640 1 --2- 172.21.15.16:0/490226008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd18003f30 0x7efd18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.043+0000 7efd2e59c640 1 --2- 172.21.15.16:0/490226008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd300b97c0 0x7efd180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.043+0000 7efd2ed9d640 1 --2- 172.21.15.16:0/490226008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd18004d40 0x7efd180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.043+0000 7efd2dd9b640 1 -- 172.21.15.16:0/490226008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd300b97c0 msgr2=0x7efd180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.043+0000 7efd2dd9b640 1 --2- 172.21.15.16:0/490226008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd300b97c0 0x7efd180039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.043+0000 7efd2dd9b640 1 -- 172.21.15.16:0/490226008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd18004d40 msgr2=0x7efd180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.043+0000 7efd2dd9b640 1 --2- 172.21.15.16:0/490226008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd18004d40 0x7efd180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.043+0000 7efd2dd9b640 1 -- 172.21.15.16:0/490226008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd18003520 con 0x7efd18003f30 2024-04-25T10:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.043+0000 7efd2e59c640 1 --2- 172.21.15.16:0/490226008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd300b97c0 0x7efd180039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.043+0000 7efd2ed9d640 1 --2- 172.21.15.16:0/490226008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd18004d40 0x7efd180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.044+0000 7efd2dd9b640 1 --2- 172.21.15.16:0/490226008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd18003f30 0x7efd18004800 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7efd24009f90 tx=0x7efd18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db83cb47e53a277 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.044+0000 7efd2d59a640 1 -- 172.21.15.16:0/490226008 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd24009c10 con 0x7efd18003f30 2024-04-25T10:10:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.044+0000 7efd2d59a640 1 -- 172.21.15.16:0/490226008 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd24036070 con 0x7efd18003f30 2024-04-25T10:10:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.044+0000 7efd2d59a640 1 -- 172.21.15.16:0/490226008 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd24031850 con 0x7efd18003f30 2024-04-25T10:10:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.044+0000 7efd34e6b640 1 -- 172.21.15.16:0/490226008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd18003f30 msgr2=0x7efd18004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.044+0000 7efd34e6b640 1 --2- 172.21.15.16:0/490226008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd18003f30 0x7efd18004800 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7efd24009f90 tx=0x7efd18005830 comp rx=0 tx=0).stop 2024-04-25T10:10:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.044+0000 7efd2ed9d640 1 -- 172.21.15.16:0/490226008 reap_dead start 2024-04-25T10:10:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.045+0000 7efd34e6b640 1 -- 172.21.15.16:0/490226008 shutdown_connections 2024-04-25T10:10:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.045+0000 7efd34e6b640 1 -- 172.21.15.16:0/490226008 >> 172.21.15.16:0/490226008 conn(0x7efd300d04f0 msgr2=0x7efd300d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.045+0000 7efd34e6b640 1 -- 172.21.15.16:0/490226008 shutdown_connections 2024-04-25T10:10:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.045+0000 7efd34e6b640 1 -- 172.21.15.16:0/490226008 wait complete. 2024-04-25T10:10:54.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.046+0000 7efd34e6b640 1 Processor -- start 2024-04-25T10:10:54.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.046+0000 7efd34e6b640 1 -- start start 2024-04-25T10:10:54.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.047+0000 7efd34e6b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd18003f30 0x7efd300cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.047+0000 7efd34e6b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd3015c6b0 0x7efd300b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.047+0000 7efd34e6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3015cb30 0x7efd300b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.047+0000 7efd34e6b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd30161db0 con 0x7efd3015cb30 2024-04-25T10:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.047+0000 7efd34e6b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3015e060 con 0x7efd18003f30 2024-04-25T10:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.047+0000 7efd34e6b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd30161f30 con 0x7efd3015c6b0 2024-04-25T10:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.047+0000 7efd2ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3015cb30 0x7efd300b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.047+0000 7efd2ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3015cb30 0x7efd300b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55254/0 (socket says 172.21.15.16:55254) 2024-04-25T10:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.047+0000 7efd2ed9d640 1 -- 172.21.15.16:0/455297240 learned_addr learned my addr 172.21.15.16:0/455297240 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.047+0000 7efd2e59c640 1 --2- 172.21.15.16:0/455297240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd18003f30 0x7efd300cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.047+0000 7efd2dd9b640 1 --2- 172.21.15.16:0/455297240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd3015c6b0 0x7efd300b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.047+0000 7efd2ed9d640 1 -- 172.21.15.16:0/455297240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd3015c6b0 msgr2=0x7efd300b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.047+0000 7efd2ed9d640 1 --2- 172.21.15.16:0/455297240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd3015c6b0 0x7efd300b1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.047+0000 7efd2ed9d640 1 -- 172.21.15.16:0/455297240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd18003f30 msgr2=0x7efd300cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.047+0000 7efd2ed9d640 1 --2- 172.21.15.16:0/455297240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd18003f30 0x7efd300cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.047+0000 7efd2ed9d640 1 -- 172.21.15.16:0/455297240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd301f7810 con 0x7efd3015cb30 2024-04-25T10:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.047+0000 7efd2ed9d640 1 --2- 172.21.15.16:0/455297240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3015cb30 0x7efd300b2450 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7efd20000c00 tx=0x7efd2000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.048+0000 7efd1effd640 1 -- 172.21.15.16:0/455297240 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2000c610 con 0x7efd3015cb30 2024-04-25T10:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.048+0000 7efd1effd640 1 -- 172.21.15.16:0/455297240 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd20018070 con 0x7efd3015cb30 2024-04-25T10:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.048+0000 7efd1effd640 1 -- 172.21.15.16:0/455297240 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2000ca10 con 0x7efd3015cb30 2024-04-25T10:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.048+0000 7efd34e6b640 1 -- 172.21.15.16:0/455297240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd300aa580 con 0x7efd3015cb30 2024-04-25T10:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.048+0000 7efd34e6b640 1 -- 172.21.15.16:0/455297240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd300aab40 con 0x7efd3015cb30 2024-04-25T10:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.049+0000 7efd34e6b640 1 -- 172.21.15.16:0/455297240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd300750b0 con 0x7efd3015cb30 2024-04-25T10:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.049+0000 7efd1effd640 1 -- 172.21.15.16:0/455297240 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd2000cbb0 con 0x7efd3015cb30 2024-04-25T10:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.049+0000 7efd1effd640 1 --2- 172.21.15.16:0/455297240 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efd1006b4d0 0x7efd1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.050+0000 7efd1effd640 1 -- 172.21.15.16:0/455297240 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5029+0+0 (secure 0 0 0) 0x7efd20089970 con 0x7efd3015cb30 2024-04-25T10:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.050+0000 7efd2e59c640 1 --2- 172.21.15.16:0/455297240 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efd1006b4d0 0x7efd1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.050+0000 7efd2e59c640 1 --2- 172.21.15.16:0/455297240 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efd1006b4d0 0x7efd1006d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efd28006fd0 tx=0x7efd28008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:54.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.054+0000 7efd1effd640 1 -- 172.21.15.16:0/455297240 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd200584d0 con 0x7efd3015cb30 2024-04-25T10:10:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.234+0000 7efd34e6b640 1 -- 172.21.15.16:0/455297240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd300d1780 con 0x7efd3015cb30 2024-04-25T10:10:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.235+0000 7efd1effd640 1 -- 172.21.15.16:0/455297240 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd2005c570 con 0x7efd3015cb30 2024-04-25T10:10:54.240 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:10:54.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.237+0000 7efd34e6b640 1 -- 172.21.15.16:0/455297240 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efd1006b4d0 msgr2=0x7efd1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.237+0000 7efd34e6b640 1 --2- 172.21.15.16:0/455297240 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efd1006b4d0 0x7efd1006d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efd28006fd0 tx=0x7efd28008040 comp rx=0 tx=0).stop 2024-04-25T10:10:54.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.237+0000 7efd34e6b640 1 -- 172.21.15.16:0/455297240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3015cb30 msgr2=0x7efd300b2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.237+0000 7efd34e6b640 1 --2- 172.21.15.16:0/455297240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3015cb30 0x7efd300b2450 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7efd20000c00 tx=0x7efd2000eb90 comp rx=0 tx=0).stop 2024-04-25T10:10:54.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.237+0000 7efd34e6b640 1 -- 172.21.15.16:0/455297240 shutdown_connections 2024-04-25T10:10:54.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.237+0000 7efd34e6b640 1 --2- 172.21.15.16:0/455297240 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efd1006b4d0 0x7efd1006d990 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.237+0000 7efd34e6b640 1 --2- 172.21.15.16:0/455297240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3015cb30 0x7efd300b2450 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.237+0000 7efd34e6b640 1 --2- 172.21.15.16:0/455297240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd3015c6b0 0x7efd300b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.237+0000 7efd34e6b640 1 --2- 172.21.15.16:0/455297240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd18003f30 0x7efd300cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.237+0000 7efd34e6b640 1 -- 172.21.15.16:0/455297240 >> 172.21.15.16:0/455297240 conn(0x7efd300d04f0 msgr2=0x7efd300b7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:54.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.237+0000 7efd34e6b640 1 -- 172.21.15.16:0/455297240 shutdown_connections 2024-04-25T10:10:54.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.237+0000 7efd34e6b640 1 -- 172.21.15.16:0/455297240 wait complete. 2024-04-25T10:10:54.253 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-25T10:10:54.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.349+0000 7fbd71f89640 1 Processor -- start 2024-04-25T10:10:54.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.349+0000 7fbd71f89640 1 -- start start 2024-04-25T10:10:54.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.350+0000 7fbd71f89640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbd6c15bf00 0x7fbd6c0cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.350+0000 7fbd71f89640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd6c06e8f0 0x7fbd6c1654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.350+0000 7fbd71f89640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd6c0ce6c0 con 0x7fbd6c0cc310 2024-04-25T10:10:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.350+0000 7fbd71f89640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6c1455e0 con 0x7fbd6c15bf00 2024-04-25T10:10:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.350+0000 7fbd71f89640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c06d450 con 0x7fbd6c06e8f0 2024-04-25T10:10:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.350+0000 7fbd6affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbd6c0cc310 0x7fbd6c15c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46236/0 (socket says 172.21.15.16:46236) 2024-04-25T10:10:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.350+0000 7fbd6affd640 1 -- 172.21.15.16:0/308396554 learned_addr learned my addr 172.21.15.16:0/308396554 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.350+0000 7fbd6bfff640 1 --2- 172.21.15.16:0/308396554 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd6c06e8f0 0x7fbd6c1654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.350+0000 7fbd6b7fe640 1 --2- 172.21.15.16:0/308396554 >> v2:172.21.15.186:3300/0 conn(0x7fbd6c15bf00 0x7fbd6c0cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.350+0000 7fbd6a7fc640 1 -- 172.21.15.16:0/308396554 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957263152 0 0) 0x7fbd5c001110 con 0x7fbd6c0cc310 2024-04-25T10:10:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.350+0000 7fbd6a7fc640 1 -- 172.21.15.16:0/308396554 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd54003520 con 0x7fbd6c0cc310 2024-04-25T10:10:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.350+0000 7fbd6b7fe640 1 -- 172.21.15.16:0/308396554 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd6c06e8f0 msgr2=0x7fbd6c1654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.350+0000 7fbd6b7fe640 1 --2- 172.21.15.16:0/308396554 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd6c06e8f0 0x7fbd6c1654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.350+0000 7fbd6b7fe640 1 -- 172.21.15.16:0/308396554 >> v1:172.21.15.16:6789/0 conn(0x7fbd6c0cc310 legacy=0x7fbd6c15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.350+0000 7fbd6b7fe640 1 -- 172.21.15.16:0/308396554 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c0cc7b0 con 0x7fbd6c15bf00 2024-04-25T10:10:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.350+0000 7fbd6bfff640 1 --2- 172.21.15.16:0/308396554 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd6c06e8f0 0x7fbd6c1654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:10:54.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.351+0000 7fbd6b7fe640 1 --2- 172.21.15.16:0/308396554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd6c15bf00 0x7fbd6c0cbdd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbd600097a0 tx=0x7fbd6002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb657bb319a67065 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:54.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.351+0000 7fbd6a7fc640 1 -- 172.21.15.16:0/308396554 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60037070 con 0x7fbd6c15bf00 2024-04-25T10:10:54.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.351+0000 7fbd6a7fc640 1 -- 172.21.15.16:0/308396554 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd60032440 con 0x7fbd6c15bf00 2024-04-25T10:10:54.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.351+0000 7fbd6a7fc640 1 -- 172.21.15.16:0/308396554 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd600327f0 con 0x7fbd6c15bf00 2024-04-25T10:10:54.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.352+0000 7fbd71f89640 1 -- 172.21.15.16:0/308396554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd6c15bf00 msgr2=0x7fbd6c0cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.352+0000 7fbd71f89640 1 --2- 172.21.15.16:0/308396554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd6c15bf00 0x7fbd6c0cbdd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbd600097a0 tx=0x7fbd6002a180 comp rx=0 tx=0).stop 2024-04-25T10:10:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.352+0000 7fbd71f89640 1 -- 172.21.15.16:0/308396554 shutdown_connections 2024-04-25T10:10:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.352+0000 7fbd71f89640 1 --2- 172.21.15.16:0/308396554 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd6c06e8f0 0x7fbd6c1654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.352+0000 7fbd71f89640 1 --2- 172.21.15.16:0/308396554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd6c15bf00 0x7fbd6c0cbdd0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.352+0000 7fbd71f89640 1 -- 172.21.15.16:0/308396554 >> 172.21.15.16:0/308396554 conn(0x7fbd6c06aba0 msgr2=0x7fbd6c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.352+0000 7fbd71f89640 1 -- 172.21.15.16:0/308396554 shutdown_connections 2024-04-25T10:10:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.352+0000 7fbd71f89640 1 -- 172.21.15.16:0/308396554 wait complete. 2024-04-25T10:10:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.353+0000 7fbd71f89640 1 Processor -- start 2024-04-25T10:10:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.353+0000 7fbd71f89640 1 -- start start 2024-04-25T10:10:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.354+0000 7fbd71f89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd6c06e8f0 0x7fbd6c156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.354+0000 7fbd71f89640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd6c0cc310 0x7fbd6c140960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.354+0000 7fbd71f89640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd6c15bf00 0x7fbd6c1f19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.354+0000 7fbd71f89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c0cd050 con 0x7fbd6c06e8f0 2024-04-25T10:10:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.354+0000 7fbd71f89640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c0ccd70 con 0x7fbd6c0cc310 2024-04-25T10:10:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.354+0000 7fbd71f89640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c06ee60 con 0x7fbd6c15bf00 2024-04-25T10:10:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.354+0000 7fbd6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd6c06e8f0 0x7fbd6c156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.354+0000 7fbd6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd6c06e8f0 0x7fbd6c156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55260/0 (socket says 172.21.15.16:55260) 2024-04-25T10:10:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.354+0000 7fbd6b7fe640 1 -- 172.21.15.16:0/4279837719 learned_addr learned my addr 172.21.15.16:0/4279837719 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.354+0000 7fbd6affd640 1 --2- 172.21.15.16:0/4279837719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd6c0cc310 0x7fbd6c140960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.354+0000 7fbd6bfff640 1 --2- 172.21.15.16:0/4279837719 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd6c15bf00 0x7fbd6c1f19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.354+0000 7fbd6b7fe640 1 -- 172.21.15.16:0/4279837719 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd6c15bf00 msgr2=0x7fbd6c1f19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.354+0000 7fbd6b7fe640 1 --2- 172.21.15.16:0/4279837719 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd6c15bf00 0x7fbd6c1f19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.354+0000 7fbd6b7fe640 1 -- 172.21.15.16:0/4279837719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd6c0cc310 msgr2=0x7fbd6c140960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.354+0000 7fbd6b7fe640 1 --2- 172.21.15.16:0/4279837719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd6c0cc310 0x7fbd6c140960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.354+0000 7fbd6b7fe640 1 -- 172.21.15.16:0/4279837719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c1f2540 con 0x7fbd6c06e8f0 2024-04-25T10:10:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.354+0000 7fbd6bfff640 1 --2- 172.21.15.16:0/4279837719 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd6c15bf00 0x7fbd6c1f19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.354+0000 7fbd6affd640 1 --2- 172.21.15.16:0/4279837719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd6c0cc310 0x7fbd6c140960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.355+0000 7fbd6b7fe640 1 --2- 172.21.15.16:0/4279837719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd6c06e8f0 0x7fbd6c156dd0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fbd6002a1c0 tx=0x7fbd600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.355+0000 7fbd53fff640 1 -- 172.21.15.16:0/4279837719 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60037040 con 0x7fbd6c06e8f0 2024-04-25T10:10:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.355+0000 7fbd53fff640 1 -- 172.21.15.16:0/4279837719 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd60042070 con 0x7fbd6c06e8f0 2024-04-25T10:10:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.355+0000 7fbd71f89640 1 -- 172.21.15.16:0/4279837719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6c1f2820 con 0x7fbd6c06e8f0 2024-04-25T10:10:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.355+0000 7fbd53fff640 1 -- 172.21.15.16:0/4279837719 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6003b930 con 0x7fbd6c06e8f0 2024-04-25T10:10:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.355+0000 7fbd71f89640 1 -- 172.21.15.16:0/4279837719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6c13c5c0 con 0x7fbd6c06e8f0 2024-04-25T10:10:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.356+0000 7fbd71f89640 1 -- 172.21.15.16:0/4279837719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6c06bb30 con 0x7fbd6c06e8f0 2024-04-25T10:10:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.356+0000 7fbd53fff640 1 -- 172.21.15.16:0/4279837719 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd6003bad0 con 0x7fbd6c06e8f0 2024-04-25T10:10:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.357+0000 7fbd53fff640 1 --2- 172.21.15.16:0/4279837719 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbd5806b4d0 0x7fbd5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.357+0000 7fbd53fff640 1 -- 172.21.15.16:0/4279837719 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fbd600aa8f0 con 0x7fbd6c06e8f0 2024-04-25T10:10:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.357+0000 7fbd6affd640 1 --2- 172.21.15.16:0/4279837719 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbd5806b4d0 0x7fbd5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.358+0000 7fbd6affd640 1 --2- 172.21.15.16:0/4279837719 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbd5806b4d0 0x7fbd5806d990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0012b0 tx=0x7fbd5c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.361+0000 7fbd53fff640 1 -- 172.21.15.16:0/4279837719 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd60079970 con 0x7fbd6c06e8f0 2024-04-25T10:10:54.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.610+0000 7fbd71f89640 1 -- 172.21.15.16:0/4279837719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7fbd6c15edf0 con 0x7fbd6c06e8f0 2024-04-25T10:10:54.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.618+0000 7fbd53fff640 1 -- 172.21.15.16:0/4279837719 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v290) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbd60079270 con 0x7fbd6c06e8f0 2024-04-25T10:10:54.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.620+0000 7fbd71f89640 1 -- 172.21.15.16:0/4279837719 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbd5806b4d0 msgr2=0x7fbd5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.620+0000 7fbd71f89640 1 --2- 172.21.15.16:0/4279837719 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbd5806b4d0 0x7fbd5806d990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0012b0 tx=0x7fbd5c009040 comp rx=0 tx=0).stop 2024-04-25T10:10:54.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.620+0000 7fbd71f89640 1 -- 172.21.15.16:0/4279837719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd6c06e8f0 msgr2=0x7fbd6c156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.620+0000 7fbd71f89640 1 --2- 172.21.15.16:0/4279837719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd6c06e8f0 0x7fbd6c156dd0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fbd6002a1c0 tx=0x7fbd600046c0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.620+0000 7fbd71f89640 1 -- 172.21.15.16:0/4279837719 shutdown_connections 2024-04-25T10:10:54.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.620+0000 7fbd71f89640 1 --2- 172.21.15.16:0/4279837719 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbd5806b4d0 0x7fbd5806d990 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.620+0000 7fbd71f89640 1 --2- 172.21.15.16:0/4279837719 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd6c15bf00 0x7fbd6c1f19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.620+0000 7fbd71f89640 1 --2- 172.21.15.16:0/4279837719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd6c0cc310 0x7fbd6c140960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.620+0000 7fbd71f89640 1 --2- 172.21.15.16:0/4279837719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd6c06e8f0 0x7fbd6c156dd0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.620+0000 7fbd71f89640 1 -- 172.21.15.16:0/4279837719 >> 172.21.15.16:0/4279837719 conn(0x7fbd6c06aba0 msgr2=0x7fbd6c071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:54.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.620+0000 7fbd71f89640 1 -- 172.21.15.16:0/4279837719 shutdown_connections 2024-04-25T10:10:54.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.620+0000 7fbd71f89640 1 -- 172.21.15.16:0/4279837719 wait complete. 2024-04-25T10:10:54.636 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:10:54.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.735+0000 7f7a59cfc640 1 Processor -- start 2024-04-25T10:10:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.735+0000 7f7a59cfc640 1 -- start start 2024-04-25T10:10:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.736+0000 7f7a59cfc640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7a5415dca0 0x7f7a540ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.736+0000 7f7a59cfc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a540d3b90 0x7f7a540d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.736+0000 7f7a59cfc640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a540d4090 con 0x7f7a540cee80 2024-04-25T10:10:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.736+0000 7f7a59cfc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a540b6b20 con 0x7f7a540d3b90 2024-04-25T10:10:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.736+0000 7f7a59cfc640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a540d4370 con 0x7f7a5415dca0 2024-04-25T10:10:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.736+0000 7f7a52ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a540d3b90 0x7f7a540d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.736+0000 7f7a52ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a540d3b90 0x7f7a540d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55264/0 (socket says 172.21.15.16:55264) 2024-04-25T10:10:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.736+0000 7f7a52ffd640 1 -- 172.21.15.16:0/1793515916 learned_addr learned my addr 172.21.15.16:0/1793515916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.736+0000 7f7a537fe640 1 --2- 172.21.15.16:0/1793515916 >> v2:172.21.15.186:3300/0 conn(0x7f7a5415dca0 0x7f7a540ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.737+0000 7f7a52ffd640 1 -- 172.21.15.16:0/1793515916 >> v2:172.21.15.186:3300/0 conn(0x7f7a5415dca0 msgr2=0x7f7a540ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:54.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.737+0000 7f7a52ffd640 1 --2- 172.21.15.16:0/1793515916 >> v2:172.21.15.186:3300/0 conn(0x7f7a5415dca0 0x7f7a540ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.737+0000 7f7a52ffd640 1 -- 172.21.15.16:0/1793515916 >> v1:172.21.15.186:6789/0 conn(0x7f7a540cee80 legacy=0x7f7a540b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.737+0000 7f7a52ffd640 1 -- 172.21.15.16:0/1793515916 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a540d4650 con 0x7f7a540d3b90 2024-04-25T10:10:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.737+0000 7f7a52ffd640 1 --2- 172.21.15.16:0/1793515916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a540d3b90 0x7f7a540d0a20 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f7a540b6340 tx=0x7f7a4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4a95eac44900bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.737+0000 7f7a527fc640 1 -- 172.21.15.16:0/1793515916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a48037070 con 0x7f7a540d3b90 2024-04-25T10:10:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.737+0000 7f7a527fc640 1 -- 172.21.15.16:0/1793515916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a48032440 con 0x7f7a540d3b90 2024-04-25T10:10:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.737+0000 7f7a527fc640 1 -- 172.21.15.16:0/1793515916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a48032810 con 0x7f7a540d3b90 2024-04-25T10:10:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.738+0000 7f7a59cfc640 1 -- 172.21.15.16:0/1793515916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a540d3b90 msgr2=0x7f7a540d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.738+0000 7f7a59cfc640 1 --2- 172.21.15.16:0/1793515916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a540d3b90 0x7f7a540d0a20 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f7a540b6340 tx=0x7f7a4802a270 comp rx=0 tx=0).stop 2024-04-25T10:10:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.738+0000 7f7a59cfc640 1 -- 172.21.15.16:0/1793515916 shutdown_connections 2024-04-25T10:10:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.738+0000 7f7a59cfc640 1 --2- 172.21.15.16:0/1793515916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a540d3b90 0x7f7a540d0a20 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.738+0000 7f7a59cfc640 1 --2- 172.21.15.16:0/1793515916 >> v2:172.21.15.186:3300/0 conn(0x7f7a5415dca0 0x7f7a540ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.738+0000 7f7a59cfc640 1 -- 172.21.15.16:0/1793515916 >> 172.21.15.16:0/1793515916 conn(0x7f7a540726c0 msgr2=0x7f7a54072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.738+0000 7f7a59cfc640 1 -- 172.21.15.16:0/1793515916 shutdown_connections 2024-04-25T10:10:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.738+0000 7f7a59cfc640 1 -- 172.21.15.16:0/1793515916 wait complete. 2024-04-25T10:10:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.739+0000 7f7a59cfc640 1 Processor -- start 2024-04-25T10:10:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.739+0000 7f7a59cfc640 1 -- start start 2024-04-25T10:10:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.740+0000 7f7a59cfc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a540cee80 0x7f7a541f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.740+0000 7f7a59cfc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a540d3b90 0x7f7a54157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.740+0000 7f7a59cfc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a5415dca0 0x7f7a541583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.740+0000 7f7a59cfc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5415e200 con 0x7f7a5415dca0 2024-04-25T10:10:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.740+0000 7f7a59cfc640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5415e080 con 0x7f7a540d3b90 2024-04-25T10:10:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.740+0000 7f7a59cfc640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a540d0f60 con 0x7f7a540cee80 2024-04-25T10:10:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.740+0000 7f7a53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a5415dca0 0x7f7a541583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.740+0000 7f7a53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a5415dca0 0x7f7a541583a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55276/0 (socket says 172.21.15.16:55276) 2024-04-25T10:10:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.740+0000 7f7a53fff640 1 -- 172.21.15.16:0/2131647340 learned_addr learned my addr 172.21.15.16:0/2131647340 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.740+0000 7f7a52ffd640 1 --2- 172.21.15.16:0/2131647340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a540d3b90 0x7f7a54157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.740+0000 7f7a537fe640 1 --2- 172.21.15.16:0/2131647340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a540cee80 0x7f7a541f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.741+0000 7f7a53fff640 1 -- 172.21.15.16:0/2131647340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a540cee80 msgr2=0x7f7a541f38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.741+0000 7f7a53fff640 1 --2- 172.21.15.16:0/2131647340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a540cee80 0x7f7a541f38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.741+0000 7f7a53fff640 1 -- 172.21.15.16:0/2131647340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a540d3b90 msgr2=0x7f7a54157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.741+0000 7f7a53fff640 1 --2- 172.21.15.16:0/2131647340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a540d3b90 0x7f7a54157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.741+0000 7f7a53fff640 1 -- 172.21.15.16:0/2131647340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a540c8560 con 0x7f7a5415dca0 2024-04-25T10:10:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.741+0000 7f7a52ffd640 1 --2- 172.21.15.16:0/2131647340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a540d3b90 0x7f7a54157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.741+0000 7f7a53fff640 1 --2- 172.21.15.16:0/2131647340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a5415dca0 0x7f7a541583a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00c7a0 tx=0x7f7a4c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.741+0000 7f7a3bfff640 1 -- 172.21.15.16:0/2131647340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c003b70 con 0x7f7a5415dca0 2024-04-25T10:10:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.741+0000 7f7a3bfff640 1 -- 172.21.15.16:0/2131647340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c003d30 con 0x7f7a5415dca0 2024-04-25T10:10:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.741+0000 7f7a59cfc640 1 -- 172.21.15.16:0/2131647340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a54160580 con 0x7f7a5415dca0 2024-04-25T10:10:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.741+0000 7f7a3bfff640 1 -- 172.21.15.16:0/2131647340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c00aa90 con 0x7f7a5415dca0 2024-04-25T10:10:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.741+0000 7f7a59cfc640 1 -- 172.21.15.16:0/2131647340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a540c8ae0 con 0x7f7a5415dca0 2024-04-25T10:10:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.742+0000 7f7a59cfc640 1 -- 172.21.15.16:0/2131647340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a540750b0 con 0x7f7a5415dca0 2024-04-25T10:10:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.742+0000 7f7a3bfff640 1 -- 172.21.15.16:0/2131647340 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a4c00ac30 con 0x7f7a5415dca0 2024-04-25T10:10:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.743+0000 7f7a3bfff640 1 --2- 172.21.15.16:0/2131647340 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a4006b4d0 0x7f7a4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.743+0000 7f7a3bfff640 1 -- 172.21.15.16:0/2131647340 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f7a4c089eb0 con 0x7f7a5415dca0 2024-04-25T10:10:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.743+0000 7f7a537fe640 1 --2- 172.21.15.16:0/2131647340 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a4006b4d0 0x7f7a4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.744+0000 7f7a537fe640 1 --2- 172.21.15.16:0/2131647340 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a4006b4d0 0x7f7a4006d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7a540ca7a0 tx=0x7f7a44008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:54.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.747+0000 7f7a3bfff640 1 -- 172.21.15.16:0/2131647340 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a4c058fd0 con 0x7f7a5415dca0 2024-04-25T10:10:54.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.932+0000 7f7a59cfc640 1 -- 172.21.15.16:0/2131647340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7a540cdb90 con 0x7f7a5415dca0 2024-04-25T10:10:54.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.935+0000 7f7a3bfff640 1 -- 172.21.15.16:0/2131647340 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7a4c0588d0 con 0x7f7a5415dca0 2024-04-25T10:10:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.937+0000 7f7a59cfc640 1 -- 172.21.15.16:0/2131647340 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a4006b4d0 msgr2=0x7f7a4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.937+0000 7f7a59cfc640 1 --2- 172.21.15.16:0/2131647340 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a4006b4d0 0x7f7a4006d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7a540ca7a0 tx=0x7f7a44008040 comp rx=0 tx=0).stop 2024-04-25T10:10:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.937+0000 7f7a59cfc640 1 -- 172.21.15.16:0/2131647340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a5415dca0 msgr2=0x7f7a541583a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.937+0000 7f7a59cfc640 1 --2- 172.21.15.16:0/2131647340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a5415dca0 0x7f7a541583a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00c7a0 tx=0x7f7a4c00cc60 comp rx=0 tx=0).stop 2024-04-25T10:10:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.937+0000 7f7a59cfc640 1 -- 172.21.15.16:0/2131647340 shutdown_connections 2024-04-25T10:10:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.937+0000 7f7a59cfc640 1 --2- 172.21.15.16:0/2131647340 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7a4006b4d0 0x7f7a4006d990 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.937+0000 7f7a59cfc640 1 --2- 172.21.15.16:0/2131647340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a5415dca0 0x7f7a541583a0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.937+0000 7f7a59cfc640 1 --2- 172.21.15.16:0/2131647340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a540d3b90 0x7f7a54157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.937+0000 7f7a59cfc640 1 --2- 172.21.15.16:0/2131647340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7a540cee80 0x7f7a541f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.937+0000 7f7a59cfc640 1 -- 172.21.15.16:0/2131647340 >> 172.21.15.16:0/2131647340 conn(0x7f7a540726c0 msgr2=0x7f7a540d1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:54.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.938+0000 7f7a59cfc640 1 -- 172.21.15.16:0/2131647340 shutdown_connections 2024-04-25T10:10:54.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:54.938+0000 7f7a59cfc640 1 -- 172.21.15.16:0/2131647340 wait complete. 2024-04-25T10:10:54.954 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:10:54.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:10:55.005 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:10:55.055 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:10:55.055 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:10:55.055 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:55.070 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:10:55.071 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:10:55.071 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:55.090 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:10:55.090 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:10:55.090 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:55.127 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:10:55.127 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:10:55.127 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:55.147 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:10:55.147 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:10:55.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:55.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.263+0000 7f5c2c89e640 1 Processor -- start 2024-04-25T10:10:55.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.263+0000 7f5c2c89e640 1 -- start start 2024-04-25T10:10:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.264+0000 7f5c2c89e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5c2815ba10 0x7f5c28099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.264+0000 7f5c2c89e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c2806c900 con 0x7f5c2806c3c0 2024-04-25T10:10:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.264+0000 7f5c2c89e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c2806cbe0 con 0x7f5c28099f80 2024-04-25T10:10:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.264+0000 7f5c2c89e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c280cbe50 con 0x7f5c2815ba10 2024-04-25T10:10:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.264+0000 7f5c26d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5c2806c3c0 0x7f5c280cd220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46238/0 (socket says 172.21.15.16:46238) 2024-04-25T10:10:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.264+0000 7f5c26d9d640 1 -- 172.21.15.16:0/3609309111 learned_addr learned my addr 172.21.15.16:0/3609309111 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.264+0000 7f5c2559a640 1 -- 172.21.15.16:0/3609309111 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473198040 0 0) 0x7f5c2806c900 con 0x7f5c2806c3c0 2024-04-25T10:10:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.264+0000 7f5c2559a640 1 -- 172.21.15.16:0/3609309111 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c10003520 con 0x7f5c2806c3c0 2024-04-25T10:10:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c2659c640 1 --2- 172.21.15.16:0/3609309111 >> v2:172.21.15.186:3300/0 conn(0x7f5c2815ba10 0x7f5c28099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c2559a640 1 -- 172.21.15.16:0/3609309111 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1249509494 0 0) 0x7f5c10003520 con 0x7f5c2806c3c0 2024-04-25T10:10:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c2559a640 1 -- 172.21.15.16:0/3609309111 >> v2:172.21.15.186:3300/0 conn(0x7f5c2815ba10 msgr2=0x7f5c28099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c2559a640 1 --2- 172.21.15.16:0/3609309111 >> v2:172.21.15.186:3300/0 conn(0x7f5c2815ba10 0x7f5c28099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c2559a640 1 -- 172.21.15.16:0/3609309111 >> v1:172.21.15.186:6789/0 conn(0x7f5c28099f80 legacy=0x7f5c2806bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c2559a640 1 -- 172.21.15.16:0/3609309111 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2806cec0 con 0x7f5c2806c3c0 2024-04-25T10:10:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c2559a640 1 -- 172.21.15.16:0/3609309111 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f5c180019f0 con 0x7f5c2806c3c0 2024-04-25T10:10:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c2559a640 1 -- 172.21.15.16:0/3609309111 >> v1:172.21.15.16:6789/0 conn(0x7f5c2806c3c0 legacy=0x7f5c280cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c2559a640 1 --2- 172.21.15.16:0/3609309111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c10003a00 0x7f5c10003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c2559a640 1 --2- 172.21.15.16:0/3609309111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c10004800 0x7f5c10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c2559a640 1 --2- 172.21.15.16:0/3609309111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c100051c0 0x7f5c1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c2559a640 1 -- 172.21.15.16:0/3609309111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1000c090 con 0x7f5c100051c0 2024-04-25T10:10:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c2559a640 1 -- 172.21.15.16:0/3609309111 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1000c240 con 0x7f5c10003a00 2024-04-25T10:10:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c2559a640 1 -- 172.21.15.16:0/3609309111 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1000c440 con 0x7f5c10004800 2024-04-25T10:10:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c26d9d640 1 --2- 172.21.15.16:0/3609309111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c100051c0 0x7f5c1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c2659c640 1 --2- 172.21.15.16:0/3609309111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c10003a00 0x7f5c10003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c25d9b640 1 --2- 172.21.15.16:0/3609309111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c10004800 0x7f5c10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c26d9d640 1 -- 172.21.15.16:0/3609309111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c10004800 msgr2=0x7f5c10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c26d9d640 1 --2- 172.21.15.16:0/3609309111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c10004800 0x7f5c10004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c26d9d640 1 -- 172.21.15.16:0/3609309111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c10003a00 msgr2=0x7f5c10003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c26d9d640 1 --2- 172.21.15.16:0/3609309111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c10003a00 0x7f5c10003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.265+0000 7f5c26d9d640 1 -- 172.21.15.16:0/3609309111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c10003520 con 0x7f5c100051c0 2024-04-25T10:10:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.266+0000 7f5c26d9d640 1 -- 172.21.15.16:0/3609309111 reap_dead start 2024-04-25T10:10:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.266+0000 7f5c26d9d640 1 --2- 172.21.15.16:0/3609309111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c100051c0 0x7f5c1000b9c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800a780 tx=0x7f5c10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612b6c9acf222013 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.266+0000 7f5c2559a640 1 -- 172.21.15.16:0/3609309111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1800a3d0 con 0x7f5c100051c0 2024-04-25T10:10:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.266+0000 7f5c2559a640 1 -- 172.21.15.16:0/3609309111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1802f070 con 0x7f5c100051c0 2024-04-25T10:10:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.266+0000 7f5c2559a640 1 -- 172.21.15.16:0/3609309111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1802bc00 con 0x7f5c100051c0 2024-04-25T10:10:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.267+0000 7f5c2c89e640 1 -- 172.21.15.16:0/3609309111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c100051c0 msgr2=0x7f5c1000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.267+0000 7f5c2c89e640 1 --2- 172.21.15.16:0/3609309111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c100051c0 0x7f5c1000b9c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800a780 tx=0x7f5c10005830 comp rx=0 tx=0).stop 2024-04-25T10:10:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.267+0000 7f5c2c89e640 1 -- 172.21.15.16:0/3609309111 shutdown_connections 2024-04-25T10:10:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.267+0000 7f5c2c89e640 1 --2- 172.21.15.16:0/3609309111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c100051c0 0x7f5c1000b9c0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.267+0000 7f5c2c89e640 1 -- 172.21.15.16:0/3609309111 >> 172.21.15.16:0/3609309111 conn(0x7f5c280b32f0 msgr2=0x7f5c280b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.267+0000 7f5c2c89e640 1 -- 172.21.15.16:0/3609309111 shutdown_connections 2024-04-25T10:10:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.267+0000 7f5c2c89e640 1 -- 172.21.15.16:0/3609309111 wait complete. 2024-04-25T10:10:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.268+0000 7f5c2c89e640 1 Processor -- start 2024-04-25T10:10:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.268+0000 7f5c2c89e640 1 -- start start 2024-04-25T10:10:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.269+0000 7f5c2c89e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c100051c0 0x7f5c280c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.269+0000 7f5c2c89e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c2815f2a0 0x7f5c2815f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.269+0000 7f5c2c89e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c2815dad0 0x7f5c2815df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.269+0000 7f5c2c89e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c28099dd0 con 0x7f5c2815f2a0 2024-04-25T10:10:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.269+0000 7f5c2c89e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c28099c50 con 0x7f5c2815dad0 2024-04-25T10:10:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.269+0000 7f5c2c89e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2809a3d0 con 0x7f5c100051c0 2024-04-25T10:10:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.269+0000 7f5c25d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c2815f2a0 0x7f5c2815f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.269+0000 7f5c25d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c2815f2a0 0x7f5c2815f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55290/0 (socket says 172.21.15.16:55290) 2024-04-25T10:10:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.269+0000 7f5c25d9b640 1 -- 172.21.15.16:0/2238824442 learned_addr learned my addr 172.21.15.16:0/2238824442 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.269+0000 7f5c2659c640 1 --2- 172.21.15.16:0/2238824442 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c100051c0 0x7f5c280c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.269+0000 7f5c26d9d640 1 --2- 172.21.15.16:0/2238824442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c2815dad0 0x7f5c2815df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.269+0000 7f5c25d9b640 1 -- 172.21.15.16:0/2238824442 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c100051c0 msgr2=0x7f5c280c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.269+0000 7f5c25d9b640 1 --2- 172.21.15.16:0/2238824442 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c100051c0 0x7f5c280c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.269+0000 7f5c25d9b640 1 -- 172.21.15.16:0/2238824442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c2815dad0 msgr2=0x7f5c2815df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.269+0000 7f5c25d9b640 1 --2- 172.21.15.16:0/2238824442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c2815dad0 0x7f5c2815df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.269+0000 7f5c25d9b640 1 -- 172.21.15.16:0/2238824442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c280c81f0 con 0x7f5c2815f2a0 2024-04-25T10:10:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.269+0000 7f5c25d9b640 1 --2- 172.21.15.16:0/2238824442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c2815f2a0 0x7f5c2815f720 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00e4b0 tx=0x7f5c1c00e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.270+0000 7f5c16ffd640 1 -- 172.21.15.16:0/2238824442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c0052e0 con 0x7f5c2815f2a0 2024-04-25T10:10:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.270+0000 7f5c16ffd640 1 -- 172.21.15.16:0/2238824442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1c018070 con 0x7f5c2815f2a0 2024-04-25T10:10:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.270+0000 7f5c2c89e640 1 -- 172.21.15.16:0/2238824442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c281f75a0 con 0x7f5c2815f2a0 2024-04-25T10:10:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.270+0000 7f5c16ffd640 1 -- 172.21.15.16:0/2238824442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c003830 con 0x7f5c2815f2a0 2024-04-25T10:10:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.270+0000 7f5c2c89e640 1 -- 172.21.15.16:0/2238824442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c281f7b40 con 0x7f5c2815f2a0 2024-04-25T10:10:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.271+0000 7f5c2c89e640 1 -- 172.21.15.16:0/2238824442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c280749b0 con 0x7f5c2815f2a0 2024-04-25T10:10:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.271+0000 7f5c16ffd640 1 -- 172.21.15.16:0/2238824442 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c1c0039d0 con 0x7f5c2815f2a0 2024-04-25T10:10:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.272+0000 7f5c16ffd640 1 --2- 172.21.15.16:0/2238824442 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5c0806b4d0 0x7f5c0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.272+0000 7f5c16ffd640 1 -- 172.21.15.16:0/2238824442 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f5c1c089b50 con 0x7f5c2815f2a0 2024-04-25T10:10:55.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.272+0000 7f5c2659c640 1 --2- 172.21.15.16:0/2238824442 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5c0806b4d0 0x7f5c0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.272+0000 7f5c2659c640 1 --2- 172.21.15.16:0/2238824442 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5c0806b4d0 0x7f5c0806d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009a80 tx=0x7f5c20008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.276+0000 7f5c16ffd640 1 -- 172.21.15.16:0/2238824442 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c1c058c70 con 0x7f5c2815f2a0 2024-04-25T10:10:55.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.492+0000 7f5c2c89e640 1 -- 172.21.15.16:0/2238824442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c2809a570 con 0x7f5c2815f2a0 2024-04-25T10:10:55.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.492+0000 7f5c16ffd640 1 -- 172.21.15.16:0/2238824442 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f5c1c058570 con 0x7f5c2815f2a0 2024-04-25T10:10:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:55.498 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6070,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2805428272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2805428272},{"type":"v1","addr":"172.21.15.186:6835","nonce":2805428272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6178,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1052667029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1052667029},{"type":"v1","addr":"172.21.15.186:6837","nonce":1052667029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2897154125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2897154125},{"type":"v1","addr":"172.21.15.16:6833","nonce":2897154125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":7254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/881985120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":881985120},{"type":"v1","addr":"172.21.15.16:6835","nonce":881985120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[]} 2024-04-25T10:10:55.499 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 46 2024-04-25T10:10:55.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.494+0000 7f5c2c89e640 1 -- 172.21.15.16:0/2238824442 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5c0806b4d0 msgr2=0x7f5c0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.494+0000 7f5c2c89e640 1 --2- 172.21.15.16:0/2238824442 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5c0806b4d0 0x7f5c0806d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009a80 tx=0x7f5c20008040 comp rx=0 tx=0).stop 2024-04-25T10:10:55.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.494+0000 7f5c2c89e640 1 -- 172.21.15.16:0/2238824442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c2815f2a0 msgr2=0x7f5c2815f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.494+0000 7f5c2c89e640 1 --2- 172.21.15.16:0/2238824442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c2815f2a0 0x7f5c2815f720 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00e4b0 tx=0x7f5c1c00e970 comp rx=0 tx=0).stop 2024-04-25T10:10:55.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.494+0000 7f5c2c89e640 1 -- 172.21.15.16:0/2238824442 shutdown_connections 2024-04-25T10:10:55.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.494+0000 7f5c2c89e640 1 --2- 172.21.15.16:0/2238824442 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5c0806b4d0 0x7f5c0806d990 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.494+0000 7f5c2c89e640 1 --2- 172.21.15.16:0/2238824442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c2815dad0 0x7f5c2815df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.494+0000 7f5c2c89e640 1 --2- 172.21.15.16:0/2238824442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c2815f2a0 0x7f5c2815f720 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.494+0000 7f5c2c89e640 1 --2- 172.21.15.16:0/2238824442 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c100051c0 0x7f5c280c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.494+0000 7f5c2c89e640 1 -- 172.21.15.16:0/2238824442 >> 172.21.15.16:0/2238824442 conn(0x7f5c280b32f0 msgr2=0x7f5c280d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:55.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.494+0000 7f5c2c89e640 1 -- 172.21.15.16:0/2238824442 shutdown_connections 2024-04-25T10:10:55.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.494+0000 7f5c2c89e640 1 -- 172.21.15.16:0/2238824442 wait complete. 2024-04-25T10:10:55.511 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:10:55.511 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:10:55.511 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:10:55.511 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:10:55.512 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:10:55.512 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:10:55.512 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:10:55.512 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:10:55.512 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:10:55.512 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:10:55.513 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:10:55.513 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:10:55.513 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:10:55.513 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:10:55.513 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:10:55.513 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:10:55.521 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:10:55.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:10:55.522 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:10:55.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:10:55.524 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:10:55.524 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:10:55.525 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:10:55.525 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:10:55.564 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:10:55.564 DEBUG:teuthology.parallel:result is None 2024-04-25T10:10:55.565 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:10:55.565 DEBUG:teuthology.parallel:result is None 2024-04-25T10:10:55.567 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:10:55.568 DEBUG:teuthology.parallel:result is None 2024-04-25T10:10:55.568 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:10:55.568 DEBUG:teuthology.parallel:result is None 2024-04-25T10:10:55.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:10:55.639 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-04-25T10:10:55.639 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-04-25T10:10:55.649 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2024-04-25T10:10:55.650 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2024-04-25T10:10:55.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.669+0000 7f3ac9d13640 1 Processor -- start 2024-04-25T10:10:55.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.669+0000 7f3ac9d13640 1 -- start start 2024-04-25T10:10:55.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.670+0000 7f3ac9d13640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ac40b9720 0x7f3ac40ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.670+0000 7f3ac9d13640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3ac40cece0 0x7f3ac40d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.670+0000 7f3ac9d13640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac4161f80 con 0x7f3ac4161090 2024-04-25T10:10:55.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.670+0000 7f3ac9d13640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac40cf130 con 0x7f3ac40cece0 2024-04-25T10:10:55.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.670+0000 7f3ac9d13640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac40d21a0 con 0x7f3ac40b9720 2024-04-25T10:10:55.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.670+0000 7f3ac3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3ac40cece0 0x7f3ac40d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.670+0000 7f3ac3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3ac40cece0 0x7f3ac40d2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55338/0 (socket says 172.21.15.16:55338) 2024-04-25T10:10:55.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.670+0000 7f3ac3fff640 1 -- 172.21.15.16:0/1333791294 learned_addr learned my addr 172.21.15.16:0/1333791294 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.671+0000 7f3ac2ffd640 1 --2- 172.21.15.16:0/1333791294 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ac40b9720 0x7f3ac40ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.671+0000 7f3ac3fff640 1 -- 172.21.15.16:0/1333791294 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ac40b9720 msgr2=0x7f3ac40ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.671+0000 7f3ac3fff640 1 --2- 172.21.15.16:0/1333791294 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ac40b9720 0x7f3ac40ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.671+0000 7f3ac3fff640 1 -- 172.21.15.16:0/1333791294 >> v1:172.21.15.186:6789/0 conn(0x7f3ac4161090 legacy=0x7f3ac4161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.671+0000 7f3ac3fff640 1 -- 172.21.15.16:0/1333791294 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac4162260 con 0x7f3ac40cece0 2024-04-25T10:10:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.671+0000 7f3ac2ffd640 1 --2- 172.21.15.16:0/1333791294 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ac40b9720 0x7f3ac40ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.672+0000 7f3ac3fff640 1 --2- 172.21.15.16:0/1333791294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ac40cece0 0x7f3ac40d2630 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009e50 tx=0x7f3abc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af80425add0a8084 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.672+0000 7f3ac27fc640 1 -- 172.21.15.16:0/1333791294 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc036070 con 0x7f3ac40cece0 2024-04-25T10:10:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.672+0000 7f3ac27fc640 1 -- 172.21.15.16:0/1333791294 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc031600 con 0x7f3ac40cece0 2024-04-25T10:10:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.672+0000 7f3ac27fc640 1 -- 172.21.15.16:0/1333791294 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc0319d0 con 0x7f3ac40cece0 2024-04-25T10:10:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.672+0000 7f3ac9d13640 1 -- 172.21.15.16:0/1333791294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ac40cece0 msgr2=0x7f3ac40d2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.672+0000 7f3ac9d13640 1 --2- 172.21.15.16:0/1333791294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ac40cece0 0x7f3ac40d2630 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009e50 tx=0x7f3abc02a710 comp rx=0 tx=0).stop 2024-04-25T10:10:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.672+0000 7f3ac9d13640 1 -- 172.21.15.16:0/1333791294 shutdown_connections 2024-04-25T10:10:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.672+0000 7f3ac9d13640 1 --2- 172.21.15.16:0/1333791294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ac40cece0 0x7f3ac40d2630 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.672+0000 7f3ac9d13640 1 --2- 172.21.15.16:0/1333791294 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ac40b9720 0x7f3ac40ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.672+0000 7f3ac9d13640 1 -- 172.21.15.16:0/1333791294 >> 172.21.15.16:0/1333791294 conn(0x7f3ac40b55e0 msgr2=0x7f3ac40b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:55.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.672+0000 7f3ac9d13640 1 -- 172.21.15.16:0/1333791294 shutdown_connections 2024-04-25T10:10:55.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.672+0000 7f3ac9d13640 1 -- 172.21.15.16:0/1333791294 wait complete. 2024-04-25T10:10:55.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.673+0000 7f3ac9d13640 1 Processor -- start 2024-04-25T10:10:55.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.674+0000 7f3ac9d13640 1 -- start start 2024-04-25T10:10:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.674+0000 7f3ac9d13640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ac40b9720 0x7f3ac4146e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.674+0000 7f3ac9d13640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ac40cece0 0x7f3ac40cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3ac9d13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ac4161090 0x7f3ac40d3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3ac9d13640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac4161470 con 0x7f3ac4161090 2024-04-25T10:10:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3ac9d13640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac40d1e30 con 0x7f3ac40b9720 2024-04-25T10:10:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3ac9d13640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac40d2b70 con 0x7f3ac40cece0 2024-04-25T10:10:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3ac3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ac4161090 0x7f3ac40d3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3ac3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ac4161090 0x7f3ac40d3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55350/0 (socket says 172.21.15.16:55350) 2024-04-25T10:10:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3ac3fff640 1 -- 172.21.15.16:0/3848675045 learned_addr learned my addr 172.21.15.16:0/3848675045 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3ac37fe640 1 --2- 172.21.15.16:0/3848675045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ac40b9720 0x7f3ac4146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3ac2ffd640 1 --2- 172.21.15.16:0/3848675045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ac40cece0 0x7f3ac40cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3ac3fff640 1 -- 172.21.15.16:0/3848675045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ac40cece0 msgr2=0x7f3ac40cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3ac3fff640 1 --2- 172.21.15.16:0/3848675045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ac40cece0 0x7f3ac40cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3ac3fff640 1 -- 172.21.15.16:0/3848675045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ac40b9720 msgr2=0x7f3ac4146e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3ac3fff640 1 --2- 172.21.15.16:0/3848675045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ac40b9720 0x7f3ac4146e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3ac3fff640 1 -- 172.21.15.16:0/3848675045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac4147710 con 0x7f3ac4161090 2024-04-25T10:10:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3ac3fff640 1 --2- 172.21.15.16:0/3848675045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ac4161090 0x7f3ac40d3a80 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3abc034040 tx=0x7f3abc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3aabfff640 1 -- 172.21.15.16:0/3848675045 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc036050 con 0x7f3ac4161090 2024-04-25T10:10:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3aabfff640 1 -- 172.21.15.16:0/3848675045 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc042070 con 0x7f3ac4161090 2024-04-25T10:10:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3ac9d13640 1 -- 172.21.15.16:0/3848675045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac41479f0 con 0x7f3ac4161090 2024-04-25T10:10:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3aabfff640 1 -- 172.21.15.16:0/3848675045 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc009510 con 0x7f3ac4161090 2024-04-25T10:10:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.675+0000 7f3ac9d13640 1 -- 172.21.15.16:0/3848675045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac40cac40 con 0x7f3ac4161090 2024-04-25T10:10:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.676+0000 7f3ac9d13640 1 -- 172.21.15.16:0/3848675045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac40750b0 con 0x7f3ac4161090 2024-04-25T10:10:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.677+0000 7f3aabfff640 1 -- 172.21.15.16:0/3848675045 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3abc007bd0 con 0x7f3ac4161090 2024-04-25T10:10:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.677+0000 7f3aabfff640 1 --2- 172.21.15.16:0/3848675045 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3ab006b4d0 0x7f3ab006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.677+0000 7f3aabfff640 1 -- 172.21.15.16:0/3848675045 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f3abc0aae30 con 0x7f3ac4161090 2024-04-25T10:10:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.678+0000 7f3ac37fe640 1 --2- 172.21.15.16:0/3848675045 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3ab006b4d0 0x7f3ab006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.678+0000 7f3ac37fe640 1 --2- 172.21.15.16:0/3848675045 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3ab006b4d0 0x7f3ab006d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3ab40016e0 tx=0x7f3ab4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.682+0000 7f3aabfff640 1 -- 172.21.15.16:0/3848675045 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3abc079880 con 0x7f3ac4161090 2024-04-25T10:10:55.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:55.873+0000 7f3ac9d13640 1 -- 172.21.15.16:0/3848675045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3ac40d1700 con 0x7f3ac4161090 2024-04-25T10:10:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.632+0000 7f3aabfff640 1 -- 172.21.15.16:0/3848675045 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v80) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3abc07d920 con 0x7f3ac4161090 2024-04-25T10:10:56.638 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-04-25T10:10:56.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.635+0000 7f3ac9d13640 1 -- 172.21.15.16:0/3848675045 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3ab006b4d0 msgr2=0x7f3ab006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.635+0000 7f3ac9d13640 1 --2- 172.21.15.16:0/3848675045 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3ab006b4d0 0x7f3ab006d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3ab40016e0 tx=0x7f3ab4009040 comp rx=0 tx=0).stop 2024-04-25T10:10:56.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.635+0000 7f3ac9d13640 1 -- 172.21.15.16:0/3848675045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ac4161090 msgr2=0x7f3ac40d3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.635+0000 7f3ac9d13640 1 --2- 172.21.15.16:0/3848675045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ac4161090 0x7f3ac40d3a80 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3abc034040 tx=0x7f3abc004a40 comp rx=0 tx=0).stop 2024-04-25T10:10:56.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.635+0000 7f3ac9d13640 1 -- 172.21.15.16:0/3848675045 shutdown_connections 2024-04-25T10:10:56.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.635+0000 7f3ac9d13640 1 --2- 172.21.15.16:0/3848675045 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3ab006b4d0 0x7f3ab006d990 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.635+0000 7f3ac9d13640 1 --2- 172.21.15.16:0/3848675045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ac4161090 0x7f3ac40d3a80 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.635+0000 7f3ac9d13640 1 --2- 172.21.15.16:0/3848675045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ac40cece0 0x7f3ac40cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.635+0000 7f3ac9d13640 1 --2- 172.21.15.16:0/3848675045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ac40b9720 0x7f3ac4146e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.635+0000 7f3ac9d13640 1 -- 172.21.15.16:0/3848675045 >> 172.21.15.16:0/3848675045 conn(0x7f3ac40b55e0 msgr2=0x7f3ac40b74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:56.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.636+0000 7f3ac9d13640 1 -- 172.21.15.16:0/3848675045 shutdown_connections 2024-04-25T10:10:56.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.636+0000 7f3ac9d13640 1 -- 172.21.15.16:0/3848675045 wait complete. 2024-04-25T10:10:56.656 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:10:56.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.758+0000 7f03f9c4d640 1 Processor -- start 2024-04-25T10:10:56.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.758+0000 7f03f9c4d640 1 -- start start 2024-04-25T10:10:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.759+0000 7f03f9c4d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f03f415dce0 0x7f03f40d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.759+0000 7f03f9c4d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f415e0c0 con 0x7f03f40d35b0 2024-04-25T10:10:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.759+0000 7f03f9c4d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f40d2510 con 0x7f03f40b5b60 2024-04-25T10:10:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.759+0000 7f03f9c4d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f40d27f0 con 0x7f03f415dce0 2024-04-25T10:10:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.759+0000 7f03f2ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f03f40d35b0 0x7f03f40b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46244/0 (socket says 172.21.15.16:46244) 2024-04-25T10:10:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.759+0000 7f03f2ffd640 1 -- 172.21.15.16:0/2629000255 learned_addr learned my addr 172.21.15.16:0/2629000255 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f37fe640 1 --2- 172.21.15.16:0/2629000255 >> v2:172.21.15.186:3300/0 conn(0x7f03f415dce0 0x7f03f40d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f27fc640 1 -- 172.21.15.16:0/2629000255 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3658545709 0 0) 0x7f03f415e0c0 con 0x7f03f40d35b0 2024-04-25T10:10:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f27fc640 1 -- 172.21.15.16:0/2629000255 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03dc003520 con 0x7f03f40d35b0 2024-04-25T10:10:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f27fc640 1 -- 172.21.15.16:0/2629000255 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419648600 0 0) 0x7f03f40d2510 con 0x7f03f40b5b60 2024-04-25T10:10:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f27fc640 1 -- 172.21.15.16:0/2629000255 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03f415e0c0 con 0x7f03f40b5b60 2024-04-25T10:10:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f27fc640 1 -- 172.21.15.16:0/2629000255 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3636078794 0 0) 0x7f03dc003520 con 0x7f03f40d35b0 2024-04-25T10:10:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f27fc640 1 -- 172.21.15.16:0/2629000255 >> v2:172.21.15.186:3300/0 conn(0x7f03f415dce0 msgr2=0x7f03f40d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f27fc640 1 --2- 172.21.15.16:0/2629000255 >> v2:172.21.15.186:3300/0 conn(0x7f03f415dce0 0x7f03f40d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f27fc640 1 -- 172.21.15.16:0/2629000255 >> v1:172.21.15.186:6789/0 conn(0x7f03f40b5b60 legacy=0x7f03f40b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f27fc640 1 -- 172.21.15.16:0/2629000255 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f40d2ad0 con 0x7f03f40d35b0 2024-04-25T10:10:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f27fc640 1 -- 172.21.15.16:0/2629000255 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f03e8001670 con 0x7f03f40d35b0 2024-04-25T10:10:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f27fc640 1 -- 172.21.15.16:0/2629000255 >> v1:172.21.15.16:6789/0 conn(0x7f03f40d35b0 legacy=0x7f03f40b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f27fc640 1 --2- 172.21.15.16:0/2629000255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03dc003b40 0x7f03dc003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f27fc640 1 --2- 172.21.15.16:0/2629000255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03dc004800 0x7f03dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f27fc640 1 --2- 172.21.15.16:0/2629000255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03dc0051c0 0x7f03dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f27fc640 1 -- 172.21.15.16:0/2629000255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f40d27f0 con 0x7f03dc003b40 2024-04-25T10:10:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f27fc640 1 -- 172.21.15.16:0/2629000255 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc00c040 con 0x7f03dc004800 2024-04-25T10:10:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f27fc640 1 -- 172.21.15.16:0/2629000255 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc00c240 con 0x7f03dc0051c0 2024-04-25T10:10:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f37fe640 1 --2- 172.21.15.16:0/2629000255 >> v2:172.21.15.186:3300/0 conn(0x7f03f415dce0 0x7f03f40d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:10:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f37fe640 1 --2- 172.21.15.16:0/2629000255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03dc003b40 0x7f03dc003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f2ffd640 1 --2- 172.21.15.16:0/2629000255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03dc004800 0x7f03dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.760+0000 7f03f3fff640 1 --2- 172.21.15.16:0/2629000255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03dc0051c0 0x7f03dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.761+0000 7f03f37fe640 1 -- 172.21.15.16:0/2629000255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03dc0051c0 msgr2=0x7f03dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.761+0000 7f03f37fe640 1 --2- 172.21.15.16:0/2629000255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03dc0051c0 0x7f03dc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.761+0000 7f03f37fe640 1 -- 172.21.15.16:0/2629000255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03dc004800 msgr2=0x7f03dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.761+0000 7f03f37fe640 1 --2- 172.21.15.16:0/2629000255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03dc004800 0x7f03dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.761+0000 7f03f37fe640 1 -- 172.21.15.16:0/2629000255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc003520 con 0x7f03dc003b40 2024-04-25T10:10:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.761+0000 7f03f3fff640 1 -- 172.21.15.16:0/2629000255 reap_dead start 2024-04-25T10:10:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.761+0000 7f03f37fe640 1 --2- 172.21.15.16:0/2629000255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03dc003b40 0x7f03dc003f40 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009c70 tx=0x7f03dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc68ab5f45f6daae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.761+0000 7f03f27fc640 1 -- 172.21.15.16:0/2629000255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e4009840 con 0x7f03dc003b40 2024-04-25T10:10:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.761+0000 7f03f27fc640 1 -- 172.21.15.16:0/2629000255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e4004020 con 0x7f03dc003b40 2024-04-25T10:10:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.761+0000 7f03f27fc640 1 -- 172.21.15.16:0/2629000255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e40043d0 con 0x7f03dc003b40 2024-04-25T10:10:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.762+0000 7f03f9c4d640 1 -- 172.21.15.16:0/2629000255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03dc003b40 msgr2=0x7f03dc003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.762+0000 7f03f9c4d640 1 --2- 172.21.15.16:0/2629000255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03dc003b40 0x7f03dc003f40 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009c70 tx=0x7f03dc005830 comp rx=0 tx=0).stop 2024-04-25T10:10:56.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.762+0000 7f03f9c4d640 1 -- 172.21.15.16:0/2629000255 shutdown_connections 2024-04-25T10:10:56.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.762+0000 7f03f9c4d640 1 --2- 172.21.15.16:0/2629000255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03dc003b40 0x7f03dc003f40 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.762+0000 7f03f9c4d640 1 -- 172.21.15.16:0/2629000255 >> 172.21.15.16:0/2629000255 conn(0x7f03f4072900 msgr2=0x7f03f40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:56.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.762+0000 7f03f9c4d640 1 -- 172.21.15.16:0/2629000255 shutdown_connections 2024-04-25T10:10:56.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.762+0000 7f03f9c4d640 1 -- 172.21.15.16:0/2629000255 wait complete. 2024-04-25T10:10:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.763+0000 7f03f9c4d640 1 Processor -- start 2024-04-25T10:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.763+0000 7f03f9c4d640 1 -- start start 2024-04-25T10:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.764+0000 7f03f9c4d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03dc003b40 0x7f03f41570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.764+0000 7f03f9c4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f415f500 0x7f03f40ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.764+0000 7f03f9c4d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f40ccff0 0x7f03f415e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.764+0000 7f03f9c4d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f40b9820 con 0x7f03f415f500 2024-04-25T10:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.764+0000 7f03f9c4d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f40d3090 con 0x7f03f40ccff0 2024-04-25T10:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.764+0000 7f03f9c4d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f40b99a0 con 0x7f03dc003b40 2024-04-25T10:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.764+0000 7f03f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f415f500 0x7f03f40ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.764+0000 7f03f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f415f500 0x7f03f40ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55372/0 (socket says 172.21.15.16:55372) 2024-04-25T10:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.764+0000 7f03f2ffd640 1 -- 172.21.15.16:0/2192000693 learned_addr learned my addr 172.21.15.16:0/2192000693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.764+0000 7f03f37fe640 1 --2- 172.21.15.16:0/2192000693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03dc003b40 0x7f03f41570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.764+0000 7f03f3fff640 1 --2- 172.21.15.16:0/2192000693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f40ccff0 0x7f03f415e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.765+0000 7f03f2ffd640 1 -- 172.21.15.16:0/2192000693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03dc003b40 msgr2=0x7f03f41570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.765+0000 7f03f2ffd640 1 --2- 172.21.15.16:0/2192000693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03dc003b40 0x7f03f41570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.765+0000 7f03f2ffd640 1 -- 172.21.15.16:0/2192000693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f40ccff0 msgr2=0x7f03f415e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.765+0000 7f03f2ffd640 1 --2- 172.21.15.16:0/2192000693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f40ccff0 0x7f03f415e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.765+0000 7f03f2ffd640 1 -- 172.21.15.16:0/2192000693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f41575e0 con 0x7f03f415f500 2024-04-25T10:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.765+0000 7f03f2ffd640 1 --2- 172.21.15.16:0/2192000693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f415f500 0x7f03f40ccab0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f03e8004fa0 tx=0x7f03e8007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.765+0000 7f03dbfff640 1 -- 172.21.15.16:0/2192000693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e800f780 con 0x7f03f415f500 2024-04-25T10:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.765+0000 7f03dbfff640 1 -- 172.21.15.16:0/2192000693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e8018070 con 0x7f03f415f500 2024-04-25T10:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.765+0000 7f03f9c4d640 1 -- 172.21.15.16:0/2192000693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f41f4440 con 0x7f03f415f500 2024-04-25T10:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.765+0000 7f03dbfff640 1 -- 172.21.15.16:0/2192000693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e800fb80 con 0x7f03f415f500 2024-04-25T10:10:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.765+0000 7f03f9c4d640 1 -- 172.21.15.16:0/2192000693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f41f3ba0 con 0x7f03f415f500 2024-04-25T10:10:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.766+0000 7f03f9c4d640 1 -- 172.21.15.16:0/2192000693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f40749b0 con 0x7f03f415f500 2024-04-25T10:10:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.767+0000 7f03dbfff640 1 -- 172.21.15.16:0/2192000693 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f03e800fd20 con 0x7f03f415f500 2024-04-25T10:10:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.767+0000 7f03dbfff640 1 --2- 172.21.15.16:0/2192000693 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f03e006b500 0x7f03e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.767+0000 7f03dbfff640 1 -- 172.21.15.16:0/2192000693 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f03e8089d30 con 0x7f03f415f500 2024-04-25T10:10:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.767+0000 7f03f37fe640 1 --2- 172.21.15.16:0/2192000693 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f03e006b500 0x7f03e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.768+0000 7f03f37fe640 1 --2- 172.21.15.16:0/2192000693 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f03e006b500 0x7f03e006d9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f03e4032000 tx=0x7f03e4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:56.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:56.772+0000 7f03dbfff640 1 -- 172.21.15.16:0/2192000693 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03e8058f00 con 0x7f03f415f500 2024-04-25T10:10:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.021+0000 7f03f9c4d640 1 -- 172.21.15.16:0/2192000693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f03f40cd3d0 con 0x7f03f415f500 2024-04-25T10:10:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.022+0000 7f03dbfff640 1 -- 172.21.15.16:0/2192000693 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f03e8058800 con 0x7f03f415f500 2024-04-25T10:10:57.030 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:57.030 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:10:57.030 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-04-25T10:10:57.030 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.031 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-04-25T10:10:57.031 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmj+xmBBAAaWdHmoiYcXzFnMVQSioT2Q==", 2024-04-25T10:10:57.031 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.031 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:10:57.031 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:10:57.031 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:10:57.031 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:10:57.031 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.031 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:10:57.032 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.032 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-04-25T10:10:57.032 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmUES+DxAAu5It7dQmb7/BM/4x1fUYXA==", 2024-04-25T10:10:57.032 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.032 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:10:57.032 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:10:57.032 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:10:57.032 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:10:57.032 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.032 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:10:57.032 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.032 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-04-25T10:10:57.032 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipm+LtFCxAAGG79kv8eWzly8l5qzQwfww==", 2024-04-25T10:10:57.032 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.032 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:10:57.033 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:10:57.033 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:10:57.033 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:10:57.033 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.033 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:10:57.033 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.033 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-04-25T10:10:57.033 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmqbSSFhAAZtrLsAGSyaMGgv45Hy0WpA==", 2024-04-25T10:10:57.033 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.033 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:10:57.033 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:10:57.033 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:10:57.033 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:10:57.033 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.034 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:10:57.034 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.034 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-04-25T10:10:57.034 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCaKipm2+UdOxAAP8GBunvYsue8EILYV5Uq7A==", 2024-04-25T10:10:57.034 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.034 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:57.034 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:10:57.034 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:10:57.034 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.034 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:10:57.034 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.034 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-04-25T10:10:57.034 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCdKipmSv87ORAAigVU7nyzkcXqMEHKsSAsqg==", 2024-04-25T10:10:57.034 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.035 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:57.035 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:10:57.035 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:10:57.035 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.035 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:10:57.035 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.035 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-04-25T10:10:57.035 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCgKipm2VMLOBAAJETFfWgq7H/imQAfw93Ruw==", 2024-04-25T10:10:57.035 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.035 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:57.035 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:10:57.035 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:10:57.035 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.035 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:10:57.035 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.036 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-04-25T10:10:57.036 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCjKipmb1g0NhAADH2uHQPhW84KLYyWbOjQdw==", 2024-04-25T10:10:57.036 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.036 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:57.036 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:10:57.036 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:10:57.036 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.036 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:10:57.036 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.036 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-04-25T10:10:57.036 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCpKipmnH0ROBAAEb8i4nfswYQ/w5mEw48kIg==", 2024-04-25T10:10:57.036 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.036 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:57.036 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:10:57.037 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:10:57.037 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.037 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:10:57.037 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.037 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-04-25T10:10:57.037 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCsKipm8HCRNhAA3FhcMlUWxMAiFRXbBWWvNA==", 2024-04-25T10:10:57.037 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.037 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:57.037 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:10:57.037 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:10:57.037 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.037 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:10:57.037 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.037 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-04-25T10:10:57.037 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCvKipmfgqpNRAAtPhpSDU9q23Sfa+V7m+3NA==", 2024-04-25T10:10:57.038 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.038 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:57.038 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:10:57.038 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:10:57.038 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.038 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:10:57.038 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.038 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-04-25T10:10:57.038 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCyKipm3ndyMxAAeMNODGxNupb043CsA/SP1A==", 2024-04-25T10:10:57.038 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.038 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:57.038 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:10:57.038 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:10:57.038 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:10:57.039 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.039 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-04-25T10:10:57.039 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmAph/GhAAG3x1zC98yHQ13fjpmrlPkg==", 2024-04-25T10:10:57.039 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.039 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:10:57.039 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-04-25T10:10:57.039 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:10:57.039 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:10:57.039 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.039 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-04-25T10:10:57.039 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmrccTIxAAuDZamFXTe6J2d+HAcBe+DA==", 2024-04-25T10:10:57.039 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.039 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:10:57.040 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:10:57.040 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:10:57.040 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:10:57.040 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.040 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:10:57.040 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.040 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-04-25T10:10:57.040 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmsFACIBAAFuWsmuQlF58yzIcYUx/6Iw==", 2024-04-25T10:10:57.040 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.040 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:10:57.040 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:10:57.040 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:10:57.040 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:10:57.040 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.040 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:10:57.041 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.041 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-04-25T10:10:57.041 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmAe1/KBAAXzLkscuwpCwMnrIKQny9+Q==", 2024-04-25T10:10:57.041 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.041 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:10:57.041 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:10:57.041 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:10:57.041 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:10:57.041 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:10:57.041 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.041 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-04-25T10:10:57.041 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCYKipmVFQ2JBAAdqG2p/M746PZPMDcpQjwfw==", 2024-04-25T10:10:57.041 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.042 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:10:57.042 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:10:57.042 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:10:57.042 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:10:57.042 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.042 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:10:57.042 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.042 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-04-25T10:10:57.042 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmiPVHARAAGPbv3P3yVj3tlo0b2gmcyw==", 2024-04-25T10:10:57.042 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.042 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:10:57.042 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-04-25T10:10:57.042 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:10:57.043 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.043 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:10:57.043 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.043 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-04-25T10:10:57.043 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCYKipmghhyOBAAcCm0hQzU+Wlw45yZQETTTg==", 2024-04-25T10:10:57.043 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:10:57.043 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:10:57.043 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-04-25T10:10:57.043 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:10:57.043 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.044 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.044 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-04-25T10:10:57.044 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:10:57.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.024+0000 7f03f9c4d640 1 -- 172.21.15.16:0/2192000693 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f03e006b500 msgr2=0x7f03e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.024+0000 7f03f9c4d640 1 --2- 172.21.15.16:0/2192000693 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f03e006b500 0x7f03e006d9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f03e4032000 tx=0x7f03e4008030 comp rx=0 tx=0).stop 2024-04-25T10:10:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.024+0000 7f03f9c4d640 1 -- 172.21.15.16:0/2192000693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f415f500 msgr2=0x7f03f40ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.024+0000 7f03f9c4d640 1 --2- 172.21.15.16:0/2192000693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f415f500 0x7f03f40ccab0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f03e8004fa0 tx=0x7f03e8007430 comp rx=0 tx=0).stop 2024-04-25T10:10:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.024+0000 7f03f9c4d640 1 -- 172.21.15.16:0/2192000693 shutdown_connections 2024-04-25T10:10:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.024+0000 7f03f9c4d640 1 --2- 172.21.15.16:0/2192000693 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f03e006b500 0x7f03e006d9c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.024+0000 7f03f9c4d640 1 --2- 172.21.15.16:0/2192000693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f40ccff0 0x7f03f415e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.024+0000 7f03f9c4d640 1 --2- 172.21.15.16:0/2192000693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f415f500 0x7f03f40ccab0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.024+0000 7f03f9c4d640 1 --2- 172.21.15.16:0/2192000693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03dc003b40 0x7f03f41570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.024+0000 7f03f9c4d640 1 -- 172.21.15.16:0/2192000693 >> 172.21.15.16:0/2192000693 conn(0x7f03f4072900 msgr2=0x7f03f40b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.024+0000 7f03f9c4d640 1 -- 172.21.15.16:0/2192000693 shutdown_connections 2024-04-25T10:10:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.024+0000 7f03f9c4d640 1 -- 172.21.15.16:0/2192000693 wait complete. 2024-04-25T10:10:57.047 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.144+0000 7fa75a67d640 1 Processor -- start 2024-04-25T10:10:57.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.144+0000 7fa75a67d640 1 -- start start 2024-04-25T10:10:57.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.145+0000 7fa75a67d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa75415dce0 0x7fa7540d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.145+0000 7fa75a67d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7540b5b60 0x7fa7540b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.145+0000 7fa75a67d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75415e0c0 con 0x7fa7540d35b0 2024-04-25T10:10:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.145+0000 7fa75a67d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7540d2510 con 0x7fa75415dce0 2024-04-25T10:10:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.145+0000 7fa75a67d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7540d27f0 con 0x7fa7540b5b60 2024-04-25T10:10:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.145+0000 7fa753fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa75415dce0 0x7fa7540d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.145+0000 7fa753fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa75415dce0 0x7fa7540d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55386/0 (socket says 172.21.15.16:55386) 2024-04-25T10:10:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.145+0000 7fa753fff640 1 -- 172.21.15.16:0/3760011476 learned_addr learned my addr 172.21.15.16:0/3760011476 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.145+0000 7fa758c1a640 1 --2- 172.21.15.16:0/3760011476 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7540b5b60 0x7fa7540b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.145+0000 7fa753fff640 1 -- 172.21.15.16:0/3760011476 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7540b5b60 msgr2=0x7fa7540b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.145+0000 7fa753fff640 1 --2- 172.21.15.16:0/3760011476 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7540b5b60 0x7fa7540b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.145+0000 7fa753fff640 1 -- 172.21.15.16:0/3760011476 >> v1:172.21.15.186:6789/0 conn(0x7fa7540d35b0 legacy=0x7fa7540b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.145+0000 7fa753fff640 1 -- 172.21.15.16:0/3760011476 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7540d2ad0 con 0x7fa75415dce0 2024-04-25T10:10:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.146+0000 7fa753fff640 1 --2- 172.21.15.16:0/3760011476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa75415dce0 0x7fa7540d1a50 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fa744009870 tx=0x7fa74402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6633294338e8e771 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.146+0000 7fa752ffd640 1 -- 172.21.15.16:0/3760011476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744037070 con 0x7fa75415dce0 2024-04-25T10:10:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.146+0000 7fa752ffd640 1 -- 172.21.15.16:0/3760011476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa744032440 con 0x7fa75415dce0 2024-04-25T10:10:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.146+0000 7fa752ffd640 1 -- 172.21.15.16:0/3760011476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744032810 con 0x7fa75415dce0 2024-04-25T10:10:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.147+0000 7fa75a67d640 1 -- 172.21.15.16:0/3760011476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa75415dce0 msgr2=0x7fa7540d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.147+0000 7fa75a67d640 1 --2- 172.21.15.16:0/3760011476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa75415dce0 0x7fa7540d1a50 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fa744009870 tx=0x7fa74402a270 comp rx=0 tx=0).stop 2024-04-25T10:10:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.147+0000 7fa75a67d640 1 -- 172.21.15.16:0/3760011476 shutdown_connections 2024-04-25T10:10:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.147+0000 7fa75a67d640 1 --2- 172.21.15.16:0/3760011476 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7540b5b60 0x7fa7540b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.147+0000 7fa75a67d640 1 --2- 172.21.15.16:0/3760011476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa75415dce0 0x7fa7540d1a50 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.147+0000 7fa75a67d640 1 -- 172.21.15.16:0/3760011476 >> 172.21.15.16:0/3760011476 conn(0x7fa754072900 msgr2=0x7fa7540d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.147+0000 7fa75a67d640 1 -- 172.21.15.16:0/3760011476 shutdown_connections 2024-04-25T10:10:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.147+0000 7fa75a67d640 1 -- 172.21.15.16:0/3760011476 wait complete. 2024-04-25T10:10:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.148+0000 7fa75a67d640 1 Processor -- start 2024-04-25T10:10:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.148+0000 7fa75a67d640 1 -- start start 2024-04-25T10:10:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.149+0000 7fa75a67d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7540b5b60 0x7fa7541f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.149+0000 7fa75a67d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7540d35b0 0x7fa75415ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.149+0000 7fa75a67d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa75415dce0 0x7fa75415ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.149+0000 7fa75a67d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7540b9820 con 0x7fa75415dce0 2024-04-25T10:10:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.149+0000 7fa75a67d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7540d3090 con 0x7fa7540b5b60 2024-04-25T10:10:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.149+0000 7fa75a67d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7540b99a0 con 0x7fa7540d35b0 2024-04-25T10:10:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.149+0000 7fa758c1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa75415dce0 0x7fa75415ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.149+0000 7fa758c1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa75415dce0 0x7fa75415ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55394/0 (socket says 172.21.15.16:55394) 2024-04-25T10:10:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.149+0000 7fa758c1a640 1 -- 172.21.15.16:0/1053910232 learned_addr learned my addr 172.21.15.16:0/1053910232 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.149+0000 7fa753fff640 1 --2- 172.21.15.16:0/1053910232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7540b5b60 0x7fa7541f7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.149+0000 7fa7537fe640 1 --2- 172.21.15.16:0/1053910232 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7540d35b0 0x7fa75415ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.149+0000 7fa758c1a640 1 -- 172.21.15.16:0/1053910232 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7540d35b0 msgr2=0x7fa75415ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.149+0000 7fa758c1a640 1 --2- 172.21.15.16:0/1053910232 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7540d35b0 0x7fa75415ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.149+0000 7fa758c1a640 1 -- 172.21.15.16:0/1053910232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7540b5b60 msgr2=0x7fa7541f7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.149+0000 7fa758c1a640 1 --2- 172.21.15.16:0/1053910232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7540b5b60 0x7fa7541f7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.149+0000 7fa758c1a640 1 -- 172.21.15.16:0/1053910232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa754160980 con 0x7fa75415dce0 2024-04-25T10:10:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.149+0000 7fa7537fe640 1 --2- 172.21.15.16:0/1053910232 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7540d35b0 0x7fa75415ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.150+0000 7fa758c1a640 1 --2- 172.21.15.16:0/1053910232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa75415dce0 0x7fa75415ff00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fa74c007e40 tx=0x7fa74c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.150+0000 7fa750ff9640 1 -- 172.21.15.16:0/1053910232 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c014620 con 0x7fa75415dce0 2024-04-25T10:10:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.150+0000 7fa750ff9640 1 -- 172.21.15.16:0/1053910232 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74c019070 con 0x7fa75415dce0 2024-04-25T10:10:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.150+0000 7fa75a67d640 1 -- 172.21.15.16:0/1053910232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7541f7100 con 0x7fa75415dce0 2024-04-25T10:10:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.150+0000 7fa750ff9640 1 -- 172.21.15.16:0/1053910232 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c014a20 con 0x7fa75415dce0 2024-04-25T10:10:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.150+0000 7fa75a67d640 1 -- 172.21.15.16:0/1053910232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7541f76c0 con 0x7fa75415dce0 2024-04-25T10:10:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.151+0000 7fa75a67d640 1 -- 172.21.15.16:0/1053910232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7540749b0 con 0x7fa75415dce0 2024-04-25T10:10:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.151+0000 7fa750ff9640 1 -- 172.21.15.16:0/1053910232 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa74c014bc0 con 0x7fa75415dce0 2024-04-25T10:10:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.152+0000 7fa750ff9640 1 --2- 172.21.15.16:0/1053910232 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa74006b500 0x7fa74006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.152+0000 7fa750ff9640 1 -- 172.21.15.16:0/1053910232 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa74c08a6f0 con 0x7fa75415dce0 2024-04-25T10:10:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.152+0000 7fa753fff640 1 --2- 172.21.15.16:0/1053910232 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa74006b500 0x7fa74006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.153+0000 7fa753fff640 1 --2- 172.21.15.16:0/1053910232 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa74006b500 0x7fa74006d9c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa744034000 tx=0x7fa74402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.156+0000 7fa750ff9640 1 -- 172.21.15.16:0/1053910232 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa74c0598b0 con 0x7fa75415dce0 2024-04-25T10:10:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.336+0000 7fa75a67d640 1 -- 172.21.15.16:0/1053910232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7540b9170 con 0x7fa75415dce0 2024-04-25T10:10:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.337+0000 7fa750ff9640 1 -- 172.21.15.16:0/1053910232 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+12892 (secure 0 0 0) 0x7fa74c0591b0 con 0x7fa75415dce0 2024-04-25T10:10:57.343 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:57.343 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:10:56.640623+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.339+0000 7fa75a67d640 1 -- 172.21.15.16:0/1053910232 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa74006b500 msgr2=0x7fa74006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.339+0000 7fa75a67d640 1 --2- 172.21.15.16:0/1053910232 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa74006b500 0x7fa74006d9c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa744034000 tx=0x7fa74402e000 comp rx=0 tx=0).stop 2024-04-25T10:10:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.339+0000 7fa75a67d640 1 -- 172.21.15.16:0/1053910232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa75415dce0 msgr2=0x7fa75415ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.339+0000 7fa75a67d640 1 --2- 172.21.15.16:0/1053910232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa75415dce0 0x7fa75415ff00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fa74c007e40 tx=0x7fa74c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.339+0000 7fa75a67d640 1 -- 172.21.15.16:0/1053910232 shutdown_connections 2024-04-25T10:10:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.339+0000 7fa75a67d640 1 --2- 172.21.15.16:0/1053910232 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa74006b500 0x7fa74006d9c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.339+0000 7fa75a67d640 1 --2- 172.21.15.16:0/1053910232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa75415dce0 0x7fa75415ff00 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.339+0000 7fa75a67d640 1 --2- 172.21.15.16:0/1053910232 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7540d35b0 0x7fa75415ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.339+0000 7fa75a67d640 1 --2- 172.21.15.16:0/1053910232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7540b5b60 0x7fa7541f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.339+0000 7fa75a67d640 1 -- 172.21.15.16:0/1053910232 >> 172.21.15.16:0/1053910232 conn(0x7fa754072900 msgr2=0x7fa7540d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.339+0000 7fa75a67d640 1 -- 172.21.15.16:0/1053910232 shutdown_connections 2024-04-25T10:10:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.339+0000 7fa75a67d640 1 -- 172.21.15.16:0/1053910232 wait complete. 2024-04-25T10:10:57.355 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:10:57.356 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:10:57.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.455+0000 7f93d9770640 1 Processor -- start 2024-04-25T10:10:57.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.455+0000 7f93d9770640 1 -- start start 2024-04-25T10:10:57.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.456+0000 7f93d9770640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93d415b7b0 0x7f93d40ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.456+0000 7f93d9770640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93d40cf760 0x7f93d40cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.456+0000 7f93d9770640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93d415f000 con 0x7f93d415e110 2024-04-25T10:10:57.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.456+0000 7f93d9770640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93d40d0100 con 0x7f93d415b7b0 2024-04-25T10:10:57.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.456+0000 7f93d9770640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d40ced00 con 0x7f93d40cf760 2024-04-25T10:10:57.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.456+0000 7f93d2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93d415b7b0 0x7f93d40ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.456+0000 7f93d37fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f93d415e110 0x7f93d40cc080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46254/0 (socket says 172.21.15.16:46254) 2024-04-25T10:10:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.456+0000 7f93d37fe640 1 -- 172.21.15.16:0/2161606546 learned_addr learned my addr 172.21.15.16:0/2161606546 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.456+0000 7f93cbfff640 1 --2- 172.21.15.16:0/2161606546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93d40cf760 0x7f93d40cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.456+0000 7f93d2ffd640 1 -- 172.21.15.16:0/2161606546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93d40cf760 msgr2=0x7f93d40cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.456+0000 7f93d2ffd640 1 --2- 172.21.15.16:0/2161606546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93d40cf760 0x7f93d40cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.456+0000 7f93d2ffd640 1 -- 172.21.15.16:0/2161606546 >> v1:172.21.15.16:6789/0 conn(0x7f93d415e110 legacy=0x7f93d40cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.456+0000 7f93d2ffd640 1 -- 172.21.15.16:0/2161606546 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d415ed20 con 0x7f93d415b7b0 2024-04-25T10:10:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.456+0000 7f93d27fc640 1 -- 172.21.15.16:0/2161606546 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2742455381 0 0) 0x7f93d415f000 con 0x7f93d415e110 2024-04-25T10:10:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.457+0000 7f93d2ffd640 1 --2- 172.21.15.16:0/2161606546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d415b7b0 0x7f93d40ce7c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f93c40097a0 tx=0x7f93c402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7032e7f67b6bec16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.457+0000 7f93d27fc640 1 -- 172.21.15.16:0/2161606546 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c4037070 con 0x7f93d415b7b0 2024-04-25T10:10:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.457+0000 7f93d27fc640 1 -- 172.21.15.16:0/2161606546 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c4032440 con 0x7f93d415b7b0 2024-04-25T10:10:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.457+0000 7f93d27fc640 1 -- 172.21.15.16:0/2161606546 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c4032810 con 0x7f93d415b7b0 2024-04-25T10:10:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.458+0000 7f93d9770640 1 -- 172.21.15.16:0/2161606546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d415b7b0 msgr2=0x7f93d40ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.458+0000 7f93d9770640 1 --2- 172.21.15.16:0/2161606546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d415b7b0 0x7f93d40ce7c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f93c40097a0 tx=0x7f93c402a180 comp rx=0 tx=0).stop 2024-04-25T10:10:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.458+0000 7f93d9770640 1 -- 172.21.15.16:0/2161606546 shutdown_connections 2024-04-25T10:10:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.458+0000 7f93d9770640 1 --2- 172.21.15.16:0/2161606546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93d40cf760 0x7f93d40cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.458+0000 7f93d9770640 1 --2- 172.21.15.16:0/2161606546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d415b7b0 0x7f93d40ce7c0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.458+0000 7f93d9770640 1 -- 172.21.15.16:0/2161606546 >> 172.21.15.16:0/2161606546 conn(0x7f93d406f4a0 msgr2=0x7f93d406f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:57.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.458+0000 7f93d9770640 1 -- 172.21.15.16:0/2161606546 shutdown_connections 2024-04-25T10:10:57.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.458+0000 7f93d9770640 1 -- 172.21.15.16:0/2161606546 wait complete. 2024-04-25T10:10:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.459+0000 7f93d9770640 1 Processor -- start 2024-04-25T10:10:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.459+0000 7f93d9770640 1 -- start start 2024-04-25T10:10:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93d9770640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93d40cf760 0x7f93d415a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93d9770640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93d415b7b0 0x7f93d40b5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93d9770640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d415e110 0x7f93d40b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93d9770640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d40cdfc0 con 0x7f93d415e110 2024-04-25T10:10:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93d9770640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d40ce330 con 0x7f93d40cf760 2024-04-25T10:10:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93d9770640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d41619f0 con 0x7f93d415b7b0 2024-04-25T10:10:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d415e110 0x7f93d40b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d415e110 0x7f93d40b6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55418/0 (socket says 172.21.15.16:55418) 2024-04-25T10:10:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93d37fe640 1 -- 172.21.15.16:0/2299414698 learned_addr learned my addr 172.21.15.16:0/2299414698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93cbfff640 1 --2- 172.21.15.16:0/2299414698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93d415b7b0 0x7f93d40b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93d2ffd640 1 --2- 172.21.15.16:0/2299414698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93d40cf760 0x7f93d415a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93d37fe640 1 -- 172.21.15.16:0/2299414698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93d415b7b0 msgr2=0x7f93d40b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93d37fe640 1 --2- 172.21.15.16:0/2299414698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93d415b7b0 0x7f93d40b5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93d37fe640 1 -- 172.21.15.16:0/2299414698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93d40cf760 msgr2=0x7f93d415a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93d37fe640 1 --2- 172.21.15.16:0/2299414698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93d40cf760 0x7f93d415a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93d37fe640 1 -- 172.21.15.16:0/2299414698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d40b5e40 con 0x7f93d415e110 2024-04-25T10:10:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93cbfff640 1 --2- 172.21.15.16:0/2299414698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93d415b7b0 0x7f93d40b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93d37fe640 1 --2- 172.21.15.16:0/2299414698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d415e110 0x7f93d40b6ee0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00c7a0 tx=0x7f93cc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93cb7fe640 1 -- 172.21.15.16:0/2299414698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc018070 con 0x7f93d415e110 2024-04-25T10:10:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93cb7fe640 1 -- 172.21.15.16:0/2299414698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93cc00a640 con 0x7f93d415e110 2024-04-25T10:10:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93d9770640 1 -- 172.21.15.16:0/2299414698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d40b6120 con 0x7f93d415e110 2024-04-25T10:10:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.460+0000 7f93cb7fe640 1 -- 172.21.15.16:0/2299414698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc00aab0 con 0x7f93d415e110 2024-04-25T10:10:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.461+0000 7f93d9770640 1 -- 172.21.15.16:0/2299414698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d40b66e0 con 0x7f93d415e110 2024-04-25T10:10:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.461+0000 7f93d9770640 1 -- 172.21.15.16:0/2299414698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d40750b0 con 0x7f93d415e110 2024-04-25T10:10:57.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.462+0000 7f93cb7fe640 1 -- 172.21.15.16:0/2299414698 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f93cc01b070 con 0x7f93d415e110 2024-04-25T10:10:57.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.462+0000 7f93cb7fe640 1 --2- 172.21.15.16:0/2299414698 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f93b406b4d0 0x7f93b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.463+0000 7f93cb7fe640 1 -- 172.21.15.16:0/2299414698 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f93cc089430 con 0x7f93d415e110 2024-04-25T10:10:57.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.463+0000 7f93d2ffd640 1 --2- 172.21.15.16:0/2299414698 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f93b406b4d0 0x7f93b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.463+0000 7f93d2ffd640 1 --2- 172.21.15.16:0/2299414698 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f93b406b4d0 0x7f93b406d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f93c4034000 tx=0x7f93c402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.467+0000 7f93cb7fe640 1 -- 172.21.15.16:0/2299414698 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f93cc058690 con 0x7f93d415e110 2024-04-25T10:10:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.647+0000 7f93d9770640 1 -- 172.21.15.16:0/2299414698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f93d415a700 con 0x7f93d415e110 2024-04-25T10:10:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.647+0000 7f93cb7fe640 1 -- 172.21.15.16:0/2299414698 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f93cc057f90 con 0x7f93d415e110 2024-04-25T10:10:57.652 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:10:57.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.649+0000 7f93d9770640 1 -- 172.21.15.16:0/2299414698 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f93b406b4d0 msgr2=0x7f93b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.649+0000 7f93d9770640 1 --2- 172.21.15.16:0/2299414698 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f93b406b4d0 0x7f93b406d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f93c4034000 tx=0x7f93c402e000 comp rx=0 tx=0).stop 2024-04-25T10:10:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.649+0000 7f93d9770640 1 -- 172.21.15.16:0/2299414698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d415e110 msgr2=0x7f93d40b6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.649+0000 7f93d9770640 1 --2- 172.21.15.16:0/2299414698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d415e110 0x7f93d40b6ee0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00c7a0 tx=0x7f93cc00cc60 comp rx=0 tx=0).stop 2024-04-25T10:10:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.649+0000 7f93d9770640 1 -- 172.21.15.16:0/2299414698 shutdown_connections 2024-04-25T10:10:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.649+0000 7f93d9770640 1 --2- 172.21.15.16:0/2299414698 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f93b406b4d0 0x7f93b406d990 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.649+0000 7f93d9770640 1 --2- 172.21.15.16:0/2299414698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d415e110 0x7f93d40b6ee0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.649+0000 7f93d9770640 1 --2- 172.21.15.16:0/2299414698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93d415b7b0 0x7f93d40b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.649+0000 7f93d9770640 1 --2- 172.21.15.16:0/2299414698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93d40cf760 0x7f93d415a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.649+0000 7f93d9770640 1 -- 172.21.15.16:0/2299414698 >> 172.21.15.16:0/2299414698 conn(0x7f93d406f4a0 msgr2=0x7f93d406cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.649+0000 7f93d9770640 1 -- 172.21.15.16:0/2299414698 shutdown_connections 2024-04-25T10:10:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.649+0000 7f93d9770640 1 -- 172.21.15.16:0/2299414698 wait complete. 2024-04-25T10:10:57.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:10:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.763+0000 7fb2b9ae1640 1 Processor -- start 2024-04-25T10:10:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.763+0000 7fb2b9ae1640 1 -- start start 2024-04-25T10:10:57.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.763+0000 7fb2b9ae1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb2b415dce0 0x7fb2b40d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.763+0000 7fb2b9ae1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2b40d35b0 0x7fb2b40b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.764+0000 7fb2b9ae1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b415e0c0 con 0x7fb2b40d1f90 2024-04-25T10:10:57.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.764+0000 7fb2b9ae1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b40b7b10 con 0x7fb2b415dce0 2024-04-25T10:10:57.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.764+0000 7fb2b9ae1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b40d2510 con 0x7fb2b40d35b0 2024-04-25T10:10:57.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.764+0000 7fb2b37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb2b415dce0 0x7fb2b40d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.764+0000 7fb2b3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb2b40d1f90 0x7fb2b406bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46264/0 (socket says 172.21.15.16:46264) 2024-04-25T10:10:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.764+0000 7fb2b3fff640 1 -- 172.21.15.16:0/2093042136 learned_addr learned my addr 172.21.15.16:0/2093042136 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.764+0000 7fb2aaffd640 1 --2- 172.21.15.16:0/2093042136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2b40d35b0 0x7fb2b40b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.764+0000 7fb2b37fe640 1 -- 172.21.15.16:0/2093042136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2b40d35b0 msgr2=0x7fb2b40b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.764+0000 7fb2b37fe640 1 --2- 172.21.15.16:0/2093042136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2b40d35b0 0x7fb2b40b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.764+0000 7fb2b37fe640 1 -- 172.21.15.16:0/2093042136 >> v1:172.21.15.16:6789/0 conn(0x7fb2b40d1f90 legacy=0x7fb2b406bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.764+0000 7fb2b2ffd640 1 -- 172.21.15.16:0/2093042136 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 620179007 0 0) 0x7fb2b415e0c0 con 0x7fb2b40d1f90 2024-04-25T10:10:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.764+0000 7fb2b37fe640 1 -- 172.21.15.16:0/2093042136 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b40d27f0 con 0x7fb2b415dce0 2024-04-25T10:10:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.764+0000 7fb2aaffd640 1 --2- 172.21.15.16:0/2093042136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2b40d35b0 0x7fb2b40b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.765+0000 7fb2b37fe640 1 --2- 172.21.15.16:0/2093042136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b415dce0 0x7fb2b40d1a50 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009d90 tx=0x7fb2a402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8cd846a766b0c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.765+0000 7fb2b2ffd640 1 -- 172.21.15.16:0/2093042136 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4031600 con 0x7fb2b415dce0 2024-04-25T10:10:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.765+0000 7fb2b2ffd640 1 -- 172.21.15.16:0/2093042136 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a40317a0 con 0x7fb2b415dce0 2024-04-25T10:10:57.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.765+0000 7fb2b2ffd640 1 -- 172.21.15.16:0/2093042136 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4031b70 con 0x7fb2b415dce0 2024-04-25T10:10:57.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.765+0000 7fb2b9ae1640 1 -- 172.21.15.16:0/2093042136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b415dce0 msgr2=0x7fb2b40d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.765+0000 7fb2b9ae1640 1 --2- 172.21.15.16:0/2093042136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b415dce0 0x7fb2b40d1a50 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009d90 tx=0x7fb2a402a5d0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.765+0000 7fb2b9ae1640 1 -- 172.21.15.16:0/2093042136 shutdown_connections 2024-04-25T10:10:57.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.765+0000 7fb2b9ae1640 1 --2- 172.21.15.16:0/2093042136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2b40d35b0 0x7fb2b40b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.765+0000 7fb2b9ae1640 1 --2- 172.21.15.16:0/2093042136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b415dce0 0x7fb2b40d1a50 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.765+0000 7fb2b9ae1640 1 -- 172.21.15.16:0/2093042136 >> 172.21.15.16:0/2093042136 conn(0x7fb2b4072900 msgr2=0x7fb2b40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:57.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.766+0000 7fb2b9ae1640 1 -- 172.21.15.16:0/2093042136 shutdown_connections 2024-04-25T10:10:57.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.766+0000 7fb2b9ae1640 1 -- 172.21.15.16:0/2093042136 wait complete. 2024-04-25T10:10:57.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.767+0000 7fb2b9ae1640 1 Processor -- start 2024-04-25T10:10:57.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.767+0000 7fb2b9ae1640 1 -- start start 2024-04-25T10:10:57.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b9ae1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2b40d1f90 0x7fb2b4160a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b9ae1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2b40d35b0 0x7fb2b41534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b9ae1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b415dce0 0x7fb2b4153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b9ae1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b40d3090 con 0x7fb2b415dce0 2024-04-25T10:10:57.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b9ae1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b40d2db0 con 0x7fb2b40d35b0 2024-04-25T10:10:57.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b9ae1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b40b9820 con 0x7fb2b40d1f90 2024-04-25T10:10:57.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b415dce0 0x7fb2b4153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b415dce0 0x7fb2b4153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55434/0 (socket says 172.21.15.16:55434) 2024-04-25T10:10:57.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b3fff640 1 -- 172.21.15.16:0/442340237 learned_addr learned my addr 172.21.15.16:0/442340237 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:57.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b37fe640 1 --2- 172.21.15.16:0/442340237 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2b40d1f90 0x7fb2b4160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2aaffd640 1 --2- 172.21.15.16:0/442340237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2b40d35b0 0x7fb2b41534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b3fff640 1 -- 172.21.15.16:0/442340237 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2b40d1f90 msgr2=0x7fb2b4160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b3fff640 1 --2- 172.21.15.16:0/442340237 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2b40d1f90 0x7fb2b4160a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b3fff640 1 -- 172.21.15.16:0/442340237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2b40d35b0 msgr2=0x7fb2b41534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b3fff640 1 --2- 172.21.15.16:0/442340237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2b40d35b0 0x7fb2b41534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b3fff640 1 -- 172.21.15.16:0/442340237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b40b46c0 con 0x7fb2b415dce0 2024-04-25T10:10:57.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b3fff640 1 --2- 172.21.15.16:0/442340237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b415dce0 0x7fb2b4153a30 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00c410 tx=0x7fb2ac00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b0ff9640 1 -- 172.21.15.16:0/442340237 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0074e0 con 0x7fb2b415dce0 2024-04-25T10:10:57.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b0ff9640 1 -- 172.21.15.16:0/442340237 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac0047e0 con 0x7fb2b415dce0 2024-04-25T10:10:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b9ae1640 1 -- 172.21.15.16:0/442340237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b40b49a0 con 0x7fb2b415dce0 2024-04-25T10:10:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b0ff9640 1 -- 172.21.15.16:0/442340237 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac004bd0 con 0x7fb2b415dce0 2024-04-25T10:10:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.768+0000 7fb2b9ae1640 1 -- 172.21.15.16:0/442340237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b40b4f60 con 0x7fb2b415dce0 2024-04-25T10:10:57.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.769+0000 7fb2b9ae1640 1 -- 172.21.15.16:0/442340237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b40b5540 con 0x7fb2b415dce0 2024-04-25T10:10:57.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.770+0000 7fb2b0ff9640 1 -- 172.21.15.16:0/442340237 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2ac011070 con 0x7fb2b415dce0 2024-04-25T10:10:57.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.770+0000 7fb2b0ff9640 1 --2- 172.21.15.16:0/442340237 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb29406b4d0 0x7fb29406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.770+0000 7fb2b0ff9640 1 -- 172.21.15.16:0/442340237 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb2ac089a50 con 0x7fb2b415dce0 2024-04-25T10:10:57.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.770+0000 7fb2b37fe640 1 --2- 172.21.15.16:0/442340237 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb29406b4d0 0x7fb29406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.771+0000 7fb2b37fe640 1 --2- 172.21.15.16:0/442340237 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb29406b4d0 0x7fb29406d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4034000 tx=0x7fb2a402f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.774+0000 7fb2b0ff9640 1 -- 172.21.15.16:0/442340237 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb2b40b5540 con 0x7fb2b415dce0 2024-04-25T10:10:57.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.957+0000 7fb2b9ae1640 1 -- 172.21.15.16:0/442340237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb2b40b5130 con 0x7fb2b415dce0 2024-04-25T10:10:57.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.958+0000 7fb2b0ff9640 1 -- 172.21.15.16:0/442340237 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb2ac01a2b0 con 0x7fb2b415dce0 2024-04-25T10:10:57.963 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:10:57.963 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-04-25T10:10:57.963 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:10:57.963 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-04-25T10:10:57.964 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-04-25T10:10:57.964 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:10:57.964 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-04-25T10:10:57.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.960+0000 7fb2b9ae1640 1 -- 172.21.15.16:0/442340237 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb29406b4d0 msgr2=0x7fb29406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.960+0000 7fb2b9ae1640 1 --2- 172.21.15.16:0/442340237 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb29406b4d0 0x7fb29406d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4034000 tx=0x7fb2a402f000 comp rx=0 tx=0).stop 2024-04-25T10:10:57.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.960+0000 7fb2b9ae1640 1 -- 172.21.15.16:0/442340237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b415dce0 msgr2=0x7fb2b4153a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.960+0000 7fb2b9ae1640 1 --2- 172.21.15.16:0/442340237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b415dce0 0x7fb2b4153a30 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00c410 tx=0x7fb2ac00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.960+0000 7fb2b9ae1640 1 -- 172.21.15.16:0/442340237 shutdown_connections 2024-04-25T10:10:57.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.960+0000 7fb2b9ae1640 1 --2- 172.21.15.16:0/442340237 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb29406b4d0 0x7fb29406d990 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.960+0000 7fb2b9ae1640 1 --2- 172.21.15.16:0/442340237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2b415dce0 0x7fb2b4153a30 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.960+0000 7fb2b9ae1640 1 --2- 172.21.15.16:0/442340237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2b40d35b0 0x7fb2b41534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.960+0000 7fb2b9ae1640 1 --2- 172.21.15.16:0/442340237 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2b40d1f90 0x7fb2b4160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.960+0000 7fb2b9ae1640 1 -- 172.21.15.16:0/442340237 >> 172.21.15.16:0/442340237 conn(0x7fb2b4072900 msgr2=0x7fb2b40d3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:57.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.960+0000 7fb2b9ae1640 1 -- 172.21.15.16:0/442340237 shutdown_connections 2024-04-25T10:10:57.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:57.960+0000 7fb2b9ae1640 1 -- 172.21.15.16:0/442340237 wait complete. 2024-04-25T10:10:57.976 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:10:57.976 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:10:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.076+0000 7f236f876640 1 Processor -- start 2024-04-25T10:10:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.076+0000 7f236f876640 1 -- start start 2024-04-25T10:10:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.077+0000 7f236f876640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f236815dc50 0x7f2368070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.077+0000 7f236f876640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23680cf600 0x7f23680713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.077+0000 7f236f876640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236806c220 con 0x7f23680cd940 2024-04-25T10:10:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.077+0000 7f236f876640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23680cfe00 con 0x7f23680cf600 2024-04-25T10:10:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.077+0000 7f236f876640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23680cec10 con 0x7f236815dc50 2024-04-25T10:10:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.077+0000 7f236de13640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23680cf600 0x7f23680713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.077+0000 7f236ce11640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f23680cd940 0x7f2368161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46266/0 (socket says 172.21.15.16:46266) 2024-04-25T10:10:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.077+0000 7f236ce11640 1 -- 172.21.15.16:0/3733917331 learned_addr learned my addr 172.21.15.16:0/3733917331 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.077+0000 7f236d612640 1 --2- 172.21.15.16:0/3733917331 >> v2:172.21.15.186:3300/0 conn(0x7f236815dc50 0x7f2368070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.077+0000 7f236de13640 1 -- 172.21.15.16:0/3733917331 >> v2:172.21.15.186:3300/0 conn(0x7f236815dc50 msgr2=0x7f2368070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.077+0000 7f236de13640 1 --2- 172.21.15.16:0/3733917331 >> v2:172.21.15.186:3300/0 conn(0x7f236815dc50 0x7f2368070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.077+0000 7f236de13640 1 -- 172.21.15.16:0/3733917331 >> v1:172.21.15.16:6789/0 conn(0x7f23680cd940 legacy=0x7f2368161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.077+0000 7f235ffff640 1 -- 172.21.15.16:0/3733917331 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706236366 0 0) 0x7f236806c220 con 0x7f23680cd940 2024-04-25T10:10:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.077+0000 7f236de13640 1 -- 172.21.15.16:0/3733917331 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23680d0050 con 0x7f23680cf600 2024-04-25T10:10:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.077+0000 7f236d612640 1 --2- 172.21.15.16:0/3733917331 >> v2:172.21.15.186:3300/0 conn(0x7f236815dc50 0x7f2368070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.078+0000 7f236de13640 1 --2- 172.21.15.16:0/3733917331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23680cf600 0x7f23680713c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f2368072fc0 tx=0x7f236002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5866597e6d62fb57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.078+0000 7f235ffff640 1 -- 172.21.15.16:0/3733917331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360007740 con 0x7f23680cf600 2024-04-25T10:10:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.078+0000 7f235ffff640 1 -- 172.21.15.16:0/3733917331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23600078e0 con 0x7f23680cf600 2024-04-25T10:10:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.078+0000 7f235ffff640 1 -- 172.21.15.16:0/3733917331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360007cb0 con 0x7f23680cf600 2024-04-25T10:10:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.078+0000 7f236f876640 1 -- 172.21.15.16:0/3733917331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23680cf600 msgr2=0x7f23680713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.078+0000 7f236f876640 1 --2- 172.21.15.16:0/3733917331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23680cf600 0x7f23680713c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f2368072fc0 tx=0x7f236002a5f0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.078+0000 7f236f876640 1 -- 172.21.15.16:0/3733917331 shutdown_connections 2024-04-25T10:10:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.078+0000 7f236f876640 1 --2- 172.21.15.16:0/3733917331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23680cf600 0x7f23680713c0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.078+0000 7f236f876640 1 --2- 172.21.15.16:0/3733917331 >> v2:172.21.15.186:3300/0 conn(0x7f236815dc50 0x7f2368070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.078+0000 7f236f876640 1 -- 172.21.15.16:0/3733917331 >> 172.21.15.16:0/3733917331 conn(0x7f236806f430 msgr2=0x7f236806f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.079+0000 7f236f876640 1 -- 172.21.15.16:0/3733917331 shutdown_connections 2024-04-25T10:10:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.079+0000 7f236f876640 1 -- 172.21.15.16:0/3733917331 wait complete. 2024-04-25T10:10:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.080+0000 7f236f876640 1 Processor -- start 2024-04-25T10:10:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.080+0000 7f236f876640 1 -- start start 2024-04-25T10:10:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.080+0000 7f236f876640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23680cd940 0x7f2368160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.080+0000 7f236f876640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23680cf600 0x7f2368151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.080+0000 7f236f876640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f236815dc50 0x7f23680caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.080+0000 7f236f876640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23680ce8a0 con 0x7f23680cd940 2024-04-25T10:10:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.080+0000 7f236f876640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23680cc0c0 con 0x7f236815dc50 2024-04-25T10:10:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.080+0000 7f236f876640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23680cd620 con 0x7f23680cf600 2024-04-25T10:10:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.081+0000 7f236d612640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23680cd940 0x7f2368160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.081+0000 7f236d612640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23680cd940 0x7f2368160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55454/0 (socket says 172.21.15.16:55454) 2024-04-25T10:10:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.081+0000 7f236d612640 1 -- 172.21.15.16:0/1922064343 learned_addr learned my addr 172.21.15.16:0/1922064343 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.081+0000 7f236ce11640 1 --2- 172.21.15.16:0/1922064343 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23680cf600 0x7f2368151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.081+0000 7f236d612640 1 -- 172.21.15.16:0/1922064343 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23680cf600 msgr2=0x7f2368151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.081+0000 7f236de13640 1 --2- 172.21.15.16:0/1922064343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f236815dc50 0x7f23680caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.081+0000 7f236d612640 1 --2- 172.21.15.16:0/1922064343 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23680cf600 0x7f2368151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.081+0000 7f236d612640 1 -- 172.21.15.16:0/1922064343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f236815dc50 msgr2=0x7f23680caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.081+0000 7f236d612640 1 --2- 172.21.15.16:0/1922064343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f236815dc50 0x7f23680caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.081+0000 7f236d612640 1 -- 172.21.15.16:0/1922064343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236815fb00 con 0x7f23680cd940 2024-04-25T10:10:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.081+0000 7f236ce11640 1 --2- 172.21.15.16:0/1922064343 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23680cf600 0x7f2368151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.081+0000 7f236de13640 1 --2- 172.21.15.16:0/1922064343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f236815dc50 0x7f23680caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.081+0000 7f236d612640 1 --2- 172.21.15.16:0/1922064343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23680cd940 0x7f2368160ba0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f236400d7d0 tx=0x7f236400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.081+0000 7f235dffb640 1 -- 172.21.15.16:0/1922064343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364004030 con 0x7f23680cd940 2024-04-25T10:10:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.081+0000 7f235dffb640 1 -- 172.21.15.16:0/1922064343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23640041d0 con 0x7f23680cd940 2024-04-25T10:10:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.081+0000 7f236f876640 1 -- 172.21.15.16:0/1922064343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236815fde0 con 0x7f23680cd940 2024-04-25T10:10:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.081+0000 7f235dffb640 1 -- 172.21.15.16:0/1922064343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236400b6c0 con 0x7f23680cd940 2024-04-25T10:10:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.082+0000 7f236f876640 1 -- 172.21.15.16:0/1922064343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23681603a0 con 0x7f23680cd940 2024-04-25T10:10:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.082+0000 7f236f876640 1 -- 172.21.15.16:0/1922064343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23680750b0 con 0x7f23680cd940 2024-04-25T10:10:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.083+0000 7f235dffb640 1 -- 172.21.15.16:0/1922064343 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f236400b860 con 0x7f23680cd940 2024-04-25T10:10:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.083+0000 7f235dffb640 1 --2- 172.21.15.16:0/1922064343 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f234c06b500 0x7f234c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.083+0000 7f235dffb640 1 -- 172.21.15.16:0/1922064343 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f236408a0d0 con 0x7f23680cd940 2024-04-25T10:10:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.083+0000 7f236ce11640 1 --2- 172.21.15.16:0/1922064343 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f234c06b500 0x7f234c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.084+0000 7f236ce11640 1 --2- 172.21.15.16:0/1922064343 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f234c06b500 0x7f234c06d9c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f23680ce740 tx=0x7f2358009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.087+0000 7f235dffb640 1 -- 172.21.15.16:0/1922064343 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2364059330 con 0x7f23680cd940 2024-04-25T10:10:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.268+0000 7f236f876640 1 -- 172.21.15.16:0/1922064343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f236806c220 con 0x7f23680cd940 2024-04-25T10:10:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.638+0000 7f235dffb640 1 -- 172.21.15.16:0/1922064343 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2364058c30 con 0x7f23680cd940 2024-04-25T10:10:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-04-25T10:10:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.641+0000 7f236f876640 1 -- 172.21.15.16:0/1922064343 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f234c06b500 msgr2=0x7f234c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.641+0000 7f236f876640 1 --2- 172.21.15.16:0/1922064343 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f234c06b500 0x7f234c06d9c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f23680ce740 tx=0x7f2358009040 comp rx=0 tx=0).stop 2024-04-25T10:10:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.641+0000 7f236f876640 1 -- 172.21.15.16:0/1922064343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23680cd940 msgr2=0x7f2368160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.641+0000 7f236f876640 1 --2- 172.21.15.16:0/1922064343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23680cd940 0x7f2368160ba0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f236400d7d0 tx=0x7f236400dc90 comp rx=0 tx=0).stop 2024-04-25T10:10:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.641+0000 7f236f876640 1 -- 172.21.15.16:0/1922064343 shutdown_connections 2024-04-25T10:10:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.641+0000 7f236f876640 1 --2- 172.21.15.16:0/1922064343 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f234c06b500 0x7f234c06d9c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.641+0000 7f236f876640 1 --2- 172.21.15.16:0/1922064343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f236815dc50 0x7f23680caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.641+0000 7f236f876640 1 --2- 172.21.15.16:0/1922064343 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23680cf600 0x7f2368151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.641+0000 7f236f876640 1 --2- 172.21.15.16:0/1922064343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23680cd940 0x7f2368160ba0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.641+0000 7f236f876640 1 -- 172.21.15.16:0/1922064343 >> 172.21.15.16:0/1922064343 conn(0x7f236806f430 msgr2=0x7f23680b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.641+0000 7f236f876640 1 -- 172.21.15.16:0/1922064343 shutdown_connections 2024-04-25T10:10:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.641+0000 7f236f876640 1 -- 172.21.15.16:0/1922064343 wait complete. 2024-04-25T10:10:58.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:10:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.764+0000 7fee3e8ee640 1 Processor -- start 2024-04-25T10:10:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.764+0000 7fee3e8ee640 1 -- start start 2024-04-25T10:10:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.765+0000 7fee3e8ee640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee380b9290 0x7fee380d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.765+0000 7fee3e8ee640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3815e070 con 0x7fee380b96d0 2024-04-25T10:10:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.765+0000 7fee3e8ee640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee380d1b10 con 0x7fee3815dc90 2024-04-25T10:10:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.765+0000 7fee3e8ee640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee380d1df0 con 0x7fee380b9290 2024-04-25T10:10:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.765+0000 7fee3ce8b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fee380b96d0 0x7fee380b6aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46282/0 (socket says 172.21.15.16:46282) 2024-04-25T10:10:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.765+0000 7fee3ce8b640 1 -- 172.21.15.16:0/1425545707 learned_addr learned my addr 172.21.15.16:0/1425545707 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.765+0000 7fee36ffd640 1 -- 172.21.15.16:0/1425545707 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138582910 0 0) 0x7fee3815e070 con 0x7fee380b96d0 2024-04-25T10:10:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.765+0000 7fee36ffd640 1 -- 172.21.15.16:0/1425545707 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee20003520 con 0x7fee380b96d0 2024-04-25T10:10:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.765+0000 7fee377fe640 1 --2- 172.21.15.16:0/1425545707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee380b9290 0x7fee380d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.765+0000 7fee36ffd640 1 -- 172.21.15.16:0/1425545707 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4218583662 0 0) 0x7fee20003520 con 0x7fee380b96d0 2024-04-25T10:10:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.765+0000 7fee36ffd640 1 -- 172.21.15.16:0/1425545707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee380b9290 msgr2=0x7fee380d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.765+0000 7fee36ffd640 1 --2- 172.21.15.16:0/1425545707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee380b9290 0x7fee380d0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.765+0000 7fee36ffd640 1 -- 172.21.15.16:0/1425545707 >> v1:172.21.15.186:6789/0 conn(0x7fee3815dc90 legacy=0x7fee380d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.765+0000 7fee36ffd640 1 -- 172.21.15.16:0/1425545707 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee380d20d0 con 0x7fee380b96d0 2024-04-25T10:10:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.765+0000 7fee36ffd640 1 -- 172.21.15.16:0/1425545707 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fee300019f0 con 0x7fee380b96d0 2024-04-25T10:10:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.765+0000 7fee36ffd640 1 -- 172.21.15.16:0/1425545707 >> v1:172.21.15.16:6789/0 conn(0x7fee380b96d0 legacy=0x7fee380b6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.765+0000 7fee36ffd640 1 --2- 172.21.15.16:0/1425545707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee200039e0 0x7fee20003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.765+0000 7fee36ffd640 1 --2- 172.21.15.16:0/1425545707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee20004800 0x7fee20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.766+0000 7fee36ffd640 1 --2- 172.21.15.16:0/1425545707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee380b9290 0x7fee200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.766+0000 7fee36ffd640 1 -- 172.21.15.16:0/1425545707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee380d1df0 con 0x7fee200039e0 2024-04-25T10:10:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.766+0000 7fee36ffd640 1 -- 172.21.15.16:0/1425545707 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2000baa0 con 0x7fee20004800 2024-04-25T10:10:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.766+0000 7fee36ffd640 1 -- 172.21.15.16:0/1425545707 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2000bca0 con 0x7fee380b9290 2024-04-25T10:10:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.766+0000 7fee37fff640 1 --2- 172.21.15.16:0/1425545707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee200039e0 0x7fee20003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.766+0000 7fee3ce8b640 1 --2- 172.21.15.16:0/1425545707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee380b9290 0x7fee200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.766+0000 7fee377fe640 1 --2- 172.21.15.16:0/1425545707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee20004800 0x7fee20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.766+0000 7fee37fff640 1 -- 172.21.15.16:0/1425545707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee380b9290 msgr2=0x7fee200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.766+0000 7fee37fff640 1 --2- 172.21.15.16:0/1425545707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee380b9290 0x7fee200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.766+0000 7fee37fff640 1 -- 172.21.15.16:0/1425545707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee20004800 msgr2=0x7fee20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.766+0000 7fee37fff640 1 --2- 172.21.15.16:0/1425545707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee20004800 0x7fee20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.766+0000 7fee37fff640 1 -- 172.21.15.16:0/1425545707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee20003520 con 0x7fee200039e0 2024-04-25T10:10:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.767+0000 7fee37fff640 1 --2- 172.21.15.16:0/1425545707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee200039e0 0x7fee20003e00 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fee28001720 tx=0x7fee20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32b2466de1bffbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.767+0000 7fee36ffd640 1 -- 172.21.15.16:0/1425545707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee28001370 con 0x7fee200039e0 2024-04-25T10:10:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.767+0000 7fee36ffd640 1 -- 172.21.15.16:0/1425545707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee280052d0 con 0x7fee200039e0 2024-04-25T10:10:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.767+0000 7fee36ffd640 1 -- 172.21.15.16:0/1425545707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2802ab90 con 0x7fee200039e0 2024-04-25T10:10:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.767+0000 7fee3e8ee640 1 -- 172.21.15.16:0/1425545707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee200039e0 msgr2=0x7fee20003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.767+0000 7fee3e8ee640 1 --2- 172.21.15.16:0/1425545707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee200039e0 0x7fee20003e00 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fee28001720 tx=0x7fee20005830 comp rx=0 tx=0).stop 2024-04-25T10:10:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.767+0000 7fee3ce8b640 1 -- 172.21.15.16:0/1425545707 reap_dead start 2024-04-25T10:10:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.767+0000 7fee3e8ee640 1 -- 172.21.15.16:0/1425545707 shutdown_connections 2024-04-25T10:10:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.767+0000 7fee3e8ee640 1 -- 172.21.15.16:0/1425545707 >> 172.21.15.16:0/1425545707 conn(0x7fee38072900 msgr2=0x7fee3806c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.767+0000 7fee3e8ee640 1 -- 172.21.15.16:0/1425545707 shutdown_connections 2024-04-25T10:10:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.767+0000 7fee3e8ee640 1 -- 172.21.15.16:0/1425545707 wait complete. 2024-04-25T10:10:58.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.769+0000 7fee3e8ee640 1 Processor -- start 2024-04-25T10:10:58.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.769+0000 7fee3e8ee640 1 -- start start 2024-04-25T10:10:58.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.769+0000 7fee3e8ee640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee200039e0 0x7fee38152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.769+0000 7fee3e8ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee38153660 0x7fee380b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.769+0000 7fee3e8ee640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee38153ae0 0x7fee380b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.769+0000 7fee3e8ee640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee380d3480 con 0x7fee38153660 2024-04-25T10:10:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.769+0000 7fee3e8ee640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee380d2690 con 0x7fee200039e0 2024-04-25T10:10:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.769+0000 7fee3e8ee640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee380d3600 con 0x7fee38153ae0 2024-04-25T10:10:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.770+0000 7fee377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee38153660 0x7fee380b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.770+0000 7fee377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee38153660 0x7fee380b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55464/0 (socket says 172.21.15.16:55464) 2024-04-25T10:10:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.770+0000 7fee377fe640 1 -- 172.21.15.16:0/1311602594 learned_addr learned my addr 172.21.15.16:0/1311602594 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.770+0000 7fee37fff640 1 --2- 172.21.15.16:0/1311602594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee200039e0 0x7fee38152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.770+0000 7fee3ce8b640 1 --2- 172.21.15.16:0/1311602594 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee38153ae0 0x7fee380b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.770+0000 7fee377fe640 1 -- 172.21.15.16:0/1311602594 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee38153ae0 msgr2=0x7fee380b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.770+0000 7fee377fe640 1 --2- 172.21.15.16:0/1311602594 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee38153ae0 0x7fee380b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.770+0000 7fee377fe640 1 -- 172.21.15.16:0/1311602594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee200039e0 msgr2=0x7fee38152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.770+0000 7fee377fe640 1 --2- 172.21.15.16:0/1311602594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee200039e0 0x7fee38152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.770+0000 7fee377fe640 1 -- 172.21.15.16:0/1311602594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee381562b0 con 0x7fee38153660 2024-04-25T10:10:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.770+0000 7fee37fff640 1 --2- 172.21.15.16:0/1311602594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee200039e0 0x7fee38152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.770+0000 7fee377fe640 1 --2- 172.21.15.16:0/1311602594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee38153660 0x7fee380b4450 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fee2c011940 tx=0x7fee2c011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.770+0000 7fee34ff9640 1 -- 172.21.15.16:0/1311602594 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c00faf0 con 0x7fee38153660 2024-04-25T10:10:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.770+0000 7fee34ff9640 1 -- 172.21.15.16:0/1311602594 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c00fc90 con 0x7fee38153660 2024-04-25T10:10:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.770+0000 7fee3e8ee640 1 -- 172.21.15.16:0/1311602594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee38156590 con 0x7fee38153660 2024-04-25T10:10:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.770+0000 7fee34ff9640 1 -- 172.21.15.16:0/1311602594 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c014440 con 0x7fee38153660 2024-04-25T10:10:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.770+0000 7fee3e8ee640 1 -- 172.21.15.16:0/1311602594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee38156f60 con 0x7fee38153660 2024-04-25T10:10:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.772+0000 7fee3e8ee640 1 -- 172.21.15.16:0/1311602594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee380749b0 con 0x7fee38153660 2024-04-25T10:10:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.772+0000 7fee34ff9640 1 -- 172.21.15.16:0/1311602594 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee2c012040 con 0x7fee38153660 2024-04-25T10:10:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.772+0000 7fee34ff9640 1 --2- 172.21.15.16:0/1311602594 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fee2406b500 0x7fee2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.773+0000 7fee34ff9640 1 -- 172.21.15.16:0/1311602594 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fee2c08a580 con 0x7fee38153660 2024-04-25T10:10:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.773+0000 7fee37fff640 1 --2- 172.21.15.16:0/1311602594 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fee2406b500 0x7fee2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.773+0000 7fee37fff640 1 --2- 172.21.15.16:0/1311602594 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fee2406b500 0x7fee2406d9c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fee28032000 tx=0x7fee2802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.777+0000 7fee34ff9640 1 -- 172.21.15.16:0/1311602594 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee2c019030 con 0x7fee38153660 2024-04-25T10:10:58.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:58.963+0000 7fee3e8ee640 1 -- 172.21.15.16:0/1311602594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fee3815a200 con 0x7fee38153660 2024-04-25T10:10:59.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.640+0000 7fee34ff9640 1 -- 172.21.15.16:0/1311602594 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) v1 ==== 170+0+0 (secure 0 0 0) 0x7fee2c0595d0 con 0x7fee38153660 2024-04-25T10:10:59.646 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-04-25T10:10:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.646+0000 7fee3e8ee640 1 -- 172.21.15.16:0/1311602594 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fee2406b500 msgr2=0x7fee2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.646+0000 7fee3e8ee640 1 --2- 172.21.15.16:0/1311602594 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fee2406b500 0x7fee2406d9c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fee28032000 tx=0x7fee2802e000 comp rx=0 tx=0).stop 2024-04-25T10:10:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.646+0000 7fee3e8ee640 1 -- 172.21.15.16:0/1311602594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee38153660 msgr2=0x7fee380b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.646+0000 7fee3e8ee640 1 --2- 172.21.15.16:0/1311602594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee38153660 0x7fee380b4450 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fee2c011940 tx=0x7fee2c011e00 comp rx=0 tx=0).stop 2024-04-25T10:10:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.647+0000 7fee3e8ee640 1 -- 172.21.15.16:0/1311602594 shutdown_connections 2024-04-25T10:10:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.647+0000 7fee3e8ee640 1 --2- 172.21.15.16:0/1311602594 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fee2406b500 0x7fee2406d9c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.647+0000 7fee3e8ee640 1 --2- 172.21.15.16:0/1311602594 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee38153ae0 0x7fee380b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.647+0000 7fee3e8ee640 1 --2- 172.21.15.16:0/1311602594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee38153660 0x7fee380b4450 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.647+0000 7fee3e8ee640 1 --2- 172.21.15.16:0/1311602594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee200039e0 0x7fee38152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.647+0000 7fee3e8ee640 1 -- 172.21.15.16:0/1311602594 >> 172.21.15.16:0/1311602594 conn(0x7fee38072900 msgr2=0x7fee380b8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.653+0000 7fee3e8ee640 1 -- 172.21.15.16:0/1311602594 shutdown_connections 2024-04-25T10:10:59.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.655+0000 7fee3e8ee640 1 -- 172.21.15.16:0/1311602594 wait complete. 2024-04-25T10:10:59.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:10:59.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.781+0000 7f99620f5640 1 Processor -- start 2024-04-25T10:10:59.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.782+0000 7f99620f5640 1 -- start start 2024-04-25T10:10:59.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.782+0000 7f99620f5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f995c15bbe0 0x7f995c0cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.782+0000 7f99620f5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f995c0d0610 0x7f995c0cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.782+0000 7f99620f5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995c0cc830 con 0x7f995c0cf510 2024-04-25T10:10:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.782+0000 7f99620f5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f995c0ccb10 con 0x7f995c0d0610 2024-04-25T10:10:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.782+0000 7f99620f5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f995c0ccdf0 con 0x7f995c15bbe0 2024-04-25T10:10:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.783+0000 7f995bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f995c0d0610 0x7f995c0cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.783+0000 7f995affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f995c0cf510 0x7f995c0cff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46294/0 (socket says 172.21.15.16:46294) 2024-04-25T10:10:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.783+0000 7f995affd640 1 -- 172.21.15.16:0/2007537171 learned_addr learned my addr 172.21.15.16:0/2007537171 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:10:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.783+0000 7f995bfff640 1 --2- 172.21.15.16:0/2007537171 >> v2:172.21.15.16:3300/0 conn(0x7f995c0d0610 0x7f995c0cf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55472/0 (socket says 172.21.15.16:55472) 2024-04-25T10:10:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.783+0000 7f995b7fe640 1 --2- 172.21.15.16:0/2007537171 >> v2:172.21.15.186:3300/0 conn(0x7f995c15bbe0 0x7f995c0cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.783+0000 7f995bfff640 1 -- 172.21.15.16:0/2007537171 >> v2:172.21.15.186:3300/0 conn(0x7f995c15bbe0 msgr2=0x7f995c0cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.783+0000 7f995bfff640 1 --2- 172.21.15.16:0/2007537171 >> v2:172.21.15.186:3300/0 conn(0x7f995c15bbe0 0x7f995c0cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.783+0000 7f995bfff640 1 -- 172.21.15.16:0/2007537171 >> v1:172.21.15.16:6789/0 conn(0x7f995c0cf510 legacy=0x7f995c0cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.783+0000 7f995a7fc640 1 -- 172.21.15.16:0/2007537171 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 442744894 0 0) 0x7f995c0cc830 con 0x7f995c0cf510 2024-04-25T10:10:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.783+0000 7f995bfff640 1 -- 172.21.15.16:0/2007537171 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995c0cd0d0 con 0x7f995c0d0610 2024-04-25T10:10:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.784+0000 7f995bfff640 1 --2- 172.21.15.16:0/2007537171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c0d0610 0x7f995c0cf970 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f995c09a700 tx=0x7f995402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485d10c58a7d3a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.784+0000 7f995a7fc640 1 -- 172.21.15.16:0/2007537171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954007740 con 0x7f995c0d0610 2024-04-25T10:10:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.784+0000 7f995a7fc640 1 -- 172.21.15.16:0/2007537171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99540078e0 con 0x7f995c0d0610 2024-04-25T10:10:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.784+0000 7f995a7fc640 1 -- 172.21.15.16:0/2007537171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954007cb0 con 0x7f995c0d0610 2024-04-25T10:10:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.784+0000 7f99620f5640 1 -- 172.21.15.16:0/2007537171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c0d0610 msgr2=0x7f995c0cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.784+0000 7f99620f5640 1 --2- 172.21.15.16:0/2007537171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c0d0610 0x7f995c0cf970 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f995c09a700 tx=0x7f995402a680 comp rx=0 tx=0).stop 2024-04-25T10:10:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.784+0000 7f99620f5640 1 -- 172.21.15.16:0/2007537171 shutdown_connections 2024-04-25T10:10:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.784+0000 7f99620f5640 1 --2- 172.21.15.16:0/2007537171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c0d0610 0x7f995c0cf970 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.784+0000 7f99620f5640 1 --2- 172.21.15.16:0/2007537171 >> v2:172.21.15.186:3300/0 conn(0x7f995c15bbe0 0x7f995c0cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.784+0000 7f99620f5640 1 -- 172.21.15.16:0/2007537171 >> 172.21.15.16:0/2007537171 conn(0x7f995c0b3430 msgr2=0x7f995c0b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.784+0000 7f99620f5640 1 -- 172.21.15.16:0/2007537171 shutdown_connections 2024-04-25T10:10:59.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.784+0000 7f99620f5640 1 -- 172.21.15.16:0/2007537171 wait complete. 2024-04-25T10:10:59.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.785+0000 7f99620f5640 1 Processor -- start 2024-04-25T10:10:59.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.786+0000 7f99620f5640 1 -- start start 2024-04-25T10:10:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.786+0000 7f99620f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c0cf510 0x7f995c14b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.786+0000 7f99620f5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f995c0d0610 0x7f995c14d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.786+0000 7f99620f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f995c15bbe0 0x7f995c14dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.786+0000 7f99620f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c0cde10 con 0x7f995c0cf510 2024-04-25T10:10:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.786+0000 7f99620f5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c15bfc0 con 0x7f995c0d0610 2024-04-25T10:10:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.786+0000 7f99620f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995c0cc450 con 0x7f995c15bbe0 2024-04-25T10:10:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.787+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c0cf510 0x7f995c14b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.787+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c0cf510 0x7f995c14b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55480/0 (socket says 172.21.15.16:55480) 2024-04-25T10:10:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.787+0000 7f995b7fe640 1 -- 172.21.15.16:0/3286459776 learned_addr learned my addr 172.21.15.16:0/3286459776 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:10:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.787+0000 7f995b7fe640 1 -- 172.21.15.16:0/3286459776 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f995c15bbe0 msgr2=0x7f995c14dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.787+0000 7f995bfff640 1 --2- 172.21.15.16:0/3286459776 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f995c15bbe0 0x7f995c14dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.787+0000 7f995affd640 1 --2- 172.21.15.16:0/3286459776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f995c0d0610 0x7f995c14d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.787+0000 7f995b7fe640 1 --2- 172.21.15.16:0/3286459776 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f995c15bbe0 0x7f995c14dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.787+0000 7f995b7fe640 1 -- 172.21.15.16:0/3286459776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f995c0d0610 msgr2=0x7f995c14d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.787+0000 7f995b7fe640 1 --2- 172.21.15.16:0/3286459776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f995c0d0610 0x7f995c14d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.787+0000 7f995b7fe640 1 -- 172.21.15.16:0/3286459776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995c15e8b0 con 0x7f995c0cf510 2024-04-25T10:10:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.787+0000 7f995b7fe640 1 --2- 172.21.15.16:0/3286459776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c0cf510 0x7f995c14b020 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f994c00d800 tx=0x7f994c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.787+0000 7f9943fff640 1 -- 172.21.15.16:0/3286459776 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c00b6c0 con 0x7f995c0cf510 2024-04-25T10:10:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.787+0000 7f9943fff640 1 -- 172.21.15.16:0/3286459776 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994c019070 con 0x7f995c0cf510 2024-04-25T10:10:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.787+0000 7f99620f5640 1 -- 172.21.15.16:0/3286459776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995c14b960 con 0x7f995c0cf510 2024-04-25T10:10:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.787+0000 7f9943fff640 1 -- 172.21.15.16:0/3286459776 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c00ba80 con 0x7f995c0cf510 2024-04-25T10:10:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.787+0000 7f99620f5640 1 -- 172.21.15.16:0/3286459776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995c14bf20 con 0x7f995c0cf510 2024-04-25T10:10:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.788+0000 7f99620f5640 1 -- 172.21.15.16:0/3286459776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f995c0750b0 con 0x7f995c0cf510 2024-04-25T10:10:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.789+0000 7f9943fff640 1 -- 172.21.15.16:0/3286459776 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f994c00bc20 con 0x7f995c0cf510 2024-04-25T10:10:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.789+0000 7f9943fff640 1 --2- 172.21.15.16:0/3286459776 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f994806b4d0 0x7f994806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.789+0000 7f9943fff640 1 -- 172.21.15.16:0/3286459776 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f994c08b1a0 con 0x7f995c0cf510 2024-04-25T10:10:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.790+0000 7f995affd640 1 --2- 172.21.15.16:0/3286459776 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f994806b4d0 0x7f994806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.790+0000 7f995affd640 1 --2- 172.21.15.16:0/3286459776 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f994806b4d0 0x7f994806d990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f995c15f140 tx=0x7f9950009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:59.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:10:59.793+0000 7f9943fff640 1 -- 172.21.15.16:0/3286459776 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f994c05a060 con 0x7f995c0cf510 2024-04-25T10:11:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.013+0000 7f99620f5640 1 -- 172.21.15.16:0/3286459776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f995c0c4bb0 con 0x7f995c0cf510 2024-04-25T10:11:00.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.658+0000 7f9943fff640 1 -- 172.21.15.16:0/3286459776 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v49) v1 ==== 174+0+0 (secure 0 0 0) 0x7f994c059960 con 0x7f995c0cf510 2024-04-25T10:11:00.664 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 12 and data pool 13 2024-04-25T10:11:00.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.663+0000 7f99620f5640 1 -- 172.21.15.16:0/3286459776 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f994806b4d0 msgr2=0x7f994806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.663+0000 7f99620f5640 1 --2- 172.21.15.16:0/3286459776 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f994806b4d0 0x7f994806d990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f995c15f140 tx=0x7f9950009040 comp rx=0 tx=0).stop 2024-04-25T10:11:00.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.663+0000 7f99620f5640 1 -- 172.21.15.16:0/3286459776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c0cf510 msgr2=0x7f995c14b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.663+0000 7f99620f5640 1 --2- 172.21.15.16:0/3286459776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c0cf510 0x7f995c14b020 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f994c00d800 tx=0x7f994c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.665+0000 7f99620f5640 1 -- 172.21.15.16:0/3286459776 shutdown_connections 2024-04-25T10:11:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.665+0000 7f99620f5640 1 --2- 172.21.15.16:0/3286459776 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f994806b4d0 0x7f994806d990 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.665+0000 7f99620f5640 1 --2- 172.21.15.16:0/3286459776 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f995c15bbe0 0x7f995c14dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.665+0000 7f99620f5640 1 --2- 172.21.15.16:0/3286459776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f995c0d0610 0x7f995c14d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.665+0000 7f99620f5640 1 --2- 172.21.15.16:0/3286459776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c0cf510 0x7f995c14b020 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.665+0000 7f99620f5640 1 -- 172.21.15.16:0/3286459776 >> 172.21.15.16:0/3286459776 conn(0x7f995c0b3430 msgr2=0x7f995c0cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.666+0000 7f99620f5640 1 -- 172.21.15.16:0/3286459776 shutdown_connections 2024-04-25T10:11:00.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.666+0000 7f99620f5640 1 -- 172.21.15.16:0/3286459776 wait complete. 2024-04-25T10:11:00.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.789+0000 7fd2cf553640 1 Processor -- start 2024-04-25T10:11:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.789+0000 7fd2cf553640 1 -- start start 2024-04-25T10:11:00.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.790+0000 7fd2cf553640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd2c80d35b0 0x7fd2c80b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.790+0000 7fd2cf553640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd2c80b5990 0x7fd2c80b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.790+0000 7fd2cf553640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c815e0c0 con 0x7fd2c815dce0 2024-04-25T10:11:00.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.790+0000 7fd2cf553640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c80b5de0 con 0x7fd2c80b5990 2024-04-25T10:11:00.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.790+0000 7fd2cf553640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80d2510 con 0x7fd2c80d35b0 2024-04-25T10:11:00.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.790+0000 7fd2cdaf0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd2c80b5990 0x7fd2c80b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.790+0000 7fd2cdaf0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd2c80b5990 0x7fd2c80b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55492/0 (socket says 172.21.15.16:55492) 2024-04-25T10:11:00.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.790+0000 7fd2cdaf0640 1 -- 172.21.15.16:0/2981416735 learned_addr learned my addr 172.21.15.16:0/2981416735 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:00.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.790+0000 7fd2cd2ef640 1 --1- 172.21.15.16:0/2981416735 >> v1:172.21.15.16:6789/0 conn(0x7fd2c815dce0 0x7fd2c80d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46302/0 (socket says 172.21.15.16:46302) 2024-04-25T10:11:00.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.791+0000 7fd2ccaee640 1 --2- 172.21.15.16:0/2981416735 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd2c80d35b0 0x7fd2c80b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.791+0000 7fd2cdaf0640 1 -- 172.21.15.16:0/2981416735 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd2c80d35b0 msgr2=0x7fd2c80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:00.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.791+0000 7fd2cdaf0640 1 --2- 172.21.15.16:0/2981416735 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd2c80d35b0 0x7fd2c80b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.791+0000 7fd2cdaf0640 1 -- 172.21.15.16:0/2981416735 >> v1:172.21.15.16:6789/0 conn(0x7fd2c815dce0 legacy=0x7fd2c80d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.791+0000 7fd2c7fff640 1 -- 172.21.15.16:0/2981416735 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784274169 0 0) 0x7fd2c815e0c0 con 0x7fd2c815dce0 2024-04-25T10:11:00.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.791+0000 7fd2cdaf0640 1 -- 172.21.15.16:0/2981416735 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c80d27f0 con 0x7fd2c80b5990 2024-04-25T10:11:00.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.791+0000 7fd2ccaee640 1 --2- 172.21.15.16:0/2981416735 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd2c80d35b0 0x7fd2c80b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:00.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.791+0000 7fd2cdaf0640 1 --2- 172.21.15.16:0/2981416735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c80b5990 0x7fd2c80b6b00 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009e50 tx=0x7fd2b802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82884a62f9787b0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:00.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.792+0000 7fd2c7fff640 1 -- 172.21.15.16:0/2981416735 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8036070 con 0x7fd2c80b5990 2024-04-25T10:11:00.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.792+0000 7fd2c7fff640 1 -- 172.21.15.16:0/2981416735 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b8031600 con 0x7fd2c80b5990 2024-04-25T10:11:00.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.792+0000 7fd2c7fff640 1 -- 172.21.15.16:0/2981416735 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b80319d0 con 0x7fd2c80b5990 2024-04-25T10:11:00.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.792+0000 7fd2cf553640 1 -- 172.21.15.16:0/2981416735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c80b5990 msgr2=0x7fd2c80b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.792+0000 7fd2cf553640 1 --2- 172.21.15.16:0/2981416735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c80b5990 0x7fd2c80b6b00 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009e50 tx=0x7fd2b802a710 comp rx=0 tx=0).stop 2024-04-25T10:11:00.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.792+0000 7fd2cf553640 1 -- 172.21.15.16:0/2981416735 shutdown_connections 2024-04-25T10:11:00.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.792+0000 7fd2cf553640 1 --2- 172.21.15.16:0/2981416735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c80b5990 0x7fd2c80b6b00 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.792+0000 7fd2cf553640 1 --2- 172.21.15.16:0/2981416735 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd2c80d35b0 0x7fd2c80b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.792+0000 7fd2cf553640 1 -- 172.21.15.16:0/2981416735 >> 172.21.15.16:0/2981416735 conn(0x7fd2c8072900 msgr2=0x7fd2c80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:00.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.792+0000 7fd2cf553640 1 -- 172.21.15.16:0/2981416735 shutdown_connections 2024-04-25T10:11:00.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.792+0000 7fd2cf553640 1 -- 172.21.15.16:0/2981416735 wait complete. 2024-04-25T10:11:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.793+0000 7fd2cf553640 1 Processor -- start 2024-04-25T10:11:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.794+0000 7fd2cf553640 1 -- start start 2024-04-25T10:11:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.794+0000 7fd2cf553640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd2c80b5990 0x7fd2c81520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.794+0000 7fd2cf553640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c80d35b0 0x7fd2c815f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.794+0000 7fd2cf553640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c815dce0 0x7fd2c815f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.794+0000 7fd2cf553640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80d3090 con 0x7fd2c80d35b0 2024-04-25T10:11:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.794+0000 7fd2cf553640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80d2db0 con 0x7fd2c815dce0 2024-04-25T10:11:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.794+0000 7fd2cf553640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80d2160 con 0x7fd2c80b5990 2024-04-25T10:11:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.794+0000 7fd2ccaee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c80d35b0 0x7fd2c815f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.794+0000 7fd2ccaee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c80d35b0 0x7fd2c815f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55498/0 (socket says 172.21.15.16:55498) 2024-04-25T10:11:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.794+0000 7fd2ccaee640 1 -- 172.21.15.16:0/1197552520 learned_addr learned my addr 172.21.15.16:0/1197552520 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.795+0000 7fd2cdaf0640 1 --2- 172.21.15.16:0/1197552520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c815dce0 0x7fd2c815f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.795+0000 7fd2cd2ef640 1 --2- 172.21.15.16:0/1197552520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd2c80b5990 0x7fd2c81520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.795+0000 7fd2ccaee640 1 -- 172.21.15.16:0/1197552520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd2c80b5990 msgr2=0x7fd2c81520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.795+0000 7fd2ccaee640 1 --2- 172.21.15.16:0/1197552520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd2c80b5990 0x7fd2c81520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.795+0000 7fd2ccaee640 1 -- 172.21.15.16:0/1197552520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c815dce0 msgr2=0x7fd2c815f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.795+0000 7fd2ccaee640 1 --2- 172.21.15.16:0/1197552520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c815dce0 0x7fd2c815f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.795+0000 7fd2ccaee640 1 -- 172.21.15.16:0/1197552520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c80cd530 con 0x7fd2c80d35b0 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.795+0000 7fd2cd2ef640 1 --2- 172.21.15.16:0/1197552520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd2c80b5990 0x7fd2c81520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.795+0000 7fd2ccaee640 1 --2- 172.21.15.16:0/1197552520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c80d35b0 0x7fd2c815f0a0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00e810 tx=0x7fd2bc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.795+0000 7fd2c5ffb640 1 -- 172.21.15.16:0/1197552520 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc00f040 con 0x7fd2c80d35b0 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.795+0000 7fd2c5ffb640 1 -- 172.21.15.16:0/1197552520 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc004030 con 0x7fd2c80d35b0 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.795+0000 7fd2cf553640 1 -- 172.21.15.16:0/1197552520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c80c76a0 con 0x7fd2c80d35b0 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.795+0000 7fd2c5ffb640 1 -- 172.21.15.16:0/1197552520 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc004400 con 0x7fd2c80d35b0 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.795+0000 7fd2cf553640 1 -- 172.21.15.16:0/1197552520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c80c7c20 con 0x7fd2c80d35b0 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.796+0000 7fd2cf553640 1 -- 172.21.15.16:0/1197552520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c80749b0 con 0x7fd2c80d35b0 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.797+0000 7fd2c5ffb640 1 -- 172.21.15.16:0/1197552520 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd2bc00c6d0 con 0x7fd2c80d35b0 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.797+0000 7fd2c5ffb640 1 --2- 172.21.15.16:0/1197552520 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd2b406b500 0x7fd2b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.797+0000 7fd2c5ffb640 1 -- 172.21.15.16:0/1197552520 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2bc089ea0 con 0x7fd2c80d35b0 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.797+0000 7fd2cd2ef640 1 --2- 172.21.15.16:0/1197552520 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd2b406b500 0x7fd2b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.798+0000 7fd2cd2ef640 1 --2- 172.21.15.16:0/1197552520 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd2b406b500 0x7fd2b406d9c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0000f60 tx=0x7fd2c0007ca0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:00.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.801+0000 7fd2c5ffb640 1 -- 172.21.15.16:0/1197552520 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd2bc017020 con 0x7fd2c80d35b0 2024-04-25T10:11:00.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.980+0000 7fd2cf553640 1 -- 172.21.15.16:0/1197552520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2c80d1f40 con 0x7fd2c80d35b0 2024-04-25T10:11:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.982+0000 7fd2c5ffb640 1 -- 172.21.15.16:0/1197552520 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16685 (secure 0 0 0) 0x7fd2bc0586d0 con 0x7fd2c80d35b0 2024-04-25T10:11:00.987 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:00.987 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":85,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:11:00.014756+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:58.299405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:58.994953+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.983+0000 7fd2cf553640 1 -- 172.21.15.16:0/1197552520 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd2b406b500 msgr2=0x7fd2b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.983+0000 7fd2cf553640 1 --2- 172.21.15.16:0/1197552520 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd2b406b500 0x7fd2b406d9c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0000f60 tx=0x7fd2c0007ca0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.983+0000 7fd2cf553640 1 -- 172.21.15.16:0/1197552520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c80d35b0 msgr2=0x7fd2c815f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.983+0000 7fd2cf553640 1 --2- 172.21.15.16:0/1197552520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c80d35b0 0x7fd2c815f0a0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00e810 tx=0x7fd2bc00ecd0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.984+0000 7fd2cf553640 1 -- 172.21.15.16:0/1197552520 shutdown_connections 2024-04-25T10:11:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.984+0000 7fd2cf553640 1 --2- 172.21.15.16:0/1197552520 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd2b406b500 0x7fd2b406d9c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.984+0000 7fd2cf553640 1 --2- 172.21.15.16:0/1197552520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c815dce0 0x7fd2c815f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.984+0000 7fd2cf553640 1 --2- 172.21.15.16:0/1197552520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c80d35b0 0x7fd2c815f0a0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.984+0000 7fd2cf553640 1 --2- 172.21.15.16:0/1197552520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd2c80b5990 0x7fd2c81520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.984+0000 7fd2cf553640 1 -- 172.21.15.16:0/1197552520 >> 172.21.15.16:0/1197552520 conn(0x7fd2c8072900 msgr2=0x7fd2c80d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.984+0000 7fd2cf553640 1 -- 172.21.15.16:0/1197552520 shutdown_connections 2024-04-25T10:11:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:00.984+0000 7fd2cf553640 1 -- 172.21.15.16:0/1197552520 wait complete. 2024-04-25T10:11:01.000 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.095+0000 7f633f170640 1 Processor -- start 2024-04-25T10:11:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.095+0000 7f633f170640 1 -- start start 2024-04-25T10:11:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.096+0000 7f633f170640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f633815dce0 0x7f63380d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.096+0000 7f633f170640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f63380d35b0 0x7f63380b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.096+0000 7f633f170640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f63380d1f90 0x7f63380b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.096+0000 7f633f170640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633815e0c0 con 0x7f63380d1f90 2024-04-25T10:11:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.096+0000 7f633f170640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63380d2510 con 0x7f63380d35b0 2024-04-25T10:11:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.096+0000 7f633f170640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63380d27f0 con 0x7f633815dce0 2024-04-25T10:11:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.096+0000 7f633d70d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f63380d1f90 0x7f63380b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.096+0000 7f633d70d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f63380d1f90 0x7f63380b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55514/0 (socket says 172.21.15.16:55514) 2024-04-25T10:11:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.096+0000 7f633d70d640 1 -- 172.21.15.16:0/3735456415 learned_addr learned my addr 172.21.15.16:0/3735456415 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.096+0000 7f633cf0c640 1 --2- 172.21.15.16:0/3735456415 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f633815dce0 0x7f63380d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.096+0000 7f6337fff640 1 --2- 172.21.15.16:0/3735456415 >> v2:172.21.15.186:3300/0 conn(0x7f63380d35b0 0x7f63380b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.096+0000 7f633d70d640 1 -- 172.21.15.16:0/3735456415 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f633815dce0 msgr2=0x7f63380d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.096+0000 7f633d70d640 1 --2- 172.21.15.16:0/3735456415 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f633815dce0 0x7f63380d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.096+0000 7f633d70d640 1 -- 172.21.15.16:0/3735456415 >> v2:172.21.15.186:3300/0 conn(0x7f63380d35b0 msgr2=0x7f63380b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.096+0000 7f633d70d640 1 --2- 172.21.15.16:0/3735456415 >> v2:172.21.15.186:3300/0 conn(0x7f63380d35b0 0x7f63380b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.096+0000 7f633d70d640 1 -- 172.21.15.16:0/3735456415 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63380d2ad0 con 0x7f63380d1f90 2024-04-25T10:11:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.097+0000 7f633d70d640 1 --2- 172.21.15.16:0/3735456415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63380d1f90 0x7f63380b5990 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f633806d0d0 tx=0x7f633002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a071a62208e01d98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.097+0000 7f63377fe640 1 -- 172.21.15.16:0/3735456415 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330036070 con 0x7f63380d1f90 2024-04-25T10:11:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.097+0000 7f63377fe640 1 -- 172.21.15.16:0/3735456415 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6330031600 con 0x7f63380d1f90 2024-04-25T10:11:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.097+0000 7f63377fe640 1 -- 172.21.15.16:0/3735456415 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63300319d0 con 0x7f63380d1f90 2024-04-25T10:11:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.098+0000 7f633f170640 1 -- 172.21.15.16:0/3735456415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63380d1f90 msgr2=0x7f63380b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.098+0000 7f633f170640 1 --2- 172.21.15.16:0/3735456415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63380d1f90 0x7f63380b5990 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f633806d0d0 tx=0x7f633002a610 comp rx=0 tx=0).stop 2024-04-25T10:11:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.098+0000 7f633f170640 1 -- 172.21.15.16:0/3735456415 shutdown_connections 2024-04-25T10:11:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.098+0000 7f633f170640 1 --2- 172.21.15.16:0/3735456415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63380d1f90 0x7f63380b5990 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.098+0000 7f633f170640 1 --2- 172.21.15.16:0/3735456415 >> v2:172.21.15.186:3300/0 conn(0x7f63380d35b0 0x7f63380b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.098+0000 7f633f170640 1 --2- 172.21.15.16:0/3735456415 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f633815dce0 0x7f63380d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.098+0000 7f633f170640 1 -- 172.21.15.16:0/3735456415 >> 172.21.15.16:0/3735456415 conn(0x7f6338072900 msgr2=0x7f63380d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.098+0000 7f633f170640 1 -- 172.21.15.16:0/3735456415 shutdown_connections 2024-04-25T10:11:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.098+0000 7f633f170640 1 -- 172.21.15.16:0/3735456415 wait complete. 2024-04-25T10:11:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.099+0000 7f633f170640 1 Processor -- start 2024-04-25T10:11:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.099+0000 7f633f170640 1 -- start start 2024-04-25T10:11:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.100+0000 7f633f170640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63380d1f90 0x7f6338149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.100+0000 7f633f170640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63380d35b0 0x7f63380c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.100+0000 7f633f170640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f633815dce0 0x7f63380c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.100+0000 7f633f170640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63380b9820 con 0x7f63380d1f90 2024-04-25T10:11:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.100+0000 7f633f170640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63380d3090 con 0x7f63380d35b0 2024-04-25T10:11:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.100+0000 7f633f170640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63380b99a0 con 0x7f633815dce0 2024-04-25T10:11:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.100+0000 7f633cf0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63380d1f90 0x7f6338149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.100+0000 7f633cf0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63380d1f90 0x7f6338149cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55524/0 (socket says 172.21.15.16:55524) 2024-04-25T10:11:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.100+0000 7f633cf0c640 1 -- 172.21.15.16:0/1958276588 learned_addr learned my addr 172.21.15.16:0/1958276588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.100+0000 7f633d70d640 1 --2- 172.21.15.16:0/1958276588 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f633815dce0 0x7f63380c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.100+0000 7f6337fff640 1 --2- 172.21.15.16:0/1958276588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63380d35b0 0x7f63380c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.100+0000 7f633cf0c640 1 -- 172.21.15.16:0/1958276588 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f633815dce0 msgr2=0x7f63380c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.100+0000 7f633cf0c640 1 --2- 172.21.15.16:0/1958276588 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f633815dce0 0x7f63380c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.100+0000 7f633cf0c640 1 -- 172.21.15.16:0/1958276588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63380d35b0 msgr2=0x7f63380c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.100+0000 7f633cf0c640 1 --2- 172.21.15.16:0/1958276588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63380d35b0 0x7f63380c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.100+0000 7f633cf0c640 1 -- 172.21.15.16:0/1958276588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6338160e90 con 0x7f63380d1f90 2024-04-25T10:11:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.100+0000 7f633cf0c640 1 --2- 172.21.15.16:0/1958276588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63380d1f90 0x7f6338149cd0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f632c00e9e0 tx=0x7f632c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.101+0000 7f63357fa640 1 -- 172.21.15.16:0/1958276588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c017070 con 0x7f63380d1f90 2024-04-25T10:11:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.101+0000 7f63357fa640 1 -- 172.21.15.16:0/1958276588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632c00c9d0 con 0x7f63380d1f90 2024-04-25T10:11:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.101+0000 7f63357fa640 1 -- 172.21.15.16:0/1958276588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00cdc0 con 0x7f63380d1f90 2024-04-25T10:11:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.101+0000 7f633f170640 1 -- 172.21.15.16:0/1958276588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633814a490 con 0x7f63380d1f90 2024-04-25T10:11:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.101+0000 7f633f170640 1 -- 172.21.15.16:0/1958276588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633814aa50 con 0x7f63380d1f90 2024-04-25T10:11:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.102+0000 7f633f170640 1 -- 172.21.15.16:0/1958276588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6338067490 con 0x7f63380d1f90 2024-04-25T10:11:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.102+0000 7f63357fa640 1 -- 172.21.15.16:0/1958276588 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f632c01a070 con 0x7f63380d1f90 2024-04-25T10:11:01.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.102+0000 7f63357fa640 1 --2- 172.21.15.16:0/1958276588 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f631c06b4d0 0x7f631c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.103+0000 7f63357fa640 1 -- 172.21.15.16:0/1958276588 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f632c08a500 con 0x7f63380d1f90 2024-04-25T10:11:01.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.103+0000 7f6337fff640 1 --2- 172.21.15.16:0/1958276588 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f631c06b4d0 0x7f631c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.103+0000 7f6337fff640 1 --2- 172.21.15.16:0/1958276588 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f631c06b4d0 0x7f631c06d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f63380d1e80 tx=0x7f6324009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:01.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.107+0000 7f63357fa640 1 -- 172.21.15.16:0/1958276588 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6338067490 con 0x7f63380d1f90 2024-04-25T10:11:01.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.290+0000 7f633f170640 1 -- 172.21.15.16:0/1958276588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63380749b0 con 0x7f63380d1f90 2024-04-25T10:11:01.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.291+0000 7f63357fa640 1 -- 172.21.15.16:0/1958276588 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16685 (secure 0 0 0) 0x7f632c058dc0 con 0x7f63380d1f90 2024-04-25T10:11:01.297 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:01.297 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":85,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:11:00.014756+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:58.299405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:58.994953+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.293+0000 7f633f170640 1 -- 172.21.15.16:0/1958276588 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f631c06b4d0 msgr2=0x7f631c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.293+0000 7f633f170640 1 --2- 172.21.15.16:0/1958276588 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f631c06b4d0 0x7f631c06d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f63380d1e80 tx=0x7f6324009190 comp rx=0 tx=0).stop 2024-04-25T10:11:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.293+0000 7f633f170640 1 -- 172.21.15.16:0/1958276588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63380d1f90 msgr2=0x7f6338149cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.293+0000 7f633f170640 1 --2- 172.21.15.16:0/1958276588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63380d1f90 0x7f6338149cd0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f632c00e9e0 tx=0x7f632c00eea0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.293+0000 7f633f170640 1 -- 172.21.15.16:0/1958276588 shutdown_connections 2024-04-25T10:11:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.293+0000 7f633f170640 1 --2- 172.21.15.16:0/1958276588 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f631c06b4d0 0x7f631c06d990 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.294+0000 7f633f170640 1 --2- 172.21.15.16:0/1958276588 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f633815dce0 0x7f63380c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.294+0000 7f633f170640 1 --2- 172.21.15.16:0/1958276588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63380d35b0 0x7f63380c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.294+0000 7f633f170640 1 --2- 172.21.15.16:0/1958276588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63380d1f90 0x7f6338149cd0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.294+0000 7f633f170640 1 -- 172.21.15.16:0/1958276588 >> 172.21.15.16:0/1958276588 conn(0x7f6338072900 msgr2=0x7f63380b8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.294+0000 7f633f170640 1 -- 172.21.15.16:0/1958276588 shutdown_connections 2024-04-25T10:11:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.294+0000 7f633f170640 1 -- 172.21.15.16:0/1958276588 wait complete. 2024-04-25T10:11:01.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:11:01.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.408+0000 7efea119e640 1 Processor -- start 2024-04-25T10:11:01.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.408+0000 7efea119e640 1 -- start start 2024-04-25T10:11:01.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.409+0000 7efea119e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efe9c15dca0 0x7efe9c0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.409+0000 7efea119e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7efe9c0d3b90 0x7efe9c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.409+0000 7efea119e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe9c0cee80 0x7efe9c0d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.409+0000 7efea119e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9c0b6b20 con 0x7efe9c15dca0 2024-04-25T10:11:01.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.409+0000 7efea119e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9c0d4090 con 0x7efe9c0d3b90 2024-04-25T10:11:01.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.409+0000 7efea119e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0d4370 con 0x7efe9c0cee80 2024-04-25T10:11:01.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.409+0000 7efe9ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efe9c15dca0 0x7efe9c0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.409+0000 7efe9ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efe9c15dca0 0x7efe9c0ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55526/0 (socket says 172.21.15.16:55526) 2024-04-25T10:11:01.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.409+0000 7efe9ad9d640 1 -- 172.21.15.16:0/4194314031 learned_addr learned my addr 172.21.15.16:0/4194314031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:01.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.409+0000 7efe9b59e640 1 --2- 172.21.15.16:0/4194314031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe9c0cee80 0x7efe9c0d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.409+0000 7efe9a59c640 1 --2- 172.21.15.16:0/4194314031 >> v2:172.21.15.186:3300/0 conn(0x7efe9c0d3b90 0x7efe9c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.409+0000 7efe9ad9d640 1 -- 172.21.15.16:0/4194314031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe9c0cee80 msgr2=0x7efe9c0d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:01.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.409+0000 7efe9ad9d640 1 --2- 172.21.15.16:0/4194314031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe9c0cee80 0x7efe9c0d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.409+0000 7efe9ad9d640 1 -- 172.21.15.16:0/4194314031 >> v2:172.21.15.186:3300/0 conn(0x7efe9c0d3b90 msgr2=0x7efe9c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:01.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.409+0000 7efe9ad9d640 1 --2- 172.21.15.16:0/4194314031 >> v2:172.21.15.186:3300/0 conn(0x7efe9c0d3b90 0x7efe9c0d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.410+0000 7efe9ad9d640 1 -- 172.21.15.16:0/4194314031 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c0d4650 con 0x7efe9c15dca0 2024-04-25T10:11:01.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.410+0000 7efe9b59e640 1 --2- 172.21.15.16:0/4194314031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe9c0cee80 0x7efe9c0d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:01.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.410+0000 7efe9ad9d640 1 --2- 172.21.15.16:0/4194314031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c15dca0 0x7efe9c0ce940 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0b6560 tx=0x7efe9402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274476df629f2509 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:01.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.410+0000 7efe99d9b640 1 -- 172.21.15.16:0/4194314031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94037070 con 0x7efe9c15dca0 2024-04-25T10:11:01.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.410+0000 7efe99d9b640 1 -- 172.21.15.16:0/4194314031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe94032440 con 0x7efe9c15dca0 2024-04-25T10:11:01.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.410+0000 7efe99d9b640 1 -- 172.21.15.16:0/4194314031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94032810 con 0x7efe9c15dca0 2024-04-25T10:11:01.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.411+0000 7efea119e640 1 -- 172.21.15.16:0/4194314031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c15dca0 msgr2=0x7efe9c0ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.411+0000 7efea119e640 1 --2- 172.21.15.16:0/4194314031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c15dca0 0x7efe9c0ce940 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0b6560 tx=0x7efe9402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.411+0000 7efea119e640 1 -- 172.21.15.16:0/4194314031 shutdown_connections 2024-04-25T10:11:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.411+0000 7efea119e640 1 --2- 172.21.15.16:0/4194314031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe9c0cee80 0x7efe9c0d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.411+0000 7efea119e640 1 --2- 172.21.15.16:0/4194314031 >> v2:172.21.15.186:3300/0 conn(0x7efe9c0d3b90 0x7efe9c0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.411+0000 7efea119e640 1 --2- 172.21.15.16:0/4194314031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c15dca0 0x7efe9c0ce940 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.411+0000 7efea119e640 1 -- 172.21.15.16:0/4194314031 >> 172.21.15.16:0/4194314031 conn(0x7efe9c0726c0 msgr2=0x7efe9c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.411+0000 7efea119e640 1 -- 172.21.15.16:0/4194314031 shutdown_connections 2024-04-25T10:11:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.411+0000 7efea119e640 1 -- 172.21.15.16:0/4194314031 wait complete. 2024-04-25T10:11:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.412+0000 7efea119e640 1 Processor -- start 2024-04-25T10:11:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.413+0000 7efea119e640 1 -- start start 2024-04-25T10:11:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.413+0000 7efea119e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c0cee80 0x7efe9c161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.413+0000 7efea119e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe9c0d3b90 0x7efe9c1f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.413+0000 7efea119e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe9c15dca0 0x7efe9c0cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.413+0000 7efea119e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c15e200 con 0x7efe9c0cee80 2024-04-25T10:11:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.413+0000 7efea119e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c15e080 con 0x7efe9c0d3b90 2024-04-25T10:11:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.413+0000 7efea119e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0b2500 con 0x7efe9c15dca0 2024-04-25T10:11:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.413+0000 7efe9ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c0cee80 0x7efe9c161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.413+0000 7efe9ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c0cee80 0x7efe9c161010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55536/0 (socket says 172.21.15.16:55536) 2024-04-25T10:11:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.413+0000 7efe9ad9d640 1 -- 172.21.15.16:0/3776311981 learned_addr learned my addr 172.21.15.16:0/3776311981 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.414+0000 7efe9b59e640 1 --2- 172.21.15.16:0/3776311981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe9c15dca0 0x7efe9c0cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.414+0000 7efe9a59c640 1 --2- 172.21.15.16:0/3776311981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe9c0d3b90 0x7efe9c1f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.414+0000 7efe9ad9d640 1 -- 172.21.15.16:0/3776311981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe9c15dca0 msgr2=0x7efe9c0cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.414+0000 7efe9ad9d640 1 --2- 172.21.15.16:0/3776311981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe9c15dca0 0x7efe9c0cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.414+0000 7efe9ad9d640 1 -- 172.21.15.16:0/3776311981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe9c0d3b90 msgr2=0x7efe9c1f0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.414+0000 7efe9ad9d640 1 --2- 172.21.15.16:0/3776311981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe9c0d3b90 0x7efe9c1f0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.414+0000 7efe9ad9d640 1 -- 172.21.15.16:0/3776311981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c1f12e0 con 0x7efe9c0cee80 2024-04-25T10:11:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.414+0000 7efe9ad9d640 1 --2- 172.21.15.16:0/3776311981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c0cee80 0x7efe9c161010 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7efe9402a220 tx=0x7efe940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.414+0000 7efe837fe640 1 -- 172.21.15.16:0/3776311981 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94041020 con 0x7efe9c0cee80 2024-04-25T10:11:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.414+0000 7efe837fe640 1 -- 172.21.15.16:0/3776311981 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe940306c0 con 0x7efe9c0cee80 2024-04-25T10:11:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.414+0000 7efea119e640 1 -- 172.21.15.16:0/3776311981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9c1f4170 con 0x7efe9c0cee80 2024-04-25T10:11:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.414+0000 7efe837fe640 1 -- 172.21.15.16:0/3776311981 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94037040 con 0x7efe9c0cee80 2024-04-25T10:11:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.414+0000 7efea119e640 1 -- 172.21.15.16:0/3776311981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9c158480 con 0x7efe9c0cee80 2024-04-25T10:11:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.415+0000 7efea119e640 1 -- 172.21.15.16:0/3776311981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe9c0750b0 con 0x7efe9c0cee80 2024-04-25T10:11:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.416+0000 7efe837fe640 1 -- 172.21.15.16:0/3776311981 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe94044020 con 0x7efe9c0cee80 2024-04-25T10:11:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.416+0000 7efe837fe640 1 --2- 172.21.15.16:0/3776311981 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efe8806b4d0 0x7efe8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.416+0000 7efe837fe640 1 -- 172.21.15.16:0/3776311981 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe940aa990 con 0x7efe9c0cee80 2024-04-25T10:11:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.416+0000 7efe9a59c640 1 --2- 172.21.15.16:0/3776311981 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efe8806b4d0 0x7efe8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.417+0000 7efe9a59c640 1 --2- 172.21.15.16:0/3776311981 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efe8806b4d0 0x7efe8806d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0b82a0 tx=0x7efe90008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.420+0000 7efe837fe640 1 -- 172.21.15.16:0/3776311981 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe9404b030 con 0x7efe9c0cee80 2024-04-25T10:11:01.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.640+0000 7efea119e640 1 -- 172.21.15.16:0/3776311981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7efe9c0d2320 con 0x7efe9c0cee80 2024-04-25T10:11:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.657+0000 7efe837fe640 1 -- 172.21.15.16:0/3776311981 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v50) v1 ==== 118+0+0 (secure 0 0 0) 0x7efe94079810 con 0x7efe9c0cee80 2024-04-25T10:11:01.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.660+0000 7efea119e640 1 -- 172.21.15.16:0/3776311981 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efe8806b4d0 msgr2=0x7efe8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.660+0000 7efea119e640 1 --2- 172.21.15.16:0/3776311981 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efe8806b4d0 0x7efe8806d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0b82a0 tx=0x7efe90008040 comp rx=0 tx=0).stop 2024-04-25T10:11:01.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.660+0000 7efea119e640 1 -- 172.21.15.16:0/3776311981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c0cee80 msgr2=0x7efe9c161010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.660+0000 7efea119e640 1 --2- 172.21.15.16:0/3776311981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c0cee80 0x7efe9c161010 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7efe9402a220 tx=0x7efe940046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.660+0000 7efea119e640 1 -- 172.21.15.16:0/3776311981 shutdown_connections 2024-04-25T10:11:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.660+0000 7efea119e640 1 --2- 172.21.15.16:0/3776311981 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efe8806b4d0 0x7efe8806d990 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.660+0000 7efea119e640 1 --2- 172.21.15.16:0/3776311981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe9c15dca0 0x7efe9c0cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.660+0000 7efea119e640 1 --2- 172.21.15.16:0/3776311981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe9c0d3b90 0x7efe9c1f0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.660+0000 7efea119e640 1 --2- 172.21.15.16:0/3776311981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c0cee80 0x7efe9c161010 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.660+0000 7efea119e640 1 -- 172.21.15.16:0/3776311981 >> 172.21.15.16:0/3776311981 conn(0x7efe9c0726c0 msgr2=0x7efe9c0b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.660+0000 7efea119e640 1 -- 172.21.15.16:0/3776311981 shutdown_connections 2024-04-25T10:11:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.660+0000 7efea119e640 1 -- 172.21.15.16:0/3776311981 wait complete. 2024-04-25T10:11:01.677 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:11:01.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:11:01.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.776+0000 7f94b1742640 1 Processor -- start 2024-04-25T10:11:01.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.776+0000 7f94b1742640 1 -- start start 2024-04-25T10:11:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.777+0000 7f94b1742640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f94ac15a260 0x7f94ac071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.777+0000 7f94b1742640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ac15dc30 con 0x7f94ac06ec00 2024-04-25T10:11:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.777+0000 7f94b1742640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ac15df10 con 0x7f94ac071760 2024-04-25T10:11:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.777+0000 7f94b1742640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ac071bc0 con 0x7f94ac15a260 2024-04-25T10:11:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.777+0000 7f94ab7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f94ac06ec00 0x7f94ac0b2650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46314/0 (socket says 172.21.15.16:46314) 2024-04-25T10:11:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.777+0000 7f94ab7fe640 1 -- 172.21.15.16:0/307431552 learned_addr learned my addr 172.21.15.16:0/307431552 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.777+0000 7f94aaffd640 1 --2- 172.21.15.16:0/307431552 >> v2:172.21.15.186:3300/0 conn(0x7f94ac15a260 0x7f94ac071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.777+0000 7f94a9ffb640 1 -- 172.21.15.16:0/307431552 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242065421 0 0) 0x7f94ac15dc30 con 0x7f94ac06ec00 2024-04-25T10:11:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.777+0000 7f94a9ffb640 1 -- 172.21.15.16:0/307431552 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9494004630 con 0x7f94ac06ec00 2024-04-25T10:11:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.777+0000 7f94a9ffb640 1 -- 172.21.15.16:0/307431552 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801926030 0 0) 0x7f94ac15df10 con 0x7f94ac071760 2024-04-25T10:11:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.777+0000 7f94a9ffb640 1 -- 172.21.15.16:0/307431552 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94ac15dc30 con 0x7f94ac071760 2024-04-25T10:11:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.777+0000 7f94a9ffb640 1 -- 172.21.15.16:0/307431552 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2450198728 0 0) 0x7f9494004630 con 0x7f94ac06ec00 2024-04-25T10:11:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.777+0000 7f94a9ffb640 1 -- 172.21.15.16:0/307431552 >> v2:172.21.15.186:3300/0 conn(0x7f94ac15a260 msgr2=0x7f94ac071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.777+0000 7f94a9ffb640 1 --2- 172.21.15.16:0/307431552 >> v2:172.21.15.186:3300/0 conn(0x7f94ac15a260 0x7f94ac071220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.777+0000 7f94a9ffb640 1 -- 172.21.15.16:0/307431552 >> v1:172.21.15.186:6789/0 conn(0x7f94ac071760 legacy=0x7f94ac06e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.777+0000 7f94a9ffb640 1 -- 172.21.15.16:0/307431552 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac15e1f0 con 0x7f94ac06ec00 2024-04-25T10:11:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.778+0000 7f94a9ffb640 1 -- 172.21.15.16:0/307431552 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f949c0019f0 con 0x7f94ac06ec00 2024-04-25T10:11:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.778+0000 7f94a9ffb640 1 -- 172.21.15.16:0/307431552 >> v1:172.21.15.16:6789/0 conn(0x7f94ac06ec00 legacy=0x7f94ac0b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.778+0000 7f94a9ffb640 1 --2- 172.21.15.16:0/307431552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9494004c50 0x7f9494005050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.778+0000 7f94a9ffb640 1 --2- 172.21.15.16:0/307431552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494005910 0x7f9494005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.778+0000 7f94a9ffb640 1 --2- 172.21.15.16:0/307431552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94940062d0 0x7f9494003440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.778+0000 7f94a9ffb640 1 -- 172.21.15.16:0/307431552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9494003b60 con 0x7f9494005910 2024-04-25T10:11:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.778+0000 7f94a9ffb640 1 -- 172.21.15.16:0/307431552 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9494003d10 con 0x7f94940062d0 2024-04-25T10:11:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.778+0000 7f94a9ffb640 1 -- 172.21.15.16:0/307431552 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9494003f10 con 0x7f9494004c50 2024-04-25T10:11:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.778+0000 7f94aa7fc640 1 --2- 172.21.15.16:0/307431552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494005910 0x7f9494005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.778+0000 7f94aaffd640 1 --2- 172.21.15.16:0/307431552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9494004c50 0x7f9494005050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.778+0000 7f94ab7fe640 1 --2- 172.21.15.16:0/307431552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94940062d0 0x7f9494003440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.778+0000 7f94aa7fc640 1 -- 172.21.15.16:0/307431552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9494004c50 msgr2=0x7f9494005050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:01.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.778+0000 7f94aa7fc640 1 --2- 172.21.15.16:0/307431552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9494004c50 0x7f9494005050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.778+0000 7f94aa7fc640 1 -- 172.21.15.16:0/307431552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94940062d0 msgr2=0x7f9494003440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:01.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.778+0000 7f94aa7fc640 1 --2- 172.21.15.16:0/307431552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94940062d0 0x7f9494003440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.778+0000 7f94ab7fe640 1 -- 172.21.15.16:0/307431552 reap_dead start 2024-04-25T10:11:01.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.778+0000 7f94aa7fc640 1 -- 172.21.15.16:0/307431552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9494004630 con 0x7f9494005910 2024-04-25T10:11:01.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.779+0000 7f94aa7fc640 1 --2- 172.21.15.16:0/307431552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494005910 0x7f9494005d90 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f94a00018d0 tx=0x7f9494006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f20b1725a41963d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:01.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.779+0000 7f94a9ffb640 1 -- 172.21.15.16:0/307431552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a00014f0 con 0x7f9494005910 2024-04-25T10:11:01.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.779+0000 7f94a9ffb640 1 -- 172.21.15.16:0/307431552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a00052d0 con 0x7f9494005910 2024-04-25T10:11:01.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.779+0000 7f94a9ffb640 1 -- 172.21.15.16:0/307431552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a002ac70 con 0x7f9494005910 2024-04-25T10:11:01.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.779+0000 7f94b1742640 1 -- 172.21.15.16:0/307431552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494005910 msgr2=0x7f9494005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.779+0000 7f94b1742640 1 --2- 172.21.15.16:0/307431552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494005910 0x7f9494005d90 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f94a00018d0 tx=0x7f9494006940 comp rx=0 tx=0).stop 2024-04-25T10:11:01.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.779+0000 7f94b1742640 1 -- 172.21.15.16:0/307431552 shutdown_connections 2024-04-25T10:11:01.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.779+0000 7f94b1742640 1 --2- 172.21.15.16:0/307431552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494005910 0x7f9494005d90 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.779+0000 7f94b1742640 1 -- 172.21.15.16:0/307431552 >> 172.21.15.16:0/307431552 conn(0x7f94ac06aba0 msgr2=0x7f94ac06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:01.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.779+0000 7f94b1742640 1 -- 172.21.15.16:0/307431552 shutdown_connections 2024-04-25T10:11:01.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.780+0000 7f94b1742640 1 -- 172.21.15.16:0/307431552 wait complete. 2024-04-25T10:11:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.781+0000 7f94b1742640 1 Processor -- start 2024-04-25T10:11:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.781+0000 7f94b1742640 1 -- start start 2024-04-25T10:11:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.781+0000 7f94b1742640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9494005910 0x7f94ac151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.781+0000 7f94b1742640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ac15cf20 0x7f94ac1ef8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94b1742640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f94ac1efdf0 0x7f94ac1f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94b1742640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac071220 con 0x7f94ac15cf20 2024-04-25T10:11:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94b1742640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac15e7b0 con 0x7f9494005910 2024-04-25T10:11:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94b1742640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0713a0 con 0x7f94ac1efdf0 2024-04-25T10:11:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94aa7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ac15cf20 0x7f94ac1ef8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94aa7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ac15cf20 0x7f94ac1ef8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55558/0 (socket says 172.21.15.16:55558) 2024-04-25T10:11:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94aa7fc640 1 -- 172.21.15.16:0/694506094 learned_addr learned my addr 172.21.15.16:0/694506094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94aaffd640 1 --2- 172.21.15.16:0/694506094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9494005910 0x7f94ac151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94aa7fc640 1 -- 172.21.15.16:0/694506094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f94ac1efdf0 msgr2=0x7f94ac1f2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94ab7fe640 1 --2- 172.21.15.16:0/694506094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f94ac1efdf0 0x7f94ac1f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94aa7fc640 1 --2- 172.21.15.16:0/694506094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f94ac1efdf0 0x7f94ac1f2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94aa7fc640 1 -- 172.21.15.16:0/694506094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9494005910 msgr2=0x7f94ac151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94aa7fc640 1 --2- 172.21.15.16:0/694506094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9494005910 0x7f94ac151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94aa7fc640 1 -- 172.21.15.16:0/694506094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac151590 con 0x7f94ac15cf20 2024-04-25T10:11:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94aaffd640 1 --2- 172.21.15.16:0/694506094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9494005910 0x7f94ac151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94aa7fc640 1 --2- 172.21.15.16:0/694506094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ac15cf20 0x7f94ac1ef8b0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f94a00019f0 tx=0x7f94a00012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94937fe640 1 -- 172.21.15.16:0/694506094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a003f020 con 0x7f94ac15cf20 2024-04-25T10:11:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94937fe640 1 -- 172.21.15.16:0/694506094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a0005480 con 0x7f94ac15cf20 2024-04-25T10:11:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94b1742640 1 -- 172.21.15.16:0/694506094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ac1f0280 con 0x7f94ac15cf20 2024-04-25T10:11:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94937fe640 1 -- 172.21.15.16:0/694506094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a0037040 con 0x7f94ac15cf20 2024-04-25T10:11:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.782+0000 7f94b1742640 1 -- 172.21.15.16:0/694506094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ac15ad00 con 0x7f94ac15cf20 2024-04-25T10:11:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.783+0000 7f94b1742640 1 -- 172.21.15.16:0/694506094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ac0750b0 con 0x7f94ac15cf20 2024-04-25T10:11:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.784+0000 7f94937fe640 1 -- 172.21.15.16:0/694506094 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f94a0042020 con 0x7f94ac15cf20 2024-04-25T10:11:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.784+0000 7f94937fe640 1 --2- 172.21.15.16:0/694506094 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f949806b500 0x7f949806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.784+0000 7f94937fe640 1 -- 172.21.15.16:0/694506094 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94a00a9e50 con 0x7f94ac15cf20 2024-04-25T10:11:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.784+0000 7f94aaffd640 1 --2- 172.21.15.16:0/694506094 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f949806b500 0x7f949806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.785+0000 7f94aaffd640 1 --2- 172.21.15.16:0/694506094 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f949806b500 0x7f949806d9c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f94a4006d40 tx=0x7f94a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:01.789+0000 7f94937fe640 1 -- 172.21.15.16:0/694506094 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f94a0049030 con 0x7f94ac15cf20 2024-04-25T10:11:02.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.006+0000 7f94b1742640 1 -- 172.21.15.16:0/694506094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f94ac072310 con 0x7f94ac15cf20 2024-04-25T10:11:02.644 INFO:tasks.ceph.mds.b.smithi186.stderr:2024-04-25T10:11:02.638+0000 7fa67df9e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:11:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.671+0000 7f94937fe640 1 -- 172.21.15.16:0/694506094 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v51) v1 ==== 122+0+0 (secure 0 0 0) 0x7f94a0078cd0 con 0x7f94ac15cf20 2024-04-25T10:11:02.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.673+0000 7f94b1742640 1 -- 172.21.15.16:0/694506094 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f949806b500 msgr2=0x7f949806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.673+0000 7f94b1742640 1 --2- 172.21.15.16:0/694506094 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f949806b500 0x7f949806d9c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f94a4006d40 tx=0x7f94a4008040 comp rx=0 tx=0).stop 2024-04-25T10:11:02.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.673+0000 7f94b1742640 1 -- 172.21.15.16:0/694506094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ac15cf20 msgr2=0x7f94ac1ef8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.673+0000 7f94b1742640 1 --2- 172.21.15.16:0/694506094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ac15cf20 0x7f94ac1ef8b0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f94a00019f0 tx=0x7f94a00012d0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.674+0000 7f94b1742640 1 -- 172.21.15.16:0/694506094 shutdown_connections 2024-04-25T10:11:02.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.674+0000 7f94b1742640 1 --2- 172.21.15.16:0/694506094 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f949806b500 0x7f949806d9c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.674+0000 7f94b1742640 1 --2- 172.21.15.16:0/694506094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f94ac1efdf0 0x7f94ac1f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.674+0000 7f94b1742640 1 --2- 172.21.15.16:0/694506094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ac15cf20 0x7f94ac1ef8b0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.674+0000 7f94b1742640 1 --2- 172.21.15.16:0/694506094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9494005910 0x7f94ac151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.674+0000 7f94b1742640 1 -- 172.21.15.16:0/694506094 >> 172.21.15.16:0/694506094 conn(0x7f94ac06aba0 msgr2=0x7f94ac0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:02.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.674+0000 7f94b1742640 1 -- 172.21.15.16:0/694506094 shutdown_connections 2024-04-25T10:11:02.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.674+0000 7f94b1742640 1 -- 172.21.15.16:0/694506094 wait complete. 2024-04-25T10:11:02.692 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:11:02.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:02.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.794+0000 7fe8d0a58640 1 Processor -- start 2024-04-25T10:11:02.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.794+0000 7fe8d0a58640 1 -- start start 2024-04-25T10:11:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.794+0000 7fe8d0a58640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc15ba10 0x7fe8cc099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.794+0000 7fe8d0a58640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe8cc099f80 0x7fe8cc06bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.794+0000 7fe8d0a58640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe8cc06c1f0 0x7fe8cc0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.794+0000 7fe8d0a58640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8cc0cbe50 con 0x7fe8cc099f80 2024-04-25T10:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.794+0000 7fe8d0a58640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8cc0cfe80 con 0x7fe8cc06c1f0 2024-04-25T10:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.794+0000 7fe8d0a58640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc09a3a0 con 0x7fe8cc15ba10 2024-04-25T10:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.795+0000 7fe8c9d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe8cc099f80 0x7fe8cc06bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.795+0000 7fe8c9d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe8cc099f80 0x7fe8cc06bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53138/0 (socket says 172.21.15.16:53138) 2024-04-25T10:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.795+0000 7fe8c9d9b640 1 -- 172.21.15.16:0/1527101985 learned_addr learned my addr 172.21.15.16:0/1527101985 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.795+0000 7fe8cad9d640 1 --2- 172.21.15.16:0/1527101985 >> v2:172.21.15.186:3300/0 conn(0x7fe8cc06c1f0 0x7fe8cc0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.795+0000 7fe8ca59c640 1 --2- 172.21.15.16:0/1527101985 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc15ba10 0x7fe8cc099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.795+0000 7fe8c9d9b640 1 -- 172.21.15.16:0/1527101985 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc15ba10 msgr2=0x7fe8cc099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.795+0000 7fe8c9d9b640 1 --2- 172.21.15.16:0/1527101985 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc15ba10 0x7fe8cc099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.795+0000 7fe8c9d9b640 1 -- 172.21.15.16:0/1527101985 >> v2:172.21.15.186:3300/0 conn(0x7fe8cc06c1f0 msgr2=0x7fe8cc0cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.795+0000 7fe8c9d9b640 1 --2- 172.21.15.16:0/1527101985 >> v2:172.21.15.186:3300/0 conn(0x7fe8cc06c1f0 0x7fe8cc0cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.795+0000 7fe8c9d9b640 1 -- 172.21.15.16:0/1527101985 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc06c680 con 0x7fe8cc099f80 2024-04-25T10:11:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.795+0000 7fe8ca59c640 1 --2- 172.21.15.16:0/1527101985 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc15ba10 0x7fe8cc099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.796+0000 7fe8c9d9b640 1 --2- 172.21.15.16:0/1527101985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc099f80 0x7fe8cc06bcb0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc1616a0 tx=0x7fe8c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c427fad328f0f4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.796+0000 7fe8c959a640 1 -- 172.21.15.16:0/1527101985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0037070 con 0x7fe8cc099f80 2024-04-25T10:11:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.796+0000 7fe8c959a640 1 -- 172.21.15.16:0/1527101985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c0032440 con 0x7fe8cc099f80 2024-04-25T10:11:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.796+0000 7fe8c959a640 1 -- 172.21.15.16:0/1527101985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0032810 con 0x7fe8cc099f80 2024-04-25T10:11:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.796+0000 7fe8d0a58640 1 -- 172.21.15.16:0/1527101985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc099f80 msgr2=0x7fe8cc06bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.796+0000 7fe8d0a58640 1 --2- 172.21.15.16:0/1527101985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc099f80 0x7fe8cc06bcb0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc1616a0 tx=0x7fe8c002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.797+0000 7fe8d0a58640 1 -- 172.21.15.16:0/1527101985 shutdown_connections 2024-04-25T10:11:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.797+0000 7fe8d0a58640 1 --2- 172.21.15.16:0/1527101985 >> v2:172.21.15.186:3300/0 conn(0x7fe8cc06c1f0 0x7fe8cc0cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.797+0000 7fe8d0a58640 1 --2- 172.21.15.16:0/1527101985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc099f80 0x7fe8cc06bcb0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.797+0000 7fe8d0a58640 1 --2- 172.21.15.16:0/1527101985 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc15ba10 0x7fe8cc099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.797+0000 7fe8d0a58640 1 -- 172.21.15.16:0/1527101985 >> 172.21.15.16:0/1527101985 conn(0x7fe8cc0b32f0 msgr2=0x7fe8cc0b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.797+0000 7fe8d0a58640 1 -- 172.21.15.16:0/1527101985 shutdown_connections 2024-04-25T10:11:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.797+0000 7fe8d0a58640 1 -- 172.21.15.16:0/1527101985 wait complete. 2024-04-25T10:11:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.798+0000 7fe8d0a58640 1 Processor -- start 2024-04-25T10:11:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.798+0000 7fe8d0a58640 1 -- start start 2024-04-25T10:11:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.799+0000 7fe8d0a58640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc06c1f0 0x7fe8cc15a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.799+0000 7fe8d0a58640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc099f80 0x7fe8cc0c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.799+0000 7fe8d0a58640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc15ba10 0x7fe8cc0c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.799+0000 7fe8d0a58640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc06cec0 con 0x7fe8cc099f80 2024-04-25T10:11:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.799+0000 7fe8d0a58640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc06cbe0 con 0x7fe8cc15ba10 2024-04-25T10:11:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.799+0000 7fe8d0a58640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc0cd760 con 0x7fe8cc06c1f0 2024-04-25T10:11:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.799+0000 7fe8c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc099f80 0x7fe8cc0c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.799+0000 7fe8c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc099f80 0x7fe8cc0c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53154/0 (socket says 172.21.15.16:53154) 2024-04-25T10:11:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.799+0000 7fe8c9d9b640 1 -- 172.21.15.16:0/2338483551 learned_addr learned my addr 172.21.15.16:0/2338483551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.799+0000 7fe8cad9d640 1 --2- 172.21.15.16:0/2338483551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc15ba10 0x7fe8cc0c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.799+0000 7fe8ca59c640 1 --2- 172.21.15.16:0/2338483551 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc06c1f0 0x7fe8cc15a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.799+0000 7fe8c9d9b640 1 -- 172.21.15.16:0/2338483551 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc06c1f0 msgr2=0x7fe8cc15a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.799+0000 7fe8c9d9b640 1 --2- 172.21.15.16:0/2338483551 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc06c1f0 0x7fe8cc15a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.799+0000 7fe8c9d9b640 1 -- 172.21.15.16:0/2338483551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc15ba10 msgr2=0x7fe8cc0c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.799+0000 7fe8c9d9b640 1 --2- 172.21.15.16:0/2338483551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc15ba10 0x7fe8cc0c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.799+0000 7fe8c9d9b640 1 -- 172.21.15.16:0/2338483551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc0cb620 con 0x7fe8cc099f80 2024-04-25T10:11:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.799+0000 7fe8c9d9b640 1 --2- 172.21.15.16:0/2338483551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc099f80 0x7fe8cc0c8fc0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0000c00 tx=0x7fe8c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.799+0000 7fe8baffd640 1 -- 172.21.15.16:0/2338483551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0041020 con 0x7fe8cc099f80 2024-04-25T10:11:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.800+0000 7fe8baffd640 1 -- 172.21.15.16:0/2338483551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c0030970 con 0x7fe8cc099f80 2024-04-25T10:11:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.800+0000 7fe8baffd640 1 -- 172.21.15.16:0/2338483551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0037040 con 0x7fe8cc099f80 2024-04-25T10:11:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.800+0000 7fe8d0a58640 1 -- 172.21.15.16:0/2338483551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8cc160d90 con 0x7fe8cc099f80 2024-04-25T10:11:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.800+0000 7fe8d0a58640 1 -- 172.21.15.16:0/2338483551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8cc0c6620 con 0x7fe8cc099f80 2024-04-25T10:11:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.801+0000 7fe8d0a58640 1 -- 172.21.15.16:0/2338483551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8cc15b370 con 0x7fe8cc099f80 2024-04-25T10:11:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.801+0000 7fe8baffd640 1 -- 172.21.15.16:0/2338483551 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8c0044020 con 0x7fe8cc099f80 2024-04-25T10:11:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.801+0000 7fe8baffd640 1 --2- 172.21.15.16:0/2338483551 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe8ac06b4d0 0x7fe8ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.802+0000 7fe8baffd640 1 -- 172.21.15.16:0/2338483551 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8c00aabe0 con 0x7fe8cc099f80 2024-04-25T10:11:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.802+0000 7fe8ca59c640 1 --2- 172.21.15.16:0/2338483551 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe8ac06b4d0 0x7fe8ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.802+0000 7fe8ca59c640 1 --2- 172.21.15.16:0/2338483551 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe8ac06b4d0 0x7fe8ac06d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400a8d0 tx=0x7fe8c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:02.806+0000 7fe8baffd640 1 -- 172.21.15.16:0/2338483551 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8c004b030 con 0x7fe8cc099f80 2024-04-25T10:11:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.023+0000 7fe8d0a58640 1 -- 172.21.15.16:0/2338483551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8cc0749b0 con 0x7fe8cc099f80 2024-04-25T10:11:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.023+0000 7fe8baffd640 1 -- 172.21.15.16:0/2338483551 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe8cc0cb620 con 0x7fe8cc099f80 2024-04-25T10:11:03.029 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:03.029 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4044509901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4044509901},{"type":"v1","addr":"172.21.15.16:6835","nonce":4044509901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/84349474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":84349474},{"type":"v1","addr":"172.21.15.16:6834","nonce":84349474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7335,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/3950732718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3950732718},{"type":"v1","addr":"172.21.15.186:6836","nonce":3950732718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:00.015075+0000","modified":"2024-04-25T10:11:02.007788+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35852":{"gid":35852,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/3092291936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":3092291936},{"type":"v1","addr":"172.21.15.186:6837","nonce":3092291936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:03.029 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 51 2024-04-25T10:11:03.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.025+0000 7fe8d0a58640 1 -- 172.21.15.16:0/2338483551 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe8ac06b4d0 msgr2=0x7fe8ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.025+0000 7fe8d0a58640 1 --2- 172.21.15.16:0/2338483551 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe8ac06b4d0 0x7fe8ac06d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400a8d0 tx=0x7fe8c4008040 comp rx=0 tx=0).stop 2024-04-25T10:11:03.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.025+0000 7fe8d0a58640 1 -- 172.21.15.16:0/2338483551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc099f80 msgr2=0x7fe8cc0c8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.025+0000 7fe8d0a58640 1 --2- 172.21.15.16:0/2338483551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc099f80 0x7fe8cc0c8fc0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0000c00 tx=0x7fe8c00046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.025+0000 7fe8d0a58640 1 -- 172.21.15.16:0/2338483551 shutdown_connections 2024-04-25T10:11:03.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.025+0000 7fe8d0a58640 1 --2- 172.21.15.16:0/2338483551 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe8ac06b4d0 0x7fe8ac06d990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.025+0000 7fe8d0a58640 1 --2- 172.21.15.16:0/2338483551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc15ba10 0x7fe8cc0c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.025+0000 7fe8d0a58640 1 --2- 172.21.15.16:0/2338483551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc099f80 0x7fe8cc0c8fc0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.025+0000 7fe8d0a58640 1 --2- 172.21.15.16:0/2338483551 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc06c1f0 0x7fe8cc15a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.025+0000 7fe8d0a58640 1 -- 172.21.15.16:0/2338483551 >> 172.21.15.16:0/2338483551 conn(0x7fe8cc0b32f0 msgr2=0x7fe8cc0d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.025+0000 7fe8d0a58640 1 -- 172.21.15.16:0/2338483551 shutdown_connections 2024-04-25T10:11:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.025+0000 7fe8d0a58640 1 -- 172.21.15.16:0/2338483551 wait complete. 2024-04-25T10:11:03.042 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.137+0000 7fdbb5942640 1 Processor -- start 2024-04-25T10:11:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.138+0000 7fdbb5942640 1 -- start start 2024-04-25T10:11:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.138+0000 7fdbb5942640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fdbb009a150 0x7fdbb006bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.138+0000 7fdbb5942640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb006c660 con 0x7fdbb015ba10 2024-04-25T10:11:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.138+0000 7fdbb5942640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb006c900 con 0x7fdbb006c1f0 2024-04-25T10:11:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.138+0000 7fdbb5942640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb00cbe50 con 0x7fdbb009a150 2024-04-25T10:11:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.138+0000 7fdbaeffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdbb015ba10 0x7fdbb0099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55478/0 (socket says 172.21.15.16:55478) 2024-04-25T10:11:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.138+0000 7fdbaeffd640 1 -- 172.21.15.16:0/342729530 learned_addr learned my addr 172.21.15.16:0/342729530 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbadffb640 1 -- 172.21.15.16:0/342729530 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341411742 0 0) 0x7fdbb006c660 con 0x7fdbb015ba10 2024-04-25T10:11:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbadffb640 1 -- 172.21.15.16:0/342729530 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb9c003520 con 0x7fdbb015ba10 2024-04-25T10:11:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbae7fc640 1 --2- 172.21.15.16:0/342729530 >> v2:172.21.15.186:3300/0 conn(0x7fdbb009a150 0x7fdbb006bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbadffb640 1 -- 172.21.15.16:0/342729530 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1510403083 0 0) 0x7fdb9c003520 con 0x7fdbb015ba10 2024-04-25T10:11:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbadffb640 1 -- 172.21.15.16:0/342729530 >> v2:172.21.15.186:3300/0 conn(0x7fdbb009a150 msgr2=0x7fdbb006bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbadffb640 1 --2- 172.21.15.16:0/342729530 >> v2:172.21.15.186:3300/0 conn(0x7fdbb009a150 0x7fdbb006bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbadffb640 1 -- 172.21.15.16:0/342729530 >> v1:172.21.15.186:6789/0 conn(0x7fdbb006c1f0 legacy=0x7fdbb00cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbadffb640 1 -- 172.21.15.16:0/342729530 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb006cbe0 con 0x7fdbb015ba10 2024-04-25T10:11:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbadffb640 1 -- 172.21.15.16:0/342729530 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fdba8001670 con 0x7fdbb015ba10 2024-04-25T10:11:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbadffb640 1 -- 172.21.15.16:0/342729530 >> v1:172.21.15.16:6789/0 conn(0x7fdbb015ba10 legacy=0x7fdbb0099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbadffb640 1 --2- 172.21.15.16:0/342729530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9c0039b0 0x7fdb9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbadffb640 1 --2- 172.21.15.16:0/342729530 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9c004800 0x7fdb9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbadffb640 1 --2- 172.21.15.16:0/342729530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9c0051c0 0x7fdb9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbadffb640 1 -- 172.21.15.16:0/342729530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c00c0e0 con 0x7fdb9c0039b0 2024-04-25T10:11:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbadffb640 1 -- 172.21.15.16:0/342729530 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c00c290 con 0x7fdb9c0051c0 2024-04-25T10:11:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbadffb640 1 -- 172.21.15.16:0/342729530 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c00c490 con 0x7fdb9c004800 2024-04-25T10:11:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbaeffd640 1 --2- 172.21.15.16:0/342729530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9c0039b0 0x7fdb9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbae7fc640 1 --2- 172.21.15.16:0/342729530 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9c004800 0x7fdb9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbaf7fe640 1 --2- 172.21.15.16:0/342729530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9c0051c0 0x7fdb9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbaeffd640 1 -- 172.21.15.16:0/342729530 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9c004800 msgr2=0x7fdb9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbaeffd640 1 --2- 172.21.15.16:0/342729530 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9c004800 0x7fdb9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbaeffd640 1 -- 172.21.15.16:0/342729530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9c0051c0 msgr2=0x7fdb9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbaeffd640 1 --2- 172.21.15.16:0/342729530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9c0051c0 0x7fdb9c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbaeffd640 1 -- 172.21.15.16:0/342729530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c003520 con 0x7fdb9c0039b0 2024-04-25T10:11:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.139+0000 7fdbaf7fe640 1 -- 172.21.15.16:0/342729530 reap_dead start 2024-04-25T10:11:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.140+0000 7fdbaeffd640 1 --2- 172.21.15.16:0/342729530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9c0039b0 0x7fdb9c003dd0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fdba800c210 tx=0x7fdb9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b63e64529cd618 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.140+0000 7fdbadffb640 1 -- 172.21.15.16:0/342729530 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8007d30 con 0x7fdb9c0039b0 2024-04-25T10:11:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.140+0000 7fdbadffb640 1 -- 172.21.15.16:0/342729530 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba8005ad0 con 0x7fdb9c0039b0 2024-04-25T10:11:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.140+0000 7fdbadffb640 1 -- 172.21.15.16:0/342729530 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8007070 con 0x7fdb9c0039b0 2024-04-25T10:11:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.141+0000 7fdbb5942640 1 -- 172.21.15.16:0/342729530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9c0039b0 msgr2=0x7fdb9c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.141+0000 7fdbb5942640 1 --2- 172.21.15.16:0/342729530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9c0039b0 0x7fdb9c003dd0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fdba800c210 tx=0x7fdb9c005830 comp rx=0 tx=0).stop 2024-04-25T10:11:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.141+0000 7fdbb5942640 1 -- 172.21.15.16:0/342729530 shutdown_connections 2024-04-25T10:11:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.141+0000 7fdbb5942640 1 --2- 172.21.15.16:0/342729530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9c0039b0 0x7fdb9c003dd0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.141+0000 7fdbb5942640 1 -- 172.21.15.16:0/342729530 >> 172.21.15.16:0/342729530 conn(0x7fdbb00b32f0 msgr2=0x7fdbb00b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.141+0000 7fdbb5942640 1 -- 172.21.15.16:0/342729530 shutdown_connections 2024-04-25T10:11:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.141+0000 7fdbb5942640 1 -- 172.21.15.16:0/342729530 wait complete. 2024-04-25T10:11:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.142+0000 7fdbb5942640 1 Processor -- start 2024-04-25T10:11:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.142+0000 7fdbb5942640 1 -- start start 2024-04-25T10:11:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.143+0000 7fdbb5942640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9c0039b0 0x7fdbb00b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.143+0000 7fdbb5942640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb0153f50 0x7fdbb01f7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.143+0000 7fdbb5942640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbb00c76a0 0x7fdbb00c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.143+0000 7fdbb5942640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb006bcb0 con 0x7fdbb00c76a0 2024-04-25T10:11:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.143+0000 7fdbb5942640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb00cfe80 con 0x7fdbb0153f50 2024-04-25T10:11:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.143+0000 7fdbb5942640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb006be30 con 0x7fdb9c0039b0 2024-04-25T10:11:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.143+0000 7fdbaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbb00c76a0 0x7fdbb00c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.143+0000 7fdbaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbb00c76a0 0x7fdbb00c7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53178/0 (socket says 172.21.15.16:53178) 2024-04-25T10:11:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.143+0000 7fdbaf7fe640 1 -- 172.21.15.16:0/398995986 learned_addr learned my addr 172.21.15.16:0/398995986 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.143+0000 7fdbaeffd640 1 --2- 172.21.15.16:0/398995986 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9c0039b0 0x7fdbb00b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.143+0000 7fdbae7fc640 1 --2- 172.21.15.16:0/398995986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb0153f50 0x7fdbb01f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.143+0000 7fdbaf7fe640 1 -- 172.21.15.16:0/398995986 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9c0039b0 msgr2=0x7fdbb00b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.143+0000 7fdbaf7fe640 1 --2- 172.21.15.16:0/398995986 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9c0039b0 0x7fdbb00b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.143+0000 7fdbaf7fe640 1 -- 172.21.15.16:0/398995986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb0153f50 msgr2=0x7fdbb01f7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.143+0000 7fdbaf7fe640 1 --2- 172.21.15.16:0/398995986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb0153f50 0x7fdbb01f7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.143+0000 7fdbaf7fe640 1 -- 172.21.15.16:0/398995986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb0157960 con 0x7fdbb00c76a0 2024-04-25T10:11:03.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.144+0000 7fdbaf7fe640 1 --2- 172.21.15.16:0/398995986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbb00c76a0 0x7fdbb00c7b50 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fdba0007fd0 tx=0x7fdba000e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.144+0000 7fdba7fff640 1 -- 172.21.15.16:0/398995986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0019070 con 0x7fdbb00c76a0 2024-04-25T10:11:03.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.144+0000 7fdba7fff640 1 -- 172.21.15.16:0/398995986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba0006e20 con 0x7fdbb00c76a0 2024-04-25T10:11:03.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.144+0000 7fdba7fff640 1 -- 172.21.15.16:0/398995986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0003bb0 con 0x7fdbb00c76a0 2024-04-25T10:11:03.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.144+0000 7fdbb5942640 1 -- 172.21.15.16:0/398995986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb0154a10 con 0x7fdbb00c76a0 2024-04-25T10:11:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.144+0000 7fdbb5942640 1 -- 172.21.15.16:0/398995986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb00c8090 con 0x7fdbb00c76a0 2024-04-25T10:11:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.145+0000 7fdbb5942640 1 -- 172.21.15.16:0/398995986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb00749b0 con 0x7fdbb00c76a0 2024-04-25T10:11:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.145+0000 7fdba7fff640 1 -- 172.21.15.16:0/398995986 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdba001c070 con 0x7fdbb00c76a0 2024-04-25T10:11:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.146+0000 7fdba7fff640 1 --2- 172.21.15.16:0/398995986 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb9006b4d0 0x7fdb9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.146+0000 7fdba7fff640 1 -- 172.21.15.16:0/398995986 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdba0089750 con 0x7fdbb00c76a0 2024-04-25T10:11:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.146+0000 7fdbaeffd640 1 --2- 172.21.15.16:0/398995986 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb9006b4d0 0x7fdb9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.147+0000 7fdbaeffd640 1 --2- 172.21.15.16:0/398995986 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb9006b4d0 0x7fdb9006d990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdba8033000 tx=0x7fdba802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.150+0000 7fdba7fff640 1 -- 172.21.15.16:0/398995986 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdba00585c0 con 0x7fdbb00c76a0 2024-04-25T10:11:03.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.330+0000 7fdbb5942640 1 -- 172.21.15.16:0/398995986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbb009a570 con 0x7fdbb00c76a0 2024-04-25T10:11:03.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.331+0000 7fdba7fff640 1 -- 172.21.15.16:0/398995986 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16672 (secure 0 0 0) 0x7fdba0057ec0 con 0x7fdbb00c76a0 2024-04-25T10:11:03.337 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:03.337 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:11:01.657760+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:58.299405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:58.994953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:03.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.334+0000 7fdbb5942640 1 -- 172.21.15.16:0/398995986 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb9006b4d0 msgr2=0x7fdb9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.334+0000 7fdbb5942640 1 --2- 172.21.15.16:0/398995986 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb9006b4d0 0x7fdb9006d990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdba8033000 tx=0x7fdba802e000 comp rx=0 tx=0).stop 2024-04-25T10:11:03.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.334+0000 7fdbb5942640 1 -- 172.21.15.16:0/398995986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbb00c76a0 msgr2=0x7fdbb00c7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.334+0000 7fdbb5942640 1 --2- 172.21.15.16:0/398995986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbb00c76a0 0x7fdbb00c7b50 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fdba0007fd0 tx=0x7fdba000e940 comp rx=0 tx=0).stop 2024-04-25T10:11:03.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.334+0000 7fdbb5942640 1 -- 172.21.15.16:0/398995986 shutdown_connections 2024-04-25T10:11:03.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.334+0000 7fdbb5942640 1 --2- 172.21.15.16:0/398995986 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb9006b4d0 0x7fdb9006d990 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.334+0000 7fdbb5942640 1 --2- 172.21.15.16:0/398995986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbb00c76a0 0x7fdbb00c7b50 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.334+0000 7fdbb5942640 1 --2- 172.21.15.16:0/398995986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb0153f50 0x7fdbb01f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.334+0000 7fdbb5942640 1 --2- 172.21.15.16:0/398995986 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9c0039b0 0x7fdbb00b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.334+0000 7fdbb5942640 1 -- 172.21.15.16:0/398995986 >> 172.21.15.16:0/398995986 conn(0x7fdbb00b32f0 msgr2=0x7fdbb0099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:03.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.334+0000 7fdbb5942640 1 -- 172.21.15.16:0/398995986 shutdown_connections 2024-04-25T10:11:03.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.334+0000 7fdbb5942640 1 -- 172.21.15.16:0/398995986 wait complete. 2024-04-25T10:11:03.351 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:11:03.351 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.447+0000 7fc76541f640 1 Processor -- start 2024-04-25T10:11:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.447+0000 7fc76541f640 1 -- start start 2024-04-25T10:11:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc76541f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7600b5b60 0x7fc7601687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc76541f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc76015e0c0 con 0x7fc76015dce0 2024-04-25T10:11:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc76541f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7600d2510 con 0x7fc7600d35b0 2024-04-25T10:11:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc76541f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7600b7b10 con 0x7fc7600b5b60 2024-04-25T10:11:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc75effd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc76015dce0 0x7fc7600d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55482/0 (socket says 172.21.15.16:55482) 2024-04-25T10:11:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc75effd640 1 -- 172.21.15.16:0/1395712355 learned_addr learned my addr 172.21.15.16:0/1395712355 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc75dffb640 1 -- 172.21.15.16:0/1395712355 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809296266 0 0) 0x7fc76015e0c0 con 0x7fc76015dce0 2024-04-25T10:11:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc75dffb640 1 -- 172.21.15.16:0/1395712355 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc74c003520 con 0x7fc76015dce0 2024-04-25T10:11:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc75f7fe640 1 --2- 172.21.15.16:0/1395712355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7600b5b60 0x7fc7601687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc75dffb640 1 -- 172.21.15.16:0/1395712355 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1783311623 0 0) 0x7fc74c003520 con 0x7fc76015dce0 2024-04-25T10:11:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc75dffb640 1 -- 172.21.15.16:0/1395712355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7600b5b60 msgr2=0x7fc7601687f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc75dffb640 1 --2- 172.21.15.16:0/1395712355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7600b5b60 0x7fc7601687f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc75dffb640 1 -- 172.21.15.16:0/1395712355 >> v1:172.21.15.186:6789/0 conn(0x7fc7600d35b0 legacy=0x7fc7600b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc75dffb640 1 -- 172.21.15.16:0/1395712355 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7600d27f0 con 0x7fc76015dce0 2024-04-25T10:11:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc75dffb640 1 -- 172.21.15.16:0/1395712355 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fc758001670 con 0x7fc76015dce0 2024-04-25T10:11:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc75dffb640 1 -- 172.21.15.16:0/1395712355 >> v1:172.21.15.16:6789/0 conn(0x7fc76015dce0 legacy=0x7fc7600d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc75dffb640 1 --2- 172.21.15.16:0/1395712355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc74c0039e0 0x7fc74c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc75dffb640 1 --2- 172.21.15.16:0/1395712355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc74c004800 0x7fc74c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.448+0000 7fc75dffb640 1 --2- 172.21.15.16:0/1395712355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7600b5b60 0x7fc74c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.449+0000 7fc75dffb640 1 -- 172.21.15.16:0/1395712355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7600b7b10 con 0x7fc74c0039e0 2024-04-25T10:11:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.449+0000 7fc75dffb640 1 -- 172.21.15.16:0/1395712355 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c00ba70 con 0x7fc74c004800 2024-04-25T10:11:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.449+0000 7fc75dffb640 1 -- 172.21.15.16:0/1395712355 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c00bc70 con 0x7fc7600b5b60 2024-04-25T10:11:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.449+0000 7fc75effd640 1 --2- 172.21.15.16:0/1395712355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc74c0039e0 0x7fc74c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.449+0000 7fc75e7fc640 1 --2- 172.21.15.16:0/1395712355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc74c004800 0x7fc74c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.449+0000 7fc75f7fe640 1 --2- 172.21.15.16:0/1395712355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7600b5b60 0x7fc74c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.449+0000 7fc75effd640 1 -- 172.21.15.16:0/1395712355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7600b5b60 msgr2=0x7fc74c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.449+0000 7fc75effd640 1 --2- 172.21.15.16:0/1395712355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7600b5b60 0x7fc74c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.449+0000 7fc75effd640 1 -- 172.21.15.16:0/1395712355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc74c004800 msgr2=0x7fc74c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.449+0000 7fc75effd640 1 --2- 172.21.15.16:0/1395712355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc74c004800 0x7fc74c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.449+0000 7fc75effd640 1 -- 172.21.15.16:0/1395712355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74c003520 con 0x7fc74c0039e0 2024-04-25T10:11:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.449+0000 7fc75f7fe640 1 --2- 172.21.15.16:0/1395712355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7600b5b60 0x7fc74c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.450+0000 7fc75effd640 1 --2- 172.21.15.16:0/1395712355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc74c0039e0 0x7fc74c003e00 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fc75800c210 tx=0x7fc74c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ec4ea8ee91fd99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.450+0000 7fc75dffb640 1 -- 172.21.15.16:0/1395712355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc758007d10 con 0x7fc74c0039e0 2024-04-25T10:11:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.450+0000 7fc75dffb640 1 -- 172.21.15.16:0/1395712355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc758005ad0 con 0x7fc74c0039e0 2024-04-25T10:11:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.450+0000 7fc75dffb640 1 -- 172.21.15.16:0/1395712355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc758007070 con 0x7fc74c0039e0 2024-04-25T10:11:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.450+0000 7fc76541f640 1 -- 172.21.15.16:0/1395712355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc74c0039e0 msgr2=0x7fc74c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.450+0000 7fc76541f640 1 --2- 172.21.15.16:0/1395712355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc74c0039e0 0x7fc74c003e00 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fc75800c210 tx=0x7fc74c005830 comp rx=0 tx=0).stop 2024-04-25T10:11:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.450+0000 7fc75f7fe640 1 -- 172.21.15.16:0/1395712355 reap_dead start 2024-04-25T10:11:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.450+0000 7fc76541f640 1 -- 172.21.15.16:0/1395712355 shutdown_connections 2024-04-25T10:11:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.450+0000 7fc76541f640 1 -- 172.21.15.16:0/1395712355 >> 172.21.15.16:0/1395712355 conn(0x7fc760072900 msgr2=0x7fc7600d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.450+0000 7fc76541f640 1 -- 172.21.15.16:0/1395712355 shutdown_connections 2024-04-25T10:11:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.450+0000 7fc76541f640 1 -- 172.21.15.16:0/1395712355 wait complete. 2024-04-25T10:11:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.452+0000 7fc76541f640 1 Processor -- start 2024-04-25T10:11:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.452+0000 7fc76541f640 1 -- start start 2024-04-25T10:11:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.452+0000 7fc76541f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc74c0039e0 0x7fc7601583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.452+0000 7fc76541f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7600b1cc0 0x7fc76015fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.452+0000 7fc76541f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc760157030 0x7fc7600ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.452+0000 7fc76541f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7600d3090 con 0x7fc7600b1cc0 2024-04-25T10:11:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.452+0000 7fc76541f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7600d2db0 con 0x7fc74c0039e0 2024-04-25T10:11:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.452+0000 7fc76541f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7600d2160 con 0x7fc760157030 2024-04-25T10:11:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.453+0000 7fc75e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7600b1cc0 0x7fc76015fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.453+0000 7fc75e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7600b1cc0 0x7fc76015fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53196/0 (socket says 172.21.15.16:53196) 2024-04-25T10:11:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.453+0000 7fc75e7fc640 1 -- 172.21.15.16:0/2438060476 learned_addr learned my addr 172.21.15.16:0/2438060476 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.453+0000 7fc75f7fe640 1 --2- 172.21.15.16:0/2438060476 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc760157030 0x7fc7600ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.453+0000 7fc75effd640 1 --2- 172.21.15.16:0/2438060476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc74c0039e0 0x7fc7601583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.453+0000 7fc75e7fc640 1 -- 172.21.15.16:0/2438060476 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc760157030 msgr2=0x7fc7600ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.453+0000 7fc75e7fc640 1 --2- 172.21.15.16:0/2438060476 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc760157030 0x7fc7600ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.453+0000 7fc75e7fc640 1 -- 172.21.15.16:0/2438060476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc74c0039e0 msgr2=0x7fc7601583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.453+0000 7fc75e7fc640 1 --2- 172.21.15.16:0/2438060476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc74c0039e0 0x7fc7601583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.453+0000 7fc75e7fc640 1 -- 172.21.15.16:0/2438060476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc760158910 con 0x7fc7600b1cc0 2024-04-25T10:11:03.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.453+0000 7fc75e7fc640 1 --2- 172.21.15.16:0/2438060476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7600b1cc0 0x7fc76015fb90 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fc74800f5c0 tx=0x7fc74800fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.453+0000 7fc7577fe640 1 -- 172.21.15.16:0/2438060476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7480052e0 con 0x7fc7600b1cc0 2024-04-25T10:11:03.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.453+0000 7fc7577fe640 1 -- 172.21.15.16:0/2438060476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc748018070 con 0x7fc7600b1cc0 2024-04-25T10:11:03.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.453+0000 7fc76541f640 1 -- 172.21.15.16:0/2438060476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7601603c0 con 0x7fc7600b1cc0 2024-04-25T10:11:03.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.453+0000 7fc7577fe640 1 -- 172.21.15.16:0/2438060476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748003810 con 0x7fc7600b1cc0 2024-04-25T10:11:03.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.453+0000 7fc76541f640 1 -- 172.21.15.16:0/2438060476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc760157bd0 con 0x7fc7600b1cc0 2024-04-25T10:11:03.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.454+0000 7fc76541f640 1 -- 172.21.15.16:0/2438060476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7600749b0 con 0x7fc7600b1cc0 2024-04-25T10:11:03.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.455+0000 7fc7577fe640 1 -- 172.21.15.16:0/2438060476 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7480039b0 con 0x7fc7600b1cc0 2024-04-25T10:11:03.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.455+0000 7fc7577fe640 1 --2- 172.21.15.16:0/2438060476 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc74006b500 0x7fc74006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.455+0000 7fc7577fe640 1 -- 172.21.15.16:0/2438060476 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc748089ee0 con 0x7fc7600b1cc0 2024-04-25T10:11:03.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.455+0000 7fc75effd640 1 --2- 172.21.15.16:0/2438060476 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc74006b500 0x7fc74006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.456+0000 7fc75effd640 1 --2- 172.21.15.16:0/2438060476 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc74006b500 0x7fc74006d9c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc758032000 tx=0x7fc75802d000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.459+0000 7fc7577fe640 1 -- 172.21.15.16:0/2438060476 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc748058df0 con 0x7fc7600b1cc0 2024-04-25T10:11:03.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.638+0000 7fc76541f640 1 -- 172.21.15.16:0/2438060476 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7601600d0 con 0x7fc74006b500 2024-04-25T10:11:03.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.644+0000 7fc7577fe640 1 -- 172.21.15.16:0/2438060476 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+238130 (secure 0 0 0) 0x7fc7601600d0 con 0x7fc74006b500 2024-04-25T10:11:03.653 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:03.654 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:11:03.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.647+0000 7fc76541f640 1 -- 172.21.15.16:0/2438060476 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc74006b500 msgr2=0x7fc74006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.647+0000 7fc76541f640 1 --2- 172.21.15.16:0/2438060476 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc74006b500 0x7fc74006d9c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc758032000 tx=0x7fc75802d000 comp rx=0 tx=0).stop 2024-04-25T10:11:03.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.647+0000 7fc76541f640 1 -- 172.21.15.16:0/2438060476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7600b1cc0 msgr2=0x7fc76015fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.647+0000 7fc76541f640 1 --2- 172.21.15.16:0/2438060476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7600b1cc0 0x7fc76015fb90 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fc74800f5c0 tx=0x7fc74800fa80 comp rx=0 tx=0).stop 2024-04-25T10:11:03.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.647+0000 7fc76541f640 1 -- 172.21.15.16:0/2438060476 shutdown_connections 2024-04-25T10:11:03.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.647+0000 7fc76541f640 1 --2- 172.21.15.16:0/2438060476 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc74006b500 0x7fc74006d9c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.647+0000 7fc76541f640 1 --2- 172.21.15.16:0/2438060476 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc760157030 0x7fc7600ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.647+0000 7fc76541f640 1 --2- 172.21.15.16:0/2438060476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7600b1cc0 0x7fc76015fb90 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.647+0000 7fc76541f640 1 --2- 172.21.15.16:0/2438060476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc74c0039e0 0x7fc7601583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.647+0000 7fc76541f640 1 -- 172.21.15.16:0/2438060476 >> 172.21.15.16:0/2438060476 conn(0x7fc760072900 msgr2=0x7fc7600ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:03.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.647+0000 7fc76541f640 1 -- 172.21.15.16:0/2438060476 shutdown_connections 2024-04-25T10:11:03.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.647+0000 7fc76541f640 1 -- 172.21.15.16:0/2438060476 wait complete. 2024-04-25T10:11:03.663 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2024-04-25T10:11:01.866605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":250132,"kb_used_data":8968,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754724588,"statfs":{"total":773094113280,"available":772837978112,"internally_reserved":0,"allocated":9183232,"data_stored":4550480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000941"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.648641+0000","last_change":"2024-04-25T10:10:59.648641+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.648641+0000","last_undegraded":"2024-04-25T10:10:59.648641+0000","last_fullsized":"2024-04-25T10:10:59.648641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.651673+0000","last_change":"2024-04-25T10:10:59.651673+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.651673+0000","last_undegraded":"2024-04-25T10:10:59.651673+0000","last_fullsized":"2024-04-25T10:10:59.651673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.651157+0000","last_change":"2024-04-25T10:10:59.651157+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.651157+0000","last_undegraded":"2024-04-25T10:10:59.651157+0000","last_fullsized":"2024-04-25T10:10:59.651157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T10:10:59.664299+0000","last_change":"2024-04-25T10:10:59.658687+0000","last_active":"2024-04-25T10:10:59.664299+0000","last_peered":"2024-04-25T10:10:59.664299+0000","last_clean":"2024-04-25T10:10:59.664299+0000","last_became_active":"2024-04-25T10:10:59.658380+0000","last_became_peered":"2024-04-25T10:10:59.658380+0000","last_unstale":"2024-04-25T10:10:59.664299+0000","last_undegraded":"2024-04-25T10:10:59.664299+0000","last_fullsized":"2024-04-25T10:10:59.664299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:14.321738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.652342+0000","last_change":"2024-04-25T10:10:59.652342+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.652342+0000","last_undegraded":"2024-04-25T10:10:59.652342+0000","last_fullsized":"2024-04-25T10:10:59.652342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T10:10:59.666667+0000","last_change":"2024-04-25T10:10:59.662230+0000","last_active":"2024-04-25T10:10:59.666667+0000","last_peered":"2024-04-25T10:10:59.666667+0000","last_clean":"2024-04-25T10:10:59.666667+0000","last_became_active":"2024-04-25T10:10:59.661953+0000","last_became_peered":"2024-04-25T10:10:59.661953+0000","last_unstale":"2024-04-25T10:10:59.666667+0000","last_undegraded":"2024-04-25T10:10:59.666667+0000","last_fullsized":"2024-04-25T10:10:59.666667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:31.766942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.653171+0000","last_change":"2024-04-25T10:10:59.653171+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.653171+0000","last_undegraded":"2024-04-25T10:10:59.653171+0000","last_fullsized":"2024-04-25T10:10:59.653171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:59.639238+0000","last_change":"2024-04-25T10:10:59.639238+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.639238+0000","last_undegraded":"2024-04-25T10:10:59.639238+0000","last_fullsized":"2024-04-25T10:10:59.639238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:10:58.634346+0000","last_change":"2024-04-25T10:10:58.634346+0000","last_active":"2024-04-25T10:10:58.634346+0000","last_peered":"2024-04-25T10:10:58.634346+0000","last_clean":"2024-04-25T10:10:58.634346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:58.634346+0000","last_undegraded":"2024-04-25T10:10:58.634346+0000","last_fullsized":"2024-04-25T10:10:58.634346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":292,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:10:56.645329+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:10:56.645329+0000","last_peered":"2024-04-25T10:10:56.645329+0000","last_clean":"2024-04-25T10:10:56.645329+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:10:56.645329+0000","last_undegraded":"2024-04-25T10:10:56.645329+0000","last_fullsized":"2024-04-25T10:10:56.645329+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":4,"acting":4,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509515,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":987136,"data_stored":416066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":421123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509516,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":999424,"data_stored":421123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":421123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":999424,"data_stored":421123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509515,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1593344,"data_stored":1011491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":421123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1605632,"data_stored":1017308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:03.668 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:03.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.799+0000 7f90ef1c8640 1 Processor -- start 2024-04-25T10:11:03.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.799+0000 7f90ef1c8640 1 -- start start 2024-04-25T10:11:03.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.800+0000 7f90ef1c8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90e815dce0 0x7f90e80d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.800+0000 7f90ef1c8640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f90e80b5b60 0x7f90e80b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.800+0000 7f90ef1c8640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e815e0c0 con 0x7f90e80d35b0 2024-04-25T10:11:03.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.800+0000 7f90ef1c8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e80d2510 con 0x7f90e815dce0 2024-04-25T10:11:03.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.800+0000 7f90ef1c8640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e80d27f0 con 0x7f90e80b5b60 2024-04-25T10:11:03.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.800+0000 7f90ecf64640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90e815dce0 0x7f90e80d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.800+0000 7f90ecf64640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90e815dce0 0x7f90e80d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53206/0 (socket says 172.21.15.16:53206) 2024-04-25T10:11:03.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.800+0000 7f90ecf64640 1 -- 172.21.15.16:0/2258473935 learned_addr learned my addr 172.21.15.16:0/2258473935 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.800+0000 7f90ed765640 1 --2- 172.21.15.16:0/2258473935 >> v2:172.21.15.186:3300/0 conn(0x7f90e80b5b60 0x7f90e80b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.800+0000 7f90ecf64640 1 -- 172.21.15.16:0/2258473935 >> v2:172.21.15.186:3300/0 conn(0x7f90e80b5b60 msgr2=0x7f90e80b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.800+0000 7f90ecf64640 1 --2- 172.21.15.16:0/2258473935 >> v2:172.21.15.186:3300/0 conn(0x7f90e80b5b60 0x7f90e80b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.800+0000 7f90ecf64640 1 -- 172.21.15.16:0/2258473935 >> v1:172.21.15.186:6789/0 conn(0x7f90e80d35b0 legacy=0x7f90e80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.800+0000 7f90ecf64640 1 -- 172.21.15.16:0/2258473935 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e80d2ad0 con 0x7f90e815dce0 2024-04-25T10:11:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.801+0000 7f90ecf64640 1 --2- 172.21.15.16:0/2258473935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90e815dce0 0x7f90e80d1a50 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f90e80b3390 tx=0x7f90d802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4dc2c16d25adf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.801+0000 7f90e77fe640 1 -- 172.21.15.16:0/2258473935 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d8037070 con 0x7f90e815dce0 2024-04-25T10:11:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.801+0000 7f90e77fe640 1 -- 172.21.15.16:0/2258473935 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d8032440 con 0x7f90e815dce0 2024-04-25T10:11:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.801+0000 7f90e77fe640 1 -- 172.21.15.16:0/2258473935 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d8032810 con 0x7f90e815dce0 2024-04-25T10:11:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.802+0000 7f90ef1c8640 1 -- 172.21.15.16:0/2258473935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90e815dce0 msgr2=0x7f90e80d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.802+0000 7f90ef1c8640 1 --2- 172.21.15.16:0/2258473935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90e815dce0 0x7f90e80d1a50 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f90e80b3390 tx=0x7f90d802a230 comp rx=0 tx=0).stop 2024-04-25T10:11:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.802+0000 7f90ef1c8640 1 -- 172.21.15.16:0/2258473935 shutdown_connections 2024-04-25T10:11:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.802+0000 7f90ef1c8640 1 --2- 172.21.15.16:0/2258473935 >> v2:172.21.15.186:3300/0 conn(0x7f90e80b5b60 0x7f90e80b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.802+0000 7f90ef1c8640 1 --2- 172.21.15.16:0/2258473935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90e815dce0 0x7f90e80d1a50 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.802+0000 7f90ef1c8640 1 -- 172.21.15.16:0/2258473935 >> 172.21.15.16:0/2258473935 conn(0x7f90e8072900 msgr2=0x7f90e80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.802+0000 7f90ef1c8640 1 -- 172.21.15.16:0/2258473935 shutdown_connections 2024-04-25T10:11:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.802+0000 7f90ef1c8640 1 -- 172.21.15.16:0/2258473935 wait complete. 2024-04-25T10:11:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.803+0000 7f90ef1c8640 1 Processor -- start 2024-04-25T10:11:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.803+0000 7f90ef1c8640 1 -- start start 2024-04-25T10:11:03.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.804+0000 7f90ef1c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90e80b5b60 0x7f90e815e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.804+0000 7f90ef1c8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90e80d35b0 0x7f90e814ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.804+0000 7f90ef1c8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90e815dce0 0x7f90e814b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.804+0000 7f90ef1c8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e80b9820 con 0x7f90e80b5b60 2024-04-25T10:11:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.804+0000 7f90ef1c8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e80d3090 con 0x7f90e80d35b0 2024-04-25T10:11:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.804+0000 7f90ef1c8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e80b99a0 con 0x7f90e815dce0 2024-04-25T10:11:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.804+0000 7f90ecf64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90e80b5b60 0x7f90e815e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.804+0000 7f90ecf64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90e80b5b60 0x7f90e815e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53210/0 (socket says 172.21.15.16:53210) 2024-04-25T10:11:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.804+0000 7f90ecf64640 1 -- 172.21.15.16:0/1650784236 learned_addr learned my addr 172.21.15.16:0/1650784236 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.804+0000 7f90ed765640 1 --2- 172.21.15.16:0/1650784236 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90e815dce0 0x7f90e814b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.804+0000 7f90e7fff640 1 --2- 172.21.15.16:0/1650784236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90e80d35b0 0x7f90e814ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.804+0000 7f90ecf64640 1 -- 172.21.15.16:0/1650784236 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90e815dce0 msgr2=0x7f90e814b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.804+0000 7f90ecf64640 1 --2- 172.21.15.16:0/1650784236 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90e815dce0 0x7f90e814b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.804+0000 7f90ecf64640 1 -- 172.21.15.16:0/1650784236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90e80d35b0 msgr2=0x7f90e814ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.804+0000 7f90ecf64640 1 --2- 172.21.15.16:0/1650784236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90e80d35b0 0x7f90e814ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.804+0000 7f90ecf64640 1 -- 172.21.15.16:0/1650784236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e814b320 con 0x7f90e80b5b60 2024-04-25T10:11:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.805+0000 7f90ecf64640 1 --2- 172.21.15.16:0/1650784236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90e80b5b60 0x7f90e815e2b0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f90d8000c00 tx=0x7f90d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.805+0000 7f90e57fa640 1 -- 172.21.15.16:0/1650784236 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d8041020 con 0x7f90e80b5b60 2024-04-25T10:11:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.805+0000 7f90e57fa640 1 -- 172.21.15.16:0/1650784236 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d80306c0 con 0x7f90e80b5b60 2024-04-25T10:11:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.805+0000 7f90ef1c8640 1 -- 172.21.15.16:0/1650784236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e80c8930 con 0x7f90e80b5b60 2024-04-25T10:11:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.805+0000 7f90e57fa640 1 -- 172.21.15.16:0/1650784236 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d8037040 con 0x7f90e80b5b60 2024-04-25T10:11:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.805+0000 7f90ef1c8640 1 -- 172.21.15.16:0/1650784236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e80c8ef0 con 0x7f90e80b5b60 2024-04-25T10:11:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.806+0000 7f90ef1c8640 1 -- 172.21.15.16:0/1650784236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e80749b0 con 0x7f90e80b5b60 2024-04-25T10:11:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.806+0000 7f90e57fa640 1 -- 172.21.15.16:0/1650784236 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90d8044020 con 0x7f90e80b5b60 2024-04-25T10:11:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.807+0000 7f90e57fa640 1 --2- 172.21.15.16:0/1650784236 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f90d406b4d0 0x7f90d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.807+0000 7f90e57fa640 1 -- 172.21.15.16:0/1650784236 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90d80aaa30 con 0x7f90e80b5b60 2024-04-25T10:11:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.807+0000 7f90e7fff640 1 --2- 172.21.15.16:0/1650784236 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f90d406b4d0 0x7f90d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.807+0000 7f90e7fff640 1 --2- 172.21.15.16:0/1650784236 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f90d406b4d0 0x7f90d406d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f90e8157760 tx=0x7f90dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.811+0000 7f90e57fa640 1 -- 172.21.15.16:0/1650784236 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90d804b030 con 0x7f90e80b5b60 2024-04-25T10:11:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.988+0000 7f90ef1c8640 1 -- 172.21.15.16:0/1650784236 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90e80c86e0 con 0x7f90d406b4d0 2024-04-25T10:11:03.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.994+0000 7f90e57fa640 1 -- 172.21.15.16:0/1650784236 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251662 (secure 0 0 0) 0x7f90e80c86e0 con 0x7f90d406b4d0 2024-04-25T10:11:04.002 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:11:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.997+0000 7f90ef1c8640 1 -- 172.21.15.16:0/1650784236 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f90d406b4d0 msgr2=0x7f90d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.997+0000 7f90ef1c8640 1 --2- 172.21.15.16:0/1650784236 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f90d406b4d0 0x7f90d406d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f90e8157760 tx=0x7f90dc009040 comp rx=0 tx=0).stop 2024-04-25T10:11:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.997+0000 7f90ef1c8640 1 -- 172.21.15.16:0/1650784236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90e80b5b60 msgr2=0x7f90e815e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.997+0000 7f90ef1c8640 1 --2- 172.21.15.16:0/1650784236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90e80b5b60 0x7f90e815e2b0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f90d8000c00 tx=0x7f90d80046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.997+0000 7f90ef1c8640 1 -- 172.21.15.16:0/1650784236 shutdown_connections 2024-04-25T10:11:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.997+0000 7f90ef1c8640 1 --2- 172.21.15.16:0/1650784236 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f90d406b4d0 0x7f90d406d990 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.997+0000 7f90ef1c8640 1 --2- 172.21.15.16:0/1650784236 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90e815dce0 0x7f90e814b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.997+0000 7f90ef1c8640 1 --2- 172.21.15.16:0/1650784236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90e80d35b0 0x7f90e814ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.997+0000 7f90ef1c8640 1 --2- 172.21.15.16:0/1650784236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90e80b5b60 0x7f90e815e2b0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.997+0000 7f90ef1c8640 1 -- 172.21.15.16:0/1650784236 >> 172.21.15.16:0/1650784236 conn(0x7f90e8072900 msgr2=0x7f90e80b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.997+0000 7f90ef1c8640 1 -- 172.21.15.16:0/1650784236 shutdown_connections 2024-04-25T10:11:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:03.997+0000 7f90ef1c8640 1 -- 172.21.15.16:0/1650784236 wait complete. 2024-04-25T10:11:04.013 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-04-25T10:11:03.867060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":198,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":267160,"kb_used_data":9364,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754707560,"statfs":{"total":773094113280,"available":772820541440,"internally_reserved":0,"allocated":9588736,"data_stored":4734880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.226999"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669584+0000","last_change":"2024-04-25T10:11:00.665795+0000","last_active":"2024-04-25T10:11:01.669584+0000","last_peered":"2024-04-25T10:11:01.669584+0000","last_clean":"2024-04-25T10:11:01.669584+0000","last_became_active":"2024-04-25T10:11:00.664672+0000","last_became_peered":"2024-04-25T10:11:00.664672+0000","last_unstale":"2024-04-25T10:11:01.669584+0000","last_undegraded":"2024-04-25T10:11:01.669584+0000","last_fullsized":"2024-04-25T10:11:01.669584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:30.244884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672240+0000","last_change":"2024-04-25T10:11:00.658574+0000","last_active":"2024-04-25T10:11:01.672240+0000","last_peered":"2024-04-25T10:11:01.672240+0000","last_clean":"2024-04-25T10:11:01.672240+0000","last_became_active":"2024-04-25T10:11:00.658280+0000","last_became_peered":"2024-04-25T10:11:00.658280+0000","last_unstale":"2024-04-25T10:11:01.672240+0000","last_undegraded":"2024-04-25T10:11:01.672240+0000","last_fullsized":"2024-04-25T10:11:01.672240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:38.681757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666472+0000","last_change":"2024-04-25T10:11:00.664492+0000","last_active":"2024-04-25T10:11:01.666472+0000","last_peered":"2024-04-25T10:11:01.666472+0000","last_clean":"2024-04-25T10:11:01.666472+0000","last_became_active":"2024-04-25T10:11:00.664205+0000","last_became_peered":"2024-04-25T10:11:00.664205+0000","last_unstale":"2024-04-25T10:11:01.666472+0000","last_undegraded":"2024-04-25T10:11:01.666472+0000","last_fullsized":"2024-04-25T10:11:01.666472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.860030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.648641+0000","last_change":"2024-04-25T10:10:59.648641+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.648641+0000","last_undegraded":"2024-04-25T10:10:59.648641+0000","last_fullsized":"2024-04-25T10:10:59.648641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791419+0000","last_change":"2024-04-25T10:11:00.674341+0000","last_active":"2024-04-25T10:11:01.791419+0000","last_peered":"2024-04-25T10:11:01.791419+0000","last_clean":"2024-04-25T10:11:01.791419+0000","last_became_active":"2024-04-25T10:11:00.674016+0000","last_became_peered":"2024-04-25T10:11:00.674016+0000","last_unstale":"2024-04-25T10:11:01.791419+0000","last_undegraded":"2024-04-25T10:11:01.791419+0000","last_fullsized":"2024-04-25T10:11:01.791419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:17.318511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666458+0000","last_change":"2024-04-25T10:11:00.666821+0000","last_active":"2024-04-25T10:11:01.666458+0000","last_peered":"2024-04-25T10:11:01.666458+0000","last_clean":"2024-04-25T10:11:01.666458+0000","last_became_active":"2024-04-25T10:11:00.666552+0000","last_became_peered":"2024-04-25T10:11:00.666552+0000","last_unstale":"2024-04-25T10:11:01.666458+0000","last_undegraded":"2024-04-25T10:11:01.666458+0000","last_fullsized":"2024-04-25T10:11:01.666458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:43.541677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.671974+0000","last_change":"2024-04-25T10:11:00.666060+0000","last_active":"2024-04-25T10:11:01.671974+0000","last_peered":"2024-04-25T10:11:01.671974+0000","last_clean":"2024-04-25T10:11:01.671974+0000","last_became_active":"2024-04-25T10:11:00.665760+0000","last_became_peered":"2024-04-25T10:11:00.665760+0000","last_unstale":"2024-04-25T10:11:01.671974+0000","last_undegraded":"2024-04-25T10:11:01.671974+0000","last_fullsized":"2024-04-25T10:11:01.671974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:56.991014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790545+0000","last_change":"2024-04-25T10:11:00.672928+0000","last_active":"2024-04-25T10:11:01.790545+0000","last_peered":"2024-04-25T10:11:01.790545+0000","last_clean":"2024-04-25T10:11:01.790545+0000","last_became_active":"2024-04-25T10:11:00.672604+0000","last_became_peered":"2024-04-25T10:11:00.672604+0000","last_unstale":"2024-04-25T10:11:01.790545+0000","last_undegraded":"2024-04-25T10:11:01.790545+0000","last_fullsized":"2024-04-25T10:11:01.790545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:43.541677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669584+0000","last_change":"2024-04-25T10:11:00.665452+0000","last_active":"2024-04-25T10:11:01.669584+0000","last_peered":"2024-04-25T10:11:01.669584+0000","last_clean":"2024-04-25T10:11:01.669584+0000","last_became_active":"2024-04-25T10:11:00.665186+0000","last_became_peered":"2024-04-25T10:11:00.665186+0000","last_unstale":"2024-04-25T10:11:01.669584+0000","last_undegraded":"2024-04-25T10:11:01.669584+0000","last_fullsized":"2024-04-25T10:11:01.669584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:13.745574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666508+0000","last_change":"2024-04-25T10:11:00.657395+0000","last_active":"2024-04-25T10:11:01.666508+0000","last_peered":"2024-04-25T10:11:01.666508+0000","last_clean":"2024-04-25T10:11:01.666508+0000","last_became_active":"2024-04-25T10:11:00.657121+0000","last_became_peered":"2024-04-25T10:11:00.657121+0000","last_unstale":"2024-04-25T10:11:01.666508+0000","last_undegraded":"2024-04-25T10:11:01.666508+0000","last_fullsized":"2024-04-25T10:11:01.666508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:28.012019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669989+0000","last_change":"2024-04-25T10:11:00.674199+0000","last_active":"2024-04-25T10:11:01.669989+0000","last_peered":"2024-04-25T10:11:01.669989+0000","last_clean":"2024-04-25T10:11:01.669989+0000","last_became_active":"2024-04-25T10:11:00.673935+0000","last_became_peered":"2024-04-25T10:11:00.673935+0000","last_unstale":"2024-04-25T10:11:01.669989+0000","last_undegraded":"2024-04-25T10:11:01.669989+0000","last_fullsized":"2024-04-25T10:11:01.669989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:35.284964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790779+0000","last_change":"2024-04-25T10:11:00.675112+0000","last_active":"2024-04-25T10:11:01.790779+0000","last_peered":"2024-04-25T10:11:01.790779+0000","last_clean":"2024-04-25T10:11:01.790779+0000","last_became_active":"2024-04-25T10:11:00.674812+0000","last_became_peered":"2024-04-25T10:11:00.674812+0000","last_unstale":"2024-04-25T10:11:01.790779+0000","last_undegraded":"2024-04-25T10:11:01.790779+0000","last_fullsized":"2024-04-25T10:11:01.790779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:56.966365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791569+0000","last_change":"2024-04-25T10:11:00.660277+0000","last_active":"2024-04-25T10:11:01.791569+0000","last_peered":"2024-04-25T10:11:01.791569+0000","last_clean":"2024-04-25T10:11:01.791569+0000","last_became_active":"2024-04-25T10:11:00.659982+0000","last_became_peered":"2024-04-25T10:11:00.659982+0000","last_unstale":"2024-04-25T10:11:01.791569+0000","last_undegraded":"2024-04-25T10:11:01.791569+0000","last_fullsized":"2024-04-25T10:11:01.791569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:26.638020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791242+0000","last_change":"2024-04-25T10:11:00.672864+0000","last_active":"2024-04-25T10:11:01.791242+0000","last_peered":"2024-04-25T10:11:01.791242+0000","last_clean":"2024-04-25T10:11:01.791242+0000","last_became_active":"2024-04-25T10:11:00.672565+0000","last_became_peered":"2024-04-25T10:11:00.672565+0000","last_unstale":"2024-04-25T10:11:01.791242+0000","last_undegraded":"2024-04-25T10:11:01.791242+0000","last_fullsized":"2024-04-25T10:11:01.791242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:38.932561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.671980+0000","last_change":"2024-04-25T10:11:00.675075+0000","last_active":"2024-04-25T10:11:01.671980+0000","last_peered":"2024-04-25T10:11:01.671980+0000","last_clean":"2024-04-25T10:11:01.671980+0000","last_became_active":"2024-04-25T10:11:00.674791+0000","last_became_peered":"2024-04-25T10:11:00.674791+0000","last_unstale":"2024-04-25T10:11:01.671980+0000","last_undegraded":"2024-04-25T10:11:01.671980+0000","last_fullsized":"2024-04-25T10:11:01.671980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:14.897123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666210+0000","last_change":"2024-04-25T10:11:00.658578+0000","last_active":"2024-04-25T10:11:01.666210+0000","last_peered":"2024-04-25T10:11:01.666210+0000","last_clean":"2024-04-25T10:11:01.666210+0000","last_became_active":"2024-04-25T10:11:00.658309+0000","last_became_peered":"2024-04-25T10:11:00.658309+0000","last_unstale":"2024-04-25T10:11:01.666210+0000","last_undegraded":"2024-04-25T10:11:01.666210+0000","last_fullsized":"2024-04-25T10:11:01.666210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:40.828899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790928+0000","last_change":"2024-04-25T10:11:00.674321+0000","last_active":"2024-04-25T10:11:01.790928+0000","last_peered":"2024-04-25T10:11:01.790928+0000","last_clean":"2024-04-25T10:11:01.790928+0000","last_became_active":"2024-04-25T10:11:00.674037+0000","last_became_peered":"2024-04-25T10:11:00.674037+0000","last_unstale":"2024-04-25T10:11:01.790928+0000","last_undegraded":"2024-04-25T10:11:01.790928+0000","last_fullsized":"2024-04-25T10:11:01.790928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:37.420962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669990+0000","last_change":"2024-04-25T10:11:00.673650+0000","last_active":"2024-04-25T10:11:01.669990+0000","last_peered":"2024-04-25T10:11:01.669990+0000","last_clean":"2024-04-25T10:11:01.669990+0000","last_became_active":"2024-04-25T10:11:00.673318+0000","last_became_peered":"2024-04-25T10:11:00.673318+0000","last_unstale":"2024-04-25T10:11:01.669990+0000","last_undegraded":"2024-04-25T10:11:01.669990+0000","last_fullsized":"2024-04-25T10:11:01.669990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:18.528132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791780+0000","last_change":"2024-04-25T10:11:00.672749+0000","last_active":"2024-04-25T10:11:01.791780+0000","last_peered":"2024-04-25T10:11:01.791780+0000","last_clean":"2024-04-25T10:11:01.791780+0000","last_became_active":"2024-04-25T10:11:00.671982+0000","last_became_peered":"2024-04-25T10:11:00.671982+0000","last_unstale":"2024-04-25T10:11:01.791780+0000","last_undegraded":"2024-04-25T10:11:01.791780+0000","last_fullsized":"2024-04-25T10:11:01.791780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.235679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791396+0000","last_change":"2024-04-25T10:11:00.674897+0000","last_active":"2024-04-25T10:11:01.791396+0000","last_peered":"2024-04-25T10:11:01.791396+0000","last_clean":"2024-04-25T10:11:01.791396+0000","last_became_active":"2024-04-25T10:11:00.674582+0000","last_became_peered":"2024-04-25T10:11:00.674582+0000","last_unstale":"2024-04-25T10:11:01.791396+0000","last_undegraded":"2024-04-25T10:11:01.791396+0000","last_fullsized":"2024-04-25T10:11:01.791396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.603074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.651673+0000","last_change":"2024-04-25T10:10:59.651673+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.651673+0000","last_undegraded":"2024-04-25T10:10:59.651673+0000","last_fullsized":"2024-04-25T10:10:59.651673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791410+0000","last_change":"2024-04-25T10:11:00.670165+0000","last_active":"2024-04-25T10:11:01.791410+0000","last_peered":"2024-04-25T10:11:01.791410+0000","last_clean":"2024-04-25T10:11:01.791410+0000","last_became_active":"2024-04-25T10:11:00.669866+0000","last_became_peered":"2024-04-25T10:11:00.669866+0000","last_unstale":"2024-04-25T10:11:01.791410+0000","last_undegraded":"2024-04-25T10:11:01.791410+0000","last_fullsized":"2024-04-25T10:11:01.791410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:43.529739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790610+0000","last_change":"2024-04-25T10:11:00.674422+0000","last_active":"2024-04-25T10:11:01.790610+0000","last_peered":"2024-04-25T10:11:01.790610+0000","last_clean":"2024-04-25T10:11:01.790610+0000","last_became_active":"2024-04-25T10:11:00.674142+0000","last_became_peered":"2024-04-25T10:11:00.674142+0000","last_unstale":"2024-04-25T10:11:01.790610+0000","last_undegraded":"2024-04-25T10:11:01.790610+0000","last_fullsized":"2024-04-25T10:11:01.790610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:28.012019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669720+0000","last_change":"2024-04-25T10:11:00.672113+0000","last_active":"2024-04-25T10:11:01.669720+0000","last_peered":"2024-04-25T10:11:01.669720+0000","last_clean":"2024-04-25T10:11:01.669720+0000","last_became_active":"2024-04-25T10:11:00.671835+0000","last_became_peered":"2024-04-25T10:11:00.671835+0000","last_unstale":"2024-04-25T10:11:01.669720+0000","last_undegraded":"2024-04-25T10:11:01.669720+0000","last_fullsized":"2024-04-25T10:11:01.669720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:56.991014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.670008+0000","last_change":"2024-04-25T10:11:00.679427+0000","last_active":"2024-04-25T10:11:01.670008+0000","last_peered":"2024-04-25T10:11:01.670008+0000","last_clean":"2024-04-25T10:11:01.670008+0000","last_became_active":"2024-04-25T10:11:00.678268+0000","last_became_peered":"2024-04-25T10:11:00.678268+0000","last_unstale":"2024-04-25T10:11:01.670008+0000","last_undegraded":"2024-04-25T10:11:01.670008+0000","last_fullsized":"2024-04-25T10:11:01.670008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:57.428737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791705+0000","last_change":"2024-04-25T10:11:00.671179+0000","last_active":"2024-04-25T10:11:01.791705+0000","last_peered":"2024-04-25T10:11:01.791705+0000","last_clean":"2024-04-25T10:11:01.791705+0000","last_became_active":"2024-04-25T10:11:00.670882+0000","last_became_peered":"2024-04-25T10:11:00.670882+0000","last_unstale":"2024-04-25T10:11:01.791705+0000","last_undegraded":"2024-04-25T10:11:01.791705+0000","last_fullsized":"2024-04-25T10:11:01.791705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:22.210854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791595+0000","last_change":"2024-04-25T10:11:00.672506+0000","last_active":"2024-04-25T10:11:01.791595+0000","last_peered":"2024-04-25T10:11:01.791595+0000","last_clean":"2024-04-25T10:11:01.791595+0000","last_became_active":"2024-04-25T10:11:00.672018+0000","last_became_peered":"2024-04-25T10:11:00.672018+0000","last_unstale":"2024-04-25T10:11:01.791595+0000","last_undegraded":"2024-04-25T10:11:01.791595+0000","last_fullsized":"2024-04-25T10:11:01.791595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:14.645638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669413+0000","last_change":"2024-04-25T10:11:00.668016+0000","last_active":"2024-04-25T10:11:01.669413+0000","last_peered":"2024-04-25T10:11:01.669413+0000","last_clean":"2024-04-25T10:11:01.669413+0000","last_became_active":"2024-04-25T10:11:00.667742+0000","last_became_peered":"2024-04-25T10:11:00.667742+0000","last_unstale":"2024-04-25T10:11:01.669413+0000","last_undegraded":"2024-04-25T10:11:01.669413+0000","last_fullsized":"2024-04-25T10:11:01.669413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:35.210017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.651157+0000","last_change":"2024-04-25T10:10:59.651157+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.651157+0000","last_undegraded":"2024-04-25T10:10:59.651157+0000","last_fullsized":"2024-04-25T10:10:59.651157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791586+0000","last_change":"2024-04-25T10:11:00.672908+0000","last_active":"2024-04-25T10:11:01.791586+0000","last_peered":"2024-04-25T10:11:01.791586+0000","last_clean":"2024-04-25T10:11:01.791586+0000","last_became_active":"2024-04-25T10:11:00.672634+0000","last_became_peered":"2024-04-25T10:11:00.672634+0000","last_unstale":"2024-04-25T10:11:01.791586+0000","last_undegraded":"2024-04-25T10:11:01.791586+0000","last_fullsized":"2024-04-25T10:11:01.791586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:37.071670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666210+0000","last_change":"2024-04-25T10:11:00.667457+0000","last_active":"2024-04-25T10:11:01.666210+0000","last_peered":"2024-04-25T10:11:01.666210+0000","last_clean":"2024-04-25T10:11:01.666210+0000","last_became_active":"2024-04-25T10:11:00.667205+0000","last_became_peered":"2024-04-25T10:11:00.667205+0000","last_unstale":"2024-04-25T10:11:01.666210+0000","last_undegraded":"2024-04-25T10:11:01.666210+0000","last_fullsized":"2024-04-25T10:11:01.666210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:07.289251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666783+0000","last_change":"2024-04-25T10:11:00.668155+0000","last_active":"2024-04-25T10:11:01.666783+0000","last_peered":"2024-04-25T10:11:01.666783+0000","last_clean":"2024-04-25T10:11:01.666783+0000","last_became_active":"2024-04-25T10:11:00.667848+0000","last_became_peered":"2024-04-25T10:11:00.667848+0000","last_unstale":"2024-04-25T10:11:01.666783+0000","last_undegraded":"2024-04-25T10:11:01.666783+0000","last_fullsized":"2024-04-25T10:11:01.666783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:55.212072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790745+0000","last_change":"2024-04-25T10:11:00.664212+0000","last_active":"2024-04-25T10:11:01.790745+0000","last_peered":"2024-04-25T10:11:01.790745+0000","last_clean":"2024-04-25T10:11:01.790745+0000","last_became_active":"2024-04-25T10:11:00.663662+0000","last_became_peered":"2024-04-25T10:11:00.663662+0000","last_unstale":"2024-04-25T10:11:01.790745+0000","last_undegraded":"2024-04-25T10:11:01.790745+0000","last_fullsized":"2024-04-25T10:11:01.790745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:14.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"85'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791831+0000","last_change":"2024-04-25T10:10:59.661968+0000","last_active":"2024-04-25T10:11:01.791831+0000","last_peered":"2024-04-25T10:11:01.791831+0000","last_clean":"2024-04-25T10:11:01.791831+0000","last_became_active":"2024-04-25T10:10:59.661557+0000","last_became_peered":"2024-04-25T10:10:59.661557+0000","last_unstale":"2024-04-25T10:11:01.791831+0000","last_undegraded":"2024-04-25T10:11:01.791831+0000","last_fullsized":"2024-04-25T10:11:01.791831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:29.166481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669745+0000","last_change":"2024-04-25T10:11:00.664765+0000","last_active":"2024-04-25T10:11:01.669745+0000","last_peered":"2024-04-25T10:11:01.669745+0000","last_clean":"2024-04-25T10:11:01.669745+0000","last_became_active":"2024-04-25T10:11:00.664306+0000","last_became_peered":"2024-04-25T10:11:00.664306+0000","last_unstale":"2024-04-25T10:11:01.669745+0000","last_undegraded":"2024-04-25T10:11:01.669745+0000","last_fullsized":"2024-04-25T10:11:01.669745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:51.779177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666622+0000","last_change":"2024-04-25T10:10:59.659985+0000","last_active":"2024-04-25T10:11:01.666622+0000","last_peered":"2024-04-25T10:11:01.666622+0000","last_clean":"2024-04-25T10:11:01.666622+0000","last_became_active":"2024-04-25T10:10:59.659641+0000","last_became_peered":"2024-04-25T10:10:59.659641+0000","last_unstale":"2024-04-25T10:11:01.666622+0000","last_undegraded":"2024-04-25T10:11:01.666622+0000","last_fullsized":"2024-04-25T10:11:01.666622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:35.098559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790943+0000","last_change":"2024-04-25T10:11:00.666415+0000","last_active":"2024-04-25T10:11:01.790943+0000","last_peered":"2024-04-25T10:11:01.790943+0000","last_clean":"2024-04-25T10:11:01.790943+0000","last_became_active":"2024-04-25T10:11:00.666151+0000","last_became_peered":"2024-04-25T10:11:00.666151+0000","last_unstale":"2024-04-25T10:11:01.790943+0000","last_undegraded":"2024-04-25T10:11:01.790943+0000","last_fullsized":"2024-04-25T10:11:01.790943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:43.188026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669929+0000","last_change":"2024-04-25T10:10:59.654631+0000","last_active":"2024-04-25T10:11:01.669929+0000","last_peered":"2024-04-25T10:11:01.669929+0000","last_clean":"2024-04-25T10:11:01.669929+0000","last_became_active":"2024-04-25T10:10:59.654288+0000","last_became_peered":"2024-04-25T10:10:59.654288+0000","last_unstale":"2024-04-25T10:11:01.669929+0000","last_undegraded":"2024-04-25T10:11:01.669929+0000","last_fullsized":"2024-04-25T10:11:01.669929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:34.205125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669497+0000","last_change":"2024-04-25T10:11:00.677043+0000","last_active":"2024-04-25T10:11:01.669497+0000","last_peered":"2024-04-25T10:11:01.669497+0000","last_clean":"2024-04-25T10:11:01.669497+0000","last_became_active":"2024-04-25T10:11:00.675514+0000","last_became_peered":"2024-04-25T10:11:00.675514+0000","last_unstale":"2024-04-25T10:11:01.669497+0000","last_undegraded":"2024-04-25T10:11:01.669497+0000","last_fullsized":"2024-04-25T10:11:01.669497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:46.822338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T10:10:59.664299+0000","last_change":"2024-04-25T10:10:59.658687+0000","last_active":"2024-04-25T10:10:59.664299+0000","last_peered":"2024-04-25T10:10:59.664299+0000","last_clean":"2024-04-25T10:10:59.664299+0000","last_became_active":"2024-04-25T10:10:59.658380+0000","last_became_peered":"2024-04-25T10:10:59.658380+0000","last_unstale":"2024-04-25T10:10:59.664299+0000","last_undegraded":"2024-04-25T10:10:59.664299+0000","last_fullsized":"2024-04-25T10:10:59.664299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:14.321738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.652342+0000","last_change":"2024-04-25T10:10:59.652342+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.652342+0000","last_undegraded":"2024-04-25T10:10:59.652342+0000","last_fullsized":"2024-04-25T10:10:59.652342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790565+0000","last_change":"2024-04-25T10:10:59.658839+0000","last_active":"2024-04-25T10:11:01.790565+0000","last_peered":"2024-04-25T10:11:01.790565+0000","last_clean":"2024-04-25T10:11:01.790565+0000","last_became_active":"2024-04-25T10:10:59.658557+0000","last_became_peered":"2024-04-25T10:10:59.658557+0000","last_unstale":"2024-04-25T10:11:01.790565+0000","last_undegraded":"2024-04-25T10:11:01.790565+0000","last_fullsized":"2024-04-25T10:11:01.790565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:35.098559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666691+0000","last_change":"2024-04-25T10:11:00.670164+0000","last_active":"2024-04-25T10:11:01.666691+0000","last_peered":"2024-04-25T10:11:01.666691+0000","last_clean":"2024-04-25T10:11:01.666691+0000","last_became_active":"2024-04-25T10:11:00.669856+0000","last_became_peered":"2024-04-25T10:11:00.669856+0000","last_unstale":"2024-04-25T10:11:01.666691+0000","last_undegraded":"2024-04-25T10:11:01.666691+0000","last_fullsized":"2024-04-25T10:11:01.666691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:56.966365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666588+0000","last_change":"2024-04-25T10:10:59.660519+0000","last_active":"2024-04-25T10:11:01.666588+0000","last_peered":"2024-04-25T10:11:01.666588+0000","last_clean":"2024-04-25T10:11:01.666588+0000","last_became_active":"2024-04-25T10:10:59.659571+0000","last_became_peered":"2024-04-25T10:10:59.659571+0000","last_unstale":"2024-04-25T10:11:01.666588+0000","last_undegraded":"2024-04-25T10:11:01.666588+0000","last_fullsized":"2024-04-25T10:11:01.666588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:47.638024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669228+0000","last_change":"2024-04-25T10:11:00.672808+0000","last_active":"2024-04-25T10:11:01.669228+0000","last_peered":"2024-04-25T10:11:01.669228+0000","last_clean":"2024-04-25T10:11:01.669228+0000","last_became_active":"2024-04-25T10:11:00.672479+0000","last_became_peered":"2024-04-25T10:11:00.672479+0000","last_unstale":"2024-04-25T10:11:01.669228+0000","last_undegraded":"2024-04-25T10:11:01.669228+0000","last_fullsized":"2024-04-25T10:11:01.669228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:43.529739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791149+0000","last_change":"2024-04-25T10:10:59.662636+0000","last_active":"2024-04-25T10:11:01.791149+0000","last_peered":"2024-04-25T10:11:01.791149+0000","last_clean":"2024-04-25T10:11:01.791149+0000","last_became_active":"2024-04-25T10:10:59.661790+0000","last_became_peered":"2024-04-25T10:10:59.661790+0000","last_unstale":"2024-04-25T10:11:01.791149+0000","last_undegraded":"2024-04-25T10:11:01.791149+0000","last_fullsized":"2024-04-25T10:11:01.791149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:10.609819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791191+0000","last_change":"2024-04-25T10:11:00.671182+0000","last_active":"2024-04-25T10:11:01.791191+0000","last_peered":"2024-04-25T10:11:01.791191+0000","last_clean":"2024-04-25T10:11:01.791191+0000","last_became_active":"2024-04-25T10:11:00.670863+0000","last_became_peered":"2024-04-25T10:11:00.670863+0000","last_unstale":"2024-04-25T10:11:01.791191+0000","last_undegraded":"2024-04-25T10:11:01.791191+0000","last_fullsized":"2024-04-25T10:11:01.791191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:21.622898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791076+0000","last_change":"2024-04-25T10:10:59.661931+0000","last_active":"2024-04-25T10:11:01.791076+0000","last_peered":"2024-04-25T10:11:01.791076+0000","last_clean":"2024-04-25T10:11:01.791076+0000","last_became_active":"2024-04-25T10:10:59.661173+0000","last_became_peered":"2024-04-25T10:10:59.661173+0000","last_unstale":"2024-04-25T10:11:01.791076+0000","last_undegraded":"2024-04-25T10:11:01.791076+0000","last_fullsized":"2024-04-25T10:11:01.791076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:35.826027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791331+0000","last_change":"2024-04-25T10:11:00.676424+0000","last_active":"2024-04-25T10:11:01.791331+0000","last_peered":"2024-04-25T10:11:01.791331+0000","last_clean":"2024-04-25T10:11:01.791331+0000","last_became_active":"2024-04-25T10:11:00.676088+0000","last_became_peered":"2024-04-25T10:11:00.676088+0000","last_unstale":"2024-04-25T10:11:01.791331+0000","last_undegraded":"2024-04-25T10:11:01.791331+0000","last_fullsized":"2024-04-25T10:11:01.791331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:50.887593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669814+0000","last_change":"2024-04-25T10:10:59.660535+0000","last_active":"2024-04-25T10:11:01.669814+0000","last_peered":"2024-04-25T10:11:01.669814+0000","last_clean":"2024-04-25T10:11:01.669814+0000","last_became_active":"2024-04-25T10:10:59.660260+0000","last_became_peered":"2024-04-25T10:10:59.660260+0000","last_unstale":"2024-04-25T10:11:01.669814+0000","last_undegraded":"2024-04-25T10:11:01.669814+0000","last_fullsized":"2024-04-25T10:11:01.669814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:23.530134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669744+0000","last_change":"2024-04-25T10:11:00.684312+0000","last_active":"2024-04-25T10:11:01.669744+0000","last_peered":"2024-04-25T10:11:01.669744+0000","last_clean":"2024-04-25T10:11:01.669744+0000","last_became_active":"2024-04-25T10:11:00.683971+0000","last_became_peered":"2024-04-25T10:11:00.683971+0000","last_unstale":"2024-04-25T10:11:01.669744+0000","last_undegraded":"2024-04-25T10:11:01.669744+0000","last_fullsized":"2024-04-25T10:11:01.669744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:40.703987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666638+0000","last_change":"2024-04-25T10:10:59.658424+0000","last_active":"2024-04-25T10:11:01.666638+0000","last_peered":"2024-04-25T10:11:01.666638+0000","last_clean":"2024-04-25T10:11:01.666638+0000","last_became_active":"2024-04-25T10:10:59.658128+0000","last_became_peered":"2024-04-25T10:10:59.658128+0000","last_unstale":"2024-04-25T10:11:01.666638+0000","last_undegraded":"2024-04-25T10:11:01.666638+0000","last_fullsized":"2024-04-25T10:11:01.666638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:42.113345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669659+0000","last_change":"2024-04-25T10:11:00.678536+0000","last_active":"2024-04-25T10:11:01.669659+0000","last_peered":"2024-04-25T10:11:01.669659+0000","last_clean":"2024-04-25T10:11:01.669659+0000","last_became_active":"2024-04-25T10:11:00.678276+0000","last_became_peered":"2024-04-25T10:11:00.678276+0000","last_unstale":"2024-04-25T10:11:01.669659+0000","last_undegraded":"2024-04-25T10:11:01.669659+0000","last_fullsized":"2024-04-25T10:11:01.669659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:48.642917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"85'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669547+0000","last_change":"2024-04-25T10:10:59.653278+0000","last_active":"2024-04-25T10:11:01.669547+0000","last_peered":"2024-04-25T10:11:01.669547+0000","last_clean":"2024-04-25T10:11:01.669547+0000","last_became_active":"2024-04-25T10:10:59.652974+0000","last_became_peered":"2024-04-25T10:10:59.652974+0000","last_unstale":"2024-04-25T10:11:01.669547+0000","last_undegraded":"2024-04-25T10:11:01.669547+0000","last_fullsized":"2024-04-25T10:11:01.669547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:45.820044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791449+0000","last_change":"2024-04-25T10:11:00.684587+0000","last_active":"2024-04-25T10:11:01.791449+0000","last_peered":"2024-04-25T10:11:01.791449+0000","last_clean":"2024-04-25T10:11:01.791449+0000","last_became_active":"2024-04-25T10:11:00.684306+0000","last_became_peered":"2024-04-25T10:11:00.684306+0000","last_unstale":"2024-04-25T10:11:01.791449+0000","last_undegraded":"2024-04-25T10:11:01.791449+0000","last_fullsized":"2024-04-25T10:11:01.791449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:39.955023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"85'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790926+0000","last_change":"2024-04-25T10:10:59.662733+0000","last_active":"2024-04-25T10:11:01.790926+0000","last_peered":"2024-04-25T10:11:01.790926+0000","last_clean":"2024-04-25T10:11:01.790926+0000","last_became_active":"2024-04-25T10:10:59.662290+0000","last_became_peered":"2024-04-25T10:10:59.662290+0000","last_unstale":"2024-04-25T10:11:01.790926+0000","last_undegraded":"2024-04-25T10:11:01.790926+0000","last_fullsized":"2024-04-25T10:11:01.790926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:34.280071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669718+0000","last_change":"2024-04-25T10:11:00.669181+0000","last_active":"2024-04-25T10:11:01.669718+0000","last_peered":"2024-04-25T10:11:01.669718+0000","last_clean":"2024-04-25T10:11:01.669718+0000","last_became_active":"2024-04-25T10:11:00.668758+0000","last_became_peered":"2024-04-25T10:11:00.668758+0000","last_unstale":"2024-04-25T10:11:01.669718+0000","last_undegraded":"2024-04-25T10:11:01.669718+0000","last_fullsized":"2024-04-25T10:11:01.669718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:38.681757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T10:10:59.666667+0000","last_change":"2024-04-25T10:10:59.662230+0000","last_active":"2024-04-25T10:10:59.666667+0000","last_peered":"2024-04-25T10:10:59.666667+0000","last_clean":"2024-04-25T10:10:59.666667+0000","last_became_active":"2024-04-25T10:10:59.661953+0000","last_became_peered":"2024-04-25T10:10:59.661953+0000","last_unstale":"2024-04-25T10:10:59.666667+0000","last_undegraded":"2024-04-25T10:10:59.666667+0000","last_fullsized":"2024-04-25T10:10:59.666667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:31.766942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669387+0000","last_change":"2024-04-25T10:11:00.680010+0000","last_active":"2024-04-25T10:11:01.669387+0000","last_peered":"2024-04-25T10:11:01.669387+0000","last_clean":"2024-04-25T10:11:01.669387+0000","last_became_active":"2024-04-25T10:11:00.679712+0000","last_became_peered":"2024-04-25T10:11:00.679712+0000","last_unstale":"2024-04-25T10:11:01.669387+0000","last_undegraded":"2024-04-25T10:11:01.669387+0000","last_fullsized":"2024-04-25T10:11:01.669387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:22.568376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672076+0000","last_change":"2024-04-25T10:10:59.659677+0000","last_active":"2024-04-25T10:11:01.672076+0000","last_peered":"2024-04-25T10:11:01.672076+0000","last_clean":"2024-04-25T10:11:01.672076+0000","last_became_active":"2024-04-25T10:10:59.659395+0000","last_became_peered":"2024-04-25T10:10:59.659395+0000","last_unstale":"2024-04-25T10:11:01.672076+0000","last_undegraded":"2024-04-25T10:11:01.672076+0000","last_fullsized":"2024-04-25T10:11:01.672076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:21.670434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790957+0000","last_change":"2024-04-25T10:11:00.678832+0000","last_active":"2024-04-25T10:11:01.790957+0000","last_peered":"2024-04-25T10:11:01.790957+0000","last_clean":"2024-04-25T10:11:01.790957+0000","last_became_active":"2024-04-25T10:11:00.678560+0000","last_became_peered":"2024-04-25T10:11:00.678560+0000","last_unstale":"2024-04-25T10:11:01.790957+0000","last_undegraded":"2024-04-25T10:11:01.790957+0000","last_fullsized":"2024-04-25T10:11:01.790957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:19.719591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666238+0000","last_change":"2024-04-25T10:10:59.656540+0000","last_active":"2024-04-25T10:11:01.666238+0000","last_peered":"2024-04-25T10:11:01.666238+0000","last_clean":"2024-04-25T10:11:01.666238+0000","last_became_active":"2024-04-25T10:10:59.656276+0000","last_became_peered":"2024-04-25T10:10:59.656276+0000","last_unstale":"2024-04-25T10:11:01.666238+0000","last_undegraded":"2024-04-25T10:11:01.666238+0000","last_fullsized":"2024-04-25T10:11:01.666238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:13.892230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669548+0000","last_change":"2024-04-25T10:11:00.680212+0000","last_active":"2024-04-25T10:11:01.669548+0000","last_peered":"2024-04-25T10:11:01.669548+0000","last_clean":"2024-04-25T10:11:01.669548+0000","last_became_active":"2024-04-25T10:11:00.679944+0000","last_became_peered":"2024-04-25T10:11:00.679944+0000","last_unstale":"2024-04-25T10:11:01.669548+0000","last_undegraded":"2024-04-25T10:11:01.669548+0000","last_fullsized":"2024-04-25T10:11:01.669548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:14.897123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"85'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669582+0000","last_change":"2024-04-25T10:10:59.656545+0000","last_active":"2024-04-25T10:11:01.669582+0000","last_peered":"2024-04-25T10:11:01.669582+0000","last_clean":"2024-04-25T10:11:01.669582+0000","last_became_active":"2024-04-25T10:10:59.656262+0000","last_became_peered":"2024-04-25T10:10:59.656262+0000","last_unstale":"2024-04-25T10:11:01.669582+0000","last_undegraded":"2024-04-25T10:11:01.669582+0000","last_fullsized":"2024-04-25T10:11:01.669582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:12.740681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790809+0000","last_change":"2024-04-25T10:11:00.660578+0000","last_active":"2024-04-25T10:11:01.790809+0000","last_peered":"2024-04-25T10:11:01.790809+0000","last_clean":"2024-04-25T10:11:01.790809+0000","last_became_active":"2024-04-25T10:11:00.660258+0000","last_became_peered":"2024-04-25T10:11:00.660258+0000","last_unstale":"2024-04-25T10:11:01.790809+0000","last_undegraded":"2024-04-25T10:11:01.790809+0000","last_fullsized":"2024-04-25T10:11:01.790809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:20.151958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791898+0000","last_change":"2024-04-25T10:10:59.660067+0000","last_active":"2024-04-25T10:11:01.791898+0000","last_peered":"2024-04-25T10:11:01.791898+0000","last_clean":"2024-04-25T10:11:01.791898+0000","last_became_active":"2024-04-25T10:10:59.659717+0000","last_became_peered":"2024-04-25T10:10:59.659717+0000","last_unstale":"2024-04-25T10:11:01.791898+0000","last_undegraded":"2024-04-25T10:11:01.791898+0000","last_fullsized":"2024-04-25T10:11:01.791898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:55.225255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666704+0000","last_change":"2024-04-25T10:11:00.664267+0000","last_active":"2024-04-25T10:11:01.666704+0000","last_peered":"2024-04-25T10:11:01.666704+0000","last_clean":"2024-04-25T10:11:01.666704+0000","last_became_active":"2024-04-25T10:11:00.664038+0000","last_became_peered":"2024-04-25T10:11:00.664038+0000","last_unstale":"2024-04-25T10:11:01.666704+0000","last_undegraded":"2024-04-25T10:11:01.666704+0000","last_fullsized":"2024-04-25T10:11:01.666704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:20.151958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791555+0000","last_change":"2024-04-25T10:10:59.661418+0000","last_active":"2024-04-25T10:11:01.791555+0000","last_peered":"2024-04-25T10:11:01.791555+0000","last_clean":"2024-04-25T10:11:01.791555+0000","last_became_active":"2024-04-25T10:10:59.661121+0000","last_became_peered":"2024-04-25T10:10:59.661121+0000","last_unstale":"2024-04-25T10:11:01.791555+0000","last_undegraded":"2024-04-25T10:11:01.791555+0000","last_fullsized":"2024-04-25T10:11:01.791555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:53.008632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790835+0000","last_change":"2024-04-25T10:11:00.662883+0000","last_active":"2024-04-25T10:11:01.790835+0000","last_peered":"2024-04-25T10:11:01.790835+0000","last_clean":"2024-04-25T10:11:01.790835+0000","last_became_active":"2024-04-25T10:11:00.662578+0000","last_became_peered":"2024-04-25T10:11:00.662578+0000","last_unstale":"2024-04-25T10:11:01.790835+0000","last_undegraded":"2024-04-25T10:11:01.790835+0000","last_fullsized":"2024-04-25T10:11:01.790835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:55.212072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791634+0000","last_change":"2024-04-25T10:10:59.663404+0000","last_active":"2024-04-25T10:11:01.791634+0000","last_peered":"2024-04-25T10:11:01.791634+0000","last_clean":"2024-04-25T10:11:01.791634+0000","last_became_active":"2024-04-25T10:10:59.663138+0000","last_became_peered":"2024-04-25T10:10:59.663138+0000","last_unstale":"2024-04-25T10:11:01.791634+0000","last_undegraded":"2024-04-25T10:11:01.791634+0000","last_fullsized":"2024-04-25T10:11:01.791634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:38.050152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672109+0000","last_change":"2024-04-25T10:11:00.672494+0000","last_active":"2024-04-25T10:11:01.672109+0000","last_peered":"2024-04-25T10:11:01.672109+0000","last_clean":"2024-04-25T10:11:01.672109+0000","last_became_active":"2024-04-25T10:11:00.672238+0000","last_became_peered":"2024-04-25T10:11:00.672238+0000","last_unstale":"2024-04-25T10:11:01.672109+0000","last_undegraded":"2024-04-25T10:11:01.672109+0000","last_fullsized":"2024-04-25T10:11:01.672109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:48.642917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791130+0000","last_change":"2024-04-25T10:10:59.662706+0000","last_active":"2024-04-25T10:11:01.791130+0000","last_peered":"2024-04-25T10:11:01.791130+0000","last_clean":"2024-04-25T10:11:01.791130+0000","last_became_active":"2024-04-25T10:10:59.662399+0000","last_became_peered":"2024-04-25T10:10:59.662399+0000","last_unstale":"2024-04-25T10:11:01.791130+0000","last_undegraded":"2024-04-25T10:11:01.791130+0000","last_fullsized":"2024-04-25T10:11:01.791130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:56.481855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791760+0000","last_change":"2024-04-25T10:11:00.684360+0000","last_active":"2024-04-25T10:11:01.791760+0000","last_peered":"2024-04-25T10:11:01.791760+0000","last_clean":"2024-04-25T10:11:01.791760+0000","last_became_active":"2024-04-25T10:11:00.683915+0000","last_became_peered":"2024-04-25T10:11:00.683915+0000","last_unstale":"2024-04-25T10:11:01.791760+0000","last_undegraded":"2024-04-25T10:11:01.791760+0000","last_fullsized":"2024-04-25T10:11:01.791760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:44.016510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672207+0000","last_change":"2024-04-25T10:10:59.664474+0000","last_active":"2024-04-25T10:11:01.672207+0000","last_peered":"2024-04-25T10:11:01.672207+0000","last_clean":"2024-04-25T10:11:01.672207+0000","last_became_active":"2024-04-25T10:10:59.664182+0000","last_became_peered":"2024-04-25T10:10:59.664182+0000","last_unstale":"2024-04-25T10:11:01.672207+0000","last_undegraded":"2024-04-25T10:11:01.672207+0000","last_fullsized":"2024-04-25T10:11:01.672207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:51.674567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.653171+0000","last_change":"2024-04-25T10:10:59.653171+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.653171+0000","last_undegraded":"2024-04-25T10:10:59.653171+0000","last_fullsized":"2024-04-25T10:10:59.653171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669493+0000","last_change":"2024-04-25T10:10:59.656426+0000","last_active":"2024-04-25T10:11:01.669493+0000","last_peered":"2024-04-25T10:11:01.669493+0000","last_clean":"2024-04-25T10:11:01.669493+0000","last_became_active":"2024-04-25T10:10:59.656089+0000","last_became_peered":"2024-04-25T10:10:59.656089+0000","last_unstale":"2024-04-25T10:11:01.669493+0000","last_undegraded":"2024-04-25T10:11:01.669493+0000","last_fullsized":"2024-04-25T10:11:01.669493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:50.774285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790576+0000","last_change":"2024-04-25T10:11:00.675256+0000","last_active":"2024-04-25T10:11:01.790576+0000","last_peered":"2024-04-25T10:11:01.790576+0000","last_clean":"2024-04-25T10:11:01.790576+0000","last_became_active":"2024-04-25T10:11:00.674969+0000","last_became_peered":"2024-04-25T10:11:00.674969+0000","last_unstale":"2024-04-25T10:11:01.790576+0000","last_undegraded":"2024-04-25T10:11:01.790576+0000","last_fullsized":"2024-04-25T10:11:01.790576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.860030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791546+0000","last_change":"2024-04-25T10:10:59.662798+0000","last_active":"2024-04-25T10:11:01.791546+0000","last_peered":"2024-04-25T10:11:01.791546+0000","last_clean":"2024-04-25T10:11:01.791546+0000","last_became_active":"2024-04-25T10:10:59.661497+0000","last_became_peered":"2024-04-25T10:10:59.661497+0000","last_unstale":"2024-04-25T10:11:01.791546+0000","last_undegraded":"2024-04-25T10:11:01.791546+0000","last_fullsized":"2024-04-25T10:11:01.791546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:29.530495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791719+0000","last_change":"2024-04-25T10:11:00.666183+0000","last_active":"2024-04-25T10:11:01.791719+0000","last_peered":"2024-04-25T10:11:01.791719+0000","last_clean":"2024-04-25T10:11:01.791719+0000","last_became_active":"2024-04-25T10:11:00.665905+0000","last_became_peered":"2024-04-25T10:11:00.665905+0000","last_unstale":"2024-04-25T10:11:01.791719+0000","last_undegraded":"2024-04-25T10:11:01.791719+0000","last_fullsized":"2024-04-25T10:11:01.791719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.427341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791772+0000","last_change":"2024-04-25T10:10:59.661863+0000","last_active":"2024-04-25T10:11:01.791772+0000","last_peered":"2024-04-25T10:11:01.791772+0000","last_clean":"2024-04-25T10:11:01.791772+0000","last_became_active":"2024-04-25T10:10:59.661598+0000","last_became_peered":"2024-04-25T10:10:59.661598+0000","last_unstale":"2024-04-25T10:11:01.791772+0000","last_undegraded":"2024-04-25T10:11:01.791772+0000","last_fullsized":"2024-04-25T10:11:01.791772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:45.820044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791785+0000","last_change":"2024-04-25T10:11:00.666115+0000","last_active":"2024-04-25T10:11:01.791785+0000","last_peered":"2024-04-25T10:11:01.791785+0000","last_clean":"2024-04-25T10:11:01.791785+0000","last_became_active":"2024-04-25T10:11:00.665817+0000","last_became_peered":"2024-04-25T10:11:00.665817+0000","last_unstale":"2024-04-25T10:11:01.791785+0000","last_undegraded":"2024-04-25T10:11:01.791785+0000","last_fullsized":"2024-04-25T10:11:01.791785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:10.823889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.670527+0000","last_change":"2024-04-25T10:10:59.654401+0000","last_active":"2024-04-25T10:11:01.670527+0000","last_peered":"2024-04-25T10:11:01.670527+0000","last_clean":"2024-04-25T10:11:01.670527+0000","last_became_active":"2024-04-25T10:10:59.654067+0000","last_became_peered":"2024-04-25T10:10:59.654067+0000","last_unstale":"2024-04-25T10:11:01.670527+0000","last_undegraded":"2024-04-25T10:11:01.670527+0000","last_fullsized":"2024-04-25T10:11:01.670527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:45.817445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791696+0000","last_change":"2024-04-25T10:11:00.671112+0000","last_active":"2024-04-25T10:11:01.791696+0000","last_peered":"2024-04-25T10:11:01.791696+0000","last_clean":"2024-04-25T10:11:01.791696+0000","last_became_active":"2024-04-25T10:11:00.670792+0000","last_became_peered":"2024-04-25T10:11:00.670792+0000","last_unstale":"2024-04-25T10:11:01.791696+0000","last_undegraded":"2024-04-25T10:11:01.791696+0000","last_fullsized":"2024-04-25T10:11:01.791696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:00.467722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790958+0000","last_change":"2024-04-25T10:10:59.660751+0000","last_active":"2024-04-25T10:11:01.790958+0000","last_peered":"2024-04-25T10:11:01.790958+0000","last_clean":"2024-04-25T10:11:01.790958+0000","last_became_active":"2024-04-25T10:10:59.660439+0000","last_became_peered":"2024-04-25T10:10:59.660439+0000","last_unstale":"2024-04-25T10:11:01.790958+0000","last_undegraded":"2024-04-25T10:11:01.790958+0000","last_fullsized":"2024-04-25T10:11:01.790958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:17.523240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666630+0000","last_change":"2024-04-25T10:11:00.662687+0000","last_active":"2024-04-25T10:11:01.666630+0000","last_peered":"2024-04-25T10:11:01.666630+0000","last_clean":"2024-04-25T10:11:01.666630+0000","last_became_active":"2024-04-25T10:11:00.662430+0000","last_became_peered":"2024-04-25T10:11:00.662430+0000","last_unstale":"2024-04-25T10:11:01.666630+0000","last_undegraded":"2024-04-25T10:11:01.666630+0000","last_fullsized":"2024-04-25T10:11:01.666630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:14.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666396+0000","last_change":"2024-04-25T10:10:59.660156+0000","last_active":"2024-04-25T10:11:01.666396+0000","last_peered":"2024-04-25T10:11:01.666396+0000","last_clean":"2024-04-25T10:11:01.666396+0000","last_became_active":"2024-04-25T10:10:59.659878+0000","last_became_peered":"2024-04-25T10:10:59.659878+0000","last_unstale":"2024-04-25T10:11:01.666396+0000","last_undegraded":"2024-04-25T10:11:01.666396+0000","last_fullsized":"2024-04-25T10:11:01.666396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:55.986122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669254+0000","last_change":"2024-04-25T10:11:00.664855+0000","last_active":"2024-04-25T10:11:01.669254+0000","last_peered":"2024-04-25T10:11:01.669254+0000","last_clean":"2024-04-25T10:11:01.669254+0000","last_became_active":"2024-04-25T10:11:00.664174+0000","last_became_peered":"2024-04-25T10:11:00.664174+0000","last_unstale":"2024-04-25T10:11:01.669254+0000","last_undegraded":"2024-04-25T10:11:01.669254+0000","last_fullsized":"2024-04-25T10:11:01.669254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.235679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672299+0000","last_change":"2024-04-25T10:10:59.656397+0000","last_active":"2024-04-25T10:11:01.672299+0000","last_peered":"2024-04-25T10:11:01.672299+0000","last_clean":"2024-04-25T10:11:01.672299+0000","last_became_active":"2024-04-25T10:10:59.656114+0000","last_became_peered":"2024-04-25T10:10:59.656114+0000","last_unstale":"2024-04-25T10:11:01.672299+0000","last_undegraded":"2024-04-25T10:11:01.672299+0000","last_fullsized":"2024-04-25T10:11:01.672299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:39.088943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669867+0000","last_change":"2024-04-25T10:11:00.675967+0000","last_active":"2024-04-25T10:11:01.669867+0000","last_peered":"2024-04-25T10:11:01.669867+0000","last_clean":"2024-04-25T10:11:01.669867+0000","last_became_active":"2024-04-25T10:11:00.673547+0000","last_became_peered":"2024-04-25T10:11:00.673547+0000","last_unstale":"2024-04-25T10:11:01.669867+0000","last_undegraded":"2024-04-25T10:11:01.669867+0000","last_fullsized":"2024-04-25T10:11:01.669867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:43.118238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666689+0000","last_change":"2024-04-25T10:10:59.661201+0000","last_active":"2024-04-25T10:11:01.666689+0000","last_peered":"2024-04-25T10:11:01.666689+0000","last_clean":"2024-04-25T10:11:01.666689+0000","last_became_active":"2024-04-25T10:10:59.660913+0000","last_became_peered":"2024-04-25T10:10:59.660913+0000","last_unstale":"2024-04-25T10:11:01.666689+0000","last_undegraded":"2024-04-25T10:11:01.666689+0000","last_fullsized":"2024-04-25T10:11:01.666689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:37.676864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669208+0000","last_change":"2024-04-25T10:11:00.664505+0000","last_active":"2024-04-25T10:11:01.669208+0000","last_peered":"2024-04-25T10:11:01.669208+0000","last_clean":"2024-04-25T10:11:01.669208+0000","last_became_active":"2024-04-25T10:11:00.664218+0000","last_became_peered":"2024-04-25T10:11:00.664218+0000","last_unstale":"2024-04-25T10:11:01.669208+0000","last_undegraded":"2024-04-25T10:11:01.669208+0000","last_fullsized":"2024-04-25T10:11:01.669208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.603074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669199+0000","last_change":"2024-04-25T10:10:59.652758+0000","last_active":"2024-04-25T10:11:01.669199+0000","last_peered":"2024-04-25T10:11:01.669199+0000","last_clean":"2024-04-25T10:11:01.669199+0000","last_became_active":"2024-04-25T10:10:59.652456+0000","last_became_peered":"2024-04-25T10:10:59.652456+0000","last_unstale":"2024-04-25T10:11:01.669199+0000","last_undegraded":"2024-04-25T10:11:01.669199+0000","last_fullsized":"2024-04-25T10:11:01.669199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:53.008632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669239+0000","last_change":"2024-04-25T10:11:00.660736+0000","last_active":"2024-04-25T10:11:01.669239+0000","last_peered":"2024-04-25T10:11:01.669239+0000","last_clean":"2024-04-25T10:11:01.669239+0000","last_became_active":"2024-04-25T10:11:00.660471+0000","last_became_peered":"2024-04-25T10:11:00.660471+0000","last_unstale":"2024-04-25T10:11:01.669239+0000","last_undegraded":"2024-04-25T10:11:01.669239+0000","last_fullsized":"2024-04-25T10:11:01.669239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:37.071670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790521+0000","last_change":"2024-04-25T10:10:59.660087+0000","last_active":"2024-04-25T10:11:01.790521+0000","last_peered":"2024-04-25T10:11:01.790521+0000","last_clean":"2024-04-25T10:11:01.790521+0000","last_became_active":"2024-04-25T10:10:59.659815+0000","last_became_peered":"2024-04-25T10:10:59.659815+0000","last_unstale":"2024-04-25T10:11:01.790521+0000","last_undegraded":"2024-04-25T10:11:01.790521+0000","last_fullsized":"2024-04-25T10:11:01.790521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:42.113345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669804+0000","last_change":"2024-04-25T10:11:00.674881+0000","last_active":"2024-04-25T10:11:01.669804+0000","last_peered":"2024-04-25T10:11:01.669804+0000","last_clean":"2024-04-25T10:11:01.669804+0000","last_became_active":"2024-04-25T10:11:00.674632+0000","last_became_peered":"2024-04-25T10:11:00.674632+0000","last_unstale":"2024-04-25T10:11:01.669804+0000","last_undegraded":"2024-04-25T10:11:01.669804+0000","last_fullsized":"2024-04-25T10:11:01.669804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:36.103452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791702+0000","last_change":"2024-04-25T10:10:59.663122+0000","last_active":"2024-04-25T10:11:01.791702+0000","last_peered":"2024-04-25T10:11:01.791702+0000","last_clean":"2024-04-25T10:11:01.791702+0000","last_became_active":"2024-04-25T10:10:59.662664+0000","last_became_peered":"2024-04-25T10:10:59.662664+0000","last_unstale":"2024-04-25T10:11:01.791702+0000","last_undegraded":"2024-04-25T10:11:01.791702+0000","last_fullsized":"2024-04-25T10:11:01.791702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:16.256009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":301,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669306+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:11:01.669306+0000","last_peered":"2024-04-25T10:11:01.669306+0000","last_clean":"2024-04-25T10:11:01.669306+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:11:01.669306+0000","last_undegraded":"2024-04-25T10:11:01.669306+0000","last_fullsized":"2024-04-25T10:11:01.669306+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1052672,"data_stored":446360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509516,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1056768,"data_stored":446962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509517,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1052672,"data_stored":446324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335460,"statfs":{"total":96636764160,"available":96599511040,"internally_reserved":0,"allocated":1060864,"data_stored":446964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509518,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335472,"statfs":{"total":96636764160,"available":96599523328,"internally_reserved":0,"allocated":1048576,"data_stored":446290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509516,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1646592,"data_stored":1036748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509518,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1064960,"data_stored":447924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1605632,"data_stored":1017308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:04.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.150+0000 7f120adb7640 1 Processor -- start 2024-04-25T10:11:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.150+0000 7f120adb7640 1 -- start start 2024-04-25T10:11:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.150+0000 7f120adb7640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f12040d35b0 0x7f12040b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.150+0000 7f120adb7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12040b5990 0x7f12040b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.150+0000 7f120adb7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120415e0c0 con 0x7f120415dce0 2024-04-25T10:11:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.150+0000 7f120adb7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12040d2510 con 0x7f12040b5990 2024-04-25T10:11:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.150+0000 7f120adb7640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12040d27f0 con 0x7f12040d35b0 2024-04-25T10:11:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.151+0000 7f1209354640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12040b5990 0x7f12040b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.151+0000 7f1208b53640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f120415dce0 0x7f12040d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55494/0 (socket says 172.21.15.16:55494) 2024-04-25T10:11:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.151+0000 7f1208b53640 1 -- 172.21.15.16:0/2495334411 learned_addr learned my addr 172.21.15.16:0/2495334411 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.151+0000 7f1203fff640 1 --2- 172.21.15.16:0/2495334411 >> v2:172.21.15.186:3300/0 conn(0x7f12040d35b0 0x7f12040b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.151+0000 7f1209354640 1 -- 172.21.15.16:0/2495334411 >> v2:172.21.15.186:3300/0 conn(0x7f12040d35b0 msgr2=0x7f12040b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.151+0000 7f1209354640 1 --2- 172.21.15.16:0/2495334411 >> v2:172.21.15.186:3300/0 conn(0x7f12040d35b0 0x7f12040b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.151+0000 7f1209354640 1 -- 172.21.15.16:0/2495334411 >> v1:172.21.15.16:6789/0 conn(0x7f120415dce0 legacy=0x7f12040d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.151+0000 7f1209354640 1 -- 172.21.15.16:0/2495334411 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12040d2ad0 con 0x7f12040b5990 2024-04-25T10:11:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.151+0000 7f12037fe640 1 -- 172.21.15.16:0/2495334411 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611076053 0 0) 0x7f120415e0c0 con 0x7f120415dce0 2024-04-25T10:11:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.152+0000 7f1209354640 1 --2- 172.21.15.16:0/2495334411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12040b5990 0x7f12040b6b00 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f12040cdfa0 tx=0x7f11f402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1376f3f37f3a81c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.152+0000 7f12037fe640 1 -- 172.21.15.16:0/2495334411 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4037070 con 0x7f12040b5990 2024-04-25T10:11:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.152+0000 7f12037fe640 1 -- 172.21.15.16:0/2495334411 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f4032440 con 0x7f12040b5990 2024-04-25T10:11:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.152+0000 7f12037fe640 1 -- 172.21.15.16:0/2495334411 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4032810 con 0x7f12040b5990 2024-04-25T10:11:04.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.152+0000 7f120adb7640 1 -- 172.21.15.16:0/2495334411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12040b5990 msgr2=0x7f12040b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.152+0000 7f120adb7640 1 --2- 172.21.15.16:0/2495334411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12040b5990 0x7f12040b6b00 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f12040cdfa0 tx=0x7f11f402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.152+0000 7f120adb7640 1 -- 172.21.15.16:0/2495334411 shutdown_connections 2024-04-25T10:11:04.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.152+0000 7f120adb7640 1 --2- 172.21.15.16:0/2495334411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12040b5990 0x7f12040b6b00 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.152+0000 7f120adb7640 1 --2- 172.21.15.16:0/2495334411 >> v2:172.21.15.186:3300/0 conn(0x7f12040d35b0 0x7f12040b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.152+0000 7f120adb7640 1 -- 172.21.15.16:0/2495334411 >> 172.21.15.16:0/2495334411 conn(0x7f1204072900 msgr2=0x7f12040d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.152+0000 7f120adb7640 1 -- 172.21.15.16:0/2495334411 shutdown_connections 2024-04-25T10:11:04.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.152+0000 7f120adb7640 1 -- 172.21.15.16:0/2495334411 wait complete. 2024-04-25T10:11:04.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.154+0000 7f120adb7640 1 Processor -- start 2024-04-25T10:11:04.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.154+0000 7f120adb7640 1 -- start start 2024-04-25T10:11:04.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.154+0000 7f120adb7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12040b5990 0x7f12040b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.154+0000 7f120adb7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12040d35b0 0x7f120415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.154+0000 7f120adb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120415dce0 0x7f12040ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.154+0000 7f120adb7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12040d2160 con 0x7f120415dce0 2024-04-25T10:11:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.154+0000 7f120adb7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12040d3090 con 0x7f12040b5990 2024-04-25T10:11:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.154+0000 7f120adb7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12040aa7e0 con 0x7f12040d35b0 2024-04-25T10:11:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.155+0000 7f1209354640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120415dce0 0x7f12040ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.155+0000 7f1209354640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120415dce0 0x7f12040ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53242/0 (socket says 172.21.15.16:53242) 2024-04-25T10:11:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.155+0000 7f1209354640 1 -- 172.21.15.16:0/684555966 learned_addr learned my addr 172.21.15.16:0/684555966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.155+0000 7f1208b53640 1 --2- 172.21.15.16:0/684555966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12040b5990 0x7f12040b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.155+0000 7f1203fff640 1 --2- 172.21.15.16:0/684555966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12040d35b0 0x7f120415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.155+0000 7f1209354640 1 -- 172.21.15.16:0/684555966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12040d35b0 msgr2=0x7f120415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.155+0000 7f1209354640 1 --2- 172.21.15.16:0/684555966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12040d35b0 0x7f120415ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.155+0000 7f1209354640 1 -- 172.21.15.16:0/684555966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12040b5990 msgr2=0x7f12040b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.155+0000 7f1209354640 1 --2- 172.21.15.16:0/684555966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12040b5990 0x7f12040b1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.155+0000 7f1209354640 1 -- 172.21.15.16:0/684555966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12040aaac0 con 0x7f120415dce0 2024-04-25T10:11:04.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.155+0000 7f1208b53640 1 --2- 172.21.15.16:0/684555966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12040b5990 0x7f12040b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:04.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.155+0000 7f1203fff640 1 --2- 172.21.15.16:0/684555966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12040d35b0 0x7f120415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:04.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.155+0000 7f1209354640 1 --2- 172.21.15.16:0/684555966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120415dce0 0x7f12040ccab0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f11f402a1e0 tx=0x7f11f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.155+0000 7f12017fa640 1 -- 172.21.15.16:0/684555966 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4037040 con 0x7f120415dce0 2024-04-25T10:11:04.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.155+0000 7f12017fa640 1 -- 172.21.15.16:0/684555966 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f4042070 con 0x7f120415dce0 2024-04-25T10:11:04.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.155+0000 7f120adb7640 1 -- 172.21.15.16:0/684555966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12040aada0 con 0x7f120415dce0 2024-04-25T10:11:04.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.155+0000 7f12017fa640 1 -- 172.21.15.16:0/684555966 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4030b50 con 0x7f120415dce0 2024-04-25T10:11:04.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.155+0000 7f120adb7640 1 -- 172.21.15.16:0/684555966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12040ab360 con 0x7f120415dce0 2024-04-25T10:11:04.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.157+0000 7f120adb7640 1 -- 172.21.15.16:0/684555966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12040749b0 con 0x7f120415dce0 2024-04-25T10:11:04.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.157+0000 7f12017fa640 1 -- 172.21.15.16:0/684555966 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11f4030cf0 con 0x7f120415dce0 2024-04-25T10:11:04.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.157+0000 7f12017fa640 1 --2- 172.21.15.16:0/684555966 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f11e406b4d0 0x7f11e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.157+0000 7f12017fa640 1 -- 172.21.15.16:0/684555966 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11f40aad50 con 0x7f120415dce0 2024-04-25T10:11:04.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.158+0000 7f1208b53640 1 --2- 172.21.15.16:0/684555966 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f11e406b4d0 0x7f11e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.158+0000 7f1208b53640 1 --2- 172.21.15.16:0/684555966 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f11e406b4d0 0x7f11e406d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f11fc007fd0 tx=0x7f11fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.161+0000 7f12017fa640 1 -- 172.21.15.16:0/684555966 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11f40ae030 con 0x7f120415dce0 2024-04-25T10:11:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.336+0000 7f120adb7640 1 -- 172.21.15.16:0/684555966 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f12040b1d70 con 0x7f11e406b4d0 2024-04-25T10:11:04.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.343+0000 7f12017fa640 1 -- 172.21.15.16:0/684555966 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251662 (secure 0 0 0) 0x7f12040b1d70 con 0x7f11e406b4d0 2024-04-25T10:11:04.351 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:11:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.346+0000 7f120adb7640 1 -- 172.21.15.16:0/684555966 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f11e406b4d0 msgr2=0x7f11e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.346+0000 7f120adb7640 1 --2- 172.21.15.16:0/684555966 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f11e406b4d0 0x7f11e406d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f11fc007fd0 tx=0x7f11fc009040 comp rx=0 tx=0).stop 2024-04-25T10:11:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.346+0000 7f120adb7640 1 -- 172.21.15.16:0/684555966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120415dce0 msgr2=0x7f12040ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.346+0000 7f120adb7640 1 --2- 172.21.15.16:0/684555966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120415dce0 0x7f12040ccab0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f11f402a1e0 tx=0x7f11f40046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.346+0000 7f120adb7640 1 -- 172.21.15.16:0/684555966 shutdown_connections 2024-04-25T10:11:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.346+0000 7f120adb7640 1 --2- 172.21.15.16:0/684555966 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f11e406b4d0 0x7f11e406d990 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.346+0000 7f120adb7640 1 --2- 172.21.15.16:0/684555966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120415dce0 0x7f12040ccab0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.346+0000 7f120adb7640 1 --2- 172.21.15.16:0/684555966 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12040d35b0 0x7f120415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.346+0000 7f120adb7640 1 --2- 172.21.15.16:0/684555966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12040b5990 0x7f12040b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.346+0000 7f120adb7640 1 -- 172.21.15.16:0/684555966 >> 172.21.15.16:0/684555966 conn(0x7f1204072900 msgr2=0x7f12040b8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.346+0000 7f120adb7640 1 -- 172.21.15.16:0/684555966 shutdown_connections 2024-04-25T10:11:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.346+0000 7f120adb7640 1 -- 172.21.15.16:0/684555966 wait complete. 2024-04-25T10:11:04.362 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-04-25T10:11:03.867060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":198,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":267160,"kb_used_data":9364,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754707560,"statfs":{"total":773094113280,"available":772820541440,"internally_reserved":0,"allocated":9588736,"data_stored":4734880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.226999"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669584+0000","last_change":"2024-04-25T10:11:00.665795+0000","last_active":"2024-04-25T10:11:01.669584+0000","last_peered":"2024-04-25T10:11:01.669584+0000","last_clean":"2024-04-25T10:11:01.669584+0000","last_became_active":"2024-04-25T10:11:00.664672+0000","last_became_peered":"2024-04-25T10:11:00.664672+0000","last_unstale":"2024-04-25T10:11:01.669584+0000","last_undegraded":"2024-04-25T10:11:01.669584+0000","last_fullsized":"2024-04-25T10:11:01.669584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:30.244884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672240+0000","last_change":"2024-04-25T10:11:00.658574+0000","last_active":"2024-04-25T10:11:01.672240+0000","last_peered":"2024-04-25T10:11:01.672240+0000","last_clean":"2024-04-25T10:11:01.672240+0000","last_became_active":"2024-04-25T10:11:00.658280+0000","last_became_peered":"2024-04-25T10:11:00.658280+0000","last_unstale":"2024-04-25T10:11:01.672240+0000","last_undegraded":"2024-04-25T10:11:01.672240+0000","last_fullsized":"2024-04-25T10:11:01.672240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:38.681757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666472+0000","last_change":"2024-04-25T10:11:00.664492+0000","last_active":"2024-04-25T10:11:01.666472+0000","last_peered":"2024-04-25T10:11:01.666472+0000","last_clean":"2024-04-25T10:11:01.666472+0000","last_became_active":"2024-04-25T10:11:00.664205+0000","last_became_peered":"2024-04-25T10:11:00.664205+0000","last_unstale":"2024-04-25T10:11:01.666472+0000","last_undegraded":"2024-04-25T10:11:01.666472+0000","last_fullsized":"2024-04-25T10:11:01.666472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.860030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.648641+0000","last_change":"2024-04-25T10:10:59.648641+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.648641+0000","last_undegraded":"2024-04-25T10:10:59.648641+0000","last_fullsized":"2024-04-25T10:10:59.648641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791419+0000","last_change":"2024-04-25T10:11:00.674341+0000","last_active":"2024-04-25T10:11:01.791419+0000","last_peered":"2024-04-25T10:11:01.791419+0000","last_clean":"2024-04-25T10:11:01.791419+0000","last_became_active":"2024-04-25T10:11:00.674016+0000","last_became_peered":"2024-04-25T10:11:00.674016+0000","last_unstale":"2024-04-25T10:11:01.791419+0000","last_undegraded":"2024-04-25T10:11:01.791419+0000","last_fullsized":"2024-04-25T10:11:01.791419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:17.318511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666458+0000","last_change":"2024-04-25T10:11:00.666821+0000","last_active":"2024-04-25T10:11:01.666458+0000","last_peered":"2024-04-25T10:11:01.666458+0000","last_clean":"2024-04-25T10:11:01.666458+0000","last_became_active":"2024-04-25T10:11:00.666552+0000","last_became_peered":"2024-04-25T10:11:00.666552+0000","last_unstale":"2024-04-25T10:11:01.666458+0000","last_undegraded":"2024-04-25T10:11:01.666458+0000","last_fullsized":"2024-04-25T10:11:01.666458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:43.541677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.671974+0000","last_change":"2024-04-25T10:11:00.666060+0000","last_active":"2024-04-25T10:11:01.671974+0000","last_peered":"2024-04-25T10:11:01.671974+0000","last_clean":"2024-04-25T10:11:01.671974+0000","last_became_active":"2024-04-25T10:11:00.665760+0000","last_became_peered":"2024-04-25T10:11:00.665760+0000","last_unstale":"2024-04-25T10:11:01.671974+0000","last_undegraded":"2024-04-25T10:11:01.671974+0000","last_fullsized":"2024-04-25T10:11:01.671974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:56.991014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790545+0000","last_change":"2024-04-25T10:11:00.672928+0000","last_active":"2024-04-25T10:11:01.790545+0000","last_peered":"2024-04-25T10:11:01.790545+0000","last_clean":"2024-04-25T10:11:01.790545+0000","last_became_active":"2024-04-25T10:11:00.672604+0000","last_became_peered":"2024-04-25T10:11:00.672604+0000","last_unstale":"2024-04-25T10:11:01.790545+0000","last_undegraded":"2024-04-25T10:11:01.790545+0000","last_fullsized":"2024-04-25T10:11:01.790545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:43.541677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669584+0000","last_change":"2024-04-25T10:11:00.665452+0000","last_active":"2024-04-25T10:11:01.669584+0000","last_peered":"2024-04-25T10:11:01.669584+0000","last_clean":"2024-04-25T10:11:01.669584+0000","last_became_active":"2024-04-25T10:11:00.665186+0000","last_became_peered":"2024-04-25T10:11:00.665186+0000","last_unstale":"2024-04-25T10:11:01.669584+0000","last_undegraded":"2024-04-25T10:11:01.669584+0000","last_fullsized":"2024-04-25T10:11:01.669584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:13.745574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666508+0000","last_change":"2024-04-25T10:11:00.657395+0000","last_active":"2024-04-25T10:11:01.666508+0000","last_peered":"2024-04-25T10:11:01.666508+0000","last_clean":"2024-04-25T10:11:01.666508+0000","last_became_active":"2024-04-25T10:11:00.657121+0000","last_became_peered":"2024-04-25T10:11:00.657121+0000","last_unstale":"2024-04-25T10:11:01.666508+0000","last_undegraded":"2024-04-25T10:11:01.666508+0000","last_fullsized":"2024-04-25T10:11:01.666508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:28.012019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669989+0000","last_change":"2024-04-25T10:11:00.674199+0000","last_active":"2024-04-25T10:11:01.669989+0000","last_peered":"2024-04-25T10:11:01.669989+0000","last_clean":"2024-04-25T10:11:01.669989+0000","last_became_active":"2024-04-25T10:11:00.673935+0000","last_became_peered":"2024-04-25T10:11:00.673935+0000","last_unstale":"2024-04-25T10:11:01.669989+0000","last_undegraded":"2024-04-25T10:11:01.669989+0000","last_fullsized":"2024-04-25T10:11:01.669989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:35.284964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790779+0000","last_change":"2024-04-25T10:11:00.675112+0000","last_active":"2024-04-25T10:11:01.790779+0000","last_peered":"2024-04-25T10:11:01.790779+0000","last_clean":"2024-04-25T10:11:01.790779+0000","last_became_active":"2024-04-25T10:11:00.674812+0000","last_became_peered":"2024-04-25T10:11:00.674812+0000","last_unstale":"2024-04-25T10:11:01.790779+0000","last_undegraded":"2024-04-25T10:11:01.790779+0000","last_fullsized":"2024-04-25T10:11:01.790779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:56.966365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791569+0000","last_change":"2024-04-25T10:11:00.660277+0000","last_active":"2024-04-25T10:11:01.791569+0000","last_peered":"2024-04-25T10:11:01.791569+0000","last_clean":"2024-04-25T10:11:01.791569+0000","last_became_active":"2024-04-25T10:11:00.659982+0000","last_became_peered":"2024-04-25T10:11:00.659982+0000","last_unstale":"2024-04-25T10:11:01.791569+0000","last_undegraded":"2024-04-25T10:11:01.791569+0000","last_fullsized":"2024-04-25T10:11:01.791569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:26.638020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791242+0000","last_change":"2024-04-25T10:11:00.672864+0000","last_active":"2024-04-25T10:11:01.791242+0000","last_peered":"2024-04-25T10:11:01.791242+0000","last_clean":"2024-04-25T10:11:01.791242+0000","last_became_active":"2024-04-25T10:11:00.672565+0000","last_became_peered":"2024-04-25T10:11:00.672565+0000","last_unstale":"2024-04-25T10:11:01.791242+0000","last_undegraded":"2024-04-25T10:11:01.791242+0000","last_fullsized":"2024-04-25T10:11:01.791242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:38.932561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.671980+0000","last_change":"2024-04-25T10:11:00.675075+0000","last_active":"2024-04-25T10:11:01.671980+0000","last_peered":"2024-04-25T10:11:01.671980+0000","last_clean":"2024-04-25T10:11:01.671980+0000","last_became_active":"2024-04-25T10:11:00.674791+0000","last_became_peered":"2024-04-25T10:11:00.674791+0000","last_unstale":"2024-04-25T10:11:01.671980+0000","last_undegraded":"2024-04-25T10:11:01.671980+0000","last_fullsized":"2024-04-25T10:11:01.671980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:14.897123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666210+0000","last_change":"2024-04-25T10:11:00.658578+0000","last_active":"2024-04-25T10:11:01.666210+0000","last_peered":"2024-04-25T10:11:01.666210+0000","last_clean":"2024-04-25T10:11:01.666210+0000","last_became_active":"2024-04-25T10:11:00.658309+0000","last_became_peered":"2024-04-25T10:11:00.658309+0000","last_unstale":"2024-04-25T10:11:01.666210+0000","last_undegraded":"2024-04-25T10:11:01.666210+0000","last_fullsized":"2024-04-25T10:11:01.666210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:40.828899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790928+0000","last_change":"2024-04-25T10:11:00.674321+0000","last_active":"2024-04-25T10:11:01.790928+0000","last_peered":"2024-04-25T10:11:01.790928+0000","last_clean":"2024-04-25T10:11:01.790928+0000","last_became_active":"2024-04-25T10:11:00.674037+0000","last_became_peered":"2024-04-25T10:11:00.674037+0000","last_unstale":"2024-04-25T10:11:01.790928+0000","last_undegraded":"2024-04-25T10:11:01.790928+0000","last_fullsized":"2024-04-25T10:11:01.790928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:37.420962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669990+0000","last_change":"2024-04-25T10:11:00.673650+0000","last_active":"2024-04-25T10:11:01.669990+0000","last_peered":"2024-04-25T10:11:01.669990+0000","last_clean":"2024-04-25T10:11:01.669990+0000","last_became_active":"2024-04-25T10:11:00.673318+0000","last_became_peered":"2024-04-25T10:11:00.673318+0000","last_unstale":"2024-04-25T10:11:01.669990+0000","last_undegraded":"2024-04-25T10:11:01.669990+0000","last_fullsized":"2024-04-25T10:11:01.669990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:18.528132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791780+0000","last_change":"2024-04-25T10:11:00.672749+0000","last_active":"2024-04-25T10:11:01.791780+0000","last_peered":"2024-04-25T10:11:01.791780+0000","last_clean":"2024-04-25T10:11:01.791780+0000","last_became_active":"2024-04-25T10:11:00.671982+0000","last_became_peered":"2024-04-25T10:11:00.671982+0000","last_unstale":"2024-04-25T10:11:01.791780+0000","last_undegraded":"2024-04-25T10:11:01.791780+0000","last_fullsized":"2024-04-25T10:11:01.791780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.235679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791396+0000","last_change":"2024-04-25T10:11:00.674897+0000","last_active":"2024-04-25T10:11:01.791396+0000","last_peered":"2024-04-25T10:11:01.791396+0000","last_clean":"2024-04-25T10:11:01.791396+0000","last_became_active":"2024-04-25T10:11:00.674582+0000","last_became_peered":"2024-04-25T10:11:00.674582+0000","last_unstale":"2024-04-25T10:11:01.791396+0000","last_undegraded":"2024-04-25T10:11:01.791396+0000","last_fullsized":"2024-04-25T10:11:01.791396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.603074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.651673+0000","last_change":"2024-04-25T10:10:59.651673+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.651673+0000","last_undegraded":"2024-04-25T10:10:59.651673+0000","last_fullsized":"2024-04-25T10:10:59.651673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791410+0000","last_change":"2024-04-25T10:11:00.670165+0000","last_active":"2024-04-25T10:11:01.791410+0000","last_peered":"2024-04-25T10:11:01.791410+0000","last_clean":"2024-04-25T10:11:01.791410+0000","last_became_active":"2024-04-25T10:11:00.669866+0000","last_became_peered":"2024-04-25T10:11:00.669866+0000","last_unstale":"2024-04-25T10:11:01.791410+0000","last_undegraded":"2024-04-25T10:11:01.791410+0000","last_fullsized":"2024-04-25T10:11:01.791410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:43.529739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790610+0000","last_change":"2024-04-25T10:11:00.674422+0000","last_active":"2024-04-25T10:11:01.790610+0000","last_peered":"2024-04-25T10:11:01.790610+0000","last_clean":"2024-04-25T10:11:01.790610+0000","last_became_active":"2024-04-25T10:11:00.674142+0000","last_became_peered":"2024-04-25T10:11:00.674142+0000","last_unstale":"2024-04-25T10:11:01.790610+0000","last_undegraded":"2024-04-25T10:11:01.790610+0000","last_fullsized":"2024-04-25T10:11:01.790610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:28.012019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669720+0000","last_change":"2024-04-25T10:11:00.672113+0000","last_active":"2024-04-25T10:11:01.669720+0000","last_peered":"2024-04-25T10:11:01.669720+0000","last_clean":"2024-04-25T10:11:01.669720+0000","last_became_active":"2024-04-25T10:11:00.671835+0000","last_became_peered":"2024-04-25T10:11:00.671835+0000","last_unstale":"2024-04-25T10:11:01.669720+0000","last_undegraded":"2024-04-25T10:11:01.669720+0000","last_fullsized":"2024-04-25T10:11:01.669720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:56.991014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.670008+0000","last_change":"2024-04-25T10:11:00.679427+0000","last_active":"2024-04-25T10:11:01.670008+0000","last_peered":"2024-04-25T10:11:01.670008+0000","last_clean":"2024-04-25T10:11:01.670008+0000","last_became_active":"2024-04-25T10:11:00.678268+0000","last_became_peered":"2024-04-25T10:11:00.678268+0000","last_unstale":"2024-04-25T10:11:01.670008+0000","last_undegraded":"2024-04-25T10:11:01.670008+0000","last_fullsized":"2024-04-25T10:11:01.670008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:57.428737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791705+0000","last_change":"2024-04-25T10:11:00.671179+0000","last_active":"2024-04-25T10:11:01.791705+0000","last_peered":"2024-04-25T10:11:01.791705+0000","last_clean":"2024-04-25T10:11:01.791705+0000","last_became_active":"2024-04-25T10:11:00.670882+0000","last_became_peered":"2024-04-25T10:11:00.670882+0000","last_unstale":"2024-04-25T10:11:01.791705+0000","last_undegraded":"2024-04-25T10:11:01.791705+0000","last_fullsized":"2024-04-25T10:11:01.791705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:22.210854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791595+0000","last_change":"2024-04-25T10:11:00.672506+0000","last_active":"2024-04-25T10:11:01.791595+0000","last_peered":"2024-04-25T10:11:01.791595+0000","last_clean":"2024-04-25T10:11:01.791595+0000","last_became_active":"2024-04-25T10:11:00.672018+0000","last_became_peered":"2024-04-25T10:11:00.672018+0000","last_unstale":"2024-04-25T10:11:01.791595+0000","last_undegraded":"2024-04-25T10:11:01.791595+0000","last_fullsized":"2024-04-25T10:11:01.791595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:14.645638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669413+0000","last_change":"2024-04-25T10:11:00.668016+0000","last_active":"2024-04-25T10:11:01.669413+0000","last_peered":"2024-04-25T10:11:01.669413+0000","last_clean":"2024-04-25T10:11:01.669413+0000","last_became_active":"2024-04-25T10:11:00.667742+0000","last_became_peered":"2024-04-25T10:11:00.667742+0000","last_unstale":"2024-04-25T10:11:01.669413+0000","last_undegraded":"2024-04-25T10:11:01.669413+0000","last_fullsized":"2024-04-25T10:11:01.669413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:35.210017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.651157+0000","last_change":"2024-04-25T10:10:59.651157+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.651157+0000","last_undegraded":"2024-04-25T10:10:59.651157+0000","last_fullsized":"2024-04-25T10:10:59.651157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791586+0000","last_change":"2024-04-25T10:11:00.672908+0000","last_active":"2024-04-25T10:11:01.791586+0000","last_peered":"2024-04-25T10:11:01.791586+0000","last_clean":"2024-04-25T10:11:01.791586+0000","last_became_active":"2024-04-25T10:11:00.672634+0000","last_became_peered":"2024-04-25T10:11:00.672634+0000","last_unstale":"2024-04-25T10:11:01.791586+0000","last_undegraded":"2024-04-25T10:11:01.791586+0000","last_fullsized":"2024-04-25T10:11:01.791586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:37.071670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666210+0000","last_change":"2024-04-25T10:11:00.667457+0000","last_active":"2024-04-25T10:11:01.666210+0000","last_peered":"2024-04-25T10:11:01.666210+0000","last_clean":"2024-04-25T10:11:01.666210+0000","last_became_active":"2024-04-25T10:11:00.667205+0000","last_became_peered":"2024-04-25T10:11:00.667205+0000","last_unstale":"2024-04-25T10:11:01.666210+0000","last_undegraded":"2024-04-25T10:11:01.666210+0000","last_fullsized":"2024-04-25T10:11:01.666210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:07.289251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666783+0000","last_change":"2024-04-25T10:11:00.668155+0000","last_active":"2024-04-25T10:11:01.666783+0000","last_peered":"2024-04-25T10:11:01.666783+0000","last_clean":"2024-04-25T10:11:01.666783+0000","last_became_active":"2024-04-25T10:11:00.667848+0000","last_became_peered":"2024-04-25T10:11:00.667848+0000","last_unstale":"2024-04-25T10:11:01.666783+0000","last_undegraded":"2024-04-25T10:11:01.666783+0000","last_fullsized":"2024-04-25T10:11:01.666783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:55.212072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790745+0000","last_change":"2024-04-25T10:11:00.664212+0000","last_active":"2024-04-25T10:11:01.790745+0000","last_peered":"2024-04-25T10:11:01.790745+0000","last_clean":"2024-04-25T10:11:01.790745+0000","last_became_active":"2024-04-25T10:11:00.663662+0000","last_became_peered":"2024-04-25T10:11:00.663662+0000","last_unstale":"2024-04-25T10:11:01.790745+0000","last_undegraded":"2024-04-25T10:11:01.790745+0000","last_fullsized":"2024-04-25T10:11:01.790745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:14.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"85'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791831+0000","last_change":"2024-04-25T10:10:59.661968+0000","last_active":"2024-04-25T10:11:01.791831+0000","last_peered":"2024-04-25T10:11:01.791831+0000","last_clean":"2024-04-25T10:11:01.791831+0000","last_became_active":"2024-04-25T10:10:59.661557+0000","last_became_peered":"2024-04-25T10:10:59.661557+0000","last_unstale":"2024-04-25T10:11:01.791831+0000","last_undegraded":"2024-04-25T10:11:01.791831+0000","last_fullsized":"2024-04-25T10:11:01.791831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:29.166481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669745+0000","last_change":"2024-04-25T10:11:00.664765+0000","last_active":"2024-04-25T10:11:01.669745+0000","last_peered":"2024-04-25T10:11:01.669745+0000","last_clean":"2024-04-25T10:11:01.669745+0000","last_became_active":"2024-04-25T10:11:00.664306+0000","last_became_peered":"2024-04-25T10:11:00.664306+0000","last_unstale":"2024-04-25T10:11:01.669745+0000","last_undegraded":"2024-04-25T10:11:01.669745+0000","last_fullsized":"2024-04-25T10:11:01.669745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:51.779177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666622+0000","last_change":"2024-04-25T10:10:59.659985+0000","last_active":"2024-04-25T10:11:01.666622+0000","last_peered":"2024-04-25T10:11:01.666622+0000","last_clean":"2024-04-25T10:11:01.666622+0000","last_became_active":"2024-04-25T10:10:59.659641+0000","last_became_peered":"2024-04-25T10:10:59.659641+0000","last_unstale":"2024-04-25T10:11:01.666622+0000","last_undegraded":"2024-04-25T10:11:01.666622+0000","last_fullsized":"2024-04-25T10:11:01.666622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:35.098559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790943+0000","last_change":"2024-04-25T10:11:00.666415+0000","last_active":"2024-04-25T10:11:01.790943+0000","last_peered":"2024-04-25T10:11:01.790943+0000","last_clean":"2024-04-25T10:11:01.790943+0000","last_became_active":"2024-04-25T10:11:00.666151+0000","last_became_peered":"2024-04-25T10:11:00.666151+0000","last_unstale":"2024-04-25T10:11:01.790943+0000","last_undegraded":"2024-04-25T10:11:01.790943+0000","last_fullsized":"2024-04-25T10:11:01.790943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:43.188026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669929+0000","last_change":"2024-04-25T10:10:59.654631+0000","last_active":"2024-04-25T10:11:01.669929+0000","last_peered":"2024-04-25T10:11:01.669929+0000","last_clean":"2024-04-25T10:11:01.669929+0000","last_became_active":"2024-04-25T10:10:59.654288+0000","last_became_peered":"2024-04-25T10:10:59.654288+0000","last_unstale":"2024-04-25T10:11:01.669929+0000","last_undegraded":"2024-04-25T10:11:01.669929+0000","last_fullsized":"2024-04-25T10:11:01.669929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:34.205125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669497+0000","last_change":"2024-04-25T10:11:00.677043+0000","last_active":"2024-04-25T10:11:01.669497+0000","last_peered":"2024-04-25T10:11:01.669497+0000","last_clean":"2024-04-25T10:11:01.669497+0000","last_became_active":"2024-04-25T10:11:00.675514+0000","last_became_peered":"2024-04-25T10:11:00.675514+0000","last_unstale":"2024-04-25T10:11:01.669497+0000","last_undegraded":"2024-04-25T10:11:01.669497+0000","last_fullsized":"2024-04-25T10:11:01.669497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:46.822338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T10:10:59.664299+0000","last_change":"2024-04-25T10:10:59.658687+0000","last_active":"2024-04-25T10:10:59.664299+0000","last_peered":"2024-04-25T10:10:59.664299+0000","last_clean":"2024-04-25T10:10:59.664299+0000","last_became_active":"2024-04-25T10:10:59.658380+0000","last_became_peered":"2024-04-25T10:10:59.658380+0000","last_unstale":"2024-04-25T10:10:59.664299+0000","last_undegraded":"2024-04-25T10:10:59.664299+0000","last_fullsized":"2024-04-25T10:10:59.664299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:14.321738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.652342+0000","last_change":"2024-04-25T10:10:59.652342+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.652342+0000","last_undegraded":"2024-04-25T10:10:59.652342+0000","last_fullsized":"2024-04-25T10:10:59.652342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790565+0000","last_change":"2024-04-25T10:10:59.658839+0000","last_active":"2024-04-25T10:11:01.790565+0000","last_peered":"2024-04-25T10:11:01.790565+0000","last_clean":"2024-04-25T10:11:01.790565+0000","last_became_active":"2024-04-25T10:10:59.658557+0000","last_became_peered":"2024-04-25T10:10:59.658557+0000","last_unstale":"2024-04-25T10:11:01.790565+0000","last_undegraded":"2024-04-25T10:11:01.790565+0000","last_fullsized":"2024-04-25T10:11:01.790565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:35.098559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666691+0000","last_change":"2024-04-25T10:11:00.670164+0000","last_active":"2024-04-25T10:11:01.666691+0000","last_peered":"2024-04-25T10:11:01.666691+0000","last_clean":"2024-04-25T10:11:01.666691+0000","last_became_active":"2024-04-25T10:11:00.669856+0000","last_became_peered":"2024-04-25T10:11:00.669856+0000","last_unstale":"2024-04-25T10:11:01.666691+0000","last_undegraded":"2024-04-25T10:11:01.666691+0000","last_fullsized":"2024-04-25T10:11:01.666691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:56.966365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666588+0000","last_change":"2024-04-25T10:10:59.660519+0000","last_active":"2024-04-25T10:11:01.666588+0000","last_peered":"2024-04-25T10:11:01.666588+0000","last_clean":"2024-04-25T10:11:01.666588+0000","last_became_active":"2024-04-25T10:10:59.659571+0000","last_became_peered":"2024-04-25T10:10:59.659571+0000","last_unstale":"2024-04-25T10:11:01.666588+0000","last_undegraded":"2024-04-25T10:11:01.666588+0000","last_fullsized":"2024-04-25T10:11:01.666588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:47.638024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669228+0000","last_change":"2024-04-25T10:11:00.672808+0000","last_active":"2024-04-25T10:11:01.669228+0000","last_peered":"2024-04-25T10:11:01.669228+0000","last_clean":"2024-04-25T10:11:01.669228+0000","last_became_active":"2024-04-25T10:11:00.672479+0000","last_became_peered":"2024-04-25T10:11:00.672479+0000","last_unstale":"2024-04-25T10:11:01.669228+0000","last_undegraded":"2024-04-25T10:11:01.669228+0000","last_fullsized":"2024-04-25T10:11:01.669228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:43.529739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791149+0000","last_change":"2024-04-25T10:10:59.662636+0000","last_active":"2024-04-25T10:11:01.791149+0000","last_peered":"2024-04-25T10:11:01.791149+0000","last_clean":"2024-04-25T10:11:01.791149+0000","last_became_active":"2024-04-25T10:10:59.661790+0000","last_became_peered":"2024-04-25T10:10:59.661790+0000","last_unstale":"2024-04-25T10:11:01.791149+0000","last_undegraded":"2024-04-25T10:11:01.791149+0000","last_fullsized":"2024-04-25T10:11:01.791149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:10.609819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791191+0000","last_change":"2024-04-25T10:11:00.671182+0000","last_active":"2024-04-25T10:11:01.791191+0000","last_peered":"2024-04-25T10:11:01.791191+0000","last_clean":"2024-04-25T10:11:01.791191+0000","last_became_active":"2024-04-25T10:11:00.670863+0000","last_became_peered":"2024-04-25T10:11:00.670863+0000","last_unstale":"2024-04-25T10:11:01.791191+0000","last_undegraded":"2024-04-25T10:11:01.791191+0000","last_fullsized":"2024-04-25T10:11:01.791191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:21.622898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791076+0000","last_change":"2024-04-25T10:10:59.661931+0000","last_active":"2024-04-25T10:11:01.791076+0000","last_peered":"2024-04-25T10:11:01.791076+0000","last_clean":"2024-04-25T10:11:01.791076+0000","last_became_active":"2024-04-25T10:10:59.661173+0000","last_became_peered":"2024-04-25T10:10:59.661173+0000","last_unstale":"2024-04-25T10:11:01.791076+0000","last_undegraded":"2024-04-25T10:11:01.791076+0000","last_fullsized":"2024-04-25T10:11:01.791076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:35.826027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791331+0000","last_change":"2024-04-25T10:11:00.676424+0000","last_active":"2024-04-25T10:11:01.791331+0000","last_peered":"2024-04-25T10:11:01.791331+0000","last_clean":"2024-04-25T10:11:01.791331+0000","last_became_active":"2024-04-25T10:11:00.676088+0000","last_became_peered":"2024-04-25T10:11:00.676088+0000","last_unstale":"2024-04-25T10:11:01.791331+0000","last_undegraded":"2024-04-25T10:11:01.791331+0000","last_fullsized":"2024-04-25T10:11:01.791331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:50.887593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669814+0000","last_change":"2024-04-25T10:10:59.660535+0000","last_active":"2024-04-25T10:11:01.669814+0000","last_peered":"2024-04-25T10:11:01.669814+0000","last_clean":"2024-04-25T10:11:01.669814+0000","last_became_active":"2024-04-25T10:10:59.660260+0000","last_became_peered":"2024-04-25T10:10:59.660260+0000","last_unstale":"2024-04-25T10:11:01.669814+0000","last_undegraded":"2024-04-25T10:11:01.669814+0000","last_fullsized":"2024-04-25T10:11:01.669814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:23.530134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669744+0000","last_change":"2024-04-25T10:11:00.684312+0000","last_active":"2024-04-25T10:11:01.669744+0000","last_peered":"2024-04-25T10:11:01.669744+0000","last_clean":"2024-04-25T10:11:01.669744+0000","last_became_active":"2024-04-25T10:11:00.683971+0000","last_became_peered":"2024-04-25T10:11:00.683971+0000","last_unstale":"2024-04-25T10:11:01.669744+0000","last_undegraded":"2024-04-25T10:11:01.669744+0000","last_fullsized":"2024-04-25T10:11:01.669744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:40.703987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666638+0000","last_change":"2024-04-25T10:10:59.658424+0000","last_active":"2024-04-25T10:11:01.666638+0000","last_peered":"2024-04-25T10:11:01.666638+0000","last_clean":"2024-04-25T10:11:01.666638+0000","last_became_active":"2024-04-25T10:10:59.658128+0000","last_became_peered":"2024-04-25T10:10:59.658128+0000","last_unstale":"2024-04-25T10:11:01.666638+0000","last_undegraded":"2024-04-25T10:11:01.666638+0000","last_fullsized":"2024-04-25T10:11:01.666638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:42.113345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669659+0000","last_change":"2024-04-25T10:11:00.678536+0000","last_active":"2024-04-25T10:11:01.669659+0000","last_peered":"2024-04-25T10:11:01.669659+0000","last_clean":"2024-04-25T10:11:01.669659+0000","last_became_active":"2024-04-25T10:11:00.678276+0000","last_became_peered":"2024-04-25T10:11:00.678276+0000","last_unstale":"2024-04-25T10:11:01.669659+0000","last_undegraded":"2024-04-25T10:11:01.669659+0000","last_fullsized":"2024-04-25T10:11:01.669659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:48.642917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"85'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669547+0000","last_change":"2024-04-25T10:10:59.653278+0000","last_active":"2024-04-25T10:11:01.669547+0000","last_peered":"2024-04-25T10:11:01.669547+0000","last_clean":"2024-04-25T10:11:01.669547+0000","last_became_active":"2024-04-25T10:10:59.652974+0000","last_became_peered":"2024-04-25T10:10:59.652974+0000","last_unstale":"2024-04-25T10:11:01.669547+0000","last_undegraded":"2024-04-25T10:11:01.669547+0000","last_fullsized":"2024-04-25T10:11:01.669547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:45.820044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791449+0000","last_change":"2024-04-25T10:11:00.684587+0000","last_active":"2024-04-25T10:11:01.791449+0000","last_peered":"2024-04-25T10:11:01.791449+0000","last_clean":"2024-04-25T10:11:01.791449+0000","last_became_active":"2024-04-25T10:11:00.684306+0000","last_became_peered":"2024-04-25T10:11:00.684306+0000","last_unstale":"2024-04-25T10:11:01.791449+0000","last_undegraded":"2024-04-25T10:11:01.791449+0000","last_fullsized":"2024-04-25T10:11:01.791449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:39.955023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"85'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790926+0000","last_change":"2024-04-25T10:10:59.662733+0000","last_active":"2024-04-25T10:11:01.790926+0000","last_peered":"2024-04-25T10:11:01.790926+0000","last_clean":"2024-04-25T10:11:01.790926+0000","last_became_active":"2024-04-25T10:10:59.662290+0000","last_became_peered":"2024-04-25T10:10:59.662290+0000","last_unstale":"2024-04-25T10:11:01.790926+0000","last_undegraded":"2024-04-25T10:11:01.790926+0000","last_fullsized":"2024-04-25T10:11:01.790926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:34.280071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669718+0000","last_change":"2024-04-25T10:11:00.669181+0000","last_active":"2024-04-25T10:11:01.669718+0000","last_peered":"2024-04-25T10:11:01.669718+0000","last_clean":"2024-04-25T10:11:01.669718+0000","last_became_active":"2024-04-25T10:11:00.668758+0000","last_became_peered":"2024-04-25T10:11:00.668758+0000","last_unstale":"2024-04-25T10:11:01.669718+0000","last_undegraded":"2024-04-25T10:11:01.669718+0000","last_fullsized":"2024-04-25T10:11:01.669718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:38.681757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T10:10:59.666667+0000","last_change":"2024-04-25T10:10:59.662230+0000","last_active":"2024-04-25T10:10:59.666667+0000","last_peered":"2024-04-25T10:10:59.666667+0000","last_clean":"2024-04-25T10:10:59.666667+0000","last_became_active":"2024-04-25T10:10:59.661953+0000","last_became_peered":"2024-04-25T10:10:59.661953+0000","last_unstale":"2024-04-25T10:10:59.666667+0000","last_undegraded":"2024-04-25T10:10:59.666667+0000","last_fullsized":"2024-04-25T10:10:59.666667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:31.766942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669387+0000","last_change":"2024-04-25T10:11:00.680010+0000","last_active":"2024-04-25T10:11:01.669387+0000","last_peered":"2024-04-25T10:11:01.669387+0000","last_clean":"2024-04-25T10:11:01.669387+0000","last_became_active":"2024-04-25T10:11:00.679712+0000","last_became_peered":"2024-04-25T10:11:00.679712+0000","last_unstale":"2024-04-25T10:11:01.669387+0000","last_undegraded":"2024-04-25T10:11:01.669387+0000","last_fullsized":"2024-04-25T10:11:01.669387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:22.568376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672076+0000","last_change":"2024-04-25T10:10:59.659677+0000","last_active":"2024-04-25T10:11:01.672076+0000","last_peered":"2024-04-25T10:11:01.672076+0000","last_clean":"2024-04-25T10:11:01.672076+0000","last_became_active":"2024-04-25T10:10:59.659395+0000","last_became_peered":"2024-04-25T10:10:59.659395+0000","last_unstale":"2024-04-25T10:11:01.672076+0000","last_undegraded":"2024-04-25T10:11:01.672076+0000","last_fullsized":"2024-04-25T10:11:01.672076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:21.670434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790957+0000","last_change":"2024-04-25T10:11:00.678832+0000","last_active":"2024-04-25T10:11:01.790957+0000","last_peered":"2024-04-25T10:11:01.790957+0000","last_clean":"2024-04-25T10:11:01.790957+0000","last_became_active":"2024-04-25T10:11:00.678560+0000","last_became_peered":"2024-04-25T10:11:00.678560+0000","last_unstale":"2024-04-25T10:11:01.790957+0000","last_undegraded":"2024-04-25T10:11:01.790957+0000","last_fullsized":"2024-04-25T10:11:01.790957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:19.719591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666238+0000","last_change":"2024-04-25T10:10:59.656540+0000","last_active":"2024-04-25T10:11:01.666238+0000","last_peered":"2024-04-25T10:11:01.666238+0000","last_clean":"2024-04-25T10:11:01.666238+0000","last_became_active":"2024-04-25T10:10:59.656276+0000","last_became_peered":"2024-04-25T10:10:59.656276+0000","last_unstale":"2024-04-25T10:11:01.666238+0000","last_undegraded":"2024-04-25T10:11:01.666238+0000","last_fullsized":"2024-04-25T10:11:01.666238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:13.892230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669548+0000","last_change":"2024-04-25T10:11:00.680212+0000","last_active":"2024-04-25T10:11:01.669548+0000","last_peered":"2024-04-25T10:11:01.669548+0000","last_clean":"2024-04-25T10:11:01.669548+0000","last_became_active":"2024-04-25T10:11:00.679944+0000","last_became_peered":"2024-04-25T10:11:00.679944+0000","last_unstale":"2024-04-25T10:11:01.669548+0000","last_undegraded":"2024-04-25T10:11:01.669548+0000","last_fullsized":"2024-04-25T10:11:01.669548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:14.897123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"85'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669582+0000","last_change":"2024-04-25T10:10:59.656545+0000","last_active":"2024-04-25T10:11:01.669582+0000","last_peered":"2024-04-25T10:11:01.669582+0000","last_clean":"2024-04-25T10:11:01.669582+0000","last_became_active":"2024-04-25T10:10:59.656262+0000","last_became_peered":"2024-04-25T10:10:59.656262+0000","last_unstale":"2024-04-25T10:11:01.669582+0000","last_undegraded":"2024-04-25T10:11:01.669582+0000","last_fullsized":"2024-04-25T10:11:01.669582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:12.740681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790809+0000","last_change":"2024-04-25T10:11:00.660578+0000","last_active":"2024-04-25T10:11:01.790809+0000","last_peered":"2024-04-25T10:11:01.790809+0000","last_clean":"2024-04-25T10:11:01.790809+0000","last_became_active":"2024-04-25T10:11:00.660258+0000","last_became_peered":"2024-04-25T10:11:00.660258+0000","last_unstale":"2024-04-25T10:11:01.790809+0000","last_undegraded":"2024-04-25T10:11:01.790809+0000","last_fullsized":"2024-04-25T10:11:01.790809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:20.151958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791898+0000","last_change":"2024-04-25T10:10:59.660067+0000","last_active":"2024-04-25T10:11:01.791898+0000","last_peered":"2024-04-25T10:11:01.791898+0000","last_clean":"2024-04-25T10:11:01.791898+0000","last_became_active":"2024-04-25T10:10:59.659717+0000","last_became_peered":"2024-04-25T10:10:59.659717+0000","last_unstale":"2024-04-25T10:11:01.791898+0000","last_undegraded":"2024-04-25T10:11:01.791898+0000","last_fullsized":"2024-04-25T10:11:01.791898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:55.225255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666704+0000","last_change":"2024-04-25T10:11:00.664267+0000","last_active":"2024-04-25T10:11:01.666704+0000","last_peered":"2024-04-25T10:11:01.666704+0000","last_clean":"2024-04-25T10:11:01.666704+0000","last_became_active":"2024-04-25T10:11:00.664038+0000","last_became_peered":"2024-04-25T10:11:00.664038+0000","last_unstale":"2024-04-25T10:11:01.666704+0000","last_undegraded":"2024-04-25T10:11:01.666704+0000","last_fullsized":"2024-04-25T10:11:01.666704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:20.151958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791555+0000","last_change":"2024-04-25T10:10:59.661418+0000","last_active":"2024-04-25T10:11:01.791555+0000","last_peered":"2024-04-25T10:11:01.791555+0000","last_clean":"2024-04-25T10:11:01.791555+0000","last_became_active":"2024-04-25T10:10:59.661121+0000","last_became_peered":"2024-04-25T10:10:59.661121+0000","last_unstale":"2024-04-25T10:11:01.791555+0000","last_undegraded":"2024-04-25T10:11:01.791555+0000","last_fullsized":"2024-04-25T10:11:01.791555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:53.008632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790835+0000","last_change":"2024-04-25T10:11:00.662883+0000","last_active":"2024-04-25T10:11:01.790835+0000","last_peered":"2024-04-25T10:11:01.790835+0000","last_clean":"2024-04-25T10:11:01.790835+0000","last_became_active":"2024-04-25T10:11:00.662578+0000","last_became_peered":"2024-04-25T10:11:00.662578+0000","last_unstale":"2024-04-25T10:11:01.790835+0000","last_undegraded":"2024-04-25T10:11:01.790835+0000","last_fullsized":"2024-04-25T10:11:01.790835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:55.212072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791634+0000","last_change":"2024-04-25T10:10:59.663404+0000","last_active":"2024-04-25T10:11:01.791634+0000","last_peered":"2024-04-25T10:11:01.791634+0000","last_clean":"2024-04-25T10:11:01.791634+0000","last_became_active":"2024-04-25T10:10:59.663138+0000","last_became_peered":"2024-04-25T10:10:59.663138+0000","last_unstale":"2024-04-25T10:11:01.791634+0000","last_undegraded":"2024-04-25T10:11:01.791634+0000","last_fullsized":"2024-04-25T10:11:01.791634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:38.050152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672109+0000","last_change":"2024-04-25T10:11:00.672494+0000","last_active":"2024-04-25T10:11:01.672109+0000","last_peered":"2024-04-25T10:11:01.672109+0000","last_clean":"2024-04-25T10:11:01.672109+0000","last_became_active":"2024-04-25T10:11:00.672238+0000","last_became_peered":"2024-04-25T10:11:00.672238+0000","last_unstale":"2024-04-25T10:11:01.672109+0000","last_undegraded":"2024-04-25T10:11:01.672109+0000","last_fullsized":"2024-04-25T10:11:01.672109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:48.642917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791130+0000","last_change":"2024-04-25T10:10:59.662706+0000","last_active":"2024-04-25T10:11:01.791130+0000","last_peered":"2024-04-25T10:11:01.791130+0000","last_clean":"2024-04-25T10:11:01.791130+0000","last_became_active":"2024-04-25T10:10:59.662399+0000","last_became_peered":"2024-04-25T10:10:59.662399+0000","last_unstale":"2024-04-25T10:11:01.791130+0000","last_undegraded":"2024-04-25T10:11:01.791130+0000","last_fullsized":"2024-04-25T10:11:01.791130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:56.481855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791760+0000","last_change":"2024-04-25T10:11:00.684360+0000","last_active":"2024-04-25T10:11:01.791760+0000","last_peered":"2024-04-25T10:11:01.791760+0000","last_clean":"2024-04-25T10:11:01.791760+0000","last_became_active":"2024-04-25T10:11:00.683915+0000","last_became_peered":"2024-04-25T10:11:00.683915+0000","last_unstale":"2024-04-25T10:11:01.791760+0000","last_undegraded":"2024-04-25T10:11:01.791760+0000","last_fullsized":"2024-04-25T10:11:01.791760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:44.016510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672207+0000","last_change":"2024-04-25T10:10:59.664474+0000","last_active":"2024-04-25T10:11:01.672207+0000","last_peered":"2024-04-25T10:11:01.672207+0000","last_clean":"2024-04-25T10:11:01.672207+0000","last_became_active":"2024-04-25T10:10:59.664182+0000","last_became_peered":"2024-04-25T10:10:59.664182+0000","last_unstale":"2024-04-25T10:11:01.672207+0000","last_undegraded":"2024-04-25T10:11:01.672207+0000","last_fullsized":"2024-04-25T10:11:01.672207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:51.674567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.653171+0000","last_change":"2024-04-25T10:10:59.653171+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.653171+0000","last_undegraded":"2024-04-25T10:10:59.653171+0000","last_fullsized":"2024-04-25T10:10:59.653171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669493+0000","last_change":"2024-04-25T10:10:59.656426+0000","last_active":"2024-04-25T10:11:01.669493+0000","last_peered":"2024-04-25T10:11:01.669493+0000","last_clean":"2024-04-25T10:11:01.669493+0000","last_became_active":"2024-04-25T10:10:59.656089+0000","last_became_peered":"2024-04-25T10:10:59.656089+0000","last_unstale":"2024-04-25T10:11:01.669493+0000","last_undegraded":"2024-04-25T10:11:01.669493+0000","last_fullsized":"2024-04-25T10:11:01.669493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:50.774285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790576+0000","last_change":"2024-04-25T10:11:00.675256+0000","last_active":"2024-04-25T10:11:01.790576+0000","last_peered":"2024-04-25T10:11:01.790576+0000","last_clean":"2024-04-25T10:11:01.790576+0000","last_became_active":"2024-04-25T10:11:00.674969+0000","last_became_peered":"2024-04-25T10:11:00.674969+0000","last_unstale":"2024-04-25T10:11:01.790576+0000","last_undegraded":"2024-04-25T10:11:01.790576+0000","last_fullsized":"2024-04-25T10:11:01.790576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.860030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791546+0000","last_change":"2024-04-25T10:10:59.662798+0000","last_active":"2024-04-25T10:11:01.791546+0000","last_peered":"2024-04-25T10:11:01.791546+0000","last_clean":"2024-04-25T10:11:01.791546+0000","last_became_active":"2024-04-25T10:10:59.661497+0000","last_became_peered":"2024-04-25T10:10:59.661497+0000","last_unstale":"2024-04-25T10:11:01.791546+0000","last_undegraded":"2024-04-25T10:11:01.791546+0000","last_fullsized":"2024-04-25T10:11:01.791546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:29.530495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791719+0000","last_change":"2024-04-25T10:11:00.666183+0000","last_active":"2024-04-25T10:11:01.791719+0000","last_peered":"2024-04-25T10:11:01.791719+0000","last_clean":"2024-04-25T10:11:01.791719+0000","last_became_active":"2024-04-25T10:11:00.665905+0000","last_became_peered":"2024-04-25T10:11:00.665905+0000","last_unstale":"2024-04-25T10:11:01.791719+0000","last_undegraded":"2024-04-25T10:11:01.791719+0000","last_fullsized":"2024-04-25T10:11:01.791719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.427341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791772+0000","last_change":"2024-04-25T10:10:59.661863+0000","last_active":"2024-04-25T10:11:01.791772+0000","last_peered":"2024-04-25T10:11:01.791772+0000","last_clean":"2024-04-25T10:11:01.791772+0000","last_became_active":"2024-04-25T10:10:59.661598+0000","last_became_peered":"2024-04-25T10:10:59.661598+0000","last_unstale":"2024-04-25T10:11:01.791772+0000","last_undegraded":"2024-04-25T10:11:01.791772+0000","last_fullsized":"2024-04-25T10:11:01.791772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:45.820044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791785+0000","last_change":"2024-04-25T10:11:00.666115+0000","last_active":"2024-04-25T10:11:01.791785+0000","last_peered":"2024-04-25T10:11:01.791785+0000","last_clean":"2024-04-25T10:11:01.791785+0000","last_became_active":"2024-04-25T10:11:00.665817+0000","last_became_peered":"2024-04-25T10:11:00.665817+0000","last_unstale":"2024-04-25T10:11:01.791785+0000","last_undegraded":"2024-04-25T10:11:01.791785+0000","last_fullsized":"2024-04-25T10:11:01.791785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:10.823889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.670527+0000","last_change":"2024-04-25T10:10:59.654401+0000","last_active":"2024-04-25T10:11:01.670527+0000","last_peered":"2024-04-25T10:11:01.670527+0000","last_clean":"2024-04-25T10:11:01.670527+0000","last_became_active":"2024-04-25T10:10:59.654067+0000","last_became_peered":"2024-04-25T10:10:59.654067+0000","last_unstale":"2024-04-25T10:11:01.670527+0000","last_undegraded":"2024-04-25T10:11:01.670527+0000","last_fullsized":"2024-04-25T10:11:01.670527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:45.817445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791696+0000","last_change":"2024-04-25T10:11:00.671112+0000","last_active":"2024-04-25T10:11:01.791696+0000","last_peered":"2024-04-25T10:11:01.791696+0000","last_clean":"2024-04-25T10:11:01.791696+0000","last_became_active":"2024-04-25T10:11:00.670792+0000","last_became_peered":"2024-04-25T10:11:00.670792+0000","last_unstale":"2024-04-25T10:11:01.791696+0000","last_undegraded":"2024-04-25T10:11:01.791696+0000","last_fullsized":"2024-04-25T10:11:01.791696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:00.467722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790958+0000","last_change":"2024-04-25T10:10:59.660751+0000","last_active":"2024-04-25T10:11:01.790958+0000","last_peered":"2024-04-25T10:11:01.790958+0000","last_clean":"2024-04-25T10:11:01.790958+0000","last_became_active":"2024-04-25T10:10:59.660439+0000","last_became_peered":"2024-04-25T10:10:59.660439+0000","last_unstale":"2024-04-25T10:11:01.790958+0000","last_undegraded":"2024-04-25T10:11:01.790958+0000","last_fullsized":"2024-04-25T10:11:01.790958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:17.523240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666630+0000","last_change":"2024-04-25T10:11:00.662687+0000","last_active":"2024-04-25T10:11:01.666630+0000","last_peered":"2024-04-25T10:11:01.666630+0000","last_clean":"2024-04-25T10:11:01.666630+0000","last_became_active":"2024-04-25T10:11:00.662430+0000","last_became_peered":"2024-04-25T10:11:00.662430+0000","last_unstale":"2024-04-25T10:11:01.666630+0000","last_undegraded":"2024-04-25T10:11:01.666630+0000","last_fullsized":"2024-04-25T10:11:01.666630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:14.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666396+0000","last_change":"2024-04-25T10:10:59.660156+0000","last_active":"2024-04-25T10:11:01.666396+0000","last_peered":"2024-04-25T10:11:01.666396+0000","last_clean":"2024-04-25T10:11:01.666396+0000","last_became_active":"2024-04-25T10:10:59.659878+0000","last_became_peered":"2024-04-25T10:10:59.659878+0000","last_unstale":"2024-04-25T10:11:01.666396+0000","last_undegraded":"2024-04-25T10:11:01.666396+0000","last_fullsized":"2024-04-25T10:11:01.666396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:55.986122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669254+0000","last_change":"2024-04-25T10:11:00.664855+0000","last_active":"2024-04-25T10:11:01.669254+0000","last_peered":"2024-04-25T10:11:01.669254+0000","last_clean":"2024-04-25T10:11:01.669254+0000","last_became_active":"2024-04-25T10:11:00.664174+0000","last_became_peered":"2024-04-25T10:11:00.664174+0000","last_unstale":"2024-04-25T10:11:01.669254+0000","last_undegraded":"2024-04-25T10:11:01.669254+0000","last_fullsized":"2024-04-25T10:11:01.669254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.235679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672299+0000","last_change":"2024-04-25T10:10:59.656397+0000","last_active":"2024-04-25T10:11:01.672299+0000","last_peered":"2024-04-25T10:11:01.672299+0000","last_clean":"2024-04-25T10:11:01.672299+0000","last_became_active":"2024-04-25T10:10:59.656114+0000","last_became_peered":"2024-04-25T10:10:59.656114+0000","last_unstale":"2024-04-25T10:11:01.672299+0000","last_undegraded":"2024-04-25T10:11:01.672299+0000","last_fullsized":"2024-04-25T10:11:01.672299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:39.088943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669867+0000","last_change":"2024-04-25T10:11:00.675967+0000","last_active":"2024-04-25T10:11:01.669867+0000","last_peered":"2024-04-25T10:11:01.669867+0000","last_clean":"2024-04-25T10:11:01.669867+0000","last_became_active":"2024-04-25T10:11:00.673547+0000","last_became_peered":"2024-04-25T10:11:00.673547+0000","last_unstale":"2024-04-25T10:11:01.669867+0000","last_undegraded":"2024-04-25T10:11:01.669867+0000","last_fullsized":"2024-04-25T10:11:01.669867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:43.118238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666689+0000","last_change":"2024-04-25T10:10:59.661201+0000","last_active":"2024-04-25T10:11:01.666689+0000","last_peered":"2024-04-25T10:11:01.666689+0000","last_clean":"2024-04-25T10:11:01.666689+0000","last_became_active":"2024-04-25T10:10:59.660913+0000","last_became_peered":"2024-04-25T10:10:59.660913+0000","last_unstale":"2024-04-25T10:11:01.666689+0000","last_undegraded":"2024-04-25T10:11:01.666689+0000","last_fullsized":"2024-04-25T10:11:01.666689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:37.676864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669208+0000","last_change":"2024-04-25T10:11:00.664505+0000","last_active":"2024-04-25T10:11:01.669208+0000","last_peered":"2024-04-25T10:11:01.669208+0000","last_clean":"2024-04-25T10:11:01.669208+0000","last_became_active":"2024-04-25T10:11:00.664218+0000","last_became_peered":"2024-04-25T10:11:00.664218+0000","last_unstale":"2024-04-25T10:11:01.669208+0000","last_undegraded":"2024-04-25T10:11:01.669208+0000","last_fullsized":"2024-04-25T10:11:01.669208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.603074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669199+0000","last_change":"2024-04-25T10:10:59.652758+0000","last_active":"2024-04-25T10:11:01.669199+0000","last_peered":"2024-04-25T10:11:01.669199+0000","last_clean":"2024-04-25T10:11:01.669199+0000","last_became_active":"2024-04-25T10:10:59.652456+0000","last_became_peered":"2024-04-25T10:10:59.652456+0000","last_unstale":"2024-04-25T10:11:01.669199+0000","last_undegraded":"2024-04-25T10:11:01.669199+0000","last_fullsized":"2024-04-25T10:11:01.669199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:53.008632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669239+0000","last_change":"2024-04-25T10:11:00.660736+0000","last_active":"2024-04-25T10:11:01.669239+0000","last_peered":"2024-04-25T10:11:01.669239+0000","last_clean":"2024-04-25T10:11:01.669239+0000","last_became_active":"2024-04-25T10:11:00.660471+0000","last_became_peered":"2024-04-25T10:11:00.660471+0000","last_unstale":"2024-04-25T10:11:01.669239+0000","last_undegraded":"2024-04-25T10:11:01.669239+0000","last_fullsized":"2024-04-25T10:11:01.669239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:37.071670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790521+0000","last_change":"2024-04-25T10:10:59.660087+0000","last_active":"2024-04-25T10:11:01.790521+0000","last_peered":"2024-04-25T10:11:01.790521+0000","last_clean":"2024-04-25T10:11:01.790521+0000","last_became_active":"2024-04-25T10:10:59.659815+0000","last_became_peered":"2024-04-25T10:10:59.659815+0000","last_unstale":"2024-04-25T10:11:01.790521+0000","last_undegraded":"2024-04-25T10:11:01.790521+0000","last_fullsized":"2024-04-25T10:11:01.790521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:42.113345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669804+0000","last_change":"2024-04-25T10:11:00.674881+0000","last_active":"2024-04-25T10:11:01.669804+0000","last_peered":"2024-04-25T10:11:01.669804+0000","last_clean":"2024-04-25T10:11:01.669804+0000","last_became_active":"2024-04-25T10:11:00.674632+0000","last_became_peered":"2024-04-25T10:11:00.674632+0000","last_unstale":"2024-04-25T10:11:01.669804+0000","last_undegraded":"2024-04-25T10:11:01.669804+0000","last_fullsized":"2024-04-25T10:11:01.669804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:36.103452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791702+0000","last_change":"2024-04-25T10:10:59.663122+0000","last_active":"2024-04-25T10:11:01.791702+0000","last_peered":"2024-04-25T10:11:01.791702+0000","last_clean":"2024-04-25T10:11:01.791702+0000","last_became_active":"2024-04-25T10:10:59.662664+0000","last_became_peered":"2024-04-25T10:10:59.662664+0000","last_unstale":"2024-04-25T10:11:01.791702+0000","last_undegraded":"2024-04-25T10:11:01.791702+0000","last_fullsized":"2024-04-25T10:11:01.791702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:16.256009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":301,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669306+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:11:01.669306+0000","last_peered":"2024-04-25T10:11:01.669306+0000","last_clean":"2024-04-25T10:11:01.669306+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:11:01.669306+0000","last_undegraded":"2024-04-25T10:11:01.669306+0000","last_fullsized":"2024-04-25T10:11:01.669306+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1052672,"data_stored":446360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509516,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1056768,"data_stored":446962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509517,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1052672,"data_stored":446324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335460,"statfs":{"total":96636764160,"available":96599511040,"internally_reserved":0,"allocated":1060864,"data_stored":446964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509518,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335472,"statfs":{"total":96636764160,"available":96599523328,"internally_reserved":0,"allocated":1048576,"data_stored":446290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509516,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1646592,"data_stored":1036748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509518,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1064960,"data_stored":447924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1605632,"data_stored":1017308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:04.366 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3c is not active+clean 2024-04-25T10:11:04.366 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:10:59.648641+0000', 'last_change': '2024-04-25T10:10:59.648641+0000', 'last_active': '2024-04-25T10:10:59.639238+0000', 'last_peered': '2024-04-25T10:10:59.639238+0000', 'last_clean': '2024-04-25T10:10:59.639238+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:10:59.648641+0000', 'last_undegraded': '2024-04-25T10:10:59.648641+0000', 'last_fullsized': '2024-04-25T10:10:59.648641+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:10:59.639238+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:10:59.639238+0000', 'last_clean_scrub_stamp': '2024-04-25T10:10:59.639238+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:11:04.366 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2b is not active+clean 2024-04-25T10:11:04.366 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:10:59.651673+0000', 'last_change': '2024-04-25T10:10:59.651673+0000', 'last_active': '2024-04-25T10:10:59.639238+0000', 'last_peered': '2024-04-25T10:10:59.639238+0000', 'last_clean': '2024-04-25T10:10:59.639238+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:10:59.651673+0000', 'last_undegraded': '2024-04-25T10:10:59.651673+0000', 'last_fullsized': '2024-04-25T10:10:59.651673+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:10:59.639238+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:10:59.639238+0000', 'last_clean_scrub_stamp': '2024-04-25T10:10:59.639238+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:11:04.366 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.23 is not active+clean 2024-04-25T10:11:04.367 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:10:59.651157+0000', 'last_change': '2024-04-25T10:10:59.651157+0000', 'last_active': '2024-04-25T10:10:59.639238+0000', 'last_peered': '2024-04-25T10:10:59.639238+0000', 'last_clean': '2024-04-25T10:10:59.639238+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:10:59.651157+0000', 'last_undegraded': '2024-04-25T10:10:59.651157+0000', 'last_fullsized': '2024-04-25T10:10:59.651157+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:10:59.639238+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:10:59.639238+0000', 'last_clean_scrub_stamp': '2024-04-25T10:10:59.639238+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:11:04.367 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1a is not active+clean 2024-04-25T10:11:04.367 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:10:59.652342+0000', 'last_change': '2024-04-25T10:10:59.652342+0000', 'last_active': '2024-04-25T10:10:59.639238+0000', 'last_peered': '2024-04-25T10:10:59.639238+0000', 'last_clean': '2024-04-25T10:10:59.639238+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:10:59.652342+0000', 'last_undegraded': '2024-04-25T10:10:59.652342+0000', 'last_fullsized': '2024-04-25T10:10:59.652342+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:10:59.639238+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:10:59.639238+0000', 'last_clean_scrub_stamp': '2024-04-25T10:10:59.639238+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:11:04.367 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.8 is not active+clean 2024-04-25T10:11:04.367 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:10:59.653171+0000', 'last_change': '2024-04-25T10:10:59.653171+0000', 'last_active': '2024-04-25T10:10:59.639238+0000', 'last_peered': '2024-04-25T10:10:59.639238+0000', 'last_clean': '2024-04-25T10:10:59.639238+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:10:59.653171+0000', 'last_undegraded': '2024-04-25T10:10:59.653171+0000', 'last_fullsized': '2024-04-25T10:10:59.653171+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:10:59.639238+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:10:59.639238+0000', 'last_clean_scrub_stamp': '2024-04-25T10:10:59.639238+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:11:04.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:11:04.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.496+0000 7f1ab888f640 1 Processor -- start 2024-04-25T10:11:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.496+0000 7f1ab888f640 1 -- start start 2024-04-25T10:11:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.497+0000 7f1ab888f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ab4099a40 0x7f1ab409a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.497+0000 7f1ab888f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1ab4099e20 0x7f1ab40cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.497+0000 7f1ab888f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ab40cfb90 con 0x7f1ab415bd40 2024-04-25T10:11:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.497+0000 7f1ab888f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab415cfd0 con 0x7f1ab4099e20 2024-04-25T10:11:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.497+0000 7f1ab888f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab415c210 con 0x7f1ab4099a40 2024-04-25T10:11:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.497+0000 7f1ab1d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1ab415bd40 0x7f1ab40cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55506/0 (socket says 172.21.15.16:55506) 2024-04-25T10:11:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.497+0000 7f1ab1d9b640 1 -- 172.21.15.16:0/218181035 learned_addr learned my addr 172.21.15.16:0/218181035 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.497+0000 7f1ab2d9d640 1 --2- 172.21.15.16:0/218181035 >> v2:172.21.15.186:3300/0 conn(0x7f1ab4099e20 0x7f1ab40cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.497+0000 7f1ab259c640 1 --2- 172.21.15.16:0/218181035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ab4099a40 0x7f1ab409a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.497+0000 7f1ab159a640 1 -- 172.21.15.16:0/218181035 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363446428 0 0) 0x7f1ab40cfb90 con 0x7f1ab415bd40 2024-04-25T10:11:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.497+0000 7f1ab159a640 1 -- 172.21.15.16:0/218181035 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a9c003520 con 0x7f1ab415bd40 2024-04-25T10:11:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab159a640 1 -- 172.21.15.16:0/218181035 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 833672167 0 0) 0x7f1a9c003520 con 0x7f1ab415bd40 2024-04-25T10:11:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab159a640 1 -- 172.21.15.16:0/218181035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ab4099a40 msgr2=0x7f1ab409a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab159a640 1 --2- 172.21.15.16:0/218181035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ab4099a40 0x7f1ab409a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab159a640 1 -- 172.21.15.16:0/218181035 >> v2:172.21.15.186:3300/0 conn(0x7f1ab4099e20 msgr2=0x7f1ab40cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab159a640 1 --2- 172.21.15.16:0/218181035 >> v2:172.21.15.186:3300/0 conn(0x7f1ab4099e20 0x7f1ab40cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab159a640 1 -- 172.21.15.16:0/218181035 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab415c4f0 con 0x7f1ab415bd40 2024-04-25T10:11:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab159a640 1 -- 172.21.15.16:0/218181035 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f1aa8001670 con 0x7f1ab415bd40 2024-04-25T10:11:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab2d9d640 1 --2- 172.21.15.16:0/218181035 >> v2:172.21.15.186:3300/0 conn(0x7f1ab4099e20 0x7f1ab40cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab159a640 1 -- 172.21.15.16:0/218181035 >> v1:172.21.15.16:6789/0 conn(0x7f1ab415bd40 legacy=0x7f1ab40cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab159a640 1 --2- 172.21.15.16:0/218181035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ab4099a40 0x7f1a9c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab159a640 1 --2- 172.21.15.16:0/218181035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a9c003f30 0x7f1a9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab159a640 1 --2- 172.21.15.16:0/218181035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a9c004d40 0x7f1a9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab159a640 1 -- 172.21.15.16:0/218181035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab415cfd0 con 0x7f1a9c004d40 2024-04-25T10:11:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab159a640 1 -- 172.21.15.16:0/218181035 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab415c210 con 0x7f1a9c003f30 2024-04-25T10:11:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab159a640 1 -- 172.21.15.16:0/218181035 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c00ba10 con 0x7f1ab4099a40 2024-04-25T10:11:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab2d9d640 1 --2- 172.21.15.16:0/218181035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a9c004d40 0x7f1a9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab1d9b640 1 --2- 172.21.15.16:0/218181035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a9c003f30 0x7f1a9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab259c640 1 --2- 172.21.15.16:0/218181035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ab4099a40 0x7f1a9c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab2d9d640 1 -- 172.21.15.16:0/218181035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ab4099a40 msgr2=0x7f1a9c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab2d9d640 1 --2- 172.21.15.16:0/218181035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ab4099a40 0x7f1a9c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab2d9d640 1 -- 172.21.15.16:0/218181035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a9c003f30 msgr2=0x7f1a9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab2d9d640 1 --2- 172.21.15.16:0/218181035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a9c003f30 0x7f1a9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.498+0000 7f1ab2d9d640 1 -- 172.21.15.16:0/218181035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c003520 con 0x7f1a9c004d40 2024-04-25T10:11:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.499+0000 7f1ab2d9d640 1 --2- 172.21.15.16:0/218181035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a9c004d40 0x7f1a9c0051c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009cc0 tx=0x7f1a9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2062e1c0761b2604 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.499+0000 7f1ab159a640 1 -- 172.21.15.16:0/218181035 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac036070 con 0x7f1a9c004d40 2024-04-25T10:11:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.499+0000 7f1ab159a640 1 -- 172.21.15.16:0/218181035 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac031400 con 0x7f1a9c004d40 2024-04-25T10:11:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.499+0000 7f1ab159a640 1 -- 172.21.15.16:0/218181035 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac009860 con 0x7f1a9c004d40 2024-04-25T10:11:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.500+0000 7f1ab888f640 1 -- 172.21.15.16:0/218181035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a9c004d40 msgr2=0x7f1a9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.500+0000 7f1ab888f640 1 --2- 172.21.15.16:0/218181035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a9c004d40 0x7f1a9c0051c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009cc0 tx=0x7f1a9c005830 comp rx=0 tx=0).stop 2024-04-25T10:11:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.500+0000 7f1ab2d9d640 1 -- 172.21.15.16:0/218181035 reap_dead start 2024-04-25T10:11:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.500+0000 7f1ab888f640 1 -- 172.21.15.16:0/218181035 shutdown_connections 2024-04-25T10:11:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.500+0000 7f1ab888f640 1 -- 172.21.15.16:0/218181035 >> 172.21.15.16:0/218181035 conn(0x7f1ab40b32f0 msgr2=0x7f1ab40b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.500+0000 7f1ab888f640 1 -- 172.21.15.16:0/218181035 shutdown_connections 2024-04-25T10:11:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.500+0000 7f1ab888f640 1 -- 172.21.15.16:0/218181035 wait complete. 2024-04-25T10:11:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.501+0000 7f1ab888f640 1 Processor -- start 2024-04-25T10:11:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.501+0000 7f1ab888f640 1 -- start start 2024-04-25T10:11:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.502+0000 7f1ab888f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a9c004d40 0x7f1ab40c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.502+0000 7f1ab888f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ab406bcb0 0x7f1ab4150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.502+0000 7f1ab888f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ab40c9730 0x7f1ab40c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.502+0000 7f1ab888f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab409a2b0 con 0x7f1a9c004d40 2024-04-25T10:11:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.502+0000 7f1ab888f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab415cab0 con 0x7f1ab406bcb0 2024-04-25T10:11:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.502+0000 7f1ab888f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab409a9c0 con 0x7f1ab40c9730 2024-04-25T10:11:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.502+0000 7f1ab259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a9c004d40 0x7f1ab40c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.502+0000 7f1ab259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a9c004d40 0x7f1ab40c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53256/0 (socket says 172.21.15.16:53256) 2024-04-25T10:11:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.502+0000 7f1ab259c640 1 -- 172.21.15.16:0/3454961070 learned_addr learned my addr 172.21.15.16:0/3454961070 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.502+0000 7f1ab2d9d640 1 --2- 172.21.15.16:0/3454961070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ab40c9730 0x7f1ab40c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.502+0000 7f1ab1d9b640 1 --2- 172.21.15.16:0/3454961070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ab406bcb0 0x7f1ab4150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.502+0000 7f1ab259c640 1 -- 172.21.15.16:0/3454961070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ab40c9730 msgr2=0x7f1ab40c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.502+0000 7f1ab259c640 1 --2- 172.21.15.16:0/3454961070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ab40c9730 0x7f1ab40c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.502+0000 7f1ab259c640 1 -- 172.21.15.16:0/3454961070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ab406bcb0 msgr2=0x7f1ab4150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.502+0000 7f1ab259c640 1 --2- 172.21.15.16:0/3454961070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ab406bcb0 0x7f1ab4150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.502+0000 7f1ab259c640 1 -- 172.21.15.16:0/3454961070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab406c4c0 con 0x7f1a9c004d40 2024-04-25T10:11:04.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.502+0000 7f1ab1d9b640 1 --2- 172.21.15.16:0/3454961070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ab406bcb0 0x7f1ab4150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:04.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.502+0000 7f1ab259c640 1 --2- 172.21.15.16:0/3454961070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a9c004d40 0x7f1ab40c7990 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4011e30 tx=0x7f1aa400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.503+0000 7f1aa2ffd640 1 -- 172.21.15.16:0/3454961070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4015630 con 0x7f1a9c004d40 2024-04-25T10:11:04.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.503+0000 7f1aa2ffd640 1 -- 172.21.15.16:0/3454961070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa40157d0 con 0x7f1a9c004d40 2024-04-25T10:11:04.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.503+0000 7f1ab888f640 1 -- 172.21.15.16:0/3454961070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab406c7a0 con 0x7f1a9c004d40 2024-04-25T10:11:04.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.503+0000 7f1aa2ffd640 1 -- 172.21.15.16:0/3454961070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4015ba0 con 0x7f1a9c004d40 2024-04-25T10:11:04.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.503+0000 7f1ab888f640 1 -- 172.21.15.16:0/3454961070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab40c54d0 con 0x7f1a9c004d40 2024-04-25T10:11:04.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.504+0000 7f1ab888f640 1 -- 172.21.15.16:0/3454961070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab406ccc0 con 0x7f1a9c004d40 2024-04-25T10:11:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.504+0000 7f1aa2ffd640 1 -- 172.21.15.16:0/3454961070 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1aa401d400 con 0x7f1a9c004d40 2024-04-25T10:11:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.505+0000 7f1aa2ffd640 1 --2- 172.21.15.16:0/3454961070 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1a9406b500 0x7f1a9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.505+0000 7f1aa2ffd640 1 -- 172.21.15.16:0/3454961070 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1aa408b350 con 0x7f1a9c004d40 2024-04-25T10:11:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.505+0000 7f1ab1d9b640 1 --2- 172.21.15.16:0/3454961070 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1a9406b500 0x7f1a9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.505+0000 7f1ab1d9b640 1 --2- 172.21.15.16:0/3454961070 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1a9406b500 0x7f1a9406d9c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1ab415ab10 tx=0x7f1aa8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.509+0000 7f1aa2ffd640 1 -- 172.21.15.16:0/3454961070 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ab406ccc0 con 0x7f1a9c004d40 2024-04-25T10:11:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.760+0000 7f1ab888f640 1 -- 172.21.15.16:0/3454961070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1ab40749b0 con 0x7f1a9c004d40 2024-04-25T10:11:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.760+0000 7f1aa2ffd640 1 -- 172.21.15.16:0/3454961070 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1101 (secure 0 0 0) 0x7f1aa4019030 con 0x7f1a9c004d40 2024-04-25T10:11:04.766 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:04.766 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":378,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":86,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039497,"num_in_osds":8,"osd_in_since":1714039488,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":92},{"state_name":"creating+peering","count":5}],"num_pgs":97,"num_pools":3,"num_objects":21,"data_bytes":592744,"bytes_used":273571840,"bytes_avail":772820541440,"bytes_total":773094113280,"inactive_pgs_ratio":0.051546391099691391,"write_bytes_sec":1371,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":51,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":35852}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T10:10:49.864022+0000","services":{}},"progress_events":{}} 2024-04-25T10:11:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.762+0000 7f1ab888f640 1 -- 172.21.15.16:0/3454961070 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1a9406b500 msgr2=0x7f1a9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.762+0000 7f1ab888f640 1 --2- 172.21.15.16:0/3454961070 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1a9406b500 0x7f1a9406d9c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1ab415ab10 tx=0x7f1aa8009040 comp rx=0 tx=0).stop 2024-04-25T10:11:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.762+0000 7f1ab888f640 1 -- 172.21.15.16:0/3454961070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a9c004d40 msgr2=0x7f1ab40c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.762+0000 7f1ab888f640 1 --2- 172.21.15.16:0/3454961070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a9c004d40 0x7f1ab40c7990 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4011e30 tx=0x7f1aa400f580 comp rx=0 tx=0).stop 2024-04-25T10:11:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.763+0000 7f1ab888f640 1 -- 172.21.15.16:0/3454961070 shutdown_connections 2024-04-25T10:11:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.763+0000 7f1ab888f640 1 --2- 172.21.15.16:0/3454961070 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1a9406b500 0x7f1a9406d9c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.763+0000 7f1ab888f640 1 --2- 172.21.15.16:0/3454961070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ab40c9730 0x7f1ab40c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.763+0000 7f1ab888f640 1 --2- 172.21.15.16:0/3454961070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ab406bcb0 0x7f1ab4150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.763+0000 7f1ab888f640 1 --2- 172.21.15.16:0/3454961070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a9c004d40 0x7f1ab40c7990 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.763+0000 7f1ab888f640 1 -- 172.21.15.16:0/3454961070 >> 172.21.15.16:0/3454961070 conn(0x7f1ab40b32f0 msgr2=0x7f1ab40cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.763+0000 7f1ab888f640 1 -- 172.21.15.16:0/3454961070 shutdown_connections 2024-04-25T10:11:04.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.763+0000 7f1ab888f640 1 -- 172.21.15.16:0/3454961070 wait complete. 2024-04-25T10:11:04.779 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:11:04.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.874+0000 7f360476a640 1 Processor -- start 2024-04-25T10:11:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.874+0000 7f360476a640 1 -- start start 2024-04-25T10:11:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.875+0000 7f360476a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35fc15dce0 0x7f35fc0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.875+0000 7f360476a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35fc0b5b60 0x7f35fc0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.875+0000 7f360476a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35fc15e0c0 con 0x7f35fc0d35b0 2024-04-25T10:11:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.875+0000 7f360476a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35fc0d2510 con 0x7f35fc15dce0 2024-04-25T10:11:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.875+0000 7f360476a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0d27f0 con 0x7f35fc0b5b60 2024-04-25T10:11:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.875+0000 7f3602506640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35fc15dce0 0x7f35fc0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.875+0000 7f3601d05640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f35fc0d35b0 0x7f35fc0b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55508/0 (socket says 172.21.15.16:55508) 2024-04-25T10:11:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.875+0000 7f3601d05640 1 -- 172.21.15.16:0/823663781 learned_addr learned my addr 172.21.15.16:0/823663781 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.876+0000 7f3602506640 1 -- 172.21.15.16:0/823663781 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35fc0b5b60 msgr2=0x7f35fc0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.876+0000 7f3602506640 1 --2- 172.21.15.16:0/823663781 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35fc0b5b60 0x7f35fc0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.876+0000 7f3602506640 1 -- 172.21.15.16:0/823663781 >> v1:172.21.15.16:6789/0 conn(0x7f35fc0d35b0 legacy=0x7f35fc0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.876+0000 7f3602506640 1 -- 172.21.15.16:0/823663781 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc0d2ad0 con 0x7f35fc15dce0 2024-04-25T10:11:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.876+0000 7f3601504640 1 -- 172.21.15.16:0/823663781 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185852475 0 0) 0x7f35fc15e0c0 con 0x7f35fc0d35b0 2024-04-25T10:11:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.876+0000 7f3602506640 1 --2- 172.21.15.16:0/823663781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35fc15dce0 0x7f35fc0d1a50 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f35f40097a0 tx=0x7f35f402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c6550d46dbf079 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.876+0000 7f3601504640 1 -- 172.21.15.16:0/823663781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4037070 con 0x7f35fc15dce0 2024-04-25T10:11:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.876+0000 7f3601504640 1 -- 172.21.15.16:0/823663781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f4032440 con 0x7f35fc15dce0 2024-04-25T10:11:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.877+0000 7f3601504640 1 -- 172.21.15.16:0/823663781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4032810 con 0x7f35fc15dce0 2024-04-25T10:11:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.877+0000 7f360476a640 1 -- 172.21.15.16:0/823663781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35fc15dce0 msgr2=0x7f35fc0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.877+0000 7f360476a640 1 --2- 172.21.15.16:0/823663781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35fc15dce0 0x7f35fc0d1a50 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f35f40097a0 tx=0x7f35f402a180 comp rx=0 tx=0).stop 2024-04-25T10:11:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.877+0000 7f360476a640 1 -- 172.21.15.16:0/823663781 shutdown_connections 2024-04-25T10:11:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.877+0000 7f360476a640 1 --2- 172.21.15.16:0/823663781 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35fc0b5b60 0x7f35fc0b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.877+0000 7f360476a640 1 --2- 172.21.15.16:0/823663781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35fc15dce0 0x7f35fc0d1a50 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.877+0000 7f360476a640 1 -- 172.21.15.16:0/823663781 >> 172.21.15.16:0/823663781 conn(0x7f35fc072900 msgr2=0x7f35fc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.877+0000 7f360476a640 1 -- 172.21.15.16:0/823663781 shutdown_connections 2024-04-25T10:11:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.877+0000 7f360476a640 1 -- 172.21.15.16:0/823663781 wait complete. 2024-04-25T10:11:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.878+0000 7f360476a640 1 Processor -- start 2024-04-25T10:11:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.878+0000 7f360476a640 1 -- start start 2024-04-25T10:11:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.879+0000 7f360476a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35fc0b5b60 0x7f35fc1f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.879+0000 7f360476a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35fc0d35b0 0x7f35fc15eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.879+0000 7f360476a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35fc15dce0 0x7f35fc15f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.879+0000 7f360476a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0b9820 con 0x7f35fc0b5b60 2024-04-25T10:11:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.879+0000 7f360476a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0d3090 con 0x7f35fc0d35b0 2024-04-25T10:11:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.879+0000 7f360476a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0b99a0 con 0x7f35fc15dce0 2024-04-25T10:11:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.879+0000 7f3602506640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35fc0b5b60 0x7f35fc1f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.879+0000 7f3602506640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35fc0b5b60 0x7f35fc1f2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53282/0 (socket says 172.21.15.16:53282) 2024-04-25T10:11:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.879+0000 7f3602506640 1 -- 172.21.15.16:0/2291002283 learned_addr learned my addr 172.21.15.16:0/2291002283 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.879+0000 7f3601d05640 1 --2- 172.21.15.16:0/2291002283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35fc0d35b0 0x7f35fc15eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.879+0000 7f3602d07640 1 --2- 172.21.15.16:0/2291002283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35fc15dce0 0x7f35fc15f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.879+0000 7f3602506640 1 -- 172.21.15.16:0/2291002283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35fc15dce0 msgr2=0x7f35fc15f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.879+0000 7f3602506640 1 --2- 172.21.15.16:0/2291002283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35fc15dce0 0x7f35fc15f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.879+0000 7f3602506640 1 -- 172.21.15.16:0/2291002283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35fc0d35b0 msgr2=0x7f35fc15eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.879+0000 7f3602506640 1 --2- 172.21.15.16:0/2291002283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35fc0d35b0 0x7f35fc15eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.879+0000 7f3602506640 1 -- 172.21.15.16:0/2291002283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc15f2e0 con 0x7f35fc0b5b60 2024-04-25T10:11:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.880+0000 7f3601d05640 1 --2- 172.21.15.16:0/2291002283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35fc0d35b0 0x7f35fc15eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.880+0000 7f3602506640 1 --2- 172.21.15.16:0/2291002283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35fc0b5b60 0x7f35fc1f2e70 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f35f40097a0 tx=0x7f35f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.880+0000 7f35faffd640 1 -- 172.21.15.16:0/2291002283 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4041070 con 0x7f35fc0b5b60 2024-04-25T10:11:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.880+0000 7f35faffd640 1 -- 172.21.15.16:0/2291002283 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f403c870 con 0x7f35fc0b5b60 2024-04-25T10:11:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.880+0000 7f360476a640 1 -- 172.21.15.16:0/2291002283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35fc1f6ef0 con 0x7f35fc0b5b60 2024-04-25T10:11:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.880+0000 7f35faffd640 1 -- 172.21.15.16:0/2291002283 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4037040 con 0x7f35fc0b5b60 2024-04-25T10:11:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.880+0000 7f360476a640 1 -- 172.21.15.16:0/2291002283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35fc1f74b0 con 0x7f35fc0b5b60 2024-04-25T10:11:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.881+0000 7f360476a640 1 -- 172.21.15.16:0/2291002283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35fc0749b0 con 0x7f35fc0b5b60 2024-04-25T10:11:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.881+0000 7f35faffd640 1 -- 172.21.15.16:0/2291002283 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f35f4044070 con 0x7f35fc0b5b60 2024-04-25T10:11:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.882+0000 7f35faffd640 1 --2- 172.21.15.16:0/2291002283 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f35e806b500 0x7f35e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.882+0000 7f35faffd640 1 -- 172.21.15.16:0/2291002283 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35f40aac70 con 0x7f35fc0b5b60 2024-04-25T10:11:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.882+0000 7f3601d05640 1 --2- 172.21.15.16:0/2291002283 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f35e806b500 0x7f35e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.883+0000 7f3601d05640 1 --2- 172.21.15.16:0/2291002283 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f35e806b500 0x7f35e806d9c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f35fc067f30 tx=0x7f35f0007c10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:04.886+0000 7f35faffd640 1 -- 172.21.15.16:0/2291002283 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f35f4079ca0 con 0x7f35fc0b5b60 2024-04-25T10:11:05.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:05.063+0000 7f360476a640 1 -- 172.21.15.16:0/2291002283 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f35fc15fb50 con 0x7f35e806b500 2024-04-25T10:11:05.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:05.070+0000 7f35faffd640 1 -- 172.21.15.16:0/2291002283 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251662 (secure 0 0 0) 0x7f35fc15fb50 con 0x7f35e806b500 2024-04-25T10:11:05.078 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:05.079 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:11:05.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:05.073+0000 7f360476a640 1 -- 172.21.15.16:0/2291002283 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f35e806b500 msgr2=0x7f35e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:05.073+0000 7f360476a640 1 --2- 172.21.15.16:0/2291002283 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f35e806b500 0x7f35e806d9c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f35fc067f30 tx=0x7f35f0007c10 comp rx=0 tx=0).stop 2024-04-25T10:11:05.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:05.073+0000 7f360476a640 1 -- 172.21.15.16:0/2291002283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35fc0b5b60 msgr2=0x7f35fc1f2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:05.073+0000 7f360476a640 1 --2- 172.21.15.16:0/2291002283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35fc0b5b60 0x7f35fc1f2e70 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f35f40097a0 tx=0x7f35f40046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:05.073+0000 7f360476a640 1 -- 172.21.15.16:0/2291002283 shutdown_connections 2024-04-25T10:11:05.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:05.073+0000 7f360476a640 1 --2- 172.21.15.16:0/2291002283 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f35e806b500 0x7f35e806d9c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:05.073+0000 7f360476a640 1 --2- 172.21.15.16:0/2291002283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35fc15dce0 0x7f35fc15f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:05.073+0000 7f360476a640 1 --2- 172.21.15.16:0/2291002283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35fc0d35b0 0x7f35fc15eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:05.073+0000 7f360476a640 1 --2- 172.21.15.16:0/2291002283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35fc0b5b60 0x7f35fc1f2e70 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:05.073+0000 7f360476a640 1 -- 172.21.15.16:0/2291002283 >> 172.21.15.16:0/2291002283 conn(0x7f35fc072900 msgr2=0x7f35fc0b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:05.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:05.073+0000 7f360476a640 1 -- 172.21.15.16:0/2291002283 shutdown_connections 2024-04-25T10:11:05.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:05.073+0000 7f360476a640 1 -- 172.21.15.16:0/2291002283 wait complete. 2024-04-25T10:11:05.089 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-04-25T10:11:03.867060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":198,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":267160,"kb_used_data":9364,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754707560,"statfs":{"total":773094113280,"available":772820541440,"internally_reserved":0,"allocated":9588736,"data_stored":4734880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.226999"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669584+0000","last_change":"2024-04-25T10:11:00.665795+0000","last_active":"2024-04-25T10:11:01.669584+0000","last_peered":"2024-04-25T10:11:01.669584+0000","last_clean":"2024-04-25T10:11:01.669584+0000","last_became_active":"2024-04-25T10:11:00.664672+0000","last_became_peered":"2024-04-25T10:11:00.664672+0000","last_unstale":"2024-04-25T10:11:01.669584+0000","last_undegraded":"2024-04-25T10:11:01.669584+0000","last_fullsized":"2024-04-25T10:11:01.669584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:30.244884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672240+0000","last_change":"2024-04-25T10:11:00.658574+0000","last_active":"2024-04-25T10:11:01.672240+0000","last_peered":"2024-04-25T10:11:01.672240+0000","last_clean":"2024-04-25T10:11:01.672240+0000","last_became_active":"2024-04-25T10:11:00.658280+0000","last_became_peered":"2024-04-25T10:11:00.658280+0000","last_unstale":"2024-04-25T10:11:01.672240+0000","last_undegraded":"2024-04-25T10:11:01.672240+0000","last_fullsized":"2024-04-25T10:11:01.672240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:38.681757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666472+0000","last_change":"2024-04-25T10:11:00.664492+0000","last_active":"2024-04-25T10:11:01.666472+0000","last_peered":"2024-04-25T10:11:01.666472+0000","last_clean":"2024-04-25T10:11:01.666472+0000","last_became_active":"2024-04-25T10:11:00.664205+0000","last_became_peered":"2024-04-25T10:11:00.664205+0000","last_unstale":"2024-04-25T10:11:01.666472+0000","last_undegraded":"2024-04-25T10:11:01.666472+0000","last_fullsized":"2024-04-25T10:11:01.666472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.860030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.648641+0000","last_change":"2024-04-25T10:10:59.648641+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.648641+0000","last_undegraded":"2024-04-25T10:10:59.648641+0000","last_fullsized":"2024-04-25T10:10:59.648641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791419+0000","last_change":"2024-04-25T10:11:00.674341+0000","last_active":"2024-04-25T10:11:01.791419+0000","last_peered":"2024-04-25T10:11:01.791419+0000","last_clean":"2024-04-25T10:11:01.791419+0000","last_became_active":"2024-04-25T10:11:00.674016+0000","last_became_peered":"2024-04-25T10:11:00.674016+0000","last_unstale":"2024-04-25T10:11:01.791419+0000","last_undegraded":"2024-04-25T10:11:01.791419+0000","last_fullsized":"2024-04-25T10:11:01.791419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:17.318511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666458+0000","last_change":"2024-04-25T10:11:00.666821+0000","last_active":"2024-04-25T10:11:01.666458+0000","last_peered":"2024-04-25T10:11:01.666458+0000","last_clean":"2024-04-25T10:11:01.666458+0000","last_became_active":"2024-04-25T10:11:00.666552+0000","last_became_peered":"2024-04-25T10:11:00.666552+0000","last_unstale":"2024-04-25T10:11:01.666458+0000","last_undegraded":"2024-04-25T10:11:01.666458+0000","last_fullsized":"2024-04-25T10:11:01.666458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:43.541677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.671974+0000","last_change":"2024-04-25T10:11:00.666060+0000","last_active":"2024-04-25T10:11:01.671974+0000","last_peered":"2024-04-25T10:11:01.671974+0000","last_clean":"2024-04-25T10:11:01.671974+0000","last_became_active":"2024-04-25T10:11:00.665760+0000","last_became_peered":"2024-04-25T10:11:00.665760+0000","last_unstale":"2024-04-25T10:11:01.671974+0000","last_undegraded":"2024-04-25T10:11:01.671974+0000","last_fullsized":"2024-04-25T10:11:01.671974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:56.991014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790545+0000","last_change":"2024-04-25T10:11:00.672928+0000","last_active":"2024-04-25T10:11:01.790545+0000","last_peered":"2024-04-25T10:11:01.790545+0000","last_clean":"2024-04-25T10:11:01.790545+0000","last_became_active":"2024-04-25T10:11:00.672604+0000","last_became_peered":"2024-04-25T10:11:00.672604+0000","last_unstale":"2024-04-25T10:11:01.790545+0000","last_undegraded":"2024-04-25T10:11:01.790545+0000","last_fullsized":"2024-04-25T10:11:01.790545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:43.541677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669584+0000","last_change":"2024-04-25T10:11:00.665452+0000","last_active":"2024-04-25T10:11:01.669584+0000","last_peered":"2024-04-25T10:11:01.669584+0000","last_clean":"2024-04-25T10:11:01.669584+0000","last_became_active":"2024-04-25T10:11:00.665186+0000","last_became_peered":"2024-04-25T10:11:00.665186+0000","last_unstale":"2024-04-25T10:11:01.669584+0000","last_undegraded":"2024-04-25T10:11:01.669584+0000","last_fullsized":"2024-04-25T10:11:01.669584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:13.745574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666508+0000","last_change":"2024-04-25T10:11:00.657395+0000","last_active":"2024-04-25T10:11:01.666508+0000","last_peered":"2024-04-25T10:11:01.666508+0000","last_clean":"2024-04-25T10:11:01.666508+0000","last_became_active":"2024-04-25T10:11:00.657121+0000","last_became_peered":"2024-04-25T10:11:00.657121+0000","last_unstale":"2024-04-25T10:11:01.666508+0000","last_undegraded":"2024-04-25T10:11:01.666508+0000","last_fullsized":"2024-04-25T10:11:01.666508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:28.012019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669989+0000","last_change":"2024-04-25T10:11:00.674199+0000","last_active":"2024-04-25T10:11:01.669989+0000","last_peered":"2024-04-25T10:11:01.669989+0000","last_clean":"2024-04-25T10:11:01.669989+0000","last_became_active":"2024-04-25T10:11:00.673935+0000","last_became_peered":"2024-04-25T10:11:00.673935+0000","last_unstale":"2024-04-25T10:11:01.669989+0000","last_undegraded":"2024-04-25T10:11:01.669989+0000","last_fullsized":"2024-04-25T10:11:01.669989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:35.284964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790779+0000","last_change":"2024-04-25T10:11:00.675112+0000","last_active":"2024-04-25T10:11:01.790779+0000","last_peered":"2024-04-25T10:11:01.790779+0000","last_clean":"2024-04-25T10:11:01.790779+0000","last_became_active":"2024-04-25T10:11:00.674812+0000","last_became_peered":"2024-04-25T10:11:00.674812+0000","last_unstale":"2024-04-25T10:11:01.790779+0000","last_undegraded":"2024-04-25T10:11:01.790779+0000","last_fullsized":"2024-04-25T10:11:01.790779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:56.966365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791569+0000","last_change":"2024-04-25T10:11:00.660277+0000","last_active":"2024-04-25T10:11:01.791569+0000","last_peered":"2024-04-25T10:11:01.791569+0000","last_clean":"2024-04-25T10:11:01.791569+0000","last_became_active":"2024-04-25T10:11:00.659982+0000","last_became_peered":"2024-04-25T10:11:00.659982+0000","last_unstale":"2024-04-25T10:11:01.791569+0000","last_undegraded":"2024-04-25T10:11:01.791569+0000","last_fullsized":"2024-04-25T10:11:01.791569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:26.638020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791242+0000","last_change":"2024-04-25T10:11:00.672864+0000","last_active":"2024-04-25T10:11:01.791242+0000","last_peered":"2024-04-25T10:11:01.791242+0000","last_clean":"2024-04-25T10:11:01.791242+0000","last_became_active":"2024-04-25T10:11:00.672565+0000","last_became_peered":"2024-04-25T10:11:00.672565+0000","last_unstale":"2024-04-25T10:11:01.791242+0000","last_undegraded":"2024-04-25T10:11:01.791242+0000","last_fullsized":"2024-04-25T10:11:01.791242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:38.932561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.671980+0000","last_change":"2024-04-25T10:11:00.675075+0000","last_active":"2024-04-25T10:11:01.671980+0000","last_peered":"2024-04-25T10:11:01.671980+0000","last_clean":"2024-04-25T10:11:01.671980+0000","last_became_active":"2024-04-25T10:11:00.674791+0000","last_became_peered":"2024-04-25T10:11:00.674791+0000","last_unstale":"2024-04-25T10:11:01.671980+0000","last_undegraded":"2024-04-25T10:11:01.671980+0000","last_fullsized":"2024-04-25T10:11:01.671980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:14.897123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666210+0000","last_change":"2024-04-25T10:11:00.658578+0000","last_active":"2024-04-25T10:11:01.666210+0000","last_peered":"2024-04-25T10:11:01.666210+0000","last_clean":"2024-04-25T10:11:01.666210+0000","last_became_active":"2024-04-25T10:11:00.658309+0000","last_became_peered":"2024-04-25T10:11:00.658309+0000","last_unstale":"2024-04-25T10:11:01.666210+0000","last_undegraded":"2024-04-25T10:11:01.666210+0000","last_fullsized":"2024-04-25T10:11:01.666210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:40.828899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790928+0000","last_change":"2024-04-25T10:11:00.674321+0000","last_active":"2024-04-25T10:11:01.790928+0000","last_peered":"2024-04-25T10:11:01.790928+0000","last_clean":"2024-04-25T10:11:01.790928+0000","last_became_active":"2024-04-25T10:11:00.674037+0000","last_became_peered":"2024-04-25T10:11:00.674037+0000","last_unstale":"2024-04-25T10:11:01.790928+0000","last_undegraded":"2024-04-25T10:11:01.790928+0000","last_fullsized":"2024-04-25T10:11:01.790928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:37.420962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669990+0000","last_change":"2024-04-25T10:11:00.673650+0000","last_active":"2024-04-25T10:11:01.669990+0000","last_peered":"2024-04-25T10:11:01.669990+0000","last_clean":"2024-04-25T10:11:01.669990+0000","last_became_active":"2024-04-25T10:11:00.673318+0000","last_became_peered":"2024-04-25T10:11:00.673318+0000","last_unstale":"2024-04-25T10:11:01.669990+0000","last_undegraded":"2024-04-25T10:11:01.669990+0000","last_fullsized":"2024-04-25T10:11:01.669990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:18.528132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791780+0000","last_change":"2024-04-25T10:11:00.672749+0000","last_active":"2024-04-25T10:11:01.791780+0000","last_peered":"2024-04-25T10:11:01.791780+0000","last_clean":"2024-04-25T10:11:01.791780+0000","last_became_active":"2024-04-25T10:11:00.671982+0000","last_became_peered":"2024-04-25T10:11:00.671982+0000","last_unstale":"2024-04-25T10:11:01.791780+0000","last_undegraded":"2024-04-25T10:11:01.791780+0000","last_fullsized":"2024-04-25T10:11:01.791780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.235679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791396+0000","last_change":"2024-04-25T10:11:00.674897+0000","last_active":"2024-04-25T10:11:01.791396+0000","last_peered":"2024-04-25T10:11:01.791396+0000","last_clean":"2024-04-25T10:11:01.791396+0000","last_became_active":"2024-04-25T10:11:00.674582+0000","last_became_peered":"2024-04-25T10:11:00.674582+0000","last_unstale":"2024-04-25T10:11:01.791396+0000","last_undegraded":"2024-04-25T10:11:01.791396+0000","last_fullsized":"2024-04-25T10:11:01.791396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.603074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.651673+0000","last_change":"2024-04-25T10:10:59.651673+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.651673+0000","last_undegraded":"2024-04-25T10:10:59.651673+0000","last_fullsized":"2024-04-25T10:10:59.651673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791410+0000","last_change":"2024-04-25T10:11:00.670165+0000","last_active":"2024-04-25T10:11:01.791410+0000","last_peered":"2024-04-25T10:11:01.791410+0000","last_clean":"2024-04-25T10:11:01.791410+0000","last_became_active":"2024-04-25T10:11:00.669866+0000","last_became_peered":"2024-04-25T10:11:00.669866+0000","last_unstale":"2024-04-25T10:11:01.791410+0000","last_undegraded":"2024-04-25T10:11:01.791410+0000","last_fullsized":"2024-04-25T10:11:01.791410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:43.529739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790610+0000","last_change":"2024-04-25T10:11:00.674422+0000","last_active":"2024-04-25T10:11:01.790610+0000","last_peered":"2024-04-25T10:11:01.790610+0000","last_clean":"2024-04-25T10:11:01.790610+0000","last_became_active":"2024-04-25T10:11:00.674142+0000","last_became_peered":"2024-04-25T10:11:00.674142+0000","last_unstale":"2024-04-25T10:11:01.790610+0000","last_undegraded":"2024-04-25T10:11:01.790610+0000","last_fullsized":"2024-04-25T10:11:01.790610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:28.012019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669720+0000","last_change":"2024-04-25T10:11:00.672113+0000","last_active":"2024-04-25T10:11:01.669720+0000","last_peered":"2024-04-25T10:11:01.669720+0000","last_clean":"2024-04-25T10:11:01.669720+0000","last_became_active":"2024-04-25T10:11:00.671835+0000","last_became_peered":"2024-04-25T10:11:00.671835+0000","last_unstale":"2024-04-25T10:11:01.669720+0000","last_undegraded":"2024-04-25T10:11:01.669720+0000","last_fullsized":"2024-04-25T10:11:01.669720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:56.991014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.670008+0000","last_change":"2024-04-25T10:11:00.679427+0000","last_active":"2024-04-25T10:11:01.670008+0000","last_peered":"2024-04-25T10:11:01.670008+0000","last_clean":"2024-04-25T10:11:01.670008+0000","last_became_active":"2024-04-25T10:11:00.678268+0000","last_became_peered":"2024-04-25T10:11:00.678268+0000","last_unstale":"2024-04-25T10:11:01.670008+0000","last_undegraded":"2024-04-25T10:11:01.670008+0000","last_fullsized":"2024-04-25T10:11:01.670008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:57.428737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791705+0000","last_change":"2024-04-25T10:11:00.671179+0000","last_active":"2024-04-25T10:11:01.791705+0000","last_peered":"2024-04-25T10:11:01.791705+0000","last_clean":"2024-04-25T10:11:01.791705+0000","last_became_active":"2024-04-25T10:11:00.670882+0000","last_became_peered":"2024-04-25T10:11:00.670882+0000","last_unstale":"2024-04-25T10:11:01.791705+0000","last_undegraded":"2024-04-25T10:11:01.791705+0000","last_fullsized":"2024-04-25T10:11:01.791705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:22.210854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791595+0000","last_change":"2024-04-25T10:11:00.672506+0000","last_active":"2024-04-25T10:11:01.791595+0000","last_peered":"2024-04-25T10:11:01.791595+0000","last_clean":"2024-04-25T10:11:01.791595+0000","last_became_active":"2024-04-25T10:11:00.672018+0000","last_became_peered":"2024-04-25T10:11:00.672018+0000","last_unstale":"2024-04-25T10:11:01.791595+0000","last_undegraded":"2024-04-25T10:11:01.791595+0000","last_fullsized":"2024-04-25T10:11:01.791595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:14.645638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669413+0000","last_change":"2024-04-25T10:11:00.668016+0000","last_active":"2024-04-25T10:11:01.669413+0000","last_peered":"2024-04-25T10:11:01.669413+0000","last_clean":"2024-04-25T10:11:01.669413+0000","last_became_active":"2024-04-25T10:11:00.667742+0000","last_became_peered":"2024-04-25T10:11:00.667742+0000","last_unstale":"2024-04-25T10:11:01.669413+0000","last_undegraded":"2024-04-25T10:11:01.669413+0000","last_fullsized":"2024-04-25T10:11:01.669413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:35.210017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.651157+0000","last_change":"2024-04-25T10:10:59.651157+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.651157+0000","last_undegraded":"2024-04-25T10:10:59.651157+0000","last_fullsized":"2024-04-25T10:10:59.651157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791586+0000","last_change":"2024-04-25T10:11:00.672908+0000","last_active":"2024-04-25T10:11:01.791586+0000","last_peered":"2024-04-25T10:11:01.791586+0000","last_clean":"2024-04-25T10:11:01.791586+0000","last_became_active":"2024-04-25T10:11:00.672634+0000","last_became_peered":"2024-04-25T10:11:00.672634+0000","last_unstale":"2024-04-25T10:11:01.791586+0000","last_undegraded":"2024-04-25T10:11:01.791586+0000","last_fullsized":"2024-04-25T10:11:01.791586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:37.071670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666210+0000","last_change":"2024-04-25T10:11:00.667457+0000","last_active":"2024-04-25T10:11:01.666210+0000","last_peered":"2024-04-25T10:11:01.666210+0000","last_clean":"2024-04-25T10:11:01.666210+0000","last_became_active":"2024-04-25T10:11:00.667205+0000","last_became_peered":"2024-04-25T10:11:00.667205+0000","last_unstale":"2024-04-25T10:11:01.666210+0000","last_undegraded":"2024-04-25T10:11:01.666210+0000","last_fullsized":"2024-04-25T10:11:01.666210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:07.289251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666783+0000","last_change":"2024-04-25T10:11:00.668155+0000","last_active":"2024-04-25T10:11:01.666783+0000","last_peered":"2024-04-25T10:11:01.666783+0000","last_clean":"2024-04-25T10:11:01.666783+0000","last_became_active":"2024-04-25T10:11:00.667848+0000","last_became_peered":"2024-04-25T10:11:00.667848+0000","last_unstale":"2024-04-25T10:11:01.666783+0000","last_undegraded":"2024-04-25T10:11:01.666783+0000","last_fullsized":"2024-04-25T10:11:01.666783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:55.212072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790745+0000","last_change":"2024-04-25T10:11:00.664212+0000","last_active":"2024-04-25T10:11:01.790745+0000","last_peered":"2024-04-25T10:11:01.790745+0000","last_clean":"2024-04-25T10:11:01.790745+0000","last_became_active":"2024-04-25T10:11:00.663662+0000","last_became_peered":"2024-04-25T10:11:00.663662+0000","last_unstale":"2024-04-25T10:11:01.790745+0000","last_undegraded":"2024-04-25T10:11:01.790745+0000","last_fullsized":"2024-04-25T10:11:01.790745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:14.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"85'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791831+0000","last_change":"2024-04-25T10:10:59.661968+0000","last_active":"2024-04-25T10:11:01.791831+0000","last_peered":"2024-04-25T10:11:01.791831+0000","last_clean":"2024-04-25T10:11:01.791831+0000","last_became_active":"2024-04-25T10:10:59.661557+0000","last_became_peered":"2024-04-25T10:10:59.661557+0000","last_unstale":"2024-04-25T10:11:01.791831+0000","last_undegraded":"2024-04-25T10:11:01.791831+0000","last_fullsized":"2024-04-25T10:11:01.791831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:29.166481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669745+0000","last_change":"2024-04-25T10:11:00.664765+0000","last_active":"2024-04-25T10:11:01.669745+0000","last_peered":"2024-04-25T10:11:01.669745+0000","last_clean":"2024-04-25T10:11:01.669745+0000","last_became_active":"2024-04-25T10:11:00.664306+0000","last_became_peered":"2024-04-25T10:11:00.664306+0000","last_unstale":"2024-04-25T10:11:01.669745+0000","last_undegraded":"2024-04-25T10:11:01.669745+0000","last_fullsized":"2024-04-25T10:11:01.669745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:51.779177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666622+0000","last_change":"2024-04-25T10:10:59.659985+0000","last_active":"2024-04-25T10:11:01.666622+0000","last_peered":"2024-04-25T10:11:01.666622+0000","last_clean":"2024-04-25T10:11:01.666622+0000","last_became_active":"2024-04-25T10:10:59.659641+0000","last_became_peered":"2024-04-25T10:10:59.659641+0000","last_unstale":"2024-04-25T10:11:01.666622+0000","last_undegraded":"2024-04-25T10:11:01.666622+0000","last_fullsized":"2024-04-25T10:11:01.666622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:35.098559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790943+0000","last_change":"2024-04-25T10:11:00.666415+0000","last_active":"2024-04-25T10:11:01.790943+0000","last_peered":"2024-04-25T10:11:01.790943+0000","last_clean":"2024-04-25T10:11:01.790943+0000","last_became_active":"2024-04-25T10:11:00.666151+0000","last_became_peered":"2024-04-25T10:11:00.666151+0000","last_unstale":"2024-04-25T10:11:01.790943+0000","last_undegraded":"2024-04-25T10:11:01.790943+0000","last_fullsized":"2024-04-25T10:11:01.790943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:43.188026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669929+0000","last_change":"2024-04-25T10:10:59.654631+0000","last_active":"2024-04-25T10:11:01.669929+0000","last_peered":"2024-04-25T10:11:01.669929+0000","last_clean":"2024-04-25T10:11:01.669929+0000","last_became_active":"2024-04-25T10:10:59.654288+0000","last_became_peered":"2024-04-25T10:10:59.654288+0000","last_unstale":"2024-04-25T10:11:01.669929+0000","last_undegraded":"2024-04-25T10:11:01.669929+0000","last_fullsized":"2024-04-25T10:11:01.669929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:34.205125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669497+0000","last_change":"2024-04-25T10:11:00.677043+0000","last_active":"2024-04-25T10:11:01.669497+0000","last_peered":"2024-04-25T10:11:01.669497+0000","last_clean":"2024-04-25T10:11:01.669497+0000","last_became_active":"2024-04-25T10:11:00.675514+0000","last_became_peered":"2024-04-25T10:11:00.675514+0000","last_unstale":"2024-04-25T10:11:01.669497+0000","last_undegraded":"2024-04-25T10:11:01.669497+0000","last_fullsized":"2024-04-25T10:11:01.669497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:46.822338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T10:10:59.664299+0000","last_change":"2024-04-25T10:10:59.658687+0000","last_active":"2024-04-25T10:10:59.664299+0000","last_peered":"2024-04-25T10:10:59.664299+0000","last_clean":"2024-04-25T10:10:59.664299+0000","last_became_active":"2024-04-25T10:10:59.658380+0000","last_became_peered":"2024-04-25T10:10:59.658380+0000","last_unstale":"2024-04-25T10:10:59.664299+0000","last_undegraded":"2024-04-25T10:10:59.664299+0000","last_fullsized":"2024-04-25T10:10:59.664299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:14.321738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.652342+0000","last_change":"2024-04-25T10:10:59.652342+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.652342+0000","last_undegraded":"2024-04-25T10:10:59.652342+0000","last_fullsized":"2024-04-25T10:10:59.652342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790565+0000","last_change":"2024-04-25T10:10:59.658839+0000","last_active":"2024-04-25T10:11:01.790565+0000","last_peered":"2024-04-25T10:11:01.790565+0000","last_clean":"2024-04-25T10:11:01.790565+0000","last_became_active":"2024-04-25T10:10:59.658557+0000","last_became_peered":"2024-04-25T10:10:59.658557+0000","last_unstale":"2024-04-25T10:11:01.790565+0000","last_undegraded":"2024-04-25T10:11:01.790565+0000","last_fullsized":"2024-04-25T10:11:01.790565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:35.098559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666691+0000","last_change":"2024-04-25T10:11:00.670164+0000","last_active":"2024-04-25T10:11:01.666691+0000","last_peered":"2024-04-25T10:11:01.666691+0000","last_clean":"2024-04-25T10:11:01.666691+0000","last_became_active":"2024-04-25T10:11:00.669856+0000","last_became_peered":"2024-04-25T10:11:00.669856+0000","last_unstale":"2024-04-25T10:11:01.666691+0000","last_undegraded":"2024-04-25T10:11:01.666691+0000","last_fullsized":"2024-04-25T10:11:01.666691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:56.966365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666588+0000","last_change":"2024-04-25T10:10:59.660519+0000","last_active":"2024-04-25T10:11:01.666588+0000","last_peered":"2024-04-25T10:11:01.666588+0000","last_clean":"2024-04-25T10:11:01.666588+0000","last_became_active":"2024-04-25T10:10:59.659571+0000","last_became_peered":"2024-04-25T10:10:59.659571+0000","last_unstale":"2024-04-25T10:11:01.666588+0000","last_undegraded":"2024-04-25T10:11:01.666588+0000","last_fullsized":"2024-04-25T10:11:01.666588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:47.638024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669228+0000","last_change":"2024-04-25T10:11:00.672808+0000","last_active":"2024-04-25T10:11:01.669228+0000","last_peered":"2024-04-25T10:11:01.669228+0000","last_clean":"2024-04-25T10:11:01.669228+0000","last_became_active":"2024-04-25T10:11:00.672479+0000","last_became_peered":"2024-04-25T10:11:00.672479+0000","last_unstale":"2024-04-25T10:11:01.669228+0000","last_undegraded":"2024-04-25T10:11:01.669228+0000","last_fullsized":"2024-04-25T10:11:01.669228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:43.529739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791149+0000","last_change":"2024-04-25T10:10:59.662636+0000","last_active":"2024-04-25T10:11:01.791149+0000","last_peered":"2024-04-25T10:11:01.791149+0000","last_clean":"2024-04-25T10:11:01.791149+0000","last_became_active":"2024-04-25T10:10:59.661790+0000","last_became_peered":"2024-04-25T10:10:59.661790+0000","last_unstale":"2024-04-25T10:11:01.791149+0000","last_undegraded":"2024-04-25T10:11:01.791149+0000","last_fullsized":"2024-04-25T10:11:01.791149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:10.609819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791191+0000","last_change":"2024-04-25T10:11:00.671182+0000","last_active":"2024-04-25T10:11:01.791191+0000","last_peered":"2024-04-25T10:11:01.791191+0000","last_clean":"2024-04-25T10:11:01.791191+0000","last_became_active":"2024-04-25T10:11:00.670863+0000","last_became_peered":"2024-04-25T10:11:00.670863+0000","last_unstale":"2024-04-25T10:11:01.791191+0000","last_undegraded":"2024-04-25T10:11:01.791191+0000","last_fullsized":"2024-04-25T10:11:01.791191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:21.622898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791076+0000","last_change":"2024-04-25T10:10:59.661931+0000","last_active":"2024-04-25T10:11:01.791076+0000","last_peered":"2024-04-25T10:11:01.791076+0000","last_clean":"2024-04-25T10:11:01.791076+0000","last_became_active":"2024-04-25T10:10:59.661173+0000","last_became_peered":"2024-04-25T10:10:59.661173+0000","last_unstale":"2024-04-25T10:11:01.791076+0000","last_undegraded":"2024-04-25T10:11:01.791076+0000","last_fullsized":"2024-04-25T10:11:01.791076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:35.826027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791331+0000","last_change":"2024-04-25T10:11:00.676424+0000","last_active":"2024-04-25T10:11:01.791331+0000","last_peered":"2024-04-25T10:11:01.791331+0000","last_clean":"2024-04-25T10:11:01.791331+0000","last_became_active":"2024-04-25T10:11:00.676088+0000","last_became_peered":"2024-04-25T10:11:00.676088+0000","last_unstale":"2024-04-25T10:11:01.791331+0000","last_undegraded":"2024-04-25T10:11:01.791331+0000","last_fullsized":"2024-04-25T10:11:01.791331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:50.887593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669814+0000","last_change":"2024-04-25T10:10:59.660535+0000","last_active":"2024-04-25T10:11:01.669814+0000","last_peered":"2024-04-25T10:11:01.669814+0000","last_clean":"2024-04-25T10:11:01.669814+0000","last_became_active":"2024-04-25T10:10:59.660260+0000","last_became_peered":"2024-04-25T10:10:59.660260+0000","last_unstale":"2024-04-25T10:11:01.669814+0000","last_undegraded":"2024-04-25T10:11:01.669814+0000","last_fullsized":"2024-04-25T10:11:01.669814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:23.530134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669744+0000","last_change":"2024-04-25T10:11:00.684312+0000","last_active":"2024-04-25T10:11:01.669744+0000","last_peered":"2024-04-25T10:11:01.669744+0000","last_clean":"2024-04-25T10:11:01.669744+0000","last_became_active":"2024-04-25T10:11:00.683971+0000","last_became_peered":"2024-04-25T10:11:00.683971+0000","last_unstale":"2024-04-25T10:11:01.669744+0000","last_undegraded":"2024-04-25T10:11:01.669744+0000","last_fullsized":"2024-04-25T10:11:01.669744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:40.703987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666638+0000","last_change":"2024-04-25T10:10:59.658424+0000","last_active":"2024-04-25T10:11:01.666638+0000","last_peered":"2024-04-25T10:11:01.666638+0000","last_clean":"2024-04-25T10:11:01.666638+0000","last_became_active":"2024-04-25T10:10:59.658128+0000","last_became_peered":"2024-04-25T10:10:59.658128+0000","last_unstale":"2024-04-25T10:11:01.666638+0000","last_undegraded":"2024-04-25T10:11:01.666638+0000","last_fullsized":"2024-04-25T10:11:01.666638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:42.113345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669659+0000","last_change":"2024-04-25T10:11:00.678536+0000","last_active":"2024-04-25T10:11:01.669659+0000","last_peered":"2024-04-25T10:11:01.669659+0000","last_clean":"2024-04-25T10:11:01.669659+0000","last_became_active":"2024-04-25T10:11:00.678276+0000","last_became_peered":"2024-04-25T10:11:00.678276+0000","last_unstale":"2024-04-25T10:11:01.669659+0000","last_undegraded":"2024-04-25T10:11:01.669659+0000","last_fullsized":"2024-04-25T10:11:01.669659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:48.642917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"85'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669547+0000","last_change":"2024-04-25T10:10:59.653278+0000","last_active":"2024-04-25T10:11:01.669547+0000","last_peered":"2024-04-25T10:11:01.669547+0000","last_clean":"2024-04-25T10:11:01.669547+0000","last_became_active":"2024-04-25T10:10:59.652974+0000","last_became_peered":"2024-04-25T10:10:59.652974+0000","last_unstale":"2024-04-25T10:11:01.669547+0000","last_undegraded":"2024-04-25T10:11:01.669547+0000","last_fullsized":"2024-04-25T10:11:01.669547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:45.820044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791449+0000","last_change":"2024-04-25T10:11:00.684587+0000","last_active":"2024-04-25T10:11:01.791449+0000","last_peered":"2024-04-25T10:11:01.791449+0000","last_clean":"2024-04-25T10:11:01.791449+0000","last_became_active":"2024-04-25T10:11:00.684306+0000","last_became_peered":"2024-04-25T10:11:00.684306+0000","last_unstale":"2024-04-25T10:11:01.791449+0000","last_undegraded":"2024-04-25T10:11:01.791449+0000","last_fullsized":"2024-04-25T10:11:01.791449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:39.955023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"85'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790926+0000","last_change":"2024-04-25T10:10:59.662733+0000","last_active":"2024-04-25T10:11:01.790926+0000","last_peered":"2024-04-25T10:11:01.790926+0000","last_clean":"2024-04-25T10:11:01.790926+0000","last_became_active":"2024-04-25T10:10:59.662290+0000","last_became_peered":"2024-04-25T10:10:59.662290+0000","last_unstale":"2024-04-25T10:11:01.790926+0000","last_undegraded":"2024-04-25T10:11:01.790926+0000","last_fullsized":"2024-04-25T10:11:01.790926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:34.280071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669718+0000","last_change":"2024-04-25T10:11:00.669181+0000","last_active":"2024-04-25T10:11:01.669718+0000","last_peered":"2024-04-25T10:11:01.669718+0000","last_clean":"2024-04-25T10:11:01.669718+0000","last_became_active":"2024-04-25T10:11:00.668758+0000","last_became_peered":"2024-04-25T10:11:00.668758+0000","last_unstale":"2024-04-25T10:11:01.669718+0000","last_undegraded":"2024-04-25T10:11:01.669718+0000","last_fullsized":"2024-04-25T10:11:01.669718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:38.681757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T10:10:59.666667+0000","last_change":"2024-04-25T10:10:59.662230+0000","last_active":"2024-04-25T10:10:59.666667+0000","last_peered":"2024-04-25T10:10:59.666667+0000","last_clean":"2024-04-25T10:10:59.666667+0000","last_became_active":"2024-04-25T10:10:59.661953+0000","last_became_peered":"2024-04-25T10:10:59.661953+0000","last_unstale":"2024-04-25T10:10:59.666667+0000","last_undegraded":"2024-04-25T10:10:59.666667+0000","last_fullsized":"2024-04-25T10:10:59.666667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:31.766942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669387+0000","last_change":"2024-04-25T10:11:00.680010+0000","last_active":"2024-04-25T10:11:01.669387+0000","last_peered":"2024-04-25T10:11:01.669387+0000","last_clean":"2024-04-25T10:11:01.669387+0000","last_became_active":"2024-04-25T10:11:00.679712+0000","last_became_peered":"2024-04-25T10:11:00.679712+0000","last_unstale":"2024-04-25T10:11:01.669387+0000","last_undegraded":"2024-04-25T10:11:01.669387+0000","last_fullsized":"2024-04-25T10:11:01.669387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:22.568376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672076+0000","last_change":"2024-04-25T10:10:59.659677+0000","last_active":"2024-04-25T10:11:01.672076+0000","last_peered":"2024-04-25T10:11:01.672076+0000","last_clean":"2024-04-25T10:11:01.672076+0000","last_became_active":"2024-04-25T10:10:59.659395+0000","last_became_peered":"2024-04-25T10:10:59.659395+0000","last_unstale":"2024-04-25T10:11:01.672076+0000","last_undegraded":"2024-04-25T10:11:01.672076+0000","last_fullsized":"2024-04-25T10:11:01.672076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:21.670434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790957+0000","last_change":"2024-04-25T10:11:00.678832+0000","last_active":"2024-04-25T10:11:01.790957+0000","last_peered":"2024-04-25T10:11:01.790957+0000","last_clean":"2024-04-25T10:11:01.790957+0000","last_became_active":"2024-04-25T10:11:00.678560+0000","last_became_peered":"2024-04-25T10:11:00.678560+0000","last_unstale":"2024-04-25T10:11:01.790957+0000","last_undegraded":"2024-04-25T10:11:01.790957+0000","last_fullsized":"2024-04-25T10:11:01.790957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:19.719591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666238+0000","last_change":"2024-04-25T10:10:59.656540+0000","last_active":"2024-04-25T10:11:01.666238+0000","last_peered":"2024-04-25T10:11:01.666238+0000","last_clean":"2024-04-25T10:11:01.666238+0000","last_became_active":"2024-04-25T10:10:59.656276+0000","last_became_peered":"2024-04-25T10:10:59.656276+0000","last_unstale":"2024-04-25T10:11:01.666238+0000","last_undegraded":"2024-04-25T10:11:01.666238+0000","last_fullsized":"2024-04-25T10:11:01.666238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:13.892230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669548+0000","last_change":"2024-04-25T10:11:00.680212+0000","last_active":"2024-04-25T10:11:01.669548+0000","last_peered":"2024-04-25T10:11:01.669548+0000","last_clean":"2024-04-25T10:11:01.669548+0000","last_became_active":"2024-04-25T10:11:00.679944+0000","last_became_peered":"2024-04-25T10:11:00.679944+0000","last_unstale":"2024-04-25T10:11:01.669548+0000","last_undegraded":"2024-04-25T10:11:01.669548+0000","last_fullsized":"2024-04-25T10:11:01.669548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:14.897123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"85'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669582+0000","last_change":"2024-04-25T10:10:59.656545+0000","last_active":"2024-04-25T10:11:01.669582+0000","last_peered":"2024-04-25T10:11:01.669582+0000","last_clean":"2024-04-25T10:11:01.669582+0000","last_became_active":"2024-04-25T10:10:59.656262+0000","last_became_peered":"2024-04-25T10:10:59.656262+0000","last_unstale":"2024-04-25T10:11:01.669582+0000","last_undegraded":"2024-04-25T10:11:01.669582+0000","last_fullsized":"2024-04-25T10:11:01.669582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:12.740681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790809+0000","last_change":"2024-04-25T10:11:00.660578+0000","last_active":"2024-04-25T10:11:01.790809+0000","last_peered":"2024-04-25T10:11:01.790809+0000","last_clean":"2024-04-25T10:11:01.790809+0000","last_became_active":"2024-04-25T10:11:00.660258+0000","last_became_peered":"2024-04-25T10:11:00.660258+0000","last_unstale":"2024-04-25T10:11:01.790809+0000","last_undegraded":"2024-04-25T10:11:01.790809+0000","last_fullsized":"2024-04-25T10:11:01.790809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:20.151958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791898+0000","last_change":"2024-04-25T10:10:59.660067+0000","last_active":"2024-04-25T10:11:01.791898+0000","last_peered":"2024-04-25T10:11:01.791898+0000","last_clean":"2024-04-25T10:11:01.791898+0000","last_became_active":"2024-04-25T10:10:59.659717+0000","last_became_peered":"2024-04-25T10:10:59.659717+0000","last_unstale":"2024-04-25T10:11:01.791898+0000","last_undegraded":"2024-04-25T10:11:01.791898+0000","last_fullsized":"2024-04-25T10:11:01.791898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:55.225255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666704+0000","last_change":"2024-04-25T10:11:00.664267+0000","last_active":"2024-04-25T10:11:01.666704+0000","last_peered":"2024-04-25T10:11:01.666704+0000","last_clean":"2024-04-25T10:11:01.666704+0000","last_became_active":"2024-04-25T10:11:00.664038+0000","last_became_peered":"2024-04-25T10:11:00.664038+0000","last_unstale":"2024-04-25T10:11:01.666704+0000","last_undegraded":"2024-04-25T10:11:01.666704+0000","last_fullsized":"2024-04-25T10:11:01.666704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:20.151958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791555+0000","last_change":"2024-04-25T10:10:59.661418+0000","last_active":"2024-04-25T10:11:01.791555+0000","last_peered":"2024-04-25T10:11:01.791555+0000","last_clean":"2024-04-25T10:11:01.791555+0000","last_became_active":"2024-04-25T10:10:59.661121+0000","last_became_peered":"2024-04-25T10:10:59.661121+0000","last_unstale":"2024-04-25T10:11:01.791555+0000","last_undegraded":"2024-04-25T10:11:01.791555+0000","last_fullsized":"2024-04-25T10:11:01.791555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:53.008632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790835+0000","last_change":"2024-04-25T10:11:00.662883+0000","last_active":"2024-04-25T10:11:01.790835+0000","last_peered":"2024-04-25T10:11:01.790835+0000","last_clean":"2024-04-25T10:11:01.790835+0000","last_became_active":"2024-04-25T10:11:00.662578+0000","last_became_peered":"2024-04-25T10:11:00.662578+0000","last_unstale":"2024-04-25T10:11:01.790835+0000","last_undegraded":"2024-04-25T10:11:01.790835+0000","last_fullsized":"2024-04-25T10:11:01.790835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:55.212072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791634+0000","last_change":"2024-04-25T10:10:59.663404+0000","last_active":"2024-04-25T10:11:01.791634+0000","last_peered":"2024-04-25T10:11:01.791634+0000","last_clean":"2024-04-25T10:11:01.791634+0000","last_became_active":"2024-04-25T10:10:59.663138+0000","last_became_peered":"2024-04-25T10:10:59.663138+0000","last_unstale":"2024-04-25T10:11:01.791634+0000","last_undegraded":"2024-04-25T10:11:01.791634+0000","last_fullsized":"2024-04-25T10:11:01.791634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:38.050152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672109+0000","last_change":"2024-04-25T10:11:00.672494+0000","last_active":"2024-04-25T10:11:01.672109+0000","last_peered":"2024-04-25T10:11:01.672109+0000","last_clean":"2024-04-25T10:11:01.672109+0000","last_became_active":"2024-04-25T10:11:00.672238+0000","last_became_peered":"2024-04-25T10:11:00.672238+0000","last_unstale":"2024-04-25T10:11:01.672109+0000","last_undegraded":"2024-04-25T10:11:01.672109+0000","last_fullsized":"2024-04-25T10:11:01.672109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:48.642917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791130+0000","last_change":"2024-04-25T10:10:59.662706+0000","last_active":"2024-04-25T10:11:01.791130+0000","last_peered":"2024-04-25T10:11:01.791130+0000","last_clean":"2024-04-25T10:11:01.791130+0000","last_became_active":"2024-04-25T10:10:59.662399+0000","last_became_peered":"2024-04-25T10:10:59.662399+0000","last_unstale":"2024-04-25T10:11:01.791130+0000","last_undegraded":"2024-04-25T10:11:01.791130+0000","last_fullsized":"2024-04-25T10:11:01.791130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:56.481855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791760+0000","last_change":"2024-04-25T10:11:00.684360+0000","last_active":"2024-04-25T10:11:01.791760+0000","last_peered":"2024-04-25T10:11:01.791760+0000","last_clean":"2024-04-25T10:11:01.791760+0000","last_became_active":"2024-04-25T10:11:00.683915+0000","last_became_peered":"2024-04-25T10:11:00.683915+0000","last_unstale":"2024-04-25T10:11:01.791760+0000","last_undegraded":"2024-04-25T10:11:01.791760+0000","last_fullsized":"2024-04-25T10:11:01.791760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:44.016510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672207+0000","last_change":"2024-04-25T10:10:59.664474+0000","last_active":"2024-04-25T10:11:01.672207+0000","last_peered":"2024-04-25T10:11:01.672207+0000","last_clean":"2024-04-25T10:11:01.672207+0000","last_became_active":"2024-04-25T10:10:59.664182+0000","last_became_peered":"2024-04-25T10:10:59.664182+0000","last_unstale":"2024-04-25T10:11:01.672207+0000","last_undegraded":"2024-04-25T10:11:01.672207+0000","last_fullsized":"2024-04-25T10:11:01.672207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:51.674567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.653171+0000","last_change":"2024-04-25T10:10:59.653171+0000","last_active":"2024-04-25T10:10:59.639238+0000","last_peered":"2024-04-25T10:10:59.639238+0000","last_clean":"2024-04-25T10:10:59.639238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.653171+0000","last_undegraded":"2024-04-25T10:10:59.653171+0000","last_fullsized":"2024-04-25T10:10:59.653171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669493+0000","last_change":"2024-04-25T10:10:59.656426+0000","last_active":"2024-04-25T10:11:01.669493+0000","last_peered":"2024-04-25T10:11:01.669493+0000","last_clean":"2024-04-25T10:11:01.669493+0000","last_became_active":"2024-04-25T10:10:59.656089+0000","last_became_peered":"2024-04-25T10:10:59.656089+0000","last_unstale":"2024-04-25T10:11:01.669493+0000","last_undegraded":"2024-04-25T10:11:01.669493+0000","last_fullsized":"2024-04-25T10:11:01.669493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:50.774285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790576+0000","last_change":"2024-04-25T10:11:00.675256+0000","last_active":"2024-04-25T10:11:01.790576+0000","last_peered":"2024-04-25T10:11:01.790576+0000","last_clean":"2024-04-25T10:11:01.790576+0000","last_became_active":"2024-04-25T10:11:00.674969+0000","last_became_peered":"2024-04-25T10:11:00.674969+0000","last_unstale":"2024-04-25T10:11:01.790576+0000","last_undegraded":"2024-04-25T10:11:01.790576+0000","last_fullsized":"2024-04-25T10:11:01.790576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.860030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791546+0000","last_change":"2024-04-25T10:10:59.662798+0000","last_active":"2024-04-25T10:11:01.791546+0000","last_peered":"2024-04-25T10:11:01.791546+0000","last_clean":"2024-04-25T10:11:01.791546+0000","last_became_active":"2024-04-25T10:10:59.661497+0000","last_became_peered":"2024-04-25T10:10:59.661497+0000","last_unstale":"2024-04-25T10:11:01.791546+0000","last_undegraded":"2024-04-25T10:11:01.791546+0000","last_fullsized":"2024-04-25T10:11:01.791546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:29.530495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791719+0000","last_change":"2024-04-25T10:11:00.666183+0000","last_active":"2024-04-25T10:11:01.791719+0000","last_peered":"2024-04-25T10:11:01.791719+0000","last_clean":"2024-04-25T10:11:01.791719+0000","last_became_active":"2024-04-25T10:11:00.665905+0000","last_became_peered":"2024-04-25T10:11:00.665905+0000","last_unstale":"2024-04-25T10:11:01.791719+0000","last_undegraded":"2024-04-25T10:11:01.791719+0000","last_fullsized":"2024-04-25T10:11:01.791719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.427341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791772+0000","last_change":"2024-04-25T10:10:59.661863+0000","last_active":"2024-04-25T10:11:01.791772+0000","last_peered":"2024-04-25T10:11:01.791772+0000","last_clean":"2024-04-25T10:11:01.791772+0000","last_became_active":"2024-04-25T10:10:59.661598+0000","last_became_peered":"2024-04-25T10:10:59.661598+0000","last_unstale":"2024-04-25T10:11:01.791772+0000","last_undegraded":"2024-04-25T10:11:01.791772+0000","last_fullsized":"2024-04-25T10:11:01.791772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:45.820044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791785+0000","last_change":"2024-04-25T10:11:00.666115+0000","last_active":"2024-04-25T10:11:01.791785+0000","last_peered":"2024-04-25T10:11:01.791785+0000","last_clean":"2024-04-25T10:11:01.791785+0000","last_became_active":"2024-04-25T10:11:00.665817+0000","last_became_peered":"2024-04-25T10:11:00.665817+0000","last_unstale":"2024-04-25T10:11:01.791785+0000","last_undegraded":"2024-04-25T10:11:01.791785+0000","last_fullsized":"2024-04-25T10:11:01.791785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:10.823889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.670527+0000","last_change":"2024-04-25T10:10:59.654401+0000","last_active":"2024-04-25T10:11:01.670527+0000","last_peered":"2024-04-25T10:11:01.670527+0000","last_clean":"2024-04-25T10:11:01.670527+0000","last_became_active":"2024-04-25T10:10:59.654067+0000","last_became_peered":"2024-04-25T10:10:59.654067+0000","last_unstale":"2024-04-25T10:11:01.670527+0000","last_undegraded":"2024-04-25T10:11:01.670527+0000","last_fullsized":"2024-04-25T10:11:01.670527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:45.817445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791696+0000","last_change":"2024-04-25T10:11:00.671112+0000","last_active":"2024-04-25T10:11:01.791696+0000","last_peered":"2024-04-25T10:11:01.791696+0000","last_clean":"2024-04-25T10:11:01.791696+0000","last_became_active":"2024-04-25T10:11:00.670792+0000","last_became_peered":"2024-04-25T10:11:00.670792+0000","last_unstale":"2024-04-25T10:11:01.791696+0000","last_undegraded":"2024-04-25T10:11:01.791696+0000","last_fullsized":"2024-04-25T10:11:01.791696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:00.467722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790958+0000","last_change":"2024-04-25T10:10:59.660751+0000","last_active":"2024-04-25T10:11:01.790958+0000","last_peered":"2024-04-25T10:11:01.790958+0000","last_clean":"2024-04-25T10:11:01.790958+0000","last_became_active":"2024-04-25T10:10:59.660439+0000","last_became_peered":"2024-04-25T10:10:59.660439+0000","last_unstale":"2024-04-25T10:11:01.790958+0000","last_undegraded":"2024-04-25T10:11:01.790958+0000","last_fullsized":"2024-04-25T10:11:01.790958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:17.523240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666630+0000","last_change":"2024-04-25T10:11:00.662687+0000","last_active":"2024-04-25T10:11:01.666630+0000","last_peered":"2024-04-25T10:11:01.666630+0000","last_clean":"2024-04-25T10:11:01.666630+0000","last_became_active":"2024-04-25T10:11:00.662430+0000","last_became_peered":"2024-04-25T10:11:00.662430+0000","last_unstale":"2024-04-25T10:11:01.666630+0000","last_undegraded":"2024-04-25T10:11:01.666630+0000","last_fullsized":"2024-04-25T10:11:01.666630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:14.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666396+0000","last_change":"2024-04-25T10:10:59.660156+0000","last_active":"2024-04-25T10:11:01.666396+0000","last_peered":"2024-04-25T10:11:01.666396+0000","last_clean":"2024-04-25T10:11:01.666396+0000","last_became_active":"2024-04-25T10:10:59.659878+0000","last_became_peered":"2024-04-25T10:10:59.659878+0000","last_unstale":"2024-04-25T10:11:01.666396+0000","last_undegraded":"2024-04-25T10:11:01.666396+0000","last_fullsized":"2024-04-25T10:11:01.666396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:55.986122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669254+0000","last_change":"2024-04-25T10:11:00.664855+0000","last_active":"2024-04-25T10:11:01.669254+0000","last_peered":"2024-04-25T10:11:01.669254+0000","last_clean":"2024-04-25T10:11:01.669254+0000","last_became_active":"2024-04-25T10:11:00.664174+0000","last_became_peered":"2024-04-25T10:11:00.664174+0000","last_unstale":"2024-04-25T10:11:01.669254+0000","last_undegraded":"2024-04-25T10:11:01.669254+0000","last_fullsized":"2024-04-25T10:11:01.669254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.235679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672299+0000","last_change":"2024-04-25T10:10:59.656397+0000","last_active":"2024-04-25T10:11:01.672299+0000","last_peered":"2024-04-25T10:11:01.672299+0000","last_clean":"2024-04-25T10:11:01.672299+0000","last_became_active":"2024-04-25T10:10:59.656114+0000","last_became_peered":"2024-04-25T10:10:59.656114+0000","last_unstale":"2024-04-25T10:11:01.672299+0000","last_undegraded":"2024-04-25T10:11:01.672299+0000","last_fullsized":"2024-04-25T10:11:01.672299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:39.088943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669867+0000","last_change":"2024-04-25T10:11:00.675967+0000","last_active":"2024-04-25T10:11:01.669867+0000","last_peered":"2024-04-25T10:11:01.669867+0000","last_clean":"2024-04-25T10:11:01.669867+0000","last_became_active":"2024-04-25T10:11:00.673547+0000","last_became_peered":"2024-04-25T10:11:00.673547+0000","last_unstale":"2024-04-25T10:11:01.669867+0000","last_undegraded":"2024-04-25T10:11:01.669867+0000","last_fullsized":"2024-04-25T10:11:01.669867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:43.118238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666689+0000","last_change":"2024-04-25T10:10:59.661201+0000","last_active":"2024-04-25T10:11:01.666689+0000","last_peered":"2024-04-25T10:11:01.666689+0000","last_clean":"2024-04-25T10:11:01.666689+0000","last_became_active":"2024-04-25T10:10:59.660913+0000","last_became_peered":"2024-04-25T10:10:59.660913+0000","last_unstale":"2024-04-25T10:11:01.666689+0000","last_undegraded":"2024-04-25T10:11:01.666689+0000","last_fullsized":"2024-04-25T10:11:01.666689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:37.676864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669208+0000","last_change":"2024-04-25T10:11:00.664505+0000","last_active":"2024-04-25T10:11:01.669208+0000","last_peered":"2024-04-25T10:11:01.669208+0000","last_clean":"2024-04-25T10:11:01.669208+0000","last_became_active":"2024-04-25T10:11:00.664218+0000","last_became_peered":"2024-04-25T10:11:00.664218+0000","last_unstale":"2024-04-25T10:11:01.669208+0000","last_undegraded":"2024-04-25T10:11:01.669208+0000","last_fullsized":"2024-04-25T10:11:01.669208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.603074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669199+0000","last_change":"2024-04-25T10:10:59.652758+0000","last_active":"2024-04-25T10:11:01.669199+0000","last_peered":"2024-04-25T10:11:01.669199+0000","last_clean":"2024-04-25T10:11:01.669199+0000","last_became_active":"2024-04-25T10:10:59.652456+0000","last_became_peered":"2024-04-25T10:10:59.652456+0000","last_unstale":"2024-04-25T10:11:01.669199+0000","last_undegraded":"2024-04-25T10:11:01.669199+0000","last_fullsized":"2024-04-25T10:11:01.669199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:53.008632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669239+0000","last_change":"2024-04-25T10:11:00.660736+0000","last_active":"2024-04-25T10:11:01.669239+0000","last_peered":"2024-04-25T10:11:01.669239+0000","last_clean":"2024-04-25T10:11:01.669239+0000","last_became_active":"2024-04-25T10:11:00.660471+0000","last_became_peered":"2024-04-25T10:11:00.660471+0000","last_unstale":"2024-04-25T10:11:01.669239+0000","last_undegraded":"2024-04-25T10:11:01.669239+0000","last_fullsized":"2024-04-25T10:11:01.669239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:37.071670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790521+0000","last_change":"2024-04-25T10:10:59.660087+0000","last_active":"2024-04-25T10:11:01.790521+0000","last_peered":"2024-04-25T10:11:01.790521+0000","last_clean":"2024-04-25T10:11:01.790521+0000","last_became_active":"2024-04-25T10:10:59.659815+0000","last_became_peered":"2024-04-25T10:10:59.659815+0000","last_unstale":"2024-04-25T10:11:01.790521+0000","last_undegraded":"2024-04-25T10:11:01.790521+0000","last_fullsized":"2024-04-25T10:11:01.790521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:42.113345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669804+0000","last_change":"2024-04-25T10:11:00.674881+0000","last_active":"2024-04-25T10:11:01.669804+0000","last_peered":"2024-04-25T10:11:01.669804+0000","last_clean":"2024-04-25T10:11:01.669804+0000","last_became_active":"2024-04-25T10:11:00.674632+0000","last_became_peered":"2024-04-25T10:11:00.674632+0000","last_unstale":"2024-04-25T10:11:01.669804+0000","last_undegraded":"2024-04-25T10:11:01.669804+0000","last_fullsized":"2024-04-25T10:11:01.669804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:36.103452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791702+0000","last_change":"2024-04-25T10:10:59.663122+0000","last_active":"2024-04-25T10:11:01.791702+0000","last_peered":"2024-04-25T10:11:01.791702+0000","last_clean":"2024-04-25T10:11:01.791702+0000","last_became_active":"2024-04-25T10:10:59.662664+0000","last_became_peered":"2024-04-25T10:10:59.662664+0000","last_unstale":"2024-04-25T10:11:01.791702+0000","last_undegraded":"2024-04-25T10:11:01.791702+0000","last_fullsized":"2024-04-25T10:11:01.791702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:16.256009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":301,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669306+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:11:01.669306+0000","last_peered":"2024-04-25T10:11:01.669306+0000","last_clean":"2024-04-25T10:11:01.669306+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:11:01.669306+0000","last_undegraded":"2024-04-25T10:11:01.669306+0000","last_fullsized":"2024-04-25T10:11:01.669306+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1052672,"data_stored":446360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509516,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1056768,"data_stored":446962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509517,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1052672,"data_stored":446324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335460,"statfs":{"total":96636764160,"available":96599511040,"internally_reserved":0,"allocated":1060864,"data_stored":446964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509518,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335472,"statfs":{"total":96636764160,"available":96599523328,"internally_reserved":0,"allocated":1048576,"data_stored":446290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509516,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1646592,"data_stored":1036748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509518,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1064960,"data_stored":447924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1605632,"data_stored":1017308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:08.077 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:08.080 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:08.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:08.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.226+0000 7fe2e80b3640 1 Processor -- start 2024-04-25T10:11:08.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.227+0000 7fe2e80b3640 1 -- start start 2024-04-25T10:11:08.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.227+0000 7fe2e80b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2e015dc30 0x7fe2e006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.227+0000 7fe2e80b3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe2e00b1b30 0x7fe2e00d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.228+0000 7fe2e80b3640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e006c1f0 con 0x7fe2e006cc00 2024-04-25T10:11:08.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.228+0000 7fe2e80b3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e00cfb40 con 0x7fe2e00b1b30 2024-04-25T10:11:08.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.228+0000 7fe2e80b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00b2060 con 0x7fe2e015dc30 2024-04-25T10:11:08.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.228+0000 7fe2e6650640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe2e00b1b30 0x7fe2e00d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.228+0000 7fe2e6650640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe2e00b1b30 0x7fe2e00d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53286/0 (socket says 172.21.15.16:53286) 2024-04-25T10:11:08.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.228+0000 7fe2e6650640 1 -- 172.21.15.16:0/912371380 learned_addr learned my addr 172.21.15.16:0/912371380 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:08.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.228+0000 7fe2e6650640 1 -- 172.21.15.16:0/912371380 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2e015dc30 msgr2=0x7fe2e006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:08.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.228+0000 7fe2e6650640 1 --2- 172.21.15.16:0/912371380 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2e015dc30 0x7fe2e006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.228+0000 7fe2e6650640 1 -- 172.21.15.16:0/912371380 >> v1:172.21.15.186:6789/0 conn(0x7fe2e006cc00 legacy=0x7fe2e00b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:08.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.228+0000 7fe2e6650640 1 -- 172.21.15.16:0/912371380 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e00b2340 con 0x7fe2e00b1b30 2024-04-25T10:11:08.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.229+0000 7fe2e6650640 1 --2- 172.21.15.16:0/912371380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e00b1b30 0x7fe2e00d0300 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8009e50 tx=0x7fe2d802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ac1e4a2d445c16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.229+0000 7fe2e4e4d640 1 -- 172.21.15.16:0/912371380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8036070 con 0x7fe2e00b1b30 2024-04-25T10:11:08.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.229+0000 7fe2e4e4d640 1 -- 172.21.15.16:0/912371380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d8031600 con 0x7fe2e00b1b30 2024-04-25T10:11:08.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.229+0000 7fe2e4e4d640 1 -- 172.21.15.16:0/912371380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d80319d0 con 0x7fe2e00b1b30 2024-04-25T10:11:08.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.229+0000 7fe2e80b3640 1 -- 172.21.15.16:0/912371380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e00b1b30 msgr2=0x7fe2e00d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.229+0000 7fe2e80b3640 1 --2- 172.21.15.16:0/912371380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e00b1b30 0x7fe2e00d0300 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8009e50 tx=0x7fe2d802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.230+0000 7fe2e80b3640 1 -- 172.21.15.16:0/912371380 shutdown_connections 2024-04-25T10:11:08.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.230+0000 7fe2e80b3640 1 --2- 172.21.15.16:0/912371380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e00b1b30 0x7fe2e00d0300 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.230+0000 7fe2e80b3640 1 --2- 172.21.15.16:0/912371380 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2e015dc30 0x7fe2e006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.230+0000 7fe2e80b3640 1 -- 172.21.15.16:0/912371380 >> 172.21.15.16:0/912371380 conn(0x7fe2e00d1900 msgr2=0x7fe2e00d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:08.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.230+0000 7fe2e80b3640 1 -- 172.21.15.16:0/912371380 shutdown_connections 2024-04-25T10:11:08.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.230+0000 7fe2e80b3640 1 -- 172.21.15.16:0/912371380 wait complete. 2024-04-25T10:11:08.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.231+0000 7fe2e80b3640 1 Processor -- start 2024-04-25T10:11:08.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.231+0000 7fe2e80b3640 1 -- start start 2024-04-25T10:11:08.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.231+0000 7fe2e80b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e006cc00 0x7fe2e01f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e80b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2e00b1b30 0x7fe2e015e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e80b3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2e015dc30 0x7fe2e00c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e80b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e015e010 con 0x7fe2e006cc00 2024-04-25T10:11:08.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e80b3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00cfde0 con 0x7fe2e015dc30 2024-04-25T10:11:08.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e80b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00d0840 con 0x7fe2e00b1b30 2024-04-25T10:11:08.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e5e4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e006cc00 0x7fe2e01f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e5e4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e006cc00 0x7fe2e01f46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53300/0 (socket says 172.21.15.16:53300) 2024-04-25T10:11:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e5e4f640 1 -- 172.21.15.16:0/552310984 learned_addr learned my addr 172.21.15.16:0/552310984 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e564e640 1 --2- 172.21.15.16:0/552310984 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2e00b1b30 0x7fe2e015e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e6650640 1 --2- 172.21.15.16:0/552310984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2e015dc30 0x7fe2e00c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e5e4f640 1 -- 172.21.15.16:0/552310984 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2e00b1b30 msgr2=0x7fe2e015e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e5e4f640 1 --2- 172.21.15.16:0/552310984 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2e00b1b30 0x7fe2e015e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e5e4f640 1 -- 172.21.15.16:0/552310984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2e015dc30 msgr2=0x7fe2e00c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e5e4f640 1 --2- 172.21.15.16:0/552310984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2e015dc30 0x7fe2e00c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e5e4f640 1 -- 172.21.15.16:0/552310984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e015e6d0 con 0x7fe2e006cc00 2024-04-25T10:11:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e564e640 1 --2- 172.21.15.16:0/552310984 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2e00b1b30 0x7fe2e015e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e5e4f640 1 --2- 172.21.15.16:0/552310984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e006cc00 0x7fe2e01f46a0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00b410 tx=0x7fe2dc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2d67fc640 1 -- 172.21.15.16:0/552310984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc015020 con 0x7fe2e006cc00 2024-04-25T10:11:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2d67fc640 1 -- 172.21.15.16:0/552310984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2dc011850 con 0x7fe2e006cc00 2024-04-25T10:11:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e80b3640 1 -- 172.21.15.16:0/552310984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e015e9a0 con 0x7fe2e006cc00 2024-04-25T10:11:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2d67fc640 1 -- 172.21.15.16:0/552310984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc011c40 con 0x7fe2e006cc00 2024-04-25T10:11:08.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.232+0000 7fe2e80b3640 1 -- 172.21.15.16:0/552310984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e015ef60 con 0x7fe2e006cc00 2024-04-25T10:11:08.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.233+0000 7fe2e80b3640 1 -- 172.21.15.16:0/552310984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e00749b0 con 0x7fe2e006cc00 2024-04-25T10:11:08.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.234+0000 7fe2d67fc640 1 -- 172.21.15.16:0/552310984 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2dc018020 con 0x7fe2e006cc00 2024-04-25T10:11:08.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.234+0000 7fe2d67fc640 1 --2- 172.21.15.16:0/552310984 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe2c406b4d0 0x7fe2c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.234+0000 7fe2d67fc640 1 -- 172.21.15.16:0/552310984 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2dc08a130 con 0x7fe2e006cc00 2024-04-25T10:11:08.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.235+0000 7fe2e564e640 1 --2- 172.21.15.16:0/552310984 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe2c406b4d0 0x7fe2c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.235+0000 7fe2e564e640 1 --2- 172.21.15.16:0/552310984 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe2c406b4d0 0x7fe2c406d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0007fd0 tx=0x7fe2d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.239+0000 7fe2d67fc640 1 -- 172.21.15.16:0/552310984 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2dc01f020 con 0x7fe2e006cc00 2024-04-25T10:11:08.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.415+0000 7fe2e80b3640 1 -- 172.21.15.16:0/552310984 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe2e006e330 con 0x7fe2c406b4d0 2024-04-25T10:11:08.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.421+0000 7fe2d67fc640 1 -- 172.21.15.16:0/552310984 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251996 (secure 0 0 0) 0x7fe2e006e330 con 0x7fe2c406b4d0 2024-04-25T10:11:08.429 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:08.430 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:11:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.424+0000 7fe2e80b3640 1 -- 172.21.15.16:0/552310984 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe2c406b4d0 msgr2=0x7fe2c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.424+0000 7fe2e80b3640 1 --2- 172.21.15.16:0/552310984 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe2c406b4d0 0x7fe2c406d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0007fd0 tx=0x7fe2d0009040 comp rx=0 tx=0).stop 2024-04-25T10:11:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.424+0000 7fe2e80b3640 1 -- 172.21.15.16:0/552310984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e006cc00 msgr2=0x7fe2e01f46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.424+0000 7fe2e80b3640 1 --2- 172.21.15.16:0/552310984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e006cc00 0x7fe2e01f46a0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00b410 tx=0x7fe2dc00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.424+0000 7fe2e80b3640 1 -- 172.21.15.16:0/552310984 shutdown_connections 2024-04-25T10:11:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.424+0000 7fe2e80b3640 1 --2- 172.21.15.16:0/552310984 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe2c406b4d0 0x7fe2c406d990 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.424+0000 7fe2e80b3640 1 --2- 172.21.15.16:0/552310984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2e015dc30 0x7fe2e00c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.424+0000 7fe2e80b3640 1 --2- 172.21.15.16:0/552310984 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2e00b1b30 0x7fe2e015e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.424+0000 7fe2e80b3640 1 --2- 172.21.15.16:0/552310984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e006cc00 0x7fe2e01f46a0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.424+0000 7fe2e80b3640 1 -- 172.21.15.16:0/552310984 >> 172.21.15.16:0/552310984 conn(0x7fe2e00d1900 msgr2=0x7fe2e00b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.424+0000 7fe2e80b3640 1 -- 172.21.15.16:0/552310984 shutdown_connections 2024-04-25T10:11:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.424+0000 7fe2e80b3640 1 -- 172.21.15.16:0/552310984 wait complete. 2024-04-25T10:11:08.440 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-04-25T10:11:07.868120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":267208,"kb_used_data":9416,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754707512,"statfs":{"total":773094113280,"available":772820492288,"internally_reserved":0,"allocated":9641984,"data_stored":4755876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001956"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669584+0000","last_change":"2024-04-25T10:11:00.665795+0000","last_active":"2024-04-25T10:11:01.669584+0000","last_peered":"2024-04-25T10:11:01.669584+0000","last_clean":"2024-04-25T10:11:01.669584+0000","last_became_active":"2024-04-25T10:11:00.664672+0000","last_became_peered":"2024-04-25T10:11:00.664672+0000","last_unstale":"2024-04-25T10:11:01.669584+0000","last_undegraded":"2024-04-25T10:11:01.669584+0000","last_fullsized":"2024-04-25T10:11:01.669584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:30.244884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672240+0000","last_change":"2024-04-25T10:11:00.658574+0000","last_active":"2024-04-25T10:11:01.672240+0000","last_peered":"2024-04-25T10:11:01.672240+0000","last_clean":"2024-04-25T10:11:01.672240+0000","last_became_active":"2024-04-25T10:11:00.658280+0000","last_became_peered":"2024-04-25T10:11:00.658280+0000","last_unstale":"2024-04-25T10:11:01.672240+0000","last_undegraded":"2024-04-25T10:11:01.672240+0000","last_fullsized":"2024-04-25T10:11:01.672240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:38.681757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666472+0000","last_change":"2024-04-25T10:11:00.664492+0000","last_active":"2024-04-25T10:11:01.666472+0000","last_peered":"2024-04-25T10:11:01.666472+0000","last_clean":"2024-04-25T10:11:01.666472+0000","last_became_active":"2024-04-25T10:11:00.664205+0000","last_became_peered":"2024-04-25T10:11:00.664205+0000","last_unstale":"2024-04-25T10:11:01.666472+0000","last_undegraded":"2024-04-25T10:11:01.666472+0000","last_fullsized":"2024-04-25T10:11:01.666472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.860030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791966+0000","last_change":"2024-04-25T10:11:00.665672+0000","last_active":"2024-04-25T10:11:01.791966+0000","last_peered":"2024-04-25T10:11:01.791966+0000","last_clean":"2024-04-25T10:11:01.791966+0000","last_became_active":"2024-04-25T10:11:00.665198+0000","last_became_peered":"2024-04-25T10:11:00.665198+0000","last_unstale":"2024-04-25T10:11:01.791966+0000","last_undegraded":"2024-04-25T10:11:01.791966+0000","last_fullsized":"2024-04-25T10:11:01.791966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:58.968301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791419+0000","last_change":"2024-04-25T10:11:00.674341+0000","last_active":"2024-04-25T10:11:01.791419+0000","last_peered":"2024-04-25T10:11:01.791419+0000","last_clean":"2024-04-25T10:11:01.791419+0000","last_became_active":"2024-04-25T10:11:00.674016+0000","last_became_peered":"2024-04-25T10:11:00.674016+0000","last_unstale":"2024-04-25T10:11:01.791419+0000","last_undegraded":"2024-04-25T10:11:01.791419+0000","last_fullsized":"2024-04-25T10:11:01.791419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:17.318511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666458+0000","last_change":"2024-04-25T10:11:00.666821+0000","last_active":"2024-04-25T10:11:01.666458+0000","last_peered":"2024-04-25T10:11:01.666458+0000","last_clean":"2024-04-25T10:11:01.666458+0000","last_became_active":"2024-04-25T10:11:00.666552+0000","last_became_peered":"2024-04-25T10:11:00.666552+0000","last_unstale":"2024-04-25T10:11:01.666458+0000","last_undegraded":"2024-04-25T10:11:01.666458+0000","last_fullsized":"2024-04-25T10:11:01.666458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:43.541677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.671974+0000","last_change":"2024-04-25T10:11:00.666060+0000","last_active":"2024-04-25T10:11:01.671974+0000","last_peered":"2024-04-25T10:11:01.671974+0000","last_clean":"2024-04-25T10:11:01.671974+0000","last_became_active":"2024-04-25T10:11:00.665760+0000","last_became_peered":"2024-04-25T10:11:00.665760+0000","last_unstale":"2024-04-25T10:11:01.671974+0000","last_undegraded":"2024-04-25T10:11:01.671974+0000","last_fullsized":"2024-04-25T10:11:01.671974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:56.991014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790545+0000","last_change":"2024-04-25T10:11:00.672928+0000","last_active":"2024-04-25T10:11:01.790545+0000","last_peered":"2024-04-25T10:11:01.790545+0000","last_clean":"2024-04-25T10:11:01.790545+0000","last_became_active":"2024-04-25T10:11:00.672604+0000","last_became_peered":"2024-04-25T10:11:00.672604+0000","last_unstale":"2024-04-25T10:11:01.790545+0000","last_undegraded":"2024-04-25T10:11:01.790545+0000","last_fullsized":"2024-04-25T10:11:01.790545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:43.541677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669584+0000","last_change":"2024-04-25T10:11:00.665452+0000","last_active":"2024-04-25T10:11:01.669584+0000","last_peered":"2024-04-25T10:11:01.669584+0000","last_clean":"2024-04-25T10:11:01.669584+0000","last_became_active":"2024-04-25T10:11:00.665186+0000","last_became_peered":"2024-04-25T10:11:00.665186+0000","last_unstale":"2024-04-25T10:11:01.669584+0000","last_undegraded":"2024-04-25T10:11:01.669584+0000","last_fullsized":"2024-04-25T10:11:01.669584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:13.745574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666508+0000","last_change":"2024-04-25T10:11:00.657395+0000","last_active":"2024-04-25T10:11:01.666508+0000","last_peered":"2024-04-25T10:11:01.666508+0000","last_clean":"2024-04-25T10:11:01.666508+0000","last_became_active":"2024-04-25T10:11:00.657121+0000","last_became_peered":"2024-04-25T10:11:00.657121+0000","last_unstale":"2024-04-25T10:11:01.666508+0000","last_undegraded":"2024-04-25T10:11:01.666508+0000","last_fullsized":"2024-04-25T10:11:01.666508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:28.012019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669989+0000","last_change":"2024-04-25T10:11:00.674199+0000","last_active":"2024-04-25T10:11:01.669989+0000","last_peered":"2024-04-25T10:11:01.669989+0000","last_clean":"2024-04-25T10:11:01.669989+0000","last_became_active":"2024-04-25T10:11:00.673935+0000","last_became_peered":"2024-04-25T10:11:00.673935+0000","last_unstale":"2024-04-25T10:11:01.669989+0000","last_undegraded":"2024-04-25T10:11:01.669989+0000","last_fullsized":"2024-04-25T10:11:01.669989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:35.284964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790779+0000","last_change":"2024-04-25T10:11:00.675112+0000","last_active":"2024-04-25T10:11:01.790779+0000","last_peered":"2024-04-25T10:11:01.790779+0000","last_clean":"2024-04-25T10:11:01.790779+0000","last_became_active":"2024-04-25T10:11:00.674812+0000","last_became_peered":"2024-04-25T10:11:00.674812+0000","last_unstale":"2024-04-25T10:11:01.790779+0000","last_undegraded":"2024-04-25T10:11:01.790779+0000","last_fullsized":"2024-04-25T10:11:01.790779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:56.966365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791569+0000","last_change":"2024-04-25T10:11:00.660277+0000","last_active":"2024-04-25T10:11:01.791569+0000","last_peered":"2024-04-25T10:11:01.791569+0000","last_clean":"2024-04-25T10:11:01.791569+0000","last_became_active":"2024-04-25T10:11:00.659982+0000","last_became_peered":"2024-04-25T10:11:00.659982+0000","last_unstale":"2024-04-25T10:11:01.791569+0000","last_undegraded":"2024-04-25T10:11:01.791569+0000","last_fullsized":"2024-04-25T10:11:01.791569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:26.638020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791242+0000","last_change":"2024-04-25T10:11:00.672864+0000","last_active":"2024-04-25T10:11:01.791242+0000","last_peered":"2024-04-25T10:11:01.791242+0000","last_clean":"2024-04-25T10:11:01.791242+0000","last_became_active":"2024-04-25T10:11:00.672565+0000","last_became_peered":"2024-04-25T10:11:00.672565+0000","last_unstale":"2024-04-25T10:11:01.791242+0000","last_undegraded":"2024-04-25T10:11:01.791242+0000","last_fullsized":"2024-04-25T10:11:01.791242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:38.932561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.671980+0000","last_change":"2024-04-25T10:11:00.675075+0000","last_active":"2024-04-25T10:11:01.671980+0000","last_peered":"2024-04-25T10:11:01.671980+0000","last_clean":"2024-04-25T10:11:01.671980+0000","last_became_active":"2024-04-25T10:11:00.674791+0000","last_became_peered":"2024-04-25T10:11:00.674791+0000","last_unstale":"2024-04-25T10:11:01.671980+0000","last_undegraded":"2024-04-25T10:11:01.671980+0000","last_fullsized":"2024-04-25T10:11:01.671980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:14.897123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666210+0000","last_change":"2024-04-25T10:11:00.658578+0000","last_active":"2024-04-25T10:11:01.666210+0000","last_peered":"2024-04-25T10:11:01.666210+0000","last_clean":"2024-04-25T10:11:01.666210+0000","last_became_active":"2024-04-25T10:11:00.658309+0000","last_became_peered":"2024-04-25T10:11:00.658309+0000","last_unstale":"2024-04-25T10:11:01.666210+0000","last_undegraded":"2024-04-25T10:11:01.666210+0000","last_fullsized":"2024-04-25T10:11:01.666210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:40.828899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790928+0000","last_change":"2024-04-25T10:11:00.674321+0000","last_active":"2024-04-25T10:11:01.790928+0000","last_peered":"2024-04-25T10:11:01.790928+0000","last_clean":"2024-04-25T10:11:01.790928+0000","last_became_active":"2024-04-25T10:11:00.674037+0000","last_became_peered":"2024-04-25T10:11:00.674037+0000","last_unstale":"2024-04-25T10:11:01.790928+0000","last_undegraded":"2024-04-25T10:11:01.790928+0000","last_fullsized":"2024-04-25T10:11:01.790928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:37.420962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669990+0000","last_change":"2024-04-25T10:11:00.673650+0000","last_active":"2024-04-25T10:11:01.669990+0000","last_peered":"2024-04-25T10:11:01.669990+0000","last_clean":"2024-04-25T10:11:01.669990+0000","last_became_active":"2024-04-25T10:11:00.673318+0000","last_became_peered":"2024-04-25T10:11:00.673318+0000","last_unstale":"2024-04-25T10:11:01.669990+0000","last_undegraded":"2024-04-25T10:11:01.669990+0000","last_fullsized":"2024-04-25T10:11:01.669990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:18.528132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791780+0000","last_change":"2024-04-25T10:11:00.672749+0000","last_active":"2024-04-25T10:11:01.791780+0000","last_peered":"2024-04-25T10:11:01.791780+0000","last_clean":"2024-04-25T10:11:01.791780+0000","last_became_active":"2024-04-25T10:11:00.671982+0000","last_became_peered":"2024-04-25T10:11:00.671982+0000","last_unstale":"2024-04-25T10:11:01.791780+0000","last_undegraded":"2024-04-25T10:11:01.791780+0000","last_fullsized":"2024-04-25T10:11:01.791780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.235679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791396+0000","last_change":"2024-04-25T10:11:00.674897+0000","last_active":"2024-04-25T10:11:01.791396+0000","last_peered":"2024-04-25T10:11:01.791396+0000","last_clean":"2024-04-25T10:11:01.791396+0000","last_became_active":"2024-04-25T10:11:00.674582+0000","last_became_peered":"2024-04-25T10:11:00.674582+0000","last_unstale":"2024-04-25T10:11:01.791396+0000","last_undegraded":"2024-04-25T10:11:01.791396+0000","last_fullsized":"2024-04-25T10:11:01.791396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.603074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791262+0000","last_change":"2024-04-25T10:11:00.675839+0000","last_active":"2024-04-25T10:11:01.791262+0000","last_peered":"2024-04-25T10:11:01.791262+0000","last_clean":"2024-04-25T10:11:01.791262+0000","last_became_active":"2024-04-25T10:11:00.674812+0000","last_became_peered":"2024-04-25T10:11:00.674812+0000","last_unstale":"2024-04-25T10:11:01.791262+0000","last_undegraded":"2024-04-25T10:11:01.791262+0000","last_fullsized":"2024-04-25T10:11:01.791262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:45.304788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791410+0000","last_change":"2024-04-25T10:11:00.670165+0000","last_active":"2024-04-25T10:11:01.791410+0000","last_peered":"2024-04-25T10:11:01.791410+0000","last_clean":"2024-04-25T10:11:01.791410+0000","last_became_active":"2024-04-25T10:11:00.669866+0000","last_became_peered":"2024-04-25T10:11:00.669866+0000","last_unstale":"2024-04-25T10:11:01.791410+0000","last_undegraded":"2024-04-25T10:11:01.791410+0000","last_fullsized":"2024-04-25T10:11:01.791410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:43.529739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790610+0000","last_change":"2024-04-25T10:11:00.674422+0000","last_active":"2024-04-25T10:11:01.790610+0000","last_peered":"2024-04-25T10:11:01.790610+0000","last_clean":"2024-04-25T10:11:01.790610+0000","last_became_active":"2024-04-25T10:11:00.674142+0000","last_became_peered":"2024-04-25T10:11:00.674142+0000","last_unstale":"2024-04-25T10:11:01.790610+0000","last_undegraded":"2024-04-25T10:11:01.790610+0000","last_fullsized":"2024-04-25T10:11:01.790610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:28.012019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669720+0000","last_change":"2024-04-25T10:11:00.672113+0000","last_active":"2024-04-25T10:11:01.669720+0000","last_peered":"2024-04-25T10:11:01.669720+0000","last_clean":"2024-04-25T10:11:01.669720+0000","last_became_active":"2024-04-25T10:11:00.671835+0000","last_became_peered":"2024-04-25T10:11:00.671835+0000","last_unstale":"2024-04-25T10:11:01.669720+0000","last_undegraded":"2024-04-25T10:11:01.669720+0000","last_fullsized":"2024-04-25T10:11:01.669720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:56.991014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.670008+0000","last_change":"2024-04-25T10:11:00.679427+0000","last_active":"2024-04-25T10:11:01.670008+0000","last_peered":"2024-04-25T10:11:01.670008+0000","last_clean":"2024-04-25T10:11:01.670008+0000","last_became_active":"2024-04-25T10:11:00.678268+0000","last_became_peered":"2024-04-25T10:11:00.678268+0000","last_unstale":"2024-04-25T10:11:01.670008+0000","last_undegraded":"2024-04-25T10:11:01.670008+0000","last_fullsized":"2024-04-25T10:11:01.670008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:57.428737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791705+0000","last_change":"2024-04-25T10:11:00.671179+0000","last_active":"2024-04-25T10:11:01.791705+0000","last_peered":"2024-04-25T10:11:01.791705+0000","last_clean":"2024-04-25T10:11:01.791705+0000","last_became_active":"2024-04-25T10:11:00.670882+0000","last_became_peered":"2024-04-25T10:11:00.670882+0000","last_unstale":"2024-04-25T10:11:01.791705+0000","last_undegraded":"2024-04-25T10:11:01.791705+0000","last_fullsized":"2024-04-25T10:11:01.791705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:22.210854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791595+0000","last_change":"2024-04-25T10:11:00.672506+0000","last_active":"2024-04-25T10:11:01.791595+0000","last_peered":"2024-04-25T10:11:01.791595+0000","last_clean":"2024-04-25T10:11:01.791595+0000","last_became_active":"2024-04-25T10:11:00.672018+0000","last_became_peered":"2024-04-25T10:11:00.672018+0000","last_unstale":"2024-04-25T10:11:01.791595+0000","last_undegraded":"2024-04-25T10:11:01.791595+0000","last_fullsized":"2024-04-25T10:11:01.791595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:14.645638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669413+0000","last_change":"2024-04-25T10:11:00.668016+0000","last_active":"2024-04-25T10:11:01.669413+0000","last_peered":"2024-04-25T10:11:01.669413+0000","last_clean":"2024-04-25T10:11:01.669413+0000","last_became_active":"2024-04-25T10:11:00.667742+0000","last_became_peered":"2024-04-25T10:11:00.667742+0000","last_unstale":"2024-04-25T10:11:01.669413+0000","last_undegraded":"2024-04-25T10:11:01.669413+0000","last_fullsized":"2024-04-25T10:11:01.669413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:35.210017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791447+0000","last_change":"2024-04-25T10:11:00.665612+0000","last_active":"2024-04-25T10:11:01.791447+0000","last_peered":"2024-04-25T10:11:01.791447+0000","last_clean":"2024-04-25T10:11:01.791447+0000","last_became_active":"2024-04-25T10:11:00.665327+0000","last_became_peered":"2024-04-25T10:11:00.665327+0000","last_unstale":"2024-04-25T10:11:01.791447+0000","last_undegraded":"2024-04-25T10:11:01.791447+0000","last_fullsized":"2024-04-25T10:11:01.791447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:36.768794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791586+0000","last_change":"2024-04-25T10:11:00.672908+0000","last_active":"2024-04-25T10:11:01.791586+0000","last_peered":"2024-04-25T10:11:01.791586+0000","last_clean":"2024-04-25T10:11:01.791586+0000","last_became_active":"2024-04-25T10:11:00.672634+0000","last_became_peered":"2024-04-25T10:11:00.672634+0000","last_unstale":"2024-04-25T10:11:01.791586+0000","last_undegraded":"2024-04-25T10:11:01.791586+0000","last_fullsized":"2024-04-25T10:11:01.791586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:37.071670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666210+0000","last_change":"2024-04-25T10:11:00.667457+0000","last_active":"2024-04-25T10:11:01.666210+0000","last_peered":"2024-04-25T10:11:01.666210+0000","last_clean":"2024-04-25T10:11:01.666210+0000","last_became_active":"2024-04-25T10:11:00.667205+0000","last_became_peered":"2024-04-25T10:11:00.667205+0000","last_unstale":"2024-04-25T10:11:01.666210+0000","last_undegraded":"2024-04-25T10:11:01.666210+0000","last_fullsized":"2024-04-25T10:11:01.666210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:07.289251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666783+0000","last_change":"2024-04-25T10:11:00.668155+0000","last_active":"2024-04-25T10:11:01.666783+0000","last_peered":"2024-04-25T10:11:01.666783+0000","last_clean":"2024-04-25T10:11:01.666783+0000","last_became_active":"2024-04-25T10:11:00.667848+0000","last_became_peered":"2024-04-25T10:11:00.667848+0000","last_unstale":"2024-04-25T10:11:01.666783+0000","last_undegraded":"2024-04-25T10:11:01.666783+0000","last_fullsized":"2024-04-25T10:11:01.666783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:55.212072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790745+0000","last_change":"2024-04-25T10:11:00.664212+0000","last_active":"2024-04-25T10:11:01.790745+0000","last_peered":"2024-04-25T10:11:01.790745+0000","last_clean":"2024-04-25T10:11:01.790745+0000","last_became_active":"2024-04-25T10:11:00.663662+0000","last_became_peered":"2024-04-25T10:11:00.663662+0000","last_unstale":"2024-04-25T10:11:01.790745+0000","last_undegraded":"2024-04-25T10:11:01.790745+0000","last_fullsized":"2024-04-25T10:11:01.790745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:14.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"85'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791831+0000","last_change":"2024-04-25T10:10:59.661968+0000","last_active":"2024-04-25T10:11:01.791831+0000","last_peered":"2024-04-25T10:11:01.791831+0000","last_clean":"2024-04-25T10:11:01.791831+0000","last_became_active":"2024-04-25T10:10:59.661557+0000","last_became_peered":"2024-04-25T10:10:59.661557+0000","last_unstale":"2024-04-25T10:11:01.791831+0000","last_undegraded":"2024-04-25T10:11:01.791831+0000","last_fullsized":"2024-04-25T10:11:01.791831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:29.166481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669745+0000","last_change":"2024-04-25T10:11:00.664765+0000","last_active":"2024-04-25T10:11:01.669745+0000","last_peered":"2024-04-25T10:11:01.669745+0000","last_clean":"2024-04-25T10:11:01.669745+0000","last_became_active":"2024-04-25T10:11:00.664306+0000","last_became_peered":"2024-04-25T10:11:00.664306+0000","last_unstale":"2024-04-25T10:11:01.669745+0000","last_undegraded":"2024-04-25T10:11:01.669745+0000","last_fullsized":"2024-04-25T10:11:01.669745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:51.779177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666622+0000","last_change":"2024-04-25T10:10:59.659985+0000","last_active":"2024-04-25T10:11:01.666622+0000","last_peered":"2024-04-25T10:11:01.666622+0000","last_clean":"2024-04-25T10:11:01.666622+0000","last_became_active":"2024-04-25T10:10:59.659641+0000","last_became_peered":"2024-04-25T10:10:59.659641+0000","last_unstale":"2024-04-25T10:11:01.666622+0000","last_undegraded":"2024-04-25T10:11:01.666622+0000","last_fullsized":"2024-04-25T10:11:01.666622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:35.098559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790943+0000","last_change":"2024-04-25T10:11:00.666415+0000","last_active":"2024-04-25T10:11:01.790943+0000","last_peered":"2024-04-25T10:11:01.790943+0000","last_clean":"2024-04-25T10:11:01.790943+0000","last_became_active":"2024-04-25T10:11:00.666151+0000","last_became_peered":"2024-04-25T10:11:00.666151+0000","last_unstale":"2024-04-25T10:11:01.790943+0000","last_undegraded":"2024-04-25T10:11:01.790943+0000","last_fullsized":"2024-04-25T10:11:01.790943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:43.188026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669929+0000","last_change":"2024-04-25T10:10:59.654631+0000","last_active":"2024-04-25T10:11:01.669929+0000","last_peered":"2024-04-25T10:11:01.669929+0000","last_clean":"2024-04-25T10:11:01.669929+0000","last_became_active":"2024-04-25T10:10:59.654288+0000","last_became_peered":"2024-04-25T10:10:59.654288+0000","last_unstale":"2024-04-25T10:11:01.669929+0000","last_undegraded":"2024-04-25T10:11:01.669929+0000","last_fullsized":"2024-04-25T10:11:01.669929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:34.205125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669497+0000","last_change":"2024-04-25T10:11:00.677043+0000","last_active":"2024-04-25T10:11:01.669497+0000","last_peered":"2024-04-25T10:11:01.669497+0000","last_clean":"2024-04-25T10:11:01.669497+0000","last_became_active":"2024-04-25T10:11:00.675514+0000","last_became_peered":"2024-04-25T10:11:00.675514+0000","last_unstale":"2024-04-25T10:11:01.669497+0000","last_undegraded":"2024-04-25T10:11:01.669497+0000","last_fullsized":"2024-04-25T10:11:01.669497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:46.822338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791948+0000","last_change":"2024-04-25T10:10:59.658687+0000","last_active":"2024-04-25T10:11:01.791948+0000","last_peered":"2024-04-25T10:11:01.791948+0000","last_clean":"2024-04-25T10:11:01.791948+0000","last_became_active":"2024-04-25T10:10:59.658380+0000","last_became_peered":"2024-04-25T10:10:59.658380+0000","last_unstale":"2024-04-25T10:11:01.791948+0000","last_undegraded":"2024-04-25T10:11:01.791948+0000","last_fullsized":"2024-04-25T10:11:01.791948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:26.830980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791356+0000","last_change":"2024-04-25T10:11:00.671818+0000","last_active":"2024-04-25T10:11:01.791356+0000","last_peered":"2024-04-25T10:11:01.791356+0000","last_clean":"2024-04-25T10:11:01.791356+0000","last_became_active":"2024-04-25T10:11:00.671536+0000","last_became_peered":"2024-04-25T10:11:00.671536+0000","last_unstale":"2024-04-25T10:11:01.791356+0000","last_undegraded":"2024-04-25T10:11:01.791356+0000","last_fullsized":"2024-04-25T10:11:01.791356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:20.822168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790565+0000","last_change":"2024-04-25T10:10:59.658839+0000","last_active":"2024-04-25T10:11:01.790565+0000","last_peered":"2024-04-25T10:11:01.790565+0000","last_clean":"2024-04-25T10:11:01.790565+0000","last_became_active":"2024-04-25T10:10:59.658557+0000","last_became_peered":"2024-04-25T10:10:59.658557+0000","last_unstale":"2024-04-25T10:11:01.790565+0000","last_undegraded":"2024-04-25T10:11:01.790565+0000","last_fullsized":"2024-04-25T10:11:01.790565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:35.098559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666691+0000","last_change":"2024-04-25T10:11:00.670164+0000","last_active":"2024-04-25T10:11:01.666691+0000","last_peered":"2024-04-25T10:11:01.666691+0000","last_clean":"2024-04-25T10:11:01.666691+0000","last_became_active":"2024-04-25T10:11:00.669856+0000","last_became_peered":"2024-04-25T10:11:00.669856+0000","last_unstale":"2024-04-25T10:11:01.666691+0000","last_undegraded":"2024-04-25T10:11:01.666691+0000","last_fullsized":"2024-04-25T10:11:01.666691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:56.966365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666588+0000","last_change":"2024-04-25T10:10:59.660519+0000","last_active":"2024-04-25T10:11:01.666588+0000","last_peered":"2024-04-25T10:11:01.666588+0000","last_clean":"2024-04-25T10:11:01.666588+0000","last_became_active":"2024-04-25T10:10:59.659571+0000","last_became_peered":"2024-04-25T10:10:59.659571+0000","last_unstale":"2024-04-25T10:11:01.666588+0000","last_undegraded":"2024-04-25T10:11:01.666588+0000","last_fullsized":"2024-04-25T10:11:01.666588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:47.638024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669228+0000","last_change":"2024-04-25T10:11:00.672808+0000","last_active":"2024-04-25T10:11:01.669228+0000","last_peered":"2024-04-25T10:11:01.669228+0000","last_clean":"2024-04-25T10:11:01.669228+0000","last_became_active":"2024-04-25T10:11:00.672479+0000","last_became_peered":"2024-04-25T10:11:00.672479+0000","last_unstale":"2024-04-25T10:11:01.669228+0000","last_undegraded":"2024-04-25T10:11:01.669228+0000","last_fullsized":"2024-04-25T10:11:01.669228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:43.529739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791149+0000","last_change":"2024-04-25T10:10:59.662636+0000","last_active":"2024-04-25T10:11:01.791149+0000","last_peered":"2024-04-25T10:11:01.791149+0000","last_clean":"2024-04-25T10:11:01.791149+0000","last_became_active":"2024-04-25T10:10:59.661790+0000","last_became_peered":"2024-04-25T10:10:59.661790+0000","last_unstale":"2024-04-25T10:11:01.791149+0000","last_undegraded":"2024-04-25T10:11:01.791149+0000","last_fullsized":"2024-04-25T10:11:01.791149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:10.609819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791191+0000","last_change":"2024-04-25T10:11:00.671182+0000","last_active":"2024-04-25T10:11:01.791191+0000","last_peered":"2024-04-25T10:11:01.791191+0000","last_clean":"2024-04-25T10:11:01.791191+0000","last_became_active":"2024-04-25T10:11:00.670863+0000","last_became_peered":"2024-04-25T10:11:00.670863+0000","last_unstale":"2024-04-25T10:11:01.791191+0000","last_undegraded":"2024-04-25T10:11:01.791191+0000","last_fullsized":"2024-04-25T10:11:01.791191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:21.622898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791076+0000","last_change":"2024-04-25T10:10:59.661931+0000","last_active":"2024-04-25T10:11:01.791076+0000","last_peered":"2024-04-25T10:11:01.791076+0000","last_clean":"2024-04-25T10:11:01.791076+0000","last_became_active":"2024-04-25T10:10:59.661173+0000","last_became_peered":"2024-04-25T10:10:59.661173+0000","last_unstale":"2024-04-25T10:11:01.791076+0000","last_undegraded":"2024-04-25T10:11:01.791076+0000","last_fullsized":"2024-04-25T10:11:01.791076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:35.826027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791331+0000","last_change":"2024-04-25T10:11:00.676424+0000","last_active":"2024-04-25T10:11:01.791331+0000","last_peered":"2024-04-25T10:11:01.791331+0000","last_clean":"2024-04-25T10:11:01.791331+0000","last_became_active":"2024-04-25T10:11:00.676088+0000","last_became_peered":"2024-04-25T10:11:00.676088+0000","last_unstale":"2024-04-25T10:11:01.791331+0000","last_undegraded":"2024-04-25T10:11:01.791331+0000","last_fullsized":"2024-04-25T10:11:01.791331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:50.887593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669814+0000","last_change":"2024-04-25T10:10:59.660535+0000","last_active":"2024-04-25T10:11:01.669814+0000","last_peered":"2024-04-25T10:11:01.669814+0000","last_clean":"2024-04-25T10:11:01.669814+0000","last_became_active":"2024-04-25T10:10:59.660260+0000","last_became_peered":"2024-04-25T10:10:59.660260+0000","last_unstale":"2024-04-25T10:11:01.669814+0000","last_undegraded":"2024-04-25T10:11:01.669814+0000","last_fullsized":"2024-04-25T10:11:01.669814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:23.530134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669744+0000","last_change":"2024-04-25T10:11:00.684312+0000","last_active":"2024-04-25T10:11:01.669744+0000","last_peered":"2024-04-25T10:11:01.669744+0000","last_clean":"2024-04-25T10:11:01.669744+0000","last_became_active":"2024-04-25T10:11:00.683971+0000","last_became_peered":"2024-04-25T10:11:00.683971+0000","last_unstale":"2024-04-25T10:11:01.669744+0000","last_undegraded":"2024-04-25T10:11:01.669744+0000","last_fullsized":"2024-04-25T10:11:01.669744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:40.703987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666638+0000","last_change":"2024-04-25T10:10:59.658424+0000","last_active":"2024-04-25T10:11:01.666638+0000","last_peered":"2024-04-25T10:11:01.666638+0000","last_clean":"2024-04-25T10:11:01.666638+0000","last_became_active":"2024-04-25T10:10:59.658128+0000","last_became_peered":"2024-04-25T10:10:59.658128+0000","last_unstale":"2024-04-25T10:11:01.666638+0000","last_undegraded":"2024-04-25T10:11:01.666638+0000","last_fullsized":"2024-04-25T10:11:01.666638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:42.113345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669659+0000","last_change":"2024-04-25T10:11:00.678536+0000","last_active":"2024-04-25T10:11:01.669659+0000","last_peered":"2024-04-25T10:11:01.669659+0000","last_clean":"2024-04-25T10:11:01.669659+0000","last_became_active":"2024-04-25T10:11:00.678276+0000","last_became_peered":"2024-04-25T10:11:00.678276+0000","last_unstale":"2024-04-25T10:11:01.669659+0000","last_undegraded":"2024-04-25T10:11:01.669659+0000","last_fullsized":"2024-04-25T10:11:01.669659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:48.642917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"85'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669547+0000","last_change":"2024-04-25T10:10:59.653278+0000","last_active":"2024-04-25T10:11:01.669547+0000","last_peered":"2024-04-25T10:11:01.669547+0000","last_clean":"2024-04-25T10:11:01.669547+0000","last_became_active":"2024-04-25T10:10:59.652974+0000","last_became_peered":"2024-04-25T10:10:59.652974+0000","last_unstale":"2024-04-25T10:11:01.669547+0000","last_undegraded":"2024-04-25T10:11:01.669547+0000","last_fullsized":"2024-04-25T10:11:01.669547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:45.820044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791449+0000","last_change":"2024-04-25T10:11:00.684587+0000","last_active":"2024-04-25T10:11:01.791449+0000","last_peered":"2024-04-25T10:11:01.791449+0000","last_clean":"2024-04-25T10:11:01.791449+0000","last_became_active":"2024-04-25T10:11:00.684306+0000","last_became_peered":"2024-04-25T10:11:00.684306+0000","last_unstale":"2024-04-25T10:11:01.791449+0000","last_undegraded":"2024-04-25T10:11:01.791449+0000","last_fullsized":"2024-04-25T10:11:01.791449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:39.955023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"85'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790926+0000","last_change":"2024-04-25T10:10:59.662733+0000","last_active":"2024-04-25T10:11:01.790926+0000","last_peered":"2024-04-25T10:11:01.790926+0000","last_clean":"2024-04-25T10:11:01.790926+0000","last_became_active":"2024-04-25T10:10:59.662290+0000","last_became_peered":"2024-04-25T10:10:59.662290+0000","last_unstale":"2024-04-25T10:11:01.790926+0000","last_undegraded":"2024-04-25T10:11:01.790926+0000","last_fullsized":"2024-04-25T10:11:01.790926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:34.280071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669718+0000","last_change":"2024-04-25T10:11:00.669181+0000","last_active":"2024-04-25T10:11:01.669718+0000","last_peered":"2024-04-25T10:11:01.669718+0000","last_clean":"2024-04-25T10:11:01.669718+0000","last_became_active":"2024-04-25T10:11:00.668758+0000","last_became_peered":"2024-04-25T10:11:00.668758+0000","last_unstale":"2024-04-25T10:11:01.669718+0000","last_undegraded":"2024-04-25T10:11:01.669718+0000","last_fullsized":"2024-04-25T10:11:01.669718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:38.681757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791557+0000","last_change":"2024-04-25T10:10:59.662230+0000","last_active":"2024-04-25T10:11:01.791557+0000","last_peered":"2024-04-25T10:11:01.791557+0000","last_clean":"2024-04-25T10:11:01.791557+0000","last_became_active":"2024-04-25T10:10:59.661953+0000","last_became_peered":"2024-04-25T10:10:59.661953+0000","last_unstale":"2024-04-25T10:11:01.791557+0000","last_undegraded":"2024-04-25T10:11:01.791557+0000","last_fullsized":"2024-04-25T10:11:01.791557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:43.354438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669387+0000","last_change":"2024-04-25T10:11:00.680010+0000","last_active":"2024-04-25T10:11:01.669387+0000","last_peered":"2024-04-25T10:11:01.669387+0000","last_clean":"2024-04-25T10:11:01.669387+0000","last_became_active":"2024-04-25T10:11:00.679712+0000","last_became_peered":"2024-04-25T10:11:00.679712+0000","last_unstale":"2024-04-25T10:11:01.669387+0000","last_undegraded":"2024-04-25T10:11:01.669387+0000","last_fullsized":"2024-04-25T10:11:01.669387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:22.568376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672076+0000","last_change":"2024-04-25T10:10:59.659677+0000","last_active":"2024-04-25T10:11:01.672076+0000","last_peered":"2024-04-25T10:11:01.672076+0000","last_clean":"2024-04-25T10:11:01.672076+0000","last_became_active":"2024-04-25T10:10:59.659395+0000","last_became_peered":"2024-04-25T10:10:59.659395+0000","last_unstale":"2024-04-25T10:11:01.672076+0000","last_undegraded":"2024-04-25T10:11:01.672076+0000","last_fullsized":"2024-04-25T10:11:01.672076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:21.670434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790957+0000","last_change":"2024-04-25T10:11:00.678832+0000","last_active":"2024-04-25T10:11:01.790957+0000","last_peered":"2024-04-25T10:11:01.790957+0000","last_clean":"2024-04-25T10:11:01.790957+0000","last_became_active":"2024-04-25T10:11:00.678560+0000","last_became_peered":"2024-04-25T10:11:00.678560+0000","last_unstale":"2024-04-25T10:11:01.790957+0000","last_undegraded":"2024-04-25T10:11:01.790957+0000","last_fullsized":"2024-04-25T10:11:01.790957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:19.719591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666238+0000","last_change":"2024-04-25T10:10:59.656540+0000","last_active":"2024-04-25T10:11:01.666238+0000","last_peered":"2024-04-25T10:11:01.666238+0000","last_clean":"2024-04-25T10:11:01.666238+0000","last_became_active":"2024-04-25T10:10:59.656276+0000","last_became_peered":"2024-04-25T10:10:59.656276+0000","last_unstale":"2024-04-25T10:11:01.666238+0000","last_undegraded":"2024-04-25T10:11:01.666238+0000","last_fullsized":"2024-04-25T10:11:01.666238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:13.892230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669548+0000","last_change":"2024-04-25T10:11:00.680212+0000","last_active":"2024-04-25T10:11:01.669548+0000","last_peered":"2024-04-25T10:11:01.669548+0000","last_clean":"2024-04-25T10:11:01.669548+0000","last_became_active":"2024-04-25T10:11:00.679944+0000","last_became_peered":"2024-04-25T10:11:00.679944+0000","last_unstale":"2024-04-25T10:11:01.669548+0000","last_undegraded":"2024-04-25T10:11:01.669548+0000","last_fullsized":"2024-04-25T10:11:01.669548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:14.897123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"85'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669582+0000","last_change":"2024-04-25T10:10:59.656545+0000","last_active":"2024-04-25T10:11:01.669582+0000","last_peered":"2024-04-25T10:11:01.669582+0000","last_clean":"2024-04-25T10:11:01.669582+0000","last_became_active":"2024-04-25T10:10:59.656262+0000","last_became_peered":"2024-04-25T10:10:59.656262+0000","last_unstale":"2024-04-25T10:11:01.669582+0000","last_undegraded":"2024-04-25T10:11:01.669582+0000","last_fullsized":"2024-04-25T10:11:01.669582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:12.740681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790809+0000","last_change":"2024-04-25T10:11:00.660578+0000","last_active":"2024-04-25T10:11:01.790809+0000","last_peered":"2024-04-25T10:11:01.790809+0000","last_clean":"2024-04-25T10:11:01.790809+0000","last_became_active":"2024-04-25T10:11:00.660258+0000","last_became_peered":"2024-04-25T10:11:00.660258+0000","last_unstale":"2024-04-25T10:11:01.790809+0000","last_undegraded":"2024-04-25T10:11:01.790809+0000","last_fullsized":"2024-04-25T10:11:01.790809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:20.151958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791898+0000","last_change":"2024-04-25T10:10:59.660067+0000","last_active":"2024-04-25T10:11:01.791898+0000","last_peered":"2024-04-25T10:11:01.791898+0000","last_clean":"2024-04-25T10:11:01.791898+0000","last_became_active":"2024-04-25T10:10:59.659717+0000","last_became_peered":"2024-04-25T10:10:59.659717+0000","last_unstale":"2024-04-25T10:11:01.791898+0000","last_undegraded":"2024-04-25T10:11:01.791898+0000","last_fullsized":"2024-04-25T10:11:01.791898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:55.225255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666704+0000","last_change":"2024-04-25T10:11:00.664267+0000","last_active":"2024-04-25T10:11:01.666704+0000","last_peered":"2024-04-25T10:11:01.666704+0000","last_clean":"2024-04-25T10:11:01.666704+0000","last_became_active":"2024-04-25T10:11:00.664038+0000","last_became_peered":"2024-04-25T10:11:00.664038+0000","last_unstale":"2024-04-25T10:11:01.666704+0000","last_undegraded":"2024-04-25T10:11:01.666704+0000","last_fullsized":"2024-04-25T10:11:01.666704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:20.151958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791555+0000","last_change":"2024-04-25T10:10:59.661418+0000","last_active":"2024-04-25T10:11:01.791555+0000","last_peered":"2024-04-25T10:11:01.791555+0000","last_clean":"2024-04-25T10:11:01.791555+0000","last_became_active":"2024-04-25T10:10:59.661121+0000","last_became_peered":"2024-04-25T10:10:59.661121+0000","last_unstale":"2024-04-25T10:11:01.791555+0000","last_undegraded":"2024-04-25T10:11:01.791555+0000","last_fullsized":"2024-04-25T10:11:01.791555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:53.008632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790835+0000","last_change":"2024-04-25T10:11:00.662883+0000","last_active":"2024-04-25T10:11:01.790835+0000","last_peered":"2024-04-25T10:11:01.790835+0000","last_clean":"2024-04-25T10:11:01.790835+0000","last_became_active":"2024-04-25T10:11:00.662578+0000","last_became_peered":"2024-04-25T10:11:00.662578+0000","last_unstale":"2024-04-25T10:11:01.790835+0000","last_undegraded":"2024-04-25T10:11:01.790835+0000","last_fullsized":"2024-04-25T10:11:01.790835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:55.212072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791634+0000","last_change":"2024-04-25T10:10:59.663404+0000","last_active":"2024-04-25T10:11:01.791634+0000","last_peered":"2024-04-25T10:11:01.791634+0000","last_clean":"2024-04-25T10:11:01.791634+0000","last_became_active":"2024-04-25T10:10:59.663138+0000","last_became_peered":"2024-04-25T10:10:59.663138+0000","last_unstale":"2024-04-25T10:11:01.791634+0000","last_undegraded":"2024-04-25T10:11:01.791634+0000","last_fullsized":"2024-04-25T10:11:01.791634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:38.050152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672109+0000","last_change":"2024-04-25T10:11:00.672494+0000","last_active":"2024-04-25T10:11:01.672109+0000","last_peered":"2024-04-25T10:11:01.672109+0000","last_clean":"2024-04-25T10:11:01.672109+0000","last_became_active":"2024-04-25T10:11:00.672238+0000","last_became_peered":"2024-04-25T10:11:00.672238+0000","last_unstale":"2024-04-25T10:11:01.672109+0000","last_undegraded":"2024-04-25T10:11:01.672109+0000","last_fullsized":"2024-04-25T10:11:01.672109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:48.642917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791130+0000","last_change":"2024-04-25T10:10:59.662706+0000","last_active":"2024-04-25T10:11:01.791130+0000","last_peered":"2024-04-25T10:11:01.791130+0000","last_clean":"2024-04-25T10:11:01.791130+0000","last_became_active":"2024-04-25T10:10:59.662399+0000","last_became_peered":"2024-04-25T10:10:59.662399+0000","last_unstale":"2024-04-25T10:11:01.791130+0000","last_undegraded":"2024-04-25T10:11:01.791130+0000","last_fullsized":"2024-04-25T10:11:01.791130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:56.481855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791760+0000","last_change":"2024-04-25T10:11:00.684360+0000","last_active":"2024-04-25T10:11:01.791760+0000","last_peered":"2024-04-25T10:11:01.791760+0000","last_clean":"2024-04-25T10:11:01.791760+0000","last_became_active":"2024-04-25T10:11:00.683915+0000","last_became_peered":"2024-04-25T10:11:00.683915+0000","last_unstale":"2024-04-25T10:11:01.791760+0000","last_undegraded":"2024-04-25T10:11:01.791760+0000","last_fullsized":"2024-04-25T10:11:01.791760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:44.016510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672207+0000","last_change":"2024-04-25T10:10:59.664474+0000","last_active":"2024-04-25T10:11:01.672207+0000","last_peered":"2024-04-25T10:11:01.672207+0000","last_clean":"2024-04-25T10:11:01.672207+0000","last_became_active":"2024-04-25T10:10:59.664182+0000","last_became_peered":"2024-04-25T10:10:59.664182+0000","last_unstale":"2024-04-25T10:11:01.672207+0000","last_undegraded":"2024-04-25T10:11:01.672207+0000","last_fullsized":"2024-04-25T10:11:01.672207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:51.674567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790910+0000","last_change":"2024-04-25T10:11:00.671858+0000","last_active":"2024-04-25T10:11:01.790910+0000","last_peered":"2024-04-25T10:11:01.790910+0000","last_clean":"2024-04-25T10:11:01.790910+0000","last_became_active":"2024-04-25T10:11:00.671566+0000","last_became_peered":"2024-04-25T10:11:00.671566+0000","last_unstale":"2024-04-25T10:11:01.790910+0000","last_undegraded":"2024-04-25T10:11:01.790910+0000","last_fullsized":"2024-04-25T10:11:01.790910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:36.037775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669493+0000","last_change":"2024-04-25T10:10:59.656426+0000","last_active":"2024-04-25T10:11:01.669493+0000","last_peered":"2024-04-25T10:11:01.669493+0000","last_clean":"2024-04-25T10:11:01.669493+0000","last_became_active":"2024-04-25T10:10:59.656089+0000","last_became_peered":"2024-04-25T10:10:59.656089+0000","last_unstale":"2024-04-25T10:11:01.669493+0000","last_undegraded":"2024-04-25T10:11:01.669493+0000","last_fullsized":"2024-04-25T10:11:01.669493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:50.774285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790576+0000","last_change":"2024-04-25T10:11:00.675256+0000","last_active":"2024-04-25T10:11:01.790576+0000","last_peered":"2024-04-25T10:11:01.790576+0000","last_clean":"2024-04-25T10:11:01.790576+0000","last_became_active":"2024-04-25T10:11:00.674969+0000","last_became_peered":"2024-04-25T10:11:00.674969+0000","last_unstale":"2024-04-25T10:11:01.790576+0000","last_undegraded":"2024-04-25T10:11:01.790576+0000","last_fullsized":"2024-04-25T10:11:01.790576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.860030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791546+0000","last_change":"2024-04-25T10:10:59.662798+0000","last_active":"2024-04-25T10:11:01.791546+0000","last_peered":"2024-04-25T10:11:01.791546+0000","last_clean":"2024-04-25T10:11:01.791546+0000","last_became_active":"2024-04-25T10:10:59.661497+0000","last_became_peered":"2024-04-25T10:10:59.661497+0000","last_unstale":"2024-04-25T10:11:01.791546+0000","last_undegraded":"2024-04-25T10:11:01.791546+0000","last_fullsized":"2024-04-25T10:11:01.791546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:29.530495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791719+0000","last_change":"2024-04-25T10:11:00.666183+0000","last_active":"2024-04-25T10:11:01.791719+0000","last_peered":"2024-04-25T10:11:01.791719+0000","last_clean":"2024-04-25T10:11:01.791719+0000","last_became_active":"2024-04-25T10:11:00.665905+0000","last_became_peered":"2024-04-25T10:11:00.665905+0000","last_unstale":"2024-04-25T10:11:01.791719+0000","last_undegraded":"2024-04-25T10:11:01.791719+0000","last_fullsized":"2024-04-25T10:11:01.791719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.427341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791772+0000","last_change":"2024-04-25T10:10:59.661863+0000","last_active":"2024-04-25T10:11:01.791772+0000","last_peered":"2024-04-25T10:11:01.791772+0000","last_clean":"2024-04-25T10:11:01.791772+0000","last_became_active":"2024-04-25T10:10:59.661598+0000","last_became_peered":"2024-04-25T10:10:59.661598+0000","last_unstale":"2024-04-25T10:11:01.791772+0000","last_undegraded":"2024-04-25T10:11:01.791772+0000","last_fullsized":"2024-04-25T10:11:01.791772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:45.820044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791785+0000","last_change":"2024-04-25T10:11:00.666115+0000","last_active":"2024-04-25T10:11:01.791785+0000","last_peered":"2024-04-25T10:11:01.791785+0000","last_clean":"2024-04-25T10:11:01.791785+0000","last_became_active":"2024-04-25T10:11:00.665817+0000","last_became_peered":"2024-04-25T10:11:00.665817+0000","last_unstale":"2024-04-25T10:11:01.791785+0000","last_undegraded":"2024-04-25T10:11:01.791785+0000","last_fullsized":"2024-04-25T10:11:01.791785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:10.823889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.670527+0000","last_change":"2024-04-25T10:10:59.654401+0000","last_active":"2024-04-25T10:11:01.670527+0000","last_peered":"2024-04-25T10:11:01.670527+0000","last_clean":"2024-04-25T10:11:01.670527+0000","last_became_active":"2024-04-25T10:10:59.654067+0000","last_became_peered":"2024-04-25T10:10:59.654067+0000","last_unstale":"2024-04-25T10:11:01.670527+0000","last_undegraded":"2024-04-25T10:11:01.670527+0000","last_fullsized":"2024-04-25T10:11:01.670527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:45.817445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791696+0000","last_change":"2024-04-25T10:11:00.671112+0000","last_active":"2024-04-25T10:11:01.791696+0000","last_peered":"2024-04-25T10:11:01.791696+0000","last_clean":"2024-04-25T10:11:01.791696+0000","last_became_active":"2024-04-25T10:11:00.670792+0000","last_became_peered":"2024-04-25T10:11:00.670792+0000","last_unstale":"2024-04-25T10:11:01.791696+0000","last_undegraded":"2024-04-25T10:11:01.791696+0000","last_fullsized":"2024-04-25T10:11:01.791696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:00.467722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790958+0000","last_change":"2024-04-25T10:10:59.660751+0000","last_active":"2024-04-25T10:11:01.790958+0000","last_peered":"2024-04-25T10:11:01.790958+0000","last_clean":"2024-04-25T10:11:01.790958+0000","last_became_active":"2024-04-25T10:10:59.660439+0000","last_became_peered":"2024-04-25T10:10:59.660439+0000","last_unstale":"2024-04-25T10:11:01.790958+0000","last_undegraded":"2024-04-25T10:11:01.790958+0000","last_fullsized":"2024-04-25T10:11:01.790958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:17.523240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666630+0000","last_change":"2024-04-25T10:11:00.662687+0000","last_active":"2024-04-25T10:11:01.666630+0000","last_peered":"2024-04-25T10:11:01.666630+0000","last_clean":"2024-04-25T10:11:01.666630+0000","last_became_active":"2024-04-25T10:11:00.662430+0000","last_became_peered":"2024-04-25T10:11:00.662430+0000","last_unstale":"2024-04-25T10:11:01.666630+0000","last_undegraded":"2024-04-25T10:11:01.666630+0000","last_fullsized":"2024-04-25T10:11:01.666630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:14.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666396+0000","last_change":"2024-04-25T10:10:59.660156+0000","last_active":"2024-04-25T10:11:01.666396+0000","last_peered":"2024-04-25T10:11:01.666396+0000","last_clean":"2024-04-25T10:11:01.666396+0000","last_became_active":"2024-04-25T10:10:59.659878+0000","last_became_peered":"2024-04-25T10:10:59.659878+0000","last_unstale":"2024-04-25T10:11:01.666396+0000","last_undegraded":"2024-04-25T10:11:01.666396+0000","last_fullsized":"2024-04-25T10:11:01.666396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:55.986122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669254+0000","last_change":"2024-04-25T10:11:00.664855+0000","last_active":"2024-04-25T10:11:01.669254+0000","last_peered":"2024-04-25T10:11:01.669254+0000","last_clean":"2024-04-25T10:11:01.669254+0000","last_became_active":"2024-04-25T10:11:00.664174+0000","last_became_peered":"2024-04-25T10:11:00.664174+0000","last_unstale":"2024-04-25T10:11:01.669254+0000","last_undegraded":"2024-04-25T10:11:01.669254+0000","last_fullsized":"2024-04-25T10:11:01.669254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.235679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.672299+0000","last_change":"2024-04-25T10:10:59.656397+0000","last_active":"2024-04-25T10:11:01.672299+0000","last_peered":"2024-04-25T10:11:01.672299+0000","last_clean":"2024-04-25T10:11:01.672299+0000","last_became_active":"2024-04-25T10:10:59.656114+0000","last_became_peered":"2024-04-25T10:10:59.656114+0000","last_unstale":"2024-04-25T10:11:01.672299+0000","last_undegraded":"2024-04-25T10:11:01.672299+0000","last_fullsized":"2024-04-25T10:11:01.672299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:39.088943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669867+0000","last_change":"2024-04-25T10:11:00.675967+0000","last_active":"2024-04-25T10:11:01.669867+0000","last_peered":"2024-04-25T10:11:01.669867+0000","last_clean":"2024-04-25T10:11:01.669867+0000","last_became_active":"2024-04-25T10:11:00.673547+0000","last_became_peered":"2024-04-25T10:11:00.673547+0000","last_unstale":"2024-04-25T10:11:01.669867+0000","last_undegraded":"2024-04-25T10:11:01.669867+0000","last_fullsized":"2024-04-25T10:11:01.669867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:43.118238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.666689+0000","last_change":"2024-04-25T10:10:59.661201+0000","last_active":"2024-04-25T10:11:01.666689+0000","last_peered":"2024-04-25T10:11:01.666689+0000","last_clean":"2024-04-25T10:11:01.666689+0000","last_became_active":"2024-04-25T10:10:59.660913+0000","last_became_peered":"2024-04-25T10:10:59.660913+0000","last_unstale":"2024-04-25T10:11:01.666689+0000","last_undegraded":"2024-04-25T10:11:01.666689+0000","last_fullsized":"2024-04-25T10:11:01.666689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:37.676864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669208+0000","last_change":"2024-04-25T10:11:00.664505+0000","last_active":"2024-04-25T10:11:01.669208+0000","last_peered":"2024-04-25T10:11:01.669208+0000","last_clean":"2024-04-25T10:11:01.669208+0000","last_became_active":"2024-04-25T10:11:00.664218+0000","last_became_peered":"2024-04-25T10:11:00.664218+0000","last_unstale":"2024-04-25T10:11:01.669208+0000","last_undegraded":"2024-04-25T10:11:01.669208+0000","last_fullsized":"2024-04-25T10:11:01.669208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.603074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669199+0000","last_change":"2024-04-25T10:10:59.652758+0000","last_active":"2024-04-25T10:11:01.669199+0000","last_peered":"2024-04-25T10:11:01.669199+0000","last_clean":"2024-04-25T10:11:01.669199+0000","last_became_active":"2024-04-25T10:10:59.652456+0000","last_became_peered":"2024-04-25T10:10:59.652456+0000","last_unstale":"2024-04-25T10:11:01.669199+0000","last_undegraded":"2024-04-25T10:11:01.669199+0000","last_fullsized":"2024-04-25T10:11:01.669199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:53.008632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669239+0000","last_change":"2024-04-25T10:11:00.660736+0000","last_active":"2024-04-25T10:11:01.669239+0000","last_peered":"2024-04-25T10:11:01.669239+0000","last_clean":"2024-04-25T10:11:01.669239+0000","last_became_active":"2024-04-25T10:11:00.660471+0000","last_became_peered":"2024-04-25T10:11:00.660471+0000","last_unstale":"2024-04-25T10:11:01.669239+0000","last_undegraded":"2024-04-25T10:11:01.669239+0000","last_fullsized":"2024-04-25T10:11:01.669239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:37.071670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.790521+0000","last_change":"2024-04-25T10:10:59.660087+0000","last_active":"2024-04-25T10:11:01.790521+0000","last_peered":"2024-04-25T10:11:01.790521+0000","last_clean":"2024-04-25T10:11:01.790521+0000","last_became_active":"2024-04-25T10:10:59.659815+0000","last_became_peered":"2024-04-25T10:10:59.659815+0000","last_unstale":"2024-04-25T10:11:01.790521+0000","last_undegraded":"2024-04-25T10:11:01.790521+0000","last_fullsized":"2024-04-25T10:11:01.790521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:42.113345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669804+0000","last_change":"2024-04-25T10:11:00.674881+0000","last_active":"2024-04-25T10:11:01.669804+0000","last_peered":"2024-04-25T10:11:01.669804+0000","last_clean":"2024-04-25T10:11:01.669804+0000","last_became_active":"2024-04-25T10:11:00.674632+0000","last_became_peered":"2024-04-25T10:11:00.674632+0000","last_unstale":"2024-04-25T10:11:01.669804+0000","last_undegraded":"2024-04-25T10:11:01.669804+0000","last_fullsized":"2024-04-25T10:11:01.669804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.639238+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.639238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:36.103452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.791702+0000","last_change":"2024-04-25T10:10:59.663122+0000","last_active":"2024-04-25T10:11:01.791702+0000","last_peered":"2024-04-25T10:11:01.791702+0000","last_clean":"2024-04-25T10:11:01.791702+0000","last_became_active":"2024-04-25T10:10:59.662664+0000","last_became_peered":"2024-04-25T10:10:59.662664+0000","last_unstale":"2024-04-25T10:11:01.791702+0000","last_undegraded":"2024-04-25T10:11:01.791702+0000","last_fullsized":"2024-04-25T10:11:01.791702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.634346+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.634346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:16.256009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":301,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T10:11:01.669306+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:11:01.669306+0000","last_peered":"2024-04-25T10:11:01.669306+0000","last_clean":"2024-04-25T10:11:01.669306+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:11:01.669306+0000","last_undegraded":"2024-04-25T10:11:01.669306+0000","last_fullsized":"2024-04-25T10:11:01.669306+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509517,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1052672,"data_stored":446360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509517,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1056768,"data_stored":446962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509518,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1052672,"data_stored":446324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509518,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335460,"statfs":{"total":96636764160,"available":96599511040,"internally_reserved":0,"allocated":1060864,"data_stored":446964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509519,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335472,"statfs":{"total":96636764160,"available":96599523328,"internally_reserved":0,"allocated":1048576,"data_stored":446290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509517,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1646592,"data_stored":1036748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509519,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1064960,"data_stored":447924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1658880,"data_stored":1038304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:08.444 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:11:08.445 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:11:08.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.584+0000 7fcbb3f7e640 1 Processor -- start 2024-04-25T10:11:08.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.584+0000 7fcbb3f7e640 1 -- start start 2024-04-25T10:11:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.584+0000 7fcbb3f7e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fcbac0d03e0 0x7fcbac0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.584+0000 7fcbb3f7e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbac15dd90 0x7fcbac0d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.585+0000 7fcbb3f7e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcbac0d1a30 0x7fcbac0b5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.585+0000 7fcbb3f7e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbac0d0d00 con 0x7fcbac0d1a30 2024-04-25T10:11:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.585+0000 7fcbb3f7e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbac0d2410 con 0x7fcbac0d03e0 2024-04-25T10:11:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.585+0000 7fcbb3f7e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0d26f0 con 0x7fcbac15dd90 2024-04-25T10:11:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.585+0000 7fcbb251b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcbac0d1a30 0x7fcbac0b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.585+0000 7fcbb251b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcbac0d1a30 0x7fcbac0b5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53316/0 (socket says 172.21.15.16:53316) 2024-04-25T10:11:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.585+0000 7fcbb251b640 1 -- 172.21.15.16:0/304229346 learned_addr learned my addr 172.21.15.16:0/304229346 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.585+0000 7fcbb1d1a640 1 --2- 172.21.15.16:0/304229346 >> v2:172.21.15.186:3300/0 conn(0x7fcbac0d03e0 0x7fcbac0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.585+0000 7fcbb1519640 1 --2- 172.21.15.16:0/304229346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbac15dd90 0x7fcbac0d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.585+0000 7fcbb251b640 1 -- 172.21.15.16:0/304229346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbac15dd90 msgr2=0x7fcbac0d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.585+0000 7fcbb251b640 1 --2- 172.21.15.16:0/304229346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbac15dd90 0x7fcbac0d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.585+0000 7fcbb251b640 1 -- 172.21.15.16:0/304229346 >> v2:172.21.15.186:3300/0 conn(0x7fcbac0d03e0 msgr2=0x7fcbac0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.585+0000 7fcbb251b640 1 --2- 172.21.15.16:0/304229346 >> v2:172.21.15.186:3300/0 conn(0x7fcbac0d03e0 0x7fcbac0d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.585+0000 7fcbb251b640 1 -- 172.21.15.16:0/304229346 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac0d29d0 con 0x7fcbac0d1a30 2024-04-25T10:11:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.586+0000 7fcbb251b640 1 --2- 172.21.15.16:0/304229346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac0d1a30 0x7fcbac0b5490 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0d1400 tx=0x7fcba402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547b1b73eaa3a8c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.586+0000 7fcbb0d18640 1 -- 172.21.15.16:0/304229346 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4036070 con 0x7fcbac0d1a30 2024-04-25T10:11:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.586+0000 7fcbb0d18640 1 -- 172.21.15.16:0/304229346 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba4031600 con 0x7fcbac0d1a30 2024-04-25T10:11:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.586+0000 7fcbb0d18640 1 -- 172.21.15.16:0/304229346 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba40319d0 con 0x7fcbac0d1a30 2024-04-25T10:11:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.586+0000 7fcbb3f7e640 1 -- 172.21.15.16:0/304229346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac0d1a30 msgr2=0x7fcbac0b5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.586+0000 7fcbb3f7e640 1 --2- 172.21.15.16:0/304229346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac0d1a30 0x7fcbac0b5490 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0d1400 tx=0x7fcba402a730 comp rx=0 tx=0).stop 2024-04-25T10:11:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.586+0000 7fcbb3f7e640 1 -- 172.21.15.16:0/304229346 shutdown_connections 2024-04-25T10:11:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.586+0000 7fcbb3f7e640 1 --2- 172.21.15.16:0/304229346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac0d1a30 0x7fcbac0b5490 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.586+0000 7fcbb3f7e640 1 --2- 172.21.15.16:0/304229346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbac15dd90 0x7fcbac0d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.586+0000 7fcbb3f7e640 1 --2- 172.21.15.16:0/304229346 >> v2:172.21.15.186:3300/0 conn(0x7fcbac0d03e0 0x7fcbac0d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.586+0000 7fcbb3f7e640 1 -- 172.21.15.16:0/304229346 >> 172.21.15.16:0/304229346 conn(0x7fcbac072900 msgr2=0x7fcbac06c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:08.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.587+0000 7fcbb3f7e640 1 -- 172.21.15.16:0/304229346 shutdown_connections 2024-04-25T10:11:08.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.587+0000 7fcbb3f7e640 1 -- 172.21.15.16:0/304229346 wait complete. 2024-04-25T10:11:08.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.588+0000 7fcbb3f7e640 1 Processor -- start 2024-04-25T10:11:08.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.588+0000 7fcbb3f7e640 1 -- start start 2024-04-25T10:11:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.588+0000 7fcbb3f7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac0d03e0 0x7fcbac0c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.588+0000 7fcbb3f7e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbac0d1a30 0x7fcbac0cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.588+0000 7fcbb3f7e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbac15dd90 0x7fcbac0ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.588+0000 7fcbb3f7e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0b9820 con 0x7fcbac0d03e0 2024-04-25T10:11:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.588+0000 7fcbb3f7e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0d2f90 con 0x7fcbac15dd90 2024-04-25T10:11:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.588+0000 7fcbb3f7e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0b99a0 con 0x7fcbac0d1a30 2024-04-25T10:11:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.589+0000 7fcbb1d1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac0d03e0 0x7fcbac0c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.589+0000 7fcbb1d1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac0d03e0 0x7fcbac0c6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53320/0 (socket says 172.21.15.16:53320) 2024-04-25T10:11:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.589+0000 7fcbb1d1a640 1 -- 172.21.15.16:0/392227270 learned_addr learned my addr 172.21.15.16:0/392227270 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.589+0000 7fcbb1d1a640 1 -- 172.21.15.16:0/392227270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbac0d1a30 msgr2=0x7fcbac0cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.589+0000 7fcbb1d1a640 1 --2- 172.21.15.16:0/392227270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbac0d1a30 0x7fcbac0cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.589+0000 7fcbb1d1a640 1 -- 172.21.15.16:0/392227270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbac15dd90 msgr2=0x7fcbac0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.589+0000 7fcbb1d1a640 1 --2- 172.21.15.16:0/392227270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbac15dd90 0x7fcbac0ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.589+0000 7fcbb1d1a640 1 -- 172.21.15.16:0/392227270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac158830 con 0x7fcbac0d03e0 2024-04-25T10:11:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.589+0000 7fcbb1d1a640 1 --2- 172.21.15.16:0/392227270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac0d03e0 0x7fcbac0c6750 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fcba800e870 tx=0x7fcba800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.589+0000 7fcba27fc640 1 -- 172.21.15.16:0/392227270 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8017070 con 0x7fcbac0d03e0 2024-04-25T10:11:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.589+0000 7fcba27fc640 1 -- 172.21.15.16:0/392227270 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba800c940 con 0x7fcbac0d03e0 2024-04-25T10:11:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.589+0000 7fcbb3f7e640 1 -- 172.21.15.16:0/392227270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbac158b10 con 0x7fcbac0d03e0 2024-04-25T10:11:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.589+0000 7fcba27fc640 1 -- 172.21.15.16:0/392227270 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba800cd30 con 0x7fcbac0d03e0 2024-04-25T10:11:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.589+0000 7fcbb3f7e640 1 -- 172.21.15.16:0/392227270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbac1590d0 con 0x7fcbac0d03e0 2024-04-25T10:11:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.590+0000 7fcbb3f7e640 1 -- 172.21.15.16:0/392227270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbac0749b0 con 0x7fcbac0d03e0 2024-04-25T10:11:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.591+0000 7fcba27fc640 1 -- 172.21.15.16:0/392227270 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcba801a070 con 0x7fcbac0d03e0 2024-04-25T10:11:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.591+0000 7fcba27fc640 1 --2- 172.21.15.16:0/392227270 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb9006b4d0 0x7fcb9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.591+0000 7fcba27fc640 1 -- 172.21.15.16:0/392227270 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcba808a3c0 con 0x7fcbac0d03e0 2024-04-25T10:11:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.591+0000 7fcbb1519640 1 --2- 172.21.15.16:0/392227270 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb9006b4d0 0x7fcb9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.592+0000 7fcbb1519640 1 --2- 172.21.15.16:0/392227270 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb9006b4d0 0x7fcb9006d990 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c006a30 tx=0x7fcb9c0069c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.595+0000 7fcba27fc640 1 -- 172.21.15.16:0/392227270 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcba80592d0 con 0x7fcbac0d03e0 2024-04-25T10:11:08.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.851+0000 7fcbb3f7e640 1 -- 172.21.15.16:0/392227270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fcbac06b1d0 con 0x7fcbac0d03e0 2024-04-25T10:11:08.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.853+0000 7fcba27fc640 1 -- 172.21.15.16:0/392227270 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7fcba8058bd0 con 0x7fcbac0d03e0 2024-04-25T10:11:08.859 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-04-25T10:11:08.859 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCZKipmAph/GhAAG3x1zC98yHQ13fjpmrlPkg== 2024-04-25T10:11:08.859 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-04-25T10:11:08.859 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-04-25T10:11:08.859 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:11:08.859 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-04-25T10:11:08.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.855+0000 7fcbb3f7e640 1 -- 172.21.15.16:0/392227270 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb9006b4d0 msgr2=0x7fcb9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.855+0000 7fcbb3f7e640 1 --2- 172.21.15.16:0/392227270 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb9006b4d0 0x7fcb9006d990 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c006a30 tx=0x7fcb9c0069c0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.855+0000 7fcbb3f7e640 1 -- 172.21.15.16:0/392227270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac0d03e0 msgr2=0x7fcbac0c6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.855+0000 7fcbb3f7e640 1 --2- 172.21.15.16:0/392227270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac0d03e0 0x7fcbac0c6750 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fcba800e870 tx=0x7fcba800ed30 comp rx=0 tx=0).stop 2024-04-25T10:11:08.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.855+0000 7fcbb3f7e640 1 -- 172.21.15.16:0/392227270 shutdown_connections 2024-04-25T10:11:08.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.855+0000 7fcbb3f7e640 1 --2- 172.21.15.16:0/392227270 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcb9006b4d0 0x7fcb9006d990 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.855+0000 7fcbb3f7e640 1 --2- 172.21.15.16:0/392227270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbac15dd90 0x7fcbac0ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.855+0000 7fcbb3f7e640 1 --2- 172.21.15.16:0/392227270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbac0d1a30 0x7fcbac0cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.855+0000 7fcbb3f7e640 1 --2- 172.21.15.16:0/392227270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac0d03e0 0x7fcbac0c6750 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.855+0000 7fcbb3f7e640 1 -- 172.21.15.16:0/392227270 >> 172.21.15.16:0/392227270 conn(0x7fcbac072900 msgr2=0x7fcbac0b8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.855+0000 7fcbb3f7e640 1 -- 172.21.15.16:0/392227270 shutdown_connections 2024-04-25T10:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.855+0000 7fcbb3f7e640 1 -- 172.21.15.16:0/392227270 wait complete. 2024-04-25T10:11:08.871 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:08.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.967+0000 7f82f010a640 1 Processor -- start 2024-04-25T10:11:08.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.967+0000 7f82f010a640 1 -- start start 2024-04-25T10:11:08.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.968+0000 7f82f010a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82e815dce0 0x7f82e80d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.968+0000 7f82f010a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f82e80b5b60 0x7f82e80b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.968+0000 7f82f010a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e815e0c0 con 0x7f82e80d35b0 2024-04-25T10:11:08.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.968+0000 7f82f010a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e80d2510 con 0x7f82e80b5b60 2024-04-25T10:11:08.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.968+0000 7f82f010a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80d27f0 con 0x7f82e815dce0 2024-04-25T10:11:08.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.969+0000 7f82ee6a7640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f82e80b5b60 0x7f82e80b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.969+0000 7f82edea6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82e815dce0 0x7f82e80d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.969+0000 7f82edea6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82e815dce0 0x7f82e80d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:35836/0 (socket says 172.21.15.16:35836) 2024-04-25T10:11:08.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.969+0000 7f82edea6640 1 -- 172.21.15.16:0/550640716 learned_addr learned my addr 172.21.15.16:0/550640716 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:08.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.969+0000 7f82ed6a5640 1 --1- 172.21.15.16:0/550640716 >> v1:172.21.15.186:6789/0 conn(0x7f82e80d35b0 0x7f82e80b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:47336/0 (socket says 172.21.15.16:47336) 2024-04-25T10:11:08.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.969+0000 7f82edea6640 1 -- 172.21.15.16:0/550640716 >> v2:172.21.15.186:3300/0 conn(0x7f82e80b5b60 msgr2=0x7f82e80b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:08.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.969+0000 7f82edea6640 1 --2- 172.21.15.16:0/550640716 >> v2:172.21.15.186:3300/0 conn(0x7f82e80b5b60 0x7f82e80b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.969+0000 7f82edea6640 1 -- 172.21.15.16:0/550640716 >> v1:172.21.15.186:6789/0 conn(0x7f82e80d35b0 legacy=0x7f82e80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.969+0000 7f82edea6640 1 -- 172.21.15.16:0/550640716 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e80d2ad0 con 0x7f82e815dce0 2024-04-25T10:11:08.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.970+0000 7f82edea6640 1 --2- 172.21.15.16:0/550640716 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82e815dce0 0x7f82e80d1a50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f82d8009e50 tx=0x7f82d802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b32af546c5f1007f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.970+0000 7f82ecea4640 1 -- 172.21.15.16:0/550640716 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d8036070 con 0x7f82e815dce0 2024-04-25T10:11:08.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.970+0000 7f82ecea4640 1 -- 172.21.15.16:0/550640716 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d8031600 con 0x7f82e815dce0 2024-04-25T10:11:08.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.970+0000 7f82ecea4640 1 -- 172.21.15.16:0/550640716 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d80319d0 con 0x7f82e815dce0 2024-04-25T10:11:08.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.970+0000 7f82f010a640 1 -- 172.21.15.16:0/550640716 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82e815dce0 msgr2=0x7f82e80d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.970+0000 7f82f010a640 1 --2- 172.21.15.16:0/550640716 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82e815dce0 0x7f82e80d1a50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f82d8009e50 tx=0x7f82d802a710 comp rx=0 tx=0).stop 2024-04-25T10:11:08.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.970+0000 7f82f010a640 1 -- 172.21.15.16:0/550640716 shutdown_connections 2024-04-25T10:11:08.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.970+0000 7f82f010a640 1 --2- 172.21.15.16:0/550640716 >> v2:172.21.15.186:3300/0 conn(0x7f82e80b5b60 0x7f82e80b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.970+0000 7f82f010a640 1 --2- 172.21.15.16:0/550640716 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82e815dce0 0x7f82e80d1a50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.970+0000 7f82f010a640 1 -- 172.21.15.16:0/550640716 >> 172.21.15.16:0/550640716 conn(0x7f82e8072900 msgr2=0x7f82e80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:08.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.971+0000 7f82f010a640 1 -- 172.21.15.16:0/550640716 shutdown_connections 2024-04-25T10:11:08.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.971+0000 7f82f010a640 1 -- 172.21.15.16:0/550640716 wait complete. 2024-04-25T10:11:08.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.972+0000 7f82f010a640 1 Processor -- start 2024-04-25T10:11:08.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.972+0000 7f82f010a640 1 -- start start 2024-04-25T10:11:08.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.972+0000 7f82f010a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82e80b5b60 0x7f82e81f6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82f010a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82e80d35b0 0x7f82e80cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82f010a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82e815dce0 0x7f82e80cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82f010a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80b9820 con 0x7f82e80d35b0 2024-04-25T10:11:08.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82f010a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80d3090 con 0x7f82e80b5b60 2024-04-25T10:11:08.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82f010a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80b99a0 con 0x7f82e815dce0 2024-04-25T10:11:08.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82ed6a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82e80d35b0 0x7f82e80cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82ed6a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82e80d35b0 0x7f82e80cbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53326/0 (socket says 172.21.15.16:53326) 2024-04-25T10:11:08.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82ed6a5640 1 -- 172.21.15.16:0/2395889837 learned_addr learned my addr 172.21.15.16:0/2395889837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:08.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82ee6a7640 1 --2- 172.21.15.16:0/2395889837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82e815dce0 0x7f82e80cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82edea6640 1 --2- 172.21.15.16:0/2395889837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82e80b5b60 0x7f82e81f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82ed6a5640 1 -- 172.21.15.16:0/2395889837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82e815dce0 msgr2=0x7f82e80cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82ed6a5640 1 --2- 172.21.15.16:0/2395889837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82e815dce0 0x7f82e80cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82ed6a5640 1 -- 172.21.15.16:0/2395889837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82e80b5b60 msgr2=0x7f82e81f6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82ed6a5640 1 --2- 172.21.15.16:0/2395889837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82e80b5b60 0x7f82e81f6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82ed6a5640 1 -- 172.21.15.16:0/2395889837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e80c94e0 con 0x7f82e80d35b0 2024-04-25T10:11:08.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82ed6a5640 1 --2- 172.21.15.16:0/2395889837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82e80d35b0 0x7f82e80cbf50 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00c400 tx=0x7f82dc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82e67fc640 1 -- 172.21.15.16:0/2395889837 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0074e0 con 0x7f82e80d35b0 2024-04-25T10:11:08.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82e67fc640 1 -- 172.21.15.16:0/2395889837 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc007680 con 0x7f82e80d35b0 2024-04-25T10:11:08.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82f010a640 1 -- 172.21.15.16:0/2395889837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e80c97c0 con 0x7f82e80d35b0 2024-04-25T10:11:08.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.973+0000 7f82e67fc640 1 -- 172.21.15.16:0/2395889837 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc004a00 con 0x7f82e80d35b0 2024-04-25T10:11:08.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.974+0000 7f82f010a640 1 -- 172.21.15.16:0/2395889837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e80c9d80 con 0x7f82e80d35b0 2024-04-25T10:11:08.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.974+0000 7f82f010a640 1 -- 172.21.15.16:0/2395889837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e80749b0 con 0x7f82e80d35b0 2024-04-25T10:11:08.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.975+0000 7f82e67fc640 1 -- 172.21.15.16:0/2395889837 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82dc01a020 con 0x7f82e80d35b0 2024-04-25T10:11:08.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.975+0000 7f82e67fc640 1 --2- 172.21.15.16:0/2395889837 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f82d406b4d0 0x7f82d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.976+0000 7f82e67fc640 1 -- 172.21.15.16:0/2395889837 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82dc089a30 con 0x7f82e80d35b0 2024-04-25T10:11:08.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.976+0000 7f82edea6640 1 --2- 172.21.15.16:0/2395889837 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f82d406b4d0 0x7f82d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.976+0000 7f82edea6640 1 --2- 172.21.15.16:0/2395889837 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f82d406b4d0 0x7f82d406d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f82d8033000 tx=0x7f82d8009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:08.980+0000 7f82e67fc640 1 -- 172.21.15.16:0/2395889837 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82dc058820 con 0x7f82e80d35b0 2024-04-25T10:11:09.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.204+0000 7f82f010a640 1 -- 172.21.15.16:0/2395889837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82e80d0680 con 0x7f82e80d35b0 2024-04-25T10:11:09.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.205+0000 7f82e67fc640 1 -- 172.21.15.16:0/2395889837 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f82dc058120 con 0x7f82e80d35b0 2024-04-25T10:11:09.210 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:09.211 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4044509901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4044509901},{"type":"v1","addr":"172.21.15.16:6835","nonce":4044509901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/84349474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":84349474},{"type":"v1","addr":"172.21.15.16:6834","nonce":84349474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7335,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/3950732718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3950732718},{"type":"v1","addr":"172.21.15.186:6836","nonce":3950732718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:00.015075+0000","modified":"2024-04-25T10:11:02.007788+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35852":{"gid":35852,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/3092291936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":3092291936},{"type":"v1","addr":"172.21.15.186:6837","nonce":3092291936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:09.211 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 51 2024-04-25T10:11:09.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.207+0000 7f82f010a640 1 -- 172.21.15.16:0/2395889837 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f82d406b4d0 msgr2=0x7f82d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.207+0000 7f82f010a640 1 --2- 172.21.15.16:0/2395889837 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f82d406b4d0 0x7f82d406d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f82d8033000 tx=0x7f82d8009000 comp rx=0 tx=0).stop 2024-04-25T10:11:09.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.207+0000 7f82f010a640 1 -- 172.21.15.16:0/2395889837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82e80d35b0 msgr2=0x7f82e80cbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.207+0000 7f82f010a640 1 --2- 172.21.15.16:0/2395889837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82e80d35b0 0x7f82e80cbf50 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00c400 tx=0x7f82dc00c8c0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.207+0000 7f82f010a640 1 -- 172.21.15.16:0/2395889837 shutdown_connections 2024-04-25T10:11:09.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.207+0000 7f82f010a640 1 --2- 172.21.15.16:0/2395889837 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f82d406b4d0 0x7f82d406d990 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.207+0000 7f82f010a640 1 --2- 172.21.15.16:0/2395889837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82e815dce0 0x7f82e80cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.207+0000 7f82f010a640 1 --2- 172.21.15.16:0/2395889837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82e80d35b0 0x7f82e80cbf50 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.207+0000 7f82f010a640 1 --2- 172.21.15.16:0/2395889837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82e80b5b60 0x7f82e81f6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.207+0000 7f82f010a640 1 -- 172.21.15.16:0/2395889837 >> 172.21.15.16:0/2395889837 conn(0x7f82e8072900 msgr2=0x7f82e80b53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.207+0000 7f82f010a640 1 -- 172.21.15.16:0/2395889837 shutdown_connections 2024-04-25T10:11:09.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.207+0000 7f82f010a640 1 -- 172.21.15.16:0/2395889837 wait complete. 2024-04-25T10:11:09.627 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:00.015075+0000', 'modified': '2024-04-25T10:11:02.007788+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 35852}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_35852': {'gid': 35852, 'name': 'b', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.186:6837/3092291936', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6835', 'nonce': 3092291936}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 3092291936}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:09.627 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:11:09.628 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:11:09.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.727+0000 7f3e817d5640 1 Processor -- start 2024-04-25T10:11:09.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.727+0000 7f3e817d5640 1 -- start start 2024-04-25T10:11:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.728+0000 7f3e817d5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3e7c0b0740 0x7f3e7c0b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.728+0000 7f3e817d5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e7c15f660 con 0x7f3e7c15ef60 2024-04-25T10:11:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.728+0000 7f3e817d5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e7c15f940 con 0x7f3e7c0cfd60 2024-04-25T10:11:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.728+0000 7f3e817d5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7c15fc20 con 0x7f3e7c0b0740 2024-04-25T10:11:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.728+0000 7f3e7affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3e7c15ef60 0x7f3e7c0ce4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55510/0 (socket says 172.21.15.16:55510) 2024-04-25T10:11:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.728+0000 7f3e7affd640 1 -- 172.21.15.16:0/3021848637 learned_addr learned my addr 172.21.15.16:0/3021848637 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.729+0000 7f3e7a7fc640 1 --2- 172.21.15.16:0/3021848637 >> v2:172.21.15.186:3300/0 conn(0x7f3e7c0b0740 0x7f3e7c0b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.729+0000 7f3e79ffb640 1 -- 172.21.15.16:0/3021848637 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2324486588 0 0) 0x7f3e7c15f660 con 0x7f3e7c15ef60 2024-04-25T10:11:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.729+0000 7f3e79ffb640 1 -- 172.21.15.16:0/3021848637 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e64003520 con 0x7f3e7c15ef60 2024-04-25T10:11:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.729+0000 7f3e7a7fc640 1 -- 172.21.15.16:0/3021848637 >> v1:172.21.15.186:6789/0 conn(0x7f3e7c0cfd60 legacy=0x7f3e7c0d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.729+0000 7f3e7a7fc640 1 -- 172.21.15.16:0/3021848637 >> v1:172.21.15.16:6789/0 conn(0x7f3e7c15ef60 legacy=0x7f3e7c0ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.729+0000 7f3e7a7fc640 1 -- 172.21.15.16:0/3021848637 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c15ff00 con 0x7f3e7c0b0740 2024-04-25T10:11:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.730+0000 7f3e7a7fc640 1 --2- 172.21.15.16:0/3021848637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c0b0740 0x7f3e7c0b0bd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3e700097a0 tx=0x7f3e7002a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f9cf54cd4e7ae1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.730+0000 7f3e79ffb640 1 -- 172.21.15.16:0/3021848637 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70037070 con 0x7f3e7c0b0740 2024-04-25T10:11:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.730+0000 7f3e79ffb640 1 -- 172.21.15.16:0/3021848637 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e70032640 con 0x7f3e7c0b0740 2024-04-25T10:11:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.730+0000 7f3e79ffb640 1 -- 172.21.15.16:0/3021848637 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70032a10 con 0x7f3e7c0b0740 2024-04-25T10:11:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.730+0000 7f3e817d5640 1 -- 172.21.15.16:0/3021848637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c0b0740 msgr2=0x7f3e7c0b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.730+0000 7f3e817d5640 1 --2- 172.21.15.16:0/3021848637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c0b0740 0x7f3e7c0b0bd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3e700097a0 tx=0x7f3e7002a2c0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.730+0000 7f3e817d5640 1 -- 172.21.15.16:0/3021848637 shutdown_connections 2024-04-25T10:11:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.730+0000 7f3e817d5640 1 --2- 172.21.15.16:0/3021848637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c0b0740 0x7f3e7c0b0bd0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.730+0000 7f3e817d5640 1 -- 172.21.15.16:0/3021848637 >> 172.21.15.16:0/3021848637 conn(0x7f3e7c06f4c0 msgr2=0x7f3e7c06f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.730+0000 7f3e817d5640 1 -- 172.21.15.16:0/3021848637 shutdown_connections 2024-04-25T10:11:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.730+0000 7f3e817d5640 1 -- 172.21.15.16:0/3021848637 wait complete. 2024-04-25T10:11:09.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.732+0000 7f3e817d5640 1 Processor -- start 2024-04-25T10:11:09.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.732+0000 7f3e817d5640 1 -- start start 2024-04-25T10:11:09.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.732+0000 7f3e817d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e7c0b0740 0x7f3e7c0afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.732+0000 7f3e817d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e7c0cfd60 0x7f3e7c15b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.732+0000 7f3e817d5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c15ef60 0x7f3e7c15bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.732+0000 7f3e817d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c15f340 con 0x7f3e7c0cfd60 2024-04-25T10:11:09.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.732+0000 7f3e817d5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c1604c0 con 0x7f3e7c15ef60 2024-04-25T10:11:09.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.732+0000 7f3e817d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0cebd0 con 0x7f3e7c0b0740 2024-04-25T10:11:09.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.733+0000 7f3e7a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e7c0cfd60 0x7f3e7c15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.733+0000 7f3e7a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e7c0cfd60 0x7f3e7c15b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53328/0 (socket says 172.21.15.16:53328) 2024-04-25T10:11:09.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.733+0000 7f3e7a7fc640 1 -- 172.21.15.16:0/3689787162 learned_addr learned my addr 172.21.15.16:0/3689787162 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:09.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.733+0000 7f3e7affd640 1 --2- 172.21.15.16:0/3689787162 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e7c0b0740 0x7f3e7c0afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.733+0000 7f3e7b7fe640 1 --2- 172.21.15.16:0/3689787162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c15ef60 0x7f3e7c15bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.733+0000 7f3e7a7fc640 1 -- 172.21.15.16:0/3689787162 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e7c0b0740 msgr2=0x7f3e7c0afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.733+0000 7f3e7a7fc640 1 --2- 172.21.15.16:0/3689787162 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e7c0b0740 0x7f3e7c0afc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.733+0000 7f3e7a7fc640 1 -- 172.21.15.16:0/3689787162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c15ef60 msgr2=0x7f3e7c15bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.733+0000 7f3e7a7fc640 1 --2- 172.21.15.16:0/3689787162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c15ef60 0x7f3e7c15bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.733+0000 7f3e7a7fc640 1 -- 172.21.15.16:0/3689787162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c0b01b0 con 0x7f3e7c0cfd60 2024-04-25T10:11:09.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.733+0000 7f3e7a7fc640 1 --2- 172.21.15.16:0/3689787162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e7c0cfd60 0x7f3e7c15b860 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3e70035040 tx=0x7f3e700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.733+0000 7f3e637fe640 1 -- 172.21.15.16:0/3689787162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70037040 con 0x7f3e7c0cfd60 2024-04-25T10:11:09.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.733+0000 7f3e637fe640 1 -- 172.21.15.16:0/3689787162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e70032440 con 0x7f3e7c0cfd60 2024-04-25T10:11:09.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.733+0000 7f3e817d5640 1 -- 172.21.15.16:0/3689787162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7c0c7050 con 0x7f3e7c0cfd60 2024-04-25T10:11:09.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.733+0000 7f3e637fe640 1 -- 172.21.15.16:0/3689787162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e700309d0 con 0x7f3e7c0cfd60 2024-04-25T10:11:09.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.733+0000 7f3e817d5640 1 -- 172.21.15.16:0/3689787162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c0c7610 con 0x7f3e7c0cfd60 2024-04-25T10:11:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.735+0000 7f3e637fe640 1 -- 172.21.15.16:0/3689787162 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e70044020 con 0x7f3e7c0cfd60 2024-04-25T10:11:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.735+0000 7f3e637fe640 1 --2- 172.21.15.16:0/3689787162 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3e6806b4d0 0x7f3e6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.735+0000 7f3e637fe640 1 -- 172.21.15.16:0/3689787162 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e700aa480 con 0x7f3e7c0cfd60 2024-04-25T10:11:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.736+0000 7f3e7affd640 1 --2- 172.21.15.16:0/3689787162 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3e6806b4d0 0x7f3e6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.736+0000 7f3e7affd640 1 --2- 172.21.15.16:0/3689787162 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3e6806b4d0 0x7f3e6806d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3e740011e0 tx=0x7f3e74009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.736+0000 7f3e845ba740 1 Processor -- start 2024-04-25T10:11:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.737+0000 7f3e845ba740 1 -- start start 2024-04-25T10:11:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.737+0000 7f3e845ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb5e7e0 0x56461bb36e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.737+0000 7f3e845ba740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb5ebc0 0x56461bb61040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.737+0000 7f3e845ba740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb61580 0x56461bb34de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.737+0000 7f3e845ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56461ba99a10 con 0x56461bb5e7e0 2024-04-25T10:11:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.737+0000 7f3e845ba740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56461ba42b30 con 0x56461bb61580 2024-04-25T10:11:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.737+0000 7f3e845ba740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56461ba95c50 con 0x56461bb5ebc0 2024-04-25T10:11:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.737+0000 7f3e7affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb5e7e0 0x56461bb36e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.737+0000 7f3e7affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb5e7e0 0x56461bb36e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53342/0 (socket says 172.21.15.16:53342) 2024-04-25T10:11:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.737+0000 7f3e7affd640 1 -- 172.21.15.16:0/2855893656 learned_addr learned my addr 172.21.15.16:0/2855893656 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.737+0000 7f3e7b7fe640 1 --2- 172.21.15.16:0/2855893656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb61580 0x56461bb34de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.737+0000 7f3e7a7fc640 1 --2- 172.21.15.16:0/2855893656 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb5ebc0 0x56461bb61040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.738+0000 7f3e7affd640 1 -- 172.21.15.16:0/2855893656 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb5ebc0 msgr2=0x56461bb61040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.738+0000 7f3e7affd640 1 --2- 172.21.15.16:0/2855893656 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb5ebc0 0x56461bb61040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.738+0000 7f3e7affd640 1 -- 172.21.15.16:0/2855893656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb61580 msgr2=0x56461bb34de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.738+0000 7f3e7affd640 1 --2- 172.21.15.16:0/2855893656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb61580 0x56461bb34de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.738+0000 7f3e7affd640 1 -- 172.21.15.16:0/2855893656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56461ba57e10 con 0x56461bb5e7e0 2024-04-25T10:11:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.738+0000 7f3e7a7fc640 1 --2- 172.21.15.16:0/2855893656 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb5ebc0 0x56461bb61040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.738+0000 7f3e7affd640 1 --2- 172.21.15.16:0/2855893656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb5e7e0 0x56461bb36e40 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f3e74005cd0 tx=0x7f3e74010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27986ab670dc666 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.738+0000 7f3e60ff9640 1 -- 172.21.15.16:0/2855893656 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74007560 con 0x56461bb5e7e0 2024-04-25T10:11:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.738+0000 7f3e60ff9640 1 -- 172.21.15.16:0/2855893656 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e74007700 con 0x56461bb5e7e0 2024-04-25T10:11:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.738+0000 7f3e60ff9640 1 -- 172.21.15.16:0/2855893656 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74014bb0 con 0x56461bb5e7e0 2024-04-25T10:11:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.739+0000 7f3e845ba740 1 -- 172.21.15.16:0/2855893656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb5e7e0 msgr2=0x56461bb36e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.739+0000 7f3e845ba740 1 --2- 172.21.15.16:0/2855893656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb5e7e0 0x56461bb36e40 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f3e74005cd0 tx=0x7f3e74010000 comp rx=0 tx=0).stop 2024-04-25T10:11:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.739+0000 7f3e845ba740 1 -- 172.21.15.16:0/2855893656 shutdown_connections 2024-04-25T10:11:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.739+0000 7f3e845ba740 1 --2- 172.21.15.16:0/2855893656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb61580 0x56461bb34de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.739+0000 7f3e845ba740 1 --2- 172.21.15.16:0/2855893656 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb5ebc0 0x56461bb61040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.739+0000 7f3e845ba740 1 --2- 172.21.15.16:0/2855893656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb5e7e0 0x56461bb36e40 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.739+0000 7f3e845ba740 1 -- 172.21.15.16:0/2855893656 >> 172.21.15.16:0/2855893656 conn(0x56461bb33840 msgr2=0x56461bb7c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.739+0000 7f3e845ba740 1 -- 172.21.15.16:0/2855893656 shutdown_connections 2024-04-25T10:11:09.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.739+0000 7f3e845ba740 1 -- 172.21.15.16:0/2855893656 wait complete. 2024-04-25T10:11:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.739+0000 7f3e845ba740 1 -- start start 2024-04-25T10:11:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.739+0000 7f3e845ba740 1 Processor -- start 2024-04-25T10:11:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.740+0000 7f3e845ba740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb5ebc0 0x56461bb639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.740+0000 7f3e845ba740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb61580 0x56461bb4fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.740+0000 7f3e845ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb730b0 0x56461bb5fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.740+0000 7f3e845ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56461ba560c0 con 0x56461bb730b0 2024-04-25T10:11:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.740+0000 7f3e845ba740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56461bb79d40 con 0x56461bb61580 2024-04-25T10:11:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.740+0000 7f3e845ba740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56461bb922c0 con 0x56461bb5ebc0 2024-04-25T10:11:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.740+0000 7f3e845ba740 10 client.0 _ll_register_callbacks cb 0x56461ba49ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.740+0000 7f3e7b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb730b0 0x56461bb5fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.740+0000 7f3e7b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb730b0 0x56461bb5fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53350/0 (socket says 172.21.15.16:53350) 2024-04-25T10:11:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.740+0000 7f3e7b7fe640 1 -- 172.21.15.16:0/3976620507 learned_addr learned my addr 172.21.15.16:0/3976620507 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.740+0000 7f3e7affd640 1 --2- 172.21.15.16:0/3976620507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb5ebc0 0x56461bb639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.740+0000 7f3e7a7fc640 1 --2- 172.21.15.16:0/3976620507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb61580 0x56461bb4fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e7b7fe640 1 -- 172.21.15.16:0/3976620507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb5ebc0 msgr2=0x56461bb639d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e7b7fe640 1 --2- 172.21.15.16:0/3976620507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb5ebc0 0x56461bb639d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e7b7fe640 1 -- 172.21.15.16:0/3976620507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb61580 msgr2=0x56461bb4fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e7b7fe640 1 --2- 172.21.15.16:0/3976620507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb61580 0x56461bb4fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e7b7fe640 1 -- 172.21.15.16:0/3976620507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56461b7ae2d0 con 0x56461bb730b0 2024-04-25T10:11:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e7a7fc640 1 --2- 172.21.15.16:0/3976620507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb61580 0x56461bb4fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e7b7fe640 1 --2- 172.21.15.16:0/3976620507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb730b0 0x56461bb5fdd0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c010610 tx=0x7f3e6c010ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c10010a4d539ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:11:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e59ffb640 1 -- 172.21.15.16:0/3976620507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c003da0 con 0x56461bb730b0 2024-04-25T10:11:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e59ffb640 1 -- 172.21.15.16:0/3976620507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e6c0050d0 con 0x56461bb730b0 2024-04-25T10:11:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e845ba740 1 -- 172.21.15.16:0/3976620507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e74007700 con 0x56461bb730b0 2024-04-25T10:11:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e59ffb640 1 -- 172.21.15.16:0/3976620507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e6c021050 con 0x56461bb730b0 2024-04-25T10:11:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e845ba740 10 client.7500 fetch_fsmap learned FSMap version 51 2024-04-25T10:11:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e845ba740 1 -- 172.21.15.16:0/3976620507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x56461b8786b0 con 0x56461bb730b0 2024-04-25T10:11:09.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e59ffb640 1 -- 172.21.15.16:0/3976620507 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e74007700 con 0x56461bb730b0 2024-04-25T10:11:09.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e59ffb640 1 -- 172.21.15.16:0/3976620507 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3e6c024020 con 0x56461bb730b0 2024-04-25T10:11:09.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e845ba740 10 client.7500 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T10:11:09.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e845ba740 10 client.7500 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:11:09.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e845ba740 20 client.7500 populate_metadata read hostname 'smithi016' 2024-04-25T10:11:09.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e845ba740 1 --2- 172.21.15.16:0/3976620507 >> [v2:172.21.15.186:6835/3092291936,v1:172.21.15.186:6837/3092291936] conn(0x56461bb629a0 0x56461bb62d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e845ba740 4 client.7500 mds_command: new command op to 35852 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:09.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.741+0000 7f3e845ba740 1 -- 172.21.15.16:0/3976620507 --> [v2:172.21.15.186:6835/3092291936,v1:172.21.15.186:6837/3092291936] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56461baf8920 con 0x56461bb629a0 2024-04-25T10:11:09.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.742+0000 7f3e7affd640 1 --2- 172.21.15.16:0/3976620507 >> [v2:172.21.15.186:6835/3092291936,v1:172.21.15.186:6837/3092291936] conn(0x56461bb629a0 0x56461bb62d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.742+0000 7f3e7affd640 1 --2- 172.21.15.16:0/3976620507 >> [v2:172.21.15.186:6835/3092291936,v1:172.21.15.186:6837/3092291936] conn(0x56461bb629a0 0x56461bb62d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e58190c1a4a6cf5 server_cookie=e9ec006ee274af2b in_seq=0 out_seq=0 2024-04-25T10:11:09.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.742+0000 7f3e59ffb640 10 client.7500 ms_handle_connect on v2:172.21.15.186:6835/3092291936 2024-04-25T10:11:09.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.743+0000 7f3e59ffb640 1 -- 172.21.15.16:0/3976620507 <== mds.0 v2:172.21.15.186:6835/3092291936 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56461baf8920 con 0x56461bb629a0 2024-04-25T10:11:09.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.743+0000 7f3e59ffb640 10 client.7500 handle_command_reply: tid=0 2024-04-25T10:11:09.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.743+0000 7f3e845ba740 1 client.7500 shutdown 2024-04-25T10:11:09.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.743+0000 7f3e845ba740 1 -- 172.21.15.16:0/3976620507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb730b0 msgr2=0x56461bb5fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.743+0000 7f3e845ba740 1 --2- 172.21.15.16:0/3976620507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb730b0 0x56461bb5fdd0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c010610 tx=0x7f3e6c010ad0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.743+0000 7f3e845ba740 1 -- 172.21.15.16:0/3976620507 shutdown_connections 2024-04-25T10:11:09.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.743+0000 7f3e845ba740 1 --2- 172.21.15.16:0/3976620507 >> [v2:172.21.15.186:6835/3092291936,v1:172.21.15.186:6837/3092291936] conn(0x56461bb629a0 0x56461bb62d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.743+0000 7f3e845ba740 1 --2- 172.21.15.16:0/3976620507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb730b0 0x56461bb5fdd0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.743+0000 7f3e845ba740 1 --2- 172.21.15.16:0/3976620507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb61580 0x56461bb4fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.743+0000 7f3e59ffb640 0 client.7500 ms_handle_reset on v2:172.21.15.186:6835/3092291936 2024-04-25T10:11:09.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.743+0000 7f3e845ba740 1 --2- 172.21.15.16:0/3976620507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb5ebc0 0x56461bb639d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.743+0000 7f3e845ba740 1 -- 172.21.15.16:0/3976620507 >> 172.21.15.16:0/3976620507 conn(0x56461bb5e7e0 msgr2=0x56461bb36980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.743+0000 7f3e845ba740 1 -- 172.21.15.16:0/3976620507 shutdown_connections 2024-04-25T10:11:09.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.743+0000 7f3e845ba740 1 -- 172.21.15.16:0/3976620507 wait complete. 2024-04-25T10:11:09.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.743+0000 7f3e845ba740 20 client.7500 trim_cache size 0 max 16384 2024-04-25T10:11:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.759+0000 7f3e845ba740 1 Processor -- start 2024-04-25T10:11:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.759+0000 7f3e845ba740 1 -- start start 2024-04-25T10:11:09.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.760+0000 7f3e845ba740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb5ebc0 0x56461bbe3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.760+0000 7f3e845ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb61580 0x56461bb30c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.760+0000 7f3e845ba740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb730b0 0x56461bb31150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.760+0000 7f3e845ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c06d0b0 con 0x56461bb61580 2024-04-25T10:11:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.760+0000 7f3e845ba740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0b1d40 con 0x56461bb5ebc0 2024-04-25T10:11:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.760+0000 7f3e845ba740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0b2340 con 0x56461bb730b0 2024-04-25T10:11:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.760+0000 7f3e7a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb61580 0x56461bb30c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.760+0000 7f3e7a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb61580 0x56461bb30c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53354/0 (socket says 172.21.15.16:53354) 2024-04-25T10:11:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.760+0000 7f3e7a7fc640 1 -- 172.21.15.16:0/2193867311 learned_addr learned my addr 172.21.15.16:0/2193867311 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.760+0000 7f3e7a7fc640 1 -- 172.21.15.16:0/2193867311 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb730b0 msgr2=0x56461bb31150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.760+0000 7f3e7a7fc640 1 --2- 172.21.15.16:0/2193867311 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb730b0 0x56461bb31150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.760+0000 7f3e7a7fc640 1 -- 172.21.15.16:0/2193867311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb5ebc0 msgr2=0x56461bbe3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.760+0000 7f3e7a7fc640 1 --2- 172.21.15.16:0/2193867311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb5ebc0 0x56461bbe3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.760+0000 7f3e7a7fc640 1 -- 172.21.15.16:0/2193867311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56461bb5e410 con 0x56461bb61580 2024-04-25T10:11:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.760+0000 7f3e7a7fc640 1 --2- 172.21.15.16:0/2193867311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb61580 0x56461bb30c10 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3e70078930 tx=0x7f3e700aa040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af15c95fc25ad4e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.760+0000 7f3e5bfff640 1 -- 172.21.15.16:0/2193867311 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70079330 con 0x56461bb61580 2024-04-25T10:11:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.760+0000 7f3e5bfff640 1 -- 172.21.15.16:0/2193867311 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e70078c30 con 0x56461bb61580 2024-04-25T10:11:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.760+0000 7f3e5bfff640 1 -- 172.21.15.16:0/2193867311 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7007cd80 con 0x56461bb61580 2024-04-25T10:11:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.761+0000 7f3e845ba740 1 -- 172.21.15.16:0/2193867311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb61580 msgr2=0x56461bb30c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.761+0000 7f3e845ba740 1 --2- 172.21.15.16:0/2193867311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb61580 0x56461bb30c10 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3e70078930 tx=0x7f3e700aa040 comp rx=0 tx=0).stop 2024-04-25T10:11:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.761+0000 7f3e845ba740 1 -- 172.21.15.16:0/2193867311 shutdown_connections 2024-04-25T10:11:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.761+0000 7f3e845ba740 1 --2- 172.21.15.16:0/2193867311 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb730b0 0x56461bb31150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.761+0000 7f3e845ba740 1 --2- 172.21.15.16:0/2193867311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb61580 0x56461bb30c10 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.761+0000 7f3e845ba740 1 --2- 172.21.15.16:0/2193867311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb5ebc0 0x56461bbe3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.761+0000 7f3e845ba740 1 -- 172.21.15.16:0/2193867311 >> 172.21.15.16:0/2193867311 conn(0x56461bb5e7e0 msgr2=0x56461bb4f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.761+0000 7f3e845ba740 1 -- 172.21.15.16:0/2193867311 shutdown_connections 2024-04-25T10:11:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.761+0000 7f3e845ba740 1 -- 172.21.15.16:0/2193867311 wait complete. 2024-04-25T10:11:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.761+0000 7f3e845ba740 1 -- start start 2024-04-25T10:11:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.761+0000 7f3e845ba740 1 Processor -- start 2024-04-25T10:11:09.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.762+0000 7f3e845ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb5ebc0 0x56461bb30c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.762+0000 7f3e845ba740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb61580 0x56461bb31150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.762+0000 7f3e845ba740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb730b0 0x56461bb62d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.762+0000 7f3e845ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56461bb652c0 con 0x56461bb5ebc0 2024-04-25T10:11:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.762+0000 7f3e845ba740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56461bb64b80 con 0x56461bb730b0 2024-04-25T10:11:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.762+0000 7f3e845ba740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56461bb91940 con 0x56461bb61580 2024-04-25T10:11:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.762+0000 7f3e845ba740 10 client.0 _ll_register_callbacks cb 0x7f3e74007c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.762+0000 7f3e7affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb5ebc0 0x56461bb30c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.762+0000 7f3e7affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb5ebc0 0x56461bb30c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53356/0 (socket says 172.21.15.16:53356) 2024-04-25T10:11:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.762+0000 7f3e7affd640 1 -- 172.21.15.16:0/1772217200 learned_addr learned my addr 172.21.15.16:0/1772217200 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:09.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.762+0000 7f3e7a7fc640 1 --2- 172.21.15.16:0/1772217200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb61580 0x56461bb31150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.762+0000 7f3e7b7fe640 1 --2- 172.21.15.16:0/1772217200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb730b0 0x56461bb62d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.762+0000 7f3e7affd640 1 -- 172.21.15.16:0/1772217200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb61580 msgr2=0x56461bb31150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.762+0000 7f3e7affd640 1 --2- 172.21.15.16:0/1772217200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb61580 0x56461bb31150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.762+0000 7f3e7affd640 1 -- 172.21.15.16:0/1772217200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb730b0 msgr2=0x56461bb62d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.762+0000 7f3e7affd640 1 --2- 172.21.15.16:0/1772217200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb730b0 0x56461bb62d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.763+0000 7f3e7affd640 1 -- 172.21.15.16:0/1772217200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56461bb91de0 con 0x56461bb5ebc0 2024-04-25T10:11:09.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.763+0000 7f3e7b7fe640 1 --2- 172.21.15.16:0/1772217200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb730b0 0x56461bb62d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:09.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.763+0000 7f3e7affd640 1 --2- 172.21.15.16:0/1772217200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb5ebc0 0x56461bb30c10 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3e74005c70 tx=0x7f3e74001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02e63416a369b77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.763+0000 7f3e59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:11:09.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.763+0000 7f3e59ffb640 1 -- 172.21.15.16:0/1772217200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e740149b0 con 0x56461bb5ebc0 2024-04-25T10:11:09.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.763+0000 7f3e59ffb640 1 -- 172.21.15.16:0/1772217200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e74014b50 con 0x56461bb5ebc0 2024-04-25T10:11:09.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.763+0000 7f3e845ba740 1 -- 172.21.15.16:0/1772217200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e70078c30 con 0x56461bb5ebc0 2024-04-25T10:11:09.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.763+0000 7f3e59ffb640 1 -- 172.21.15.16:0/1772217200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e7403b050 con 0x56461bb5ebc0 2024-04-25T10:11:09.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.763+0000 7f3e845ba740 10 client.7506 fetch_fsmap learned FSMap version 51 2024-04-25T10:11:09.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.763+0000 7f3e845ba740 1 -- 172.21.15.16:0/1772217200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x56461bb63a50 con 0x56461bb5ebc0 2024-04-25T10:11:09.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.763+0000 7f3e59ffb640 1 -- 172.21.15.16:0/1772217200 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e70078c30 con 0x56461bb5ebc0 2024-04-25T10:11:09.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.763+0000 7f3e59ffb640 1 -- 172.21.15.16:0/1772217200 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3e7403e020 con 0x56461bb5ebc0 2024-04-25T10:11:09.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.763+0000 7f3e845ba740 10 client.7506 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T10:11:09.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.763+0000 7f3e845ba740 10 client.7506 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:11:09.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.763+0000 7f3e845ba740 20 client.7506 populate_metadata read hostname 'smithi016' 2024-04-25T10:11:09.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.763+0000 7f3e845ba740 1 --2- 172.21.15.16:0/1772217200 >> [v2:172.21.15.186:6835/3092291936,v1:172.21.15.186:6837/3092291936] conn(0x56461bb63310 0x56461bb36df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.763+0000 7f3e845ba740 4 client.7506 mds_command: new command op to 35852 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:11:09.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.763+0000 7f3e845ba740 1 -- 172.21.15.16:0/1772217200 --> [v2:172.21.15.186:6835/3092291936,v1:172.21.15.186:6837/3092291936] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56461bb60a10 con 0x56461bb63310 2024-04-25T10:11:09.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.764+0000 7f3e7a7fc640 1 --2- 172.21.15.16:0/1772217200 >> [v2:172.21.15.186:6835/3092291936,v1:172.21.15.186:6837/3092291936] conn(0x56461bb63310 0x56461bb36df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.764+0000 7f3e7a7fc640 1 --2- 172.21.15.16:0/1772217200 >> [v2:172.21.15.186:6835/3092291936,v1:172.21.15.186:6837/3092291936] conn(0x56461bb63310 0x56461bb36df0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=572f350f5bcda502 server_cookie=43183e9acd0c7394 in_seq=0 out_seq=0 2024-04-25T10:11:09.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.764+0000 7f3e59ffb640 10 client.7506 ms_handle_connect on v2:172.21.15.186:6835/3092291936 2024-04-25T10:11:09.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.764+0000 7f3e59ffb640 1 -- 172.21.15.16:0/1772217200 <== mds.0 v2:172.21.15.186:6835/3092291936 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56461bb60a10 con 0x56461bb63310 2024-04-25T10:11:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.764+0000 7f3e59ffb640 10 client.7506 handle_command_reply: tid=0 2024-04-25T10:11:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.764+0000 7f3e845ba740 1 client.7506 shutdown 2024-04-25T10:11:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.764+0000 7f3e845ba740 1 -- 172.21.15.16:0/1772217200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb5ebc0 msgr2=0x56461bb30c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.764+0000 7f3e845ba740 1 --2- 172.21.15.16:0/1772217200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb5ebc0 0x56461bb30c10 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3e74005c70 tx=0x7f3e74001150 comp rx=0 tx=0).stop 2024-04-25T10:11:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.764+0000 7f3e845ba740 1 -- 172.21.15.16:0/1772217200 shutdown_connections 2024-04-25T10:11:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.764+0000 7f3e845ba740 1 --2- 172.21.15.16:0/1772217200 >> [v2:172.21.15.186:6835/3092291936,v1:172.21.15.186:6837/3092291936] conn(0x56461bb63310 0x56461bb36df0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.764+0000 7f3e845ba740 1 --2- 172.21.15.16:0/1772217200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461bb730b0 0x56461bb62d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.764+0000 7f3e845ba740 1 --2- 172.21.15.16:0/1772217200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56461bb61580 0x56461bb31150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.764+0000 7f3e845ba740 1 --2- 172.21.15.16:0/1772217200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56461bb5ebc0 0x56461bb30c10 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.764+0000 7f3e59ffb640 0 client.7506 ms_handle_reset on v2:172.21.15.186:6835/3092291936 2024-04-25T10:11:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.764+0000 7f3e845ba740 1 -- 172.21.15.16:0/1772217200 >> 172.21.15.16:0/1772217200 conn(0x56461bb5e7e0 msgr2=0x56461bb7f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.764+0000 7f3e845ba740 1 -- 172.21.15.16:0/1772217200 shutdown_connections 2024-04-25T10:11:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.764+0000 7f3e845ba740 1 -- 172.21.15.16:0/1772217200 wait complete. 2024-04-25T10:11:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.764+0000 7f3e845ba740 20 client.7506 trim_cache size 0 max 16384 2024-04-25T10:11:09.777 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:11:09.777 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "34bc5eca-3a21-42a6-a26f-07da0905e8bc", 2024-04-25T10:11:09.777 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-04-25T10:11:09.777 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 35852, 2024-04-25T10:11:09.777 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-04-25T10:11:09.777 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-04-25T10:11:09.777 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-04-25T10:11:09.777 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 9.1057148869999995, 2024-04-25T10:11:09.777 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 51, 2024-04-25T10:11:09.777 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 86, 2024-04-25T10:11:09.778 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T10:11:09.778 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 14.118383207000001 2024-04-25T10:11:09.778 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:11:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.765+0000 7f3e817d5640 1 -- 172.21.15.16:0/3689787162 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3e6806b4d0 msgr2=0x7f3e6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.765+0000 7f3e817d5640 1 --2- 172.21.15.16:0/3689787162 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3e6806b4d0 0x7f3e6806d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3e740011e0 tx=0x7f3e74009040 comp rx=0 tx=0).stop 2024-04-25T10:11:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.765+0000 7f3e817d5640 1 -- 172.21.15.16:0/3689787162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e7c0cfd60 msgr2=0x7f3e7c15b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.765+0000 7f3e817d5640 1 --2- 172.21.15.16:0/3689787162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e7c0cfd60 0x7f3e7c15b860 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3e70035040 tx=0x7f3e700046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.765+0000 7f3e817d5640 1 -- 172.21.15.16:0/3689787162 shutdown_connections 2024-04-25T10:11:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.765+0000 7f3e817d5640 1 --2- 172.21.15.16:0/3689787162 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3e6806b4d0 0x7f3e6806d990 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.765+0000 7f3e817d5640 1 --2- 172.21.15.16:0/3689787162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c15ef60 0x7f3e7c15bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.765+0000 7f3e817d5640 1 --2- 172.21.15.16:0/3689787162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e7c0cfd60 0x7f3e7c15b860 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.765+0000 7f3e817d5640 1 --2- 172.21.15.16:0/3689787162 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e7c0b0740 0x7f3e7c0afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.765+0000 7f3e817d5640 1 -- 172.21.15.16:0/3689787162 >> 172.21.15.16:0/3689787162 conn(0x7f3e7c06f4c0 msgr2=0x7f3e7c15e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.765+0000 7f3e817d5640 1 -- 172.21.15.16:0/3689787162 shutdown_connections 2024-04-25T10:11:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.765+0000 7f3e817d5640 1 -- 172.21.15.16:0/3689787162 wait complete. 2024-04-25T10:11:09.781 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:11:09.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.876+0000 7fb82cf70640 1 Processor -- start 2024-04-25T10:11:09.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.876+0000 7fb82cf70640 1 -- start start 2024-04-25T10:11:09.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.877+0000 7fb82cf70640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb82815bf00 0x7fb8280cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.877+0000 7fb82cf70640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb8280cc310 0x7fb82815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.877+0000 7fb82cf70640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8280ce6c0 con 0x7fb82815c820 2024-04-25T10:11:09.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.877+0000 7fb82cf70640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8281455e0 con 0x7fb8280cc310 2024-04-25T10:11:09.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.877+0000 7fb82cf70640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82806d450 con 0x7fb82815bf00 2024-04-25T10:11:09.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.878+0000 7fb82659c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb82815bf00 0x7fb8280cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.878+0000 7fb825d9b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb8280cc310 0x7fb82815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.878+0000 7fb82659c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb82815bf00 0x7fb8280cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:35900/0 (socket says 172.21.15.16:35900) 2024-04-25T10:11:09.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.878+0000 7fb82659c640 1 -- 172.21.15.16:0/3386310219 learned_addr learned my addr 172.21.15.16:0/3386310219 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.878+0000 7fb82659c640 1 -- 172.21.15.16:0/3386310219 >> v2:172.21.15.186:3300/0 conn(0x7fb8280cc310 msgr2=0x7fb82815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.878+0000 7fb82659c640 1 --2- 172.21.15.16:0/3386310219 >> v2:172.21.15.186:3300/0 conn(0x7fb8280cc310 0x7fb82815c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.878+0000 7fb82659c640 1 -- 172.21.15.16:0/3386310219 >> v1:172.21.15.186:6789/0 conn(0x7fb82815c820 legacy=0x7fb82806d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.878+0000 7fb82659c640 1 -- 172.21.15.16:0/3386310219 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8280cc7b0 con 0x7fb82815bf00 2024-04-25T10:11:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.879+0000 7fb82659c640 1 --2- 172.21.15.16:0/3386310219 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb82815bf00 0x7fb8280cbdd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb818009e50 tx=0x7fb81802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=510278312e392adb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.879+0000 7fb82559a640 1 -- 172.21.15.16:0/3386310219 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818036070 con 0x7fb82815bf00 2024-04-25T10:11:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.879+0000 7fb82559a640 1 -- 172.21.15.16:0/3386310219 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb818031600 con 0x7fb82815bf00 2024-04-25T10:11:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.879+0000 7fb82559a640 1 -- 172.21.15.16:0/3386310219 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180319d0 con 0x7fb82815bf00 2024-04-25T10:11:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.879+0000 7fb82cf70640 1 -- 172.21.15.16:0/3386310219 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb82815bf00 msgr2=0x7fb8280cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.879+0000 7fb82cf70640 1 --2- 172.21.15.16:0/3386310219 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb82815bf00 0x7fb8280cbdd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb818009e50 tx=0x7fb81802a710 comp rx=0 tx=0).stop 2024-04-25T10:11:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.880+0000 7fb82cf70640 1 -- 172.21.15.16:0/3386310219 shutdown_connections 2024-04-25T10:11:09.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.880+0000 7fb82cf70640 1 --2- 172.21.15.16:0/3386310219 >> v2:172.21.15.186:3300/0 conn(0x7fb8280cc310 0x7fb82815c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.880+0000 7fb82cf70640 1 --2- 172.21.15.16:0/3386310219 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb82815bf00 0x7fb8280cbdd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.880+0000 7fb82cf70640 1 -- 172.21.15.16:0/3386310219 >> 172.21.15.16:0/3386310219 conn(0x7fb82806aba0 msgr2=0x7fb82806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.880+0000 7fb82cf70640 1 -- 172.21.15.16:0/3386310219 shutdown_connections 2024-04-25T10:11:09.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.880+0000 7fb82cf70640 1 -- 172.21.15.16:0/3386310219 wait complete. 2024-04-25T10:11:09.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.881+0000 7fb82cf70640 1 Processor -- start 2024-04-25T10:11:09.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.881+0000 7fb82cf70640 1 -- start start 2024-04-25T10:11:09.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.882+0000 7fb82cf70640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8280cc310 0x7fb8280caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.882+0000 7fb82cf70640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb82815bf00 0x7fb8281f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.882+0000 7fb82cf70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb82815c820 0x7fb8281404e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.882+0000 7fb82cf70640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8280cd050 con 0x7fb82815c820 2024-04-25T10:11:09.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.882+0000 7fb82cf70640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8280ccd70 con 0x7fb8280cc310 2024-04-25T10:11:09.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.882+0000 7fb82cf70640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82815cca0 con 0x7fb82815bf00 2024-04-25T10:11:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.882+0000 7fb826d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb82815c820 0x7fb8281404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.882+0000 7fb826d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb82815c820 0x7fb8281404e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53358/0 (socket says 172.21.15.16:53358) 2024-04-25T10:11:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.882+0000 7fb826d9d640 1 -- 172.21.15.16:0/1649632494 learned_addr learned my addr 172.21.15.16:0/1649632494 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.882+0000 7fb82659c640 1 --2- 172.21.15.16:0/1649632494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8280cc310 0x7fb8280caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.882+0000 7fb825d9b640 1 --2- 172.21.15.16:0/1649632494 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb82815bf00 0x7fb8281f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.882+0000 7fb826d9d640 1 -- 172.21.15.16:0/1649632494 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb82815bf00 msgr2=0x7fb8281f7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.882+0000 7fb826d9d640 1 --2- 172.21.15.16:0/1649632494 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb82815bf00 0x7fb8281f7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.882+0000 7fb826d9d640 1 -- 172.21.15.16:0/1649632494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8280cc310 msgr2=0x7fb8280caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.882+0000 7fb826d9d640 1 --2- 172.21.15.16:0/1649632494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8280cc310 0x7fb8280caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.882+0000 7fb826d9d640 1 -- 172.21.15.16:0/1649632494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8280b3240 con 0x7fb82815c820 2024-04-25T10:11:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.882+0000 7fb826d9d640 1 --2- 172.21.15.16:0/1649632494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb82815c820 0x7fb8281404e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb82000c390 tx=0x7fb82000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.882+0000 7fb81effd640 1 -- 172.21.15.16:0/1649632494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820017070 con 0x7fb82815c820 2024-04-25T10:11:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.883+0000 7fb81effd640 1 -- 172.21.15.16:0/1649632494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8200052d0 con 0x7fb82815c820 2024-04-25T10:11:09.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.883+0000 7fb82cf70640 1 -- 172.21.15.16:0/1649632494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8280b3520 con 0x7fb82815c820 2024-04-25T10:11:09.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.883+0000 7fb81effd640 1 -- 172.21.15.16:0/1649632494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820003810 con 0x7fb82815c820 2024-04-25T10:11:09.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.883+0000 7fb82cf70640 1 -- 172.21.15.16:0/1649632494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8280c48f0 con 0x7fb82815c820 2024-04-25T10:11:09.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.884+0000 7fb82cf70640 1 -- 172.21.15.16:0/1649632494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82806bb30 con 0x7fb82815c820 2024-04-25T10:11:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.884+0000 7fb81effd640 1 -- 172.21.15.16:0/1649632494 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb82001a070 con 0x7fb82815c820 2024-04-25T10:11:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.885+0000 7fb81effd640 1 --2- 172.21.15.16:0/1649632494 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb80806b4d0 0x7fb80806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.885+0000 7fb81effd640 1 -- 172.21.15.16:0/1649632494 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb820089390 con 0x7fb82815c820 2024-04-25T10:11:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.885+0000 7fb82659c640 1 --2- 172.21.15.16:0/1649632494 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb80806b4d0 0x7fb80806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.886+0000 7fb82659c640 1 --2- 172.21.15.16:0/1649632494 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb80806b4d0 0x7fb80806d990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb818033000 tx=0x7fb818009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:09.889+0000 7fb81effd640 1 -- 172.21.15.16:0/1649632494 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb8280ce6c0 con 0x7fb82815c820 2024-04-25T10:11:10.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:10.105+0000 7fb82cf70640 1 -- 172.21.15.16:0/1649632494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb8280749b0 con 0x7fb82815c820 2024-04-25T10:11:10.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:10.105+0000 7fb81effd640 1 -- 172.21.15.16:0/1649632494 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb820057c30 con 0x7fb82815c820 2024-04-25T10:11:10.111 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:11:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:10.107+0000 7fb82cf70640 1 -- 172.21.15.16:0/1649632494 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb80806b4d0 msgr2=0x7fb80806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:10.107+0000 7fb82cf70640 1 --2- 172.21.15.16:0/1649632494 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb80806b4d0 0x7fb80806d990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb818033000 tx=0x7fb818009000 comp rx=0 tx=0).stop 2024-04-25T10:11:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:10.107+0000 7fb82cf70640 1 -- 172.21.15.16:0/1649632494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb82815c820 msgr2=0x7fb8281404e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:10.107+0000 7fb82cf70640 1 --2- 172.21.15.16:0/1649632494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb82815c820 0x7fb8281404e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb82000c390 tx=0x7fb82000c850 comp rx=0 tx=0).stop 2024-04-25T10:11:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:10.107+0000 7fb82cf70640 1 -- 172.21.15.16:0/1649632494 shutdown_connections 2024-04-25T10:11:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:10.107+0000 7fb82cf70640 1 --2- 172.21.15.16:0/1649632494 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb80806b4d0 0x7fb80806d990 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:10.107+0000 7fb82cf70640 1 --2- 172.21.15.16:0/1649632494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb82815c820 0x7fb8281404e0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:10.107+0000 7fb82cf70640 1 --2- 172.21.15.16:0/1649632494 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb82815bf00 0x7fb8281f7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:10.107+0000 7fb82cf70640 1 --2- 172.21.15.16:0/1649632494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8280cc310 0x7fb8280caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:10.107+0000 7fb82cf70640 1 -- 172.21.15.16:0/1649632494 >> 172.21.15.16:0/1649632494 conn(0x7fb82806aba0 msgr2=0x7fb828071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:10.107+0000 7fb82cf70640 1 -- 172.21.15.16:0/1649632494 shutdown_connections 2024-04-25T10:11:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:10.107+0000 7fb82cf70640 1 -- 172.21.15.16:0/1649632494 wait complete. 2024-04-25T10:11:10.123 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:11:10.123 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:11:10.123 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-04-25T10:11:10.123 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:11:10.124 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:10.124 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:11:10.124 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:11:10.124 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:11:10.124 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:10.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:10.124 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-04-25T10:11:10.140 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:11:10.140 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:11:10.140 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:11:10.140 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:10.140 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:11:10.140 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:10.140 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:10.140 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:10.140 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-04-25T10:11:10.140 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:10.140 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:10.140 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-04-25T10:11:10.141 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:11:10.141 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:10.141 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-04-25T10:11:10.141 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:11:10.141 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42290sec preferred_lft 42290sec 2024-04-25T10:11:10.141 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-25T10:11:10.141 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:10.141 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:10.141 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:10.141 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-04-25T10:11:10.141 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:11:10.141 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:10.141 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:11:10.141 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:10.141 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f06b:fdff:fed4:a335/64 scope link 2024-04-25T10:11:10.142 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:10.142 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:10.142 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:10.142 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::90ad:4fff:fe91:3c33/64 scope link 2024-04-25T10:11:10.142 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:10.142 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:10.142 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether be:14:4a:3c:16:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:10.142 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::bc14:4aff:fe3c:1605/64 scope link 2024-04-25T10:11:10.142 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:10.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:10.143 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:11:10.203 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:11:10.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:10.204 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-04-25T10:11:10.261 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-04-25T10:11:10.261 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:11:10.262 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:11:10.262 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:11:10.262 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:11:10.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:10.262 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:11:10.262 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:11:10.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:11:10.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:11:10.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:11:10.263 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:11:10.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:10.347 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:11:10.362 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:11:10.362 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:11:10.362 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:10.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:10.363 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-04-25T10:11:10.428 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:10.428 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:10.483 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:10.484 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:10.484 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:10.539 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:11:10.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:10.539 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:10.621 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:10.621 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:10.621 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:11:10.630 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:10.630 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:10.630 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:11:10.630 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:11:10.630 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:11:10.636 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:10.636 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:10.636 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:11:10.636 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:11:10.636 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:11:10.636 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:11:10.636 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:11:10.636 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:11:10.637 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:11:10.637 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:11:10.637 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:11:10.637 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:11:10.637 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:11:10.638 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:11:10.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:10.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:10.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:10.706 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:11:10.769 INFO:teuthology.orchestra.run.smithi016.stdout:client6682 2024-04-25T10:11:10.769 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:11:10.769 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:10.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:10.825 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-04-25T10:11:10.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:10.886 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-04-25T10:11:10.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:10.943 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-25T10:11:11.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:11.000 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-04-25T10:11:11.059 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:11:11.059 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:11:11.060 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:11:11.060 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:11:11.060 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-04-25T10:11:11.060 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:11:11.166 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627776 2024-04-25T10:11:11.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:11:11.175 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:11:11.175 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:11.230 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:11:11.231 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:11.286 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:11.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.421+0000 7f93cf81f640 1 Processor -- start 2024-04-25T10:11:11.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.421+0000 7f93cf81f640 1 -- start start 2024-04-25T10:11:11.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.422+0000 7f93cf81f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f93c806efc0 0x7f93c80c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.422+0000 7f93cf81f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c80719f0 con 0x7f93c81476d0 2024-04-25T10:11:11.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.422+0000 7f93cf81f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c8071cd0 con 0x7f93c80cd060 2024-04-25T10:11:11.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.422+0000 7f93cf81f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c8071fb0 con 0x7f93c806efc0 2024-04-25T10:11:11.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.422+0000 7f93cd5bb640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f93c81476d0 0x7f93c80ca110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55520/0 (socket says 172.21.15.16:55520) 2024-04-25T10:11:11.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.422+0000 7f93cd5bb640 1 -- 172.21.15.16:0/828267795 learned_addr learned my addr 172.21.15.16:0/828267795 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:11.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93cddbc640 1 --2- 172.21.15.16:0/828267795 >> v2:172.21.15.186:3300/0 conn(0x7f93c806efc0 0x7f93c80c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93c7fff640 1 -- 172.21.15.16:0/828267795 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198368042 0 0) 0x7f93c80719f0 con 0x7f93c81476d0 2024-04-25T10:11:11.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93c7fff640 1 -- 172.21.15.16:0/828267795 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93b0003520 con 0x7f93c81476d0 2024-04-25T10:11:11.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93c7fff640 1 -- 172.21.15.16:0/828267795 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2157876806 0 0) 0x7f93b0003520 con 0x7f93c81476d0 2024-04-25T10:11:11.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93c7fff640 1 -- 172.21.15.16:0/828267795 >> v2:172.21.15.186:3300/0 conn(0x7f93c806efc0 msgr2=0x7f93c80c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93c7fff640 1 --2- 172.21.15.16:0/828267795 >> v2:172.21.15.186:3300/0 conn(0x7f93c806efc0 0x7f93c80c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93c7fff640 1 -- 172.21.15.16:0/828267795 >> v1:172.21.15.186:6789/0 conn(0x7f93c80cd060 legacy=0x7f93c806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93c7fff640 1 -- 172.21.15.16:0/828267795 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c8072290 con 0x7f93c81476d0 2024-04-25T10:11:11.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93c7fff640 1 -- 172.21.15.16:0/828267795 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f93c0001670 con 0x7f93c81476d0 2024-04-25T10:11:11.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93c7fff640 1 -- 172.21.15.16:0/828267795 >> v1:172.21.15.16:6789/0 conn(0x7f93c81476d0 legacy=0x7f93c80ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93c7fff640 1 --2- 172.21.15.16:0/828267795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93b00039b0 0x7f93b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93c7fff640 1 --2- 172.21.15.16:0/828267795 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93b00043b0 0x7f93b0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93c7fff640 1 --2- 172.21.15.16:0/828267795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93b0005080 0x7f93b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93c7fff640 1 -- 172.21.15.16:0/828267795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c8071fb0 con 0x7f93b0005080 2024-04-25T10:11:11.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93c7fff640 1 -- 172.21.15.16:0/828267795 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b000bf30 con 0x7f93b00039b0 2024-04-25T10:11:11.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93c7fff640 1 -- 172.21.15.16:0/828267795 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b000c130 con 0x7f93b00043b0 2024-04-25T10:11:11.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93cddbc640 1 --2- 172.21.15.16:0/828267795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93b0005080 0x7f93b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93ccdba640 1 --2- 172.21.15.16:0/828267795 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93b00043b0 0x7f93b0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93cd5bb640 1 --2- 172.21.15.16:0/828267795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93b00039b0 0x7f93b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93cddbc640 1 -- 172.21.15.16:0/828267795 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93b00043b0 msgr2=0x7f93b0004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93cddbc640 1 --2- 172.21.15.16:0/828267795 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93b00043b0 0x7f93b0004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93cddbc640 1 -- 172.21.15.16:0/828267795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93b00039b0 msgr2=0x7f93b0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93cddbc640 1 --2- 172.21.15.16:0/828267795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93b00039b0 0x7f93b0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.423+0000 7f93cddbc640 1 -- 172.21.15.16:0/828267795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b0003520 con 0x7f93b0005080 2024-04-25T10:11:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.424+0000 7f93cddbc640 1 -- 172.21.15.16:0/828267795 reap_dead start 2024-04-25T10:11:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.424+0000 7f93cddbc640 1 --2- 172.21.15.16:0/828267795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93b0005080 0x7f93b000b9c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f93c8071410 tx=0x7f93b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403b0000c20f8a20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.424+0000 7f93c7fff640 1 -- 172.21.15.16:0/828267795 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b8035020 con 0x7f93b0005080 2024-04-25T10:11:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.424+0000 7f93c7fff640 1 -- 172.21.15.16:0/828267795 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b8003b90 con 0x7f93b0005080 2024-04-25T10:11:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.424+0000 7f93c7fff640 1 -- 172.21.15.16:0/828267795 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b8009c10 con 0x7f93b0005080 2024-04-25T10:11:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.425+0000 7f93cf81f640 1 -- 172.21.15.16:0/828267795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93b0005080 msgr2=0x7f93b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.425+0000 7f93cf81f640 1 --2- 172.21.15.16:0/828267795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93b0005080 0x7f93b000b9c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f93c8071410 tx=0x7f93b0005830 comp rx=0 tx=0).stop 2024-04-25T10:11:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.425+0000 7f93cf81f640 1 -- 172.21.15.16:0/828267795 shutdown_connections 2024-04-25T10:11:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.425+0000 7f93cf81f640 1 --2- 172.21.15.16:0/828267795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93b0005080 0x7f93b000b9c0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.425+0000 7f93cf81f640 1 -- 172.21.15.16:0/828267795 >> 172.21.15.16:0/828267795 conn(0x7f93c80cc540 msgr2=0x7f93c80cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.425+0000 7f93cf81f640 1 -- 172.21.15.16:0/828267795 shutdown_connections 2024-04-25T10:11:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.425+0000 7f93cf81f640 1 -- 172.21.15.16:0/828267795 wait complete. 2024-04-25T10:11:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.426+0000 7f93cf81f640 1 Processor -- start 2024-04-25T10:11:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.426+0000 7f93cf81f640 1 -- start start 2024-04-25T10:11:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93cf81f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93b0005080 0x7f93c815f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93cf81f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c81f71e0 0x7f93c815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93cf81f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93c80ac660 0x7f93c80cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93cf81f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c806d810 con 0x7f93c81f71e0 2024-04-25T10:11:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93cf81f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c806d690 con 0x7f93c80ac660 2024-04-25T10:11:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93cf81f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c806d990 con 0x7f93b0005080 2024-04-25T10:11:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93ccdba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c81f71e0 0x7f93c815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93ccdba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c81f71e0 0x7f93c815d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53372/0 (socket says 172.21.15.16:53372) 2024-04-25T10:11:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93ccdba640 1 -- 172.21.15.16:0/1130156693 learned_addr learned my addr 172.21.15.16:0/1130156693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93cddbc640 1 --2- 172.21.15.16:0/1130156693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93c80ac660 0x7f93c80cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93ccdba640 1 -- 172.21.15.16:0/1130156693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93b0005080 msgr2=0x7f93c815f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93cd5bb640 1 --2- 172.21.15.16:0/1130156693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93b0005080 0x7f93c815f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93ccdba640 1 --2- 172.21.15.16:0/1130156693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93b0005080 0x7f93c815f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93ccdba640 1 -- 172.21.15.16:0/1130156693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93c80ac660 msgr2=0x7f93c80cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93ccdba640 1 --2- 172.21.15.16:0/1130156693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93c80ac660 0x7f93c80cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93ccdba640 1 -- 172.21.15.16:0/1130156693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c815f940 con 0x7f93c81f71e0 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93cddbc640 1 --2- 172.21.15.16:0/1130156693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93c80ac660 0x7f93c80cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93cd5bb640 1 --2- 172.21.15.16:0/1130156693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93b0005080 0x7f93c815f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93ccdba640 1 --2- 172.21.15.16:0/1130156693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c81f71e0 0x7f93c815d4a0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00e580 tx=0x7f93bc00ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.427+0000 7f93c5ffb640 1 -- 172.21.15.16:0/1130156693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc0052e0 con 0x7f93c81f71e0 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.428+0000 7f93c5ffb640 1 -- 172.21.15.16:0/1130156693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93bc018070 con 0x7f93c81f71e0 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.428+0000 7f93cf81f640 1 -- 172.21.15.16:0/1130156693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c815fc20 con 0x7f93c81f71e0 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.428+0000 7f93c5ffb640 1 -- 172.21.15.16:0/1130156693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc003830 con 0x7f93c81f71e0 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.428+0000 7f93cf81f640 1 -- 172.21.15.16:0/1130156693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c81601e0 con 0x7f93c81f71e0 2024-04-25T10:11:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.429+0000 7f93cf81f640 1 -- 172.21.15.16:0/1130156693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c80749b0 con 0x7f93c81f71e0 2024-04-25T10:11:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.429+0000 7f93c5ffb640 1 -- 172.21.15.16:0/1130156693 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f93bc0039d0 con 0x7f93c81f71e0 2024-04-25T10:11:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.429+0000 7f93c5ffb640 1 --2- 172.21.15.16:0/1130156693 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f93b406b4d0 0x7f93b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.430+0000 7f93c5ffb640 1 -- 172.21.15.16:0/1130156693 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93bc089ff0 con 0x7f93c81f71e0 2024-04-25T10:11:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.430+0000 7f93cd5bb640 1 --2- 172.21.15.16:0/1130156693 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f93b406b4d0 0x7f93b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.430+0000 7f93cd5bb640 1 --2- 172.21.15.16:0/1130156693 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f93b406b4d0 0x7f93b406d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f93c00057f0 tx=0x7f93c0000ed0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.434+0000 7f93c5ffb640 1 -- 172.21.15.16:0/1130156693 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f93bc058f00 con 0x7f93c81f71e0 2024-04-25T10:11:11.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.614+0000 7f93cf81f640 1 -- 172.21.15.16:0/1130156693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93c80ca290 con 0x7f93c81f71e0 2024-04-25T10:11:11.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.615+0000 7f93c5ffb640 1 -- 172.21.15.16:0/1130156693 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16672 (secure 0 0 0) 0x7f93bc058800 con 0x7f93c81f71e0 2024-04-25T10:11:11.620 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:11.621 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:11:01.657760+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:58.299405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:58.994953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:11.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.617+0000 7f93cf81f640 1 -- 172.21.15.16:0/1130156693 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f93b406b4d0 msgr2=0x7f93b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.617+0000 7f93cf81f640 1 --2- 172.21.15.16:0/1130156693 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f93b406b4d0 0x7f93b406d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f93c00057f0 tx=0x7f93c0000ed0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.617+0000 7f93cf81f640 1 -- 172.21.15.16:0/1130156693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c81f71e0 msgr2=0x7f93c815d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.617+0000 7f93cf81f640 1 --2- 172.21.15.16:0/1130156693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c81f71e0 0x7f93c815d4a0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00e580 tx=0x7f93bc00ea40 comp rx=0 tx=0).stop 2024-04-25T10:11:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.617+0000 7f93cf81f640 1 -- 172.21.15.16:0/1130156693 shutdown_connections 2024-04-25T10:11:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.617+0000 7f93cf81f640 1 --2- 172.21.15.16:0/1130156693 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f93b406b4d0 0x7f93b406d990 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.617+0000 7f93cf81f640 1 --2- 172.21.15.16:0/1130156693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93c80ac660 0x7f93c80cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.617+0000 7f93cf81f640 1 --2- 172.21.15.16:0/1130156693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c81f71e0 0x7f93c815d4a0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.617+0000 7f93cf81f640 1 --2- 172.21.15.16:0/1130156693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93b0005080 0x7f93c815f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.617+0000 7f93cf81f640 1 -- 172.21.15.16:0/1130156693 >> 172.21.15.16:0/1130156693 conn(0x7f93c80cc540 msgr2=0x7f93c80b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.617+0000 7f93cf81f640 1 -- 172.21.15.16:0/1130156693 shutdown_connections 2024-04-25T10:11:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.617+0000 7f93cf81f640 1 -- 172.21.15.16:0/1130156693 wait complete. 2024-04-25T10:11:11.633 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:10:58.299405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:10:58.994953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:11.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.733+0000 7fb1daea1640 1 Processor -- start 2024-04-25T10:11:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.733+0000 7fb1daea1640 1 -- start start 2024-04-25T10:11:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.734+0000 7fb1daea1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1d415dca0 0x7fb1d40ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.734+0000 7fb1daea1640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb1d40d1130 0x7fb1d40b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.734+0000 7fb1daea1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d40d4090 con 0x7fb1d40d3b90 2024-04-25T10:11:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.734+0000 7fb1daea1640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d40b6b20 con 0x7fb1d40d1130 2024-04-25T10:11:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.734+0000 7fb1daea1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d40d4370 con 0x7fb1d415dca0 2024-04-25T10:11:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.734+0000 7fb1d3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb1d40d3b90 0x7fb1d40d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55536/0 (socket says 172.21.15.16:55536) 2024-04-25T10:11:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.734+0000 7fb1d3fff640 1 -- 172.21.15.16:0/566156150 learned_addr learned my addr 172.21.15.16:0/566156150 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.734+0000 7fb1d8c3d640 1 --2- 172.21.15.16:0/566156150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1d415dca0 0x7fb1d40ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.734+0000 7fb1d943e640 1 --2- 172.21.15.16:0/566156150 >> v2:172.21.15.186:3300/0 conn(0x7fb1d40d1130 0x7fb1d40b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.734+0000 7fb1d37fe640 1 -- 172.21.15.16:0/566156150 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471197449 0 0) 0x7fb1d40d4090 con 0x7fb1d40d3b90 2024-04-25T10:11:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.734+0000 7fb1d37fe640 1 -- 172.21.15.16:0/566156150 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1c4003520 con 0x7fb1d40d3b90 2024-04-25T10:11:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.734+0000 7fb1d37fe640 1 -- 172.21.15.16:0/566156150 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 205241949 0 0) 0x7fb1c4003520 con 0x7fb1d40d3b90 2024-04-25T10:11:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.734+0000 7fb1d37fe640 1 -- 172.21.15.16:0/566156150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1d415dca0 msgr2=0x7fb1d40ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.734+0000 7fb1d37fe640 1 --2- 172.21.15.16:0/566156150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1d415dca0 0x7fb1d40ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.734+0000 7fb1d37fe640 1 -- 172.21.15.16:0/566156150 >> v2:172.21.15.186:3300/0 conn(0x7fb1d40d1130 msgr2=0x7fb1d40b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.734+0000 7fb1d37fe640 1 --2- 172.21.15.16:0/566156150 >> v2:172.21.15.186:3300/0 conn(0x7fb1d40d1130 0x7fb1d40b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.734+0000 7fb1d37fe640 1 -- 172.21.15.16:0/566156150 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d40d4650 con 0x7fb1d40d3b90 2024-04-25T10:11:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.734+0000 7fb1d37fe640 1 -- 172.21.15.16:0/566156150 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fb1c0001670 con 0x7fb1d40d3b90 2024-04-25T10:11:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.735+0000 7fb1d37fe640 1 -- 172.21.15.16:0/566156150 >> v1:172.21.15.16:6789/0 conn(0x7fb1d40d3b90 legacy=0x7fb1d40d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.735+0000 7fb1d37fe640 1 --2- 172.21.15.16:0/566156150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1c40039b0 0x7fb1c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.735+0000 7fb1d37fe640 1 --2- 172.21.15.16:0/566156150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1d415dca0 0x7fb1c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.735+0000 7fb1d37fe640 1 --2- 172.21.15.16:0/566156150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1c4004d40 0x7fb1c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.735+0000 7fb1d37fe640 1 -- 172.21.15.16:0/566156150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d40d4370 con 0x7fb1c4004d40 2024-04-25T10:11:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.735+0000 7fb1d37fe640 1 -- 172.21.15.16:0/566156150 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c400b9c0 con 0x7fb1c40039b0 2024-04-25T10:11:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.735+0000 7fb1d37fe640 1 -- 172.21.15.16:0/566156150 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c400bb40 con 0x7fb1d415dca0 2024-04-25T10:11:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.735+0000 7fb1d943e640 1 --2- 172.21.15.16:0/566156150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1c4004d40 0x7fb1c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.735+0000 7fb1d8c3d640 1 --2- 172.21.15.16:0/566156150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1c40039b0 0x7fb1c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.735+0000 7fb1d3fff640 1 --2- 172.21.15.16:0/566156150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1d415dca0 0x7fb1c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.735+0000 7fb1d943e640 1 -- 172.21.15.16:0/566156150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1d415dca0 msgr2=0x7fb1c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.735+0000 7fb1d943e640 1 --2- 172.21.15.16:0/566156150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1d415dca0 0x7fb1c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.735+0000 7fb1d943e640 1 -- 172.21.15.16:0/566156150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1c40039b0 msgr2=0x7fb1c4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.735+0000 7fb1d943e640 1 --2- 172.21.15.16:0/566156150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1c40039b0 0x7fb1c4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.735+0000 7fb1d943e640 1 -- 172.21.15.16:0/566156150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c4003520 con 0x7fb1c4004d40 2024-04-25T10:11:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.736+0000 7fb1d943e640 1 --2- 172.21.15.16:0/566156150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1c4004d40 0x7fb1c40051c0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40d0320 tx=0x7fb1c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c64c991084b4c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.736+0000 7fb1d37fe640 1 -- 172.21.15.16:0/566156150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc009c10 con 0x7fb1c4004d40 2024-04-25T10:11:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.736+0000 7fb1d37fe640 1 -- 172.21.15.16:0/566156150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1cc036070 con 0x7fb1c4004d40 2024-04-25T10:11:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.736+0000 7fb1d37fe640 1 -- 172.21.15.16:0/566156150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc031850 con 0x7fb1c4004d40 2024-04-25T10:11:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.736+0000 7fb1daea1640 1 -- 172.21.15.16:0/566156150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1c4004d40 msgr2=0x7fb1c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.736+0000 7fb1daea1640 1 --2- 172.21.15.16:0/566156150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1c4004d40 0x7fb1c40051c0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40d0320 tx=0x7fb1c4005830 comp rx=0 tx=0).stop 2024-04-25T10:11:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.736+0000 7fb1d943e640 1 -- 172.21.15.16:0/566156150 reap_dead start 2024-04-25T10:11:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.736+0000 7fb1daea1640 1 -- 172.21.15.16:0/566156150 shutdown_connections 2024-04-25T10:11:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.736+0000 7fb1daea1640 1 -- 172.21.15.16:0/566156150 >> 172.21.15.16:0/566156150 conn(0x7fb1d40726c0 msgr2=0x7fb1d4072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.736+0000 7fb1daea1640 1 -- 172.21.15.16:0/566156150 shutdown_connections 2024-04-25T10:11:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.737+0000 7fb1daea1640 1 -- 172.21.15.16:0/566156150 wait complete. 2024-04-25T10:11:11.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.738+0000 7fb1daea1640 1 Processor -- start 2024-04-25T10:11:11.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.738+0000 7fb1daea1640 1 -- start start 2024-04-25T10:11:11.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1daea1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1c4004d40 0x7fb1d40ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1daea1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d41f7040 0x7fb1d40b7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1daea1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1d41f74c0 0x7fb1d40b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1daea1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d415e200 con 0x7fb1d41f7040 2024-04-25T10:11:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1daea1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d415e080 con 0x7fb1c4004d40 2024-04-25T10:11:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1daea1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d40ce940 con 0x7fb1d41f74c0 2024-04-25T10:11:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d41f7040 0x7fb1d40b7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d41f7040 0x7fb1d40b7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53396/0 (socket says 172.21.15.16:53396) 2024-04-25T10:11:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1d3fff640 1 -- 172.21.15.16:0/811414339 learned_addr learned my addr 172.21.15.16:0/811414339 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1d8c3d640 1 --2- 172.21.15.16:0/811414339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1c4004d40 0x7fb1d40ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1d943e640 1 --2- 172.21.15.16:0/811414339 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1d41f74c0 0x7fb1d40b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1d3fff640 1 -- 172.21.15.16:0/811414339 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1d41f74c0 msgr2=0x7fb1d40b7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1d3fff640 1 --2- 172.21.15.16:0/811414339 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1d41f74c0 0x7fb1d40b7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1d3fff640 1 -- 172.21.15.16:0/811414339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1c4004d40 msgr2=0x7fb1d40ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1d3fff640 1 --2- 172.21.15.16:0/811414339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1c4004d40 0x7fb1d40ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1d3fff640 1 -- 172.21.15.16:0/811414339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d4155190 con 0x7fb1d41f7040 2024-04-25T10:11:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1d3fff640 1 --2- 172.21.15.16:0/811414339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d41f7040 0x7fb1d40b7690 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0007340 tx=0x7fb1c0007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1d17fa640 1 -- 172.21.15.16:0/811414339 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c001a070 con 0x7fb1d41f7040 2024-04-25T10:11:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1d17fa640 1 -- 172.21.15.16:0/811414339 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c000f920 con 0x7fb1d41f7040 2024-04-25T10:11:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1daea1640 1 -- 172.21.15.16:0/811414339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d41f7a00 con 0x7fb1d41f7040 2024-04-25T10:11:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1d17fa640 1 -- 172.21.15.16:0/811414339 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c000fd10 con 0x7fb1d41f7040 2024-04-25T10:11:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.739+0000 7fb1daea1640 1 -- 172.21.15.16:0/811414339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d4155430 con 0x7fb1d41f7040 2024-04-25T10:11:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.740+0000 7fb1daea1640 1 -- 172.21.15.16:0/811414339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d41f7f20 con 0x7fb1d41f7040 2024-04-25T10:11:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.741+0000 7fb1d17fa640 1 -- 172.21.15.16:0/811414339 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb1c001e3f0 con 0x7fb1d41f7040 2024-04-25T10:11:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.741+0000 7fb1d17fa640 1 --2- 172.21.15.16:0/811414339 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb1b806b500 0x7fb1b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.741+0000 7fb1d17fa640 1 -- 172.21.15.16:0/811414339 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1c00899d0 con 0x7fb1d41f7040 2024-04-25T10:11:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.741+0000 7fb1d8c3d640 1 --2- 172.21.15.16:0/811414339 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb1b806b500 0x7fb1b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.742+0000 7fb1d8c3d640 1 --2- 172.21.15.16:0/811414339 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb1b806b500 0x7fb1b806d9c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009a80 tx=0x7fb1c8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.745+0000 7fb1d17fa640 1 -- 172.21.15.16:0/811414339 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1d41f7f20 con 0x7fb1d41f7040 2024-04-25T10:11:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.924+0000 7fb1daea1640 1 -- 172.21.15.16:0/811414339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1d4075090 con 0x7fb1d41f7040 2024-04-25T10:11:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.925+0000 7fb1d17fa640 1 -- 172.21.15.16:0/811414339 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb1c0058850 con 0x7fb1d41f7040 2024-04-25T10:11:11.930 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:11:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.927+0000 7fb1daea1640 1 -- 172.21.15.16:0/811414339 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb1b806b500 msgr2=0x7fb1b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.927+0000 7fb1daea1640 1 --2- 172.21.15.16:0/811414339 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb1b806b500 0x7fb1b806d9c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009a80 tx=0x7fb1c8008040 comp rx=0 tx=0).stop 2024-04-25T10:11:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.927+0000 7fb1daea1640 1 -- 172.21.15.16:0/811414339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d41f7040 msgr2=0x7fb1d40b7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.927+0000 7fb1daea1640 1 --2- 172.21.15.16:0/811414339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d41f7040 0x7fb1d40b7690 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0007340 tx=0x7fb1c0007800 comp rx=0 tx=0).stop 2024-04-25T10:11:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.927+0000 7fb1daea1640 1 -- 172.21.15.16:0/811414339 shutdown_connections 2024-04-25T10:11:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.927+0000 7fb1daea1640 1 --2- 172.21.15.16:0/811414339 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb1b806b500 0x7fb1b806d9c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.927+0000 7fb1daea1640 1 --2- 172.21.15.16:0/811414339 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1d41f74c0 0x7fb1d40b7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.927+0000 7fb1daea1640 1 --2- 172.21.15.16:0/811414339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d41f7040 0x7fb1d40b7690 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.927+0000 7fb1daea1640 1 --2- 172.21.15.16:0/811414339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1c4004d40 0x7fb1d40ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.927+0000 7fb1daea1640 1 -- 172.21.15.16:0/811414339 >> 172.21.15.16:0/811414339 conn(0x7fb1d40726c0 msgr2=0x7fb1d4160390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.927+0000 7fb1daea1640 1 -- 172.21.15.16:0/811414339 shutdown_connections 2024-04-25T10:11:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:11.927+0000 7fb1daea1640 1 -- 172.21.15.16:0/811414339 wait complete. 2024-04-25T10:11:11.943 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:12.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.043+0000 7fa85b370640 1 Processor -- start 2024-04-25T10:11:12.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.043+0000 7fa85b370640 1 -- start start 2024-04-25T10:11:12.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.044+0000 7fa85b370640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa85415b7b0 0x7fa8540ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.044+0000 7fa85b370640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8540cf760 0x7fa8540cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.044+0000 7fa85b370640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa85415e110 0x7fa85415e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.044+0000 7fa85b370640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8540d0100 con 0x7fa85415b7b0 2024-04-25T10:11:12.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.044+0000 7fa85b370640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8540ced00 con 0x7fa85415e110 2024-04-25T10:11:12.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.044+0000 7fa85b370640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85415eae0 con 0x7fa8540cf760 2024-04-25T10:11:12.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.044+0000 7fa85910c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa85415b7b0 0x7fa8540ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.044+0000 7fa85910c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa85415b7b0 0x7fa8540ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53410/0 (socket says 172.21.15.16:53410) 2024-04-25T10:11:12.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.044+0000 7fa85910c640 1 -- 172.21.15.16:0/2508684255 learned_addr learned my addr 172.21.15.16:0/2508684255 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:12.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.044+0000 7fa85890b640 1 --2- 172.21.15.16:0/2508684255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8540cf760 0x7fa8540cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.044+0000 7fa85990d640 1 --2- 172.21.15.16:0/2508684255 >> v2:172.21.15.186:3300/0 conn(0x7fa85415e110 0x7fa85415e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.044+0000 7fa85910c640 1 -- 172.21.15.16:0/2508684255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8540cf760 msgr2=0x7fa8540cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.044+0000 7fa85910c640 1 --2- 172.21.15.16:0/2508684255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8540cf760 0x7fa8540cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.044+0000 7fa85910c640 1 -- 172.21.15.16:0/2508684255 >> v2:172.21.15.186:3300/0 conn(0x7fa85415e110 msgr2=0x7fa85415e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.044+0000 7fa85910c640 1 --2- 172.21.15.16:0/2508684255 >> v2:172.21.15.186:3300/0 conn(0x7fa85415e110 0x7fa85415e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.044+0000 7fa85910c640 1 -- 172.21.15.16:0/2508684255 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85415f000 con 0x7fa85415b7b0 2024-04-25T10:11:12.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.044+0000 7fa85990d640 1 --2- 172.21.15.16:0/2508684255 >> v2:172.21.15.186:3300/0 conn(0x7fa85415e110 0x7fa85415e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.045+0000 7fa85910c640 1 --2- 172.21.15.16:0/2508684255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa85415b7b0 0x7fa8540ce7c0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fa854161670 tx=0x7fa85002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4b3d7f2c397e1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.045+0000 7fa84bfff640 1 -- 172.21.15.16:0/2508684255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850037070 con 0x7fa85415b7b0 2024-04-25T10:11:12.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.045+0000 7fa84bfff640 1 -- 172.21.15.16:0/2508684255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa850032440 con 0x7fa85415b7b0 2024-04-25T10:11:12.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.045+0000 7fa84bfff640 1 -- 172.21.15.16:0/2508684255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850032810 con 0x7fa85415b7b0 2024-04-25T10:11:12.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.045+0000 7fa85b370640 1 -- 172.21.15.16:0/2508684255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa85415b7b0 msgr2=0x7fa8540ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.045+0000 7fa85b370640 1 --2- 172.21.15.16:0/2508684255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa85415b7b0 0x7fa8540ce7c0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fa854161670 tx=0x7fa85002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.045+0000 7fa85b370640 1 -- 172.21.15.16:0/2508684255 shutdown_connections 2024-04-25T10:11:12.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.046+0000 7fa85b370640 1 --2- 172.21.15.16:0/2508684255 >> v2:172.21.15.186:3300/0 conn(0x7fa85415e110 0x7fa85415e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.046+0000 7fa85b370640 1 --2- 172.21.15.16:0/2508684255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8540cf760 0x7fa8540cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.046+0000 7fa85b370640 1 --2- 172.21.15.16:0/2508684255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa85415b7b0 0x7fa8540ce7c0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.046+0000 7fa85b370640 1 -- 172.21.15.16:0/2508684255 >> 172.21.15.16:0/2508684255 conn(0x7fa85406f4a0 msgr2=0x7fa85406f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.046+0000 7fa85b370640 1 -- 172.21.15.16:0/2508684255 shutdown_connections 2024-04-25T10:11:12.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.046+0000 7fa85b370640 1 -- 172.21.15.16:0/2508684255 wait complete. 2024-04-25T10:11:12.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.047+0000 7fa85b370640 1 Processor -- start 2024-04-25T10:11:12.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.047+0000 7fa85b370640 1 -- start start 2024-04-25T10:11:12.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.047+0000 7fa85b370640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8540cf760 0x7fa8541588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa85b370640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa85415b7b0 0x7fa8540b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa85b370640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa85415e110 0x7fa8540b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa85b370640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540cdce0 con 0x7fa85415e110 2024-04-25T10:11:12.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa85b370640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540ce330 con 0x7fa8540cf760 2024-04-25T10:11:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa85b370640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540cdfc0 con 0x7fa85415b7b0 2024-04-25T10:11:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa85990d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa85415e110 0x7fa8540b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa85990d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa85415e110 0x7fa8540b6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53426/0 (socket says 172.21.15.16:53426) 2024-04-25T10:11:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa85990d640 1 -- 172.21.15.16:0/3260021765 learned_addr learned my addr 172.21.15.16:0/3260021765 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa85910c640 1 --2- 172.21.15.16:0/3260021765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8540cf760 0x7fa8541588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa85890b640 1 --2- 172.21.15.16:0/3260021765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa85415b7b0 0x7fa8540b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa85990d640 1 -- 172.21.15.16:0/3260021765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa85415b7b0 msgr2=0x7fa8540b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa85990d640 1 --2- 172.21.15.16:0/3260021765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa85415b7b0 0x7fa8540b5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa85990d640 1 -- 172.21.15.16:0/3260021765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8540cf760 msgr2=0x7fa8541588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa85990d640 1 --2- 172.21.15.16:0/3260021765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8540cf760 0x7fa8541588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa85990d640 1 -- 172.21.15.16:0/3260021765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8540b69d0 con 0x7fa85415e110 2024-04-25T10:11:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa85910c640 1 --2- 172.21.15.16:0/3260021765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8540cf760 0x7fa8541588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa85990d640 1 --2- 172.21.15.16:0/3260021765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa85415e110 0x7fa8540b6460 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00cf20 tx=0x7fa84c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa849ffb640 1 -- 172.21.15.16:0/3260021765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c01a070 con 0x7fa85415e110 2024-04-25T10:11:12.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa849ffb640 1 -- 172.21.15.16:0/3260021765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84c00d840 con 0x7fa85415e110 2024-04-25T10:11:12.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa85b370640 1 -- 172.21.15.16:0/3260021765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8540b2430 con 0x7fa85415e110 2024-04-25T10:11:12.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.048+0000 7fa849ffb640 1 -- 172.21.15.16:0/3260021765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c00dc30 con 0x7fa85415e110 2024-04-25T10:11:12.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.049+0000 7fa85b370640 1 -- 172.21.15.16:0/3260021765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8540b2c70 con 0x7fa85415e110 2024-04-25T10:11:12.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.049+0000 7fa85b370640 1 -- 172.21.15.16:0/3260021765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa85406dc80 con 0x7fa85415e110 2024-04-25T10:11:12.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.050+0000 7fa849ffb640 1 -- 172.21.15.16:0/3260021765 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa84c01d070 con 0x7fa85415e110 2024-04-25T10:11:12.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.050+0000 7fa849ffb640 1 --2- 172.21.15.16:0/3260021765 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa83806b4d0 0x7fa83806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.050+0000 7fa849ffb640 1 -- 172.21.15.16:0/3260021765 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa84c08aa00 con 0x7fa85415e110 2024-04-25T10:11:12.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.051+0000 7fa85910c640 1 --2- 172.21.15.16:0/3260021765 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa83806b4d0 0x7fa83806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.051+0000 7fa85910c640 1 --2- 172.21.15.16:0/3260021765 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa83806b4d0 0x7fa83806d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa850034000 tx=0x7fa85002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.055+0000 7fa849ffb640 1 -- 172.21.15.16:0/3260021765 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa85406dc80 con 0x7fa85415e110 2024-04-25T10:11:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.232+0000 7fa85b370640 1 -- 172.21.15.16:0/3260021765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa8540750b0 con 0x7fa85415e110 2024-04-25T10:11:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.232+0000 7fa849ffb640 1 -- 172.21.15.16:0/3260021765 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa8540b2430 con 0x7fa85415e110 2024-04-25T10:11:12.238 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.234+0000 7fa85b370640 1 -- 172.21.15.16:0/3260021765 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa83806b4d0 msgr2=0x7fa83806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.234+0000 7fa85b370640 1 --2- 172.21.15.16:0/3260021765 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa83806b4d0 0x7fa83806d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa850034000 tx=0x7fa85002e000 comp rx=0 tx=0).stop 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.234+0000 7fa85b370640 1 -- 172.21.15.16:0/3260021765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa85415e110 msgr2=0x7fa8540b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.234+0000 7fa85b370640 1 --2- 172.21.15.16:0/3260021765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa85415e110 0x7fa8540b6460 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00cf20 tx=0x7fa84c00a7d0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.234+0000 7fa85b370640 1 -- 172.21.15.16:0/3260021765 shutdown_connections 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.234+0000 7fa85b370640 1 --2- 172.21.15.16:0/3260021765 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa83806b4d0 0x7fa83806d990 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.234+0000 7fa85b370640 1 --2- 172.21.15.16:0/3260021765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa85415e110 0x7fa8540b6460 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.234+0000 7fa85b370640 1 --2- 172.21.15.16:0/3260021765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa85415b7b0 0x7fa8540b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.234+0000 7fa85b370640 1 --2- 172.21.15.16:0/3260021765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8540cf760 0x7fa8541588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.234+0000 7fa85b370640 1 -- 172.21.15.16:0/3260021765 >> 172.21.15.16:0/3260021765 conn(0x7fa85406f4a0 msgr2=0x7fa854159a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.234+0000 7fa85b370640 1 -- 172.21.15.16:0/3260021765 shutdown_connections 2024-04-25T10:11:12.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.234+0000 7fa85b370640 1 -- 172.21.15.16:0/3260021765 wait complete. 2024-04-25T10:11:12.250 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:12.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.349+0000 7fb8be1f9640 1 Processor -- start 2024-04-25T10:11:12.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.349+0000 7fb8be1f9640 1 -- start start 2024-04-25T10:11:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.350+0000 7fb8be1f9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8b815e820 0x7fb8b80ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.350+0000 7fb8be1f9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b815f000 con 0x7fb8b80cfe70 2024-04-25T10:11:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.350+0000 7fb8be1f9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b815ed20 con 0x7fb8b815b7b0 2024-04-25T10:11:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.350+0000 7fb8be1f9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b80ced00 con 0x7fb8b815e820 2024-04-25T10:11:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.350+0000 7fb8b7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8b815e820 0x7fb8b80ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.350+0000 7fb8b6ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb8b80cfe70 0x7fb8b815e110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55550/0 (socket says 172.21.15.16:55550) 2024-04-25T10:11:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.350+0000 7fb8b6ffd640 1 -- 172.21.15.16:0/1425085926 learned_addr learned my addr 172.21.15.16:0/1425085926 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.350+0000 7fb8b7fff640 1 -- 172.21.15.16:0/1425085926 >> v1:172.21.15.186:6789/0 conn(0x7fb8b815b7b0 legacy=0x7fb8b80cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.350+0000 7fb8b7fff640 1 -- 172.21.15.16:0/1425085926 >> v1:172.21.15.16:6789/0 conn(0x7fb8b80cfe70 legacy=0x7fb8b815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.350+0000 7fb8b7fff640 1 -- 172.21.15.16:0/1425085926 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b80cdce0 con 0x7fb8b815e820 2024-04-25T10:11:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.350+0000 7fb8b67fc640 1 -- 172.21.15.16:0/1425085926 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2707015361 0 0) 0x7fb8b815f000 con 0x7fb8b80cfe70 2024-04-25T10:11:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.351+0000 7fb8b7fff640 1 --2- 172.21.15.16:0/1425085926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b815e820 0x7fb8b80ce7c0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0009d90 tx=0x7fb8b002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4bf5a15ca6c6504 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.351+0000 7fb8b67fc640 1 -- 172.21.15.16:0/1425085926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0007740 con 0x7fb8b815e820 2024-04-25T10:11:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.351+0000 7fb8b67fc640 1 -- 172.21.15.16:0/1425085926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b00078e0 con 0x7fb8b815e820 2024-04-25T10:11:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.351+0000 7fb8b67fc640 1 -- 172.21.15.16:0/1425085926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0007cb0 con 0x7fb8b815e820 2024-04-25T10:11:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.352+0000 7fb8be1f9640 1 -- 172.21.15.16:0/1425085926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b815e820 msgr2=0x7fb8b80ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.352+0000 7fb8be1f9640 1 --2- 172.21.15.16:0/1425085926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b815e820 0x7fb8b80ce7c0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0009d90 tx=0x7fb8b002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.352+0000 7fb8be1f9640 1 -- 172.21.15.16:0/1425085926 shutdown_connections 2024-04-25T10:11:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.352+0000 7fb8be1f9640 1 --2- 172.21.15.16:0/1425085926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b815e820 0x7fb8b80ce7c0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.352+0000 7fb8be1f9640 1 -- 172.21.15.16:0/1425085926 >> 172.21.15.16:0/1425085926 conn(0x7fb8b806f4a0 msgr2=0x7fb8b806f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.352+0000 7fb8be1f9640 1 -- 172.21.15.16:0/1425085926 shutdown_connections 2024-04-25T10:11:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.352+0000 7fb8be1f9640 1 -- 172.21.15.16:0/1425085926 wait complete. 2024-04-25T10:11:12.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.353+0000 7fb8be1f9640 1 Processor -- start 2024-04-25T10:11:12.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.353+0000 7fb8be1f9640 1 -- start start 2024-04-25T10:11:12.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.354+0000 7fb8be1f9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8b80cfe70 0x7fb8b815d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.354+0000 7fb8be1f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b815b7b0 0x7fb8b81f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.354+0000 7fb8be1f9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8b815e820 0x7fb8b81f7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.354+0000 7fb8be1f9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b81619f0 con 0x7fb8b815b7b0 2024-04-25T10:11:12.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.354+0000 7fb8be1f9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b80ce330 con 0x7fb8b80cfe70 2024-04-25T10:11:12.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.354+0000 7fb8be1f9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b80cc080 con 0x7fb8b815e820 2024-04-25T10:11:12.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.354+0000 7fb8b6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b815b7b0 0x7fb8b81f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.354+0000 7fb8b6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b815b7b0 0x7fb8b81f7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53440/0 (socket says 172.21.15.16:53440) 2024-04-25T10:11:12.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.354+0000 7fb8b6ffd640 1 -- 172.21.15.16:0/3253935852 learned_addr learned my addr 172.21.15.16:0/3253935852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:12.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.354+0000 7fb8b77fe640 1 --2- 172.21.15.16:0/3253935852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8b80cfe70 0x7fb8b815d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.354+0000 7fb8b7fff640 1 --2- 172.21.15.16:0/3253935852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8b815e820 0x7fb8b81f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.354+0000 7fb8b6ffd640 1 -- 172.21.15.16:0/3253935852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8b815e820 msgr2=0x7fb8b81f7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.354+0000 7fb8b6ffd640 1 --2- 172.21.15.16:0/3253935852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8b815e820 0x7fb8b81f7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.354+0000 7fb8b6ffd640 1 -- 172.21.15.16:0/3253935852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8b80cfe70 msgr2=0x7fb8b815d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.354+0000 7fb8b6ffd640 1 --2- 172.21.15.16:0/3253935852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8b80cfe70 0x7fb8b815d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.354+0000 7fb8b6ffd640 1 -- 172.21.15.16:0/3253935852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b80cb840 con 0x7fb8b815b7b0 2024-04-25T10:11:12.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.354+0000 7fb8b6ffd640 1 --2- 172.21.15.16:0/3253935852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b815b7b0 0x7fb8b81f7350 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00c4d0 tx=0x7fb8ac00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.354+0000 7fb89ffff640 1 -- 172.21.15.16:0/3253935852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac018070 con 0x7fb8b815b7b0 2024-04-25T10:11:12.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.355+0000 7fb89ffff640 1 -- 172.21.15.16:0/3253935852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ac007500 con 0x7fb8b815b7b0 2024-04-25T10:11:12.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.355+0000 7fb8be1f9640 1 -- 172.21.15.16:0/3253935852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b81f7ec0 con 0x7fb8b815b7b0 2024-04-25T10:11:12.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.355+0000 7fb89ffff640 1 -- 172.21.15.16:0/3253935852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac004a20 con 0x7fb8b815b7b0 2024-04-25T10:11:12.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.355+0000 7fb8be1f9640 1 -- 172.21.15.16:0/3253935852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b80c5700 con 0x7fb8b815b7b0 2024-04-25T10:11:12.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.356+0000 7fb8be1f9640 1 -- 172.21.15.16:0/3253935852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b80750b0 con 0x7fb8b815b7b0 2024-04-25T10:11:12.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.356+0000 7fb89ffff640 1 -- 172.21.15.16:0/3253935852 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb8ac01b070 con 0x7fb8b815b7b0 2024-04-25T10:11:12.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.357+0000 7fb89ffff640 1 --2- 172.21.15.16:0/3253935852 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb8a406b4d0 0x7fb8a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.357+0000 7fb89ffff640 1 -- 172.21.15.16:0/3253935852 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8ac089460 con 0x7fb8b815b7b0 2024-04-25T10:11:12.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.357+0000 7fb8b77fe640 1 --2- 172.21.15.16:0/3253935852 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb8a406b4d0 0x7fb8a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.357+0000 7fb8b77fe640 1 --2- 172.21.15.16:0/3253935852 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb8a406b4d0 0x7fb8a406d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8004f60 tx=0x7fb8a8007e30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.361+0000 7fb89ffff640 1 -- 172.21.15.16:0/3253935852 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb8ac057c90 con 0x7fb8b815b7b0 2024-04-25T10:11:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.538+0000 7fb8be1f9640 1 -- 172.21.15.16:0/3253935852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb8b80b7870 con 0x7fb8b815b7b0 2024-04-25T10:11:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.539+0000 7fb89ffff640 1 -- 172.21.15.16:0/3253935852 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb8ac05bd30 con 0x7fb8b815b7b0 2024-04-25T10:11:12.544 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:11:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.540+0000 7fb8be1f9640 1 -- 172.21.15.16:0/3253935852 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb8a406b4d0 msgr2=0x7fb8a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.540+0000 7fb8be1f9640 1 --2- 172.21.15.16:0/3253935852 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb8a406b4d0 0x7fb8a406d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8004f60 tx=0x7fb8a8007e30 comp rx=0 tx=0).stop 2024-04-25T10:11:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.540+0000 7fb8be1f9640 1 -- 172.21.15.16:0/3253935852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b815b7b0 msgr2=0x7fb8b81f7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.540+0000 7fb8be1f9640 1 --2- 172.21.15.16:0/3253935852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b815b7b0 0x7fb8b81f7350 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00c4d0 tx=0x7fb8ac00c990 comp rx=0 tx=0).stop 2024-04-25T10:11:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.541+0000 7fb8be1f9640 1 -- 172.21.15.16:0/3253935852 shutdown_connections 2024-04-25T10:11:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.541+0000 7fb8be1f9640 1 --2- 172.21.15.16:0/3253935852 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb8a406b4d0 0x7fb8a406d990 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.541+0000 7fb8be1f9640 1 --2- 172.21.15.16:0/3253935852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8b815e820 0x7fb8b81f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.541+0000 7fb8be1f9640 1 --2- 172.21.15.16:0/3253935852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b815b7b0 0x7fb8b81f7350 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.541+0000 7fb8be1f9640 1 --2- 172.21.15.16:0/3253935852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8b80cfe70 0x7fb8b815d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.541+0000 7fb8be1f9640 1 -- 172.21.15.16:0/3253935852 >> 172.21.15.16:0/3253935852 conn(0x7fb8b806f4a0 msgr2=0x7fb8b806f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.541+0000 7fb8be1f9640 1 -- 172.21.15.16:0/3253935852 shutdown_connections 2024-04-25T10:11:12.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.541+0000 7fb8be1f9640 1 -- 172.21.15.16:0/3253935852 wait complete. 2024-04-25T10:11:12.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:11:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.653+0000 7fd7fa353640 1 Processor -- start 2024-04-25T10:11:12.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.653+0000 7fd7fa353640 1 -- start start 2024-04-25T10:11:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.654+0000 7fd7fa353640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd7f41610c0 0x7fd7f40acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.654+0000 7fd7fa353640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f406c580 con 0x7fd7f40ce240 2024-04-25T10:11:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.654+0000 7fd7fa353640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f406dde0 con 0x7fd7f406bac0 2024-04-25T10:11:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.654+0000 7fd7fa353640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f41614a0 con 0x7fd7f41610c0 2024-04-25T10:11:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.654+0000 7fd7f3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd7f41610c0 0x7fd7f40acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.654+0000 7fd7f88f0640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd7f40ce240 0x7fd7f406eba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55556/0 (socket says 172.21.15.16:55556) 2024-04-25T10:11:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.654+0000 7fd7f88f0640 1 -- 172.21.15.16:0/4001311792 learned_addr learned my addr 172.21.15.16:0/4001311792 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.654+0000 7fd7f3fff640 1 -- 172.21.15.16:0/4001311792 >> v1:172.21.15.186:6789/0 conn(0x7fd7f406bac0 legacy=0x7fd7f40cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.654+0000 7fd7f2ffd640 1 -- 172.21.15.16:0/4001311792 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2497049627 0 0) 0x7fd7f406c580 con 0x7fd7f40ce240 2024-04-25T10:11:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.655+0000 7fd7f3fff640 1 -- 172.21.15.16:0/4001311792 >> v1:172.21.15.16:6789/0 conn(0x7fd7f40ce240 legacy=0x7fd7f406eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.655+0000 7fd7f3fff640 1 -- 172.21.15.16:0/4001311792 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f406e0c0 con 0x7fd7f41610c0 2024-04-25T10:11:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.655+0000 7fd7f3fff640 1 --2- 172.21.15.16:0/4001311792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f41610c0 0x7fd7f40acdf0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40097a0 tx=0x7fd7e402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a46aa70a1a72f66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.655+0000 7fd7f2ffd640 1 -- 172.21.15.16:0/4001311792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4037070 con 0x7fd7f41610c0 2024-04-25T10:11:12.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.655+0000 7fd7f2ffd640 1 -- 172.21.15.16:0/4001311792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e4032440 con 0x7fd7f41610c0 2024-04-25T10:11:12.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.655+0000 7fd7f2ffd640 1 -- 172.21.15.16:0/4001311792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4032810 con 0x7fd7f41610c0 2024-04-25T10:11:12.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.656+0000 7fd7fa353640 1 -- 172.21.15.16:0/4001311792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f41610c0 msgr2=0x7fd7f40acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.656+0000 7fd7fa353640 1 --2- 172.21.15.16:0/4001311792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f41610c0 0x7fd7f40acdf0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40097a0 tx=0x7fd7e402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.656+0000 7fd7fa353640 1 -- 172.21.15.16:0/4001311792 shutdown_connections 2024-04-25T10:11:12.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.656+0000 7fd7fa353640 1 --2- 172.21.15.16:0/4001311792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f41610c0 0x7fd7f40acdf0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.656+0000 7fd7fa353640 1 -- 172.21.15.16:0/4001311792 >> 172.21.15.16:0/4001311792 conn(0x7fd7f40cc9e0 msgr2=0x7fd7f40ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.656+0000 7fd7fa353640 1 -- 172.21.15.16:0/4001311792 shutdown_connections 2024-04-25T10:11:12.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.656+0000 7fd7fa353640 1 -- 172.21.15.16:0/4001311792 wait complete. 2024-04-25T10:11:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.657+0000 7fd7fa353640 1 Processor -- start 2024-04-25T10:11:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.657+0000 7fd7fa353640 1 -- start start 2024-04-25T10:11:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.658+0000 7fd7fa353640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7f406bac0 0x7fd7f415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.658+0000 7fd7fa353640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7f40ce240 0x7fd7f415a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.658+0000 7fd7fa353640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f41610c0 0x7fd7f4157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.658+0000 7fd7fa353640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f406add0 con 0x7fd7f41610c0 2024-04-25T10:11:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.658+0000 7fd7fa353640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f406e680 con 0x7fd7f40ce240 2024-04-25T10:11:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.658+0000 7fd7fa353640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f406bf50 con 0x7fd7f406bac0 2024-04-25T10:11:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.658+0000 7fd7f88f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f41610c0 0x7fd7f4157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.658+0000 7fd7f88f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f41610c0 0x7fd7f4157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53464/0 (socket says 172.21.15.16:53464) 2024-04-25T10:11:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.658+0000 7fd7f88f0640 1 -- 172.21.15.16:0/118904929 learned_addr learned my addr 172.21.15.16:0/118904929 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.658+0000 7fd7f37fe640 1 --2- 172.21.15.16:0/118904929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7f40ce240 0x7fd7f415a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.658+0000 7fd7f3fff640 1 --2- 172.21.15.16:0/118904929 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7f406bac0 0x7fd7f415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.659+0000 7fd7f88f0640 1 -- 172.21.15.16:0/118904929 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7f406bac0 msgr2=0x7fd7f415eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.659+0000 7fd7f88f0640 1 --2- 172.21.15.16:0/118904929 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7f406bac0 0x7fd7f415eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.659+0000 7fd7f88f0640 1 -- 172.21.15.16:0/118904929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7f40ce240 msgr2=0x7fd7f415a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.659+0000 7fd7f88f0640 1 --2- 172.21.15.16:0/118904929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7f40ce240 0x7fd7f415a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.659+0000 7fd7f88f0640 1 -- 172.21.15.16:0/118904929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f415b1a0 con 0x7fd7f41610c0 2024-04-25T10:11:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.659+0000 7fd7f88f0640 1 --2- 172.21.15.16:0/118904929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f41610c0 0x7fd7f4157f00 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00c7a0 tx=0x7fd7ec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.659+0000 7fd7f0ff9640 1 -- 172.21.15.16:0/118904929 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec018070 con 0x7fd7f41610c0 2024-04-25T10:11:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.659+0000 7fd7f0ff9640 1 -- 172.21.15.16:0/118904929 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ec00a640 con 0x7fd7f41610c0 2024-04-25T10:11:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.659+0000 7fd7fa353640 1 -- 172.21.15.16:0/118904929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f415b480 con 0x7fd7f41610c0 2024-04-25T10:11:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.659+0000 7fd7f0ff9640 1 -- 172.21.15.16:0/118904929 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec00aab0 con 0x7fd7f41610c0 2024-04-25T10:11:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.659+0000 7fd7fa353640 1 -- 172.21.15.16:0/118904929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f415ba40 con 0x7fd7f41610c0 2024-04-25T10:11:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.660+0000 7fd7fa353640 1 -- 172.21.15.16:0/118904929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f40cdd60 con 0x7fd7f41610c0 2024-04-25T10:11:12.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.660+0000 7fd7f0ff9640 1 -- 172.21.15.16:0/118904929 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7ec01b070 con 0x7fd7f41610c0 2024-04-25T10:11:12.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.661+0000 7fd7f0ff9640 1 --2- 172.21.15.16:0/118904929 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd7e006b4d0 0x7fd7e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.661+0000 7fd7f0ff9640 1 -- 172.21.15.16:0/118904929 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7ec0896e0 con 0x7fd7f41610c0 2024-04-25T10:11:12.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.661+0000 7fd7f3fff640 1 --2- 172.21.15.16:0/118904929 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd7e006b4d0 0x7fd7e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.662+0000 7fd7f3fff640 1 --2- 172.21.15.16:0/118904929 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd7e006b4d0 0x7fd7e006d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4034000 tx=0x7fd7e402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.665+0000 7fd7f0ff9640 1 -- 172.21.15.16:0/118904929 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7f40cdd60 con 0x7fd7f41610c0 2024-04-25T10:11:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.859+0000 7fd7fa353640 1 -- 172.21.15.16:0/118904929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd7f40cd8f0 con 0x7fd7f41610c0 2024-04-25T10:11:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.859+0000 7fd7f0ff9640 1 -- 172.21.15.16:0/118904929 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v86) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd7ec057fa0 con 0x7fd7f41610c0 2024-04-25T10:11:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-04-25T10:11:12.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.861+0000 7fd7fa353640 1 -- 172.21.15.16:0/118904929 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd7e006b4d0 msgr2=0x7fd7e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.861+0000 7fd7fa353640 1 --2- 172.21.15.16:0/118904929 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd7e006b4d0 0x7fd7e006d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4034000 tx=0x7fd7e402e000 comp rx=0 tx=0).stop 2024-04-25T10:11:12.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.861+0000 7fd7fa353640 1 -- 172.21.15.16:0/118904929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f41610c0 msgr2=0x7fd7f4157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.861+0000 7fd7fa353640 1 --2- 172.21.15.16:0/118904929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f41610c0 0x7fd7f4157f00 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00c7a0 tx=0x7fd7ec00cc60 comp rx=0 tx=0).stop 2024-04-25T10:11:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.861+0000 7fd7fa353640 1 -- 172.21.15.16:0/118904929 shutdown_connections 2024-04-25T10:11:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.861+0000 7fd7fa353640 1 --2- 172.21.15.16:0/118904929 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd7e006b4d0 0x7fd7e006d990 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.861+0000 7fd7fa353640 1 --2- 172.21.15.16:0/118904929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f41610c0 0x7fd7f4157f00 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.861+0000 7fd7fa353640 1 --2- 172.21.15.16:0/118904929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7f40ce240 0x7fd7f415a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.861+0000 7fd7fa353640 1 --2- 172.21.15.16:0/118904929 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7f406bac0 0x7fd7f415eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.861+0000 7fd7fa353640 1 -- 172.21.15.16:0/118904929 >> 172.21.15.16:0/118904929 conn(0x7fd7f40cc9e0 msgr2=0x7fd7f40ae060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.861+0000 7fd7fa353640 1 -- 172.21.15.16:0/118904929 shutdown_connections 2024-04-25T10:11:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:12.861+0000 7fd7fa353640 1 -- 172.21.15.16:0/118904929 wait complete. 2024-04-25T10:11:12.878 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:11:12.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:12.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:12.923 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:11:12.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:12.923 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:11:12.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.115+0000 7faea37e0640 1 Processor -- start 2024-04-25T10:11:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.115+0000 7faea37e0640 1 -- start start 2024-04-25T10:11:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.116+0000 7faea37e0640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fae9c15a730 0x7fae9c0ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.116+0000 7faea37e0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae9c0b4020 0x7fae9c0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.116+0000 7faea37e0640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9c0cc020 con 0x7fae9c06d1b0 2024-04-25T10:11:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.116+0000 7faea37e0640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9c0b49a0 con 0x7fae9c15a730 2024-04-25T10:11:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.116+0000 7faea37e0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c0b32b0 con 0x7fae9c0b4020 2024-04-25T10:11:13.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.116+0000 7faea0d7b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae9c0b4020 0x7fae9c0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.116+0000 7faea157c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fae9c15a730 0x7fae9c0ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.116+0000 7faea1d7d640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fae9c06d1b0 0x7fae9c06d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:35524/0 (socket says 172.21.15.16:35524) 2024-04-25T10:11:13.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.116+0000 7faea1d7d640 1 -- 172.21.15.16:0/3074317727 learned_addr learned my addr 172.21.15.16:0/3074317727 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:13.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.117+0000 7fae9bfff640 1 -- 172.21.15.16:0/3074317727 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140318238 0 0) 0x7fae9c0cc020 con 0x7fae9c06d1b0 2024-04-25T10:11:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.117+0000 7fae9bfff640 1 -- 172.21.15.16:0/3074317727 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae84003520 con 0x7fae9c06d1b0 2024-04-25T10:11:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.117+0000 7faea0d7b640 1 -- 172.21.15.16:0/3074317727 >> v2:172.21.15.186:3300/0 conn(0x7fae9c15a730 msgr2=0x7fae9c0ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.117+0000 7faea0d7b640 1 --2- 172.21.15.16:0/3074317727 >> v2:172.21.15.186:3300/0 conn(0x7fae9c15a730 0x7fae9c0ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.117+0000 7faea0d7b640 1 -- 172.21.15.16:0/3074317727 >> v1:172.21.15.186:6789/0 conn(0x7fae9c06d1b0 legacy=0x7fae9c06d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.117+0000 7faea0d7b640 1 -- 172.21.15.16:0/3074317727 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c0b3590 con 0x7fae9c0b4020 2024-04-25T10:11:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.117+0000 7faea157c640 1 --2- 172.21.15.16:0/3074317727 >> v2:172.21.15.186:3300/0 conn(0x7fae9c15a730 0x7fae9c0ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.117+0000 7faea0d7b640 1 --2- 172.21.15.16:0/3074317727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae9c0b4020 0x7fae9c0b4460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0b4b20 tx=0x7fae9002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6468a210112eb7dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.117+0000 7fae9bfff640 1 -- 172.21.15.16:0/3074317727 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae90037070 con 0x7fae9c0b4020 2024-04-25T10:11:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.118+0000 7fae9bfff640 1 -- 172.21.15.16:0/3074317727 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae90032440 con 0x7fae9c0b4020 2024-04-25T10:11:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.118+0000 7fae9bfff640 1 -- 172.21.15.16:0/3074317727 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae90032810 con 0x7fae9c0b4020 2024-04-25T10:11:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.118+0000 7faea37e0640 1 -- 172.21.15.16:0/3074317727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae9c0b4020 msgr2=0x7fae9c0b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.118+0000 7faea37e0640 1 --2- 172.21.15.16:0/3074317727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae9c0b4020 0x7fae9c0b4460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0b4b20 tx=0x7fae9002a180 comp rx=0 tx=0).stop 2024-04-25T10:11:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.118+0000 7faea37e0640 1 -- 172.21.15.16:0/3074317727 shutdown_connections 2024-04-25T10:11:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.118+0000 7faea37e0640 1 --2- 172.21.15.16:0/3074317727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae9c0b4020 0x7fae9c0b4460 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.118+0000 7faea37e0640 1 --2- 172.21.15.16:0/3074317727 >> v2:172.21.15.186:3300/0 conn(0x7fae9c15a730 0x7fae9c0ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.118+0000 7faea37e0640 1 -- 172.21.15.16:0/3074317727 >> 172.21.15.16:0/3074317727 conn(0x7fae9c06aba0 msgr2=0x7fae9c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.118+0000 7faea37e0640 1 -- 172.21.15.16:0/3074317727 shutdown_connections 2024-04-25T10:11:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.118+0000 7faea37e0640 1 -- 172.21.15.16:0/3074317727 wait complete. 2024-04-25T10:11:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.119+0000 7faea37e0640 1 Processor -- start 2024-04-25T10:11:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.119+0000 7faea37e0640 1 -- start start 2024-04-25T10:11:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.120+0000 7faea37e0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae9c06d1b0 0x7fae9c154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.120+0000 7faea37e0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae9c0b4020 0x7fae9c1524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.120+0000 7faea37e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae9c15a730 0x7fae9c144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.120+0000 7faea37e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c0cd620 con 0x7fae9c15a730 2024-04-25T10:11:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.120+0000 7faea37e0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c0b3b50 con 0x7fae9c06d1b0 2024-04-25T10:11:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.120+0000 7faea37e0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c0aeef0 con 0x7fae9c0b4020 2024-04-25T10:11:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.120+0000 7faea1d7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae9c15a730 0x7fae9c144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.120+0000 7faea1d7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae9c15a730 0x7fae9c144750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52806/0 (socket says 172.21.15.16:52806) 2024-04-25T10:11:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.120+0000 7faea1d7d640 1 -- 172.21.15.16:0/2327193628 learned_addr learned my addr 172.21.15.16:0/2327193628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.120+0000 7faea157c640 1 --2- 172.21.15.16:0/2327193628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae9c06d1b0 0x7fae9c154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.120+0000 7faea0d7b640 1 --2- 172.21.15.16:0/2327193628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae9c0b4020 0x7fae9c1524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.121+0000 7faea1d7d640 1 -- 172.21.15.16:0/2327193628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae9c0b4020 msgr2=0x7fae9c1524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.121+0000 7faea1d7d640 1 --2- 172.21.15.16:0/2327193628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae9c0b4020 0x7fae9c1524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.121+0000 7faea1d7d640 1 -- 172.21.15.16:0/2327193628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae9c06d1b0 msgr2=0x7fae9c154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.121+0000 7faea1d7d640 1 --2- 172.21.15.16:0/2327193628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae9c06d1b0 0x7fae9c154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.121+0000 7faea1d7d640 1 -- 172.21.15.16:0/2327193628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c155020 con 0x7fae9c15a730 2024-04-25T10:11:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.121+0000 7faea1d7d640 1 --2- 172.21.15.16:0/2327193628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae9c15a730 0x7fae9c144750 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00cb30 tx=0x7fae8c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.121+0000 7fae99ffb640 1 -- 172.21.15.16:0/2327193628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c017070 con 0x7fae9c15a730 2024-04-25T10:11:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.121+0000 7fae99ffb640 1 -- 172.21.15.16:0/2327193628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c00aca0 con 0x7fae9c15a730 2024-04-25T10:11:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.121+0000 7faea37e0640 1 -- 172.21.15.16:0/2327193628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9c155300 con 0x7fae9c15a730 2024-04-25T10:11:13.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.121+0000 7fae99ffb640 1 -- 172.21.15.16:0/2327193628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c012600 con 0x7fae9c15a730 2024-04-25T10:11:13.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.121+0000 7faea37e0640 1 -- 172.21.15.16:0/2327193628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9c145210 con 0x7fae9c15a730 2024-04-25T10:11:13.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.122+0000 7faea37e0640 1 -- 172.21.15.16:0/2327193628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae9c0d0530 con 0x7fae9c15a730 2024-04-25T10:11:13.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.122+0000 7fae99ffb640 1 -- 172.21.15.16:0/2327193628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae8c01a070 con 0x7fae9c15a730 2024-04-25T10:11:13.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.123+0000 7fae99ffb640 1 --2- 172.21.15.16:0/2327193628 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae8806b4d0 0x7fae8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.123+0000 7fae99ffb640 1 -- 172.21.15.16:0/2327193628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae8c089e80 con 0x7fae9c15a730 2024-04-25T10:11:13.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.123+0000 7faea157c640 1 --2- 172.21.15.16:0/2327193628 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae8806b4d0 0x7fae8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.124+0000 7faea157c640 1 --2- 172.21.15.16:0/2327193628 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae8806b4d0 0x7fae8806d990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fae94009560 tx=0x7fae94009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.127+0000 7fae99ffb640 1 -- 172.21.15.16:0/2327193628 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae9c0d0530 con 0x7fae9c15a730 2024-04-25T10:11:13.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.342+0000 7faea37e0640 1 -- 172.21.15.16:0/2327193628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae9c0aeaa0 con 0x7fae9c15a730 2024-04-25T10:11:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.343+0000 7fae99ffb640 1 -- 172.21.15.16:0/2327193628 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fae8c058740 con 0x7fae9c15a730 2024-04-25T10:11:13.348 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:13.349 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4044509901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4044509901},{"type":"v1","addr":"172.21.15.16:6835","nonce":4044509901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/84349474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":84349474},{"type":"v1","addr":"172.21.15.16:6834","nonce":84349474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7335,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/3950732718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3950732718},{"type":"v1","addr":"172.21.15.186:6836","nonce":3950732718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:00.015075+0000","modified":"2024-04-25T10:11:02.007788+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35852":{"gid":35852,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/3092291936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":3092291936},{"type":"v1","addr":"172.21.15.186:6837","nonce":3092291936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 51 2024-04-25T10:11:13.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.344+0000 7faea37e0640 1 -- 172.21.15.16:0/2327193628 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae8806b4d0 msgr2=0x7fae8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.344+0000 7faea37e0640 1 --2- 172.21.15.16:0/2327193628 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae8806b4d0 0x7fae8806d990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fae94009560 tx=0x7fae94009190 comp rx=0 tx=0).stop 2024-04-25T10:11:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.344+0000 7faea37e0640 1 -- 172.21.15.16:0/2327193628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae9c15a730 msgr2=0x7fae9c144750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.344+0000 7faea37e0640 1 --2- 172.21.15.16:0/2327193628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae9c15a730 0x7fae9c144750 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00cb30 tx=0x7fae8c00a060 comp rx=0 tx=0).stop 2024-04-25T10:11:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.345+0000 7faea37e0640 1 -- 172.21.15.16:0/2327193628 shutdown_connections 2024-04-25T10:11:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.345+0000 7faea37e0640 1 --2- 172.21.15.16:0/2327193628 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae8806b4d0 0x7fae8806d990 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.345+0000 7faea37e0640 1 --2- 172.21.15.16:0/2327193628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae9c15a730 0x7fae9c144750 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.345+0000 7faea37e0640 1 --2- 172.21.15.16:0/2327193628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae9c0b4020 0x7fae9c1524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.345+0000 7faea37e0640 1 --2- 172.21.15.16:0/2327193628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae9c06d1b0 0x7fae9c154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.345+0000 7faea37e0640 1 -- 172.21.15.16:0/2327193628 >> 172.21.15.16:0/2327193628 conn(0x7fae9c06aba0 msgr2=0x7fae9c0b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.345+0000 7faea37e0640 1 -- 172.21.15.16:0/2327193628 shutdown_connections 2024-04-25T10:11:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.345+0000 7faea37e0640 1 -- 172.21.15.16:0/2327193628 wait complete. 2024-04-25T10:11:13.361 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-25T10:11:13.499 INFO:teuthology.orchestra.run.smithi186.stdout:{"message":"","return_code":0} 2024-04-25T10:11:13.499 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:11:13.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:11:13.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.596+0000 7f5a0914c640 1 Processor -- start 2024-04-25T10:11:13.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.596+0000 7f5a0914c640 1 -- start start 2024-04-25T10:11:13.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.597+0000 7f5a0914c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a040aedc0 0x7f5a0415ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.597+0000 7f5a0914c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a0406d260 0x7f5a040b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.597+0000 7f5a0914c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a0406da40 con 0x7f5a040b4730 2024-04-25T10:11:13.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.597+0000 7f5a0914c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a040b4ba0 con 0x7f5a0406d260 2024-04-25T10:11:13.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.597+0000 7f5a0914c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0415c050 con 0x7f5a040aedc0 2024-04-25T10:11:13.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.597+0000 7f5a0259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a0406d260 0x7f5a040b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.597+0000 7f5a0259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a0406d260 0x7f5a040b41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52818/0 (socket says 172.21.15.16:52818) 2024-04-25T10:11:13.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.597+0000 7f5a0259c640 1 -- 172.21.15.16:0/1924118296 learned_addr learned my addr 172.21.15.16:0/1924118296 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:13.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.598+0000 7f5a02d9d640 1 --2- 172.21.15.16:0/1924118296 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a040aedc0 0x7f5a0415ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.598+0000 7f5a0259c640 1 -- 172.21.15.16:0/1924118296 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a040aedc0 msgr2=0x7f5a0415ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.598+0000 7f5a0259c640 1 --2- 172.21.15.16:0/1924118296 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a040aedc0 0x7f5a0415ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.598+0000 7f5a0259c640 1 -- 172.21.15.16:0/1924118296 >> v1:172.21.15.186:6789/0 conn(0x7f5a040b4730 legacy=0x7f5a040b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.598+0000 7f5a0259c640 1 -- 172.21.15.16:0/1924118296 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0406dd20 con 0x7f5a0406d260 2024-04-25T10:11:13.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.598+0000 7f5a0259c640 1 --2- 172.21.15.16:0/1924118296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a0406d260 0x7f5a040b41f0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f59f800bb10 tx=0x7f59f802c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc2111b94de1a79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.599+0000 7f5a01d9b640 1 -- 172.21.15.16:0/1924118296 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8032790 con 0x7f5a0406d260 2024-04-25T10:11:13.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.599+0000 7f5a01d9b640 1 -- 172.21.15.16:0/1924118296 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f8038070 con 0x7f5a0406d260 2024-04-25T10:11:13.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.599+0000 7f5a01d9b640 1 -- 172.21.15.16:0/1924118296 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8032b50 con 0x7f5a0406d260 2024-04-25T10:11:13.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.599+0000 7f5a0914c640 1 -- 172.21.15.16:0/1924118296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a0406d260 msgr2=0x7f5a040b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.599+0000 7f5a0914c640 1 --2- 172.21.15.16:0/1924118296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a0406d260 0x7f5a040b41f0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f59f800bb10 tx=0x7f59f802c890 comp rx=0 tx=0).stop 2024-04-25T10:11:13.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.599+0000 7f5a0914c640 1 -- 172.21.15.16:0/1924118296 shutdown_connections 2024-04-25T10:11:13.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.599+0000 7f5a0914c640 1 --2- 172.21.15.16:0/1924118296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a0406d260 0x7f5a040b41f0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.599+0000 7f5a0914c640 1 --2- 172.21.15.16:0/1924118296 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a040aedc0 0x7f5a0415ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.599+0000 7f5a0914c640 1 -- 172.21.15.16:0/1924118296 >> 172.21.15.16:0/1924118296 conn(0x7f5a0406aba0 msgr2=0x7f5a0406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.599+0000 7f5a0914c640 1 -- 172.21.15.16:0/1924118296 shutdown_connections 2024-04-25T10:11:13.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.599+0000 7f5a0914c640 1 -- 172.21.15.16:0/1924118296 wait complete. 2024-04-25T10:11:13.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.601+0000 7f5a0914c640 1 Processor -- start 2024-04-25T10:11:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.601+0000 7f5a0914c640 1 -- start start 2024-04-25T10:11:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.601+0000 7f5a0914c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a0406d260 0x7f5a040adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.601+0000 7f5a0914c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a040aedc0 0x7f5a040a86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.601+0000 7f5a0914c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a040b4730 0x7f5a040a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.601+0000 7f5a0914c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0415c2a0 con 0x7f5a040aedc0 2024-04-25T10:11:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.601+0000 7f5a0914c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a040af1a0 con 0x7f5a040b4730 2024-04-25T10:11:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.601+0000 7f5a0914c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0406d6c0 con 0x7f5a0406d260 2024-04-25T10:11:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.601+0000 7f5a0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a040aedc0 0x7f5a040a86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.602+0000 7f5a0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a040aedc0 0x7f5a040a86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52824/0 (socket says 172.21.15.16:52824) 2024-04-25T10:11:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.602+0000 7f5a0259c640 1 -- 172.21.15.16:0/2759000729 learned_addr learned my addr 172.21.15.16:0/2759000729 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.602+0000 7f5a02d9d640 1 --2- 172.21.15.16:0/2759000729 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a0406d260 0x7f5a040adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.602+0000 7f5a0359e640 1 --2- 172.21.15.16:0/2759000729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a040b4730 0x7f5a040a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.602+0000 7f5a0259c640 1 -- 172.21.15.16:0/2759000729 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a0406d260 msgr2=0x7f5a040adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.602+0000 7f5a0259c640 1 --2- 172.21.15.16:0/2759000729 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a0406d260 0x7f5a040adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.602+0000 7f5a0259c640 1 -- 172.21.15.16:0/2759000729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a040b4730 msgr2=0x7f5a040a8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.602+0000 7f5a0259c640 1 --2- 172.21.15.16:0/2759000729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a040b4730 0x7f5a040a8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.602+0000 7f5a0259c640 1 -- 172.21.15.16:0/2759000729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a040a9150 con 0x7f5a040aedc0 2024-04-25T10:11:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.602+0000 7f5a02d9d640 1 --2- 172.21.15.16:0/2759000729 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a0406d260 0x7f5a040adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.602+0000 7f5a0259c640 1 --2- 172.21.15.16:0/2759000729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a040aedc0 0x7f5a040a86d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f59f802c8d0 tx=0x7f59f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.602+0000 7f59eb7fe640 1 -- 172.21.15.16:0/2759000729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8033620 con 0x7f5a040aedc0 2024-04-25T10:11:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.602+0000 7f59eb7fe640 1 -- 172.21.15.16:0/2759000729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f80337c0 con 0x7f5a040aedc0 2024-04-25T10:11:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.602+0000 7f59eb7fe640 1 -- 172.21.15.16:0/2759000729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8033bb0 con 0x7f5a040aedc0 2024-04-25T10:11:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.602+0000 7f5a0914c640 1 -- 172.21.15.16:0/2759000729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a040a9400 con 0x7f5a040aedc0 2024-04-25T10:11:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.602+0000 7f5a0914c640 1 -- 172.21.15.16:0/2759000729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a040a99c0 con 0x7f5a040aedc0 2024-04-25T10:11:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.603+0000 7f5a0914c640 1 -- 172.21.15.16:0/2759000729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a040750b0 con 0x7f5a040aedc0 2024-04-25T10:11:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.604+0000 7f59eb7fe640 1 -- 172.21.15.16:0/2759000729 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f59f804d060 con 0x7f5a040aedc0 2024-04-25T10:11:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.604+0000 7f59eb7fe640 1 --2- 172.21.15.16:0/2759000729 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f59f006b4d0 0x7f59f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.604+0000 7f59eb7fe640 1 -- 172.21.15.16:0/2759000729 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59f80acda0 con 0x7f5a040aedc0 2024-04-25T10:11:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.604+0000 7f5a02d9d640 1 --2- 172.21.15.16:0/2759000729 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f59f006b4d0 0x7f59f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.605+0000 7f5a02d9d640 1 --2- 172.21.15.16:0/2759000729 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f59f006b4d0 0x7f59f006d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009ac0 tx=0x7f59f4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.608+0000 7f59eb7fe640 1 -- 172.21.15.16:0/2759000729 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f59f8041070 con 0x7f5a040aedc0 2024-04-25T10:11:13.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.826+0000 7f5a0914c640 1 -- 172.21.15.16:0/2759000729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5a040ceb90 con 0x7f5a040aedc0 2024-04-25T10:11:13.831 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-04-25T10:11:13.826+0000 7ff13f9bb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:11:13.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.828+0000 7f59eb7fe640 1 -- 172.21.15.16:0/2759000729 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v52) v1 ==== 161+0+0 (secure 0 0 0) 0x7f59f807bcb0 con 0x7f5a040aedc0 2024-04-25T10:11:13.833 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:11:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.830+0000 7f5a0914c640 1 -- 172.21.15.16:0/2759000729 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f59f006b4d0 msgr2=0x7f59f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.830+0000 7f5a0914c640 1 --2- 172.21.15.16:0/2759000729 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f59f006b4d0 0x7f59f006d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009ac0 tx=0x7f59f4008040 comp rx=0 tx=0).stop 2024-04-25T10:11:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.830+0000 7f5a0914c640 1 -- 172.21.15.16:0/2759000729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a040aedc0 msgr2=0x7f5a040a86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.830+0000 7f5a0914c640 1 --2- 172.21.15.16:0/2759000729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a040aedc0 0x7f5a040a86d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f59f802c8d0 tx=0x7f59f80046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.830+0000 7f5a0914c640 1 -- 172.21.15.16:0/2759000729 shutdown_connections 2024-04-25T10:11:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.830+0000 7f5a0914c640 1 --2- 172.21.15.16:0/2759000729 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f59f006b4d0 0x7f59f006d990 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.830+0000 7f5a0914c640 1 --2- 172.21.15.16:0/2759000729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a040b4730 0x7f5a040a8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.830+0000 7f5a0914c640 1 --2- 172.21.15.16:0/2759000729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a040aedc0 0x7f5a040a86d0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.830+0000 7f5a0914c640 1 --2- 172.21.15.16:0/2759000729 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a0406d260 0x7f5a040adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.830+0000 7f5a0914c640 1 -- 172.21.15.16:0/2759000729 >> 172.21.15.16:0/2759000729 conn(0x7f5a0406aba0 msgr2=0x7f5a0406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.830+0000 7f5a0914c640 1 -- 172.21.15.16:0/2759000729 shutdown_connections 2024-04-25T10:11:13.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.830+0000 7f5a0914c640 1 -- 172.21.15.16:0/2759000729 wait complete. 2024-04-25T10:11:13.841 INFO:tasks.ceph.mds.b.smithi186.stderr: -4944> 2024-04-25T10:11:02.638+0000 7fa67df9e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:11:13.847 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-04-25T10:11:13.871 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2024-04-25T10:11:13.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.883+0000 7f440112df40 1 Processor -- start 2024-04-25T10:11:13.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.884+0000 7f440112df40 1 -- start start 2024-04-25T10:11:13.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.884+0000 7f440112df40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581f886db00 0x5581f883ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.884+0000 7f440112df40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5581f886dee0 0x5581f8912220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.884+0000 7f440112df40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581f8916e50 0x5581f89031b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.884+0000 7f440112df40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f8913540 con 0x5581f886db00 2024-04-25T10:11:13.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.884+0000 7f440112df40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f87e4df0 con 0x5581f8916e50 2024-04-25T10:11:13.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.884+0000 7f440112df40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581f88855b0 con 0x5581f886dee0 2024-04-25T10:11:13.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.884+0000 7f43fe1c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581f886db00 0x5581f883ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.884+0000 7f43fe1c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581f886db00 0x5581f883ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52842/0 (socket says 172.21.15.16:52842) 2024-04-25T10:11:13.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.884+0000 7f43fe1c7640 1 -- 172.21.15.16:0/4279987291 learned_addr learned my addr 172.21.15.16:0/4279987291 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:13.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.884+0000 7f43fd9c6640 1 --2- 172.21.15.16:0/4279987291 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5581f886dee0 0x5581f8912220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.884+0000 7f43fe9c8640 1 --2- 172.21.15.16:0/4279987291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581f8916e50 0x5581f89031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.885+0000 7f43fe1c7640 1 -- 172.21.15.16:0/4279987291 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5581f886dee0 msgr2=0x5581f8912220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.885+0000 7f43fe1c7640 1 --2- 172.21.15.16:0/4279987291 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5581f886dee0 0x5581f8912220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.885+0000 7f43fe1c7640 1 -- 172.21.15.16:0/4279987291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581f8916e50 msgr2=0x5581f89031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.885+0000 7f43fe1c7640 1 --2- 172.21.15.16:0/4279987291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581f8916e50 0x5581f89031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.885+0000 7f43fe1c7640 1 -- 172.21.15.16:0/4279987291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581f876ac50 con 0x5581f886db00 2024-04-25T10:11:13.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.885+0000 7f43fe1c7640 1 --2- 172.21.15.16:0/4279987291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581f886db00 0x5581f883ebc0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f43f8025c40 tx=0x7f43f8051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69935a886bedb97f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.885+0000 7f43fc9c4640 1 -- 172.21.15.16:0/4279987291 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f805a070 con 0x5581f886db00 2024-04-25T10:11:13.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.885+0000 7f43fc9c4640 1 -- 172.21.15.16:0/4279987291 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f8064070 con 0x5581f886db00 2024-04-25T10:11:13.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.885+0000 7f43fc9c4640 1 -- 172.21.15.16:0/4279987291 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f8055c40 con 0x5581f886db00 2024-04-25T10:11:13.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.885+0000 7f440112df40 1 -- 172.21.15.16:0/4279987291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581f886db00 msgr2=0x5581f883ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.885+0000 7f440112df40 1 --2- 172.21.15.16:0/4279987291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581f886db00 0x5581f883ebc0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f43f8025c40 tx=0x7f43f8051000 comp rx=0 tx=0).stop 2024-04-25T10:11:13.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.885+0000 7f440112df40 1 -- 172.21.15.16:0/4279987291 shutdown_connections 2024-04-25T10:11:13.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.885+0000 7f440112df40 1 --2- 172.21.15.16:0/4279987291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581f8916e50 0x5581f89031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.885+0000 7f440112df40 1 --2- 172.21.15.16:0/4279987291 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5581f886dee0 0x5581f8912220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.885+0000 7f440112df40 1 --2- 172.21.15.16:0/4279987291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581f886db00 0x5581f883ebc0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.885+0000 7f440112df40 1 -- 172.21.15.16:0/4279987291 >> 172.21.15.16:0/4279987291 conn(0x5581f88683d0 msgr2=0x5581f8869040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.886+0000 7f440112df40 1 -- 172.21.15.16:0/4279987291 shutdown_connections 2024-04-25T10:11:13.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.886+0000 7f440112df40 1 -- 172.21.15.16:0/4279987291 wait complete. 2024-04-25T10:11:13.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.886+0000 7f440112df40 1 Processor -- start 2024-04-25T10:11:13.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.886+0000 7f440112df40 1 -- start start 2024-04-25T10:11:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.887+0000 7f440112df40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581f886db00 0x5581f89075a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.887+0000 7f440112df40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581f886dee0 0x5581f88fde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.887+0000 7f440112df40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5581f8916e50 0x5581f886cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.887+0000 7f440112df40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f89109c0 con 0x5581f886db00 2024-04-25T10:11:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.887+0000 7f440112df40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f883f100 con 0x5581f886dee0 2024-04-25T10:11:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.887+0000 7f440112df40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581f89168a0 con 0x5581f8916e50 2024-04-25T10:11:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.887+0000 7f43fe1c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581f886db00 0x5581f89075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.887+0000 7f43fe1c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581f886db00 0x5581f89075a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52854/0 (socket says 172.21.15.16:52854) 2024-04-25T10:11:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.887+0000 7f43fe1c7640 1 -- 172.21.15.16:0/4164912619 learned_addr learned my addr 172.21.15.16:0/4164912619 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.887+0000 7f43fe9c8640 1 --2- 172.21.15.16:0/4164912619 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5581f8916e50 0x5581f886cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.887+0000 7f43fd9c6640 1 --2- 172.21.15.16:0/4164912619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581f886dee0 0x5581f88fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.887+0000 7f43fe1c7640 1 -- 172.21.15.16:0/4164912619 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5581f8916e50 msgr2=0x5581f886cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.887+0000 7f43fe1c7640 1 --2- 172.21.15.16:0/4164912619 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5581f8916e50 0x5581f886cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.887+0000 7f43fe1c7640 1 -- 172.21.15.16:0/4164912619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581f886dee0 msgr2=0x5581f88fde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.887+0000 7f43fe1c7640 1 --2- 172.21.15.16:0/4164912619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581f886dee0 0x5581f88fde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.887+0000 7f43fe1c7640 1 -- 172.21.15.16:0/4164912619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581f88fe3c0 con 0x5581f886db00 2024-04-25T10:11:13.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.887+0000 7f43fe1c7640 1 --2- 172.21.15.16:0/4164912619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581f886db00 0x5581f89075a0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f43f8025c40 tx=0x7f43f8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.888+0000 7f43f77fe640 1 -- 172.21.15.16:0/4164912619 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f805a040 con 0x5581f886db00 2024-04-25T10:11:13.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.888+0000 7f43f77fe640 1 -- 172.21.15.16:0/4164912619 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f805b040 con 0x5581f886db00 2024-04-25T10:11:13.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.888+0000 7f440112df40 1 -- 172.21.15.16:0/4164912619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5581f8914170 con 0x5581f886db00 2024-04-25T10:11:13.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.888+0000 7f43f77fe640 1 -- 172.21.15.16:0/4164912619 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f8055d20 con 0x5581f886db00 2024-04-25T10:11:13.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.888+0000 7f440112df40 1 -- 172.21.15.16:0/4164912619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581f8868bc0 con 0x5581f886db00 2024-04-25T10:11:13.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.889+0000 7f43f77fe640 1 -- 172.21.15.16:0/4164912619 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43f8071020 con 0x5581f886db00 2024-04-25T10:11:13.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.889+0000 7f43f77fe640 1 --2- 172.21.15.16:0/4164912619 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f43e0069eb0 0x7f43e006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.889+0000 7f43f77fe640 1 -- 172.21.15.16:0/4164912619 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f43f80cde20 con 0x5581f886db00 2024-04-25T10:11:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.890+0000 7f43fd9c6640 1 --2- 172.21.15.16:0/4164912619 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f43e0069eb0 0x7f43e006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.890+0000 7f440112df40 1 --2- 172.21.15.16:0/4164912619 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x5581f89ab240 0x5581f886b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.890+0000 7f440112df40 1 -- 172.21.15.16:0/4164912619 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e87) v9 -- 0x5581f886bbb0 con 0x5581f89ab240 2024-04-25T10:11:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.890+0000 7f43fe9c8640 1 --2- 172.21.15.16:0/4164912619 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x5581f89ab240 0x5581f886b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.890+0000 7f43fd9c6640 1 --2- 172.21.15.16:0/4164912619 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f43e0069eb0 0x7f43e006c370 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f43e8000c00 tx=0x7f43e80017f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.890+0000 7f43fe9c8640 1 --2- 172.21.15.16:0/4164912619 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x5581f89ab240 0x5581f886b670 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.893+0000 7f43fe9c8640 1 -- 172.21.15.16:0/4164912619 <== osd.6 v2:172.21.15.186:6816/1123185664 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v87'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f43f000d850 con 0x5581f89ab240 2024-04-25T10:11:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.893+0000 7f440112df40 1 -- 172.21.15.16:0/4164912619 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x5581f89ab240 msgr2=0x5581f886b670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.893+0000 7f440112df40 1 --2- 172.21.15.16:0/4164912619 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x5581f89ab240 0x5581f886b670 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.893+0000 7f440112df40 1 -- 172.21.15.16:0/4164912619 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f43e0069eb0 msgr2=0x7f43e006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.893+0000 7f440112df40 1 --2- 172.21.15.16:0/4164912619 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f43e0069eb0 0x7f43e006c370 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f43e8000c00 tx=0x7f43e80017f0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.893+0000 7f440112df40 1 -- 172.21.15.16:0/4164912619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581f886db00 msgr2=0x5581f89075a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.893+0000 7f440112df40 1 --2- 172.21.15.16:0/4164912619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581f886db00 0x5581f89075a0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f43f8025c40 tx=0x7f43f8002fb0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.893+0000 7f43fe9c8640 1 -- 172.21.15.16:0/4164912619 reap_dead start 2024-04-25T10:11:13.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.893+0000 7f440112df40 1 -- 172.21.15.16:0/4164912619 shutdown_connections 2024-04-25T10:11:13.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.893+0000 7f440112df40 1 -- 172.21.15.16:0/4164912619 >> 172.21.15.16:0/4164912619 conn(0x5581f88683d0 msgr2=0x5581f88ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.894+0000 7f440112df40 1 -- 172.21.15.16:0/4164912619 shutdown_connections 2024-04-25T10:11:13.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.894+0000 7f440112df40 1 -- 172.21.15.16:0/4164912619 wait complete. 2024-04-25T10:11:13.902 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-04-25T10:11:13.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.977+0000 7fb9be80e280 1 -- start start 2024-04-25T10:11:13.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.977+0000 7fb9be80e280 1 Processor -- start 2024-04-25T10:11:13.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9be80e280 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b084b50c00 0x55b084bc0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9be80e280 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b084b51800 0x55b084bc0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9be80e280 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b084b51c00 0x55b084bc1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9be80e280 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0858e0600 con 0x55b084b50c00 2024-04-25T10:11:13.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9be80e280 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0858e0480 con 0x55b084b51800 2024-04-25T10:11:13.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9be80e280 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0858e0300 con 0x55b084b51c00 2024-04-25T10:11:13.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9bc26d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b084b50c00 0x55b084bc0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9bc26d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b084b50c00 0x55b084bc0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52866/0 (socket says 172.21.15.16:52866) 2024-04-25T10:11:13.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9bc26d640 1 -- 172.21.15.16:0/1326709471 learned_addr learned my addr 172.21.15.16:0/1326709471 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:13.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9bba6c640 1 --2- 172.21.15.16:0/1326709471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b084b51800 0x55b084bc0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9bca6e640 1 --2- 172.21.15.16:0/1326709471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b084b51c00 0x55b084bc1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9bc26d640 1 -- 172.21.15.16:0/1326709471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b084b51c00 msgr2=0x55b084bc1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9bc26d640 1 --2- 172.21.15.16:0/1326709471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b084b51c00 0x55b084bc1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9bc26d640 1 -- 172.21.15.16:0/1326709471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b084b51800 msgr2=0x55b084bc0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9bc26d640 1 --2- 172.21.15.16:0/1326709471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b084b51800 0x55b084bc0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9bc26d640 1 -- 172.21.15.16:0/1326709471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b084b86780 con 0x55b084b50c00 2024-04-25T10:11:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9bc26d640 1 --2- 172.21.15.16:0/1326709471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b084b50c00 0x55b084bc0580 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x55b08596b350 tx=0x55b085a08540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5294af43f4e3351 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9b9a68640 1 -- 172.21.15.16:0/1326709471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b084b76680 con 0x55b084b50c00 2024-04-25T10:11:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9b9a68640 1 -- 172.21.15.16:0/1326709471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b084b7ce00 con 0x55b084b50c00 2024-04-25T10:11:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9b9a68640 1 -- 172.21.15.16:0/1326709471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b084b76b60 con 0x55b084b50c00 2024-04-25T10:11:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9be80e280 1 -- 172.21.15.16:0/1326709471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b084b874a0 con 0x55b084b50c00 2024-04-25T10:11:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9be80e280 1 -- 172.21.15.16:0/1326709471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b084b86000 con 0x55b084b50c00 2024-04-25T10:11:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.978+0000 7fb9be80e280 4 waiting for MDS map... 2024-04-25T10:11:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.979+0000 7fb9b9a68640 1 -- 172.21.15.16:0/1326709471 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x55b085a20600 con 0x55b084b50c00 2024-04-25T10:11:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.979+0000 7fb9b9a68640 1 -- 172.21.15.16:0/1326709471 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 52) v1 ==== 2723+0+0 (secure 0 0 0) 0x55b084bab180 con 0x55b084b50c00 2024-04-25T10:11:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.979+0000 7fb9be80e280 4 Got MDS map 52 2024-04-25T10:11:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.979+0000 7fb9be80e280 10 main: JournalTool::main 2024-04-25T10:11:13.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.979+0000 7fb9be80e280 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:11:13.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.979+0000 7fb9be80e280 1 Processor -- start 2024-04-25T10:11:13.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.979+0000 7fb9be80e280 1 -- start start 2024-04-25T10:11:13.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9be80e280 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b085923000 0x55b084bc1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9be80e280 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b085923400 0x55b084bc1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9be80e280 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b085923800 0x55b084bc2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9be80e280 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0858e0180 con 0x55b085923400 2024-04-25T10:11:13.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9be80e280 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b084bb2480 con 0x55b085923800 2024-04-25T10:11:13.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9be80e280 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b084bb2900 con 0x55b085923000 2024-04-25T10:11:13.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9bc26d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b085923400 0x55b084bc1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9bc26d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b085923400 0x55b084bc1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52874/0 (socket says 172.21.15.16:52874) 2024-04-25T10:11:13.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9bc26d640 1 -- 172.21.15.16:0/1004844898 learned_addr learned my addr 172.21.15.16:0/1004844898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:13.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9bca6e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b085923000 0x55b084bc1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9bba6c640 1 --2- 172.21.15.16:0/1004844898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b085923800 0x55b084bc2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9bc26d640 1 -- 172.21.15.16:0/1004844898 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b085923000 msgr2=0x55b084bc1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9bc26d640 1 --2- 172.21.15.16:0/1004844898 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b085923000 0x55b084bc1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9bc26d640 1 -- 172.21.15.16:0/1004844898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b085923800 msgr2=0x55b084bc2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9bc26d640 1 --2- 172.21.15.16:0/1004844898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b085923800 0x55b084bc2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9bc26d640 1 -- 172.21.15.16:0/1004844898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b084b863c0 con 0x55b085923400 2024-04-25T10:11:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9bba6c640 1 --2- 172.21.15.16:0/1004844898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b085923800 0x55b084bc2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9bc26d640 1 --2- 172.21.15.16:0/1004844898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b085923400 0x55b084bc1b80 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x55b085a28630 tx=0x55b085a08060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca07c968626beac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9b7a64640 1 -- 172.21.15.16:0/1004844898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b084b76820 con 0x55b085923400 2024-04-25T10:11:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9b7a64640 1 -- 172.21.15.16:0/1004844898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b084b7ca80 con 0x55b085923400 2024-04-25T10:11:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.980+0000 7fb9b7a64640 1 -- 172.21.15.16:0/1004844898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b084b76680 con 0x55b085923400 2024-04-25T10:11:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.981+0000 7fb9be80e280 1 -- 172.21.15.16:0/1004844898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b085923400 msgr2=0x55b084bc1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.981+0000 7fb9be80e280 1 --2- 172.21.15.16:0/1004844898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b085923400 0x55b084bc1b80 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x55b085a28630 tx=0x55b085a08060 comp rx=0 tx=0).stop 2024-04-25T10:11:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.981+0000 7fb9be80e280 1 -- 172.21.15.16:0/1004844898 shutdown_connections 2024-04-25T10:11:13.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.981+0000 7fb9be80e280 1 --2- 172.21.15.16:0/1004844898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b085923800 0x55b084bc2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.981+0000 7fb9be80e280 1 --2- 172.21.15.16:0/1004844898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b085923400 0x55b084bc1b80 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.981+0000 7fb9be80e280 1 --2- 172.21.15.16:0/1004844898 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b085923000 0x55b084bc1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.981+0000 7fb9be80e280 1 -- 172.21.15.16:0/1004844898 >> 172.21.15.16:0/1004844898 conn(0x55b085922c00 msgr2=0x55b084c36800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.981+0000 7fb9be80e280 1 -- 172.21.15.16:0/1004844898 shutdown_connections 2024-04-25T10:11:13.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.981+0000 7fb9be80e280 1 -- 172.21.15.16:0/1004844898 wait complete. 2024-04-25T10:11:13.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.981+0000 7fb9be80e280 1 Processor -- start 2024-04-25T10:11:13.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.982+0000 7fb9be80e280 1 -- start start 2024-04-25T10:11:13.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.982+0000 7fb9be80e280 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b085923000 0x55b084bc3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.982+0000 7fb9be80e280 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b085923400 0x55b084bc2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.982+0000 7fb9be80e280 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b085923800 0x55b084bc2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.982+0000 7fb9be80e280 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0858e1380 con 0x55b085923400 2024-04-25T10:11:13.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.982+0000 7fb9be80e280 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0858e1500 con 0x55b085923000 2024-04-25T10:11:13.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.982+0000 7fb9be80e280 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0858e0180 con 0x55b085923800 2024-04-25T10:11:13.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.982+0000 7fb9bc26d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b085923400 0x55b084bc2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.982+0000 7fb9bc26d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b085923400 0x55b084bc2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52886/0 (socket says 172.21.15.16:52886) 2024-04-25T10:11:13.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.982+0000 7fb9bc26d640 1 -- 172.21.15.16:0/233812449 learned_addr learned my addr 172.21.15.16:0/233812449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:13.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.982+0000 7fb9bba6c640 1 --2- 172.21.15.16:0/233812449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b085923800 0x55b084bc2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.982+0000 7fb9bca6e640 1 --2- 172.21.15.16:0/233812449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b085923000 0x55b084bc3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.982+0000 7fb9bc26d640 1 -- 172.21.15.16:0/233812449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b085923800 msgr2=0x55b084bc2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.982+0000 7fb9bc26d640 1 --2- 172.21.15.16:0/233812449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b085923800 0x55b084bc2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.982+0000 7fb9bc26d640 1 -- 172.21.15.16:0/233812449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b085923000 msgr2=0x55b084bc3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.982+0000 7fb9bc26d640 1 --2- 172.21.15.16:0/233812449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b085923000 0x55b084bc3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.982+0000 7fb9bc26d640 1 -- 172.21.15.16:0/233812449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b084b86780 con 0x55b085923400 2024-04-25T10:11:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.983+0000 7fb9bc26d640 1 --2- 172.21.15.16:0/233812449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b085923400 0x55b084bc2c00 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x55b085a288a0 tx=0x55b085a7a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.983+0000 7fb9b6261640 1 -- 172.21.15.16:0/233812449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b084b76b60 con 0x55b085923400 2024-04-25T10:11:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.983+0000 7fb9b6261640 1 -- 172.21.15.16:0/233812449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b084b7cc40 con 0x55b085923400 2024-04-25T10:11:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.983+0000 7fb9be80e280 1 -- 172.21.15.16:0/233812449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b084b874a0 con 0x55b085923400 2024-04-25T10:11:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.983+0000 7fb9b6261640 1 -- 172.21.15.16:0/233812449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b084b764e0 con 0x55b085923400 2024-04-25T10:11:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.983+0000 7fb9be80e280 1 -- 172.21.15.16:0/233812449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b084b86000 con 0x55b085923400 2024-04-25T10:11:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.983+0000 7fb9be80e280 4 main: JournalTool: resolving pool 12 2024-04-25T10:11:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.984+0000 7fb9b6261640 1 -- 172.21.15.16:0/233812449 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x55b085a7cc00 con 0x55b085923400 2024-04-25T10:11:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.984+0000 7fb9b6261640 1 --2- 172.21.15.16:0/233812449 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55b084eba000 0x55b084bc3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.984+0000 7fb9b6261640 1 -- 172.21.15.16:0/233812449 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x55b085a96600 con 0x55b085923400 2024-04-25T10:11:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.984+0000 7fb9be80e280 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:11:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.985+0000 7fb9be80e280 4 main: Executing for rank 0 2024-04-25T10:11:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.985+0000 7fb9bca6e640 1 --2- 172.21.15.16:0/233812449 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55b084eba000 0x55b084bc3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.985+0000 7fb9bca6e640 1 --2- 172.21.15.16:0/233812449 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55b084eba000 0x55b084bc3700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55b0858d0840 tx=0x55b084ec8c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.985+0000 7fb9be80e280 1 -- start start 2024-04-25T10:11:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.985+0000 7fb9be80e280 1 Processor -- start 2024-04-25T10:11:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.986+0000 7fb9be80e280 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b084ebb400 0x55b084bc2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.986+0000 7fb9be80e280 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b084ebb800 0x55b084bc1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.986+0000 7fb9be80e280 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b084ebbc00 0x55b084bc1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.986+0000 7fb9be80e280 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0858e1800 con 0x55b084ebb800 2024-04-25T10:11:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.986+0000 7fb9be80e280 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0858e1980 con 0x55b084ebbc00 2024-04-25T10:11:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.986+0000 7fb9be80e280 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b084bb2900 con 0x55b084ebb400 2024-04-25T10:11:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.986+0000 7fb9bc26d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b084ebb800 0x55b084bc1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.986+0000 7fb9bc26d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b084ebb800 0x55b084bc1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52896/0 (socket says 172.21.15.16:52896) 2024-04-25T10:11:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.986+0000 7fb9bc26d640 1 -- 172.21.15.16:0/2751743165 learned_addr learned my addr 172.21.15.16:0/2751743165 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.986+0000 7fb9bca6e640 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b084ebb400 0x55b084bc2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.986+0000 7fb9bba6c640 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b084ebbc00 0x55b084bc1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.986+0000 7fb9bc26d640 1 -- 172.21.15.16:0/2751743165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b084ebb400 msgr2=0x55b084bc2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.986+0000 7fb9bc26d640 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b084ebb400 0x55b084bc2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.986+0000 7fb9bc26d640 1 -- 172.21.15.16:0/2751743165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b084ebbc00 msgr2=0x55b084bc1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.986+0000 7fb9bc26d640 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b084ebbc00 0x55b084bc1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.986+0000 7fb9bc26d640 1 -- 172.21.15.16:0/2751743165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b084b86960 con 0x55b084ebb800 2024-04-25T10:11:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.986+0000 7fb9bc26d640 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b084ebb800 0x55b084bc1b80 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x55b084e856b0 tx=0x55b085a7a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e5e787a1f0e820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.987+0000 7fb9b325b640 1 -- 172.21.15.16:0/2751743165 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b084e4c4e0 con 0x55b084ebb800 2024-04-25T10:11:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.987+0000 7fb9b325b640 1 -- 172.21.15.16:0/2751743165 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b084b7c1c0 con 0x55b084ebb800 2024-04-25T10:11:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.987+0000 7fb9b325b640 1 -- 172.21.15.16:0/2751743165 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b084e4c680 con 0x55b084ebb800 2024-04-25T10:11:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.987+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b084b863c0 con 0x55b084ebb800 2024-04-25T10:11:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.987+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b084b86780 con 0x55b084ebb800 2024-04-25T10:11:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.987+0000 7fb9be80e280 4 waiting for MDS map... 2024-04-25T10:11:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.987+0000 7fb9b325b640 1 -- 172.21.15.16:0/2751743165 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x55b084f1e400 con 0x55b084ebb800 2024-04-25T10:11:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.987+0000 7fb9b325b640 1 -- 172.21.15.16:0/2751743165 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 52) v1 ==== 2723+0+0 (secure 0 0 0) 0x55b084bab880 con 0x55b084ebb800 2024-04-25T10:11:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.987+0000 7fb9be80e280 4 Got MDS map 52 2024-04-25T10:11:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.987+0000 7fb9be80e280 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55b084f1d800 0x55b084f38000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.987+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) v9 -- 0x55b084f1dc00 con 0x55b084f1d800 2024-04-25T10:11:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.987+0000 7fb9bca6e640 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55b084f1d800 0x55b084f38000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.987+0000 7fb9bca6e640 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55b084f1d800 0x55b084f38000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=1d9584580ff1eb15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.988+0000 7fb9bca6e640 1 -- 172.21.15.16:0/2751743165 <== osd.3 v2:172.21.15.16:6824/256577261 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x55b084c4c000 con 0x55b084f1d800 2024-04-25T10:11:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.988+0000 7fb9be80e280 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55b084f40000 0x55b084f38580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.988+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:2 12.14 12:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) v9 -- 0x55b084f40400 con 0x55b084f40000 2024-04-25T10:11:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.988+0000 7fb9bba6c640 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55b084f40000 0x55b084f38580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.988+0000 7fb9bba6c640 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55b084f40000 0x55b084f38580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=c0b1271d5a3c7b86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.990+0000 7fb9bba6c640 1 -- 172.21.15.16:0/2751743165 <== osd.0 v2:172.21.15.16:6816/2167750256 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x55b084c4c500 con 0x55b084f40000 2024-04-25T10:11:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.990+0000 7fb9b1a58640 1 -- 172.21.15.16:0/2751743165 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:3 12.14 12:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e87) v9 -- 0x55b085927400 con 0x55b084f40000 2024-04-25T10:11:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.990+0000 7fb9b1a58640 1 -- 172.21.15.16:0/2751743165 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:4 12.1b 12:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e87) v9 -- 0x55b085927000 con 0x55b084f40000 2024-04-25T10:11:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.991+0000 7fb9bba6c640 1 -- 172.21.15.16:0/2751743165 <== osd.0 v2:172.21.15.16:6816/2167750256 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x55b084c4c500 con 0x55b084f40000 2024-04-25T10:11:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.991+0000 7fb9bba6c640 1 -- 172.21.15.16:0/2751743165 <== osd.0 v2:172.21.15.16:6816/2167750256 3 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b084c4c500 con 0x55b084f40000 2024-04-25T10:11:14.004 INFO:teuthology.orchestra.run.smithi016.stdout:old journal was 4212724~2605 2024-04-25T10:11:14.004 INFO:teuthology.orchestra.run.smithi016.stdout:new journal start will be 8388608 (4173279 bytes past old end) 2024-04-25T10:11:14.004 INFO:teuthology.orchestra.run.smithi016.stdout:writing journal head 2024-04-25T10:11:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.991+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:5 12.14 12:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) v9 -- 0x55b084f40800 con 0x55b084f40000 2024-04-25T10:11:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.992+0000 7fb9bba6c640 1 -- 172.21.15.16:0/2751743165 <== osd.0 v2:172.21.15.16:6816/2167750256 4 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v87'21 uv21 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55b084c4c500 con 0x55b084f40000 2024-04-25T10:11:14.005 INFO:teuthology.orchestra.run.smithi016.stdout:writing EResetJournal entry 2024-04-25T10:11:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.992+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:6 12.1b 12:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) v9 -- 0x55b084f40c00 con 0x55b084f40000 2024-04-25T10:11:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.992+0000 7fb9be80e280 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55b084f41000 0x55b084f38b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.992+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:7 12.b 12:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) v9 -- 0x55b084f41400 con 0x55b084f41000 2024-04-25T10:11:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.992+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:8 12.1f 12:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) v9 -- 0x55b084f41800 con 0x55b084f1d800 2024-04-25T10:11:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.992+0000 7fb9be80e280 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55b084f40400 0x55b084f39080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.992+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:9 12.1a 12:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) v9 -- 0x55b084f41c00 con 0x55b084f40400 2024-04-25T10:11:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.992+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:10 12.1e 12:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) v9 -- 0x55b085926c00 con 0x55b084f40400 2024-04-25T10:11:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.992+0000 7fb9be80e280 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55b084f4a000 0x55b084f39600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.992+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:11 12.1d 12:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) v9 -- 0x55b084f4a400 con 0x55b084f4a000 2024-04-25T10:11:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.992+0000 7fb9bca6e640 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55b084f41000 0x55b084f38b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.992+0000 7fb9bc26d640 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55b084f40400 0x55b084f39080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.992+0000 7fb9bba6c640 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55b084f4a000 0x55b084f39600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.993+0000 7fb9bca6e640 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55b084f41000 0x55b084f38b00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=daa2926363eaac04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.993+0000 7fb9bc26d640 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55b084f40400 0x55b084f39080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=f30569b104a9eb94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.993+0000 7fb9bba6c640 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55b084f4a000 0x55b084f39600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=a91a221bc104a918 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.993+0000 7fb9bba6c640 1 -- 172.21.15.16:0/2751743165 <== osd.0 v2:172.21.15.16:6816/2167750256 5 ==== osd_op_reply(6 200.00000002 [delete] v87'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b084c4c500 con 0x55b084f40000 2024-04-25T10:11:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.993+0000 7fb9bca6e640 1 -- 172.21.15.16:0/2751743165 <== osd.3 v2:172.21.15.16:6824/256577261 2 ==== osd_op_reply(8 200.00000004 [delete] v87'7 uv6 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b084c4c000 con 0x55b084f1d800 2024-04-25T10:11:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.994+0000 7fb9bca6e640 1 -- 172.21.15.16:0/2751743165 <== osd.7 v2:172.21.15.186:6808/1627307043 1 ==== osd_op_reply(7 200.00000003 [delete] v87'4 uv3 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b084c4c000 con 0x55b084f41000 2024-04-25T10:11:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.994+0000 7fb9b1a58640 1 -- 172.21.15.16:0/2751743165 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:12 12.1b 12:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) v9 -- 0x55b085926800 con 0x55b084f40000 2024-04-25T10:11:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.994+0000 7fb9bba6c640 1 -- 172.21.15.16:0/2751743165 <== osd.6 v2:172.21.15.186:6816/1123185664 1 ==== osd_op_reply(11 200.00000007 [delete] v87'3 uv2 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b084c4c500 con 0x55b084f4a000 2024-04-25T10:11:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9bba6c640 1 -- 172.21.15.16:0/2751743165 <== osd.0 v2:172.21.15.16:6816/2167750256 6 ==== osd_op_reply(12 200.00000002 [write 0~48] v87'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55b084c4c500 con 0x55b084f40000 2024-04-25T10:11:14.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9bc26d640 1 -- 172.21.15.16:0/2751743165 <== osd.2 v2:172.21.15.16:6800/2094356376 1 ==== osd_op_reply(9 200.00000005 [delete] v87'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b084c4c780 con 0x55b084f40400 2024-04-25T10:11:14.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9bc26d640 1 -- 172.21.15.16:0/2751743165 <== osd.2 v2:172.21.15.16:6800/2094356376 2 ==== osd_op_reply(10 200.00000006 [delete] v87'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b084c4c780 con 0x55b084f40400 2024-04-25T10:11:14.009 INFO:teuthology.orchestra.run.smithi016.stdout:done 2024-04-25T10:11:14.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55b084f40000 msgr2=0x55b084f38580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9be80e280 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55b084f40000 0x55b084f38580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55b084f40400 msgr2=0x55b084f39080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9be80e280 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55b084f40400 0x55b084f39080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55b084f1d800 msgr2=0x55b084f38000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9be80e280 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55b084f1d800 0x55b084f38000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55b084f4a000 msgr2=0x55b084f39600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9be80e280 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55b084f4a000 0x55b084f39600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55b084f41000 msgr2=0x55b084f38b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9be80e280 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55b084f41000 0x55b084f38b00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9bba6c640 1 -- 172.21.15.16:0/2751743165 reap_dead start 2024-04-25T10:11:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9bba6c640 1 -- 172.21.15.16:0/2751743165 reap_dead start 2024-04-25T10:11:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b084ebb800 msgr2=0x55b084bc1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9be80e280 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b084ebb800 0x55b084bc1b80 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x55b084e856b0 tx=0x55b085a7a8a0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 shutdown_connections 2024-04-25T10:11:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9be80e280 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55b084f41000 0x55b084f38b00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9be80e280 1 --2- 172.21.15.16:0/2751743165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b084ebb800 0x55b084bc1b80 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 >> 172.21.15.16:0/2751743165 conn(0x55b084ebb000 msgr2=0x55b084c36000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 shutdown_connections 2024-04-25T10:11:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.996+0000 7fb9be80e280 1 -- 172.21.15.16:0/2751743165 wait complete. 2024-04-25T10:11:14.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 -- 172.21.15.16:0/233812449 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55b084eba000 msgr2=0x55b084bc3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 --2- 172.21.15.16:0/233812449 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55b084eba000 0x55b084bc3700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55b0858d0840 tx=0x55b084ec8c00 comp rx=0 tx=0).stop 2024-04-25T10:11:14.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 -- 172.21.15.16:0/233812449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b085923400 msgr2=0x55b084bc2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 --2- 172.21.15.16:0/233812449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b085923400 0x55b084bc2c00 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x55b085a288a0 tx=0x55b085a7a300 comp rx=0 tx=0).stop 2024-04-25T10:11:14.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 -- 172.21.15.16:0/233812449 shutdown_connections 2024-04-25T10:11:14.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 --2- 172.21.15.16:0/233812449 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55b084eba000 0x55b084bc3700 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 --2- 172.21.15.16:0/233812449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b085923800 0x55b084bc2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 --2- 172.21.15.16:0/233812449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b085923400 0x55b084bc2c00 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 --2- 172.21.15.16:0/233812449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b085923000 0x55b084bc3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 -- 172.21.15.16:0/233812449 >> 172.21.15.16:0/233812449 conn(0x55b085922c00 msgr2=0x55b084c36800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 -- 172.21.15.16:0/233812449 shutdown_connections 2024-04-25T10:11:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 -- 172.21.15.16:0/233812449 wait complete. 2024-04-25T10:11:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 -- 172.21.15.16:0/1326709471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b084b50c00 msgr2=0x55b084bc0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 --2- 172.21.15.16:0/1326709471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b084b50c00 0x55b084bc0580 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x55b08596b350 tx=0x55b085a08540 comp rx=0 tx=0).stop 2024-04-25T10:11:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 -- 172.21.15.16:0/1326709471 shutdown_connections 2024-04-25T10:11:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 --2- 172.21.15.16:0/1326709471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b084b51c00 0x55b084bc1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 --2- 172.21.15.16:0/1326709471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b084b51800 0x55b084bc0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 --2- 172.21.15.16:0/1326709471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b084b50c00 0x55b084bc0580 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 -- 172.21.15.16:0/1326709471 >> 172.21.15.16:0/1326709471 conn(0x55b084b51000 msgr2=0x55b084c37000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 -- 172.21.15.16:0/1326709471 shutdown_connections 2024-04-25T10:11:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:13.997+0000 7fb9be80e280 1 -- 172.21.15.16:0/1326709471 wait complete. 2024-04-25T10:11:14.016 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.114652, result: old journal was 4212724~2605 new journal start will be 8388608 (4173279 bytes past old end) writing journal head writing EResetJournal entry done 2024-04-25T10:11:14.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:11:14.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.152+0000 7ff5e16b5640 1 Processor -- start 2024-04-25T10:11:14.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.152+0000 7ff5e16b5640 1 -- start start 2024-04-25T10:11:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.153+0000 7ff5e16b5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5dc15dca0 0x7ff5dc0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.153+0000 7ff5e16b5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff5dc0d1130 0x7ff5dc0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.153+0000 7ff5e16b5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5dc0d4090 con 0x7ff5dc0d3b90 2024-04-25T10:11:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.153+0000 7ff5e16b5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5dc0b6b20 con 0x7ff5dc0d1130 2024-04-25T10:11:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.153+0000 7ff5e16b5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc0d4370 con 0x7ff5dc15dca0 2024-04-25T10:11:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.153+0000 7ff5da7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff5dc0d3b90 0x7ff5dc0d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42112/0 (socket says 172.21.15.16:42112) 2024-04-25T10:11:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.153+0000 7ff5da7fc640 1 -- 172.21.15.16:0/438634886 learned_addr learned my addr 172.21.15.16:0/438634886 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.153+0000 7ff5db7fe640 1 --2- 172.21.15.16:0/438634886 >> v2:172.21.15.186:3300/0 conn(0x7ff5dc0d1130 0x7ff5dc0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.153+0000 7ff5daffd640 1 --2- 172.21.15.16:0/438634886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5dc15dca0 0x7ff5dc0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.153+0000 7ff5d9ffb640 1 -- 172.21.15.16:0/438634886 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 547554461 0 0) 0x7ff5dc0d4090 con 0x7ff5dc0d3b90 2024-04-25T10:11:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.153+0000 7ff5d9ffb640 1 -- 172.21.15.16:0/438634886 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5c4003520 con 0x7ff5dc0d3b90 2024-04-25T10:11:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.153+0000 7ff5db7fe640 1 -- 172.21.15.16:0/438634886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5dc15dca0 msgr2=0x7ff5dc0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.153+0000 7ff5d9ffb640 1 -- 172.21.15.16:0/438634886 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1631477498 0 0) 0x7ff5c4003520 con 0x7ff5dc0d3b90 2024-04-25T10:11:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.153+0000 7ff5db7fe640 1 --2- 172.21.15.16:0/438634886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5dc15dca0 0x7ff5dc0ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.153+0000 7ff5db7fe640 1 -- 172.21.15.16:0/438634886 >> v1:172.21.15.16:6789/0 conn(0x7ff5dc0d3b90 legacy=0x7ff5dc0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.153+0000 7ff5db7fe640 1 -- 172.21.15.16:0/438634886 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5dc0d4650 con 0x7ff5dc0d1130 2024-04-25T10:11:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.154+0000 7ff5db7fe640 1 --2- 172.21.15.16:0/438634886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5dc0d1130 0x7ff5dc0b2500 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc009d90 tx=0x7ff5cc02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d165c6459c08f3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.154+0000 7ff5d9ffb640 1 -- 172.21.15.16:0/438634886 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0001670 con 0x7ff5dc0d1130 2024-04-25T10:11:14.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.154+0000 7ff5d9ffb640 1 -- 172.21.15.16:0/438634886 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5cc007540 con 0x7ff5dc0d1130 2024-04-25T10:11:14.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.154+0000 7ff5d9ffb640 1 -- 172.21.15.16:0/438634886 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc007910 con 0x7ff5dc0d1130 2024-04-25T10:11:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.155+0000 7ff5e16b5640 1 -- 172.21.15.16:0/438634886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5dc0d1130 msgr2=0x7ff5dc0b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.155+0000 7ff5e16b5640 1 --2- 172.21.15.16:0/438634886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5dc0d1130 0x7ff5dc0b2500 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc009d90 tx=0x7ff5cc02a5f0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.155+0000 7ff5e16b5640 1 -- 172.21.15.16:0/438634886 shutdown_connections 2024-04-25T10:11:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.155+0000 7ff5e16b5640 1 --2- 172.21.15.16:0/438634886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5dc0d1130 0x7ff5dc0b2500 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.155+0000 7ff5e16b5640 1 --2- 172.21.15.16:0/438634886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5dc15dca0 0x7ff5dc0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.155+0000 7ff5e16b5640 1 -- 172.21.15.16:0/438634886 >> 172.21.15.16:0/438634886 conn(0x7ff5dc0726c0 msgr2=0x7ff5dc072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.155+0000 7ff5e16b5640 1 -- 172.21.15.16:0/438634886 shutdown_connections 2024-04-25T10:11:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.155+0000 7ff5e16b5640 1 -- 172.21.15.16:0/438634886 wait complete. 2024-04-25T10:11:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.156+0000 7ff5e16b5640 1 Processor -- start 2024-04-25T10:11:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.156+0000 7ff5e16b5640 1 -- start start 2024-04-25T10:11:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.157+0000 7ff5e16b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5dc0d1130 0x7ff5dc15b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.157+0000 7ff5e16b5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5dc0d3b90 0x7ff5dc0cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.157+0000 7ff5e16b5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5dc15dca0 0x7ff5dc0cd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.157+0000 7ff5e16b5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc15e200 con 0x7ff5dc0d1130 2024-04-25T10:11:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.157+0000 7ff5e16b5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc15e080 con 0x7ff5dc0d3b90 2024-04-25T10:11:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.157+0000 7ff5e16b5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc0cee80 con 0x7ff5dc15dca0 2024-04-25T10:11:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.157+0000 7ff5daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5dc0d1130 0x7ff5dc15b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.157+0000 7ff5daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5dc0d1130 0x7ff5dc15b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52904/0 (socket says 172.21.15.16:52904) 2024-04-25T10:11:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.157+0000 7ff5daffd640 1 -- 172.21.15.16:0/2351914869 learned_addr learned my addr 172.21.15.16:0/2351914869 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.157+0000 7ff5daffd640 1 -- 172.21.15.16:0/2351914869 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5dc15dca0 msgr2=0x7ff5dc0cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.157+0000 7ff5daffd640 1 --2- 172.21.15.16:0/2351914869 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5dc15dca0 0x7ff5dc0cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.157+0000 7ff5daffd640 1 -- 172.21.15.16:0/2351914869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5dc0d3b90 msgr2=0x7ff5dc0cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.157+0000 7ff5daffd640 1 --2- 172.21.15.16:0/2351914869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5dc0d3b90 0x7ff5dc0cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.157+0000 7ff5daffd640 1 -- 172.21.15.16:0/2351914869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5dc0aa620 con 0x7ff5dc0d1130 2024-04-25T10:11:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.157+0000 7ff5daffd640 1 --2- 172.21.15.16:0/2351914869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5dc0d1130 0x7ff5dc15b330 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400e870 tx=0x7ff5d400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.157+0000 7ff5c37fe640 1 -- 172.21.15.16:0/2351914869 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4017070 con 0x7ff5dc0d1130 2024-04-25T10:11:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.158+0000 7ff5c37fe640 1 -- 172.21.15.16:0/2351914869 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d400c940 con 0x7ff5dc0d1130 2024-04-25T10:11:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.158+0000 7ff5e16b5640 1 -- 172.21.15.16:0/2351914869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5dc0aa900 con 0x7ff5dc0d1130 2024-04-25T10:11:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.158+0000 7ff5c37fe640 1 -- 172.21.15.16:0/2351914869 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d400cd30 con 0x7ff5dc0d1130 2024-04-25T10:11:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.158+0000 7ff5e16b5640 1 -- 172.21.15.16:0/2351914869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5dc0aaec0 con 0x7ff5dc0d1130 2024-04-25T10:11:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.159+0000 7ff5e16b5640 1 -- 172.21.15.16:0/2351914869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5dc0750b0 con 0x7ff5dc0d1130 2024-04-25T10:11:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.159+0000 7ff5c37fe640 1 -- 172.21.15.16:0/2351914869 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5d401a070 con 0x7ff5dc0d1130 2024-04-25T10:11:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.159+0000 7ff5c37fe640 1 --2- 172.21.15.16:0/2351914869 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff5c806b4d0 0x7ff5c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.160+0000 7ff5c37fe640 1 -- 172.21.15.16:0/2351914869 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff5d408a510 con 0x7ff5dc0d1130 2024-04-25T10:11:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.160+0000 7ff5da7fc640 1 --2- 172.21.15.16:0/2351914869 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff5c806b4d0 0x7ff5c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.160+0000 7ff5da7fc640 1 --2- 172.21.15.16:0/2351914869 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff5c806b4d0 0x7ff5c806d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc0abf00 tx=0x7ff5d0001180 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.164+0000 7ff5c37fe640 1 -- 172.21.15.16:0/2351914869 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5d40592b0 con 0x7ff5dc0d1130 2024-04-25T10:11:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.383+0000 7ff5e16b5640 1 -- 172.21.15.16:0/2351914869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7ff5dc06b1d0 con 0x7ff5dc0d1130 2024-04-25T10:11:14.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.835+0000 7ff5c37fe640 1 -- 172.21.15.16:0/2351914869 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v53) v1 ==== 162+0+0 (secure 0 0 0) 0x7ff5d4058bb0 con 0x7ff5dc0d1130 2024-04-25T10:11:14.840 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:11:14.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.837+0000 7ff5e16b5640 1 -- 172.21.15.16:0/2351914869 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff5c806b4d0 msgr2=0x7ff5c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.837+0000 7ff5e16b5640 1 --2- 172.21.15.16:0/2351914869 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff5c806b4d0 0x7ff5c806d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc0abf00 tx=0x7ff5d0001180 comp rx=0 tx=0).stop 2024-04-25T10:11:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.837+0000 7ff5e16b5640 1 -- 172.21.15.16:0/2351914869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5dc0d1130 msgr2=0x7ff5dc15b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.837+0000 7ff5e16b5640 1 --2- 172.21.15.16:0/2351914869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5dc0d1130 0x7ff5dc15b330 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400e870 tx=0x7ff5d400ed30 comp rx=0 tx=0).stop 2024-04-25T10:11:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.837+0000 7ff5e16b5640 1 -- 172.21.15.16:0/2351914869 shutdown_connections 2024-04-25T10:11:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.837+0000 7ff5e16b5640 1 --2- 172.21.15.16:0/2351914869 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff5c806b4d0 0x7ff5c806d990 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.837+0000 7ff5e16b5640 1 --2- 172.21.15.16:0/2351914869 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff5dc15dca0 0x7ff5dc0cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.837+0000 7ff5e16b5640 1 --2- 172.21.15.16:0/2351914869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5dc0d3b90 0x7ff5dc0cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.837+0000 7ff5e16b5640 1 --2- 172.21.15.16:0/2351914869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5dc0d1130 0x7ff5dc15b330 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.837+0000 7ff5e16b5640 1 -- 172.21.15.16:0/2351914869 >> 172.21.15.16:0/2351914869 conn(0x7ff5dc0726c0 msgr2=0x7ff5dc0b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.837+0000 7ff5e16b5640 1 -- 172.21.15.16:0/2351914869 shutdown_connections 2024-04-25T10:11:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.837+0000 7ff5e16b5640 1 -- 172.21.15.16:0/2351914869 wait complete. 2024-04-25T10:11:14.857 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.956+0000 7f5e07b70640 1 Processor -- start 2024-04-25T10:11:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.957+0000 7f5e07b70640 1 -- start start 2024-04-25T10:11:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.957+0000 7f5e07b70640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e000d35b0 0x7f5e000b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.958+0000 7f5e07b70640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e000b5990 0x7f5e000b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.958+0000 7f5e07b70640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e0015e0c0 con 0x7f5e0015dce0 2024-04-25T10:11:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.958+0000 7f5e07b70640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e000b5de0 con 0x7f5e000b5990 2024-04-25T10:11:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.958+0000 7f5e07b70640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e000d2510 con 0x7f5e000d35b0 2024-04-25T10:11:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.958+0000 7f5e0610d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e000b5990 0x7f5e000b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.958+0000 7f5e0590c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5e0015dce0 0x7f5e000d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42124/0 (socket says 172.21.15.16:42124) 2024-04-25T10:11:14.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.958+0000 7f5e0590c640 1 -- 172.21.15.16:0/2308144644 learned_addr learned my addr 172.21.15.16:0/2308144644 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:14.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.958+0000 7f5e0610d640 1 -- 172.21.15.16:0/2308144644 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e000d35b0 msgr2=0x7f5e000b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.958+0000 7f5e0610d640 1 --2- 172.21.15.16:0/2308144644 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e000d35b0 0x7f5e000b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.958+0000 7f5e0610d640 1 -- 172.21.15.16:0/2308144644 >> v1:172.21.15.16:6789/0 conn(0x7f5e0015dce0 legacy=0x7f5e000d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.958+0000 7f5e0610d640 1 -- 172.21.15.16:0/2308144644 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e000d27f0 con 0x7f5e000b5990 2024-04-25T10:11:14.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.958+0000 7f5e0490a640 1 -- 172.21.15.16:0/2308144644 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309458302 0 0) 0x7f5e0015e0c0 con 0x7f5e0015dce0 2024-04-25T10:11:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.959+0000 7f5e0610d640 1 --2- 172.21.15.16:0/2308144644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e000b5990 0x7f5e000b6b00 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009d90 tx=0x7f5df802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d024f1cb126c2bac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.959+0000 7f5e0490a640 1 -- 172.21.15.16:0/2308144644 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df8007740 con 0x7f5e000b5990 2024-04-25T10:11:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.959+0000 7f5e0490a640 1 -- 172.21.15.16:0/2308144644 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df80078e0 con 0x7f5e000b5990 2024-04-25T10:11:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.959+0000 7f5e0490a640 1 -- 172.21.15.16:0/2308144644 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df8007cb0 con 0x7f5e000b5990 2024-04-25T10:11:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.959+0000 7f5e07b70640 1 -- 172.21.15.16:0/2308144644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e000b5990 msgr2=0x7f5e000b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.959+0000 7f5e07b70640 1 --2- 172.21.15.16:0/2308144644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e000b5990 0x7f5e000b6b00 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009d90 tx=0x7f5df802a660 comp rx=0 tx=0).stop 2024-04-25T10:11:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.959+0000 7f5e07b70640 1 -- 172.21.15.16:0/2308144644 shutdown_connections 2024-04-25T10:11:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.959+0000 7f5e07b70640 1 --2- 172.21.15.16:0/2308144644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e000b5990 0x7f5e000b6b00 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.959+0000 7f5e07b70640 1 --2- 172.21.15.16:0/2308144644 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e000d35b0 0x7f5e000b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.959+0000 7f5e07b70640 1 -- 172.21.15.16:0/2308144644 >> 172.21.15.16:0/2308144644 conn(0x7f5e00072900 msgr2=0x7f5e000d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.960+0000 7f5e07b70640 1 -- 172.21.15.16:0/2308144644 shutdown_connections 2024-04-25T10:11:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.960+0000 7f5e07b70640 1 -- 172.21.15.16:0/2308144644 wait complete. 2024-04-25T10:11:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.961+0000 7f5e07b70640 1 Processor -- start 2024-04-25T10:11:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.961+0000 7f5e07b70640 1 -- start start 2024-04-25T10:11:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.961+0000 7f5e07b70640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e000b5990 0x7f5e0015fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.961+0000 7f5e07b70640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e000d35b0 0x7f5e00160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.961+0000 7f5e07b70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e0015dce0 0x7f5e00154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.961+0000 7f5e07b70640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e000d3090 con 0x7f5e0015dce0 2024-04-25T10:11:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.961+0000 7f5e07b70640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e000d2db0 con 0x7f5e000b5990 2024-04-25T10:11:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.961+0000 7f5e07b70640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e000d2160 con 0x7f5e000d35b0 2024-04-25T10:11:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.962+0000 7f5e0610d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e0015dce0 0x7f5e00154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.962+0000 7f5e0610d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e0015dce0 0x7f5e00154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52914/0 (socket says 172.21.15.16:52914) 2024-04-25T10:11:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.962+0000 7f5e0610d640 1 -- 172.21.15.16:0/1913729248 learned_addr learned my addr 172.21.15.16:0/1913729248 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.962+0000 7f5e0590c640 1 --2- 172.21.15.16:0/1913729248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e000b5990 0x7f5e0015fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.962+0000 7f5e0610d640 1 -- 172.21.15.16:0/1913729248 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e000d35b0 msgr2=0x7f5e00160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.962+0000 7f5e0510b640 1 --2- 172.21.15.16:0/1913729248 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e000d35b0 0x7f5e00160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.962+0000 7f5e0610d640 1 --2- 172.21.15.16:0/1913729248 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e000d35b0 0x7f5e00160a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.962+0000 7f5e0610d640 1 -- 172.21.15.16:0/1913729248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e000b5990 msgr2=0x7f5e0015fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.962+0000 7f5e0610d640 1 --2- 172.21.15.16:0/1913729248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e000b5990 0x7f5e0015fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.962+0000 7f5e0610d640 1 -- 172.21.15.16:0/1913729248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0015f010 con 0x7f5e0015dce0 2024-04-25T10:11:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.962+0000 7f5e0590c640 1 --2- 172.21.15.16:0/1913729248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e000b5990 0x7f5e0015fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.962+0000 7f5e0610d640 1 --2- 172.21.15.16:0/1913729248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e0015dce0 0x7f5e00154370 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009eb0 tx=0x7f5df8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.962+0000 7f5df67fc640 1 -- 172.21.15.16:0/1913729248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df8040070 con 0x7f5e0015dce0 2024-04-25T10:11:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.962+0000 7f5df67fc640 1 -- 172.21.15.16:0/1913729248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df8031c90 con 0x7f5e0015dce0 2024-04-25T10:11:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.962+0000 7f5e07b70640 1 -- 172.21.15.16:0/1913729248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0015f290 con 0x7f5e0015dce0 2024-04-25T10:11:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.962+0000 7f5df67fc640 1 -- 172.21.15.16:0/1913729248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df8045680 con 0x7f5e0015dce0 2024-04-25T10:11:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.962+0000 7f5e07b70640 1 -- 172.21.15.16:0/1913729248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0015f830 con 0x7f5e0015dce0 2024-04-25T10:11:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.963+0000 7f5e07b70640 1 -- 172.21.15.16:0/1913729248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e000749b0 con 0x7f5e0015dce0 2024-04-25T10:11:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.964+0000 7f5df67fc640 1 -- 172.21.15.16:0/1913729248 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5df8039040 con 0x7f5e0015dce0 2024-04-25T10:11:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.964+0000 7f5df67fc640 1 --2- 172.21.15.16:0/1913729248 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5de406b4d0 0x7f5de406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.964+0000 7f5df67fc640 1 -- 172.21.15.16:0/1913729248 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5df80abbf0 con 0x7f5e0015dce0 2024-04-25T10:11:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.964+0000 7f5e0590c640 1 --2- 172.21.15.16:0/1913729248 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5de406b4d0 0x7f5de406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.965+0000 7f5e0590c640 1 --2- 172.21.15.16:0/1913729248 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5de406b4d0 0x7f5de406d990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc000f60 tx=0x7f5dfc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:14.968+0000 7f5df67fc640 1 -- 172.21.15.16:0/1913729248 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e0015f010 con 0x7f5e0015dce0 2024-04-25T10:11:15.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:15.182+0000 7f5e07b70640 1 -- 172.21.15.16:0/1913729248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e000b6660 con 0x7f5e0015dce0 2024-04-25T10:11:15.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:15.183+0000 7f5df67fc640 1 -- 172.21.15.16:0/1913729248 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f5df807ab60 con 0x7f5e0015dce0 2024-04-25T10:11:15.188 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:15.188 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4044509901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4044509901},{"type":"v1","addr":"172.21.15.16:6835","nonce":4044509901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1494230493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1494230493},{"type":"v1","addr":"172.21.15.186:6837","nonce":1494230493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/84349474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":84349474},{"type":"v1","addr":"172.21.15.16:6834","nonce":84349474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:00.015075+0000","modified":"2024-04-25T10:11:14.837423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"d","rank":0,"incarnation":54,"state":"up:replay","state_seq":1,"addr":"172.21.15.186:6836/3950732718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3950732718},{"type":"v1","addr":"172.21.15.186:6836","nonce":3950732718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:15.189 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 54 2024-04-25T10:11:15.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:15.185+0000 7f5e07b70640 1 -- 172.21.15.16:0/1913729248 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5de406b4d0 msgr2=0x7f5de406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:15.185+0000 7f5e07b70640 1 --2- 172.21.15.16:0/1913729248 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5de406b4d0 0x7f5de406d990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc000f60 tx=0x7f5dfc009040 comp rx=0 tx=0).stop 2024-04-25T10:11:15.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:15.185+0000 7f5e07b70640 1 -- 172.21.15.16:0/1913729248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e0015dce0 msgr2=0x7f5e00154370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:15.185+0000 7f5e07b70640 1 --2- 172.21.15.16:0/1913729248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e0015dce0 0x7f5e00154370 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009eb0 tx=0x7f5df8004a40 comp rx=0 tx=0).stop 2024-04-25T10:11:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:15.185+0000 7f5e07b70640 1 -- 172.21.15.16:0/1913729248 shutdown_connections 2024-04-25T10:11:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:15.185+0000 7f5e07b70640 1 --2- 172.21.15.16:0/1913729248 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5de406b4d0 0x7f5de406d990 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:15.185+0000 7f5e07b70640 1 --2- 172.21.15.16:0/1913729248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e0015dce0 0x7f5e00154370 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:15.185+0000 7f5e07b70640 1 --2- 172.21.15.16:0/1913729248 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e000d35b0 0x7f5e00160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:15.185+0000 7f5e07b70640 1 --2- 172.21.15.16:0/1913729248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e000b5990 0x7f5e0015fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:15.185+0000 7f5e07b70640 1 -- 172.21.15.16:0/1913729248 >> 172.21.15.16:0/1913729248 conn(0x7f5e00072900 msgr2=0x7f5e000b8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:15.185+0000 7f5e07b70640 1 -- 172.21.15.16:0/1913729248 shutdown_connections 2024-04-25T10:11:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:15.185+0000 7f5e07b70640 1 -- 172.21.15.16:0/1913729248 wait complete. 2024-04-25T10:11:15.605 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:00.015075+0000', 'modified': '2024-04-25T10:11:14.837423+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 87, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7335}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7335': {'gid': 7335, 'name': 'd', 'rank': 0, 'incarnation': 54, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.186:6836/3950732718', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 3950732718}, {'type': 'v1', 'addr': '172.21.15.186:6836', 'nonce': 3950732718}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:15.605 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7335:up:replay 2024-04-25T10:11:16.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:16.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.717+0000 7f6926f5c640 1 Processor -- start 2024-04-25T10:11:16.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.717+0000 7f6926f5c640 1 -- start start 2024-04-25T10:11:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.718+0000 7f6926f5c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69200b5b60 0x7f69201687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.718+0000 7f6926f5c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692015e0c0 con 0x7f692015dce0 2024-04-25T10:11:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.718+0000 7f6926f5c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69200d2510 con 0x7f69200d35b0 2024-04-25T10:11:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.718+0000 7f6926f5c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69200b7b10 con 0x7f69200b5b60 2024-04-25T10:11:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.718+0000 7f6924cf8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f692015dce0 0x7f69200d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42138/0 (socket says 172.21.15.16:42138) 2024-04-25T10:11:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.718+0000 7f6924cf8640 1 -- 172.21.15.16:0/3366198981 learned_addr learned my addr 172.21.15.16:0/3366198981 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.718+0000 7f69254f9640 1 --2- 172.21.15.16:0/3366198981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69200b5b60 0x7f69201687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.719+0000 7f691f7fe640 1 -- 172.21.15.16:0/3366198981 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139337623 0 0) 0x7f692015e0c0 con 0x7f692015dce0 2024-04-25T10:11:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.719+0000 7f691f7fe640 1 -- 172.21.15.16:0/3366198981 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6908003520 con 0x7f692015dce0 2024-04-25T10:11:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.719+0000 7f691f7fe640 1 -- 172.21.15.16:0/3366198981 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511095199 0 0) 0x7f69200d2510 con 0x7f69200d35b0 2024-04-25T10:11:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.719+0000 7f691f7fe640 1 -- 172.21.15.16:0/3366198981 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f692015e0c0 con 0x7f69200d35b0 2024-04-25T10:11:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.719+0000 7f69254f9640 1 -- 172.21.15.16:0/3366198981 >> v1:172.21.15.186:6789/0 conn(0x7f69200d35b0 legacy=0x7f69200b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.719+0000 7f69254f9640 1 -- 172.21.15.16:0/3366198981 >> v1:172.21.15.16:6789/0 conn(0x7f692015dce0 legacy=0x7f69200d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.719+0000 7f69254f9640 1 -- 172.21.15.16:0/3366198981 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69200d27f0 con 0x7f69200b5b60 2024-04-25T10:11:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.720+0000 7f69254f9640 1 --2- 172.21.15.16:0/3366198981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69200b5b60 0x7f69201687f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6918009dc0 tx=0x7f691802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97fbe7d0d6735cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.720+0000 7f691f7fe640 1 -- 172.21.15.16:0/3366198981 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918007740 con 0x7f69200b5b60 2024-04-25T10:11:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.720+0000 7f691f7fe640 1 -- 172.21.15.16:0/3366198981 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69180078e0 con 0x7f69200b5b60 2024-04-25T10:11:16.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.720+0000 7f691f7fe640 1 -- 172.21.15.16:0/3366198981 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918007c90 con 0x7f69200b5b60 2024-04-25T10:11:16.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.720+0000 7f6926f5c640 1 -- 172.21.15.16:0/3366198981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69200b5b60 msgr2=0x7f69201687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.720+0000 7f6926f5c640 1 --2- 172.21.15.16:0/3366198981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69200b5b60 0x7f69201687f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6918009dc0 tx=0x7f691802a6d0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.720+0000 7f6926f5c640 1 -- 172.21.15.16:0/3366198981 shutdown_connections 2024-04-25T10:11:16.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.720+0000 7f6926f5c640 1 --2- 172.21.15.16:0/3366198981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69200b5b60 0x7f69201687f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.720+0000 7f6926f5c640 1 -- 172.21.15.16:0/3366198981 >> 172.21.15.16:0/3366198981 conn(0x7f6920072900 msgr2=0x7f69200d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.720+0000 7f6926f5c640 1 -- 172.21.15.16:0/3366198981 shutdown_connections 2024-04-25T10:11:16.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.720+0000 7f6926f5c640 1 -- 172.21.15.16:0/3366198981 wait complete. 2024-04-25T10:11:16.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.722+0000 7f6926f5c640 1 Processor -- start 2024-04-25T10:11:16.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.722+0000 7f6926f5c640 1 -- start start 2024-04-25T10:11:16.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.722+0000 7f6926f5c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69200b5b60 0x7f69200cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.722+0000 7f6926f5c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69200d35b0 0x7f6920160d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.722+0000 7f6926f5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f692015dce0 0x7f6920143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.722+0000 7f6926f5c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69200d3090 con 0x7f692015dce0 2024-04-25T10:11:16.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.722+0000 7f6926f5c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69200d2db0 con 0x7f69200b5b60 2024-04-25T10:11:16.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.722+0000 7f6926f5c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69200d2160 con 0x7f69200d35b0 2024-04-25T10:11:16.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.723+0000 7f69254f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f692015dce0 0x7f6920143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.723+0000 7f69254f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f692015dce0 0x7f6920143c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52924/0 (socket says 172.21.15.16:52924) 2024-04-25T10:11:16.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.723+0000 7f69254f9640 1 -- 172.21.15.16:0/3341399001 learned_addr learned my addr 172.21.15.16:0/3341399001 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:16.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.723+0000 7f6924cf8640 1 --2- 172.21.15.16:0/3341399001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69200b5b60 0x7f69200cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.723+0000 7f69254f9640 1 -- 172.21.15.16:0/3341399001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69200d35b0 msgr2=0x7f6920160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.723+0000 7f691ffff640 1 --2- 172.21.15.16:0/3341399001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69200d35b0 0x7f6920160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.723+0000 7f69254f9640 1 --2- 172.21.15.16:0/3341399001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69200d35b0 0x7f6920160d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.723+0000 7f69254f9640 1 -- 172.21.15.16:0/3341399001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69200b5b60 msgr2=0x7f69200cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.723+0000 7f69254f9640 1 --2- 172.21.15.16:0/3341399001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69200b5b60 0x7f69200cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.723+0000 7f69254f9640 1 -- 172.21.15.16:0/3341399001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69201612a0 con 0x7f692015dce0 2024-04-25T10:11:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.723+0000 7f6924cf8640 1 --2- 172.21.15.16:0/3341399001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69200b5b60 0x7f69200cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.723+0000 7f69254f9640 1 --2- 172.21.15.16:0/3341399001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f692015dce0 0x7f6920143c90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f691802acc0 tx=0x7f6918004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.723+0000 7f691d7fa640 1 -- 172.21.15.16:0/3341399001 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918040070 con 0x7f692015dce0 2024-04-25T10:11:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.723+0000 7f691d7fa640 1 -- 172.21.15.16:0/3341399001 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6918031c40 con 0x7f692015dce0 2024-04-25T10:11:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.723+0000 7f6926f5c640 1 -- 172.21.15.16:0/3341399001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6920153f00 con 0x7f692015dce0 2024-04-25T10:11:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.723+0000 7f691d7fa640 1 -- 172.21.15.16:0/3341399001 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918009450 con 0x7f692015dce0 2024-04-25T10:11:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.723+0000 7f6926f5c640 1 -- 172.21.15.16:0/3341399001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69200cc880 con 0x7f692015dce0 2024-04-25T10:11:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.724+0000 7f6926f5c640 1 -- 172.21.15.16:0/3341399001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69200749b0 con 0x7f692015dce0 2024-04-25T10:11:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.725+0000 7f691d7fa640 1 -- 172.21.15.16:0/3341399001 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6918039040 con 0x7f692015dce0 2024-04-25T10:11:16.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.725+0000 7f691d7fa640 1 --2- 172.21.15.16:0/3341399001 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f690c06b4d0 0x7f690c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.725+0000 7f691d7fa640 1 -- 172.21.15.16:0/3341399001 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f69180abae0 con 0x7f692015dce0 2024-04-25T10:11:16.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.726+0000 7f6924cf8640 1 --2- 172.21.15.16:0/3341399001 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f690c06b4d0 0x7f690c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.726+0000 7f6924cf8640 1 --2- 172.21.15.16:0/3341399001 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f690c06b4d0 0x7f690c06d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6910000f60 tx=0x7f6910009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.730+0000 7f691d7fa640 1 -- 172.21.15.16:0/3341399001 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f691807a8f0 con 0x7f692015dce0 2024-04-25T10:11:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.948+0000 7f6926f5c640 1 -- 172.21.15.16:0/3341399001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6920156df0 con 0x7f692015dce0 2024-04-25T10:11:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.949+0000 7f691d7fa640 1 -- 172.21.15.16:0/3341399001 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f691807a1f0 con 0x7f692015dce0 2024-04-25T10:11:16.954 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:16.954 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4044509901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4044509901},{"type":"v1","addr":"172.21.15.16:6835","nonce":4044509901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1494230493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1494230493},{"type":"v1","addr":"172.21.15.186:6837","nonce":1494230493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/84349474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":84349474},{"type":"v1","addr":"172.21.15.16:6834","nonce":84349474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:00.015075+0000","modified":"2024-04-25T10:11:15.841958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"d","rank":0,"incarnation":54,"state":"up:rejoin","state_seq":7,"addr":"172.21.15.186:6836/3950732718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3950732718},{"type":"v1","addr":"172.21.15.186:6836","nonce":3950732718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 56 2024-04-25T10:11:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.951+0000 7f6926f5c640 1 -- 172.21.15.16:0/3341399001 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f690c06b4d0 msgr2=0x7f690c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.951+0000 7f6926f5c640 1 --2- 172.21.15.16:0/3341399001 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f690c06b4d0 0x7f690c06d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6910000f60 tx=0x7f6910009040 comp rx=0 tx=0).stop 2024-04-25T10:11:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.951+0000 7f6926f5c640 1 -- 172.21.15.16:0/3341399001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f692015dce0 msgr2=0x7f6920143c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.951+0000 7f6926f5c640 1 --2- 172.21.15.16:0/3341399001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f692015dce0 0x7f6920143c90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f691802acc0 tx=0x7f6918004a40 comp rx=0 tx=0).stop 2024-04-25T10:11:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.951+0000 7f6926f5c640 1 -- 172.21.15.16:0/3341399001 shutdown_connections 2024-04-25T10:11:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.951+0000 7f6926f5c640 1 --2- 172.21.15.16:0/3341399001 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f690c06b4d0 0x7f690c06d990 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.951+0000 7f6926f5c640 1 --2- 172.21.15.16:0/3341399001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f692015dce0 0x7f6920143c90 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.951+0000 7f6926f5c640 1 --2- 172.21.15.16:0/3341399001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f69200d35b0 0x7f6920160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.951+0000 7f6926f5c640 1 --2- 172.21.15.16:0/3341399001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69200b5b60 0x7f69200cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.951+0000 7f6926f5c640 1 -- 172.21.15.16:0/3341399001 >> 172.21.15.16:0/3341399001 conn(0x7f6920072900 msgr2=0x7f69200d3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.951+0000 7f6926f5c640 1 -- 172.21.15.16:0/3341399001 shutdown_connections 2024-04-25T10:11:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:16.951+0000 7f6926f5c640 1 -- 172.21.15.16:0/3341399001 wait complete. 2024-04-25T10:11:17.370 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:00.015075+0000', 'modified': '2024-04-25T10:11:15.841958+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 87, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7335}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7335': {'gid': 7335, 'name': 'd', 'rank': 0, 'incarnation': 54, 'state': 'up:rejoin', 'state_seq': 7, 'addr': '172.21.15.186:6836/3950732718', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 3950732718}, {'type': 'v1', 'addr': '172.21.15.186:6836', 'nonce': 3950732718}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:17.370 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7335:up:rejoin 2024-04-25T10:11:18.371 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:18.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.479+0000 7f713fe76640 1 Processor -- start 2024-04-25T10:11:18.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.479+0000 7f713fe76640 1 -- start start 2024-04-25T10:11:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.480+0000 7f713fe76640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f713815dce0 0x7f71380d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.480+0000 7f713fe76640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713815e0c0 con 0x7f71380d35b0 2024-04-25T10:11:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.480+0000 7f713fe76640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71380d2510 con 0x7f71380b5b60 2024-04-25T10:11:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.480+0000 7f713fe76640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71380d27f0 con 0x7f713815dce0 2024-04-25T10:11:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.480+0000 7f713d411640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f71380d35b0 0x7f71380b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42152/0 (socket says 172.21.15.16:42152) 2024-04-25T10:11:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.480+0000 7f713d411640 1 -- 172.21.15.16:0/3145020949 learned_addr learned my addr 172.21.15.16:0/3145020949 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713dc12640 1 --2- 172.21.15.16:0/3145020949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f713815dce0 0x7f71380d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713cc10640 1 -- 172.21.15.16:0/3145020949 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4179613191 0 0) 0x7f713815e0c0 con 0x7f71380d35b0 2024-04-25T10:11:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713cc10640 1 -- 172.21.15.16:0/3145020949 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7124003520 con 0x7f71380d35b0 2024-04-25T10:11:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713cc10640 1 -- 172.21.15.16:0/3145020949 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4029575455 0 0) 0x7f7124003520 con 0x7f71380d35b0 2024-04-25T10:11:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713cc10640 1 -- 172.21.15.16:0/3145020949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f713815dce0 msgr2=0x7f71380d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713cc10640 1 --2- 172.21.15.16:0/3145020949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f713815dce0 0x7f71380d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713cc10640 1 -- 172.21.15.16:0/3145020949 >> v1:172.21.15.186:6789/0 conn(0x7f71380b5b60 legacy=0x7f71380b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713cc10640 1 -- 172.21.15.16:0/3145020949 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71380d2ad0 con 0x7f71380d35b0 2024-04-25T10:11:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713cc10640 1 -- 172.21.15.16:0/3145020949 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f7128001670 con 0x7f71380d35b0 2024-04-25T10:11:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713cc10640 1 -- 172.21.15.16:0/3145020949 >> v1:172.21.15.16:6789/0 conn(0x7f71380d35b0 legacy=0x7f71380b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713cc10640 1 --2- 172.21.15.16:0/3145020949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f713815dce0 0x7f71240039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713cc10640 1 --2- 172.21.15.16:0/3145020949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7124003f60 0x7f7124004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713cc10640 1 --2- 172.21.15.16:0/3145020949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7124004d40 0x7f71240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713cc10640 1 -- 172.21.15.16:0/3145020949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71380d27f0 con 0x7f7124004d40 2024-04-25T10:11:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713cc10640 1 -- 172.21.15.16:0/3145020949 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712400baa0 con 0x7f7124003f60 2024-04-25T10:11:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713cc10640 1 -- 172.21.15.16:0/3145020949 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712400bca0 con 0x7f713815dce0 2024-04-25T10:11:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713e413640 1 --2- 172.21.15.16:0/3145020949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7124004d40 0x7f71240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713dc12640 1 --2- 172.21.15.16:0/3145020949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f713815dce0 0x7f71240039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.481+0000 7f713d411640 1 --2- 172.21.15.16:0/3145020949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7124003f60 0x7f7124004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.482+0000 7f713e413640 1 -- 172.21.15.16:0/3145020949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f713815dce0 msgr2=0x7f71240039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.482+0000 7f713e413640 1 --2- 172.21.15.16:0/3145020949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f713815dce0 0x7f71240039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.482+0000 7f713e413640 1 -- 172.21.15.16:0/3145020949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7124003f60 msgr2=0x7f7124004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.482+0000 7f713e413640 1 --2- 172.21.15.16:0/3145020949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7124003f60 0x7f7124004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.482+0000 7f713e413640 1 -- 172.21.15.16:0/3145020949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7124003520 con 0x7f7124004d40 2024-04-25T10:11:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.482+0000 7f713e413640 1 --2- 172.21.15.16:0/3145020949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7124004d40 0x7f71240051c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f7130001e50 tx=0x7f7124005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20187a7bcdef258 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.482+0000 7f713cc10640 1 -- 172.21.15.16:0/3145020949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130001a70 con 0x7f7124004d40 2024-04-25T10:11:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.482+0000 7f713cc10640 1 -- 172.21.15.16:0/3145020949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7130003970 con 0x7f7124004d40 2024-04-25T10:11:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.482+0000 7f713cc10640 1 -- 172.21.15.16:0/3145020949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130003d60 con 0x7f7124004d40 2024-04-25T10:11:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.483+0000 7f713fe76640 1 -- 172.21.15.16:0/3145020949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7124004d40 msgr2=0x7f71240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.483+0000 7f713fe76640 1 --2- 172.21.15.16:0/3145020949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7124004d40 0x7f71240051c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f7130001e50 tx=0x7f7124005830 comp rx=0 tx=0).stop 2024-04-25T10:11:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.483+0000 7f713e413640 1 -- 172.21.15.16:0/3145020949 reap_dead start 2024-04-25T10:11:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.483+0000 7f713fe76640 1 -- 172.21.15.16:0/3145020949 shutdown_connections 2024-04-25T10:11:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.483+0000 7f713fe76640 1 -- 172.21.15.16:0/3145020949 >> 172.21.15.16:0/3145020949 conn(0x7f7138072900 msgr2=0x7f71380d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.483+0000 7f713fe76640 1 -- 172.21.15.16:0/3145020949 shutdown_connections 2024-04-25T10:11:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.483+0000 7f713fe76640 1 -- 172.21.15.16:0/3145020949 wait complete. 2024-04-25T10:11:18.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.484+0000 7f713fe76640 1 Processor -- start 2024-04-25T10:11:18.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.484+0000 7f713fe76640 1 -- start start 2024-04-25T10:11:18.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.485+0000 7f713fe76640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7124004d40 0x7f71381f0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.485+0000 7f713fe76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71380c9250 0x7f71380c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.485+0000 7f713fe76640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71380b3d60 0x7f71380b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.485+0000 7f713fe76640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71380b9820 con 0x7f71380c9250 2024-04-25T10:11:18.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.485+0000 7f713fe76640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71380d3090 con 0x7f71380b3d60 2024-04-25T10:11:18.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.485+0000 7f713fe76640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71380b99a0 con 0x7f7124004d40 2024-04-25T10:11:18.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.485+0000 7f713d411640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71380c9250 0x7f71380c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.485+0000 7f713d411640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71380c9250 0x7f71380c9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52942/0 (socket says 172.21.15.16:52942) 2024-04-25T10:11:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.485+0000 7f713d411640 1 -- 172.21.15.16:0/2984119686 learned_addr learned my addr 172.21.15.16:0/2984119686 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.485+0000 7f713e413640 1 --2- 172.21.15.16:0/2984119686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71380b3d60 0x7f71380b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.485+0000 7f713dc12640 1 --2- 172.21.15.16:0/2984119686 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7124004d40 0x7f71381f0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.486+0000 7f713d411640 1 -- 172.21.15.16:0/2984119686 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7124004d40 msgr2=0x7f71381f0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.486+0000 7f713d411640 1 --2- 172.21.15.16:0/2984119686 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7124004d40 0x7f71381f0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.486+0000 7f713d411640 1 -- 172.21.15.16:0/2984119686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71380b3d60 msgr2=0x7f71380b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.486+0000 7f713d411640 1 --2- 172.21.15.16:0/2984119686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71380b3d60 0x7f71380b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.486+0000 7f713d411640 1 -- 172.21.15.16:0/2984119686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71380ca1e0 con 0x7f71380c9250 2024-04-25T10:11:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.486+0000 7f713d411640 1 --2- 172.21.15.16:0/2984119686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71380c9250 0x7f71380c9870 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f71280072c0 tx=0x7f71280076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.486+0000 7f712e7fc640 1 -- 172.21.15.16:0/2984119686 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712800f8e0 con 0x7f71380c9250 2024-04-25T10:11:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.486+0000 7f712e7fc640 1 -- 172.21.15.16:0/2984119686 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7128018070 con 0x7f71380c9250 2024-04-25T10:11:18.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.486+0000 7f713fe76640 1 -- 172.21.15.16:0/2984119686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71381f7110 con 0x7f71380c9250 2024-04-25T10:11:18.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.486+0000 7f712e7fc640 1 -- 172.21.15.16:0/2984119686 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712800fcc0 con 0x7f71380c9250 2024-04-25T10:11:18.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.486+0000 7f713fe76640 1 -- 172.21.15.16:0/2984119686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71381f1290 con 0x7f71380c9250 2024-04-25T10:11:18.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.487+0000 7f713fe76640 1 -- 172.21.15.16:0/2984119686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71380749b0 con 0x7f71380c9250 2024-04-25T10:11:18.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.488+0000 7f712e7fc640 1 -- 172.21.15.16:0/2984119686 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f712801d460 con 0x7f71380c9250 2024-04-25T10:11:18.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.488+0000 7f712e7fc640 1 --2- 172.21.15.16:0/2984119686 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f711c06b4d0 0x7f711c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.488+0000 7f712e7fc640 1 -- 172.21.15.16:0/2984119686 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f712808a2a0 con 0x7f71380c9250 2024-04-25T10:11:18.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.488+0000 7f713dc12640 1 --2- 172.21.15.16:0/2984119686 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f711c06b4d0 0x7f711c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.489+0000 7f713dc12640 1 --2- 172.21.15.16:0/2984119686 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f711c06b4d0 0x7f711c06d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7134009bc0 tx=0x7f7134009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.492+0000 7f712e7fc640 1 -- 172.21.15.16:0/2984119686 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f712800fe60 con 0x7f71380c9250 2024-04-25T10:11:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.712+0000 7f713fe76640 1 -- 172.21.15.16:0/2984119686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71380c6e00 con 0x7f71380c9250 2024-04-25T10:11:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.713+0000 7f712e7fc640 1 -- 172.21.15.16:0/2984119686 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f71280590c0 con 0x7f71380c9250 2024-04-25T10:11:18.718 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:18.719 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4044509901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4044509901},{"type":"v1","addr":"172.21.15.16:6835","nonce":4044509901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1494230493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1494230493},{"type":"v1","addr":"172.21.15.186:6837","nonce":1494230493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/84349474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":84349474},{"type":"v1","addr":"172.21.15.16:6834","nonce":84349474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:00.015075+0000","modified":"2024-04-25T10:11:16.850790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"d","rank":0,"incarnation":54,"state":"up:active","state_seq":8,"addr":"172.21.15.186:6836/3950732718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3950732718},{"type":"v1","addr":"172.21.15.186:6836","nonce":3950732718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 57 2024-04-25T10:11:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.715+0000 7f713fe76640 1 -- 172.21.15.16:0/2984119686 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f711c06b4d0 msgr2=0x7f711c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.715+0000 7f713fe76640 1 --2- 172.21.15.16:0/2984119686 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f711c06b4d0 0x7f711c06d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7134009bc0 tx=0x7f7134009190 comp rx=0 tx=0).stop 2024-04-25T10:11:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.715+0000 7f713fe76640 1 -- 172.21.15.16:0/2984119686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71380c9250 msgr2=0x7f71380c9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.715+0000 7f713fe76640 1 --2- 172.21.15.16:0/2984119686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71380c9250 0x7f71380c9870 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f71280072c0 tx=0x7f71280076c0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.715+0000 7f713fe76640 1 -- 172.21.15.16:0/2984119686 shutdown_connections 2024-04-25T10:11:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.715+0000 7f713fe76640 1 --2- 172.21.15.16:0/2984119686 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f711c06b4d0 0x7f711c06d990 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.715+0000 7f713fe76640 1 --2- 172.21.15.16:0/2984119686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71380b3d60 0x7f71380b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.715+0000 7f713fe76640 1 --2- 172.21.15.16:0/2984119686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71380c9250 0x7f71380c9870 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.715+0000 7f713fe76640 1 --2- 172.21.15.16:0/2984119686 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7124004d40 0x7f71381f0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.715+0000 7f713fe76640 1 -- 172.21.15.16:0/2984119686 >> 172.21.15.16:0/2984119686 conn(0x7f7138072900 msgr2=0x7f71380d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.715+0000 7f713fe76640 1 -- 172.21.15.16:0/2984119686 shutdown_connections 2024-04-25T10:11:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:18.715+0000 7f713fe76640 1 -- 172.21.15.16:0/2984119686 wait complete. 2024-04-25T10:11:19.133 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:00.015075+0000', 'modified': '2024-04-25T10:11:16.850790+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 87, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7335}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7335': {'gid': 7335, 'name': 'd', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.186:6836/3950732718', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 3950732718}, {'type': 'v1', 'addr': '172.21.15.186:6836', 'nonce': 3950732718}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:19.134 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:11:19.134 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:11:19.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.234+0000 7efe3a709640 1 Processor -- start 2024-04-25T10:11:19.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.234+0000 7efe3a709640 1 -- start start 2024-04-25T10:11:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.234+0000 7efe3a709640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efe340cef40 0x7efe340b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.234+0000 7efe3a709640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7efe341618d0 0x7efe340d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.234+0000 7efe3a709640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe34162040 con 0x7efe341610b0 2024-04-25T10:11:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.234+0000 7efe3a709640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe340d1ea0 con 0x7efe340cef40 2024-04-25T10:11:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.234+0000 7efe3a709640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe34162320 con 0x7efe341618d0 2024-04-25T10:11:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.235+0000 7efe337fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efe340cef40 0x7efe340b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.235+0000 7efe337fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efe340cef40 0x7efe340b9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52956/0 (socket says 172.21.15.16:52956) 2024-04-25T10:11:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.235+0000 7efe337fe640 1 -- 172.21.15.16:0/3263566357 learned_addr learned my addr 172.21.15.16:0/3263566357 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.235+0000 7efe337fe640 1 -- 172.21.15.16:0/3263566357 >> v2:172.21.15.186:3300/0 conn(0x7efe341618d0 msgr2=0x7efe340d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.235+0000 7efe337fe640 1 --2- 172.21.15.16:0/3263566357 >> v2:172.21.15.186:3300/0 conn(0x7efe341618d0 0x7efe340d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.235+0000 7efe337fe640 1 -- 172.21.15.16:0/3263566357 >> v1:172.21.15.186:6789/0 conn(0x7efe341610b0 legacy=0x7efe340ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.235+0000 7efe337fe640 1 -- 172.21.15.16:0/3263566357 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe34162600 con 0x7efe340cef40 2024-04-25T10:11:19.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.236+0000 7efe337fe640 1 --2- 172.21.15.16:0/3263566357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe340cef40 0x7efe340b9580 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7efe340b9190 tx=0x7efe2802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d62b771226be10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.236+0000 7efe32ffd640 1 -- 172.21.15.16:0/3263566357 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe28037070 con 0x7efe340cef40 2024-04-25T10:11:19.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.236+0000 7efe32ffd640 1 -- 172.21.15.16:0/3263566357 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe28032440 con 0x7efe340cef40 2024-04-25T10:11:19.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.236+0000 7efe32ffd640 1 -- 172.21.15.16:0/3263566357 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe28032810 con 0x7efe340cef40 2024-04-25T10:11:19.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.236+0000 7efe3a709640 1 -- 172.21.15.16:0/3263566357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe340cef40 msgr2=0x7efe340b9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.236+0000 7efe3a709640 1 --2- 172.21.15.16:0/3263566357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe340cef40 0x7efe340b9580 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7efe340b9190 tx=0x7efe2802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.236+0000 7efe3a709640 1 -- 172.21.15.16:0/3263566357 shutdown_connections 2024-04-25T10:11:19.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.236+0000 7efe3a709640 1 --2- 172.21.15.16:0/3263566357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe340cef40 0x7efe340b9580 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.236+0000 7efe3a709640 1 --2- 172.21.15.16:0/3263566357 >> v2:172.21.15.186:3300/0 conn(0x7efe341618d0 0x7efe340d2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.236+0000 7efe3a709640 1 -- 172.21.15.16:0/3263566357 >> 172.21.15.16:0/3263566357 conn(0x7efe340b5570 msgr2=0x7efe340b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.236+0000 7efe3a709640 1 -- 172.21.15.16:0/3263566357 shutdown_connections 2024-04-25T10:11:19.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.236+0000 7efe3a709640 1 -- 172.21.15.16:0/3263566357 wait complete. 2024-04-25T10:11:19.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.238+0000 7efe3a709640 1 Processor -- start 2024-04-25T10:11:19.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.238+0000 7efe3a709640 1 -- start start 2024-04-25T10:11:19.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.238+0000 7efe3a709640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe340cef40 0x7efe34157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.238+0000 7efe3a709640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe341610b0 0x7efe34153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.238+0000 7efe3a709640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe341618d0 0x7efe34145f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.238+0000 7efe3a709640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe34161d40 con 0x7efe341618d0 2024-04-25T10:11:19.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.238+0000 7efe3a709640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe340b6c00 con 0x7efe340cef40 2024-04-25T10:11:19.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.238+0000 7efe3a709640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe34161490 con 0x7efe341610b0 2024-04-25T10:11:19.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.239+0000 7efe38ca6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe341618d0 0x7efe34145f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.239+0000 7efe38ca6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe341618d0 0x7efe34145f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52970/0 (socket says 172.21.15.16:52970) 2024-04-25T10:11:19.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.239+0000 7efe38ca6640 1 -- 172.21.15.16:0/755433213 learned_addr learned my addr 172.21.15.16:0/755433213 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:19.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.239+0000 7efe33fff640 1 --2- 172.21.15.16:0/755433213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe340cef40 0x7efe34157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.239+0000 7efe337fe640 1 --2- 172.21.15.16:0/755433213 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe341610b0 0x7efe34153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.239+0000 7efe38ca6640 1 -- 172.21.15.16:0/755433213 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe341610b0 msgr2=0x7efe34153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.239+0000 7efe38ca6640 1 --2- 172.21.15.16:0/755433213 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe341610b0 0x7efe34153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.239+0000 7efe38ca6640 1 -- 172.21.15.16:0/755433213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe340cef40 msgr2=0x7efe34157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.239+0000 7efe38ca6640 1 --2- 172.21.15.16:0/755433213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe340cef40 0x7efe34157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.239+0000 7efe38ca6640 1 -- 172.21.15.16:0/755433213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe341553f0 con 0x7efe341618d0 2024-04-25T10:11:19.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.239+0000 7efe38ca6640 1 --2- 172.21.15.16:0/755433213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe341618d0 0x7efe34145f50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00b9a0 tx=0x7efe2c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.239+0000 7efe30ff9640 1 -- 172.21.15.16:0/755433213 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c019070 con 0x7efe341618d0 2024-04-25T10:11:19.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.239+0000 7efe30ff9640 1 -- 172.21.15.16:0/755433213 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c00cbc0 con 0x7efe341618d0 2024-04-25T10:11:19.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.239+0000 7efe3a709640 1 -- 172.21.15.16:0/755433213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe34156450 con 0x7efe341618d0 2024-04-25T10:11:19.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.239+0000 7efe30ff9640 1 -- 172.21.15.16:0/755433213 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c014440 con 0x7efe341618d0 2024-04-25T10:11:19.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.239+0000 7efe3a709640 1 -- 172.21.15.16:0/755433213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe34155c50 con 0x7efe341618d0 2024-04-25T10:11:19.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.241+0000 7efe30ff9640 1 -- 172.21.15.16:0/755433213 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe2c01c070 con 0x7efe341618d0 2024-04-25T10:11:19.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.242+0000 7efe30ff9640 1 --2- 172.21.15.16:0/755433213 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efe2006b4d0 0x7efe2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.242+0000 7efe30ff9640 1 -- 172.21.15.16:0/755433213 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7efe2c089f00 con 0x7efe341618d0 2024-04-25T10:11:19.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.242+0000 7efe33fff640 1 --2- 172.21.15.16:0/755433213 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efe2006b4d0 0x7efe2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.242+0000 7efe3d414740 1 Processor -- start 2024-04-25T10:11:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.242+0000 7efe3d414740 1 -- start start 2024-04-25T10:11:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.243+0000 7efe33fff640 1 --2- 172.21.15.16:0/755433213 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efe2006b4d0 0x7efe2006d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efe24007fd0 tx=0x7efe24009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.243+0000 7efe3d414740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5c3230 0x55854c5c3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.243+0000 7efe3d414740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5e0ad0 0x55854c5e1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.243+0000 7efe3d414740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c598f10 0x55854c5df190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.243+0000 7efe3d414740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55854c4fba10 con 0x55854c5e0ad0 2024-04-25T10:11:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.243+0000 7efe3d414740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55854c4a4b30 con 0x55854c598f10 2024-04-25T10:11:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.243+0000 7efe3d414740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55854c4f7c50 con 0x55854c5c3230 2024-04-25T10:11:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.243+0000 7efe337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5e0ad0 0x55854c5e1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.243+0000 7efe337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5e0ad0 0x55854c5e1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52980/0 (socket says 172.21.15.16:52980) 2024-04-25T10:11:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.243+0000 7efe337fe640 1 -- 172.21.15.16:0/1541584298 learned_addr learned my addr 172.21.15.16:0/1541584298 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.243+0000 7efe33fff640 1 --2- 172.21.15.16:0/1541584298 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5c3230 0x55854c5c3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.243+0000 7efe38ca6640 1 --2- 172.21.15.16:0/1541584298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c598f10 0x55854c5df190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.244+0000 7efe337fe640 1 -- 172.21.15.16:0/1541584298 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5c3230 msgr2=0x55854c5c3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.244+0000 7efe337fe640 1 --2- 172.21.15.16:0/1541584298 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5c3230 0x55854c5c3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.244+0000 7efe337fe640 1 -- 172.21.15.16:0/1541584298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c598f10 msgr2=0x55854c5df190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.244+0000 7efe337fe640 1 --2- 172.21.15.16:0/1541584298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c598f10 0x55854c5df190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.244+0000 7efe337fe640 1 -- 172.21.15.16:0/1541584298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55854c4b9e10 con 0x55854c5e0ad0 2024-04-25T10:11:19.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.244+0000 7efe337fe640 1 --2- 172.21.15.16:0/1541584298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5e0ad0 0x55854c5e1020 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7efe2802a1e0 tx=0x7efe2802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a1ef932c69d964 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.244+0000 7efe19ffb640 1 -- 172.21.15.16:0/1541584298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe28037040 con 0x55854c5e0ad0 2024-04-25T10:11:19.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.244+0000 7efe19ffb640 1 -- 172.21.15.16:0/1541584298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe280326c0 con 0x55854c5e0ad0 2024-04-25T10:11:19.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.244+0000 7efe19ffb640 1 -- 172.21.15.16:0/1541584298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe28032a90 con 0x55854c5e0ad0 2024-04-25T10:11:19.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.245+0000 7efe3d414740 1 -- 172.21.15.16:0/1541584298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5e0ad0 msgr2=0x55854c5e1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.245+0000 7efe3d414740 1 --2- 172.21.15.16:0/1541584298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5e0ad0 0x55854c5e1020 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7efe2802a1e0 tx=0x7efe2802c000 comp rx=0 tx=0).stop 2024-04-25T10:11:19.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.245+0000 7efe3d414740 1 -- 172.21.15.16:0/1541584298 shutdown_connections 2024-04-25T10:11:19.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.245+0000 7efe3d414740 1 --2- 172.21.15.16:0/1541584298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c598f10 0x55854c5df190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.245+0000 7efe3d414740 1 --2- 172.21.15.16:0/1541584298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5e0ad0 0x55854c5e1020 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.245+0000 7efe3d414740 1 --2- 172.21.15.16:0/1541584298 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5c3230 0x55854c5c3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.245+0000 7efe3d414740 1 -- 172.21.15.16:0/1541584298 >> 172.21.15.16:0/1541584298 conn(0x55854c595840 msgr2=0x55854c5de040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.245+0000 7efe3d414740 1 -- 172.21.15.16:0/1541584298 shutdown_connections 2024-04-25T10:11:19.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.245+0000 7efe3d414740 1 -- 172.21.15.16:0/1541584298 wait complete. 2024-04-25T10:11:19.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.245+0000 7efe3d414740 1 -- start start 2024-04-25T10:11:19.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.245+0000 7efe3d414740 1 Processor -- start 2024-04-25T10:11:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.246+0000 7efe3d414740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c5c3230 0x55854c5f0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.246+0000 7efe3d414740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5e0ad0 0x55854c5d50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.246+0000 7efe3d414740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5d5630 0x55854c5c0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.246+0000 7efe3d414740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55854c599cb0 con 0x55854c5e0ad0 2024-04-25T10:11:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.246+0000 7efe3d414740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55854c4b80c0 con 0x55854c5c3230 2024-04-25T10:11:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.246+0000 7efe3d414740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55854c593770 con 0x55854c5d5630 2024-04-25T10:11:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.246+0000 7efe3d414740 10 client.0 _ll_register_callbacks cb 0x55854c4abec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.246+0000 7efe337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5e0ad0 0x55854c5d50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.246+0000 7efe337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5e0ad0 0x55854c5d50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52982/0 (socket says 172.21.15.16:52982) 2024-04-25T10:11:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.246+0000 7efe337fe640 1 -- 172.21.15.16:0/3441599403 learned_addr learned my addr 172.21.15.16:0/3441599403 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.246+0000 7efe33fff640 1 --2- 172.21.15.16:0/3441599403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c5c3230 0x55854c5f0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.246+0000 7efe38ca6640 1 --2- 172.21.15.16:0/3441599403 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5d5630 0x55854c5c0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe337fe640 1 -- 172.21.15.16:0/3441599403 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5d5630 msgr2=0x55854c5c0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe337fe640 1 --2- 172.21.15.16:0/3441599403 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5d5630 0x55854c5c0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe337fe640 1 -- 172.21.15.16:0/3441599403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c5c3230 msgr2=0x55854c5f0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe337fe640 1 --2- 172.21.15.16:0/3441599403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c5c3230 0x55854c5f0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe337fe640 1 -- 172.21.15.16:0/3441599403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55854c2102d0 con 0x55854c5e0ad0 2024-04-25T10:11:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe33fff640 1 --2- 172.21.15.16:0/3441599403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c5c3230 0x55854c5f0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe38ca6640 1 --2- 172.21.15.16:0/3441599403 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5d5630 0x55854c5c0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe337fe640 1 --2- 172.21.15.16:0/3441599403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5e0ad0 0x55854c5d50f0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7efe2802a790 tx=0x7efe28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bb81c74f9dfac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:11:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe12ffd640 1 -- 172.21.15.16:0/3441599403 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2804b060 con 0x55854c5e0ad0 2024-04-25T10:11:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe12ffd640 1 -- 172.21.15.16:0/3441599403 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7efe28032a80 con 0x55854c5e0ad0 2024-04-25T10:11:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe3d414740 1 -- 172.21.15.16:0/3441599403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe280326c0 con 0x55854c5e0ad0 2024-04-25T10:11:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe12ffd640 1 -- 172.21.15.16:0/3441599403 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe28051050 con 0x55854c5e0ad0 2024-04-25T10:11:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe3d414740 10 client.7626 fetch_fsmap learned FSMap version 57 2024-04-25T10:11:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe3d414740 1 -- 172.21.15.16:0/3441599403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55854c2da6b0 con 0x55854c5e0ad0 2024-04-25T10:11:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe12ffd640 1 -- 172.21.15.16:0/3441599403 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe280326c0 con 0x55854c5e0ad0 2024-04-25T10:11:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe12ffd640 1 -- 172.21.15.16:0/3441599403 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efe28054020 con 0x55854c5e0ad0 2024-04-25T10:11:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe3d414740 10 client.7626 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:11:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe3d414740 10 client.7626 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:11:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe3d414740 20 client.7626 populate_metadata read hostname 'smithi016' 2024-04-25T10:11:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe3d414740 1 --2- 172.21.15.16:0/3441599403 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55854c5b0ed0 0x55854c5fbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe3d414740 4 client.7626 mds_command: new command op to 7335 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.247+0000 7efe3d414740 1 -- 172.21.15.16:0/3441599403 --> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55854c55a920 con 0x55854c5b0ed0 2024-04-25T10:11:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.248+0000 7efe33fff640 1 --2- 172.21.15.16:0/3441599403 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55854c5b0ed0 0x55854c5fbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.248+0000 7efe33fff640 1 --2- 172.21.15.16:0/3441599403 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55854c5b0ed0 0x55854c5fbc50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21e7798cc01cae30 server_cookie=191f0dfe2723a27d in_seq=0 out_seq=0 2024-04-25T10:11:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.248+0000 7efe12ffd640 10 client.7626 ms_handle_connect on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.249+0000 7efe12ffd640 1 -- 172.21.15.16:0/3441599403 <== mds.0 v2:172.21.15.186:6834/3950732718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55854c55a920 con 0x55854c5b0ed0 2024-04-25T10:11:19.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.249+0000 7efe12ffd640 10 client.7626 handle_command_reply: tid=0 2024-04-25T10:11:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.249+0000 7efe3d414740 1 client.7626 shutdown 2024-04-25T10:11:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.249+0000 7efe3d414740 1 -- 172.21.15.16:0/3441599403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5e0ad0 msgr2=0x55854c5d50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.249+0000 7efe3d414740 1 --2- 172.21.15.16:0/3441599403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5e0ad0 0x55854c5d50f0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7efe2802a790 tx=0x7efe28004730 comp rx=0 tx=0).stop 2024-04-25T10:11:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.249+0000 7efe3d414740 1 -- 172.21.15.16:0/3441599403 shutdown_connections 2024-04-25T10:11:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.249+0000 7efe3d414740 1 --2- 172.21.15.16:0/3441599403 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55854c5b0ed0 0x55854c5fbc50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.249+0000 7efe3d414740 1 --2- 172.21.15.16:0/3441599403 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5d5630 0x55854c5c0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.249+0000 7efe3d414740 1 --2- 172.21.15.16:0/3441599403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5e0ad0 0x55854c5d50f0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.249+0000 7efe12ffd640 0 client.7626 ms_handle_reset on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.249+0000 7efe3d414740 1 --2- 172.21.15.16:0/3441599403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c5c3230 0x55854c5f0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.249+0000 7efe3d414740 1 -- 172.21.15.16:0/3441599403 >> 172.21.15.16:0/3441599403 conn(0x55854c598f10 msgr2=0x55854c5dfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.249+0000 7efe3d414740 1 -- 172.21.15.16:0/3441599403 shutdown_connections 2024-04-25T10:11:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.249+0000 7efe3d414740 1 -- 172.21.15.16:0/3441599403 wait complete. 2024-04-25T10:11:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.249+0000 7efe3d414740 20 client.7626 trim_cache size 0 max 16384 2024-04-25T10:11:19.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.265+0000 7efe3d414740 1 Processor -- start 2024-04-25T10:11:19.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.265+0000 7efe3d414740 1 -- start start 2024-04-25T10:11:19.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.266+0000 7efe3d414740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5b0ed0 0x55854c5bfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.266+0000 7efe3d414740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5c3230 0x55854c5924a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.266+0000 7efe3d414740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c5d5630 0x55854c5929e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.266+0000 7efe3d414740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe341ff240 con 0x55854c5b0ed0 2024-04-25T10:11:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.266+0000 7efe3d414740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe341fe670 con 0x55854c5d5630 2024-04-25T10:11:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.266+0000 7efe3d414740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe341fec70 con 0x55854c5c3230 2024-04-25T10:11:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.266+0000 7efe33fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5b0ed0 0x55854c5bfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.266+0000 7efe33fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5b0ed0 0x55854c5bfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52990/0 (socket says 172.21.15.16:52990) 2024-04-25T10:11:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.266+0000 7efe33fff640 1 -- 172.21.15.16:0/3180019023 learned_addr learned my addr 172.21.15.16:0/3180019023 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.266+0000 7efe33fff640 1 -- 172.21.15.16:0/3180019023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5c3230 msgr2=0x55854c5924a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.266+0000 7efe33fff640 1 --2- 172.21.15.16:0/3180019023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5c3230 0x55854c5924a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.266+0000 7efe33fff640 1 -- 172.21.15.16:0/3180019023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c5d5630 msgr2=0x55854c5929e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.266+0000 7efe33fff640 1 --2- 172.21.15.16:0/3180019023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c5d5630 0x55854c5929e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.266+0000 7efe33fff640 1 -- 172.21.15.16:0/3180019023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55854c5f0a50 con 0x55854c5b0ed0 2024-04-25T10:11:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.266+0000 7efe33fff640 1 --2- 172.21.15.16:0/3180019023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5b0ed0 0x55854c5bfef0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7efe240013d0 tx=0x7efe24017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b516e8ea91d21a3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.266+0000 7efe197fa640 1 -- 172.21.15.16:0/3180019023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe240035f0 con 0x55854c5b0ed0 2024-04-25T10:11:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.266+0000 7efe197fa640 1 -- 172.21.15.16:0/3180019023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2402b020 con 0x55854c5b0ed0 2024-04-25T10:11:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.266+0000 7efe197fa640 1 -- 172.21.15.16:0/3180019023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe240037d0 con 0x55854c5b0ed0 2024-04-25T10:11:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.267+0000 7efe3d414740 1 -- 172.21.15.16:0/3180019023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5b0ed0 msgr2=0x55854c5bfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.267+0000 7efe3d414740 1 --2- 172.21.15.16:0/3180019023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5b0ed0 0x55854c5bfef0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7efe240013d0 tx=0x7efe24017040 comp rx=0 tx=0).stop 2024-04-25T10:11:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.267+0000 7efe3d414740 1 -- 172.21.15.16:0/3180019023 shutdown_connections 2024-04-25T10:11:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.267+0000 7efe3d414740 1 --2- 172.21.15.16:0/3180019023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c5d5630 0x55854c5929e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.267+0000 7efe3d414740 1 --2- 172.21.15.16:0/3180019023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5c3230 0x55854c5924a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.267+0000 7efe3d414740 1 --2- 172.21.15.16:0/3180019023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5b0ed0 0x55854c5bfef0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.267+0000 7efe3d414740 1 -- 172.21.15.16:0/3180019023 >> 172.21.15.16:0/3180019023 conn(0x55854c598f10 msgr2=0x55854c5dfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.267+0000 7efe3d414740 1 -- 172.21.15.16:0/3180019023 shutdown_connections 2024-04-25T10:11:19.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.267+0000 7efe3d414740 1 -- 172.21.15.16:0/3180019023 wait complete. 2024-04-25T10:11:19.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.268+0000 7efe3d414740 1 -- start start 2024-04-25T10:11:19.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.268+0000 7efe3d414740 1 Processor -- start 2024-04-25T10:11:19.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.268+0000 7efe3d414740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5b0ed0 0x55854c5bdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.268+0000 7efe3d414740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c5c3230 0x55854c648430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.268+0000 7efe3d414740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5d5630 0x55854c63ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.268+0000 7efe3d414740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55854c5b0d50 con 0x55854c5b0ed0 2024-04-25T10:11:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe3d414740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55854c5c0e90 con 0x55854c5c3230 2024-04-25T10:11:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe3d414740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55854c5b12f0 con 0x55854c5d5630 2024-04-25T10:11:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe3d414740 10 client.0 _ll_register_callbacks cb 0x55854c593c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe33fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5b0ed0 0x55854c5bdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe33fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5b0ed0 0x55854c5bdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52994/0 (socket says 172.21.15.16:52994) 2024-04-25T10:11:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe33fff640 1 -- 172.21.15.16:0/892080186 learned_addr learned my addr 172.21.15.16:0/892080186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe38ca6640 1 --2- 172.21.15.16:0/892080186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5d5630 0x55854c63ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe337fe640 1 --2- 172.21.15.16:0/892080186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c5c3230 0x55854c648430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe33fff640 1 -- 172.21.15.16:0/892080186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5d5630 msgr2=0x55854c63ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe33fff640 1 --2- 172.21.15.16:0/892080186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5d5630 0x55854c63ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe33fff640 1 -- 172.21.15.16:0/892080186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c5c3230 msgr2=0x55854c648430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe33fff640 1 --2- 172.21.15.16:0/892080186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c5c3230 0x55854c648430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe33fff640 1 -- 172.21.15.16:0/892080186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55854c5be200 con 0x55854c5b0ed0 2024-04-25T10:11:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe33fff640 1 --2- 172.21.15.16:0/892080186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5b0ed0 0x55854c5bdcc0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7efe24029040 tx=0x7efe24000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7ec8dc1b7164cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:11:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe12ffd640 1 -- 172.21.15.16:0/892080186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24034070 con 0x55854c5b0ed0 2024-04-25T10:11:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe12ffd640 1 -- 172.21.15.16:0/892080186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7efe24039b10 con 0x55854c5b0ed0 2024-04-25T10:11:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe3d414740 1 -- 172.21.15.16:0/892080186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe2402b020 con 0x55854c5b0ed0 2024-04-25T10:11:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe12ffd640 1 -- 172.21.15.16:0/892080186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe2402a050 con 0x55854c5b0ed0 2024-04-25T10:11:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe3d414740 10 client.7632 fetch_fsmap learned FSMap version 57 2024-04-25T10:11:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.269+0000 7efe3d414740 1 -- 172.21.15.16:0/892080186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55854c600360 con 0x55854c5b0ed0 2024-04-25T10:11:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.270+0000 7efe12ffd640 1 -- 172.21.15.16:0/892080186 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2402b020 con 0x55854c5b0ed0 2024-04-25T10:11:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.270+0000 7efe12ffd640 1 -- 172.21.15.16:0/892080186 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efe24003940 con 0x55854c5b0ed0 2024-04-25T10:11:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.270+0000 7efe3d414740 10 client.7632 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:11:19.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.270+0000 7efe3d414740 10 client.7632 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:11:19.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.270+0000 7efe3d414740 20 client.7632 populate_metadata read hostname 'smithi016' 2024-04-25T10:11:19.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.270+0000 7efe3d414740 1 --2- 172.21.15.16:0/892080186 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55854c5e0ad0 0x55854c6403b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.270+0000 7efe3d414740 4 client.7632 mds_command: new command op to 7335 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:11:19.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.270+0000 7efe3d414740 1 -- 172.21.15.16:0/892080186 --> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55854c508f50 con 0x55854c5e0ad0 2024-04-25T10:11:19.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.270+0000 7efe337fe640 1 --2- 172.21.15.16:0/892080186 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55854c5e0ad0 0x55854c6403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.270+0000 7efe337fe640 1 --2- 172.21.15.16:0/892080186 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55854c5e0ad0 0x55854c6403b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=234e235430aad9cb server_cookie=5cfe87f5a1279519 in_seq=0 out_seq=0 2024-04-25T10:11:19.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.270+0000 7efe12ffd640 10 client.7632 ms_handle_connect on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:19.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.270+0000 7efe12ffd640 1 -- 172.21.15.16:0/892080186 <== mds.0 v2:172.21.15.186:6834/3950732718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55854c508f50 con 0x55854c5e0ad0 2024-04-25T10:11:19.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.270+0000 7efe12ffd640 10 client.7632 handle_command_reply: tid=0 2024-04-25T10:11:19.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.270+0000 7efe3d414740 1 client.7632 shutdown 2024-04-25T10:11:19.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3d414740 1 -- 172.21.15.16:0/892080186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5b0ed0 msgr2=0x55854c5bdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3d414740 1 --2- 172.21.15.16:0/892080186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5b0ed0 0x55854c5bdcc0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7efe24029040 tx=0x7efe24000fb0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3d414740 1 -- 172.21.15.16:0/892080186 shutdown_connections 2024-04-25T10:11:19.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3d414740 1 --2- 172.21.15.16:0/892080186 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55854c5e0ad0 0x55854c6403b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3d414740 1 --2- 172.21.15.16:0/892080186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55854c5d5630 0x55854c63ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe12ffd640 0 client.7632 ms_handle_reset on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:19.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3d414740 1 --2- 172.21.15.16:0/892080186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55854c5c3230 0x55854c648430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3d414740 1 --2- 172.21.15.16:0/892080186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55854c5b0ed0 0x55854c5bdcc0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3d414740 1 -- 172.21.15.16:0/892080186 >> 172.21.15.16:0/892080186 conn(0x55854c598f10 msgr2=0x55854c5c2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3d414740 1 -- 172.21.15.16:0/892080186 shutdown_connections 2024-04-25T10:11:19.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3d414740 1 -- 172.21.15.16:0/892080186 wait complete. 2024-04-25T10:11:19.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3d414740 20 client.7632 trim_cache size 0 max 16384 2024-04-25T10:11:19.284 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:11:19.284 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "34bc5eca-3a21-42a6-a26f-07da0905e8bc", 2024-04-25T10:11:19.284 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-04-25T10:11:19.284 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7335, 2024-04-25T10:11:19.284 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-04-25T10:11:19.284 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-04-25T10:11:19.285 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-04-25T10:11:19.285 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 4.4322865570000003, 2024-04-25T10:11:19.285 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 57, 2024-04-25T10:11:19.285 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 87, 2024-04-25T10:11:19.285 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T10:11:19.285 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 23.624761283000002 2024-04-25T10:11:19.285 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:11:19.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3a709640 1 -- 172.21.15.16:0/755433213 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efe2006b4d0 msgr2=0x7efe2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3a709640 1 --2- 172.21.15.16:0/755433213 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efe2006b4d0 0x7efe2006d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efe24007fd0 tx=0x7efe24009040 comp rx=0 tx=0).stop 2024-04-25T10:11:19.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3a709640 1 -- 172.21.15.16:0/755433213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe341618d0 msgr2=0x7efe34145f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3a709640 1 --2- 172.21.15.16:0/755433213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe341618d0 0x7efe34145f50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00b9a0 tx=0x7efe2c00be60 comp rx=0 tx=0).stop 2024-04-25T10:11:19.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3a709640 1 -- 172.21.15.16:0/755433213 shutdown_connections 2024-04-25T10:11:19.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3a709640 1 --2- 172.21.15.16:0/755433213 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efe2006b4d0 0x7efe2006d990 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3a709640 1 --2- 172.21.15.16:0/755433213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe341618d0 0x7efe34145f50 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3a709640 1 --2- 172.21.15.16:0/755433213 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe341610b0 0x7efe34153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3a709640 1 --2- 172.21.15.16:0/755433213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe340cef40 0x7efe34157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3a709640 1 -- 172.21.15.16:0/755433213 >> 172.21.15.16:0/755433213 conn(0x7efe340b5570 msgr2=0x7efe340b7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.271+0000 7efe3a709640 1 -- 172.21.15.16:0/755433213 shutdown_connections 2024-04-25T10:11:19.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.272+0000 7efe3a709640 1 -- 172.21.15.16:0/755433213 wait complete. 2024-04-25T10:11:19.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:11:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.384+0000 7fe502353640 1 Processor -- start 2024-04-25T10:11:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.384+0000 7fe502353640 1 -- start start 2024-04-25T10:11:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe502353640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4fc0aecf0 0x7fe4fc0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe502353640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4fc0712c0 con 0x7fe4fc15d980 2024-04-25T10:11:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe502353640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4fc0715a0 con 0x7fe4fc15ed70 2024-04-25T10:11:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe502353640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc071ed0 con 0x7fe4fc0aecf0 2024-04-25T10:11:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4fbfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe4fc15d980 0x7fe4fc0ae5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42154/0 (socket says 172.21.15.16:42154) 2024-04-25T10:11:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4fbfff640 1 -- 172.21.15.16:0/369009574 learned_addr learned my addr 172.21.15.16:0/369009574 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4fb7fe640 1 --2- 172.21.15.16:0/369009574 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4fc0aecf0 0x7fe4fc0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4faffd640 1 -- 172.21.15.16:0/369009574 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1694437102 0 0) 0x7fe4fc0712c0 con 0x7fe4fc15d980 2024-04-25T10:11:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4faffd640 1 -- 172.21.15.16:0/369009574 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4e4003520 con 0x7fe4fc15d980 2024-04-25T10:11:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4faffd640 1 -- 172.21.15.16:0/369009574 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187171374 0 0) 0x7fe4fc0715a0 con 0x7fe4fc15ed70 2024-04-25T10:11:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4faffd640 1 -- 172.21.15.16:0/369009574 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4fc0712c0 con 0x7fe4fc15ed70 2024-04-25T10:11:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4faffd640 1 -- 172.21.15.16:0/369009574 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 596478926 0 0) 0x7fe4e4003520 con 0x7fe4fc15d980 2024-04-25T10:11:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4faffd640 1 -- 172.21.15.16:0/369009574 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4fc0aecf0 msgr2=0x7fe4fc0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4faffd640 1 --2- 172.21.15.16:0/369009574 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4fc0aecf0 0x7fe4fc0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4faffd640 1 -- 172.21.15.16:0/369009574 >> v1:172.21.15.186:6789/0 conn(0x7fe4fc15ed70 legacy=0x7fe4fc0acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4faffd640 1 -- 172.21.15.16:0/369009574 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc071880 con 0x7fe4fc15d980 2024-04-25T10:11:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4faffd640 1 -- 172.21.15.16:0/369009574 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fe4f0001670 con 0x7fe4fc15d980 2024-04-25T10:11:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4fb7fe640 1 --2- 172.21.15.16:0/369009574 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4fc0aecf0 0x7fe4fc0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4faffd640 1 -- 172.21.15.16:0/369009574 >> v1:172.21.15.16:6789/0 conn(0x7fe4fc15d980 legacy=0x7fe4fc0ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4faffd640 1 --2- 172.21.15.16:0/369009574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4e4003b70 0x7fe4e4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4faffd640 1 --2- 172.21.15.16:0/369009574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e4004800 0x7fe4e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4faffd640 1 --2- 172.21.15.16:0/369009574 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4e40051c0 0x7fe4e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.385+0000 7fe4faffd640 1 -- 172.21.15.16:0/369009574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc071ed0 con 0x7fe4e4004800 2024-04-25T10:11:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.386+0000 7fe4faffd640 1 -- 172.21.15.16:0/369009574 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e400bf00 con 0x7fe4e4003b70 2024-04-25T10:11:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.386+0000 7fe4faffd640 1 -- 172.21.15.16:0/369009574 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e400c080 con 0x7fe4e40051c0 2024-04-25T10:11:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.386+0000 7fe4fb7fe640 1 --2- 172.21.15.16:0/369009574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e4004800 0x7fe4e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.386+0000 7fe4fbfff640 1 --2- 172.21.15.16:0/369009574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4e4003b70 0x7fe4e4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.386+0000 7fe5008f0640 1 --2- 172.21.15.16:0/369009574 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4e40051c0 0x7fe4e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.386+0000 7fe4fb7fe640 1 -- 172.21.15.16:0/369009574 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4e40051c0 msgr2=0x7fe4e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.386+0000 7fe4fb7fe640 1 --2- 172.21.15.16:0/369009574 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4e40051c0 0x7fe4e400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.386+0000 7fe4fb7fe640 1 -- 172.21.15.16:0/369009574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4e4003b70 msgr2=0x7fe4e4003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.386+0000 7fe4fb7fe640 1 --2- 172.21.15.16:0/369009574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4e4003b70 0x7fe4e4003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.386+0000 7fe4fb7fe640 1 -- 172.21.15.16:0/369009574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e4003520 con 0x7fe4e4004800 2024-04-25T10:11:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.386+0000 7fe5008f0640 1 --2- 172.21.15.16:0/369009574 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4e40051c0 0x7fe4e400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.387+0000 7fe4fb7fe640 1 --2- 172.21.15.16:0/369009574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e4004800 0x7fe4e4004c80 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009c60 tx=0x7fe4e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6de6768ab37681f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.387+0000 7fe4faffd640 1 -- 172.21.15.16:0/369009574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec036070 con 0x7fe4e4004800 2024-04-25T10:11:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.387+0000 7fe4faffd640 1 -- 172.21.15.16:0/369009574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec031400 con 0x7fe4e4004800 2024-04-25T10:11:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.387+0000 7fe4faffd640 1 -- 172.21.15.16:0/369009574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec009860 con 0x7fe4e4004800 2024-04-25T10:11:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.387+0000 7fe502353640 1 -- 172.21.15.16:0/369009574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e4004800 msgr2=0x7fe4e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.387+0000 7fe502353640 1 --2- 172.21.15.16:0/369009574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e4004800 0x7fe4e4004c80 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009c60 tx=0x7fe4e4005830 comp rx=0 tx=0).stop 2024-04-25T10:11:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.387+0000 7fe5008f0640 1 -- 172.21.15.16:0/369009574 reap_dead start 2024-04-25T10:11:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.387+0000 7fe502353640 1 -- 172.21.15.16:0/369009574 shutdown_connections 2024-04-25T10:11:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.387+0000 7fe502353640 1 -- 172.21.15.16:0/369009574 >> 172.21.15.16:0/369009574 conn(0x7fe4fc06aba0 msgr2=0x7fe4fc06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.387+0000 7fe502353640 1 -- 172.21.15.16:0/369009574 shutdown_connections 2024-04-25T10:11:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.387+0000 7fe502353640 1 -- 172.21.15.16:0/369009574 wait complete. 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.388+0000 7fe502353640 1 Processor -- start 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.389+0000 7fe502353640 1 -- start start 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.389+0000 7fe502353640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e4004800 0x7fe4fc152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.389+0000 7fe502353640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4fc14ecb0 0x7fe4fc14f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.389+0000 7fe502353640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4fc155190 0x7fe4fc1555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.389+0000 7fe502353640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0cf400 con 0x7fe4e4004800 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.389+0000 7fe502353640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0cf280 con 0x7fe4fc14ecb0 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.389+0000 7fe502353640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0cf580 con 0x7fe4fc155190 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.389+0000 7fe4fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e4004800 0x7fe4fc152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.389+0000 7fe4fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e4004800 0x7fe4fc152b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53014/0 (socket says 172.21.15.16:53014) 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.389+0000 7fe4fbfff640 1 -- 172.21.15.16:0/2128046895 learned_addr learned my addr 172.21.15.16:0/2128046895 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.390+0000 7fe4fb7fe640 1 --2- 172.21.15.16:0/2128046895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4fc14ecb0 0x7fe4fc14f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.390+0000 7fe4fbfff640 1 -- 172.21.15.16:0/2128046895 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4fc155190 msgr2=0x7fe4fc1555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.390+0000 7fe4fbfff640 1 --2- 172.21.15.16:0/2128046895 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4fc155190 0x7fe4fc1555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.390+0000 7fe4fbfff640 1 -- 172.21.15.16:0/2128046895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4fc14ecb0 msgr2=0x7fe4fc14f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.390+0000 7fe4fbfff640 1 --2- 172.21.15.16:0/2128046895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4fc14ecb0 0x7fe4fc14f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.390+0000 7fe4fbfff640 1 -- 172.21.15.16:0/2128046895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc153770 con 0x7fe4e4004800 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.390+0000 7fe4fbfff640 1 --2- 172.21.15.16:0/2128046895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e4004800 0x7fe4fc152b90 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00072c0 tx=0x7fe4f0007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.390+0000 7fe4f8ff9640 1 -- 172.21.15.16:0/2128046895 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0001650 con 0x7fe4e4004800 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.390+0000 7fe4f8ff9640 1 -- 172.21.15.16:0/2128046895 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f0018070 con 0x7fe4e4004800 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.390+0000 7fe502353640 1 -- 172.21.15.16:0/2128046895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4fc153a50 con 0x7fe4e4004800 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.390+0000 7fe4f8ff9640 1 -- 172.21.15.16:0/2128046895 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f000fba0 con 0x7fe4e4004800 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.390+0000 7fe502353640 1 -- 172.21.15.16:0/2128046895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc154010 con 0x7fe4e4004800 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.391+0000 7fe502353640 1 -- 172.21.15.16:0/2128046895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4fc154210 con 0x7fe4e4004800 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.392+0000 7fe4f8ff9640 1 -- 172.21.15.16:0/2128046895 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe4f000fd40 con 0x7fe4e4004800 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.392+0000 7fe4f8ff9640 1 --2- 172.21.15.16:0/2128046895 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe4e806b500 0x7fe4e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.392+0000 7fe4f8ff9640 1 -- 172.21.15.16:0/2128046895 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe4f008a230 con 0x7fe4e4004800 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.392+0000 7fe4fb7fe640 1 --2- 172.21.15.16:0/2128046895 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe4e806b500 0x7fe4e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.393+0000 7fe4fb7fe640 1 --2- 172.21.15.16:0/2128046895 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe4e806b500 0x7fe4e806d9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec033000 tx=0x7fe4ec008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.396+0000 7fe4f8ff9640 1 -- 172.21.15.16:0/2128046895 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe4fc154210 con 0x7fe4e4004800 2024-04-25T10:11:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.613+0000 7fe502353640 1 -- 172.21.15.16:0/2128046895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe4fc06d4c0 con 0x7fe4e4004800 2024-04-25T10:11:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.613+0000 7fe4f8ff9640 1 -- 172.21.15.16:0/2128046895 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe4f0059050 con 0x7fe4e4004800 2024-04-25T10:11:19.618 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:11:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.615+0000 7fe502353640 1 -- 172.21.15.16:0/2128046895 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe4e806b500 msgr2=0x7fe4e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.615+0000 7fe502353640 1 --2- 172.21.15.16:0/2128046895 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe4e806b500 0x7fe4e806d9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec033000 tx=0x7fe4ec008030 comp rx=0 tx=0).stop 2024-04-25T10:11:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.615+0000 7fe502353640 1 -- 172.21.15.16:0/2128046895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e4004800 msgr2=0x7fe4fc152b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.615+0000 7fe502353640 1 --2- 172.21.15.16:0/2128046895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e4004800 0x7fe4fc152b90 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00072c0 tx=0x7fe4f0007780 comp rx=0 tx=0).stop 2024-04-25T10:11:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.615+0000 7fe502353640 1 -- 172.21.15.16:0/2128046895 shutdown_connections 2024-04-25T10:11:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.615+0000 7fe502353640 1 --2- 172.21.15.16:0/2128046895 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe4e806b500 0x7fe4e806d9c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.615+0000 7fe502353640 1 --2- 172.21.15.16:0/2128046895 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4fc155190 0x7fe4fc1555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.615+0000 7fe502353640 1 --2- 172.21.15.16:0/2128046895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4fc14ecb0 0x7fe4fc14f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.615+0000 7fe502353640 1 --2- 172.21.15.16:0/2128046895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e4004800 0x7fe4fc152b90 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.615+0000 7fe502353640 1 -- 172.21.15.16:0/2128046895 >> 172.21.15.16:0/2128046895 conn(0x7fe4fc06aba0 msgr2=0x7fe4fc1608e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.615+0000 7fe502353640 1 -- 172.21.15.16:0/2128046895 shutdown_connections 2024-04-25T10:11:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:19.615+0000 7fe502353640 1 -- 172.21.15.16:0/2128046895 wait complete. 2024-04-25T10:11:19.631 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:11:19.631 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:11:19.631 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-04-25T10:11:19.631 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:11:19.631 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:19.631 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:11:19.631 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:11:19.632 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:11:19.632 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:19.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:19.632 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-04-25T10:11:19.648 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:11:19.648 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:11:19.648 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:11:19.649 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:19.649 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:11:19.649 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:19.649 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:19.649 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:19.649 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-04-25T10:11:19.649 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:19.649 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:19.649 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-04-25T10:11:19.649 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:11:19.649 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:19.649 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-04-25T10:11:19.650 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:11:19.650 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42281sec preferred_lft 42281sec 2024-04-25T10:11:19.650 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-25T10:11:19.650 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:19.650 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:19.650 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:19.650 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-04-25T10:11:19.650 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:11:19.650 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:19.650 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:11:19.650 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:19.650 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f06b:fdff:fed4:a335/64 scope link 2024-04-25T10:11:19.650 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:19.651 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:19.651 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:19.651 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::90ad:4fff:fe91:3c33/64 scope link 2024-04-25T10:11:19.651 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:19.651 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:19.651 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether be:14:4a:3c:16:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:19.651 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::bc14:4aff:fe3c:1605/64 scope link 2024-04-25T10:11:19.651 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:19.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:19.652 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:11:19.713 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:11:19.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:19.715 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-04-25T10:11:19.771 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-04-25T10:11:19.771 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:11:19.771 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:11:19.771 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:11:19.771 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:11:19.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:19.772 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:11:19.772 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:11:19.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:11:19.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:11:19.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:11:19.772 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:11:19.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:19.857 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:11:19.873 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:11:19.873 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:11:19.873 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:19.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:19.873 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-04-25T10:11:19.937 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:19.938 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:19.993 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:19.994 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:19.994 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:20.049 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:11:20.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:20.049 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:20.132 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:20.132 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:20.132 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:11:20.140 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:20.140 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:20.140 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:11:20.140 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:11:20.140 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:11:20.160 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:20.160 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:20.160 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:11:20.160 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:11:20.160 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:11:20.160 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:11:20.160 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:11:20.160 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:11:20.161 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:11:20.161 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:11:20.161 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:11:20.161 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:11:20.161 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:11:20.162 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:11:20.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:20.162 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:20.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:20.226 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:11:20.283 INFO:teuthology.orchestra.run.smithi016.stdout:client6775 2024-04-25T10:11:20.284 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:11:20.284 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:20.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:20.340 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-04-25T10:11:20.398 INFO:tasks.ceph.mds.d.smithi186.stderr:2024-04-25T10:11:20.392+0000 7f98024a4640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-25T10:11:20.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:11:20.399 INFO:teuthology.orchestra.run.smithi016.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-04-25T10:11:20.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:20.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-04-25T10:11:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.538+0000 7f35778e1640 1 Processor -- start 2024-04-25T10:11:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.539+0000 7f35778e1640 1 -- start start 2024-04-25T10:11:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.539+0000 7f35778e1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35700b0740 0x7f35700b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.539+0000 7f35778e1640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f35700cfd60 0x7f35700d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.539+0000 7f35778e1640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f357015f660 con 0x7f357015ef60 2024-04-25T10:11:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.539+0000 7f35778e1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f357015f940 con 0x7f35700b0740 2024-04-25T10:11:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.539+0000 7f35778e1640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f357015fc20 con 0x7f35700cfd60 2024-04-25T10:11:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.540+0000 7f3574e7c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35700b0740 0x7f35700b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.540+0000 7f3574e7c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35700b0740 0x7f35700b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53024/0 (socket says 172.21.15.16:53024) 2024-04-25T10:11:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.540+0000 7f3574e7c640 1 -- 172.21.15.16:0/3064431259 learned_addr learned my addr 172.21.15.16:0/3064431259 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.540+0000 7f3575e7e640 1 --2- 172.21.15.16:0/3064431259 >> v2:172.21.15.186:3300/0 conn(0x7f35700cfd60 0x7f35700d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.540+0000 7f3574e7c640 1 -- 172.21.15.16:0/3064431259 >> v2:172.21.15.186:3300/0 conn(0x7f35700cfd60 msgr2=0x7f35700d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.540+0000 7f3574e7c640 1 --2- 172.21.15.16:0/3064431259 >> v2:172.21.15.186:3300/0 conn(0x7f35700cfd60 0x7f35700d0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.540+0000 7f3574e7c640 1 -- 172.21.15.16:0/3064431259 >> v1:172.21.15.186:6789/0 conn(0x7f357015ef60 legacy=0x7f35700ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.540+0000 7f3574e7c640 1 -- 172.21.15.16:0/3064431259 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357015ff00 con 0x7f35700b0740 2024-04-25T10:11:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.541+0000 7f3574e7c640 1 --2- 172.21.15.16:0/3064431259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35700b0740 0x7f35700b0bd0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f35700700e0 tx=0x7f356002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef412d2f82fa25c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.541+0000 7f3567fff640 1 -- 172.21.15.16:0/3064431259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3560037070 con 0x7f35700b0740 2024-04-25T10:11:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.541+0000 7f3567fff640 1 -- 172.21.15.16:0/3064431259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3560032440 con 0x7f35700b0740 2024-04-25T10:11:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.541+0000 7f3567fff640 1 -- 172.21.15.16:0/3064431259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3560032810 con 0x7f35700b0740 2024-04-25T10:11:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.541+0000 7f35778e1640 1 -- 172.21.15.16:0/3064431259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35700b0740 msgr2=0x7f35700b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.541+0000 7f35778e1640 1 --2- 172.21.15.16:0/3064431259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35700b0740 0x7f35700b0bd0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f35700700e0 tx=0x7f356002a2c0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.541+0000 7f35778e1640 1 -- 172.21.15.16:0/3064431259 shutdown_connections 2024-04-25T10:11:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.541+0000 7f35778e1640 1 --2- 172.21.15.16:0/3064431259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35700b0740 0x7f35700b0bd0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.541+0000 7f35778e1640 1 --2- 172.21.15.16:0/3064431259 >> v2:172.21.15.186:3300/0 conn(0x7f35700cfd60 0x7f35700d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.541+0000 7f35778e1640 1 -- 172.21.15.16:0/3064431259 >> 172.21.15.16:0/3064431259 conn(0x7f357006f4c0 msgr2=0x7f357006f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.541+0000 7f35778e1640 1 -- 172.21.15.16:0/3064431259 shutdown_connections 2024-04-25T10:11:20.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.541+0000 7f35778e1640 1 -- 172.21.15.16:0/3064431259 wait complete. 2024-04-25T10:11:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.542+0000 7f35778e1640 1 Processor -- start 2024-04-25T10:11:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.543+0000 7f35778e1640 1 -- start start 2024-04-25T10:11:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.543+0000 7f35778e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35700b0740 0x7f3570147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.543+0000 7f35778e1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35700cfd60 0x7f35700b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.543+0000 7f35778e1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f357015ef60 0x7f3570149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.543+0000 7f35778e1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357015f340 con 0x7f35700b0740 2024-04-25T10:11:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.543+0000 7f35778e1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35701604c0 con 0x7f357015ef60 2024-04-25T10:11:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.543+0000 7f35778e1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35700cebd0 con 0x7f35700cfd60 2024-04-25T10:11:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.543+0000 7f357567d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35700b0740 0x7f3570147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.543+0000 7f357567d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35700b0740 0x7f3570147670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53026/0 (socket says 172.21.15.16:53026) 2024-04-25T10:11:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.543+0000 7f357567d640 1 -- 172.21.15.16:0/3218270351 learned_addr learned my addr 172.21.15.16:0/3218270351 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.544+0000 7f3574e7c640 1 --2- 172.21.15.16:0/3218270351 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35700cfd60 0x7f35700b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.544+0000 7f3575e7e640 1 --2- 172.21.15.16:0/3218270351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f357015ef60 0x7f3570149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.544+0000 7f357567d640 1 -- 172.21.15.16:0/3218270351 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35700cfd60 msgr2=0x7f35700b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.544+0000 7f357567d640 1 --2- 172.21.15.16:0/3218270351 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35700cfd60 0x7f35700b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.544+0000 7f357567d640 1 -- 172.21.15.16:0/3218270351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f357015ef60 msgr2=0x7f3570149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.544+0000 7f357567d640 1 --2- 172.21.15.16:0/3218270351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f357015ef60 0x7f3570149e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.544+0000 7f357567d640 1 -- 172.21.15.16:0/3218270351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35700b1420 con 0x7f35700b0740 2024-04-25T10:11:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.544+0000 7f357567d640 1 --2- 172.21.15.16:0/3218270351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35700b0740 0x7f3570147670 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f356c00b420 tx=0x7f356c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.544+0000 7f3565ffb640 1 -- 172.21.15.16:0/3218270351 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c011830 con 0x7f35700b0740 2024-04-25T10:11:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.544+0000 7f3565ffb640 1 -- 172.21.15.16:0/3218270351 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356c016070 con 0x7f35700b0740 2024-04-25T10:11:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.544+0000 7f35778e1640 1 -- 172.21.15.16:0/3218270351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35700b3580 con 0x7f35700b0740 2024-04-25T10:11:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.544+0000 7f3565ffb640 1 -- 172.21.15.16:0/3218270351 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c011c10 con 0x7f35700b0740 2024-04-25T10:11:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.544+0000 7f35778e1640 1 -- 172.21.15.16:0/3218270351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35700b3b40 con 0x7f35700b0740 2024-04-25T10:11:20.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.546+0000 7f3565ffb640 1 -- 172.21.15.16:0/3218270351 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f356c01b460 con 0x7f35700b0740 2024-04-25T10:11:20.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.546+0000 7f3565ffb640 1 --2- 172.21.15.16:0/3218270351 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f355406b4d0 0x7f355406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.546+0000 7f3565ffb640 1 -- 172.21.15.16:0/3218270351 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f356c089cb0 con 0x7f35700b0740 2024-04-25T10:11:20.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.547+0000 7f3574e7c640 1 --2- 172.21.15.16:0/3218270351 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f355406b4d0 0x7f355406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.547+0000 7f3574e7c640 1 --2- 172.21.15.16:0/3218270351 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f355406b4d0 0x7f355406d990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3560034000 tx=0x7f356002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.547+0000 7f357a5e8740 1 Processor -- start 2024-04-25T10:11:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.547+0000 7f357a5e8740 1 -- start start 2024-04-25T10:11:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.548+0000 7f357a5e8740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de2080 0x55c3a3de4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.548+0000 7f357a5e8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2460 0x55c3a3de4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.548+0000 7f357a5e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3db4ce0 0x55c3a3db84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.548+0000 7f357a5e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a3d1daf0 con 0x55c3a3db4ce0 2024-04-25T10:11:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.548+0000 7f357a5e8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a3cc6de0 con 0x55c3a3de2460 2024-04-25T10:11:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.548+0000 7f357a5e8740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3a3d19b90 con 0x55c3a3de2080 2024-04-25T10:11:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.548+0000 7f3575e7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3db4ce0 0x55c3a3db84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.548+0000 7f3575e7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3db4ce0 0x55c3a3db84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53028/0 (socket says 172.21.15.16:53028) 2024-04-25T10:11:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.548+0000 7f3575e7e640 1 -- 172.21.15.16:0/537438707 learned_addr learned my addr 172.21.15.16:0/537438707 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.548+0000 7f3574e7c640 1 --2- 172.21.15.16:0/537438707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2460 0x55c3a3de4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.548+0000 7f357567d640 1 --2- 172.21.15.16:0/537438707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de2080 0x55c3a3de4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.549+0000 7f3575e7e640 1 -- 172.21.15.16:0/537438707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de2080 msgr2=0x55c3a3de4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.549+0000 7f3575e7e640 1 --2- 172.21.15.16:0/537438707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de2080 0x55c3a3de4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.549+0000 7f3575e7e640 1 -- 172.21.15.16:0/537438707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2460 msgr2=0x55c3a3de4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.549+0000 7f3575e7e640 1 --2- 172.21.15.16:0/537438707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2460 0x55c3a3de4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.549+0000 7f3575e7e640 1 -- 172.21.15.16:0/537438707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3a3afe7b0 con 0x55c3a3db4ce0 2024-04-25T10:11:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.549+0000 7f3575e7e640 1 --2- 172.21.15.16:0/537438707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3db4ce0 0x55c3a3db84f0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f3568007400 tx=0x7f35680078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1186fe98254db391 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.549+0000 7f355affd640 1 -- 172.21.15.16:0/537438707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35680147e0 con 0x55c3a3db4ce0 2024-04-25T10:11:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.549+0000 7f355affd640 1 -- 172.21.15.16:0/537438707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356801a070 con 0x55c3a3db4ce0 2024-04-25T10:11:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.549+0000 7f355affd640 1 -- 172.21.15.16:0/537438707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3568014ba0 con 0x55c3a3db4ce0 2024-04-25T10:11:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.549+0000 7f357a5e8740 1 -- 172.21.15.16:0/537438707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3db4ce0 msgr2=0x55c3a3db84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.549+0000 7f357a5e8740 1 --2- 172.21.15.16:0/537438707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3db4ce0 0x55c3a3db84f0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f3568007400 tx=0x7f35680078c0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.550+0000 7f357a5e8740 1 -- 172.21.15.16:0/537438707 shutdown_connections 2024-04-25T10:11:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.550+0000 7f357a5e8740 1 --2- 172.21.15.16:0/537438707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3db4ce0 0x55c3a3db84f0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.550+0000 7f357a5e8740 1 --2- 172.21.15.16:0/537438707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2460 0x55c3a3de4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.550+0000 7f357a5e8740 1 --2- 172.21.15.16:0/537438707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de2080 0x55c3a3de4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.550+0000 7f357a5e8740 1 -- 172.21.15.16:0/537438707 >> 172.21.15.16:0/537438707 conn(0x55c3a3db6f50 msgr2=0x55c3a3dff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.550+0000 7f357a5e8740 1 -- 172.21.15.16:0/537438707 shutdown_connections 2024-04-25T10:11:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.550+0000 7f357a5e8740 1 -- 172.21.15.16:0/537438707 wait complete. 2024-04-25T10:11:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.550+0000 7f357a5e8740 1 -- start start 2024-04-25T10:11:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.550+0000 7f357a5e8740 1 Processor -- start 2024-04-25T10:11:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.551+0000 7f357a5e8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2080 0x55c3a3e0f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.551+0000 7f357a5e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de2460 0x55c3a3e10430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.551+0000 7f357a5e8740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de0700 0x55c3a3de0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.551+0000 7f357a5e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a3dba100 con 0x55c3a3de2460 2024-04-25T10:11:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.551+0000 7f357a5e8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a3cbc3e0 con 0x55c3a3de2080 2024-04-25T10:11:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.551+0000 7f357a5e8740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3a3dba3e0 con 0x55c3a3de0700 2024-04-25T10:11:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.551+0000 7f357a5e8740 10 client.0 _ll_register_callbacks cb 0x55c3a3cd9d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.551+0000 7f3574e7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de2460 0x55c3a3e10430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.551+0000 7f3574e7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de2460 0x55c3a3e10430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53042/0 (socket says 172.21.15.16:53042) 2024-04-25T10:11:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.551+0000 7f3574e7c640 1 -- 172.21.15.16:0/1732512150 learned_addr learned my addr 172.21.15.16:0/1732512150 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.551+0000 7f357567d640 1 --2- 172.21.15.16:0/1732512150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2080 0x55c3a3e0f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.551+0000 7f3575e7e640 1 --2- 172.21.15.16:0/1732512150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de0700 0x55c3a3de0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f3574e7c640 1 -- 172.21.15.16:0/1732512150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de0700 msgr2=0x55c3a3de0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f3574e7c640 1 --2- 172.21.15.16:0/1732512150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de0700 0x55c3a3de0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f3574e7c640 1 -- 172.21.15.16:0/1732512150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2080 msgr2=0x55c3a3e0f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f3574e7c640 1 --2- 172.21.15.16:0/1732512150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2080 0x55c3a3e0f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f3574e7c640 1 -- 172.21.15.16:0/1732512150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3a3b5ef40 con 0x55c3a3de2460 2024-04-25T10:11:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f357567d640 1 --2- 172.21.15.16:0/1732512150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2080 0x55c3a3e0f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f3574e7c640 1 --2- 172.21.15.16:0/1732512150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de2460 0x55c3a3e10430 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f3560009960 tx=0x7f356002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64ffb33c0ad534b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f354ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:11:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f354ffff640 1 -- 172.21.15.16:0/1732512150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356004e060 con 0x55c3a3de2460 2024-04-25T10:11:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f354ffff640 1 -- 172.21.15.16:0/1732512150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f356003ed40 con 0x55c3a3de2460 2024-04-25T10:11:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f357a5e8740 1 -- 172.21.15.16:0/1732512150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f356801a070 con 0x55c3a3de2460 2024-04-25T10:11:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f354ffff640 1 -- 172.21.15.16:0/1732512150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3560054050 con 0x55c3a3de2460 2024-04-25T10:11:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f357a5e8740 10 client.7650 fetch_fsmap learned FSMap version 57 2024-04-25T10:11:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f357a5e8740 1 -- 172.21.15.16:0/1732512150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55c3a39f20a0 con 0x55c3a3de2460 2024-04-25T10:11:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f354ffff640 1 -- 172.21.15.16:0/1732512150 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356801a070 con 0x55c3a3de2460 2024-04-25T10:11:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f354ffff640 1 -- 172.21.15.16:0/1732512150 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3560057020 con 0x55c3a3de2460 2024-04-25T10:11:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f357a5e8740 10 client.7650 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:11:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f357a5e8740 10 client.7650 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T10:11:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f357a5e8740 20 client.7650 populate_metadata read hostname 'smithi016' 2024-04-25T10:11:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f357a5e8740 1 --2- 172.21.15.16:0/1732512150 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55c3a3e12450 0x55c3a3e12880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f357a5e8740 4 client.7650 mds_command: new command op to 7335 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.552+0000 7f357a5e8740 1 -- 172.21.15.16:0/1732512150 --> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3a3d2b030 con 0x55c3a3e12450 2024-04-25T10:11:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.553+0000 7f357567d640 1 --2- 172.21.15.16:0/1732512150 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55c3a3e12450 0x55c3a3e12880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.553+0000 7f354ffff640 10 client.7650 ms_handle_connect on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.553+0000 7f357567d640 1 --2- 172.21.15.16:0/1732512150 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55c3a3e12450 0x55c3a3e12880 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2ffe2614ed26b8c server_cookie=f55cf5cafea9bb7f in_seq=0 out_seq=0 2024-04-25T10:11:20.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.554+0000 7f354ffff640 1 -- 172.21.15.16:0/1732512150 <== mds.0 v2:172.21.15.186:6834/3950732718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c3a3d2b030 con 0x55c3a3e12450 2024-04-25T10:11:20.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.554+0000 7f354ffff640 10 client.7650 handle_command_reply: tid=0 2024-04-25T10:11:20.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.554+0000 7f357a5e8740 1 client.7650 shutdown 2024-04-25T10:11:20.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.554+0000 7f357a5e8740 1 -- 172.21.15.16:0/1732512150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de2460 msgr2=0x55c3a3e10430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.554+0000 7f357a5e8740 1 --2- 172.21.15.16:0/1732512150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de2460 0x55c3a3e10430 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f3560009960 tx=0x7f356002c070 comp rx=0 tx=0).stop 2024-04-25T10:11:20.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.554+0000 7f357a5e8740 1 -- 172.21.15.16:0/1732512150 shutdown_connections 2024-04-25T10:11:20.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.554+0000 7f357a5e8740 1 --2- 172.21.15.16:0/1732512150 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55c3a3e12450 0x55c3a3e12880 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.554+0000 7f357a5e8740 1 --2- 172.21.15.16:0/1732512150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de0700 0x55c3a3de0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.554+0000 7f357a5e8740 1 --2- 172.21.15.16:0/1732512150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de2460 0x55c3a3e10430 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.554+0000 7f357a5e8740 1 --2- 172.21.15.16:0/1732512150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2080 0x55c3a3e0f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.554+0000 7f354ffff640 0 client.7650 ms_handle_reset on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:20.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.554+0000 7f357a5e8740 1 -- 172.21.15.16:0/1732512150 >> 172.21.15.16:0/1732512150 conn(0x55c3a3db4ce0 msgr2=0x55c3a3de1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.554+0000 7f357a5e8740 1 -- 172.21.15.16:0/1732512150 shutdown_connections 2024-04-25T10:11:20.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.554+0000 7f357a5e8740 1 -- 172.21.15.16:0/1732512150 wait complete. 2024-04-25T10:11:20.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.554+0000 7f357a5e8740 20 client.7650 trim_cache size 0 max 16384 2024-04-25T10:11:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.569+0000 7f357a5e8740 1 Processor -- start 2024-04-25T10:11:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.569+0000 7f357a5e8740 1 -- start start 2024-04-25T10:11:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.570+0000 7f357a5e8740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de0700 0x55c3a3e07810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.570+0000 7f357a5e8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2080 0x55c3a3de4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.570+0000 7f357a5e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de2460 0x55c3a3ddeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.570+0000 7f357a5e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35701fde10 con 0x55c3a3de2460 2024-04-25T10:11:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.570+0000 7f357a5e8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35701fd120 con 0x55c3a3de2080 2024-04-25T10:11:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.570+0000 7f357a5e8740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357015cb00 con 0x55c3a3de0700 2024-04-25T10:11:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.570+0000 7f3575e7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de2460 0x55c3a3ddeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.570+0000 7f3575e7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de2460 0x55c3a3ddeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53044/0 (socket says 172.21.15.16:53044) 2024-04-25T10:11:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.570+0000 7f3575e7e640 1 -- 172.21.15.16:0/1710146654 learned_addr learned my addr 172.21.15.16:0/1710146654 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.570+0000 7f3575e7e640 1 -- 172.21.15.16:0/1710146654 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de0700 msgr2=0x55c3a3e07810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.570+0000 7f3575e7e640 1 --2- 172.21.15.16:0/1710146654 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de0700 0x55c3a3e07810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.570+0000 7f3575e7e640 1 -- 172.21.15.16:0/1710146654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2080 msgr2=0x55c3a3de4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.570+0000 7f3575e7e640 1 --2- 172.21.15.16:0/1710146654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2080 0x55c3a3de4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.570+0000 7f3575e7e640 1 -- 172.21.15.16:0/1710146654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3a3dfc6f0 con 0x55c3a3de2460 2024-04-25T10:11:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.571+0000 7f3575e7e640 1 --2- 172.21.15.16:0/1710146654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de2460 0x55c3a3ddeaf0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f3568008010 tx=0x7f3568014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b0fdd76d10b8cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.571+0000 7f355a7fc640 1 -- 172.21.15.16:0/1710146654 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356802e070 con 0x55c3a3de2460 2024-04-25T10:11:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.571+0000 7f355a7fc640 1 -- 172.21.15.16:0/1710146654 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356801a040 con 0x55c3a3de2460 2024-04-25T10:11:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.571+0000 7f355a7fc640 1 -- 172.21.15.16:0/1710146654 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3568014a20 con 0x55c3a3de2460 2024-04-25T10:11:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.571+0000 7f357a5e8740 1 -- 172.21.15.16:0/1710146654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de2460 msgr2=0x55c3a3ddeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.571+0000 7f357a5e8740 1 --2- 172.21.15.16:0/1710146654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de2460 0x55c3a3ddeaf0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f3568008010 tx=0x7f3568014000 comp rx=0 tx=0).stop 2024-04-25T10:11:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.571+0000 7f357a5e8740 1 -- 172.21.15.16:0/1710146654 shutdown_connections 2024-04-25T10:11:20.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.571+0000 7f357a5e8740 1 --2- 172.21.15.16:0/1710146654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de2460 0x55c3a3ddeaf0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.571+0000 7f357a5e8740 1 --2- 172.21.15.16:0/1710146654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2080 0x55c3a3de4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.571+0000 7f357a5e8740 1 --2- 172.21.15.16:0/1710146654 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de0700 0x55c3a3e07810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.571+0000 7f357a5e8740 1 -- 172.21.15.16:0/1710146654 >> 172.21.15.16:0/1710146654 conn(0x55c3a3db4ce0 msgr2=0x55c3a3db03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.571+0000 7f357a5e8740 1 -- 172.21.15.16:0/1710146654 shutdown_connections 2024-04-25T10:11:20.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.571+0000 7f357a5e8740 1 -- 172.21.15.16:0/1710146654 wait complete. 2024-04-25T10:11:20.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.572+0000 7f357a5e8740 1 -- start start 2024-04-25T10:11:20.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.572+0000 7f357a5e8740 1 Processor -- start 2024-04-25T10:11:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f357a5e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de0700 0x55c3a3df5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f357a5e8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2080 0x55c3a3db9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f357a5e8740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de2460 0x55c3a3ddeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f357a5e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a3d2a170 con 0x55c3a3de0700 2024-04-25T10:11:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f357a5e8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357015d100 con 0x55c3a3de2080 2024-04-25T10:11:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f357a5e8740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3a3cddc60 con 0x55c3a3de2460 2024-04-25T10:11:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f357a5e8740 10 client.0 _ll_register_callbacks cb 0x7f35600096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f357567d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de0700 0x55c3a3df5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f357567d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de0700 0x55c3a3df5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53050/0 (socket says 172.21.15.16:53050) 2024-04-25T10:11:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f357567d640 1 -- 172.21.15.16:0/2640571764 learned_addr learned my addr 172.21.15.16:0/2640571764 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f3574e7c640 1 --2- 172.21.15.16:0/2640571764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2080 0x55c3a3db9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f3575e7e640 1 --2- 172.21.15.16:0/2640571764 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de2460 0x55c3a3ddeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f357567d640 1 -- 172.21.15.16:0/2640571764 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de2460 msgr2=0x55c3a3ddeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f357567d640 1 --2- 172.21.15.16:0/2640571764 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de2460 0x55c3a3ddeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f357567d640 1 -- 172.21.15.16:0/2640571764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2080 msgr2=0x55c3a3db9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f357567d640 1 --2- 172.21.15.16:0/2640571764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2080 0x55c3a3db9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f357567d640 1 -- 172.21.15.16:0/2640571764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3a3ddad40 con 0x55c3a3de0700 2024-04-25T10:11:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f357567d640 1 --2- 172.21.15.16:0/2640571764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de0700 0x55c3a3df5ce0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f356c0580d0 tx=0x7f356c054670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fca2680a223868d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.573+0000 7f354ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:11:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.574+0000 7f354ffff640 1 -- 172.21.15.16:0/2640571764 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c060650 con 0x55c3a3de0700 2024-04-25T10:11:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.574+0000 7f354ffff640 1 -- 172.21.15.16:0/2640571764 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f356c0607f0 con 0x55c3a3de0700 2024-04-25T10:11:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.574+0000 7f357a5e8740 1 -- 172.21.15.16:0/2640571764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f356801a040 con 0x55c3a3de0700 2024-04-25T10:11:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.574+0000 7f354ffff640 1 -- 172.21.15.16:0/2640571764 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f356c057820 con 0x55c3a3de0700 2024-04-25T10:11:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.574+0000 7f357a5e8740 10 client.7656 fetch_fsmap learned FSMap version 57 2024-04-25T10:11:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.574+0000 7f357a5e8740 1 -- 172.21.15.16:0/2640571764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55c3a3ddb640 con 0x55c3a3de0700 2024-04-25T10:11:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.574+0000 7f354ffff640 1 -- 172.21.15.16:0/2640571764 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356801a040 con 0x55c3a3de0700 2024-04-25T10:11:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.574+0000 7f354ffff640 1 -- 172.21.15.16:0/2640571764 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f356c075070 con 0x55c3a3de0700 2024-04-25T10:11:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.574+0000 7f357a5e8740 10 client.7656 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:11:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.574+0000 7f357a5e8740 10 client.7656 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T10:11:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.574+0000 7f357a5e8740 20 client.7656 populate_metadata read hostname 'smithi016' 2024-04-25T10:11:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.574+0000 7f357a5e8740 1 --2- 172.21.15.16:0/2640571764 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55c3a3ddf030 0x55c3a3e73fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.574+0000 7f357a5e8740 4 client.7656 mds_command: new command op to 7335 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-04-25T10:11:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.574+0000 7f357a5e8740 1 -- 172.21.15.16:0/2640571764 --> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) v1 -- 0x55c3a3d25380 con 0x55c3a3ddf030 2024-04-25T10:11:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.574+0000 7f3574e7c640 1 --2- 172.21.15.16:0/2640571764 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55c3a3ddf030 0x55c3a3e73fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.575+0000 7f354ffff640 10 client.7656 ms_handle_connect on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.575+0000 7f3574e7c640 1 --2- 172.21.15.16:0/2640571764 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55c3a3ddf030 0x55c3a3e73fd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5309eaaf93f70101 server_cookie=d265c24fe3dc836e in_seq=0 out_seq=0 2024-04-25T10:11:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.575+0000 7f354ffff640 1 -- 172.21.15.16:0/2640571764 <== mds.0 v2:172.21.15.186:6834/3950732718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c3a3d25380 con 0x55c3a3ddf030 2024-04-25T10:11:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.575+0000 7f354ffff640 10 client.7656 handle_command_reply: tid=0 2024-04-25T10:11:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.575+0000 7f357a5e8740 1 client.7656 shutdown 2024-04-25T10:11:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.575+0000 7f357a5e8740 1 -- 172.21.15.16:0/2640571764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de0700 msgr2=0x55c3a3df5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.575+0000 7f357a5e8740 1 --2- 172.21.15.16:0/2640571764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de0700 0x55c3a3df5ce0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f356c0580d0 tx=0x7f356c054670 comp rx=0 tx=0).stop 2024-04-25T10:11:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.575+0000 7f357a5e8740 1 -- 172.21.15.16:0/2640571764 shutdown_connections 2024-04-25T10:11:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.575+0000 7f357a5e8740 1 --2- 172.21.15.16:0/2640571764 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55c3a3ddf030 0x55c3a3e73fd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.575+0000 7f357a5e8740 1 --2- 172.21.15.16:0/2640571764 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c3a3de2460 0x55c3a3ddeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.575+0000 7f354ffff640 0 client.7656 ms_handle_reset on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.575+0000 7f357a5e8740 1 --2- 172.21.15.16:0/2640571764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c3a3de2080 0x55c3a3db9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.575+0000 7f357a5e8740 1 --2- 172.21.15.16:0/2640571764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3a3de0700 0x55c3a3df5ce0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.575+0000 7f357a5e8740 1 -- 172.21.15.16:0/2640571764 >> 172.21.15.16:0/2640571764 conn(0x55c3a3db4ce0 msgr2=0x55c3a3de4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.575+0000 7f357a5e8740 1 -- 172.21.15.16:0/2640571764 shutdown_connections 2024-04-25T10:11:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.575+0000 7f357a5e8740 1 -- 172.21.15.16:0/2640571764 wait complete. 2024-04-25T10:11:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.575+0000 7f357a5e8740 20 client.7656 trim_cache size 0 max 16384 2024-04-25T10:11:20.588 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:11:20.588 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-04-25T10:11:20.589 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "c0af51ca-53be-4dde-84e5-e81e982a0f98", 2024-04-25T10:11:20.589 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-04-25T10:11:20.589 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:11:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.576+0000 7f35778e1640 1 -- 172.21.15.16:0/3218270351 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f355406b4d0 msgr2=0x7f355406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.576+0000 7f35778e1640 1 --2- 172.21.15.16:0/3218270351 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f355406b4d0 0x7f355406d990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3560034000 tx=0x7f356002e000 comp rx=0 tx=0).stop 2024-04-25T10:11:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.576+0000 7f35778e1640 1 -- 172.21.15.16:0/3218270351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35700b0740 msgr2=0x7f3570147670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.576+0000 7f35778e1640 1 --2- 172.21.15.16:0/3218270351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35700b0740 0x7f3570147670 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f356c00b420 tx=0x7f356c00b8e0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.576+0000 7f35778e1640 1 -- 172.21.15.16:0/3218270351 shutdown_connections 2024-04-25T10:11:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.576+0000 7f35778e1640 1 --2- 172.21.15.16:0/3218270351 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f355406b4d0 0x7f355406d990 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.576+0000 7f35778e1640 1 --2- 172.21.15.16:0/3218270351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f357015ef60 0x7f3570149e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.576+0000 7f35778e1640 1 --2- 172.21.15.16:0/3218270351 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35700cfd60 0x7f35700b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.576+0000 7f35778e1640 1 --2- 172.21.15.16:0/3218270351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35700b0740 0x7f3570147670 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.576+0000 7f35778e1640 1 -- 172.21.15.16:0/3218270351 >> 172.21.15.16:0/3218270351 conn(0x7f357006f4c0 msgr2=0x7f35700c8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.576+0000 7f35778e1640 1 -- 172.21.15.16:0/3218270351 shutdown_connections 2024-04-25T10:11:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.576+0000 7f35778e1640 1 -- 172.21.15.16:0/3218270351 wait complete. 2024-04-25T10:11:20.591 INFO:tasks.ceph.mds.d.smithi186.stderr:2024-04-25T10:11:20.576+0000 7f98024a4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-04-25T10:11:10.936139+0000 1=1+0) n(v0 rc2024-04-25T10:11:11.052137+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x55cf190d0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:11:10.936139+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:11:11.052137+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-25T10:11:20.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:20.593 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:11:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.690+0000 7ff8f588f640 1 Processor -- start 2024-04-25T10:11:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.690+0000 7ff8f588f640 1 -- start start 2024-04-25T10:11:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.691+0000 7ff8f588f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff8f00b98b0 0x7ff8f015eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.691+0000 7ff8f588f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff8f00b5a30 0x7ff8f00b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.691+0000 7ff8f588f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f015f080 con 0x7ff8f00b5580 2024-04-25T10:11:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.691+0000 7ff8f588f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f015bb80 con 0x7ff8f00b98b0 2024-04-25T10:11:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.691+0000 7ff8f588f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f006c4b0 con 0x7ff8f00b5a30 2024-04-25T10:11:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.691+0000 7ff8eeffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff8f00b98b0 0x7ff8f015eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.691+0000 7ff8e7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff8f00b5580 0x7ff8f00b24c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42170/0 (socket says 172.21.15.16:42170) 2024-04-25T10:11:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.691+0000 7ff8e7fff640 1 -- 172.21.15.16:0/982219777 learned_addr learned my addr 172.21.15.16:0/982219777 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.691+0000 7ff8ef7fe640 1 --2- 172.21.15.16:0/982219777 >> v2:172.21.15.186:3300/0 conn(0x7ff8f00b5a30 0x7ff8f00b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.691+0000 7ff8eeffd640 1 -- 172.21.15.16:0/982219777 >> v2:172.21.15.186:3300/0 conn(0x7ff8f00b5a30 msgr2=0x7ff8f00b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.691+0000 7ff8eeffd640 1 --2- 172.21.15.16:0/982219777 >> v2:172.21.15.186:3300/0 conn(0x7ff8f00b5a30 0x7ff8f00b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.691+0000 7ff8ee7fc640 1 -- 172.21.15.16:0/982219777 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1277951353 0 0) 0x7ff8f015f080 con 0x7ff8f00b5580 2024-04-25T10:11:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.691+0000 7ff8eeffd640 1 -- 172.21.15.16:0/982219777 >> v1:172.21.15.16:6789/0 conn(0x7ff8f00b5580 legacy=0x7ff8f00b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.691+0000 7ff8eeffd640 1 -- 172.21.15.16:0/982219777 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f006c790 con 0x7ff8f00b98b0 2024-04-25T10:11:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.691+0000 7ff8ef7fe640 1 --2- 172.21.15.16:0/982219777 >> v2:172.21.15.186:3300/0 conn(0x7ff8f00b5a30 0x7ff8f00b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.692+0000 7ff8eeffd640 1 --2- 172.21.15.16:0/982219777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8f00b98b0 0x7ff8f015eb40 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0162790 tx=0x7ff8e002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba89fb6287bae00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.692+0000 7ff8ee7fc640 1 -- 172.21.15.16:0/982219777 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0037070 con 0x7ff8f00b98b0 2024-04-25T10:11:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.692+0000 7ff8ee7fc640 1 -- 172.21.15.16:0/982219777 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e002add0 con 0x7ff8f00b98b0 2024-04-25T10:11:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.692+0000 7ff8ee7fc640 1 -- 172.21.15.16:0/982219777 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0032660 con 0x7ff8f00b98b0 2024-04-25T10:11:20.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.693+0000 7ff8f588f640 1 -- 172.21.15.16:0/982219777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8f00b98b0 msgr2=0x7ff8f015eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.693+0000 7ff8f588f640 1 --2- 172.21.15.16:0/982219777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8f00b98b0 0x7ff8f015eb40 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0162790 tx=0x7ff8e002a080 comp rx=0 tx=0).stop 2024-04-25T10:11:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.693+0000 7ff8f588f640 1 -- 172.21.15.16:0/982219777 shutdown_connections 2024-04-25T10:11:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.693+0000 7ff8f588f640 1 --2- 172.21.15.16:0/982219777 >> v2:172.21.15.186:3300/0 conn(0x7ff8f00b5a30 0x7ff8f00b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.693+0000 7ff8f588f640 1 --2- 172.21.15.16:0/982219777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8f00b98b0 0x7ff8f015eb40 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.693+0000 7ff8f588f640 1 -- 172.21.15.16:0/982219777 >> 172.21.15.16:0/982219777 conn(0x7ff8f00d0580 msgr2=0x7ff8f00d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.693+0000 7ff8f588f640 1 -- 172.21.15.16:0/982219777 shutdown_connections 2024-04-25T10:11:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.693+0000 7ff8f588f640 1 -- 172.21.15.16:0/982219777 wait complete. 2024-04-25T10:11:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.694+0000 7ff8f588f640 1 Processor -- start 2024-04-25T10:11:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.694+0000 7ff8f588f640 1 -- start start 2024-04-25T10:11:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8f588f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f00b5580 0x7ff8f01569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8f588f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8f00b5a30 0x7ff8f01586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8f588f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8f0156150 0x7ff8f0161680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8f588f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f00b5e60 con 0x7ff8f00b5a30 2024-04-25T10:11:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8f588f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f006cd50 con 0x7ff8f00b5580 2024-04-25T10:11:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8f588f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f006c140 con 0x7ff8f0156150 2024-04-25T10:11:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8e7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8f00b5a30 0x7ff8f01586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8e7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8f00b5a30 0x7ff8f01586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53056/0 (socket says 172.21.15.16:53056) 2024-04-25T10:11:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8e7fff640 1 -- 172.21.15.16:0/3816468077 learned_addr learned my addr 172.21.15.16:0/3816468077 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8e7fff640 1 -- 172.21.15.16:0/3816468077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8f0156150 msgr2=0x7ff8f0161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8eeffd640 1 --2- 172.21.15.16:0/3816468077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f00b5580 0x7ff8f01569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8ef7fe640 1 --2- 172.21.15.16:0/3816468077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8f0156150 0x7ff8f0161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8e7fff640 1 --2- 172.21.15.16:0/3816468077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8f0156150 0x7ff8f0161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8e7fff640 1 -- 172.21.15.16:0/3816468077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f00b5580 msgr2=0x7ff8f01569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8e7fff640 1 --2- 172.21.15.16:0/3816468077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f00b5580 0x7ff8f01569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8e7fff640 1 -- 172.21.15.16:0/3816468077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f0158c00 con 0x7ff8f00b5a30 2024-04-25T10:11:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8eeffd640 1 --2- 172.21.15.16:0/3816468077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f00b5580 0x7ff8f01569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8ef7fe640 1 --2- 172.21.15.16:0/3816468077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8f0156150 0x7ff8f0161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.695+0000 7ff8e7fff640 1 --2- 172.21.15.16:0/3816468077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8f00b5a30 0x7ff8f01586c0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c4a0 tx=0x7ff8d800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.696+0000 7ff8ecff9640 1 -- 172.21.15.16:0/3816468077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8018070 con 0x7ff8f00b5a30 2024-04-25T10:11:20.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.696+0000 7ff8ecff9640 1 -- 172.21.15.16:0/3816468077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d80074e0 con 0x7ff8f00b5a30 2024-04-25T10:11:20.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.696+0000 7ff8f588f640 1 -- 172.21.15.16:0/3816468077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f0157030 con 0x7ff8f00b5a30 2024-04-25T10:11:20.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.696+0000 7ff8ecff9640 1 -- 172.21.15.16:0/3816468077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8004a20 con 0x7ff8f00b5a30 2024-04-25T10:11:20.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.696+0000 7ff8f588f640 1 -- 172.21.15.16:0/3816468077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f01575f0 con 0x7ff8f00b5a30 2024-04-25T10:11:20.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.697+0000 7ff8ecff9640 1 -- 172.21.15.16:0/3816468077 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff8d801b070 con 0x7ff8f00b5a30 2024-04-25T10:11:20.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.697+0000 7ff8ecff9640 1 --2- 172.21.15.16:0/3816468077 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff8d006b4d0 0x7ff8d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.698+0000 7ff8ecff9640 1 -- 172.21.15.16:0/3816468077 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff8d8089560 con 0x7ff8f00b5a30 2024-04-25T10:11:20.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.698+0000 7ff8eeffd640 1 --2- 172.21.15.16:0/3816468077 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff8d006b4d0 0x7ff8d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.698+0000 7ff8eeffd640 1 --2- 172.21.15.16:0/3816468077 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff8d006b4d0 0x7ff8d006d990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0034000 tx=0x7ff8e002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.698+0000 7ff8f858e740 1 Processor -- start 2024-04-25T10:11:20.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.698+0000 7ff8f858e740 1 -- start start 2024-04-25T10:11:20.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.699+0000 7ff8f858e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d21b10 0x5570d5d24320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.699+0000 7ff8f858e740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d21ef0 0x5570d5d24860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.699+0000 7ff8f858e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5cf4770 0x5570d5cf7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.699+0000 7ff8f858e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570d5c5dda0 con 0x5570d5cf4770 2024-04-25T10:11:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.699+0000 7ff8f858e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570d5c06b80 con 0x5570d5d21b10 2024-04-25T10:11:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.699+0000 7ff8f858e740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5570d5c59e40 con 0x5570d5d21ef0 2024-04-25T10:11:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.699+0000 7ff8ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5cf4770 0x5570d5cf7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.699+0000 7ff8ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5cf4770 0x5570d5cf7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53058/0 (socket says 172.21.15.16:53058) 2024-04-25T10:11:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.699+0000 7ff8ef7fe640 1 -- 172.21.15.16:0/4033912341 learned_addr learned my addr 172.21.15.16:0/4033912341 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.700+0000 7ff8e7fff640 1 --2- 172.21.15.16:0/4033912341 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d21ef0 0x5570d5d24860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.700+0000 7ff8eeffd640 1 --2- 172.21.15.16:0/4033912341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d21b10 0x5570d5d24320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.700+0000 7ff8ef7fe640 1 -- 172.21.15.16:0/4033912341 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d21ef0 msgr2=0x5570d5d24860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.700+0000 7ff8ef7fe640 1 --2- 172.21.15.16:0/4033912341 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d21ef0 0x5570d5d24860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.700+0000 7ff8ef7fe640 1 -- 172.21.15.16:0/4033912341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d21b10 msgr2=0x5570d5d24320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.700+0000 7ff8ef7fe640 1 --2- 172.21.15.16:0/4033912341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d21b10 0x5570d5d24320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.700+0000 7ff8ef7fe640 1 -- 172.21.15.16:0/4033912341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570d5c10f10 con 0x5570d5cf4770 2024-04-25T10:11:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.700+0000 7ff8ef7fe640 1 --2- 172.21.15.16:0/4033912341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5cf4770 0x5570d5cf7f80 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009fd0 tx=0x7ff8e8007890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5fad42d5542abb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.700+0000 7ff8e57fa640 1 -- 172.21.15.16:0/4033912341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e80137d0 con 0x5570d5cf4770 2024-04-25T10:11:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.700+0000 7ff8e57fa640 1 -- 172.21.15.16:0/4033912341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e8019070 con 0x5570d5cf4770 2024-04-25T10:11:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.700+0000 7ff8e57fa640 1 -- 172.21.15.16:0/4033912341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8013b70 con 0x5570d5cf4770 2024-04-25T10:11:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.701+0000 7ff8f858e740 1 -- 172.21.15.16:0/4033912341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5cf4770 msgr2=0x5570d5cf7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.701+0000 7ff8f858e740 1 --2- 172.21.15.16:0/4033912341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5cf4770 0x5570d5cf7f80 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009fd0 tx=0x7ff8e8007890 comp rx=0 tx=0).stop 2024-04-25T10:11:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.701+0000 7ff8f858e740 1 -- 172.21.15.16:0/4033912341 shutdown_connections 2024-04-25T10:11:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.701+0000 7ff8f858e740 1 --2- 172.21.15.16:0/4033912341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5cf4770 0x5570d5cf7f80 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.701+0000 7ff8f858e740 1 --2- 172.21.15.16:0/4033912341 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d21ef0 0x5570d5d24860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.701+0000 7ff8f858e740 1 --2- 172.21.15.16:0/4033912341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d21b10 0x5570d5d24320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.701+0000 7ff8f858e740 1 -- 172.21.15.16:0/4033912341 >> 172.21.15.16:0/4033912341 conn(0x5570d5cf69e0 msgr2=0x5570d5d3f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.701+0000 7ff8f858e740 1 -- 172.21.15.16:0/4033912341 shutdown_connections 2024-04-25T10:11:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.701+0000 7ff8f858e740 1 -- 172.21.15.16:0/4033912341 wait complete. 2024-04-25T10:11:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.701+0000 7ff8f858e740 1 -- start start 2024-04-25T10:11:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.701+0000 7ff8f858e740 1 Processor -- start 2024-04-25T10:11:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.702+0000 7ff8f858e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d21b10 0x5570d5d189d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.702+0000 7ff8f858e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21ef0 0x5570d5d42910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.702+0000 7ff8f858e740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d42e50 0x5570d5d40450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.702+0000 7ff8f858e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570d5cf9b90 con 0x5570d5d21ef0 2024-04-25T10:11:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.702+0000 7ff8f858e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570d5bfc4b0 con 0x5570d5d21b10 2024-04-25T10:11:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.702+0000 7ff8f858e740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5570d5cf9e70 con 0x5570d5d42e50 2024-04-25T10:11:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.702+0000 7ff8f858e740 10 client.0 _ll_register_callbacks cb 0x5570d5c3c180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.702+0000 7ff8e7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21ef0 0x5570d5d42910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.702+0000 7ff8e7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21ef0 0x5570d5d42910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53072/0 (socket says 172.21.15.16:53072) 2024-04-25T10:11:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.702+0000 7ff8e7fff640 1 -- 172.21.15.16:0/130055647 learned_addr learned my addr 172.21.15.16:0/130055647 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.702+0000 7ff8eeffd640 1 --2- 172.21.15.16:0/130055647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d21b10 0x5570d5d189d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8ef7fe640 1 --2- 172.21.15.16:0/130055647 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d42e50 0x5570d5d40450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8e7fff640 1 -- 172.21.15.16:0/130055647 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d42e50 msgr2=0x5570d5d40450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8e7fff640 1 --2- 172.21.15.16:0/130055647 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d42e50 0x5570d5d40450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8e7fff640 1 -- 172.21.15.16:0/130055647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d21b10 msgr2=0x5570d5d189d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8e7fff640 1 --2- 172.21.15.16:0/130055647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d21b10 0x5570d5d189d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8e7fff640 1 -- 172.21.15.16:0/130055647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570d5a3c7d0 con 0x5570d5d21ef0 2024-04-25T10:11:20.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8e7fff640 1 --2- 172.21.15.16:0/130055647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21ef0 0x5570d5d42910 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80540e0 tx=0x7ff8d80584f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac3b9a924972dab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:11:20.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8ce7fc640 1 -- 172.21.15.16:0/130055647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8060100 con 0x5570d5d21ef0 2024-04-25T10:11:20.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8ce7fc640 1 -- 172.21.15.16:0/130055647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff8d80890b0 con 0x5570d5d21ef0 2024-04-25T10:11:20.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8f858e740 1 -- 172.21.15.16:0/130055647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8e8019070 con 0x5570d5d21ef0 2024-04-25T10:11:20.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8ce7fc640 1 -- 172.21.15.16:0/130055647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8d80892b0 con 0x5570d5d21ef0 2024-04-25T10:11:20.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8f858e740 10 client.7671 fetch_fsmap learned FSMap version 57 2024-04-25T10:11:20.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8f858e740 1 -- 172.21.15.16:0/130055647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5570d59723f0 con 0x5570d5d21ef0 2024-04-25T10:11:20.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8ce7fc640 1 -- 172.21.15.16:0/130055647 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e8019070 con 0x5570d5d21ef0 2024-04-25T10:11:20.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8ce7fc640 1 -- 172.21.15.16:0/130055647 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff8d80570d0 con 0x5570d5d21ef0 2024-04-25T10:11:20.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8f858e740 10 client.7671 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:11:20.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8f858e740 10 client.7671 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T10:11:20.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8f858e740 20 client.7671 populate_metadata read hostname 'smithi016' 2024-04-25T10:11:20.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8f858e740 1 --2- 172.21.15.16:0/130055647 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x5570d5d19690 0x5570d5d19ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8f858e740 4 client.7671 mds_command: new command op to 7335 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:20.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.703+0000 7ff8f858e740 1 -- 172.21.15.16:0/130055647 --> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5570d5c6b2e0 con 0x5570d5d19690 2024-04-25T10:11:20.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.704+0000 7ff8eeffd640 1 --2- 172.21.15.16:0/130055647 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x5570d5d19690 0x5570d5d19ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.704+0000 7ff8eeffd640 1 --2- 172.21.15.16:0/130055647 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x5570d5d19690 0x5570d5d19ad0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2c97ac8281489fb server_cookie=4bf439fcee468ea4 in_seq=0 out_seq=0 2024-04-25T10:11:20.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.704+0000 7ff8ce7fc640 10 client.7671 ms_handle_connect on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:20.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.705+0000 7ff8ce7fc640 1 -- 172.21.15.16:0/130055647 <== mds.0 v2:172.21.15.186:6834/3950732718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5570d5c6b2e0 con 0x5570d5d19690 2024-04-25T10:11:20.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.705+0000 7ff8ce7fc640 10 client.7671 handle_command_reply: tid=0 2024-04-25T10:11:20.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.705+0000 7ff8f858e740 1 client.7671 shutdown 2024-04-25T10:11:20.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.705+0000 7ff8f858e740 1 -- 172.21.15.16:0/130055647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21ef0 msgr2=0x5570d5d42910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.705+0000 7ff8f858e740 1 --2- 172.21.15.16:0/130055647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21ef0 0x5570d5d42910 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80540e0 tx=0x7ff8d80584f0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.705+0000 7ff8f858e740 1 -- 172.21.15.16:0/130055647 shutdown_connections 2024-04-25T10:11:20.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.705+0000 7ff8f858e740 1 --2- 172.21.15.16:0/130055647 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x5570d5d19690 0x5570d5d19ad0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.705+0000 7ff8f858e740 1 --2- 172.21.15.16:0/130055647 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d42e50 0x5570d5d40450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.705+0000 7ff8f858e740 1 --2- 172.21.15.16:0/130055647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21ef0 0x5570d5d42910 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.705+0000 7ff8f858e740 1 --2- 172.21.15.16:0/130055647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d21b10 0x5570d5d189d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.705+0000 7ff8ce7fc640 0 client.7671 ms_handle_reset on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:20.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.705+0000 7ff8f858e740 1 -- 172.21.15.16:0/130055647 >> 172.21.15.16:0/130055647 conn(0x5570d5cf4770 msgr2=0x5570d5d20f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.705+0000 7ff8f858e740 1 -- 172.21.15.16:0/130055647 shutdown_connections 2024-04-25T10:11:20.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.705+0000 7ff8f858e740 1 -- 172.21.15.16:0/130055647 wait complete. 2024-04-25T10:11:20.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.705+0000 7ff8f858e740 20 client.7671 trim_cache size 0 max 16384 2024-04-25T10:11:20.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.720+0000 7ff8f858e740 1 Processor -- start 2024-04-25T10:11:20.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.720+0000 7ff8f858e740 1 -- start start 2024-04-25T10:11:20.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.721+0000 7ff8f858e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d21b10 0x5570d5d1cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.721+0000 7ff8f858e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21ef0 0x5570d5cf3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.721+0000 7ff8f858e740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d42e50 0x5570d5cf3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.721+0000 7ff8f858e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f01fff60 con 0x5570d5d21ef0 2024-04-25T10:11:20.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.721+0000 7ff8f858e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f01ff0e0 con 0x5570d5d21b10 2024-04-25T10:11:20.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.721+0000 7ff8f858e740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f00cd230 con 0x5570d5d42e50 2024-04-25T10:11:20.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.721+0000 7ff8e7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21ef0 0x5570d5cf3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.721+0000 7ff8e7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21ef0 0x5570d5cf3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53086/0 (socket says 172.21.15.16:53086) 2024-04-25T10:11:20.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.721+0000 7ff8e7fff640 1 -- 172.21.15.16:0/2741674971 learned_addr learned my addr 172.21.15.16:0/2741674971 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:20.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.721+0000 7ff8e7fff640 1 -- 172.21.15.16:0/2741674971 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d42e50 msgr2=0x5570d5cf3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.721+0000 7ff8e7fff640 1 --2- 172.21.15.16:0/2741674971 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d42e50 0x5570d5cf3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.721+0000 7ff8e7fff640 1 -- 172.21.15.16:0/2741674971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d21b10 msgr2=0x5570d5d1cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.721+0000 7ff8e7fff640 1 --2- 172.21.15.16:0/2741674971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d21b10 0x5570d5d1cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.721+0000 7ff8e7fff640 1 -- 172.21.15.16:0/2741674971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570d59320f0 con 0x5570d5d21ef0 2024-04-25T10:11:20.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.722+0000 7ff8e7fff640 1 --2- 172.21.15.16:0/2741674971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21ef0 0x5570d5cf3600 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8057980 tx=0x7ff8d805e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ec7124cc400ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.722+0000 7ff8e4ff9640 1 -- 172.21.15.16:0/2741674971 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d805ff00 con 0x5570d5d21ef0 2024-04-25T10:11:20.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.722+0000 7ff8e4ff9640 1 -- 172.21.15.16:0/2741674971 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d8087e50 con 0x5570d5d21ef0 2024-04-25T10:11:20.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.722+0000 7ff8e4ff9640 1 -- 172.21.15.16:0/2741674971 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8053c10 con 0x5570d5d21ef0 2024-04-25T10:11:20.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.722+0000 7ff8f858e740 1 -- 172.21.15.16:0/2741674971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21ef0 msgr2=0x5570d5cf3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.722+0000 7ff8f858e740 1 --2- 172.21.15.16:0/2741674971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21ef0 0x5570d5cf3600 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8057980 tx=0x7ff8d805e000 comp rx=0 tx=0).stop 2024-04-25T10:11:20.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.722+0000 7ff8f858e740 1 -- 172.21.15.16:0/2741674971 shutdown_connections 2024-04-25T10:11:20.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.722+0000 7ff8f858e740 1 --2- 172.21.15.16:0/2741674971 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d42e50 0x5570d5cf3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.722+0000 7ff8f858e740 1 --2- 172.21.15.16:0/2741674971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21ef0 0x5570d5cf3600 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.722+0000 7ff8f858e740 1 --2- 172.21.15.16:0/2741674971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d21b10 0x5570d5d1cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.722+0000 7ff8f858e740 1 -- 172.21.15.16:0/2741674971 >> 172.21.15.16:0/2741674971 conn(0x5570d5cf4770 msgr2=0x5570d5d42720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.722+0000 7ff8f858e740 1 -- 172.21.15.16:0/2741674971 shutdown_connections 2024-04-25T10:11:20.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.722+0000 7ff8f858e740 1 -- 172.21.15.16:0/2741674971 wait complete. 2024-04-25T10:11:20.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.723+0000 7ff8f858e740 1 -- start start 2024-04-25T10:11:20.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.723+0000 7ff8f858e740 1 Processor -- start 2024-04-25T10:11:20.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8f858e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21b10 0x5570d5d27160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8f858e740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d21ef0 0x5570d5d3f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8f858e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d42e50 0x5570d5d16df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8f858e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f00cd830 con 0x5570d5d21b10 2024-04-25T10:11:20.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8f858e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570d5d619f0 con 0x5570d5d42e50 2024-04-25T10:11:20.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8f858e740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5570d5d17330 con 0x5570d5d21ef0 2024-04-25T10:11:20.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8f858e740 10 client.0 _ll_register_callbacks cb 0x7ff8e80098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:20.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21b10 0x5570d5d27160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21b10 0x5570d5d27160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53100/0 (socket says 172.21.15.16:53100) 2024-04-25T10:11:20.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8eeffd640 1 -- 172.21.15.16:0/3400448983 learned_addr learned my addr 172.21.15.16:0/3400448983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:20.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8e7fff640 1 --2- 172.21.15.16:0/3400448983 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d21ef0 0x5570d5d3f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8ef7fe640 1 --2- 172.21.15.16:0/3400448983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d42e50 0x5570d5d16df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8eeffd640 1 -- 172.21.15.16:0/3400448983 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d21ef0 msgr2=0x5570d5d3f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8eeffd640 1 --2- 172.21.15.16:0/3400448983 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d21ef0 0x5570d5d3f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8eeffd640 1 -- 172.21.15.16:0/3400448983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d42e50 msgr2=0x5570d5d16df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8eeffd640 1 --2- 172.21.15.16:0/3400448983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d42e50 0x5570d5d16df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8eeffd640 1 -- 172.21.15.16:0/3400448983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570d5d40580 con 0x5570d5d21b10 2024-04-25T10:11:20.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8ef7fe640 1 --2- 172.21.15.16:0/3400448983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d42e50 0x5570d5d16df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:20.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8eeffd640 1 --2- 172.21.15.16:0/3400448983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21b10 0x5570d5d27160 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0035040 tx=0x7ff8e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6837aa119c1fd0b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:11:20.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8ce7fc640 1 -- 172.21.15.16:0/3400448983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0037040 con 0x5570d5d21b10 2024-04-25T10:11:20.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8ce7fc640 1 -- 172.21.15.16:0/3400448983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff8e003de40 con 0x5570d5d21b10 2024-04-25T10:11:20.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.724+0000 7ff8f858e740 1 -- 172.21.15.16:0/3400448983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8d8087e50 con 0x5570d5d21b10 2024-04-25T10:11:20.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.725+0000 7ff8ce7fc640 1 -- 172.21.15.16:0/3400448983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8e005d050 con 0x5570d5d21b10 2024-04-25T10:11:20.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.725+0000 7ff8f858e740 10 client.7677 fetch_fsmap learned FSMap version 57 2024-04-25T10:11:20.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.725+0000 7ff8f858e740 1 -- 172.21.15.16:0/3400448983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5570d5d1d2e0 con 0x5570d5d21b10 2024-04-25T10:11:20.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.725+0000 7ff8ce7fc640 1 -- 172.21.15.16:0/3400448983 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d8087e50 con 0x5570d5d21b10 2024-04-25T10:11:20.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.725+0000 7ff8ce7fc640 1 -- 172.21.15.16:0/3400448983 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff8e0060020 con 0x5570d5d21b10 2024-04-25T10:11:20.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.725+0000 7ff8f858e740 10 client.7677 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:11:20.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.725+0000 7ff8f858e740 10 client.7677 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T10:11:20.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.725+0000 7ff8f858e740 20 client.7677 populate_metadata read hostname 'smithi016' 2024-04-25T10:11:20.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.725+0000 7ff8f858e740 1 --2- 172.21.15.16:0/3400448983 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x5570d5d17c00 0x5570d5cf4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.725+0000 7ff8f858e740 4 client.7677 mds_command: new command op to 7335 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:11:20.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.725+0000 7ff8f858e740 1 -- 172.21.15.16:0/3400448983 --> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5570d5d3ef30 con 0x5570d5d17c00 2024-04-25T10:11:20.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.725+0000 7ff8e7fff640 1 --2- 172.21.15.16:0/3400448983 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x5570d5d17c00 0x5570d5cf4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.726+0000 7ff8e7fff640 1 --2- 172.21.15.16:0/3400448983 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x5570d5d17c00 0x5570d5cf4c50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccefb85968a47913 server_cookie=7a9a662072e87d7 in_seq=0 out_seq=0 2024-04-25T10:11:20.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.726+0000 7ff8ce7fc640 10 client.7677 ms_handle_connect on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:20.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.726+0000 7ff8ce7fc640 1 -- 172.21.15.16:0/3400448983 <== mds.0 v2:172.21.15.186:6834/3950732718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5570d5d3ef30 con 0x5570d5d17c00 2024-04-25T10:11:20.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.726+0000 7ff8ce7fc640 10 client.7677 handle_command_reply: tid=0 2024-04-25T10:11:20.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.726+0000 7ff8f858e740 1 client.7677 shutdown 2024-04-25T10:11:20.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.726+0000 7ff8f858e740 1 -- 172.21.15.16:0/3400448983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21b10 msgr2=0x5570d5d27160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.726+0000 7ff8f858e740 1 --2- 172.21.15.16:0/3400448983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21b10 0x5570d5d27160 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0035040 tx=0x7ff8e00046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.726+0000 7ff8f858e740 1 -- 172.21.15.16:0/3400448983 shutdown_connections 2024-04-25T10:11:20.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.726+0000 7ff8f858e740 1 --2- 172.21.15.16:0/3400448983 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x5570d5d17c00 0x5570d5cf4c50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.726+0000 7ff8f858e740 1 --2- 172.21.15.16:0/3400448983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5570d5d42e50 0x5570d5d16df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.726+0000 7ff8f858e740 1 --2- 172.21.15.16:0/3400448983 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5570d5d21ef0 0x5570d5d3f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.726+0000 7ff8ce7fc640 0 client.7677 ms_handle_reset on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:20.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.726+0000 7ff8f858e740 1 --2- 172.21.15.16:0/3400448983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570d5d21b10 0x5570d5d27160 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.726+0000 7ff8f858e740 1 -- 172.21.15.16:0/3400448983 >> 172.21.15.16:0/3400448983 conn(0x5570d5cf4770 msgr2=0x5570d5cf3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.726+0000 7ff8f858e740 1 -- 172.21.15.16:0/3400448983 shutdown_connections 2024-04-25T10:11:20.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.726+0000 7ff8f858e740 1 -- 172.21.15.16:0/3400448983 wait complete. 2024-04-25T10:11:20.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.726+0000 7ff8f858e740 20 client.7677 trim_cache size 0 max 16384 2024-04-25T10:11:20.740 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:11:20.740 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-04-25T10:11:20.741 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-04-25T10:11:20.741 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:11:20.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.726+0000 7ff8f588f640 1 -- 172.21.15.16:0/3816468077 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff8d006b4d0 msgr2=0x7ff8d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.727+0000 7ff8f588f640 1 --2- 172.21.15.16:0/3816468077 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff8d006b4d0 0x7ff8d006d990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0034000 tx=0x7ff8e002e000 comp rx=0 tx=0).stop 2024-04-25T10:11:20.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.727+0000 7ff8f588f640 1 -- 172.21.15.16:0/3816468077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8f00b5a30 msgr2=0x7ff8f01586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.727+0000 7ff8f588f640 1 --2- 172.21.15.16:0/3816468077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8f00b5a30 0x7ff8f01586c0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c4a0 tx=0x7ff8d800c960 comp rx=0 tx=0).stop 2024-04-25T10:11:20.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.727+0000 7ff8f588f640 1 -- 172.21.15.16:0/3816468077 shutdown_connections 2024-04-25T10:11:20.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.727+0000 7ff8f588f640 1 --2- 172.21.15.16:0/3816468077 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff8d006b4d0 0x7ff8d006d990 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.727+0000 7ff8f588f640 1 --2- 172.21.15.16:0/3816468077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8f0156150 0x7ff8f0161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.727+0000 7ff8f588f640 1 --2- 172.21.15.16:0/3816468077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8f00b5a30 0x7ff8f01586c0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.727+0000 7ff8f588f640 1 --2- 172.21.15.16:0/3816468077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8f00b5580 0x7ff8f01569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.727+0000 7ff8f588f640 1 -- 172.21.15.16:0/3816468077 >> 172.21.15.16:0/3816468077 conn(0x7ff8f00d0580 msgr2=0x7ff8f00b46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.727+0000 7ff8f588f640 1 -- 172.21.15.16:0/3816468077 shutdown_connections 2024-04-25T10:11:20.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.727+0000 7ff8f588f640 1 -- 172.21.15.16:0/3816468077 wait complete. 2024-04-25T10:11:20.744 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:11:20.744 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.838+0000 7f9defe1f640 1 Processor -- start 2024-04-25T10:11:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.838+0000 7f9defe1f640 1 -- start start 2024-04-25T10:11:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.839+0000 7f9defe1f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9de815bb90 0x7f9de8160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.839+0000 7f9defe1f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9de806aba0 0x7f9de80cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.839+0000 7f9defe1f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de806b310 con 0x7f9de80cc780 2024-04-25T10:11:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.839+0000 7f9defe1f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de80b0010 con 0x7f9de815bb90 2024-04-25T10:11:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.839+0000 7f9defe1f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de806b5f0 con 0x7f9de806aba0 2024-04-25T10:11:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.839+0000 7f9dedbbb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9de815bb90 0x7f9de8160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.839+0000 7f9dee3bc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9de80cc780 0x7f9de80cd080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42182/0 (socket says 172.21.15.16:42182) 2024-04-25T10:11:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.839+0000 7f9dee3bc640 1 -- 172.21.15.16:0/679365262 learned_addr learned my addr 172.21.15.16:0/679365262 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.839+0000 7f9ded3ba640 1 --2- 172.21.15.16:0/679365262 >> v2:172.21.15.186:3300/0 conn(0x7f9de806aba0 0x7f9de80cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.839+0000 7f9dedbbb640 1 -- 172.21.15.16:0/679365262 >> v2:172.21.15.186:3300/0 conn(0x7f9de806aba0 msgr2=0x7f9de80cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.839+0000 7f9dedbbb640 1 --2- 172.21.15.16:0/679365262 >> v2:172.21.15.186:3300/0 conn(0x7f9de806aba0 0x7f9de80cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.839+0000 7f9dedbbb640 1 -- 172.21.15.16:0/679365262 >> v1:172.21.15.16:6789/0 conn(0x7f9de80cc780 legacy=0x7f9de80cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.839+0000 7f9decbb9640 1 -- 172.21.15.16:0/679365262 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150504715 0 0) 0x7f9de806b310 con 0x7f9de80cc780 2024-04-25T10:11:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.839+0000 7f9dedbbb640 1 -- 172.21.15.16:0/679365262 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de80b0260 con 0x7f9de815bb90 2024-04-25T10:11:20.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.840+0000 7f9dedbbb640 1 --2- 172.21.15.16:0/679365262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de815bb90 0x7f9de8160070 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f9de806dda0 tx=0x7f9de402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b4e2926e3c32ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.840+0000 7f9decbb9640 1 -- 172.21.15.16:0/679365262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de4037070 con 0x7f9de815bb90 2024-04-25T10:11:20.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.840+0000 7f9decbb9640 1 -- 172.21.15.16:0/679365262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de4032440 con 0x7f9de815bb90 2024-04-25T10:11:20.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.840+0000 7f9decbb9640 1 -- 172.21.15.16:0/679365262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de402ae60 con 0x7f9de815bb90 2024-04-25T10:11:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.841+0000 7f9defe1f640 1 -- 172.21.15.16:0/679365262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de815bb90 msgr2=0x7f9de8160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.841+0000 7f9defe1f640 1 --2- 172.21.15.16:0/679365262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de815bb90 0x7f9de8160070 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f9de806dda0 tx=0x7f9de402a110 comp rx=0 tx=0).stop 2024-04-25T10:11:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.841+0000 7f9defe1f640 1 -- 172.21.15.16:0/679365262 shutdown_connections 2024-04-25T10:11:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.841+0000 7f9defe1f640 1 --2- 172.21.15.16:0/679365262 >> v2:172.21.15.186:3300/0 conn(0x7f9de806aba0 0x7f9de80cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.841+0000 7f9defe1f640 1 --2- 172.21.15.16:0/679365262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de815bb90 0x7f9de8160070 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.841+0000 7f9defe1f640 1 -- 172.21.15.16:0/679365262 >> 172.21.15.16:0/679365262 conn(0x7f9de80d06b0 msgr2=0x7f9de80d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.841+0000 7f9defe1f640 1 -- 172.21.15.16:0/679365262 shutdown_connections 2024-04-25T10:11:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.841+0000 7f9defe1f640 1 -- 172.21.15.16:0/679365262 wait complete. 2024-04-25T10:11:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.842+0000 7f9defe1f640 1 Processor -- start 2024-04-25T10:11:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.842+0000 7f9defe1f640 1 -- start start 2024-04-25T10:11:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9defe1f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9de806aba0 0x7f9de80c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9defe1f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9de80cc780 0x7f9de815aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9defe1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de815bb90 0x7f9de81e4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9defe1f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de80ccc10 con 0x7f9de815bb90 2024-04-25T10:11:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9defe1f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de806b030 con 0x7f9de806aba0 2024-04-25T10:11:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9defe1f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de80b1a70 con 0x7f9de80cc780 2024-04-25T10:11:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9dee3bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de815bb90 0x7f9de81e4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9dee3bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de815bb90 0x7f9de81e4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53114/0 (socket says 172.21.15.16:53114) 2024-04-25T10:11:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9dee3bc640 1 -- 172.21.15.16:0/2941976620 learned_addr learned my addr 172.21.15.16:0/2941976620 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9dedbbb640 1 --2- 172.21.15.16:0/2941976620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9de806aba0 0x7f9de80c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9ded3ba640 1 --2- 172.21.15.16:0/2941976620 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9de80cc780 0x7f9de815aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9dee3bc640 1 -- 172.21.15.16:0/2941976620 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9de80cc780 msgr2=0x7f9de815aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9dee3bc640 1 --2- 172.21.15.16:0/2941976620 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9de80cc780 0x7f9de815aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9dee3bc640 1 -- 172.21.15.16:0/2941976620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9de806aba0 msgr2=0x7f9de80c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9dee3bc640 1 --2- 172.21.15.16:0/2941976620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9de806aba0 0x7f9de80c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9dee3bc640 1 -- 172.21.15.16:0/2941976620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de815d160 con 0x7f9de815bb90 2024-04-25T10:11:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9dedbbb640 1 --2- 172.21.15.16:0/2941976620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9de806aba0 0x7f9de80c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9ded3ba640 1 --2- 172.21.15.16:0/2941976620 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9de80cc780 0x7f9de815aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9dee3bc640 1 --2- 172.21.15.16:0/2941976620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de815bb90 0x7f9de81e4f60 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f9de000c7a0 tx=0x7f9de000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.843+0000 7f9dde7fc640 1 -- 172.21.15.16:0/2941976620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de0018070 con 0x7f9de815bb90 2024-04-25T10:11:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.844+0000 7f9dde7fc640 1 -- 172.21.15.16:0/2941976620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de000a640 con 0x7f9de815bb90 2024-04-25T10:11:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.844+0000 7f9defe1f640 1 -- 172.21.15.16:0/2941976620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de80c9480 con 0x7f9de815bb90 2024-04-25T10:11:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.844+0000 7f9dde7fc640 1 -- 172.21.15.16:0/2941976620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de000aab0 con 0x7f9de815bb90 2024-04-25T10:11:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.844+0000 7f9defe1f640 1 -- 172.21.15.16:0/2941976620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de815d400 con 0x7f9de815bb90 2024-04-25T10:11:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.845+0000 7f9defe1f640 1 -- 172.21.15.16:0/2941976620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de8070ae0 con 0x7f9de815bb90 2024-04-25T10:11:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.845+0000 7f9dde7fc640 1 -- 172.21.15.16:0/2941976620 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9de001b070 con 0x7f9de815bb90 2024-04-25T10:11:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.845+0000 7f9dde7fc640 1 --2- 172.21.15.16:0/2941976620 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9dcc06b4d0 0x7f9dcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.846+0000 7f9dde7fc640 1 -- 172.21.15.16:0/2941976620 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9de00898f0 con 0x7f9de815bb90 2024-04-25T10:11:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.846+0000 7f9dedbbb640 1 --2- 172.21.15.16:0/2941976620 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9dcc06b4d0 0x7f9dcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.846+0000 7f9dedbbb640 1 --2- 172.21.15.16:0/2941976620 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9dcc06b4d0 0x7f9dcc06d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9de4034000 tx=0x7f9de402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:20.850+0000 7f9dde7fc640 1 -- 172.21.15.16:0/2941976620 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9de8070ae0 con 0x7f9de815bb90 2024-04-25T10:11:20.860 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-04-25T10:11:20.854+0000 7ff1421c0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:11:21.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.065+0000 7f9defe1f640 1 -- 172.21.15.16:0/2941976620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9de80749b0 con 0x7f9de815bb90 2024-04-25T10:11:21.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.066+0000 7f9dde7fc640 1 -- 172.21.15.16:0/2941976620 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f9de0058670 con 0x7f9de815bb90 2024-04-25T10:11:21.172 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:21.172 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4044509901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4044509901},{"type":"v1","addr":"172.21.15.16:6835","nonce":4044509901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1494230493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1494230493},{"type":"v1","addr":"172.21.15.186:6837","nonce":1494230493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/84349474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":84349474},{"type":"v1","addr":"172.21.15.16:6834","nonce":84349474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:00.015075+0000","modified":"2024-04-25T10:11:20.851789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"d","rank":0,"incarnation":54,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6836/3950732718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3950732718},{"type":"v1","addr":"172.21.15.186:6836","nonce":3950732718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 58 2024-04-25T10:11:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.068+0000 7f9defe1f640 1 -- 172.21.15.16:0/2941976620 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9dcc06b4d0 msgr2=0x7f9dcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.068+0000 7f9defe1f640 1 --2- 172.21.15.16:0/2941976620 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9dcc06b4d0 0x7f9dcc06d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9de4034000 tx=0x7f9de402e000 comp rx=0 tx=0).stop 2024-04-25T10:11:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.068+0000 7f9defe1f640 1 -- 172.21.15.16:0/2941976620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de815bb90 msgr2=0x7f9de81e4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.068+0000 7f9defe1f640 1 --2- 172.21.15.16:0/2941976620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de815bb90 0x7f9de81e4f60 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f9de000c7a0 tx=0x7f9de000cc60 comp rx=0 tx=0).stop 2024-04-25T10:11:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.068+0000 7f9defe1f640 1 -- 172.21.15.16:0/2941976620 shutdown_connections 2024-04-25T10:11:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.068+0000 7f9defe1f640 1 --2- 172.21.15.16:0/2941976620 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9dcc06b4d0 0x7f9dcc06d990 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.068+0000 7f9defe1f640 1 --2- 172.21.15.16:0/2941976620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de815bb90 0x7f9de81e4f60 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.068+0000 7f9defe1f640 1 --2- 172.21.15.16:0/2941976620 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9de80cc780 0x7f9de815aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.068+0000 7f9defe1f640 1 --2- 172.21.15.16:0/2941976620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9de806aba0 0x7f9de80c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.068+0000 7f9defe1f640 1 -- 172.21.15.16:0/2941976620 >> 172.21.15.16:0/2941976620 conn(0x7f9de80d06b0 msgr2=0x7f9de80b52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.068+0000 7f9defe1f640 1 -- 172.21.15.16:0/2941976620 shutdown_connections 2024-04-25T10:11:21.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.068+0000 7f9defe1f640 1 -- 172.21.15.16:0/2941976620 wait complete. 2024-04-25T10:11:21.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-04-25T10:11:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.272+0000 7f0334ff9640 1 Processor -- start 2024-04-25T10:11:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.272+0000 7f0334ff9640 1 -- start start 2024-04-25T10:11:21.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.273+0000 7f0334ff9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f03300ce9d0 0x7f03300cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.273+0000 7f0334ff9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03300cc300 0x7f03300cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.273+0000 7f0334ff9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033006dcb0 con 0x7f0330144e70 2024-04-25T10:11:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.273+0000 7f0334ff9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033015bfe0 con 0x7f03300cc300 2024-04-25T10:11:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.273+0000 7f0334ff9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033006df90 con 0x7f03300ce9d0 2024-04-25T10:11:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.273+0000 7f032ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03300cc300 0x7f03300cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.273+0000 7f032ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03300cc300 0x7f03300cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53124/0 (socket says 172.21.15.16:53124) 2024-04-25T10:11:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.273+0000 7f032ed9d640 1 -- 172.21.15.16:0/75643820 learned_addr learned my addr 172.21.15.16:0/75643820 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.273+0000 7f0327fff640 1 --2- 172.21.15.16:0/75643820 >> v2:172.21.15.186:3300/0 conn(0x7f03300ce9d0 0x7f03300cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.273+0000 7f032ed9d640 1 -- 172.21.15.16:0/75643820 >> v2:172.21.15.186:3300/0 conn(0x7f03300ce9d0 msgr2=0x7f03300cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.273+0000 7f032ed9d640 1 --2- 172.21.15.16:0/75643820 >> v2:172.21.15.186:3300/0 conn(0x7f03300ce9d0 0x7f03300cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.273+0000 7f032ed9d640 1 -- 172.21.15.16:0/75643820 >> v1:172.21.15.186:6789/0 conn(0x7f0330144e70 legacy=0x7f033006d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.273+0000 7f032ed9d640 1 -- 172.21.15.16:0/75643820 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033006e270 con 0x7f03300cc300 2024-04-25T10:11:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.273+0000 7f0327fff640 1 --2- 172.21.15.16:0/75643820 >> v2:172.21.15.186:3300/0 conn(0x7f03300ce9d0 0x7f03300cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:21.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.274+0000 7f032ed9d640 1 --2- 172.21.15.16:0/75643820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03300cc300 0x7f03300cc740 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f0330072560 tx=0x7f032802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9649bf49301af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.274+0000 7f032dd9b640 1 -- 172.21.15.16:0/75643820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328036070 con 0x7f03300cc300 2024-04-25T10:11:21.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.274+0000 7f032dd9b640 1 -- 172.21.15.16:0/75643820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0328031600 con 0x7f03300cc300 2024-04-25T10:11:21.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.274+0000 7f032dd9b640 1 -- 172.21.15.16:0/75643820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03280319d0 con 0x7f03300cc300 2024-04-25T10:11:21.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.275+0000 7f0334ff9640 1 -- 172.21.15.16:0/75643820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03300cc300 msgr2=0x7f03300cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.275+0000 7f0334ff9640 1 --2- 172.21.15.16:0/75643820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03300cc300 0x7f03300cc740 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f0330072560 tx=0x7f032802a660 comp rx=0 tx=0).stop 2024-04-25T10:11:21.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.275+0000 7f0334ff9640 1 -- 172.21.15.16:0/75643820 shutdown_connections 2024-04-25T10:11:21.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.275+0000 7f0334ff9640 1 --2- 172.21.15.16:0/75643820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03300cc300 0x7f03300cc740 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.275+0000 7f0334ff9640 1 --2- 172.21.15.16:0/75643820 >> v2:172.21.15.186:3300/0 conn(0x7f03300ce9d0 0x7f03300cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.275+0000 7f0334ff9640 1 -- 172.21.15.16:0/75643820 >> 172.21.15.16:0/75643820 conn(0x7f033006aba0 msgr2=0x7f033006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.275+0000 7f0334ff9640 1 -- 172.21.15.16:0/75643820 shutdown_connections 2024-04-25T10:11:21.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.275+0000 7f0334ff9640 1 -- 172.21.15.16:0/75643820 wait complete. 2024-04-25T10:11:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.276+0000 7f0334ff9640 1 Processor -- start 2024-04-25T10:11:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.276+0000 7f0334ff9640 1 -- start start 2024-04-25T10:11:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f0334ff9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03300cc300 0x7f03300b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f0334ff9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03300ce9d0 0x7f03300b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f0334ff9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0330144e70 0x7f03300afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f0334ff9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033006d970 con 0x7f03300ce9d0 2024-04-25T10:11:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f0334ff9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03300cf3a0 con 0x7f0330144e70 2024-04-25T10:11:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f0334ff9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03301454b0 con 0x7f03300cc300 2024-04-25T10:11:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f0327fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03300ce9d0 0x7f03300b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f0327fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03300ce9d0 0x7f03300b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53128/0 (socket says 172.21.15.16:53128) 2024-04-25T10:11:21.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f0327fff640 1 -- 172.21.15.16:0/3079175992 learned_addr learned my addr 172.21.15.16:0/3079175992 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:21.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f032ed9d640 1 --2- 172.21.15.16:0/3079175992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0330144e70 0x7f03300afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f032e59c640 1 --2- 172.21.15.16:0/3079175992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03300cc300 0x7f03300b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f0327fff640 1 -- 172.21.15.16:0/3079175992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03300cc300 msgr2=0x7f03300b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f0327fff640 1 --2- 172.21.15.16:0/3079175992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03300cc300 0x7f03300b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f0327fff640 1 -- 172.21.15.16:0/3079175992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0330144e70 msgr2=0x7f03300afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f0327fff640 1 --2- 172.21.15.16:0/3079175992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0330144e70 0x7f03300afcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f0327fff640 1 -- 172.21.15.16:0/3079175992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0330159bb0 con 0x7f03300ce9d0 2024-04-25T10:11:21.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f032ed9d640 1 --2- 172.21.15.16:0/3079175992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0330144e70 0x7f03300afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:21.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f032e59c640 1 --2- 172.21.15.16:0/3079175992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03300cc300 0x7f03300b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:21.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.277+0000 7f0327fff640 1 --2- 172.21.15.16:0/3079175992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03300ce9d0 0x7f03300b4370 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f031800c7f0 tx=0x7f031800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.278+0000 7f0326ffd640 1 -- 172.21.15.16:0/3079175992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318015070 con 0x7f03300ce9d0 2024-04-25T10:11:21.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.278+0000 7f0326ffd640 1 -- 172.21.15.16:0/3079175992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0318007a30 con 0x7f03300ce9d0 2024-04-25T10:11:21.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.278+0000 7f0334ff9640 1 -- 172.21.15.16:0/3079175992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03300b32d0 con 0x7f03300ce9d0 2024-04-25T10:11:21.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.278+0000 7f0326ffd640 1 -- 172.21.15.16:0/3079175992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318007e00 con 0x7f03300ce9d0 2024-04-25T10:11:21.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.278+0000 7f0334ff9640 1 -- 172.21.15.16:0/3079175992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03300b3890 con 0x7f03300ce9d0 2024-04-25T10:11:21.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.279+0000 7f0326ffd640 1 -- 172.21.15.16:0/3079175992 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0318018070 con 0x7f03300ce9d0 2024-04-25T10:11:21.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.279+0000 7f0326ffd640 1 --2- 172.21.15.16:0/3079175992 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f031006b4d0 0x7f031006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.280+0000 7f0326ffd640 1 -- 172.21.15.16:0/3079175992 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f031808a2b0 con 0x7f03300ce9d0 2024-04-25T10:11:21.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.280+0000 7f032e59c640 1 --2- 172.21.15.16:0/3079175992 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f031006b4d0 0x7f031006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.280+0000 7f0337dd8740 1 Processor -- start 2024-04-25T10:11:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.280+0000 7f032e59c640 1 --2- 172.21.15.16:0/3079175992 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f031006b4d0 0x7f031006d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f032000a830 tx=0x7f0320008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.281+0000 7f0337dd8740 1 -- start start 2024-04-25T10:11:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.281+0000 7f0337dd8740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b66937d590 0x55b66937f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.281+0000 7f0337dd8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b6693b9c60 0x55b66937d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.281+0000 7f0337dd8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 0x55b6693a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.281+0000 7f0337dd8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6692e0c60 con 0x55b6693a8050 2024-04-25T10:11:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.281+0000 7f0337dd8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66927fd80 con 0x55b6693b9c60 2024-04-25T10:11:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.281+0000 7f0337dd8740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6692dcd00 con 0x55b66937d590 2024-04-25T10:11:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.281+0000 7f032ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 0x55b6693a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.281+0000 7f032ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 0x55b6693a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53134/0 (socket says 172.21.15.16:53134) 2024-04-25T10:11:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.281+0000 7f032ed9d640 1 -- 172.21.15.16:0/51934528 learned_addr learned my addr 172.21.15.16:0/51934528 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.281+0000 7f0327fff640 1 --2- 172.21.15.16:0/51934528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b6693b9c60 0x55b66937d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.281+0000 7f032e59c640 1 --2- 172.21.15.16:0/51934528 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b66937d590 0x55b66937f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.282+0000 7f032ed9d640 1 -- 172.21.15.16:0/51934528 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b66937d590 msgr2=0x55b66937f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.282+0000 7f032ed9d640 1 --2- 172.21.15.16:0/51934528 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b66937d590 0x55b66937f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.282+0000 7f032ed9d640 1 -- 172.21.15.16:0/51934528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b6693b9c60 msgr2=0x55b66937d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.282+0000 7f032ed9d640 1 --2- 172.21.15.16:0/51934528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b6693b9c60 0x55b66937d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.282+0000 7f032ed9d640 1 -- 172.21.15.16:0/51934528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6690c17c0 con 0x55b6693a8050 2024-04-25T10:11:21.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.282+0000 7f032ed9d640 1 --2- 172.21.15.16:0/51934528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 0x55b6693a84d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f0328009f40 tx=0x7f032802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b488cbb232b792 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.282+0000 7f030ffff640 1 -- 172.21.15.16:0/51934528 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328036040 con 0x55b6693a8050 2024-04-25T10:11:21.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.282+0000 7f030ffff640 1 -- 172.21.15.16:0/51934528 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03280315c0 con 0x55b6693a8050 2024-04-25T10:11:21.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.282+0000 7f030ffff640 1 -- 172.21.15.16:0/51934528 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328031970 con 0x55b6693a8050 2024-04-25T10:11:21.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.283+0000 7f0337dd8740 1 -- 172.21.15.16:0/51934528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 msgr2=0x55b6693a84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.283+0000 7f0337dd8740 1 --2- 172.21.15.16:0/51934528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 0x55b6693a84d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f0328009f40 tx=0x7f032802e000 comp rx=0 tx=0).stop 2024-04-25T10:11:21.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.283+0000 7f0337dd8740 1 -- 172.21.15.16:0/51934528 shutdown_connections 2024-04-25T10:11:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.283+0000 7f0337dd8740 1 --2- 172.21.15.16:0/51934528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 0x55b6693a84d0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.283+0000 7f0337dd8740 1 --2- 172.21.15.16:0/51934528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b6693b9c60 0x55b66937d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.283+0000 7f0337dd8740 1 --2- 172.21.15.16:0/51934528 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b66937d590 0x55b66937f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.283+0000 7f0337dd8740 1 -- 172.21.15.16:0/51934528 >> 172.21.15.16:0/51934528 conn(0x55b6693456e0 msgr2=0x55b6693c3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.283+0000 7f0337dd8740 1 -- 172.21.15.16:0/51934528 shutdown_connections 2024-04-25T10:11:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.283+0000 7f0337dd8740 1 -- 172.21.15.16:0/51934528 wait complete. 2024-04-25T10:11:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.283+0000 7f0337dd8740 1 -- start start 2024-04-25T10:11:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.284+0000 7f0337dd8740 1 Processor -- start 2024-04-25T10:11:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.284+0000 7f0337dd8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66937d590 0x55b6693a5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.284+0000 7f0337dd8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 0x55b6693c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.284+0000 7f0337dd8740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b6693b9c60 0x55b6693d3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.284+0000 7f0337dd8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66927d840 con 0x55b6693a8050 2024-04-25T10:11:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.284+0000 7f0337dd8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6693a5ba0 con 0x55b66937d590 2024-04-25T10:11:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.284+0000 7f0337dd8740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6693bdaf0 con 0x55b6693b9c60 2024-04-25T10:11:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f0337dd8740 10 client.0 _ll_register_callbacks cb 0x55b669297980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f0327fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 0x55b6693c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f0327fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 0x55b6693c6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53140/0 (socket says 172.21.15.16:53140) 2024-04-25T10:11:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f0327fff640 1 -- 172.21.15.16:0/2590796758 learned_addr learned my addr 172.21.15.16:0/2590796758 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f032ed9d640 1 --2- 172.21.15.16:0/2590796758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b6693b9c60 0x55b6693d3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f032e59c640 1 --2- 172.21.15.16:0/2590796758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66937d590 0x55b6693a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f0327fff640 1 -- 172.21.15.16:0/2590796758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b6693b9c60 msgr2=0x55b6693d3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f0327fff640 1 --2- 172.21.15.16:0/2590796758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b6693b9c60 0x55b6693d3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f0327fff640 1 -- 172.21.15.16:0/2590796758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66937d590 msgr2=0x55b6693a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f0327fff640 1 --2- 172.21.15.16:0/2590796758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66937d590 0x55b6693a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f0327fff640 1 -- 172.21.15.16:0/2590796758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b669121f50 con 0x55b6693a8050 2024-04-25T10:11:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f0327fff640 1 --2- 172.21.15.16:0/2590796758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 0x55b6693c6b30 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0318007410 tx=0x7f031808a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a5c4961f85574e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f030d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:11:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f030d7fa640 1 -- 172.21.15.16:0/2590796758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318070020 con 0x55b6693a8050 2024-04-25T10:11:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f030d7fa640 1 -- 172.21.15.16:0/2590796758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f03180703c0 con 0x55b6693a8050 2024-04-25T10:11:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f0337dd8740 1 -- 172.21.15.16:0/2590796758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03280315c0 con 0x55b6693a8050 2024-04-25T10:11:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f030d7fa640 1 -- 172.21.15.16:0/2590796758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f031801f050 con 0x55b6693a8050 2024-04-25T10:11:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f0337dd8740 10 client.7698 fetch_fsmap learned FSMap version 58 2024-04-25T10:11:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f0337dd8740 1 -- 172.21.15.16:0/2590796758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x55b668fb50b0 con 0x55b6693a8050 2024-04-25T10:11:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.285+0000 7f030d7fa640 1 -- 172.21.15.16:0/2590796758 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03280315c0 con 0x55b6693a8050 2024-04-25T10:11:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.286+0000 7f030d7fa640 1 -- 172.21.15.16:0/2590796758 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0318088260 con 0x55b6693a8050 2024-04-25T10:11:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.286+0000 7f0337dd8740 10 client.7698 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T10:11:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.286+0000 7f0337dd8740 10 client.7698 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:11:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.286+0000 7f0337dd8740 20 client.7698 populate_metadata read hostname 'smithi016' 2024-04-25T10:11:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.286+0000 7f0337dd8740 1 --2- 172.21.15.16:0/2590796758 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55b6693d2ca0 0x55b6693d3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.286+0000 7f0337dd8740 4 client.7698 mds_command: new command op to 7335 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:21.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.286+0000 7f0337dd8740 1 -- 172.21.15.16:0/2590796758 --> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6692a0e60 con 0x55b6693d2ca0 2024-04-25T10:11:21.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.286+0000 7f032e59c640 1 --2- 172.21.15.16:0/2590796758 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55b6693d2ca0 0x55b6693d3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.286+0000 7f032e59c640 1 --2- 172.21.15.16:0/2590796758 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55b6693d2ca0 0x55b6693d3080 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4cb000703b553d8 server_cookie=a0cbf328586473b0 in_seq=0 out_seq=0 2024-04-25T10:11:21.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.286+0000 7f030d7fa640 10 client.7698 ms_handle_connect on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:21.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.287+0000 7f030d7fa640 1 -- 172.21.15.16:0/2590796758 <== mds.0 v2:172.21.15.186:6834/3950732718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b6692a0e60 con 0x55b6693d2ca0 2024-04-25T10:11:21.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.287+0000 7f030d7fa640 10 client.7698 handle_command_reply: tid=0 2024-04-25T10:11:21.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.287+0000 7f0337dd8740 1 client.7698 shutdown 2024-04-25T10:11:21.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.287+0000 7f0337dd8740 1 -- 172.21.15.16:0/2590796758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 msgr2=0x55b6693c6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.287+0000 7f0337dd8740 1 --2- 172.21.15.16:0/2590796758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 0x55b6693c6b30 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0318007410 tx=0x7f031808a1f0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.287+0000 7f0337dd8740 1 -- 172.21.15.16:0/2590796758 shutdown_connections 2024-04-25T10:11:21.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.287+0000 7f0337dd8740 1 --2- 172.21.15.16:0/2590796758 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55b6693d2ca0 0x55b6693d3080 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.287+0000 7f0337dd8740 1 --2- 172.21.15.16:0/2590796758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b6693b9c60 0x55b6693d3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.287+0000 7f0337dd8740 1 --2- 172.21.15.16:0/2590796758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 0x55b6693c6b30 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.287+0000 7f0337dd8740 1 --2- 172.21.15.16:0/2590796758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b66937d590 0x55b6693a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.287+0000 7f030d7fa640 0 client.7698 ms_handle_reset on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:21.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.287+0000 7f0337dd8740 1 -- 172.21.15.16:0/2590796758 >> 172.21.15.16:0/2590796758 conn(0x55b6693456e0 msgr2=0x55b6693a5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.287+0000 7f0337dd8740 1 -- 172.21.15.16:0/2590796758 shutdown_connections 2024-04-25T10:11:21.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.287+0000 7f0337dd8740 1 -- 172.21.15.16:0/2590796758 wait complete. 2024-04-25T10:11:21.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.287+0000 7f0337dd8740 20 client.7698 trim_cache size 0 max 16384 2024-04-25T10:11:21.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.299+0000 7f0337dd8740 1 Processor -- start 2024-04-25T10:11:21.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.299+0000 7f0337dd8740 1 -- start start 2024-04-25T10:11:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.300+0000 7f0337dd8740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b66937d590 0x55b6693a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.300+0000 7f0337dd8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 0x55b6693a1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.300+0000 7f0337dd8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b6693b9c60 0x55b6693d3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.300+0000 7f0337dd8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03301ff150 con 0x55b6693a8050 2024-04-25T10:11:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.300+0000 7f0337dd8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03300ad5d0 con 0x55b6693b9c60 2024-04-25T10:11:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.300+0000 7f0337dd8740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0330150310 con 0x55b66937d590 2024-04-25T10:11:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.300+0000 7f0327fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 0x55b6693a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.300+0000 7f0327fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 0x55b6693a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53152/0 (socket says 172.21.15.16:53152) 2024-04-25T10:11:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.300+0000 7f0327fff640 1 -- 172.21.15.16:0/1725764166 learned_addr learned my addr 172.21.15.16:0/1725764166 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.300+0000 7f032ed9d640 1 --2- 172.21.15.16:0/1725764166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b6693b9c60 0x55b6693d3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.300+0000 7f032e59c640 1 --2- 172.21.15.16:0/1725764166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b66937d590 0x55b6693a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.300+0000 7f0327fff640 1 -- 172.21.15.16:0/1725764166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b66937d590 msgr2=0x55b6693a51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.300+0000 7f0327fff640 1 --2- 172.21.15.16:0/1725764166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b66937d590 0x55b6693a51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.300+0000 7f0327fff640 1 -- 172.21.15.16:0/1725764166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b6693b9c60 msgr2=0x55b6693d3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.300+0000 7f0327fff640 1 --2- 172.21.15.16:0/1725764166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b6693b9c60 0x55b6693d3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.300+0000 7f0327fff640 1 -- 172.21.15.16:0/1725764166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b669379d80 con 0x55b6693a8050 2024-04-25T10:11:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.300+0000 7f0327fff640 1 --2- 172.21.15.16:0/1725764166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 0x55b6693a1870 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f03180552b0 tx=0x7f031801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7911b1030f80bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.301+0000 7f030f7fe640 1 -- 172.21.15.16:0/1725764166 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031807c070 con 0x55b6693a8050 2024-04-25T10:11:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.301+0000 7f030f7fe640 1 -- 172.21.15.16:0/1725764166 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0318077350 con 0x55b6693a8050 2024-04-25T10:11:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.301+0000 7f030f7fe640 1 -- 172.21.15.16:0/1725764166 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318070050 con 0x55b6693a8050 2024-04-25T10:11:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.301+0000 7f0337dd8740 1 -- 172.21.15.16:0/1725764166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 msgr2=0x55b6693a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.301+0000 7f0337dd8740 1 --2- 172.21.15.16:0/1725764166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 0x55b6693a1870 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f03180552b0 tx=0x7f031801c000 comp rx=0 tx=0).stop 2024-04-25T10:11:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.301+0000 7f0337dd8740 1 -- 172.21.15.16:0/1725764166 shutdown_connections 2024-04-25T10:11:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.301+0000 7f0337dd8740 1 --2- 172.21.15.16:0/1725764166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b6693b9c60 0x55b6693d3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.301+0000 7f0337dd8740 1 --2- 172.21.15.16:0/1725764166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6693a8050 0x55b6693a1870 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.301+0000 7f0337dd8740 1 --2- 172.21.15.16:0/1725764166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b66937d590 0x55b6693a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.301+0000 7f0337dd8740 1 -- 172.21.15.16:0/1725764166 >> 172.21.15.16:0/1725764166 conn(0x55b6693456e0 msgr2=0x55b6693a45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.301+0000 7f0337dd8740 1 -- 172.21.15.16:0/1725764166 shutdown_connections 2024-04-25T10:11:21.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.301+0000 7f0337dd8740 1 -- 172.21.15.16:0/1725764166 wait complete. 2024-04-25T10:11:21.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.302+0000 7f0337dd8740 1 -- start start 2024-04-25T10:11:21.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.302+0000 7f0337dd8740 1 Processor -- start 2024-04-25T10:11:21.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f0337dd8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b66937d590 0x55b66942c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f0337dd8740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b6693a8050 0x55b6693d3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f0337dd8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b6693b9c60 0x55b6693d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f0337dd8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6693d4400 con 0x55b66937d590 2024-04-25T10:11:21.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f0337dd8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6693c7070 con 0x55b6693b9c60 2024-04-25T10:11:21.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f0337dd8740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6693d64d0 con 0x55b6693a8050 2024-04-25T10:11:21.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f0337dd8740 10 client.0 _ll_register_callbacks cb 0x7f0328034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:21.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f032e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b66937d590 0x55b66942c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f032e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b66937d590 0x55b66942c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53168/0 (socket says 172.21.15.16:53168) 2024-04-25T10:11:21.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f032e59c640 1 -- 172.21.15.16:0/893904766 learned_addr learned my addr 172.21.15.16:0/893904766 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:21.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f0327fff640 1 --2- 172.21.15.16:0/893904766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b6693a8050 0x55b6693d3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f032ed9d640 1 --2- 172.21.15.16:0/893904766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b6693b9c60 0x55b6693d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f032e59c640 1 -- 172.21.15.16:0/893904766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b6693a8050 msgr2=0x55b6693d3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f032e59c640 1 --2- 172.21.15.16:0/893904766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b6693a8050 0x55b6693d3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f032e59c640 1 -- 172.21.15.16:0/893904766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b6693b9c60 msgr2=0x55b6693d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f032e59c640 1 --2- 172.21.15.16:0/893904766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b6693b9c60 0x55b6693d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f032e59c640 1 -- 172.21.15.16:0/893904766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b66937f010 con 0x55b66937d590 2024-04-25T10:11:21.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f032e59c640 1 --2- 172.21.15.16:0/893904766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b66937d590 0x55b66942c910 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x55b6693bd2d0 tx=0x7f0320004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad06575542887c3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f030d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:11:21.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f030d7fa640 1 -- 172.21.15.16:0/893904766 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0320013ba0 con 0x55b66937d590 2024-04-25T10:11:21.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f030d7fa640 1 -- 172.21.15.16:0/893904766 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0320013d40 con 0x55b66937d590 2024-04-25T10:11:21.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.303+0000 7f0337dd8740 1 -- 172.21.15.16:0/893904766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0318077350 con 0x55b66937d590 2024-04-25T10:11:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.304+0000 7f030d7fa640 1 -- 172.21.15.16:0/893904766 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0320037050 con 0x55b66937d590 2024-04-25T10:11:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.304+0000 7f0337dd8740 10 client.7704 fetch_fsmap learned FSMap version 58 2024-04-25T10:11:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.304+0000 7f0337dd8740 1 -- 172.21.15.16:0/893904766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x55b6693b89e0 con 0x55b66937d590 2024-04-25T10:11:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.304+0000 7f030d7fa640 1 -- 172.21.15.16:0/893904766 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0318077350 con 0x55b66937d590 2024-04-25T10:11:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.304+0000 7f030d7fa640 1 -- 172.21.15.16:0/893904766 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f032003a020 con 0x55b66937d590 2024-04-25T10:11:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.304+0000 7f0337dd8740 10 client.7704 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T10:11:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.304+0000 7f0337dd8740 10 client.7704 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:11:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.304+0000 7f0337dd8740 20 client.7704 populate_metadata read hostname 'smithi016' 2024-04-25T10:11:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.304+0000 7f0337dd8740 1 --2- 172.21.15.16:0/893904766 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55b6693d5100 0x55b6693d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.304+0000 7f0337dd8740 4 client.7704 mds_command: new command op to 7335 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:11:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.304+0000 7f0337dd8740 1 -- 172.21.15.16:0/893904766 --> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55b6692ee1a0 con 0x55b6693d5100 2024-04-25T10:11:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.304+0000 7f0327fff640 1 --2- 172.21.15.16:0/893904766 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55b6693d5100 0x55b6693d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.304+0000 7f030d7fa640 10 client.7704 ms_handle_connect on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.304+0000 7f0327fff640 1 --2- 172.21.15.16:0/893904766 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55b6693d5100 0x55b6693d15c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbadc766ed48ca99 server_cookie=e236635f389610a2 in_seq=0 out_seq=0 2024-04-25T10:11:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.305+0000 7f030d7fa640 1 -- 172.21.15.16:0/893904766 <== mds.0 v2:172.21.15.186:6834/3950732718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+419 (crc 0 0 0) 0x55b6692ee1a0 con 0x55b6693d5100 2024-04-25T10:11:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.305+0000 7f030d7fa640 10 client.7704 handle_command_reply: tid=0 2024-04-25T10:11:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.305+0000 7f0337dd8740 1 client.7704 shutdown 2024-04-25T10:11:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.305+0000 7f0337dd8740 1 -- 172.21.15.16:0/893904766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b66937d590 msgr2=0x55b66942c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.305+0000 7f0337dd8740 1 --2- 172.21.15.16:0/893904766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b66937d590 0x55b66942c910 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x55b6693bd2d0 tx=0x7f0320004680 comp rx=0 tx=0).stop 2024-04-25T10:11:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.305+0000 7f0337dd8740 1 -- 172.21.15.16:0/893904766 shutdown_connections 2024-04-25T10:11:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.305+0000 7f0337dd8740 1 --2- 172.21.15.16:0/893904766 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55b6693d5100 0x55b6693d15c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.305+0000 7f0337dd8740 1 --2- 172.21.15.16:0/893904766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b6693b9c60 0x55b6693d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.305+0000 7f030d7fa640 0 client.7704 ms_handle_reset on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.305+0000 7f0337dd8740 1 --2- 172.21.15.16:0/893904766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55b6693a8050 0x55b6693d3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.305+0000 7f0337dd8740 1 --2- 172.21.15.16:0/893904766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b66937d590 0x55b66942c910 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.305+0000 7f0337dd8740 1 -- 172.21.15.16:0/893904766 >> 172.21.15.16:0/893904766 conn(0x55b6693456e0 msgr2=0x55b6693a8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.305+0000 7f0337dd8740 1 -- 172.21.15.16:0/893904766 shutdown_connections 2024-04-25T10:11:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.305+0000 7f0337dd8740 1 -- 172.21.15.16:0/893904766 wait complete. 2024-04-25T10:11:21.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.305+0000 7f0337dd8740 20 client.7704 trim_cache size 0 max 16384 2024-04-25T10:11:21.318 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:21.318 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-04-25T10:11:21.318 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:11:21.318 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-04-25T10:11:21.318 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 369417969, 2024-04-25T10:11:21.318 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511627777, 2024-04-25T10:11:21.318 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/testdir/hardlink" 2024-04-25T10:11:21.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:11:21.318 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:11:21.319 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "dir_frag", 2024-04-25T10:11:21.319 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1051714378, 2024-04-25T10:11:21.319 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511627776, 2024-04-25T10:11:21.319 INFO:teuthology.orchestra.run.smithi016.stdout: "frag": "*", 2024-04-25T10:11:21.319 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/dir" 2024-04-25T10:11:21.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:11:21.319 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:11:21.319 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-04-25T10:11:21.319 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1447701495, 2024-04-25T10:11:21.319 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511627776, 2024-04-25T10:11:21.319 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/dir" 2024-04-25T10:11:21.319 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:11:21.319 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-04-25T10:11:21.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.306+0000 7f0334ff9640 1 -- 172.21.15.16:0/3079175992 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f031006b4d0 msgr2=0x7f031006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.306+0000 7f0334ff9640 1 --2- 172.21.15.16:0/3079175992 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f031006b4d0 0x7f031006d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f032000a830 tx=0x7f0320008040 comp rx=0 tx=0).stop 2024-04-25T10:11:21.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.306+0000 7f0334ff9640 1 -- 172.21.15.16:0/3079175992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03300ce9d0 msgr2=0x7f03300b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.306+0000 7f0334ff9640 1 --2- 172.21.15.16:0/3079175992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03300ce9d0 0x7f03300b4370 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f031800c7f0 tx=0x7f031800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.306+0000 7f0334ff9640 1 -- 172.21.15.16:0/3079175992 shutdown_connections 2024-04-25T10:11:21.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.306+0000 7f0334ff9640 1 --2- 172.21.15.16:0/3079175992 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f031006b4d0 0x7f031006d990 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.306+0000 7f0334ff9640 1 --2- 172.21.15.16:0/3079175992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0330144e70 0x7f03300afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.306+0000 7f0334ff9640 1 --2- 172.21.15.16:0/3079175992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03300ce9d0 0x7f03300b4370 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.306+0000 7f0334ff9640 1 --2- 172.21.15.16:0/3079175992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03300cc300 0x7f03300b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.306+0000 7f0334ff9640 1 -- 172.21.15.16:0/3079175992 >> 172.21.15.16:0/3079175992 conn(0x7f033006aba0 msgr2=0x7f033013ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.306+0000 7f0334ff9640 1 -- 172.21.15.16:0/3079175992 shutdown_connections 2024-04-25T10:11:21.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.306+0000 7f0334ff9640 1 -- 172.21.15.16:0/3079175992 wait complete. 2024-04-25T10:11:21.323 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:11:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.416+0000 7fb1e7ec6640 1 Processor -- start 2024-04-25T10:11:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.417+0000 7fb1e7ec6640 1 -- start start 2024-04-25T10:11:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.417+0000 7fb1e7ec6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb1e009a9c0 0x7fb1e0166910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.417+0000 7fb1e7ec6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e009adc0 con 0x7fb1e0099a40 2024-04-25T10:11:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.417+0000 7fb1e7ec6640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e00cfb90 con 0x7fb1e015bd40 2024-04-25T10:11:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.417+0000 7fb1e7ec6640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e015cfd0 con 0x7fb1e009a9c0 2024-04-25T10:11:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.418+0000 7fb1e5c62640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb1e0099a40 0x7fb1e00cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42192/0 (socket says 172.21.15.16:42192) 2024-04-25T10:11:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.418+0000 7fb1e5c62640 1 -- 172.21.15.16:0/4226319425 learned_addr learned my addr 172.21.15.16:0/4226319425 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.418+0000 7fb1e6463640 1 --2- 172.21.15.16:0/4226319425 >> v2:172.21.15.186:3300/0 conn(0x7fb1e009a9c0 0x7fb1e0166910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.418+0000 7fb1e4c60640 1 -- 172.21.15.16:0/4226319425 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175589407 0 0) 0x7fb1e009adc0 con 0x7fb1e0099a40 2024-04-25T10:11:21.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.418+0000 7fb1e4c60640 1 -- 172.21.15.16:0/4226319425 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1cc003520 con 0x7fb1e0099a40 2024-04-25T10:11:21.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.418+0000 7fb1e4c60640 1 -- 172.21.15.16:0/4226319425 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 642879381 0 0) 0x7fb1cc003520 con 0x7fb1e0099a40 2024-04-25T10:11:21.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.418+0000 7fb1e6463640 1 -- 172.21.15.16:0/4226319425 >> v1:172.21.15.186:6789/0 conn(0x7fb1e015bd40 legacy=0x7fb1e0099e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.418+0000 7fb1e6463640 1 -- 172.21.15.16:0/4226319425 >> v1:172.21.15.16:6789/0 conn(0x7fb1e0099a40 legacy=0x7fb1e00cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.418+0000 7fb1e6463640 1 -- 172.21.15.16:0/4226319425 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e015c210 con 0x7fb1e009a9c0 2024-04-25T10:11:21.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.419+0000 7fb1e6463640 1 --2- 172.21.15.16:0/4226319425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e009a9c0 0x7fb1e0166910 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009d90 tx=0x7fb1d802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f913ae2111e6603 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.419+0000 7fb1e4c60640 1 -- 172.21.15.16:0/4226319425 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc001670 con 0x7fb1e009a9c0 2024-04-25T10:11:21.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.419+0000 7fb1e4c60640 1 -- 172.21.15.16:0/4226319425 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d8007540 con 0x7fb1e009a9c0 2024-04-25T10:11:21.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.419+0000 7fb1e4c60640 1 -- 172.21.15.16:0/4226319425 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d80078f0 con 0x7fb1e009a9c0 2024-04-25T10:11:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.420+0000 7fb1e7ec6640 1 -- 172.21.15.16:0/4226319425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e009a9c0 msgr2=0x7fb1e0166910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.420+0000 7fb1e7ec6640 1 --2- 172.21.15.16:0/4226319425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e009a9c0 0x7fb1e0166910 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009d90 tx=0x7fb1d802a680 comp rx=0 tx=0).stop 2024-04-25T10:11:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.420+0000 7fb1e7ec6640 1 -- 172.21.15.16:0/4226319425 shutdown_connections 2024-04-25T10:11:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.420+0000 7fb1e7ec6640 1 --2- 172.21.15.16:0/4226319425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e009a9c0 0x7fb1e0166910 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.420+0000 7fb1e7ec6640 1 -- 172.21.15.16:0/4226319425 >> 172.21.15.16:0/4226319425 conn(0x7fb1e00b32f0 msgr2=0x7fb1e00b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.420+0000 7fb1e7ec6640 1 -- 172.21.15.16:0/4226319425 shutdown_connections 2024-04-25T10:11:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.420+0000 7fb1e7ec6640 1 -- 172.21.15.16:0/4226319425 wait complete. 2024-04-25T10:11:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.421+0000 7fb1e7ec6640 1 Processor -- start 2024-04-25T10:11:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.421+0000 7fb1e7ec6640 1 -- start start 2024-04-25T10:11:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e7ec6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1e0099a40 0x7fb1e01e71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e7ec6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e009a9c0 0x7fb1e01f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e7ec6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e015bd40 0x7fb1e00af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e7ec6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e015cab0 con 0x7fb1e015bd40 2024-04-25T10:11:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e7ec6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e015c7d0 con 0x7fb1e009a9c0 2024-04-25T10:11:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e7ec6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e00cd400 con 0x7fb1e0099a40 2024-04-25T10:11:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e6463640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e015bd40 0x7fb1e00af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e6463640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e015bd40 0x7fb1e00af840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53172/0 (socket says 172.21.15.16:53172) 2024-04-25T10:11:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e6463640 1 -- 172.21.15.16:0/1201380961 learned_addr learned my addr 172.21.15.16:0/1201380961 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e5c62640 1 --2- 172.21.15.16:0/1201380961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1e0099a40 0x7fb1e01e71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e5461640 1 --2- 172.21.15.16:0/1201380961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e009a9c0 0x7fb1e01f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e6463640 1 -- 172.21.15.16:0/1201380961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1e0099a40 msgr2=0x7fb1e01e71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e6463640 1 --2- 172.21.15.16:0/1201380961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1e0099a40 0x7fb1e01e71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e6463640 1 -- 172.21.15.16:0/1201380961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e009a9c0 msgr2=0x7fb1e01f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e6463640 1 --2- 172.21.15.16:0/1201380961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e009a9c0 0x7fb1e01f3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e6463640 1 -- 172.21.15.16:0/1201380961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e00b7580 con 0x7fb1e015bd40 2024-04-25T10:11:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e6463640 1 --2- 172.21.15.16:0/1201380961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e015bd40 0x7fb1e00af840 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009eb0 tx=0x7fb1d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1d67fc640 1 -- 172.21.15.16:0/1201380961 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8040070 con 0x7fb1e015bd40 2024-04-25T10:11:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1d67fc640 1 -- 172.21.15.16:0/1201380961 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d8031c40 con 0x7fb1e015bd40 2024-04-25T10:11:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e7ec6640 1 -- 172.21.15.16:0/1201380961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e00b6240 con 0x7fb1e015bd40 2024-04-25T10:11:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1d67fc640 1 -- 172.21.15.16:0/1201380961 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8009450 con 0x7fb1e015bd40 2024-04-25T10:11:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.422+0000 7fb1e7ec6640 1 -- 172.21.15.16:0/1201380961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e00b6aa0 con 0x7fb1e015bd40 2024-04-25T10:11:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.423+0000 7fb1e7ec6640 1 -- 172.21.15.16:0/1201380961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e009a570 con 0x7fb1e015bd40 2024-04-25T10:11:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.424+0000 7fb1d67fc640 1 -- 172.21.15.16:0/1201380961 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb1d8039040 con 0x7fb1e015bd40 2024-04-25T10:11:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.424+0000 7fb1d67fc640 1 --2- 172.21.15.16:0/1201380961 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb1c406b4d0 0x7fb1c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.424+0000 7fb1d67fc640 1 -- 172.21.15.16:0/1201380961 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb1d80ab6f0 con 0x7fb1e015bd40 2024-04-25T10:11:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.425+0000 7fb1e5c62640 1 --2- 172.21.15.16:0/1201380961 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb1c406b4d0 0x7fb1c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.425+0000 7fb1e5c62640 1 --2- 172.21.15.16:0/1201380961 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb1c406b4d0 0x7fb1c406d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc005880 tx=0x7fb1dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.429+0000 7fb1d67fc640 1 -- 172.21.15.16:0/1201380961 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1d804c350 con 0x7fb1e015bd40 2024-04-25T10:11:21.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.646+0000 7fb1e7ec6640 1 -- 172.21.15.16:0/1201380961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb1e00744a0 con 0x7fb1e015bd40 2024-04-25T10:11:21.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.646+0000 7fb1d67fc640 1 -- 172.21.15.16:0/1201380961 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fb1d807a520 con 0x7fb1e015bd40 2024-04-25T10:11:21.653 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:21.653 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:11:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.648+0000 7fb1e7ec6640 1 -- 172.21.15.16:0/1201380961 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb1c406b4d0 msgr2=0x7fb1c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.648+0000 7fb1e7ec6640 1 --2- 172.21.15.16:0/1201380961 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb1c406b4d0 0x7fb1c406d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc005880 tx=0x7fb1dc009040 comp rx=0 tx=0).stop 2024-04-25T10:11:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.648+0000 7fb1e7ec6640 1 -- 172.21.15.16:0/1201380961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e015bd40 msgr2=0x7fb1e00af840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.648+0000 7fb1e7ec6640 1 --2- 172.21.15.16:0/1201380961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e015bd40 0x7fb1e00af840 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009eb0 tx=0x7fb1d8004a40 comp rx=0 tx=0).stop 2024-04-25T10:11:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.648+0000 7fb1e7ec6640 1 -- 172.21.15.16:0/1201380961 shutdown_connections 2024-04-25T10:11:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.648+0000 7fb1e7ec6640 1 --2- 172.21.15.16:0/1201380961 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb1c406b4d0 0x7fb1c406d990 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.648+0000 7fb1e7ec6640 1 --2- 172.21.15.16:0/1201380961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e015bd40 0x7fb1e00af840 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.648+0000 7fb1e7ec6640 1 --2- 172.21.15.16:0/1201380961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1e009a9c0 0x7fb1e01f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.648+0000 7fb1e7ec6640 1 --2- 172.21.15.16:0/1201380961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1e0099a40 0x7fb1e01e71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.648+0000 7fb1e7ec6640 1 -- 172.21.15.16:0/1201380961 >> 172.21.15.16:0/1201380961 conn(0x7fb1e00b32f0 msgr2=0x7fb1e00ae790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.648+0000 7fb1e7ec6640 1 -- 172.21.15.16:0/1201380961 shutdown_connections 2024-04-25T10:11:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.648+0000 7fb1e7ec6640 1 -- 172.21.15.16:0/1201380961 wait complete. 2024-04-25T10:11:21.665 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:11:21.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:21.665 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-25T10:11:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.764+0000 7fc9f2f61640 1 Processor -- start 2024-04-25T10:11:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.764+0000 7fc9f2f61640 1 -- start start 2024-04-25T10:11:21.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.765+0000 7fc9f2f61640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9ec0b4900 0x7fc9ec1654c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.765+0000 7fc9f2f61640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ec06da40 con 0x7fc9ec0aedc0 2024-04-25T10:11:21.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.765+0000 7fc9f2f61640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ec06dd20 con 0x7fc9ec15ae20 2024-04-25T10:11:21.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.765+0000 7fc9f2f61640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ec15c050 con 0x7fc9ec0b4900 2024-04-25T10:11:21.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.765+0000 7fc9f14fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9ec0b4900 0x7fc9ec1654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.765+0000 7fc9f0cfd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc9ec0aedc0 0x7fc9ec06d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42194/0 (socket says 172.21.15.16:42194) 2024-04-25T10:11:21.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.765+0000 7fc9f0cfd640 1 -- 172.21.15.16:0/671888354 learned_addr learned my addr 172.21.15.16:0/671888354 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.765+0000 7fc9eb7fe640 1 -- 172.21.15.16:0/671888354 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505317320 0 0) 0x7fc9ec06da40 con 0x7fc9ec0aedc0 2024-04-25T10:11:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.765+0000 7fc9eb7fe640 1 -- 172.21.15.16:0/671888354 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9dc003520 con 0x7fc9ec0aedc0 2024-04-25T10:11:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.765+0000 7fc9f14fe640 1 -- 172.21.15.16:0/671888354 >> v1:172.21.15.186:6789/0 conn(0x7fc9ec15ae20 legacy=0x7fc9ec0b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.765+0000 7fc9f14fe640 1 -- 172.21.15.16:0/671888354 >> v1:172.21.15.16:6789/0 conn(0x7fc9ec0aedc0 legacy=0x7fc9ec06d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.765+0000 7fc9f14fe640 1 -- 172.21.15.16:0/671888354 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ec06e000 con 0x7fc9ec0b4900 2024-04-25T10:11:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.766+0000 7fc9f14fe640 1 --2- 172.21.15.16:0/671888354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ec0b4900 0x7fc9ec1654c0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009d90 tx=0x7fc9e402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cfd7d9418b74f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.766+0000 7fc9eb7fe640 1 -- 172.21.15.16:0/671888354 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4031600 con 0x7fc9ec0b4900 2024-04-25T10:11:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.766+0000 7fc9eb7fe640 1 -- 172.21.15.16:0/671888354 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e40317a0 con 0x7fc9ec0b4900 2024-04-25T10:11:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.766+0000 7fc9eb7fe640 1 -- 172.21.15.16:0/671888354 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4031b70 con 0x7fc9ec0b4900 2024-04-25T10:11:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.767+0000 7fc9f2f61640 1 -- 172.21.15.16:0/671888354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ec0b4900 msgr2=0x7fc9ec1654c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.767+0000 7fc9f2f61640 1 --2- 172.21.15.16:0/671888354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ec0b4900 0x7fc9ec1654c0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009d90 tx=0x7fc9e402a610 comp rx=0 tx=0).stop 2024-04-25T10:11:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.767+0000 7fc9f2f61640 1 -- 172.21.15.16:0/671888354 shutdown_connections 2024-04-25T10:11:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.767+0000 7fc9f2f61640 1 --2- 172.21.15.16:0/671888354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ec0b4900 0x7fc9ec1654c0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.767+0000 7fc9f2f61640 1 -- 172.21.15.16:0/671888354 >> 172.21.15.16:0/671888354 conn(0x7fc9ec06aba0 msgr2=0x7fc9ec06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.767+0000 7fc9f2f61640 1 -- 172.21.15.16:0/671888354 shutdown_connections 2024-04-25T10:11:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.767+0000 7fc9f2f61640 1 -- 172.21.15.16:0/671888354 wait complete. 2024-04-25T10:11:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.768+0000 7fc9f2f61640 1 Processor -- start 2024-04-25T10:11:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.768+0000 7fc9f2f61640 1 -- start start 2024-04-25T10:11:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.769+0000 7fc9f2f61640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9ec0aedc0 0x7fc9ec0c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.769+0000 7fc9f2f61640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9ec0b4900 0x7fc9ec14eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.769+0000 7fc9f2f61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ec15ae20 0x7fc9ec0cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.769+0000 7fc9f2f61640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec15d7d0 con 0x7fc9ec15ae20 2024-04-25T10:11:21.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.769+0000 7fc9f2f61640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec0af1a0 con 0x7fc9ec0aedc0 2024-04-25T10:11:21.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.769+0000 7fc9f2f61640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec15d950 con 0x7fc9ec0b4900 2024-04-25T10:11:21.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.769+0000 7fc9f14fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ec15ae20 0x7fc9ec0cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.769+0000 7fc9f14fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ec15ae20 0x7fc9ec0cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53184/0 (socket says 172.21.15.16:53184) 2024-04-25T10:11:21.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.769+0000 7fc9f14fe640 1 -- 172.21.15.16:0/4110333723 learned_addr learned my addr 172.21.15.16:0/4110333723 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:21.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.769+0000 7fc9f0cfd640 1 --2- 172.21.15.16:0/4110333723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9ec0aedc0 0x7fc9ec0c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.769+0000 7fc9ebfff640 1 --2- 172.21.15.16:0/4110333723 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9ec0b4900 0x7fc9ec14eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.769+0000 7fc9f14fe640 1 -- 172.21.15.16:0/4110333723 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9ec0b4900 msgr2=0x7fc9ec14eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.769+0000 7fc9f14fe640 1 --2- 172.21.15.16:0/4110333723 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9ec0b4900 0x7fc9ec14eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.769+0000 7fc9f14fe640 1 -- 172.21.15.16:0/4110333723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9ec0aedc0 msgr2=0x7fc9ec0c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.769+0000 7fc9f14fe640 1 --2- 172.21.15.16:0/4110333723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9ec0aedc0 0x7fc9ec0c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.769+0000 7fc9f14fe640 1 -- 172.21.15.16:0/4110333723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ec1503b0 con 0x7fc9ec15ae20 2024-04-25T10:11:21.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.769+0000 7fc9f0cfd640 1 --2- 172.21.15.16:0/4110333723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9ec0aedc0 0x7fc9ec0c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:21.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.769+0000 7fc9ebfff640 1 --2- 172.21.15.16:0/4110333723 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9ec0b4900 0x7fc9ec14eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:21.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.770+0000 7fc9f14fe640 1 --2- 172.21.15.16:0/4110333723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ec15ae20 0x7fc9ec0cb380 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009eb0 tx=0x7fc9e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.770+0000 7fc9e97fa640 1 -- 172.21.15.16:0/4110333723 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4040070 con 0x7fc9ec15ae20 2024-04-25T10:11:21.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.770+0000 7fc9e97fa640 1 -- 172.21.15.16:0/4110333723 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e4030b60 con 0x7fc9ec15ae20 2024-04-25T10:11:21.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.770+0000 7fc9f2f61640 1 -- 172.21.15.16:0/4110333723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ec1536f0 con 0x7fc9ec15ae20 2024-04-25T10:11:21.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.770+0000 7fc9e97fa640 1 -- 172.21.15.16:0/4110333723 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4009450 con 0x7fc9ec15ae20 2024-04-25T10:11:21.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.770+0000 7fc9f2f61640 1 -- 172.21.15.16:0/4110333723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ec14fb10 con 0x7fc9ec15ae20 2024-04-25T10:11:21.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.771+0000 7fc9e97fa640 1 -- 172.21.15.16:0/4110333723 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc9e4039040 con 0x7fc9ec15ae20 2024-04-25T10:11:21.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.772+0000 7fc9e97fa640 1 --2- 172.21.15.16:0/4110333723 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc9d006b4d0 0x7fc9d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.772+0000 7fc9e97fa640 1 -- 172.21.15.16:0/4110333723 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc9e40ab450 con 0x7fc9ec15ae20 2024-04-25T10:11:21.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.772+0000 7fc9f0cfd640 1 --2- 172.21.15.16:0/4110333723 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc9d006b4d0 0x7fc9d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.773+0000 7fc9f0cfd640 1 --2- 172.21.15.16:0/4110333723 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc9d006b4d0 0x7fc9d006d990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0007fd0 tx=0x7fc9e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.773+0000 7fc9f5c5e740 1 Processor -- start 2024-04-25T10:11:21.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.773+0000 7fc9f5c5e740 1 -- start start 2024-04-25T10:11:21.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.773+0000 7fc9f5c5e740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9564cbb0 0x556a9564cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.773+0000 7fc9f5c5e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a95647830 0x556a9564b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.773+0000 7fc9f5c5e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9564bcf0 0x556a9564c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.773+0000 7fc9f5c5e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a955b0ac0 con 0x556a95647830 2024-04-25T10:11:21.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.773+0000 7fc9f5c5e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a95559d80 con 0x556a9564bcf0 2024-04-25T10:11:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.773+0000 7fc9f5c5e740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a955acb60 con 0x556a9564cbb0 2024-04-25T10:11:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.774+0000 7fc9ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a95647830 0x556a9564b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.774+0000 7fc9ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a95647830 0x556a9564b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53200/0 (socket says 172.21.15.16:53200) 2024-04-25T10:11:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.774+0000 7fc9ebfff640 1 -- 172.21.15.16:0/1596698946 learned_addr learned my addr 172.21.15.16:0/1596698946 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.774+0000 7fc9f0cfd640 1 --2- 172.21.15.16:0/1596698946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9564cbb0 0x556a9564cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.774+0000 7fc9f14fe640 1 --2- 172.21.15.16:0/1596698946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9564bcf0 0x556a9564c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.774+0000 7fc9ebfff640 1 -- 172.21.15.16:0/1596698946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9564cbb0 msgr2=0x556a9564cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.774+0000 7fc9ebfff640 1 --2- 172.21.15.16:0/1596698946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9564cbb0 0x556a9564cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.774+0000 7fc9ebfff640 1 -- 172.21.15.16:0/1596698946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9564bcf0 msgr2=0x556a9564c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.774+0000 7fc9ebfff640 1 --2- 172.21.15.16:0/1596698946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9564bcf0 0x556a9564c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.774+0000 7fc9ebfff640 1 -- 172.21.15.16:0/1596698946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a9538d7e0 con 0x556a95647830 2024-04-25T10:11:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.774+0000 7fc9ebfff640 1 --2- 172.21.15.16:0/1596698946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a95647830 0x556a9564b7b0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8006fd0 tx=0x7fc9d800bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d461c3116b3438e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.774+0000 7fc9d67fc640 1 -- 172.21.15.16:0/1596698946 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8004020 con 0x556a95647830 2024-04-25T10:11:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.774+0000 7fc9d67fc640 1 -- 172.21.15.16:0/1596698946 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d80041c0 con 0x556a95647830 2024-04-25T10:11:21.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.774+0000 7fc9d67fc640 1 -- 172.21.15.16:0/1596698946 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80043b0 con 0x556a95647830 2024-04-25T10:11:21.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.775+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/1596698946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a95647830 msgr2=0x556a9564b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.775+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/1596698946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a95647830 0x556a9564b7b0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8006fd0 tx=0x7fc9d800bc60 comp rx=0 tx=0).stop 2024-04-25T10:11:21.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.775+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/1596698946 shutdown_connections 2024-04-25T10:11:21.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.775+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/1596698946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9564bcf0 0x556a9564c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.775+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/1596698946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a95647830 0x556a9564b7b0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.775+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/1596698946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9564cbb0 0x556a9564cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.775+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/1596698946 >> 172.21.15.16:0/1596698946 conn(0x556a95649cb0 msgr2=0x556a95692770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.775+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/1596698946 shutdown_connections 2024-04-25T10:11:21.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.775+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/1596698946 wait complete. 2024-04-25T10:11:21.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.775+0000 7fc9f5c5e740 1 -- start start 2024-04-25T10:11:21.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.776+0000 7fc9f5c5e740 1 Processor -- start 2024-04-25T10:11:21.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.776+0000 7fc9f5c5e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564bcf0 0x556a95669bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.776+0000 7fc9f5c5e740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9564cbb0 0x556a95698a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.776+0000 7fc9f5c5e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9569ac20 0x556a95698fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.776+0000 7fc9f5c5e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a95678740 con 0x556a9564bcf0 2024-04-25T10:11:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.776+0000 7fc9f5c5e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a9554f530 con 0x556a9569ac20 2024-04-25T10:11:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.776+0000 7fc9f5c5e740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a95677bc0 con 0x556a9564cbb0 2024-04-25T10:11:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.776+0000 7fc9f5c5e740 10 client.0 _ll_register_callbacks cb 0x556a955a7130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.776+0000 7fc9f0cfd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564bcf0 0x556a95669bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.776+0000 7fc9f0cfd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564bcf0 0x556a95669bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53202/0 (socket says 172.21.15.16:53202) 2024-04-25T10:11:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.777+0000 7fc9f0cfd640 1 -- 172.21.15.16:0/3833676296 learned_addr learned my addr 172.21.15.16:0/3833676296 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.777+0000 7fc9ebfff640 1 --2- 172.21.15.16:0/3833676296 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9564cbb0 0x556a95698a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.777+0000 7fc9f14fe640 1 --2- 172.21.15.16:0/3833676296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9569ac20 0x556a95698fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.777+0000 7fc9f0cfd640 1 -- 172.21.15.16:0/3833676296 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9564cbb0 msgr2=0x556a95698a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.777+0000 7fc9f0cfd640 1 --2- 172.21.15.16:0/3833676296 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9564cbb0 0x556a95698a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.777+0000 7fc9f0cfd640 1 -- 172.21.15.16:0/3833676296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9569ac20 msgr2=0x556a95698fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.777+0000 7fc9f0cfd640 1 --2- 172.21.15.16:0/3833676296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9569ac20 0x556a95698fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.777+0000 7fc9f0cfd640 1 -- 172.21.15.16:0/3833676296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a953edf70 con 0x556a9564bcf0 2024-04-25T10:11:21.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.777+0000 7fc9f0cfd640 1 --2- 172.21.15.16:0/3833676296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564bcf0 0x556a95669bd0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00070d0 tx=0x7fc9e0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcdc9c347bb6d397 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.777+0000 7fc9cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:11:21.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.777+0000 7fc9cb7fe640 1 -- 172.21.15.16:0/3833676296 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0027060 con 0x556a9564bcf0 2024-04-25T10:11:21.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.777+0000 7fc9cb7fe640 1 -- 172.21.15.16:0/3833676296 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc9e0010040 con 0x556a9564bcf0 2024-04-25T10:11:21.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.777+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/3833676296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9d80041c0 con 0x556a9564bcf0 2024-04-25T10:11:21.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.777+0000 7fc9cb7fe640 1 -- 172.21.15.16:0/3833676296 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9e0007590 con 0x556a9564bcf0 2024-04-25T10:11:21.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.777+0000 7fc9f5c5e740 10 client.7725 fetch_fsmap learned FSMap version 58 2024-04-25T10:11:21.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.777+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/3833676296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x556a95285170 con 0x556a9564bcf0 2024-04-25T10:11:21.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.777+0000 7fc9cb7fe640 1 -- 172.21.15.16:0/3833676296 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d80041c0 con 0x556a9564bcf0 2024-04-25T10:11:21.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.778+0000 7fc9cb7fe640 1 -- 172.21.15.16:0/3833676296 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc9e0013420 con 0x556a9564bcf0 2024-04-25T10:11:21.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.778+0000 7fc9f5c5e740 10 client.7725 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T10:11:21.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.778+0000 7fc9f5c5e740 10 client.7725 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T10:11:21.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.778+0000 7fc9f5c5e740 20 client.7725 populate_metadata read hostname 'smithi016' 2024-04-25T10:11:21.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.778+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/3833676296 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x556a95670d40 0x556a95697310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.778+0000 7fc9f5c5e740 4 client.7725 mds_command: new command op to 7335 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:21.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.778+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/3833676296 --> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a955be000 con 0x556a95670d40 2024-04-25T10:11:21.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.778+0000 7fc9ebfff640 1 --2- 172.21.15.16:0/3833676296 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x556a95670d40 0x556a95697310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.778+0000 7fc9ebfff640 1 --2- 172.21.15.16:0/3833676296 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x556a95670d40 0x556a95697310 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a1d13bd4d83c4d8 server_cookie=32a6abf66e2d9002 in_seq=0 out_seq=0 2024-04-25T10:11:21.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.778+0000 7fc9cb7fe640 10 client.7725 ms_handle_connect on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:21.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.779+0000 7fc9cb7fe640 1 -- 172.21.15.16:0/3833676296 <== mds.0 v2:172.21.15.186:6834/3950732718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556a955be000 con 0x556a95670d40 2024-04-25T10:11:21.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.779+0000 7fc9cb7fe640 10 client.7725 handle_command_reply: tid=0 2024-04-25T10:11:21.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.779+0000 7fc9f5c5e740 1 client.7725 shutdown 2024-04-25T10:11:21.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.779+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/3833676296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564bcf0 msgr2=0x556a95669bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.779+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/3833676296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564bcf0 0x556a95669bd0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00070d0 tx=0x7fc9e0000ed0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.779+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/3833676296 shutdown_connections 2024-04-25T10:11:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.779+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/3833676296 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x556a95670d40 0x556a95697310 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.779+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/3833676296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9569ac20 0x556a95698fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.779+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/3833676296 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9564cbb0 0x556a95698a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.779+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/3833676296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564bcf0 0x556a95669bd0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.779+0000 7fc9cb7fe640 0 client.7725 ms_handle_reset on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.779+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/3833676296 >> 172.21.15.16:0/3833676296 conn(0x556a95647830 msgr2=0x556a9564d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.779+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/3833676296 shutdown_connections 2024-04-25T10:11:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.779+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/3833676296 wait complete. 2024-04-25T10:11:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.779+0000 7fc9f5c5e740 20 client.7725 trim_cache size 0 max 16384 2024-04-25T10:11:21.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.795+0000 7fc9f5c5e740 1 Processor -- start 2024-04-25T10:11:21.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.795+0000 7fc9f5c5e740 1 -- start start 2024-04-25T10:11:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.795+0000 7fc9f5c5e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9564bcf0 0x556a956b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.795+0000 7fc9f5c5e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564cbb0 0x556a95679c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.795+0000 7fc9f5c5e740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9569ac20 0x556a95697500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.795+0000 7fc9f5c5e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec1ffeb0 con 0x556a9564cbb0 2024-04-25T10:11:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.795+0000 7fc9f5c5e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec1ff070 con 0x556a9564bcf0 2024-04-25T10:11:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.795+0000 7fc9f5c5e740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec0c5680 con 0x556a9569ac20 2024-04-25T10:11:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.795+0000 7fc9ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564cbb0 0x556a95679c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.796+0000 7fc9ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564cbb0 0x556a95679c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53214/0 (socket says 172.21.15.16:53214) 2024-04-25T10:11:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.796+0000 7fc9ebfff640 1 -- 172.21.15.16:0/475385402 learned_addr learned my addr 172.21.15.16:0/475385402 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.796+0000 7fc9f0cfd640 1 --2- 172.21.15.16:0/475385402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9564bcf0 0x556a956b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.796+0000 7fc9f14fe640 1 --2- 172.21.15.16:0/475385402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9569ac20 0x556a95697500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.796+0000 7fc9ebfff640 1 -- 172.21.15.16:0/475385402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9569ac20 msgr2=0x556a95697500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.796+0000 7fc9ebfff640 1 --2- 172.21.15.16:0/475385402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9569ac20 0x556a95697500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.796+0000 7fc9ebfff640 1 -- 172.21.15.16:0/475385402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9564bcf0 msgr2=0x556a956b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.796+0000 7fc9ebfff640 1 --2- 172.21.15.16:0/475385402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9564bcf0 0x556a956b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.796+0000 7fc9ebfff640 1 -- 172.21.15.16:0/475385402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a9566c330 con 0x556a9564cbb0 2024-04-25T10:11:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.796+0000 7fc9f0cfd640 1 --2- 172.21.15.16:0/475385402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9564bcf0 0x556a956b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.796+0000 7fc9f14fe640 1 --2- 172.21.15.16:0/475385402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9569ac20 0x556a95697500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.796+0000 7fc9ebfff640 1 --2- 172.21.15.16:0/475385402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564cbb0 0x556a95679c50 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8014000 tx=0x7fc9d8009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4280898fff8dbe70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.796+0000 7fc9d5ffb640 1 -- 172.21.15.16:0/475385402 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d802a070 con 0x556a9564cbb0 2024-04-25T10:11:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.796+0000 7fc9d5ffb640 1 -- 172.21.15.16:0/475385402 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d8024b00 con 0x556a9564cbb0 2024-04-25T10:11:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.796+0000 7fc9d5ffb640 1 -- 172.21.15.16:0/475385402 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d800d9d0 con 0x556a9564cbb0 2024-04-25T10:11:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.797+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/475385402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564cbb0 msgr2=0x556a95679c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.797+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/475385402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564cbb0 0x556a95679c50 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8014000 tx=0x7fc9d8009000 comp rx=0 tx=0).stop 2024-04-25T10:11:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.797+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/475385402 shutdown_connections 2024-04-25T10:11:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.797+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/475385402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9569ac20 0x556a95697500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.797+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/475385402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564cbb0 0x556a95679c50 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.797+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/475385402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9564bcf0 0x556a956b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.797+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/475385402 >> 172.21.15.16:0/475385402 conn(0x556a95647830 msgr2=0x556a9564a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.797+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/475385402 shutdown_connections 2024-04-25T10:11:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.797+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/475385402 wait complete. 2024-04-25T10:11:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.797+0000 7fc9f5c5e740 1 -- start start 2024-04-25T10:11:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.797+0000 7fc9f5c5e740 1 Processor -- start 2024-04-25T10:11:21.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.798+0000 7fc9f5c5e740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9564bcf0 0x556a956b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.798+0000 7fc9f5c5e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564cbb0 0x556a95697590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.798+0000 7fc9f5c5e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9569ac20 0x556a95698080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.798+0000 7fc9f5c5e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec0c5c80 con 0x556a9564cbb0 2024-04-25T10:11:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.798+0000 7fc9f5c5e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a956b4140 con 0x556a9569ac20 2024-04-25T10:11:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.798+0000 7fc9f5c5e740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a9564db00 con 0x556a9564bcf0 2024-04-25T10:11:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.798+0000 7fc9f5c5e740 10 client.0 _ll_register_callbacks cb 0x7fc9d800b200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.798+0000 7fc9ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564cbb0 0x556a95697590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.798+0000 7fc9ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564cbb0 0x556a95697590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53230/0 (socket says 172.21.15.16:53230) 2024-04-25T10:11:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.798+0000 7fc9ebfff640 1 -- 172.21.15.16:0/1145396599 learned_addr learned my addr 172.21.15.16:0/1145396599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.798+0000 7fc9f0cfd640 1 --2- 172.21.15.16:0/1145396599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9564bcf0 0x556a956b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.798+0000 7fc9f14fe640 1 --2- 172.21.15.16:0/1145396599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9569ac20 0x556a95698080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9ebfff640 1 -- 172.21.15.16:0/1145396599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9564bcf0 msgr2=0x556a956b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9ebfff640 1 --2- 172.21.15.16:0/1145396599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9564bcf0 0x556a956b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9ebfff640 1 -- 172.21.15.16:0/1145396599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9569ac20 msgr2=0x556a95698080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9ebfff640 1 --2- 172.21.15.16:0/1145396599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9569ac20 0x556a95698080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9ebfff640 1 -- 172.21.15.16:0/1145396599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a956b3c70 con 0x556a9564cbb0 2024-04-25T10:11:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9ebfff640 1 --2- 172.21.15.16:0/1145396599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564cbb0 0x556a95697590 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800b720 tx=0x7fc9d8004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98d7bd1e9460268 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:11:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9cb7fe640 1 -- 172.21.15.16:0/1145396599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d802a070 con 0x556a9564cbb0 2024-04-25T10:11:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9cb7fe640 1 -- 172.21.15.16:0/1145396599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc9d800dbe0 con 0x556a9564cbb0 2024-04-25T10:11:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/1145396599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9d8024b00 con 0x556a9564cbb0 2024-04-25T10:11:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9cb7fe640 1 -- 172.21.15.16:0/1145396599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9d8037050 con 0x556a9564cbb0 2024-04-25T10:11:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9f5c5e740 10 client.7731 fetch_fsmap learned FSMap version 58 2024-04-25T10:11:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/1145396599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x556a956b3e80 con 0x556a9564cbb0 2024-04-25T10:11:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9cb7fe640 1 -- 172.21.15.16:0/1145396599 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d8024b00 con 0x556a9564cbb0 2024-04-25T10:11:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9cb7fe640 1 -- 172.21.15.16:0/1145396599 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc9d8024300 con 0x556a9564cbb0 2024-04-25T10:11:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9f5c5e740 10 client.7731 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T10:11:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9f5c5e740 10 client.7731 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T10:11:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9f5c5e740 20 client.7731 populate_metadata read hostname 'smithi016' 2024-04-25T10:11:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/1145396599 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x556a95697ad0 0x556a956985c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9f5c5e740 4 client.7731 mds_command: new command op to 7335 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:11:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.799+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/1145396599 --> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x556a955b8350 con 0x556a95697ad0 2024-04-25T10:11:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.800+0000 7fc9f0cfd640 1 --2- 172.21.15.16:0/1145396599 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x556a95697ad0 0x556a956985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.800+0000 7fc9f0cfd640 1 --2- 172.21.15.16:0/1145396599 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x556a95697ad0 0x556a956985c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e21a6eeee28fbd5 server_cookie=cf043e6e832ffc47 in_seq=0 out_seq=0 2024-04-25T10:11:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.800+0000 7fc9cb7fe640 10 client.7731 ms_handle_connect on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.800+0000 7fc9cb7fe640 1 -- 172.21.15.16:0/1145396599 <== mds.0 v2:172.21.15.186:6834/3950732718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556a955b8350 con 0x556a95697ad0 2024-04-25T10:11:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9cb7fe640 10 client.7731 handle_command_reply: tid=0 2024-04-25T10:11:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f5c5e740 1 client.7731 shutdown 2024-04-25T10:11:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/1145396599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564cbb0 msgr2=0x556a95697590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/1145396599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564cbb0 0x556a95697590 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800b720 tx=0x7fc9d8004680 comp rx=0 tx=0).stop 2024-04-25T10:11:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/1145396599 shutdown_connections 2024-04-25T10:11:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/1145396599 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x556a95697ad0 0x556a956985c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/1145396599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556a9569ac20 0x556a95698080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/1145396599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a9564cbb0 0x556a95697590 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9cb7fe640 0 client.7731 ms_handle_reset on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f5c5e740 1 --2- 172.21.15.16:0/1145396599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556a9564bcf0 0x556a956b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/1145396599 >> 172.21.15.16:0/1145396599 conn(0x556a95647830 msgr2=0x556a95695880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/1145396599 shutdown_connections 2024-04-25T10:11:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f5c5e740 1 -- 172.21.15.16:0/1145396599 wait complete. 2024-04-25T10:11:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f5c5e740 20 client.7731 trim_cache size 0 max 16384 2024-04-25T10:11:21.815 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:11:21.815 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-04-25T10:11:21.815 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "e9beceb6-7061-4119-ba99-0fcb66a4a0a4", 2024-04-25T10:11:21.815 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-04-25T10:11:21.815 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:11:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f2f61640 1 -- 172.21.15.16:0/4110333723 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc9d006b4d0 msgr2=0x7fc9d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f2f61640 1 --2- 172.21.15.16:0/4110333723 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc9d006b4d0 0x7fc9d006d990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0007fd0 tx=0x7fc9e0009040 comp rx=0 tx=0).stop 2024-04-25T10:11:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f2f61640 1 -- 172.21.15.16:0/4110333723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ec15ae20 msgr2=0x7fc9ec0cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f2f61640 1 --2- 172.21.15.16:0/4110333723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ec15ae20 0x7fc9ec0cb380 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009eb0 tx=0x7fc9e4004a40 comp rx=0 tx=0).stop 2024-04-25T10:11:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f2f61640 1 -- 172.21.15.16:0/4110333723 shutdown_connections 2024-04-25T10:11:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f2f61640 1 --2- 172.21.15.16:0/4110333723 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc9d006b4d0 0x7fc9d006d990 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f2f61640 1 --2- 172.21.15.16:0/4110333723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9ec15ae20 0x7fc9ec0cb380 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f2f61640 1 --2- 172.21.15.16:0/4110333723 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9ec0b4900 0x7fc9ec14eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f2f61640 1 --2- 172.21.15.16:0/4110333723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9ec0aedc0 0x7fc9ec0c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.801+0000 7fc9f2f61640 1 -- 172.21.15.16:0/4110333723 >> 172.21.15.16:0/4110333723 conn(0x7fc9ec06aba0 msgr2=0x7fc9ec15a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.802+0000 7fc9f2f61640 1 -- 172.21.15.16:0/4110333723 shutdown_connections 2024-04-25T10:11:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.802+0000 7fc9f2f61640 1 -- 172.21.15.16:0/4110333723 wait complete. 2024-04-25T10:11:21.817 INFO:tasks.ceph.mds.d.smithi186.stderr:2024-04-25T10:11:21.802+0000 7f98024a4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv18 ap=3 DIRTYPARENT f(v0 m2024-04-25T10:11:10.936139+0000 1=1+0)->f(v1 m2024-04-25T10:11:10.936139+0000) n(v0 rc2024-04-25T10:11:11.052137+0000 2=1+1)->n(v0 rc2024-04-25T10:11:11.052137+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55cf190d0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:11:10.936139+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:11:11.052137+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-25T10:11:21.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:21.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:11:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.917+0000 7f898fcbd640 1 Processor -- start 2024-04-25T10:11:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.917+0000 7f898fcbd640 1 -- start start 2024-04-25T10:11:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.918+0000 7f898fcbd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8988161120 0x7f89881617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.918+0000 7f898fcbd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89880b5c60 0x7f89880d2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.918+0000 7f898fcbd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8988161500 con 0x7f8988161ce0 2024-04-25T10:11:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.918+0000 7f898fcbd640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8988162740 con 0x7f8988161120 2024-04-25T10:11:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.918+0000 7f898fcbd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8988162180 con 0x7f89880b5c60 2024-04-25T10:11:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.919+0000 7f898da59640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8988161120 0x7f89881617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.919+0000 7f898e25a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89880b5c60 0x7f89880d2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.919+0000 7f898e25a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89880b5c60 0x7f89880d2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:55094/0 (socket says 172.21.15.16:55094) 2024-04-25T10:11:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.919+0000 7f898d258640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f8988161ce0 0x7f89880b5550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:35660/0 (socket says 172.21.15.16:35660) 2024-04-25T10:11:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.919+0000 7f898da59640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8988161120 0x7f89881617a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:58434/0 (socket says 172.21.15.16:58434) 2024-04-25T10:11:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.919+0000 7f898e25a640 1 -- 172.21.15.16:0/3648230935 learned_addr learned my addr 172.21.15.16:0/3648230935 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.919+0000 7f898da59640 1 -- 172.21.15.16:0/3648230935 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89880b5c60 msgr2=0x7f89880d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.919+0000 7f898da59640 1 --2- 172.21.15.16:0/3648230935 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89880b5c60 0x7f89880d2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.919+0000 7f898da59640 1 -- 172.21.15.16:0/3648230935 >> v1:172.21.15.186:6789/0 conn(0x7f8988161ce0 legacy=0x7f89880b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.919+0000 7f898da59640 1 -- 172.21.15.16:0/3648230935 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8988162460 con 0x7f8988161120 2024-04-25T10:11:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.919+0000 7f898e25a640 1 --2- 172.21.15.16:0/3648230935 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89880b5c60 0x7f89880d2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.920+0000 7f898da59640 1 --2- 172.21.15.16:0/3648230935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8988161120 0x7f89881617a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f89800097a0 tx=0x7f898002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f0178df3748b728 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.920+0000 7f898ca57640 1 -- 172.21.15.16:0/3648230935 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980037070 con 0x7f8988161120 2024-04-25T10:11:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.920+0000 7f898ca57640 1 -- 172.21.15.16:0/3648230935 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898002ae50 con 0x7f8988161120 2024-04-25T10:11:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.920+0000 7f898ca57640 1 -- 172.21.15.16:0/3648230935 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980032660 con 0x7f8988161120 2024-04-25T10:11:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.920+0000 7f898fcbd640 1 -- 172.21.15.16:0/3648230935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8988161120 msgr2=0x7f89881617a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.920+0000 7f898fcbd640 1 --2- 172.21.15.16:0/3648230935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8988161120 0x7f89881617a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f89800097a0 tx=0x7f898002a100 comp rx=0 tx=0).stop 2024-04-25T10:11:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.921+0000 7f898fcbd640 1 -- 172.21.15.16:0/3648230935 shutdown_connections 2024-04-25T10:11:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.921+0000 7f898fcbd640 1 --2- 172.21.15.16:0/3648230935 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89880b5c60 0x7f89880d2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.921+0000 7f898fcbd640 1 --2- 172.21.15.16:0/3648230935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8988161120 0x7f89881617a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.921+0000 7f898fcbd640 1 -- 172.21.15.16:0/3648230935 >> 172.21.15.16:0/3648230935 conn(0x7f89880d0550 msgr2=0x7f89880d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.921+0000 7f898fcbd640 1 -- 172.21.15.16:0/3648230935 shutdown_connections 2024-04-25T10:11:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.921+0000 7f898fcbd640 1 -- 172.21.15.16:0/3648230935 wait complete. 2024-04-25T10:11:21.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.922+0000 7f898fcbd640 1 Processor -- start 2024-04-25T10:11:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.922+0000 7f898fcbd640 1 -- start start 2024-04-25T10:11:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.922+0000 7f898fcbd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89880b5c60 0x7f89880cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.922+0000 7f898fcbd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8988161120 0x7f898815fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.922+0000 7f898fcbd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8988161ce0 0x7f8988154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.922+0000 7f898fcbd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89880b95f0 con 0x7f8988161120 2024-04-25T10:11:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.923+0000 7f898fcbd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89880d2040 con 0x7f8988161ce0 2024-04-25T10:11:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.923+0000 7f898fcbd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89880b9770 con 0x7f89880b5c60 2024-04-25T10:11:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.923+0000 7f898d258640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8988161120 0x7f898815fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.923+0000 7f898d258640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8988161120 0x7f898815fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53236/0 (socket says 172.21.15.16:53236) 2024-04-25T10:11:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.923+0000 7f898d258640 1 -- 172.21.15.16:0/2132187492 learned_addr learned my addr 172.21.15.16:0/2132187492 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.923+0000 7f898da59640 1 --2- 172.21.15.16:0/2132187492 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89880b5c60 0x7f89880cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.923+0000 7f898e25a640 1 --2- 172.21.15.16:0/2132187492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8988161ce0 0x7f8988154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.923+0000 7f898d258640 1 -- 172.21.15.16:0/2132187492 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89880b5c60 msgr2=0x7f89880cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.923+0000 7f898d258640 1 --2- 172.21.15.16:0/2132187492 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89880b5c60 0x7f89880cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.923+0000 7f898d258640 1 -- 172.21.15.16:0/2132187492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8988161ce0 msgr2=0x7f8988154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.923+0000 7f898d258640 1 --2- 172.21.15.16:0/2132187492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8988161ce0 0x7f8988154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.923+0000 7f898d258640 1 -- 172.21.15.16:0/2132187492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89880cbfd0 con 0x7f8988161120 2024-04-25T10:11:21.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.923+0000 7f898d258640 1 --2- 172.21.15.16:0/2132187492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8988161120 0x7f898815fec0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f897800c3f0 tx=0x7f897800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.923+0000 7f897e7fc640 1 -- 172.21.15.16:0/2132187492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89780074e0 con 0x7f8988161120 2024-04-25T10:11:21.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.923+0000 7f897e7fc640 1 -- 172.21.15.16:0/2132187492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8978007680 con 0x7f8988161120 2024-04-25T10:11:21.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.923+0000 7f898fcbd640 1 -- 172.21.15.16:0/2132187492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89880cc2b0 con 0x7f8988161120 2024-04-25T10:11:21.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.923+0000 7f897e7fc640 1 -- 172.21.15.16:0/2132187492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978004a00 con 0x7f8988161120 2024-04-25T10:11:21.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.923+0000 7f898fcbd640 1 -- 172.21.15.16:0/2132187492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89880cc870 con 0x7f8988161120 2024-04-25T10:11:21.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.925+0000 7f897e7fc640 1 -- 172.21.15.16:0/2132187492 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f897801a020 con 0x7f8988161120 2024-04-25T10:11:21.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.925+0000 7f897e7fc640 1 --2- 172.21.15.16:0/2132187492 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f896c06b4d0 0x7f896c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.925+0000 7f897e7fc640 1 -- 172.21.15.16:0/2132187492 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f89780897e0 con 0x7f8988161120 2024-04-25T10:11:21.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.926+0000 7f898da59640 1 --2- 172.21.15.16:0/2132187492 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f896c06b4d0 0x7f896c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.926+0000 7f898da59640 1 --2- 172.21.15.16:0/2132187492 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f896c06b4d0 0x7f896c06d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8980034000 tx=0x7f898002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.926+0000 7f89929c0740 1 Processor -- start 2024-04-25T10:11:21.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.926+0000 7f89929c0740 1 -- start start 2024-04-25T10:11:21.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.927+0000 7f89929c0740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c7b10 0x55d93b3ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.927+0000 7f89929c0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c7ef0 0x55d93b3ca860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.927+0000 7f89929c0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b39a770 0x55d93b39df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.927+0000 7f89929c0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d93b303da0 con 0x55d93b39a770 2024-04-25T10:11:21.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.927+0000 7f89929c0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d93b2acb80 con 0x55d93b3c7ef0 2024-04-25T10:11:21.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.927+0000 7f89929c0740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d93b2ffe40 con 0x55d93b3c7b10 2024-04-25T10:11:21.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.927+0000 7f898e25a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b39a770 0x55d93b39df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.927+0000 7f898e25a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b39a770 0x55d93b39df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53250/0 (socket says 172.21.15.16:53250) 2024-04-25T10:11:21.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.927+0000 7f898e25a640 1 -- 172.21.15.16:0/1068123611 learned_addr learned my addr 172.21.15.16:0/1068123611 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:21.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.927+0000 7f898d258640 1 --2- 172.21.15.16:0/1068123611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c7ef0 0x55d93b3ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.927+0000 7f898da59640 1 --2- 172.21.15.16:0/1068123611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c7b10 0x55d93b3ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.927+0000 7f898e25a640 1 -- 172.21.15.16:0/1068123611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c7b10 msgr2=0x55d93b3ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.928+0000 7f898e25a640 1 --2- 172.21.15.16:0/1068123611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c7b10 0x55d93b3ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.928+0000 7f898e25a640 1 -- 172.21.15.16:0/1068123611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c7ef0 msgr2=0x55d93b3ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.928+0000 7f898e25a640 1 --2- 172.21.15.16:0/1068123611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c7ef0 0x55d93b3ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.928+0000 7f898e25a640 1 -- 172.21.15.16:0/1068123611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d93b2b6f10 con 0x55d93b39a770 2024-04-25T10:11:21.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.928+0000 7f898e25a640 1 --2- 172.21.15.16:0/1068123611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b39a770 0x55d93b39df80 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f8984011580 tx=0x7f8984009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ab4e15d47ca000 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.928+0000 7f896b7fe640 1 -- 172.21.15.16:0/1068123611 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984019070 con 0x55d93b39a770 2024-04-25T10:11:21.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.928+0000 7f896b7fe640 1 -- 172.21.15.16:0/1068123611 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8984003b90 con 0x55d93b39a770 2024-04-25T10:11:21.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.928+0000 7f896b7fe640 1 -- 172.21.15.16:0/1068123611 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984004020 con 0x55d93b39a770 2024-04-25T10:11:21.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.928+0000 7f89929c0740 1 -- 172.21.15.16:0/1068123611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b39a770 msgr2=0x55d93b39df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.928+0000 7f89929c0740 1 --2- 172.21.15.16:0/1068123611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b39a770 0x55d93b39df80 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f8984011580 tx=0x7f8984009810 comp rx=0 tx=0).stop 2024-04-25T10:11:21.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.928+0000 7f89929c0740 1 -- 172.21.15.16:0/1068123611 shutdown_connections 2024-04-25T10:11:21.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.928+0000 7f89929c0740 1 --2- 172.21.15.16:0/1068123611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b39a770 0x55d93b39df80 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.928+0000 7f89929c0740 1 --2- 172.21.15.16:0/1068123611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c7ef0 0x55d93b3ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.928+0000 7f89929c0740 1 --2- 172.21.15.16:0/1068123611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c7b10 0x55d93b3ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.928+0000 7f89929c0740 1 -- 172.21.15.16:0/1068123611 >> 172.21.15.16:0/1068123611 conn(0x55d93b39c9e0 msgr2=0x55d93b3e5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.929+0000 7f89929c0740 1 -- 172.21.15.16:0/1068123611 shutdown_connections 2024-04-25T10:11:21.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.929+0000 7f89929c0740 1 -- 172.21.15.16:0/1068123611 wait complete. 2024-04-25T10:11:21.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.929+0000 7f89929c0740 1 -- start start 2024-04-25T10:11:21.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.929+0000 7f89929c0740 1 Processor -- start 2024-04-25T10:11:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.930+0000 7f89929c0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c7b10 0x55d93b3bd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.930+0000 7f89929c0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c7ef0 0x55d93b3e84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.930+0000 7f89929c0740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c2c10 0x55d93b3c6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.930+0000 7f89929c0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d93b39fb90 con 0x55d93b3c7b10 2024-04-25T10:11:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.930+0000 7f89929c0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d93b2a24b0 con 0x55d93b3c7ef0 2024-04-25T10:11:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.930+0000 7f89929c0740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d93b39fe70 con 0x55d93b3c2c10 2024-04-25T10:11:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.930+0000 7f89929c0740 10 client.0 _ll_register_callbacks cb 0x55d93b2e2180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.930+0000 7f898da59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c7b10 0x55d93b3bd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.930+0000 7f898da59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c7b10 0x55d93b3bd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53262/0 (socket says 172.21.15.16:53262) 2024-04-25T10:11:21.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.930+0000 7f898da59640 1 -- 172.21.15.16:0/1503550812 learned_addr learned my addr 172.21.15.16:0/1503550812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:21.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.930+0000 7f898d258640 1 --2- 172.21.15.16:0/1503550812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c7ef0 0x55d93b3e84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.930+0000 7f898e25a640 1 --2- 172.21.15.16:0/1503550812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c2c10 0x55d93b3c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.930+0000 7f898da59640 1 -- 172.21.15.16:0/1503550812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c2c10 msgr2=0x55d93b3c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.930+0000 7f898da59640 1 --2- 172.21.15.16:0/1503550812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c2c10 0x55d93b3c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.930+0000 7f898da59640 1 -- 172.21.15.16:0/1503550812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c7ef0 msgr2=0x55d93b3e84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.930+0000 7f898da59640 1 --2- 172.21.15.16:0/1503550812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c7ef0 0x55d93b3e84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.930+0000 7f898da59640 1 -- 172.21.15.16:0/1503550812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d93b0e27d0 con 0x55d93b3c7b10 2024-04-25T10:11:21.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.931+0000 7f898da59640 1 --2- 172.21.15.16:0/1503550812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c7b10 0x55d93b3bd6c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f898002a6f0 tx=0x7f8980004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7f83a9673c7d77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.931+0000 7f8968ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:11:21.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.931+0000 7f8968ff9640 1 -- 172.21.15.16:0/1503550812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980037040 con 0x55d93b3c7b10 2024-04-25T10:11:21.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.931+0000 7f8968ff9640 1 -- 172.21.15.16:0/1503550812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f898003c650 con 0x55d93b3c7b10 2024-04-25T10:11:21.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.931+0000 7f89929c0740 1 -- 172.21.15.16:0/1503550812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8984003b90 con 0x55d93b3c7b10 2024-04-25T10:11:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.931+0000 7f8968ff9640 1 -- 172.21.15.16:0/1503550812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8980055050 con 0x55d93b3c7b10 2024-04-25T10:11:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.931+0000 7f89929c0740 10 client.7740 fetch_fsmap learned FSMap version 58 2024-04-25T10:11:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.931+0000 7f89929c0740 1 -- 172.21.15.16:0/1503550812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x55d93b0183f0 con 0x55d93b3c7b10 2024-04-25T10:11:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.931+0000 7f8968ff9640 1 -- 172.21.15.16:0/1503550812 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8984003b90 con 0x55d93b3c7b10 2024-04-25T10:11:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.931+0000 7f8968ff9640 1 -- 172.21.15.16:0/1503550812 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8980058020 con 0x55d93b3c7b10 2024-04-25T10:11:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.931+0000 7f89929c0740 10 client.7740 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T10:11:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.931+0000 7f89929c0740 10 client.7740 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T10:11:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.931+0000 7f89929c0740 20 client.7740 populate_metadata read hostname 'smithi016' 2024-04-25T10:11:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.931+0000 7f89929c0740 1 --2- 172.21.15.16:0/1503550812 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55d93b3c20c0 0x55d93b3e6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.931+0000 7f89929c0740 4 client.7740 mds_command: new command op to 7335 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.931+0000 7f89929c0740 1 -- 172.21.15.16:0/1503550812 --> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d93b3112e0 con 0x55d93b3c20c0 2024-04-25T10:11:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.932+0000 7f898d258640 1 --2- 172.21.15.16:0/1503550812 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55d93b3c20c0 0x55d93b3e6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.932+0000 7f898d258640 1 --2- 172.21.15.16:0/1503550812 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55d93b3c20c0 0x55d93b3e6450 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7352dab5bbe01926 server_cookie=a58b4858204aa86f in_seq=0 out_seq=0 2024-04-25T10:11:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.932+0000 7f8968ff9640 10 client.7740 ms_handle_connect on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.933+0000 7f8968ff9640 1 -- 172.21.15.16:0/1503550812 <== mds.0 v2:172.21.15.186:6834/3950732718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d93b3112e0 con 0x55d93b3c20c0 2024-04-25T10:11:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.933+0000 7f8968ff9640 10 client.7740 handle_command_reply: tid=0 2024-04-25T10:11:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.933+0000 7f89929c0740 1 client.7740 shutdown 2024-04-25T10:11:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.933+0000 7f89929c0740 1 -- 172.21.15.16:0/1503550812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c7b10 msgr2=0x55d93b3bd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.933+0000 7f89929c0740 1 --2- 172.21.15.16:0/1503550812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c7b10 0x55d93b3bd6c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f898002a6f0 tx=0x7f8980004730 comp rx=0 tx=0).stop 2024-04-25T10:11:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.933+0000 7f89929c0740 1 -- 172.21.15.16:0/1503550812 shutdown_connections 2024-04-25T10:11:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.933+0000 7f89929c0740 1 --2- 172.21.15.16:0/1503550812 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55d93b3c20c0 0x55d93b3e6450 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.933+0000 7f89929c0740 1 --2- 172.21.15.16:0/1503550812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c2c10 0x55d93b3c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.933+0000 7f89929c0740 1 --2- 172.21.15.16:0/1503550812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c7ef0 0x55d93b3e84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.933+0000 7f8968ff9640 0 client.7740 ms_handle_reset on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.933+0000 7f89929c0740 1 --2- 172.21.15.16:0/1503550812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c7b10 0x55d93b3bd6c0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.933+0000 7f89929c0740 1 -- 172.21.15.16:0/1503550812 >> 172.21.15.16:0/1503550812 conn(0x55d93b39a770 msgr2=0x55d93b39f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.933+0000 7f89929c0740 1 -- 172.21.15.16:0/1503550812 shutdown_connections 2024-04-25T10:11:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.933+0000 7f89929c0740 1 -- 172.21.15.16:0/1503550812 wait complete. 2024-04-25T10:11:21.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.933+0000 7f89929c0740 20 client.7740 trim_cache size 0 max 16384 2024-04-25T10:11:21.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.948+0000 7f89929c0740 1 Processor -- start 2024-04-25T10:11:21.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.948+0000 7f89929c0740 1 -- start start 2024-04-25T10:11:21.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.949+0000 7f89929c0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c2c10 0x55d93b3ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.949+0000 7f89929c0740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c7b10 0x55d93b39afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.949+0000 7f89929c0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c7ef0 0x55d93b39b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.949+0000 7f89929c0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8988200050 con 0x55d93b3c2c10 2024-04-25T10:11:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.949+0000 7f89929c0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89881ff320 con 0x55d93b3c7ef0 2024-04-25T10:11:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.949+0000 7f89929c0740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89881430d0 con 0x55d93b3c7b10 2024-04-25T10:11:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.949+0000 7f898da59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c2c10 0x55d93b3ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.949+0000 7f898da59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c2c10 0x55d93b3ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53276/0 (socket says 172.21.15.16:53276) 2024-04-25T10:11:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.949+0000 7f898da59640 1 -- 172.21.15.16:0/3110901897 learned_addr learned my addr 172.21.15.16:0/3110901897 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.949+0000 7f898da59640 1 -- 172.21.15.16:0/3110901897 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c7b10 msgr2=0x55d93b39afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.949+0000 7f898da59640 1 --2- 172.21.15.16:0/3110901897 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c7b10 0x55d93b39afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.949+0000 7f898da59640 1 -- 172.21.15.16:0/3110901897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c7ef0 msgr2=0x55d93b39b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.949+0000 7f898da59640 1 --2- 172.21.15.16:0/3110901897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c7ef0 0x55d93b39b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.949+0000 7f898da59640 1 -- 172.21.15.16:0/3110901897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d93afd80f0 con 0x55d93b3c2c10 2024-04-25T10:11:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.950+0000 7f898da59640 1 --2- 172.21.15.16:0/3110901897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c2c10 0x55d93b3ce010 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f8980009250 tx=0x7f8980046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2aa1d9232520d40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.950+0000 7f896affd640 1 -- 172.21.15.16:0/3110901897 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89800093c0 con 0x55d93b3c2c10 2024-04-25T10:11:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.950+0000 7f896affd640 1 -- 172.21.15.16:0/3110901897 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898003c650 con 0x55d93b3c2c10 2024-04-25T10:11:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.950+0000 7f896affd640 1 -- 172.21.15.16:0/3110901897 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898003caa0 con 0x55d93b3c2c10 2024-04-25T10:11:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.950+0000 7f89929c0740 1 -- 172.21.15.16:0/3110901897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c2c10 msgr2=0x55d93b3ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.950+0000 7f89929c0740 1 --2- 172.21.15.16:0/3110901897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c2c10 0x55d93b3ce010 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f8980009250 tx=0x7f8980046000 comp rx=0 tx=0).stop 2024-04-25T10:11:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.950+0000 7f89929c0740 1 -- 172.21.15.16:0/3110901897 shutdown_connections 2024-04-25T10:11:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.950+0000 7f89929c0740 1 --2- 172.21.15.16:0/3110901897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c7ef0 0x55d93b39b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.950+0000 7f89929c0740 1 --2- 172.21.15.16:0/3110901897 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c7b10 0x55d93b39afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.950+0000 7f89929c0740 1 --2- 172.21.15.16:0/3110901897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c2c10 0x55d93b3ce010 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.950+0000 7f89929c0740 1 -- 172.21.15.16:0/3110901897 >> 172.21.15.16:0/3110901897 conn(0x55d93b39a770 msgr2=0x55d93b395b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.950+0000 7f89929c0740 1 -- 172.21.15.16:0/3110901897 shutdown_connections 2024-04-25T10:11:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.950+0000 7f89929c0740 1 -- 172.21.15.16:0/3110901897 wait complete. 2024-04-25T10:11:21.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.951+0000 7f89929c0740 1 -- start start 2024-04-25T10:11:21.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.951+0000 7f89929c0740 1 Processor -- start 2024-04-25T10:11:21.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f89929c0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c2c10 0x55d93b3dacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f89929c0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c7b10 0x55d93b44fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f89929c0740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c7ef0 0x55d93b3e12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f89929c0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d93b3ff1b0 con 0x55d93b3c7b10 2024-04-25T10:11:21.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f89929c0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89881436d0 con 0x55d93b3c2c10 2024-04-25T10:11:21.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f89929c0740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d93b3a0ab0 con 0x55d93b3c7ef0 2024-04-25T10:11:21.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f89929c0740 10 client.0 _ll_register_callbacks cb 0x7f8984009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:21.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f898d258640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c7b10 0x55d93b44fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f898d258640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c7b10 0x55d93b44fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53290/0 (socket says 172.21.15.16:53290) 2024-04-25T10:11:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f898d258640 1 -- 172.21.15.16:0/1859173751 learned_addr learned my addr 172.21.15.16:0/1859173751 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f898e25a640 1 --2- 172.21.15.16:0/1859173751 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c7ef0 0x55d93b3e12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f898da59640 1 --2- 172.21.15.16:0/1859173751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c2c10 0x55d93b3dacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f898d258640 1 -- 172.21.15.16:0/1859173751 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c7ef0 msgr2=0x55d93b3e12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f898d258640 1 --2- 172.21.15.16:0/1859173751 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c7ef0 0x55d93b3e12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f898d258640 1 -- 172.21.15.16:0/1859173751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c2c10 msgr2=0x55d93b3dacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f898d258640 1 --2- 172.21.15.16:0/1859173751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c2c10 0x55d93b3dacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f898d258640 1 -- 172.21.15.16:0/1859173751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d93b39b850 con 0x55d93b3c7b10 2024-04-25T10:11:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f898e25a640 1 --2- 172.21.15.16:0/1859173751 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c7ef0 0x55d93b3e12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f898da59640 1 --2- 172.21.15.16:0/1859173751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c2c10 0x55d93b3dacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f898d258640 1 --2- 172.21.15.16:0/1859173751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c7b10 0x55d93b44fcd0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f897800cdb0 tx=0x7f89780561a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8f49f5884765d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f8968ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:11:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f8968ff9640 1 -- 172.21.15.16:0/1859173751 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978085020 con 0x55d93b3c7b10 2024-04-25T10:11:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f8968ff9640 1 -- 172.21.15.16:0/1859173751 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f89780584a0 con 0x55d93b3c7b10 2024-04-25T10:11:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.952+0000 7f89929c0740 1 -- 172.21.15.16:0/1859173751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f898003c650 con 0x55d93b3c7b10 2024-04-25T10:11:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.953+0000 7f8968ff9640 1 -- 172.21.15.16:0/1859173751 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8978076e60 con 0x55d93b3c7b10 2024-04-25T10:11:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.953+0000 7f89929c0740 10 client.7746 fetch_fsmap learned FSMap version 58 2024-04-25T10:11:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.953+0000 7f89929c0740 1 -- 172.21.15.16:0/1859173751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x55d93b3db370 con 0x55d93b3c7b10 2024-04-25T10:11:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.953+0000 7f8968ff9640 1 -- 172.21.15.16:0/1859173751 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898003c650 con 0x55d93b3c7b10 2024-04-25T10:11:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.953+0000 7f8968ff9640 1 -- 172.21.15.16:0/1859173751 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8978060210 con 0x55d93b3c7b10 2024-04-25T10:11:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.953+0000 7f89929c0740 10 client.7746 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T10:11:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.953+0000 7f89929c0740 10 client.7746 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T10:11:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.953+0000 7f89929c0740 20 client.7746 populate_metadata read hostname 'smithi016' 2024-04-25T10:11:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.953+0000 7f89929c0740 1 --2- 172.21.15.16:0/1859173751 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55d93b3ce080 0x55d93b3e17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.953+0000 7f89929c0740 4 client.7746 mds_command: new command op to 7335 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:11:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.953+0000 7f89929c0740 1 -- 172.21.15.16:0/1859173751 --> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d93b30b630 con 0x55d93b3ce080 2024-04-25T10:11:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.953+0000 7f898da59640 1 --2- 172.21.15.16:0/1859173751 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55d93b3ce080 0x55d93b3e17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.954+0000 7f898da59640 1 --2- 172.21.15.16:0/1859173751 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55d93b3ce080 0x55d93b3e17e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cbd2c81bc76d9b7 server_cookie=493dcd30cbde733e in_seq=0 out_seq=0 2024-04-25T10:11:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.954+0000 7f8968ff9640 10 client.7746 ms_handle_connect on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.954+0000 7f8968ff9640 1 -- 172.21.15.16:0/1859173751 <== mds.0 v2:172.21.15.186:6834/3950732718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x55d93b30b630 con 0x55d93b3ce080 2024-04-25T10:11:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.954+0000 7f8968ff9640 10 client.7746 handle_command_reply: tid=0 2024-04-25T10:11:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.954+0000 7f89929c0740 1 client.7746 shutdown 2024-04-25T10:11:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.954+0000 7f89929c0740 1 -- 172.21.15.16:0/1859173751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c7b10 msgr2=0x55d93b44fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.954+0000 7f89929c0740 1 --2- 172.21.15.16:0/1859173751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c7b10 0x55d93b44fcd0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f897800cdb0 tx=0x7f89780561a0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.954+0000 7f89929c0740 1 -- 172.21.15.16:0/1859173751 shutdown_connections 2024-04-25T10:11:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.954+0000 7f89929c0740 1 --2- 172.21.15.16:0/1859173751 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55d93b3ce080 0x55d93b3e17e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.954+0000 7f89929c0740 1 --2- 172.21.15.16:0/1859173751 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55d93b3c7ef0 0x55d93b3e12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.954+0000 7f8968ff9640 0 client.7746 ms_handle_reset on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.954+0000 7f89929c0740 1 --2- 172.21.15.16:0/1859173751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d93b3c7b10 0x55d93b44fcd0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.954+0000 7f89929c0740 1 --2- 172.21.15.16:0/1859173751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d93b3c2c10 0x55d93b3dacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.954+0000 7f89929c0740 1 -- 172.21.15.16:0/1859173751 >> 172.21.15.16:0/1859173751 conn(0x55d93b39a770 msgr2=0x55d93b3c34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.954+0000 7f89929c0740 1 -- 172.21.15.16:0/1859173751 shutdown_connections 2024-04-25T10:11:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.954+0000 7f89929c0740 1 -- 172.21.15.16:0/1859173751 wait complete. 2024-04-25T10:11:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.954+0000 7f89929c0740 20 client.7746 trim_cache size 0 max 16384 2024-04-25T10:11:21.972 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:11:21.972 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:11:21.972 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-04-25T10:11:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: "e9beceb6-7061-4119-ba99-0fcb66a4a0a4": { 2024-04-25T10:11:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/dir", 2024-04-25T10:11:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "e9beceb6-7061-4119-ba99-0fcb66a4a0a4", 2024-04-25T10:11:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,repair" 2024-04-25T10:11:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:11:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:11:21.973 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:11:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.955+0000 7f898fcbd640 1 -- 172.21.15.16:0/2132187492 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f896c06b4d0 msgr2=0x7f896c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.955+0000 7f898fcbd640 1 --2- 172.21.15.16:0/2132187492 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f896c06b4d0 0x7f896c06d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8980034000 tx=0x7f898002e000 comp rx=0 tx=0).stop 2024-04-25T10:11:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.955+0000 7f898fcbd640 1 -- 172.21.15.16:0/2132187492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8988161120 msgr2=0x7f898815fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.955+0000 7f898fcbd640 1 --2- 172.21.15.16:0/2132187492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8988161120 0x7f898815fec0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f897800c3f0 tx=0x7f897800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.955+0000 7f898fcbd640 1 -- 172.21.15.16:0/2132187492 shutdown_connections 2024-04-25T10:11:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.955+0000 7f898fcbd640 1 --2- 172.21.15.16:0/2132187492 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f896c06b4d0 0x7f896c06d990 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.955+0000 7f898fcbd640 1 --2- 172.21.15.16:0/2132187492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8988161ce0 0x7f8988154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.955+0000 7f898fcbd640 1 --2- 172.21.15.16:0/2132187492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8988161120 0x7f898815fec0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.955+0000 7f898fcbd640 1 --2- 172.21.15.16:0/2132187492 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89880b5c60 0x7f89880cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.955+0000 7f898fcbd640 1 -- 172.21.15.16:0/2132187492 >> 172.21.15.16:0/2132187492 conn(0x7f89880d0550 msgr2=0x7f89880cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.955+0000 7f898fcbd640 1 -- 172.21.15.16:0/2132187492 shutdown_connections 2024-04-25T10:11:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:21.955+0000 7f898fcbd640 1 -- 172.21.15.16:0/2132187492 wait complete. 2024-04-25T10:11:21.975 INFO:tasks.cephfs.filesystem:scrub status for tag:e9beceb6-7061-4119-ba99-0fcb66a4a0a4 - {'path': '/dir', 'tag': 'e9beceb6-7061-4119-ba99-0fcb66a4a0a4', 'options': 'recursive,repair'} 2024-04-25T10:11:22.644 INFO:tasks.ceph.mds.d.smithi186.stderr:2024-04-25T10:11:22.639+0000 7f98064ac640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:11:38.208 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:38.212 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:51.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:51.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:11:52.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.087+0000 7f23d17d5640 1 Processor -- start 2024-04-25T10:11:52.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.087+0000 7f23d17d5640 1 -- start start 2024-04-25T10:11:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.088+0000 7f23d17d5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f23cc0d3b90 0x7f23cc0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.088+0000 7f23d17d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d0f60 0x7f23cc0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.088+0000 7f23d17d5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23cc0d4090 con 0x7f23cc15dca0 2024-04-25T10:11:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.088+0000 7f23d17d5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23cc0b6b20 con 0x7f23cc0d3b90 2024-04-25T10:11:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.088+0000 7f23d17d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0d1390 con 0x7f23cc0d0f60 2024-04-25T10:11:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.089+0000 7f23cb7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d0f60 0x7f23cc0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.089+0000 7f23ca7fc640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f23cc0d3b90 0x7f23cc0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.089+0000 7f23cb7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d0f60 0x7f23cc0b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:54684/0 (socket says 172.21.15.16:54684) 2024-04-25T10:11:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.089+0000 7f23ca7fc640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f23cc0d3b90 0x7f23cc0d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:39690/0 (socket says 172.21.15.16:39690) 2024-04-25T10:11:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.089+0000 7f23cb7fe640 1 -- 172.21.15.16:0/736929495 learned_addr learned my addr 172.21.15.16:0/736929495 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.089+0000 7f23ca7fc640 1 -- 172.21.15.16:0/736929495 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d0f60 msgr2=0x7f23cc0b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.089+0000 7f23ca7fc640 1 --2- 172.21.15.16:0/736929495 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d0f60 0x7f23cc0b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.089+0000 7f23ca7fc640 1 -- 172.21.15.16:0/736929495 >> v1:172.21.15.186:6789/0 conn(0x7f23cc15dca0 legacy=0x7f23cc0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.089+0000 7f23c9ffb640 1 -- 172.21.15.16:0/736929495 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2188847843 0 0) 0x7f23cc0d4090 con 0x7f23cc15dca0 2024-04-25T10:11:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.089+0000 7f23ca7fc640 1 -- 172.21.15.16:0/736929495 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc0d4370 con 0x7f23cc0d3b90 2024-04-25T10:11:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.089+0000 7f23cb7fe640 1 --2- 172.21.15.16:0/736929495 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d0f60 0x7f23cc0b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.090+0000 7f23ca7fc640 1 --2- 172.21.15.16:0/736929495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc0d3b90 0x7f23cc0d0a20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f23c00097d0 tx=0x7f23c002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a155e945993a06c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.090+0000 7f23c9ffb640 1 -- 172.21.15.16:0/736929495 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0037070 con 0x7f23cc0d3b90 2024-04-25T10:11:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.090+0000 7f23c9ffb640 1 -- 172.21.15.16:0/736929495 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c0032440 con 0x7f23cc0d3b90 2024-04-25T10:11:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.090+0000 7f23c9ffb640 1 -- 172.21.15.16:0/736929495 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0032810 con 0x7f23cc0d3b90 2024-04-25T10:11:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.091+0000 7f23d17d5640 1 -- 172.21.15.16:0/736929495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc0d3b90 msgr2=0x7f23cc0d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.091+0000 7f23d17d5640 1 --2- 172.21.15.16:0/736929495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc0d3b90 0x7f23cc0d0a20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f23c00097d0 tx=0x7f23c002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.091+0000 7f23d17d5640 1 -- 172.21.15.16:0/736929495 shutdown_connections 2024-04-25T10:11:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.091+0000 7f23d17d5640 1 --2- 172.21.15.16:0/736929495 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d0f60 0x7f23cc0b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.091+0000 7f23d17d5640 1 --2- 172.21.15.16:0/736929495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc0d3b90 0x7f23cc0d0a20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.091+0000 7f23d17d5640 1 -- 172.21.15.16:0/736929495 >> 172.21.15.16:0/736929495 conn(0x7f23cc0726c0 msgr2=0x7f23cc072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.091+0000 7f23d17d5640 1 -- 172.21.15.16:0/736929495 shutdown_connections 2024-04-25T10:11:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.091+0000 7f23d17d5640 1 -- 172.21.15.16:0/736929495 wait complete. 2024-04-25T10:11:52.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.092+0000 7f23d17d5640 1 Processor -- start 2024-04-25T10:11:52.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.092+0000 7f23d17d5640 1 -- start start 2024-04-25T10:11:52.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.093+0000 7f23d17d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d0f60 0x7f23cc0ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.093+0000 7f23d17d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23cc0d3b90 0x7f23cc15ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.093+0000 7f23d17d5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc15dca0 0x7f23cc160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.093+0000 7f23d17d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0cf050 con 0x7f23cc0d3b90 2024-04-25T10:11:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.093+0000 7f23d17d5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0b6d50 con 0x7f23cc15dca0 2024-04-25T10:11:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.093+0000 7f23d17d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc15e080 con 0x7f23cc0d0f60 2024-04-25T10:11:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.093+0000 7f23ca7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23cc0d3b90 0x7f23cc15ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.093+0000 7f23ca7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23cc0d3b90 0x7f23cc15ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40014/0 (socket says 172.21.15.16:40014) 2024-04-25T10:11:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.093+0000 7f23ca7fc640 1 -- 172.21.15.16:0/3611270659 learned_addr learned my addr 172.21.15.16:0/3611270659 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.093+0000 7f23cb7fe640 1 --2- 172.21.15.16:0/3611270659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc15dca0 0x7f23cc160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.093+0000 7f23caffd640 1 --2- 172.21.15.16:0/3611270659 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d0f60 0x7f23cc0ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.093+0000 7f23ca7fc640 1 -- 172.21.15.16:0/3611270659 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d0f60 msgr2=0x7f23cc0ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.093+0000 7f23ca7fc640 1 --2- 172.21.15.16:0/3611270659 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d0f60 0x7f23cc0ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.093+0000 7f23ca7fc640 1 -- 172.21.15.16:0/3611270659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc15dca0 msgr2=0x7f23cc160b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.093+0000 7f23ca7fc640 1 --2- 172.21.15.16:0/3611270659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc15dca0 0x7f23cc160b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.093+0000 7f23ca7fc640 1 -- 172.21.15.16:0/3611270659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc15f320 con 0x7f23cc0d3b90 2024-04-25T10:11:52.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.093+0000 7f23caffd640 1 --2- 172.21.15.16:0/3611270659 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d0f60 0x7f23cc0ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:52.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.094+0000 7f23ca7fc640 1 --2- 172.21.15.16:0/3611270659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23cc0d3b90 0x7f23cc15ede0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f23c00097d0 tx=0x7f23c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.094+0000 7f23b37fe640 1 -- 172.21.15.16:0/3611270659 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0037050 con 0x7f23cc0d3b90 2024-04-25T10:11:52.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.094+0000 7f23d17d5640 1 -- 172.21.15.16:0/3611270659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23cc15f7d0 con 0x7f23cc0d3b90 2024-04-25T10:11:52.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.094+0000 7f23b37fe640 1 -- 172.21.15.16:0/3611270659 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c0038040 con 0x7f23cc0d3b90 2024-04-25T10:11:52.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.094+0000 7f23b37fe640 1 -- 172.21.15.16:0/3611270659 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0004490 con 0x7f23cc0d3b90 2024-04-25T10:11:52.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.094+0000 7f23d17d5640 1 -- 172.21.15.16:0/3611270659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23cc15fd90 con 0x7f23cc0d3b90 2024-04-25T10:11:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.095+0000 7f23b37fe640 1 -- 172.21.15.16:0/3611270659 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f23c003c6d0 con 0x7f23cc0d3b90 2024-04-25T10:11:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.096+0000 7f23b37fe640 1 --2- 172.21.15.16:0/3611270659 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f23b806b500 0x7f23b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.096+0000 7f23b37fe640 1 -- 172.21.15.16:0/3611270659 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f23c00aabf0 con 0x7f23cc0d3b90 2024-04-25T10:11:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.096+0000 7f23caffd640 1 --2- 172.21.15.16:0/3611270659 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f23b806b500 0x7f23b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.097+0000 7f23caffd640 1 --2- 172.21.15.16:0/3611270659 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f23b806b500 0x7f23b806d9c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f23c4000f60 tx=0x7f23c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.097+0000 7f23d45b9740 1 Processor -- start 2024-04-25T10:11:52.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.097+0000 7f23d45b9740 1 -- start start 2024-04-25T10:11:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.098+0000 7f23d45b9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bfa7b10 0x562f6bfaa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.098+0000 7f23d45b9740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7ef0 0x562f6bfaa860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.098+0000 7f23d45b9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bf7a770 0x562f6bf7df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.098+0000 7f23d45b9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6bee3da0 con 0x562f6bf7a770 2024-04-25T10:11:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.098+0000 7f23d45b9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6be8cb80 con 0x562f6bfa7b10 2024-04-25T10:11:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.098+0000 7f23d45b9740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f6bedfe40 con 0x562f6bfa7ef0 2024-04-25T10:11:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.098+0000 7f23cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bf7a770 0x562f6bf7df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.098+0000 7f23cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bf7a770 0x562f6bf7df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40022/0 (socket says 172.21.15.16:40022) 2024-04-25T10:11:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.098+0000 7f23cb7fe640 1 -- 172.21.15.16:0/2304078836 learned_addr learned my addr 172.21.15.16:0/2304078836 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.098+0000 7f23caffd640 1 --2- 172.21.15.16:0/2304078836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bfa7b10 0x562f6bfaa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.098+0000 7f23ca7fc640 1 --2- 172.21.15.16:0/2304078836 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7ef0 0x562f6bfaa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.098+0000 7f23cb7fe640 1 -- 172.21.15.16:0/2304078836 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7ef0 msgr2=0x562f6bfaa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.098+0000 7f23cb7fe640 1 --2- 172.21.15.16:0/2304078836 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7ef0 0x562f6bfaa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.098+0000 7f23cb7fe640 1 -- 172.21.15.16:0/2304078836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bfa7b10 msgr2=0x562f6bfaa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.098+0000 7f23cb7fe640 1 --2- 172.21.15.16:0/2304078836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bfa7b10 0x562f6bfaa320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.098+0000 7f23cb7fe640 1 -- 172.21.15.16:0/2304078836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f6be96f10 con 0x562f6bf7a770 2024-04-25T10:11:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.099+0000 7f23cb7fe640 1 --2- 172.21.15.16:0/2304078836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bf7a770 0x562f6bf7df80 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009370 tx=0x7f23bc009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b035947bda7b21c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.099+0000 7f23b0ff9640 1 -- 172.21.15.16:0/2304078836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc0147f0 con 0x562f6bf7a770 2024-04-25T10:11:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.099+0000 7f23b0ff9640 1 -- 172.21.15.16:0/2304078836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc01a070 con 0x562f6bf7a770 2024-04-25T10:11:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.099+0000 7f23b0ff9640 1 -- 172.21.15.16:0/2304078836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc014bb0 con 0x562f6bf7a770 2024-04-25T10:11:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.099+0000 7f23d45b9740 1 -- 172.21.15.16:0/2304078836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bf7a770 msgr2=0x562f6bf7df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.099+0000 7f23d45b9740 1 --2- 172.21.15.16:0/2304078836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bf7a770 0x562f6bf7df80 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009370 tx=0x7f23bc009810 comp rx=0 tx=0).stop 2024-04-25T10:11:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.099+0000 7f23d45b9740 1 -- 172.21.15.16:0/2304078836 shutdown_connections 2024-04-25T10:11:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.099+0000 7f23d45b9740 1 --2- 172.21.15.16:0/2304078836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bf7a770 0x562f6bf7df80 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.099+0000 7f23d45b9740 1 --2- 172.21.15.16:0/2304078836 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7ef0 0x562f6bfaa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.099+0000 7f23d45b9740 1 --2- 172.21.15.16:0/2304078836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bfa7b10 0x562f6bfaa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.099+0000 7f23d45b9740 1 -- 172.21.15.16:0/2304078836 >> 172.21.15.16:0/2304078836 conn(0x562f6bf7c9e0 msgr2=0x562f6bfc5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.100+0000 7f23d45b9740 1 -- 172.21.15.16:0/2304078836 shutdown_connections 2024-04-25T10:11:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.100+0000 7f23d45b9740 1 -- 172.21.15.16:0/2304078836 wait complete. 2024-04-25T10:11:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.100+0000 7f23d45b9740 1 -- start start 2024-04-25T10:11:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.100+0000 7f23d45b9740 1 Processor -- start 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.101+0000 7f23d45b9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bfa7b10 0x562f6bf9e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.101+0000 7f23d45b9740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7ef0 0x562f6bfc8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.101+0000 7f23d45b9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bf9d1f0 0x562f6bf9f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.101+0000 7f23d45b9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6bf7fb90 con 0x562f6bfa7b10 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.101+0000 7f23d45b9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6be824b0 con 0x562f6bf9d1f0 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.101+0000 7f23d45b9740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f6bf7fe70 con 0x562f6bfa7ef0 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.101+0000 7f23d45b9740 10 client.0 _ll_register_callbacks cb 0x562f6bec2180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.101+0000 7f23caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bfa7b10 0x562f6bf9e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.101+0000 7f23caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bfa7b10 0x562f6bf9e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40028/0 (socket says 172.21.15.16:40028) 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.101+0000 7f23caffd640 1 -- 172.21.15.16:0/2086755308 learned_addr learned my addr 172.21.15.16:0/2086755308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.101+0000 7f23cb7fe640 1 --2- 172.21.15.16:0/2086755308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bf9d1f0 0x562f6bf9f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.101+0000 7f23ca7fc640 1 --2- 172.21.15.16:0/2086755308 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7ef0 0x562f6bfc8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.101+0000 7f23caffd640 1 -- 172.21.15.16:0/2086755308 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7ef0 msgr2=0x562f6bfc8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.101+0000 7f23caffd640 1 --2- 172.21.15.16:0/2086755308 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7ef0 0x562f6bfc8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.101+0000 7f23caffd640 1 -- 172.21.15.16:0/2086755308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bf9d1f0 msgr2=0x562f6bf9f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.101+0000 7f23caffd640 1 --2- 172.21.15.16:0/2086755308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bf9d1f0 0x562f6bf9f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.101+0000 7f23caffd640 1 -- 172.21.15.16:0/2086755308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f6bcc27d0 con 0x562f6bfa7b10 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.102+0000 7f23caffd640 1 --2- 172.21.15.16:0/2086755308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bfa7b10 0x562f6bf9e810 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f23c400ddb0 tx=0x7f23c4001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828e4be30df96a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.102+0000 7f23a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.102+0000 7f23a9ffb640 1 -- 172.21.15.16:0/2086755308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4026060 con 0x562f6bfa7b10 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.102+0000 7f23a9ffb640 1 -- 172.21.15.16:0/2086755308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f23c4021b00 con 0x562f6bfa7b10 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.102+0000 7f23d45b9740 1 -- 172.21.15.16:0/2086755308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23bc01a070 con 0x562f6bfa7b10 2024-04-25T10:11:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.102+0000 7f23a9ffb640 1 -- 172.21.15.16:0/2086755308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23c4021d00 con 0x562f6bfa7b10 2024-04-25T10:11:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.102+0000 7f23d45b9740 10 client.7755 fetch_fsmap learned FSMap version 58 2024-04-25T10:11:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.102+0000 7f23d45b9740 1 -- 172.21.15.16:0/2086755308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x562f6bbf83f0 con 0x562f6bfa7b10 2024-04-25T10:11:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.102+0000 7f23a9ffb640 1 -- 172.21.15.16:0/2086755308 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc01a070 con 0x562f6bfa7b10 2024-04-25T10:11:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.102+0000 7f23a9ffb640 1 -- 172.21.15.16:0/2086755308 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f23c4021020 con 0x562f6bfa7b10 2024-04-25T10:11:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.102+0000 7f23d45b9740 10 client.7755 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T10:11:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.102+0000 7f23d45b9740 10 client.7755 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T10:11:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.102+0000 7f23d45b9740 20 client.7755 populate_metadata read hostname 'smithi016' 2024-04-25T10:11:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.102+0000 7f23d45b9740 1 --2- 172.21.15.16:0/2086755308 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x562f6bfc44c0 0x562f6bfc48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.102+0000 7f23d45b9740 4 client.7755 mds_command: new command op to 7335 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.102+0000 7f23d45b9740 1 -- 172.21.15.16:0/2086755308 --> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f6bef12e0 con 0x562f6bfc44c0 2024-04-25T10:11:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.103+0000 7f23ca7fc640 1 --2- 172.21.15.16:0/2086755308 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x562f6bfc44c0 0x562f6bfc48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.103+0000 7f23ca7fc640 1 --2- 172.21.15.16:0/2086755308 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x562f6bfc44c0 0x562f6bfc48d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4e798622defef43 server_cookie=f1b1d0efdb54ff6e in_seq=0 out_seq=0 2024-04-25T10:11:52.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.103+0000 7f23a9ffb640 10 client.7755 ms_handle_connect on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:52.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.104+0000 7f23a9ffb640 1 -- 172.21.15.16:0/2086755308 <== mds.0 v2:172.21.15.186:6834/3950732718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562f6bef12e0 con 0x562f6bfc44c0 2024-04-25T10:11:52.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.104+0000 7f23a9ffb640 10 client.7755 handle_command_reply: tid=0 2024-04-25T10:11:52.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.104+0000 7f23d45b9740 1 client.7755 shutdown 2024-04-25T10:11:52.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.104+0000 7f23d45b9740 1 -- 172.21.15.16:0/2086755308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bfa7b10 msgr2=0x562f6bf9e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.104+0000 7f23d45b9740 1 --2- 172.21.15.16:0/2086755308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bfa7b10 0x562f6bf9e810 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f23c400ddb0 tx=0x7f23c4001080 comp rx=0 tx=0).stop 2024-04-25T10:11:52.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.104+0000 7f23d45b9740 1 -- 172.21.15.16:0/2086755308 shutdown_connections 2024-04-25T10:11:52.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.104+0000 7f23d45b9740 1 --2- 172.21.15.16:0/2086755308 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x562f6bfc44c0 0x562f6bfc48d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.104+0000 7f23d45b9740 1 --2- 172.21.15.16:0/2086755308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bf9d1f0 0x562f6bf9f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.104+0000 7f23d45b9740 1 --2- 172.21.15.16:0/2086755308 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7ef0 0x562f6bfc8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.104+0000 7f23d45b9740 1 --2- 172.21.15.16:0/2086755308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bfa7b10 0x562f6bf9e810 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.104+0000 7f23a9ffb640 0 client.7755 ms_handle_reset on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:52.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.104+0000 7f23d45b9740 1 -- 172.21.15.16:0/2086755308 >> 172.21.15.16:0/2086755308 conn(0x562f6bf7a770 msgr2=0x562f6bf7b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.104+0000 7f23d45b9740 1 -- 172.21.15.16:0/2086755308 shutdown_connections 2024-04-25T10:11:52.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.104+0000 7f23d45b9740 1 -- 172.21.15.16:0/2086755308 wait complete. 2024-04-25T10:11:52.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.104+0000 7f23d45b9740 20 client.7755 trim_cache size 0 max 16384 2024-04-25T10:11:52.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.119+0000 7f23d45b9740 1 Processor -- start 2024-04-25T10:11:52.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.119+0000 7f23d45b9740 1 -- start start 2024-04-25T10:11:52.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.120+0000 7f23d45b9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bf9d1f0 0x562f6bf79f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.120+0000 7f23d45b9740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7b10 0x562f6bf9e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.120+0000 7f23d45b9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bfa7ef0 0x562f6bfc5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.120+0000 7f23d45b9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6bfdf1e0 con 0x562f6bf9d1f0 2024-04-25T10:11:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.120+0000 7f23d45b9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc1ffe80 con 0x562f6bfa7ef0 2024-04-25T10:11:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.120+0000 7f23d45b9740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0cc080 con 0x562f6bfa7b10 2024-04-25T10:11:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.120+0000 7f23caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bf9d1f0 0x562f6bf79f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.120+0000 7f23caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bf9d1f0 0x562f6bf79f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40032/0 (socket says 172.21.15.16:40032) 2024-04-25T10:11:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.120+0000 7f23caffd640 1 -- 172.21.15.16:0/3241532012 learned_addr learned my addr 172.21.15.16:0/3241532012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.120+0000 7f23caffd640 1 -- 172.21.15.16:0/3241532012 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7b10 msgr2=0x562f6bf9e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.120+0000 7f23caffd640 1 --2- 172.21.15.16:0/3241532012 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7b10 0x562f6bf9e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.120+0000 7f23caffd640 1 -- 172.21.15.16:0/3241532012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bfa7ef0 msgr2=0x562f6bfc5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.120+0000 7f23caffd640 1 --2- 172.21.15.16:0/3241532012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bfa7ef0 0x562f6bfc5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.120+0000 7f23caffd640 1 -- 172.21.15.16:0/3241532012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f6bbb80f0 con 0x562f6bf9d1f0 2024-04-25T10:11:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.121+0000 7f23caffd640 1 --2- 172.21.15.16:0/3241532012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bf9d1f0 0x562f6bf79f40 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f23c4001e30 tx=0x7f23c4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cab3806672aa97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.121+0000 7f23abfff640 1 -- 172.21.15.16:0/3241532012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4021520 con 0x562f6bf9d1f0 2024-04-25T10:11:52.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.121+0000 7f23abfff640 1 -- 172.21.15.16:0/3241532012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c40216c0 con 0x562f6bf9d1f0 2024-04-25T10:11:52.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.121+0000 7f23abfff640 1 -- 172.21.15.16:0/3241532012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4021a70 con 0x562f6bf9d1f0 2024-04-25T10:11:52.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.121+0000 7f23d45b9740 1 -- 172.21.15.16:0/3241532012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bf9d1f0 msgr2=0x562f6bf79f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.121+0000 7f23d45b9740 1 --2- 172.21.15.16:0/3241532012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bf9d1f0 0x562f6bf79f40 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f23c4001e30 tx=0x7f23c4019000 comp rx=0 tx=0).stop 2024-04-25T10:11:52.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.121+0000 7f23d45b9740 1 -- 172.21.15.16:0/3241532012 shutdown_connections 2024-04-25T10:11:52.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.121+0000 7f23d45b9740 1 --2- 172.21.15.16:0/3241532012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bfa7ef0 0x562f6bfc5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.121+0000 7f23d45b9740 1 --2- 172.21.15.16:0/3241532012 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7b10 0x562f6bf9e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.121+0000 7f23d45b9740 1 --2- 172.21.15.16:0/3241532012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bf9d1f0 0x562f6bf79f40 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.121+0000 7f23d45b9740 1 -- 172.21.15.16:0/3241532012 >> 172.21.15.16:0/3241532012 conn(0x562f6bf7a770 msgr2=0x562f6bfe7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.121+0000 7f23d45b9740 1 -- 172.21.15.16:0/3241532012 shutdown_connections 2024-04-25T10:11:52.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.121+0000 7f23d45b9740 1 -- 172.21.15.16:0/3241532012 wait complete. 2024-04-25T10:11:52.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.122+0000 7f23d45b9740 1 -- start start 2024-04-25T10:11:52.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.122+0000 7f23d45b9740 1 Processor -- start 2024-04-25T10:11:52.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23d45b9740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bf9d1f0 0x562f6c02ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23d45b9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bfa7b10 0x562f6bface40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23d45b9740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7ef0 0x562f6bfad380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23d45b9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0cba80 con 0x562f6bfa7b10 2024-04-25T10:11:52.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23d45b9740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0cb480 con 0x562f6bf9d1f0 2024-04-25T10:11:52.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23d45b9740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f6bf7f1c0 con 0x562f6bfa7ef0 2024-04-25T10:11:52.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23d45b9740 10 client.0 _ll_register_callbacks cb 0x7f23c4001d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:52.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23ca7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bfa7b10 0x562f6bface40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23ca7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bfa7b10 0x562f6bface40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40036/0 (socket says 172.21.15.16:40036) 2024-04-25T10:11:52.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23ca7fc640 1 -- 172.21.15.16:0/2192829431 learned_addr learned my addr 172.21.15.16:0/2192829431 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:52.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23ca7fc640 1 -- 172.21.15.16:0/2192829431 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7ef0 msgr2=0x562f6bfad380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23caffd640 1 --2- 172.21.15.16:0/2192829431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bf9d1f0 0x562f6c02ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23cb7fe640 1 --2- 172.21.15.16:0/2192829431 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7ef0 0x562f6bfad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23ca7fc640 1 --2- 172.21.15.16:0/2192829431 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7ef0 0x562f6bfad380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23ca7fc640 1 -- 172.21.15.16:0/2192829431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bf9d1f0 msgr2=0x562f6c02ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23ca7fc640 1 --2- 172.21.15.16:0/2192829431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bf9d1f0 0x562f6c02ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23ca7fc640 1 -- 172.21.15.16:0/2192829431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f6bfd4600 con 0x562f6bfa7b10 2024-04-25T10:11:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23caffd640 1 --2- 172.21.15.16:0/2192829431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bf9d1f0 0x562f6c02ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23cb7fe640 1 --2- 172.21.15.16:0/2192829431 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7ef0 0x562f6bfad380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23ca7fc640 1 --2- 172.21.15.16:0/2192829431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bfa7b10 0x562f6bface40 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f23c0075780 tx=0x7f23c0009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3431f470a6c1ac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:11:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23a9ffb640 1 -- 172.21.15.16:0/2192829431 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0075990 con 0x562f6bfa7b10 2024-04-25T10:11:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23a9ffb640 1 -- 172.21.15.16:0/2192829431 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f23c007d0c0 con 0x562f6bfa7b10 2024-04-25T10:11:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23d45b9740 1 -- 172.21.15.16:0/2192829431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23c40216c0 con 0x562f6bfa7b10 2024-04-25T10:11:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.123+0000 7f23a9ffb640 1 -- 172.21.15.16:0/2192829431 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23c0075b30 con 0x562f6bfa7b10 2024-04-25T10:11:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.124+0000 7f23d45b9740 10 client.7761 fetch_fsmap learned FSMap version 58 2024-04-25T10:11:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.124+0000 7f23d45b9740 1 -- 172.21.15.16:0/2192829431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x562f6bfc7930 con 0x562f6bfa7b10 2024-04-25T10:11:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.124+0000 7f23a9ffb640 1 -- 172.21.15.16:0/2192829431 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c40216c0 con 0x562f6bfa7b10 2024-04-25T10:11:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.124+0000 7f23a9ffb640 1 -- 172.21.15.16:0/2192829431 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f23c00815a0 con 0x562f6bfa7b10 2024-04-25T10:11:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.124+0000 7f23d45b9740 10 client.7761 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T10:11:52.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.124+0000 7f23d45b9740 10 client.7761 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T10:11:52.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.124+0000 7f23d45b9740 20 client.7761 populate_metadata read hostname 'smithi016' 2024-04-25T10:11:52.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.124+0000 7f23d45b9740 1 --2- 172.21.15.16:0/2192829431 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x562f6bf9b660 0x562f6bfc5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.124+0000 7f23d45b9740 4 client.7761 mds_command: new command op to 7335 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:11:52.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.124+0000 7f23d45b9740 1 -- 172.21.15.16:0/2192829431 --> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562f6beeb630 con 0x562f6bf9b660 2024-04-25T10:11:52.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.124+0000 7f23caffd640 1 --2- 172.21.15.16:0/2192829431 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x562f6bf9b660 0x562f6bfc5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.124+0000 7f23caffd640 1 --2- 172.21.15.16:0/2192829431 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x562f6bf9b660 0x562f6bfc5320 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d760fc30e3196694 server_cookie=e6d72c1ad5c80e2 in_seq=0 out_seq=0 2024-04-25T10:11:52.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.124+0000 7f23a9ffb640 10 client.7761 ms_handle_connect on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:52.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23a9ffb640 1 -- 172.21.15.16:0/2192829431 <== mds.0 v2:172.21.15.186:6834/3950732718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562f6beeb630 con 0x562f6bf9b660 2024-04-25T10:11:52.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23a9ffb640 10 client.7761 handle_command_reply: tid=0 2024-04-25T10:11:52.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d45b9740 1 client.7761 shutdown 2024-04-25T10:11:52.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d45b9740 1 -- 172.21.15.16:0/2192829431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bfa7b10 msgr2=0x562f6bface40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d45b9740 1 --2- 172.21.15.16:0/2192829431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bfa7b10 0x562f6bface40 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f23c0075780 tx=0x7f23c0009190 comp rx=0 tx=0).stop 2024-04-25T10:11:52.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d45b9740 1 -- 172.21.15.16:0/2192829431 shutdown_connections 2024-04-25T10:11:52.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d45b9740 1 --2- 172.21.15.16:0/2192829431 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x562f6bf9b660 0x562f6bfc5320 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d45b9740 1 --2- 172.21.15.16:0/2192829431 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x562f6bfa7ef0 0x562f6bfad380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d45b9740 1 --2- 172.21.15.16:0/2192829431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6bfa7b10 0x562f6bface40 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23a9ffb640 0 client.7761 ms_handle_reset on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:52.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d45b9740 1 --2- 172.21.15.16:0/2192829431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f6bf9d1f0 0x562f6c02ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d45b9740 1 -- 172.21.15.16:0/2192829431 >> 172.21.15.16:0/2192829431 conn(0x562f6bf7a770 msgr2=0x562f6bfa2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d45b9740 1 -- 172.21.15.16:0/2192829431 shutdown_connections 2024-04-25T10:11:52.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d45b9740 1 -- 172.21.15.16:0/2192829431 wait complete. 2024-04-25T10:11:52.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d45b9740 20 client.7761 trim_cache size 0 max 16384 2024-04-25T10:11:52.136 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:11:52.137 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-04-25T10:11:52.137 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-04-25T10:11:52.137 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:11:52.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d17d5640 1 -- 172.21.15.16:0/3611270659 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f23b806b500 msgr2=0x7f23b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d17d5640 1 --2- 172.21.15.16:0/3611270659 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f23b806b500 0x7f23b806d9c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f23c4000f60 tx=0x7f23c4009040 comp rx=0 tx=0).stop 2024-04-25T10:11:52.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d17d5640 1 -- 172.21.15.16:0/3611270659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23cc0d3b90 msgr2=0x7f23cc15ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d17d5640 1 --2- 172.21.15.16:0/3611270659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23cc0d3b90 0x7f23cc15ede0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f23c00097d0 tx=0x7f23c00046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d17d5640 1 -- 172.21.15.16:0/3611270659 shutdown_connections 2024-04-25T10:11:52.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d17d5640 1 --2- 172.21.15.16:0/3611270659 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f23b806b500 0x7f23b806d9c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d17d5640 1 --2- 172.21.15.16:0/3611270659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc15dca0 0x7f23cc160b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d17d5640 1 --2- 172.21.15.16:0/3611270659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23cc0d3b90 0x7f23cc15ede0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d17d5640 1 --2- 172.21.15.16:0/3611270659 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d0f60 0x7f23cc0ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.125+0000 7f23d17d5640 1 -- 172.21.15.16:0/3611270659 >> 172.21.15.16:0/3611270659 conn(0x7f23cc0726c0 msgr2=0x7f23cc0b7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.126+0000 7f23d17d5640 1 -- 172.21.15.16:0/3611270659 shutdown_connections 2024-04-25T10:11:52.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.126+0000 7f23d17d5640 1 -- 172.21.15.16:0/3611270659 wait complete. 2024-04-25T10:11:52.141 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:11:52.141 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:52.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.237+0000 7f3de26bd640 1 Processor -- start 2024-04-25T10:11:52.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.237+0000 7f3de26bd640 1 -- start start 2024-04-25T10:11:52.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.238+0000 7f3de26bd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3ddc15dce0 0x7f3ddc0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.238+0000 7f3de26bd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ddc0d35b0 0x7f3ddc0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.238+0000 7f3de26bd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ddc15e0c0 con 0x7f3ddc0d1f90 2024-04-25T10:11:52.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.238+0000 7f3de26bd640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ddc0b7b10 con 0x7f3ddc15dce0 2024-04-25T10:11:52.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.238+0000 7f3de26bd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc0d2510 con 0x7f3ddc0d35b0 2024-04-25T10:11:52.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.238+0000 7f3ddb7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ddc0d35b0 0x7f3ddc0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.238+0000 7f3ddbfff640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3ddc15dce0 0x7f3ddc0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.238+0000 7f3de0c5a640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f3ddc0d1f90 0x7f3ddc06bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:47076/0 (socket says 172.21.15.16:47076) 2024-04-25T10:11:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.238+0000 7f3de0c5a640 1 -- 172.21.15.16:0/3073976720 learned_addr learned my addr 172.21.15.16:0/3073976720 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.238+0000 7f3ddbfff640 1 --2- 172.21.15.16:0/3073976720 >> v2:172.21.15.186:3300/0 conn(0x7f3ddc15dce0 0x7f3ddc0d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:39748/0 (socket says 172.21.15.16:39748) 2024-04-25T10:11:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.238+0000 7f3ddb7fe640 1 --2- 172.21.15.16:0/3073976720 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ddc0d35b0 0x7f3ddc0b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:54728/0 (socket says 172.21.15.16:54728) 2024-04-25T10:11:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.238+0000 7f3ddb7fe640 1 -- 172.21.15.16:0/3073976720 >> v2:172.21.15.186:3300/0 conn(0x7f3ddc15dce0 msgr2=0x7f3ddc0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.238+0000 7f3ddb7fe640 1 --2- 172.21.15.16:0/3073976720 >> v2:172.21.15.186:3300/0 conn(0x7f3ddc15dce0 0x7f3ddc0d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.238+0000 7f3ddb7fe640 1 -- 172.21.15.16:0/3073976720 >> v1:172.21.15.186:6789/0 conn(0x7f3ddc0d1f90 legacy=0x7f3ddc06bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.238+0000 7f3ddb7fe640 1 -- 172.21.15.16:0/3073976720 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc0d27f0 con 0x7f3ddc0d35b0 2024-04-25T10:11:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.238+0000 7f3ddbfff640 1 --2- 172.21.15.16:0/3073976720 >> v2:172.21.15.186:3300/0 conn(0x7f3ddc15dce0 0x7f3ddc0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.239+0000 7f3ddb7fe640 1 --2- 172.21.15.16:0/3073976720 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ddc0d35b0 0x7f3ddc0b5450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0097a0 tx=0x7f3dcc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b3a86cc31e6aebb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.239+0000 7f3ddaffd640 1 -- 172.21.15.16:0/3073976720 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc037070 con 0x7f3ddc0d35b0 2024-04-25T10:11:52.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.239+0000 7f3ddaffd640 1 -- 172.21.15.16:0/3073976720 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dcc02ae50 con 0x7f3ddc0d35b0 2024-04-25T10:11:52.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.239+0000 7f3ddaffd640 1 -- 172.21.15.16:0/3073976720 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc032660 con 0x7f3ddc0d35b0 2024-04-25T10:11:52.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.240+0000 7f3de26bd640 1 -- 172.21.15.16:0/3073976720 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ddc0d35b0 msgr2=0x7f3ddc0b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.240+0000 7f3de26bd640 1 --2- 172.21.15.16:0/3073976720 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ddc0d35b0 0x7f3ddc0b5450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0097a0 tx=0x7f3dcc02a100 comp rx=0 tx=0).stop 2024-04-25T10:11:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.240+0000 7f3de26bd640 1 -- 172.21.15.16:0/3073976720 shutdown_connections 2024-04-25T10:11:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.240+0000 7f3de26bd640 1 --2- 172.21.15.16:0/3073976720 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ddc0d35b0 0x7f3ddc0b5450 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.240+0000 7f3de26bd640 1 --2- 172.21.15.16:0/3073976720 >> v2:172.21.15.186:3300/0 conn(0x7f3ddc15dce0 0x7f3ddc0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.240+0000 7f3de26bd640 1 -- 172.21.15.16:0/3073976720 >> 172.21.15.16:0/3073976720 conn(0x7f3ddc072900 msgr2=0x7f3ddc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.240+0000 7f3de26bd640 1 -- 172.21.15.16:0/3073976720 shutdown_connections 2024-04-25T10:11:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.240+0000 7f3de26bd640 1 -- 172.21.15.16:0/3073976720 wait complete. 2024-04-25T10:11:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.241+0000 7f3de26bd640 1 Processor -- start 2024-04-25T10:11:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.241+0000 7f3de26bd640 1 -- start start 2024-04-25T10:11:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.242+0000 7f3de26bd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ddc0d1f90 0x7f3ddc153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.242+0000 7f3de26bd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ddc0d35b0 0x7f3ddc1f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.242+0000 7f3de26bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ddc15dce0 0x7f3ddc155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.242+0000 7f3de26bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc0d3090 con 0x7f3ddc15dce0 2024-04-25T10:11:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.242+0000 7f3de26bd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc0d2db0 con 0x7f3ddc0d1f90 2024-04-25T10:11:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.242+0000 7f3de26bd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc0b9820 con 0x7f3ddc0d35b0 2024-04-25T10:11:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.242+0000 7f3de0c5a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ddc15dce0 0x7f3ddc155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.242+0000 7f3de0c5a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ddc15dce0 0x7f3ddc155eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40042/0 (socket says 172.21.15.16:40042) 2024-04-25T10:11:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.242+0000 7f3de0c5a640 1 -- 172.21.15.16:0/4108587521 learned_addr learned my addr 172.21.15.16:0/4108587521 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.242+0000 7f3ddb7fe640 1 --2- 172.21.15.16:0/4108587521 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ddc0d35b0 0x7f3ddc1f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.242+0000 7f3ddbfff640 1 --2- 172.21.15.16:0/4108587521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ddc0d1f90 0x7f3ddc153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.242+0000 7f3de0c5a640 1 -- 172.21.15.16:0/4108587521 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ddc0d35b0 msgr2=0x7f3ddc1f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.242+0000 7f3de0c5a640 1 --2- 172.21.15.16:0/4108587521 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ddc0d35b0 0x7f3ddc1f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.242+0000 7f3de0c5a640 1 -- 172.21.15.16:0/4108587521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ddc0d1f90 msgr2=0x7f3ddc153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.242+0000 7f3de0c5a640 1 --2- 172.21.15.16:0/4108587521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ddc0d1f90 0x7f3ddc153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.242+0000 7f3de0c5a640 1 -- 172.21.15.16:0/4108587521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc153410 con 0x7f3ddc15dce0 2024-04-25T10:11:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.243+0000 7f3de0c5a640 1 --2- 172.21.15.16:0/4108587521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ddc15dce0 0x7f3ddc155eb0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400ca80 tx=0x7f3dd400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.243+0000 7f3dd8ff9640 1 -- 172.21.15.16:0/4108587521 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4017070 con 0x7f3ddc15dce0 2024-04-25T10:11:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.243+0000 7f3dd8ff9640 1 -- 172.21.15.16:0/4108587521 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd400ac30 con 0x7f3ddc15dce0 2024-04-25T10:11:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.243+0000 7f3dd8ff9640 1 -- 172.21.15.16:0/4108587521 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4012600 con 0x7f3ddc15dce0 2024-04-25T10:11:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.243+0000 7f3de26bd640 1 -- 172.21.15.16:0/4108587521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ddc15f260 con 0x7f3ddc15dce0 2024-04-25T10:11:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.243+0000 7f3de26bd640 1 -- 172.21.15.16:0/4108587521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ddc15f820 con 0x7f3ddc15dce0 2024-04-25T10:11:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.244+0000 7f3de26bd640 1 -- 172.21.15.16:0/4108587521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ddc0d0b50 con 0x7f3ddc15dce0 2024-04-25T10:11:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.244+0000 7f3dd8ff9640 1 -- 172.21.15.16:0/4108587521 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3dd401a070 con 0x7f3ddc15dce0 2024-04-25T10:11:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.245+0000 7f3dd8ff9640 1 --2- 172.21.15.16:0/4108587521 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3dc806b500 0x7f3dc806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.245+0000 7f3dd8ff9640 1 -- 172.21.15.16:0/4108587521 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3dd4089ea0 con 0x7f3ddc15dce0 2024-04-25T10:11:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.245+0000 7f3ddbfff640 1 --2- 172.21.15.16:0/4108587521 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3dc806b500 0x7f3dc806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.245+0000 7f3ddbfff640 1 --2- 172.21.15.16:0/4108587521 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3dc806b500 0x7f3dc806d9c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009560 tx=0x7f3dd0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.249+0000 7f3dd8ff9640 1 -- 172.21.15.16:0/4108587521 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ddc0d0b50 con 0x7f3ddc15dce0 2024-04-25T10:11:52.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.466+0000 7f3de26bd640 1 -- 172.21.15.16:0/4108587521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ddc0749b0 con 0x7f3ddc15dce0 2024-04-25T10:11:52.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.467+0000 7f3dd8ff9640 1 -- 172.21.15.16:0/4108587521 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f3dd400ae20 con 0x7f3ddc15dce0 2024-04-25T10:11:52.474 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:52.474 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4044509901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4044509901},{"type":"v1","addr":"172.21.15.16:6835","nonce":4044509901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1494230493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1494230493},{"type":"v1","addr":"172.21.15.186:6837","nonce":1494230493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/84349474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":84349474},{"type":"v1","addr":"172.21.15.16:6834","nonce":84349474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:00.015075+0000","modified":"2024-04-25T10:11:20.851789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"d","rank":0,"incarnation":54,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6836/3950732718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3950732718},{"type":"v1","addr":"172.21.15.186:6836","nonce":3950732718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 58 2024-04-25T10:11:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.469+0000 7f3de26bd640 1 -- 172.21.15.16:0/4108587521 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3dc806b500 msgr2=0x7f3dc806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.469+0000 7f3de26bd640 1 --2- 172.21.15.16:0/4108587521 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3dc806b500 0x7f3dc806d9c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009560 tx=0x7f3dd0009190 comp rx=0 tx=0).stop 2024-04-25T10:11:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.469+0000 7f3de26bd640 1 -- 172.21.15.16:0/4108587521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ddc15dce0 msgr2=0x7f3ddc155eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.469+0000 7f3de26bd640 1 --2- 172.21.15.16:0/4108587521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ddc15dce0 0x7f3ddc155eb0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400ca80 tx=0x7f3dd400cf40 comp rx=0 tx=0).stop 2024-04-25T10:11:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.469+0000 7f3de26bd640 1 -- 172.21.15.16:0/4108587521 shutdown_connections 2024-04-25T10:11:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.469+0000 7f3de26bd640 1 --2- 172.21.15.16:0/4108587521 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3dc806b500 0x7f3dc806d9c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.469+0000 7f3de26bd640 1 --2- 172.21.15.16:0/4108587521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ddc15dce0 0x7f3ddc155eb0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.469+0000 7f3de26bd640 1 --2- 172.21.15.16:0/4108587521 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ddc0d35b0 0x7f3ddc1f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.469+0000 7f3de26bd640 1 --2- 172.21.15.16:0/4108587521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ddc0d1f90 0x7f3ddc153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.469+0000 7f3de26bd640 1 -- 172.21.15.16:0/4108587521 >> 172.21.15.16:0/4108587521 conn(0x7f3ddc072900 msgr2=0x7f3ddc0cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.469+0000 7f3de26bd640 1 -- 172.21.15.16:0/4108587521 shutdown_connections 2024-04-25T10:11:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.469+0000 7f3de26bd640 1 -- 172.21.15.16:0/4108587521 wait complete. 2024-04-25T10:11:52.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-04-25T10:11:52.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.582+0000 7f6a9c888640 1 Processor -- start 2024-04-25T10:11:52.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.582+0000 7f6a9c888640 1 -- start start 2024-04-25T10:11:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.583+0000 7f6a9c888640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6a980d3b90 0x7f6a980d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.583+0000 7f6a9c888640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a980d0f60 0x7f6a980b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.583+0000 7f6a9c888640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a980d4090 con 0x7f6a9815dca0 2024-04-25T10:11:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.583+0000 7f6a9c888640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a980b6b20 con 0x7f6a980d3b90 2024-04-25T10:11:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.583+0000 7f6a9c888640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a980d1390 con 0x7f6a980d0f60 2024-04-25T10:11:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.583+0000 7f6a8ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6a980d3b90 0x7f6a980d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.583+0000 7f6a9659c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6a9815dca0 0x7f6a980ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36984/0 (socket says 172.21.15.16:36984) 2024-04-25T10:11:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.583+0000 7f6a8ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6a980d3b90 0x7f6a980d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40050/0 (socket says 172.21.15.16:40050) 2024-04-25T10:11:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.583+0000 7f6a9659c640 1 -- 172.21.15.16:0/4182629213 learned_addr learned my addr 172.21.15.16:0/4182629213 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.583+0000 7f6a96d9d640 1 --2- 172.21.15.16:0/4182629213 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a980d0f60 0x7f6a980b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.583+0000 7f6a8ffff640 1 -- 172.21.15.16:0/4182629213 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a980d0f60 msgr2=0x7f6a980b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.583+0000 7f6a8ffff640 1 --2- 172.21.15.16:0/4182629213 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a980d0f60 0x7f6a980b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.583+0000 7f6a8ffff640 1 -- 172.21.15.16:0/4182629213 >> v1:172.21.15.16:6789/0 conn(0x7f6a9815dca0 legacy=0x7f6a980ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.583+0000 7f6a8ffff640 1 -- 172.21.15.16:0/4182629213 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a980d4370 con 0x7f6a980d3b90 2024-04-25T10:11:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.584+0000 7f6a8ffff640 1 --2- 172.21.15.16:0/4182629213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a980d3b90 0x7f6a980d0a20 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f6a800097d0 tx=0x7f6a8002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6a422c3697695b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.584+0000 7f6a95d9b640 1 -- 172.21.15.16:0/4182629213 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a80037070 con 0x7f6a980d3b90 2024-04-25T10:11:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.584+0000 7f6a95d9b640 1 -- 172.21.15.16:0/4182629213 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a80032440 con 0x7f6a980d3b90 2024-04-25T10:11:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.584+0000 7f6a95d9b640 1 -- 172.21.15.16:0/4182629213 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a80032810 con 0x7f6a980d3b90 2024-04-25T10:11:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.585+0000 7f6a9c888640 1 -- 172.21.15.16:0/4182629213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a980d3b90 msgr2=0x7f6a980d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.585+0000 7f6a9c888640 1 --2- 172.21.15.16:0/4182629213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a980d3b90 0x7f6a980d0a20 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f6a800097d0 tx=0x7f6a8002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.585+0000 7f6a9c888640 1 -- 172.21.15.16:0/4182629213 shutdown_connections 2024-04-25T10:11:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.585+0000 7f6a9c888640 1 --2- 172.21.15.16:0/4182629213 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a980d0f60 0x7f6a980b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.585+0000 7f6a9c888640 1 --2- 172.21.15.16:0/4182629213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a980d3b90 0x7f6a980d0a20 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.585+0000 7f6a9c888640 1 -- 172.21.15.16:0/4182629213 >> 172.21.15.16:0/4182629213 conn(0x7f6a980726c0 msgr2=0x7f6a98072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.585+0000 7f6a9c888640 1 -- 172.21.15.16:0/4182629213 shutdown_connections 2024-04-25T10:11:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.585+0000 7f6a9c888640 1 -- 172.21.15.16:0/4182629213 wait complete. 2024-04-25T10:11:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.586+0000 7f6a9c888640 1 Processor -- start 2024-04-25T10:11:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.586+0000 7f6a9c888640 1 -- start start 2024-04-25T10:11:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.587+0000 7f6a9c888640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a980d0f60 0x7f6a980b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.587+0000 7f6a9c888640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a980d3b90 0x7f6a981f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.587+0000 7f6a9c888640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a9815dca0 0x7f6a981f4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.587+0000 7f6a9c888640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a980cf050 con 0x7f6a980d0f60 2024-04-25T10:11:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.587+0000 7f6a9c888640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a980b6d50 con 0x7f6a980d3b90 2024-04-25T10:11:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.587+0000 7f6a9c888640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9815e080 con 0x7f6a9815dca0 2024-04-25T10:11:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.587+0000 7f6a9659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a980d0f60 0x7f6a980b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.587+0000 7f6a9659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a980d0f60 0x7f6a980b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40060/0 (socket says 172.21.15.16:40060) 2024-04-25T10:11:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.587+0000 7f6a9659c640 1 -- 172.21.15.16:0/2338184429 learned_addr learned my addr 172.21.15.16:0/2338184429 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.587+0000 7f6a96d9d640 1 --2- 172.21.15.16:0/2338184429 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a9815dca0 0x7f6a981f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.587+0000 7f6a8ffff640 1 --2- 172.21.15.16:0/2338184429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a980d3b90 0x7f6a981f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.587+0000 7f6a9659c640 1 -- 172.21.15.16:0/2338184429 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a9815dca0 msgr2=0x7f6a981f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.587+0000 7f6a9659c640 1 --2- 172.21.15.16:0/2338184429 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a9815dca0 0x7f6a981f4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.587+0000 7f6a9659c640 1 -- 172.21.15.16:0/2338184429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a980d3b90 msgr2=0x7f6a981f3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.587+0000 7f6a9659c640 1 --2- 172.21.15.16:0/2338184429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a980d3b90 0x7f6a981f3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.587+0000 7f6a9659c640 1 -- 172.21.15.16:0/2338184429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a980c9670 con 0x7f6a980d0f60 2024-04-25T10:11:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.587+0000 7f6a9659c640 1 --2- 172.21.15.16:0/2338184429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a980d0f60 0x7f6a980b3760 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800c3c0 tx=0x7f6a8800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.588+0000 7f6a8effd640 1 -- 172.21.15.16:0/2338184429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a88017070 con 0x7f6a980d0f60 2024-04-25T10:11:52.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.588+0000 7f6a8effd640 1 -- 172.21.15.16:0/2338184429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a88005ce0 con 0x7f6a980d0f60 2024-04-25T10:11:52.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.588+0000 7f6a9c888640 1 -- 172.21.15.16:0/2338184429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a980c8610 con 0x7f6a980d0f60 2024-04-25T10:11:52.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.588+0000 7f6a8effd640 1 -- 172.21.15.16:0/2338184429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a880072f0 con 0x7f6a980d0f60 2024-04-25T10:11:52.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.588+0000 7f6a9c888640 1 -- 172.21.15.16:0/2338184429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a980c8e70 con 0x7f6a980d0f60 2024-04-25T10:11:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.589+0000 7f6a8effd640 1 -- 172.21.15.16:0/2338184429 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a8801a070 con 0x7f6a980d0f60 2024-04-25T10:11:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.590+0000 7f6a8effd640 1 --2- 172.21.15.16:0/2338184429 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6a7806b4d0 0x7f6a7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.590+0000 7f6a8effd640 1 -- 172.21.15.16:0/2338184429 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6a88089640 con 0x7f6a980d0f60 2024-04-25T10:11:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.590+0000 7f6a8ffff640 1 --2- 172.21.15.16:0/2338184429 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6a7806b4d0 0x7f6a7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.591+0000 7f6a8ffff640 1 --2- 172.21.15.16:0/2338184429 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6a7806b4d0 0x7f6a7806d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6a80034000 tx=0x7f6a8002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.591+0000 7f6a9f588740 1 Processor -- start 2024-04-25T10:11:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.591+0000 7f6a9f588740 1 -- start start 2024-04-25T10:11:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a9f588740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78df886f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a9f588740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfc2c60 0x55a78df86970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a9f588740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfb1050 0x55a78dfb14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a9f588740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78dee9c60 con 0x55a78dfc2c60 2024-04-25T10:11:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a9f588740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78de88d80 con 0x55a78df86590 2024-04-25T10:11:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a9f588740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a78dee5d00 con 0x55a78dfb1050 2024-04-25T10:11:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfc2c60 0x55a78df86970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfc2c60 0x55a78df86970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40076/0 (socket says 172.21.15.16:40076) 2024-04-25T10:11:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a8ffff640 1 -- 172.21.15.16:0/3470681466 learned_addr learned my addr 172.21.15.16:0/3470681466 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a96d9d640 1 --2- 172.21.15.16:0/3470681466 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfb1050 0x55a78dfb14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a9659c640 1 --2- 172.21.15.16:0/3470681466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78df886f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a8ffff640 1 -- 172.21.15.16:0/3470681466 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfb1050 msgr2=0x55a78dfb14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a8ffff640 1 --2- 172.21.15.16:0/3470681466 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfb1050 0x55a78dfb14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a8ffff640 1 -- 172.21.15.16:0/3470681466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 msgr2=0x55a78df886f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a8ffff640 1 --2- 172.21.15.16:0/3470681466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78df886f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a8ffff640 1 -- 172.21.15.16:0/3470681466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a78dcca7c0 con 0x55a78dfc2c60 2024-04-25T10:11:52.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a9659c640 1 --2- 172.21.15.16:0/3470681466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78df886f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:52.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a8ffff640 1 --2- 172.21.15.16:0/3470681466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfc2c60 0x55a78df86970 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f6a800098c0 tx=0x7f6a8002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4ec535e9b106d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a77fff640 1 -- 172.21.15.16:0/3470681466 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8004e070 con 0x55a78dfc2c60 2024-04-25T10:11:52.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a77fff640 1 -- 172.21.15.16:0/3470681466 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a80032e30 con 0x55a78dfc2c60 2024-04-25T10:11:52.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.592+0000 7f6a77fff640 1 -- 172.21.15.16:0/3470681466 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a80037040 con 0x55a78dfc2c60 2024-04-25T10:11:52.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.593+0000 7f6a9f588740 1 -- 172.21.15.16:0/3470681466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfc2c60 msgr2=0x55a78df86970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.593+0000 7f6a9f588740 1 --2- 172.21.15.16:0/3470681466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfc2c60 0x55a78df86970 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f6a800098c0 tx=0x7f6a8002c000 comp rx=0 tx=0).stop 2024-04-25T10:11:52.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.593+0000 7f6a9f588740 1 -- 172.21.15.16:0/3470681466 shutdown_connections 2024-04-25T10:11:52.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.593+0000 7f6a9f588740 1 --2- 172.21.15.16:0/3470681466 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfb1050 0x55a78dfb14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.593+0000 7f6a9f588740 1 --2- 172.21.15.16:0/3470681466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfc2c60 0x55a78df86970 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.593+0000 7f6a9f588740 1 --2- 172.21.15.16:0/3470681466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78df886f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.593+0000 7f6a9f588740 1 -- 172.21.15.16:0/3470681466 >> 172.21.15.16:0/3470681466 conn(0x55a78df4e6e0 msgr2=0x55a78dfcc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.593+0000 7f6a9f588740 1 -- 172.21.15.16:0/3470681466 shutdown_connections 2024-04-25T10:11:52.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.593+0000 7f6a9f588740 1 -- 172.21.15.16:0/3470681466 wait complete. 2024-04-25T10:11:52.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.594+0000 7f6a9f588740 1 -- start start 2024-04-25T10:11:52.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.594+0000 7f6a9f588740 1 Processor -- start 2024-04-25T10:11:52.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a9f588740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78dfb2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a9f588740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfb1050 0x55a78dfdce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a9f588740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfc2c60 0x55a78dfdd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a9f588740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78de86840 con 0x55a78dfb1050 2024-04-25T10:11:52.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a9f588740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78dfaeba0 con 0x55a78df86590 2024-04-25T10:11:52.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a9f588740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a78dfb3090 con 0x55a78dfc2c60 2024-04-25T10:11:52.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a9f588740 10 client.0 _ll_register_callbacks cb 0x55a78dea0980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:52.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfb1050 0x55a78dfdce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfb1050 0x55a78dfdce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40084/0 (socket says 172.21.15.16:40084) 2024-04-25T10:11:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a8ffff640 1 -- 172.21.15.16:0/2236822660 learned_addr learned my addr 172.21.15.16:0/2236822660 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a96d9d640 1 --2- 172.21.15.16:0/2236822660 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfc2c60 0x55a78dfdd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a9659c640 1 --2- 172.21.15.16:0/2236822660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78dfb2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a8ffff640 1 -- 172.21.15.16:0/2236822660 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfc2c60 msgr2=0x55a78dfdd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a8ffff640 1 --2- 172.21.15.16:0/2236822660 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfc2c60 0x55a78dfdd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a8ffff640 1 -- 172.21.15.16:0/2236822660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 msgr2=0x55a78dfb2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a8ffff640 1 --2- 172.21.15.16:0/2236822660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78dfb2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a8ffff640 1 -- 172.21.15.16:0/2236822660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a78dd2af50 con 0x55a78dfb1050 2024-04-25T10:11:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a9659c640 1 --2- 172.21.15.16:0/2236822660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78dfb2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a8ffff640 1 --2- 172.21.15.16:0/2236822660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfb1050 0x55a78dfdce40 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f6a8004c040 tx=0x7f6a80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577d2a08d6b221f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.595+0000 7f6a757fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:11:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.596+0000 7f6a757fa640 1 -- 172.21.15.16:0/2236822660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a80059060 con 0x55a78dfb1050 2024-04-25T10:11:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.596+0000 7f6a757fa640 1 -- 172.21.15.16:0/2236822660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6a80049d10 con 0x55a78dfb1050 2024-04-25T10:11:52.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.596+0000 7f6a9f588740 1 -- 172.21.15.16:0/2236822660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a80032e30 con 0x55a78dfb1050 2024-04-25T10:11:52.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.596+0000 7f6a757fa640 1 -- 172.21.15.16:0/2236822660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a8004e070 con 0x55a78dfb1050 2024-04-25T10:11:52.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.596+0000 7f6a9f588740 10 client.7779 fetch_fsmap learned FSMap version 58 2024-04-25T10:11:52.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.596+0000 7f6a9f588740 1 -- 172.21.15.16:0/2236822660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x55a78dbbe0b0 con 0x55a78dfb1050 2024-04-25T10:11:52.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.596+0000 7f6a757fa640 1 -- 172.21.15.16:0/2236822660 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a80032e30 con 0x55a78dfb1050 2024-04-25T10:11:52.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.596+0000 7f6a757fa640 1 -- 172.21.15.16:0/2236822660 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6a8003e460 con 0x55a78dfb1050 2024-04-25T10:11:52.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.596+0000 7f6a9f588740 10 client.7779 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T10:11:52.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.596+0000 7f6a9f588740 10 client.7779 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:11:52.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.596+0000 7f6a9f588740 20 client.7779 populate_metadata read hostname 'smithi016' 2024-04-25T10:11:52.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.596+0000 7f6a9f588740 1 --2- 172.21.15.16:0/2236822660 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55a78dfdf9b0 0x55a78dfdfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.596+0000 7f6a9f588740 4 client.7779 mds_command: new command op to 7335 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:52.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.596+0000 7f6a9f588740 1 -- 172.21.15.16:0/2236822660 --> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a78dea9e60 con 0x55a78dfdf9b0 2024-04-25T10:11:52.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.596+0000 7f6a9659c640 1 --2- 172.21.15.16:0/2236822660 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55a78dfdf9b0 0x55a78dfdfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.597+0000 7f6a757fa640 10 client.7779 ms_handle_connect on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.597+0000 7f6a9659c640 1 --2- 172.21.15.16:0/2236822660 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55a78dfdf9b0 0x55a78dfdfd90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7849c0c982fa9a60 server_cookie=d77568725c663618 in_seq=0 out_seq=0 2024-04-25T10:11:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.597+0000 7f6a757fa640 1 -- 172.21.15.16:0/2236822660 <== mds.0 v2:172.21.15.186:6834/3950732718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a78dea9e60 con 0x55a78dfdf9b0 2024-04-25T10:11:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.597+0000 7f6a757fa640 10 client.7779 handle_command_reply: tid=0 2024-04-25T10:11:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.597+0000 7f6a9f588740 1 client.7779 shutdown 2024-04-25T10:11:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.598+0000 7f6a9f588740 1 -- 172.21.15.16:0/2236822660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfb1050 msgr2=0x55a78dfdce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.598+0000 7f6a9f588740 1 --2- 172.21.15.16:0/2236822660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfb1050 0x55a78dfdce40 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f6a8004c040 tx=0x7f6a80004730 comp rx=0 tx=0).stop 2024-04-25T10:11:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.598+0000 7f6a9f588740 1 -- 172.21.15.16:0/2236822660 shutdown_connections 2024-04-25T10:11:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.598+0000 7f6a9f588740 1 --2- 172.21.15.16:0/2236822660 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55a78dfdf9b0 0x55a78dfdfd90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.598+0000 7f6a9f588740 1 --2- 172.21.15.16:0/2236822660 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfc2c60 0x55a78dfdd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.598+0000 7f6a9f588740 1 --2- 172.21.15.16:0/2236822660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfb1050 0x55a78dfdce40 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.598+0000 7f6a9f588740 1 --2- 172.21.15.16:0/2236822660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78dfb2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.598+0000 7f6a9f588740 1 -- 172.21.15.16:0/2236822660 >> 172.21.15.16:0/2236822660 conn(0x55a78df4e6e0 msgr2=0x55a78dfcf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.598+0000 7f6a757fa640 0 client.7779 ms_handle_reset on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.598+0000 7f6a9f588740 1 -- 172.21.15.16:0/2236822660 shutdown_connections 2024-04-25T10:11:52.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.598+0000 7f6a9f588740 1 -- 172.21.15.16:0/2236822660 wait complete. 2024-04-25T10:11:52.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.598+0000 7f6a9f588740 20 client.7779 trim_cache size 0 max 16384 2024-04-25T10:11:52.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.610+0000 7f6a9f588740 1 Processor -- start 2024-04-25T10:11:52.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.610+0000 7f6a9f588740 1 -- start start 2024-04-25T10:11:52.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.610+0000 7f6a9f588740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78dfcf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.610+0000 7f6a9f588740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfb1050 0x55a78dfcf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.610+0000 7f6a9f588740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfc2c60 0x55a78dfac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.610+0000 7f6a9f588740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a981ff250 con 0x55a78dfc2c60 2024-04-25T10:11:52.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.610+0000 7f6a9f588740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a981f7b90 con 0x55a78df86590 2024-04-25T10:11:52.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.610+0000 7f6a9f588740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a980cb730 con 0x55a78dfb1050 2024-04-25T10:11:52.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.610+0000 7f6a96d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfc2c60 0x55a78dfac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.611+0000 7f6a96d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfc2c60 0x55a78dfac1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40092/0 (socket says 172.21.15.16:40092) 2024-04-25T10:11:52.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.611+0000 7f6a96d9d640 1 -- 172.21.15.16:0/2986704629 learned_addr learned my addr 172.21.15.16:0/2986704629 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:52.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.611+0000 7f6a8ffff640 1 --2- 172.21.15.16:0/2986704629 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfb1050 0x55a78dfcf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.611+0000 7f6a9659c640 1 --2- 172.21.15.16:0/2986704629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78dfcf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.611+0000 7f6a96d9d640 1 -- 172.21.15.16:0/2986704629 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfb1050 msgr2=0x55a78dfcf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.611+0000 7f6a96d9d640 1 --2- 172.21.15.16:0/2986704629 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfb1050 0x55a78dfcf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.611+0000 7f6a96d9d640 1 -- 172.21.15.16:0/2986704629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 msgr2=0x55a78dfcf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.611+0000 7f6a96d9d640 1 --2- 172.21.15.16:0/2986704629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78dfcf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.611+0000 7f6a96d9d640 1 -- 172.21.15.16:0/2986704629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a78df86fb0 con 0x55a78dfc2c60 2024-04-25T10:11:52.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.611+0000 7f6a9659c640 1 --2- 172.21.15.16:0/2986704629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78dfcf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:52.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.611+0000 7f6a96d9d640 1 --2- 172.21.15.16:0/2986704629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfc2c60 0x55a78dfac1c0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f6a900145e0 tx=0x7f6a90009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a890eb17e233f709 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.611+0000 7f6a777fe640 1 -- 172.21.15.16:0/2986704629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9001d070 con 0x55a78dfc2c60 2024-04-25T10:11:52.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.611+0000 7f6a777fe640 1 -- 172.21.15.16:0/2986704629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a90016870 con 0x55a78dfc2c60 2024-04-25T10:11:52.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.611+0000 7f6a777fe640 1 -- 172.21.15.16:0/2986704629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90016c20 con 0x55a78dfc2c60 2024-04-25T10:11:52.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.612+0000 7f6a9f588740 1 -- 172.21.15.16:0/2986704629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfc2c60 msgr2=0x55a78dfac1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.612+0000 7f6a9f588740 1 --2- 172.21.15.16:0/2986704629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfc2c60 0x55a78dfac1c0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f6a900145e0 tx=0x7f6a90009750 comp rx=0 tx=0).stop 2024-04-25T10:11:52.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.612+0000 7f6a9f588740 1 -- 172.21.15.16:0/2986704629 shutdown_connections 2024-04-25T10:11:52.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.612+0000 7f6a9f588740 1 --2- 172.21.15.16:0/2986704629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfc2c60 0x55a78dfac1c0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.612+0000 7f6a9f588740 1 --2- 172.21.15.16:0/2986704629 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfb1050 0x55a78dfcf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.612+0000 7f6a9f588740 1 --2- 172.21.15.16:0/2986704629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78dfcf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.612+0000 7f6a9f588740 1 -- 172.21.15.16:0/2986704629 >> 172.21.15.16:0/2986704629 conn(0x55a78df4e6e0 msgr2=0x55a78dfb2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.612+0000 7f6a9f588740 1 -- 172.21.15.16:0/2986704629 shutdown_connections 2024-04-25T10:11:52.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.612+0000 7f6a9f588740 1 -- 172.21.15.16:0/2986704629 wait complete. 2024-04-25T10:11:52.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.612+0000 7f6a9f588740 1 -- start start 2024-04-25T10:11:52.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.612+0000 7f6a9f588740 1 Processor -- start 2024-04-25T10:11:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.613+0000 7f6a9f588740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78dfb4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.613+0000 7f6a9f588740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfb1050 0x55a78dfb49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.613+0000 7f6a9f588740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfc2c60 0x55a78dfac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.613+0000 7f6a9f588740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78dfde030 con 0x55a78dfb1050 2024-04-25T10:11:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.613+0000 7f6a9f588740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78dfb2350 con 0x55a78df86590 2024-04-25T10:11:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.613+0000 7f6a9f588740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a78dfddc30 con 0x55a78dfc2c60 2024-04-25T10:11:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.613+0000 7f6a9f588740 10 client.0 _ll_register_callbacks cb 0x7f6a80035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.613+0000 7f6a8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfb1050 0x55a78dfb49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.613+0000 7f6a8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfb1050 0x55a78dfb49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40102/0 (socket says 172.21.15.16:40102) 2024-04-25T10:11:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.613+0000 7f6a8ffff640 1 -- 172.21.15.16:0/2787494301 learned_addr learned my addr 172.21.15.16:0/2787494301 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.613+0000 7f6a9659c640 1 --2- 172.21.15.16:0/2787494301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78dfb4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.613+0000 7f6a96d9d640 1 --2- 172.21.15.16:0/2787494301 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfc2c60 0x55a78dfac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a8ffff640 1 -- 172.21.15.16:0/2787494301 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfc2c60 msgr2=0x55a78dfac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a8ffff640 1 --2- 172.21.15.16:0/2787494301 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfc2c60 0x55a78dfac1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a8ffff640 1 -- 172.21.15.16:0/2787494301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 msgr2=0x55a78dfb4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a8ffff640 1 --2- 172.21.15.16:0/2787494301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78dfb4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a8ffff640 1 -- 172.21.15.16:0/2787494301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a78dfc9c20 con 0x55a78dfb1050 2024-04-25T10:11:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a8ffff640 1 --2- 172.21.15.16:0/2787494301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfb1050 0x55a78dfb49d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6a80000c00 tx=0x7f6a80008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc6da6b134291ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a757fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:11:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a757fa640 1 -- 172.21.15.16:0/2787494301 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a80037040 con 0x55a78dfb1050 2024-04-25T10:11:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a757fa640 1 -- 172.21.15.16:0/2787494301 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6a80049d20 con 0x55a78dfb1050 2024-04-25T10:11:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a9f588740 1 -- 172.21.15.16:0/2787494301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a90016870 con 0x55a78dfb1050 2024-04-25T10:11:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a757fa640 1 -- 172.21.15.16:0/2787494301 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a8006a050 con 0x55a78dfb1050 2024-04-25T10:11:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a9f588740 10 client.7785 fetch_fsmap learned FSMap version 58 2024-04-25T10:11:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a9f588740 1 -- 172.21.15.16:0/2787494301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x55a78dfc7b80 con 0x55a78dfb1050 2024-04-25T10:11:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a757fa640 1 -- 172.21.15.16:0/2787494301 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a90016870 con 0x55a78dfb1050 2024-04-25T10:11:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a757fa640 1 -- 172.21.15.16:0/2787494301 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6a8006d020 con 0x55a78dfb1050 2024-04-25T10:11:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a9f588740 10 client.7785 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T10:11:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a9f588740 10 client.7785 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:11:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a9f588740 20 client.7785 populate_metadata read hostname 'smithi016' 2024-04-25T10:11:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a9f588740 1 --2- 172.21.15.16:0/2787494301 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55a78dfacd00 0x55a78dfc6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a9f588740 4 client.7785 mds_command: new command op to 7335 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:11:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.614+0000 7f6a9f588740 1 -- 172.21.15.16:0/2787494301 --> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55a78def71a0 con 0x55a78dfacd00 2024-04-25T10:11:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.615+0000 7f6a9659c640 1 --2- 172.21.15.16:0/2787494301 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55a78dfacd00 0x55a78dfc6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.615+0000 7f6a9659c640 1 --2- 172.21.15.16:0/2787494301 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55a78dfacd00 0x55a78dfc6b00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6eaea9615f55a2d server_cookie=64ef9271c5321916 in_seq=0 out_seq=0 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.615+0000 7f6a757fa640 10 client.7785 ms_handle_connect on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.615+0000 7f6a757fa640 1 -- 172.21.15.16:0/2787494301 <== mds.0 v2:172.21.15.186:6834/3950732718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+143 (crc 0 0 0) 0x55a78def71a0 con 0x55a78dfacd00 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.615+0000 7f6a757fa640 10 client.7785 handle_command_reply: tid=0 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.615+0000 7f6a9f588740 1 client.7785 shutdown 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9f588740 1 -- 172.21.15.16:0/2787494301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfb1050 msgr2=0x55a78dfb49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9f588740 1 --2- 172.21.15.16:0/2787494301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfb1050 0x55a78dfb49d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6a80000c00 tx=0x7f6a80008030 comp rx=0 tx=0).stop 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9f588740 1 -- 172.21.15.16:0/2787494301 shutdown_connections 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9f588740 1 --2- 172.21.15.16:0/2787494301 >> [v2:172.21.15.186:6834/3950732718,v1:172.21.15.186:6836/3950732718] conn(0x55a78dfacd00 0x55a78dfc6b00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9f588740 1 --2- 172.21.15.16:0/2787494301 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a78dfc2c60 0x55a78dfac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9f588740 1 --2- 172.21.15.16:0/2787494301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a78dfb1050 0x55a78dfb49d0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a757fa640 0 client.7785 ms_handle_reset on v2:172.21.15.186:6834/3950732718 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9f588740 1 --2- 172.21.15.16:0/2787494301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a78df86590 0x55a78dfb4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9f588740 1 -- 172.21.15.16:0/2787494301 >> 172.21.15.16:0/2787494301 conn(0x55a78df4e6e0 msgr2=0x55a78dfdc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9f588740 1 -- 172.21.15.16:0/2787494301 shutdown_connections 2024-04-25T10:11:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9f588740 1 -- 172.21.15.16:0/2787494301 wait complete. 2024-04-25T10:11:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9f588740 20 client.7785 trim_cache size 0 max 16384 2024-04-25T10:11:52.627 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:52.627 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-04-25T10:11:52.627 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:11:52.627 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-04-25T10:11:52.627 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 369417969, 2024-04-25T10:11:52.627 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511627777, 2024-04-25T10:11:52.628 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/testdir/hardlink" 2024-04-25T10:11:52.628 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:11:52.628 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-04-25T10:11:52.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9c888640 1 -- 172.21.15.16:0/2338184429 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6a7806b4d0 msgr2=0x7f6a7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9c888640 1 --2- 172.21.15.16:0/2338184429 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6a7806b4d0 0x7f6a7806d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6a80034000 tx=0x7f6a8002e000 comp rx=0 tx=0).stop 2024-04-25T10:11:52.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9c888640 1 -- 172.21.15.16:0/2338184429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a980d0f60 msgr2=0x7f6a980b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9c888640 1 --2- 172.21.15.16:0/2338184429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a980d0f60 0x7f6a980b3760 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800c3c0 tx=0x7f6a8800c880 comp rx=0 tx=0).stop 2024-04-25T10:11:52.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9c888640 1 -- 172.21.15.16:0/2338184429 shutdown_connections 2024-04-25T10:11:52.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9c888640 1 --2- 172.21.15.16:0/2338184429 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6a7806b4d0 0x7f6a7806d990 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9c888640 1 --2- 172.21.15.16:0/2338184429 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a9815dca0 0x7f6a981f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9c888640 1 --2- 172.21.15.16:0/2338184429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a980d3b90 0x7f6a981f3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9c888640 1 --2- 172.21.15.16:0/2338184429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a980d0f60 0x7f6a980b3760 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.616+0000 7f6a9c888640 1 -- 172.21.15.16:0/2338184429 >> 172.21.15.16:0/2338184429 conn(0x7f6a980726c0 msgr2=0x7f6a9815e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.617+0000 7f6a9c888640 1 -- 172.21.15.16:0/2338184429 shutdown_connections 2024-04-25T10:11:52.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.617+0000 7f6a9c888640 1 -- 172.21.15.16:0/2338184429 wait complete. 2024-04-25T10:11:52.632 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:11:52.632 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:52.648 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:11:52.648 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:52.703 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:52.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.839+0000 7f1ec66e1640 1 Processor -- start 2024-04-25T10:11:52.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.839+0000 7f1ec66e1640 1 -- start start 2024-04-25T10:11:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.839+0000 7f1ec66e1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ec015dc30 0x7f1ec006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.839+0000 7f1ec66e1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ec006cc00 0x7f1ec00b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.839+0000 7f1ec66e1640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec006c1f0 con 0x7f1ec00b1960 2024-04-25T10:11:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.839+0000 7f1ec66e1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec00cfb40 con 0x7f1ec006cc00 2024-04-25T10:11:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.839+0000 7f1ec66e1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec00b2060 con 0x7f1ec015dc30 2024-04-25T10:11:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.840+0000 7f1ebf7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ec006cc00 0x7f1ec00b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.840+0000 7f1ebf7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ec006cc00 0x7f1ec00b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49268/0 (socket says 172.21.15.16:49268) 2024-04-25T10:11:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.840+0000 7f1ebf7fe640 1 -- 172.21.15.16:0/2355186844 learned_addr learned my addr 172.21.15.16:0/2355186844 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.840+0000 7f1ebffff640 1 --2- 172.21.15.16:0/2355186844 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ec015dc30 0x7f1ec006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.840+0000 7f1ebf7fe640 1 -- 172.21.15.16:0/2355186844 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ec015dc30 msgr2=0x7f1ec006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.840+0000 7f1ebf7fe640 1 --2- 172.21.15.16:0/2355186844 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ec015dc30 0x7f1ec006c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.840+0000 7f1ebf7fe640 1 -- 172.21.15.16:0/2355186844 >> v1:172.21.15.186:6789/0 conn(0x7f1ec00b1960 legacy=0x7f1ec0099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.840+0000 7f1ebf7fe640 1 -- 172.21.15.16:0/2355186844 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec00b2340 con 0x7f1ec006cc00 2024-04-25T10:11:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.840+0000 7f1ebffff640 1 --2- 172.21.15.16:0/2355186844 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ec015dc30 0x7f1ec006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.841+0000 7f1ebf7fe640 1 --2- 172.21.15.16:0/2355186844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec006cc00 0x7f1ec00b1420 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009870 tx=0x7f1eb402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c37c1096fe66a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.841+0000 7f1ebeffd640 1 -- 172.21.15.16:0/2355186844 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4037070 con 0x7f1ec006cc00 2024-04-25T10:11:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.841+0000 7f1ebeffd640 1 -- 172.21.15.16:0/2355186844 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb4032440 con 0x7f1ec006cc00 2024-04-25T10:11:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.841+0000 7f1ebeffd640 1 -- 172.21.15.16:0/2355186844 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4032810 con 0x7f1ec006cc00 2024-04-25T10:11:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.841+0000 7f1ec66e1640 1 -- 172.21.15.16:0/2355186844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec006cc00 msgr2=0x7f1ec00b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.841+0000 7f1ec66e1640 1 --2- 172.21.15.16:0/2355186844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec006cc00 0x7f1ec00b1420 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009870 tx=0x7f1eb402a200 comp rx=0 tx=0).stop 2024-04-25T10:11:52.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.841+0000 7f1ec66e1640 1 -- 172.21.15.16:0/2355186844 shutdown_connections 2024-04-25T10:11:52.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.841+0000 7f1ec66e1640 1 --2- 172.21.15.16:0/2355186844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec006cc00 0x7f1ec00b1420 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.841+0000 7f1ec66e1640 1 --2- 172.21.15.16:0/2355186844 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ec015dc30 0x7f1ec006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.841+0000 7f1ec66e1640 1 -- 172.21.15.16:0/2355186844 >> 172.21.15.16:0/2355186844 conn(0x7f1ec00d1900 msgr2=0x7f1ec00d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.841+0000 7f1ec66e1640 1 -- 172.21.15.16:0/2355186844 shutdown_connections 2024-04-25T10:11:52.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.841+0000 7f1ec66e1640 1 -- 172.21.15.16:0/2355186844 wait complete. 2024-04-25T10:11:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.842+0000 7f1ec66e1640 1 Processor -- start 2024-04-25T10:11:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.843+0000 7f1ec66e1640 1 -- start start 2024-04-25T10:11:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.843+0000 7f1ec66e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec006cc00 0x7f1ec0154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.843+0000 7f1ec66e1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ec00b1960 0x7f1ec006f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.843+0000 7f1ec66e1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ec015dc30 0x7f1ec006f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.843+0000 7f1ec66e1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec00b1d80 con 0x7f1ec006cc00 2024-04-25T10:11:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.843+0000 7f1ec66e1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec00cfde0 con 0x7f1ec00b1960 2024-04-25T10:11:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.843+0000 7f1ec66e1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec015e010 con 0x7f1ec015dc30 2024-04-25T10:11:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.843+0000 7f1ebffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec006cc00 0x7f1ec0154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.843+0000 7f1ebffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec006cc00 0x7f1ec0154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49274/0 (socket says 172.21.15.16:49274) 2024-04-25T10:11:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.843+0000 7f1ebffff640 1 -- 172.21.15.16:0/1146907101 learned_addr learned my addr 172.21.15.16:0/1146907101 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.844+0000 7f1ec4c7e640 1 --2- 172.21.15.16:0/1146907101 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ec015dc30 0x7f1ec006f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.844+0000 7f1ebf7fe640 1 --2- 172.21.15.16:0/1146907101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ec00b1960 0x7f1ec006f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.844+0000 7f1ebffff640 1 -- 172.21.15.16:0/1146907101 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ec015dc30 msgr2=0x7f1ec006f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.844+0000 7f1ebffff640 1 --2- 172.21.15.16:0/1146907101 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ec015dc30 0x7f1ec006f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.844+0000 7f1ebffff640 1 -- 172.21.15.16:0/1146907101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ec00b1960 msgr2=0x7f1ec006f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.844+0000 7f1ebffff640 1 --2- 172.21.15.16:0/1146907101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ec00b1960 0x7f1ec006f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.844+0000 7f1ebffff640 1 -- 172.21.15.16:0/1146907101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec01604f0 con 0x7f1ec006cc00 2024-04-25T10:11:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.844+0000 7f1ebffff640 1 --2- 172.21.15.16:0/1146907101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec006cc00 0x7f1ec0154f60 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c820 tx=0x7f1eb000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.844+0000 7f1ebcff9640 1 -- 172.21.15.16:0/1146907101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0015070 con 0x7f1ec006cc00 2024-04-25T10:11:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.844+0000 7f1ebcff9640 1 -- 172.21.15.16:0/1146907101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb0007a30 con 0x7f1ec006cc00 2024-04-25T10:11:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.844+0000 7f1ec66e1640 1 -- 172.21.15.16:0/1146907101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec01607d0 con 0x7f1ec006cc00 2024-04-25T10:11:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.844+0000 7f1ebcff9640 1 -- 172.21.15.16:0/1146907101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0007de0 con 0x7f1ec006cc00 2024-04-25T10:11:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.844+0000 7f1ec66e1640 1 -- 172.21.15.16:0/1146907101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec0160d90 con 0x7f1ec006cc00 2024-04-25T10:11:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.845+0000 7f1ec66e1640 1 -- 172.21.15.16:0/1146907101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec0067850 con 0x7f1ec006cc00 2024-04-25T10:11:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.846+0000 7f1ebcff9640 1 -- 172.21.15.16:0/1146907101 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1eb0018070 con 0x7f1ec006cc00 2024-04-25T10:11:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.846+0000 7f1ebcff9640 1 --2- 172.21.15.16:0/1146907101 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1eac06b4d0 0x7f1eac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.846+0000 7f1ebcff9640 1 -- 172.21.15.16:0/1146907101 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1eb008a190 con 0x7f1ec006cc00 2024-04-25T10:11:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.846+0000 7f1ebf7fe640 1 --2- 172.21.15.16:0/1146907101 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1eac06b4d0 0x7f1eac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.847+0000 7f1ebf7fe640 1 --2- 172.21.15.16:0/1146907101 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1eac06b4d0 0x7f1eac06d990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4034000 tx=0x7f1eb402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:52.850+0000 7f1ebcff9640 1 -- 172.21.15.16:0/1146907101 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1eb0058fb0 con 0x7f1ec006cc00 2024-04-25T10:11:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.034+0000 7f1ec66e1640 1 -- 172.21.15.16:0/1146907101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ec00749b0 con 0x7f1ec006cc00 2024-04-25T10:11:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.035+0000 7f1ebcff9640 1 -- 172.21.15.16:0/1146907101 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16803 (secure 0 0 0) 0x7f1eb00588b0 con 0x7f1ec006cc00 2024-04-25T10:11:53.040 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:53.040 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":87,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:11:13.827370+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:58.299405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:58.994953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.186:6837/3092291936":"2024-04-26T10:11:13.827348+0000","172.21.15.186:6835/3092291936":"2024-04-26T10:11:13.827348+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.037+0000 7f1ec66e1640 1 -- 172.21.15.16:0/1146907101 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1eac06b4d0 msgr2=0x7f1eac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.037+0000 7f1ec66e1640 1 --2- 172.21.15.16:0/1146907101 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1eac06b4d0 0x7f1eac06d990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4034000 tx=0x7f1eb402e000 comp rx=0 tx=0).stop 2024-04-25T10:11:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.037+0000 7f1ec66e1640 1 -- 172.21.15.16:0/1146907101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec006cc00 msgr2=0x7f1ec0154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.037+0000 7f1ec66e1640 1 --2- 172.21.15.16:0/1146907101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec006cc00 0x7f1ec0154f60 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c820 tx=0x7f1eb000cce0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.037+0000 7f1ec66e1640 1 -- 172.21.15.16:0/1146907101 shutdown_connections 2024-04-25T10:11:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.037+0000 7f1ec66e1640 1 --2- 172.21.15.16:0/1146907101 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1eac06b4d0 0x7f1eac06d990 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.037+0000 7f1ec66e1640 1 --2- 172.21.15.16:0/1146907101 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1ec015dc30 0x7f1ec006f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.037+0000 7f1ec66e1640 1 --2- 172.21.15.16:0/1146907101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ec00b1960 0x7f1ec006f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.037+0000 7f1ec66e1640 1 --2- 172.21.15.16:0/1146907101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec006cc00 0x7f1ec0154f60 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.037+0000 7f1ec66e1640 1 -- 172.21.15.16:0/1146907101 >> 172.21.15.16:0/1146907101 conn(0x7f1ec00d1900 msgr2=0x7f1ec00d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.037+0000 7f1ec66e1640 1 -- 172.21.15.16:0/1146907101 shutdown_connections 2024-04-25T10:11:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.037+0000 7f1ec66e1640 1 -- 172.21.15.16:0/1146907101 wait complete. 2024-04-25T10:11:53.052 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:10:58.299405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:10:58.994953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:53.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:53.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.151+0000 7f7d83aac640 1 Processor -- start 2024-04-25T10:11:53.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.151+0000 7f7d83aac640 1 -- start start 2024-04-25T10:11:53.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.152+0000 7f7d83aac640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7d7c15dbd0 0x7f7d7c06c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.152+0000 7f7d83aac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d7c066e10 0x7f7d7c0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.152+0000 7f7d83aac640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7c15efc0 con 0x7f7d7c06aba0 2024-04-25T10:11:53.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.152+0000 7f7d83aac640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d7c06b0a0 con 0x7f7d7c066e10 2024-04-25T10:11:53.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.152+0000 7f7d83aac640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d7c06b380 con 0x7f7d7c15dbd0 2024-04-25T10:11:53.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.152+0000 7f7d81047640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d7c066e10 0x7f7d7c0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.152+0000 7f7d81047640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d7c066e10 0x7f7d7c0cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49280/0 (socket says 172.21.15.16:49280) 2024-04-25T10:11:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.152+0000 7f7d81047640 1 -- 172.21.15.16:0/3701340282 learned_addr learned my addr 172.21.15.16:0/3701340282 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.152+0000 7f7d81848640 1 --2- 172.21.15.16:0/3701340282 >> v2:172.21.15.186:3300/0 conn(0x7f7d7c15dbd0 0x7f7d7c06c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.152+0000 7f7d81047640 1 -- 172.21.15.16:0/3701340282 >> v2:172.21.15.186:3300/0 conn(0x7f7d7c15dbd0 msgr2=0x7f7d7c06c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.152+0000 7f7d81047640 1 --2- 172.21.15.16:0/3701340282 >> v2:172.21.15.186:3300/0 conn(0x7f7d7c15dbd0 0x7f7d7c06c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.152+0000 7f7d81047640 1 -- 172.21.15.16:0/3701340282 >> v1:172.21.15.186:6789/0 conn(0x7f7d7c06aba0 legacy=0x7f7d7c06dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.152+0000 7f7d81047640 1 -- 172.21.15.16:0/3701340282 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c06b660 con 0x7f7d7c066e10 2024-04-25T10:11:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.153+0000 7f7d81047640 1 --2- 172.21.15.16:0/3701340282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d7c066e10 0x7f7d7c0cdb50 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c071600 tx=0x7f7d6c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c93771118a9599 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.153+0000 7f7d80846640 1 -- 172.21.15.16:0/3701340282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c037070 con 0x7f7d7c066e10 2024-04-25T10:11:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.153+0000 7f7d80846640 1 -- 172.21.15.16:0/3701340282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6c032440 con 0x7f7d7c066e10 2024-04-25T10:11:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.153+0000 7f7d80846640 1 -- 172.21.15.16:0/3701340282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c032810 con 0x7f7d7c066e10 2024-04-25T10:11:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.154+0000 7f7d83aac640 1 -- 172.21.15.16:0/3701340282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d7c066e10 msgr2=0x7f7d7c0cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.154+0000 7f7d83aac640 1 --2- 172.21.15.16:0/3701340282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d7c066e10 0x7f7d7c0cdb50 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c071600 tx=0x7f7d6c02a230 comp rx=0 tx=0).stop 2024-04-25T10:11:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.154+0000 7f7d83aac640 1 -- 172.21.15.16:0/3701340282 shutdown_connections 2024-04-25T10:11:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.154+0000 7f7d83aac640 1 --2- 172.21.15.16:0/3701340282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d7c066e10 0x7f7d7c0cdb50 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.154+0000 7f7d83aac640 1 --2- 172.21.15.16:0/3701340282 >> v2:172.21.15.186:3300/0 conn(0x7f7d7c15dbd0 0x7f7d7c06c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.154+0000 7f7d83aac640 1 -- 172.21.15.16:0/3701340282 >> 172.21.15.16:0/3701340282 conn(0x7f7d7c0cca00 msgr2=0x7f7d7c0cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.154+0000 7f7d83aac640 1 -- 172.21.15.16:0/3701340282 shutdown_connections 2024-04-25T10:11:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.154+0000 7f7d83aac640 1 -- 172.21.15.16:0/3701340282 wait complete. 2024-04-25T10:11:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.155+0000 7f7d83aac640 1 Processor -- start 2024-04-25T10:11:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.155+0000 7f7d83aac640 1 -- start start 2024-04-25T10:11:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d83aac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d7c066e10 0x7f7d7c15c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d83aac640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d7c06aba0 0x7f7d7c14fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d83aac640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d7c15dbd0 0x7f7d7c0cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d83aac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c161720 con 0x7f7d7c066e10 2024-04-25T10:11:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d83aac640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c06bc20 con 0x7f7d7c15dbd0 2024-04-25T10:11:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d83aac640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c1618a0 con 0x7f7d7c06aba0 2024-04-25T10:11:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d81848640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d7c066e10 0x7f7d7c15c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d81848640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d7c066e10 0x7f7d7c15c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49282/0 (socket says 172.21.15.16:49282) 2024-04-25T10:11:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d81848640 1 -- 172.21.15.16:0/3686360850 learned_addr learned my addr 172.21.15.16:0/3686360850 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d82049640 1 --2- 172.21.15.16:0/3686360850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d7c15dbd0 0x7f7d7c0cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d81047640 1 --2- 172.21.15.16:0/3686360850 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d7c06aba0 0x7f7d7c14fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d81848640 1 -- 172.21.15.16:0/3686360850 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d7c06aba0 msgr2=0x7f7d7c14fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d81848640 1 --2- 172.21.15.16:0/3686360850 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d7c06aba0 0x7f7d7c14fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d81848640 1 -- 172.21.15.16:0/3686360850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d7c15dbd0 msgr2=0x7f7d7c0cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d81848640 1 --2- 172.21.15.16:0/3686360850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d7c15dbd0 0x7f7d7c0cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d81848640 1 -- 172.21.15.16:0/3686360850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c0b4e90 con 0x7f7d7c066e10 2024-04-25T10:11:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d82049640 1 --2- 172.21.15.16:0/3686360850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d7c15dbd0 0x7f7d7c0cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d81848640 1 --2- 172.21.15.16:0/3686360850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d7c066e10 0x7f7d7c15c5c0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800d800 tx=0x7f7d7800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d727fc640 1 -- 172.21.15.16:0/3686360850 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7800b6c0 con 0x7f7d7c066e10 2024-04-25T10:11:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.156+0000 7f7d727fc640 1 -- 172.21.15.16:0/3686360850 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d78019070 con 0x7f7d7c066e10 2024-04-25T10:11:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.157+0000 7f7d83aac640 1 -- 172.21.15.16:0/3686360850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7c0b5170 con 0x7f7d7c066e10 2024-04-25T10:11:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.157+0000 7f7d727fc640 1 -- 172.21.15.16:0/3686360850 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7800ba80 con 0x7f7d7c066e10 2024-04-25T10:11:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.157+0000 7f7d83aac640 1 -- 172.21.15.16:0/3686360850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7c15cb00 con 0x7f7d7c066e10 2024-04-25T10:11:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.158+0000 7f7d83aac640 1 -- 172.21.15.16:0/3686360850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7c0750b0 con 0x7f7d7c066e10 2024-04-25T10:11:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.158+0000 7f7d727fc640 1 -- 172.21.15.16:0/3686360850 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d7800bc20 con 0x7f7d7c066e10 2024-04-25T10:11:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.158+0000 7f7d727fc640 1 --2- 172.21.15.16:0/3686360850 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7d6006b4d0 0x7f7d6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.159+0000 7f7d727fc640 1 -- 172.21.15.16:0/3686360850 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7d7808a740 con 0x7f7d7c066e10 2024-04-25T10:11:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.159+0000 7f7d81047640 1 --2- 172.21.15.16:0/3686360850 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7d6006b4d0 0x7f7d6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.159+0000 7f7d81047640 1 --2- 172.21.15.16:0/3686360850 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7d6006b4d0 0x7f7d6006d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c034000 tx=0x7f7d6c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.163+0000 7f7d727fc640 1 -- 172.21.15.16:0/3686360850 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7d7801c360 con 0x7f7d7c066e10 2024-04-25T10:11:53.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.346+0000 7f7d83aac640 1 -- 172.21.15.16:0/3686360850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d7c066b10 con 0x7f7d7c066e10 2024-04-25T10:11:53.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.346+0000 7f7d727fc640 1 -- 172.21.15.16:0/3686360850 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7d78059560 con 0x7f7d7c066e10 2024-04-25T10:11:53.351 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:11:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.348+0000 7f7d83aac640 1 -- 172.21.15.16:0/3686360850 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7d6006b4d0 msgr2=0x7f7d6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.348+0000 7f7d83aac640 1 --2- 172.21.15.16:0/3686360850 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7d6006b4d0 0x7f7d6006d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c034000 tx=0x7f7d6c02e000 comp rx=0 tx=0).stop 2024-04-25T10:11:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.348+0000 7f7d83aac640 1 -- 172.21.15.16:0/3686360850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d7c066e10 msgr2=0x7f7d7c15c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.348+0000 7f7d83aac640 1 --2- 172.21.15.16:0/3686360850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d7c066e10 0x7f7d7c15c5c0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800d800 tx=0x7f7d7800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.348+0000 7f7d83aac640 1 -- 172.21.15.16:0/3686360850 shutdown_connections 2024-04-25T10:11:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.348+0000 7f7d83aac640 1 --2- 172.21.15.16:0/3686360850 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7d6006b4d0 0x7f7d6006d990 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.348+0000 7f7d83aac640 1 --2- 172.21.15.16:0/3686360850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d7c15dbd0 0x7f7d7c0cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.348+0000 7f7d83aac640 1 --2- 172.21.15.16:0/3686360850 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d7c06aba0 0x7f7d7c14fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.348+0000 7f7d83aac640 1 --2- 172.21.15.16:0/3686360850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d7c066e10 0x7f7d7c15c5c0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.348+0000 7f7d83aac640 1 -- 172.21.15.16:0/3686360850 >> 172.21.15.16:0/3686360850 conn(0x7f7d7c0cca00 msgr2=0x7f7d7c06dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.349+0000 7f7d83aac640 1 -- 172.21.15.16:0/3686360850 shutdown_connections 2024-04-25T10:11:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.349+0000 7f7d83aac640 1 -- 172.21.15.16:0/3686360850 wait complete. 2024-04-25T10:11:53.364 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:53.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.462+0000 7fb82ba6f640 1 Processor -- start 2024-04-25T10:11:53.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.463+0000 7fb82ba6f640 1 -- start start 2024-04-25T10:11:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.463+0000 7fb82ba6f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb8240b97c0 0x7fb8240b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.463+0000 7fb82ba6f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8240b5a30 0x7fb8240d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.463+0000 7fb82ba6f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8240b5e40 con 0x7fb82415dc80 2024-04-25T10:11:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.463+0000 7fb82ba6f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb824161f80 con 0x7fb8240b5a30 2024-04-25T10:11:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.463+0000 7fb82ba6f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8241627a0 con 0x7fb8240b97c0 2024-04-25T10:11:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.463+0000 7fb82a00c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8240b5a30 0x7fb8240d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.463+0000 7fb82980b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb82415dc80 0x7fb824161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49436/0 (socket says 172.21.15.16:49436) 2024-04-25T10:11:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.463+0000 7fb82980b640 1 -- 172.21.15.16:0/3451125379 learned_addr learned my addr 172.21.15.16:0/3451125379 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.464+0000 7fb82a00c640 1 -- 172.21.15.16:0/3451125379 >> v2:172.21.15.186:3300/0 conn(0x7fb8240b97c0 msgr2=0x7fb8240b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.464+0000 7fb82900a640 1 --2- 172.21.15.16:0/3451125379 >> v2:172.21.15.186:3300/0 conn(0x7fb8240b97c0 0x7fb8240b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.464+0000 7fb82a00c640 1 --2- 172.21.15.16:0/3451125379 >> v2:172.21.15.186:3300/0 conn(0x7fb8240b97c0 0x7fb8240b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.464+0000 7fb82a00c640 1 -- 172.21.15.16:0/3451125379 >> v1:172.21.15.16:6789/0 conn(0x7fb82415dc80 legacy=0x7fb824161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.464+0000 7fb828809640 1 -- 172.21.15.16:0/3451125379 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 606477782 0 0) 0x7fb814001490 con 0x7fb82415dc80 2024-04-25T10:11:53.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.464+0000 7fb82a00c640 1 -- 172.21.15.16:0/3451125379 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8241621e0 con 0x7fb8240b5a30 2024-04-25T10:11:53.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.464+0000 7fb82900a640 1 --2- 172.21.15.16:0/3451125379 >> v2:172.21.15.186:3300/0 conn(0x7fb8240b97c0 0x7fb8240b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:53.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.464+0000 7fb82a00c640 1 --2- 172.21.15.16:0/3451125379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8240b5a30 0x7fb8240d2470 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb82406ba70 tx=0x7fb81c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9351c1f8ebe6dfa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.465+0000 7fb828809640 1 -- 172.21.15.16:0/3451125379 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c037070 con 0x7fb8240b5a30 2024-04-25T10:11:53.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.465+0000 7fb828809640 1 -- 172.21.15.16:0/3451125379 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c032440 con 0x7fb8240b5a30 2024-04-25T10:11:53.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.465+0000 7fb828809640 1 -- 172.21.15.16:0/3451125379 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c032810 con 0x7fb8240b5a30 2024-04-25T10:11:53.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.465+0000 7fb82ba6f640 1 -- 172.21.15.16:0/3451125379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8240b5a30 msgr2=0x7fb8240d2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.465+0000 7fb82ba6f640 1 --2- 172.21.15.16:0/3451125379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8240b5a30 0x7fb8240d2470 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb82406ba70 tx=0x7fb81c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.465+0000 7fb82ba6f640 1 -- 172.21.15.16:0/3451125379 shutdown_connections 2024-04-25T10:11:53.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.465+0000 7fb82ba6f640 1 --2- 172.21.15.16:0/3451125379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8240b5a30 0x7fb8240d2470 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.465+0000 7fb82ba6f640 1 --2- 172.21.15.16:0/3451125379 >> v2:172.21.15.186:3300/0 conn(0x7fb8240b97c0 0x7fb8240b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.465+0000 7fb82ba6f640 1 -- 172.21.15.16:0/3451125379 >> 172.21.15.16:0/3451125379 conn(0x7fb8240d04f0 msgr2=0x7fb8240d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.465+0000 7fb82ba6f640 1 -- 172.21.15.16:0/3451125379 shutdown_connections 2024-04-25T10:11:53.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.465+0000 7fb82ba6f640 1 -- 172.21.15.16:0/3451125379 wait complete. 2024-04-25T10:11:53.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.466+0000 7fb82ba6f640 1 Processor -- start 2024-04-25T10:11:53.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.467+0000 7fb82ba6f640 1 -- start start 2024-04-25T10:11:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.467+0000 7fb82ba6f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8240b5a30 0x7fb82415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.467+0000 7fb82ba6f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8240b97c0 0x7fb8240c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.467+0000 7fb82ba6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb82415dc80 0x7fb8240b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.467+0000 7fb82ba6f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82415e060 con 0x7fb82415dc80 2024-04-25T10:11:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.467+0000 7fb82ba6f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8240d1c70 con 0x7fb8240b97c0 2024-04-25T10:11:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.467+0000 7fb82ba6f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8240d29b0 con 0x7fb8240b5a30 2024-04-25T10:11:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.467+0000 7fb82a00c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb82415dc80 0x7fb8240b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.467+0000 7fb82a00c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb82415dc80 0x7fb8240b6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49298/0 (socket says 172.21.15.16:49298) 2024-04-25T10:11:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.467+0000 7fb82a00c640 1 -- 172.21.15.16:0/4106697125 learned_addr learned my addr 172.21.15.16:0/4106697125 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.468+0000 7fb82900a640 1 --2- 172.21.15.16:0/4106697125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8240b97c0 0x7fb8240c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.468+0000 7fb82980b640 1 --2- 172.21.15.16:0/4106697125 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8240b5a30 0x7fb82415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.468+0000 7fb82a00c640 1 -- 172.21.15.16:0/4106697125 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8240b5a30 msgr2=0x7fb82415aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.468+0000 7fb82a00c640 1 --2- 172.21.15.16:0/4106697125 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8240b5a30 0x7fb82415aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.468+0000 7fb82a00c640 1 -- 172.21.15.16:0/4106697125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8240b97c0 msgr2=0x7fb8240c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.468+0000 7fb82a00c640 1 --2- 172.21.15.16:0/4106697125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8240b97c0 0x7fb8240c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.468+0000 7fb82a00c640 1 -- 172.21.15.16:0/4106697125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82415b530 con 0x7fb82415dc80 2024-04-25T10:11:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.468+0000 7fb82980b640 1 --2- 172.21.15.16:0/4106697125 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8240b5a30 0x7fb82415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.468+0000 7fb82a00c640 1 --2- 172.21.15.16:0/4106697125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb82415dc80 0x7fb8240b6fb0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb81c035040 tx=0x7fb81c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.468+0000 7fb8227fc640 1 -- 172.21.15.16:0/4106697125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c037040 con 0x7fb82415dc80 2024-04-25T10:11:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.468+0000 7fb8227fc640 1 -- 172.21.15.16:0/4106697125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c042070 con 0x7fb82415dc80 2024-04-25T10:11:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.468+0000 7fb8227fc640 1 -- 172.21.15.16:0/4106697125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c030b30 con 0x7fb82415dc80 2024-04-25T10:11:53.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.468+0000 7fb82ba6f640 1 -- 172.21.15.16:0/4106697125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8241f3980 con 0x7fb82415dc80 2024-04-25T10:11:53.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.468+0000 7fb82ba6f640 1 -- 172.21.15.16:0/4106697125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8240c8070 con 0x7fb82415dc80 2024-04-25T10:11:53.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.469+0000 7fb82ba6f640 1 -- 172.21.15.16:0/4106697125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8240750b0 con 0x7fb82415dc80 2024-04-25T10:11:53.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.470+0000 7fb8227fc640 1 -- 172.21.15.16:0/4106697125 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb81c030cd0 con 0x7fb82415dc80 2024-04-25T10:11:53.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.470+0000 7fb8227fc640 1 --2- 172.21.15.16:0/4106697125 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb81006b4d0 0x7fb81006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.470+0000 7fb8227fc640 1 -- 172.21.15.16:0/4106697125 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb81c0aad90 con 0x7fb82415dc80 2024-04-25T10:11:53.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.470+0000 7fb82980b640 1 --2- 172.21.15.16:0/4106697125 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb81006b4d0 0x7fb81006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.471+0000 7fb82980b640 1 --2- 172.21.15.16:0/4106697125 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb81006b4d0 0x7fb81006d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb8140012e0 tx=0x7fb814005b90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.474+0000 7fb8227fc640 1 -- 172.21.15.16:0/4106697125 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb82415b530 con 0x7fb82415dc80 2024-04-25T10:11:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.653+0000 7fb82ba6f640 1 -- 172.21.15.16:0/4106697125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb82415d1f0 con 0x7fb82415dc80 2024-04-25T10:11:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.653+0000 7fb8227fc640 1 -- 172.21.15.16:0/4106697125 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb81c079c40 con 0x7fb82415dc80 2024-04-25T10:11:53.658 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:11:53.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.655+0000 7fb82ba6f640 1 -- 172.21.15.16:0/4106697125 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb81006b4d0 msgr2=0x7fb81006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.655+0000 7fb82ba6f640 1 --2- 172.21.15.16:0/4106697125 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb81006b4d0 0x7fb81006d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb8140012e0 tx=0x7fb814005b90 comp rx=0 tx=0).stop 2024-04-25T10:11:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.655+0000 7fb82ba6f640 1 -- 172.21.15.16:0/4106697125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb82415dc80 msgr2=0x7fb8240b6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.655+0000 7fb82ba6f640 1 --2- 172.21.15.16:0/4106697125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb82415dc80 0x7fb8240b6fb0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb81c035040 tx=0x7fb81c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.655+0000 7fb82ba6f640 1 -- 172.21.15.16:0/4106697125 shutdown_connections 2024-04-25T10:11:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.655+0000 7fb82ba6f640 1 --2- 172.21.15.16:0/4106697125 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb81006b4d0 0x7fb81006d990 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.655+0000 7fb82ba6f640 1 --2- 172.21.15.16:0/4106697125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb82415dc80 0x7fb8240b6fb0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.655+0000 7fb82ba6f640 1 --2- 172.21.15.16:0/4106697125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8240b97c0 0x7fb8240c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.655+0000 7fb82ba6f640 1 --2- 172.21.15.16:0/4106697125 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8240b5a30 0x7fb82415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.655+0000 7fb82ba6f640 1 -- 172.21.15.16:0/4106697125 >> 172.21.15.16:0/4106697125 conn(0x7fb8240d04f0 msgr2=0x7fb8240ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.655+0000 7fb82ba6f640 1 -- 172.21.15.16:0/4106697125 shutdown_connections 2024-04-25T10:11:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.655+0000 7fb82ba6f640 1 -- 172.21.15.16:0/4106697125 wait complete. 2024-04-25T10:11:53.671 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:53.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.769+0000 7f3d61f42640 1 Processor -- start 2024-04-25T10:11:53.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.769+0000 7f3d61f42640 1 -- start start 2024-04-25T10:11:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.770+0000 7f3d61f42640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3d5c0d3b90 0x7f3d5c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.770+0000 7f3d61f42640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3d5c0d0f60 0x7f3d5c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.770+0000 7f3d61f42640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d5c0d4090 con 0x7f3d5c15dca0 2024-04-25T10:11:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.770+0000 7f3d61f42640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5c0b6b20 con 0x7f3d5c0d3b90 2024-04-25T10:11:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.770+0000 7f3d61f42640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5c0d1390 con 0x7f3d5c0d0f60 2024-04-25T10:11:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.770+0000 7f3d5affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3d5c0d3b90 0x7f3d5c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.770+0000 7f3d5affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3d5c0d3b90 0x7f3d5c0d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49310/0 (socket says 172.21.15.16:49310) 2024-04-25T10:11:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.770+0000 7f3d5affd640 1 -- 172.21.15.16:0/1313882597 learned_addr learned my addr 172.21.15.16:0/1313882597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.770+0000 7f3d5bfff640 1 --2- 172.21.15.16:0/1313882597 >> v2:172.21.15.186:3300/0 conn(0x7f3d5c0d0f60 0x7f3d5c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.770+0000 7f3d5affd640 1 -- 172.21.15.16:0/1313882597 >> v2:172.21.15.186:3300/0 conn(0x7f3d5c0d0f60 msgr2=0x7f3d5c0b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.770+0000 7f3d5affd640 1 --2- 172.21.15.16:0/1313882597 >> v2:172.21.15.186:3300/0 conn(0x7f3d5c0d0f60 0x7f3d5c0b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.770+0000 7f3d5affd640 1 -- 172.21.15.16:0/1313882597 >> v1:172.21.15.186:6789/0 conn(0x7f3d5c15dca0 legacy=0x7f3d5c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.770+0000 7f3d5affd640 1 -- 172.21.15.16:0/1313882597 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c0d4370 con 0x7f3d5c0d3b90 2024-04-25T10:11:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.771+0000 7f3d5affd640 1 --2- 172.21.15.16:0/1313882597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d5c0d3b90 0x7f3d5c0d0a20 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0b9640 tx=0x7f3d5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d70d191ddf41db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.771+0000 7f3d5a7fc640 1 -- 172.21.15.16:0/1313882597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d50037070 con 0x7f3d5c0d3b90 2024-04-25T10:11:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.771+0000 7f3d5a7fc640 1 -- 172.21.15.16:0/1313882597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d50032440 con 0x7f3d5c0d3b90 2024-04-25T10:11:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.771+0000 7f3d5a7fc640 1 -- 172.21.15.16:0/1313882597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d50032810 con 0x7f3d5c0d3b90 2024-04-25T10:11:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.772+0000 7f3d61f42640 1 -- 172.21.15.16:0/1313882597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d5c0d3b90 msgr2=0x7f3d5c0d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.772+0000 7f3d61f42640 1 --2- 172.21.15.16:0/1313882597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d5c0d3b90 0x7f3d5c0d0a20 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0b9640 tx=0x7f3d5002a270 comp rx=0 tx=0).stop 2024-04-25T10:11:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.772+0000 7f3d61f42640 1 -- 172.21.15.16:0/1313882597 shutdown_connections 2024-04-25T10:11:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.772+0000 7f3d61f42640 1 --2- 172.21.15.16:0/1313882597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d5c0d3b90 0x7f3d5c0d0a20 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.772+0000 7f3d61f42640 1 --2- 172.21.15.16:0/1313882597 >> v2:172.21.15.186:3300/0 conn(0x7f3d5c0d0f60 0x7f3d5c0b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.772+0000 7f3d61f42640 1 -- 172.21.15.16:0/1313882597 >> 172.21.15.16:0/1313882597 conn(0x7f3d5c0726c0 msgr2=0x7f3d5c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.772+0000 7f3d61f42640 1 -- 172.21.15.16:0/1313882597 shutdown_connections 2024-04-25T10:11:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.772+0000 7f3d61f42640 1 -- 172.21.15.16:0/1313882597 wait complete. 2024-04-25T10:11:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.773+0000 7f3d61f42640 1 Processor -- start 2024-04-25T10:11:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.773+0000 7f3d61f42640 1 -- start start 2024-04-25T10:11:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.774+0000 7f3d61f42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d5c0d0f60 0x7f3d5c0aa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.774+0000 7f3d61f42640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d5c0d3b90 0x7f3d5c0c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.774+0000 7f3d61f42640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d5c15dca0 0x7f3d5c15fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.774+0000 7f3d61f42640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c0cf050 con 0x7f3d5c0d0f60 2024-04-25T10:11:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.774+0000 7f3d61f42640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c0b6d50 con 0x7f3d5c0d3b90 2024-04-25T10:11:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.774+0000 7f3d61f42640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c15e080 con 0x7f3d5c15dca0 2024-04-25T10:11:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.774+0000 7f3d5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d5c0d0f60 0x7f3d5c0aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.774+0000 7f3d5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d5c0d0f60 0x7f3d5c0aa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49316/0 (socket says 172.21.15.16:49316) 2024-04-25T10:11:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.774+0000 7f3d5b7fe640 1 -- 172.21.15.16:0/594910259 learned_addr learned my addr 172.21.15.16:0/594910259 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.774+0000 7f3d5bfff640 1 --2- 172.21.15.16:0/594910259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d5c15dca0 0x7f3d5c15fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.774+0000 7f3d5affd640 1 --2- 172.21.15.16:0/594910259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d5c0d3b90 0x7f3d5c0c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.774+0000 7f3d5b7fe640 1 -- 172.21.15.16:0/594910259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d5c15dca0 msgr2=0x7f3d5c15fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.774+0000 7f3d5b7fe640 1 --2- 172.21.15.16:0/594910259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d5c15dca0 0x7f3d5c15fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.774+0000 7f3d5b7fe640 1 -- 172.21.15.16:0/594910259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d5c0d3b90 msgr2=0x7f3d5c0c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.774+0000 7f3d5b7fe640 1 --2- 172.21.15.16:0/594910259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d5c0d3b90 0x7f3d5c0c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.774+0000 7f3d5b7fe640 1 -- 172.21.15.16:0/594910259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c0c7140 con 0x7f3d5c0d0f60 2024-04-25T10:11:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.775+0000 7f3d5b7fe640 1 --2- 172.21.15.16:0/594910259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d5c0d0f60 0x7f3d5c0aa480 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00b3f0 tx=0x7f3d4c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.775+0000 7f3d43fff640 1 -- 172.21.15.16:0/594910259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c015020 con 0x7f3d5c0d0f60 2024-04-25T10:11:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.775+0000 7f3d43fff640 1 -- 172.21.15.16:0/594910259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c011830 con 0x7f3d5c0d0f60 2024-04-25T10:11:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.775+0000 7f3d61f42640 1 -- 172.21.15.16:0/594910259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5c0c7760 con 0x7f3d5c0d0f60 2024-04-25T10:11:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.775+0000 7f3d43fff640 1 -- 172.21.15.16:0/594910259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c011c20 con 0x7f3d5c0d0f60 2024-04-25T10:11:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.775+0000 7f3d61f42640 1 -- 172.21.15.16:0/594910259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5c15c6f0 con 0x7f3d5c0d0f60 2024-04-25T10:11:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.776+0000 7f3d61f42640 1 -- 172.21.15.16:0/594910259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5c0750b0 con 0x7f3d5c0d0f60 2024-04-25T10:11:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.776+0000 7f3d43fff640 1 -- 172.21.15.16:0/594910259 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d4c018020 con 0x7f3d5c0d0f60 2024-04-25T10:11:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.777+0000 7f3d43fff640 1 --2- 172.21.15.16:0/594910259 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3d4806b4d0 0x7f3d4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.777+0000 7f3d43fff640 1 -- 172.21.15.16:0/594910259 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3d4c089d80 con 0x7f3d5c0d0f60 2024-04-25T10:11:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.777+0000 7f3d5affd640 1 --2- 172.21.15.16:0/594910259 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3d4806b4d0 0x7f3d4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.777+0000 7f3d5affd640 1 --2- 172.21.15.16:0/594910259 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3d4806b4d0 0x7f3d4806d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3d50034000 tx=0x7f3d5002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.781+0000 7f3d43fff640 1 -- 172.21.15.16:0/594910259 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d4c01f030 con 0x7f3d5c0d0f60 2024-04-25T10:11:53.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.959+0000 7f3d61f42640 1 -- 172.21.15.16:0/594910259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3d5c0b2fb0 con 0x7f3d5c0d0f60 2024-04-25T10:11:53.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.960+0000 7f3d43fff640 1 -- 172.21.15.16:0/594910259 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3d4c058550 con 0x7f3d5c0d0f60 2024-04-25T10:11:53.965 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:11:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.962+0000 7f3d61f42640 1 -- 172.21.15.16:0/594910259 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3d4806b4d0 msgr2=0x7f3d4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.962+0000 7f3d61f42640 1 --2- 172.21.15.16:0/594910259 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3d4806b4d0 0x7f3d4806d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3d50034000 tx=0x7f3d5002e000 comp rx=0 tx=0).stop 2024-04-25T10:11:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.962+0000 7f3d61f42640 1 -- 172.21.15.16:0/594910259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d5c0d0f60 msgr2=0x7f3d5c0aa480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.962+0000 7f3d61f42640 1 --2- 172.21.15.16:0/594910259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d5c0d0f60 0x7f3d5c0aa480 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00b3f0 tx=0x7f3d4c00b8b0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.962+0000 7f3d61f42640 1 -- 172.21.15.16:0/594910259 shutdown_connections 2024-04-25T10:11:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.962+0000 7f3d61f42640 1 --2- 172.21.15.16:0/594910259 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3d4806b4d0 0x7f3d4806d990 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.962+0000 7f3d61f42640 1 --2- 172.21.15.16:0/594910259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d5c15dca0 0x7f3d5c15fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.962+0000 7f3d61f42640 1 --2- 172.21.15.16:0/594910259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d5c0d3b90 0x7f3d5c0c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.962+0000 7f3d61f42640 1 --2- 172.21.15.16:0/594910259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d5c0d0f60 0x7f3d5c0aa480 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.962+0000 7f3d61f42640 1 -- 172.21.15.16:0/594910259 >> 172.21.15.16:0/594910259 conn(0x7f3d5c0726c0 msgr2=0x7f3d5c0b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.962+0000 7f3d61f42640 1 -- 172.21.15.16:0/594910259 shutdown_connections 2024-04-25T10:11:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:53.962+0000 7f3d61f42640 1 -- 172.21.15.16:0/594910259 wait complete. 2024-04-25T10:11:53.978 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:11:54.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.075+0000 7f807d28f640 1 Processor -- start 2024-04-25T10:11:54.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.075+0000 7f807d28f640 1 -- start start 2024-04-25T10:11:54.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.075+0000 7f807d28f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f80780cef40 0x7f80780b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.075+0000 7f807d28f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8078162040 con 0x7f80781610b0 2024-04-25T10:11:54.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.075+0000 7f807d28f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8078162320 con 0x7f80781618d0 2024-04-25T10:11:54.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.075+0000 7f807d28f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80780d1ea0 con 0x7f80780cef40 2024-04-25T10:11:54.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.075+0000 7f8076ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f80781610b0 0x7f80780ce830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49442/0 (socket says 172.21.15.16:49442) 2024-04-25T10:11:54.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f8076ffd640 1 -- 172.21.15.16:0/4168093482 learned_addr learned my addr 172.21.15.16:0/4168093482 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:54.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f8075ffb640 1 -- 172.21.15.16:0/4168093482 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3294361588 0 0) 0x7f8078162040 con 0x7f80781610b0 2024-04-25T10:11:54.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f80767fc640 1 --2- 172.21.15.16:0/4168093482 >> v2:172.21.15.186:3300/0 conn(0x7f80780cef40 0x7f80780b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f8075ffb640 1 -- 172.21.15.16:0/4168093482 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8060003520 con 0x7f80781610b0 2024-04-25T10:11:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f8075ffb640 1 -- 172.21.15.16:0/4168093482 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3391780399 0 0) 0x7f8060003520 con 0x7f80781610b0 2024-04-25T10:11:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f8075ffb640 1 -- 172.21.15.16:0/4168093482 >> v2:172.21.15.186:3300/0 conn(0x7f80780cef40 msgr2=0x7f80780b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f8075ffb640 1 --2- 172.21.15.16:0/4168093482 >> v2:172.21.15.186:3300/0 conn(0x7f80780cef40 0x7f80780b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f8075ffb640 1 -- 172.21.15.16:0/4168093482 >> v1:172.21.15.186:6789/0 conn(0x7f80781618d0 legacy=0x7f80780d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f8075ffb640 1 -- 172.21.15.16:0/4168093482 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8078162600 con 0x7f80781610b0 2024-04-25T10:11:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f8075ffb640 1 -- 172.21.15.16:0/4168093482 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f8068001670 con 0x7f80781610b0 2024-04-25T10:11:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f8075ffb640 1 -- 172.21.15.16:0/4168093482 >> v1:172.21.15.16:6789/0 conn(0x7f80781610b0 legacy=0x7f80780ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f8075ffb640 1 --2- 172.21.15.16:0/4168093482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80600039b0 0x7f8060003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f8075ffb640 1 --2- 172.21.15.16:0/4168093482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8060004800 0x7f8060004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f8075ffb640 1 --2- 172.21.15.16:0/4168093482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8060005210 0x7f806000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f8075ffb640 1 -- 172.21.15.16:0/4168093482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80780d1ea0 con 0x7f8060005210 2024-04-25T10:11:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f8075ffb640 1 -- 172.21.15.16:0/4168093482 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806000c140 con 0x7f8060004800 2024-04-25T10:11:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f8075ffb640 1 -- 172.21.15.16:0/4168093482 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806000c340 con 0x7f80600039b0 2024-04-25T10:11:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f80777fe640 1 --2- 172.21.15.16:0/4168093482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8060005210 0x7f806000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f8076ffd640 1 --2- 172.21.15.16:0/4168093482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80600039b0 0x7f8060003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.076+0000 7f80767fc640 1 --2- 172.21.15.16:0/4168093482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8060004800 0x7f8060004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.077+0000 7f80777fe640 1 -- 172.21.15.16:0/4168093482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80600039b0 msgr2=0x7f8060003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.077+0000 7f80777fe640 1 --2- 172.21.15.16:0/4168093482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80600039b0 0x7f8060003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.077+0000 7f80777fe640 1 -- 172.21.15.16:0/4168093482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8060004800 msgr2=0x7f8060004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.077+0000 7f80777fe640 1 --2- 172.21.15.16:0/4168093482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8060004800 0x7f8060004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.077+0000 7f80777fe640 1 -- 172.21.15.16:0/4168093482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8060003520 con 0x7f8060005210 2024-04-25T10:11:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.077+0000 7f80777fe640 1 -- 172.21.15.16:0/4168093482 reap_dead start 2024-04-25T10:11:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.077+0000 7f80777fe640 1 --2- 172.21.15.16:0/4168093482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8060005210 0x7f806000b9c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f8070001e50 tx=0x7f8060005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9a99254d0fd8e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.077+0000 7f8075ffb640 1 -- 172.21.15.16:0/4168093482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070001a70 con 0x7f8060005210 2024-04-25T10:11:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.077+0000 7f8075ffb640 1 -- 172.21.15.16:0/4168093482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8070003970 con 0x7f8060005210 2024-04-25T10:11:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.077+0000 7f8075ffb640 1 -- 172.21.15.16:0/4168093482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070003d60 con 0x7f8060005210 2024-04-25T10:11:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.078+0000 7f807d28f640 1 -- 172.21.15.16:0/4168093482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8060005210 msgr2=0x7f806000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.078+0000 7f807d28f640 1 --2- 172.21.15.16:0/4168093482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8060005210 0x7f806000b9c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f8070001e50 tx=0x7f8060005830 comp rx=0 tx=0).stop 2024-04-25T10:11:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.078+0000 7f807d28f640 1 -- 172.21.15.16:0/4168093482 shutdown_connections 2024-04-25T10:11:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.078+0000 7f807d28f640 1 --2- 172.21.15.16:0/4168093482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8060005210 0x7f806000b9c0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.078+0000 7f807d28f640 1 -- 172.21.15.16:0/4168093482 >> 172.21.15.16:0/4168093482 conn(0x7f80780b5570 msgr2=0x7f80780b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.078+0000 7f807d28f640 1 -- 172.21.15.16:0/4168093482 shutdown_connections 2024-04-25T10:11:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.078+0000 7f807d28f640 1 -- 172.21.15.16:0/4168093482 wait complete. 2024-04-25T10:11:54.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.079+0000 7f807d28f640 1 Processor -- start 2024-04-25T10:11:54.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.079+0000 7f807d28f640 1 -- start start 2024-04-25T10:11:54.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.080+0000 7f807d28f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8060005210 0x7f80780c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.080+0000 7f807d28f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80781502f0 0x7f8078151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.080+0000 7f807d28f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80781548c0 0x7f8078154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.080+0000 7f807d28f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8078161d40 con 0x7f8060005210 2024-04-25T10:11:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.080+0000 7f807d28f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80780b6c00 con 0x7f80781502f0 2024-04-25T10:11:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.080+0000 7f807d28f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8078161490 con 0x7f80781548c0 2024-04-25T10:11:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.080+0000 7f8076ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8060005210 0x7f80780c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.080+0000 7f8076ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8060005210 0x7f80780c98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49338/0 (socket says 172.21.15.16:49338) 2024-04-25T10:11:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.080+0000 7f8076ffd640 1 -- 172.21.15.16:0/2970413643 learned_addr learned my addr 172.21.15.16:0/2970413643 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.080+0000 7f80767fc640 1 --2- 172.21.15.16:0/2970413643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80781502f0 0x7f8078151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.080+0000 7f80777fe640 1 --2- 172.21.15.16:0/2970413643 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80781548c0 0x7f8078154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.081+0000 7f8076ffd640 1 -- 172.21.15.16:0/2970413643 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80781548c0 msgr2=0x7f8078154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.081+0000 7f8076ffd640 1 --2- 172.21.15.16:0/2970413643 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80781548c0 0x7f8078154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.081+0000 7f8076ffd640 1 -- 172.21.15.16:0/2970413643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80781502f0 msgr2=0x7f8078151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.081+0000 7f8076ffd640 1 --2- 172.21.15.16:0/2970413643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80781502f0 0x7f8078151450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.081+0000 7f8076ffd640 1 -- 172.21.15.16:0/2970413643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80780bfb90 con 0x7f8060005210 2024-04-25T10:11:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.081+0000 7f8076ffd640 1 --2- 172.21.15.16:0/2970413643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8060005210 0x7f80780c98a0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f8068007370 tx=0x7f8068007770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.081+0000 7f805f7fe640 1 -- 172.21.15.16:0/2970413643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806801a070 con 0x7f8060005210 2024-04-25T10:11:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.081+0000 7f805f7fe640 1 -- 172.21.15.16:0/2970413643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806800f960 con 0x7f8060005210 2024-04-25T10:11:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.081+0000 7f807d28f640 1 -- 172.21.15.16:0/2970413643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8078150980 con 0x7f8060005210 2024-04-25T10:11:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.081+0000 7f805f7fe640 1 -- 172.21.15.16:0/2970413643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806800fd50 con 0x7f8060005210 2024-04-25T10:11:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.081+0000 7f807d28f640 1 -- 172.21.15.16:0/2970413643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80780caeb0 con 0x7f8060005210 2024-04-25T10:11:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.082+0000 7f807d28f640 1 -- 172.21.15.16:0/2970413643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80780750b0 con 0x7f8060005210 2024-04-25T10:11:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.082+0000 7f805f7fe640 1 -- 172.21.15.16:0/2970413643 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f806801d070 con 0x7f8060005210 2024-04-25T10:11:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.083+0000 7f805f7fe640 1 --2- 172.21.15.16:0/2970413643 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f806406b4d0 0x7f806406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.083+0000 7f805f7fe640 1 -- 172.21.15.16:0/2970413643 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8068089a50 con 0x7f8060005210 2024-04-25T10:11:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.083+0000 7f80767fc640 1 --2- 172.21.15.16:0/2970413643 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f806406b4d0 0x7f806406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.084+0000 7f80767fc640 1 --2- 172.21.15.16:0/2970413643 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f806406b4d0 0x7f806406d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f80781f7460 tx=0x7f806c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.087+0000 7f805f7fe640 1 -- 172.21.15.16:0/2970413643 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8068058180 con 0x7f8060005210 2024-04-25T10:11:54.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.280+0000 7f807d28f640 1 -- 172.21.15.16:0/2970413643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f80781506d0 con 0x7f8060005210 2024-04-25T10:11:54.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.281+0000 7f805f7fe640 1 -- 172.21.15.16:0/2970413643 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v87) v1 ==== 80+0+124 (secure 0 0 0) 0x7f806805c220 con 0x7f8060005210 2024-04-25T10:11:54.286 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.186:6837/3092291936 2024-04-26T10:11:13.827348+0000 2024-04-25T10:11:54.286 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.186:6835/3092291936 2024-04-26T10:11:13.827348+0000 2024-04-25T10:11:54.286 INFO:teuthology.orchestra.run.smithi016.stderr:listed 2 entries 2024-04-25T10:11:54.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.283+0000 7f807d28f640 1 -- 172.21.15.16:0/2970413643 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f806406b4d0 msgr2=0x7f806406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.283+0000 7f807d28f640 1 --2- 172.21.15.16:0/2970413643 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f806406b4d0 0x7f806406d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f80781f7460 tx=0x7f806c009190 comp rx=0 tx=0).stop 2024-04-25T10:11:54.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.283+0000 7f807d28f640 1 -- 172.21.15.16:0/2970413643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8060005210 msgr2=0x7f80780c98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.283+0000 7f807d28f640 1 --2- 172.21.15.16:0/2970413643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8060005210 0x7f80780c98a0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f8068007370 tx=0x7f8068007770 comp rx=0 tx=0).stop 2024-04-25T10:11:54.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.283+0000 7f807d28f640 1 -- 172.21.15.16:0/2970413643 shutdown_connections 2024-04-25T10:11:54.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.283+0000 7f807d28f640 1 --2- 172.21.15.16:0/2970413643 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f806406b4d0 0x7f806406d990 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.283+0000 7f807d28f640 1 --2- 172.21.15.16:0/2970413643 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80781548c0 0x7f8078154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.283+0000 7f807d28f640 1 --2- 172.21.15.16:0/2970413643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80781502f0 0x7f8078151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.283+0000 7f807d28f640 1 --2- 172.21.15.16:0/2970413643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8060005210 0x7f80780c98a0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.283+0000 7f807d28f640 1 -- 172.21.15.16:0/2970413643 >> 172.21.15.16:0/2970413643 conn(0x7f80780b5570 msgr2=0x7f8078074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.283+0000 7f807d28f640 1 -- 172.21.15.16:0/2970413643 shutdown_connections 2024-04-25T10:11:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.283+0000 7f807d28f640 1 -- 172.21.15.16:0/2970413643 wait complete. 2024-04-25T10:11:54.299 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:11:54.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:54.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:54.326 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:11:54.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:54.326 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:11:54.382 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:54.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.518+0000 7f3eb3529640 1 Processor -- start 2024-04-25T10:11:54.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.518+0000 7f3eb3529640 1 -- start start 2024-04-25T10:11:54.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.519+0000 7f3eb3529640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3eac15ba10 0x7f3eac099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.519+0000 7f3eb3529640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3eac099f80 0x7f3eac06bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.519+0000 7f3eb3529640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3eac06c1f0 0x7f3eac0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.519+0000 7f3eb3529640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eac0cbe50 con 0x7f3eac15ba10 2024-04-25T10:11:54.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.519+0000 7f3eb3529640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eac0cfe80 con 0x7f3eac06c1f0 2024-04-25T10:11:54.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.519+0000 7f3eb3529640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac09a3a0 con 0x7f3eac099f80 2024-04-25T10:11:54.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.519+0000 7f3eb12c5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3eac15ba10 0x7f3eac099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.519+0000 7f3eb12c5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3eac15ba10 0x7f3eac099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49344/0 (socket says 172.21.15.16:49344) 2024-04-25T10:11:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.519+0000 7f3eb12c5640 1 -- 172.21.15.16:0/3361347951 learned_addr learned my addr 172.21.15.16:0/3361347951 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.519+0000 7f3eb1ac6640 1 --2- 172.21.15.16:0/3361347951 >> v2:172.21.15.186:3300/0 conn(0x7f3eac06c1f0 0x7f3eac0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.519+0000 7f3eb0ac4640 1 --2- 172.21.15.16:0/3361347951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3eac099f80 0x7f3eac06bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.519+0000 7f3eb12c5640 1 -- 172.21.15.16:0/3361347951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3eac099f80 msgr2=0x7f3eac06bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.519+0000 7f3eb12c5640 1 --2- 172.21.15.16:0/3361347951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3eac099f80 0x7f3eac06bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.519+0000 7f3eb12c5640 1 -- 172.21.15.16:0/3361347951 >> v2:172.21.15.186:3300/0 conn(0x7f3eac06c1f0 msgr2=0x7f3eac0cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.519+0000 7f3eb12c5640 1 --2- 172.21.15.16:0/3361347951 >> v2:172.21.15.186:3300/0 conn(0x7f3eac06c1f0 0x7f3eac0cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.519+0000 7f3eb12c5640 1 -- 172.21.15.16:0/3361347951 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac06c680 con 0x7f3eac15ba10 2024-04-25T10:11:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.520+0000 7f3eb0ac4640 1 --2- 172.21.15.16:0/3361347951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3eac099f80 0x7f3eac06bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.520+0000 7f3eb12c5640 1 --2- 172.21.15.16:0/3361347951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eac15ba10 0x7f3eac099a40 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f3eac1616a0 tx=0x7f3ea802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dad46ed3f62693f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.520+0000 7f3ea3fff640 1 -- 172.21.15.16:0/3361347951 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8037070 con 0x7f3eac15ba10 2024-04-25T10:11:54.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.520+0000 7f3ea3fff640 1 -- 172.21.15.16:0/3361347951 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea8032440 con 0x7f3eac15ba10 2024-04-25T10:11:54.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.520+0000 7f3ea3fff640 1 -- 172.21.15.16:0/3361347951 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8032810 con 0x7f3eac15ba10 2024-04-25T10:11:54.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.521+0000 7f3eb3529640 1 -- 172.21.15.16:0/3361347951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eac15ba10 msgr2=0x7f3eac099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.521+0000 7f3eb3529640 1 --2- 172.21.15.16:0/3361347951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eac15ba10 0x7f3eac099a40 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f3eac1616a0 tx=0x7f3ea802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.521+0000 7f3eb3529640 1 -- 172.21.15.16:0/3361347951 shutdown_connections 2024-04-25T10:11:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.521+0000 7f3eb3529640 1 --2- 172.21.15.16:0/3361347951 >> v2:172.21.15.186:3300/0 conn(0x7f3eac06c1f0 0x7f3eac0cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.521+0000 7f3eb3529640 1 --2- 172.21.15.16:0/3361347951 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3eac099f80 0x7f3eac06bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.521+0000 7f3eb3529640 1 --2- 172.21.15.16:0/3361347951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eac15ba10 0x7f3eac099a40 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.521+0000 7f3eb3529640 1 -- 172.21.15.16:0/3361347951 >> 172.21.15.16:0/3361347951 conn(0x7f3eac0b32f0 msgr2=0x7f3eac0b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.521+0000 7f3eb3529640 1 -- 172.21.15.16:0/3361347951 shutdown_connections 2024-04-25T10:11:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.521+0000 7f3eb3529640 1 -- 172.21.15.16:0/3361347951 wait complete. 2024-04-25T10:11:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.522+0000 7f3eb3529640 1 Processor -- start 2024-04-25T10:11:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.522+0000 7f3eb3529640 1 -- start start 2024-04-25T10:11:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.523+0000 7f3eb3529640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3eac06c1f0 0x7f3eac0af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.523+0000 7f3eb3529640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3eac099f80 0x7f3eac147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.523+0000 7f3eb3529640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eac15ba10 0x7f3eac147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.523+0000 7f3eb3529640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac06cec0 con 0x7f3eac15ba10 2024-04-25T10:11:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.523+0000 7f3eb3529640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac06cbe0 con 0x7f3eac06c1f0 2024-04-25T10:11:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.523+0000 7f3eb3529640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0cd760 con 0x7f3eac099f80 2024-04-25T10:11:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.523+0000 7f3eb1ac6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eac15ba10 0x7f3eac147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.523+0000 7f3eb1ac6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eac15ba10 0x7f3eac147f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49352/0 (socket says 172.21.15.16:49352) 2024-04-25T10:11:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.523+0000 7f3eb1ac6640 1 -- 172.21.15.16:0/2557416656 learned_addr learned my addr 172.21.15.16:0/2557416656 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.523+0000 7f3eb12c5640 1 --2- 172.21.15.16:0/2557416656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3eac06c1f0 0x7f3eac0af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.523+0000 7f3eb0ac4640 1 --2- 172.21.15.16:0/2557416656 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3eac099f80 0x7f3eac147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.523+0000 7f3eb1ac6640 1 -- 172.21.15.16:0/2557416656 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3eac099f80 msgr2=0x7f3eac147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.523+0000 7f3eb1ac6640 1 --2- 172.21.15.16:0/2557416656 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3eac099f80 0x7f3eac147a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.523+0000 7f3eb1ac6640 1 -- 172.21.15.16:0/2557416656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3eac06c1f0 msgr2=0x7f3eac0af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.523+0000 7f3eb1ac6640 1 --2- 172.21.15.16:0/2557416656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3eac06c1f0 0x7f3eac0af7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.523+0000 7f3eb1ac6640 1 -- 172.21.15.16:0/2557416656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac159370 con 0x7f3eac15ba10 2024-04-25T10:11:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.524+0000 7f3eb1ac6640 1 --2- 172.21.15.16:0/2557416656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eac15ba10 0x7f3eac147f80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4007e70 tx=0x7f3ea400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.524+0000 7f3ea1ffb640 1 -- 172.21.15.16:0/2557416656 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4014620 con 0x7f3eac15ba10 2024-04-25T10:11:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.524+0000 7f3ea1ffb640 1 -- 172.21.15.16:0/2557416656 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea4019070 con 0x7f3eac15ba10 2024-04-25T10:11:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.524+0000 7f3eb3529640 1 -- 172.21.15.16:0/2557416656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eac0aed50 con 0x7f3eac15ba10 2024-04-25T10:11:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.524+0000 7f3ea1ffb640 1 -- 172.21.15.16:0/2557416656 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4014a20 con 0x7f3eac15ba10 2024-04-25T10:11:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.524+0000 7f3eb3529640 1 -- 172.21.15.16:0/2557416656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eac0c53d0 con 0x7f3eac15ba10 2024-04-25T10:11:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.525+0000 7f3eb3529640 1 -- 172.21.15.16:0/2557416656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eac0749b0 con 0x7f3eac15ba10 2024-04-25T10:11:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.525+0000 7f3ea1ffb640 1 -- 172.21.15.16:0/2557416656 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ea4014bc0 con 0x7f3eac15ba10 2024-04-25T10:11:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.525+0000 7f3ea1ffb640 1 --2- 172.21.15.16:0/2557416656 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3e9006b4d0 0x7f3e9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.526+0000 7f3ea1ffb640 1 -- 172.21.15.16:0/2557416656 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3ea408aab0 con 0x7f3eac15ba10 2024-04-25T10:11:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.526+0000 7f3eb12c5640 1 --2- 172.21.15.16:0/2557416656 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3e9006b4d0 0x7f3e9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.526+0000 7f3eb12c5640 1 --2- 172.21.15.16:0/2557416656 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3e9006b4d0 0x7f3e9006d990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8034000 tx=0x7f3ea802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.530+0000 7f3ea1ffb640 1 -- 172.21.15.16:0/2557416656 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3eac159370 con 0x7f3eac15ba10 2024-04-25T10:11:54.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.747+0000 7f3eb3529640 1 -- 172.21.15.16:0/2557416656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3eac1614b0 con 0x7f3eac15ba10 2024-04-25T10:11:54.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.748+0000 7f3ea1ffb640 1 -- 172.21.15.16:0/2557416656 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f3ea40591f0 con 0x7f3eac15ba10 2024-04-25T10:11:54.753 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:54.753 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4044509901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4044509901},{"type":"v1","addr":"172.21.15.16:6835","nonce":4044509901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1494230493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1494230493},{"type":"v1","addr":"172.21.15.186:6837","nonce":1494230493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/84349474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":84349474},{"type":"v1","addr":"172.21.15.16:6834","nonce":84349474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:00.015075+0000","modified":"2024-04-25T10:11:20.851789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"d","rank":0,"incarnation":54,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6836/3950732718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3950732718},{"type":"v1","addr":"172.21.15.186:6836","nonce":3950732718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 58 2024-04-25T10:11:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.749+0000 7f3eb3529640 1 -- 172.21.15.16:0/2557416656 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3e9006b4d0 msgr2=0x7f3e9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.749+0000 7f3eb3529640 1 --2- 172.21.15.16:0/2557416656 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3e9006b4d0 0x7f3e9006d990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8034000 tx=0x7f3ea802e000 comp rx=0 tx=0).stop 2024-04-25T10:11:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.749+0000 7f3eb3529640 1 -- 172.21.15.16:0/2557416656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eac15ba10 msgr2=0x7f3eac147f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.749+0000 7f3eb3529640 1 --2- 172.21.15.16:0/2557416656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eac15ba10 0x7f3eac147f80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4007e70 tx=0x7f3ea400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.750+0000 7f3eb3529640 1 -- 172.21.15.16:0/2557416656 shutdown_connections 2024-04-25T10:11:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.750+0000 7f3eb3529640 1 --2- 172.21.15.16:0/2557416656 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3e9006b4d0 0x7f3e9006d990 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.750+0000 7f3eb3529640 1 --2- 172.21.15.16:0/2557416656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eac15ba10 0x7f3eac147f80 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.750+0000 7f3eb3529640 1 --2- 172.21.15.16:0/2557416656 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3eac099f80 0x7f3eac147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.750+0000 7f3eb3529640 1 --2- 172.21.15.16:0/2557416656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3eac06c1f0 0x7f3eac0af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.750+0000 7f3eb3529640 1 -- 172.21.15.16:0/2557416656 >> 172.21.15.16:0/2557416656 conn(0x7f3eac0b32f0 msgr2=0x7f3eac15ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.750+0000 7f3eb3529640 1 -- 172.21.15.16:0/2557416656 shutdown_connections 2024-04-25T10:11:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.750+0000 7f3eb3529640 1 -- 172.21.15.16:0/2557416656 wait complete. 2024-04-25T10:11:54.765 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-25T10:11:54.901 INFO:teuthology.orchestra.run.smithi186.stdout:{"message":"","return_code":0} 2024-04-25T10:11:54.901 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:11:54.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:11:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.999+0000 7ff3fa6b3640 1 Processor -- start 2024-04-25T10:11:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:54.999+0000 7ff3fa6b3640 1 -- start start 2024-04-25T10:11:55.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.000+0000 7ff3fa6b3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff3f40cef40 0x7ff3f40b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.000+0000 7ff3fa6b3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff3f41618d0 0x7ff3f40d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.000+0000 7ff3fa6b3640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f4162040 con 0x7ff3f41610b0 2024-04-25T10:11:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.000+0000 7ff3fa6b3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f40d1ea0 con 0x7ff3f41618d0 2024-04-25T10:11:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.000+0000 7ff3fa6b3640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f4162320 con 0x7ff3f40cef40 2024-04-25T10:11:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.000+0000 7ff3f8c50640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff3f41618d0 0x7ff3f40d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.000+0000 7ff3f8c50640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff3f41618d0 0x7ff3f40d2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49362/0 (socket says 172.21.15.16:49362) 2024-04-25T10:11:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.000+0000 7ff3f8c50640 1 -- 172.21.15.16:0/28659815 learned_addr learned my addr 172.21.15.16:0/28659815 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.000+0000 7ff3f37fe640 1 --2- 172.21.15.16:0/28659815 >> v2:172.21.15.186:3300/0 conn(0x7ff3f40cef40 0x7ff3f40b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.000+0000 7ff3f8c50640 1 -- 172.21.15.16:0/28659815 >> v2:172.21.15.186:3300/0 conn(0x7ff3f40cef40 msgr2=0x7ff3f40b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.000+0000 7ff3f8c50640 1 --2- 172.21.15.16:0/28659815 >> v2:172.21.15.186:3300/0 conn(0x7ff3f40cef40 0x7ff3f40b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.000+0000 7ff3f8c50640 1 -- 172.21.15.16:0/28659815 >> v1:172.21.15.186:6789/0 conn(0x7ff3f41610b0 legacy=0x7ff3f40ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.000+0000 7ff3f8c50640 1 -- 172.21.15.16:0/28659815 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f4162600 con 0x7ff3f41618d0 2024-04-25T10:11:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.001+0000 7ff3f8c50640 1 --2- 172.21.15.16:0/28659815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f41618d0 0x7ff3f40d2610 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff3f406d320 tx=0x7ff3ec02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59008b588177f2f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.001+0000 7ff3f2ffd640 1 -- 172.21.15.16:0/28659815 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec036070 con 0x7ff3f41618d0 2024-04-25T10:11:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.001+0000 7ff3f2ffd640 1 -- 172.21.15.16:0/28659815 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ec031600 con 0x7ff3f41618d0 2024-04-25T10:11:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.001+0000 7ff3f2ffd640 1 -- 172.21.15.16:0/28659815 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec0319d0 con 0x7ff3f41618d0 2024-04-25T10:11:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.001+0000 7ff3fa6b3640 1 -- 172.21.15.16:0/28659815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f41618d0 msgr2=0x7ff3f40d2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.001+0000 7ff3fa6b3640 1 --2- 172.21.15.16:0/28659815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f41618d0 0x7ff3f40d2610 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff3f406d320 tx=0x7ff3ec02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.002+0000 7ff3fa6b3640 1 -- 172.21.15.16:0/28659815 shutdown_connections 2024-04-25T10:11:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.002+0000 7ff3fa6b3640 1 --2- 172.21.15.16:0/28659815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f41618d0 0x7ff3f40d2610 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.002+0000 7ff3fa6b3640 1 --2- 172.21.15.16:0/28659815 >> v2:172.21.15.186:3300/0 conn(0x7ff3f40cef40 0x7ff3f40b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.002+0000 7ff3fa6b3640 1 -- 172.21.15.16:0/28659815 >> 172.21.15.16:0/28659815 conn(0x7ff3f40b5570 msgr2=0x7ff3f40b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.002+0000 7ff3fa6b3640 1 -- 172.21.15.16:0/28659815 shutdown_connections 2024-04-25T10:11:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.002+0000 7ff3fa6b3640 1 -- 172.21.15.16:0/28659815 wait complete. 2024-04-25T10:11:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.003+0000 7ff3fa6b3640 1 Processor -- start 2024-04-25T10:11:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.003+0000 7ff3fa6b3640 1 -- start start 2024-04-25T10:11:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3fa6b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3f40cef40 0x7ff3f40cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3fa6b3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3f41610b0 0x7ff3f40c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3fa6b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f41618d0 0x7ff3f41f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3fa6b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f4161d40 con 0x7ff3f41618d0 2024-04-25T10:11:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3fa6b3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f40b6c00 con 0x7ff3f41610b0 2024-04-25T10:11:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3fa6b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f4161490 con 0x7ff3f40cef40 2024-04-25T10:11:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3f8c50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f41618d0 0x7ff3f41f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3f8c50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f41618d0 0x7ff3f41f3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49372/0 (socket says 172.21.15.16:49372) 2024-04-25T10:11:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3f8c50640 1 -- 172.21.15.16:0/1151737925 learned_addr learned my addr 172.21.15.16:0/1151737925 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3f37fe640 1 --2- 172.21.15.16:0/1151737925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3f41610b0 0x7ff3f40c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3f3fff640 1 --2- 172.21.15.16:0/1151737925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3f40cef40 0x7ff3f40cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3f8c50640 1 -- 172.21.15.16:0/1151737925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3f40cef40 msgr2=0x7ff3f40cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3f8c50640 1 --2- 172.21.15.16:0/1151737925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3f40cef40 0x7ff3f40cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3f8c50640 1 -- 172.21.15.16:0/1151737925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3f41610b0 msgr2=0x7ff3f40c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3f8c50640 1 --2- 172.21.15.16:0/1151737925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3f41610b0 0x7ff3f40c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3f8c50640 1 -- 172.21.15.16:0/1151737925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f4158be0 con 0x7ff3f41618d0 2024-04-25T10:11:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3f8c50640 1 --2- 172.21.15.16:0/1151737925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f41618d0 0x7ff3f41f3890 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec02ac90 tx=0x7ff3ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3f0ff9640 1 -- 172.21.15.16:0/1151737925 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec03f020 con 0x7ff3f41618d0 2024-04-25T10:11:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3f0ff9640 1 -- 172.21.15.16:0/1151737925 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ec037040 con 0x7ff3f41618d0 2024-04-25T10:11:55.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.004+0000 7ff3fa6b3640 1 -- 172.21.15.16:0/1151737925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f41f4330 con 0x7ff3f41618d0 2024-04-25T10:11:55.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.005+0000 7ff3f0ff9640 1 -- 172.21.15.16:0/1151737925 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec036070 con 0x7ff3f41618d0 2024-04-25T10:11:55.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.005+0000 7ff3fa6b3640 1 -- 172.21.15.16:0/1151737925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f4158e80 con 0x7ff3f41618d0 2024-04-25T10:11:55.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.006+0000 7ff3fa6b3640 1 -- 172.21.15.16:0/1151737925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f40750b0 con 0x7ff3f41618d0 2024-04-25T10:11:55.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.006+0000 7ff3f0ff9640 1 -- 172.21.15.16:0/1151737925 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3ec042020 con 0x7ff3f41618d0 2024-04-25T10:11:55.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.007+0000 7ff3f0ff9640 1 --2- 172.21.15.16:0/1151737925 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff3e006b4d0 0x7ff3e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.007+0000 7ff3f0ff9640 1 -- 172.21.15.16:0/1151737925 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff3ec0aa530 con 0x7ff3f41618d0 2024-04-25T10:11:55.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.007+0000 7ff3f3fff640 1 --2- 172.21.15.16:0/1151737925 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff3e006b4d0 0x7ff3e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.007+0000 7ff3f3fff640 1 --2- 172.21.15.16:0/1151737925 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff3e006b4d0 0x7ff3e006d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8001780 tx=0x7ff3e8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.011+0000 7ff3f0ff9640 1 -- 172.21.15.16:0/1151737925 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3ec049030 con 0x7ff3f41618d0 2024-04-25T10:11:55.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.230+0000 7ff3fa6b3640 1 -- 172.21.15.16:0/1151737925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff3f40d1770 con 0x7ff3f41618d0 2024-04-25T10:11:55.235 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-04-25T10:11:55.230+0000 7ff13f9bb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:11:55.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.232+0000 7ff3f0ff9640 1 -- 172.21.15.16:0/1151737925 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff3ec078d00 con 0x7ff3f41618d0 2024-04-25T10:11:55.236 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:11:55.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.234+0000 7ff3fa6b3640 1 -- 172.21.15.16:0/1151737925 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff3e006b4d0 msgr2=0x7ff3e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.234+0000 7ff3fa6b3640 1 --2- 172.21.15.16:0/1151737925 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff3e006b4d0 0x7ff3e006d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8001780 tx=0x7ff3e8009040 comp rx=0 tx=0).stop 2024-04-25T10:11:55.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.234+0000 7ff3fa6b3640 1 -- 172.21.15.16:0/1151737925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f41618d0 msgr2=0x7ff3f41f3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.234+0000 7ff3fa6b3640 1 --2- 172.21.15.16:0/1151737925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f41618d0 0x7ff3f41f3890 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec02ac90 tx=0x7ff3ec004a40 comp rx=0 tx=0).stop 2024-04-25T10:11:55.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.234+0000 7ff3fa6b3640 1 -- 172.21.15.16:0/1151737925 shutdown_connections 2024-04-25T10:11:55.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.234+0000 7ff3fa6b3640 1 --2- 172.21.15.16:0/1151737925 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff3e006b4d0 0x7ff3e006d990 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.234+0000 7ff3fa6b3640 1 --2- 172.21.15.16:0/1151737925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f41618d0 0x7ff3f41f3890 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.234+0000 7ff3fa6b3640 1 --2- 172.21.15.16:0/1151737925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3f41610b0 0x7ff3f40c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.234+0000 7ff3fa6b3640 1 --2- 172.21.15.16:0/1151737925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3f40cef40 0x7ff3f40cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.234+0000 7ff3fa6b3640 1 -- 172.21.15.16:0/1151737925 >> 172.21.15.16:0/1151737925 conn(0x7ff3f40b5570 msgr2=0x7ff3f415df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.235+0000 7ff3fa6b3640 1 -- 172.21.15.16:0/1151737925 shutdown_connections 2024-04-25T10:11:55.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.235+0000 7ff3fa6b3640 1 -- 172.21.15.16:0/1151737925 wait complete. 2024-04-25T10:11:55.242 INFO:tasks.ceph.mds.d.smithi186.stderr: -2799> 2024-04-25T10:11:20.392+0000 7f98024a4640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-25T10:11:55.242 INFO:tasks.ceph.mds.d.smithi186.stderr: -2637> 2024-04-25T10:11:20.576+0000 7f98024a4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-04-25T10:11:10.936139+0000 1=1+0) n(v0 rc2024-04-25T10:11:11.052137+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x55cf190d0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:11:10.936139+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:11:11.052137+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-25T10:11:55.243 INFO:tasks.ceph.mds.d.smithi186.stderr: -2272> 2024-04-25T10:11:21.802+0000 7f98024a4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv18 ap=3 DIRTYPARENT f(v0 m2024-04-25T10:11:10.936139+0000 1=1+0)->f(v1 m2024-04-25T10:11:10.936139+0000) n(v0 rc2024-04-25T10:11:11.052137+0000 2=1+1)->n(v0 rc2024-04-25T10:11:11.052137+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55cf190d0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:11:10.936139+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:11:11.052137+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-25T10:11:55.243 INFO:tasks.ceph.mds.d.smithi186.stderr: -2057> 2024-04-25T10:11:22.639+0000 7f98064ac640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:11:55.250 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-25T10:11:55.268 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2024-04-25T10:11:55.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.284+0000 7ff262817740 1 -- start start 2024-04-25T10:11:55.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.285+0000 7ff262817740 1 Processor -- start 2024-04-25T10:11:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.285+0000 7ff262817740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592317eec00 0x55923185e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.285+0000 7ff262817740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592317ef800 0x55923185eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.285+0000 7ff262817740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592317efc00 0x55923185f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.285+0000 7ff262817740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592318ca600 con 0x5592317eec00 2024-04-25T10:11:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.285+0000 7ff262817740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592318ca480 con 0x5592317efc00 2024-04-25T10:11:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.285+0000 7ff262817740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592318ca300 con 0x5592317ef800 2024-04-25T10:11:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.285+0000 7ff26054f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592317eec00 0x55923185e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.285+0000 7ff26054f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592317eec00 0x55923185e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49380/0 (socket says 172.21.15.16:49380) 2024-04-25T10:11:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.285+0000 7ff26054f640 1 -- 172.21.15.16:0/1979124729 learned_addr learned my addr 172.21.15.16:0/1979124729 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.285+0000 7ff25fd4e640 1 --2- 172.21.15.16:0/1979124729 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592317ef800 0x55923185eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.285+0000 7ff260d50640 1 --2- 172.21.15.16:0/1979124729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592317efc00 0x55923185f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.286+0000 7ff26054f640 1 -- 172.21.15.16:0/1979124729 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592317ef800 msgr2=0x55923185eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.286+0000 7ff26054f640 1 --2- 172.21.15.16:0/1979124729 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592317ef800 0x55923185eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.286+0000 7ff26054f640 1 -- 172.21.15.16:0/1979124729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592317efc00 msgr2=0x55923185f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.286+0000 7ff26054f640 1 --2- 172.21.15.16:0/1979124729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592317efc00 0x55923185f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.286+0000 7ff26054f640 1 -- 172.21.15.16:0/1979124729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55923181b680 con 0x5592317eec00 2024-04-25T10:11:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.286+0000 7ff26054f640 1 --2- 172.21.15.16:0/1979124729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592317eec00 0x55923185e580 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x5592325f9d40 tx=0x5592326b6540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345829ea3eefd647 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.286+0000 7ff25dd4a640 1 -- 172.21.15.16:0/1979124729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559231814680 con 0x5592317eec00 2024-04-25T10:11:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.286+0000 7ff25dd4a640 1 -- 172.21.15.16:0/1979124729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5592318381c0 con 0x5592317eec00 2024-04-25T10:11:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.286+0000 7ff25dd4a640 1 -- 172.21.15.16:0/1979124729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559231814b60 con 0x5592317eec00 2024-04-25T10:11:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.286+0000 7ff262817740 1 -- 172.21.15.16:0/1979124729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55923181b860 con 0x5592317eec00 2024-04-25T10:11:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.286+0000 7ff262817740 1 -- 172.21.15.16:0/1979124729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55923181b2c0 con 0x5592317eec00 2024-04-25T10:11:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.286+0000 7ff262817740 4 waiting for MDS map... 2024-04-25T10:11:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.286+0000 7ff25dd4a640 1 -- 172.21.15.16:0/1979124729 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x5592326cc600 con 0x5592317eec00 2024-04-25T10:11:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.286+0000 7ff25dd4a640 1 -- 172.21.15.16:0/1979124729 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 59) v1 ==== 2723+0+0 (secure 0 0 0) 0x559231847180 con 0x5592317eec00 2024-04-25T10:11:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.286+0000 7ff262817740 4 Got MDS map 59 2024-04-25T10:11:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.286+0000 7ff262817740 4 datascan.main: Using metadata pool output 2024-04-25T10:11:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.286+0000 7ff262817740 4 datascan.main: connecting to RADOS... 2024-04-25T10:11:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.287+0000 7ff262817740 1 Processor -- start 2024-04-25T10:11:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.287+0000 7ff262817740 1 -- start start 2024-04-25T10:11:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.287+0000 7ff262817740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592326b3000 0x55923185f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.287+0000 7ff262817740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592326b3400 0x55923185fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.287+0000 7ff262817740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592326b3800 0x559231860100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.287+0000 7ff262817740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592318ca180 con 0x5592326b3000 2024-04-25T10:11:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.287+0000 7ff262817740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923184e480 con 0x5592326b3800 2024-04-25T10:11:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.287+0000 7ff262817740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55923184e900 con 0x5592326b3400 2024-04-25T10:11:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.288+0000 7ff260d50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592326b3000 0x55923185f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.288+0000 7ff260d50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592326b3000 0x55923185f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49392/0 (socket says 172.21.15.16:49392) 2024-04-25T10:11:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.288+0000 7ff260d50640 1 -- 172.21.15.16:0/2547179723 learned_addr learned my addr 172.21.15.16:0/2547179723 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.288+0000 7ff25fd4e640 1 --2- 172.21.15.16:0/2547179723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592326b3800 0x559231860100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.288+0000 7ff26054f640 1 --2- 172.21.15.16:0/2547179723 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592326b3400 0x55923185fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.288+0000 7ff260d50640 1 -- 172.21.15.16:0/2547179723 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592326b3400 msgr2=0x55923185fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.288+0000 7ff260d50640 1 --2- 172.21.15.16:0/2547179723 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592326b3400 0x55923185fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.288+0000 7ff260d50640 1 -- 172.21.15.16:0/2547179723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592326b3800 msgr2=0x559231860100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.288+0000 7ff260d50640 1 --2- 172.21.15.16:0/2547179723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592326b3800 0x559231860100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.288+0000 7ff260d50640 1 -- 172.21.15.16:0/2547179723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55923181b4a0 con 0x5592326b3000 2024-04-25T10:11:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.288+0000 7ff25fd4e640 1 --2- 172.21.15.16:0/2547179723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592326b3800 0x559231860100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.288+0000 7ff260d50640 1 --2- 172.21.15.16:0/2547179723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592326b3000 0x55923185f600 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x5592326bf290 tx=0x559232738000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6fcfbe19a2a38b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.288+0000 7ff25bd46640 1 -- 172.21.15.16:0/2547179723 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559231814680 con 0x5592326b3000 2024-04-25T10:11:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.288+0000 7ff25bd46640 1 -- 172.21.15.16:0/2547179723 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559231838540 con 0x5592326b3000 2024-04-25T10:11:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.288+0000 7ff25bd46640 1 -- 172.21.15.16:0/2547179723 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559231814b60 con 0x5592326b3000 2024-04-25T10:11:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.289+0000 7ff262817740 1 -- 172.21.15.16:0/2547179723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592326b3000 msgr2=0x55923185f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.289+0000 7ff262817740 1 --2- 172.21.15.16:0/2547179723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592326b3000 0x55923185f600 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x5592326bf290 tx=0x559232738000 comp rx=0 tx=0).stop 2024-04-25T10:11:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.289+0000 7ff262817740 1 -- 172.21.15.16:0/2547179723 shutdown_connections 2024-04-25T10:11:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.289+0000 7ff262817740 1 --2- 172.21.15.16:0/2547179723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592326b3800 0x559231860100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.289+0000 7ff262817740 1 --2- 172.21.15.16:0/2547179723 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592326b3400 0x55923185fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.289+0000 7ff262817740 1 --2- 172.21.15.16:0/2547179723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592326b3000 0x55923185f600 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.289+0000 7ff262817740 1 -- 172.21.15.16:0/2547179723 >> 172.21.15.16:0/2547179723 conn(0x5592326b2c00 msgr2=0x5592318d4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.289+0000 7ff262817740 1 -- 172.21.15.16:0/2547179723 shutdown_connections 2024-04-25T10:11:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.289+0000 7ff262817740 1 -- 172.21.15.16:0/2547179723 wait complete. 2024-04-25T10:11:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.289+0000 7ff262817740 1 Processor -- start 2024-04-25T10:11:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.289+0000 7ff262817740 1 -- start start 2024-04-25T10:11:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff262817740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592326b3000 0x559231861180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff262817740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592326b3400 0x559231860c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff262817740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592326b3800 0x559231860680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff262817740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592318cb380 con 0x5592326b3000 2024-04-25T10:11:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff262817740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592318ca180 con 0x5592326b3800 2024-04-25T10:11:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff262817740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592318cb500 con 0x5592326b3400 2024-04-25T10:11:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff260d50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592326b3000 0x559231861180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff260d50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592326b3000 0x559231861180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49406/0 (socket says 172.21.15.16:49406) 2024-04-25T10:11:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 learned_addr learned my addr 172.21.15.16:0/41348775 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff25fd4e640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592326b3800 0x559231860680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff26054f640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592326b3400 0x559231860c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592326b3400 msgr2=0x559231860c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff260d50640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592326b3400 0x559231860c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592326b3800 msgr2=0x559231860680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff260d50640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592326b3800 0x559231860680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55923181b680 con 0x5592326b3000 2024-04-25T10:11:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff260d50640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592326b3000 0x559231861180 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x5592326bff50 tx=0x5592326b64e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff25a543640 1 -- 172.21.15.16:0/41348775 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5592318144e0 con 0x5592326b3000 2024-04-25T10:11:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff25a543640 1 -- 172.21.15.16:0/41348775 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559231838a80 con 0x5592326b3000 2024-04-25T10:11:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.290+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55923181b860 con 0x5592326b3000 2024-04-25T10:11:55.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.291+0000 7ff25a543640 1 -- 172.21.15.16:0/41348775 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559231814680 con 0x5592326b3000 2024-04-25T10:11:55.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.291+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55923181b0e0 con 0x5592326b3000 2024-04-25T10:11:55.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.291+0000 7ff262817740 4 datascan.init: resolving metadata pool 12 2024-04-25T10:11:55.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.292+0000 7ff25a543640 1 -- 172.21.15.16:0/41348775 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x55923273ec00 con 0x5592326b3000 2024-04-25T10:11:55.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.292+0000 7ff25a543640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55923273a800 0x559231861700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.292+0000 7ff25a543640 1 -- 172.21.15.16:0/41348775 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x55923273ca00 con 0x5592326b3000 2024-04-25T10:11:55.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.292+0000 7ff262817740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:11:55.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.292+0000 7ff262817740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-04-25T10:11:55.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.292+0000 7ff262817740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:11:55.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.292+0000 7ff262817740 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55923273ac00 0x559231860100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.292+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x55923273b000 con 0x55923273ac00 2024-04-25T10:11:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.292+0000 7ff26054f640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55923273a800 0x559231861700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.292+0000 7ff25fd4e640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55923273ac00 0x559231860100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.293+0000 7ff26054f640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55923273a800 0x559231861700 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x5592326bf0b0 tx=0x5592326b6060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.293+0000 7ff25fd4e640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55923273ac00 0x559231860100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.293+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.1 v2:172.21.15.16:6808/1916704159 1 ==== osd_op_reply(1 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x55923273ac00 2024-04-25T10:11:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.293+0000 7ff25b545640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x5592325c3400 0x559231b7a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.293+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:2 13.20 13:04000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325c3000 con 0x5592325c3400 2024-04-25T10:11:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.293+0000 7ff260d50640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x5592325c3400 0x559231b7a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.293+0000 7ff260d50640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x5592325c3400 0x559231b7a000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.294+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.2 v2:172.21.15.16:6800/2094356376 1 ==== osd_op_reply(2 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x5592325c3400 2024-04-25T10:11:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.294+0000 7ff25bd46640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x5592325c2c00 0x559231b7a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.294+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:3 13.10 13:08000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x55923273b000 con 0x5592325c2c00 2024-04-25T10:11:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.294+0000 7ff26054f640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x5592325c2c00 0x559231b7a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.294+0000 7ff26054f640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x5592325c2c00 0x559231b7a580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.297+0000 7ff26054f640 1 -- 172.21.15.16:0/41348775 <== osd.7 v2:172.21.15.186:6808/1627307043 1 ==== osd_op_reply(3 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc780 con 0x5592325c2c00 2024-04-25T10:11:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.297+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:4 13.30 13:0c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325c2800 con 0x5592325c3400 2024-04-25T10:11:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.297+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.2 v2:172.21.15.16:6800/2094356376 2 ==== osd_op_reply(4 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x5592325c3400 2024-04-25T10:11:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.297+0000 7ff25bd46640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x559231b82000 0x559231b7ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.297+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:5 13.8 13:10000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b82400 con 0x559231b82000 2024-04-25T10:11:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.297+0000 7ff25fd4e640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x559231b82000 0x559231b7ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.297+0000 7ff25fd4e640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x559231b82000 0x559231b7ab00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.299+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.5 v2:172.21.15.186:6800/1827108421 1 ==== osd_op_reply(5 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x559231b82000 2024-04-25T10:11:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.299+0000 7ff25b545640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x559231b82c00 0x559231b7b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.299+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:6 13.28 13:14000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b83000 con 0x559231b82c00 2024-04-25T10:11:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.300+0000 7ff260d50640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x559231b82c00 0x559231b7b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.300+0000 7ff260d50640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x559231b82c00 0x559231b7b600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.302+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.6 v2:172.21.15.186:6816/1123185664 1 ==== osd_op_reply(6 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x559231b82c00 2024-04-25T10:11:55.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.302+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:7 13.18 13:18000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b82800 con 0x55923273ac00 2024-04-25T10:11:55.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.302+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.1 v2:172.21.15.16:6808/1916704159 2 ==== osd_op_reply(7 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x55923273ac00 2024-04-25T10:11:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.302+0000 7ff25b545640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x559231b83400 0x559231b7bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.302+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:8 13.38 13:1c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b83800 con 0x559231b83400 2024-04-25T10:11:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.302+0000 7ff26054f640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x559231b83400 0x559231b7bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.302+0000 7ff26054f640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x559231b83400 0x559231b7bb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.304+0000 7ff26054f640 1 -- 172.21.15.16:0/41348775 <== osd.0 v2:172.21.15.16:6816/2167750256 1 ==== osd_op_reply(8 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc780 con 0x559231b83400 2024-04-25T10:11:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.304+0000 7ff25bd46640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x5592325e0000 0x559231b7b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.304+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:9 13.4 13:20000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e0400 con 0x5592325e0000 2024-04-25T10:11:55.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.304+0000 7ff25fd4e640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x5592325e0000 0x559231b7b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.305+0000 7ff25fd4e640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x5592325e0000 0x559231b7b080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.307+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.3 v2:172.21.15.16:6824/256577261 1 ==== osd_op_reply(9 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x5592325e0000 2024-04-25T10:11:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.307+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:10 13.24 13:24000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b83c00 con 0x55923273ac00 2024-04-25T10:11:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.307+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.1 v2:172.21.15.16:6808/1916704159 3 ==== osd_op_reply(10 [pgnls start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x55923273ac00 2024-04-25T10:11:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.307+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:11 13.14 13:28000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e0800 con 0x559231b82c00 2024-04-25T10:11:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.308+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.6 v2:172.21.15.186:6816/1123185664 2 ==== osd_op_reply(11 [pgnls start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x559231b82c00 2024-04-25T10:11:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.308+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:12 13.34 13:2c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e0800 con 0x559231b83400 2024-04-25T10:11:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.308+0000 7ff26054f640 1 -- 172.21.15.16:0/41348775 <== osd.0 v2:172.21.15.16:6816/2167750256 2 ==== osd_op_reply(12 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc780 con 0x559231b83400 2024-04-25T10:11:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.308+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:13 13.c 13:30000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e0c00 con 0x559231b82c00 2024-04-25T10:11:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.308+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.6 v2:172.21.15.186:6816/1123185664 3 ==== osd_op_reply(13 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x559231b82c00 2024-04-25T10:11:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.308+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:14 13.2c 13:34000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e1000 con 0x5592325c2c00 2024-04-25T10:11:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.308+0000 7ff26054f640 1 -- 172.21.15.16:0/41348775 <== osd.7 v2:172.21.15.186:6808/1627307043 2 ==== osd_op_reply(14 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc780 con 0x5592325c2c00 2024-04-25T10:11:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.308+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:15 13.1c 13:38000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e1400 con 0x559231b83400 2024-04-25T10:11:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.309+0000 7ff26054f640 1 -- 172.21.15.16:0/41348775 <== osd.0 v2:172.21.15.16:6816/2167750256 3 ==== osd_op_reply(15 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc780 con 0x559231b83400 2024-04-25T10:11:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.309+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:16 13.3c 13:3c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b83000 con 0x559231b82000 2024-04-25T10:11:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.309+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.5 v2:172.21.15.186:6800/1827108421 2 ==== osd_op_reply(16 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x559231b82000 2024-04-25T10:11:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.309+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:17 13.2 13:40000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e1800 con 0x55923273ac00 2024-04-25T10:11:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.309+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.1 v2:172.21.15.16:6808/1916704159 4 ==== osd_op_reply(17 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x55923273ac00 2024-04-25T10:11:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.309+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:18 13.22 13:44000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325c3000 con 0x5592325c2c00 2024-04-25T10:11:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.310+0000 7ff26054f640 1 -- 172.21.15.16:0/41348775 <== osd.7 v2:172.21.15.186:6808/1627307043 3 ==== osd_op_reply(18 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc780 con 0x5592325c2c00 2024-04-25T10:11:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.310+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:19 13.12 13:48000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e1c00 con 0x559231b82c00 2024-04-25T10:11:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.310+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.6 v2:172.21.15.186:6816/1123185664 4 ==== osd_op_reply(19 [pgnls start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x559231b82c00 2024-04-25T10:11:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.310+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:20 13.32 13:4c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325c2800 con 0x5592325c2c00 2024-04-25T10:11:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.310+0000 7ff26054f640 1 -- 172.21.15.16:0/41348775 <== osd.7 v2:172.21.15.186:6808/1627307043 4 ==== osd_op_reply(20 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc780 con 0x5592325c2c00 2024-04-25T10:11:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.310+0000 7ff25bd46640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x559231b82400 0x559231b7c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.310+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:21 13.a 13:50000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b82800 con 0x559231b82400 2024-04-25T10:11:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.310+0000 7ff260d50640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x559231b82400 0x559231b7c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.311+0000 7ff260d50640 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x559231b82400 0x559231b7c100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.311+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.4 v2:172.21.15.186:6824/3091472889 1 ==== osd_op_reply(21 [pgnls start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x559231b82400 2024-04-25T10:11:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.311+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:22 13.2a 13:54000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e0400 con 0x5592325c2c00 2024-04-25T10:11:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.311+0000 7ff26054f640 1 -- 172.21.15.16:0/41348775 <== osd.7 v2:172.21.15.186:6808/1627307043 5 ==== osd_op_reply(22 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc780 con 0x5592325c2c00 2024-04-25T10:11:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.311+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:23 13.1a 13:58000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b83c00 con 0x559231b82000 2024-04-25T10:11:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.312+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.5 v2:172.21.15.186:6800/1827108421 3 ==== osd_op_reply(23 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x559231b82000 2024-04-25T10:11:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.312+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:24 13.3a 13:5c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e1800 con 0x5592325c3400 2024-04-25T10:11:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.312+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.2 v2:172.21.15.16:6800/2094356376 3 ==== osd_op_reply(24 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x5592325c3400 2024-04-25T10:11:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.312+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:25 13.6 13:60000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b83000 con 0x5592325e0000 2024-04-25T10:11:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.312+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.3 v2:172.21.15.16:6824/256577261 2 ==== osd_op_reply(25 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x5592325e0000 2024-04-25T10:11:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.312+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:26 13.26 13:64000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e0800 con 0x5592325e0000 2024-04-25T10:11:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.312+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.3 v2:172.21.15.16:6824/256577261 3 ==== osd_op_reply(26 [pgnls start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x5592325e0000 2024-04-25T10:11:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.312+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:27 13.16 13:68000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b83800 con 0x5592325c2c00 2024-04-25T10:11:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.313+0000 7ff26054f640 1 -- 172.21.15.16:0/41348775 <== osd.7 v2:172.21.15.186:6808/1627307043 6 ==== osd_op_reply(27 [pgnls start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc780 con 0x5592325c2c00 2024-04-25T10:11:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.313+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:28 13.36 13:6c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e1000 con 0x5592325c3400 2024-04-25T10:11:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.313+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.2 v2:172.21.15.16:6800/2094356376 4 ==== osd_op_reply(28 [pgnls start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x5592325c3400 2024-04-25T10:11:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.313+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:29 13.e 13:70000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x55923273b000 con 0x559231b83400 2024-04-25T10:11:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.314+0000 7ff26054f640 1 -- 172.21.15.16:0/41348775 <== osd.0 v2:172.21.15.16:6816/2167750256 4 ==== osd_op_reply(29 [pgnls start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc780 con 0x559231b83400 2024-04-25T10:11:55.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.314+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:30 13.2e 13:74000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e1400 con 0x559231b82c00 2024-04-25T10:11:55.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.314+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.6 v2:172.21.15.186:6816/1123185664 5 ==== osd_op_reply(30 [pgnls start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x559231b82c00 2024-04-25T10:11:55.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.314+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:31 13.1e 13:78000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e0c00 con 0x559231b83400 2024-04-25T10:11:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.314+0000 7ff26054f640 1 -- 172.21.15.16:0/41348775 <== osd.0 v2:172.21.15.16:6816/2167750256 5 ==== osd_op_reply(31 [pgnls start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc780 con 0x559231b83400 2024-04-25T10:11:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.314+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:32 13.3e 13:7c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e0400 con 0x559231b82400 2024-04-25T10:11:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.315+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.4 v2:172.21.15.186:6824/3091472889 2 ==== osd_op_reply(32 [pgnls start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x559231b82400 2024-04-25T10:11:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.315+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:33 13.1 13:80000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e1c00 con 0x55923273ac00 2024-04-25T10:11:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.315+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.1 v2:172.21.15.16:6808/1916704159 5 ==== osd_op_reply(33 [pgnls start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x55923273ac00 2024-04-25T10:11:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.315+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:34 13.21 13:84000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e0c00 con 0x5592325c3400 2024-04-25T10:11:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.315+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.2 v2:172.21.15.16:6800/2094356376 5 ==== osd_op_reply(34 [pgnls start_epoch 88 out=49b] v86'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x5592325c3400 2024-04-25T10:11:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.315+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:35 13.11 13:88000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e1800 con 0x559231b82c00 2024-04-25T10:11:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.316+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.6 v2:172.21.15.186:6816/1123185664 6 ==== osd_op_reply(35 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x559231b82c00 2024-04-25T10:11:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.316+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:36 13.31 13:8c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x55923273b000 con 0x559231b82400 2024-04-25T10:11:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.316+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.4 v2:172.21.15.186:6824/3091472889 3 ==== osd_op_reply(36 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x559231b82400 2024-04-25T10:11:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.316+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:37 13.9 13:90000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e1000 con 0x559231b83400 2024-04-25T10:11:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.316+0000 7ff26054f640 1 -- 172.21.15.16:0/41348775 <== osd.0 v2:172.21.15.16:6816/2167750256 6 ==== osd_op_reply(37 [pgnls start_epoch 88 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc780 con 0x559231b83400 2024-04-25T10:11:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.316+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:38 13.29 13:94000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b83800 con 0x559231b83400 2024-04-25T10:11:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.316+0000 7ff26054f640 1 -- 172.21.15.16:0/41348775 <== osd.0 v2:172.21.15.16:6816/2167750256 7 ==== osd_op_reply(38 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc780 con 0x559231b83400 2024-04-25T10:11:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.317+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:39 13.19 13:98000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b82800 con 0x5592325c2c00 2024-04-25T10:11:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.317+0000 7ff26054f640 1 -- 172.21.15.16:0/41348775 <== osd.7 v2:172.21.15.186:6808/1627307043 7 ==== osd_op_reply(39 [pgnls start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc780 con 0x5592325c2c00 2024-04-25T10:11:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.317+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:40 13.39 13:9c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325c3000 con 0x559231b82400 2024-04-25T10:11:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.317+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.4 v2:172.21.15.186:6824/3091472889 4 ==== osd_op_reply(40 [pgnls start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x559231b82400 2024-04-25T10:11:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.317+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:41 13.5 13:a0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325c3000 con 0x5592325c3400 2024-04-25T10:11:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.317+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.2 v2:172.21.15.16:6800/2094356376 6 ==== osd_op_reply(41 [pgnls start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x5592325c3400 2024-04-25T10:11:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.317+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:42 13.25 13:a4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325c2800 con 0x5592325e0000 2024-04-25T10:11:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.318+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.3 v2:172.21.15.16:6824/256577261 4 ==== osd_op_reply(42 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x5592325e0000 2024-04-25T10:11:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.318+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:43 13.15 13:a8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x55923273b000 con 0x5592325e0000 2024-04-25T10:11:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.318+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.3 v2:172.21.15.16:6824/256577261 5 ==== osd_op_reply(43 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x5592325e0000 2024-04-25T10:11:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.318+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:44 13.35 13:ac000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b9c000 con 0x559231b82c00 2024-04-25T10:11:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.318+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.6 v2:172.21.15.186:6816/1123185664 7 ==== osd_op_reply(44 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x559231b82c00 2024-04-25T10:11:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.318+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:45 13.d 13:b0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e1400 con 0x559231b83400 2024-04-25T10:11:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.318+0000 7ff26054f640 1 -- 172.21.15.16:0/41348775 <== osd.0 v2:172.21.15.16:6816/2167750256 8 ==== osd_op_reply(45 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc780 con 0x559231b83400 2024-04-25T10:11:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.319+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:46 13.2d 13:b4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b9c400 con 0x5592325c2c00 2024-04-25T10:11:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.319+0000 7ff26054f640 1 -- 172.21.15.16:0/41348775 <== osd.7 v2:172.21.15.186:6808/1627307043 8 ==== osd_op_reply(46 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc780 con 0x5592325c2c00 2024-04-25T10:11:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.319+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:47 13.1d 13:b8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e0400 con 0x55923273ac00 2024-04-25T10:11:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.319+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.1 v2:172.21.15.16:6808/1916704159 6 ==== osd_op_reply(47 [pgnls start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x55923273ac00 2024-04-25T10:11:55.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.319+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:48 13.3d 13:bc000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b9c800 con 0x5592325c3400 2024-04-25T10:11:55.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.319+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.2 v2:172.21.15.16:6800/2094356376 7 ==== osd_op_reply(48 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x5592325c3400 2024-04-25T10:11:55.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.319+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:49 13.3 13:c0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e0c00 con 0x559231b82c00 2024-04-25T10:11:55.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.320+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.6 v2:172.21.15.186:6816/1123185664 8 ==== osd_op_reply(49 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x559231b82c00 2024-04-25T10:11:55.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.320+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:50 13.23 13:c4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b9cc00 con 0x559231b82000 2024-04-25T10:11:55.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.320+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.5 v2:172.21.15.186:6800/1827108421 4 ==== osd_op_reply(50 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x559231b82000 2024-04-25T10:11:55.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.320+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:51 13.13 13:c8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e1800 con 0x55923273ac00 2024-04-25T10:11:55.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.320+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.1 v2:172.21.15.16:6808/1916704159 7 ==== osd_op_reply(51 [pgnls start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x55923273ac00 2024-04-25T10:11:55.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.320+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:52 13.33 13:cc000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b9d000 con 0x5592325e0000 2024-04-25T10:11:55.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.321+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.3 v2:172.21.15.16:6824/256577261 6 ==== osd_op_reply(52 [pgnls start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x5592325e0000 2024-04-25T10:11:55.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.321+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:53 13.b 13:d0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e0800 con 0x5592325e0000 2024-04-25T10:11:55.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.321+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.3 v2:172.21.15.16:6824/256577261 7 ==== osd_op_reply(53 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x5592325e0000 2024-04-25T10:11:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.321+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:54 13.2b 13:d4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b9d400 con 0x559231b82000 2024-04-25T10:11:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.321+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.5 v2:172.21.15.186:6800/1827108421 5 ==== osd_op_reply(54 [pgnls start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x559231b82000 2024-04-25T10:11:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.321+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:55 13.1b 13:d8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e1c00 con 0x5592325c3400 2024-04-25T10:11:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.322+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.2 v2:172.21.15.16:6800/2094356376 8 ==== osd_op_reply(55 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x5592325c3400 2024-04-25T10:11:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.322+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:56 13.3b 13:dc000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b9d800 con 0x5592325e0000 2024-04-25T10:11:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.322+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.3 v2:172.21.15.16:6824/256577261 8 ==== osd_op_reply(56 [pgnls start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x5592325e0000 2024-04-25T10:11:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.322+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:57 13.7 13:e0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e0400 con 0x5592325e0000 2024-04-25T10:11:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.322+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.3 v2:172.21.15.16:6824/256577261 9 ==== osd_op_reply(57 [pgnls start_epoch 88 out=81b] v86'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5592318dc000 con 0x5592325e0000 2024-04-25T10:11:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.322+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:58 13.7 13:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x55923273b400 con 0x5592325e0000 2024-04-25T10:11:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.322+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.3 v2:172.21.15.16:6824/256577261 10 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5592318dc000 con 0x5592325e0000 2024-04-25T10:11:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.322+0000 7ff262817740 10 datascan.operator(): handling object 1099511627777.0 2024-04-25T10:11:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.322+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:59 13.7 13:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v9 -- 0x55923273b800 con 0x5592325e0000 2024-04-25T10:11:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.324+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.3 v2:172.21.15.16:6824/256577261 11 ==== osd_op_reply(59 10000000001.00000000 [call] v88'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5592318dc000 con 0x5592325e0000 2024-04-25T10:11:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.324+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:60 13.27 13:e4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x55923273bc00 con 0x559231b82c00 2024-04-25T10:11:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.324+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.6 v2:172.21.15.186:6816/1123185664 9 ==== osd_op_reply(60 [pgnls start_epoch 88 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x559231b82c00 2024-04-25T10:11:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.324+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:61 13.17 13:e8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x55923273b000 con 0x55923273ac00 2024-04-25T10:11:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.324+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.1 v2:172.21.15.16:6808/1916704159 8 ==== osd_op_reply(61 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x55923273ac00 2024-04-25T10:11:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.324+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:62 13.37 13:ec000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b9dc00 con 0x55923273ac00 2024-04-25T10:11:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.325+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.1 v2:172.21.15.16:6808/1916704159 9 ==== osd_op_reply(62 [pgnls start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x55923273ac00 2024-04-25T10:11:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.325+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:63 13.f 13:f0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325c2800 con 0x5592325c3400 2024-04-25T10:11:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.325+0000 7ff260d50640 1 -- 172.21.15.16:0/41348775 <== osd.2 v2:172.21.15.16:6800/2094356376 9 ==== osd_op_reply(63 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc500 con 0x5592325c3400 2024-04-25T10:11:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.325+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:64 13.2f 13:f4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x5592325e0c00 con 0x559231b83400 2024-04-25T10:11:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.325+0000 7ff26054f640 1 -- 172.21.15.16:0/41348775 <== osd.0 v2:172.21.15.16:6816/2167750256 9 ==== osd_op_reply(64 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc780 con 0x559231b83400 2024-04-25T10:11:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.325+0000 7ff25bd46640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:65 13.1f 13:f8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b83c00 con 0x55923273ac00 2024-04-25T10:11:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.325+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.1 v2:172.21.15.16:6808/1916704159 10 ==== osd_op_reply(65 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x55923273ac00 2024-04-25T10:11:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.325+0000 7ff25b545640 1 -- 172.21.15.16:0/41348775 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:66 13.3f 13:fc000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x559231b9c000 con 0x55923273ac00 2024-04-25T10:11:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 <== osd.1 v2:172.21.15.16:6808/1916704159 11 ==== osd_op_reply(66 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5592318dc000 con 0x55923273ac00 2024-04-25T10:11:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x559231b83400 msgr2=0x559231b7bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x559231b83400 0x559231b7bb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55923273ac00 msgr2=0x559231860100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55923273ac00 0x559231860100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x5592325c3400 msgr2=0x559231b7a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x5592325c3400 0x559231b7a000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x5592325e0000 msgr2=0x559231b7b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x5592325e0000 0x559231b7b080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x559231b82400 msgr2=0x559231b7c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x559231b82400 0x559231b7c100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x559231b82000 msgr2=0x559231b7ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 reap_dead start 2024-04-25T10:11:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x559231b82000 0x559231b7ab00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 reap_dead start 2024-04-25T10:11:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x559231b82c00 msgr2=0x559231b7b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x559231b82c00 0x559231b7b600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x5592325c2c00 msgr2=0x559231b7a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x5592325c2c00 0x559231b7a580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff25fd4e640 1 -- 172.21.15.16:0/41348775 reap_dead start 2024-04-25T10:11:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55923273a800 msgr2=0x559231861700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55923273a800 0x559231861700 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x5592326bf0b0 tx=0x5592326b6060 comp rx=0 tx=0).stop 2024-04-25T10:11:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592326b3000 msgr2=0x559231861180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592326b3000 0x559231861180 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x5592326bff50 tx=0x5592326b64e0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 shutdown_connections 2024-04-25T10:11:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55923273a800 0x559231861700 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 --2- 172.21.15.16:0/41348775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592326b3000 0x559231861180 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 >> 172.21.15.16:0/41348775 conn(0x5592326b2c00 msgr2=0x5592318d4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 shutdown_connections 2024-04-25T10:11:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.326+0000 7ff262817740 1 -- 172.21.15.16:0/41348775 wait complete. 2024-04-25T10:11:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.327+0000 7ff262817740 1 -- 172.21.15.16:0/1979124729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592317eec00 msgr2=0x55923185e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.327+0000 7ff262817740 1 --2- 172.21.15.16:0/1979124729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592317eec00 0x55923185e580 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x5592325f9d40 tx=0x5592326b6540 comp rx=0 tx=0).stop 2024-04-25T10:11:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.327+0000 7ff262817740 1 -- 172.21.15.16:0/1979124729 shutdown_connections 2024-04-25T10:11:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.327+0000 7ff262817740 1 --2- 172.21.15.16:0/1979124729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5592317efc00 0x55923185f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.327+0000 7ff262817740 1 --2- 172.21.15.16:0/1979124729 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5592317ef800 0x55923185eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.327+0000 7ff262817740 1 --2- 172.21.15.16:0/1979124729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592317eec00 0x55923185e580 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.327+0000 7ff262817740 1 -- 172.21.15.16:0/1979124729 >> 172.21.15.16:0/1979124729 conn(0x5592317ef000 msgr2=0x5592318d5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.327+0000 7ff262817740 1 -- 172.21.15.16:0/1979124729 shutdown_connections 2024-04-25T10:11:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.327+0000 7ff262817740 1 -- 172.21.15.16:0/1979124729 wait complete. 2024-04-25T10:11:55.340 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.089285, result: 2024-04-25T10:11:55.340 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-04-25T10:11:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.409+0000 7ffb5be2d740 1 -- start start 2024-04-25T10:11:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.409+0000 7ffb5be2d740 1 Processor -- start 2024-04-25T10:11:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.410+0000 7ffb5be2d740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a04ba4c00 0x560a04c15b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.410+0000 7ffb5be2d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a04ba5800 0x560a04c15600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.410+0000 7ffb5be2d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a04ba5c00 0x560a04c15080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.410+0000 7ffb5be2d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a04d18180 con 0x560a04ba5c00 2024-04-25T10:11:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.410+0000 7ffb5be2d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a04c04900 con 0x560a04ba5800 2024-04-25T10:11:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.410+0000 7ffb5be2d740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a04c04480 con 0x560a04ba4c00 2024-04-25T10:11:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.410+0000 7ffb59f91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a04ba5c00 0x560a04c15080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.410+0000 7ffb59f91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a04ba5c00 0x560a04c15080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49426/0 (socket says 172.21.15.16:49426) 2024-04-25T10:11:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.410+0000 7ffb59f91640 1 -- 172.21.15.16:0/2128746244 learned_addr learned my addr 172.21.15.16:0/2128746244 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.410+0000 7ffb59790640 1 --2- 172.21.15.16:0/2128746244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a04ba4c00 0x560a04c15b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.410+0000 7ffb58f8f640 1 --2- 172.21.15.16:0/2128746244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a04ba5800 0x560a04c15600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.411+0000 7ffb59f91640 1 -- 172.21.15.16:0/2128746244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a04ba4c00 msgr2=0x560a04c15b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.411+0000 7ffb59f91640 1 --2- 172.21.15.16:0/2128746244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a04ba4c00 0x560a04c15b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.411+0000 7ffb59f91640 1 -- 172.21.15.16:0/2128746244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a04ba5800 msgr2=0x560a04c15600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.411+0000 7ffb59f91640 1 --2- 172.21.15.16:0/2128746244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a04ba5800 0x560a04c15600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.411+0000 7ffb59f91640 1 -- 172.21.15.16:0/2128746244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a04bd1680 con 0x560a04ba5c00 2024-04-25T10:11:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.411+0000 7ffb59f91640 1 --2- 172.21.15.16:0/2128746244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a04ba5c00 0x560a04c15080 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x560a059bdc50 tx=0x560a05976300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1930ef6b52aba05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.411+0000 7ffb56f8b640 1 -- 172.21.15.16:0/2128746244 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560a04bca680 con 0x560a04ba5c00 2024-04-25T10:11:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.411+0000 7ffb56f8b640 1 -- 172.21.15.16:0/2128746244 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560a04bee1c0 con 0x560a04ba5c00 2024-04-25T10:11:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.411+0000 7ffb56f8b640 1 -- 172.21.15.16:0/2128746244 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560a04bcab60 con 0x560a04ba5c00 2024-04-25T10:11:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.411+0000 7ffb5be2d740 1 -- 172.21.15.16:0/2128746244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a04bd1860 con 0x560a04ba5c00 2024-04-25T10:11:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.411+0000 7ffb5be2d740 1 -- 172.21.15.16:0/2128746244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x560a04bd12c0 con 0x560a04ba5c00 2024-04-25T10:11:55.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.411+0000 7ffb5be2d740 4 waiting for MDS map... 2024-04-25T10:11:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.411+0000 7ffb56f8b640 1 -- 172.21.15.16:0/2128746244 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x560a05a72600 con 0x560a04ba5c00 2024-04-25T10:11:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.411+0000 7ffb56f8b640 1 -- 172.21.15.16:0/2128746244 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 59) v1 ==== 2723+0+0 (secure 0 0 0) 0x560a04bfd180 con 0x560a04ba5c00 2024-04-25T10:11:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.411+0000 7ffb5be2d740 4 Got MDS map 59 2024-04-25T10:11:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.411+0000 7ffb5be2d740 4 datascan.main: Using metadata pool output 2024-04-25T10:11:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.411+0000 7ffb5be2d740 4 datascan.main: connecting to RADOS... 2024-04-25T10:11:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.412+0000 7ffb5be2d740 1 Processor -- start 2024-04-25T10:11:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.412+0000 7ffb5be2d740 1 -- start start 2024-04-25T10:11:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.412+0000 7ffb5be2d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0597d000 0x560a04c14b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.412+0000 7ffb5be2d740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0597d400 0x560a04c14580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.412+0000 7ffb5be2d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a0597d800 0x560a04c16100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.412+0000 7ffb5be2d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a04c04180 con 0x560a0597d800 2024-04-25T10:11:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.412+0000 7ffb5be2d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a04d18d80 con 0x560a0597d000 2024-04-25T10:11:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.412+0000 7ffb5be2d740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a04d18f00 con 0x560a0597d400 2024-04-25T10:11:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.413+0000 7ffb58f8f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a0597d800 0x560a04c16100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.413+0000 7ffb58f8f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a0597d800 0x560a04c16100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49442/0 (socket says 172.21.15.16:49442) 2024-04-25T10:11:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.413+0000 7ffb58f8f640 1 -- 172.21.15.16:0/2796605367 learned_addr learned my addr 172.21.15.16:0/2796605367 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.413+0000 7ffb59790640 1 --2- 172.21.15.16:0/2796605367 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0597d400 0x560a04c14580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.413+0000 7ffb58f8f640 1 --2- 172.21.15.16:0/2796605367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0597d000 0x560a04c14b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.413+0000 7ffb58f8f640 1 -- 172.21.15.16:0/2796605367 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0597d400 msgr2=0x560a04c14580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.413+0000 7ffb58f8f640 1 --2- 172.21.15.16:0/2796605367 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0597d400 0x560a04c14580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.413+0000 7ffb58f8f640 1 -- 172.21.15.16:0/2796605367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0597d000 msgr2=0x560a04c14b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.413+0000 7ffb58f8f640 1 --2- 172.21.15.16:0/2796605367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0597d000 0x560a04c14b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.413+0000 7ffb58f8f640 1 -- 172.21.15.16:0/2796605367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a04bd14a0 con 0x560a0597d800 2024-04-25T10:11:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.413+0000 7ffb58f8f640 1 --2- 172.21.15.16:0/2796605367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a0597d800 0x560a04c16100 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x560a05a66a80 tx=0x560a05ad60c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de1f62a044968ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.413+0000 7ffb54f87640 1 -- 172.21.15.16:0/2796605367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560a04bca680 con 0x560a0597d800 2024-04-25T10:11:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.413+0000 7ffb54f87640 1 -- 172.21.15.16:0/2796605367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560a04beee00 con 0x560a0597d800 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.413+0000 7ffb54f87640 1 -- 172.21.15.16:0/2796605367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560a04bcab60 con 0x560a0597d800 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.414+0000 7ffb5be2d740 1 -- 172.21.15.16:0/2796605367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a0597d800 msgr2=0x560a04c16100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.414+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/2796605367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a0597d800 0x560a04c16100 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x560a05a66a80 tx=0x560a05ad60c0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.414+0000 7ffb5be2d740 1 -- 172.21.15.16:0/2796605367 shutdown_connections 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.414+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/2796605367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a0597d800 0x560a04c16100 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.414+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/2796605367 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0597d400 0x560a04c14580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.414+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/2796605367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0597d000 0x560a04c14b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.414+0000 7ffb5be2d740 1 -- 172.21.15.16:0/2796605367 >> 172.21.15.16:0/2796605367 conn(0x560a0597cc00 msgr2=0x560a04c8d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.414+0000 7ffb5be2d740 1 -- 172.21.15.16:0/2796605367 shutdown_connections 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.414+0000 7ffb5be2d740 1 -- 172.21.15.16:0/2796605367 wait complete. 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.414+0000 7ffb5be2d740 1 Processor -- start 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.414+0000 7ffb5be2d740 1 -- start start 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb5be2d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0597d000 0x560a04c14b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb5be2d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a0597d400 0x560a04c14580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb5be2d740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0597d800 0x560a04c16100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb5be2d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a04d19200 con 0x560a0597d400 2024-04-25T10:11:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb5be2d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a04d19080 con 0x560a0597d000 2024-04-25T10:11:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb5be2d740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a04d19380 con 0x560a0597d800 2024-04-25T10:11:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb59790640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a0597d400 0x560a04c14580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb59790640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a0597d400 0x560a04c14580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49456/0 (socket says 172.21.15.16:49456) 2024-04-25T10:11:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 learned_addr learned my addr 172.21.15.16:0/3926332243 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb58f8f640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0597d800 0x560a04c16100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb58f8f640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0597d000 0x560a04c14b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0597d800 msgr2=0x560a04c16100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb59790640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0597d800 0x560a04c16100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0597d000 msgr2=0x560a04c14b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb59790640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0597d000 0x560a04c14b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a04bd1680 con 0x560a0597d400 2024-04-25T10:11:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb58f8f640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0597d800 0x560a04c16100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb59790640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a0597d400 0x560a04c14580 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x560a059bd2f0 tx=0x560a05976120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb53784640 1 -- 172.21.15.16:0/3926332243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560a04bca9c0 con 0x560a0597d400 2024-04-25T10:11:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb53784640 1 -- 172.21.15.16:0/3926332243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560a04bee540 con 0x560a0597d400 2024-04-25T10:11:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.415+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560a04bd1860 con 0x560a0597d400 2024-04-25T10:11:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.416+0000 7ffb53784640 1 -- 172.21.15.16:0/3926332243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560a04bca680 con 0x560a0597d400 2024-04-25T10:11:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.416+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a04bd10e0 con 0x560a0597d400 2024-04-25T10:11:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.416+0000 7ffb5be2d740 4 datascan.init: resolving metadata pool 12 2024-04-25T10:11:55.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.417+0000 7ffb53784640 1 -- 172.21.15.16:0/3926332243 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x560a05ae8c00 con 0x560a0597d400 2024-04-25T10:11:55.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.417+0000 7ffb53784640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x560a05ae0800 0x560a04f10000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.417+0000 7ffb53784640 1 -- 172.21.15.16:0/3926332243 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x560a05ae6c00 con 0x560a0597d400 2024-04-25T10:11:55.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.417+0000 7ffb5be2d740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:11:55.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.417+0000 7ffb5be2d740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-04-25T10:11:55.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.417+0000 7ffb5be2d740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:11:55.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.417+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x560a05ae0c00 0x560a04c16680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.417+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a05ae1000 con 0x560a05ae0c00 2024-04-25T10:11:55.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.417+0000 7ffb58f8f640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x560a05ae0c00 0x560a04c16680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.417+0000 7ffb58f8f640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x560a05ae0800 0x560a04f10000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.418+0000 7ffb58f8f640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x560a05ae0c00 0x560a04c16680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.418+0000 7ffb58f8f640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x560a05ae0800 0x560a04f10000 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x560a05a66bd0 tx=0x560a05ad6300 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.418+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.0 v2:172.21.15.16:6816/2167750256 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv24 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x560a04c92000 con 0x560a05ae0c00 2024-04-25T10:11:55.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.418+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x560a05ae1400 0x560a04c16c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.418+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:2 12.1f 12:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a05ae1800 con 0x560a05ae1400 2024-04-25T10:11:55.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.418+0000 7ffb59f91640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x560a05ae1400 0x560a04c16c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.418+0000 7ffb59f91640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x560a05ae1400 0x560a04c16c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.419+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.419+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x560a05ae1c00 0x560a04c17180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.419+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:3 13.0 13:00000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f26000 con 0x560a05ae1c00 2024-04-25T10:11:55.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.419+0000 7ffb59790640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x560a05ae1c00 0x560a04c17180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.419+0000 7ffb59790640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x560a05ae1c00 0x560a04c17180 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.419+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.1 v2:172.21.15.16:6808/1916704159 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a05ae1c00 2024-04-25T10:11:55.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.419+0000 7ffb54786640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x560a04f27800 0x560a04f10580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.419+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:4 13.20 13:04000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f27c00 con 0x560a04f27800 2024-04-25T10:11:55.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.420+0000 7ffb58f8f640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x560a04f27800 0x560a04f10580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.420+0000 7ffb58f8f640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x560a04f27800 0x560a04f10580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.420+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.2 v2:172.21.15.16:6800/2094356376 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f27800 2024-04-25T10:11:55.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.420+0000 7ffb54f87640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x560a04f36000 0x560a04f10b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.420+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:5 13.10 13:08000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f36400 con 0x560a04f36000 2024-04-25T10:11:55.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.420+0000 7ffb59f91640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x560a04f36000 0x560a04f10b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.421+0000 7ffb59f91640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x560a04f36000 0x560a04f10b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.421+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.7 v2:172.21.15.186:6808/1627307043 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a04f36000 2024-04-25T10:11:55.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.421+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:6 13.30 13:0c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f36c00 con 0x560a04f27800 2024-04-25T10:11:55.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.421+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.2 v2:172.21.15.16:6800/2094356376 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f27800 2024-04-25T10:11:55.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.421+0000 7ffb54f87640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x560a04f36800 0x560a04f11080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.421+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:7 13.8 13:10000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f27c00 con 0x560a04f36800 2024-04-25T10:11:55.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.421+0000 7ffb59790640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x560a04f36800 0x560a04f11080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.422+0000 7ffb59790640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x560a04f36800 0x560a04f11080 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.422+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.5 v2:172.21.15.186:6800/1827108421 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a04f36800 2024-04-25T10:11:55.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.422+0000 7ffb54786640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x560a04f37000 0x560a04f11b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.422+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:8 13.28 13:14000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f37400 con 0x560a04f37000 2024-04-25T10:11:55.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.422+0000 7ffb58f8f640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x560a04f37000 0x560a04f11b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.422+0000 7ffb58f8f640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x560a04f37000 0x560a04f11b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.423+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.423+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:9 13.18 13:18000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a05ae0400 con 0x560a05ae1c00 2024-04-25T10:11:55.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.423+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.1 v2:172.21.15.16:6808/1916704159 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a05ae1c00 2024-04-25T10:11:55.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.423+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:10 13.38 13:1c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f36c00 con 0x560a05ae0c00 2024-04-25T10:11:55.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.423+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.0 v2:172.21.15.16:6816/2167750256 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a05ae0c00 2024-04-25T10:11:55.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.423+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:11 13.4 13:20000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a05ae1000 con 0x560a05ae1400 2024-04-25T10:11:55.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.424+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.424+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:12 13.24 13:24000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f37800 con 0x560a05ae1c00 2024-04-25T10:11:55.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.424+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.1 v2:172.21.15.16:6808/1916704159 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a05ae1c00 2024-04-25T10:11:55.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.424+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:13 13.14 13:28000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a0597c800 con 0x560a04f37000 2024-04-25T10:11:55.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.424+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 2 ==== osd_op_reply(13 [pgnls-filter start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.424+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:14 13.34 13:2c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f37c00 con 0x560a05ae0c00 2024-04-25T10:11:55.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.424+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.0 v2:172.21.15.16:6816/2167750256 3 ==== osd_op_reply(14 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a05ae0c00 2024-04-25T10:11:55.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.424+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:15 13.c 13:30000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f26000 con 0x560a04f37000 2024-04-25T10:11:55.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.425+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.425+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:16 13.2c 13:34000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f37400 con 0x560a04f36000 2024-04-25T10:11:55.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.425+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.7 v2:172.21.15.186:6808/1627307043 2 ==== osd_op_reply(16 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a04f36000 2024-04-25T10:11:55.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.425+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:17 13.1c 13:38000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f27c00 con 0x560a05ae0c00 2024-04-25T10:11:55.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.425+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.0 v2:172.21.15.16:6816/2167750256 4 ==== osd_op_reply(17 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a05ae0c00 2024-04-25T10:11:55.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.425+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:18 13.3c 13:3c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f36c00 con 0x560a04f36800 2024-04-25T10:11:55.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.425+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.5 v2:172.21.15.186:6800/1827108421 2 ==== osd_op_reply(18 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a04f36800 2024-04-25T10:11:55.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.425+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:19 13.2 13:40000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04c81400 con 0x560a05ae1c00 2024-04-25T10:11:55.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.426+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.1 v2:172.21.15.16:6808/1916704159 4 ==== osd_op_reply(19 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a05ae1c00 2024-04-25T10:11:55.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.426+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:20 13.22 13:44000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a0597c800 con 0x560a04f36000 2024-04-25T10:11:55.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.426+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.7 v2:172.21.15.186:6808/1627307043 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a04f36000 2024-04-25T10:11:55.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.426+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:21 13.12 13:48000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a05ae0400 con 0x560a04f37000 2024-04-25T10:11:55.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.426+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 4 ==== osd_op_reply(21 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.426+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:22 13.32 13:4c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a0597c400 con 0x560a04f36000 2024-04-25T10:11:55.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.427+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.7 v2:172.21.15.186:6808/1627307043 4 ==== osd_op_reply(22 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a04f36000 2024-04-25T10:11:55.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.427+0000 7ffb54f87640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x560a05ae1800 0x560a04f11600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.427+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:23 13.a 13:50000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f37c00 con 0x560a05ae1800 2024-04-25T10:11:55.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.427+0000 7ffb59f91640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x560a05ae1800 0x560a04f11600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.427+0000 7ffb59f91640 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x560a05ae1800 0x560a04f11600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.428+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.4 v2:172.21.15.186:6824/3091472889 1 ==== osd_op_reply(23 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1800 2024-04-25T10:11:55.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.428+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:24 13.2a 13:54000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f26000 con 0x560a04f36000 2024-04-25T10:11:55.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.428+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.7 v2:172.21.15.186:6808/1627307043 5 ==== osd_op_reply(24 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a04f36000 2024-04-25T10:11:55.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.428+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:25 13.1a 13:58000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f37800 con 0x560a04f36800 2024-04-25T10:11:55.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.428+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.5 v2:172.21.15.186:6800/1827108421 3 ==== osd_op_reply(25 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a04f36800 2024-04-25T10:11:55.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.428+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:26 13.3a 13:5c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a05ae1000 con 0x560a04f27800 2024-04-25T10:11:55.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.428+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.2 v2:172.21.15.16:6800/2094356376 3 ==== osd_op_reply(26 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f27800 2024-04-25T10:11:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.428+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:27 13.6 13:60000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f36400 con 0x560a05ae1400 2024-04-25T10:11:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.429+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 3 ==== osd_op_reply(27 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.429+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:28 13.26 13:64000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f37400 con 0x560a05ae1400 2024-04-25T10:11:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.429+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 4 ==== osd_op_reply(28 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.429+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:29 13.16 13:68000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4a000 con 0x560a04f36000 2024-04-25T10:11:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.429+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.7 v2:172.21.15.186:6808/1627307043 6 ==== osd_op_reply(29 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a04f36000 2024-04-25T10:11:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.429+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:30 13.36 13:6c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f37c00 con 0x560a04f27800 2024-04-25T10:11:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.430+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.2 v2:172.21.15.16:6800/2094356376 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f27800 2024-04-25T10:11:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.430+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:31 13.e 13:70000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4a400 con 0x560a05ae0c00 2024-04-25T10:11:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.430+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.0 v2:172.21.15.16:6816/2167750256 5 ==== osd_op_reply(31 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a05ae0c00 2024-04-25T10:11:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.430+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:32 13.2e 13:74000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f36400 con 0x560a04f37000 2024-04-25T10:11:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.430+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.430+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:33 13.1e 13:78000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4a800 con 0x560a05ae0c00 2024-04-25T10:11:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.430+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.0 v2:172.21.15.16:6816/2167750256 6 ==== osd_op_reply(33 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a05ae0c00 2024-04-25T10:11:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.430+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:34 13.3e 13:7c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f26000 con 0x560a05ae1800 2024-04-25T10:11:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.431+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.4 v2:172.21.15.186:6824/3091472889 2 ==== osd_op_reply(34 [pgnls-filter start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1800 2024-04-25T10:11:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.431+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:35 13.1 13:80000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4ac00 con 0x560a05ae1c00 2024-04-25T10:11:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.431+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.1 v2:172.21.15.16:6808/1916704159 5 ==== osd_op_reply(35 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a05ae1c00 2024-04-25T10:11:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.431+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:36 13.21 13:84000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a0597c800 con 0x560a04f27800 2024-04-25T10:11:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.431+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.2 v2:172.21.15.16:6800/2094356376 5 ==== osd_op_reply(36 [pgnls-filter start_epoch 88 out=49b] v86'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f27800 2024-04-25T10:11:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.431+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:37 13.11 13:88000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4b000 con 0x560a04f37000 2024-04-25T10:11:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.431+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 6 ==== osd_op_reply(37 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.431+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:38 13.31 13:8c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a0597c400 con 0x560a05ae1800 2024-04-25T10:11:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.432+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.4 v2:172.21.15.186:6824/3091472889 3 ==== osd_op_reply(38 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1800 2024-04-25T10:11:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.432+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:39 13.9 13:90000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4b400 con 0x560a05ae0c00 2024-04-25T10:11:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.432+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.0 v2:172.21.15.16:6816/2167750256 7 ==== osd_op_reply(39 [pgnls-filter start_epoch 88 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a05ae0c00 2024-04-25T10:11:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.432+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:40 13.29 13:94000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4a400 con 0x560a05ae0c00 2024-04-25T10:11:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.432+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.0 v2:172.21.15.16:6816/2167750256 8 ==== osd_op_reply(40 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a05ae0c00 2024-04-25T10:11:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.432+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:41 13.19 13:98000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4e000 con 0x560a04f36000 2024-04-25T10:11:55.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.433+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.7 v2:172.21.15.186:6808/1627307043 7 ==== osd_op_reply(41 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a04f36000 2024-04-25T10:11:55.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.433+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:42 13.39 13:9c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4b800 con 0x560a05ae1800 2024-04-25T10:11:55.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.433+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.4 v2:172.21.15.186:6824/3091472889 4 ==== osd_op_reply(42 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1800 2024-04-25T10:11:55.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.433+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:43 13.5 13:a0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4e400 con 0x560a04f27800 2024-04-25T10:11:55.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.433+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.2 v2:172.21.15.16:6800/2094356376 6 ==== osd_op_reply(43 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f27800 2024-04-25T10:11:55.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.433+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:44 13.25 13:a4000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4bc00 con 0x560a05ae1400 2024-04-25T10:11:55.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.434+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.434+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:45 13.15 13:a8000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4e800 con 0x560a05ae1400 2024-04-25T10:11:55.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.434+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 6 ==== osd_op_reply(45 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.434+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:46 13.35 13:ac000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f37c00 con 0x560a04f37000 2024-04-25T10:11:55.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.434+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 7 ==== osd_op_reply(46 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.434+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:47 13.d 13:b0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4ec00 con 0x560a05ae0c00 2024-04-25T10:11:55.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.435+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.0 v2:172.21.15.16:6816/2167750256 9 ==== osd_op_reply(47 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a05ae0c00 2024-04-25T10:11:55.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.435+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:48 13.2d 13:b4000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f36400 con 0x560a04f36000 2024-04-25T10:11:55.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.435+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.7 v2:172.21.15.186:6808/1627307043 8 ==== osd_op_reply(48 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a04f36000 2024-04-25T10:11:55.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.435+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:49 13.1d 13:b8000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4f000 con 0x560a05ae1c00 2024-04-25T10:11:55.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.435+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.1 v2:172.21.15.16:6808/1916704159 6 ==== osd_op_reply(49 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a05ae1c00 2024-04-25T10:11:55.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.435+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:50 13.3d 13:bc000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a05ae0400 con 0x560a04f27800 2024-04-25T10:11:55.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.435+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.2 v2:172.21.15.16:6800/2094356376 7 ==== osd_op_reply(50 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f27800 2024-04-25T10:11:55.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.435+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:51 13.3 13:c0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4f400 con 0x560a04f37000 2024-04-25T10:11:55.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.436+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 8 ==== osd_op_reply(51 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.436+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:52 13.23 13:c4000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a05ae1000 con 0x560a04f36800 2024-04-25T10:11:55.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.436+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.5 v2:172.21.15.186:6800/1827108421 4 ==== osd_op_reply(52 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a04f36800 2024-04-25T10:11:55.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.436+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:53 13.13 13:c8000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4f800 con 0x560a05ae1c00 2024-04-25T10:11:55.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.436+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.1 v2:172.21.15.16:6808/1916704159 7 ==== osd_op_reply(53 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a05ae1c00 2024-04-25T10:11:55.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.436+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:54 13.33 13:cc000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f27c00 con 0x560a05ae1400 2024-04-25T10:11:55.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.437+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 7 ==== osd_op_reply(54 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.437+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:55 13.b 13:d0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a0597c800 con 0x560a05ae1400 2024-04-25T10:11:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.437+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 8 ==== osd_op_reply(55 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.437+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:56 13.2b 13:d4000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4f000 con 0x560a04f36800 2024-04-25T10:11:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.437+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.5 v2:172.21.15.186:6800/1827108421 5 ==== osd_op_reply(56 [pgnls-filter start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a04f36800 2024-04-25T10:11:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.437+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:57 13.1b 13:d8000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f26000 con 0x560a04f27800 2024-04-25T10:11:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.438+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.2 v2:172.21.15.16:6800/2094356376 8 ==== osd_op_reply(57 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f27800 2024-04-25T10:11:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.438+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:58 13.3b 13:dc000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4e400 con 0x560a05ae1400 2024-04-25T10:11:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.438+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 9 ==== osd_op_reply(58 [pgnls-filter start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.438+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:59 13.7 13:e0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f36c00 con 0x560a05ae1400 2024-04-25T10:11:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.438+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 10 ==== osd_op_reply(59 [pgnls-filter start_epoch 88 out=81b] v88'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.438+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:60 13.0 13:00000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f26400 con 0x560a05ae1c00 2024-04-25T10:11:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.438+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.1 v2:172.21.15.16:6808/1916704159 8 ==== osd_op_reply(60 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a05ae1c00 2024-04-25T10:11:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.438+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:61 13.20 13:04000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f37800 con 0x560a04f27800 2024-04-25T10:11:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.439+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.2 v2:172.21.15.16:6800/2094356376 9 ==== osd_op_reply(61 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f27800 2024-04-25T10:11:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.439+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:62 13.10 13:08000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4e000 con 0x560a04f36000 2024-04-25T10:11:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.439+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.7 v2:172.21.15.186:6808/1627307043 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a04f36000 2024-04-25T10:11:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.439+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:63 13.30 13:0c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f37c00 con 0x560a04f27800 2024-04-25T10:11:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.439+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.2 v2:172.21.15.16:6800/2094356376 10 ==== osd_op_reply(63 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f27800 2024-04-25T10:11:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.439+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:64 13.8 13:10000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4fc00 con 0x560a04f36800 2024-04-25T10:11:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.440+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.5 v2:172.21.15.186:6800/1827108421 6 ==== osd_op_reply(64 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a04f36800 2024-04-25T10:11:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.440+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:65 13.28 13:14000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f37400 con 0x560a04f37000 2024-04-25T10:11:55.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.440+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 9 ==== osd_op_reply(65 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.440+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:66 13.18 13:18000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a0597c400 con 0x560a05ae1c00 2024-04-25T10:11:55.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.440+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.1 v2:172.21.15.16:6808/1916704159 9 ==== osd_op_reply(66 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a05ae1c00 2024-04-25T10:11:55.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.440+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:67 13.38 13:1c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a05ae1000 con 0x560a05ae0c00 2024-04-25T10:11:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.441+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.0 v2:172.21.15.16:6816/2167750256 10 ==== osd_op_reply(67 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a05ae0c00 2024-04-25T10:11:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.441+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:68 13.4 13:20000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a05ae0400 con 0x560a05ae1400 2024-04-25T10:11:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.441+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 11 ==== osd_op_reply(68 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.441+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:69 13.24 13:24000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04c81400 con 0x560a05ae1c00 2024-04-25T10:11:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.441+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.1 v2:172.21.15.16:6808/1916704159 10 ==== osd_op_reply(69 [pgnls-filter start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a05ae1c00 2024-04-25T10:11:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.441+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:70 13.14 13:28000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a05ae1000 con 0x560a04f37000 2024-04-25T10:11:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.441+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 10 ==== osd_op_reply(70 [pgnls-filter start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.441+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:71 13.34 13:2c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a05ae0400 con 0x560a05ae0c00 2024-04-25T10:11:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.442+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.0 v2:172.21.15.16:6816/2167750256 11 ==== osd_op_reply(71 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a05ae0c00 2024-04-25T10:11:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.442+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:72 13.c 13:30000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f37800 con 0x560a04f37000 2024-04-25T10:11:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.442+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.442+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:73 13.2c 13:34000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4e800 con 0x560a04f36000 2024-04-25T10:11:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.442+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.7 v2:172.21.15.186:6808/1627307043 10 ==== osd_op_reply(73 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a04f36000 2024-04-25T10:11:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.442+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:74 13.1c 13:38000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f37c00 con 0x560a05ae0c00 2024-04-25T10:11:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.442+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.0 v2:172.21.15.16:6816/2167750256 12 ==== osd_op_reply(74 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a05ae0c00 2024-04-25T10:11:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.442+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:75 13.3c 13:3c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4e400 con 0x560a04f36800 2024-04-25T10:11:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.443+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.5 v2:172.21.15.186:6800/1827108421 7 ==== osd_op_reply(75 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a04f36800 2024-04-25T10:11:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.443+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:76 13.2 13:40000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f37400 con 0x560a05ae1c00 2024-04-25T10:11:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.443+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.1 v2:172.21.15.16:6808/1916704159 11 ==== osd_op_reply(76 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a05ae1c00 2024-04-25T10:11:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.443+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:77 13.22 13:44000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4e000 con 0x560a04f36000 2024-04-25T10:11:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.443+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.7 v2:172.21.15.186:6808/1627307043 11 ==== osd_op_reply(77 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a04f36000 2024-04-25T10:11:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.443+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:78 13.12 13:48000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f26000 con 0x560a04f37000 2024-04-25T10:11:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.444+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 12 ==== osd_op_reply(78 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.444+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:79 13.32 13:4c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f36400 con 0x560a04f36000 2024-04-25T10:11:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.444+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.7 v2:172.21.15.186:6808/1627307043 12 ==== osd_op_reply(79 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a04f36000 2024-04-25T10:11:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.444+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:80 13.a 13:50000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4ec00 con 0x560a05ae1800 2024-04-25T10:11:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.444+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.4 v2:172.21.15.186:6824/3091472889 5 ==== osd_op_reply(80 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1800 2024-04-25T10:11:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.444+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:81 13.2a 13:54000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f36c00 con 0x560a04f36000 2024-04-25T10:11:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.445+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.7 v2:172.21.15.186:6808/1627307043 13 ==== osd_op_reply(81 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a04f36000 2024-04-25T10:11:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.445+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:82 13.1a 13:58000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4f400 con 0x560a04f36800 2024-04-25T10:11:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.445+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.5 v2:172.21.15.186:6800/1827108421 8 ==== osd_op_reply(82 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a04f36800 2024-04-25T10:11:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.445+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:83 13.3a 13:5c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a0597c800 con 0x560a04f27800 2024-04-25T10:11:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.446+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.2 v2:172.21.15.16:6800/2094356376 11 ==== osd_op_reply(83 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f27800 2024-04-25T10:11:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.446+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:84 13.6 13:60000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4f800 con 0x560a05ae1400 2024-04-25T10:11:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.446+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 12 ==== osd_op_reply(84 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.446+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:85 13.26 13:64000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f27c00 con 0x560a05ae1400 2024-04-25T10:11:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.446+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 13 ==== osd_op_reply(85 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.446+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:86 13.16 13:68000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4f000 con 0x560a04f36000 2024-04-25T10:11:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.447+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.7 v2:172.21.15.186:6808/1627307043 14 ==== osd_op_reply(86 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a04f36000 2024-04-25T10:11:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.447+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:87 13.36 13:6c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f5a000 con 0x560a04f27800 2024-04-25T10:11:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.447+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.2 v2:172.21.15.16:6800/2094356376 12 ==== osd_op_reply(87 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f27800 2024-04-25T10:11:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.447+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:88 13.e 13:70000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4fc00 con 0x560a05ae0c00 2024-04-25T10:11:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.447+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.0 v2:172.21.15.16:6816/2167750256 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a05ae0c00 2024-04-25T10:11:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.447+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:89 13.2e 13:74000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f5a400 con 0x560a04f37000 2024-04-25T10:11:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.447+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 13 ==== osd_op_reply(89 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.447+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:90 13.1e 13:78000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4e400 con 0x560a05ae0c00 2024-04-25T10:11:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.448+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.0 v2:172.21.15.16:6816/2167750256 14 ==== osd_op_reply(90 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a05ae0c00 2024-04-25T10:11:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.448+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:91 13.3e 13:7c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f5a800 con 0x560a05ae1800 2024-04-25T10:11:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.448+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.4 v2:172.21.15.186:6824/3091472889 6 ==== osd_op_reply(91 [pgnls-filter start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1800 2024-04-25T10:11:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.448+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:92 13.1 13:80000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04c81400 con 0x560a05ae1c00 2024-04-25T10:11:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.448+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.1 v2:172.21.15.16:6808/1916704159 12 ==== osd_op_reply(92 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a05ae1c00 2024-04-25T10:11:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.448+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:93 13.21 13:84000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f5ac00 con 0x560a04f27800 2024-04-25T10:11:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.448+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.2 v2:172.21.15.16:6800/2094356376 13 ==== osd_op_reply(93 [pgnls-filter start_epoch 88 out=49b] v86'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f27800 2024-04-25T10:11:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.449+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:94 13.11 13:88000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a0597c400 con 0x560a04f37000 2024-04-25T10:11:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.449+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 14 ==== osd_op_reply(94 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.449+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:95 13.31 13:8c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f5b000 con 0x560a05ae1800 2024-04-25T10:11:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.449+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.4 v2:172.21.15.186:6824/3091472889 7 ==== osd_op_reply(95 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1800 2024-04-25T10:11:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.449+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:96 13.9 13:90000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f26400 con 0x560a05ae0c00 2024-04-25T10:11:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.449+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.0 v2:172.21.15.16:6816/2167750256 15 ==== osd_op_reply(96 [pgnls-filter start_epoch 88 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a05ae0c00 2024-04-25T10:11:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.450+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:97 13.29 13:94000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f5b400 con 0x560a05ae0c00 2024-04-25T10:11:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.450+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.0 v2:172.21.15.16:6816/2167750256 16 ==== osd_op_reply(97 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a05ae0c00 2024-04-25T10:11:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.450+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:98 13.19 13:98000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4e800 con 0x560a04f36000 2024-04-25T10:11:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.450+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.7 v2:172.21.15.186:6808/1627307043 15 ==== osd_op_reply(98 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a04f36000 2024-04-25T10:11:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.450+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:99 13.39 13:9c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f5b800 con 0x560a05ae1800 2024-04-25T10:11:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.450+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.4 v2:172.21.15.186:6824/3091472889 8 ==== osd_op_reply(99 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1800 2024-04-25T10:11:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.450+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:100 13.5 13:a0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4e000 con 0x560a04f27800 2024-04-25T10:11:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.451+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.2 v2:172.21.15.16:6800/2094356376 14 ==== osd_op_reply(100 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f27800 2024-04-25T10:11:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.451+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:101 13.25 13:a4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f5bc00 con 0x560a05ae1400 2024-04-25T10:11:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.451+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 14 ==== osd_op_reply(101 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.451+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:102 13.15 13:a8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4ec00 con 0x560a05ae1400 2024-04-25T10:11:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.451+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 15 ==== osd_op_reply(102 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.451+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:103 13.35 13:ac000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f5a800 con 0x560a04f37000 2024-04-25T10:11:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.452+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 15 ==== osd_op_reply(103 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.452+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:104 13.d 13:b0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a0597c800 con 0x560a05ae0c00 2024-04-25T10:11:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.452+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.0 v2:172.21.15.16:6816/2167750256 17 ==== osd_op_reply(104 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a05ae0c00 2024-04-25T10:11:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.452+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:105 13.2d 13:b4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f5a000 con 0x560a04f36000 2024-04-25T10:11:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.452+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.7 v2:172.21.15.186:6808/1627307043 16 ==== osd_op_reply(105 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a04f36000 2024-04-25T10:11:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.452+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:106 13.1d 13:b8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f27c00 con 0x560a05ae1c00 2024-04-25T10:11:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.452+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.1 v2:172.21.15.16:6808/1916704159 13 ==== osd_op_reply(106 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a05ae1c00 2024-04-25T10:11:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.452+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:107 13.3d 13:bc000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f5a400 con 0x560a04f27800 2024-04-25T10:11:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.453+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.2 v2:172.21.15.16:6800/2094356376 15 ==== osd_op_reply(107 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f27800 2024-04-25T10:11:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.453+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:108 13.3 13:c0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f26000 con 0x560a04f37000 2024-04-25T10:11:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.453+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 16 ==== osd_op_reply(108 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.453+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:109 13.23 13:c4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4f800 con 0x560a04f36800 2024-04-25T10:11:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.453+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.5 v2:172.21.15.186:6800/1827108421 9 ==== osd_op_reply(109 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a04f36800 2024-04-25T10:11:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.453+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:110 13.13 13:c8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f36400 con 0x560a05ae1c00 2024-04-25T10:11:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.454+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.1 v2:172.21.15.16:6808/1916704159 14 ==== osd_op_reply(110 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a05ae1c00 2024-04-25T10:11:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.454+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:111 13.33 13:cc000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4f000 con 0x560a05ae1400 2024-04-25T10:11:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.454+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 16 ==== osd_op_reply(111 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.454+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:112 13.b 13:d0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f36c00 con 0x560a05ae1400 2024-04-25T10:11:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.454+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 17 ==== osd_op_reply(112 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.454+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:113 13.2b 13:d4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4fc00 con 0x560a04f36800 2024-04-25T10:11:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.454+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.5 v2:172.21.15.186:6800/1827108421 10 ==== osd_op_reply(113 [pgnls-filter start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a04f36800 2024-04-25T10:11:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.454+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:114 13.1b 13:d8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f37800 con 0x560a04f27800 2024-04-25T10:11:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.455+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.2 v2:172.21.15.16:6800/2094356376 16 ==== osd_op_reply(114 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f27800 2024-04-25T10:11:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.455+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:115 13.3b 13:dc000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f26400 con 0x560a05ae1400 2024-04-25T10:11:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.455+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 18 ==== osd_op_reply(115 [pgnls-filter start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.455+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:116 13.7 13:e0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f37c00 con 0x560a05ae1400 2024-04-25T10:11:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.455+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 19 ==== osd_op_reply(116 [pgnls-filter start_epoch 88 out=81b] v88'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.455+0000 7ffb5be2d740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-04-25T10:11:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.455+0000 7ffb5be2d740 10 datascan.operator(): handling object 10000000001.0 2024-04-25T10:11:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.455+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:117 13.7 13:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f26c00 con 0x560a05ae1400 2024-04-25T10:11:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.456+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 20 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.456+0000 7ffb5be2d740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-04-25T10:11:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.456+0000 7ffb5be2d740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-04-25T10:11:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.456+0000 7ffb5be2d740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-04-25T10:11:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.456+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:118 12.10 12:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f27000 con 0x560a04f37000 2024-04-25T10:11:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.456+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 17 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+59 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.456+0000 7ffb5be2d740 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-04-25T10:11:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.456+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f27400 con 0x560a05ae1400 2024-04-25T10:11:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.456+0000 7ffb59f91640 1 -- 172.21.15.16:0/3926332243 <== osd.3 v2:172.21.15.16:6824/256577261 21 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=581b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+581 (crc 0 0 0) 0x560a04c92500 con 0x560a05ae1400 2024-04-25T10:11:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.456+0000 7ffb5be2d740 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-04-25T10:11:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.456+0000 7ffb5be2d740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-04-25T10:11:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.456+0000 7ffb5be2d740 20 datascan.get_frag_of: frag is * 2024-04-25T10:11:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.456+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:120 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4e800 con 0x560a04f37000 2024-04-25T10:11:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.457+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 18 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.457+0000 7ffb5be2d740 10 datascan.find_or_create_dirfrag: read_version = 17 2024-04-25T10:11:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.457+0000 7ffb5be2d740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-04-25T10:11:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.457+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:121 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4ec00 con 0x560a04f37000 2024-04-25T10:11:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.457+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 19 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.457+0000 7ffb5be2d740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T10:11:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.457+0000 7ffb5be2d740 20 datascan.read_dentry: file_head not found in result 2024-04-25T10:11:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.457+0000 7ffb5be2d740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-04-25T10:11:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.457+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:122 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f4f000 con 0x560a04f37000 2024-04-25T10:11:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.459+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 20 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v88'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.459+0000 7ffb5be2d740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T10:11:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.459+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:123 13.27 13:e4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f5b000 con 0x560a04f37000 2024-04-25T10:11:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.459+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.6 v2:172.21.15.186:6816/1123185664 21 ==== osd_op_reply(123 [pgnls-filter start_epoch 88 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f37000 2024-04-25T10:11:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.459+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:124 13.17 13:e8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a05ae1000 con 0x560a05ae1c00 2024-04-25T10:11:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.459+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.1 v2:172.21.15.16:6808/1916704159 15 ==== osd_op_reply(124 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a05ae1c00 2024-04-25T10:11:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.459+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:125 13.37 13:ec000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f26000 con 0x560a05ae1c00 2024-04-25T10:11:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.460+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.1 v2:172.21.15.16:6808/1916704159 16 ==== osd_op_reply(125 [pgnls-filter start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a05ae1c00 2024-04-25T10:11:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.460+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:126 13.f 13:f0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a05ae0400 con 0x560a04f27800 2024-04-25T10:11:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.460+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.2 v2:172.21.15.16:6800/2094356376 17 ==== osd_op_reply(126 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a04f27800 2024-04-25T10:11:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.460+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:127 13.2f 13:f4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f5ac00 con 0x560a05ae0c00 2024-04-25T10:11:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.460+0000 7ffb58f8f640 1 -- 172.21.15.16:0/3926332243 <== osd.0 v2:172.21.15.16:6816/2167750256 18 ==== osd_op_reply(127 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92000 con 0x560a05ae0c00 2024-04-25T10:11:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.460+0000 7ffb54786640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:128 13.1f 13:f8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f37400 con 0x560a05ae1c00 2024-04-25T10:11:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.460+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.1 v2:172.21.15.16:6808/1916704159 17 ==== osd_op_reply(128 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a05ae1c00 2024-04-25T10:11:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.460+0000 7ffb54f87640 1 -- 172.21.15.16:0/3926332243 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:129 13.3f 13:fc000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x560a04f5b400 con 0x560a05ae1c00 2024-04-25T10:11:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 <== osd.1 v2:172.21.15.16:6808/1916704159 18 ==== osd_op_reply(129 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560a04c92780 con 0x560a05ae1c00 2024-04-25T10:11:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x560a05ae0c00 msgr2=0x560a04c16680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x560a05ae0c00 0x560a04c16680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x560a05ae1c00 msgr2=0x560a04c17180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x560a05ae1c00 0x560a04c17180 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x560a04f27800 msgr2=0x560a04f10580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x560a04f27800 0x560a04f10580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x560a05ae1400 msgr2=0x560a04c16c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x560a05ae1400 0x560a04c16c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x560a05ae1800 msgr2=0x560a04f11600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 reap_dead start 2024-04-25T10:11:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x560a05ae1800 0x560a04f11600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x560a04f36800 msgr2=0x560a04f11080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x560a04f36800 0x560a04f11080 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x560a04f37000 msgr2=0x560a04f11b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x560a04f37000 0x560a04f11b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x560a04f36000 msgr2=0x560a04f10b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x560a04f36000 0x560a04f10b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x560a05ae0800 msgr2=0x560a04f10000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x560a05ae0800 0x560a04f10000 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x560a05a66bd0 tx=0x560a05ad6300 comp rx=0 tx=0).stop 2024-04-25T10:11:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a0597d400 msgr2=0x560a04c14580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb59790640 1 -- 172.21.15.16:0/3926332243 reap_dead start 2024-04-25T10:11:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a0597d400 0x560a04c14580 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x560a059bd2f0 tx=0x560a05976120 comp rx=0 tx=0).stop 2024-04-25T10:11:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 shutdown_connections 2024-04-25T10:11:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/3926332243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a0597d400 0x560a04c14580 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 >> 172.21.15.16:0/3926332243 conn(0x560a0597cc00 msgr2=0x560a04c8d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 shutdown_connections 2024-04-25T10:11:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 -- 172.21.15.16:0/3926332243 wait complete. 2024-04-25T10:11:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 -- 172.21.15.16:0/2128746244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a04ba5c00 msgr2=0x560a04c15080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/2128746244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a04ba5c00 0x560a04c15080 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x560a059bdc50 tx=0x560a05976300 comp rx=0 tx=0).stop 2024-04-25T10:11:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 -- 172.21.15.16:0/2128746244 shutdown_connections 2024-04-25T10:11:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/2128746244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a04ba5c00 0x560a04c15080 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/2128746244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a04ba5800 0x560a04c15600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 --2- 172.21.15.16:0/2128746244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a04ba4c00 0x560a04c15b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.461+0000 7ffb5be2d740 1 -- 172.21.15.16:0/2128746244 >> 172.21.15.16:0/2128746244 conn(0x560a04ba5000 msgr2=0x560a04c8c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.462+0000 7ffb5be2d740 1 -- 172.21.15.16:0/2128746244 shutdown_connections 2024-04-25T10:11:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.462+0000 7ffb5be2d740 1 -- 172.21.15.16:0/2128746244 wait complete. 2024-04-25T10:11:55.484 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.144582, result: 2024-04-25T10:11:55.485 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-25T10:11:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.545+0000 7fb04a63a740 1 -- start start 2024-04-25T10:11:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.545+0000 7fb04a63a740 1 Processor -- start 2024-04-25T10:11:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04a63a740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555db7256c00 0x555db72c6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04a63a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db7257800 0x555db72c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04a63a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555db7257c00 0x555db72c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04a63a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db73ca600 con 0x555db7257800 2024-04-25T10:11:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04a63a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db73ca480 con 0x555db7257c00 2024-04-25T10:11:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04a63a740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x555db73ca300 con 0x555db7256c00 2024-04-25T10:11:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04775c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db7257800 0x555db72c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04775c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db7257800 0x555db72c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49464/0 (socket says 172.21.15.16:49464) 2024-04-25T10:11:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04775c640 1 -- 172.21.15.16:0/2094128105 learned_addr learned my addr 172.21.15.16:0/2094128105 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb047f5d640 1 --2- 172.21.15.16:0/2094128105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555db7256c00 0x555db72c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04875e640 1 --2- 172.21.15.16:0/2094128105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555db7257c00 0x555db72c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04775c640 1 -- 172.21.15.16:0/2094128105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555db7256c00 msgr2=0x555db72c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04775c640 1 --2- 172.21.15.16:0/2094128105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555db7256c00 0x555db72c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04775c640 1 -- 172.21.15.16:0/2094128105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555db7257c00 msgr2=0x555db72c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04775c640 1 --2- 172.21.15.16:0/2094128105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555db7257c00 0x555db72c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04775c640 1 -- 172.21.15.16:0/2094128105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555db7283680 con 0x555db7257800 2024-04-25T10:11:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04775c640 1 --2- 172.21.15.16:0/2094128105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db7257800 0x555db72c6b00 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x555db8069200 tx=0x555db8112120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09ba3ed31e37272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb045758640 1 -- 172.21.15.16:0/2094128105 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555db727c9c0 con 0x555db7257800 2024-04-25T10:11:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb045758640 1 -- 172.21.15.16:0/2094128105 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555db72a0c40 con 0x555db7257800 2024-04-25T10:11:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb045758640 1 -- 172.21.15.16:0/2094128105 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555db727c680 con 0x555db7257800 2024-04-25T10:11:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04a63a740 1 -- 172.21.15.16:0/2094128105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555db7283860 con 0x555db7257800 2024-04-25T10:11:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04a63a740 1 -- 172.21.15.16:0/2094128105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x555db72832c0 con 0x555db7257800 2024-04-25T10:11:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.546+0000 7fb04a63a740 4 waiting for MDS map... 2024-04-25T10:11:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.547+0000 7fb045758640 1 -- 172.21.15.16:0/2094128105 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x555db8124200 con 0x555db7257800 2024-04-25T10:11:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.547+0000 7fb045758640 1 -- 172.21.15.16:0/2094128105 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 59) v1 ==== 2723+0+0 (secure 0 0 0) 0x555db72af180 con 0x555db7257800 2024-04-25T10:11:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.547+0000 7fb04a63a740 4 Got MDS map 59 2024-04-25T10:11:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.547+0000 7fb04a63a740 4 datascan.main: Using metadata pool output 2024-04-25T10:11:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.547+0000 7fb04a63a740 4 datascan.main: connecting to RADOS... 2024-04-25T10:11:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.547+0000 7fb04a63a740 1 Processor -- start 2024-04-25T10:11:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.547+0000 7fb04a63a740 1 -- start start 2024-04-25T10:11:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.548+0000 7fb04a63a740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555db7333c00 0x555db72c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.548+0000 7fb04a63a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db8178000 0x555db72c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.548+0000 7fb04a63a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555db8178400 0x555db72c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.548+0000 7fb04a63a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db73ca180 con 0x555db8178000 2024-04-25T10:11:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.548+0000 7fb04a63a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db72b6480 con 0x555db8178400 2024-04-25T10:11:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.548+0000 7fb04a63a740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x555db72b6900 con 0x555db7333c00 2024-04-25T10:11:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.548+0000 7fb047f5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db8178000 0x555db72c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.548+0000 7fb047f5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db8178000 0x555db72c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49472/0 (socket says 172.21.15.16:49472) 2024-04-25T10:11:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.548+0000 7fb047f5d640 1 -- 172.21.15.16:0/4166432135 learned_addr learned my addr 172.21.15.16:0/4166432135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.548+0000 7fb04875e640 1 --2- 172.21.15.16:0/4166432135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555db7333c00 0x555db72c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.548+0000 7fb04775c640 1 --2- 172.21.15.16:0/4166432135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555db8178400 0x555db72c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.548+0000 7fb047f5d640 1 -- 172.21.15.16:0/4166432135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555db7333c00 msgr2=0x555db72c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.548+0000 7fb047f5d640 1 --2- 172.21.15.16:0/4166432135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555db7333c00 0x555db72c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.548+0000 7fb047f5d640 1 -- 172.21.15.16:0/4166432135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555db8178400 msgr2=0x555db72c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.548+0000 7fb047f5d640 1 --2- 172.21.15.16:0/4166432135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555db8178400 0x555db72c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.548+0000 7fb047f5d640 1 -- 172.21.15.16:0/4166432135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555db72834a0 con 0x555db8178000 2024-04-25T10:11:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.548+0000 7fb047f5d640 1 --2- 172.21.15.16:0/4166432135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db8178000 0x555db72c7b80 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x555db804a810 tx=0x555db8190000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdff090d6d5d567b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.549+0000 7fb043754640 1 -- 172.21.15.16:0/4166432135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555db727c9c0 con 0x555db8178000 2024-04-25T10:11:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.549+0000 7fb043754640 1 -- 172.21.15.16:0/4166432135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555db72a0a80 con 0x555db8178000 2024-04-25T10:11:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.549+0000 7fb043754640 1 -- 172.21.15.16:0/4166432135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555db727c680 con 0x555db8178000 2024-04-25T10:11:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.549+0000 7fb04a63a740 1 -- 172.21.15.16:0/4166432135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db8178000 msgr2=0x555db72c7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.549+0000 7fb04a63a740 1 --2- 172.21.15.16:0/4166432135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db8178000 0x555db72c7b80 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x555db804a810 tx=0x555db8190000 comp rx=0 tx=0).stop 2024-04-25T10:11:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.549+0000 7fb04a63a740 1 -- 172.21.15.16:0/4166432135 shutdown_connections 2024-04-25T10:11:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.549+0000 7fb04a63a740 1 --2- 172.21.15.16:0/4166432135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555db8178400 0x555db72c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.549+0000 7fb04a63a740 1 --2- 172.21.15.16:0/4166432135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db8178000 0x555db72c7b80 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.549+0000 7fb04a63a740 1 --2- 172.21.15.16:0/4166432135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555db7333c00 0x555db72c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.549+0000 7fb04a63a740 1 -- 172.21.15.16:0/4166432135 >> 172.21.15.16:0/4166432135 conn(0x555db7333800 msgr2=0x555db733c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.549+0000 7fb04a63a740 1 -- 172.21.15.16:0/4166432135 shutdown_connections 2024-04-25T10:11:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.549+0000 7fb04a63a740 1 -- 172.21.15.16:0/4166432135 wait complete. 2024-04-25T10:11:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.550+0000 7fb04a63a740 1 Processor -- start 2024-04-25T10:11:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.550+0000 7fb04a63a740 1 -- start start 2024-04-25T10:11:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.550+0000 7fb04a63a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db7333800 0x555db72c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.550+0000 7fb04a63a740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555db8179000 0x555db72c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.550+0000 7fb04a63a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555db8178c00 0x555db72c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.550+0000 7fb04a63a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db73cb380 con 0x555db7333800 2024-04-25T10:11:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.550+0000 7fb04a63a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db73ca180 con 0x555db8178c00 2024-04-25T10:11:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.550+0000 7fb04a63a740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x555db73cb500 con 0x555db8179000 2024-04-25T10:11:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.550+0000 7fb04875e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db7333800 0x555db72c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.550+0000 7fb04875e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db7333800 0x555db72c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49482/0 (socket says 172.21.15.16:49482) 2024-04-25T10:11:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.550+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 learned_addr learned my addr 172.21.15.16:0/3407194514 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.550+0000 7fb047f5d640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555db8179000 0x555db72c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.551+0000 7fb04775c640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555db8178c00 0x555db72c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.551+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555db8179000 msgr2=0x555db72c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.551+0000 7fb04875e640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555db8179000 0x555db72c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.551+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555db8178c00 msgr2=0x555db72c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.551+0000 7fb04875e640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555db8178c00 0x555db72c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.551+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555db72830e0 con 0x555db7333800 2024-04-25T10:11:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.551+0000 7fb04875e640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db7333800 0x555db72c9180 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x555db804bbc0 tx=0x555db8190120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.551+0000 7fb041f51640 1 -- 172.21.15.16:0/3407194514 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555db727c9c0 con 0x555db7333800 2024-04-25T10:11:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.551+0000 7fb041f51640 1 -- 172.21.15.16:0/3407194514 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555db72a1500 con 0x555db7333800 2024-04-25T10:11:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.551+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555db7283a40 con 0x555db7333800 2024-04-25T10:11:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.551+0000 7fb041f51640 1 -- 172.21.15.16:0/3407194514 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555db727c680 con 0x555db7333800 2024-04-25T10:11:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.551+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555db72823c0 con 0x555db7333800 2024-04-25T10:11:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.551+0000 7fb04a63a740 4 datascan.init: resolving metadata pool 12 2024-04-25T10:11:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.552+0000 7fb041f51640 1 -- 172.21.15.16:0/3407194514 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x555db8196c00 con 0x555db7333800 2024-04-25T10:11:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.553+0000 7fb041f51640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x555db819ac00 0x555db72c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.553+0000 7fb041f51640 1 -- 172.21.15.16:0/3407194514 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x555db817fa00 con 0x555db7333800 2024-04-25T10:11:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.553+0000 7fb04a63a740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:11:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.553+0000 7fb04a63a740 4 datascan.main: resolving metadata pool 12 2024-04-25T10:11:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.553+0000 7fb04a63a740 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x555db8178800 0x555db72c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.553+0000 7fb047f5d640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x555db819ac00 0x555db72c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.553+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db8178400 con 0x555db8178800 2024-04-25T10:11:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.553+0000 7fb04775c640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x555db8178800 0x555db72c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.553+0000 7fb047f5d640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x555db819ac00 0x555db72c9700 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x555db804a810 tx=0x555db8112480 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.553+0000 7fb04775c640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x555db8178800 0x555db72c8100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.554+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.0 v2:172.21.15.16:6816/2167750256 1 ==== osd_op_reply(1 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db8178800 2024-04-25T10:11:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.554+0000 7fb04775c640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x555db8178400 0x555db75c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.554+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:2 12.10 12:08000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db7333400 con 0x555db8178400 2024-04-25T10:11:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.554+0000 7fb04875e640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x555db8178400 0x555db75c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.554+0000 7fb04875e640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x555db8178400 0x555db75c8000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.555+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 1 ==== osd_op_reply(2 [pgnls start_epoch 88 out=177b] v88'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.555+0000 7fb04875e640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x555db7333400 0x555db75c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.555+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:3 12.8 12:10000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db819b000 con 0x555db7333400 2024-04-25T10:11:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.555+0000 7fb047f5d640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x555db7333400 0x555db75c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.555+0000 7fb047f5d640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x555db7333400 0x555db75c8580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.556+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.7 v2:172.21.15.186:6808/1627307043 1 ==== osd_op_reply(3 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344780 con 0x555db7333400 2024-04-25T10:11:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.556+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:4 12.18 12:18000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db819b000 con 0x555db7333400 2024-04-25T10:11:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.556+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.7 v2:172.21.15.186:6808/1627307043 2 ==== osd_op_reply(4 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344780 con 0x555db7333400 2024-04-25T10:11:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.556+0000 7fb047f5d640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x555db819b000 0x555db75c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.556+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:5 12.4 12:20000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db8179c00 con 0x555db819b000 2024-04-25T10:11:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.556+0000 7fb04775c640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x555db819b000 0x555db75c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.556+0000 7fb04775c640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x555db819b000 0x555db75c8b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.557+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.2 v2:172.21.15.16:6800/2094356376 1 ==== osd_op_reply(5 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db819b000 2024-04-25T10:11:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.557+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:6 12.14 12:28000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db8179c00 con 0x555db8178800 2024-04-25T10:11:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.557+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.0 v2:172.21.15.16:6816/2167750256 2 ==== osd_op_reply(6 [pgnls start_epoch 88 out=149b] v87'27 uv27 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x555db7344000 con 0x555db8178800 2024-04-25T10:11:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.557+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:7 12.c 12:30000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db8179c00 con 0x555db7333400 2024-04-25T10:11:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.557+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.7 v2:172.21.15.186:6808/1627307043 3 ==== osd_op_reply(7 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344780 con 0x555db7333400 2024-04-25T10:11:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.557+0000 7fb047f5d640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x555db8179c00 0x555db75c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.557+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:8 12.1c 12:38000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db819bc00 con 0x555db8179c00 2024-04-25T10:11:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.557+0000 7fb04875e640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x555db8179c00 0x555db75c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.558+0000 7fb04875e640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x555db8179c00 0x555db75c9080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.558+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.5 v2:172.21.15.186:6800/1827108421 1 ==== osd_op_reply(8 [pgnls start_epoch 88 out=73b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555db7344500 con 0x555db8179c00 2024-04-25T10:11:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.558+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:9 12.2 12:40000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db819bc00 con 0x555db819b000 2024-04-25T10:11:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.559+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.2 v2:172.21.15.16:6800/2094356376 2 ==== osd_op_reply(9 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db819b000 2024-04-25T10:11:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.559+0000 7fb04775c640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x555db819bc00 0x555db75c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.559+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:10 12.12 12:48000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8000 con 0x555db819bc00 2024-04-25T10:11:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.559+0000 7fb047f5d640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x555db819bc00 0x555db75c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.559+0000 7fb047f5d640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x555db819bc00 0x555db75c9b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.559+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.4 v2:172.21.15.186:6824/3091472889 1 ==== osd_op_reply(10 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344780 con 0x555db819bc00 2024-04-25T10:11:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.559+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:11 12.a 12:50000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8000 con 0x555db819bc00 2024-04-25T10:11:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.560+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.4 v2:172.21.15.186:6824/3091472889 2 ==== osd_op_reply(11 [pgnls start_epoch 88 out=81b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555db7344780 con 0x555db819bc00 2024-04-25T10:11:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.560+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:12 12.1a 12:58000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8000 con 0x555db819b000 2024-04-25T10:11:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.560+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.2 v2:172.21.15.16:6800/2094356376 3 ==== osd_op_reply(12 [pgnls start_epoch 88 out=49b] v87'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db819b000 2024-04-25T10:11:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.560+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:13 12.6 12:60000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8000 con 0x555db8178400 2024-04-25T10:11:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.560+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 2 ==== osd_op_reply(13 [pgnls start_epoch 88 out=77b] v87'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.560+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:14 12.16 12:68000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8000 con 0x555db819b000 2024-04-25T10:11:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.561+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.2 v2:172.21.15.16:6800/2094356376 4 ==== osd_op_reply(14 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db819b000 2024-04-25T10:11:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.561+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:15 12.e 12:70000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8000 con 0x555db7333400 2024-04-25T10:11:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.561+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.7 v2:172.21.15.186:6808/1627307043 4 ==== osd_op_reply(15 [pgnls start_epoch 88 out=73b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555db7344780 con 0x555db7333400 2024-04-25T10:11:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.561+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:16 12.1e 12:78000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8000 con 0x555db819b000 2024-04-25T10:11:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.561+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.2 v2:172.21.15.16:6800/2094356376 5 ==== osd_op_reply(16 [pgnls start_epoch 88 out=49b] v87'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db819b000 2024-04-25T10:11:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.561+0000 7fb04775c640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x555db75e8000 0x555db75ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.561+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:17 12.1 12:80000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8400 con 0x555db75e8000 2024-04-25T10:11:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.561+0000 7fb04775c640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x555db75e8000 0x555db75ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.561+0000 7fb04775c640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x555db75e8000 0x555db75ca100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.562+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.1 v2:172.21.15.16:6808/1916704159 1 ==== osd_op_reply(17 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db75e8000 2024-04-25T10:11:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.562+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:18 12.11 12:88000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8400 con 0x555db819b000 2024-04-25T10:11:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.562+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.2 v2:172.21.15.16:6800/2094356376 6 ==== osd_op_reply(18 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db819b000 2024-04-25T10:11:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.562+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:19 12.9 12:90000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8400 con 0x555db8178400 2024-04-25T10:11:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.562+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 3 ==== osd_op_reply(19 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.562+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:20 12.19 12:98000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8400 con 0x555db7333400 2024-04-25T10:11:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.563+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.7 v2:172.21.15.186:6808/1627307043 5 ==== osd_op_reply(20 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344780 con 0x555db7333400 2024-04-25T10:11:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.563+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:21 12.5 12:a0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8400 con 0x555db8178800 2024-04-25T10:11:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.563+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.0 v2:172.21.15.16:6816/2167750256 3 ==== osd_op_reply(21 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db8178800 2024-04-25T10:11:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.563+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:22 12.15 12:a8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8400 con 0x555db75e8000 2024-04-25T10:11:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.563+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.1 v2:172.21.15.16:6808/1916704159 2 ==== osd_op_reply(22 [pgnls start_epoch 88 out=97b] v87'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x555db7344000 con 0x555db75e8000 2024-04-25T10:11:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.563+0000 7fb04775c640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x555db75e8400 0x555db75ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.563+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:23 12.d 12:b0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8800 con 0x555db75e8400 2024-04-25T10:11:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.563+0000 7fb04875e640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x555db75e8400 0x555db75ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.563+0000 7fb04875e640 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x555db75e8400 0x555db75ca680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.564+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.3 v2:172.21.15.16:6824/256577261 1 ==== osd_op_reply(23 [pgnls start_epoch 88 out=74b] v85'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x555db7344500 con 0x555db75e8400 2024-04-25T10:11:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.564+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:24 12.1d 12:b8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8800 con 0x555db8178400 2024-04-25T10:11:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.564+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 4 ==== osd_op_reply(24 [pgnls start_epoch 88 out=73b] v87'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.564+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:25 12.3 12:c0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8800 con 0x555db819bc00 2024-04-25T10:11:55.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.564+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.4 v2:172.21.15.186:6824/3091472889 3 ==== osd_op_reply(25 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344780 con 0x555db819bc00 2024-04-25T10:11:55.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.565+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:26 12.13 12:c8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8800 con 0x555db8179c00 2024-04-25T10:11:55.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.565+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.5 v2:172.21.15.186:6800/1827108421 2 ==== osd_op_reply(26 [pgnls start_epoch 88 out=98b] v87'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x555db7344500 con 0x555db8179c00 2024-04-25T10:11:55.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.565+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:27 12.b 12:d0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8800 con 0x555db7333400 2024-04-25T10:11:55.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.565+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.7 v2:172.21.15.186:6808/1627307043 6 ==== osd_op_reply(27 [pgnls start_epoch 88 out=76b] v87'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x555db7344780 con 0x555db7333400 2024-04-25T10:11:55.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.565+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:28 12.1b 12:d8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8800 con 0x555db8178800 2024-04-25T10:11:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.565+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.0 v2:172.21.15.16:6816/2167750256 4 ==== osd_op_reply(28 [pgnls start_epoch 88 out=73b] v87'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555db7344000 con 0x555db8178800 2024-04-25T10:11:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.565+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:29 12.7 12:e0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8800 con 0x555db7333400 2024-04-25T10:11:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.566+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.7 v2:172.21.15.186:6808/1627307043 7 ==== osd_op_reply(29 [pgnls start_epoch 88 out=73b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555db7344780 con 0x555db7333400 2024-04-25T10:11:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.566+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:30 12.17 12:e8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8800 con 0x555db75e8000 2024-04-25T10:11:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.566+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.1 v2:172.21.15.16:6808/1916704159 3 ==== osd_op_reply(30 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db75e8000 2024-04-25T10:11:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.566+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:31 12.f 12:f0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8800 con 0x555db75e8400 2024-04-25T10:11:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.566+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.3 v2:172.21.15.16:6824/256577261 2 ==== osd_op_reply(31 [pgnls start_epoch 88 out=73b] v85'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555db7344500 con 0x555db75e8400 2024-04-25T10:11:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.566+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:32 12.1f 12:f8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8800 con 0x555db75e8400 2024-04-25T10:11:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.566+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.3 v2:172.21.15.16:6824/256577261 3 ==== osd_op_reply(32 [pgnls start_epoch 88 out=101b] v87'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x555db7344500 con 0x555db75e8400 2024-04-25T10:11:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.566+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-25T10:11:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.566+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:33 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db8178000 con 0x555db8178400 2024-04-25T10:11:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.567+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.567+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-25T10:11:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.567+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:34 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e9000 con 0x555db8178400 2024-04-25T10:11:55.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.567+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.567+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-25T10:11:55.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.567+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:35 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e9400 con 0x555db8178400 2024-04-25T10:11:55.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.568+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.568+0000 7fb04a63a740 20 datascan.scan_links: injected ino detected 2024-04-25T10:11:55.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.568+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-25T10:11:55.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.568+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:36 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e9800 con 0x555db8178400 2024-04-25T10:11:55.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.568+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.568+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-25T10:11:55.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.568+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:37 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e9c00 con 0x555db8178400 2024-04-25T10:11:55.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.568+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.568+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-25T10:11:55.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.568+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:11:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.568+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-25T10:11:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.568+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:11:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.568+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-25T10:11:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.568+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:11:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.568+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-25T10:11:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.568+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:38 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e8800 con 0x555db8178800 2024-04-25T10:11:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.569+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.0 v2:172.21.15.16:6816/2167750256 5 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv25 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344000 con 0x555db8178800 2024-04-25T10:11:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.569+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-25T10:11:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.569+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:39 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e9000 con 0x555db8179c00 2024-04-25T10:11:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.569+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.5 v2:172.21.15.186:6800/1827108421 3 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344500 con 0x555db8179c00 2024-04-25T10:11:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.569+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-04-25T10:11:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.569+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:40 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db8178000 con 0x555db819bc00 2024-04-25T10:11:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.570+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.4 v2:172.21.15.186:6824/3091472889 4 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x555db7344780 con 0x555db819bc00 2024-04-25T10:11:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.570+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-25T10:11:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.570+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:11:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.570+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-25T10:11:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.570+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:41 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db819b800 con 0x555db7333400 2024-04-25T10:11:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.570+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.7 v2:172.21.15.186:6808/1627307043 8 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344780 con 0x555db7333400 2024-04-25T10:11:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.570+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-25T10:11:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.570+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:42 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db819b400 con 0x555db75e8000 2024-04-25T10:11:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.570+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.1 v2:172.21.15.16:6808/1916704159 4 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344000 con 0x555db75e8000 2024-04-25T10:11:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.570+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-25T10:11:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.570+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:11:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.570+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-25T10:11:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.570+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:11:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.570+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-25T10:11:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.570+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:43 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fa000 con 0x555db8178400 2024-04-25T10:11:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.570+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 10 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.570+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-25T10:11:55.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.571+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:11:55.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.571+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-25T10:11:55.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.571+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:44 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fa400 con 0x555db8179c00 2024-04-25T10:11:55.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.571+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.5 v2:172.21.15.186:6800/1827108421 4 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x555db7344500 con 0x555db8179c00 2024-04-25T10:11:55.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.571+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-25T10:11:55.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.571+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:11:55.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.571+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 200.00000002 2024-04-25T10:11:55.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.571+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-25T10:11:55.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.571+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-25T10:11:55.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.571+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:45 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fa800 con 0x555db7333400 2024-04-25T10:11:55.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.571+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.7 v2:172.21.15.186:6808/1627307043 9 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344780 con 0x555db7333400 2024-04-25T10:11:55.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.571+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-25T10:11:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.571+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:11:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.571+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-25T10:11:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.571+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:11:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.571+0000 7fb04a63a740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-25T10:11:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.571+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:46 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fac00 con 0x555db75e8400 2024-04-25T10:11:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.572+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.3 v2:172.21.15.16:6824/256577261 4 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1163 (crc 0 0 0) 0x555db7344500 con 0x555db75e8400 2024-04-25T10:11:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.572+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:47 12.0 12:00000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db8178800 2024-04-25T10:11:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.572+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.0 v2:172.21.15.16:6816/2167750256 6 ==== osd_op_reply(47 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db8178800 2024-04-25T10:11:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.572+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:48 12.10 12:08000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db8178400 2024-04-25T10:11:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.572+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 11 ==== osd_op_reply(48 [pgnls start_epoch 88 out=177b] v88'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.572+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:49 12.8 12:10000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db7333400 2024-04-25T10:11:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.573+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.7 v2:172.21.15.186:6808/1627307043 10 ==== osd_op_reply(49 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344780 con 0x555db7333400 2024-04-25T10:11:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.573+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:50 12.18 12:18000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db7333400 2024-04-25T10:11:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.573+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.7 v2:172.21.15.186:6808/1627307043 11 ==== osd_op_reply(50 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344780 con 0x555db7333400 2024-04-25T10:11:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.573+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:51 12.4 12:20000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db819b000 2024-04-25T10:11:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.573+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.2 v2:172.21.15.16:6800/2094356376 7 ==== osd_op_reply(51 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db819b000 2024-04-25T10:11:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.573+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:52 12.14 12:28000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db8178800 2024-04-25T10:11:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.573+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.0 v2:172.21.15.16:6816/2167750256 7 ==== osd_op_reply(52 [pgnls start_epoch 88 out=149b] v87'27 uv27 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x555db7344000 con 0x555db8178800 2024-04-25T10:11:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.573+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:53 12.c 12:30000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db7333400 2024-04-25T10:11:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.574+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.7 v2:172.21.15.186:6808/1627307043 12 ==== osd_op_reply(53 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344780 con 0x555db7333400 2024-04-25T10:11:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.574+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:54 12.1c 12:38000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db8179c00 2024-04-25T10:11:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.574+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.5 v2:172.21.15.186:6800/1827108421 5 ==== osd_op_reply(54 [pgnls start_epoch 88 out=73b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555db7344500 con 0x555db8179c00 2024-04-25T10:11:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.574+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:55 12.2 12:40000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db819b000 2024-04-25T10:11:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.575+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.2 v2:172.21.15.16:6800/2094356376 8 ==== osd_op_reply(55 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db819b000 2024-04-25T10:11:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.575+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:56 12.12 12:48000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db819bc00 2024-04-25T10:11:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.575+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.4 v2:172.21.15.186:6824/3091472889 5 ==== osd_op_reply(56 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344780 con 0x555db819bc00 2024-04-25T10:11:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.575+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:57 12.a 12:50000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db819bc00 2024-04-25T10:11:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.575+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.4 v2:172.21.15.186:6824/3091472889 6 ==== osd_op_reply(57 [pgnls start_epoch 88 out=81b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555db7344780 con 0x555db819bc00 2024-04-25T10:11:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.575+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:58 12.1a 12:58000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db819b000 2024-04-25T10:11:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.575+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.2 v2:172.21.15.16:6800/2094356376 9 ==== osd_op_reply(58 [pgnls start_epoch 88 out=49b] v87'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db819b000 2024-04-25T10:11:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.575+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:59 12.6 12:60000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db8178400 2024-04-25T10:11:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.576+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 12 ==== osd_op_reply(59 [pgnls start_epoch 88 out=77b] v87'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.576+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:60 12.16 12:68000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db819b000 2024-04-25T10:11:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.576+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.2 v2:172.21.15.16:6800/2094356376 10 ==== osd_op_reply(60 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db819b000 2024-04-25T10:11:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.576+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:61 12.e 12:70000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db7333400 2024-04-25T10:11:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.576+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.7 v2:172.21.15.186:6808/1627307043 13 ==== osd_op_reply(61 [pgnls start_epoch 88 out=73b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555db7344780 con 0x555db7333400 2024-04-25T10:11:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.576+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:62 12.1e 12:78000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db819b000 2024-04-25T10:11:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.576+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.2 v2:172.21.15.16:6800/2094356376 11 ==== osd_op_reply(62 [pgnls start_epoch 88 out=49b] v87'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db819b000 2024-04-25T10:11:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.577+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:63 12.1 12:80000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db75e8000 2024-04-25T10:11:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.577+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.1 v2:172.21.15.16:6808/1916704159 5 ==== osd_op_reply(63 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db75e8000 2024-04-25T10:11:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.577+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:64 12.11 12:88000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db819b000 2024-04-25T10:11:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.577+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.2 v2:172.21.15.16:6800/2094356376 12 ==== osd_op_reply(64 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db819b000 2024-04-25T10:11:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.577+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:65 12.9 12:90000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db8178400 2024-04-25T10:11:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.577+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 13 ==== osd_op_reply(65 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.577+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:66 12.19 12:98000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db7333400 2024-04-25T10:11:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.578+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.7 v2:172.21.15.186:6808/1627307043 14 ==== osd_op_reply(66 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344780 con 0x555db7333400 2024-04-25T10:11:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.578+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:67 12.5 12:a0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db8178800 2024-04-25T10:11:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.578+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.0 v2:172.21.15.16:6816/2167750256 8 ==== osd_op_reply(67 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db8178800 2024-04-25T10:11:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.578+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:68 12.15 12:a8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db75e8000 2024-04-25T10:11:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.578+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.1 v2:172.21.15.16:6808/1916704159 6 ==== osd_op_reply(68 [pgnls start_epoch 88 out=97b] v87'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x555db7344000 con 0x555db75e8000 2024-04-25T10:11:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.578+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:69 12.d 12:b0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db75e8400 2024-04-25T10:11:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.578+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.3 v2:172.21.15.16:6824/256577261 5 ==== osd_op_reply(69 [pgnls start_epoch 88 out=74b] v85'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x555db7344500 con 0x555db75e8400 2024-04-25T10:11:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.578+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:70 12.1d 12:b8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db8178400 2024-04-25T10:11:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.579+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 14 ==== osd_op_reply(70 [pgnls start_epoch 88 out=73b] v87'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.579+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:71 12.3 12:c0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db819bc00 2024-04-25T10:11:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.579+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.4 v2:172.21.15.186:6824/3091472889 7 ==== osd_op_reply(71 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344780 con 0x555db819bc00 2024-04-25T10:11:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.579+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:72 12.13 12:c8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db8179c00 2024-04-25T10:11:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.579+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.5 v2:172.21.15.186:6800/1827108421 6 ==== osd_op_reply(72 [pgnls start_epoch 88 out=98b] v87'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x555db7344500 con 0x555db8179c00 2024-04-25T10:11:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.579+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:73 12.b 12:d0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db7333400 2024-04-25T10:11:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.580+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.7 v2:172.21.15.186:6808/1627307043 15 ==== osd_op_reply(73 [pgnls start_epoch 88 out=76b] v87'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x555db7344780 con 0x555db7333400 2024-04-25T10:11:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.580+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:74 12.1b 12:d8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db8178800 2024-04-25T10:11:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.580+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.0 v2:172.21.15.16:6816/2167750256 9 ==== osd_op_reply(74 [pgnls start_epoch 88 out=73b] v87'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555db7344000 con 0x555db8178800 2024-04-25T10:11:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.580+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:75 12.7 12:e0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db7333400 2024-04-25T10:11:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.580+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.7 v2:172.21.15.186:6808/1627307043 16 ==== osd_op_reply(75 [pgnls start_epoch 88 out=73b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555db7344780 con 0x555db7333400 2024-04-25T10:11:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.580+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:76 12.17 12:e8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db75e8000 2024-04-25T10:11:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.581+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.1 v2:172.21.15.16:6808/1916704159 7 ==== osd_op_reply(76 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555db7344000 con 0x555db75e8000 2024-04-25T10:11:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.581+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:77 12.f 12:f0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db75e8400 2024-04-25T10:11:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.581+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.3 v2:172.21.15.16:6824/256577261 6 ==== osd_op_reply(77 [pgnls start_epoch 88 out=73b] v85'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x555db7344500 con 0x555db75e8400 2024-04-25T10:11:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.581+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:78 12.1f 12:f8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db75e8400 2024-04-25T10:11:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.581+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.3 v2:172.21.15.16:6824/256577261 7 ==== osd_op_reply(78 [pgnls start_epoch 88 out=101b] v87'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x555db7344500 con 0x555db75e8400 2024-04-25T10:11:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.581+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-25T10:11:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.581+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:79 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb400 con 0x555db8178400 2024-04-25T10:11:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.582+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 15 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.582+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-25T10:11:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.582+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:80 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb800 con 0x555db8178400 2024-04-25T10:11:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.582+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 16 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.582+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-25T10:11:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.582+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:81 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fbc00 con 0x555db8178400 2024-04-25T10:11:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.582+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 17 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.582+0000 7fb04a63a740 20 datascan.scan_links: injected ino detected 2024-04-25T10:11:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.582+0000 7fb04a63a740 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-04-25T10:11:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.582+0000 7fb04a63a740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-04-25T10:11:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.582+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-25T10:11:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.582+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:82 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fa000 con 0x555db8178400 2024-04-25T10:11:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.583+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 18 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.583+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-25T10:11:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.583+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:83 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fa400 con 0x555db8178400 2024-04-25T10:11:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.583+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 19 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.583+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-25T10:11:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.583+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:11:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.583+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-25T10:11:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.583+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:11:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.583+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-25T10:11:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.583+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:11:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.583+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-25T10:11:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.583+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:84 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e9400 con 0x555db8178800 2024-04-25T10:11:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.583+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.0 v2:172.21.15.16:6816/2167750256 10 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv25 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344000 con 0x555db8178800 2024-04-25T10:11:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.583+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-25T10:11:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.583+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:85 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e9800 con 0x555db8179c00 2024-04-25T10:11:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.584+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.5 v2:172.21.15.186:6800/1827108421 7 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344500 con 0x555db8179c00 2024-04-25T10:11:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.584+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-04-25T10:11:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.584+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:86 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e9c00 con 0x555db819bc00 2024-04-25T10:11:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.584+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.4 v2:172.21.15.186:6824/3091472889 8 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x555db7344780 con 0x555db819bc00 2024-04-25T10:11:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.584+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-25T10:11:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.584+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:11:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.584+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-25T10:11:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.584+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:87 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e9000 con 0x555db7333400 2024-04-25T10:11:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.584+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.7 v2:172.21.15.186:6808/1627307043 17 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344780 con 0x555db7333400 2024-04-25T10:11:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.584+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-25T10:11:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.584+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:88 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e9c00 con 0x555db75e8000 2024-04-25T10:11:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04775c640 1 -- 172.21.15.16:0/3407194514 <== osd.1 v2:172.21.15.16:6808/1916704159 8 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344000 con 0x555db75e8000 2024-04-25T10:11:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-25T10:11:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:11:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-25T10:11:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:11:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-25T10:11:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75e9000 con 0x555db8178400 2024-04-25T10:11:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 20 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-25T10:11:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:11:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-25T10:11:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:90 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db819b800 con 0x555db8179c00 2024-04-25T10:11:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.5 v2:172.21.15.186:6800/1827108421 8 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x555db7344500 con 0x555db8179c00 2024-04-25T10:11:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-25T10:11:55.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:11:55.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 200.00000002 2024-04-25T10:11:55.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-25T10:11:55.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-25T10:11:55.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.585+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:91 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db8178000 con 0x555db7333400 2024-04-25T10:11:55.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.586+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 <== osd.7 v2:172.21.15.186:6808/1627307043 18 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x555db7344780 con 0x555db7333400 2024-04-25T10:11:55.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.586+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-25T10:11:55.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.586+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:11:55.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.586+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-25T10:11:55.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.586+0000 7fb04a63a740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:11:55.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.586+0000 7fb04a63a740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-25T10:11:55.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.586+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:92 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fa000 con 0x555db75e8400 2024-04-25T10:11:55.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.586+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.3 v2:172.21.15.16:6824/256577261 8 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1163 (crc 0 0 0) 0x555db7344500 con 0x555db75e8400 2024-04-25T10:11:55.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.586+0000 7fb04a63a740 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-04-25T10:11:55.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.586+0000 7fb04a63a740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-25T10:11:55.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.586+0000 7fb04a63a740 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-04-25T10:11:55.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.586+0000 7fb04a63a740 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-04-25T10:11:55.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.586+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:93 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fa400 con 0x555db8178400 2024-04-25T10:11:55.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.586+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 21 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.587+0000 7fb04a63a740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T10:11:55.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.587+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:94 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fa800 con 0x555db8178400 2024-04-25T10:11:55.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.588+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 22 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v88'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.588+0000 7fb04a63a740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T10:11:55.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.588+0000 7fb04a63a740 10 datascan.scan_links: processing 1 injected_inos 2024-04-25T10:11:55.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.588+0000 7fb04a63a740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-04-25T10:11:55.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.588+0000 7fb04a63a740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-04-25T10:11:55.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.588+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:95 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fac00 con 0x555db8178400 2024-04-25T10:11:55.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.588+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 23 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.588+0000 7fb04a63a740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T10:11:55.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.588+0000 7fb04a63a740 20 datascan.scan_links: first is now 2 2024-04-25T10:11:55.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.588+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:96 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb000 con 0x555db8178400 2024-04-25T10:11:55.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.592+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.6 v2:172.21.15.186:6816/1123185664 24 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v88'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555db7344500 con 0x555db8178400 2024-04-25T10:11:55.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.592+0000 7fb04a63a740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T10:11:55.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.592+0000 7fb04a63a740 10 datascan.scan_links: updating inotable 2024-04-25T10:11:55.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.592+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:97 12.13 12:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb400 con 0x555db8179c00 2024-04-25T10:11:55.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.592+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.5 v2:172.21.15.186:6800/1827108421 9 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x555db7344280 con 0x555db8179c00 2024-04-25T10:11:55.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.592+0000 7fb04a63a740 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-04-25T10:11:55.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.592+0000 7fb04a63a740 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-04-25T10:11:55.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.592+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:98 12.13 12:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fb800 con 0x555db8179c00 2024-04-25T10:11:55.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.593+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.5 v2:172.21.15.186:6800/1827108421 10 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v88'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x555db7344280 con 0x555db8179c00 2024-04-25T10:11:55.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.593+0000 7fb04a63a740 10 datascan.scan_links: updating snaptable 2024-04-25T10:11:55.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.593+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:99 12.d 12:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v9 -- 0x555db75fbc00 con 0x555db75e8400 2024-04-25T10:11:55.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04875e640 1 -- 172.21.15.16:0/3407194514 <== osd.3 v2:172.21.15.16:6824/256577261 9 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x555db7344280 con 0x555db75e8400 2024-04-25T10:11:55.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x555db8178800 msgr2=0x555db72c8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x555db8178800 0x555db72c8100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x555db75e8000 msgr2=0x555db75ca100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x555db75e8000 0x555db75ca100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x555db819b000 msgr2=0x555db75c8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x555db819b000 0x555db75c8b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x555db75e8400 msgr2=0x555db75ca680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x555db75e8400 0x555db75ca680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 reap_dead start 2024-04-25T10:11:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x555db819bc00 msgr2=0x555db75c9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x555db819bc00 0x555db75c9b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 reap_dead start 2024-04-25T10:11:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x555db8179c00 msgr2=0x555db75c9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x555db8179c00 0x555db75c9080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x555db8178400 msgr2=0x555db75c8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x555db8178400 0x555db75c8000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x555db7333400 msgr2=0x555db75c8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x555db7333400 0x555db75c8580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x555db819ac00 msgr2=0x555db72c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x555db819ac00 0x555db72c9700 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x555db804a810 tx=0x555db8112480 comp rx=0 tx=0).stop 2024-04-25T10:11:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db7333800 msgr2=0x555db72c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 --2- 172.21.15.16:0/3407194514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db7333800 0x555db72c9180 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x555db804bbc0 tx=0x555db8190120 comp rx=0 tx=0).stop 2024-04-25T10:11:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb047f5d640 1 -- 172.21.15.16:0/3407194514 reap_dead start 2024-04-25T10:11:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 shutdown_connections 2024-04-25T10:11:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.594+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 >> 172.21.15.16:0/3407194514 conn(0x555db7333c00 msgr2=0x555db733c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.595+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 shutdown_connections 2024-04-25T10:11:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.595+0000 7fb04a63a740 1 -- 172.21.15.16:0/3407194514 wait complete. 2024-04-25T10:11:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.595+0000 7fb04a63a740 1 -- 172.21.15.16:0/2094128105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db7257800 msgr2=0x555db72c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.595+0000 7fb04a63a740 1 --2- 172.21.15.16:0/2094128105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db7257800 0x555db72c6b00 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x555db8069200 tx=0x555db8112120 comp rx=0 tx=0).stop 2024-04-25T10:11:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.595+0000 7fb04a63a740 1 -- 172.21.15.16:0/2094128105 shutdown_connections 2024-04-25T10:11:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.595+0000 7fb04a63a740 1 --2- 172.21.15.16:0/2094128105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555db7257c00 0x555db72c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.595+0000 7fb04a63a740 1 --2- 172.21.15.16:0/2094128105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555db7257800 0x555db72c6b00 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.595+0000 7fb04a63a740 1 --2- 172.21.15.16:0/2094128105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555db7256c00 0x555db72c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.595+0000 7fb04a63a740 1 -- 172.21.15.16:0/2094128105 >> 172.21.15.16:0/2094128105 conn(0x555db7257000 msgr2=0x555db733d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.595+0000 7fb04a63a740 1 -- 172.21.15.16:0/2094128105 shutdown_connections 2024-04-25T10:11:55.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.595+0000 7fb04a63a740 1 -- 172.21.15.16:0/2094128105 wait complete. 2024-04-25T10:11:55.616 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.131404, result: 2024-04-25T10:11:55.616 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:11:55.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.753+0000 7fc3980f5640 1 Processor -- start 2024-04-25T10:11:55.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.753+0000 7fc3980f5640 1 -- start start 2024-04-25T10:11:55.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.754+0000 7fc3980f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc39015dca0 0x7fc3900ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.754+0000 7fc3980f5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3900d3b90 0x7fc3900d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.754+0000 7fc3980f5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3900d4090 con 0x7fc3900cee80 2024-04-25T10:11:55.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.754+0000 7fc3980f5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3900b6b20 con 0x7fc3900d3b90 2024-04-25T10:11:55.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.754+0000 7fc3980f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3900d4370 con 0x7fc39015dca0 2024-04-25T10:11:55.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.754+0000 7fc395690640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3900d3b90 0x7fc3900d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.754+0000 7fc396692640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc3900cee80 0x7fc3900b2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49468/0 (socket says 172.21.15.16:49468) 2024-04-25T10:11:55.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.754+0000 7fc395690640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3900d3b90 0x7fc3900d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49486/0 (socket says 172.21.15.16:49486) 2024-04-25T10:11:55.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.754+0000 7fc396692640 1 -- 172.21.15.16:0/1829021200 learned_addr learned my addr 172.21.15.16:0/1829021200 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:55.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.754+0000 7fc395e91640 1 --2- 172.21.15.16:0/1829021200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc39015dca0 0x7fc3900ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.754+0000 7fc395690640 1 -- 172.21.15.16:0/1829021200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc39015dca0 msgr2=0x7fc3900ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.754+0000 7fc395690640 1 --2- 172.21.15.16:0/1829021200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc39015dca0 0x7fc3900ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.754+0000 7fc395690640 1 -- 172.21.15.16:0/1829021200 >> v1:172.21.15.16:6789/0 conn(0x7fc3900cee80 legacy=0x7fc3900b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.754+0000 7fc394e8f640 1 -- 172.21.15.16:0/1829021200 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024710066 0 0) 0x7fc3900d4090 con 0x7fc3900cee80 2024-04-25T10:11:55.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.754+0000 7fc395690640 1 -- 172.21.15.16:0/1829021200 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3900d4650 con 0x7fc3900d3b90 2024-04-25T10:11:55.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.755+0000 7fc395690640 1 --2- 172.21.15.16:0/1829021200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3900d3b90 0x7fc3900d0a20 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fc3800097a0 tx=0x7fc38002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5699b0d32deab711 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.755+0000 7fc394e8f640 1 -- 172.21.15.16:0/1829021200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380037070 con 0x7fc3900d3b90 2024-04-25T10:11:55.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.755+0000 7fc394e8f640 1 -- 172.21.15.16:0/1829021200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc380032440 con 0x7fc3900d3b90 2024-04-25T10:11:55.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.755+0000 7fc394e8f640 1 -- 172.21.15.16:0/1829021200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380032810 con 0x7fc3900d3b90 2024-04-25T10:11:55.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.756+0000 7fc3980f5640 1 -- 172.21.15.16:0/1829021200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3900d3b90 msgr2=0x7fc3900d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.756+0000 7fc3980f5640 1 --2- 172.21.15.16:0/1829021200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3900d3b90 0x7fc3900d0a20 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fc3800097a0 tx=0x7fc38002a180 comp rx=0 tx=0).stop 2024-04-25T10:11:55.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.756+0000 7fc3980f5640 1 -- 172.21.15.16:0/1829021200 shutdown_connections 2024-04-25T10:11:55.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.756+0000 7fc3980f5640 1 --2- 172.21.15.16:0/1829021200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3900d3b90 0x7fc3900d0a20 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.756+0000 7fc3980f5640 1 --2- 172.21.15.16:0/1829021200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc39015dca0 0x7fc3900ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.756+0000 7fc3980f5640 1 -- 172.21.15.16:0/1829021200 >> 172.21.15.16:0/1829021200 conn(0x7fc3900726c0 msgr2=0x7fc390072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.756+0000 7fc3980f5640 1 -- 172.21.15.16:0/1829021200 shutdown_connections 2024-04-25T10:11:55.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.756+0000 7fc3980f5640 1 -- 172.21.15.16:0/1829021200 wait complete. 2024-04-25T10:11:55.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.757+0000 7fc3980f5640 1 Processor -- start 2024-04-25T10:11:55.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.757+0000 7fc3980f5640 1 -- start start 2024-04-25T10:11:55.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.758+0000 7fc3980f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3900cee80 0x7fc3900c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.758+0000 7fc3980f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3900d3b90 0x7fc3901e9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.758+0000 7fc3980f5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc39015dca0 0x7fc3901e9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.758+0000 7fc3980f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39015e200 con 0x7fc3900cee80 2024-04-25T10:11:55.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.758+0000 7fc3980f5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39015e080 con 0x7fc39015dca0 2024-04-25T10:11:55.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.758+0000 7fc3980f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3900d0f60 con 0x7fc3900d3b90 2024-04-25T10:11:55.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.758+0000 7fc395e91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3900cee80 0x7fc3900c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.758+0000 7fc395e91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3900cee80 0x7fc3900c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49494/0 (socket says 172.21.15.16:49494) 2024-04-25T10:11:55.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.758+0000 7fc395e91640 1 -- 172.21.15.16:0/3851439940 learned_addr learned my addr 172.21.15.16:0/3851439940 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:55.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.758+0000 7fc395690640 1 --2- 172.21.15.16:0/3851439940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3900d3b90 0x7fc3901e9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.758+0000 7fc396692640 1 --2- 172.21.15.16:0/3851439940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc39015dca0 0x7fc3901e9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.758+0000 7fc395e91640 1 -- 172.21.15.16:0/3851439940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3900d3b90 msgr2=0x7fc3901e9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.758+0000 7fc395e91640 1 --2- 172.21.15.16:0/3851439940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3900d3b90 0x7fc3901e9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.758+0000 7fc395e91640 1 -- 172.21.15.16:0/3851439940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc39015dca0 msgr2=0x7fc3901e9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.758+0000 7fc395e91640 1 --2- 172.21.15.16:0/3851439940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc39015dca0 0x7fc3901e9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.758+0000 7fc395e91640 1 -- 172.21.15.16:0/3851439940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc390147e40 con 0x7fc3900cee80 2024-04-25T10:11:55.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.758+0000 7fc395e91640 1 --2- 172.21.15.16:0/3851439940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3900cee80 0x7fc3900c9fd0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fc38c00d600 tx=0x7fc38c00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.759+0000 7fc3867fc640 1 -- 172.21.15.16:0/3851439940 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c0125f0 con 0x7fc3900cee80 2024-04-25T10:11:55.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.759+0000 7fc3867fc640 1 -- 172.21.15.16:0/3851439940 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c019070 con 0x7fc3900cee80 2024-04-25T10:11:55.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.759+0000 7fc3867fc640 1 -- 172.21.15.16:0/3851439940 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c0129f0 con 0x7fc3900cee80 2024-04-25T10:11:55.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.759+0000 7fc3980f5640 1 -- 172.21.15.16:0/3851439940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc390148120 con 0x7fc3900cee80 2024-04-25T10:11:55.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.759+0000 7fc3980f5640 1 -- 172.21.15.16:0/3851439940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3901486e0 con 0x7fc3900cee80 2024-04-25T10:11:55.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.760+0000 7fc3980f5640 1 -- 172.21.15.16:0/3851439940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3900750b0 con 0x7fc3900cee80 2024-04-25T10:11:55.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.760+0000 7fc3867fc640 1 -- 172.21.15.16:0/3851439940 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc38c012b90 con 0x7fc3900cee80 2024-04-25T10:11:55.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.761+0000 7fc3867fc640 1 --2- 172.21.15.16:0/3851439940 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc37406b4d0 0x7fc37406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.761+0000 7fc3867fc640 1 -- 172.21.15.16:0/3851439940 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fc38c08a6a0 con 0x7fc3900cee80 2024-04-25T10:11:55.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.761+0000 7fc395690640 1 --2- 172.21.15.16:0/3851439940 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc37406b4d0 0x7fc37406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.761+0000 7fc395690640 1 --2- 172.21.15.16:0/3851439940 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc37406b4d0 0x7fc37406d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc380034000 tx=0x7fc38002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.765+0000 7fc3867fc640 1 -- 172.21.15.16:0/3851439940 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc38c059500 con 0x7fc3900cee80 2024-04-25T10:11:55.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:55.989+0000 7fc3980f5640 1 -- 172.21.15.16:0/3851439940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fc3900d3400 con 0x7fc3900cee80 2024-04-25T10:11:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.236+0000 7fc3867fc640 1 -- 172.21.15.16:0/3851439940 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v60) v1 ==== 162+0+0 (secure 0 0 0) 0x7fc38c058e00 con 0x7fc3900cee80 2024-04-25T10:11:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:11:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.238+0000 7fc3980f5640 1 -- 172.21.15.16:0/3851439940 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc37406b4d0 msgr2=0x7fc37406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.238+0000 7fc3980f5640 1 --2- 172.21.15.16:0/3851439940 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc37406b4d0 0x7fc37406d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc380034000 tx=0x7fc38002e000 comp rx=0 tx=0).stop 2024-04-25T10:11:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.238+0000 7fc3980f5640 1 -- 172.21.15.16:0/3851439940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3900cee80 msgr2=0x7fc3900c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.238+0000 7fc3980f5640 1 --2- 172.21.15.16:0/3851439940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3900cee80 0x7fc3900c9fd0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fc38c00d600 tx=0x7fc38c00dac0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.239+0000 7fc3980f5640 1 -- 172.21.15.16:0/3851439940 shutdown_connections 2024-04-25T10:11:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.239+0000 7fc3980f5640 1 --2- 172.21.15.16:0/3851439940 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc37406b4d0 0x7fc37406d990 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.239+0000 7fc3980f5640 1 --2- 172.21.15.16:0/3851439940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc39015dca0 0x7fc3901e9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.239+0000 7fc3980f5640 1 --2- 172.21.15.16:0/3851439940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3900d3b90 0x7fc3901e9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.239+0000 7fc3980f5640 1 --2- 172.21.15.16:0/3851439940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3900cee80 0x7fc3900c9fd0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.239+0000 7fc3980f5640 1 -- 172.21.15.16:0/3851439940 >> 172.21.15.16:0/3851439940 conn(0x7fc3900726c0 msgr2=0x7fc3900b8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.239+0000 7fc3980f5640 1 -- 172.21.15.16:0/3851439940 shutdown_connections 2024-04-25T10:11:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.239+0000 7fc3980f5640 1 -- 172.21.15.16:0/3851439940 wait complete. 2024-04-25T10:11:56.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.352+0000 7f320966f640 1 Processor -- start 2024-04-25T10:11:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.352+0000 7f320966f640 1 -- start start 2024-04-25T10:11:56.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.353+0000 7f320966f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f320415bf00 0x7f32040cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.353+0000 7f320966f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f320406e8f0 0x7f32041654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.353+0000 7f320966f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32040ce6c0 con 0x7f32040cc310 2024-04-25T10:11:56.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.353+0000 7f320966f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32041455e0 con 0x7f320406e8f0 2024-04-25T10:11:56.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.353+0000 7f320966f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f320406d450 con 0x7f320415bf00 2024-04-25T10:11:56.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.353+0000 7f32037fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f320406e8f0 0x7f32041654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.353+0000 7f32027fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f32040cc310 0x7f320415c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49470/0 (socket says 172.21.15.16:49470) 2024-04-25T10:11:56.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.353+0000 7f32027fc640 1 -- 172.21.15.16:0/242641001 learned_addr learned my addr 172.21.15.16:0/242641001 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:56.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.353+0000 7f3202ffd640 1 --2- 172.21.15.16:0/242641001 >> v2:172.21.15.186:3300/0 conn(0x7f320415bf00 0x7f32040cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.353+0000 7f32037fe640 1 -- 172.21.15.16:0/242641001 >> v2:172.21.15.186:3300/0 conn(0x7f320415bf00 msgr2=0x7f32040cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.353+0000 7f32037fe640 1 --2- 172.21.15.16:0/242641001 >> v2:172.21.15.186:3300/0 conn(0x7f320415bf00 0x7f32040cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.353+0000 7f32037fe640 1 -- 172.21.15.16:0/242641001 >> v1:172.21.15.16:6789/0 conn(0x7f32040cc310 legacy=0x7f320415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.353+0000 7f32037fe640 1 -- 172.21.15.16:0/242641001 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32040cc7b0 con 0x7f320406e8f0 2024-04-25T10:11:56.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.353+0000 7f3201ffb640 1 -- 172.21.15.16:0/242641001 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1872326729 0 0) 0x7f31ec001110 con 0x7f32040cc310 2024-04-25T10:11:56.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.354+0000 7f32037fe640 1 --2- 172.21.15.16:0/242641001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f320406e8f0 0x7f32041654b0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f320406ba10 tx=0x7f31f402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f84323a41ead664 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.354+0000 7f3201ffb640 1 -- 172.21.15.16:0/242641001 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4007740 con 0x7f320406e8f0 2024-04-25T10:11:56.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.354+0000 7f3201ffb640 1 -- 172.21.15.16:0/242641001 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f40078e0 con 0x7f320406e8f0 2024-04-25T10:11:56.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.354+0000 7f3201ffb640 1 -- 172.21.15.16:0/242641001 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4007cb0 con 0x7f320406e8f0 2024-04-25T10:11:56.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.355+0000 7f320966f640 1 -- 172.21.15.16:0/242641001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f320406e8f0 msgr2=0x7f32041654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.355+0000 7f320966f640 1 --2- 172.21.15.16:0/242641001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f320406e8f0 0x7f32041654b0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f320406ba10 tx=0x7f31f402a680 comp rx=0 tx=0).stop 2024-04-25T10:11:56.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.355+0000 7f320966f640 1 -- 172.21.15.16:0/242641001 shutdown_connections 2024-04-25T10:11:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.355+0000 7f320966f640 1 --2- 172.21.15.16:0/242641001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f320406e8f0 0x7f32041654b0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.355+0000 7f320966f640 1 --2- 172.21.15.16:0/242641001 >> v2:172.21.15.186:3300/0 conn(0x7f320415bf00 0x7f32040cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.355+0000 7f320966f640 1 -- 172.21.15.16:0/242641001 >> 172.21.15.16:0/242641001 conn(0x7f320406aba0 msgr2=0x7f320406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.355+0000 7f320966f640 1 -- 172.21.15.16:0/242641001 shutdown_connections 2024-04-25T10:11:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.355+0000 7f320966f640 1 -- 172.21.15.16:0/242641001 wait complete. 2024-04-25T10:11:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.356+0000 7f320966f640 1 Processor -- start 2024-04-25T10:11:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.356+0000 7f320966f640 1 -- start start 2024-04-25T10:11:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.357+0000 7f320966f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f320406e8f0 0x7f3204137fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.357+0000 7f320966f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32040cc310 0x7f3204139560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.357+0000 7f320966f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320415bf00 0x7f3204139aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.357+0000 7f320966f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32040cd050 con 0x7f32040cc310 2024-04-25T10:11:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.357+0000 7f320966f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32040ccd70 con 0x7f320415bf00 2024-04-25T10:11:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.357+0000 7f320966f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320406ee60 con 0x7f320406e8f0 2024-04-25T10:11:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.357+0000 7f32027fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32040cc310 0x7f3204139560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.357+0000 7f32027fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32040cc310 0x7f3204139560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49512/0 (socket says 172.21.15.16:49512) 2024-04-25T10:11:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.357+0000 7f32027fc640 1 -- 172.21.15.16:0/2538920873 learned_addr learned my addr 172.21.15.16:0/2538920873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.357+0000 7f32037fe640 1 --2- 172.21.15.16:0/2538920873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320415bf00 0x7f3204139aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.357+0000 7f3202ffd640 1 --2- 172.21.15.16:0/2538920873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f320406e8f0 0x7f3204137fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.357+0000 7f32027fc640 1 -- 172.21.15.16:0/2538920873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f320406e8f0 msgr2=0x7f3204137fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.357+0000 7f32027fc640 1 --2- 172.21.15.16:0/2538920873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f320406e8f0 0x7f3204137fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.357+0000 7f32027fc640 1 -- 172.21.15.16:0/2538920873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320415bf00 msgr2=0x7f3204139aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.357+0000 7f32027fc640 1 --2- 172.21.15.16:0/2538920873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320415bf00 0x7f3204139aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.358+0000 7f32027fc640 1 -- 172.21.15.16:0/2538920873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32041e5910 con 0x7f32040cc310 2024-04-25T10:11:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.358+0000 7f32027fc640 1 --2- 172.21.15.16:0/2538920873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32040cc310 0x7f3204139560 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00c780 tx=0x7f31ec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.358+0000 7f31fb7fe640 1 -- 172.21.15.16:0/2538920873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec018070 con 0x7f32040cc310 2024-04-25T10:11:56.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.358+0000 7f31fb7fe640 1 -- 172.21.15.16:0/2538920873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec0072d0 con 0x7f32040cc310 2024-04-25T10:11:56.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.358+0000 7f320966f640 1 -- 172.21.15.16:0/2538920873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32041e5bf0 con 0x7f32040cc310 2024-04-25T10:11:56.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.358+0000 7f31fb7fe640 1 -- 172.21.15.16:0/2538920873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec0047e0 con 0x7f32040cc310 2024-04-25T10:11:56.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.358+0000 7f320966f640 1 -- 172.21.15.16:0/2538920873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320413bb90 con 0x7f32040cc310 2024-04-25T10:11:56.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.359+0000 7f320966f640 1 -- 172.21.15.16:0/2538920873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320406bb30 con 0x7f32040cc310 2024-04-25T10:11:56.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.359+0000 7f31fb7fe640 1 -- 172.21.15.16:0/2538920873 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f31ec01b070 con 0x7f32040cc310 2024-04-25T10:11:56.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.360+0000 7f31fb7fe640 1 --2- 172.21.15.16:0/2538920873 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f31e406b4d0 0x7f31e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.360+0000 7f31fb7fe640 1 -- 172.21.15.16:0/2538920873 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f31ec089510 con 0x7f32040cc310 2024-04-25T10:11:56.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.360+0000 7f3202ffd640 1 --2- 172.21.15.16:0/2538920873 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f31e406b4d0 0x7f31e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.361+0000 7f3202ffd640 1 --2- 172.21.15.16:0/2538920873 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f31e406b4d0 0x7f31e406d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0079a0 tx=0x7f31fc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.364+0000 7f31fb7fe640 1 -- 172.21.15.16:0/2538920873 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f32040ce6c0 con 0x7f32040cc310 2024-04-25T10:11:56.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.582+0000 7f320966f640 1 -- 172.21.15.16:0/2538920873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32040749b0 con 0x7f32040cc310 2024-04-25T10:11:56.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.582+0000 7f31fb7fe640 1 -- 172.21.15.16:0/2538920873 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f31ec057c90 con 0x7f32040cc310 2024-04-25T10:11:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:56.587 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4044509901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4044509901},{"type":"v1","addr":"172.21.15.16:6835","nonce":4044509901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1494230493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1494230493},{"type":"v1","addr":"172.21.15.186:6837","nonce":1494230493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/1050183001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1050183001},{"type":"v1","addr":"172.21.15.186:6836","nonce":1050183001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:00.015075+0000","modified":"2024-04-25T10:11:56.239184+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7323},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7323":{"gid":7323,"name":"a","rank":0,"incarnation":61,"state":"up:replay","state_seq":1,"addr":"172.21.15.16:6834/84349474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":84349474},{"type":"v1","addr":"172.21.15.16:6834","nonce":84349474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:56.588 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 61 2024-04-25T10:11:56.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.584+0000 7f320966f640 1 -- 172.21.15.16:0/2538920873 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f31e406b4d0 msgr2=0x7f31e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.584+0000 7f320966f640 1 --2- 172.21.15.16:0/2538920873 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f31e406b4d0 0x7f31e406d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0079a0 tx=0x7f31fc008040 comp rx=0 tx=0).stop 2024-04-25T10:11:56.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.584+0000 7f320966f640 1 -- 172.21.15.16:0/2538920873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32040cc310 msgr2=0x7f3204139560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.584+0000 7f320966f640 1 --2- 172.21.15.16:0/2538920873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32040cc310 0x7f3204139560 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00c780 tx=0x7f31ec00cc40 comp rx=0 tx=0).stop 2024-04-25T10:11:56.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.584+0000 7f320966f640 1 -- 172.21.15.16:0/2538920873 shutdown_connections 2024-04-25T10:11:56.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.584+0000 7f320966f640 1 --2- 172.21.15.16:0/2538920873 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f31e406b4d0 0x7f31e406d990 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.584+0000 7f320966f640 1 --2- 172.21.15.16:0/2538920873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320415bf00 0x7f3204139aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.584+0000 7f320966f640 1 --2- 172.21.15.16:0/2538920873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32040cc310 0x7f3204139560 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.584+0000 7f320966f640 1 --2- 172.21.15.16:0/2538920873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f320406e8f0 0x7f3204137fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.584+0000 7f320966f640 1 -- 172.21.15.16:0/2538920873 >> 172.21.15.16:0/2538920873 conn(0x7f320406aba0 msgr2=0x7f320415eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.584+0000 7f320966f640 1 -- 172.21.15.16:0/2538920873 shutdown_connections 2024-04-25T10:11:56.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:56.584+0000 7f320966f640 1 -- 172.21.15.16:0/2538920873 wait complete. 2024-04-25T10:11:57.003 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:00.015075+0000', 'modified': '2024-04-25T10:11:56.239184+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 88, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7323}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7323': {'gid': 7323, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.16:6834/84349474', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6832', 'nonce': 84349474}, {'type': 'v1', 'addr': '172.21.15.16:6834', 'nonce': 84349474}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:57.003 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7323:up:replay 2024-04-25T10:11:58.004 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.113+0000 7f4e5f488640 1 Processor -- start 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.114+0000 7f4e5f488640 1 -- start start 2024-04-25T10:11:58.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.114+0000 7f4e5f488640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4e5815dc30 0x7f4e5806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.114+0000 7f4e5f488640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e580b1b30 0x7f4e580d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.114+0000 7f4e5f488640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e5806c1f0 con 0x7f4e5806cc00 2024-04-25T10:11:58.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.114+0000 7f4e5f488640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e580cfb40 con 0x7f4e580b1b30 2024-04-25T10:11:58.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.114+0000 7f4e5f488640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e580b2060 con 0x7f4e5815dc30 2024-04-25T10:11:58.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.115+0000 7f4e5da25640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e580b1b30 0x7f4e580d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.115+0000 7f4e5ca23640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4e5806cc00 0x7f4e580b1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49474/0 (socket says 172.21.15.16:49474) 2024-04-25T10:11:58.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.115+0000 7f4e5ca23640 1 -- 172.21.15.16:0/3435946612 learned_addr learned my addr 172.21.15.16:0/3435946612 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:58.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.115+0000 7f4e5d224640 1 --2- 172.21.15.16:0/3435946612 >> v2:172.21.15.186:3300/0 conn(0x7f4e5815dc30 0x7f4e5806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.115+0000 7f4e5da25640 1 -- 172.21.15.16:0/3435946612 >> v2:172.21.15.186:3300/0 conn(0x7f4e5815dc30 msgr2=0x7f4e5806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:58.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.115+0000 7f4e5da25640 1 --2- 172.21.15.16:0/3435946612 >> v2:172.21.15.186:3300/0 conn(0x7f4e5815dc30 0x7f4e5806c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.115+0000 7f4e5da25640 1 -- 172.21.15.16:0/3435946612 >> v1:172.21.15.16:6789/0 conn(0x7f4e5806cc00 legacy=0x7f4e580b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.115+0000 7f4e5da25640 1 -- 172.21.15.16:0/3435946612 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e580b2340 con 0x7f4e580b1b30 2024-04-25T10:11:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.115+0000 7f4e4ffff640 1 -- 172.21.15.16:0/3435946612 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424536397 0 0) 0x7f4e48001110 con 0x7f4e5806cc00 2024-04-25T10:11:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.115+0000 7f4e5d224640 1 --2- 172.21.15.16:0/3435946612 >> v2:172.21.15.186:3300/0 conn(0x7f4e5815dc30 0x7f4e5806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.116+0000 7f4e5da25640 1 --2- 172.21.15.16:0/3435946612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e580b1b30 0x7f4e580d0300 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f4e5809ab70 tx=0x7f4e5402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc86c28614822f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.116+0000 7f4e4ffff640 1 -- 172.21.15.16:0/3435946612 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e54037070 con 0x7f4e580b1b30 2024-04-25T10:11:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.116+0000 7f4e4ffff640 1 -- 172.21.15.16:0/3435946612 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e54032440 con 0x7f4e580b1b30 2024-04-25T10:11:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.116+0000 7f4e4ffff640 1 -- 172.21.15.16:0/3435946612 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e54032810 con 0x7f4e580b1b30 2024-04-25T10:11:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.116+0000 7f4e5f488640 1 -- 172.21.15.16:0/3435946612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e580b1b30 msgr2=0x7f4e580d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.116+0000 7f4e5f488640 1 --2- 172.21.15.16:0/3435946612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e580b1b30 0x7f4e580d0300 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f4e5809ab70 tx=0x7f4e5402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.116+0000 7f4e5f488640 1 -- 172.21.15.16:0/3435946612 shutdown_connections 2024-04-25T10:11:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.116+0000 7f4e5f488640 1 --2- 172.21.15.16:0/3435946612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e580b1b30 0x7f4e580d0300 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.116+0000 7f4e5f488640 1 --2- 172.21.15.16:0/3435946612 >> v2:172.21.15.186:3300/0 conn(0x7f4e5815dc30 0x7f4e5806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.116+0000 7f4e5f488640 1 -- 172.21.15.16:0/3435946612 >> 172.21.15.16:0/3435946612 conn(0x7f4e580d1900 msgr2=0x7f4e580d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.117+0000 7f4e5f488640 1 -- 172.21.15.16:0/3435946612 shutdown_connections 2024-04-25T10:11:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.117+0000 7f4e5f488640 1 -- 172.21.15.16:0/3435946612 wait complete. 2024-04-25T10:11:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.118+0000 7f4e5f488640 1 Processor -- start 2024-04-25T10:11:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.118+0000 7f4e5f488640 1 -- start start 2024-04-25T10:11:58.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e5f488640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5806cc00 0x7f4e58158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e5f488640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e580b1b30 0x7f4e580c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e5f488640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e5815dc30 0x7f4e580c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e5f488640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5815e010 con 0x7f4e5806cc00 2024-04-25T10:11:58.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e5f488640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580cfde0 con 0x7f4e580b1b30 2024-04-25T10:11:58.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e5f488640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580d0840 con 0x7f4e5815dc30 2024-04-25T10:11:58.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e5d224640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5806cc00 0x7f4e58158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e5d224640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5806cc00 0x7f4e58158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49528/0 (socket says 172.21.15.16:49528) 2024-04-25T10:11:58.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e5d224640 1 -- 172.21.15.16:0/631321922 learned_addr learned my addr 172.21.15.16:0/631321922 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:58.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e5ca23640 1 --2- 172.21.15.16:0/631321922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e580b1b30 0x7f4e580c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e5da25640 1 --2- 172.21.15.16:0/631321922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e5815dc30 0x7f4e580c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e5d224640 1 -- 172.21.15.16:0/631321922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e5815dc30 msgr2=0x7f4e580c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e5d224640 1 --2- 172.21.15.16:0/631321922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e5815dc30 0x7f4e580c7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e5d224640 1 -- 172.21.15.16:0/631321922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e580b1b30 msgr2=0x7f4e580c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e5d224640 1 --2- 172.21.15.16:0/631321922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e580b1b30 0x7f4e580c73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e5d224640 1 -- 172.21.15.16:0/631321922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e580b5430 con 0x7f4e5806cc00 2024-04-25T10:11:58.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e5d224640 1 --2- 172.21.15.16:0/631321922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5806cc00 0x7f4e58158120 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000ce30 tx=0x7f4e5000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e4dffb640 1 -- 172.21.15.16:0/631321922 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5000f890 con 0x7f4e5806cc00 2024-04-25T10:11:58.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e4dffb640 1 -- 172.21.15.16:0/631321922 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5000fa30 con 0x7f4e5806cc00 2024-04-25T10:11:58.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e5f488640 1 -- 172.21.15.16:0/631321922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e580c5d20 con 0x7f4e5806cc00 2024-04-25T10:11:58.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.119+0000 7f4e4dffb640 1 -- 172.21.15.16:0/631321922 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5000fe00 con 0x7f4e5806cc00 2024-04-25T10:11:58.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.120+0000 7f4e5f488640 1 -- 172.21.15.16:0/631321922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e580c62e0 con 0x7f4e5806cc00 2024-04-25T10:11:58.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.121+0000 7f4e5f488640 1 -- 172.21.15.16:0/631321922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e58067850 con 0x7f4e5806cc00 2024-04-25T10:11:58.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.121+0000 7f4e4dffb640 1 -- 172.21.15.16:0/631321922 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e50018400 con 0x7f4e5806cc00 2024-04-25T10:11:58.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.122+0000 7f4e4dffb640 1 --2- 172.21.15.16:0/631321922 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4e3c06b4d0 0x7f4e3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.122+0000 7f4e4dffb640 1 -- 172.21.15.16:0/631321922 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f4e5008b0e0 con 0x7f4e5806cc00 2024-04-25T10:11:58.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.122+0000 7f4e5ca23640 1 --2- 172.21.15.16:0/631321922 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4e3c06b4d0 0x7f4e3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.122+0000 7f4e5ca23640 1 --2- 172.21.15.16:0/631321922 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4e3c06b4d0 0x7f4e3c06d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4e5815fca0 tx=0x7f4e48009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.126+0000 7f4e4dffb640 1 -- 172.21.15.16:0/631321922 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e50014030 con 0x7f4e5806cc00 2024-04-25T10:11:58.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.345+0000 7f4e5f488640 1 -- 172.21.15.16:0/631321922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e580749b0 con 0x7f4e5806cc00 2024-04-25T10:11:58.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.345+0000 7f4e4dffb640 1 -- 172.21.15.16:0/631321922 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f4e5000ae20 con 0x7f4e5806cc00 2024-04-25T10:11:58.350 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:11:58.351 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4044509901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4044509901},{"type":"v1","addr":"172.21.15.16:6835","nonce":4044509901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1494230493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1494230493},{"type":"v1","addr":"172.21.15.186:6837","nonce":1494230493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/1050183001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1050183001},{"type":"v1","addr":"172.21.15.186:6836","nonce":1050183001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:00.015075+0000","modified":"2024-04-25T10:11:57.244104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7323},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7323":{"gid":7323,"name":"a","rank":0,"incarnation":61,"state":"up:rejoin","state_seq":18,"addr":"172.21.15.16:6834/84349474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":84349474},{"type":"v1","addr":"172.21.15.16:6834","nonce":84349474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:58.351 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 63 2024-04-25T10:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.348+0000 7f4e5f488640 1 -- 172.21.15.16:0/631321922 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4e3c06b4d0 msgr2=0x7f4e3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.348+0000 7f4e5f488640 1 --2- 172.21.15.16:0/631321922 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4e3c06b4d0 0x7f4e3c06d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4e5815fca0 tx=0x7f4e48009040 comp rx=0 tx=0).stop 2024-04-25T10:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.348+0000 7f4e5f488640 1 -- 172.21.15.16:0/631321922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5806cc00 msgr2=0x7f4e58158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.348+0000 7f4e5f488640 1 --2- 172.21.15.16:0/631321922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5806cc00 0x7f4e58158120 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000ce30 tx=0x7f4e5000a650 comp rx=0 tx=0).stop 2024-04-25T10:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.348+0000 7f4e5f488640 1 -- 172.21.15.16:0/631321922 shutdown_connections 2024-04-25T10:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.348+0000 7f4e5f488640 1 --2- 172.21.15.16:0/631321922 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4e3c06b4d0 0x7f4e3c06d990 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.348+0000 7f4e5f488640 1 --2- 172.21.15.16:0/631321922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e5815dc30 0x7f4e580c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.348+0000 7f4e5f488640 1 --2- 172.21.15.16:0/631321922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e580b1b30 0x7f4e580c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.348+0000 7f4e5f488640 1 --2- 172.21.15.16:0/631321922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5806cc00 0x7f4e58158120 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.348+0000 7f4e5f488640 1 -- 172.21.15.16:0/631321922 >> 172.21.15.16:0/631321922 conn(0x7f4e580d1900 msgr2=0x7f4e580d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.348+0000 7f4e5f488640 1 -- 172.21.15.16:0/631321922 shutdown_connections 2024-04-25T10:11:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:58.348+0000 7f4e5f488640 1 -- 172.21.15.16:0/631321922 wait complete. 2024-04-25T10:11:58.766 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:00.015075+0000', 'modified': '2024-04-25T10:11:57.244104+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 88, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7323}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7323': {'gid': 7323, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:rejoin', 'state_seq': 18, 'addr': '172.21.15.16:6834/84349474', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6832', 'nonce': 84349474}, {'type': 'v1', 'addr': '172.21.15.16:6834', 'nonce': 84349474}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:58.766 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7323:up:rejoin 2024-04-25T10:11:59.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.876+0000 7f870d6f5640 1 Processor -- start 2024-04-25T10:11:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.876+0000 7f870d6f5640 1 -- start start 2024-04-25T10:11:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.877+0000 7f870d6f5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f87080b9460 0x7f87080ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.877+0000 7f870d6f5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87080d2440 con 0x7f870815dd50 2024-04-25T10:11:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.877+0000 7f870d6f5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87080d2720 con 0x7f87080d34e0 2024-04-25T10:11:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.877+0000 7f870d6f5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87080d2a00 con 0x7f87080b9460 2024-04-25T10:11:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.877+0000 7f8706ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f870815dd50 0x7f87080d1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49476/0 (socket says 172.21.15.16:49476) 2024-04-25T10:11:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.877+0000 7f8706ffd640 1 -- 172.21.15.16:0/700878918 learned_addr learned my addr 172.21.15.16:0/700878918 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:11:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.877+0000 7f86fffff640 1 --2- 172.21.15.16:0/700878918 >> v2:172.21.15.186:3300/0 conn(0x7f87080b9460 0x7f87080ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.877+0000 7f87067fc640 1 -- 172.21.15.16:0/700878918 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181410735 0 0) 0x7f87080d2440 con 0x7f870815dd50 2024-04-25T10:11:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.877+0000 7f87067fc640 1 -- 172.21.15.16:0/700878918 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f4003520 con 0x7f870815dd50 2024-04-25T10:11:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.878+0000 7f87067fc640 1 -- 172.21.15.16:0/700878918 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4077520681 0 0) 0x7f86f4003520 con 0x7f870815dd50 2024-04-25T10:11:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.878+0000 7f86fffff640 1 -- 172.21.15.16:0/700878918 >> v1:172.21.15.186:6789/0 conn(0x7f87080d34e0 legacy=0x7f87080b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.878+0000 7f86fffff640 1 -- 172.21.15.16:0/700878918 >> v1:172.21.15.16:6789/0 conn(0x7f870815dd50 legacy=0x7f87080d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.878+0000 7f86fffff640 1 -- 172.21.15.16:0/700878918 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87080d2ce0 con 0x7f87080b9460 2024-04-25T10:11:59.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.878+0000 7f86fffff640 1 --2- 172.21.15.16:0/700878918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87080b9460 0x7f87080ce7a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f86f00097a0 tx=0x7f86f002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbac7a29a261ede7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.879+0000 7f87067fc640 1 -- 172.21.15.16:0/700878918 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0037070 con 0x7f87080b9460 2024-04-25T10:11:59.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.879+0000 7f87067fc640 1 -- 172.21.15.16:0/700878918 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f002abc0 con 0x7f87080b9460 2024-04-25T10:11:59.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.879+0000 7f87067fc640 1 -- 172.21.15.16:0/700878918 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f8001670 con 0x7f87080b9460 2024-04-25T10:11:59.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.879+0000 7f870d6f5640 1 -- 172.21.15.16:0/700878918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87080b9460 msgr2=0x7f87080ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.879+0000 7f870d6f5640 1 --2- 172.21.15.16:0/700878918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87080b9460 0x7f87080ce7a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f86f00097a0 tx=0x7f86f002a130 comp rx=0 tx=0).stop 2024-04-25T10:11:59.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.879+0000 7f870d6f5640 1 -- 172.21.15.16:0/700878918 shutdown_connections 2024-04-25T10:11:59.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.879+0000 7f870d6f5640 1 --2- 172.21.15.16:0/700878918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87080b9460 0x7f87080ce7a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.879+0000 7f870d6f5640 1 -- 172.21.15.16:0/700878918 >> 172.21.15.16:0/700878918 conn(0x7f87080722d0 msgr2=0x7f87080726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:59.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.879+0000 7f870d6f5640 1 -- 172.21.15.16:0/700878918 shutdown_connections 2024-04-25T10:11:59.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.879+0000 7f870d6f5640 1 -- 172.21.15.16:0/700878918 wait complete. 2024-04-25T10:11:59.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.880+0000 7f870d6f5640 1 Processor -- start 2024-04-25T10:11:59.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.880+0000 7f870d6f5640 1 -- start start 2024-04-25T10:11:59.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.881+0000 7f870d6f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f87080b9460 0x7f87080c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.881+0000 7f870d6f5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87080d34e0 0x7f870814c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.881+0000 7f870d6f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870815dd50 0x7f870815abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.881+0000 7f870d6f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87080cee60 con 0x7f870815dd50 2024-04-25T10:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.881+0000 7f870d6f5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87080cece0 con 0x7f87080d34e0 2024-04-25T10:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.881+0000 7f870d6f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87080cefe0 con 0x7f87080b9460 2024-04-25T10:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.881+0000 7f87077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870815dd50 0x7f870815abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.881+0000 7f8706ffd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f87080b9460 0x7f87080c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.881+0000 7f86fffff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87080d34e0 0x7f870814c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.881+0000 7f87077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870815dd50 0x7f870815abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49536/0 (socket says 172.21.15.16:49536) 2024-04-25T10:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.881+0000 7f87077fe640 1 -- 172.21.15.16:0/4246254206 learned_addr learned my addr 172.21.15.16:0/4246254206 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.882+0000 7f87077fe640 1 -- 172.21.15.16:0/4246254206 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f87080b9460 msgr2=0x7f87080c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.882+0000 7f87077fe640 1 --2- 172.21.15.16:0/4246254206 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f87080b9460 0x7f87080c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.882+0000 7f87077fe640 1 -- 172.21.15.16:0/4246254206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87080d34e0 msgr2=0x7f870814c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.882+0000 7f87077fe640 1 --2- 172.21.15.16:0/4246254206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87080d34e0 0x7f870814c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.882+0000 7f87077fe640 1 -- 172.21.15.16:0/4246254206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870815e130 con 0x7f870815dd50 2024-04-25T10:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.882+0000 7f87077fe640 1 --2- 172.21.15.16:0/4246254206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870815dd50 0x7f870815abc0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f870000c760 tx=0x7f870000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.882+0000 7f86ff7fe640 1 -- 172.21.15.16:0/4246254206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8700003b70 con 0x7f870815dd50 2024-04-25T10:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.882+0000 7f86ff7fe640 1 -- 172.21.15.16:0/4246254206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8700003d30 con 0x7f870815dd50 2024-04-25T10:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.882+0000 7f870d6f5640 1 -- 172.21.15.16:0/4246254206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87080b59c0 con 0x7f870815dd50 2024-04-25T10:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.882+0000 7f86ff7fe640 1 -- 172.21.15.16:0/4246254206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870000aa90 con 0x7f870815dd50 2024-04-25T10:11:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.882+0000 7f870d6f5640 1 -- 172.21.15.16:0/4246254206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87080b5f80 con 0x7f870815dd50 2024-04-25T10:11:59.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.883+0000 7f870d6f5640 1 -- 172.21.15.16:0/4246254206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8708067850 con 0x7f870815dd50 2024-04-25T10:11:59.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.884+0000 7f86ff7fe640 1 -- 172.21.15.16:0/4246254206 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f870000ac30 con 0x7f870815dd50 2024-04-25T10:11:59.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.884+0000 7f86ff7fe640 1 --2- 172.21.15.16:0/4246254206 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f86e806b500 0x7f86e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.884+0000 7f86ff7fe640 1 -- 172.21.15.16:0/4246254206 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f8700089f80 con 0x7f870815dd50 2024-04-25T10:11:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.884+0000 7f8706ffd640 1 --2- 172.21.15.16:0/4246254206 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f86e806b500 0x7f86e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.885+0000 7f8706ffd640 1 --2- 172.21.15.16:0/4246254206 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f86e806b500 0x7f86e806d9c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f86f8005880 tx=0x7f86f8000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:11:59.888+0000 7f86ff7fe640 1 -- 172.21.15.16:0/4246254206 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8700058790 con 0x7f870815dd50 2024-04-25T10:12:00.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.108+0000 7f870d6f5640 1 -- 172.21.15.16:0/4246254206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87080d1740 con 0x7f870815dd50 2024-04-25T10:12:00.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.108+0000 7f86ff7fe640 1 -- 172.21.15.16:0/4246254206 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f870005c830 con 0x7f870815dd50 2024-04-25T10:12:00.113 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:00.114 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4044509901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4044509901},{"type":"v1","addr":"172.21.15.16:6835","nonce":4044509901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1494230493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1494230493},{"type":"v1","addr":"172.21.15.186:6837","nonce":1494230493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/1050183001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1050183001},{"type":"v1","addr":"172.21.15.186:6836","nonce":1050183001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:00.015075+0000","modified":"2024-04-25T10:11:58.248543+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7323},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7323":{"gid":7323,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":19,"addr":"172.21.15.16:6834/84349474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":84349474},{"type":"v1","addr":"172.21.15.16:6834","nonce":84349474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:12:00.114 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 64 2024-04-25T10:12:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.110+0000 7f870d6f5640 1 -- 172.21.15.16:0/4246254206 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f86e806b500 msgr2=0x7f86e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.110+0000 7f870d6f5640 1 --2- 172.21.15.16:0/4246254206 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f86e806b500 0x7f86e806d9c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f86f8005880 tx=0x7f86f8000bc0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.110+0000 7f870d6f5640 1 -- 172.21.15.16:0/4246254206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870815dd50 msgr2=0x7f870815abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.110+0000 7f870d6f5640 1 --2- 172.21.15.16:0/4246254206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870815dd50 0x7f870815abc0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f870000c760 tx=0x7f870000cc20 comp rx=0 tx=0).stop 2024-04-25T10:12:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.110+0000 7f870d6f5640 1 -- 172.21.15.16:0/4246254206 shutdown_connections 2024-04-25T10:12:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.110+0000 7f870d6f5640 1 --2- 172.21.15.16:0/4246254206 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f86e806b500 0x7f86e806d9c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.110+0000 7f870d6f5640 1 --2- 172.21.15.16:0/4246254206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870815dd50 0x7f870815abc0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.110+0000 7f870d6f5640 1 --2- 172.21.15.16:0/4246254206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87080d34e0 0x7f870814c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.110+0000 7f870d6f5640 1 --2- 172.21.15.16:0/4246254206 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f87080b9460 0x7f87080c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.110+0000 7f870d6f5640 1 -- 172.21.15.16:0/4246254206 >> 172.21.15.16:0/4246254206 conn(0x7f87080722d0 msgr2=0x7f8708159580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.111+0000 7f870d6f5640 1 -- 172.21.15.16:0/4246254206 shutdown_connections 2024-04-25T10:12:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.111+0000 7f870d6f5640 1 -- 172.21.15.16:0/4246254206 wait complete. 2024-04-25T10:12:00.529 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:00.015075+0000', 'modified': '2024-04-25T10:11:58.248543+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 88, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7323}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7323': {'gid': 7323, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.16:6834/84349474', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6832', 'nonce': 84349474}, {'type': 'v1', 'addr': '172.21.15.16:6834', 'nonce': 84349474}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:12:00.529 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:12:00.529 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:12:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.630+0000 7f286849e640 1 Processor -- start 2024-04-25T10:12:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.630+0000 7f286849e640 1 -- start start 2024-04-25T10:12:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.631+0000 7f286849e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f28600cef40 0x7f28600b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.631+0000 7f286849e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28601618d0 0x7f28600d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.631+0000 7f286849e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2860161d80 con 0x7f28601610b0 2024-04-25T10:12:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.631+0000 7f286849e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28600d1ea0 con 0x7f28600cef40 2024-04-25T10:12:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.631+0000 7f286849e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2860162040 con 0x7f28601618d0 2024-04-25T10:12:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.631+0000 7f286623a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f28601610b0 0x7f28600ce830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49480/0 (socket says 172.21.15.16:49480) 2024-04-25T10:12:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.632+0000 7f286623a640 1 -- 172.21.15.16:0/2649329463 learned_addr learned my addr 172.21.15.16:0/2649329463 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.632+0000 7f2865a39640 1 --2- 172.21.15.16:0/2649329463 >> v2:172.21.15.186:3300/0 conn(0x7f28600cef40 0x7f28600b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.632+0000 7f2866a3b640 1 --2- 172.21.15.16:0/2649329463 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28601618d0 0x7f28600d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.632+0000 7f2865238640 1 -- 172.21.15.16:0/2649329463 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317918657 0 0) 0x7f2860161d80 con 0x7f28601610b0 2024-04-25T10:12:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.632+0000 7f2865238640 1 -- 172.21.15.16:0/2649329463 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f284c003520 con 0x7f28601610b0 2024-04-25T10:12:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.632+0000 7f2866a3b640 1 -- 172.21.15.16:0/2649329463 >> v2:172.21.15.186:3300/0 conn(0x7f28600cef40 msgr2=0x7f28600b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.632+0000 7f2866a3b640 1 --2- 172.21.15.16:0/2649329463 >> v2:172.21.15.186:3300/0 conn(0x7f28600cef40 0x7f28600b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.632+0000 7f2866a3b640 1 -- 172.21.15.16:0/2649329463 >> v1:172.21.15.16:6789/0 conn(0x7f28601610b0 legacy=0x7f28600ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.632+0000 7f2866a3b640 1 -- 172.21.15.16:0/2649329463 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2860162320 con 0x7f28601618d0 2024-04-25T10:12:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.632+0000 7f2865a39640 1 --2- 172.21.15.16:0/2649329463 >> v2:172.21.15.186:3300/0 conn(0x7f28600cef40 0x7f28600b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:00.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.633+0000 7f2866a3b640 1 --2- 172.21.15.16:0/2649329463 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28601618d0 0x7f28600d2610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2858009d90 tx=0x7f285802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcd978e39fbd10ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.633+0000 7f2865238640 1 -- 172.21.15.16:0/2649329463 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858031600 con 0x7f28601618d0 2024-04-25T10:12:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.633+0000 7f2865238640 1 -- 172.21.15.16:0/2649329463 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28580317a0 con 0x7f28601618d0 2024-04-25T10:12:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.633+0000 7f2865238640 1 -- 172.21.15.16:0/2649329463 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858031b70 con 0x7f28601618d0 2024-04-25T10:12:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.634+0000 7f286849e640 1 -- 172.21.15.16:0/2649329463 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28601618d0 msgr2=0x7f28600d2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.634+0000 7f286849e640 1 --2- 172.21.15.16:0/2649329463 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28601618d0 0x7f28600d2610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2858009d90 tx=0x7f285802a5d0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.634+0000 7f286849e640 1 -- 172.21.15.16:0/2649329463 shutdown_connections 2024-04-25T10:12:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.634+0000 7f286849e640 1 --2- 172.21.15.16:0/2649329463 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28601618d0 0x7f28600d2610 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.634+0000 7f286849e640 1 --2- 172.21.15.16:0/2649329463 >> v2:172.21.15.186:3300/0 conn(0x7f28600cef40 0x7f28600b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.634+0000 7f286849e640 1 -- 172.21.15.16:0/2649329463 >> 172.21.15.16:0/2649329463 conn(0x7f28600b5570 msgr2=0x7f28600b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.634+0000 7f286849e640 1 -- 172.21.15.16:0/2649329463 shutdown_connections 2024-04-25T10:12:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.634+0000 7f286849e640 1 -- 172.21.15.16:0/2649329463 wait complete. 2024-04-25T10:12:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.635+0000 7f286849e640 1 Processor -- start 2024-04-25T10:12:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.635+0000 7f286849e640 1 -- start start 2024-04-25T10:12:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f286849e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28600cef40 0x7f28600cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f286849e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28601610b0 0x7f28600cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f286849e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28601618d0 0x7f2860151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f286849e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28600b6c00 con 0x7f28600cef40 2024-04-25T10:12:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f286849e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28600d20f0 con 0x7f28601610b0 2024-04-25T10:12:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f286849e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2860161490 con 0x7f28601618d0 2024-04-25T10:12:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f286623a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28600cef40 0x7f28600cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f286623a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28600cef40 0x7f28600cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49550/0 (socket says 172.21.15.16:49550) 2024-04-25T10:12:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f286623a640 1 -- 172.21.15.16:0/909771773 learned_addr learned my addr 172.21.15.16:0/909771773 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f2866a3b640 1 --2- 172.21.15.16:0/909771773 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28601618d0 0x7f2860151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f2865a39640 1 --2- 172.21.15.16:0/909771773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28601610b0 0x7f28600cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f286623a640 1 -- 172.21.15.16:0/909771773 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28601618d0 msgr2=0x7f2860151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f286623a640 1 --2- 172.21.15.16:0/909771773 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28601618d0 0x7f2860151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f286623a640 1 -- 172.21.15.16:0/909771773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28601610b0 msgr2=0x7f28600cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f286623a640 1 --2- 172.21.15.16:0/909771773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28601610b0 0x7f28600cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f286623a640 1 -- 172.21.15.16:0/909771773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28600cd900 con 0x7f28600cef40 2024-04-25T10:12:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f2866a3b640 1 --2- 172.21.15.16:0/909771773 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28601618d0 0x7f2860151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f2865a39640 1 --2- 172.21.15.16:0/909771773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28601610b0 0x7f28600cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f286623a640 1 --2- 172.21.15.16:0/909771773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28600cef40 0x7f28600cd3c0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f285c00c430 tx=0x7f285c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f2856ffd640 1 -- 172.21.15.16:0/909771773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c017070 con 0x7f28600cef40 2024-04-25T10:12:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f2856ffd640 1 -- 172.21.15.16:0/909771773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c005ce0 con 0x7f28600cef40 2024-04-25T10:12:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f286849e640 1 -- 172.21.15.16:0/909771773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28601514a0 con 0x7f28600cef40 2024-04-25T10:12:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f2856ffd640 1 -- 172.21.15.16:0/909771773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c0072f0 con 0x7f28600cef40 2024-04-25T10:12:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.636+0000 7f286849e640 1 -- 172.21.15.16:0/909771773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286015f5d0 con 0x7f28600cef40 2024-04-25T10:12:00.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.638+0000 7f2856ffd640 1 -- 172.21.15.16:0/909771773 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f285c01a070 con 0x7f28600cef40 2024-04-25T10:12:00.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.639+0000 7f2856ffd640 1 --2- 172.21.15.16:0/909771773 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f284406b500 0x7f284406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.639+0000 7f2856ffd640 1 -- 172.21.15.16:0/909771773 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f285c089740 con 0x7f28600cef40 2024-04-25T10:12:00.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.639+0000 7f2865a39640 1 --2- 172.21.15.16:0/909771773 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f284406b500 0x7f284406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.639+0000 7f2865a39640 1 --2- 172.21.15.16:0/909771773 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f284406b500 0x7f284406d9c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f28600cf4b0 tx=0x7f2850008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.640+0000 7f286b19f740 1 Processor -- start 2024-04-25T10:12:00.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.640+0000 7f286b19f740 1 -- start start 2024-04-25T10:12:00.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f286b19f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36753230 0x55bb36753610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f286b19f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36770ad0 0x55bb36771020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f286b19f740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36728f10 0x55bb3676f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f286b19f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3668ba10 con 0x55bb36753230 2024-04-25T10:12:00.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f286b19f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb36634b30 con 0x55bb36770ad0 2024-04-25T10:12:00.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f286b19f740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb36687c50 con 0x55bb36728f10 2024-04-25T10:12:00.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f286623a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36753230 0x55bb36753610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f286623a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36753230 0x55bb36753610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49560/0 (socket says 172.21.15.16:49560) 2024-04-25T10:12:00.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f286623a640 1 -- 172.21.15.16:0/643264525 learned_addr learned my addr 172.21.15.16:0/643264525 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:00.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f2865a39640 1 --2- 172.21.15.16:0/643264525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36770ad0 0x55bb36771020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f2866a3b640 1 --2- 172.21.15.16:0/643264525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36728f10 0x55bb3676f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f286623a640 1 -- 172.21.15.16:0/643264525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36728f10 msgr2=0x55bb3676f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f286623a640 1 --2- 172.21.15.16:0/643264525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36728f10 0x55bb3676f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f286623a640 1 -- 172.21.15.16:0/643264525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36770ad0 msgr2=0x55bb36771020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f286623a640 1 --2- 172.21.15.16:0/643264525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36770ad0 0x55bb36771020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f286623a640 1 -- 172.21.15.16:0/643264525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb36649e10 con 0x55bb36753230 2024-04-25T10:12:00.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f2865a39640 1 --2- 172.21.15.16:0/643264525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36770ad0 0x55bb36771020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:00.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f2866a3b640 1 --2- 172.21.15.16:0/643264525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36728f10 0x55bb3676f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:00.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f286623a640 1 --2- 172.21.15.16:0/643264525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36753230 0x55bb36753610 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f285c087be0 tx=0x7f285c057a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64672d5352064239 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.641+0000 7f2843fff640 1 -- 172.21.15.16:0/643264525 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c056110 con 0x55bb36753230 2024-04-25T10:12:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.642+0000 7f2843fff640 1 -- 172.21.15.16:0/643264525 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c054480 con 0x55bb36753230 2024-04-25T10:12:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.642+0000 7f2843fff640 1 -- 172.21.15.16:0/643264525 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c007490 con 0x55bb36753230 2024-04-25T10:12:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.642+0000 7f286b19f740 1 -- 172.21.15.16:0/643264525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36753230 msgr2=0x55bb36753610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.642+0000 7f286b19f740 1 --2- 172.21.15.16:0/643264525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36753230 0x55bb36753610 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f285c087be0 tx=0x7f285c057a80 comp rx=0 tx=0).stop 2024-04-25T10:12:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.642+0000 7f286b19f740 1 -- 172.21.15.16:0/643264525 shutdown_connections 2024-04-25T10:12:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.642+0000 7f286b19f740 1 --2- 172.21.15.16:0/643264525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36728f10 0x55bb3676f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.642+0000 7f286b19f740 1 --2- 172.21.15.16:0/643264525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36770ad0 0x55bb36771020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.642+0000 7f286b19f740 1 --2- 172.21.15.16:0/643264525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36753230 0x55bb36753610 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.642+0000 7f286b19f740 1 -- 172.21.15.16:0/643264525 >> 172.21.15.16:0/643264525 conn(0x55bb36725840 msgr2=0x55bb3676e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.642+0000 7f286b19f740 1 -- 172.21.15.16:0/643264525 shutdown_connections 2024-04-25T10:12:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.642+0000 7f286b19f740 1 -- 172.21.15.16:0/643264525 wait complete. 2024-04-25T10:12:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.643+0000 7f286b19f740 1 -- start start 2024-04-25T10:12:00.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.643+0000 7f286b19f740 1 Processor -- start 2024-04-25T10:12:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.644+0000 7f286b19f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36753230 0x55bb3674eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.644+0000 7f286b19f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36770ad0 0x55bb36765ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.644+0000 7f286b19f740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36751af0 0x55bb36751f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.644+0000 7f286b19f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3668ba10 con 0x55bb36770ad0 2024-04-25T10:12:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.644+0000 7f286b19f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb366480c0 con 0x55bb36753230 2024-04-25T10:12:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.644+0000 7f286b19f740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb36729cb0 con 0x55bb36751af0 2024-04-25T10:12:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.644+0000 7f286b19f740 10 client.0 _ll_register_callbacks cb 0x55bb3663bec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.644+0000 7f2865a39640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36770ad0 0x55bb36765ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.644+0000 7f2865a39640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36770ad0 0x55bb36765ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49562/0 (socket says 172.21.15.16:49562) 2024-04-25T10:12:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.644+0000 7f2865a39640 1 -- 172.21.15.16:0/194960376 learned_addr learned my addr 172.21.15.16:0/194960376 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.644+0000 7f286623a640 1 --2- 172.21.15.16:0/194960376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36753230 0x55bb3674eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.644+0000 7f2866a3b640 1 --2- 172.21.15.16:0/194960376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36751af0 0x55bb36751f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.644+0000 7f2865a39640 1 -- 172.21.15.16:0/194960376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36751af0 msgr2=0x55bb36751f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.644+0000 7f2865a39640 1 --2- 172.21.15.16:0/194960376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36751af0 0x55bb36751f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.644+0000 7f2865a39640 1 -- 172.21.15.16:0/194960376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36753230 msgr2=0x55bb3674eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.644+0000 7f2865a39640 1 --2- 172.21.15.16:0/194960376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36753230 0x55bb3674eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.644+0000 7f2865a39640 1 -- 172.21.15.16:0/194960376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb363a02d0 con 0x55bb36770ad0 2024-04-25T10:12:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.645+0000 7f2865a39640 1 --2- 172.21.15.16:0/194960376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36770ad0 0x55bb36765ae0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f28500157c0 tx=0x7f28500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e9427cba764d16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.645+0000 7f28417fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:12:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.645+0000 7f28417fa640 1 -- 172.21.15.16:0/194960376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2850025070 con 0x55bb36770ad0 2024-04-25T10:12:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.645+0000 7f28417fa640 1 -- 172.21.15.16:0/194960376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f2850014960 con 0x55bb36770ad0 2024-04-25T10:12:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.645+0000 7f286b19f740 1 -- 172.21.15.16:0/194960376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f285c054480 con 0x55bb36770ad0 2024-04-25T10:12:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.645+0000 7f28417fa640 1 -- 172.21.15.16:0/194960376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2850030050 con 0x55bb36770ad0 2024-04-25T10:12:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.645+0000 7f286b19f740 10 client.7917 fetch_fsmap learned FSMap version 64 2024-04-25T10:12:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.645+0000 7f286b19f740 1 -- 172.21.15.16:0/194960376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55bb3646a6b0 con 0x55bb36770ad0 2024-04-25T10:12:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.645+0000 7f28417fa640 1 -- 172.21.15.16:0/194960376 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c054480 con 0x55bb36770ad0 2024-04-25T10:12:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.645+0000 7f28417fa640 1 -- 172.21.15.16:0/194960376 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2850033020 con 0x55bb36770ad0 2024-04-25T10:12:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.645+0000 7f286b19f740 10 client.7917 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T10:12:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.645+0000 7f286b19f740 10 client.7917 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:12:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.645+0000 7f286b19f740 20 client.7917 populate_metadata read hostname 'smithi016' 2024-04-25T10:12:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.645+0000 7f286b19f740 1 --2- 172.21.15.16:0/194960376 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55bb367544a0 0x55bb36754880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.645+0000 7f286b19f740 4 client.7917 mds_command: new command op to 7323 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.645+0000 7f286b19f740 1 -- 172.21.15.16:0/194960376 --> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb366ea920 con 0x55bb367544a0 2024-04-25T10:12:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.645+0000 7f286623a640 1 --2- 172.21.15.16:0/194960376 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55bb367544a0 0x55bb36754880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.646+0000 7f28417fa640 10 client.7917 ms_handle_connect on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.646+0000 7f286623a640 1 --2- 172.21.15.16:0/194960376 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55bb367544a0 0x55bb36754880 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d332c2700640e0f server_cookie=e89c3ce4b0564849 in_seq=0 out_seq=0 2024-04-25T10:12:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.646+0000 7f28417fa640 1 -- 172.21.15.16:0/194960376 <== mds.0 v2:172.21.15.16:6832/84349474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bb366ea920 con 0x55bb367544a0 2024-04-25T10:12:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.646+0000 7f28417fa640 10 client.7917 handle_command_reply: tid=0 2024-04-25T10:12:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.646+0000 7f286b19f740 1 client.7917 shutdown 2024-04-25T10:12:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.646+0000 7f286b19f740 1 -- 172.21.15.16:0/194960376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36770ad0 msgr2=0x55bb36765ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.646+0000 7f286b19f740 1 --2- 172.21.15.16:0/194960376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36770ad0 0x55bb36765ae0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f28500157c0 tx=0x7f28500046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.646+0000 7f286b19f740 1 -- 172.21.15.16:0/194960376 shutdown_connections 2024-04-25T10:12:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.646+0000 7f286b19f740 1 --2- 172.21.15.16:0/194960376 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55bb367544a0 0x55bb36754880 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.647+0000 7f286b19f740 1 --2- 172.21.15.16:0/194960376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36751af0 0x55bb36751f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.647+0000 7f286b19f740 1 --2- 172.21.15.16:0/194960376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36770ad0 0x55bb36765ae0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.647+0000 7f28417fa640 0 client.7917 ms_handle_reset on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.647+0000 7f286b19f740 1 --2- 172.21.15.16:0/194960376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36753230 0x55bb3674eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.647+0000 7f286b19f740 1 -- 172.21.15.16:0/194960376 >> 172.21.15.16:0/194960376 conn(0x55bb36728f10 msgr2=0x55bb367281b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.647+0000 7f286b19f740 1 -- 172.21.15.16:0/194960376 shutdown_connections 2024-04-25T10:12:00.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.647+0000 7f286b19f740 1 -- 172.21.15.16:0/194960376 wait complete. 2024-04-25T10:12:00.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.647+0000 7f286b19f740 20 client.7917 trim_cache size 0 max 16384 2024-04-25T10:12:00.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.662+0000 7f286b19f740 1 Processor -- start 2024-04-25T10:12:00.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.662+0000 7f286b19f740 1 -- start start 2024-04-25T10:12:00.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.663+0000 7f286b19f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36751af0 0x55bb36754880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.663+0000 7f286b19f740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36753230 0x55bb36750bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.663+0000 7f286b19f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36770ad0 0x55bb367670e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.663+0000 7f286b19f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28600cb6c0 con 0x55bb36751af0 2024-04-25T10:12:00.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.663+0000 7f286b19f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28600cab10 con 0x55bb36770ad0 2024-04-25T10:12:00.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.663+0000 7f286b19f740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28600cb0c0 con 0x55bb36753230 2024-04-25T10:12:00.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.663+0000 7f286623a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36751af0 0x55bb36754880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.663+0000 7f286623a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36751af0 0x55bb36754880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49570/0 (socket says 172.21.15.16:49570) 2024-04-25T10:12:00.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.663+0000 7f286623a640 1 -- 172.21.15.16:0/4128101215 learned_addr learned my addr 172.21.15.16:0/4128101215 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:00.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.663+0000 7f286623a640 1 -- 172.21.15.16:0/4128101215 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36753230 msgr2=0x55bb36750bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.663+0000 7f286623a640 1 --2- 172.21.15.16:0/4128101215 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36753230 0x55bb36750bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.663+0000 7f286623a640 1 -- 172.21.15.16:0/4128101215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36770ad0 msgr2=0x55bb367670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.663+0000 7f286623a640 1 --2- 172.21.15.16:0/4128101215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36770ad0 0x55bb367670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.663+0000 7f286623a640 1 -- 172.21.15.16:0/4128101215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb36791660 con 0x55bb36751af0 2024-04-25T10:12:00.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.664+0000 7f286623a640 1 --2- 172.21.15.16:0/4128101215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36751af0 0x55bb36754880 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f285c089a60 tx=0x7f285c089040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3972be09526a8979 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.664+0000 7f28437fe640 1 -- 172.21.15.16:0/4128101215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c054480 con 0x55bb36751af0 2024-04-25T10:12:00.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.664+0000 7f28437fe640 1 -- 172.21.15.16:0/4128101215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c056110 con 0x55bb36751af0 2024-04-25T10:12:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.664+0000 7f28437fe640 1 -- 172.21.15.16:0/4128101215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c06fba0 con 0x55bb36751af0 2024-04-25T10:12:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.664+0000 7f286b19f740 1 -- 172.21.15.16:0/4128101215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36751af0 msgr2=0x55bb36754880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.664+0000 7f286b19f740 1 --2- 172.21.15.16:0/4128101215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36751af0 0x55bb36754880 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f285c089a60 tx=0x7f285c089040 comp rx=0 tx=0).stop 2024-04-25T10:12:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.664+0000 7f286b19f740 1 -- 172.21.15.16:0/4128101215 shutdown_connections 2024-04-25T10:12:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.664+0000 7f286b19f740 1 --2- 172.21.15.16:0/4128101215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36770ad0 0x55bb367670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.664+0000 7f286b19f740 1 --2- 172.21.15.16:0/4128101215 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36753230 0x55bb36750bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.664+0000 7f286b19f740 1 --2- 172.21.15.16:0/4128101215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36751af0 0x55bb36754880 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.664+0000 7f286b19f740 1 -- 172.21.15.16:0/4128101215 >> 172.21.15.16:0/4128101215 conn(0x55bb36728f10 msgr2=0x55bb36729580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.664+0000 7f286b19f740 1 -- 172.21.15.16:0/4128101215 shutdown_connections 2024-04-25T10:12:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.664+0000 7f286b19f740 1 -- 172.21.15.16:0/4128101215 wait complete. 2024-04-25T10:12:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.665+0000 7f286b19f740 1 -- start start 2024-04-25T10:12:00.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.665+0000 7f286b19f740 1 Processor -- start 2024-04-25T10:12:00.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f286b19f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36751af0 0x55bb367c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f286b19f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36753230 0x55bb36750bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f286b19f740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36770ad0 0x55bb36751100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f286b19f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3677f840 con 0x55bb36751af0 2024-04-25T10:12:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f286b19f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb367669d0 con 0x55bb36753230 2024-04-25T10:12:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f286b19f740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb36757020 con 0x55bb36770ad0 2024-04-25T10:12:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f286b19f740 10 client.0 _ll_register_callbacks cb 0x55bb36723c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f286623a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36751af0 0x55bb367c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f286623a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36751af0 0x55bb367c20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49584/0 (socket says 172.21.15.16:49584) 2024-04-25T10:12:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f286623a640 1 -- 172.21.15.16:0/2844440622 learned_addr learned my addr 172.21.15.16:0/2844440622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f2865a39640 1 --2- 172.21.15.16:0/2844440622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36753230 0x55bb36750bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f2866a3b640 1 --2- 172.21.15.16:0/2844440622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36770ad0 0x55bb36751100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f286623a640 1 -- 172.21.15.16:0/2844440622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36770ad0 msgr2=0x55bb36751100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f286623a640 1 --2- 172.21.15.16:0/2844440622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36770ad0 0x55bb36751100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f286623a640 1 -- 172.21.15.16:0/2844440622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36753230 msgr2=0x55bb36750bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f286623a640 1 --2- 172.21.15.16:0/2844440622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36753230 0x55bb36750bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f286623a640 1 -- 172.21.15.16:0/2844440622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb367dd110 con 0x55bb36751af0 2024-04-25T10:12:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f286623a640 1 --2- 172.21.15.16:0/2844440622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36751af0 0x55bb367c20e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f285c057fd0 tx=0x7f285c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75eb505b00258c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f28417fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:12:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f28417fa640 1 -- 172.21.15.16:0/2844440622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c07a020 con 0x55bb36751af0 2024-04-25T10:12:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.666+0000 7f28417fa640 1 -- 172.21.15.16:0/2844440622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f285c05e070 con 0x55bb36751af0 2024-04-25T10:12:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.667+0000 7f286b19f740 1 -- 172.21.15.16:0/2844440622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f285c056110 con 0x55bb36751af0 2024-04-25T10:12:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.667+0000 7f28417fa640 1 -- 172.21.15.16:0/2844440622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f285c073050 con 0x55bb36751af0 2024-04-25T10:12:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.667+0000 7f286b19f740 10 client.7923 fetch_fsmap learned FSMap version 64 2024-04-25T10:12:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.667+0000 7f286b19f740 1 -- 172.21.15.16:0/2844440622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55bb36727700 con 0x55bb36751af0 2024-04-25T10:12:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.667+0000 7f28417fa640 1 -- 172.21.15.16:0/2844440622 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c056110 con 0x55bb36751af0 2024-04-25T10:12:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.667+0000 7f28417fa640 1 -- 172.21.15.16:0/2844440622 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f285c076ce0 con 0x55bb36751af0 2024-04-25T10:12:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.667+0000 7f286b19f740 10 client.7923 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T10:12:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.667+0000 7f286b19f740 10 client.7923 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:12:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.667+0000 7f286b19f740 20 client.7923 populate_metadata read hostname 'smithi016' 2024-04-25T10:12:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.667+0000 7f286b19f740 1 --2- 172.21.15.16:0/2844440622 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55bb367228c0 0x55bb36722ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.667+0000 7f286b19f740 4 client.7923 mds_command: new command op to 7323 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:12:00.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.667+0000 7f286b19f740 1 -- 172.21.15.16:0/2844440622 --> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bb36698f50 con 0x55bb367228c0 2024-04-25T10:12:00.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.667+0000 7f2865a39640 1 --2- 172.21.15.16:0/2844440622 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55bb367228c0 0x55bb36722ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.667+0000 7f2865a39640 1 --2- 172.21.15.16:0/2844440622 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55bb367228c0 0x55bb36722ca0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=135a305bbed16419 server_cookie=2065d18577a643f9 in_seq=0 out_seq=0 2024-04-25T10:12:00.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.667+0000 7f28417fa640 10 client.7923 ms_handle_connect on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:00.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.667+0000 7f28417fa640 1 -- 172.21.15.16:0/2844440622 <== mds.0 v2:172.21.15.16:6832/84349474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55bb36698f50 con 0x55bb367228c0 2024-04-25T10:12:00.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f28417fa640 10 client.7923 handle_command_reply: tid=0 2024-04-25T10:12:00.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286b19f740 1 client.7923 shutdown 2024-04-25T10:12:00.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286b19f740 1 -- 172.21.15.16:0/2844440622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36751af0 msgr2=0x55bb367c20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286b19f740 1 --2- 172.21.15.16:0/2844440622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36751af0 0x55bb367c20e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f285c057fd0 tx=0x7f285c01e040 comp rx=0 tx=0).stop 2024-04-25T10:12:00.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286b19f740 1 -- 172.21.15.16:0/2844440622 shutdown_connections 2024-04-25T10:12:00.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286b19f740 1 --2- 172.21.15.16:0/2844440622 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55bb367228c0 0x55bb36722ca0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286b19f740 1 --2- 172.21.15.16:0/2844440622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb36770ad0 0x55bb36751100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286b19f740 1 --2- 172.21.15.16:0/2844440622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb36753230 0x55bb36750bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f28417fa640 0 client.7923 ms_handle_reset on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:00.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286b19f740 1 --2- 172.21.15.16:0/2844440622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb36751af0 0x55bb367c20e0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286b19f740 1 -- 172.21.15.16:0/2844440622 >> 172.21.15.16:0/2844440622 conn(0x55bb36728f10 msgr2=0x55bb36754880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286b19f740 1 -- 172.21.15.16:0/2844440622 shutdown_connections 2024-04-25T10:12:00.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286b19f740 1 -- 172.21.15.16:0/2844440622 wait complete. 2024-04-25T10:12:00.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286b19f740 20 client.7923 trim_cache size 0 max 16384 2024-04-25T10:12:00.679 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:12:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "34bc5eca-3a21-42a6-a26f-07da0905e8bc", 2024-04-25T10:12:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-04-25T10:12:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7323, 2024-04-25T10:12:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-04-25T10:12:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-04-25T10:12:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-04-25T10:12:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 4.4277283069999998, 2024-04-25T10:12:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 64, 2024-04-25T10:12:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 88, 2024-04-25T10:12:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T10:12:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 65.032259694000004 2024-04-25T10:12:00.680 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:12:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286849e640 1 -- 172.21.15.16:0/909771773 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f284406b500 msgr2=0x7f284406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286849e640 1 --2- 172.21.15.16:0/909771773 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f284406b500 0x7f284406d9c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f28600cf4b0 tx=0x7f2850008040 comp rx=0 tx=0).stop 2024-04-25T10:12:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286849e640 1 -- 172.21.15.16:0/909771773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28600cef40 msgr2=0x7f28600cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286849e640 1 --2- 172.21.15.16:0/909771773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28600cef40 0x7f28600cd3c0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f285c00c430 tx=0x7f285c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286849e640 1 -- 172.21.15.16:0/909771773 shutdown_connections 2024-04-25T10:12:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286849e640 1 --2- 172.21.15.16:0/909771773 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f284406b500 0x7f284406d9c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286849e640 1 --2- 172.21.15.16:0/909771773 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28601618d0 0x7f2860151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286849e640 1 --2- 172.21.15.16:0/909771773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28601610b0 0x7f28600cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286849e640 1 --2- 172.21.15.16:0/909771773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28600cef40 0x7f28600cd3c0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.668+0000 7f286849e640 1 -- 172.21.15.16:0/909771773 >> 172.21.15.16:0/909771773 conn(0x7f28600b5570 msgr2=0x7f28600b8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.669+0000 7f286849e640 1 -- 172.21.15.16:0/909771773 shutdown_connections 2024-04-25T10:12:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.669+0000 7f286849e640 1 -- 172.21.15.16:0/909771773 wait complete. 2024-04-25T10:12:00.684 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:12:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.780+0000 7f4f52007640 1 Processor -- start 2024-04-25T10:12:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.780+0000 7f4f52007640 1 -- start start 2024-04-25T10:12:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.780+0000 7f4f52007640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f4c15a870 0x7f4f4c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.780+0000 7f4f52007640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4f4c0cfc40 0x7f4f4c15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.780+0000 7f4f52007640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f4c0cdb10 con 0x7f4f4c06e2e0 2024-04-25T10:12:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f52007640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f4c0ce720 con 0x7f4f4c0cfc40 2024-04-25T10:12:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f52007640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c0cddf0 con 0x7f4f4c15a870 2024-04-25T10:12:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4f4c06e2e0 0x7f4f4c0cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49494/0 (socket says 172.21.15.16:49494) 2024-04-25T10:12:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4affd640 1 -- 172.21.15.16:0/4113465417 learned_addr learned my addr 172.21.15.16:0/4113465417 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4b7fe640 1 --2- 172.21.15.16:0/4113465417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f4c15a870 0x7f4f4c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4bfff640 1 --2- 172.21.15.16:0/4113465417 >> v2:172.21.15.186:3300/0 conn(0x7f4f4c0cfc40 0x7f4f4c15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4a7fc640 1 -- 172.21.15.16:0/4113465417 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 710497906 0 0) 0x7f4f4c0cdb10 con 0x7f4f4c06e2e0 2024-04-25T10:12:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4a7fc640 1 -- 172.21.15.16:0/4113465417 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f34003520 con 0x7f4f4c06e2e0 2024-04-25T10:12:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4a7fc640 1 -- 172.21.15.16:0/4113465417 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1834964323 0 0) 0x7f4f34003520 con 0x7f4f4c06e2e0 2024-04-25T10:12:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4a7fc640 1 -- 172.21.15.16:0/4113465417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f4c15a870 msgr2=0x7f4f4c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4a7fc640 1 --2- 172.21.15.16:0/4113465417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f4c15a870 0x7f4f4c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4a7fc640 1 -- 172.21.15.16:0/4113465417 >> v2:172.21.15.186:3300/0 conn(0x7f4f4c0cfc40 msgr2=0x7f4f4c15ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4a7fc640 1 --2- 172.21.15.16:0/4113465417 >> v2:172.21.15.186:3300/0 conn(0x7f4f4c0cfc40 0x7f4f4c15ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4a7fc640 1 -- 172.21.15.16:0/4113465417 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c0ce0d0 con 0x7f4f4c06e2e0 2024-04-25T10:12:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4a7fc640 1 -- 172.21.15.16:0/4113465417 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f4f40001670 con 0x7f4f4c06e2e0 2024-04-25T10:12:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4bfff640 1 --2- 172.21.15.16:0/4113465417 >> v2:172.21.15.186:3300/0 conn(0x7f4f4c0cfc40 0x7f4f4c15ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4a7fc640 1 -- 172.21.15.16:0/4113465417 >> v1:172.21.15.16:6789/0 conn(0x7f4f4c06e2e0 legacy=0x7f4f4c0cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4a7fc640 1 --2- 172.21.15.16:0/4113465417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f340039b0 0x7f4f34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4a7fc640 1 --2- 172.21.15.16:0/4113465417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f4c15a870 0x7f4f34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4a7fc640 1 --2- 172.21.15.16:0/4113465417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f34004d40 0x7f4f340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4a7fc640 1 -- 172.21.15.16:0/4113465417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c0ce720 con 0x7f4f340039b0 2024-04-25T10:12:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4a7fc640 1 -- 172.21.15.16:0/4113465417 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c0cddf0 con 0x7f4f34004d40 2024-04-25T10:12:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.781+0000 7f4f4a7fc640 1 -- 172.21.15.16:0/4113465417 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3400b9c0 con 0x7f4f4c15a870 2024-04-25T10:12:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.782+0000 7f4f4b7fe640 1 --2- 172.21.15.16:0/4113465417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f340039b0 0x7f4f34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.782+0000 7f4f4affd640 1 --2- 172.21.15.16:0/4113465417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f4c15a870 0x7f4f34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.782+0000 7f4f4bfff640 1 --2- 172.21.15.16:0/4113465417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f34004d40 0x7f4f340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.782+0000 7f4f4b7fe640 1 -- 172.21.15.16:0/4113465417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f4c15a870 msgr2=0x7f4f34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.782+0000 7f4f4b7fe640 1 --2- 172.21.15.16:0/4113465417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f4c15a870 0x7f4f34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.782+0000 7f4f4b7fe640 1 -- 172.21.15.16:0/4113465417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f34004d40 msgr2=0x7f4f340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.782+0000 7f4f4b7fe640 1 --2- 172.21.15.16:0/4113465417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f34004d40 0x7f4f340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.782+0000 7f4f4b7fe640 1 -- 172.21.15.16:0/4113465417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f34003520 con 0x7f4f340039b0 2024-04-25T10:12:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.782+0000 7f4f4bfff640 1 --2- 172.21.15.16:0/4113465417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f34004d40 0x7f4f340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.783+0000 7f4f4b7fe640 1 --2- 172.21.15.16:0/4113465417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f340039b0 0x7f4f34003dd0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009ac0 tx=0x7f4f34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c173796e02ab1e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.783+0000 7f4f4a7fc640 1 -- 172.21.15.16:0/4113465417 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c036070 con 0x7f4f340039b0 2024-04-25T10:12:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.783+0000 7f4f4a7fc640 1 -- 172.21.15.16:0/4113465417 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3c031400 con 0x7f4f340039b0 2024-04-25T10:12:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.783+0000 7f4f4a7fc640 1 -- 172.21.15.16:0/4113465417 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c009690 con 0x7f4f340039b0 2024-04-25T10:12:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.783+0000 7f4f52007640 1 -- 172.21.15.16:0/4113465417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f340039b0 msgr2=0x7f4f34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.783+0000 7f4f52007640 1 --2- 172.21.15.16:0/4113465417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f340039b0 0x7f4f34003dd0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009ac0 tx=0x7f4f34005830 comp rx=0 tx=0).stop 2024-04-25T10:12:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.783+0000 7f4f4bfff640 1 -- 172.21.15.16:0/4113465417 reap_dead start 2024-04-25T10:12:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.783+0000 7f4f52007640 1 -- 172.21.15.16:0/4113465417 shutdown_connections 2024-04-25T10:12:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.783+0000 7f4f52007640 1 -- 172.21.15.16:0/4113465417 >> 172.21.15.16:0/4113465417 conn(0x7f4f4c06aba0 msgr2=0x7f4f4c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.783+0000 7f4f52007640 1 -- 172.21.15.16:0/4113465417 shutdown_connections 2024-04-25T10:12:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.783+0000 7f4f52007640 1 -- 172.21.15.16:0/4113465417 wait complete. 2024-04-25T10:12:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.784+0000 7f4f52007640 1 Processor -- start 2024-04-25T10:12:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.785+0000 7f4f52007640 1 -- start start 2024-04-25T10:12:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.785+0000 7f4f52007640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f340039b0 0x7f4f4c152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.785+0000 7f4f52007640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4c159490 0x7f4f4c1528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.785+0000 7f4f52007640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f4c1e73f0 0x7f4f4c1e78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.785+0000 7f4f52007640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c0cd700 con 0x7f4f4c1e73f0 2024-04-25T10:12:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.785+0000 7f4f52007640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c0cd580 con 0x7f4f4c159490 2024-04-25T10:12:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.785+0000 7f4f52007640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c0cd880 con 0x7f4f340039b0 2024-04-25T10:12:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.785+0000 7f4f4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f4c1e73f0 0x7f4f4c1e78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.786+0000 7f4f4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f4c1e73f0 0x7f4f4c1e78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49608/0 (socket says 172.21.15.16:49608) 2024-04-25T10:12:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.786+0000 7f4f4bfff640 1 -- 172.21.15.16:0/3662681830 learned_addr learned my addr 172.21.15.16:0/3662681830 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.786+0000 7f4f4b7fe640 1 --2- 172.21.15.16:0/3662681830 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f340039b0 0x7f4f4c152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.786+0000 7f4f4affd640 1 --2- 172.21.15.16:0/3662681830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4c159490 0x7f4f4c1528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.786+0000 7f4f4bfff640 1 -- 172.21.15.16:0/3662681830 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f340039b0 msgr2=0x7f4f4c152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.786+0000 7f4f4bfff640 1 --2- 172.21.15.16:0/3662681830 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f340039b0 0x7f4f4c152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.786+0000 7f4f4bfff640 1 -- 172.21.15.16:0/3662681830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4c159490 msgr2=0x7f4f4c1528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.786+0000 7f4f4bfff640 1 --2- 172.21.15.16:0/3662681830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4c159490 0x7f4f4c1528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.786+0000 7f4f4bfff640 1 -- 172.21.15.16:0/3662681830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c153230 con 0x7f4f4c1e73f0 2024-04-25T10:12:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.786+0000 7f4f4bfff640 1 --2- 172.21.15.16:0/3662681830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f4c1e73f0 0x7f4f4c1e78a0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0d0050 tx=0x7f4f440115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.786+0000 7f4f33fff640 1 -- 172.21.15.16:0/3662681830 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f44003990 con 0x7f4f4c1e73f0 2024-04-25T10:12:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.786+0000 7f4f33fff640 1 -- 172.21.15.16:0/3662681830 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4401b070 con 0x7f4f4c1e73f0 2024-04-25T10:12:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.786+0000 7f4f52007640 1 -- 172.21.15.16:0/3662681830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4c153510 con 0x7f4f4c1e73f0 2024-04-25T10:12:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.786+0000 7f4f33fff640 1 -- 172.21.15.16:0/3662681830 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f44003b30 con 0x7f4f4c1e73f0 2024-04-25T10:12:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.786+0000 7f4f52007640 1 -- 172.21.15.16:0/3662681830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f4c1f3910 con 0x7f4f4c1e73f0 2024-04-25T10:12:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.787+0000 7f4f52007640 1 -- 172.21.15.16:0/3662681830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f4c1f3b10 con 0x7f4f4c1e73f0 2024-04-25T10:12:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.788+0000 7f4f33fff640 1 -- 172.21.15.16:0/3662681830 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f44005d10 con 0x7f4f4c1e73f0 2024-04-25T10:12:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.788+0000 7f4f33fff640 1 --2- 172.21.15.16:0/3662681830 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4f3806b500 0x7f4f3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.788+0000 7f4f33fff640 1 -- 172.21.15.16:0/3662681830 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f4f4408a880 con 0x7f4f4c1e73f0 2024-04-25T10:12:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.788+0000 7f4f4b7fe640 1 --2- 172.21.15.16:0/3662681830 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4f3806b500 0x7f4f3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.789+0000 7f4f4b7fe640 1 --2- 172.21.15.16:0/3662681830 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4f3806b500 0x7f4f3806d9c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c033000 tx=0x7f4f3c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:00.792+0000 7f4f33fff640 1 -- 172.21.15.16:0/3662681830 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f4c1f3b10 con 0x7f4f4c1e73f0 2024-04-25T10:12:01.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.011+0000 7f4f52007640 1 -- 172.21.15.16:0/3662681830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4f4c0749b0 con 0x7f4f4c1e73f0 2024-04-25T10:12:01.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.011+0000 7f4f33fff640 1 -- 172.21.15.16:0/3662681830 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4f44025020 con 0x7f4f4c1e73f0 2024-04-25T10:12:01.016 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:12:01.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.013+0000 7f4f52007640 1 -- 172.21.15.16:0/3662681830 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4f3806b500 msgr2=0x7f4f3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.013+0000 7f4f52007640 1 --2- 172.21.15.16:0/3662681830 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4f3806b500 0x7f4f3806d9c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c033000 tx=0x7f4f3c008030 comp rx=0 tx=0).stop 2024-04-25T10:12:01.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.013+0000 7f4f52007640 1 -- 172.21.15.16:0/3662681830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f4c1e73f0 msgr2=0x7f4f4c1e78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.013+0000 7f4f52007640 1 --2- 172.21.15.16:0/3662681830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f4c1e73f0 0x7f4f4c1e78a0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0d0050 tx=0x7f4f440115a0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.013+0000 7f4f52007640 1 -- 172.21.15.16:0/3662681830 shutdown_connections 2024-04-25T10:12:01.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.013+0000 7f4f52007640 1 --2- 172.21.15.16:0/3662681830 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4f3806b500 0x7f4f3806d9c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.013+0000 7f4f52007640 1 --2- 172.21.15.16:0/3662681830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f4c1e73f0 0x7f4f4c1e78a0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.013+0000 7f4f52007640 1 --2- 172.21.15.16:0/3662681830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4c159490 0x7f4f4c1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.013+0000 7f4f52007640 1 --2- 172.21.15.16:0/3662681830 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f340039b0 0x7f4f4c152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.013+0000 7f4f52007640 1 -- 172.21.15.16:0/3662681830 >> 172.21.15.16:0/3662681830 conn(0x7f4f4c06aba0 msgr2=0x7f4f4c0af150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:01.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.013+0000 7f4f52007640 1 -- 172.21.15.16:0/3662681830 shutdown_connections 2024-04-25T10:12:01.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.013+0000 7f4f52007640 1 -- 172.21.15.16:0/3662681830 wait complete. 2024-04-25T10:12:01.029 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:12:01.029 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:12:01.029 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-04-25T10:12:01.029 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:12:01.029 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:01.029 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:12:01.029 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:12:01.029 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:12:01.029 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:01.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:01.030 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-04-25T10:12:01.045 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:12:01.045 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:12:01.045 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:12:01.045 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:01.046 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:12:01.046 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:12:01.046 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:01.046 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-04-25T10:12:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:12:01.046 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:01.046 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-04-25T10:12:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:12:01.046 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:01.046 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-04-25T10:12:01.046 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:12:01.046 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42239sec preferred_lft 42239sec 2024-04-25T10:12:01.046 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-25T10:12:01.047 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:12:01.047 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:01.047 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-04-25T10:12:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:12:01.047 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:01.047 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:12:01.047 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:01.047 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f06b:fdff:fed4:a335/64 scope link 2024-04-25T10:12:01.047 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:12:01.047 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:01.047 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::90ad:4fff:fe91:3c33/64 scope link 2024-04-25T10:12:01.047 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:12:01.048 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether be:14:4a:3c:16:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:01.048 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::bc14:4aff:fe3c:1605/64 scope link 2024-04-25T10:12:01.048 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:01.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:01.048 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:12:01.109 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:12:01.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:01.110 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-04-25T10:12:01.167 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-04-25T10:12:01.167 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:12:01.167 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:12:01.167 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:12:01.168 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:12:01.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:01.168 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:12:01.168 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:12:01.168 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:12:01.168 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:12:01.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:12:01.169 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:12:01.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:01.253 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:12:01.270 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:12:01.271 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:12:01.271 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:01.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:01.271 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-04-25T10:12:01.335 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:01.335 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:01.391 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:01.392 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:01.392 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:01.447 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:12:01.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:01.447 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:01.530 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:01.530 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:12:01.530 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:12:01.538 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:01.538 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:12:01.538 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:12:01.539 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:12:01.539 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:12:01.543 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:01.543 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:12:01.543 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:12:01.543 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:12:01.543 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:12:01.543 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:12:01.543 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:12:01.543 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:12:01.544 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:12:01.544 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:12:01.544 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:12:01.544 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:12:01.544 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:12:01.546 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:12:01.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:01.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:01.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:01.612 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:12:01.643 INFO:tasks.ceph.mds.a.smithi016.stderr:2024-04-25T10:12:01.638+0000 7f82eb6d7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:12:01.659 INFO:teuthology.orchestra.run.smithi016.stdout:client7932 2024-04-25T10:12:01.659 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:12:01.659 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:01.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:01.715 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-25T10:12:01.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.856+0000 7f93acad8640 1 Processor -- start 2024-04-25T10:12:01.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.856+0000 7f93acad8640 1 -- start start 2024-04-25T10:12:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.856+0000 7f93acad8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93a81610b0 0x7f93a80b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.856+0000 7f93acad8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a8162040 con 0x7f93a80cef40 2024-04-25T10:12:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.856+0000 7f93acad8640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a8162320 con 0x7f93a81618d0 2024-04-25T10:12:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.856+0000 7f93acad8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a80d1ea0 con 0x7f93a81610b0 2024-04-25T10:12:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.857+0000 7f93a659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93a81610b0 0x7f93a80b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.857+0000 7f93a6d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f93a80cef40 0x7f93a80d2610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49506/0 (socket says 172.21.15.16:49506) 2024-04-25T10:12:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.857+0000 7f93a6d9d640 1 -- 172.21.15.16:0/2280280219 learned_addr learned my addr 172.21.15.16:0/2280280219 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.857+0000 7f93a659c640 1 -- 172.21.15.16:0/2280280219 >> v1:172.21.15.186:6789/0 conn(0x7f93a81618d0 legacy=0x7f93a80ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.857+0000 7f93a659c640 1 -- 172.21.15.16:0/2280280219 >> v1:172.21.15.16:6789/0 conn(0x7f93a80cef40 legacy=0x7f93a80d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.857+0000 7f93a659c640 1 -- 172.21.15.16:0/2280280219 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a8162600 con 0x7f93a81610b0 2024-04-25T10:12:01.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.858+0000 7f93a659c640 1 --2- 172.21.15.16:0/2280280219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a81610b0 0x7f93a80b9580 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f93980097a0 tx=0x7f939802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbde346fa347a70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:01.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.858+0000 7f93a5d9b640 1 -- 172.21.15.16:0/2280280219 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398037070 con 0x7f93a81610b0 2024-04-25T10:12:01.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.858+0000 7f93a5d9b640 1 -- 172.21.15.16:0/2280280219 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939802ae50 con 0x7f93a81610b0 2024-04-25T10:12:01.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.858+0000 7f93a5d9b640 1 -- 172.21.15.16:0/2280280219 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398032660 con 0x7f93a81610b0 2024-04-25T10:12:01.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.858+0000 7f93acad8640 1 -- 172.21.15.16:0/2280280219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a81610b0 msgr2=0x7f93a80b9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.858+0000 7f93acad8640 1 --2- 172.21.15.16:0/2280280219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a81610b0 0x7f93a80b9580 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f93980097a0 tx=0x7f939802a140 comp rx=0 tx=0).stop 2024-04-25T10:12:01.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.858+0000 7f93acad8640 1 -- 172.21.15.16:0/2280280219 shutdown_connections 2024-04-25T10:12:01.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.858+0000 7f93acad8640 1 --2- 172.21.15.16:0/2280280219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a81610b0 0x7f93a80b9580 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.858+0000 7f93acad8640 1 -- 172.21.15.16:0/2280280219 >> 172.21.15.16:0/2280280219 conn(0x7f93a80b5570 msgr2=0x7f93a80b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:01.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.858+0000 7f93acad8640 1 -- 172.21.15.16:0/2280280219 shutdown_connections 2024-04-25T10:12:01.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.858+0000 7f93acad8640 1 -- 172.21.15.16:0/2280280219 wait complete. 2024-04-25T10:12:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.860+0000 7f93acad8640 1 Processor -- start 2024-04-25T10:12:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.860+0000 7f93acad8640 1 -- start start 2024-04-25T10:12:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.860+0000 7f93acad8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93a80cef40 0x7f93a81f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.860+0000 7f93acad8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93a81610b0 0x7f93a80c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.860+0000 7f93acad8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a81618d0 0x7f93a815c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.860+0000 7f93acad8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a8161d10 con 0x7f93a81618d0 2024-04-25T10:12:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.860+0000 7f93acad8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a80b6c00 con 0x7f93a81610b0 2024-04-25T10:12:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.860+0000 7f93acad8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a8161490 con 0x7f93a80cef40 2024-04-25T10:12:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.861+0000 7f93a6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a81618d0 0x7f93a815c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.861+0000 7f93a6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a81618d0 0x7f93a815c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49616/0 (socket says 172.21.15.16:49616) 2024-04-25T10:12:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.861+0000 7f93a6d9d640 1 -- 172.21.15.16:0/35224549 learned_addr learned my addr 172.21.15.16:0/35224549 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.861+0000 7f93a6d9d640 1 -- 172.21.15.16:0/35224549 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93a80cef40 msgr2=0x7f93a81f4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.861+0000 7f93a6d9d640 1 --2- 172.21.15.16:0/35224549 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93a80cef40 0x7f93a81f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.861+0000 7f93a6d9d640 1 -- 172.21.15.16:0/35224549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93a81610b0 msgr2=0x7f93a80c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.861+0000 7f939ffff640 1 --2- 172.21.15.16:0/35224549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93a81610b0 0x7f93a80c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.861+0000 7f93a6d9d640 1 --2- 172.21.15.16:0/35224549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93a81610b0 0x7f93a80c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.861+0000 7f93a6d9d640 1 -- 172.21.15.16:0/35224549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a8158140 con 0x7f93a81618d0 2024-04-25T10:12:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.861+0000 7f93a6d9d640 1 --2- 172.21.15.16:0/35224549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a81618d0 0x7f93a815c010 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f93a000cbf0 tx=0x7f93a000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.861+0000 7f939effd640 1 -- 172.21.15.16:0/35224549 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0017070 con 0x7f93a81618d0 2024-04-25T10:12:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.861+0000 7f939effd640 1 -- 172.21.15.16:0/35224549 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a000ad40 con 0x7f93a81618d0 2024-04-25T10:12:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.861+0000 7f93acad8640 1 -- 172.21.15.16:0/35224549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a81586d0 con 0x7f93a81618d0 2024-04-25T10:12:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.861+0000 7f939effd640 1 -- 172.21.15.16:0/35224549 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0012600 con 0x7f93a81618d0 2024-04-25T10:12:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.861+0000 7f93acad8640 1 -- 172.21.15.16:0/35224549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a815c920 con 0x7f93a81618d0 2024-04-25T10:12:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.863+0000 7f939effd640 1 -- 172.21.15.16:0/35224549 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f93a001a070 con 0x7f93a81618d0 2024-04-25T10:12:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.863+0000 7f939effd640 1 --2- 172.21.15.16:0/35224549 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f938806b4d0 0x7f938806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.863+0000 7f939effd640 1 -- 172.21.15.16:0/35224549 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f93a0089f00 con 0x7f93a81618d0 2024-04-25T10:12:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.863+0000 7f93a659c640 1 --2- 172.21.15.16:0/35224549 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f938806b4d0 0x7f938806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.864+0000 7f93a659c640 1 --2- 172.21.15.16:0/35224549 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f938806b4d0 0x7f938806d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9398034000 tx=0x7f939802c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.864+0000 7f93af7d7740 1 Processor -- start 2024-04-25T10:12:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.864+0000 7f93af7d7740 1 -- start start 2024-04-25T10:12:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.865+0000 7f93af7d7740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c62abb0 0x55613c62af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.865+0000 7f93af7d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c625830 0x55613c6297b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.865+0000 7f93af7d7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c629cf0 0x55613c62a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.865+0000 7f93af7d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55613c58eac0 con 0x55613c625830 2024-04-25T10:12:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.865+0000 7f93af7d7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55613c537d80 con 0x55613c629cf0 2024-04-25T10:12:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.865+0000 7f93af7d7740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55613c58ab60 con 0x55613c62abb0 2024-04-25T10:12:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.865+0000 7f939ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c625830 0x55613c6297b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.865+0000 7f939ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c625830 0x55613c6297b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49630/0 (socket says 172.21.15.16:49630) 2024-04-25T10:12:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.865+0000 7f939ffff640 1 -- 172.21.15.16:0/4030719886 learned_addr learned my addr 172.21.15.16:0/4030719886 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.865+0000 7f93a659c640 1 --2- 172.21.15.16:0/4030719886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c62abb0 0x55613c62af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.865+0000 7f93a6d9d640 1 --2- 172.21.15.16:0/4030719886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c629cf0 0x55613c62a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.865+0000 7f939ffff640 1 -- 172.21.15.16:0/4030719886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c62abb0 msgr2=0x55613c62af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.865+0000 7f939ffff640 1 --2- 172.21.15.16:0/4030719886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c62abb0 0x55613c62af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.865+0000 7f939ffff640 1 -- 172.21.15.16:0/4030719886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c629cf0 msgr2=0x55613c62a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.865+0000 7f939ffff640 1 --2- 172.21.15.16:0/4030719886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c629cf0 0x55613c62a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.865+0000 7f939ffff640 1 -- 172.21.15.16:0/4030719886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55613c36b7e0 con 0x55613c625830 2024-04-25T10:12:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.865+0000 7f93a659c640 1 --2- 172.21.15.16:0/4030719886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c62abb0 0x55613c62af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.865+0000 7f93a6d9d640 1 --2- 172.21.15.16:0/4030719886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c629cf0 0x55613c62a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.866+0000 7f939ffff640 1 --2- 172.21.15.16:0/4030719886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c625830 0x55613c6297b0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f939000a7f0 tx=0x7f939000acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad2b4f33383e70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.866+0000 7f9387fff640 1 -- 172.21.15.16:0/4030719886 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390015070 con 0x55613c625830 2024-04-25T10:12:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.866+0000 7f9387fff640 1 -- 172.21.15.16:0/4030719886 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939000cb40 con 0x55613c625830 2024-04-25T10:12:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.866+0000 7f9387fff640 1 -- 172.21.15.16:0/4030719886 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93900113f0 con 0x55613c625830 2024-04-25T10:12:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.866+0000 7f93af7d7740 1 -- 172.21.15.16:0/4030719886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c625830 msgr2=0x55613c6297b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.866+0000 7f93af7d7740 1 --2- 172.21.15.16:0/4030719886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c625830 0x55613c6297b0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f939000a7f0 tx=0x7f939000acb0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.866+0000 7f93af7d7740 1 -- 172.21.15.16:0/4030719886 shutdown_connections 2024-04-25T10:12:01.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.866+0000 7f93af7d7740 1 --2- 172.21.15.16:0/4030719886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c629cf0 0x55613c62a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.866+0000 7f93af7d7740 1 --2- 172.21.15.16:0/4030719886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c625830 0x55613c6297b0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.866+0000 7f93af7d7740 1 --2- 172.21.15.16:0/4030719886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c62abb0 0x55613c62af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.866+0000 7f93af7d7740 1 -- 172.21.15.16:0/4030719886 >> 172.21.15.16:0/4030719886 conn(0x55613c627cb0 msgr2=0x55613c670770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:01.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.867+0000 7f93af7d7740 1 -- 172.21.15.16:0/4030719886 shutdown_connections 2024-04-25T10:12:01.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.867+0000 7f93af7d7740 1 -- 172.21.15.16:0/4030719886 wait complete. 2024-04-25T10:12:01.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.867+0000 7f93af7d7740 1 -- start start 2024-04-25T10:12:01.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.867+0000 7f93af7d7740 1 Processor -- start 2024-04-25T10:12:01.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.868+0000 7f93af7d7740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c629cf0 0x55613c647b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.868+0000 7f93af7d7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c62abb0 0x55613c648150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.868+0000 7f93af7d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c64e9a0 0x55613c64b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.868+0000 7f93af7d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55613c656740 con 0x55613c64e9a0 2024-04-25T10:12:01.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.868+0000 7f93af7d7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55613c52d530 con 0x55613c62abb0 2024-04-25T10:12:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.868+0000 7f93af7d7740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55613c655bc0 con 0x55613c629cf0 2024-04-25T10:12:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.868+0000 7f93af7d7740 10 client.0 _ll_register_callbacks cb 0x55613c585130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.868+0000 7f93a6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c64e9a0 0x55613c64b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f93a6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c64e9a0 0x55613c64b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49640/0 (socket says 172.21.15.16:49640) 2024-04-25T10:12:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f93a6d9d640 1 -- 172.21.15.16:0/3924094847 learned_addr learned my addr 172.21.15.16:0/3924094847 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f93a659c640 1 --2- 172.21.15.16:0/3924094847 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c629cf0 0x55613c647b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f939ffff640 1 --2- 172.21.15.16:0/3924094847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c62abb0 0x55613c648150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f93a6d9d640 1 -- 172.21.15.16:0/3924094847 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c629cf0 msgr2=0x55613c647b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f93a6d9d640 1 --2- 172.21.15.16:0/3924094847 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c629cf0 0x55613c647b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f93a6d9d640 1 -- 172.21.15.16:0/3924094847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c62abb0 msgr2=0x55613c648150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f93a6d9d640 1 --2- 172.21.15.16:0/3924094847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c62abb0 0x55613c648150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f93a6d9d640 1 -- 172.21.15.16:0/3924094847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55613c3cbf70 con 0x55613c64e9a0 2024-04-25T10:12:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f93a6d9d640 1 --2- 172.21.15.16:0/3924094847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c64e9a0 0x55613c64b750 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f93a000a930 tx=0x7f93a0060da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9526063e1231ec74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f93857fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:12:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f93857fa640 1 -- 172.21.15.16:0/3924094847 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0004b60 con 0x55613c64e9a0 2024-04-25T10:12:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f93857fa640 1 -- 172.21.15.16:0/3924094847 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f93a0004d00 con 0x55613c64e9a0 2024-04-25T10:12:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f93af7d7740 1 -- 172.21.15.16:0/3924094847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f939000cb40 con 0x55613c64e9a0 2024-04-25T10:12:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f93857fa640 1 -- 172.21.15.16:0/3924094847 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93a0088050 con 0x55613c64e9a0 2024-04-25T10:12:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f93af7d7740 10 client.7947 fetch_fsmap learned FSMap version 64 2024-04-25T10:12:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f93af7d7740 1 -- 172.21.15.16:0/3924094847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55613c263170 con 0x55613c64e9a0 2024-04-25T10:12:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f93857fa640 1 -- 172.21.15.16:0/3924094847 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939000cb40 con 0x55613c64e9a0 2024-04-25T10:12:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.869+0000 7f93857fa640 1 -- 172.21.15.16:0/3924094847 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f93a0081ce0 con 0x55613c64e9a0 2024-04-25T10:12:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.870+0000 7f93af7d7740 10 client.7947 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T10:12:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.870+0000 7f93af7d7740 10 client.7947 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:12:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.870+0000 7f93af7d7740 20 client.7947 populate_metadata read hostname 'smithi016' 2024-04-25T10:12:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.870+0000 7f93af7d7740 1 --2- 172.21.15.16:0/3924094847 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55613c649dc0 0x55613c64a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.870+0000 7f93af7d7740 4 client.7947 mds_command: new command op to 7323 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.870+0000 7f93af7d7740 1 -- 172.21.15.16:0/3924094847 --> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55613c59c000 con 0x55613c649dc0 2024-04-25T10:12:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.870+0000 7f93a659c640 1 --2- 172.21.15.16:0/3924094847 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55613c649dc0 0x55613c64a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.870+0000 7f93857fa640 10 client.7947 ms_handle_connect on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.870+0000 7f93a659c640 1 --2- 172.21.15.16:0/3924094847 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55613c649dc0 0x55613c64a1d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3cbe7701062a01d server_cookie=a083258d0222b398 in_seq=0 out_seq=0 2024-04-25T10:12:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.871+0000 7f93857fa640 1 -- 172.21.15.16:0/3924094847 <== mds.0 v2:172.21.15.16:6832/84349474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55613c59c000 con 0x55613c649dc0 2024-04-25T10:12:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.871+0000 7f93857fa640 10 client.7947 handle_command_reply: tid=0 2024-04-25T10:12:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.871+0000 7f93af7d7740 1 client.7947 shutdown 2024-04-25T10:12:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.871+0000 7f93af7d7740 1 -- 172.21.15.16:0/3924094847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c64e9a0 msgr2=0x55613c64b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.871+0000 7f93af7d7740 1 --2- 172.21.15.16:0/3924094847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c64e9a0 0x55613c64b750 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f93a000a930 tx=0x7f93a0060da0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.871+0000 7f93af7d7740 1 -- 172.21.15.16:0/3924094847 shutdown_connections 2024-04-25T10:12:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.871+0000 7f93af7d7740 1 --2- 172.21.15.16:0/3924094847 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55613c649dc0 0x55613c64a1d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.871+0000 7f93af7d7740 1 --2- 172.21.15.16:0/3924094847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c64e9a0 0x55613c64b750 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.871+0000 7f93af7d7740 1 --2- 172.21.15.16:0/3924094847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c62abb0 0x55613c648150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.871+0000 7f93857fa640 0 client.7947 ms_handle_reset on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.871+0000 7f93af7d7740 1 --2- 172.21.15.16:0/3924094847 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c629cf0 0x55613c647b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.871+0000 7f93af7d7740 1 -- 172.21.15.16:0/3924094847 >> 172.21.15.16:0/3924094847 conn(0x55613c625830 msgr2=0x55613c62b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.871+0000 7f93af7d7740 1 -- 172.21.15.16:0/3924094847 shutdown_connections 2024-04-25T10:12:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.871+0000 7f93af7d7740 1 -- 172.21.15.16:0/3924094847 wait complete. 2024-04-25T10:12:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.871+0000 7f93af7d7740 20 client.7947 trim_cache size 0 max 16384 2024-04-25T10:12:01.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.886+0000 7f93af7d7740 1 Processor -- start 2024-04-25T10:12:01.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.886+0000 7f93af7d7740 1 -- start start 2024-04-25T10:12:01.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.887+0000 7f93af7d7740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c629cf0 0x55613c6581d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.887+0000 7f93af7d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c62abb0 0x55613c6d5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.887+0000 7f93af7d7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c64e9a0 0x55613c6246c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.887+0000 7f93af7d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a8200080 con 0x55613c62abb0 2024-04-25T10:12:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.887+0000 7f93af7d7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a81ff220 con 0x55613c64e9a0 2024-04-25T10:12:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.887+0000 7f93af7d7740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a81f7a40 con 0x55613c629cf0 2024-04-25T10:12:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.887+0000 7f939ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c62abb0 0x55613c6d5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.887+0000 7f939ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c62abb0 0x55613c6d5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49654/0 (socket says 172.21.15.16:49654) 2024-04-25T10:12:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.887+0000 7f939ffff640 1 -- 172.21.15.16:0/2462008490 learned_addr learned my addr 172.21.15.16:0/2462008490 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.887+0000 7f939ffff640 1 -- 172.21.15.16:0/2462008490 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c629cf0 msgr2=0x55613c6581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.887+0000 7f939ffff640 1 --2- 172.21.15.16:0/2462008490 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c629cf0 0x55613c6581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.887+0000 7f939ffff640 1 -- 172.21.15.16:0/2462008490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c64e9a0 msgr2=0x55613c6246c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.887+0000 7f939ffff640 1 --2- 172.21.15.16:0/2462008490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c64e9a0 0x55613c6246c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.887+0000 7f939ffff640 1 -- 172.21.15.16:0/2462008490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55613c64eeb0 con 0x55613c62abb0 2024-04-25T10:12:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.887+0000 7f939ffff640 1 --2- 172.21.15.16:0/2462008490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c62abb0 0x55613c6d5e90 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f939000a7c0 tx=0x7f9390012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b3b81e3b2b3fb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.888+0000 7f93877fe640 1 -- 172.21.15.16:0/2462008490 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390028070 con 0x55613c62abb0 2024-04-25T10:12:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.888+0000 7f93877fe640 1 -- 172.21.15.16:0/2462008490 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9390004240 con 0x55613c62abb0 2024-04-25T10:12:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.888+0000 7f93877fe640 1 -- 172.21.15.16:0/2462008490 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390015040 con 0x55613c62abb0 2024-04-25T10:12:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.888+0000 7f93af7d7740 1 -- 172.21.15.16:0/2462008490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c62abb0 msgr2=0x55613c6d5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.888+0000 7f93af7d7740 1 --2- 172.21.15.16:0/2462008490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c62abb0 0x55613c6d5e90 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f939000a7c0 tx=0x7f9390012000 comp rx=0 tx=0).stop 2024-04-25T10:12:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.888+0000 7f93af7d7740 1 -- 172.21.15.16:0/2462008490 shutdown_connections 2024-04-25T10:12:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.888+0000 7f93af7d7740 1 --2- 172.21.15.16:0/2462008490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c64e9a0 0x55613c6246c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.888+0000 7f93af7d7740 1 --2- 172.21.15.16:0/2462008490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c62abb0 0x55613c6d5e90 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.888+0000 7f93af7d7740 1 --2- 172.21.15.16:0/2462008490 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c629cf0 0x55613c6581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.888+0000 7f93af7d7740 1 -- 172.21.15.16:0/2462008490 >> 172.21.15.16:0/2462008490 conn(0x55613c625830 msgr2=0x55613c676e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.888+0000 7f93af7d7740 1 -- 172.21.15.16:0/2462008490 shutdown_connections 2024-04-25T10:12:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.888+0000 7f93af7d7740 1 -- 172.21.15.16:0/2462008490 wait complete. 2024-04-25T10:12:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.889+0000 7f93af7d7740 1 -- start start 2024-04-25T10:12:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.889+0000 7f93af7d7740 1 Processor -- start 2024-04-25T10:12:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.890+0000 7f93af7d7740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c629cf0 0x55613c6dc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.890+0000 7f93af7d7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c62abb0 0x55613c670340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.890+0000 7f93af7d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c64e9a0 0x55613c6742c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.890+0000 7f93af7d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a8159610 con 0x55613c64e9a0 2024-04-25T10:12:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.890+0000 7f93af7d7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55613c6dcb40 con 0x55613c62abb0 2024-04-25T10:12:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.890+0000 7f93af7d7740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55613c64b3e0 con 0x55613c629cf0 2024-04-25T10:12:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.890+0000 7f93af7d7740 10 client.0 _ll_register_callbacks cb 0x7f9390006f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.890+0000 7f93a6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c64e9a0 0x55613c6742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.890+0000 7f93a6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c64e9a0 0x55613c6742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49658/0 (socket says 172.21.15.16:49658) 2024-04-25T10:12:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.890+0000 7f93a6d9d640 1 -- 172.21.15.16:0/3956022747 learned_addr learned my addr 172.21.15.16:0/3956022747 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.890+0000 7f93a659c640 1 --2- 172.21.15.16:0/3956022747 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c629cf0 0x55613c6dc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.890+0000 7f939ffff640 1 --2- 172.21.15.16:0/3956022747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c62abb0 0x55613c670340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.890+0000 7f93a6d9d640 1 -- 172.21.15.16:0/3956022747 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c629cf0 msgr2=0x55613c6dc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.890+0000 7f93a6d9d640 1 --2- 172.21.15.16:0/3956022747 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c629cf0 0x55613c6dc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.890+0000 7f93a6d9d640 1 -- 172.21.15.16:0/3956022747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c62abb0 msgr2=0x55613c670340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.890+0000 7f93a6d9d640 1 --2- 172.21.15.16:0/3956022747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c62abb0 0x55613c670340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.890+0000 7f93a6d9d640 1 -- 172.21.15.16:0/3956022747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55613c670ee0 con 0x55613c64e9a0 2024-04-25T10:12:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.891+0000 7f93a6d9d640 1 --2- 172.21.15.16:0/3956022747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c64e9a0 0x55613c6742c0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f93a0058380 tx=0x7f93a000aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5739f83de0cadbfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.891+0000 7f93857fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:12:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.891+0000 7f93857fa640 1 -- 172.21.15.16:0/3956022747 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0076020 con 0x55613c64e9a0 2024-04-25T10:12:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.891+0000 7f93857fa640 1 -- 172.21.15.16:0/3956022747 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f93a0004b60 con 0x55613c64e9a0 2024-04-25T10:12:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.891+0000 7f93af7d7740 1 -- 172.21.15.16:0/3956022747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9390004240 con 0x55613c64e9a0 2024-04-25T10:12:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.891+0000 7f93857fa640 1 -- 172.21.15.16:0/3956022747 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93a007c050 con 0x55613c64e9a0 2024-04-25T10:12:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.891+0000 7f93af7d7740 10 client.7953 fetch_fsmap learned FSMap version 64 2024-04-25T10:12:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.891+0000 7f93af7d7740 1 -- 172.21.15.16:0/3956022747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55613c629b20 con 0x55613c64e9a0 2024-04-25T10:12:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.891+0000 7f93857fa640 1 -- 172.21.15.16:0/3956022747 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9390004240 con 0x55613c64e9a0 2024-04-25T10:12:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.891+0000 7f93857fa640 1 -- 172.21.15.16:0/3956022747 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f93a007f020 con 0x55613c64e9a0 2024-04-25T10:12:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.891+0000 7f93af7d7740 10 client.7953 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T10:12:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.891+0000 7f93af7d7740 10 client.7953 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:12:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.891+0000 7f93af7d7740 20 client.7953 populate_metadata read hostname 'smithi016' 2024-04-25T10:12:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.891+0000 7f93af7d7740 1 --2- 172.21.15.16:0/3956022747 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55613c670880 0x55613c674800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.891+0000 7f93af7d7740 4 client.7953 mds_command: new command op to 7323 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:12:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.891+0000 7f93af7d7740 1 -- 172.21.15.16:0/3956022747 --> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x55613c596350 con 0x55613c670880 2024-04-25T10:12:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.891+0000 7f93a659c640 1 --2- 172.21.15.16:0/3956022747 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55613c670880 0x55613c674800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.892+0000 7f93a659c640 1 --2- 172.21.15.16:0/3956022747 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55613c670880 0x55613c674800 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8e3bde8fc05eb31 server_cookie=a057f451db7505b8 in_seq=0 out_seq=0 2024-04-25T10:12:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.892+0000 7f93857fa640 10 client.7953 ms_handle_connect on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.892+0000 7f93857fa640 1 -- 172.21.15.16:0/3956022747 <== mds.0 v2:172.21.15.16:6832/84349474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55613c596350 con 0x55613c670880 2024-04-25T10:12:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.892+0000 7f93857fa640 10 client.7953 handle_command_reply: tid=0 2024-04-25T10:12:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.892+0000 7f93af7d7740 1 client.7953 shutdown 2024-04-25T10:12:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.892+0000 7f93af7d7740 1 -- 172.21.15.16:0/3956022747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c64e9a0 msgr2=0x55613c6742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.892+0000 7f93af7d7740 1 --2- 172.21.15.16:0/3956022747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c64e9a0 0x55613c6742c0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f93a0058380 tx=0x7f93a000aef0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.892+0000 7f93af7d7740 1 -- 172.21.15.16:0/3956022747 shutdown_connections 2024-04-25T10:12:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.892+0000 7f93af7d7740 1 --2- 172.21.15.16:0/3956022747 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55613c670880 0x55613c674800 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.892+0000 7f93af7d7740 1 --2- 172.21.15.16:0/3956022747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55613c64e9a0 0x55613c6742c0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.892+0000 7f93af7d7740 1 --2- 172.21.15.16:0/3956022747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55613c62abb0 0x55613c670340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.892+0000 7f93857fa640 0 client.7953 ms_handle_reset on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.892+0000 7f93af7d7740 1 --2- 172.21.15.16:0/3956022747 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55613c629cf0 0x55613c6dc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.892+0000 7f93af7d7740 1 -- 172.21.15.16:0/3956022747 >> 172.21.15.16:0/3956022747 conn(0x55613c625830 msgr2=0x55613c62af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.892+0000 7f93af7d7740 1 -- 172.21.15.16:0/3956022747 shutdown_connections 2024-04-25T10:12:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.892+0000 7f93af7d7740 1 -- 172.21.15.16:0/3956022747 wait complete. 2024-04-25T10:12:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.892+0000 7f93af7d7740 20 client.7953 trim_cache size 0 max 16384 2024-04-25T10:12:01.903 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:12:01.904 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-04-25T10:12:01.904 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "ffe8e886-2ba7-49ab-94fc-d98b0719b9d7", 2024-04-25T10:12:01.904 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-04-25T10:12:01.904 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:12:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.893+0000 7f93acad8640 1 -- 172.21.15.16:0/35224549 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f938806b4d0 msgr2=0x7f938806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.893+0000 7f93acad8640 1 --2- 172.21.15.16:0/35224549 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f938806b4d0 0x7f938806d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9398034000 tx=0x7f939802c000 comp rx=0 tx=0).stop 2024-04-25T10:12:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.893+0000 7f93acad8640 1 -- 172.21.15.16:0/35224549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a81618d0 msgr2=0x7f93a815c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.893+0000 7f93acad8640 1 --2- 172.21.15.16:0/35224549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a81618d0 0x7f93a815c010 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f93a000cbf0 tx=0x7f93a000a430 comp rx=0 tx=0).stop 2024-04-25T10:12:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.893+0000 7f93acad8640 1 -- 172.21.15.16:0/35224549 shutdown_connections 2024-04-25T10:12:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.893+0000 7f93acad8640 1 --2- 172.21.15.16:0/35224549 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f938806b4d0 0x7f938806d990 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.893+0000 7f93acad8640 1 --2- 172.21.15.16:0/35224549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a81618d0 0x7f93a815c010 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.893+0000 7f93acad8640 1 --2- 172.21.15.16:0/35224549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93a81610b0 0x7f93a80c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.893+0000 7f93acad8640 1 --2- 172.21.15.16:0/35224549 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93a80cef40 0x7f93a81f4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.893+0000 7f93acad8640 1 -- 172.21.15.16:0/35224549 >> 172.21.15.16:0/35224549 conn(0x7f93a80b5570 msgr2=0x7f93a80ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.893+0000 7f93acad8640 1 -- 172.21.15.16:0/35224549 shutdown_connections 2024-04-25T10:12:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:01.893+0000 7f93acad8640 1 -- 172.21.15.16:0/35224549 wait complete. 2024-04-25T10:12:01.906 INFO:tasks.ceph.mds.a.smithi016.stderr:2024-04-25T10:12:01.894+0000 7f82e76cf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x563b8cd90680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:12:01.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:01.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:12:02.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.007+0000 7f8ed561b640 1 Processor -- start 2024-04-25T10:12:02.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.007+0000 7f8ed561b640 1 -- start start 2024-04-25T10:12:02.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.008+0000 7f8ed561b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8ed00b5550 0x7f8ed00b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.008+0000 7f8ed561b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ed00d2560 0x7f8ed00d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.008+0000 7f8ed561b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed0161500 con 0x7f8ed0161120 2024-04-25T10:12:02.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.008+0000 7f8ed561b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed00b5ef0 con 0x7f8ed00b5550 2024-04-25T10:12:02.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.008+0000 7f8ed561b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed0162740 con 0x7f8ed00d2560 2024-04-25T10:12:02.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.008+0000 7f8eceffd640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f8ed0161120 0x7f8ed01617a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:58464/0 (socket says 172.21.15.16:58464) 2024-04-25T10:12:02.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.009+0000 7f8eceffd640 1 -- 172.21.15.16:0/3202987695 learned_addr learned my addr 172.21.15.16:0/3202987695 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:02.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.009+0000 7f8ecf7fe640 1 --2- 172.21.15.16:0/3202987695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ed00d2560 0x7f8ed00d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.009+0000 7f8ece7fc640 1 --2- 172.21.15.16:0/3202987695 >> v2:172.21.15.186:3300/0 conn(0x7f8ed00b5550 0x7f8ed00b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.009+0000 7f8ecdffb640 1 -- 172.21.15.16:0/3202987695 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2854627330 0 0) 0x7f8ed0161500 con 0x7f8ed0161120 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.009+0000 7f8ecdffb640 1 -- 172.21.15.16:0/3202987695 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ebc003520 con 0x7f8ed0161120 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.009+0000 7f8ece7fc640 1 -- 172.21.15.16:0/3202987695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ed00d2560 msgr2=0x7f8ed00d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.009+0000 7f8ece7fc640 1 --2- 172.21.15.16:0/3202987695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ed00d2560 0x7f8ed00d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.009+0000 7f8ece7fc640 1 -- 172.21.15.16:0/3202987695 >> v1:172.21.15.186:6789/0 conn(0x7f8ed0161120 legacy=0x7f8ed01617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.009+0000 7f8ece7fc640 1 -- 172.21.15.16:0/3202987695 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed0161eb0 con 0x7f8ed00b5550 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.009+0000 7f8ecf7fe640 1 --2- 172.21.15.16:0/3202987695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ed00d2560 0x7f8ed00d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.010+0000 7f8ece7fc640 1 --2- 172.21.15.16:0/3202987695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ed00b5550 0x7f8ed00b59b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8eb80097a0 tx=0x7f8eb802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c86d7127708d49ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.010+0000 7f8ecdffb640 1 -- 172.21.15.16:0/3202987695 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8037070 con 0x7f8ed00b5550 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.010+0000 7f8ecdffb640 1 -- 172.21.15.16:0/3202987695 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb802ae50 con 0x7f8ed00b5550 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.010+0000 7f8ecdffb640 1 -- 172.21.15.16:0/3202987695 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8032640 con 0x7f8ed00b5550 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.010+0000 7f8ed561b640 1 -- 172.21.15.16:0/3202987695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ed00b5550 msgr2=0x7f8ed00b59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.010+0000 7f8ed561b640 1 --2- 172.21.15.16:0/3202987695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ed00b5550 0x7f8ed00b59b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8eb80097a0 tx=0x7f8eb802a100 comp rx=0 tx=0).stop 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.010+0000 7f8ed561b640 1 -- 172.21.15.16:0/3202987695 shutdown_connections 2024-04-25T10:12:02.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.010+0000 7f8ed561b640 1 --2- 172.21.15.16:0/3202987695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ed00d2560 0x7f8ed00d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.010+0000 7f8ed561b640 1 --2- 172.21.15.16:0/3202987695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ed00b5550 0x7f8ed00b59b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.010+0000 7f8ed561b640 1 -- 172.21.15.16:0/3202987695 >> 172.21.15.16:0/3202987695 conn(0x7f8ed00d0550 msgr2=0x7f8ed00d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:02.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.010+0000 7f8ed561b640 1 -- 172.21.15.16:0/3202987695 shutdown_connections 2024-04-25T10:12:02.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.010+0000 7f8ed561b640 1 -- 172.21.15.16:0/3202987695 wait complete. 2024-04-25T10:12:02.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.012+0000 7f8ed561b640 1 Processor -- start 2024-04-25T10:12:02.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.012+0000 7f8ed561b640 1 -- start start 2024-04-25T10:12:02.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.012+0000 7f8ed561b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ed00b5550 0x7f8ed00caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.012+0000 7f8ed561b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ed00d2560 0x7f8ed0155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.012+0000 7f8ed561b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ed0161120 0x7f8ed0155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.012+0000 7f8ed561b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed00d1d60 con 0x7f8ed00b5550 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.012+0000 7f8ed561b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed0162460 con 0x7f8ed0161120 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.012+0000 7f8ed561b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed00d2040 con 0x7f8ed00d2560 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.013+0000 7f8eceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ed00b5550 0x7f8ed00caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.013+0000 7f8eceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ed00b5550 0x7f8ed00caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49672/0 (socket says 172.21.15.16:49672) 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.013+0000 7f8eceffd640 1 -- 172.21.15.16:0/2052013555 learned_addr learned my addr 172.21.15.16:0/2052013555 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.013+0000 7f8ecf7fe640 1 --2- 172.21.15.16:0/2052013555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ed0161120 0x7f8ed0155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.013+0000 7f8ece7fc640 1 --2- 172.21.15.16:0/2052013555 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ed00d2560 0x7f8ed0155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.013+0000 7f8eceffd640 1 -- 172.21.15.16:0/2052013555 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ed00d2560 msgr2=0x7f8ed0155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.013+0000 7f8eceffd640 1 --2- 172.21.15.16:0/2052013555 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ed00d2560 0x7f8ed0155700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.013+0000 7f8eceffd640 1 -- 172.21.15.16:0/2052013555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ed0161120 msgr2=0x7f8ed0155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.013+0000 7f8eceffd640 1 --2- 172.21.15.16:0/2052013555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ed0161120 0x7f8ed0155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.013+0000 7f8eceffd640 1 -- 172.21.15.16:0/2052013555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed00c9560 con 0x7f8ed00b5550 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.013+0000 7f8eceffd640 1 --2- 172.21.15.16:0/2052013555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ed00b5550 0x7f8ed00caba0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80010e0 tx=0x7f8ec800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.013+0000 7f8ec77fe640 1 -- 172.21.15.16:0/2052013555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8017070 con 0x7f8ed00b5550 2024-04-25T10:12:02.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.013+0000 7f8ec77fe640 1 -- 172.21.15.16:0/2052013555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec8005d00 con 0x7f8ed00b5550 2024-04-25T10:12:02.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.013+0000 7f8ec77fe640 1 -- 172.21.15.16:0/2052013555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8007310 con 0x7f8ed00b5550 2024-04-25T10:12:02.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.013+0000 7f8ed561b640 1 -- 172.21.15.16:0/2052013555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed00c9840 con 0x7f8ed00b5550 2024-04-25T10:12:02.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.013+0000 7f8ed561b640 1 -- 172.21.15.16:0/2052013555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed00c9e00 con 0x7f8ed00b5550 2024-04-25T10:12:02.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.015+0000 7f8ec77fe640 1 -- 172.21.15.16:0/2052013555 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ec801a070 con 0x7f8ed00b5550 2024-04-25T10:12:02.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.015+0000 7f8ec77fe640 1 --2- 172.21.15.16:0/2052013555 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8eb006b4d0 0x7f8eb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.015+0000 7f8ec77fe640 1 -- 172.21.15.16:0/2052013555 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f8ec8089610 con 0x7f8ed00b5550 2024-04-25T10:12:02.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.015+0000 7f8ece7fc640 1 --2- 172.21.15.16:0/2052013555 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8eb006b4d0 0x7f8eb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.016+0000 7f8ece7fc640 1 --2- 172.21.15.16:0/2052013555 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8eb006b4d0 0x7f8eb006d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8034000 tx=0x7f8eb802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.016+0000 7f8ed83f0740 1 Processor -- start 2024-04-25T10:12:02.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.017+0000 7f8ed83f0740 1 -- start start 2024-04-25T10:12:02.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.017+0000 7f8ed83f0740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe6716bb10 0x55fe6716e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.017+0000 7f8ed83f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bef0 0x55fe6716e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.017+0000 7f8ed83f0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe6713e770 0x55fe67141f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.017+0000 7f8ed83f0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe670a7da0 con 0x55fe6716bef0 2024-04-25T10:12:02.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.017+0000 7f8ed83f0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe67050b80 con 0x55fe6713e770 2024-04-25T10:12:02.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.017+0000 7f8ed83f0740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe670a3e40 con 0x55fe6716bb10 2024-04-25T10:12:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.017+0000 7f8ece7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bef0 0x55fe6716e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.017+0000 7f8ece7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bef0 0x55fe6716e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49682/0 (socket says 172.21.15.16:49682) 2024-04-25T10:12:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.017+0000 7f8ece7fc640 1 -- 172.21.15.16:0/2228336408 learned_addr learned my addr 172.21.15.16:0/2228336408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.018+0000 7f8eceffd640 1 --2- 172.21.15.16:0/2228336408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe6716bb10 0x55fe6716e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.018+0000 7f8ecf7fe640 1 --2- 172.21.15.16:0/2228336408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe6713e770 0x55fe67141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.018+0000 7f8ece7fc640 1 -- 172.21.15.16:0/2228336408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe6716bb10 msgr2=0x55fe6716e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.018+0000 7f8ece7fc640 1 --2- 172.21.15.16:0/2228336408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe6716bb10 0x55fe6716e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.018+0000 7f8ece7fc640 1 -- 172.21.15.16:0/2228336408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe6713e770 msgr2=0x55fe67141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.018+0000 7f8ece7fc640 1 --2- 172.21.15.16:0/2228336408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe6713e770 0x55fe67141f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.018+0000 7f8ece7fc640 1 -- 172.21.15.16:0/2228336408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe6705af10 con 0x55fe6716bef0 2024-04-25T10:12:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.018+0000 7f8eceffd640 1 --2- 172.21.15.16:0/2228336408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe6716bb10 0x55fe6716e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.018+0000 7f8ece7fc640 1 --2- 172.21.15.16:0/2228336408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bef0 0x55fe6716e860 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8035040 tx=0x7f8eb802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2585f89f66f3e4bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.018+0000 7f8ec4ff9640 1 -- 172.21.15.16:0/2228336408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb804d070 con 0x55fe6716bef0 2024-04-25T10:12:02.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.018+0000 7f8ec4ff9640 1 -- 172.21.15.16:0/2228336408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb8032e00 con 0x55fe6716bef0 2024-04-25T10:12:02.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.018+0000 7f8ec4ff9640 1 -- 172.21.15.16:0/2228336408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8037040 con 0x55fe6716bef0 2024-04-25T10:12:02.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.019+0000 7f8ed83f0740 1 -- 172.21.15.16:0/2228336408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bef0 msgr2=0x55fe6716e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.019+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/2228336408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bef0 0x55fe6716e860 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8035040 tx=0x7f8eb802c000 comp rx=0 tx=0).stop 2024-04-25T10:12:02.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.019+0000 7f8ed83f0740 1 -- 172.21.15.16:0/2228336408 shutdown_connections 2024-04-25T10:12:02.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.019+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/2228336408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe6713e770 0x55fe67141f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.019+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/2228336408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bef0 0x55fe6716e860 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.019+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/2228336408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe6716bb10 0x55fe6716e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.019+0000 7f8ed83f0740 1 -- 172.21.15.16:0/2228336408 >> 172.21.15.16:0/2228336408 conn(0x55fe671409e0 msgr2=0x55fe67189320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:02.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.019+0000 7f8ed83f0740 1 -- 172.21.15.16:0/2228336408 shutdown_connections 2024-04-25T10:12:02.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.019+0000 7f8ed83f0740 1 -- 172.21.15.16:0/2228336408 wait complete. 2024-04-25T10:12:02.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.019+0000 7f8ed83f0740 1 -- start start 2024-04-25T10:12:02.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.019+0000 7f8ed83f0740 1 Processor -- start 2024-04-25T10:12:02.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.020+0000 7f8ed83f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bb10 0x55fe67186220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.020+0000 7f8ed83f0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe6716bef0 0x55fe67142f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.020+0000 7f8ed83f0740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe671628e0 0x55fe671434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.020+0000 7f8ed83f0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe67143b90 con 0x55fe6716bb10 2024-04-25T10:12:02.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.020+0000 7f8ed83f0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe670464b0 con 0x55fe6716bef0 2024-04-25T10:12:02.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.020+0000 7f8ed83f0740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe67143e70 con 0x55fe671628e0 2024-04-25T10:12:02.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.020+0000 7f8ed83f0740 10 client.0 _ll_register_callbacks cb 0x55fe67086180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:02.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.020+0000 7f8eceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bb10 0x55fe67186220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.020+0000 7f8eceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bb10 0x55fe67186220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49686/0 (socket says 172.21.15.16:49686) 2024-04-25T10:12:02.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.020+0000 7f8eceffd640 1 -- 172.21.15.16:0/411636517 learned_addr learned my addr 172.21.15.16:0/411636517 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:02.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.020+0000 7f8ece7fc640 1 --2- 172.21.15.16:0/411636517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe6716bef0 0x55fe67142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.020+0000 7f8ecf7fe640 1 --2- 172.21.15.16:0/411636517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe671628e0 0x55fe671434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8eceffd640 1 -- 172.21.15.16:0/411636517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe671628e0 msgr2=0x55fe671434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:02.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8eceffd640 1 --2- 172.21.15.16:0/411636517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe671628e0 0x55fe671434d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8eceffd640 1 -- 172.21.15.16:0/411636517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe6716bef0 msgr2=0x55fe67142f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:02.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8eceffd640 1 --2- 172.21.15.16:0/411636517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe6716bef0 0x55fe67142f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8eceffd640 1 -- 172.21.15.16:0/411636517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe66e867d0 con 0x55fe6716bb10 2024-04-25T10:12:02.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8ecf7fe640 1 --2- 172.21.15.16:0/411636517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe671628e0 0x55fe671434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:02.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8eceffd640 1 --2- 172.21.15.16:0/411636517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bb10 0x55fe67186220 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8057a70 tx=0x7f8ec8057eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73b0246293c8b9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8eadffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:12:02.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8eadffb640 1 -- 172.21.15.16:0/411636517 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80074b0 con 0x55fe6716bb10 2024-04-25T10:12:02.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8eadffb640 1 -- 172.21.15.16:0/411636517 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f8ec8007650 con 0x55fe6716bb10 2024-04-25T10:12:02.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8ed83f0740 1 -- 172.21.15.16:0/411636517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8eb8032e00 con 0x55fe6716bb10 2024-04-25T10:12:02.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8eadffb640 1 -- 172.21.15.16:0/411636517 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ec80571c0 con 0x55fe6716bb10 2024-04-25T10:12:02.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8ed83f0740 10 client.7962 fetch_fsmap learned FSMap version 64 2024-04-25T10:12:02.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8ed83f0740 1 -- 172.21.15.16:0/411636517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55fe66dbc3f0 con 0x55fe6716bb10 2024-04-25T10:12:02.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8eadffb640 1 -- 172.21.15.16:0/411636517 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb8032e00 con 0x55fe6716bb10 2024-04-25T10:12:02.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8eadffb640 1 -- 172.21.15.16:0/411636517 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8ec8057360 con 0x55fe6716bb10 2024-04-25T10:12:02.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8ed83f0740 10 client.7962 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T10:12:02.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8ed83f0740 10 client.7962 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:12:02.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8ed83f0740 20 client.7962 populate_metadata read hostname 'smithi016' 2024-04-25T10:12:02.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/411636517 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55fe6719ac10 0x55fe6719b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8ed83f0740 4 client.7962 mds_command: new command op to 7323 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:02.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8ed83f0740 1 -- 172.21.15.16:0/411636517 --> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe670b52e0 con 0x55fe6719ac10 2024-04-25T10:12:02.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.021+0000 7f8ece7fc640 1 --2- 172.21.15.16:0/411636517 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55fe6719ac10 0x55fe6719b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.022+0000 7f8eadffb640 10 client.7962 ms_handle_connect on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:02.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.022+0000 7f8ece7fc640 1 --2- 172.21.15.16:0/411636517 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55fe6719ac10 0x55fe6719b080 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71437664c317a49f server_cookie=34fe1bfe59c80cb9 in_seq=0 out_seq=0 2024-04-25T10:12:02.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.022+0000 7f8eadffb640 1 -- 172.21.15.16:0/411636517 <== mds.0 v2:172.21.15.16:6832/84349474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fe670b52e0 con 0x55fe6719ac10 2024-04-25T10:12:02.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.022+0000 7f8eadffb640 10 client.7962 handle_command_reply: tid=0 2024-04-25T10:12:02.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.022+0000 7f8ed83f0740 1 client.7962 shutdown 2024-04-25T10:12:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.022+0000 7f8ed83f0740 1 -- 172.21.15.16:0/411636517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bb10 msgr2=0x55fe67186220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.022+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/411636517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bb10 0x55fe67186220 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8057a70 tx=0x7f8ec8057eb0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.022+0000 7f8ed83f0740 1 -- 172.21.15.16:0/411636517 shutdown_connections 2024-04-25T10:12:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.022+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/411636517 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55fe6719ac10 0x55fe6719b080 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.022+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/411636517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe671628e0 0x55fe671434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.022+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/411636517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe6716bef0 0x55fe67142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.022+0000 7f8eadffb640 0 client.7962 ms_handle_reset on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.022+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/411636517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bb10 0x55fe67186220 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.022+0000 7f8ed83f0740 1 -- 172.21.15.16:0/411636517 >> 172.21.15.16:0/411636517 conn(0x55fe6713e770 msgr2=0x55fe6716a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.023+0000 7f8ed83f0740 1 -- 172.21.15.16:0/411636517 shutdown_connections 2024-04-25T10:12:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.023+0000 7f8ed83f0740 1 -- 172.21.15.16:0/411636517 wait complete. 2024-04-25T10:12:02.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.023+0000 7f8ed83f0740 20 client.7962 trim_cache size 0 max 16384 2024-04-25T10:12:02.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.038+0000 7f8ed83f0740 1 Processor -- start 2024-04-25T10:12:02.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.038+0000 7f8ed83f0740 1 -- start start 2024-04-25T10:12:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.038+0000 7f8ed83f0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe671628e0 0x55fe6713ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.038+0000 7f8ed83f0740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe6716bb10 0x55fe6716a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.038+0000 7f8ed83f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bef0 0x55fe6716a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.038+0000 7f8ed83f0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed01ffc70 con 0x55fe6716bef0 2024-04-25T10:12:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.038+0000 7f8ed83f0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed0158810 con 0x55fe671628e0 2024-04-25T10:12:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.038+0000 7f8ed83f0740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed0157c10 con 0x55fe6716bb10 2024-04-25T10:12:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.039+0000 7f8ecf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bef0 0x55fe6716a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.039+0000 7f8ecf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bef0 0x55fe6716a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49702/0 (socket says 172.21.15.16:49702) 2024-04-25T10:12:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.039+0000 7f8ecf7fe640 1 -- 172.21.15.16:0/1023387117 learned_addr learned my addr 172.21.15.16:0/1023387117 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.039+0000 7f8ece7fc640 1 --2- 172.21.15.16:0/1023387117 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe6716bb10 0x55fe6716a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.039+0000 7f8eceffd640 1 --2- 172.21.15.16:0/1023387117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe671628e0 0x55fe6713ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.039+0000 7f8ecf7fe640 1 -- 172.21.15.16:0/1023387117 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe6716bb10 msgr2=0x55fe6716a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.039+0000 7f8ecf7fe640 1 --2- 172.21.15.16:0/1023387117 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe6716bb10 0x55fe6716a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.039+0000 7f8ecf7fe640 1 -- 172.21.15.16:0/1023387117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe671628e0 msgr2=0x55fe6713ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.039+0000 7f8ecf7fe640 1 --2- 172.21.15.16:0/1023387117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe671628e0 0x55fe6713ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.039+0000 7f8ecf7fe640 1 -- 172.21.15.16:0/1023387117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe6718ab50 con 0x55fe6716bef0 2024-04-25T10:12:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.039+0000 7f8ecf7fe640 1 --2- 172.21.15.16:0/1023387117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bef0 0x55fe6716a870 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f8ec00076e0 tx=0x7f8ec0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e11ef6c3ed6eb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.039+0000 7f8eaffff640 1 -- 172.21.15.16:0/1023387117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec001d070 con 0x55fe6716bef0 2024-04-25T10:12:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.039+0000 7f8eaffff640 1 -- 172.21.15.16:0/1023387117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec0014950 con 0x55fe6716bef0 2024-04-25T10:12:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.039+0000 7f8eaffff640 1 -- 172.21.15.16:0/1023387117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0014d40 con 0x55fe6716bef0 2024-04-25T10:12:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.040+0000 7f8ed83f0740 1 -- 172.21.15.16:0/1023387117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bef0 msgr2=0x55fe6716a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.040+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/1023387117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bef0 0x55fe6716a870 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f8ec00076e0 tx=0x7f8ec0009750 comp rx=0 tx=0).stop 2024-04-25T10:12:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.040+0000 7f8ed83f0740 1 -- 172.21.15.16:0/1023387117 shutdown_connections 2024-04-25T10:12:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.040+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/1023387117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe6716bef0 0x55fe6716a870 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.040+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/1023387117 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe6716bb10 0x55fe6716a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.040+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/1023387117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe671628e0 0x55fe6713ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.040+0000 7f8ed83f0740 1 -- 172.21.15.16:0/1023387117 >> 172.21.15.16:0/1023387117 conn(0x55fe6713e770 msgr2=0x55fe671447a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.040+0000 7f8ed83f0740 1 -- 172.21.15.16:0/1023387117 shutdown_connections 2024-04-25T10:12:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.040+0000 7f8ed83f0740 1 -- 172.21.15.16:0/1023387117 wait complete. 2024-04-25T10:12:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.040+0000 7f8ed83f0740 1 -- start start 2024-04-25T10:12:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.040+0000 7f8ed83f0740 1 Processor -- start 2024-04-25T10:12:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.041+0000 7f8ed83f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe671628e0 0x55fe671f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.041+0000 7f8ed83f0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe6716bb10 0x55fe6718b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.041+0000 7f8ed83f0740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe6716bef0 0x55fe6718b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.041+0000 7f8ed83f0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6716b140 con 0x55fe671628e0 2024-04-25T10:12:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.041+0000 7f8ed83f0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed0158210 con 0x55fe6716bb10 2024-04-25T10:12:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.041+0000 7f8ed83f0740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe671f6e10 con 0x55fe6716bef0 2024-04-25T10:12:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.041+0000 7f8ed83f0740 10 client.0 _ll_register_callbacks cb 0x7f8eb8035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.041+0000 7f8eceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe671628e0 0x55fe671f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.041+0000 7f8eceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe671628e0 0x55fe671f66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49718/0 (socket says 172.21.15.16:49718) 2024-04-25T10:12:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.041+0000 7f8eceffd640 1 -- 172.21.15.16:0/132628429 learned_addr learned my addr 172.21.15.16:0/132628429 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.041+0000 7f8ecf7fe640 1 --2- 172.21.15.16:0/132628429 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe6716bef0 0x55fe6718b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.041+0000 7f8ece7fc640 1 --2- 172.21.15.16:0/132628429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe6716bb10 0x55fe6718b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.041+0000 7f8eceffd640 1 -- 172.21.15.16:0/132628429 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe6716bef0 msgr2=0x55fe6718b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.041+0000 7f8eceffd640 1 --2- 172.21.15.16:0/132628429 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe6716bef0 0x55fe6718b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.041+0000 7f8eceffd640 1 -- 172.21.15.16:0/132628429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe6716bb10 msgr2=0x55fe6718b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.041+0000 7f8eceffd640 1 --2- 172.21.15.16:0/132628429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe6716bb10 0x55fe6718b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.041+0000 7f8eceffd640 1 -- 172.21.15.16:0/132628429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe66d7c0f0 con 0x55fe671628e0 2024-04-25T10:12:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.042+0000 7f8eceffd640 1 --2- 172.21.15.16:0/132628429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe671628e0 0x55fe671f66e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f8ec805ff50 tx=0x7f8ec8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f8a93280788c0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.042+0000 7f8eadffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:12:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.042+0000 7f8eadffb640 1 -- 172.21.15.16:0/132628429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8073070 con 0x55fe671628e0 2024-04-25T10:12:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.042+0000 7f8eadffb640 1 -- 172.21.15.16:0/132628429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f8ec8053db0 con 0x55fe671628e0 2024-04-25T10:12:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.042+0000 7f8ed83f0740 1 -- 172.21.15.16:0/132628429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ec0014950 con 0x55fe671628e0 2024-04-25T10:12:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.042+0000 7f8eadffb640 1 -- 172.21.15.16:0/132628429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ec80076c0 con 0x55fe671628e0 2024-04-25T10:12:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.042+0000 7f8ed83f0740 10 client.7968 fetch_fsmap learned FSMap version 64 2024-04-25T10:12:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.042+0000 7f8ed83f0740 1 -- 172.21.15.16:0/132628429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55fe6716a4b0 con 0x55fe671628e0 2024-04-25T10:12:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.042+0000 7f8eadffb640 1 -- 172.21.15.16:0/132628429 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec0014950 con 0x55fe671628e0 2024-04-25T10:12:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.042+0000 7f8eadffb640 1 -- 172.21.15.16:0/132628429 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8ec806fb00 con 0x55fe671628e0 2024-04-25T10:12:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.042+0000 7f8ed83f0740 10 client.7968 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T10:12:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.042+0000 7f8ed83f0740 10 client.7968 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:12:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.042+0000 7f8ed83f0740 20 client.7968 populate_metadata read hostname 'smithi016' 2024-04-25T10:12:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.042+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/132628429 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55fe6713da50 0x55fe6717eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.042+0000 7f8ed83f0740 4 client.7968 mds_command: new command op to 7323 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:12:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.042+0000 7f8ed83f0740 1 -- 172.21.15.16:0/132628429 --> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fe670af630 con 0x55fe6713da50 2024-04-25T10:12:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.042+0000 7f8ece7fc640 1 --2- 172.21.15.16:0/132628429 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55fe6713da50 0x55fe6717eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8eadffb640 10 client.7968 ms_handle_connect on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8ece7fc640 1 --2- 172.21.15.16:0/132628429 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55fe6713da50 0x55fe6717eee0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df4e87e8bc42f9e6 server_cookie=a1e48569bc56403f in_seq=0 out_seq=0 2024-04-25T10:12:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8eadffb640 1 -- 172.21.15.16:0/132628429 <== mds.0 v2:172.21.15.16:6832/84349474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x55fe670af630 con 0x55fe6713da50 2024-04-25T10:12:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8eadffb640 10 client.7968 handle_command_reply: tid=0 2024-04-25T10:12:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8ed83f0740 1 client.7968 shutdown 2024-04-25T10:12:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8ed83f0740 1 -- 172.21.15.16:0/132628429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe671628e0 msgr2=0x55fe671f66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/132628429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe671628e0 0x55fe671f66e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f8ec805ff50 tx=0x7f8ec8014040 comp rx=0 tx=0).stop 2024-04-25T10:12:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8ed83f0740 1 -- 172.21.15.16:0/132628429 shutdown_connections 2024-04-25T10:12:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/132628429 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x55fe6713da50 0x55fe6717eee0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/132628429 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fe6716bef0 0x55fe6718b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/132628429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fe6716bb10 0x55fe6718b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8eadffb640 0 client.7968 ms_handle_reset on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8ed83f0740 1 --2- 172.21.15.16:0/132628429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe671628e0 0x55fe671f66e0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8ed83f0740 1 -- 172.21.15.16:0/132628429 >> 172.21.15.16:0/132628429 conn(0x55fe6713e770 msgr2=0x55fe671447a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8ed83f0740 1 -- 172.21.15.16:0/132628429 shutdown_connections 2024-04-25T10:12:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8ed83f0740 1 -- 172.21.15.16:0/132628429 wait complete. 2024-04-25T10:12:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8ed83f0740 20 client.7968 trim_cache size 0 max 16384 2024-04-25T10:12:02.055 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:12:02.055 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:12:02.055 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-04-25T10:12:02.056 INFO:teuthology.orchestra.run.smithi016.stdout: "ffe8e886-2ba7-49ab-94fc-d98b0719b9d7": { 2024-04-25T10:12:02.056 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/dir", 2024-04-25T10:12:02.056 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "ffe8e886-2ba7-49ab-94fc-d98b0719b9d7", 2024-04-25T10:12:02.056 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,repair" 2024-04-25T10:12:02.056 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:02.056 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:02.056 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:12:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8ed561b640 1 -- 172.21.15.16:0/2052013555 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8eb006b4d0 msgr2=0x7f8eb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8ed561b640 1 --2- 172.21.15.16:0/2052013555 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8eb006b4d0 0x7f8eb006d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8034000 tx=0x7f8eb802e000 comp rx=0 tx=0).stop 2024-04-25T10:12:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8ed561b640 1 -- 172.21.15.16:0/2052013555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ed00b5550 msgr2=0x7f8ed00caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.043+0000 7f8ed561b640 1 --2- 172.21.15.16:0/2052013555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ed00b5550 0x7f8ed00caba0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80010e0 tx=0x7f8ec800c890 comp rx=0 tx=0).stop 2024-04-25T10:12:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.044+0000 7f8ed561b640 1 -- 172.21.15.16:0/2052013555 shutdown_connections 2024-04-25T10:12:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.044+0000 7f8ed561b640 1 --2- 172.21.15.16:0/2052013555 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8eb006b4d0 0x7f8eb006d990 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.044+0000 7f8ed561b640 1 --2- 172.21.15.16:0/2052013555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ed0161120 0x7f8ed0155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.044+0000 7f8ed561b640 1 --2- 172.21.15.16:0/2052013555 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ed00d2560 0x7f8ed0155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.044+0000 7f8ed561b640 1 --2- 172.21.15.16:0/2052013555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ed00b5550 0x7f8ed00caba0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.044+0000 7f8ed561b640 1 -- 172.21.15.16:0/2052013555 >> 172.21.15.16:0/2052013555 conn(0x7f8ed00d0550 msgr2=0x7f8ed00d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.044+0000 7f8ed561b640 1 -- 172.21.15.16:0/2052013555 shutdown_connections 2024-04-25T10:12:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:02.044+0000 7f8ed561b640 1 -- 172.21.15.16:0/2052013555 wait complete. 2024-04-25T10:12:02.059 INFO:tasks.cephfs.filesystem:scrub status for tag:ffe8e886-2ba7-49ab-94fc-d98b0719b9d7 - {'path': '/dir', 'tag': 'ffe8e886-2ba7-49ab-94fc-d98b0719b9d7', 'options': 'recursive,repair'} 2024-04-25T10:12:08.346 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:08.349 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:32.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:32.061 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:12:32.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.172+0000 7f00c850a640 1 Processor -- start 2024-04-25T10:12:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.172+0000 7f00c850a640 1 -- start start 2024-04-25T10:12:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.173+0000 7f00c850a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f00c00d3b90 0x7f00c00d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.173+0000 7f00c850a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c00d4090 con 0x7f00c00d0f60 2024-04-25T10:12:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.173+0000 7f00c850a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c00d4370 con 0x7f00c015dca0 2024-04-25T10:12:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.173+0000 7f00c850a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c00b6b20 con 0x7f00c00d3b90 2024-04-25T10:12:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.173+0000 7f00c6aa7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f00c00d0f60 0x7f00c00b2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53324/0 (socket says 172.21.15.16:53324) 2024-04-25T10:12:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.173+0000 7f00c6aa7640 1 -- 172.21.15.16:0/89977853 learned_addr learned my addr 172.21.15.16:0/89977853 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.173+0000 7f00c5aa5640 1 --2- 172.21.15.16:0/89977853 >> v2:172.21.15.186:3300/0 conn(0x7f00c00d3b90 0x7f00c00d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.173+0000 7f00c52a4640 1 -- 172.21.15.16:0/89977853 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1326362343 0 0) 0x7f00c00d4090 con 0x7f00c00d0f60 2024-04-25T10:12:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.173+0000 7f00c52a4640 1 -- 172.21.15.16:0/89977853 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00ac003520 con 0x7f00c00d0f60 2024-04-25T10:12:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.173+0000 7f00c52a4640 1 -- 172.21.15.16:0/89977853 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1383791534 0 0) 0x7f00ac003520 con 0x7f00c00d0f60 2024-04-25T10:12:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.173+0000 7f00c52a4640 1 -- 172.21.15.16:0/89977853 >> v2:172.21.15.186:3300/0 conn(0x7f00c00d3b90 msgr2=0x7f00c00d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.173+0000 7f00c52a4640 1 --2- 172.21.15.16:0/89977853 >> v2:172.21.15.186:3300/0 conn(0x7f00c00d3b90 0x7f00c00d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.173+0000 7f00c52a4640 1 -- 172.21.15.16:0/89977853 >> v1:172.21.15.186:6789/0 conn(0x7f00c015dca0 legacy=0x7f00c00ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.173+0000 7f00c52a4640 1 -- 172.21.15.16:0/89977853 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c00d4650 con 0x7f00c00d0f60 2024-04-25T10:12:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.173+0000 7f00c52a4640 1 -- 172.21.15.16:0/89977853 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f00b80019f0 con 0x7f00c00d0f60 2024-04-25T10:12:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.173+0000 7f00c52a4640 1 -- 172.21.15.16:0/89977853 >> v1:172.21.15.16:6789/0 conn(0x7f00c00d0f60 legacy=0x7f00c00b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.174+0000 7f00c52a4640 1 --2- 172.21.15.16:0/89977853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00ac0039b0 0x7f00ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.174+0000 7f00c52a4640 1 --2- 172.21.15.16:0/89977853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00ac004800 0x7f00ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.174+0000 7f00c52a4640 1 --2- 172.21.15.16:0/89977853 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00ac0051c0 0x7f00ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.174+0000 7f00c52a4640 1 -- 172.21.15.16:0/89977853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac00c030 con 0x7f00ac0039b0 2024-04-25T10:12:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.174+0000 7f00c52a4640 1 -- 172.21.15.16:0/89977853 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac00c1e0 con 0x7f00ac004800 2024-04-25T10:12:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.174+0000 7f00c52a4640 1 -- 172.21.15.16:0/89977853 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac00c3e0 con 0x7f00ac0051c0 2024-04-25T10:12:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.174+0000 7f00c62a6640 1 --2- 172.21.15.16:0/89977853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00ac0039b0 0x7f00ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.174+0000 7f00c5aa5640 1 --2- 172.21.15.16:0/89977853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00ac004800 0x7f00ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.174+0000 7f00c6aa7640 1 --2- 172.21.15.16:0/89977853 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00ac0051c0 0x7f00ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.174+0000 7f00c62a6640 1 -- 172.21.15.16:0/89977853 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00ac0051c0 msgr2=0x7f00ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.174+0000 7f00c62a6640 1 --2- 172.21.15.16:0/89977853 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00ac0051c0 0x7f00ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.174+0000 7f00c62a6640 1 -- 172.21.15.16:0/89977853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00ac004800 msgr2=0x7f00ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.174+0000 7f00c62a6640 1 --2- 172.21.15.16:0/89977853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00ac004800 0x7f00ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.174+0000 7f00c62a6640 1 -- 172.21.15.16:0/89977853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac003520 con 0x7f00ac0039b0 2024-04-25T10:12:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.174+0000 7f00c6aa7640 1 -- 172.21.15.16:0/89977853 reap_dead start 2024-04-25T10:12:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.175+0000 7f00c62a6640 1 --2- 172.21.15.16:0/89977853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00ac0039b0 0x7f00ac003dd0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f00bc001750 tx=0x7f00ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8595aa71337d6c3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.175+0000 7f00c52a4640 1 -- 172.21.15.16:0/89977853 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc001370 con 0x7f00ac0039b0 2024-04-25T10:12:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.175+0000 7f00c52a4640 1 -- 172.21.15.16:0/89977853 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc0052d0 con 0x7f00ac0039b0 2024-04-25T10:12:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.175+0000 7f00c52a4640 1 -- 172.21.15.16:0/89977853 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc02abe0 con 0x7f00ac0039b0 2024-04-25T10:12:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.175+0000 7f00c850a640 1 -- 172.21.15.16:0/89977853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00ac0039b0 msgr2=0x7f00ac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.175+0000 7f00c850a640 1 --2- 172.21.15.16:0/89977853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00ac0039b0 0x7f00ac003dd0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f00bc001750 tx=0x7f00ac005830 comp rx=0 tx=0).stop 2024-04-25T10:12:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.175+0000 7f00c850a640 1 -- 172.21.15.16:0/89977853 shutdown_connections 2024-04-25T10:12:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.176+0000 7f00c850a640 1 --2- 172.21.15.16:0/89977853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00ac0039b0 0x7f00ac003dd0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.176+0000 7f00c850a640 1 -- 172.21.15.16:0/89977853 >> 172.21.15.16:0/89977853 conn(0x7f00c00726c0 msgr2=0x7f00c0072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.176+0000 7f00c850a640 1 -- 172.21.15.16:0/89977853 shutdown_connections 2024-04-25T10:12:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.176+0000 7f00c850a640 1 -- 172.21.15.16:0/89977853 wait complete. 2024-04-25T10:12:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.177+0000 7f00c850a640 1 Processor -- start 2024-04-25T10:12:32.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.177+0000 7f00c850a640 1 -- start start 2024-04-25T10:12:32.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.177+0000 7f00c850a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00ac0039b0 0x7f00c0159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.177+0000 7f00c850a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c0157e40 0x7f00c01582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00c850a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00c0155a70 0x7f00c0155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00c850a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c015e080 con 0x7f00c0157e40 2024-04-25T10:12:32.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00c850a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c00d1390 con 0x7f00c0155a70 2024-04-25T10:12:32.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00c850a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c015e200 con 0x7f00ac0039b0 2024-04-25T10:12:32.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00c5aa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c0157e40 0x7f00c01582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00c5aa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c0157e40 0x7f00c01582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48832/0 (socket says 172.21.15.16:48832) 2024-04-25T10:12:32.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00c5aa5640 1 -- 172.21.15.16:0/2445526255 learned_addr learned my addr 172.21.15.16:0/2445526255 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:32.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00c62a6640 1 --2- 172.21.15.16:0/2445526255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00ac0039b0 0x7f00c0159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00c6aa7640 1 --2- 172.21.15.16:0/2445526255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00c0155a70 0x7f00c0155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00c5aa5640 1 -- 172.21.15.16:0/2445526255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00ac0039b0 msgr2=0x7f00c0159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00c5aa5640 1 --2- 172.21.15.16:0/2445526255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00ac0039b0 0x7f00c0159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00c5aa5640 1 -- 172.21.15.16:0/2445526255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00c0155a70 msgr2=0x7f00c0155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00c5aa5640 1 --2- 172.21.15.16:0/2445526255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00c0155a70 0x7f00c0155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00c5aa5640 1 -- 172.21.15.16:0/2445526255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c015e580 con 0x7f00c0157e40 2024-04-25T10:12:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00c5aa5640 1 --2- 172.21.15.16:0/2445526255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c0157e40 0x7f00c01582c0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f00b00118c0 tx=0x7f00b0011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00b6ffd640 1 -- 172.21.15.16:0/2445526255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b0019070 con 0x7f00c0157e40 2024-04-25T10:12:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00b6ffd640 1 -- 172.21.15.16:0/2445526255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b0004240 con 0x7f00c0157e40 2024-04-25T10:12:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00c850a640 1 -- 172.21.15.16:0/2445526255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c015e860 con 0x7f00c0157e40 2024-04-25T10:12:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.178+0000 7f00b6ffd640 1 -- 172.21.15.16:0/2445526255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b000f7c0 con 0x7f00c0157e40 2024-04-25T10:12:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.179+0000 7f00c850a640 1 -- 172.21.15.16:0/2445526255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c015ee20 con 0x7f00c0157e40 2024-04-25T10:12:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.180+0000 7f00b6ffd640 1 -- 172.21.15.16:0/2445526255 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f00b001c070 con 0x7f00c0157e40 2024-04-25T10:12:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.181+0000 7f00b6ffd640 1 --2- 172.21.15.16:0/2445526255 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f00a406b4d0 0x7f00a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.181+0000 7f00b6ffd640 1 -- 172.21.15.16:0/2445526255 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f00b008a510 con 0x7f00c0157e40 2024-04-25T10:12:32.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.181+0000 7f00c62a6640 1 --2- 172.21.15.16:0/2445526255 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f00a406b4d0 0x7f00a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.182+0000 7f00c62a6640 1 --2- 172.21.15.16:0/2445526255 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f00a406b4d0 0x7f00a406d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f00bc032000 tx=0x7f00bc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.182+0000 7f00cb206740 1 Processor -- start 2024-04-25T10:12:32.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.182+0000 7f00cb206740 1 -- start start 2024-04-25T10:12:32.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.183+0000 7f00cb206740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559150e72b10 0x559150e75320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.183+0000 7f00cb206740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 0x559150e75860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.183+0000 7f00cb206740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559150e45770 0x559150e48f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.183+0000 7f00cb206740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559150daeda0 con 0x559150e72ef0 2024-04-25T10:12:32.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.183+0000 7f00cb206740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559150d57b80 con 0x559150e72b10 2024-04-25T10:12:32.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.183+0000 7f00cb206740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x559150daae40 con 0x559150e45770 2024-04-25T10:12:32.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.183+0000 7f00c5aa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 0x559150e75860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.183+0000 7f00c5aa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 0x559150e75860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48838/0 (socket says 172.21.15.16:48838) 2024-04-25T10:12:32.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.183+0000 7f00c5aa5640 1 -- 172.21.15.16:0/4079612653 learned_addr learned my addr 172.21.15.16:0/4079612653 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.183+0000 7f00c62a6640 1 --2- 172.21.15.16:0/4079612653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559150e72b10 0x559150e75320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.183+0000 7f00c6aa7640 1 --2- 172.21.15.16:0/4079612653 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559150e45770 0x559150e48f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.183+0000 7f00c5aa5640 1 -- 172.21.15.16:0/4079612653 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559150e45770 msgr2=0x559150e48f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.183+0000 7f00c5aa5640 1 --2- 172.21.15.16:0/4079612653 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559150e45770 0x559150e48f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.183+0000 7f00c5aa5640 1 -- 172.21.15.16:0/4079612653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559150e72b10 msgr2=0x559150e75320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.183+0000 7f00c5aa5640 1 --2- 172.21.15.16:0/4079612653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559150e72b10 0x559150e75320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.183+0000 7f00c5aa5640 1 -- 172.21.15.16:0/4079612653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559150d61f10 con 0x559150e72ef0 2024-04-25T10:12:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.184+0000 7f00c5aa5640 1 --2- 172.21.15.16:0/4079612653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 0x559150e75860 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f00b0088900 tx=0x7f00b00586b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ff0ce8f759441 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.184+0000 7f00a3fff640 1 -- 172.21.15.16:0/4079612653 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b0023040 con 0x559150e72ef0 2024-04-25T10:12:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.184+0000 7f00a3fff640 1 -- 172.21.15.16:0/4079612653 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b00612f0 con 0x559150e72ef0 2024-04-25T10:12:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.184+0000 7f00a3fff640 1 -- 172.21.15.16:0/4079612653 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b0055300 con 0x559150e72ef0 2024-04-25T10:12:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.184+0000 7f00cb206740 1 -- 172.21.15.16:0/4079612653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 msgr2=0x559150e75860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.184+0000 7f00cb206740 1 --2- 172.21.15.16:0/4079612653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 0x559150e75860 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f00b0088900 tx=0x7f00b00586b0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.184+0000 7f00cb206740 1 -- 172.21.15.16:0/4079612653 shutdown_connections 2024-04-25T10:12:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.184+0000 7f00cb206740 1 --2- 172.21.15.16:0/4079612653 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559150e45770 0x559150e48f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.184+0000 7f00cb206740 1 --2- 172.21.15.16:0/4079612653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 0x559150e75860 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.184+0000 7f00cb206740 1 --2- 172.21.15.16:0/4079612653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559150e72b10 0x559150e75320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.184+0000 7f00cb206740 1 -- 172.21.15.16:0/4079612653 >> 172.21.15.16:0/4079612653 conn(0x559150e479e0 msgr2=0x559150e90320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.185+0000 7f00cb206740 1 -- 172.21.15.16:0/4079612653 shutdown_connections 2024-04-25T10:12:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.185+0000 7f00cb206740 1 -- 172.21.15.16:0/4079612653 wait complete. 2024-04-25T10:12:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.185+0000 7f00cb206740 1 -- start start 2024-04-25T10:12:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.185+0000 7f00cb206740 1 Processor -- start 2024-04-25T10:12:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.186+0000 7f00cb206740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559150e72b10 0x559150e68580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.186+0000 7f00cb206740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 0x559150e8cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.186+0000 7f00cb206740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559150e92670 0x559150e92ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.186+0000 7f00cb206740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559150e4ab90 con 0x559150e72ef0 2024-04-25T10:12:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.186+0000 7f00cb206740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559150d4d4b0 con 0x559150e72b10 2024-04-25T10:12:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.186+0000 7f00cb206740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x559150e4ae70 con 0x559150e92670 2024-04-25T10:12:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.186+0000 7f00cb206740 10 client.0 _ll_register_callbacks cb 0x559150d8d180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.186+0000 7f00c5aa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 0x559150e8cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.186+0000 7f00c5aa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 0x559150e8cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48840/0 (socket says 172.21.15.16:48840) 2024-04-25T10:12:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.186+0000 7f00c5aa5640 1 -- 172.21.15.16:0/72446267 learned_addr learned my addr 172.21.15.16:0/72446267 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.186+0000 7f00c6aa7640 1 --2- 172.21.15.16:0/72446267 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559150e92670 0x559150e92ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.186+0000 7f00c62a6640 1 --2- 172.21.15.16:0/72446267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559150e72b10 0x559150e68580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00c5aa5640 1 -- 172.21.15.16:0/72446267 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559150e92670 msgr2=0x559150e92ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00c5aa5640 1 --2- 172.21.15.16:0/72446267 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559150e92670 0x559150e92ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00c5aa5640 1 -- 172.21.15.16:0/72446267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559150e72b10 msgr2=0x559150e68580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00c5aa5640 1 --2- 172.21.15.16:0/72446267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559150e72b10 0x559150e68580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00c5aa5640 1 -- 172.21.15.16:0/72446267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559150b8d7d0 con 0x559150e72ef0 2024-04-25T10:12:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00c5aa5640 1 --2- 172.21.15.16:0/72446267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 0x559150e8cea0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f00b0058990 tx=0x7f00b0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0609d68cc7db1ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:12:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00a17fa640 1 -- 172.21.15.16:0/72446267 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b0023040 con 0x559150e72ef0 2024-04-25T10:12:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00a17fa640 1 -- 172.21.15.16:0/72446267 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f00b0061060 con 0x559150e72ef0 2024-04-25T10:12:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00cb206740 1 -- 172.21.15.16:0/72446267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00b00612f0 con 0x559150e72ef0 2024-04-25T10:12:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00a17fa640 1 -- 172.21.15.16:0/72446267 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00b006cba0 con 0x559150e72ef0 2024-04-25T10:12:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00cb206740 10 client.7980 fetch_fsmap learned FSMap version 64 2024-04-25T10:12:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00cb206740 1 -- 172.21.15.16:0/72446267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x559150ac33f0 con 0x559150e72ef0 2024-04-25T10:12:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00a17fa640 1 -- 172.21.15.16:0/72446267 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b00612f0 con 0x559150e72ef0 2024-04-25T10:12:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00a17fa640 1 -- 172.21.15.16:0/72446267 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f00b0083070 con 0x559150e72ef0 2024-04-25T10:12:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00cb206740 10 client.7980 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T10:12:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00cb206740 10 client.7980 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:12:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00cb206740 20 client.7980 populate_metadata read hostname 'smithi016' 2024-04-25T10:12:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00cb206740 1 --2- 172.21.15.16:0/72446267 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x559150e8ae20 0x559150e8b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00cb206740 4 client.7980 mds_command: new command op to 7323 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.187+0000 7f00cb206740 1 -- 172.21.15.16:0/72446267 --> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559150dbc2e0 con 0x559150e8ae20 2024-04-25T10:12:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.188+0000 7f00c62a6640 1 --2- 172.21.15.16:0/72446267 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x559150e8ae20 0x559150e8b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.188+0000 7f00a17fa640 10 client.7980 ms_handle_connect on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.188+0000 7f00c62a6640 1 --2- 172.21.15.16:0/72446267 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x559150e8ae20 0x559150e8b200 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d75ba84a27b0dda server_cookie=aff4df33ab02a110 in_seq=0 out_seq=0 2024-04-25T10:12:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.188+0000 7f00a17fa640 1 -- 172.21.15.16:0/72446267 <== mds.0 v2:172.21.15.16:6832/84349474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559150dbc2e0 con 0x559150e8ae20 2024-04-25T10:12:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.188+0000 7f00a17fa640 10 client.7980 handle_command_reply: tid=0 2024-04-25T10:12:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.188+0000 7f00cb206740 1 client.7980 shutdown 2024-04-25T10:12:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.189+0000 7f00cb206740 1 -- 172.21.15.16:0/72446267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 msgr2=0x559150e8cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.189+0000 7f00cb206740 1 --2- 172.21.15.16:0/72446267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 0x559150e8cea0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f00b0058990 tx=0x7f00b0000bc0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.189+0000 7f00cb206740 1 -- 172.21.15.16:0/72446267 shutdown_connections 2024-04-25T10:12:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.189+0000 7f00cb206740 1 --2- 172.21.15.16:0/72446267 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x559150e8ae20 0x559150e8b200 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.189+0000 7f00cb206740 1 --2- 172.21.15.16:0/72446267 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559150e92670 0x559150e92ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.189+0000 7f00cb206740 1 --2- 172.21.15.16:0/72446267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 0x559150e8cea0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.189+0000 7f00cb206740 1 --2- 172.21.15.16:0/72446267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559150e72b10 0x559150e68580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.189+0000 7f00a17fa640 0 client.7980 ms_handle_reset on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:32.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.189+0000 7f00cb206740 1 -- 172.21.15.16:0/72446267 >> 172.21.15.16:0/72446267 conn(0x559150e45770 msgr2=0x559150e71ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.189+0000 7f00cb206740 1 -- 172.21.15.16:0/72446267 shutdown_connections 2024-04-25T10:12:32.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.189+0000 7f00cb206740 1 -- 172.21.15.16:0/72446267 wait complete. 2024-04-25T10:12:32.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.189+0000 7f00cb206740 20 client.7980 trim_cache size 0 max 16384 2024-04-25T10:12:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.204+0000 7f00cb206740 1 Processor -- start 2024-04-25T10:12:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.204+0000 7f00cb206740 1 -- start start 2024-04-25T10:12:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.205+0000 7f00cb206740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559150e72b10 0x559150e93b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.205+0000 7f00cb206740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559150e72ef0 0x559150e4a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.205+0000 7f00cb206740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e92670 0x559150e90320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.205+0000 7f00cb206740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559150e98a40 con 0x559150e92670 2024-04-25T10:12:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.205+0000 7f00cb206740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c0200820 con 0x559150e72ef0 2024-04-25T10:12:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.205+0000 7f00cb206740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c01ff9e0 con 0x559150e72b10 2024-04-25T10:12:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.205+0000 7f00c6aa7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e92670 0x559150e90320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.205+0000 7f00c6aa7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e92670 0x559150e90320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48852/0 (socket says 172.21.15.16:48852) 2024-04-25T10:12:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.205+0000 7f00c6aa7640 1 -- 172.21.15.16:0/874351746 learned_addr learned my addr 172.21.15.16:0/874351746 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.205+0000 7f00c6aa7640 1 -- 172.21.15.16:0/874351746 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559150e72b10 msgr2=0x559150e93b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.205+0000 7f00c6aa7640 1 --2- 172.21.15.16:0/874351746 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559150e72b10 0x559150e93b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.205+0000 7f00c6aa7640 1 -- 172.21.15.16:0/874351746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559150e72ef0 msgr2=0x559150e4a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.205+0000 7f00c6aa7640 1 --2- 172.21.15.16:0/874351746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559150e72ef0 0x559150e4a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.205+0000 7f00c6aa7640 1 -- 172.21.15.16:0/874351746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559150a830f0 con 0x559150e92670 2024-04-25T10:12:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.206+0000 7f00c6aa7640 1 --2- 172.21.15.16:0/874351746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e92670 0x559150e90320 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f00c008ce10 tx=0x7f00b8002130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f74a2e74715fc86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.206+0000 7f00a37fe640 1 -- 172.21.15.16:0/874351746 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b8004b60 con 0x559150e92670 2024-04-25T10:12:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.206+0000 7f00a37fe640 1 -- 172.21.15.16:0/874351746 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b8004d00 con 0x559150e92670 2024-04-25T10:12:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.206+0000 7f00a37fe640 1 -- 172.21.15.16:0/874351746 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b8006330 con 0x559150e92670 2024-04-25T10:12:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.206+0000 7f00cb206740 1 -- 172.21.15.16:0/874351746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e92670 msgr2=0x559150e90320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.206+0000 7f00cb206740 1 --2- 172.21.15.16:0/874351746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e92670 0x559150e90320 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f00c008ce10 tx=0x7f00b8002130 comp rx=0 tx=0).stop 2024-04-25T10:12:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.206+0000 7f00cb206740 1 -- 172.21.15.16:0/874351746 shutdown_connections 2024-04-25T10:12:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.206+0000 7f00cb206740 1 --2- 172.21.15.16:0/874351746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e92670 0x559150e90320 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.206+0000 7f00cb206740 1 --2- 172.21.15.16:0/874351746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559150e72ef0 0x559150e4a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.206+0000 7f00cb206740 1 --2- 172.21.15.16:0/874351746 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559150e72b10 0x559150e93b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.206+0000 7f00cb206740 1 -- 172.21.15.16:0/874351746 >> 172.21.15.16:0/874351746 conn(0x559150e45770 msgr2=0x559150e8cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.206+0000 7f00cb206740 1 -- 172.21.15.16:0/874351746 shutdown_connections 2024-04-25T10:12:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.206+0000 7f00cb206740 1 -- 172.21.15.16:0/874351746 wait complete. 2024-04-25T10:12:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.207+0000 7f00cb206740 1 -- start start 2024-04-25T10:12:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.207+0000 7f00cb206740 1 Processor -- start 2024-04-25T10:12:32.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.208+0000 7f00cb206740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559150e72b10 0x559150e41060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.208+0000 7f00cb206740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 0x559150e8b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.208+0000 7f00cb206740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559150e92670 0x559150ef9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.208+0000 7f00cb206740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c01ff3e0 con 0x559150e72ef0 2024-04-25T10:12:32.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.208+0000 7f00cb206740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c01550b0 con 0x559150e72b10 2024-04-25T10:12:32.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.208+0000 7f00cb206740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x559150e8bbd0 con 0x559150e92670 2024-04-25T10:12:32.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.208+0000 7f00cb206740 10 client.0 _ll_register_callbacks cb 0x7f00ac0036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:32.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.208+0000 7f00c5aa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 0x559150e8b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.208+0000 7f00c5aa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 0x559150e8b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48862/0 (socket says 172.21.15.16:48862) 2024-04-25T10:12:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.208+0000 7f00c5aa5640 1 -- 172.21.15.16:0/2656242313 learned_addr learned my addr 172.21.15.16:0/2656242313 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.208+0000 7f00c5aa5640 1 -- 172.21.15.16:0/2656242313 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559150e92670 msgr2=0x559150ef9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.208+0000 7f00c5aa5640 1 --2- 172.21.15.16:0/2656242313 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559150e92670 0x559150ef9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.208+0000 7f00c5aa5640 1 -- 172.21.15.16:0/2656242313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559150e72b10 msgr2=0x559150e41060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.208+0000 7f00c5aa5640 1 --2- 172.21.15.16:0/2656242313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559150e72b10 0x559150e41060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.208+0000 7f00c5aa5640 1 -- 172.21.15.16:0/2656242313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559150ef99b0 con 0x559150e72ef0 2024-04-25T10:12:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.209+0000 7f00c5aa5640 1 --2- 172.21.15.16:0/2656242313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 0x559150e8b200 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f00b0055610 tx=0x7f00b0088860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36cda60d136b6245 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.209+0000 7f00a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:12:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.209+0000 7f00a17fa640 1 -- 172.21.15.16:0/2656242313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b0023030 con 0x559150e72ef0 2024-04-25T10:12:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.209+0000 7f00a17fa640 1 -- 172.21.15.16:0/2656242313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f00b0010dd0 con 0x559150e72ef0 2024-04-25T10:12:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.209+0000 7f00cb206740 1 -- 172.21.15.16:0/2656242313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00b8004d00 con 0x559150e72ef0 2024-04-25T10:12:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.209+0000 7f00a17fa640 1 -- 172.21.15.16:0/2656242313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00b006ce30 con 0x559150e72ef0 2024-04-25T10:12:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.209+0000 7f00cb206740 10 client.7986 fetch_fsmap learned FSMap version 64 2024-04-25T10:12:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.209+0000 7f00cb206740 1 -- 172.21.15.16:0/2656242313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x559150e4b190 con 0x559150e72ef0 2024-04-25T10:12:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.209+0000 7f00a17fa640 1 -- 172.21.15.16:0/2656242313 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b8004d00 con 0x559150e72ef0 2024-04-25T10:12:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.209+0000 7f00a17fa640 1 -- 172.21.15.16:0/2656242313 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f00b00549e0 con 0x559150e72ef0 2024-04-25T10:12:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.209+0000 7f00cb206740 10 client.7986 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T10:12:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.209+0000 7f00cb206740 10 client.7986 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:12:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.209+0000 7f00cb206740 20 client.7986 populate_metadata read hostname 'smithi016' 2024-04-25T10:12:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.209+0000 7f00cb206740 1 --2- 172.21.15.16:0/2656242313 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x559150e86450 0x559150e86850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.209+0000 7f00cb206740 4 client.7986 mds_command: new command op to 7323 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:12:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.209+0000 7f00cb206740 1 -- 172.21.15.16:0/2656242313 --> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559150db6630 con 0x559150e86450 2024-04-25T10:12:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.209+0000 7f00c62a6640 1 --2- 172.21.15.16:0/2656242313 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x559150e86450 0x559150e86850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.210+0000 7f00a17fa640 10 client.7986 ms_handle_connect on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.210+0000 7f00c62a6640 1 --2- 172.21.15.16:0/2656242313 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x559150e86450 0x559150e86850 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8115e3d984286de server_cookie=122b2527d47c89da in_seq=0 out_seq=0 2024-04-25T10:12:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.210+0000 7f00a17fa640 1 -- 172.21.15.16:0/2656242313 <== mds.0 v2:172.21.15.16:6832/84349474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559150db6630 con 0x559150e86450 2024-04-25T10:12:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.210+0000 7f00a17fa640 10 client.7986 handle_command_reply: tid=0 2024-04-25T10:12:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.210+0000 7f00cb206740 1 client.7986 shutdown 2024-04-25T10:12:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.210+0000 7f00cb206740 1 -- 172.21.15.16:0/2656242313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 msgr2=0x559150e8b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.210+0000 7f00cb206740 1 --2- 172.21.15.16:0/2656242313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 0x559150e8b200 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f00b0055610 tx=0x7f00b0088860 comp rx=0 tx=0).stop 2024-04-25T10:12:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.210+0000 7f00cb206740 1 -- 172.21.15.16:0/2656242313 shutdown_connections 2024-04-25T10:12:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.210+0000 7f00cb206740 1 --2- 172.21.15.16:0/2656242313 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x559150e86450 0x559150e86850 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.210+0000 7f00cb206740 1 --2- 172.21.15.16:0/2656242313 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x559150e92670 0x559150ef9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.210+0000 7f00cb206740 1 --2- 172.21.15.16:0/2656242313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559150e72ef0 0x559150e8b200 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.210+0000 7f00cb206740 1 --2- 172.21.15.16:0/2656242313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559150e72b10 0x559150e41060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.210+0000 7f00a17fa640 0 client.7986 ms_handle_reset on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.210+0000 7f00cb206740 1 -- 172.21.15.16:0/2656242313 >> 172.21.15.16:0/2656242313 conn(0x559150e45770 msgr2=0x559150e85cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.210+0000 7f00cb206740 1 -- 172.21.15.16:0/2656242313 shutdown_connections 2024-04-25T10:12:32.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.210+0000 7f00cb206740 1 -- 172.21.15.16:0/2656242313 wait complete. 2024-04-25T10:12:32.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.210+0000 7f00cb206740 20 client.7986 trim_cache size 0 max 16384 2024-04-25T10:12:32.222 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:12:32.222 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-04-25T10:12:32.222 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-04-25T10:12:32.222 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:12:32.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.211+0000 7f00c850a640 1 -- 172.21.15.16:0/2445526255 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f00a406b4d0 msgr2=0x7f00a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.211+0000 7f00c850a640 1 --2- 172.21.15.16:0/2445526255 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f00a406b4d0 0x7f00a406d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f00bc032000 tx=0x7f00bc02e000 comp rx=0 tx=0).stop 2024-04-25T10:12:32.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.211+0000 7f00c850a640 1 -- 172.21.15.16:0/2445526255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c0157e40 msgr2=0x7f00c01582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.211+0000 7f00c850a640 1 --2- 172.21.15.16:0/2445526255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c0157e40 0x7f00c01582c0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f00b00118c0 tx=0x7f00b0011d80 comp rx=0 tx=0).stop 2024-04-25T10:12:32.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.211+0000 7f00c850a640 1 -- 172.21.15.16:0/2445526255 shutdown_connections 2024-04-25T10:12:32.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.211+0000 7f00c850a640 1 --2- 172.21.15.16:0/2445526255 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f00a406b4d0 0x7f00a406d990 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.211+0000 7f00c850a640 1 --2- 172.21.15.16:0/2445526255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00c0155a70 0x7f00c0155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.211+0000 7f00c850a640 1 --2- 172.21.15.16:0/2445526255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c0157e40 0x7f00c01582c0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.211+0000 7f00c850a640 1 --2- 172.21.15.16:0/2445526255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00ac0039b0 0x7f00c0159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.211+0000 7f00c850a640 1 -- 172.21.15.16:0/2445526255 >> 172.21.15.16:0/2445526255 conn(0x7f00c00726c0 msgr2=0x7f00c015cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.211+0000 7f00c850a640 1 -- 172.21.15.16:0/2445526255 shutdown_connections 2024-04-25T10:12:32.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.211+0000 7f00c850a640 1 -- 172.21.15.16:0/2445526255 wait complete. 2024-04-25T10:12:32.226 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:12:32.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:32.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.324+0000 7fa824f5c640 1 Processor -- start 2024-04-25T10:12:32.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.324+0000 7fa824f5c640 1 -- start start 2024-04-25T10:12:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.325+0000 7fa824f5c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa8200b5b60 0x7fa8201687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.325+0000 7fa824f5c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa82015e0c0 con 0x7fa82015dce0 2024-04-25T10:12:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.325+0000 7fa824f5c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8200d2510 con 0x7fa8200d35b0 2024-04-25T10:12:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.325+0000 7fa824f5c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8200b7b10 con 0x7fa8200b5b60 2024-04-25T10:12:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.325+0000 7fa81e59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa82015dce0 0x7fa8200d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53334/0 (socket says 172.21.15.16:53334) 2024-04-25T10:12:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.325+0000 7fa81e59c640 1 -- 172.21.15.16:0/4199121074 learned_addr learned my addr 172.21.15.16:0/4199121074 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.325+0000 7fa81ed9d640 1 --2- 172.21.15.16:0/4199121074 >> v2:172.21.15.186:3300/0 conn(0x7fa8200b5b60 0x7fa8201687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.325+0000 7fa81d59a640 1 -- 172.21.15.16:0/4199121074 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239234107 0 0) 0x7fa82015e0c0 con 0x7fa82015dce0 2024-04-25T10:12:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.325+0000 7fa81d59a640 1 -- 172.21.15.16:0/4199121074 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa808003520 con 0x7fa82015dce0 2024-04-25T10:12:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.325+0000 7fa81ed9d640 1 -- 172.21.15.16:0/4199121074 >> v1:172.21.15.186:6789/0 conn(0x7fa8200d35b0 legacy=0x7fa8200b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.325+0000 7fa81ed9d640 1 -- 172.21.15.16:0/4199121074 >> v1:172.21.15.16:6789/0 conn(0x7fa82015dce0 legacy=0x7fa8200d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.325+0000 7fa81ed9d640 1 -- 172.21.15.16:0/4199121074 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8200d27f0 con 0x7fa8200b5b60 2024-04-25T10:12:32.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.326+0000 7fa81ed9d640 1 --2- 172.21.15.16:0/4199121074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8200b5b60 0x7fa8201687f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa818009d90 tx=0x7fa81802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6b9e006ef611fa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.326+0000 7fa81d59a640 1 -- 172.21.15.16:0/4199121074 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818007740 con 0x7fa8200b5b60 2024-04-25T10:12:32.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.326+0000 7fa81d59a640 1 -- 172.21.15.16:0/4199121074 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8180078e0 con 0x7fa8200b5b60 2024-04-25T10:12:32.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.326+0000 7fa81d59a640 1 -- 172.21.15.16:0/4199121074 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818007c90 con 0x7fa8200b5b60 2024-04-25T10:12:32.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.327+0000 7fa824f5c640 1 -- 172.21.15.16:0/4199121074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8200b5b60 msgr2=0x7fa8201687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.327+0000 7fa824f5c640 1 --2- 172.21.15.16:0/4199121074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8200b5b60 0x7fa8201687f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa818009d90 tx=0x7fa81802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.327+0000 7fa824f5c640 1 -- 172.21.15.16:0/4199121074 shutdown_connections 2024-04-25T10:12:32.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.327+0000 7fa824f5c640 1 --2- 172.21.15.16:0/4199121074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8200b5b60 0x7fa8201687f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.327+0000 7fa824f5c640 1 -- 172.21.15.16:0/4199121074 >> 172.21.15.16:0/4199121074 conn(0x7fa820072900 msgr2=0x7fa8200d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.327+0000 7fa824f5c640 1 -- 172.21.15.16:0/4199121074 shutdown_connections 2024-04-25T10:12:32.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.327+0000 7fa824f5c640 1 -- 172.21.15.16:0/4199121074 wait complete. 2024-04-25T10:12:32.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.328+0000 7fa824f5c640 1 Processor -- start 2024-04-25T10:12:32.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.328+0000 7fa824f5c640 1 -- start start 2024-04-25T10:12:32.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa824f5c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8200b5b60 0x7fa82014de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa824f5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8200d35b0 0x7fa8200cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa824f5c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa82015dce0 0x7fa82015b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa824f5c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8200d3090 con 0x7fa8200d35b0 2024-04-25T10:12:32.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa824f5c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8200d2db0 con 0x7fa82015dce0 2024-04-25T10:12:32.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa824f5c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8200d2160 con 0x7fa8200b5b60 2024-04-25T10:12:32.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa81dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8200d35b0 0x7fa8200cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa81dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8200d35b0 0x7fa8200cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48874/0 (socket says 172.21.15.16:48874) 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa81dd9b640 1 -- 172.21.15.16:0/1694167034 learned_addr learned my addr 172.21.15.16:0/1694167034 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa81e59c640 1 --2- 172.21.15.16:0/1694167034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8200b5b60 0x7fa82014de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa81ed9d640 1 --2- 172.21.15.16:0/1694167034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa82015dce0 0x7fa82015b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa81dd9b640 1 -- 172.21.15.16:0/1694167034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8200b5b60 msgr2=0x7fa82014de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa81dd9b640 1 --2- 172.21.15.16:0/1694167034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8200b5b60 0x7fa82014de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa81dd9b640 1 -- 172.21.15.16:0/1694167034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa82015dce0 msgr2=0x7fa82015b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa81dd9b640 1 --2- 172.21.15.16:0/1694167034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa82015dce0 0x7fa82015b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa81dd9b640 1 -- 172.21.15.16:0/1694167034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8200cd900 con 0x7fa8200d35b0 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa81e59c640 1 --2- 172.21.15.16:0/1694167034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8200b5b60 0x7fa82014de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa81ed9d640 1 --2- 172.21.15.16:0/1694167034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa82015dce0 0x7fa82015b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.329+0000 7fa81dd9b640 1 --2- 172.21.15.16:0/1694167034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8200d35b0 0x7fa8200cc010 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fa81400c440 tx=0x7fa814001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.330+0000 7fa80effd640 1 -- 172.21.15.16:0/1694167034 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8140052d0 con 0x7fa8200d35b0 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.330+0000 7fa80effd640 1 -- 172.21.15.16:0/1694167034 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa814005470 con 0x7fa8200d35b0 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.330+0000 7fa824f5c640 1 -- 172.21.15.16:0/1694167034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82015fc70 con 0x7fa8200d35b0 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.330+0000 7fa80effd640 1 -- 172.21.15.16:0/1694167034 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814007260 con 0x7fa8200d35b0 2024-04-25T10:12:32.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.330+0000 7fa824f5c640 1 -- 172.21.15.16:0/1694167034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8200ccfd0 con 0x7fa8200d35b0 2024-04-25T10:12:32.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.331+0000 7fa824f5c640 1 -- 172.21.15.16:0/1694167034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8200749b0 con 0x7fa8200d35b0 2024-04-25T10:12:32.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.331+0000 7fa80effd640 1 -- 172.21.15.16:0/1694167034 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa814007400 con 0x7fa8200d35b0 2024-04-25T10:12:32.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.331+0000 7fa80effd640 1 --2- 172.21.15.16:0/1694167034 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa80006b4d0 0x7fa80006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.332+0000 7fa80effd640 1 -- 172.21.15.16:0/1694167034 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fa8140897c0 con 0x7fa8200d35b0 2024-04-25T10:12:32.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.332+0000 7fa81e59c640 1 --2- 172.21.15.16:0/1694167034 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa80006b4d0 0x7fa80006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.332+0000 7fa81e59c640 1 --2- 172.21.15.16:0/1694167034 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa80006b4d0 0x7fa80006d990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa8100011e0 tx=0x7fa810009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.336+0000 7fa80effd640 1 -- 172.21.15.16:0/1694167034 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa82015fc70 con 0x7fa8200d35b0 2024-04-25T10:12:32.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.552+0000 7fa824f5c640 1 -- 172.21.15.16:0/1694167034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8200b89e0 con 0x7fa8200d35b0 2024-04-25T10:12:32.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.553+0000 7fa80effd640 1 -- 172.21.15.16:0/1694167034 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fa814058590 con 0x7fa8200d35b0 2024-04-25T10:12:32.558 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:32.558 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4044509901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4044509901},{"type":"v1","addr":"172.21.15.16:6835","nonce":4044509901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1494230493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1494230493},{"type":"v1","addr":"172.21.15.186:6837","nonce":1494230493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/1050183001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1050183001},{"type":"v1","addr":"172.21.15.186:6836","nonce":1050183001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:00.015075+0000","modified":"2024-04-25T10:11:58.248543+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7323},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7323":{"gid":7323,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":19,"addr":"172.21.15.16:6834/84349474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":84349474},{"type":"v1","addr":"172.21.15.16:6834","nonce":84349474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:12:32.558 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 64 2024-04-25T10:12:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.555+0000 7fa824f5c640 1 -- 172.21.15.16:0/1694167034 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa80006b4d0 msgr2=0x7fa80006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.555+0000 7fa824f5c640 1 --2- 172.21.15.16:0/1694167034 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa80006b4d0 0x7fa80006d990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa8100011e0 tx=0x7fa810009040 comp rx=0 tx=0).stop 2024-04-25T10:12:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.555+0000 7fa824f5c640 1 -- 172.21.15.16:0/1694167034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8200d35b0 msgr2=0x7fa8200cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.555+0000 7fa824f5c640 1 --2- 172.21.15.16:0/1694167034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8200d35b0 0x7fa8200cc010 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fa81400c440 tx=0x7fa814001e70 comp rx=0 tx=0).stop 2024-04-25T10:12:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.555+0000 7fa824f5c640 1 -- 172.21.15.16:0/1694167034 shutdown_connections 2024-04-25T10:12:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.555+0000 7fa824f5c640 1 --2- 172.21.15.16:0/1694167034 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa80006b4d0 0x7fa80006d990 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.555+0000 7fa824f5c640 1 --2- 172.21.15.16:0/1694167034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa82015dce0 0x7fa82015b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.555+0000 7fa824f5c640 1 --2- 172.21.15.16:0/1694167034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8200d35b0 0x7fa8200cc010 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.555+0000 7fa824f5c640 1 --2- 172.21.15.16:0/1694167034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8200b5b60 0x7fa82014de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.555+0000 7fa824f5c640 1 -- 172.21.15.16:0/1694167034 >> 172.21.15.16:0/1694167034 conn(0x7fa820072900 msgr2=0x7fa8200ce250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.555+0000 7fa824f5c640 1 -- 172.21.15.16:0/1694167034 shutdown_connections 2024-04-25T10:12:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.555+0000 7fa824f5c640 1 -- 172.21.15.16:0/1694167034 wait complete. 2024-04-25T10:12:32.570 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:12:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.668+0000 7f75fb73b640 1 Processor -- start 2024-04-25T10:12:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.668+0000 7f75fb73b640 1 -- start start 2024-04-25T10:12:32.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.669+0000 7f75fb73b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f75f4144e70 0x7f75f406d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.669+0000 7f75fb73b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75f40cf0e0 0x7f75f40cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.669+0000 7f75fb73b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f406dcb0 con 0x7f75f406d7a0 2024-04-25T10:12:32.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.669+0000 7f75fb73b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f415bfe0 con 0x7f75f4144e70 2024-04-25T10:12:32.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.669+0000 7f75fb73b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f406df90 con 0x7f75f40cf0e0 2024-04-25T10:12:32.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.669+0000 7f75f94d7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f75f4144e70 0x7f75f406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.669+0000 7f75f94d7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f75f4144e70 0x7f75f406d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48884/0 (socket says 172.21.15.16:48884) 2024-04-25T10:12:32.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.669+0000 7f75f94d7640 1 -- 172.21.15.16:0/2077286389 learned_addr learned my addr 172.21.15.16:0/2077286389 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:32.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.669+0000 7f75f9cd8640 1 --2- 172.21.15.16:0/2077286389 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75f40cf0e0 0x7f75f40cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.669+0000 7f75f94d7640 1 -- 172.21.15.16:0/2077286389 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75f40cf0e0 msgr2=0x7f75f40cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.669+0000 7f75f94d7640 1 --2- 172.21.15.16:0/2077286389 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75f40cf0e0 0x7f75f40cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.669+0000 7f75f94d7640 1 -- 172.21.15.16:0/2077286389 >> v1:172.21.15.186:6789/0 conn(0x7f75f406d7a0 legacy=0x7f75f40ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.669+0000 7f75f94d7640 1 -- 172.21.15.16:0/2077286389 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f406e270 con 0x7f75f4144e70 2024-04-25T10:12:32.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.670+0000 7f75f94d7640 1 --2- 172.21.15.16:0/2077286389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f4144e70 0x7f75f406d260 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009870 tx=0x7f75f002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531a381e704cee9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.670+0000 7f75ebfff640 1 -- 172.21.15.16:0/2077286389 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0037070 con 0x7f75f4144e70 2024-04-25T10:12:32.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.670+0000 7f75ebfff640 1 -- 172.21.15.16:0/2077286389 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f0032440 con 0x7f75f4144e70 2024-04-25T10:12:32.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.670+0000 7f75ebfff640 1 -- 172.21.15.16:0/2077286389 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0032810 con 0x7f75f4144e70 2024-04-25T10:12:32.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.671+0000 7f75fb73b640 1 -- 172.21.15.16:0/2077286389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f4144e70 msgr2=0x7f75f406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.671+0000 7f75fb73b640 1 --2- 172.21.15.16:0/2077286389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f4144e70 0x7f75f406d260 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009870 tx=0x7f75f002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.671+0000 7f75fb73b640 1 -- 172.21.15.16:0/2077286389 shutdown_connections 2024-04-25T10:12:32.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.671+0000 7f75fb73b640 1 --2- 172.21.15.16:0/2077286389 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75f40cf0e0 0x7f75f40cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.671+0000 7f75fb73b640 1 --2- 172.21.15.16:0/2077286389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f4144e70 0x7f75f406d260 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.671+0000 7f75fb73b640 1 -- 172.21.15.16:0/2077286389 >> 172.21.15.16:0/2077286389 conn(0x7f75f406aba0 msgr2=0x7f75f406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.671+0000 7f75fb73b640 1 -- 172.21.15.16:0/2077286389 shutdown_connections 2024-04-25T10:12:32.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.671+0000 7f75fb73b640 1 -- 172.21.15.16:0/2077286389 wait complete. 2024-04-25T10:12:32.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.672+0000 7f75fb73b640 1 Processor -- start 2024-04-25T10:12:32.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.672+0000 7f75fb73b640 1 -- start start 2024-04-25T10:12:32.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75fb73b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75f406d7a0 0x7f75f413bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75fb73b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f40cf0e0 0x7f75f40b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75fb73b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75f4144e70 0x7f75f40b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75fb73b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f4145630 con 0x7f75f40cf0e0 2024-04-25T10:12:32.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75fb73b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f41454b0 con 0x7f75f406d7a0 2024-04-25T10:12:32.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75fb73b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f40cc840 con 0x7f75f4144e70 2024-04-25T10:12:32.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75f8cd6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f40cf0e0 0x7f75f40b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75f8cd6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f40cf0e0 0x7f75f40b0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48896/0 (socket says 172.21.15.16:48896) 2024-04-25T10:12:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75f8cd6640 1 -- 172.21.15.16:0/1255907031 learned_addr learned my addr 172.21.15.16:0/1255907031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75f94d7640 1 --2- 172.21.15.16:0/1255907031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75f406d7a0 0x7f75f413bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75f9cd8640 1 --2- 172.21.15.16:0/1255907031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75f4144e70 0x7f75f40b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75f8cd6640 1 -- 172.21.15.16:0/1255907031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75f4144e70 msgr2=0x7f75f40b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75f8cd6640 1 --2- 172.21.15.16:0/1255907031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75f4144e70 0x7f75f40b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75f8cd6640 1 -- 172.21.15.16:0/1255907031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75f406d7a0 msgr2=0x7f75f413bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75f8cd6640 1 --2- 172.21.15.16:0/1255907031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75f406d7a0 0x7f75f413bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75f8cd6640 1 -- 172.21.15.16:0/1255907031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f41f2ec0 con 0x7f75f40cf0e0 2024-04-25T10:12:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75f94d7640 1 --2- 172.21.15.16:0/1255907031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75f406d7a0 0x7f75f413bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75f8cd6640 1 --2- 172.21.15.16:0/1255907031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f40cf0e0 0x7f75f40b0e60 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f75e400c430 tx=0x7f75e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75e9ffb640 1 -- 172.21.15.16:0/1255907031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4017070 con 0x7f75f40cf0e0 2024-04-25T10:12:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75e9ffb640 1 -- 172.21.15.16:0/1255907031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e40052d0 con 0x7f75f40cf0e0 2024-04-25T10:12:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75fb73b640 1 -- 172.21.15.16:0/1255907031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f41f31a0 con 0x7f75f40cf0e0 2024-04-25T10:12:32.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.673+0000 7f75e9ffb640 1 -- 172.21.15.16:0/1255907031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4003850 con 0x7f75f40cf0e0 2024-04-25T10:12:32.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.674+0000 7f75fb73b640 1 -- 172.21.15.16:0/1255907031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f41f3a40 con 0x7f75f40cf0e0 2024-04-25T10:12:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.675+0000 7f75e9ffb640 1 -- 172.21.15.16:0/1255907031 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75e401a070 con 0x7f75f40cf0e0 2024-04-25T10:12:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.675+0000 7f75e9ffb640 1 --2- 172.21.15.16:0/1255907031 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f75d806b4d0 0x7f75d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.676+0000 7f75e9ffb640 1 -- 172.21.15.16:0/1255907031 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f75e40896a0 con 0x7f75f40cf0e0 2024-04-25T10:12:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.676+0000 7f75f94d7640 1 --2- 172.21.15.16:0/1255907031 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f75d806b4d0 0x7f75d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.676+0000 7f75f94d7640 1 --2- 172.21.15.16:0/1255907031 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f75d806b4d0 0x7f75d806d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f75f0034000 tx=0x7f75f002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.676+0000 7f75fe440740 1 Processor -- start 2024-04-25T10:12:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.677+0000 7f75fe440740 1 -- start start 2024-04-25T10:12:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.677+0000 7f75fe440740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 0x5587bb8776f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.677+0000 7f75fe440740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8b1c60 0x5587bb875970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.677+0000 7f75fe440740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb8a0050 0x5587bb8a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.677+0000 7f75fe440740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587bb7d8c60 con 0x5587bb875590 2024-04-25T10:12:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.677+0000 7f75fe440740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587bb777d80 con 0x5587bb8a0050 2024-04-25T10:12:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.677+0000 7f75fe440740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587bb7d4d00 con 0x5587bb8b1c60 2024-04-25T10:12:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.677+0000 7f75f94d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 0x5587bb8776f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.677+0000 7f75f94d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 0x5587bb8776f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48908/0 (socket says 172.21.15.16:48908) 2024-04-25T10:12:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.677+0000 7f75f94d7640 1 -- 172.21.15.16:0/1349645837 learned_addr learned my addr 172.21.15.16:0/1349645837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.678+0000 7f75f8cd6640 1 --2- 172.21.15.16:0/1349645837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8b1c60 0x5587bb875970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.678+0000 7f75f9cd8640 1 --2- 172.21.15.16:0/1349645837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb8a0050 0x5587bb8a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.678+0000 7f75f94d7640 1 -- 172.21.15.16:0/1349645837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8b1c60 msgr2=0x5587bb875970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.678+0000 7f75f94d7640 1 --2- 172.21.15.16:0/1349645837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8b1c60 0x5587bb875970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.678+0000 7f75f94d7640 1 -- 172.21.15.16:0/1349645837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb8a0050 msgr2=0x5587bb8a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.678+0000 7f75f94d7640 1 --2- 172.21.15.16:0/1349645837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb8a0050 0x5587bb8a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.678+0000 7f75f94d7640 1 -- 172.21.15.16:0/1349645837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587bb5b97c0 con 0x5587bb875590 2024-04-25T10:12:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.678+0000 7f75f8cd6640 1 --2- 172.21.15.16:0/1349645837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8b1c60 0x5587bb875970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.678+0000 7f75f9cd8640 1 --2- 172.21.15.16:0/1349645837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb8a0050 0x5587bb8a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.678+0000 7f75f94d7640 1 --2- 172.21.15.16:0/1349645837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 0x5587bb8776f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f75f002a7b0 tx=0x7f75f002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d07206d69df9e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.678+0000 7f75deffd640 1 -- 172.21.15.16:0/1349645837 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f004e070 con 0x5587bb875590 2024-04-25T10:12:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.678+0000 7f75deffd640 1 -- 172.21.15.16:0/1349645837 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f003e460 con 0x5587bb875590 2024-04-25T10:12:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.678+0000 7f75deffd640 1 -- 172.21.15.16:0/1349645837 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0037040 con 0x5587bb875590 2024-04-25T10:12:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.678+0000 7f75fe440740 1 -- 172.21.15.16:0/1349645837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 msgr2=0x5587bb8776f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.678+0000 7f75fe440740 1 --2- 172.21.15.16:0/1349645837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 0x5587bb8776f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f75f002a7b0 tx=0x7f75f002c000 comp rx=0 tx=0).stop 2024-04-25T10:12:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.679+0000 7f75fe440740 1 -- 172.21.15.16:0/1349645837 shutdown_connections 2024-04-25T10:12:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.679+0000 7f75fe440740 1 --2- 172.21.15.16:0/1349645837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb8a0050 0x5587bb8a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.679+0000 7f75fe440740 1 --2- 172.21.15.16:0/1349645837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8b1c60 0x5587bb875970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.679+0000 7f75fe440740 1 --2- 172.21.15.16:0/1349645837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 0x5587bb8776f0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.679+0000 7f75fe440740 1 -- 172.21.15.16:0/1349645837 >> 172.21.15.16:0/1349645837 conn(0x5587bb83d6e0 msgr2=0x5587bb8bb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.679+0000 7f75fe440740 1 -- 172.21.15.16:0/1349645837 shutdown_connections 2024-04-25T10:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.679+0000 7f75fe440740 1 -- 172.21.15.16:0/1349645837 wait complete. 2024-04-25T10:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.679+0000 7f75fe440740 1 -- start start 2024-04-25T10:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.679+0000 7f75fe440740 1 Processor -- start 2024-04-25T10:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.680+0000 7f75fe440740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 0x5587bb8a3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.680+0000 7f75fe440740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb8a0050 0x5587bb891770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.680+0000 7f75fe440740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8b1c60 0x5587bb8cada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.680+0000 7f75fe440740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587bb775840 con 0x5587bb875590 2024-04-25T10:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.680+0000 7f75fe440740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587bb89dba0 con 0x5587bb8a0050 2024-04-25T10:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.680+0000 7f75fe440740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587bb8cc040 con 0x5587bb8b1c60 2024-04-25T10:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.680+0000 7f75fe440740 10 client.0 _ll_register_callbacks cb 0x5587bb78f980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.680+0000 7f75f94d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 0x5587bb8a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.680+0000 7f75f94d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 0x5587bb8a3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48920/0 (socket says 172.21.15.16:48920) 2024-04-25T10:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.680+0000 7f75f94d7640 1 -- 172.21.15.16:0/2156845937 learned_addr learned my addr 172.21.15.16:0/2156845937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.680+0000 7f75f9cd8640 1 --2- 172.21.15.16:0/2156845937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8b1c60 0x5587bb8cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.680+0000 7f75f8cd6640 1 --2- 172.21.15.16:0/2156845937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb8a0050 0x5587bb891770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.680+0000 7f75f94d7640 1 -- 172.21.15.16:0/2156845937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8b1c60 msgr2=0x5587bb8cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.680+0000 7f75f94d7640 1 --2- 172.21.15.16:0/2156845937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8b1c60 0x5587bb8cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.680+0000 7f75f94d7640 1 -- 172.21.15.16:0/2156845937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb8a0050 msgr2=0x5587bb891770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.680+0000 7f75f94d7640 1 --2- 172.21.15.16:0/2156845937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb8a0050 0x5587bb891770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.680+0000 7f75f94d7640 1 -- 172.21.15.16:0/2156845937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587bb619f50 con 0x5587bb875590 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.681+0000 7f75f8cd6640 1 --2- 172.21.15.16:0/2156845937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb8a0050 0x5587bb891770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.681+0000 7f75f9cd8640 1 --2- 172.21.15.16:0/2156845937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8b1c60 0x5587bb8cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.681+0000 7f75f94d7640 1 --2- 172.21.15.16:0/2156845937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 0x5587bb8a3270 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f75f002a7b0 tx=0x7f75f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb886ab78420872 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.681+0000 7f75d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.681+0000 7f75d3fff640 1 -- 172.21.15.16:0/2156845937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0057060 con 0x5587bb875590 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.681+0000 7f75d3fff640 1 -- 172.21.15.16:0/2156845937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f75f0044040 con 0x5587bb875590 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.681+0000 7f75fe440740 1 -- 172.21.15.16:0/2156845937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75f003e460 con 0x5587bb875590 2024-04-25T10:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.681+0000 7f75d3fff640 1 -- 172.21.15.16:0/2156845937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75f005d050 con 0x5587bb875590 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.681+0000 7f75fe440740 10 client.8004 fetch_fsmap learned FSMap version 64 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.681+0000 7f75fe440740 1 -- 172.21.15.16:0/2156845937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x5587bb4ad0b0 con 0x5587bb875590 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.681+0000 7f75d3fff640 1 -- 172.21.15.16:0/2156845937 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f003e460 con 0x5587bb875590 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.681+0000 7f75d3fff640 1 -- 172.21.15.16:0/2156845937 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f75f0060020 con 0x5587bb875590 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.681+0000 7f75fe440740 10 client.8004 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.681+0000 7f75fe440740 10 client.8004 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.681+0000 7f75fe440740 20 client.8004 populate_metadata read hostname 'smithi016' 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.681+0000 7f75fe440740 1 --2- 172.21.15.16:0/2156845937 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x5587bb8d64d0 0x5587bb8d88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.681+0000 7f75fe440740 4 client.8004 mds_command: new command op to 7323 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.681+0000 7f75fe440740 1 -- 172.21.15.16:0/2156845937 --> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587bb798e60 con 0x5587bb8d64d0 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.682+0000 7f75f8cd6640 1 --2- 172.21.15.16:0/2156845937 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x5587bb8d64d0 0x5587bb8d88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.682+0000 7f75d3fff640 10 client.8004 ms_handle_connect on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.682+0000 7f75f8cd6640 1 --2- 172.21.15.16:0/2156845937 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x5587bb8d64d0 0x5587bb8d88c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6adc7e7fb8798536 server_cookie=30a920de9cb61107 in_seq=0 out_seq=0 2024-04-25T10:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.682+0000 7f75d3fff640 1 -- 172.21.15.16:0/2156845937 <== mds.0 v2:172.21.15.16:6832/84349474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5587bb798e60 con 0x5587bb8d64d0 2024-04-25T10:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.682+0000 7f75d3fff640 10 client.8004 handle_command_reply: tid=0 2024-04-25T10:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.682+0000 7f75fe440740 1 client.8004 shutdown 2024-04-25T10:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.682+0000 7f75fe440740 1 -- 172.21.15.16:0/2156845937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 msgr2=0x5587bb8a3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.682+0000 7f75fe440740 1 --2- 172.21.15.16:0/2156845937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 0x5587bb8a3270 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f75f002a7b0 tx=0x7f75f0004730 comp rx=0 tx=0).stop 2024-04-25T10:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.683+0000 7f75fe440740 1 -- 172.21.15.16:0/2156845937 shutdown_connections 2024-04-25T10:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.683+0000 7f75fe440740 1 --2- 172.21.15.16:0/2156845937 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x5587bb8d64d0 0x5587bb8d88c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.683+0000 7f75fe440740 1 --2- 172.21.15.16:0/2156845937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8b1c60 0x5587bb8cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.683+0000 7f75fe440740 1 --2- 172.21.15.16:0/2156845937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb8a0050 0x5587bb891770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.683+0000 7f75fe440740 1 --2- 172.21.15.16:0/2156845937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 0x5587bb8a3270 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.683+0000 7f75d3fff640 0 client.8004 ms_handle_reset on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.683+0000 7f75fe440740 1 -- 172.21.15.16:0/2156845937 >> 172.21.15.16:0/2156845937 conn(0x5587bb83d6e0 msgr2=0x5587bb89d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.683+0000 7f75fe440740 1 -- 172.21.15.16:0/2156845937 shutdown_connections 2024-04-25T10:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.683+0000 7f75fe440740 1 -- 172.21.15.16:0/2156845937 wait complete. 2024-04-25T10:12:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.683+0000 7f75fe440740 20 client.8004 trim_cache size 0 max 16384 2024-04-25T10:12:32.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.695+0000 7f75fe440740 1 Processor -- start 2024-04-25T10:12:32.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.695+0000 7f75fe440740 1 -- start start 2024-04-25T10:12:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.695+0000 7f75fe440740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 0x5587bb8ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.695+0000 7f75fe440740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8a0050 0x5587bb890a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.695+0000 7f75fe440740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb8b1c60 0x5587bb8a2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.695+0000 7f75fe440740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f41fd050 con 0x5587bb875590 2024-04-25T10:12:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.695+0000 7f75fe440740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f41e4ec0 con 0x5587bb8b1c60 2024-04-25T10:12:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.695+0000 7f75fe440740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f41fcaa0 con 0x5587bb8a0050 2024-04-25T10:12:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.695+0000 7f75f94d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 0x5587bb8ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.695+0000 7f75f94d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 0x5587bb8ca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48932/0 (socket says 172.21.15.16:48932) 2024-04-25T10:12:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.695+0000 7f75f94d7640 1 -- 172.21.15.16:0/4000109379 learned_addr learned my addr 172.21.15.16:0/4000109379 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.696+0000 7f75f94d7640 1 -- 172.21.15.16:0/4000109379 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8a0050 msgr2=0x5587bb890a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.696+0000 7f75f94d7640 1 --2- 172.21.15.16:0/4000109379 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8a0050 0x5587bb890a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.696+0000 7f75f94d7640 1 -- 172.21.15.16:0/4000109379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb8b1c60 msgr2=0x5587bb8a2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.696+0000 7f75f94d7640 1 --2- 172.21.15.16:0/4000109379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb8b1c60 0x5587bb8a2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.696+0000 7f75f94d7640 1 -- 172.21.15.16:0/4000109379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587bb8729e0 con 0x5587bb875590 2024-04-25T10:12:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.696+0000 7f75f94d7640 1 --2- 172.21.15.16:0/4000109379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 0x5587bb8ca880 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009730 tx=0x7f75f0037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5e98e42f989423 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.696+0000 7f75de7fc640 1 -- 172.21.15.16:0/4000109379 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f004e030 con 0x5587bb875590 2024-04-25T10:12:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.696+0000 7f75de7fc640 1 -- 172.21.15.16:0/4000109379 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f003e460 con 0x5587bb875590 2024-04-25T10:12:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.696+0000 7f75de7fc640 1 -- 172.21.15.16:0/4000109379 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0037070 con 0x5587bb875590 2024-04-25T10:12:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.696+0000 7f75fe440740 1 -- 172.21.15.16:0/4000109379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 msgr2=0x5587bb8ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.696+0000 7f75fe440740 1 --2- 172.21.15.16:0/4000109379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 0x5587bb8ca880 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009730 tx=0x7f75f0037000 comp rx=0 tx=0).stop 2024-04-25T10:12:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.696+0000 7f75fe440740 1 -- 172.21.15.16:0/4000109379 shutdown_connections 2024-04-25T10:12:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.696+0000 7f75fe440740 1 --2- 172.21.15.16:0/4000109379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb8b1c60 0x5587bb8a2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.696+0000 7f75fe440740 1 --2- 172.21.15.16:0/4000109379 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8a0050 0x5587bb890a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.696+0000 7f75fe440740 1 --2- 172.21.15.16:0/4000109379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb875590 0x5587bb8ca880 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.696+0000 7f75fe440740 1 -- 172.21.15.16:0/4000109379 >> 172.21.15.16:0/4000109379 conn(0x5587bb83d6e0 msgr2=0x5587bb86f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.696+0000 7f75fe440740 1 -- 172.21.15.16:0/4000109379 shutdown_connections 2024-04-25T10:12:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.696+0000 7f75fe440740 1 -- 172.21.15.16:0/4000109379 wait complete. 2024-04-25T10:12:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.697+0000 7f75fe440740 1 -- start start 2024-04-25T10:12:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.697+0000 7f75fe440740 1 Processor -- start 2024-04-25T10:12:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.698+0000 7f75fe440740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb875590 0x5587bb8b6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.698+0000 7f75fe440740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8a0050 0x5587bb8b6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.698+0000 7f75fe440740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb8b1c60 0x5587bb8d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.698+0000 7f75fe440740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587bb8a1460 con 0x5587bb8b1c60 2024-04-25T10:12:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.698+0000 7f75fe440740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587bb8a25c0 con 0x5587bb875590 2024-04-25T10:12:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.698+0000 7f75fe440740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587bb8cb9a0 con 0x5587bb8a0050 2024-04-25T10:12:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.698+0000 7f75fe440740 10 client.0 _ll_register_callbacks cb 0x7f75f0035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.698+0000 7f75f9cd8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb8b1c60 0x5587bb8d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.698+0000 7f75f9cd8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb8b1c60 0x5587bb8d42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60866/0 (socket says 172.21.15.16:60866) 2024-04-25T10:12:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.698+0000 7f75f9cd8640 1 -- 172.21.15.16:0/1863700800 learned_addr learned my addr 172.21.15.16:0/1863700800 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.698+0000 7f75f94d7640 1 --2- 172.21.15.16:0/1863700800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb875590 0x5587bb8b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.698+0000 7f75f8cd6640 1 --2- 172.21.15.16:0/1863700800 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8a0050 0x5587bb8b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.698+0000 7f75f9cd8640 1 -- 172.21.15.16:0/1863700800 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8a0050 msgr2=0x5587bb8b6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.698+0000 7f75f9cd8640 1 --2- 172.21.15.16:0/1863700800 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8a0050 0x5587bb8b6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.698+0000 7f75f9cd8640 1 -- 172.21.15.16:0/1863700800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb875590 msgr2=0x5587bb8b6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.698+0000 7f75f9cd8640 1 --2- 172.21.15.16:0/1863700800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb875590 0x5587bb8b6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.698+0000 7f75f9cd8640 1 -- 172.21.15.16:0/1863700800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587bb88f860 con 0x5587bb8b1c60 2024-04-25T10:12:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.698+0000 7f75f8cd6640 1 --2- 172.21.15.16:0/1863700800 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8a0050 0x5587bb8b6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.699+0000 7f75f9cd8640 1 --2- 172.21.15.16:0/1863700800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb8b1c60 0x5587bb8d42a0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f75ec007560 tx=0x7f75ec007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25510b69c244f292 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.699+0000 7f75d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:12:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.699+0000 7f75d3fff640 1 -- 172.21.15.16:0/1863700800 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec01c020 con 0x5587bb8b1c60 2024-04-25T10:12:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.699+0000 7f75d3fff640 1 -- 172.21.15.16:0/1863700800 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f75ec017870 con 0x5587bb8b1c60 2024-04-25T10:12:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.699+0000 7f75fe440740 1 -- 172.21.15.16:0/1863700800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75f003e460 con 0x5587bb8b1c60 2024-04-25T10:12:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.699+0000 7f75d3fff640 1 -- 172.21.15.16:0/1863700800 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75ec026050 con 0x5587bb8b1c60 2024-04-25T10:12:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.699+0000 7f75fe440740 10 client.8010 fetch_fsmap learned FSMap version 64 2024-04-25T10:12:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.699+0000 7f75fe440740 1 -- 172.21.15.16:0/1863700800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x5587bb8dcec0 con 0x5587bb8b1c60 2024-04-25T10:12:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.699+0000 7f75d3fff640 1 -- 172.21.15.16:0/1863700800 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f003e460 con 0x5587bb8b1c60 2024-04-25T10:12:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.699+0000 7f75d3fff640 1 -- 172.21.15.16:0/1863700800 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f75ec029020 con 0x5587bb8b1c60 2024-04-25T10:12:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.699+0000 7f75fe440740 10 client.8010 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T10:12:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.699+0000 7f75fe440740 10 client.8010 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:12:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.699+0000 7f75fe440740 20 client.8010 populate_metadata read hostname 'smithi016' 2024-04-25T10:12:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.699+0000 7f75fe440740 1 --2- 172.21.15.16:0/1863700800 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x5587bb8d4de0 0x5587bb89e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.699+0000 7f75fe440740 4 client.8010 mds_command: new command op to 7323 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:12:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.699+0000 7f75fe440740 1 -- 172.21.15.16:0/1863700800 --> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5587bb7e61a0 con 0x5587bb8d4de0 2024-04-25T10:12:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.699+0000 7f75f94d7640 1 --2- 172.21.15.16:0/1863700800 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x5587bb8d4de0 0x5587bb89e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.700+0000 7f75d3fff640 10 client.8010 ms_handle_connect on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.700+0000 7f75f94d7640 1 --2- 172.21.15.16:0/1863700800 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x5587bb8d4de0 0x5587bb89e920 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1847e7625030a488 server_cookie=95fc33996950dc43 in_seq=0 out_seq=0 2024-04-25T10:12:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.700+0000 7f75d3fff640 1 -- 172.21.15.16:0/1863700800 <== mds.0 v2:172.21.15.16:6832/84349474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5587bb7e61a0 con 0x5587bb8d4de0 2024-04-25T10:12:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.700+0000 7f75d3fff640 10 client.8010 handle_command_reply: tid=0 2024-04-25T10:12:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.700+0000 7f75fe440740 1 client.8010 shutdown 2024-04-25T10:12:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.700+0000 7f75fe440740 1 -- 172.21.15.16:0/1863700800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb8b1c60 msgr2=0x5587bb8d42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.700+0000 7f75fe440740 1 --2- 172.21.15.16:0/1863700800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb8b1c60 0x5587bb8d42a0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f75ec007560 tx=0x7f75ec007a20 comp rx=0 tx=0).stop 2024-04-25T10:12:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.700+0000 7f75fe440740 1 -- 172.21.15.16:0/1863700800 shutdown_connections 2024-04-25T10:12:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.700+0000 7f75fe440740 1 --2- 172.21.15.16:0/1863700800 >> [v2:172.21.15.16:6832/84349474,v1:172.21.15.16:6834/84349474] conn(0x5587bb8d4de0 0x5587bb89e920 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.700+0000 7f75fe440740 1 --2- 172.21.15.16:0/1863700800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587bb8b1c60 0x5587bb8d42a0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.700+0000 7f75fe440740 1 --2- 172.21.15.16:0/1863700800 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5587bb8a0050 0x5587bb8b6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.700+0000 7f75d3fff640 0 client.8010 ms_handle_reset on v2:172.21.15.16:6832/84349474 2024-04-25T10:12:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.700+0000 7f75fe440740 1 --2- 172.21.15.16:0/1863700800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587bb875590 0x5587bb8b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.700+0000 7f75fe440740 1 -- 172.21.15.16:0/1863700800 >> 172.21.15.16:0/1863700800 conn(0x5587bb83d6e0 msgr2=0x5587bb8bc180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.700+0000 7f75fe440740 1 -- 172.21.15.16:0/1863700800 shutdown_connections 2024-04-25T10:12:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.700+0000 7f75fe440740 1 -- 172.21.15.16:0/1863700800 wait complete. 2024-04-25T10:12:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.700+0000 7f75fe440740 20 client.8010 trim_cache size 0 max 16384 2024-04-25T10:12:32.713 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:32.713 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-04-25T10:12:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.701+0000 7f75fb73b640 1 -- 172.21.15.16:0/1255907031 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f75d806b4d0 msgr2=0x7f75d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.701+0000 7f75fb73b640 1 --2- 172.21.15.16:0/1255907031 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f75d806b4d0 0x7f75d806d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f75f0034000 tx=0x7f75f002e000 comp rx=0 tx=0).stop 2024-04-25T10:12:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.701+0000 7f75fb73b640 1 -- 172.21.15.16:0/1255907031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f40cf0e0 msgr2=0x7f75f40b0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.701+0000 7f75fb73b640 1 --2- 172.21.15.16:0/1255907031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f40cf0e0 0x7f75f40b0e60 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f75e400c430 tx=0x7f75e400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.701+0000 7f75fb73b640 1 -- 172.21.15.16:0/1255907031 shutdown_connections 2024-04-25T10:12:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.701+0000 7f75fb73b640 1 --2- 172.21.15.16:0/1255907031 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f75d806b4d0 0x7f75d806d990 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.701+0000 7f75fb73b640 1 --2- 172.21.15.16:0/1255907031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75f4144e70 0x7f75f40b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.701+0000 7f75fb73b640 1 --2- 172.21.15.16:0/1255907031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75f40cf0e0 0x7f75f40b0e60 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.701+0000 7f75fb73b640 1 --2- 172.21.15.16:0/1255907031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75f406d7a0 0x7f75f413bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.701+0000 7f75fb73b640 1 -- 172.21.15.16:0/1255907031 >> 172.21.15.16:0/1255907031 conn(0x7f75f406aba0 msgr2=0x7f75f415c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.701+0000 7f75fb73b640 1 -- 172.21.15.16:0/1255907031 shutdown_connections 2024-04-25T10:12:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.701+0000 7f75fb73b640 1 -- 172.21.15.16:0/1255907031 wait complete. 2024-04-25T10:12:32.716 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:12:32.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.811+0000 7f7746658640 1 Processor -- start 2024-04-25T10:12:32.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.811+0000 7f7746658640 1 -- start start 2024-04-25T10:12:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.811+0000 7f7746658640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f77400b5b60 0x7f77401687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.811+0000 7f7746658640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f774015e0c0 con 0x7f77400d35b0 2024-04-25T10:12:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.811+0000 7f7746658640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77400d2510 con 0x7f774015dce0 2024-04-25T10:12:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.811+0000 7f7746658640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77400b7b10 con 0x7f77400b5b60 2024-04-25T10:12:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.812+0000 7f773f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f77400d35b0 0x7f77400b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49118/0 (socket says 172.21.15.16:49118) 2024-04-25T10:12:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.812+0000 7f773f7fe640 1 -- 172.21.15.16:0/1901318405 learned_addr learned my addr 172.21.15.16:0/1901318405 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.812+0000 7f7744bf5640 1 --2- 172.21.15.16:0/1901318405 >> v2:172.21.15.186:3300/0 conn(0x7f77400b5b60 0x7f77401687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.812+0000 7f773effd640 1 -- 172.21.15.16:0/1901318405 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3971352660 0 0) 0x7f774015e0c0 con 0x7f77400d35b0 2024-04-25T10:12:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.812+0000 7f773effd640 1 -- 172.21.15.16:0/1901318405 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7728003520 con 0x7f77400d35b0 2024-04-25T10:12:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.812+0000 7f7744bf5640 1 -- 172.21.15.16:0/1901318405 >> v1:172.21.15.186:6789/0 conn(0x7f774015dce0 legacy=0x7f77400d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.812+0000 7f7744bf5640 1 -- 172.21.15.16:0/1901318405 >> v1:172.21.15.16:6789/0 conn(0x7f77400d35b0 legacy=0x7f77400b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.812+0000 7f773effd640 1 -- 172.21.15.16:0/1901318405 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880211177 0 0) 0x7f77400d2510 con 0x7f774015dce0 2024-04-25T10:12:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.812+0000 7f7744bf5640 1 -- 172.21.15.16:0/1901318405 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77400d27f0 con 0x7f77400b5b60 2024-04-25T10:12:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.813+0000 7f7744bf5640 1 --2- 172.21.15.16:0/1901318405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77400b5b60 0x7f77401687f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7738009d90 tx=0x7f773802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f7afa95a7c8aa4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.813+0000 7f773effd640 1 -- 172.21.15.16:0/1901318405 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738007740 con 0x7f77400b5b60 2024-04-25T10:12:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.813+0000 7f773effd640 1 -- 172.21.15.16:0/1901318405 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77380078e0 con 0x7f77400b5b60 2024-04-25T10:12:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.813+0000 7f773effd640 1 -- 172.21.15.16:0/1901318405 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738007cb0 con 0x7f77400b5b60 2024-04-25T10:12:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.813+0000 7f7746658640 1 -- 172.21.15.16:0/1901318405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77400b5b60 msgr2=0x7f77401687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.813+0000 7f7746658640 1 --2- 172.21.15.16:0/1901318405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77400b5b60 0x7f77401687f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7738009d90 tx=0x7f773802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.814+0000 7f7746658640 1 -- 172.21.15.16:0/1901318405 shutdown_connections 2024-04-25T10:12:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.814+0000 7f7746658640 1 --2- 172.21.15.16:0/1901318405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77400b5b60 0x7f77401687f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.814+0000 7f7746658640 1 -- 172.21.15.16:0/1901318405 >> 172.21.15.16:0/1901318405 conn(0x7f7740072900 msgr2=0x7f77400d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.814+0000 7f7746658640 1 -- 172.21.15.16:0/1901318405 shutdown_connections 2024-04-25T10:12:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.814+0000 7f7746658640 1 -- 172.21.15.16:0/1901318405 wait complete. 2024-04-25T10:12:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.815+0000 7f7746658640 1 Processor -- start 2024-04-25T10:12:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.815+0000 7f7746658640 1 -- start start 2024-04-25T10:12:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f7746658640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77400b5b60 0x7f7740143840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f7746658640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77400d35b0 0x7f77401537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f7746658640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f774015dce0 0x7f7740153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f7746658640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77400d3090 con 0x7f77400b5b60 2024-04-25T10:12:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f7746658640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77400d2db0 con 0x7f77400d35b0 2024-04-25T10:12:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f7746658640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77400d2160 con 0x7f774015dce0 2024-04-25T10:12:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f773ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77400b5b60 0x7f7740143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f773ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77400b5b60 0x7f7740143840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60868/0 (socket says 172.21.15.16:60868) 2024-04-25T10:12:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f773ffff640 1 -- 172.21.15.16:0/3208686393 learned_addr learned my addr 172.21.15.16:0/3208686393 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f7744bf5640 1 --2- 172.21.15.16:0/3208686393 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f774015dce0 0x7f7740153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f773f7fe640 1 --2- 172.21.15.16:0/3208686393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77400d35b0 0x7f77401537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f773ffff640 1 -- 172.21.15.16:0/3208686393 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f774015dce0 msgr2=0x7f7740153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f773ffff640 1 --2- 172.21.15.16:0/3208686393 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f774015dce0 0x7f7740153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f773ffff640 1 -- 172.21.15.16:0/3208686393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77400d35b0 msgr2=0x7f77401537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f773ffff640 1 --2- 172.21.15.16:0/3208686393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77400d35b0 0x7f77401537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f773ffff640 1 -- 172.21.15.16:0/3208686393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7740143d80 con 0x7f77400b5b60 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f773ffff640 1 --2- 172.21.15.16:0/3208686393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77400b5b60 0x7f7740143840 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f773000c410 tx=0x7f773000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f773cff9640 1 -- 172.21.15.16:0/3208686393 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7730017070 con 0x7f77400b5b60 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f773cff9640 1 -- 172.21.15.16:0/3208686393 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77300035f0 con 0x7f77400b5b60 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f7746658640 1 -- 172.21.15.16:0/3208686393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7740156940 con 0x7f77400b5b60 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f773cff9640 1 -- 172.21.15.16:0/3208686393 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77300039e0 con 0x7f77400b5b60 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.816+0000 7f7746658640 1 -- 172.21.15.16:0/3208686393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77400cc020 con 0x7f77400b5b60 2024-04-25T10:12:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.817+0000 7f7746658640 1 -- 172.21.15.16:0/3208686393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77400749b0 con 0x7f77400b5b60 2024-04-25T10:12:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.818+0000 7f773cff9640 1 -- 172.21.15.16:0/3208686393 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f773001a070 con 0x7f77400b5b60 2024-04-25T10:12:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.818+0000 7f773cff9640 1 --2- 172.21.15.16:0/3208686393 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f772c06b500 0x7f772c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.818+0000 7f773cff9640 1 -- 172.21.15.16:0/3208686393 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f77300896c0 con 0x7f77400b5b60 2024-04-25T10:12:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.819+0000 7f773f7fe640 1 --2- 172.21.15.16:0/3208686393 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f772c06b500 0x7f772c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.819+0000 7f773f7fe640 1 --2- 172.21.15.16:0/3208686393 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f772c06b500 0x7f772c06d9c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7740146990 tx=0x7f7734009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:32.823+0000 7f773cff9640 1 -- 172.21.15.16:0/3208686393 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7730058490 con 0x7f77400b5b60 2024-04-25T10:12:33.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.041+0000 7f7746658640 1 -- 172.21.15.16:0/3208686393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f77400d1f40 con 0x7f77400b5b60 2024-04-25T10:12:33.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.042+0000 7f773cff9640 1 -- 172.21.15.16:0/3208686393 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f7730057d90 con 0x7f77400b5b60 2024-04-25T10:12:33.046 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:33.046 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:12:33.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.043+0000 7f7746658640 1 -- 172.21.15.16:0/3208686393 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f772c06b500 msgr2=0x7f772c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.043+0000 7f7746658640 1 --2- 172.21.15.16:0/3208686393 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f772c06b500 0x7f772c06d9c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7740146990 tx=0x7f7734009040 comp rx=0 tx=0).stop 2024-04-25T10:12:33.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.043+0000 7f7746658640 1 -- 172.21.15.16:0/3208686393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77400b5b60 msgr2=0x7f7740143840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.043+0000 7f7746658640 1 --2- 172.21.15.16:0/3208686393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77400b5b60 0x7f7740143840 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f773000c410 tx=0x7f773000c8d0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.043+0000 7f7746658640 1 -- 172.21.15.16:0/3208686393 shutdown_connections 2024-04-25T10:12:33.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.043+0000 7f7746658640 1 --2- 172.21.15.16:0/3208686393 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f772c06b500 0x7f772c06d9c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.043+0000 7f7746658640 1 --2- 172.21.15.16:0/3208686393 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f774015dce0 0x7f7740153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.043+0000 7f7746658640 1 --2- 172.21.15.16:0/3208686393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77400d35b0 0x7f77401537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.043+0000 7f7746658640 1 --2- 172.21.15.16:0/3208686393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77400b5b60 0x7f7740143840 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.044+0000 7f7746658640 1 -- 172.21.15.16:0/3208686393 >> 172.21.15.16:0/3208686393 conn(0x7f7740072900 msgr2=0x7f77400b84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.044+0000 7f7746658640 1 -- 172.21.15.16:0/3208686393 shutdown_connections 2024-04-25T10:12:33.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.044+0000 7f7746658640 1 -- 172.21.15.16:0/3208686393 wait complete. 2024-04-25T10:12:33.059 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:12:33.059 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:12:33.059 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:33.099 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:33.143 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:12:33.143 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:12:33.143 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:33.159 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:12:33.159 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:33.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:33.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.352+0000 7ffbc0db7640 1 Processor -- start 2024-04-25T10:12:33.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.352+0000 7ffbc0db7640 1 -- start start 2024-04-25T10:12:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.353+0000 7ffbc0db7640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ffbbc0b5b60 0x7ffbbc1687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.353+0000 7ffbc0db7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbbc15e0c0 con 0x7ffbbc15dce0 2024-04-25T10:12:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.353+0000 7ffbc0db7640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbbc0d2510 con 0x7ffbbc0d35b0 2024-04-25T10:12:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.353+0000 7ffbc0db7640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbbc0b7b10 con 0x7ffbbc0b5b60 2024-04-25T10:12:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.353+0000 7ffbba59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffbbc15dce0 0x7ffbbc0d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49128/0 (socket says 172.21.15.16:49128) 2024-04-25T10:12:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.353+0000 7ffbba59c640 1 -- 172.21.15.16:0/3535216858 learned_addr learned my addr 172.21.15.16:0/3535216858 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbb959a640 1 -- 172.21.15.16:0/3535216858 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647051912 0 0) 0x7ffbbc15e0c0 con 0x7ffbbc15dce0 2024-04-25T10:12:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbb959a640 1 -- 172.21.15.16:0/3535216858 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffba8003520 con 0x7ffbbc15dce0 2024-04-25T10:12:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbbad9d640 1 --2- 172.21.15.16:0/3535216858 >> v2:172.21.15.186:3300/0 conn(0x7ffbbc0b5b60 0x7ffbbc1687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbb959a640 1 -- 172.21.15.16:0/3535216858 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2642882476 0 0) 0x7ffba8003520 con 0x7ffbbc15dce0 2024-04-25T10:12:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbb959a640 1 -- 172.21.15.16:0/3535216858 >> v2:172.21.15.186:3300/0 conn(0x7ffbbc0b5b60 msgr2=0x7ffbbc1687f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbb959a640 1 --2- 172.21.15.16:0/3535216858 >> v2:172.21.15.186:3300/0 conn(0x7ffbbc0b5b60 0x7ffbbc1687f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbb959a640 1 -- 172.21.15.16:0/3535216858 >> v1:172.21.15.186:6789/0 conn(0x7ffbbc0d35b0 legacy=0x7ffbbc0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbb959a640 1 -- 172.21.15.16:0/3535216858 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc0d27f0 con 0x7ffbbc15dce0 2024-04-25T10:12:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbb959a640 1 -- 172.21.15.16:0/3535216858 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7ffbb4001670 con 0x7ffbbc15dce0 2024-04-25T10:12:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbb959a640 1 -- 172.21.15.16:0/3535216858 >> v1:172.21.15.16:6789/0 conn(0x7ffbbc15dce0 legacy=0x7ffbbc0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbb959a640 1 --2- 172.21.15.16:0/3535216858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba8003a00 0x7ffba8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbb959a640 1 --2- 172.21.15.16:0/3535216858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffba8004800 0x7ffba8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbb959a640 1 --2- 172.21.15.16:0/3535216858 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba80051c0 0x7ffba800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbb959a640 1 -- 172.21.15.16:0/3535216858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0b7b10 con 0x7ffba8003a00 2024-04-25T10:12:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbb959a640 1 -- 172.21.15.16:0/3535216858 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba800c0c0 con 0x7ffba8004800 2024-04-25T10:12:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbb959a640 1 -- 172.21.15.16:0/3535216858 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba800c2c0 con 0x7ffba80051c0 2024-04-25T10:12:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbba59c640 1 --2- 172.21.15.16:0/3535216858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba8003a00 0x7ffba8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbb9d9b640 1 --2- 172.21.15.16:0/3535216858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffba8004800 0x7ffba8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbbad9d640 1 --2- 172.21.15.16:0/3535216858 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba80051c0 0x7ffba800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbba59c640 1 -- 172.21.15.16:0/3535216858 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba80051c0 msgr2=0x7ffba800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbba59c640 1 --2- 172.21.15.16:0/3535216858 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba80051c0 0x7ffba800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbba59c640 1 -- 172.21.15.16:0/3535216858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffba8004800 msgr2=0x7ffba8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbba59c640 1 --2- 172.21.15.16:0/3535216858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffba8004800 0x7ffba8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbba59c640 1 -- 172.21.15.16:0/3535216858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba8003520 con 0x7ffba8003a00 2024-04-25T10:12:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.354+0000 7ffbb9d9b640 1 --2- 172.21.15.16:0/3535216858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffba8004800 0x7ffba8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.355+0000 7ffbbad9d640 1 -- 172.21.15.16:0/3535216858 reap_dead start 2024-04-25T10:12:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.355+0000 7ffbba59c640 1 --2- 172.21.15.16:0/3535216858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba8003a00 0x7ffba8003e20 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400c110 tx=0x7ffba8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b3c3656ef4242c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.355+0000 7ffbb959a640 1 -- 172.21.15.16:0/3535216858 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4007c40 con 0x7ffba8003a00 2024-04-25T10:12:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.355+0000 7ffbb959a640 1 -- 172.21.15.16:0/3535216858 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb4005af0 con 0x7ffba8003a00 2024-04-25T10:12:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.355+0000 7ffbb959a640 1 -- 172.21.15.16:0/3535216858 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4007070 con 0x7ffba8003a00 2024-04-25T10:12:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.356+0000 7ffbc0db7640 1 -- 172.21.15.16:0/3535216858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba8003a00 msgr2=0x7ffba8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.356+0000 7ffbc0db7640 1 --2- 172.21.15.16:0/3535216858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba8003a00 0x7ffba8003e20 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400c110 tx=0x7ffba8005830 comp rx=0 tx=0).stop 2024-04-25T10:12:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.356+0000 7ffbc0db7640 1 -- 172.21.15.16:0/3535216858 shutdown_connections 2024-04-25T10:12:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.356+0000 7ffbc0db7640 1 --2- 172.21.15.16:0/3535216858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba8003a00 0x7ffba8003e20 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.356+0000 7ffbc0db7640 1 -- 172.21.15.16:0/3535216858 >> 172.21.15.16:0/3535216858 conn(0x7ffbbc072900 msgr2=0x7ffbbc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.356+0000 7ffbc0db7640 1 -- 172.21.15.16:0/3535216858 shutdown_connections 2024-04-25T10:12:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.356+0000 7ffbc0db7640 1 -- 172.21.15.16:0/3535216858 wait complete. 2024-04-25T10:12:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.357+0000 7ffbc0db7640 1 Processor -- start 2024-04-25T10:12:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.357+0000 7ffbc0db7640 1 -- start start 2024-04-25T10:12:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.358+0000 7ffbc0db7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba8003a00 0x7ffbbc153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.358+0000 7ffbc0db7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbbc152ab0 0x7ffbbc145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.358+0000 7ffbc0db7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbbc145f50 0x7ffbbc1f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.358+0000 7ffbc0db7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0d3090 con 0x7ffbbc145f50 2024-04-25T10:12:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.358+0000 7ffbc0db7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0d2db0 con 0x7ffbbc152ab0 2024-04-25T10:12:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.358+0000 7ffbc0db7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0b9820 con 0x7ffba8003a00 2024-04-25T10:12:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.358+0000 7ffbbad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbbc145f50 0x7ffbbc1f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.358+0000 7ffbbad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbbc145f50 0x7ffbbc1f6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60876/0 (socket says 172.21.15.16:60876) 2024-04-25T10:12:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.358+0000 7ffbbad9d640 1 -- 172.21.15.16:0/1614020939 learned_addr learned my addr 172.21.15.16:0/1614020939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.358+0000 7ffbba59c640 1 --2- 172.21.15.16:0/1614020939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba8003a00 0x7ffbbc153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.358+0000 7ffbb9d9b640 1 --2- 172.21.15.16:0/1614020939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbbc152ab0 0x7ffbbc145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.359+0000 7ffbbad9d640 1 -- 172.21.15.16:0/1614020939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba8003a00 msgr2=0x7ffbbc153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.359+0000 7ffbbad9d640 1 --2- 172.21.15.16:0/1614020939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba8003a00 0x7ffbbc153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.359+0000 7ffbbad9d640 1 -- 172.21.15.16:0/1614020939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbbc152ab0 msgr2=0x7ffbbc145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.359+0000 7ffbbad9d640 1 --2- 172.21.15.16:0/1614020939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbbc152ab0 0x7ffbbc145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.359+0000 7ffbbad9d640 1 -- 172.21.15.16:0/1614020939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc1563d0 con 0x7ffbbc145f50 2024-04-25T10:12:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.359+0000 7ffbbad9d640 1 --2- 172.21.15.16:0/1614020939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbbc145f50 0x7ffbbc1f6da0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc161710 tx=0x7ffbac0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.359+0000 7ffbb2ffd640 1 -- 172.21.15.16:0/1614020939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac011de0 con 0x7ffbbc145f50 2024-04-25T10:12:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.359+0000 7ffbb2ffd640 1 -- 172.21.15.16:0/1614020939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac01b070 con 0x7ffbbc145f50 2024-04-25T10:12:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.359+0000 7ffbc0db7640 1 -- 172.21.15.16:0/1614020939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbbc1566b0 con 0x7ffbbc145f50 2024-04-25T10:12:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.359+0000 7ffbb2ffd640 1 -- 172.21.15.16:0/1614020939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac003990 con 0x7ffbbc145f50 2024-04-25T10:12:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.359+0000 7ffbc0db7640 1 -- 172.21.15.16:0/1614020939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbbc156c70 con 0x7ffbbc145f50 2024-04-25T10:12:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.360+0000 7ffbc0db7640 1 -- 172.21.15.16:0/1614020939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbbc156eb0 con 0x7ffbbc145f50 2024-04-25T10:12:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.360+0000 7ffbb2ffd640 1 -- 172.21.15.16:0/1614020939 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffbac003b30 con 0x7ffbbc145f50 2024-04-25T10:12:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.361+0000 7ffbb2ffd640 1 --2- 172.21.15.16:0/1614020939 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffb9c06b4d0 0x7ffb9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.361+0000 7ffbb2ffd640 1 -- 172.21.15.16:0/1614020939 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7ffbac08a930 con 0x7ffbbc145f50 2024-04-25T10:12:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.361+0000 7ffbba59c640 1 --2- 172.21.15.16:0/1614020939 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffb9c06b4d0 0x7ffb9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.362+0000 7ffbba59c640 1 --2- 172.21.15.16:0/1614020939 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffb9c06b4d0 0x7ffb9c06d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4032000 tx=0x7ffbb402d000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.365+0000 7ffbb2ffd640 1 -- 172.21.15.16:0/1614020939 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffbbc156eb0 con 0x7ffbbc145f50 2024-04-25T10:12:33.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.546+0000 7ffbc0db7640 1 -- 172.21.15.16:0/1614020939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffbbc0cdce0 con 0x7ffbbc145f50 2024-04-25T10:12:33.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.547+0000 7ffbb2ffd640 1 -- 172.21.15.16:0/1614020939 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16935 (secure 0 0 0) 0x7ffbbc0cdce0 con 0x7ffbbc145f50 2024-04-25T10:12:33.552 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:33.552 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:11:55.231494+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:58.299405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:58.994953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.186:6836/3950732718":"2024-04-26T10:11:55.231471+0000","172.21.15.186:6834/3950732718":"2024-04-26T10:11:55.231471+0000","172.21.15.186:6837/3092291936":"2024-04-26T10:11:13.827348+0000","172.21.15.186:6835/3092291936":"2024-04-26T10:11:13.827348+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:33.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.549+0000 7ffbc0db7640 1 -- 172.21.15.16:0/1614020939 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffb9c06b4d0 msgr2=0x7ffb9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.549+0000 7ffbc0db7640 1 --2- 172.21.15.16:0/1614020939 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffb9c06b4d0 0x7ffb9c06d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4032000 tx=0x7ffbb402d000 comp rx=0 tx=0).stop 2024-04-25T10:12:33.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.549+0000 7ffbc0db7640 1 -- 172.21.15.16:0/1614020939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbbc145f50 msgr2=0x7ffbbc1f6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.549+0000 7ffbc0db7640 1 --2- 172.21.15.16:0/1614020939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbbc145f50 0x7ffbbc1f6da0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc161710 tx=0x7ffbac0115a0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.549+0000 7ffbc0db7640 1 -- 172.21.15.16:0/1614020939 shutdown_connections 2024-04-25T10:12:33.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.549+0000 7ffbc0db7640 1 --2- 172.21.15.16:0/1614020939 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffb9c06b4d0 0x7ffb9c06d990 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.549+0000 7ffbc0db7640 1 --2- 172.21.15.16:0/1614020939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbbc145f50 0x7ffbbc1f6da0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.549+0000 7ffbc0db7640 1 --2- 172.21.15.16:0/1614020939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbbc152ab0 0x7ffbbc145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.549+0000 7ffbc0db7640 1 --2- 172.21.15.16:0/1614020939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba8003a00 0x7ffbbc153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.549+0000 7ffbc0db7640 1 -- 172.21.15.16:0/1614020939 >> 172.21.15.16:0/1614020939 conn(0x7ffbbc072900 msgr2=0x7ffbbc0d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.549+0000 7ffbc0db7640 1 -- 172.21.15.16:0/1614020939 shutdown_connections 2024-04-25T10:12:33.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.549+0000 7ffbc0db7640 1 -- 172.21.15.16:0/1614020939 wait complete. 2024-04-25T10:12:33.565 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:10:58.299405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:10:58.994953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:33.565 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.664+0000 7fecb6846640 1 Processor -- start 2024-04-25T10:12:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.664+0000 7fecb6846640 1 -- start start 2024-04-25T10:12:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.665+0000 7fecb6846640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fecb015b7b0 0x7fecb00ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.665+0000 7fecb6846640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fecb00cf760 0x7fecb00cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.665+0000 7fecb6846640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb015f000 con 0x7fecb015e110 2024-04-25T10:12:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.665+0000 7fecb6846640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb00ced00 con 0x7fecb00cf760 2024-04-25T10:12:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.665+0000 7fecb6846640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb00ce330 con 0x7fecb015b7b0 2024-04-25T10:12:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.665+0000 7fecaf7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fecb00cf760 0x7fecb00cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.665+0000 7fecaf7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fecb00cf760 0x7fecb00cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60882/0 (socket says 172.21.15.16:60882) 2024-04-25T10:12:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.665+0000 7fecaf7fe640 1 -- 172.21.15.16:0/4129764231 learned_addr learned my addr 172.21.15.16:0/4129764231 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.665+0000 7fecaffff640 1 --2- 172.21.15.16:0/4129764231 >> v2:172.21.15.186:3300/0 conn(0x7fecb015b7b0 0x7fecb00ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.665+0000 7fecaf7fe640 1 -- 172.21.15.16:0/4129764231 >> v2:172.21.15.186:3300/0 conn(0x7fecb015b7b0 msgr2=0x7fecb00ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.665+0000 7fecaf7fe640 1 --2- 172.21.15.16:0/4129764231 >> v2:172.21.15.186:3300/0 conn(0x7fecb015b7b0 0x7fecb00ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.665+0000 7fecaf7fe640 1 -- 172.21.15.16:0/4129764231 >> v1:172.21.15.186:6789/0 conn(0x7fecb015e110 legacy=0x7fecb015e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:33.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.665+0000 7fecaf7fe640 1 -- 172.21.15.16:0/4129764231 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb015ed20 con 0x7fecb00cf760 2024-04-25T10:12:33.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.666+0000 7fecaf7fe640 1 --2- 172.21.15.16:0/4129764231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb00cf760 0x7fecb00cfbf0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7feca4009870 tx=0x7feca402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3c9e3c04812e2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.666+0000 7fecaeffd640 1 -- 172.21.15.16:0/4129764231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca4037070 con 0x7fecb00cf760 2024-04-25T10:12:33.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.666+0000 7fecaeffd640 1 -- 172.21.15.16:0/4129764231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca4032440 con 0x7fecb00cf760 2024-04-25T10:12:33.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.666+0000 7fecaeffd640 1 -- 172.21.15.16:0/4129764231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca4032810 con 0x7fecb00cf760 2024-04-25T10:12:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.667+0000 7fecb6846640 1 -- 172.21.15.16:0/4129764231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb00cf760 msgr2=0x7fecb00cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.667+0000 7fecb6846640 1 --2- 172.21.15.16:0/4129764231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb00cf760 0x7fecb00cfbf0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7feca4009870 tx=0x7feca402a270 comp rx=0 tx=0).stop 2024-04-25T10:12:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.667+0000 7fecb6846640 1 -- 172.21.15.16:0/4129764231 shutdown_connections 2024-04-25T10:12:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.667+0000 7fecb6846640 1 --2- 172.21.15.16:0/4129764231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb00cf760 0x7fecb00cfbf0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.667+0000 7fecb6846640 1 --2- 172.21.15.16:0/4129764231 >> v2:172.21.15.186:3300/0 conn(0x7fecb015b7b0 0x7fecb00ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.667+0000 7fecb6846640 1 -- 172.21.15.16:0/4129764231 >> 172.21.15.16:0/4129764231 conn(0x7fecb006f4a0 msgr2=0x7fecb006f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.667+0000 7fecb6846640 1 -- 172.21.15.16:0/4129764231 shutdown_connections 2024-04-25T10:12:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.667+0000 7fecb6846640 1 -- 172.21.15.16:0/4129764231 wait complete. 2024-04-25T10:12:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.668+0000 7fecb6846640 1 Processor -- start 2024-04-25T10:12:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.668+0000 7fecb6846640 1 -- start start 2024-04-25T10:12:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.669+0000 7fecb6846640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecb00cf760 0x7fecb00b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.669+0000 7fecb6846640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb015b7b0 0x7fecb0158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.669+0000 7fecb6846640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb015e110 0x7fecb01591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.669+0000 7fecb6846640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb01619f0 con 0x7fecb015b7b0 2024-04-25T10:12:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.669+0000 7fecb6846640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00cdfc0 con 0x7fecb015e110 2024-04-25T10:12:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.669+0000 7fecb6846640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00cc080 con 0x7fecb00cf760 2024-04-25T10:12:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.669+0000 7fecaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb015b7b0 0x7fecb0158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.669+0000 7fecaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb015b7b0 0x7fecb0158cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60896/0 (socket says 172.21.15.16:60896) 2024-04-25T10:12:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.669+0000 7fecaf7fe640 1 -- 172.21.15.16:0/2811992519 learned_addr learned my addr 172.21.15.16:0/2811992519 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.669+0000 7fecaffff640 1 --2- 172.21.15.16:0/2811992519 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecb00cf760 0x7fecb00b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.669+0000 7fecb4de3640 1 --2- 172.21.15.16:0/2811992519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb015e110 0x7fecb01591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.669+0000 7fecaf7fe640 1 -- 172.21.15.16:0/2811992519 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecb00cf760 msgr2=0x7fecb00b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.669+0000 7fecaf7fe640 1 --2- 172.21.15.16:0/2811992519 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecb00cf760 0x7fecb00b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.669+0000 7fecaf7fe640 1 -- 172.21.15.16:0/2811992519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb015e110 msgr2=0x7fecb01591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.669+0000 7fecaf7fe640 1 --2- 172.21.15.16:0/2811992519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb015e110 0x7fecb01591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.669+0000 7fecaf7fe640 1 -- 172.21.15.16:0/2811992519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb00b3370 con 0x7fecb015b7b0 2024-04-25T10:12:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.669+0000 7fecaffff640 1 --2- 172.21.15.16:0/2811992519 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecb00cf760 0x7fecb00b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.670+0000 7fecaf7fe640 1 --2- 172.21.15.16:0/2811992519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb015b7b0 0x7fecb0158cb0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7feca4009870 tx=0x7feca40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.670+0000 7fecacff9640 1 -- 172.21.15.16:0/2811992519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca4037040 con 0x7fecb015b7b0 2024-04-25T10:12:33.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.670+0000 7fecacff9640 1 -- 172.21.15.16:0/2811992519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca4004230 con 0x7fecb015b7b0 2024-04-25T10:12:33.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.670+0000 7fecb6846640 1 -- 172.21.15.16:0/2811992519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb00b3650 con 0x7fecb015b7b0 2024-04-25T10:12:33.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.670+0000 7fecacff9640 1 -- 172.21.15.16:0/2811992519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca4003610 con 0x7fecb015b7b0 2024-04-25T10:12:33.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.670+0000 7fecb6846640 1 -- 172.21.15.16:0/2811992519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb00b3c10 con 0x7fecb015b7b0 2024-04-25T10:12:33.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.671+0000 7fecb6846640 1 -- 172.21.15.16:0/2811992519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb00750b0 con 0x7fecb015b7b0 2024-04-25T10:12:33.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.671+0000 7fecacff9640 1 -- 172.21.15.16:0/2811992519 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feca40037b0 con 0x7fecb015b7b0 2024-04-25T10:12:33.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.672+0000 7fecacff9640 1 --2- 172.21.15.16:0/2811992519 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fec9c06b4d0 0x7fec9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.672+0000 7fecacff9640 1 -- 172.21.15.16:0/2811992519 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7feca40ab040 con 0x7fecb015b7b0 2024-04-25T10:12:33.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.672+0000 7fecaffff640 1 --2- 172.21.15.16:0/2811992519 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fec9c06b4d0 0x7fec9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.673+0000 7fecaffff640 1 --2- 172.21.15.16:0/2811992519 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fec9c06b4d0 0x7fec9c06d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7feca0006fd0 tx=0x7feca0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.676+0000 7fecacff9640 1 -- 172.21.15.16:0/2811992519 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feca4079f30 con 0x7fecb015b7b0 2024-04-25T10:12:33.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.856+0000 7fecb6846640 1 -- 172.21.15.16:0/2811992519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fecb00d0ec0 con 0x7fecb015b7b0 2024-04-25T10:12:33.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.857+0000 7fecacff9640 1 -- 172.21.15.16:0/2811992519 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7feca4079830 con 0x7fecb015b7b0 2024-04-25T10:12:33.862 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:12:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.859+0000 7fecb6846640 1 -- 172.21.15.16:0/2811992519 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fec9c06b4d0 msgr2=0x7fec9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.859+0000 7fecb6846640 1 --2- 172.21.15.16:0/2811992519 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fec9c06b4d0 0x7fec9c06d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7feca0006fd0 tx=0x7feca0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.859+0000 7fecb6846640 1 -- 172.21.15.16:0/2811992519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb015b7b0 msgr2=0x7fecb0158cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.859+0000 7fecb6846640 1 --2- 172.21.15.16:0/2811992519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb015b7b0 0x7fecb0158cb0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7feca4009870 tx=0x7feca40046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.859+0000 7fecb6846640 1 -- 172.21.15.16:0/2811992519 shutdown_connections 2024-04-25T10:12:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.859+0000 7fecb6846640 1 --2- 172.21.15.16:0/2811992519 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fec9c06b4d0 0x7fec9c06d990 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.859+0000 7fecb6846640 1 --2- 172.21.15.16:0/2811992519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb015e110 0x7fecb01591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.859+0000 7fecb6846640 1 --2- 172.21.15.16:0/2811992519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb015b7b0 0x7fecb0158cb0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.859+0000 7fecb6846640 1 --2- 172.21.15.16:0/2811992519 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecb00cf760 0x7fecb00b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.859+0000 7fecb6846640 1 -- 172.21.15.16:0/2811992519 >> 172.21.15.16:0/2811992519 conn(0x7fecb006f4a0 msgr2=0x7fecb006bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.859+0000 7fecb6846640 1 -- 172.21.15.16:0/2811992519 shutdown_connections 2024-04-25T10:12:33.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.859+0000 7fecb6846640 1 -- 172.21.15.16:0/2811992519 wait complete. 2024-04-25T10:12:33.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.973+0000 7f03a9d9e640 1 Processor -- start 2024-04-25T10:12:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.974+0000 7f03a9d9e640 1 -- start start 2024-04-25T10:12:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.974+0000 7f03a9d9e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f03a40b2c00 0x7f03a40b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.974+0000 7f03a9d9e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a40b1c00 con 0x7f03a415b7f0 2024-04-25T10:12:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.974+0000 7f03a9d9e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a40b0ea0 con 0x7f03a40b1800 2024-04-25T10:12:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.974+0000 7f03a9d9e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a40afe20 con 0x7f03a40b2c00 2024-04-25T10:12:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.974+0000 7f03a8d9c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f03a415b7f0 0x7f03a415bbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49130/0 (socket says 172.21.15.16:49130) 2024-04-25T10:12:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.974+0000 7f03a8d9c640 1 -- 172.21.15.16:0/2281207208 learned_addr learned my addr 172.21.15.16:0/2281207208 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.975+0000 7f03a3fff640 1 --2- 172.21.15.16:0/2281207208 >> v2:172.21.15.186:3300/0 conn(0x7f03a40b2c00 0x7f03a40b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.975+0000 7f03a37fe640 1 -- 172.21.15.16:0/2281207208 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392131351 0 0) 0x7f03a40b1c00 con 0x7f03a415b7f0 2024-04-25T10:12:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.975+0000 7f03a37fe640 1 -- 172.21.15.16:0/2281207208 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f038c003520 con 0x7f03a415b7f0 2024-04-25T10:12:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.975+0000 7f03a37fe640 1 -- 172.21.15.16:0/2281207208 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869727466 0 0) 0x7f03a40b0ea0 con 0x7f03a40b1800 2024-04-25T10:12:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.975+0000 7f03a37fe640 1 -- 172.21.15.16:0/2281207208 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03a40b1c00 con 0x7f03a40b1800 2024-04-25T10:12:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.975+0000 7f03a3fff640 1 -- 172.21.15.16:0/2281207208 >> v1:172.21.15.186:6789/0 conn(0x7f03a40b1800 legacy=0x7f03a40cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.975+0000 7f03a3fff640 1 -- 172.21.15.16:0/2281207208 >> v1:172.21.15.16:6789/0 conn(0x7f03a415b7f0 legacy=0x7f03a415bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.975+0000 7f03a3fff640 1 -- 172.21.15.16:0/2281207208 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a40b00e0 con 0x7f03a40b2c00 2024-04-25T10:12:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.976+0000 7f03a3fff640 1 --2- 172.21.15.16:0/2281207208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03a40b2c00 0x7f03a40b12c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f03980097a0 tx=0x7f039802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b67658734d9c88f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.976+0000 7f03a37fe640 1 -- 172.21.15.16:0/2281207208 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0398037070 con 0x7f03a40b2c00 2024-04-25T10:12:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.976+0000 7f03a37fe640 1 -- 172.21.15.16:0/2281207208 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0398032440 con 0x7f03a40b2c00 2024-04-25T10:12:33.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.976+0000 7f03a37fe640 1 -- 172.21.15.16:0/2281207208 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0398032810 con 0x7f03a40b2c00 2024-04-25T10:12:33.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.976+0000 7f03a9d9e640 1 -- 172.21.15.16:0/2281207208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03a40b2c00 msgr2=0x7f03a40b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.976+0000 7f03a9d9e640 1 --2- 172.21.15.16:0/2281207208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03a40b2c00 0x7f03a40b12c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f03980097a0 tx=0x7f039802a230 comp rx=0 tx=0).stop 2024-04-25T10:12:33.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.976+0000 7f03a9d9e640 1 -- 172.21.15.16:0/2281207208 shutdown_connections 2024-04-25T10:12:33.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.976+0000 7f03a9d9e640 1 --2- 172.21.15.16:0/2281207208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03a40b2c00 0x7f03a40b12c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.976+0000 7f03a9d9e640 1 -- 172.21.15.16:0/2281207208 >> 172.21.15.16:0/2281207208 conn(0x7f03a4099a40 msgr2=0x7f03a40b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.977+0000 7f03a9d9e640 1 -- 172.21.15.16:0/2281207208 shutdown_connections 2024-04-25T10:12:33.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.977+0000 7f03a9d9e640 1 -- 172.21.15.16:0/2281207208 wait complete. 2024-04-25T10:12:33.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.978+0000 7f03a9d9e640 1 Processor -- start 2024-04-25T10:12:33.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.978+0000 7f03a9d9e640 1 -- start start 2024-04-25T10:12:33.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.979+0000 7f03a9d9e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03a40b1800 0x7f03a41f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.979+0000 7f03a9d9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a40b2c00 0x7f03a40b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.979+0000 7f03a9d9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03a415b7f0 0x7f03a40b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.979+0000 7f03a9d9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a40b0980 con 0x7f03a40b2c00 2024-04-25T10:12:33.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.979+0000 7f03a9d9e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a40b06a0 con 0x7f03a40b1800 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.979+0000 7f03a9d9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a4099ec0 con 0x7f03a415b7f0 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.979+0000 7f03a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a40b2c00 0x7f03a40b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.979+0000 7f03a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a40b2c00 0x7f03a40b54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60906/0 (socket says 172.21.15.16:60906) 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.979+0000 7f03a3fff640 1 -- 172.21.15.16:0/274139631 learned_addr learned my addr 172.21.15.16:0/274139631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.979+0000 7f03a8d9c640 1 --2- 172.21.15.16:0/274139631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03a40b1800 0x7f03a41f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.979+0000 7f03a959d640 1 --2- 172.21.15.16:0/274139631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03a415b7f0 0x7f03a40b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.979+0000 7f03a3fff640 1 -- 172.21.15.16:0/274139631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03a415b7f0 msgr2=0x7f03a40b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.979+0000 7f03a3fff640 1 --2- 172.21.15.16:0/274139631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03a415b7f0 0x7f03a40b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.979+0000 7f03a3fff640 1 -- 172.21.15.16:0/274139631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03a40b1800 msgr2=0x7f03a41f1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.979+0000 7f03a3fff640 1 --2- 172.21.15.16:0/274139631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03a40b1800 0x7f03a41f1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.979+0000 7f03a3fff640 1 -- 172.21.15.16:0/274139631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a40b5c30 con 0x7f03a40b2c00 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.979+0000 7f03a3fff640 1 --2- 172.21.15.16:0/274139631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a40b2c00 0x7f03a40b54e0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f0398035040 tx=0x7f03980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.980+0000 7f03a17fa640 1 -- 172.21.15.16:0/274139631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0398041030 con 0x7f03a40b2c00 2024-04-25T10:12:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.980+0000 7f03a17fa640 1 -- 172.21.15.16:0/274139631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039803d4e0 con 0x7f03a40b2c00 2024-04-25T10:12:33.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.980+0000 7f03a9d9e640 1 -- 172.21.15.16:0/274139631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a40b5f10 con 0x7f03a40b2c00 2024-04-25T10:12:33.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.980+0000 7f03a17fa640 1 -- 172.21.15.16:0/274139631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0398037040 con 0x7f03a40b2c00 2024-04-25T10:12:33.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.980+0000 7f03a9d9e640 1 -- 172.21.15.16:0/274139631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a40b3600 con 0x7f03a40b2c00 2024-04-25T10:12:33.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.981+0000 7f03a9d9e640 1 -- 172.21.15.16:0/274139631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a40749b0 con 0x7f03a40b2c00 2024-04-25T10:12:33.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.981+0000 7f03a17fa640 1 -- 172.21.15.16:0/274139631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0398044070 con 0x7f03a40b2c00 2024-04-25T10:12:33.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.981+0000 7f03a17fa640 1 --2- 172.21.15.16:0/274139631 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f039006b4d0 0x7f039006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.981+0000 7f03a17fa640 1 -- 172.21.15.16:0/274139631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f03980aa920 con 0x7f03a40b2c00 2024-04-25T10:12:33.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.982+0000 7f03a8d9c640 1 --2- 172.21.15.16:0/274139631 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f039006b4d0 0x7f039006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.982+0000 7f03a8d9c640 1 --2- 172.21.15.16:0/274139631 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f039006b4d0 0x7f039006d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0394000f60 tx=0x7f0394009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:33.986+0000 7f03a17fa640 1 -- 172.21.15.16:0/274139631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0398079820 con 0x7f03a40b2c00 2024-04-25T10:12:34.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.167+0000 7f03a9d9e640 1 -- 172.21.15.16:0/274139631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03a415bfe0 con 0x7f03a40b2c00 2024-04-25T10:12:34.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.167+0000 7f03a17fa640 1 -- 172.21.15.16:0/274139631 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0398079120 con 0x7f03a40b2c00 2024-04-25T10:12:34.172 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:12:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.169+0000 7f03a9d9e640 1 -- 172.21.15.16:0/274139631 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f039006b4d0 msgr2=0x7f039006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.169+0000 7f03a9d9e640 1 --2- 172.21.15.16:0/274139631 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f039006b4d0 0x7f039006d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0394000f60 tx=0x7f0394009040 comp rx=0 tx=0).stop 2024-04-25T10:12:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.169+0000 7f03a9d9e640 1 -- 172.21.15.16:0/274139631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a40b2c00 msgr2=0x7f03a40b54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.169+0000 7f03a9d9e640 1 --2- 172.21.15.16:0/274139631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a40b2c00 0x7f03a40b54e0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f0398035040 tx=0x7f03980046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.169+0000 7f03a9d9e640 1 -- 172.21.15.16:0/274139631 shutdown_connections 2024-04-25T10:12:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.169+0000 7f03a9d9e640 1 --2- 172.21.15.16:0/274139631 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f039006b4d0 0x7f039006d990 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.169+0000 7f03a9d9e640 1 --2- 172.21.15.16:0/274139631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03a415b7f0 0x7f03a40b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.169+0000 7f03a9d9e640 1 --2- 172.21.15.16:0/274139631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a40b2c00 0x7f03a40b54e0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.169+0000 7f03a9d9e640 1 --2- 172.21.15.16:0/274139631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03a40b1800 0x7f03a41f1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.169+0000 7f03a9d9e640 1 -- 172.21.15.16:0/274139631 >> 172.21.15.16:0/274139631 conn(0x7f03a4099a40 msgr2=0x7f03a40af420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:34.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.169+0000 7f03a9d9e640 1 -- 172.21.15.16:0/274139631 shutdown_connections 2024-04-25T10:12:34.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.169+0000 7f03a9d9e640 1 -- 172.21.15.16:0/274139631 wait complete. 2024-04-25T10:12:34.185 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:34.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.283+0000 7f6010c9e640 1 Processor -- start 2024-04-25T10:12:34.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.283+0000 7f6010c9e640 1 -- start start 2024-04-25T10:12:34.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.284+0000 7f6010c9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f600c144e70 0x7f600c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.284+0000 7f6010c9e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f600c0cf0e0 0x7f600c0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.284+0000 7f6010c9e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600c06dcb0 con 0x7f600c06d7a0 2024-04-25T10:12:34.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.284+0000 7f6010c9e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600c15bfe0 con 0x7f600c0cf0e0 2024-04-25T10:12:34.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.284+0000 7f6010c9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c06df90 con 0x7f600c144e70 2024-04-25T10:12:34.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.285+0000 7f600ad9d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f600c0cf0e0 0x7f600c0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.285+0000 7f600a59c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f600c144e70 0x7f600c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.285+0000 7f600a59c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f600c144e70 0x7f600c06d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:34726/0 (socket says 172.21.15.16:34726) 2024-04-25T10:12:34.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.285+0000 7f600a59c640 1 -- 172.21.15.16:0/585166928 learned_addr learned my addr 172.21.15.16:0/585166928 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:34.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.285+0000 7f600a59c640 1 -- 172.21.15.16:0/585166928 >> v2:172.21.15.186:3300/0 conn(0x7f600c0cf0e0 msgr2=0x7f600c0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:34.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.285+0000 7f600a59c640 1 --2- 172.21.15.16:0/585166928 >> v2:172.21.15.186:3300/0 conn(0x7f600c0cf0e0 0x7f600c0cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.285+0000 7f600a59c640 1 -- 172.21.15.16:0/585166928 >> v1:172.21.15.186:6789/0 conn(0x7f600c06d7a0 legacy=0x7f600c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.285+0000 7f600a59c640 1 -- 172.21.15.16:0/585166928 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c06e270 con 0x7f600c144e70 2024-04-25T10:12:34.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.285+0000 7f600ad9d640 1 --2- 172.21.15.16:0/585166928 >> v2:172.21.15.186:3300/0 conn(0x7f600c0cf0e0 0x7f600c0cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:34.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.286+0000 7f600a59c640 1 --2- 172.21.15.16:0/585166928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f600c144e70 0x7f600c06d260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6004009e50 tx=0x7f600402a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b4125f377431e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.286+0000 7f600959a640 1 -- 172.21.15.16:0/585166928 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004036070 con 0x7f600c144e70 2024-04-25T10:12:34.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.286+0000 7f600959a640 1 -- 172.21.15.16:0/585166928 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6004031600 con 0x7f600c144e70 2024-04-25T10:12:34.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.286+0000 7f600959a640 1 -- 172.21.15.16:0/585166928 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60040319d0 con 0x7f600c144e70 2024-04-25T10:12:34.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.287+0000 7f6010c9e640 1 -- 172.21.15.16:0/585166928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f600c144e70 msgr2=0x7f600c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.287+0000 7f6010c9e640 1 --2- 172.21.15.16:0/585166928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f600c144e70 0x7f600c06d260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6004009e50 tx=0x7f600402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.287+0000 7f6010c9e640 1 -- 172.21.15.16:0/585166928 shutdown_connections 2024-04-25T10:12:34.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.287+0000 7f6010c9e640 1 --2- 172.21.15.16:0/585166928 >> v2:172.21.15.186:3300/0 conn(0x7f600c0cf0e0 0x7f600c0cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.287+0000 7f6010c9e640 1 --2- 172.21.15.16:0/585166928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f600c144e70 0x7f600c06d260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.287+0000 7f6010c9e640 1 -- 172.21.15.16:0/585166928 >> 172.21.15.16:0/585166928 conn(0x7f600c06aba0 msgr2=0x7f600c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:34.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.287+0000 7f6010c9e640 1 -- 172.21.15.16:0/585166928 shutdown_connections 2024-04-25T10:12:34.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.287+0000 7f6010c9e640 1 -- 172.21.15.16:0/585166928 wait complete. 2024-04-25T10:12:34.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.288+0000 7f6010c9e640 1 Processor -- start 2024-04-25T10:12:34.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.288+0000 7f6010c9e640 1 -- start start 2024-04-25T10:12:34.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.289+0000 7f6010c9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f600c06d7a0 0x7f600c15b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.289+0000 7f6010c9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f600c0cf0e0 0x7f600c0ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.289+0000 7f6010c9e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f600c144e70 0x7f600c0ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.289+0000 7f6010c9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c145630 con 0x7f600c06d7a0 2024-04-25T10:12:34.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.289+0000 7f6010c9e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c1454b0 con 0x7f600c144e70 2024-04-25T10:12:34.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.289+0000 7f6010c9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0cc840 con 0x7f600c0cf0e0 2024-04-25T10:12:34.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.289+0000 7f600a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f600c06d7a0 0x7f600c15b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.289+0000 7f600a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f600c06d7a0 0x7f600c15b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60922/0 (socket says 172.21.15.16:60922) 2024-04-25T10:12:34.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.289+0000 7f600a59c640 1 -- 172.21.15.16:0/1413274384 learned_addr learned my addr 172.21.15.16:0/1413274384 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.289+0000 7f600ad9d640 1 --2- 172.21.15.16:0/1413274384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f600c144e70 0x7f600c0ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.289+0000 7f6009d9b640 1 --2- 172.21.15.16:0/1413274384 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f600c0cf0e0 0x7f600c0ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.289+0000 7f600a59c640 1 -- 172.21.15.16:0/1413274384 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f600c0cf0e0 msgr2=0x7f600c0ab3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.289+0000 7f600a59c640 1 --2- 172.21.15.16:0/1413274384 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f600c0cf0e0 0x7f600c0ab3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.289+0000 7f600a59c640 1 -- 172.21.15.16:0/1413274384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f600c144e70 msgr2=0x7f600c0ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.289+0000 7f600a59c640 1 --2- 172.21.15.16:0/1413274384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f600c144e70 0x7f600c0ab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.289+0000 7f600a59c640 1 -- 172.21.15.16:0/1413274384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c1e68c0 con 0x7f600c06d7a0 2024-04-25T10:12:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.289+0000 7f600ad9d640 1 --2- 172.21.15.16:0/1413274384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f600c144e70 0x7f600c0ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.289+0000 7f600a59c640 1 --2- 172.21.15.16:0/1413274384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f600c06d7a0 0x7f600c15b430 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f6004034040 tx=0x7f6004004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.290+0000 7f5ffaffd640 1 -- 172.21.15.16:0/1413274384 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004041020 con 0x7f600c06d7a0 2024-04-25T10:12:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.290+0000 7f5ffaffd640 1 -- 172.21.15.16:0/1413274384 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6004007820 con 0x7f600c06d7a0 2024-04-25T10:12:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.290+0000 7f6010c9e640 1 -- 172.21.15.16:0/1413274384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600c1e6ba0 con 0x7f600c06d7a0 2024-04-25T10:12:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.290+0000 7f5ffaffd640 1 -- 172.21.15.16:0/1413274384 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004036040 con 0x7f600c06d7a0 2024-04-25T10:12:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.290+0000 7f6010c9e640 1 -- 172.21.15.16:0/1413274384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600c1e7160 con 0x7f600c06d7a0 2024-04-25T10:12:34.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.291+0000 7f6010c9e640 1 -- 172.21.15.16:0/1413274384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600c0750b0 con 0x7f600c06d7a0 2024-04-25T10:12:34.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.291+0000 7f5ffaffd640 1 -- 172.21.15.16:0/1413274384 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6004044020 con 0x7f600c06d7a0 2024-04-25T10:12:34.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.292+0000 7f5ffaffd640 1 --2- 172.21.15.16:0/1413274384 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5fec06b4d0 0x7f5fec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.292+0000 7f5ffaffd640 1 -- 172.21.15.16:0/1413274384 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f60040aaf40 con 0x7f600c06d7a0 2024-04-25T10:12:34.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.292+0000 7f6009d9b640 1 --2- 172.21.15.16:0/1413274384 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5fec06b4d0 0x7f5fec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.293+0000 7f6009d9b640 1 --2- 172.21.15.16:0/1413274384 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5fec06b4d0 0x7f5fec06d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f600c1e7a30 tx=0x7f6000009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.296+0000 7f5ffaffd640 1 -- 172.21.15.16:0/1413274384 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f600404b030 con 0x7f600c06d7a0 2024-04-25T10:12:34.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.475+0000 7f6010c9e640 1 -- 172.21.15.16:0/1413274384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f600c0aebe0 con 0x7f600c06d7a0 2024-04-25T10:12:34.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.475+0000 7f5ffaffd640 1 -- 172.21.15.16:0/1413274384 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6004079c90 con 0x7f600c06d7a0 2024-04-25T10:12:34.480 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:12:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.477+0000 7f6010c9e640 1 -- 172.21.15.16:0/1413274384 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5fec06b4d0 msgr2=0x7f5fec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.477+0000 7f6010c9e640 1 --2- 172.21.15.16:0/1413274384 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5fec06b4d0 0x7f5fec06d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f600c1e7a30 tx=0x7f6000009040 comp rx=0 tx=0).stop 2024-04-25T10:12:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.477+0000 7f6010c9e640 1 -- 172.21.15.16:0/1413274384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f600c06d7a0 msgr2=0x7f600c15b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.477+0000 7f6010c9e640 1 --2- 172.21.15.16:0/1413274384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f600c06d7a0 0x7f600c15b430 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f6004034040 tx=0x7f6004004a40 comp rx=0 tx=0).stop 2024-04-25T10:12:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.477+0000 7f6010c9e640 1 -- 172.21.15.16:0/1413274384 shutdown_connections 2024-04-25T10:12:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.477+0000 7f6010c9e640 1 --2- 172.21.15.16:0/1413274384 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5fec06b4d0 0x7f5fec06d990 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.477+0000 7f6010c9e640 1 --2- 172.21.15.16:0/1413274384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f600c144e70 0x7f600c0ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.477+0000 7f6010c9e640 1 --2- 172.21.15.16:0/1413274384 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f600c0cf0e0 0x7f600c0ab3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.477+0000 7f6010c9e640 1 --2- 172.21.15.16:0/1413274384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f600c06d7a0 0x7f600c15b430 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.477+0000 7f6010c9e640 1 -- 172.21.15.16:0/1413274384 >> 172.21.15.16:0/1413274384 conn(0x7f600c06aba0 msgr2=0x7f600c0cf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.477+0000 7f6010c9e640 1 -- 172.21.15.16:0/1413274384 shutdown_connections 2024-04-25T10:12:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.477+0000 7f6010c9e640 1 -- 172.21.15.16:0/1413274384 wait complete. 2024-04-25T10:12:34.493 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:12:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.590+0000 7fe1e0ccb640 1 Processor -- start 2024-04-25T10:12:34.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.590+0000 7fe1e0ccb640 1 -- start start 2024-04-25T10:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.591+0000 7fe1e0ccb640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe1dc1610b0 0x7fe1dc0b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.591+0000 7fe1e0ccb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1dc1618d0 0x7fe1dc0ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.591+0000 7fe1e0ccb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe1dc0ced70 0x7fe1dc06c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.591+0000 7fe1e0ccb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1dc0d1ea0 con 0x7fe1dc0ced70 2024-04-25T10:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.591+0000 7fe1e0ccb640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1dc162040 con 0x7fe1dc1610b0 2024-04-25T10:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.591+0000 7fe1e0ccb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc162320 con 0x7fe1dc1618d0 2024-04-25T10:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.591+0000 7fe1dad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe1dc0ced70 0x7fe1dc06c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.591+0000 7fe1dad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe1dc0ced70 0x7fe1dc06c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60930/0 (socket says 172.21.15.16:60930) 2024-04-25T10:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.591+0000 7fe1dad9d640 1 -- 172.21.15.16:0/2002054727 learned_addr learned my addr 172.21.15.16:0/2002054727 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.591+0000 7fe1d1d9b640 1 --2- 172.21.15.16:0/2002054727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1dc1618d0 0x7fe1dc0ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.591+0000 7fe1da59c640 1 --2- 172.21.15.16:0/2002054727 >> v2:172.21.15.186:3300/0 conn(0x7fe1dc1610b0 0x7fe1dc0b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.591+0000 7fe1dad9d640 1 -- 172.21.15.16:0/2002054727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1dc1618d0 msgr2=0x7fe1dc0ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.591+0000 7fe1dad9d640 1 --2- 172.21.15.16:0/2002054727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1dc1618d0 0x7fe1dc0ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.591+0000 7fe1dad9d640 1 -- 172.21.15.16:0/2002054727 >> v2:172.21.15.186:3300/0 conn(0x7fe1dc1610b0 msgr2=0x7fe1dc0b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.591+0000 7fe1dad9d640 1 --2- 172.21.15.16:0/2002054727 >> v2:172.21.15.186:3300/0 conn(0x7fe1dc1610b0 0x7fe1dc0b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.591+0000 7fe1dad9d640 1 -- 172.21.15.16:0/2002054727 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc162600 con 0x7fe1dc0ced70 2024-04-25T10:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.591+0000 7fe1da59c640 1 --2- 172.21.15.16:0/2002054727 >> v2:172.21.15.186:3300/0 conn(0x7fe1dc1610b0 0x7fe1dc0b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.591+0000 7fe1d1d9b640 1 --2- 172.21.15.16:0/2002054727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1dc1618d0 0x7fe1dc0ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.592+0000 7fe1dad9d640 1 --2- 172.21.15.16:0/2002054727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1dc0ced70 0x7fe1dc06c250 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc08c410 tx=0x7fe1d402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139e1fc195f58835 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.592+0000 7fe1d9d9b640 1 -- 172.21.15.16:0/2002054727 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4036070 con 0x7fe1dc0ced70 2024-04-25T10:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.592+0000 7fe1d9d9b640 1 -- 172.21.15.16:0/2002054727 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d4031600 con 0x7fe1dc0ced70 2024-04-25T10:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.592+0000 7fe1d9d9b640 1 -- 172.21.15.16:0/2002054727 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d40319d0 con 0x7fe1dc0ced70 2024-04-25T10:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.592+0000 7fe1e0ccb640 1 -- 172.21.15.16:0/2002054727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1dc0ced70 msgr2=0x7fe1dc06c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.592+0000 7fe1e0ccb640 1 --2- 172.21.15.16:0/2002054727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1dc0ced70 0x7fe1dc06c250 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc08c410 tx=0x7fe1d402a610 comp rx=0 tx=0).stop 2024-04-25T10:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.593+0000 7fe1e0ccb640 1 -- 172.21.15.16:0/2002054727 shutdown_connections 2024-04-25T10:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.593+0000 7fe1e0ccb640 1 --2- 172.21.15.16:0/2002054727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1dc0ced70 0x7fe1dc06c250 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.593+0000 7fe1e0ccb640 1 --2- 172.21.15.16:0/2002054727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1dc1618d0 0x7fe1dc0ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.593+0000 7fe1e0ccb640 1 --2- 172.21.15.16:0/2002054727 >> v2:172.21.15.186:3300/0 conn(0x7fe1dc1610b0 0x7fe1dc0b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.593+0000 7fe1e0ccb640 1 -- 172.21.15.16:0/2002054727 >> 172.21.15.16:0/2002054727 conn(0x7fe1dc0b5570 msgr2=0x7fe1dc0b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:34.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.593+0000 7fe1e0ccb640 1 -- 172.21.15.16:0/2002054727 shutdown_connections 2024-04-25T10:12:34.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.593+0000 7fe1e0ccb640 1 -- 172.21.15.16:0/2002054727 wait complete. 2024-04-25T10:12:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.594+0000 7fe1e0ccb640 1 Processor -- start 2024-04-25T10:12:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.594+0000 7fe1e0ccb640 1 -- start start 2024-04-25T10:12:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1e0ccb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1dc0ced70 0x7fe1dc14f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1e0ccb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1dc1610b0 0x7fe1dc0cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1e0ccb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1dc1618d0 0x7fe1dc15c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1e0ccb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc161d10 con 0x7fe1dc1610b0 2024-04-25T10:12:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1e0ccb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc0b6c00 con 0x7fe1dc1618d0 2024-04-25T10:12:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1e0ccb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc161490 con 0x7fe1dc0ced70 2024-04-25T10:12:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1d1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1dc1610b0 0x7fe1dc0cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1d1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1dc1610b0 0x7fe1dc0cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60940/0 (socket says 172.21.15.16:60940) 2024-04-25T10:12:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1d1d9b640 1 -- 172.21.15.16:0/3297919940 learned_addr learned my addr 172.21.15.16:0/3297919940 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1dad9d640 1 --2- 172.21.15.16:0/3297919940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1dc1618d0 0x7fe1dc15c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1da59c640 1 --2- 172.21.15.16:0/3297919940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1dc0ced70 0x7fe1dc14f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1d1d9b640 1 -- 172.21.15.16:0/3297919940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1dc0ced70 msgr2=0x7fe1dc14f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1d1d9b640 1 --2- 172.21.15.16:0/3297919940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1dc0ced70 0x7fe1dc14f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1d1d9b640 1 -- 172.21.15.16:0/3297919940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1dc1618d0 msgr2=0x7fe1dc15c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1d1d9b640 1 --2- 172.21.15.16:0/3297919940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1dc1618d0 0x7fe1dc15c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1d1d9b640 1 -- 172.21.15.16:0/3297919940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc0cce80 con 0x7fe1dc1610b0 2024-04-25T10:12:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1da59c640 1 --2- 172.21.15.16:0/3297919940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1dc0ced70 0x7fe1dc14f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1d1d9b640 1 --2- 172.21.15.16:0/3297919940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1dc1610b0 0x7fe1dc0cc510 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400c820 tx=0x7fe1c400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.595+0000 7fe1d37fe640 1 -- 172.21.15.16:0/3297919940 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4015070 con 0x7fe1dc1610b0 2024-04-25T10:12:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.596+0000 7fe1d37fe640 1 -- 172.21.15.16:0/3297919940 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c4007a30 con 0x7fe1dc1610b0 2024-04-25T10:12:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.596+0000 7fe1e0ccb640 1 -- 172.21.15.16:0/3297919940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1dc0cd160 con 0x7fe1dc1610b0 2024-04-25T10:12:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.596+0000 7fe1d37fe640 1 -- 172.21.15.16:0/3297919940 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4007e00 con 0x7fe1dc1610b0 2024-04-25T10:12:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.596+0000 7fe1e0ccb640 1 -- 172.21.15.16:0/3297919940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1dc0cd720 con 0x7fe1dc1610b0 2024-04-25T10:12:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.597+0000 7fe1e0ccb640 1 -- 172.21.15.16:0/3297919940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1dc0750b0 con 0x7fe1dc1610b0 2024-04-25T10:12:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.597+0000 7fe1d37fe640 1 -- 172.21.15.16:0/3297919940 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1c4018070 con 0x7fe1dc1610b0 2024-04-25T10:12:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.597+0000 7fe1d37fe640 1 --2- 172.21.15.16:0/3297919940 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe1bc06b4d0 0x7fe1bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.597+0000 7fe1d37fe640 1 -- 172.21.15.16:0/3297919940 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fe1c408a160 con 0x7fe1dc1610b0 2024-04-25T10:12:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.598+0000 7fe1da59c640 1 --2- 172.21.15.16:0/3297919940 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe1bc06b4d0 0x7fe1bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.598+0000 7fe1da59c640 1 --2- 172.21.15.16:0/3297919940 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe1bc06b4d0 0x7fe1bc06d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc006fd0 tx=0x7fe1cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.602+0000 7fe1d37fe640 1 -- 172.21.15.16:0/3297919940 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1c4058fc0 con 0x7fe1dc1610b0 2024-04-25T10:12:34.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.799+0000 7fe1e0ccb640 1 -- 172.21.15.16:0/3297919940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe1dc14f610 con 0x7fe1dc1610b0 2024-04-25T10:12:34.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.800+0000 7fe1d37fe640 1 -- 172.21.15.16:0/3297919940 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v88) v1 ==== 80+0+248 (secure 0 0 0) 0x7fe1c40588c0 con 0x7fe1dc1610b0 2024-04-25T10:12:34.805 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.186:6836/3950732718 2024-04-26T10:11:55.231471+0000 2024-04-25T10:12:34.805 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.186:6834/3950732718 2024-04-26T10:11:55.231471+0000 2024-04-25T10:12:34.805 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.186:6837/3092291936 2024-04-26T10:11:13.827348+0000 2024-04-25T10:12:34.805 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.186:6835/3092291936 2024-04-26T10:11:13.827348+0000 2024-04-25T10:12:34.805 INFO:teuthology.orchestra.run.smithi016.stderr:listed 4 entries 2024-04-25T10:12:34.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.802+0000 7fe1e0ccb640 1 -- 172.21.15.16:0/3297919940 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe1bc06b4d0 msgr2=0x7fe1bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.802+0000 7fe1e0ccb640 1 --2- 172.21.15.16:0/3297919940 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe1bc06b4d0 0x7fe1bc06d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc006fd0 tx=0x7fe1cc008040 comp rx=0 tx=0).stop 2024-04-25T10:12:34.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.802+0000 7fe1e0ccb640 1 -- 172.21.15.16:0/3297919940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1dc1610b0 msgr2=0x7fe1dc0cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.802+0000 7fe1e0ccb640 1 --2- 172.21.15.16:0/3297919940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1dc1610b0 0x7fe1dc0cc510 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400c820 tx=0x7fe1c400cc20 comp rx=0 tx=0).stop 2024-04-25T10:12:34.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.802+0000 7fe1e0ccb640 1 -- 172.21.15.16:0/3297919940 shutdown_connections 2024-04-25T10:12:34.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.802+0000 7fe1e0ccb640 1 --2- 172.21.15.16:0/3297919940 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe1bc06b4d0 0x7fe1bc06d990 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.802+0000 7fe1e0ccb640 1 --2- 172.21.15.16:0/3297919940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1dc1618d0 0x7fe1dc15c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.802+0000 7fe1e0ccb640 1 --2- 172.21.15.16:0/3297919940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1dc1610b0 0x7fe1dc0cc510 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.802+0000 7fe1e0ccb640 1 --2- 172.21.15.16:0/3297919940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1dc0ced70 0x7fe1dc14f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.802+0000 7fe1e0ccb640 1 -- 172.21.15.16:0/3297919940 >> 172.21.15.16:0/3297919940 conn(0x7fe1dc0b5570 msgr2=0x7fe1dc159bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:34.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.802+0000 7fe1e0ccb640 1 -- 172.21.15.16:0/3297919940 shutdown_connections 2024-04-25T10:12:34.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:34.802+0000 7fe1e0ccb640 1 -- 172.21.15.16:0/3297919940 wait complete. 2024-04-25T10:12:34.817 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:12:34.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:34.817 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:34.844 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:12:34.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:34.845 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:12:34.900 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:12:34.900 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:34.919 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:12:34.920 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:12:34.920 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:34.955 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:12:34.956 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:12:34.956 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:34.977 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:12:34.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:35.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.090+0000 7fd77fdcf640 1 Processor -- start 2024-04-25T10:12:35.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.091+0000 7fd77fdcf640 1 -- start start 2024-04-25T10:12:35.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.091+0000 7fd77fdcf640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77815dce0 0x7fd7780d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.091+0000 7fd77fdcf640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd7780b5b60 0x7fd7780b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.091+0000 7fd77fdcf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd77815e0c0 con 0x7fd7780d35b0 2024-04-25T10:12:35.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.091+0000 7fd77fdcf640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7780d2510 con 0x7fd7780b5b60 2024-04-25T10:12:35.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.091+0000 7fd77fdcf640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7780d27f0 con 0x7fd77815dce0 2024-04-25T10:12:35.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.091+0000 7fd77d36a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd7780d35b0 0x7fd7780b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49132/0 (socket says 172.21.15.16:49132) 2024-04-25T10:12:35.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.091+0000 7fd77d36a640 1 -- 172.21.15.16:0/2711080282 learned_addr learned my addr 172.21.15.16:0/2711080282 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:35.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77cb69640 1 -- 172.21.15.16:0/2711080282 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093027277 0 0) 0x7fd77815e0c0 con 0x7fd7780d35b0 2024-04-25T10:12:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77cb69640 1 -- 172.21.15.16:0/2711080282 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd760003520 con 0x7fd7780d35b0 2024-04-25T10:12:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77e36c640 1 --2- 172.21.15.16:0/2711080282 >> v2:172.21.15.186:3300/0 conn(0x7fd7780b5b60 0x7fd7780b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77db6b640 1 --2- 172.21.15.16:0/2711080282 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77815dce0 0x7fd7780d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77cb69640 1 -- 172.21.15.16:0/2711080282 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 272971892 0 0) 0x7fd760003520 con 0x7fd7780d35b0 2024-04-25T10:12:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77cb69640 1 -- 172.21.15.16:0/2711080282 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77815dce0 msgr2=0x7fd7780d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77cb69640 1 --2- 172.21.15.16:0/2711080282 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77815dce0 0x7fd7780d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77cb69640 1 -- 172.21.15.16:0/2711080282 >> v2:172.21.15.186:3300/0 conn(0x7fd7780b5b60 msgr2=0x7fd7780b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77cb69640 1 --2- 172.21.15.16:0/2711080282 >> v2:172.21.15.186:3300/0 conn(0x7fd7780b5b60 0x7fd7780b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77cb69640 1 -- 172.21.15.16:0/2711080282 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7780d2ad0 con 0x7fd7780d35b0 2024-04-25T10:12:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77cb69640 1 -- 172.21.15.16:0/2711080282 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fd76c001670 con 0x7fd7780d35b0 2024-04-25T10:12:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77cb69640 1 -- 172.21.15.16:0/2711080282 >> v1:172.21.15.16:6789/0 conn(0x7fd7780d35b0 legacy=0x7fd7780b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77cb69640 1 --2- 172.21.15.16:0/2711080282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7600039b0 0x7fd760003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77cb69640 1 --2- 172.21.15.16:0/2711080282 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77815dce0 0x7fd760004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77cb69640 1 --2- 172.21.15.16:0/2711080282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd760004d40 0x7fd7600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77cb69640 1 -- 172.21.15.16:0/2711080282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7780d2510 con 0x7fd760004d40 2024-04-25T10:12:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77cb69640 1 -- 172.21.15.16:0/2711080282 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7780d27f0 con 0x7fd7600039b0 2024-04-25T10:12:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77cb69640 1 -- 172.21.15.16:0/2711080282 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76000b9c0 con 0x7fd77815dce0 2024-04-25T10:12:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77e36c640 1 --2- 172.21.15.16:0/2711080282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd760004d40 0x7fd7600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77db6b640 1 --2- 172.21.15.16:0/2711080282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7600039b0 0x7fd760003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.092+0000 7fd77d36a640 1 --2- 172.21.15.16:0/2711080282 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77815dce0 0x7fd760004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.093+0000 7fd77e36c640 1 -- 172.21.15.16:0/2711080282 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77815dce0 msgr2=0x7fd760004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.093+0000 7fd77e36c640 1 --2- 172.21.15.16:0/2711080282 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77815dce0 0x7fd760004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.093+0000 7fd77e36c640 1 -- 172.21.15.16:0/2711080282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7600039b0 msgr2=0x7fd760003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.093+0000 7fd77e36c640 1 --2- 172.21.15.16:0/2711080282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7600039b0 0x7fd760003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.093+0000 7fd77e36c640 1 -- 172.21.15.16:0/2711080282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd760003520 con 0x7fd760004d40 2024-04-25T10:12:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.093+0000 7fd77e36c640 1 --2- 172.21.15.16:0/2711080282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd760004d40 0x7fd7600051c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd768009fc0 tx=0x7fd760005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58fc378ac988578 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.093+0000 7fd77cb69640 1 -- 172.21.15.16:0/2711080282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd768009c10 con 0x7fd760004d40 2024-04-25T10:12:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.093+0000 7fd77cb69640 1 -- 172.21.15.16:0/2711080282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd768036070 con 0x7fd760004d40 2024-04-25T10:12:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.094+0000 7fd77cb69640 1 -- 172.21.15.16:0/2711080282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76802d070 con 0x7fd760004d40 2024-04-25T10:12:35.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.094+0000 7fd77fdcf640 1 -- 172.21.15.16:0/2711080282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd760004d40 msgr2=0x7fd7600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.094+0000 7fd77fdcf640 1 --2- 172.21.15.16:0/2711080282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd760004d40 0x7fd7600051c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd768009fc0 tx=0x7fd760005830 comp rx=0 tx=0).stop 2024-04-25T10:12:35.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.094+0000 7fd77e36c640 1 -- 172.21.15.16:0/2711080282 reap_dead start 2024-04-25T10:12:35.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.094+0000 7fd77fdcf640 1 -- 172.21.15.16:0/2711080282 shutdown_connections 2024-04-25T10:12:35.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.094+0000 7fd77fdcf640 1 -- 172.21.15.16:0/2711080282 >> 172.21.15.16:0/2711080282 conn(0x7fd778072900 msgr2=0x7fd7780d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:35.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.094+0000 7fd77fdcf640 1 -- 172.21.15.16:0/2711080282 shutdown_connections 2024-04-25T10:12:35.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.094+0000 7fd77fdcf640 1 -- 172.21.15.16:0/2711080282 wait complete. 2024-04-25T10:12:35.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.095+0000 7fd77fdcf640 1 Processor -- start 2024-04-25T10:12:35.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.095+0000 7fd77fdcf640 1 -- start start 2024-04-25T10:12:35.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.096+0000 7fd77fdcf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd760004d40 0x7fd7781f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.096+0000 7fd77fdcf640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7780cbaf0 0x7fd7780c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.096+0000 7fd77fdcf640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7780cbf50 0x7fd7780c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.096+0000 7fd77fdcf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7780b9820 con 0x7fd760004d40 2024-04-25T10:12:35.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.096+0000 7fd77fdcf640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7780d3090 con 0x7fd7780cbf50 2024-04-25T10:12:35.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.096+0000 7fd77fdcf640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7780b99a0 con 0x7fd7780cbaf0 2024-04-25T10:12:35.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.096+0000 7fd77db6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd760004d40 0x7fd7781f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.096+0000 7fd77db6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd760004d40 0x7fd7781f3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60960/0 (socket says 172.21.15.16:60960) 2024-04-25T10:12:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.096+0000 7fd77db6b640 1 -- 172.21.15.16:0/856281257 learned_addr learned my addr 172.21.15.16:0/856281257 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.096+0000 7fd77d36a640 1 --2- 172.21.15.16:0/856281257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7780cbaf0 0x7fd7780c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.096+0000 7fd77e36c640 1 --2- 172.21.15.16:0/856281257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7780cbf50 0x7fd7780c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.097+0000 7fd77db6b640 1 -- 172.21.15.16:0/856281257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7780cbaf0 msgr2=0x7fd7780c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.097+0000 7fd77db6b640 1 --2- 172.21.15.16:0/856281257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7780cbaf0 0x7fd7780c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.097+0000 7fd77db6b640 1 -- 172.21.15.16:0/856281257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7780cbf50 msgr2=0x7fd7780c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.097+0000 7fd77db6b640 1 --2- 172.21.15.16:0/856281257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7780cbf50 0x7fd7780c7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.097+0000 7fd77db6b640 1 -- 172.21.15.16:0/856281257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7780cc4b0 con 0x7fd760004d40 2024-04-25T10:12:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.097+0000 7fd77e36c640 1 --2- 172.21.15.16:0/856281257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7780cbf50 0x7fd7780c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.097+0000 7fd77db6b640 1 --2- 172.21.15.16:0/856281257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd760004d40 0x7fd7781f3430 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd770011910 tx=0x7fd770011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.097+0000 7fd7767fc640 1 -- 172.21.15.16:0/856281257 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77000faf0 con 0x7fd760004d40 2024-04-25T10:12:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.097+0000 7fd7767fc640 1 -- 172.21.15.16:0/856281257 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77000fc90 con 0x7fd760004d40 2024-04-25T10:12:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.097+0000 7fd77fdcf640 1 -- 172.21.15.16:0/856281257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7780cc790 con 0x7fd760004d40 2024-04-25T10:12:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.097+0000 7fd7767fc640 1 -- 172.21.15.16:0/856281257 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd770014440 con 0x7fd760004d40 2024-04-25T10:12:35.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.097+0000 7fd77fdcf640 1 -- 172.21.15.16:0/856281257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7780ccd50 con 0x7fd760004d40 2024-04-25T10:12:35.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.098+0000 7fd77fdcf640 1 -- 172.21.15.16:0/856281257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7780749b0 con 0x7fd760004d40 2024-04-25T10:12:35.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.099+0000 7fd7767fc640 1 -- 172.21.15.16:0/856281257 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd770012040 con 0x7fd760004d40 2024-04-25T10:12:35.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.099+0000 7fd7767fc640 1 --2- 172.21.15.16:0/856281257 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd76406b500 0x7fd76406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.099+0000 7fd7767fc640 1 -- 172.21.15.16:0/856281257 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fd77008a850 con 0x7fd760004d40 2024-04-25T10:12:35.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.099+0000 7fd77d36a640 1 --2- 172.21.15.16:0/856281257 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd76406b500 0x7fd76406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.100+0000 7fd77d36a640 1 --2- 172.21.15.16:0/856281257 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd76406b500 0x7fd76406d9c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd778154240 tx=0x7fd76c007970 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.103+0000 7fd7767fc640 1 -- 172.21.15.16:0/856281257 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd770019030 con 0x7fd760004d40 2024-04-25T10:12:35.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.320+0000 7fd77fdcf640 1 -- 172.21.15.16:0/856281257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7780b40e0 con 0x7fd760004d40 2024-04-25T10:12:35.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.321+0000 7fd7767fc640 1 -- 172.21.15.16:0/856281257 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fd770059590 con 0x7fd760004d40 2024-04-25T10:12:35.326 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:35.326 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4044509901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4044509901},{"type":"v1","addr":"172.21.15.16:6835","nonce":4044509901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1494230493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1494230493},{"type":"v1","addr":"172.21.15.186:6837","nonce":1494230493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/1050183001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1050183001},{"type":"v1","addr":"172.21.15.186:6836","nonce":1050183001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:00.015075+0000","modified":"2024-04-25T10:11:58.248543+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7323},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7323":{"gid":7323,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":19,"addr":"172.21.15.16:6834/84349474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":84349474},{"type":"v1","addr":"172.21.15.16:6834","nonce":84349474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:12:35.326 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 64 2024-04-25T10:12:35.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.323+0000 7fd77fdcf640 1 -- 172.21.15.16:0/856281257 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd76406b500 msgr2=0x7fd76406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.323+0000 7fd77fdcf640 1 --2- 172.21.15.16:0/856281257 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd76406b500 0x7fd76406d9c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd778154240 tx=0x7fd76c007970 comp rx=0 tx=0).stop 2024-04-25T10:12:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.323+0000 7fd77fdcf640 1 -- 172.21.15.16:0/856281257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd760004d40 msgr2=0x7fd7781f3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.323+0000 7fd77fdcf640 1 --2- 172.21.15.16:0/856281257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd760004d40 0x7fd7781f3430 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd770011910 tx=0x7fd770011dd0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.323+0000 7fd77fdcf640 1 -- 172.21.15.16:0/856281257 shutdown_connections 2024-04-25T10:12:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.323+0000 7fd77fdcf640 1 --2- 172.21.15.16:0/856281257 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd76406b500 0x7fd76406d9c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.323+0000 7fd77fdcf640 1 --2- 172.21.15.16:0/856281257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7780cbf50 0x7fd7780c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.323+0000 7fd77fdcf640 1 --2- 172.21.15.16:0/856281257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7780cbaf0 0x7fd7780c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.323+0000 7fd77fdcf640 1 --2- 172.21.15.16:0/856281257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd760004d40 0x7fd7781f3430 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.323+0000 7fd77fdcf640 1 -- 172.21.15.16:0/856281257 >> 172.21.15.16:0/856281257 conn(0x7fd778072900 msgr2=0x7fd7780b50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.323+0000 7fd77fdcf640 1 -- 172.21.15.16:0/856281257 shutdown_connections 2024-04-25T10:12:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.323+0000 7fd77fdcf640 1 -- 172.21.15.16:0/856281257 wait complete. 2024-04-25T10:12:35.338 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.434+0000 7fce4c89e640 1 Processor -- start 2024-04-25T10:12:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.434+0000 7fce4c89e640 1 -- start start 2024-04-25T10:12:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.435+0000 7fce4c89e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce4815dce0 0x7fce480d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.435+0000 7fce4c89e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fce480b5b60 0x7fce480b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.435+0000 7fce4c89e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4815e0c0 con 0x7fce480d35b0 2024-04-25T10:12:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.435+0000 7fce4c89e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce480d2510 con 0x7fce4815dce0 2024-04-25T10:12:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.435+0000 7fce4c89e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce480d27f0 con 0x7fce480b5b60 2024-04-25T10:12:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.435+0000 7fce4659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce4815dce0 0x7fce480d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.435+0000 7fce45d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fce480d35b0 0x7fce480b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49142/0 (socket says 172.21.15.16:49142) 2024-04-25T10:12:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.435+0000 7fce45d9b640 1 -- 172.21.15.16:0/405274817 learned_addr learned my addr 172.21.15.16:0/405274817 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.436+0000 7fce46d9d640 1 --2- 172.21.15.16:0/405274817 >> v2:172.21.15.186:3300/0 conn(0x7fce480b5b60 0x7fce480b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.436+0000 7fce4659c640 1 -- 172.21.15.16:0/405274817 >> v2:172.21.15.186:3300/0 conn(0x7fce480b5b60 msgr2=0x7fce480b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.436+0000 7fce4659c640 1 --2- 172.21.15.16:0/405274817 >> v2:172.21.15.186:3300/0 conn(0x7fce480b5b60 0x7fce480b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.436+0000 7fce4659c640 1 -- 172.21.15.16:0/405274817 >> v1:172.21.15.16:6789/0 conn(0x7fce480d35b0 legacy=0x7fce480b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.436+0000 7fce4659c640 1 -- 172.21.15.16:0/405274817 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce480d2ad0 con 0x7fce4815dce0 2024-04-25T10:12:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.436+0000 7fce4559a640 1 -- 172.21.15.16:0/405274817 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2185305 0 0) 0x7fce4815e0c0 con 0x7fce480d35b0 2024-04-25T10:12:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.436+0000 7fce4659c640 1 --2- 172.21.15.16:0/405274817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4815dce0 0x7fce480d1a50 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fce480b3390 tx=0x7fce3802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b302a63119d1e1f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.436+0000 7fce4559a640 1 -- 172.21.15.16:0/405274817 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38007740 con 0x7fce4815dce0 2024-04-25T10:12:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.437+0000 7fce4559a640 1 -- 172.21.15.16:0/405274817 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce380078e0 con 0x7fce4815dce0 2024-04-25T10:12:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.437+0000 7fce4559a640 1 -- 172.21.15.16:0/405274817 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38007cb0 con 0x7fce4815dce0 2024-04-25T10:12:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.437+0000 7fce4c89e640 1 -- 172.21.15.16:0/405274817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4815dce0 msgr2=0x7fce480d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.437+0000 7fce4c89e640 1 --2- 172.21.15.16:0/405274817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4815dce0 0x7fce480d1a50 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fce480b3390 tx=0x7fce3802a680 comp rx=0 tx=0).stop 2024-04-25T10:12:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.437+0000 7fce4c89e640 1 -- 172.21.15.16:0/405274817 shutdown_connections 2024-04-25T10:12:35.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.437+0000 7fce4c89e640 1 --2- 172.21.15.16:0/405274817 >> v2:172.21.15.186:3300/0 conn(0x7fce480b5b60 0x7fce480b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.437+0000 7fce4c89e640 1 --2- 172.21.15.16:0/405274817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4815dce0 0x7fce480d1a50 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.437+0000 7fce4c89e640 1 -- 172.21.15.16:0/405274817 >> 172.21.15.16:0/405274817 conn(0x7fce48072900 msgr2=0x7fce480d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:35.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.437+0000 7fce4c89e640 1 -- 172.21.15.16:0/405274817 shutdown_connections 2024-04-25T10:12:35.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.437+0000 7fce4c89e640 1 -- 172.21.15.16:0/405274817 wait complete. 2024-04-25T10:12:35.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.438+0000 7fce4c89e640 1 Processor -- start 2024-04-25T10:12:35.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.438+0000 7fce4c89e640 1 -- start start 2024-04-25T10:12:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.439+0000 7fce4c89e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce480b5b60 0x7fce480c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.439+0000 7fce4c89e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce480d35b0 0x7fce48159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.439+0000 7fce4c89e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce4815dce0 0x7fce48159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.439+0000 7fce4c89e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce480b9820 con 0x7fce480b5b60 2024-04-25T10:12:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.439+0000 7fce4c89e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce480d3090 con 0x7fce4815dce0 2024-04-25T10:12:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.439+0000 7fce4c89e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce480b99a0 con 0x7fce480d35b0 2024-04-25T10:12:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.439+0000 7fce4659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce480b5b60 0x7fce480c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.439+0000 7fce4659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce480b5b60 0x7fce480c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60990/0 (socket says 172.21.15.16:60990) 2024-04-25T10:12:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.439+0000 7fce4659c640 1 -- 172.21.15.16:0/2534280857 learned_addr learned my addr 172.21.15.16:0/2534280857 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.439+0000 7fce45d9b640 1 --2- 172.21.15.16:0/2534280857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce480d35b0 0x7fce48159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.439+0000 7fce46d9d640 1 --2- 172.21.15.16:0/2534280857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce4815dce0 0x7fce48159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.440+0000 7fce4659c640 1 -- 172.21.15.16:0/2534280857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce480d35b0 msgr2=0x7fce48159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.440+0000 7fce4659c640 1 --2- 172.21.15.16:0/2534280857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce480d35b0 0x7fce48159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.440+0000 7fce4659c640 1 -- 172.21.15.16:0/2534280857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce4815dce0 msgr2=0x7fce48159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.440+0000 7fce4659c640 1 --2- 172.21.15.16:0/2534280857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce4815dce0 0x7fce48159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.440+0000 7fce4659c640 1 -- 172.21.15.16:0/2534280857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce480c9a70 con 0x7fce480b5b60 2024-04-25T10:12:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.440+0000 7fce45d9b640 1 --2- 172.21.15.16:0/2534280857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce480d35b0 0x7fce48159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.440+0000 7fce46d9d640 1 --2- 172.21.15.16:0/2534280857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce4815dce0 0x7fce48159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.440+0000 7fce4659c640 1 --2- 172.21.15.16:0/2534280857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce480b5b60 0x7fce480c9530 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fce38009e80 tx=0x7fce38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.440+0000 7fce3effd640 1 -- 172.21.15.16:0/2534280857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3803f070 con 0x7fce480b5b60 2024-04-25T10:12:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.440+0000 7fce3effd640 1 -- 172.21.15.16:0/2534280857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce38031c40 con 0x7fce480b5b60 2024-04-25T10:12:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.440+0000 7fce3effd640 1 -- 172.21.15.16:0/2534280857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38044460 con 0x7fce480b5b60 2024-04-25T10:12:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.440+0000 7fce4c89e640 1 -- 172.21.15.16:0/2534280857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4815a230 con 0x7fce480b5b60 2024-04-25T10:12:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.440+0000 7fce4c89e640 1 -- 172.21.15.16:0/2534280857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4815ef20 con 0x7fce480b5b60 2024-04-25T10:12:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.441+0000 7fce4c89e640 1 -- 172.21.15.16:0/2534280857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4815ed10 con 0x7fce480b5b60 2024-04-25T10:12:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.442+0000 7fce3effd640 1 -- 172.21.15.16:0/2534280857 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce3803b410 con 0x7fce480b5b60 2024-04-25T10:12:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.442+0000 7fce3effd640 1 --2- 172.21.15.16:0/2534280857 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fce2806b4d0 0x7fce2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.442+0000 7fce3effd640 1 -- 172.21.15.16:0/2534280857 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fce380abaa0 con 0x7fce480b5b60 2024-04-25T10:12:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.442+0000 7fce45d9b640 1 --2- 172.21.15.16:0/2534280857 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fce2806b4d0 0x7fce2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.443+0000 7fce45d9b640 1 --2- 172.21.15.16:0/2534280857 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fce2806b4d0 0x7fce2806d990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fce4815ec40 tx=0x7fce30007c10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.446+0000 7fce3effd640 1 -- 172.21.15.16:0/2534280857 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce480c9a70 con 0x7fce480b5b60 2024-04-25T10:12:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.629+0000 7fce4c89e640 1 -- 172.21.15.16:0/2534280857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce480749b0 con 0x7fce480b5b60 2024-04-25T10:12:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.631+0000 7fce3effd640 1 -- 172.21.15.16:0/2534280857 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16935 (secure 0 0 0) 0x7fce3807a900 con 0x7fce480b5b60 2024-04-25T10:12:35.635 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:35.636 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:11:55.231494+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:58.299405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:58.994953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.186:6836/3950732718":"2024-04-26T10:11:55.231471+0000","172.21.15.186:6834/3950732718":"2024-04-26T10:11:55.231471+0000","172.21.15.186:6837/3092291936":"2024-04-26T10:11:13.827348+0000","172.21.15.186:6835/3092291936":"2024-04-26T10:11:13.827348+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.633+0000 7fce4c89e640 1 -- 172.21.15.16:0/2534280857 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fce2806b4d0 msgr2=0x7fce2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.633+0000 7fce4c89e640 1 --2- 172.21.15.16:0/2534280857 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fce2806b4d0 0x7fce2806d990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fce4815ec40 tx=0x7fce30007c10 comp rx=0 tx=0).stop 2024-04-25T10:12:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.633+0000 7fce4c89e640 1 -- 172.21.15.16:0/2534280857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce480b5b60 msgr2=0x7fce480c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.633+0000 7fce4c89e640 1 --2- 172.21.15.16:0/2534280857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce480b5b60 0x7fce480c9530 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fce38009e80 tx=0x7fce38004a40 comp rx=0 tx=0).stop 2024-04-25T10:12:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.633+0000 7fce4c89e640 1 -- 172.21.15.16:0/2534280857 shutdown_connections 2024-04-25T10:12:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.633+0000 7fce4c89e640 1 --2- 172.21.15.16:0/2534280857 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fce2806b4d0 0x7fce2806d990 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.633+0000 7fce4c89e640 1 --2- 172.21.15.16:0/2534280857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce4815dce0 0x7fce48159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.633+0000 7fce4c89e640 1 --2- 172.21.15.16:0/2534280857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce480d35b0 0x7fce48159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.633+0000 7fce4c89e640 1 --2- 172.21.15.16:0/2534280857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce480b5b60 0x7fce480c9530 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.633+0000 7fce4c89e640 1 -- 172.21.15.16:0/2534280857 >> 172.21.15.16:0/2534280857 conn(0x7fce48072900 msgr2=0x7fce480b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.633+0000 7fce4c89e640 1 -- 172.21.15.16:0/2534280857 shutdown_connections 2024-04-25T10:12:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.633+0000 7fce4c89e640 1 -- 172.21.15.16:0/2534280857 wait complete. 2024-04-25T10:12:35.648 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:10:58.299405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:10:58.994953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:35.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:35.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.749+0000 7f78c76bd640 1 Processor -- start 2024-04-25T10:12:35.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.749+0000 7f78c76bd640 1 -- start start 2024-04-25T10:12:35.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.750+0000 7f78c76bd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78c0144e70 0x7f78c006d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.750+0000 7f78c76bd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c006dcb0 con 0x7f78c006d7a0 2024-04-25T10:12:35.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.750+0000 7f78c76bd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c006df90 con 0x7f78c00cf0e0 2024-04-25T10:12:35.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.750+0000 7f78c76bd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c015bfe0 con 0x7f78c0144e70 2024-04-25T10:12:35.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.750+0000 7f78c4c58640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f78c006d7a0 0x7f78c00ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49154/0 (socket says 172.21.15.16:49154) 2024-04-25T10:12:35.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.750+0000 7f78c4c58640 1 -- 172.21.15.16:0/2501703384 learned_addr learned my addr 172.21.15.16:0/2501703384 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:35.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78c5459640 1 --2- 172.21.15.16:0/2501703384 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78c0144e70 0x7f78c006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78b7fff640 1 -- 172.21.15.16:0/2501703384 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202439427 0 0) 0x7f78c006dcb0 con 0x7f78c006d7a0 2024-04-25T10:12:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78b7fff640 1 -- 172.21.15.16:0/2501703384 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78ac003520 con 0x7f78c006d7a0 2024-04-25T10:12:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78b7fff640 1 -- 172.21.15.16:0/2501703384 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435826245 0 0) 0x7f78c006df90 con 0x7f78c00cf0e0 2024-04-25T10:12:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78b7fff640 1 -- 172.21.15.16:0/2501703384 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78c006dcb0 con 0x7f78c00cf0e0 2024-04-25T10:12:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78b7fff640 1 -- 172.21.15.16:0/2501703384 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3554630950 0 0) 0x7f78ac003520 con 0x7f78c006d7a0 2024-04-25T10:12:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78b7fff640 1 -- 172.21.15.16:0/2501703384 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78c0144e70 msgr2=0x7f78c006d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78b7fff640 1 --2- 172.21.15.16:0/2501703384 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78c0144e70 0x7f78c006d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78b7fff640 1 -- 172.21.15.16:0/2501703384 >> v1:172.21.15.186:6789/0 conn(0x7f78c00cf0e0 legacy=0x7f78c00cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78b7fff640 1 -- 172.21.15.16:0/2501703384 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c006e270 con 0x7f78c006d7a0 2024-04-25T10:12:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78b7fff640 1 -- 172.21.15.16:0/2501703384 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f78b0001690 con 0x7f78c006d7a0 2024-04-25T10:12:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78b7fff640 1 -- 172.21.15.16:0/2501703384 >> v1:172.21.15.16:6789/0 conn(0x7f78c006d7a0 legacy=0x7f78c00ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78b7fff640 1 --2- 172.21.15.16:0/2501703384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ac003b70 0x7f78ac003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78b7fff640 1 --2- 172.21.15.16:0/2501703384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78ac004800 0x7f78ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78b7fff640 1 --2- 172.21.15.16:0/2501703384 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78c0144e70 0x7f78ac0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78b7fff640 1 -- 172.21.15.16:0/2501703384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c015bfe0 con 0x7f78ac003b70 2024-04-25T10:12:35.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78b7fff640 1 -- 172.21.15.16:0/2501703384 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac00bad0 con 0x7f78ac004800 2024-04-25T10:12:35.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78b7fff640 1 -- 172.21.15.16:0/2501703384 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac00bcd0 con 0x7f78c0144e70 2024-04-25T10:12:35.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78c5459640 1 --2- 172.21.15.16:0/2501703384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ac003b70 0x7f78ac003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78c4c58640 1 --2- 172.21.15.16:0/2501703384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78ac004800 0x7f78ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.751+0000 7f78c5c5a640 1 --2- 172.21.15.16:0/2501703384 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78c0144e70 0x7f78ac0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.752+0000 7f78c5459640 1 -- 172.21.15.16:0/2501703384 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78c0144e70 msgr2=0x7f78ac0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.752+0000 7f78c5459640 1 --2- 172.21.15.16:0/2501703384 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78c0144e70 0x7f78ac0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.752+0000 7f78c5459640 1 -- 172.21.15.16:0/2501703384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78ac004800 msgr2=0x7f78ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.752+0000 7f78c5459640 1 --2- 172.21.15.16:0/2501703384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78ac004800 0x7f78ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.752+0000 7f78c5459640 1 -- 172.21.15.16:0/2501703384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac003520 con 0x7f78ac003b70 2024-04-25T10:12:35.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.752+0000 7f78c5c5a640 1 --2- 172.21.15.16:0/2501703384 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78c0144e70 0x7f78ac0051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:35.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.752+0000 7f78c5459640 1 --2- 172.21.15.16:0/2501703384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ac003b70 0x7f78ac003f70 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009ac0 tx=0x7f78ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75164473bd78e5e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.752+0000 7f78b7fff640 1 -- 172.21.15.16:0/2501703384 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8036070 con 0x7f78ac003b70 2024-04-25T10:12:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.752+0000 7f78b7fff640 1 -- 172.21.15.16:0/2501703384 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b8031400 con 0x7f78ac003b70 2024-04-25T10:12:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.752+0000 7f78b7fff640 1 -- 172.21.15.16:0/2501703384 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8009690 con 0x7f78ac003b70 2024-04-25T10:12:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.753+0000 7f78c76bd640 1 -- 172.21.15.16:0/2501703384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ac003b70 msgr2=0x7f78ac003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.753+0000 7f78c76bd640 1 --2- 172.21.15.16:0/2501703384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ac003b70 0x7f78ac003f70 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009ac0 tx=0x7f78ac005830 comp rx=0 tx=0).stop 2024-04-25T10:12:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.753+0000 7f78c5c5a640 1 -- 172.21.15.16:0/2501703384 reap_dead start 2024-04-25T10:12:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.753+0000 7f78c76bd640 1 -- 172.21.15.16:0/2501703384 shutdown_connections 2024-04-25T10:12:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.753+0000 7f78c76bd640 1 -- 172.21.15.16:0/2501703384 >> 172.21.15.16:0/2501703384 conn(0x7f78c006aba0 msgr2=0x7f78c006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.753+0000 7f78c76bd640 1 -- 172.21.15.16:0/2501703384 shutdown_connections 2024-04-25T10:12:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.753+0000 7f78c76bd640 1 -- 172.21.15.16:0/2501703384 wait complete. 2024-04-25T10:12:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.754+0000 7f78c76bd640 1 Processor -- start 2024-04-25T10:12:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.754+0000 7f78c76bd640 1 -- start start 2024-04-25T10:12:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.755+0000 7f78c76bd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78ac003b70 0x7f78c00b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.755+0000 7f78c76bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c015b470 0x7f78c013acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.755+0000 7f78c76bd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78c013c620 0x7f78c015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.755+0000 7f78c76bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c0145630 con 0x7f78c015b470 2024-04-25T10:12:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.755+0000 7f78c76bd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c01454b0 con 0x7f78c013c620 2024-04-25T10:12:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.755+0000 7f78c76bd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c006d260 con 0x7f78ac003b70 2024-04-25T10:12:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.755+0000 7f78c4c58640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c015b470 0x7f78c013acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.755+0000 7f78c4c58640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c015b470 0x7f78c013acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32778/0 (socket says 172.21.15.16:32778) 2024-04-25T10:12:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.755+0000 7f78c4c58640 1 -- 172.21.15.16:0/4134443812 learned_addr learned my addr 172.21.15.16:0/4134443812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.755+0000 7f78c5c5a640 1 --2- 172.21.15.16:0/4134443812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78c013c620 0x7f78c015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.755+0000 7f78c5459640 1 --2- 172.21.15.16:0/4134443812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78ac003b70 0x7f78c00b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.755+0000 7f78c4c58640 1 -- 172.21.15.16:0/4134443812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78ac003b70 msgr2=0x7f78c00b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.755+0000 7f78c4c58640 1 --2- 172.21.15.16:0/4134443812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78ac003b70 0x7f78c00b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.755+0000 7f78c4c58640 1 -- 172.21.15.16:0/4134443812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78c013c620 msgr2=0x7f78c015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.755+0000 7f78c4c58640 1 --2- 172.21.15.16:0/4134443812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78c013c620 0x7f78c015e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.755+0000 7f78c4c58640 1 -- 172.21.15.16:0/4134443812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c013b580 con 0x7f78c015b470 2024-04-25T10:12:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.756+0000 7f78c4c58640 1 --2- 172.21.15.16:0/4134443812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c015b470 0x7f78c013acf0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f78b0004f80 tx=0x7f78b00074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.756+0000 7f78b5ffb640 1 -- 172.21.15.16:0/4134443812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b0007740 con 0x7f78c015b470 2024-04-25T10:12:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.756+0000 7f78b5ffb640 1 -- 172.21.15.16:0/4134443812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b0018070 con 0x7f78c015b470 2024-04-25T10:12:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.756+0000 7f78c76bd640 1 -- 172.21.15.16:0/4134443812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c013b860 con 0x7f78c015b470 2024-04-25T10:12:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.756+0000 7f78b5ffb640 1 -- 172.21.15.16:0/4134443812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b000fbe0 con 0x7f78c015b470 2024-04-25T10:12:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.756+0000 7f78c76bd640 1 -- 172.21.15.16:0/4134443812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c013be20 con 0x7f78c015b470 2024-04-25T10:12:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.757+0000 7f78c76bd640 1 -- 172.21.15.16:0/4134443812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c00b2b50 con 0x7f78c015b470 2024-04-25T10:12:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.757+0000 7f78b5ffb640 1 -- 172.21.15.16:0/4134443812 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78b001d460 con 0x7f78c015b470 2024-04-25T10:12:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.758+0000 7f78b5ffb640 1 --2- 172.21.15.16:0/4134443812 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f78a406b6a0 0x7f78a406db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.758+0000 7f78b5ffb640 1 -- 172.21.15.16:0/4134443812 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f78b008a210 con 0x7f78c015b470 2024-04-25T10:12:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.758+0000 7f78c5459640 1 --2- 172.21.15.16:0/4134443812 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f78a406b6a0 0x7f78a406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.759+0000 7f78c5459640 1 --2- 172.21.15.16:0/4134443812 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f78a406b6a0 0x7f78a406db60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f78b8033000 tx=0x7f78b8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.762+0000 7f78b5ffb640 1 -- 172.21.15.16:0/4134443812 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78c00b2b50 con 0x7f78c015b470 2024-04-25T10:12:35.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.942+0000 7f78c76bd640 1 -- 172.21.15.16:0/4134443812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f78c00750b0 con 0x7f78c015b470 2024-04-25T10:12:35.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.942+0000 7f78b5ffb640 1 -- 172.21.15.16:0/4134443812 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f78b0059070 con 0x7f78c015b470 2024-04-25T10:12:35.947 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:12:35.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.944+0000 7f78c76bd640 1 -- 172.21.15.16:0/4134443812 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f78a406b6a0 msgr2=0x7f78a406db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.944+0000 7f78c76bd640 1 --2- 172.21.15.16:0/4134443812 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f78a406b6a0 0x7f78a406db60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f78b8033000 tx=0x7f78b8008030 comp rx=0 tx=0).stop 2024-04-25T10:12:35.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.944+0000 7f78c76bd640 1 -- 172.21.15.16:0/4134443812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c015b470 msgr2=0x7f78c013acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.944+0000 7f78c76bd640 1 --2- 172.21.15.16:0/4134443812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c015b470 0x7f78c013acf0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f78b0004f80 tx=0x7f78b00074f0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.944+0000 7f78c76bd640 1 -- 172.21.15.16:0/4134443812 shutdown_connections 2024-04-25T10:12:35.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.944+0000 7f78c76bd640 1 --2- 172.21.15.16:0/4134443812 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f78a406b6a0 0x7f78a406db60 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.944+0000 7f78c76bd640 1 --2- 172.21.15.16:0/4134443812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78c013c620 0x7f78c015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.944+0000 7f78c76bd640 1 --2- 172.21.15.16:0/4134443812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c015b470 0x7f78c013acf0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.944+0000 7f78c76bd640 1 --2- 172.21.15.16:0/4134443812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78ac003b70 0x7f78c00b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.944+0000 7f78c76bd640 1 -- 172.21.15.16:0/4134443812 >> 172.21.15.16:0/4134443812 conn(0x7f78c006aba0 msgr2=0x7f78c00b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:35.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.944+0000 7f78c76bd640 1 -- 172.21.15.16:0/4134443812 shutdown_connections 2024-04-25T10:12:35.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:35.944+0000 7f78c76bd640 1 -- 172.21.15.16:0/4134443812 wait complete. 2024-04-25T10:12:35.959 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.057+0000 7f0bf7688640 1 Processor -- start 2024-04-25T10:12:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.057+0000 7f0bf7688640 1 -- start start 2024-04-25T10:12:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.058+0000 7f0bf7688640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0bf00d3b90 0x7f0bf00d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.058+0000 7f0bf7688640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bf00d0f60 0x7f0bf00b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.058+0000 7f0bf7688640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf00d4090 con 0x7f0bf015dca0 2024-04-25T10:12:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.058+0000 7f0bf7688640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf00b6b20 con 0x7f0bf00d3b90 2024-04-25T10:12:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.058+0000 7f0bf7688640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf00d1390 con 0x7f0bf00d0f60 2024-04-25T10:12:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.058+0000 7f0bf5424640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0bf015dca0 0x7f0bf00ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49166/0 (socket says 172.21.15.16:49166) 2024-04-25T10:12:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.058+0000 7f0bf5424640 1 -- 172.21.15.16:0/2546837570 learned_addr learned my addr 172.21.15.16:0/2546837570 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.058+0000 7f0bf4c23640 1 --2- 172.21.15.16:0/2546837570 >> v2:172.21.15.16:3300/0 conn(0x7f0bf00d3b90 0x7f0bf00d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.058+0000 7f0bf5c25640 1 --2- 172.21.15.16:0/2546837570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bf00d0f60 0x7f0bf00b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.058+0000 7f0be7fff640 1 -- 172.21.15.16:0/2546837570 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124794040 0 0) 0x7f0bf00d4090 con 0x7f0bf015dca0 2024-04-25T10:12:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.058+0000 7f0be7fff640 1 -- 172.21.15.16:0/2546837570 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bdc003520 con 0x7f0bf015dca0 2024-04-25T10:12:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.058+0000 7f0bf4c23640 1 -- 172.21.15.16:0/2546837570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bf00d0f60 msgr2=0x7f0bf00b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.058+0000 7f0bf4c23640 1 --2- 172.21.15.16:0/2546837570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bf00d0f60 0x7f0bf00b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.058+0000 7f0bf4c23640 1 -- 172.21.15.16:0/2546837570 >> v1:172.21.15.16:6789/0 conn(0x7f0bf015dca0 legacy=0x7f0bf00ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.058+0000 7f0bf4c23640 1 -- 172.21.15.16:0/2546837570 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf00d4370 con 0x7f0bf00d3b90 2024-04-25T10:12:36.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.059+0000 7f0bf5c25640 1 --2- 172.21.15.16:0/2546837570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bf00d0f60 0x7f0bf00b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:36.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.059+0000 7f0bf4c23640 1 --2- 172.21.15.16:0/2546837570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bf00d3b90 0x7f0bf00d0a20 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f0be00097d0 tx=0x7f0be002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6544f3358ea0e32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.059+0000 7f0be7fff640 1 -- 172.21.15.16:0/2546837570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0037070 con 0x7f0bf00d3b90 2024-04-25T10:12:36.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.059+0000 7f0be7fff640 1 -- 172.21.15.16:0/2546837570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be0032440 con 0x7f0bf00d3b90 2024-04-25T10:12:36.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.059+0000 7f0be7fff640 1 -- 172.21.15.16:0/2546837570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0032810 con 0x7f0bf00d3b90 2024-04-25T10:12:36.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.060+0000 7f0bf7688640 1 -- 172.21.15.16:0/2546837570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bf00d3b90 msgr2=0x7f0bf00d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.060+0000 7f0bf7688640 1 --2- 172.21.15.16:0/2546837570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bf00d3b90 0x7f0bf00d0a20 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f0be00097d0 tx=0x7f0be002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.060+0000 7f0bf7688640 1 -- 172.21.15.16:0/2546837570 shutdown_connections 2024-04-25T10:12:36.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.060+0000 7f0bf7688640 1 --2- 172.21.15.16:0/2546837570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bf00d0f60 0x7f0bf00b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.060+0000 7f0bf7688640 1 --2- 172.21.15.16:0/2546837570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bf00d3b90 0x7f0bf00d0a20 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.060+0000 7f0bf7688640 1 -- 172.21.15.16:0/2546837570 >> 172.21.15.16:0/2546837570 conn(0x7f0bf00726c0 msgr2=0x7f0bf0072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.060+0000 7f0bf7688640 1 -- 172.21.15.16:0/2546837570 shutdown_connections 2024-04-25T10:12:36.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.060+0000 7f0bf7688640 1 -- 172.21.15.16:0/2546837570 wait complete. 2024-04-25T10:12:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.061+0000 7f0bf7688640 1 Processor -- start 2024-04-25T10:12:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.061+0000 7f0bf7688640 1 -- start start 2024-04-25T10:12:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf7688640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bf00d0f60 0x7f0bf0150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf7688640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bf00d3b90 0x7f0bf00cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf7688640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bf015dca0 0x7f0bf015b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf7688640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf00cf050 con 0x7f0bf00d3b90 2024-04-25T10:12:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf7688640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf00b6d50 con 0x7f0bf015dca0 2024-04-25T10:12:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf7688640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf015e080 con 0x7f0bf00d0f60 2024-04-25T10:12:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf4c23640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bf00d3b90 0x7f0bf00cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf4c23640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bf00d3b90 0x7f0bf00cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32792/0 (socket says 172.21.15.16:32792) 2024-04-25T10:12:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf4c23640 1 -- 172.21.15.16:0/2939965758 learned_addr learned my addr 172.21.15.16:0/2939965758 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf4c23640 1 -- 172.21.15.16:0/2939965758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bf00d0f60 msgr2=0x7f0bf0150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf5424640 1 --2- 172.21.15.16:0/2939965758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bf00d0f60 0x7f0bf0150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf5c25640 1 --2- 172.21.15.16:0/2939965758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bf015dca0 0x7f0bf015b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf4c23640 1 --2- 172.21.15.16:0/2939965758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bf00d0f60 0x7f0bf0150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf4c23640 1 -- 172.21.15.16:0/2939965758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bf015dca0 msgr2=0x7f0bf015b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf4c23640 1 --2- 172.21.15.16:0/2939965758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bf015dca0 0x7f0bf015b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf4c23640 1 -- 172.21.15.16:0/2939965758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf0150da0 con 0x7f0bf00d3b90 2024-04-25T10:12:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf5424640 1 --2- 172.21.15.16:0/2939965758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bf00d0f60 0x7f0bf0150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf5c25640 1 --2- 172.21.15.16:0/2939965758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bf015dca0 0x7f0bf015b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.062+0000 7f0bf4c23640 1 --2- 172.21.15.16:0/2939965758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bf00d3b90 0x7f0bf00cab60 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f0be0035040 tx=0x7f0be00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.063+0000 7f0be5ffb640 1 -- 172.21.15.16:0/2939965758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0037040 con 0x7f0bf00d3b90 2024-04-25T10:12:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.063+0000 7f0be5ffb640 1 -- 172.21.15.16:0/2939965758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be0030500 con 0x7f0bf00d3b90 2024-04-25T10:12:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.063+0000 7f0bf7688640 1 -- 172.21.15.16:0/2939965758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf00cb540 con 0x7f0bf00d3b90 2024-04-25T10:12:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.063+0000 7f0be5ffb640 1 -- 172.21.15.16:0/2939965758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be00308f0 con 0x7f0bf00d3b90 2024-04-25T10:12:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.063+0000 7f0bf7688640 1 -- 172.21.15.16:0/2939965758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf00cc050 con 0x7f0bf00d3b90 2024-04-25T10:12:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.064+0000 7f0bf7688640 1 -- 172.21.15.16:0/2939965758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf00750b0 con 0x7f0bf00d3b90 2024-04-25T10:12:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.064+0000 7f0be5ffb640 1 -- 172.21.15.16:0/2939965758 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0be0044020 con 0x7f0bf00d3b90 2024-04-25T10:12:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.065+0000 7f0be5ffb640 1 --2- 172.21.15.16:0/2939965758 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0bd406b4d0 0x7f0bd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.065+0000 7f0be5ffb640 1 -- 172.21.15.16:0/2939965758 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f0be00aa530 con 0x7f0bf00d3b90 2024-04-25T10:12:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.065+0000 7f0bf5424640 1 --2- 172.21.15.16:0/2939965758 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0bd406b4d0 0x7f0bd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.065+0000 7f0bf5424640 1 --2- 172.21.15.16:0/2939965758 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0bd406b4d0 0x7f0bd406d990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0bec005880 tx=0x7f0bec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.069+0000 7f0be5ffb640 1 -- 172.21.15.16:0/2939965758 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0be0079300 con 0x7f0bf00d3b90 2024-04-25T10:12:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.251+0000 7f0bf7688640 1 -- 172.21.15.16:0/2939965758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0bf006b1d0 con 0x7f0bf00d3b90 2024-04-25T10:12:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.251+0000 7f0be5ffb640 1 -- 172.21.15.16:0/2939965758 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0be0078c00 con 0x7f0bf00d3b90 2024-04-25T10:12:36.256 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:12:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.253+0000 7f0bf7688640 1 -- 172.21.15.16:0/2939965758 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0bd406b4d0 msgr2=0x7f0bd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.253+0000 7f0bf7688640 1 --2- 172.21.15.16:0/2939965758 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0bd406b4d0 0x7f0bd406d990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0bec005880 tx=0x7f0bec009040 comp rx=0 tx=0).stop 2024-04-25T10:12:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.253+0000 7f0bf7688640 1 -- 172.21.15.16:0/2939965758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bf00d3b90 msgr2=0x7f0bf00cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.253+0000 7f0bf7688640 1 --2- 172.21.15.16:0/2939965758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bf00d3b90 0x7f0bf00cab60 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f0be0035040 tx=0x7f0be00046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.253+0000 7f0bf7688640 1 -- 172.21.15.16:0/2939965758 shutdown_connections 2024-04-25T10:12:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.253+0000 7f0bf7688640 1 --2- 172.21.15.16:0/2939965758 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0bd406b4d0 0x7f0bd406d990 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.253+0000 7f0bf7688640 1 --2- 172.21.15.16:0/2939965758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bf015dca0 0x7f0bf015b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.253+0000 7f0bf7688640 1 --2- 172.21.15.16:0/2939965758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bf00d3b90 0x7f0bf00cab60 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.253+0000 7f0bf7688640 1 --2- 172.21.15.16:0/2939965758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bf00d0f60 0x7f0bf0150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.253+0000 7f0bf7688640 1 -- 172.21.15.16:0/2939965758 >> 172.21.15.16:0/2939965758 conn(0x7f0bf00726c0 msgr2=0x7f0bf00b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.253+0000 7f0bf7688640 1 -- 172.21.15.16:0/2939965758 shutdown_connections 2024-04-25T10:12:36.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.253+0000 7f0bf7688640 1 -- 172.21.15.16:0/2939965758 wait complete. 2024-04-25T10:12:36.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:36.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.368+0000 7fee54fa5640 1 Processor -- start 2024-04-25T10:12:36.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.368+0000 7fee54fa5640 1 -- start start 2024-04-25T10:12:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee54fa5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee5015dca0 0x7fee500ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee54fa5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee500d4090 con 0x7fee500d1130 2024-04-25T10:12:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee54fa5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee500d4370 con 0x7fee500d3b90 2024-04-25T10:12:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee54fa5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee500b6b20 con 0x7fee5015dca0 2024-04-25T10:12:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4ed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fee500d1130 0x7fee500b2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49176/0 (socket says 172.21.15.16:49176) 2024-04-25T10:12:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4ed9d640 1 -- 172.21.15.16:0/4035094210 learned_addr learned my addr 172.21.15.16:0/4035094210 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4e59c640 1 --2- 172.21.15.16:0/4035094210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee5015dca0 0x7fee500ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4d59a640 1 -- 172.21.15.16:0/4035094210 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1976834264 0 0) 0x7fee500d4090 con 0x7fee500d1130 2024-04-25T10:12:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4d59a640 1 -- 172.21.15.16:0/4035094210 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee38003520 con 0x7fee500d1130 2024-04-25T10:12:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4d59a640 1 -- 172.21.15.16:0/4035094210 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742380596 0 0) 0x7fee500d4370 con 0x7fee500d3b90 2024-04-25T10:12:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4d59a640 1 -- 172.21.15.16:0/4035094210 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee500d4090 con 0x7fee500d3b90 2024-04-25T10:12:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4d59a640 1 -- 172.21.15.16:0/4035094210 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2006690350 0 0) 0x7fee38003520 con 0x7fee500d1130 2024-04-25T10:12:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4d59a640 1 -- 172.21.15.16:0/4035094210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee5015dca0 msgr2=0x7fee500ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4d59a640 1 --2- 172.21.15.16:0/4035094210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee5015dca0 0x7fee500ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4d59a640 1 -- 172.21.15.16:0/4035094210 >> v1:172.21.15.186:6789/0 conn(0x7fee500d3b90 legacy=0x7fee500d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4d59a640 1 -- 172.21.15.16:0/4035094210 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee500d4650 con 0x7fee500d1130 2024-04-25T10:12:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4d59a640 1 -- 172.21.15.16:0/4035094210 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fee480019f0 con 0x7fee500d1130 2024-04-25T10:12:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4d59a640 1 -- 172.21.15.16:0/4035094210 >> v1:172.21.15.16:6789/0 conn(0x7fee500d1130 legacy=0x7fee500b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4d59a640 1 --2- 172.21.15.16:0/4035094210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee5015dca0 0x7fee38003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4d59a640 1 --2- 172.21.15.16:0/4035094210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee38004120 0x7fee38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4d59a640 1 --2- 172.21.15.16:0/4035094210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee38004d40 0x7fee380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4d59a640 1 -- 172.21.15.16:0/4035094210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3800bad0 con 0x7fee38004120 2024-04-25T10:12:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4d59a640 1 -- 172.21.15.16:0/4035094210 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3800bc80 con 0x7fee38004d40 2024-04-25T10:12:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.369+0000 7fee4d59a640 1 -- 172.21.15.16:0/4035094210 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3800be80 con 0x7fee5015dca0 2024-04-25T10:12:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.370+0000 7fee4dd9b640 1 --2- 172.21.15.16:0/4035094210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee38004120 0x7fee38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.370+0000 7fee4e59c640 1 --2- 172.21.15.16:0/4035094210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee5015dca0 0x7fee38003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.370+0000 7fee4ed9d640 1 --2- 172.21.15.16:0/4035094210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee38004d40 0x7fee380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.370+0000 7fee4dd9b640 1 -- 172.21.15.16:0/4035094210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee5015dca0 msgr2=0x7fee38003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.370+0000 7fee4dd9b640 1 --2- 172.21.15.16:0/4035094210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee5015dca0 0x7fee38003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.370+0000 7fee4dd9b640 1 -- 172.21.15.16:0/4035094210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee38004d40 msgr2=0x7fee380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.370+0000 7fee4dd9b640 1 --2- 172.21.15.16:0/4035094210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee38004d40 0x7fee380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.370+0000 7fee4dd9b640 1 -- 172.21.15.16:0/4035094210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee38003520 con 0x7fee38004120 2024-04-25T10:12:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.371+0000 7fee4dd9b640 1 --2- 172.21.15.16:0/4035094210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee38004120 0x7fee38004800 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fee440018d0 tx=0x7fee38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1a27de35f6d2c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.371+0000 7fee4d59a640 1 -- 172.21.15.16:0/4035094210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee440014f0 con 0x7fee38004120 2024-04-25T10:12:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.371+0000 7fee4d59a640 1 -- 172.21.15.16:0/4035094210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee440052d0 con 0x7fee38004120 2024-04-25T10:12:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.371+0000 7fee4d59a640 1 -- 172.21.15.16:0/4035094210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4402abe0 con 0x7fee38004120 2024-04-25T10:12:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.371+0000 7fee54fa5640 1 -- 172.21.15.16:0/4035094210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee38004120 msgr2=0x7fee38004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.371+0000 7fee54fa5640 1 --2- 172.21.15.16:0/4035094210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee38004120 0x7fee38004800 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fee440018d0 tx=0x7fee38005830 comp rx=0 tx=0).stop 2024-04-25T10:12:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.371+0000 7fee4ed9d640 1 -- 172.21.15.16:0/4035094210 reap_dead start 2024-04-25T10:12:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.371+0000 7fee54fa5640 1 -- 172.21.15.16:0/4035094210 shutdown_connections 2024-04-25T10:12:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.371+0000 7fee54fa5640 1 -- 172.21.15.16:0/4035094210 >> 172.21.15.16:0/4035094210 conn(0x7fee500726c0 msgr2=0x7fee50072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.371+0000 7fee54fa5640 1 -- 172.21.15.16:0/4035094210 shutdown_connections 2024-04-25T10:12:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.371+0000 7fee54fa5640 1 -- 172.21.15.16:0/4035094210 wait complete. 2024-04-25T10:12:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.373+0000 7fee54fa5640 1 Processor -- start 2024-04-25T10:12:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.373+0000 7fee54fa5640 1 -- start start 2024-04-25T10:12:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.373+0000 7fee54fa5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee38004120 0x7fee501f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.373+0000 7fee54fa5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee5014de40 0x7fee5014e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.373+0000 7fee54fa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5014e2a0 0x7fee5014ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.373+0000 7fee54fa5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5015e200 con 0x7fee5014e2a0 2024-04-25T10:12:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.373+0000 7fee54fa5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5015e080 con 0x7fee38004120 2024-04-25T10:12:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.373+0000 7fee54fa5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee500b8280 con 0x7fee5014de40 2024-04-25T10:12:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.373+0000 7fee4ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5014e2a0 0x7fee5014ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.373+0000 7fee4ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5014e2a0 0x7fee5014ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32812/0 (socket says 172.21.15.16:32812) 2024-04-25T10:12:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.374+0000 7fee4ed9d640 1 -- 172.21.15.16:0/2792336994 learned_addr learned my addr 172.21.15.16:0/2792336994 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.374+0000 7fee4e59c640 1 --2- 172.21.15.16:0/2792336994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee38004120 0x7fee501f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.374+0000 7fee4dd9b640 1 --2- 172.21.15.16:0/2792336994 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee5014de40 0x7fee5014e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.374+0000 7fee4ed9d640 1 -- 172.21.15.16:0/2792336994 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee5014de40 msgr2=0x7fee5014e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.374+0000 7fee4ed9d640 1 --2- 172.21.15.16:0/2792336994 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee5014de40 0x7fee5014e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.374+0000 7fee4ed9d640 1 -- 172.21.15.16:0/2792336994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee38004120 msgr2=0x7fee501f3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.374+0000 7fee4ed9d640 1 --2- 172.21.15.16:0/2792336994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee38004120 0x7fee501f3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.374+0000 7fee4ed9d640 1 -- 172.21.15.16:0/2792336994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5015f5a0 con 0x7fee5014e2a0 2024-04-25T10:12:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.374+0000 7fee4ed9d640 1 --2- 172.21.15.16:0/2792336994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5014e2a0 0x7fee5014ec50 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fee500b6580 tx=0x7fee48007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.374+0000 7fee3effd640 1 -- 172.21.15.16:0/2792336994 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4800a9f0 con 0x7fee5014e2a0 2024-04-25T10:12:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.374+0000 7fee3effd640 1 -- 172.21.15.16:0/2792336994 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee48019070 con 0x7fee5014e2a0 2024-04-25T10:12:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.374+0000 7fee54fa5640 1 -- 172.21.15.16:0/2792336994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5015f880 con 0x7fee5014e2a0 2024-04-25T10:12:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.374+0000 7fee3effd640 1 -- 172.21.15.16:0/2792336994 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4800ae10 con 0x7fee5014e2a0 2024-04-25T10:12:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.374+0000 7fee54fa5640 1 -- 172.21.15.16:0/2792336994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5015fe40 con 0x7fee5014e2a0 2024-04-25T10:12:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.375+0000 7fee54fa5640 1 -- 172.21.15.16:0/2792336994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee50160040 con 0x7fee5014e2a0 2024-04-25T10:12:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.376+0000 7fee3effd640 1 -- 172.21.15.16:0/2792336994 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee4801e460 con 0x7fee5014e2a0 2024-04-25T10:12:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.376+0000 7fee3effd640 1 --2- 172.21.15.16:0/2792336994 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fee3006b500 0x7fee3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.376+0000 7fee3effd640 1 -- 172.21.15.16:0/2792336994 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fee4808a590 con 0x7fee5014e2a0 2024-04-25T10:12:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.376+0000 7fee4e59c640 1 --2- 172.21.15.16:0/2792336994 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fee3006b500 0x7fee3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.377+0000 7fee4e59c640 1 --2- 172.21.15.16:0/2792336994 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fee3006b500 0x7fee3006d9c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fee40009ae0 tx=0x7fee40009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.380+0000 7fee3effd640 1 -- 172.21.15.16:0/2792336994 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee50160040 con 0x7fee5014e2a0 2024-04-25T10:12:36.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.559+0000 7fee54fa5640 1 -- 172.21.15.16:0/2792336994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fee500750b0 con 0x7fee5014e2a0 2024-04-25T10:12:36.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.560+0000 7fee3effd640 1 -- 172.21.15.16:0/2792336994 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fee48059360 con 0x7fee5014e2a0 2024-04-25T10:12:36.564 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:12:36.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.561+0000 7fee54fa5640 1 -- 172.21.15.16:0/2792336994 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fee3006b500 msgr2=0x7fee3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.561+0000 7fee54fa5640 1 --2- 172.21.15.16:0/2792336994 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fee3006b500 0x7fee3006d9c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fee40009ae0 tx=0x7fee40009190 comp rx=0 tx=0).stop 2024-04-25T10:12:36.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.561+0000 7fee54fa5640 1 -- 172.21.15.16:0/2792336994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5014e2a0 msgr2=0x7fee5014ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.561+0000 7fee54fa5640 1 --2- 172.21.15.16:0/2792336994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5014e2a0 0x7fee5014ec50 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fee500b6580 tx=0x7fee48007b10 comp rx=0 tx=0).stop 2024-04-25T10:12:36.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.561+0000 7fee54fa5640 1 -- 172.21.15.16:0/2792336994 shutdown_connections 2024-04-25T10:12:36.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.561+0000 7fee54fa5640 1 --2- 172.21.15.16:0/2792336994 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fee3006b500 0x7fee3006d9c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.561+0000 7fee54fa5640 1 --2- 172.21.15.16:0/2792336994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5014e2a0 0x7fee5014ec50 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.561+0000 7fee54fa5640 1 --2- 172.21.15.16:0/2792336994 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee5014de40 0x7fee5014e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.561+0000 7fee54fa5640 1 --2- 172.21.15.16:0/2792336994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee38004120 0x7fee501f3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.561+0000 7fee54fa5640 1 -- 172.21.15.16:0/2792336994 >> 172.21.15.16:0/2792336994 conn(0x7fee500726c0 msgr2=0x7fee500d2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.561+0000 7fee54fa5640 1 -- 172.21.15.16:0/2792336994 shutdown_connections 2024-04-25T10:12:36.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.561+0000 7fee54fa5640 1 -- 172.21.15.16:0/2792336994 wait complete. 2024-04-25T10:12:36.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:36.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.674+0000 7fed58722640 1 Processor -- start 2024-04-25T10:12:36.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.674+0000 7fed58722640 1 -- start start 2024-04-25T10:12:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.675+0000 7fed58722640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fed5015dce0 0x7fed500d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.675+0000 7fed58722640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fed500d35b0 0x7fed500b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.675+0000 7fed58722640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed500d1f90 0x7fed500b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.675+0000 7fed58722640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed5015e0c0 con 0x7fed5015dce0 2024-04-25T10:12:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.675+0000 7fed58722640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed500d2510 con 0x7fed500d35b0 2024-04-25T10:12:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.675+0000 7fed58722640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed500d27f0 con 0x7fed500d1f90 2024-04-25T10:12:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.676+0000 7fed564be640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fed5015dce0 0x7fed500d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.676+0000 7fed564be640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fed5015dce0 0x7fed500d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32818/0 (socket says 172.21.15.16:32818) 2024-04-25T10:12:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.676+0000 7fed564be640 1 -- 172.21.15.16:0/2739899742 learned_addr learned my addr 172.21.15.16:0/2739899742 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.676+0000 7fed55cbd640 1 --2- 172.21.15.16:0/2739899742 >> v2:172.21.15.186:3300/0 conn(0x7fed500d35b0 0x7fed500b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.676+0000 7fed56cbf640 1 --2- 172.21.15.16:0/2739899742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed500d1f90 0x7fed500b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.676+0000 7fed564be640 1 -- 172.21.15.16:0/2739899742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed500d1f90 msgr2=0x7fed500b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.676+0000 7fed564be640 1 --2- 172.21.15.16:0/2739899742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed500d1f90 0x7fed500b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.676+0000 7fed564be640 1 -- 172.21.15.16:0/2739899742 >> v2:172.21.15.186:3300/0 conn(0x7fed500d35b0 msgr2=0x7fed500b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.676+0000 7fed564be640 1 --2- 172.21.15.16:0/2739899742 >> v2:172.21.15.186:3300/0 conn(0x7fed500d35b0 0x7fed500b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.676+0000 7fed564be640 1 -- 172.21.15.16:0/2739899742 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed500d2ad0 con 0x7fed5015dce0 2024-04-25T10:12:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.676+0000 7fed55cbd640 1 --2- 172.21.15.16:0/2739899742 >> v2:172.21.15.186:3300/0 conn(0x7fed500d35b0 0x7fed500b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.677+0000 7fed564be640 1 --2- 172.21.15.16:0/2739899742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed5015dce0 0x7fed500d1a50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fed500cdfa0 tx=0x7fed4c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6890368408f35d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.677+0000 7fed554bc640 1 -- 172.21.15.16:0/2739899742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c037070 con 0x7fed5015dce0 2024-04-25T10:12:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.677+0000 7fed554bc640 1 -- 172.21.15.16:0/2739899742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4c032440 con 0x7fed5015dce0 2024-04-25T10:12:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.677+0000 7fed554bc640 1 -- 172.21.15.16:0/2739899742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c032810 con 0x7fed5015dce0 2024-04-25T10:12:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.677+0000 7fed58722640 1 -- 172.21.15.16:0/2739899742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed5015dce0 msgr2=0x7fed500d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.677+0000 7fed58722640 1 --2- 172.21.15.16:0/2739899742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed5015dce0 0x7fed500d1a50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fed500cdfa0 tx=0x7fed4c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.677+0000 7fed58722640 1 -- 172.21.15.16:0/2739899742 shutdown_connections 2024-04-25T10:12:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.677+0000 7fed58722640 1 --2- 172.21.15.16:0/2739899742 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed500d1f90 0x7fed500b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.677+0000 7fed58722640 1 --2- 172.21.15.16:0/2739899742 >> v2:172.21.15.186:3300/0 conn(0x7fed500d35b0 0x7fed500b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.677+0000 7fed58722640 1 --2- 172.21.15.16:0/2739899742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed5015dce0 0x7fed500d1a50 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.678+0000 7fed58722640 1 -- 172.21.15.16:0/2739899742 >> 172.21.15.16:0/2739899742 conn(0x7fed50072900 msgr2=0x7fed500d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.678+0000 7fed58722640 1 -- 172.21.15.16:0/2739899742 shutdown_connections 2024-04-25T10:12:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.678+0000 7fed58722640 1 -- 172.21.15.16:0/2739899742 wait complete. 2024-04-25T10:12:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.679+0000 7fed58722640 1 Processor -- start 2024-04-25T10:12:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.679+0000 7fed58722640 1 -- start start 2024-04-25T10:12:36.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.679+0000 7fed58722640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed500d1f90 0x7fed5015f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.679+0000 7fed58722640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed500d35b0 0x7fed5014dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed58722640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed5015dce0 0x7fed5015a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed58722640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed500b9820 con 0x7fed5015dce0 2024-04-25T10:12:36.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed58722640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed500d3090 con 0x7fed500d35b0 2024-04-25T10:12:36.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed58722640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed500b99a0 con 0x7fed500d1f90 2024-04-25T10:12:36.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed56cbf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed5015dce0 0x7fed5015a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed56cbf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed5015dce0 0x7fed5015a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32826/0 (socket says 172.21.15.16:32826) 2024-04-25T10:12:36.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed56cbf640 1 -- 172.21.15.16:0/1173152828 learned_addr learned my addr 172.21.15.16:0/1173152828 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed55cbd640 1 --2- 172.21.15.16:0/1173152828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed500d35b0 0x7fed5014dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed564be640 1 --2- 172.21.15.16:0/1173152828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed500d1f90 0x7fed5015f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed56cbf640 1 -- 172.21.15.16:0/1173152828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed500d1f90 msgr2=0x7fed5015f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed56cbf640 1 --2- 172.21.15.16:0/1173152828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed500d1f90 0x7fed5015f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed56cbf640 1 -- 172.21.15.16:0/1173152828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed500d35b0 msgr2=0x7fed5014dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed56cbf640 1 --2- 172.21.15.16:0/1173152828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed500d35b0 0x7fed5014dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed56cbf640 1 -- 172.21.15.16:0/1173152828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed501f6d90 con 0x7fed5015dce0 2024-04-25T10:12:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed56cbf640 1 --2- 172.21.15.16:0/1173152828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed5015dce0 0x7fed5015a310 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fed4800eed0 tx=0x7fed4800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed46ffd640 1 -- 172.21.15.16:0/1173152828 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48018020 con 0x7fed5015dce0 2024-04-25T10:12:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed46ffd640 1 -- 172.21.15.16:0/1173152828 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4800bd20 con 0x7fed5015dce0 2024-04-25T10:12:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed58722640 1 -- 172.21.15.16:0/1173152828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed501f7070 con 0x7fed5015dce0 2024-04-25T10:12:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed46ffd640 1 -- 172.21.15.16:0/1173152828 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48014640 con 0x7fed5015dce0 2024-04-25T10:12:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.680+0000 7fed58722640 1 -- 172.21.15.16:0/1173152828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed501f7630 con 0x7fed5015dce0 2024-04-25T10:12:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.681+0000 7fed58722640 1 -- 172.21.15.16:0/1173152828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed500749b0 con 0x7fed5015dce0 2024-04-25T10:12:36.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.682+0000 7fed46ffd640 1 -- 172.21.15.16:0/1173152828 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed48005d10 con 0x7fed5015dce0 2024-04-25T10:12:36.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.682+0000 7fed46ffd640 1 --2- 172.21.15.16:0/1173152828 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed3406b4d0 0x7fed3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.682+0000 7fed46ffd640 1 -- 172.21.15.16:0/1173152828 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fed4808a7a0 con 0x7fed5015dce0 2024-04-25T10:12:36.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.683+0000 7fed564be640 1 --2- 172.21.15.16:0/1173152828 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed3406b4d0 0x7fed3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.683+0000 7fed564be640 1 --2- 172.21.15.16:0/1173152828 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed3406b4d0 0x7fed3406d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fed4c034000 tx=0x7fed4c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.687+0000 7fed46ffd640 1 -- 172.21.15.16:0/1173152828 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed48022070 con 0x7fed5015dce0 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.902+0000 7fed58722640 1 -- 172.21.15.16:0/1173152828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed5015d2c0 con 0x7fed5015dce0 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.903+0000 7fed46ffd640 1 -- 172.21.15.16:0/1173152828 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fed48058f20 con 0x7fed5015dce0 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4044509901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4044509901},{"type":"v1","addr":"172.21.15.16:6835","nonce":4044509901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1494230493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1494230493},{"type":"v1","addr":"172.21.15.186:6837","nonce":1494230493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/1050183001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1050183001},{"type":"v1","addr":"172.21.15.186:6836","nonce":1050183001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:00.015075+0000","modified":"2024-04-25T10:11:58.248543+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7323},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7323":{"gid":7323,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":19,"addr":"172.21.15.16:6834/84349474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":84349474},{"type":"v1","addr":"172.21.15.16:6834","nonce":84349474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 64 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.904+0000 7fed58722640 1 -- 172.21.15.16:0/1173152828 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed3406b4d0 msgr2=0x7fed3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.904+0000 7fed58722640 1 --2- 172.21.15.16:0/1173152828 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed3406b4d0 0x7fed3406d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fed4c034000 tx=0x7fed4c02e000 comp rx=0 tx=0).stop 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.904+0000 7fed58722640 1 -- 172.21.15.16:0/1173152828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed5015dce0 msgr2=0x7fed5015a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.904+0000 7fed58722640 1 --2- 172.21.15.16:0/1173152828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed5015dce0 0x7fed5015a310 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fed4800eed0 tx=0x7fed4800b480 comp rx=0 tx=0).stop 2024-04-25T10:12:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.904+0000 7fed58722640 1 -- 172.21.15.16:0/1173152828 shutdown_connections 2024-04-25T10:12:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.904+0000 7fed58722640 1 --2- 172.21.15.16:0/1173152828 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed3406b4d0 0x7fed3406d990 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.904+0000 7fed58722640 1 --2- 172.21.15.16:0/1173152828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed5015dce0 0x7fed5015a310 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.904+0000 7fed58722640 1 --2- 172.21.15.16:0/1173152828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed500d35b0 0x7fed5014dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.904+0000 7fed58722640 1 --2- 172.21.15.16:0/1173152828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed500d1f90 0x7fed5015f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.905+0000 7fed58722640 1 -- 172.21.15.16:0/1173152828 >> 172.21.15.16:0/1173152828 conn(0x7fed50072900 msgr2=0x7fed500b6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.905+0000 7fed58722640 1 -- 172.21.15.16:0/1173152828 shutdown_connections 2024-04-25T10:12:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:36.905+0000 7fed58722640 1 -- 172.21.15.16:0/1173152828 wait complete. 2024-04-25T10:12:36.920 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:37.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.016+0000 7f066b6fc640 1 Processor -- start 2024-04-25T10:12:37.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.016+0000 7f066b6fc640 1 -- start start 2024-04-25T10:12:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.016+0000 7f066b6fc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f066409a150 0x7f066406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f066b6fc640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f066406c680 con 0x7f066406c1f0 2024-04-25T10:12:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f066b6fc640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f066406c900 con 0x7f066415ba10 2024-04-25T10:12:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f066b6fc640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06640cbe50 con 0x7f066409a150 2024-04-25T10:12:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0669c99640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f066406c1f0 0x7f06640cd220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49186/0 (socket says 172.21.15.16:49186) 2024-04-25T10:12:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0669c99640 1 -- 172.21.15.16:0/1955085660 learned_addr learned my addr 172.21.15.16:0/1955085660 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0668c97640 1 --2- 172.21.15.16:0/1955085660 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f066409a150 0x7f066406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0663fff640 1 -- 172.21.15.16:0/1955085660 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907769687 0 0) 0x7f0654001490 con 0x7f066406c1f0 2024-04-25T10:12:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0663fff640 1 -- 172.21.15.16:0/1955085660 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f064c003520 con 0x7f066406c1f0 2024-04-25T10:12:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0663fff640 1 -- 172.21.15.16:0/1955085660 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2667773832 0 0) 0x7f064c003520 con 0x7f066406c1f0 2024-04-25T10:12:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0663fff640 1 -- 172.21.15.16:0/1955085660 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f066409a150 msgr2=0x7f066406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0663fff640 1 --2- 172.21.15.16:0/1955085660 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f066409a150 0x7f066406bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0663fff640 1 -- 172.21.15.16:0/1955085660 >> v1:172.21.15.186:6789/0 conn(0x7f066415ba10 legacy=0x7f0664099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0663fff640 1 -- 172.21.15.16:0/1955085660 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066406cbe0 con 0x7f066406c1f0 2024-04-25T10:12:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0663fff640 1 -- 172.21.15.16:0/1955085660 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f0654001bc0 con 0x7f066406c1f0 2024-04-25T10:12:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0663fff640 1 -- 172.21.15.16:0/1955085660 >> v1:172.21.15.16:6789/0 conn(0x7f066406c1f0 legacy=0x7f06640cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0663fff640 1 --2- 172.21.15.16:0/1955085660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f064c0039e0 0x7f064c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0663fff640 1 --2- 172.21.15.16:0/1955085660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f064c004390 0x7f064c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0663fff640 1 --2- 172.21.15.16:0/1955085660 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f066409a150 0x7f064c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0663fff640 1 -- 172.21.15.16:0/1955085660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c00b9c0 con 0x7f064c0039e0 2024-04-25T10:12:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0663fff640 1 -- 172.21.15.16:0/1955085660 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c00bb40 con 0x7f064c004390 2024-04-25T10:12:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0663fff640 1 -- 172.21.15.16:0/1955085660 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064c00bd40 con 0x7f066409a150 2024-04-25T10:12:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.017+0000 7f0669498640 1 --2- 172.21.15.16:0/1955085660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f064c0039e0 0x7f064c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.018+0000 7f0669c99640 1 --2- 172.21.15.16:0/1955085660 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f066409a150 0x7f064c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.018+0000 7f0668c97640 1 --2- 172.21.15.16:0/1955085660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f064c004390 0x7f064c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.018+0000 7f0669498640 1 -- 172.21.15.16:0/1955085660 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f066409a150 msgr2=0x7f064c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.018+0000 7f0669498640 1 --2- 172.21.15.16:0/1955085660 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f066409a150 0x7f064c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.018+0000 7f0669498640 1 -- 172.21.15.16:0/1955085660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f064c004390 msgr2=0x7f064c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.018+0000 7f0669498640 1 --2- 172.21.15.16:0/1955085660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f064c004390 0x7f064c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.018+0000 7f0669498640 1 -- 172.21.15.16:0/1955085660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c003520 con 0x7f064c0039e0 2024-04-25T10:12:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.018+0000 7f0668c97640 1 --2- 172.21.15.16:0/1955085660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f064c004390 0x7f064c004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.018+0000 7f0669498640 1 --2- 172.21.15.16:0/1955085660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f064c0039e0 0x7f064c003dc0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f065c001860 tx=0x7f064c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1db9d14cc602 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.019+0000 7f0663fff640 1 -- 172.21.15.16:0/1955085660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c037070 con 0x7f064c0039e0 2024-04-25T10:12:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.019+0000 7f0663fff640 1 -- 172.21.15.16:0/1955085660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065c02ac30 con 0x7f064c0039e0 2024-04-25T10:12:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.019+0000 7f0663fff640 1 -- 172.21.15.16:0/1955085660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c001430 con 0x7f064c0039e0 2024-04-25T10:12:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.019+0000 7f066b6fc640 1 -- 172.21.15.16:0/1955085660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f064c0039e0 msgr2=0x7f064c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.019+0000 7f066b6fc640 1 --2- 172.21.15.16:0/1955085660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f064c0039e0 0x7f064c003dc0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f065c001860 tx=0x7f064c005830 comp rx=0 tx=0).stop 2024-04-25T10:12:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.019+0000 7f0669c99640 1 -- 172.21.15.16:0/1955085660 reap_dead start 2024-04-25T10:12:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.019+0000 7f066b6fc640 1 -- 172.21.15.16:0/1955085660 shutdown_connections 2024-04-25T10:12:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.019+0000 7f066b6fc640 1 -- 172.21.15.16:0/1955085660 >> 172.21.15.16:0/1955085660 conn(0x7f06640b32f0 msgr2=0x7f06640b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.019+0000 7f066b6fc640 1 -- 172.21.15.16:0/1955085660 shutdown_connections 2024-04-25T10:12:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.019+0000 7f066b6fc640 1 -- 172.21.15.16:0/1955085660 wait complete. 2024-04-25T10:12:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.020+0000 7f066b6fc640 1 Processor -- start 2024-04-25T10:12:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.020+0000 7f066b6fc640 1 -- start start 2024-04-25T10:12:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.021+0000 7f066b6fc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f064c0039e0 0x7f06640c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.021+0000 7f066b6fc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f066415fe30 0x7f0664153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.021+0000 7f066b6fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06640c7940 0x7f06640c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.021+0000 7f066b6fc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06640d1320 con 0x7f06640c7940 2024-04-25T10:12:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.021+0000 7f066b6fc640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06640cfe80 con 0x7f066415fe30 2024-04-25T10:12:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.021+0000 7f066b6fc640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06640d14a0 con 0x7f064c0039e0 2024-04-25T10:12:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.021+0000 7f0669c99640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06640c7940 0x7f06640c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.021+0000 7f0669c99640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06640c7940 0x7f06640c7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32844/0 (socket says 172.21.15.16:32844) 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.021+0000 7f0669c99640 1 -- 172.21.15.16:0/1570950104 learned_addr learned my addr 172.21.15.16:0/1570950104 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.021+0000 7f0668c97640 1 --2- 172.21.15.16:0/1570950104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f066415fe30 0x7f0664153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.021+0000 7f0669498640 1 --2- 172.21.15.16:0/1570950104 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f064c0039e0 0x7f06640c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.022+0000 7f0669c99640 1 -- 172.21.15.16:0/1570950104 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f064c0039e0 msgr2=0x7f06640c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.022+0000 7f0669c99640 1 --2- 172.21.15.16:0/1570950104 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f064c0039e0 0x7f06640c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.022+0000 7f0669c99640 1 -- 172.21.15.16:0/1570950104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f066415fe30 msgr2=0x7f0664153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.022+0000 7f0669c99640 1 --2- 172.21.15.16:0/1570950104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f066415fe30 0x7f0664153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.022+0000 7f0669c99640 1 -- 172.21.15.16:0/1570950104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0664160e90 con 0x7f06640c7940 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.022+0000 7f0668c97640 1 --2- 172.21.15.16:0/1570950104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f066415fe30 0x7f0664153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.022+0000 7f0669c99640 1 --2- 172.21.15.16:0/1570950104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06640c7940 0x7f06640c7dc0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f0654002230 tx=0x7f065400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.022+0000 7f0661ffb640 1 -- 172.21.15.16:0/1570950104 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654019070 con 0x7f06640c7940 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.022+0000 7f0661ffb640 1 -- 172.21.15.16:0/1570950104 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065400ebc0 con 0x7f06640c7940 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.022+0000 7f066b6fc640 1 -- 172.21.15.16:0/1570950104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06641605f0 con 0x7f06640c7940 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.022+0000 7f0661ffb640 1 -- 172.21.15.16:0/1570950104 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654014480 con 0x7f06640c7940 2024-04-25T10:12:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.022+0000 7f066b6fc640 1 -- 172.21.15.16:0/1570950104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0664160bb0 con 0x7f06640c7940 2024-04-25T10:12:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.023+0000 7f066b6fc640 1 -- 172.21.15.16:0/1570950104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06640749b0 con 0x7f06640c7940 2024-04-25T10:12:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.023+0000 7f0661ffb640 1 -- 172.21.15.16:0/1570950104 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f065401c070 con 0x7f06640c7940 2024-04-25T10:12:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.024+0000 7f0661ffb640 1 --2- 172.21.15.16:0/1570950104 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f065006b500 0x7f065006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.024+0000 7f0661ffb640 1 -- 172.21.15.16:0/1570950104 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f0654089e20 con 0x7f06640c7940 2024-04-25T10:12:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.024+0000 7f0669498640 1 --2- 172.21.15.16:0/1570950104 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f065006b500 0x7f065006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.025+0000 7f0669498640 1 --2- 172.21.15.16:0/1570950104 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f065006b500 0x7f065006d9c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f065c001860 tx=0x7f065c009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.028+0000 7f0661ffb640 1 -- 172.21.15.16:0/1570950104 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f066406c680 con 0x7f06640c7940 2024-04-25T10:12:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.210+0000 7f066b6fc640 1 -- 172.21.15.16:0/1570950104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f066409a570 con 0x7f06640c7940 2024-04-25T10:12:37.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.211+0000 7f0661ffb640 1 -- 172.21.15.16:0/1570950104 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16935 (secure 0 0 0) 0x7f0654058c80 con 0x7f06640c7940 2024-04-25T10:12:37.216 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:37.216 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:11:55.231494+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:58.299405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:58.994953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.186:6836/3950732718":"2024-04-26T10:11:55.231471+0000","172.21.15.186:6834/3950732718":"2024-04-26T10:11:55.231471+0000","172.21.15.186:6837/3092291936":"2024-04-26T10:11:13.827348+0000","172.21.15.186:6835/3092291936":"2024-04-26T10:11:13.827348+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.213+0000 7f066b6fc640 1 -- 172.21.15.16:0/1570950104 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f065006b500 msgr2=0x7f065006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.213+0000 7f066b6fc640 1 --2- 172.21.15.16:0/1570950104 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f065006b500 0x7f065006d9c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f065c001860 tx=0x7f065c009030 comp rx=0 tx=0).stop 2024-04-25T10:12:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.213+0000 7f066b6fc640 1 -- 172.21.15.16:0/1570950104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06640c7940 msgr2=0x7f06640c7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.213+0000 7f066b6fc640 1 --2- 172.21.15.16:0/1570950104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06640c7940 0x7f06640c7dc0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f0654002230 tx=0x7f065400e530 comp rx=0 tx=0).stop 2024-04-25T10:12:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.213+0000 7f066b6fc640 1 -- 172.21.15.16:0/1570950104 shutdown_connections 2024-04-25T10:12:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.213+0000 7f066b6fc640 1 --2- 172.21.15.16:0/1570950104 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f065006b500 0x7f065006d9c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.213+0000 7f066b6fc640 1 --2- 172.21.15.16:0/1570950104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06640c7940 0x7f06640c7dc0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.213+0000 7f066b6fc640 1 --2- 172.21.15.16:0/1570950104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f066415fe30 0x7f0664153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.213+0000 7f066b6fc640 1 --2- 172.21.15.16:0/1570950104 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f064c0039e0 0x7f06640c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.213+0000 7f066b6fc640 1 -- 172.21.15.16:0/1570950104 >> 172.21.15.16:0/1570950104 conn(0x7f06640b32f0 msgr2=0x7f06640afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.213+0000 7f066b6fc640 1 -- 172.21.15.16:0/1570950104 shutdown_connections 2024-04-25T10:12:37.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.213+0000 7f066b6fc640 1 -- 172.21.15.16:0/1570950104 wait complete. 2024-04-25T10:12:37.229 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:12:37.229 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.325+0000 7f37a8d09640 1 Processor -- start 2024-04-25T10:12:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.325+0000 7f37a8d09640 1 -- start start 2024-04-25T10:12:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.326+0000 7f37a8d09640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37a415ba10 0x7f37a4099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.326+0000 7f37a8d09640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a4099f80 0x7f37a406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.326+0000 7f37a8d09640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f37a406c1f0 0x7f37a40cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.326+0000 7f37a8d09640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a40cbe50 con 0x7f37a415ba10 2024-04-25T10:12:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.326+0000 7f37a8d09640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a40cfe80 con 0x7f37a406c1f0 2024-04-25T10:12:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.326+0000 7f37a8d09640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a409a3a0 con 0x7f37a4099f80 2024-04-25T10:12:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.326+0000 7f37a259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37a415ba10 0x7f37a4099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.326+0000 7f37a259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37a415ba10 0x7f37a4099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32860/0 (socket says 172.21.15.16:32860) 2024-04-25T10:12:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.326+0000 7f37a259c640 1 -- 172.21.15.16:0/2536584934 learned_addr learned my addr 172.21.15.16:0/2536584934 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.326+0000 7f37a259c640 1 -- 172.21.15.16:0/2536584934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a4099f80 msgr2=0x7f37a406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.326+0000 7f37a2d9d640 1 --2- 172.21.15.16:0/2536584934 >> v2:172.21.15.186:3300/0 conn(0x7f37a406c1f0 0x7f37a40cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.326+0000 7f379bfff640 1 --2- 172.21.15.16:0/2536584934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a4099f80 0x7f37a406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.326+0000 7f37a259c640 1 --2- 172.21.15.16:0/2536584934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a4099f80 0x7f37a406bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.326+0000 7f37a259c640 1 -- 172.21.15.16:0/2536584934 >> v2:172.21.15.186:3300/0 conn(0x7f37a406c1f0 msgr2=0x7f37a40cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.326+0000 7f37a259c640 1 --2- 172.21.15.16:0/2536584934 >> v2:172.21.15.186:3300/0 conn(0x7f37a406c1f0 0x7f37a40cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.326+0000 7f37a259c640 1 -- 172.21.15.16:0/2536584934 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a406c680 con 0x7f37a415ba10 2024-04-25T10:12:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.327+0000 7f37a259c640 1 --2- 172.21.15.16:0/2536584934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a415ba10 0x7f37a4099a40 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f37a41616a0 tx=0x7f379402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a537592614a0eda5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.327+0000 7f37a1d9b640 1 -- 172.21.15.16:0/2536584934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3794037070 con 0x7f37a415ba10 2024-04-25T10:12:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.327+0000 7f37a1d9b640 1 -- 172.21.15.16:0/2536584934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3794032440 con 0x7f37a415ba10 2024-04-25T10:12:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.327+0000 7f37a1d9b640 1 -- 172.21.15.16:0/2536584934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3794032810 con 0x7f37a415ba10 2024-04-25T10:12:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.328+0000 7f37a8d09640 1 -- 172.21.15.16:0/2536584934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a415ba10 msgr2=0x7f37a4099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.328+0000 7f37a8d09640 1 --2- 172.21.15.16:0/2536584934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a415ba10 0x7f37a4099a40 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f37a41616a0 tx=0x7f379402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.328+0000 7f37a8d09640 1 -- 172.21.15.16:0/2536584934 shutdown_connections 2024-04-25T10:12:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.328+0000 7f37a8d09640 1 --2- 172.21.15.16:0/2536584934 >> v2:172.21.15.186:3300/0 conn(0x7f37a406c1f0 0x7f37a40cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.328+0000 7f37a8d09640 1 --2- 172.21.15.16:0/2536584934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a4099f80 0x7f37a406bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.328+0000 7f37a8d09640 1 --2- 172.21.15.16:0/2536584934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a415ba10 0x7f37a4099a40 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.328+0000 7f37a8d09640 1 -- 172.21.15.16:0/2536584934 >> 172.21.15.16:0/2536584934 conn(0x7f37a40b32f0 msgr2=0x7f37a40b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.328+0000 7f37a8d09640 1 -- 172.21.15.16:0/2536584934 shutdown_connections 2024-04-25T10:12:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.328+0000 7f37a8d09640 1 -- 172.21.15.16:0/2536584934 wait complete. 2024-04-25T10:12:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.329+0000 7f37a8d09640 1 Processor -- start 2024-04-25T10:12:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.329+0000 7f37a8d09640 1 -- start start 2024-04-25T10:12:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f37a8d09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a406c1f0 0x7f37a40b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f37a8d09640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37a4099f80 0x7f37a41581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f37a8d09640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a415ba10 0x7f37a40c6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f37a8d09640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a406cec0 con 0x7f37a406c1f0 2024-04-25T10:12:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f37a8d09640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a406cbe0 con 0x7f37a4099f80 2024-04-25T10:12:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f37a8d09640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a40cd760 con 0x7f37a415ba10 2024-04-25T10:12:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f37a259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a406c1f0 0x7f37a40b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f37a259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a406c1f0 0x7f37a40b4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32862/0 (socket says 172.21.15.16:32862) 2024-04-25T10:12:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f37a259c640 1 -- 172.21.15.16:0/2663850259 learned_addr learned my addr 172.21.15.16:0/2663850259 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f379bfff640 1 --2- 172.21.15.16:0/2663850259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37a4099f80 0x7f37a41581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f37a2d9d640 1 --2- 172.21.15.16:0/2663850259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a415ba10 0x7f37a40c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f37a259c640 1 -- 172.21.15.16:0/2663850259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a415ba10 msgr2=0x7f37a40c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f37a259c640 1 --2- 172.21.15.16:0/2663850259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a415ba10 0x7f37a40c6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f37a259c640 1 -- 172.21.15.16:0/2663850259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37a4099f80 msgr2=0x7f37a41581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f37a259c640 1 --2- 172.21.15.16:0/2663850259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37a4099f80 0x7f37a41581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f37a259c640 1 -- 172.21.15.16:0/2663850259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a40b4960 con 0x7f37a406c1f0 2024-04-25T10:12:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f379bfff640 1 --2- 172.21.15.16:0/2663850259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37a4099f80 0x7f37a41581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f37a259c640 1 --2- 172.21.15.16:0/2663850259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a406c1f0 0x7f37a40b4420 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f379402a220 tx=0x7f37940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f379affd640 1 -- 172.21.15.16:0/2663850259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3794037040 con 0x7f37a406c1f0 2024-04-25T10:12:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.330+0000 7f379affd640 1 -- 172.21.15.16:0/2663850259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3794042070 con 0x7f37a406c1f0 2024-04-25T10:12:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.331+0000 7f37a8d09640 1 -- 172.21.15.16:0/2663850259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a40c6bf0 con 0x7f37a406c1f0 2024-04-25T10:12:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.331+0000 7f379affd640 1 -- 172.21.15.16:0/2663850259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3794030bb0 con 0x7f37a406c1f0 2024-04-25T10:12:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.331+0000 7f37a8d09640 1 -- 172.21.15.16:0/2663850259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a40ca300 con 0x7f37a406c1f0 2024-04-25T10:12:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.331+0000 7f37a8d09640 1 -- 172.21.15.16:0/2663850259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a40749b0 con 0x7f37a406c1f0 2024-04-25T10:12:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.332+0000 7f379affd640 1 -- 172.21.15.16:0/2663850259 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3794047460 con 0x7f37a406c1f0 2024-04-25T10:12:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.332+0000 7f379affd640 1 --2- 172.21.15.16:0/2663850259 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f378406b4d0 0x7f378406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.332+0000 7f379affd640 1 -- 172.21.15.16:0/2663850259 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f37940aaf30 con 0x7f37a406c1f0 2024-04-25T10:12:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.333+0000 7f379bfff640 1 --2- 172.21.15.16:0/2663850259 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f378406b4d0 0x7f378406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.333+0000 7f379bfff640 1 --2- 172.21.15.16:0/2663850259 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f378406b4d0 0x7f378406d990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f37a41f7910 tx=0x7f378c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.337+0000 7f379affd640 1 -- 172.21.15.16:0/2663850259 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f37a406c680 con 0x7f37a406c1f0 2024-04-25T10:12:37.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.553+0000 7f37a8d09640 1 -- 172.21.15.16:0/2663850259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37a41614b0 con 0x7f37a406c1f0 2024-04-25T10:12:37.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.554+0000 7f379affd640 1 -- 172.21.15.16:0/2663850259 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f3794045360 con 0x7f37a406c1f0 2024-04-25T10:12:37.559 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:37.559 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4044509901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4044509901},{"type":"v1","addr":"172.21.15.16:6835","nonce":4044509901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1494230493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1494230493},{"type":"v1","addr":"172.21.15.186:6837","nonce":1494230493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/1050183001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1050183001},{"type":"v1","addr":"172.21.15.186:6836","nonce":1050183001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:00.015075+0000","modified":"2024-04-25T10:11:58.248543+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7323},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7323":{"gid":7323,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":19,"addr":"172.21.15.16:6834/84349474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":84349474},{"type":"v1","addr":"172.21.15.16:6834","nonce":84349474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:12:37.559 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 64 2024-04-25T10:12:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.556+0000 7f37a8d09640 1 -- 172.21.15.16:0/2663850259 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f378406b4d0 msgr2=0x7f378406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.556+0000 7f37a8d09640 1 --2- 172.21.15.16:0/2663850259 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f378406b4d0 0x7f378406d990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f37a41f7910 tx=0x7f378c008040 comp rx=0 tx=0).stop 2024-04-25T10:12:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.556+0000 7f37a8d09640 1 -- 172.21.15.16:0/2663850259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a406c1f0 msgr2=0x7f37a40b4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.556+0000 7f37a8d09640 1 --2- 172.21.15.16:0/2663850259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a406c1f0 0x7f37a40b4420 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f379402a220 tx=0x7f37940046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.556+0000 7f37a8d09640 1 -- 172.21.15.16:0/2663850259 shutdown_connections 2024-04-25T10:12:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.556+0000 7f37a8d09640 1 --2- 172.21.15.16:0/2663850259 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f378406b4d0 0x7f378406d990 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.556+0000 7f37a8d09640 1 --2- 172.21.15.16:0/2663850259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37a415ba10 0x7f37a40c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.556+0000 7f37a8d09640 1 --2- 172.21.15.16:0/2663850259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37a4099f80 0x7f37a41581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.556+0000 7f37a8d09640 1 --2- 172.21.15.16:0/2663850259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a406c1f0 0x7f37a40b4420 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.556+0000 7f37a8d09640 1 -- 172.21.15.16:0/2663850259 >> 172.21.15.16:0/2663850259 conn(0x7f37a40b32f0 msgr2=0x7f37a415ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.556+0000 7f37a8d09640 1 -- 172.21.15.16:0/2663850259 shutdown_connections 2024-04-25T10:12:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.556+0000 7f37a8d09640 1 -- 172.21.15.16:0/2663850259 wait complete. 2024-04-25T10:12:37.571 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.667+0000 7f91722ea640 1 Processor -- start 2024-04-25T10:12:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.667+0000 7f91722ea640 1 -- start start 2024-04-25T10:12:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.668+0000 7f91722ea640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f916c0d35b0 0x7f916c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.668+0000 7f91722ea640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f916c0b5990 0x7f916c0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.668+0000 7f91722ea640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916c15e0c0 con 0x7f916c15dce0 2024-04-25T10:12:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.668+0000 7f91722ea640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916c0d2510 con 0x7f916c0d35b0 2024-04-25T10:12:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.668+0000 7f91722ea640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0d27f0 con 0x7f916c0b5990 2024-04-25T10:12:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.669+0000 7f916b7fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f916c0d35b0 0x7f916c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.669+0000 7f9170887640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f916c0b5990 0x7f916c0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.669+0000 7f916bfff640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f916c15dce0 0x7f916c0d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:44482/0 (socket says 172.21.15.16:44482) 2024-04-25T10:12:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.669+0000 7f916b7fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f916c0d35b0 0x7f916c0b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:53328/0 (socket says 172.21.15.16:53328) 2024-04-25T10:12:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.669+0000 7f9170887640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f916c0b5990 0x7f916c0b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:34932/0 (socket says 172.21.15.16:34932) 2024-04-25T10:12:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.669+0000 7f916bfff640 1 -- 172.21.15.16:0/2318048247 learned_addr learned my addr 172.21.15.16:0/2318048247 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.669+0000 7f916b7fe640 1 -- 172.21.15.16:0/2318048247 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f916c0b5990 msgr2=0x7f916c0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.669+0000 7f916b7fe640 1 --2- 172.21.15.16:0/2318048247 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f916c0b5990 0x7f916c0b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.669+0000 7f916b7fe640 1 -- 172.21.15.16:0/2318048247 >> v1:172.21.15.186:6789/0 conn(0x7f916c15dce0 legacy=0x7f916c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.669+0000 7f916affd640 1 -- 172.21.15.16:0/2318048247 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359067310 0 0) 0x7f916c15e0c0 con 0x7f916c15dce0 2024-04-25T10:12:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.669+0000 7f916b7fe640 1 -- 172.21.15.16:0/2318048247 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c0d2ad0 con 0x7f916c0d35b0 2024-04-25T10:12:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.670+0000 7f916b7fe640 1 --2- 172.21.15.16:0/2318048247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f916c0d35b0 0x7f916c0b5450 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f91540097a0 tx=0x7f915402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f20d0d68a5b2734 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.670+0000 7f916affd640 1 -- 172.21.15.16:0/2318048247 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9154037070 con 0x7f916c0d35b0 2024-04-25T10:12:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.670+0000 7f916affd640 1 -- 172.21.15.16:0/2318048247 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9154032440 con 0x7f916c0d35b0 2024-04-25T10:12:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.670+0000 7f916affd640 1 -- 172.21.15.16:0/2318048247 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9154032810 con 0x7f916c0d35b0 2024-04-25T10:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.670+0000 7f91722ea640 1 -- 172.21.15.16:0/2318048247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f916c0d35b0 msgr2=0x7f916c0b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.670+0000 7f91722ea640 1 --2- 172.21.15.16:0/2318048247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f916c0d35b0 0x7f916c0b5450 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f91540097a0 tx=0x7f915402a180 comp rx=0 tx=0).stop 2024-04-25T10:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.670+0000 7f91722ea640 1 -- 172.21.15.16:0/2318048247 shutdown_connections 2024-04-25T10:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.670+0000 7f91722ea640 1 --2- 172.21.15.16:0/2318048247 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f916c0b5990 0x7f916c0b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.670+0000 7f91722ea640 1 --2- 172.21.15.16:0/2318048247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f916c0d35b0 0x7f916c0b5450 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.670+0000 7f91722ea640 1 -- 172.21.15.16:0/2318048247 >> 172.21.15.16:0/2318048247 conn(0x7f916c072900 msgr2=0x7f916c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.671+0000 7f91722ea640 1 -- 172.21.15.16:0/2318048247 shutdown_connections 2024-04-25T10:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.671+0000 7f91722ea640 1 -- 172.21.15.16:0/2318048247 wait complete. 2024-04-25T10:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.672+0000 7f91722ea640 1 Processor -- start 2024-04-25T10:12:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.672+0000 7f91722ea640 1 -- start start 2024-04-25T10:12:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.672+0000 7f91722ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916c0b5990 0x7f916c0c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.672+0000 7f91722ea640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f916c0d35b0 0x7f916c0ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f91722ea640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f916c15dce0 0x7f916c0cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f91722ea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0d2160 con 0x7f916c0b5990 2024-04-25T10:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f91722ea640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0d3090 con 0x7f916c15dce0 2024-04-25T10:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f91722ea640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0ac950 con 0x7f916c0d35b0 2024-04-25T10:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f916bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916c0b5990 0x7f916c0c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f916bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916c0b5990 0x7f916c0c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32868/0 (socket says 172.21.15.16:32868) 2024-04-25T10:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f916bfff640 1 -- 172.21.15.16:0/608962709 learned_addr learned my addr 172.21.15.16:0/608962709 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f9170887640 1 --2- 172.21.15.16:0/608962709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f916c15dce0 0x7f916c0cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f916b7fe640 1 --2- 172.21.15.16:0/608962709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f916c0d35b0 0x7f916c0ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f916bfff640 1 -- 172.21.15.16:0/608962709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f916c0d35b0 msgr2=0x7f916c0ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f916bfff640 1 --2- 172.21.15.16:0/608962709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f916c0d35b0 0x7f916c0ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f916bfff640 1 -- 172.21.15.16:0/608962709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f916c15dce0 msgr2=0x7f916c0cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f916bfff640 1 --2- 172.21.15.16:0/608962709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f916c15dce0 0x7f916c0cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f916bfff640 1 -- 172.21.15.16:0/608962709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c0acba0 con 0x7f916c0b5990 2024-04-25T10:12:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f916bfff640 1 --2- 172.21.15.16:0/608962709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916c0b5990 0x7f916c0c9cb0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c430 tx=0x7f915c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f9168ff9640 1 -- 172.21.15.16:0/608962709 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c0074e0 con 0x7f916c0b5990 2024-04-25T10:12:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f91722ea640 1 -- 172.21.15.16:0/608962709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916c0ace80 con 0x7f916c0b5990 2024-04-25T10:12:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f9168ff9640 1 -- 172.21.15.16:0/608962709 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915c0047e0 con 0x7f916c0b5990 2024-04-25T10:12:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f9168ff9640 1 -- 172.21.15.16:0/608962709 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c004bd0 con 0x7f916c0b5990 2024-04-25T10:12:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.673+0000 7f91722ea640 1 -- 172.21.15.16:0/608962709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c0ad440 con 0x7f916c0b5990 2024-04-25T10:12:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.674+0000 7f91722ea640 1 -- 172.21.15.16:0/608962709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916c0ad680 con 0x7f916c0b5990 2024-04-25T10:12:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.675+0000 7f9168ff9640 1 -- 172.21.15.16:0/608962709 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f915c011070 con 0x7f916c0b5990 2024-04-25T10:12:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.675+0000 7f9168ff9640 1 --2- 172.21.15.16:0/608962709 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f914c06b4d0 0x7f914c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.675+0000 7f9168ff9640 1 -- 172.21.15.16:0/608962709 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f915c089f00 con 0x7f916c0b5990 2024-04-25T10:12:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.675+0000 7f916b7fe640 1 --2- 172.21.15.16:0/608962709 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f914c06b4d0 0x7f914c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.676+0000 7f916b7fe640 1 --2- 172.21.15.16:0/608962709 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f914c06b4d0 0x7f914c06d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9154034000 tx=0x7f915402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.679+0000 7f9168ff9640 1 -- 172.21.15.16:0/608962709 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f915c058560 con 0x7f916c0b5990 2024-04-25T10:12:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.896+0000 7f91722ea640 1 -- 172.21.15.16:0/608962709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f916c0749b0 con 0x7f916c0b5990 2024-04-25T10:12:37.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.897+0000 7f9168ff9640 1 -- 172.21.15.16:0/608962709 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f915c05c600 con 0x7f916c0b5990 2024-04-25T10:12:37.902 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:37.902 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4044509901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4044509901},{"type":"v1","addr":"172.21.15.16:6835","nonce":4044509901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1494230493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1494230493},{"type":"v1","addr":"172.21.15.186:6837","nonce":1494230493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/1050183001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1050183001},{"type":"v1","addr":"172.21.15.186:6836","nonce":1050183001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:00.015075+0000","modified":"2024-04-25T10:11:58.248543+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7323},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7323":{"gid":7323,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":19,"addr":"172.21.15.16:6834/84349474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":84349474},{"type":"v1","addr":"172.21.15.16:6834","nonce":84349474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:12:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 64 2024-04-25T10:12:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.899+0000 7f91722ea640 1 -- 172.21.15.16:0/608962709 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f914c06b4d0 msgr2=0x7f914c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.899+0000 7f91722ea640 1 --2- 172.21.15.16:0/608962709 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f914c06b4d0 0x7f914c06d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9154034000 tx=0x7f915402e000 comp rx=0 tx=0).stop 2024-04-25T10:12:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.899+0000 7f91722ea640 1 -- 172.21.15.16:0/608962709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916c0b5990 msgr2=0x7f916c0c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.899+0000 7f91722ea640 1 --2- 172.21.15.16:0/608962709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916c0b5990 0x7f916c0c9cb0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c430 tx=0x7f915c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.899+0000 7f91722ea640 1 -- 172.21.15.16:0/608962709 shutdown_connections 2024-04-25T10:12:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.899+0000 7f91722ea640 1 --2- 172.21.15.16:0/608962709 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f914c06b4d0 0x7f914c06d990 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.899+0000 7f91722ea640 1 --2- 172.21.15.16:0/608962709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f916c15dce0 0x7f916c0cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.899+0000 7f91722ea640 1 --2- 172.21.15.16:0/608962709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f916c0d35b0 0x7f916c0ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.899+0000 7f91722ea640 1 --2- 172.21.15.16:0/608962709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916c0b5990 0x7f916c0c9cb0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.899+0000 7f91722ea640 1 -- 172.21.15.16:0/608962709 >> 172.21.15.16:0/608962709 conn(0x7f916c072900 msgr2=0x7f916c15cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.899+0000 7f91722ea640 1 -- 172.21.15.16:0/608962709 shutdown_connections 2024-04-25T10:12:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:37.899+0000 7f91722ea640 1 -- 172.21.15.16:0/608962709 wait complete. 2024-04-25T10:12:37.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:38.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.011+0000 7f841bd22640 1 Processor -- start 2024-04-25T10:12:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.011+0000 7f841bd22640 1 -- start start 2024-04-25T10:12:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.012+0000 7f841bd22640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f841415bb90 0x7f8414160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.012+0000 7f841bd22640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f84140cc950 0x7f84140b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.012+0000 7f841bd22640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841406b310 con 0x7f841406aba0 2024-04-25T10:12:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.012+0000 7f841bd22640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84140b0010 con 0x7f841415bb90 2024-04-25T10:12:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.012+0000 7f841bd22640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f841406b5f0 con 0x7f84140cc950 2024-04-25T10:12:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.012+0000 7f8419abe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f841415bb90 0x7f8414160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.012+0000 7f8419abe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f841415bb90 0x7f8414160070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32872/0 (socket says 172.21.15.16:32872) 2024-04-25T10:12:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.012+0000 7f8419abe640 1 -- 172.21.15.16:0/2818244268 learned_addr learned my addr 172.21.15.16:0/2818244268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.012+0000 7f841a2bf640 1 --2- 172.21.15.16:0/2818244268 >> v2:172.21.15.186:3300/0 conn(0x7f84140cc950 0x7f84140b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.012+0000 7f8419abe640 1 -- 172.21.15.16:0/2818244268 >> v2:172.21.15.186:3300/0 conn(0x7f84140cc950 msgr2=0x7f84140b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.012+0000 7f8419abe640 1 --2- 172.21.15.16:0/2818244268 >> v2:172.21.15.186:3300/0 conn(0x7f84140cc950 0x7f84140b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.012+0000 7f8419abe640 1 -- 172.21.15.16:0/2818244268 >> v1:172.21.15.186:6789/0 conn(0x7f841406aba0 legacy=0x7f84140cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.012+0000 7f8419abe640 1 -- 172.21.15.16:0/2818244268 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84140b0260 con 0x7f841415bb90 2024-04-25T10:12:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.013+0000 7f8419abe640 1 --2- 172.21.15.16:0/2818244268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841415bb90 0x7f8414160070 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f84140d0640 tx=0x7f841002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148a7be74d6f9e24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.013+0000 7f8418abc640 1 -- 172.21.15.16:0/2818244268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410037070 con 0x7f841415bb90 2024-04-25T10:12:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.013+0000 7f8418abc640 1 -- 172.21.15.16:0/2818244268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841002ae20 con 0x7f841415bb90 2024-04-25T10:12:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.013+0000 7f8418abc640 1 -- 172.21.15.16:0/2818244268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410032660 con 0x7f841415bb90 2024-04-25T10:12:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.014+0000 7f841bd22640 1 -- 172.21.15.16:0/2818244268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841415bb90 msgr2=0x7f8414160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.014+0000 7f841bd22640 1 --2- 172.21.15.16:0/2818244268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841415bb90 0x7f8414160070 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f84140d0640 tx=0x7f841002a110 comp rx=0 tx=0).stop 2024-04-25T10:12:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.014+0000 7f841bd22640 1 -- 172.21.15.16:0/2818244268 shutdown_connections 2024-04-25T10:12:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.014+0000 7f841bd22640 1 --2- 172.21.15.16:0/2818244268 >> v2:172.21.15.186:3300/0 conn(0x7f84140cc950 0x7f84140b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.014+0000 7f841bd22640 1 --2- 172.21.15.16:0/2818244268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841415bb90 0x7f8414160070 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.014+0000 7f841bd22640 1 -- 172.21.15.16:0/2818244268 >> 172.21.15.16:0/2818244268 conn(0x7f84140d06b0 msgr2=0x7f84140d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.014+0000 7f841bd22640 1 -- 172.21.15.16:0/2818244268 shutdown_connections 2024-04-25T10:12:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.014+0000 7f841bd22640 1 -- 172.21.15.16:0/2818244268 wait complete. 2024-04-25T10:12:38.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.015+0000 7f841bd22640 1 Processor -- start 2024-04-25T10:12:38.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.015+0000 7f841bd22640 1 -- start start 2024-04-25T10:12:38.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f841bd22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841406aba0 0x7f84140c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f841bd22640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84140cc950 0x7f84140c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f841bd22640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f841415bb90 0x7f841415a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f841bd22640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84140b0fa0 con 0x7f841406aba0 2024-04-25T10:12:38.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f841bd22640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841406b030 con 0x7f841415bb90 2024-04-25T10:12:38.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f841bd22640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84140b1120 con 0x7f84140cc950 2024-04-25T10:12:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f8419abe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841406aba0 0x7f84140c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f8419abe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841406aba0 0x7f84140c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32886/0 (socket says 172.21.15.16:32886) 2024-04-25T10:12:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f8419abe640 1 -- 172.21.15.16:0/3892025791 learned_addr learned my addr 172.21.15.16:0/3892025791 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f841a2bf640 1 --2- 172.21.15.16:0/3892025791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f841415bb90 0x7f841415a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f84192bd640 1 --2- 172.21.15.16:0/3892025791 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84140cc950 0x7f84140c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f8419abe640 1 -- 172.21.15.16:0/3892025791 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84140cc950 msgr2=0x7f84140c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f8419abe640 1 --2- 172.21.15.16:0/3892025791 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84140cc950 0x7f84140c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f8419abe640 1 -- 172.21.15.16:0/3892025791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f841415bb90 msgr2=0x7f841415a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f8419abe640 1 --2- 172.21.15.16:0/3892025791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f841415bb90 0x7f841415a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f8419abe640 1 -- 172.21.15.16:0/3892025791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841415b140 con 0x7f841406aba0 2024-04-25T10:12:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f841a2bf640 1 --2- 172.21.15.16:0/3892025791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f841415bb90 0x7f841415a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f84192bd640 1 --2- 172.21.15.16:0/3892025791 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84140cc950 0x7f84140c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.016+0000 7f8419abe640 1 --2- 172.21.15.16:0/3892025791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841406aba0 0x7f84140c9dc0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f8410000c00 tx=0x7f84100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.017+0000 7f840a7fc640 1 -- 172.21.15.16:0/3892025791 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410042070 con 0x7f841406aba0 2024-04-25T10:12:38.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.017+0000 7f840a7fc640 1 -- 172.21.15.16:0/3892025791 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841003e670 con 0x7f841406aba0 2024-04-25T10:12:38.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.017+0000 7f841bd22640 1 -- 172.21.15.16:0/3892025791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84140ca430 con 0x7f841406aba0 2024-04-25T10:12:38.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.017+0000 7f840a7fc640 1 -- 172.21.15.16:0/3892025791 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410037040 con 0x7f841406aba0 2024-04-25T10:12:38.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.017+0000 7f841bd22640 1 -- 172.21.15.16:0/3892025791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84140c86c0 con 0x7f841406aba0 2024-04-25T10:12:38.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.018+0000 7f841bd22640 1 -- 172.21.15.16:0/3892025791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84140c81f0 con 0x7f841406aba0 2024-04-25T10:12:38.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.018+0000 7f840a7fc640 1 -- 172.21.15.16:0/3892025791 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8410045070 con 0x7f841406aba0 2024-04-25T10:12:38.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.018+0000 7f840a7fc640 1 --2- 172.21.15.16:0/3892025791 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f83f806b500 0x7f83f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.019+0000 7f840a7fc640 1 -- 172.21.15.16:0/3892025791 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f84100aa840 con 0x7f841406aba0 2024-04-25T10:12:38.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.019+0000 7f84192bd640 1 --2- 172.21.15.16:0/3892025791 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f83f806b500 0x7f83f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.019+0000 7f84192bd640 1 --2- 172.21.15.16:0/3892025791 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f83f806b500 0x7f83f806d9c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f84140aeaf0 tx=0x7f8404009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.023+0000 7f840a7fc640 1 -- 172.21.15.16:0/3892025791 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84100797c0 con 0x7f841406aba0 2024-04-25T10:12:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.205+0000 7f841bd22640 1 -- 172.21.15.16:0/3892025791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84140ad860 con 0x7f841406aba0 2024-04-25T10:12:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.206+0000 7f840a7fc640 1 -- 172.21.15.16:0/3892025791 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16935 (secure 0 0 0) 0x7f841002ae20 con 0x7f841406aba0 2024-04-25T10:12:38.211 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:38.211 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:11:55.231494+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:58.299405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:58.994953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.186:6836/3950732718":"2024-04-26T10:11:55.231471+0000","172.21.15.186:6834/3950732718":"2024-04-26T10:11:55.231471+0000","172.21.15.186:6837/3092291936":"2024-04-26T10:11:13.827348+0000","172.21.15.186:6835/3092291936":"2024-04-26T10:11:13.827348+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.208+0000 7f841bd22640 1 -- 172.21.15.16:0/3892025791 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f83f806b500 msgr2=0x7f83f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.208+0000 7f841bd22640 1 --2- 172.21.15.16:0/3892025791 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f83f806b500 0x7f83f806d9c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f84140aeaf0 tx=0x7f8404009040 comp rx=0 tx=0).stop 2024-04-25T10:12:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.208+0000 7f841bd22640 1 -- 172.21.15.16:0/3892025791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841406aba0 msgr2=0x7f84140c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.208+0000 7f841bd22640 1 --2- 172.21.15.16:0/3892025791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841406aba0 0x7f84140c9dc0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f8410000c00 tx=0x7f84100046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.208+0000 7f841bd22640 1 -- 172.21.15.16:0/3892025791 shutdown_connections 2024-04-25T10:12:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.208+0000 7f841bd22640 1 --2- 172.21.15.16:0/3892025791 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f83f806b500 0x7f83f806d9c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.208+0000 7f841bd22640 1 --2- 172.21.15.16:0/3892025791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f841415bb90 0x7f841415a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.208+0000 7f841bd22640 1 --2- 172.21.15.16:0/3892025791 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f84140cc950 0x7f84140c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.208+0000 7f841bd22640 1 --2- 172.21.15.16:0/3892025791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841406aba0 0x7f84140c9dc0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.208+0000 7f841bd22640 1 -- 172.21.15.16:0/3892025791 >> 172.21.15.16:0/3892025791 conn(0x7f84140d06b0 msgr2=0x7f84140b52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.208+0000 7f841bd22640 1 -- 172.21.15.16:0/3892025791 shutdown_connections 2024-04-25T10:12:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.208+0000 7f841bd22640 1 -- 172.21.15.16:0/3892025791 wait complete. 2024-04-25T10:12:38.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:12:38.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.321+0000 7f6e17e4a640 1 Processor -- start 2024-04-25T10:12:38.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.321+0000 7f6e17e4a640 1 -- start start 2024-04-25T10:12:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.322+0000 7f6e17e4a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e10070080 0x7f6e10162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.322+0000 7f6e17e4a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e100d01a0 con 0x7f6e1006b2b0 2024-04-25T10:12:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.322+0000 7f6e17e4a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e10161950 con 0x7f6e100cc9a0 2024-04-25T10:12:38.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.322+0000 7f6e17e4a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e10161c30 con 0x7f6e10070080 2024-04-25T10:12:38.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.322+0000 7f6e15be6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e10070080 0x7f6e10162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.322+0000 7f6e15be6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e10070080 0x7f6e10162430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32902/0 (socket says 172.21.15.16:32902) 2024-04-25T10:12:38.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.322+0000 7f6e15be6640 1 -- 172.21.15.16:0/2201127416 learned_addr learned my addr 172.21.15.16:0/2201127416 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:38.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.323+0000 7f6e15be6640 1 -- 172.21.15.16:0/2201127416 >> v1:172.21.15.186:6789/0 conn(0x7f6e100cc9a0 legacy=0x7f6e1006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.323+0000 7f6e15be6640 1 -- 172.21.15.16:0/2201127416 >> v1:172.21.15.16:6789/0 conn(0x7f6e1006b2b0 legacy=0x7f6e1006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.323+0000 7f6e14be4640 1 -- 172.21.15.16:0/2201127416 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442346499 0 0) 0x7f6e100d01a0 con 0x7f6e1006b2b0 2024-04-25T10:12:38.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.323+0000 7f6e15be6640 1 -- 172.21.15.16:0/2201127416 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e10070460 con 0x7f6e10070080 2024-04-25T10:12:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.324+0000 7f6e15be6640 1 --2- 172.21.15.16:0/2201127416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e10070080 0x7f6e10162430 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009870 tx=0x7f6e0802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facb88a325c4fd27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.324+0000 7f6e14be4640 1 -- 172.21.15.16:0/2201127416 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08037070 con 0x7f6e10070080 2024-04-25T10:12:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.324+0000 7f6e14be4640 1 -- 172.21.15.16:0/2201127416 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e08032440 con 0x7f6e10070080 2024-04-25T10:12:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.324+0000 7f6e14be4640 1 -- 172.21.15.16:0/2201127416 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08032810 con 0x7f6e10070080 2024-04-25T10:12:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.324+0000 7f6e17e4a640 1 -- 172.21.15.16:0/2201127416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e10070080 msgr2=0x7f6e10162430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.324+0000 7f6e17e4a640 1 --2- 172.21.15.16:0/2201127416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e10070080 0x7f6e10162430 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009870 tx=0x7f6e0802a270 comp rx=0 tx=0).stop 2024-04-25T10:12:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.324+0000 7f6e17e4a640 1 -- 172.21.15.16:0/2201127416 shutdown_connections 2024-04-25T10:12:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.324+0000 7f6e17e4a640 1 --2- 172.21.15.16:0/2201127416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e10070080 0x7f6e10162430 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.324+0000 7f6e17e4a640 1 -- 172.21.15.16:0/2201127416 >> 172.21.15.16:0/2201127416 conn(0x7f6e10099a40 msgr2=0x7f6e100d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.324+0000 7f6e17e4a640 1 -- 172.21.15.16:0/2201127416 shutdown_connections 2024-04-25T10:12:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.325+0000 7f6e17e4a640 1 -- 172.21.15.16:0/2201127416 wait complete. 2024-04-25T10:12:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.326+0000 7f6e17e4a640 1 Processor -- start 2024-04-25T10:12:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.326+0000 7f6e17e4a640 1 -- start start 2024-04-25T10:12:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.326+0000 7f6e17e4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e1006b2b0 0x7f6e1014fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.326+0000 7f6e17e4a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e10070080 0x7f6e10150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.326+0000 7f6e17e4a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e100cc9a0 0x7f6e10160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.326+0000 7f6e17e4a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e100ce6f0 con 0x7f6e1006b2b0 2024-04-25T10:12:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.326+0000 7f6e17e4a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e100ce570 con 0x7f6e100cc9a0 2024-04-25T10:12:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.326+0000 7f6e17e4a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e100cfb20 con 0x7f6e10070080 2024-04-25T10:12:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.326+0000 7f6e15be6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e1006b2b0 0x7f6e1014fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.326+0000 7f6e15be6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e1006b2b0 0x7f6e1014fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32916/0 (socket says 172.21.15.16:32916) 2024-04-25T10:12:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.326+0000 7f6e15be6640 1 -- 172.21.15.16:0/1820615789 learned_addr learned my addr 172.21.15.16:0/1820615789 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.327+0000 7f6e153e5640 1 --2- 172.21.15.16:0/1820615789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e10070080 0x7f6e10150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.327+0000 7f6e163e7640 1 --2- 172.21.15.16:0/1820615789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e100cc9a0 0x7f6e10160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.327+0000 7f6e15be6640 1 -- 172.21.15.16:0/1820615789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e10070080 msgr2=0x7f6e10150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.327+0000 7f6e15be6640 1 --2- 172.21.15.16:0/1820615789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e10070080 0x7f6e10150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.327+0000 7f6e15be6640 1 -- 172.21.15.16:0/1820615789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e100cc9a0 msgr2=0x7f6e10160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.327+0000 7f6e15be6640 1 --2- 172.21.15.16:0/1820615789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e100cc9a0 0x7f6e10160500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.327+0000 7f6e15be6640 1 -- 172.21.15.16:0/1820615789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e100c44b0 con 0x7f6e1006b2b0 2024-04-25T10:12:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.327+0000 7f6e163e7640 1 --2- 172.21.15.16:0/1820615789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e100cc9a0 0x7f6e10160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.327+0000 7f6e15be6640 1 --2- 172.21.15.16:0/1820615789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e1006b2b0 0x7f6e1014fe00 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009870 tx=0x7f6e080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.327+0000 7f6e067fc640 1 -- 172.21.15.16:0/1820615789 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08041020 con 0x7f6e1006b2b0 2024-04-25T10:12:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.327+0000 7f6e067fc640 1 -- 172.21.15.16:0/1820615789 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e08030970 con 0x7f6e1006b2b0 2024-04-25T10:12:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.327+0000 7f6e067fc640 1 -- 172.21.15.16:0/1820615789 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08037040 con 0x7f6e1006b2b0 2024-04-25T10:12:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.327+0000 7f6e17e4a640 1 -- 172.21.15.16:0/1820615789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e100c4790 con 0x7f6e1006b2b0 2024-04-25T10:12:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.327+0000 7f6e17e4a640 1 -- 172.21.15.16:0/1820615789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e100c4d50 con 0x7f6e1006b2b0 2024-04-25T10:12:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.328+0000 7f6e17e4a640 1 -- 172.21.15.16:0/1820615789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e100750b0 con 0x7f6e1006b2b0 2024-04-25T10:12:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.328+0000 7f6e067fc640 1 -- 172.21.15.16:0/1820615789 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e08044020 con 0x7f6e1006b2b0 2024-04-25T10:12:38.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.329+0000 7f6e067fc640 1 --2- 172.21.15.16:0/1820615789 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6df406b4d0 0x7f6df406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.329+0000 7f6e067fc640 1 -- 172.21.15.16:0/1820615789 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f6e080aacc0 con 0x7f6e1006b2b0 2024-04-25T10:12:38.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.329+0000 7f6e153e5640 1 --2- 172.21.15.16:0/1820615789 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6df406b4d0 0x7f6df406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.330+0000 7f6e153e5640 1 --2- 172.21.15.16:0/1820615789 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6df406b4d0 0x7f6df406d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6e101e8250 tx=0x7f6e00009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.333+0000 7f6e067fc640 1 -- 172.21.15.16:0/1820615789 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e0804b030 con 0x7f6e1006b2b0 2024-04-25T10:12:38.476 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:38.479 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:38.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.553+0000 7f6e17e4a640 1 -- 172.21.15.16:0/1820615789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6e10099fd0 con 0x7f6e1006b2b0 2024-04-25T10:12:38.558 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-04-25T10:12:38.553+0000 7ff13f9bb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:12:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.555+0000 7f6e067fc640 1 -- 172.21.15.16:0/1820615789 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v65) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6e08079a90 con 0x7f6e1006b2b0 2024-04-25T10:12:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:12:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.558+0000 7f6e17e4a640 1 -- 172.21.15.16:0/1820615789 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6df406b4d0 msgr2=0x7f6df406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.558+0000 7f6e17e4a640 1 --2- 172.21.15.16:0/1820615789 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6df406b4d0 0x7f6df406d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6e101e8250 tx=0x7f6e00009040 comp rx=0 tx=0).stop 2024-04-25T10:12:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.558+0000 7f6e17e4a640 1 -- 172.21.15.16:0/1820615789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e1006b2b0 msgr2=0x7f6e1014fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.558+0000 7f6e17e4a640 1 --2- 172.21.15.16:0/1820615789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e1006b2b0 0x7f6e1014fe00 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009870 tx=0x7f6e080046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.558+0000 7f6e17e4a640 1 -- 172.21.15.16:0/1820615789 shutdown_connections 2024-04-25T10:12:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.558+0000 7f6e17e4a640 1 --2- 172.21.15.16:0/1820615789 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6df406b4d0 0x7f6df406d990 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.558+0000 7f6e17e4a640 1 --2- 172.21.15.16:0/1820615789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e100cc9a0 0x7f6e10160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.558+0000 7f6e17e4a640 1 --2- 172.21.15.16:0/1820615789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e10070080 0x7f6e10150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.558+0000 7f6e17e4a640 1 --2- 172.21.15.16:0/1820615789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e1006b2b0 0x7f6e1014fe00 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.558+0000 7f6e17e4a640 1 -- 172.21.15.16:0/1820615789 >> 172.21.15.16:0/1820615789 conn(0x7f6e10099a40 msgr2=0x7f6e100b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.558+0000 7f6e17e4a640 1 -- 172.21.15.16:0/1820615789 shutdown_connections 2024-04-25T10:12:38.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.558+0000 7f6e17e4a640 1 -- 172.21.15.16:0/1820615789 wait complete. 2024-04-25T10:12:38.565 INFO:tasks.ceph.mds.a.smithi016.stderr: -2201> 2024-04-25T10:12:01.638+0000 7f82eb6d7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:12:38.565 INFO:tasks.ceph.mds.a.smithi016.stderr: -1909> 2024-04-25T10:12:01.894+0000 7f82e76cf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x563b8cd90680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:12:38.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:12:38.592 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-04-25T10:12:38.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.689+0000 7f2cc239e640 1 Processor -- start 2024-04-25T10:12:38.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.689+0000 7f2cc239e640 1 -- start start 2024-04-25T10:12:38.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.690+0000 7f2cc239e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cbc0b42c0 0x7f2cbc15ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.690+0000 7f2cc239e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cbc15aef0 0x7f2cbc0cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.690+0000 7f2cc239e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cbc15b370 con 0x7f2cbc0b46a0 2024-04-25T10:12:38.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.690+0000 7f2cc239e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc0d05d0 con 0x7f2cbc15aef0 2024-04-25T10:12:38.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.690+0000 7f2cc239e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc15b650 con 0x7f2cbc0b42c0 2024-04-25T10:12:38.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.690+0000 7f2cbb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cbc15aef0 0x7f2cbc0cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.690+0000 7f2cc093b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2cbc0b46a0 0x7f2cbc1549d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49208/0 (socket says 172.21.15.16:49208) 2024-04-25T10:12:38.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.690+0000 7f2cc093b640 1 -- 172.21.15.16:0/1386327514 learned_addr learned my addr 172.21.15.16:0/1386327514 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:38.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.690+0000 7f2cbbfff640 1 --2- 172.21.15.16:0/1386327514 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cbc0b42c0 0x7f2cbc15ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.691+0000 7f2cbb7fe640 1 -- 172.21.15.16:0/1386327514 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cbc0b42c0 msgr2=0x7f2cbc15ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.691+0000 7f2cbb7fe640 1 --2- 172.21.15.16:0/1386327514 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cbc0b42c0 0x7f2cbc15ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.691+0000 7f2cbb7fe640 1 -- 172.21.15.16:0/1386327514 >> v1:172.21.15.16:6789/0 conn(0x7f2cbc0b46a0 legacy=0x7f2cbc1549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.691+0000 7f2cbb7fe640 1 -- 172.21.15.16:0/1386327514 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc15b930 con 0x7f2cbc15aef0 2024-04-25T10:12:38.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.691+0000 7f2cbaffd640 1 -- 172.21.15.16:0/1386327514 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503767036 0 0) 0x7f2cbc15b370 con 0x7f2cbc0b46a0 2024-04-25T10:12:38.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.691+0000 7f2cbb7fe640 1 --2- 172.21.15.16:0/1386327514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cbc15aef0 0x7f2cbc0cf440 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40097a0 tx=0x7f2ca402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1f2c7e60c9f14a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.691+0000 7f2cbaffd640 1 -- 172.21.15.16:0/1386327514 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4037070 con 0x7f2cbc15aef0 2024-04-25T10:12:38.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.691+0000 7f2cbaffd640 1 -- 172.21.15.16:0/1386327514 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca402adb0 con 0x7f2cbc15aef0 2024-04-25T10:12:38.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.692+0000 7f2cbaffd640 1 -- 172.21.15.16:0/1386327514 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4032660 con 0x7f2cbc15aef0 2024-04-25T10:12:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.692+0000 7f2cc239e640 1 -- 172.21.15.16:0/1386327514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cbc15aef0 msgr2=0x7f2cbc0cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.692+0000 7f2cc239e640 1 --2- 172.21.15.16:0/1386327514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cbc15aef0 0x7f2cbc0cf440 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40097a0 tx=0x7f2ca402a060 comp rx=0 tx=0).stop 2024-04-25T10:12:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.692+0000 7f2cc239e640 1 -- 172.21.15.16:0/1386327514 shutdown_connections 2024-04-25T10:12:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.692+0000 7f2cc239e640 1 --2- 172.21.15.16:0/1386327514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cbc15aef0 0x7f2cbc0cf440 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.692+0000 7f2cc239e640 1 --2- 172.21.15.16:0/1386327514 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cbc0b42c0 0x7f2cbc15ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.692+0000 7f2cc239e640 1 -- 172.21.15.16:0/1386327514 >> 172.21.15.16:0/1386327514 conn(0x7f2cbc06aba0 msgr2=0x7f2cbc06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.692+0000 7f2cc239e640 1 -- 172.21.15.16:0/1386327514 shutdown_connections 2024-04-25T10:12:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.692+0000 7f2cc239e640 1 -- 172.21.15.16:0/1386327514 wait complete. 2024-04-25T10:12:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.693+0000 7f2cc239e640 1 Processor -- start 2024-04-25T10:12:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.693+0000 7f2cc239e640 1 -- start start 2024-04-25T10:12:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.694+0000 7f2cc239e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cbc0b42c0 0x7f2cbc1518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.694+0000 7f2cc239e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cbc0b46a0 0x7f2cbc156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.694+0000 7f2cc239e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cbc15aef0 0x7f2cbc1e4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.694+0000 7f2cc239e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc06eae0 con 0x7f2cbc0b46a0 2024-04-25T10:12:38.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.694+0000 7f2cc239e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0cf980 con 0x7f2cbc0b42c0 2024-04-25T10:12:38.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.694+0000 7f2cc239e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc06ec60 con 0x7f2cbc15aef0 2024-04-25T10:12:38.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.694+0000 7f2cbb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cbc0b46a0 0x7f2cbc156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.694+0000 7f2cbb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cbc0b46a0 0x7f2cbc156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32950/0 (socket says 172.21.15.16:32950) 2024-04-25T10:12:38.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.694+0000 7f2cbb7fe640 1 -- 172.21.15.16:0/703851122 learned_addr learned my addr 172.21.15.16:0/703851122 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:38.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.694+0000 7f2cbbfff640 1 --2- 172.21.15.16:0/703851122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cbc0b42c0 0x7f2cbc1518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.694+0000 7f2cc093b640 1 --2- 172.21.15.16:0/703851122 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cbc15aef0 0x7f2cbc1e4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.695+0000 7f2cbb7fe640 1 -- 172.21.15.16:0/703851122 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cbc15aef0 msgr2=0x7f2cbc1e4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.695+0000 7f2cbb7fe640 1 --2- 172.21.15.16:0/703851122 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cbc15aef0 0x7f2cbc1e4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.695+0000 7f2cbb7fe640 1 -- 172.21.15.16:0/703851122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cbc0b42c0 msgr2=0x7f2cbc1518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.695+0000 7f2cbb7fe640 1 --2- 172.21.15.16:0/703851122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cbc0b42c0 0x7f2cbc1518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.695+0000 7f2cbb7fe640 1 -- 172.21.15.16:0/703851122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc1567d0 con 0x7f2cbc0b46a0 2024-04-25T10:12:38.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.695+0000 7f2cbb7fe640 1 --2- 172.21.15.16:0/703851122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cbc0b46a0 0x7f2cbc156290 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4035040 tx=0x7f2ca40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.695+0000 7f2cb8ff9640 1 -- 172.21.15.16:0/703851122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4041070 con 0x7f2cbc0b46a0 2024-04-25T10:12:38.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.695+0000 7f2cb8ff9640 1 -- 172.21.15.16:0/703851122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca403c460 con 0x7f2cbc0b46a0 2024-04-25T10:12:38.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.695+0000 7f2cc239e640 1 -- 172.21.15.16:0/703851122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc151f30 con 0x7f2cbc0b46a0 2024-04-25T10:12:38.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.695+0000 7f2cb8ff9640 1 -- 172.21.15.16:0/703851122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4037040 con 0x7f2cbc0b46a0 2024-04-25T10:12:38.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.695+0000 7f2cc239e640 1 -- 172.21.15.16:0/703851122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc1527d0 con 0x7f2cbc0b46a0 2024-04-25T10:12:38.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.696+0000 7f2cc239e640 1 -- 172.21.15.16:0/703851122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cbc0750b0 con 0x7f2cbc0b46a0 2024-04-25T10:12:38.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.696+0000 7f2cb8ff9640 1 -- 172.21.15.16:0/703851122 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ca403a070 con 0x7f2cbc0b46a0 2024-04-25T10:12:38.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.697+0000 7f2cb8ff9640 1 --2- 172.21.15.16:0/703851122 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2c9c06b4d0 0x7f2c9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.697+0000 7f2cb8ff9640 1 -- 172.21.15.16:0/703851122 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f2ca40aa700 con 0x7f2cbc0b46a0 2024-04-25T10:12:38.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.697+0000 7f2cbbfff640 1 --2- 172.21.15.16:0/703851122 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2c9c06b4d0 0x7f2c9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.698+0000 7f2cbbfff640 1 --2- 172.21.15.16:0/703851122 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2c9c06b4d0 0x7f2c9c06d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009b40 tx=0x7f2cac009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.701+0000 7f2cb8ff9640 1 -- 172.21.15.16:0/703851122 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2cbc1567d0 con 0x7f2cbc0b46a0 2024-04-25T10:12:38.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:38.921+0000 7f2cc239e640 1 -- 172.21.15.16:0/703851122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2cbc0b17f0 con 0x7f2cbc0b46a0 2024-04-25T10:12:39.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.556+0000 7f2cb8ff9640 1 -- 172.21.15.16:0/703851122 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v66) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2ca4078e30 con 0x7f2cbc0b46a0 2024-04-25T10:12:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.559+0000 7f2cc239e640 1 -- 172.21.15.16:0/703851122 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2c9c06b4d0 msgr2=0x7f2c9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.559+0000 7f2cc239e640 1 --2- 172.21.15.16:0/703851122 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2c9c06b4d0 0x7f2c9c06d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009b40 tx=0x7f2cac009190 comp rx=0 tx=0).stop 2024-04-25T10:12:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.559+0000 7f2cc239e640 1 -- 172.21.15.16:0/703851122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cbc0b46a0 msgr2=0x7f2cbc156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.559+0000 7f2cc239e640 1 --2- 172.21.15.16:0/703851122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cbc0b46a0 0x7f2cbc156290 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4035040 tx=0x7f2ca40046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.559+0000 7f2cc239e640 1 -- 172.21.15.16:0/703851122 shutdown_connections 2024-04-25T10:12:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.559+0000 7f2cc239e640 1 --2- 172.21.15.16:0/703851122 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2c9c06b4d0 0x7f2c9c06d990 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.559+0000 7f2cc239e640 1 --2- 172.21.15.16:0/703851122 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cbc15aef0 0x7f2cbc1e4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.559+0000 7f2cc239e640 1 --2- 172.21.15.16:0/703851122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cbc0b46a0 0x7f2cbc156290 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.559+0000 7f2cc239e640 1 --2- 172.21.15.16:0/703851122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cbc0b42c0 0x7f2cbc1518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.559+0000 7f2cc239e640 1 -- 172.21.15.16:0/703851122 >> 172.21.15.16:0/703851122 conn(0x7f2cbc06aba0 msgr2=0x7f2cbc15a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.559+0000 7f2cc239e640 1 -- 172.21.15.16:0/703851122 shutdown_connections 2024-04-25T10:12:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.559+0000 7f2cc239e640 1 -- 172.21.15.16:0/703851122 wait complete. 2024-04-25T10:12:39.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:12:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.684+0000 7f16325e0640 1 Processor -- start 2024-04-25T10:12:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.684+0000 7f16325e0640 1 -- start start 2024-04-25T10:12:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.685+0000 7f16325e0640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f162c15dc50 0x7f162c070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.685+0000 7f16325e0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f162c0cd940 0x7f162c0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.685+0000 7f16325e0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f162c0cf600 0x7f162c161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.685+0000 7f16325e0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162c0cfe00 con 0x7f162c0cf600 2024-04-25T10:12:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.685+0000 7f16325e0640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162c06c220 con 0x7f162c15dc50 2024-04-25T10:12:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.685+0000 7f16325e0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162c0cec10 con 0x7f162c0cd940 2024-04-25T10:12:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.685+0000 7f1630b7d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f162c0cf600 0x7f162c161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.685+0000 7f1630b7d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f162c0cf600 0x7f162c161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32960/0 (socket says 172.21.15.16:32960) 2024-04-25T10:12:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.685+0000 7f1630b7d640 1 -- 172.21.15.16:0/2134955961 learned_addr learned my addr 172.21.15.16:0/2134955961 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.685+0000 7f162bfff640 1 --2- 172.21.15.16:0/2134955961 >> v2:172.21.15.186:3300/0 conn(0x7f162c15dc50 0x7f162c070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.685+0000 7f162b7fe640 1 --2- 172.21.15.16:0/2134955961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f162c0cd940 0x7f162c0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.685+0000 7f1630b7d640 1 -- 172.21.15.16:0/2134955961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f162c0cd940 msgr2=0x7f162c0713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.685+0000 7f1630b7d640 1 --2- 172.21.15.16:0/2134955961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f162c0cd940 0x7f162c0713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.685+0000 7f1630b7d640 1 -- 172.21.15.16:0/2134955961 >> v2:172.21.15.186:3300/0 conn(0x7f162c15dc50 msgr2=0x7f162c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.685+0000 7f1630b7d640 1 --2- 172.21.15.16:0/2134955961 >> v2:172.21.15.186:3300/0 conn(0x7f162c15dc50 0x7f162c070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.685+0000 7f1630b7d640 1 -- 172.21.15.16:0/2134955961 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162c0d0050 con 0x7f162c0cf600 2024-04-25T10:12:39.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.686+0000 7f1630b7d640 1 --2- 172.21.15.16:0/2134955961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f162c0cf600 0x7f162c161870 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f162c072fc0 tx=0x7f162402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4068c1ec39ad1c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.686+0000 7f162affd640 1 -- 172.21.15.16:0/2134955961 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624036070 con 0x7f162c0cf600 2024-04-25T10:12:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.686+0000 7f162affd640 1 -- 172.21.15.16:0/2134955961 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1624031600 con 0x7f162c0cf600 2024-04-25T10:12:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.686+0000 7f162affd640 1 -- 172.21.15.16:0/2134955961 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16240319d0 con 0x7f162c0cf600 2024-04-25T10:12:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.687+0000 7f16325e0640 1 -- 172.21.15.16:0/2134955961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f162c0cf600 msgr2=0x7f162c161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.687+0000 7f16325e0640 1 --2- 172.21.15.16:0/2134955961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f162c0cf600 0x7f162c161870 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f162c072fc0 tx=0x7f162402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.687+0000 7f16325e0640 1 -- 172.21.15.16:0/2134955961 shutdown_connections 2024-04-25T10:12:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.687+0000 7f16325e0640 1 --2- 172.21.15.16:0/2134955961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f162c0cf600 0x7f162c161870 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.687+0000 7f16325e0640 1 --2- 172.21.15.16:0/2134955961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f162c0cd940 0x7f162c0713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.687+0000 7f16325e0640 1 --2- 172.21.15.16:0/2134955961 >> v2:172.21.15.186:3300/0 conn(0x7f162c15dc50 0x7f162c070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.687+0000 7f16325e0640 1 -- 172.21.15.16:0/2134955961 >> 172.21.15.16:0/2134955961 conn(0x7f162c06f430 msgr2=0x7f162c06f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.687+0000 7f16325e0640 1 -- 172.21.15.16:0/2134955961 shutdown_connections 2024-04-25T10:12:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.687+0000 7f16325e0640 1 -- 172.21.15.16:0/2134955961 wait complete. 2024-04-25T10:12:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.688+0000 7f16325e0640 1 Processor -- start 2024-04-25T10:12:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.688+0000 7f16325e0640 1 -- start start 2024-04-25T10:12:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.689+0000 7f16325e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f162c0cd940 0x7f162c0c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.689+0000 7f16325e0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f162c0cf600 0x7f162c156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.689+0000 7f16325e0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f162c15dc50 0x7f162c0c5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.689+0000 7f16325e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c0ce8a0 con 0x7f162c0cd940 2024-04-25T10:12:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.689+0000 7f16325e0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c0cc0c0 con 0x7f162c15dc50 2024-04-25T10:12:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.689+0000 7f16325e0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162c15e030 con 0x7f162c0cf600 2024-04-25T10:12:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.689+0000 7f162bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f162c0cd940 0x7f162c0c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.689+0000 7f162bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f162c0cd940 0x7f162c0c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32968/0 (socket says 172.21.15.16:32968) 2024-04-25T10:12:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.689+0000 7f162bfff640 1 -- 172.21.15.16:0/2732460662 learned_addr learned my addr 172.21.15.16:0/2732460662 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.689+0000 7f162b7fe640 1 --2- 172.21.15.16:0/2732460662 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f162c0cf600 0x7f162c156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.689+0000 7f1630b7d640 1 --2- 172.21.15.16:0/2732460662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f162c15dc50 0x7f162c0c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.689+0000 7f162bfff640 1 -- 172.21.15.16:0/2732460662 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f162c0cf600 msgr2=0x7f162c156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.689+0000 7f162bfff640 1 --2- 172.21.15.16:0/2732460662 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f162c0cf600 0x7f162c156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.689+0000 7f162bfff640 1 -- 172.21.15.16:0/2732460662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f162c15dc50 msgr2=0x7f162c0c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.689+0000 7f162bfff640 1 --2- 172.21.15.16:0/2732460662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f162c15dc50 0x7f162c0c5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.689+0000 7f162bfff640 1 -- 172.21.15.16:0/2732460662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162c157620 con 0x7f162c0cd940 2024-04-25T10:12:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.690+0000 7f162bfff640 1 --2- 172.21.15.16:0/2732460662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f162c0cd940 0x7f162c0c7d20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f161400e9e0 tx=0x7f161400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.690+0000 7f1628ff9640 1 -- 172.21.15.16:0/2732460662 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614017070 con 0x7f162c0cd940 2024-04-25T10:12:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.690+0000 7f1628ff9640 1 -- 172.21.15.16:0/2732460662 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161400c9d0 con 0x7f162c0cd940 2024-04-25T10:12:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.690+0000 7f1628ff9640 1 -- 172.21.15.16:0/2732460662 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161400cdc0 con 0x7f162c0cd940 2024-04-25T10:12:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.690+0000 7f16325e0640 1 -- 172.21.15.16:0/2732460662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162c157900 con 0x7f162c0cd940 2024-04-25T10:12:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.690+0000 7f16325e0640 1 -- 172.21.15.16:0/2732460662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162c157ec0 con 0x7f162c0cd940 2024-04-25T10:12:39.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.691+0000 7f16325e0640 1 -- 172.21.15.16:0/2732460662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162c0750b0 con 0x7f162c0cd940 2024-04-25T10:12:39.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.691+0000 7f1628ff9640 1 -- 172.21.15.16:0/2732460662 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f161401a070 con 0x7f162c0cd940 2024-04-25T10:12:39.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.692+0000 7f1628ff9640 1 --2- 172.21.15.16:0/2732460662 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f160c06b4d0 0x7f160c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.692+0000 7f1628ff9640 1 -- 172.21.15.16:0/2732460662 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f161408a5e0 con 0x7f162c0cd940 2024-04-25T10:12:39.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.692+0000 7f162b7fe640 1 --2- 172.21.15.16:0/2732460662 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f160c06b4d0 0x7f160c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:39.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.692+0000 7f162b7fe640 1 --2- 172.21.15.16:0/2732460662 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f160c06b4d0 0x7f160c06d990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f162c0d2480 tx=0x7f161c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:39.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.696+0000 7f1628ff9640 1 -- 172.21.15.16:0/2732460662 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16140593f0 con 0x7f162c0cd940 2024-04-25T10:12:39.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:39.875+0000 7f16325e0640 1 -- 172.21.15.16:0/2732460662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f162c0b77a0 con 0x7f162c0cd940 2024-04-25T10:12:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.561+0000 7f1628ff9640 1 -- 172.21.15.16:0/2732460662 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1614058cf0 con 0x7f162c0cd940 2024-04-25T10:12:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:12:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.565+0000 7f16325e0640 1 -- 172.21.15.16:0/2732460662 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f160c06b4d0 msgr2=0x7f160c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.565+0000 7f16325e0640 1 --2- 172.21.15.16:0/2732460662 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f160c06b4d0 0x7f160c06d990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f162c0d2480 tx=0x7f161c008040 comp rx=0 tx=0).stop 2024-04-25T10:12:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.565+0000 7f16325e0640 1 -- 172.21.15.16:0/2732460662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f162c0cd940 msgr2=0x7f162c0c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.565+0000 7f16325e0640 1 --2- 172.21.15.16:0/2732460662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f162c0cd940 0x7f162c0c7d20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f161400e9e0 tx=0x7f161400eea0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.566+0000 7f16325e0640 1 -- 172.21.15.16:0/2732460662 shutdown_connections 2024-04-25T10:12:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.566+0000 7f16325e0640 1 --2- 172.21.15.16:0/2732460662 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f160c06b4d0 0x7f160c06d990 secure :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f162c0d2480 tx=0x7f161c008040 comp rx=0 tx=0).stop 2024-04-25T10:12:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.566+0000 7f16325e0640 1 --2- 172.21.15.16:0/2732460662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f162c15dc50 0x7f162c0c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.566+0000 7f16325e0640 1 --2- 172.21.15.16:0/2732460662 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f162c0cf600 0x7f162c156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.566+0000 7f16325e0640 1 --2- 172.21.15.16:0/2732460662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f162c0cd940 0x7f162c0c7d20 secure :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f161400e9e0 tx=0x7f161400eea0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.566+0000 7f16325e0640 1 -- 172.21.15.16:0/2732460662 >> 172.21.15.16:0/2732460662 conn(0x7f162c06f430 msgr2=0x7f162c0b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.566+0000 7f16325e0640 1 -- 172.21.15.16:0/2732460662 shutdown_connections 2024-04-25T10:12:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.566+0000 7f16325e0640 1 -- 172.21.15.16:0/2732460662 wait complete. 2024-04-25T10:12:40.586 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:12:40.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.692+0000 7f86277e0640 1 Processor -- start 2024-04-25T10:12:40.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.693+0000 7f86277e0640 1 -- start start 2024-04-25T10:12:40.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.693+0000 7f86277e0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f862006b040 0x7f86200ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.693+0000 7f86277e0640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f862006b420 0x7f86200b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.693+0000 7f86277e0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f86200b2ee0 0x7f8620161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.693+0000 7f86277e0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86200cf980 con 0x7f86200b2ee0 2024-04-25T10:12:40.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.693+0000 7f86277e0640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86200cfc20 con 0x7f862006b420 2024-04-25T10:12:40.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.693+0000 7f86277e0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86200cff00 con 0x7f862006b040 2024-04-25T10:12:40.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.694+0000 7f8625d7d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f86200b2ee0 0x7f8620161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:40.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.694+0000 7f8625d7d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f86200b2ee0 0x7f8620161f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32976/0 (socket says 172.21.15.16:32976) 2024-04-25T10:12:40.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.694+0000 7f8625d7d640 1 -- 172.21.15.16:0/3944936787 learned_addr learned my addr 172.21.15.16:0/3944936787 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:40.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.694+0000 7f862557c640 1 --2- 172.21.15.16:0/3944936787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f862006b040 0x7f86200ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:40.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.694+0000 7f8624d7b640 1 --2- 172.21.15.16:0/3944936787 >> v2:172.21.15.186:3300/0 conn(0x7f862006b420 0x7f86200b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:40.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.694+0000 7f8625d7d640 1 -- 172.21.15.16:0/3944936787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f862006b040 msgr2=0x7f86200ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:40.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.694+0000 7f8625d7d640 1 --2- 172.21.15.16:0/3944936787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f862006b040 0x7f86200ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.694+0000 7f8625d7d640 1 -- 172.21.15.16:0/3944936787 >> v2:172.21.15.186:3300/0 conn(0x7f862006b420 msgr2=0x7f86200b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:40.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.694+0000 7f8625d7d640 1 --2- 172.21.15.16:0/3944936787 >> v2:172.21.15.186:3300/0 conn(0x7f862006b420 0x7f86200b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.694+0000 7f8625d7d640 1 -- 172.21.15.16:0/3944936787 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86200d01e0 con 0x7f86200b2ee0 2024-04-25T10:12:40.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.695+0000 7f8625d7d640 1 --2- 172.21.15.16:0/3944936787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86200b2ee0 0x7f8620161f10 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f86200ce300 tx=0x7f861802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e4d271e3d8619 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:40.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.695+0000 7f8617fff640 1 -- 172.21.15.16:0/3944936787 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618036070 con 0x7f86200b2ee0 2024-04-25T10:12:40.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.695+0000 7f8617fff640 1 -- 172.21.15.16:0/3944936787 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8618031600 con 0x7f86200b2ee0 2024-04-25T10:12:40.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.695+0000 7f8617fff640 1 -- 172.21.15.16:0/3944936787 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86180319d0 con 0x7f86200b2ee0 2024-04-25T10:12:40.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.695+0000 7f86277e0640 1 -- 172.21.15.16:0/3944936787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86200b2ee0 msgr2=0x7f8620161f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.695+0000 7f86277e0640 1 --2- 172.21.15.16:0/3944936787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86200b2ee0 0x7f8620161f10 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f86200ce300 tx=0x7f861802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.695+0000 7f86277e0640 1 -- 172.21.15.16:0/3944936787 shutdown_connections 2024-04-25T10:12:40.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.695+0000 7f86277e0640 1 --2- 172.21.15.16:0/3944936787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86200b2ee0 0x7f8620161f10 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.695+0000 7f86277e0640 1 --2- 172.21.15.16:0/3944936787 >> v2:172.21.15.186:3300/0 conn(0x7f862006b420 0x7f86200b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.695+0000 7f86277e0640 1 --2- 172.21.15.16:0/3944936787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f862006b040 0x7f86200ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.695+0000 7f86277e0640 1 -- 172.21.15.16:0/3944936787 >> 172.21.15.16:0/3944936787 conn(0x7f8620099a40 msgr2=0x7f86200b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:40.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.695+0000 7f86277e0640 1 -- 172.21.15.16:0/3944936787 shutdown_connections 2024-04-25T10:12:40.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.695+0000 7f86277e0640 1 -- 172.21.15.16:0/3944936787 wait complete. 2024-04-25T10:12:40.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.696+0000 7f86277e0640 1 Processor -- start 2024-04-25T10:12:40.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.697+0000 7f86277e0640 1 -- start start 2024-04-25T10:12:40.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.697+0000 7f86277e0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f862006b040 0x7f86200b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.697+0000 7f86277e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f862006b420 0x7f86200b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.697+0000 7f86277e0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86200b2ee0 0x7f86201503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.697+0000 7f86277e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86200cd530 con 0x7f862006b420 2024-04-25T10:12:40.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.697+0000 7f86277e0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86200d04c0 con 0x7f86200b2ee0 2024-04-25T10:12:40.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.697+0000 7f86277e0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86200cd6b0 con 0x7f862006b040 2024-04-25T10:12:40.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.697+0000 7f8624d7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f862006b420 0x7f86200b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:40.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.697+0000 7f8624d7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f862006b420 0x7f86200b38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32980/0 (socket says 172.21.15.16:32980) 2024-04-25T10:12:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.697+0000 7f8624d7b640 1 -- 172.21.15.16:0/1494985057 learned_addr learned my addr 172.21.15.16:0/1494985057 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.697+0000 7f8625d7d640 1 --2- 172.21.15.16:0/1494985057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86200b2ee0 0x7f86201503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.697+0000 7f862557c640 1 --2- 172.21.15.16:0/1494985057 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f862006b040 0x7f86200b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.698+0000 7f8624d7b640 1 -- 172.21.15.16:0/1494985057 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f862006b040 msgr2=0x7f86200b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.698+0000 7f8624d7b640 1 --2- 172.21.15.16:0/1494985057 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f862006b040 0x7f86200b33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.698+0000 7f8624d7b640 1 -- 172.21.15.16:0/1494985057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86200b2ee0 msgr2=0x7f86201503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.698+0000 7f8624d7b640 1 --2- 172.21.15.16:0/1494985057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86200b2ee0 0x7f86201503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.698+0000 7f8624d7b640 1 -- 172.21.15.16:0/1494985057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86200b3f80 con 0x7f862006b420 2024-04-25T10:12:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.698+0000 7f8624d7b640 1 --2- 172.21.15.16:0/1494985057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f862006b420 0x7f86200b38e0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f861000d5f0 tx=0x7f861000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.698+0000 7f8615ffb640 1 -- 172.21.15.16:0/1494985057 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86100125f0 con 0x7f862006b420 2024-04-25T10:12:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.698+0000 7f8615ffb640 1 -- 172.21.15.16:0/1494985057 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8610019070 con 0x7f862006b420 2024-04-25T10:12:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.698+0000 7f86277e0640 1 -- 172.21.15.16:0/1494985057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86200b4260 con 0x7f862006b420 2024-04-25T10:12:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.698+0000 7f8615ffb640 1 -- 172.21.15.16:0/1494985057 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86100129d0 con 0x7f862006b420 2024-04-25T10:12:40.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.698+0000 7f86277e0640 1 -- 172.21.15.16:0/1494985057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86200b4820 con 0x7f862006b420 2024-04-25T10:12:40.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.699+0000 7f86277e0640 1 -- 172.21.15.16:0/1494985057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86200750b0 con 0x7f862006b420 2024-04-25T10:12:40.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.700+0000 7f8615ffb640 1 -- 172.21.15.16:0/1494985057 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8610012b70 con 0x7f862006b420 2024-04-25T10:12:40.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.700+0000 7f8615ffb640 1 --2- 172.21.15.16:0/1494985057 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f860406b4d0 0x7f860406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.700+0000 7f8615ffb640 1 -- 172.21.15.16:0/1494985057 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5529+0+0 (secure 0 0 0) 0x7f861008a460 con 0x7f862006b420 2024-04-25T10:12:40.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.700+0000 7f862557c640 1 --2- 172.21.15.16:0/1494985057 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f860406b4d0 0x7f860406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:40.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.701+0000 7f862557c640 1 --2- 172.21.15.16:0/1494985057 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f860406b4d0 0x7f860406d990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f86200b4f00 tx=0x7f861c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:40.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.705+0000 7f8615ffb640 1 -- 172.21.15.16:0/1494985057 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8610058d40 con 0x7f862006b420 2024-04-25T10:12:40.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:40.887+0000 7f86277e0640 1 -- 172.21.15.16:0/1494985057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f862006f0e0 con 0x7f862006b420 2024-04-25T10:12:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.562+0000 7f8615ffb640 1 -- 172.21.15.16:0/1494985057 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) v1 ==== 169+0+0 (secure 0 0 0) 0x7f861005cde0 con 0x7f862006b420 2024-04-25T10:12:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-04-25T10:12:41.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.567+0000 7f86277e0640 1 -- 172.21.15.16:0/1494985057 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f860406b4d0 msgr2=0x7f860406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.567+0000 7f86277e0640 1 --2- 172.21.15.16:0/1494985057 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f860406b4d0 0x7f860406d990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f86200b4f00 tx=0x7f861c008040 comp rx=0 tx=0).stop 2024-04-25T10:12:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.567+0000 7f86277e0640 1 -- 172.21.15.16:0/1494985057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f862006b420 msgr2=0x7f86200b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.567+0000 7f86277e0640 1 --2- 172.21.15.16:0/1494985057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f862006b420 0x7f86200b38e0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f861000d5f0 tx=0x7f861000dab0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.568+0000 7f86277e0640 1 -- 172.21.15.16:0/1494985057 shutdown_connections 2024-04-25T10:12:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.568+0000 7f86277e0640 1 --2- 172.21.15.16:0/1494985057 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f860406b4d0 0x7f860406d990 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.568+0000 7f86277e0640 1 --2- 172.21.15.16:0/1494985057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86200b2ee0 0x7f86201503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.568+0000 7f86277e0640 1 --2- 172.21.15.16:0/1494985057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f862006b420 0x7f86200b38e0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.568+0000 7f86277e0640 1 --2- 172.21.15.16:0/1494985057 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f862006b040 0x7f86200b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.568+0000 7f86277e0640 1 -- 172.21.15.16:0/1494985057 >> 172.21.15.16:0/1494985057 conn(0x7f8620099a40 msgr2=0x7f86200adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:41.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.569+0000 7f86277e0640 1 -- 172.21.15.16:0/1494985057 shutdown_connections 2024-04-25T10:12:41.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.569+0000 7f86277e0640 1 -- 172.21.15.16:0/1494985057 wait complete. 2024-04-25T10:12:41.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:12:41.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.696+0000 7f569f25d640 1 Processor -- start 2024-04-25T10:12:41.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.696+0000 7f569f25d640 1 -- start start 2024-04-25T10:12:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.696+0000 7f569f25d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5698161120 0x7f56981617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.696+0000 7f569f25d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f56980b5c60 0x7f56980d2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.696+0000 7f569f25d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5698161500 con 0x7f5698161ce0 2024-04-25T10:12:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.696+0000 7f569f25d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5698162740 con 0x7f56980b5c60 2024-04-25T10:12:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.696+0000 7f569f25d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5698162180 con 0x7f5698161120 2024-04-25T10:12:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f5697fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5698161ce0 0x7f56980b5550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49222/0 (socket says 172.21.15.16:49222) 2024-04-25T10:12:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f5697fff640 1 -- 172.21.15.16:0/270079404 learned_addr learned my addr 172.21.15.16:0/270079404 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f569cff9640 1 --2- 172.21.15.16:0/270079404 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5698161120 0x7f56981617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f569d7fa640 1 --2- 172.21.15.16:0/270079404 >> v2:172.21.15.186:3300/0 conn(0x7f56980b5c60 0x7f56980d2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f56977fe640 1 -- 172.21.15.16:0/270079404 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541226905 0 0) 0x7f5698161500 con 0x7f5698161ce0 2024-04-25T10:12:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f56977fe640 1 -- 172.21.15.16:0/270079404 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5688003520 con 0x7f5698161ce0 2024-04-25T10:12:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f56977fe640 1 -- 172.21.15.16:0/270079404 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1621466576 0 0) 0x7f5688003520 con 0x7f5698161ce0 2024-04-25T10:12:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f56977fe640 1 -- 172.21.15.16:0/270079404 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5698161120 msgr2=0x7f56981617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f56977fe640 1 --2- 172.21.15.16:0/270079404 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5698161120 0x7f56981617a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f56977fe640 1 -- 172.21.15.16:0/270079404 >> v2:172.21.15.186:3300/0 conn(0x7f56980b5c60 msgr2=0x7f56980d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f56977fe640 1 --2- 172.21.15.16:0/270079404 >> v2:172.21.15.186:3300/0 conn(0x7f56980b5c60 0x7f56980d2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f56977fe640 1 -- 172.21.15.16:0/270079404 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5698162460 con 0x7f5698161ce0 2024-04-25T10:12:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f56977fe640 1 -- 172.21.15.16:0/270079404 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f5684001670 con 0x7f5698161ce0 2024-04-25T10:12:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f56977fe640 1 -- 172.21.15.16:0/270079404 >> v1:172.21.15.16:6789/0 conn(0x7f5698161ce0 legacy=0x7f56980b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f56977fe640 1 --2- 172.21.15.16:0/270079404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56880039b0 0x7f5688003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f56977fe640 1 --2- 172.21.15.16:0/270079404 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5698161120 0x7f5688004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f56977fe640 1 --2- 172.21.15.16:0/270079404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5688004d40 0x7f56880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f56977fe640 1 -- 172.21.15.16:0/270079404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5698162740 con 0x7f5688004d40 2024-04-25T10:12:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f56977fe640 1 -- 172.21.15.16:0/270079404 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5698162180 con 0x7f56880039b0 2024-04-25T10:12:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f56977fe640 1 -- 172.21.15.16:0/270079404 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568800b9c0 con 0x7f5698161120 2024-04-25T10:12:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f569d7fa640 1 --2- 172.21.15.16:0/270079404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5688004d40 0x7f56880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f569cff9640 1 --2- 172.21.15.16:0/270079404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56880039b0 0x7f5688003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.697+0000 7f5697fff640 1 --2- 172.21.15.16:0/270079404 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5698161120 0x7f5688004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.698+0000 7f569d7fa640 1 -- 172.21.15.16:0/270079404 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5698161120 msgr2=0x7f5688004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.698+0000 7f569d7fa640 1 --2- 172.21.15.16:0/270079404 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5698161120 0x7f5688004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.698+0000 7f569d7fa640 1 -- 172.21.15.16:0/270079404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56880039b0 msgr2=0x7f5688003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.698+0000 7f569d7fa640 1 --2- 172.21.15.16:0/270079404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56880039b0 0x7f5688003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.698+0000 7f569d7fa640 1 -- 172.21.15.16:0/270079404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5688003520 con 0x7f5688004d40 2024-04-25T10:12:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.698+0000 7f569d7fa640 1 --2- 172.21.15.16:0/270079404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5688004d40 0x7f56880051c0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f5690009f90 tx=0x7f5688005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6032687a9a0a8dd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.698+0000 7f56977fe640 1 -- 172.21.15.16:0/270079404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690009c10 con 0x7f5688004d40 2024-04-25T10:12:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.699+0000 7f56977fe640 1 -- 172.21.15.16:0/270079404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5690036070 con 0x7f5688004d40 2024-04-25T10:12:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.699+0000 7f56977fe640 1 -- 172.21.15.16:0/270079404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690031880 con 0x7f5688004d40 2024-04-25T10:12:41.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.699+0000 7f569f25d640 1 -- 172.21.15.16:0/270079404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5688004d40 msgr2=0x7f56880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.699+0000 7f569f25d640 1 --2- 172.21.15.16:0/270079404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5688004d40 0x7f56880051c0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f5690009f90 tx=0x7f5688005830 comp rx=0 tx=0).stop 2024-04-25T10:12:41.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.699+0000 7f569d7fa640 1 -- 172.21.15.16:0/270079404 reap_dead start 2024-04-25T10:12:41.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.699+0000 7f569f25d640 1 -- 172.21.15.16:0/270079404 shutdown_connections 2024-04-25T10:12:41.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.699+0000 7f569f25d640 1 -- 172.21.15.16:0/270079404 >> 172.21.15.16:0/270079404 conn(0x7f56980d0550 msgr2=0x7f56980d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:41.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.699+0000 7f569f25d640 1 -- 172.21.15.16:0/270079404 shutdown_connections 2024-04-25T10:12:41.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.699+0000 7f569f25d640 1 -- 172.21.15.16:0/270079404 wait complete. 2024-04-25T10:12:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.700+0000 7f569f25d640 1 Processor -- start 2024-04-25T10:12:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.700+0000 7f569f25d640 1 -- start start 2024-04-25T10:12:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.701+0000 7f569f25d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5688004d40 0x7f56980c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.701+0000 7f569f25d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f569815d2b0 0x7f569815f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.701+0000 7f569f25d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f569814c320 0x7f569814c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.701+0000 7f569f25d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56980b95f0 con 0x7f5688004d40 2024-04-25T10:12:41.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.701+0000 7f569f25d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56980d2040 con 0x7f569814c320 2024-04-25T10:12:41.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.701+0000 7f569f25d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56980b9770 con 0x7f569815d2b0 2024-04-25T10:12:41.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.701+0000 7f569cff9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5688004d40 0x7f56980c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.701+0000 7f569cff9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5688004d40 0x7f56980c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33010/0 (socket says 172.21.15.16:33010) 2024-04-25T10:12:41.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.701+0000 7f569cff9640 1 -- 172.21.15.16:0/595313933 learned_addr learned my addr 172.21.15.16:0/595313933 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:41.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.702+0000 7f5697fff640 1 --2- 172.21.15.16:0/595313933 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f569815d2b0 0x7f569815f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.702+0000 7f569d7fa640 1 --2- 172.21.15.16:0/595313933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f569814c320 0x7f569814c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.702+0000 7f569cff9640 1 -- 172.21.15.16:0/595313933 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f569815d2b0 msgr2=0x7f569815f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.702+0000 7f569cff9640 1 --2- 172.21.15.16:0/595313933 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f569815d2b0 0x7f569815f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.702+0000 7f569cff9640 1 -- 172.21.15.16:0/595313933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f569814c320 msgr2=0x7f569814c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.702+0000 7f569cff9640 1 --2- 172.21.15.16:0/595313933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f569814c320 0x7f569814c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.702+0000 7f569cff9640 1 -- 172.21.15.16:0/595313933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5698160710 con 0x7f5688004d40 2024-04-25T10:12:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.702+0000 7f5697fff640 1 --2- 172.21.15.16:0/595313933 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f569815d2b0 0x7f569815f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.702+0000 7f569cff9640 1 --2- 172.21.15.16:0/595313933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5688004d40 0x7f56980c6f90 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f568c0118e0 tx=0x7f568c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.702+0000 7f56957fa640 1 -- 172.21.15.16:0/595313933 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c00faf0 con 0x7f5688004d40 2024-04-25T10:12:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.702+0000 7f56957fa640 1 -- 172.21.15.16:0/595313933 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568c00fc90 con 0x7f5688004d40 2024-04-25T10:12:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.702+0000 7f569f25d640 1 -- 172.21.15.16:0/595313933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569815d710 con 0x7f5688004d40 2024-04-25T10:12:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.702+0000 7f56957fa640 1 -- 172.21.15.16:0/595313933 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c014440 con 0x7f5688004d40 2024-04-25T10:12:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.702+0000 7f569f25d640 1 -- 172.21.15.16:0/595313933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569815fe70 con 0x7f5688004d40 2024-04-25T10:12:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.703+0000 7f569f25d640 1 -- 172.21.15.16:0/595313933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56980b9130 con 0x7f5688004d40 2024-04-25T10:12:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.704+0000 7f56957fa640 1 -- 172.21.15.16:0/595313933 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f568c012040 con 0x7f5688004d40 2024-04-25T10:12:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.704+0000 7f56957fa640 1 --2- 172.21.15.16:0/595313933 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f567c06b4d0 0x7f567c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.704+0000 7f56957fa640 1 -- 172.21.15.16:0/595313933 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f568c08a6e0 con 0x7f5688004d40 2024-04-25T10:12:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.704+0000 7f5697fff640 1 --2- 172.21.15.16:0/595313933 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f567c06b4d0 0x7f567c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.705+0000 7f5697fff640 1 --2- 172.21.15.16:0/595313933 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f567c06b4d0 0x7f567c06d990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f56840017e0 tx=0x7f5684009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.709+0000 7f56957fa640 1 -- 172.21.15.16:0/595313933 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56980b9130 con 0x7f5688004d40 2024-04-25T10:12:41.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.891+0000 7f569f25d640 1 -- 172.21.15.16:0/595313933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f56980750b0 con 0x7f5688004d40 2024-04-25T10:12:41.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.894+0000 7f56957fa640 1 -- 172.21.15.16:0/595313933 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f568c019030 con 0x7f5688004d40 2024-04-25T10:12:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.896+0000 7f569f25d640 1 -- 172.21.15.16:0/595313933 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f567c06b4d0 msgr2=0x7f567c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.896+0000 7f569f25d640 1 --2- 172.21.15.16:0/595313933 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f567c06b4d0 0x7f567c06d990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f56840017e0 tx=0x7f5684009040 comp rx=0 tx=0).stop 2024-04-25T10:12:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.896+0000 7f569f25d640 1 -- 172.21.15.16:0/595313933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5688004d40 msgr2=0x7f56980c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.896+0000 7f569f25d640 1 --2- 172.21.15.16:0/595313933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5688004d40 0x7f56980c6f90 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f568c0118e0 tx=0x7f568c011da0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.896+0000 7f569f25d640 1 -- 172.21.15.16:0/595313933 shutdown_connections 2024-04-25T10:12:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.896+0000 7f569f25d640 1 --2- 172.21.15.16:0/595313933 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f567c06b4d0 0x7f567c06d990 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.896+0000 7f569f25d640 1 --2- 172.21.15.16:0/595313933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f569814c320 0x7f569814c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.896+0000 7f569f25d640 1 --2- 172.21.15.16:0/595313933 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f569815d2b0 0x7f569815f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.896+0000 7f569f25d640 1 --2- 172.21.15.16:0/595313933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5688004d40 0x7f56980c6f90 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.896+0000 7f569f25d640 1 -- 172.21.15.16:0/595313933 >> 172.21.15.16:0/595313933 conn(0x7f56980d0550 msgr2=0x7f56980d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.896+0000 7f569f25d640 1 -- 172.21.15.16:0/595313933 shutdown_connections 2024-04-25T10:12:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:41.896+0000 7f569f25d640 1 -- 172.21.15.16:0/595313933 wait complete. 2024-04-25T10:12:41.912 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-25T10:12:42.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.008+0000 7f4e29309640 1 Processor -- start 2024-04-25T10:12:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.008+0000 7f4e29309640 1 -- start start 2024-04-25T10:12:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.009+0000 7f4e29309640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e240d35b0 0x7f4e240b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.009+0000 7f4e29309640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4e240b5990 0x7f4e240b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.009+0000 7f4e29309640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e2415e0c0 con 0x7f4e2415dce0 2024-04-25T10:12:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.009+0000 7f4e29309640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e240b5de0 con 0x7f4e240b5990 2024-04-25T10:12:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.009+0000 7f4e29309640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240d2510 con 0x7f4e240d35b0 2024-04-25T10:12:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.009+0000 7f4e1a7fc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e240d35b0 0x7f4e240b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.009+0000 7f4e237fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4e240b5990 0x7f4e240b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.009+0000 7f4e1a7fc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e240d35b0 0x7f4e240b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:35106/0 (socket says 172.21.15.16:35106) 2024-04-25T10:12:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.009+0000 7f4e1a7fc640 1 -- 172.21.15.16:0/411860241 learned_addr learned my addr 172.21.15.16:0/411860241 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.009+0000 7f4e1a7fc640 1 -- 172.21.15.16:0/411860241 >> v2:172.21.15.186:3300/0 conn(0x7f4e240b5990 msgr2=0x7f4e240b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.009+0000 7f4e1a7fc640 1 --2- 172.21.15.16:0/411860241 >> v2:172.21.15.186:3300/0 conn(0x7f4e240b5990 0x7f4e240b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.009+0000 7f4e1a7fc640 1 -- 172.21.15.16:0/411860241 >> v1:172.21.15.186:6789/0 conn(0x7f4e2415dce0 legacy=0x7f4e240d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.009+0000 7f4e1a7fc640 1 -- 172.21.15.16:0/411860241 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e240d27f0 con 0x7f4e240d35b0 2024-04-25T10:12:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.009+0000 7f4e237fe640 1 --2- 172.21.15.16:0/411860241 >> v2:172.21.15.186:3300/0 conn(0x7f4e240b5990 0x7f4e240b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.010+0000 7f4e1a7fc640 1 --2- 172.21.15.16:0/411860241 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e240d35b0 0x7f4e240b5450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c009870 tx=0x7f4e0c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b23e6fbf793e0fe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.010+0000 7f4e227fc640 1 -- 172.21.15.16:0/411860241 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c037070 con 0x7f4e240d35b0 2024-04-25T10:12:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.010+0000 7f4e227fc640 1 -- 172.21.15.16:0/411860241 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0c032440 con 0x7f4e240d35b0 2024-04-25T10:12:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.010+0000 7f4e227fc640 1 -- 172.21.15.16:0/411860241 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c032810 con 0x7f4e240d35b0 2024-04-25T10:12:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.011+0000 7f4e29309640 1 -- 172.21.15.16:0/411860241 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e240d35b0 msgr2=0x7f4e240b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.011+0000 7f4e29309640 1 --2- 172.21.15.16:0/411860241 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e240d35b0 0x7f4e240b5450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c009870 tx=0x7f4e0c02a270 comp rx=0 tx=0).stop 2024-04-25T10:12:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.011+0000 7f4e29309640 1 -- 172.21.15.16:0/411860241 shutdown_connections 2024-04-25T10:12:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.011+0000 7f4e29309640 1 --2- 172.21.15.16:0/411860241 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e240d35b0 0x7f4e240b5450 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.011+0000 7f4e29309640 1 --2- 172.21.15.16:0/411860241 >> v2:172.21.15.186:3300/0 conn(0x7f4e240b5990 0x7f4e240b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.011+0000 7f4e29309640 1 -- 172.21.15.16:0/411860241 >> 172.21.15.16:0/411860241 conn(0x7f4e24072900 msgr2=0x7f4e240d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.011+0000 7f4e29309640 1 -- 172.21.15.16:0/411860241 shutdown_connections 2024-04-25T10:12:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.011+0000 7f4e29309640 1 -- 172.21.15.16:0/411860241 wait complete. 2024-04-25T10:12:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.012+0000 7f4e29309640 1 Processor -- start 2024-04-25T10:12:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.012+0000 7f4e29309640 1 -- start start 2024-04-25T10:12:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.013+0000 7f4e29309640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e240b5990 0x7f4e2414ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.013+0000 7f4e29309640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e240d35b0 0x7f4e24160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.013+0000 7f4e29309640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e2415dce0 0x7f4e24159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.013+0000 7f4e29309640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240d3090 con 0x7f4e240b5990 2024-04-25T10:12:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.013+0000 7f4e29309640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240d2db0 con 0x7f4e240d35b0 2024-04-25T10:12:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.013+0000 7f4e29309640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240d2160 con 0x7f4e2415dce0 2024-04-25T10:12:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.013+0000 7f4e22ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e240b5990 0x7f4e2414ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.013+0000 7f4e22ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e240b5990 0x7f4e2414ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33026/0 (socket says 172.21.15.16:33026) 2024-04-25T10:12:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.013+0000 7f4e22ffd640 1 -- 172.21.15.16:0/1264692617 learned_addr learned my addr 172.21.15.16:0/1264692617 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.013+0000 7f4e1a7fc640 1 --2- 172.21.15.16:0/1264692617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e240d35b0 0x7f4e24160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.013+0000 7f4e237fe640 1 --2- 172.21.15.16:0/1264692617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e2415dce0 0x7f4e24159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.013+0000 7f4e22ffd640 1 -- 172.21.15.16:0/1264692617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e2415dce0 msgr2=0x7f4e24159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.013+0000 7f4e22ffd640 1 --2- 172.21.15.16:0/1264692617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e2415dce0 0x7f4e24159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.014+0000 7f4e22ffd640 1 -- 172.21.15.16:0/1264692617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e240d35b0 msgr2=0x7f4e24160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.014+0000 7f4e22ffd640 1 --2- 172.21.15.16:0/1264692617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e240d35b0 0x7f4e24160aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.014+0000 7f4e22ffd640 1 -- 172.21.15.16:0/1264692617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e240ca380 con 0x7f4e240b5990 2024-04-25T10:12:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.014+0000 7f4e22ffd640 1 --2- 172.21.15.16:0/1264692617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e240b5990 0x7f4e2414ae70 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400c400 tx=0x7f4e1400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.014+0000 7f4e1bfff640 1 -- 172.21.15.16:0/1264692617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14017070 con 0x7f4e240b5990 2024-04-25T10:12:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.014+0000 7f4e1bfff640 1 -- 172.21.15.16:0/1264692617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e140052d0 con 0x7f4e240b5990 2024-04-25T10:12:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.014+0000 7f4e1bfff640 1 -- 172.21.15.16:0/1264692617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14003830 con 0x7f4e240b5990 2024-04-25T10:12:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.014+0000 7f4e29309640 1 -- 172.21.15.16:0/1264692617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e241587d0 con 0x7f4e240b5990 2024-04-25T10:12:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.014+0000 7f4e29309640 1 -- 172.21.15.16:0/1264692617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e24158d90 con 0x7f4e240b5990 2024-04-25T10:12:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.015+0000 7f4e29309640 1 -- 172.21.15.16:0/1264692617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e240749b0 con 0x7f4e240b5990 2024-04-25T10:12:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.015+0000 7f4e1bfff640 1 -- 172.21.15.16:0/1264692617 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e1401a070 con 0x7f4e240b5990 2024-04-25T10:12:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.016+0000 7f4e1bfff640 1 --2- 172.21.15.16:0/1264692617 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4e0406b4d0 0x7f4e0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.016+0000 7f4e1bfff640 1 -- 172.21.15.16:0/1264692617 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f4e140891e0 con 0x7f4e240b5990 2024-04-25T10:12:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.016+0000 7f4e1a7fc640 1 --2- 172.21.15.16:0/1264692617 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4e0406b4d0 0x7f4e0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.017+0000 7f4e1a7fc640 1 --2- 172.21.15.16:0/1264692617 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4e0406b4d0 0x7f4e0406d990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c034000 tx=0x7f4e0c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:42.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.020+0000 7f4e1bfff640 1 -- 172.21.15.16:0/1264692617 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e14057c60 con 0x7f4e240b5990 2024-04-25T10:12:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.280+0000 7f4e29309640 1 -- 172.21.15.16:0/1264692617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7f4e240b7b30 con 0x7f4e240b5990 2024-04-25T10:12:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.572+0000 7f4e1bfff640 1 -- 172.21.15.16:0/1264692617 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v366) v1 ==== 167+0+0 (secure 0 0 0) 0x7f4e1405bd00 con 0x7f4e240b5990 2024-04-25T10:12:42.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.575+0000 7f4e29309640 1 -- 172.21.15.16:0/1264692617 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4e0406b4d0 msgr2=0x7f4e0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.575+0000 7f4e29309640 1 --2- 172.21.15.16:0/1264692617 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4e0406b4d0 0x7f4e0406d990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c034000 tx=0x7f4e0c02e000 comp rx=0 tx=0).stop 2024-04-25T10:12:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.575+0000 7f4e29309640 1 -- 172.21.15.16:0/1264692617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e240b5990 msgr2=0x7f4e2414ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.575+0000 7f4e29309640 1 --2- 172.21.15.16:0/1264692617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e240b5990 0x7f4e2414ae70 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400c400 tx=0x7f4e1400c800 comp rx=0 tx=0).stop 2024-04-25T10:12:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.575+0000 7f4e29309640 1 -- 172.21.15.16:0/1264692617 shutdown_connections 2024-04-25T10:12:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.575+0000 7f4e29309640 1 --2- 172.21.15.16:0/1264692617 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4e0406b4d0 0x7f4e0406d990 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.575+0000 7f4e29309640 1 --2- 172.21.15.16:0/1264692617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e2415dce0 0x7f4e24159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.575+0000 7f4e29309640 1 --2- 172.21.15.16:0/1264692617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e240d35b0 0x7f4e24160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.575+0000 7f4e29309640 1 --2- 172.21.15.16:0/1264692617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e240b5990 0x7f4e2414ae70 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.575+0000 7f4e29309640 1 -- 172.21.15.16:0/1264692617 >> 172.21.15.16:0/1264692617 conn(0x7f4e24072900 msgr2=0x7f4e240b6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.575+0000 7f4e29309640 1 -- 172.21.15.16:0/1264692617 shutdown_connections 2024-04-25T10:12:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.575+0000 7f4e29309640 1 -- 172.21.15.16:0/1264692617 wait complete. 2024-04-25T10:12:42.591 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T10:12:42.591 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:12:42.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.688+0000 7ff93cbf5640 1 Processor -- start 2024-04-25T10:12:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.688+0000 7ff93cbf5640 1 -- start start 2024-04-25T10:12:42.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.689+0000 7ff93cbf5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff93815dce0 0x7ff9380d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.689+0000 7ff93cbf5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff9380d35b0 0x7ff9380b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.689+0000 7ff93cbf5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff93815e0c0 con 0x7ff9380d1f90 2024-04-25T10:12:42.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.689+0000 7ff93cbf5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9380b7b10 con 0x7ff9380d35b0 2024-04-25T10:12:42.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.689+0000 7ff93cbf5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9380d2510 con 0x7ff93815dce0 2024-04-25T10:12:42.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.689+0000 7ff935d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff9380d35b0 0x7ff9380b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.689+0000 7ff936d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff9380d1f90 0x7ff93806bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49226/0 (socket says 172.21.15.16:49226) 2024-04-25T10:12:42.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.689+0000 7ff936d9d640 1 -- 172.21.15.16:0/1518234973 learned_addr learned my addr 172.21.15.16:0/1518234973 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:42.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.690+0000 7ff93659c640 1 --2- 172.21.15.16:0/1518234973 >> v2:172.21.15.186:3300/0 conn(0x7ff93815dce0 0x7ff9380d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.690+0000 7ff935d9b640 1 -- 172.21.15.16:0/1518234973 >> v2:172.21.15.186:3300/0 conn(0x7ff93815dce0 msgr2=0x7ff9380d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:42.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.690+0000 7ff935d9b640 1 --2- 172.21.15.16:0/1518234973 >> v2:172.21.15.186:3300/0 conn(0x7ff93815dce0 0x7ff9380d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.690+0000 7ff935d9b640 1 -- 172.21.15.16:0/1518234973 >> v1:172.21.15.16:6789/0 conn(0x7ff9380d1f90 legacy=0x7ff93806bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.690+0000 7ff93559a640 1 -- 172.21.15.16:0/1518234973 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557697771 0 0) 0x7ff93815e0c0 con 0x7ff9380d1f90 2024-04-25T10:12:42.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.690+0000 7ff935d9b640 1 -- 172.21.15.16:0/1518234973 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9380d27f0 con 0x7ff9380d35b0 2024-04-25T10:12:42.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.690+0000 7ff935d9b640 1 --2- 172.21.15.16:0/1518234973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9380d35b0 0x7ff9380b5450 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7ff9380ce220 tx=0x7ff92002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73bebf1390ebf51b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:42.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.691+0000 7ff93559a640 1 -- 172.21.15.16:0/1518234973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff920037070 con 0x7ff9380d35b0 2024-04-25T10:12:42.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.691+0000 7ff93559a640 1 -- 172.21.15.16:0/1518234973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff920032440 con 0x7ff9380d35b0 2024-04-25T10:12:42.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.691+0000 7ff93559a640 1 -- 172.21.15.16:0/1518234973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff920032810 con 0x7ff9380d35b0 2024-04-25T10:12:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.691+0000 7ff93cbf5640 1 -- 172.21.15.16:0/1518234973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9380d35b0 msgr2=0x7ff9380b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.691+0000 7ff93cbf5640 1 --2- 172.21.15.16:0/1518234973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9380d35b0 0x7ff9380b5450 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7ff9380ce220 tx=0x7ff92002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.691+0000 7ff93cbf5640 1 -- 172.21.15.16:0/1518234973 shutdown_connections 2024-04-25T10:12:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.691+0000 7ff93cbf5640 1 --2- 172.21.15.16:0/1518234973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9380d35b0 0x7ff9380b5450 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.691+0000 7ff93cbf5640 1 --2- 172.21.15.16:0/1518234973 >> v2:172.21.15.186:3300/0 conn(0x7ff93815dce0 0x7ff9380d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.691+0000 7ff93cbf5640 1 -- 172.21.15.16:0/1518234973 >> 172.21.15.16:0/1518234973 conn(0x7ff938072900 msgr2=0x7ff9380d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.691+0000 7ff93cbf5640 1 -- 172.21.15.16:0/1518234973 shutdown_connections 2024-04-25T10:12:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.691+0000 7ff93cbf5640 1 -- 172.21.15.16:0/1518234973 wait complete. 2024-04-25T10:12:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.693+0000 7ff93cbf5640 1 Processor -- start 2024-04-25T10:12:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.693+0000 7ff93cbf5640 1 -- start start 2024-04-25T10:12:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.693+0000 7ff93cbf5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9380d1f90 0x7ff938153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.693+0000 7ff93cbf5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9380d35b0 0x7ff9380cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.693+0000 7ff93cbf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93815dce0 0x7ff9380c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.693+0000 7ff93cbf5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9380d3090 con 0x7ff93815dce0 2024-04-25T10:12:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.693+0000 7ff93cbf5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9380d2db0 con 0x7ff9380d35b0 2024-04-25T10:12:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.693+0000 7ff93cbf5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9380b9820 con 0x7ff9380d1f90 2024-04-25T10:12:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.693+0000 7ff936d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93815dce0 0x7ff9380c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.693+0000 7ff936d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93815dce0 0x7ff9380c9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33038/0 (socket says 172.21.15.16:33038) 2024-04-25T10:12:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.693+0000 7ff936d9d640 1 -- 172.21.15.16:0/2546544643 learned_addr learned my addr 172.21.15.16:0/2546544643 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.694+0000 7ff935d9b640 1 --2- 172.21.15.16:0/2546544643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9380d35b0 0x7ff9380cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.694+0000 7ff93659c640 1 --2- 172.21.15.16:0/2546544643 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9380d1f90 0x7ff938153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.694+0000 7ff936d9d640 1 -- 172.21.15.16:0/2546544643 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9380d1f90 msgr2=0x7ff938153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.694+0000 7ff936d9d640 1 --2- 172.21.15.16:0/2546544643 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9380d1f90 0x7ff938153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.694+0000 7ff936d9d640 1 -- 172.21.15.16:0/2546544643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9380d35b0 msgr2=0x7ff9380cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.694+0000 7ff936d9d640 1 --2- 172.21.15.16:0/2546544643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9380d35b0 0x7ff9380cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.694+0000 7ff936d9d640 1 -- 172.21.15.16:0/2546544643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9380ccfc0 con 0x7ff93815dce0 2024-04-25T10:12:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.694+0000 7ff935d9b640 1 --2- 172.21.15.16:0/2546544643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9380d35b0 0x7ff9380cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.694+0000 7ff936d9d640 1 --2- 172.21.15.16:0/2546544643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93815dce0 0x7ff9380c9120 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7ff92800c7a0 tx=0x7ff92800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.694+0000 7ff92effd640 1 -- 172.21.15.16:0/2546544643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928018070 con 0x7ff93815dce0 2024-04-25T10:12:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.694+0000 7ff92effd640 1 -- 172.21.15.16:0/2546544643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92800a640 con 0x7ff93815dce0 2024-04-25T10:12:42.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.694+0000 7ff93cbf5640 1 -- 172.21.15.16:0/2546544643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9380cd2a0 con 0x7ff93815dce0 2024-04-25T10:12:42.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.694+0000 7ff92effd640 1 -- 172.21.15.16:0/2546544643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92800aab0 con 0x7ff93815dce0 2024-04-25T10:12:42.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.694+0000 7ff93cbf5640 1 -- 172.21.15.16:0/2546544643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff938153230 con 0x7ff93815dce0 2024-04-25T10:12:42.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.695+0000 7ff93cbf5640 1 -- 172.21.15.16:0/2546544643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff93815eb10 con 0x7ff93815dce0 2024-04-25T10:12:42.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.696+0000 7ff92effd640 1 -- 172.21.15.16:0/2546544643 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff92800ac50 con 0x7ff93815dce0 2024-04-25T10:12:42.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.696+0000 7ff92effd640 1 --2- 172.21.15.16:0/2546544643 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff91806b4d0 0x7ff91806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.696+0000 7ff92effd640 1 -- 172.21.15.16:0/2546544643 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5115+0+0 (secure 0 0 0) 0x7ff928089610 con 0x7ff93815dce0 2024-04-25T10:12:42.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.696+0000 7ff93659c640 1 --2- 172.21.15.16:0/2546544643 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff91806b4d0 0x7ff91806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.697+0000 7ff93659c640 1 --2- 172.21.15.16:0/2546544643 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff91806b4d0 0x7ff91806d990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff930009aa0 tx=0x7ff930009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:42.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.700+0000 7ff92effd640 1 -- 172.21.15.16:0/2546544643 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff928058010 con 0x7ff93815dce0 2024-04-25T10:12:42.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.886+0000 7ff93cbf5640 1 -- 172.21.15.16:0/2546544643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9380b5dd0 con 0x7ff93815dce0 2024-04-25T10:12:42.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.887+0000 7ff92effd640 1 -- 172.21.15.16:0/2546544643 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+13281 (secure 0 0 0) 0x7ff928089030 con 0x7ff93815dce0 2024-04-25T10:12:42.891 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:42.892 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":91,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:12:41.562143+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6834/84349474":"2024-04-26T10:12:38.554526+0000","172.21.15.16:6832/84349474":"2024-04-26T10:12:38.554526+0000","172.21.15.186:6836/3950732718":"2024-04-26T10:11:55.231471+0000","172.21.15.186:6834/3950732718":"2024-04-26T10:11:55.231471+0000","172.21.15.186:6837/3092291936":"2024-04-26T10:11:13.827348+0000","172.21.15.186:6835/3092291936":"2024-04-26T10:11:13.827348+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:42.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.889+0000 7ff93cbf5640 1 -- 172.21.15.16:0/2546544643 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff91806b4d0 msgr2=0x7ff91806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.889+0000 7ff93cbf5640 1 --2- 172.21.15.16:0/2546544643 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff91806b4d0 0x7ff91806d990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff930009aa0 tx=0x7ff930009190 comp rx=0 tx=0).stop 2024-04-25T10:12:42.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.889+0000 7ff93cbf5640 1 -- 172.21.15.16:0/2546544643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93815dce0 msgr2=0x7ff9380c9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.889+0000 7ff93cbf5640 1 --2- 172.21.15.16:0/2546544643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93815dce0 0x7ff9380c9120 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7ff92800c7a0 tx=0x7ff92800cc60 comp rx=0 tx=0).stop 2024-04-25T10:12:42.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.889+0000 7ff93cbf5640 1 -- 172.21.15.16:0/2546544643 shutdown_connections 2024-04-25T10:12:42.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.889+0000 7ff93cbf5640 1 --2- 172.21.15.16:0/2546544643 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff91806b4d0 0x7ff91806d990 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.889+0000 7ff93cbf5640 1 --2- 172.21.15.16:0/2546544643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93815dce0 0x7ff9380c9120 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.889+0000 7ff93cbf5640 1 --2- 172.21.15.16:0/2546544643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9380d35b0 0x7ff9380cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.889+0000 7ff93cbf5640 1 --2- 172.21.15.16:0/2546544643 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9380d1f90 0x7ff938153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.889+0000 7ff93cbf5640 1 -- 172.21.15.16:0/2546544643 >> 172.21.15.16:0/2546544643 conn(0x7ff938072900 msgr2=0x7ff9380d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.889+0000 7ff93cbf5640 1 -- 172.21.15.16:0/2546544643 shutdown_connections 2024-04-25T10:12:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:42.889+0000 7ff93cbf5640 1 -- 172.21.15.16:0/2546544643 wait complete. 2024-04-25T10:12:42.905 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:42.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:43.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.004+0000 7ffb1273b640 1 Processor -- start 2024-04-25T10:12:43.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.004+0000 7ffb1273b640 1 -- start start 2024-04-25T10:12:43.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.005+0000 7ffb1273b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb0c15dc80 0x7ffb0c161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.005+0000 7ffb1273b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0c1627a0 con 0x7ffb0c0b5c00 2024-04-25T10:12:43.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.005+0000 7ffb1273b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0c1621e0 con 0x7ffb0c0b97c0 2024-04-25T10:12:43.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.005+0000 7ffb1273b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c1624c0 con 0x7ffb0c15dc80 2024-04-25T10:12:43.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.005+0000 7ffb10cd8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffb0c0b5c00 0x7ffb0c0d2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45114/0 (socket says 172.21.15.16:45114) 2024-04-25T10:12:43.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.005+0000 7ffb10cd8640 1 -- 172.21.15.16:0/3202904507 learned_addr learned my addr 172.21.15.16:0/3202904507 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:43.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.005+0000 7ffb0affd640 1 -- 172.21.15.16:0/3202904507 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219935433 0 0) 0x7ffb0c1627a0 con 0x7ffb0c0b5c00 2024-04-25T10:12:43.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.005+0000 7ffb0bfff640 1 --2- 172.21.15.16:0/3202904507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb0c15dc80 0x7ffb0c161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.005+0000 7ffb0affd640 1 -- 172.21.15.16:0/3202904507 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaf8003520 con 0x7ffb0c0b5c00 2024-04-25T10:12:43.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0affd640 1 -- 172.21.15.16:0/3202904507 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1251503832 0 0) 0x7ffaf8003520 con 0x7ffb0c0b5c00 2024-04-25T10:12:43.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0affd640 1 -- 172.21.15.16:0/3202904507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb0c15dc80 msgr2=0x7ffb0c161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0affd640 1 --2- 172.21.15.16:0/3202904507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb0c15dc80 0x7ffb0c161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0affd640 1 -- 172.21.15.16:0/3202904507 >> v1:172.21.15.186:6789/0 conn(0x7ffb0c0b97c0 legacy=0x7ffb0c0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0affd640 1 -- 172.21.15.16:0/3202904507 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c0d1c70 con 0x7ffb0c0b5c00 2024-04-25T10:12:43.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0affd640 1 -- 172.21.15.16:0/3202904507 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7ffb040019f0 con 0x7ffb0c0b5c00 2024-04-25T10:12:43.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0affd640 1 -- 172.21.15.16:0/3202904507 >> v1:172.21.15.16:6789/0 conn(0x7ffb0c0b5c00 legacy=0x7ffb0c0d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0affd640 1 --2- 172.21.15.16:0/3202904507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf80039e0 0x7ffaf8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0affd640 1 --2- 172.21.15.16:0/3202904507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffaf8004800 0x7ffaf8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0affd640 1 --2- 172.21.15.16:0/3202904507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb0c15dc80 0x7ffaf80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0affd640 1 -- 172.21.15.16:0/3202904507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c1624c0 con 0x7ffaf80039e0 2024-04-25T10:12:43.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0affd640 1 -- 172.21.15.16:0/3202904507 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf800ba70 con 0x7ffaf8004800 2024-04-25T10:12:43.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0affd640 1 -- 172.21.15.16:0/3202904507 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf800bc70 con 0x7ffb0c15dc80 2024-04-25T10:12:43.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0bfff640 1 --2- 172.21.15.16:0/3202904507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf80039e0 0x7ffaf8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb10cd8640 1 --2- 172.21.15.16:0/3202904507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb0c15dc80 0x7ffaf80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0b7fe640 1 --2- 172.21.15.16:0/3202904507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffaf8004800 0x7ffaf8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0bfff640 1 -- 172.21.15.16:0/3202904507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb0c15dc80 msgr2=0x7ffaf80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0bfff640 1 --2- 172.21.15.16:0/3202904507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb0c15dc80 0x7ffaf80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0bfff640 1 -- 172.21.15.16:0/3202904507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffaf8004800 msgr2=0x7ffaf8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0bfff640 1 --2- 172.21.15.16:0/3202904507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffaf8004800 0x7ffaf8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0bfff640 1 -- 172.21.15.16:0/3202904507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf8003520 con 0x7ffaf80039e0 2024-04-25T10:12:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.006+0000 7ffb0b7fe640 1 --2- 172.21.15.16:0/3202904507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffaf8004800 0x7ffaf8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.007+0000 7ffb0bfff640 1 --2- 172.21.15.16:0/3202904507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf80039e0 0x7ffaf8003e00 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009a90 tx=0x7ffaf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6afa1fac6a842357 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.007+0000 7ffb0affd640 1 -- 172.21.15.16:0/3202904507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4036070 con 0x7ffaf80039e0 2024-04-25T10:12:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.007+0000 7ffb0affd640 1 -- 172.21.15.16:0/3202904507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf4031400 con 0x7ffaf80039e0 2024-04-25T10:12:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.007+0000 7ffb0affd640 1 -- 172.21.15.16:0/3202904507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4009690 con 0x7ffaf80039e0 2024-04-25T10:12:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.007+0000 7ffb1273b640 1 -- 172.21.15.16:0/3202904507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf80039e0 msgr2=0x7ffaf8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.007+0000 7ffb1273b640 1 --2- 172.21.15.16:0/3202904507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf80039e0 0x7ffaf8003e00 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009a90 tx=0x7ffaf8005830 comp rx=0 tx=0).stop 2024-04-25T10:12:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.007+0000 7ffb10cd8640 1 -- 172.21.15.16:0/3202904507 reap_dead start 2024-04-25T10:12:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.008+0000 7ffb1273b640 1 -- 172.21.15.16:0/3202904507 shutdown_connections 2024-04-25T10:12:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.008+0000 7ffb1273b640 1 -- 172.21.15.16:0/3202904507 >> 172.21.15.16:0/3202904507 conn(0x7ffb0c0d04f0 msgr2=0x7ffb0c0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.008+0000 7ffb1273b640 1 -- 172.21.15.16:0/3202904507 shutdown_connections 2024-04-25T10:12:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.008+0000 7ffb1273b640 1 -- 172.21.15.16:0/3202904507 wait complete. 2024-04-25T10:12:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.009+0000 7ffb1273b640 1 Processor -- start 2024-04-25T10:12:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.009+0000 7ffb1273b640 1 -- start start 2024-04-25T10:12:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.010+0000 7ffb1273b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffaf80039e0 0x7ffb0c14fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.010+0000 7ffb1273b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb0c160940 0x7ffb0c160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.010+0000 7ffb1273b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb0c15e440 0x7ffb0c15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.010+0000 7ffb1273b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c1619f0 con 0x7ffb0c160940 2024-04-25T10:12:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.010+0000 7ffb1273b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c161870 con 0x7ffaf80039e0 2024-04-25T10:12:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.010+0000 7ffb1273b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c161b70 con 0x7ffb0c15e440 2024-04-25T10:12:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.010+0000 7ffb0b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb0c160940 0x7ffb0c160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.010+0000 7ffb0b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb0c160940 0x7ffb0c160dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60800/0 (socket says 172.21.15.16:60800) 2024-04-25T10:12:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.010+0000 7ffb0b7fe640 1 -- 172.21.15.16:0/1369190971 learned_addr learned my addr 172.21.15.16:0/1369190971 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.010+0000 7ffb0bfff640 1 --2- 172.21.15.16:0/1369190971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffaf80039e0 0x7ffb0c14fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.010+0000 7ffb10cd8640 1 --2- 172.21.15.16:0/1369190971 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb0c15e440 0x7ffb0c15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.010+0000 7ffb0b7fe640 1 -- 172.21.15.16:0/1369190971 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb0c15e440 msgr2=0x7ffb0c15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.010+0000 7ffb0b7fe640 1 --2- 172.21.15.16:0/1369190971 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb0c15e440 0x7ffb0c15e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.010+0000 7ffb0b7fe640 1 -- 172.21.15.16:0/1369190971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffaf80039e0 msgr2=0x7ffb0c14fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.010+0000 7ffb0b7fe640 1 --2- 172.21.15.16:0/1369190971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffaf80039e0 0x7ffb0c14fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.010+0000 7ffb0b7fe640 1 -- 172.21.15.16:0/1369190971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c15f1b0 con 0x7ffb0c160940 2024-04-25T10:12:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.010+0000 7ffb0b7fe640 1 --2- 172.21.15.16:0/1369190971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb0c160940 0x7ffb0c160dc0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00e4e0 tx=0x7ffafc00e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.011+0000 7ffb08ff9640 1 -- 172.21.15.16:0/1369190971 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc019070 con 0x7ffb0c160940 2024-04-25T10:12:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.011+0000 7ffb08ff9640 1 -- 172.21.15.16:0/1369190971 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffafc0052e0 con 0x7ffb0c160940 2024-04-25T10:12:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.011+0000 7ffb1273b640 1 -- 172.21.15.16:0/1369190971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0c15f490 con 0x7ffb0c160940 2024-04-25T10:12:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.011+0000 7ffb08ff9640 1 -- 172.21.15.16:0/1369190971 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc003830 con 0x7ffb0c160940 2024-04-25T10:12:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.011+0000 7ffb1273b640 1 -- 172.21.15.16:0/1369190971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0c15fa50 con 0x7ffb0c160940 2024-04-25T10:12:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.012+0000 7ffb1273b640 1 -- 172.21.15.16:0/1369190971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0c0750b0 con 0x7ffb0c160940 2024-04-25T10:12:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.012+0000 7ffb08ff9640 1 -- 172.21.15.16:0/1369190971 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffafc01c070 con 0x7ffb0c160940 2024-04-25T10:12:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.013+0000 7ffb08ff9640 1 --2- 172.21.15.16:0/1369190971 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffaec06b4d0 0x7ffaec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.013+0000 7ffb08ff9640 1 -- 172.21.15.16:0/1369190971 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5115+0+0 (secure 0 0 0) 0x7ffafc0894a0 con 0x7ffb0c160940 2024-04-25T10:12:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.013+0000 7ffb0bfff640 1 --2- 172.21.15.16:0/1369190971 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffaec06b4d0 0x7ffaec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.014+0000 7ffb0bfff640 1 --2- 172.21.15.16:0/1369190971 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffaec06b4d0 0x7ffaec06d990 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4033000 tx=0x7ffaf4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.017+0000 7ffb08ff9640 1 -- 172.21.15.16:0/1369190971 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffafc057fe0 con 0x7ffb0c160940 2024-04-25T10:12:43.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.196+0000 7ffb1273b640 1 -- 172.21.15.16:0/1369190971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb0c0d1780 con 0x7ffb0c160940 2024-04-25T10:12:43.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.197+0000 7ffb08ff9640 1 -- 172.21.15.16:0/1369190971 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffafc05c080 con 0x7ffb0c160940 2024-04-25T10:12:43.201 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:12:43.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.198+0000 7ffb1273b640 1 -- 172.21.15.16:0/1369190971 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffaec06b4d0 msgr2=0x7ffaec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.198+0000 7ffb1273b640 1 --2- 172.21.15.16:0/1369190971 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffaec06b4d0 0x7ffaec06d990 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4033000 tx=0x7ffaf4008030 comp rx=0 tx=0).stop 2024-04-25T10:12:43.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.198+0000 7ffb1273b640 1 -- 172.21.15.16:0/1369190971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb0c160940 msgr2=0x7ffb0c160dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.198+0000 7ffb1273b640 1 --2- 172.21.15.16:0/1369190971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb0c160940 0x7ffb0c160dc0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00e4e0 tx=0x7ffafc00e8e0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.198+0000 7ffb1273b640 1 -- 172.21.15.16:0/1369190971 shutdown_connections 2024-04-25T10:12:43.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.198+0000 7ffb1273b640 1 --2- 172.21.15.16:0/1369190971 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffaec06b4d0 0x7ffaec06d990 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.198+0000 7ffb1273b640 1 --2- 172.21.15.16:0/1369190971 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffb0c15e440 0x7ffb0c15e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.198+0000 7ffb1273b640 1 --2- 172.21.15.16:0/1369190971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb0c160940 0x7ffb0c160dc0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.198+0000 7ffb1273b640 1 --2- 172.21.15.16:0/1369190971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffaf80039e0 0x7ffb0c14fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.198+0000 7ffb1273b640 1 -- 172.21.15.16:0/1369190971 >> 172.21.15.16:0/1369190971 conn(0x7ffb0c0d04f0 msgr2=0x7ffb0c0d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.199+0000 7ffb1273b640 1 -- 172.21.15.16:0/1369190971 shutdown_connections 2024-04-25T10:12:43.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.199+0000 7ffb1273b640 1 -- 172.21.15.16:0/1369190971 wait complete. 2024-04-25T10:12:43.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-25T10:12:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.310+0000 7f1409130640 1 Processor -- start 2024-04-25T10:12:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.310+0000 7f1409130640 1 -- start start 2024-04-25T10:12:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.311+0000 7f1409130640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14040b5b60 0x7f14041687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.311+0000 7f1409130640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f140415e0c0 con 0x7f140415dce0 2024-04-25T10:12:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.311+0000 7f1409130640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14040d2510 con 0x7f14040d35b0 2024-04-25T10:12:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.311+0000 7f1409130640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14040b7b10 con 0x7f14040b5b60 2024-04-25T10:12:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.311+0000 7f1402d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f140415dce0 0x7f14040d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45128/0 (socket says 172.21.15.16:45128) 2024-04-25T10:12:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.311+0000 7f1402d9d640 1 -- 172.21.15.16:0/2959610301 learned_addr learned my addr 172.21.15.16:0/2959610301 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.312+0000 7f140359e640 1 --2- 172.21.15.16:0/2959610301 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14040b5b60 0x7f14041687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.312+0000 7f1401d9b640 1 -- 172.21.15.16:0/2959610301 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129690653 0 0) 0x7f140415e0c0 con 0x7f140415dce0 2024-04-25T10:12:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.312+0000 7f1401d9b640 1 -- 172.21.15.16:0/2959610301 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13ec003520 con 0x7f140415dce0 2024-04-25T10:12:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.312+0000 7f1401d9b640 1 -- 172.21.15.16:0/2959610301 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956305788 0 0) 0x7f14040d2510 con 0x7f14040d35b0 2024-04-25T10:12:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.312+0000 7f140359e640 1 -- 172.21.15.16:0/2959610301 >> v1:172.21.15.186:6789/0 conn(0x7f14040d35b0 legacy=0x7f14040b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.312+0000 7f140359e640 1 -- 172.21.15.16:0/2959610301 >> v1:172.21.15.16:6789/0 conn(0x7f140415dce0 legacy=0x7f14040d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.312+0000 7f140359e640 1 -- 172.21.15.16:0/2959610301 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14040d27f0 con 0x7f14040b5b60 2024-04-25T10:12:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.313+0000 7f140359e640 1 --2- 172.21.15.16:0/2959610301 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14040b5b60 0x7f14041687f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009dc0 tx=0x7f13fc02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfa80d27e5521758 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.313+0000 7f1401d9b640 1 -- 172.21.15.16:0/2959610301 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc007740 con 0x7f14040b5b60 2024-04-25T10:12:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.313+0000 7f1401d9b640 1 -- 172.21.15.16:0/2959610301 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13fc0078e0 con 0x7f14040b5b60 2024-04-25T10:12:43.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.313+0000 7f1401d9b640 1 -- 172.21.15.16:0/2959610301 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc007c90 con 0x7f14040b5b60 2024-04-25T10:12:43.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.313+0000 7f1409130640 1 -- 172.21.15.16:0/2959610301 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14040b5b60 msgr2=0x7f14041687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.313+0000 7f1409130640 1 --2- 172.21.15.16:0/2959610301 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14040b5b60 0x7f14041687f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009dc0 tx=0x7f13fc02a6d0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.313+0000 7f1409130640 1 -- 172.21.15.16:0/2959610301 shutdown_connections 2024-04-25T10:12:43.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.313+0000 7f1409130640 1 --2- 172.21.15.16:0/2959610301 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14040b5b60 0x7f14041687f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.313+0000 7f1409130640 1 -- 172.21.15.16:0/2959610301 >> 172.21.15.16:0/2959610301 conn(0x7f1404072900 msgr2=0x7f14040d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.313+0000 7f1409130640 1 -- 172.21.15.16:0/2959610301 shutdown_connections 2024-04-25T10:12:43.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.313+0000 7f1409130640 1 -- 172.21.15.16:0/2959610301 wait complete. 2024-04-25T10:12:43.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.315+0000 7f1409130640 1 Processor -- start 2024-04-25T10:12:43.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.315+0000 7f1409130640 1 -- start start 2024-04-25T10:12:43.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.316+0000 7f1409130640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14040b5b60 0x7f140414e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.316+0000 7f1409130640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14040d35b0 0x7f140415f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.316+0000 7f1409130640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f140415dce0 0x7f140415b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.316+0000 7f1409130640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14040d3090 con 0x7f14040d35b0 2024-04-25T10:12:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.316+0000 7f1409130640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14040d2db0 con 0x7f140415dce0 2024-04-25T10:12:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.316+0000 7f1409130640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14040d2160 con 0x7f14040b5b60 2024-04-25T10:12:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.316+0000 7f140259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14040d35b0 0x7f140415f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.316+0000 7f140259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14040d35b0 0x7f140415f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60802/0 (socket says 172.21.15.16:60802) 2024-04-25T10:12:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.316+0000 7f140259c640 1 -- 172.21.15.16:0/3323517919 learned_addr learned my addr 172.21.15.16:0/3323517919 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.316+0000 7f140359e640 1 --2- 172.21.15.16:0/3323517919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f140415dce0 0x7f140415b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.316+0000 7f1402d9d640 1 --2- 172.21.15.16:0/3323517919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14040b5b60 0x7f140414e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.316+0000 7f140259c640 1 -- 172.21.15.16:0/3323517919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14040b5b60 msgr2=0x7f140414e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.316+0000 7f140259c640 1 --2- 172.21.15.16:0/3323517919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14040b5b60 0x7f140414e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.316+0000 7f140259c640 1 -- 172.21.15.16:0/3323517919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f140415dce0 msgr2=0x7f140415b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.316+0000 7f140259c640 1 --2- 172.21.15.16:0/3323517919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f140415dce0 0x7f140415b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.316+0000 7f140259c640 1 -- 172.21.15.16:0/3323517919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14040cce90 con 0x7f14040d35b0 2024-04-25T10:12:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.316+0000 7f140359e640 1 --2- 172.21.15.16:0/3323517919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f140415dce0 0x7f140415b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.316+0000 7f140259c640 1 --2- 172.21.15.16:0/3323517919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14040d35b0 0x7f140415f4f0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f13f800c520 tx=0x7f13f800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.317+0000 7f13eb7fe640 1 -- 172.21.15.16:0/3323517919 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8017070 con 0x7f14040d35b0 2024-04-25T10:12:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.317+0000 7f13eb7fe640 1 -- 172.21.15.16:0/3323517919 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f80035f0 con 0x7f14040d35b0 2024-04-25T10:12:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.317+0000 7f1409130640 1 -- 172.21.15.16:0/3323517919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14040cd170 con 0x7f14040d35b0 2024-04-25T10:12:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.317+0000 7f13eb7fe640 1 -- 172.21.15.16:0/3323517919 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f80039e0 con 0x7f14040d35b0 2024-04-25T10:12:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.317+0000 7f1409130640 1 -- 172.21.15.16:0/3323517919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14040cd730 con 0x7f14040d35b0 2024-04-25T10:12:43.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.318+0000 7f1409130640 1 -- 172.21.15.16:0/3323517919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14040749b0 con 0x7f14040d35b0 2024-04-25T10:12:43.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.318+0000 7f13eb7fe640 1 -- 172.21.15.16:0/3323517919 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f13f801a070 con 0x7f14040d35b0 2024-04-25T10:12:43.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.318+0000 7f13eb7fe640 1 --2- 172.21.15.16:0/3323517919 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f13f006b4d0 0x7f13f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.319+0000 7f13eb7fe640 1 -- 172.21.15.16:0/3323517919 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f13f8089380 con 0x7f14040d35b0 2024-04-25T10:12:43.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.319+0000 7f1402d9d640 1 --2- 172.21.15.16:0/3323517919 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f13f006b4d0 0x7f13f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.319+0000 7f1402d9d640 1 --2- 172.21.15.16:0/3323517919 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f13f006b4d0 0x7f13f006d990 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f13f4000f60 tx=0x7f13f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.323+0000 7f13eb7fe640 1 -- 172.21.15.16:0/3323517919 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f13f80584e0 con 0x7f14040d35b0 2024-04-25T10:12:43.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.570+0000 7f1409130640 1 -- 172.21.15.16:0/3323517919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7f14040cf5b0 con 0x7f14040d35b0 2024-04-25T10:12:43.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.573+0000 7f13eb7fe640 1 -- 172.21.15.16:0/3323517919 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v367) v1 ==== 149+0+0 (secure 0 0 0) 0x7f13f8057de0 con 0x7f14040d35b0 2024-04-25T10:12:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.575+0000 7f1409130640 1 -- 172.21.15.16:0/3323517919 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f13f006b4d0 msgr2=0x7f13f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.575+0000 7f1409130640 1 --2- 172.21.15.16:0/3323517919 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f13f006b4d0 0x7f13f006d990 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f13f4000f60 tx=0x7f13f4009040 comp rx=0 tx=0).stop 2024-04-25T10:12:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.575+0000 7f1409130640 1 -- 172.21.15.16:0/3323517919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14040d35b0 msgr2=0x7f140415f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.575+0000 7f1409130640 1 --2- 172.21.15.16:0/3323517919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14040d35b0 0x7f140415f4f0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f13f800c520 tx=0x7f13f800c9e0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.575+0000 7f1409130640 1 -- 172.21.15.16:0/3323517919 shutdown_connections 2024-04-25T10:12:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.575+0000 7f1409130640 1 --2- 172.21.15.16:0/3323517919 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f13f006b4d0 0x7f13f006d990 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.575+0000 7f1409130640 1 --2- 172.21.15.16:0/3323517919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f140415dce0 0x7f140415b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.575+0000 7f1409130640 1 --2- 172.21.15.16:0/3323517919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14040d35b0 0x7f140415f4f0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.575+0000 7f1409130640 1 --2- 172.21.15.16:0/3323517919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14040b5b60 0x7f140414e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.575+0000 7f1409130640 1 -- 172.21.15.16:0/3323517919 >> 172.21.15.16:0/3323517919 conn(0x7f1404072900 msgr2=0x7f14040ce250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.575+0000 7f1409130640 1 -- 172.21.15.16:0/3323517919 shutdown_connections 2024-04-25T10:12:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.575+0000 7f1409130640 1 -- 172.21.15.16:0/3323517919 wait complete. 2024-04-25T10:12:43.591 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:12:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.690+0000 7fe7bbb69640 1 Processor -- start 2024-04-25T10:12:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.691+0000 7fe7bbb69640 1 -- start start 2024-04-25T10:12:43.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.691+0000 7fe7bbb69640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7b415dc80 0x7fe7b4161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.691+0000 7fe7bbb69640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7b40b97c0 0x7fe7b40b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.691+0000 7fe7bbb69640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b41627a0 con 0x7fe7b40b5a30 2024-04-25T10:12:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.691+0000 7fe7bbb69640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b40b5ea0 con 0x7fe7b415dc80 2024-04-25T10:12:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.691+0000 7fe7bbb69640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b41621e0 con 0x7fe7b40b97c0 2024-04-25T10:12:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.692+0000 7fe7b9905640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7b415dc80 0x7fe7b4161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.692+0000 7fe7b9905640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7b415dc80 0x7fe7b4161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60814/0 (socket says 172.21.15.16:60814) 2024-04-25T10:12:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.692+0000 7fe7b9905640 1 -- 172.21.15.16:0/4150934635 learned_addr learned my addr 172.21.15.16:0/4150934635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.692+0000 7fe7b9104640 1 --2- 172.21.15.16:0/4150934635 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7b40b97c0 0x7fe7b40b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.692+0000 7fe7b9905640 1 -- 172.21.15.16:0/4150934635 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7b40b97c0 msgr2=0x7fe7b40b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.692+0000 7fe7b9905640 1 --2- 172.21.15.16:0/4150934635 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7b40b97c0 0x7fe7b40b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.692+0000 7fe7b9905640 1 -- 172.21.15.16:0/4150934635 >> v1:172.21.15.186:6789/0 conn(0x7fe7b40b5a30 legacy=0x7fe7b406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.692+0000 7fe7b9905640 1 -- 172.21.15.16:0/4150934635 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b41624c0 con 0x7fe7b415dc80 2024-04-25T10:12:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.692+0000 7fe7b9104640 1 --2- 172.21.15.16:0/4150934635 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7b40b97c0 0x7fe7b40b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.693+0000 7fe7b9905640 1 --2- 172.21.15.16:0/4150934635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7b415dc80 0x7fe7b4161870 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009870 tx=0x7fe7b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3baadeefafa59a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.693+0000 7fe7b8903640 1 -- 172.21.15.16:0/4150934635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0037070 con 0x7fe7b415dc80 2024-04-25T10:12:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.693+0000 7fe7b8903640 1 -- 172.21.15.16:0/4150934635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b0032440 con 0x7fe7b415dc80 2024-04-25T10:12:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.693+0000 7fe7b8903640 1 -- 172.21.15.16:0/4150934635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0032810 con 0x7fe7b415dc80 2024-04-25T10:12:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.693+0000 7fe7bbb69640 1 -- 172.21.15.16:0/4150934635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7b415dc80 msgr2=0x7fe7b4161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.693+0000 7fe7bbb69640 1 --2- 172.21.15.16:0/4150934635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7b415dc80 0x7fe7b4161870 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009870 tx=0x7fe7b002a270 comp rx=0 tx=0).stop 2024-04-25T10:12:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.693+0000 7fe7bbb69640 1 -- 172.21.15.16:0/4150934635 shutdown_connections 2024-04-25T10:12:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.693+0000 7fe7bbb69640 1 --2- 172.21.15.16:0/4150934635 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7b40b97c0 0x7fe7b40b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.693+0000 7fe7bbb69640 1 --2- 172.21.15.16:0/4150934635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7b415dc80 0x7fe7b4161870 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.693+0000 7fe7bbb69640 1 -- 172.21.15.16:0/4150934635 >> 172.21.15.16:0/4150934635 conn(0x7fe7b40d04f0 msgr2=0x7fe7b40d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.693+0000 7fe7bbb69640 1 -- 172.21.15.16:0/4150934635 shutdown_connections 2024-04-25T10:12:43.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.693+0000 7fe7bbb69640 1 -- 172.21.15.16:0/4150934635 wait complete. 2024-04-25T10:12:43.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.695+0000 7fe7bbb69640 1 Processor -- start 2024-04-25T10:12:43.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.695+0000 7fe7bbb69640 1 -- start start 2024-04-25T10:12:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.695+0000 7fe7bbb69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7b40b5a30 0x7fe7b40c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.695+0000 7fe7bbb69640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7b40b97c0 0x7fe7b415f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.695+0000 7fe7bbb69640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7b415dc80 0x7fe7b415f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.695+0000 7fe7bbb69640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b4161db0 con 0x7fe7b40b5a30 2024-04-25T10:12:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.695+0000 7fe7bbb69640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b415e060 con 0x7fe7b415dc80 2024-04-25T10:12:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.695+0000 7fe7bbb69640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b4161f30 con 0x7fe7b40b97c0 2024-04-25T10:12:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.695+0000 7fe7b9905640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7b40b5a30 0x7fe7b40c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.695+0000 7fe7b9905640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7b40b5a30 0x7fe7b40c6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60822/0 (socket says 172.21.15.16:60822) 2024-04-25T10:12:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.695+0000 7fe7b9905640 1 -- 172.21.15.16:0/2052173641 learned_addr learned my addr 172.21.15.16:0/2052173641 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.696+0000 7fe7ba106640 1 --2- 172.21.15.16:0/2052173641 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7b415dc80 0x7fe7b415f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.696+0000 7fe7b9905640 1 -- 172.21.15.16:0/2052173641 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7b40b97c0 msgr2=0x7fe7b415f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.696+0000 7fe7b9905640 1 --2- 172.21.15.16:0/2052173641 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7b40b97c0 0x7fe7b415f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.696+0000 7fe7b9905640 1 -- 172.21.15.16:0/2052173641 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7b415dc80 msgr2=0x7fe7b415f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.696+0000 7fe7b9905640 1 --2- 172.21.15.16:0/2052173641 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7b415dc80 0x7fe7b415f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.696+0000 7fe7b9905640 1 -- 172.21.15.16:0/2052173641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b40cdc50 con 0x7fe7b40b5a30 2024-04-25T10:12:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.696+0000 7fe7b9905640 1 --2- 172.21.15.16:0/2052173641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7b40b5a30 0x7fe7b40c6f50 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fe7b002a2b0 tx=0x7fe7b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.696+0000 7fe7aa7fc640 1 -- 172.21.15.16:0/2052173641 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0041020 con 0x7fe7b40b5a30 2024-04-25T10:12:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.696+0000 7fe7aa7fc640 1 -- 172.21.15.16:0/2052173641 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b00306c0 con 0x7fe7b40b5a30 2024-04-25T10:12:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.696+0000 7fe7bbb69640 1 -- 172.21.15.16:0/2052173641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b40cb060 con 0x7fe7b40b5a30 2024-04-25T10:12:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.696+0000 7fe7aa7fc640 1 -- 172.21.15.16:0/2052173641 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0037040 con 0x7fe7b40b5a30 2024-04-25T10:12:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.696+0000 7fe7bbb69640 1 -- 172.21.15.16:0/2052173641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b40cd8b0 con 0x7fe7b40b5a30 2024-04-25T10:12:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.697+0000 7fe7bbb69640 1 -- 172.21.15.16:0/2052173641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b4148ae0 con 0x7fe7b40b5a30 2024-04-25T10:12:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.697+0000 7fe7aa7fc640 1 -- 172.21.15.16:0/2052173641 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7b0044020 con 0x7fe7b40b5a30 2024-04-25T10:12:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.698+0000 7fe7aa7fc640 1 --2- 172.21.15.16:0/2052173641 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe79806b4d0 0x7fe79806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.698+0000 7fe7aa7fc640 1 -- 172.21.15.16:0/2052173641 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fe7b00aa700 con 0x7fe7b40b5a30 2024-04-25T10:12:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.698+0000 7fe7b9104640 1 --2- 172.21.15.16:0/2052173641 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe79806b4d0 0x7fe79806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.699+0000 7fe7b9104640 1 --2- 172.21.15.16:0/2052173641 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe79806b4d0 0x7fe79806d990 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4006fd0 tx=0x7fe7a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.702+0000 7fe7aa7fc640 1 -- 172.21.15.16:0/2052173641 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe7b004b030 con 0x7fe7b40b5a30 2024-04-25T10:12:43.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.882+0000 7fe7bbb69640 1 -- 172.21.15.16:0/2052173641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe7b40750b0 con 0x7fe7b40b5a30 2024-04-25T10:12:43.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.885+0000 7fe7aa7fc640 1 -- 172.21.15.16:0/2052173641 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe7b0079860 con 0x7fe7b40b5a30 2024-04-25T10:12:43.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.887+0000 7fe7bbb69640 1 -- 172.21.15.16:0/2052173641 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe79806b4d0 msgr2=0x7fe79806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.887+0000 7fe7bbb69640 1 --2- 172.21.15.16:0/2052173641 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe79806b4d0 0x7fe79806d990 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4006fd0 tx=0x7fe7a4008040 comp rx=0 tx=0).stop 2024-04-25T10:12:43.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.887+0000 7fe7bbb69640 1 -- 172.21.15.16:0/2052173641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7b40b5a30 msgr2=0x7fe7b40c6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.887+0000 7fe7bbb69640 1 --2- 172.21.15.16:0/2052173641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7b40b5a30 0x7fe7b40c6f50 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fe7b002a2b0 tx=0x7fe7b00046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.887+0000 7fe7bbb69640 1 -- 172.21.15.16:0/2052173641 shutdown_connections 2024-04-25T10:12:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.887+0000 7fe7bbb69640 1 --2- 172.21.15.16:0/2052173641 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe79806b4d0 0x7fe79806d990 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.887+0000 7fe7bbb69640 1 --2- 172.21.15.16:0/2052173641 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7b415dc80 0x7fe7b415f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.887+0000 7fe7bbb69640 1 --2- 172.21.15.16:0/2052173641 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7b40b97c0 0x7fe7b415f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.887+0000 7fe7bbb69640 1 --2- 172.21.15.16:0/2052173641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7b40b5a30 0x7fe7b40c6f50 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.887+0000 7fe7bbb69640 1 -- 172.21.15.16:0/2052173641 >> 172.21.15.16:0/2052173641 conn(0x7fe7b40d04f0 msgr2=0x7fe7b40b7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.887+0000 7fe7bbb69640 1 -- 172.21.15.16:0/2052173641 shutdown_connections 2024-04-25T10:12:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:43.887+0000 7fe7bbb69640 1 -- 172.21.15.16:0/2052173641 wait complete. 2024-04-25T10:12:43.903 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:12:43.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:43.950 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:43.996 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:12:43.996 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:12:43.996 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:44.012 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:12:44.012 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:12:44.012 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:44.029 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:12:44.029 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:12:44.029 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:44.068 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:12:44.068 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:12:44.068 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:44.086 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:12:44.086 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:12:44.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:44.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.204+0000 7f8d0275c640 1 Processor -- start 2024-04-25T10:12:44.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.204+0000 7f8d0275c640 1 -- start start 2024-04-25T10:12:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.205+0000 7f8d0275c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8cfc0d35b0 0x7f8cfc0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.205+0000 7f8d0275c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8cfc0b5990 0x7f8cfc0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.205+0000 7f8d0275c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cfc15e0c0 con 0x7f8cfc15dce0 2024-04-25T10:12:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.205+0000 7f8d0275c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cfc0d2510 con 0x7f8cfc0d35b0 2024-04-25T10:12:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.205+0000 7f8d0275c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0d27f0 con 0x7f8cfc0b5990 2024-04-25T10:12:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.205+0000 7f8cfbfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8cfc15dce0 0x7f8cfc0d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45144/0 (socket says 172.21.15.16:45144) 2024-04-25T10:12:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.205+0000 7f8cfbfff640 1 -- 172.21.15.16:0/3285994391 learned_addr learned my addr 172.21.15.16:0/3285994391 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.205+0000 7f8cfaffd640 1 -- 172.21.15.16:0/3285994391 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1288250397 0 0) 0x7f8cfc15e0c0 con 0x7f8cfc15dce0 2024-04-25T10:12:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.205+0000 7f8cfb7fe640 1 --2- 172.21.15.16:0/3285994391 >> v2:172.21.15.186:3300/0 conn(0x7f8cfc0d35b0 0x7f8cfc0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.205+0000 7f8cfaffd640 1 -- 172.21.15.16:0/3285994391 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ce4003520 con 0x7f8cfc15dce0 2024-04-25T10:12:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.205+0000 7f8d00cf9640 1 --2- 172.21.15.16:0/3285994391 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8cfc0b5990 0x7f8cfc0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8cfaffd640 1 -- 172.21.15.16:0/3285994391 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3484371796 0 0) 0x7f8ce4003520 con 0x7f8cfc15dce0 2024-04-25T10:12:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8cfaffd640 1 -- 172.21.15.16:0/3285994391 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8cfc0b5990 msgr2=0x7f8cfc0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8cfaffd640 1 --2- 172.21.15.16:0/3285994391 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8cfc0b5990 0x7f8cfc0b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8cfaffd640 1 -- 172.21.15.16:0/3285994391 >> v2:172.21.15.186:3300/0 conn(0x7f8cfc0d35b0 msgr2=0x7f8cfc0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8cfaffd640 1 --2- 172.21.15.16:0/3285994391 >> v2:172.21.15.186:3300/0 conn(0x7f8cfc0d35b0 0x7f8cfc0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8cfaffd640 1 -- 172.21.15.16:0/3285994391 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc0d2ad0 con 0x7f8cfc15dce0 2024-04-25T10:12:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8cfaffd640 1 -- 172.21.15.16:0/3285994391 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f8cec001670 con 0x7f8cfc15dce0 2024-04-25T10:12:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8cfaffd640 1 -- 172.21.15.16:0/3285994391 >> v1:172.21.15.16:6789/0 conn(0x7f8cfc15dce0 legacy=0x7f8cfc0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8cfaffd640 1 --2- 172.21.15.16:0/3285994391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ce40039b0 0x7f8ce4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8cfaffd640 1 --2- 172.21.15.16:0/3285994391 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8cfc0b5990 0x7f8ce4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8cfaffd640 1 --2- 172.21.15.16:0/3285994391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce4004d40 0x7f8ce40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8cfaffd640 1 -- 172.21.15.16:0/3285994391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0d2510 con 0x7f8ce4004d40 2024-04-25T10:12:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8cfaffd640 1 -- 172.21.15.16:0/3285994391 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0d27f0 con 0x7f8ce40039b0 2024-04-25T10:12:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8cfaffd640 1 -- 172.21.15.16:0/3285994391 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce400b9c0 con 0x7f8cfc0b5990 2024-04-25T10:12:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8cfbfff640 1 --2- 172.21.15.16:0/3285994391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ce40039b0 0x7f8ce4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8d00cf9640 1 --2- 172.21.15.16:0/3285994391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce4004d40 0x7f8ce40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8cfb7fe640 1 --2- 172.21.15.16:0/3285994391 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8cfc0b5990 0x7f8ce4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8d00cf9640 1 -- 172.21.15.16:0/3285994391 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8cfc0b5990 msgr2=0x7f8ce4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8d00cf9640 1 --2- 172.21.15.16:0/3285994391 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8cfc0b5990 0x7f8ce4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8d00cf9640 1 -- 172.21.15.16:0/3285994391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ce40039b0 msgr2=0x7f8ce4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8d00cf9640 1 --2- 172.21.15.16:0/3285994391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ce40039b0 0x7f8ce4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.206+0000 7f8d00cf9640 1 -- 172.21.15.16:0/3285994391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce4003520 con 0x7f8ce4004d40 2024-04-25T10:12:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.207+0000 7f8d00cf9640 1 --2- 172.21.15.16:0/3285994391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce4004d40 0x7f8ce40051c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400e170 tx=0x7f8ce4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e9fb5cc94655a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.207+0000 7f8cfaffd640 1 -- 172.21.15.16:0/3285994391 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4034020 con 0x7f8ce4004d40 2024-04-25T10:12:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.207+0000 7f8cfaffd640 1 -- 172.21.15.16:0/3285994391 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf40305f0 con 0x7f8ce4004d40 2024-04-25T10:12:44.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.207+0000 7f8cfaffd640 1 -- 172.21.15.16:0/3285994391 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4009c10 con 0x7f8ce4004d40 2024-04-25T10:12:44.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.208+0000 7f8d0275c640 1 -- 172.21.15.16:0/3285994391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce4004d40 msgr2=0x7f8ce40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.208+0000 7f8d0275c640 1 --2- 172.21.15.16:0/3285994391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce4004d40 0x7f8ce40051c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400e170 tx=0x7f8ce4005830 comp rx=0 tx=0).stop 2024-04-25T10:12:44.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.208+0000 7f8d00cf9640 1 -- 172.21.15.16:0/3285994391 reap_dead start 2024-04-25T10:12:44.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.208+0000 7f8d0275c640 1 -- 172.21.15.16:0/3285994391 shutdown_connections 2024-04-25T10:12:44.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.208+0000 7f8d0275c640 1 -- 172.21.15.16:0/3285994391 >> 172.21.15.16:0/3285994391 conn(0x7f8cfc072900 msgr2=0x7f8cfc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:44.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.208+0000 7f8d0275c640 1 -- 172.21.15.16:0/3285994391 shutdown_connections 2024-04-25T10:12:44.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.208+0000 7f8d0275c640 1 -- 172.21.15.16:0/3285994391 wait complete. 2024-04-25T10:12:44.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.209+0000 7f8d0275c640 1 Processor -- start 2024-04-25T10:12:44.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.209+0000 7f8d0275c640 1 -- start start 2024-04-25T10:12:44.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.210+0000 7f8d0275c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc160c80 0x7f8cfc0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.210+0000 7f8d0275c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cfc156470 0x7f8cfc1568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.210+0000 7f8d0275c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8cfc1f3b30 0x7f8cfc1f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.210+0000 7f8d0275c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0d2160 con 0x7f8cfc160c80 2024-04-25T10:12:44.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.210+0000 7f8d0275c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0d3090 con 0x7f8cfc156470 2024-04-25T10:12:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.210+0000 7f8d0275c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0cd530 con 0x7f8cfc1f3b30 2024-04-25T10:12:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.210+0000 7f8cfbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc160c80 0x7f8cfc0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.210+0000 7f8cfbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc160c80 0x7f8cfc0cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60834/0 (socket says 172.21.15.16:60834) 2024-04-25T10:12:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.210+0000 7f8cfbfff640 1 -- 172.21.15.16:0/2691633971 learned_addr learned my addr 172.21.15.16:0/2691633971 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.210+0000 7f8cfb7fe640 1 --2- 172.21.15.16:0/2691633971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cfc156470 0x7f8cfc1568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.210+0000 7f8cfbfff640 1 -- 172.21.15.16:0/2691633971 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8cfc1f3b30 msgr2=0x7f8cfc1f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.210+0000 7f8cfbfff640 1 --2- 172.21.15.16:0/2691633971 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8cfc1f3b30 0x7f8cfc1f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.210+0000 7f8cfbfff640 1 -- 172.21.15.16:0/2691633971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cfc156470 msgr2=0x7f8cfc1568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.210+0000 7f8cfbfff640 1 --2- 172.21.15.16:0/2691633971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cfc156470 0x7f8cfc1568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.210+0000 7f8cfbfff640 1 -- 172.21.15.16:0/2691633971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc0cc490 con 0x7f8cfc160c80 2024-04-25T10:12:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.210+0000 7f8cfb7fe640 1 --2- 172.21.15.16:0/2691633971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cfc156470 0x7f8cfc1568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.210+0000 7f8cfbfff640 1 --2- 172.21.15.16:0/2691633971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc160c80 0x7f8cfc0cbaf0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f8cec007290 tx=0x7f8cec007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.210+0000 7f8cf8ff9640 1 -- 172.21.15.16:0/2691633971 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec00f940 con 0x7f8cfc160c80 2024-04-25T10:12:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.211+0000 7f8cf8ff9640 1 -- 172.21.15.16:0/2691633971 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cec018070 con 0x7f8cfc160c80 2024-04-25T10:12:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.211+0000 7f8d0275c640 1 -- 172.21.15.16:0/2691633971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc0cc770 con 0x7f8cfc160c80 2024-04-25T10:12:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.211+0000 7f8cf8ff9640 1 -- 172.21.15.16:0/2691633971 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec00fd60 con 0x7f8cfc160c80 2024-04-25T10:12:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.211+0000 7f8d0275c640 1 -- 172.21.15.16:0/2691633971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc0ccd30 con 0x7f8cfc160c80 2024-04-25T10:12:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.211+0000 7f8d0275c640 1 -- 172.21.15.16:0/2691633971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc0749b0 con 0x7f8cfc160c80 2024-04-25T10:12:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.212+0000 7f8cf8ff9640 1 -- 172.21.15.16:0/2691633971 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8cec01d460 con 0x7f8cfc160c80 2024-04-25T10:12:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.212+0000 7f8cf8ff9640 1 --2- 172.21.15.16:0/2691633971 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8ce806b500 0x7f8ce806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.212+0000 7f8cf8ff9640 1 -- 172.21.15.16:0/2691633971 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f8cec08a0a0 con 0x7f8cfc160c80 2024-04-25T10:12:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.213+0000 7f8cfb7fe640 1 --2- 172.21.15.16:0/2691633971 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8ce806b500 0x7f8ce806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.213+0000 7f8cfb7fe640 1 --2- 172.21.15.16:0/2691633971 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8ce806b500 0x7f8ce806d9c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc15ed90 tx=0x7f8cf0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:44.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.217+0000 7f8cf8ff9640 1 -- 172.21.15.16:0/2691633971 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8cec059160 con 0x7f8cfc160c80 2024-04-25T10:12:44.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.435+0000 7f8d0275c640 1 -- 172.21.15.16:0/2691633971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cfc0b3cf0 con 0x7f8cfc160c80 2024-04-25T10:12:44.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.436+0000 7f8cf8ff9640 1 -- 172.21.15.16:0/2691633971 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f8cec058a60 con 0x7f8cfc160c80 2024-04-25T10:12:44.440 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:44.441 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":66,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4044509901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4044509901},{"type":"v1","addr":"172.21.15.16:6835","nonce":4044509901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1494230493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1494230493},{"type":"v1","addr":"172.21.15.186:6837","nonce":1494230493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/1050183001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1050183001},{"type":"v1","addr":"172.21.15.186:6836","nonce":1050183001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8121,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3103611057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3103611057},{"type":"v1","addr":"172.21.15.16:6834","nonce":3103611057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[]} 2024-04-25T10:12:44.441 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 66 2024-04-25T10:12:44.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.437+0000 7f8d0275c640 1 -- 172.21.15.16:0/2691633971 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8ce806b500 msgr2=0x7f8ce806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.438+0000 7f8d0275c640 1 --2- 172.21.15.16:0/2691633971 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8ce806b500 0x7f8ce806d9c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc15ed90 tx=0x7f8cf0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:44.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.438+0000 7f8d0275c640 1 -- 172.21.15.16:0/2691633971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc160c80 msgr2=0x7f8cfc0cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.438+0000 7f8d0275c640 1 --2- 172.21.15.16:0/2691633971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc160c80 0x7f8cfc0cbaf0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f8cec007290 tx=0x7f8cec007750 comp rx=0 tx=0).stop 2024-04-25T10:12:44.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.438+0000 7f8d0275c640 1 -- 172.21.15.16:0/2691633971 shutdown_connections 2024-04-25T10:12:44.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.438+0000 7f8d0275c640 1 --2- 172.21.15.16:0/2691633971 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8ce806b500 0x7f8ce806d9c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.438+0000 7f8d0275c640 1 --2- 172.21.15.16:0/2691633971 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8cfc1f3b30 0x7f8cfc1f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.438+0000 7f8d0275c640 1 --2- 172.21.15.16:0/2691633971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cfc156470 0x7f8cfc1568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.438+0000 7f8d0275c640 1 --2- 172.21.15.16:0/2691633971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc160c80 0x7f8cfc0cbaf0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.438+0000 7f8d0275c640 1 -- 172.21.15.16:0/2691633971 >> 172.21.15.16:0/2691633971 conn(0x7f8cfc072900 msgr2=0x7f8cfc0b8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:44.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.438+0000 7f8d0275c640 1 -- 172.21.15.16:0/2691633971 shutdown_connections 2024-04-25T10:12:44.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.438+0000 7f8d0275c640 1 -- 172.21.15.16:0/2691633971 wait complete. 2024-04-25T10:12:44.454 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:12:44.454 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:12:44.454 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:12:44.455 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:12:44.455 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:12:44.455 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:12:44.455 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:12:44.455 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:12:44.455 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:12:44.455 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:12:44.456 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:12:44.456 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:12:44.456 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:12:44.456 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:12:44.456 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:12:44.456 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:12:44.466 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:12:44.466 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:12:44.467 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:12:44.467 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:12:44.468 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:12:44.469 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:12:44.469 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:12:44.469 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:12:44.509 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:12:44.509 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:44.510 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:12:44.511 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:44.511 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:12:44.511 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:44.512 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:12:44.512 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:44.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:12:44.585 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-04-25T10:12:44.585 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-04-25T10:12:44.592 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2024-04-25T10:12:44.592 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2024-04-25T10:12:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.614+0000 7f2cf6b1b640 1 Processor -- start 2024-04-25T10:12:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.614+0000 7f2cf6b1b640 1 -- start start 2024-04-25T10:12:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.614+0000 7f2cf6b1b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cf0161090 0x7f2cf0161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.614+0000 7f2cf6b1b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cf00b9720 0x7f2cf00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.615+0000 7f2cf6b1b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf0161f80 con 0x7f2cf00cece0 2024-04-25T10:12:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.615+0000 7f2cf6b1b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf00cf130 con 0x7f2cf0161090 2024-04-25T10:12:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.615+0000 7f2cf6b1b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf00d21a0 con 0x7f2cf00b9720 2024-04-25T10:12:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.615+0000 7f2cf48b7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cf0161090 0x7f2cf0161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.615+0000 7f2cf48b7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cf0161090 0x7f2cf0161730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60856/0 (socket says 172.21.15.16:60856) 2024-04-25T10:12:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.615+0000 7f2cf48b7640 1 -- 172.21.15.16:0/105515863 learned_addr learned my addr 172.21.15.16:0/105515863 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.615+0000 7f2ceffff640 1 --2- 172.21.15.16:0/105515863 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cf00b9720 0x7f2cf00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.615+0000 7f2cf48b7640 1 -- 172.21.15.16:0/105515863 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cf00b9720 msgr2=0x7f2cf00ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.615+0000 7f2cf48b7640 1 --2- 172.21.15.16:0/105515863 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cf00b9720 0x7f2cf00ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.615+0000 7f2cf48b7640 1 -- 172.21.15.16:0/105515863 >> v1:172.21.15.186:6789/0 conn(0x7f2cf00cece0 legacy=0x7f2cf006c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.615+0000 7f2cf48b7640 1 -- 172.21.15.16:0/105515863 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf0162260 con 0x7f2cf0161090 2024-04-25T10:12:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.616+0000 7f2cf48b7640 1 --2- 172.21.15.16:0/105515863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0161090 0x7f2cf0161730 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009870 tx=0x7f2ce402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6675ebca75c07961 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.616+0000 7f2cef7fe640 1 -- 172.21.15.16:0/105515863 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4037070 con 0x7f2cf0161090 2024-04-25T10:12:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.616+0000 7f2cef7fe640 1 -- 172.21.15.16:0/105515863 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce4032440 con 0x7f2cf0161090 2024-04-25T10:12:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.616+0000 7f2cef7fe640 1 -- 172.21.15.16:0/105515863 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4032810 con 0x7f2cf0161090 2024-04-25T10:12:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.616+0000 7f2cf6b1b640 1 -- 172.21.15.16:0/105515863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0161090 msgr2=0x7f2cf0161730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.616+0000 7f2cf6b1b640 1 --2- 172.21.15.16:0/105515863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0161090 0x7f2cf0161730 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009870 tx=0x7f2ce402a270 comp rx=0 tx=0).stop 2024-04-25T10:12:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.616+0000 7f2cf6b1b640 1 -- 172.21.15.16:0/105515863 shutdown_connections 2024-04-25T10:12:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.616+0000 7f2cf6b1b640 1 --2- 172.21.15.16:0/105515863 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cf00b9720 0x7f2cf00ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.616+0000 7f2cf6b1b640 1 --2- 172.21.15.16:0/105515863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0161090 0x7f2cf0161730 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.616+0000 7f2cf6b1b640 1 -- 172.21.15.16:0/105515863 >> 172.21.15.16:0/105515863 conn(0x7f2cf00b55e0 msgr2=0x7f2cf00b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.616+0000 7f2cf6b1b640 1 -- 172.21.15.16:0/105515863 shutdown_connections 2024-04-25T10:12:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.616+0000 7f2cf6b1b640 1 -- 172.21.15.16:0/105515863 wait complete. 2024-04-25T10:12:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.618+0000 7f2cf6b1b640 1 Processor -- start 2024-04-25T10:12:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.618+0000 7f2cf6b1b640 1 -- start start 2024-04-25T10:12:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.618+0000 7f2cf6b1b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf00b9720 0x7f2cf00caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.618+0000 7f2cf6b1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf00cece0 0x7f2cf014ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.618+0000 7f2cf6b1b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cf0161090 0x7f2cf00cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.618+0000 7f2cf6b1b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf0161470 con 0x7f2cf00cece0 2024-04-25T10:12:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.618+0000 7f2cf6b1b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf00d1e30 con 0x7f2cf00b9720 2024-04-25T10:12:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.618+0000 7f2cf6b1b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf0161c70 con 0x7f2cf0161090 2024-04-25T10:12:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.619+0000 7f2ceffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf00cece0 0x7f2cf014ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.619+0000 7f2ceffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf00cece0 0x7f2cf014ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60860/0 (socket says 172.21.15.16:60860) 2024-04-25T10:12:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.619+0000 7f2ceffff640 1 -- 172.21.15.16:0/1843301371 learned_addr learned my addr 172.21.15.16:0/1843301371 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.619+0000 7f2ceffff640 1 -- 172.21.15.16:0/1843301371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cf0161090 msgr2=0x7f2cf00cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.619+0000 7f2cf50b8640 1 --2- 172.21.15.16:0/1843301371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cf0161090 0x7f2cf00cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.619+0000 7f2ceffff640 1 --2- 172.21.15.16:0/1843301371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cf0161090 0x7f2cf00cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.619+0000 7f2cf48b7640 1 --2- 172.21.15.16:0/1843301371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf00b9720 0x7f2cf00caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.619+0000 7f2ceffff640 1 -- 172.21.15.16:0/1843301371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf00b9720 msgr2=0x7f2cf00caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.619+0000 7f2ceffff640 1 --2- 172.21.15.16:0/1843301371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf00b9720 0x7f2cf00caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.619+0000 7f2ceffff640 1 -- 172.21.15.16:0/1843301371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf01f3bd0 con 0x7f2cf00cece0 2024-04-25T10:12:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.619+0000 7f2ceffff640 1 --2- 172.21.15.16:0/1843301371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf00cece0 0x7f2cf014ae50 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00e9b0 tx=0x7f2cdc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.619+0000 7f2ced7fa640 1 -- 172.21.15.16:0/1843301371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc017070 con 0x7f2cf00cece0 2024-04-25T10:12:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.619+0000 7f2ced7fa640 1 -- 172.21.15.16:0/1843301371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc00c9a0 con 0x7f2cf00cece0 2024-04-25T10:12:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.619+0000 7f2ced7fa640 1 -- 172.21.15.16:0/1843301371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc00cd90 con 0x7f2cf00cece0 2024-04-25T10:12:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.619+0000 7f2cf6b1b640 1 -- 172.21.15.16:0/1843301371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf01f3eb0 con 0x7f2cf00cece0 2024-04-25T10:12:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.619+0000 7f2cf6b1b640 1 -- 172.21.15.16:0/1843301371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf01f4470 con 0x7f2cf00cece0 2024-04-25T10:12:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.620+0000 7f2cf6b1b640 1 -- 172.21.15.16:0/1843301371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf00750b0 con 0x7f2cf00cece0 2024-04-25T10:12:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.621+0000 7f2ced7fa640 1 -- 172.21.15.16:0/1843301371 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2cdc01a070 con 0x7f2cf00cece0 2024-04-25T10:12:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.621+0000 7f2ced7fa640 1 --2- 172.21.15.16:0/1843301371 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2cd406b4d0 0x7f2cd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.621+0000 7f2ced7fa640 1 -- 172.21.15.16:0/1843301371 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f2cdc089f60 con 0x7f2cf00cece0 2024-04-25T10:12:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.622+0000 7f2cf48b7640 1 --2- 172.21.15.16:0/1843301371 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2cd406b4d0 0x7f2cd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.622+0000 7f2cf48b7640 1 --2- 172.21.15.16:0/1843301371 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2cd406b4d0 0x7f2cd406d990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4034000 tx=0x7f2ce402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.626+0000 7f2ced7fa640 1 -- 172.21.15.16:0/1843301371 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2cdc0590c0 con 0x7f2cf00cece0 2024-04-25T10:12:44.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:44.820+0000 7f2cf6b1b640 1 -- 172.21.15.16:0/1843301371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2cf00d1700 con 0x7f2cf00cece0 2024-04-25T10:12:45.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:45.897+0000 7f2ced7fa640 1 -- 172.21.15.16:0/1843301371 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v96) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2cdc0589c0 con 0x7f2cf00cece0 2024-04-25T10:12:45.902 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-04-25T10:12:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:45.901+0000 7f2cf6b1b640 1 -- 172.21.15.16:0/1843301371 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2cd406b4d0 msgr2=0x7f2cd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:45.901+0000 7f2cf6b1b640 1 --2- 172.21.15.16:0/1843301371 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2cd406b4d0 0x7f2cd406d990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4034000 tx=0x7f2ce402e000 comp rx=0 tx=0).stop 2024-04-25T10:12:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:45.901+0000 7f2cf6b1b640 1 -- 172.21.15.16:0/1843301371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf00cece0 msgr2=0x7f2cf014ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:45.901+0000 7f2cf6b1b640 1 --2- 172.21.15.16:0/1843301371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf00cece0 0x7f2cf014ae50 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00e9b0 tx=0x7f2cdc00ee70 comp rx=0 tx=0).stop 2024-04-25T10:12:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:45.901+0000 7f2cf6b1b640 1 -- 172.21.15.16:0/1843301371 shutdown_connections 2024-04-25T10:12:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:45.901+0000 7f2cf6b1b640 1 --2- 172.21.15.16:0/1843301371 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2cd406b4d0 0x7f2cd406d990 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:45.901+0000 7f2cf6b1b640 1 --2- 172.21.15.16:0/1843301371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cf0161090 0x7f2cf00cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:45.901+0000 7f2cf6b1b640 1 --2- 172.21.15.16:0/1843301371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf00cece0 0x7f2cf014ae50 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:45.901+0000 7f2cf6b1b640 1 --2- 172.21.15.16:0/1843301371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf00b9720 0x7f2cf00caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:45.901+0000 7f2cf6b1b640 1 -- 172.21.15.16:0/1843301371 >> 172.21.15.16:0/1843301371 conn(0x7f2cf00b55e0 msgr2=0x7f2cf00d3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:45.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:45.901+0000 7f2cf6b1b640 1 -- 172.21.15.16:0/1843301371 shutdown_connections 2024-04-25T10:12:45.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:45.901+0000 7f2cf6b1b640 1 -- 172.21.15.16:0/1843301371 wait complete. 2024-04-25T10:12:45.920 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:12:46.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.021+0000 7f606059e640 1 Processor -- start 2024-04-25T10:12:46.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.022+0000 7f606059e640 1 -- start start 2024-04-25T10:12:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.022+0000 7f606059e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f60580cd320 0x7f60580cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.022+0000 7f606059e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60580cd930 con 0x7f60580cfe80 2024-04-25T10:12:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.022+0000 7f606059e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60580cdc10 con 0x7f605815df40 2024-04-25T10:12:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.022+0000 7f606059e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60580cdef0 con 0x7f60580cd320 2024-04-25T10:12:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.023+0000 7f605db39640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f60580cd320 0x7f60580cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.023+0000 7f605eb3b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f60580cfe80 0x7f60580cb5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45156/0 (socket says 172.21.15.16:45156) 2024-04-25T10:12:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.023+0000 7f605eb3b640 1 -- 172.21.15.16:0/3165017883 learned_addr learned my addr 172.21.15.16:0/3165017883 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.023+0000 7f605db39640 1 -- 172.21.15.16:0/3165017883 >> v1:172.21.15.186:6789/0 conn(0x7f605815df40 legacy=0x7f60580b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.023+0000 7f605db39640 1 -- 172.21.15.16:0/3165017883 >> v1:172.21.15.16:6789/0 conn(0x7f60580cfe80 legacy=0x7f60580cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.023+0000 7f605db39640 1 -- 172.21.15.16:0/3165017883 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60580cada0 con 0x7f60580cd320 2024-04-25T10:12:46.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.024+0000 7f605db39640 1 --2- 172.21.15.16:0/3165017883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60580cd320 0x7f60580cf940 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f60480097a0 tx=0x7f604802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e066f2870be0e857 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.024+0000 7f605d338640 1 -- 172.21.15.16:0/3165017883 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048037070 con 0x7f60580cd320 2024-04-25T10:12:46.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.024+0000 7f605d338640 1 -- 172.21.15.16:0/3165017883 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6048032440 con 0x7f60580cd320 2024-04-25T10:12:46.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.024+0000 7f605d338640 1 -- 172.21.15.16:0/3165017883 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048032810 con 0x7f60580cd320 2024-04-25T10:12:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.024+0000 7f606059e640 1 -- 172.21.15.16:0/3165017883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60580cd320 msgr2=0x7f60580cf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.024+0000 7f606059e640 1 --2- 172.21.15.16:0/3165017883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60580cd320 0x7f60580cf940 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f60480097a0 tx=0x7f604802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.024+0000 7f606059e640 1 -- 172.21.15.16:0/3165017883 shutdown_connections 2024-04-25T10:12:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.024+0000 7f606059e640 1 --2- 172.21.15.16:0/3165017883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60580cd320 0x7f60580cf940 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.025+0000 7f606059e640 1 -- 172.21.15.16:0/3165017883 >> 172.21.15.16:0/3165017883 conn(0x7f60580d06b0 msgr2=0x7f60580d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.025+0000 7f606059e640 1 -- 172.21.15.16:0/3165017883 shutdown_connections 2024-04-25T10:12:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.025+0000 7f606059e640 1 -- 172.21.15.16:0/3165017883 wait complete. 2024-04-25T10:12:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.026+0000 7f606059e640 1 Processor -- start 2024-04-25T10:12:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.026+0000 7f606059e640 1 -- start start 2024-04-25T10:12:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.026+0000 7f606059e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60580cd320 0x7f60580b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.026+0000 7f606059e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60580cfe80 0x7f60580c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.026+0000 7f606059e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f605815df40 0x7f605814c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f606059e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60580c4200 con 0x7f60580cd320 2024-04-25T10:12:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f606059e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60580d0290 con 0x7f605815df40 2024-04-25T10:12:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f606059e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60580c4380 con 0x7f60580cfe80 2024-04-25T10:12:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f605e33a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60580cd320 0x7f60580b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f605e33a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60580cd320 0x7f60580b51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60874/0 (socket says 172.21.15.16:60874) 2024-04-25T10:12:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f605e33a640 1 -- 172.21.15.16:0/2464391734 learned_addr learned my addr 172.21.15.16:0/2464391734 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f605db39640 1 --2- 172.21.15.16:0/2464391734 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60580cfe80 0x7f60580c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f605eb3b640 1 --2- 172.21.15.16:0/2464391734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f605815df40 0x7f605814c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f605e33a640 1 -- 172.21.15.16:0/2464391734 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60580cfe80 msgr2=0x7f60580c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f605e33a640 1 --2- 172.21.15.16:0/2464391734 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60580cfe80 0x7f60580c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f605e33a640 1 -- 172.21.15.16:0/2464391734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f605815df40 msgr2=0x7f605814c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f605e33a640 1 --2- 172.21.15.16:0/2464391734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f605815df40 0x7f605814c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f605e33a640 1 -- 172.21.15.16:0/2464391734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605815e870 con 0x7f60580cd320 2024-04-25T10:12:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f605e33a640 1 --2- 172.21.15.16:0/2464391734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60580cd320 0x7f60580b51c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f605400c3c0 tx=0x7f605400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f604effd640 1 -- 172.21.15.16:0/2464391734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60540052d0 con 0x7f60580cd320 2024-04-25T10:12:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f604effd640 1 -- 172.21.15.16:0/2464391734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6054005490 con 0x7f60580cd320 2024-04-25T10:12:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f606059e640 1 -- 172.21.15.16:0/2464391734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605815eb50 con 0x7f60580cd320 2024-04-25T10:12:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f604effd640 1 -- 172.21.15.16:0/2464391734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6054007260 con 0x7f60580cd320 2024-04-25T10:12:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.027+0000 7f606059e640 1 -- 172.21.15.16:0/2464391734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605815f110 con 0x7f60580cd320 2024-04-25T10:12:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.028+0000 7f606059e640 1 -- 172.21.15.16:0/2464391734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60580749b0 con 0x7f60580cd320 2024-04-25T10:12:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.029+0000 7f604effd640 1 -- 172.21.15.16:0/2464391734 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6054007400 con 0x7f60580cd320 2024-04-25T10:12:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.029+0000 7f604effd640 1 --2- 172.21.15.16:0/2464391734 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f603c06b500 0x7f603c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.029+0000 7f604effd640 1 -- 172.21.15.16:0/2464391734 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6054089270 con 0x7f60580cd320 2024-04-25T10:12:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.030+0000 7f605db39640 1 --2- 172.21.15.16:0/2464391734 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f603c06b500 0x7f603c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.030+0000 7f605db39640 1 --2- 172.21.15.16:0/2464391734 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f603c06b500 0x7f603c06d9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6048034000 tx=0x7f604802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.034+0000 7f604effd640 1 -- 172.21.15.16:0/2464391734 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f60540584d0 con 0x7f60580cd320 2024-04-25T10:12:46.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.281+0000 7f606059e640 1 -- 172.21.15.16:0/2464391734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f605815f530 con 0x7f60580cd320 2024-04-25T10:12:46.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.282+0000 7f604effd640 1 -- 172.21.15.16:0/2464391734 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6054057dd0 con 0x7f60580cd320 2024-04-25T10:12:46.287 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:46.287 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:12:46.287 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-04-25T10:12:46.287 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.287 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-04-25T10:12:46.287 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmj+xmBBAAaWdHmoiYcXzFnMVQSioT2Q==", 2024-04-25T10:12:46.287 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.287 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:12:46.287 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:12:46.287 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:12:46.287 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:12:46.288 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:12:46.288 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.288 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-04-25T10:12:46.288 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmUES+DxAAu5It7dQmb7/BM/4x1fUYXA==", 2024-04-25T10:12:46.288 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.288 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:12:46.288 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:12:46.288 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:12:46.288 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:12:46.288 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:12:46.288 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.288 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-04-25T10:12:46.288 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipm+LtFCxAAGG79kv8eWzly8l5qzQwfww==", 2024-04-25T10:12:46.288 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:12:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:12:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:12:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:12:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:12:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-04-25T10:12:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmqbSSFhAAZtrLsAGSyaMGgv45Hy0WpA==", 2024-04-25T10:12:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:12:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:12:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:12:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:12:46.289 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:12:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-04-25T10:12:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCaKipm2+UdOxAAP8GBunvYsue8EILYV5Uq7A==", 2024-04-25T10:12:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:12:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:12:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:12:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-04-25T10:12:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCdKipmSv87ORAAigVU7nyzkcXqMEHKsSAsqg==", 2024-04-25T10:12:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.290 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:12:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:12:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:12:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-04-25T10:12:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCgKipm2VMLOBAAJETFfWgq7H/imQAfw93Ruw==", 2024-04-25T10:12:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:12:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:12:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:12:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.291 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-04-25T10:12:46.292 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCjKipmb1g0NhAADH2uHQPhW84KLYyWbOjQdw==", 2024-04-25T10:12:46.292 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.292 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:46.292 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:12:46.292 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:12:46.292 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:12:46.292 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.292 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-04-25T10:12:46.292 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCpKipmnH0ROBAAEb8i4nfswYQ/w5mEw48kIg==", 2024-04-25T10:12:46.292 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.292 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:46.292 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:12:46.292 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:12:46.292 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:12:46.293 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.293 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-04-25T10:12:46.293 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCsKipm8HCRNhAA3FhcMlUWxMAiFRXbBWWvNA==", 2024-04-25T10:12:46.293 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.293 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:46.293 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:12:46.293 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:12:46.293 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:12:46.293 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.293 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-04-25T10:12:46.293 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCvKipmfgqpNRAAtPhpSDU9q23Sfa+V7m+3NA==", 2024-04-25T10:12:46.293 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.293 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:46.293 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:12:46.294 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:12:46.294 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:12:46.294 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.294 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-04-25T10:12:46.294 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCyKipm3ndyMxAAeMNODGxNupb043CsA/SP1A==", 2024-04-25T10:12:46.294 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.294 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:46.294 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:12:46.294 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:12:46.294 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:12:46.294 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.294 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-04-25T10:12:46.295 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmAph/GhAAG3x1zC98yHQ13fjpmrlPkg==", 2024-04-25T10:12:46.295 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.295 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:12:46.295 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-04-25T10:12:46.295 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:12:46.295 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:12:46.295 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.295 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-04-25T10:12:46.295 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmrccTIxAAuDZamFXTe6J2d+HAcBe+DA==", 2024-04-25T10:12:46.295 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.295 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:12:46.295 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:12:46.295 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:12:46.296 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:12:46.296 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:12:46.296 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.296 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-04-25T10:12:46.296 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmsFACIBAAFuWsmuQlF58yzIcYUx/6Iw==", 2024-04-25T10:12:46.296 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.296 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:12:46.296 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:12:46.296 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:12:46.296 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:12:46.296 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:12:46.296 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.296 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-04-25T10:12:46.297 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmAe1/KBAAXzLkscuwpCwMnrIKQny9+Q==", 2024-04-25T10:12:46.297 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.297 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:12:46.297 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:12:46.297 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:12:46.297 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:12:46.297 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.297 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:12:46.297 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.297 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-04-25T10:12:46.297 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCYKipmVFQ2JBAAdqG2p/M746PZPMDcpQjwfw==", 2024-04-25T10:12:46.297 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.297 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:12:46.297 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:12:46.297 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:12:46.298 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:12:46.298 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.298 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:12:46.298 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.298 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-04-25T10:12:46.298 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmiPVHARAAGPbv3P3yVj3tlo0b2gmcyw==", 2024-04-25T10:12:46.298 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.298 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:12:46.298 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-04-25T10:12:46.298 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:12:46.298 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.298 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:12:46.298 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:46.298 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-04-25T10:12:46.299 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCYKipmghhyOBAAcCm0hQzU+Wlw45yZQETTTg==", 2024-04-25T10:12:46.299 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:12:46.299 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:12:46.299 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-04-25T10:12:46.299 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:12:46.299 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.299 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:46.299 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-04-25T10:12:46.299 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:12:46.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.284+0000 7f606059e640 1 -- 172.21.15.16:0/2464391734 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f603c06b500 msgr2=0x7f603c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.284+0000 7f606059e640 1 --2- 172.21.15.16:0/2464391734 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f603c06b500 0x7f603c06d9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6048034000 tx=0x7f604802e000 comp rx=0 tx=0).stop 2024-04-25T10:12:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.284+0000 7f606059e640 1 -- 172.21.15.16:0/2464391734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60580cd320 msgr2=0x7f60580b51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.284+0000 7f606059e640 1 --2- 172.21.15.16:0/2464391734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60580cd320 0x7f60580b51c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f605400c3c0 tx=0x7f605400c880 comp rx=0 tx=0).stop 2024-04-25T10:12:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.284+0000 7f606059e640 1 -- 172.21.15.16:0/2464391734 shutdown_connections 2024-04-25T10:12:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.284+0000 7f606059e640 1 --2- 172.21.15.16:0/2464391734 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f603c06b500 0x7f603c06d9c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.284+0000 7f606059e640 1 --2- 172.21.15.16:0/2464391734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f605815df40 0x7f605814c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.284+0000 7f606059e640 1 --2- 172.21.15.16:0/2464391734 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60580cfe80 0x7f60580c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.284+0000 7f606059e640 1 --2- 172.21.15.16:0/2464391734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60580cd320 0x7f60580b51c0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.284+0000 7f606059e640 1 -- 172.21.15.16:0/2464391734 >> 172.21.15.16:0/2464391734 conn(0x7f60580d06b0 msgr2=0x7f605806e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.284+0000 7f606059e640 1 -- 172.21.15.16:0/2464391734 shutdown_connections 2024-04-25T10:12:46.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.284+0000 7f606059e640 1 -- 172.21.15.16:0/2464391734 wait complete. 2024-04-25T10:12:46.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:46.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.398+0000 7f9e59522640 1 Processor -- start 2024-04-25T10:12:46.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.398+0000 7f9e59522640 1 -- start start 2024-04-25T10:12:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.399+0000 7f9e59522640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9e5415dce0 0x7f9e540d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.399+0000 7f9e59522640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e540b5b60 0x7f9e540b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.399+0000 7f9e59522640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e5415e0c0 con 0x7f9e540d35b0 2024-04-25T10:12:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.399+0000 7f9e59522640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e540d2510 con 0x7f9e5415dce0 2024-04-25T10:12:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.399+0000 7f9e59522640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e540d27f0 con 0x7f9e540b5b60 2024-04-25T10:12:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.399+0000 7f9e527fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9e540d35b0 0x7f9e540b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45158/0 (socket says 172.21.15.16:45158) 2024-04-25T10:12:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.399+0000 7f9e527fc640 1 -- 172.21.15.16:0/2513246524 learned_addr learned my addr 172.21.15.16:0/2513246524 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.399+0000 7f9e52ffd640 1 --2- 172.21.15.16:0/2513246524 >> v2:172.21.15.186:3300/0 conn(0x7f9e5415dce0 0x7f9e540d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.399+0000 7f9e537fe640 1 --2- 172.21.15.16:0/2513246524 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e540b5b60 0x7f9e540b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.399+0000 7f9e51ffb640 1 -- 172.21.15.16:0/2513246524 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409428825 0 0) 0x7f9e5415e0c0 con 0x7f9e540d35b0 2024-04-25T10:12:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.399+0000 7f9e51ffb640 1 -- 172.21.15.16:0/2513246524 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e3c003520 con 0x7f9e540d35b0 2024-04-25T10:12:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e51ffb640 1 -- 172.21.15.16:0/2513246524 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1564219417 0 0) 0x7f9e3c003520 con 0x7f9e540d35b0 2024-04-25T10:12:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e51ffb640 1 -- 172.21.15.16:0/2513246524 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e540b5b60 msgr2=0x7f9e540b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e51ffb640 1 --2- 172.21.15.16:0/2513246524 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e540b5b60 0x7f9e540b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e51ffb640 1 -- 172.21.15.16:0/2513246524 >> v2:172.21.15.186:3300/0 conn(0x7f9e5415dce0 msgr2=0x7f9e540d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e51ffb640 1 --2- 172.21.15.16:0/2513246524 >> v2:172.21.15.186:3300/0 conn(0x7f9e5415dce0 0x7f9e540d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e51ffb640 1 -- 172.21.15.16:0/2513246524 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e540d2ad0 con 0x7f9e540d35b0 2024-04-25T10:12:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e51ffb640 1 -- 172.21.15.16:0/2513246524 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f9e48001670 con 0x7f9e540d35b0 2024-04-25T10:12:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e51ffb640 1 -- 172.21.15.16:0/2513246524 >> v1:172.21.15.16:6789/0 conn(0x7f9e540d35b0 legacy=0x7f9e540b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e51ffb640 1 --2- 172.21.15.16:0/2513246524 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e3c0039b0 0x7f9e3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e51ffb640 1 --2- 172.21.15.16:0/2513246524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e3c0043a0 0x7f9e3c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e51ffb640 1 --2- 172.21.15.16:0/2513246524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e3c004ef0 0x7f9e3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e51ffb640 1 -- 172.21.15.16:0/2513246524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e540d2510 con 0x7f9e3c0043a0 2024-04-25T10:12:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e51ffb640 1 -- 172.21.15.16:0/2513246524 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e540d27f0 con 0x7f9e3c004ef0 2024-04-25T10:12:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e51ffb640 1 -- 172.21.15.16:0/2513246524 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c00bf00 con 0x7f9e3c0039b0 2024-04-25T10:12:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e52ffd640 1 --2- 172.21.15.16:0/2513246524 >> v2:172.21.15.186:3300/0 conn(0x7f9e5415dce0 0x7f9e540d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e537fe640 1 --2- 172.21.15.16:0/2513246524 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e540b5b60 0x7f9e540b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e527fc640 1 --2- 172.21.15.16:0/2513246524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e3c0043a0 0x7f9e3c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e537fe640 1 --2- 172.21.15.16:0/2513246524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e3c004ef0 0x7f9e3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e52ffd640 1 --2- 172.21.15.16:0/2513246524 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e3c0039b0 0x7f9e3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e527fc640 1 -- 172.21.15.16:0/2513246524 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e3c0039b0 msgr2=0x7f9e3c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e527fc640 1 --2- 172.21.15.16:0/2513246524 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e3c0039b0 0x7f9e3c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e527fc640 1 -- 172.21.15.16:0/2513246524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e3c004ef0 msgr2=0x7f9e3c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e527fc640 1 --2- 172.21.15.16:0/2513246524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e3c004ef0 0x7f9e3c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.400+0000 7f9e527fc640 1 -- 172.21.15.16:0/2513246524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c003520 con 0x7f9e3c0043a0 2024-04-25T10:12:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.401+0000 7f9e527fc640 1 --2- 172.21.15.16:0/2513246524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e3c0043a0 0x7f9e3c004880 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f9e48007fd0 tx=0x7f9e3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cbbb772fd57e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.401+0000 7f9e51ffb640 1 -- 172.21.15.16:0/2513246524 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e48007d50 con 0x7f9e3c0043a0 2024-04-25T10:12:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.401+0000 7f9e51ffb640 1 -- 172.21.15.16:0/2513246524 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e48005ad0 con 0x7f9e3c0043a0 2024-04-25T10:12:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.401+0000 7f9e51ffb640 1 -- 172.21.15.16:0/2513246524 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e48007070 con 0x7f9e3c0043a0 2024-04-25T10:12:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.401+0000 7f9e59522640 1 -- 172.21.15.16:0/2513246524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e3c0043a0 msgr2=0x7f9e3c004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.401+0000 7f9e59522640 1 --2- 172.21.15.16:0/2513246524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e3c0043a0 0x7f9e3c004880 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f9e48007fd0 tx=0x7f9e3c005830 comp rx=0 tx=0).stop 2024-04-25T10:12:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.401+0000 7f9e537fe640 1 -- 172.21.15.16:0/2513246524 reap_dead start 2024-04-25T10:12:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.401+0000 7f9e59522640 1 -- 172.21.15.16:0/2513246524 shutdown_connections 2024-04-25T10:12:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.401+0000 7f9e59522640 1 -- 172.21.15.16:0/2513246524 >> 172.21.15.16:0/2513246524 conn(0x7f9e54072900 msgr2=0x7f9e540d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.402+0000 7f9e59522640 1 -- 172.21.15.16:0/2513246524 shutdown_connections 2024-04-25T10:12:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.402+0000 7f9e59522640 1 -- 172.21.15.16:0/2513246524 wait complete. 2024-04-25T10:12:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.403+0000 7f9e59522640 1 Processor -- start 2024-04-25T10:12:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.403+0000 7f9e59522640 1 -- start start 2024-04-25T10:12:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.404+0000 7f9e59522640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e3c0043a0 0x7f9e5415eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.404+0000 7f9e59522640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e541f7660 0x7f9e541539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.404+0000 7f9e59522640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e54153ef0 0x7f9e54155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.404+0000 7f9e59522640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e540b9820 con 0x7f9e541f7660 2024-04-25T10:12:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.404+0000 7f9e59522640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e540d3090 con 0x7f9e3c0043a0 2024-04-25T10:12:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.404+0000 7f9e59522640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e540b99a0 con 0x7f9e54153ef0 2024-04-25T10:12:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.404+0000 7f9e527fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e541f7660 0x7f9e541539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.404+0000 7f9e527fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e541f7660 0x7f9e541539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60904/0 (socket says 172.21.15.16:60904) 2024-04-25T10:12:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.404+0000 7f9e527fc640 1 -- 172.21.15.16:0/147050510 learned_addr learned my addr 172.21.15.16:0/147050510 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.404+0000 7f9e52ffd640 1 --2- 172.21.15.16:0/147050510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e3c0043a0 0x7f9e5415eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.404+0000 7f9e537fe640 1 --2- 172.21.15.16:0/147050510 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e54153ef0 0x7f9e54155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.404+0000 7f9e527fc640 1 -- 172.21.15.16:0/147050510 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e54153ef0 msgr2=0x7f9e54155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.404+0000 7f9e527fc640 1 --2- 172.21.15.16:0/147050510 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e54153ef0 0x7f9e54155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.404+0000 7f9e527fc640 1 -- 172.21.15.16:0/147050510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e3c0043a0 msgr2=0x7f9e5415eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.404+0000 7f9e527fc640 1 --2- 172.21.15.16:0/147050510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e3c0043a0 0x7f9e5415eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.404+0000 7f9e527fc640 1 -- 172.21.15.16:0/147050510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5415f540 con 0x7f9e541f7660 2024-04-25T10:12:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.404+0000 7f9e527fc640 1 --2- 172.21.15.16:0/147050510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e541f7660 0x7f9e541539b0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800c380 tx=0x7f9e540d2ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.405+0000 7f9e3b7fe640 1 -- 172.21.15.16:0/147050510 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e48042030 con 0x7f9e541f7660 2024-04-25T10:12:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.405+0000 7f9e3b7fe640 1 -- 172.21.15.16:0/147050510 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e480049f0 con 0x7f9e541f7660 2024-04-25T10:12:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.405+0000 7f9e3b7fe640 1 -- 172.21.15.16:0/147050510 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e48001c80 con 0x7f9e541f7660 2024-04-25T10:12:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.405+0000 7f9e59522640 1 -- 172.21.15.16:0/147050510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5415f800 con 0x7f9e541f7660 2024-04-25T10:12:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.405+0000 7f9e59522640 1 -- 172.21.15.16:0/147050510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5415fdc0 con 0x7f9e541f7660 2024-04-25T10:12:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.406+0000 7f9e59522640 1 -- 172.21.15.16:0/147050510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e540749b0 con 0x7f9e541f7660 2024-04-25T10:12:46.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.406+0000 7f9e3b7fe640 1 -- 172.21.15.16:0/147050510 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e48045070 con 0x7f9e541f7660 2024-04-25T10:12:46.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.406+0000 7f9e3b7fe640 1 --2- 172.21.15.16:0/147050510 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9e4006b500 0x7f9e4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.406+0000 7f9e3b7fe640 1 -- 172.21.15.16:0/147050510 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9e480a9df0 con 0x7f9e541f7660 2024-04-25T10:12:46.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.407+0000 7f9e52ffd640 1 --2- 172.21.15.16:0/147050510 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9e4006b500 0x7f9e4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.407+0000 7f9e52ffd640 1 --2- 172.21.15.16:0/147050510 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9e4006b500 0x7f9e4006d9c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9e440093d0 tx=0x7f9e44008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.410+0000 7f9e3b7fe640 1 -- 172.21.15.16:0/147050510 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9e480788d0 con 0x7f9e541f7660 2024-04-25T10:12:46.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.592+0000 7f9e59522640 1 -- 172.21.15.16:0/147050510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e541542d0 con 0x7f9e541f7660 2024-04-25T10:12:46.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.593+0000 7f9e3b7fe640 1 -- 172.21.15.16:0/147050510 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+12892 (secure 0 0 0) 0x7f9e4807c970 con 0x7f9e541f7660 2024-04-25T10:12:46.597 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:46.597 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":96,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:12:45.896890+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.595+0000 7f9e59522640 1 -- 172.21.15.16:0/147050510 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9e4006b500 msgr2=0x7f9e4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.595+0000 7f9e59522640 1 --2- 172.21.15.16:0/147050510 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9e4006b500 0x7f9e4006d9c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9e440093d0 tx=0x7f9e44008040 comp rx=0 tx=0).stop 2024-04-25T10:12:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.595+0000 7f9e59522640 1 -- 172.21.15.16:0/147050510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e541f7660 msgr2=0x7f9e541539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.595+0000 7f9e59522640 1 --2- 172.21.15.16:0/147050510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e541f7660 0x7f9e541539b0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800c380 tx=0x7f9e540d2ca0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.595+0000 7f9e59522640 1 -- 172.21.15.16:0/147050510 shutdown_connections 2024-04-25T10:12:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.595+0000 7f9e59522640 1 --2- 172.21.15.16:0/147050510 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9e4006b500 0x7f9e4006d9c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.595+0000 7f9e59522640 1 --2- 172.21.15.16:0/147050510 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e54153ef0 0x7f9e54155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.595+0000 7f9e59522640 1 --2- 172.21.15.16:0/147050510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e541f7660 0x7f9e541539b0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.595+0000 7f9e59522640 1 --2- 172.21.15.16:0/147050510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e3c0043a0 0x7f9e5415eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.595+0000 7f9e59522640 1 -- 172.21.15.16:0/147050510 >> 172.21.15.16:0/147050510 conn(0x7f9e54072900 msgr2=0x7f9e540b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.595+0000 7f9e59522640 1 -- 172.21.15.16:0/147050510 shutdown_connections 2024-04-25T10:12:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.595+0000 7f9e59522640 1 -- 172.21.15.16:0/147050510 wait complete. 2024-04-25T10:12:46.611 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:46.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.710+0000 7f4342d7e640 1 Processor -- start 2024-04-25T10:12:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.710+0000 7f4342d7e640 1 -- start start 2024-04-25T10:12:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.711+0000 7f4342d7e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f433c0b98b0 0x7f433c15eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.711+0000 7f4342d7e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f433c0b5580 0x7f433c0b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.711+0000 7f4342d7e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f433c15f080 con 0x7f433c0b24c0 2024-04-25T10:12:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.711+0000 7f4342d7e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f433c15bb80 con 0x7f433c0b98b0 2024-04-25T10:12:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.711+0000 7f4342d7e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433c06c4b0 con 0x7f433c0b5580 2024-04-25T10:12:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.711+0000 7f434131b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f433c0b24c0 0x7f433c0b2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45170/0 (socket says 172.21.15.16:45170) 2024-04-25T10:12:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.711+0000 7f434131b640 1 -- 172.21.15.16:0/3330078949 learned_addr learned my addr 172.21.15.16:0/3330078949 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433b7fe640 1 -- 172.21.15.16:0/3330078949 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 619445493 0 0) 0x7f433c15f080 con 0x7f433c0b24c0 2024-04-25T10:12:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f4340b1a640 1 --2- 172.21.15.16:0/3330078949 >> v2:172.21.15.186:3300/0 conn(0x7f433c0b98b0 0x7f433c15eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433b7fe640 1 -- 172.21.15.16:0/3330078949 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4328003520 con 0x7f433c0b24c0 2024-04-25T10:12:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433bfff640 1 --2- 172.21.15.16:0/3330078949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f433c0b5580 0x7f433c0b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433b7fe640 1 -- 172.21.15.16:0/3330078949 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2925990613 0 0) 0x7f4328003520 con 0x7f433c0b24c0 2024-04-25T10:12:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433b7fe640 1 -- 172.21.15.16:0/3330078949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f433c0b5580 msgr2=0x7f433c0b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433b7fe640 1 --2- 172.21.15.16:0/3330078949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f433c0b5580 0x7f433c0b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433b7fe640 1 -- 172.21.15.16:0/3330078949 >> v2:172.21.15.186:3300/0 conn(0x7f433c0b98b0 msgr2=0x7f433c15eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433b7fe640 1 --2- 172.21.15.16:0/3330078949 >> v2:172.21.15.186:3300/0 conn(0x7f433c0b98b0 0x7f433c15eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433b7fe640 1 -- 172.21.15.16:0/3330078949 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c06c790 con 0x7f433c0b24c0 2024-04-25T10:12:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433b7fe640 1 -- 172.21.15.16:0/3330078949 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f4324001670 con 0x7f433c0b24c0 2024-04-25T10:12:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433b7fe640 1 -- 172.21.15.16:0/3330078949 >> v1:172.21.15.16:6789/0 conn(0x7f433c0b24c0 legacy=0x7f433c0b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433b7fe640 1 --2- 172.21.15.16:0/3330078949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43280039b0 0x7f4328003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433b7fe640 1 --2- 172.21.15.16:0/3330078949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4328004800 0x7f4328004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433b7fe640 1 --2- 172.21.15.16:0/3330078949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f433c0b5580 0x7f43280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433b7fe640 1 -- 172.21.15.16:0/3330078949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c15bb80 con 0x7f4328004800 2024-04-25T10:12:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433b7fe640 1 -- 172.21.15.16:0/3330078949 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c06c4b0 con 0x7f43280039b0 2024-04-25T10:12:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433b7fe640 1 -- 172.21.15.16:0/3330078949 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432800ba70 con 0x7f433c0b5580 2024-04-25T10:12:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433bfff640 1 --2- 172.21.15.16:0/3330078949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4328004800 0x7f4328004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f4340b1a640 1 --2- 172.21.15.16:0/3330078949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43280039b0 0x7f4328003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f434131b640 1 --2- 172.21.15.16:0/3330078949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f433c0b5580 0x7f43280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433bfff640 1 -- 172.21.15.16:0/3330078949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f433c0b5580 msgr2=0x7f43280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433bfff640 1 --2- 172.21.15.16:0/3330078949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f433c0b5580 0x7f43280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433bfff640 1 -- 172.21.15.16:0/3330078949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43280039b0 msgr2=0x7f4328003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433bfff640 1 --2- 172.21.15.16:0/3330078949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43280039b0 0x7f4328003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f433bfff640 1 -- 172.21.15.16:0/3330078949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4328003520 con 0x7f4328004800 2024-04-25T10:12:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.712+0000 7f4340b1a640 1 --2- 172.21.15.16:0/3330078949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43280039b0 0x7f4328003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.713+0000 7f433bfff640 1 --2- 172.21.15.16:0/3330078949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4328004800 0x7f4328004c80 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f432c009ac0 tx=0x7f4328005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f04cea26d24db1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.713+0000 7f433b7fe640 1 -- 172.21.15.16:0/3330078949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c036070 con 0x7f4328004800 2024-04-25T10:12:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.713+0000 7f433b7fe640 1 -- 172.21.15.16:0/3330078949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432c031400 con 0x7f4328004800 2024-04-25T10:12:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.713+0000 7f433b7fe640 1 -- 172.21.15.16:0/3330078949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c009690 con 0x7f4328004800 2024-04-25T10:12:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.714+0000 7f4342d7e640 1 -- 172.21.15.16:0/3330078949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4328004800 msgr2=0x7f4328004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.714+0000 7f4342d7e640 1 --2- 172.21.15.16:0/3330078949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4328004800 0x7f4328004c80 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f432c009ac0 tx=0x7f4328005830 comp rx=0 tx=0).stop 2024-04-25T10:12:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.714+0000 7f434131b640 1 -- 172.21.15.16:0/3330078949 reap_dead start 2024-04-25T10:12:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.714+0000 7f4342d7e640 1 -- 172.21.15.16:0/3330078949 shutdown_connections 2024-04-25T10:12:46.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.714+0000 7f4342d7e640 1 -- 172.21.15.16:0/3330078949 >> 172.21.15.16:0/3330078949 conn(0x7f433c0d0580 msgr2=0x7f433c0d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.714+0000 7f4342d7e640 1 -- 172.21.15.16:0/3330078949 shutdown_connections 2024-04-25T10:12:46.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.714+0000 7f4342d7e640 1 -- 172.21.15.16:0/3330078949 wait complete. 2024-04-25T10:12:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.715+0000 7f4342d7e640 1 Processor -- start 2024-04-25T10:12:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.715+0000 7f4342d7e640 1 -- start start 2024-04-25T10:12:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.716+0000 7f4342d7e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4328004800 0x7f433c0ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.716+0000 7f4342d7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f433c0cbdf0 0x7f433c0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.716+0000 7f4342d7e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f433c0c9530 0x7f433c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.716+0000 7f4342d7e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c06c140 con 0x7f433c0cbdf0 2024-04-25T10:12:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.716+0000 7f4342d7e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c06cd50 con 0x7f4328004800 2024-04-25T10:12:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.716+0000 7f4342d7e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433c0cec40 con 0x7f433c0c9530 2024-04-25T10:12:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.716+0000 7f433bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f433c0cbdf0 0x7f433c0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.716+0000 7f433bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f433c0cbdf0 0x7f433c0caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60932/0 (socket says 172.21.15.16:60932) 2024-04-25T10:12:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.716+0000 7f433bfff640 1 -- 172.21.15.16:0/1894905380 learned_addr learned my addr 172.21.15.16:0/1894905380 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.716+0000 7f4340b1a640 1 --2- 172.21.15.16:0/1894905380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4328004800 0x7f433c0ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.716+0000 7f434131b640 1 --2- 172.21.15.16:0/1894905380 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f433c0c9530 0x7f433c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.716+0000 7f433bfff640 1 -- 172.21.15.16:0/1894905380 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f433c0c9530 msgr2=0x7f433c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.716+0000 7f433bfff640 1 --2- 172.21.15.16:0/1894905380 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f433c0c9530 0x7f433c156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.716+0000 7f433bfff640 1 -- 172.21.15.16:0/1894905380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4328004800 msgr2=0x7f433c0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.716+0000 7f433bfff640 1 --2- 172.21.15.16:0/1894905380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4328004800 0x7f433c0ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.716+0000 7f433bfff640 1 -- 172.21.15.16:0/1894905380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c0c8770 con 0x7f433c0cbdf0 2024-04-25T10:12:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.716+0000 7f4340b1a640 1 --2- 172.21.15.16:0/1894905380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4328004800 0x7f433c0ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.716+0000 7f433bfff640 1 --2- 172.21.15.16:0/1894905380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f433c0cbdf0 0x7f433c0caea0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f432c009ac0 tx=0x7f432c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.717+0000 7f43397fa640 1 -- 172.21.15.16:0/1894905380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c040020 con 0x7f433c0cbdf0 2024-04-25T10:12:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.717+0000 7f43397fa640 1 -- 172.21.15.16:0/1894905380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432c003b40 con 0x7f433c0cbdf0 2024-04-25T10:12:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.717+0000 7f4342d7e640 1 -- 172.21.15.16:0/1894905380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433c0c8a50 con 0x7f433c0cbdf0 2024-04-25T10:12:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.717+0000 7f43397fa640 1 -- 172.21.15.16:0/1894905380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c036040 con 0x7f433c0cbdf0 2024-04-25T10:12:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.717+0000 7f4342d7e640 1 -- 172.21.15.16:0/1894905380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f433c0c8420 con 0x7f433c0cbdf0 2024-04-25T10:12:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.718+0000 7f4342d7e640 1 -- 172.21.15.16:0/1894905380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f433c0750b0 con 0x7f433c0cbdf0 2024-04-25T10:12:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.718+0000 7f43397fa640 1 -- 172.21.15.16:0/1894905380 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f432c043020 con 0x7f433c0cbdf0 2024-04-25T10:12:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.718+0000 7f43397fa640 1 --2- 172.21.15.16:0/1894905380 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f431c06b4d0 0x7f431c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.719+0000 7f43397fa640 1 -- 172.21.15.16:0/1894905380 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f432c0a9f90 con 0x7f433c0cbdf0 2024-04-25T10:12:46.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.719+0000 7f4340b1a640 1 --2- 172.21.15.16:0/1894905380 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f431c06b4d0 0x7f431c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.719+0000 7f4340b1a640 1 --2- 172.21.15.16:0/1894905380 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f431c06b4d0 0x7f431c06d990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4334007560 tx=0x7f433400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.723+0000 7f43397fa640 1 -- 172.21.15.16:0/1894905380 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f432c04a070 con 0x7f433c0cbdf0 2024-04-25T10:12:46.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.901+0000 7f4342d7e640 1 -- 172.21.15.16:0/1894905380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f433c0b46a0 con 0x7f433c0cbdf0 2024-04-25T10:12:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.901+0000 7f43397fa640 1 -- 172.21.15.16:0/1894905380 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f432c0044e0 con 0x7f433c0cbdf0 2024-04-25T10:12:46.906 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:12:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.903+0000 7f4342d7e640 1 -- 172.21.15.16:0/1894905380 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f431c06b4d0 msgr2=0x7f431c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.903+0000 7f4342d7e640 1 --2- 172.21.15.16:0/1894905380 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f431c06b4d0 0x7f431c06d990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4334007560 tx=0x7f433400b040 comp rx=0 tx=0).stop 2024-04-25T10:12:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.903+0000 7f4342d7e640 1 -- 172.21.15.16:0/1894905380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f433c0cbdf0 msgr2=0x7f433c0caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.903+0000 7f4342d7e640 1 --2- 172.21.15.16:0/1894905380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f433c0cbdf0 0x7f433c0caea0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f432c009ac0 tx=0x7f432c004730 comp rx=0 tx=0).stop 2024-04-25T10:12:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.903+0000 7f4342d7e640 1 -- 172.21.15.16:0/1894905380 shutdown_connections 2024-04-25T10:12:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.903+0000 7f4342d7e640 1 --2- 172.21.15.16:0/1894905380 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f431c06b4d0 0x7f431c06d990 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.903+0000 7f4342d7e640 1 --2- 172.21.15.16:0/1894905380 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f433c0c9530 0x7f433c156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.903+0000 7f4342d7e640 1 --2- 172.21.15.16:0/1894905380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f433c0cbdf0 0x7f433c0caea0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.903+0000 7f4342d7e640 1 --2- 172.21.15.16:0/1894905380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4328004800 0x7f433c0ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.903+0000 7f4342d7e640 1 -- 172.21.15.16:0/1894905380 >> 172.21.15.16:0/1894905380 conn(0x7f433c0d0580 msgr2=0x7f433c0b4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.903+0000 7f4342d7e640 1 -- 172.21.15.16:0/1894905380 shutdown_connections 2024-04-25T10:12:46.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:46.903+0000 7f4342d7e640 1 -- 172.21.15.16:0/1894905380 wait complete. 2024-04-25T10:12:46.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:12:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.015+0000 7f9498a1f640 1 Processor -- start 2024-04-25T10:12:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.015+0000 7f9498a1f640 1 -- start start 2024-04-25T10:12:47.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.015+0000 7f9498a1f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f94940d35b0 0x7f94940b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.015+0000 7f9498a1f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f94940b5990 0x7f94940b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.015+0000 7f9498a1f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949415e0c0 con 0x7f949415dce0 2024-04-25T10:12:47.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.015+0000 7f9498a1f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94940d2510 con 0x7f94940d35b0 2024-04-25T10:12:47.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.015+0000 7f9498a1f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94940d27f0 con 0x7f94940b5990 2024-04-25T10:12:47.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.015+0000 7f949259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f949415dce0 0x7f94940d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45180/0 (socket says 172.21.15.16:45180) 2024-04-25T10:12:47.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f949259c640 1 -- 172.21.15.16:0/1930569926 learned_addr learned my addr 172.21.15.16:0/1930569926 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:47.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f9492d9d640 1 --2- 172.21.15.16:0/1930569926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f94940b5990 0x7f94940b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f948bfff640 1 --2- 172.21.15.16:0/1930569926 >> v2:172.21.15.186:3300/0 conn(0x7f94940d35b0 0x7f94940b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f9491d9b640 1 -- 172.21.15.16:0/1930569926 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076914327 0 0) 0x7f949415e0c0 con 0x7f949415dce0 2024-04-25T10:12:47.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f9491d9b640 1 -- 172.21.15.16:0/1930569926 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9480003520 con 0x7f949415dce0 2024-04-25T10:12:47.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f9491d9b640 1 -- 172.21.15.16:0/1930569926 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3142169551 0 0) 0x7f9480003520 con 0x7f949415dce0 2024-04-25T10:12:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f9491d9b640 1 -- 172.21.15.16:0/1930569926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f94940b5990 msgr2=0x7f94940b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f9491d9b640 1 --2- 172.21.15.16:0/1930569926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f94940b5990 0x7f94940b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f9491d9b640 1 -- 172.21.15.16:0/1930569926 >> v2:172.21.15.186:3300/0 conn(0x7f94940d35b0 msgr2=0x7f94940b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f9491d9b640 1 --2- 172.21.15.16:0/1930569926 >> v2:172.21.15.186:3300/0 conn(0x7f94940d35b0 0x7f94940b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f9491d9b640 1 -- 172.21.15.16:0/1930569926 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94940d2ad0 con 0x7f949415dce0 2024-04-25T10:12:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f9491d9b640 1 -- 172.21.15.16:0/1930569926 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f9484001670 con 0x7f949415dce0 2024-04-25T10:12:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f9491d9b640 1 -- 172.21.15.16:0/1930569926 >> v1:172.21.15.16:6789/0 conn(0x7f949415dce0 legacy=0x7f94940d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f9491d9b640 1 --2- 172.21.15.16:0/1930569926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94800039b0 0x7f9480003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f9491d9b640 1 --2- 172.21.15.16:0/1930569926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f94940b5990 0x7f9480004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f9491d9b640 1 --2- 172.21.15.16:0/1930569926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9480004d40 0x7f94800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f9491d9b640 1 -- 172.21.15.16:0/1930569926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94940d2510 con 0x7f9480004d40 2024-04-25T10:12:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f9491d9b640 1 -- 172.21.15.16:0/1930569926 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94940d27f0 con 0x7f94800039b0 2024-04-25T10:12:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f9491d9b640 1 -- 172.21.15.16:0/1930569926 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948000b9c0 con 0x7f94940b5990 2024-04-25T10:12:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f9492d9d640 1 --2- 172.21.15.16:0/1930569926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9480004d40 0x7f94800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f948bfff640 1 --2- 172.21.15.16:0/1930569926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f94940b5990 0x7f9480004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.016+0000 7f949259c640 1 --2- 172.21.15.16:0/1930569926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94800039b0 0x7f9480003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.017+0000 7f9492d9d640 1 -- 172.21.15.16:0/1930569926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f94940b5990 msgr2=0x7f9480004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.017+0000 7f9492d9d640 1 --2- 172.21.15.16:0/1930569926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f94940b5990 0x7f9480004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.017+0000 7f9492d9d640 1 -- 172.21.15.16:0/1930569926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94800039b0 msgr2=0x7f9480003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.017+0000 7f9492d9d640 1 --2- 172.21.15.16:0/1930569926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94800039b0 0x7f9480003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.017+0000 7f9492d9d640 1 -- 172.21.15.16:0/1930569926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9480003520 con 0x7f9480004d40 2024-04-25T10:12:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.017+0000 7f9492d9d640 1 --2- 172.21.15.16:0/1930569926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9480004d40 0x7f94800051c0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f948c00e170 tx=0x7f9480005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c50564c8b63df5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.017+0000 7f9491d9b640 1 -- 172.21.15.16:0/1930569926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c034020 con 0x7f9480004d40 2024-04-25T10:12:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.017+0000 7f9491d9b640 1 -- 172.21.15.16:0/1930569926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c0305f0 con 0x7f9480004d40 2024-04-25T10:12:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.017+0000 7f9491d9b640 1 -- 172.21.15.16:0/1930569926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c009c10 con 0x7f9480004d40 2024-04-25T10:12:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.018+0000 7f9498a1f640 1 -- 172.21.15.16:0/1930569926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9480004d40 msgr2=0x7f94800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.018+0000 7f9498a1f640 1 --2- 172.21.15.16:0/1930569926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9480004d40 0x7f94800051c0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f948c00e170 tx=0x7f9480005830 comp rx=0 tx=0).stop 2024-04-25T10:12:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.018+0000 7f9492d9d640 1 -- 172.21.15.16:0/1930569926 reap_dead start 2024-04-25T10:12:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.018+0000 7f9498a1f640 1 -- 172.21.15.16:0/1930569926 shutdown_connections 2024-04-25T10:12:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.018+0000 7f9498a1f640 1 -- 172.21.15.16:0/1930569926 >> 172.21.15.16:0/1930569926 conn(0x7f9494072900 msgr2=0x7f94940d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.018+0000 7f9498a1f640 1 -- 172.21.15.16:0/1930569926 shutdown_connections 2024-04-25T10:12:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.018+0000 7f9498a1f640 1 -- 172.21.15.16:0/1930569926 wait complete. 2024-04-25T10:12:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.019+0000 7f9498a1f640 1 Processor -- start 2024-04-25T10:12:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.019+0000 7f9498a1f640 1 -- start start 2024-04-25T10:12:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.020+0000 7f9498a1f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9480004d40 0x7f9494156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.020+0000 7f9498a1f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9494152e40 0x7f9494154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.020+0000 7f9498a1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494154840 0x7f949415fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.020+0000 7f9498a1f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94940d2160 con 0x7f9494154840 2024-04-25T10:12:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.020+0000 7f9498a1f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94940d3090 con 0x7f9494152e40 2024-04-25T10:12:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.020+0000 7f9498a1f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949415f230 con 0x7f9480004d40 2024-04-25T10:12:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.020+0000 7f9492d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494154840 0x7f949415fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.020+0000 7f9492d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494154840 0x7f949415fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60948/0 (socket says 172.21.15.16:60948) 2024-04-25T10:12:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.020+0000 7f9492d9d640 1 -- 172.21.15.16:0/905463985 learned_addr learned my addr 172.21.15.16:0/905463985 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.021+0000 7f949259c640 1 --2- 172.21.15.16:0/905463985 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9480004d40 0x7f9494156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.021+0000 7f948bfff640 1 --2- 172.21.15.16:0/905463985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9494152e40 0x7f9494154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.021+0000 7f9492d9d640 1 -- 172.21.15.16:0/905463985 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9480004d40 msgr2=0x7f9494156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.021+0000 7f9492d9d640 1 --2- 172.21.15.16:0/905463985 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9480004d40 0x7f9494156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.021+0000 7f9492d9d640 1 -- 172.21.15.16:0/905463985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9494152e40 msgr2=0x7f9494154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.021+0000 7f9492d9d640 1 --2- 172.21.15.16:0/905463985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9494152e40 0x7f9494154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.021+0000 7f9492d9d640 1 -- 172.21.15.16:0/905463985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94940cd430 con 0x7f9494154840 2024-04-25T10:12:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.021+0000 7f949259c640 1 --2- 172.21.15.16:0/905463985 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9480004d40 0x7f9494156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.021+0000 7f9492d9d640 1 --2- 172.21.15.16:0/905463985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494154840 0x7f949415fca0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f948c00e170 tx=0x7f948c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.021+0000 7f948affd640 1 -- 172.21.15.16:0/905463985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c034040 con 0x7f9494154840 2024-04-25T10:12:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.021+0000 7f948affd640 1 -- 172.21.15.16:0/905463985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c003990 con 0x7f9494154840 2024-04-25T10:12:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.021+0000 7f9498a1f640 1 -- 172.21.15.16:0/905463985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949415f4a0 con 0x7f9494154840 2024-04-25T10:12:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.021+0000 7f948affd640 1 -- 172.21.15.16:0/905463985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c009c10 con 0x7f9494154840 2024-04-25T10:12:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.021+0000 7f9498a1f640 1 -- 172.21.15.16:0/905463985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9494161190 con 0x7f9494154840 2024-04-25T10:12:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.022+0000 7f9498a1f640 1 -- 172.21.15.16:0/905463985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94940749b0 con 0x7f9494154840 2024-04-25T10:12:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.022+0000 7f948affd640 1 -- 172.21.15.16:0/905463985 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f948c037020 con 0x7f9494154840 2024-04-25T10:12:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.023+0000 7f948affd640 1 --2- 172.21.15.16:0/905463985 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f947406b4d0 0x7f947406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.023+0000 7f948affd640 1 -- 172.21.15.16:0/905463985 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f948c0aaba0 con 0x7f9494154840 2024-04-25T10:12:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.023+0000 7f949259c640 1 --2- 172.21.15.16:0/905463985 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f947406b4d0 0x7f947406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.024+0000 7f949259c640 1 --2- 172.21.15.16:0/905463985 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f947406b4d0 0x7f947406d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9484007e10 tx=0x7f9484009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.027+0000 7f948affd640 1 -- 172.21.15.16:0/905463985 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f948c049060 con 0x7f9494154840 2024-04-25T10:12:47.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.209+0000 7f9498a1f640 1 -- 172.21.15.16:0/905463985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f949415d500 con 0x7f9494154840 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.209+0000 7f948affd640 1 -- 172.21.15.16:0/905463985 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v96) v1 ==== 84+0+65 (secure 0 0 0) 0x7f948c079ce0 con 0x7f9494154840 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.211+0000 7f9498a1f640 1 -- 172.21.15.16:0/905463985 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f947406b4d0 msgr2=0x7f947406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.211+0000 7f9498a1f640 1 --2- 172.21.15.16:0/905463985 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f947406b4d0 0x7f947406d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9484007e10 tx=0x7f9484009040 comp rx=0 tx=0).stop 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.211+0000 7f9498a1f640 1 -- 172.21.15.16:0/905463985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494154840 msgr2=0x7f949415fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.211+0000 7f9498a1f640 1 --2- 172.21.15.16:0/905463985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494154840 0x7f949415fca0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f948c00e170 tx=0x7f948c009750 comp rx=0 tx=0).stop 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.211+0000 7f9498a1f640 1 -- 172.21.15.16:0/905463985 shutdown_connections 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.211+0000 7f9498a1f640 1 --2- 172.21.15.16:0/905463985 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f947406b4d0 0x7f947406d990 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.211+0000 7f9498a1f640 1 --2- 172.21.15.16:0/905463985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494154840 0x7f949415fca0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.211+0000 7f9498a1f640 1 --2- 172.21.15.16:0/905463985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9494152e40 0x7f9494154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.211+0000 7f9498a1f640 1 --2- 172.21.15.16:0/905463985 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9480004d40 0x7f9494156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.211+0000 7f9498a1f640 1 -- 172.21.15.16:0/905463985 >> 172.21.15.16:0/905463985 conn(0x7f9494072900 msgr2=0x7f94940d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.211+0000 7f9498a1f640 1 -- 172.21.15.16:0/905463985 shutdown_connections 2024-04-25T10:12:47.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.211+0000 7f9498a1f640 1 -- 172.21.15.16:0/905463985 wait complete. 2024-04-25T10:12:47.227 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:12:47.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:12:47.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.327+0000 7fb66b88f640 1 Processor -- start 2024-04-25T10:12:47.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.327+0000 7fb66b88f640 1 -- start start 2024-04-25T10:12:47.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.328+0000 7fb66b88f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb6641610b0 0x7fb6640b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.328+0000 7fb66b88f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6641618d0 0x7fb6640ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.328+0000 7fb66b88f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb664162040 con 0x7fb6640ced70 2024-04-25T10:12:47.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.328+0000 7fb66b88f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6640d1ea0 con 0x7fb6641610b0 2024-04-25T10:12:47.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.328+0000 7fb66b88f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb664162320 con 0x7fb6641618d0 2024-04-25T10:12:47.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.328+0000 7fb66962b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb6641610b0 0x7fb6640b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.328+0000 7fb668e2a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6641618d0 0x7fb6640ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.328+0000 7fb669e2c640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fb6640ced70 0x7fb66406c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:45478/0 (socket says 172.21.15.16:45478) 2024-04-25T10:12:47.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.328+0000 7fb669e2c640 1 -- 172.21.15.16:0/2930842381 learned_addr learned my addr 172.21.15.16:0/2930842381 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:47.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.328+0000 7fb668e2a640 1 -- 172.21.15.16:0/2930842381 >> v2:172.21.15.186:3300/0 conn(0x7fb6641610b0 msgr2=0x7fb6640b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.328+0000 7fb668e2a640 1 --2- 172.21.15.16:0/2930842381 >> v2:172.21.15.186:3300/0 conn(0x7fb6641610b0 0x7fb6640b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.328+0000 7fb668e2a640 1 -- 172.21.15.16:0/2930842381 >> v1:172.21.15.186:6789/0 conn(0x7fb6640ced70 legacy=0x7fb66406c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.328+0000 7fb668e2a640 1 -- 172.21.15.16:0/2930842381 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb664162600 con 0x7fb6641618d0 2024-04-25T10:12:47.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.329+0000 7fb668e2a640 1 --2- 172.21.15.16:0/2930842381 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6641618d0 0x7fb6640ce830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb6540097a0 tx=0x7fb65402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71b988f2d0b7ac24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.329+0000 7fb65bfff640 1 -- 172.21.15.16:0/2930842381 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb654037070 con 0x7fb6641618d0 2024-04-25T10:12:47.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.329+0000 7fb65bfff640 1 -- 172.21.15.16:0/2930842381 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65402ae50 con 0x7fb6641618d0 2024-04-25T10:12:47.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.329+0000 7fb65bfff640 1 -- 172.21.15.16:0/2930842381 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb654032660 con 0x7fb6641618d0 2024-04-25T10:12:47.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.330+0000 7fb66b88f640 1 -- 172.21.15.16:0/2930842381 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6641618d0 msgr2=0x7fb6640ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.330+0000 7fb66b88f640 1 --2- 172.21.15.16:0/2930842381 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6641618d0 0x7fb6640ce830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb6540097a0 tx=0x7fb65402a100 comp rx=0 tx=0).stop 2024-04-25T10:12:47.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.330+0000 7fb66b88f640 1 -- 172.21.15.16:0/2930842381 shutdown_connections 2024-04-25T10:12:47.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.330+0000 7fb66b88f640 1 --2- 172.21.15.16:0/2930842381 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6641618d0 0x7fb6640ce830 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.330+0000 7fb66b88f640 1 --2- 172.21.15.16:0/2930842381 >> v2:172.21.15.186:3300/0 conn(0x7fb6641610b0 0x7fb6640b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.330+0000 7fb66b88f640 1 -- 172.21.15.16:0/2930842381 >> 172.21.15.16:0/2930842381 conn(0x7fb6640b5570 msgr2=0x7fb6640b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.330+0000 7fb66b88f640 1 -- 172.21.15.16:0/2930842381 shutdown_connections 2024-04-25T10:12:47.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.330+0000 7fb66b88f640 1 -- 172.21.15.16:0/2930842381 wait complete. 2024-04-25T10:12:47.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.331+0000 7fb66b88f640 1 Processor -- start 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.331+0000 7fb66b88f640 1 -- start start 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.332+0000 7fb66b88f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6640ced70 0x7fb664157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.332+0000 7fb66b88f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6641610b0 0x7fb66415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.332+0000 7fb66b88f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6641618d0 0x7fb66415a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.332+0000 7fb66b88f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb664161d10 con 0x7fb6640ced70 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.332+0000 7fb66b88f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6640b6c00 con 0x7fb6641618d0 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.332+0000 7fb66b88f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6640d3620 con 0x7fb6641610b0 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.332+0000 7fb66962b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6640ced70 0x7fb664157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.332+0000 7fb66962b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6640ced70 0x7fb664157e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60964/0 (socket says 172.21.15.16:60964) 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.332+0000 7fb66962b640 1 -- 172.21.15.16:0/2715028150 learned_addr learned my addr 172.21.15.16:0/2715028150 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.332+0000 7fb669e2c640 1 --2- 172.21.15.16:0/2715028150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6641618d0 0x7fb66415a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.332+0000 7fb668e2a640 1 --2- 172.21.15.16:0/2715028150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6641610b0 0x7fb66415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.332+0000 7fb66962b640 1 -- 172.21.15.16:0/2715028150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6641610b0 msgr2=0x7fb66415a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.332+0000 7fb66962b640 1 --2- 172.21.15.16:0/2715028150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6641610b0 0x7fb66415a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.332+0000 7fb66962b640 1 -- 172.21.15.16:0/2715028150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6641618d0 msgr2=0x7fb66415a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.332+0000 7fb66962b640 1 --2- 172.21.15.16:0/2715028150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6641618d0 0x7fb66415a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.332+0000 7fb66962b640 1 -- 172.21.15.16:0/2715028150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66415cb20 con 0x7fb6640ced70 2024-04-25T10:12:47.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.333+0000 7fb66962b640 1 --2- 172.21.15.16:0/2715028150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6640ced70 0x7fb664157e10 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fb66000e9b0 tx=0x7fb66000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.333+0000 7fb659ffb640 1 -- 172.21.15.16:0/2715028150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660017070 con 0x7fb6640ced70 2024-04-25T10:12:47.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.333+0000 7fb659ffb640 1 -- 172.21.15.16:0/2715028150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66000c9a0 con 0x7fb6640ced70 2024-04-25T10:12:47.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.333+0000 7fb659ffb640 1 -- 172.21.15.16:0/2715028150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66000cd90 con 0x7fb6640ced70 2024-04-25T10:12:47.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.333+0000 7fb66b88f640 1 -- 172.21.15.16:0/2715028150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66415cdd0 con 0x7fb6640ced70 2024-04-25T10:12:47.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.333+0000 7fb66b88f640 1 -- 172.21.15.16:0/2715028150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66415d390 con 0x7fb6640ced70 2024-04-25T10:12:47.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.334+0000 7fb66b88f640 1 -- 172.21.15.16:0/2715028150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6640750b0 con 0x7fb6640ced70 2024-04-25T10:12:47.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.334+0000 7fb659ffb640 1 -- 172.21.15.16:0/2715028150 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb66001a070 con 0x7fb6640ced70 2024-04-25T10:12:47.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.335+0000 7fb659ffb640 1 --2- 172.21.15.16:0/2715028150 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb64806b4d0 0x7fb64806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.335+0000 7fb659ffb640 1 -- 172.21.15.16:0/2715028150 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb66008a150 con 0x7fb6640ced70 2024-04-25T10:12:47.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.335+0000 7fb668e2a640 1 --2- 172.21.15.16:0/2715028150 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb64806b4d0 0x7fb64806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.336+0000 7fb668e2a640 1 --2- 172.21.15.16:0/2715028150 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb64806b4d0 0x7fb64806d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb654034000 tx=0x7fb65402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.339+0000 7fb659ffb640 1 -- 172.21.15.16:0/2715028150 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb660058d60 con 0x7fb6640ced70 2024-04-25T10:12:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.519+0000 7fb66b88f640 1 -- 172.21.15.16:0/2715028150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb6640d1770 con 0x7fb6640ced70 2024-04-25T10:12:47.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.589+0000 7fb659ffb640 1 -- 172.21.15.16:0/2715028150 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v97) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb66005ce00 con 0x7fb6640ced70 2024-04-25T10:12:47.593 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-04-25T10:12:47.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.592+0000 7fb66b88f640 1 -- 172.21.15.16:0/2715028150 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb64806b4d0 msgr2=0x7fb64806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.592+0000 7fb66b88f640 1 --2- 172.21.15.16:0/2715028150 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb64806b4d0 0x7fb64806d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb654034000 tx=0x7fb65402e000 comp rx=0 tx=0).stop 2024-04-25T10:12:47.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.592+0000 7fb66b88f640 1 -- 172.21.15.16:0/2715028150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6640ced70 msgr2=0x7fb664157e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.592+0000 7fb66b88f640 1 --2- 172.21.15.16:0/2715028150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6640ced70 0x7fb664157e10 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fb66000e9b0 tx=0x7fb66000ee70 comp rx=0 tx=0).stop 2024-04-25T10:12:47.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.592+0000 7fb66b88f640 1 -- 172.21.15.16:0/2715028150 shutdown_connections 2024-04-25T10:12:47.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.592+0000 7fb66b88f640 1 --2- 172.21.15.16:0/2715028150 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb64806b4d0 0x7fb64806d990 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.592+0000 7fb66b88f640 1 --2- 172.21.15.16:0/2715028150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6641618d0 0x7fb66415a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.592+0000 7fb66b88f640 1 --2- 172.21.15.16:0/2715028150 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6641610b0 0x7fb66415a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.592+0000 7fb66b88f640 1 --2- 172.21.15.16:0/2715028150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6640ced70 0x7fb664157e10 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.592+0000 7fb66b88f640 1 -- 172.21.15.16:0/2715028150 >> 172.21.15.16:0/2715028150 conn(0x7fb6640b5570 msgr2=0x7fb6640b8520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.592+0000 7fb66b88f640 1 -- 172.21.15.16:0/2715028150 shutdown_connections 2024-04-25T10:12:47.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.592+0000 7fb66b88f640 1 -- 172.21.15.16:0/2715028150 wait complete. 2024-04-25T10:12:47.610 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:12:47.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.713+0000 7fbdde7d5640 1 Processor -- start 2024-04-25T10:12:47.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.713+0000 7fbdde7d5640 1 -- start start 2024-04-25T10:12:47.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.714+0000 7fbdde7d5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbdd80cc0c0 0x7fbdd80b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.714+0000 7fbdde7d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdd815f2c0 0x7fbdd80cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.714+0000 7fbdde7d5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd80b0060 con 0x7fbdd815ee30 2024-04-25T10:12:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.714+0000 7fbdde7d5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd80b1600 con 0x7fbdd80cc0c0 2024-04-25T10:12:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.714+0000 7fbdde7d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd80b0c30 con 0x7fbdd815f2c0 2024-04-25T10:12:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.714+0000 7fbdd7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbdd80cc0c0 0x7fbdd80b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.714+0000 7fbdd7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbdd80cc0c0 0x7fbdd80b10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60972/0 (socket says 172.21.15.16:60972) 2024-04-25T10:12:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.714+0000 7fbdd7fff640 1 -- 172.21.15.16:0/3644112995 learned_addr learned my addr 172.21.15.16:0/3644112995 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.715+0000 7fbdd7fff640 1 -- 172.21.15.16:0/3644112995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdd815f2c0 msgr2=0x7fbdd80cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.715+0000 7fbddcd72640 1 --2- 172.21.15.16:0/3644112995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdd815f2c0 0x7fbdd80cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.715+0000 7fbdd7fff640 1 --2- 172.21.15.16:0/3644112995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdd815f2c0 0x7fbdd80cae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.715+0000 7fbdd7fff640 1 -- 172.21.15.16:0/3644112995 >> v1:172.21.15.186:6789/0 conn(0x7fbdd815ee30 legacy=0x7fbdd80cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.715+0000 7fbdd7fff640 1 -- 172.21.15.16:0/3644112995 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd80b0300 con 0x7fbdd80cc0c0 2024-04-25T10:12:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.715+0000 7fbdd7fff640 1 --2- 172.21.15.16:0/3644112995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd80cc0c0 0x7fbdd80b10c0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80d0720 tx=0x7fbdc802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f366f10c093eab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.716+0000 7fbdd6ffd640 1 -- 172.21.15.16:0/3644112995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8037070 con 0x7fbdd80cc0c0 2024-04-25T10:12:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.716+0000 7fbdd6ffd640 1 -- 172.21.15.16:0/3644112995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc802ae50 con 0x7fbdd80cc0c0 2024-04-25T10:12:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.716+0000 7fbdd6ffd640 1 -- 172.21.15.16:0/3644112995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8032660 con 0x7fbdd80cc0c0 2024-04-25T10:12:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.716+0000 7fbdde7d5640 1 -- 172.21.15.16:0/3644112995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd80cc0c0 msgr2=0x7fbdd80b10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.716+0000 7fbdde7d5640 1 --2- 172.21.15.16:0/3644112995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd80cc0c0 0x7fbdd80b10c0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80d0720 tx=0x7fbdc802a140 comp rx=0 tx=0).stop 2024-04-25T10:12:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.716+0000 7fbdde7d5640 1 -- 172.21.15.16:0/3644112995 shutdown_connections 2024-04-25T10:12:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.716+0000 7fbdde7d5640 1 --2- 172.21.15.16:0/3644112995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdd815f2c0 0x7fbdd80cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.716+0000 7fbdde7d5640 1 --2- 172.21.15.16:0/3644112995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd80cc0c0 0x7fbdd80b10c0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.716+0000 7fbdde7d5640 1 -- 172.21.15.16:0/3644112995 >> 172.21.15.16:0/3644112995 conn(0x7fbdd8099a40 msgr2=0x7fbdd80b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.716+0000 7fbdde7d5640 1 -- 172.21.15.16:0/3644112995 shutdown_connections 2024-04-25T10:12:47.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.716+0000 7fbdde7d5640 1 -- 172.21.15.16:0/3644112995 wait complete. 2024-04-25T10:12:47.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.717+0000 7fbdde7d5640 1 Processor -- start 2024-04-25T10:12:47.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.718+0000 7fbdde7d5640 1 -- start start 2024-04-25T10:12:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.718+0000 7fbdde7d5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdd80cc0c0 0x7fbdd8155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.718+0000 7fbdde7d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdd815ee30 0x7fbdd80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.718+0000 7fbdde7d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd815f2c0 0x7fbdd80b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.718+0000 7fbdde7d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd80cc4a0 con 0x7fbdd815f2c0 2024-04-25T10:12:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.718+0000 7fbdde7d5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd80b08c0 con 0x7fbdd80cc0c0 2024-04-25T10:12:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.718+0000 7fbdde7d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd80b1a20 con 0x7fbdd815ee30 2024-04-25T10:12:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.718+0000 7fbddcd72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd815f2c0 0x7fbdd80b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.718+0000 7fbddcd72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd815f2c0 0x7fbdd80b53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60976/0 (socket says 172.21.15.16:60976) 2024-04-25T10:12:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.718+0000 7fbddcd72640 1 -- 172.21.15.16:0/2985912727 learned_addr learned my addr 172.21.15.16:0/2985912727 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.719+0000 7fbdd7fff640 1 --2- 172.21.15.16:0/2985912727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdd80cc0c0 0x7fbdd8155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.719+0000 7fbdd77fe640 1 --2- 172.21.15.16:0/2985912727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdd815ee30 0x7fbdd80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.719+0000 7fbddcd72640 1 -- 172.21.15.16:0/2985912727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdd815ee30 msgr2=0x7fbdd80b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.719+0000 7fbddcd72640 1 --2- 172.21.15.16:0/2985912727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdd815ee30 0x7fbdd80b4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.719+0000 7fbddcd72640 1 -- 172.21.15.16:0/2985912727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdd80cc0c0 msgr2=0x7fbdd8155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.719+0000 7fbddcd72640 1 --2- 172.21.15.16:0/2985912727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdd80cc0c0 0x7fbdd8155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.719+0000 7fbddcd72640 1 -- 172.21.15.16:0/2985912727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd80c4360 con 0x7fbdd815f2c0 2024-04-25T10:12:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.719+0000 7fbdd77fe640 1 --2- 172.21.15.16:0/2985912727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdd815ee30 0x7fbdd80b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.719+0000 7fbdd7fff640 1 --2- 172.21.15.16:0/2985912727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdd80cc0c0 0x7fbdd8155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.719+0000 7fbddcd72640 1 --2- 172.21.15.16:0/2985912727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd815f2c0 0x7fbdd80b53e0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000cd20 tx=0x7fbdd000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.719+0000 7fbdd4ff9640 1 -- 172.21.15.16:0/2985912727 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0003b90 con 0x7fbdd815f2c0 2024-04-25T10:12:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.719+0000 7fbdd4ff9640 1 -- 172.21.15.16:0/2985912727 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd0003d30 con 0x7fbdd815f2c0 2024-04-25T10:12:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.719+0000 7fbdde7d5640 1 -- 172.21.15.16:0/2985912727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd80c4640 con 0x7fbdd815f2c0 2024-04-25T10:12:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.719+0000 7fbdd4ff9640 1 -- 172.21.15.16:0/2985912727 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0004240 con 0x7fbdd815f2c0 2024-04-25T10:12:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.719+0000 7fbdde7d5640 1 -- 172.21.15.16:0/2985912727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd80c4c00 con 0x7fbdd815f2c0 2024-04-25T10:12:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.720+0000 7fbdde7d5640 1 -- 172.21.15.16:0/2985912727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd80749b0 con 0x7fbdd815f2c0 2024-04-25T10:12:47.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.721+0000 7fbdd4ff9640 1 -- 172.21.15.16:0/2985912727 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbdd00043e0 con 0x7fbdd815f2c0 2024-04-25T10:12:47.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.721+0000 7fbdd4ff9640 1 --2- 172.21.15.16:0/2985912727 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbdc406b4d0 0x7fbdc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.721+0000 7fbdd4ff9640 1 -- 172.21.15.16:0/2985912727 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fbdd008ab20 con 0x7fbdd815f2c0 2024-04-25T10:12:47.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.721+0000 7fbdd7fff640 1 --2- 172.21.15.16:0/2985912727 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbdc406b4d0 0x7fbdc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.722+0000 7fbdd7fff640 1 --2- 172.21.15.16:0/2985912727 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbdc406b4d0 0x7fbdc406d990 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8034000 tx=0x7fbdc802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.726+0000 7fbdd4ff9640 1 -- 172.21.15.16:0/2985912727 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbdd0059b80 con 0x7fbdd815f2c0 2024-04-25T10:12:47.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:47.907+0000 7fbdde7d5640 1 -- 172.21.15.16:0/2985912727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbdd80cdc50 con 0x7fbdd815f2c0 2024-04-25T10:12:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.591+0000 7fbdd4ff9640 1 -- 172.21.15.16:0/2985912727 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v98) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbdd0059480 con 0x7fbdd815f2c0 2024-04-25T10:12:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-04-25T10:12:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.595+0000 7fbdde7d5640 1 -- 172.21.15.16:0/2985912727 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbdc406b4d0 msgr2=0x7fbdc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.595+0000 7fbdde7d5640 1 --2- 172.21.15.16:0/2985912727 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbdc406b4d0 0x7fbdc406d990 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8034000 tx=0x7fbdc802e000 comp rx=0 tx=0).stop 2024-04-25T10:12:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.595+0000 7fbdde7d5640 1 -- 172.21.15.16:0/2985912727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd815f2c0 msgr2=0x7fbdd80b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.595+0000 7fbdde7d5640 1 --2- 172.21.15.16:0/2985912727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd815f2c0 0x7fbdd80b53e0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000cd20 tx=0x7fbdd000a660 comp rx=0 tx=0).stop 2024-04-25T10:12:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.596+0000 7fbdde7d5640 1 -- 172.21.15.16:0/2985912727 shutdown_connections 2024-04-25T10:12:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.596+0000 7fbdde7d5640 1 --2- 172.21.15.16:0/2985912727 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbdc406b4d0 0x7fbdc406d990 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.596+0000 7fbdde7d5640 1 --2- 172.21.15.16:0/2985912727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd815f2c0 0x7fbdd80b53e0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.596+0000 7fbdde7d5640 1 --2- 172.21.15.16:0/2985912727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdd815ee30 0x7fbdd80b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.596+0000 7fbdde7d5640 1 --2- 172.21.15.16:0/2985912727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdd80cc0c0 0x7fbdd8155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.596+0000 7fbdde7d5640 1 -- 172.21.15.16:0/2985912727 >> 172.21.15.16:0/2985912727 conn(0x7fbdd8099a40 msgr2=0x7fbdd806f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.597+0000 7fbdde7d5640 1 -- 172.21.15.16:0/2985912727 shutdown_connections 2024-04-25T10:12:48.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.597+0000 7fbdde7d5640 1 -- 172.21.15.16:0/2985912727 wait complete. 2024-04-25T10:12:48.624 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:12:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.728+0000 7fd4d5d29640 1 Processor -- start 2024-04-25T10:12:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.728+0000 7fd4d5d29640 1 -- start start 2024-04-25T10:12:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.729+0000 7fd4d5d29640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4d00d3b90 0x7fd4d00d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.729+0000 7fd4d5d29640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4d00d0f60 0x7fd4d00b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.729+0000 7fd4d5d29640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d00d4090 con 0x7fd4d015dca0 2024-04-25T10:12:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.729+0000 7fd4d5d29640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d00b6b20 con 0x7fd4d00d3b90 2024-04-25T10:12:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.729+0000 7fd4d5d29640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d00d1390 con 0x7fd4d00d0f60 2024-04-25T10:12:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.729+0000 7fd4ceffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4d00d3b90 0x7fd4d00d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.729+0000 7fd4cf7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd4d015dca0 0x7fd4d00ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45192/0 (socket says 172.21.15.16:45192) 2024-04-25T10:12:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.729+0000 7fd4cf7fe640 1 -- 172.21.15.16:0/898554309 learned_addr learned my addr 172.21.15.16:0/898554309 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.730+0000 7fd4ceffd640 1 -- 172.21.15.16:0/898554309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4d00d0f60 msgr2=0x7fd4d00b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.730+0000 7fd4ceffd640 1 --2- 172.21.15.16:0/898554309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4d00d0f60 0x7fd4d00b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.730+0000 7fd4ceffd640 1 -- 172.21.15.16:0/898554309 >> v1:172.21.15.16:6789/0 conn(0x7fd4d015dca0 legacy=0x7fd4d00ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.730+0000 7fd4ceffd640 1 -- 172.21.15.16:0/898554309 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d00d4370 con 0x7fd4d00d3b90 2024-04-25T10:12:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.730+0000 7fd4ce7fc640 1 -- 172.21.15.16:0/898554309 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2089221496 0 0) 0x7fd4d00d4090 con 0x7fd4d015dca0 2024-04-25T10:12:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.730+0000 7fd4ceffd640 1 --2- 172.21.15.16:0/898554309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d00d3b90 0x7fd4d00d0a20 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40097d0 tx=0x7fd4c402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70dcd989a329fa42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.730+0000 7fd4ce7fc640 1 -- 172.21.15.16:0/898554309 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4037070 con 0x7fd4d00d3b90 2024-04-25T10:12:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.731+0000 7fd4ce7fc640 1 -- 172.21.15.16:0/898554309 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c4032440 con 0x7fd4d00d3b90 2024-04-25T10:12:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.731+0000 7fd4ce7fc640 1 -- 172.21.15.16:0/898554309 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4032810 con 0x7fd4d00d3b90 2024-04-25T10:12:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.731+0000 7fd4d5d29640 1 -- 172.21.15.16:0/898554309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d00d3b90 msgr2=0x7fd4d00d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.731+0000 7fd4d5d29640 1 --2- 172.21.15.16:0/898554309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d00d3b90 0x7fd4d00d0a20 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40097d0 tx=0x7fd4c402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.731+0000 7fd4d5d29640 1 -- 172.21.15.16:0/898554309 shutdown_connections 2024-04-25T10:12:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.731+0000 7fd4d5d29640 1 --2- 172.21.15.16:0/898554309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4d00d0f60 0x7fd4d00b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.731+0000 7fd4d5d29640 1 --2- 172.21.15.16:0/898554309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d00d3b90 0x7fd4d00d0a20 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.731+0000 7fd4d5d29640 1 -- 172.21.15.16:0/898554309 >> 172.21.15.16:0/898554309 conn(0x7fd4d00726c0 msgr2=0x7fd4d0072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.731+0000 7fd4d5d29640 1 -- 172.21.15.16:0/898554309 shutdown_connections 2024-04-25T10:12:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.731+0000 7fd4d5d29640 1 -- 172.21.15.16:0/898554309 wait complete. 2024-04-25T10:12:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.732+0000 7fd4d5d29640 1 Processor -- start 2024-04-25T10:12:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.732+0000 7fd4d5d29640 1 -- start start 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.733+0000 7fd4d5d29640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4d00d0f60 0x7fd4d0156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.733+0000 7fd4d5d29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d00d3b90 0x7fd4d0157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.733+0000 7fd4d5d29640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4d015dca0 0x7fd4d01609e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.733+0000 7fd4d5d29640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d00cf050 con 0x7fd4d00d3b90 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.733+0000 7fd4d5d29640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d00b6d50 con 0x7fd4d015dca0 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.733+0000 7fd4d5d29640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d015e080 con 0x7fd4d00d0f60 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.733+0000 7fd4ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d00d3b90 0x7fd4d0157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.733+0000 7fd4ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d00d3b90 0x7fd4d0157010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60996/0 (socket says 172.21.15.16:60996) 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.733+0000 7fd4ceffd640 1 -- 172.21.15.16:0/775740990 learned_addr learned my addr 172.21.15.16:0/775740990 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.733+0000 7fd4cf7fe640 1 --2- 172.21.15.16:0/775740990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4d00d0f60 0x7fd4d0156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.733+0000 7fd4cffff640 1 --2- 172.21.15.16:0/775740990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4d015dca0 0x7fd4d01609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.734+0000 7fd4ceffd640 1 -- 172.21.15.16:0/775740990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4d00d0f60 msgr2=0x7fd4d0156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.734+0000 7fd4ceffd640 1 --2- 172.21.15.16:0/775740990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4d00d0f60 0x7fd4d0156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.734+0000 7fd4ceffd640 1 -- 172.21.15.16:0/775740990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4d015dca0 msgr2=0x7fd4d01609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.734+0000 7fd4ceffd640 1 --2- 172.21.15.16:0/775740990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4d015dca0 0x7fd4d01609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.734+0000 7fd4ceffd640 1 -- 172.21.15.16:0/775740990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d015f940 con 0x7fd4d00d3b90 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.734+0000 7fd4ceffd640 1 --2- 172.21.15.16:0/775740990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d00d3b90 0x7fd4d0157010 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4035040 tx=0x7fd4c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.734+0000 7fd4b7fff640 1 -- 172.21.15.16:0/775740990 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4037040 con 0x7fd4d00d3b90 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.734+0000 7fd4b7fff640 1 -- 172.21.15.16:0/775740990 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c4004250 con 0x7fd4d00d3b90 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.734+0000 7fd4d5d29640 1 -- 172.21.15.16:0/775740990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d015fc20 con 0x7fd4d00d3b90 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.734+0000 7fd4b7fff640 1 -- 172.21.15.16:0/775740990 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4032e40 con 0x7fd4d00d3b90 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.734+0000 7fd4d5d29640 1 -- 172.21.15.16:0/775740990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d01601e0 con 0x7fd4d00d3b90 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.735+0000 7fd4d5d29640 1 -- 172.21.15.16:0/775740990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d00750b0 con 0x7fd4d00d3b90 2024-04-25T10:12:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.736+0000 7fd4b7fff640 1 -- 172.21.15.16:0/775740990 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4c4030770 con 0x7fd4d00d3b90 2024-04-25T10:12:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.736+0000 7fd4b7fff640 1 --2- 172.21.15.16:0/775740990 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd4bc06b4d0 0x7fd4bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.736+0000 7fd4b7fff640 1 -- 172.21.15.16:0/775740990 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fd4c40ab3b0 con 0x7fd4d00d3b90 2024-04-25T10:12:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.736+0000 7fd4cf7fe640 1 --2- 172.21.15.16:0/775740990 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd4bc06b4d0 0x7fd4bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.737+0000 7fd4cf7fe640 1 --2- 172.21.15.16:0/775740990 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd4bc06b4d0 0x7fd4bc06d990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0007fd0 tx=0x7fd4c0007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.740+0000 7fd4b7fff640 1 -- 172.21.15.16:0/775740990 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4c4079bc0 con 0x7fd4d00d3b90 2024-04-25T10:12:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:48.957+0000 7fd4d5d29640 1 -- 172.21.15.16:0/775740990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd4d006ced0 con 0x7fd4d00d3b90 2024-04-25T10:12:49.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.602+0000 7fd4b7fff640 1 -- 172.21.15.16:0/775740990 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v68) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd4c407dc60 con 0x7fd4d00d3b90 2024-04-25T10:12:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 14 and data pool 15 2024-04-25T10:12:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.610+0000 7fd4d5d29640 1 -- 172.21.15.16:0/775740990 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd4bc06b4d0 msgr2=0x7fd4bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.610+0000 7fd4d5d29640 1 --2- 172.21.15.16:0/775740990 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd4bc06b4d0 0x7fd4bc06d990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0007fd0 tx=0x7fd4c0007de0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.610+0000 7fd4d5d29640 1 -- 172.21.15.16:0/775740990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d00d3b90 msgr2=0x7fd4d0157010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.610+0000 7fd4d5d29640 1 --2- 172.21.15.16:0/775740990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d00d3b90 0x7fd4d0157010 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4035040 tx=0x7fd4c40046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.611+0000 7fd4d5d29640 1 -- 172.21.15.16:0/775740990 shutdown_connections 2024-04-25T10:12:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.611+0000 7fd4d5d29640 1 --2- 172.21.15.16:0/775740990 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd4bc06b4d0 0x7fd4bc06d990 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.611+0000 7fd4d5d29640 1 --2- 172.21.15.16:0/775740990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4d015dca0 0x7fd4d01609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.611+0000 7fd4d5d29640 1 --2- 172.21.15.16:0/775740990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d00d3b90 0x7fd4d0157010 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.611+0000 7fd4d5d29640 1 --2- 172.21.15.16:0/775740990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4d00d0f60 0x7fd4d0156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.611+0000 7fd4d5d29640 1 -- 172.21.15.16:0/775740990 >> 172.21.15.16:0/775740990 conn(0x7fd4d00726c0 msgr2=0x7fd4d00b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.611+0000 7fd4d5d29640 1 -- 172.21.15.16:0/775740990 shutdown_connections 2024-04-25T10:12:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.612+0000 7fd4d5d29640 1 -- 172.21.15.16:0/775740990 wait complete. 2024-04-25T10:12:49.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:49.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.747+0000 7ff58166b640 1 Processor -- start 2024-04-25T10:12:49.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.747+0000 7ff58166b640 1 -- start start 2024-04-25T10:12:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.748+0000 7ff58166b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff57c15dce0 0x7ff57c0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.748+0000 7ff58166b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff57c15e0c0 con 0x7ff57c0b5b60 2024-04-25T10:12:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.748+0000 7ff58166b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff57c0d2510 con 0x7ff57c0d35b0 2024-04-25T10:12:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.748+0000 7ff58166b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff57c0d27f0 con 0x7ff57c15dce0 2024-04-25T10:12:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.748+0000 7ff57affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff57c15dce0 0x7ff57c0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.748+0000 7ff57b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff57c0b5b60 0x7ff57c0b6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45194/0 (socket says 172.21.15.16:45194) 2024-04-25T10:12:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.748+0000 7ff57b7fe640 1 -- 172.21.15.16:0/2033707412 learned_addr learned my addr 172.21.15.16:0/2033707412 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.748+0000 7ff57affd640 1 -- 172.21.15.16:0/2033707412 >> v1:172.21.15.186:6789/0 conn(0x7ff57c0d35b0 legacy=0x7ff57c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.748+0000 7ff57affd640 1 -- 172.21.15.16:0/2033707412 >> v1:172.21.15.16:6789/0 conn(0x7ff57c0b5b60 legacy=0x7ff57c0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.748+0000 7ff57affd640 1 -- 172.21.15.16:0/2033707412 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff57c0d2ad0 con 0x7ff57c15dce0 2024-04-25T10:12:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.748+0000 7ff579ffb640 1 -- 172.21.15.16:0/2033707412 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774255892 0 0) 0x7ff57c15e0c0 con 0x7ff57c0b5b60 2024-04-25T10:12:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.749+0000 7ff57affd640 1 --2- 172.21.15.16:0/2033707412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff57c15dce0 0x7ff57c0d1a50 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009d90 tx=0x7ff56c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bed5cee292b5727 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.749+0000 7ff579ffb640 1 -- 172.21.15.16:0/2033707412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c007740 con 0x7ff57c15dce0 2024-04-25T10:12:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.749+0000 7ff579ffb640 1 -- 172.21.15.16:0/2033707412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56c0078e0 con 0x7ff57c15dce0 2024-04-25T10:12:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.749+0000 7ff579ffb640 1 -- 172.21.15.16:0/2033707412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c007cb0 con 0x7ff57c15dce0 2024-04-25T10:12:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.750+0000 7ff58166b640 1 -- 172.21.15.16:0/2033707412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff57c15dce0 msgr2=0x7ff57c0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.750+0000 7ff58166b640 1 --2- 172.21.15.16:0/2033707412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff57c15dce0 0x7ff57c0d1a50 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009d90 tx=0x7ff56c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.750+0000 7ff58166b640 1 -- 172.21.15.16:0/2033707412 shutdown_connections 2024-04-25T10:12:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.750+0000 7ff58166b640 1 --2- 172.21.15.16:0/2033707412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff57c15dce0 0x7ff57c0d1a50 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.750+0000 7ff58166b640 1 -- 172.21.15.16:0/2033707412 >> 172.21.15.16:0/2033707412 conn(0x7ff57c072900 msgr2=0x7ff57c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.750+0000 7ff58166b640 1 -- 172.21.15.16:0/2033707412 shutdown_connections 2024-04-25T10:12:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.750+0000 7ff58166b640 1 -- 172.21.15.16:0/2033707412 wait complete. 2024-04-25T10:12:49.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.751+0000 7ff58166b640 1 Processor -- start 2024-04-25T10:12:49.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.751+0000 7ff58166b640 1 -- start start 2024-04-25T10:12:49.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.752+0000 7ff58166b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff57c0b5b60 0x7ff57c0b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.752+0000 7ff58166b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff57c0d35b0 0x7ff57c0b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.752+0000 7ff58166b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff57c15dce0 0x7ff57c0c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.752+0000 7ff58166b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c0b9820 con 0x7ff57c15dce0 2024-04-25T10:12:49.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.752+0000 7ff58166b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c0d3090 con 0x7ff57c0b5b60 2024-04-25T10:12:49.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.752+0000 7ff58166b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c0b99a0 con 0x7ff57c0d35b0 2024-04-25T10:12:49.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.752+0000 7ff57b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff57c15dce0 0x7ff57c0c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.752+0000 7ff57b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff57c15dce0 0x7ff57c0c90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32780/0 (socket says 172.21.15.16:32780) 2024-04-25T10:12:49.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.752+0000 7ff57b7fe640 1 -- 172.21.15.16:0/4210372239 learned_addr learned my addr 172.21.15.16:0/4210372239 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:49.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.752+0000 7ff57affd640 1 --2- 172.21.15.16:0/4210372239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff57c0b5b60 0x7ff57c0b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.752+0000 7ff57a7fc640 1 --2- 172.21.15.16:0/4210372239 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff57c0d35b0 0x7ff57c0b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.752+0000 7ff57b7fe640 1 -- 172.21.15.16:0/4210372239 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff57c0d35b0 msgr2=0x7ff57c0b6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.752+0000 7ff57b7fe640 1 --2- 172.21.15.16:0/4210372239 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff57c0d35b0 0x7ff57c0b6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.752+0000 7ff57b7fe640 1 -- 172.21.15.16:0/4210372239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff57c0b5b60 msgr2=0x7ff57c0b7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.752+0000 7ff57b7fe640 1 --2- 172.21.15.16:0/4210372239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff57c0b5b60 0x7ff57c0b7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.752+0000 7ff57b7fe640 1 -- 172.21.15.16:0/4210372239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff57c153750 con 0x7ff57c15dce0 2024-04-25T10:12:49.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.753+0000 7ff57b7fe640 1 --2- 172.21.15.16:0/4210372239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff57c15dce0 0x7ff57c0c90f0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7ff57400c450 tx=0x7ff57400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.753+0000 7ff5737fe640 1 -- 172.21.15.16:0/4210372239 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff574018070 con 0x7ff57c15dce0 2024-04-25T10:12:49.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.753+0000 7ff5737fe640 1 -- 172.21.15.16:0/4210372239 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff574007500 con 0x7ff57c15dce0 2024-04-25T10:12:49.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.753+0000 7ff58166b640 1 -- 172.21.15.16:0/4210372239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff57c15e450 con 0x7ff57c15dce0 2024-04-25T10:12:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.753+0000 7ff5737fe640 1 -- 172.21.15.16:0/4210372239 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff574004a20 con 0x7ff57c15dce0 2024-04-25T10:12:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.753+0000 7ff58166b640 1 -- 172.21.15.16:0/4210372239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff57c15ea10 con 0x7ff57c15dce0 2024-04-25T10:12:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.754+0000 7ff58166b640 1 -- 172.21.15.16:0/4210372239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff57c0749b0 con 0x7ff57c15dce0 2024-04-25T10:12:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.754+0000 7ff5737fe640 1 -- 172.21.15.16:0/4210372239 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff574004bc0 con 0x7ff57c15dce0 2024-04-25T10:12:49.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.755+0000 7ff5737fe640 1 --2- 172.21.15.16:0/4210372239 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff55c06b4d0 0x7ff55c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.755+0000 7ff5737fe640 1 -- 172.21.15.16:0/4210372239 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff574089460 con 0x7ff57c15dce0 2024-04-25T10:12:49.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.755+0000 7ff57affd640 1 --2- 172.21.15.16:0/4210372239 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff55c06b4d0 0x7ff55c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.755+0000 7ff57affd640 1 --2- 172.21.15.16:0/4210372239 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff55c06b4d0 0x7ff55c06d990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff56c034000 tx=0x7ff56c02f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.759+0000 7ff5737fe640 1 -- 172.21.15.16:0/4210372239 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff574057b70 con 0x7ff57c15dce0 2024-04-25T10:12:49.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.939+0000 7ff58166b640 1 -- 172.21.15.16:0/4210372239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff57c0b25a0 con 0x7ff57c15dce0 2024-04-25T10:12:49.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.941+0000 7ff5737fe640 1 -- 172.21.15.16:0/4210372239 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16621 (secure 0 0 0) 0x7ff57405bc10 con 0x7ff57c15dce0 2024-04-25T10:12:49.945 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:49.946 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":99,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:12:48.958630+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:47.551100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:47.939289+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:49.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.943+0000 7ff58166b640 1 -- 172.21.15.16:0/4210372239 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff55c06b4d0 msgr2=0x7ff55c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.943+0000 7ff58166b640 1 --2- 172.21.15.16:0/4210372239 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff55c06b4d0 0x7ff55c06d990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff56c034000 tx=0x7ff56c02f000 comp rx=0 tx=0).stop 2024-04-25T10:12:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.943+0000 7ff58166b640 1 -- 172.21.15.16:0/4210372239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff57c15dce0 msgr2=0x7ff57c0c90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.943+0000 7ff58166b640 1 --2- 172.21.15.16:0/4210372239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff57c15dce0 0x7ff57c0c90f0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7ff57400c450 tx=0x7ff57400c910 comp rx=0 tx=0).stop 2024-04-25T10:12:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.943+0000 7ff58166b640 1 -- 172.21.15.16:0/4210372239 shutdown_connections 2024-04-25T10:12:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.943+0000 7ff58166b640 1 --2- 172.21.15.16:0/4210372239 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff55c06b4d0 0x7ff55c06d990 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.943+0000 7ff58166b640 1 --2- 172.21.15.16:0/4210372239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff57c15dce0 0x7ff57c0c90f0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.943+0000 7ff58166b640 1 --2- 172.21.15.16:0/4210372239 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff57c0d35b0 0x7ff57c0b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.943+0000 7ff58166b640 1 --2- 172.21.15.16:0/4210372239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff57c0b5b60 0x7ff57c0b7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.943+0000 7ff58166b640 1 -- 172.21.15.16:0/4210372239 >> 172.21.15.16:0/4210372239 conn(0x7ff57c072900 msgr2=0x7ff57c15cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.943+0000 7ff58166b640 1 -- 172.21.15.16:0/4210372239 shutdown_connections 2024-04-25T10:12:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:49.943+0000 7ff58166b640 1 -- 172.21.15.16:0/4210372239 wait complete. 2024-04-25T10:12:49.959 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:50.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.055+0000 7f42d516a640 1 Processor -- start 2024-04-25T10:12:50.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.055+0000 7f42d516a640 1 -- start start 2024-04-25T10:12:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.056+0000 7f42d516a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f42d015dce0 0x7f42d00d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.056+0000 7f42d516a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f42d00d35b0 0x7f42d00b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.056+0000 7f42d516a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42d00d1f90 0x7f42d00b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.056+0000 7f42d516a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d015e0c0 con 0x7f42d015dce0 2024-04-25T10:12:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.056+0000 7f42d516a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d00d2510 con 0x7f42d00d35b0 2024-04-25T10:12:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.056+0000 7f42d516a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d00d27f0 con 0x7f42d00d1f90 2024-04-25T10:12:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.056+0000 7f42ced9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f42d015dce0 0x7f42d00d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.056+0000 7f42ced9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f42d015dce0 0x7f42d00d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32796/0 (socket says 172.21.15.16:32796) 2024-04-25T10:12:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.056+0000 7f42ced9d640 1 -- 172.21.15.16:0/439531530 learned_addr learned my addr 172.21.15.16:0/439531530 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.056+0000 7f42ce59c640 1 --2- 172.21.15.16:0/439531530 >> v2:172.21.15.186:3300/0 conn(0x7f42d00d35b0 0x7f42d00b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.056+0000 7f42cf59e640 1 --2- 172.21.15.16:0/439531530 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42d00d1f90 0x7f42d00b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.056+0000 7f42ced9d640 1 -- 172.21.15.16:0/439531530 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42d00d1f90 msgr2=0x7f42d00b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.056+0000 7f42ced9d640 1 --2- 172.21.15.16:0/439531530 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42d00d1f90 0x7f42d00b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.056+0000 7f42ced9d640 1 -- 172.21.15.16:0/439531530 >> v2:172.21.15.186:3300/0 conn(0x7f42d00d35b0 msgr2=0x7f42d00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.056+0000 7f42ced9d640 1 --2- 172.21.15.16:0/439531530 >> v2:172.21.15.186:3300/0 conn(0x7f42d00d35b0 0x7f42d00b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.056+0000 7f42ced9d640 1 -- 172.21.15.16:0/439531530 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d00d2ad0 con 0x7f42d015dce0 2024-04-25T10:12:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.057+0000 7f42ced9d640 1 --2- 172.21.15.16:0/439531530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d015dce0 0x7f42d00d1a50 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f42d00cdfa0 tx=0x7f42c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2be90968f4991d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.057+0000 7f42cdd9b640 1 -- 172.21.15.16:0/439531530 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c0037070 con 0x7f42d015dce0 2024-04-25T10:12:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.057+0000 7f42cdd9b640 1 -- 172.21.15.16:0/439531530 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c0032440 con 0x7f42d015dce0 2024-04-25T10:12:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.057+0000 7f42cdd9b640 1 -- 172.21.15.16:0/439531530 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c0032810 con 0x7f42d015dce0 2024-04-25T10:12:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.058+0000 7f42d516a640 1 -- 172.21.15.16:0/439531530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d015dce0 msgr2=0x7f42d00d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.058+0000 7f42d516a640 1 --2- 172.21.15.16:0/439531530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d015dce0 0x7f42d00d1a50 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f42d00cdfa0 tx=0x7f42c002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.058+0000 7f42d516a640 1 -- 172.21.15.16:0/439531530 shutdown_connections 2024-04-25T10:12:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.058+0000 7f42d516a640 1 --2- 172.21.15.16:0/439531530 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42d00d1f90 0x7f42d00b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.058+0000 7f42d516a640 1 --2- 172.21.15.16:0/439531530 >> v2:172.21.15.186:3300/0 conn(0x7f42d00d35b0 0x7f42d00b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.058+0000 7f42d516a640 1 --2- 172.21.15.16:0/439531530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d015dce0 0x7f42d00d1a50 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.058+0000 7f42d516a640 1 -- 172.21.15.16:0/439531530 >> 172.21.15.16:0/439531530 conn(0x7f42d0072900 msgr2=0x7f42d00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.058+0000 7f42d516a640 1 -- 172.21.15.16:0/439531530 shutdown_connections 2024-04-25T10:12:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.058+0000 7f42d516a640 1 -- 172.21.15.16:0/439531530 wait complete. 2024-04-25T10:12:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.059+0000 7f42d516a640 1 Processor -- start 2024-04-25T10:12:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.059+0000 7f42d516a640 1 -- start start 2024-04-25T10:12:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.060+0000 7f42d516a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42d00d1f90 0x7f42d0157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.060+0000 7f42d516a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42d00d35b0 0x7f42d01f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.060+0000 7f42d516a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d015dce0 0x7f42d00b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.060+0000 7f42d516a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d00b9820 con 0x7f42d015dce0 2024-04-25T10:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.060+0000 7f42d516a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d00d3090 con 0x7f42d00d35b0 2024-04-25T10:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.060+0000 7f42d516a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d00b99a0 con 0x7f42d00d1f90 2024-04-25T10:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.060+0000 7f42cf59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d015dce0 0x7f42d00b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.060+0000 7f42cf59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d015dce0 0x7f42d00b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32800/0 (socket says 172.21.15.16:32800) 2024-04-25T10:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.060+0000 7f42cf59e640 1 -- 172.21.15.16:0/3783976587 learned_addr learned my addr 172.21.15.16:0/3783976587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.060+0000 7f42ced9d640 1 --2- 172.21.15.16:0/3783976587 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42d00d1f90 0x7f42d0157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.060+0000 7f42ce59c640 1 --2- 172.21.15.16:0/3783976587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42d00d35b0 0x7f42d01f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.060+0000 7f42cf59e640 1 -- 172.21.15.16:0/3783976587 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42d00d1f90 msgr2=0x7f42d0157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.060+0000 7f42cf59e640 1 --2- 172.21.15.16:0/3783976587 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42d00d1f90 0x7f42d0157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.060+0000 7f42cf59e640 1 -- 172.21.15.16:0/3783976587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42d00d35b0 msgr2=0x7f42d01f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.060+0000 7f42cf59e640 1 --2- 172.21.15.16:0/3783976587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42d00d35b0 0x7f42d01f36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.060+0000 7f42cf59e640 1 -- 172.21.15.16:0/3783976587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d01f40d0 con 0x7f42d015dce0 2024-04-25T10:12:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.060+0000 7f42ced9d640 1 --2- 172.21.15.16:0/3783976587 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42d00d1f90 0x7f42d0157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.061+0000 7f42cf59e640 1 --2- 172.21.15.16:0/3783976587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d015dce0 0x7f42d00b48e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f42c8007d20 tx=0x7f42c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.061+0000 7f42b77fe640 1 -- 172.21.15.16:0/3783976587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c8014420 con 0x7f42d015dce0 2024-04-25T10:12:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.061+0000 7f42b77fe640 1 -- 172.21.15.16:0/3783976587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c8019070 con 0x7f42d015dce0 2024-04-25T10:12:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.061+0000 7f42d516a640 1 -- 172.21.15.16:0/3783976587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d01f43b0 con 0x7f42d015dce0 2024-04-25T10:12:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.061+0000 7f42b77fe640 1 -- 172.21.15.16:0/3783976587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c8014820 con 0x7f42d015dce0 2024-04-25T10:12:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.061+0000 7f42d516a640 1 -- 172.21.15.16:0/3783976587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d015f540 con 0x7f42d015dce0 2024-04-25T10:12:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.062+0000 7f42d516a640 1 -- 172.21.15.16:0/3783976587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d00749b0 con 0x7f42d015dce0 2024-04-25T10:12:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.062+0000 7f42b77fe640 1 -- 172.21.15.16:0/3783976587 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42c80149c0 con 0x7f42d015dce0 2024-04-25T10:12:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.063+0000 7f42b77fe640 1 --2- 172.21.15.16:0/3783976587 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f42bc06b4d0 0x7f42bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.063+0000 7f42b77fe640 1 -- 172.21.15.16:0/3783976587 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42c808a8b0 con 0x7f42d015dce0 2024-04-25T10:12:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.063+0000 7f42ced9d640 1 --2- 172.21.15.16:0/3783976587 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f42bc06b4d0 0x7f42bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.063+0000 7f42ced9d640 1 --2- 172.21.15.16:0/3783976587 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f42bc06b4d0 0x7f42bc06d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f42c0034000 tx=0x7f42c002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.067+0000 7f42b77fe640 1 -- 172.21.15.16:0/3783976587 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f42c80597c0 con 0x7f42d015dce0 2024-04-25T10:12:50.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.246+0000 7f42d516a640 1 -- 172.21.15.16:0/3783976587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42d015d2c0 con 0x7f42d015dce0 2024-04-25T10:12:50.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.247+0000 7f42b77fe640 1 -- 172.21.15.16:0/3783976587 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16621 (secure 0 0 0) 0x7f42c80590c0 con 0x7f42d015dce0 2024-04-25T10:12:50.252 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:50.252 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":99,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:12:48.958630+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:47.551100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:47.939289+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.249+0000 7f42d516a640 1 -- 172.21.15.16:0/3783976587 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f42bc06b4d0 msgr2=0x7f42bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.249+0000 7f42d516a640 1 --2- 172.21.15.16:0/3783976587 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f42bc06b4d0 0x7f42bc06d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f42c0034000 tx=0x7f42c002e000 comp rx=0 tx=0).stop 2024-04-25T10:12:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.249+0000 7f42d516a640 1 -- 172.21.15.16:0/3783976587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d015dce0 msgr2=0x7f42d00b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.249+0000 7f42d516a640 1 --2- 172.21.15.16:0/3783976587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d015dce0 0x7f42d00b48e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f42c8007d20 tx=0x7f42c800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.249+0000 7f42d516a640 1 -- 172.21.15.16:0/3783976587 shutdown_connections 2024-04-25T10:12:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.249+0000 7f42d516a640 1 --2- 172.21.15.16:0/3783976587 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f42bc06b4d0 0x7f42bc06d990 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.249+0000 7f42d516a640 1 --2- 172.21.15.16:0/3783976587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d015dce0 0x7f42d00b48e0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.249+0000 7f42d516a640 1 --2- 172.21.15.16:0/3783976587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42d00d35b0 0x7f42d01f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.249+0000 7f42d516a640 1 --2- 172.21.15.16:0/3783976587 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42d00d1f90 0x7f42d0157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.249+0000 7f42d516a640 1 -- 172.21.15.16:0/3783976587 >> 172.21.15.16:0/3783976587 conn(0x7f42d0072900 msgr2=0x7f42d00b6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.249+0000 7f42d516a640 1 -- 172.21.15.16:0/3783976587 shutdown_connections 2024-04-25T10:12:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.250+0000 7f42d516a640 1 -- 172.21.15.16:0/3783976587 wait complete. 2024-04-25T10:12:50.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:12:50.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.364+0000 7f39c585d640 1 Processor -- start 2024-04-25T10:12:50.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.365+0000 7f39c585d640 1 -- start start 2024-04-25T10:12:50.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.365+0000 7f39c585d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f39c00b98b0 0x7f39c015eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.365+0000 7f39c585d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f39c00b5a30 0x7f39c00b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.365+0000 7f39c585d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c015f080 con 0x7f39c00b5580 2024-04-25T10:12:50.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.365+0000 7f39c585d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39c015bb80 con 0x7f39c00b5a30 2024-04-25T10:12:50.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.365+0000 7f39c585d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39c006c4b0 con 0x7f39c00b98b0 2024-04-25T10:12:50.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.365+0000 7f39bf7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f39c00b5a30 0x7f39c00b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.365+0000 7f39bf7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f39c00b5a30 0x7f39c00b2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32810/0 (socket says 172.21.15.16:32810) 2024-04-25T10:12:50.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.365+0000 7f39bf7fe640 1 -- 172.21.15.16:0/45187831 learned_addr learned my addr 172.21.15.16:0/45187831 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:50.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.366+0000 7f39beffd640 1 --2- 172.21.15.16:0/45187831 >> v2:172.21.15.186:3300/0 conn(0x7f39c00b98b0 0x7f39c015eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.366+0000 7f39bf7fe640 1 -- 172.21.15.16:0/45187831 >> v2:172.21.15.186:3300/0 conn(0x7f39c00b98b0 msgr2=0x7f39c015eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:50.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.366+0000 7f39bf7fe640 1 --2- 172.21.15.16:0/45187831 >> v2:172.21.15.186:3300/0 conn(0x7f39c00b98b0 0x7f39c015eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.366+0000 7f39bf7fe640 1 -- 172.21.15.16:0/45187831 >> v1:172.21.15.186:6789/0 conn(0x7f39c00b5580 legacy=0x7f39c00b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.366+0000 7f39bf7fe640 1 -- 172.21.15.16:0/45187831 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c006c790 con 0x7f39c00b5a30 2024-04-25T10:12:50.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.366+0000 7f39bf7fe640 1 --2- 172.21.15.16:0/45187831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39c00b5a30 0x7f39c00b2bd0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f39c00d4b10 tx=0x7f39b802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eefdc9a2f3a3c24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.367+0000 7f39bdffb640 1 -- 172.21.15.16:0/45187831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8036070 con 0x7f39c00b5a30 2024-04-25T10:12:50.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.367+0000 7f39bdffb640 1 -- 172.21.15.16:0/45187831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b8031600 con 0x7f39c00b5a30 2024-04-25T10:12:50.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.367+0000 7f39bdffb640 1 -- 172.21.15.16:0/45187831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b80319d0 con 0x7f39c00b5a30 2024-04-25T10:12:50.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.367+0000 7f39c585d640 1 -- 172.21.15.16:0/45187831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39c00b5a30 msgr2=0x7f39c00b2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.367+0000 7f39c585d640 1 --2- 172.21.15.16:0/45187831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39c00b5a30 0x7f39c00b2bd0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f39c00d4b10 tx=0x7f39b802a610 comp rx=0 tx=0).stop 2024-04-25T10:12:50.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.367+0000 7f39c585d640 1 -- 172.21.15.16:0/45187831 shutdown_connections 2024-04-25T10:12:50.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.367+0000 7f39c585d640 1 --2- 172.21.15.16:0/45187831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39c00b5a30 0x7f39c00b2bd0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.367+0000 7f39c585d640 1 --2- 172.21.15.16:0/45187831 >> v2:172.21.15.186:3300/0 conn(0x7f39c00b98b0 0x7f39c015eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.367+0000 7f39c585d640 1 -- 172.21.15.16:0/45187831 >> 172.21.15.16:0/45187831 conn(0x7f39c00d0580 msgr2=0x7f39c00d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:50.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.367+0000 7f39c585d640 1 -- 172.21.15.16:0/45187831 shutdown_connections 2024-04-25T10:12:50.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.367+0000 7f39c585d640 1 -- 172.21.15.16:0/45187831 wait complete. 2024-04-25T10:12:50.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.369+0000 7f39c585d640 1 Processor -- start 2024-04-25T10:12:50.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.369+0000 7f39c585d640 1 -- start start 2024-04-25T10:12:50.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.369+0000 7f39c585d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39c00b5580 0x7f39c014c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.369+0000 7f39c585d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39c00b5a30 0x7f39c00c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.369+0000 7f39c585d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39c00cbbe0 0x7f39c00cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.369+0000 7f39c585d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c00b5e60 con 0x7f39c00cbbe0 2024-04-25T10:12:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.369+0000 7f39c585d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c006cd50 con 0x7f39c00b5a30 2024-04-25T10:12:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.369+0000 7f39c585d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c006c140 con 0x7f39c00b5580 2024-04-25T10:12:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.370+0000 7f39bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39c00cbbe0 0x7f39c00cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.370+0000 7f39bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39c00cbbe0 0x7f39c00cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32816/0 (socket says 172.21.15.16:32816) 2024-04-25T10:12:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.370+0000 7f39bf7fe640 1 -- 172.21.15.16:0/2198111017 learned_addr learned my addr 172.21.15.16:0/2198111017 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.370+0000 7f39beffd640 1 --2- 172.21.15.16:0/2198111017 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39c00b5580 0x7f39c014c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.370+0000 7f39be7fc640 1 --2- 172.21.15.16:0/2198111017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39c00b5a30 0x7f39c00c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.370+0000 7f39bf7fe640 1 -- 172.21.15.16:0/2198111017 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39c00b5580 msgr2=0x7f39c014c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.370+0000 7f39bf7fe640 1 --2- 172.21.15.16:0/2198111017 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39c00b5580 0x7f39c014c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.370+0000 7f39bf7fe640 1 -- 172.21.15.16:0/2198111017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39c00b5a30 msgr2=0x7f39c00c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.370+0000 7f39bf7fe640 1 --2- 172.21.15.16:0/2198111017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39c00b5a30 0x7f39c00c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.370+0000 7f39bf7fe640 1 -- 172.21.15.16:0/2198111017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c00c7b70 con 0x7f39c00cbbe0 2024-04-25T10:12:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.370+0000 7f39bf7fe640 1 --2- 172.21.15.16:0/2198111017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39c00cbbe0 0x7f39c00cc090 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009f40 tx=0x7f39b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.370+0000 7f39a7fff640 1 -- 172.21.15.16:0/2198111017 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8036050 con 0x7f39c00cbbe0 2024-04-25T10:12:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.370+0000 7f39a7fff640 1 -- 172.21.15.16:0/2198111017 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b8042070 con 0x7f39c00cbbe0 2024-04-25T10:12:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.370+0000 7f39a7fff640 1 -- 172.21.15.16:0/2198111017 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8007bd0 con 0x7f39c00cbbe0 2024-04-25T10:12:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.370+0000 7f39c585d640 1 -- 172.21.15.16:0/2198111017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c014cf10 con 0x7f39c00cbbe0 2024-04-25T10:12:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.370+0000 7f39c585d640 1 -- 172.21.15.16:0/2198111017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c014d4d0 con 0x7f39c00cbbe0 2024-04-25T10:12:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.371+0000 7f39c585d640 1 -- 172.21.15.16:0/2198111017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c00750b0 con 0x7f39c00cbbe0 2024-04-25T10:12:50.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.372+0000 7f39a7fff640 1 -- 172.21.15.16:0/2198111017 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f39b8047460 con 0x7f39c00cbbe0 2024-04-25T10:12:50.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.372+0000 7f39a7fff640 1 --2- 172.21.15.16:0/2198111017 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f39ac06b500 0x7f39ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.372+0000 7f39a7fff640 1 -- 172.21.15.16:0/2198111017 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39b80ab130 con 0x7f39c00cbbe0 2024-04-25T10:12:50.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.372+0000 7f39beffd640 1 --2- 172.21.15.16:0/2198111017 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f39ac06b500 0x7f39ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.373+0000 7f39beffd640 1 --2- 172.21.15.16:0/2198111017 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f39ac06b500 0x7f39ac06d9c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f39b0006fd0 tx=0x7f39b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.377+0000 7f39a7fff640 1 -- 172.21.15.16:0/2198111017 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f39b8079960 con 0x7f39c00cbbe0 2024-04-25T10:12:50.592 INFO:tasks.ceph.mds.b.smithi186.stderr:2024-04-25T10:12:50.588+0000 7f583d4e9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:12:50.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:50.597+0000 7f39c585d640 1 -- 172.21.15.16:0/2198111017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f39c00b3dd0 con 0x7f39c00cbbe0 2024-04-25T10:12:51.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.475+0000 7f39a7fff640 1 -- 172.21.15.16:0/2198111017 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) v1 ==== 118+0+0 (secure 0 0 0) 0x7f39b807da00 con 0x7f39c00cbbe0 2024-04-25T10:12:51.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.478+0000 7f39c585d640 1 -- 172.21.15.16:0/2198111017 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f39ac06b500 msgr2=0x7f39ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.478+0000 7f39c585d640 1 --2- 172.21.15.16:0/2198111017 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f39ac06b500 0x7f39ac06d9c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f39b0006fd0 tx=0x7f39b0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:51.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.478+0000 7f39c585d640 1 -- 172.21.15.16:0/2198111017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39c00cbbe0 msgr2=0x7f39c00cc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.478+0000 7f39c585d640 1 --2- 172.21.15.16:0/2198111017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39c00cbbe0 0x7f39c00cc090 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009f40 tx=0x7f39b8004a40 comp rx=0 tx=0).stop 2024-04-25T10:12:51.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.478+0000 7f39c585d640 1 -- 172.21.15.16:0/2198111017 shutdown_connections 2024-04-25T10:12:51.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.478+0000 7f39c585d640 1 --2- 172.21.15.16:0/2198111017 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f39ac06b500 0x7f39ac06d9c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.478+0000 7f39c585d640 1 --2- 172.21.15.16:0/2198111017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39c00cbbe0 0x7f39c00cc090 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.478+0000 7f39c585d640 1 --2- 172.21.15.16:0/2198111017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39c00b5a30 0x7f39c00c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.478+0000 7f39c585d640 1 --2- 172.21.15.16:0/2198111017 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39c00b5580 0x7f39c014c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.478+0000 7f39c585d640 1 -- 172.21.15.16:0/2198111017 >> 172.21.15.16:0/2198111017 conn(0x7f39c00d0580 msgr2=0x7f39c00d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:51.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.478+0000 7f39c585d640 1 -- 172.21.15.16:0/2198111017 shutdown_connections 2024-04-25T10:12:51.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.478+0000 7f39c585d640 1 -- 172.21.15.16:0/2198111017 wait complete. 2024-04-25T10:12:51.496 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:12:51.496 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:12:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.602+0000 7f3ab181f640 1 Processor -- start 2024-04-25T10:12:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.602+0000 7f3ab181f640 1 -- start start 2024-04-25T10:12:51.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.603+0000 7f3ab181f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3aac071930 0x7f3aac06e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.603+0000 7f3ab181f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3aac06ea30 0x7f3aac0b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.603+0000 7f3ab181f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aac15dc30 con 0x7f3aac15a260 2024-04-25T10:12:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.603+0000 7f3ab181f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aac15df10 con 0x7f3aac071930 2024-04-25T10:12:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.603+0000 7f3ab181f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac15e1f0 con 0x7f3aac06ea30 2024-04-25T10:12:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.603+0000 7f3aaa7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3aac071930 0x7f3aac06e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.603+0000 7f3aaaffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3aac15a260 0x7f3aac071220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45208/0 (socket says 172.21.15.16:45208) 2024-04-25T10:12:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.603+0000 7f3aaaffd640 1 -- 172.21.15.16:0/1496134852 learned_addr learned my addr 172.21.15.16:0/1496134852 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.603+0000 7f3aab7fe640 1 --2- 172.21.15.16:0/1496134852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3aac06ea30 0x7f3aac0b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.603+0000 7f3aaa7fc640 1 -- 172.21.15.16:0/1496134852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3aac06ea30 msgr2=0x7f3aac0b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.603+0000 7f3aaa7fc640 1 --2- 172.21.15.16:0/1496134852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3aac06ea30 0x7f3aac0b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.603+0000 7f3aaa7fc640 1 -- 172.21.15.16:0/1496134852 >> v1:172.21.15.16:6789/0 conn(0x7f3aac15a260 legacy=0x7f3aac071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.603+0000 7f3aa9ffb640 1 -- 172.21.15.16:0/1496134852 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584857027 0 0) 0x7f3aac15dc30 con 0x7f3aac15a260 2024-04-25T10:12:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.603+0000 7f3aaa7fc640 1 -- 172.21.15.16:0/1496134852 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac15e4d0 con 0x7f3aac071930 2024-04-25T10:12:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.603+0000 7f3aab7fe640 1 --2- 172.21.15.16:0/1496134852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3aac06ea30 0x7f3aac0b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.604+0000 7f3aaa7fc640 1 --2- 172.21.15.16:0/1496134852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aac071930 0x7f3aac06e4f0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00097a0 tx=0x7f3aa002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e962420ebe41809 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.604+0000 7f3aa9ffb640 1 -- 172.21.15.16:0/1496134852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0037070 con 0x7f3aac071930 2024-04-25T10:12:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.604+0000 7f3aa9ffb640 1 -- 172.21.15.16:0/1496134852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa0032440 con 0x7f3aac071930 2024-04-25T10:12:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.604+0000 7f3aa9ffb640 1 -- 172.21.15.16:0/1496134852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0032810 con 0x7f3aac071930 2024-04-25T10:12:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.605+0000 7f3ab181f640 1 -- 172.21.15.16:0/1496134852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aac071930 msgr2=0x7f3aac06e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.605+0000 7f3ab181f640 1 --2- 172.21.15.16:0/1496134852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aac071930 0x7f3aac06e4f0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00097a0 tx=0x7f3aa002a180 comp rx=0 tx=0).stop 2024-04-25T10:12:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.605+0000 7f3ab181f640 1 -- 172.21.15.16:0/1496134852 shutdown_connections 2024-04-25T10:12:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.605+0000 7f3ab181f640 1 --2- 172.21.15.16:0/1496134852 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3aac06ea30 0x7f3aac0b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.605+0000 7f3ab181f640 1 --2- 172.21.15.16:0/1496134852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aac071930 0x7f3aac06e4f0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.605+0000 7f3ab181f640 1 -- 172.21.15.16:0/1496134852 >> 172.21.15.16:0/1496134852 conn(0x7f3aac06aba0 msgr2=0x7f3aac06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.605+0000 7f3ab181f640 1 -- 172.21.15.16:0/1496134852 shutdown_connections 2024-04-25T10:12:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.605+0000 7f3ab181f640 1 -- 172.21.15.16:0/1496134852 wait complete. 2024-04-25T10:12:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.606+0000 7f3ab181f640 1 Processor -- start 2024-04-25T10:12:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.606+0000 7f3ab181f640 1 -- start start 2024-04-25T10:12:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.607+0000 7f3ab181f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3aac06ea30 0x7f3aac14a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.607+0000 7f3ab181f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aac071930 0x7f3aac14e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.607+0000 7f3ab181f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3aac15a260 0x7f3aac0b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.607+0000 7f3ab181f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac06efc0 con 0x7f3aac071930 2024-04-25T10:12:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.607+0000 7f3ab181f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac06ee40 con 0x7f3aac15a260 2024-04-25T10:12:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.607+0000 7f3ab181f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac0cf6f0 con 0x7f3aac06ea30 2024-04-25T10:12:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.607+0000 7f3aaa7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aac071930 0x7f3aac14e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.607+0000 7f3aaa7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aac071930 0x7f3aac14e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32830/0 (socket says 172.21.15.16:32830) 2024-04-25T10:12:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.607+0000 7f3aaa7fc640 1 -- 172.21.15.16:0/2966679322 learned_addr learned my addr 172.21.15.16:0/2966679322 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.607+0000 7f3aaaffd640 1 --2- 172.21.15.16:0/2966679322 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3aac06ea30 0x7f3aac14a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.607+0000 7f3aab7fe640 1 --2- 172.21.15.16:0/2966679322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3aac15a260 0x7f3aac0b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.607+0000 7f3aaa7fc640 1 -- 172.21.15.16:0/2966679322 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3aac06ea30 msgr2=0x7f3aac14a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.607+0000 7f3aaa7fc640 1 --2- 172.21.15.16:0/2966679322 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3aac06ea30 0x7f3aac14a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.607+0000 7f3aaa7fc640 1 -- 172.21.15.16:0/2966679322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3aac15a260 msgr2=0x7f3aac0b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.607+0000 7f3aaa7fc640 1 --2- 172.21.15.16:0/2966679322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3aac15a260 0x7f3aac0b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.607+0000 7f3aaa7fc640 1 -- 172.21.15.16:0/2966679322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac14e950 con 0x7f3aac071930 2024-04-25T10:12:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.607+0000 7f3aab7fe640 1 --2- 172.21.15.16:0/2966679322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3aac15a260 0x7f3aac0b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.608+0000 7f3aaa7fc640 1 --2- 172.21.15.16:0/2966679322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aac071930 0x7f3aac14e410 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0035040 tx=0x7f3aa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.608+0000 7f3a93fff640 1 -- 172.21.15.16:0/2966679322 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0037040 con 0x7f3aac071930 2024-04-25T10:12:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.608+0000 7f3a93fff640 1 -- 172.21.15.16:0/2966679322 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa0004250 con 0x7f3aac071930 2024-04-25T10:12:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.608+0000 7f3a93fff640 1 -- 172.21.15.16:0/2966679322 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0032e40 con 0x7f3aac071930 2024-04-25T10:12:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.608+0000 7f3ab181f640 1 -- 172.21.15.16:0/2966679322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aac14ebf0 con 0x7f3aac071930 2024-04-25T10:12:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.608+0000 7f3ab181f640 1 -- 172.21.15.16:0/2966679322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aac14f1b0 con 0x7f3aac071930 2024-04-25T10:12:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.609+0000 7f3ab181f640 1 -- 172.21.15.16:0/2966679322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aac0750b0 con 0x7f3aac071930 2024-04-25T10:12:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.609+0000 7f3a93fff640 1 -- 172.21.15.16:0/2966679322 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3aa00306f0 con 0x7f3aac071930 2024-04-25T10:12:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.610+0000 7f3a93fff640 1 --2- 172.21.15.16:0/2966679322 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a9806b4d0 0x7f3a9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.610+0000 7f3a93fff640 1 -- 172.21.15.16:0/2966679322 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3aa00ab2f0 con 0x7f3aac071930 2024-04-25T10:12:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.610+0000 7f3aaaffd640 1 --2- 172.21.15.16:0/2966679322 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a9806b4d0 0x7f3a9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.611+0000 7f3aaaffd640 1 --2- 172.21.15.16:0/2966679322 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a9806b4d0 0x7f3a9806d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4007fd0 tx=0x7f3aa4007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.614+0000 7f3a93fff640 1 -- 172.21.15.16:0/2966679322 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3aa007a170 con 0x7f3aac071930 2024-04-25T10:12:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:51.838+0000 7f3ab181f640 1 -- 172.21.15.16:0/2966679322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3aac0ae500 con 0x7f3aac071930 2024-04-25T10:12:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.622+0000 7f3a93fff640 1 -- 172.21.15.16:0/2966679322 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3aa0079a70 con 0x7f3aac071930 2024-04-25T10:12:52.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.626+0000 7f3ab181f640 1 -- 172.21.15.16:0/2966679322 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a9806b4d0 msgr2=0x7f3a9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.626+0000 7f3ab181f640 1 --2- 172.21.15.16:0/2966679322 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a9806b4d0 0x7f3a9806d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4007fd0 tx=0x7f3aa4007de0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.626+0000 7f3ab181f640 1 -- 172.21.15.16:0/2966679322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aac071930 msgr2=0x7f3aac14e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.626+0000 7f3ab181f640 1 --2- 172.21.15.16:0/2966679322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aac071930 0x7f3aac14e410 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0035040 tx=0x7f3aa00046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.626+0000 7f3ab181f640 1 -- 172.21.15.16:0/2966679322 shutdown_connections 2024-04-25T10:12:52.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.626+0000 7f3ab181f640 1 --2- 172.21.15.16:0/2966679322 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a9806b4d0 0x7f3a9806d990 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.626+0000 7f3ab181f640 1 --2- 172.21.15.16:0/2966679322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3aac15a260 0x7f3aac0b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.626+0000 7f3ab181f640 1 --2- 172.21.15.16:0/2966679322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aac071930 0x7f3aac14e410 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.626+0000 7f3ab181f640 1 --2- 172.21.15.16:0/2966679322 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3aac06ea30 0x7f3aac14a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.626+0000 7f3ab181f640 1 -- 172.21.15.16:0/2966679322 >> 172.21.15.16:0/2966679322 conn(0x7f3aac06aba0 msgr2=0x7f3aac06d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:52.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.626+0000 7f3ab181f640 1 -- 172.21.15.16:0/2966679322 shutdown_connections 2024-04-25T10:12:52.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.626+0000 7f3ab181f640 1 -- 172.21.15.16:0/2966679322 wait complete. 2024-04-25T10:12:52.645 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:12:52.645 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.751+0000 7f1f7c42c640 1 Processor -- start 2024-04-25T10:12:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.751+0000 7f1f7c42c640 1 -- start start 2024-04-25T10:12:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.752+0000 7f1f7c42c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1f740b5b60 0x7f1f741687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.752+0000 7f1f7c42c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7415e0c0 con 0x7f1f7415dce0 2024-04-25T10:12:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.752+0000 7f1f7c42c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f740d2510 con 0x7f1f740d35b0 2024-04-25T10:12:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.752+0000 7f1f7c42c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f740b7b10 con 0x7f1f740b5b60 2024-04-25T10:12:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.752+0000 7f1f7a1c8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1f7415dce0 0x7f1f740d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35812/0 (socket says 172.21.15.16:35812) 2024-04-25T10:12:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.752+0000 7f1f7a1c8640 1 -- 172.21.15.16:0/2407464330 learned_addr learned my addr 172.21.15.16:0/2407464330 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.752+0000 7f1f7a9c9640 1 --2- 172.21.15.16:0/2407464330 >> v2:172.21.15.186:3300/0 conn(0x7f1f740b5b60 0x7f1f741687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.752+0000 7f1f791c6640 1 -- 172.21.15.16:0/2407464330 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518739317 0 0) 0x7f1f7415e0c0 con 0x7f1f7415dce0 2024-04-25T10:12:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.752+0000 7f1f791c6640 1 -- 172.21.15.16:0/2407464330 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f60003520 con 0x7f1f7415dce0 2024-04-25T10:12:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.752+0000 7f1f791c6640 1 -- 172.21.15.16:0/2407464330 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2639744634 0 0) 0x7f1f60003520 con 0x7f1f7415dce0 2024-04-25T10:12:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.752+0000 7f1f791c6640 1 -- 172.21.15.16:0/2407464330 >> v2:172.21.15.186:3300/0 conn(0x7f1f740b5b60 msgr2=0x7f1f741687f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.752+0000 7f1f791c6640 1 --2- 172.21.15.16:0/2407464330 >> v2:172.21.15.186:3300/0 conn(0x7f1f740b5b60 0x7f1f741687f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.752+0000 7f1f791c6640 1 -- 172.21.15.16:0/2407464330 >> v1:172.21.15.186:6789/0 conn(0x7f1f740d35b0 legacy=0x7f1f740b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.752+0000 7f1f791c6640 1 -- 172.21.15.16:0/2407464330 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f740d27f0 con 0x7f1f7415dce0 2024-04-25T10:12:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.753+0000 7f1f791c6640 1 -- 172.21.15.16:0/2407464330 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f1f70001670 con 0x7f1f7415dce0 2024-04-25T10:12:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.753+0000 7f1f7a9c9640 1 --2- 172.21.15.16:0/2407464330 >> v2:172.21.15.186:3300/0 conn(0x7f1f740b5b60 0x7f1f741687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.753+0000 7f1f791c6640 1 -- 172.21.15.16:0/2407464330 >> v1:172.21.15.16:6789/0 conn(0x7f1f7415dce0 legacy=0x7f1f740d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.753+0000 7f1f791c6640 1 --2- 172.21.15.16:0/2407464330 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f600039b0 0x7f1f60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.753+0000 7f1f791c6640 1 --2- 172.21.15.16:0/2407464330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f60004800 0x7f1f60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.753+0000 7f1f791c6640 1 --2- 172.21.15.16:0/2407464330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f600051c0 0x7f1f6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.753+0000 7f1f791c6640 1 -- 172.21.15.16:0/2407464330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740b7b10 con 0x7f1f600051c0 2024-04-25T10:12:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.753+0000 7f1f791c6640 1 -- 172.21.15.16:0/2407464330 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6000c060 con 0x7f1f60004800 2024-04-25T10:12:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.753+0000 7f1f791c6640 1 -- 172.21.15.16:0/2407464330 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6000c260 con 0x7f1f600039b0 2024-04-25T10:12:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.753+0000 7f1f7a9c9640 1 --2- 172.21.15.16:0/2407464330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f600051c0 0x7f1f6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.753+0000 7f1f7a1c8640 1 --2- 172.21.15.16:0/2407464330 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f600039b0 0x7f1f60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.753+0000 7f1f799c7640 1 --2- 172.21.15.16:0/2407464330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f60004800 0x7f1f60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.753+0000 7f1f7a9c9640 1 -- 172.21.15.16:0/2407464330 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f600039b0 msgr2=0x7f1f60003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.753+0000 7f1f7a9c9640 1 --2- 172.21.15.16:0/2407464330 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f600039b0 0x7f1f60003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.753+0000 7f1f7a9c9640 1 -- 172.21.15.16:0/2407464330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f60004800 msgr2=0x7f1f60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.753+0000 7f1f7a9c9640 1 --2- 172.21.15.16:0/2407464330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f60004800 0x7f1f60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.753+0000 7f1f7a9c9640 1 -- 172.21.15.16:0/2407464330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f60003520 con 0x7f1f600051c0 2024-04-25T10:12:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.754+0000 7f1f7a9c9640 1 -- 172.21.15.16:0/2407464330 reap_dead start 2024-04-25T10:12:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.754+0000 7f1f7a9c9640 1 --2- 172.21.15.16:0/2407464330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f600051c0 0x7f1f6000b9c0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c00e350 tx=0x7f1f60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1bdccf200f25c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.754+0000 7f1f791c6640 1 -- 172.21.15.16:0/2407464330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c009de0 con 0x7f1f600051c0 2024-04-25T10:12:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.754+0000 7f1f791c6640 1 -- 172.21.15.16:0/2407464330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c003bd0 con 0x7f1f600051c0 2024-04-25T10:12:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.754+0000 7f1f791c6640 1 -- 172.21.15.16:0/2407464330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c004020 con 0x7f1f600051c0 2024-04-25T10:12:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.754+0000 7f1f7c42c640 1 -- 172.21.15.16:0/2407464330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f600051c0 msgr2=0x7f1f6000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.754+0000 7f1f7c42c640 1 --2- 172.21.15.16:0/2407464330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f600051c0 0x7f1f6000b9c0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c00e350 tx=0x7f1f60005830 comp rx=0 tx=0).stop 2024-04-25T10:12:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.755+0000 7f1f7c42c640 1 -- 172.21.15.16:0/2407464330 shutdown_connections 2024-04-25T10:12:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.755+0000 7f1f7c42c640 1 --2- 172.21.15.16:0/2407464330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f600051c0 0x7f1f6000b9c0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.755+0000 7f1f7c42c640 1 -- 172.21.15.16:0/2407464330 >> 172.21.15.16:0/2407464330 conn(0x7f1f74072900 msgr2=0x7f1f740d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.755+0000 7f1f7c42c640 1 -- 172.21.15.16:0/2407464330 shutdown_connections 2024-04-25T10:12:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.755+0000 7f1f7c42c640 1 -- 172.21.15.16:0/2407464330 wait complete. 2024-04-25T10:12:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.756+0000 7f1f7c42c640 1 Processor -- start 2024-04-25T10:12:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.756+0000 7f1f7c42c640 1 -- start start 2024-04-25T10:12:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f7c42c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f600051c0 0x7f1f740c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f7c42c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f74160f30 0x7f1f740cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f7c42c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f7415e2b0 0x7f1f7415e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f7c42c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740d3090 con 0x7f1f74160f30 2024-04-25T10:12:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f7c42c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740d2db0 con 0x7f1f600051c0 2024-04-25T10:12:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f7c42c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740b9820 con 0x7f1f7415e2b0 2024-04-25T10:12:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f799c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f74160f30 0x7f1f740cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f799c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f74160f30 0x7f1f740cbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60632/0 (socket says 172.21.15.16:60632) 2024-04-25T10:12:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f799c7640 1 -- 172.21.15.16:0/2596453482 learned_addr learned my addr 172.21.15.16:0/2596453482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f7a9c9640 1 --2- 172.21.15.16:0/2596453482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f7415e2b0 0x7f1f7415e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f7a1c8640 1 --2- 172.21.15.16:0/2596453482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f600051c0 0x7f1f740c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f799c7640 1 -- 172.21.15.16:0/2596453482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f7415e2b0 msgr2=0x7f1f7415e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f799c7640 1 --2- 172.21.15.16:0/2596453482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f7415e2b0 0x7f1f7415e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f799c7640 1 -- 172.21.15.16:0/2596453482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f600051c0 msgr2=0x7f1f740c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f799c7640 1 --2- 172.21.15.16:0/2596453482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f600051c0 0x7f1f740c9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f799c7640 1 -- 172.21.15.16:0/2596453482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f740cc510 con 0x7f1f74160f30 2024-04-25T10:12:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f799c7640 1 --2- 172.21.15.16:0/2596453482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f74160f30 0x7f1f740cbc10 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400f5f0 tx=0x7f1f6400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f6affd640 1 -- 172.21.15.16:0/2596453482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f640052e0 con 0x7f1f74160f30 2024-04-25T10:12:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f6affd640 1 -- 172.21.15.16:0/2596453482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f64018070 con 0x7f1f74160f30 2024-04-25T10:12:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.757+0000 7f1f7c42c640 1 -- 172.21.15.16:0/2596453482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f740cc7f0 con 0x7f1f74160f30 2024-04-25T10:12:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.758+0000 7f1f6affd640 1 -- 172.21.15.16:0/2596453482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f64003830 con 0x7f1f74160f30 2024-04-25T10:12:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.758+0000 7f1f7c42c640 1 -- 172.21.15.16:0/2596453482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f740cc150 con 0x7f1f74160f30 2024-04-25T10:12:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.759+0000 7f1f7c42c640 1 -- 172.21.15.16:0/2596453482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f740749b0 con 0x7f1f74160f30 2024-04-25T10:12:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.759+0000 7f1f6affd640 1 -- 172.21.15.16:0/2596453482 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f640039d0 con 0x7f1f74160f30 2024-04-25T10:12:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.759+0000 7f1f6affd640 1 --2- 172.21.15.16:0/2596453482 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1f5806b4d0 0x7f1f5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.760+0000 7f1f6affd640 1 -- 172.21.15.16:0/2596453482 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f64089f70 con 0x7f1f74160f30 2024-04-25T10:12:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.760+0000 7f1f7a1c8640 1 --2- 172.21.15.16:0/2596453482 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1f5806b4d0 0x7f1f5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.760+0000 7f1f7a1c8640 1 --2- 172.21.15.16:0/2596453482 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1f5806b4d0 0x7f1f5806d990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1f70001050 tx=0x7f1f70009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.764+0000 7f1f6affd640 1 -- 172.21.15.16:0/2596453482 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f64058df0 con 0x7f1f74160f30 2024-04-25T10:12:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.985+0000 7f1f7c42c640 1 -- 172.21.15.16:0/2596453482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f7415a570 con 0x7f1f74160f30 2024-04-25T10:12:52.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.985+0000 7f1f6affd640 1 -- 172.21.15.16:0/2596453482 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f1f640586f0 con 0x7f1f74160f30 2024-04-25T10:12:52.990 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:52.990 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7243,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/109406141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":109406141},{"type":"v1","addr":"172.21.15.186:6837","nonce":109406141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2704401503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2704401503},{"type":"v1","addr":"172.21.15.16:6835","nonce":2704401503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/939221717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":939221717},{"type":"v1","addr":"172.21.15.16:6834","nonce":939221717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:48.958938+0000","modified":"2024-04-25T10:12:51.839702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8202},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8202":{"gid":8202,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6836/1371188714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1371188714},{"type":"v1","addr":"172.21.15.186:6836","nonce":1371188714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:52.990 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 71 2024-04-25T10:12:52.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.987+0000 7f1f7c42c640 1 -- 172.21.15.16:0/2596453482 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1f5806b4d0 msgr2=0x7f1f5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.987+0000 7f1f7c42c640 1 --2- 172.21.15.16:0/2596453482 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1f5806b4d0 0x7f1f5806d990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1f70001050 tx=0x7f1f70009040 comp rx=0 tx=0).stop 2024-04-25T10:12:52.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.987+0000 7f1f7c42c640 1 -- 172.21.15.16:0/2596453482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f74160f30 msgr2=0x7f1f740cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.987+0000 7f1f7c42c640 1 --2- 172.21.15.16:0/2596453482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f74160f30 0x7f1f740cbc10 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400f5f0 tx=0x7f1f6400fab0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.987+0000 7f1f7c42c640 1 -- 172.21.15.16:0/2596453482 shutdown_connections 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.987+0000 7f1f7c42c640 1 --2- 172.21.15.16:0/2596453482 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1f5806b4d0 0x7f1f5806d990 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.987+0000 7f1f7c42c640 1 --2- 172.21.15.16:0/2596453482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f7415e2b0 0x7f1f7415e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.987+0000 7f1f7c42c640 1 --2- 172.21.15.16:0/2596453482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f74160f30 0x7f1f740cbc10 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.987+0000 7f1f7c42c640 1 --2- 172.21.15.16:0/2596453482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f600051c0 0x7f1f740c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.987+0000 7f1f7c42c640 1 -- 172.21.15.16:0/2596453482 >> 172.21.15.16:0/2596453482 conn(0x7f1f74072900 msgr2=0x7f1f740b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.987+0000 7f1f7c42c640 1 -- 172.21.15.16:0/2596453482 shutdown_connections 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:52.987+0000 7f1f7c42c640 1 -- 172.21.15.16:0/2596453482 wait complete. 2024-04-25T10:12:53.003 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:53.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.100+0000 7ff87a14c640 1 Processor -- start 2024-04-25T10:12:53.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.100+0000 7ff87a14c640 1 -- start start 2024-04-25T10:12:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.100+0000 7ff87a14c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87409a150 0x7ff87406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.100+0000 7ff87a14c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87406c680 con 0x7ff87415ba10 2024-04-25T10:12:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.100+0000 7ff87a14c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87406c900 con 0x7ff87406c1f0 2024-04-25T10:12:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.100+0000 7ff87a14c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740cbe50 con 0x7ff87409a150 2024-04-25T10:12:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff8737fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff87415ba10 0x7ff874099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35822/0 (socket says 172.21.15.16:35822) 2024-04-25T10:12:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff8737fe640 1 -- 172.21.15.16:0/348803108 learned_addr learned my addr 172.21.15.16:0/348803108 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff872ffd640 1 --2- 172.21.15.16:0/348803108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87409a150 0x7ff87406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff8727fc640 1 -- 172.21.15.16:0/348803108 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2418956887 0 0) 0x7ff864001110 con 0x7ff87415ba10 2024-04-25T10:12:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff8727fc640 1 -- 172.21.15.16:0/348803108 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff85c003520 con 0x7ff87415ba10 2024-04-25T10:12:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff8727fc640 1 -- 172.21.15.16:0/348803108 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 286564750 0 0) 0x7ff85c003520 con 0x7ff87415ba10 2024-04-25T10:12:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff8727fc640 1 -- 172.21.15.16:0/348803108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87409a150 msgr2=0x7ff87406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff8727fc640 1 --2- 172.21.15.16:0/348803108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87409a150 0x7ff87406bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff8727fc640 1 -- 172.21.15.16:0/348803108 >> v1:172.21.15.186:6789/0 conn(0x7ff87406c1f0 legacy=0x7ff8740cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff8727fc640 1 -- 172.21.15.16:0/348803108 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87406cbe0 con 0x7ff87415ba10 2024-04-25T10:12:53.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff8727fc640 1 -- 172.21.15.16:0/348803108 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7ff864001840 con 0x7ff87415ba10 2024-04-25T10:12:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff8727fc640 1 -- 172.21.15.16:0/348803108 >> v1:172.21.15.16:6789/0 conn(0x7ff87415ba10 legacy=0x7ff874099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff8727fc640 1 --2- 172.21.15.16:0/348803108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff85c0039e0 0x7ff85c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff8727fc640 1 --2- 172.21.15.16:0/348803108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff85c004390 0x7ff85c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff8727fc640 1 --2- 172.21.15.16:0/348803108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87409a150 0x7ff85c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff8727fc640 1 -- 172.21.15.16:0/348803108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c00b9c0 con 0x7ff85c004390 2024-04-25T10:12:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff8727fc640 1 -- 172.21.15.16:0/348803108 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c00bb40 con 0x7ff85c0039e0 2024-04-25T10:12:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff8727fc640 1 -- 172.21.15.16:0/348803108 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c00bd40 con 0x7ff87409a150 2024-04-25T10:12:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.101+0000 7ff872ffd640 1 --2- 172.21.15.16:0/348803108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff85c004390 0x7ff85c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.102+0000 7ff8737fe640 1 --2- 172.21.15.16:0/348803108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff85c0039e0 0x7ff85c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.102+0000 7ff873fff640 1 --2- 172.21.15.16:0/348803108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87409a150 0x7ff85c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.102+0000 7ff872ffd640 1 -- 172.21.15.16:0/348803108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87409a150 msgr2=0x7ff85c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.102+0000 7ff872ffd640 1 --2- 172.21.15.16:0/348803108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87409a150 0x7ff85c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.102+0000 7ff872ffd640 1 -- 172.21.15.16:0/348803108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff85c0039e0 msgr2=0x7ff85c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.102+0000 7ff872ffd640 1 --2- 172.21.15.16:0/348803108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff85c0039e0 0x7ff85c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.102+0000 7ff872ffd640 1 -- 172.21.15.16:0/348803108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85c003520 con 0x7ff85c004390 2024-04-25T10:12:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.102+0000 7ff873fff640 1 --2- 172.21.15.16:0/348803108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff87409a150 0x7ff85c0050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.102+0000 7ff872ffd640 1 --2- 172.21.15.16:0/348803108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff85c004390 0x7ff85c004860 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7ff868009ac0 tx=0x7ff85c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2b86f0f4a62a7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.103+0000 7ff8727fc640 1 -- 172.21.15.16:0/348803108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868036070 con 0x7ff85c004390 2024-04-25T10:12:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.103+0000 7ff8727fc640 1 -- 172.21.15.16:0/348803108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff868031400 con 0x7ff85c004390 2024-04-25T10:12:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.103+0000 7ff8727fc640 1 -- 172.21.15.16:0/348803108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868009690 con 0x7ff85c004390 2024-04-25T10:12:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.103+0000 7ff87a14c640 1 -- 172.21.15.16:0/348803108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff85c004390 msgr2=0x7ff85c004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.103+0000 7ff87a14c640 1 --2- 172.21.15.16:0/348803108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff85c004390 0x7ff85c004860 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7ff868009ac0 tx=0x7ff85c005830 comp rx=0 tx=0).stop 2024-04-25T10:12:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.103+0000 7ff873fff640 1 -- 172.21.15.16:0/348803108 reap_dead start 2024-04-25T10:12:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.103+0000 7ff87a14c640 1 -- 172.21.15.16:0/348803108 shutdown_connections 2024-04-25T10:12:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.103+0000 7ff87a14c640 1 -- 172.21.15.16:0/348803108 >> 172.21.15.16:0/348803108 conn(0x7ff8740b32f0 msgr2=0x7ff8740b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.103+0000 7ff87a14c640 1 -- 172.21.15.16:0/348803108 shutdown_connections 2024-04-25T10:12:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.103+0000 7ff87a14c640 1 -- 172.21.15.16:0/348803108 wait complete. 2024-04-25T10:12:53.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.104+0000 7ff87a14c640 1 Processor -- start 2024-04-25T10:12:53.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.105+0000 7ff87a14c640 1 -- start start 2024-04-25T10:12:53.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.105+0000 7ff87a14c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff85c004390 0x7ff8741f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.105+0000 7ff87a14c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8741f3e10 0x7ff8740c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.105+0000 7ff87a14c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8740c78a0 0x7ff8740704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.105+0000 7ff87a14c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740d1320 con 0x7ff8740c78a0 2024-04-25T10:12:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.105+0000 7ff87a14c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740cfe80 con 0x7ff85c004390 2024-04-25T10:12:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.105+0000 7ff87a14c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740d14a0 con 0x7ff8741f3e10 2024-04-25T10:12:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.105+0000 7ff873fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8740c78a0 0x7ff8740704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.105+0000 7ff873fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8740c78a0 0x7ff8740704b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60646/0 (socket says 172.21.15.16:60646) 2024-04-25T10:12:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.105+0000 7ff873fff640 1 -- 172.21.15.16:0/5413962 learned_addr learned my addr 172.21.15.16:0/5413962 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.106+0000 7ff8737fe640 1 --2- 172.21.15.16:0/5413962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff85c004390 0x7ff8741f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.106+0000 7ff872ffd640 1 --2- 172.21.15.16:0/5413962 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8741f3e10 0x7ff8740c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.106+0000 7ff873fff640 1 -- 172.21.15.16:0/5413962 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8741f3e10 msgr2=0x7ff8740c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.106+0000 7ff873fff640 1 --2- 172.21.15.16:0/5413962 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8741f3e10 0x7ff8740c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.106+0000 7ff873fff640 1 -- 172.21.15.16:0/5413962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff85c004390 msgr2=0x7ff8741f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.106+0000 7ff873fff640 1 --2- 172.21.15.16:0/5413962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff85c004390 0x7ff8741f5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.106+0000 7ff873fff640 1 -- 172.21.15.16:0/5413962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8740c81e0 con 0x7ff8740c78a0 2024-04-25T10:12:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.106+0000 7ff873fff640 1 --2- 172.21.15.16:0/5413962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8740c78a0 0x7ff8740704b0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7ff8740685f0 tx=0x7ff86c00ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.106+0000 7ff85bfff640 1 -- 172.21.15.16:0/5413962 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c019070 con 0x7ff8740c78a0 2024-04-25T10:12:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.106+0000 7ff85bfff640 1 -- 172.21.15.16:0/5413962 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c006b50 con 0x7ff8740c78a0 2024-04-25T10:12:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.106+0000 7ff87a14c640 1 -- 172.21.15.16:0/5413962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8741f4290 con 0x7ff8740c78a0 2024-04-25T10:12:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.106+0000 7ff85bfff640 1 -- 172.21.15.16:0/5413962 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c003970 con 0x7ff8740c78a0 2024-04-25T10:12:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.106+0000 7ff87a14c640 1 -- 172.21.15.16:0/5413962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8741f4550 con 0x7ff8740c78a0 2024-04-25T10:12:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.107+0000 7ff87a14c640 1 -- 172.21.15.16:0/5413962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8740749b0 con 0x7ff8740c78a0 2024-04-25T10:12:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.108+0000 7ff85bfff640 1 -- 172.21.15.16:0/5413962 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff86c01c070 con 0x7ff8740c78a0 2024-04-25T10:12:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.108+0000 7ff85bfff640 1 --2- 172.21.15.16:0/5413962 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff86006b4d0 0x7ff86006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.108+0000 7ff85bfff640 1 -- 172.21.15.16:0/5413962 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff86c0895a0 con 0x7ff8740c78a0 2024-04-25T10:12:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.108+0000 7ff8737fe640 1 --2- 172.21.15.16:0/5413962 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff86006b4d0 0x7ff86006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.109+0000 7ff8737fe640 1 --2- 172.21.15.16:0/5413962 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff86006b4d0 0x7ff86006d990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff8640012b0 tx=0x7ff864009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.112+0000 7ff85bfff640 1 -- 172.21.15.16:0/5413962 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff86c058420 con 0x7ff8740c78a0 2024-04-25T10:12:53.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.292+0000 7ff87a14c640 1 -- 172.21.15.16:0/5413962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff87409a570 con 0x7ff8740c78a0 2024-04-25T10:12:53.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.293+0000 7ff85bfff640 1 -- 172.21.15.16:0/5413962 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16610 (secure 0 0 0) 0x7ff86c057d20 con 0x7ff8740c78a0 2024-04-25T10:12:53.298 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:53.298 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":100,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:12:50.612438+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:47.551100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:47.939289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.295+0000 7ff87a14c640 1 -- 172.21.15.16:0/5413962 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff86006b4d0 msgr2=0x7ff86006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.295+0000 7ff87a14c640 1 --2- 172.21.15.16:0/5413962 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff86006b4d0 0x7ff86006d990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff8640012b0 tx=0x7ff864009040 comp rx=0 tx=0).stop 2024-04-25T10:12:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.295+0000 7ff87a14c640 1 -- 172.21.15.16:0/5413962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8740c78a0 msgr2=0x7ff8740704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.295+0000 7ff87a14c640 1 --2- 172.21.15.16:0/5413962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8740c78a0 0x7ff8740704b0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7ff8740685f0 tx=0x7ff86c00ebe0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.295+0000 7ff87a14c640 1 -- 172.21.15.16:0/5413962 shutdown_connections 2024-04-25T10:12:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.295+0000 7ff87a14c640 1 --2- 172.21.15.16:0/5413962 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff86006b4d0 0x7ff86006d990 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.295+0000 7ff87a14c640 1 --2- 172.21.15.16:0/5413962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8740c78a0 0x7ff8740704b0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.295+0000 7ff87a14c640 1 --2- 172.21.15.16:0/5413962 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8741f3e10 0x7ff8740c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.295+0000 7ff87a14c640 1 --2- 172.21.15.16:0/5413962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff85c004390 0x7ff8741f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.295+0000 7ff87a14c640 1 -- 172.21.15.16:0/5413962 >> 172.21.15.16:0/5413962 conn(0x7ff8740b32f0 msgr2=0x7ff874159630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.295+0000 7ff87a14c640 1 -- 172.21.15.16:0/5413962 shutdown_connections 2024-04-25T10:12:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.295+0000 7ff87a14c640 1 -- 172.21.15.16:0/5413962 wait complete. 2024-04-25T10:12:53.311 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:12:53.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:12:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.408+0000 7f3e7181f640 1 Processor -- start 2024-04-25T10:12:53.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.408+0000 7f3e7181f640 1 -- start start 2024-04-25T10:12:53.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.409+0000 7f3e7181f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e6c15dce0 0x7f3e6c0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.409+0000 7f3e7181f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e6c15e0c0 con 0x7f3e6c0b5b60 2024-04-25T10:12:53.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.409+0000 7f3e7181f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e6c0d2510 con 0x7f3e6c0d35b0 2024-04-25T10:12:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.409+0000 7f3e7181f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c0d27f0 con 0x7f3e6c15dce0 2024-04-25T10:12:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.409+0000 7f3e6b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3e6c0b5b60 0x7f3e6c0b6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35838/0 (socket says 172.21.15.16:35838) 2024-04-25T10:12:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.409+0000 7f3e6b7fe640 1 -- 172.21.15.16:0/2328816830 learned_addr learned my addr 172.21.15.16:0/2328816830 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.409+0000 7f3e6affd640 1 --2- 172.21.15.16:0/2328816830 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e6c15dce0 0x7f3e6c0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.409+0000 7f3e69ffb640 1 -- 172.21.15.16:0/2328816830 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695453339 0 0) 0x7f3e6c15e0c0 con 0x7f3e6c0b5b60 2024-04-25T10:12:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.409+0000 7f3e69ffb640 1 -- 172.21.15.16:0/2328816830 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e58003520 con 0x7f3e6c0b5b60 2024-04-25T10:12:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.409+0000 7f3e69ffb640 1 -- 172.21.15.16:0/2328816830 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1861057923 0 0) 0x7f3e58003520 con 0x7f3e6c0b5b60 2024-04-25T10:12:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e69ffb640 1 -- 172.21.15.16:0/2328816830 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e6c15dce0 msgr2=0x7f3e6c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e69ffb640 1 --2- 172.21.15.16:0/2328816830 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e6c15dce0 0x7f3e6c0d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e69ffb640 1 -- 172.21.15.16:0/2328816830 >> v1:172.21.15.186:6789/0 conn(0x7f3e6c0d35b0 legacy=0x7f3e6c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e69ffb640 1 -- 172.21.15.16:0/2328816830 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c0d2ad0 con 0x7f3e6c0b5b60 2024-04-25T10:12:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e69ffb640 1 -- 172.21.15.16:0/2328816830 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f3e5c0019f0 con 0x7f3e6c0b5b60 2024-04-25T10:12:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e69ffb640 1 -- 172.21.15.16:0/2328816830 >> v1:172.21.15.16:6789/0 conn(0x7f3e6c0b5b60 legacy=0x7f3e6c0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e69ffb640 1 --2- 172.21.15.16:0/2328816830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e580039e0 0x7f3e58003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e69ffb640 1 --2- 172.21.15.16:0/2328816830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e58004390 0x7f3e58004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e69ffb640 1 --2- 172.21.15.16:0/2328816830 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e6c15dce0 0x7f3e580050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e69ffb640 1 -- 172.21.15.16:0/2328816830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c0d27f0 con 0x7f3e580039e0 2024-04-25T10:12:53.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e69ffb640 1 -- 172.21.15.16:0/2328816830 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5800b9c0 con 0x7f3e58004390 2024-04-25T10:12:53.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e69ffb640 1 -- 172.21.15.16:0/2328816830 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5800bbc0 con 0x7f3e6c15dce0 2024-04-25T10:12:53.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e6affd640 1 --2- 172.21.15.16:0/2328816830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e580039e0 0x7f3e58003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e6a7fc640 1 --2- 172.21.15.16:0/2328816830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e58004390 0x7f3e58004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e6b7fe640 1 --2- 172.21.15.16:0/2328816830 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e6c15dce0 0x7f3e580050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e6affd640 1 -- 172.21.15.16:0/2328816830 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e6c15dce0 msgr2=0x7f3e580050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:53.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e6affd640 1 --2- 172.21.15.16:0/2328816830 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e6c15dce0 0x7f3e580050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e6affd640 1 -- 172.21.15.16:0/2328816830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e58004390 msgr2=0x7f3e58004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:53.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e6affd640 1 --2- 172.21.15.16:0/2328816830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e58004390 0x7f3e58004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e6affd640 1 -- 172.21.15.16:0/2328816830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e58003520 con 0x7f3e580039e0 2024-04-25T10:12:53.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.410+0000 7f3e6a7fc640 1 --2- 172.21.15.16:0/2328816830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e58004390 0x7f3e58004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:53.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.411+0000 7f3e6affd640 1 --2- 172.21.15.16:0/2328816830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e580039e0 0x7f3e58003dc0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009ac0 tx=0x7f3e58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9128bb9dcf61f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.411+0000 7f3e69ffb640 1 -- 172.21.15.16:0/2328816830 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64036070 con 0x7f3e580039e0 2024-04-25T10:12:53.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.411+0000 7f3e69ffb640 1 -- 172.21.15.16:0/2328816830 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e64031400 con 0x7f3e580039e0 2024-04-25T10:12:53.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.411+0000 7f3e69ffb640 1 -- 172.21.15.16:0/2328816830 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64009690 con 0x7f3e580039e0 2024-04-25T10:12:53.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.411+0000 7f3e7181f640 1 -- 172.21.15.16:0/2328816830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e580039e0 msgr2=0x7f3e58003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.411+0000 7f3e7181f640 1 --2- 172.21.15.16:0/2328816830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e580039e0 0x7f3e58003dc0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009ac0 tx=0x7f3e58005830 comp rx=0 tx=0).stop 2024-04-25T10:12:53.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.411+0000 7f3e6b7fe640 1 -- 172.21.15.16:0/2328816830 reap_dead start 2024-04-25T10:12:53.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.411+0000 7f3e7181f640 1 -- 172.21.15.16:0/2328816830 shutdown_connections 2024-04-25T10:12:53.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.411+0000 7f3e7181f640 1 -- 172.21.15.16:0/2328816830 >> 172.21.15.16:0/2328816830 conn(0x7f3e6c072900 msgr2=0x7f3e6c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:53.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.412+0000 7f3e7181f640 1 -- 172.21.15.16:0/2328816830 shutdown_connections 2024-04-25T10:12:53.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.412+0000 7f3e7181f640 1 -- 172.21.15.16:0/2328816830 wait complete. 2024-04-25T10:12:53.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.413+0000 7f3e7181f640 1 Processor -- start 2024-04-25T10:12:53.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.413+0000 7f3e7181f640 1 -- start start 2024-04-25T10:12:53.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e7181f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e580039e0 0x7f3e6c0aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e7181f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e6c15a000 0x7f3e6c0ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e7181f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e6c160b60 0x7f3e6c1f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e7181f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c0b9820 con 0x7f3e580039e0 2024-04-25T10:12:53.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e7181f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c0d3090 con 0x7f3e6c160b60 2024-04-25T10:12:53.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e7181f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c0b99a0 con 0x7f3e6c15a000 2024-04-25T10:12:53.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e6affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e580039e0 0x7f3e6c0aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e6affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e580039e0 0x7f3e6c0aacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60668/0 (socket says 172.21.15.16:60668) 2024-04-25T10:12:53.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e6affd640 1 -- 172.21.15.16:0/1404787776 learned_addr learned my addr 172.21.15.16:0/1404787776 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:53.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e6affd640 1 -- 172.21.15.16:0/1404787776 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e6c15a000 msgr2=0x7f3e6c0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e6affd640 1 --2- 172.21.15.16:0/1404787776 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e6c15a000 0x7f3e6c0ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e6affd640 1 -- 172.21.15.16:0/1404787776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e6c160b60 msgr2=0x7f3e6c1f38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e6affd640 1 --2- 172.21.15.16:0/1404787776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e6c160b60 0x7f3e6c1f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e6affd640 1 -- 172.21.15.16:0/1404787776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c15fe00 con 0x7f3e580039e0 2024-04-25T10:12:53.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e6affd640 1 --2- 172.21.15.16:0/1404787776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e580039e0 0x7f3e6c0aacc0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f3e64034040 tx=0x7f3e64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e63fff640 1 -- 172.21.15.16:0/1404787776 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64040020 con 0x7f3e580039e0 2024-04-25T10:12:53.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e63fff640 1 -- 172.21.15.16:0/1404787776 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e64003b40 con 0x7f3e580039e0 2024-04-25T10:12:53.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e7181f640 1 -- 172.21.15.16:0/1404787776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6c15a7e0 con 0x7f3e580039e0 2024-04-25T10:12:53.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e63fff640 1 -- 172.21.15.16:0/1404787776 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64036040 con 0x7f3e580039e0 2024-04-25T10:12:53.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.414+0000 7f3e7181f640 1 -- 172.21.15.16:0/1404787776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6c160380 con 0x7f3e580039e0 2024-04-25T10:12:53.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.415+0000 7f3e7181f640 1 -- 172.21.15.16:0/1404787776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e6c0749b0 con 0x7f3e580039e0 2024-04-25T10:12:53.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.416+0000 7f3e63fff640 1 -- 172.21.15.16:0/1404787776 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e64043020 con 0x7f3e580039e0 2024-04-25T10:12:53.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.416+0000 7f3e63fff640 1 --2- 172.21.15.16:0/1404787776 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3e4c06b500 0x7f3e4c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.416+0000 7f3e63fff640 1 -- 172.21.15.16:0/1404787776 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e640aa1b0 con 0x7f3e580039e0 2024-04-25T10:12:53.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.416+0000 7f3e6a7fc640 1 --2- 172.21.15.16:0/1404787776 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3e4c06b500 0x7f3e4c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.417+0000 7f3e6a7fc640 1 --2- 172.21.15.16:0/1404787776 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3e4c06b500 0x7f3e4c06d9c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3e54007ac0 tx=0x7f3e54009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.421+0000 7f3e63fff640 1 -- 172.21.15.16:0/1404787776 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3e6404a070 con 0x7f3e580039e0 2024-04-25T10:12:53.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.598+0000 7f3e7181f640 1 -- 172.21.15.16:0/1404787776 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e6c0b5460 con 0x7f3e4c06b500 2024-04-25T10:12:53.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.605+0000 7f3e63fff640 1 -- 172.21.15.16:0/1404787776 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+241830 (secure 0 0 0) 0x7f3e6c0b5460 con 0x7f3e4c06b500 2024-04-25T10:12:53.612 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:53.613 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:12:53.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.608+0000 7f3e7181f640 1 -- 172.21.15.16:0/1404787776 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3e4c06b500 msgr2=0x7f3e4c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.608+0000 7f3e7181f640 1 --2- 172.21.15.16:0/1404787776 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3e4c06b500 0x7f3e4c06d9c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3e54007ac0 tx=0x7f3e54009040 comp rx=0 tx=0).stop 2024-04-25T10:12:53.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.608+0000 7f3e7181f640 1 -- 172.21.15.16:0/1404787776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e580039e0 msgr2=0x7f3e6c0aacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.608+0000 7f3e7181f640 1 --2- 172.21.15.16:0/1404787776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e580039e0 0x7f3e6c0aacc0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f3e64034040 tx=0x7f3e64004730 comp rx=0 tx=0).stop 2024-04-25T10:12:53.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.608+0000 7f3e7181f640 1 -- 172.21.15.16:0/1404787776 shutdown_connections 2024-04-25T10:12:53.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.608+0000 7f3e7181f640 1 --2- 172.21.15.16:0/1404787776 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3e4c06b500 0x7f3e4c06d9c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.608+0000 7f3e7181f640 1 --2- 172.21.15.16:0/1404787776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e6c160b60 0x7f3e6c1f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.608+0000 7f3e7181f640 1 --2- 172.21.15.16:0/1404787776 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e6c15a000 0x7f3e6c0ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.608+0000 7f3e7181f640 1 --2- 172.21.15.16:0/1404787776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e580039e0 0x7f3e6c0aacc0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.608+0000 7f3e7181f640 1 -- 172.21.15.16:0/1404787776 >> 172.21.15.16:0/1404787776 conn(0x7f3e6c072900 msgr2=0x7f3e6c0b4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:53.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.608+0000 7f3e7181f640 1 -- 172.21.15.16:0/1404787776 shutdown_connections 2024-04-25T10:12:53.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.608+0000 7f3e7181f640 1 -- 172.21.15.16:0/1404787776 wait complete. 2024-04-25T10:12:53.623 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-04-25T10:12:51.894598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590402,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":162,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":84,"acting":84,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":272336,"kb_used_data":10400,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754702384,"statfs":{"total":773094113280,"available":772815241216,"internally_reserved":0,"allocated":10649600,"data_stored":5218668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.303625"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.632133+0000","last_change":"2024-04-25T10:12:49.628211+0000","last_active":"2024-04-25T10:12:49.632133+0000","last_peered":"2024-04-25T10:12:49.632133+0000","last_clean":"2024-04-25T10:12:49.632133+0000","last_became_active":"2024-04-25T10:12:49.627922+0000","last_became_peered":"2024-04-25T10:12:49.627922+0000","last_unstale":"2024-04-25T10:12:49.632133+0000","last_undegraded":"2024-04-25T10:12:49.632133+0000","last_fullsized":"2024-04-25T10:12:49.632133+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:12.892619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.637443+0000","last_change":"2024-04-25T10:12:49.633994+0000","last_active":"2024-04-25T10:12:49.637443+0000","last_peered":"2024-04-25T10:12:49.637443+0000","last_clean":"2024-04-25T10:12:49.637443+0000","last_became_active":"2024-04-25T10:12:49.633734+0000","last_became_peered":"2024-04-25T10:12:49.633734+0000","last_unstale":"2024-04-25T10:12:49.637443+0000","last_undegraded":"2024-04-25T10:12:49.637443+0000","last_fullsized":"2024-04-25T10:12:49.637443+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:59.164107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.637321+0000","last_change":"2024-04-25T10:12:49.632860+0000","last_active":"2024-04-25T10:12:49.637321+0000","last_peered":"2024-04-25T10:12:49.637321+0000","last_clean":"2024-04-25T10:12:49.637321+0000","last_became_active":"2024-04-25T10:12:49.632364+0000","last_became_peered":"2024-04-25T10:12:49.632364+0000","last_unstale":"2024-04-25T10:12:49.637321+0000","last_undegraded":"2024-04-25T10:12:49.637321+0000","last_fullsized":"2024-04-25T10:12:49.637321+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:02.903983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.597778+0000","last_change":"2024-04-25T10:12:47.597778+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.597778+0000","last_undegraded":"2024-04-25T10:12:47.597778+0000","last_fullsized":"2024-04-25T10:12:47.597778+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.596687+0000","last_change":"2024-04-25T10:12:47.596687+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.596687+0000","last_undegraded":"2024-04-25T10:12:47.596687+0000","last_fullsized":"2024-04-25T10:12:47.596687+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.592782+0000","last_change":"2024-04-25T10:12:47.592782+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.592782+0000","last_undegraded":"2024-04-25T10:12:47.592782+0000","last_fullsized":"2024-04-25T10:12:47.592782+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.595549+0000","last_change":"2024-04-25T10:12:47.595549+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.595549+0000","last_undegraded":"2024-04-25T10:12:47.595549+0000","last_fullsized":"2024-04-25T10:12:47.595549+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.597024+0000","last_change":"2024-04-25T10:12:47.597024+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.597024+0000","last_undegraded":"2024-04-25T10:12:47.597024+0000","last_fullsized":"2024-04-25T10:12:47.597024+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.592796+0000","last_change":"2024-04-25T10:12:47.592796+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.592796+0000","last_undegraded":"2024-04-25T10:12:47.592796+0000","last_fullsized":"2024-04-25T10:12:47.592796+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.596655+0000","last_change":"2024-04-25T10:12:47.596655+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.596655+0000","last_undegraded":"2024-04-25T10:12:47.596655+0000","last_fullsized":"2024-04-25T10:12:47.596655+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.616583+0000","last_change":"2024-04-25T10:12:48.617160+0000","last_active":"2024-04-25T10:12:49.616583+0000","last_peered":"2024-04-25T10:12:49.616583+0000","last_clean":"2024-04-25T10:12:49.616583+0000","last_became_active":"2024-04-25T10:12:48.616865+0000","last_became_peered":"2024-04-25T10:12:48.616865+0000","last_unstale":"2024-04-25T10:12:49.616583+0000","last_undegraded":"2024-04-25T10:12:49.616583+0000","last_fullsized":"2024-04-25T10:12:49.616583+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:36.175763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.592965+0000","last_change":"2024-04-25T10:12:47.592965+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.592965+0000","last_undegraded":"2024-04-25T10:12:47.592965+0000","last_fullsized":"2024-04-25T10:12:47.592965+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.596070+0000","last_change":"2024-04-25T10:12:47.596070+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.596070+0000","last_undegraded":"2024-04-25T10:12:47.596070+0000","last_fullsized":"2024-04-25T10:12:47.596070+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.597182+0000","last_change":"2024-04-25T10:12:47.597182+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.597182+0000","last_undegraded":"2024-04-25T10:12:47.597182+0000","last_fullsized":"2024-04-25T10:12:47.597182+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.592383+0000","last_change":"2024-04-25T10:12:47.592383+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.592383+0000","last_undegraded":"2024-04-25T10:12:47.592383+0000","last_fullsized":"2024-04-25T10:12:47.592383+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"99'2","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.632486+0000","last_change":"2024-04-25T10:12:48.612576+0000","last_active":"2024-04-25T10:12:49.632486+0000","last_peered":"2024-04-25T10:12:49.632486+0000","last_clean":"2024-04-25T10:12:49.632486+0000","last_became_active":"2024-04-25T10:12:48.612301+0000","last_became_peered":"2024-04-25T10:12:48.612301+0000","last_unstale":"2024-04-25T10:12:49.632486+0000","last_undegraded":"2024-04-25T10:12:49.632486+0000","last_fullsized":"2024-04-25T10:12:49.632486+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:41.978908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.596477+0000","last_change":"2024-04-25T10:12:47.596477+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.596477+0000","last_undegraded":"2024-04-25T10:12:47.596477+0000","last_fullsized":"2024-04-25T10:12:47.596477+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.592401+0000","last_change":"2024-04-25T10:12:47.592401+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.592401+0000","last_undegraded":"2024-04-25T10:12:47.592401+0000","last_fullsized":"2024-04-25T10:12:47.592401+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.596428+0000","last_change":"2024-04-25T10:12:47.596428+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.596428+0000","last_undegraded":"2024-04-25T10:12:47.596428+0000","last_fullsized":"2024-04-25T10:12:47.596428+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.592385+0000","last_change":"2024-04-25T10:12:47.592385+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.592385+0000","last_undegraded":"2024-04-25T10:12:47.592385+0000","last_fullsized":"2024-04-25T10:12:47.592385+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.596719+0000","last_change":"2024-04-25T10:12:47.596719+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.596719+0000","last_undegraded":"2024-04-25T10:12:47.596719+0000","last_fullsized":"2024-04-25T10:12:47.596719+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.616144+0000","last_change":"2024-04-25T10:12:48.612122+0000","last_active":"2024-04-25T10:12:49.616144+0000","last_peered":"2024-04-25T10:12:49.616144+0000","last_clean":"2024-04-25T10:12:49.616144+0000","last_became_active":"2024-04-25T10:12:48.611790+0000","last_became_peered":"2024-04-25T10:12:48.611790+0000","last_unstale":"2024-04-25T10:12:49.616144+0000","last_undegraded":"2024-04-25T10:12:49.616144+0000","last_fullsized":"2024-04-25T10:12:49.616144+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:06.583645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.630435+0000","last_change":"2024-04-25T10:12:49.625164+0000","last_active":"2024-04-25T10:12:49.630435+0000","last_peered":"2024-04-25T10:12:49.630435+0000","last_clean":"2024-04-25T10:12:49.630435+0000","last_became_active":"2024-04-25T10:12:49.624850+0000","last_became_peered":"2024-04-25T10:12:49.624850+0000","last_unstale":"2024-04-25T10:12:49.630435+0000","last_undegraded":"2024-04-25T10:12:49.630435+0000","last_fullsized":"2024-04-25T10:12:49.630435+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:11.948655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.596426+0000","last_change":"2024-04-25T10:12:47.596426+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.596426+0000","last_undegraded":"2024-04-25T10:12:47.596426+0000","last_fullsized":"2024-04-25T10:12:47.596426+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.637441+0000","last_change":"2024-04-25T10:12:49.633660+0000","last_active":"2024-04-25T10:12:49.637441+0000","last_peered":"2024-04-25T10:12:49.637441+0000","last_clean":"2024-04-25T10:12:49.637441+0000","last_became_active":"2024-04-25T10:12:49.633389+0000","last_became_peered":"2024-04-25T10:12:49.633389+0000","last_unstale":"2024-04-25T10:12:49.637441+0000","last_undegraded":"2024-04-25T10:12:49.637441+0000","last_fullsized":"2024-04-25T10:12:49.637441+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:12.735103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.594853+0000","last_change":"2024-04-25T10:12:47.594853+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.594853+0000","last_undegraded":"2024-04-25T10:12:47.594853+0000","last_fullsized":"2024-04-25T10:12:47.594853+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.593475+0000","last_change":"2024-04-25T10:12:47.593475+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.593475+0000","last_undegraded":"2024-04-25T10:12:47.593475+0000","last_fullsized":"2024-04-25T10:12:47.593475+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.593358+0000","last_change":"2024-04-25T10:12:47.593358+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.593358+0000","last_undegraded":"2024-04-25T10:12:47.593358+0000","last_fullsized":"2024-04-25T10:12:47.593358+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.631690+0000","last_change":"2024-04-25T10:12:49.626564+0000","last_active":"2024-04-25T10:12:49.631690+0000","last_peered":"2024-04-25T10:12:49.631690+0000","last_clean":"2024-04-25T10:12:49.631690+0000","last_became_active":"2024-04-25T10:12:49.626253+0000","last_became_peered":"2024-04-25T10:12:49.626253+0000","last_unstale":"2024-04-25T10:12:49.631690+0000","last_undegraded":"2024-04-25T10:12:49.631690+0000","last_fullsized":"2024-04-25T10:12:49.631690+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:23.187839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.592383+0000","last_change":"2024-04-25T10:12:47.592383+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.592383+0000","last_undegraded":"2024-04-25T10:12:47.592383+0000","last_fullsized":"2024-04-25T10:12:47.592383+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.632576+0000","last_change":"2024-04-25T10:12:49.629290+0000","last_active":"2024-04-25T10:12:49.632576+0000","last_peered":"2024-04-25T10:12:49.632576+0000","last_clean":"2024-04-25T10:12:49.632576+0000","last_became_active":"2024-04-25T10:12:49.628982+0000","last_became_peered":"2024-04-25T10:12:49.628982+0000","last_unstale":"2024-04-25T10:12:49.632576+0000","last_undegraded":"2024-04-25T10:12:49.632576+0000","last_fullsized":"2024-04-25T10:12:49.632576+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:14.913579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.613170+0000","last_change":"2024-04-25T10:12:48.617283+0000","last_active":"2024-04-25T10:12:49.613170+0000","last_peered":"2024-04-25T10:12:49.613170+0000","last_clean":"2024-04-25T10:12:49.613170+0000","last_became_active":"2024-04-25T10:12:48.616924+0000","last_became_peered":"2024-04-25T10:12:48.616924+0000","last_unstale":"2024-04-25T10:12:49.613170+0000","last_undegraded":"2024-04-25T10:12:49.613170+0000","last_fullsized":"2024-04-25T10:12:49.613170+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:11.626114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.612338+0000","last_change":"2024-04-25T10:12:48.611644+0000","last_active":"2024-04-25T10:12:49.612338+0000","last_peered":"2024-04-25T10:12:49.612338+0000","last_clean":"2024-04-25T10:12:49.612338+0000","last_became_active":"2024-04-25T10:12:48.611350+0000","last_became_peered":"2024-04-25T10:12:48.611350+0000","last_unstale":"2024-04-25T10:12:49.612338+0000","last_undegraded":"2024-04-25T10:12:49.612338+0000","last_fullsized":"2024-04-25T10:12:49.612338+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:34.540070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.594370+0000","last_change":"2024-04-25T10:12:47.594370+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.594370+0000","last_undegraded":"2024-04-25T10:12:47.594370+0000","last_fullsized":"2024-04-25T10:12:47.594370+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.628320+0000","last_change":"2024-04-25T10:12:49.624411+0000","last_active":"2024-04-25T10:12:49.628320+0000","last_peered":"2024-04-25T10:12:49.628320+0000","last_clean":"2024-04-25T10:12:49.628320+0000","last_became_active":"2024-04-25T10:12:49.623581+0000","last_became_peered":"2024-04-25T10:12:49.623581+0000","last_unstale":"2024-04-25T10:12:49.628320+0000","last_undegraded":"2024-04-25T10:12:49.628320+0000","last_fullsized":"2024-04-25T10:12:49.628320+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:47.880397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.592773+0000","last_change":"2024-04-25T10:12:47.592773+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.592773+0000","last_undegraded":"2024-04-25T10:12:47.592773+0000","last_fullsized":"2024-04-25T10:12:47.592773+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.594377+0000","last_change":"2024-04-25T10:12:47.594377+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.594377+0000","last_undegraded":"2024-04-25T10:12:47.594377+0000","last_fullsized":"2024-04-25T10:12:47.594377+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.625166+0000","last_change":"2024-04-25T10:12:49.620038+0000","last_active":"2024-04-25T10:12:49.625166+0000","last_peered":"2024-04-25T10:12:49.625166+0000","last_clean":"2024-04-25T10:12:49.625166+0000","last_became_active":"2024-04-25T10:12:49.619771+0000","last_became_peered":"2024-04-25T10:12:49.619771+0000","last_unstale":"2024-04-25T10:12:49.625166+0000","last_undegraded":"2024-04-25T10:12:49.625166+0000","last_fullsized":"2024-04-25T10:12:49.625166+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:44.781605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.597943+0000","last_change":"2024-04-25T10:12:47.597943+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.597943+0000","last_undegraded":"2024-04-25T10:12:47.597943+0000","last_fullsized":"2024-04-25T10:12:47.597943+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.592782+0000","last_change":"2024-04-25T10:12:47.592782+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.592782+0000","last_undegraded":"2024-04-25T10:12:47.592782+0000","last_fullsized":"2024-04-25T10:12:47.592782+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:12:48.590133+0000","last_change":"2024-04-25T10:12:48.590133+0000","last_active":"2024-04-25T10:12:48.590133+0000","last_peered":"2024-04-25T10:12:48.590133+0000","last_clean":"2024-04-25T10:12:48.590133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:48.590133+0000","last_undegraded":"2024-04-25T10:12:48.590133+0000","last_fullsized":"2024-04-25T10:12:48.590133+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-25T10:12:47.592893+0000","last_change":"2024-04-25T10:12:47.592893+0000","last_active":"2024-04-25T10:12:47.585530+0000","last_peered":"2024-04-25T10:12:47.585530+0000","last_clean":"2024-04-25T10:12:47.585530+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:47.592893+0000","last_undegraded":"2024-04-25T10:12:47.592893+0000","last_fullsized":"2024-04-25T10:12:47.592893+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":323,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:12:47.593300+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:12:47.593300+0000","last_peered":"2024-04-25T10:12:47.593300+0000","last_clean":"2024-04-25T10:12:47.593300+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:12:47.593300+0000","last_undegraded":"2024-04-25T10:12:47.593300+0000","last_fullsized":"2024-04-25T10:12:47.593300+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":1},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509537,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":503230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509537,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":503230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509538,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1179648,"data_stored":503230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509538,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36496,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335344,"statfs":{"total":96636764160,"available":96599392256,"internally_reserved":0,"allocated":1179648,"data_stored":503230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36492,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335348,"statfs":{"total":96636764160,"available":96599396352,"internally_reserved":0,"allocated":1179648,"data_stored":503230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509537,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37072,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334768,"statfs":{"total":96636764160,"available":96598802432,"internally_reserved":0,"allocated":1773568,"data_stored":1093598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":503230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574946,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32936,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338904,"statfs":{"total":96636764160,"available":96603037696,"internally_reserved":0,"allocated":1798144,"data_stored":1105690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:12:53.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:12:53.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.766+0000 7f5d2436a640 1 Processor -- start 2024-04-25T10:12:53.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.766+0000 7f5d2436a640 1 -- start start 2024-04-25T10:12:53.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.767+0000 7f5d2436a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d1c15dce0 0x7f5d1c0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.767+0000 7f5d2436a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d1c0b5b60 0x7f5d1c0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.767+0000 7f5d2436a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d1c15e0c0 con 0x7f5d1c0d35b0 2024-04-25T10:12:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.767+0000 7f5d2436a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d1c0d2510 con 0x7f5d1c15dce0 2024-04-25T10:12:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.767+0000 7f5d2436a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c0d27f0 con 0x7f5d1c0b5b60 2024-04-25T10:12:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.768+0000 7f5d22106640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d1c15dce0 0x7f5d1c0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.768+0000 7f5d22106640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d1c15dce0 0x7f5d1c0d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60676/0 (socket says 172.21.15.16:60676) 2024-04-25T10:12:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.768+0000 7f5d22106640 1 -- 172.21.15.16:0/3121660622 learned_addr learned my addr 172.21.15.16:0/3121660622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.768+0000 7f5d22907640 1 --2- 172.21.15.16:0/3121660622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d1c0b5b60 0x7f5d1c0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.768+0000 7f5d22106640 1 -- 172.21.15.16:0/3121660622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d1c0b5b60 msgr2=0x7f5d1c0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.768+0000 7f5d22106640 1 --2- 172.21.15.16:0/3121660622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d1c0b5b60 0x7f5d1c0b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.768+0000 7f5d22106640 1 -- 172.21.15.16:0/3121660622 >> v1:172.21.15.186:6789/0 conn(0x7f5d1c0d35b0 legacy=0x7f5d1c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.768+0000 7f5d22106640 1 -- 172.21.15.16:0/3121660622 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1c0d2ad0 con 0x7f5d1c15dce0 2024-04-25T10:12:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.768+0000 7f5d22907640 1 --2- 172.21.15.16:0/3121660622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d1c0b5b60 0x7f5d1c0b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:53.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.769+0000 7f5d22106640 1 --2- 172.21.15.16:0/3121660622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1c15dce0 0x7f5d1c0d1a50 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009870 tx=0x7f5d1402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4c43f254d37986 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.769+0000 7f5d21104640 1 -- 172.21.15.16:0/3121660622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14037070 con 0x7f5d1c15dce0 2024-04-25T10:12:53.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.769+0000 7f5d21104640 1 -- 172.21.15.16:0/3121660622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d14032440 con 0x7f5d1c15dce0 2024-04-25T10:12:53.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.769+0000 7f5d21104640 1 -- 172.21.15.16:0/3121660622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14032810 con 0x7f5d1c15dce0 2024-04-25T10:12:53.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.769+0000 7f5d2436a640 1 -- 172.21.15.16:0/3121660622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1c15dce0 msgr2=0x7f5d1c0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.769+0000 7f5d2436a640 1 --2- 172.21.15.16:0/3121660622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1c15dce0 0x7f5d1c0d1a50 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009870 tx=0x7f5d1402a210 comp rx=0 tx=0).stop 2024-04-25T10:12:53.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.769+0000 7f5d2436a640 1 -- 172.21.15.16:0/3121660622 shutdown_connections 2024-04-25T10:12:53.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.769+0000 7f5d2436a640 1 --2- 172.21.15.16:0/3121660622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d1c0b5b60 0x7f5d1c0b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.769+0000 7f5d2436a640 1 --2- 172.21.15.16:0/3121660622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1c15dce0 0x7f5d1c0d1a50 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.769+0000 7f5d2436a640 1 -- 172.21.15.16:0/3121660622 >> 172.21.15.16:0/3121660622 conn(0x7f5d1c072900 msgr2=0x7f5d1c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.769+0000 7f5d2436a640 1 -- 172.21.15.16:0/3121660622 shutdown_connections 2024-04-25T10:12:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.769+0000 7f5d2436a640 1 -- 172.21.15.16:0/3121660622 wait complete. 2024-04-25T10:12:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.770+0000 7f5d2436a640 1 Processor -- start 2024-04-25T10:12:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.770+0000 7f5d2436a640 1 -- start start 2024-04-25T10:12:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.771+0000 7f5d2436a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1c0b5b60 0x7f5d1c14c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.771+0000 7f5d2436a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d1c0d35b0 0x7f5d1c15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.771+0000 7f5d2436a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d1c15dce0 0x7f5d1c0b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.771+0000 7f5d2436a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c0b9820 con 0x7f5d1c0b5b60 2024-04-25T10:12:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.771+0000 7f5d2436a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c0d3090 con 0x7f5d1c15dce0 2024-04-25T10:12:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.771+0000 7f5d2436a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c0b99a0 con 0x7f5d1c0d35b0 2024-04-25T10:12:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.771+0000 7f5d22106640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1c0b5b60 0x7f5d1c14c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.771+0000 7f5d22106640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1c0b5b60 0x7f5d1c14c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60678/0 (socket says 172.21.15.16:60678) 2024-04-25T10:12:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.771+0000 7f5d22106640 1 -- 172.21.15.16:0/1676369923 learned_addr learned my addr 172.21.15.16:0/1676369923 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.771+0000 7f5d22907640 1 --2- 172.21.15.16:0/1676369923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d1c15dce0 0x7f5d1c0b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.771+0000 7f5d21905640 1 --2- 172.21.15.16:0/1676369923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d1c0d35b0 0x7f5d1c15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.772+0000 7f5d22106640 1 -- 172.21.15.16:0/1676369923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d1c0d35b0 msgr2=0x7f5d1c15e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.772+0000 7f5d22106640 1 --2- 172.21.15.16:0/1676369923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d1c0d35b0 0x7f5d1c15e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.772+0000 7f5d22106640 1 -- 172.21.15.16:0/1676369923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d1c15dce0 msgr2=0x7f5d1c0b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.772+0000 7f5d22106640 1 --2- 172.21.15.16:0/1676369923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d1c15dce0 0x7f5d1c0b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.772+0000 7f5d22106640 1 -- 172.21.15.16:0/1676369923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1c14cef0 con 0x7f5d1c0b5b60 2024-04-25T10:12:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.772+0000 7f5d22106640 1 --2- 172.21.15.16:0/1676369923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1c0b5b60 0x7f5d1c14c9b0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009870 tx=0x7f5d140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.772+0000 7f5d1affd640 1 -- 172.21.15.16:0/1676369923 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14037040 con 0x7f5d1c0b5b60 2024-04-25T10:12:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.772+0000 7f5d1affd640 1 -- 172.21.15.16:0/1676369923 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d14042070 con 0x7f5d1c0b5b60 2024-04-25T10:12:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.772+0000 7f5d2436a640 1 -- 172.21.15.16:0/1676369923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1c14a240 con 0x7f5d1c0b5b60 2024-04-25T10:12:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.772+0000 7f5d1affd640 1 -- 172.21.15.16:0/1676369923 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14030bb0 con 0x7f5d1c0b5b60 2024-04-25T10:12:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.772+0000 7f5d2436a640 1 -- 172.21.15.16:0/1676369923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d1c14aae0 con 0x7f5d1c0b5b60 2024-04-25T10:12:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.773+0000 7f5d2436a640 1 -- 172.21.15.16:0/1676369923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d1c14a630 con 0x7f5d1c0b5b60 2024-04-25T10:12:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.773+0000 7f5d1affd640 1 -- 172.21.15.16:0/1676369923 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d14047460 con 0x7f5d1c0b5b60 2024-04-25T10:12:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.774+0000 7f5d1affd640 1 --2- 172.21.15.16:0/1676369923 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5d0806b4d0 0x7f5d0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.774+0000 7f5d1affd640 1 -- 172.21.15.16:0/1676369923 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d140aae20 con 0x7f5d1c0b5b60 2024-04-25T10:12:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.774+0000 7f5d21905640 1 --2- 172.21.15.16:0/1676369923 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5d0806b4d0 0x7f5d0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.775+0000 7f5d21905640 1 --2- 172.21.15.16:0/1676369923 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5d0806b4d0 0x7f5d0806d990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0c9c40 tx=0x7f5d10009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.778+0000 7f5d1affd640 1 -- 172.21.15.16:0/1676369923 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d1c14cef0 con 0x7f5d1c0b5b60 2024-04-25T10:12:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.954+0000 7f5d2436a640 1 -- 172.21.15.16:0/1676369923 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5d1c0b5790 con 0x7f5d0806b4d0 2024-04-25T10:12:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.961+0000 7f5d1affd640 1 -- 172.21.15.16:0/1676369923 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252156 (secure 0 0 0) 0x7f5d1c0b5790 con 0x7f5d0806b4d0 2024-04-25T10:12:53.968 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:53.969 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:12:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.964+0000 7f5d2436a640 1 -- 172.21.15.16:0/1676369923 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5d0806b4d0 msgr2=0x7f5d0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.964+0000 7f5d2436a640 1 --2- 172.21.15.16:0/1676369923 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5d0806b4d0 0x7f5d0806d990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0c9c40 tx=0x7f5d10009040 comp rx=0 tx=0).stop 2024-04-25T10:12:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.964+0000 7f5d2436a640 1 -- 172.21.15.16:0/1676369923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1c0b5b60 msgr2=0x7f5d1c14c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.964+0000 7f5d2436a640 1 --2- 172.21.15.16:0/1676369923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1c0b5b60 0x7f5d1c14c9b0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009870 tx=0x7f5d140046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.964+0000 7f5d2436a640 1 -- 172.21.15.16:0/1676369923 shutdown_connections 2024-04-25T10:12:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.964+0000 7f5d2436a640 1 --2- 172.21.15.16:0/1676369923 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5d0806b4d0 0x7f5d0806d990 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.964+0000 7f5d2436a640 1 --2- 172.21.15.16:0/1676369923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d1c15dce0 0x7f5d1c0b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.964+0000 7f5d2436a640 1 --2- 172.21.15.16:0/1676369923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d1c0d35b0 0x7f5d1c15e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.964+0000 7f5d2436a640 1 --2- 172.21.15.16:0/1676369923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1c0b5b60 0x7f5d1c14c9b0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.964+0000 7f5d2436a640 1 -- 172.21.15.16:0/1676369923 >> 172.21.15.16:0/1676369923 conn(0x7f5d1c072900 msgr2=0x7f5d1c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.964+0000 7f5d2436a640 1 -- 172.21.15.16:0/1676369923 shutdown_connections 2024-04-25T10:12:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:53.964+0000 7f5d2436a640 1 -- 172.21.15.16:0/1676369923 wait complete. 2024-04-25T10:12:53.979 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-04-25T10:12:53.895079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":272756,"kb_used_data":10780,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754701964,"statfs":{"total":773094113280,"available":772814811136,"internally_reserved":0,"allocated":11038720,"data_stored":5397989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001303"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.795674+0000","last_change":"2024-04-25T10:12:49.632848+0000","last_active":"2024-04-25T10:12:50.795674+0000","last_peered":"2024-04-25T10:12:50.795674+0000","last_clean":"2024-04-25T10:12:50.795674+0000","last_became_active":"2024-04-25T10:12:49.632208+0000","last_became_peered":"2024-04-25T10:12:49.632208+0000","last_unstale":"2024-04-25T10:12:50.795674+0000","last_undegraded":"2024-04-25T10:12:50.795674+0000","last_fullsized":"2024-04-25T10:12:50.795674+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:56.681174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.795668+0000","last_change":"2024-04-25T10:12:49.621487+0000","last_active":"2024-04-25T10:12:50.795668+0000","last_peered":"2024-04-25T10:12:50.795668+0000","last_clean":"2024-04-25T10:12:50.795668+0000","last_became_active":"2024-04-25T10:12:49.620997+0000","last_became_peered":"2024-04-25T10:12:49.620997+0000","last_unstale":"2024-04-25T10:12:50.795668+0000","last_undegraded":"2024-04-25T10:12:50.795668+0000","last_fullsized":"2024-04-25T10:12:50.795668+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:43.987984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.619328+0000","last_change":"2024-04-25T10:12:49.616457+0000","last_active":"2024-04-25T10:12:50.619328+0000","last_peered":"2024-04-25T10:12:50.619328+0000","last_clean":"2024-04-25T10:12:50.619328+0000","last_became_active":"2024-04-25T10:12:49.616152+0000","last_became_peered":"2024-04-25T10:12:49.616152+0000","last_unstale":"2024-04-25T10:12:50.619328+0000","last_undegraded":"2024-04-25T10:12:50.619328+0000","last_fullsized":"2024-04-25T10:12:50.619328+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:04.680436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.795493+0000","last_change":"2024-04-25T10:12:49.627831+0000","last_active":"2024-04-25T10:12:50.795493+0000","last_peered":"2024-04-25T10:12:50.795493+0000","last_clean":"2024-04-25T10:12:50.795493+0000","last_became_active":"2024-04-25T10:12:49.627534+0000","last_became_peered":"2024-04-25T10:12:49.627534+0000","last_unstale":"2024-04-25T10:12:50.795493+0000","last_undegraded":"2024-04-25T10:12:50.795493+0000","last_fullsized":"2024-04-25T10:12:50.795493+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:55.852710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.851073+0000","last_change":"2024-04-25T10:12:49.616450+0000","last_active":"2024-04-25T10:12:50.851073+0000","last_peered":"2024-04-25T10:12:50.851073+0000","last_clean":"2024-04-25T10:12:50.851073+0000","last_became_active":"2024-04-25T10:12:49.616144+0000","last_became_peered":"2024-04-25T10:12:49.616144+0000","last_unstale":"2024-04-25T10:12:50.851073+0000","last_undegraded":"2024-04-25T10:12:50.851073+0000","last_fullsized":"2024-04-25T10:12:50.851073+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:17.654471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.850990+0000","last_change":"2024-04-25T10:12:49.620396+0000","last_active":"2024-04-25T10:12:50.850990+0000","last_peered":"2024-04-25T10:12:50.850990+0000","last_clean":"2024-04-25T10:12:50.850990+0000","last_became_active":"2024-04-25T10:12:49.620108+0000","last_became_peered":"2024-04-25T10:12:49.620108+0000","last_unstale":"2024-04-25T10:12:50.850990+0000","last_undegraded":"2024-04-25T10:12:50.850990+0000","last_fullsized":"2024-04-25T10:12:50.850990+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:34.246224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.850599+0000","last_change":"2024-04-25T10:12:49.617368+0000","last_active":"2024-04-25T10:12:50.850599+0000","last_peered":"2024-04-25T10:12:50.850599+0000","last_clean":"2024-04-25T10:12:50.850599+0000","last_became_active":"2024-04-25T10:12:49.617106+0000","last_became_peered":"2024-04-25T10:12:49.617106+0000","last_unstale":"2024-04-25T10:12:50.850599+0000","last_undegraded":"2024-04-25T10:12:50.850599+0000","last_fullsized":"2024-04-25T10:12:50.850599+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:14.967466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618701+0000","last_change":"2024-04-25T10:12:49.617348+0000","last_active":"2024-04-25T10:12:50.618701+0000","last_peered":"2024-04-25T10:12:50.618701+0000","last_clean":"2024-04-25T10:12:50.618701+0000","last_became_active":"2024-04-25T10:12:49.616851+0000","last_became_peered":"2024-04-25T10:12:49.616851+0000","last_unstale":"2024-04-25T10:12:50.618701+0000","last_undegraded":"2024-04-25T10:12:50.618701+0000","last_fullsized":"2024-04-25T10:12:50.618701+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:39.264666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.851016+0000","last_change":"2024-04-25T10:12:49.623253+0000","last_active":"2024-04-25T10:12:50.851016+0000","last_peered":"2024-04-25T10:12:50.851016+0000","last_clean":"2024-04-25T10:12:50.851016+0000","last_became_active":"2024-04-25T10:12:49.622953+0000","last_became_peered":"2024-04-25T10:12:49.622953+0000","last_unstale":"2024-04-25T10:12:50.851016+0000","last_undegraded":"2024-04-25T10:12:50.851016+0000","last_fullsized":"2024-04-25T10:12:50.851016+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:06.841187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.850716+0000","last_change":"2024-04-25T10:12:49.622833+0000","last_active":"2024-04-25T10:12:50.850716+0000","last_peered":"2024-04-25T10:12:50.850716+0000","last_clean":"2024-04-25T10:12:50.850716+0000","last_became_active":"2024-04-25T10:12:49.622473+0000","last_became_peered":"2024-04-25T10:12:49.622473+0000","last_unstale":"2024-04-25T10:12:50.850716+0000","last_undegraded":"2024-04-25T10:12:50.850716+0000","last_fullsized":"2024-04-25T10:12:50.850716+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:29.604622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.851117+0000","last_change":"2024-04-25T10:12:49.620970+0000","last_active":"2024-04-25T10:12:50.851117+0000","last_peered":"2024-04-25T10:12:50.851117+0000","last_clean":"2024-04-25T10:12:50.851117+0000","last_became_active":"2024-04-25T10:12:49.620639+0000","last_became_peered":"2024-04-25T10:12:49.620639+0000","last_unstale":"2024-04-25T10:12:50.851117+0000","last_undegraded":"2024-04-25T10:12:50.851117+0000","last_fullsized":"2024-04-25T10:12:50.851117+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:04.709673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618927+0000","last_change":"2024-04-25T10:12:49.632041+0000","last_active":"2024-04-25T10:12:50.618927+0000","last_peered":"2024-04-25T10:12:50.618927+0000","last_clean":"2024-04-25T10:12:50.618927+0000","last_became_active":"2024-04-25T10:12:49.631709+0000","last_became_peered":"2024-04-25T10:12:49.631709+0000","last_unstale":"2024-04-25T10:12:50.618927+0000","last_undegraded":"2024-04-25T10:12:50.618927+0000","last_fullsized":"2024-04-25T10:12:50.618927+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:01.080830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.851935+0000","last_change":"2024-04-25T10:12:49.627396+0000","last_active":"2024-04-25T10:12:50.851935+0000","last_peered":"2024-04-25T10:12:50.851935+0000","last_clean":"2024-04-25T10:12:50.851935+0000","last_became_active":"2024-04-25T10:12:49.627092+0000","last_became_peered":"2024-04-25T10:12:49.627092+0000","last_unstale":"2024-04-25T10:12:50.851935+0000","last_undegraded":"2024-04-25T10:12:50.851935+0000","last_fullsized":"2024-04-25T10:12:50.851935+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:21.332769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.795219+0000","last_change":"2024-04-25T10:12:49.623080+0000","last_active":"2024-04-25T10:12:50.795219+0000","last_peered":"2024-04-25T10:12:50.795219+0000","last_clean":"2024-04-25T10:12:50.795219+0000","last_became_active":"2024-04-25T10:12:49.622826+0000","last_became_peered":"2024-04-25T10:12:49.622826+0000","last_unstale":"2024-04-25T10:12:50.795219+0000","last_undegraded":"2024-04-25T10:12:50.795219+0000","last_fullsized":"2024-04-25T10:12:50.795219+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:11.302292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.795168+0000","last_change":"2024-04-25T10:12:49.625560+0000","last_active":"2024-04-25T10:12:50.795168+0000","last_peered":"2024-04-25T10:12:50.795168+0000","last_clean":"2024-04-25T10:12:50.795168+0000","last_became_active":"2024-04-25T10:12:49.625107+0000","last_became_peered":"2024-04-25T10:12:49.625107+0000","last_unstale":"2024-04-25T10:12:50.795168+0000","last_undegraded":"2024-04-25T10:12:50.795168+0000","last_fullsized":"2024-04-25T10:12:50.795168+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:05.807816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.850593+0000","last_change":"2024-04-25T10:12:49.625289+0000","last_active":"2024-04-25T10:12:50.850593+0000","last_peered":"2024-04-25T10:12:50.850593+0000","last_clean":"2024-04-25T10:12:50.850593+0000","last_became_active":"2024-04-25T10:12:49.625001+0000","last_became_peered":"2024-04-25T10:12:49.625001+0000","last_unstale":"2024-04-25T10:12:50.850593+0000","last_undegraded":"2024-04-25T10:12:50.850593+0000","last_fullsized":"2024-04-25T10:12:50.850593+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:17.095138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.632133+0000","last_change":"2024-04-25T10:12:49.628211+0000","last_active":"2024-04-25T10:12:49.632133+0000","last_peered":"2024-04-25T10:12:49.632133+0000","last_clean":"2024-04-25T10:12:49.632133+0000","last_became_active":"2024-04-25T10:12:49.627922+0000","last_became_peered":"2024-04-25T10:12:49.627922+0000","last_unstale":"2024-04-25T10:12:49.632133+0000","last_undegraded":"2024-04-25T10:12:49.632133+0000","last_fullsized":"2024-04-25T10:12:49.632133+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:12.892619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.795708+0000","last_change":"2024-04-25T10:12:49.624613+0000","last_active":"2024-04-25T10:12:50.795708+0000","last_peered":"2024-04-25T10:12:50.795708+0000","last_clean":"2024-04-25T10:12:50.795708+0000","last_became_active":"2024-04-25T10:12:49.624272+0000","last_became_peered":"2024-04-25T10:12:49.624272+0000","last_unstale":"2024-04-25T10:12:50.795708+0000","last_undegraded":"2024-04-25T10:12:50.795708+0000","last_fullsized":"2024-04-25T10:12:50.795708+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:07.868911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.637443+0000","last_change":"2024-04-25T10:12:49.633994+0000","last_active":"2024-04-25T10:12:49.637443+0000","last_peered":"2024-04-25T10:12:49.637443+0000","last_clean":"2024-04-25T10:12:49.637443+0000","last_became_active":"2024-04-25T10:12:49.633734+0000","last_became_peered":"2024-04-25T10:12:49.633734+0000","last_unstale":"2024-04-25T10:12:49.637443+0000","last_undegraded":"2024-04-25T10:12:49.637443+0000","last_fullsized":"2024-04-25T10:12:49.637443+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:59.164107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.619075+0000","last_change":"2024-04-25T10:12:49.624116+0000","last_active":"2024-04-25T10:12:50.619075+0000","last_peered":"2024-04-25T10:12:50.619075+0000","last_clean":"2024-04-25T10:12:50.619075+0000","last_became_active":"2024-04-25T10:12:49.623648+0000","last_became_peered":"2024-04-25T10:12:49.623648+0000","last_unstale":"2024-04-25T10:12:50.619075+0000","last_undegraded":"2024-04-25T10:12:50.619075+0000","last_fullsized":"2024-04-25T10:12:50.619075+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:06.269406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.619495+0000","last_change":"2024-04-25T10:12:49.616263+0000","last_active":"2024-04-25T10:12:50.619495+0000","last_peered":"2024-04-25T10:12:50.619495+0000","last_clean":"2024-04-25T10:12:50.619495+0000","last_became_active":"2024-04-25T10:12:49.615967+0000","last_became_peered":"2024-04-25T10:12:49.615967+0000","last_unstale":"2024-04-25T10:12:50.619495+0000","last_undegraded":"2024-04-25T10:12:50.619495+0000","last_fullsized":"2024-04-25T10:12:50.619495+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:29.604622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618915+0000","last_change":"2024-04-25T10:12:49.619588+0000","last_active":"2024-04-25T10:12:50.618915+0000","last_peered":"2024-04-25T10:12:50.618915+0000","last_clean":"2024-04-25T10:12:50.618915+0000","last_became_active":"2024-04-25T10:12:49.619314+0000","last_became_peered":"2024-04-25T10:12:49.619314+0000","last_unstale":"2024-04-25T10:12:50.618915+0000","last_undegraded":"2024-04-25T10:12:50.618915+0000","last_fullsized":"2024-04-25T10:12:50.618915+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:55.822790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.795254+0000","last_change":"2024-04-25T10:12:49.631074+0000","last_active":"2024-04-25T10:12:50.795254+0000","last_peered":"2024-04-25T10:12:50.795254+0000","last_clean":"2024-04-25T10:12:50.795254+0000","last_became_active":"2024-04-25T10:12:49.630817+0000","last_became_peered":"2024-04-25T10:12:49.630817+0000","last_unstale":"2024-04-25T10:12:50.795254+0000","last_undegraded":"2024-04-25T10:12:50.795254+0000","last_fullsized":"2024-04-25T10:12:50.795254+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:18.298533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618343+0000","last_change":"2024-04-25T10:12:49.626697+0000","last_active":"2024-04-25T10:12:50.618343+0000","last_peered":"2024-04-25T10:12:50.618343+0000","last_clean":"2024-04-25T10:12:50.618343+0000","last_became_active":"2024-04-25T10:12:49.626400+0000","last_became_peered":"2024-04-25T10:12:49.626400+0000","last_unstale":"2024-04-25T10:12:50.618343+0000","last_undegraded":"2024-04-25T10:12:50.618343+0000","last_fullsized":"2024-04-25T10:12:50.618343+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:45.297295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.851053+0000","last_change":"2024-04-25T10:12:49.627557+0000","last_active":"2024-04-25T10:12:50.851053+0000","last_peered":"2024-04-25T10:12:50.851053+0000","last_clean":"2024-04-25T10:12:50.851053+0000","last_became_active":"2024-04-25T10:12:49.627269+0000","last_became_peered":"2024-04-25T10:12:49.627269+0000","last_unstale":"2024-04-25T10:12:50.851053+0000","last_undegraded":"2024-04-25T10:12:50.851053+0000","last_fullsized":"2024-04-25T10:12:50.851053+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:32.840558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.851591+0000","last_change":"2024-04-25T10:12:49.626557+0000","last_active":"2024-04-25T10:12:50.851591+0000","last_peered":"2024-04-25T10:12:50.851591+0000","last_clean":"2024-04-25T10:12:50.851591+0000","last_became_active":"2024-04-25T10:12:49.626262+0000","last_became_peered":"2024-04-25T10:12:49.626262+0000","last_unstale":"2024-04-25T10:12:50.851591+0000","last_undegraded":"2024-04-25T10:12:50.851591+0000","last_fullsized":"2024-04-25T10:12:50.851591+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:37.853789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618366+0000","last_change":"2024-04-25T10:12:49.618116+0000","last_active":"2024-04-25T10:12:50.618366+0000","last_peered":"2024-04-25T10:12:50.618366+0000","last_clean":"2024-04-25T10:12:50.618366+0000","last_became_active":"2024-04-25T10:12:49.617824+0000","last_became_peered":"2024-04-25T10:12:49.617824+0000","last_unstale":"2024-04-25T10:12:50.618366+0000","last_undegraded":"2024-04-25T10:12:50.618366+0000","last_fullsized":"2024-04-25T10:12:50.618366+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:43.987984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.617980+0000","last_change":"2024-04-25T10:12:49.621478+0000","last_active":"2024-04-25T10:12:50.617980+0000","last_peered":"2024-04-25T10:12:50.617980+0000","last_clean":"2024-04-25T10:12:50.617980+0000","last_became_active":"2024-04-25T10:12:49.621193+0000","last_became_peered":"2024-04-25T10:12:49.621193+0000","last_unstale":"2024-04-25T10:12:50.617980+0000","last_undegraded":"2024-04-25T10:12:50.617980+0000","last_fullsized":"2024-04-25T10:12:50.617980+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:11.302292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.852082+0000","last_change":"2024-04-25T10:12:49.617457+0000","last_active":"2024-04-25T10:12:50.852082+0000","last_peered":"2024-04-25T10:12:50.852082+0000","last_clean":"2024-04-25T10:12:50.852082+0000","last_became_active":"2024-04-25T10:12:49.617206+0000","last_became_peered":"2024-04-25T10:12:49.617206+0000","last_unstale":"2024-04-25T10:12:50.852082+0000","last_undegraded":"2024-04-25T10:12:50.852082+0000","last_fullsized":"2024-04-25T10:12:50.852082+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:06.211796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.851980+0000","last_change":"2024-04-25T10:12:49.629203+0000","last_active":"2024-04-25T10:12:50.851980+0000","last_peered":"2024-04-25T10:12:50.851980+0000","last_clean":"2024-04-25T10:12:50.851980+0000","last_became_active":"2024-04-25T10:12:49.628934+0000","last_became_peered":"2024-04-25T10:12:49.628934+0000","last_unstale":"2024-04-25T10:12:50.851980+0000","last_undegraded":"2024-04-25T10:12:50.851980+0000","last_fullsized":"2024-04-25T10:12:50.851980+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:47.322902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618835+0000","last_change":"2024-04-25T10:12:49.615432+0000","last_active":"2024-04-25T10:12:50.618835+0000","last_peered":"2024-04-25T10:12:50.618835+0000","last_clean":"2024-04-25T10:12:50.618835+0000","last_became_active":"2024-04-25T10:12:49.615165+0000","last_became_peered":"2024-04-25T10:12:49.615165+0000","last_unstale":"2024-04-25T10:12:50.618835+0000","last_undegraded":"2024-04-25T10:12:50.618835+0000","last_fullsized":"2024-04-25T10:12:50.618835+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:26.329852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.795165+0000","last_change":"2024-04-25T10:12:49.628788+0000","last_active":"2024-04-25T10:12:50.795165+0000","last_peered":"2024-04-25T10:12:50.795165+0000","last_clean":"2024-04-25T10:12:50.795165+0000","last_became_active":"2024-04-25T10:12:49.628382+0000","last_became_peered":"2024-04-25T10:12:49.628382+0000","last_unstale":"2024-04-25T10:12:50.795165+0000","last_undegraded":"2024-04-25T10:12:50.795165+0000","last_fullsized":"2024-04-25T10:12:50.795165+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:03.593921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.637321+0000","last_change":"2024-04-25T10:12:49.632860+0000","last_active":"2024-04-25T10:12:49.637321+0000","last_peered":"2024-04-25T10:12:49.637321+0000","last_clean":"2024-04-25T10:12:49.637321+0000","last_became_active":"2024-04-25T10:12:49.632364+0000","last_became_peered":"2024-04-25T10:12:49.632364+0000","last_unstale":"2024-04-25T10:12:49.637321+0000","last_undegraded":"2024-04-25T10:12:49.637321+0000","last_fullsized":"2024-04-25T10:12:49.637321+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:02.903983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"99'4","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.852042+0000","last_change":"2024-04-25T10:12:48.614778+0000","last_active":"2024-04-25T10:12:50.852042+0000","last_peered":"2024-04-25T10:12:50.852042+0000","last_clean":"2024-04-25T10:12:50.852042+0000","last_became_active":"2024-04-25T10:12:48.614494+0000","last_became_peered":"2024-04-25T10:12:48.614494+0000","last_unstale":"2024-04-25T10:12:50.852042+0000","last_undegraded":"2024-04-25T10:12:50.852042+0000","last_fullsized":"2024-04-25T10:12:50.852042+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:25.367254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.851388+0000","last_change":"2024-04-25T10:12:49.618162+0000","last_active":"2024-04-25T10:12:50.851388+0000","last_peered":"2024-04-25T10:12:50.851388+0000","last_clean":"2024-04-25T10:12:50.851388+0000","last_became_active":"2024-04-25T10:12:49.617908+0000","last_became_peered":"2024-04-25T10:12:49.617908+0000","last_unstale":"2024-04-25T10:12:50.851388+0000","last_undegraded":"2024-04-25T10:12:50.851388+0000","last_fullsized":"2024-04-25T10:12:50.851388+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:07.010972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.850795+0000","last_change":"2024-04-25T10:12:48.610189+0000","last_active":"2024-04-25T10:12:50.850795+0000","last_peered":"2024-04-25T10:12:50.850795+0000","last_clean":"2024-04-25T10:12:50.850795+0000","last_became_active":"2024-04-25T10:12:48.609913+0000","last_became_peered":"2024-04-25T10:12:48.609913+0000","last_unstale":"2024-04-25T10:12:50.850795+0000","last_undegraded":"2024-04-25T10:12:50.850795+0000","last_fullsized":"2024-04-25T10:12:50.850795+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:00.685723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.795626+0000","last_change":"2024-04-25T10:12:49.622084+0000","last_active":"2024-04-25T10:12:50.795626+0000","last_peered":"2024-04-25T10:12:50.795626+0000","last_clean":"2024-04-25T10:12:50.795626+0000","last_became_active":"2024-04-25T10:12:49.621826+0000","last_became_peered":"2024-04-25T10:12:49.621826+0000","last_unstale":"2024-04-25T10:12:50.795626+0000","last_undegraded":"2024-04-25T10:12:50.795626+0000","last_fullsized":"2024-04-25T10:12:50.795626+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:45.297295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.619385+0000","last_change":"2024-04-25T10:12:48.610699+0000","last_active":"2024-04-25T10:12:50.619385+0000","last_peered":"2024-04-25T10:12:50.619385+0000","last_clean":"2024-04-25T10:12:50.619385+0000","last_became_active":"2024-04-25T10:12:48.610313+0000","last_became_peered":"2024-04-25T10:12:48.610313+0000","last_unstale":"2024-04-25T10:12:50.619385+0000","last_undegraded":"2024-04-25T10:12:50.619385+0000","last_fullsized":"2024-04-25T10:12:50.619385+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:00.076227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.850828+0000","last_change":"2024-04-25T10:12:49.618224+0000","last_active":"2024-04-25T10:12:50.850828+0000","last_peered":"2024-04-25T10:12:50.850828+0000","last_clean":"2024-04-25T10:12:50.850828+0000","last_became_active":"2024-04-25T10:12:49.617954+0000","last_became_peered":"2024-04-25T10:12:49.617954+0000","last_unstale":"2024-04-25T10:12:50.850828+0000","last_undegraded":"2024-04-25T10:12:50.850828+0000","last_fullsized":"2024-04-25T10:12:50.850828+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:10.035341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.795566+0000","last_change":"2024-04-25T10:12:48.617347+0000","last_active":"2024-04-25T10:12:50.795566+0000","last_peered":"2024-04-25T10:12:50.795566+0000","last_clean":"2024-04-25T10:12:50.795566+0000","last_became_active":"2024-04-25T10:12:48.617060+0000","last_became_peered":"2024-04-25T10:12:48.617060+0000","last_unstale":"2024-04-25T10:12:50.795566+0000","last_undegraded":"2024-04-25T10:12:50.795566+0000","last_fullsized":"2024-04-25T10:12:50.795566+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:36.061592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618400+0000","last_change":"2024-04-25T10:12:49.620587+0000","last_active":"2024-04-25T10:12:50.618400+0000","last_peered":"2024-04-25T10:12:50.618400+0000","last_clean":"2024-04-25T10:12:50.618400+0000","last_became_active":"2024-04-25T10:12:49.620299+0000","last_became_peered":"2024-04-25T10:12:49.620299+0000","last_unstale":"2024-04-25T10:12:50.618400+0000","last_undegraded":"2024-04-25T10:12:50.618400+0000","last_fullsized":"2024-04-25T10:12:50.618400+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:23.048635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.795106+0000","last_change":"2024-04-25T10:12:48.612021+0000","last_active":"2024-04-25T10:12:50.795106+0000","last_peered":"2024-04-25T10:12:50.795106+0000","last_clean":"2024-04-25T10:12:50.795106+0000","last_became_active":"2024-04-25T10:12:48.611730+0000","last_became_peered":"2024-04-25T10:12:48.611730+0000","last_unstale":"2024-04-25T10:12:50.795106+0000","last_undegraded":"2024-04-25T10:12:50.795106+0000","last_fullsized":"2024-04-25T10:12:50.795106+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:31.962802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.851175+0000","last_change":"2024-04-25T10:12:49.619926+0000","last_active":"2024-04-25T10:12:50.851175+0000","last_peered":"2024-04-25T10:12:50.851175+0000","last_clean":"2024-04-25T10:12:50.851175+0000","last_became_active":"2024-04-25T10:12:49.619589+0000","last_became_peered":"2024-04-25T10:12:49.619589+0000","last_unstale":"2024-04-25T10:12:50.851175+0000","last_undegraded":"2024-04-25T10:12:50.851175+0000","last_fullsized":"2024-04-25T10:12:50.851175+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:58.932421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618942+0000","last_change":"2024-04-25T10:12:48.606927+0000","last_active":"2024-04-25T10:12:50.618942+0000","last_peered":"2024-04-25T10:12:50.618942+0000","last_clean":"2024-04-25T10:12:50.618942+0000","last_became_active":"2024-04-25T10:12:48.606625+0000","last_became_peered":"2024-04-25T10:12:48.606625+0000","last_unstale":"2024-04-25T10:12:50.618942+0000","last_undegraded":"2024-04-25T10:12:50.618942+0000","last_fullsized":"2024-04-25T10:12:50.618942+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:02.591929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.851853+0000","last_change":"2024-04-25T10:12:49.618851+0000","last_active":"2024-04-25T10:12:50.851853+0000","last_peered":"2024-04-25T10:12:50.851853+0000","last_clean":"2024-04-25T10:12:50.851853+0000","last_became_active":"2024-04-25T10:12:49.618591+0000","last_became_peered":"2024-04-25T10:12:49.618591+0000","last_unstale":"2024-04-25T10:12:50.851853+0000","last_undegraded":"2024-04-25T10:12:50.851853+0000","last_fullsized":"2024-04-25T10:12:50.851853+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:43.475165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.850796+0000","last_change":"2024-04-25T10:12:48.608956+0000","last_active":"2024-04-25T10:12:50.850796+0000","last_peered":"2024-04-25T10:12:50.850796+0000","last_clean":"2024-04-25T10:12:50.850796+0000","last_became_active":"2024-04-25T10:12:48.608636+0000","last_became_peered":"2024-04-25T10:12:48.608636+0000","last_unstale":"2024-04-25T10:12:50.850796+0000","last_undegraded":"2024-04-25T10:12:50.850796+0000","last_fullsized":"2024-04-25T10:12:50.850796+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:26.536731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618261+0000","last_change":"2024-04-25T10:12:49.619603+0000","last_active":"2024-04-25T10:12:50.618261+0000","last_peered":"2024-04-25T10:12:50.618261+0000","last_clean":"2024-04-25T10:12:50.618261+0000","last_became_active":"2024-04-25T10:12:49.619289+0000","last_became_peered":"2024-04-25T10:12:49.619289+0000","last_unstale":"2024-04-25T10:12:50.618261+0000","last_undegraded":"2024-04-25T10:12:50.618261+0000","last_fullsized":"2024-04-25T10:12:50.618261+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:19.343496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.616583+0000","last_change":"2024-04-25T10:12:48.617160+0000","last_active":"2024-04-25T10:12:49.616583+0000","last_peered":"2024-04-25T10:12:49.616583+0000","last_clean":"2024-04-25T10:12:49.616583+0000","last_became_active":"2024-04-25T10:12:48.616865+0000","last_became_peered":"2024-04-25T10:12:48.616865+0000","last_unstale":"2024-04-25T10:12:49.616583+0000","last_undegraded":"2024-04-25T10:12:49.616583+0000","last_fullsized":"2024-04-25T10:12:49.616583+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:36.175763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.795776+0000","last_change":"2024-04-25T10:12:49.625425+0000","last_active":"2024-04-25T10:12:50.795776+0000","last_peered":"2024-04-25T10:12:50.795776+0000","last_clean":"2024-04-25T10:12:50.795776+0000","last_became_active":"2024-04-25T10:12:49.625066+0000","last_became_peered":"2024-04-25T10:12:49.625066+0000","last_unstale":"2024-04-25T10:12:50.795776+0000","last_undegraded":"2024-04-25T10:12:50.795776+0000","last_fullsized":"2024-04-25T10:12:50.795776+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:36.776087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.619460+0000","last_change":"2024-04-25T10:12:48.610213+0000","last_active":"2024-04-25T10:12:50.619460+0000","last_peered":"2024-04-25T10:12:50.619460+0000","last_clean":"2024-04-25T10:12:50.619460+0000","last_became_active":"2024-04-25T10:12:48.609969+0000","last_became_peered":"2024-04-25T10:12:48.609969+0000","last_unstale":"2024-04-25T10:12:50.619460+0000","last_undegraded":"2024-04-25T10:12:50.619460+0000","last_fullsized":"2024-04-25T10:12:50.619460+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:27.901314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618264+0000","last_change":"2024-04-25T10:12:49.619415+0000","last_active":"2024-04-25T10:12:50.618264+0000","last_peered":"2024-04-25T10:12:50.618264+0000","last_clean":"2024-04-25T10:12:50.618264+0000","last_became_active":"2024-04-25T10:12:49.619138+0000","last_became_peered":"2024-04-25T10:12:49.619138+0000","last_unstale":"2024-04-25T10:12:50.618264+0000","last_undegraded":"2024-04-25T10:12:50.618264+0000","last_fullsized":"2024-04-25T10:12:50.618264+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:55.852710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.795748+0000","last_change":"2024-04-25T10:12:48.617416+0000","last_active":"2024-04-25T10:12:50.795748+0000","last_peered":"2024-04-25T10:12:50.795748+0000","last_clean":"2024-04-25T10:12:50.795748+0000","last_became_active":"2024-04-25T10:12:48.617067+0000","last_became_peered":"2024-04-25T10:12:48.617067+0000","last_unstale":"2024-04-25T10:12:50.795748+0000","last_undegraded":"2024-04-25T10:12:50.795748+0000","last_fullsized":"2024-04-25T10:12:50.795748+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:58.770181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.850976+0000","last_change":"2024-04-25T10:12:49.628860+0000","last_active":"2024-04-25T10:12:50.850976+0000","last_peered":"2024-04-25T10:12:50.850976+0000","last_clean":"2024-04-25T10:12:50.850976+0000","last_became_active":"2024-04-25T10:12:49.628591+0000","last_became_peered":"2024-04-25T10:12:49.628591+0000","last_unstale":"2024-04-25T10:12:50.850976+0000","last_undegraded":"2024-04-25T10:12:50.850976+0000","last_fullsized":"2024-04-25T10:12:50.850976+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:25.186199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"99'3","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.851166+0000","last_change":"2024-04-25T10:12:48.605012+0000","last_active":"2024-04-25T10:12:50.851166+0000","last_peered":"2024-04-25T10:12:50.851166+0000","last_clean":"2024-04-25T10:12:50.851166+0000","last_became_active":"2024-04-25T10:12:48.604710+0000","last_became_peered":"2024-04-25T10:12:48.604710+0000","last_unstale":"2024-04-25T10:12:50.851166+0000","last_undegraded":"2024-04-25T10:12:50.851166+0000","last_fullsized":"2024-04-25T10:12:50.851166+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.450303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618223+0000","last_change":"2024-04-25T10:12:49.626635+0000","last_active":"2024-04-25T10:12:50.618223+0000","last_peered":"2024-04-25T10:12:50.618223+0000","last_clean":"2024-04-25T10:12:50.618223+0000","last_became_active":"2024-04-25T10:12:49.626398+0000","last_became_peered":"2024-04-25T10:12:49.626398+0000","last_unstale":"2024-04-25T10:12:50.618223+0000","last_undegraded":"2024-04-25T10:12:50.618223+0000","last_fullsized":"2024-04-25T10:12:50.618223+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:28.981565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"99'6","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.619008+0000","last_change":"2024-04-25T10:12:48.607766+0000","last_active":"2024-04-25T10:12:50.619008+0000","last_peered":"2024-04-25T10:12:50.619008+0000","last_clean":"2024-04-25T10:12:50.619008+0000","last_became_active":"2024-04-25T10:12:48.607494+0000","last_became_peered":"2024-04-25T10:12:48.607494+0000","last_unstale":"2024-04-25T10:12:50.619008+0000","last_undegraded":"2024-04-25T10:12:50.619008+0000","last_fullsized":"2024-04-25T10:12:50.619008+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:20.328166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.795247+0000","last_change":"2024-04-25T10:12:49.634348+0000","last_active":"2024-04-25T10:12:50.795247+0000","last_peered":"2024-04-25T10:12:50.795247+0000","last_clean":"2024-04-25T10:12:50.795247+0000","last_became_active":"2024-04-25T10:12:49.633327+0000","last_became_peered":"2024-04-25T10:12:49.633327+0000","last_unstale":"2024-04-25T10:12:50.795247+0000","last_undegraded":"2024-04-25T10:12:50.795247+0000","last_fullsized":"2024-04-25T10:12:50.795247+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:15.536905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"99'2","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.632486+0000","last_change":"2024-04-25T10:12:48.612576+0000","last_active":"2024-04-25T10:12:49.632486+0000","last_peered":"2024-04-25T10:12:49.632486+0000","last_clean":"2024-04-25T10:12:49.632486+0000","last_became_active":"2024-04-25T10:12:48.612301+0000","last_became_peered":"2024-04-25T10:12:48.612301+0000","last_unstale":"2024-04-25T10:12:49.632486+0000","last_undegraded":"2024-04-25T10:12:49.632486+0000","last_fullsized":"2024-04-25T10:12:49.632486+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:41.978908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.619243+0000","last_change":"2024-04-25T10:12:49.622887+0000","last_active":"2024-04-25T10:12:50.619243+0000","last_peered":"2024-04-25T10:12:50.619243+0000","last_clean":"2024-04-25T10:12:50.619243+0000","last_became_active":"2024-04-25T10:12:49.622434+0000","last_became_peered":"2024-04-25T10:12:49.622434+0000","last_unstale":"2024-04-25T10:12:50.619243+0000","last_undegraded":"2024-04-25T10:12:50.619243+0000","last_fullsized":"2024-04-25T10:12:50.619243+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:59.072488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.795273+0000","last_change":"2024-04-25T10:12:48.612870+0000","last_active":"2024-04-25T10:12:50.795273+0000","last_peered":"2024-04-25T10:12:50.795273+0000","last_clean":"2024-04-25T10:12:50.795273+0000","last_became_active":"2024-04-25T10:12:48.612487+0000","last_became_peered":"2024-04-25T10:12:48.612487+0000","last_unstale":"2024-04-25T10:12:50.795273+0000","last_undegraded":"2024-04-25T10:12:50.795273+0000","last_fullsized":"2024-04-25T10:12:50.795273+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:35.801634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.850647+0000","last_change":"2024-04-25T10:12:49.620744+0000","last_active":"2024-04-25T10:12:50.850647+0000","last_peered":"2024-04-25T10:12:50.850647+0000","last_clean":"2024-04-25T10:12:50.850647+0000","last_became_active":"2024-04-25T10:12:49.620205+0000","last_became_peered":"2024-04-25T10:12:49.620205+0000","last_unstale":"2024-04-25T10:12:50.850647+0000","last_undegraded":"2024-04-25T10:12:50.850647+0000","last_fullsized":"2024-04-25T10:12:50.850647+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:18.655253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618428+0000","last_change":"2024-04-25T10:12:48.613055+0000","last_active":"2024-04-25T10:12:50.618428+0000","last_peered":"2024-04-25T10:12:50.618428+0000","last_clean":"2024-04-25T10:12:50.618428+0000","last_became_active":"2024-04-25T10:12:48.612577+0000","last_became_peered":"2024-04-25T10:12:48.612577+0000","last_unstale":"2024-04-25T10:12:50.618428+0000","last_undegraded":"2024-04-25T10:12:50.618428+0000","last_fullsized":"2024-04-25T10:12:50.618428+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:05.207193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618928+0000","last_change":"2024-04-25T10:12:49.622038+0000","last_active":"2024-04-25T10:12:50.618928+0000","last_peered":"2024-04-25T10:12:50.618928+0000","last_clean":"2024-04-25T10:12:50.618928+0000","last_became_active":"2024-04-25T10:12:49.621736+0000","last_became_peered":"2024-04-25T10:12:49.621736+0000","last_unstale":"2024-04-25T10:12:50.618928+0000","last_undegraded":"2024-04-25T10:12:50.618928+0000","last_fullsized":"2024-04-25T10:12:50.618928+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:34.149678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"99'8","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.795243+0000","last_change":"2024-04-25T10:12:48.613622+0000","last_active":"2024-04-25T10:12:50.795243+0000","last_peered":"2024-04-25T10:12:50.795243+0000","last_clean":"2024-04-25T10:12:50.795243+0000","last_became_active":"2024-04-25T10:12:48.612865+0000","last_became_peered":"2024-04-25T10:12:48.612865+0000","last_unstale":"2024-04-25T10:12:50.795243+0000","last_undegraded":"2024-04-25T10:12:50.795243+0000","last_fullsized":"2024-04-25T10:12:50.795243+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:36.395167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618415+0000","last_change":"2024-04-25T10:12:49.618894+0000","last_active":"2024-04-25T10:12:50.618415+0000","last_peered":"2024-04-25T10:12:50.618415+0000","last_clean":"2024-04-25T10:12:50.618415+0000","last_became_active":"2024-04-25T10:12:49.618483+0000","last_became_peered":"2024-04-25T10:12:49.618483+0000","last_unstale":"2024-04-25T10:12:50.618415+0000","last_undegraded":"2024-04-25T10:12:50.618415+0000","last_fullsized":"2024-04-25T10:12:50.618415+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:56.681174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618839+0000","last_change":"2024-04-25T10:12:48.607452+0000","last_active":"2024-04-25T10:12:50.618839+0000","last_peered":"2024-04-25T10:12:50.618839+0000","last_clean":"2024-04-25T10:12:50.618839+0000","last_became_active":"2024-04-25T10:12:48.607157+0000","last_became_peered":"2024-04-25T10:12:48.607157+0000","last_unstale":"2024-04-25T10:12:50.618839+0000","last_undegraded":"2024-04-25T10:12:50.618839+0000","last_fullsized":"2024-04-25T10:12:50.618839+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:42.470562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618142+0000","last_change":"2024-04-25T10:12:49.621719+0000","last_active":"2024-04-25T10:12:50.618142+0000","last_peered":"2024-04-25T10:12:50.618142+0000","last_clean":"2024-04-25T10:12:50.618142+0000","last_became_active":"2024-04-25T10:12:49.621354+0000","last_became_peered":"2024-04-25T10:12:49.621354+0000","last_unstale":"2024-04-25T10:12:50.618142+0000","last_undegraded":"2024-04-25T10:12:50.618142+0000","last_fullsized":"2024-04-25T10:12:50.618142+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:18.298533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.851202+0000","last_change":"2024-04-25T10:12:48.610525+0000","last_active":"2024-04-25T10:12:50.851202+0000","last_peered":"2024-04-25T10:12:50.851202+0000","last_clean":"2024-04-25T10:12:50.851202+0000","last_became_active":"2024-04-25T10:12:48.609675+0000","last_became_peered":"2024-04-25T10:12:48.609675+0000","last_unstale":"2024-04-25T10:12:50.851202+0000","last_undegraded":"2024-04-25T10:12:50.851202+0000","last_fullsized":"2024-04-25T10:12:50.851202+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:42.627034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618758+0000","last_change":"2024-04-25T10:12:49.623762+0000","last_active":"2024-04-25T10:12:50.618758+0000","last_peered":"2024-04-25T10:12:50.618758+0000","last_clean":"2024-04-25T10:12:50.618758+0000","last_became_active":"2024-04-25T10:12:49.623454+0000","last_became_peered":"2024-04-25T10:12:49.623454+0000","last_unstale":"2024-04-25T10:12:50.618758+0000","last_undegraded":"2024-04-25T10:12:50.618758+0000","last_fullsized":"2024-04-25T10:12:50.618758+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:00.533137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.616144+0000","last_change":"2024-04-25T10:12:48.612122+0000","last_active":"2024-04-25T10:12:49.616144+0000","last_peered":"2024-04-25T10:12:49.616144+0000","last_clean":"2024-04-25T10:12:49.616144+0000","last_became_active":"2024-04-25T10:12:48.611790+0000","last_became_peered":"2024-04-25T10:12:48.611790+0000","last_unstale":"2024-04-25T10:12:49.616144+0000","last_undegraded":"2024-04-25T10:12:49.616144+0000","last_fullsized":"2024-04-25T10:12:49.616144+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:06.583645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.630435+0000","last_change":"2024-04-25T10:12:49.625164+0000","last_active":"2024-04-25T10:12:49.630435+0000","last_peered":"2024-04-25T10:12:49.630435+0000","last_clean":"2024-04-25T10:12:49.630435+0000","last_became_active":"2024-04-25T10:12:49.624850+0000","last_became_peered":"2024-04-25T10:12:49.624850+0000","last_unstale":"2024-04-25T10:12:49.630435+0000","last_undegraded":"2024-04-25T10:12:49.630435+0000","last_fullsized":"2024-04-25T10:12:49.630435+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:11.948655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.795218+0000","last_change":"2024-04-25T10:12:48.618521+0000","last_active":"2024-04-25T10:12:50.795218+0000","last_peered":"2024-04-25T10:12:50.795218+0000","last_clean":"2024-04-25T10:12:50.795218+0000","last_became_active":"2024-04-25T10:12:48.618212+0000","last_became_peered":"2024-04-25T10:12:48.618212+0000","last_unstale":"2024-04-25T10:12:50.795218+0000","last_undegraded":"2024-04-25T10:12:50.795218+0000","last_fullsized":"2024-04-25T10:12:50.795218+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:19.839224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.637441+0000","last_change":"2024-04-25T10:12:49.633660+0000","last_active":"2024-04-25T10:12:49.637441+0000","last_peered":"2024-04-25T10:12:49.637441+0000","last_clean":"2024-04-25T10:12:49.637441+0000","last_became_active":"2024-04-25T10:12:49.633389+0000","last_became_peered":"2024-04-25T10:12:49.633389+0000","last_unstale":"2024-04-25T10:12:49.637441+0000","last_undegraded":"2024-04-25T10:12:49.637441+0000","last_fullsized":"2024-04-25T10:12:49.637441+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:12.735103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.617900+0000","last_change":"2024-04-25T10:12:48.607763+0000","last_active":"2024-04-25T10:12:50.617900+0000","last_peered":"2024-04-25T10:12:50.617900+0000","last_clean":"2024-04-25T10:12:50.617900+0000","last_became_active":"2024-04-25T10:12:48.607506+0000","last_became_peered":"2024-04-25T10:12:48.607506+0000","last_unstale":"2024-04-25T10:12:50.617900+0000","last_undegraded":"2024-04-25T10:12:50.617900+0000","last_fullsized":"2024-04-25T10:12:50.617900+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:58.067884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.795397+0000","last_change":"2024-04-25T10:12:49.621715+0000","last_active":"2024-04-25T10:12:50.795397+0000","last_peered":"2024-04-25T10:12:50.795397+0000","last_clean":"2024-04-25T10:12:50.795397+0000","last_became_active":"2024-04-25T10:12:49.621395+0000","last_became_peered":"2024-04-25T10:12:49.621395+0000","last_unstale":"2024-04-25T10:12:50.795397+0000","last_undegraded":"2024-04-25T10:12:50.795397+0000","last_fullsized":"2024-04-25T10:12:50.795397+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:28.981565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.850645+0000","last_change":"2024-04-25T10:12:48.607662+0000","last_active":"2024-04-25T10:12:50.850645+0000","last_peered":"2024-04-25T10:12:50.850645+0000","last_clean":"2024-04-25T10:12:50.850645+0000","last_became_active":"2024-04-25T10:12:48.607258+0000","last_became_peered":"2024-04-25T10:12:48.607258+0000","last_unstale":"2024-04-25T10:12:50.850645+0000","last_undegraded":"2024-04-25T10:12:50.850645+0000","last_fullsized":"2024-04-25T10:12:50.850645+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:54.818186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618003+0000","last_change":"2024-04-25T10:12:49.623956+0000","last_active":"2024-04-25T10:12:50.618003+0000","last_peered":"2024-04-25T10:12:50.618003+0000","last_clean":"2024-04-25T10:12:50.618003+0000","last_became_active":"2024-04-25T10:12:49.622006+0000","last_became_peered":"2024-04-25T10:12:49.622006+0000","last_unstale":"2024-04-25T10:12:50.618003+0000","last_undegraded":"2024-04-25T10:12:50.618003+0000","last_fullsized":"2024-04-25T10:12:50.618003+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:15.536905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618616+0000","last_change":"2024-04-25T10:12:48.610018+0000","last_active":"2024-04-25T10:12:50.618616+0000","last_peered":"2024-04-25T10:12:50.618616+0000","last_clean":"2024-04-25T10:12:50.618616+0000","last_became_active":"2024-04-25T10:12:48.609487+0000","last_became_peered":"2024-04-25T10:12:48.609487+0000","last_unstale":"2024-04-25T10:12:50.618616+0000","last_undegraded":"2024-04-25T10:12:50.618616+0000","last_fullsized":"2024-04-25T10:12:50.618616+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:59.528534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.631690+0000","last_change":"2024-04-25T10:12:49.626564+0000","last_active":"2024-04-25T10:12:49.631690+0000","last_peered":"2024-04-25T10:12:49.631690+0000","last_clean":"2024-04-25T10:12:49.631690+0000","last_became_active":"2024-04-25T10:12:49.626253+0000","last_became_peered":"2024-04-25T10:12:49.626253+0000","last_unstale":"2024-04-25T10:12:49.631690+0000","last_undegraded":"2024-04-25T10:12:49.631690+0000","last_fullsized":"2024-04-25T10:12:49.631690+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:23.187839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618968+0000","last_change":"2024-04-25T10:12:48.612343+0000","last_active":"2024-04-25T10:12:50.618968+0000","last_peered":"2024-04-25T10:12:50.618968+0000","last_clean":"2024-04-25T10:12:50.618968+0000","last_became_active":"2024-04-25T10:12:48.612036+0000","last_became_peered":"2024-04-25T10:12:48.612036+0000","last_unstale":"2024-04-25T10:12:50.618968+0000","last_undegraded":"2024-04-25T10:12:50.618968+0000","last_fullsized":"2024-04-25T10:12:50.618968+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:46.318299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.632576+0000","last_change":"2024-04-25T10:12:49.629290+0000","last_active":"2024-04-25T10:12:49.632576+0000","last_peered":"2024-04-25T10:12:49.632576+0000","last_clean":"2024-04-25T10:12:49.632576+0000","last_became_active":"2024-04-25T10:12:49.628982+0000","last_became_peered":"2024-04-25T10:12:49.628982+0000","last_unstale":"2024-04-25T10:12:49.632576+0000","last_undegraded":"2024-04-25T10:12:49.632576+0000","last_fullsized":"2024-04-25T10:12:49.632576+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:14.913579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.613170+0000","last_change":"2024-04-25T10:12:48.617283+0000","last_active":"2024-04-25T10:12:49.613170+0000","last_peered":"2024-04-25T10:12:49.613170+0000","last_clean":"2024-04-25T10:12:49.613170+0000","last_became_active":"2024-04-25T10:12:48.616924+0000","last_became_peered":"2024-04-25T10:12:48.616924+0000","last_unstale":"2024-04-25T10:12:49.613170+0000","last_undegraded":"2024-04-25T10:12:49.613170+0000","last_fullsized":"2024-04-25T10:12:49.613170+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:11.626114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.619321+0000","last_change":"2024-04-25T10:12:49.617243+0000","last_active":"2024-04-25T10:12:50.619321+0000","last_peered":"2024-04-25T10:12:50.619321+0000","last_clean":"2024-04-25T10:12:50.619321+0000","last_became_active":"2024-04-25T10:12:49.616950+0000","last_became_peered":"2024-04-25T10:12:49.616950+0000","last_unstale":"2024-04-25T10:12:50.619321+0000","last_undegraded":"2024-04-25T10:12:50.619321+0000","last_fullsized":"2024-04-25T10:12:50.619321+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:13.444453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.612338+0000","last_change":"2024-04-25T10:12:48.611644+0000","last_active":"2024-04-25T10:12:49.612338+0000","last_peered":"2024-04-25T10:12:49.612338+0000","last_clean":"2024-04-25T10:12:49.612338+0000","last_became_active":"2024-04-25T10:12:48.611350+0000","last_became_peered":"2024-04-25T10:12:48.611350+0000","last_unstale":"2024-04-25T10:12:49.612338+0000","last_undegraded":"2024-04-25T10:12:49.612338+0000","last_fullsized":"2024-04-25T10:12:49.612338+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:34.540070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.851145+0000","last_change":"2024-04-25T10:12:49.616770+0000","last_active":"2024-04-25T10:12:50.851145+0000","last_peered":"2024-04-25T10:12:50.851145+0000","last_clean":"2024-04-25T10:12:50.851145+0000","last_became_active":"2024-04-25T10:12:49.615887+0000","last_became_peered":"2024-04-25T10:12:49.615887+0000","last_unstale":"2024-04-25T10:12:50.851145+0000","last_undegraded":"2024-04-25T10:12:50.851145+0000","last_fullsized":"2024-04-25T10:12:50.851145+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:43.243225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618074+0000","last_change":"2024-04-25T10:12:48.606310+0000","last_active":"2024-04-25T10:12:50.618074+0000","last_peered":"2024-04-25T10:12:50.618074+0000","last_clean":"2024-04-25T10:12:50.618074+0000","last_became_active":"2024-04-25T10:12:48.605988+0000","last_became_peered":"2024-04-25T10:12:48.605988+0000","last_unstale":"2024-04-25T10:12:50.618074+0000","last_undegraded":"2024-04-25T10:12:50.618074+0000","last_fullsized":"2024-04-25T10:12:50.618074+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:33.145075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.628320+0000","last_change":"2024-04-25T10:12:49.624411+0000","last_active":"2024-04-25T10:12:49.628320+0000","last_peered":"2024-04-25T10:12:49.628320+0000","last_clean":"2024-04-25T10:12:49.628320+0000","last_became_active":"2024-04-25T10:12:49.623581+0000","last_became_peered":"2024-04-25T10:12:49.623581+0000","last_unstale":"2024-04-25T10:12:49.628320+0000","last_undegraded":"2024-04-25T10:12:49.628320+0000","last_fullsized":"2024-04-25T10:12:49.628320+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:47.880397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.619304+0000","last_change":"2024-04-25T10:12:48.613868+0000","last_active":"2024-04-25T10:12:50.619304+0000","last_peered":"2024-04-25T10:12:50.619304+0000","last_clean":"2024-04-25T10:12:50.619304+0000","last_became_active":"2024-04-25T10:12:48.613601+0000","last_became_peered":"2024-04-25T10:12:48.613601+0000","last_unstale":"2024-04-25T10:12:50.619304+0000","last_undegraded":"2024-04-25T10:12:50.619304+0000","last_fullsized":"2024-04-25T10:12:50.619304+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:05.264803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.851202+0000","last_change":"2024-04-25T10:12:49.622436+0000","last_active":"2024-04-25T10:12:50.851202+0000","last_peered":"2024-04-25T10:12:50.851202+0000","last_clean":"2024-04-25T10:12:50.851202+0000","last_became_active":"2024-04-25T10:12:49.622156+0000","last_became_peered":"2024-04-25T10:12:49.622156+0000","last_unstale":"2024-04-25T10:12:50.851202+0000","last_undegraded":"2024-04-25T10:12:50.851202+0000","last_fullsized":"2024-04-25T10:12:50.851202+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:07.426796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618022+0000","last_change":"2024-04-25T10:12:48.609269+0000","last_active":"2024-04-25T10:12:50.618022+0000","last_peered":"2024-04-25T10:12:50.618022+0000","last_clean":"2024-04-25T10:12:50.618022+0000","last_became_active":"2024-04-25T10:12:48.608978+0000","last_became_peered":"2024-04-25T10:12:48.608978+0000","last_unstale":"2024-04-25T10:12:50.618022+0000","last_undegraded":"2024-04-25T10:12:50.618022+0000","last_fullsized":"2024-04-25T10:12:50.618022+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:12.439849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.625166+0000","last_change":"2024-04-25T10:12:49.620038+0000","last_active":"2024-04-25T10:12:49.625166+0000","last_peered":"2024-04-25T10:12:49.625166+0000","last_clean":"2024-04-25T10:12:49.625166+0000","last_became_active":"2024-04-25T10:12:49.619771+0000","last_became_peered":"2024-04-25T10:12:49.619771+0000","last_unstale":"2024-04-25T10:12:49.625166+0000","last_undegraded":"2024-04-25T10:12:49.625166+0000","last_fullsized":"2024-04-25T10:12:49.625166+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:44.781605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.851390+0000","last_change":"2024-04-25T10:12:48.614034+0000","last_active":"2024-04-25T10:12:50.851390+0000","last_peered":"2024-04-25T10:12:50.851390+0000","last_clean":"2024-04-25T10:12:50.851390+0000","last_became_active":"2024-04-25T10:12:48.613745+0000","last_became_peered":"2024-04-25T10:12:48.613745+0000","last_unstale":"2024-04-25T10:12:50.851390+0000","last_undegraded":"2024-04-25T10:12:50.851390+0000","last_fullsized":"2024-04-25T10:12:50.851390+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:43.158364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618389+0000","last_change":"2024-04-25T10:12:49.624229+0000","last_active":"2024-04-25T10:12:50.618389+0000","last_peered":"2024-04-25T10:12:50.618389+0000","last_clean":"2024-04-25T10:12:50.618389+0000","last_became_active":"2024-04-25T10:12:49.623965+0000","last_became_peered":"2024-04-25T10:12:49.623965+0000","last_unstale":"2024-04-25T10:12:50.618389+0000","last_undegraded":"2024-04-25T10:12:50.618389+0000","last_fullsized":"2024-04-25T10:12:50.618389+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:17.178391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618793+0000","last_change":"2024-04-25T10:12:48.609800+0000","last_active":"2024-04-25T10:12:50.618793+0000","last_peered":"2024-04-25T10:12:50.618793+0000","last_clean":"2024-04-25T10:12:50.618793+0000","last_became_active":"2024-04-25T10:12:48.609137+0000","last_became_peered":"2024-04-25T10:12:48.609137+0000","last_unstale":"2024-04-25T10:12:50.618793+0000","last_undegraded":"2024-04-25T10:12:50.618793+0000","last_fullsized":"2024-04-25T10:12:50.618793+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:14.584312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.619553+0000","last_change":"2024-04-25T10:12:49.623262+0000","last_active":"2024-04-25T10:12:50.619553+0000","last_peered":"2024-04-25T10:12:50.619553+0000","last_clean":"2024-04-25T10:12:50.619553+0000","last_became_active":"2024-04-25T10:12:49.622979+0000","last_became_peered":"2024-04-25T10:12:49.622979+0000","last_unstale":"2024-04-25T10:12:50.619553+0000","last_undegraded":"2024-04-25T10:12:50.619553+0000","last_fullsized":"2024-04-25T10:12:50.619553+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:48.590133+0000","last_clean_scrub_stamp":"2024-04-25T10:12:48.590133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:10.035341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.619005+0000","last_change":"2024-04-25T10:12:48.608944+0000","last_active":"2024-04-25T10:12:50.619005+0000","last_peered":"2024-04-25T10:12:50.619005+0000","last_clean":"2024-04-25T10:12:50.619005+0000","last_became_active":"2024-04-25T10:12:48.608672+0000","last_became_peered":"2024-04-25T10:12:48.608672+0000","last_unstale":"2024-04-25T10:12:50.619005+0000","last_undegraded":"2024-04-25T10:12:50.619005+0000","last_fullsized":"2024-04-25T10:12:50.619005+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.585530+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.585530+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:36.849186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":329,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T10:12:50.618597+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:12:50.618597+0000","last_peered":"2024-04-25T10:12:50.618597+0000","last_clean":"2024-04-25T10:12:50.618597+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:12:50.618597+0000","last_undegraded":"2024-04-25T10:12:50.618597+0000","last_fullsized":"2024-04-25T10:12:50.618597+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509538,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1232896,"data_stored":528979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509538,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1228800,"data_stored":528397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509539,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1232896,"data_stored":528979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509539,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36560,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335280,"statfs":{"total":96636764160,"available":96599326720,"internally_reserved":0,"allocated":1241088,"data_stored":528591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509540,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36552,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335288,"statfs":{"total":96636764160,"available":96599334912,"internally_reserved":0,"allocated":1232896,"data_stored":528467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509538,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37148,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334692,"statfs":{"total":96636764160,"available":96598724608,"internally_reserved":0,"allocated":1843200,"data_stored":1120489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509540,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1228800,"data_stored":528397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574946,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32936,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338904,"statfs":{"total":96636764160,"available":96603037696,"internally_reserved":0,"allocated":1798144,"data_stored":1105690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:12:53.983 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:12:53.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:12:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.123+0000 7f6fa445d640 1 Processor -- start 2024-04-25T10:12:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.123+0000 7f6fa445d640 1 -- start start 2024-04-25T10:12:54.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.124+0000 7f6fa445d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f9c0d1670 0x7f6f9c0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.124+0000 7f6fa445d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f9c0d0690 0x7f6f9c0d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.124+0000 7f6fa445d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6f9c0b46e0 0x7f6f9c0b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.124+0000 7f6fa445d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9c0d1f90 con 0x7f6f9c0d1670 2024-04-25T10:12:54.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.124+0000 7f6fa445d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9c0cf5f0 con 0x7f6f9c0b46e0 2024-04-25T10:12:54.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.124+0000 7f6fa445d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0cf8d0 con 0x7f6f9c0d0690 2024-04-25T10:12:54.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.124+0000 7f6fa21f9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f9c0d1670 0x7f6f9c0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.124+0000 7f6fa21f9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f9c0d1670 0x7f6f9c0d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60690/0 (socket says 172.21.15.16:60690) 2024-04-25T10:12:54.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.124+0000 7f6fa21f9640 1 -- 172.21.15.16:0/2202731873 learned_addr learned my addr 172.21.15.16:0/2202731873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:54.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.124+0000 7f6fa21f9640 1 -- 172.21.15.16:0/2202731873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f9c0d0690 msgr2=0x7f6f9c0d0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.124+0000 7f6fa21f9640 1 --2- 172.21.15.16:0/2202731873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f9c0d0690 0x7f6f9c0d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.124+0000 7f6fa21f9640 1 -- 172.21.15.16:0/2202731873 >> v2:172.21.15.186:3300/0 conn(0x7f6f9c0b46e0 msgr2=0x7f6f9c0b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.124+0000 7f6fa21f9640 1 --2- 172.21.15.16:0/2202731873 >> v2:172.21.15.186:3300/0 conn(0x7f6f9c0b46e0 0x7f6f9c0b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.124+0000 7f6fa21f9640 1 -- 172.21.15.16:0/2202731873 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c0cfbb0 con 0x7f6f9c0d1670 2024-04-25T10:12:54.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.125+0000 7f6fa21f9640 1 --2- 172.21.15.16:0/2202731873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f9c0d1670 0x7f6f9c0d1a50 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0673a0 tx=0x7f6f9802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e2fc717acb6569 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.125+0000 7f6fa11f7640 1 -- 172.21.15.16:0/2202731873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f98037070 con 0x7f6f9c0d1670 2024-04-25T10:12:54.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.125+0000 7f6fa11f7640 1 -- 172.21.15.16:0/2202731873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f98032440 con 0x7f6f9c0d1670 2024-04-25T10:12:54.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.125+0000 7f6fa11f7640 1 -- 172.21.15.16:0/2202731873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f98032810 con 0x7f6f9c0d1670 2024-04-25T10:12:54.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.125+0000 7f6fa445d640 1 -- 172.21.15.16:0/2202731873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f9c0d1670 msgr2=0x7f6f9c0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.125+0000 7f6fa445d640 1 --2- 172.21.15.16:0/2202731873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f9c0d1670 0x7f6f9c0d1a50 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0673a0 tx=0x7f6f9802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.126+0000 7f6fa445d640 1 -- 172.21.15.16:0/2202731873 shutdown_connections 2024-04-25T10:12:54.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.126+0000 7f6fa445d640 1 --2- 172.21.15.16:0/2202731873 >> v2:172.21.15.186:3300/0 conn(0x7f6f9c0b46e0 0x7f6f9c0b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.126+0000 7f6fa445d640 1 --2- 172.21.15.16:0/2202731873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f9c0d0690 0x7f6f9c0d0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.126+0000 7f6fa445d640 1 --2- 172.21.15.16:0/2202731873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f9c0d1670 0x7f6f9c0d1a50 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.126+0000 7f6fa445d640 1 -- 172.21.15.16:0/2202731873 >> 172.21.15.16:0/2202731873 conn(0x7f6f9c0b33e0 msgr2=0x7f6f9c0b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.126+0000 7f6fa445d640 1 -- 172.21.15.16:0/2202731873 shutdown_connections 2024-04-25T10:12:54.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.126+0000 7f6fa445d640 1 -- 172.21.15.16:0/2202731873 wait complete. 2024-04-25T10:12:54.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.127+0000 7f6fa445d640 1 Processor -- start 2024-04-25T10:12:54.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.127+0000 7f6fa445d640 1 -- start start 2024-04-25T10:12:54.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.127+0000 7f6fa445d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f9c0b46e0 0x7f6f9c147710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.127+0000 7f6fa445d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f9c0d0690 0x7f6f9c1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.127+0000 7f6fa445d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f9c0d1670 0x7f6f9c0b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.127+0000 7f6fa445d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c15f240 con 0x7f6f9c0b46e0 2024-04-25T10:12:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.127+0000 7f6fa445d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0d0170 con 0x7f6f9c0d0690 2024-04-25T10:12:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.127+0000 7f6fa445d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c15f3c0 con 0x7f6f9c0d1670 2024-04-25T10:12:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.128+0000 7f6fa21f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f9c0b46e0 0x7f6f9c147710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.128+0000 7f6fa21f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f9c0b46e0 0x7f6f9c147710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60698/0 (socket says 172.21.15.16:60698) 2024-04-25T10:12:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.128+0000 7f6fa21f9640 1 -- 172.21.15.16:0/2300406865 learned_addr learned my addr 172.21.15.16:0/2300406865 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.128+0000 7f6fa19f8640 1 --2- 172.21.15.16:0/2300406865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f9c0d0690 0x7f6f9c1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.128+0000 7f6fa21f9640 1 -- 172.21.15.16:0/2300406865 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f9c0d1670 msgr2=0x7f6f9c0b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.128+0000 7f6fa29fa640 1 --2- 172.21.15.16:0/2300406865 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f9c0d1670 0x7f6f9c0b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.128+0000 7f6fa21f9640 1 --2- 172.21.15.16:0/2300406865 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f9c0d1670 0x7f6f9c0b53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.128+0000 7f6fa21f9640 1 -- 172.21.15.16:0/2300406865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f9c0d0690 msgr2=0x7f6f9c1568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.128+0000 7f6fa21f9640 1 --2- 172.21.15.16:0/2300406865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f9c0d0690 0x7f6f9c1568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.128+0000 7f6fa21f9640 1 -- 172.21.15.16:0/2300406865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c147c50 con 0x7f6f9c0b46e0 2024-04-25T10:12:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.128+0000 7f6fa19f8640 1 --2- 172.21.15.16:0/2300406865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f9c0d0690 0x7f6f9c1568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.128+0000 7f6fa29fa640 1 --2- 172.21.15.16:0/2300406865 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f9c0d1670 0x7f6f9c0b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.128+0000 7f6fa21f9640 1 --2- 172.21.15.16:0/2300406865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f9c0b46e0 0x7f6f9c147710 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f6f9802a220 tx=0x7f6f980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.128+0000 7f6f92ffd640 1 -- 172.21.15.16:0/2300406865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f98041070 con 0x7f6f9c0b46e0 2024-04-25T10:12:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.128+0000 7f6f92ffd640 1 -- 172.21.15.16:0/2300406865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9803c870 con 0x7f6f9c0b46e0 2024-04-25T10:12:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.128+0000 7f6fa445d640 1 -- 172.21.15.16:0/2300406865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9c158250 con 0x7f6f9c0b46e0 2024-04-25T10:12:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.128+0000 7f6f92ffd640 1 -- 172.21.15.16:0/2300406865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f98037040 con 0x7f6f9c0b46e0 2024-04-25T10:12:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.128+0000 7f6fa445d640 1 -- 172.21.15.16:0/2300406865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c1450a0 con 0x7f6f9c0b46e0 2024-04-25T10:12:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.129+0000 7f6fa445d640 1 -- 172.21.15.16:0/2300406865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9c0749b0 con 0x7f6f9c0b46e0 2024-04-25T10:12:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.130+0000 7f6f92ffd640 1 -- 172.21.15.16:0/2300406865 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f98044070 con 0x7f6f9c0b46e0 2024-04-25T10:12:54.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.130+0000 7f6f92ffd640 1 --2- 172.21.15.16:0/2300406865 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f8006b4d0 0x7f6f8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.130+0000 7f6f92ffd640 1 -- 172.21.15.16:0/2300406865 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f980aade0 con 0x7f6f9c0b46e0 2024-04-25T10:12:54.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.131+0000 7f6fa19f8640 1 --2- 172.21.15.16:0/2300406865 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f8006b4d0 0x7f6f8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.131+0000 7f6fa19f8640 1 --2- 172.21.15.16:0/2300406865 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f8006b4d0 0x7f6f8006d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0b3070 tx=0x7f6f8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.135+0000 7f6f92ffd640 1 -- 172.21.15.16:0/2300406865 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6f98079d70 con 0x7f6f9c0b46e0 2024-04-25T10:12:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.382+0000 7f6fa445d640 1 -- 172.21.15.16:0/2300406865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6f9c06e4f0 con 0x7f6f9c0b46e0 2024-04-25T10:12:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.384+0000 7f6f92ffd640 1 -- 172.21.15.16:0/2300406865 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7f6f98079670 con 0x7f6f9c0b46e0 2024-04-25T10:12:54.388 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-04-25T10:12:54.388 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCZKipmAph/GhAAG3x1zC98yHQ13fjpmrlPkg== 2024-04-25T10:12:54.389 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-04-25T10:12:54.389 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-04-25T10:12:54.389 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:12:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-04-25T10:12:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.386+0000 7f6fa445d640 1 -- 172.21.15.16:0/2300406865 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f8006b4d0 msgr2=0x7f6f8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.386+0000 7f6fa445d640 1 --2- 172.21.15.16:0/2300406865 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f8006b4d0 0x7f6f8006d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0b3070 tx=0x7f6f8c008040 comp rx=0 tx=0).stop 2024-04-25T10:12:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.386+0000 7f6fa445d640 1 -- 172.21.15.16:0/2300406865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f9c0b46e0 msgr2=0x7f6f9c147710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.386+0000 7f6fa445d640 1 --2- 172.21.15.16:0/2300406865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f9c0b46e0 0x7f6f9c147710 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f6f9802a220 tx=0x7f6f980046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.386+0000 7f6fa445d640 1 -- 172.21.15.16:0/2300406865 shutdown_connections 2024-04-25T10:12:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.386+0000 7f6fa445d640 1 --2- 172.21.15.16:0/2300406865 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f8006b4d0 0x7f6f8006d990 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.386+0000 7f6fa445d640 1 --2- 172.21.15.16:0/2300406865 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f9c0d1670 0x7f6f9c0b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.386+0000 7f6fa445d640 1 --2- 172.21.15.16:0/2300406865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f9c0d0690 0x7f6f9c1568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.386+0000 7f6fa445d640 1 --2- 172.21.15.16:0/2300406865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f9c0b46e0 0x7f6f9c147710 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.386+0000 7f6fa445d640 1 -- 172.21.15.16:0/2300406865 >> 172.21.15.16:0/2300406865 conn(0x7f6f9c0b33e0 msgr2=0x7f6f9c0b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.386+0000 7f6fa445d640 1 -- 172.21.15.16:0/2300406865 shutdown_connections 2024-04-25T10:12:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.386+0000 7f6fa445d640 1 -- 172.21.15.16:0/2300406865 wait complete. 2024-04-25T10:12:54.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:54.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.497+0000 7f0d1a934640 1 Processor -- start 2024-04-25T10:12:54.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.497+0000 7f0d1a934640 1 -- start start 2024-04-25T10:12:54.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.498+0000 7f0d1a934640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0d1415dce0 0x7f0d140d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.498+0000 7f0d1a934640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d140d35b0 0x7f0d140b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.498+0000 7f0d1a934640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1415e0c0 con 0x7f0d140d1f90 2024-04-25T10:12:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.498+0000 7f0d1a934640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d140b7b10 con 0x7f0d1415dce0 2024-04-25T10:12:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.498+0000 7f0d1a934640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140d2510 con 0x7f0d140d35b0 2024-04-25T10:12:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.498+0000 7f0d18ed1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0d140d1f90 0x7f0d1406bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35848/0 (socket says 172.21.15.16:35848) 2024-04-25T10:12:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.498+0000 7f0d18ed1640 1 -- 172.21.15.16:0/3368608691 learned_addr learned my addr 172.21.15.16:0/3368608691 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d13fff640 1 --2- 172.21.15.16:0/3368608691 >> v2:172.21.15.186:3300/0 conn(0x7f0d1415dce0 0x7f0d140d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3368608691 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190949227 0 0) 0x7f0d1415e0c0 con 0x7f0d140d1f90 2024-04-25T10:12:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d137fe640 1 --2- 172.21.15.16:0/3368608691 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d140d35b0 0x7f0d140b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3368608691 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d00003520 con 0x7f0d140d1f90 2024-04-25T10:12:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3368608691 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3363416668 0 0) 0x7f0d00003520 con 0x7f0d140d1f90 2024-04-25T10:12:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3368608691 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d140d35b0 msgr2=0x7f0d140b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d12ffd640 1 --2- 172.21.15.16:0/3368608691 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d140d35b0 0x7f0d140b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3368608691 >> v2:172.21.15.186:3300/0 conn(0x7f0d1415dce0 msgr2=0x7f0d140d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d12ffd640 1 --2- 172.21.15.16:0/3368608691 >> v2:172.21.15.186:3300/0 conn(0x7f0d1415dce0 0x7f0d140d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3368608691 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d140d27f0 con 0x7f0d140d1f90 2024-04-25T10:12:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3368608691 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f0d0c0019f0 con 0x7f0d140d1f90 2024-04-25T10:12:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3368608691 >> v1:172.21.15.16:6789/0 conn(0x7f0d140d1f90 legacy=0x7f0d1406bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d12ffd640 1 --2- 172.21.15.16:0/3368608691 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d140d35b0 0x7f0d000039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d12ffd640 1 --2- 172.21.15.16:0/3368608691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d00003f30 0x7f0d00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d12ffd640 1 --2- 172.21.15.16:0/3368608691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d00004d40 0x7f0d000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3368608691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140b7b10 con 0x7f0d00003f30 2024-04-25T10:12:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3368608691 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140d2510 con 0x7f0d00004d40 2024-04-25T10:12:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3368608691 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0000ba10 con 0x7f0d140d35b0 2024-04-25T10:12:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d137fe640 1 --2- 172.21.15.16:0/3368608691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d00003f30 0x7f0d00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d13fff640 1 --2- 172.21.15.16:0/3368608691 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d140d35b0 0x7f0d000039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d18ed1640 1 --2- 172.21.15.16:0/3368608691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d00004d40 0x7f0d000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d137fe640 1 -- 172.21.15.16:0/3368608691 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d140d35b0 msgr2=0x7f0d000039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d137fe640 1 --2- 172.21.15.16:0/3368608691 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d140d35b0 0x7f0d000039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d137fe640 1 -- 172.21.15.16:0/3368608691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d00004d40 msgr2=0x7f0d000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d137fe640 1 --2- 172.21.15.16:0/3368608691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d00004d40 0x7f0d000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.499+0000 7f0d137fe640 1 -- 172.21.15.16:0/3368608691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d00003520 con 0x7f0d00003f30 2024-04-25T10:12:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.500+0000 7f0d137fe640 1 --2- 172.21.15.16:0/3368608691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d00003f30 0x7f0d00004800 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009af0 tx=0x7f0d00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528c1a4f0df7631b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.500+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3368608691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04036070 con 0x7f0d00003f30 2024-04-25T10:12:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.500+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3368608691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d04031400 con 0x7f0d00003f30 2024-04-25T10:12:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.500+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3368608691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04009690 con 0x7f0d00003f30 2024-04-25T10:12:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.501+0000 7f0d1a934640 1 -- 172.21.15.16:0/3368608691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d00003f30 msgr2=0x7f0d00004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.501+0000 7f0d1a934640 1 --2- 172.21.15.16:0/3368608691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d00003f30 0x7f0d00004800 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009af0 tx=0x7f0d00005830 comp rx=0 tx=0).stop 2024-04-25T10:12:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.501+0000 7f0d18ed1640 1 -- 172.21.15.16:0/3368608691 reap_dead start 2024-04-25T10:12:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.501+0000 7f0d1a934640 1 -- 172.21.15.16:0/3368608691 shutdown_connections 2024-04-25T10:12:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.501+0000 7f0d1a934640 1 -- 172.21.15.16:0/3368608691 >> 172.21.15.16:0/3368608691 conn(0x7f0d14072900 msgr2=0x7f0d140d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.501+0000 7f0d1a934640 1 -- 172.21.15.16:0/3368608691 shutdown_connections 2024-04-25T10:12:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.501+0000 7f0d1a934640 1 -- 172.21.15.16:0/3368608691 wait complete. 2024-04-25T10:12:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.502+0000 7f0d1a934640 1 Processor -- start 2024-04-25T10:12:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.502+0000 7f0d1a934640 1 -- start start 2024-04-25T10:12:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.503+0000 7f0d1a934640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d00003f30 0x7f0d1415fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.503+0000 7f0d1a934640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d14160240 0x7f0d140b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.503+0000 7f0d1a934640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d140c97e0 0x7f0d140b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.503+0000 7f0d1a934640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140d3090 con 0x7f0d14160240 2024-04-25T10:12:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.503+0000 7f0d1a934640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140d2db0 con 0x7f0d00003f30 2024-04-25T10:12:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.503+0000 7f0d1a934640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140b9820 con 0x7f0d140c97e0 2024-04-25T10:12:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.503+0000 7f0d137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d14160240 0x7f0d140b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.503+0000 7f0d137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d14160240 0x7f0d140b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60702/0 (socket says 172.21.15.16:60702) 2024-04-25T10:12:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.503+0000 7f0d137fe640 1 -- 172.21.15.16:0/2624995904 learned_addr learned my addr 172.21.15.16:0/2624995904 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.503+0000 7f0d18ed1640 1 --2- 172.21.15.16:0/2624995904 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d140c97e0 0x7f0d140b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.503+0000 7f0d13fff640 1 --2- 172.21.15.16:0/2624995904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d00003f30 0x7f0d1415fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.503+0000 7f0d137fe640 1 -- 172.21.15.16:0/2624995904 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d140c97e0 msgr2=0x7f0d140b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.503+0000 7f0d137fe640 1 --2- 172.21.15.16:0/2624995904 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d140c97e0 0x7f0d140b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.503+0000 7f0d137fe640 1 -- 172.21.15.16:0/2624995904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d00003f30 msgr2=0x7f0d1415fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.503+0000 7f0d137fe640 1 --2- 172.21.15.16:0/2624995904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d00003f30 0x7f0d1415fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.503+0000 7f0d137fe640 1 -- 172.21.15.16:0/2624995904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d14160750 con 0x7f0d14160240 2024-04-25T10:12:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.503+0000 7f0d13fff640 1 --2- 172.21.15.16:0/2624995904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d00003f30 0x7f0d1415fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.504+0000 7f0d137fe640 1 --2- 172.21.15.16:0/2624995904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d14160240 0x7f0d140b4940 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009af0 tx=0x7f0d04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.504+0000 7f0d10ff9640 1 -- 172.21.15.16:0/2624995904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04040020 con 0x7f0d14160240 2024-04-25T10:12:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.504+0000 7f0d10ff9640 1 -- 172.21.15.16:0/2624995904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d04003c60 con 0x7f0d14160240 2024-04-25T10:12:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.504+0000 7f0d10ff9640 1 -- 172.21.15.16:0/2624995904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04036040 con 0x7f0d14160240 2024-04-25T10:12:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.504+0000 7f0d1a934640 1 -- 172.21.15.16:0/2624995904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d14160a30 con 0x7f0d14160240 2024-04-25T10:12:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.504+0000 7f0d1a934640 1 -- 172.21.15.16:0/2624995904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d14160ff0 con 0x7f0d14160240 2024-04-25T10:12:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.505+0000 7f0d1a934640 1 -- 172.21.15.16:0/2624995904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d140749b0 con 0x7f0d14160240 2024-04-25T10:12:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.505+0000 7f0d10ff9640 1 -- 172.21.15.16:0/2624995904 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d04043020 con 0x7f0d14160240 2024-04-25T10:12:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.506+0000 7f0d10ff9640 1 --2- 172.21.15.16:0/2624995904 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0cf406b4d0 0x7f0cf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.506+0000 7f0d10ff9640 1 -- 172.21.15.16:0/2624995904 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d040aa580 con 0x7f0d14160240 2024-04-25T10:12:54.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.506+0000 7f0d13fff640 1 --2- 172.21.15.16:0/2624995904 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0cf406b4d0 0x7f0cf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.506+0000 7f0d13fff640 1 --2- 172.21.15.16:0/2624995904 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0cf406b4d0 0x7f0cf406d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009c40 tx=0x7f0cfc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.510+0000 7f0d10ff9640 1 -- 172.21.15.16:0/2624995904 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d0404a030 con 0x7f0d14160240 2024-04-25T10:12:54.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.731+0000 7f0d1a934640 1 -- 172.21.15.16:0/2624995904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d140c9e80 con 0x7f0d14160240 2024-04-25T10:12:54.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.732+0000 7f0d10ff9640 1 -- 172.21.15.16:0/2624995904 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f0d040044e0 con 0x7f0d14160240 2024-04-25T10:12:54.737 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:54.737 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7243,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/109406141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":109406141},{"type":"v1","addr":"172.21.15.186:6837","nonce":109406141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2704401503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2704401503},{"type":"v1","addr":"172.21.15.16:6835","nonce":2704401503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/939221717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":939221717},{"type":"v1","addr":"172.21.15.16:6834","nonce":939221717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:48.958938+0000","modified":"2024-04-25T10:12:51.839702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8202},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8202":{"gid":8202,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6836/1371188714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1371188714},{"type":"v1","addr":"172.21.15.186:6836","nonce":1371188714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:54.737 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 71 2024-04-25T10:12:54.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.734+0000 7f0d1a934640 1 -- 172.21.15.16:0/2624995904 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0cf406b4d0 msgr2=0x7f0cf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.734+0000 7f0d1a934640 1 --2- 172.21.15.16:0/2624995904 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0cf406b4d0 0x7f0cf406d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009c40 tx=0x7f0cfc008040 comp rx=0 tx=0).stop 2024-04-25T10:12:54.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.734+0000 7f0d1a934640 1 -- 172.21.15.16:0/2624995904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d14160240 msgr2=0x7f0d140b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.734+0000 7f0d1a934640 1 --2- 172.21.15.16:0/2624995904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d14160240 0x7f0d140b4940 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009af0 tx=0x7f0d04004730 comp rx=0 tx=0).stop 2024-04-25T10:12:54.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.734+0000 7f0d1a934640 1 -- 172.21.15.16:0/2624995904 shutdown_connections 2024-04-25T10:12:54.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.734+0000 7f0d1a934640 1 --2- 172.21.15.16:0/2624995904 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0cf406b4d0 0x7f0cf406d990 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.734+0000 7f0d1a934640 1 --2- 172.21.15.16:0/2624995904 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d140c97e0 0x7f0d140b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.734+0000 7f0d1a934640 1 --2- 172.21.15.16:0/2624995904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d14160240 0x7f0d140b4940 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.734+0000 7f0d1a934640 1 --2- 172.21.15.16:0/2624995904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d00003f30 0x7f0d1415fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.734+0000 7f0d1a934640 1 -- 172.21.15.16:0/2624995904 >> 172.21.15.16:0/2624995904 conn(0x7f0d14072900 msgr2=0x7f0d140ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.734+0000 7f0d1a934640 1 -- 172.21.15.16:0/2624995904 shutdown_connections 2024-04-25T10:12:54.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:54.734+0000 7f0d1a934640 1 -- 172.21.15.16:0/2624995904 wait complete. 2024-04-25T10:12:55.152 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:12:48.958938+0000', 'modified': '2024-04-25T10:12:51.839702+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8202}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8202': {'gid': 8202, 'name': 'b', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.186:6836/1371188714', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 1371188714}, {'type': 'v1', 'addr': '172.21.15.186:6836', 'nonce': 1371188714}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:12:55.152 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:12:55.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:12:55.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.252+0000 7fdbd867c640 1 Processor -- start 2024-04-25T10:12:55.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.252+0000 7fdbd867c640 1 -- start start 2024-04-25T10:12:55.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.253+0000 7fdbd867c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fdbd015ef60 0x7fdbd00ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.253+0000 7fdbd867c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdbd00cea00 0x7fdbd00b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.253+0000 7fdbd867c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd00b1060 con 0x7fdbd00b0c80 2024-04-25T10:12:55.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.253+0000 7fdbd867c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd015f660 con 0x7fdbd00cea00 2024-04-25T10:12:55.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.253+0000 7fdbd867c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd015f940 con 0x7fdbd015ef60 2024-04-25T10:12:55.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.253+0000 7fdbd5c17640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdbd00cea00 0x7fdbd00b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.253+0000 7fdbd6c19640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdbd00b0c80 0x7fdbd00b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35852/0 (socket says 172.21.15.16:35852) 2024-04-25T10:12:55.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.253+0000 7fdbd6c19640 1 -- 172.21.15.16:0/1647666396 learned_addr learned my addr 172.21.15.16:0/1647666396 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:55.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.254+0000 7fdbd6418640 1 --2- 172.21.15.16:0/1647666396 >> v2:172.21.15.186:3300/0 conn(0x7fdbd015ef60 0x7fdbd00ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.254+0000 7fdbd5c17640 1 -- 172.21.15.16:0/1647666396 >> v2:172.21.15.186:3300/0 conn(0x7fdbd015ef60 msgr2=0x7fdbd00ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:55.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.254+0000 7fdbd5c17640 1 --2- 172.21.15.16:0/1647666396 >> v2:172.21.15.186:3300/0 conn(0x7fdbd015ef60 0x7fdbd00ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.254+0000 7fdbd5c17640 1 -- 172.21.15.16:0/1647666396 >> v1:172.21.15.16:6789/0 conn(0x7fdbd00b0c80 legacy=0x7fdbd00b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.254+0000 7fdbd5416640 1 -- 172.21.15.16:0/1647666396 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443500856 0 0) 0x7fdbd00b1060 con 0x7fdbd00b0c80 2024-04-25T10:12:55.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.254+0000 7fdbd5c17640 1 -- 172.21.15.16:0/1647666396 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd015fc20 con 0x7fdbd00cea00 2024-04-25T10:12:55.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.254+0000 7fdbd6418640 1 --2- 172.21.15.16:0/1647666396 >> v2:172.21.15.186:3300/0 conn(0x7fdbd015ef60 0x7fdbd00ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.254+0000 7fdbd5c17640 1 --2- 172.21.15.16:0/1647666396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd00cea00 0x7fdbd00b0740 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0070140 tx=0x7fdbc402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd174fc5d692d482 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.255+0000 7fdbd5416640 1 -- 172.21.15.16:0/1647666396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4037070 con 0x7fdbd00cea00 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.255+0000 7fdbd5416640 1 -- 172.21.15.16:0/1647666396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc4032440 con 0x7fdbd00cea00 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.255+0000 7fdbd5416640 1 -- 172.21.15.16:0/1647666396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4032810 con 0x7fdbd00cea00 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.255+0000 7fdbd867c640 1 -- 172.21.15.16:0/1647666396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd00cea00 msgr2=0x7fdbd00b0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.255+0000 7fdbd867c640 1 --2- 172.21.15.16:0/1647666396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd00cea00 0x7fdbd00b0740 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0070140 tx=0x7fdbc402a230 comp rx=0 tx=0).stop 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.255+0000 7fdbd867c640 1 -- 172.21.15.16:0/1647666396 shutdown_connections 2024-04-25T10:12:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.255+0000 7fdbd867c640 1 --2- 172.21.15.16:0/1647666396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd00cea00 0x7fdbd00b0740 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.255+0000 7fdbd867c640 1 --2- 172.21.15.16:0/1647666396 >> v2:172.21.15.186:3300/0 conn(0x7fdbd015ef60 0x7fdbd00ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.255+0000 7fdbd867c640 1 -- 172.21.15.16:0/1647666396 >> 172.21.15.16:0/1647666396 conn(0x7fdbd006f4c0 msgr2=0x7fdbd006f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.255+0000 7fdbd867c640 1 -- 172.21.15.16:0/1647666396 shutdown_connections 2024-04-25T10:12:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.255+0000 7fdbd867c640 1 -- 172.21.15.16:0/1647666396 wait complete. 2024-04-25T10:12:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.256+0000 7fdbd867c640 1 Processor -- start 2024-04-25T10:12:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.256+0000 7fdbd867c640 1 -- start start 2024-04-25T10:12:55.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.257+0000 7fdbd867c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd00b0c80 0x7fdbd00b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.257+0000 7fdbd867c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbd00cea00 0x7fdbd0158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.257+0000 7fdbd867c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdbd015ef60 0x7fdbd014e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.257+0000 7fdbd867c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd01604c0 con 0x7fdbd00b0c80 2024-04-25T10:12:55.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.257+0000 7fdbd867c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd01601e0 con 0x7fdbd00cea00 2024-04-25T10:12:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.257+0000 7fdbd867c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd015f340 con 0x7fdbd015ef60 2024-04-25T10:12:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.257+0000 7fdbd6418640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd00b0c80 0x7fdbd00b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.257+0000 7fdbd6418640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd00b0c80 0x7fdbd00b1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60720/0 (socket says 172.21.15.16:60720) 2024-04-25T10:12:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.257+0000 7fdbd6418640 1 -- 172.21.15.16:0/947114800 learned_addr learned my addr 172.21.15.16:0/947114800 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.258+0000 7fdbd6c19640 1 --2- 172.21.15.16:0/947114800 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdbd015ef60 0x7fdbd014e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.258+0000 7fdbd5c17640 1 --2- 172.21.15.16:0/947114800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbd00cea00 0x7fdbd0158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.258+0000 7fdbd6418640 1 -- 172.21.15.16:0/947114800 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdbd015ef60 msgr2=0x7fdbd014e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.258+0000 7fdbd6418640 1 --2- 172.21.15.16:0/947114800 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdbd015ef60 0x7fdbd014e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.258+0000 7fdbd6418640 1 -- 172.21.15.16:0/947114800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbd00cea00 msgr2=0x7fdbd0158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.258+0000 7fdbd6418640 1 --2- 172.21.15.16:0/947114800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbd00cea00 0x7fdbd0158850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.258+0000 7fdbd6418640 1 -- 172.21.15.16:0/947114800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd00cb800 con 0x7fdbd00b0c80 2024-04-25T10:12:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.258+0000 7fdbd5c17640 1 --2- 172.21.15.16:0/947114800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbd00cea00 0x7fdbd0158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.258+0000 7fdbd6c19640 1 --2- 172.21.15.16:0/947114800 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdbd015ef60 0x7fdbd014e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.258+0000 7fdbd6418640 1 --2- 172.21.15.16:0/947114800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd00b0c80 0x7fdbd00b1210 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800c7f0 tx=0x7fdbc800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.258+0000 7fdbceffd640 1 -- 172.21.15.16:0/947114800 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8015070 con 0x7fdbd00b0c80 2024-04-25T10:12:55.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.258+0000 7fdbceffd640 1 -- 172.21.15.16:0/947114800 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc8007a30 con 0x7fdbd00b0c80 2024-04-25T10:12:55.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.258+0000 7fdbd867c640 1 -- 172.21.15.16:0/947114800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd0156ea0 con 0x7fdbd00b0c80 2024-04-25T10:12:55.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.258+0000 7fdbceffd640 1 -- 172.21.15.16:0/947114800 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8007e00 con 0x7fdbd00b0c80 2024-04-25T10:12:55.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.258+0000 7fdbd867c640 1 -- 172.21.15.16:0/947114800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd006d9e0 con 0x7fdbd00b0c80 2024-04-25T10:12:55.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.259+0000 7fdbceffd640 1 -- 172.21.15.16:0/947114800 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdbc8018070 con 0x7fdbd00b0c80 2024-04-25T10:12:55.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.260+0000 7fdbceffd640 1 --2- 172.21.15.16:0/947114800 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdbbc06b4d0 0x7fdbbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.260+0000 7fdbceffd640 1 -- 172.21.15.16:0/947114800 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbc808a160 con 0x7fdbd00b0c80 2024-04-25T10:12:55.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.260+0000 7fdbd5c17640 1 --2- 172.21.15.16:0/947114800 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdbbc06b4d0 0x7fdbbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.261+0000 7fdbd5c17640 1 --2- 172.21.15.16:0/947114800 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdbbc06b4d0 0x7fdbbc06d990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4034000 tx=0x7fdbc402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.261+0000 7fdbdb377740 1 Processor -- start 2024-04-25T10:12:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.261+0000 7fdbdb377740 1 -- start start 2024-04-25T10:12:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbdb377740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba39230 0x560cdba39610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbdb377740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba56ad0 0x560cdba57020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbdb377740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba0ef10 0x560cdba55190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbdb377740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cdb971a10 con 0x560cdba39230 2024-04-25T10:12:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbdb377740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cdb91ab30 con 0x560cdba56ad0 2024-04-25T10:12:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbdb377740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560cdb96dc50 con 0x560cdba0ef10 2024-04-25T10:12:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbd6418640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba39230 0x560cdba39610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbd6418640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba39230 0x560cdba39610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60732/0 (socket says 172.21.15.16:60732) 2024-04-25T10:12:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbd6418640 1 -- 172.21.15.16:0/1253214237 learned_addr learned my addr 172.21.15.16:0/1253214237 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbd5c17640 1 --2- 172.21.15.16:0/1253214237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba56ad0 0x560cdba57020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbd6c19640 1 --2- 172.21.15.16:0/1253214237 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba0ef10 0x560cdba55190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbd6418640 1 -- 172.21.15.16:0/1253214237 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba0ef10 msgr2=0x560cdba55190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbd6418640 1 --2- 172.21.15.16:0/1253214237 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba0ef10 0x560cdba55190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbd6418640 1 -- 172.21.15.16:0/1253214237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba56ad0 msgr2=0x560cdba57020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbd6418640 1 --2- 172.21.15.16:0/1253214237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba56ad0 0x560cdba57020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbd6418640 1 -- 172.21.15.16:0/1253214237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cdb92fe10 con 0x560cdba39230 2024-04-25T10:12:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbd5c17640 1 --2- 172.21.15.16:0/1253214237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba56ad0 0x560cdba57020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbd6c19640 1 --2- 172.21.15.16:0/1253214237 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba0ef10 0x560cdba55190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbd6418640 1 --2- 172.21.15.16:0/1253214237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba39230 0x560cdba39610 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8054d30 tx=0x7fdbc8059090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16953c04e4db899 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbb7fff640 1 -- 172.21.15.16:0/1253214237 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc801f040 con 0x560cdba39230 2024-04-25T10:12:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbb7fff640 1 -- 172.21.15.16:0/1253214237 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc8060fe0 con 0x560cdba39230 2024-04-25T10:12:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.262+0000 7fdbb7fff640 1 -- 172.21.15.16:0/1253214237 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8054f40 con 0x560cdba39230 2024-04-25T10:12:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.263+0000 7fdbdb377740 1 -- 172.21.15.16:0/1253214237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba39230 msgr2=0x560cdba39610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.263+0000 7fdbdb377740 1 --2- 172.21.15.16:0/1253214237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba39230 0x560cdba39610 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8054d30 tx=0x7fdbc8059090 comp rx=0 tx=0).stop 2024-04-25T10:12:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.263+0000 7fdbdb377740 1 -- 172.21.15.16:0/1253214237 shutdown_connections 2024-04-25T10:12:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.263+0000 7fdbdb377740 1 --2- 172.21.15.16:0/1253214237 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba0ef10 0x560cdba55190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.263+0000 7fdbdb377740 1 --2- 172.21.15.16:0/1253214237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba56ad0 0x560cdba57020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.263+0000 7fdbdb377740 1 --2- 172.21.15.16:0/1253214237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba39230 0x560cdba39610 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.263+0000 7fdbdb377740 1 -- 172.21.15.16:0/1253214237 >> 172.21.15.16:0/1253214237 conn(0x560cdba0b840 msgr2=0x560cdba54040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.263+0000 7fdbdb377740 1 -- 172.21.15.16:0/1253214237 shutdown_connections 2024-04-25T10:12:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.263+0000 7fdbdb377740 1 -- 172.21.15.16:0/1253214237 wait complete. 2024-04-25T10:12:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.264+0000 7fdbdb377740 1 -- start start 2024-04-25T10:12:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.264+0000 7fdbdb377740 1 Processor -- start 2024-04-25T10:12:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbdb377740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba39230 0x560cdba29200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbdb377740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba56ad0 0x560cdba64a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbdb377740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba644e0 0x560cdba3c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbdb377740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cdba0fcb0 con 0x560cdba56ad0 2024-04-25T10:12:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbdb377740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cdb92e0c0 con 0x560cdba644e0 2024-04-25T10:12:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbdb377740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560cdba09770 con 0x560cdba39230 2024-04-25T10:12:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbdb377740 10 client.0 _ll_register_callbacks cb 0x560cdb921ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbd5c17640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba56ad0 0x560cdba64a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbd5c17640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba56ad0 0x560cdba64a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60738/0 (socket says 172.21.15.16:60738) 2024-04-25T10:12:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbd5c17640 1 -- 172.21.15.16:0/3726927980 learned_addr learned my addr 172.21.15.16:0/3726927980 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbd6c19640 1 --2- 172.21.15.16:0/3726927980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba644e0 0x560cdba3c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbd6418640 1 --2- 172.21.15.16:0/3726927980 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba39230 0x560cdba29200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbd5c17640 1 -- 172.21.15.16:0/3726927980 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba39230 msgr2=0x560cdba29200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbd5c17640 1 --2- 172.21.15.16:0/3726927980 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba39230 0x560cdba29200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbd5c17640 1 -- 172.21.15.16:0/3726927980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba644e0 msgr2=0x560cdba3c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbd5c17640 1 --2- 172.21.15.16:0/3726927980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba644e0 0x560cdba3c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbd5c17640 1 -- 172.21.15.16:0/3726927980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560cdb6862d0 con 0x560cdba56ad0 2024-04-25T10:12:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbd5c17640 1 --2- 172.21.15.16:0/3726927980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba56ad0 0x560cdba64a80 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4006ce0 tx=0x7fdbc402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57542506cd46a933 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:12:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbb57fa640 1 -- 172.21.15.16:0/3726927980 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4037040 con 0x560cdba56ad0 2024-04-25T10:12:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbb57fa640 1 -- 172.21.15.16:0/3726927980 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbc403e400 con 0x560cdba56ad0 2024-04-25T10:12:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbdb377740 1 -- 172.21.15.16:0/3726927980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbc8060fe0 con 0x560cdba56ad0 2024-04-25T10:12:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.265+0000 7fdbb57fa640 1 -- 172.21.15.16:0/3726927980 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbc4057050 con 0x560cdba56ad0 2024-04-25T10:12:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.266+0000 7fdbdb377740 10 client.8337 fetch_fsmap learned FSMap version 71 2024-04-25T10:12:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.266+0000 7fdbdb377740 1 -- 172.21.15.16:0/3726927980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x560cdb7506b0 con 0x560cdba56ad0 2024-04-25T10:12:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.266+0000 7fdbb57fa640 1 -- 172.21.15.16:0/3726927980 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc8060fe0 con 0x560cdba56ad0 2024-04-25T10:12:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.266+0000 7fdbb57fa640 1 -- 172.21.15.16:0/3726927980 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdbc405a020 con 0x560cdba56ad0 2024-04-25T10:12:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.266+0000 7fdbdb377740 10 client.8337 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:12:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.266+0000 7fdbdb377740 10 client.8337 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:12:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.266+0000 7fdbdb377740 20 client.8337 populate_metadata read hostname 'smithi016' 2024-04-25T10:12:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.266+0000 7fdbdb377740 1 --2- 172.21.15.16:0/3726927980 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x560cdba67f10 0x560cdba682f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.266+0000 7fdbdb377740 4 client.8337 mds_command: new command op to 8202 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.266+0000 7fdbdb377740 1 -- 172.21.15.16:0/3726927980 --> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cdb9d0920 con 0x560cdba67f10 2024-04-25T10:12:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.266+0000 7fdbd6418640 1 --2- 172.21.15.16:0/3726927980 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x560cdba67f10 0x560cdba682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.267+0000 7fdbd6418640 1 --2- 172.21.15.16:0/3726927980 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x560cdba67f10 0x560cdba682f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b620edbd7f2f7949 server_cookie=dfbc9e5e319398ce in_seq=0 out_seq=0 2024-04-25T10:12:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.267+0000 7fdbb57fa640 10 client.8337 ms_handle_connect on v2:172.21.15.186:6834/1371188714 2024-04-25T10:12:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.267+0000 7fdbb57fa640 1 -- 172.21.15.16:0/3726927980 <== mds.0 v2:172.21.15.186:6834/1371188714 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560cdb9d0920 con 0x560cdba67f10 2024-04-25T10:12:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.267+0000 7fdbb57fa640 10 client.8337 handle_command_reply: tid=0 2024-04-25T10:12:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.267+0000 7fdbdb377740 1 client.8337 shutdown 2024-04-25T10:12:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.267+0000 7fdbdb377740 1 -- 172.21.15.16:0/3726927980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba56ad0 msgr2=0x560cdba64a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.267+0000 7fdbdb377740 1 --2- 172.21.15.16:0/3726927980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba56ad0 0x560cdba64a80 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4006ce0 tx=0x7fdbc402c070 comp rx=0 tx=0).stop 2024-04-25T10:12:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.267+0000 7fdbdb377740 1 -- 172.21.15.16:0/3726927980 shutdown_connections 2024-04-25T10:12:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.267+0000 7fdbdb377740 1 --2- 172.21.15.16:0/3726927980 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x560cdba67f10 0x560cdba682f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.268+0000 7fdbdb377740 1 --2- 172.21.15.16:0/3726927980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba644e0 0x560cdba3c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.268+0000 7fdbdb377740 1 --2- 172.21.15.16:0/3726927980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba56ad0 0x560cdba64a80 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.268+0000 7fdbb57fa640 0 client.8337 ms_handle_reset on v2:172.21.15.186:6834/1371188714 2024-04-25T10:12:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.268+0000 7fdbdb377740 1 --2- 172.21.15.16:0/3726927980 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba39230 0x560cdba29200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.268+0000 7fdbdb377740 1 -- 172.21.15.16:0/3726927980 >> 172.21.15.16:0/3726927980 conn(0x560cdba0ef10 msgr2=0x560cdba575c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.268+0000 7fdbdb377740 1 -- 172.21.15.16:0/3726927980 shutdown_connections 2024-04-25T10:12:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.268+0000 7fdbdb377740 1 -- 172.21.15.16:0/3726927980 wait complete. 2024-04-25T10:12:55.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.268+0000 7fdbdb377740 20 client.8337 trim_cache size 0 max 16384 2024-04-25T10:12:55.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.283+0000 7fdbdb377740 1 Processor -- start 2024-04-25T10:12:55.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.284+0000 7fdbdb377740 1 -- start start 2024-04-25T10:12:55.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.284+0000 7fdbdb377740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba39230 0x560cdba57410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.284+0000 7fdbdb377740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba56ad0 0x560cdba57950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.284+0000 7fdbdb377740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba644e0 0x560cdba084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.284+0000 7fdbdb377740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00b3a00 con 0x560cdba644e0 2024-04-25T10:12:55.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.284+0000 7fdbdb377740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00b2e00 con 0x560cdba39230 2024-04-25T10:12:55.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.284+0000 7fdbdb377740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00b3400 con 0x560cdba56ad0 2024-04-25T10:12:55.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.284+0000 7fdbd6c19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba644e0 0x560cdba084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.284+0000 7fdbd6c19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba644e0 0x560cdba084a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60742/0 (socket says 172.21.15.16:60742) 2024-04-25T10:12:55.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.284+0000 7fdbd6c19640 1 -- 172.21.15.16:0/2369024648 learned_addr learned my addr 172.21.15.16:0/2369024648 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:55.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.284+0000 7fdbd6418640 1 --2- 172.21.15.16:0/2369024648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba39230 0x560cdba57410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.284+0000 7fdbd5c17640 1 --2- 172.21.15.16:0/2369024648 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba56ad0 0x560cdba57950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.285+0000 7fdbd6c19640 1 -- 172.21.15.16:0/2369024648 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba56ad0 msgr2=0x560cdba57950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.285+0000 7fdbd6c19640 1 --2- 172.21.15.16:0/2369024648 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba56ad0 0x560cdba57950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.285+0000 7fdbd6c19640 1 -- 172.21.15.16:0/2369024648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba39230 msgr2=0x560cdba57410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.285+0000 7fdbd6c19640 1 --2- 172.21.15.16:0/2369024648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba39230 0x560cdba57410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.285+0000 7fdbd6c19640 1 -- 172.21.15.16:0/2369024648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cdba4af20 con 0x560cdba644e0 2024-04-25T10:12:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.285+0000 7fdbd6c19640 1 --2- 172.21.15.16:0/2369024648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba644e0 0x560cdba084a0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0000f80 tx=0x7fdbc0007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71dc594aab92518 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.285+0000 7fdbb77fe640 1 -- 172.21.15.16:0/2369024648 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0019070 con 0x560cdba644e0 2024-04-25T10:12:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.285+0000 7fdbb77fe640 1 -- 172.21.15.16:0/2369024648 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc0012c30 con 0x560cdba644e0 2024-04-25T10:12:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.285+0000 7fdbb77fe640 1 -- 172.21.15.16:0/2369024648 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0015440 con 0x560cdba644e0 2024-04-25T10:12:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.285+0000 7fdbdb377740 1 -- 172.21.15.16:0/2369024648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba644e0 msgr2=0x560cdba084a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.286+0000 7fdbdb377740 1 --2- 172.21.15.16:0/2369024648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba644e0 0x560cdba084a0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0000f80 tx=0x7fdbc0007a10 comp rx=0 tx=0).stop 2024-04-25T10:12:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.286+0000 7fdbdb377740 1 -- 172.21.15.16:0/2369024648 shutdown_connections 2024-04-25T10:12:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.286+0000 7fdbdb377740 1 --2- 172.21.15.16:0/2369024648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba644e0 0x560cdba084a0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.286+0000 7fdbdb377740 1 --2- 172.21.15.16:0/2369024648 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba56ad0 0x560cdba57950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.286+0000 7fdbdb377740 1 --2- 172.21.15.16:0/2369024648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba39230 0x560cdba57410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.286+0000 7fdbdb377740 1 -- 172.21.15.16:0/2369024648 >> 172.21.15.16:0/2369024648 conn(0x560cdba0ef10 msgr2=0x560cdba3bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.286+0000 7fdbdb377740 1 -- 172.21.15.16:0/2369024648 shutdown_connections 2024-04-25T10:12:55.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.286+0000 7fdbdb377740 1 -- 172.21.15.16:0/2369024648 wait complete. 2024-04-25T10:12:55.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.286+0000 7fdbdb377740 1 -- start start 2024-04-25T10:12:55.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.286+0000 7fdbdb377740 1 Processor -- start 2024-04-25T10:12:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.287+0000 7fdbdb377740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba39230 0x560cdbac27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.287+0000 7fdbdb377740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba56ad0 0x560cdba35870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.287+0000 7fdbdb377740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba644e0 0x560cdba35db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.287+0000 7fdbdb377740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cdba27630 con 0x560cdba644e0 2024-04-25T10:12:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.287+0000 7fdbdb377740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cdba4b9a0 con 0x560cdba39230 2024-04-25T10:12:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.287+0000 7fdbdb377740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560cdba26ec0 con 0x560cdba56ad0 2024-04-25T10:12:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.287+0000 7fdbdb377740 10 client.0 _ll_register_callbacks cb 0x560cdba09c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.287+0000 7fdbd6c19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba644e0 0x560cdba35db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.287+0000 7fdbd6c19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba644e0 0x560cdba35db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60750/0 (socket says 172.21.15.16:60750) 2024-04-25T10:12:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.287+0000 7fdbd6c19640 1 -- 172.21.15.16:0/275643786 learned_addr learned my addr 172.21.15.16:0/275643786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.287+0000 7fdbd5c17640 1 --2- 172.21.15.16:0/275643786 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba56ad0 0x560cdba35870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.287+0000 7fdbd6418640 1 --2- 172.21.15.16:0/275643786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba39230 0x560cdbac27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbd6c19640 1 -- 172.21.15.16:0/275643786 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba56ad0 msgr2=0x560cdba35870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbd6c19640 1 --2- 172.21.15.16:0/275643786 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba56ad0 0x560cdba35870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbd6c19640 1 -- 172.21.15.16:0/275643786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba39230 msgr2=0x560cdbac27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbd6c19640 1 --2- 172.21.15.16:0/275643786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba39230 0x560cdbac27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbd6c19640 1 -- 172.21.15.16:0/275643786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560cdba579a0 con 0x560cdba644e0 2024-04-25T10:12:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbd6c19640 1 --2- 172.21.15.16:0/275643786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba644e0 0x560cdba35db0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00012e0 tx=0x7fdbc0007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b372503b5e333bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:12:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbb57fa640 1 -- 172.21.15.16:0/275643786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc002c060 con 0x560cdba644e0 2024-04-25T10:12:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbb57fa640 1 -- 172.21.15.16:0/275643786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbc001e910 con 0x560cdba644e0 2024-04-25T10:12:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbdb377740 1 -- 172.21.15.16:0/275643786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbc0012c30 con 0x560cdba644e0 2024-04-25T10:12:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbb57fa640 1 -- 172.21.15.16:0/275643786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbc0032050 con 0x560cdba644e0 2024-04-25T10:12:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbdb377740 10 client.8343 fetch_fsmap learned FSMap version 71 2024-04-25T10:12:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbdb377740 1 -- 172.21.15.16:0/275643786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x560cdba504a0 con 0x560cdba644e0 2024-04-25T10:12:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbb57fa640 1 -- 172.21.15.16:0/275643786 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc0012c30 con 0x560cdba644e0 2024-04-25T10:12:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbb57fa640 1 -- 172.21.15.16:0/275643786 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdbc0035020 con 0x560cdba644e0 2024-04-25T10:12:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbdb377740 10 client.8343 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:12:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbdb377740 10 client.8343 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:12:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbdb377740 20 client.8343 populate_metadata read hostname 'smithi016' 2024-04-25T10:12:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbdb377740 1 --2- 172.21.15.16:0/275643786 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x560cdbabfc10 0x560cdba25210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbdb377740 4 client.8343 mds_command: new command op to 8202 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:12:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.288+0000 7fdbdb377740 1 -- 172.21.15.16:0/275643786 --> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560cdb97ef50 con 0x560cdbabfc10 2024-04-25T10:12:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.289+0000 7fdbd6418640 1 --2- 172.21.15.16:0/275643786 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x560cdbabfc10 0x560cdba25210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.289+0000 7fdbd6418640 1 --2- 172.21.15.16:0/275643786 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x560cdbabfc10 0x560cdba25210 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21a965a569fb7685 server_cookie=537c20ec26bba473 in_seq=0 out_seq=0 2024-04-25T10:12:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.289+0000 7fdbb57fa640 10 client.8343 ms_handle_connect on v2:172.21.15.186:6834/1371188714 2024-04-25T10:12:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.289+0000 7fdbb57fa640 1 -- 172.21.15.16:0/275643786 <== mds.0 v2:172.21.15.186:6834/1371188714 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x560cdb97ef50 con 0x560cdbabfc10 2024-04-25T10:12:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.289+0000 7fdbb57fa640 10 client.8343 handle_command_reply: tid=0 2024-04-25T10:12:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.289+0000 7fdbdb377740 1 client.8343 shutdown 2024-04-25T10:12:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.289+0000 7fdbdb377740 1 -- 172.21.15.16:0/275643786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba644e0 msgr2=0x560cdba35db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.289+0000 7fdbdb377740 1 --2- 172.21.15.16:0/275643786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba644e0 0x560cdba35db0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00012e0 tx=0x7fdbc0007cc0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbdb377740 1 -- 172.21.15.16:0/275643786 shutdown_connections 2024-04-25T10:12:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbdb377740 1 --2- 172.21.15.16:0/275643786 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x560cdbabfc10 0x560cdba25210 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbdb377740 1 --2- 172.21.15.16:0/275643786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cdba644e0 0x560cdba35db0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbdb377740 1 --2- 172.21.15.16:0/275643786 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560cdba56ad0 0x560cdba35870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbdb377740 1 --2- 172.21.15.16:0/275643786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560cdba39230 0x560cdbac27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbb57fa640 0 client.8343 ms_handle_reset on v2:172.21.15.186:6834/1371188714 2024-04-25T10:12:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbdb377740 1 -- 172.21.15.16:0/275643786 >> 172.21.15.16:0/275643786 conn(0x560cdba0ef10 msgr2=0x560cdba29010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbdb377740 1 -- 172.21.15.16:0/275643786 shutdown_connections 2024-04-25T10:12:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbdb377740 1 -- 172.21.15.16:0/275643786 wait complete. 2024-04-25T10:12:55.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbdb377740 20 client.8343 trim_cache size 0 max 16384 2024-04-25T10:12:55.301 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:12:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "34bc5eca-3a21-42a6-a26f-07da0905e8bc", 2024-04-25T10:12:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-04-25T10:12:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 8202, 2024-04-25T10:12:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-04-25T10:12:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-04-25T10:12:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-04-25T10:12:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 5.6866890110000003, 2024-04-25T10:12:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 71, 2024-04-25T10:12:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 100, 2024-04-25T10:12:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 100, 2024-04-25T10:12:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 10.700579174 2024-04-25T10:12:55.302 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:12:55.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbd867c640 1 -- 172.21.15.16:0/947114800 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdbbc06b4d0 msgr2=0x7fdbbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbd867c640 1 --2- 172.21.15.16:0/947114800 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdbbc06b4d0 0x7fdbbc06d990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4034000 tx=0x7fdbc402e000 comp rx=0 tx=0).stop 2024-04-25T10:12:55.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbd867c640 1 -- 172.21.15.16:0/947114800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd00b0c80 msgr2=0x7fdbd00b1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbd867c640 1 --2- 172.21.15.16:0/947114800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd00b0c80 0x7fdbd00b1210 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800c7f0 tx=0x7fdbc800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbd867c640 1 -- 172.21.15.16:0/947114800 shutdown_connections 2024-04-25T10:12:55.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbd867c640 1 --2- 172.21.15.16:0/947114800 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdbbc06b4d0 0x7fdbbc06d990 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbd867c640 1 --2- 172.21.15.16:0/947114800 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdbd015ef60 0x7fdbd014e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbd867c640 1 --2- 172.21.15.16:0/947114800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbd00cea00 0x7fdbd0158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbd867c640 1 --2- 172.21.15.16:0/947114800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd00b0c80 0x7fdbd00b1210 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbd867c640 1 -- 172.21.15.16:0/947114800 >> 172.21.15.16:0/947114800 conn(0x7fdbd006f4c0 msgr2=0x7fdbd015ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbd867c640 1 -- 172.21.15.16:0/947114800 shutdown_connections 2024-04-25T10:12:55.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.290+0000 7fdbd867c640 1 -- 172.21.15.16:0/947114800 wait complete. 2024-04-25T10:12:55.305 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:12:55.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.402+0000 7f9d66c8f640 1 Processor -- start 2024-04-25T10:12:55.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.402+0000 7f9d66c8f640 1 -- start start 2024-04-25T10:12:55.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.402+0000 7f9d66c8f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9d600b5b60 0x7f9d601687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.402+0000 7f9d66c8f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d6015e0c0 con 0x7f9d600d35b0 2024-04-25T10:12:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.402+0000 7f9d66c8f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d600d2510 con 0x7f9d6015dce0 2024-04-25T10:12:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.402+0000 7f9d66c8f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d600b7b10 con 0x7f9d600b5b60 2024-04-25T10:12:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.403+0000 7f9d6522c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9d600b5b60 0x7f9d601687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.403+0000 7f9d57fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9d600d35b0 0x7f9d600b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35856/0 (socket says 172.21.15.16:35856) 2024-04-25T10:12:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.403+0000 7f9d57fff640 1 -- 172.21.15.16:0/1376218059 learned_addr learned my addr 172.21.15.16:0/1376218059 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.403+0000 7f9d6522c640 1 -- 172.21.15.16:0/1376218059 >> v1:172.21.15.186:6789/0 conn(0x7f9d6015dce0 legacy=0x7f9d600d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.403+0000 7f9d6522c640 1 -- 172.21.15.16:0/1376218059 >> v1:172.21.15.16:6789/0 conn(0x7f9d600d35b0 legacy=0x7f9d600b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.403+0000 7f9d6522c640 1 -- 172.21.15.16:0/1376218059 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d600d27f0 con 0x7f9d600b5b60 2024-04-25T10:12:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.403+0000 7f9d577fe640 1 -- 172.21.15.16:0/1376218059 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819545517 0 0) 0x7f9d6015e0c0 con 0x7f9d600d35b0 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.404+0000 7f9d6522c640 1 --2- 172.21.15.16:0/1376218059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d600b5b60 0x7f9d601687f0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009d90 tx=0x7f9d5802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf00e7f7acf9c6cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.404+0000 7f9d577fe640 1 -- 172.21.15.16:0/1376218059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d58007740 con 0x7f9d600b5b60 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.404+0000 7f9d577fe640 1 -- 172.21.15.16:0/1376218059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d580078e0 con 0x7f9d600b5b60 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.404+0000 7f9d577fe640 1 -- 172.21.15.16:0/1376218059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d58007cb0 con 0x7f9d600b5b60 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.404+0000 7f9d66c8f640 1 -- 172.21.15.16:0/1376218059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d600b5b60 msgr2=0x7f9d601687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.404+0000 7f9d66c8f640 1 --2- 172.21.15.16:0/1376218059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d600b5b60 0x7f9d601687f0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009d90 tx=0x7f9d5802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.404+0000 7f9d66c8f640 1 -- 172.21.15.16:0/1376218059 shutdown_connections 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.404+0000 7f9d66c8f640 1 --2- 172.21.15.16:0/1376218059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d600b5b60 0x7f9d601687f0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.404+0000 7f9d66c8f640 1 -- 172.21.15.16:0/1376218059 >> 172.21.15.16:0/1376218059 conn(0x7f9d60072900 msgr2=0x7f9d600d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.404+0000 7f9d66c8f640 1 -- 172.21.15.16:0/1376218059 shutdown_connections 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.404+0000 7f9d66c8f640 1 -- 172.21.15.16:0/1376218059 wait complete. 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.405+0000 7f9d66c8f640 1 Processor -- start 2024-04-25T10:12:55.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.406+0000 7f9d66c8f640 1 -- start start 2024-04-25T10:12:55.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.406+0000 7f9d66c8f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9d600b5b60 0x7f9d60160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.406+0000 7f9d66c8f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d600d35b0 0x7f9d601f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.406+0000 7f9d66c8f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d6015dce0 0x7f9d6015f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.406+0000 7f9d66c8f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d600d3090 con 0x7f9d600d35b0 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.406+0000 7f9d66c8f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d600d2db0 con 0x7f9d6015dce0 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.406+0000 7f9d66c8f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d600d2160 con 0x7f9d600b5b60 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.406+0000 7f9d57fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d600d35b0 0x7f9d601f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.407+0000 7f9d57fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d600d35b0 0x7f9d601f2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60766/0 (socket says 172.21.15.16:60766) 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.407+0000 7f9d57fff640 1 -- 172.21.15.16:0/2417450105 learned_addr learned my addr 172.21.15.16:0/2417450105 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.407+0000 7f9d6522c640 1 --2- 172.21.15.16:0/2417450105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d6015dce0 0x7f9d6015f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.407+0000 7f9d64a2b640 1 --2- 172.21.15.16:0/2417450105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9d600b5b60 0x7f9d60160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.407+0000 7f9d57fff640 1 -- 172.21.15.16:0/2417450105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9d600b5b60 msgr2=0x7f9d60160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.407+0000 7f9d57fff640 1 --2- 172.21.15.16:0/2417450105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9d600b5b60 0x7f9d60160d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.407+0000 7f9d57fff640 1 -- 172.21.15.16:0/2417450105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d6015dce0 msgr2=0x7f9d6015f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.407+0000 7f9d57fff640 1 --2- 172.21.15.16:0/2417450105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d6015dce0 0x7f9d6015f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.407+0000 7f9d57fff640 1 -- 172.21.15.16:0/2417450105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d601603e0 con 0x7f9d600d35b0 2024-04-25T10:12:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.407+0000 7f9d6522c640 1 --2- 172.21.15.16:0/2417450105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d6015dce0 0x7f9d6015f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.407+0000 7f9d64a2b640 1 --2- 172.21.15.16:0/2417450105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9d600b5b60 0x7f9d60160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.407+0000 7f9d57fff640 1 --2- 172.21.15.16:0/2417450105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d600d35b0 0x7f9d601f2cd0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000c460 tx=0x7f9d5000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.407+0000 7f9d557fa640 1 -- 172.21.15.16:0/2417450105 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50018070 con 0x7f9d600d35b0 2024-04-25T10:12:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.407+0000 7f9d557fa640 1 -- 172.21.15.16:0/2417450105 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d50007520 con 0x7f9d600d35b0 2024-04-25T10:12:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.407+0000 7f9d66c8f640 1 -- 172.21.15.16:0/2417450105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d601606c0 con 0x7f9d600d35b0 2024-04-25T10:12:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.407+0000 7f9d557fa640 1 -- 172.21.15.16:0/2417450105 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50004a20 con 0x7f9d600d35b0 2024-04-25T10:12:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.407+0000 7f9d66c8f640 1 -- 172.21.15.16:0/2417450105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6015f040 con 0x7f9d600d35b0 2024-04-25T10:12:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.408+0000 7f9d66c8f640 1 -- 172.21.15.16:0/2417450105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d600749b0 con 0x7f9d600d35b0 2024-04-25T10:12:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.409+0000 7f9d557fa640 1 -- 172.21.15.16:0/2417450105 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d5001b070 con 0x7f9d600d35b0 2024-04-25T10:12:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.409+0000 7f9d557fa640 1 --2- 172.21.15.16:0/2417450105 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9d4c06b4d0 0x7f9d4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.409+0000 7f9d557fa640 1 -- 172.21.15.16:0/2417450105 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d500895e0 con 0x7f9d600d35b0 2024-04-25T10:12:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.409+0000 7f9d64a2b640 1 --2- 172.21.15.16:0/2417450105 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9d4c06b4d0 0x7f9d4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.410+0000 7f9d64a2b640 1 --2- 172.21.15.16:0/2417450105 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9d4c06b4d0 0x7f9d4c06d990 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c000f60 tx=0x7f9d5c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.413+0000 7f9d557fa640 1 -- 172.21.15.16:0/2417450105 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d50058460 con 0x7f9d600d35b0 2024-04-25T10:12:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.633+0000 7f9d66c8f640 1 -- 172.21.15.16:0/2417450105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9d6006b080 con 0x7f9d600d35b0 2024-04-25T10:12:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.634+0000 7f9d557fa640 1 -- 172.21.15.16:0/2417450105 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9d50057d60 con 0x7f9d600d35b0 2024-04-25T10:12:55.638 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:12:55.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.636+0000 7f9d66c8f640 1 -- 172.21.15.16:0/2417450105 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9d4c06b4d0 msgr2=0x7f9d4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.636+0000 7f9d66c8f640 1 --2- 172.21.15.16:0/2417450105 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9d4c06b4d0 0x7f9d4c06d990 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c000f60 tx=0x7f9d5c009040 comp rx=0 tx=0).stop 2024-04-25T10:12:55.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.636+0000 7f9d66c8f640 1 -- 172.21.15.16:0/2417450105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d600d35b0 msgr2=0x7f9d601f2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.636+0000 7f9d66c8f640 1 --2- 172.21.15.16:0/2417450105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d600d35b0 0x7f9d601f2cd0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000c460 tx=0x7f9d5000c920 comp rx=0 tx=0).stop 2024-04-25T10:12:55.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.636+0000 7f9d66c8f640 1 -- 172.21.15.16:0/2417450105 shutdown_connections 2024-04-25T10:12:55.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.636+0000 7f9d66c8f640 1 --2- 172.21.15.16:0/2417450105 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9d4c06b4d0 0x7f9d4c06d990 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.636+0000 7f9d66c8f640 1 --2- 172.21.15.16:0/2417450105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d6015dce0 0x7f9d6015f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.636+0000 7f9d66c8f640 1 --2- 172.21.15.16:0/2417450105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d600d35b0 0x7f9d601f2cd0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.636+0000 7f9d66c8f640 1 --2- 172.21.15.16:0/2417450105 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9d600b5b60 0x7f9d60160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.636+0000 7f9d66c8f640 1 -- 172.21.15.16:0/2417450105 >> 172.21.15.16:0/2417450105 conn(0x7f9d60072900 msgr2=0x7f9d600d3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.636+0000 7f9d66c8f640 1 -- 172.21.15.16:0/2417450105 shutdown_connections 2024-04-25T10:12:55.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:55.636+0000 7f9d66c8f640 1 -- 172.21.15.16:0/2417450105 wait complete. 2024-04-25T10:12:55.651 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:12:55.651 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:12:55.651 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-04-25T10:12:55.651 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:12:55.651 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:55.651 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:12:55.652 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:12:55.652 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:12:55.652 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:55.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:55.652 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-04-25T10:12:55.667 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:12:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:12:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:12:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:12:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:55.668 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:12:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-04-25T10:12:55.668 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:12:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-04-25T10:12:55.668 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:12:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-04-25T10:12:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:12:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42185sec preferred_lft 42185sec 2024-04-25T10:12:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-25T10:12:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:55.669 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:12:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-04-25T10:12:55.669 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:12:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:12:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f06b:fdff:fed4:a335/64 scope link 2024-04-25T10:12:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:55.669 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:12:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::90ad:4fff:fe91:3c33/64 scope link 2024-04-25T10:12:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:55.670 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:12:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether be:14:4a:3c:16:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::bc14:4aff:fe3c:1605/64 scope link 2024-04-25T10:12:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:55.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:55.670 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:12:55.731 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:12:55.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:55.732 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-04-25T10:12:55.788 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-04-25T10:12:55.788 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:12:55.788 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:12:55.788 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:12:55.789 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:12:55.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:55.789 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:12:55.789 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:12:55.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:12:55.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:12:55.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:12:55.789 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:12:55.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:55.874 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:12:55.889 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:12:55.889 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:12:55.889 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:55.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:55.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-04-25T10:12:55.953 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:55.954 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:56.009 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:56.010 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:56.010 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:56.065 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:12:56.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:56.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:56.147 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:56.148 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:12:56.148 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:12:56.156 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:56.156 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:12:56.156 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:12:56.156 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:12:56.156 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:12:56.161 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:56.161 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:12:56.162 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:12:56.162 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:12:56.162 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:12:56.162 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:12:56.162 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:12:56.162 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:12:56.163 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:12:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:12:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:12:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:12:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:12:56.164 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:12:56.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:56.164 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:56.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:56.229 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:12:56.292 INFO:teuthology.orchestra.run.smithi016.stdout:client36470 2024-04-25T10:12:56.292 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:12:56.292 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:56.348 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:12:56.348 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:56.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:56.404 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:12:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:6+0 records in 2024-04-25T10:12:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:6+0 records out 2024-04-25T10:12:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0346018 s, 182 MB/s 2024-04-25T10:12:56.501 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:12:56.501 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:56.519 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:12:56.519 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:56.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:56.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.710+0000 7f15aaefc640 1 Processor -- start 2024-04-25T10:12:56.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.711+0000 7f15aaefc640 1 -- start start 2024-04-25T10:12:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.711+0000 7f15aaefc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15a415dce0 0x7f15a40d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.711+0000 7f15aaefc640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f15a40b5b60 0x7f15a40b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.711+0000 7f15aaefc640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a415e0c0 con 0x7f15a40d35b0 2024-04-25T10:12:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.711+0000 7f15aaefc640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a40d2510 con 0x7f15a40b5b60 2024-04-25T10:12:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.711+0000 7f15aaefc640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a40d27f0 con 0x7f15a415dce0 2024-04-25T10:12:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f15a40d35b0 0x7f15a40b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35860/0 (socket says 172.21.15.16:35860) 2024-04-25T10:12:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a3fff640 1 -- 172.21.15.16:0/3230096618 learned_addr learned my addr 172.21.15.16:0/3230096618 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a37fe640 1 -- 172.21.15.16:0/3230096618 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646706156 0 0) 0x7f15a415e0c0 con 0x7f15a40d35b0 2024-04-25T10:12:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a37fe640 1 -- 172.21.15.16:0/3230096618 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f158c003520 con 0x7f15a40d35b0 2024-04-25T10:12:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a9499640 1 --2- 172.21.15.16:0/3230096618 >> v2:172.21.15.186:3300/0 conn(0x7f15a40b5b60 0x7f15a40b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a8c98640 1 --2- 172.21.15.16:0/3230096618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15a415dce0 0x7f15a40d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a37fe640 1 -- 172.21.15.16:0/3230096618 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3754286452 0 0) 0x7f158c003520 con 0x7f15a40d35b0 2024-04-25T10:12:56.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a37fe640 1 -- 172.21.15.16:0/3230096618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15a415dce0 msgr2=0x7f15a40d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:56.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a37fe640 1 --2- 172.21.15.16:0/3230096618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15a415dce0 0x7f15a40d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a37fe640 1 -- 172.21.15.16:0/3230096618 >> v2:172.21.15.186:3300/0 conn(0x7f15a40b5b60 msgr2=0x7f15a40b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:56.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a37fe640 1 --2- 172.21.15.16:0/3230096618 >> v2:172.21.15.186:3300/0 conn(0x7f15a40b5b60 0x7f15a40b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a37fe640 1 -- 172.21.15.16:0/3230096618 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a40d2ad0 con 0x7f15a40d35b0 2024-04-25T10:12:56.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a37fe640 1 -- 172.21.15.16:0/3230096618 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f1598001670 con 0x7f15a40d35b0 2024-04-25T10:12:56.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a37fe640 1 -- 172.21.15.16:0/3230096618 >> v1:172.21.15.16:6789/0 conn(0x7f15a40d35b0 legacy=0x7f15a40b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a37fe640 1 --2- 172.21.15.16:0/3230096618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15a415dce0 0x7f158c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a37fe640 1 --2- 172.21.15.16:0/3230096618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f158c003f30 0x7f158c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a37fe640 1 --2- 172.21.15.16:0/3230096618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f158c004d40 0x7f158c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a37fe640 1 -- 172.21.15.16:0/3230096618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a40d2510 con 0x7f158c003f30 2024-04-25T10:12:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a37fe640 1 -- 172.21.15.16:0/3230096618 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a40d27f0 con 0x7f158c004d40 2024-04-25T10:12:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a37fe640 1 -- 172.21.15.16:0/3230096618 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158c00b9c0 con 0x7f15a415dce0 2024-04-25T10:12:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a3fff640 1 --2- 172.21.15.16:0/3230096618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f158c003f30 0x7f158c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a8c98640 1 --2- 172.21.15.16:0/3230096618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15a415dce0 0x7f158c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.712+0000 7f15a9499640 1 --2- 172.21.15.16:0/3230096618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f158c004d40 0x7f158c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.713+0000 7f15a3fff640 1 -- 172.21.15.16:0/3230096618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15a415dce0 msgr2=0x7f158c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.713+0000 7f15a3fff640 1 --2- 172.21.15.16:0/3230096618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15a415dce0 0x7f158c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.713+0000 7f15a3fff640 1 -- 172.21.15.16:0/3230096618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f158c004d40 msgr2=0x7f158c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.713+0000 7f15a3fff640 1 --2- 172.21.15.16:0/3230096618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f158c004d40 0x7f158c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.713+0000 7f15a3fff640 1 -- 172.21.15.16:0/3230096618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c003520 con 0x7f158c003f30 2024-04-25T10:12:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.713+0000 7f15a3fff640 1 --2- 172.21.15.16:0/3230096618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f158c003f30 0x7f158c004800 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f1598007fd0 tx=0x7f158c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2847a8a25037251 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.713+0000 7f15a37fe640 1 -- 172.21.15.16:0/3230096618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598007d50 con 0x7f158c003f30 2024-04-25T10:12:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.713+0000 7f15a37fe640 1 -- 172.21.15.16:0/3230096618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1598005ad0 con 0x7f158c003f30 2024-04-25T10:12:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.713+0000 7f15a37fe640 1 -- 172.21.15.16:0/3230096618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598007070 con 0x7f158c003f30 2024-04-25T10:12:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.714+0000 7f15aaefc640 1 -- 172.21.15.16:0/3230096618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f158c003f30 msgr2=0x7f158c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.714+0000 7f15aaefc640 1 --2- 172.21.15.16:0/3230096618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f158c003f30 0x7f158c004800 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f1598007fd0 tx=0x7f158c005830 comp rx=0 tx=0).stop 2024-04-25T10:12:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.714+0000 7f15a9499640 1 -- 172.21.15.16:0/3230096618 reap_dead start 2024-04-25T10:12:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.714+0000 7f15aaefc640 1 -- 172.21.15.16:0/3230096618 shutdown_connections 2024-04-25T10:12:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.714+0000 7f15aaefc640 1 -- 172.21.15.16:0/3230096618 >> 172.21.15.16:0/3230096618 conn(0x7f15a4072900 msgr2=0x7f15a40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.714+0000 7f15aaefc640 1 -- 172.21.15.16:0/3230096618 shutdown_connections 2024-04-25T10:12:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.714+0000 7f15aaefc640 1 -- 172.21.15.16:0/3230096618 wait complete. 2024-04-25T10:12:56.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.715+0000 7f15aaefc640 1 Processor -- start 2024-04-25T10:12:56.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.715+0000 7f15aaefc640 1 -- start start 2024-04-25T10:12:56.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.716+0000 7f15aaefc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f158c003f30 0x7f15a4156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.716+0000 7f15aaefc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a41f3760 0x7f15a41f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.716+0000 7f15aaefc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15a4154340 0x7f15a41547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.716+0000 7f15aaefc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a40b9820 con 0x7f15a41f3760 2024-04-25T10:12:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.716+0000 7f15aaefc640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a40d3090 con 0x7f15a4154340 2024-04-25T10:12:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.716+0000 7f15aaefc640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a40b99a0 con 0x7f158c003f30 2024-04-25T10:12:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.716+0000 7f15a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a41f3760 0x7f15a41f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.716+0000 7f15a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a41f3760 0x7f15a41f3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60796/0 (socket says 172.21.15.16:60796) 2024-04-25T10:12:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.716+0000 7f15a3fff640 1 -- 172.21.15.16:0/3169127334 learned_addr learned my addr 172.21.15.16:0/3169127334 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.717+0000 7f15a8c98640 1 --2- 172.21.15.16:0/3169127334 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f158c003f30 0x7f15a4156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.717+0000 7f15a9499640 1 --2- 172.21.15.16:0/3169127334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15a4154340 0x7f15a41547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.717+0000 7f15a3fff640 1 -- 172.21.15.16:0/3169127334 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f158c003f30 msgr2=0x7f15a4156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.717+0000 7f15a3fff640 1 --2- 172.21.15.16:0/3169127334 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f158c003f30 0x7f15a4156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.717+0000 7f15a3fff640 1 -- 172.21.15.16:0/3169127334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15a4154340 msgr2=0x7f15a41547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.717+0000 7f15a3fff640 1 --2- 172.21.15.16:0/3169127334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15a4154340 0x7f15a41547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.717+0000 7f15a3fff640 1 -- 172.21.15.16:0/3169127334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a4154d30 con 0x7f15a41f3760 2024-04-25T10:12:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.717+0000 7f15a3fff640 1 --2- 172.21.15.16:0/3169127334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a41f3760 0x7f15a41f3bc0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f159800c300 tx=0x7f15a40d2ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.717+0000 7f15a17fa640 1 -- 172.21.15.16:0/3169127334 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598042020 con 0x7f15a41f3760 2024-04-25T10:12:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.717+0000 7f15a17fa640 1 -- 172.21.15.16:0/3169127334 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1598004c10 con 0x7f15a41f3760 2024-04-25T10:12:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.717+0000 7f15aaefc640 1 -- 172.21.15.16:0/3169127334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a41f42e0 con 0x7f15a41f3760 2024-04-25T10:12:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.717+0000 7f15a17fa640 1 -- 172.21.15.16:0/3169127334 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598007070 con 0x7f15a41f3760 2024-04-25T10:12:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.717+0000 7f15aaefc640 1 -- 172.21.15.16:0/3169127334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a40cbac0 con 0x7f15a41f3760 2024-04-25T10:12:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.718+0000 7f15aaefc640 1 -- 172.21.15.16:0/3169127334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a40749b0 con 0x7f15a41f3760 2024-04-25T10:12:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.719+0000 7f15a17fa640 1 -- 172.21.15.16:0/3169127334 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1598045020 con 0x7f15a41f3760 2024-04-25T10:12:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.719+0000 7f15a17fa640 1 --2- 172.21.15.16:0/3169127334 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f159006b4d0 0x7f159006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.719+0000 7f15a17fa640 1 -- 172.21.15.16:0/3169127334 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f15980aa900 con 0x7f15a41f3760 2024-04-25T10:12:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.719+0000 7f15a8c98640 1 --2- 172.21.15.16:0/3169127334 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f159006b4d0 0x7f159006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.720+0000 7f15a8c98640 1 --2- 172.21.15.16:0/3169127334 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f159006b4d0 0x7f159006d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1594009ae0 tx=0x7f1594009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.723+0000 7f15a17fa640 1 -- 172.21.15.16:0/3169127334 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15980798a0 con 0x7f15a41f3760 2024-04-25T10:12:56.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.904+0000 7f15aaefc640 1 -- 172.21.15.16:0/3169127334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15a40b5090 con 0x7f15a41f3760 2024-04-25T10:12:56.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.905+0000 7f15a17fa640 1 -- 172.21.15.16:0/3169127334 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16610 (secure 0 0 0) 0x7f15980791a0 con 0x7f15a41f3760 2024-04-25T10:12:56.910 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:56.910 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":100,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:12:50.612438+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:47.551100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:47.939289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:56.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.907+0000 7f15aaefc640 1 -- 172.21.15.16:0/3169127334 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f159006b4d0 msgr2=0x7f159006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.907+0000 7f15aaefc640 1 --2- 172.21.15.16:0/3169127334 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f159006b4d0 0x7f159006d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1594009ae0 tx=0x7f1594009190 comp rx=0 tx=0).stop 2024-04-25T10:12:56.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.907+0000 7f15aaefc640 1 -- 172.21.15.16:0/3169127334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a41f3760 msgr2=0x7f15a41f3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.907+0000 7f15aaefc640 1 --2- 172.21.15.16:0/3169127334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a41f3760 0x7f15a41f3bc0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f159800c300 tx=0x7f15a40d2ca0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.907+0000 7f15aaefc640 1 -- 172.21.15.16:0/3169127334 shutdown_connections 2024-04-25T10:12:56.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.907+0000 7f15aaefc640 1 --2- 172.21.15.16:0/3169127334 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f159006b4d0 0x7f159006d990 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.907+0000 7f15aaefc640 1 --2- 172.21.15.16:0/3169127334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15a4154340 0x7f15a41547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.907+0000 7f15aaefc640 1 --2- 172.21.15.16:0/3169127334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a41f3760 0x7f15a41f3bc0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.907+0000 7f15aaefc640 1 --2- 172.21.15.16:0/3169127334 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f158c003f30 0x7f15a4156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.907+0000 7f15aaefc640 1 -- 172.21.15.16:0/3169127334 >> 172.21.15.16:0/3169127334 conn(0x7f15a4072900 msgr2=0x7f15a40b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:56.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.907+0000 7f15aaefc640 1 -- 172.21.15.16:0/3169127334 shutdown_connections 2024-04-25T10:12:56.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:56.907+0000 7f15aaefc640 1 -- 172.21.15.16:0/3169127334 wait complete. 2024-04-25T10:12:56.923 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:47.551100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:47.939289+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:56.924 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:57.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.023+0000 7fda2f3bd640 1 Processor -- start 2024-04-25T10:12:57.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.023+0000 7fda2f3bd640 1 -- start start 2024-04-25T10:12:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.024+0000 7fda2f3bd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda280ce3a0 0x7fda280b1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.024+0000 7fda2f3bd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fda2815d450 0x7fda2815d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.024+0000 7fda2f3bd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda280b1f60 con 0x7fda2806aba0 2024-04-25T10:12:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.024+0000 7fda2f3bd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda280b0130 con 0x7fda2815d450 2024-04-25T10:12:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.024+0000 7fda2f3bd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda280b09d0 con 0x7fda280ce3a0 2024-04-25T10:12:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.024+0000 7fda2d95a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fda2815d450 0x7fda2815d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.024+0000 7fda2d159640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fda2806aba0 0x7fda2806af80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35864/0 (socket says 172.21.15.16:35864) 2024-04-25T10:12:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.024+0000 7fda2d159640 1 -- 172.21.15.16:0/191719034 learned_addr learned my addr 172.21.15.16:0/191719034 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.024+0000 7fda2c958640 1 --2- 172.21.15.16:0/191719034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda280ce3a0 0x7fda280b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.024+0000 7fda2d95a640 1 -- 172.21.15.16:0/191719034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda280ce3a0 msgr2=0x7fda280b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:57.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.024+0000 7fda2d95a640 1 --2- 172.21.15.16:0/191719034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda280ce3a0 0x7fda280b1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.024+0000 7fda2d95a640 1 -- 172.21.15.16:0/191719034 >> v1:172.21.15.16:6789/0 conn(0x7fda2806aba0 legacy=0x7fda2806af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.024+0000 7fda1ffff640 1 -- 172.21.15.16:0/191719034 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3602624489 0 0) 0x7fda280b1f60 con 0x7fda2806aba0 2024-04-25T10:12:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.024+0000 7fda2d95a640 1 -- 172.21.15.16:0/191719034 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda280b0380 con 0x7fda2815d450 2024-04-25T10:12:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.025+0000 7fda2d95a640 1 --2- 172.21.15.16:0/191719034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda2815d450 0x7fda2815d8e0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fda20009dc0 tx=0x7fda2002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca30893dc8458429 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.025+0000 7fda1ffff640 1 -- 172.21.15.16:0/191719034 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2002ae60 con 0x7fda2815d450 2024-04-25T10:12:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.025+0000 7fda1ffff640 1 -- 172.21.15.16:0/191719034 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda20007740 con 0x7fda2815d450 2024-04-25T10:12:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.025+0000 7fda1ffff640 1 -- 172.21.15.16:0/191719034 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20007b10 con 0x7fda2815d450 2024-04-25T10:12:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.026+0000 7fda2f3bd640 1 -- 172.21.15.16:0/191719034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda2815d450 msgr2=0x7fda2815d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.026+0000 7fda2f3bd640 1 --2- 172.21.15.16:0/191719034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda2815d450 0x7fda2815d8e0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fda20009dc0 tx=0x7fda2002a680 comp rx=0 tx=0).stop 2024-04-25T10:12:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.026+0000 7fda2f3bd640 1 -- 172.21.15.16:0/191719034 shutdown_connections 2024-04-25T10:12:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.026+0000 7fda2f3bd640 1 --2- 172.21.15.16:0/191719034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda2815d450 0x7fda2815d8e0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.026+0000 7fda2f3bd640 1 --2- 172.21.15.16:0/191719034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda280ce3a0 0x7fda280b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.026+0000 7fda2f3bd640 1 -- 172.21.15.16:0/191719034 >> 172.21.15.16:0/191719034 conn(0x7fda280d07d0 msgr2=0x7fda280d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.026+0000 7fda2f3bd640 1 -- 172.21.15.16:0/191719034 shutdown_connections 2024-04-25T10:12:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.026+0000 7fda2f3bd640 1 -- 172.21.15.16:0/191719034 wait complete. 2024-04-25T10:12:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.027+0000 7fda2f3bd640 1 Processor -- start 2024-04-25T10:12:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.027+0000 7fda2f3bd640 1 -- start start 2024-04-25T10:12:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2f3bd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2806aba0 0x7fda280c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2f3bd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda280ce3a0 0x7fda28153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2f3bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda2815d450 0x7fda280c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2f3bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda280b0e60 con 0x7fda2815d450 2024-04-25T10:12:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2f3bd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2806b880 con 0x7fda2806aba0 2024-04-25T10:12:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2f3bd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda280b0fe0 con 0x7fda280ce3a0 2024-04-25T10:12:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2d95a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda2815d450 0x7fda280c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2d95a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda2815d450 0x7fda280c6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60816/0 (socket says 172.21.15.16:60816) 2024-04-25T10:12:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2d95a640 1 -- 172.21.15.16:0/81551281 learned_addr learned my addr 172.21.15.16:0/81551281 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2c958640 1 --2- 172.21.15.16:0/81551281 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda280ce3a0 0x7fda28153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2d95a640 1 -- 172.21.15.16:0/81551281 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda280ce3a0 msgr2=0x7fda28153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2d159640 1 --2- 172.21.15.16:0/81551281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2806aba0 0x7fda280c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2d95a640 1 --2- 172.21.15.16:0/81551281 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda280ce3a0 0x7fda28153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2d95a640 1 -- 172.21.15.16:0/81551281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2806aba0 msgr2=0x7fda280c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2d95a640 1 --2- 172.21.15.16:0/81551281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2806aba0 0x7fda280c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2d95a640 1 -- 172.21.15.16:0/81551281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2809a8b0 con 0x7fda2815d450 2024-04-25T10:12:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2c958640 1 --2- 172.21.15.16:0/81551281 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda280ce3a0 0x7fda28153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2d159640 1 --2- 172.21.15.16:0/81551281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2806aba0 0x7fda280c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.028+0000 7fda2d95a640 1 --2- 172.21.15.16:0/81551281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda2815d450 0x7fda280c6f50 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fda20009ee0 tx=0x7fda20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.029+0000 7fda1dffb640 1 -- 172.21.15.16:0/81551281 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2003f070 con 0x7fda2815d450 2024-04-25T10:12:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.029+0000 7fda1dffb640 1 -- 172.21.15.16:0/81551281 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda20031d40 con 0x7fda2815d450 2024-04-25T10:12:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.029+0000 7fda2f3bd640 1 -- 172.21.15.16:0/81551281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda280c53e0 con 0x7fda2815d450 2024-04-25T10:12:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.029+0000 7fda1dffb640 1 -- 172.21.15.16:0/81551281 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20009450 con 0x7fda2815d450 2024-04-25T10:12:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.029+0000 7fda2f3bd640 1 -- 172.21.15.16:0/81551281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda280c59a0 con 0x7fda2815d450 2024-04-25T10:12:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.030+0000 7fda2f3bd640 1 -- 172.21.15.16:0/81551281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda280750b0 con 0x7fda2815d450 2024-04-25T10:12:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.030+0000 7fda1dffb640 1 -- 172.21.15.16:0/81551281 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda2003b410 con 0x7fda2815d450 2024-04-25T10:12:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.030+0000 7fda1dffb640 1 --2- 172.21.15.16:0/81551281 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fda0c06b4d0 0x7fda0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.031+0000 7fda1dffb640 1 -- 172.21.15.16:0/81551281 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda200ab390 con 0x7fda2815d450 2024-04-25T10:12:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.031+0000 7fda2d159640 1 --2- 172.21.15.16:0/81551281 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fda0c06b4d0 0x7fda0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.031+0000 7fda2d159640 1 --2- 172.21.15.16:0/81551281 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fda0c06b4d0 0x7fda0c06d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fda24000f60 tx=0x7fda24009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.035+0000 7fda1dffb640 1 -- 172.21.15.16:0/81551281 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fda20079c50 con 0x7fda2815d450 2024-04-25T10:12:57.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.216+0000 7fda2f3bd640 1 -- 172.21.15.16:0/81551281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda280b1b60 con 0x7fda2815d450 2024-04-25T10:12:57.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.216+0000 7fda1dffb640 1 -- 172.21.15.16:0/81551281 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda2007dcf0 con 0x7fda2815d450 2024-04-25T10:12:57.221 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:12:57.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.219+0000 7fda2f3bd640 1 -- 172.21.15.16:0/81551281 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fda0c06b4d0 msgr2=0x7fda0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.219+0000 7fda2f3bd640 1 --2- 172.21.15.16:0/81551281 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fda0c06b4d0 0x7fda0c06d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fda24000f60 tx=0x7fda24009040 comp rx=0 tx=0).stop 2024-04-25T10:12:57.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.219+0000 7fda2f3bd640 1 -- 172.21.15.16:0/81551281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda2815d450 msgr2=0x7fda280c6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.219+0000 7fda2f3bd640 1 --2- 172.21.15.16:0/81551281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda2815d450 0x7fda280c6f50 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fda20009ee0 tx=0x7fda20004a40 comp rx=0 tx=0).stop 2024-04-25T10:12:57.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.219+0000 7fda2f3bd640 1 -- 172.21.15.16:0/81551281 shutdown_connections 2024-04-25T10:12:57.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.219+0000 7fda2f3bd640 1 --2- 172.21.15.16:0/81551281 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fda0c06b4d0 0x7fda0c06d990 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.219+0000 7fda2f3bd640 1 --2- 172.21.15.16:0/81551281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda2815d450 0x7fda280c6f50 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.219+0000 7fda2f3bd640 1 --2- 172.21.15.16:0/81551281 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda280ce3a0 0x7fda28153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.219+0000 7fda2f3bd640 1 --2- 172.21.15.16:0/81551281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2806aba0 0x7fda280c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.219+0000 7fda2f3bd640 1 -- 172.21.15.16:0/81551281 >> 172.21.15.16:0/81551281 conn(0x7fda280d07d0 msgr2=0x7fda2806f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.219+0000 7fda2f3bd640 1 -- 172.21.15.16:0/81551281 shutdown_connections 2024-04-25T10:12:57.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.219+0000 7fda2f3bd640 1 -- 172.21.15.16:0/81551281 wait complete. 2024-04-25T10:12:57.234 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:57.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.333+0000 7f8b407f5640 1 Processor -- start 2024-04-25T10:12:57.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.333+0000 7f8b407f5640 1 -- start start 2024-04-25T10:12:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.334+0000 7f8b407f5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8b3815dca0 0x7f8b380ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.334+0000 7f8b407f5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8b380d1130 0x7f8b380b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.334+0000 7f8b407f5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b380d4090 con 0x7f8b380d3b90 2024-04-25T10:12:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.334+0000 7f8b407f5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b380b6b20 con 0x7f8b3815dca0 2024-04-25T10:12:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.334+0000 7f8b407f5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b380d4370 con 0x7f8b380d1130 2024-04-25T10:12:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.334+0000 7f8b3e591640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8b3815dca0 0x7f8b380ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.334+0000 7f8b3dd90640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8b380d3b90 0x7f8b380d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35878/0 (socket says 172.21.15.16:35878) 2024-04-25T10:12:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.334+0000 7f8b3dd90640 1 -- 172.21.15.16:0/2266033476 learned_addr learned my addr 172.21.15.16:0/2266033476 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.334+0000 7f8b3ed92640 1 --2- 172.21.15.16:0/2266033476 >> v2:172.21.15.186:3300/0 conn(0x7f8b380d1130 0x7f8b380b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.334+0000 7f8b3e591640 1 -- 172.21.15.16:0/2266033476 >> v2:172.21.15.186:3300/0 conn(0x7f8b380d1130 msgr2=0x7f8b380b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.334+0000 7f8b3e591640 1 --2- 172.21.15.16:0/2266033476 >> v2:172.21.15.186:3300/0 conn(0x7f8b380d1130 0x7f8b380b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.334+0000 7f8b3e591640 1 -- 172.21.15.16:0/2266033476 >> v1:172.21.15.16:6789/0 conn(0x7f8b380d3b90 legacy=0x7f8b380d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.334+0000 7f8b3d58f640 1 -- 172.21.15.16:0/2266033476 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222325095 0 0) 0x7f8b380d4090 con 0x7f8b380d3b90 2024-04-25T10:12:57.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.334+0000 7f8b3e591640 1 -- 172.21.15.16:0/2266033476 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b380d4650 con 0x7f8b3815dca0 2024-04-25T10:12:57.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.335+0000 7f8b3e591640 1 --2- 172.21.15.16:0/2266033476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b3815dca0 0x7f8b380ce940 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f8b380b9640 tx=0x7f8b3002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6402839b32140aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.335+0000 7f8b3d58f640 1 -- 172.21.15.16:0/2266033476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30037070 con 0x7f8b3815dca0 2024-04-25T10:12:57.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.335+0000 7f8b3d58f640 1 -- 172.21.15.16:0/2266033476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b30032440 con 0x7f8b3815dca0 2024-04-25T10:12:57.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.335+0000 7f8b3d58f640 1 -- 172.21.15.16:0/2266033476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30032810 con 0x7f8b3815dca0 2024-04-25T10:12:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.336+0000 7f8b407f5640 1 -- 172.21.15.16:0/2266033476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b3815dca0 msgr2=0x7f8b380ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.336+0000 7f8b407f5640 1 --2- 172.21.15.16:0/2266033476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b3815dca0 0x7f8b380ce940 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f8b380b9640 tx=0x7f8b3002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.336+0000 7f8b407f5640 1 -- 172.21.15.16:0/2266033476 shutdown_connections 2024-04-25T10:12:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.336+0000 7f8b407f5640 1 --2- 172.21.15.16:0/2266033476 >> v2:172.21.15.186:3300/0 conn(0x7f8b380d1130 0x7f8b380b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.336+0000 7f8b407f5640 1 --2- 172.21.15.16:0/2266033476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b3815dca0 0x7f8b380ce940 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.336+0000 7f8b407f5640 1 -- 172.21.15.16:0/2266033476 >> 172.21.15.16:0/2266033476 conn(0x7f8b380726c0 msgr2=0x7f8b38072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.336+0000 7f8b407f5640 1 -- 172.21.15.16:0/2266033476 shutdown_connections 2024-04-25T10:12:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.336+0000 7f8b407f5640 1 -- 172.21.15.16:0/2266033476 wait complete. 2024-04-25T10:12:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.337+0000 7f8b407f5640 1 Processor -- start 2024-04-25T10:12:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.337+0000 7f8b407f5640 1 -- start start 2024-04-25T10:12:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b407f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b380d1130 0x7f8b380cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b407f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b380d3b90 0x7f8b381565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b407f5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b3815dca0 0x7f8b38153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b407f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3815e200 con 0x7f8b380d3b90 2024-04-25T10:12:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b407f5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3815e080 con 0x7f8b3815dca0 2024-04-25T10:12:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b407f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b380cee80 con 0x7f8b380d1130 2024-04-25T10:12:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b3dd90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b380d3b90 0x7f8b381565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b3dd90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b380d3b90 0x7f8b381565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60840/0 (socket says 172.21.15.16:60840) 2024-04-25T10:12:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b3dd90640 1 -- 172.21.15.16:0/2489441608 learned_addr learned my addr 172.21.15.16:0/2489441608 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b3e591640 1 --2- 172.21.15.16:0/2489441608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b380d1130 0x7f8b380cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b3dd90640 1 -- 172.21.15.16:0/2489441608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b380d1130 msgr2=0x7f8b380cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b3ed92640 1 --2- 172.21.15.16:0/2489441608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b3815dca0 0x7f8b38153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b3dd90640 1 --2- 172.21.15.16:0/2489441608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b380d1130 0x7f8b380cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b3dd90640 1 -- 172.21.15.16:0/2489441608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b3815dca0 msgr2=0x7f8b38153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b3dd90640 1 --2- 172.21.15.16:0/2489441608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b3815dca0 0x7f8b38153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b3dd90640 1 -- 172.21.15.16:0/2489441608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b38153540 con 0x7f8b380d3b90 2024-04-25T10:12:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b3e591640 1 --2- 172.21.15.16:0/2489441608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b380d1130 0x7f8b380cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b3ed92640 1 --2- 172.21.15.16:0/2489441608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b3815dca0 0x7f8b38153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b3dd90640 1 --2- 172.21.15.16:0/2489441608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b380d3b90 0x7f8b381565b0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00c4b0 tx=0x7f8b2c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.338+0000 7f8b36ffd640 1 -- 172.21.15.16:0/2489441608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c018070 con 0x7f8b380d3b90 2024-04-25T10:12:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.339+0000 7f8b36ffd640 1 -- 172.21.15.16:0/2489441608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c007500 con 0x7f8b380d3b90 2024-04-25T10:12:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.339+0000 7f8b407f5640 1 -- 172.21.15.16:0/2489441608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b380cc460 con 0x7f8b380d3b90 2024-04-25T10:12:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.339+0000 7f8b36ffd640 1 -- 172.21.15.16:0/2489441608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c004a20 con 0x7f8b380d3b90 2024-04-25T10:12:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.339+0000 7f8b407f5640 1 -- 172.21.15.16:0/2489441608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b380cca20 con 0x7f8b380d3b90 2024-04-25T10:12:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.339+0000 7f8b407f5640 1 -- 172.21.15.16:0/2489441608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b380750b0 con 0x7f8b380d3b90 2024-04-25T10:12:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.340+0000 7f8b36ffd640 1 -- 172.21.15.16:0/2489441608 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b2c01b070 con 0x7f8b380d3b90 2024-04-25T10:12:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.340+0000 7f8b36ffd640 1 --2- 172.21.15.16:0/2489441608 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8b2406b4d0 0x7f8b2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.340+0000 7f8b36ffd640 1 -- 172.21.15.16:0/2489441608 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b2c089530 con 0x7f8b380d3b90 2024-04-25T10:12:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.341+0000 7f8b3e591640 1 --2- 172.21.15.16:0/2489441608 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8b2406b4d0 0x7f8b2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.341+0000 7f8b3e591640 1 --2- 172.21.15.16:0/2489441608 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8b2406b4d0 0x7f8b2406d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8b30034000 tx=0x7f8b3002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.345+0000 7f8b36ffd640 1 -- 172.21.15.16:0/2489441608 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b2c058470 con 0x7f8b380d3b90 2024-04-25T10:12:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.522+0000 7f8b407f5640 1 -- 172.21.15.16:0/2489441608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8b380b4f90 con 0x7f8b380d3b90 2024-04-25T10:12:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.522+0000 7f8b36ffd640 1 -- 172.21.15.16:0/2489441608 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8b2c057d70 con 0x7f8b380d3b90 2024-04-25T10:12:57.527 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:12:57.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.524+0000 7f8b407f5640 1 -- 172.21.15.16:0/2489441608 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8b2406b4d0 msgr2=0x7f8b2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.524+0000 7f8b407f5640 1 --2- 172.21.15.16:0/2489441608 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8b2406b4d0 0x7f8b2406d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8b30034000 tx=0x7f8b3002e000 comp rx=0 tx=0).stop 2024-04-25T10:12:57.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.524+0000 7f8b407f5640 1 -- 172.21.15.16:0/2489441608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b380d3b90 msgr2=0x7f8b381565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.524+0000 7f8b407f5640 1 --2- 172.21.15.16:0/2489441608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b380d3b90 0x7f8b381565b0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00c4b0 tx=0x7f8b2c00c970 comp rx=0 tx=0).stop 2024-04-25T10:12:57.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.524+0000 7f8b407f5640 1 -- 172.21.15.16:0/2489441608 shutdown_connections 2024-04-25T10:12:57.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.524+0000 7f8b407f5640 1 --2- 172.21.15.16:0/2489441608 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8b2406b4d0 0x7f8b2406d990 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.524+0000 7f8b407f5640 1 --2- 172.21.15.16:0/2489441608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b3815dca0 0x7f8b38153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.524+0000 7f8b407f5640 1 --2- 172.21.15.16:0/2489441608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b380d3b90 0x7f8b381565b0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.524+0000 7f8b407f5640 1 --2- 172.21.15.16:0/2489441608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b380d1130 0x7f8b380cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.524+0000 7f8b407f5640 1 -- 172.21.15.16:0/2489441608 >> 172.21.15.16:0/2489441608 conn(0x7f8b380726c0 msgr2=0x7f8b380d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.524+0000 7f8b407f5640 1 -- 172.21.15.16:0/2489441608 shutdown_connections 2024-04-25T10:12:57.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.524+0000 7f8b407f5640 1 -- 172.21.15.16:0/2489441608 wait complete. 2024-04-25T10:12:57.542 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:57.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.644+0000 7f905f7ce640 1 Processor -- start 2024-04-25T10:12:57.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.644+0000 7f905f7ce640 1 -- start start 2024-04-25T10:12:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.645+0000 7f905f7ce640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f905815dc80 0x7f9058161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.645+0000 7f905f7ce640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90580b97c0 0x7f90580b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.645+0000 7f905f7ce640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90581627a0 con 0x7f90580b5a30 2024-04-25T10:12:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.645+0000 7f905f7ce640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90580b5ea0 con 0x7f90580b97c0 2024-04-25T10:12:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.645+0000 7f905f7ce640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90581621e0 con 0x7f905815dc80 2024-04-25T10:12:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.645+0000 7f905cd69640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90580b97c0 0x7f90580b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.645+0000 7f905dd6b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f90580b5a30 0x7f905806bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35880/0 (socket says 172.21.15.16:35880) 2024-04-25T10:12:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.645+0000 7f905dd6b640 1 -- 172.21.15.16:0/934411751 learned_addr learned my addr 172.21.15.16:0/934411751 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.645+0000 7f905d56a640 1 --2- 172.21.15.16:0/934411751 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f905815dc80 0x7f9058161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.645+0000 7f904ffff640 1 -- 172.21.15.16:0/934411751 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557448526 0 0) 0x7f90581627a0 con 0x7f90580b5a30 2024-04-25T10:12:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.645+0000 7f904ffff640 1 -- 172.21.15.16:0/934411751 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9044003520 con 0x7f90580b5a30 2024-04-25T10:12:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.645+0000 7f905cd69640 1 -- 172.21.15.16:0/934411751 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f905815dc80 msgr2=0x7f9058161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.645+0000 7f905cd69640 1 --2- 172.21.15.16:0/934411751 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f905815dc80 0x7f9058161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.645+0000 7f905cd69640 1 -- 172.21.15.16:0/934411751 >> v1:172.21.15.16:6789/0 conn(0x7f90580b5a30 legacy=0x7f905806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.645+0000 7f905cd69640 1 -- 172.21.15.16:0/934411751 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90581624c0 con 0x7f90580b97c0 2024-04-25T10:12:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.646+0000 7f905cd69640 1 --2- 172.21.15.16:0/934411751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90580b97c0 0x7f90580b54f0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f90480097a0 tx=0x7f904802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c36b63ee714b2de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.646+0000 7f904ffff640 1 -- 172.21.15.16:0/934411751 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048037070 con 0x7f90580b97c0 2024-04-25T10:12:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.646+0000 7f904ffff640 1 -- 172.21.15.16:0/934411751 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9048032440 con 0x7f90580b97c0 2024-04-25T10:12:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.646+0000 7f904ffff640 1 -- 172.21.15.16:0/934411751 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048032810 con 0x7f90580b97c0 2024-04-25T10:12:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.647+0000 7f905f7ce640 1 -- 172.21.15.16:0/934411751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90580b97c0 msgr2=0x7f90580b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.647+0000 7f905f7ce640 1 --2- 172.21.15.16:0/934411751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90580b97c0 0x7f90580b54f0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f90480097a0 tx=0x7f904802a180 comp rx=0 tx=0).stop 2024-04-25T10:12:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.647+0000 7f905f7ce640 1 -- 172.21.15.16:0/934411751 shutdown_connections 2024-04-25T10:12:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.647+0000 7f905f7ce640 1 --2- 172.21.15.16:0/934411751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90580b97c0 0x7f90580b54f0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.647+0000 7f905f7ce640 1 --2- 172.21.15.16:0/934411751 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f905815dc80 0x7f9058161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.647+0000 7f905f7ce640 1 -- 172.21.15.16:0/934411751 >> 172.21.15.16:0/934411751 conn(0x7f90580d04f0 msgr2=0x7f90580d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.647+0000 7f905f7ce640 1 -- 172.21.15.16:0/934411751 shutdown_connections 2024-04-25T10:12:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.647+0000 7f905f7ce640 1 -- 172.21.15.16:0/934411751 wait complete. 2024-04-25T10:12:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.648+0000 7f905f7ce640 1 Processor -- start 2024-04-25T10:12:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.648+0000 7f905f7ce640 1 -- start start 2024-04-25T10:12:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.649+0000 7f905f7ce640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90580b5a30 0x7f905815bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.649+0000 7f905f7ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90580b97c0 0x7f90580cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.649+0000 7f905f7ce640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f905815dc80 0x7f90580c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.649+0000 7f905f7ce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9058161db0 con 0x7f90580b97c0 2024-04-25T10:12:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.649+0000 7f905f7ce640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905815e060 con 0x7f90580b5a30 2024-04-25T10:12:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.649+0000 7f905f7ce640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9058161f30 con 0x7f905815dc80 2024-04-25T10:12:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.649+0000 7f905cd69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90580b97c0 0x7f90580cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.649+0000 7f905cd69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90580b97c0 0x7f90580cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60858/0 (socket says 172.21.15.16:60858) 2024-04-25T10:12:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.649+0000 7f905cd69640 1 -- 172.21.15.16:0/1592691592 learned_addr learned my addr 172.21.15.16:0/1592691592 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.649+0000 7f905cd69640 1 -- 172.21.15.16:0/1592691592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f905815dc80 msgr2=0x7f90580c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.649+0000 7f905cd69640 1 --2- 172.21.15.16:0/1592691592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f905815dc80 0x7f90580c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.649+0000 7f905cd69640 1 -- 172.21.15.16:0/1592691592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90580b5a30 msgr2=0x7f905815bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.649+0000 7f905cd69640 1 --2- 172.21.15.16:0/1592691592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90580b5a30 0x7f905815bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.649+0000 7f905cd69640 1 -- 172.21.15.16:0/1592691592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905815f7b0 con 0x7f90580b97c0 2024-04-25T10:12:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.649+0000 7f905cd69640 1 --2- 172.21.15.16:0/1592691592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90580b97c0 0x7f90580cd540 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f9048035040 tx=0x7f90480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.649+0000 7f904dffb640 1 -- 172.21.15.16:0/1592691592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048037040 con 0x7f90580b97c0 2024-04-25T10:12:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.649+0000 7f904dffb640 1 -- 172.21.15.16:0/1592691592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9048030500 con 0x7f90580b97c0 2024-04-25T10:12:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.649+0000 7f905f7ce640 1 -- 172.21.15.16:0/1592691592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905815f4d0 con 0x7f90580b97c0 2024-04-25T10:12:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.650+0000 7f904dffb640 1 -- 172.21.15.16:0/1592691592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048030910 con 0x7f90580b97c0 2024-04-25T10:12:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.650+0000 7f905f7ce640 1 -- 172.21.15.16:0/1592691592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90580cd020 con 0x7f90580b97c0 2024-04-25T10:12:57.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.651+0000 7f905f7ce640 1 -- 172.21.15.16:0/1592691592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90580750b0 con 0x7f90580b97c0 2024-04-25T10:12:57.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.651+0000 7f904dffb640 1 -- 172.21.15.16:0/1592691592 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9048044020 con 0x7f90580b97c0 2024-04-25T10:12:57.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.651+0000 7f904dffb640 1 --2- 172.21.15.16:0/1592691592 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f903c06b500 0x7f903c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.652+0000 7f904dffb640 1 -- 172.21.15.16:0/1592691592 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90480aa380 con 0x7f90580b97c0 2024-04-25T10:12:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.652+0000 7f905d56a640 1 --2- 172.21.15.16:0/1592691592 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f903c06b500 0x7f903c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.652+0000 7f905d56a640 1 --2- 172.21.15.16:0/1592691592 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f903c06b500 0x7f903c06d9c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9054006fd0 tx=0x7f9054008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.656+0000 7f904dffb640 1 -- 172.21.15.16:0/1592691592 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9048079290 con 0x7f90580b97c0 2024-04-25T10:12:57.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.835+0000 7f905f7ce640 1 -- 172.21.15.16:0/1592691592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f90580cd1f0 con 0x7f90580b97c0 2024-04-25T10:12:57.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.835+0000 7f904dffb640 1 -- 172.21.15.16:0/1592691592 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9048078b90 con 0x7f90580b97c0 2024-04-25T10:12:57.840 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:12:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.837+0000 7f905f7ce640 1 -- 172.21.15.16:0/1592691592 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f903c06b500 msgr2=0x7f903c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.837+0000 7f905f7ce640 1 --2- 172.21.15.16:0/1592691592 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f903c06b500 0x7f903c06d9c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9054006fd0 tx=0x7f9054008040 comp rx=0 tx=0).stop 2024-04-25T10:12:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.837+0000 7f905f7ce640 1 -- 172.21.15.16:0/1592691592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90580b97c0 msgr2=0x7f90580cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.837+0000 7f905f7ce640 1 --2- 172.21.15.16:0/1592691592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90580b97c0 0x7f90580cd540 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f9048035040 tx=0x7f90480046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.837+0000 7f905f7ce640 1 -- 172.21.15.16:0/1592691592 shutdown_connections 2024-04-25T10:12:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.837+0000 7f905f7ce640 1 --2- 172.21.15.16:0/1592691592 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f903c06b500 0x7f903c06d9c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.837+0000 7f905f7ce640 1 --2- 172.21.15.16:0/1592691592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f905815dc80 0x7f90580c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.837+0000 7f905f7ce640 1 --2- 172.21.15.16:0/1592691592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90580b97c0 0x7f90580cd540 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.837+0000 7f905f7ce640 1 --2- 172.21.15.16:0/1592691592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90580b5a30 0x7f905815bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.837+0000 7f905f7ce640 1 -- 172.21.15.16:0/1592691592 >> 172.21.15.16:0/1592691592 conn(0x7f90580d04f0 msgr2=0x7f90580b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.837+0000 7f905f7ce640 1 -- 172.21.15.16:0/1592691592 shutdown_connections 2024-04-25T10:12:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.837+0000 7f905f7ce640 1 -- 172.21.15.16:0/1592691592 wait complete. 2024-04-25T10:12:57.853 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:12:57.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.950+0000 7f1fdedb7640 1 Processor -- start 2024-04-25T10:12:57.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.950+0000 7f1fdedb7640 1 -- start start 2024-04-25T10:12:57.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.950+0000 7f1fdedb7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1fd80ceeb0 0x7f1fd8168a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.950+0000 7f1fdedb7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd8161f80 con 0x7f1fd8161090 2024-04-25T10:12:57.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.950+0000 7f1fdedb7640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd8162260 con 0x7f1fd80b9720 2024-04-25T10:12:57.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.950+0000 7f1fdedb7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd80d21a0 con 0x7f1fd80ceeb0 2024-04-25T10:12:57.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.951+0000 7f1fdcb53640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1fd8161090 0x7f1fd8161730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35892/0 (socket says 172.21.15.16:35892) 2024-04-25T10:12:57.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.951+0000 7f1fdcb53640 1 -- 172.21.15.16:0/1187402830 learned_addr learned my addr 172.21.15.16:0/1187402830 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:57.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.951+0000 7f1fdd354640 1 --2- 172.21.15.16:0/1187402830 >> v2:172.21.15.16:3300/0 conn(0x7f1fd80ceeb0 0x7f1fd8168a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.951+0000 7f1fd77fe640 1 -- 172.21.15.16:0/1187402830 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336977083 0 0) 0x7f1fd8161f80 con 0x7f1fd8161090 2024-04-25T10:12:57.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.951+0000 7f1fd77fe640 1 -- 172.21.15.16:0/1187402830 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fc8003520 con 0x7f1fd8161090 2024-04-25T10:12:57.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.951+0000 7f1fdd354640 1 -- 172.21.15.16:0/1187402830 >> v1:172.21.15.186:6789/0 conn(0x7f1fd80b9720 legacy=0x7f1fd80ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:57.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.951+0000 7f1fdd354640 1 -- 172.21.15.16:0/1187402830 >> v1:172.21.15.16:6789/0 conn(0x7f1fd8161090 legacy=0x7f1fd8161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.951+0000 7f1fdd354640 1 -- 172.21.15.16:0/1187402830 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd8162540 con 0x7f1fd80ceeb0 2024-04-25T10:12:57.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.952+0000 7f1fdd354640 1 --2- 172.21.15.16:0/1187402830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fd80ceeb0 0x7f1fd8168a60 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80d2320 tx=0x7f1fd002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144b11e347daecbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.952+0000 7f1fd77fe640 1 -- 172.21.15.16:0/1187402830 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0031600 con 0x7f1fd80ceeb0 2024-04-25T10:12:57.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.952+0000 7f1fd77fe640 1 -- 172.21.15.16:0/1187402830 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd00317a0 con 0x7f1fd80ceeb0 2024-04-25T10:12:57.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.952+0000 7f1fd77fe640 1 -- 172.21.15.16:0/1187402830 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0031b70 con 0x7f1fd80ceeb0 2024-04-25T10:12:57.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.952+0000 7f1fdedb7640 1 -- 172.21.15.16:0/1187402830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fd80ceeb0 msgr2=0x7f1fd8168a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.952+0000 7f1fdedb7640 1 --2- 172.21.15.16:0/1187402830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fd80ceeb0 0x7f1fd8168a60 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80d2320 tx=0x7f1fd002a610 comp rx=0 tx=0).stop 2024-04-25T10:12:57.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.952+0000 7f1fdedb7640 1 -- 172.21.15.16:0/1187402830 shutdown_connections 2024-04-25T10:12:57.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.952+0000 7f1fdedb7640 1 --2- 172.21.15.16:0/1187402830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fd80ceeb0 0x7f1fd8168a60 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.952+0000 7f1fdedb7640 1 -- 172.21.15.16:0/1187402830 >> 172.21.15.16:0/1187402830 conn(0x7f1fd80b55e0 msgr2=0x7f1fd80b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.952+0000 7f1fdedb7640 1 -- 172.21.15.16:0/1187402830 shutdown_connections 2024-04-25T10:12:57.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.952+0000 7f1fdedb7640 1 -- 172.21.15.16:0/1187402830 wait complete. 2024-04-25T10:12:57.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.954+0000 7f1fdedb7640 1 Processor -- start 2024-04-25T10:12:57.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.954+0000 7f1fdedb7640 1 -- start start 2024-04-25T10:12:57.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.954+0000 7f1fdedb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fd80b9720 0x7f1fd8155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.954+0000 7f1fdedb7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1fd80ceeb0 0x7f1fd80c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.954+0000 7f1fdedb7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fd8161090 0x7f1fd815bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.954+0000 7f1fdedb7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80d3640 con 0x7f1fd80b9720 2024-04-25T10:12:57.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.955+0000 7f1fdedb7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd8161470 con 0x7f1fd8161090 2024-04-25T10:12:57.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.955+0000 7f1fdedb7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80d37c0 con 0x7f1fd80ceeb0 2024-04-25T10:12:57.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.955+0000 7f1fdcb53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fd80b9720 0x7f1fd8155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.955+0000 7f1fdcb53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fd80b9720 0x7f1fd8155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60880/0 (socket says 172.21.15.16:60880) 2024-04-25T10:12:57.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.955+0000 7f1fdcb53640 1 -- 172.21.15.16:0/3014664038 learned_addr learned my addr 172.21.15.16:0/3014664038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:57.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.955+0000 7f1fdd354640 1 --2- 172.21.15.16:0/3014664038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fd8161090 0x7f1fd815bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.955+0000 7f1fd7fff640 1 --2- 172.21.15.16:0/3014664038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1fd80ceeb0 0x7f1fd80c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.955+0000 7f1fdcb53640 1 -- 172.21.15.16:0/3014664038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1fd80ceeb0 msgr2=0x7f1fd80c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.955+0000 7f1fdcb53640 1 --2- 172.21.15.16:0/3014664038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1fd80ceeb0 0x7f1fd80c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.955+0000 7f1fdcb53640 1 -- 172.21.15.16:0/3014664038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fd8161090 msgr2=0x7f1fd815bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.955+0000 7f1fdcb53640 1 --2- 172.21.15.16:0/3014664038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fd8161090 0x7f1fd815bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.955+0000 7f1fdcb53640 1 -- 172.21.15.16:0/3014664038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd81581f0 con 0x7f1fd80b9720 2024-04-25T10:12:57.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.955+0000 7f1fdcb53640 1 --2- 172.21.15.16:0/3014664038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fd80b9720 0x7f1fd8155d10 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00c430 tx=0x7f1fcc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.955+0000 7f1fd57fa640 1 -- 172.21.15.16:0/3014664038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc017070 con 0x7f1fd80b9720 2024-04-25T10:12:57.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.955+0000 7f1fd57fa640 1 -- 172.21.15.16:0/3014664038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fcc005ce0 con 0x7f1fd80b9720 2024-04-25T10:12:57.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.955+0000 7f1fdedb7640 1 -- 172.21.15.16:0/3014664038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd81584b0 con 0x7f1fd80b9720 2024-04-25T10:12:57.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.955+0000 7f1fd57fa640 1 -- 172.21.15.16:0/3014664038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc0072f0 con 0x7f1fd80b9720 2024-04-25T10:12:57.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.955+0000 7f1fdedb7640 1 -- 172.21.15.16:0/3014664038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd8158a70 con 0x7f1fd80b9720 2024-04-25T10:12:57.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.956+0000 7f1fdedb7640 1 -- 172.21.15.16:0/3014664038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd80750b0 con 0x7f1fd80b9720 2024-04-25T10:12:57.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.957+0000 7f1fd57fa640 1 -- 172.21.15.16:0/3014664038 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1fcc01a070 con 0x7f1fd80b9720 2024-04-25T10:12:57.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.957+0000 7f1fd57fa640 1 --2- 172.21.15.16:0/3014664038 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.957+0000 7f1fd57fa640 1 -- 172.21.15.16:0/3014664038 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fcc089770 con 0x7f1fd80b9720 2024-04-25T10:12:57.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.957+0000 7f1fd7fff640 1 --2- 172.21.15.16:0/3014664038 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.958+0000 7f1fd7fff640 1 --2- 172.21.15.16:0/3014664038 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4006fd0 tx=0x7f1fc4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:57.961+0000 7f1fd57fa640 1 -- 172.21.15.16:0/3014664038 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1fcc058560 con 0x7f1fd80b9720 2024-04-25T10:12:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.158+0000 7f1fdedb7640 1 -- 172.21.15.16:0/3014664038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1fd80d1700 con 0x7f1fd80b9720 2024-04-25T10:12:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.159+0000 7f1fd57fa640 1 -- 172.21.15.16:0/3014664038 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v100) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1fcc057e60 con 0x7f1fd80b9720 2024-04-25T10:12:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-04-25T10:12:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.161+0000 7f1fdedb7640 1 -- 172.21.15.16:0/3014664038 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1fbc06b4d0 msgr2=0x7f1fbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.161+0000 7f1fdedb7640 1 --2- 172.21.15.16:0/3014664038 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4006fd0 tx=0x7f1fc4008040 comp rx=0 tx=0).stop 2024-04-25T10:12:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.161+0000 7f1fdedb7640 1 -- 172.21.15.16:0/3014664038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fd80b9720 msgr2=0x7f1fd8155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.161+0000 7f1fdedb7640 1 --2- 172.21.15.16:0/3014664038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fd80b9720 0x7f1fd8155d10 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00c430 tx=0x7f1fcc00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.161+0000 7f1fdedb7640 1 -- 172.21.15.16:0/3014664038 shutdown_connections 2024-04-25T10:12:58.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.161+0000 7f1fdedb7640 1 --2- 172.21.15.16:0/3014664038 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.161+0000 7f1fdedb7640 1 --2- 172.21.15.16:0/3014664038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fd8161090 0x7f1fd815bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.161+0000 7f1fdedb7640 1 --2- 172.21.15.16:0/3014664038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1fd80ceeb0 0x7f1fd80c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.161+0000 7f1fdedb7640 1 --2- 172.21.15.16:0/3014664038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fd80b9720 0x7f1fd8155d10 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.161+0000 7f1fdedb7640 1 -- 172.21.15.16:0/3014664038 >> 172.21.15.16:0/3014664038 conn(0x7f1fd80b55e0 msgr2=0x7f1fd806c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.161+0000 7f1fdedb7640 1 -- 172.21.15.16:0/3014664038 shutdown_connections 2024-04-25T10:12:58.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.161+0000 7f1fdedb7640 1 -- 172.21.15.16:0/3014664038 wait complete. 2024-04-25T10:12:58.176 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:12:58.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:58.176 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:58.210 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:12:58.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:58.210 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:12:58.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:58.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.402+0000 7f455516a640 1 Processor -- start 2024-04-25T10:12:58.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.402+0000 7f455516a640 1 -- start start 2024-04-25T10:12:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.402+0000 7f455516a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f455015df40 0x7f45500cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.402+0000 7f455516a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f45500b19c0 0x7f45500cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.402+0000 7f455516a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45500cd930 con 0x7f45500cfe80 2024-04-25T10:12:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.402+0000 7f455516a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45500cc5b0 con 0x7f45500b19c0 2024-04-25T10:12:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.402+0000 7f455516a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45500cdc10 con 0x7f455015df40 2024-04-25T10:12:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.403+0000 7f454e59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f45500b19c0 0x7f45500cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.403+0000 7f454e59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f45500b19c0 0x7f45500cf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60894/0 (socket says 172.21.15.16:60894) 2024-04-25T10:12:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.403+0000 7f454e59c640 1 -- 172.21.15.16:0/3020369698 learned_addr learned my addr 172.21.15.16:0/3020369698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.403+0000 7f454ed9d640 1 --2- 172.21.15.16:0/3020369698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f455015df40 0x7f45500cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.403+0000 7f454e59c640 1 -- 172.21.15.16:0/3020369698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f455015df40 msgr2=0x7f45500cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.403+0000 7f454e59c640 1 --2- 172.21.15.16:0/3020369698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f455015df40 0x7f45500cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.403+0000 7f454e59c640 1 -- 172.21.15.16:0/3020369698 >> v1:172.21.15.186:6789/0 conn(0x7f45500cfe80 legacy=0x7f455006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.403+0000 7f454e59c640 1 -- 172.21.15.16:0/3020369698 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45500cdef0 con 0x7f45500b19c0 2024-04-25T10:12:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.404+0000 7f454e59c640 1 --2- 172.21.15.16:0/3020369698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45500b19c0 0x7f45500cf940 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f4544009870 tx=0x7f454402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78cc19af5c60774 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.404+0000 7f454dd9b640 1 -- 172.21.15.16:0/3020369698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4544037070 con 0x7f45500b19c0 2024-04-25T10:12:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.404+0000 7f454dd9b640 1 -- 172.21.15.16:0/3020369698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4544032440 con 0x7f45500b19c0 2024-04-25T10:12:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.404+0000 7f454dd9b640 1 -- 172.21.15.16:0/3020369698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4544032810 con 0x7f45500b19c0 2024-04-25T10:12:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.404+0000 7f455516a640 1 -- 172.21.15.16:0/3020369698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45500b19c0 msgr2=0x7f45500cf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.404+0000 7f455516a640 1 --2- 172.21.15.16:0/3020369698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45500b19c0 0x7f45500cf940 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f4544009870 tx=0x7f454402a180 comp rx=0 tx=0).stop 2024-04-25T10:12:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.404+0000 7f455516a640 1 -- 172.21.15.16:0/3020369698 shutdown_connections 2024-04-25T10:12:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.404+0000 7f455516a640 1 --2- 172.21.15.16:0/3020369698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45500b19c0 0x7f45500cf940 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.404+0000 7f455516a640 1 --2- 172.21.15.16:0/3020369698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f455015df40 0x7f45500cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.404+0000 7f455516a640 1 -- 172.21.15.16:0/3020369698 >> 172.21.15.16:0/3020369698 conn(0x7f45500d06b0 msgr2=0x7f45500d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.404+0000 7f455516a640 1 -- 172.21.15.16:0/3020369698 shutdown_connections 2024-04-25T10:12:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.404+0000 7f455516a640 1 -- 172.21.15.16:0/3020369698 wait complete. 2024-04-25T10:12:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.406+0000 7f455516a640 1 Processor -- start 2024-04-25T10:12:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.406+0000 7f455516a640 1 -- start start 2024-04-25T10:12:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.406+0000 7f455516a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45500b19c0 0x7f45500c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f455516a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45500cfe80 0x7f45500b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f455516a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f455015df40 0x7f45500b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f455516a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45500d0310 con 0x7f45500b19c0 2024-04-25T10:12:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f455516a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45500cb080 con 0x7f455015df40 2024-04-25T10:12:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f455516a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45500b1e20 con 0x7f45500cfe80 2024-04-25T10:12:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f454ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45500b19c0 0x7f45500c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f454ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45500b19c0 0x7f45500c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60898/0 (socket says 172.21.15.16:60898) 2024-04-25T10:12:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f454ed9d640 1 -- 172.21.15.16:0/3732630104 learned_addr learned my addr 172.21.15.16:0/3732630104 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f454ed9d640 1 -- 172.21.15.16:0/3732630104 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45500cfe80 msgr2=0x7f45500b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f454ed9d640 1 --2- 172.21.15.16:0/3732630104 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45500cfe80 0x7f45500b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f454ed9d640 1 -- 172.21.15.16:0/3732630104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f455015df40 msgr2=0x7f45500b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f454ed9d640 1 --2- 172.21.15.16:0/3732630104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f455015df40 0x7f45500b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f454ed9d640 1 -- 172.21.15.16:0/3732630104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45501502c0 con 0x7f45500b19c0 2024-04-25T10:12:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f454ed9d640 1 --2- 172.21.15.16:0/3732630104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45500b19c0 0x7f45500c9c10 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f454800d830 tx=0x7f454800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f45377fe640 1 -- 172.21.15.16:0/3732630104 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454800b660 con 0x7f45500b19c0 2024-04-25T10:12:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f45377fe640 1 -- 172.21.15.16:0/3732630104 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4548019070 con 0x7f45500b19c0 2024-04-25T10:12:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f455516a640 1 -- 172.21.15.16:0/3732630104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45500b32b0 con 0x7f45500b19c0 2024-04-25T10:12:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f45377fe640 1 -- 172.21.15.16:0/3732630104 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454800ba20 con 0x7f45500b19c0 2024-04-25T10:12:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.407+0000 7f455516a640 1 -- 172.21.15.16:0/3732630104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45500b3870 con 0x7f45500b19c0 2024-04-25T10:12:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.408+0000 7f455516a640 1 -- 172.21.15.16:0/3732630104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45500749b0 con 0x7f45500b19c0 2024-04-25T10:12:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.409+0000 7f45377fe640 1 -- 172.21.15.16:0/3732630104 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f454801e460 con 0x7f45500b19c0 2024-04-25T10:12:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.409+0000 7f45377fe640 1 --2- 172.21.15.16:0/3732630104 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f453c06b4d0 0x7f453c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.409+0000 7f45377fe640 1 -- 172.21.15.16:0/3732630104 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f454808a8a0 con 0x7f45500b19c0 2024-04-25T10:12:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.409+0000 7f454e59c640 1 --2- 172.21.15.16:0/3732630104 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f453c06b4d0 0x7f453c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.410+0000 7f454e59c640 1 --2- 172.21.15.16:0/3732630104 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f453c06b4d0 0x7f453c06d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4544034000 tx=0x7f454402c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.413+0000 7f45377fe640 1 -- 172.21.15.16:0/3732630104 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4548059720 con 0x7f45500b19c0 2024-04-25T10:12:58.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.632+0000 7f455516a640 1 -- 172.21.15.16:0/3732630104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f455006e3e0 con 0x7f45500b19c0 2024-04-25T10:12:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.632+0000 7f45377fe640 1 -- 172.21.15.16:0/3732630104 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f4548059020 con 0x7f45500b19c0 2024-04-25T10:12:58.637 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:58.637 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7243,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/109406141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":109406141},{"type":"v1","addr":"172.21.15.186:6837","nonce":109406141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2704401503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2704401503},{"type":"v1","addr":"172.21.15.16:6835","nonce":2704401503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/939221717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":939221717},{"type":"v1","addr":"172.21.15.16:6834","nonce":939221717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:48.958938+0000","modified":"2024-04-25T10:12:51.839702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8202},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8202":{"gid":8202,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6836/1371188714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1371188714},{"type":"v1","addr":"172.21.15.186:6836","nonce":1371188714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 71 2024-04-25T10:12:58.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.634+0000 7f455516a640 1 -- 172.21.15.16:0/3732630104 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f453c06b4d0 msgr2=0x7f453c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.634+0000 7f455516a640 1 --2- 172.21.15.16:0/3732630104 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f453c06b4d0 0x7f453c06d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4544034000 tx=0x7f454402c000 comp rx=0 tx=0).stop 2024-04-25T10:12:58.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.634+0000 7f455516a640 1 -- 172.21.15.16:0/3732630104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45500b19c0 msgr2=0x7f45500c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.634+0000 7f455516a640 1 --2- 172.21.15.16:0/3732630104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45500b19c0 0x7f45500c9c10 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f454800d830 tx=0x7f454800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.634+0000 7f455516a640 1 -- 172.21.15.16:0/3732630104 shutdown_connections 2024-04-25T10:12:58.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.634+0000 7f455516a640 1 --2- 172.21.15.16:0/3732630104 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f453c06b4d0 0x7f453c06d990 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.634+0000 7f455516a640 1 --2- 172.21.15.16:0/3732630104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f455015df40 0x7f45500b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.634+0000 7f455516a640 1 --2- 172.21.15.16:0/3732630104 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45500cfe80 0x7f45500b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.634+0000 7f455516a640 1 --2- 172.21.15.16:0/3732630104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45500b19c0 0x7f45500c9c10 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.634+0000 7f455516a640 1 -- 172.21.15.16:0/3732630104 >> 172.21.15.16:0/3732630104 conn(0x7f45500d06b0 msgr2=0x7f45500c6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.634+0000 7f455516a640 1 -- 172.21.15.16:0/3732630104 shutdown_connections 2024-04-25T10:12:58.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.634+0000 7f455516a640 1 -- 172.21.15.16:0/3732630104 wait complete. 2024-04-25T10:12:58.651 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-25T10:12:58.787 INFO:teuthology.orchestra.run.smithi186.stdout:{"message":"","return_code":0} 2024-04-25T10:12:58.788 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:12:58.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.884+0000 7f80b5e5d640 1 Processor -- start 2024-04-25T10:12:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.884+0000 7f80b5e5d640 1 -- start start 2024-04-25T10:12:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.885+0000 7f80b5e5d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80b015dce0 0x7f80b00d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.885+0000 7f80b5e5d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b015e0c0 con 0x7f80b00b5b60 2024-04-25T10:12:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.885+0000 7f80b5e5d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b00d2510 con 0x7f80b00d35b0 2024-04-25T10:12:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.885+0000 7f80b5e5d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00d27f0 con 0x7f80b015dce0 2024-04-25T10:12:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.885+0000 7f80affff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f80b00b5b60 0x7f80b00b6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35894/0 (socket says 172.21.15.16:35894) 2024-04-25T10:12:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.885+0000 7f80affff640 1 -- 172.21.15.16:0/2883492358 learned_addr learned my addr 172.21.15.16:0/2883492358 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80af7fe640 1 --2- 172.21.15.16:0/2883492358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80b015dce0 0x7f80b00d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80ae7fc640 1 -- 172.21.15.16:0/2883492358 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617432778 0 0) 0x7f80b015e0c0 con 0x7f80b00b5b60 2024-04-25T10:12:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80ae7fc640 1 -- 172.21.15.16:0/2883492358 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8098003520 con 0x7f80b00b5b60 2024-04-25T10:12:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80ae7fc640 1 -- 172.21.15.16:0/2883492358 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3046103991 0 0) 0x7f8098003520 con 0x7f80b00b5b60 2024-04-25T10:12:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80ae7fc640 1 -- 172.21.15.16:0/2883492358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80b015dce0 msgr2=0x7f80b00d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80ae7fc640 1 --2- 172.21.15.16:0/2883492358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80b015dce0 0x7f80b00d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80ae7fc640 1 -- 172.21.15.16:0/2883492358 >> v1:172.21.15.186:6789/0 conn(0x7f80b00d35b0 legacy=0x7f80b00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80ae7fc640 1 -- 172.21.15.16:0/2883492358 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b00d2ad0 con 0x7f80b00b5b60 2024-04-25T10:12:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80ae7fc640 1 -- 172.21.15.16:0/2883492358 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f80a80019f0 con 0x7f80b00b5b60 2024-04-25T10:12:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80ae7fc640 1 -- 172.21.15.16:0/2883492358 >> v1:172.21.15.16:6789/0 conn(0x7f80b00b5b60 legacy=0x7f80b00b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80ae7fc640 1 --2- 172.21.15.16:0/2883492358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80b015dce0 0x7f80980039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80ae7fc640 1 --2- 172.21.15.16:0/2883492358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8098003f60 0x7f8098004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80ae7fc640 1 --2- 172.21.15.16:0/2883492358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8098004d40 0x7f80980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80ae7fc640 1 -- 172.21.15.16:0/2883492358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00d27f0 con 0x7f8098003f60 2024-04-25T10:12:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80ae7fc640 1 -- 172.21.15.16:0/2883492358 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809800baa0 con 0x7f8098004d40 2024-04-25T10:12:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80ae7fc640 1 -- 172.21.15.16:0/2883492358 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809800bca0 con 0x7f80b015dce0 2024-04-25T10:12:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80aeffd640 1 --2- 172.21.15.16:0/2883492358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8098003f60 0x7f8098004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80af7fe640 1 --2- 172.21.15.16:0/2883492358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80b015dce0 0x7f80980039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80affff640 1 --2- 172.21.15.16:0/2883492358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8098004d40 0x7f80980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80aeffd640 1 -- 172.21.15.16:0/2883492358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80b015dce0 msgr2=0x7f80980039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80aeffd640 1 --2- 172.21.15.16:0/2883492358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80b015dce0 0x7f80980039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80aeffd640 1 -- 172.21.15.16:0/2883492358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8098004d40 msgr2=0x7f80980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80aeffd640 1 --2- 172.21.15.16:0/2883492358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8098004d40 0x7f80980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.886+0000 7f80aeffd640 1 -- 172.21.15.16:0/2883492358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8098003520 con 0x7f8098003f60 2024-04-25T10:12:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.887+0000 7f80aeffd640 1 --2- 172.21.15.16:0/2883492358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8098003f60 0x7f8098004800 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f80a4001750 tx=0x7f8098005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637514eda98f436a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.887+0000 7f80ae7fc640 1 -- 172.21.15.16:0/2883492358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4001370 con 0x7f8098003f60 2024-04-25T10:12:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.887+0000 7f80ae7fc640 1 -- 172.21.15.16:0/2883492358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a40052d0 con 0x7f8098003f60 2024-04-25T10:12:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.887+0000 7f80ae7fc640 1 -- 172.21.15.16:0/2883492358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a402abe0 con 0x7f8098003f60 2024-04-25T10:12:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.888+0000 7f80b5e5d640 1 -- 172.21.15.16:0/2883492358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8098003f60 msgr2=0x7f8098004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.888+0000 7f80b5e5d640 1 --2- 172.21.15.16:0/2883492358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8098003f60 0x7f8098004800 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f80a4001750 tx=0x7f8098005830 comp rx=0 tx=0).stop 2024-04-25T10:12:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.888+0000 7f80affff640 1 -- 172.21.15.16:0/2883492358 reap_dead start 2024-04-25T10:12:58.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.888+0000 7f80b5e5d640 1 -- 172.21.15.16:0/2883492358 shutdown_connections 2024-04-25T10:12:58.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.888+0000 7f80b5e5d640 1 -- 172.21.15.16:0/2883492358 >> 172.21.15.16:0/2883492358 conn(0x7f80b0072900 msgr2=0x7f80b00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.888+0000 7f80b5e5d640 1 -- 172.21.15.16:0/2883492358 shutdown_connections 2024-04-25T10:12:58.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.888+0000 7f80b5e5d640 1 -- 172.21.15.16:0/2883492358 wait complete. 2024-04-25T10:12:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.889+0000 7f80b5e5d640 1 Processor -- start 2024-04-25T10:12:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.889+0000 7f80b5e5d640 1 -- start start 2024-04-25T10:12:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.890+0000 7f80b5e5d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8098003f60 0x7f80b0156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.890+0000 7f80b5e5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80b0153bd0 0x7f80b0154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.890+0000 7f80b5e5d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80b00cbaf0 0x7f80b00cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.890+0000 7f80b5e5d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00b9820 con 0x7f80b0153bd0 2024-04-25T10:12:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.890+0000 7f80b5e5d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00d3090 con 0x7f8098003f60 2024-04-25T10:12:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.890+0000 7f80b5e5d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00b99a0 con 0x7f80b00cbaf0 2024-04-25T10:12:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.890+0000 7f80aeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80b0153bd0 0x7f80b0154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.890+0000 7f80aeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80b0153bd0 0x7f80b0154030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60908/0 (socket says 172.21.15.16:60908) 2024-04-25T10:12:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.890+0000 7f80aeffd640 1 -- 172.21.15.16:0/314052811 learned_addr learned my addr 172.21.15.16:0/314052811 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.890+0000 7f80affff640 1 --2- 172.21.15.16:0/314052811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80b00cbaf0 0x7f80b00cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.890+0000 7f80af7fe640 1 --2- 172.21.15.16:0/314052811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8098003f60 0x7f80b0156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.890+0000 7f80aeffd640 1 -- 172.21.15.16:0/314052811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80b00cbaf0 msgr2=0x7f80b00cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.890+0000 7f80aeffd640 1 --2- 172.21.15.16:0/314052811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80b00cbaf0 0x7f80b00cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.890+0000 7f80aeffd640 1 -- 172.21.15.16:0/314052811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8098003f60 msgr2=0x7f80b0156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.890+0000 7f80aeffd640 1 --2- 172.21.15.16:0/314052811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8098003f60 0x7f80b0156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.890+0000 7f80aeffd640 1 -- 172.21.15.16:0/314052811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b00cc7e0 con 0x7f80b0153bd0 2024-04-25T10:12:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.890+0000 7f80af7fe640 1 --2- 172.21.15.16:0/314052811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8098003f60 0x7f80b0156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.890+0000 7f80affff640 1 --2- 172.21.15.16:0/314052811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80b00cbaf0 0x7f80b00cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.891+0000 7f80aeffd640 1 --2- 172.21.15.16:0/314052811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80b0153bd0 0x7f80b0154030 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f80a4005040 tx=0x7f80a4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.891+0000 7f8097fff640 1 -- 172.21.15.16:0/314052811 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4042070 con 0x7f80b0153bd0 2024-04-25T10:12:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.891+0000 7f8097fff640 1 -- 172.21.15.16:0/314052811 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a402be50 con 0x7f80b0153bd0 2024-04-25T10:12:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.891+0000 7f80b5e5d640 1 -- 172.21.15.16:0/314052811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b00ccac0 con 0x7f80b0153bd0 2024-04-25T10:12:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.891+0000 7f8097fff640 1 -- 172.21.15.16:0/314052811 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4037040 con 0x7f80b0153bd0 2024-04-25T10:12:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.891+0000 7f80b5e5d640 1 -- 172.21.15.16:0/314052811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b00cd080 con 0x7f80b0153bd0 2024-04-25T10:12:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.892+0000 7f80b5e5d640 1 -- 172.21.15.16:0/314052811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b00749b0 con 0x7f80b0153bd0 2024-04-25T10:12:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.892+0000 7f8097fff640 1 -- 172.21.15.16:0/314052811 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f80a4045070 con 0x7f80b0153bd0 2024-04-25T10:12:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.893+0000 7f8097fff640 1 --2- 172.21.15.16:0/314052811 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f809c06b500 0x7f809c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.893+0000 7f8097fff640 1 -- 172.21.15.16:0/314052811 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80a40a9990 con 0x7f80b0153bd0 2024-04-25T10:12:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.893+0000 7f80af7fe640 1 --2- 172.21.15.16:0/314052811 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f809c06b500 0x7f809c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.893+0000 7f80af7fe640 1 --2- 172.21.15.16:0/314052811 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f809c06b500 0x7f809c06d9c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009ab0 tx=0x7f80a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:58.897+0000 7f8097fff640 1 -- 172.21.15.16:0/314052811 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f80a40787f0 con 0x7f80b0153bd0 2024-04-25T10:12:59.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.113+0000 7f80b5e5d640 1 -- 172.21.15.16:0/314052811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80b00b6da0 con 0x7f80b0153bd0 2024-04-25T10:12:59.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.114+0000 7f8097fff640 1 -- 172.21.15.16:0/314052811 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f80a40780f0 con 0x7f80b0153bd0 2024-04-25T10:12:59.118 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:12:59.118 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7243,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/109406141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":109406141},{"type":"v1","addr":"172.21.15.186:6837","nonce":109406141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2704401503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2704401503},{"type":"v1","addr":"172.21.15.16:6835","nonce":2704401503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/939221717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":939221717},{"type":"v1","addr":"172.21.15.16:6834","nonce":939221717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:48.958938+0000","modified":"2024-04-25T10:12:51.839702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8202},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8202":{"gid":8202,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6836/1371188714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1371188714},{"type":"v1","addr":"172.21.15.186:6836","nonce":1371188714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:59.119 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 71 2024-04-25T10:12:59.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.115+0000 7f80b5e5d640 1 -- 172.21.15.16:0/314052811 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f809c06b500 msgr2=0x7f809c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.115+0000 7f80b5e5d640 1 --2- 172.21.15.16:0/314052811 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f809c06b500 0x7f809c06d9c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009ab0 tx=0x7f80a0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:59.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.115+0000 7f80b5e5d640 1 -- 172.21.15.16:0/314052811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80b0153bd0 msgr2=0x7f80b0154030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.115+0000 7f80b5e5d640 1 --2- 172.21.15.16:0/314052811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80b0153bd0 0x7f80b0154030 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f80a4005040 tx=0x7f80a4001120 comp rx=0 tx=0).stop 2024-04-25T10:12:59.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.115+0000 7f80b5e5d640 1 -- 172.21.15.16:0/314052811 shutdown_connections 2024-04-25T10:12:59.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.115+0000 7f80b5e5d640 1 --2- 172.21.15.16:0/314052811 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f809c06b500 0x7f809c06d9c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.115+0000 7f80b5e5d640 1 --2- 172.21.15.16:0/314052811 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80b00cbaf0 0x7f80b00cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.116+0000 7f80b5e5d640 1 --2- 172.21.15.16:0/314052811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80b0153bd0 0x7f80b0154030 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.116+0000 7f80b5e5d640 1 --2- 172.21.15.16:0/314052811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8098003f60 0x7f80b0156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.116+0000 7f80b5e5d640 1 -- 172.21.15.16:0/314052811 >> 172.21.15.16:0/314052811 conn(0x7f80b0072900 msgr2=0x7f80b006bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:59.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.116+0000 7f80b5e5d640 1 -- 172.21.15.16:0/314052811 shutdown_connections 2024-04-25T10:12:59.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.116+0000 7f80b5e5d640 1 -- 172.21.15.16:0/314052811 wait complete. 2024-04-25T10:12:59.131 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-04-25T10:12:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.170+0000 7f3b88a11f40 1 Processor -- start 2024-04-25T10:12:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.170+0000 7f3b88a11f40 1 -- start start 2024-04-25T10:12:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.170+0000 7f3b88a11f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55774ff062a0 0x55774ff95c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.170+0000 7f3b88a11f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774ff08690 0x55774ff028f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.171+0000 7f3b88a11f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55774ff79270 0x55774ff02e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.171+0000 7f3b88a11f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774ff08ea0 con 0x55774ff062a0 2024-04-25T10:12:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.171+0000 7f3b88a11f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774fe65df0 con 0x55774ff08690 2024-04-25T10:12:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.171+0000 7f3b88a11f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55774ff09140 con 0x55774ff79270 2024-04-25T10:12:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.171+0000 7f3b8589f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55774ff062a0 0x55774ff95c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.171+0000 7f3b8589f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55774ff062a0 0x55774ff95c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60912/0 (socket says 172.21.15.16:60912) 2024-04-25T10:12:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.171+0000 7f3b8589f640 1 -- 172.21.15.16:0/2377091361 learned_addr learned my addr 172.21.15.16:0/2377091361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.171+0000 7f3b8589f640 1 -- 172.21.15.16:0/2377091361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55774ff79270 msgr2=0x55774ff02e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.171+0000 7f3b860a0640 1 --2- 172.21.15.16:0/2377091361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55774ff79270 0x55774ff02e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.171+0000 7f3b8509e640 1 --2- 172.21.15.16:0/2377091361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774ff08690 0x55774ff028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.171+0000 7f3b8589f640 1 --2- 172.21.15.16:0/2377091361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55774ff79270 0x55774ff02e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.171+0000 7f3b8589f640 1 -- 172.21.15.16:0/2377091361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774ff08690 msgr2=0x55774ff028f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.171+0000 7f3b8589f640 1 --2- 172.21.15.16:0/2377091361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774ff08690 0x55774ff028f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.171+0000 7f3b8589f640 1 -- 172.21.15.16:0/2377091361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55774fdebc50 con 0x55774ff062a0 2024-04-25T10:12:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.171+0000 7f3b8589f640 1 --2- 172.21.15.16:0/2377091361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55774ff062a0 0x55774ff95c80 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f3b80057040 tx=0x7f3b8004e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0f7e904a41555d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.171+0000 7f3b7ffff640 1 -- 172.21.15.16:0/2377091361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80065070 con 0x55774ff062a0 2024-04-25T10:12:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.171+0000 7f3b7ffff640 1 -- 172.21.15.16:0/2377091361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b800549f0 con 0x55774ff062a0 2024-04-25T10:12:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.171+0000 7f3b7ffff640 1 -- 172.21.15.16:0/2377091361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80059040 con 0x55774ff062a0 2024-04-25T10:12:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.172+0000 7f3b88a11f40 1 -- 172.21.15.16:0/2377091361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55774ff062a0 msgr2=0x55774ff95c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.172+0000 7f3b88a11f40 1 --2- 172.21.15.16:0/2377091361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55774ff062a0 0x55774ff95c80 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f3b80057040 tx=0x7f3b8004e000 comp rx=0 tx=0).stop 2024-04-25T10:12:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.172+0000 7f3b88a11f40 1 -- 172.21.15.16:0/2377091361 shutdown_connections 2024-04-25T10:12:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.172+0000 7f3b88a11f40 1 --2- 172.21.15.16:0/2377091361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55774ff79270 0x55774ff02e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.172+0000 7f3b88a11f40 1 --2- 172.21.15.16:0/2377091361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774ff08690 0x55774ff028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.172+0000 7f3b88a11f40 1 --2- 172.21.15.16:0/2377091361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55774ff062a0 0x55774ff95c80 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.172+0000 7f3b88a11f40 1 -- 172.21.15.16:0/2377091361 >> 172.21.15.16:0/2377091361 conn(0x55774fee93d0 msgr2=0x55774feea040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.172+0000 7f3b88a11f40 1 -- 172.21.15.16:0/2377091361 shutdown_connections 2024-04-25T10:12:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.172+0000 7f3b88a11f40 1 -- 172.21.15.16:0/2377091361 wait complete. 2024-04-25T10:12:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.172+0000 7f3b88a11f40 1 Processor -- start 2024-04-25T10:12:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.173+0000 7f3b88a11f40 1 -- start start 2024-04-25T10:12:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.173+0000 7f3b88a11f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774ff062a0 0x55774ff8f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.173+0000 7f3b88a11f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55774ff08690 0x557750026170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.173+0000 7f3b88a11f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55774ff79270 0x5577500266b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.173+0000 7f3b88a11f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774ff961c0 con 0x55774ff08690 2024-04-25T10:12:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.173+0000 7f3b88a11f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774fee8eb0 con 0x55774ff062a0 2024-04-25T10:12:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.173+0000 7f3b88a11f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55774ff05770 con 0x55774ff79270 2024-04-25T10:12:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.173+0000 7f3b8509e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55774ff08690 0x557750026170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.173+0000 7f3b8509e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55774ff08690 0x557750026170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60924/0 (socket says 172.21.15.16:60924) 2024-04-25T10:12:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.173+0000 7f3b8509e640 1 -- 172.21.15.16:0/915252777 learned_addr learned my addr 172.21.15.16:0/915252777 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.173+0000 7f3b860a0640 1 --2- 172.21.15.16:0/915252777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55774ff79270 0x5577500266b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.173+0000 7f3b8589f640 1 --2- 172.21.15.16:0/915252777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774ff062a0 0x55774ff8f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.174+0000 7f3b8509e640 1 -- 172.21.15.16:0/915252777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55774ff79270 msgr2=0x5577500266b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.174+0000 7f3b8509e640 1 --2- 172.21.15.16:0/915252777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55774ff79270 0x5577500266b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.174+0000 7f3b8509e640 1 -- 172.21.15.16:0/915252777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774ff062a0 msgr2=0x55774ff8f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.174+0000 7f3b8509e640 1 --2- 172.21.15.16:0/915252777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55774ff062a0 0x55774ff8f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.174+0000 7f3b8509e640 1 -- 172.21.15.16:0/915252777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55774feedf20 con 0x55774ff08690 2024-04-25T10:12:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.174+0000 7f3b8509e640 1 --2- 172.21.15.16:0/915252777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55774ff08690 0x557750026170 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x55774feef670 tx=0x7f3b70008400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.174+0000 7f3b7effd640 1 -- 172.21.15.16:0/915252777 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b700145f0 con 0x55774ff08690 2024-04-25T10:12:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.174+0000 7f3b7effd640 1 -- 172.21.15.16:0/915252777 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b70019070 con 0x55774ff08690 2024-04-25T10:12:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.174+0000 7f3b88a11f40 1 -- 172.21.15.16:0/915252777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557750024980 con 0x55774ff08690 2024-04-25T10:12:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.174+0000 7f3b7effd640 1 -- 172.21.15.16:0/915252777 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b700149f0 con 0x55774ff08690 2024-04-25T10:12:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.174+0000 7f3b88a11f40 1 -- 172.21.15.16:0/915252777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557750024f40 con 0x55774ff08690 2024-04-25T10:12:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.175+0000 7f3b7effd640 1 -- 172.21.15.16:0/915252777 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b70014b90 con 0x55774ff08690 2024-04-25T10:12:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.176+0000 7f3b7effd640 1 --2- 172.21.15.16:0/915252777 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b68069eb0 0x7f3b6806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.176+0000 7f3b7effd640 1 -- 172.21.15.16:0/915252777 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b70088d40 con 0x55774ff08690 2024-04-25T10:12:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.176+0000 7f3b88a11f40 1 --2- 172.21.15.16:0/915252777 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55774ff77170 0x55774fee9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.176+0000 7f3b88a11f40 1 -- 172.21.15.16:0/915252777 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) v9 -- 0x557750025740 con 0x55774ff77170 2024-04-25T10:12:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.176+0000 7f3b8589f640 1 --2- 172.21.15.16:0/915252777 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b68069eb0 0x7f3b6806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.176+0000 7f3b860a0640 1 --2- 172.21.15.16:0/915252777 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55774ff77170 0x55774fee9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.177+0000 7f3b8589f640 1 --2- 172.21.15.16:0/915252777 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b68069eb0 0x7f3b6806c370 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3b80002000 tx=0x7f3b80051000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.177+0000 7f3b860a0640 1 --2- 172.21.15.16:0/915252777 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55774ff77170 0x55774fee9cc0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.177+0000 7f3b860a0640 1 -- 172.21.15.16:0/915252777 <== osd.5 v2:172.21.15.186:6800/1827108421 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x7f3b78001920 con 0x55774ff77170 2024-04-25T10:12:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.177+0000 7f3b88a11f40 1 -- 172.21.15.16:0/915252777 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55774ff77170 msgr2=0x55774fee9cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.177+0000 7f3b88a11f40 1 --2- 172.21.15.16:0/915252777 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55774ff77170 0x55774fee9cc0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.177+0000 7f3b88a11f40 1 -- 172.21.15.16:0/915252777 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b68069eb0 msgr2=0x7f3b6806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.177+0000 7f3b88a11f40 1 --2- 172.21.15.16:0/915252777 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b68069eb0 0x7f3b6806c370 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3b80002000 tx=0x7f3b80051000 comp rx=0 tx=0).stop 2024-04-25T10:12:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.177+0000 7f3b88a11f40 1 -- 172.21.15.16:0/915252777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55774ff08690 msgr2=0x557750026170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.177+0000 7f3b88a11f40 1 --2- 172.21.15.16:0/915252777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55774ff08690 0x557750026170 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x55774feef670 tx=0x7f3b70008400 comp rx=0 tx=0).stop 2024-04-25T10:12:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.177+0000 7f3b860a0640 1 -- 172.21.15.16:0/915252777 reap_dead start 2024-04-25T10:12:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.177+0000 7f3b88a11f40 1 -- 172.21.15.16:0/915252777 shutdown_connections 2024-04-25T10:12:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.177+0000 7f3b88a11f40 1 -- 172.21.15.16:0/915252777 >> 172.21.15.16:0/915252777 conn(0x55774fee93d0 msgr2=0x55774ff028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.177+0000 7f3b88a11f40 1 -- 172.21.15.16:0/915252777 shutdown_connections 2024-04-25T10:12:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.177+0000 7f3b88a11f40 1 -- 172.21.15.16:0/915252777 wait complete. 2024-04-25T10:12:59.191 INFO:teuthology.orchestra.run.smithi016.stdout:����� 2024-04-25T10:12:59.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:12:59.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.327+0000 7f7ed366f640 1 Processor -- start 2024-04-25T10:12:59.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.327+0000 7f7ed366f640 1 -- start start 2024-04-25T10:12:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.328+0000 7f7ed366f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc0b5b60 0x7f7ecc1687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.328+0000 7f7ed366f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ecc15e0c0 con 0x7f7ecc15dce0 2024-04-25T10:12:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.328+0000 7f7ed366f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ecc0d2510 con 0x7f7ecc0d35b0 2024-04-25T10:12:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.328+0000 7f7ed366f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc0b7b10 con 0x7f7ecc0b5b60 2024-04-25T10:12:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.328+0000 7f7ed140b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7ecc15dce0 0x7f7ecc0d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35912/0 (socket says 172.21.15.16:35912) 2024-04-25T10:12:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.328+0000 7f7ed140b640 1 -- 172.21.15.16:0/4258220122 learned_addr learned my addr 172.21.15.16:0/4258220122 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:12:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.328+0000 7f7ed1c0c640 1 --2- 172.21.15.16:0/4258220122 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc0b5b60 0x7f7ecc1687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.328+0000 7f7ec3fff640 1 -- 172.21.15.16:0/4258220122 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599387505 0 0) 0x7f7ecc15e0c0 con 0x7f7ecc15dce0 2024-04-25T10:12:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.328+0000 7f7ec3fff640 1 -- 172.21.15.16:0/4258220122 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eb8003520 con 0x7f7ecc15dce0 2024-04-25T10:12:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.328+0000 7f7ec3fff640 1 -- 172.21.15.16:0/4258220122 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 899755131 0 0) 0x7f7eb8003520 con 0x7f7ecc15dce0 2024-04-25T10:12:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.328+0000 7f7ec3fff640 1 -- 172.21.15.16:0/4258220122 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc0b5b60 msgr2=0x7f7ecc1687f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.328+0000 7f7ec3fff640 1 --2- 172.21.15.16:0/4258220122 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc0b5b60 0x7f7ecc1687f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.328+0000 7f7ec3fff640 1 -- 172.21.15.16:0/4258220122 >> v1:172.21.15.186:6789/0 conn(0x7f7ecc0d35b0 legacy=0x7f7ecc0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.328+0000 7f7ec3fff640 1 -- 172.21.15.16:0/4258220122 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc0d27f0 con 0x7f7ecc15dce0 2024-04-25T10:12:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.328+0000 7f7ec3fff640 1 -- 172.21.15.16:0/4258220122 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f7ec8001670 con 0x7f7ecc15dce0 2024-04-25T10:12:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.328+0000 7f7ed1c0c640 1 --2- 172.21.15.16:0/4258220122 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc0b5b60 0x7f7ecc1687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.328+0000 7f7ec3fff640 1 -- 172.21.15.16:0/4258220122 >> v1:172.21.15.16:6789/0 conn(0x7f7ecc15dce0 legacy=0x7f7ecc0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.329+0000 7f7ec3fff640 1 --2- 172.21.15.16:0/4258220122 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc0b5b60 0x7f7eb80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.329+0000 7f7ec3fff640 1 --2- 172.21.15.16:0/4258220122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7eb8003f20 0x7f7eb8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.329+0000 7f7ec3fff640 1 --2- 172.21.15.16:0/4258220122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eb8004d40 0x7f7eb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.329+0000 7f7ec3fff640 1 -- 172.21.15.16:0/4258220122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc0b7b10 con 0x7f7eb8004d40 2024-04-25T10:12:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.329+0000 7f7ec3fff640 1 -- 172.21.15.16:0/4258220122 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb800ba20 con 0x7f7eb8003f20 2024-04-25T10:12:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.329+0000 7f7ec3fff640 1 -- 172.21.15.16:0/4258220122 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb800bc20 con 0x7f7ecc0b5b60 2024-04-25T10:12:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.329+0000 7f7ed1c0c640 1 --2- 172.21.15.16:0/4258220122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eb8004d40 0x7f7eb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.329+0000 7f7ed140b640 1 --2- 172.21.15.16:0/4258220122 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc0b5b60 0x7f7eb80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.329+0000 7f7ed0c0a640 1 --2- 172.21.15.16:0/4258220122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7eb8003f20 0x7f7eb8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.329+0000 7f7ed1c0c640 1 -- 172.21.15.16:0/4258220122 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc0b5b60 msgr2=0x7f7eb80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.329+0000 7f7ed1c0c640 1 --2- 172.21.15.16:0/4258220122 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc0b5b60 0x7f7eb80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.329+0000 7f7ed1c0c640 1 -- 172.21.15.16:0/4258220122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7eb8003f20 msgr2=0x7f7eb8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.329+0000 7f7ed1c0c640 1 --2- 172.21.15.16:0/4258220122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7eb8003f20 0x7f7eb8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.329+0000 7f7ed1c0c640 1 -- 172.21.15.16:0/4258220122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb8003520 con 0x7f7eb8004d40 2024-04-25T10:12:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.330+0000 7f7ed1c0c640 1 --2- 172.21.15.16:0/4258220122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eb8004d40 0x7f7eb80051c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400e350 tx=0x7f7eb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d9664fdc338dac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.330+0000 7f7ec3fff640 1 -- 172.21.15.16:0/4258220122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4009da0 con 0x7f7eb8004d40 2024-04-25T10:12:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.330+0000 7f7ec3fff640 1 -- 172.21.15.16:0/4258220122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec4003b90 con 0x7f7eb8004d40 2024-04-25T10:12:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.330+0000 7f7ec3fff640 1 -- 172.21.15.16:0/4258220122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4004030 con 0x7f7eb8004d40 2024-04-25T10:12:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.330+0000 7f7ed366f640 1 -- 172.21.15.16:0/4258220122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eb8004d40 msgr2=0x7f7eb80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.330+0000 7f7ed366f640 1 --2- 172.21.15.16:0/4258220122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eb8004d40 0x7f7eb80051c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400e350 tx=0x7f7eb8005830 comp rx=0 tx=0).stop 2024-04-25T10:12:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.330+0000 7f7ed1c0c640 1 -- 172.21.15.16:0/4258220122 reap_dead start 2024-04-25T10:12:59.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.330+0000 7f7ed366f640 1 -- 172.21.15.16:0/4258220122 shutdown_connections 2024-04-25T10:12:59.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.330+0000 7f7ed366f640 1 -- 172.21.15.16:0/4258220122 >> 172.21.15.16:0/4258220122 conn(0x7f7ecc072900 msgr2=0x7f7ecc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:59.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.330+0000 7f7ed366f640 1 -- 172.21.15.16:0/4258220122 shutdown_connections 2024-04-25T10:12:59.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.330+0000 7f7ed366f640 1 -- 172.21.15.16:0/4258220122 wait complete. 2024-04-25T10:12:59.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.332+0000 7f7ed366f640 1 Processor -- start 2024-04-25T10:12:59.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.332+0000 7f7ed366f640 1 -- start start 2024-04-25T10:12:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.332+0000 7f7ed366f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ecc14ed90 0x7f7ecc1e88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.332+0000 7f7ed366f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc1e8e00 0x7f7ecc1e6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.332+0000 7f7ed366f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc1e67c0 0x7f7ecc15f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.332+0000 7f7ed366f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc0d3090 con 0x7f7ecc1e67c0 2024-04-25T10:12:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.332+0000 7f7ed366f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc0d2db0 con 0x7f7ecc14ed90 2024-04-25T10:12:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.332+0000 7f7ed366f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc0d2160 con 0x7f7ecc1e8e00 2024-04-25T10:12:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.332+0000 7f7ed1c0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc1e67c0 0x7f7ecc15f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.332+0000 7f7ed1c0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc1e67c0 0x7f7ecc15f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60936/0 (socket says 172.21.15.16:60936) 2024-04-25T10:12:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.332+0000 7f7ed1c0c640 1 -- 172.21.15.16:0/2612072669 learned_addr learned my addr 172.21.15.16:0/2612072669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:12:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.333+0000 7f7ed0c0a640 1 --2- 172.21.15.16:0/2612072669 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc1e8e00 0x7f7ecc1e6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.333+0000 7f7ed140b640 1 --2- 172.21.15.16:0/2612072669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ecc14ed90 0x7f7ecc1e88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.333+0000 7f7ed1c0c640 1 -- 172.21.15.16:0/2612072669 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc1e8e00 msgr2=0x7f7ecc1e6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.333+0000 7f7ed1c0c640 1 --2- 172.21.15.16:0/2612072669 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc1e8e00 0x7f7ecc1e6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.333+0000 7f7ed1c0c640 1 -- 172.21.15.16:0/2612072669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ecc14ed90 msgr2=0x7f7ecc1e88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.333+0000 7f7ed1c0c640 1 --2- 172.21.15.16:0/2612072669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ecc14ed90 0x7f7ecc1e88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.333+0000 7f7ed1c0c640 1 -- 172.21.15.16:0/2612072669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc1e6fb0 con 0x7f7ecc1e67c0 2024-04-25T10:12:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.333+0000 7f7ed1c0c640 1 --2- 172.21.15.16:0/2612072669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc1e67c0 0x7f7ecc15f300 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f7ec402b040 tx=0x7f7ec4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.333+0000 7f7ec1ffb640 1 -- 172.21.15.16:0/2612072669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4009da0 con 0x7f7ecc1e67c0 2024-04-25T10:12:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.333+0000 7f7ec1ffb640 1 -- 172.21.15.16:0/2612072669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec402f460 con 0x7f7ecc1e67c0 2024-04-25T10:12:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.333+0000 7f7ed366f640 1 -- 172.21.15.16:0/2612072669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ecc1e7290 con 0x7f7ecc1e67c0 2024-04-25T10:12:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.333+0000 7f7ec1ffb640 1 -- 172.21.15.16:0/2612072669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec402f830 con 0x7f7ecc1e67c0 2024-04-25T10:12:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.333+0000 7f7ed366f640 1 -- 172.21.15.16:0/2612072669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ecc1e7850 con 0x7f7ecc1e67c0 2024-04-25T10:12:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.334+0000 7f7ed366f640 1 -- 172.21.15.16:0/2612072669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ecc0749b0 con 0x7f7ecc1e67c0 2024-04-25T10:12:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.335+0000 7f7ec1ffb640 1 -- 172.21.15.16:0/2612072669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ec4041070 con 0x7f7ecc1e67c0 2024-04-25T10:12:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.335+0000 7f7ec1ffb640 1 --2- 172.21.15.16:0/2612072669 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7eb006b4d0 0x7f7eb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.335+0000 7f7ec1ffb640 1 -- 172.21.15.16:0/2612072669 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ec40aa8e0 con 0x7f7ecc1e67c0 2024-04-25T10:12:59.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.336+0000 7f7ed140b640 1 --2- 172.21.15.16:0/2612072669 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7eb006b4d0 0x7f7eb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.336+0000 7f7ed140b640 1 --2- 172.21.15.16:0/2612072669 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7eb006b4d0 0x7f7eb006d990 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80011e0 tx=0x7f7ec8001170 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.339+0000 7f7ec1ffb640 1 -- 172.21.15.16:0/2612072669 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ec4033020 con 0x7f7ecc1e67c0 2024-04-25T10:12:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.557+0000 7f7ed366f640 1 -- 172.21.15.16:0/2612072669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7ecc0cea40 con 0x7f7ecc1e67c0 2024-04-25T10:12:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.558+0000 7f7ec1ffb640 1 -- 172.21.15.16:0/2612072669 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7ec40791c0 con 0x7f7ecc1e67c0 2024-04-25T10:12:59.562 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:12:59.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.560+0000 7f7ed366f640 1 -- 172.21.15.16:0/2612072669 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7eb006b4d0 msgr2=0x7f7eb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.560+0000 7f7ed366f640 1 --2- 172.21.15.16:0/2612072669 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7eb006b4d0 0x7f7eb006d990 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80011e0 tx=0x7f7ec8001170 comp rx=0 tx=0).stop 2024-04-25T10:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.560+0000 7f7ed366f640 1 -- 172.21.15.16:0/2612072669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc1e67c0 msgr2=0x7f7ecc15f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.560+0000 7f7ed366f640 1 --2- 172.21.15.16:0/2612072669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc1e67c0 0x7f7ecc15f300 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f7ec402b040 tx=0x7f7ec4009750 comp rx=0 tx=0).stop 2024-04-25T10:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.560+0000 7f7ed366f640 1 -- 172.21.15.16:0/2612072669 shutdown_connections 2024-04-25T10:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.560+0000 7f7ed366f640 1 --2- 172.21.15.16:0/2612072669 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7eb006b4d0 0x7f7eb006d990 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.560+0000 7f7ed366f640 1 --2- 172.21.15.16:0/2612072669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ecc1e67c0 0x7f7ecc15f300 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.560+0000 7f7ed366f640 1 --2- 172.21.15.16:0/2612072669 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc1e8e00 0x7f7ecc1e6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.560+0000 7f7ed366f640 1 --2- 172.21.15.16:0/2612072669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ecc14ed90 0x7f7ecc1e88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.560+0000 7f7ed366f640 1 -- 172.21.15.16:0/2612072669 >> 172.21.15.16:0/2612072669 conn(0x7f7ecc072900 msgr2=0x7f7ecc0cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.560+0000 7f7ed366f640 1 -- 172.21.15.16:0/2612072669 shutdown_connections 2024-04-25T10:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:12:59.560+0000 7f7ed366f640 1 -- 172.21.15.16:0/2612072669 wait complete. 2024-04-25T10:12:59.575 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:12:59.575 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:12:59.575 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-04-25T10:12:59.575 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:12:59.575 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:59.575 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:12:59.575 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:12:59.575 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:12:59.575 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:59.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:59.576 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-04-25T10:12:59.591 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:12:59.592 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:12:59.592 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:12:59.592 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:59.592 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:12:59.592 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:59.592 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:12:59.592 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:59.592 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-04-25T10:12:59.592 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:12:59.592 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:59.592 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-04-25T10:12:59.592 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:12:59.592 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:59.592 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-04-25T10:12:59.593 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:12:59.593 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42181sec preferred_lft 42181sec 2024-04-25T10:12:59.593 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-25T10:12:59.593 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:59.593 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:12:59.593 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:59.593 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-04-25T10:12:59.593 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:12:59.593 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:59.593 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:12:59.593 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:59.593 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f06b:fdff:fed4:a335/64 scope link 2024-04-25T10:12:59.593 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:59.593 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:12:59.593 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:59.593 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::90ad:4fff:fe91:3c33/64 scope link 2024-04-25T10:12:59.594 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:59.594 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:12:59.594 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether be:14:4a:3c:16:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:59.594 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::bc14:4aff:fe3c:1605/64 scope link 2024-04-25T10:12:59.594 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:59.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:59.594 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:12:59.655 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:12:59.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:59.657 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-04-25T10:12:59.713 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-04-25T10:12:59.713 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:12:59.713 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:12:59.713 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:12:59.713 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:12:59.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:59.714 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:12:59.714 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:12:59.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:12:59.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:12:59.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:12:59.714 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:12:59.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:59.799 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:12:59.814 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:12:59.814 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:12:59.814 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:59.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:59.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-04-25T10:12:59.878 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:59.879 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:59.934 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:59.935 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:59.935 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:59.990 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:12:59.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:59.990 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:00.072 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:00.072 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:00.072 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:13:00.081 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:00.081 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:00.081 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:13:00.081 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:13:00.081 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:13:00.086 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:00.086 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:00.086 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:13:00.087 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:13:00.087 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:13:00.087 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:13:00.087 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:13:00.087 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:13:00.087 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:13:00.087 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:13:00.087 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:13:00.087 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:13:00.087 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:13:00.089 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:13:00.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:00.089 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:00.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:00.154 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:13:00.211 INFO:teuthology.orchestra.run.smithi016.stdout:client36506 2024-04-25T10:13:00.211 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:13:00.211 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:00.266 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:13:00.266 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:00.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:00.323 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:13:00.423 INFO:teuthology.orchestra.run.smithi016.stderr:6+0 records in 2024-04-25T10:13:00.423 INFO:teuthology.orchestra.run.smithi016.stderr:6+0 records out 2024-04-25T10:13:00.424 INFO:teuthology.orchestra.run.smithi016.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.042026 s, 150 MB/s 2024-04-25T10:13:00.424 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:13:00.424 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:00.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:00.440 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:13:00.538 INFO:teuthology.orchestra.run.smithi016.stderr:6+0 records in 2024-04-25T10:13:00.538 INFO:teuthology.orchestra.run.smithi016.stderr:6+0 records out 2024-04-25T10:13:00.538 INFO:teuthology.orchestra.run.smithi016.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0406491 s, 155 MB/s 2024-04-25T10:13:00.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:00.538 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:13:00.555 INFO:teuthology.orchestra.run.smithi016.stdout:./ 2024-04-25T10:13:00.555 INFO:teuthology.orchestra.run.smithi016.stdout:./file2_sixmegs 2024-04-25T10:13:00.555 INFO:teuthology.orchestra.run.smithi016.stdout:./file3_sixmegs 2024-04-25T10:13:00.555 INFO:teuthology.orchestra.run.smithi016.stdout:./file1_sixmegs 2024-04-25T10:13:00.556 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:13:00.556 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:13:00.556 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:13:00.556 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:13:00.556 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:13:00.556 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:13:00.662 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:13:00.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:00.672 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:13:00.672 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:13:00.672 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:13:00.672 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:13:00.672 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-04-25T10:13:00.672 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:13:00.779 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627777 2024-04-25T10:13:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:00.787 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:13:00.788 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:13:00.788 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:13:00.788 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:13:00.788 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-04-25T10:13:00.788 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:13:00.894 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627778 2024-04-25T10:13:00.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:00.903 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:13:00.903 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:13:00.903 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:13:00.903 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:13:00.903 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-04-25T10:13:00.903 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:13:01.010 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627776 2024-04-25T10:13:01.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:01.019 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:13:01.019 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:01.075 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:13:01.075 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:01.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.267+0000 7fa4130f5640 1 Processor -- start 2024-04-25T10:13:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.268+0000 7fa4130f5640 1 -- start start 2024-04-25T10:13:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.268+0000 7fa4130f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa40c15ba10 0x7fa40c099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.268+0000 7fa4130f5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa40c06c3c0 0x7fa40c0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.268+0000 7fa4130f5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40c09a3a0 con 0x7fa40c099f80 2024-04-25T10:13:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.268+0000 7fa4130f5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40c0cbe50 con 0x7fa40c06c3c0 2024-04-25T10:13:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.268+0000 7fa4130f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0cfe80 con 0x7fa40c15ba10 2024-04-25T10:13:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.268+0000 7fa40bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa40c099f80 0x7fa40c06bcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35924/0 (socket says 172.21.15.16:35924) 2024-04-25T10:13:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.268+0000 7fa40bfff640 1 -- 172.21.15.16:0/3099656782 learned_addr learned my addr 172.21.15.16:0/3099656782 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:13:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa40b7fe640 1 -- 172.21.15.16:0/3099656782 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699996904 0 0) 0x7fa40c09a3a0 con 0x7fa40c099f80 2024-04-25T10:13:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa40b7fe640 1 -- 172.21.15.16:0/3099656782 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3fc003520 con 0x7fa40c099f80 2024-04-25T10:13:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa411692640 1 --2- 172.21.15.16:0/3099656782 >> v2:172.21.15.186:3300/0 conn(0x7fa40c06c3c0 0x7fa40c0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa410e91640 1 --2- 172.21.15.16:0/3099656782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa40c15ba10 0x7fa40c099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa40b7fe640 1 -- 172.21.15.16:0/3099656782 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 834521159 0 0) 0x7fa3fc003520 con 0x7fa40c099f80 2024-04-25T10:13:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa40b7fe640 1 -- 172.21.15.16:0/3099656782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa40c15ba10 msgr2=0x7fa40c099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa40b7fe640 1 --2- 172.21.15.16:0/3099656782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa40c15ba10 0x7fa40c099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa40b7fe640 1 -- 172.21.15.16:0/3099656782 >> v2:172.21.15.186:3300/0 conn(0x7fa40c06c3c0 msgr2=0x7fa40c0cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa40b7fe640 1 --2- 172.21.15.16:0/3099656782 >> v2:172.21.15.186:3300/0 conn(0x7fa40c06c3c0 0x7fa40c0cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa40b7fe640 1 -- 172.21.15.16:0/3099656782 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c06c900 con 0x7fa40c099f80 2024-04-25T10:13:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa40b7fe640 1 -- 172.21.15.16:0/3099656782 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fa3f8001670 con 0x7fa40c099f80 2024-04-25T10:13:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa40b7fe640 1 -- 172.21.15.16:0/3099656782 >> v1:172.21.15.16:6789/0 conn(0x7fa40c099f80 legacy=0x7fa40c06bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa40b7fe640 1 --2- 172.21.15.16:0/3099656782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3fc0039b0 0x7fa3fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa40b7fe640 1 --2- 172.21.15.16:0/3099656782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa40c15ba10 0x7fa3fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa40b7fe640 1 --2- 172.21.15.16:0/3099656782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3fc004d40 0x7fa3fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa40b7fe640 1 -- 172.21.15.16:0/3099656782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0cfe80 con 0x7fa3fc0039b0 2024-04-25T10:13:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa40b7fe640 1 -- 172.21.15.16:0/3099656782 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc00b9c0 con 0x7fa3fc004d40 2024-04-25T10:13:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa40b7fe640 1 -- 172.21.15.16:0/3099656782 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc00bb40 con 0x7fa40c15ba10 2024-04-25T10:13:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa410e91640 1 --2- 172.21.15.16:0/3099656782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3fc0039b0 0x7fa3fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa40bfff640 1 --2- 172.21.15.16:0/3099656782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa40c15ba10 0x7fa3fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.269+0000 7fa411692640 1 --2- 172.21.15.16:0/3099656782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3fc004d40 0x7fa3fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.270+0000 7fa410e91640 1 -- 172.21.15.16:0/3099656782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa40c15ba10 msgr2=0x7fa3fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.270+0000 7fa410e91640 1 --2- 172.21.15.16:0/3099656782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa40c15ba10 0x7fa3fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.270+0000 7fa410e91640 1 -- 172.21.15.16:0/3099656782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3fc004d40 msgr2=0x7fa3fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.270+0000 7fa410e91640 1 --2- 172.21.15.16:0/3099656782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3fc004d40 0x7fa3fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.270+0000 7fa410e91640 1 -- 172.21.15.16:0/3099656782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3fc003520 con 0x7fa3fc0039b0 2024-04-25T10:13:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.270+0000 7fa410e91640 1 --2- 172.21.15.16:0/3099656782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3fc0039b0 0x7fa3fc003dd0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fa400009ac0 tx=0x7fa3fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80866eaaf17f8c29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.270+0000 7fa40b7fe640 1 -- 172.21.15.16:0/3099656782 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400036070 con 0x7fa3fc0039b0 2024-04-25T10:13:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.270+0000 7fa40b7fe640 1 -- 172.21.15.16:0/3099656782 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa400031400 con 0x7fa3fc0039b0 2024-04-25T10:13:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.270+0000 7fa40b7fe640 1 -- 172.21.15.16:0/3099656782 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400009690 con 0x7fa3fc0039b0 2024-04-25T10:13:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.271+0000 7fa4130f5640 1 -- 172.21.15.16:0/3099656782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3fc0039b0 msgr2=0x7fa3fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.271+0000 7fa4130f5640 1 --2- 172.21.15.16:0/3099656782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3fc0039b0 0x7fa3fc003dd0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fa400009ac0 tx=0x7fa3fc005830 comp rx=0 tx=0).stop 2024-04-25T10:13:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.271+0000 7fa411692640 1 -- 172.21.15.16:0/3099656782 reap_dead start 2024-04-25T10:13:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.271+0000 7fa4130f5640 1 -- 172.21.15.16:0/3099656782 shutdown_connections 2024-04-25T10:13:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.271+0000 7fa4130f5640 1 -- 172.21.15.16:0/3099656782 >> 172.21.15.16:0/3099656782 conn(0x7fa40c0b32f0 msgr2=0x7fa40c0b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.271+0000 7fa4130f5640 1 -- 172.21.15.16:0/3099656782 shutdown_connections 2024-04-25T10:13:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.271+0000 7fa4130f5640 1 -- 172.21.15.16:0/3099656782 wait complete. 2024-04-25T10:13:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.272+0000 7fa4130f5640 1 Processor -- start 2024-04-25T10:13:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.272+0000 7fa4130f5640 1 -- start start 2024-04-25T10:13:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.273+0000 7fa4130f5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3fc0039b0 0x7fa40c15bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.273+0000 7fa4130f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa40c0b6d40 0x7fa40c0b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.273+0000 7fa4130f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa40c15af50 0x7fa40c0c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.273+0000 7fa4130f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c099a40 con 0x7fa40c0b6d40 2024-04-25T10:13:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.273+0000 7fa4130f5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c06cec0 con 0x7fa3fc0039b0 2024-04-25T10:13:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.273+0000 7fa4130f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c099bc0 con 0x7fa40c15af50 2024-04-25T10:13:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.273+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa40c0b6d40 0x7fa40c0b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.273+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa40c0b6d40 0x7fa40c0b4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60950/0 (socket says 172.21.15.16:60950) 2024-04-25T10:13:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.273+0000 7fa40bfff640 1 -- 172.21.15.16:0/3032863876 learned_addr learned my addr 172.21.15.16:0/3032863876 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.273+0000 7fa411692640 1 --2- 172.21.15.16:0/3032863876 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa40c15af50 0x7fa40c0c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.273+0000 7fa410e91640 1 --2- 172.21.15.16:0/3032863876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3fc0039b0 0x7fa40c15bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.273+0000 7fa40bfff640 1 -- 172.21.15.16:0/3032863876 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa40c15af50 msgr2=0x7fa40c0c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.273+0000 7fa40bfff640 1 --2- 172.21.15.16:0/3032863876 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa40c15af50 0x7fa40c0c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.273+0000 7fa40bfff640 1 -- 172.21.15.16:0/3032863876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3fc0039b0 msgr2=0x7fa40c15bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.273+0000 7fa40bfff640 1 --2- 172.21.15.16:0/3032863876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3fc0039b0 0x7fa40c15bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.273+0000 7fa40bfff640 1 -- 172.21.15.16:0/3032863876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c0b4b00 con 0x7fa40c0b6d40 2024-04-25T10:13:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.274+0000 7fa40bfff640 1 --2- 172.21.15.16:0/3032863876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa40c0b6d40 0x7fa40c0b4390 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800e8a0 tx=0x7fa3f800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.274+0000 7fa4097fa640 1 -- 172.21.15.16:0/3032863876 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f800c8d0 con 0x7fa40c0b6d40 2024-04-25T10:13:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.274+0000 7fa4097fa640 1 -- 172.21.15.16:0/3032863876 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f8018070 con 0x7fa40c0b6d40 2024-04-25T10:13:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.274+0000 7fa4130f5640 1 -- 172.21.15.16:0/3032863876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40c0b4de0 con 0x7fa40c0b6d40 2024-04-25T10:13:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.274+0000 7fa4097fa640 1 -- 172.21.15.16:0/3032863876 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f800ccf0 con 0x7fa40c0b6d40 2024-04-25T10:13:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.274+0000 7fa4130f5640 1 -- 172.21.15.16:0/3032863876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40c0b53a0 con 0x7fa40c0b6d40 2024-04-25T10:13:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.275+0000 7fa4130f5640 1 -- 172.21.15.16:0/3032863876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40c0749b0 con 0x7fa40c0b6d40 2024-04-25T10:13:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.275+0000 7fa4097fa640 1 -- 172.21.15.16:0/3032863876 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3f8013420 con 0x7fa40c0b6d40 2024-04-25T10:13:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.276+0000 7fa4097fa640 1 --2- 172.21.15.16:0/3032863876 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa3f006b500 0x7fa3f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.276+0000 7fa4097fa640 1 -- 172.21.15.16:0/3032863876 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3f8089e10 con 0x7fa40c0b6d40 2024-04-25T10:13:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.276+0000 7fa410e91640 1 --2- 172.21.15.16:0/3032863876 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa3f006b500 0x7fa3f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.277+0000 7fa410e91640 1 --2- 172.21.15.16:0/3032863876 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa3f006b500 0x7fa3f006d9c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa400033000 tx=0x7fa400008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.280+0000 7fa4097fa640 1 -- 172.21.15.16:0/3032863876 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa3f8058c90 con 0x7fa40c0b6d40 2024-04-25T10:13:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.461+0000 7fa4130f5640 1 -- 172.21.15.16:0/3032863876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa40c09a570 con 0x7fa40c0b6d40 2024-04-25T10:13:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.462+0000 7fa4097fa640 1 -- 172.21.15.16:0/3032863876 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16610 (secure 0 0 0) 0x7fa3f8058590 con 0x7fa40c0b6d40 2024-04-25T10:13:01.467 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:13:01.467 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":100,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:12:50.612438+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:47.551100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:47.939289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:01.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.464+0000 7fa4130f5640 1 -- 172.21.15.16:0/3032863876 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa3f006b500 msgr2=0x7fa3f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.464+0000 7fa4130f5640 1 --2- 172.21.15.16:0/3032863876 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa3f006b500 0x7fa3f006d9c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa400033000 tx=0x7fa400008030 comp rx=0 tx=0).stop 2024-04-25T10:13:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.464+0000 7fa4130f5640 1 -- 172.21.15.16:0/3032863876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa40c0b6d40 msgr2=0x7fa40c0b4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.464+0000 7fa4130f5640 1 --2- 172.21.15.16:0/3032863876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa40c0b6d40 0x7fa40c0b4390 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800e8a0 tx=0x7fa3f800ed60 comp rx=0 tx=0).stop 2024-04-25T10:13:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.464+0000 7fa4130f5640 1 -- 172.21.15.16:0/3032863876 shutdown_connections 2024-04-25T10:13:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.464+0000 7fa4130f5640 1 --2- 172.21.15.16:0/3032863876 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa3f006b500 0x7fa3f006d9c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.464+0000 7fa4130f5640 1 --2- 172.21.15.16:0/3032863876 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa40c15af50 0x7fa40c0c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.464+0000 7fa4130f5640 1 --2- 172.21.15.16:0/3032863876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa40c0b6d40 0x7fa40c0b4390 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.464+0000 7fa4130f5640 1 --2- 172.21.15.16:0/3032863876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3fc0039b0 0x7fa40c15bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.464+0000 7fa4130f5640 1 -- 172.21.15.16:0/3032863876 >> 172.21.15.16:0/3032863876 conn(0x7fa40c0b32f0 msgr2=0x7fa40c0ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.464+0000 7fa4130f5640 1 -- 172.21.15.16:0/3032863876 shutdown_connections 2024-04-25T10:13:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.464+0000 7fa4130f5640 1 -- 172.21.15.16:0/3032863876 wait complete. 2024-04-25T10:13:01.479 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:47.551100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:47.939289+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:01.480 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:01.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.578+0000 7f394cb22640 1 Processor -- start 2024-04-25T10:13:01.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.578+0000 7f394cb22640 1 -- start start 2024-04-25T10:13:01.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.579+0000 7f394cb22640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f394815dc80 0x7f3948161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.579+0000 7f394cb22640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f39480b97c0 0x7f39480b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.579+0000 7f394cb22640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f39480b5a30 0x7f394806bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.579+0000 7f394cb22640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39480b5ea0 con 0x7f39480b97c0 2024-04-25T10:13:01.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.579+0000 7f394cb22640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39481627a0 con 0x7f39480b5a30 2024-04-25T10:13:01.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.579+0000 7f394cb22640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39481621e0 con 0x7f394815dc80 2024-04-25T10:13:01.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.579+0000 7f3945d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f39480b97c0 0x7f39480b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.579+0000 7f3945d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f39480b97c0 0x7f39480b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60952/0 (socket says 172.21.15.16:60952) 2024-04-25T10:13:01.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.579+0000 7f3945d9b640 1 -- 172.21.15.16:0/1688340978 learned_addr learned my addr 172.21.15.16:0/1688340978 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:01.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.580+0000 7f394659c640 1 --2- 172.21.15.16:0/1688340978 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f394815dc80 0x7f3948161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.580+0000 7f3946d9d640 1 --2- 172.21.15.16:0/1688340978 >> v2:172.21.15.186:3300/0 conn(0x7f39480b5a30 0x7f394806bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.580+0000 7f3945d9b640 1 -- 172.21.15.16:0/1688340978 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f394815dc80 msgr2=0x7f3948161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.580+0000 7f3945d9b640 1 --2- 172.21.15.16:0/1688340978 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f394815dc80 0x7f3948161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.580+0000 7f3945d9b640 1 -- 172.21.15.16:0/1688340978 >> v2:172.21.15.186:3300/0 conn(0x7f39480b5a30 msgr2=0x7f394806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.580+0000 7f3945d9b640 1 --2- 172.21.15.16:0/1688340978 >> v2:172.21.15.186:3300/0 conn(0x7f39480b5a30 0x7f394806bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.580+0000 7f3945d9b640 1 -- 172.21.15.16:0/1688340978 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39481624c0 con 0x7f39480b97c0 2024-04-25T10:13:01.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.580+0000 7f3945d9b640 1 --2- 172.21.15.16:0/1688340978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39480b97c0 0x7f39480b54f0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f39480d16d0 tx=0x7f393002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2952a34b3ba792b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.581+0000 7f394559a640 1 -- 172.21.15.16:0/1688340978 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930037070 con 0x7f39480b97c0 2024-04-25T10:13:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.581+0000 7f394559a640 1 -- 172.21.15.16:0/1688340978 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3930032440 con 0x7f39480b97c0 2024-04-25T10:13:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.581+0000 7f394559a640 1 -- 172.21.15.16:0/1688340978 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930032810 con 0x7f39480b97c0 2024-04-25T10:13:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.581+0000 7f394cb22640 1 -- 172.21.15.16:0/1688340978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39480b97c0 msgr2=0x7f39480b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.581+0000 7f394cb22640 1 --2- 172.21.15.16:0/1688340978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39480b97c0 0x7f39480b54f0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f39480d16d0 tx=0x7f393002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.581+0000 7f394cb22640 1 -- 172.21.15.16:0/1688340978 shutdown_connections 2024-04-25T10:13:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.581+0000 7f394cb22640 1 --2- 172.21.15.16:0/1688340978 >> v2:172.21.15.186:3300/0 conn(0x7f39480b5a30 0x7f394806bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.581+0000 7f394cb22640 1 --2- 172.21.15.16:0/1688340978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39480b97c0 0x7f39480b54f0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.581+0000 7f394cb22640 1 --2- 172.21.15.16:0/1688340978 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f394815dc80 0x7f3948161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.581+0000 7f394cb22640 1 -- 172.21.15.16:0/1688340978 >> 172.21.15.16:0/1688340978 conn(0x7f39480d04f0 msgr2=0x7f39480d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.581+0000 7f394cb22640 1 -- 172.21.15.16:0/1688340978 shutdown_connections 2024-04-25T10:13:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.581+0000 7f394cb22640 1 -- 172.21.15.16:0/1688340978 wait complete. 2024-04-25T10:13:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.583+0000 7f394cb22640 1 Processor -- start 2024-04-25T10:13:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.583+0000 7f394cb22640 1 -- start start 2024-04-25T10:13:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.583+0000 7f394cb22640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39480b5a30 0x7f39481e9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.583+0000 7f394cb22640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39480b97c0 0x7f39480d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.583+0000 7f394cb22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394815dc80 0x7f3948153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.583+0000 7f394cb22640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3948161db0 con 0x7f394815dc80 2024-04-25T10:13:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.583+0000 7f394cb22640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394815e060 con 0x7f39480b97c0 2024-04-25T10:13:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.583+0000 7f394cb22640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3948161f30 con 0x7f39480b5a30 2024-04-25T10:13:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.583+0000 7f3946d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394815dc80 0x7f3948153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.584+0000 7f3946d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394815dc80 0x7f3948153e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60962/0 (socket says 172.21.15.16:60962) 2024-04-25T10:13:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.584+0000 7f3946d9d640 1 -- 172.21.15.16:0/3580523990 learned_addr learned my addr 172.21.15.16:0/3580523990 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.584+0000 7f394659c640 1 --2- 172.21.15.16:0/3580523990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39480b5a30 0x7f39481e9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.584+0000 7f3945d9b640 1 --2- 172.21.15.16:0/3580523990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39480b97c0 0x7f39480d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.584+0000 7f3946d9d640 1 -- 172.21.15.16:0/3580523990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39480b5a30 msgr2=0x7f39481e9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.584+0000 7f3946d9d640 1 --2- 172.21.15.16:0/3580523990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39480b5a30 0x7f39481e9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.584+0000 7f3946d9d640 1 -- 172.21.15.16:0/3580523990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39480b97c0 msgr2=0x7f39480d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.584+0000 7f3946d9d640 1 --2- 172.21.15.16:0/3580523990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39480b97c0 0x7f39480d2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.584+0000 7f3946d9d640 1 -- 172.21.15.16:0/3580523990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3948154a80 con 0x7f394815dc80 2024-04-25T10:13:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.584+0000 7f3946d9d640 1 --2- 172.21.15.16:0/3580523990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394815dc80 0x7f3948153e30 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f3938007d20 tx=0x7f393800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.584+0000 7f393effd640 1 -- 172.21.15.16:0/3580523990 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938014420 con 0x7f394815dc80 2024-04-25T10:13:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.584+0000 7f393effd640 1 -- 172.21.15.16:0/3580523990 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3938019070 con 0x7f394815dc80 2024-04-25T10:13:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.584+0000 7f394cb22640 1 -- 172.21.15.16:0/3580523990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39480c9780 con 0x7f394815dc80 2024-04-25T10:13:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.584+0000 7f393effd640 1 -- 172.21.15.16:0/3580523990 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938014820 con 0x7f394815dc80 2024-04-25T10:13:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.584+0000 7f394cb22640 1 -- 172.21.15.16:0/3580523990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39480c9d40 con 0x7f394815dc80 2024-04-25T10:13:01.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.585+0000 7f394cb22640 1 -- 172.21.15.16:0/3580523990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39480750b0 con 0x7f394815dc80 2024-04-25T10:13:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.585+0000 7f393effd640 1 -- 172.21.15.16:0/3580523990 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f39380149c0 con 0x7f394815dc80 2024-04-25T10:13:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.586+0000 7f393effd640 1 --2- 172.21.15.16:0/3580523990 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f392806b500 0x7f392806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.586+0000 7f393effd640 1 -- 172.21.15.16:0/3580523990 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f393808a8f0 con 0x7f394815dc80 2024-04-25T10:13:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.586+0000 7f394659c640 1 --2- 172.21.15.16:0/3580523990 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f392806b500 0x7f392806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.587+0000 7f394659c640 1 --2- 172.21.15.16:0/3580523990 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f392806b500 0x7f392806d9c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3940006fd0 tx=0x7f3940008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.590+0000 7f393effd640 1 -- 172.21.15.16:0/3580523990 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3938059800 con 0x7f394815dc80 2024-04-25T10:13:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.770+0000 7f394cb22640 1 -- 172.21.15.16:0/3580523990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39480d1780 con 0x7f394815dc80 2024-04-25T10:13:01.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.770+0000 7f393effd640 1 -- 172.21.15.16:0/3580523990 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3938059100 con 0x7f394815dc80 2024-04-25T10:13:01.775 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:13:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.772+0000 7f394cb22640 1 -- 172.21.15.16:0/3580523990 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f392806b500 msgr2=0x7f392806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.772+0000 7f394cb22640 1 --2- 172.21.15.16:0/3580523990 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f392806b500 0x7f392806d9c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3940006fd0 tx=0x7f3940008040 comp rx=0 tx=0).stop 2024-04-25T10:13:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.772+0000 7f394cb22640 1 -- 172.21.15.16:0/3580523990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394815dc80 msgr2=0x7f3948153e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.772+0000 7f394cb22640 1 --2- 172.21.15.16:0/3580523990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394815dc80 0x7f3948153e30 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f3938007d20 tx=0x7f393800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.772+0000 7f394cb22640 1 -- 172.21.15.16:0/3580523990 shutdown_connections 2024-04-25T10:13:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.772+0000 7f394cb22640 1 --2- 172.21.15.16:0/3580523990 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f392806b500 0x7f392806d9c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.772+0000 7f394cb22640 1 --2- 172.21.15.16:0/3580523990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394815dc80 0x7f3948153e30 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.772+0000 7f394cb22640 1 --2- 172.21.15.16:0/3580523990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39480b97c0 0x7f39480d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.772+0000 7f394cb22640 1 --2- 172.21.15.16:0/3580523990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39480b5a30 0x7f39481e9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.772+0000 7f394cb22640 1 -- 172.21.15.16:0/3580523990 >> 172.21.15.16:0/3580523990 conn(0x7f39480d04f0 msgr2=0x7f39480b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.772+0000 7f394cb22640 1 -- 172.21.15.16:0/3580523990 shutdown_connections 2024-04-25T10:13:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.772+0000 7f394cb22640 1 -- 172.21.15.16:0/3580523990 wait complete. 2024-04-25T10:13:01.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:01.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.887+0000 7f5cf5637640 1 Processor -- start 2024-04-25T10:13:01.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.887+0000 7f5cf5637640 1 -- start start 2024-04-25T10:13:01.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.888+0000 7f5cf5637640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5cf015bbe0 0x7f5cf00cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.888+0000 7f5cf5637640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf00cc830 con 0x7f5cf00d0610 2024-04-25T10:13:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.888+0000 7f5cf5637640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf00ccb10 con 0x7f5cf00cf510 2024-04-25T10:13:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.888+0000 7f5cf5637640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf00ccdf0 con 0x7f5cf015bbe0 2024-04-25T10:13:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.888+0000 7f5cef7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5cf00d0610 0x7f5cf00c5490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35934/0 (socket says 172.21.15.16:35934) 2024-04-25T10:13:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.888+0000 7f5cef7fe640 1 -- 172.21.15.16:0/3426810583 learned_addr learned my addr 172.21.15.16:0/3426810583 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:13:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.888+0000 7f5ceeffd640 1 --2- 172.21.15.16:0/3426810583 >> v2:172.21.15.186:3300/0 conn(0x7f5cf015bbe0 0x7f5cf00cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.889+0000 7f5cedffb640 1 -- 172.21.15.16:0/3426810583 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4233398714 0 0) 0x7f5cf00cc830 con 0x7f5cf00d0610 2024-04-25T10:13:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.889+0000 7f5cedffb640 1 -- 172.21.15.16:0/3426810583 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cdc003520 con 0x7f5cf00d0610 2024-04-25T10:13:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.889+0000 7f5ceeffd640 1 -- 172.21.15.16:0/3426810583 >> v1:172.21.15.186:6789/0 conn(0x7f5cf00cf510 legacy=0x7f5cf00cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.889+0000 7f5ceeffd640 1 -- 172.21.15.16:0/3426810583 >> v1:172.21.15.16:6789/0 conn(0x7f5cf00d0610 legacy=0x7f5cf00c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.889+0000 7f5ceeffd640 1 -- 172.21.15.16:0/3426810583 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf00cd0d0 con 0x7f5cf015bbe0 2024-04-25T10:13:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.890+0000 7f5ceeffd640 1 --2- 172.21.15.16:0/3426810583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cf015bbe0 0x7f5cf00cd8d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80097a0 tx=0x7f5ce802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1977cf65abf061c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.890+0000 7f5cedffb640 1 -- 172.21.15.16:0/3426810583 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8037070 con 0x7f5cf015bbe0 2024-04-25T10:13:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.890+0000 7f5cedffb640 1 -- 172.21.15.16:0/3426810583 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce8032440 con 0x7f5cf015bbe0 2024-04-25T10:13:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.890+0000 7f5cedffb640 1 -- 172.21.15.16:0/3426810583 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8032810 con 0x7f5cf015bbe0 2024-04-25T10:13:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.890+0000 7f5cf5637640 1 -- 172.21.15.16:0/3426810583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cf015bbe0 msgr2=0x7f5cf00cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.890+0000 7f5cf5637640 1 --2- 172.21.15.16:0/3426810583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cf015bbe0 0x7f5cf00cd8d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80097a0 tx=0x7f5ce802a230 comp rx=0 tx=0).stop 2024-04-25T10:13:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.890+0000 7f5cf5637640 1 -- 172.21.15.16:0/3426810583 shutdown_connections 2024-04-25T10:13:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.890+0000 7f5cf5637640 1 --2- 172.21.15.16:0/3426810583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cf015bbe0 0x7f5cf00cd8d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.890+0000 7f5cf5637640 1 -- 172.21.15.16:0/3426810583 >> 172.21.15.16:0/3426810583 conn(0x7f5cf00b3430 msgr2=0x7f5cf00b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.890+0000 7f5cf5637640 1 -- 172.21.15.16:0/3426810583 shutdown_connections 2024-04-25T10:13:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.890+0000 7f5cf5637640 1 -- 172.21.15.16:0/3426810583 wait complete. 2024-04-25T10:13:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.892+0000 7f5cf5637640 1 Processor -- start 2024-04-25T10:13:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.892+0000 7f5cf5637640 1 -- start start 2024-04-25T10:13:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.892+0000 7f5cf5637640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cf00cf510 0x7f5cf00cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.892+0000 7f5cf5637640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00d0610 0x7f5cf0159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.892+0000 7f5cf5637640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5cf015bbe0 0x7f5cf015a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.892+0000 7f5cf5637640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf00cde10 con 0x7f5cf00d0610 2024-04-25T10:13:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.892+0000 7f5cf5637640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf015bfc0 con 0x7f5cf00cf510 2024-04-25T10:13:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.892+0000 7f5cf5637640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf00cc450 con 0x7f5cf015bbe0 2024-04-25T10:13:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.892+0000 7f5cee7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00d0610 0x7f5cf0159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.892+0000 7f5cee7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00d0610 0x7f5cf0159b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60970/0 (socket says 172.21.15.16:60970) 2024-04-25T10:13:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.892+0000 7f5cee7fc640 1 -- 172.21.15.16:0/2915007428 learned_addr learned my addr 172.21.15.16:0/2915007428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.893+0000 7f5ceeffd640 1 --2- 172.21.15.16:0/2915007428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cf00cf510 0x7f5cf00cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.893+0000 7f5cef7fe640 1 --2- 172.21.15.16:0/2915007428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5cf015bbe0 0x7f5cf015a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.893+0000 7f5cee7fc640 1 -- 172.21.15.16:0/2915007428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5cf015bbe0 msgr2=0x7f5cf015a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.893+0000 7f5cee7fc640 1 --2- 172.21.15.16:0/2915007428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5cf015bbe0 0x7f5cf015a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.893+0000 7f5cee7fc640 1 -- 172.21.15.16:0/2915007428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cf00cf510 msgr2=0x7f5cf00cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.893+0000 7f5cee7fc640 1 --2- 172.21.15.16:0/2915007428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cf00cf510 0x7f5cf00cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.893+0000 7f5cee7fc640 1 -- 172.21.15.16:0/2915007428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf015c140 con 0x7f5cf00d0610 2024-04-25T10:13:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.893+0000 7f5ceeffd640 1 --2- 172.21.15.16:0/2915007428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cf00cf510 0x7f5cf00cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.893+0000 7f5cee7fc640 1 --2- 172.21.15.16:0/2915007428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00d0610 0x7f5cf0159b80 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800c3c0 tx=0x7f5cd800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.893+0000 7f5ce77fe640 1 -- 172.21.15.16:0/2915007428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd80052d0 con 0x7f5cf00d0610 2024-04-25T10:13:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.893+0000 7f5ce77fe640 1 -- 172.21.15.16:0/2915007428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd8005490 con 0x7f5cf00d0610 2024-04-25T10:13:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.893+0000 7f5cf5637640 1 -- 172.21.15.16:0/2915007428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf00b6110 con 0x7f5cf00d0610 2024-04-25T10:13:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.893+0000 7f5ce77fe640 1 -- 172.21.15.16:0/2915007428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8007260 con 0x7f5cf00d0610 2024-04-25T10:13:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.893+0000 7f5cf5637640 1 -- 172.21.15.16:0/2915007428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf00b66d0 con 0x7f5cf00d0610 2024-04-25T10:13:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.894+0000 7f5cf5637640 1 -- 172.21.15.16:0/2915007428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf00750b0 con 0x7f5cf00d0610 2024-04-25T10:13:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.894+0000 7f5ce77fe640 1 -- 172.21.15.16:0/2915007428 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5cd8007400 con 0x7f5cf00d0610 2024-04-25T10:13:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.895+0000 7f5ce77fe640 1 --2- 172.21.15.16:0/2915007428 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5cd006b4d0 0x7f5cd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.895+0000 7f5ce77fe640 1 -- 172.21.15.16:0/2915007428 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5cd8089650 con 0x7f5cf00d0610 2024-04-25T10:13:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.895+0000 7f5ceeffd640 1 --2- 172.21.15.16:0/2915007428 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5cd006b4d0 0x7f5cd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.896+0000 7f5ceeffd640 1 --2- 172.21.15.16:0/2915007428 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5cd006b4d0 0x7f5cd006d990 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8034000 tx=0x7f5ce802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:01.899+0000 7f5ce77fe640 1 -- 172.21.15.16:0/2915007428 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5cd8058560 con 0x7f5cf00d0610 2024-04-25T10:13:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.080+0000 7f5cf5637640 1 -- 172.21.15.16:0/2915007428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5cf00b42d0 con 0x7f5cf00d0610 2024-04-25T10:13:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.080+0000 7f5ce77fe640 1 -- 172.21.15.16:0/2915007428 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5cd8057e60 con 0x7f5cf00d0610 2024-04-25T10:13:02.085 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:13:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.082+0000 7f5cf5637640 1 -- 172.21.15.16:0/2915007428 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5cd006b4d0 msgr2=0x7f5cd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.082+0000 7f5cf5637640 1 --2- 172.21.15.16:0/2915007428 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5cd006b4d0 0x7f5cd006d990 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8034000 tx=0x7f5ce802e000 comp rx=0 tx=0).stop 2024-04-25T10:13:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.082+0000 7f5cf5637640 1 -- 172.21.15.16:0/2915007428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00d0610 msgr2=0x7f5cf0159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.082+0000 7f5cf5637640 1 --2- 172.21.15.16:0/2915007428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00d0610 0x7f5cf0159b80 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800c3c0 tx=0x7f5cd800c880 comp rx=0 tx=0).stop 2024-04-25T10:13:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.082+0000 7f5cf5637640 1 -- 172.21.15.16:0/2915007428 shutdown_connections 2024-04-25T10:13:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.082+0000 7f5cf5637640 1 --2- 172.21.15.16:0/2915007428 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5cd006b4d0 0x7f5cd006d990 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.082+0000 7f5cf5637640 1 --2- 172.21.15.16:0/2915007428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5cf015bbe0 0x7f5cf015a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.082+0000 7f5cf5637640 1 --2- 172.21.15.16:0/2915007428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00d0610 0x7f5cf0159b80 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.082+0000 7f5cf5637640 1 --2- 172.21.15.16:0/2915007428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cf00cf510 0x7f5cf00cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.082+0000 7f5cf5637640 1 -- 172.21.15.16:0/2915007428 >> 172.21.15.16:0/2915007428 conn(0x7f5cf00b3430 msgr2=0x7f5cf00cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.082+0000 7f5cf5637640 1 -- 172.21.15.16:0/2915007428 shutdown_connections 2024-04-25T10:13:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.082+0000 7f5cf5637640 1 -- 172.21.15.16:0/2915007428 wait complete. 2024-04-25T10:13:02.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:02.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.197+0000 7f171b407640 1 Processor -- start 2024-04-25T10:13:02.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.198+0000 7f171b407640 1 -- start start 2024-04-25T10:13:02.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.198+0000 7f171b407640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17140b42c0 0x7f171415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.198+0000 7f171b407640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f171415aef0 0x7f17140cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.198+0000 7f171b407640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171415b370 con 0x7f17140b46a0 2024-04-25T10:13:02.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.198+0000 7f171b407640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17140d05d0 con 0x7f171415aef0 2024-04-25T10:13:02.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.198+0000 7f171b407640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171415b650 con 0x7f17140b42c0 2024-04-25T10:13:02.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f17199a4640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f17140b46a0 0x7f17141549d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35938/0 (socket says 172.21.15.16:35938) 2024-04-25T10:13:02.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f17199a4640 1 -- 172.21.15.16:0/1107895963 learned_addr learned my addr 172.21.15.16:0/1107895963 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:13:02.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f17189a2640 1 --2- 172.21.15.16:0/1107895963 >> v2:172.21.15.186:3300/0 conn(0x7f171415aef0 0x7f17140cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f17191a3640 1 --2- 172.21.15.16:0/1107895963 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17140b42c0 0x7f171415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f170bfff640 1 -- 172.21.15.16:0/1107895963 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1975386797 0 0) 0x7f171415b370 con 0x7f17140b46a0 2024-04-25T10:13:02.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f170bfff640 1 -- 172.21.15.16:0/1107895963 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1700003520 con 0x7f17140b46a0 2024-04-25T10:13:02.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f170bfff640 1 -- 172.21.15.16:0/1107895963 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1929170018 0 0) 0x7f1700003520 con 0x7f17140b46a0 2024-04-25T10:13:02.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f170bfff640 1 -- 172.21.15.16:0/1107895963 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17140b42c0 msgr2=0x7f171415ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:02.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f170bfff640 1 --2- 172.21.15.16:0/1107895963 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17140b42c0 0x7f171415ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f170bfff640 1 -- 172.21.15.16:0/1107895963 >> v2:172.21.15.186:3300/0 conn(0x7f171415aef0 msgr2=0x7f17140cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:02.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f170bfff640 1 --2- 172.21.15.16:0/1107895963 >> v2:172.21.15.186:3300/0 conn(0x7f171415aef0 0x7f17140cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f170bfff640 1 -- 172.21.15.16:0/1107895963 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171415b930 con 0x7f17140b46a0 2024-04-25T10:13:02.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f170bfff640 1 -- 172.21.15.16:0/1107895963 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f170c0019f0 con 0x7f17140b46a0 2024-04-25T10:13:02.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f17191a3640 1 --2- 172.21.15.16:0/1107895963 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17140b42c0 0x7f171415ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:02.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f170bfff640 1 -- 172.21.15.16:0/1107895963 >> v1:172.21.15.16:6789/0 conn(0x7f17140b46a0 legacy=0x7f17141549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f170bfff640 1 --2- 172.21.15.16:0/1107895963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17000039b0 0x7f1700003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f170bfff640 1 --2- 172.21.15.16:0/1107895963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1700004800 0x7f1700004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f170bfff640 1 --2- 172.21.15.16:0/1107895963 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17140b42c0 0x7f17000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f170bfff640 1 -- 172.21.15.16:0/1107895963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17140d05d0 con 0x7f17000039b0 2024-04-25T10:13:02.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f170bfff640 1 -- 172.21.15.16:0/1107895963 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171415b650 con 0x7f1700004800 2024-04-25T10:13:02.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f170bfff640 1 -- 172.21.15.16:0/1107895963 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170000ba10 con 0x7f17140b42c0 2024-04-25T10:13:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f17189a2640 1 --2- 172.21.15.16:0/1107895963 >> v2:172.21.15.186:3300/0 conn(0x7f171415aef0 0x7f17140cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.199+0000 7f17191a3640 1 --2- 172.21.15.16:0/1107895963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17000039b0 0x7f1700003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.200+0000 7f17199a4640 1 --2- 172.21.15.16:0/1107895963 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17140b42c0 0x7f17000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.200+0000 7f17189a2640 1 --2- 172.21.15.16:0/1107895963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1700004800 0x7f1700004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.200+0000 7f17191a3640 1 -- 172.21.15.16:0/1107895963 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17140b42c0 msgr2=0x7f17000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.200+0000 7f17191a3640 1 --2- 172.21.15.16:0/1107895963 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17140b42c0 0x7f17000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.200+0000 7f17191a3640 1 -- 172.21.15.16:0/1107895963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1700004800 msgr2=0x7f1700004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.200+0000 7f17191a3640 1 --2- 172.21.15.16:0/1107895963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1700004800 0x7f1700004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.200+0000 7f17191a3640 1 -- 172.21.15.16:0/1107895963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1700003520 con 0x7f17000039b0 2024-04-25T10:13:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.200+0000 7f17199a4640 1 --2- 172.21.15.16:0/1107895963 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17140b42c0 0x7f17000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.200+0000 7f17191a3640 1 --2- 172.21.15.16:0/1107895963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17000039b0 0x7f1700003dd0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f1710009c90 tx=0x7f1700005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baaa9bfc43261927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.200+0000 7f170bfff640 1 -- 172.21.15.16:0/1107895963 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710036070 con 0x7f17000039b0 2024-04-25T10:13:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.200+0000 7f170bfff640 1 -- 172.21.15.16:0/1107895963 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1710031400 con 0x7f17000039b0 2024-04-25T10:13:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.200+0000 7f170bfff640 1 -- 172.21.15.16:0/1107895963 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710009860 con 0x7f17000039b0 2024-04-25T10:13:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.201+0000 7f171b407640 1 -- 172.21.15.16:0/1107895963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17000039b0 msgr2=0x7f1700003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.201+0000 7f171b407640 1 --2- 172.21.15.16:0/1107895963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17000039b0 0x7f1700003dd0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f1710009c90 tx=0x7f1700005830 comp rx=0 tx=0).stop 2024-04-25T10:13:02.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.201+0000 7f17199a4640 1 -- 172.21.15.16:0/1107895963 reap_dead start 2024-04-25T10:13:02.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.201+0000 7f171b407640 1 -- 172.21.15.16:0/1107895963 shutdown_connections 2024-04-25T10:13:02.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.201+0000 7f171b407640 1 -- 172.21.15.16:0/1107895963 >> 172.21.15.16:0/1107895963 conn(0x7f171406aba0 msgr2=0x7f171406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:02.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.201+0000 7f171b407640 1 -- 172.21.15.16:0/1107895963 shutdown_connections 2024-04-25T10:13:02.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.201+0000 7f171b407640 1 -- 172.21.15.16:0/1107895963 wait complete. 2024-04-25T10:13:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.202+0000 7f171b407640 1 Processor -- start 2024-04-25T10:13:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.202+0000 7f171b407640 1 -- start start 2024-04-25T10:13:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.203+0000 7f171b407640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17000039b0 0x7f171414d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.203+0000 7f171b407640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f171414f580 0x7f17140c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.203+0000 7f171b407640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17140c4be0 0x7f17141562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.203+0000 7f171b407640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171406eae0 con 0x7f171414f580 2024-04-25T10:13:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.203+0000 7f171b407640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17140cf980 con 0x7f17140c4be0 2024-04-25T10:13:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.203+0000 7f171b407640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171406ec60 con 0x7f17000039b0 2024-04-25T10:13:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.203+0000 7f17189a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f171414f580 0x7f17140c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.203+0000 7f17189a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f171414f580 0x7f17140c46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60994/0 (socket says 172.21.15.16:60994) 2024-04-25T10:13:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.203+0000 7f17189a2640 1 -- 172.21.15.16:0/1748858394 learned_addr learned my addr 172.21.15.16:0/1748858394 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.203+0000 7f17191a3640 1 --2- 172.21.15.16:0/1748858394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17000039b0 0x7f171414d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.203+0000 7f17189a2640 1 -- 172.21.15.16:0/1748858394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17000039b0 msgr2=0x7f171414d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.203+0000 7f17199a4640 1 --2- 172.21.15.16:0/1748858394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17140c4be0 0x7f17141562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.203+0000 7f17189a2640 1 --2- 172.21.15.16:0/1748858394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17000039b0 0x7f171414d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.203+0000 7f17189a2640 1 -- 172.21.15.16:0/1748858394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17140c4be0 msgr2=0x7f17141562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.203+0000 7f17189a2640 1 --2- 172.21.15.16:0/1748858394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17140c4be0 0x7f17141562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.203+0000 7f17189a2640 1 -- 172.21.15.16:0/1748858394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1714157ef0 con 0x7f171414f580 2024-04-25T10:13:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.203+0000 7f17199a4640 1 --2- 172.21.15.16:0/1748858394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17140c4be0 0x7f17141562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.204+0000 7f17189a2640 1 --2- 172.21.15.16:0/1748858394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f171414f580 0x7f17140c46a0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f1704011960 tx=0x7f1704011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.204+0000 7f1709ffb640 1 -- 172.21.15.16:0/1748858394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170400fbd0 con 0x7f171414f580 2024-04-25T10:13:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.204+0000 7f1709ffb640 1 -- 172.21.15.16:0/1748858394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170400fd70 con 0x7f171414f580 2024-04-25T10:13:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.204+0000 7f171b407640 1 -- 172.21.15.16:0/1748858394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17141581d0 con 0x7f171414f580 2024-04-25T10:13:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.204+0000 7f1709ffb640 1 -- 172.21.15.16:0/1748858394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704015660 con 0x7f171414f580 2024-04-25T10:13:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.204+0000 7f171b407640 1 -- 172.21.15.16:0/1748858394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1714158790 con 0x7f171414f580 2024-04-25T10:13:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.205+0000 7f171b407640 1 -- 172.21.15.16:0/1748858394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17140750b0 con 0x7f171414f580 2024-04-25T10:13:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.205+0000 7f1709ffb640 1 -- 172.21.15.16:0/1748858394 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1704015800 con 0x7f171414f580 2024-04-25T10:13:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.206+0000 7f1709ffb640 1 --2- 172.21.15.16:0/1748858394 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f16f806b500 0x7f16f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.206+0000 7f1709ffb640 1 -- 172.21.15.16:0/1748858394 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f170408a800 con 0x7f171414f580 2024-04-25T10:13:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.206+0000 7f17191a3640 1 --2- 172.21.15.16:0/1748858394 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f16f806b500 0x7f16f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.207+0000 7f17191a3640 1 --2- 172.21.15.16:0/1748858394 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f16f806b500 0x7f16f806d9c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1710033000 tx=0x7f1710008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.210+0000 7f1709ffb640 1 -- 172.21.15.16:0/1748858394 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1704019030 con 0x7f171414f580 2024-04-25T10:13:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.392+0000 7f171b407640 1 -- 172.21.15.16:0/1748858394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f171406e700 con 0x7f171414f580 2024-04-25T10:13:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.392+0000 7f1709ffb640 1 -- 172.21.15.16:0/1748858394 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1704059670 con 0x7f171414f580 2024-04-25T10:13:02.397 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:13:02.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.394+0000 7f171b407640 1 -- 172.21.15.16:0/1748858394 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f16f806b500 msgr2=0x7f16f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.394+0000 7f171b407640 1 --2- 172.21.15.16:0/1748858394 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f16f806b500 0x7f16f806d9c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1710033000 tx=0x7f1710008030 comp rx=0 tx=0).stop 2024-04-25T10:13:02.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.394+0000 7f171b407640 1 -- 172.21.15.16:0/1748858394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f171414f580 msgr2=0x7f17140c46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.394+0000 7f171b407640 1 --2- 172.21.15.16:0/1748858394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f171414f580 0x7f17140c46a0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f1704011960 tx=0x7f1704011e20 comp rx=0 tx=0).stop 2024-04-25T10:13:02.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.394+0000 7f171b407640 1 -- 172.21.15.16:0/1748858394 shutdown_connections 2024-04-25T10:13:02.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.394+0000 7f171b407640 1 --2- 172.21.15.16:0/1748858394 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f16f806b500 0x7f16f806d9c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.394+0000 7f171b407640 1 --2- 172.21.15.16:0/1748858394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17140c4be0 0x7f17141562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.394+0000 7f171b407640 1 --2- 172.21.15.16:0/1748858394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f171414f580 0x7f17140c46a0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.394+0000 7f171b407640 1 --2- 172.21.15.16:0/1748858394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17000039b0 0x7f171414d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.394+0000 7f171b407640 1 -- 172.21.15.16:0/1748858394 >> 172.21.15.16:0/1748858394 conn(0x7f171406aba0 msgr2=0x7f171415cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:02.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.394+0000 7f171b407640 1 -- 172.21.15.16:0/1748858394 shutdown_connections 2024-04-25T10:13:02.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.394+0000 7f171b407640 1 -- 172.21.15.16:0/1748858394 wait complete. 2024-04-25T10:13:02.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:13:02.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.508+0000 7fcdf489a640 1 Processor -- start 2024-04-25T10:13:02.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.508+0000 7fcdf489a640 1 -- start start 2024-04-25T10:13:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.509+0000 7fcdf489a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcdf00cef40 0x7fcdf00b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.509+0000 7fcdf489a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcdf01618d0 0x7fcdf00d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.509+0000 7fcdf489a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf0161d80 con 0x7fcdf01610b0 2024-04-25T10:13:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.509+0000 7fcdf489a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf00d1ea0 con 0x7fcdf01618d0 2024-04-25T10:13:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.509+0000 7fcdf489a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0162040 con 0x7fcdf00cef40 2024-04-25T10:13:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.509+0000 7fcdeed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcdf01618d0 0x7fcdf00d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.509+0000 7fcdeed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcdf01618d0 0x7fcdf00d2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32768/0 (socket says 172.21.15.16:32768) 2024-04-25T10:13:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.509+0000 7fcdeed9d640 1 -- 172.21.15.16:0/3189836123 learned_addr learned my addr 172.21.15.16:0/3189836123 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.509+0000 7fcde7fff640 1 --2- 172.21.15.16:0/3189836123 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcdf00cef40 0x7fcdf00b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.509+0000 7fcdeed9d640 1 -- 172.21.15.16:0/3189836123 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcdf00cef40 msgr2=0x7fcdf00b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.509+0000 7fcdeed9d640 1 --2- 172.21.15.16:0/3189836123 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcdf00cef40 0x7fcdf00b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.509+0000 7fcdeed9d640 1 -- 172.21.15.16:0/3189836123 >> v1:172.21.15.186:6789/0 conn(0x7fcdf01610b0 legacy=0x7fcdf00ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.509+0000 7fcdeed9d640 1 -- 172.21.15.16:0/3189836123 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf0162320 con 0x7fcdf01618d0 2024-04-25T10:13:02.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.510+0000 7fcdeed9d640 1 --2- 172.21.15.16:0/3189836123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf01618d0 0x7fcdf00d2610 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009e50 tx=0x7fcde802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24438f058db7d096 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.510+0000 7fcdedd9b640 1 -- 172.21.15.16:0/3189836123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8036070 con 0x7fcdf01618d0 2024-04-25T10:13:02.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.510+0000 7fcdedd9b640 1 -- 172.21.15.16:0/3189836123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde8031600 con 0x7fcdf01618d0 2024-04-25T10:13:02.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.510+0000 7fcdedd9b640 1 -- 172.21.15.16:0/3189836123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde80319d0 con 0x7fcdf01618d0 2024-04-25T10:13:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.511+0000 7fcdf489a640 1 -- 172.21.15.16:0/3189836123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf01618d0 msgr2=0x7fcdf00d2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.511+0000 7fcdf489a640 1 --2- 172.21.15.16:0/3189836123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf01618d0 0x7fcdf00d2610 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009e50 tx=0x7fcde802a680 comp rx=0 tx=0).stop 2024-04-25T10:13:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.511+0000 7fcdf489a640 1 -- 172.21.15.16:0/3189836123 shutdown_connections 2024-04-25T10:13:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.511+0000 7fcdf489a640 1 --2- 172.21.15.16:0/3189836123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf01618d0 0x7fcdf00d2610 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.511+0000 7fcdf489a640 1 --2- 172.21.15.16:0/3189836123 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcdf00cef40 0x7fcdf00b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.511+0000 7fcdf489a640 1 -- 172.21.15.16:0/3189836123 >> 172.21.15.16:0/3189836123 conn(0x7fcdf00b5570 msgr2=0x7fcdf00b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.511+0000 7fcdf489a640 1 -- 172.21.15.16:0/3189836123 shutdown_connections 2024-04-25T10:13:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.511+0000 7fcdf489a640 1 -- 172.21.15.16:0/3189836123 wait complete. 2024-04-25T10:13:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.512+0000 7fcdf489a640 1 Processor -- start 2024-04-25T10:13:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.512+0000 7fcdf489a640 1 -- start start 2024-04-25T10:13:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcdf489a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcdf00cef40 0x7fcdf00c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcdf489a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdf01610b0 0x7fcdf0155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcdf489a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf01618d0 0x7fcdf006c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcdf489a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00b6c00 con 0x7fcdf01618d0 2024-04-25T10:13:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcdf489a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00d20f0 con 0x7fcdf01610b0 2024-04-25T10:13:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcdf489a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0161490 con 0x7fcdf00cef40 2024-04-25T10:13:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcdeed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf01618d0 0x7fcdf006c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcdeed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf01618d0 0x7fcdf006c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32782/0 (socket says 172.21.15.16:32782) 2024-04-25T10:13:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcdeed9d640 1 -- 172.21.15.16:0/958527936 learned_addr learned my addr 172.21.15.16:0/958527936 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcde7fff640 1 --2- 172.21.15.16:0/958527936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdf01610b0 0x7fcdf0155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcdee59c640 1 --2- 172.21.15.16:0/958527936 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcdf00cef40 0x7fcdf00c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcdeed9d640 1 -- 172.21.15.16:0/958527936 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcdf00cef40 msgr2=0x7fcdf00c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcdeed9d640 1 --2- 172.21.15.16:0/958527936 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcdf00cef40 0x7fcdf00c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcdeed9d640 1 -- 172.21.15.16:0/958527936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdf01610b0 msgr2=0x7fcdf0155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcdeed9d640 1 --2- 172.21.15.16:0/958527936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdf01610b0 0x7fcdf0155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcdeed9d640 1 -- 172.21.15.16:0/958527936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf0153fd0 con 0x7fcdf01618d0 2024-04-25T10:13:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcdee59c640 1 --2- 172.21.15.16:0/958527936 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcdf00cef40 0x7fcdf00c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcde7fff640 1 --2- 172.21.15.16:0/958527936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdf01610b0 0x7fcdf0155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcdeed9d640 1 --2- 172.21.15.16:0/958527936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf01618d0 0x7fcdf006c220 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fcde8034040 tx=0x7fcde8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcde6ffd640 1 -- 172.21.15.16:0/958527936 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8036050 con 0x7fcdf01618d0 2024-04-25T10:13:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcde6ffd640 1 -- 172.21.15.16:0/958527936 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde8042070 con 0x7fcdf01618d0 2024-04-25T10:13:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcdf489a640 1 -- 172.21.15.16:0/958527936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf01542b0 con 0x7fcdf01618d0 2024-04-25T10:13:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.513+0000 7fcde6ffd640 1 -- 172.21.15.16:0/958527936 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8009510 con 0x7fcdf01618d0 2024-04-25T10:13:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.514+0000 7fcdf489a640 1 -- 172.21.15.16:0/958527936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf0154870 con 0x7fcdf01618d0 2024-04-25T10:13:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.514+0000 7fcdf489a640 1 -- 172.21.15.16:0/958527936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf00750b0 con 0x7fcdf01618d0 2024-04-25T10:13:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.515+0000 7fcde6ffd640 1 -- 172.21.15.16:0/958527936 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcde8007bd0 con 0x7fcdf01618d0 2024-04-25T10:13:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.515+0000 7fcde6ffd640 1 --2- 172.21.15.16:0/958527936 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcdd006b4d0 0x7fcdd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.516+0000 7fcde6ffd640 1 -- 172.21.15.16:0/958527936 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcde80ab190 con 0x7fcdf01618d0 2024-04-25T10:13:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.516+0000 7fcdee59c640 1 --2- 172.21.15.16:0/958527936 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcdd006b4d0 0x7fcdd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.516+0000 7fcdee59c640 1 --2- 172.21.15.16:0/958527936 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcdd006b4d0 0x7fcdd006d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcde0001700 tx=0x7fcde0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.520+0000 7fcde6ffd640 1 -- 172.21.15.16:0/958527936 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcde8045360 con 0x7fcdf01618d0 2024-04-25T10:13:02.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.714+0000 7fcdf489a640 1 -- 172.21.15.16:0/958527936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcdf00d1770 con 0x7fcdf01618d0 2024-04-25T10:13:02.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.714+0000 7fcde6ffd640 1 -- 172.21.15.16:0/958527936 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v100) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcde8079f70 con 0x7fcdf01618d0 2024-04-25T10:13:02.719 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-04-25T10:13:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.717+0000 7fcdf489a640 1 -- 172.21.15.16:0/958527936 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcdd006b4d0 msgr2=0x7fcdd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.717+0000 7fcdf489a640 1 --2- 172.21.15.16:0/958527936 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcdd006b4d0 0x7fcdd006d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcde0001700 tx=0x7fcde0009040 comp rx=0 tx=0).stop 2024-04-25T10:13:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.717+0000 7fcdf489a640 1 -- 172.21.15.16:0/958527936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf01618d0 msgr2=0x7fcdf006c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.717+0000 7fcdf489a640 1 --2- 172.21.15.16:0/958527936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf01618d0 0x7fcdf006c220 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fcde8034040 tx=0x7fcde8004a40 comp rx=0 tx=0).stop 2024-04-25T10:13:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.717+0000 7fcdf489a640 1 -- 172.21.15.16:0/958527936 shutdown_connections 2024-04-25T10:13:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.717+0000 7fcdf489a640 1 --2- 172.21.15.16:0/958527936 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcdd006b4d0 0x7fcdd006d990 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.717+0000 7fcdf489a640 1 --2- 172.21.15.16:0/958527936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf01618d0 0x7fcdf006c220 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.717+0000 7fcdf489a640 1 --2- 172.21.15.16:0/958527936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcdf01610b0 0x7fcdf0155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.717+0000 7fcdf489a640 1 --2- 172.21.15.16:0/958527936 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcdf00cef40 0x7fcdf00c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.717+0000 7fcdf489a640 1 -- 172.21.15.16:0/958527936 >> 172.21.15.16:0/958527936 conn(0x7fcdf00b5570 msgr2=0x7fcdf00b6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.717+0000 7fcdf489a640 1 -- 172.21.15.16:0/958527936 shutdown_connections 2024-04-25T10:13:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.717+0000 7fcdf489a640 1 -- 172.21.15.16:0/958527936 wait complete. 2024-04-25T10:13:02.732 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:13:02.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:02.732 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:02.779 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:13:02.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:02.779 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:13:02.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.939+0000 7f0c51e37640 1 Processor -- start 2024-04-25T10:13:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.939+0000 7f0c51e37640 1 -- start start 2024-04-25T10:13:02.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.940+0000 7f0c51e37640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0c4c15f120 0x7f0c4c0ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.940+0000 7f0c51e37640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0c4c0ce830 0x7f0c4c0cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.940+0000 7f0c51e37640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4c0cdfe0 con 0x7f0c4c0cfbe0 2024-04-25T10:13:02.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.940+0000 7f0c51e37640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c4c0d09f0 con 0x7f0c4c0ce830 2024-04-25T10:13:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.940+0000 7f0c51e37640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c4c0cec70 con 0x7f0c4c15f120 2024-04-25T10:13:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.940+0000 7f0c4affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0c4c0ce830 0x7f0c4c0cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.940+0000 7f0c4bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0c4c0cfbe0 0x7f0c4c099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53810/0 (socket says 172.21.15.16:53810) 2024-04-25T10:13:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.940+0000 7f0c4bfff640 1 -- 172.21.15.16:0/3493164888 learned_addr learned my addr 172.21.15.16:0/3493164888 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:13:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.941+0000 7f0c4b7fe640 1 --2- 172.21.15.16:0/3493164888 >> v2:172.21.15.186:3300/0 conn(0x7f0c4c15f120 0x7f0c4c0ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.941+0000 7f0c4affd640 1 -- 172.21.15.16:0/3493164888 >> v2:172.21.15.186:3300/0 conn(0x7f0c4c15f120 msgr2=0x7f0c4c0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.941+0000 7f0c4affd640 1 --2- 172.21.15.16:0/3493164888 >> v2:172.21.15.186:3300/0 conn(0x7f0c4c15f120 0x7f0c4c0ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.941+0000 7f0c4affd640 1 -- 172.21.15.16:0/3493164888 >> v1:172.21.15.16:6789/0 conn(0x7f0c4c0cfbe0 legacy=0x7f0c4c099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.941+0000 7f0c4affd640 1 -- 172.21.15.16:0/3493164888 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c0d0430 con 0x7f0c4c0ce830 2024-04-25T10:13:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.941+0000 7f0c4a7fc640 1 -- 172.21.15.16:0/3493164888 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3789852292 0 0) 0x7f0c4c0cdfe0 con 0x7f0c4c0cfbe0 2024-04-25T10:13:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.941+0000 7f0c4b7fe640 1 --2- 172.21.15.16:0/3493164888 >> v2:172.21.15.186:3300/0 conn(0x7f0c4c15f120 0x7f0c4c0ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.941+0000 7f0c4affd640 1 --2- 172.21.15.16:0/3493164888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c0ce830 0x7f0c4c0cf6a0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c09b000 tx=0x7f0c3c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c3eafb15e11907 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.942+0000 7f0c4a7fc640 1 -- 172.21.15.16:0/3493164888 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c037070 con 0x7f0c4c0ce830 2024-04-25T10:13:02.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.942+0000 7f0c4a7fc640 1 -- 172.21.15.16:0/3493164888 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3c032440 con 0x7f0c4c0ce830 2024-04-25T10:13:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.942+0000 7f0c4a7fc640 1 -- 172.21.15.16:0/3493164888 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c032810 con 0x7f0c4c0ce830 2024-04-25T10:13:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.942+0000 7f0c51e37640 1 -- 172.21.15.16:0/3493164888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c0ce830 msgr2=0x7f0c4c0cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.942+0000 7f0c51e37640 1 --2- 172.21.15.16:0/3493164888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c0ce830 0x7f0c4c0cf6a0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c09b000 tx=0x7f0c3c02a230 comp rx=0 tx=0).stop 2024-04-25T10:13:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.942+0000 7f0c51e37640 1 -- 172.21.15.16:0/3493164888 shutdown_connections 2024-04-25T10:13:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.942+0000 7f0c51e37640 1 --2- 172.21.15.16:0/3493164888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c0ce830 0x7f0c4c0cf6a0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.942+0000 7f0c51e37640 1 --2- 172.21.15.16:0/3493164888 >> v2:172.21.15.186:3300/0 conn(0x7f0c4c15f120 0x7f0c4c0ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.942+0000 7f0c51e37640 1 -- 172.21.15.16:0/3493164888 >> 172.21.15.16:0/3493164888 conn(0x7f0c4c0d1900 msgr2=0x7f0c4c0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.942+0000 7f0c51e37640 1 -- 172.21.15.16:0/3493164888 shutdown_connections 2024-04-25T10:13:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.942+0000 7f0c51e37640 1 -- 172.21.15.16:0/3493164888 wait complete. 2024-04-25T10:13:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.943+0000 7f0c51e37640 1 Processor -- start 2024-04-25T10:13:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.944+0000 7f0c51e37640 1 -- start start 2024-04-25T10:13:02.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.944+0000 7f0c51e37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c0ce830 0x7f0c4c1f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.944+0000 7f0c51e37640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c4c0cfbe0 0x7f0c4c1544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.944+0000 7f0c51e37640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c4c15f120 0x7f0c4c154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.944+0000 7f0c51e37640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c0ccab0 con 0x7f0c4c0ce830 2024-04-25T10:13:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.944+0000 7f0c51e37640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c0cc7d0 con 0x7f0c4c15f120 2024-04-25T10:13:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.944+0000 7f0c51e37640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c0cc480 con 0x7f0c4c0cfbe0 2024-04-25T10:13:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.944+0000 7f0c4b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c0ce830 0x7f0c4c1f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.944+0000 7f0c4b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c0ce830 0x7f0c4c1f7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41004/0 (socket says 172.21.15.16:41004) 2024-04-25T10:13:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.944+0000 7f0c4b7fe640 1 -- 172.21.15.16:0/2885224448 learned_addr learned my addr 172.21.15.16:0/2885224448 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.945+0000 7f0c4bfff640 1 --2- 172.21.15.16:0/2885224448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c4c15f120 0x7f0c4c154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.945+0000 7f0c4affd640 1 --2- 172.21.15.16:0/2885224448 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c4c0cfbe0 0x7f0c4c1544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.945+0000 7f0c4b7fe640 1 -- 172.21.15.16:0/2885224448 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c4c0cfbe0 msgr2=0x7f0c4c1544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.945+0000 7f0c4b7fe640 1 --2- 172.21.15.16:0/2885224448 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c4c0cfbe0 0x7f0c4c1544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.945+0000 7f0c4b7fe640 1 -- 172.21.15.16:0/2885224448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c4c15f120 msgr2=0x7f0c4c154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.945+0000 7f0c4b7fe640 1 --2- 172.21.15.16:0/2885224448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c4c15f120 0x7f0c4c154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.945+0000 7f0c4b7fe640 1 -- 172.21.15.16:0/2885224448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c0afee0 con 0x7f0c4c0ce830 2024-04-25T10:13:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.945+0000 7f0c4affd640 1 --2- 172.21.15.16:0/2885224448 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c4c0cfbe0 0x7f0c4c1544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.945+0000 7f0c4b7fe640 1 --2- 172.21.15.16:0/2885224448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c0ce830 0x7f0c4c1f7cd0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000d7d0 tx=0x7f0c4000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.945+0000 7f0c33fff640 1 -- 172.21.15.16:0/2885224448 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c40004030 con 0x7f0c4c0ce830 2024-04-25T10:13:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.945+0000 7f0c33fff640 1 -- 172.21.15.16:0/2885224448 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c400041d0 con 0x7f0c4c0ce830 2024-04-25T10:13:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.945+0000 7f0c33fff640 1 -- 172.21.15.16:0/2885224448 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c400096a0 con 0x7f0c4c0ce830 2024-04-25T10:13:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.945+0000 7f0c51e37640 1 -- 172.21.15.16:0/2885224448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4c0b01c0 con 0x7f0c4c0ce830 2024-04-25T10:13:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.945+0000 7f0c51e37640 1 -- 172.21.15.16:0/2885224448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4c0b0780 con 0x7f0c4c0ce830 2024-04-25T10:13:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.946+0000 7f0c51e37640 1 -- 172.21.15.16:0/2885224448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4c067850 con 0x7f0c4c0ce830 2024-04-25T10:13:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.946+0000 7f0c33fff640 1 -- 172.21.15.16:0/2885224448 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c40009840 con 0x7f0c4c0ce830 2024-04-25T10:13:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.947+0000 7f0c33fff640 1 --2- 172.21.15.16:0/2885224448 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0c3806b4d0 0x7f0c3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.947+0000 7f0c33fff640 1 -- 172.21.15.16:0/2885224448 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c4008a530 con 0x7f0c4c0ce830 2024-04-25T10:13:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.947+0000 7f0c4affd640 1 --2- 172.21.15.16:0/2885224448 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0c3806b4d0 0x7f0c3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.948+0000 7f0c4affd640 1 --2- 172.21.15.16:0/2885224448 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0c3806b4d0 0x7f0c3806d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c034000 tx=0x7f0c3c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:02.951+0000 7f0c33fff640 1 -- 172.21.15.16:0/2885224448 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c400593b0 con 0x7f0c4c0ce830 2024-04-25T10:13:03.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.167+0000 7f0c51e37640 1 -- 172.21.15.16:0/2885224448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c4c0749b0 con 0x7f0c4c0ce830 2024-04-25T10:13:03.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.167+0000 7f0c33fff640 1 -- 172.21.15.16:0/2885224448 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f0c40058cb0 con 0x7f0c4c0ce830 2024-04-25T10:13:03.172 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:13:03.172 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7243,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/109406141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":109406141},{"type":"v1","addr":"172.21.15.186:6837","nonce":109406141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2704401503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2704401503},{"type":"v1","addr":"172.21.15.16:6835","nonce":2704401503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/939221717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":939221717},{"type":"v1","addr":"172.21.15.16:6834","nonce":939221717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:48.958938+0000","modified":"2024-04-25T10:12:51.839702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8202},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8202":{"gid":8202,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6836/1371188714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1371188714},{"type":"v1","addr":"172.21.15.186:6836","nonce":1371188714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:03.172 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 71 2024-04-25T10:13:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.169+0000 7f0c51e37640 1 -- 172.21.15.16:0/2885224448 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0c3806b4d0 msgr2=0x7f0c3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.169+0000 7f0c51e37640 1 --2- 172.21.15.16:0/2885224448 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0c3806b4d0 0x7f0c3806d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c034000 tx=0x7f0c3c02e000 comp rx=0 tx=0).stop 2024-04-25T10:13:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.169+0000 7f0c51e37640 1 -- 172.21.15.16:0/2885224448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c0ce830 msgr2=0x7f0c4c1f7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.169+0000 7f0c51e37640 1 --2- 172.21.15.16:0/2885224448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c0ce830 0x7f0c4c1f7cd0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000d7d0 tx=0x7f0c4000dc90 comp rx=0 tx=0).stop 2024-04-25T10:13:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.169+0000 7f0c51e37640 1 -- 172.21.15.16:0/2885224448 shutdown_connections 2024-04-25T10:13:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.169+0000 7f0c51e37640 1 --2- 172.21.15.16:0/2885224448 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0c3806b4d0 0x7f0c3806d990 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.169+0000 7f0c51e37640 1 --2- 172.21.15.16:0/2885224448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c4c15f120 0x7f0c4c154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.169+0000 7f0c51e37640 1 --2- 172.21.15.16:0/2885224448 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c4c0cfbe0 0x7f0c4c1544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.169+0000 7f0c51e37640 1 --2- 172.21.15.16:0/2885224448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c0ce830 0x7f0c4c1f7cd0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.169+0000 7f0c51e37640 1 -- 172.21.15.16:0/2885224448 >> 172.21.15.16:0/2885224448 conn(0x7f0c4c0d1900 msgr2=0x7f0c4c0d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.170+0000 7f0c51e37640 1 -- 172.21.15.16:0/2885224448 shutdown_connections 2024-04-25T10:13:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.170+0000 7f0c51e37640 1 -- 172.21.15.16:0/2885224448 wait complete. 2024-04-25T10:13:03.185 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-25T10:13:03.322 INFO:teuthology.orchestra.run.smithi186.stdout:{"message":"","return_code":0} 2024-04-25T10:13:03.323 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:13:03.323 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:13:03.323 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:03.339 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:13:03.339 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:13:03.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:03.339 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:13:03.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:13:03.544 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T10:13:03.545 DEBUG:teuthology.orchestra.run.smithi016:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T10:13:03.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:03.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-25T10:13:03.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.693+0000 7f7ef92ac640 1 Processor -- start 2024-04-25T10:13:03.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.693+0000 7f7ef92ac640 1 -- start start 2024-04-25T10:13:03.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.693+0000 7f7ef92ac640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ef4161090 0x7f7ef4161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.693+0000 7f7ef92ac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ef40b9720 0x7f7ef40ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.693+0000 7f7ef92ac640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7ef40cece0 0x7f7ef406c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.693+0000 7f7ef92ac640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef40cf130 con 0x7f7ef40b9720 2024-04-25T10:13:03.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.693+0000 7f7ef92ac640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef40d21a0 con 0x7f7ef40cece0 2024-04-25T10:13:03.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.693+0000 7f7ef92ac640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef4161f80 con 0x7f7ef4161090 2024-04-25T10:13:03.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.694+0000 7f7ef27fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ef40b9720 0x7f7ef40ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.694+0000 7f7ef27fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ef40b9720 0x7f7ef40ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41020/0 (socket says 172.21.15.16:41020) 2024-04-25T10:13:03.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.694+0000 7f7ef27fc640 1 -- 172.21.15.16:0/730644486 learned_addr learned my addr 172.21.15.16:0/730644486 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:03.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.694+0000 7f7ef2ffd640 1 --2- 172.21.15.16:0/730644486 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ef4161090 0x7f7ef4161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.694+0000 7f7ef37fe640 1 --2- 172.21.15.16:0/730644486 >> v2:172.21.15.186:3300/0 conn(0x7f7ef40cece0 0x7f7ef406c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.694+0000 7f7ef27fc640 1 -- 172.21.15.16:0/730644486 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ef4161090 msgr2=0x7f7ef4161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.694+0000 7f7ef27fc640 1 --2- 172.21.15.16:0/730644486 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ef4161090 0x7f7ef4161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.694+0000 7f7ef27fc640 1 -- 172.21.15.16:0/730644486 >> v2:172.21.15.186:3300/0 conn(0x7f7ef40cece0 msgr2=0x7f7ef406c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.694+0000 7f7ef27fc640 1 --2- 172.21.15.16:0/730644486 >> v2:172.21.15.186:3300/0 conn(0x7f7ef40cece0 0x7f7ef406c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.694+0000 7f7ef27fc640 1 -- 172.21.15.16:0/730644486 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef4162260 con 0x7f7ef40b9720 2024-04-25T10:13:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.695+0000 7f7ef27fc640 1 --2- 172.21.15.16:0/730644486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef40b9720 0x7f7ef40ce7a0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40cab30 tx=0x7f7ee402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a221aa0817e3d8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.695+0000 7f7ef1ffb640 1 -- 172.21.15.16:0/730644486 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4037070 con 0x7f7ef40b9720 2024-04-25T10:13:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.695+0000 7f7ef1ffb640 1 -- 172.21.15.16:0/730644486 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee4032440 con 0x7f7ef40b9720 2024-04-25T10:13:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.695+0000 7f7ef1ffb640 1 -- 172.21.15.16:0/730644486 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4032810 con 0x7f7ef40b9720 2024-04-25T10:13:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.696+0000 7f7ef92ac640 1 -- 172.21.15.16:0/730644486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef40b9720 msgr2=0x7f7ef40ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.696+0000 7f7ef92ac640 1 --2- 172.21.15.16:0/730644486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef40b9720 0x7f7ef40ce7a0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40cab30 tx=0x7f7ee402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.696+0000 7f7ef92ac640 1 -- 172.21.15.16:0/730644486 shutdown_connections 2024-04-25T10:13:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.696+0000 7f7ef92ac640 1 --2- 172.21.15.16:0/730644486 >> v2:172.21.15.186:3300/0 conn(0x7f7ef40cece0 0x7f7ef406c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.696+0000 7f7ef92ac640 1 --2- 172.21.15.16:0/730644486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef40b9720 0x7f7ef40ce7a0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.696+0000 7f7ef92ac640 1 --2- 172.21.15.16:0/730644486 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ef4161090 0x7f7ef4161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.696+0000 7f7ef92ac640 1 -- 172.21.15.16:0/730644486 >> 172.21.15.16:0/730644486 conn(0x7f7ef40b55e0 msgr2=0x7f7ef40b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.696+0000 7f7ef92ac640 1 -- 172.21.15.16:0/730644486 shutdown_connections 2024-04-25T10:13:03.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.696+0000 7f7ef92ac640 1 -- 172.21.15.16:0/730644486 wait complete. 2024-04-25T10:13:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.697+0000 7f7ef92ac640 1 Processor -- start 2024-04-25T10:13:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.697+0000 7f7ef92ac640 1 -- start start 2024-04-25T10:13:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.698+0000 7f7ef92ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef40b9720 0x7f7ef40c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.698+0000 7f7ef92ac640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ef40cece0 0x7f7ef4156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.698+0000 7f7ef92ac640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ef4161090 0x7f7ef40ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.698+0000 7f7ef92ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef4161470 con 0x7f7ef40b9720 2024-04-25T10:13:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.698+0000 7f7ef92ac640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef40d1e30 con 0x7f7ef40cece0 2024-04-25T10:13:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.698+0000 7f7ef92ac640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef4161c70 con 0x7f7ef4161090 2024-04-25T10:13:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.698+0000 7f7ef2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef40b9720 0x7f7ef40c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.698+0000 7f7ef2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef40b9720 0x7f7ef40c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41032/0 (socket says 172.21.15.16:41032) 2024-04-25T10:13:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.698+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/4032787531 learned_addr learned my addr 172.21.15.16:0/4032787531 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.698+0000 7f7ef37fe640 1 --2- 172.21.15.16:0/4032787531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ef4161090 0x7f7ef40ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.698+0000 7f7ef27fc640 1 --2- 172.21.15.16:0/4032787531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ef40cece0 0x7f7ef4156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.699+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/4032787531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ef4161090 msgr2=0x7f7ef40ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.699+0000 7f7ef2ffd640 1 --2- 172.21.15.16:0/4032787531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ef4161090 0x7f7ef40ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.699+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/4032787531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ef40cece0 msgr2=0x7f7ef4156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.699+0000 7f7ef2ffd640 1 --2- 172.21.15.16:0/4032787531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ef40cece0 0x7f7ef4156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.699+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/4032787531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef4155d40 con 0x7f7ef40b9720 2024-04-25T10:13:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.699+0000 7f7ef2ffd640 1 --2- 172.21.15.16:0/4032787531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef40b9720 0x7f7ef40c9460 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800e8a0 tx=0x7f7ee800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.699+0000 7f7edb7fe640 1 -- 172.21.15.16:0/4032787531 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8017070 con 0x7f7ef40b9720 2024-04-25T10:13:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.699+0000 7f7edb7fe640 1 -- 172.21.15.16:0/4032787531 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee800c940 con 0x7f7ef40b9720 2024-04-25T10:13:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.699+0000 7f7ef92ac640 1 -- 172.21.15.16:0/4032787531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef4155ff0 con 0x7f7ef40b9720 2024-04-25T10:13:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.699+0000 7f7edb7fe640 1 -- 172.21.15.16:0/4032787531 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee800cd30 con 0x7f7ef40b9720 2024-04-25T10:13:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.699+0000 7f7ef92ac640 1 -- 172.21.15.16:0/4032787531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef4156890 con 0x7f7ef40b9720 2024-04-25T10:13:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.701+0000 7f7edb7fe640 1 -- 172.21.15.16:0/4032787531 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ee801a070 con 0x7f7ef40b9720 2024-04-25T10:13:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.701+0000 7f7edb7fe640 1 --2- 172.21.15.16:0/4032787531 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7ee006b4d0 0x7f7ee006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.701+0000 7f7edb7fe640 1 -- 172.21.15.16:0/4032787531 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ee8089fe0 con 0x7f7ef40b9720 2024-04-25T10:13:03.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.702+0000 7f7ef27fc640 1 --2- 172.21.15.16:0/4032787531 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7ee006b4d0 0x7f7ee006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.702+0000 7f7efbfaf740 1 Processor -- start 2024-04-25T10:13:03.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.702+0000 7f7ef27fc640 1 --2- 172.21.15.16:0/4032787531 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7ee006b4d0 0x7f7ee006d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4034000 tx=0x7f7ee402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.702+0000 7f7efbfaf740 1 -- start start 2024-04-25T10:13:03.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.703+0000 7f7efbfaf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 0x55ea50cd5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.703+0000 7f7efbfaf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cd0830 0x55ea50cd47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.703+0000 7f7efbfaf740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd4cf0 0x55ea50cd5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.703+0000 7f7efbfaf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea50c39ac0 con 0x55ea50cd5bb0 2024-04-25T10:13:03.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.703+0000 7f7efbfaf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea50be2d80 con 0x55ea50cd0830 2024-04-25T10:13:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.703+0000 7f7efbfaf740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea50c35b60 con 0x55ea50cd4cf0 2024-04-25T10:13:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.703+0000 7f7ef2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 0x55ea50cd5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.703+0000 7f7ef2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 0x55ea50cd5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41038/0 (socket says 172.21.15.16:41038) 2024-04-25T10:13:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.703+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/3171301287 learned_addr learned my addr 172.21.15.16:0/3171301287 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.703+0000 7f7ef37fe640 1 --2- 172.21.15.16:0/3171301287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd4cf0 0x55ea50cd5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.703+0000 7f7ef27fc640 1 --2- 172.21.15.16:0/3171301287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cd0830 0x55ea50cd47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.703+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/3171301287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd4cf0 msgr2=0x55ea50cd5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.703+0000 7f7ef2ffd640 1 --2- 172.21.15.16:0/3171301287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd4cf0 0x55ea50cd5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.703+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/3171301287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cd0830 msgr2=0x55ea50cd47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.703+0000 7f7ef2ffd640 1 --2- 172.21.15.16:0/3171301287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cd0830 0x55ea50cd47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.703+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/3171301287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea50a167e0 con 0x55ea50cd5bb0 2024-04-25T10:13:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.704+0000 7f7ef2ffd640 1 --2- 172.21.15.16:0/3171301287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 0x55ea50cd5f90 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8088530 tx=0x7f7ee8058450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515db13076c26394 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.704+0000 7f7ed8ff9640 1 -- 172.21.15.16:0/3171301287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8021040 con 0x55ea50cd5bb0 2024-04-25T10:13:03.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.704+0000 7f7ed8ff9640 1 -- 172.21.15.16:0/3171301287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee8060e70 con 0x55ea50cd5bb0 2024-04-25T10:13:03.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.704+0000 7f7ed8ff9640 1 -- 172.21.15.16:0/3171301287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8070180 con 0x55ea50cd5bb0 2024-04-25T10:13:03.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.704+0000 7f7efbfaf740 1 -- 172.21.15.16:0/3171301287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 msgr2=0x55ea50cd5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.704+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/3171301287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 0x55ea50cd5f90 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8088530 tx=0x7f7ee8058450 comp rx=0 tx=0).stop 2024-04-25T10:13:03.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.704+0000 7f7efbfaf740 1 -- 172.21.15.16:0/3171301287 shutdown_connections 2024-04-25T10:13:03.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.704+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/3171301287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd4cf0 0x55ea50cd5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.704+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/3171301287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cd0830 0x55ea50cd47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.704+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/3171301287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 0x55ea50cd5f90 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.705+0000 7f7efbfaf740 1 -- 172.21.15.16:0/3171301287 >> 172.21.15.16:0/3171301287 conn(0x55ea50cd2cb0 msgr2=0x55ea50d1b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.705+0000 7f7efbfaf740 1 -- 172.21.15.16:0/3171301287 shutdown_connections 2024-04-25T10:13:03.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.705+0000 7f7efbfaf740 1 -- 172.21.15.16:0/3171301287 wait complete. 2024-04-25T10:13:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.705+0000 7f7efbfaf740 1 -- start start 2024-04-25T10:13:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.706+0000 7f7efbfaf740 1 Processor -- start 2024-04-25T10:13:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.707+0000 7f7efbfaf740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd4cf0 0x55ea50d23ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.707+0000 7f7efbfaf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 0x55ea50cf58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.707+0000 7f7efbfaf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cf3450 0x55ea50cf5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.707+0000 7f7efbfaf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea50d01740 con 0x55ea50cd5bb0 2024-04-25T10:13:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.707+0000 7f7efbfaf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea50bd8530 con 0x55ea50cf3450 2024-04-25T10:13:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.707+0000 7f7efbfaf740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea50d00bc0 con 0x55ea50cd4cf0 2024-04-25T10:13:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.707+0000 7f7efbfaf740 10 client.0 _ll_register_callbacks cb 0x55ea50c30130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.707+0000 7f7ef27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 0x55ea50cf58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.707+0000 7f7ef27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 0x55ea50cf58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41048/0 (socket says 172.21.15.16:41048) 2024-04-25T10:13:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.707+0000 7f7ef27fc640 1 -- 172.21.15.16:0/3166126574 learned_addr learned my addr 172.21.15.16:0/3166126574 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.707+0000 7f7ef2ffd640 1 --2- 172.21.15.16:0/3166126574 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd4cf0 0x55ea50d23ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.707+0000 7f7ef37fe640 1 --2- 172.21.15.16:0/3166126574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cf3450 0x55ea50cf5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.707+0000 7f7ef27fc640 1 -- 172.21.15.16:0/3166126574 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd4cf0 msgr2=0x55ea50d23ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.707+0000 7f7ef27fc640 1 --2- 172.21.15.16:0/3166126574 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd4cf0 0x55ea50d23ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.707+0000 7f7ef27fc640 1 -- 172.21.15.16:0/3166126574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cf3450 msgr2=0x55ea50cf5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.707+0000 7f7ef27fc640 1 --2- 172.21.15.16:0/3166126574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cf3450 0x55ea50cf5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.707+0000 7f7ef27fc640 1 -- 172.21.15.16:0/3166126574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea50a76f70 con 0x55ea50cd5bb0 2024-04-25T10:13:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.707+0000 7f7ef37fe640 1 --2- 172.21.15.16:0/3166126574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cf3450 0x55ea50cf5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.708+0000 7f7ef27fc640 1 --2- 172.21.15.16:0/3166126574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 0x55ea50cf58e0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009960 tx=0x7f7ee4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1169f729421c0203 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.708+0000 7f7ed1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:13:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.708+0000 7f7ed1ffb640 1 -- 172.21.15.16:0/3166126574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee404e070 con 0x55ea50cd5bb0 2024-04-25T10:13:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.708+0000 7f7ed1ffb640 1 -- 172.21.15.16:0/3166126574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ee403c640 con 0x55ea50cd5bb0 2024-04-25T10:13:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.708+0000 7f7efbfaf740 1 -- 172.21.15.16:0/3166126574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ee8060e70 con 0x55ea50cd5bb0 2024-04-25T10:13:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.708+0000 7f7ed1ffb640 1 -- 172.21.15.16:0/3166126574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ee4037040 con 0x55ea50cd5bb0 2024-04-25T10:13:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.708+0000 7f7efbfaf740 10 client.8472 fetch_fsmap learned FSMap version 71 2024-04-25T10:13:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.708+0000 7f7efbfaf740 1 -- 172.21.15.16:0/3166126574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55ea5090e170 con 0x55ea50cd5bb0 2024-04-25T10:13:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.708+0000 7f7ed1ffb640 1 -- 172.21.15.16:0/3166126574 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee8060e70 con 0x55ea50cd5bb0 2024-04-25T10:13:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.708+0000 7f7ed1ffb640 1 -- 172.21.15.16:0/3166126574 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7ee4046040 con 0x55ea50cd5bb0 2024-04-25T10:13:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.708+0000 7f7efbfaf740 10 client.8472 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:13:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.708+0000 7f7efbfaf740 10 client.8472 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:13:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.708+0000 7f7efbfaf740 20 client.8472 populate_metadata read hostname 'smithi016' 2024-04-25T10:13:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.708+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/3166126574 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x55ea50d223b0 0x55ea50d227c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.708+0000 7f7efbfaf740 4 client.8472 mds_command: new command op to 8202 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.708+0000 7f7efbfaf740 1 -- 172.21.15.16:0/3166126574 --> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea50c47000 con 0x55ea50d223b0 2024-04-25T10:13:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.709+0000 7f7ef2ffd640 1 --2- 172.21.15.16:0/3166126574 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x55ea50d223b0 0x55ea50d227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.709+0000 7f7ed1ffb640 10 client.8472 ms_handle_connect on v2:172.21.15.186:6834/1371188714 2024-04-25T10:13:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.709+0000 7f7ef2ffd640 1 --2- 172.21.15.16:0/3166126574 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x55ea50d223b0 0x55ea50d227c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=840da883a9305382 server_cookie=f38eb69ceeb6ec59 in_seq=0 out_seq=0 2024-04-25T10:13:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.710+0000 7f7ed1ffb640 1 -- 172.21.15.16:0/3166126574 <== mds.0 v2:172.21.15.186:6834/1371188714 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ea50c47000 con 0x55ea50d223b0 2024-04-25T10:13:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.710+0000 7f7ed1ffb640 10 client.8472 handle_command_reply: tid=0 2024-04-25T10:13:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.710+0000 7f7efbfaf740 1 client.8472 shutdown 2024-04-25T10:13:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.710+0000 7f7efbfaf740 1 -- 172.21.15.16:0/3166126574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 msgr2=0x55ea50cf58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.710+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/3166126574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 0x55ea50cf58e0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009960 tx=0x7f7ee4004730 comp rx=0 tx=0).stop 2024-04-25T10:13:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.710+0000 7f7efbfaf740 1 -- 172.21.15.16:0/3166126574 shutdown_connections 2024-04-25T10:13:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.710+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/3166126574 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x55ea50d223b0 0x55ea50d227c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.710+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/3166126574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cf3450 0x55ea50cf5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.710+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/3166126574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 0x55ea50cf58e0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.710+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/3166126574 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd4cf0 0x55ea50d23ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.710+0000 7f7ed1ffb640 0 client.8472 ms_handle_reset on v2:172.21.15.186:6834/1371188714 2024-04-25T10:13:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.710+0000 7f7efbfaf740 1 -- 172.21.15.16:0/3166126574 >> 172.21.15.16:0/3166126574 conn(0x55ea50cd0830 msgr2=0x55ea50ccff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.710+0000 7f7efbfaf740 1 -- 172.21.15.16:0/3166126574 shutdown_connections 2024-04-25T10:13:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.710+0000 7f7efbfaf740 1 -- 172.21.15.16:0/3166126574 wait complete. 2024-04-25T10:13:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.710+0000 7f7efbfaf740 20 client.8472 trim_cache size 0 max 16384 2024-04-25T10:13:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.716+0000 7f477c85d640 1 Processor -- start 2024-04-25T10:13:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.716+0000 7f477c85d640 1 -- start start 2024-04-25T10:13:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.716+0000 7f477c85d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47780cea80 0x7f4778162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.716+0000 7f477c85d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477815f090 con 0x7f47780b5bb0 2024-04-25T10:13:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.716+0000 7f477c85d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47780cf060 con 0x7f477815dca0 2024-04-25T10:13:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.716+0000 7f477c85d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47780cf340 con 0x7f47780cea80 2024-04-25T10:13:03.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.717+0000 7f4776d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47780cea80 0x7f4778162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.717+0000 7f4776d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47780cea80 0x7f4778162200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41054/0 (socket says 172.21.15.16:41054) 2024-04-25T10:13:03.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.717+0000 7f4776d9d640 1 -- 172.21.15.16:0/3218779657 learned_addr learned my addr 172.21.15.16:0/3218779657 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:03.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.717+0000 7f476dd9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f47780b5bb0 0x7f47780d1c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53824/0 (socket says 172.21.15.16:53824) 2024-04-25T10:13:03.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.717+0000 7f4776d9d640 1 -- 172.21.15.16:0/3218779657 >> v1:172.21.15.186:6789/0 conn(0x7f477815dca0 legacy=0x7f47780b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.717+0000 7f4776d9d640 1 -- 172.21.15.16:0/3218779657 >> v1:172.21.15.16:6789/0 conn(0x7f47780b5bb0 legacy=0x7f47780d1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.717+0000 7f4775d9b640 1 -- 172.21.15.16:0/3218779657 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889660159 0 0) 0x7f477815f090 con 0x7f47780b5bb0 2024-04-25T10:13:03.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.717+0000 7f4776d9d640 1 -- 172.21.15.16:0/3218779657 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477806c1a0 con 0x7f47780cea80 2024-04-25T10:13:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.718+0000 7f4776d9d640 1 --2- 172.21.15.16:0/3218779657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47780cea80 0x7f4778162200 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f4770009e50 tx=0x7f477002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f938f6bc30bd99c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.718+0000 7f4775d9b640 1 -- 172.21.15.16:0/3218779657 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770036070 con 0x7f47780cea80 2024-04-25T10:13:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.718+0000 7f4775d9b640 1 -- 172.21.15.16:0/3218779657 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4770031600 con 0x7f47780cea80 2024-04-25T10:13:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.718+0000 7f4775d9b640 1 -- 172.21.15.16:0/3218779657 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47700319d0 con 0x7f47780cea80 2024-04-25T10:13:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.718+0000 7f477c85d640 1 -- 172.21.15.16:0/3218779657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47780cea80 msgr2=0x7f4778162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.718+0000 7f477c85d640 1 --2- 172.21.15.16:0/3218779657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47780cea80 0x7f4778162200 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f4770009e50 tx=0x7f477002a730 comp rx=0 tx=0).stop 2024-04-25T10:13:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.718+0000 7f477c85d640 1 -- 172.21.15.16:0/3218779657 shutdown_connections 2024-04-25T10:13:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.718+0000 7f477c85d640 1 --2- 172.21.15.16:0/3218779657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47780cea80 0x7f4778162200 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.718+0000 7f477c85d640 1 -- 172.21.15.16:0/3218779657 >> 172.21.15.16:0/3218779657 conn(0x7f47780d04a0 msgr2=0x7f47780d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.718+0000 7f477c85d640 1 -- 172.21.15.16:0/3218779657 shutdown_connections 2024-04-25T10:13:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.718+0000 7f477c85d640 1 -- 172.21.15.16:0/3218779657 wait complete. 2024-04-25T10:13:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.720+0000 7f477c85d640 1 Processor -- start 2024-04-25T10:13:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.720+0000 7f477c85d640 1 -- start start 2024-04-25T10:13:03.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f477c85d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47780b5bb0 0x7f47780c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f477c85d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47780cea80 0x7f47780aa390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f477c85d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f477815dca0 0x7f477814c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f477c85d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477806c760 con 0x7f47780b5bb0 2024-04-25T10:13:03.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f477c85d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477806c480 con 0x7f47780cea80 2024-04-25T10:13:03.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f477c85d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477806cc80 con 0x7f477815dca0 2024-04-25T10:13:03.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f477659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47780b5bb0 0x7f47780c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f477659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47780b5bb0 0x7f47780c9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41066/0 (socket says 172.21.15.16:41066) 2024-04-25T10:13:03.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f477659c640 1 -- 172.21.15.16:0/1053652108 learned_addr learned my addr 172.21.15.16:0/1053652108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:03.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f476dd9b640 1 --2- 172.21.15.16:0/1053652108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47780cea80 0x7f47780aa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f477659c640 1 -- 172.21.15.16:0/1053652108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f477815dca0 msgr2=0x7f477814c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f4776d9d640 1 --2- 172.21.15.16:0/1053652108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f477815dca0 0x7f477814c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f477659c640 1 --2- 172.21.15.16:0/1053652108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f477815dca0 0x7f477814c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f477659c640 1 -- 172.21.15.16:0/1053652108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47780cea80 msgr2=0x7f47780aa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f477659c640 1 --2- 172.21.15.16:0/1053652108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47780cea80 0x7f47780aa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f477659c640 1 -- 172.21.15.16:0/1053652108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47780cb6c0 con 0x7f47780b5bb0 2024-04-25T10:13:03.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f476dd9b640 1 --2- 172.21.15.16:0/1053652108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47780cea80 0x7f47780aa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:03.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f4776d9d640 1 --2- 172.21.15.16:0/1053652108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f477815dca0 0x7f477814c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:03.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f477659c640 1 --2- 172.21.15.16:0/1053652108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47780b5bb0 0x7f47780c9d80 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f476800c400 tx=0x7f476800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f476f7fe640 1 -- 172.21.15.16:0/1053652108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4768017070 con 0x7f47780b5bb0 2024-04-25T10:13:03.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f476f7fe640 1 -- 172.21.15.16:0/1053652108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47680052d0 con 0x7f47780b5bb0 2024-04-25T10:13:03.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.721+0000 7f477c85d640 1 -- 172.21.15.16:0/1053652108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477814a4a0 con 0x7f47780b5bb0 2024-04-25T10:13:03.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.722+0000 7f476f7fe640 1 -- 172.21.15.16:0/1053652108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4768003830 con 0x7f47780b5bb0 2024-04-25T10:13:03.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.722+0000 7f477c85d640 1 -- 172.21.15.16:0/1053652108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477814aa60 con 0x7f47780b5bb0 2024-04-25T10:13:03.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.722+0000 7f477c85d640 1 -- 172.21.15.16:0/1053652108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f47780749b0 con 0x7f47780b5bb0 2024-04-25T10:13:03.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.723+0000 7f476f7fe640 1 -- 172.21.15.16:0/1053652108 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f476801a070 con 0x7f47780b5bb0 2024-04-25T10:13:03.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.723+0000 7f476f7fe640 1 --2- 172.21.15.16:0/1053652108 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f475806b4d0 0x7f475806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.724+0000 7f476f7fe640 1 -- 172.21.15.16:0/1053652108 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4768089c90 con 0x7f47780b5bb0 2024-04-25T10:13:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.724+0000 7f476dd9b640 1 --2- 172.21.15.16:0/1053652108 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f475806b4d0 0x7f475806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.724+0000 7f476f7fe640 1 -- 172.21.15.16:0/1053652108 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+512 (secure 0 0 0) 0x7f4768021070 con 0x7f47780b5bb0 2024-04-25T10:13:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.724+0000 7f477c85d640 1 -- 172.21.15.16:0/1053652108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f47780d1810 con 0x7f47780b5bb0 2024-04-25T10:13:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.724+0000 7f476dd9b640 1 --2- 172.21.15.16:0/1053652108 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f475806b4d0 0x7f475806d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f47780ce2e0 tx=0x7f4760009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.727+0000 7f7efbfaf740 1 Processor -- start 2024-04-25T10:13:03.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.727+0000 7f7efbfaf740 1 -- start start 2024-04-25T10:13:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.727+0000 7f7efbfaf740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd4cf0 0x55ea50cd50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.727+0000 7f7efbfaf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 0x55ea50d35a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.727+0000 7f7efbfaf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cf3450 0x55ea50cf6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.727+0000 7f7efbfaf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea50cd3bc0 con 0x55ea50cd5bb0 2024-04-25T10:13:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.727+0000 7f7efbfaf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef41fff90 con 0x55ea50cf3450 2024-04-25T10:13:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.727+0000 7f7efbfaf740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef40cb920 con 0x55ea50cd4cf0 2024-04-25T10:13:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.728+0000 7f7ef27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 0x55ea50d35a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.728+0000 7f7ef27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 0x55ea50d35a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41076/0 (socket says 172.21.15.16:41076) 2024-04-25T10:13:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.728+0000 7f7ef27fc640 1 -- 172.21.15.16:0/3749223347 learned_addr learned my addr 172.21.15.16:0/3749223347 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.728+0000 7f7ef2ffd640 1 --2- 172.21.15.16:0/3749223347 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd4cf0 0x55ea50cd50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.728+0000 7f7ef37fe640 1 --2- 172.21.15.16:0/3749223347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cf3450 0x55ea50cf6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.728+0000 7f7ef27fc640 1 -- 172.21.15.16:0/3749223347 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd4cf0 msgr2=0x55ea50cd50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.728+0000 7f7ef27fc640 1 --2- 172.21.15.16:0/3749223347 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd4cf0 0x55ea50cd50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.728+0000 7f7ef27fc640 1 -- 172.21.15.16:0/3749223347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cf3450 msgr2=0x55ea50cf6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.728+0000 7f7ef27fc640 1 --2- 172.21.15.16:0/3749223347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cf3450 0x55ea50cf6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.728+0000 7f7ef27fc640 1 -- 172.21.15.16:0/3749223347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea50cd2780 con 0x55ea50cd5bb0 2024-04-25T10:13:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.728+0000 7f7ef27fc640 1 --2- 172.21.15.16:0/3749223347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 0x55ea50d35a50 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4032460 tx=0x7f7ee4047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0ef9989a0f3ac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.728+0000 7f7ed3fff640 1 -- 172.21.15.16:0/3749223347 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee404e040 con 0x55ea50cd5bb0 2024-04-25T10:13:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.728+0000 7f7ed3fff640 1 -- 172.21.15.16:0/3749223347 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee403cd60 con 0x55ea50cd5bb0 2024-04-25T10:13:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.728+0000 7f7ed3fff640 1 -- 172.21.15.16:0/3749223347 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee403c660 con 0x55ea50cd5bb0 2024-04-25T10:13:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.729+0000 7f7efbfaf740 1 -- 172.21.15.16:0/3749223347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 msgr2=0x55ea50d35a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.729+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/3749223347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 0x55ea50d35a50 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4032460 tx=0x7f7ee4047000 comp rx=0 tx=0).stop 2024-04-25T10:13:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.729+0000 7f7efbfaf740 1 -- 172.21.15.16:0/3749223347 shutdown_connections 2024-04-25T10:13:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.729+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/3749223347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cf3450 0x55ea50cf6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.729+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/3749223347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd5bb0 0x55ea50d35a50 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.729+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/3749223347 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd4cf0 0x55ea50cd50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.729+0000 7f7efbfaf740 1 -- 172.21.15.16:0/3749223347 >> 172.21.15.16:0/3749223347 conn(0x55ea50cd0830 msgr2=0x55ea50d1f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.729+0000 7f7efbfaf740 1 -- 172.21.15.16:0/3749223347 shutdown_connections 2024-04-25T10:13:03.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.729+0000 7f7efbfaf740 1 -- 172.21.15.16:0/3749223347 wait complete. 2024-04-25T10:13:03.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.729+0000 7f7efbfaf740 1 -- start start 2024-04-25T10:13:03.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.729+0000 7f7efbfaf740 1 Processor -- start 2024-04-25T10:13:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.730+0000 7f7efbfaf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd4cf0 0x55ea50d6f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.730+0000 7f7efbfaf740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd5bb0 0x55ea50cf65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.730+0000 7f7efbfaf740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cf3450 0x55ea50cf6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.730+0000 7f7efbfaf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef40cb320 con 0x55ea50cd4cf0 2024-04-25T10:13:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.730+0000 7f7efbfaf740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef40cad20 con 0x55ea50cf3450 2024-04-25T10:13:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.730+0000 7f7efbfaf740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea50d11ff0 con 0x55ea50cd5bb0 2024-04-25T10:13:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.730+0000 7f7efbfaf740 10 client.0 _ll_register_callbacks cb 0x7f7eec009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.730+0000 7f7ef2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd4cf0 0x55ea50d6f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.730+0000 7f7ef2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd4cf0 0x55ea50d6f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41092/0 (socket says 172.21.15.16:41092) 2024-04-25T10:13:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.730+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/2246768873 learned_addr learned my addr 172.21.15.16:0/2246768873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7ef37fe640 1 --2- 172.21.15.16:0/2246768873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cf3450 0x55ea50cf6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7ef27fc640 1 --2- 172.21.15.16:0/2246768873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd5bb0 0x55ea50cf65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/2246768873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd5bb0 msgr2=0x55ea50cf65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7ef2ffd640 1 --2- 172.21.15.16:0/2246768873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd5bb0 0x55ea50cf65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/2246768873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cf3450 msgr2=0x55ea50cf6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7ef2ffd640 1 --2- 172.21.15.16:0/2246768873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cf3450 0x55ea50cf6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7ef2ffd640 1 -- 172.21.15.16:0/2246768873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea50d1fe20 con 0x55ea50cd4cf0 2024-04-25T10:13:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7ef27fc640 1 --2- 172.21.15.16:0/2246768873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd5bb0 0x55ea50cf65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7ef2ffd640 1 --2- 172.21.15.16:0/2246768873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd4cf0 0x55ea50d6f7d0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800cf80 tx=0x7f7ee8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db8d8ae0e520936 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7ed1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:13:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7ed1ffb640 1 -- 172.21.15.16:0/2246768873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8075060 con 0x55ea50cd4cf0 2024-04-25T10:13:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7ed1ffb640 1 -- 172.21.15.16:0/2246768873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ee8070180 con 0x55ea50cd4cf0 2024-04-25T10:13:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7efbfaf740 1 -- 172.21.15.16:0/2246768873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ee403cd60 con 0x55ea50cd4cf0 2024-04-25T10:13:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7ed1ffb640 1 -- 172.21.15.16:0/2246768873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ee8087040 con 0x55ea50cd4cf0 2024-04-25T10:13:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7efbfaf740 10 client.8487 fetch_fsmap learned FSMap version 71 2024-04-25T10:13:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7efbfaf740 1 -- 172.21.15.16:0/2246768873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55ea50d871a0 con 0x55ea50cd4cf0 2024-04-25T10:13:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7ed1ffb640 1 -- 172.21.15.16:0/2246768873 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee403cd60 con 0x55ea50cd4cf0 2024-04-25T10:13:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7ed1ffb640 1 -- 172.21.15.16:0/2246768873 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7ee8024060 con 0x55ea50cd4cf0 2024-04-25T10:13:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7efbfaf740 10 client.8487 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:13:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7efbfaf740 10 client.8487 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:13:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7efbfaf740 20 client.8487 populate_metadata read hostname 'smithi016' 2024-04-25T10:13:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/2246768873 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x55ea50cf94e0 0x55ea50d2a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7efbfaf740 4 client.8487 mds_command: new command op to 8202 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:13:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.731+0000 7f7efbfaf740 1 -- 172.21.15.16:0/2246768873 --> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55ea50c41350 con 0x55ea50cf94e0 2024-04-25T10:13:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.732+0000 7f7ef27fc640 1 --2- 172.21.15.16:0/2246768873 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x55ea50cf94e0 0x55ea50d2a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.732+0000 7f7ed1ffb640 10 client.8487 ms_handle_connect on v2:172.21.15.186:6834/1371188714 2024-04-25T10:13:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.732+0000 7f7ef27fc640 1 --2- 172.21.15.16:0/2246768873 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x55ea50cf94e0 0x55ea50d2a4a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ceada7049af0ef server_cookie=4ce005a4dba66626 in_seq=0 out_seq=0 2024-04-25T10:13:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.733+0000 7f7ed1ffb640 1 -- 172.21.15.16:0/2246768873 <== mds.0 v2:172.21.15.186:6834/1371188714 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ea50c41350 con 0x55ea50cf94e0 2024-04-25T10:13:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.733+0000 7f7ed1ffb640 10 client.8487 handle_command_reply: tid=0 2024-04-25T10:13:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.733+0000 7f7efbfaf740 1 client.8487 shutdown 2024-04-25T10:13:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.733+0000 7f7efbfaf740 1 -- 172.21.15.16:0/2246768873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd4cf0 msgr2=0x55ea50d6f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.733+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/2246768873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd4cf0 0x55ea50d6f7d0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800cf80 tx=0x7f7ee8000bc0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.733+0000 7f7efbfaf740 1 -- 172.21.15.16:0/2246768873 shutdown_connections 2024-04-25T10:13:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.733+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/2246768873 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x55ea50cf94e0 0x55ea50d2a4a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.733+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/2246768873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea50cf3450 0x55ea50cf6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.733+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/2246768873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ea50cd5bb0 0x55ea50cf65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.733+0000 7f7ed1ffb640 0 client.8487 ms_handle_reset on v2:172.21.15.186:6834/1371188714 2024-04-25T10:13:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.733+0000 7f7efbfaf740 1 --2- 172.21.15.16:0/2246768873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea50cd4cf0 0x55ea50d6f7d0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.733+0000 7f7efbfaf740 1 -- 172.21.15.16:0/2246768873 >> 172.21.15.16:0/2246768873 conn(0x55ea50cd0830 msgr2=0x55ea50d35840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.733+0000 7f7efbfaf740 1 -- 172.21.15.16:0/2246768873 shutdown_connections 2024-04-25T10:13:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.733+0000 7f7efbfaf740 1 -- 172.21.15.16:0/2246768873 wait complete. 2024-04-25T10:13:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.733+0000 7f7efbfaf740 20 client.8487 trim_cache size 0 max 16384 2024-04-25T10:13:03.744 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:13:03.744 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-04-25T10:13:03.744 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "8fdaec76-0790-4692-99ba-909902a5d389", 2024-04-25T10:13:03.744 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-04-25T10:13:03.744 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:13:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.733+0000 7f7ef92ac640 1 -- 172.21.15.16:0/4032787531 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7ee006b4d0 msgr2=0x7f7ee006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.733+0000 7f7ef92ac640 1 --2- 172.21.15.16:0/4032787531 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7ee006b4d0 0x7f7ee006d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4034000 tx=0x7f7ee402e000 comp rx=0 tx=0).stop 2024-04-25T10:13:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.734+0000 7f7ef92ac640 1 -- 172.21.15.16:0/4032787531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef40b9720 msgr2=0x7f7ef40c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.734+0000 7f7ef92ac640 1 --2- 172.21.15.16:0/4032787531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef40b9720 0x7f7ef40c9460 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800e8a0 tx=0x7f7ee800ed60 comp rx=0 tx=0).stop 2024-04-25T10:13:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.734+0000 7f7ef92ac640 1 -- 172.21.15.16:0/4032787531 shutdown_connections 2024-04-25T10:13:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.734+0000 7f7ef92ac640 1 --2- 172.21.15.16:0/4032787531 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f7ee006b4d0 0x7f7ee006d990 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.734+0000 7f7ef92ac640 1 --2- 172.21.15.16:0/4032787531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ef4161090 0x7f7ef40ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.734+0000 7f7ef92ac640 1 --2- 172.21.15.16:0/4032787531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ef40cece0 0x7f7ef4156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.734+0000 7f7ef92ac640 1 --2- 172.21.15.16:0/4032787531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef40b9720 0x7f7ef40c9460 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.734+0000 7f7ef92ac640 1 -- 172.21.15.16:0/4032787531 >> 172.21.15.16:0/4032787531 conn(0x7f7ef40b55e0 msgr2=0x7f7ef40d39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.734+0000 7f7ef92ac640 1 -- 172.21.15.16:0/4032787531 shutdown_connections 2024-04-25T10:13:03.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.734+0000 7f7ef92ac640 1 -- 172.21.15.16:0/4032787531 wait complete. 2024-04-25T10:13:03.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:03.749 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:13:03.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.847+0000 7f05e91c1640 1 Processor -- start 2024-04-25T10:13:03.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.847+0000 7f05e91c1640 1 -- start start 2024-04-25T10:13:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.848+0000 7f05e91c1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05e40d3b90 0x7f05e40d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.848+0000 7f05e91c1640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f05e40d0f60 0x7f05e40b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.848+0000 7f05e91c1640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e40d4090 con 0x7f05e415dca0 2024-04-25T10:13:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.848+0000 7f05e91c1640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e40b6b20 con 0x7f05e40d0f60 2024-04-25T10:13:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.848+0000 7f05e91c1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e40d4370 con 0x7f05e40d3b90 2024-04-25T10:13:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.848+0000 7f05e359e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f05e40d0f60 0x7f05e40b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.848+0000 7f05e259c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05e40d3b90 0x7f05e40d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.848+0000 7f05e2d9d640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f05e415dca0 0x7f05e40ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:44408/0 (socket says 172.21.15.16:44408) 2024-04-25T10:13:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.848+0000 7f05e259c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05e40d3b90 0x7f05e40d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:36014/0 (socket says 172.21.15.16:36014) 2024-04-25T10:13:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.848+0000 7f05e2d9d640 1 -- 172.21.15.16:0/1782315629 learned_addr learned my addr 172.21.15.16:0/1782315629 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:13:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.848+0000 7f05e359e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f05e40d0f60 0x7f05e40b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:43516/0 (socket says 172.21.15.16:43516) 2024-04-25T10:13:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.849+0000 7f05e1d9b640 1 -- 172.21.15.16:0/1782315629 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119110665 0 0) 0x7f05e40d4090 con 0x7f05e415dca0 2024-04-25T10:13:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.849+0000 7f05e1d9b640 1 -- 172.21.15.16:0/1782315629 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05d0003520 con 0x7f05e415dca0 2024-04-25T10:13:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.849+0000 7f05e259c640 1 -- 172.21.15.16:0/1782315629 >> v2:172.21.15.186:3300/0 conn(0x7f05e40d0f60 msgr2=0x7f05e40b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.849+0000 7f05e259c640 1 --2- 172.21.15.16:0/1782315629 >> v2:172.21.15.186:3300/0 conn(0x7f05e40d0f60 0x7f05e40b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.849+0000 7f05e259c640 1 -- 172.21.15.16:0/1782315629 >> v1:172.21.15.186:6789/0 conn(0x7f05e415dca0 legacy=0x7f05e40ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.849+0000 7f05e259c640 1 -- 172.21.15.16:0/1782315629 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e40d4650 con 0x7f05e40d3b90 2024-04-25T10:13:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.849+0000 7f05e359e640 1 --2- 172.21.15.16:0/1782315629 >> v2:172.21.15.186:3300/0 conn(0x7f05e40d0f60 0x7f05e40b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.849+0000 7f05e259c640 1 --2- 172.21.15.16:0/1782315629 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05e40d3b90 0x7f05e40d0a20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0097a0 tx=0x7f05cc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dc2849ca41919c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.850+0000 7f05e1d9b640 1 -- 172.21.15.16:0/1782315629 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc037070 con 0x7f05e40d3b90 2024-04-25T10:13:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.850+0000 7f05e1d9b640 1 -- 172.21.15.16:0/1782315629 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05cc032440 con 0x7f05e40d3b90 2024-04-25T10:13:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.850+0000 7f05e1d9b640 1 -- 172.21.15.16:0/1782315629 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc032810 con 0x7f05e40d3b90 2024-04-25T10:13:03.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.850+0000 7f05e91c1640 1 -- 172.21.15.16:0/1782315629 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05e40d3b90 msgr2=0x7f05e40d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.850+0000 7f05e91c1640 1 --2- 172.21.15.16:0/1782315629 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05e40d3b90 0x7f05e40d0a20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0097a0 tx=0x7f05cc02a180 comp rx=0 tx=0).stop 2024-04-25T10:13:03.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.850+0000 7f05e91c1640 1 -- 172.21.15.16:0/1782315629 shutdown_connections 2024-04-25T10:13:03.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.850+0000 7f05e91c1640 1 --2- 172.21.15.16:0/1782315629 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05e40d3b90 0x7f05e40d0a20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.850+0000 7f05e91c1640 1 --2- 172.21.15.16:0/1782315629 >> v2:172.21.15.186:3300/0 conn(0x7f05e40d0f60 0x7f05e40b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.850+0000 7f05e91c1640 1 -- 172.21.15.16:0/1782315629 >> 172.21.15.16:0/1782315629 conn(0x7f05e40726c0 msgr2=0x7f05e4072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.850+0000 7f05e91c1640 1 -- 172.21.15.16:0/1782315629 shutdown_connections 2024-04-25T10:13:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.850+0000 7f05e91c1640 1 -- 172.21.15.16:0/1782315629 wait complete. 2024-04-25T10:13:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.852+0000 7f05e91c1640 1 Processor -- start 2024-04-25T10:13:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.852+0000 7f05e91c1640 1 -- start start 2024-04-25T10:13:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.852+0000 7f05e91c1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05e40d0f60 0x7f05e40ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.852+0000 7f05e91c1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05e40d3b90 0x7f05e40cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.852+0000 7f05e91c1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05e415dca0 0x7f05e40cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.852+0000 7f05e91c1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e415e080 con 0x7f05e415dca0 2024-04-25T10:13:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.852+0000 7f05e91c1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e40cf050 con 0x7f05e40d3b90 2024-04-25T10:13:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.852+0000 7f05e91c1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e415e200 con 0x7f05e40d0f60 2024-04-25T10:13:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.853+0000 7f05e359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05e415dca0 0x7f05e40cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.853+0000 7f05e359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05e415dca0 0x7f05e40cbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41106/0 (socket says 172.21.15.16:41106) 2024-04-25T10:13:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.853+0000 7f05e359e640 1 -- 172.21.15.16:0/3006737807 learned_addr learned my addr 172.21.15.16:0/3006737807 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.853+0000 7f05e2d9d640 1 --2- 172.21.15.16:0/3006737807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05e40d0f60 0x7f05e40ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.853+0000 7f05e259c640 1 --2- 172.21.15.16:0/3006737807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05e40d3b90 0x7f05e40cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.853+0000 7f05e359e640 1 -- 172.21.15.16:0/3006737807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05e40d0f60 msgr2=0x7f05e40ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.853+0000 7f05e359e640 1 --2- 172.21.15.16:0/3006737807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05e40d0f60 0x7f05e40ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.853+0000 7f05e359e640 1 -- 172.21.15.16:0/3006737807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05e40d3b90 msgr2=0x7f05e40cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.853+0000 7f05e359e640 1 --2- 172.21.15.16:0/3006737807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05e40d3b90 0x7f05e40cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.853+0000 7f05e359e640 1 -- 172.21.15.16:0/3006737807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e40cac00 con 0x7f05e415dca0 2024-04-25T10:13:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.853+0000 7f05e259c640 1 --2- 172.21.15.16:0/3006737807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05e40d3b90 0x7f05e40cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.853+0000 7f05e359e640 1 --2- 172.21.15.16:0/3006737807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05e415dca0 0x7f05e40cbc60 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009f00 tx=0x7f05d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.853+0000 7f05db7fe640 1 -- 172.21.15.16:0/3006737807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4018030 con 0x7f05e415dca0 2024-04-25T10:13:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.853+0000 7f05db7fe640 1 -- 172.21.15.16:0/3006737807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d400ce50 con 0x7f05e415dca0 2024-04-25T10:13:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.853+0000 7f05e91c1640 1 -- 172.21.15.16:0/3006737807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e40caea0 con 0x7f05e415dca0 2024-04-25T10:13:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.853+0000 7f05db7fe640 1 -- 172.21.15.16:0/3006737807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4003bb0 con 0x7f05e415dca0 2024-04-25T10:13:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.853+0000 7f05e91c1640 1 -- 172.21.15.16:0/3006737807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e40cb460 con 0x7f05e415dca0 2024-04-25T10:13:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.855+0000 7f05db7fe640 1 -- 172.21.15.16:0/3006737807 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05d4005d10 con 0x7f05e415dca0 2024-04-25T10:13:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.855+0000 7f05db7fe640 1 --2- 172.21.15.16:0/3006737807 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f05c406b4d0 0x7f05c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.855+0000 7f05db7fe640 1 -- 172.21.15.16:0/3006737807 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05d408ac30 con 0x7f05e415dca0 2024-04-25T10:13:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.856+0000 7f05e2d9d640 1 --2- 172.21.15.16:0/3006737807 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f05c406b4d0 0x7f05c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.856+0000 7f05e2d9d640 1 --2- 172.21.15.16:0/3006737807 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f05c406b4d0 0x7f05c406d990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f05dc005880 tx=0x7f05dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.856+0000 7f05ebf9e740 1 Processor -- start 2024-04-25T10:13:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.856+0000 7f05ebf9e740 1 -- start start 2024-04-25T10:13:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.857+0000 7f05ebf9e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba683b10 0x555bba686320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.857+0000 7f05ebf9e740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba683ef0 0x555bba686860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.857+0000 7f05ebf9e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba656770 0x555bba659f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.857+0000 7f05ebf9e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bba5bfda0 con 0x555bba656770 2024-04-25T10:13:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.857+0000 7f05ebf9e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bba568b80 con 0x555bba683b10 2024-04-25T10:13:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.857+0000 7f05ebf9e740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bba5bbe40 con 0x555bba683ef0 2024-04-25T10:13:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.857+0000 7f05e359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba656770 0x555bba659f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.857+0000 7f05e359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba656770 0x555bba659f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41110/0 (socket says 172.21.15.16:41110) 2024-04-25T10:13:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.857+0000 7f05e359e640 1 -- 172.21.15.16:0/1517009394 learned_addr learned my addr 172.21.15.16:0/1517009394 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.857+0000 7f05e2d9d640 1 --2- 172.21.15.16:0/1517009394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba683b10 0x555bba686320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.857+0000 7f05e259c640 1 --2- 172.21.15.16:0/1517009394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba683ef0 0x555bba686860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.857+0000 7f05e359e640 1 -- 172.21.15.16:0/1517009394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba683ef0 msgr2=0x555bba686860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.857+0000 7f05e359e640 1 --2- 172.21.15.16:0/1517009394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba683ef0 0x555bba686860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.857+0000 7f05e359e640 1 -- 172.21.15.16:0/1517009394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba683b10 msgr2=0x555bba686320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.857+0000 7f05e359e640 1 --2- 172.21.15.16:0/1517009394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba683b10 0x555bba686320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.857+0000 7f05e359e640 1 -- 172.21.15.16:0/1517009394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bba572f10 con 0x555bba656770 2024-04-25T10:13:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.857+0000 7f05e259c640 1 --2- 172.21.15.16:0/1517009394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba683ef0 0x555bba686860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.858+0000 7f05e359e640 1 --2- 172.21.15.16:0/1517009394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba656770 0x555bba659f80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f05d4089100 tx=0x7f05d4059bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d97a9c289b7f681 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.858+0000 7f05d8ff9640 1 -- 172.21.15.16:0/1517009394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4061ac0 con 0x555bba656770 2024-04-25T10:13:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.858+0000 7f05d8ff9640 1 -- 172.21.15.16:0/1517009394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d40576b0 con 0x555bba656770 2024-04-25T10:13:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.858+0000 7f05d8ff9640 1 -- 172.21.15.16:0/1517009394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d405d0b0 con 0x555bba656770 2024-04-25T10:13:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.858+0000 7f05ebf9e740 1 -- 172.21.15.16:0/1517009394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba656770 msgr2=0x555bba659f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.858+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/1517009394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba656770 0x555bba659f80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f05d4089100 tx=0x7f05d4059bf0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.858+0000 7f05ebf9e740 1 -- 172.21.15.16:0/1517009394 shutdown_connections 2024-04-25T10:13:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.858+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/1517009394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba656770 0x555bba659f80 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.858+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/1517009394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba683ef0 0x555bba686860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.858+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/1517009394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba683b10 0x555bba686320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.858+0000 7f05ebf9e740 1 -- 172.21.15.16:0/1517009394 >> 172.21.15.16:0/1517009394 conn(0x555bba6589e0 msgr2=0x555bba6a1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.859+0000 7f05ebf9e740 1 -- 172.21.15.16:0/1517009394 shutdown_connections 2024-04-25T10:13:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.859+0000 7f05ebf9e740 1 -- 172.21.15.16:0/1517009394 wait complete. 2024-04-25T10:13:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.859+0000 7f05ebf9e740 1 -- start start 2024-04-25T10:13:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.859+0000 7f05ebf9e740 1 Processor -- start 2024-04-25T10:13:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.860+0000 7f05ebf9e740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba683b10 0x555bba67d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.860+0000 7f05ebf9e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 0x555bba6b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.860+0000 7f05ebf9e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba682190 0x555bba6825f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.860+0000 7f05ebf9e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bba65bb90 con 0x555bba683ef0 2024-04-25T10:13:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.860+0000 7f05ebf9e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bba55e4b0 con 0x555bba682190 2024-04-25T10:13:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.860+0000 7f05ebf9e740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bba65be70 con 0x555bba683b10 2024-04-25T10:13:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.860+0000 7f05ebf9e740 10 client.0 _ll_register_callbacks cb 0x555bba59e180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.860+0000 7f05e259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 0x555bba6b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.860+0000 7f05e259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 0x555bba6b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41114/0 (socket says 172.21.15.16:41114) 2024-04-25T10:13:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.860+0000 7f05e259c640 1 -- 172.21.15.16:0/843166878 learned_addr learned my addr 172.21.15.16:0/843166878 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.860+0000 7f05e2d9d640 1 --2- 172.21.15.16:0/843166878 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba683b10 0x555bba67d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.860+0000 7f05e359e640 1 --2- 172.21.15.16:0/843166878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba682190 0x555bba6825f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.860+0000 7f05e259c640 1 -- 172.21.15.16:0/843166878 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba683b10 msgr2=0x555bba67d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.860+0000 7f05e259c640 1 --2- 172.21.15.16:0/843166878 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba683b10 0x555bba67d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.860+0000 7f05e259c640 1 -- 172.21.15.16:0/843166878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba682190 msgr2=0x555bba6825f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.860+0000 7f05e259c640 1 --2- 172.21.15.16:0/843166878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba682190 0x555bba6825f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.860+0000 7f05e259c640 1 -- 172.21.15.16:0/843166878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bba39e7d0 con 0x555bba683ef0 2024-04-25T10:13:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.861+0000 7f05e259c640 1 --2- 172.21.15.16:0/843166878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 0x555bba6b1c70 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009890 tx=0x7f05cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31364a2a01a5a3eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.861+0000 7f05c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:13:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.861+0000 7f05c1ffb640 1 -- 172.21.15.16:0/843166878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc043070 con 0x555bba683ef0 2024-04-25T10:13:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.861+0000 7f05c1ffb640 1 -- 172.21.15.16:0/843166878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05cc03c440 con 0x555bba683ef0 2024-04-25T10:13:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.861+0000 7f05ebf9e740 1 -- 172.21.15.16:0/843166878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05d40576b0 con 0x555bba683ef0 2024-04-25T10:13:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.861+0000 7f05c1ffb640 1 -- 172.21.15.16:0/843166878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05cc04e050 con 0x555bba683ef0 2024-04-25T10:13:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.861+0000 7f05ebf9e740 10 client.8496 fetch_fsmap learned FSMap version 71 2024-04-25T10:13:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.861+0000 7f05ebf9e740 1 -- 172.21.15.16:0/843166878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x555bba2d43f0 con 0x555bba683ef0 2024-04-25T10:13:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.861+0000 7f05c1ffb640 1 -- 172.21.15.16:0/843166878 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d40576b0 con 0x555bba683ef0 2024-04-25T10:13:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.861+0000 7f05c1ffb640 1 -- 172.21.15.16:0/843166878 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f05cc051020 con 0x555bba683ef0 2024-04-25T10:13:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.861+0000 7f05ebf9e740 10 client.8496 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:13:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.861+0000 7f05ebf9e740 10 client.8496 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:13:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.861+0000 7f05ebf9e740 20 client.8496 populate_metadata read hostname 'smithi016' 2024-04-25T10:13:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.861+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/843166878 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x555bba6a2970 0x555bba6a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.861+0000 7f05ebf9e740 4 client.8496 mds_command: new command op to 8202 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.861+0000 7f05ebf9e740 1 -- 172.21.15.16:0/843166878 --> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bba5cd2e0 con 0x555bba6a2970 2024-04-25T10:13:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.861+0000 7f05e2d9d640 1 --2- 172.21.15.16:0/843166878 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x555bba6a2970 0x555bba6a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.862+0000 7f05e2d9d640 1 --2- 172.21.15.16:0/843166878 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x555bba6a2970 0x555bba6a2d50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c670b47088ed160f server_cookie=522852e16212b48e in_seq=0 out_seq=0 2024-04-25T10:13:03.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.862+0000 7f05c1ffb640 10 client.8496 ms_handle_connect on v2:172.21.15.186:6834/1371188714 2024-04-25T10:13:03.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.862+0000 7f05c1ffb640 1 -- 172.21.15.16:0/843166878 <== mds.0 v2:172.21.15.186:6834/1371188714 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555bba5cd2e0 con 0x555bba6a2970 2024-04-25T10:13:03.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.862+0000 7f05c1ffb640 10 client.8496 handle_command_reply: tid=0 2024-04-25T10:13:03.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.862+0000 7f05ebf9e740 1 client.8496 shutdown 2024-04-25T10:13:03.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.862+0000 7f05ebf9e740 1 -- 172.21.15.16:0/843166878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 msgr2=0x555bba6b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.862+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/843166878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 0x555bba6b1c70 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009890 tx=0x7f05cc004730 comp rx=0 tx=0).stop 2024-04-25T10:13:03.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.862+0000 7f05ebf9e740 1 -- 172.21.15.16:0/843166878 shutdown_connections 2024-04-25T10:13:03.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.862+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/843166878 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x555bba6a2970 0x555bba6a2d50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.863+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/843166878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba682190 0x555bba6825f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.863+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/843166878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 0x555bba6b1c70 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.863+0000 7f05c1ffb640 0 client.8496 ms_handle_reset on v2:172.21.15.186:6834/1371188714 2024-04-25T10:13:03.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.863+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/843166878 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba683b10 0x555bba67d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.863+0000 7f05ebf9e740 1 -- 172.21.15.16:0/843166878 >> 172.21.15.16:0/843166878 conn(0x555bba656770 msgr2=0x555bba67e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.863+0000 7f05ebf9e740 1 -- 172.21.15.16:0/843166878 shutdown_connections 2024-04-25T10:13:03.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.863+0000 7f05ebf9e740 1 -- 172.21.15.16:0/843166878 wait complete. 2024-04-25T10:13:03.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.863+0000 7f05ebf9e740 20 client.8496 trim_cache size 0 max 16384 2024-04-25T10:13:03.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.878+0000 7f05ebf9e740 1 Processor -- start 2024-04-25T10:13:03.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.878+0000 7f05ebf9e740 1 -- start start 2024-04-25T10:13:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05ebf9e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba682190 0x555bba656c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05ebf9e740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba683b10 0x555bba657190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05ebf9e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 0x555bba6b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05ebf9e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e41ffdc0 con 0x555bba683ef0 2024-04-25T10:13:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05ebf9e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e41f4720 con 0x555bba682190 2024-04-25T10:13:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05ebf9e740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e41f3b50 con 0x555bba683b10 2024-04-25T10:13:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05e359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 0x555bba6b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05e359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 0x555bba6b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41118/0 (socket says 172.21.15.16:41118) 2024-04-25T10:13:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05e359e640 1 -- 172.21.15.16:0/533766310 learned_addr learned my addr 172.21.15.16:0/533766310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05e259c640 1 --2- 172.21.15.16:0/533766310 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba683b10 0x555bba657190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05e2d9d640 1 --2- 172.21.15.16:0/533766310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba682190 0x555bba656c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05e359e640 1 -- 172.21.15.16:0/533766310 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba683b10 msgr2=0x555bba657190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05e359e640 1 --2- 172.21.15.16:0/533766310 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba683b10 0x555bba657190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05e359e640 1 -- 172.21.15.16:0/533766310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba682190 msgr2=0x555bba656c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05e359e640 1 --2- 172.21.15.16:0/533766310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba682190 0x555bba656c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05e359e640 1 -- 172.21.15.16:0/533766310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bba2940f0 con 0x555bba683ef0 2024-04-25T10:13:03.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05e2d9d640 1 --2- 172.21.15.16:0/533766310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba682190 0x555bba656c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:03.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05e259c640 1 --2- 172.21.15.16:0/533766310 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba683b10 0x555bba657190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:03.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05e359e640 1 --2- 172.21.15.16:0/533766310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 0x555bba6b1260 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f05d4059060 tx=0x7f05d405b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122b6b6af8852238 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05c3fff640 1 -- 172.21.15.16:0/533766310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4057450 con 0x555bba683ef0 2024-04-25T10:13:03.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05c3fff640 1 -- 172.21.15.16:0/533766310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d40576b0 con 0x555bba683ef0 2024-04-25T10:13:03.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.879+0000 7f05c3fff640 1 -- 172.21.15.16:0/533766310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d405d0b0 con 0x555bba683ef0 2024-04-25T10:13:03.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.880+0000 7f05ebf9e740 1 -- 172.21.15.16:0/533766310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 msgr2=0x555bba6b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.880+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/533766310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 0x555bba6b1260 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f05d4059060 tx=0x7f05d405b000 comp rx=0 tx=0).stop 2024-04-25T10:13:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.880+0000 7f05ebf9e740 1 -- 172.21.15.16:0/533766310 shutdown_connections 2024-04-25T10:13:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.880+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/533766310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 0x555bba6b1260 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.880+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/533766310 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba683b10 0x555bba657190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.880+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/533766310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba682190 0x555bba656c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.880+0000 7f05ebf9e740 1 -- 172.21.15.16:0/533766310 >> 172.21.15.16:0/533766310 conn(0x555bba656770 msgr2=0x555bba67d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.880+0000 7f05ebf9e740 1 -- 172.21.15.16:0/533766310 shutdown_connections 2024-04-25T10:13:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.880+0000 7f05ebf9e740 1 -- 172.21.15.16:0/533766310 wait complete. 2024-04-25T10:13:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.880+0000 7f05ebf9e740 1 -- start start 2024-04-25T10:13:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.881+0000 7f05ebf9e740 1 Processor -- start 2024-04-25T10:13:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.881+0000 7f05ebf9e740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba682190 0x555bba68a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.881+0000 7f05ebf9e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba683b10 0x555bba70bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.881+0000 7f05ebf9e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 0x555bba696cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.881+0000 7f05ebf9e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bba658630 con 0x555bba683ef0 2024-04-25T10:13:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.881+0000 7f05ebf9e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e41f4120 con 0x555bba683b10 2024-04-25T10:13:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.881+0000 7f05ebf9e740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bba5c22b0 con 0x555bba682190 2024-04-25T10:13:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.881+0000 7f05ebf9e740 10 client.0 _ll_register_callbacks cb 0x7f05d4059180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.881+0000 7f05e359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 0x555bba696cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.881+0000 7f05e359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 0x555bba696cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41132/0 (socket says 172.21.15.16:41132) 2024-04-25T10:13:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.881+0000 7f05e359e640 1 -- 172.21.15.16:0/1908680607 learned_addr learned my addr 172.21.15.16:0/1908680607 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.881+0000 7f05e2d9d640 1 --2- 172.21.15.16:0/1908680607 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba682190 0x555bba68a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.881+0000 7f05e259c640 1 --2- 172.21.15.16:0/1908680607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba683b10 0x555bba70bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05e359e640 1 -- 172.21.15.16:0/1908680607 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba682190 msgr2=0x555bba68a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05e359e640 1 --2- 172.21.15.16:0/1908680607 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba682190 0x555bba68a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05e359e640 1 -- 172.21.15.16:0/1908680607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba683b10 msgr2=0x555bba70bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05e359e640 1 --2- 172.21.15.16:0/1908680607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba683b10 0x555bba70bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05e359e640 1 -- 172.21.15.16:0/1908680607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bba6b07b0 con 0x555bba683ef0 2024-04-25T10:13:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05e359e640 1 --2- 172.21.15.16:0/1908680607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 0x555bba696cb0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f05d40590b0 tx=0x7f05d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a8b6dfb85e82c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:13:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05c1ffb640 1 -- 172.21.15.16:0/1908680607 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4080060 con 0x555bba683ef0 2024-04-25T10:13:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05c1ffb640 1 -- 172.21.15.16:0/1908680607 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05d4058a00 con 0x555bba683ef0 2024-04-25T10:13:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05ebf9e740 1 -- 172.21.15.16:0/1908680607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05d40576b0 con 0x555bba683ef0 2024-04-25T10:13:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05c1ffb640 1 -- 172.21.15.16:0/1908680607 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05d4055100 con 0x555bba683ef0 2024-04-25T10:13:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05ebf9e740 10 client.8502 fetch_fsmap learned FSMap version 71 2024-04-25T10:13:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05ebf9e740 1 -- 172.21.15.16:0/1908680607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x555bba689b90 con 0x555bba683ef0 2024-04-25T10:13:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05c1ffb640 1 -- 172.21.15.16:0/1908680607 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d40576b0 con 0x555bba683ef0 2024-04-25T10:13:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05c1ffb640 1 -- 172.21.15.16:0/1908680607 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f05d40552a0 con 0x555bba683ef0 2024-04-25T10:13:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05ebf9e740 10 client.8502 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:13:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05ebf9e740 10 client.8502 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:13:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05ebf9e740 20 client.8502 populate_metadata read hostname 'smithi016' 2024-04-25T10:13:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/1908680607 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x555bba6c3b50 0x555bba6971f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05ebf9e740 4 client.8502 mds_command: new command op to 8202 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:13:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.882+0000 7f05ebf9e740 1 -- 172.21.15.16:0/1908680607 --> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555bba5c7630 con 0x555bba6c3b50 2024-04-25T10:13:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.883+0000 7f05e2d9d640 1 --2- 172.21.15.16:0/1908680607 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x555bba6c3b50 0x555bba6971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.883+0000 7f05c1ffb640 10 client.8502 ms_handle_connect on v2:172.21.15.186:6834/1371188714 2024-04-25T10:13:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.883+0000 7f05e2d9d640 1 --2- 172.21.15.16:0/1908680607 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x555bba6c3b50 0x555bba6971f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ed52bbfff061f9 server_cookie=a93bf9a29c17d04 in_seq=0 out_seq=0 2024-04-25T10:13:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.883+0000 7f05c1ffb640 1 -- 172.21.15.16:0/1908680607 <== mds.0 v2:172.21.15.186:6834/1371188714 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x555bba5c7630 con 0x555bba6c3b50 2024-04-25T10:13:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.883+0000 7f05c1ffb640 10 client.8502 handle_command_reply: tid=0 2024-04-25T10:13:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.883+0000 7f05ebf9e740 1 client.8502 shutdown 2024-04-25T10:13:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.883+0000 7f05ebf9e740 1 -- 172.21.15.16:0/1908680607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 msgr2=0x555bba696cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.883+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/1908680607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 0x555bba696cb0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f05d40590b0 tx=0x7f05d4004a40 comp rx=0 tx=0).stop 2024-04-25T10:13:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.883+0000 7f05ebf9e740 1 -- 172.21.15.16:0/1908680607 shutdown_connections 2024-04-25T10:13:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.883+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/1908680607 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x555bba6c3b50 0x555bba6971f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.883+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/1908680607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bba683ef0 0x555bba696cb0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.883+0000 7f05c1ffb640 0 client.8502 ms_handle_reset on v2:172.21.15.186:6834/1371188714 2024-04-25T10:13:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.883+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/1908680607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555bba683b10 0x555bba70bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.883+0000 7f05ebf9e740 1 --2- 172.21.15.16:0/1908680607 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555bba682190 0x555bba68a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.883+0000 7f05ebf9e740 1 -- 172.21.15.16:0/1908680607 >> 172.21.15.16:0/1908680607 conn(0x555bba656770 msgr2=0x555bba655c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.883+0000 7f05ebf9e740 1 -- 172.21.15.16:0/1908680607 shutdown_connections 2024-04-25T10:13:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.883+0000 7f05ebf9e740 1 -- 172.21.15.16:0/1908680607 wait complete. 2024-04-25T10:13:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.884+0000 7f05ebf9e740 20 client.8502 trim_cache size 0 max 16384 2024-04-25T10:13:03.899 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:13:03.899 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:13:03.899 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-04-25T10:13:03.899 INFO:teuthology.orchestra.run.smithi016.stdout: "8fdaec76-0790-4692-99ba-909902a5d389": { 2024-04-25T10:13:03.899 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-04-25T10:13:03.899 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "8fdaec76-0790-4692-99ba-909902a5d389", 2024-04-25T10:13:03.899 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,repair" 2024-04-25T10:13:03.899 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:13:03.899 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:13:03.899 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:13:03.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.884+0000 7f05e91c1640 1 -- 172.21.15.16:0/3006737807 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f05c406b4d0 msgr2=0x7f05c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.884+0000 7f05e91c1640 1 --2- 172.21.15.16:0/3006737807 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f05c406b4d0 0x7f05c406d990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f05dc005880 tx=0x7f05dc009040 comp rx=0 tx=0).stop 2024-04-25T10:13:03.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.884+0000 7f05e91c1640 1 -- 172.21.15.16:0/3006737807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05e415dca0 msgr2=0x7f05e40cbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.884+0000 7f05e91c1640 1 --2- 172.21.15.16:0/3006737807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05e415dca0 0x7f05e40cbc60 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009f00 tx=0x7f05d400c490 comp rx=0 tx=0).stop 2024-04-25T10:13:03.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.884+0000 7f05e91c1640 1 -- 172.21.15.16:0/3006737807 shutdown_connections 2024-04-25T10:13:03.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.884+0000 7f05e91c1640 1 --2- 172.21.15.16:0/3006737807 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f05c406b4d0 0x7f05c406d990 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.884+0000 7f05e91c1640 1 --2- 172.21.15.16:0/3006737807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05e415dca0 0x7f05e40cbc60 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.884+0000 7f05e91c1640 1 --2- 172.21.15.16:0/3006737807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05e40d3b90 0x7f05e40cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.884+0000 7f05e91c1640 1 --2- 172.21.15.16:0/3006737807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05e40d0f60 0x7f05e40ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.884+0000 7f05e91c1640 1 -- 172.21.15.16:0/3006737807 >> 172.21.15.16:0/3006737807 conn(0x7f05e40726c0 msgr2=0x7f05e40d1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.884+0000 7f05e91c1640 1 -- 172.21.15.16:0/3006737807 shutdown_connections 2024-04-25T10:13:03.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:03.884+0000 7f05e91c1640 1 -- 172.21.15.16:0/3006737807 wait complete. 2024-04-25T10:13:03.901 INFO:tasks.cephfs.filesystem:scrub status for tag:8fdaec76-0790-4692-99ba-909902a5d389 - {'path': '/', 'tag': '8fdaec76-0790-4692-99ba-909902a5d389', 'options': 'recursive,repair'} 2024-04-25T10:13:04.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:04.656+0000 7f476f7fe640 1 -- 172.21.15.16:0/1053652108 <== mon.0 v2:172.21.15.16:3300/0 7 ==== log(5 entries from seq 1 at 2024-04-25T10:13:03.733553+0000) v1 ==== 846+0+0 (secure 0 0 0) 0x7f47680882d0 con 0x7f47780b5bb0 2024-04-25T10:13:04.662 INFO:teuthology.orchestra.run.smithi016.stdout: cluster: 2024-04-25T10:13:04.662 INFO:teuthology.orchestra.run.smithi016.stdout: id: 34bc5eca-3a21-42a6-a26f-07da0905e8bc 2024-04-25T10:13:04.662 INFO:teuthology.orchestra.run.smithi016.stdout: health: HEALTH_OK 2024-04-25T10:13:04.662 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:13:04.663 INFO:teuthology.orchestra.run.smithi016.stdout: services: 2024-04-25T10:13:04.663 INFO:teuthology.orchestra.run.smithi016.stdout: mon: 3 daemons, quorum a,b,c (age 8m) 2024-04-25T10:13:04.663 INFO:teuthology.orchestra.run.smithi016.stdout: mgr: x(active, since 8m), standbys: y 2024-04-25T10:13:04.663 INFO:teuthology.orchestra.run.smithi016.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T10:13:04.663 INFO:teuthology.orchestra.run.smithi016.stdout: osd: 8 osds: 8 up (since 8m), 8 in (since 8m) 2024-04-25T10:13:04.663 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:13:04.664 INFO:teuthology.orchestra.run.smithi016.stdout: data: 2024-04-25T10:13:04.664 INFO:teuthology.orchestra.run.smithi016.stdout: volumes: 1/1 healthy 2024-04-25T10:13:04.664 INFO:teuthology.orchestra.run.smithi016.stdout: pools: 3 pools, 97 pgs 2024-04-25T10:13:04.664 INFO:teuthology.orchestra.run.smithi016.stdout: objects: 26 objects, 6.6 MiB 2024-04-25T10:13:04.664 INFO:teuthology.orchestra.run.smithi016.stdout: usage: 278 MiB used, 720 GiB / 720 GiB avail 2024-04-25T10:13:04.665 INFO:teuthology.orchestra.run.smithi016.stdout: pgs: 97 active+clean 2024-04-25T10:13:04.665 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:13:04.665 INFO:teuthology.orchestra.run.smithi016.stdout: io: 2024-04-25T10:13:04.665 INFO:teuthology.orchestra.run.smithi016.stdout: client: 90 B/s rd, 546 KiB/s wr, 0 op/s rd, 4 op/s wr 2024-04-25T10:13:04.665 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:13:04.666 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:13:04.666 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:13:03.733553+0000 mds.b [INF] scrub queued for path: / 2024-04-25T10:13:04.666 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:13:03.733556+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:13:04.666 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:13:03.733560+0000 mds.b [INF] scrub summary: active paths [/] 2024-04-25T10:13:04.666 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:13:03.734827+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:13:04.666 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:13:04.592686+0000 mds.b [INF] scrub summary: idle 2024-04-25T10:13:08.615 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:08.620 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:33.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:33.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:13:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.015+0000 7f6875190640 1 Processor -- start 2024-04-25T10:13:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.015+0000 7f6875190640 1 -- start start 2024-04-25T10:13:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.016+0000 7f6875190640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f687015dca0 0x7f68700ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.016+0000 7f6875190640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68700d3b90 0x7f68700d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.016+0000 7f6875190640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68700d4090 con 0x7f68700cee80 2024-04-25T10:13:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.016+0000 7f6875190640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68700b6b20 con 0x7f687015dca0 2024-04-25T10:13:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.016+0000 7f6875190640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68700d4370 con 0x7f68700d3b90 2024-04-25T10:13:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.016+0000 7f686ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f687015dca0 0x7f68700ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.016+0000 7f686ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f687015dca0 0x7f68700ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32770/0 (socket says 172.21.15.16:32770) 2024-04-25T10:13:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.016+0000 7f686ed9d640 1 -- 172.21.15.16:0/182353125 learned_addr learned my addr 172.21.15.16:0/182353125 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.016+0000 7f686659c640 1 --2- 172.21.15.16:0/182353125 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68700d3b90 0x7f68700d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.016+0000 7f686ed9d640 1 -- 172.21.15.16:0/182353125 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68700d3b90 msgr2=0x7f68700d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.016+0000 7f686ed9d640 1 --2- 172.21.15.16:0/182353125 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68700d3b90 0x7f68700d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.016+0000 7f686ed9d640 1 -- 172.21.15.16:0/182353125 >> v1:172.21.15.186:6789/0 conn(0x7f68700cee80 legacy=0x7f68700b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.016+0000 7f686ed9d640 1 -- 172.21.15.16:0/182353125 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68700d4650 con 0x7f687015dca0 2024-04-25T10:13:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.017+0000 7f686ed9d640 1 --2- 172.21.15.16:0/182353125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f687015dca0 0x7f68700ce940 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f6860009870 tx=0x7f686002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581e90451caa63ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.017+0000 7f686e59c640 1 -- 172.21.15.16:0/182353125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860037070 con 0x7f687015dca0 2024-04-25T10:13:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.017+0000 7f686e59c640 1 -- 172.21.15.16:0/182353125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6860032440 con 0x7f687015dca0 2024-04-25T10:13:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.017+0000 7f686e59c640 1 -- 172.21.15.16:0/182353125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860032810 con 0x7f687015dca0 2024-04-25T10:13:34.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.018+0000 7f6875190640 1 -- 172.21.15.16:0/182353125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f687015dca0 msgr2=0x7f68700ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.018+0000 7f6875190640 1 --2- 172.21.15.16:0/182353125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f687015dca0 0x7f68700ce940 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f6860009870 tx=0x7f686002a210 comp rx=0 tx=0).stop 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.018+0000 7f6875190640 1 -- 172.21.15.16:0/182353125 shutdown_connections 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.018+0000 7f6875190640 1 --2- 172.21.15.16:0/182353125 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68700d3b90 0x7f68700d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.018+0000 7f6875190640 1 --2- 172.21.15.16:0/182353125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f687015dca0 0x7f68700ce940 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.018+0000 7f6875190640 1 -- 172.21.15.16:0/182353125 >> 172.21.15.16:0/182353125 conn(0x7f68700726c0 msgr2=0x7f6870072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.018+0000 7f6875190640 1 -- 172.21.15.16:0/182353125 shutdown_connections 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.018+0000 7f6875190640 1 -- 172.21.15.16:0/182353125 wait complete. 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.019+0000 7f6875190640 1 Processor -- start 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.019+0000 7f6875190640 1 -- start start 2024-04-25T10:13:34.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.020+0000 7f6875190640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68700cee80 0x7f68700cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.020+0000 7f6875190640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68700d3b90 0x7f68700cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.020+0000 7f6875190640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f687015dca0 0x7f6870158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.020+0000 7f6875190640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687015e200 con 0x7f687015dca0 2024-04-25T10:13:34.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.020+0000 7f6875190640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687015e080 con 0x7f68700cee80 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.020+0000 7f6875190640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68700d0f60 con 0x7f68700d3b90 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.020+0000 7f686f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f687015dca0 0x7f6870158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.020+0000 7f686f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f687015dca0 0x7f6870158b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32786/0 (socket says 172.21.15.16:32786) 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.020+0000 7f686f59e640 1 -- 172.21.15.16:0/2652709759 learned_addr learned my addr 172.21.15.16:0/2652709759 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.020+0000 7f686659c640 1 --2- 172.21.15.16:0/2652709759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68700d3b90 0x7f68700cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.021+0000 7f686ed9d640 1 --2- 172.21.15.16:0/2652709759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68700cee80 0x7f68700cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.021+0000 7f686f59e640 1 -- 172.21.15.16:0/2652709759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68700d3b90 msgr2=0x7f68700cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.021+0000 7f686f59e640 1 --2- 172.21.15.16:0/2652709759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68700d3b90 0x7f68700cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.021+0000 7f686f59e640 1 -- 172.21.15.16:0/2652709759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68700cee80 msgr2=0x7f68700cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.021+0000 7f686f59e640 1 --2- 172.21.15.16:0/2652709759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68700cee80 0x7f68700cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.021+0000 7f686f59e640 1 -- 172.21.15.16:0/2652709759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68700cb140 con 0x7f687015dca0 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.021+0000 7f686659c640 1 --2- 172.21.15.16:0/2652709759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68700d3b90 0x7f68700cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.021+0000 7f686ed9d640 1 --2- 172.21.15.16:0/2652709759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68700cee80 0x7f68700cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.021+0000 7f686f59e640 1 --2- 172.21.15.16:0/2652709759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f687015dca0 0x7f6870158b00 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f686800c7b0 tx=0x7f686800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.021+0000 7f6867fff640 1 -- 172.21.15.16:0/2652709759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868004b60 con 0x7f687015dca0 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.021+0000 7f6867fff640 1 -- 172.21.15.16:0/2652709759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6868004d00 con 0x7f687015dca0 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.021+0000 7f6875190640 1 -- 172.21.15.16:0/2652709759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68700cb420 con 0x7f687015dca0 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.021+0000 7f6867fff640 1 -- 172.21.15.16:0/2652709759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686800aa90 con 0x7f687015dca0 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.021+0000 7f6875190640 1 -- 172.21.15.16:0/2652709759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68700cb9e0 con 0x7f687015dca0 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.023+0000 7f6867fff640 1 -- 172.21.15.16:0/2652709759 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f686800ac30 con 0x7f687015dca0 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.023+0000 7f6867fff640 1 --2- 172.21.15.16:0/2652709759 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f685006b4d0 0x7f685006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.023+0000 7f6867fff640 1 -- 172.21.15.16:0/2652709759 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f686808a840 con 0x7f687015dca0 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.023+0000 7f686ed9d640 1 --2- 172.21.15.16:0/2652709759 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f685006b4d0 0x7f685006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.024+0000 7f686ed9d640 1 --2- 172.21.15.16:0/2652709759 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f685006b4d0 0x7f685006d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6860034000 tx=0x7f686002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.024+0000 7f6877f72740 1 Processor -- start 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.024+0000 7f6877f72740 1 -- start start 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.025+0000 7f6877f72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdb10 0x5632317d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.025+0000 7f6877f72740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5632317cdef0 0x5632317d0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.025+0000 7f6877f72740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317a0770 0x5632317a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.025+0000 7f6877f72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563231709da0 con 0x5632317cdb10 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.025+0000 7f6877f72740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632316b2b80 con 0x5632317a0770 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.025+0000 7f6877f72740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x563231705e40 con 0x5632317cdef0 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.025+0000 7f686ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdb10 0x5632317d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.025+0000 7f686ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdb10 0x5632317d0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32802/0 (socket says 172.21.15.16:32802) 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.025+0000 7f686ed9d640 1 -- 172.21.15.16:0/1030648108 learned_addr learned my addr 172.21.15.16:0/1030648108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.025+0000 7f686659c640 1 --2- 172.21.15.16:0/1030648108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5632317cdef0 0x5632317d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.025+0000 7f686f59e640 1 --2- 172.21.15.16:0/1030648108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317a0770 0x5632317a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.026+0000 7f686ed9d640 1 -- 172.21.15.16:0/1030648108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5632317cdef0 msgr2=0x5632317d0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.026+0000 7f686ed9d640 1 --2- 172.21.15.16:0/1030648108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5632317cdef0 0x5632317d0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.026+0000 7f686ed9d640 1 -- 172.21.15.16:0/1030648108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317a0770 msgr2=0x5632317a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.026+0000 7f686ed9d640 1 --2- 172.21.15.16:0/1030648108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317a0770 0x5632317a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.026+0000 7f686ed9d640 1 -- 172.21.15.16:0/1030648108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632316bcf10 con 0x5632317cdb10 2024-04-25T10:13:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.026+0000 7f686ed9d640 1 --2- 172.21.15.16:0/1030648108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdb10 0x5632317d0320 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f686002a800 tx=0x7f686002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66badbee50900e6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.026+0000 7f6864d99640 1 -- 172.21.15.16:0/1030648108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860037040 con 0x5632317cdb10 2024-04-25T10:13:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.026+0000 7f6864d99640 1 -- 172.21.15.16:0/1030648108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686003e660 con 0x5632317cdb10 2024-04-25T10:13:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.026+0000 7f6864d99640 1 -- 172.21.15.16:0/1030648108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686003ea10 con 0x5632317cdb10 2024-04-25T10:13:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.027+0000 7f6877f72740 1 -- 172.21.15.16:0/1030648108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdb10 msgr2=0x5632317d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.027+0000 7f6877f72740 1 --2- 172.21.15.16:0/1030648108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdb10 0x5632317d0320 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f686002a800 tx=0x7f686002c000 comp rx=0 tx=0).stop 2024-04-25T10:13:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.027+0000 7f6877f72740 1 -- 172.21.15.16:0/1030648108 shutdown_connections 2024-04-25T10:13:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.027+0000 7f6877f72740 1 --2- 172.21.15.16:0/1030648108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317a0770 0x5632317a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.027+0000 7f6877f72740 1 --2- 172.21.15.16:0/1030648108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5632317cdef0 0x5632317d0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.027+0000 7f6877f72740 1 --2- 172.21.15.16:0/1030648108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdb10 0x5632317d0320 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.027+0000 7f6877f72740 1 -- 172.21.15.16:0/1030648108 >> 172.21.15.16:0/1030648108 conn(0x5632317a29e0 msgr2=0x5632317eb320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.027+0000 7f6877f72740 1 -- 172.21.15.16:0/1030648108 shutdown_connections 2024-04-25T10:13:34.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.027+0000 7f6877f72740 1 -- 172.21.15.16:0/1030648108 wait complete. 2024-04-25T10:13:34.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.027+0000 7f6877f72740 1 -- start start 2024-04-25T10:13:34.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.027+0000 7f6877f72740 1 Processor -- start 2024-04-25T10:13:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.028+0000 7f6877f72740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317cdb10 0x5632317c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.028+0000 7f6877f72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdef0 0x5632317d4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.028+0000 7f6877f72740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5632317b56a0 0x5632317b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.028+0000 7f6877f72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632317a5b90 con 0x5632317cdef0 2024-04-25T10:13:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.028+0000 7f6877f72740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632316a84b0 con 0x5632317cdb10 2024-04-25T10:13:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.028+0000 7f6877f72740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632317a5e70 con 0x5632317b56a0 2024-04-25T10:13:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.028+0000 7f6877f72740 10 client.0 _ll_register_callbacks cb 0x5632316e8180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.028+0000 7f686659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdef0 0x5632317d4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.028+0000 7f686659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdef0 0x5632317d4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32804/0 (socket says 172.21.15.16:32804) 2024-04-25T10:13:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.028+0000 7f686659c640 1 -- 172.21.15.16:0/2012223943 learned_addr learned my addr 172.21.15.16:0/2012223943 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.028+0000 7f686ed9d640 1 --2- 172.21.15.16:0/2012223943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317cdb10 0x5632317c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.028+0000 7f686f59e640 1 --2- 172.21.15.16:0/2012223943 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5632317b56a0 0x5632317b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.029+0000 7f686659c640 1 -- 172.21.15.16:0/2012223943 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5632317b56a0 msgr2=0x5632317b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.029+0000 7f686659c640 1 --2- 172.21.15.16:0/2012223943 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5632317b56a0 0x5632317b5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.029+0000 7f686659c640 1 -- 172.21.15.16:0/2012223943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317cdb10 msgr2=0x5632317c3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.029+0000 7f686659c640 1 --2- 172.21.15.16:0/2012223943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317cdb10 0x5632317c3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.029+0000 7f686659c640 1 -- 172.21.15.16:0/2012223943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632314e87d0 con 0x5632317cdef0 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.029+0000 7f686659c640 1 --2- 172.21.15.16:0/2012223943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdef0 0x5632317d4170 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f68700d3fc0 tx=0x7f6858012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07683ac87a60726 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.029+0000 7f684dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.029+0000 7f684dffb640 1 -- 172.21.15.16:0/2012223943 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685801a070 con 0x5632317cdef0 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.029+0000 7f684dffb640 1 -- 172.21.15.16:0/2012223943 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6858004300 con 0x5632317cdef0 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.029+0000 7f6877f72740 1 -- 172.21.15.16:0/2012223943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f686003e660 con 0x5632317cdef0 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.029+0000 7f684dffb640 1 -- 172.21.15.16:0/2012223943 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6858025050 con 0x5632317cdef0 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.029+0000 7f6877f72740 10 client.8514 fetch_fsmap learned FSMap version 71 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.029+0000 7f6877f72740 1 -- 172.21.15.16:0/2012223943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x56323141e3f0 con 0x5632317cdef0 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.029+0000 7f684dffb640 1 -- 172.21.15.16:0/2012223943 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686003e660 con 0x5632317cdef0 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.030+0000 7f684dffb640 1 -- 172.21.15.16:0/2012223943 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6858028020 con 0x5632317cdef0 2024-04-25T10:13:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.030+0000 7f6877f72740 10 client.8514 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:13:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.030+0000 7f6877f72740 10 client.8514 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:13:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.030+0000 7f6877f72740 20 client.8514 populate_metadata read hostname 'smithi016' 2024-04-25T10:13:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.030+0000 7f6877f72740 1 --2- 172.21.15.16:0/2012223943 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x5632317ed970 0x5632317edd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.030+0000 7f6877f72740 4 client.8514 mds_command: new command op to 8202 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.030+0000 7f6877f72740 1 -- 172.21.15.16:0/2012223943 --> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632317172e0 con 0x5632317ed970 2024-04-25T10:13:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.030+0000 7f686ed9d640 1 --2- 172.21.15.16:0/2012223943 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x5632317ed970 0x5632317edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.030+0000 7f686ed9d640 1 --2- 172.21.15.16:0/2012223943 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x5632317ed970 0x5632317edd80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e018b36d6afb2eb server_cookie=2b68b52da0fb577a in_seq=0 out_seq=0 2024-04-25T10:13:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.030+0000 7f684dffb640 10 client.8514 ms_handle_connect on v2:172.21.15.186:6834/1371188714 2024-04-25T10:13:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.031+0000 7f684dffb640 1 -- 172.21.15.16:0/2012223943 <== mds.0 v2:172.21.15.186:6834/1371188714 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5632317172e0 con 0x5632317ed970 2024-04-25T10:13:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.031+0000 7f684dffb640 10 client.8514 handle_command_reply: tid=0 2024-04-25T10:13:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.031+0000 7f6877f72740 1 client.8514 shutdown 2024-04-25T10:13:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.031+0000 7f6877f72740 1 -- 172.21.15.16:0/2012223943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdef0 msgr2=0x5632317d4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.031+0000 7f6877f72740 1 --2- 172.21.15.16:0/2012223943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdef0 0x5632317d4170 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f68700d3fc0 tx=0x7f6858012760 comp rx=0 tx=0).stop 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.031+0000 7f6877f72740 1 -- 172.21.15.16:0/2012223943 shutdown_connections 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.031+0000 7f6877f72740 1 --2- 172.21.15.16:0/2012223943 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x5632317ed970 0x5632317edd80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.031+0000 7f6877f72740 1 --2- 172.21.15.16:0/2012223943 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5632317b56a0 0x5632317b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.031+0000 7f684dffb640 0 client.8514 ms_handle_reset on v2:172.21.15.186:6834/1371188714 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.031+0000 7f6877f72740 1 --2- 172.21.15.16:0/2012223943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdef0 0x5632317d4170 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.031+0000 7f6877f72740 1 --2- 172.21.15.16:0/2012223943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317cdb10 0x5632317c3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.031+0000 7f6877f72740 1 -- 172.21.15.16:0/2012223943 >> 172.21.15.16:0/2012223943 conn(0x5632317a0770 msgr2=0x56323179f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.031+0000 7f6877f72740 1 -- 172.21.15.16:0/2012223943 shutdown_connections 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.031+0000 7f6877f72740 1 -- 172.21.15.16:0/2012223943 wait complete. 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.031+0000 7f6877f72740 20 client.8514 trim_cache size 0 max 16384 2024-04-25T10:13:34.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.046+0000 7f6877f72740 1 Processor -- start 2024-04-25T10:13:34.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.046+0000 7f6877f72740 1 -- start start 2024-04-25T10:13:34.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.047+0000 7f6877f72740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5632317b56a0 0x5632317d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.047+0000 7f6877f72740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317cdb10 0x5632317d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.047+0000 7f6877f72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdef0 0x5632317d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.047+0000 7f6877f72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68701ffe30 con 0x5632317cdef0 2024-04-25T10:13:34.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.047+0000 7f6877f72740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68701587d0 con 0x5632317cdb10 2024-04-25T10:13:34.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.047+0000 7f6877f72740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6870157bd0 con 0x5632317b56a0 2024-04-25T10:13:34.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.047+0000 7f686f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdef0 0x5632317d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.047+0000 7f686f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdef0 0x5632317d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32806/0 (socket says 172.21.15.16:32806) 2024-04-25T10:13:34.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.047+0000 7f686f59e640 1 -- 172.21.15.16:0/50401131 learned_addr learned my addr 172.21.15.16:0/50401131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.047+0000 7f686f59e640 1 -- 172.21.15.16:0/50401131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5632317b56a0 msgr2=0x5632317d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.047+0000 7f686f59e640 1 --2- 172.21.15.16:0/50401131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5632317b56a0 0x5632317d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.048+0000 7f686f59e640 1 -- 172.21.15.16:0/50401131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317cdb10 msgr2=0x5632317d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.048+0000 7f686659c640 1 --2- 172.21.15.16:0/50401131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317cdb10 0x5632317d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.048+0000 7f686f59e640 1 --2- 172.21.15.16:0/50401131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317cdb10 0x5632317d02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.048+0000 7f686f59e640 1 -- 172.21.15.16:0/50401131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632313de0f0 con 0x5632317cdef0 2024-04-25T10:13:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.048+0000 7f686f59e640 1 --2- 172.21.15.16:0/50401131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdef0 0x5632317d0820 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f68680553a0 tx=0x7f6868055260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51749740b51b144b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.048+0000 7f684ffff640 1 -- 172.21.15.16:0/50401131 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868074030 con 0x5632317cdef0 2024-04-25T10:13:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.048+0000 7f684ffff640 1 -- 172.21.15.16:0/50401131 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6868056fe0 con 0x5632317cdef0 2024-04-25T10:13:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.048+0000 7f684ffff640 1 -- 172.21.15.16:0/50401131 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868022da0 con 0x5632317cdef0 2024-04-25T10:13:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.048+0000 7f6877f72740 1 -- 172.21.15.16:0/50401131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdef0 msgr2=0x5632317d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.048+0000 7f6877f72740 1 --2- 172.21.15.16:0/50401131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdef0 0x5632317d0820 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f68680553a0 tx=0x7f6868055260 comp rx=0 tx=0).stop 2024-04-25T10:13:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.048+0000 7f6877f72740 1 -- 172.21.15.16:0/50401131 shutdown_connections 2024-04-25T10:13:34.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.048+0000 7f6877f72740 1 --2- 172.21.15.16:0/50401131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdef0 0x5632317d0820 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.048+0000 7f6877f72740 1 --2- 172.21.15.16:0/50401131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317cdb10 0x5632317d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.048+0000 7f6877f72740 1 --2- 172.21.15.16:0/50401131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5632317b56a0 0x5632317d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.048+0000 7f6877f72740 1 -- 172.21.15.16:0/50401131 >> 172.21.15.16:0/50401131 conn(0x5632317a0770 msgr2=0x5632317e8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.048+0000 7f6877f72740 1 -- 172.21.15.16:0/50401131 shutdown_connections 2024-04-25T10:13:34.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.048+0000 7f6877f72740 1 -- 172.21.15.16:0/50401131 wait complete. 2024-04-25T10:13:34.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.049+0000 7f6877f72740 1 -- start start 2024-04-25T10:13:34.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.049+0000 7f6877f72740 1 Processor -- start 2024-04-25T10:13:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.050+0000 7f6877f72740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317b56a0 0x5632317b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.050+0000 7f6877f72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdb10 0x5632318556b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.050+0000 7f6877f72740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5632317cdef0 0x56323179f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.050+0000 7f6877f72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632317c4fd0 con 0x5632317cdb10 2024-04-25T10:13:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.050+0000 7f6877f72740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68701581d0 con 0x5632317b56a0 2024-04-25T10:13:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.050+0000 7f6877f72740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632317fe010 con 0x5632317cdef0 2024-04-25T10:13:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.050+0000 7f6877f72740 10 client.0 _ll_register_callbacks cb 0x7f6860035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.050+0000 7f686659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdb10 0x5632318556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.050+0000 7f686659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdb10 0x5632318556b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32814/0 (socket says 172.21.15.16:32814) 2024-04-25T10:13:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.050+0000 7f686659c640 1 -- 172.21.15.16:0/107851930 learned_addr learned my addr 172.21.15.16:0/107851930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.050+0000 7f686ed9d640 1 --2- 172.21.15.16:0/107851930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317b56a0 0x5632317b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.050+0000 7f686659c640 1 -- 172.21.15.16:0/107851930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5632317cdef0 msgr2=0x56323179f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.050+0000 7f686659c640 1 --2- 172.21.15.16:0/107851930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5632317cdef0 0x56323179f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.050+0000 7f686659c640 1 -- 172.21.15.16:0/107851930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317b56a0 msgr2=0x5632317b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.050+0000 7f686659c640 1 --2- 172.21.15.16:0/107851930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317b56a0 0x5632317b4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.050+0000 7f686659c640 1 -- 172.21.15.16:0/107851930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632317ea9a0 con 0x5632317cdb10 2024-04-25T10:13:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.050+0000 7f686659c640 1 --2- 172.21.15.16:0/107851930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdb10 0x5632318556b0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f6858003db0 tx=0x7f68580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9000b380fc11458b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.050+0000 7f684dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:13:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.051+0000 7f684dffb640 1 -- 172.21.15.16:0/107851930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685801a040 con 0x5632317cdb10 2024-04-25T10:13:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.051+0000 7f684dffb640 1 -- 172.21.15.16:0/107851930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6858004300 con 0x5632317cdb10 2024-04-25T10:13:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.051+0000 7f6877f72740 1 -- 172.21.15.16:0/107851930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6868056fe0 con 0x5632317cdb10 2024-04-25T10:13:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.051+0000 7f684dffb640 1 -- 172.21.15.16:0/107851930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6858039050 con 0x5632317cdb10 2024-04-25T10:13:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.051+0000 7f6877f72740 10 client.8520 fetch_fsmap learned FSMap version 71 2024-04-25T10:13:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.051+0000 7f6877f72740 1 -- 172.21.15.16:0/107851930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x563231804be0 con 0x5632317cdb10 2024-04-25T10:13:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.051+0000 7f684dffb640 1 -- 172.21.15.16:0/107851930 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6868056fe0 con 0x5632317cdb10 2024-04-25T10:13:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.051+0000 7f684dffb640 1 -- 172.21.15.16:0/107851930 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f685803c020 con 0x5632317cdb10 2024-04-25T10:13:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.051+0000 7f6877f72740 10 client.8520 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T10:13:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.051+0000 7f6877f72740 10 client.8520 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:13:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.051+0000 7f6877f72740 20 client.8520 populate_metadata read hostname 'smithi016' 2024-04-25T10:13:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.051+0000 7f6877f72740 1 --2- 172.21.15.16:0/107851930 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x56323179bd50 0x56323179fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.051+0000 7f6877f72740 4 client.8520 mds_command: new command op to 8202 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:13:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.051+0000 7f6877f72740 1 -- 172.21.15.16:0/107851930 --> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5632317d29a0 con 0x56323179bd50 2024-04-25T10:13:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.051+0000 7f686ed9d640 1 --2- 172.21.15.16:0/107851930 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x56323179bd50 0x56323179fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.052+0000 7f686ed9d640 1 --2- 172.21.15.16:0/107851930 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x56323179bd50 0x56323179fb40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=881a0d8fac7ab121 server_cookie=d7cbf53bbf760530 in_seq=0 out_seq=0 2024-04-25T10:13:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.052+0000 7f684dffb640 10 client.8520 ms_handle_connect on v2:172.21.15.186:6834/1371188714 2024-04-25T10:13:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.052+0000 7f684dffb640 1 -- 172.21.15.16:0/107851930 <== mds.0 v2:172.21.15.186:6834/1371188714 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5632317d29a0 con 0x56323179bd50 2024-04-25T10:13:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.052+0000 7f684dffb640 10 client.8520 handle_command_reply: tid=0 2024-04-25T10:13:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.052+0000 7f6877f72740 1 client.8520 shutdown 2024-04-25T10:13:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.052+0000 7f6877f72740 1 -- 172.21.15.16:0/107851930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdb10 msgr2=0x5632318556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.052+0000 7f6877f72740 1 --2- 172.21.15.16:0/107851930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdb10 0x5632318556b0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f6858003db0 tx=0x7f68580046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.052+0000 7f6877f72740 1 -- 172.21.15.16:0/107851930 shutdown_connections 2024-04-25T10:13:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.052+0000 7f6877f72740 1 --2- 172.21.15.16:0/107851930 >> [v2:172.21.15.186:6834/1371188714,v1:172.21.15.186:6836/1371188714] conn(0x56323179bd50 0x56323179fb40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.052+0000 7f6877f72740 1 --2- 172.21.15.16:0/107851930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5632317cdef0 0x56323179f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.052+0000 7f6877f72740 1 --2- 172.21.15.16:0/107851930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632317cdb10 0x5632318556b0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.052+0000 7f684dffb640 0 client.8520 ms_handle_reset on v2:172.21.15.186:6834/1371188714 2024-04-25T10:13:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.052+0000 7f6877f72740 1 --2- 172.21.15.16:0/107851930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632317b56a0 0x5632317b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.052+0000 7f6877f72740 1 -- 172.21.15.16:0/107851930 >> 172.21.15.16:0/107851930 conn(0x5632317a0770 msgr2=0x5632317a0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.052+0000 7f6877f72740 1 -- 172.21.15.16:0/107851930 shutdown_connections 2024-04-25T10:13:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.052+0000 7f6877f72740 1 -- 172.21.15.16:0/107851930 wait complete. 2024-04-25T10:13:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.052+0000 7f6877f72740 20 client.8520 trim_cache size 0 max 16384 2024-04-25T10:13:34.063 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:13:34.063 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-04-25T10:13:34.063 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-04-25T10:13:34.063 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:13:34.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.053+0000 7f6875190640 1 -- 172.21.15.16:0/2652709759 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f685006b4d0 msgr2=0x7f685006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.053+0000 7f6875190640 1 --2- 172.21.15.16:0/2652709759 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f685006b4d0 0x7f685006d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6860034000 tx=0x7f686002e000 comp rx=0 tx=0).stop 2024-04-25T10:13:34.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.053+0000 7f6875190640 1 -- 172.21.15.16:0/2652709759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f687015dca0 msgr2=0x7f6870158b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.053+0000 7f6875190640 1 --2- 172.21.15.16:0/2652709759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f687015dca0 0x7f6870158b00 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f686800c7b0 tx=0x7f686800cc70 comp rx=0 tx=0).stop 2024-04-25T10:13:34.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.053+0000 7f6875190640 1 -- 172.21.15.16:0/2652709759 shutdown_connections 2024-04-25T10:13:34.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.053+0000 7f6875190640 1 --2- 172.21.15.16:0/2652709759 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f685006b4d0 0x7f685006d990 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.053+0000 7f6875190640 1 --2- 172.21.15.16:0/2652709759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f687015dca0 0x7f6870158b00 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.053+0000 7f6875190640 1 --2- 172.21.15.16:0/2652709759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68700d3b90 0x7f68700cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.053+0000 7f6875190640 1 --2- 172.21.15.16:0/2652709759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68700cee80 0x7f68700cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.053+0000 7f6875190640 1 -- 172.21.15.16:0/2652709759 >> 172.21.15.16:0/2652709759 conn(0x7f68700726c0 msgr2=0x7f68700b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.053+0000 7f6875190640 1 -- 172.21.15.16:0/2652709759 shutdown_connections 2024-04-25T10:13:34.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.053+0000 7f6875190640 1 -- 172.21.15.16:0/2652709759 wait complete. 2024-04-25T10:13:34.068 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:13:34.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:13:34.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.269+0000 7fa3d401f640 1 Processor -- start 2024-04-25T10:13:34.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.269+0000 7fa3d401f640 1 -- start start 2024-04-25T10:13:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.270+0000 7fa3d401f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa3cc1610b0 0x7fa3cc0b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.270+0000 7fa3d401f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa3cc1618d0 0x7fa3cc0ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.270+0000 7fa3d401f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3cc162040 con 0x7fa3cc0ced70 2024-04-25T10:13:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.270+0000 7fa3d401f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3cc0cf1c0 con 0x7fa3cc1618d0 2024-04-25T10:13:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.270+0000 7fa3d401f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3cc0d1ea0 con 0x7fa3cc1610b0 2024-04-25T10:13:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.270+0000 7fa3d15ba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa3cc1618d0 0x7fa3cc0ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.270+0000 7fa3d25bc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa3cc0ced70 0x7fa3cc06c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41162/0 (socket says 172.21.15.16:41162) 2024-04-25T10:13:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.270+0000 7fa3d25bc640 1 -- 172.21.15.16:0/4289247525 learned_addr learned my addr 172.21.15.16:0/4289247525 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:13:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.270+0000 7fa3d1dbb640 1 --2- 172.21.15.16:0/4289247525 >> v2:172.21.15.186:3300/0 conn(0x7fa3cc1610b0 0x7fa3cc0b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.270+0000 7fa3d15ba640 1 -- 172.21.15.16:0/4289247525 >> v2:172.21.15.186:3300/0 conn(0x7fa3cc1610b0 msgr2=0x7fa3cc0b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.270+0000 7fa3d15ba640 1 --2- 172.21.15.16:0/4289247525 >> v2:172.21.15.186:3300/0 conn(0x7fa3cc1610b0 0x7fa3cc0b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.270+0000 7fa3d15ba640 1 -- 172.21.15.16:0/4289247525 >> v1:172.21.15.16:6789/0 conn(0x7fa3cc0ced70 legacy=0x7fa3cc06c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.270+0000 7fa3d15ba640 1 -- 172.21.15.16:0/4289247525 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc162320 con 0x7fa3cc1618d0 2024-04-25T10:13:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.270+0000 7fa3d0db9640 1 -- 172.21.15.16:0/4289247525 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482744868 0 0) 0x7fa3cc162040 con 0x7fa3cc0ced70 2024-04-25T10:13:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.271+0000 7fa3d15ba640 1 --2- 172.21.15.16:0/4289247525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3cc1618d0 0x7fa3cc0ce830 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc08c410 tx=0x7fa3bc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9119bf7a1e6f27a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.271+0000 7fa3d0db9640 1 -- 172.21.15.16:0/4289247525 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc037070 con 0x7fa3cc1618d0 2024-04-25T10:13:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.271+0000 7fa3d0db9640 1 -- 172.21.15.16:0/4289247525 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3bc032440 con 0x7fa3cc1618d0 2024-04-25T10:13:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.271+0000 7fa3d0db9640 1 -- 172.21.15.16:0/4289247525 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc02ae60 con 0x7fa3cc1618d0 2024-04-25T10:13:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.272+0000 7fa3d401f640 1 -- 172.21.15.16:0/4289247525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3cc1618d0 msgr2=0x7fa3cc0ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.272+0000 7fa3d401f640 1 --2- 172.21.15.16:0/4289247525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3cc1618d0 0x7fa3cc0ce830 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc08c410 tx=0x7fa3bc02a110 comp rx=0 tx=0).stop 2024-04-25T10:13:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.272+0000 7fa3d401f640 1 -- 172.21.15.16:0/4289247525 shutdown_connections 2024-04-25T10:13:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.272+0000 7fa3d401f640 1 --2- 172.21.15.16:0/4289247525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3cc1618d0 0x7fa3cc0ce830 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.272+0000 7fa3d401f640 1 --2- 172.21.15.16:0/4289247525 >> v2:172.21.15.186:3300/0 conn(0x7fa3cc1610b0 0x7fa3cc0b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.272+0000 7fa3d401f640 1 -- 172.21.15.16:0/4289247525 >> 172.21.15.16:0/4289247525 conn(0x7fa3cc0b5570 msgr2=0x7fa3cc0b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.272+0000 7fa3d401f640 1 -- 172.21.15.16:0/4289247525 shutdown_connections 2024-04-25T10:13:34.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.272+0000 7fa3d401f640 1 -- 172.21.15.16:0/4289247525 wait complete. 2024-04-25T10:13:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.273+0000 7fa3d401f640 1 Processor -- start 2024-04-25T10:13:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.273+0000 7fa3d401f640 1 -- start start 2024-04-25T10:13:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d401f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3cc0ced70 0x7fa3cc0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d401f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3cc1610b0 0x7fa3cc15b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d401f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3cc1618d0 0x7fa3cc157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d401f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc0b6c00 con 0x7fa3cc1610b0 2024-04-25T10:13:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d401f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc0d20f0 con 0x7fa3cc0ced70 2024-04-25T10:13:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d401f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc161d10 con 0x7fa3cc1618d0 2024-04-25T10:13:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d15ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3cc1610b0 0x7fa3cc15b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d15ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3cc1610b0 0x7fa3cc15b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32832/0 (socket says 172.21.15.16:32832) 2024-04-25T10:13:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d15ba640 1 -- 172.21.15.16:0/2221807999 learned_addr learned my addr 172.21.15.16:0/2221807999 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d25bc640 1 --2- 172.21.15.16:0/2221807999 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3cc1618d0 0x7fa3cc157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d1dbb640 1 --2- 172.21.15.16:0/2221807999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3cc0ced70 0x7fa3cc0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d15ba640 1 -- 172.21.15.16:0/2221807999 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3cc1618d0 msgr2=0x7fa3cc157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d15ba640 1 --2- 172.21.15.16:0/2221807999 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3cc1618d0 0x7fa3cc157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d15ba640 1 -- 172.21.15.16:0/2221807999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3cc0ced70 msgr2=0x7fa3cc0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d15ba640 1 --2- 172.21.15.16:0/2221807999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3cc0ced70 0x7fa3cc0cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d15ba640 1 -- 172.21.15.16:0/2221807999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc15bc30 con 0x7fa3cc1610b0 2024-04-25T10:13:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d15ba640 1 --2- 172.21.15.16:0/2221807999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3cc1610b0 0x7fa3cc15b120 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc000c00 tx=0x7fa3bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3c27fc640 1 -- 172.21.15.16:0/2221807999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc037040 con 0x7fa3cc1610b0 2024-04-25T10:13:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3c27fc640 1 -- 172.21.15.16:0/2221807999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3bc042070 con 0x7fa3cc1610b0 2024-04-25T10:13:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3c27fc640 1 -- 172.21.15.16:0/2221807999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc030bd0 con 0x7fa3cc1610b0 2024-04-25T10:13:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d401f640 1 -- 172.21.15.16:0/2221807999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3cc15bf10 con 0x7fa3cc1610b0 2024-04-25T10:13:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.274+0000 7fa3d401f640 1 -- 172.21.15.16:0/2221807999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3cc0cb020 con 0x7fa3cc1610b0 2024-04-25T10:13:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.275+0000 7fa3d401f640 1 -- 172.21.15.16:0/2221807999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3cc0750b0 con 0x7fa3cc1610b0 2024-04-25T10:13:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.276+0000 7fa3c27fc640 1 -- 172.21.15.16:0/2221807999 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3bc047460 con 0x7fa3cc1610b0 2024-04-25T10:13:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.276+0000 7fa3c27fc640 1 --2- 172.21.15.16:0/2221807999 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa3b006b4d0 0x7fa3b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.276+0000 7fa3c27fc640 1 -- 172.21.15.16:0/2221807999 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3bc0aade0 con 0x7fa3cc1610b0 2024-04-25T10:13:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.277+0000 7fa3d1dbb640 1 --2- 172.21.15.16:0/2221807999 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa3b006b4d0 0x7fa3b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.277+0000 7fa3d1dbb640 1 --2- 172.21.15.16:0/2221807999 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa3b006b4d0 0x7fa3b006d990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8006fd0 tx=0x7fa3c8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.281+0000 7fa3c27fc640 1 -- 172.21.15.16:0/2221807999 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa3bc032e20 con 0x7fa3cc1610b0 2024-04-25T10:13:34.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.498+0000 7fa3d401f640 1 -- 172.21.15.16:0/2221807999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa3cc0d1770 con 0x7fa3cc1610b0 2024-04-25T10:13:34.503 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-04-25T10:13:34.499+0000 7ff13f9bb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:13:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.500+0000 7fa3c27fc640 1 -- 172.21.15.16:0/2221807999 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa3bc079c60 con 0x7fa3cc1610b0 2024-04-25T10:13:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:13:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.503+0000 7fa3d401f640 1 -- 172.21.15.16:0/2221807999 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa3b006b4d0 msgr2=0x7fa3b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.503+0000 7fa3d401f640 1 --2- 172.21.15.16:0/2221807999 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa3b006b4d0 0x7fa3b006d990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8006fd0 tx=0x7fa3c8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.503+0000 7fa3d401f640 1 -- 172.21.15.16:0/2221807999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3cc1610b0 msgr2=0x7fa3cc15b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.503+0000 7fa3d401f640 1 --2- 172.21.15.16:0/2221807999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3cc1610b0 0x7fa3cc15b120 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc000c00 tx=0x7fa3bc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.503+0000 7fa3d401f640 1 -- 172.21.15.16:0/2221807999 shutdown_connections 2024-04-25T10:13:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.503+0000 7fa3d401f640 1 --2- 172.21.15.16:0/2221807999 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa3b006b4d0 0x7fa3b006d990 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.503+0000 7fa3d401f640 1 --2- 172.21.15.16:0/2221807999 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3cc1618d0 0x7fa3cc157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.503+0000 7fa3d401f640 1 --2- 172.21.15.16:0/2221807999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3cc1610b0 0x7fa3cc15b120 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.503+0000 7fa3d401f640 1 --2- 172.21.15.16:0/2221807999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3cc0ced70 0x7fa3cc0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.503+0000 7fa3d401f640 1 -- 172.21.15.16:0/2221807999 >> 172.21.15.16:0/2221807999 conn(0x7fa3cc0b5570 msgr2=0x7fa3cc0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.503+0000 7fa3d401f640 1 -- 172.21.15.16:0/2221807999 shutdown_connections 2024-04-25T10:13:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.503+0000 7fa3d401f640 1 -- 172.21.15.16:0/2221807999 wait complete. 2024-04-25T10:13:34.512 INFO:tasks.ceph.mds.b.smithi186.stderr: -6996> 2024-04-25T10:12:50.588+0000 7f583d4e9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:34.519 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-04-25T10:13:34.545 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2024-04-25T10:13:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.557+0000 7f6e6c52a280 1 -- start start 2024-04-25T10:13:34.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.557+0000 7f6e6c52a280 1 Processor -- start 2024-04-25T10:13:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.558+0000 7f6e6c52a280 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f76ff5ec00 0x55f76ffce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.558+0000 7f6e6c52a280 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f76ff5f800 0x55f76ffceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.558+0000 7f6e6c52a280 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f76ff5fc00 0x55f76ffcf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.558+0000 7f6e6c52a280 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f770d30600 con 0x55f76ff5fc00 2024-04-25T10:13:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.558+0000 7f6e6c52a280 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f770d30480 con 0x55f76ff5f800 2024-04-25T10:13:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.558+0000 7f6e6c52a280 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f770d30300 con 0x55f76ff5ec00 2024-04-25T10:13:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.558+0000 7f6e6aa6e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f76ff5fc00 0x55f76ffcf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.558+0000 7f6e6aa6e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f76ff5fc00 0x55f76ffcf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32846/0 (socket says 172.21.15.16:32846) 2024-04-25T10:13:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.558+0000 7f6e6aa6e640 1 -- 172.21.15.16:0/577410237 learned_addr learned my addr 172.21.15.16:0/577410237 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.558+0000 7f6e69a6c640 1 --2- 172.21.15.16:0/577410237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f76ff5f800 0x55f76ffceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.558+0000 7f6e6a26d640 1 --2- 172.21.15.16:0/577410237 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f76ff5ec00 0x55f76ffce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.558+0000 7f6e6aa6e640 1 -- 172.21.15.16:0/577410237 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f76ff5ec00 msgr2=0x55f76ffce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.558+0000 7f6e6aa6e640 1 --2- 172.21.15.16:0/577410237 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f76ff5ec00 0x55f76ffce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.558+0000 7f6e6aa6e640 1 -- 172.21.15.16:0/577410237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f76ff5f800 msgr2=0x55f76ffceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.558+0000 7f6e6aa6e640 1 --2- 172.21.15.16:0/577410237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f76ff5f800 0x55f76ffceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.558+0000 7f6e6aa6e640 1 -- 172.21.15.16:0/577410237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f76ff94780 con 0x55f76ff5fc00 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.558+0000 7f6e69a6c640 1 --2- 172.21.15.16:0/577410237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f76ff5f800 0x55f76ffceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.559+0000 7f6e6aa6e640 1 --2- 172.21.15.16:0/577410237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f76ff5fc00 0x55f76ffcf080 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x55f770d793b0 tx=0x55f770e14540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143d2d5427e16cb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.559+0000 7f6e67a68640 1 -- 172.21.15.16:0/577410237 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f76ff84680 con 0x55f76ff5fc00 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.559+0000 7f6e67a68640 1 -- 172.21.15.16:0/577410237 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f76ff8ae00 con 0x55f76ff5fc00 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.559+0000 7f6e67a68640 1 -- 172.21.15.16:0/577410237 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f76ff84b60 con 0x55f76ff5fc00 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.559+0000 7f6e6c52a280 1 -- 172.21.15.16:0/577410237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f76ff943c0 con 0x55f76ff5fc00 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.559+0000 7f6e6c52a280 1 -- 172.21.15.16:0/577410237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55f76ff95680 con 0x55f76ff5fc00 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.559+0000 7f6e6c52a280 4 waiting for MDS map... 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.559+0000 7f6e67a68640 1 -- 172.21.15.16:0/577410237 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x55f770e26600 con 0x55f76ff5fc00 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.559+0000 7f6e67a68640 1 -- 172.21.15.16:0/577410237 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 72) v1 ==== 2723+0+0 (secure 0 0 0) 0x55f76ffb9180 con 0x55f76ff5fc00 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.559+0000 7f6e6c52a280 4 Got MDS map 72 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.559+0000 7f6e6c52a280 10 main: JournalTool::main 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.559+0000 7f6e6c52a280 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.560+0000 7f6e6c52a280 1 Processor -- start 2024-04-25T10:13:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.560+0000 7f6e6c52a280 1 -- start start 2024-04-25T10:13:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.560+0000 7f6e6c52a280 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f770e29000 0x55f76ffcf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.560+0000 7f6e6c52a280 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f770e29400 0x55f76ffcfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.560+0000 7f6e6c52a280 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f770e29800 0x55f76ffd0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.560+0000 7f6e6c52a280 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f770d30180 con 0x55f770e29400 2024-04-25T10:13:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.560+0000 7f6e6c52a280 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76ffc0480 con 0x55f770e29000 2024-04-25T10:13:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.560+0000 7f6e6c52a280 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f76ffc0900 con 0x55f770e29800 2024-04-25T10:13:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.560+0000 7f6e6a26d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f770e29400 0x55f76ffcfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.560+0000 7f6e6a26d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f770e29400 0x55f76ffcfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32854/0 (socket says 172.21.15.16:32854) 2024-04-25T10:13:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.560+0000 7f6e6a26d640 1 -- 172.21.15.16:0/1408501995 learned_addr learned my addr 172.21.15.16:0/1408501995 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e69a6c640 1 --2- 172.21.15.16:0/1408501995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f770e29000 0x55f76ffcf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e69a6c640 1 --2- 172.21.15.16:0/1408501995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f770e29800 0x55f76ffd0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e6a26d640 1 -- 172.21.15.16:0/1408501995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f770e29800 msgr2=0x55f76ffd0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e6a26d640 1 --2- 172.21.15.16:0/1408501995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f770e29800 0x55f76ffd0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e6a26d640 1 -- 172.21.15.16:0/1408501995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f770e29000 msgr2=0x55f76ffcf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e6a26d640 1 --2- 172.21.15.16:0/1408501995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f770e29000 0x55f76ffcf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e6a26d640 1 -- 172.21.15.16:0/1408501995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f76ff954a0 con 0x55f770e29400 2024-04-25T10:13:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e6a26d640 1 --2- 172.21.15.16:0/1408501995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f770e29400 0x55f76ffcfb80 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x55f770d78a80 tx=0x55f770e142a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8deb64f5fab76e3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e65a64640 1 -- 172.21.15.16:0/1408501995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f76ff849c0 con 0x55f770e29400 2024-04-25T10:13:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e65a64640 1 -- 172.21.15.16:0/1408501995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f76ff8ac40 con 0x55f770e29400 2024-04-25T10:13:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e65a64640 1 -- 172.21.15.16:0/1408501995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f76ff84680 con 0x55f770e29400 2024-04-25T10:13:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e6c52a280 1 -- 172.21.15.16:0/1408501995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f770e29400 msgr2=0x55f76ffcfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e6c52a280 1 --2- 172.21.15.16:0/1408501995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f770e29400 0x55f76ffcfb80 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x55f770d78a80 tx=0x55f770e142a0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e6c52a280 1 -- 172.21.15.16:0/1408501995 shutdown_connections 2024-04-25T10:13:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e6c52a280 1 --2- 172.21.15.16:0/1408501995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f770e29800 0x55f76ffd0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e6c52a280 1 --2- 172.21.15.16:0/1408501995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f770e29400 0x55f76ffcfb80 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e6c52a280 1 --2- 172.21.15.16:0/1408501995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f770e29000 0x55f76ffcf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e6c52a280 1 -- 172.21.15.16:0/1408501995 >> 172.21.15.16:0/1408501995 conn(0x55f770e28c00 msgr2=0x55f770044800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e6c52a280 1 -- 172.21.15.16:0/1408501995 shutdown_connections 2024-04-25T10:13:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.561+0000 7f6e6c52a280 1 -- 172.21.15.16:0/1408501995 wait complete. 2024-04-25T10:13:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.562+0000 7f6e6c52a280 1 Processor -- start 2024-04-25T10:13:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.562+0000 7f6e6c52a280 1 -- start start 2024-04-25T10:13:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.562+0000 7f6e6c52a280 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f770e29000 0x55f76ffd1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.562+0000 7f6e6c52a280 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f770e29400 0x55f76ffd0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.562+0000 7f6e6c52a280 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f770e29800 0x55f76ffd0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.562+0000 7f6e6c52a280 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f770d31200 con 0x55f770e29000 2024-04-25T10:13:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e6c52a280 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f770d30180 con 0x55f770e29400 2024-04-25T10:13:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e6c52a280 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f770d31380 con 0x55f770e29800 2024-04-25T10:13:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e69a6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f770e29000 0x55f76ffd1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e69a6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f770e29000 0x55f76ffd1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32866/0 (socket says 172.21.15.16:32866) 2024-04-25T10:13:34.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e69a6c640 1 -- 172.21.15.16:0/500602998 learned_addr learned my addr 172.21.15.16:0/500602998 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e6a26d640 1 --2- 172.21.15.16:0/500602998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f770e29400 0x55f76ffd0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e69a6c640 1 --2- 172.21.15.16:0/500602998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f770e29800 0x55f76ffd0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e69a6c640 1 -- 172.21.15.16:0/500602998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f770e29800 msgr2=0x55f76ffd0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e69a6c640 1 --2- 172.21.15.16:0/500602998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f770e29800 0x55f76ffd0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e69a6c640 1 -- 172.21.15.16:0/500602998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f770e29400 msgr2=0x55f76ffd0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e69a6c640 1 --2- 172.21.15.16:0/500602998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f770e29400 0x55f76ffd0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e69a6c640 1 -- 172.21.15.16:0/500602998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f76ff94780 con 0x55f770e29000 2024-04-25T10:13:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e69a6c640 1 --2- 172.21.15.16:0/500602998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f770e29000 0x55f76ffd1180 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x55f770e39440 tx=0x55f770ea2060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e64261640 1 -- 172.21.15.16:0/500602998 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f76ff849c0 con 0x55f770e29000 2024-04-25T10:13:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e64261640 1 -- 172.21.15.16:0/500602998 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f76ff8aa80 con 0x55f770e29000 2024-04-25T10:13:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e6c52a280 1 -- 172.21.15.16:0/500602998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f76ff943c0 con 0x55f770e29000 2024-04-25T10:13:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e64261640 1 -- 172.21.15.16:0/500602998 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f76ff84680 con 0x55f770e29000 2024-04-25T10:13:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e6c52a280 1 -- 172.21.15.16:0/500602998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f76ff94000 con 0x55f770e29000 2024-04-25T10:13:34.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.563+0000 7f6e6c52a280 4 main: JournalTool: resolving pool 14 2024-04-25T10:13:34.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.565+0000 7f6e64261640 1 -- 172.21.15.16:0/500602998 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x55f770e92c00 con 0x55f770e29000 2024-04-25T10:13:34.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.565+0000 7f6e64261640 1 --2- 172.21.15.16:0/500602998 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55f770eaa800 0x55f76ffd1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.565+0000 7f6e64261640 1 -- 172.21.15.16:0/500602998 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x55f770ea8a00 con 0x55f770e29000 2024-04-25T10:13:34.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.565+0000 7f6e6c52a280 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:13:34.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.565+0000 7f6e6c52a280 4 main: Executing for rank 0 2024-04-25T10:13:34.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.565+0000 7f6e6c52a280 4 Filtering by inode '1099511627777' 2024-04-25T10:13:34.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.565+0000 7f6e6c52a280 1 --2- 172.21.15.16:0/500602998 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55f770eaac00 0x55f76ffd0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.565+0000 7f6e6a26d640 1 --2- 172.21.15.16:0/500602998 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55f770eaa800 0x55f76ffd1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.565+0000 7f6e6c52a280 1 -- 172.21.15.16:0/500602998 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) v9 -- 0x55f770eab000 con 0x55f770eaac00 2024-04-25T10:13:34.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.565+0000 7f6e69a6c640 1 --2- 172.21.15.16:0/500602998 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55f770eaac00 0x55f76ffd0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.565+0000 7f6e6a26d640 1 --2- 172.21.15.16:0/500602998 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55f770eaa800 0x55f76ffd1700 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x55f770d78a80 tx=0x55f770e141e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.566+0000 7f6e69a6c640 1 --2- 172.21.15.16:0/500602998 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55f770eaac00 0x55f76ffd0100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.566+0000 7f6e69a6c640 1 -- 172.21.15.16:0/500602998 <== osd.1 v2:172.21.15.16:6808/1916704159 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x55f77005a000 con 0x55f770eaac00 2024-04-25T10:13:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.566+0000 7f6e6c52a280 4 Pointer 400.00000000 is readable 2024-04-25T10:13:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.566+0000 7f6e6c52a280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-25T10:13:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.566+0000 7f6e6c52a280 1 -- 172.21.15.16:0/500602998 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) v9 -- 0x55f770eab400 con 0x55f770eaac00 2024-04-25T10:13:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.566+0000 7f6e69a6c640 1 -- 172.21.15.16:0/500602998 <== osd.1 v2:172.21.15.16:6808/1916704159 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x55f77005a000 con 0x55f770eaac00 2024-04-25T10:13:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.566+0000 7f6e6c52a280 10 Header 0x400000 0x407d21 0x40809d 2024-04-25T10:13:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.566+0000 7f6e6c52a280 10 Starting journal scan from offset 0x407d21 2024-04-25T10:13:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.566+0000 7f6e6c52a280 1 -- 172.21.15.16:0/500602998 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 32033~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) v9 -- 0x55f770eab800 con 0x55f770eaac00 2024-04-25T10:13:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.567+0000 7f6e69a6c640 1 -- 172.21.15.16:0/500602998 <== osd.1 v2:172.21.15.16:6808/1916704159 3 ==== osd_op_reply(3 200.00000001 [read 32033~892 out=892b] v0'0 uv28 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x55f77005a000 con 0x55f770eaac00 2024-04-25T10:13:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.567+0000 7f6e6c52a280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-25T10:13:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.567+0000 7f6e6c52a280 10 Parsing data, 0x37c bytes available 2024-04-25T10:13:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.567+0000 7f6e6c52a280 10 Attempting decode at 0x407d21 2024-04-25T10:13:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.567+0000 7f6e6c52a280 10 Consumed 0x37c bytes 2024-04-25T10:13:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.567+0000 7f6e6c52a280 10 Valid entry at 0x407d21 2024-04-25T10:13:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.567+0000 7f6e6c52a280 1 --2- 172.21.15.16:0/500602998 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55f770eabc00 0x55f76ffcfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.567+0000 7f6e6c52a280 1 -- 172.21.15.16:0/500602998 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) v9 -- 0x55f7702de000 con 0x55f770eabc00 2024-04-25T10:13:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.567+0000 7f6e6aa6e640 1 --2- 172.21.15.16:0/500602998 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55f770eabc00 0x55f76ffcfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.568+0000 7f6e6aa6e640 1 --2- 172.21.15.16:0/500602998 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55f770eabc00 0x55f76ffcfb80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6aa6e640 1 -- 172.21.15.16:0/500602998 <== osd.6 v2:172.21.15.186:6816/1123185664 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55f77005a500 con 0x55f770eabc00 2024-04-25T10:13:34.580 INFO:teuthology.orchestra.run.smithi016.stdout:Events by type: 2024-04-25T10:13:34.580 INFO:teuthology.orchestra.run.smithi016.stdout:Errors: 0 2024-04-25T10:13:34.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 4 Reached end of journal objects 2024-04-25T10:13:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 4 Scanned objects, 0 missing, 1 valid 2024-04-25T10:13:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 4 Events scanned, 0 gaps 2024-04-25T10:13:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 4 Found 1 valid events 2024-04-25T10:13:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 4 Selected 0 events events for processing 2024-04-25T10:13:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 4 0 events 2024-04-25T10:13:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 -- 172.21.15.16:0/500602998 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55f770eaac00 msgr2=0x55f76ffd0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 --2- 172.21.15.16:0/500602998 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55f770eaac00 0x55f76ffd0100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 -- 172.21.15.16:0/500602998 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55f770eabc00 msgr2=0x55f76ffcfb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 --2- 172.21.15.16:0/500602998 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55f770eabc00 0x55f76ffcfb80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 -- 172.21.15.16:0/500602998 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55f770eaa800 msgr2=0x55f76ffd1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 --2- 172.21.15.16:0/500602998 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55f770eaa800 0x55f76ffd1700 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x55f770d78a80 tx=0x55f770e141e0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 -- 172.21.15.16:0/500602998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f770e29000 msgr2=0x55f76ffd1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 --2- 172.21.15.16:0/500602998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f770e29000 0x55f76ffd1180 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x55f770e39440 tx=0x55f770ea2060 comp rx=0 tx=0).stop 2024-04-25T10:13:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6a26d640 1 -- 172.21.15.16:0/500602998 reap_dead start 2024-04-25T10:13:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6a26d640 1 -- 172.21.15.16:0/500602998 reap_dead start 2024-04-25T10:13:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 -- 172.21.15.16:0/500602998 shutdown_connections 2024-04-25T10:13:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 -- 172.21.15.16:0/500602998 >> 172.21.15.16:0/500602998 conn(0x55f770e28c00 msgr2=0x55f770044800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 -- 172.21.15.16:0/500602998 shutdown_connections 2024-04-25T10:13:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 -- 172.21.15.16:0/500602998 wait complete. 2024-04-25T10:13:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 -- 172.21.15.16:0/577410237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f76ff5fc00 msgr2=0x55f76ffcf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 --2- 172.21.15.16:0/577410237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f76ff5fc00 0x55f76ffcf080 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x55f770d793b0 tx=0x55f770e14540 comp rx=0 tx=0).stop 2024-04-25T10:13:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 -- 172.21.15.16:0/577410237 shutdown_connections 2024-04-25T10:13:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 --2- 172.21.15.16:0/577410237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f76ff5fc00 0x55f76ffcf080 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 --2- 172.21.15.16:0/577410237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f76ff5f800 0x55f76ffceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 --2- 172.21.15.16:0/577410237 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f76ff5ec00 0x55f76ffce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 -- 172.21.15.16:0/577410237 >> 172.21.15.16:0/577410237 conn(0x55f76ff5f000 msgr2=0x55f770045000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 -- 172.21.15.16:0/577410237 shutdown_connections 2024-04-25T10:13:34.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.570+0000 7f6e6c52a280 1 -- 172.21.15.16:0/577410237 wait complete. 2024-04-25T10:13:34.585 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.066015, result: Events by type: Errors: 0 2024-04-25T10:13:34.585 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-04-25T10:13:34.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.663+0000 7f7c94a8d280 1 -- start start 2024-04-25T10:13:34.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.663+0000 7f7c94a8d280 1 Processor -- start 2024-04-25T10:13:34.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c94a8d280 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfdc7e000 0x557bfcf0f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c94a8d280 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfdc7e400 0x557bfcf0eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c94a8d280 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557bfce9f800 0x557bfcf0fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c94a8d280 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bfdc88480 con 0x557bfdc7e000 2024-04-25T10:13:34.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c94a8d280 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bfdc88600 con 0x557bfdc7e400 2024-04-25T10:13:34.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c94a8d280 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bfdc88780 con 0x557bfce9f800 2024-04-25T10:13:34.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c93239640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfdc7e000 0x557bfcf0f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c93239640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfdc7e000 0x557bfcf0f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32882/0 (socket says 172.21.15.16:32882) 2024-04-25T10:13:34.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c93239640 1 -- 172.21.15.16:0/4185816081 learned_addr learned my addr 172.21.15.16:0/4185816081 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:34.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c92a38640 1 --2- 172.21.15.16:0/4185816081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfdc7e400 0x557bfcf0eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c93a3a640 1 --2- 172.21.15.16:0/4185816081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557bfce9f800 0x557bfcf0fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c93239640 1 -- 172.21.15.16:0/4185816081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557bfce9f800 msgr2=0x557bfcf0fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c93239640 1 --2- 172.21.15.16:0/4185816081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557bfce9f800 0x557bfcf0fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c93239640 1 -- 172.21.15.16:0/4185816081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfdc7e400 msgr2=0x557bfcf0eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c93239640 1 --2- 172.21.15.16:0/4185816081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfdc7e400 0x557bfcf0eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c93239640 1 -- 172.21.15.16:0/4185816081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bfced4780 con 0x557bfdc7e000 2024-04-25T10:13:34.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c93239640 1 --2- 172.21.15.16:0/4185816081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfdc7e000 0x557bfcf0f080 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x557bfdcc2d80 tx=0x557bfdc31740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e78a197c9bbd99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c90a34640 1 -- 172.21.15.16:0/4185816081 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557bfcec4d00 con 0x557bfdc7e000 2024-04-25T10:13:34.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c90a34640 1 -- 172.21.15.16:0/4185816081 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557bfceca700 con 0x557bfdc7e000 2024-04-25T10:13:34.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c90a34640 1 -- 172.21.15.16:0/4185816081 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557bfcec4680 con 0x557bfdc7e000 2024-04-25T10:13:34.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c94a8d280 1 -- 172.21.15.16:0/4185816081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bfced54a0 con 0x557bfdc7e000 2024-04-25T10:13:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c94a8d280 1 -- 172.21.15.16:0/4185816081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x557bfced4960 con 0x557bfdc7e000 2024-04-25T10:13:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.664+0000 7f7c94a8d280 4 waiting for MDS map... 2024-04-25T10:13:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.665+0000 7f7c90a34640 1 -- 172.21.15.16:0/4185816081 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x557bfdd70400 con 0x557bfdc7e000 2024-04-25T10:13:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.665+0000 7f7c90a34640 1 -- 172.21.15.16:0/4185816081 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 72) v1 ==== 2723+0+0 (secure 0 0 0) 0x557bfcef9180 con 0x557bfdc7e000 2024-04-25T10:13:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.665+0000 7f7c94a8d280 4 Got MDS map 72 2024-04-25T10:13:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.665+0000 7f7c94a8d280 10 main: JournalTool::main 2024-04-25T10:13:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.665+0000 7f7c94a8d280 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:13:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.665+0000 7f7c94a8d280 1 Processor -- start 2024-04-25T10:13:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.665+0000 7f7c94a8d280 1 -- start start 2024-04-25T10:13:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.666+0000 7f7c94a8d280 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557bfce9fc00 0x557bfcf10100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.666+0000 7f7c94a8d280 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfdc7e800 0x557bfcf10680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.666+0000 7f7c94a8d280 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfddca000 0x557bfcf10c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.666+0000 7f7c94a8d280 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bfcf00180 con 0x557bfddca000 2024-04-25T10:13:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.666+0000 7f7c94a8d280 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bfdc88f00 con 0x557bfdc7e800 2024-04-25T10:13:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.666+0000 7f7c94a8d280 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bfdc89080 con 0x557bfce9fc00 2024-04-25T10:13:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.666+0000 7f7c92a38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfddca000 0x557bfcf10c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.666+0000 7f7c92a38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfddca000 0x557bfcf10c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32890/0 (socket says 172.21.15.16:32890) 2024-04-25T10:13:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.666+0000 7f7c92a38640 1 -- 172.21.15.16:0/1414620063 learned_addr learned my addr 172.21.15.16:0/1414620063 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.666+0000 7f7c93239640 1 --2- 172.21.15.16:0/1414620063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfdc7e800 0x557bfcf10680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.666+0000 7f7c93a3a640 1 --2- 172.21.15.16:0/1414620063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557bfce9fc00 0x557bfcf10100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.666+0000 7f7c92a38640 1 -- 172.21.15.16:0/1414620063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557bfce9fc00 msgr2=0x557bfcf10100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.666+0000 7f7c92a38640 1 --2- 172.21.15.16:0/1414620063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557bfce9fc00 0x557bfcf10100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.666+0000 7f7c92a38640 1 -- 172.21.15.16:0/1414620063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfdc7e800 msgr2=0x557bfcf10680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.666+0000 7f7c92a38640 1 --2- 172.21.15.16:0/1414620063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfdc7e800 0x557bfcf10680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.666+0000 7f7c92a38640 1 -- 172.21.15.16:0/1414620063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bfced43c0 con 0x557bfddca000 2024-04-25T10:13:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.666+0000 7f7c92a38640 1 --2- 172.21.15.16:0/1414620063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfddca000 0x557bfcf10c00 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x557bfdd78660 tx=0x557bfddcc000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311d01660a1a4f29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.667+0000 7f7c8ea30640 1 -- 172.21.15.16:0/1414620063 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557bfcec4d00 con 0x557bfddca000 2024-04-25T10:13:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.667+0000 7f7c8ea30640 1 -- 172.21.15.16:0/1414620063 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557bfcecafc0 con 0x557bfddca000 2024-04-25T10:13:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.667+0000 7f7c8ea30640 1 -- 172.21.15.16:0/1414620063 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557bfcec4680 con 0x557bfddca000 2024-04-25T10:13:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.667+0000 7f7c94a8d280 1 -- 172.21.15.16:0/1414620063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfddca000 msgr2=0x557bfcf10c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.667+0000 7f7c94a8d280 1 --2- 172.21.15.16:0/1414620063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfddca000 0x557bfcf10c00 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x557bfdd78660 tx=0x557bfddcc000 comp rx=0 tx=0).stop 2024-04-25T10:13:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.667+0000 7f7c94a8d280 1 -- 172.21.15.16:0/1414620063 shutdown_connections 2024-04-25T10:13:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.667+0000 7f7c94a8d280 1 --2- 172.21.15.16:0/1414620063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfddca000 0x557bfcf10c00 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.667+0000 7f7c94a8d280 1 --2- 172.21.15.16:0/1414620063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfdc7e800 0x557bfcf10680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.667+0000 7f7c94a8d280 1 --2- 172.21.15.16:0/1414620063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557bfce9fc00 0x557bfcf10100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.667+0000 7f7c94a8d280 1 -- 172.21.15.16:0/1414620063 >> 172.21.15.16:0/1414620063 conn(0x557bfce9ec00 msgr2=0x557bfcf87800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.667+0000 7f7c94a8d280 1 -- 172.21.15.16:0/1414620063 shutdown_connections 2024-04-25T10:13:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.667+0000 7f7c94a8d280 1 -- 172.21.15.16:0/1414620063 wait complete. 2024-04-25T10:13:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.668+0000 7f7c94a8d280 1 Processor -- start 2024-04-25T10:13:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.668+0000 7f7c94a8d280 1 -- start start 2024-04-25T10:13:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.668+0000 7f7c94a8d280 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557bfce9fc00 0x557bfcf10100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.668+0000 7f7c94a8d280 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfdc7e800 0x557bfcf10680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.668+0000 7f7c94a8d280 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfddcb400 0x557bfcf10c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.668+0000 7f7c94a8d280 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bfdc89380 con 0x557bfddcb400 2024-04-25T10:13:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.668+0000 7f7c94a8d280 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bfdc89200 con 0x557bfdc7e800 2024-04-25T10:13:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.668+0000 7f7c94a8d280 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bfdc89500 con 0x557bfce9fc00 2024-04-25T10:13:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.668+0000 7f7c92a38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfddcb400 0x557bfcf10c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.668+0000 7f7c92a38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfddcb400 0x557bfcf10c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32896/0 (socket says 172.21.15.16:32896) 2024-04-25T10:13:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.668+0000 7f7c92a38640 1 -- 172.21.15.16:0/2094789902 learned_addr learned my addr 172.21.15.16:0/2094789902 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.668+0000 7f7c93239640 1 --2- 172.21.15.16:0/2094789902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfdc7e800 0x557bfcf10680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.669+0000 7f7c92a38640 1 -- 172.21.15.16:0/2094789902 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557bfce9fc00 msgr2=0x557bfcf10100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:13:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.669+0000 7f7c92a38640 1 --2- 172.21.15.16:0/2094789902 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557bfce9fc00 0x557bfcf10100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.669+0000 7f7c92a38640 1 -- 172.21.15.16:0/2094789902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfdc7e800 msgr2=0x557bfcf10680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.669+0000 7f7c92a38640 1 --2- 172.21.15.16:0/2094789902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfdc7e800 0x557bfcf10680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.669+0000 7f7c92a38640 1 -- 172.21.15.16:0/2094789902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bfced4780 con 0x557bfddcb400 2024-04-25T10:13:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.669+0000 7f7c92a38640 1 --2- 172.21.15.16:0/2094789902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfddcb400 0x557bfcf10c00 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x557bfdd79320 tx=0x557bfdc315c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.669+0000 7f7c8d22d640 1 -- 172.21.15.16:0/2094789902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557bfcec4820 con 0x557bfddcb400 2024-04-25T10:13:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.669+0000 7f7c8d22d640 1 -- 172.21.15.16:0/2094789902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557bfcecae00 con 0x557bfddcb400 2024-04-25T10:13:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.669+0000 7f7c94a8d280 1 -- 172.21.15.16:0/2094789902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557bfced4000 con 0x557bfddcb400 2024-04-25T10:13:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.669+0000 7f7c8d22d640 1 -- 172.21.15.16:0/2094789902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557bfcec4d00 con 0x557bfddcb400 2024-04-25T10:13:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.669+0000 7f7c94a8d280 1 -- 172.21.15.16:0/2094789902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bfced5680 con 0x557bfddcb400 2024-04-25T10:13:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.669+0000 7f7c94a8d280 4 main: JournalTool: resolving pool 14 2024-04-25T10:13:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.670+0000 7f7c8d22d640 1 -- 172.21.15.16:0/2094789902 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x557bfdddac00 con 0x557bfddcb400 2024-04-25T10:13:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.671+0000 7f7c8d22d640 1 --2- 172.21.15.16:0/2094789902 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x557bfdddec00 0x557bfd208000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.671+0000 7f7c8d22d640 1 -- 172.21.15.16:0/2094789902 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x557bfddcea00 con 0x557bfddcb400 2024-04-25T10:13:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.671+0000 7f7c94a8d280 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:13:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.671+0000 7f7c94a8d280 4 main: Executing for rank 0 2024-04-25T10:13:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.671+0000 7f7c94a8d280 4 Filtering by inode '1099511627778' 2024-04-25T10:13:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.671+0000 7f7c94a8d280 1 --2- 172.21.15.16:0/2094789902 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x557bfddcb000 0x557bfcf11180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.671+0000 7f7c94a8d280 1 -- 172.21.15.16:0/2094789902 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) v9 -- 0x557bfddcac00 con 0x557bfddcb000 2024-04-25T10:13:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.671+0000 7f7c93a3a640 1 --2- 172.21.15.16:0/2094789902 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x557bfdddec00 0x557bfd208000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.671+0000 7f7c93239640 1 --2- 172.21.15.16:0/2094789902 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x557bfddcb000 0x557bfcf11180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.671+0000 7f7c93239640 1 --2- 172.21.15.16:0/2094789902 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x557bfddcb000 0x557bfcf11180 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.671+0000 7f7c93a3a640 1 --2- 172.21.15.16:0/2094789902 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x557bfdddec00 0x557bfd208000 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x557bfdc1e030 tx=0x557bfddcc2a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.671+0000 7f7c93239640 1 -- 172.21.15.16:0/2094789902 <== osd.1 v2:172.21.15.16:6808/1916704159 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x557bfcf9a000 con 0x557bfddcb000 2024-04-25T10:13:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.671+0000 7f7c94a8d280 4 Pointer 400.00000000 is readable 2024-04-25T10:13:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.671+0000 7f7c94a8d280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-25T10:13:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.671+0000 7f7c94a8d280 1 -- 172.21.15.16:0/2094789902 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) v9 -- 0x557bfddca800 con 0x557bfddcb000 2024-04-25T10:13:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.672+0000 7f7c93239640 1 -- 172.21.15.16:0/2094789902 <== osd.1 v2:172.21.15.16:6808/1916704159 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x557bfcf9a000 con 0x557bfddcb000 2024-04-25T10:13:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.672+0000 7f7c94a8d280 10 Header 0x400000 0x407d21 0x40809d 2024-04-25T10:13:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.672+0000 7f7c94a8d280 10 Starting journal scan from offset 0x407d21 2024-04-25T10:13:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.672+0000 7f7c94a8d280 1 -- 172.21.15.16:0/2094789902 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 32033~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) v9 -- 0x557bfddca400 con 0x557bfddcb000 2024-04-25T10:13:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.672+0000 7f7c93239640 1 -- 172.21.15.16:0/2094789902 <== osd.1 v2:172.21.15.16:6808/1916704159 3 ==== osd_op_reply(3 200.00000001 [read 32033~892 out=892b] v0'0 uv28 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x557bfcf9a000 con 0x557bfddcb000 2024-04-25T10:13:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.672+0000 7f7c94a8d280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-25T10:13:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.672+0000 7f7c94a8d280 10 Parsing data, 0x37c bytes available 2024-04-25T10:13:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.672+0000 7f7c94a8d280 10 Attempting decode at 0x407d21 2024-04-25T10:13:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.672+0000 7f7c94a8d280 10 Consumed 0x37c bytes 2024-04-25T10:13:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.672+0000 7f7c94a8d280 10 Valid entry at 0x407d21 2024-04-25T10:13:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.672+0000 7f7c94a8d280 1 --2- 172.21.15.16:0/2094789902 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x557bfddca000 0x557bfcf11700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.672+0000 7f7c94a8d280 1 -- 172.21.15.16:0/2094789902 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) v9 -- 0x557bfdddfc00 con 0x557bfddca000 2024-04-25T10:13:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.673+0000 7f7c93a3a640 1 --2- 172.21.15.16:0/2094789902 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x557bfddca000 0x557bfcf11700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.675+0000 7f7c93a3a640 1 --2- 172.21.15.16:0/2094789902 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x557bfddca000 0x557bfcf11700 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c93a3a640 1 -- 172.21.15.16:0/2094789902 <== osd.6 v2:172.21.15.186:6816/1123185664 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x557bfcf9a500 con 0x557bfddca000 2024-04-25T10:13:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 4 Reached end of journal objects 2024-04-25T10:13:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 4 Scanned objects, 0 missing, 1 valid 2024-04-25T10:13:34.686 INFO:teuthology.orchestra.run.smithi016.stdout:Events by type: 2024-04-25T10:13:34.686 INFO:teuthology.orchestra.run.smithi016.stdout:Errors: 0 2024-04-25T10:13:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 4 Events scanned, 0 gaps 2024-04-25T10:13:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 4 Found 1 valid events 2024-04-25T10:13:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 4 Selected 0 events events for processing 2024-04-25T10:13:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 4 0 events 2024-04-25T10:13:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 -- 172.21.15.16:0/2094789902 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x557bfddcb000 msgr2=0x557bfcf11180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 --2- 172.21.15.16:0/2094789902 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x557bfddcb000 0x557bfcf11180 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 -- 172.21.15.16:0/2094789902 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x557bfddca000 msgr2=0x557bfcf11700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 --2- 172.21.15.16:0/2094789902 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x557bfddca000 0x557bfcf11700 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 -- 172.21.15.16:0/2094789902 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x557bfdddec00 msgr2=0x557bfd208000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 --2- 172.21.15.16:0/2094789902 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x557bfdddec00 0x557bfd208000 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x557bfdc1e030 tx=0x557bfddcc2a0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 -- 172.21.15.16:0/2094789902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfddcb400 msgr2=0x557bfcf10c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c92a38640 1 -- 172.21.15.16:0/2094789902 reap_dead start 2024-04-25T10:13:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 --2- 172.21.15.16:0/2094789902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfddcb400 0x557bfcf10c00 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x557bfdd79320 tx=0x557bfdc315c0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 -- 172.21.15.16:0/2094789902 shutdown_connections 2024-04-25T10:13:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 --2- 172.21.15.16:0/2094789902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfddcb400 0x557bfcf10c00 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 -- 172.21.15.16:0/2094789902 >> 172.21.15.16:0/2094789902 conn(0x557bfce9ec00 msgr2=0x557bfcf87800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 -- 172.21.15.16:0/2094789902 shutdown_connections 2024-04-25T10:13:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 -- 172.21.15.16:0/2094789902 wait complete. 2024-04-25T10:13:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 -- 172.21.15.16:0/4185816081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfdc7e000 msgr2=0x557bfcf0f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 --2- 172.21.15.16:0/4185816081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfdc7e000 0x557bfcf0f080 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x557bfdcc2d80 tx=0x557bfdc31740 comp rx=0 tx=0).stop 2024-04-25T10:13:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 -- 172.21.15.16:0/4185816081 shutdown_connections 2024-04-25T10:13:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 --2- 172.21.15.16:0/4185816081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557bfce9f800 0x557bfcf0fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 --2- 172.21.15.16:0/4185816081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfdc7e400 0x557bfcf0eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 --2- 172.21.15.16:0/4185816081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfdc7e000 0x557bfcf0f080 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 -- 172.21.15.16:0/4185816081 >> 172.21.15.16:0/4185816081 conn(0x557bfce9f000 msgr2=0x557bfcf87000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 -- 172.21.15.16:0/4185816081 shutdown_connections 2024-04-25T10:13:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.676+0000 7f7c94a8d280 1 -- 172.21.15.16:0/4185816081 wait complete. 2024-04-25T10:13:34.690 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.105328, result: Events by type: Errors: 0 2024-04-25T10:13:34.690 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-04-25T10:13:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.730+0000 7f58a0634f40 1 Processor -- start 2024-04-25T10:13:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.730+0000 7f58a0634f40 1 -- start start 2024-04-25T10:13:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.730+0000 7f58a0634f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5b66a32a0 0x55a5b669f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.730+0000 7f58a0634f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5b66a5860 0x55a5b6715370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.730+0000 7f58a0634f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a5b6716270 0x55a5b6711410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.730+0000 7f58a0634f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5b66a5ea0 con 0x55a5b66a32a0 2024-04-25T10:13:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.730+0000 7f58a0634f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5b6602df0 con 0x55a5b66a5860 2024-04-25T10:13:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.730+0000 7f58a0634f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5b66a6140 con 0x55a5b6716270 2024-04-25T10:13:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.730+0000 7f589e137640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5b66a32a0 0x55a5b669f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.730+0000 7f589e137640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5b66a32a0 0x55a5b669f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32920/0 (socket says 172.21.15.16:32920) 2024-04-25T10:13:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.730+0000 7f589e137640 1 -- 172.21.15.16:0/3980047878 learned_addr learned my addr 172.21.15.16:0/3980047878 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.731+0000 7f589e938640 1 --2- 172.21.15.16:0/3980047878 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a5b6716270 0x55a5b6711410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.731+0000 7f589d936640 1 --2- 172.21.15.16:0/3980047878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5b66a5860 0x55a5b6715370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.731+0000 7f589e137640 1 -- 172.21.15.16:0/3980047878 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a5b6716270 msgr2=0x55a5b6711410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.731+0000 7f589e137640 1 --2- 172.21.15.16:0/3980047878 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a5b6716270 0x55a5b6711410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.731+0000 7f589e137640 1 -- 172.21.15.16:0/3980047878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5b66a5860 msgr2=0x55a5b6715370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.731+0000 7f589e137640 1 --2- 172.21.15.16:0/3980047878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5b66a5860 0x55a5b6715370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.731+0000 7f589e137640 1 -- 172.21.15.16:0/3980047878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5b6588c50 con 0x55a5b66a32a0 2024-04-25T10:13:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.731+0000 7f589e137640 1 --2- 172.21.15.16:0/3980047878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5b66a32a0 0x55a5b669f8f0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f589800a7a0 tx=0x7f589800a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef23e6827f64793 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.731+0000 7f589c934640 1 -- 172.21.15.16:0/3980047878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898018070 con 0x55a5b66a32a0 2024-04-25T10:13:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.731+0000 7f589c934640 1 -- 172.21.15.16:0/3980047878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5898007600 con 0x55a5b66a32a0 2024-04-25T10:13:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.731+0000 7f589c934640 1 -- 172.21.15.16:0/3980047878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58980079b0 con 0x55a5b66a32a0 2024-04-25T10:13:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.731+0000 7f58a0634f40 1 -- 172.21.15.16:0/3980047878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5b66a32a0 msgr2=0x55a5b669f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.731+0000 7f58a0634f40 1 --2- 172.21.15.16:0/3980047878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5b66a32a0 0x55a5b669f8f0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f589800a7a0 tx=0x7f589800a890 comp rx=0 tx=0).stop 2024-04-25T10:13:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.732+0000 7f58a0634f40 1 -- 172.21.15.16:0/3980047878 shutdown_connections 2024-04-25T10:13:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.732+0000 7f58a0634f40 1 --2- 172.21.15.16:0/3980047878 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a5b6716270 0x55a5b6711410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.732+0000 7f58a0634f40 1 --2- 172.21.15.16:0/3980047878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5b66a5860 0x55a5b6715370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.732+0000 7f58a0634f40 1 --2- 172.21.15.16:0/3980047878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5b66a32a0 0x55a5b669f8f0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.732+0000 7f58a0634f40 1 -- 172.21.15.16:0/3980047878 >> 172.21.15.16:0/3980047878 conn(0x55a5b66863d0 msgr2=0x55a5b668b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.732+0000 7f58a0634f40 1 -- 172.21.15.16:0/3980047878 shutdown_connections 2024-04-25T10:13:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.732+0000 7f58a0634f40 1 -- 172.21.15.16:0/3980047878 wait complete. 2024-04-25T10:13:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.732+0000 7f58a0634f40 1 Processor -- start 2024-04-25T10:13:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.732+0000 7f58a0634f40 1 -- start start 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.733+0000 7f58a0634f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a5b66a32a0 0x55a5b668ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.733+0000 7f58a0634f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5b66a5860 0x55a5b670f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.733+0000 7f58a0634f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5b6716270 0x55a5b672e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.733+0000 7f58a0634f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5b669fe30 con 0x55a5b6716270 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.733+0000 7f58a0634f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5b668bde0 con 0x55a5b66a5860 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.733+0000 7f58a0634f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5b6685eb0 con 0x55a5b66a32a0 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.733+0000 7f589e938640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5b6716270 0x55a5b672e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.733+0000 7f589e938640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5b6716270 0x55a5b672e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32924/0 (socket says 172.21.15.16:32924) 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.733+0000 7f589e938640 1 -- 172.21.15.16:0/2545685129 learned_addr learned my addr 172.21.15.16:0/2545685129 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.733+0000 7f589e137640 1 --2- 172.21.15.16:0/2545685129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a5b66a32a0 0x55a5b668ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.733+0000 7f589d936640 1 --2- 172.21.15.16:0/2545685129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5b66a5860 0x55a5b670f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.733+0000 7f589e938640 1 -- 172.21.15.16:0/2545685129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a5b66a32a0 msgr2=0x55a5b668ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.733+0000 7f589e938640 1 --2- 172.21.15.16:0/2545685129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a5b66a32a0 0x55a5b668ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.733+0000 7f589e938640 1 -- 172.21.15.16:0/2545685129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5b66a5860 msgr2=0x55a5b670f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.733+0000 7f589e938640 1 --2- 172.21.15.16:0/2545685129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5b66a5860 0x55a5b670f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.733+0000 7f589e938640 1 -- 172.21.15.16:0/2545685129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5b668b280 con 0x55a5b6716270 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.734+0000 7f589e938640 1 --2- 172.21.15.16:0/2545685129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5b6716270 0x55a5b672e040 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f5890007000 tx=0x7f5890003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.734+0000 7f588f7fe640 1 -- 172.21.15.16:0/2545685129 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589006a070 con 0x55a5b6716270 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.734+0000 7f588f7fe640 1 -- 172.21.15.16:0/2545685129 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58900514b0 con 0x55a5b6716270 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.734+0000 7f588f7fe640 1 -- 172.21.15.16:0/2545685129 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890055040 con 0x55a5b6716270 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.734+0000 7f58a0634f40 1 -- 172.21.15.16:0/2545685129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a5b668a580 con 0x55a5b6716270 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.734+0000 7f58a0634f40 1 -- 172.21.15.16:0/2545685129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5b668a820 con 0x55a5b6716270 2024-04-25T10:13:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.735+0000 7f588f7fe640 1 -- 172.21.15.16:0/2545685129 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f589006d070 con 0x55a5b6716270 2024-04-25T10:13:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.735+0000 7f588f7fe640 1 --2- 172.21.15.16:0/2545685129 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5880069ee0 0x7f588006c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.736+0000 7f588f7fe640 1 -- 172.21.15.16:0/2545685129 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f58900cc850 con 0x55a5b6716270 2024-04-25T10:13:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.736+0000 7f589e137640 1 --2- 172.21.15.16:0/2545685129 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5880069ee0 0x7f588006c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.736+0000 7f58a0634f40 1 --2- 172.21.15.16:0/2545685129 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55a5b670feb0 0x55a5b672ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.736+0000 7f58a0634f40 1 -- 172.21.15.16:0/2545685129 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v9 -- 0x55a5b670f280 con 0x55a5b670feb0 2024-04-25T10:13:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.736+0000 7f589d936640 1 --2- 172.21.15.16:0/2545685129 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55a5b670feb0 0x55a5b672ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.736+0000 7f589e137640 1 --2- 172.21.15.16:0/2545685129 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5880069ee0 0x7f588006c3a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5898016040 tx=0x7f589800c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.736+0000 7f589d936640 1 --2- 172.21.15.16:0/2545685129 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55a5b670feb0 0x55a5b672ebd0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.738+0000 7f589d936640 1 -- 172.21.15.16:0/2545685129 <== osd.5 v2:172.21.15.186:6800/1827108421 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v101'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f589400e040 con 0x55a5b670feb0 2024-04-25T10:13:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.738+0000 7f58a0634f40 1 -- 172.21.15.16:0/2545685129 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55a5b670feb0 msgr2=0x55a5b672ebd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.738+0000 7f58a0634f40 1 --2- 172.21.15.16:0/2545685129 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55a5b670feb0 0x55a5b672ebd0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.738+0000 7f58a0634f40 1 -- 172.21.15.16:0/2545685129 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5880069ee0 msgr2=0x7f588006c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.738+0000 7f58a0634f40 1 --2- 172.21.15.16:0/2545685129 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5880069ee0 0x7f588006c3a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5898016040 tx=0x7f589800c000 comp rx=0 tx=0).stop 2024-04-25T10:13:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.738+0000 7f58a0634f40 1 -- 172.21.15.16:0/2545685129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5b6716270 msgr2=0x55a5b672e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.738+0000 7f58a0634f40 1 --2- 172.21.15.16:0/2545685129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5b6716270 0x55a5b672e040 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f5890007000 tx=0x7f5890003330 comp rx=0 tx=0).stop 2024-04-25T10:13:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.738+0000 7f589e938640 1 -- 172.21.15.16:0/2545685129 reap_dead start 2024-04-25T10:13:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.738+0000 7f58a0634f40 1 -- 172.21.15.16:0/2545685129 shutdown_connections 2024-04-25T10:13:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.738+0000 7f58a0634f40 1 -- 172.21.15.16:0/2545685129 >> 172.21.15.16:0/2545685129 conn(0x55a5b66863d0 msgr2=0x55a5b668bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.738+0000 7f58a0634f40 1 -- 172.21.15.16:0/2545685129 shutdown_connections 2024-04-25T10:13:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.738+0000 7f58a0634f40 1 -- 172.21.15.16:0/2545685129 wait complete. 2024-04-25T10:13:34.747 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:13:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.884+0000 7f29caa07640 1 Processor -- start 2024-04-25T10:13:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.884+0000 7f29caa07640 1 -- start start 2024-04-25T10:13:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.885+0000 7f29caa07640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f29c415dc80 0x7f29c4161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.885+0000 7f29caa07640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c41627a0 con 0x7f29c40b5c00 2024-04-25T10:13:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.885+0000 7f29caa07640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c41621e0 con 0x7f29c40b97c0 2024-04-25T10:13:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.885+0000 7f29caa07640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c41624c0 con 0x7f29c415dc80 2024-04-25T10:13:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.885+0000 7f29c3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f29c415dc80 0x7f29c4161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.885+0000 7f29c3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f29c415dc80 0x7f29c4161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32932/0 (socket says 172.21.15.16:32932) 2024-04-25T10:13:34.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.885+0000 7f29c3fff640 1 -- 172.21.15.16:0/3850462839 learned_addr learned my addr 172.21.15.16:0/3850462839 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:34.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.885+0000 7f29c8fa4640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f29c40b5c00 0x7f29c40d2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41182/0 (socket says 172.21.15.16:41182) 2024-04-25T10:13:34.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.886+0000 7f29c3fff640 1 -- 172.21.15.16:0/3850462839 >> v1:172.21.15.186:6789/0 conn(0x7f29c40b97c0 legacy=0x7f29c40b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.886+0000 7f29c3fff640 1 -- 172.21.15.16:0/3850462839 >> v1:172.21.15.16:6789/0 conn(0x7f29c40b5c00 legacy=0x7f29c40d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.886+0000 7f29c3fff640 1 -- 172.21.15.16:0/3850462839 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c40d1c70 con 0x7f29c415dc80 2024-04-25T10:13:34.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.886+0000 7f29c3fff640 1 --2- 172.21.15.16:0/3850462839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c415dc80 0x7f29c4161870 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009870 tx=0x7f29ac02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1880f90b94b93a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.887+0000 7f29c2ffd640 1 -- 172.21.15.16:0/3850462839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac037070 con 0x7f29c415dc80 2024-04-25T10:13:34.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.887+0000 7f29c2ffd640 1 -- 172.21.15.16:0/3850462839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac02ae50 con 0x7f29c415dc80 2024-04-25T10:13:34.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.887+0000 7f29c2ffd640 1 -- 172.21.15.16:0/3850462839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac032860 con 0x7f29c415dc80 2024-04-25T10:13:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.887+0000 7f29caa07640 1 -- 172.21.15.16:0/3850462839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c415dc80 msgr2=0x7f29c4161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.887+0000 7f29caa07640 1 --2- 172.21.15.16:0/3850462839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c415dc80 0x7f29c4161870 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009870 tx=0x7f29ac02a300 comp rx=0 tx=0).stop 2024-04-25T10:13:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.887+0000 7f29caa07640 1 -- 172.21.15.16:0/3850462839 shutdown_connections 2024-04-25T10:13:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.887+0000 7f29caa07640 1 --2- 172.21.15.16:0/3850462839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c415dc80 0x7f29c4161870 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.887+0000 7f29caa07640 1 -- 172.21.15.16:0/3850462839 >> 172.21.15.16:0/3850462839 conn(0x7f29c40d04f0 msgr2=0x7f29c40d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.887+0000 7f29caa07640 1 -- 172.21.15.16:0/3850462839 shutdown_connections 2024-04-25T10:13:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.887+0000 7f29caa07640 1 -- 172.21.15.16:0/3850462839 wait complete. 2024-04-25T10:13:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.888+0000 7f29caa07640 1 Processor -- start 2024-04-25T10:13:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.888+0000 7f29caa07640 1 -- start start 2024-04-25T10:13:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.889+0000 7f29caa07640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29c40b5c00 0x7f29c415cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.889+0000 7f29caa07640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c40b97c0 0x7f29c40cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.889+0000 7f29caa07640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29c415dc80 0x7f29c40cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.889+0000 7f29caa07640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c4161f30 con 0x7f29c40b97c0 2024-04-25T10:13:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.889+0000 7f29caa07640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c4161db0 con 0x7f29c415dc80 2024-04-25T10:13:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.889+0000 7f29caa07640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40d2b80 con 0x7f29c40b5c00 2024-04-25T10:13:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.889+0000 7f29c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c40b97c0 0x7f29c40cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.889+0000 7f29c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c40b97c0 0x7f29c40cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32944/0 (socket says 172.21.15.16:32944) 2024-04-25T10:13:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.889+0000 7f29c37fe640 1 -- 172.21.15.16:0/2137462394 learned_addr learned my addr 172.21.15.16:0/2137462394 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.889+0000 7f29c3fff640 1 --2- 172.21.15.16:0/2137462394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29c40b5c00 0x7f29c415cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.889+0000 7f29c8fa4640 1 --2- 172.21.15.16:0/2137462394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29c415dc80 0x7f29c40cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.890+0000 7f29c37fe640 1 -- 172.21.15.16:0/2137462394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29c40b5c00 msgr2=0x7f29c415cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.890+0000 7f29c37fe640 1 --2- 172.21.15.16:0/2137462394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29c40b5c00 0x7f29c415cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.890+0000 7f29c37fe640 1 -- 172.21.15.16:0/2137462394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29c415dc80 msgr2=0x7f29c40cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.890+0000 7f29c37fe640 1 --2- 172.21.15.16:0/2137462394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29c415dc80 0x7f29c40cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.890+0000 7f29c37fe640 1 -- 172.21.15.16:0/2137462394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c41f3a70 con 0x7f29c40b97c0 2024-04-25T10:13:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.890+0000 7f29c3fff640 1 --2- 172.21.15.16:0/2137462394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29c40b5c00 0x7f29c415cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.890+0000 7f29c37fe640 1 --2- 172.21.15.16:0/2137462394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c40b97c0 0x7f29c40cd2a0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f29b400c400 tx=0x7f29b400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.890+0000 7f29c0ff9640 1 -- 172.21.15.16:0/2137462394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b40052d0 con 0x7f29c40b97c0 2024-04-25T10:13:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.890+0000 7f29c0ff9640 1 -- 172.21.15.16:0/2137462394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b4005490 con 0x7f29c40b97c0 2024-04-25T10:13:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.890+0000 7f29c0ff9640 1 -- 172.21.15.16:0/2137462394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4007260 con 0x7f29c40b97c0 2024-04-25T10:13:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.890+0000 7f29caa07640 1 -- 172.21.15.16:0/2137462394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c41f3ce0 con 0x7f29c40b97c0 2024-04-25T10:13:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.890+0000 7f29caa07640 1 -- 172.21.15.16:0/2137462394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c41f42a0 con 0x7f29c40b97c0 2024-04-25T10:13:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.891+0000 7f29caa07640 1 -- 172.21.15.16:0/2137462394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c40750b0 con 0x7f29c40b97c0 2024-04-25T10:13:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.891+0000 7f29c0ff9640 1 -- 172.21.15.16:0/2137462394 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29b4007400 con 0x7f29c40b97c0 2024-04-25T10:13:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.892+0000 7f29c0ff9640 1 --2- 172.21.15.16:0/2137462394 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f29a406b4d0 0x7f29a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.892+0000 7f29c0ff9640 1 -- 172.21.15.16:0/2137462394 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f29b4089720 con 0x7f29c40b97c0 2024-04-25T10:13:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.892+0000 7f29c3fff640 1 --2- 172.21.15.16:0/2137462394 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f29a406b4d0 0x7f29a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.892+0000 7f29c3fff640 1 --2- 172.21.15.16:0/2137462394 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f29a406b4d0 0x7f29a406d990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f29ac034000 tx=0x7f29ac02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:34.896+0000 7f29c0ff9640 1 -- 172.21.15.16:0/2137462394 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f29b40585d0 con 0x7f29c40b97c0 2024-04-25T10:13:35.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.117+0000 7f29caa07640 1 -- 172.21.15.16:0/2137462394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f29c40d1780 con 0x7f29c40b97c0 2024-04-25T10:13:35.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.479+0000 7f29c0ff9640 1 -- 172.21.15.16:0/2137462394 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v73) v1 ==== 162+0+0 (secure 0 0 0) 0x7f29b4057ed0 con 0x7f29c40b97c0 2024-04-25T10:13:35.483 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:13:35.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.481+0000 7f29caa07640 1 -- 172.21.15.16:0/2137462394 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f29a406b4d0 msgr2=0x7f29a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.481+0000 7f29caa07640 1 --2- 172.21.15.16:0/2137462394 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f29a406b4d0 0x7f29a406d990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f29ac034000 tx=0x7f29ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:13:35.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.481+0000 7f29caa07640 1 -- 172.21.15.16:0/2137462394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c40b97c0 msgr2=0x7f29c40cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.481+0000 7f29caa07640 1 --2- 172.21.15.16:0/2137462394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c40b97c0 0x7f29c40cd2a0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f29b400c400 tx=0x7f29b400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.481+0000 7f29caa07640 1 -- 172.21.15.16:0/2137462394 shutdown_connections 2024-04-25T10:13:35.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.481+0000 7f29caa07640 1 --2- 172.21.15.16:0/2137462394 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f29a406b4d0 0x7f29a406d990 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.481+0000 7f29caa07640 1 --2- 172.21.15.16:0/2137462394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29c415dc80 0x7f29c40cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.481+0000 7f29caa07640 1 --2- 172.21.15.16:0/2137462394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c40b97c0 0x7f29c40cd2a0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.481+0000 7f29caa07640 1 --2- 172.21.15.16:0/2137462394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29c40b5c00 0x7f29c415cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.481+0000 7f29caa07640 1 -- 172.21.15.16:0/2137462394 >> 172.21.15.16:0/2137462394 conn(0x7f29c40d04f0 msgr2=0x7f29c40b7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:35.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.481+0000 7f29caa07640 1 -- 172.21.15.16:0/2137462394 shutdown_connections 2024-04-25T10:13:35.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.482+0000 7f29caa07640 1 -- 172.21.15.16:0/2137462394 wait complete. 2024-04-25T10:13:35.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.598+0000 7f645e812640 1 Processor -- start 2024-04-25T10:13:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.598+0000 7f645e812640 1 -- start start 2024-04-25T10:13:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.598+0000 7f645e812640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f645815ec10 0x7f645806dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.598+0000 7f645e812640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645806e6c0 con 0x7f645806e1b0 2024-04-25T10:13:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.598+0000 7f645e812640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645806e9a0 con 0x7f64580b3750 2024-04-25T10:13:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.598+0000 7f645e812640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645806f2d0 con 0x7f645815ec10 2024-04-25T10:13:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f64577fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f645806e1b0 0x7f64580b3040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41188/0 (socket says 172.21.15.16:41188) 2024-04-25T10:13:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f64577fe640 1 -- 172.21.15.16:0/4267988539 learned_addr learned my addr 172.21.15.16:0/4267988539 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:13:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f6457fff640 1 --2- 172.21.15.16:0/4267988539 >> v2:172.21.15.186:3300/0 conn(0x7f645815ec10 0x7f645806dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f6456ffd640 1 -- 172.21.15.16:0/4267988539 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4041687975 0 0) 0x7f645806e6c0 con 0x7f645806e1b0 2024-04-25T10:13:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f6456ffd640 1 -- 172.21.15.16:0/4267988539 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6440003520 con 0x7f645806e1b0 2024-04-25T10:13:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f6456ffd640 1 -- 172.21.15.16:0/4267988539 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1286883297 0 0) 0x7f6440003520 con 0x7f645806e1b0 2024-04-25T10:13:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f6456ffd640 1 -- 172.21.15.16:0/4267988539 >> v2:172.21.15.186:3300/0 conn(0x7f645815ec10 msgr2=0x7f645806dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f6456ffd640 1 --2- 172.21.15.16:0/4267988539 >> v2:172.21.15.186:3300/0 conn(0x7f645815ec10 0x7f645806dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f6456ffd640 1 -- 172.21.15.16:0/4267988539 >> v1:172.21.15.186:6789/0 conn(0x7f64580b3750 legacy=0x7f645806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f6456ffd640 1 -- 172.21.15.16:0/4267988539 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645806ec80 con 0x7f645806e1b0 2024-04-25T10:13:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f6456ffd640 1 -- 172.21.15.16:0/4267988539 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f644c001670 con 0x7f645806e1b0 2024-04-25T10:13:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f6456ffd640 1 -- 172.21.15.16:0/4267988539 >> v1:172.21.15.16:6789/0 conn(0x7f645806e1b0 legacy=0x7f64580b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f6456ffd640 1 --2- 172.21.15.16:0/4267988539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6440003a30 0x7f6440003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f6456ffd640 1 --2- 172.21.15.16:0/4267988539 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6440004800 0x7f6440004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f6456ffd640 1 --2- 172.21.15.16:0/4267988539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64400052e0 0x7f644000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f6456ffd640 1 -- 172.21.15.16:0/4267988539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645806f2d0 con 0x7f6440003a30 2024-04-25T10:13:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f6456ffd640 1 -- 172.21.15.16:0/4267988539 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644000c230 con 0x7f64400052e0 2024-04-25T10:13:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f6456ffd640 1 -- 172.21.15.16:0/4267988539 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644000c430 con 0x7f6440004800 2024-04-25T10:13:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.599+0000 7f6457fff640 1 --2- 172.21.15.16:0/4267988539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6440003a30 0x7f6440003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.600+0000 7f645cdaf640 1 --2- 172.21.15.16:0/4267988539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64400052e0 0x7f644000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.600+0000 7f64577fe640 1 --2- 172.21.15.16:0/4267988539 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6440004800 0x7f6440004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.600+0000 7f6457fff640 1 -- 172.21.15.16:0/4267988539 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6440004800 msgr2=0x7f6440004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.600+0000 7f6457fff640 1 --2- 172.21.15.16:0/4267988539 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6440004800 0x7f6440004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.600+0000 7f6457fff640 1 -- 172.21.15.16:0/4267988539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64400052e0 msgr2=0x7f644000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.600+0000 7f6457fff640 1 --2- 172.21.15.16:0/4267988539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64400052e0 0x7f644000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.600+0000 7f645cdaf640 1 -- 172.21.15.16:0/4267988539 reap_dead start 2024-04-25T10:13:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.600+0000 7f6457fff640 1 -- 172.21.15.16:0/4267988539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6440003520 con 0x7f6440003a30 2024-04-25T10:13:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.600+0000 7f64577fe640 1 --2- 172.21.15.16:0/4267988539 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6440004800 0x7f6440004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.600+0000 7f6457fff640 1 --2- 172.21.15.16:0/4267988539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6440003a30 0x7f6440003e50 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f6448009a90 tx=0x7f6440005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4adc879080cbd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.601+0000 7f6456ffd640 1 -- 172.21.15.16:0/4267988539 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6448036070 con 0x7f6440003a30 2024-04-25T10:13:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.601+0000 7f6456ffd640 1 -- 172.21.15.16:0/4267988539 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6448031400 con 0x7f6440003a30 2024-04-25T10:13:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.601+0000 7f6456ffd640 1 -- 172.21.15.16:0/4267988539 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6448009690 con 0x7f6440003a30 2024-04-25T10:13:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.601+0000 7f645e812640 1 -- 172.21.15.16:0/4267988539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6440003a30 msgr2=0x7f6440003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.601+0000 7f645e812640 1 --2- 172.21.15.16:0/4267988539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6440003a30 0x7f6440003e50 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f6448009a90 tx=0x7f6440005830 comp rx=0 tx=0).stop 2024-04-25T10:13:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.601+0000 7f645e812640 1 -- 172.21.15.16:0/4267988539 shutdown_connections 2024-04-25T10:13:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.601+0000 7f645e812640 1 --2- 172.21.15.16:0/4267988539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6440003a30 0x7f6440003e50 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.601+0000 7f645e812640 1 -- 172.21.15.16:0/4267988539 >> 172.21.15.16:0/4267988539 conn(0x7f64580b2220 msgr2=0x7f64580cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.601+0000 7f645e812640 1 -- 172.21.15.16:0/4267988539 shutdown_connections 2024-04-25T10:13:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.601+0000 7f645e812640 1 -- 172.21.15.16:0/4267988539 wait complete. 2024-04-25T10:13:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.602+0000 7f645e812640 1 Processor -- start 2024-04-25T10:13:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.603+0000 7f645e812640 1 -- start start 2024-04-25T10:13:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.603+0000 7f645e812640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6440003a30 0x7f6458141b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.603+0000 7f645e812640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64580ae900 0x7f64580aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.603+0000 7f645e812640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f64581f75d0 0x7f64581f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.603+0000 7f645e812640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645815e2e0 con 0x7f6440003a30 2024-04-25T10:13:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.603+0000 7f645e812640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645815eff0 con 0x7f64580ae900 2024-04-25T10:13:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.603+0000 7f645e812640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645815e460 con 0x7f64581f75d0 2024-04-25T10:13:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.603+0000 7f6457fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6440003a30 0x7f6458141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.604+0000 7f6457fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6440003a30 0x7f6458141b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32962/0 (socket says 172.21.15.16:32962) 2024-04-25T10:13:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.604+0000 7f6457fff640 1 -- 172.21.15.16:0/99039772 learned_addr learned my addr 172.21.15.16:0/99039772 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.604+0000 7f64577fe640 1 --2- 172.21.15.16:0/99039772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64580ae900 0x7f64580aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.604+0000 7f645cdaf640 1 --2- 172.21.15.16:0/99039772 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f64581f75d0 0x7f64581f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.604+0000 7f6457fff640 1 -- 172.21.15.16:0/99039772 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f64581f75d0 msgr2=0x7f64581f7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.604+0000 7f6457fff640 1 --2- 172.21.15.16:0/99039772 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f64581f75d0 0x7f64581f7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.604+0000 7f6457fff640 1 -- 172.21.15.16:0/99039772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64580ae900 msgr2=0x7f64580aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.604+0000 7f6457fff640 1 --2- 172.21.15.16:0/99039772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64580ae900 0x7f64580aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.604+0000 7f6457fff640 1 -- 172.21.15.16:0/99039772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6458152470 con 0x7f6440003a30 2024-04-25T10:13:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.604+0000 7f6457fff640 1 --2- 172.21.15.16:0/99039772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6440003a30 0x7f6458141b10 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f6448034040 tx=0x7f6448004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.604+0000 7f6454ff9640 1 -- 172.21.15.16:0/99039772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644803f020 con 0x7f6440003a30 2024-04-25T10:13:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.604+0000 7f6454ff9640 1 -- 172.21.15.16:0/99039772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6448037040 con 0x7f6440003a30 2024-04-25T10:13:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.604+0000 7f645e812640 1 -- 172.21.15.16:0/99039772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64580af460 con 0x7f6440003a30 2024-04-25T10:13:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.604+0000 7f6454ff9640 1 -- 172.21.15.16:0/99039772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6448036070 con 0x7f6440003a30 2024-04-25T10:13:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.604+0000 7f645e812640 1 -- 172.21.15.16:0/99039772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64580afa20 con 0x7f6440003a30 2024-04-25T10:13:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.605+0000 7f645e812640 1 -- 172.21.15.16:0/99039772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6458156f40 con 0x7f6440003a30 2024-04-25T10:13:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.606+0000 7f6454ff9640 1 -- 172.21.15.16:0/99039772 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6448042020 con 0x7f6440003a30 2024-04-25T10:13:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.606+0000 7f6454ff9640 1 --2- 172.21.15.16:0/99039772 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f644406b500 0x7f644406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.606+0000 7f6454ff9640 1 -- 172.21.15.16:0/99039772 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f64480aa250 con 0x7f6440003a30 2024-04-25T10:13:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.606+0000 7f64577fe640 1 --2- 172.21.15.16:0/99039772 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f644406b500 0x7f644406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.607+0000 7f64577fe640 1 --2- 172.21.15.16:0/99039772 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f644406b500 0x7f644406d9c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f644c000fd0 tx=0x7f644c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.610+0000 7f6454ff9640 1 -- 172.21.15.16:0/99039772 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6458156f40 con 0x7f6440003a30 2024-04-25T10:13:35.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.831+0000 7f645e812640 1 -- 172.21.15.16:0/99039772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64581603b0 con 0x7f6440003a30 2024-04-25T10:13:35.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.831+0000 7f6454ff9640 1 -- 172.21.15.16:0/99039772 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f64581603b0 con 0x7f6440003a30 2024-04-25T10:13:35.836 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:13:35.836 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7243,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/109406141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":109406141},{"type":"v1","addr":"172.21.15.186:6837","nonce":109406141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2704401503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2704401503},{"type":"v1","addr":"172.21.15.16:6835","nonce":2704401503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/939221717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":939221717},{"type":"v1","addr":"172.21.15.16:6834","nonce":939221717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:48.958938+0000","modified":"2024-04-25T10:13:35.478706+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":36575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_36575":{"gid":36575,"name":"b","rank":0,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.186:6836/2008057690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2008057690},{"type":"v1","addr":"172.21.15.186:6836","nonce":2008057690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 73 2024-04-25T10:13:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.833+0000 7f645e812640 1 -- 172.21.15.16:0/99039772 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f644406b500 msgr2=0x7f644406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.833+0000 7f645e812640 1 --2- 172.21.15.16:0/99039772 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f644406b500 0x7f644406d9c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f644c000fd0 tx=0x7f644c009040 comp rx=0 tx=0).stop 2024-04-25T10:13:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.833+0000 7f645e812640 1 -- 172.21.15.16:0/99039772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6440003a30 msgr2=0x7f6458141b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.833+0000 7f645e812640 1 --2- 172.21.15.16:0/99039772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6440003a30 0x7f6458141b10 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f6448034040 tx=0x7f6448004730 comp rx=0 tx=0).stop 2024-04-25T10:13:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.833+0000 7f645e812640 1 -- 172.21.15.16:0/99039772 shutdown_connections 2024-04-25T10:13:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.833+0000 7f645e812640 1 --2- 172.21.15.16:0/99039772 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f644406b500 0x7f644406d9c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.833+0000 7f645e812640 1 --2- 172.21.15.16:0/99039772 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f64581f75d0 0x7f64581f7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.833+0000 7f645e812640 1 --2- 172.21.15.16:0/99039772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64580ae900 0x7f64580aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.833+0000 7f645e812640 1 --2- 172.21.15.16:0/99039772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6440003a30 0x7f6458141b10 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.833+0000 7f645e812640 1 -- 172.21.15.16:0/99039772 >> 172.21.15.16:0/99039772 conn(0x7f64580b2220 msgr2=0x7f64580cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.833+0000 7f645e812640 1 -- 172.21.15.16:0/99039772 shutdown_connections 2024-04-25T10:13:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:35.833+0000 7f645e812640 1 -- 172.21.15.16:0/99039772 wait complete. 2024-04-25T10:13:36.251 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:12:48.958938+0000', 'modified': '2024-04-25T10:13:35.478706+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 101, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 36575}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_36575': {'gid': 36575, 'name': 'b', 'rank': 0, 'incarnation': 73, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.186:6836/2008057690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 2008057690}, {'type': 'v1', 'addr': '172.21.15.186:6836', 'nonce': 2008057690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:36.252 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_36575:up:replay 2024-04-25T10:13:37.253 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.362+0000 7f22dc007640 1 Processor -- start 2024-04-25T10:13:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.362+0000 7f22dc007640 1 -- start start 2024-04-25T10:13:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.363+0000 7f22dc007640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22d415ba10 0x7f22d4099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.363+0000 7f22dc007640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f22d4099f80 0x7f22d406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.363+0000 7f22dc007640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f22d406c1f0 0x7f22d40cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.363+0000 7f22dc007640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d40cbe50 con 0x7f22d4099f80 2024-04-25T10:13:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.363+0000 7f22dc007640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d40cfe80 con 0x7f22d406c1f0 2024-04-25T10:13:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.363+0000 7f22dc007640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d409a3a0 con 0x7f22d415ba10 2024-04-25T10:13:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.363+0000 7f22d95a2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f22d4099f80 0x7f22d406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.363+0000 7f22d95a2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f22d4099f80 0x7f22d406bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32970/0 (socket says 172.21.15.16:32970) 2024-04-25T10:13:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.363+0000 7f22d95a2640 1 -- 172.21.15.16:0/521690470 learned_addr learned my addr 172.21.15.16:0/521690470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.363+0000 7f22d9da3640 1 --2- 172.21.15.16:0/521690470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22d415ba10 0x7f22d4099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.363+0000 7f22da5a4640 1 --2- 172.21.15.16:0/521690470 >> v2:172.21.15.186:3300/0 conn(0x7f22d406c1f0 0x7f22d40cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.363+0000 7f22d95a2640 1 -- 172.21.15.16:0/521690470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22d415ba10 msgr2=0x7f22d4099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.363+0000 7f22d95a2640 1 --2- 172.21.15.16:0/521690470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22d415ba10 0x7f22d4099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.363+0000 7f22d95a2640 1 -- 172.21.15.16:0/521690470 >> v2:172.21.15.186:3300/0 conn(0x7f22d406c1f0 msgr2=0x7f22d40cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.363+0000 7f22d95a2640 1 --2- 172.21.15.16:0/521690470 >> v2:172.21.15.186:3300/0 conn(0x7f22d406c1f0 0x7f22d40cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.363+0000 7f22d95a2640 1 -- 172.21.15.16:0/521690470 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d406c680 con 0x7f22d4099f80 2024-04-25T10:13:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.363+0000 7f22da5a4640 1 --2- 172.21.15.16:0/521690470 >> v2:172.21.15.186:3300/0 conn(0x7f22d406c1f0 0x7f22d40cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.364+0000 7f22d95a2640 1 --2- 172.21.15.16:0/521690470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d4099f80 0x7f22d406bcb0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f22d41616a0 tx=0x7f22c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fd4047eb1ce51c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.364+0000 7f22d8da1640 1 -- 172.21.15.16:0/521690470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4037070 con 0x7f22d4099f80 2024-04-25T10:13:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.364+0000 7f22d8da1640 1 -- 172.21.15.16:0/521690470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c4032440 con 0x7f22d4099f80 2024-04-25T10:13:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.364+0000 7f22d8da1640 1 -- 172.21.15.16:0/521690470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4032810 con 0x7f22d4099f80 2024-04-25T10:13:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.365+0000 7f22dc007640 1 -- 172.21.15.16:0/521690470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d4099f80 msgr2=0x7f22d406bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.365+0000 7f22dc007640 1 --2- 172.21.15.16:0/521690470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d4099f80 0x7f22d406bcb0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f22d41616a0 tx=0x7f22c402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.365+0000 7f22dc007640 1 -- 172.21.15.16:0/521690470 shutdown_connections 2024-04-25T10:13:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.365+0000 7f22dc007640 1 --2- 172.21.15.16:0/521690470 >> v2:172.21.15.186:3300/0 conn(0x7f22d406c1f0 0x7f22d40cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.365+0000 7f22dc007640 1 --2- 172.21.15.16:0/521690470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d4099f80 0x7f22d406bcb0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.365+0000 7f22dc007640 1 --2- 172.21.15.16:0/521690470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22d415ba10 0x7f22d4099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.365+0000 7f22dc007640 1 -- 172.21.15.16:0/521690470 >> 172.21.15.16:0/521690470 conn(0x7f22d40b32f0 msgr2=0x7f22d40b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.365+0000 7f22dc007640 1 -- 172.21.15.16:0/521690470 shutdown_connections 2024-04-25T10:13:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.365+0000 7f22dc007640 1 -- 172.21.15.16:0/521690470 wait complete. 2024-04-25T10:13:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.366+0000 7f22dc007640 1 Processor -- start 2024-04-25T10:13:37.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.366+0000 7f22dc007640 1 -- start start 2024-04-25T10:13:37.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.367+0000 7f22dc007640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22d406c1f0 0x7f22d415a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.367+0000 7f22dc007640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d4099f80 0x7f22d40c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.367+0000 7f22dc007640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22d415ba10 0x7f22d40c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.367+0000 7f22dc007640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d406cec0 con 0x7f22d4099f80 2024-04-25T10:13:37.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.367+0000 7f22dc007640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d406cbe0 con 0x7f22d406c1f0 2024-04-25T10:13:37.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.367+0000 7f22dc007640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d40cd760 con 0x7f22d415ba10 2024-04-25T10:13:37.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.367+0000 7f22d95a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d4099f80 0x7f22d40c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.367+0000 7f22d95a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d4099f80 0x7f22d40c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32978/0 (socket says 172.21.15.16:32978) 2024-04-25T10:13:37.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.367+0000 7f22d95a2640 1 -- 172.21.15.16:0/293768008 learned_addr learned my addr 172.21.15.16:0/293768008 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:37.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.367+0000 7f22d9da3640 1 --2- 172.21.15.16:0/293768008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22d406c1f0 0x7f22d415a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.367+0000 7f22da5a4640 1 --2- 172.21.15.16:0/293768008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22d415ba10 0x7f22d40c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.367+0000 7f22d95a2640 1 -- 172.21.15.16:0/293768008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22d415ba10 msgr2=0x7f22d40c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.367+0000 7f22d95a2640 1 --2- 172.21.15.16:0/293768008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22d415ba10 0x7f22d40c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.367+0000 7f22d95a2640 1 -- 172.21.15.16:0/293768008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22d406c1f0 msgr2=0x7f22d415a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.367+0000 7f22d95a2640 1 --2- 172.21.15.16:0/293768008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22d406c1f0 0x7f22d415a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.367+0000 7f22d95a2640 1 -- 172.21.15.16:0/293768008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d40cb620 con 0x7f22d4099f80 2024-04-25T10:13:37.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.367+0000 7f22d9da3640 1 --2- 172.21.15.16:0/293768008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22d406c1f0 0x7f22d415a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:37.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.368+0000 7f22d95a2640 1 --2- 172.21.15.16:0/293768008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d4099f80 0x7f22d40c8fc0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f22c4000c00 tx=0x7f22c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.368+0000 7f22ca7fc640 1 -- 172.21.15.16:0/293768008 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4041020 con 0x7f22d4099f80 2024-04-25T10:13:37.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.368+0000 7f22ca7fc640 1 -- 172.21.15.16:0/293768008 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c4030970 con 0x7f22d4099f80 2024-04-25T10:13:37.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.368+0000 7f22ca7fc640 1 -- 172.21.15.16:0/293768008 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4037040 con 0x7f22d4099f80 2024-04-25T10:13:37.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.368+0000 7f22dc007640 1 -- 172.21.15.16:0/293768008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d4160d90 con 0x7f22d4099f80 2024-04-25T10:13:37.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.368+0000 7f22dc007640 1 -- 172.21.15.16:0/293768008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d40c6620 con 0x7f22d4099f80 2024-04-25T10:13:37.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.369+0000 7f22dc007640 1 -- 172.21.15.16:0/293768008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d415b370 con 0x7f22d4099f80 2024-04-25T10:13:37.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.369+0000 7f22ca7fc640 1 -- 172.21.15.16:0/293768008 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f22c4044020 con 0x7f22d4099f80 2024-04-25T10:13:37.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.370+0000 7f22ca7fc640 1 --2- 172.21.15.16:0/293768008 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f22b806b4d0 0x7f22b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.370+0000 7f22ca7fc640 1 -- 172.21.15.16:0/293768008 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f22c40aac60 con 0x7f22d4099f80 2024-04-25T10:13:37.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.370+0000 7f22d9da3640 1 --2- 172.21.15.16:0/293768008 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f22b806b4d0 0x7f22b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.371+0000 7f22d9da3640 1 --2- 172.21.15.16:0/293768008 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f22b806b4d0 0x7f22b806d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f22d0006fd0 tx=0x7f22d0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.374+0000 7f22ca7fc640 1 -- 172.21.15.16:0/293768008 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f22c404b030 con 0x7f22d4099f80 2024-04-25T10:13:37.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.592+0000 7f22dc007640 1 -- 172.21.15.16:0/293768008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22d409a570 con 0x7f22d4099f80 2024-04-25T10:13:37.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.593+0000 7f22ca7fc640 1 -- 172.21.15.16:0/293768008 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f22d40cb620 con 0x7f22d4099f80 2024-04-25T10:13:37.597 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:13:37.597 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7243,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/109406141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":109406141},{"type":"v1","addr":"172.21.15.186:6837","nonce":109406141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2704401503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2704401503},{"type":"v1","addr":"172.21.15.16:6835","nonce":2704401503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/939221717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":939221717},{"type":"v1","addr":"172.21.15.16:6834","nonce":939221717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:48.958938+0000","modified":"2024-04-25T10:13:36.484645+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":36575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_36575":{"gid":36575,"name":"b","rank":0,"incarnation":73,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.186:6836/2008057690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2008057690},{"type":"v1","addr":"172.21.15.186:6836","nonce":2008057690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 75 2024-04-25T10:13:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.595+0000 7f22dc007640 1 -- 172.21.15.16:0/293768008 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f22b806b4d0 msgr2=0x7f22b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.595+0000 7f22dc007640 1 --2- 172.21.15.16:0/293768008 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f22b806b4d0 0x7f22b806d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f22d0006fd0 tx=0x7f22d0008040 comp rx=0 tx=0).stop 2024-04-25T10:13:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.595+0000 7f22dc007640 1 -- 172.21.15.16:0/293768008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d4099f80 msgr2=0x7f22d40c8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.595+0000 7f22dc007640 1 --2- 172.21.15.16:0/293768008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d4099f80 0x7f22d40c8fc0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f22c4000c00 tx=0x7f22c40046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.595+0000 7f22dc007640 1 -- 172.21.15.16:0/293768008 shutdown_connections 2024-04-25T10:13:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.595+0000 7f22dc007640 1 --2- 172.21.15.16:0/293768008 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f22b806b4d0 0x7f22b806d990 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.595+0000 7f22dc007640 1 --2- 172.21.15.16:0/293768008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22d415ba10 0x7f22d40c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.595+0000 7f22dc007640 1 --2- 172.21.15.16:0/293768008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d4099f80 0x7f22d40c8fc0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.595+0000 7f22dc007640 1 --2- 172.21.15.16:0/293768008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22d406c1f0 0x7f22d415a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.595+0000 7f22dc007640 1 -- 172.21.15.16:0/293768008 >> 172.21.15.16:0/293768008 conn(0x7f22d40b32f0 msgr2=0x7f22d40d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.595+0000 7f22dc007640 1 -- 172.21.15.16:0/293768008 shutdown_connections 2024-04-25T10:13:37.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:37.595+0000 7f22dc007640 1 -- 172.21.15.16:0/293768008 wait complete. 2024-04-25T10:13:38.014 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:12:48.958938+0000', 'modified': '2024-04-25T10:13:36.484645+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 101, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 36575}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_36575': {'gid': 36575, 'name': 'b', 'rank': 0, 'incarnation': 73, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.186:6836/2008057690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 2008057690}, {'type': 'v1', 'addr': '172.21.15.186:6836', 'nonce': 2008057690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:38.014 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_36575:up:rejoin 2024-04-25T10:13:38.738 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:38.741 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:39.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:39.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.125+0000 7faa28ef5640 1 Processor -- start 2024-04-25T10:13:39.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.125+0000 7faa28ef5640 1 -- start start 2024-04-25T10:13:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.126+0000 7faa28ef5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa2415ba10 0x7faa24099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.126+0000 7faa28ef5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7faa24099f80 0x7faa2406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.126+0000 7faa28ef5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2409a3a0 con 0x7faa2406c1f0 2024-04-25T10:13:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.126+0000 7faa28ef5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa240cbe50 con 0x7faa24099f80 2024-04-25T10:13:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.126+0000 7faa28ef5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa240cfe80 con 0x7faa2415ba10 2024-04-25T10:13:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.126+0000 7faa2259c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa2415ba10 0x7faa24099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.126+0000 7faa2259c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa2415ba10 0x7faa24099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:37948/0 (socket says 172.21.15.16:37948) 2024-04-25T10:13:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.126+0000 7faa1bfff640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7faa24099f80 0x7faa2406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.126+0000 7faa2259c640 1 -- 172.21.15.16:0/3809071099 learned_addr learned my addr 172.21.15.16:0/3809071099 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.126+0000 7faa21d9b640 1 -- 172.21.15.16:0/3809071099 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3849356789 0 0) 0x7faa2409a3a0 con 0x7faa2406c1f0 2024-04-25T10:13:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.126+0000 7faa21d9b640 1 -- 172.21.15.16:0/3809071099 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa10003520 con 0x7faa2406c1f0 2024-04-25T10:13:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.126+0000 7faa2259c640 1 -- 172.21.15.16:0/3809071099 >> v2:172.21.15.186:3300/0 conn(0x7faa24099f80 msgr2=0x7faa2406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.126+0000 7faa2259c640 1 --2- 172.21.15.16:0/3809071099 >> v2:172.21.15.186:3300/0 conn(0x7faa24099f80 0x7faa2406bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.126+0000 7faa2259c640 1 -- 172.21.15.16:0/3809071099 >> v1:172.21.15.186:6789/0 conn(0x7faa2406c1f0 legacy=0x7faa240cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.126+0000 7faa2259c640 1 -- 172.21.15.16:0/3809071099 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2406c680 con 0x7faa2415ba10 2024-04-25T10:13:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.126+0000 7faa1bfff640 1 --2- 172.21.15.16:0/3809071099 >> v2:172.21.15.186:3300/0 conn(0x7faa24099f80 0x7faa2406bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:39.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.127+0000 7faa2259c640 1 --2- 172.21.15.16:0/3809071099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa2415ba10 0x7faa24099a40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faa14009870 tx=0x7faa1402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d9017d7a7dc878b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.127+0000 7faa21d9b640 1 -- 172.21.15.16:0/3809071099 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14037070 con 0x7faa2415ba10 2024-04-25T10:13:39.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.127+0000 7faa21d9b640 1 -- 172.21.15.16:0/3809071099 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa14032440 con 0x7faa2415ba10 2024-04-25T10:13:39.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.127+0000 7faa21d9b640 1 -- 172.21.15.16:0/3809071099 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140327f0 con 0x7faa2415ba10 2024-04-25T10:13:39.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.128+0000 7faa28ef5640 1 -- 172.21.15.16:0/3809071099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa2415ba10 msgr2=0x7faa24099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.128+0000 7faa28ef5640 1 --2- 172.21.15.16:0/3809071099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa2415ba10 0x7faa24099a40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faa14009870 tx=0x7faa1402a270 comp rx=0 tx=0).stop 2024-04-25T10:13:39.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.128+0000 7faa28ef5640 1 -- 172.21.15.16:0/3809071099 shutdown_connections 2024-04-25T10:13:39.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.128+0000 7faa28ef5640 1 --2- 172.21.15.16:0/3809071099 >> v2:172.21.15.186:3300/0 conn(0x7faa24099f80 0x7faa2406bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.128+0000 7faa28ef5640 1 --2- 172.21.15.16:0/3809071099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa2415ba10 0x7faa24099a40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.128+0000 7faa28ef5640 1 -- 172.21.15.16:0/3809071099 >> 172.21.15.16:0/3809071099 conn(0x7faa240b32f0 msgr2=0x7faa240b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.128+0000 7faa28ef5640 1 -- 172.21.15.16:0/3809071099 shutdown_connections 2024-04-25T10:13:39.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.128+0000 7faa28ef5640 1 -- 172.21.15.16:0/3809071099 wait complete. 2024-04-25T10:13:39.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.129+0000 7faa28ef5640 1 Processor -- start 2024-04-25T10:13:39.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.130+0000 7faa28ef5640 1 -- start start 2024-04-25T10:13:39.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.130+0000 7faa28ef5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa2406c1f0 0x7faa2415c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.130+0000 7faa28ef5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa24099f80 0x7faa240cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.130+0000 7faa28ef5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2415ba10 0x7faa240cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.130+0000 7faa28ef5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2406cec0 con 0x7faa2415ba10 2024-04-25T10:13:39.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.130+0000 7faa28ef5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2406cbe0 con 0x7faa2406c1f0 2024-04-25T10:13:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.130+0000 7faa28ef5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa240cd930 con 0x7faa24099f80 2024-04-25T10:13:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.130+0000 7faa22d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2415ba10 0x7faa240cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.130+0000 7faa22d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2415ba10 0x7faa240cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32988/0 (socket says 172.21.15.16:32988) 2024-04-25T10:13:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.130+0000 7faa22d9d640 1 -- 172.21.15.16:0/2102046617 learned_addr learned my addr 172.21.15.16:0/2102046617 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.130+0000 7faa2259c640 1 --2- 172.21.15.16:0/2102046617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa2406c1f0 0x7faa2415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.130+0000 7faa1bfff640 1 --2- 172.21.15.16:0/2102046617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa24099f80 0x7faa240cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.131+0000 7faa22d9d640 1 -- 172.21.15.16:0/2102046617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa24099f80 msgr2=0x7faa240cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.131+0000 7faa22d9d640 1 --2- 172.21.15.16:0/2102046617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa24099f80 0x7faa240cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.131+0000 7faa22d9d640 1 -- 172.21.15.16:0/2102046617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa2406c1f0 msgr2=0x7faa2415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.131+0000 7faa22d9d640 1 --2- 172.21.15.16:0/2102046617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa2406c1f0 0x7faa2415c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.131+0000 7faa22d9d640 1 -- 172.21.15.16:0/2102046617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2406fca0 con 0x7faa2415ba10 2024-04-25T10:13:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.131+0000 7faa22d9d640 1 --2- 172.21.15.16:0/2102046617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2415ba10 0x7faa240cce50 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00c790 tx=0x7faa1c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.131+0000 7faa1affd640 1 -- 172.21.15.16:0/2102046617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c003b70 con 0x7faa2415ba10 2024-04-25T10:13:39.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.131+0000 7faa1affd640 1 -- 172.21.15.16:0/2102046617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1c003d10 con 0x7faa2415ba10 2024-04-25T10:13:39.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.131+0000 7faa28ef5640 1 -- 172.21.15.16:0/2102046617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2415eb00 con 0x7faa2415ba10 2024-04-25T10:13:39.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.131+0000 7faa1affd640 1 -- 172.21.15.16:0/2102046617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c00aa50 con 0x7faa2415ba10 2024-04-25T10:13:39.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.131+0000 7faa28ef5640 1 -- 172.21.15.16:0/2102046617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2414ec30 con 0x7faa2415ba10 2024-04-25T10:13:39.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.132+0000 7faa28ef5640 1 -- 172.21.15.16:0/2102046617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa240749b0 con 0x7faa2415ba10 2024-04-25T10:13:39.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.133+0000 7faa1affd640 1 -- 172.21.15.16:0/2102046617 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa1c00abf0 con 0x7faa2415ba10 2024-04-25T10:13:39.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.133+0000 7faa1affd640 1 --2- 172.21.15.16:0/2102046617 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7faa0406b500 0x7faa0406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.133+0000 7faa1affd640 1 -- 172.21.15.16:0/2102046617 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x7faa1c08a050 con 0x7faa2415ba10 2024-04-25T10:13:39.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.133+0000 7faa2259c640 1 --2- 172.21.15.16:0/2102046617 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7faa0406b500 0x7faa0406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.134+0000 7faa2259c640 1 --2- 172.21.15.16:0/2102046617 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7faa0406b500 0x7faa0406d9c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7faa14034000 tx=0x7faa1402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.137+0000 7faa1affd640 1 -- 172.21.15.16:0/2102046617 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa1c058820 con 0x7faa2415ba10 2024-04-25T10:13:39.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.356+0000 7faa28ef5640 1 -- 172.21.15.16:0/2102046617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa2409a570 con 0x7faa2415ba10 2024-04-25T10:13:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.357+0000 7faa1affd640 1 -- 172.21.15.16:0/2102046617 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4945 (secure 0 0 0) 0x7faa1c05c8c0 con 0x7faa2415ba10 2024-04-25T10:13:39.361 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:13:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":76,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7243,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/109406141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":109406141},{"type":"v1","addr":"172.21.15.186:6837","nonce":109406141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2704401503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2704401503},{"type":"v1","addr":"172.21.15.16:6835","nonce":2704401503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/939221717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":939221717},{"type":"v1","addr":"172.21.15.16:6834","nonce":939221717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:48.958938+0000","modified":"2024-04-25T10:13:37.486564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":36575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_36575":{"gid":36575,"name":"b","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.186:6836/2008057690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2008057690},{"type":"v1","addr":"172.21.15.186:6836","nonce":2008057690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:39.362 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 76 2024-04-25T10:13:39.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.359+0000 7faa28ef5640 1 -- 172.21.15.16:0/2102046617 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7faa0406b500 msgr2=0x7faa0406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.359+0000 7faa28ef5640 1 --2- 172.21.15.16:0/2102046617 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7faa0406b500 0x7faa0406d9c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7faa14034000 tx=0x7faa1402e000 comp rx=0 tx=0).stop 2024-04-25T10:13:39.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.359+0000 7faa28ef5640 1 -- 172.21.15.16:0/2102046617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2415ba10 msgr2=0x7faa240cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.359+0000 7faa28ef5640 1 --2- 172.21.15.16:0/2102046617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2415ba10 0x7faa240cce50 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00c790 tx=0x7faa1c00cc50 comp rx=0 tx=0).stop 2024-04-25T10:13:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.359+0000 7faa28ef5640 1 -- 172.21.15.16:0/2102046617 shutdown_connections 2024-04-25T10:13:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.359+0000 7faa28ef5640 1 --2- 172.21.15.16:0/2102046617 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7faa0406b500 0x7faa0406d9c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.359+0000 7faa28ef5640 1 --2- 172.21.15.16:0/2102046617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2415ba10 0x7faa240cce50 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.359+0000 7faa28ef5640 1 --2- 172.21.15.16:0/2102046617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faa24099f80 0x7faa240cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.359+0000 7faa28ef5640 1 --2- 172.21.15.16:0/2102046617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa2406c1f0 0x7faa2415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.359+0000 7faa28ef5640 1 -- 172.21.15.16:0/2102046617 >> 172.21.15.16:0/2102046617 conn(0x7faa240b32f0 msgr2=0x7faa240cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.359+0000 7faa28ef5640 1 -- 172.21.15.16:0/2102046617 shutdown_connections 2024-04-25T10:13:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.359+0000 7faa28ef5640 1 -- 172.21.15.16:0/2102046617 wait complete. 2024-04-25T10:13:39.777 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:12:48.958938+0000', 'modified': '2024-04-25T10:13:37.486564+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 101, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 36575}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_36575': {'gid': 36575, 'name': 'b', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.186:6836/2008057690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 2008057690}, {'type': 'v1', 'addr': '172.21.15.186:6836', 'nonce': 2008057690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:39.777 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:13:39.778 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:13:39.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.878+0000 7f5f33f32640 1 Processor -- start 2024-04-25T10:13:39.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.878+0000 7f5f33f32640 1 -- start start 2024-04-25T10:13:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.879+0000 7f5f33f32640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5f2c0cef40 0x7f5f2c0b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.879+0000 7f5f33f32640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5f2c1618d0 0x7f5f2c0d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.879+0000 7f5f33f32640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2c162040 con 0x7f5f2c1610b0 2024-04-25T10:13:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.879+0000 7f5f33f32640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2c0d1ea0 con 0x7f5f2c0cef40 2024-04-25T10:13:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.879+0000 7f5f33f32640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2c162320 con 0x7f5f2c1618d0 2024-04-25T10:13:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.879+0000 7f5f314cd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5f2c0cef40 0x7f5f2c0b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.879+0000 7f5f31cce640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5f2c1610b0 0x7f5f2c0ce830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41204/0 (socket says 172.21.15.16:41204) 2024-04-25T10:13:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.879+0000 7f5f31cce640 1 -- 172.21.15.16:0/2191823746 learned_addr learned my addr 172.21.15.16:0/2191823746 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:13:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.879+0000 7f5f324cf640 1 --2- 172.21.15.16:0/2191823746 >> v2:172.21.15.186:3300/0 conn(0x7f5f2c1618d0 0x7f5f2c0d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.880+0000 7f5f314cd640 1 -- 172.21.15.16:0/2191823746 >> v2:172.21.15.186:3300/0 conn(0x7f5f2c1618d0 msgr2=0x7f5f2c0d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.880+0000 7f5f314cd640 1 --2- 172.21.15.16:0/2191823746 >> v2:172.21.15.186:3300/0 conn(0x7f5f2c1618d0 0x7f5f2c0d2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.880+0000 7f5f314cd640 1 -- 172.21.15.16:0/2191823746 >> v1:172.21.15.16:6789/0 conn(0x7f5f2c1610b0 legacy=0x7f5f2c0ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.880+0000 7f5f30ccc640 1 -- 172.21.15.16:0/2191823746 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227738085 0 0) 0x7f5f2c162040 con 0x7f5f2c1610b0 2024-04-25T10:13:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.880+0000 7f5f314cd640 1 -- 172.21.15.16:0/2191823746 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c162600 con 0x7f5f2c0cef40 2024-04-25T10:13:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.880+0000 7f5f324cf640 1 --2- 172.21.15.16:0/2191823746 >> v2:172.21.15.186:3300/0 conn(0x7f5f2c1618d0 0x7f5f2c0d2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.880+0000 7f5f314cd640 1 --2- 172.21.15.16:0/2191823746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c0cef40 0x7f5f2c0b9580 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0b9190 tx=0x7f5f1c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746e86a4b6db1a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.880+0000 7f5f30ccc640 1 -- 172.21.15.16:0/2191823746 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c037070 con 0x7f5f2c0cef40 2024-04-25T10:13:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.880+0000 7f5f30ccc640 1 -- 172.21.15.16:0/2191823746 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c032440 con 0x7f5f2c0cef40 2024-04-25T10:13:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.880+0000 7f5f30ccc640 1 -- 172.21.15.16:0/2191823746 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c02ae60 con 0x7f5f2c0cef40 2024-04-25T10:13:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.881+0000 7f5f33f32640 1 -- 172.21.15.16:0/2191823746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c0cef40 msgr2=0x7f5f2c0b9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.881+0000 7f5f33f32640 1 --2- 172.21.15.16:0/2191823746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c0cef40 0x7f5f2c0b9580 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0b9190 tx=0x7f5f1c02a110 comp rx=0 tx=0).stop 2024-04-25T10:13:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.881+0000 7f5f33f32640 1 -- 172.21.15.16:0/2191823746 shutdown_connections 2024-04-25T10:13:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.881+0000 7f5f33f32640 1 --2- 172.21.15.16:0/2191823746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c0cef40 0x7f5f2c0b9580 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.881+0000 7f5f33f32640 1 --2- 172.21.15.16:0/2191823746 >> v2:172.21.15.186:3300/0 conn(0x7f5f2c1618d0 0x7f5f2c0d2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.881+0000 7f5f33f32640 1 -- 172.21.15.16:0/2191823746 >> 172.21.15.16:0/2191823746 conn(0x7f5f2c0b5570 msgr2=0x7f5f2c0b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.881+0000 7f5f33f32640 1 -- 172.21.15.16:0/2191823746 shutdown_connections 2024-04-25T10:13:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.881+0000 7f5f33f32640 1 -- 172.21.15.16:0/2191823746 wait complete. 2024-04-25T10:13:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.882+0000 7f5f33f32640 1 Processor -- start 2024-04-25T10:13:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.882+0000 7f5f33f32640 1 -- start start 2024-04-25T10:13:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.883+0000 7f5f33f32640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f2c0cef40 0x7f5f2c15bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.883+0000 7f5f33f32640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f2c1610b0 0x7f5f2c159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.883+0000 7f5f33f32640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c1618d0 0x7f5f2c06c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.883+0000 7f5f33f32640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c161d40 con 0x7f5f2c1618d0 2024-04-25T10:13:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.883+0000 7f5f33f32640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0b6c00 con 0x7f5f2c0cef40 2024-04-25T10:13:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.883+0000 7f5f33f32640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c161490 con 0x7f5f2c1610b0 2024-04-25T10:13:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.883+0000 7f5f324cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c1618d0 0x7f5f2c06c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.883+0000 7f5f324cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c1618d0 0x7f5f2c06c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33004/0 (socket says 172.21.15.16:33004) 2024-04-25T10:13:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.883+0000 7f5f324cf640 1 -- 172.21.15.16:0/3798111268 learned_addr learned my addr 172.21.15.16:0/3798111268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.883+0000 7f5f314cd640 1 --2- 172.21.15.16:0/3798111268 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f2c1610b0 0x7f5f2c159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.883+0000 7f5f31cce640 1 --2- 172.21.15.16:0/3798111268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f2c0cef40 0x7f5f2c15bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.883+0000 7f5f324cf640 1 -- 172.21.15.16:0/3798111268 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f2c1610b0 msgr2=0x7f5f2c159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.883+0000 7f5f324cf640 1 --2- 172.21.15.16:0/3798111268 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f2c1610b0 0x7f5f2c159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.883+0000 7f5f324cf640 1 -- 172.21.15.16:0/3798111268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f2c0cef40 msgr2=0x7f5f2c15bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.883+0000 7f5f324cf640 1 --2- 172.21.15.16:0/3798111268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f2c0cef40 0x7f5f2c15bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.883+0000 7f5f324cf640 1 -- 172.21.15.16:0/3798111268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c159e70 con 0x7f5f2c1618d0 2024-04-25T10:13:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.884+0000 7f5f324cf640 1 --2- 172.21.15.16:0/3798111268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c1618d0 0x7f5f2c06c220 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400cd50 tx=0x7f5f2400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.884+0000 7f5f227fc640 1 -- 172.21.15.16:0/3798111268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2400f680 con 0x7f5f2c1618d0 2024-04-25T10:13:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.884+0000 7f5f227fc640 1 -- 172.21.15.16:0/3798111268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2400f840 con 0x7f5f2c1618d0 2024-04-25T10:13:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.884+0000 7f5f227fc640 1 -- 172.21.15.16:0/3798111268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2400fc30 con 0x7f5f2c1618d0 2024-04-25T10:13:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.884+0000 7f5f33f32640 1 -- 172.21.15.16:0/3798111268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c15a110 con 0x7f5f2c1618d0 2024-04-25T10:13:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.884+0000 7f5f33f32640 1 -- 172.21.15.16:0/3798111268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c15a6d0 con 0x7f5f2c1618d0 2024-04-25T10:13:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.885+0000 7f5f227fc640 1 -- 172.21.15.16:0/3798111268 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f240213f0 con 0x7f5f2c1618d0 2024-04-25T10:13:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.886+0000 7f5f227fc640 1 --2- 172.21.15.16:0/3798111268 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5f1006b4d0 0x7f5f1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.886+0000 7f5f227fc640 1 -- 172.21.15.16:0/3798111268 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5f2408afb0 con 0x7f5f2c1618d0 2024-04-25T10:13:39.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.886+0000 7f5f31cce640 1 --2- 172.21.15.16:0/3798111268 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5f1006b4d0 0x7f5f1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.886+0000 7f5f31cce640 1 --2- 172.21.15.16:0/3798111268 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5f1006b4d0 0x7f5f1006d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5f28000f60 tx=0x7f5f28007b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.887+0000 7f5f36c31740 1 Processor -- start 2024-04-25T10:13:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.887+0000 7f5f36c31740 1 -- start start 2024-04-25T10:13:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.888+0000 7f5f36c31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a2ec230 0x563b4a2ec610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.888+0000 7f5f36c31740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a309ad0 0x563b4a30a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.888+0000 7f5f36c31740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a2c1f10 0x563b4a308190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.888+0000 7f5f36c31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4a224a10 con 0x563b4a2ec230 2024-04-25T10:13:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.888+0000 7f5f36c31740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4a1cdb30 con 0x563b4a2c1f10 2024-04-25T10:13:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.888+0000 7f5f36c31740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b4a220c50 con 0x563b4a309ad0 2024-04-25T10:13:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.888+0000 7f5f31cce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a2ec230 0x563b4a2ec610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.888+0000 7f5f31cce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a2ec230 0x563b4a2ec610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33012/0 (socket says 172.21.15.16:33012) 2024-04-25T10:13:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.888+0000 7f5f31cce640 1 -- 172.21.15.16:0/3699152373 learned_addr learned my addr 172.21.15.16:0/3699152373 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.888+0000 7f5f31cce640 1 -- 172.21.15.16:0/3699152373 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a309ad0 msgr2=0x563b4a30a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.888+0000 7f5f31cce640 1 --2- 172.21.15.16:0/3699152373 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a309ad0 0x563b4a30a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.888+0000 7f5f31cce640 1 -- 172.21.15.16:0/3699152373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a2c1f10 msgr2=0x563b4a308190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.888+0000 7f5f31cce640 1 --2- 172.21.15.16:0/3699152373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a2c1f10 0x563b4a308190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.888+0000 7f5f31cce640 1 -- 172.21.15.16:0/3699152373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b4a1e2e10 con 0x563b4a2ec230 2024-04-25T10:13:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.888+0000 7f5f31cce640 1 --2- 172.21.15.16:0/3699152373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a2ec230 0x563b4a2ec610 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f5f28005d80 tx=0x7f5f28012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24410ad9c3902d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.888+0000 7f5f0f7fe640 1 -- 172.21.15.16:0/3699152373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f280071a0 con 0x563b4a2ec230 2024-04-25T10:13:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.888+0000 7f5f0f7fe640 1 -- 172.21.15.16:0/3699152373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f28015110 con 0x563b4a2ec230 2024-04-25T10:13:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.888+0000 7f5f0f7fe640 1 -- 172.21.15.16:0/3699152373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f28015bc0 con 0x563b4a2ec230 2024-04-25T10:13:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.889+0000 7f5f36c31740 1 -- 172.21.15.16:0/3699152373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a2ec230 msgr2=0x563b4a2ec610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.889+0000 7f5f36c31740 1 --2- 172.21.15.16:0/3699152373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a2ec230 0x563b4a2ec610 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f5f28005d80 tx=0x7f5f28012000 comp rx=0 tx=0).stop 2024-04-25T10:13:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.889+0000 7f5f36c31740 1 -- 172.21.15.16:0/3699152373 shutdown_connections 2024-04-25T10:13:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.889+0000 7f5f36c31740 1 --2- 172.21.15.16:0/3699152373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a2c1f10 0x563b4a308190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.889+0000 7f5f36c31740 1 --2- 172.21.15.16:0/3699152373 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a309ad0 0x563b4a30a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.889+0000 7f5f36c31740 1 --2- 172.21.15.16:0/3699152373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a2ec230 0x563b4a2ec610 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.889+0000 7f5f36c31740 1 -- 172.21.15.16:0/3699152373 >> 172.21.15.16:0/3699152373 conn(0x563b4a2be840 msgr2=0x563b4a307040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.889+0000 7f5f36c31740 1 -- 172.21.15.16:0/3699152373 shutdown_connections 2024-04-25T10:13:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.889+0000 7f5f36c31740 1 -- 172.21.15.16:0/3699152373 wait complete. 2024-04-25T10:13:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.890+0000 7f5f36c31740 1 -- start start 2024-04-25T10:13:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.890+0000 7f5f36c31740 1 Processor -- start 2024-04-25T10:13:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.890+0000 7f5f36c31740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a2ec230 0x563b4a2e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f36c31740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a309ad0 0x563b4a2e9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f36c31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 0x563b4a308440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f36c31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4a2c2cb0 con 0x563b4a307fc0 2024-04-25T10:13:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f36c31740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4a1e10c0 con 0x563b4a2ec230 2024-04-25T10:13:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f36c31740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b4a2bc770 con 0x563b4a309ad0 2024-04-25T10:13:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f36c31740 10 client.0 _ll_register_callbacks cb 0x563b4a1d4ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f324cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 0x563b4a308440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f324cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 0x563b4a308440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33018/0 (socket says 172.21.15.16:33018) 2024-04-25T10:13:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f324cf640 1 -- 172.21.15.16:0/2872319911 learned_addr learned my addr 172.21.15.16:0/2872319911 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f31cce640 1 --2- 172.21.15.16:0/2872319911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a2ec230 0x563b4a2e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f314cd640 1 --2- 172.21.15.16:0/2872319911 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a309ad0 0x563b4a2e9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f324cf640 1 -- 172.21.15.16:0/2872319911 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a309ad0 msgr2=0x563b4a2e9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f324cf640 1 --2- 172.21.15.16:0/2872319911 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a309ad0 0x563b4a2e9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f324cf640 1 -- 172.21.15.16:0/2872319911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a2ec230 msgr2=0x563b4a2e7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f324cf640 1 --2- 172.21.15.16:0/2872319911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a2ec230 0x563b4a2e7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f324cf640 1 -- 172.21.15.16:0/2872319911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b49f392d0 con 0x563b4a307fc0 2024-04-25T10:13:39.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f324cf640 1 --2- 172.21.15.16:0/2872319911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 0x563b4a308440 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f5f24061940 tx=0x7f5f2405d8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aaee406d53ef30d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:13:39.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f0cff9640 1 -- 172.21.15.16:0/2872319911 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24071020 con 0x563b4a307fc0 2024-04-25T10:13:39.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f0cff9640 1 -- 172.21.15.16:0/2872319911 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5f240619e0 con 0x563b4a307fc0 2024-04-25T10:13:39.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f36c31740 1 -- 172.21.15.16:0/2872319911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f28015110 con 0x563b4a307fc0 2024-04-25T10:13:39.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f0cff9640 1 -- 172.21.15.16:0/2872319911 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f24089050 con 0x563b4a307fc0 2024-04-25T10:13:39.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f36c31740 10 client.8610 fetch_fsmap learned FSMap version 76 2024-04-25T10:13:39.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.891+0000 7f5f36c31740 1 -- 172.21.15.16:0/2872319911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x563b4a0036b0 con 0x563b4a307fc0 2024-04-25T10:13:39.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.892+0000 7f5f0cff9640 1 -- 172.21.15.16:0/2872319911 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f28015110 con 0x563b4a307fc0 2024-04-25T10:13:39.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.892+0000 7f5f0cff9640 1 -- 172.21.15.16:0/2872319911 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5f24058c60 con 0x563b4a307fc0 2024-04-25T10:13:39.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.892+0000 7f5f36c31740 10 client.8610 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T10:13:39.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.892+0000 7f5f36c31740 10 client.8610 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:13:39.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.892+0000 7f5f36c31740 20 client.8610 populate_metadata read hostname 'smithi016' 2024-04-25T10:13:39.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.892+0000 7f5f36c31740 1 --2- 172.21.15.16:0/2872319911 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x563b4a317a80 0x563b4a317e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.892+0000 7f5f36c31740 4 client.8610 mds_command: new command op to 36575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:39.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.892+0000 7f5f36c31740 1 -- 172.21.15.16:0/2872319911 --> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b4a283920 con 0x563b4a317a80 2024-04-25T10:13:39.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.892+0000 7f5f31cce640 1 --2- 172.21.15.16:0/2872319911 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x563b4a317a80 0x563b4a317e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.892+0000 7f5f0cff9640 10 client.8610 ms_handle_connect on v2:172.21.15.186:6834/2008057690 2024-04-25T10:13:39.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.892+0000 7f5f31cce640 1 --2- 172.21.15.16:0/2872319911 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x563b4a317a80 0x563b4a317e60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf1e85785d220f90 server_cookie=2184fc76bda2436f in_seq=0 out_seq=0 2024-04-25T10:13:39.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.893+0000 7f5f0cff9640 1 -- 172.21.15.16:0/2872319911 <== mds.0 v2:172.21.15.186:6834/2008057690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563b4a283920 con 0x563b4a317a80 2024-04-25T10:13:39.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.893+0000 7f5f0cff9640 10 client.8610 handle_command_reply: tid=0 2024-04-25T10:13:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.893+0000 7f5f36c31740 1 client.8610 shutdown 2024-04-25T10:13:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.893+0000 7f5f36c31740 1 -- 172.21.15.16:0/2872319911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 msgr2=0x563b4a308440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.893+0000 7f5f36c31740 1 --2- 172.21.15.16:0/2872319911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 0x563b4a308440 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f5f24061940 tx=0x7f5f2405d8b0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.893+0000 7f5f36c31740 1 -- 172.21.15.16:0/2872319911 shutdown_connections 2024-04-25T10:13:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.893+0000 7f5f36c31740 1 --2- 172.21.15.16:0/2872319911 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x563b4a317a80 0x563b4a317e60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.893+0000 7f5f36c31740 1 --2- 172.21.15.16:0/2872319911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 0x563b4a308440 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.893+0000 7f5f36c31740 1 --2- 172.21.15.16:0/2872319911 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a309ad0 0x563b4a2e9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.893+0000 7f5f36c31740 1 --2- 172.21.15.16:0/2872319911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a2ec230 0x563b4a2e7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.893+0000 7f5f0cff9640 0 client.8610 ms_handle_reset on v2:172.21.15.186:6834/2008057690 2024-04-25T10:13:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.893+0000 7f5f36c31740 1 -- 172.21.15.16:0/2872319911 >> 172.21.15.16:0/2872319911 conn(0x563b4a2c1f10 msgr2=0x563b4a30a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.893+0000 7f5f36c31740 1 -- 172.21.15.16:0/2872319911 shutdown_connections 2024-04-25T10:13:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.893+0000 7f5f36c31740 1 -- 172.21.15.16:0/2872319911 wait complete. 2024-04-25T10:13:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.894+0000 7f5f36c31740 20 client.8610 trim_cache size 0 max 16384 2024-04-25T10:13:39.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.909+0000 7f5f36c31740 1 Processor -- start 2024-04-25T10:13:39.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.909+0000 7f5f36c31740 1 -- start start 2024-04-25T10:13:39.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.910+0000 7f5f36c31740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a2ec230 0x563b4a2f0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.910+0000 7f5f36c31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 0x563b4a2c2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.910+0000 7f5f36c31740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a309ad0 0x563b4a2e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.910+0000 7f5f36c31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c1ffcf0 con 0x563b4a307fc0 2024-04-25T10:13:39.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.910+0000 7f5f36c31740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c1fefe0 con 0x563b4a2ec230 2024-04-25T10:13:39.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.910+0000 7f5f36c31740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0cd9c0 con 0x563b4a309ad0 2024-04-25T10:13:39.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.910+0000 7f5f314cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 0x563b4a2c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.910+0000 7f5f314cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 0x563b4a2c2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33020/0 (socket says 172.21.15.16:33020) 2024-04-25T10:13:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.910+0000 7f5f314cd640 1 -- 172.21.15.16:0/2764141145 learned_addr learned my addr 172.21.15.16:0/2764141145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.911+0000 7f5f314cd640 1 -- 172.21.15.16:0/2764141145 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a309ad0 msgr2=0x563b4a2e6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.911+0000 7f5f314cd640 1 --2- 172.21.15.16:0/2764141145 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a309ad0 0x563b4a2e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.911+0000 7f5f314cd640 1 -- 172.21.15.16:0/2764141145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a2ec230 msgr2=0x563b4a2f0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.911+0000 7f5f314cd640 1 --2- 172.21.15.16:0/2764141145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a2ec230 0x563b4a2f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.911+0000 7f5f314cd640 1 -- 172.21.15.16:0/2764141145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b4a308e10 con 0x563b4a307fc0 2024-04-25T10:13:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.911+0000 7f5f314cd640 1 --2- 172.21.15.16:0/2764141145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 0x563b4a2c2520 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009890 tx=0x7f5f1c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101f44cca93edacd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.911+0000 7f5f0effd640 1 -- 172.21.15.16:0/2764141145 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c037040 con 0x563b4a307fc0 2024-04-25T10:13:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.911+0000 7f5f0effd640 1 -- 172.21.15.16:0/2764141145 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c043070 con 0x563b4a307fc0 2024-04-25T10:13:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.911+0000 7f5f0effd640 1 -- 172.21.15.16:0/2764141145 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c032a90 con 0x563b4a307fc0 2024-04-25T10:13:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.911+0000 7f5f36c31740 1 -- 172.21.15.16:0/2764141145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 msgr2=0x563b4a2c2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.911+0000 7f5f36c31740 1 --2- 172.21.15.16:0/2764141145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 0x563b4a2c2520 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009890 tx=0x7f5f1c02c000 comp rx=0 tx=0).stop 2024-04-25T10:13:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.911+0000 7f5f36c31740 1 -- 172.21.15.16:0/2764141145 shutdown_connections 2024-04-25T10:13:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.911+0000 7f5f36c31740 1 --2- 172.21.15.16:0/2764141145 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a309ad0 0x563b4a2e6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.911+0000 7f5f36c31740 1 --2- 172.21.15.16:0/2764141145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 0x563b4a2c2520 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.911+0000 7f5f36c31740 1 --2- 172.21.15.16:0/2764141145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a2ec230 0x563b4a2f0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.911+0000 7f5f36c31740 1 -- 172.21.15.16:0/2764141145 >> 172.21.15.16:0/2764141145 conn(0x563b4a2c1f10 msgr2=0x563b4a317e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.911+0000 7f5f36c31740 1 -- 172.21.15.16:0/2764141145 shutdown_connections 2024-04-25T10:13:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.911+0000 7f5f36c31740 1 -- 172.21.15.16:0/2764141145 wait complete. 2024-04-25T10:13:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.912+0000 7f5f36c31740 1 -- start start 2024-04-25T10:13:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.912+0000 7f5f36c31740 1 Processor -- start 2024-04-25T10:13:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f36c31740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a2ec230 0x563b4a2f0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f36c31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 0x563b4a2bb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f36c31740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a309ad0 0x563b4a2bb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f36c31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4a31ad50 con 0x563b4a307fc0 2024-04-25T10:13:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f36c31740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c1f4a00 con 0x563b4a309ad0 2024-04-25T10:13:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f36c31740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b4a2ea210 con 0x563b4a2ec230 2024-04-25T10:13:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f36c31740 10 client.0 _ll_register_callbacks cb 0x563b4a2bcc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f314cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 0x563b4a2bb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f314cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 0x563b4a2bb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33024/0 (socket says 172.21.15.16:33024) 2024-04-25T10:13:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f314cd640 1 -- 172.21.15.16:0/3890605733 learned_addr learned my addr 172.21.15.16:0/3890605733 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f324cf640 1 --2- 172.21.15.16:0/3890605733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a309ad0 0x563b4a2bb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f31cce640 1 --2- 172.21.15.16:0/3890605733 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a2ec230 0x563b4a2f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f314cd640 1 -- 172.21.15.16:0/3890605733 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a2ec230 msgr2=0x563b4a2f0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f314cd640 1 --2- 172.21.15.16:0/3890605733 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a2ec230 0x563b4a2f0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f314cd640 1 -- 172.21.15.16:0/3890605733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a309ad0 msgr2=0x563b4a2bb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f314cd640 1 --2- 172.21.15.16:0/3890605733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a309ad0 0x563b4a2bb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f314cd640 1 -- 172.21.15.16:0/3890605733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b4a2e7e10 con 0x563b4a307fc0 2024-04-25T10:13:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f324cf640 1 --2- 172.21.15.16:0/3890605733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a309ad0 0x563b4a2bb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.913+0000 7f5f31cce640 1 --2- 172.21.15.16:0/3890605733 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a2ec230 0x563b4a2f0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.914+0000 7f5f314cd640 1 --2- 172.21.15.16:0/3890605733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 0x563b4a2bb4a0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c02a700 tx=0x7f5f1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55a06341d35c1f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.914+0000 7f5f0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:13:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.914+0000 7f5f0cff9640 1 -- 172.21.15.16:0/3890605733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c037040 con 0x563b4a307fc0 2024-04-25T10:13:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.914+0000 7f5f0cff9640 1 -- 172.21.15.16:0/3890605733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5f1c032a40 con 0x563b4a307fc0 2024-04-25T10:13:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.914+0000 7f5f36c31740 1 -- 172.21.15.16:0/3890605733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f1c043070 con 0x563b4a307fc0 2024-04-25T10:13:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.914+0000 7f5f0cff9640 1 -- 172.21.15.16:0/3890605733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f1c003610 con 0x563b4a307fc0 2024-04-25T10:13:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.914+0000 7f5f36c31740 10 client.8616 fetch_fsmap learned FSMap version 76 2024-04-25T10:13:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.914+0000 7f5f36c31740 1 -- 172.21.15.16:0/3890605733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x563b4a3752d0 con 0x563b4a307fc0 2024-04-25T10:13:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.914+0000 7f5f0cff9640 1 -- 172.21.15.16:0/3890605733 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c043070 con 0x563b4a307fc0 2024-04-25T10:13:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.914+0000 7f5f0cff9640 1 -- 172.21.15.16:0/3890605733 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5f1c056080 con 0x563b4a307fc0 2024-04-25T10:13:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.914+0000 7f5f36c31740 10 client.8616 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T10:13:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.914+0000 7f5f36c31740 10 client.8616 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:13:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.914+0000 7f5f36c31740 20 client.8616 populate_metadata read hostname 'smithi016' 2024-04-25T10:13:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.914+0000 7f5f36c31740 1 --2- 172.21.15.16:0/3890605733 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x563b4a3717d0 0x563b4a371bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.914+0000 7f5f36c31740 4 client.8616 mds_command: new command op to 36575 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:13:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.914+0000 7f5f36c31740 1 -- 172.21.15.16:0/3890605733 --> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563b4a231f50 con 0x563b4a3717d0 2024-04-25T10:13:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.914+0000 7f5f31cce640 1 --2- 172.21.15.16:0/3890605733 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x563b4a3717d0 0x563b4a371bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.915+0000 7f5f31cce640 1 --2- 172.21.15.16:0/3890605733 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x563b4a3717d0 0x563b4a371bb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ef7705bb2070704 server_cookie=b8bccc40031a952f in_seq=0 out_seq=0 2024-04-25T10:13:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.915+0000 7f5f0cff9640 10 client.8616 ms_handle_connect on v2:172.21.15.186:6834/2008057690 2024-04-25T10:13:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.915+0000 7f5f0cff9640 1 -- 172.21.15.16:0/3890605733 <== mds.0 v2:172.21.15.186:6834/2008057690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x563b4a231f50 con 0x563b4a3717d0 2024-04-25T10:13:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.915+0000 7f5f0cff9640 10 client.8616 handle_command_reply: tid=0 2024-04-25T10:13:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.915+0000 7f5f36c31740 1 client.8616 shutdown 2024-04-25T10:13:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.915+0000 7f5f36c31740 1 -- 172.21.15.16:0/3890605733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 msgr2=0x563b4a2bb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.915+0000 7f5f36c31740 1 --2- 172.21.15.16:0/3890605733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 0x563b4a2bb4a0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c02a700 tx=0x7f5f1c004730 comp rx=0 tx=0).stop 2024-04-25T10:13:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.915+0000 7f5f36c31740 1 -- 172.21.15.16:0/3890605733 shutdown_connections 2024-04-25T10:13:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.915+0000 7f5f36c31740 1 --2- 172.21.15.16:0/3890605733 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x563b4a3717d0 0x563b4a371bb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.915+0000 7f5f36c31740 1 --2- 172.21.15.16:0/3890605733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b4a309ad0 0x563b4a2bb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.915+0000 7f5f36c31740 1 --2- 172.21.15.16:0/3890605733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b4a307fc0 0x563b4a2bb4a0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.915+0000 7f5f36c31740 1 --2- 172.21.15.16:0/3890605733 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563b4a2ec230 0x563b4a2f0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.915+0000 7f5f0cff9640 0 client.8616 ms_handle_reset on v2:172.21.15.186:6834/2008057690 2024-04-25T10:13:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.915+0000 7f5f36c31740 1 -- 172.21.15.16:0/3890605733 >> 172.21.15.16:0/3890605733 conn(0x563b4a2c1f10 msgr2=0x563b4a2c2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.915+0000 7f5f36c31740 1 -- 172.21.15.16:0/3890605733 shutdown_connections 2024-04-25T10:13:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.915+0000 7f5f36c31740 1 -- 172.21.15.16:0/3890605733 wait complete. 2024-04-25T10:13:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.915+0000 7f5f36c31740 20 client.8616 trim_cache size 0 max 16384 2024-04-25T10:13:39.930 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:13:39.931 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "34bc5eca-3a21-42a6-a26f-07da0905e8bc", 2024-04-25T10:13:39.931 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-04-25T10:13:39.931 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 36575, 2024-04-25T10:13:39.931 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-04-25T10:13:39.931 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-04-25T10:13:39.931 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-04-25T10:13:39.931 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 4.4350643329999997, 2024-04-25T10:13:39.931 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 76, 2024-04-25T10:13:39.931 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 101, 2024-04-25T10:13:39.931 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 101, 2024-04-25T10:13:39.931 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 5.3724309640000003 2024-04-25T10:13:39.931 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:13:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.916+0000 7f5f33f32640 1 -- 172.21.15.16:0/3798111268 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5f1006b4d0 msgr2=0x7f5f1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.916+0000 7f5f33f32640 1 --2- 172.21.15.16:0/3798111268 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5f1006b4d0 0x7f5f1006d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5f28000f60 tx=0x7f5f28007b50 comp rx=0 tx=0).stop 2024-04-25T10:13:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.916+0000 7f5f33f32640 1 -- 172.21.15.16:0/3798111268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c1618d0 msgr2=0x7f5f2c06c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.916+0000 7f5f33f32640 1 --2- 172.21.15.16:0/3798111268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c1618d0 0x7f5f2c06c220 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400cd50 tx=0x7f5f2400a650 comp rx=0 tx=0).stop 2024-04-25T10:13:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.916+0000 7f5f33f32640 1 -- 172.21.15.16:0/3798111268 shutdown_connections 2024-04-25T10:13:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.916+0000 7f5f33f32640 1 --2- 172.21.15.16:0/3798111268 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5f1006b4d0 0x7f5f1006d990 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.916+0000 7f5f33f32640 1 --2- 172.21.15.16:0/3798111268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c1618d0 0x7f5f2c06c220 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.916+0000 7f5f33f32640 1 --2- 172.21.15.16:0/3798111268 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f2c1610b0 0x7f5f2c159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.916+0000 7f5f33f32640 1 --2- 172.21.15.16:0/3798111268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f2c0cef40 0x7f5f2c15bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.916+0000 7f5f33f32640 1 -- 172.21.15.16:0/3798111268 >> 172.21.15.16:0/3798111268 conn(0x7f5f2c0b5570 msgr2=0x7f5f2c15dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.916+0000 7f5f33f32640 1 -- 172.21.15.16:0/3798111268 shutdown_connections 2024-04-25T10:13:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:39.916+0000 7f5f33f32640 1 -- 172.21.15.16:0/3798111268 wait complete. 2024-04-25T10:13:39.934 DEBUG:teuthology.orchestra.run.smithi016:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T10:13:39.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:39.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-25T10:13:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.037+0000 7eff6e200640 1 Processor -- start 2024-04-25T10:13:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.037+0000 7eff6e200640 1 -- start start 2024-04-25T10:13:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.038+0000 7eff6e200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff6815ef60 0x7eff680ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.038+0000 7eff6e200640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff680b0e50 0x7eff680cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.038+0000 7eff6e200640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff6815f660 con 0x7eff680cea00 2024-04-25T10:13:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.038+0000 7eff6e200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6815f940 con 0x7eff6815ef60 2024-04-25T10:13:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.038+0000 7eff6e200640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6815fc20 con 0x7eff680b0e50 2024-04-25T10:13:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.038+0000 7eff677fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff6815ef60 0x7eff680ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.038+0000 7eff677fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff6815ef60 0x7eff680ce4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33040/0 (socket says 172.21.15.16:33040) 2024-04-25T10:13:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.038+0000 7eff677fe640 1 -- 172.21.15.16:0/2208220175 learned_addr learned my addr 172.21.15.16:0/2208220175 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.038+0000 7eff67fff640 1 --2- 172.21.15.16:0/2208220175 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff680b0e50 0x7eff680cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.038+0000 7eff677fe640 1 -- 172.21.15.16:0/2208220175 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff680b0e50 msgr2=0x7eff680cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.038+0000 7eff677fe640 1 --2- 172.21.15.16:0/2208220175 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff680b0e50 0x7eff680cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.038+0000 7eff677fe640 1 -- 172.21.15.16:0/2208220175 >> v1:172.21.15.186:6789/0 conn(0x7eff680cea00 legacy=0x7eff680b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.038+0000 7eff677fe640 1 -- 172.21.15.16:0/2208220175 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6815ff00 con 0x7eff6815ef60 2024-04-25T10:13:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.039+0000 7eff677fe640 1 --2- 172.21.15.16:0/2208220175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6815ef60 0x7eff680ce4c0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7eff58009870 tx=0x7eff5802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0136bb635c837f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.039+0000 7eff667fc640 1 -- 172.21.15.16:0/2208220175 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58037070 con 0x7eff6815ef60 2024-04-25T10:13:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.039+0000 7eff667fc640 1 -- 172.21.15.16:0/2208220175 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5802ae50 con 0x7eff6815ef60 2024-04-25T10:13:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.039+0000 7eff667fc640 1 -- 172.21.15.16:0/2208220175 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58032860 con 0x7eff6815ef60 2024-04-25T10:13:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.040+0000 7eff6e200640 1 -- 172.21.15.16:0/2208220175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6815ef60 msgr2=0x7eff680ce4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.040+0000 7eff6e200640 1 --2- 172.21.15.16:0/2208220175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6815ef60 0x7eff680ce4c0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7eff58009870 tx=0x7eff5802a300 comp rx=0 tx=0).stop 2024-04-25T10:13:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.040+0000 7eff6e200640 1 -- 172.21.15.16:0/2208220175 shutdown_connections 2024-04-25T10:13:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.040+0000 7eff6e200640 1 --2- 172.21.15.16:0/2208220175 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff680b0e50 0x7eff680cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.040+0000 7eff6e200640 1 --2- 172.21.15.16:0/2208220175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6815ef60 0x7eff680ce4c0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.040+0000 7eff6e200640 1 -- 172.21.15.16:0/2208220175 >> 172.21.15.16:0/2208220175 conn(0x7eff6806f4c0 msgr2=0x7eff6806f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.040+0000 7eff6e200640 1 -- 172.21.15.16:0/2208220175 shutdown_connections 2024-04-25T10:13:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.040+0000 7eff6e200640 1 -- 172.21.15.16:0/2208220175 wait complete. 2024-04-25T10:13:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.041+0000 7eff6e200640 1 Processor -- start 2024-04-25T10:13:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.041+0000 7eff6e200640 1 -- start start 2024-04-25T10:13:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.042+0000 7eff6e200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff680b0e50 0x7eff68157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.042+0000 7eff6e200640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff680cea00 0x7eff680c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.042+0000 7eff6e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6815ef60 0x7eff680b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.042+0000 7eff6e200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6815f340 con 0x7eff6815ef60 2024-04-25T10:13:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.042+0000 7eff6e200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff681604c0 con 0x7eff680b0e50 2024-04-25T10:13:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.042+0000 7eff6e200640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff680d02a0 con 0x7eff680cea00 2024-04-25T10:13:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.042+0000 7eff67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6815ef60 0x7eff680b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.042+0000 7eff67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6815ef60 0x7eff680b4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33042/0 (socket says 172.21.15.16:33042) 2024-04-25T10:13:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.042+0000 7eff67fff640 1 -- 172.21.15.16:0/590162283 learned_addr learned my addr 172.21.15.16:0/590162283 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.043+0000 7eff66ffd640 1 --2- 172.21.15.16:0/590162283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff680cea00 0x7eff680c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.043+0000 7eff677fe640 1 --2- 172.21.15.16:0/590162283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff680b0e50 0x7eff68157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.043+0000 7eff67fff640 1 -- 172.21.15.16:0/590162283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff680cea00 msgr2=0x7eff680c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.043+0000 7eff67fff640 1 --2- 172.21.15.16:0/590162283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff680cea00 0x7eff680c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.043+0000 7eff67fff640 1 -- 172.21.15.16:0/590162283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff680b0e50 msgr2=0x7eff68157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.043+0000 7eff67fff640 1 --2- 172.21.15.16:0/590162283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff680b0e50 0x7eff68157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.043+0000 7eff67fff640 1 -- 172.21.15.16:0/590162283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff680c7c20 con 0x7eff6815ef60 2024-04-25T10:13:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.043+0000 7eff677fe640 1 --2- 172.21.15.16:0/590162283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff680b0e50 0x7eff68157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.043+0000 7eff66ffd640 1 --2- 172.21.15.16:0/590162283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff680cea00 0x7eff680c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.043+0000 7eff67fff640 1 --2- 172.21.15.16:0/590162283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6815ef60 0x7eff680b4dc0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7eff6000efc0 tx=0x7eff6000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.043+0000 7eff5ffff640 1 -- 172.21.15.16:0/590162283 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60018020 con 0x7eff6815ef60 2024-04-25T10:13:40.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.043+0000 7eff5ffff640 1 -- 172.21.15.16:0/590162283 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff60014480 con 0x7eff6815ef60 2024-04-25T10:13:40.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.043+0000 7eff6e200640 1 -- 172.21.15.16:0/590162283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff680c7f00 con 0x7eff6815ef60 2024-04-25T10:13:40.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.043+0000 7eff5ffff640 1 -- 172.21.15.16:0/590162283 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60014870 con 0x7eff6815ef60 2024-04-25T10:13:40.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.043+0000 7eff6e200640 1 -- 172.21.15.16:0/590162283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff68159f20 con 0x7eff6815ef60 2024-04-25T10:13:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.045+0000 7eff5ffff640 1 -- 172.21.15.16:0/590162283 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff60005d10 con 0x7eff6815ef60 2024-04-25T10:13:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.045+0000 7eff5ffff640 1 --2- 172.21.15.16:0/590162283 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7eff4806b500 0x7eff4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.045+0000 7eff5ffff640 1 -- 172.21.15.16:0/590162283 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x7eff6008ad10 con 0x7eff6815ef60 2024-04-25T10:13:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.046+0000 7eff677fe640 1 --2- 172.21.15.16:0/590162283 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7eff4806b500 0x7eff4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.046+0000 7eff677fe640 1 --2- 172.21.15.16:0/590162283 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7eff4806b500 0x7eff4806d9c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7eff58034000 tx=0x7eff5802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.046+0000 7eff70fdd740 1 Processor -- start 2024-04-25T10:13:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.046+0000 7eff70fdd740 1 -- start start 2024-04-25T10:13:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.047+0000 7eff70fdd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128516bb0 0x55e128516f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.047+0000 7eff70fdd740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128511830 0x55e1285157b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.047+0000 7eff70fdd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128515cf0 0x55e128516130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.047+0000 7eff70fdd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12847aac0 con 0x55e128516bb0 2024-04-25T10:13:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.047+0000 7eff70fdd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e128423d80 con 0x55e128515cf0 2024-04-25T10:13:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.047+0000 7eff70fdd740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e128476b60 con 0x55e128511830 2024-04-25T10:13:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.047+0000 7eff677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128516bb0 0x55e128516f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.047+0000 7eff677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128516bb0 0x55e128516f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33058/0 (socket says 172.21.15.16:33058) 2024-04-25T10:13:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.047+0000 7eff677fe640 1 -- 172.21.15.16:0/819256435 learned_addr learned my addr 172.21.15.16:0/819256435 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.047+0000 7eff66ffd640 1 --2- 172.21.15.16:0/819256435 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128511830 0x55e1285157b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.047+0000 7eff67fff640 1 --2- 172.21.15.16:0/819256435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128515cf0 0x55e128516130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.047+0000 7eff677fe640 1 -- 172.21.15.16:0/819256435 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128511830 msgr2=0x55e1285157b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.047+0000 7eff677fe640 1 --2- 172.21.15.16:0/819256435 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128511830 0x55e1285157b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.047+0000 7eff677fe640 1 -- 172.21.15.16:0/819256435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128515cf0 msgr2=0x55e128516130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.047+0000 7eff677fe640 1 --2- 172.21.15.16:0/819256435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128515cf0 0x55e128516130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.047+0000 7eff677fe640 1 -- 172.21.15.16:0/819256435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1282577e0 con 0x55e128516bb0 2024-04-25T10:13:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.048+0000 7eff677fe640 1 --2- 172.21.15.16:0/819256435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128516bb0 0x55e128516f90 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7eff58035040 tx=0x7eff5802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89200afc88e4bab0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.048+0000 7eff5d7fa640 1 -- 172.21.15.16:0/819256435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58037040 con 0x55e128516bb0 2024-04-25T10:13:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.048+0000 7eff5d7fa640 1 -- 172.21.15.16:0/819256435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5802ae50 con 0x55e128516bb0 2024-04-25T10:13:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.048+0000 7eff5d7fa640 1 -- 172.21.15.16:0/819256435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58004030 con 0x55e128516bb0 2024-04-25T10:13:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.049+0000 7eff70fdd740 1 -- 172.21.15.16:0/819256435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128516bb0 msgr2=0x55e128516f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.049+0000 7eff70fdd740 1 --2- 172.21.15.16:0/819256435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128516bb0 0x55e128516f90 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7eff58035040 tx=0x7eff5802c000 comp rx=0 tx=0).stop 2024-04-25T10:13:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.049+0000 7eff70fdd740 1 -- 172.21.15.16:0/819256435 shutdown_connections 2024-04-25T10:13:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.049+0000 7eff70fdd740 1 --2- 172.21.15.16:0/819256435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128515cf0 0x55e128516130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.049+0000 7eff70fdd740 1 --2- 172.21.15.16:0/819256435 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128511830 0x55e1285157b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.049+0000 7eff70fdd740 1 --2- 172.21.15.16:0/819256435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128516bb0 0x55e128516f90 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.049+0000 7eff70fdd740 1 -- 172.21.15.16:0/819256435 >> 172.21.15.16:0/819256435 conn(0x55e128513cb0 msgr2=0x55e12855c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.049+0000 7eff70fdd740 1 -- 172.21.15.16:0/819256435 shutdown_connections 2024-04-25T10:13:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.049+0000 7eff70fdd740 1 -- 172.21.15.16:0/819256435 wait complete. 2024-04-25T10:13:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.049+0000 7eff70fdd740 1 -- start start 2024-04-25T10:13:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.049+0000 7eff70fdd740 1 Processor -- start 2024-04-25T10:13:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.050+0000 7eff70fdd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128515cf0 0x55e128562de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.050+0000 7eff70fdd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128516bb0 0x55e128536ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.050+0000 7eff70fdd740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128540e10 0x55e128537400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.050+0000 7eff70fdd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e128542740 con 0x55e128516bb0 2024-04-25T10:13:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.050+0000 7eff70fdd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e128419530 con 0x55e128515cf0 2024-04-25T10:13:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.050+0000 7eff70fdd740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e128541bc0 con 0x55e128540e10 2024-04-25T10:13:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.050+0000 7eff70fdd740 10 client.0 _ll_register_callbacks cb 0x55e128471130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.050+0000 7eff66ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128516bb0 0x55e128536ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.050+0000 7eff66ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128516bb0 0x55e128536ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33068/0 (socket says 172.21.15.16:33068) 2024-04-25T10:13:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.050+0000 7eff66ffd640 1 -- 172.21.15.16:0/2136525828 learned_addr learned my addr 172.21.15.16:0/2136525828 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.050+0000 7eff677fe640 1 --2- 172.21.15.16:0/2136525828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128515cf0 0x55e128562de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.050+0000 7eff67fff640 1 --2- 172.21.15.16:0/2136525828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128540e10 0x55e128537400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff66ffd640 1 -- 172.21.15.16:0/2136525828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128540e10 msgr2=0x55e128537400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff66ffd640 1 --2- 172.21.15.16:0/2136525828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128540e10 0x55e128537400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff66ffd640 1 -- 172.21.15.16:0/2136525828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128515cf0 msgr2=0x55e128562de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff66ffd640 1 --2- 172.21.15.16:0/2136525828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128515cf0 0x55e128562de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff66ffd640 1 -- 172.21.15.16:0/2136525828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1282b7f70 con 0x55e128516bb0 2024-04-25T10:13:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff66ffd640 1 --2- 172.21.15.16:0/2136525828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128516bb0 0x55e128536ec0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7eff500104d0 tx=0x7eff50010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c4b7c5c2ea33bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff467fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:13:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff467fc640 1 -- 172.21.15.16:0/2136525828 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff500052d0 con 0x55e128516bb0 2024-04-25T10:13:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff467fc640 1 -- 172.21.15.16:0/2136525828 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x7eff50014710 con 0x55e128516bb0 2024-04-25T10:13:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff70fdd740 1 -- 172.21.15.16:0/2136525828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff5802ae50 con 0x55e128516bb0 2024-04-25T10:13:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff467fc640 1 -- 172.21.15.16:0/2136525828 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff50021050 con 0x55e128516bb0 2024-04-25T10:13:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff70fdd740 10 client.8628 fetch_fsmap learned FSMap version 76 2024-04-25T10:13:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff70fdd740 1 -- 172.21.15.16:0/2136525828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55e12814f170 con 0x55e128516bb0 2024-04-25T10:13:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff467fc640 1 -- 172.21.15.16:0/2136525828 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5802ae50 con 0x55e128516bb0 2024-04-25T10:13:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff467fc640 1 -- 172.21.15.16:0/2136525828 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7eff50024020 con 0x55e128516bb0 2024-04-25T10:13:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff70fdd740 10 client.8628 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T10:13:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff70fdd740 10 client.8628 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:13:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff70fdd740 20 client.8628 populate_metadata read hostname 'smithi016' 2024-04-25T10:13:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff70fdd740 1 --2- 172.21.15.16:0/2136525828 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x55e12856d070 0x55e12856d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff70fdd740 4 client.8628 mds_command: new command op to 36575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.051+0000 7eff70fdd740 1 -- 172.21.15.16:0/2136525828 --> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e128488000 con 0x55e12856d070 2024-04-25T10:13:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.052+0000 7eff677fe640 1 --2- 172.21.15.16:0/2136525828 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x55e12856d070 0x55e12856d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.052+0000 7eff677fe640 1 --2- 172.21.15.16:0/2136525828 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x55e12856d070 0x55e12856d4d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=882a20376aac0dc2 server_cookie=179863a9cd0ff93 in_seq=0 out_seq=0 2024-04-25T10:13:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.052+0000 7eff467fc640 10 client.8628 ms_handle_connect on v2:172.21.15.186:6834/2008057690 2024-04-25T10:13:40.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.053+0000 7eff467fc640 1 -- 172.21.15.16:0/2136525828 <== mds.0 v2:172.21.15.186:6834/2008057690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e128488000 con 0x55e12856d070 2024-04-25T10:13:40.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.053+0000 7eff467fc640 10 client.8628 handle_command_reply: tid=0 2024-04-25T10:13:40.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.053+0000 7eff70fdd740 1 client.8628 shutdown 2024-04-25T10:13:40.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.053+0000 7eff70fdd740 1 -- 172.21.15.16:0/2136525828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128516bb0 msgr2=0x55e128536ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.053+0000 7eff70fdd740 1 --2- 172.21.15.16:0/2136525828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128516bb0 0x55e128536ec0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7eff500104d0 tx=0x7eff50010990 comp rx=0 tx=0).stop 2024-04-25T10:13:40.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.053+0000 7eff70fdd740 1 -- 172.21.15.16:0/2136525828 shutdown_connections 2024-04-25T10:13:40.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.053+0000 7eff70fdd740 1 --2- 172.21.15.16:0/2136525828 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x55e12856d070 0x55e12856d4d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.053+0000 7eff467fc640 0 client.8628 ms_handle_reset on v2:172.21.15.186:6834/2008057690 2024-04-25T10:13:40.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.053+0000 7eff70fdd740 1 --2- 172.21.15.16:0/2136525828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128540e10 0x55e128537400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.053+0000 7eff70fdd740 1 --2- 172.21.15.16:0/2136525828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128516bb0 0x55e128536ec0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.053+0000 7eff70fdd740 1 --2- 172.21.15.16:0/2136525828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128515cf0 0x55e128562de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.053+0000 7eff70fdd740 1 -- 172.21.15.16:0/2136525828 >> 172.21.15.16:0/2136525828 conn(0x55e128511830 msgr2=0x55e1285176a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.053+0000 7eff70fdd740 1 -- 172.21.15.16:0/2136525828 shutdown_connections 2024-04-25T10:13:40.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.053+0000 7eff70fdd740 1 -- 172.21.15.16:0/2136525828 wait complete. 2024-04-25T10:13:40.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.053+0000 7eff70fdd740 20 client.8628 trim_cache size 0 max 16384 2024-04-25T10:13:40.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.057+0000 7f5888a5d640 1 Processor -- start 2024-04-25T10:13:40.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.057+0000 7f5888a5d640 1 -- start start 2024-04-25T10:13:40.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.058+0000 7f5888a5d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f588415dca0 0x7f58840cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.058+0000 7f5888a5d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5884162200 0x7f588406cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.058+0000 7f5888a5d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588415f090 con 0x7f58840b54a0 2024-04-25T10:13:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.058+0000 7f5888a5d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588406d1c0 con 0x7f588415dca0 2024-04-25T10:13:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.058+0000 7f5888a5d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58840cf340 con 0x7f5884162200 2024-04-25T10:13:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.058+0000 7f5882d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f58840b54a0 0x7f58840d1c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41220/0 (socket says 172.21.15.16:41220) 2024-04-25T10:13:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.058+0000 7f5882d9d640 1 -- 172.21.15.16:0/1563734218 learned_addr learned my addr 172.21.15.16:0/1563734218 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:13:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.058+0000 7f5881d9b640 1 --2- 172.21.15.16:0/1563734218 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5884162200 0x7f588406cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.058+0000 7f588259c640 1 --2- 172.21.15.16:0/1563734218 >> v2:172.21.15.186:3300/0 conn(0x7f588415dca0 0x7f58840cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.059+0000 7f588159a640 1 -- 172.21.15.16:0/1563734218 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358637922 0 0) 0x7f588415f090 con 0x7f58840b54a0 2024-04-25T10:13:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.059+0000 7f588159a640 1 -- 172.21.15.16:0/1563734218 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f586c003520 con 0x7f58840b54a0 2024-04-25T10:13:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.059+0000 7f588259c640 1 -- 172.21.15.16:0/1563734218 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5884162200 msgr2=0x7f588406cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.059+0000 7f588259c640 1 --2- 172.21.15.16:0/1563734218 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5884162200 0x7f588406cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.059+0000 7f588259c640 1 -- 172.21.15.16:0/1563734218 >> v1:172.21.15.16:6789/0 conn(0x7f58840b54a0 legacy=0x7f58840d1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.059+0000 7f588259c640 1 -- 172.21.15.16:0/1563734218 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58840cf060 con 0x7f588415dca0 2024-04-25T10:13:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.060+0000 7f588259c640 1 --2- 172.21.15.16:0/1563734218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f588415dca0 0x7f58840cea80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f58740097a0 tx=0x7f587402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d844719d09ef9cb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.060+0000 7f588159a640 1 -- 172.21.15.16:0/1563734218 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874037070 con 0x7f588415dca0 2024-04-25T10:13:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.060+0000 7f588159a640 1 -- 172.21.15.16:0/1563734218 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587402ae50 con 0x7f588415dca0 2024-04-25T10:13:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.060+0000 7f588159a640 1 -- 172.21.15.16:0/1563734218 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874032640 con 0x7f588415dca0 2024-04-25T10:13:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.060+0000 7f5888a5d640 1 -- 172.21.15.16:0/1563734218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f588415dca0 msgr2=0x7f58840cea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.060+0000 7f5888a5d640 1 --2- 172.21.15.16:0/1563734218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f588415dca0 0x7f58840cea80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f58740097a0 tx=0x7f587402a100 comp rx=0 tx=0).stop 2024-04-25T10:13:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.061+0000 7f5888a5d640 1 -- 172.21.15.16:0/1563734218 shutdown_connections 2024-04-25T10:13:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.061+0000 7f5888a5d640 1 --2- 172.21.15.16:0/1563734218 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5884162200 0x7f588406cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.061+0000 7f5888a5d640 1 --2- 172.21.15.16:0/1563734218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f588415dca0 0x7f58840cea80 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.061+0000 7f5888a5d640 1 -- 172.21.15.16:0/1563734218 >> 172.21.15.16:0/1563734218 conn(0x7f58840d04a0 msgr2=0x7f58840d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.061+0000 7f5888a5d640 1 -- 172.21.15.16:0/1563734218 shutdown_connections 2024-04-25T10:13:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.061+0000 7f5888a5d640 1 -- 172.21.15.16:0/1563734218 wait complete. 2024-04-25T10:13:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.062+0000 7f5888a5d640 1 Processor -- start 2024-04-25T10:13:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.062+0000 7f5888a5d640 1 -- start start 2024-04-25T10:13:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.063+0000 7f5888a5d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58840b54a0 0x7f58841f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.063+0000 7f5888a5d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f588415dca0 0x7f58841f6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.063+0000 7f5888a5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5884162200 0x7f58841f3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.063+0000 7f5888a5d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588406c480 con 0x7f5884162200 2024-04-25T10:13:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.063+0000 7f5888a5d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588406c1a0 con 0x7f58840b54a0 2024-04-25T10:13:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.063+0000 7f5888a5d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588406c760 con 0x7f588415dca0 2024-04-25T10:13:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.063+0000 7f5882d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5884162200 0x7f58841f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.063+0000 7f5882d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5884162200 0x7f58841f3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33072/0 (socket says 172.21.15.16:33072) 2024-04-25T10:13:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.063+0000 7f5882d9d640 1 -- 172.21.15.16:0/87647012 learned_addr learned my addr 172.21.15.16:0/87647012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.063+0000 7f588259c640 1 --2- 172.21.15.16:0/87647012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58840b54a0 0x7f58841f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.063+0000 7f5881d9b640 1 --2- 172.21.15.16:0/87647012 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f588415dca0 0x7f58841f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.063+0000 7f5882d9d640 1 -- 172.21.15.16:0/87647012 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f588415dca0 msgr2=0x7f58841f6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.063+0000 7f5882d9d640 1 --2- 172.21.15.16:0/87647012 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f588415dca0 0x7f58841f6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.063+0000 7f5882d9d640 1 -- 172.21.15.16:0/87647012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58840b54a0 msgr2=0x7f58841f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.063+0000 7f5882d9d640 1 --2- 172.21.15.16:0/87647012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58840b54a0 0x7f58841f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.063+0000 7f5882d9d640 1 -- 172.21.15.16:0/87647012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58840aa390 con 0x7f5884162200 2024-04-25T10:13:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.063+0000 7f5882d9d640 1 --2- 172.21.15.16:0/87647012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5884162200 0x7f58841f3920 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f587c00caa0 tx=0x7f587c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.064+0000 7f5872ffd640 1 -- 172.21.15.16:0/87647012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c017070 con 0x7f5884162200 2024-04-25T10:13:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.064+0000 7f5872ffd640 1 -- 172.21.15.16:0/87647012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c00ac00 con 0x7f5884162200 2024-04-25T10:13:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.064+0000 7f5888a5d640 1 -- 172.21.15.16:0/87647012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58841f41e0 con 0x7f5884162200 2024-04-25T10:13:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.064+0000 7f5872ffd640 1 -- 172.21.15.16:0/87647012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c012400 con 0x7f5884162200 2024-04-25T10:13:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.064+0000 7f5888a5d640 1 -- 172.21.15.16:0/87647012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58841f47a0 con 0x7f5884162200 2024-04-25T10:13:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.064+0000 7f5888a5d640 1 -- 172.21.15.16:0/87647012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f58840749b0 con 0x7f5884162200 2024-04-25T10:13:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.065+0000 7f5872ffd640 1 -- 172.21.15.16:0/87647012 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f587c01a070 con 0x7f5884162200 2024-04-25T10:13:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.066+0000 7f5872ffd640 1 --2- 172.21.15.16:0/87647012 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f586406b4d0 0x7f586406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.066+0000 7f5872ffd640 1 -- 172.21.15.16:0/87647012 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f587c08a760 con 0x7f5884162200 2024-04-25T10:13:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.066+0000 7f5872ffd640 1 -- 172.21.15.16:0/87647012 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+512 (secure 0 0 0) 0x7f587c021070 con 0x7f5884162200 2024-04-25T10:13:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.066+0000 7f588259c640 1 --2- 172.21.15.16:0/87647012 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f586406b4d0 0x7f586406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.066+0000 7f5888a5d640 1 -- 172.21.15.16:0/87647012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f58840d1810 con 0x7f5884162200 2024-04-25T10:13:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.066+0000 7f588259c640 1 --2- 172.21.15.16:0/87647012 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f586406b4d0 0x7f586406d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5874034000 tx=0x7f587402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.084 INFO:tasks.ceph.mds.b.smithi186.stderr:2024-04-25T10:13:40.076+0000 7ff20b1cf640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-25T10:13:40.084 INFO:tasks.ceph.mds.b.smithi186.stderr:2024-04-25T10:13:40.076+0000 7ff20b1cf640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-25T10:13:40.084 INFO:tasks.ceph.mds.b.smithi186.stderr:2024-04-25T10:13:40.076+0000 7ff20b1cf640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-25T10:13:40.084 INFO:tasks.ceph.mds.b.smithi186.stderr:2024-04-25T10:13:40.076+0000 7ff20b1cf640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-25T10:13:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.070+0000 7eff70fdd740 1 Processor -- start 2024-04-25T10:13:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.070+0000 7eff70fdd740 1 -- start start 2024-04-25T10:13:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff70fdd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128515cf0 0x55e128516f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff70fdd740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128516bb0 0x55e1285174d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff70fdd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128540e10 0x55e1285382c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff70fdd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff681fdd90 con 0x55e128515cf0 2024-04-25T10:13:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff70fdd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff681fcf70 con 0x55e128540e10 2024-04-25T10:13:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff70fdd740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6815b130 con 0x55e128516bb0 2024-04-25T10:13:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128515cf0 0x55e128516f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128515cf0 0x55e128516f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33084/0 (socket says 172.21.15.16:33084) 2024-04-25T10:13:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff677fe640 1 -- 172.21.15.16:0/3684298223 learned_addr learned my addr 172.21.15.16:0/3684298223 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff66ffd640 1 --2- 172.21.15.16:0/3684298223 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128516bb0 0x55e1285174d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff67fff640 1 --2- 172.21.15.16:0/3684298223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128540e10 0x55e1285382c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff677fe640 1 -- 172.21.15.16:0/3684298223 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128516bb0 msgr2=0x55e1285174d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff677fe640 1 --2- 172.21.15.16:0/3684298223 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128516bb0 0x55e1285174d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff677fe640 1 -- 172.21.15.16:0/3684298223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128540e10 msgr2=0x55e1285382c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff677fe640 1 --2- 172.21.15.16:0/3684298223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128540e10 0x55e1285382c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff677fe640 1 -- 172.21.15.16:0/3684298223 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e12853c2f0 con 0x55e128515cf0 2024-04-25T10:13:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff677fe640 1 --2- 172.21.15.16:0/3684298223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128515cf0 0x55e128516f90 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7eff5802a8f0 tx=0x7eff5803a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938dedd647bde667 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff5cff9640 1 -- 172.21.15.16:0/3684298223 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58037040 con 0x55e128515cf0 2024-04-25T10:13:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff5cff9640 1 -- 172.21.15.16:0/3684298223 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff58004310 con 0x55e128515cf0 2024-04-25T10:13:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.071+0000 7eff5cff9640 1 -- 172.21.15.16:0/3684298223 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58056320 con 0x55e128515cf0 2024-04-25T10:13:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.072+0000 7eff70fdd740 1 -- 172.21.15.16:0/3684298223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128515cf0 msgr2=0x55e128516f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.072+0000 7eff70fdd740 1 --2- 172.21.15.16:0/3684298223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128515cf0 0x55e128516f90 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7eff5802a8f0 tx=0x7eff5803a000 comp rx=0 tx=0).stop 2024-04-25T10:13:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.072+0000 7eff70fdd740 1 -- 172.21.15.16:0/3684298223 shutdown_connections 2024-04-25T10:13:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.072+0000 7eff70fdd740 1 --2- 172.21.15.16:0/3684298223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128540e10 0x55e1285382c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.072+0000 7eff70fdd740 1 --2- 172.21.15.16:0/3684298223 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128516bb0 0x55e1285174d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.072+0000 7eff70fdd740 1 --2- 172.21.15.16:0/3684298223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128515cf0 0x55e128516f90 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.072+0000 7eff70fdd740 1 -- 172.21.15.16:0/3684298223 >> 172.21.15.16:0/3684298223 conn(0x55e128511830 msgr2=0x55e128541340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.072+0000 7eff70fdd740 1 -- 172.21.15.16:0/3684298223 shutdown_connections 2024-04-25T10:13:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.072+0000 7eff70fdd740 1 -- 172.21.15.16:0/3684298223 wait complete. 2024-04-25T10:13:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.073+0000 7eff70fdd740 1 -- start start 2024-04-25T10:13:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.073+0000 7eff70fdd740 1 Processor -- start 2024-04-25T10:13:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff70fdd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128515cf0 0x55e12856bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff70fdd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128516bb0 0x55e1285cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff70fdd740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128540e10 0x55e1285382c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff70fdd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e128537b60 con 0x55e128515cf0 2024-04-25T10:13:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff70fdd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6815b730 con 0x55e128516bb0 2024-04-25T10:13:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff70fdd740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e1285390c0 con 0x55e128540e10 2024-04-25T10:13:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff70fdd740 10 client.0 _ll_register_callbacks cb 0x7eff58035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128515cf0 0x55e12856bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128515cf0 0x55e12856bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33100/0 (socket says 172.21.15.16:33100) 2024-04-25T10:13:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff677fe640 1 -- 172.21.15.16:0/1113306093 learned_addr learned my addr 172.21.15.16:0/1113306093 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff67fff640 1 --2- 172.21.15.16:0/1113306093 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128540e10 0x55e1285382c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff66ffd640 1 --2- 172.21.15.16:0/1113306093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128516bb0 0x55e1285cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff677fe640 1 -- 172.21.15.16:0/1113306093 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128540e10 msgr2=0x55e1285382c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff677fe640 1 --2- 172.21.15.16:0/1113306093 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128540e10 0x55e1285382c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff677fe640 1 -- 172.21.15.16:0/1113306093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128516bb0 msgr2=0x55e1285cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff677fe640 1 --2- 172.21.15.16:0/1113306093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128516bb0 0x55e1285cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff677fe640 1 -- 172.21.15.16:0/1113306093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e12856ae00 con 0x55e128515cf0 2024-04-25T10:13:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff677fe640 1 --2- 172.21.15.16:0/1113306093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128515cf0 0x55e12856bea0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7eff5802a340 tx=0x7eff58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed8692dc863b1d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff467fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:13:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff467fc640 1 -- 172.21.15.16:0/1113306093 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58065060 con 0x55e128515cf0 2024-04-25T10:13:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff467fc640 1 -- 172.21.15.16:0/1113306093 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x7eff58056120 con 0x55e128515cf0 2024-04-25T10:13:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff70fdd740 1 -- 172.21.15.16:0/1113306093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff58004310 con 0x55e128515cf0 2024-04-25T10:13:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff467fc640 1 -- 172.21.15.16:0/1113306093 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff58037040 con 0x55e128515cf0 2024-04-25T10:13:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.074+0000 7eff70fdd740 10 client.8640 fetch_fsmap learned FSMap version 76 2024-04-25T10:13:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.075+0000 7eff70fdd740 1 -- 172.21.15.16:0/1113306093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55e128538ec0 con 0x55e128515cf0 2024-04-25T10:13:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.075+0000 7eff467fc640 1 -- 172.21.15.16:0/1113306093 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff58004310 con 0x55e128515cf0 2024-04-25T10:13:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.075+0000 7eff467fc640 1 -- 172.21.15.16:0/1113306093 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7eff58056320 con 0x55e128515cf0 2024-04-25T10:13:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.075+0000 7eff70fdd740 10 client.8640 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T10:13:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.075+0000 7eff70fdd740 10 client.8640 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:13:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.075+0000 7eff70fdd740 20 client.8640 populate_metadata read hostname 'smithi016' 2024-04-25T10:13:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.075+0000 7eff70fdd740 1 --2- 172.21.15.16:0/1113306093 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x55e12856d070 0x55e128538800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.075+0000 7eff70fdd740 4 client.8640 mds_command: new command op to 36575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:13:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.075+0000 7eff70fdd740 1 -- 172.21.15.16:0/1113306093 --> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55e128482350 con 0x55e12856d070 2024-04-25T10:13:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.075+0000 7eff66ffd640 1 --2- 172.21.15.16:0/1113306093 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x55e12856d070 0x55e128538800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.075+0000 7eff467fc640 10 client.8640 ms_handle_connect on v2:172.21.15.186:6834/2008057690 2024-04-25T10:13:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.075+0000 7eff66ffd640 1 --2- 172.21.15.16:0/1113306093 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x55e12856d070 0x55e128538800 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ba2195d0fd7a288 server_cookie=7b7cae677f4b2754 in_seq=0 out_seq=0 2024-04-25T10:13:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.076+0000 7eff467fc640 1 -- 172.21.15.16:0/1113306093 <== mds.0 v2:172.21.15.186:6834/2008057690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e128482350 con 0x55e12856d070 2024-04-25T10:13:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.076+0000 7eff467fc640 10 client.8640 handle_command_reply: tid=0 2024-04-25T10:13:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.076+0000 7eff70fdd740 1 client.8640 shutdown 2024-04-25T10:13:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.076+0000 7eff70fdd740 1 -- 172.21.15.16:0/1113306093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128515cf0 msgr2=0x55e12856bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.076+0000 7eff70fdd740 1 --2- 172.21.15.16:0/1113306093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128515cf0 0x55e12856bea0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7eff5802a340 tx=0x7eff58004730 comp rx=0 tx=0).stop 2024-04-25T10:13:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.076+0000 7eff70fdd740 1 -- 172.21.15.16:0/1113306093 shutdown_connections 2024-04-25T10:13:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.076+0000 7eff70fdd740 1 --2- 172.21.15.16:0/1113306093 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x55e12856d070 0x55e128538800 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.076+0000 7eff467fc640 0 client.8640 ms_handle_reset on v2:172.21.15.186:6834/2008057690 2024-04-25T10:13:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.076+0000 7eff70fdd740 1 --2- 172.21.15.16:0/1113306093 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e128540e10 0x55e1285382c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.076+0000 7eff70fdd740 1 --2- 172.21.15.16:0/1113306093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e128516bb0 0x55e1285cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.076+0000 7eff70fdd740 1 --2- 172.21.15.16:0/1113306093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e128515cf0 0x55e12856bea0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.076+0000 7eff70fdd740 1 -- 172.21.15.16:0/1113306093 >> 172.21.15.16:0/1113306093 conn(0x55e128511830 msgr2=0x55e128541f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.076+0000 7eff70fdd740 1 -- 172.21.15.16:0/1113306093 shutdown_connections 2024-04-25T10:13:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.077+0000 7eff70fdd740 1 -- 172.21.15.16:0/1113306093 wait complete. 2024-04-25T10:13:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.077+0000 7eff70fdd740 20 client.8640 trim_cache size 0 max 16384 2024-04-25T10:13:40.093 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:13:40.093 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-04-25T10:13:40.093 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "df9eaf39-62fc-4355-8a29-77bc359eb861", 2024-04-25T10:13:40.093 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-04-25T10:13:40.093 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:13:40.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.077+0000 7eff6e200640 1 -- 172.21.15.16:0/590162283 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7eff4806b500 msgr2=0x7eff4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.077+0000 7eff6e200640 1 --2- 172.21.15.16:0/590162283 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7eff4806b500 0x7eff4806d9c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7eff58034000 tx=0x7eff5802e000 comp rx=0 tx=0).stop 2024-04-25T10:13:40.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.077+0000 7eff6e200640 1 -- 172.21.15.16:0/590162283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6815ef60 msgr2=0x7eff680b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.077+0000 7eff6e200640 1 --2- 172.21.15.16:0/590162283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6815ef60 0x7eff680b4dc0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7eff6000efc0 tx=0x7eff6000c490 comp rx=0 tx=0).stop 2024-04-25T10:13:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.077+0000 7eff6e200640 1 -- 172.21.15.16:0/590162283 shutdown_connections 2024-04-25T10:13:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.077+0000 7eff6e200640 1 --2- 172.21.15.16:0/590162283 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7eff4806b500 0x7eff4806d9c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.077+0000 7eff6e200640 1 --2- 172.21.15.16:0/590162283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6815ef60 0x7eff680b4dc0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.077+0000 7eff6e200640 1 --2- 172.21.15.16:0/590162283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff680cea00 0x7eff680c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.077+0000 7eff6e200640 1 --2- 172.21.15.16:0/590162283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff680b0e50 0x7eff68157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.077+0000 7eff6e200640 1 -- 172.21.15.16:0/590162283 >> 172.21.15.16:0/590162283 conn(0x7eff6806f4c0 msgr2=0x7eff680cb230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.077+0000 7eff6e200640 1 -- 172.21.15.16:0/590162283 shutdown_connections 2024-04-25T10:13:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.077+0000 7eff6e200640 1 -- 172.21.15.16:0/590162283 wait complete. 2024-04-25T10:13:40.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:40.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:13:40.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.195+0000 7fbd77c4a640 1 Processor -- start 2024-04-25T10:13:40.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.195+0000 7fbd77c4a640 1 -- start start 2024-04-25T10:13:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.196+0000 7fbd77c4a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd700b5550 0x7fbd700b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.196+0000 7fbd77c4a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd70161500 con 0x7fbd700d2560 2024-04-25T10:13:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.196+0000 7fbd77c4a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd70161eb0 con 0x7fbd70161120 2024-04-25T10:13:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.196+0000 7fbd761e7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbd700d2560 0x7fbd700d29f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41234/0 (socket says 172.21.15.16:41234) 2024-04-25T10:13:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.196+0000 7fbd77c4a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd700b5ef0 con 0x7fbd700b5550 2024-04-25T10:13:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.196+0000 7fbd761e7640 1 -- 172.21.15.16:0/1597004606 learned_addr learned my addr 172.21.15.16:0/1597004606 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:13:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd751e5640 1 --2- 172.21.15.16:0/1597004606 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd700b5550 0x7fbd700b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd749e4640 1 -- 172.21.15.16:0/1597004606 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193411594 0 0) 0x7fbd70161500 con 0x7fbd700d2560 2024-04-25T10:13:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd749e4640 1 -- 172.21.15.16:0/1597004606 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd5c003520 con 0x7fbd700d2560 2024-04-25T10:13:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd749e4640 1 -- 172.21.15.16:0/1597004606 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 44039233 0 0) 0x7fbd5c003520 con 0x7fbd700d2560 2024-04-25T10:13:40.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd749e4640 1 -- 172.21.15.16:0/1597004606 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd700b5550 msgr2=0x7fbd700b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd749e4640 1 --2- 172.21.15.16:0/1597004606 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd700b5550 0x7fbd700b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd749e4640 1 -- 172.21.15.16:0/1597004606 >> v1:172.21.15.186:6789/0 conn(0x7fbd70161120 legacy=0x7fbd701617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd749e4640 1 -- 172.21.15.16:0/1597004606 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd70162180 con 0x7fbd700d2560 2024-04-25T10:13:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd749e4640 1 -- 172.21.15.16:0/1597004606 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fbd680019f0 con 0x7fbd700d2560 2024-04-25T10:13:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd749e4640 1 -- 172.21.15.16:0/1597004606 >> v1:172.21.15.16:6789/0 conn(0x7fbd700d2560 legacy=0x7fbd700d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd749e4640 1 --2- 172.21.15.16:0/1597004606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd5c0039e0 0x7fbd5c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd749e4640 1 --2- 172.21.15.16:0/1597004606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd5c004800 0x7fbd5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd749e4640 1 --2- 172.21.15.16:0/1597004606 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd700b5550 0x7fbd5c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd749e4640 1 -- 172.21.15.16:0/1597004606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c00bac0 con 0x7fbd5c0039e0 2024-04-25T10:13:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd749e4640 1 -- 172.21.15.16:0/1597004606 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c00bc70 con 0x7fbd5c004800 2024-04-25T10:13:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd749e4640 1 -- 172.21.15.16:0/1597004606 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c00be70 con 0x7fbd700b5550 2024-04-25T10:13:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd759e6640 1 --2- 172.21.15.16:0/1597004606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd5c0039e0 0x7fbd5c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd761e7640 1 --2- 172.21.15.16:0/1597004606 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd700b5550 0x7fbd5c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd751e5640 1 --2- 172.21.15.16:0/1597004606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd5c004800 0x7fbd5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd759e6640 1 -- 172.21.15.16:0/1597004606 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd700b5550 msgr2=0x7fbd5c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd759e6640 1 --2- 172.21.15.16:0/1597004606 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd700b5550 0x7fbd5c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd759e6640 1 -- 172.21.15.16:0/1597004606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd5c004800 msgr2=0x7fbd5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd759e6640 1 --2- 172.21.15.16:0/1597004606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd5c004800 0x7fbd5c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.197+0000 7fbd759e6640 1 -- 172.21.15.16:0/1597004606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c003520 con 0x7fbd5c0039e0 2024-04-25T10:13:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.198+0000 7fbd759e6640 1 --2- 172.21.15.16:0/1597004606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd5c0039e0 0x7fbd5c003e00 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c001750 tx=0x7fbd5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c1202bc74ceb6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.198+0000 7fbd749e4640 1 -- 172.21.15.16:0/1597004606 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c001370 con 0x7fbd5c0039e0 2024-04-25T10:13:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.198+0000 7fbd749e4640 1 -- 172.21.15.16:0/1597004606 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c0052d0 con 0x7fbd5c0039e0 2024-04-25T10:13:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.198+0000 7fbd749e4640 1 -- 172.21.15.16:0/1597004606 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c02abe0 con 0x7fbd5c0039e0 2024-04-25T10:13:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.199+0000 7fbd77c4a640 1 -- 172.21.15.16:0/1597004606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd5c0039e0 msgr2=0x7fbd5c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.199+0000 7fbd77c4a640 1 --2- 172.21.15.16:0/1597004606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd5c0039e0 0x7fbd5c003e00 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c001750 tx=0x7fbd5c005830 comp rx=0 tx=0).stop 2024-04-25T10:13:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.199+0000 7fbd761e7640 1 -- 172.21.15.16:0/1597004606 reap_dead start 2024-04-25T10:13:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.199+0000 7fbd77c4a640 1 -- 172.21.15.16:0/1597004606 shutdown_connections 2024-04-25T10:13:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.199+0000 7fbd77c4a640 1 -- 172.21.15.16:0/1597004606 >> 172.21.15.16:0/1597004606 conn(0x7fbd700d0550 msgr2=0x7fbd700d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.199+0000 7fbd77c4a640 1 -- 172.21.15.16:0/1597004606 shutdown_connections 2024-04-25T10:13:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.199+0000 7fbd77c4a640 1 -- 172.21.15.16:0/1597004606 wait complete. 2024-04-25T10:13:40.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.200+0000 7fbd77c4a640 1 Processor -- start 2024-04-25T10:13:40.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.200+0000 7fbd77c4a640 1 -- start start 2024-04-25T10:13:40.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.201+0000 7fbd77c4a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd5c0039e0 0x7fbd700c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.201+0000 7fbd77c4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd7015d040 0x7fbd700c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.201+0000 7fbd77c4a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd700c8030 0x7fbd700c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.201+0000 7fbd77c4a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd700d1d60 con 0x7fbd7015d040 2024-04-25T10:13:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.201+0000 7fbd77c4a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd70162740 con 0x7fbd5c0039e0 2024-04-25T10:13:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.201+0000 7fbd77c4a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd700d2040 con 0x7fbd700c8030 2024-04-25T10:13:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.201+0000 7fbd751e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd7015d040 0x7fbd700c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.201+0000 7fbd751e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd7015d040 0x7fbd700c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33124/0 (socket says 172.21.15.16:33124) 2024-04-25T10:13:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.201+0000 7fbd751e5640 1 -- 172.21.15.16:0/660413229 learned_addr learned my addr 172.21.15.16:0/660413229 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.201+0000 7fbd759e6640 1 --2- 172.21.15.16:0/660413229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd5c0039e0 0x7fbd700c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.201+0000 7fbd761e7640 1 --2- 172.21.15.16:0/660413229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd700c8030 0x7fbd700c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.201+0000 7fbd751e5640 1 -- 172.21.15.16:0/660413229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd700c8030 msgr2=0x7fbd700c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.201+0000 7fbd751e5640 1 --2- 172.21.15.16:0/660413229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd700c8030 0x7fbd700c84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.201+0000 7fbd751e5640 1 -- 172.21.15.16:0/660413229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd5c0039e0 msgr2=0x7fbd700c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.201+0000 7fbd751e5640 1 --2- 172.21.15.16:0/660413229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd5c0039e0 0x7fbd700c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.201+0000 7fbd751e5640 1 -- 172.21.15.16:0/660413229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd700cc600 con 0x7fbd7015d040 2024-04-25T10:13:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.201+0000 7fbd761e7640 1 --2- 172.21.15.16:0/660413229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd700c8030 0x7fbd700c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.201+0000 7fbd751e5640 1 --2- 172.21.15.16:0/660413229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd7015d040 0x7fbd700c7af0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fbd600118e0 tx=0x7fbd60011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.202+0000 7fbd667fc640 1 -- 172.21.15.16:0/660413229 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6000faf0 con 0x7fbd7015d040 2024-04-25T10:13:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.202+0000 7fbd667fc640 1 -- 172.21.15.16:0/660413229 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6000fc90 con 0x7fbd7015d040 2024-04-25T10:13:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.202+0000 7fbd77c4a640 1 -- 172.21.15.16:0/660413229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd700cc8e0 con 0x7fbd7015d040 2024-04-25T10:13:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.202+0000 7fbd667fc640 1 -- 172.21.15.16:0/660413229 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60014440 con 0x7fbd7015d040 2024-04-25T10:13:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.202+0000 7fbd77c4a640 1 -- 172.21.15.16:0/660413229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd700ccea0 con 0x7fbd7015d040 2024-04-25T10:13:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.203+0000 7fbd667fc640 1 -- 172.21.15.16:0/660413229 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd60012040 con 0x7fbd7015d040 2024-04-25T10:13:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.204+0000 7fbd667fc640 1 --2- 172.21.15.16:0/660413229 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbd5406b4d0 0x7fbd5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.204+0000 7fbd667fc640 1 -- 172.21.15.16:0/660413229 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbd6008a860 con 0x7fbd7015d040 2024-04-25T10:13:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.204+0000 7fbd759e6640 1 --2- 172.21.15.16:0/660413229 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbd5406b4d0 0x7fbd5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.204+0000 7fbd759e6640 1 --2- 172.21.15.16:0/660413229 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbd5406b4d0 0x7fbd5406d990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c032000 tx=0x7fbd6c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.205+0000 7fbd7aa29740 1 Processor -- start 2024-04-25T10:13:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.205+0000 7fbd7aa29740 1 -- start start 2024-04-25T10:13:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd7aa29740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e639b10 0x55be7e63c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd7aa29740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be7e639ef0 0x55be7e63c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd7aa29740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55be7e60c770 0x55be7e60ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd7aa29740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be7e575da0 con 0x55be7e639b10 2024-04-25T10:13:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd7aa29740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be7e51eb80 con 0x55be7e639ef0 2024-04-25T10:13:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd7aa29740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be7e571e40 con 0x55be7e60c770 2024-04-25T10:13:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd759e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e639b10 0x55be7e63c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd759e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e639b10 0x55be7e63c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33138/0 (socket says 172.21.15.16:33138) 2024-04-25T10:13:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd759e6640 1 -- 172.21.15.16:0/468336939 learned_addr learned my addr 172.21.15.16:0/468336939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd761e7640 1 --2- 172.21.15.16:0/468336939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55be7e60c770 0x55be7e60ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd751e5640 1 --2- 172.21.15.16:0/468336939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be7e639ef0 0x55be7e63c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd759e6640 1 -- 172.21.15.16:0/468336939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55be7e60c770 msgr2=0x55be7e60ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd759e6640 1 --2- 172.21.15.16:0/468336939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55be7e60c770 0x55be7e60ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd759e6640 1 -- 172.21.15.16:0/468336939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be7e639ef0 msgr2=0x55be7e63c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd759e6640 1 --2- 172.21.15.16:0/468336939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be7e639ef0 0x55be7e63c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd759e6640 1 -- 172.21.15.16:0/468336939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be7e528f10 con 0x55be7e639b10 2024-04-25T10:13:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd759e6640 1 --2- 172.21.15.16:0/468336939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e639b10 0x55be7e63c320 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c02a590 tx=0x7fbd6c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998131d47bda8ded server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd537fe640 1 -- 172.21.15.16:0/468336939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c04b070 con 0x55be7e639b10 2024-04-25T10:13:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd537fe640 1 -- 172.21.15.16:0/468336939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c007590 con 0x55be7e639b10 2024-04-25T10:13:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.206+0000 7fbd537fe640 1 -- 172.21.15.16:0/468336939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c041040 con 0x55be7e639b10 2024-04-25T10:13:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.207+0000 7fbd7aa29740 1 -- 172.21.15.16:0/468336939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e639b10 msgr2=0x55be7e63c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.207+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/468336939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e639b10 0x55be7e63c320 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c02a590 tx=0x7fbd6c02c000 comp rx=0 tx=0).stop 2024-04-25T10:13:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.207+0000 7fbd7aa29740 1 -- 172.21.15.16:0/468336939 shutdown_connections 2024-04-25T10:13:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.207+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/468336939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55be7e60c770 0x55be7e60ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.207+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/468336939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be7e639ef0 0x55be7e63c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.207+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/468336939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e639b10 0x55be7e63c320 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.207+0000 7fbd7aa29740 1 -- 172.21.15.16:0/468336939 >> 172.21.15.16:0/468336939 conn(0x55be7e60e9e0 msgr2=0x55be7e657320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.207+0000 7fbd7aa29740 1 -- 172.21.15.16:0/468336939 shutdown_connections 2024-04-25T10:13:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.207+0000 7fbd7aa29740 1 -- 172.21.15.16:0/468336939 wait complete. 2024-04-25T10:13:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.207+0000 7fbd7aa29740 1 -- start start 2024-04-25T10:13:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.208+0000 7fbd7aa29740 1 Processor -- start 2024-04-25T10:13:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.208+0000 7fbd7aa29740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be7e639b10 0x55be7e658450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.208+0000 7fbd7aa29740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55be7e639ef0 0x55be7e659dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.208+0000 7fbd7aa29740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e654510 0x55be7e63bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd7aa29740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be7e611b90 con 0x55be7e654510 2024-04-25T10:13:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd7aa29740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be7e5144b0 con 0x55be7e639b10 2024-04-25T10:13:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd7aa29740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be7e611e70 con 0x55be7e639ef0 2024-04-25T10:13:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd7aa29740 10 client.0 _ll_register_callbacks cb 0x55be7e554180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd761e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e654510 0x55be7e63bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd761e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e654510 0x55be7e63bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33144/0 (socket says 172.21.15.16:33144) 2024-04-25T10:13:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd761e7640 1 -- 172.21.15.16:0/803580827 learned_addr learned my addr 172.21.15.16:0/803580827 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd759e6640 1 --2- 172.21.15.16:0/803580827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be7e639b10 0x55be7e658450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd751e5640 1 --2- 172.21.15.16:0/803580827 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55be7e639ef0 0x55be7e659dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd761e7640 1 -- 172.21.15.16:0/803580827 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55be7e639ef0 msgr2=0x55be7e659dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd761e7640 1 --2- 172.21.15.16:0/803580827 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55be7e639ef0 0x55be7e659dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd761e7640 1 -- 172.21.15.16:0/803580827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be7e639b10 msgr2=0x55be7e658450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd761e7640 1 --2- 172.21.15.16:0/803580827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be7e639b10 0x55be7e658450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd761e7640 1 -- 172.21.15.16:0/803580827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be7e3547d0 con 0x55be7e654510 2024-04-25T10:13:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd761e7640 1 --2- 172.21.15.16:0/803580827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e654510 0x55be7e63bc80 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fbd700b34d0 tx=0x7fbd680078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8f4a49326fda3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:13:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd50ff9640 1 -- 172.21.15.16:0/803580827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68019070 con 0x55be7e654510 2024-04-25T10:13:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd50ff9640 1 -- 172.21.15.16:0/803580827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbd6800aaa0 con 0x55be7e654510 2024-04-25T10:13:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd7aa29740 1 -- 172.21.15.16:0/803580827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd6c007590 con 0x55be7e654510 2024-04-25T10:13:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.209+0000 7fbd50ff9640 1 -- 172.21.15.16:0/803580827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd68024050 con 0x55be7e654510 2024-04-25T10:13:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.210+0000 7fbd7aa29740 10 client.8652 fetch_fsmap learned FSMap version 76 2024-04-25T10:13:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.210+0000 7fbd7aa29740 1 -- 172.21.15.16:0/803580827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55be7e28a3f0 con 0x55be7e654510 2024-04-25T10:13:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.210+0000 7fbd50ff9640 1 -- 172.21.15.16:0/803580827 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c007590 con 0x55be7e654510 2024-04-25T10:13:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.210+0000 7fbd50ff9640 1 -- 172.21.15.16:0/803580827 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbd68027020 con 0x55be7e654510 2024-04-25T10:13:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.210+0000 7fbd7aa29740 10 client.8652 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T10:13:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.210+0000 7fbd7aa29740 10 client.8652 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:13:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.210+0000 7fbd7aa29740 20 client.8652 populate_metadata read hostname 'smithi016' 2024-04-25T10:13:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.210+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/803580827 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x55be7e651e20 0x55be7e652250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.210+0000 7fbd7aa29740 4 client.8652 mds_command: new command op to 36575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.210+0000 7fbd7aa29740 1 -- 172.21.15.16:0/803580827 --> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be7e5832e0 con 0x55be7e651e20 2024-04-25T10:13:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.210+0000 7fbd759e6640 1 --2- 172.21.15.16:0/803580827 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x55be7e651e20 0x55be7e652250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.210+0000 7fbd759e6640 1 --2- 172.21.15.16:0/803580827 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x55be7e651e20 0x55be7e652250 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e0d5c1505fbbd8c server_cookie=e2128b3a569ea6cb in_seq=0 out_seq=0 2024-04-25T10:13:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.210+0000 7fbd50ff9640 10 client.8652 ms_handle_connect on v2:172.21.15.186:6834/2008057690 2024-04-25T10:13:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.211+0000 7fbd50ff9640 1 -- 172.21.15.16:0/803580827 <== mds.0 v2:172.21.15.186:6834/2008057690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be7e5832e0 con 0x55be7e651e20 2024-04-25T10:13:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.211+0000 7fbd50ff9640 10 client.8652 handle_command_reply: tid=0 2024-04-25T10:13:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.211+0000 7fbd7aa29740 1 client.8652 shutdown 2024-04-25T10:13:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.211+0000 7fbd7aa29740 1 -- 172.21.15.16:0/803580827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e654510 msgr2=0x55be7e63bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.211+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/803580827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e654510 0x55be7e63bc80 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fbd700b34d0 tx=0x7fbd680078a0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.211+0000 7fbd7aa29740 1 -- 172.21.15.16:0/803580827 shutdown_connections 2024-04-25T10:13:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.211+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/803580827 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x55be7e651e20 0x55be7e652250 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.211+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/803580827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e654510 0x55be7e63bc80 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.211+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/803580827 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55be7e639ef0 0x55be7e659dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.211+0000 7fbd50ff9640 0 client.8652 ms_handle_reset on v2:172.21.15.186:6834/2008057690 2024-04-25T10:13:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.211+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/803580827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be7e639b10 0x55be7e658450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.211+0000 7fbd7aa29740 1 -- 172.21.15.16:0/803580827 >> 172.21.15.16:0/803580827 conn(0x55be7e60c770 msgr2=0x55be7e6113a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.211+0000 7fbd7aa29740 1 -- 172.21.15.16:0/803580827 shutdown_connections 2024-04-25T10:13:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.211+0000 7fbd7aa29740 1 -- 172.21.15.16:0/803580827 wait complete. 2024-04-25T10:13:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.211+0000 7fbd7aa29740 20 client.8652 trim_cache size 0 max 16384 2024-04-25T10:13:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.227+0000 7fbd7aa29740 1 Processor -- start 2024-04-25T10:13:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.227+0000 7fbd7aa29740 1 -- start start 2024-04-25T10:13:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.228+0000 7fbd7aa29740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be7e639b10 0x55be7e62f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.228+0000 7fbd7aa29740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55be7e639ef0 0x55be7e657320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.228+0000 7fbd7aa29740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e654510 0x55be7e657860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.228+0000 7fbd7aa29740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd70200600 con 0x55be7e654510 2024-04-25T10:13:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.228+0000 7fbd7aa29740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd701ff7e0 con 0x55be7e639b10 2024-04-25T10:13:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.228+0000 7fbd7aa29740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd701fed00 con 0x55be7e639ef0 2024-04-25T10:13:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.228+0000 7fbd761e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e654510 0x55be7e657860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.228+0000 7fbd761e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e654510 0x55be7e657860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33152/0 (socket says 172.21.15.16:33152) 2024-04-25T10:13:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.228+0000 7fbd761e7640 1 -- 172.21.15.16:0/2970888398 learned_addr learned my addr 172.21.15.16:0/2970888398 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.228+0000 7fbd761e7640 1 -- 172.21.15.16:0/2970888398 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55be7e639ef0 msgr2=0x55be7e657320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.228+0000 7fbd761e7640 1 --2- 172.21.15.16:0/2970888398 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55be7e639ef0 0x55be7e657320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.228+0000 7fbd761e7640 1 -- 172.21.15.16:0/2970888398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be7e639b10 msgr2=0x55be7e62f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.228+0000 7fbd761e7640 1 --2- 172.21.15.16:0/2970888398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be7e639b10 0x55be7e62f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.228+0000 7fbd761e7640 1 -- 172.21.15.16:0/2970888398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be7e24a0f0 con 0x55be7e654510 2024-04-25T10:13:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.228+0000 7fbd761e7640 1 --2- 172.21.15.16:0/2970888398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e654510 0x55be7e657860 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fbd68007140 tx=0x7fbd6800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edf3acc279f0c87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.228+0000 7fbd52ffd640 1 -- 172.21.15.16:0/2970888398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6802d070 con 0x55be7e654510 2024-04-25T10:13:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.228+0000 7fbd52ffd640 1 -- 172.21.15.16:0/2970888398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6801e440 con 0x55be7e654510 2024-04-25T10:13:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.228+0000 7fbd52ffd640 1 -- 172.21.15.16:0/2970888398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68019070 con 0x55be7e654510 2024-04-25T10:13:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.229+0000 7fbd7aa29740 1 -- 172.21.15.16:0/2970888398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e654510 msgr2=0x55be7e657860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.229+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/2970888398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e654510 0x55be7e657860 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fbd68007140 tx=0x7fbd6800f000 comp rx=0 tx=0).stop 2024-04-25T10:13:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.229+0000 7fbd7aa29740 1 -- 172.21.15.16:0/2970888398 shutdown_connections 2024-04-25T10:13:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.229+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/2970888398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e654510 0x55be7e657860 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.229+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/2970888398 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55be7e639ef0 0x55be7e657320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.229+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/2970888398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be7e639b10 0x55be7e62f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.229+0000 7fbd7aa29740 1 -- 172.21.15.16:0/2970888398 >> 172.21.15.16:0/2970888398 conn(0x55be7e60c770 msgr2=0x55be7e6127a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.229+0000 7fbd7aa29740 1 -- 172.21.15.16:0/2970888398 shutdown_connections 2024-04-25T10:13:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.229+0000 7fbd7aa29740 1 -- 172.21.15.16:0/2970888398 wait complete. 2024-04-25T10:13:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.230+0000 7fbd7aa29740 1 -- start start 2024-04-25T10:13:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.230+0000 7fbd7aa29740 1 Processor -- start 2024-04-25T10:13:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.230+0000 7fbd7aa29740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e639b10 0x55be7e657c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.230+0000 7fbd7aa29740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55be7e639ef0 0x55be7e6c4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.230+0000 7fbd7aa29740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be7e654510 0x55be7e6301d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.230+0000 7fbd7aa29740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be7e60d600 con 0x55be7e639b10 2024-04-25T10:13:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.230+0000 7fbd7aa29740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd701ff1e0 con 0x55be7e654510 2024-04-25T10:13:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.230+0000 7fbd7aa29740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be7e60c590 con 0x55be7e639ef0 2024-04-25T10:13:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.231+0000 7fbd7aa29740 10 client.0 _ll_register_callbacks cb 0x7fbd5c004660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.231+0000 7fbd759e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e639b10 0x55be7e657c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.231+0000 7fbd759e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e639b10 0x55be7e657c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33166/0 (socket says 172.21.15.16:33166) 2024-04-25T10:13:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.231+0000 7fbd759e6640 1 -- 172.21.15.16:0/1331918519 learned_addr learned my addr 172.21.15.16:0/1331918519 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:13:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.231+0000 7fbd759e6640 1 -- 172.21.15.16:0/1331918519 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55be7e639ef0 msgr2=0x55be7e6c4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.231+0000 7fbd759e6640 1 --2- 172.21.15.16:0/1331918519 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55be7e639ef0 0x55be7e6c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.231+0000 7fbd759e6640 1 -- 172.21.15.16:0/1331918519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be7e654510 msgr2=0x55be7e6301d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.231+0000 7fbd759e6640 1 --2- 172.21.15.16:0/1331918519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be7e654510 0x55be7e6301d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.231+0000 7fbd759e6640 1 -- 172.21.15.16:0/1331918519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be7e60ba10 con 0x55be7e639b10 2024-04-25T10:13:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.231+0000 7fbd759e6640 1 --2- 172.21.15.16:0/1331918519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e639b10 0x55be7e657c60 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c001720 tx=0x7fbd6c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76205eef35f1c125 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.231+0000 7fbd50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:13:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.231+0000 7fbd50ff9640 1 -- 172.21.15.16:0/1331918519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c053020 con 0x55be7e639b10 2024-04-25T10:13:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.231+0000 7fbd50ff9640 1 -- 172.21.15.16:0/1331918519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbd6c007590 con 0x55be7e639b10 2024-04-25T10:13:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.231+0000 7fbd7aa29740 1 -- 172.21.15.16:0/1331918519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd6801e440 con 0x55be7e639b10 2024-04-25T10:13:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.231+0000 7fbd50ff9640 1 -- 172.21.15.16:0/1331918519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd6c05d050 con 0x55be7e639b10 2024-04-25T10:13:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.231+0000 7fbd7aa29740 10 client.8658 fetch_fsmap learned FSMap version 76 2024-04-25T10:13:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.231+0000 7fbd7aa29740 1 -- 172.21.15.16:0/1331918519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55be7e63a2d0 con 0x55be7e639b10 2024-04-25T10:13:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.231+0000 7fbd50ff9640 1 -- 172.21.15.16:0/1331918519 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6801e440 con 0x55be7e639b10 2024-04-25T10:13:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.232+0000 7fbd50ff9640 1 -- 172.21.15.16:0/1331918519 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbd6c060020 con 0x55be7e639b10 2024-04-25T10:13:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.232+0000 7fbd7aa29740 10 client.8658 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T10:13:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.232+0000 7fbd7aa29740 10 client.8658 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:13:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.232+0000 7fbd7aa29740 20 client.8658 populate_metadata read hostname 'smithi016' 2024-04-25T10:13:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.232+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/1331918519 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x55be7e630710 0x55be7e6384d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.232+0000 7fbd7aa29740 4 client.8658 mds_command: new command op to 36575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:13:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.232+0000 7fbd7aa29740 1 -- 172.21.15.16:0/1331918519 --> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55be7e57d630 con 0x55be7e630710 2024-04-25T10:13:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.232+0000 7fbd751e5640 1 --2- 172.21.15.16:0/1331918519 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x55be7e630710 0x55be7e6384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.232+0000 7fbd751e5640 1 --2- 172.21.15.16:0/1331918519 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x55be7e630710 0x55be7e6384d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=874f9d505a85bf76 server_cookie=1c7d830ef658ee36 in_seq=0 out_seq=0 2024-04-25T10:13:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.232+0000 7fbd50ff9640 10 client.8658 ms_handle_connect on v2:172.21.15.186:6834/2008057690 2024-04-25T10:13:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.232+0000 7fbd50ff9640 1 -- 172.21.15.16:0/1331918519 <== mds.0 v2:172.21.15.186:6834/2008057690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55be7e57d630 con 0x55be7e630710 2024-04-25T10:13:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.232+0000 7fbd50ff9640 10 client.8658 handle_command_reply: tid=0 2024-04-25T10:13:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.232+0000 7fbd7aa29740 1 client.8658 shutdown 2024-04-25T10:13:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd7aa29740 1 -- 172.21.15.16:0/1331918519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e639b10 msgr2=0x55be7e657c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/1331918519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e639b10 0x55be7e657c60 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c001720 tx=0x7fbd6c001120 comp rx=0 tx=0).stop 2024-04-25T10:13:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd7aa29740 1 -- 172.21.15.16:0/1331918519 shutdown_connections 2024-04-25T10:13:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/1331918519 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x55be7e630710 0x55be7e6384d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/1331918519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be7e654510 0x55be7e6301d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd50ff9640 0 client.8658 ms_handle_reset on v2:172.21.15.186:6834/2008057690 2024-04-25T10:13:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/1331918519 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55be7e639ef0 0x55be7e6c4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd7aa29740 1 --2- 172.21.15.16:0/1331918519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be7e639b10 0x55be7e657c60 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd7aa29740 1 -- 172.21.15.16:0/1331918519 >> 172.21.15.16:0/1331918519 conn(0x55be7e60c770 msgr2=0x55be7e659af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd7aa29740 1 -- 172.21.15.16:0/1331918519 shutdown_connections 2024-04-25T10:13:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd7aa29740 1 -- 172.21.15.16:0/1331918519 wait complete. 2024-04-25T10:13:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd7aa29740 20 client.8658 trim_cache size 0 max 16384 2024-04-25T10:13:40.245 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:13:40.245 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:13:40.245 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-04-25T10:13:40.246 INFO:teuthology.orchestra.run.smithi016.stdout: "df9eaf39-62fc-4355-8a29-77bc359eb861": { 2024-04-25T10:13:40.246 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-04-25T10:13:40.246 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "df9eaf39-62fc-4355-8a29-77bc359eb861", 2024-04-25T10:13:40.246 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,repair" 2024-04-25T10:13:40.246 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:13:40.246 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:13:40.246 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:13:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd77c4a640 1 -- 172.21.15.16:0/660413229 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbd5406b4d0 msgr2=0x7fbd5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd77c4a640 1 --2- 172.21.15.16:0/660413229 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbd5406b4d0 0x7fbd5406d990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c032000 tx=0x7fbd6c02e000 comp rx=0 tx=0).stop 2024-04-25T10:13:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd77c4a640 1 -- 172.21.15.16:0/660413229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd7015d040 msgr2=0x7fbd700c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd77c4a640 1 --2- 172.21.15.16:0/660413229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd7015d040 0x7fbd700c7af0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fbd600118e0 tx=0x7fbd60011da0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd77c4a640 1 -- 172.21.15.16:0/660413229 shutdown_connections 2024-04-25T10:13:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd77c4a640 1 --2- 172.21.15.16:0/660413229 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbd5406b4d0 0x7fbd5406d990 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd77c4a640 1 --2- 172.21.15.16:0/660413229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd700c8030 0x7fbd700c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd77c4a640 1 --2- 172.21.15.16:0/660413229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd7015d040 0x7fbd700c7af0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd77c4a640 1 --2- 172.21.15.16:0/660413229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd5c0039e0 0x7fbd700c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd77c4a640 1 -- 172.21.15.16:0/660413229 >> 172.21.15.16:0/660413229 conn(0x7fbd700d0550 msgr2=0x7fbd700b5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd77c4a640 1 -- 172.21.15.16:0/660413229 shutdown_connections 2024-04-25T10:13:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.233+0000 7fbd77c4a640 1 -- 172.21.15.16:0/660413229 wait complete. 2024-04-25T10:13:40.249 INFO:tasks.cephfs.filesystem:scrub status for tag:df9eaf39-62fc-4355-8a29-77bc359eb861 - {'path': '/', 'tag': 'df9eaf39-62fc-4355-8a29-77bc359eb861', 'options': 'recursive,repair'} 2024-04-25T10:13:40.491 INFO:tasks.ceph.mds.b.smithi186.stderr:2024-04-25T10:13:40.486+0000 7ff20f1d7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:40.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:40.723+0000 7f5872ffd640 1 -- 172.21.15.16:0/87647012 <== mon.0 v2:172.21.15.16:3300/0 7 ==== log(8 entries from seq 1 at 2024-04-25T10:13:40.077209+0000) v1 ==== 1407+0+0 (secure 0 0 0) 0x7f587c056f30 con 0x7f5884162200 2024-04-25T10:13:40.728 INFO:teuthology.orchestra.run.smithi016.stdout: cluster: 2024-04-25T10:13:40.729 INFO:teuthology.orchestra.run.smithi016.stdout: id: 34bc5eca-3a21-42a6-a26f-07da0905e8bc 2024-04-25T10:13:40.729 INFO:teuthology.orchestra.run.smithi016.stdout: health: HEALTH_OK 2024-04-25T10:13:40.729 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:13:40.729 INFO:teuthology.orchestra.run.smithi016.stdout: services: 2024-04-25T10:13:40.729 INFO:teuthology.orchestra.run.smithi016.stdout: mon: 3 daemons, quorum a,b,c (age 8m) 2024-04-25T10:13:40.729 INFO:teuthology.orchestra.run.smithi016.stdout: mgr: x(active, since 8m), standbys: y 2024-04-25T10:13:40.729 INFO:teuthology.orchestra.run.smithi016.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T10:13:40.729 INFO:teuthology.orchestra.run.smithi016.stdout: osd: 8 osds: 8 up (since 8m), 8 in (since 8m) 2024-04-25T10:13:40.729 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:13:40.729 INFO:teuthology.orchestra.run.smithi016.stdout: data: 2024-04-25T10:13:40.729 INFO:teuthology.orchestra.run.smithi016.stdout: volumes: 1/1 healthy 2024-04-25T10:13:40.729 INFO:teuthology.orchestra.run.smithi016.stdout: pools: 3 pools, 97 pgs 2024-04-25T10:13:40.729 INFO:teuthology.orchestra.run.smithi016.stdout: objects: 30 objects, 19 MiB 2024-04-25T10:13:40.729 INFO:teuthology.orchestra.run.smithi016.stdout: usage: 302 MiB used, 720 GiB / 720 GiB avail 2024-04-25T10:13:40.730 INFO:teuthology.orchestra.run.smithi016.stdout: pgs: 97 active+clean 2024-04-25T10:13:40.730 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:13:40.730 INFO:teuthology.orchestra.run.smithi016.stdout: io: 2024-04-25T10:13:40.730 INFO:teuthology.orchestra.run.smithi016.stdout: client: 1.5 KiB/s rd, 102 B/s wr, 1 op/s rd, 0 op/s wr 2024-04-25T10:13:40.730 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:13:40.730 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:13:40.730 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:13:40.077209+0000 mds.b [INF] scrub queued for path: / 2024-04-25T10:13:40.730 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:13:40.077213+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:13:40.730 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:13:40.077218+0000 mds.b [INF] scrub summary: active paths [/] 2024-04-25T10:13:40.730 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:13:40.077906+0000 mds.b [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-04-25T10:13:40.730 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:13:40.077914+0000 mds.b [ERR] inode table repaired for inode: 0x10000000001 2024-04-25T10:13:40.730 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:13:40.078030+0000 mds.b [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-04-25T10:13:40.730 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:13:40.078036+0000 mds.b [ERR] inode table repaired for inode: 0x10000000002 2024-04-25T10:13:40.730 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:13:40.078630+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:13:45.739 INFO:teuthology.orchestra.run.smithi016.stdout:2024-04-25T10:13:44.545852+0000 mds.b [INF] scrub summary: idle 2024-04-25T10:13:45.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:45.734+0000 7f5872ffd640 1 -- 172.21.15.16:0/87647012 <== mon.0 v2:172.21.15.16:3300/0 8 ==== log(1 entries from seq 9 at 2024-04-25T10:13:44.545852+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f587c057190 con 0x7f5884162200 2024-04-25T10:13:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:52.966+0000 7f5872ffd640 1 -- 172.21.15.16:0/87647012 <== mon.0 v2:172.21.15.16:3300/0 9 ==== log(8 entries from seq 148 at 2024-04-25T10:13:52.505191+0000) v1 ==== 2208+0+0 (secure 0 0 0) 0x7f587c088ca0 con 0x7f5884162200 2024-04-25T10:13:53.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:13:53.976+0000 7f5872ffd640 1 -- 172.21.15.16:0/87647012 <== mon.0 v2:172.21.15.16:3300/0 10 ==== log(4 entries from seq 1013 at 2024-04-25T10:13:52.971143+0000) v1 ==== 1079+0+0 (secure 0 0 0) 0x7f587c055500 con 0x7f5884162200 2024-04-25T10:14:08.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:08.806 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:10.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:10.250 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:14:10.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.365+0000 7fdb9af6a640 1 Processor -- start 2024-04-25T10:14:10.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.365+0000 7fdb9af6a640 1 -- start start 2024-04-25T10:14:10.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.365+0000 7fdb9af6a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9415df30 0x7fdb940cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.365+0000 7fdb9af6a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fdb940d12a0 0x7fdb94099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.365+0000 7fdb9af6a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb940d0200 con 0x7fdb9415db50 2024-04-25T10:14:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.365+0000 7fdb9af6a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb9406c020 con 0x7fdb940d12a0 2024-04-25T10:14:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.365+0000 7fdb9af6a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940d04e0 con 0x7fdb9415df30 2024-04-25T10:14:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.366+0000 7fdb99507640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fdb940d12a0 0x7fdb94099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.366+0000 7fdb93fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9415df30 0x7fdb940cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.366+0000 7fdb98d06640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fdb9415db50 0x7fdb9406cba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:50586/0 (socket says 172.21.15.16:50586) 2024-04-25T10:14:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.366+0000 7fdb98d06640 1 -- 172.21.15.16:0/291476601 learned_addr learned my addr 172.21.15.16:0/291476601 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.366+0000 7fdb937fe640 1 -- 172.21.15.16:0/291476601 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091975542 0 0) 0x7fdb940d0200 con 0x7fdb9415db50 2024-04-25T10:14:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.366+0000 7fdb937fe640 1 -- 172.21.15.16:0/291476601 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb7c003520 con 0x7fdb9415db50 2024-04-25T10:14:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.366+0000 7fdb93fff640 1 -- 172.21.15.16:0/291476601 >> v2:172.21.15.186:3300/0 conn(0x7fdb940d12a0 msgr2=0x7fdb94099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.366+0000 7fdb93fff640 1 --2- 172.21.15.16:0/291476601 >> v2:172.21.15.186:3300/0 conn(0x7fdb940d12a0 0x7fdb94099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.366+0000 7fdb93fff640 1 -- 172.21.15.16:0/291476601 >> v1:172.21.15.186:6789/0 conn(0x7fdb9415db50 legacy=0x7fdb9406cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.366+0000 7fdb93fff640 1 -- 172.21.15.16:0/291476601 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb940d07c0 con 0x7fdb9415df30 2024-04-25T10:14:10.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.366+0000 7fdb99507640 1 --2- 172.21.15.16:0/291476601 >> v2:172.21.15.186:3300/0 conn(0x7fdb940d12a0 0x7fdb94099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:10.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.367+0000 7fdb93fff640 1 --2- 172.21.15.16:0/291476601 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9415df30 0x7fdb940cfb80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdb880097a0 tx=0x7fdb8802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b49f4f84bbc0d1e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.367+0000 7fdb937fe640 1 -- 172.21.15.16:0/291476601 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88037070 con 0x7fdb9415df30 2024-04-25T10:14:10.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.367+0000 7fdb937fe640 1 -- 172.21.15.16:0/291476601 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb88032440 con 0x7fdb9415df30 2024-04-25T10:14:10.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.367+0000 7fdb937fe640 1 -- 172.21.15.16:0/291476601 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88032810 con 0x7fdb9415df30 2024-04-25T10:14:10.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.368+0000 7fdb9af6a640 1 -- 172.21.15.16:0/291476601 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9415df30 msgr2=0x7fdb940cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.368+0000 7fdb9af6a640 1 --2- 172.21.15.16:0/291476601 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9415df30 0x7fdb940cfb80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdb880097a0 tx=0x7fdb8802a180 comp rx=0 tx=0).stop 2024-04-25T10:14:10.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.368+0000 7fdb9af6a640 1 -- 172.21.15.16:0/291476601 shutdown_connections 2024-04-25T10:14:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.368+0000 7fdb9af6a640 1 --2- 172.21.15.16:0/291476601 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9415df30 0x7fdb940cfb80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.368+0000 7fdb9af6a640 1 --2- 172.21.15.16:0/291476601 >> v2:172.21.15.186:3300/0 conn(0x7fdb940d12a0 0x7fdb94099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.368+0000 7fdb9af6a640 1 -- 172.21.15.16:0/291476601 >> 172.21.15.16:0/291476601 conn(0x7fdb940b3430 msgr2=0x7fdb940b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.368+0000 7fdb9af6a640 1 -- 172.21.15.16:0/291476601 shutdown_connections 2024-04-25T10:14:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.368+0000 7fdb9af6a640 1 -- 172.21.15.16:0/291476601 wait complete. 2024-04-25T10:14:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.369+0000 7fdb9af6a640 1 Processor -- start 2024-04-25T10:14:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.369+0000 7fdb9af6a640 1 -- start start 2024-04-25T10:14:10.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.370+0000 7fdb9af6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb940d12a0 0x7fdb9409b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.370+0000 7fdb9af6a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9415db50 0x7fdb94157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.370+0000 7fdb9af6a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9415df30 0x7fdb940c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.370+0000 7fdb9af6a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb94099f80 con 0x7fdb940d12a0 2024-04-25T10:14:10.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.370+0000 7fdb9af6a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940d0d80 con 0x7fdb9415df30 2024-04-25T10:14:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.370+0000 7fdb9af6a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9409a100 con 0x7fdb9415db50 2024-04-25T10:14:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.370+0000 7fdb98d06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb940d12a0 0x7fdb9409b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.370+0000 7fdb98d06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb940d12a0 0x7fdb9409b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41652/0 (socket says 172.21.15.16:41652) 2024-04-25T10:14:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.370+0000 7fdb98d06640 1 -- 172.21.15.16:0/4192514900 learned_addr learned my addr 172.21.15.16:0/4192514900 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.370+0000 7fdb93fff640 1 --2- 172.21.15.16:0/4192514900 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9415db50 0x7fdb94157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.370+0000 7fdb99507640 1 --2- 172.21.15.16:0/4192514900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9415df30 0x7fdb940c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.371+0000 7fdb98d06640 1 -- 172.21.15.16:0/4192514900 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9415db50 msgr2=0x7fdb94157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.371+0000 7fdb98d06640 1 --2- 172.21.15.16:0/4192514900 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9415db50 0x7fdb94157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.371+0000 7fdb98d06640 1 -- 172.21.15.16:0/4192514900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9415df30 msgr2=0x7fdb940c5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.371+0000 7fdb98d06640 1 --2- 172.21.15.16:0/4192514900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9415df30 0x7fdb940c5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.371+0000 7fdb98d06640 1 -- 172.21.15.16:0/4192514900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb941591d0 con 0x7fdb940d12a0 2024-04-25T10:14:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.371+0000 7fdb98d06640 1 --2- 172.21.15.16:0/4192514900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb940d12a0 0x7fdb9409b8d0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00cb40 tx=0x7fdb8c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.371+0000 7fdb917fa640 1 -- 172.21.15.16:0/4192514900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c017070 con 0x7fdb940d12a0 2024-04-25T10:14:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.371+0000 7fdb917fa640 1 -- 172.21.15.16:0/4192514900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c00ac30 con 0x7fdb940d12a0 2024-04-25T10:14:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.371+0000 7fdb9af6a640 1 -- 172.21.15.16:0/4192514900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb941594b0 con 0x7fdb940d12a0 2024-04-25T10:14:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.371+0000 7fdb917fa640 1 -- 172.21.15.16:0/4192514900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c012620 con 0x7fdb940d12a0 2024-04-25T10:14:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.371+0000 7fdb9af6a640 1 -- 172.21.15.16:0/4192514900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb94159a70 con 0x7fdb940d12a0 2024-04-25T10:14:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.372+0000 7fdb917fa640 1 -- 172.21.15.16:0/4192514900 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb8c01a070 con 0x7fdb940d12a0 2024-04-25T10:14:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.373+0000 7fdb917fa640 1 --2- 172.21.15.16:0/4192514900 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb8006b500 0x7fdb8006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.373+0000 7fdb917fa640 1 -- 172.21.15.16:0/4192514900 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fdb8c089eb0 con 0x7fdb940d12a0 2024-04-25T10:14:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.373+0000 7fdb93fff640 1 --2- 172.21.15.16:0/4192514900 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb8006b500 0x7fdb8006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.374+0000 7fdb93fff640 1 --2- 172.21.15.16:0/4192514900 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb8006b500 0x7fdb8006d9c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdb88034000 tx=0x7fdb8802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.374+0000 7fdb9dc73740 1 Processor -- start 2024-04-25T10:14:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.374+0000 7fdb9dc73740 1 -- start start 2024-04-25T10:14:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb9dc73740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d6710eb10 0x555d67111320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb9dc73740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eef0 0x555d67111860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb9dc73740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d670e1770 0x555d670e4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb9dc73740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d6704ada0 con 0x555d6710eef0 2024-04-25T10:14:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb9dc73740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d66ff3b80 con 0x555d670e1770 2024-04-25T10:14:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb9dc73740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d67046e40 con 0x555d6710eb10 2024-04-25T10:14:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eef0 0x555d67111860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eef0 0x555d67111860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41662/0 (socket says 172.21.15.16:41662) 2024-04-25T10:14:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb93fff640 1 -- 172.21.15.16:0/971385698 learned_addr learned my addr 172.21.15.16:0/971385698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb98d06640 1 --2- 172.21.15.16:0/971385698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d6710eb10 0x555d67111320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb99507640 1 --2- 172.21.15.16:0/971385698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d670e1770 0x555d670e4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb93fff640 1 -- 172.21.15.16:0/971385698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d6710eb10 msgr2=0x555d67111320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:10.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb93fff640 1 --2- 172.21.15.16:0/971385698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d6710eb10 0x555d67111320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb93fff640 1 -- 172.21.15.16:0/971385698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d670e1770 msgr2=0x555d670e4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb93fff640 1 --2- 172.21.15.16:0/971385698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d670e1770 0x555d670e4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb93fff640 1 -- 172.21.15.16:0/971385698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d66ffdf10 con 0x555d6710eef0 2024-04-25T10:14:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb99507640 1 --2- 172.21.15.16:0/971385698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d670e1770 0x555d670e4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb98d06640 1 --2- 172.21.15.16:0/971385698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d6710eb10 0x555d67111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.375+0000 7fdb93fff640 1 --2- 172.21.15.16:0/971385698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eef0 0x555d67111860 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fdb8802a770 tx=0x7fdb8802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9db707ae4002ffa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.376+0000 7fdb7a7fc640 1 -- 172.21.15.16:0/971385698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88037040 con 0x555d6710eef0 2024-04-25T10:14:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.376+0000 7fdb7a7fc640 1 -- 172.21.15.16:0/971385698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8804d070 con 0x555d6710eef0 2024-04-25T10:14:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.376+0000 7fdb7a7fc640 1 -- 172.21.15.16:0/971385698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88032dd0 con 0x555d6710eef0 2024-04-25T10:14:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.376+0000 7fdb9dc73740 1 -- 172.21.15.16:0/971385698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eef0 msgr2=0x555d67111860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.376+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/971385698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eef0 0x555d67111860 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fdb8802a770 tx=0x7fdb8802c000 comp rx=0 tx=0).stop 2024-04-25T10:14:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.376+0000 7fdb9dc73740 1 -- 172.21.15.16:0/971385698 shutdown_connections 2024-04-25T10:14:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.376+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/971385698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d670e1770 0x555d670e4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.376+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/971385698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eef0 0x555d67111860 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.376+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/971385698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d6710eb10 0x555d67111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.376+0000 7fdb9dc73740 1 -- 172.21.15.16:0/971385698 >> 172.21.15.16:0/971385698 conn(0x555d670e39e0 msgr2=0x555d6712c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:10.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.376+0000 7fdb9dc73740 1 -- 172.21.15.16:0/971385698 shutdown_connections 2024-04-25T10:14:10.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.376+0000 7fdb9dc73740 1 -- 172.21.15.16:0/971385698 wait complete. 2024-04-25T10:14:10.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.377+0000 7fdb9dc73740 1 -- start start 2024-04-25T10:14:10.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.377+0000 7fdb9dc73740 1 Processor -- start 2024-04-25T10:14:10.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.378+0000 7fdb9dc73740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d6710eb10 0x555d67107900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.378+0000 7fdb9dc73740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eef0 0x555d67108000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.378+0000 7fdb9dc73740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d67109c10 0x555d6710a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.378+0000 7fdb9dc73740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d670e6b90 con 0x555d6710eef0 2024-04-25T10:14:10.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.378+0000 7fdb9dc73740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d66fe94b0 con 0x555d6710eb10 2024-04-25T10:14:10.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.378+0000 7fdb9dc73740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d670e6e70 con 0x555d67109c10 2024-04-25T10:14:10.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.378+0000 7fdb9dc73740 10 client.0 _ll_register_callbacks cb 0x555d67029180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:10.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.378+0000 7fdb93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eef0 0x555d67108000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.378+0000 7fdb93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eef0 0x555d67108000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41672/0 (socket says 172.21.15.16:41672) 2024-04-25T10:14:10.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.378+0000 7fdb93fff640 1 -- 172.21.15.16:0/2246528349 learned_addr learned my addr 172.21.15.16:0/2246528349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:10.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.378+0000 7fdb99507640 1 --2- 172.21.15.16:0/2246528349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d67109c10 0x555d6710a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.378+0000 7fdb98d06640 1 --2- 172.21.15.16:0/2246528349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d6710eb10 0x555d67107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.378+0000 7fdb93fff640 1 -- 172.21.15.16:0/2246528349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d67109c10 msgr2=0x555d6710a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:10.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.378+0000 7fdb93fff640 1 --2- 172.21.15.16:0/2246528349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d67109c10 0x555d6710a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.378+0000 7fdb93fff640 1 -- 172.21.15.16:0/2246528349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d6710eb10 msgr2=0x555d67107900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:10.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.378+0000 7fdb93fff640 1 --2- 172.21.15.16:0/2246528349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d6710eb10 0x555d67107900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.378+0000 7fdb93fff640 1 -- 172.21.15.16:0/2246528349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d66e297d0 con 0x555d6710eef0 2024-04-25T10:14:10.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.379+0000 7fdb93fff640 1 --2- 172.21.15.16:0/2246528349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eef0 0x555d67108000 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fdb8802a1c0 tx=0x7fdb88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde27ba1c8be72cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.379+0000 7fdb737fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:14:10.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.379+0000 7fdb737fe640 1 -- 172.21.15.16:0/2246528349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88058060 con 0x555d6710eef0 2024-04-25T10:14:10.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.379+0000 7fdb737fe640 1 -- 172.21.15.16:0/2246528349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fdb8803c850 con 0x555d6710eef0 2024-04-25T10:14:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.379+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2246528349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb8804d070 con 0x555d6710eef0 2024-04-25T10:14:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.379+0000 7fdb737fe640 1 -- 172.21.15.16:0/2246528349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb8805e050 con 0x555d6710eef0 2024-04-25T10:14:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.379+0000 7fdb9dc73740 10 client.8667 fetch_fsmap learned FSMap version 76 2024-04-25T10:14:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.379+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2246528349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x555d66d5f3f0 con 0x555d6710eef0 2024-04-25T10:14:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.379+0000 7fdb737fe640 1 -- 172.21.15.16:0/2246528349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8804d070 con 0x555d6710eef0 2024-04-25T10:14:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.379+0000 7fdb737fe640 1 -- 172.21.15.16:0/2246528349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdb88061020 con 0x555d6710eef0 2024-04-25T10:14:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.379+0000 7fdb9dc73740 10 client.8667 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T10:14:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.379+0000 7fdb9dc73740 10 client.8667 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:14:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.379+0000 7fdb9dc73740 20 client.8667 populate_metadata read hostname 'smithi016' 2024-04-25T10:14:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.379+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/2246528349 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x555d6713f970 0x555d6713fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.379+0000 7fdb9dc73740 4 client.8667 mds_command: new command op to 36575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.379+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2246528349 --> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d670582e0 con 0x555d6713f970 2024-04-25T10:14:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.380+0000 7fdb98d06640 1 --2- 172.21.15.16:0/2246528349 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x555d6713f970 0x555d6713fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.380+0000 7fdb737fe640 10 client.8667 ms_handle_connect on v2:172.21.15.186:6834/2008057690 2024-04-25T10:14:10.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.380+0000 7fdb98d06640 1 --2- 172.21.15.16:0/2246528349 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x555d6713f970 0x555d6713fd80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baa4e4575485946f server_cookie=100e0658d14db2d9 in_seq=0 out_seq=0 2024-04-25T10:14:10.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.381+0000 7fdb737fe640 1 -- 172.21.15.16:0/2246528349 <== mds.0 v2:172.21.15.186:6834/2008057690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555d670582e0 con 0x555d6713f970 2024-04-25T10:14:10.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.381+0000 7fdb737fe640 10 client.8667 handle_command_reply: tid=0 2024-04-25T10:14:10.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.381+0000 7fdb9dc73740 1 client.8667 shutdown 2024-04-25T10:14:10.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.381+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2246528349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eef0 msgr2=0x555d67108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.381+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/2246528349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eef0 0x555d67108000 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fdb8802a1c0 tx=0x7fdb88004730 comp rx=0 tx=0).stop 2024-04-25T10:14:10.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.381+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2246528349 shutdown_connections 2024-04-25T10:14:10.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.381+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/2246528349 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x555d6713f970 0x555d6713fd80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.381+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/2246528349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d67109c10 0x555d6710a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.381+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/2246528349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eef0 0x555d67108000 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.381+0000 7fdb737fe640 0 client.8667 ms_handle_reset on v2:172.21.15.186:6834/2008057690 2024-04-25T10:14:10.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.381+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/2246528349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d6710eb10 0x555d67107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.381+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2246528349 >> 172.21.15.16:0/2246528349 conn(0x555d670e1770 msgr2=0x555d6710ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:10.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.381+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2246528349 shutdown_connections 2024-04-25T10:14:10.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.381+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2246528349 wait complete. 2024-04-25T10:14:10.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.381+0000 7fdb9dc73740 20 client.8667 trim_cache size 0 max 16384 2024-04-25T10:14:10.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.396+0000 7fdb9dc73740 1 Processor -- start 2024-04-25T10:14:10.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.396+0000 7fdb9dc73740 1 -- start start 2024-04-25T10:14:10.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.397+0000 7fdb9dc73740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d67109c10 0x555d670e1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.397+0000 7fdb9dc73740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d6710eb10 0x555d670e8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.397+0000 7fdb9dc73740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d6710eef0 0x555d67107900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.397+0000 7fdb9dc73740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb941ffdb0 con 0x555d67109c10 2024-04-25T10:14:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.397+0000 7fdb9dc73740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb941fefa0 con 0x555d6710eef0 2024-04-25T10:14:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.397+0000 7fdb9dc73740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb941fe3a0 con 0x555d6710eb10 2024-04-25T10:14:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.397+0000 7fdb98d06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d67109c10 0x555d670e1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.397+0000 7fdb98d06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d67109c10 0x555d670e1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41688/0 (socket says 172.21.15.16:41688) 2024-04-25T10:14:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.397+0000 7fdb98d06640 1 -- 172.21.15.16:0/2980592507 learned_addr learned my addr 172.21.15.16:0/2980592507 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.397+0000 7fdb93fff640 1 --2- 172.21.15.16:0/2980592507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d6710eb10 0x555d670e8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.397+0000 7fdb99507640 1 --2- 172.21.15.16:0/2980592507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d6710eef0 0x555d67107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.397+0000 7fdb98d06640 1 -- 172.21.15.16:0/2980592507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d6710eb10 msgr2=0x555d670e8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.398+0000 7fdb98d06640 1 --2- 172.21.15.16:0/2980592507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d6710eb10 0x555d670e8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.398+0000 7fdb98d06640 1 -- 172.21.15.16:0/2980592507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d6710eef0 msgr2=0x555d67107900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.398+0000 7fdb98d06640 1 --2- 172.21.15.16:0/2980592507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d6710eef0 0x555d67107900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.398+0000 7fdb98d06640 1 -- 172.21.15.16:0/2980592507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d66d1f0f0 con 0x555d67109c10 2024-04-25T10:14:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.398+0000 7fdb98d06640 1 --2- 172.21.15.16:0/2980592507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d67109c10 0x555d670e1ce0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00cb10 tx=0x7fdb8c0884e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404b632d298bfa8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.398+0000 7fdb79ffb640 1 -- 172.21.15.16:0/2980592507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0569a0 con 0x555d67109c10 2024-04-25T10:14:10.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.398+0000 7fdb79ffb640 1 -- 172.21.15.16:0/2980592507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c070070 con 0x555d67109c10 2024-04-25T10:14:10.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.398+0000 7fdb79ffb640 1 -- 172.21.15.16:0/2980592507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c060870 con 0x555d67109c10 2024-04-25T10:14:10.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.398+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2980592507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d67109c10 msgr2=0x555d670e1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.398+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/2980592507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d67109c10 0x555d670e1ce0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00cb10 tx=0x7fdb8c0884e0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.398+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2980592507 shutdown_connections 2024-04-25T10:14:10.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.398+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/2980592507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d6710eef0 0x555d67107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.398+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/2980592507 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d6710eb10 0x555d670e8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.398+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/2980592507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d67109c10 0x555d670e1ce0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.398+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2980592507 >> 172.21.15.16:0/2980592507 conn(0x555d670e1770 msgr2=0x555d6710ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:10.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.398+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2980592507 shutdown_connections 2024-04-25T10:14:10.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.398+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2980592507 wait complete. 2024-04-25T10:14:10.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.399+0000 7fdb9dc73740 1 -- start start 2024-04-25T10:14:10.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.399+0000 7fdb9dc73740 1 Processor -- start 2024-04-25T10:14:10.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.400+0000 7fdb9dc73740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d67109c10 0x555d67195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.400+0000 7fdb9dc73740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eb10 0x555d6712da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.400+0000 7fdb9dc73740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d6710eef0 0x555d6713b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.400+0000 7fdb9dc73740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d670dd380 con 0x555d6710eb10 2024-04-25T10:14:10.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.400+0000 7fdb9dc73740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb941fe9a0 con 0x555d67109c10 2024-04-25T10:14:10.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.400+0000 7fdb9dc73740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d670dcb80 con 0x555d6710eef0 2024-04-25T10:14:10.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.400+0000 7fdb9dc73740 10 client.0 _ll_register_callbacks cb 0x7fdb7c0036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:10.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.400+0000 7fdb93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eb10 0x555d6712da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.400+0000 7fdb93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eb10 0x555d6712da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41704/0 (socket says 172.21.15.16:41704) 2024-04-25T10:14:10.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.400+0000 7fdb93fff640 1 -- 172.21.15.16:0/2780131426 learned_addr learned my addr 172.21.15.16:0/2780131426 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:10.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.400+0000 7fdb98d06640 1 --2- 172.21.15.16:0/2780131426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d67109c10 0x555d67195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.400+0000 7fdb99507640 1 --2- 172.21.15.16:0/2780131426 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d6710eef0 0x555d6713b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.400+0000 7fdb93fff640 1 -- 172.21.15.16:0/2780131426 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d6710eef0 msgr2=0x555d6713b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:10.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.400+0000 7fdb93fff640 1 --2- 172.21.15.16:0/2780131426 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d6710eef0 0x555d6713b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.400+0000 7fdb93fff640 1 -- 172.21.15.16:0/2780131426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d67109c10 msgr2=0x555d67195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:10.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.400+0000 7fdb93fff640 1 --2- 172.21.15.16:0/2780131426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d67109c10 0x555d67195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.400+0000 7fdb93fff640 1 -- 172.21.15.16:0/2780131426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d6712e8c0 con 0x555d6710eb10 2024-04-25T10:14:10.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.401+0000 7fdb93fff640 1 --2- 172.21.15.16:0/2780131426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eb10 0x555d6712da80 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fdb8802a770 tx=0x7fdb88008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da52b8cb707cfdc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.401+0000 7fdb737fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:14:10.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.401+0000 7fdb737fe640 1 -- 172.21.15.16:0/2780131426 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88037040 con 0x555d6710eb10 2024-04-25T10:14:10.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.401+0000 7fdb737fe640 1 -- 172.21.15.16:0/2780131426 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fdb88053460 con 0x555d6710eb10 2024-04-25T10:14:10.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.401+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2780131426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb8c070070 con 0x555d6710eb10 2024-04-25T10:14:10.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.401+0000 7fdb737fe640 1 -- 172.21.15.16:0/2780131426 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb8806e050 con 0x555d6710eb10 2024-04-25T10:14:10.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.401+0000 7fdb9dc73740 10 client.8673 fetch_fsmap learned FSMap version 76 2024-04-25T10:14:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.401+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2780131426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x555d6713aec0 con 0x555d6710eb10 2024-04-25T10:14:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.401+0000 7fdb737fe640 1 -- 172.21.15.16:0/2780131426 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c070070 con 0x555d6710eb10 2024-04-25T10:14:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.401+0000 7fdb737fe640 1 -- 172.21.15.16:0/2780131426 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdb88071020 con 0x555d6710eb10 2024-04-25T10:14:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.401+0000 7fdb9dc73740 10 client.8673 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T10:14:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.401+0000 7fdb9dc73740 10 client.8673 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:14:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.401+0000 7fdb9dc73740 20 client.8673 populate_metadata read hostname 'smithi016' 2024-04-25T10:14:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.401+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/2780131426 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x555d67107900 0x555d67107ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.401+0000 7fdb9dc73740 4 client.8673 mds_command: new command op to 36575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:14:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.401+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2780131426 --> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555d67052630 con 0x555d67107900 2024-04-25T10:14:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.401+0000 7fdb98d06640 1 --2- 172.21.15.16:0/2780131426 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x555d67107900 0x555d67107ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.402+0000 7fdb737fe640 10 client.8673 ms_handle_connect on v2:172.21.15.186:6834/2008057690 2024-04-25T10:14:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.402+0000 7fdb98d06640 1 --2- 172.21.15.16:0/2780131426 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x555d67107900 0x555d67107ce0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=236e0a180cf9da94 server_cookie=735863b21292a603 in_seq=0 out_seq=0 2024-04-25T10:14:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.402+0000 7fdb737fe640 1 -- 172.21.15.16:0/2780131426 <== mds.0 v2:172.21.15.186:6834/2008057690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555d67052630 con 0x555d67107900 2024-04-25T10:14:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.402+0000 7fdb737fe640 10 client.8673 handle_command_reply: tid=0 2024-04-25T10:14:10.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.402+0000 7fdb9dc73740 1 client.8673 shutdown 2024-04-25T10:14:10.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.402+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2780131426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eb10 msgr2=0x555d6712da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.402+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/2780131426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eb10 0x555d6712da80 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fdb8802a770 tx=0x7fdb88008030 comp rx=0 tx=0).stop 2024-04-25T10:14:10.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.402+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2780131426 shutdown_connections 2024-04-25T10:14:10.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.402+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/2780131426 >> [v2:172.21.15.186:6834/2008057690,v1:172.21.15.186:6836/2008057690] conn(0x555d67107900 0x555d67107ce0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.402+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/2780131426 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555d6710eef0 0x555d6713b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.402+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/2780131426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d6710eb10 0x555d6712da80 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.402+0000 7fdb737fe640 0 client.8673 ms_handle_reset on v2:172.21.15.186:6834/2008057690 2024-04-25T10:14:10.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.402+0000 7fdb9dc73740 1 --2- 172.21.15.16:0/2780131426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555d67109c10 0x555d67195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.402+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2780131426 >> 172.21.15.16:0/2780131426 conn(0x555d670e1770 msgr2=0x555d6710d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:10.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.402+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2780131426 shutdown_connections 2024-04-25T10:14:10.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.402+0000 7fdb9dc73740 1 -- 172.21.15.16:0/2780131426 wait complete. 2024-04-25T10:14:10.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.403+0000 7fdb9dc73740 20 client.8673 trim_cache size 0 max 16384 2024-04-25T10:14:10.414 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:14:10.414 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-04-25T10:14:10.414 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-04-25T10:14:10.414 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:14:10.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.403+0000 7fdb9af6a640 1 -- 172.21.15.16:0/4192514900 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb8006b500 msgr2=0x7fdb8006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.403+0000 7fdb9af6a640 1 --2- 172.21.15.16:0/4192514900 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb8006b500 0x7fdb8006d9c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdb88034000 tx=0x7fdb8802e000 comp rx=0 tx=0).stop 2024-04-25T10:14:10.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.403+0000 7fdb9af6a640 1 -- 172.21.15.16:0/4192514900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb940d12a0 msgr2=0x7fdb9409b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.403+0000 7fdb9af6a640 1 --2- 172.21.15.16:0/4192514900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb940d12a0 0x7fdb9409b8d0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00cb40 tx=0x7fdb8c00cf40 comp rx=0 tx=0).stop 2024-04-25T10:14:10.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.403+0000 7fdb9af6a640 1 -- 172.21.15.16:0/4192514900 shutdown_connections 2024-04-25T10:14:10.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.403+0000 7fdb9af6a640 1 --2- 172.21.15.16:0/4192514900 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb8006b500 0x7fdb8006d9c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.403+0000 7fdb9af6a640 1 --2- 172.21.15.16:0/4192514900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9415df30 0x7fdb940c5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.403+0000 7fdb9af6a640 1 --2- 172.21.15.16:0/4192514900 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb9415db50 0x7fdb94157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.403+0000 7fdb9af6a640 1 --2- 172.21.15.16:0/4192514900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb940d12a0 0x7fdb9409b8d0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.403+0000 7fdb9af6a640 1 -- 172.21.15.16:0/4192514900 >> 172.21.15.16:0/4192514900 conn(0x7fdb940b3430 msgr2=0x7fdb940b6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:10.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.403+0000 7fdb9af6a640 1 -- 172.21.15.16:0/4192514900 shutdown_connections 2024-04-25T10:14:10.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.403+0000 7fdb9af6a640 1 -- 172.21.15.16:0/4192514900 wait complete. 2024-04-25T10:14:10.418 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:14:10.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:14:10.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.617+0000 7f3ff1fb0640 1 Processor -- start 2024-04-25T10:14:10.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.617+0000 7f3ff1fb0640 1 -- start start 2024-04-25T10:14:10.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.618+0000 7f3ff1fb0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3fec15dc50 0x7f3fec070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.618+0000 7f3ff1fb0640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3fec0cd940 0x7f3fec0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.618+0000 7f3ff1fb0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fec06c220 con 0x7f3fec0cf600 2024-04-25T10:14:10.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.618+0000 7f3ff1fb0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fec0cfe00 con 0x7f3fec15dc50 2024-04-25T10:14:10.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.618+0000 7f3ff1fb0640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fec0cec10 con 0x7f3fec0cd940 2024-04-25T10:14:10.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.618+0000 7f3feb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3fec15dc50 0x7f3fec070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.618+0000 7f3febfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3fec0cf600 0x7f3fec161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59510/0 (socket says 172.21.15.16:59510) 2024-04-25T10:14:10.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.618+0000 7f3febfff640 1 -- 172.21.15.16:0/1084214041 learned_addr learned my addr 172.21.15.16:0/1084214041 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:10.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.619+0000 7f3feaffd640 1 --2- 172.21.15.16:0/1084214041 >> v2:172.21.15.186:3300/0 conn(0x7f3fec0cd940 0x7f3fec0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.619+0000 7f3feb7fe640 1 -- 172.21.15.16:0/1084214041 >> v2:172.21.15.186:3300/0 conn(0x7f3fec0cd940 msgr2=0x7f3fec0713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:10.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.619+0000 7f3feb7fe640 1 --2- 172.21.15.16:0/1084214041 >> v2:172.21.15.186:3300/0 conn(0x7f3fec0cd940 0x7f3fec0713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.619+0000 7f3feb7fe640 1 -- 172.21.15.16:0/1084214041 >> v1:172.21.15.16:6789/0 conn(0x7f3fec0cf600 legacy=0x7f3fec161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.619+0000 7f3feb7fe640 1 -- 172.21.15.16:0/1084214041 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fec0d0050 con 0x7f3fec15dc50 2024-04-25T10:14:10.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.619+0000 7f3fea7fc640 1 -- 172.21.15.16:0/1084214041 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175976768 0 0) 0x7f3fec06c220 con 0x7f3fec0cf600 2024-04-25T10:14:10.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.619+0000 7f3feb7fe640 1 --2- 172.21.15.16:0/1084214041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fec15dc50 0x7f3fec070e80 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0d2920 tx=0x7f3fdc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf7876469d7ecb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.620+0000 7f3fea7fc640 1 -- 172.21.15.16:0/1084214041 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc037070 con 0x7f3fec15dc50 2024-04-25T10:14:10.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.620+0000 7f3fea7fc640 1 -- 172.21.15.16:0/1084214041 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fdc032440 con 0x7f3fec15dc50 2024-04-25T10:14:10.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.620+0000 7f3fea7fc640 1 -- 172.21.15.16:0/1084214041 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc02ae60 con 0x7f3fec15dc50 2024-04-25T10:14:10.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.620+0000 7f3ff1fb0640 1 -- 172.21.15.16:0/1084214041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fec15dc50 msgr2=0x7f3fec070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.620+0000 7f3ff1fb0640 1 --2- 172.21.15.16:0/1084214041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fec15dc50 0x7f3fec070e80 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0d2920 tx=0x7f3fdc02a110 comp rx=0 tx=0).stop 2024-04-25T10:14:10.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.620+0000 7f3ff1fb0640 1 -- 172.21.15.16:0/1084214041 shutdown_connections 2024-04-25T10:14:10.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.620+0000 7f3ff1fb0640 1 --2- 172.21.15.16:0/1084214041 >> v2:172.21.15.186:3300/0 conn(0x7f3fec0cd940 0x7f3fec0713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.620+0000 7f3ff1fb0640 1 --2- 172.21.15.16:0/1084214041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fec15dc50 0x7f3fec070e80 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.620+0000 7f3ff1fb0640 1 -- 172.21.15.16:0/1084214041 >> 172.21.15.16:0/1084214041 conn(0x7f3fec06f430 msgr2=0x7f3fec06f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.620+0000 7f3ff1fb0640 1 -- 172.21.15.16:0/1084214041 shutdown_connections 2024-04-25T10:14:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.620+0000 7f3ff1fb0640 1 -- 172.21.15.16:0/1084214041 wait complete. 2024-04-25T10:14:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.621+0000 7f3ff1fb0640 1 Processor -- start 2024-04-25T10:14:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.622+0000 7f3ff1fb0640 1 -- start start 2024-04-25T10:14:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.622+0000 7f3ff1fb0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fec0cd940 0x7f3fec156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.622+0000 7f3ff1fb0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fec0cf600 0x7f3fec0c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.622+0000 7f3ff1fb0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3fec15dc50 0x7f3fec161660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.622+0000 7f3ff1fb0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec0ce8a0 con 0x7f3fec0cd940 2024-04-25T10:14:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.622+0000 7f3ff1fb0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec0cc0c0 con 0x7f3fec0cf600 2024-04-25T10:14:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.622+0000 7f3ff1fb0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec0cd620 con 0x7f3fec15dc50 2024-04-25T10:14:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.622+0000 7f3feb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fec0cd940 0x7f3fec156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.623+0000 7f3feb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fec0cd940 0x7f3fec156b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41730/0 (socket says 172.21.15.16:41730) 2024-04-25T10:14:10.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.623+0000 7f3feb7fe640 1 -- 172.21.15.16:0/337922 learned_addr learned my addr 172.21.15.16:0/337922 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:10.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.623+0000 7f3feaffd640 1 --2- 172.21.15.16:0/337922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fec0cf600 0x7f3fec0c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.623+0000 7f3febfff640 1 --2- 172.21.15.16:0/337922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3fec15dc50 0x7f3fec161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.623+0000 7f3feb7fe640 1 -- 172.21.15.16:0/337922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3fec15dc50 msgr2=0x7f3fec161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.623+0000 7f3feb7fe640 1 --2- 172.21.15.16:0/337922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3fec15dc50 0x7f3fec161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.623+0000 7f3feb7fe640 1 -- 172.21.15.16:0/337922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fec0cf600 msgr2=0x7f3fec0c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.623+0000 7f3feb7fe640 1 --2- 172.21.15.16:0/337922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fec0cf600 0x7f3fec0c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.623+0000 7f3feb7fe640 1 -- 172.21.15.16:0/337922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fec0c5810 con 0x7f3fec0cd940 2024-04-25T10:14:10.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.623+0000 7f3feb7fe640 1 --2- 172.21.15.16:0/337922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fec0cd940 0x7f3fec156b30 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc035040 tx=0x7f3fdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.623+0000 7f3fd3fff640 1 -- 172.21.15.16:0/337922 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc037040 con 0x7f3fec0cd940 2024-04-25T10:14:10.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.623+0000 7f3fd3fff640 1 -- 172.21.15.16:0/337922 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fdc042070 con 0x7f3fec0cd940 2024-04-25T10:14:10.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.623+0000 7f3ff1fb0640 1 -- 172.21.15.16:0/337922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fec0c5af0 con 0x7f3fec0cd940 2024-04-25T10:14:10.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.623+0000 7f3fd3fff640 1 -- 172.21.15.16:0/337922 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc030bd0 con 0x7f3fec0cd940 2024-04-25T10:14:10.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.623+0000 7f3ff1fb0640 1 -- 172.21.15.16:0/337922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fec0c60b0 con 0x7f3fec0cd940 2024-04-25T10:14:10.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.624+0000 7f3ff1fb0640 1 -- 172.21.15.16:0/337922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fec0750b0 con 0x7f3fec0cd940 2024-04-25T10:14:10.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.625+0000 7f3fd3fff640 1 -- 172.21.15.16:0/337922 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3fdc047460 con 0x7f3fec0cd940 2024-04-25T10:14:10.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.625+0000 7f3fd3fff640 1 --2- 172.21.15.16:0/337922 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3fd806b4d0 0x7f3fd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.625+0000 7f3fd3fff640 1 -- 172.21.15.16:0/337922 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f3fdc0aace0 con 0x7f3fec0cd940 2024-04-25T10:14:10.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.626+0000 7f3feaffd640 1 --2- 172.21.15.16:0/337922 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3fd806b4d0 0x7f3fd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.626+0000 7f3feaffd640 1 --2- 172.21.15.16:0/337922 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3fd806b4d0 0x7f3fd806d990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3fec157070 tx=0x7f3fe0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.630+0000 7f3fd3fff640 1 -- 172.21.15.16:0/337922 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3fdc032e20 con 0x7f3fec0cd940 2024-04-25T10:14:10.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.847+0000 7f3ff1fb0640 1 -- 172.21.15.16:0/337922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3fec06c220 con 0x7f3fec0cd940 2024-04-25T10:14:10.852 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-04-25T10:14:10.848+0000 7ff13f9bb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:14:10.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.849+0000 7f3fd3fff640 1 -- 172.21.15.16:0/337922 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v77) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3fdc079440 con 0x7f3fec0cd940 2024-04-25T10:14:10.854 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:14:10.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.852+0000 7f3ff1fb0640 1 -- 172.21.15.16:0/337922 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3fd806b4d0 msgr2=0x7f3fd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.852+0000 7f3ff1fb0640 1 --2- 172.21.15.16:0/337922 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3fd806b4d0 0x7f3fd806d990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3fec157070 tx=0x7f3fe0008040 comp rx=0 tx=0).stop 2024-04-25T10:14:10.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.852+0000 7f3ff1fb0640 1 -- 172.21.15.16:0/337922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fec0cd940 msgr2=0x7f3fec156b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.852+0000 7f3ff1fb0640 1 --2- 172.21.15.16:0/337922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fec0cd940 0x7f3fec156b30 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc035040 tx=0x7f3fdc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.852+0000 7f3ff1fb0640 1 -- 172.21.15.16:0/337922 shutdown_connections 2024-04-25T10:14:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.852+0000 7f3ff1fb0640 1 --2- 172.21.15.16:0/337922 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3fd806b4d0 0x7f3fd806d990 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.853+0000 7f3ff1fb0640 1 --2- 172.21.15.16:0/337922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3fec15dc50 0x7f3fec161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.853+0000 7f3ff1fb0640 1 --2- 172.21.15.16:0/337922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fec0cf600 0x7f3fec0c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.853+0000 7f3ff1fb0640 1 --2- 172.21.15.16:0/337922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fec0cd940 0x7f3fec156b30 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.853+0000 7f3ff1fb0640 1 -- 172.21.15.16:0/337922 >> 172.21.15.16:0/337922 conn(0x7f3fec06f430 msgr2=0x7f3fec06d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.853+0000 7f3ff1fb0640 1 -- 172.21.15.16:0/337922 shutdown_connections 2024-04-25T10:14:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.853+0000 7f3ff1fb0640 1 -- 172.21.15.16:0/337922 wait complete. 2024-04-25T10:14:10.859 INFO:tasks.ceph.mds.b.smithi186.stderr: -1165> 2024-04-25T10:13:40.076+0000 7ff20b1cf640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-25T10:14:10.859 INFO:tasks.ceph.mds.b.smithi186.stderr: -1162> 2024-04-25T10:13:40.076+0000 7ff20b1cf640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-25T10:14:10.859 INFO:tasks.ceph.mds.b.smithi186.stderr: -1141> 2024-04-25T10:13:40.076+0000 7ff20b1cf640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-25T10:14:10.859 INFO:tasks.ceph.mds.b.smithi186.stderr: -1138> 2024-04-25T10:13:40.076+0000 7ff20b1cf640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-25T10:14:10.859 INFO:tasks.ceph.mds.b.smithi186.stderr: -992> 2024-04-25T10:13:40.486+0000 7ff20f1d7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:14:10.868 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-04-25T10:14:10.885 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2024-04-25T10:14:10.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.903+0000 7f54f4b2c280 1 -- start start 2024-04-25T10:14:10.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.903+0000 7f54f4b2c280 1 Processor -- start 2024-04-25T10:14:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.903+0000 7f54f4b2c280 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556488b26c00 0x556488b96580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.903+0000 7f54f4b2c280 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556488b27800 0x556488b96b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.903+0000 7f54f4b2c280 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556488b27c00 0x556488b97080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.903+0000 7f54f4b2c280 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564898f2180 con 0x556488b27800 2024-04-25T10:14:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.903+0000 7f54f4b2c280 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564898f2300 con 0x556488b27c00 2024-04-25T10:14:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f4b2c280 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564898f2480 con 0x556488b26c00 2024-04-25T10:14:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f206c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556488b27800 0x556488b96b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f206c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556488b27800 0x556488b96b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41738/0 (socket says 172.21.15.16:41738) 2024-04-25T10:14:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f206c640 1 -- 172.21.15.16:0/454171923 learned_addr learned my addr 172.21.15.16:0/454171923 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f306e640 1 --2- 172.21.15.16:0/454171923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556488b27c00 0x556488b97080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f206c640 1 -- 172.21.15.16:0/454171923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556488b26c00 msgr2=0x556488b96580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f286d640 1 --2- 172.21.15.16:0/454171923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556488b26c00 0x556488b96580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f206c640 1 --2- 172.21.15.16:0/454171923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556488b26c00 0x556488b96580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f206c640 1 -- 172.21.15.16:0/454171923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556488b27c00 msgr2=0x556488b97080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f206c640 1 --2- 172.21.15.16:0/454171923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556488b27c00 0x556488b97080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f206c640 1 -- 172.21.15.16:0/454171923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556488b5c780 con 0x556488b27800 2024-04-25T10:14:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f306e640 1 --2- 172.21.15.16:0/454171923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556488b27c00 0x556488b97080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f206c640 1 --2- 172.21.15.16:0/454171923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556488b27800 0x556488b96b00 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x556489947200 tx=0x5564899ec420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587cfe178a28daa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f0068640 1 -- 172.21.15.16:0/454171923 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556488b4c680 con 0x556488b27800 2024-04-25T10:14:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f0068640 1 -- 172.21.15.16:0/454171923 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556488b52700 con 0x556488b27800 2024-04-25T10:14:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f0068640 1 -- 172.21.15.16:0/454171923 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556488b4cb60 con 0x556488b27800 2024-04-25T10:14:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f4b2c280 1 -- 172.21.15.16:0/454171923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556488b5d4a0 con 0x556488b27800 2024-04-25T10:14:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f4b2c280 1 -- 172.21.15.16:0/454171923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x556488b5c000 con 0x556488b27800 2024-04-25T10:14:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f4b2c280 4 waiting for MDS map... 2024-04-25T10:14:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.904+0000 7f54f0068640 1 -- 172.21.15.16:0/454171923 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(104..104 src has 1..104) v4 ==== 5997+0+0 (secure 0 0 0) 0x5564899f8600 con 0x556488b27800 2024-04-25T10:14:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.905+0000 7f54f0068640 1 -- 172.21.15.16:0/454171923 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 77) v1 ==== 2723+0+0 (secure 0 0 0) 0x556488b81180 con 0x556488b27800 2024-04-25T10:14:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.905+0000 7f54f4b2c280 4 Got MDS map 77 2024-04-25T10:14:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.905+0000 7f54f4b2c280 10 main: main 2024-04-25T10:14:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.905+0000 7f54f4b2c280 4 main: connecting to RADOS... 2024-04-25T10:14:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.905+0000 7f54f4b2c280 1 Processor -- start 2024-04-25T10:14:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.905+0000 7f54f4b2c280 1 -- start start 2024-04-25T10:14:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.906+0000 7f54f4b2c280 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564899fb000 0x556488b97600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.906+0000 7f54f4b2c280 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564899fb400 0x556488b97b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.906+0000 7f54f4b2c280 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5564899fb800 0x556488b98100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.906+0000 7f54f4b2c280 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564898f2600 con 0x5564899fb400 2024-04-25T10:14:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.906+0000 7f54f4b2c280 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564898f2780 con 0x5564899fb000 2024-04-25T10:14:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.906+0000 7f54f4b2c280 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564898f2900 con 0x5564899fb800 2024-04-25T10:14:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.906+0000 7f54f286d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564899fb400 0x556488b97b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.906+0000 7f54f286d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564899fb400 0x556488b97b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41742/0 (socket says 172.21.15.16:41742) 2024-04-25T10:14:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.906+0000 7f54f286d640 1 -- 172.21.15.16:0/1929931276 learned_addr learned my addr 172.21.15.16:0/1929931276 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.906+0000 7f54f306e640 1 --2- 172.21.15.16:0/1929931276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564899fb000 0x556488b97600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.906+0000 7f54f206c640 1 --2- 172.21.15.16:0/1929931276 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5564899fb800 0x556488b98100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.906+0000 7f54f286d640 1 -- 172.21.15.16:0/1929931276 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5564899fb800 msgr2=0x556488b98100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.906+0000 7f54f286d640 1 --2- 172.21.15.16:0/1929931276 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5564899fb800 0x556488b98100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.906+0000 7f54f286d640 1 -- 172.21.15.16:0/1929931276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564899fb000 msgr2=0x556488b97600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.906+0000 7f54f286d640 1 --2- 172.21.15.16:0/1929931276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564899fb000 0x556488b97600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.906+0000 7f54f286d640 1 -- 172.21.15.16:0/1929931276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556488b5c3c0 con 0x5564899fb400 2024-04-25T10:14:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.906+0000 7f54f286d640 1 --2- 172.21.15.16:0/1929931276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564899fb400 0x556488b97b80 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x556489a6c000 tx=0x5564899ec240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadb2c0aaae82fa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.906+0000 7f54ee064640 1 -- 172.21.15.16:0/1929931276 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556488b4c9c0 con 0x5564899fb400 2024-04-25T10:14:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.907+0000 7f54ee064640 1 -- 172.21.15.16:0/1929931276 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556488b52fc0 con 0x5564899fb400 2024-04-25T10:14:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.907+0000 7f54ee064640 1 -- 172.21.15.16:0/1929931276 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556488b4c680 con 0x5564899fb400 2024-04-25T10:14:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.907+0000 7f54f4b2c280 1 -- 172.21.15.16:0/1929931276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564899fb400 msgr2=0x556488b97b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.907+0000 7f54f4b2c280 1 --2- 172.21.15.16:0/1929931276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564899fb400 0x556488b97b80 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x556489a6c000 tx=0x5564899ec240 comp rx=0 tx=0).stop 2024-04-25T10:14:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.907+0000 7f54f4b2c280 1 -- 172.21.15.16:0/1929931276 shutdown_connections 2024-04-25T10:14:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.907+0000 7f54f4b2c280 1 --2- 172.21.15.16:0/1929931276 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5564899fb800 0x556488b98100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.907+0000 7f54f4b2c280 1 --2- 172.21.15.16:0/1929931276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564899fb400 0x556488b97b80 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.907+0000 7f54f4b2c280 1 --2- 172.21.15.16:0/1929931276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564899fb000 0x556488b97600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.907+0000 7f54f4b2c280 1 -- 172.21.15.16:0/1929931276 >> 172.21.15.16:0/1929931276 conn(0x5564899fac00 msgr2=0x556488c0f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.907+0000 7f54f4b2c280 1 -- 172.21.15.16:0/1929931276 shutdown_connections 2024-04-25T10:14:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.907+0000 7f54f4b2c280 1 -- 172.21.15.16:0/1929931276 wait complete. 2024-04-25T10:14:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.907+0000 7f54f4b2c280 1 Processor -- start 2024-04-25T10:14:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.908+0000 7f54f4b2c280 1 -- start start 2024-04-25T10:14:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.908+0000 7f54f4b2c280 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5564899fb000 0x556488b99180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.908+0000 7f54f4b2c280 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564899fb400 0x556488b98c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.908+0000 7f54f4b2c280 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564899fb800 0x556488b98680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.908+0000 7f54f4b2c280 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564898f3b00 con 0x5564899fb800 2024-04-25T10:14:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.908+0000 7f54f4b2c280 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564898f3980 con 0x5564899fb400 2024-04-25T10:14:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.908+0000 7f54f4b2c280 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564898f3800 con 0x5564899fb000 2024-04-25T10:14:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.908+0000 7f54f206c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564899fb800 0x556488b98680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.908+0000 7f54f206c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564899fb800 0x556488b98680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41746/0 (socket says 172.21.15.16:41746) 2024-04-25T10:14:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.908+0000 7f54f206c640 1 -- 172.21.15.16:0/654108611 learned_addr learned my addr 172.21.15.16:0/654108611 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.908+0000 7f54f306e640 1 --2- 172.21.15.16:0/654108611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5564899fb000 0x556488b99180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.908+0000 7f54f286d640 1 --2- 172.21.15.16:0/654108611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564899fb400 0x556488b98c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.909+0000 7f54f206c640 1 -- 172.21.15.16:0/654108611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5564899fb000 msgr2=0x556488b99180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.909+0000 7f54f206c640 1 --2- 172.21.15.16:0/654108611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5564899fb000 0x556488b99180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.909+0000 7f54f206c640 1 -- 172.21.15.16:0/654108611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564899fb400 msgr2=0x556488b98c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.909+0000 7f54f206c640 1 --2- 172.21.15.16:0/654108611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564899fb400 0x556488b98c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.909+0000 7f54f206c640 1 -- 172.21.15.16:0/654108611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556488b5c780 con 0x5564899fb800 2024-04-25T10:14:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.909+0000 7f54f286d640 1 --2- 172.21.15.16:0/654108611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564899fb400 0x556488b98c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.909+0000 7f54f306e640 1 --2- 172.21.15.16:0/654108611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5564899fb000 0x556488b99180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.909+0000 7f54f206c640 1 --2- 172.21.15.16:0/654108611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564899fb800 0x556488b98680 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x556489a6d470 tx=0x5564899ec060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.909+0000 7f54ec861640 1 -- 172.21.15.16:0/654108611 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556488b4c820 con 0x5564899fb800 2024-04-25T10:14:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.909+0000 7f54ec861640 1 -- 172.21.15.16:0/654108611 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556488b52e00 con 0x5564899fb800 2024-04-25T10:14:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.909+0000 7f54ec861640 1 -- 172.21.15.16:0/654108611 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556488b4c9c0 con 0x5564899fb800 2024-04-25T10:14:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.909+0000 7f54f4b2c280 1 -- 172.21.15.16:0/654108611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556488b5d4a0 con 0x5564899fb800 2024-04-25T10:14:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.909+0000 7f54f4b2c280 1 -- 172.21.15.16:0/654108611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556488b5c960 con 0x5564899fb800 2024-04-25T10:14:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.909+0000 7f54f4b2c280 4 main: resolving pool 14 2024-04-25T10:14:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.910+0000 7f54ec861640 1 -- 172.21.15.16:0/654108611 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x556489918c00 con 0x5564899fb800 2024-04-25T10:14:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.910+0000 7f54ec861640 1 --2- 172.21.15.16:0/654108611 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x556488e88000 0x556488b99700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.910+0000 7f54ec861640 1 -- 172.21.15.16:0/654108611 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5997+0+0 (secure 0 0 0) 0x556489920600 con 0x5564899fb800 2024-04-25T10:14:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.910+0000 7f54f4b2c280 4 main: creating IoCtx.. 2024-04-25T10:14:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.911+0000 7f54f4b2c280 1 --2- 172.21.15.16:0/654108611 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x556488e88400 0x556488b98100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.911+0000 7f54f4b2c280 1 -- 172.21.15.16:0/654108611 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104) v9 -- 0x556488e88800 con 0x556488e88400 2024-04-25T10:14:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.911+0000 7f54f306e640 1 --2- 172.21.15.16:0/654108611 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x556488e88000 0x556488b99700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.911+0000 7f54f286d640 1 --2- 172.21.15.16:0/654108611 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x556488e88400 0x556488b98100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.911+0000 7f54f306e640 1 --2- 172.21.15.16:0/654108611 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x556488e88000 0x556488b99700 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x556489947170 tx=0x556489a6b140 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.911+0000 7f54f286d640 1 --2- 172.21.15.16:0/654108611 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x556488e88400 0x556488b98100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f286d640 1 -- 172.21.15.16:0/654108611 <== osd.5 v2:172.21.15.186:6800/1827108421 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x556488c00000 con 0x556488e88400 2024-04-25T10:14:10.923 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:14:10.923 INFO:teuthology.orchestra.run.smithi016.stdout: "0": { 2024-04-25T10:14:10.923 INFO:teuthology.orchestra.run.smithi016.stdout: "data": { 2024-04-25T10:14:10.923 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 5, 2024-04-25T10:14:10.923 INFO:teuthology.orchestra.run.smithi016.stdout: "inotable": { 2024-04-25T10:14:10.923 INFO:teuthology.orchestra.run.smithi016.stdout: "projected_free": [ 2024-04-25T10:14:10.924 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:10.924 INFO:teuthology.orchestra.run.smithi016.stdout: "start": 1099511627779, 2024-04-25T10:14:10.924 INFO:teuthology.orchestra.run.smithi016.stdout: "len": 1099511627773 2024-04-25T10:14:10.924 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:10.924 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-25T10:14:10.924 INFO:teuthology.orchestra.run.smithi016.stdout: "free": [ 2024-04-25T10:14:10.924 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:10.924 INFO:teuthology.orchestra.run.smithi016.stdout: "start": 1099511627779, 2024-04-25T10:14:10.924 INFO:teuthology.orchestra.run.smithi016.stdout: "len": 1099511627773 2024-04-25T10:14:10.924 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:10.924 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-04-25T10:14:10.924 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:10.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:10.924 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0 2024-04-25T10:14:10.925 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:10.925 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:14:10.925 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 -- 172.21.15.16:0/654108611 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x556488e88400 msgr2=0x556488b98100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 --2- 172.21.15.16:0/654108611 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x556488e88400 0x556488b98100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 -- 172.21.15.16:0/654108611 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x556488e88000 msgr2=0x556488b99700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 --2- 172.21.15.16:0/654108611 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x556488e88000 0x556488b99700 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x556489947170 tx=0x556489a6b140 comp rx=0 tx=0).stop 2024-04-25T10:14:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 -- 172.21.15.16:0/654108611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564899fb800 msgr2=0x556488b98680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 --2- 172.21.15.16:0/654108611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564899fb800 0x556488b98680 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x556489a6d470 tx=0x5564899ec060 comp rx=0 tx=0).stop 2024-04-25T10:14:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f286d640 1 -- 172.21.15.16:0/654108611 reap_dead start 2024-04-25T10:14:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 -- 172.21.15.16:0/654108611 shutdown_connections 2024-04-25T10:14:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 -- 172.21.15.16:0/654108611 >> 172.21.15.16:0/654108611 conn(0x5564899fac00 msgr2=0x556488c0f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 -- 172.21.15.16:0/654108611 shutdown_connections 2024-04-25T10:14:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 -- 172.21.15.16:0/654108611 wait complete. 2024-04-25T10:14:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 -- 172.21.15.16:0/454171923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556488b27800 msgr2=0x556488b96b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 --2- 172.21.15.16:0/454171923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556488b27800 0x556488b96b00 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x556489947200 tx=0x5564899ec420 comp rx=0 tx=0).stop 2024-04-25T10:14:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 -- 172.21.15.16:0/454171923 shutdown_connections 2024-04-25T10:14:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 --2- 172.21.15.16:0/454171923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556488b27c00 0x556488b97080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 --2- 172.21.15.16:0/454171923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556488b27800 0x556488b96b00 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 --2- 172.21.15.16:0/454171923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x556488b26c00 0x556488b96580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 -- 172.21.15.16:0/454171923 >> 172.21.15.16:0/454171923 conn(0x556488b27000 msgr2=0x556488c0e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 -- 172.21.15.16:0/454171923 shutdown_connections 2024-04-25T10:14:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:10.914+0000 7f54f4b2c280 1 -- 172.21.15.16:0/454171923 wait complete. 2024-04-25T10:14:10.929 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.061038, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-04-25T10:14:10.929 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:14:10.929 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:11.007 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:11.049 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:14:11.050 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:14:11.050 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:11.066 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:14:11.066 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:14:11.066 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:11.106 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:11.106 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:14:11.106 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:11.122 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:14:11.123 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:14:11.123 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:11.162 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:11.162 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.259+0000 7f6fad84d640 1 Processor -- start 2024-04-25T10:14:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.259+0000 7f6fad84d640 1 -- start start 2024-04-25T10:14:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.260+0000 7f6fad84d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fa815dce0 0x7f6fa80d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.260+0000 7f6fad84d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6fa80b5b60 0x7f6fa80b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.260+0000 7f6fad84d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa815e0c0 con 0x7f6fa80d35b0 2024-04-25T10:14:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.260+0000 7f6fad84d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa80d2510 con 0x7f6fa815dce0 2024-04-25T10:14:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.260+0000 7f6fad84d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa80d27f0 con 0x7f6fa80b5b60 2024-04-25T10:14:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.260+0000 7f6fa6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fa815dce0 0x7f6fa80d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.260+0000 7f6fa6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fa815dce0 0x7f6fa80d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41758/0 (socket says 172.21.15.16:41758) 2024-04-25T10:14:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.260+0000 7f6fa6ffd640 1 -- 172.21.15.16:0/3882854136 learned_addr learned my addr 172.21.15.16:0/3882854136 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.260+0000 7f6f9ffff640 1 --1- 172.21.15.16:0/3882854136 >> v1:172.21.15.16:6789/0 conn(0x7f6fa80d35b0 0x7f6fa80b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59522/0 (socket says 172.21.15.16:59522) 2024-04-25T10:14:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.260+0000 7f6fa77fe640 1 --2- 172.21.15.16:0/3882854136 >> v2:172.21.15.186:3300/0 conn(0x7f6fa80b5b60 0x7f6fa80b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.260+0000 7f6fa6ffd640 1 -- 172.21.15.16:0/3882854136 >> v2:172.21.15.186:3300/0 conn(0x7f6fa80b5b60 msgr2=0x7f6fa80b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:11.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.260+0000 7f6fa6ffd640 1 --2- 172.21.15.16:0/3882854136 >> v2:172.21.15.186:3300/0 conn(0x7f6fa80b5b60 0x7f6fa80b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.260+0000 7f6fa6ffd640 1 -- 172.21.15.16:0/3882854136 >> v1:172.21.15.16:6789/0 conn(0x7f6fa80d35b0 legacy=0x7f6fa80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.260+0000 7f6fa6ffd640 1 -- 172.21.15.16:0/3882854136 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa80d2ad0 con 0x7f6fa815dce0 2024-04-25T10:14:11.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.260+0000 7f6fa67fc640 1 -- 172.21.15.16:0/3882854136 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4138546705 0 0) 0x7f6fa815e0c0 con 0x7f6fa80d35b0 2024-04-25T10:14:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.261+0000 7f6fa6ffd640 1 --2- 172.21.15.16:0/3882854136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa815dce0 0x7f6fa80d1a50 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80b3390 tx=0x7f6f9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d677f388604eab70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.261+0000 7f6fa67fc640 1 -- 172.21.15.16:0/3882854136 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f98037070 con 0x7f6fa815dce0 2024-04-25T10:14:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.261+0000 7f6fa67fc640 1 -- 172.21.15.16:0/3882854136 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f98032440 con 0x7f6fa815dce0 2024-04-25T10:14:11.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.261+0000 7f6fa67fc640 1 -- 172.21.15.16:0/3882854136 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f98032810 con 0x7f6fa815dce0 2024-04-25T10:14:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.262+0000 7f6fad84d640 1 -- 172.21.15.16:0/3882854136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa815dce0 msgr2=0x7f6fa80d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.262+0000 7f6fad84d640 1 --2- 172.21.15.16:0/3882854136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa815dce0 0x7f6fa80d1a50 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80b3390 tx=0x7f6f9802a270 comp rx=0 tx=0).stop 2024-04-25T10:14:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.262+0000 7f6fad84d640 1 -- 172.21.15.16:0/3882854136 shutdown_connections 2024-04-25T10:14:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.262+0000 7f6fad84d640 1 --2- 172.21.15.16:0/3882854136 >> v2:172.21.15.186:3300/0 conn(0x7f6fa80b5b60 0x7f6fa80b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.262+0000 7f6fad84d640 1 --2- 172.21.15.16:0/3882854136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa815dce0 0x7f6fa80d1a50 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.262+0000 7f6fad84d640 1 -- 172.21.15.16:0/3882854136 >> 172.21.15.16:0/3882854136 conn(0x7f6fa8072900 msgr2=0x7f6fa80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.262+0000 7f6fad84d640 1 -- 172.21.15.16:0/3882854136 shutdown_connections 2024-04-25T10:14:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.262+0000 7f6fad84d640 1 -- 172.21.15.16:0/3882854136 wait complete. 2024-04-25T10:14:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.263+0000 7f6fad84d640 1 Processor -- start 2024-04-25T10:14:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.263+0000 7f6fad84d640 1 -- start start 2024-04-25T10:14:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.264+0000 7f6fad84d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fa80b5b60 0x7f6fa81467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.264+0000 7f6fad84d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fa80d35b0 0x7f6fa80ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.264+0000 7f6fad84d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa815dce0 0x7f6fa81f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.264+0000 7f6fad84d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa80b9820 con 0x7f6fa815dce0 2024-04-25T10:14:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.264+0000 7f6fad84d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa80d3090 con 0x7f6fa80b5b60 2024-04-25T10:14:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.264+0000 7f6fad84d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa80b99a0 con 0x7f6fa80d35b0 2024-04-25T10:14:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.264+0000 7f6fa77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa815dce0 0x7f6fa81f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.264+0000 7f6fa77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa815dce0 0x7f6fa81f34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41762/0 (socket says 172.21.15.16:41762) 2024-04-25T10:14:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.264+0000 7f6fa77fe640 1 -- 172.21.15.16:0/2580805907 learned_addr learned my addr 172.21.15.16:0/2580805907 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.264+0000 7f6fa6ffd640 1 --2- 172.21.15.16:0/2580805907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fa80b5b60 0x7f6fa81467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.264+0000 7f6fa77fe640 1 -- 172.21.15.16:0/2580805907 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fa80d35b0 msgr2=0x7f6fa80ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.264+0000 7f6fa77fe640 1 --2- 172.21.15.16:0/2580805907 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fa80d35b0 0x7f6fa80ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.264+0000 7f6fa77fe640 1 -- 172.21.15.16:0/2580805907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fa80b5b60 msgr2=0x7f6fa81467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.264+0000 7f6fa77fe640 1 --2- 172.21.15.16:0/2580805907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fa80b5b60 0x7f6fa81467c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.264+0000 7f6fa77fe640 1 -- 172.21.15.16:0/2580805907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa81f3ec0 con 0x7f6fa815dce0 2024-04-25T10:14:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.264+0000 7f6fa77fe640 1 --2- 172.21.15.16:0/2580805907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa815dce0 0x7f6fa81f34e0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0007e40 tx=0x7f6fa000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.265+0000 7f6fa4ff9640 1 -- 172.21.15.16:0/2580805907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0014620 con 0x7f6fa815dce0 2024-04-25T10:14:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.265+0000 7f6fa4ff9640 1 -- 172.21.15.16:0/2580805907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa0019070 con 0x7f6fa815dce0 2024-04-25T10:14:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.265+0000 7f6fad84d640 1 -- 172.21.15.16:0/2580805907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa81f41a0 con 0x7f6fa815dce0 2024-04-25T10:14:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.265+0000 7f6fa4ff9640 1 -- 172.21.15.16:0/2580805907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0014a20 con 0x7f6fa815dce0 2024-04-25T10:14:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.265+0000 7f6fad84d640 1 -- 172.21.15.16:0/2580805907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa81462a0 con 0x7f6fa815dce0 2024-04-25T10:14:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.266+0000 7f6fad84d640 1 -- 172.21.15.16:0/2580805907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa80749b0 con 0x7f6fa815dce0 2024-04-25T10:14:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.266+0000 7f6fa4ff9640 1 -- 172.21.15.16:0/2580805907 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fa0014bc0 con 0x7f6fa815dce0 2024-04-25T10:14:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.267+0000 7f6fa4ff9640 1 --2- 172.21.15.16:0/2580805907 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f8806b500 0x7f6f8806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.267+0000 7f6fa4ff9640 1 -- 172.21.15.16:0/2580805907 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f6fa008ab50 con 0x7f6fa815dce0 2024-04-25T10:14:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.267+0000 7f6fa6ffd640 1 --2- 172.21.15.16:0/2580805907 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f8806b500 0x7f6f8806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.267+0000 7f6fa6ffd640 1 --2- 172.21.15.16:0/2580805907 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f8806b500 0x7f6f8806d9c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6f98034000 tx=0x7f6f9802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.271+0000 7f6fa4ff9640 1 -- 172.21.15.16:0/2580805907 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6fa0059940 con 0x7f6fa815dce0 2024-04-25T10:14:11.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.488+0000 7f6fad84d640 1 -- 172.21.15.16:0/2580805907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fa8146470 con 0x7f6fa815dce0 2024-04-25T10:14:11.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.488+0000 7f6fa4ff9640 1 -- 172.21.15.16:0/2580805907 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4140 (secure 0 0 0) 0x7f6fa0059240 con 0x7f6fa815dce0 2024-04-25T10:14:11.493 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:11.493 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7243,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/109406141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":109406141},{"type":"v1","addr":"172.21.15.186:6837","nonce":109406141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2704401503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2704401503},{"type":"v1","addr":"172.21.15.16:6835","nonce":2704401503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/939221717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":939221717},{"type":"v1","addr":"172.21.15.16:6834","nonce":939221717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":77,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:48.958938+0000","modified":"2024-04-25T10:14:10.849139+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:14:11.493 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 77 2024-04-25T10:14:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.490+0000 7f6fad84d640 1 -- 172.21.15.16:0/2580805907 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f8806b500 msgr2=0x7f6f8806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.490+0000 7f6fad84d640 1 --2- 172.21.15.16:0/2580805907 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f8806b500 0x7f6f8806d9c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6f98034000 tx=0x7f6f9802e000 comp rx=0 tx=0).stop 2024-04-25T10:14:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.490+0000 7f6fad84d640 1 -- 172.21.15.16:0/2580805907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa815dce0 msgr2=0x7f6fa81f34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.490+0000 7f6fad84d640 1 --2- 172.21.15.16:0/2580805907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa815dce0 0x7f6fa81f34e0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0007e40 tx=0x7f6fa000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.490+0000 7f6fad84d640 1 -- 172.21.15.16:0/2580805907 shutdown_connections 2024-04-25T10:14:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.490+0000 7f6fad84d640 1 --2- 172.21.15.16:0/2580805907 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6f8806b500 0x7f6f8806d9c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.491+0000 7f6fad84d640 1 --2- 172.21.15.16:0/2580805907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa815dce0 0x7f6fa81f34e0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.491+0000 7f6fad84d640 1 --2- 172.21.15.16:0/2580805907 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fa80d35b0 0x7f6fa80ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.491+0000 7f6fad84d640 1 --2- 172.21.15.16:0/2580805907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fa80b5b60 0x7f6fa81467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.491+0000 7f6fad84d640 1 -- 172.21.15.16:0/2580805907 >> 172.21.15.16:0/2580805907 conn(0x7f6fa8072900 msgr2=0x7f6fa80b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:11.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.491+0000 7f6fad84d640 1 -- 172.21.15.16:0/2580805907 shutdown_connections 2024-04-25T10:14:11.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.491+0000 7f6fad84d640 1 -- 172.21.15.16:0/2580805907 wait complete. 2024-04-25T10:14:11.506 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:11.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.602+0000 7fe0e62e4640 1 Processor -- start 2024-04-25T10:14:11.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.602+0000 7fe0e62e4640 1 -- start start 2024-04-25T10:14:11.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.603+0000 7fe0e62e4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0e00d35b0 0x7fe0e00b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.603+0000 7fe0e62e4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe0e00b5990 0x7fe0e00b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.603+0000 7fe0e62e4640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0e015e0c0 con 0x7fe0e015dce0 2024-04-25T10:14:11.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.603+0000 7fe0e62e4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0e00b5de0 con 0x7fe0e00b5990 2024-04-25T10:14:11.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.603+0000 7fe0e62e4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e00d2510 con 0x7fe0e00d35b0 2024-04-25T10:14:11.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.603+0000 7fe0e4881640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe0e00b5990 0x7fe0e00b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.603+0000 7fe0e4881640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe0e00b5990 0x7fe0e00b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41770/0 (socket says 172.21.15.16:41770) 2024-04-25T10:14:11.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.603+0000 7fe0e4881640 1 -- 172.21.15.16:0/3016345610 learned_addr learned my addr 172.21.15.16:0/3016345610 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:11.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.604+0000 7fe0df7fe640 1 --2- 172.21.15.16:0/3016345610 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0e00d35b0 0x7fe0e00b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.604+0000 7fe0e4881640 1 -- 172.21.15.16:0/3016345610 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0e00d35b0 msgr2=0x7fe0e00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:11.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.604+0000 7fe0e4881640 1 --2- 172.21.15.16:0/3016345610 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0e00d35b0 0x7fe0e00b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.604+0000 7fe0e4881640 1 -- 172.21.15.16:0/3016345610 >> v1:172.21.15.186:6789/0 conn(0x7fe0e015dce0 legacy=0x7fe0e00d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.604+0000 7fe0e4881640 1 -- 172.21.15.16:0/3016345610 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e00d27f0 con 0x7fe0e00b5990 2024-04-25T10:14:11.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.604+0000 7fe0e4881640 1 --2- 172.21.15.16:0/3016345610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e00b5990 0x7fe0e00b6b00 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009e50 tx=0x7fe0d802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4a89d0f2158ded server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.604+0000 7fe0deffd640 1 -- 172.21.15.16:0/3016345610 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8036070 con 0x7fe0e00b5990 2024-04-25T10:14:11.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.605+0000 7fe0deffd640 1 -- 172.21.15.16:0/3016345610 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d8031600 con 0x7fe0e00b5990 2024-04-25T10:14:11.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.605+0000 7fe0deffd640 1 -- 172.21.15.16:0/3016345610 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d80319d0 con 0x7fe0e00b5990 2024-04-25T10:14:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.605+0000 7fe0e62e4640 1 -- 172.21.15.16:0/3016345610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e00b5990 msgr2=0x7fe0e00b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.605+0000 7fe0e62e4640 1 --2- 172.21.15.16:0/3016345610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e00b5990 0x7fe0e00b6b00 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009e50 tx=0x7fe0d802a710 comp rx=0 tx=0).stop 2024-04-25T10:14:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.605+0000 7fe0e62e4640 1 -- 172.21.15.16:0/3016345610 shutdown_connections 2024-04-25T10:14:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.605+0000 7fe0e62e4640 1 --2- 172.21.15.16:0/3016345610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e00b5990 0x7fe0e00b6b00 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.605+0000 7fe0e62e4640 1 --2- 172.21.15.16:0/3016345610 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0e00d35b0 0x7fe0e00b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.605+0000 7fe0e62e4640 1 -- 172.21.15.16:0/3016345610 >> 172.21.15.16:0/3016345610 conn(0x7fe0e0072900 msgr2=0x7fe0e00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.605+0000 7fe0e62e4640 1 -- 172.21.15.16:0/3016345610 shutdown_connections 2024-04-25T10:14:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.605+0000 7fe0e62e4640 1 -- 172.21.15.16:0/3016345610 wait complete. 2024-04-25T10:14:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.606+0000 7fe0e62e4640 1 Processor -- start 2024-04-25T10:14:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.606+0000 7fe0e62e4640 1 -- start start 2024-04-25T10:14:11.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.607+0000 7fe0e62e4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0e00b5990 0x7fe0e0152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.607+0000 7fe0e62e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e00d35b0 0x7fe0e00b7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.607+0000 7fe0e62e4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0e015dce0 0x7fe0e00b7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.607+0000 7fe0e62e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e00d3090 con 0x7fe0e00d35b0 2024-04-25T10:14:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.607+0000 7fe0e62e4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e00d2db0 con 0x7fe0e00b5990 2024-04-25T10:14:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.607+0000 7fe0e62e4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e00d2160 con 0x7fe0e015dce0 2024-04-25T10:14:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.607+0000 7fe0df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e00d35b0 0x7fe0e00b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.607+0000 7fe0df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e00d35b0 0x7fe0e00b7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41784/0 (socket says 172.21.15.16:41784) 2024-04-25T10:14:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.607+0000 7fe0df7fe640 1 -- 172.21.15.16:0/1016195177 learned_addr learned my addr 172.21.15.16:0/1016195177 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.607+0000 7fe0dffff640 1 --2- 172.21.15.16:0/1016195177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0e00b5990 0x7fe0e0152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.607+0000 7fe0e4881640 1 --2- 172.21.15.16:0/1016195177 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0e015dce0 0x7fe0e00b7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.608+0000 7fe0df7fe640 1 -- 172.21.15.16:0/1016195177 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0e015dce0 msgr2=0x7fe0e00b7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.608+0000 7fe0df7fe640 1 --2- 172.21.15.16:0/1016195177 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0e015dce0 0x7fe0e00b7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.608+0000 7fe0df7fe640 1 -- 172.21.15.16:0/1016195177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0e00b5990 msgr2=0x7fe0e0152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.608+0000 7fe0df7fe640 1 --2- 172.21.15.16:0/1016195177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0e00b5990 0x7fe0e0152200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.608+0000 7fe0df7fe640 1 -- 172.21.15.16:0/1016195177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e0152740 con 0x7fe0e00d35b0 2024-04-25T10:14:11.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.608+0000 7fe0df7fe640 1 --2- 172.21.15.16:0/1016195177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e00d35b0 0x7fe0e00b7570 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400d830 tx=0x7fe0d400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.608+0000 7fe0dcff9640 1 -- 172.21.15.16:0/1016195177 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400b6c0 con 0x7fe0e00d35b0 2024-04-25T10:14:11.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.608+0000 7fe0dcff9640 1 -- 172.21.15.16:0/1016195177 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d4019070 con 0x7fe0e00d35b0 2024-04-25T10:14:11.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.608+0000 7fe0dcff9640 1 -- 172.21.15.16:0/1016195177 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400ba80 con 0x7fe0e00d35b0 2024-04-25T10:14:11.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.608+0000 7fe0e62e4640 1 -- 172.21.15.16:0/1016195177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e00c81f0 con 0x7fe0e00d35b0 2024-04-25T10:14:11.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.608+0000 7fe0e62e4640 1 -- 172.21.15.16:0/1016195177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e00c87b0 con 0x7fe0e00d35b0 2024-04-25T10:14:11.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.609+0000 7fe0e62e4640 1 -- 172.21.15.16:0/1016195177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e00749b0 con 0x7fe0e00d35b0 2024-04-25T10:14:11.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.609+0000 7fe0dcff9640 1 -- 172.21.15.16:0/1016195177 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0d401e460 con 0x7fe0e00d35b0 2024-04-25T10:14:11.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.610+0000 7fe0dcff9640 1 --2- 172.21.15.16:0/1016195177 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe0cc06b4d0 0x7fe0cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.610+0000 7fe0dcff9640 1 -- 172.21.15.16:0/1016195177 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fe0d408a8b0 con 0x7fe0e00d35b0 2024-04-25T10:14:11.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.610+0000 7fe0dffff640 1 --2- 172.21.15.16:0/1016195177 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe0cc06b4d0 0x7fe0cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.611+0000 7fe0dffff640 1 --2- 172.21.15.16:0/1016195177 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe0cc06b4d0 0x7fe0cc06d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe0e00cda50 tx=0x7fe0d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.614+0000 7fe0dcff9640 1 -- 172.21.15.16:0/1016195177 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0d4058fc0 con 0x7fe0e00d35b0 2024-04-25T10:14:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.796+0000 7fe0e62e4640 1 -- 172.21.15.16:0/1016195177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0e00d15a0 con 0x7fe0e00d35b0 2024-04-25T10:14:11.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.797+0000 7fe0dcff9640 1 -- 172.21.15.16:0/1016195177 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+17068 (secure 0 0 0) 0x7fe0d405d060 con 0x7fe0e00d35b0 2024-04-25T10:14:11.802 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:11.802 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":104,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:14:10.849151+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:47.551100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:47.939289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":5,"to":3}]},{"pgid":"14.b","mappings":[{"from":6,"to":0}]},{"pgid":"14.e","mappings":[{"from":5,"to":0}]},{"pgid":"14.16","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.186:6836/2008057690":"2024-04-26T10:14:10.849128+0000","172.21.15.186:6834/2008057690":"2024-04-26T10:14:10.849128+0000","172.21.15.186:6836/1371188714":"2024-04-26T10:13:34.499700+0000","172.21.15.186:6834/1371188714":"2024-04-26T10:13:34.499700+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:11.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.799+0000 7fe0e62e4640 1 -- 172.21.15.16:0/1016195177 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe0cc06b4d0 msgr2=0x7fe0cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.799+0000 7fe0e62e4640 1 --2- 172.21.15.16:0/1016195177 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe0cc06b4d0 0x7fe0cc06d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe0e00cda50 tx=0x7fe0d0009040 comp rx=0 tx=0).stop 2024-04-25T10:14:11.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.799+0000 7fe0e62e4640 1 -- 172.21.15.16:0/1016195177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e00d35b0 msgr2=0x7fe0e00b7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.799+0000 7fe0e62e4640 1 --2- 172.21.15.16:0/1016195177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e00d35b0 0x7fe0e00b7570 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400d830 tx=0x7fe0d400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.799+0000 7fe0e62e4640 1 -- 172.21.15.16:0/1016195177 shutdown_connections 2024-04-25T10:14:11.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.799+0000 7fe0e62e4640 1 --2- 172.21.15.16:0/1016195177 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe0cc06b4d0 0x7fe0cc06d990 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.799+0000 7fe0e62e4640 1 --2- 172.21.15.16:0/1016195177 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0e015dce0 0x7fe0e00b7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.799+0000 7fe0e62e4640 1 --2- 172.21.15.16:0/1016195177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e00d35b0 0x7fe0e00b7570 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.799+0000 7fe0e62e4640 1 --2- 172.21.15.16:0/1016195177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0e00b5990 0x7fe0e0152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.799+0000 7fe0e62e4640 1 -- 172.21.15.16:0/1016195177 >> 172.21.15.16:0/1016195177 conn(0x7fe0e0072900 msgr2=0x7fe0e00c6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.800+0000 7fe0e62e4640 1 -- 172.21.15.16:0/1016195177 shutdown_connections 2024-04-25T10:14:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.800+0000 7fe0e62e4640 1 -- 172.21.15.16:0/1016195177 wait complete. 2024-04-25T10:14:11.815 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:47.551100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:47.939289+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:11.815 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:11.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.916+0000 7f931cd0a640 1 Processor -- start 2024-04-25T10:14:11.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.916+0000 7f931cd0a640 1 -- start start 2024-04-25T10:14:11.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.917+0000 7f931cd0a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f931815dca0 0x7f93180ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.917+0000 7f931cd0a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93180d1130 0x7f93180b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.917+0000 7f931cd0a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93180d4090 con 0x7f93180d3b90 2024-04-25T10:14:11.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.917+0000 7f931cd0a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93180b6b20 con 0x7f931815dca0 2024-04-25T10:14:11.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.917+0000 7f931cd0a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93180d4370 con 0x7f93180d1130 2024-04-25T10:14:11.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.917+0000 7f930ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f93180d3b90 0x7f93180d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59530/0 (socket says 172.21.15.16:59530) 2024-04-25T10:14:11.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.917+0000 7f930ffff640 1 -- 172.21.15.16:0/1566970465 learned_addr learned my addr 172.21.15.16:0/1566970465 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:11.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.917+0000 7f9316d9d640 1 --2- 172.21.15.16:0/1566970465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93180d1130 0x7f93180b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.917+0000 7f931659c640 1 --2- 172.21.15.16:0/1566970465 >> v2:172.21.15.186:3300/0 conn(0x7f931815dca0 0x7f93180ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.917+0000 7f9315d9b640 1 -- 172.21.15.16:0/1566970465 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228987262 0 0) 0x7f93180d4090 con 0x7f93180d3b90 2024-04-25T10:14:11.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.917+0000 7f9315d9b640 1 -- 172.21.15.16:0/1566970465 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9304003520 con 0x7f93180d3b90 2024-04-25T10:14:11.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.917+0000 7f931659c640 1 -- 172.21.15.16:0/1566970465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93180d1130 msgr2=0x7f93180b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:11.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.917+0000 7f931659c640 1 --2- 172.21.15.16:0/1566970465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93180d1130 0x7f93180b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.917+0000 7f931659c640 1 -- 172.21.15.16:0/1566970465 >> v1:172.21.15.16:6789/0 conn(0x7f93180d3b90 legacy=0x7f93180d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.917+0000 7f931659c640 1 -- 172.21.15.16:0/1566970465 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93180d4650 con 0x7f931815dca0 2024-04-25T10:14:11.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.917+0000 7f9316d9d640 1 --2- 172.21.15.16:0/1566970465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93180d1130 0x7f93180b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:11.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.918+0000 7f931659c640 1 --2- 172.21.15.16:0/1566970465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f931815dca0 0x7f93180ce940 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f93080097a0 tx=0x7f930802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a11b72919da917c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.918+0000 7f9315d9b640 1 -- 172.21.15.16:0/1566970465 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308037070 con 0x7f931815dca0 2024-04-25T10:14:11.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.918+0000 7f9315d9b640 1 -- 172.21.15.16:0/1566970465 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9308032440 con 0x7f931815dca0 2024-04-25T10:14:11.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.918+0000 7f9315d9b640 1 -- 172.21.15.16:0/1566970465 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308032810 con 0x7f931815dca0 2024-04-25T10:14:11.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.919+0000 7f931cd0a640 1 -- 172.21.15.16:0/1566970465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f931815dca0 msgr2=0x7f93180ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.919+0000 7f931cd0a640 1 --2- 172.21.15.16:0/1566970465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f931815dca0 0x7f93180ce940 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f93080097a0 tx=0x7f930802a180 comp rx=0 tx=0).stop 2024-04-25T10:14:11.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.919+0000 7f931cd0a640 1 -- 172.21.15.16:0/1566970465 shutdown_connections 2024-04-25T10:14:11.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.919+0000 7f931cd0a640 1 --2- 172.21.15.16:0/1566970465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93180d1130 0x7f93180b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.919+0000 7f931cd0a640 1 --2- 172.21.15.16:0/1566970465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f931815dca0 0x7f93180ce940 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.919+0000 7f931cd0a640 1 -- 172.21.15.16:0/1566970465 >> 172.21.15.16:0/1566970465 conn(0x7f93180726c0 msgr2=0x7f9318072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:11.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.919+0000 7f931cd0a640 1 -- 172.21.15.16:0/1566970465 shutdown_connections 2024-04-25T10:14:11.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.919+0000 7f931cd0a640 1 -- 172.21.15.16:0/1566970465 wait complete. 2024-04-25T10:14:11.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.920+0000 7f931cd0a640 1 Processor -- start 2024-04-25T10:14:11.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.920+0000 7f931cd0a640 1 -- start start 2024-04-25T10:14:11.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.921+0000 7f931cd0a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93180d1130 0x7f9318160340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.921+0000 7f931cd0a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93180d3b90 0x7f9318154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.921+0000 7f931cd0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931815dca0 0x7f931815f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.921+0000 7f931cd0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931815e200 con 0x7f931815dca0 2024-04-25T10:14:11.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.921+0000 7f931cd0a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931815e080 con 0x7f93180d3b90 2024-04-25T10:14:11.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.921+0000 7f931cd0a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93180cee80 con 0x7f93180d1130 2024-04-25T10:14:11.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.921+0000 7f9316d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931815dca0 0x7f931815f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.921+0000 7f9316d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931815dca0 0x7f931815f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41788/0 (socket says 172.21.15.16:41788) 2024-04-25T10:14:11.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.921+0000 7f9316d9d640 1 -- 172.21.15.16:0/2740142189 learned_addr learned my addr 172.21.15.16:0/2740142189 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:11.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.921+0000 7f9316d9d640 1 -- 172.21.15.16:0/2740142189 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93180d1130 msgr2=0x7f9318160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.921+0000 7f9316d9d640 1 --2- 172.21.15.16:0/2740142189 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93180d1130 0x7f9318160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.921+0000 7f9316d9d640 1 -- 172.21.15.16:0/2740142189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93180d3b90 msgr2=0x7f9318154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.921+0000 7f930ffff640 1 --2- 172.21.15.16:0/2740142189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93180d3b90 0x7f9318154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.921+0000 7f9316d9d640 1 --2- 172.21.15.16:0/2740142189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93180d3b90 0x7f9318154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.921+0000 7f9316d9d640 1 -- 172.21.15.16:0/2740142189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9318155440 con 0x7f931815dca0 2024-04-25T10:14:11.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.921+0000 7f9316d9d640 1 --2- 172.21.15.16:0/2740142189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931815dca0 0x7f931815f9b0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f9310009f00 tx=0x7f931000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.922+0000 7f930effd640 1 -- 172.21.15.16:0/2740142189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310018030 con 0x7f931815dca0 2024-04-25T10:14:11.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.922+0000 7f930effd640 1 -- 172.21.15.16:0/2740142189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931000ce50 con 0x7f931815dca0 2024-04-25T10:14:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.922+0000 7f931cd0a640 1 -- 172.21.15.16:0/2740142189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93181f3b00 con 0x7f931815dca0 2024-04-25T10:14:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.922+0000 7f930effd640 1 -- 172.21.15.16:0/2740142189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310003bb0 con 0x7f931815dca0 2024-04-25T10:14:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.922+0000 7f931cd0a640 1 -- 172.21.15.16:0/2740142189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93181f40c0 con 0x7f931815dca0 2024-04-25T10:14:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.923+0000 7f931cd0a640 1 -- 172.21.15.16:0/2740142189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93180750b0 con 0x7f931815dca0 2024-04-25T10:14:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.923+0000 7f930effd640 1 -- 172.21.15.16:0/2740142189 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9310005d10 con 0x7f931815dca0 2024-04-25T10:14:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.923+0000 7f930effd640 1 --2- 172.21.15.16:0/2740142189 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f92f806b4d0 0x7f92f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.924+0000 7f930effd640 1 -- 172.21.15.16:0/2740142189 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f931008acc0 con 0x7f931815dca0 2024-04-25T10:14:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.924+0000 7f931659c640 1 --2- 172.21.15.16:0/2740142189 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f92f806b4d0 0x7f92f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.924+0000 7f931659c640 1 --2- 172.21.15.16:0/2740142189 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f92f806b4d0 0x7f92f806d990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9308034000 tx=0x7f930802c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:11.928+0000 7f930effd640 1 -- 172.21.15.16:0/2740142189 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9310022070 con 0x7f931815dca0 2024-04-25T10:14:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.107+0000 7f931cd0a640 1 -- 172.21.15.16:0/2740142189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f93180cdb90 con 0x7f931815dca0 2024-04-25T10:14:12.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.107+0000 7f930effd640 1 -- 172.21.15.16:0/2740142189 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9310010040 con 0x7f931815dca0 2024-04-25T10:14:12.112 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:14:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.109+0000 7f931cd0a640 1 -- 172.21.15.16:0/2740142189 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f92f806b4d0 msgr2=0x7f92f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.109+0000 7f931cd0a640 1 --2- 172.21.15.16:0/2740142189 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f92f806b4d0 0x7f92f806d990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9308034000 tx=0x7f930802c000 comp rx=0 tx=0).stop 2024-04-25T10:14:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.109+0000 7f931cd0a640 1 -- 172.21.15.16:0/2740142189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931815dca0 msgr2=0x7f931815f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.109+0000 7f931cd0a640 1 --2- 172.21.15.16:0/2740142189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931815dca0 0x7f931815f9b0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f9310009f00 tx=0x7f931000c490 comp rx=0 tx=0).stop 2024-04-25T10:14:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.109+0000 7f931cd0a640 1 -- 172.21.15.16:0/2740142189 shutdown_connections 2024-04-25T10:14:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.109+0000 7f931cd0a640 1 --2- 172.21.15.16:0/2740142189 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f92f806b4d0 0x7f92f806d990 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.109+0000 7f931cd0a640 1 --2- 172.21.15.16:0/2740142189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931815dca0 0x7f931815f9b0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.109+0000 7f931cd0a640 1 --2- 172.21.15.16:0/2740142189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93180d3b90 0x7f9318154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.109+0000 7f931cd0a640 1 --2- 172.21.15.16:0/2740142189 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f93180d1130 0x7f9318160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.109+0000 7f931cd0a640 1 -- 172.21.15.16:0/2740142189 >> 172.21.15.16:0/2740142189 conn(0x7f93180726c0 msgr2=0x7f93180b7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:12.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.109+0000 7f931cd0a640 1 -- 172.21.15.16:0/2740142189 shutdown_connections 2024-04-25T10:14:12.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.109+0000 7f931cd0a640 1 -- 172.21.15.16:0/2740142189 wait complete. 2024-04-25T10:14:12.124 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.223+0000 7fec2f1f5640 1 Processor -- start 2024-04-25T10:14:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.224+0000 7fec2f1f5640 1 -- start start 2024-04-25T10:14:12.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.224+0000 7fec2f1f5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fec2815a260 0x7fec28071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.224+0000 7fec2f1f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec28071760 0x7fec2806e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.224+0000 7fec2f1f5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec2815dc30 con 0x7fec2806ea30 2024-04-25T10:14:12.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.224+0000 7fec2f1f5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec28071bc0 con 0x7fec2815a260 2024-04-25T10:14:12.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.224+0000 7fec2f1f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2815df10 con 0x7fec28071760 2024-04-25T10:14:12.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.225+0000 7fec2cf91640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fec2815a260 0x7fec28071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.225+0000 7fec2cf91640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fec2815a260 0x7fec28071220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41804/0 (socket says 172.21.15.16:41804) 2024-04-25T10:14:12.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.225+0000 7fec2cf91640 1 -- 172.21.15.16:0/2285918520 learned_addr learned my addr 172.21.15.16:0/2285918520 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:12.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.225+0000 7fec27fff640 1 --2- 172.21.15.16:0/2285918520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec28071760 0x7fec2806e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.225+0000 7fec2cf91640 1 -- 172.21.15.16:0/2285918520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec28071760 msgr2=0x7fec2806e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.225+0000 7fec2cf91640 1 --2- 172.21.15.16:0/2285918520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec28071760 0x7fec2806e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.225+0000 7fec2cf91640 1 -- 172.21.15.16:0/2285918520 >> v1:172.21.15.186:6789/0 conn(0x7fec2806ea30 legacy=0x7fec2806d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.225+0000 7fec2cf91640 1 -- 172.21.15.16:0/2285918520 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2815e1f0 con 0x7fec2815a260 2024-04-25T10:14:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.226+0000 7fec2cf91640 1 --2- 172.21.15.16:0/2285918520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec2815a260 0x7fec28071220 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fec20009e50 tx=0x7fec2002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc34fda3251f1c12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.226+0000 7fec277fe640 1 -- 172.21.15.16:0/2285918520 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20036070 con 0x7fec2815a260 2024-04-25T10:14:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.226+0000 7fec277fe640 1 -- 172.21.15.16:0/2285918520 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec20031600 con 0x7fec2815a260 2024-04-25T10:14:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.226+0000 7fec277fe640 1 -- 172.21.15.16:0/2285918520 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec200319d0 con 0x7fec2815a260 2024-04-25T10:14:12.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.226+0000 7fec2f1f5640 1 -- 172.21.15.16:0/2285918520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec2815a260 msgr2=0x7fec28071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.226+0000 7fec2f1f5640 1 --2- 172.21.15.16:0/2285918520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec2815a260 0x7fec28071220 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fec20009e50 tx=0x7fec2002a6c0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.226+0000 7fec2f1f5640 1 -- 172.21.15.16:0/2285918520 shutdown_connections 2024-04-25T10:14:12.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.226+0000 7fec2f1f5640 1 --2- 172.21.15.16:0/2285918520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec28071760 0x7fec2806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.226+0000 7fec2f1f5640 1 --2- 172.21.15.16:0/2285918520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec2815a260 0x7fec28071220 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.226+0000 7fec2f1f5640 1 -- 172.21.15.16:0/2285918520 >> 172.21.15.16:0/2285918520 conn(0x7fec2806aba0 msgr2=0x7fec2806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:12.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.226+0000 7fec2f1f5640 1 -- 172.21.15.16:0/2285918520 shutdown_connections 2024-04-25T10:14:12.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.226+0000 7fec2f1f5640 1 -- 172.21.15.16:0/2285918520 wait complete. 2024-04-25T10:14:12.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.228+0000 7fec2f1f5640 1 Processor -- start 2024-04-25T10:14:12.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.228+0000 7fec2f1f5640 1 -- start start 2024-04-25T10:14:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.228+0000 7fec2f1f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec2806ea30 0x7fec2814ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.228+0000 7fec2f1f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec28071760 0x7fec280cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.228+0000 7fec2f1f5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec2815a260 0x7fec280b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.228+0000 7fec2f1f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2806eec0 con 0x7fec28071760 2024-04-25T10:14:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.229+0000 7fec2f1f5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2815e7b0 con 0x7fec2815a260 2024-04-25T10:14:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.229+0000 7fec2f1f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2806f040 con 0x7fec2806ea30 2024-04-25T10:14:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.229+0000 7fec27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec28071760 0x7fec280cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.229+0000 7fec27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec28071760 0x7fec280cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41812/0 (socket says 172.21.15.16:41812) 2024-04-25T10:14:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.229+0000 7fec27fff640 1 -- 172.21.15.16:0/839608493 learned_addr learned my addr 172.21.15.16:0/839608493 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.229+0000 7fec27fff640 1 -- 172.21.15.16:0/839608493 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec2806ea30 msgr2=0x7fec2814ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.229+0000 7fec2cf91640 1 --2- 172.21.15.16:0/839608493 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec2806ea30 0x7fec2814ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.229+0000 7fec2d792640 1 --2- 172.21.15.16:0/839608493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec2815a260 0x7fec280b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.229+0000 7fec27fff640 1 --2- 172.21.15.16:0/839608493 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec2806ea30 0x7fec2814ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.229+0000 7fec27fff640 1 -- 172.21.15.16:0/839608493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec2815a260 msgr2=0x7fec280b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.229+0000 7fec27fff640 1 --2- 172.21.15.16:0/839608493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec2815a260 0x7fec280b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.229+0000 7fec27fff640 1 -- 172.21.15.16:0/839608493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec280acad0 con 0x7fec28071760 2024-04-25T10:14:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.229+0000 7fec27fff640 1 --2- 172.21.15.16:0/839608493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec28071760 0x7fec280cd690 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fec1000c7f0 tx=0x7fec1000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.229+0000 7fec257fa640 1 -- 172.21.15.16:0/839608493 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec10015070 con 0x7fec28071760 2024-04-25T10:14:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.229+0000 7fec257fa640 1 -- 172.21.15.16:0/839608493 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec10004240 con 0x7fec28071760 2024-04-25T10:14:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.229+0000 7fec2f1f5640 1 -- 172.21.15.16:0/839608493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec280acdb0 con 0x7fec28071760 2024-04-25T10:14:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.229+0000 7fec257fa640 1 -- 172.21.15.16:0/839608493 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec10007620 con 0x7fec28071760 2024-04-25T10:14:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.229+0000 7fec2f1f5640 1 -- 172.21.15.16:0/839608493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec280ad370 con 0x7fec28071760 2024-04-25T10:14:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.230+0000 7fec2f1f5640 1 -- 172.21.15.16:0/839608493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec280750b0 con 0x7fec28071760 2024-04-25T10:14:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.231+0000 7fec257fa640 1 -- 172.21.15.16:0/839608493 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec100079d0 con 0x7fec28071760 2024-04-25T10:14:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.231+0000 7fec257fa640 1 --2- 172.21.15.16:0/839608493 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fec0806b4d0 0x7fec0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.231+0000 7fec257fa640 1 -- 172.21.15.16:0/839608493 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fec1008a6c0 con 0x7fec28071760 2024-04-25T10:14:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.232+0000 7fec2cf91640 1 --2- 172.21.15.16:0/839608493 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fec0806b4d0 0x7fec0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.232+0000 7fec2cf91640 1 --2- 172.21.15.16:0/839608493 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fec0806b4d0 0x7fec0806d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fec20033000 tx=0x7fec20009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.235+0000 7fec257fa640 1 -- 172.21.15.16:0/839608493 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec100594b0 con 0x7fec28071760 2024-04-25T10:14:12.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.412+0000 7fec2f1f5640 1 -- 172.21.15.16:0/839608493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fec28071540 con 0x7fec28071760 2024-04-25T10:14:12.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.413+0000 7fec257fa640 1 -- 172.21.15.16:0/839608493 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) v1 ==== 104+0+11 (secure 0 0 0) 0x7fec1005de20 con 0x7fec28071760 2024-04-25T10:14:12.417 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:14:12.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.414+0000 7fec2f1f5640 1 -- 172.21.15.16:0/839608493 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fec0806b4d0 msgr2=0x7fec0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.414+0000 7fec2f1f5640 1 --2- 172.21.15.16:0/839608493 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fec0806b4d0 0x7fec0806d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fec20033000 tx=0x7fec20009000 comp rx=0 tx=0).stop 2024-04-25T10:14:12.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.414+0000 7fec2f1f5640 1 -- 172.21.15.16:0/839608493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec28071760 msgr2=0x7fec280cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.414+0000 7fec2f1f5640 1 --2- 172.21.15.16:0/839608493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec28071760 0x7fec280cd690 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fec1000c7f0 tx=0x7fec1000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.414+0000 7fec2f1f5640 1 -- 172.21.15.16:0/839608493 shutdown_connections 2024-04-25T10:14:12.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.414+0000 7fec2f1f5640 1 --2- 172.21.15.16:0/839608493 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fec0806b4d0 0x7fec0806d990 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.414+0000 7fec2f1f5640 1 --2- 172.21.15.16:0/839608493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec2815a260 0x7fec280b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.414+0000 7fec2f1f5640 1 --2- 172.21.15.16:0/839608493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec28071760 0x7fec280cd690 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.414+0000 7fec2f1f5640 1 --2- 172.21.15.16:0/839608493 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec2806ea30 0x7fec2814ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.414+0000 7fec2f1f5640 1 -- 172.21.15.16:0/839608493 >> 172.21.15.16:0/839608493 conn(0x7fec2806aba0 msgr2=0x7fec280ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:12.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.414+0000 7fec2f1f5640 1 -- 172.21.15.16:0/839608493 shutdown_connections 2024-04-25T10:14:12.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.414+0000 7fec2f1f5640 1 -- 172.21.15.16:0/839608493 wait complete. 2024-04-25T10:14:12.429 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.529+0000 7ff40b46f640 1 Processor -- start 2024-04-25T10:14:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.529+0000 7ff40b46f640 1 -- start start 2024-04-25T10:14:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.530+0000 7ff40b46f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff40415dca0 0x7ff4040ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.530+0000 7ff40b46f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4040d4090 con 0x7ff4040d3b90 2024-04-25T10:14:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.530+0000 7ff40b46f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4040d4370 con 0x7ff4040d1130 2024-04-25T10:14:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.530+0000 7ff40b46f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4040b6b20 con 0x7ff40415dca0 2024-04-25T10:14:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.530+0000 7ff408a0a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff4040d3b90 0x7ff4040d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59542/0 (socket says 172.21.15.16:59542) 2024-04-25T10:14:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.530+0000 7ff408a0a640 1 -- 172.21.15.16:0/3934763297 learned_addr learned my addr 172.21.15.16:0/3934763297 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.530+0000 7ff40920b640 1 --2- 172.21.15.16:0/3934763297 >> v2:172.21.15.186:3300/0 conn(0x7ff40415dca0 0x7ff4040ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.530+0000 7ff3fbfff640 1 -- 172.21.15.16:0/3934763297 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153027787 0 0) 0x7ff4040d4090 con 0x7ff4040d3b90 2024-04-25T10:14:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.530+0000 7ff3fbfff640 1 -- 172.21.15.16:0/3934763297 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3f0003520 con 0x7ff4040d3b90 2024-04-25T10:14:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.530+0000 7ff3fbfff640 1 -- 172.21.15.16:0/3934763297 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1099380159 0 0) 0x7ff3f0003520 con 0x7ff4040d3b90 2024-04-25T10:14:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.530+0000 7ff40920b640 1 -- 172.21.15.16:0/3934763297 >> v1:172.21.15.186:6789/0 conn(0x7ff4040d1130 legacy=0x7ff4040b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.530+0000 7ff40920b640 1 -- 172.21.15.16:0/3934763297 >> v1:172.21.15.16:6789/0 conn(0x7ff4040d3b90 legacy=0x7ff4040d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.530+0000 7ff40920b640 1 -- 172.21.15.16:0/3934763297 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4040d4650 con 0x7ff40415dca0 2024-04-25T10:14:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.531+0000 7ff40920b640 1 --2- 172.21.15.16:0/3934763297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff40415dca0 0x7ff4040ce940 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff4000097a0 tx=0x7ff40002a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bd905e69ac24e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.531+0000 7ff3fbfff640 1 -- 172.21.15.16:0/3934763297 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400037070 con 0x7ff40415dca0 2024-04-25T10:14:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.531+0000 7ff3fbfff640 1 -- 172.21.15.16:0/3934763297 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40002ab60 con 0x7ff40415dca0 2024-04-25T10:14:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.531+0000 7ff3fbfff640 1 -- 172.21.15.16:0/3934763297 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4001670 con 0x7ff40415dca0 2024-04-25T10:14:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.532+0000 7ff40b46f640 1 -- 172.21.15.16:0/3934763297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff40415dca0 msgr2=0x7ff4040ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.532+0000 7ff40b46f640 1 --2- 172.21.15.16:0/3934763297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff40415dca0 0x7ff4040ce940 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff4000097a0 tx=0x7ff40002a0d0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.532+0000 7ff40b46f640 1 -- 172.21.15.16:0/3934763297 shutdown_connections 2024-04-25T10:14:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.532+0000 7ff40b46f640 1 --2- 172.21.15.16:0/3934763297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff40415dca0 0x7ff4040ce940 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.532+0000 7ff40b46f640 1 -- 172.21.15.16:0/3934763297 >> 172.21.15.16:0/3934763297 conn(0x7ff4040726c0 msgr2=0x7ff404072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.532+0000 7ff40b46f640 1 -- 172.21.15.16:0/3934763297 shutdown_connections 2024-04-25T10:14:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.532+0000 7ff40b46f640 1 -- 172.21.15.16:0/3934763297 wait complete. 2024-04-25T10:14:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.533+0000 7ff40b46f640 1 Processor -- start 2024-04-25T10:14:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.533+0000 7ff40b46f640 1 -- start start 2024-04-25T10:14:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.534+0000 7ff40b46f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4040d1130 0x7ff404152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.534+0000 7ff40b46f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4040d3b90 0x7ff4040abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.534+0000 7ff40b46f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff40415dca0 0x7ff4040cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.534+0000 7ff40b46f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40415e200 con 0x7ff4040d3b90 2024-04-25T10:14:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.534+0000 7ff40b46f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40415e080 con 0x7ff40415dca0 2024-04-25T10:14:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.534+0000 7ff40b46f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040cee80 con 0x7ff4040d1130 2024-04-25T10:14:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.534+0000 7ff408a0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4040d3b90 0x7ff4040abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.534+0000 7ff408a0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4040d3b90 0x7ff4040abb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41828/0 (socket says 172.21.15.16:41828) 2024-04-25T10:14:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.534+0000 7ff408a0a640 1 -- 172.21.15.16:0/3224745131 learned_addr learned my addr 172.21.15.16:0/3224745131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.534+0000 7ff409a0c640 1 --2- 172.21.15.16:0/3224745131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff40415dca0 0x7ff4040cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.534+0000 7ff40920b640 1 --2- 172.21.15.16:0/3224745131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4040d1130 0x7ff404152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.534+0000 7ff408a0a640 1 -- 172.21.15.16:0/3224745131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4040d1130 msgr2=0x7ff404152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.534+0000 7ff408a0a640 1 --2- 172.21.15.16:0/3224745131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4040d1130 0x7ff404152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.534+0000 7ff408a0a640 1 -- 172.21.15.16:0/3224745131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff40415dca0 msgr2=0x7ff4040cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.534+0000 7ff408a0a640 1 --2- 172.21.15.16:0/3224745131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff40415dca0 0x7ff4040cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.534+0000 7ff408a0a640 1 -- 172.21.15.16:0/3224745131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4040cd680 con 0x7ff4040d3b90 2024-04-25T10:14:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.534+0000 7ff409a0c640 1 --2- 172.21.15.16:0/3224745131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff40415dca0 0x7ff4040cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.535+0000 7ff408a0a640 1 --2- 172.21.15.16:0/3224745131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4040d3b90 0x7ff4040abb40 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400c750 tx=0x7ff3f400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.535+0000 7ff3f9ffb640 1 -- 172.21.15.16:0/3224745131 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4017070 con 0x7ff4040d3b90 2024-04-25T10:14:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.535+0000 7ff3f9ffb640 1 -- 172.21.15.16:0/3224745131 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f4005ad0 con 0x7ff4040d3b90 2024-04-25T10:14:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.535+0000 7ff40b46f640 1 -- 172.21.15.16:0/3224745131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4040ac200 con 0x7ff4040d3b90 2024-04-25T10:14:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.535+0000 7ff3f9ffb640 1 -- 172.21.15.16:0/3224745131 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4007070 con 0x7ff4040d3b90 2024-04-25T10:14:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.535+0000 7ff40b46f640 1 -- 172.21.15.16:0/3224745131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4040ac7c0 con 0x7ff4040d3b90 2024-04-25T10:14:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.536+0000 7ff40b46f640 1 -- 172.21.15.16:0/3224745131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4040750b0 con 0x7ff4040d3b90 2024-04-25T10:14:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.536+0000 7ff3f9ffb640 1 -- 172.21.15.16:0/3224745131 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3f401a070 con 0x7ff4040d3b90 2024-04-25T10:14:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.537+0000 7ff3f9ffb640 1 --2- 172.21.15.16:0/3224745131 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff3e806b4d0 0x7ff3e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.537+0000 7ff3f9ffb640 1 -- 172.21.15.16:0/3224745131 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5997+0+0 (secure 0 0 0) 0x7ff3f40895a0 con 0x7ff4040d3b90 2024-04-25T10:14:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.537+0000 7ff40920b640 1 --2- 172.21.15.16:0/3224745131 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff3e806b4d0 0x7ff3e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.537+0000 7ff40920b640 1 --2- 172.21.15.16:0/3224745131 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff3e806b4d0 0x7ff3e806d990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff400034000 tx=0x7ff40002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.541+0000 7ff3f9ffb640 1 -- 172.21.15.16:0/3224745131 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3f4057d40 con 0x7ff4040d3b90 2024-04-25T10:14:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.719+0000 7ff40b46f640 1 -- 172.21.15.16:0/3224745131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4040b7df0 con 0x7ff4040d3b90 2024-04-25T10:14:12.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.719+0000 7ff3f9ffb640 1 -- 172.21.15.16:0/3224745131 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff3f405bde0 con 0x7ff4040d3b90 2024-04-25T10:14:12.724 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:14:12.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.721+0000 7ff40b46f640 1 -- 172.21.15.16:0/3224745131 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff3e806b4d0 msgr2=0x7ff3e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.721+0000 7ff40b46f640 1 --2- 172.21.15.16:0/3224745131 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff3e806b4d0 0x7ff3e806d990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff400034000 tx=0x7ff40002e000 comp rx=0 tx=0).stop 2024-04-25T10:14:12.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.721+0000 7ff40b46f640 1 -- 172.21.15.16:0/3224745131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4040d3b90 msgr2=0x7ff4040abb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.721+0000 7ff40b46f640 1 --2- 172.21.15.16:0/3224745131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4040d3b90 0x7ff4040abb40 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400c750 tx=0x7ff3f400cc10 comp rx=0 tx=0).stop 2024-04-25T10:14:12.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.721+0000 7ff40b46f640 1 -- 172.21.15.16:0/3224745131 shutdown_connections 2024-04-25T10:14:12.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.721+0000 7ff40b46f640 1 --2- 172.21.15.16:0/3224745131 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff3e806b4d0 0x7ff3e806d990 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.721+0000 7ff40b46f640 1 --2- 172.21.15.16:0/3224745131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff40415dca0 0x7ff4040cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.721+0000 7ff40b46f640 1 --2- 172.21.15.16:0/3224745131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4040d3b90 0x7ff4040abb40 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.721+0000 7ff40b46f640 1 --2- 172.21.15.16:0/3224745131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4040d1130 0x7ff404152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.721+0000 7ff40b46f640 1 -- 172.21.15.16:0/3224745131 >> 172.21.15.16:0/3224745131 conn(0x7ff4040726c0 msgr2=0x7ff4040b88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:12.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.721+0000 7ff40b46f640 1 -- 172.21.15.16:0/3224745131 shutdown_connections 2024-04-25T10:14:12.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.721+0000 7ff40b46f640 1 -- 172.21.15.16:0/3224745131 wait complete. 2024-04-25T10:14:12.736 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.832+0000 7efed3cb3640 1 Processor -- start 2024-04-25T10:14:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.833+0000 7efed3cb3640 1 -- start start 2024-04-25T10:14:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.833+0000 7efed3cb3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efecc15dce0 0x7efecc0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.833+0000 7efed3cb3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7efecc0d35b0 0x7efecc0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.833+0000 7efed3cb3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efecc0d1f90 0x7efecc0b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.833+0000 7efed3cb3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efecc15e0c0 con 0x7efecc15dce0 2024-04-25T10:14:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.833+0000 7efed3cb3640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efecc0d2510 con 0x7efecc0d35b0 2024-04-25T10:14:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.833+0000 7efed3cb3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc0d27f0 con 0x7efecc0d1f90 2024-04-25T10:14:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.834+0000 7efed1a4f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efecc15dce0 0x7efecc0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.834+0000 7efed1a4f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efecc15dce0 0x7efecc0d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47190/0 (socket says 172.21.15.16:47190) 2024-04-25T10:14:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.834+0000 7efed1a4f640 1 -- 172.21.15.16:0/573299517 learned_addr learned my addr 172.21.15.16:0/573299517 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.834+0000 7efed124e640 1 --2- 172.21.15.16:0/573299517 >> v2:172.21.15.186:3300/0 conn(0x7efecc0d35b0 0x7efecc0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.834+0000 7efed2250640 1 --2- 172.21.15.16:0/573299517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efecc0d1f90 0x7efecc0b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.834+0000 7efed1a4f640 1 -- 172.21.15.16:0/573299517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efecc0d1f90 msgr2=0x7efecc0b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.834+0000 7efed1a4f640 1 --2- 172.21.15.16:0/573299517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efecc0d1f90 0x7efecc0b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.834+0000 7efed1a4f640 1 -- 172.21.15.16:0/573299517 >> v2:172.21.15.186:3300/0 conn(0x7efecc0d35b0 msgr2=0x7efecc0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.834+0000 7efed1a4f640 1 --2- 172.21.15.16:0/573299517 >> v2:172.21.15.186:3300/0 conn(0x7efecc0d35b0 0x7efecc0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.834+0000 7efed1a4f640 1 -- 172.21.15.16:0/573299517 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc0d2ad0 con 0x7efecc15dce0 2024-04-25T10:14:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.835+0000 7efed1a4f640 1 --2- 172.21.15.16:0/573299517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc15dce0 0x7efecc0d1a50 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7efecc0cdfa0 tx=0x7efec402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fea9b8587a453f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.835+0000 7efed0a4d640 1 -- 172.21.15.16:0/573299517 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4037070 con 0x7efecc15dce0 2024-04-25T10:14:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.835+0000 7efed0a4d640 1 -- 172.21.15.16:0/573299517 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec4032440 con 0x7efecc15dce0 2024-04-25T10:14:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.835+0000 7efed0a4d640 1 -- 172.21.15.16:0/573299517 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4032810 con 0x7efecc15dce0 2024-04-25T10:14:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.835+0000 7efed3cb3640 1 -- 172.21.15.16:0/573299517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc15dce0 msgr2=0x7efecc0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.835+0000 7efed3cb3640 1 --2- 172.21.15.16:0/573299517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc15dce0 0x7efecc0d1a50 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7efecc0cdfa0 tx=0x7efec402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.835+0000 7efed3cb3640 1 -- 172.21.15.16:0/573299517 shutdown_connections 2024-04-25T10:14:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.835+0000 7efed3cb3640 1 --2- 172.21.15.16:0/573299517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efecc0d1f90 0x7efecc0b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.835+0000 7efed3cb3640 1 --2- 172.21.15.16:0/573299517 >> v2:172.21.15.186:3300/0 conn(0x7efecc0d35b0 0x7efecc0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.835+0000 7efed3cb3640 1 --2- 172.21.15.16:0/573299517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc15dce0 0x7efecc0d1a50 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.835+0000 7efed3cb3640 1 -- 172.21.15.16:0/573299517 >> 172.21.15.16:0/573299517 conn(0x7efecc072900 msgr2=0x7efecc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:12.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.835+0000 7efed3cb3640 1 -- 172.21.15.16:0/573299517 shutdown_connections 2024-04-25T10:14:12.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.835+0000 7efed3cb3640 1 -- 172.21.15.16:0/573299517 wait complete. 2024-04-25T10:14:12.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.837+0000 7efed3cb3640 1 Processor -- start 2024-04-25T10:14:12.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.837+0000 7efed3cb3640 1 -- start start 2024-04-25T10:14:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.837+0000 7efed3cb3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc0d1f90 0x7efecc15f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.837+0000 7efed3cb3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efecc0d35b0 0x7efecc14dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.837+0000 7efed3cb3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efecc15dce0 0x7efecc15a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.837+0000 7efed3cb3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc0b9820 con 0x7efecc0d1f90 2024-04-25T10:14:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.837+0000 7efed3cb3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc0d3090 con 0x7efecc0d35b0 2024-04-25T10:14:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.837+0000 7efed3cb3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc0b99a0 con 0x7efecc15dce0 2024-04-25T10:14:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.837+0000 7efed1a4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc0d1f90 0x7efecc15f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.837+0000 7efed1a4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc0d1f90 0x7efecc15f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47206/0 (socket says 172.21.15.16:47206) 2024-04-25T10:14:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.837+0000 7efed1a4f640 1 -- 172.21.15.16:0/3749144137 learned_addr learned my addr 172.21.15.16:0/3749144137 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.838+0000 7efed1a4f640 1 -- 172.21.15.16:0/3749144137 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efecc15dce0 msgr2=0x7efecc15a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.838+0000 7efed1a4f640 1 --2- 172.21.15.16:0/3749144137 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efecc15dce0 0x7efecc15a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.838+0000 7efed1a4f640 1 -- 172.21.15.16:0/3749144137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efecc0d35b0 msgr2=0x7efecc14dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.838+0000 7efed1a4f640 1 --2- 172.21.15.16:0/3749144137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efecc0d35b0 0x7efecc14dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.838+0000 7efed1a4f640 1 -- 172.21.15.16:0/3749144137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc1f6d90 con 0x7efecc0d1f90 2024-04-25T10:14:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.838+0000 7efed1a4f640 1 --2- 172.21.15.16:0/3749144137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc0d1f90 0x7efecc15f690 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7efec402a220 tx=0x7efec40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.838+0000 7efeca7fc640 1 -- 172.21.15.16:0/3749144137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4041020 con 0x7efecc0d1f90 2024-04-25T10:14:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.838+0000 7efeca7fc640 1 -- 172.21.15.16:0/3749144137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec40306c0 con 0x7efecc0d1f90 2024-04-25T10:14:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.838+0000 7efed3cb3640 1 -- 172.21.15.16:0/3749144137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efecc1f7070 con 0x7efecc0d1f90 2024-04-25T10:14:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.838+0000 7efeca7fc640 1 -- 172.21.15.16:0/3749144137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4037040 con 0x7efecc0d1f90 2024-04-25T10:14:12.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.838+0000 7efed3cb3640 1 -- 172.21.15.16:0/3749144137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efecc1f7630 con 0x7efecc0d1f90 2024-04-25T10:14:12.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.839+0000 7efed3cb3640 1 -- 172.21.15.16:0/3749144137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efecc0749b0 con 0x7efecc0d1f90 2024-04-25T10:14:12.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.839+0000 7efeca7fc640 1 -- 172.21.15.16:0/3749144137 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efec4044020 con 0x7efecc0d1f90 2024-04-25T10:14:12.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.840+0000 7efeca7fc640 1 --2- 172.21.15.16:0/3749144137 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efeb806b4d0 0x7efeb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.840+0000 7efeca7fc640 1 -- 172.21.15.16:0/3749144137 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5997+0+0 (secure 0 0 0) 0x7efec40aab90 con 0x7efecc0d1f90 2024-04-25T10:14:12.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.840+0000 7efed124e640 1 --2- 172.21.15.16:0/3749144137 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efeb806b4d0 0x7efeb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.841+0000 7efed124e640 1 --2- 172.21.15.16:0/3749144137 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efeb806b4d0 0x7efeb806d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7efecc161540 tx=0x7efec0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:12.844+0000 7efeca7fc640 1 -- 172.21.15.16:0/3749144137 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efec404b030 con 0x7efecc0d1f90 2024-04-25T10:14:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.070+0000 7efed3cb3640 1 -- 172.21.15.16:0/3749144137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efecc15d2c0 con 0x7efecc0d1f90 2024-04-25T10:14:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.070+0000 7efeca7fc640 1 -- 172.21.15.16:0/3749144137 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4934 (secure 0 0 0) 0x7efec40798f0 con 0x7efecc0d1f90 2024-04-25T10:14:13.075 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:13.075 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":78,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7243,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/109406141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":109406141},{"type":"v1","addr":"172.21.15.186:6837","nonce":109406141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7597,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/3732623843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3732623843},{"type":"v1","addr":"172.21.15.186:6836","nonce":3732623843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2704401503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2704401503},{"type":"v1","addr":"172.21.15.16:6835","nonce":2704401503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/939221717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":939221717},{"type":"v1","addr":"172.21.15.16:6834","nonce":939221717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":77,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:48.958938+0000","modified":"2024-04-25T10:14:10.849139+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:14:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 78 2024-04-25T10:14:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.072+0000 7efed3cb3640 1 -- 172.21.15.16:0/3749144137 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efeb806b4d0 msgr2=0x7efeb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.072+0000 7efed3cb3640 1 --2- 172.21.15.16:0/3749144137 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efeb806b4d0 0x7efeb806d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7efecc161540 tx=0x7efec0008040 comp rx=0 tx=0).stop 2024-04-25T10:14:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.072+0000 7efed3cb3640 1 -- 172.21.15.16:0/3749144137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc0d1f90 msgr2=0x7efecc15f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.072+0000 7efed3cb3640 1 --2- 172.21.15.16:0/3749144137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc0d1f90 0x7efecc15f690 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7efec402a220 tx=0x7efec40046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.072+0000 7efed3cb3640 1 -- 172.21.15.16:0/3749144137 shutdown_connections 2024-04-25T10:14:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.072+0000 7efed3cb3640 1 --2- 172.21.15.16:0/3749144137 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efeb806b4d0 0x7efeb806d990 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.072+0000 7efed3cb3640 1 --2- 172.21.15.16:0/3749144137 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efecc15dce0 0x7efecc15a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.072+0000 7efed3cb3640 1 --2- 172.21.15.16:0/3749144137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efecc0d35b0 0x7efecc14dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.072+0000 7efed3cb3640 1 --2- 172.21.15.16:0/3749144137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc0d1f90 0x7efecc15f690 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.072+0000 7efed3cb3640 1 -- 172.21.15.16:0/3749144137 >> 172.21.15.16:0/3749144137 conn(0x7efecc072900 msgr2=0x7efecc0b6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.072+0000 7efed3cb3640 1 -- 172.21.15.16:0/3749144137 shutdown_connections 2024-04-25T10:14:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.072+0000 7efed3cb3640 1 -- 172.21.15.16:0/3749144137 wait complete. 2024-04-25T10:14:13.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:13.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.184+0000 7fd4bfb70640 1 Processor -- start 2024-04-25T10:14:13.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.184+0000 7fd4bfb70640 1 -- start start 2024-04-25T10:14:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.185+0000 7fd4bfb70640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4b815dce0 0x7fd4b80d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.185+0000 7fd4bfb70640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b80d35b0 0x7fd4b80b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.185+0000 7fd4bfb70640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b815e0c0 con 0x7fd4b80d1f90 2024-04-25T10:14:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.185+0000 7fd4bfb70640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b80b7b10 con 0x7fd4b815dce0 2024-04-25T10:14:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.185+0000 7fd4bfb70640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b80d2510 con 0x7fd4b80d35b0 2024-04-25T10:14:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.185+0000 7fd4bd90c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4b815dce0 0x7fd4b80d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.185+0000 7fd4be10d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd4b80d1f90 0x7fd4b806bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47676/0 (socket says 172.21.15.16:47676) 2024-04-25T10:14:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.185+0000 7fd4be10d640 1 -- 172.21.15.16:0/3036738391 learned_addr learned my addr 172.21.15.16:0/3036738391 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.185+0000 7fd4bd10b640 1 --2- 172.21.15.16:0/3036738391 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b80d35b0 0x7fd4b80b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.185+0000 7fd4bd90c640 1 -- 172.21.15.16:0/3036738391 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b80d35b0 msgr2=0x7fd4b80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.186+0000 7fd4bd90c640 1 --2- 172.21.15.16:0/3036738391 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b80d35b0 0x7fd4b80b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.186+0000 7fd4bd90c640 1 -- 172.21.15.16:0/3036738391 >> v1:172.21.15.16:6789/0 conn(0x7fd4b80d1f90 legacy=0x7fd4b806bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.186+0000 7fd4bc90a640 1 -- 172.21.15.16:0/3036738391 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691275881 0 0) 0x7fd4b815e0c0 con 0x7fd4b80d1f90 2024-04-25T10:14:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.186+0000 7fd4bd90c640 1 -- 172.21.15.16:0/3036738391 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b80d27f0 con 0x7fd4b815dce0 2024-04-25T10:14:13.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.186+0000 7fd4bd90c640 1 --2- 172.21.15.16:0/3036738391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b815dce0 0x7fd4b80d1a50 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd4b00097a0 tx=0x7fd4b002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce5ce3732f4c44f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.186+0000 7fd4bc90a640 1 -- 172.21.15.16:0/3036738391 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0037070 con 0x7fd4b815dce0 2024-04-25T10:14:13.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.186+0000 7fd4bc90a640 1 -- 172.21.15.16:0/3036738391 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b002ae50 con 0x7fd4b815dce0 2024-04-25T10:14:13.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.186+0000 7fd4bc90a640 1 -- 172.21.15.16:0/3036738391 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0032660 con 0x7fd4b815dce0 2024-04-25T10:14:13.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.187+0000 7fd4bfb70640 1 -- 172.21.15.16:0/3036738391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b815dce0 msgr2=0x7fd4b80d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.187+0000 7fd4bfb70640 1 --2- 172.21.15.16:0/3036738391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b815dce0 0x7fd4b80d1a50 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd4b00097a0 tx=0x7fd4b002a100 comp rx=0 tx=0).stop 2024-04-25T10:14:13.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.187+0000 7fd4bfb70640 1 -- 172.21.15.16:0/3036738391 shutdown_connections 2024-04-25T10:14:13.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.187+0000 7fd4bfb70640 1 --2- 172.21.15.16:0/3036738391 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b80d35b0 0x7fd4b80b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.187+0000 7fd4bfb70640 1 --2- 172.21.15.16:0/3036738391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b815dce0 0x7fd4b80d1a50 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.187+0000 7fd4bfb70640 1 -- 172.21.15.16:0/3036738391 >> 172.21.15.16:0/3036738391 conn(0x7fd4b8072900 msgr2=0x7fd4b80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.187+0000 7fd4bfb70640 1 -- 172.21.15.16:0/3036738391 shutdown_connections 2024-04-25T10:14:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.187+0000 7fd4bfb70640 1 -- 172.21.15.16:0/3036738391 wait complete. 2024-04-25T10:14:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.188+0000 7fd4bfb70640 1 Processor -- start 2024-04-25T10:14:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.188+0000 7fd4bfb70640 1 -- start start 2024-04-25T10:14:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.189+0000 7fd4bfb70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b80d1f90 0x7fd4b8160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.189+0000 7fd4bfb70640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4b80d35b0 0x7fd4b80cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.189+0000 7fd4bfb70640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b815dce0 0x7fd4b815e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.189+0000 7fd4bfb70640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b80d3090 con 0x7fd4b80d1f90 2024-04-25T10:14:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.189+0000 7fd4bfb70640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b80d2db0 con 0x7fd4b80d35b0 2024-04-25T10:14:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.189+0000 7fd4bfb70640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b80b9820 con 0x7fd4b815dce0 2024-04-25T10:14:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.189+0000 7fd4bd90c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b80d1f90 0x7fd4b8160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.189+0000 7fd4bd90c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b80d1f90 0x7fd4b8160dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47228/0 (socket says 172.21.15.16:47228) 2024-04-25T10:14:13.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.189+0000 7fd4bd90c640 1 -- 172.21.15.16:0/186328193 learned_addr learned my addr 172.21.15.16:0/186328193 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:13.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.189+0000 7fd4bd10b640 1 --2- 172.21.15.16:0/186328193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4b80d35b0 0x7fd4b80cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.189+0000 7fd4be10d640 1 --2- 172.21.15.16:0/186328193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b815dce0 0x7fd4b815e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.189+0000 7fd4bd90c640 1 -- 172.21.15.16:0/186328193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b815dce0 msgr2=0x7fd4b815e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.189+0000 7fd4bd90c640 1 --2- 172.21.15.16:0/186328193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b815dce0 0x7fd4b815e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.189+0000 7fd4bd90c640 1 -- 172.21.15.16:0/186328193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4b80d35b0 msgr2=0x7fd4b80cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.189+0000 7fd4bd90c640 1 --2- 172.21.15.16:0/186328193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4b80d35b0 0x7fd4b80cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.189+0000 7fd4bd90c640 1 -- 172.21.15.16:0/186328193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b80cc540 con 0x7fd4b80d1f90 2024-04-25T10:14:13.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.189+0000 7fd4bd10b640 1 --2- 172.21.15.16:0/186328193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4b80d35b0 0x7fd4b80cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:13.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.189+0000 7fd4be10d640 1 --2- 172.21.15.16:0/186328193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b815dce0 0x7fd4b815e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:13.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.190+0000 7fd4bd90c640 1 --2- 172.21.15.16:0/186328193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b80d1f90 0x7fd4b8160dd0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fd4b00097a0 tx=0x7fd4b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.190+0000 7fd4b67fc640 1 -- 172.21.15.16:0/186328193 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0037040 con 0x7fd4b80d1f90 2024-04-25T10:14:13.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.190+0000 7fd4b67fc640 1 -- 172.21.15.16:0/186328193 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b0004250 con 0x7fd4b80d1f90 2024-04-25T10:14:13.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.190+0000 7fd4bfb70640 1 -- 172.21.15.16:0/186328193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b80cc820 con 0x7fd4b80d1f90 2024-04-25T10:14:13.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.190+0000 7fd4b67fc640 1 -- 172.21.15.16:0/186328193 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0003610 con 0x7fd4b80d1f90 2024-04-25T10:14:13.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.190+0000 7fd4bfb70640 1 -- 172.21.15.16:0/186328193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b80ccde0 con 0x7fd4b80d1f90 2024-04-25T10:14:13.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.191+0000 7fd4bfb70640 1 -- 172.21.15.16:0/186328193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b80749b0 con 0x7fd4b80d1f90 2024-04-25T10:14:13.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.191+0000 7fd4b67fc640 1 -- 172.21.15.16:0/186328193 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4b002ad10 con 0x7fd4b80d1f90 2024-04-25T10:14:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.192+0000 7fd4b67fc640 1 --2- 172.21.15.16:0/186328193 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd4a406b4d0 0x7fd4a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.192+0000 7fd4b67fc640 1 -- 172.21.15.16:0/186328193 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fd4b00aae60 con 0x7fd4b80d1f90 2024-04-25T10:14:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.192+0000 7fd4bd10b640 1 --2- 172.21.15.16:0/186328193 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd4a406b4d0 0x7fd4a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.192+0000 7fd4bd10b640 1 --2- 172.21.15.16:0/186328193 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd4a406b4d0 0x7fd4a406d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80b66a0 tx=0x7fd4ac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.196+0000 7fd4b67fc640 1 -- 172.21.15.16:0/186328193 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4b004c070 con 0x7fd4b80d1f90 2024-04-25T10:14:13.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.377+0000 7fd4bfb70640 1 -- 172.21.15.16:0/186328193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4b80d42c0 con 0x7fd4b80d1f90 2024-04-25T10:14:13.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.378+0000 7fd4b67fc640 1 -- 172.21.15.16:0/186328193 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+17068 (secure 0 0 0) 0x7fd4b0079d70 con 0x7fd4b80d1f90 2024-04-25T10:14:13.383 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:13.383 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":104,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:14:10.849151+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:47.551100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:47.939289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":5,"to":3}]},{"pgid":"14.b","mappings":[{"from":6,"to":0}]},{"pgid":"14.e","mappings":[{"from":5,"to":0}]},{"pgid":"14.16","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.186:6836/2008057690":"2024-04-26T10:14:10.849128+0000","172.21.15.186:6834/2008057690":"2024-04-26T10:14:10.849128+0000","172.21.15.186:6836/1371188714":"2024-04-26T10:13:34.499700+0000","172.21.15.186:6834/1371188714":"2024-04-26T10:13:34.499700+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.380+0000 7fd4bfb70640 1 -- 172.21.15.16:0/186328193 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd4a406b4d0 msgr2=0x7fd4a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.380+0000 7fd4bfb70640 1 --2- 172.21.15.16:0/186328193 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd4a406b4d0 0x7fd4a406d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80b66a0 tx=0x7fd4ac008040 comp rx=0 tx=0).stop 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.380+0000 7fd4bfb70640 1 -- 172.21.15.16:0/186328193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b80d1f90 msgr2=0x7fd4b8160dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.380+0000 7fd4bfb70640 1 --2- 172.21.15.16:0/186328193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b80d1f90 0x7fd4b8160dd0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fd4b00097a0 tx=0x7fd4b00046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.380+0000 7fd4bfb70640 1 -- 172.21.15.16:0/186328193 shutdown_connections 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.380+0000 7fd4bfb70640 1 --2- 172.21.15.16:0/186328193 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd4a406b4d0 0x7fd4a406d990 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.380+0000 7fd4bfb70640 1 --2- 172.21.15.16:0/186328193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b815dce0 0x7fd4b815e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.380+0000 7fd4bfb70640 1 --2- 172.21.15.16:0/186328193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4b80d35b0 0x7fd4b80cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.380+0000 7fd4bfb70640 1 --2- 172.21.15.16:0/186328193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b80d1f90 0x7fd4b8160dd0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.380+0000 7fd4bfb70640 1 -- 172.21.15.16:0/186328193 >> 172.21.15.16:0/186328193 conn(0x7fd4b8072900 msgr2=0x7fd4b815d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.380+0000 7fd4bfb70640 1 -- 172.21.15.16:0/186328193 shutdown_connections 2024-04-25T10:14:13.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.380+0000 7fd4bfb70640 1 -- 172.21.15.16:0/186328193 wait complete. 2024-04-25T10:14:13.396 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:14:13.396 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.492+0000 7f9f873bd640 1 Processor -- start 2024-04-25T10:14:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.492+0000 7f9f873bd640 1 -- start start 2024-04-25T10:14:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.493+0000 7f9f873bd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f8015dce0 0x7f9f800d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.493+0000 7f9f873bd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9f800b5b60 0x7f9f800b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.493+0000 7f9f873bd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8015e0c0 con 0x7f9f800d35b0 2024-04-25T10:14:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.493+0000 7f9f873bd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f800d2510 con 0x7f9f8015dce0 2024-04-25T10:14:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.493+0000 7f9f873bd640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f800d27f0 con 0x7f9f800b5b60 2024-04-25T10:14:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.493+0000 7f9f85159640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f8015dce0 0x7f9f800d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.493+0000 7f9f85159640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f8015dce0 0x7f9f800d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47236/0 (socket says 172.21.15.16:47236) 2024-04-25T10:14:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.493+0000 7f9f85159640 1 -- 172.21.15.16:0/383677717 learned_addr learned my addr 172.21.15.16:0/383677717 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.493+0000 7f9f85159640 1 -- 172.21.15.16:0/383677717 >> v2:172.21.15.186:3300/0 conn(0x7f9f800b5b60 msgr2=0x7f9f800b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.493+0000 7f9f85159640 1 --2- 172.21.15.16:0/383677717 >> v2:172.21.15.186:3300/0 conn(0x7f9f800b5b60 0x7f9f800b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.493+0000 7f9f85159640 1 -- 172.21.15.16:0/383677717 >> v1:172.21.15.186:6789/0 conn(0x7f9f800d35b0 legacy=0x7f9f800b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.493+0000 7f9f85159640 1 -- 172.21.15.16:0/383677717 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f800d2ad0 con 0x7f9f8015dce0 2024-04-25T10:14:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.494+0000 7f9f85159640 1 --2- 172.21.15.16:0/383677717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f8015dce0 0x7f9f800d1a50 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f9f800b3390 tx=0x7f9f7c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78acd2ad51fbd03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.494+0000 7f9f77fff640 1 -- 172.21.15.16:0/383677717 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c037070 con 0x7f9f8015dce0 2024-04-25T10:14:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.494+0000 7f9f77fff640 1 -- 172.21.15.16:0/383677717 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7c032440 con 0x7f9f8015dce0 2024-04-25T10:14:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.494+0000 7f9f77fff640 1 -- 172.21.15.16:0/383677717 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c032810 con 0x7f9f8015dce0 2024-04-25T10:14:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.494+0000 7f9f873bd640 1 -- 172.21.15.16:0/383677717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f8015dce0 msgr2=0x7f9f800d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.494+0000 7f9f873bd640 1 --2- 172.21.15.16:0/383677717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f8015dce0 0x7f9f800d1a50 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f9f800b3390 tx=0x7f9f7c02a230 comp rx=0 tx=0).stop 2024-04-25T10:14:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.495+0000 7f9f873bd640 1 -- 172.21.15.16:0/383677717 shutdown_connections 2024-04-25T10:14:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.495+0000 7f9f873bd640 1 --2- 172.21.15.16:0/383677717 >> v2:172.21.15.186:3300/0 conn(0x7f9f800b5b60 0x7f9f800b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.495+0000 7f9f873bd640 1 --2- 172.21.15.16:0/383677717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f8015dce0 0x7f9f800d1a50 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.495+0000 7f9f873bd640 1 -- 172.21.15.16:0/383677717 >> 172.21.15.16:0/383677717 conn(0x7f9f80072900 msgr2=0x7f9f800d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.495+0000 7f9f873bd640 1 -- 172.21.15.16:0/383677717 shutdown_connections 2024-04-25T10:14:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.495+0000 7f9f873bd640 1 -- 172.21.15.16:0/383677717 wait complete. 2024-04-25T10:14:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.496+0000 7f9f873bd640 1 Processor -- start 2024-04-25T10:14:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.496+0000 7f9f873bd640 1 -- start start 2024-04-25T10:14:13.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f873bd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f800b5b60 0x7f9f8014fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f873bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f800d35b0 0x7f9f800c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f873bd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f8015dce0 0x7f9f8015fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f873bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f800b9820 con 0x7f9f800d35b0 2024-04-25T10:14:13.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f873bd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f800d3090 con 0x7f9f8015dce0 2024-04-25T10:14:13.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f873bd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f800b99a0 con 0x7f9f800b5b60 2024-04-25T10:14:13.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f84958640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f800d35b0 0x7f9f800c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f84958640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f800d35b0 0x7f9f800c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47240/0 (socket says 172.21.15.16:47240) 2024-04-25T10:14:13.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f84958640 1 -- 172.21.15.16:0/2469543045 learned_addr learned my addr 172.21.15.16:0/2469543045 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:13.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f85159640 1 --2- 172.21.15.16:0/2469543045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f800b5b60 0x7f9f8014fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f8595a640 1 --2- 172.21.15.16:0/2469543045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f8015dce0 0x7f9f8015fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f84958640 1 -- 172.21.15.16:0/2469543045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f800b5b60 msgr2=0x7f9f8014fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f84958640 1 --2- 172.21.15.16:0/2469543045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f800b5b60 0x7f9f8014fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f84958640 1 -- 172.21.15.16:0/2469543045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f8015dce0 msgr2=0x7f9f8015fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f84958640 1 --2- 172.21.15.16:0/2469543045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f8015dce0 0x7f9f8015fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f84958640 1 -- 172.21.15.16:0/2469543045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8015eb40 con 0x7f9f800d35b0 2024-04-25T10:14:13.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f84958640 1 --2- 172.21.15.16:0/2469543045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f800d35b0 0x7f9f800c8710 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000c400 tx=0x7f9f7000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f75ffb640 1 -- 172.21.15.16:0/2469543045 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f70017070 con 0x7f9f800d35b0 2024-04-25T10:14:13.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f75ffb640 1 -- 172.21.15.16:0/2469543045 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f700035f0 con 0x7f9f800d35b0 2024-04-25T10:14:13.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f75ffb640 1 -- 172.21.15.16:0/2469543045 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f70003a20 con 0x7f9f800d35b0 2024-04-25T10:14:13.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.497+0000 7f9f873bd640 1 -- 172.21.15.16:0/2469543045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8015ee20 con 0x7f9f800d35b0 2024-04-25T10:14:13.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.498+0000 7f9f873bd640 1 -- 172.21.15.16:0/2469543045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8015f3e0 con 0x7f9f800d35b0 2024-04-25T10:14:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.499+0000 7f9f873bd640 1 -- 172.21.15.16:0/2469543045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f800749b0 con 0x7f9f800d35b0 2024-04-25T10:14:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.499+0000 7f9f75ffb640 1 -- 172.21.15.16:0/2469543045 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f7001a070 con 0x7f9f800d35b0 2024-04-25T10:14:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.499+0000 7f9f75ffb640 1 --2- 172.21.15.16:0/2469543045 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9f6406b4d0 0x7f9f6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.500+0000 7f9f75ffb640 1 -- 172.21.15.16:0/2469543045 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f9f70089630 con 0x7f9f800d35b0 2024-04-25T10:14:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.500+0000 7f9f85159640 1 --2- 172.21.15.16:0/2469543045 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9f6406b4d0 0x7f9f6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.500+0000 7f9f85159640 1 --2- 172.21.15.16:0/2469543045 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9f6406b4d0 0x7f9f6406d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c034000 tx=0x7f9f7c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.504+0000 7f9f75ffb640 1 -- 172.21.15.16:0/2469543045 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f70057d40 con 0x7f9f800d35b0 2024-04-25T10:14:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.718+0000 7f9f873bd640 1 -- 172.21.15.16:0/2469543045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f8006b080 con 0x7f9f800d35b0 2024-04-25T10:14:13.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.718+0000 7f9f75ffb640 1 -- 172.21.15.16:0/2469543045 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f9f7005bde0 con 0x7f9f800d35b0 2024-04-25T10:14:13.723 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:13.723 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":78,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7243,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/109406141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":109406141},{"type":"v1","addr":"172.21.15.186:6837","nonce":109406141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7597,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/3732623843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3732623843},{"type":"v1","addr":"172.21.15.186:6836","nonce":3732623843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2704401503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2704401503},{"type":"v1","addr":"172.21.15.16:6835","nonce":2704401503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/939221717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":939221717},{"type":"v1","addr":"172.21.15.16:6834","nonce":939221717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":77,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:48.958938+0000","modified":"2024-04-25T10:14:10.849139+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:14:13.723 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 78 2024-04-25T10:14:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.720+0000 7f9f873bd640 1 -- 172.21.15.16:0/2469543045 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9f6406b4d0 msgr2=0x7f9f6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.720+0000 7f9f873bd640 1 --2- 172.21.15.16:0/2469543045 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9f6406b4d0 0x7f9f6406d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c034000 tx=0x7f9f7c02e000 comp rx=0 tx=0).stop 2024-04-25T10:14:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.720+0000 7f9f873bd640 1 -- 172.21.15.16:0/2469543045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f800d35b0 msgr2=0x7f9f800c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.720+0000 7f9f873bd640 1 --2- 172.21.15.16:0/2469543045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f800d35b0 0x7f9f800c8710 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000c400 tx=0x7f9f7000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.720+0000 7f9f873bd640 1 -- 172.21.15.16:0/2469543045 shutdown_connections 2024-04-25T10:14:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.720+0000 7f9f873bd640 1 --2- 172.21.15.16:0/2469543045 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9f6406b4d0 0x7f9f6406d990 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.720+0000 7f9f873bd640 1 --2- 172.21.15.16:0/2469543045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f8015dce0 0x7f9f8015fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.720+0000 7f9f873bd640 1 --2- 172.21.15.16:0/2469543045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f800d35b0 0x7f9f800c8710 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.720+0000 7f9f873bd640 1 --2- 172.21.15.16:0/2469543045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f800b5b60 0x7f9f8014fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.720+0000 7f9f873bd640 1 -- 172.21.15.16:0/2469543045 >> 172.21.15.16:0/2469543045 conn(0x7f9f80072900 msgr2=0x7f9f800b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.720+0000 7f9f873bd640 1 -- 172.21.15.16:0/2469543045 shutdown_connections 2024-04-25T10:14:13.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.720+0000 7f9f873bd640 1 -- 172.21.15.16:0/2469543045 wait complete. 2024-04-25T10:14:13.736 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.832+0000 7f5293a5a640 1 Processor -- start 2024-04-25T10:14:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.832+0000 7f5293a5a640 1 -- start start 2024-04-25T10:14:13.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.833+0000 7f5293a5a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0d35b0 0x7f528c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.833+0000 7f5293a5a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f528c0b5990 0x7f528c0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.833+0000 7f5293a5a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f528c15e0c0 con 0x7f528c15dce0 2024-04-25T10:14:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.833+0000 7f5293a5a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f528c0b5de0 con 0x7f528c0b5990 2024-04-25T10:14:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.833+0000 7f5293a5a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528c0d2510 con 0x7f528c0d35b0 2024-04-25T10:14:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.833+0000 7f5291ff7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f528c0b5990 0x7f528c0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.833+0000 7f5291ff7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f528c0b5990 0x7f528c0b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47254/0 (socket says 172.21.15.16:47254) 2024-04-25T10:14:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.833+0000 7f5291ff7640 1 -- 172.21.15.16:0/3252946923 learned_addr learned my addr 172.21.15.16:0/3252946923 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.833+0000 7f5290ff5640 1 --2- 172.21.15.16:0/3252946923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0d35b0 0x7f528c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.833+0000 7f5291ff7640 1 -- 172.21.15.16:0/3252946923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0d35b0 msgr2=0x7f528c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.833+0000 7f5291ff7640 1 --2- 172.21.15.16:0/3252946923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0d35b0 0x7f528c0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.833+0000 7f5291ff7640 1 -- 172.21.15.16:0/3252946923 >> v1:172.21.15.186:6789/0 conn(0x7f528c15dce0 legacy=0x7f528c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.834+0000 7f5291ff7640 1 -- 172.21.15.16:0/3252946923 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c0d27f0 con 0x7f528c0b5990 2024-04-25T10:14:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.834+0000 7f5291ff7640 1 --2- 172.21.15.16:0/3252946923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f528c0b5990 0x7f528c0b6b00 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f5284009e50 tx=0x7f528402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbc32ae3a56b69e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.834+0000 7f5283fff640 1 -- 172.21.15.16:0/3252946923 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284036070 con 0x7f528c0b5990 2024-04-25T10:14:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.834+0000 7f5283fff640 1 -- 172.21.15.16:0/3252946923 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5284031600 con 0x7f528c0b5990 2024-04-25T10:14:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.834+0000 7f5283fff640 1 -- 172.21.15.16:0/3252946923 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52840319d0 con 0x7f528c0b5990 2024-04-25T10:14:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.835+0000 7f5293a5a640 1 -- 172.21.15.16:0/3252946923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f528c0b5990 msgr2=0x7f528c0b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.835+0000 7f5293a5a640 1 --2- 172.21.15.16:0/3252946923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f528c0b5990 0x7f528c0b6b00 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f5284009e50 tx=0x7f528402a710 comp rx=0 tx=0).stop 2024-04-25T10:14:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.835+0000 7f5293a5a640 1 -- 172.21.15.16:0/3252946923 shutdown_connections 2024-04-25T10:14:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.835+0000 7f5293a5a640 1 --2- 172.21.15.16:0/3252946923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f528c0b5990 0x7f528c0b6b00 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.835+0000 7f5293a5a640 1 --2- 172.21.15.16:0/3252946923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0d35b0 0x7f528c0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.835+0000 7f5293a5a640 1 -- 172.21.15.16:0/3252946923 >> 172.21.15.16:0/3252946923 conn(0x7f528c072900 msgr2=0x7f528c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.835+0000 7f5293a5a640 1 -- 172.21.15.16:0/3252946923 shutdown_connections 2024-04-25T10:14:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.835+0000 7f5293a5a640 1 -- 172.21.15.16:0/3252946923 wait complete. 2024-04-25T10:14:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.836+0000 7f5293a5a640 1 Processor -- start 2024-04-25T10:14:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.836+0000 7f5293a5a640 1 -- start start 2024-04-25T10:14:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.837+0000 7f5293a5a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0b5990 0x7f528c152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.837+0000 7f5293a5a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f528c0d35b0 0x7f528c0b7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.837+0000 7f5293a5a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f528c15dce0 0x7f528c0b7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.837+0000 7f5293a5a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c0d3090 con 0x7f528c0d35b0 2024-04-25T10:14:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.837+0000 7f5293a5a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c0d2db0 con 0x7f528c15dce0 2024-04-25T10:14:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.837+0000 7f5293a5a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528c0d2160 con 0x7f528c0b5990 2024-04-25T10:14:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.837+0000 7f5290ff5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f528c0d35b0 0x7f528c0b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.837+0000 7f5290ff5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f528c0d35b0 0x7f528c0b7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47270/0 (socket says 172.21.15.16:47270) 2024-04-25T10:14:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.837+0000 7f5290ff5640 1 -- 172.21.15.16:0/3226974529 learned_addr learned my addr 172.21.15.16:0/3226974529 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.837+0000 7f52917f6640 1 --2- 172.21.15.16:0/3226974529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0b5990 0x7f528c152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.837+0000 7f5291ff7640 1 --2- 172.21.15.16:0/3226974529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f528c15dce0 0x7f528c0b7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.837+0000 7f5290ff5640 1 -- 172.21.15.16:0/3226974529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0b5990 msgr2=0x7f528c152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.837+0000 7f5290ff5640 1 --2- 172.21.15.16:0/3226974529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0b5990 0x7f528c152200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.838+0000 7f5290ff5640 1 -- 172.21.15.16:0/3226974529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f528c15dce0 msgr2=0x7f528c0b7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.838+0000 7f5290ff5640 1 --2- 172.21.15.16:0/3226974529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f528c15dce0 0x7f528c0b7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.838+0000 7f5290ff5640 1 -- 172.21.15.16:0/3226974529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c152740 con 0x7f528c0d35b0 2024-04-25T10:14:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.838+0000 7f5291ff7640 1 --2- 172.21.15.16:0/3226974529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f528c15dce0 0x7f528c0b7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.838+0000 7f5290ff5640 1 --2- 172.21.15.16:0/3226974529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f528c0d35b0 0x7f528c0b7570 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f527c00d830 tx=0x7f527c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.838+0000 7f5281ffb640 1 -- 172.21.15.16:0/3226974529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c00b6c0 con 0x7f528c0d35b0 2024-04-25T10:14:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.838+0000 7f5281ffb640 1 -- 172.21.15.16:0/3226974529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527c019070 con 0x7f528c0d35b0 2024-04-25T10:14:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.838+0000 7f5293a5a640 1 -- 172.21.15.16:0/3226974529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528c0c81f0 con 0x7f528c0d35b0 2024-04-25T10:14:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.838+0000 7f5281ffb640 1 -- 172.21.15.16:0/3226974529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c00ba80 con 0x7f528c0d35b0 2024-04-25T10:14:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.838+0000 7f5293a5a640 1 -- 172.21.15.16:0/3226974529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528c0c87b0 con 0x7f528c0d35b0 2024-04-25T10:14:13.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.839+0000 7f5293a5a640 1 -- 172.21.15.16:0/3226974529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528c0749b0 con 0x7f528c0d35b0 2024-04-25T10:14:13.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.839+0000 7f5281ffb640 1 -- 172.21.15.16:0/3226974529 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f527c01e460 con 0x7f528c0d35b0 2024-04-25T10:14:13.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.840+0000 7f5281ffb640 1 --2- 172.21.15.16:0/3226974529 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f527006b4d0 0x7f527006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.840+0000 7f5281ffb640 1 -- 172.21.15.16:0/3226974529 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f527c08a960 con 0x7f528c0d35b0 2024-04-25T10:14:13.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.840+0000 7f52917f6640 1 --2- 172.21.15.16:0/3226974529 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f527006b4d0 0x7f527006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.840+0000 7f52917f6640 1 --2- 172.21.15.16:0/3226974529 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f527006b4d0 0x7f527006d990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f528c0cda50 tx=0x7f5288009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:13.844+0000 7f5281ffb640 1 -- 172.21.15.16:0/3226974529 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f527c0596c0 con 0x7f528c0d35b0 2024-04-25T10:14:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.062+0000 7f5293a5a640 1 -- 172.21.15.16:0/3226974529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f528c0d15a0 con 0x7f528c0d35b0 2024-04-25T10:14:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.063+0000 7f5281ffb640 1 -- 172.21.15.16:0/3226974529 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f527c058fc0 con 0x7f528c0d35b0 2024-04-25T10:14:14.067 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:14.067 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":78,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7243,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/109406141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":109406141},{"type":"v1","addr":"172.21.15.186:6837","nonce":109406141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7597,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/3732623843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3732623843},{"type":"v1","addr":"172.21.15.186:6836","nonce":3732623843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2704401503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2704401503},{"type":"v1","addr":"172.21.15.16:6835","nonce":2704401503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/939221717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":939221717},{"type":"v1","addr":"172.21.15.16:6834","nonce":939221717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":77,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:48.958938+0000","modified":"2024-04-25T10:14:10.849139+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:14:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 78 2024-04-25T10:14:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.065+0000 7f5293a5a640 1 -- 172.21.15.16:0/3226974529 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f527006b4d0 msgr2=0x7f527006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.065+0000 7f5293a5a640 1 --2- 172.21.15.16:0/3226974529 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f527006b4d0 0x7f527006d990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f528c0cda50 tx=0x7f5288009040 comp rx=0 tx=0).stop 2024-04-25T10:14:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.065+0000 7f5293a5a640 1 -- 172.21.15.16:0/3226974529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f528c0d35b0 msgr2=0x7f528c0b7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.065+0000 7f5293a5a640 1 --2- 172.21.15.16:0/3226974529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f528c0d35b0 0x7f528c0b7570 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f527c00d830 tx=0x7f527c00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.065+0000 7f5293a5a640 1 -- 172.21.15.16:0/3226974529 shutdown_connections 2024-04-25T10:14:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.065+0000 7f5293a5a640 1 --2- 172.21.15.16:0/3226974529 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f527006b4d0 0x7f527006d990 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.065+0000 7f5293a5a640 1 --2- 172.21.15.16:0/3226974529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f528c15dce0 0x7f528c0b7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.065+0000 7f5293a5a640 1 --2- 172.21.15.16:0/3226974529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f528c0d35b0 0x7f528c0b7570 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.065+0000 7f5293a5a640 1 --2- 172.21.15.16:0/3226974529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f528c0b5990 0x7f528c152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.065+0000 7f5293a5a640 1 -- 172.21.15.16:0/3226974529 >> 172.21.15.16:0/3226974529 conn(0x7f528c072900 msgr2=0x7f528c0c6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.065+0000 7f5293a5a640 1 -- 172.21.15.16:0/3226974529 shutdown_connections 2024-04-25T10:14:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.065+0000 7f5293a5a640 1 -- 172.21.15.16:0/3226974529 wait complete. 2024-04-25T10:14:14.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.176+0000 7fa26c2e1640 1 Processor -- start 2024-04-25T10:14:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.176+0000 7fa26c2e1640 1 -- start start 2024-04-25T10:14:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.177+0000 7fa26c2e1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2640afc80 0x7fa2640caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.177+0000 7fa26c2e1640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa26406fe50 0x7fa2640702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.177+0000 7fa26c2e1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2640b00e0 con 0x7fa26406aba0 2024-04-25T10:14:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.177+0000 7fa26c2e1640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2640ce6d0 con 0x7fa26406fe50 2024-04-25T10:14:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.177+0000 7fa26c2e1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2640cee10 con 0x7fa2640afc80 2024-04-25T10:14:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.177+0000 7fa26a07d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa26406aba0 0x7fa26406af80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47686/0 (socket says 172.21.15.16:47686) 2024-04-25T10:14:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26a07d640 1 -- 172.21.15.16:0/3313011247 learned_addr learned my addr 172.21.15.16:0/3313011247 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26a87e640 1 --2- 172.21.15.16:0/3313011247 >> v2:172.21.15.186:3300/0 conn(0x7fa26406fe50 0x7fa2640702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26987c640 1 --2- 172.21.15.16:0/3313011247 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2640afc80 0x7fa2640caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26907b640 1 -- 172.21.15.16:0/3313011247 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3851482 0 0) 0x7fa2640b00e0 con 0x7fa26406aba0 2024-04-25T10:14:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26907b640 1 -- 172.21.15.16:0/3313011247 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa250003520 con 0x7fa26406aba0 2024-04-25T10:14:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26907b640 1 -- 172.21.15.16:0/3313011247 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3690337498 0 0) 0x7fa250003520 con 0x7fa26406aba0 2024-04-25T10:14:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26907b640 1 -- 172.21.15.16:0/3313011247 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2640afc80 msgr2=0x7fa2640caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26907b640 1 --2- 172.21.15.16:0/3313011247 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2640afc80 0x7fa2640caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26907b640 1 -- 172.21.15.16:0/3313011247 >> v2:172.21.15.186:3300/0 conn(0x7fa26406fe50 msgr2=0x7fa2640702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26907b640 1 --2- 172.21.15.16:0/3313011247 >> v2:172.21.15.186:3300/0 conn(0x7fa26406fe50 0x7fa2640702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26907b640 1 -- 172.21.15.16:0/3313011247 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2640cf0f0 con 0x7fa26406aba0 2024-04-25T10:14:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26907b640 1 -- 172.21.15.16:0/3313011247 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fa260001670 con 0x7fa26406aba0 2024-04-25T10:14:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26907b640 1 -- 172.21.15.16:0/3313011247 >> v1:172.21.15.16:6789/0 conn(0x7fa26406aba0 legacy=0x7fa26406af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26907b640 1 --2- 172.21.15.16:0/3313011247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2500039b0 0x7fa250003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26907b640 1 --2- 172.21.15.16:0/3313011247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa250004800 0x7fa250004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26907b640 1 --2- 172.21.15.16:0/3313011247 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2640afc80 0x7fa2500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26907b640 1 -- 172.21.15.16:0/3313011247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2640ce6d0 con 0x7fa250004800 2024-04-25T10:14:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26907b640 1 -- 172.21.15.16:0/3313011247 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2640cee10 con 0x7fa2500039b0 2024-04-25T10:14:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26907b640 1 -- 172.21.15.16:0/3313011247 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25000b9c0 con 0x7fa2640afc80 2024-04-25T10:14:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26987c640 1 --2- 172.21.15.16:0/3313011247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa250004800 0x7fa250004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26a07d640 1 --2- 172.21.15.16:0/3313011247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2500039b0 0x7fa250003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.178+0000 7fa26a87e640 1 --2- 172.21.15.16:0/3313011247 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2640afc80 0x7fa2500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.179+0000 7fa26987c640 1 -- 172.21.15.16:0/3313011247 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2640afc80 msgr2=0x7fa2500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.179+0000 7fa26987c640 1 --2- 172.21.15.16:0/3313011247 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2640afc80 0x7fa2500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.179+0000 7fa26987c640 1 -- 172.21.15.16:0/3313011247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2500039b0 msgr2=0x7fa250003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.179+0000 7fa26987c640 1 --2- 172.21.15.16:0/3313011247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2500039b0 0x7fa250003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.179+0000 7fa26987c640 1 -- 172.21.15.16:0/3313011247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa250003520 con 0x7fa250004800 2024-04-25T10:14:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.179+0000 7fa26987c640 1 --2- 172.21.15.16:0/3313011247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa250004800 0x7fa250004c80 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fa254009ac0 tx=0x7fa250005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45177dd97e31fad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.179+0000 7fa26907b640 1 -- 172.21.15.16:0/3313011247 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254036070 con 0x7fa250004800 2024-04-25T10:14:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.179+0000 7fa26907b640 1 -- 172.21.15.16:0/3313011247 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa254031400 con 0x7fa250004800 2024-04-25T10:14:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.179+0000 7fa26907b640 1 -- 172.21.15.16:0/3313011247 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254009690 con 0x7fa250004800 2024-04-25T10:14:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.180+0000 7fa26c2e1640 1 -- 172.21.15.16:0/3313011247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa250004800 msgr2=0x7fa250004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.180+0000 7fa26c2e1640 1 --2- 172.21.15.16:0/3313011247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa250004800 0x7fa250004c80 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fa254009ac0 tx=0x7fa250005830 comp rx=0 tx=0).stop 2024-04-25T10:14:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.180+0000 7fa26a87e640 1 -- 172.21.15.16:0/3313011247 reap_dead start 2024-04-25T10:14:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.180+0000 7fa26c2e1640 1 -- 172.21.15.16:0/3313011247 shutdown_connections 2024-04-25T10:14:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.180+0000 7fa26c2e1640 1 -- 172.21.15.16:0/3313011247 >> 172.21.15.16:0/3313011247 conn(0x7fa2640b2370 msgr2=0x7fa2640b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.180+0000 7fa26c2e1640 1 -- 172.21.15.16:0/3313011247 shutdown_connections 2024-04-25T10:14:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.180+0000 7fa26c2e1640 1 -- 172.21.15.16:0/3313011247 wait complete. 2024-04-25T10:14:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.181+0000 7fa26c2e1640 1 Processor -- start 2024-04-25T10:14:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.181+0000 7fa26c2e1640 1 -- start start 2024-04-25T10:14:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.182+0000 7fa26c2e1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa250004800 0x7fa2640b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.182+0000 7fa26c2e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa264152e00 0x7fa26413f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.182+0000 7fa26c2e1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa264140180 0x7fa2640b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.182+0000 7fa26c2e1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2640ce490 con 0x7fa264152e00 2024-04-25T10:14:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.182+0000 7fa26c2e1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2640ce310 con 0x7fa250004800 2024-04-25T10:14:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.182+0000 7fa26c2e1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa264070820 con 0x7fa264140180 2024-04-25T10:14:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.182+0000 7fa26987c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa264152e00 0x7fa26413f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.182+0000 7fa26987c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa264152e00 0x7fa26413f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47284/0 (socket says 172.21.15.16:47284) 2024-04-25T10:14:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.182+0000 7fa26987c640 1 -- 172.21.15.16:0/681821558 learned_addr learned my addr 172.21.15.16:0/681821558 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.182+0000 7fa26a07d640 1 --2- 172.21.15.16:0/681821558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa250004800 0x7fa2640b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.182+0000 7fa26a87e640 1 --2- 172.21.15.16:0/681821558 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa264140180 0x7fa2640b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.182+0000 7fa26987c640 1 -- 172.21.15.16:0/681821558 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa264140180 msgr2=0x7fa2640b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.182+0000 7fa26987c640 1 --2- 172.21.15.16:0/681821558 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa264140180 0x7fa2640b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.182+0000 7fa26987c640 1 -- 172.21.15.16:0/681821558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa250004800 msgr2=0x7fa2640b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.182+0000 7fa26987c640 1 --2- 172.21.15.16:0/681821558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa250004800 0x7fa2640b61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.183+0000 7fa26987c640 1 -- 172.21.15.16:0/681821558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2640c8a00 con 0x7fa264152e00 2024-04-25T10:14:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.183+0000 7fa26987c640 1 --2- 172.21.15.16:0/681821558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa264152e00 0x7fa26413f870 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fa25402abb0 tx=0x7fa254004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.183+0000 7fa25affd640 1 -- 172.21.15.16:0/681821558 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254040020 con 0x7fa264152e00 2024-04-25T10:14:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.183+0000 7fa25affd640 1 -- 172.21.15.16:0/681821558 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa254003940 con 0x7fa264152e00 2024-04-25T10:14:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.183+0000 7fa26c2e1640 1 -- 172.21.15.16:0/681821558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2640c8ca0 con 0x7fa264152e00 2024-04-25T10:14:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.183+0000 7fa25affd640 1 -- 172.21.15.16:0/681821558 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254036040 con 0x7fa264152e00 2024-04-25T10:14:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.183+0000 7fa26c2e1640 1 -- 172.21.15.16:0/681821558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2640c9260 con 0x7fa264152e00 2024-04-25T10:14:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.184+0000 7fa26c2e1640 1 -- 172.21.15.16:0/681821558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2640749b0 con 0x7fa264152e00 2024-04-25T10:14:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.184+0000 7fa25affd640 1 -- 172.21.15.16:0/681821558 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa254043020 con 0x7fa264152e00 2024-04-25T10:14:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.185+0000 7fa25affd640 1 --2- 172.21.15.16:0/681821558 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa24806b500 0x7fa24806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.185+0000 7fa25affd640 1 -- 172.21.15.16:0/681821558 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fa2540a9d50 con 0x7fa264152e00 2024-04-25T10:14:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.185+0000 7fa26a07d640 1 --2- 172.21.15.16:0/681821558 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa24806b500 0x7fa24806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.185+0000 7fa26a07d640 1 --2- 172.21.15.16:0/681821558 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa24806b500 0x7fa24806d9c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa260001550 tx=0x7fa2600078f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.189+0000 7fa25affd640 1 -- 172.21.15.16:0/681821558 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa2540ad030 con 0x7fa264152e00 2024-04-25T10:14:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.371+0000 7fa26c2e1640 1 -- 172.21.15.16:0/681821558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2640c7320 con 0x7fa264152e00 2024-04-25T10:14:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.372+0000 7fa25affd640 1 -- 172.21.15.16:0/681821558 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+17068 (secure 0 0 0) 0x7fa254078460 con 0x7fa264152e00 2024-04-25T10:14:14.376 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:14.376 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":104,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:14:10.849151+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:47.551100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:47.939289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":5,"to":3}]},{"pgid":"14.b","mappings":[{"from":6,"to":0}]},{"pgid":"14.e","mappings":[{"from":5,"to":0}]},{"pgid":"14.16","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.186:6836/2008057690":"2024-04-26T10:14:10.849128+0000","172.21.15.186:6834/2008057690":"2024-04-26T10:14:10.849128+0000","172.21.15.186:6836/1371188714":"2024-04-26T10:13:34.499700+0000","172.21.15.186:6834/1371188714":"2024-04-26T10:13:34.499700+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.374+0000 7fa26c2e1640 1 -- 172.21.15.16:0/681821558 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa24806b500 msgr2=0x7fa24806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.374+0000 7fa26c2e1640 1 --2- 172.21.15.16:0/681821558 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa24806b500 0x7fa24806d9c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa260001550 tx=0x7fa2600078f0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.374+0000 7fa26c2e1640 1 -- 172.21.15.16:0/681821558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa264152e00 msgr2=0x7fa26413f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.374+0000 7fa26c2e1640 1 --2- 172.21.15.16:0/681821558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa264152e00 0x7fa26413f870 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fa25402abb0 tx=0x7fa254004730 comp rx=0 tx=0).stop 2024-04-25T10:14:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.374+0000 7fa26c2e1640 1 -- 172.21.15.16:0/681821558 shutdown_connections 2024-04-25T10:14:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.374+0000 7fa26c2e1640 1 --2- 172.21.15.16:0/681821558 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa24806b500 0x7fa24806d9c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.374+0000 7fa26c2e1640 1 --2- 172.21.15.16:0/681821558 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa264140180 0x7fa2640b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.374+0000 7fa26c2e1640 1 --2- 172.21.15.16:0/681821558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa264152e00 0x7fa26413f870 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.374+0000 7fa26c2e1640 1 --2- 172.21.15.16:0/681821558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa250004800 0x7fa2640b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.374+0000 7fa26c2e1640 1 -- 172.21.15.16:0/681821558 >> 172.21.15.16:0/681821558 conn(0x7fa2640b2370 msgr2=0x7fa2640c55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.374+0000 7fa26c2e1640 1 -- 172.21.15.16:0/681821558 shutdown_connections 2024-04-25T10:14:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.374+0000 7fa26c2e1640 1 -- 172.21.15.16:0/681821558 wait complete. 2024-04-25T10:14:14.390 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:14:14.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.487+0000 7fa97384d640 1 Processor -- start 2024-04-25T10:14:14.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.487+0000 7fa97384d640 1 -- start start 2024-04-25T10:14:14.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.488+0000 7fa97384d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa96c15dec0 0x7fa96c0d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.488+0000 7fa97384d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa96c0d2790 con 0x7fa96c0d22e0 2024-04-25T10:14:14.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.488+0000 7fa97384d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa96c0d2a20 con 0x7fa96c0d0b60 2024-04-25T10:14:14.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.488+0000 7fa97384d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa96c0d2d00 con 0x7fa96c15dec0 2024-04-25T10:14:14.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.488+0000 7fa970de8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa96c0d22e0 0x7fa96c0d0450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47702/0 (socket says 172.21.15.16:47702) 2024-04-25T10:14:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.488+0000 7fa970de8640 1 -- 172.21.15.16:0/2567273952 learned_addr learned my addr 172.21.15.16:0/2567273952 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.488+0000 7fa9715e9640 1 --2- 172.21.15.16:0/2567273952 >> v2:172.21.15.186:3300/0 conn(0x7fa96c15dec0 0x7fa96c0d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa96bfff640 1 -- 172.21.15.16:0/2567273952 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875661072 0 0) 0x7fa96c0d2790 con 0x7fa96c0d22e0 2024-04-25T10:14:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa96bfff640 1 -- 172.21.15.16:0/2567273952 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa954003520 con 0x7fa96c0d22e0 2024-04-25T10:14:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa96bfff640 1 -- 172.21.15.16:0/2567273952 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1721738386 0 0) 0x7fa954003520 con 0x7fa96c0d22e0 2024-04-25T10:14:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa96bfff640 1 -- 172.21.15.16:0/2567273952 >> v2:172.21.15.186:3300/0 conn(0x7fa96c15dec0 msgr2=0x7fa96c0d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa96bfff640 1 --2- 172.21.15.16:0/2567273952 >> v2:172.21.15.186:3300/0 conn(0x7fa96c15dec0 0x7fa96c0d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa96bfff640 1 -- 172.21.15.16:0/2567273952 >> v1:172.21.15.186:6789/0 conn(0x7fa96c0d0b60 legacy=0x7fa96c0b6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa96bfff640 1 -- 172.21.15.16:0/2567273952 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c0d2fe0 con 0x7fa96c0d22e0 2024-04-25T10:14:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa96bfff640 1 -- 172.21.15.16:0/2567273952 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fa960001670 con 0x7fa96c0d22e0 2024-04-25T10:14:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa9715e9640 1 --2- 172.21.15.16:0/2567273952 >> v2:172.21.15.186:3300/0 conn(0x7fa96c15dec0 0x7fa96c0d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa96bfff640 1 -- 172.21.15.16:0/2567273952 >> v1:172.21.15.16:6789/0 conn(0x7fa96c0d22e0 legacy=0x7fa96c0d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa96bfff640 1 --2- 172.21.15.16:0/2567273952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9540039b0 0x7fa954003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa96bfff640 1 --2- 172.21.15.16:0/2567273952 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa954004800 0x7fa954004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa96bfff640 1 --2- 172.21.15.16:0/2567273952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9540051c0 0x7fa95400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa96bfff640 1 -- 172.21.15.16:0/2567273952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c0d2d00 con 0x7fa9540039b0 2024-04-25T10:14:14.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa96bfff640 1 -- 172.21.15.16:0/2567273952 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95400c060 con 0x7fa9540051c0 2024-04-25T10:14:14.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa96bfff640 1 -- 172.21.15.16:0/2567273952 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95400c260 con 0x7fa954004800 2024-04-25T10:14:14.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa9715e9640 1 --2- 172.21.15.16:0/2567273952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9540039b0 0x7fa954003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa970de8640 1 --2- 172.21.15.16:0/2567273952 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa954004800 0x7fa954004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa971dea640 1 --2- 172.21.15.16:0/2567273952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9540051c0 0x7fa95400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa9715e9640 1 -- 172.21.15.16:0/2567273952 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa954004800 msgr2=0x7fa954004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa9715e9640 1 --2- 172.21.15.16:0/2567273952 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa954004800 0x7fa954004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa9715e9640 1 -- 172.21.15.16:0/2567273952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9540051c0 msgr2=0x7fa95400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa9715e9640 1 --2- 172.21.15.16:0/2567273952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9540051c0 0x7fa95400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa9715e9640 1 -- 172.21.15.16:0/2567273952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa954003520 con 0x7fa9540039b0 2024-04-25T10:14:14.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa971dea640 1 --2- 172.21.15.16:0/2567273952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9540051c0 0x7fa95400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:14.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.489+0000 7fa971dea640 1 -- 172.21.15.16:0/2567273952 reap_dead start 2024-04-25T10:14:14.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.490+0000 7fa9715e9640 1 --2- 172.21.15.16:0/2567273952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9540039b0 0x7fa954003dd0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fa964009c90 tx=0x7fa954005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de811bcc628f2356 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.490+0000 7fa96bfff640 1 -- 172.21.15.16:0/2567273952 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964036070 con 0x7fa9540039b0 2024-04-25T10:14:14.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.490+0000 7fa96bfff640 1 -- 172.21.15.16:0/2567273952 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa964031400 con 0x7fa9540039b0 2024-04-25T10:14:14.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.490+0000 7fa96bfff640 1 -- 172.21.15.16:0/2567273952 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964009860 con 0x7fa9540039b0 2024-04-25T10:14:14.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.491+0000 7fa97384d640 1 -- 172.21.15.16:0/2567273952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9540039b0 msgr2=0x7fa954003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.491+0000 7fa97384d640 1 --2- 172.21.15.16:0/2567273952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9540039b0 0x7fa954003dd0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fa964009c90 tx=0x7fa954005830 comp rx=0 tx=0).stop 2024-04-25T10:14:14.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.491+0000 7fa97384d640 1 -- 172.21.15.16:0/2567273952 shutdown_connections 2024-04-25T10:14:14.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.491+0000 7fa97384d640 1 --2- 172.21.15.16:0/2567273952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9540039b0 0x7fa954003dd0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.491+0000 7fa97384d640 1 -- 172.21.15.16:0/2567273952 >> 172.21.15.16:0/2567273952 conn(0x7fa96c072900 msgr2=0x7fa96c0b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.491+0000 7fa97384d640 1 -- 172.21.15.16:0/2567273952 shutdown_connections 2024-04-25T10:14:14.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.491+0000 7fa97384d640 1 -- 172.21.15.16:0/2567273952 wait complete. 2024-04-25T10:14:14.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.492+0000 7fa97384d640 1 Processor -- start 2024-04-25T10:14:14.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.492+0000 7fa97384d640 1 -- start start 2024-04-25T10:14:14.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.493+0000 7fa97384d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9540039b0 0x7fa96c158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.493+0000 7fa97384d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c1492b0 0x7fa96c15e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.493+0000 7fa97384d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa96c15eab0 0x7fa96c14a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.493+0000 7fa97384d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c0d1da0 con 0x7fa96c1492b0 2024-04-25T10:14:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.493+0000 7fa97384d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c0d35a0 con 0x7fa9540039b0 2024-04-25T10:14:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.493+0000 7fa97384d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c0d1f20 con 0x7fa96c15eab0 2024-04-25T10:14:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.493+0000 7fa970de8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c1492b0 0x7fa96c15e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.493+0000 7fa970de8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c1492b0 0x7fa96c15e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47298/0 (socket says 172.21.15.16:47298) 2024-04-25T10:14:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.493+0000 7fa970de8640 1 -- 172.21.15.16:0/2855716113 learned_addr learned my addr 172.21.15.16:0/2855716113 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.493+0000 7fa9715e9640 1 --2- 172.21.15.16:0/2855716113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9540039b0 0x7fa96c158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.493+0000 7fa971dea640 1 --2- 172.21.15.16:0/2855716113 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa96c15eab0 0x7fa96c14a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.493+0000 7fa970de8640 1 -- 172.21.15.16:0/2855716113 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa96c15eab0 msgr2=0x7fa96c14a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.493+0000 7fa970de8640 1 --2- 172.21.15.16:0/2855716113 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa96c15eab0 0x7fa96c14a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.493+0000 7fa970de8640 1 -- 172.21.15.16:0/2855716113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9540039b0 msgr2=0x7fa96c158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.493+0000 7fa970de8640 1 --2- 172.21.15.16:0/2855716113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9540039b0 0x7fa96c158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.493+0000 7fa970de8640 1 -- 172.21.15.16:0/2855716113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c149740 con 0x7fa96c1492b0 2024-04-25T10:14:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.494+0000 7fa970de8640 1 --2- 172.21.15.16:0/2855716113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c1492b0 0x7fa96c15e570 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fa96000e830 tx=0x7fa96000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.494+0000 7fa969ffb640 1 -- 172.21.15.16:0/2855716113 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96000c8d0 con 0x7fa96c1492b0 2024-04-25T10:14:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.494+0000 7fa969ffb640 1 -- 172.21.15.16:0/2855716113 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa960018070 con 0x7fa96c1492b0 2024-04-25T10:14:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.494+0000 7fa97384d640 1 -- 172.21.15.16:0/2855716113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96c149a20 con 0x7fa96c1492b0 2024-04-25T10:14:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.494+0000 7fa969ffb640 1 -- 172.21.15.16:0/2855716113 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96000ccb0 con 0x7fa96c1492b0 2024-04-25T10:14:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.494+0000 7fa97384d640 1 -- 172.21.15.16:0/2855716113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96c149fe0 con 0x7fa96c1492b0 2024-04-25T10:14:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.495+0000 7fa97384d640 1 -- 172.21.15.16:0/2855716113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa96c0750b0 con 0x7fa96c1492b0 2024-04-25T10:14:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.495+0000 7fa969ffb640 1 -- 172.21.15.16:0/2855716113 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa96001d460 con 0x7fa96c1492b0 2024-04-25T10:14:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.496+0000 7fa969ffb640 1 --2- 172.21.15.16:0/2855716113 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa95806b4d0 0x7fa95806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.496+0000 7fa969ffb640 1 -- 172.21.15.16:0/2855716113 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fa96008a140 con 0x7fa96c1492b0 2024-04-25T10:14:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.496+0000 7fa9715e9640 1 --2- 172.21.15.16:0/2855716113 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa95806b4d0 0x7fa95806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.496+0000 7fa9715e9640 1 --2- 172.21.15.16:0/2855716113 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa95806b4d0 0x7fa95806d990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa964033000 tx=0x7fa964008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.500+0000 7fa969ffb640 1 -- 172.21.15.16:0/2855716113 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa960058840 con 0x7fa96c1492b0 2024-04-25T10:14:14.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:14.717+0000 7fa97384d640 1 -- 172.21.15.16:0/2855716113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa96c0b4650 con 0x7fa96c1492b0 2024-04-25T10:14:15.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.015+0000 7fa969ffb640 1 -- 172.21.15.16:0/2855716113 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v79) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa96005c8e0 con 0x7fa96c1492b0 2024-04-25T10:14:15.019 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:14:15.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.017+0000 7fa97384d640 1 -- 172.21.15.16:0/2855716113 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa95806b4d0 msgr2=0x7fa95806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.017+0000 7fa97384d640 1 --2- 172.21.15.16:0/2855716113 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa95806b4d0 0x7fa95806d990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa964033000 tx=0x7fa964008030 comp rx=0 tx=0).stop 2024-04-25T10:14:15.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.017+0000 7fa97384d640 1 -- 172.21.15.16:0/2855716113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c1492b0 msgr2=0x7fa96c15e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.017+0000 7fa97384d640 1 --2- 172.21.15.16:0/2855716113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c1492b0 0x7fa96c15e570 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fa96000e830 tx=0x7fa96000ecf0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.017+0000 7fa97384d640 1 -- 172.21.15.16:0/2855716113 shutdown_connections 2024-04-25T10:14:15.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.017+0000 7fa97384d640 1 --2- 172.21.15.16:0/2855716113 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa95806b4d0 0x7fa95806d990 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.017+0000 7fa97384d640 1 --2- 172.21.15.16:0/2855716113 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa96c15eab0 0x7fa96c14a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.017+0000 7fa97384d640 1 --2- 172.21.15.16:0/2855716113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c1492b0 0x7fa96c15e570 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.017+0000 7fa97384d640 1 --2- 172.21.15.16:0/2855716113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9540039b0 0x7fa96c158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.017+0000 7fa97384d640 1 -- 172.21.15.16:0/2855716113 >> 172.21.15.16:0/2855716113 conn(0x7fa96c072900 msgr2=0x7fa96c0b4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.017+0000 7fa97384d640 1 -- 172.21.15.16:0/2855716113 shutdown_connections 2024-04-25T10:14:15.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.017+0000 7fa97384d640 1 -- 172.21.15.16:0/2855716113 wait complete. 2024-04-25T10:14:15.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:14:15.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.133+0000 7f145cf1b640 1 Processor -- start 2024-04-25T10:14:15.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.133+0000 7f145cf1b640 1 -- start start 2024-04-25T10:14:15.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.134+0000 7f145cf1b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1458161120 0x7f14581617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.134+0000 7f145cf1b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1458161ce0 0x7f14580b5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.134+0000 7f145cf1b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f14580b5a90 0x7f145806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.134+0000 7f145cf1b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1458162740 con 0x7f1458161120 2024-04-25T10:14:15.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.134+0000 7f145cf1b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1458161500 con 0x7f14580b5a90 2024-04-25T10:14:15.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.134+0000 7f145cf1b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1458162180 con 0x7f1458161ce0 2024-04-25T10:14:15.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.134+0000 7f145659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1458161120 0x7f14581617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.134+0000 7f145659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1458161120 0x7f14581617a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47314/0 (socket says 172.21.15.16:47314) 2024-04-25T10:14:15.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.134+0000 7f145659c640 1 -- 172.21.15.16:0/2845412178 learned_addr learned my addr 172.21.15.16:0/2845412178 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:15.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.134+0000 7f144ffff640 1 --2- 172.21.15.16:0/2845412178 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1458161ce0 0x7f14580b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.134+0000 7f1456d9d640 1 --2- 172.21.15.16:0/2845412178 >> v2:172.21.15.186:3300/0 conn(0x7f14580b5a90 0x7f145806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.134+0000 7f145659c640 1 -- 172.21.15.16:0/2845412178 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1458161ce0 msgr2=0x7f14580b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.134+0000 7f145659c640 1 --2- 172.21.15.16:0/2845412178 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1458161ce0 0x7f14580b5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.134+0000 7f145659c640 1 -- 172.21.15.16:0/2845412178 >> v2:172.21.15.186:3300/0 conn(0x7f14580b5a90 msgr2=0x7f145806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.134+0000 7f145659c640 1 --2- 172.21.15.16:0/2845412178 >> v2:172.21.15.186:3300/0 conn(0x7f14580b5a90 0x7f145806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.134+0000 7f145659c640 1 -- 172.21.15.16:0/2845412178 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1458162460 con 0x7f1458161120 2024-04-25T10:14:15.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.134+0000 7f144ffff640 1 --2- 172.21.15.16:0/2845412178 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1458161ce0 0x7f14580b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:15.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.134+0000 7f1456d9d640 1 --2- 172.21.15.16:0/2845412178 >> v2:172.21.15.186:3300/0 conn(0x7f14580b5a90 0x7f145806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:15.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.135+0000 7f145659c640 1 --2- 172.21.15.16:0/2845412178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1458161120 0x7f14581617a0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f14580cab30 tx=0x7f144802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a4129c58f5d8b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.135+0000 7f1455d9b640 1 -- 172.21.15.16:0/2845412178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1448037070 con 0x7f1458161120 2024-04-25T10:14:15.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.135+0000 7f1455d9b640 1 -- 172.21.15.16:0/2845412178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1448032440 con 0x7f1458161120 2024-04-25T10:14:15.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.135+0000 7f1455d9b640 1 -- 172.21.15.16:0/2845412178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1448032810 con 0x7f1458161120 2024-04-25T10:14:15.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.135+0000 7f145cf1b640 1 -- 172.21.15.16:0/2845412178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1458161120 msgr2=0x7f14581617a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.135+0000 7f145cf1b640 1 --2- 172.21.15.16:0/2845412178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1458161120 0x7f14581617a0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f14580cab30 tx=0x7f144802a140 comp rx=0 tx=0).stop 2024-04-25T10:14:15.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.136+0000 7f145cf1b640 1 -- 172.21.15.16:0/2845412178 shutdown_connections 2024-04-25T10:14:15.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.136+0000 7f145cf1b640 1 --2- 172.21.15.16:0/2845412178 >> v2:172.21.15.186:3300/0 conn(0x7f14580b5a90 0x7f145806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.136+0000 7f145cf1b640 1 --2- 172.21.15.16:0/2845412178 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1458161ce0 0x7f14580b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.136+0000 7f145cf1b640 1 --2- 172.21.15.16:0/2845412178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1458161120 0x7f14581617a0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.136+0000 7f145cf1b640 1 -- 172.21.15.16:0/2845412178 >> 172.21.15.16:0/2845412178 conn(0x7f14580d0550 msgr2=0x7f14580d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.136+0000 7f145cf1b640 1 -- 172.21.15.16:0/2845412178 shutdown_connections 2024-04-25T10:14:15.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.136+0000 7f145cf1b640 1 -- 172.21.15.16:0/2845412178 wait complete. 2024-04-25T10:14:15.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.137+0000 7f145cf1b640 1 Processor -- start 2024-04-25T10:14:15.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.137+0000 7f145cf1b640 1 -- start start 2024-04-25T10:14:15.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f145cf1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14580b5a90 0x7f14581f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f145cf1b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1458161120 0x7f14581566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f145cf1b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1458161ce0 0x7f145815bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f145cf1b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14580b95f0 con 0x7f14580b5a90 2024-04-25T10:14:15.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f145cf1b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14580d2040 con 0x7f1458161120 2024-04-25T10:14:15.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f145cf1b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14580b9770 con 0x7f1458161ce0 2024-04-25T10:14:15.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f145659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14580b5a90 0x7f14581f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f145659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14580b5a90 0x7f14581f4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47320/0 (socket says 172.21.15.16:47320) 2024-04-25T10:14:15.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f145659c640 1 -- 172.21.15.16:0/2676030885 learned_addr learned my addr 172.21.15.16:0/2676030885 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:15.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f144ffff640 1 --2- 172.21.15.16:0/2676030885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1458161120 0x7f14581566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f1456d9d640 1 --2- 172.21.15.16:0/2676030885 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1458161ce0 0x7f145815bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f145659c640 1 -- 172.21.15.16:0/2676030885 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1458161ce0 msgr2=0x7f145815bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f145659c640 1 --2- 172.21.15.16:0/2676030885 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1458161ce0 0x7f145815bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f145659c640 1 -- 172.21.15.16:0/2676030885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1458161120 msgr2=0x7f14581566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f145659c640 1 --2- 172.21.15.16:0/2676030885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1458161120 0x7f14581566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f145659c640 1 -- 172.21.15.16:0/2676030885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14581f4840 con 0x7f14580b5a90 2024-04-25T10:14:15.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f145659c640 1 --2- 172.21.15.16:0/2676030885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14580b5a90 0x7f14581f4300 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f144802a180 tx=0x7f14480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f144effd640 1 -- 172.21.15.16:0/2676030885 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1448037040 con 0x7f14580b5a90 2024-04-25T10:14:15.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f144effd640 1 -- 172.21.15.16:0/2676030885 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1448042070 con 0x7f14580b5a90 2024-04-25T10:14:15.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f145cf1b640 1 -- 172.21.15.16:0/2676030885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1458157e60 con 0x7f14580b5a90 2024-04-25T10:14:15.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.138+0000 7f144effd640 1 -- 172.21.15.16:0/2676030885 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1448030bb0 con 0x7f14580b5a90 2024-04-25T10:14:15.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.139+0000 7f145cf1b640 1 -- 172.21.15.16:0/2676030885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1458158420 con 0x7f14580b5a90 2024-04-25T10:14:15.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.140+0000 7f145cf1b640 1 -- 172.21.15.16:0/2676030885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14580750b0 con 0x7f14580b5a90 2024-04-25T10:14:15.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.140+0000 7f144effd640 1 -- 172.21.15.16:0/2676030885 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1448047460 con 0x7f14580b5a90 2024-04-25T10:14:15.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.140+0000 7f144effd640 1 --2- 172.21.15.16:0/2676030885 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f143806b4d0 0x7f143806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.141+0000 7f144effd640 1 -- 172.21.15.16:0/2676030885 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f14480aaec0 con 0x7f14580b5a90 2024-04-25T10:14:15.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.141+0000 7f144ffff640 1 --2- 172.21.15.16:0/2676030885 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f143806b4d0 0x7f143806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.141+0000 7f144ffff640 1 --2- 172.21.15.16:0/2676030885 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f143806b4d0 0x7f143806d990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f14580b34d0 tx=0x7f1440008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.145+0000 7f144effd640 1 -- 172.21.15.16:0/2676030885 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14581f4840 con 0x7f14580b5a90 2024-04-25T10:14:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:15.365+0000 7f145cf1b640 1 -- 172.21.15.16:0/2676030885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f14580d17e0 con 0x7f14580b5a90 2024-04-25T10:14:16.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.023+0000 7f144effd640 1 -- 172.21.15.16:0/2676030885 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v80) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1448045360 con 0x7f14580b5a90 2024-04-25T10:14:16.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.025+0000 7f145cf1b640 1 -- 172.21.15.16:0/2676030885 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f143806b4d0 msgr2=0x7f143806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.025+0000 7f145cf1b640 1 --2- 172.21.15.16:0/2676030885 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f143806b4d0 0x7f143806d990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f14580b34d0 tx=0x7f1440008040 comp rx=0 tx=0).stop 2024-04-25T10:14:16.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.025+0000 7f145cf1b640 1 -- 172.21.15.16:0/2676030885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14580b5a90 msgr2=0x7f14581f4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.025+0000 7f145cf1b640 1 --2- 172.21.15.16:0/2676030885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14580b5a90 0x7f14581f4300 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f144802a180 tx=0x7f14480046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.025+0000 7f145cf1b640 1 -- 172.21.15.16:0/2676030885 shutdown_connections 2024-04-25T10:14:16.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.025+0000 7f145cf1b640 1 --2- 172.21.15.16:0/2676030885 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f143806b4d0 0x7f143806d990 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.025+0000 7f145cf1b640 1 --2- 172.21.15.16:0/2676030885 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1458161ce0 0x7f145815bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.025+0000 7f145cf1b640 1 --2- 172.21.15.16:0/2676030885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1458161120 0x7f14581566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.025+0000 7f145cf1b640 1 --2- 172.21.15.16:0/2676030885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14580b5a90 0x7f14581f4300 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.025+0000 7f145cf1b640 1 -- 172.21.15.16:0/2676030885 >> 172.21.15.16:0/2676030885 conn(0x7f14580d0550 msgr2=0x7f14580ceba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:16.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.025+0000 7f145cf1b640 1 -- 172.21.15.16:0/2676030885 shutdown_connections 2024-04-25T10:14:16.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.025+0000 7f145cf1b640 1 -- 172.21.15.16:0/2676030885 wait complete. 2024-04-25T10:14:16.042 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:14:16.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.147+0000 7f3643a5d640 1 Processor -- start 2024-04-25T10:14:16.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.147+0000 7f3643a5d640 1 -- start start 2024-04-25T10:14:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.148+0000 7f3643a5d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f363c0ce1e0 0x7f363c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.148+0000 7f3643a5d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f363c0cc240 0x7f363c0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.148+0000 7f3643a5d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363c15f490 con 0x7f363c15f0b0 2024-04-25T10:14:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.148+0000 7f3643a5d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c0cebb0 con 0x7f363c0cc240 2024-04-25T10:14:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.148+0000 7f3643a5d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c0cf8f0 con 0x7f363c0ce1e0 2024-04-25T10:14:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.148+0000 7f3641ffa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f363c0cc240 0x7f363c0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.148+0000 7f3641ffa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f363c0cc240 0x7f363c0cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47332/0 (socket says 172.21.15.16:47332) 2024-04-25T10:14:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.148+0000 7f3641ffa640 1 -- 172.21.15.16:0/1123158968 learned_addr learned my addr 172.21.15.16:0/1123158968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.148+0000 7f3640ff8640 1 --2- 172.21.15.16:0/1123158968 >> v2:172.21.15.186:3300/0 conn(0x7f363c0ce1e0 0x7f363c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.148+0000 7f3641ffa640 1 -- 172.21.15.16:0/1123158968 >> v2:172.21.15.186:3300/0 conn(0x7f363c0ce1e0 msgr2=0x7f363c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.148+0000 7f3641ffa640 1 --2- 172.21.15.16:0/1123158968 >> v2:172.21.15.186:3300/0 conn(0x7f363c0ce1e0 0x7f363c0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.148+0000 7f3641ffa640 1 -- 172.21.15.16:0/1123158968 >> v1:172.21.15.186:6789/0 conn(0x7f363c15f0b0 legacy=0x7f363c0d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.148+0000 7f3641ffa640 1 -- 172.21.15.16:0/1123158968 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c0cfbd0 con 0x7f363c0cc240 2024-04-25T10:14:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.149+0000 7f3641ffa640 1 --2- 172.21.15.16:0/1123158968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363c0cc240 0x7f363c0cc670 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f363c073060 tx=0x7f363402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7796a959aa37c4ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.149+0000 7f3633fff640 1 -- 172.21.15.16:0/1123158968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634036070 con 0x7f363c0cc240 2024-04-25T10:14:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.149+0000 7f3633fff640 1 -- 172.21.15.16:0/1123158968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3634031600 con 0x7f363c0cc240 2024-04-25T10:14:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.149+0000 7f3633fff640 1 -- 172.21.15.16:0/1123158968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36340319f0 con 0x7f363c0cc240 2024-04-25T10:14:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.150+0000 7f3643a5d640 1 -- 172.21.15.16:0/1123158968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363c0cc240 msgr2=0x7f363c0cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.150+0000 7f3643a5d640 1 --2- 172.21.15.16:0/1123158968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363c0cc240 0x7f363c0cc670 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f363c073060 tx=0x7f363402a7c0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.150+0000 7f3643a5d640 1 -- 172.21.15.16:0/1123158968 shutdown_connections 2024-04-25T10:14:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.150+0000 7f3643a5d640 1 --2- 172.21.15.16:0/1123158968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363c0cc240 0x7f363c0cc670 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.150+0000 7f3643a5d640 1 --2- 172.21.15.16:0/1123158968 >> v2:172.21.15.186:3300/0 conn(0x7f363c0ce1e0 0x7f363c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.150+0000 7f3643a5d640 1 -- 172.21.15.16:0/1123158968 >> 172.21.15.16:0/1123158968 conn(0x7f363c06f430 msgr2=0x7f363c06f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.150+0000 7f3643a5d640 1 -- 172.21.15.16:0/1123158968 shutdown_connections 2024-04-25T10:14:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.150+0000 7f3643a5d640 1 -- 172.21.15.16:0/1123158968 wait complete. 2024-04-25T10:14:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.151+0000 7f3643a5d640 1 Processor -- start 2024-04-25T10:14:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.151+0000 7f3643a5d640 1 -- start start 2024-04-25T10:14:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.152+0000 7f3643a5d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f363c0cc240 0x7f363c14f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.152+0000 7f3643a5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363c0ce1e0 0x7f363c0c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.152+0000 7f3643a5d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f363c15f0b0 0x7f363c0c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.152+0000 7f3643a5d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c0d0470 con 0x7f363c0ce1e0 2024-04-25T10:14:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.152+0000 7f3643a5d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c0d0190 con 0x7f363c0cc240 2024-04-25T10:14:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.152+0000 7f3643a5d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c0ccbb0 con 0x7f363c15f0b0 2024-04-25T10:14:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.152+0000 7f3640ff8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363c0ce1e0 0x7f363c0c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.152+0000 7f3640ff8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363c0ce1e0 0x7f363c0c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47336/0 (socket says 172.21.15.16:47336) 2024-04-25T10:14:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.152+0000 7f3640ff8640 1 -- 172.21.15.16:0/1246018275 learned_addr learned my addr 172.21.15.16:0/1246018275 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.152+0000 7f36417f9640 1 --2- 172.21.15.16:0/1246018275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f363c0cc240 0x7f363c14f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.152+0000 7f3641ffa640 1 --2- 172.21.15.16:0/1246018275 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f363c15f0b0 0x7f363c0c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.152+0000 7f3640ff8640 1 -- 172.21.15.16:0/1246018275 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f363c15f0b0 msgr2=0x7f363c0c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.152+0000 7f3640ff8640 1 --2- 172.21.15.16:0/1246018275 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f363c15f0b0 0x7f363c0c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.152+0000 7f3640ff8640 1 -- 172.21.15.16:0/1246018275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f363c0cc240 msgr2=0x7f363c14f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.152+0000 7f3640ff8640 1 --2- 172.21.15.16:0/1246018275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f363c0cc240 0x7f363c14f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.152+0000 7f3640ff8640 1 -- 172.21.15.16:0/1246018275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c0c6ea0 con 0x7f363c0ce1e0 2024-04-25T10:14:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.152+0000 7f3641ffa640 1 --2- 172.21.15.16:0/1246018275 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f363c15f0b0 0x7f363c0c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.152+0000 7f3640ff8640 1 --2- 172.21.15.16:0/1246018275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363c0ce1e0 0x7f363c0c4a20 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f362c00d800 tx=0x7f362c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.153+0000 7f3631ffb640 1 -- 172.21.15.16:0/1246018275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c00b6c0 con 0x7f363c0ce1e0 2024-04-25T10:14:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.153+0000 7f3631ffb640 1 -- 172.21.15.16:0/1246018275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c019070 con 0x7f363c0ce1e0 2024-04-25T10:14:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.153+0000 7f3643a5d640 1 -- 172.21.15.16:0/1246018275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363c0c7180 con 0x7f363c0ce1e0 2024-04-25T10:14:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.153+0000 7f3631ffb640 1 -- 172.21.15.16:0/1246018275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c00ba80 con 0x7f363c0ce1e0 2024-04-25T10:14:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.153+0000 7f3643a5d640 1 -- 172.21.15.16:0/1246018275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363c0c7740 con 0x7f363c0ce1e0 2024-04-25T10:14:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.154+0000 7f3643a5d640 1 -- 172.21.15.16:0/1246018275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363c0750b0 con 0x7f363c0ce1e0 2024-04-25T10:14:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.154+0000 7f3631ffb640 1 -- 172.21.15.16:0/1246018275 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f362c00bc20 con 0x7f363c0ce1e0 2024-04-25T10:14:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.155+0000 7f3631ffb640 1 --2- 172.21.15.16:0/1246018275 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f362006b500 0x7f362006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.155+0000 7f3631ffb640 1 -- 172.21.15.16:0/1246018275 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f362c08a710 con 0x7f363c0ce1e0 2024-04-25T10:14:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.155+0000 7f36417f9640 1 --2- 172.21.15.16:0/1246018275 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f362006b500 0x7f362006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.155+0000 7f36417f9640 1 --2- 172.21.15.16:0/1246018275 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f362006b500 0x7f362006d9c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f36380016e0 tx=0x7f3638009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.159+0000 7f3631ffb640 1 -- 172.21.15.16:0/1246018275 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f362c059470 con 0x7f363c0ce1e0 2024-04-25T10:14:16.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:16.337+0000 7f3643a5d640 1 -- 172.21.15.16:0/1246018275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f363c0b5400 con 0x7f363c0ce1e0 2024-04-25T10:14:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.027+0000 7f3631ffb640 1 -- 172.21.15.16:0/1246018275 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) v1 ==== 181+0+0 (secure 0 0 0) 0x7f362c058d70 con 0x7f363c0ce1e0 2024-04-25T10:14:17.032 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:14:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.031+0000 7f3643a5d640 1 -- 172.21.15.16:0/1246018275 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f362006b500 msgr2=0x7f362006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.031+0000 7f3643a5d640 1 --2- 172.21.15.16:0/1246018275 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f362006b500 0x7f362006d9c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f36380016e0 tx=0x7f3638009040 comp rx=0 tx=0).stop 2024-04-25T10:14:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.031+0000 7f3643a5d640 1 -- 172.21.15.16:0/1246018275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363c0ce1e0 msgr2=0x7f363c0c4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.031+0000 7f3643a5d640 1 --2- 172.21.15.16:0/1246018275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363c0ce1e0 0x7f363c0c4a20 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f362c00d800 tx=0x7f362c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.031+0000 7f3643a5d640 1 -- 172.21.15.16:0/1246018275 shutdown_connections 2024-04-25T10:14:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.031+0000 7f3643a5d640 1 --2- 172.21.15.16:0/1246018275 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f362006b500 0x7f362006d9c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.031+0000 7f3643a5d640 1 --2- 172.21.15.16:0/1246018275 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f363c15f0b0 0x7f363c0c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.031+0000 7f3643a5d640 1 --2- 172.21.15.16:0/1246018275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363c0ce1e0 0x7f363c0c4a20 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.031+0000 7f3643a5d640 1 --2- 172.21.15.16:0/1246018275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f363c0cc240 0x7f363c14f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.031+0000 7f3643a5d640 1 -- 172.21.15.16:0/1246018275 >> 172.21.15.16:0/1246018275 conn(0x7f363c06f430 msgr2=0x7f363c0b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:17.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.032+0000 7f3643a5d640 1 -- 172.21.15.16:0/1246018275 shutdown_connections 2024-04-25T10:14:17.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.032+0000 7f3643a5d640 1 -- 172.21.15.16:0/1246018275 wait complete. 2024-04-25T10:14:17.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:14:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.156+0000 7fe413c15640 1 Processor -- start 2024-04-25T10:14:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.157+0000 7fe413c15640 1 -- start start 2024-04-25T10:14:17.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.157+0000 7fe413c15640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40c06b040 0x7fe40c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe413c15640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe40c161f10 0x7fe40c162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe413c15640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40c0cf980 con 0x7fe40c06b420 2024-04-25T10:14:17.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe413c15640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40c1628a0 con 0x7fe40c161f10 2024-04-25T10:14:17.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe413c15640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c0cfc20 con 0x7fe40c06b040 2024-04-25T10:14:17.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4111b0640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe40c06b420 0x7fe40c0b29a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47716/0 (socket says 172.21.15.16:47716) 2024-04-25T10:14:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4111b0640 1 -- 172.21.15.16:0/2928754450 learned_addr learned my addr 172.21.15.16:0/2928754450 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4121b2640 1 --2- 172.21.15.16:0/2928754450 >> v2:172.21.15.186:3300/0 conn(0x7fe40c161f10 0x7fe40c162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4119b1640 1 --2- 172.21.15.16:0/2928754450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40c06b040 0x7fe40c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4109af640 1 -- 172.21.15.16:0/2928754450 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 182209804 0 0) 0x7fe40c0cf980 con 0x7fe40c06b420 2024-04-25T10:14:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4109af640 1 -- 172.21.15.16:0/2928754450 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3f8003520 con 0x7fe40c06b420 2024-04-25T10:14:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4109af640 1 -- 172.21.15.16:0/2928754450 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2159802062 0 0) 0x7fe3f8003520 con 0x7fe40c06b420 2024-04-25T10:14:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4109af640 1 -- 172.21.15.16:0/2928754450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40c06b040 msgr2=0x7fe40c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4109af640 1 --2- 172.21.15.16:0/2928754450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40c06b040 0x7fe40c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4109af640 1 -- 172.21.15.16:0/2928754450 >> v2:172.21.15.186:3300/0 conn(0x7fe40c161f10 msgr2=0x7fe40c162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4109af640 1 --2- 172.21.15.16:0/2928754450 >> v2:172.21.15.186:3300/0 conn(0x7fe40c161f10 0x7fe40c162360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4109af640 1 -- 172.21.15.16:0/2928754450 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40c0cff00 con 0x7fe40c06b420 2024-04-25T10:14:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4109af640 1 -- 172.21.15.16:0/2928754450 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fe3fc001670 con 0x7fe40c06b420 2024-04-25T10:14:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4109af640 1 -- 172.21.15.16:0/2928754450 >> v1:172.21.15.16:6789/0 conn(0x7fe40c06b420 legacy=0x7fe40c0b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4109af640 1 --2- 172.21.15.16:0/2928754450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3f80039b0 0x7fe3f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4109af640 1 --2- 172.21.15.16:0/2928754450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40c06b040 0x7fe3f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4109af640 1 --2- 172.21.15.16:0/2928754450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f8004d40 0x7fe3f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4109af640 1 -- 172.21.15.16:0/2928754450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c1628a0 con 0x7fe3f8004d40 2024-04-25T10:14:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4109af640 1 -- 172.21.15.16:0/2928754450 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c0cfc20 con 0x7fe3f80039b0 2024-04-25T10:14:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.158+0000 7fe4109af640 1 -- 172.21.15.16:0/2928754450 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f800b9c0 con 0x7fe40c06b040 2024-04-25T10:14:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.159+0000 7fe4121b2640 1 --2- 172.21.15.16:0/2928754450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f8004d40 0x7fe3f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.159+0000 7fe4119b1640 1 --2- 172.21.15.16:0/2928754450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3f80039b0 0x7fe3f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.159+0000 7fe4111b0640 1 --2- 172.21.15.16:0/2928754450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40c06b040 0x7fe3f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.159+0000 7fe4121b2640 1 -- 172.21.15.16:0/2928754450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40c06b040 msgr2=0x7fe3f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.159+0000 7fe4121b2640 1 --2- 172.21.15.16:0/2928754450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40c06b040 0x7fe3f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.159+0000 7fe4121b2640 1 -- 172.21.15.16:0/2928754450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3f80039b0 msgr2=0x7fe3f8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.159+0000 7fe4121b2640 1 --2- 172.21.15.16:0/2928754450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3f80039b0 0x7fe3f8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.159+0000 7fe4121b2640 1 -- 172.21.15.16:0/2928754450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f8003520 con 0x7fe3f8004d40 2024-04-25T10:14:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.159+0000 7fe4119b1640 1 --2- 172.21.15.16:0/2928754450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3f80039b0 0x7fe3f8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.160+0000 7fe4121b2640 1 --2- 172.21.15.16:0/2928754450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f8004d40 0x7fe3f80051c0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fe404009fc0 tx=0x7fe3f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5055d08b2340ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.160+0000 7fe4109af640 1 -- 172.21.15.16:0/2928754450 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe404009c10 con 0x7fe3f8004d40 2024-04-25T10:14:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.160+0000 7fe4109af640 1 -- 172.21.15.16:0/2928754450 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe404036070 con 0x7fe3f8004d40 2024-04-25T10:14:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.160+0000 7fe4109af640 1 -- 172.21.15.16:0/2928754450 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40402d070 con 0x7fe3f8004d40 2024-04-25T10:14:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.160+0000 7fe413c15640 1 -- 172.21.15.16:0/2928754450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f8004d40 msgr2=0x7fe3f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.160+0000 7fe413c15640 1 --2- 172.21.15.16:0/2928754450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f8004d40 0x7fe3f80051c0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fe404009fc0 tx=0x7fe3f8005830 comp rx=0 tx=0).stop 2024-04-25T10:14:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.160+0000 7fe4121b2640 1 -- 172.21.15.16:0/2928754450 reap_dead start 2024-04-25T10:14:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.160+0000 7fe413c15640 1 -- 172.21.15.16:0/2928754450 shutdown_connections 2024-04-25T10:14:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.160+0000 7fe413c15640 1 -- 172.21.15.16:0/2928754450 >> 172.21.15.16:0/2928754450 conn(0x7fe40c099a40 msgr2=0x7fe40c0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.160+0000 7fe413c15640 1 -- 172.21.15.16:0/2928754450 shutdown_connections 2024-04-25T10:14:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.160+0000 7fe413c15640 1 -- 172.21.15.16:0/2928754450 wait complete. 2024-04-25T10:14:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.162+0000 7fe413c15640 1 Processor -- start 2024-04-25T10:14:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.162+0000 7fe413c15640 1 -- start start 2024-04-25T10:14:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.162+0000 7fe413c15640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe40c155be0 0x7fe40c0cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.162+0000 7fe413c15640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40c155fc0 0x7fe40c0cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.162+0000 7fe413c15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe40c0c9f40 0x7fe40c0ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.162+0000 7fe413c15640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c0ccba0 con 0x7fe40c0c9f40 2024-04-25T10:14:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.162+0000 7fe413c15640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c0b30b0 con 0x7fe40c155be0 2024-04-25T10:14:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.162+0000 7fe413c15640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c0d04c0 con 0x7fe40c155fc0 2024-04-25T10:14:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.163+0000 7fe4121b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe40c0c9f40 0x7fe40c0ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.163+0000 7fe4121b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe40c0c9f40 0x7fe40c0ca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47354/0 (socket says 172.21.15.16:47354) 2024-04-25T10:14:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.163+0000 7fe4121b2640 1 -- 172.21.15.16:0/3720949601 learned_addr learned my addr 172.21.15.16:0/3720949601 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.163+0000 7fe4119b1640 1 --2- 172.21.15.16:0/3720949601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe40c155be0 0x7fe40c0cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.163+0000 7fe4111b0640 1 --2- 172.21.15.16:0/3720949601 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40c155fc0 0x7fe40c0cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.163+0000 7fe4121b2640 1 -- 172.21.15.16:0/3720949601 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40c155fc0 msgr2=0x7fe40c0cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.163+0000 7fe4121b2640 1 --2- 172.21.15.16:0/3720949601 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40c155fc0 0x7fe40c0cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.163+0000 7fe4121b2640 1 -- 172.21.15.16:0/3720949601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe40c155be0 msgr2=0x7fe40c0cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.163+0000 7fe4121b2640 1 --2- 172.21.15.16:0/3720949601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe40c155be0 0x7fe40c0cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.163+0000 7fe4121b2640 1 -- 172.21.15.16:0/3720949601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40c159cb0 con 0x7fe40c0c9f40 2024-04-25T10:14:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.163+0000 7fe4119b1640 1 --2- 172.21.15.16:0/3720949601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe40c155be0 0x7fe40c0cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.163+0000 7fe4111b0640 1 --2- 172.21.15.16:0/3720949601 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40c155fc0 0x7fe40c0cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.163+0000 7fe4121b2640 1 --2- 172.21.15.16:0/3720949601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe40c0c9f40 0x7fe40c0ca3f0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fe404033040 tx=0x7fe404009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.163+0000 7fe4027fc640 1 -- 172.21.15.16:0/3720949601 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe408009690 con 0x7fe40c0c9f40 2024-04-25T10:14:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.163+0000 7fe4027fc640 1 -- 172.21.15.16:0/3720949601 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe404003990 con 0x7fe40c0c9f40 2024-04-25T10:14:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.163+0000 7fe413c15640 1 -- 172.21.15.16:0/3720949601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40c0b4590 con 0x7fe40c0c9f40 2024-04-25T10:14:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.163+0000 7fe4027fc640 1 -- 172.21.15.16:0/3720949601 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe404009c10 con 0x7fe40c0c9f40 2024-04-25T10:14:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.163+0000 7fe413c15640 1 -- 172.21.15.16:0/3720949601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe40c0b4830 con 0x7fe40c0c9f40 2024-04-25T10:14:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.164+0000 7fe413c15640 1 -- 172.21.15.16:0/3720949601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40c0cf2e0 con 0x7fe40c0c9f40 2024-04-25T10:14:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.165+0000 7fe4027fc640 1 -- 172.21.15.16:0/3720949601 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe404005d10 con 0x7fe40c0c9f40 2024-04-25T10:14:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.165+0000 7fe4027fc640 1 --2- 172.21.15.16:0/3720949601 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe3f006b500 0x7fe3f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.165+0000 7fe4027fc640 1 -- 172.21.15.16:0/3720949601 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5443+0+0 (secure 0 0 0) 0x7fe4040ab3a0 con 0x7fe40c0c9f40 2024-04-25T10:14:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.165+0000 7fe4119b1640 1 --2- 172.21.15.16:0/3720949601 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe3f006b500 0x7fe3f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.166+0000 7fe4119b1640 1 --2- 172.21.15.16:0/3720949601 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe3f006b500 0x7fe3f006d9c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe408009be0 tx=0x7fe408008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.169+0000 7fe4027fc640 1 -- 172.21.15.16:0/3720949601 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe40c0cf2e0 con 0x7fe40c0c9f40 2024-04-25T10:14:17.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:17.356+0000 7fe413c15640 1 -- 172.21.15.16:0/3720949601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe40c06f300 con 0x7fe40c0c9f40 2024-04-25T10:14:18.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.031+0000 7fe4027fc640 1 -- 172.21.15.16:0/3720949601 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe40c06f300 con 0x7fe40c0c9f40 2024-04-25T10:14:18.037 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-04-25T10:14:18.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.038+0000 7fe413c15640 1 -- 172.21.15.16:0/3720949601 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe3f006b500 msgr2=0x7fe3f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.038+0000 7fe413c15640 1 --2- 172.21.15.16:0/3720949601 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe3f006b500 0x7fe3f006d9c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe408009be0 tx=0x7fe408008040 comp rx=0 tx=0).stop 2024-04-25T10:14:18.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.038+0000 7fe413c15640 1 -- 172.21.15.16:0/3720949601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe40c0c9f40 msgr2=0x7fe40c0ca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.038+0000 7fe413c15640 1 --2- 172.21.15.16:0/3720949601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe40c0c9f40 0x7fe40c0ca3f0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fe404033040 tx=0x7fe404009750 comp rx=0 tx=0).stop 2024-04-25T10:14:18.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.038+0000 7fe413c15640 1 -- 172.21.15.16:0/3720949601 shutdown_connections 2024-04-25T10:14:18.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.038+0000 7fe413c15640 1 --2- 172.21.15.16:0/3720949601 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe3f006b500 0x7fe3f006d9c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.038+0000 7fe413c15640 1 --2- 172.21.15.16:0/3720949601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe40c0c9f40 0x7fe40c0ca3f0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.038+0000 7fe413c15640 1 --2- 172.21.15.16:0/3720949601 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe40c155fc0 0x7fe40c0cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.038+0000 7fe413c15640 1 --2- 172.21.15.16:0/3720949601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe40c155be0 0x7fe40c0cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.038+0000 7fe413c15640 1 -- 172.21.15.16:0/3720949601 >> 172.21.15.16:0/3720949601 conn(0x7fe40c099a40 msgr2=0x7fe40c06e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.039+0000 7fe413c15640 1 -- 172.21.15.16:0/3720949601 shutdown_connections 2024-04-25T10:14:18.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.039+0000 7fe413c15640 1 -- 172.21.15.16:0/3720949601 wait complete. 2024-04-25T10:14:18.060 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:14:18.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.168+0000 7fab9598b640 1 Processor -- start 2024-04-25T10:14:18.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.168+0000 7fab9598b640 1 -- start start 2024-04-25T10:14:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.169+0000 7fab9598b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab900d3b90 0x7fab900d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.169+0000 7fab9598b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab900d4090 con 0x7fab900d0f60 2024-04-25T10:14:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.169+0000 7fab9598b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab900d4370 con 0x7fab9015dca0 2024-04-25T10:14:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.169+0000 7fab9598b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab900b6b20 con 0x7fab900d3b90 2024-04-25T10:14:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.169+0000 7fab8f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fab900d0f60 0x7fab900b2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47726/0 (socket says 172.21.15.16:47726) 2024-04-25T10:14:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.169+0000 7fab8f7fe640 1 -- 172.21.15.16:0/2775837260 learned_addr learned my addr 172.21.15.16:0/2775837260 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.169+0000 7fab8e7fc640 1 --2- 172.21.15.16:0/2775837260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab900d3b90 0x7fab900d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.169+0000 7fab8dffb640 1 -- 172.21.15.16:0/2775837260 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3550435191 0 0) 0x7fab900d4090 con 0x7fab900d0f60 2024-04-25T10:14:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.169+0000 7fab8dffb640 1 -- 172.21.15.16:0/2775837260 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab78003520 con 0x7fab900d0f60 2024-04-25T10:14:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.169+0000 7fab8dffb640 1 -- 172.21.15.16:0/2775837260 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039129026 0 0) 0x7fab900d4370 con 0x7fab9015dca0 2024-04-25T10:14:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.169+0000 7fab8dffb640 1 -- 172.21.15.16:0/2775837260 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab900d4090 con 0x7fab9015dca0 2024-04-25T10:14:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.169+0000 7fab8dffb640 1 -- 172.21.15.16:0/2775837260 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1432527970 0 0) 0x7fab78003520 con 0x7fab900d0f60 2024-04-25T10:14:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.169+0000 7fab8dffb640 1 -- 172.21.15.16:0/2775837260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab900d3b90 msgr2=0x7fab900d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.169+0000 7fab8dffb640 1 --2- 172.21.15.16:0/2775837260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab900d3b90 0x7fab900d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.169+0000 7fab8dffb640 1 -- 172.21.15.16:0/2775837260 >> v1:172.21.15.186:6789/0 conn(0x7fab9015dca0 legacy=0x7fab900ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.169+0000 7fab8dffb640 1 -- 172.21.15.16:0/2775837260 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab900d4650 con 0x7fab900d0f60 2024-04-25T10:14:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.170+0000 7fab8dffb640 1 -- 172.21.15.16:0/2775837260 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fab88001670 con 0x7fab900d0f60 2024-04-25T10:14:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.170+0000 7fab8dffb640 1 -- 172.21.15.16:0/2775837260 >> v1:172.21.15.16:6789/0 conn(0x7fab900d0f60 legacy=0x7fab900b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.170+0000 7fab8dffb640 1 --2- 172.21.15.16:0/2775837260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab78003b70 0x7fab78003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.170+0000 7fab8dffb640 1 --2- 172.21.15.16:0/2775837260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab900d3b90 0x7fab78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.170+0000 7fab8dffb640 1 --2- 172.21.15.16:0/2775837260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab78004d40 0x7fab780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.170+0000 7fab8dffb640 1 -- 172.21.15.16:0/2775837260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7800ba70 con 0x7fab78004d40 2024-04-25T10:14:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.170+0000 7fab8dffb640 1 -- 172.21.15.16:0/2775837260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7800bc20 con 0x7fab78003b70 2024-04-25T10:14:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.170+0000 7fab8dffb640 1 -- 172.21.15.16:0/2775837260 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7800be20 con 0x7fab900d3b90 2024-04-25T10:14:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.170+0000 7fab8f7fe640 1 --2- 172.21.15.16:0/2775837260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab78004d40 0x7fab780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.170+0000 7fab8effd640 1 --2- 172.21.15.16:0/2775837260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab78003b70 0x7fab78003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.170+0000 7fab8e7fc640 1 --2- 172.21.15.16:0/2775837260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab900d3b90 0x7fab78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.170+0000 7fab8f7fe640 1 -- 172.21.15.16:0/2775837260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab900d3b90 msgr2=0x7fab78004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.170+0000 7fab8f7fe640 1 --2- 172.21.15.16:0/2775837260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab900d3b90 0x7fab78004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.170+0000 7fab8f7fe640 1 -- 172.21.15.16:0/2775837260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab78003b70 msgr2=0x7fab78003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.170+0000 7fab8f7fe640 1 --2- 172.21.15.16:0/2775837260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab78003b70 0x7fab78003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.170+0000 7fab8f7fe640 1 -- 172.21.15.16:0/2775837260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab78003520 con 0x7fab78004d40 2024-04-25T10:14:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.171+0000 7fab8f7fe640 1 --2- 172.21.15.16:0/2775837260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab78004d40 0x7fab780051c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fab88007fa0 tx=0x7fab78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894c57a440ec99c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.171+0000 7fab8dffb640 1 -- 172.21.15.16:0/2775837260 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88007bc0 con 0x7fab78004d40 2024-04-25T10:14:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.171+0000 7fab8dffb640 1 -- 172.21.15.16:0/2775837260 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab88005af0 con 0x7fab78004d40 2024-04-25T10:14:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.171+0000 7fab8dffb640 1 -- 172.21.15.16:0/2775837260 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88007070 con 0x7fab78004d40 2024-04-25T10:14:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.171+0000 7fab9598b640 1 -- 172.21.15.16:0/2775837260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab78004d40 msgr2=0x7fab780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.171+0000 7fab9598b640 1 --2- 172.21.15.16:0/2775837260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab78004d40 0x7fab780051c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fab88007fa0 tx=0x7fab78005830 comp rx=0 tx=0).stop 2024-04-25T10:14:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.171+0000 7fab8f7fe640 1 -- 172.21.15.16:0/2775837260 reap_dead start 2024-04-25T10:14:18.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.171+0000 7fab9598b640 1 -- 172.21.15.16:0/2775837260 shutdown_connections 2024-04-25T10:14:18.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.171+0000 7fab9598b640 1 -- 172.21.15.16:0/2775837260 >> 172.21.15.16:0/2775837260 conn(0x7fab900726c0 msgr2=0x7fab90072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.172+0000 7fab9598b640 1 -- 172.21.15.16:0/2775837260 shutdown_connections 2024-04-25T10:14:18.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.172+0000 7fab9598b640 1 -- 172.21.15.16:0/2775837260 wait complete. 2024-04-25T10:14:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.173+0000 7fab9598b640 1 Processor -- start 2024-04-25T10:14:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.173+0000 7fab9598b640 1 -- start start 2024-04-25T10:14:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab9598b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab901f7030 0x7fab900aa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab9598b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab9014f800 0x7fab901f7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab9598b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab901f7950 0x7fab9014cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab9598b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9015e080 con 0x7fab901f7950 2024-04-25T10:14:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab9598b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab900cf050 con 0x7fab901f7030 2024-04-25T10:14:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab9598b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9015e200 con 0x7fab9014f800 2024-04-25T10:14:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab8f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab901f7950 0x7fab9014cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab8f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab901f7950 0x7fab9014cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47370/0 (socket says 172.21.15.16:47370) 2024-04-25T10:14:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab8f7fe640 1 -- 172.21.15.16:0/3633687456 learned_addr learned my addr 172.21.15.16:0/3633687456 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab8e7fc640 1 --2- 172.21.15.16:0/3633687456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab9014f800 0x7fab901f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab8effd640 1 --2- 172.21.15.16:0/3633687456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab901f7030 0x7fab900aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab8f7fe640 1 -- 172.21.15.16:0/3633687456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab9014f800 msgr2=0x7fab901f7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab8f7fe640 1 --2- 172.21.15.16:0/3633687456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab9014f800 0x7fab901f7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab8f7fe640 1 -- 172.21.15.16:0/3633687456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab901f7030 msgr2=0x7fab900aa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab8f7fe640 1 --2- 172.21.15.16:0/3633687456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab901f7030 0x7fab900aa480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab8f7fe640 1 -- 172.21.15.16:0/3633687456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab900cbf10 con 0x7fab901f7950 2024-04-25T10:14:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab8e7fc640 1 --2- 172.21.15.16:0/3633687456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab9014f800 0x7fab901f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab8effd640 1 --2- 172.21.15.16:0/3633687456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab901f7030 0x7fab900aa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab8f7fe640 1 --2- 172.21.15.16:0/3633687456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab901f7950 0x7fab9014cbd0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fab8802abd0 tx=0x7fab900d4260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab77fff640 1 -- 172.21.15.16:0/3633687456 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88043070 con 0x7fab901f7950 2024-04-25T10:14:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab77fff640 1 -- 172.21.15.16:0/3633687456 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab88038040 con 0x7fab901f7950 2024-04-25T10:14:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab77fff640 1 -- 172.21.15.16:0/3633687456 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88007bc0 con 0x7fab901f7950 2024-04-25T10:14:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.174+0000 7fab9598b640 1 -- 172.21.15.16:0/3633687456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab900cc1f0 con 0x7fab901f7950 2024-04-25T10:14:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.175+0000 7fab9598b640 1 -- 172.21.15.16:0/3633687456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab900cc7b0 con 0x7fab901f7950 2024-04-25T10:14:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.175+0000 7fab9598b640 1 -- 172.21.15.16:0/3633687456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab900750b0 con 0x7fab901f7950 2024-04-25T10:14:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.176+0000 7fab77fff640 1 -- 172.21.15.16:0/3633687456 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab88046070 con 0x7fab901f7950 2024-04-25T10:14:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.176+0000 7fab77fff640 1 --2- 172.21.15.16:0/3633687456 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fab7c06b500 0x7fab7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.176+0000 7fab77fff640 1 -- 172.21.15.16:0/3633687456 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fab880a9a30 con 0x7fab901f7950 2024-04-25T10:14:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.177+0000 7fab8effd640 1 --2- 172.21.15.16:0/3633687456 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fab7c06b500 0x7fab7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.177+0000 7fab8effd640 1 --2- 172.21.15.16:0/3633687456 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fab7c06b500 0x7fab7c06d9c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fab80003f90 tx=0x7fab80009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.181+0000 7fab77fff640 1 -- 172.21.15.16:0/3633687456 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab88078460 con 0x7fab901f7950 2024-04-25T10:14:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.360+0000 7fab9598b640 1 -- 172.21.15.16:0/3633687456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fab9015d2f0 con 0x7fab901f7950 2024-04-25T10:14:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.363+0000 7fab77fff640 1 -- 172.21.15.16:0/3633687456 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fab8807c500 con 0x7fab901f7950 2024-04-25T10:14:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.365+0000 7fab9598b640 1 -- 172.21.15.16:0/3633687456 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fab7c06b500 msgr2=0x7fab7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.365+0000 7fab9598b640 1 --2- 172.21.15.16:0/3633687456 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fab7c06b500 0x7fab7c06d9c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fab80003f90 tx=0x7fab80009040 comp rx=0 tx=0).stop 2024-04-25T10:14:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.365+0000 7fab9598b640 1 -- 172.21.15.16:0/3633687456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab901f7950 msgr2=0x7fab9014cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.365+0000 7fab9598b640 1 --2- 172.21.15.16:0/3633687456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab901f7950 0x7fab9014cbd0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fab8802abd0 tx=0x7fab900d4260 comp rx=0 tx=0).stop 2024-04-25T10:14:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.365+0000 7fab9598b640 1 -- 172.21.15.16:0/3633687456 shutdown_connections 2024-04-25T10:14:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.365+0000 7fab9598b640 1 --2- 172.21.15.16:0/3633687456 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fab7c06b500 0x7fab7c06d9c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.365+0000 7fab9598b640 1 --2- 172.21.15.16:0/3633687456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab901f7950 0x7fab9014cbd0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.365+0000 7fab9598b640 1 --2- 172.21.15.16:0/3633687456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab9014f800 0x7fab901f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.365+0000 7fab9598b640 1 --2- 172.21.15.16:0/3633687456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab901f7030 0x7fab900aa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.365+0000 7fab9598b640 1 -- 172.21.15.16:0/3633687456 >> 172.21.15.16:0/3633687456 conn(0x7fab900726c0 msgr2=0x7fab9006c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.365+0000 7fab9598b640 1 -- 172.21.15.16:0/3633687456 shutdown_connections 2024-04-25T10:14:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.365+0000 7fab9598b640 1 -- 172.21.15.16:0/3633687456 wait complete. 2024-04-25T10:14:18.381 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-25T10:14:18.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.477+0000 7f57ec74f640 1 Processor -- start 2024-04-25T10:14:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.477+0000 7f57ec74f640 1 -- start start 2024-04-25T10:14:18.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.478+0000 7f57ec74f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f57e406bef0 0x7f57e406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.478+0000 7f57ec74f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e406dc70 con 0x7f57e40acd80 2024-04-25T10:14:18.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.478+0000 7f57ec74f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e406df10 con 0x7f57e406c2d0 2024-04-25T10:14:18.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.478+0000 7f57ec74f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e406eb20 con 0x7f57e406bef0 2024-04-25T10:14:18.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.478+0000 7f57ea4eb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f57e406bef0 0x7f57e406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.478+0000 7f57eacec640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f57e40acd80 0x7f57e406aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47732/0 (socket says 172.21.15.16:47732) 2024-04-25T10:14:18.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.478+0000 7f57eacec640 1 -- 172.21.15.16:0/1115115254 learned_addr learned my addr 172.21.15.16:0/1115115254 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:18.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.479+0000 7f57ea4eb640 1 -- 172.21.15.16:0/1115115254 >> v1:172.21.15.186:6789/0 conn(0x7f57e406c2d0 legacy=0x7f57e415c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.479+0000 7f57ea4eb640 1 -- 172.21.15.16:0/1115115254 >> v1:172.21.15.16:6789/0 conn(0x7f57e40acd80 legacy=0x7f57e406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.479+0000 7f57e94e9640 1 -- 172.21.15.16:0/1115115254 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1767321691 0 0) 0x7f57e406dc70 con 0x7f57e40acd80 2024-04-25T10:14:18.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.479+0000 7f57ea4eb640 1 -- 172.21.15.16:0/1115115254 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e406e1f0 con 0x7f57e406bef0 2024-04-25T10:14:18.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.479+0000 7f57ea4eb640 1 --2- 172.21.15.16:0/1115115254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e406bef0 0x7f57e406efb0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0097a0 tx=0x7f57dc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c217bfb44398c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.480+0000 7f57e94e9640 1 -- 172.21.15.16:0/1115115254 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc037070 con 0x7f57e406bef0 2024-04-25T10:14:18.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.480+0000 7f57e94e9640 1 -- 172.21.15.16:0/1115115254 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc032440 con 0x7f57e406bef0 2024-04-25T10:14:18.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.480+0000 7f57e94e9640 1 -- 172.21.15.16:0/1115115254 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc032810 con 0x7f57e406bef0 2024-04-25T10:14:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.480+0000 7f57ec74f640 1 -- 172.21.15.16:0/1115115254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e406bef0 msgr2=0x7f57e406efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.480+0000 7f57ec74f640 1 --2- 172.21.15.16:0/1115115254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e406bef0 0x7f57e406efb0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0097a0 tx=0x7f57dc02a230 comp rx=0 tx=0).stop 2024-04-25T10:14:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.480+0000 7f57ec74f640 1 -- 172.21.15.16:0/1115115254 shutdown_connections 2024-04-25T10:14:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.480+0000 7f57ec74f640 1 --2- 172.21.15.16:0/1115115254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e406bef0 0x7f57e406efb0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.480+0000 7f57ec74f640 1 -- 172.21.15.16:0/1115115254 >> 172.21.15.16:0/1115115254 conn(0x7f57e40b2220 msgr2=0x7f57e40cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.480+0000 7f57ec74f640 1 -- 172.21.15.16:0/1115115254 shutdown_connections 2024-04-25T10:14:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.480+0000 7f57ec74f640 1 -- 172.21.15.16:0/1115115254 wait complete. 2024-04-25T10:14:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.481+0000 7f57ec74f640 1 Processor -- start 2024-04-25T10:14:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.482+0000 7f57ec74f640 1 -- start start 2024-04-25T10:14:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.482+0000 7f57ec74f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57e406bef0 0x7f57e4146f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.482+0000 7f57ec74f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e406c2d0 0x7f57e415c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.482+0000 7f57ec74f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57e40acd80 0x7f57e41495a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.482+0000 7f57ec74f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e40ada00 con 0x7f57e406c2d0 2024-04-25T10:14:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.482+0000 7f57ec74f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e406e7b0 con 0x7f57e406bef0 2024-04-25T10:14:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.482+0000 7f57ec74f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e40adb80 con 0x7f57e40acd80 2024-04-25T10:14:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.482+0000 7f57e9cea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e406c2d0 0x7f57e415c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.482+0000 7f57e9cea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e406c2d0 0x7f57e415c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47398/0 (socket says 172.21.15.16:47398) 2024-04-25T10:14:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.482+0000 7f57e9cea640 1 -- 172.21.15.16:0/157079760 learned_addr learned my addr 172.21.15.16:0/157079760 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.483+0000 7f57ea4eb640 1 --2- 172.21.15.16:0/157079760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57e406bef0 0x7f57e4146f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.483+0000 7f57eacec640 1 --2- 172.21.15.16:0/157079760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57e40acd80 0x7f57e41495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.483+0000 7f57e9cea640 1 -- 172.21.15.16:0/157079760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57e40acd80 msgr2=0x7f57e41495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.483+0000 7f57e9cea640 1 --2- 172.21.15.16:0/157079760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57e40acd80 0x7f57e41495a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.483+0000 7f57e9cea640 1 -- 172.21.15.16:0/157079760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57e406bef0 msgr2=0x7f57e4146f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.483+0000 7f57e9cea640 1 --2- 172.21.15.16:0/157079760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57e406bef0 0x7f57e4146f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.483+0000 7f57e9cea640 1 -- 172.21.15.16:0/157079760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e40b4c70 con 0x7f57e406c2d0 2024-04-25T10:14:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.483+0000 7f57eacec640 1 --2- 172.21.15.16:0/157079760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57e40acd80 0x7f57e41495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.483+0000 7f57e9cea640 1 --2- 172.21.15.16:0/157079760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e406c2d0 0x7f57e415c3c0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f57d800c430 tx=0x7f57d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.483+0000 7f57e2ffd640 1 -- 172.21.15.16:0/157079760 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8017070 con 0x7f57e406c2d0 2024-04-25T10:14:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.483+0000 7f57e2ffd640 1 -- 172.21.15.16:0/157079760 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d80052d0 con 0x7f57e406c2d0 2024-04-25T10:14:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.483+0000 7f57ec74f640 1 -- 172.21.15.16:0/157079760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e41571b0 con 0x7f57e406c2d0 2024-04-25T10:14:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.483+0000 7f57e2ffd640 1 -- 172.21.15.16:0/157079760 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8003830 con 0x7f57e406c2d0 2024-04-25T10:14:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.483+0000 7f57ec74f640 1 -- 172.21.15.16:0/157079760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e4157770 con 0x7f57e406c2d0 2024-04-25T10:14:18.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.484+0000 7f57ec74f640 1 -- 172.21.15.16:0/157079760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e4157970 con 0x7f57e406c2d0 2024-04-25T10:14:18.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.485+0000 7f57e2ffd640 1 -- 172.21.15.16:0/157079760 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57d801a070 con 0x7f57e406c2d0 2024-04-25T10:14:18.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.485+0000 7f57e2ffd640 1 --2- 172.21.15.16:0/157079760 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f57d006b4d0 0x7f57d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.485+0000 7f57e2ffd640 1 -- 172.21.15.16:0/157079760 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f57d80892a0 con 0x7f57e406c2d0 2024-04-25T10:14:18.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.486+0000 7f57ea4eb640 1 --2- 172.21.15.16:0/157079760 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f57d006b4d0 0x7f57d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.486+0000 7f57ea4eb640 1 --2- 172.21.15.16:0/157079760 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f57d006b4d0 0x7f57d006d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f57dc034000 tx=0x7f57dc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.490+0000 7f57e2ffd640 1 -- 172.21.15.16:0/157079760 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57e4157970 con 0x7f57e406c2d0 2024-04-25T10:14:18.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:18.742+0000 7f57ec74f640 1 -- 172.21.15.16:0/157079760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7f57e40d0b70 con 0x7f57e406c2d0 2024-04-25T10:14:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.033+0000 7f57e2ffd640 1 -- 172.21.15.16:0/157079760 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v434) v1 ==== 146+0+0 (secure 0 0 0) 0x7f57d805e0c0 con 0x7f57e406c2d0 2024-04-25T10:14:19.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.035+0000 7f57ec74f640 1 -- 172.21.15.16:0/157079760 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f57d006b4d0 msgr2=0x7f57d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.035+0000 7f57ec74f640 1 --2- 172.21.15.16:0/157079760 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f57d006b4d0 0x7f57d006d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f57dc034000 tx=0x7f57dc02e000 comp rx=0 tx=0).stop 2024-04-25T10:14:19.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.035+0000 7f57ec74f640 1 -- 172.21.15.16:0/157079760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e406c2d0 msgr2=0x7f57e415c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.035+0000 7f57ec74f640 1 --2- 172.21.15.16:0/157079760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e406c2d0 0x7f57e415c3c0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f57d800c430 tx=0x7f57d800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.035+0000 7f57ec74f640 1 -- 172.21.15.16:0/157079760 shutdown_connections 2024-04-25T10:14:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.035+0000 7f57ec74f640 1 --2- 172.21.15.16:0/157079760 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f57d006b4d0 0x7f57d006d990 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.035+0000 7f57ec74f640 1 --2- 172.21.15.16:0/157079760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57e40acd80 0x7f57e41495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.035+0000 7f57ec74f640 1 --2- 172.21.15.16:0/157079760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e406c2d0 0x7f57e415c3c0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.035+0000 7f57ec74f640 1 --2- 172.21.15.16:0/157079760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57e406bef0 0x7f57e4146f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.035+0000 7f57ec74f640 1 -- 172.21.15.16:0/157079760 >> 172.21.15.16:0/157079760 conn(0x7f57e40b2220 msgr2=0x7f57e4154b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.035+0000 7f57ec74f640 1 -- 172.21.15.16:0/157079760 shutdown_connections 2024-04-25T10:14:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.035+0000 7f57ec74f640 1 -- 172.21.15.16:0/157079760 wait complete. 2024-04-25T10:14:19.052 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T10:14:19.052 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:14:19.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.150+0000 7f5854f9d640 1 Processor -- start 2024-04-25T10:14:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.150+0000 7f5854f9d640 1 -- start start 2024-04-25T10:14:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.150+0000 7f5854f9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58500d35b0 0x7f58500b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.150+0000 7f5854f9d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585015e0c0 con 0x7f58500b5990 2024-04-25T10:14:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.150+0000 7f5854f9d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58500d2510 con 0x7f585015dce0 2024-04-25T10:14:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.150+0000 7f5854f9d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58500d27f0 con 0x7f58500d35b0 2024-04-25T10:14:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.151+0000 7f5847fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58500d35b0 0x7f58500b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.151+0000 7f584ed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f58500b5990 0x7f58500b6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47738/0 (socket says 172.21.15.16:47738) 2024-04-25T10:14:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.151+0000 7f584ed9d640 1 -- 172.21.15.16:0/1746345426 learned_addr learned my addr 172.21.15.16:0/1746345426 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.151+0000 7f5847fff640 1 -- 172.21.15.16:0/1746345426 >> v1:172.21.15.186:6789/0 conn(0x7f585015dce0 legacy=0x7f58500d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.151+0000 7f5847fff640 1 -- 172.21.15.16:0/1746345426 >> v1:172.21.15.16:6789/0 conn(0x7f58500b5990 legacy=0x7f58500b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.151+0000 7f5847fff640 1 -- 172.21.15.16:0/1746345426 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58500d2ad0 con 0x7f58500d35b0 2024-04-25T10:14:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.151+0000 7f584dd9b640 1 -- 172.21.15.16:0/1746345426 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1121713559 0 0) 0x7f585015e0c0 con 0x7f58500b5990 2024-04-25T10:14:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.152+0000 7f5847fff640 1 --2- 172.21.15.16:0/1746345426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58500d35b0 0x7f58500b5450 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f58380097a0 tx=0x7f583802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a5fdd2a8e1e36e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.152+0000 7f584dd9b640 1 -- 172.21.15.16:0/1746345426 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838037070 con 0x7f58500d35b0 2024-04-25T10:14:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.152+0000 7f584dd9b640 1 -- 172.21.15.16:0/1746345426 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5838032440 con 0x7f58500d35b0 2024-04-25T10:14:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.152+0000 7f584dd9b640 1 -- 172.21.15.16:0/1746345426 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838032810 con 0x7f58500d35b0 2024-04-25T10:14:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.152+0000 7f5854f9d640 1 -- 172.21.15.16:0/1746345426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58500d35b0 msgr2=0x7f58500b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.152+0000 7f5854f9d640 1 --2- 172.21.15.16:0/1746345426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58500d35b0 0x7f58500b5450 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f58380097a0 tx=0x7f583802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.152+0000 7f5854f9d640 1 -- 172.21.15.16:0/1746345426 shutdown_connections 2024-04-25T10:14:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.152+0000 7f5854f9d640 1 --2- 172.21.15.16:0/1746345426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58500d35b0 0x7f58500b5450 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.152+0000 7f5854f9d640 1 -- 172.21.15.16:0/1746345426 >> 172.21.15.16:0/1746345426 conn(0x7f5850072900 msgr2=0x7f58500d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.152+0000 7f5854f9d640 1 -- 172.21.15.16:0/1746345426 shutdown_connections 2024-04-25T10:14:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.152+0000 7f5854f9d640 1 -- 172.21.15.16:0/1746345426 wait complete. 2024-04-25T10:14:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.154+0000 7f5854f9d640 1 Processor -- start 2024-04-25T10:14:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.154+0000 7f5854f9d640 1 -- start start 2024-04-25T10:14:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.154+0000 7f5854f9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58500b5990 0x7f58500ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.154+0000 7f5854f9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58500d35b0 0x7f585014c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.154+0000 7f5854f9d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f585015dce0 0x7f58500ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.154+0000 7f5854f9d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58500d2160 con 0x7f58500b5990 2024-04-25T10:14:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.154+0000 7f5854f9d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58500d3090 con 0x7f585015dce0 2024-04-25T10:14:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.154+0000 7f5854f9d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585015c110 con 0x7f58500d35b0 2024-04-25T10:14:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.155+0000 7f584e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58500b5990 0x7f58500ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.155+0000 7f584e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58500b5990 0x7f58500ad2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47414/0 (socket says 172.21.15.16:47414) 2024-04-25T10:14:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.155+0000 7f584e59c640 1 -- 172.21.15.16:0/2168392821 learned_addr learned my addr 172.21.15.16:0/2168392821 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.155+0000 7f584ed9d640 1 --2- 172.21.15.16:0/2168392821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f585015dce0 0x7f58500ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.155+0000 7f5847fff640 1 --2- 172.21.15.16:0/2168392821 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58500d35b0 0x7f585014c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.155+0000 7f584e59c640 1 -- 172.21.15.16:0/2168392821 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58500d35b0 msgr2=0x7f585014c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.155+0000 7f584e59c640 1 --2- 172.21.15.16:0/2168392821 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58500d35b0 0x7f585014c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.155+0000 7f584e59c640 1 -- 172.21.15.16:0/2168392821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f585015dce0 msgr2=0x7f58500ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.155+0000 7f584e59c640 1 --2- 172.21.15.16:0/2168392821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f585015dce0 0x7f58500ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.155+0000 7f584e59c640 1 -- 172.21.15.16:0/2168392821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58500aafc0 con 0x7f58500b5990 2024-04-25T10:14:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.155+0000 7f584e59c640 1 --2- 172.21.15.16:0/2168392821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58500b5990 0x7f58500ad2c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f584000b420 tx=0x7f584000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.155+0000 7f5846ffd640 1 -- 172.21.15.16:0/2168392821 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840011830 con 0x7f58500b5990 2024-04-25T10:14:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.155+0000 7f5846ffd640 1 -- 172.21.15.16:0/2168392821 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5840016070 con 0x7f58500b5990 2024-04-25T10:14:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.155+0000 7f5854f9d640 1 -- 172.21.15.16:0/2168392821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58500ab2a0 con 0x7f58500b5990 2024-04-25T10:14:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.155+0000 7f5846ffd640 1 -- 172.21.15.16:0/2168392821 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840011c10 con 0x7f58500b5990 2024-04-25T10:14:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.155+0000 7f5854f9d640 1 -- 172.21.15.16:0/2168392821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58500ab860 con 0x7f58500b5990 2024-04-25T10:14:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.156+0000 7f5854f9d640 1 -- 172.21.15.16:0/2168392821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58500749b0 con 0x7f58500b5990 2024-04-25T10:14:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.157+0000 7f5846ffd640 1 -- 172.21.15.16:0/2168392821 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f584001b460 con 0x7f58500b5990 2024-04-25T10:14:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.157+0000 7f5846ffd640 1 --2- 172.21.15.16:0/2168392821 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f583006b4d0 0x7f583006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.157+0000 7f5846ffd640 1 -- 172.21.15.16:0/2168392821 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f5840089aa0 con 0x7f58500b5990 2024-04-25T10:14:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.157+0000 7f5847fff640 1 --2- 172.21.15.16:0/2168392821 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f583006b4d0 0x7f583006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.158+0000 7f5847fff640 1 --2- 172.21.15.16:0/2168392821 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f583006b4d0 0x7f583006d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5838034000 tx=0x7f583802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.161+0000 7f5846ffd640 1 -- 172.21.15.16:0/2168392821 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5840020020 con 0x7f58500b5990 2024-04-25T10:14:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.343+0000 7f5854f9d640 1 -- 172.21.15.16:0/2168392821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58500b3ba0 con 0x7f58500b5990 2024-04-25T10:14:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.343+0000 7f5846ffd640 1 -- 172.21.15.16:0/2168392821 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+13160 (secure 0 0 0) 0x7f5840058b30 con 0x7f58500b5990 2024-04-25T10:14:19.349 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:19.349 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":106,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:14:18.029121+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.186:6836/2008057690":"2024-04-26T10:14:10.849128+0000","172.21.15.186:6834/2008057690":"2024-04-26T10:14:10.849128+0000","172.21.15.186:6836/1371188714":"2024-04-26T10:13:34.499700+0000","172.21.15.186:6834/1371188714":"2024-04-26T10:13:34.499700+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.346+0000 7f5854f9d640 1 -- 172.21.15.16:0/2168392821 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f583006b4d0 msgr2=0x7f583006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.346+0000 7f5854f9d640 1 --2- 172.21.15.16:0/2168392821 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f583006b4d0 0x7f583006d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5838034000 tx=0x7f583802e000 comp rx=0 tx=0).stop 2024-04-25T10:14:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.346+0000 7f5854f9d640 1 -- 172.21.15.16:0/2168392821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58500b5990 msgr2=0x7f58500ad2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.346+0000 7f5854f9d640 1 --2- 172.21.15.16:0/2168392821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58500b5990 0x7f58500ad2c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f584000b420 tx=0x7f584000b8e0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.346+0000 7f5854f9d640 1 -- 172.21.15.16:0/2168392821 shutdown_connections 2024-04-25T10:14:19.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.346+0000 7f5854f9d640 1 --2- 172.21.15.16:0/2168392821 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f583006b4d0 0x7f583006d990 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.346+0000 7f5854f9d640 1 --2- 172.21.15.16:0/2168392821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f585015dce0 0x7f58500ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.346+0000 7f5854f9d640 1 --2- 172.21.15.16:0/2168392821 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58500d35b0 0x7f585014c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.346+0000 7f5854f9d640 1 --2- 172.21.15.16:0/2168392821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58500b5990 0x7f58500ad2c0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.346+0000 7f5854f9d640 1 -- 172.21.15.16:0/2168392821 >> 172.21.15.16:0/2168392821 conn(0x7f5850072900 msgr2=0x7f58500ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:19.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.346+0000 7f5854f9d640 1 -- 172.21.15.16:0/2168392821 shutdown_connections 2024-04-25T10:14:19.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.346+0000 7f5854f9d640 1 -- 172.21.15.16:0/2168392821 wait complete. 2024-04-25T10:14:19.362 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:19.363 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.461+0000 7f0153ddb640 1 Processor -- start 2024-04-25T10:14:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.461+0000 7f0153ddb640 1 -- start start 2024-04-25T10:14:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.462+0000 7f0153ddb640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f014c0d3b90 0x7f014c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.462+0000 7f0153ddb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f014c0d0f60 0x7f014c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.462+0000 7f0153ddb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f014c0d4090 con 0x7f014c15dca0 2024-04-25T10:14:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.462+0000 7f0153ddb640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f014c0b6b20 con 0x7f014c0d3b90 2024-04-25T10:14:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.462+0000 7f0153ddb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014c0d1390 con 0x7f014c0d0f60 2024-04-25T10:14:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.462+0000 7f0151b77640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f014c15dca0 0x7f014c0ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47748/0 (socket says 172.21.15.16:47748) 2024-04-25T10:14:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.462+0000 7f0151b77640 1 -- 172.21.15.16:0/3523614717 learned_addr learned my addr 172.21.15.16:0/3523614717 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.462+0000 7f0150b75640 1 -- 172.21.15.16:0/3523614717 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456679602 0 0) 0x7f014c0d4090 con 0x7f014c15dca0 2024-04-25T10:14:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0151376640 1 --2- 172.21.15.16:0/3523614717 >> v2:172.21.15.186:3300/0 conn(0x7f014c0d3b90 0x7f014c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0150b75640 1 -- 172.21.15.16:0/3523614717 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0138003520 con 0x7f014c15dca0 2024-04-25T10:14:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0152378640 1 --2- 172.21.15.16:0/3523614717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f014c0d0f60 0x7f014c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0150b75640 1 -- 172.21.15.16:0/3523614717 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 251445829 0 0) 0x7f0138003520 con 0x7f014c15dca0 2024-04-25T10:14:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0150b75640 1 -- 172.21.15.16:0/3523614717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f014c0d0f60 msgr2=0x7f014c0b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0150b75640 1 --2- 172.21.15.16:0/3523614717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f014c0d0f60 0x7f014c0b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0150b75640 1 -- 172.21.15.16:0/3523614717 >> v2:172.21.15.186:3300/0 conn(0x7f014c0d3b90 msgr2=0x7f014c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0150b75640 1 --2- 172.21.15.16:0/3523614717 >> v2:172.21.15.186:3300/0 conn(0x7f014c0d3b90 0x7f014c0d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0150b75640 1 -- 172.21.15.16:0/3523614717 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c0d4370 con 0x7f014c15dca0 2024-04-25T10:14:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0150b75640 1 -- 172.21.15.16:0/3523614717 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f0148001670 con 0x7f014c15dca0 2024-04-25T10:14:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0150b75640 1 -- 172.21.15.16:0/3523614717 >> v1:172.21.15.16:6789/0 conn(0x7f014c15dca0 legacy=0x7f014c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0150b75640 1 --2- 172.21.15.16:0/3523614717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f014c0d0f60 0x7f01380039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0150b75640 1 --2- 172.21.15.16:0/3523614717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0138003f60 0x7f0138004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0150b75640 1 --2- 172.21.15.16:0/3523614717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0138004d40 0x7f01380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0150b75640 1 -- 172.21.15.16:0/3523614717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c0d1390 con 0x7f0138003f60 2024-04-25T10:14:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0150b75640 1 -- 172.21.15.16:0/3523614717 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013800b9c0 con 0x7f0138004d40 2024-04-25T10:14:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0150b75640 1 -- 172.21.15.16:0/3523614717 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013800bb40 con 0x7f014c0d0f60 2024-04-25T10:14:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0151376640 1 --2- 172.21.15.16:0/3523614717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0138003f60 0x7f0138004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0151b77640 1 --2- 172.21.15.16:0/3523614717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f014c0d0f60 0x7f01380039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0152378640 1 --2- 172.21.15.16:0/3523614717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0138004d40 0x7f01380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0151376640 1 -- 172.21.15.16:0/3523614717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f014c0d0f60 msgr2=0x7f01380039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0151376640 1 --2- 172.21.15.16:0/3523614717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f014c0d0f60 0x7f01380039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0151376640 1 -- 172.21.15.16:0/3523614717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0138004d40 msgr2=0x7f01380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0151376640 1 --2- 172.21.15.16:0/3523614717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0138004d40 0x7f01380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.463+0000 7f0151376640 1 -- 172.21.15.16:0/3523614717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0138003520 con 0x7f0138003f60 2024-04-25T10:14:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.464+0000 7f0151376640 1 --2- 172.21.15.16:0/3523614717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0138003f60 0x7f0138004800 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f013c009a90 tx=0x7f0138005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee54ff53e33d90e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.464+0000 7f0150b75640 1 -- 172.21.15.16:0/3523614717 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c036070 con 0x7f0138003f60 2024-04-25T10:14:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.464+0000 7f0150b75640 1 -- 172.21.15.16:0/3523614717 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c031400 con 0x7f0138003f60 2024-04-25T10:14:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.464+0000 7f0150b75640 1 -- 172.21.15.16:0/3523614717 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c009690 con 0x7f0138003f60 2024-04-25T10:14:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.465+0000 7f0153ddb640 1 -- 172.21.15.16:0/3523614717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0138003f60 msgr2=0x7f0138004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.465+0000 7f0153ddb640 1 --2- 172.21.15.16:0/3523614717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0138003f60 0x7f0138004800 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f013c009a90 tx=0x7f0138005830 comp rx=0 tx=0).stop 2024-04-25T10:14:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.465+0000 7f0152378640 1 -- 172.21.15.16:0/3523614717 reap_dead start 2024-04-25T10:14:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.465+0000 7f0153ddb640 1 -- 172.21.15.16:0/3523614717 shutdown_connections 2024-04-25T10:14:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.465+0000 7f0153ddb640 1 -- 172.21.15.16:0/3523614717 >> 172.21.15.16:0/3523614717 conn(0x7f014c0726c0 msgr2=0x7f014c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.465+0000 7f0153ddb640 1 -- 172.21.15.16:0/3523614717 shutdown_connections 2024-04-25T10:14:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.465+0000 7f0153ddb640 1 -- 172.21.15.16:0/3523614717 wait complete. 2024-04-25T10:14:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.466+0000 7f0153ddb640 1 Processor -- start 2024-04-25T10:14:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.466+0000 7f0153ddb640 1 -- start start 2024-04-25T10:14:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f0153ddb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0138003f60 0x7f014c15e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f0153ddb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f014c15c090 0x7f014c15b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f0153ddb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f014c1503d0 0x7f014c154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f0153ddb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c0cf050 con 0x7f0138003f60 2024-04-25T10:14:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f0153ddb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c0b6d50 con 0x7f014c1503d0 2024-04-25T10:14:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f0153ddb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014c15e080 con 0x7f014c15c090 2024-04-25T10:14:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f0151b77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0138003f60 0x7f014c15e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f0151b77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0138003f60 0x7f014c15e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47432/0 (socket says 172.21.15.16:47432) 2024-04-25T10:14:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f0151b77640 1 -- 172.21.15.16:0/3092936236 learned_addr learned my addr 172.21.15.16:0/3092936236 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f0152378640 1 --2- 172.21.15.16:0/3092936236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f014c1503d0 0x7f014c154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f0151376640 1 --2- 172.21.15.16:0/3092936236 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f014c15c090 0x7f014c15b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f0151b77640 1 -- 172.21.15.16:0/3092936236 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f014c15c090 msgr2=0x7f014c15b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f0151b77640 1 --2- 172.21.15.16:0/3092936236 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f014c15c090 0x7f014c15b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f0151b77640 1 -- 172.21.15.16:0/3092936236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f014c1503d0 msgr2=0x7f014c154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f0151b77640 1 --2- 172.21.15.16:0/3092936236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f014c1503d0 0x7f014c154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f0151b77640 1 -- 172.21.15.16:0/3092936236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c15a2c0 con 0x7f0138003f60 2024-04-25T10:14:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f0151b77640 1 --2- 172.21.15.16:0/3092936236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0138003f60 0x7f014c15e3d0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f0148007310 tx=0x7f01480077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f01427fc640 1 -- 172.21.15.16:0/3092936236 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014800f9c0 con 0x7f0138003f60 2024-04-25T10:14:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.467+0000 7f01427fc640 1 -- 172.21.15.16:0/3092936236 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0148018070 con 0x7f0138003f60 2024-04-25T10:14:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.468+0000 7f0153ddb640 1 -- 172.21.15.16:0/3092936236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014c15a560 con 0x7f0138003f60 2024-04-25T10:14:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.468+0000 7f01427fc640 1 -- 172.21.15.16:0/3092936236 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014800fde0 con 0x7f0138003f60 2024-04-25T10:14:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.468+0000 7f0153ddb640 1 -- 172.21.15.16:0/3092936236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014c15ab20 con 0x7f0138003f60 2024-04-25T10:14:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.469+0000 7f0153ddb640 1 -- 172.21.15.16:0/3092936236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014c0d0a20 con 0x7f0138003f60 2024-04-25T10:14:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.469+0000 7f01427fc640 1 -- 172.21.15.16:0/3092936236 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f014801d460 con 0x7f0138003f60 2024-04-25T10:14:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.470+0000 7f01427fc640 1 --2- 172.21.15.16:0/3092936236 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f013006b500 0x7f013006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.470+0000 7f01427fc640 1 -- 172.21.15.16:0/3092936236 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f0148089f50 con 0x7f0138003f60 2024-04-25T10:14:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.470+0000 7f0151376640 1 --2- 172.21.15.16:0/3092936236 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f013006b500 0x7f013006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.470+0000 7f0151376640 1 --2- 172.21.15.16:0/3092936236 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f013006b500 0x7f013006d9c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f013c033000 tx=0x7f013c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.474+0000 7f01427fc640 1 -- 172.21.15.16:0/3092936236 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f014c0d0a20 con 0x7f0138003f60 2024-04-25T10:14:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.656+0000 7f0153ddb640 1 -- 172.21.15.16:0/3092936236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f014c0750b0 con 0x7f0138003f60 2024-04-25T10:14:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.656+0000 7f01427fc640 1 -- 172.21.15.16:0/3092936236 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01480588f0 con 0x7f0138003f60 2024-04-25T10:14:19.662 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:14:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.659+0000 7f0153ddb640 1 -- 172.21.15.16:0/3092936236 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f013006b500 msgr2=0x7f013006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.659+0000 7f0153ddb640 1 --2- 172.21.15.16:0/3092936236 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f013006b500 0x7f013006d9c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f013c033000 tx=0x7f013c008030 comp rx=0 tx=0).stop 2024-04-25T10:14:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.659+0000 7f0153ddb640 1 -- 172.21.15.16:0/3092936236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0138003f60 msgr2=0x7f014c15e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.659+0000 7f0153ddb640 1 --2- 172.21.15.16:0/3092936236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0138003f60 0x7f014c15e3d0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f0148007310 tx=0x7f01480077d0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.659+0000 7f0153ddb640 1 -- 172.21.15.16:0/3092936236 shutdown_connections 2024-04-25T10:14:19.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.659+0000 7f0153ddb640 1 --2- 172.21.15.16:0/3092936236 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f013006b500 0x7f013006d9c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.659+0000 7f0153ddb640 1 --2- 172.21.15.16:0/3092936236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f014c1503d0 0x7f014c154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.659+0000 7f0153ddb640 1 --2- 172.21.15.16:0/3092936236 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f014c15c090 0x7f014c15b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.659+0000 7f0153ddb640 1 --2- 172.21.15.16:0/3092936236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0138003f60 0x7f014c15e3d0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.659+0000 7f0153ddb640 1 -- 172.21.15.16:0/3092936236 >> 172.21.15.16:0/3092936236 conn(0x7f014c0726c0 msgr2=0x7f014c1563b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:19.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.659+0000 7f0153ddb640 1 -- 172.21.15.16:0/3092936236 shutdown_connections 2024-04-25T10:14:19.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.659+0000 7f0153ddb640 1 -- 172.21.15.16:0/3092936236 wait complete. 2024-04-25T10:14:19.675 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-25T10:14:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.770+0000 7f0b67dd5640 1 Processor -- start 2024-04-25T10:14:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.771+0000 7f0b67dd5640 1 -- start start 2024-04-25T10:14:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.771+0000 7f0b67dd5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0b60071190 0x7f0b600ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.771+0000 7f0b67dd5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0b600b3730 0x7f0b6015e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.771+0000 7f0b67dd5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6006d550 con 0x7f0b60071570 2024-04-25T10:14:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.771+0000 7f0b67dd5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b600aeee0 con 0x7f0b600b3730 2024-04-25T10:14:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.771+0000 7f0b67dd5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6006d830 con 0x7f0b60071190 2024-04-25T10:14:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.772+0000 7f0b66372640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0b600b3730 0x7f0b6015e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.772+0000 7f0b66372640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0b600b3730 0x7f0b6015e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47446/0 (socket says 172.21.15.16:47446) 2024-04-25T10:14:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.772+0000 7f0b66372640 1 -- 172.21.15.16:0/1756222688 learned_addr learned my addr 172.21.15.16:0/1756222688 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.772+0000 7f0b65b71640 1 --2- 172.21.15.16:0/1756222688 >> v2:172.21.15.186:3300/0 conn(0x7f0b60071190 0x7f0b600ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.772+0000 7f0b66372640 1 -- 172.21.15.16:0/1756222688 >> v2:172.21.15.186:3300/0 conn(0x7f0b60071190 msgr2=0x7f0b600ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.772+0000 7f0b66372640 1 --2- 172.21.15.16:0/1756222688 >> v2:172.21.15.186:3300/0 conn(0x7f0b60071190 0x7f0b600ae9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.772+0000 7f0b66372640 1 -- 172.21.15.16:0/1756222688 >> v1:172.21.15.186:6789/0 conn(0x7f0b60071570 legacy=0x7f0b6015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.772+0000 7f0b66372640 1 -- 172.21.15.16:0/1756222688 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6006db10 con 0x7f0b600b3730 2024-04-25T10:14:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.773+0000 7f0b66372640 1 --2- 172.21.15.16:0/1756222688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b600b3730 0x7f0b6015e200 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000d150 tx=0x7f0b5002c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8567808f86b6ec5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.773+0000 7f0b64b6f640 1 -- 172.21.15.16:0/1756222688 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50009600 con 0x7f0b600b3730 2024-04-25T10:14:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.773+0000 7f0b64b6f640 1 -- 172.21.15.16:0/1756222688 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5002f070 con 0x7f0b600b3730 2024-04-25T10:14:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.773+0000 7f0b64b6f640 1 -- 172.21.15.16:0/1756222688 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b500099c0 con 0x7f0b600b3730 2024-04-25T10:14:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.773+0000 7f0b67dd5640 1 -- 172.21.15.16:0/1756222688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b600b3730 msgr2=0x7f0b6015e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.773+0000 7f0b67dd5640 1 --2- 172.21.15.16:0/1756222688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b600b3730 0x7f0b6015e200 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000d150 tx=0x7f0b5002c930 comp rx=0 tx=0).stop 2024-04-25T10:14:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.773+0000 7f0b67dd5640 1 -- 172.21.15.16:0/1756222688 shutdown_connections 2024-04-25T10:14:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.773+0000 7f0b67dd5640 1 --2- 172.21.15.16:0/1756222688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b600b3730 0x7f0b6015e200 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.773+0000 7f0b67dd5640 1 --2- 172.21.15.16:0/1756222688 >> v2:172.21.15.186:3300/0 conn(0x7f0b60071190 0x7f0b600ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.773+0000 7f0b67dd5640 1 -- 172.21.15.16:0/1756222688 >> 172.21.15.16:0/1756222688 conn(0x7f0b6006aba0 msgr2=0x7f0b6006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.774+0000 7f0b67dd5640 1 -- 172.21.15.16:0/1756222688 shutdown_connections 2024-04-25T10:14:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.774+0000 7f0b67dd5640 1 -- 172.21.15.16:0/1756222688 wait complete. 2024-04-25T10:14:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.775+0000 7f0b67dd5640 1 Processor -- start 2024-04-25T10:14:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.775+0000 7f0b67dd5640 1 -- start start 2024-04-25T10:14:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.775+0000 7f0b67dd5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b60071190 0x7f0b600a8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.775+0000 7f0b67dd5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b60071570 0x7f0b601f7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.775+0000 7f0b67dd5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b600b3730 0x7f0b60145f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.775+0000 7f0b67dd5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b60071a00 con 0x7f0b60071570 2024-04-25T10:14:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.775+0000 7f0b67dd5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6006e0d0 con 0x7f0b60071190 2024-04-25T10:14:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.775+0000 7f0b67dd5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6006e8b0 con 0x7f0b600b3730 2024-04-25T10:14:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.776+0000 7f0b65370640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b60071570 0x7f0b601f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.776+0000 7f0b65370640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b60071570 0x7f0b601f7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47456/0 (socket says 172.21.15.16:47456) 2024-04-25T10:14:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.776+0000 7f0b65370640 1 -- 172.21.15.16:0/3175774808 learned_addr learned my addr 172.21.15.16:0/3175774808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.776+0000 7f0b65b71640 1 --2- 172.21.15.16:0/3175774808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b60071190 0x7f0b600a8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.776+0000 7f0b66372640 1 --2- 172.21.15.16:0/3175774808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b600b3730 0x7f0b60145f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.776+0000 7f0b65370640 1 -- 172.21.15.16:0/3175774808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b600b3730 msgr2=0x7f0b60145f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.776+0000 7f0b65370640 1 --2- 172.21.15.16:0/3175774808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b600b3730 0x7f0b60145f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.776+0000 7f0b65370640 1 -- 172.21.15.16:0/3175774808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b60071190 msgr2=0x7f0b600a8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.776+0000 7f0b65370640 1 --2- 172.21.15.16:0/3175774808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b60071190 0x7f0b600a8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.776+0000 7f0b65370640 1 -- 172.21.15.16:0/3175774808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b601f78b0 con 0x7f0b60071570 2024-04-25T10:14:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.776+0000 7f0b66372640 1 --2- 172.21.15.16:0/3175774808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b600b3730 0x7f0b60145f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.776+0000 7f0b65370640 1 --2- 172.21.15.16:0/3175774808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b60071570 0x7f0b601f7140 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400c430 tx=0x7f0b5400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.776+0000 7f0b5e7fc640 1 -- 172.21.15.16:0/3175774808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b54011790 con 0x7f0b60071570 2024-04-25T10:14:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.776+0000 7f0b5e7fc640 1 -- 172.21.15.16:0/3175774808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b54017070 con 0x7f0b60071570 2024-04-25T10:14:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.776+0000 7f0b67dd5640 1 -- 172.21.15.16:0/3175774808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b601f7b90 con 0x7f0b60071570 2024-04-25T10:14:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.776+0000 7f0b5e7fc640 1 -- 172.21.15.16:0/3175774808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b54011bb0 con 0x7f0b60071570 2024-04-25T10:14:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.776+0000 7f0b67dd5640 1 -- 172.21.15.16:0/3175774808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b600b05a0 con 0x7f0b60071570 2024-04-25T10:14:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.777+0000 7f0b67dd5640 1 -- 172.21.15.16:0/3175774808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b600b07a0 con 0x7f0b60071570 2024-04-25T10:14:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.778+0000 7f0b5e7fc640 1 -- 172.21.15.16:0/3175774808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b5401c460 con 0x7f0b60071570 2024-04-25T10:14:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.778+0000 7f0b5e7fc640 1 --2- 172.21.15.16:0/3175774808 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0b4c06b4d0 0x7f0b4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.778+0000 7f0b5e7fc640 1 -- 172.21.15.16:0/3175774808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f0b54089dc0 con 0x7f0b60071570 2024-04-25T10:14:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.778+0000 7f0b65b71640 1 --2- 172.21.15.16:0/3175774808 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0b4c06b4d0 0x7f0b4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.779+0000 7f0b65b71640 1 --2- 172.21.15.16:0/3175774808 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0b4c06b4d0 0x7f0b4c06d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800a970 tx=0x7f0b58008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:19.782+0000 7f0b5e7fc640 1 -- 172.21.15.16:0/3175774808 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b54058f70 con 0x7f0b60071570 2024-04-25T10:14:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:20.037+0000 7f0b67dd5640 1 -- 172.21.15.16:0/3175774808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7f0b600749b0 con 0x7f0b60071570 2024-04-25T10:14:21.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.038+0000 7f0b5e7fc640 1 -- 172.21.15.16:0/3175774808 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v436) v1 ==== 147+0+0 (secure 0 0 0) 0x7f0b54058870 con 0x7f0b60071570 2024-04-25T10:14:21.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.041+0000 7f0b67dd5640 1 -- 172.21.15.16:0/3175774808 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0b4c06b4d0 msgr2=0x7f0b4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.041+0000 7f0b67dd5640 1 --2- 172.21.15.16:0/3175774808 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0b4c06b4d0 0x7f0b4c06d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800a970 tx=0x7f0b58008040 comp rx=0 tx=0).stop 2024-04-25T10:14:21.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.041+0000 7f0b67dd5640 1 -- 172.21.15.16:0/3175774808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b60071570 msgr2=0x7f0b601f7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.041+0000 7f0b67dd5640 1 --2- 172.21.15.16:0/3175774808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b60071570 0x7f0b601f7140 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400c430 tx=0x7f0b5400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.041+0000 7f0b67dd5640 1 -- 172.21.15.16:0/3175774808 shutdown_connections 2024-04-25T10:14:21.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.041+0000 7f0b67dd5640 1 --2- 172.21.15.16:0/3175774808 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0b4c06b4d0 0x7f0b4c06d990 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.041+0000 7f0b67dd5640 1 --2- 172.21.15.16:0/3175774808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b600b3730 0x7f0b60145f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.041+0000 7f0b67dd5640 1 --2- 172.21.15.16:0/3175774808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b60071570 0x7f0b601f7140 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.041+0000 7f0b67dd5640 1 --2- 172.21.15.16:0/3175774808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b60071190 0x7f0b600a8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.041+0000 7f0b67dd5640 1 -- 172.21.15.16:0/3175774808 >> 172.21.15.16:0/3175774808 conn(0x7f0b6006aba0 msgr2=0x7f0b6006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.042+0000 7f0b67dd5640 1 -- 172.21.15.16:0/3175774808 shutdown_connections 2024-04-25T10:14:21.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.042+0000 7f0b67dd5640 1 -- 172.21.15.16:0/3175774808 wait complete. 2024-04-25T10:14:21.060 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:14:21.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.165+0000 7fe7f73f4640 1 Processor -- start 2024-04-25T10:14:21.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.165+0000 7fe7f73f4640 1 -- start start 2024-04-25T10:14:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.166+0000 7fe7f73f4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7f015dca0 0x7fe7f00ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.166+0000 7fe7f73f4640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe7f00d3b90 0x7fe7f00d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.166+0000 7fe7f73f4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7f00cee80 0x7fe7f00d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.166+0000 7fe7f73f4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f00b6b20 con 0x7fe7f015dca0 2024-04-25T10:14:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.166+0000 7fe7f73f4640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f00d4090 con 0x7fe7f00d3b90 2024-04-25T10:14:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.166+0000 7fe7f73f4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f00d4370 con 0x7fe7f00cee80 2024-04-25T10:14:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.166+0000 7fe7f5190640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7f015dca0 0x7fe7f00ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.166+0000 7fe7f5190640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7f015dca0 0x7fe7f00ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47462/0 (socket says 172.21.15.16:47462) 2024-04-25T10:14:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.166+0000 7fe7f5190640 1 -- 172.21.15.16:0/1128243975 learned_addr learned my addr 172.21.15.16:0/1128243975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.166+0000 7fe7f498f640 1 --2- 172.21.15.16:0/1128243975 >> v2:172.21.15.186:3300/0 conn(0x7fe7f00d3b90 0x7fe7f00d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.166+0000 7fe7f5190640 1 -- 172.21.15.16:0/1128243975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7f00cee80 msgr2=0x7fe7f00d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.166+0000 7fe7f5991640 1 --2- 172.21.15.16:0/1128243975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7f00cee80 0x7fe7f00d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.166+0000 7fe7f5190640 1 --2- 172.21.15.16:0/1128243975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7f00cee80 0x7fe7f00d0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.166+0000 7fe7f5190640 1 -- 172.21.15.16:0/1128243975 >> v2:172.21.15.186:3300/0 conn(0x7fe7f00d3b90 msgr2=0x7fe7f00d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.166+0000 7fe7f5190640 1 --2- 172.21.15.16:0/1128243975 >> v2:172.21.15.186:3300/0 conn(0x7fe7f00d3b90 0x7fe7f00d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.166+0000 7fe7f5190640 1 -- 172.21.15.16:0/1128243975 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f00d4650 con 0x7fe7f015dca0 2024-04-25T10:14:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.166+0000 7fe7f498f640 1 --2- 172.21.15.16:0/1128243975 >> v2:172.21.15.186:3300/0 conn(0x7fe7f00d3b90 0x7fe7f00d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.167+0000 7fe7f5190640 1 --2- 172.21.15.16:0/1128243975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f015dca0 0x7fe7f00ce940 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00b6560 tx=0x7fe7ec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac90005191f3c68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.167+0000 7fe7e7fff640 1 -- 172.21.15.16:0/1128243975 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec037070 con 0x7fe7f015dca0 2024-04-25T10:14:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.167+0000 7fe7e7fff640 1 -- 172.21.15.16:0/1128243975 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ec032440 con 0x7fe7f015dca0 2024-04-25T10:14:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.167+0000 7fe7e7fff640 1 -- 172.21.15.16:0/1128243975 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec032810 con 0x7fe7f015dca0 2024-04-25T10:14:21.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.168+0000 7fe7f73f4640 1 -- 172.21.15.16:0/1128243975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f015dca0 msgr2=0x7fe7f00ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.168+0000 7fe7f73f4640 1 --2- 172.21.15.16:0/1128243975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f015dca0 0x7fe7f00ce940 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00b6560 tx=0x7fe7ec02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.168+0000 7fe7f73f4640 1 -- 172.21.15.16:0/1128243975 shutdown_connections 2024-04-25T10:14:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.168+0000 7fe7f73f4640 1 --2- 172.21.15.16:0/1128243975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7f00cee80 0x7fe7f00d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.168+0000 7fe7f73f4640 1 --2- 172.21.15.16:0/1128243975 >> v2:172.21.15.186:3300/0 conn(0x7fe7f00d3b90 0x7fe7f00d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.168+0000 7fe7f73f4640 1 --2- 172.21.15.16:0/1128243975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f015dca0 0x7fe7f00ce940 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.168+0000 7fe7f73f4640 1 -- 172.21.15.16:0/1128243975 >> 172.21.15.16:0/1128243975 conn(0x7fe7f00726c0 msgr2=0x7fe7f0072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.168+0000 7fe7f73f4640 1 -- 172.21.15.16:0/1128243975 shutdown_connections 2024-04-25T10:14:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.168+0000 7fe7f73f4640 1 -- 172.21.15.16:0/1128243975 wait complete. 2024-04-25T10:14:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.169+0000 7fe7f73f4640 1 Processor -- start 2024-04-25T10:14:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.169+0000 7fe7f73f4640 1 -- start start 2024-04-25T10:14:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f73f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f00cee80 0x7fe7f0156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f73f4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7f00d3b90 0x7fe7f0159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f73f4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f015dca0 0x7fe7f015ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f73f4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f015e200 con 0x7fe7f00cee80 2024-04-25T10:14:21.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f73f4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f015e080 con 0x7fe7f015dca0 2024-04-25T10:14:21.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f73f4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f00b2500 con 0x7fe7f00d3b90 2024-04-25T10:14:21.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f5190640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f00cee80 0x7fe7f0156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f5190640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f00cee80 0x7fe7f0156270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47468/0 (socket says 172.21.15.16:47468) 2024-04-25T10:14:21.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f5190640 1 -- 172.21.15.16:0/3149184709 learned_addr learned my addr 172.21.15.16:0/3149184709 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:21.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f5991640 1 --2- 172.21.15.16:0/3149184709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f015dca0 0x7fe7f015ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f498f640 1 --2- 172.21.15.16:0/3149184709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7f00d3b90 0x7fe7f0159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f5190640 1 -- 172.21.15.16:0/3149184709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7f00d3b90 msgr2=0x7fe7f0159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f5190640 1 --2- 172.21.15.16:0/3149184709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7f00d3b90 0x7fe7f0159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f5190640 1 -- 172.21.15.16:0/3149184709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f015dca0 msgr2=0x7fe7f015ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f5190640 1 --2- 172.21.15.16:0/3149184709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f015dca0 0x7fe7f015ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f5190640 1 -- 172.21.15.16:0/3149184709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f00c7760 con 0x7fe7f00cee80 2024-04-25T10:14:21.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f498f640 1 --2- 172.21.15.16:0/3149184709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7f00d3b90 0x7fe7f0159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:21.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f5991640 1 --2- 172.21.15.16:0/3149184709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f015dca0 0x7fe7f015ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:21.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.170+0000 7fe7f5190640 1 --2- 172.21.15.16:0/3149184709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f00cee80 0x7fe7f0156270 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec000c00 tx=0x7fe7ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.171+0000 7fe7e5ffb640 1 -- 172.21.15.16:0/3149184709 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec037040 con 0x7fe7f00cee80 2024-04-25T10:14:21.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.171+0000 7fe7e5ffb640 1 -- 172.21.15.16:0/3149184709 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ec042070 con 0x7fe7f00cee80 2024-04-25T10:14:21.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.171+0000 7fe7e5ffb640 1 -- 172.21.15.16:0/3149184709 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec030bb0 con 0x7fe7f00cee80 2024-04-25T10:14:21.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.171+0000 7fe7f73f4640 1 -- 172.21.15.16:0/3149184709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f00c7a40 con 0x7fe7f00cee80 2024-04-25T10:14:21.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.171+0000 7fe7f73f4640 1 -- 172.21.15.16:0/3149184709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f00c8000 con 0x7fe7f00cee80 2024-04-25T10:14:21.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.172+0000 7fe7f73f4640 1 -- 172.21.15.16:0/3149184709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f00750b0 con 0x7fe7f00cee80 2024-04-25T10:14:21.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.172+0000 7fe7e5ffb640 1 -- 172.21.15.16:0/3149184709 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7ec047460 con 0x7fe7f00cee80 2024-04-25T10:14:21.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.173+0000 7fe7e5ffb640 1 --2- 172.21.15.16:0/3149184709 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe7d406b4d0 0x7fe7d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.173+0000 7fe7e5ffb640 1 -- 172.21.15.16:0/3149184709 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fe7ec0aac80 con 0x7fe7f00cee80 2024-04-25T10:14:21.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.173+0000 7fe7f498f640 1 --2- 172.21.15.16:0/3149184709 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe7d406b4d0 0x7fe7d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.174+0000 7fe7f498f640 1 --2- 172.21.15.16:0/3149184709 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe7d406b4d0 0x7fe7d406d990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0006fd0 tx=0x7fe7e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.177+0000 7fe7e5ffb640 1 -- 172.21.15.16:0/3149184709 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe7ec079d20 con 0x7fe7f00cee80 2024-04-25T10:14:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.355+0000 7fe7f73f4640 1 -- 172.21.15.16:0/3149184709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe7f00cdb90 con 0x7fe7f00cee80 2024-04-25T10:14:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.358+0000 7fe7e5ffb640 1 -- 172.21.15.16:0/3149184709 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe7ec079620 con 0x7fe7f00cee80 2024-04-25T10:14:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.360+0000 7fe7f73f4640 1 -- 172.21.15.16:0/3149184709 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe7d406b4d0 msgr2=0x7fe7d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.360+0000 7fe7f73f4640 1 --2- 172.21.15.16:0/3149184709 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe7d406b4d0 0x7fe7d406d990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0006fd0 tx=0x7fe7e0008040 comp rx=0 tx=0).stop 2024-04-25T10:14:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.360+0000 7fe7f73f4640 1 -- 172.21.15.16:0/3149184709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f00cee80 msgr2=0x7fe7f0156270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.360+0000 7fe7f73f4640 1 --2- 172.21.15.16:0/3149184709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f00cee80 0x7fe7f0156270 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec000c00 tx=0x7fe7ec0046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.360+0000 7fe7f73f4640 1 -- 172.21.15.16:0/3149184709 shutdown_connections 2024-04-25T10:14:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.360+0000 7fe7f73f4640 1 --2- 172.21.15.16:0/3149184709 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe7d406b4d0 0x7fe7d406d990 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.360+0000 7fe7f73f4640 1 --2- 172.21.15.16:0/3149184709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f015dca0 0x7fe7f015ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.360+0000 7fe7f73f4640 1 --2- 172.21.15.16:0/3149184709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7f00d3b90 0x7fe7f0159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.360+0000 7fe7f73f4640 1 --2- 172.21.15.16:0/3149184709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f00cee80 0x7fe7f0156270 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.360+0000 7fe7f73f4640 1 -- 172.21.15.16:0/3149184709 >> 172.21.15.16:0/3149184709 conn(0x7fe7f00726c0 msgr2=0x7fe7f00d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.360+0000 7fe7f73f4640 1 -- 172.21.15.16:0/3149184709 shutdown_connections 2024-04-25T10:14:21.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.360+0000 7fe7f73f4640 1 -- 172.21.15.16:0/3149184709 wait complete. 2024-04-25T10:14:21.376 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:14:21.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:21.418 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:21.465 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:14:21.465 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:14:21.466 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:21.481 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:14:21.482 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:14:21.482 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:21.520 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:21.521 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:14:21.521 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:21.537 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:14:21.537 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:14:21.537 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:21.579 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:21.579 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:14:21.579 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.674+0000 7fb4f9b42640 1 Processor -- start 2024-04-25T10:14:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.674+0000 7fb4f9b42640 1 -- start start 2024-04-25T10:14:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.675+0000 7fb4f9b42640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb4f415dce0 0x7fb4f40d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.675+0000 7fb4f9b42640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4f40d35b0 0x7fb4f40b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.675+0000 7fb4f9b42640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f415e0c0 con 0x7fb4f40d1f90 2024-04-25T10:14:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.675+0000 7fb4f9b42640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f40b7b10 con 0x7fb4f40d35b0 2024-04-25T10:14:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.675+0000 7fb4f9b42640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f40d2510 con 0x7fb4f415dce0 2024-04-25T10:14:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.675+0000 7fb4f2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4f40d35b0 0x7fb4f40b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.675+0000 7fb4f3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb4f40d1f90 0x7fb4f406bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47762/0 (socket says 172.21.15.16:47762) 2024-04-25T10:14:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.675+0000 7fb4f3fff640 1 -- 172.21.15.16:0/258207318 learned_addr learned my addr 172.21.15.16:0/258207318 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.675+0000 7fb4f37fe640 1 --2- 172.21.15.16:0/258207318 >> v2:172.21.15.186:3300/0 conn(0x7fb4f415dce0 0x7fb4f40d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.675+0000 7fb4f2ffd640 1 -- 172.21.15.16:0/258207318 >> v2:172.21.15.186:3300/0 conn(0x7fb4f415dce0 msgr2=0x7fb4f40d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.675+0000 7fb4f2ffd640 1 --2- 172.21.15.16:0/258207318 >> v2:172.21.15.186:3300/0 conn(0x7fb4f415dce0 0x7fb4f40d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.675+0000 7fb4f2ffd640 1 -- 172.21.15.16:0/258207318 >> v1:172.21.15.16:6789/0 conn(0x7fb4f40d1f90 legacy=0x7fb4f406bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.675+0000 7fb4f2ffd640 1 -- 172.21.15.16:0/258207318 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f40d27f0 con 0x7fb4f40d35b0 2024-04-25T10:14:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.675+0000 7fb4f27fc640 1 -- 172.21.15.16:0/258207318 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2123015511 0 0) 0x7fb4f415e0c0 con 0x7fb4f40d1f90 2024-04-25T10:14:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.676+0000 7fb4f37fe640 1 --2- 172.21.15.16:0/258207318 >> v2:172.21.15.186:3300/0 conn(0x7fb4f415dce0 0x7fb4f40d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.676+0000 7fb4f2ffd640 1 --2- 172.21.15.16:0/258207318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f40d35b0 0x7fb4f40b5450 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40ce220 tx=0x7fb4e802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bf1316900373be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.676+0000 7fb4f27fc640 1 -- 172.21.15.16:0/258207318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8037070 con 0x7fb4f40d35b0 2024-04-25T10:14:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.676+0000 7fb4f27fc640 1 -- 172.21.15.16:0/258207318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e8032440 con 0x7fb4f40d35b0 2024-04-25T10:14:21.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.676+0000 7fb4f27fc640 1 -- 172.21.15.16:0/258207318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8032810 con 0x7fb4f40d35b0 2024-04-25T10:14:21.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.677+0000 7fb4f9b42640 1 -- 172.21.15.16:0/258207318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f40d35b0 msgr2=0x7fb4f40b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.677+0000 7fb4f9b42640 1 --2- 172.21.15.16:0/258207318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f40d35b0 0x7fb4f40b5450 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40ce220 tx=0x7fb4e802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.677+0000 7fb4f9b42640 1 -- 172.21.15.16:0/258207318 shutdown_connections 2024-04-25T10:14:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.677+0000 7fb4f9b42640 1 --2- 172.21.15.16:0/258207318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f40d35b0 0x7fb4f40b5450 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.677+0000 7fb4f9b42640 1 --2- 172.21.15.16:0/258207318 >> v2:172.21.15.186:3300/0 conn(0x7fb4f415dce0 0x7fb4f40d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.677+0000 7fb4f9b42640 1 -- 172.21.15.16:0/258207318 >> 172.21.15.16:0/258207318 conn(0x7fb4f4072900 msgr2=0x7fb4f40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.677+0000 7fb4f9b42640 1 -- 172.21.15.16:0/258207318 shutdown_connections 2024-04-25T10:14:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.677+0000 7fb4f9b42640 1 -- 172.21.15.16:0/258207318 wait complete. 2024-04-25T10:14:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.678+0000 7fb4f9b42640 1 Processor -- start 2024-04-25T10:14:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.678+0000 7fb4f9b42640 1 -- start start 2024-04-25T10:14:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.679+0000 7fb4f9b42640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4f40d1f90 0x7fb4f40ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.679+0000 7fb4f9b42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f40d35b0 0x7fb4f415fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.679+0000 7fb4f9b42640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4f415dce0 0x7fb4f4150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.679+0000 7fb4f9b42640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f40d3090 con 0x7fb4f40d35b0 2024-04-25T10:14:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.679+0000 7fb4f9b42640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f40d2db0 con 0x7fb4f40d1f90 2024-04-25T10:14:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.679+0000 7fb4f9b42640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f40b9820 con 0x7fb4f415dce0 2024-04-25T10:14:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.679+0000 7fb4f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f40d35b0 0x7fb4f415fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.679+0000 7fb4f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f40d35b0 0x7fb4f415fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47486/0 (socket says 172.21.15.16:47486) 2024-04-25T10:14:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.679+0000 7fb4f2ffd640 1 -- 172.21.15.16:0/2575465884 learned_addr learned my addr 172.21.15.16:0/2575465884 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.679+0000 7fb4f37fe640 1 --2- 172.21.15.16:0/2575465884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4f40d1f90 0x7fb4f40ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.679+0000 7fb4f3fff640 1 --2- 172.21.15.16:0/2575465884 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4f415dce0 0x7fb4f4150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.680+0000 7fb4f2ffd640 1 -- 172.21.15.16:0/2575465884 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4f415dce0 msgr2=0x7fb4f4150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.680+0000 7fb4f2ffd640 1 --2- 172.21.15.16:0/2575465884 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4f415dce0 0x7fb4f4150dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.680+0000 7fb4f2ffd640 1 -- 172.21.15.16:0/2575465884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4f40d1f90 msgr2=0x7fb4f40ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.680+0000 7fb4f2ffd640 1 --2- 172.21.15.16:0/2575465884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4f40d1f90 0x7fb4f40ab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.680+0000 7fb4f2ffd640 1 -- 172.21.15.16:0/2575465884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f41604f0 con 0x7fb4f40d35b0 2024-04-25T10:14:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.680+0000 7fb4f2ffd640 1 --2- 172.21.15.16:0/2575465884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f40d35b0 0x7fb4f415fdf0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8035040 tx=0x7fb4e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.680+0000 7fb4dbfff640 1 -- 172.21.15.16:0/2575465884 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8037050 con 0x7fb4f40d35b0 2024-04-25T10:14:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.680+0000 7fb4dbfff640 1 -- 172.21.15.16:0/2575465884 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e8038040 con 0x7fb4f40d35b0 2024-04-25T10:14:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.680+0000 7fb4dbfff640 1 -- 172.21.15.16:0/2575465884 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8004450 con 0x7fb4f40d35b0 2024-04-25T10:14:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.680+0000 7fb4f9b42640 1 -- 172.21.15.16:0/2575465884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f41607d0 con 0x7fb4f40d35b0 2024-04-25T10:14:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.680+0000 7fb4f9b42640 1 -- 172.21.15.16:0/2575465884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f4160d90 con 0x7fb4f40d35b0 2024-04-25T10:14:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.681+0000 7fb4f9b42640 1 -- 172.21.15.16:0/2575465884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f40749b0 con 0x7fb4f40d35b0 2024-04-25T10:14:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.681+0000 7fb4dbfff640 1 -- 172.21.15.16:0/2575465884 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb4e8030980 con 0x7fb4f40d35b0 2024-04-25T10:14:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.682+0000 7fb4dbfff640 1 --2- 172.21.15.16:0/2575465884 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4e006b4d0 0x7fb4e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.682+0000 7fb4dbfff640 1 -- 172.21.15.16:0/2575465884 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fb4e80aabf0 con 0x7fb4f40d35b0 2024-04-25T10:14:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.682+0000 7fb4f37fe640 1 --2- 172.21.15.16:0/2575465884 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4e006b4d0 0x7fb4e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.682+0000 7fb4f37fe640 1 --2- 172.21.15.16:0/2575465884 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4e006b4d0 0x7fb4e006d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400aac0 tx=0x7fb4e4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.686+0000 7fb4dbfff640 1 -- 172.21.15.16:0/2575465884 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb4e80797e0 con 0x7fb4f40d35b0 2024-04-25T10:14:21.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.901+0000 7fb4f9b42640 1 -- 172.21.15.16:0/2575465884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4f40b3170 con 0x7fb4f40d35b0 2024-04-25T10:14:21.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.902+0000 7fb4dbfff640 1 -- 172.21.15.16:0/2575465884 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+3699 (secure 0 0 0) 0x7fb4e807d880 con 0x7fb4f40d35b0 2024-04-25T10:14:21.907 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:21.907 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":80,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7243,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/109406141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":109406141},{"type":"v1","addr":"172.21.15.186:6837","nonce":109406141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7597,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/3732623843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3732623843},{"type":"v1","addr":"172.21.15.186:6836","nonce":3732623843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8190,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2704401503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2704401503},{"type":"v1","addr":"172.21.15.16:6835","nonce":2704401503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/939221717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":939221717},{"type":"v1","addr":"172.21.15.16:6834","nonce":939221717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[]} 2024-04-25T10:14:21.907 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 80 2024-04-25T10:14:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.904+0000 7fb4f9b42640 1 -- 172.21.15.16:0/2575465884 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4e006b4d0 msgr2=0x7fb4e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.904+0000 7fb4f9b42640 1 --2- 172.21.15.16:0/2575465884 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4e006b4d0 0x7fb4e006d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400aac0 tx=0x7fb4e4008040 comp rx=0 tx=0).stop 2024-04-25T10:14:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.904+0000 7fb4f9b42640 1 -- 172.21.15.16:0/2575465884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f40d35b0 msgr2=0x7fb4f415fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.904+0000 7fb4f9b42640 1 --2- 172.21.15.16:0/2575465884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f40d35b0 0x7fb4f415fdf0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8035040 tx=0x7fb4e80046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.904+0000 7fb4f9b42640 1 -- 172.21.15.16:0/2575465884 shutdown_connections 2024-04-25T10:14:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.904+0000 7fb4f9b42640 1 --2- 172.21.15.16:0/2575465884 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb4e006b4d0 0x7fb4e006d990 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.904+0000 7fb4f9b42640 1 --2- 172.21.15.16:0/2575465884 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4f415dce0 0x7fb4f4150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.904+0000 7fb4f9b42640 1 --2- 172.21.15.16:0/2575465884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f40d35b0 0x7fb4f415fdf0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.904+0000 7fb4f9b42640 1 --2- 172.21.15.16:0/2575465884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4f40d1f90 0x7fb4f40ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.904+0000 7fb4f9b42640 1 -- 172.21.15.16:0/2575465884 >> 172.21.15.16:0/2575465884 conn(0x7fb4f4072900 msgr2=0x7fb4f40d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.904+0000 7fb4f9b42640 1 -- 172.21.15.16:0/2575465884 shutdown_connections 2024-04-25T10:14:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:21.904+0000 7fb4f9b42640 1 -- 172.21.15.16:0/2575465884 wait complete. 2024-04-25T10:14:21.920 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:14:21.921 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:14:21.921 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:14:21.921 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:21.921 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:14:21.921 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:14:21.921 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:14:21.921 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:21.922 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:14:21.922 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:14:21.922 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:14:21.922 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:21.922 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:14:21.922 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:14:21.923 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:14:21.923 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:21.930 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:14:21.930 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:14:21.931 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:14:21.931 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:14:21.934 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:14:21.935 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:14:21.935 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:14:21.935 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:14:21.973 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:14:21.974 DEBUG:teuthology.parallel:result is None 2024-04-25T10:14:21.974 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:14:21.974 DEBUG:teuthology.parallel:result is None 2024-04-25T10:14:21.976 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:14:21.977 DEBUG:teuthology.parallel:result is None 2024-04-25T10:14:21.977 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:14:21.977 DEBUG:teuthology.parallel:result is None 2024-04-25T10:14:21.978 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:14:22.050 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-04-25T10:14:22.051 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-04-25T10:14:22.057 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2024-04-25T10:14:22.057 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2024-04-25T10:14:22.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.080+0000 7f610d67c640 1 Processor -- start 2024-04-25T10:14:22.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.080+0000 7f610d67c640 1 -- start start 2024-04-25T10:14:22.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.081+0000 7f610d67c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f610815ef60 0x7f61080ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.081+0000 7f610d67c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61080cea00 0x7f61080b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.081+0000 7f610d67c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61080b1060 con 0x7f61080b0c80 2024-04-25T10:14:22.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.081+0000 7f610d67c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f610815f660 con 0x7f61080cea00 2024-04-25T10:14:22.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.081+0000 7f610d67c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f610815f940 con 0x7f610815ef60 2024-04-25T10:14:22.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.081+0000 7f60fffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61080cea00 0x7f61080b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.081+0000 7f60fffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61080cea00 0x7f61080b0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47512/0 (socket says 172.21.15.16:47512) 2024-04-25T10:14:22.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.081+0000 7f61077fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f61080b0c80 0x7f61080b3370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47772/0 (socket says 172.21.15.16:47772) 2024-04-25T10:14:22.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.081+0000 7f60fffff640 1 -- 172.21.15.16:0/448514116 learned_addr learned my addr 172.21.15.16:0/448514116 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:22.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.081+0000 7f6106ffd640 1 --2- 172.21.15.16:0/448514116 >> v2:172.21.15.186:3300/0 conn(0x7f610815ef60 0x7f61080ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.081+0000 7f60fffff640 1 -- 172.21.15.16:0/448514116 >> v2:172.21.15.186:3300/0 conn(0x7f610815ef60 msgr2=0x7f61080ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.081+0000 7f60fffff640 1 --2- 172.21.15.16:0/448514116 >> v2:172.21.15.186:3300/0 conn(0x7f610815ef60 0x7f61080ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.081+0000 7f60fffff640 1 -- 172.21.15.16:0/448514116 >> v1:172.21.15.16:6789/0 conn(0x7f61080b0c80 legacy=0x7f61080b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.081+0000 7f61067fc640 1 -- 172.21.15.16:0/448514116 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3781468977 0 0) 0x7f61080b1060 con 0x7f61080b0c80 2024-04-25T10:14:22.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.081+0000 7f60fffff640 1 -- 172.21.15.16:0/448514116 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f610815fc20 con 0x7f61080cea00 2024-04-25T10:14:22.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.081+0000 7f6106ffd640 1 --2- 172.21.15.16:0/448514116 >> v2:172.21.15.186:3300/0 conn(0x7f610815ef60 0x7f61080ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:22.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.082+0000 7f60fffff640 1 --2- 172.21.15.16:0/448514116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61080cea00 0x7f61080b0740 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f6108070140 tx=0x7f60f002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1d3db7f70c610c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.082+0000 7f61067fc640 1 -- 172.21.15.16:0/448514116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f0037070 con 0x7f61080cea00 2024-04-25T10:14:22.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.082+0000 7f61067fc640 1 -- 172.21.15.16:0/448514116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f002ae50 con 0x7f61080cea00 2024-04-25T10:14:22.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.082+0000 7f61067fc640 1 -- 172.21.15.16:0/448514116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f0032860 con 0x7f61080cea00 2024-04-25T10:14:22.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.083+0000 7f610d67c640 1 -- 172.21.15.16:0/448514116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61080cea00 msgr2=0x7f61080b0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.083+0000 7f610d67c640 1 --2- 172.21.15.16:0/448514116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61080cea00 0x7f61080b0740 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f6108070140 tx=0x7f60f002a300 comp rx=0 tx=0).stop 2024-04-25T10:14:22.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.083+0000 7f610d67c640 1 -- 172.21.15.16:0/448514116 shutdown_connections 2024-04-25T10:14:22.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.083+0000 7f610d67c640 1 --2- 172.21.15.16:0/448514116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61080cea00 0x7f61080b0740 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.083+0000 7f610d67c640 1 --2- 172.21.15.16:0/448514116 >> v2:172.21.15.186:3300/0 conn(0x7f610815ef60 0x7f61080ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.083+0000 7f610d67c640 1 -- 172.21.15.16:0/448514116 >> 172.21.15.16:0/448514116 conn(0x7f610806f4c0 msgr2=0x7f610806f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.083+0000 7f610d67c640 1 -- 172.21.15.16:0/448514116 shutdown_connections 2024-04-25T10:14:22.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.083+0000 7f610d67c640 1 -- 172.21.15.16:0/448514116 wait complete. 2024-04-25T10:14:22.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.084+0000 7f610d67c640 1 Processor -- start 2024-04-25T10:14:22.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.084+0000 7f610d67c640 1 -- start start 2024-04-25T10:14:22.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.085+0000 7f610d67c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61080b0c80 0x7f610815b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.085+0000 7f610d67c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61080cea00 0x7f61080c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.085+0000 7f610d67c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f610815ef60 0x7f61080c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.085+0000 7f610d67c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61081604c0 con 0x7f61080cea00 2024-04-25T10:14:22.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.085+0000 7f610d67c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61081601e0 con 0x7f61080b0c80 2024-04-25T10:14:22.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.085+0000 7f610d67c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f610815f340 con 0x7f610815ef60 2024-04-25T10:14:22.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.085+0000 7f60fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61080cea00 0x7f61080c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.085+0000 7f60fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61080cea00 0x7f61080c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47522/0 (socket says 172.21.15.16:47522) 2024-04-25T10:14:22.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.085+0000 7f60fffff640 1 -- 172.21.15.16:0/1278973780 learned_addr learned my addr 172.21.15.16:0/1278973780 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:22.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.085+0000 7f6106ffd640 1 --2- 172.21.15.16:0/1278973780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61080b0c80 0x7f610815b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.085+0000 7f61077fe640 1 --2- 172.21.15.16:0/1278973780 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f610815ef60 0x7f61080c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.085+0000 7f60fffff640 1 -- 172.21.15.16:0/1278973780 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f610815ef60 msgr2=0x7f61080c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.085+0000 7f60fffff640 1 --2- 172.21.15.16:0/1278973780 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f610815ef60 0x7f61080c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.085+0000 7f60fffff640 1 -- 172.21.15.16:0/1278973780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61080b0c80 msgr2=0x7f610815b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.085+0000 7f60fffff640 1 --2- 172.21.15.16:0/1278973780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61080b0c80 0x7f610815b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.085+0000 7f60fffff640 1 -- 172.21.15.16:0/1278973780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61081f39e0 con 0x7f61080cea00 2024-04-25T10:14:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.085+0000 7f6106ffd640 1 --2- 172.21.15.16:0/1278973780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61080b0c80 0x7f610815b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.086+0000 7f60fffff640 1 --2- 172.21.15.16:0/1278973780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61080cea00 0x7f61080c8010 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f60f002a340 tx=0x7f60f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.086+0000 7f60ff7fe640 1 -- 172.21.15.16:0/1278973780 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f0041070 con 0x7f61080cea00 2024-04-25T10:14:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.086+0000 7f60ff7fe640 1 -- 172.21.15.16:0/1278973780 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f003c670 con 0x7f61080cea00 2024-04-25T10:14:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.086+0000 7f610d67c640 1 -- 172.21.15.16:0/1278973780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61081f3cc0 con 0x7f61080cea00 2024-04-25T10:14:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.086+0000 7f60ff7fe640 1 -- 172.21.15.16:0/1278973780 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f0037040 con 0x7f61080cea00 2024-04-25T10:14:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.086+0000 7f610d67c640 1 -- 172.21.15.16:0/1278973780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61081f4280 con 0x7f61080cea00 2024-04-25T10:14:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.087+0000 7f610d67c640 1 -- 172.21.15.16:0/1278973780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61080750b0 con 0x7f61080cea00 2024-04-25T10:14:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.087+0000 7f60ff7fe640 1 -- 172.21.15.16:0/1278973780 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f60f0044070 con 0x7f61080cea00 2024-04-25T10:14:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.088+0000 7f60ff7fe640 1 --2- 172.21.15.16:0/1278973780 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f60e806b4d0 0x7f60e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.088+0000 7f60ff7fe640 1 -- 172.21.15.16:0/1278973780 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f60f00aa710 con 0x7f61080cea00 2024-04-25T10:14:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.088+0000 7f6106ffd640 1 --2- 172.21.15.16:0/1278973780 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f60e806b4d0 0x7f60e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.088+0000 7f6106ffd640 1 --2- 172.21.15.16:0/1278973780 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f60e806b4d0 0x7f60e806d990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f60f8006fd0 tx=0x7f60f8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.092+0000 7f60ff7fe640 1 -- 172.21.15.16:0/1278973780 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f60f00799e0 con 0x7f61080cea00 2024-04-25T10:14:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:22.287+0000 7f610d67c640 1 -- 172.21.15.16:0/1278973780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f61080d1360 con 0x7f61080cea00 2024-04-25T10:14:23.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.369+0000 7f60ff7fe640 1 -- 172.21.15.16:0/1278973780 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v111) v1 ==== 97+0+0 (secure 0 0 0) 0x7f60f00792e0 con 0x7f61080cea00 2024-04-25T10:14:23.374 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-04-25T10:14:23.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.372+0000 7f610d67c640 1 -- 172.21.15.16:0/1278973780 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f60e806b4d0 msgr2=0x7f60e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.372+0000 7f610d67c640 1 --2- 172.21.15.16:0/1278973780 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f60e806b4d0 0x7f60e806d990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f60f8006fd0 tx=0x7f60f8008040 comp rx=0 tx=0).stop 2024-04-25T10:14:23.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.372+0000 7f610d67c640 1 -- 172.21.15.16:0/1278973780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61080cea00 msgr2=0x7f61080c8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.372+0000 7f610d67c640 1 --2- 172.21.15.16:0/1278973780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61080cea00 0x7f61080c8010 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f60f002a340 tx=0x7f60f00046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.372+0000 7f610d67c640 1 -- 172.21.15.16:0/1278973780 shutdown_connections 2024-04-25T10:14:23.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.372+0000 7f610d67c640 1 --2- 172.21.15.16:0/1278973780 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f60e806b4d0 0x7f60e806d990 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.372+0000 7f610d67c640 1 --2- 172.21.15.16:0/1278973780 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f610815ef60 0x7f61080c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.372+0000 7f610d67c640 1 --2- 172.21.15.16:0/1278973780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61080cea00 0x7f61080c8010 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.372+0000 7f610d67c640 1 --2- 172.21.15.16:0/1278973780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61080b0c80 0x7f610815b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.372+0000 7f610d67c640 1 -- 172.21.15.16:0/1278973780 >> 172.21.15.16:0/1278973780 conn(0x7f610806f4c0 msgr2=0x7f61080c89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.372+0000 7f610d67c640 1 -- 172.21.15.16:0/1278973780 shutdown_connections 2024-04-25T10:14:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.372+0000 7f610d67c640 1 -- 172.21.15.16:0/1278973780 wait complete. 2024-04-25T10:14:23.391 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:14:23.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.493+0000 7fcd525b0640 1 Processor -- start 2024-04-25T10:14:23.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.493+0000 7fcd525b0640 1 -- start start 2024-04-25T10:14:23.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.494+0000 7fcd525b0640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fcd4c0d35b0 0x7fcd4c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.494+0000 7fcd525b0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd4c0b5990 0x7fcd4c0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.494+0000 7fcd525b0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd4c15e0c0 con 0x7fcd4c15dce0 2024-04-25T10:14:23.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.494+0000 7fcd525b0640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd4c0d2510 con 0x7fcd4c0d35b0 2024-04-25T10:14:23.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.494+0000 7fcd525b0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c0d27f0 con 0x7fcd4c0b5990 2024-04-25T10:14:23.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.494+0000 7fcd4bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcd4c15dce0 0x7fcd4c0d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43536/0 (socket says 172.21.15.16:43536) 2024-04-25T10:14:23.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.494+0000 7fcd4bfff640 1 -- 172.21.15.16:0/1742398257 learned_addr learned my addr 172.21.15.16:0/1742398257 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:23.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.494+0000 7fcd4b7fe640 1 --2- 172.21.15.16:0/1742398257 >> v2:172.21.15.186:3300/0 conn(0x7fcd4c0d35b0 0x7fcd4c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.494+0000 7fcd4affd640 1 -- 172.21.15.16:0/1742398257 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4150991515 0 0) 0x7fcd4c15e0c0 con 0x7fcd4c15dce0 2024-04-25T10:14:23.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.494+0000 7fcd50b4d640 1 --2- 172.21.15.16:0/1742398257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd4c0b5990 0x7fcd4c0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.494+0000 7fcd4affd640 1 -- 172.21.15.16:0/1742398257 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd34003520 con 0x7fcd4c15dce0 2024-04-25T10:14:23.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.494+0000 7fcd4affd640 1 -- 172.21.15.16:0/1742398257 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 144473395 0 0) 0x7fcd34003520 con 0x7fcd4c15dce0 2024-04-25T10:14:23.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.494+0000 7fcd4affd640 1 -- 172.21.15.16:0/1742398257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd4c0b5990 msgr2=0x7fcd4c0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:23.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.494+0000 7fcd4affd640 1 --2- 172.21.15.16:0/1742398257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd4c0b5990 0x7fcd4c0b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.494+0000 7fcd4affd640 1 -- 172.21.15.16:0/1742398257 >> v2:172.21.15.186:3300/0 conn(0x7fcd4c0d35b0 msgr2=0x7fcd4c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:23.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.494+0000 7fcd4affd640 1 --2- 172.21.15.16:0/1742398257 >> v2:172.21.15.186:3300/0 conn(0x7fcd4c0d35b0 0x7fcd4c0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.494+0000 7fcd4affd640 1 -- 172.21.15.16:0/1742398257 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c0d2ad0 con 0x7fcd4c15dce0 2024-04-25T10:14:23.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.494+0000 7fcd4affd640 1 -- 172.21.15.16:0/1742398257 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fcd3c001670 con 0x7fcd4c15dce0 2024-04-25T10:14:23.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.495+0000 7fcd4affd640 1 -- 172.21.15.16:0/1742398257 >> v1:172.21.15.16:6789/0 conn(0x7fcd4c15dce0 legacy=0x7fcd4c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.495+0000 7fcd4affd640 1 --2- 172.21.15.16:0/1742398257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd340039b0 0x7fcd34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.495+0000 7fcd4affd640 1 --2- 172.21.15.16:0/1742398257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd4c0b5990 0x7fcd34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.495+0000 7fcd4affd640 1 --2- 172.21.15.16:0/1742398257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd34004d40 0x7fcd340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.495+0000 7fcd4affd640 1 -- 172.21.15.16:0/1742398257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c0d2510 con 0x7fcd34004d40 2024-04-25T10:14:23.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.495+0000 7fcd4affd640 1 -- 172.21.15.16:0/1742398257 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c0d27f0 con 0x7fcd340039b0 2024-04-25T10:14:23.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.495+0000 7fcd4affd640 1 -- 172.21.15.16:0/1742398257 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3400b9c0 con 0x7fcd4c0b5990 2024-04-25T10:14:23.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.495+0000 7fcd50b4d640 1 --2- 172.21.15.16:0/1742398257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd34004d40 0x7fcd340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.495+0000 7fcd4bfff640 1 --2- 172.21.15.16:0/1742398257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd340039b0 0x7fcd34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.495+0000 7fcd4b7fe640 1 --2- 172.21.15.16:0/1742398257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd4c0b5990 0x7fcd34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.495+0000 7fcd50b4d640 1 -- 172.21.15.16:0/1742398257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd4c0b5990 msgr2=0x7fcd34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:23.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.495+0000 7fcd50b4d640 1 --2- 172.21.15.16:0/1742398257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd4c0b5990 0x7fcd34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.495+0000 7fcd50b4d640 1 -- 172.21.15.16:0/1742398257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd340039b0 msgr2=0x7fcd34003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:23.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.495+0000 7fcd50b4d640 1 --2- 172.21.15.16:0/1742398257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd340039b0 0x7fcd34003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.495+0000 7fcd50b4d640 1 -- 172.21.15.16:0/1742398257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd34003520 con 0x7fcd34004d40 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.496+0000 7fcd50b4d640 1 --2- 172.21.15.16:0/1742398257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd34004d40 0x7fcd340051c0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400e170 tx=0x7fcd34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e79d31f6640e8fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.496+0000 7fcd4affd640 1 -- 172.21.15.16:0/1742398257 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44034020 con 0x7fcd34004d40 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.496+0000 7fcd4affd640 1 -- 172.21.15.16:0/1742398257 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd440305f0 con 0x7fcd34004d40 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.496+0000 7fcd4affd640 1 -- 172.21.15.16:0/1742398257 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44009c10 con 0x7fcd34004d40 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.496+0000 7fcd525b0640 1 -- 172.21.15.16:0/1742398257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd34004d40 msgr2=0x7fcd340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.496+0000 7fcd525b0640 1 --2- 172.21.15.16:0/1742398257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd34004d40 0x7fcd340051c0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400e170 tx=0x7fcd34005830 comp rx=0 tx=0).stop 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.496+0000 7fcd50b4d640 1 -- 172.21.15.16:0/1742398257 reap_dead start 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.496+0000 7fcd525b0640 1 -- 172.21.15.16:0/1742398257 shutdown_connections 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.496+0000 7fcd525b0640 1 -- 172.21.15.16:0/1742398257 >> 172.21.15.16:0/1742398257 conn(0x7fcd4c072900 msgr2=0x7fcd4c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.496+0000 7fcd525b0640 1 -- 172.21.15.16:0/1742398257 shutdown_connections 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.496+0000 7fcd525b0640 1 -- 172.21.15.16:0/1742398257 wait complete. 2024-04-25T10:14:23.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.498+0000 7fcd525b0640 1 Processor -- start 2024-04-25T10:14:23.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.498+0000 7fcd525b0640 1 -- start start 2024-04-25T10:14:23.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd525b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd4c160c80 0x7fcd4c0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd525b0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd4c156470 0x7fcd4c1568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd525b0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd4c1f3b30 0x7fcd4c1f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd525b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c0d2160 con 0x7fcd4c160c80 2024-04-25T10:14:23.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd525b0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c0d3090 con 0x7fcd4c156470 2024-04-25T10:14:23.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd525b0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c0cd530 con 0x7fcd4c1f3b30 2024-04-25T10:14:23.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd4c160c80 0x7fcd4c0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd4c160c80 0x7fcd4c0cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38994/0 (socket says 172.21.15.16:38994) 2024-04-25T10:14:23.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd4bfff640 1 -- 172.21.15.16:0/1411037242 learned_addr learned my addr 172.21.15.16:0/1411037242 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:23.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd4b7fe640 1 --2- 172.21.15.16:0/1411037242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd4c156470 0x7fcd4c1568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd50b4d640 1 --2- 172.21.15.16:0/1411037242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd4c1f3b30 0x7fcd4c1f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd4bfff640 1 -- 172.21.15.16:0/1411037242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd4c1f3b30 msgr2=0x7fcd4c1f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd4bfff640 1 --2- 172.21.15.16:0/1411037242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd4c1f3b30 0x7fcd4c1f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd4bfff640 1 -- 172.21.15.16:0/1411037242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd4c156470 msgr2=0x7fcd4c1568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd4bfff640 1 --2- 172.21.15.16:0/1411037242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd4c156470 0x7fcd4c1568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd4bfff640 1 -- 172.21.15.16:0/1411037242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c0cc490 con 0x7fcd4c160c80 2024-04-25T10:14:23.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd4bfff640 1 --2- 172.21.15.16:0/1411037242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd4c160c80 0x7fcd4c0cbaf0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c0072c0 tx=0x7fcd3c007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:23.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd48ff9640 1 -- 172.21.15.16:0/1411037242 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c00f9c0 con 0x7fcd4c160c80 2024-04-25T10:14:23.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd48ff9640 1 -- 172.21.15.16:0/1411037242 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3c018070 con 0x7fcd4c160c80 2024-04-25T10:14:23.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd48ff9640 1 -- 172.21.15.16:0/1411037242 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c00fdc0 con 0x7fcd4c160c80 2024-04-25T10:14:23.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd525b0640 1 -- 172.21.15.16:0/1411037242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd4c0cc770 con 0x7fcd4c160c80 2024-04-25T10:14:23.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.499+0000 7fcd525b0640 1 -- 172.21.15.16:0/1411037242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd4c0ccd30 con 0x7fcd4c160c80 2024-04-25T10:14:23.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.500+0000 7fcd525b0640 1 -- 172.21.15.16:0/1411037242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd4c0749b0 con 0x7fcd4c160c80 2024-04-25T10:14:23.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.501+0000 7fcd48ff9640 1 -- 172.21.15.16:0/1411037242 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd3c01d460 con 0x7fcd4c160c80 2024-04-25T10:14:23.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.501+0000 7fcd48ff9640 1 --2- 172.21.15.16:0/1411037242 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcd3806b500 0x7fcd3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.502+0000 7fcd48ff9640 1 -- 172.21.15.16:0/1411037242 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcd3c089ff0 con 0x7fcd4c160c80 2024-04-25T10:14:23.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.502+0000 7fcd4b7fe640 1 --2- 172.21.15.16:0/1411037242 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcd3806b500 0x7fcd3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.502+0000 7fcd4b7fe640 1 --2- 172.21.15.16:0/1411037242 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcd3806b500 0x7fcd3806d9c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c15ed90 tx=0x7fcd40008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:23.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.506+0000 7fcd48ff9640 1 -- 172.21.15.16:0/1411037242 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd3c0591d0 con 0x7fcd4c160c80 2024-04-25T10:14:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.751+0000 7fcd525b0640 1 -- 172.21.15.16:0/1411037242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcd4c0b3cf0 con 0x7fcd4c160c80 2024-04-25T10:14:23.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.752+0000 7fcd48ff9640 1 -- 172.21.15.16:0/1411037242 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fcd3c058ad0 con 0x7fcd4c160c80 2024-04-25T10:14:23.757 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:23.757 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:14:23.757 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-04-25T10:14:23.757 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.757 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-04-25T10:14:23.757 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmj+xmBBAAaWdHmoiYcXzFnMVQSioT2Q==", 2024-04-25T10:14:23.757 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.758 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:14:23.758 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:14:23.758 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:14:23.758 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:14:23.758 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:23.758 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.758 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-04-25T10:14:23.758 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmUES+DxAAu5It7dQmb7/BM/4x1fUYXA==", 2024-04-25T10:14:23.758 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.758 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:14:23.758 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:14:23.758 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:14:23.758 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:14:23.759 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.759 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:23.759 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.759 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-04-25T10:14:23.759 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipm+LtFCxAAGG79kv8eWzly8l5qzQwfww==", 2024-04-25T10:14:23.759 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.759 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:14:23.759 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:14:23.759 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:14:23.759 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:14:23.759 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.759 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:23.759 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.760 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-04-25T10:14:23.760 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmqbSSFhAAZtrLsAGSyaMGgv45Hy0WpA==", 2024-04-25T10:14:23.760 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.760 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:14:23.760 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:14:23.760 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:14:23.760 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:14:23.760 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.760 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:23.760 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.760 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-04-25T10:14:23.760 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCaKipm2+UdOxAAP8GBunvYsue8EILYV5Uq7A==", 2024-04-25T10:14:23.760 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.760 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:23.761 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:14:23.761 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:14:23.761 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:23.761 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.761 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-04-25T10:14:23.761 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCdKipmSv87ORAAigVU7nyzkcXqMEHKsSAsqg==", 2024-04-25T10:14:23.761 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.761 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:23.761 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:14:23.761 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:14:23.761 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:23.761 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.762 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-04-25T10:14:23.762 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCgKipm2VMLOBAAJETFfWgq7H/imQAfw93Ruw==", 2024-04-25T10:14:23.762 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.762 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:23.762 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:14:23.762 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:14:23.762 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.762 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:23.762 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.762 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-04-25T10:14:23.762 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCjKipmb1g0NhAADH2uHQPhW84KLYyWbOjQdw==", 2024-04-25T10:14:23.762 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.762 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:23.762 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:14:23.763 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:14:23.763 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:23.763 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.763 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-04-25T10:14:23.763 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCpKipmnH0ROBAAEb8i4nfswYQ/w5mEw48kIg==", 2024-04-25T10:14:23.763 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.763 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:23.763 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:14:23.763 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:14:23.763 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:23.763 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.763 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-04-25T10:14:23.764 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCsKipm8HCRNhAA3FhcMlUWxMAiFRXbBWWvNA==", 2024-04-25T10:14:23.764 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.764 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:23.764 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:14:23.764 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:14:23.764 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.764 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:23.764 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.764 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-04-25T10:14:23.764 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCvKipmfgqpNRAAtPhpSDU9q23Sfa+V7m+3NA==", 2024-04-25T10:14:23.764 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.764 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:23.764 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:14:23.765 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:14:23.765 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.765 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:23.765 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.765 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-04-25T10:14:23.765 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCyKipm3ndyMxAAeMNODGxNupb043CsA/SP1A==", 2024-04-25T10:14:23.765 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.765 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:23.765 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-04-25T10:14:23.765 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:14:23.765 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.765 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:23.765 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.765 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-04-25T10:14:23.766 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmAph/GhAAG3x1zC98yHQ13fjpmrlPkg==", 2024-04-25T10:14:23.766 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.766 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:14:23.766 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-04-25T10:14:23.766 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:14:23.766 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:23.766 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.766 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-04-25T10:14:23.766 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmrccTIxAAuDZamFXTe6J2d+HAcBe+DA==", 2024-04-25T10:14:23.766 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.766 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:14:23.766 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:14:23.766 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:14:23.767 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:14:23.767 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:23.767 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.767 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-04-25T10:14:23.767 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmsFACIBAAFuWsmuQlF58yzIcYUx/6Iw==", 2024-04-25T10:14:23.767 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.767 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:14:23.767 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:14:23.767 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:14:23.767 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:14:23.767 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:23.767 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.768 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-04-25T10:14:23.768 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmAe1/KBAAXzLkscuwpCwMnrIKQny9+Q==", 2024-04-25T10:14:23.768 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.768 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-04-25T10:14:23.768 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-04-25T10:14:23.768 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-04-25T10:14:23.768 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-04-25T10:14:23.768 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.768 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:23.768 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.768 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-04-25T10:14:23.768 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCYKipmVFQ2JBAAdqG2p/M746PZPMDcpQjwfw==", 2024-04-25T10:14:23.768 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.768 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:14:23.769 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-04-25T10:14:23.769 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-04-25T10:14:23.769 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:14:23.769 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:23.769 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.769 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-04-25T10:14:23.769 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZKipmiPVHARAAGPbv3P3yVj3tlo0b2gmcyw==", 2024-04-25T10:14:23.769 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.769 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:14:23.769 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-04-25T10:14:23.769 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:14:23.769 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-25T10:14:23.769 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:23.770 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-04-25T10:14:23.770 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCYKipmghhyOBAAcCm0hQzU+Wlw45yZQETTTg==", 2024-04-25T10:14:23.770 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-04-25T10:14:23.770 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-04-25T10:14:23.770 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-04-25T10:14:23.770 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-04-25T10:14:23.770 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.770 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:23.770 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-04-25T10:14:23.770 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:14:23.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.754+0000 7fcd525b0640 1 -- 172.21.15.16:0/1411037242 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcd3806b500 msgr2=0x7fcd3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.754+0000 7fcd525b0640 1 --2- 172.21.15.16:0/1411037242 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcd3806b500 0x7fcd3806d9c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c15ed90 tx=0x7fcd40008040 comp rx=0 tx=0).stop 2024-04-25T10:14:23.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.754+0000 7fcd525b0640 1 -- 172.21.15.16:0/1411037242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd4c160c80 msgr2=0x7fcd4c0cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.754+0000 7fcd525b0640 1 --2- 172.21.15.16:0/1411037242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd4c160c80 0x7fcd4c0cbaf0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c0072c0 tx=0x7fcd3c007780 comp rx=0 tx=0).stop 2024-04-25T10:14:23.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.754+0000 7fcd525b0640 1 -- 172.21.15.16:0/1411037242 shutdown_connections 2024-04-25T10:14:23.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.754+0000 7fcd525b0640 1 --2- 172.21.15.16:0/1411037242 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcd3806b500 0x7fcd3806d9c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.754+0000 7fcd525b0640 1 --2- 172.21.15.16:0/1411037242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd4c1f3b30 0x7fcd4c1f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.754+0000 7fcd525b0640 1 --2- 172.21.15.16:0/1411037242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd4c156470 0x7fcd4c1568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.754+0000 7fcd525b0640 1 --2- 172.21.15.16:0/1411037242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd4c160c80 0x7fcd4c0cbaf0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.754+0000 7fcd525b0640 1 -- 172.21.15.16:0/1411037242 >> 172.21.15.16:0/1411037242 conn(0x7fcd4c072900 msgr2=0x7fcd4c0b8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:23.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.754+0000 7fcd525b0640 1 -- 172.21.15.16:0/1411037242 shutdown_connections 2024-04-25T10:14:23.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.754+0000 7fcd525b0640 1 -- 172.21.15.16:0/1411037242 wait complete. 2024-04-25T10:14:23.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:23.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.868+0000 7f0c9ebb0640 1 Processor -- start 2024-04-25T10:14:23.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.868+0000 7f0c9ebb0640 1 -- start start 2024-04-25T10:14:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.869+0000 7f0c9ebb0640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0c9815ec10 0x7f0c9806dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.869+0000 7f0c9ebb0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0c980b3750 0x7f0c9806aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.869+0000 7f0c9ebb0640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c9806e6c0 con 0x7f0c9806e1b0 2024-04-25T10:14:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.869+0000 7f0c9ebb0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c9806f2d0 con 0x7f0c980b3750 2024-04-25T10:14:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.869+0000 7f0c9ebb0640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c9806e9a0 con 0x7f0c9815ec10 2024-04-25T10:14:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.869+0000 7f0c9d14d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0c980b3750 0x7f0c9806aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.869+0000 7f0c9d14d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0c980b3750 0x7f0c9806aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38996/0 (socket says 172.21.15.16:38996) 2024-04-25T10:14:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.869+0000 7f0c9d14d640 1 -- 172.21.15.16:0/3880091800 learned_addr learned my addr 172.21.15.16:0/3880091800 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.869+0000 7f0c9c94c640 1 --2- 172.21.15.16:0/3880091800 >> v2:172.21.15.186:3300/0 conn(0x7f0c9815ec10 0x7f0c9806dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.869+0000 7f0c9d14d640 1 -- 172.21.15.16:0/3880091800 >> v2:172.21.15.186:3300/0 conn(0x7f0c9815ec10 msgr2=0x7f0c9806dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.869+0000 7f0c9d14d640 1 --2- 172.21.15.16:0/3880091800 >> v2:172.21.15.186:3300/0 conn(0x7f0c9815ec10 0x7f0c9806dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.869+0000 7f0c9d14d640 1 -- 172.21.15.16:0/3880091800 >> v1:172.21.15.186:6789/0 conn(0x7f0c9806e1b0 legacy=0x7f0c980b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.869+0000 7f0c9d14d640 1 -- 172.21.15.16:0/3880091800 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9806ec80 con 0x7f0c980b3750 2024-04-25T10:14:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.870+0000 7f0c9d14d640 1 --2- 172.21.15.16:0/3880091800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c980b3750 0x7f0c9806aba0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009e80 tx=0x7f0c9002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e712d07b6c383c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.870+0000 7f0c977fe640 1 -- 172.21.15.16:0/3880091800 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90036070 con 0x7f0c980b3750 2024-04-25T10:14:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.870+0000 7f0c977fe640 1 -- 172.21.15.16:0/3880091800 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c90031600 con 0x7f0c980b3750 2024-04-25T10:14:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.870+0000 7f0c977fe640 1 -- 172.21.15.16:0/3880091800 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c900319d0 con 0x7f0c980b3750 2024-04-25T10:14:23.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.871+0000 7f0c9ebb0640 1 -- 172.21.15.16:0/3880091800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c980b3750 msgr2=0x7f0c9806aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.871+0000 7f0c9ebb0640 1 --2- 172.21.15.16:0/3880091800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c980b3750 0x7f0c9806aba0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009e80 tx=0x7f0c9002a630 comp rx=0 tx=0).stop 2024-04-25T10:14:23.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.871+0000 7f0c9ebb0640 1 -- 172.21.15.16:0/3880091800 shutdown_connections 2024-04-25T10:14:23.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.871+0000 7f0c9ebb0640 1 --2- 172.21.15.16:0/3880091800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c980b3750 0x7f0c9806aba0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.871+0000 7f0c9ebb0640 1 --2- 172.21.15.16:0/3880091800 >> v2:172.21.15.186:3300/0 conn(0x7f0c9815ec10 0x7f0c9806dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.871+0000 7f0c9ebb0640 1 -- 172.21.15.16:0/3880091800 >> 172.21.15.16:0/3880091800 conn(0x7f0c980b2220 msgr2=0x7f0c980cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:23.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.871+0000 7f0c9ebb0640 1 -- 172.21.15.16:0/3880091800 shutdown_connections 2024-04-25T10:14:23.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.871+0000 7f0c9ebb0640 1 -- 172.21.15.16:0/3880091800 wait complete. 2024-04-25T10:14:23.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.872+0000 7f0c9ebb0640 1 Processor -- start 2024-04-25T10:14:23.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.872+0000 7f0c9ebb0640 1 -- start start 2024-04-25T10:14:23.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c9ebb0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c9806e1b0 0x7f0c980b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c9ebb0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c980b3750 0x7f0c980ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c9ebb0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c9815ec10 0x7f0c98142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c9ebb0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9815e2e0 con 0x7f0c9806e1b0 2024-04-25T10:14:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c9ebb0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9815eff0 con 0x7f0c9815ec10 2024-04-25T10:14:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c9ebb0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9815e460 con 0x7f0c980b3750 2024-04-25T10:14:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c9c94c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c9806e1b0 0x7f0c980b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c9c94c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c9806e1b0 0x7f0c980b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39000/0 (socket says 172.21.15.16:39000) 2024-04-25T10:14:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c9c94c640 1 -- 172.21.15.16:0/3426570592 learned_addr learned my addr 172.21.15.16:0/3426570592 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c9d14d640 1 --2- 172.21.15.16:0/3426570592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c9815ec10 0x7f0c98142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c9c94c640 1 -- 172.21.15.16:0/3426570592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c980b3750 msgr2=0x7f0c980ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c97fff640 1 --2- 172.21.15.16:0/3426570592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c980b3750 0x7f0c980ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c9c94c640 1 --2- 172.21.15.16:0/3426570592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c980b3750 0x7f0c980ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c9c94c640 1 -- 172.21.15.16:0/3426570592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c9815ec10 msgr2=0x7f0c98142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c9c94c640 1 --2- 172.21.15.16:0/3426570592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c9815ec10 0x7f0c98142670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c9c94c640 1 -- 172.21.15.16:0/3426570592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c98145550 con 0x7f0c9806e1b0 2024-04-25T10:14:23.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c9d14d640 1 --2- 172.21.15.16:0/3426570592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c9815ec10 0x7f0c98142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:23.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c97fff640 1 --2- 172.21.15.16:0/3426570592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c980b3750 0x7f0c980ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:23.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.873+0000 7f0c9c94c640 1 --2- 172.21.15.16:0/3426570592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c9806e1b0 0x7f0c980b1160 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800d800 tx=0x7f0c8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:23.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.874+0000 7f0c957fa640 1 -- 172.21.15.16:0/3426570592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8800b6c0 con 0x7f0c9806e1b0 2024-04-25T10:14:23.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.874+0000 7f0c957fa640 1 -- 172.21.15.16:0/3426570592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c88019070 con 0x7f0c9806e1b0 2024-04-25T10:14:23.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.874+0000 7f0c9ebb0640 1 -- 172.21.15.16:0/3426570592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9814fa80 con 0x7f0c9806e1b0 2024-04-25T10:14:23.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.874+0000 7f0c957fa640 1 -- 172.21.15.16:0/3426570592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8800ba80 con 0x7f0c9806e1b0 2024-04-25T10:14:23.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.874+0000 7f0c9ebb0640 1 -- 172.21.15.16:0/3426570592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c98150040 con 0x7f0c9806e1b0 2024-04-25T10:14:23.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.875+0000 7f0c9ebb0640 1 -- 172.21.15.16:0/3426570592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c980749b0 con 0x7f0c9806e1b0 2024-04-25T10:14:23.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.875+0000 7f0c957fa640 1 -- 172.21.15.16:0/3426570592 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c8800bc20 con 0x7f0c9806e1b0 2024-04-25T10:14:23.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.875+0000 7f0c957fa640 1 --2- 172.21.15.16:0/3426570592 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0c8406b4d0 0x7f0c8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.876+0000 7f0c957fa640 1 -- 172.21.15.16:0/3426570592 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0c8808a360 con 0x7f0c9806e1b0 2024-04-25T10:14:23.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.876+0000 7f0c97fff640 1 --2- 172.21.15.16:0/3426570592 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0c8406b4d0 0x7f0c8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.876+0000 7f0c97fff640 1 --2- 172.21.15.16:0/3426570592 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0c8406b4d0 0x7f0c8406d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0c98154f50 tx=0x7f0c8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:23.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:23.880+0000 7f0c957fa640 1 -- 172.21.15.16:0/3426570592 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c98145550 con 0x7f0c9806e1b0 2024-04-25T10:14:24.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.062+0000 7f0c9ebb0640 1 -- 172.21.15.16:0/3426570592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c980d0b00 con 0x7f0c9806e1b0 2024-04-25T10:14:24.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.063+0000 7f0c957fa640 1 -- 172.21.15.16:0/3426570592 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+12897 (secure 0 0 0) 0x7f0c88059530 con 0x7f0c9806e1b0 2024-04-25T10:14:24.068 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:24.068 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":111,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:14:23.369231+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:24.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.065+0000 7f0c9ebb0640 1 -- 172.21.15.16:0/3426570592 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0c8406b4d0 msgr2=0x7f0c8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.065+0000 7f0c9ebb0640 1 --2- 172.21.15.16:0/3426570592 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0c8406b4d0 0x7f0c8406d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0c98154f50 tx=0x7f0c8c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:24.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.065+0000 7f0c9ebb0640 1 -- 172.21.15.16:0/3426570592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c9806e1b0 msgr2=0x7f0c980b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.065+0000 7f0c9ebb0640 1 --2- 172.21.15.16:0/3426570592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c9806e1b0 0x7f0c980b1160 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800d800 tx=0x7f0c8800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.065+0000 7f0c9ebb0640 1 -- 172.21.15.16:0/3426570592 shutdown_connections 2024-04-25T10:14:24.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.065+0000 7f0c9ebb0640 1 --2- 172.21.15.16:0/3426570592 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0c8406b4d0 0x7f0c8406d990 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.065+0000 7f0c9ebb0640 1 --2- 172.21.15.16:0/3426570592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c9815ec10 0x7f0c98142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.065+0000 7f0c9ebb0640 1 --2- 172.21.15.16:0/3426570592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0c980b3750 0x7f0c980ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.065+0000 7f0c9ebb0640 1 --2- 172.21.15.16:0/3426570592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c9806e1b0 0x7f0c980b1160 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.065+0000 7f0c9ebb0640 1 -- 172.21.15.16:0/3426570592 >> 172.21.15.16:0/3426570592 conn(0x7f0c980b2220 msgr2=0x7f0c980cc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:24.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.065+0000 7f0c9ebb0640 1 -- 172.21.15.16:0/3426570592 shutdown_connections 2024-04-25T10:14:24.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.065+0000 7f0c9ebb0640 1 -- 172.21.15.16:0/3426570592 wait complete. 2024-04-25T10:14:24.081 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:24.082 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:24.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.180+0000 7f2b3837e640 1 Processor -- start 2024-04-25T10:14:24.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.180+0000 7f2b3837e640 1 -- start start 2024-04-25T10:14:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.181+0000 7f2b3837e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2b3015dca0 0x7f2b300ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.181+0000 7f2b3837e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2b300d1130 0x7f2b300b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.181+0000 7f2b3837e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b300d4090 con 0x7f2b300d3b90 2024-04-25T10:14:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.181+0000 7f2b3837e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b300b6b20 con 0x7f2b3015dca0 2024-04-25T10:14:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.181+0000 7f2b3837e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b300d4370 con 0x7f2b300d1130 2024-04-25T10:14:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.181+0000 7f2b3611a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2b3015dca0 0x7f2b300ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.181+0000 7f2b3611a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2b3015dca0 0x7f2b300ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39004/0 (socket says 172.21.15.16:39004) 2024-04-25T10:14:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.181+0000 7f2b3611a640 1 -- 172.21.15.16:0/3398239006 learned_addr learned my addr 172.21.15.16:0/3398239006 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.181+0000 7f2b3611a640 1 -- 172.21.15.16:0/3398239006 >> v2:172.21.15.186:3300/0 conn(0x7f2b300d1130 msgr2=0x7f2b300b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.181+0000 7f2b3691b640 1 --2- 172.21.15.16:0/3398239006 >> v2:172.21.15.186:3300/0 conn(0x7f2b300d1130 0x7f2b300b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.181+0000 7f2b3611a640 1 --2- 172.21.15.16:0/3398239006 >> v2:172.21.15.186:3300/0 conn(0x7f2b300d1130 0x7f2b300b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.181+0000 7f2b3611a640 1 -- 172.21.15.16:0/3398239006 >> v1:172.21.15.186:6789/0 conn(0x7f2b300d3b90 legacy=0x7f2b300d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.181+0000 7f2b3611a640 1 -- 172.21.15.16:0/3398239006 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b300d4650 con 0x7f2b3015dca0 2024-04-25T10:14:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.182+0000 7f2b3611a640 1 --2- 172.21.15.16:0/3398239006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b3015dca0 0x7f2b300ce940 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f2b300b9640 tx=0x7f2b2c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad295eb839a4985 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.182+0000 7f2b35118640 1 -- 172.21.15.16:0/3398239006 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c037070 con 0x7f2b3015dca0 2024-04-25T10:14:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.182+0000 7f2b35118640 1 -- 172.21.15.16:0/3398239006 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c032440 con 0x7f2b3015dca0 2024-04-25T10:14:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.182+0000 7f2b35118640 1 -- 172.21.15.16:0/3398239006 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c032810 con 0x7f2b3015dca0 2024-04-25T10:14:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.183+0000 7f2b3837e640 1 -- 172.21.15.16:0/3398239006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b3015dca0 msgr2=0x7f2b300ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.183+0000 7f2b3837e640 1 --2- 172.21.15.16:0/3398239006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b3015dca0 0x7f2b300ce940 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f2b300b9640 tx=0x7f2b2c02a230 comp rx=0 tx=0).stop 2024-04-25T10:14:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.183+0000 7f2b3837e640 1 -- 172.21.15.16:0/3398239006 shutdown_connections 2024-04-25T10:14:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.183+0000 7f2b3837e640 1 --2- 172.21.15.16:0/3398239006 >> v2:172.21.15.186:3300/0 conn(0x7f2b300d1130 0x7f2b300b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.183+0000 7f2b3837e640 1 --2- 172.21.15.16:0/3398239006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b3015dca0 0x7f2b300ce940 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.183+0000 7f2b3837e640 1 -- 172.21.15.16:0/3398239006 >> 172.21.15.16:0/3398239006 conn(0x7f2b300726c0 msgr2=0x7f2b30072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.183+0000 7f2b3837e640 1 -- 172.21.15.16:0/3398239006 shutdown_connections 2024-04-25T10:14:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.183+0000 7f2b3837e640 1 -- 172.21.15.16:0/3398239006 wait complete. 2024-04-25T10:14:24.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.184+0000 7f2b3837e640 1 Processor -- start 2024-04-25T10:14:24.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.184+0000 7f2b3837e640 1 -- start start 2024-04-25T10:14:24.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.185+0000 7f2b3837e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b300d1130 0x7f2b3015c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.185+0000 7f2b3837e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b300d3b90 0x7f2b300aa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.185+0000 7f2b3837e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b3015dca0 0x7f2b301f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.185+0000 7f2b3837e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3015e200 con 0x7f2b300d1130 2024-04-25T10:14:24.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.185+0000 7f2b3837e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3015e080 con 0x7f2b3015dca0 2024-04-25T10:14:24.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.185+0000 7f2b3837e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b300cee80 con 0x7f2b300d3b90 2024-04-25T10:14:24.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.185+0000 7f2b3611a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b300d1130 0x7f2b3015c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.185+0000 7f2b3611a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b300d1130 0x7f2b3015c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39020/0 (socket says 172.21.15.16:39020) 2024-04-25T10:14:24.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.185+0000 7f2b3611a640 1 -- 172.21.15.16:0/3153163628 learned_addr learned my addr 172.21.15.16:0/3153163628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:24.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.185+0000 7f2b3611a640 1 -- 172.21.15.16:0/3153163628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b300d3b90 msgr2=0x7f2b300aa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.185+0000 7f2b3691b640 1 --2- 172.21.15.16:0/3153163628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b3015dca0 0x7f2b301f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.185+0000 7f2b3611a640 1 --2- 172.21.15.16:0/3153163628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b300d3b90 0x7f2b300aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.185+0000 7f2b3611a640 1 -- 172.21.15.16:0/3153163628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b3015dca0 msgr2=0x7f2b301f6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.185+0000 7f2b3611a640 1 --2- 172.21.15.16:0/3153163628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b3015dca0 0x7f2b301f6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.185+0000 7f2b3611a640 1 -- 172.21.15.16:0/3153163628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b30160520 con 0x7f2b300d1130 2024-04-25T10:14:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.185+0000 7f2b3611a640 1 --2- 172.21.15.16:0/3153163628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b300d1130 0x7f2b3015c040 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c000c00 tx=0x7f2b2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.185+0000 7f2b26ffd640 1 -- 172.21.15.16:0/3153163628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c041020 con 0x7f2b300d1130 2024-04-25T10:14:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.186+0000 7f2b26ffd640 1 -- 172.21.15.16:0/3153163628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c030990 con 0x7f2b300d1130 2024-04-25T10:14:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.186+0000 7f2b3837e640 1 -- 172.21.15.16:0/3153163628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b300ab5f0 con 0x7f2b300d1130 2024-04-25T10:14:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.186+0000 7f2b26ffd640 1 -- 172.21.15.16:0/3153163628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c037040 con 0x7f2b300d1130 2024-04-25T10:14:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.186+0000 7f2b3837e640 1 -- 172.21.15.16:0/3153163628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b301607c0 con 0x7f2b300d1130 2024-04-25T10:14:24.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.187+0000 7f2b3837e640 1 -- 172.21.15.16:0/3153163628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b300750b0 con 0x7f2b300d1130 2024-04-25T10:14:24.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.187+0000 7f2b26ffd640 1 -- 172.21.15.16:0/3153163628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b2c044020 con 0x7f2b300d1130 2024-04-25T10:14:24.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.187+0000 7f2b26ffd640 1 --2- 172.21.15.16:0/3153163628 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2b1406b4d0 0x7f2b1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.188+0000 7f2b26ffd640 1 -- 172.21.15.16:0/3153163628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2b2c0aa620 con 0x7f2b300d1130 2024-04-25T10:14:24.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.188+0000 7f2b35919640 1 --2- 172.21.15.16:0/3153163628 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2b1406b4d0 0x7f2b1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.188+0000 7f2b35919640 1 --2- 172.21.15.16:0/3153163628 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2b1406b4d0 0x7f2b1406d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2b300c8290 tx=0x7f2b200079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:24.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.192+0000 7f2b26ffd640 1 -- 172.21.15.16:0/3153163628 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2b2c04b070 con 0x7f2b300d1130 2024-04-25T10:14:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.371+0000 7f2b3837e640 1 -- 172.21.15.16:0/3153163628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2b300b3740 con 0x7f2b300d1130 2024-04-25T10:14:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.372+0000 7f2b26ffd640 1 -- 172.21.15.16:0/3153163628 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2b2c079750 con 0x7f2b300d1130 2024-04-25T10:14:24.377 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:14:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.374+0000 7f2b3837e640 1 -- 172.21.15.16:0/3153163628 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2b1406b4d0 msgr2=0x7f2b1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.374+0000 7f2b3837e640 1 --2- 172.21.15.16:0/3153163628 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2b1406b4d0 0x7f2b1406d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2b300c8290 tx=0x7f2b200079b0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.374+0000 7f2b3837e640 1 -- 172.21.15.16:0/3153163628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b300d1130 msgr2=0x7f2b3015c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.374+0000 7f2b3837e640 1 --2- 172.21.15.16:0/3153163628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b300d1130 0x7f2b3015c040 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c000c00 tx=0x7f2b2c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.374+0000 7f2b3837e640 1 -- 172.21.15.16:0/3153163628 shutdown_connections 2024-04-25T10:14:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.374+0000 7f2b3837e640 1 --2- 172.21.15.16:0/3153163628 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2b1406b4d0 0x7f2b1406d990 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.374+0000 7f2b3837e640 1 --2- 172.21.15.16:0/3153163628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b3015dca0 0x7f2b301f6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.374+0000 7f2b3837e640 1 --2- 172.21.15.16:0/3153163628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b300d3b90 0x7f2b300aa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.374+0000 7f2b3837e640 1 --2- 172.21.15.16:0/3153163628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b300d1130 0x7f2b3015c040 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.374+0000 7f2b3837e640 1 -- 172.21.15.16:0/3153163628 >> 172.21.15.16:0/3153163628 conn(0x7f2b300726c0 msgr2=0x7f2b3015cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.374+0000 7f2b3837e640 1 -- 172.21.15.16:0/3153163628 shutdown_connections 2024-04-25T10:14:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.374+0000 7f2b3837e640 1 -- 172.21.15.16:0/3153163628 wait complete. 2024-04-25T10:14:24.390 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:14:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.486+0000 7f4d059f4640 1 Processor -- start 2024-04-25T10:14:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.486+0000 7f4d059f4640 1 -- start start 2024-04-25T10:14:24.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.487+0000 7f4d059f4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d0015ec10 0x7f4d0006dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.487+0000 7f4d059f4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d0006e1b0 0x7f4d000b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.487+0000 7f4d059f4640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4d000b3580 0x7f4d001616d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.487+0000 7f4d059f4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d0006f2d0 con 0x7f4d0006e1b0 2024-04-25T10:14:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.487+0000 7f4d059f4640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d0006e6c0 con 0x7f4d000b3580 2024-04-25T10:14:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.487+0000 7f4d059f4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0006e9a0 con 0x7f4d0015ec10 2024-04-25T10:14:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.487+0000 7f4cf7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d0006e1b0 0x7f4d000b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.487+0000 7f4cf7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d0006e1b0 0x7f4d000b3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39032/0 (socket says 172.21.15.16:39032) 2024-04-25T10:14:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.487+0000 7f4cf7fff640 1 -- 172.21.15.16:0/1182166812 learned_addr learned my addr 172.21.15.16:0/1182166812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.487+0000 7f4cf7fff640 1 -- 172.21.15.16:0/1182166812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d0015ec10 msgr2=0x7f4d0006dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.487+0000 7f4cf7fff640 1 --2- 172.21.15.16:0/1182166812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d0015ec10 0x7f4d0006dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.487+0000 7f4cf7fff640 1 -- 172.21.15.16:0/1182166812 >> v2:172.21.15.186:3300/0 conn(0x7f4d000b3580 msgr2=0x7f4d001616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.487+0000 7f4cf7fff640 1 --2- 172.21.15.16:0/1182166812 >> v2:172.21.15.186:3300/0 conn(0x7f4d000b3580 0x7f4d001616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.487+0000 7f4cf7fff640 1 -- 172.21.15.16:0/1182166812 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0006ec80 con 0x7f4d0006e1b0 2024-04-25T10:14:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.488+0000 7f4cf7fff640 1 --2- 172.21.15.16:0/1182166812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d0006e1b0 0x7f4d000b3040 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009870 tx=0x7f4ce802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de76b5ac8d4b7fb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:24.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.488+0000 7f4cfe7fc640 1 -- 172.21.15.16:0/1182166812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8037070 con 0x7f4d0006e1b0 2024-04-25T10:14:24.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.488+0000 7f4cfe7fc640 1 -- 172.21.15.16:0/1182166812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce802ae10 con 0x7f4d0006e1b0 2024-04-25T10:14:24.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.488+0000 7f4cfe7fc640 1 -- 172.21.15.16:0/1182166812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8032660 con 0x7f4d0006e1b0 2024-04-25T10:14:24.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.488+0000 7f4d059f4640 1 -- 172.21.15.16:0/1182166812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d0006e1b0 msgr2=0x7f4d000b3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.488+0000 7f4d059f4640 1 --2- 172.21.15.16:0/1182166812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d0006e1b0 0x7f4d000b3040 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009870 tx=0x7f4ce802a100 comp rx=0 tx=0).stop 2024-04-25T10:14:24.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.489+0000 7f4d059f4640 1 -- 172.21.15.16:0/1182166812 shutdown_connections 2024-04-25T10:14:24.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.489+0000 7f4d059f4640 1 --2- 172.21.15.16:0/1182166812 >> v2:172.21.15.186:3300/0 conn(0x7f4d000b3580 0x7f4d001616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.489+0000 7f4d059f4640 1 --2- 172.21.15.16:0/1182166812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d0006e1b0 0x7f4d000b3040 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.489+0000 7f4d059f4640 1 --2- 172.21.15.16:0/1182166812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d0015ec10 0x7f4d0006dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.489+0000 7f4d059f4640 1 -- 172.21.15.16:0/1182166812 >> 172.21.15.16:0/1182166812 conn(0x7f4d000b2220 msgr2=0x7f4d000cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:24.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.489+0000 7f4d059f4640 1 -- 172.21.15.16:0/1182166812 shutdown_connections 2024-04-25T10:14:24.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.489+0000 7f4d059f4640 1 -- 172.21.15.16:0/1182166812 wait complete. 2024-04-25T10:14:24.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.490+0000 7f4d059f4640 1 Processor -- start 2024-04-25T10:14:24.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.490+0000 7f4d059f4640 1 -- start start 2024-04-25T10:14:24.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.490+0000 7f4d059f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d0006e1b0 0x7f4d000ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.490+0000 7f4d059f4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d000b3580 0x7f4d000ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.490+0000 7f4d059f4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d0015ec10 0x7f4d0013fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.490+0000 7f4d059f4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0015e2e0 con 0x7f4d0006e1b0 2024-04-25T10:14:24.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.490+0000 7f4d059f4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0015eff0 con 0x7f4d000b3580 2024-04-25T10:14:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.490+0000 7f4d059f4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0015e460 con 0x7f4d0015ec10 2024-04-25T10:14:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.491+0000 7f4cfeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d0006e1b0 0x7f4d000ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.491+0000 7f4cfeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d0006e1b0 0x7f4d000ae980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39036/0 (socket says 172.21.15.16:39036) 2024-04-25T10:14:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.491+0000 7f4cfeffd640 1 -- 172.21.15.16:0/976142557 learned_addr learned my addr 172.21.15.16:0/976142557 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.491+0000 7f4cff7fe640 1 --2- 172.21.15.16:0/976142557 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d0015ec10 0x7f4d0013fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.491+0000 7f4cf7fff640 1 --2- 172.21.15.16:0/976142557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d000b3580 0x7f4d000ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.491+0000 7f4cfeffd640 1 -- 172.21.15.16:0/976142557 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d0015ec10 msgr2=0x7f4d0013fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.491+0000 7f4cfeffd640 1 --2- 172.21.15.16:0/976142557 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d0015ec10 0x7f4d0013fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.491+0000 7f4cfeffd640 1 -- 172.21.15.16:0/976142557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d000b3580 msgr2=0x7f4d000ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.491+0000 7f4cfeffd640 1 --2- 172.21.15.16:0/976142557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d000b3580 0x7f4d000ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.491+0000 7f4cfeffd640 1 -- 172.21.15.16:0/976142557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d00140e30 con 0x7f4d0006e1b0 2024-04-25T10:14:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.491+0000 7f4cf7fff640 1 --2- 172.21.15.16:0/976142557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d000b3580 0x7f4d000ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.491+0000 7f4cff7fe640 1 --2- 172.21.15.16:0/976142557 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d0015ec10 0x7f4d0013fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.491+0000 7f4cfeffd640 1 --2- 172.21.15.16:0/976142557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d0006e1b0 0x7f4d000ae980 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000b4d0 tx=0x7f4cf000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.491+0000 7f4cfcff9640 1 -- 172.21.15.16:0/976142557 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0014070 con 0x7f4d0006e1b0 2024-04-25T10:14:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.491+0000 7f4cfcff9640 1 -- 172.21.15.16:0/976142557 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf0010870 con 0x7f4d0006e1b0 2024-04-25T10:14:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.491+0000 7f4d059f4640 1 -- 172.21.15.16:0/976142557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d00140360 con 0x7f4d0006e1b0 2024-04-25T10:14:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.491+0000 7f4cfcff9640 1 -- 172.21.15.16:0/976142557 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0010c60 con 0x7f4d0006e1b0 2024-04-25T10:14:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.491+0000 7f4d059f4640 1 -- 172.21.15.16:0/976142557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d000adbc0 con 0x7f4d0006e1b0 2024-04-25T10:14:24.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.492+0000 7f4d059f4640 1 -- 172.21.15.16:0/976142557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d00155bc0 con 0x7f4d0006e1b0 2024-04-25T10:14:24.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.493+0000 7f4cfcff9640 1 -- 172.21.15.16:0/976142557 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4cf0017070 con 0x7f4d0006e1b0 2024-04-25T10:14:24.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.493+0000 7f4cfcff9640 1 --2- 172.21.15.16:0/976142557 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4ce006b4d0 0x7f4ce006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.493+0000 7f4cfcff9640 1 -- 172.21.15.16:0/976142557 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4cf0089cd0 con 0x7f4d0006e1b0 2024-04-25T10:14:24.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.494+0000 7f4cf7fff640 1 --2- 172.21.15.16:0/976142557 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4ce006b4d0 0x7f4ce006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.494+0000 7f4cf7fff640 1 --2- 172.21.15.16:0/976142557 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4ce006b4d0 0x7f4ce006d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8034000 tx=0x7f4ce802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:24.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.498+0000 7f4cfcff9640 1 -- 172.21.15.16:0/976142557 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4d00155bc0 con 0x7f4d0006e1b0 2024-04-25T10:14:24.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.679+0000 7f4d059f4640 1 -- 172.21.15.16:0/976142557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4d000749b0 con 0x7f4d0006e1b0 2024-04-25T10:14:24.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.680+0000 7f4cfcff9640 1 -- 172.21.15.16:0/976142557 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v111) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4cf0058ea0 con 0x7f4d0006e1b0 2024-04-25T10:14:24.685 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:24.685 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-04-25T10:14:24.686 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-04-25T10:14:24.686 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-04-25T10:14:24.686 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-04-25T10:14:24.686 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-25T10:14:24.686 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-04-25T10:14:24.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.683+0000 7f4d059f4640 1 -- 172.21.15.16:0/976142557 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4ce006b4d0 msgr2=0x7f4ce006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.683+0000 7f4d059f4640 1 --2- 172.21.15.16:0/976142557 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4ce006b4d0 0x7f4ce006d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8034000 tx=0x7f4ce802e000 comp rx=0 tx=0).stop 2024-04-25T10:14:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.683+0000 7f4d059f4640 1 -- 172.21.15.16:0/976142557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d0006e1b0 msgr2=0x7f4d000ae980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.683+0000 7f4d059f4640 1 --2- 172.21.15.16:0/976142557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d0006e1b0 0x7f4d000ae980 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000b4d0 tx=0x7f4cf000b990 comp rx=0 tx=0).stop 2024-04-25T10:14:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.683+0000 7f4d059f4640 1 -- 172.21.15.16:0/976142557 shutdown_connections 2024-04-25T10:14:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.683+0000 7f4d059f4640 1 --2- 172.21.15.16:0/976142557 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4ce006b4d0 0x7f4ce006d990 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.683+0000 7f4d059f4640 1 --2- 172.21.15.16:0/976142557 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d0015ec10 0x7f4d0013fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.683+0000 7f4d059f4640 1 --2- 172.21.15.16:0/976142557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d000b3580 0x7f4d000ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.683+0000 7f4d059f4640 1 --2- 172.21.15.16:0/976142557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d0006e1b0 0x7f4d000ae980 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.683+0000 7f4d059f4640 1 -- 172.21.15.16:0/976142557 >> 172.21.15.16:0/976142557 conn(0x7f4d000b2220 msgr2=0x7f4d001554b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.683+0000 7f4d059f4640 1 -- 172.21.15.16:0/976142557 shutdown_connections 2024-04-25T10:14:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.683+0000 7f4d059f4640 1 -- 172.21.15.16:0/976142557 wait complete. 2024-04-25T10:14:24.698 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:14:24.698 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:14:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.797+0000 7fe7c7ff9640 1 Processor -- start 2024-04-25T10:14:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.797+0000 7fe7c7ff9640 1 -- start start 2024-04-25T10:14:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.798+0000 7fe7c7ff9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7c01610b0 0x7fe7c00b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.798+0000 7fe7c7ff9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7c01618d0 0x7fe7c00ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.798+0000 7fe7c7ff9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c0162040 con 0x7fe7c00ced70 2024-04-25T10:14:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.798+0000 7fe7c7ff9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c00d1ea0 con 0x7fe7c01618d0 2024-04-25T10:14:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.798+0000 7fe7c7ff9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c0162320 con 0x7fe7c01610b0 2024-04-25T10:14:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.798+0000 7fe7c5594640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7c01618d0 0x7fe7c00ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.798+0000 7fe7c5594640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7c01618d0 0x7fe7c00ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39050/0 (socket says 172.21.15.16:39050) 2024-04-25T10:14:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.798+0000 7fe7c5594640 1 -- 172.21.15.16:0/462342145 learned_addr learned my addr 172.21.15.16:0/462342145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.798+0000 7fe7c5d95640 1 --2- 172.21.15.16:0/462342145 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7c01610b0 0x7fe7c00b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.799+0000 7fe7c5594640 1 -- 172.21.15.16:0/462342145 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7c01610b0 msgr2=0x7fe7c00b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.799+0000 7fe7c5594640 1 --2- 172.21.15.16:0/462342145 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7c01610b0 0x7fe7c00b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.799+0000 7fe7c5594640 1 -- 172.21.15.16:0/462342145 >> v1:172.21.15.186:6789/0 conn(0x7fe7c00ced70 legacy=0x7fe7c006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.799+0000 7fe7c5594640 1 -- 172.21.15.16:0/462342145 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c0162600 con 0x7fe7c01618d0 2024-04-25T10:14:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.799+0000 7fe7c5d95640 1 --2- 172.21.15.16:0/462342145 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7c01610b0 0x7fe7c00b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.799+0000 7fe7c5594640 1 --2- 172.21.15.16:0/462342145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7c01618d0 0x7fe7c00ce830 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009870 tx=0x7fe7b002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ac0808ebd7978b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.799+0000 7fe7c4d93640 1 -- 172.21.15.16:0/462342145 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0037070 con 0x7fe7c01618d0 2024-04-25T10:14:24.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.799+0000 7fe7c4d93640 1 -- 172.21.15.16:0/462342145 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b0032440 con 0x7fe7c01618d0 2024-04-25T10:14:24.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.799+0000 7fe7c4d93640 1 -- 172.21.15.16:0/462342145 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0032810 con 0x7fe7c01618d0 2024-04-25T10:14:24.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.800+0000 7fe7c7ff9640 1 -- 172.21.15.16:0/462342145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7c01618d0 msgr2=0x7fe7c00ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.800+0000 7fe7c7ff9640 1 --2- 172.21.15.16:0/462342145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7c01618d0 0x7fe7c00ce830 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009870 tx=0x7fe7b002a180 comp rx=0 tx=0).stop 2024-04-25T10:14:24.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.800+0000 7fe7c7ff9640 1 -- 172.21.15.16:0/462342145 shutdown_connections 2024-04-25T10:14:24.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.800+0000 7fe7c7ff9640 1 --2- 172.21.15.16:0/462342145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7c01618d0 0x7fe7c00ce830 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.800+0000 7fe7c7ff9640 1 --2- 172.21.15.16:0/462342145 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7c01610b0 0x7fe7c00b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.800+0000 7fe7c7ff9640 1 -- 172.21.15.16:0/462342145 >> 172.21.15.16:0/462342145 conn(0x7fe7c00b5570 msgr2=0x7fe7c00b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.800+0000 7fe7c7ff9640 1 -- 172.21.15.16:0/462342145 shutdown_connections 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.800+0000 7fe7c7ff9640 1 -- 172.21.15.16:0/462342145 wait complete. 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.801+0000 7fe7c7ff9640 1 Processor -- start 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.801+0000 7fe7c7ff9640 1 -- start start 2024-04-25T10:14:24.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.802+0000 7fe7c7ff9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7c00ced70 0x7fe7c00cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.802+0000 7fe7c7ff9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7c01610b0 0x7fe7c00ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.802+0000 7fe7c7ff9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7c01618d0 0x7fe7c00cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.802+0000 7fe7c7ff9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c0161d10 con 0x7fe7c00ced70 2024-04-25T10:14:24.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.802+0000 7fe7c7ff9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c00b6c00 con 0x7fe7c01610b0 2024-04-25T10:14:24.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.802+0000 7fe7c7ff9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c00d3620 con 0x7fe7c01618d0 2024-04-25T10:14:24.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.802+0000 7fe7c5d95640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7c00ced70 0x7fe7c00cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.802+0000 7fe7c5d95640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7c00ced70 0x7fe7c00cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39058/0 (socket says 172.21.15.16:39058) 2024-04-25T10:14:24.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.802+0000 7fe7c5d95640 1 -- 172.21.15.16:0/4039447990 learned_addr learned my addr 172.21.15.16:0/4039447990 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:24.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.802+0000 7fe7c5594640 1 --2- 172.21.15.16:0/4039447990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7c01610b0 0x7fe7c00ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.802+0000 7fe7c6596640 1 --2- 172.21.15.16:0/4039447990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7c01618d0 0x7fe7c00cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.802+0000 7fe7c5d95640 1 -- 172.21.15.16:0/4039447990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7c01618d0 msgr2=0x7fe7c00cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.802+0000 7fe7c5d95640 1 --2- 172.21.15.16:0/4039447990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7c01618d0 0x7fe7c00cd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.802+0000 7fe7c5d95640 1 -- 172.21.15.16:0/4039447990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7c01610b0 msgr2=0x7fe7c00ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.802+0000 7fe7c5d95640 1 --2- 172.21.15.16:0/4039447990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7c01610b0 0x7fe7c00ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.802+0000 7fe7c5d95640 1 -- 172.21.15.16:0/4039447990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c0152700 con 0x7fe7c00ced70 2024-04-25T10:14:24.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.802+0000 7fe7c6596640 1 --2- 172.21.15.16:0/4039447990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7c01618d0 0x7fe7c00cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:24.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.803+0000 7fe7c5d95640 1 --2- 172.21.15.16:0/4039447990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7c00ced70 0x7fe7c00cc980 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800c7f0 tx=0x7fe7b800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:24.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.803+0000 7fe7b67fc640 1 -- 172.21.15.16:0/4039447990 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8015070 con 0x7fe7c00ced70 2024-04-25T10:14:24.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.803+0000 7fe7b67fc640 1 -- 172.21.15.16:0/4039447990 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b8007a30 con 0x7fe7c00ced70 2024-04-25T10:14:24.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.803+0000 7fe7c7ff9640 1 -- 172.21.15.16:0/4039447990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c01529e0 con 0x7fe7c00ced70 2024-04-25T10:14:24.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.803+0000 7fe7b67fc640 1 -- 172.21.15.16:0/4039447990 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8007e00 con 0x7fe7c00ced70 2024-04-25T10:14:24.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.803+0000 7fe7c7ff9640 1 -- 172.21.15.16:0/4039447990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c0152fa0 con 0x7fe7c00ced70 2024-04-25T10:14:24.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.804+0000 7fe7c7ff9640 1 -- 172.21.15.16:0/4039447990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c00cab10 con 0x7fe7c00ced70 2024-04-25T10:14:24.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.804+0000 7fe7b67fc640 1 -- 172.21.15.16:0/4039447990 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7b8018070 con 0x7fe7c00ced70 2024-04-25T10:14:24.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.805+0000 7fe7b67fc640 1 --2- 172.21.15.16:0/4039447990 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe7a406b4d0 0x7fe7a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.805+0000 7fe7b67fc640 1 -- 172.21.15.16:0/4039447990 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe7b8089e60 con 0x7fe7c00ced70 2024-04-25T10:14:24.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.805+0000 7fe7c5594640 1 --2- 172.21.15.16:0/4039447990 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe7a406b4d0 0x7fe7a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.806+0000 7fe7c5594640 1 --2- 172.21.15.16:0/4039447990 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe7a406b4d0 0x7fe7a406d990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0034000 tx=0x7fe7b002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:24.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.810+0000 7fe7b67fc640 1 -- 172.21.15.16:0/4039447990 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe7b8059030 con 0x7fe7c00ced70 2024-04-25T10:14:24.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:24.993+0000 7fe7c7ff9640 1 -- 172.21.15.16:0/4039447990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe7c00750b0 con 0x7fe7c00ced70 2024-04-25T10:14:25.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.372+0000 7fe7b67fc640 1 -- 172.21.15.16:0/4039447990 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v112) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe7b8058930 con 0x7fe7c00ced70 2024-04-25T10:14:25.377 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-04-25T10:14:25.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.375+0000 7fe7c7ff9640 1 -- 172.21.15.16:0/4039447990 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe7a406b4d0 msgr2=0x7fe7a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.375+0000 7fe7c7ff9640 1 --2- 172.21.15.16:0/4039447990 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe7a406b4d0 0x7fe7a406d990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0034000 tx=0x7fe7b002e000 comp rx=0 tx=0).stop 2024-04-25T10:14:25.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.375+0000 7fe7c7ff9640 1 -- 172.21.15.16:0/4039447990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7c00ced70 msgr2=0x7fe7c00cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.375+0000 7fe7c7ff9640 1 --2- 172.21.15.16:0/4039447990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7c00ced70 0x7fe7c00cc980 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800c7f0 tx=0x7fe7b800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.375+0000 7fe7c7ff9640 1 -- 172.21.15.16:0/4039447990 shutdown_connections 2024-04-25T10:14:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.375+0000 7fe7c7ff9640 1 --2- 172.21.15.16:0/4039447990 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe7a406b4d0 0x7fe7a406d990 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.375+0000 7fe7c7ff9640 1 --2- 172.21.15.16:0/4039447990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7c01618d0 0x7fe7c00cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.375+0000 7fe7c7ff9640 1 --2- 172.21.15.16:0/4039447990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7c01610b0 0x7fe7c00ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.375+0000 7fe7c7ff9640 1 --2- 172.21.15.16:0/4039447990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7c00ced70 0x7fe7c00cc980 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.375+0000 7fe7c7ff9640 1 -- 172.21.15.16:0/4039447990 >> 172.21.15.16:0/4039447990 conn(0x7fe7c00b5570 msgr2=0x7fe7c0156790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.375+0000 7fe7c7ff9640 1 -- 172.21.15.16:0/4039447990 shutdown_connections 2024-04-25T10:14:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.375+0000 7fe7c7ff9640 1 -- 172.21.15.16:0/4039447990 wait complete. 2024-04-25T10:14:25.394 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:14:25.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.501+0000 7f34abd89640 1 Processor -- start 2024-04-25T10:14:25.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.501+0000 7f34abd89640 1 -- start start 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.502+0000 7f34abd89640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34a40d03e0 0x7f34a40d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.502+0000 7f34abd89640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f34a415dd90 0x7f34a40d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.502+0000 7f34abd89640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a40d0d00 con 0x7f34a40d1a30 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.502+0000 7f34abd89640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a40d2410 con 0x7f34a415dd90 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.502+0000 7f34abd89640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a40d26f0 con 0x7f34a40d03e0 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.502+0000 7f34a9b25640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34a40d03e0 0x7f34a40d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.502+0000 7f34a9324640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f34a415dd90 0x7f34a40d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.502+0000 7f34aa326640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f34a40d1a30 0x7f34a40b5490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:45710/0 (socket says 172.21.15.16:45710) 2024-04-25T10:14:25.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.502+0000 7f34aa326640 1 -- 172.21.15.16:0/3903130159 learned_addr learned my addr 172.21.15.16:0/3903130159 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.503+0000 7f34a8b23640 1 -- 172.21.15.16:0/3903130159 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478145883 0 0) 0x7f34a40d0d00 con 0x7f34a40d1a30 2024-04-25T10:14:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.503+0000 7f34a8b23640 1 -- 172.21.15.16:0/3903130159 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3490003520 con 0x7f34a40d1a30 2024-04-25T10:14:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.503+0000 7f34a9324640 1 -- 172.21.15.16:0/3903130159 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34a40d03e0 msgr2=0x7f34a40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.503+0000 7f34a9324640 1 --2- 172.21.15.16:0/3903130159 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34a40d03e0 0x7f34a40d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.503+0000 7f34a9324640 1 -- 172.21.15.16:0/3903130159 >> v1:172.21.15.186:6789/0 conn(0x7f34a40d1a30 legacy=0x7f34a40b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.503+0000 7f34a9324640 1 -- 172.21.15.16:0/3903130159 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a40d29d0 con 0x7f34a415dd90 2024-04-25T10:14:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.503+0000 7f34a9b25640 1 --2- 172.21.15.16:0/3903130159 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34a40d03e0 0x7f34a40d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.503+0000 7f34a9324640 1 --2- 172.21.15.16:0/3903130159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34a415dd90 0x7f34a40d07c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3494009790 tx=0x7f349402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41fff53d8e7ca693 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.504+0000 7f34a8b23640 1 -- 172.21.15.16:0/3903130159 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3494037070 con 0x7f34a415dd90 2024-04-25T10:14:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.504+0000 7f34a8b23640 1 -- 172.21.15.16:0/3903130159 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3494032440 con 0x7f34a415dd90 2024-04-25T10:14:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.504+0000 7f34a8b23640 1 -- 172.21.15.16:0/3903130159 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3494032810 con 0x7f34a415dd90 2024-04-25T10:14:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.504+0000 7f34abd89640 1 -- 172.21.15.16:0/3903130159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34a415dd90 msgr2=0x7f34a40d07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.504+0000 7f34abd89640 1 --2- 172.21.15.16:0/3903130159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34a415dd90 0x7f34a40d07c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3494009790 tx=0x7f349402a210 comp rx=0 tx=0).stop 2024-04-25T10:14:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.504+0000 7f34abd89640 1 -- 172.21.15.16:0/3903130159 shutdown_connections 2024-04-25T10:14:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.504+0000 7f34abd89640 1 --2- 172.21.15.16:0/3903130159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34a415dd90 0x7f34a40d07c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.504+0000 7f34abd89640 1 --2- 172.21.15.16:0/3903130159 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34a40d03e0 0x7f34a40d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.504+0000 7f34abd89640 1 -- 172.21.15.16:0/3903130159 >> 172.21.15.16:0/3903130159 conn(0x7f34a4072900 msgr2=0x7f34a406c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.504+0000 7f34abd89640 1 -- 172.21.15.16:0/3903130159 shutdown_connections 2024-04-25T10:14:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.504+0000 7f34abd89640 1 -- 172.21.15.16:0/3903130159 wait complete. 2024-04-25T10:14:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.505+0000 7f34abd89640 1 Processor -- start 2024-04-25T10:14:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.505+0000 7f34abd89640 1 -- start start 2024-04-25T10:14:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.506+0000 7f34abd89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34a40d03e0 0x7f34a40b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.506+0000 7f34abd89640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34a40d1a30 0x7f34a414dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.506+0000 7f34abd89640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34a415dd90 0x7f34a41f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.506+0000 7f34abd89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a40b9820 con 0x7f34a40d03e0 2024-04-25T10:14:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.506+0000 7f34abd89640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a40d2f90 con 0x7f34a40d1a30 2024-04-25T10:14:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.506+0000 7f34abd89640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a40b99a0 con 0x7f34a415dd90 2024-04-25T10:14:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.506+0000 7f34a9b25640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34a40d03e0 0x7f34a40b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.506+0000 7f34a9b25640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34a40d03e0 0x7f34a40b31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39070/0 (socket says 172.21.15.16:39070) 2024-04-25T10:14:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.506+0000 7f34a9b25640 1 -- 172.21.15.16:0/2203890404 learned_addr learned my addr 172.21.15.16:0/2203890404 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.506+0000 7f34a9324640 1 --2- 172.21.15.16:0/2203890404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34a40d1a30 0x7f34a414dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.506+0000 7f34aa326640 1 --2- 172.21.15.16:0/2203890404 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34a415dd90 0x7f34a41f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.507+0000 7f34a9b25640 1 -- 172.21.15.16:0/2203890404 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34a415dd90 msgr2=0x7f34a41f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.507+0000 7f34a9b25640 1 --2- 172.21.15.16:0/2203890404 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34a415dd90 0x7f34a41f36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.507+0000 7f34a9b25640 1 -- 172.21.15.16:0/2203890404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34a40d1a30 msgr2=0x7f34a414dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.507+0000 7f34a9b25640 1 --2- 172.21.15.16:0/2203890404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34a40d1a30 0x7f34a414dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.507+0000 7f34a9b25640 1 -- 172.21.15.16:0/2203890404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a415f680 con 0x7f34a40d03e0 2024-04-25T10:14:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.507+0000 7f34a9324640 1 --2- 172.21.15.16:0/2203890404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34a40d1a30 0x7f34a414dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.507+0000 7f34aa326640 1 --2- 172.21.15.16:0/2203890404 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34a415dd90 0x7f34a41f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.507+0000 7f34a9b25640 1 --2- 172.21.15.16:0/2203890404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34a40d03e0 0x7f34a40b31b0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f34a000ec10 tx=0x7f34a000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.507+0000 7f349a7fc640 1 -- 172.21.15.16:0/2203890404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a0017070 con 0x7f34a40d03e0 2024-04-25T10:14:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.507+0000 7f349a7fc640 1 -- 172.21.15.16:0/2203890404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a000cc30 con 0x7f34a40d03e0 2024-04-25T10:14:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.507+0000 7f34abd89640 1 -- 172.21.15.16:0/2203890404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a415f960 con 0x7f34a40d03e0 2024-04-25T10:14:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.507+0000 7f349a7fc640 1 -- 172.21.15.16:0/2203890404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a0013470 con 0x7f34a40d03e0 2024-04-25T10:14:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.507+0000 7f34abd89640 1 -- 172.21.15.16:0/2203890404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a415ff20 con 0x7f34a40d03e0 2024-04-25T10:14:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.508+0000 7f34abd89640 1 -- 172.21.15.16:0/2203890404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a40749b0 con 0x7f34a40d03e0 2024-04-25T10:14:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.509+0000 7f349a7fc640 1 -- 172.21.15.16:0/2203890404 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34a001a070 con 0x7f34a40d03e0 2024-04-25T10:14:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.509+0000 7f349a7fc640 1 --2- 172.21.15.16:0/2203890404 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f348806b4d0 0x7f348806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.509+0000 7f349a7fc640 1 -- 172.21.15.16:0/2203890404 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f34a008a660 con 0x7f34a40d03e0 2024-04-25T10:14:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.510+0000 7f34a9324640 1 --2- 172.21.15.16:0/2203890404 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f348806b4d0 0x7f348806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.510+0000 7f34a9324640 1 --2- 172.21.15.16:0/2203890404 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f348806b4d0 0x7f348806d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3494034000 tx=0x7f349402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.513+0000 7f349a7fc640 1 -- 172.21.15.16:0/2203890404 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34a0059100 con 0x7f34a40d03e0 2024-04-25T10:14:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:25.695+0000 7f34abd89640 1 -- 172.21.15.16:0/2203890404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f34a40b28f0 con 0x7f34a40d03e0 2024-04-25T10:14:26.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.378+0000 7f349a7fc640 1 -- 172.21.15.16:0/2203890404 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v113) v1 ==== 170+0+0 (secure 0 0 0) 0x7f34a005d1a0 con 0x7f34a40d03e0 2024-04-25T10:14:26.383 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-04-25T10:14:26.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.382+0000 7f34abd89640 1 -- 172.21.15.16:0/2203890404 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f348806b4d0 msgr2=0x7f348806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.382+0000 7f34abd89640 1 --2- 172.21.15.16:0/2203890404 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f348806b4d0 0x7f348806d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3494034000 tx=0x7f349402e000 comp rx=0 tx=0).stop 2024-04-25T10:14:26.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.382+0000 7f34abd89640 1 -- 172.21.15.16:0/2203890404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34a40d03e0 msgr2=0x7f34a40b31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.382+0000 7f34abd89640 1 --2- 172.21.15.16:0/2203890404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34a40d03e0 0x7f34a40b31b0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f34a000ec10 tx=0x7f34a000c520 comp rx=0 tx=0).stop 2024-04-25T10:14:26.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.382+0000 7f34abd89640 1 -- 172.21.15.16:0/2203890404 shutdown_connections 2024-04-25T10:14:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.382+0000 7f34abd89640 1 --2- 172.21.15.16:0/2203890404 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f348806b4d0 0x7f348806d990 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.382+0000 7f34abd89640 1 --2- 172.21.15.16:0/2203890404 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34a415dd90 0x7f34a41f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.382+0000 7f34abd89640 1 --2- 172.21.15.16:0/2203890404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34a40d1a30 0x7f34a414dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.382+0000 7f34abd89640 1 --2- 172.21.15.16:0/2203890404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34a40d03e0 0x7f34a40b31b0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.382+0000 7f34abd89640 1 -- 172.21.15.16:0/2203890404 >> 172.21.15.16:0/2203890404 conn(0x7f34a4072900 msgr2=0x7f34a40b81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.382+0000 7f34abd89640 1 -- 172.21.15.16:0/2203890404 shutdown_connections 2024-04-25T10:14:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.383+0000 7f34abd89640 1 -- 172.21.15.16:0/2203890404 wait complete. 2024-04-25T10:14:26.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:14:26.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.519+0000 7fee2985d640 1 Processor -- start 2024-04-25T10:14:26.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.519+0000 7fee2985d640 1 -- start start 2024-04-25T10:14:26.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.519+0000 7fee2985d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee240d1130 0x7fee240b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.519+0000 7fee2985d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee240d4090 con 0x7fee240d3b90 2024-04-25T10:14:26.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.519+0000 7fee2985d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee240d4370 con 0x7fee2415dca0 2024-04-25T10:14:26.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.519+0000 7fee2985d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee240b6b20 con 0x7fee240d1130 2024-04-25T10:14:26.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee227fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fee240d3b90 0x7fee240d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43540/0 (socket says 172.21.15.16:43540) 2024-04-25T10:14:26.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee227fc640 1 -- 172.21.15.16:0/570851831 learned_addr learned my addr 172.21.15.16:0/570851831 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:26.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee237fe640 1 --2- 172.21.15.16:0/570851831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee240d1130 0x7fee240b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee21ffb640 1 -- 172.21.15.16:0/570851831 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112317187 0 0) 0x7fee240d4090 con 0x7fee240d3b90 2024-04-25T10:14:26.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee21ffb640 1 -- 172.21.15.16:0/570851831 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee0c003520 con 0x7fee240d3b90 2024-04-25T10:14:26.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee21ffb640 1 -- 172.21.15.16:0/570851831 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1236429994 0 0) 0x7fee0c003520 con 0x7fee240d3b90 2024-04-25T10:14:26.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee21ffb640 1 -- 172.21.15.16:0/570851831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee240d1130 msgr2=0x7fee240b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee21ffb640 1 --2- 172.21.15.16:0/570851831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee240d1130 0x7fee240b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee21ffb640 1 -- 172.21.15.16:0/570851831 >> v1:172.21.15.186:6789/0 conn(0x7fee2415dca0 legacy=0x7fee240ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee21ffb640 1 -- 172.21.15.16:0/570851831 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee240d4650 con 0x7fee240d3b90 2024-04-25T10:14:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee21ffb640 1 -- 172.21.15.16:0/570851831 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fee18001670 con 0x7fee240d3b90 2024-04-25T10:14:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee21ffb640 1 -- 172.21.15.16:0/570851831 >> v1:172.21.15.16:6789/0 conn(0x7fee240d3b90 legacy=0x7fee240d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee21ffb640 1 --2- 172.21.15.16:0/570851831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee0c0039e0 0x7fee0c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee21ffb640 1 --2- 172.21.15.16:0/570851831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee240d1130 0x7fee0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee21ffb640 1 --2- 172.21.15.16:0/570851831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee0c004d40 0x7fee0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee21ffb640 1 -- 172.21.15.16:0/570851831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c00b9c0 con 0x7fee0c0039e0 2024-04-25T10:14:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee21ffb640 1 -- 172.21.15.16:0/570851831 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c00bb70 con 0x7fee0c004d40 2024-04-25T10:14:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee21ffb640 1 -- 172.21.15.16:0/570851831 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c00bd70 con 0x7fee240d1130 2024-04-25T10:14:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee22ffd640 1 --2- 172.21.15.16:0/570851831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee0c0039e0 0x7fee0c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.520+0000 7fee227fc640 1 --2- 172.21.15.16:0/570851831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee240d1130 0x7fee0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.521+0000 7fee237fe640 1 --2- 172.21.15.16:0/570851831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee0c004d40 0x7fee0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.521+0000 7fee22ffd640 1 -- 172.21.15.16:0/570851831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee240d1130 msgr2=0x7fee0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.521+0000 7fee22ffd640 1 --2- 172.21.15.16:0/570851831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee240d1130 0x7fee0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.521+0000 7fee22ffd640 1 -- 172.21.15.16:0/570851831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee0c004d40 msgr2=0x7fee0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.521+0000 7fee22ffd640 1 --2- 172.21.15.16:0/570851831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee0c004d40 0x7fee0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.521+0000 7fee22ffd640 1 -- 172.21.15.16:0/570851831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c003520 con 0x7fee0c0039e0 2024-04-25T10:14:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.521+0000 7fee237fe640 1 --2- 172.21.15.16:0/570851831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee0c004d40 0x7fee0c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.521+0000 7fee22ffd640 1 --2- 172.21.15.16:0/570851831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee0c0039e0 0x7fee0c003dc0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fee14001860 tx=0x7fee0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c8c8b2b2bdf8d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.522+0000 7fee21ffb640 1 -- 172.21.15.16:0/570851831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee14037070 con 0x7fee0c0039e0 2024-04-25T10:14:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.522+0000 7fee21ffb640 1 -- 172.21.15.16:0/570851831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1402ab60 con 0x7fee0c0039e0 2024-04-25T10:14:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.522+0000 7fee21ffb640 1 -- 172.21.15.16:0/570851831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee14001430 con 0x7fee0c0039e0 2024-04-25T10:14:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.522+0000 7fee2985d640 1 -- 172.21.15.16:0/570851831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee0c0039e0 msgr2=0x7fee0c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.522+0000 7fee2985d640 1 --2- 172.21.15.16:0/570851831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee0c0039e0 0x7fee0c003dc0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fee14001860 tx=0x7fee0c005830 comp rx=0 tx=0).stop 2024-04-25T10:14:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.522+0000 7fee237fe640 1 -- 172.21.15.16:0/570851831 reap_dead start 2024-04-25T10:14:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.522+0000 7fee2985d640 1 -- 172.21.15.16:0/570851831 shutdown_connections 2024-04-25T10:14:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.522+0000 7fee2985d640 1 -- 172.21.15.16:0/570851831 >> 172.21.15.16:0/570851831 conn(0x7fee240726c0 msgr2=0x7fee24072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.522+0000 7fee2985d640 1 -- 172.21.15.16:0/570851831 shutdown_connections 2024-04-25T10:14:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.522+0000 7fee2985d640 1 -- 172.21.15.16:0/570851831 wait complete. 2024-04-25T10:14:26.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.523+0000 7fee2985d640 1 Processor -- start 2024-04-25T10:14:26.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.524+0000 7fee2985d640 1 -- start start 2024-04-25T10:14:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.524+0000 7fee2985d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee0c0039e0 0x7fee240cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.524+0000 7fee2985d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee241f6a30 0x7fee240ce500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.524+0000 7fee2985d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee241559b0 0x7fee24155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.524+0000 7fee2985d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2415e080 con 0x7fee241f6a30 2024-04-25T10:14:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.524+0000 7fee2985d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee240cf050 con 0x7fee0c0039e0 2024-04-25T10:14:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.524+0000 7fee2985d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2415e200 con 0x7fee241559b0 2024-04-25T10:14:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.524+0000 7fee227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee241f6a30 0x7fee240ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.524+0000 7fee227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee241f6a30 0x7fee240ce500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39076/0 (socket says 172.21.15.16:39076) 2024-04-25T10:14:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.524+0000 7fee227fc640 1 -- 172.21.15.16:0/2479801856 learned_addr learned my addr 172.21.15.16:0/2479801856 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.525+0000 7fee227fc640 1 -- 172.21.15.16:0/2479801856 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee241559b0 msgr2=0x7fee24155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.525+0000 7fee237fe640 1 --2- 172.21.15.16:0/2479801856 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee241559b0 0x7fee24155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.525+0000 7fee22ffd640 1 --2- 172.21.15.16:0/2479801856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee0c0039e0 0x7fee240cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.525+0000 7fee227fc640 1 --2- 172.21.15.16:0/2479801856 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee241559b0 0x7fee24155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.525+0000 7fee227fc640 1 -- 172.21.15.16:0/2479801856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee0c0039e0 msgr2=0x7fee240cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.525+0000 7fee227fc640 1 --2- 172.21.15.16:0/2479801856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee0c0039e0 0x7fee240cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.525+0000 7fee227fc640 1 -- 172.21.15.16:0/2479801856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee24153a00 con 0x7fee241f6a30 2024-04-25T10:14:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.525+0000 7fee22ffd640 1 --2- 172.21.15.16:0/2479801856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee0c0039e0 0x7fee240cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.525+0000 7fee237fe640 1 --2- 172.21.15.16:0/2479801856 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee241559b0 0x7fee24155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.525+0000 7fee227fc640 1 --2- 172.21.15.16:0/2479801856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee241f6a30 0x7fee240ce500 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fee18001050 tx=0x7fee180077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.525+0000 7fee0bfff640 1 -- 172.21.15.16:0/2479801856 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1800f960 con 0x7fee241f6a30 2024-04-25T10:14:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.525+0000 7fee0bfff640 1 -- 172.21.15.16:0/2479801856 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee18018070 con 0x7fee241f6a30 2024-04-25T10:14:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.525+0000 7fee2985d640 1 -- 172.21.15.16:0/2479801856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee24153ce0 con 0x7fee241f6a30 2024-04-25T10:14:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.525+0000 7fee0bfff640 1 -- 172.21.15.16:0/2479801856 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1800fd20 con 0x7fee241f6a30 2024-04-25T10:14:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.525+0000 7fee2985d640 1 -- 172.21.15.16:0/2479801856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee241563a0 con 0x7fee241f6a30 2024-04-25T10:14:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.527+0000 7fee2985d640 1 -- 172.21.15.16:0/2479801856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee240750b0 con 0x7fee241f6a30 2024-04-25T10:14:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.527+0000 7fee0bfff640 1 -- 172.21.15.16:0/2479801856 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee1801d460 con 0x7fee241f6a30 2024-04-25T10:14:26.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.527+0000 7fee0bfff640 1 --2- 172.21.15.16:0/2479801856 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fee1006b4d0 0x7fee1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.528+0000 7fee0bfff640 1 -- 172.21.15.16:0/2479801856 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fee1808a290 con 0x7fee241f6a30 2024-04-25T10:14:26.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.528+0000 7fee22ffd640 1 --2- 172.21.15.16:0/2479801856 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fee1006b4d0 0x7fee1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.528+0000 7fee22ffd640 1 --2- 172.21.15.16:0/2479801856 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fee1006b4d0 0x7fee1006d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fee14034000 tx=0x7fee14009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.532+0000 7fee0bfff640 1 -- 172.21.15.16:0/2479801856 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee18059200 con 0x7fee241f6a30 2024-04-25T10:14:26.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:26.749+0000 7fee2985d640 1 -- 172.21.15.16:0/2479801856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fee24160c90 con 0x7fee241f6a30 2024-04-25T10:14:27.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.380+0000 7fee0bfff640 1 -- 172.21.15.16:0/2479801856 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v82) v1 ==== 174+0+0 (secure 0 0 0) 0x7fee18058b00 con 0x7fee241f6a30 2024-04-25T10:14:27.384 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 16 and data pool 17 2024-04-25T10:14:27.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.384+0000 7fee2985d640 1 -- 172.21.15.16:0/2479801856 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fee1006b4d0 msgr2=0x7fee1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.384+0000 7fee2985d640 1 --2- 172.21.15.16:0/2479801856 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fee1006b4d0 0x7fee1006d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fee14034000 tx=0x7fee14009030 comp rx=0 tx=0).stop 2024-04-25T10:14:27.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.384+0000 7fee2985d640 1 -- 172.21.15.16:0/2479801856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee241f6a30 msgr2=0x7fee240ce500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.384+0000 7fee2985d640 1 --2- 172.21.15.16:0/2479801856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee241f6a30 0x7fee240ce500 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fee18001050 tx=0x7fee180077a0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.384+0000 7fee2985d640 1 -- 172.21.15.16:0/2479801856 shutdown_connections 2024-04-25T10:14:27.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.384+0000 7fee2985d640 1 --2- 172.21.15.16:0/2479801856 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fee1006b4d0 0x7fee1006d990 secure :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fee14034000 tx=0x7fee14009030 comp rx=0 tx=0).stop 2024-04-25T10:14:27.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.384+0000 7fee2985d640 1 --2- 172.21.15.16:0/2479801856 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fee241559b0 0x7fee24155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.384+0000 7fee2985d640 1 --2- 172.21.15.16:0/2479801856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee241f6a30 0x7fee240ce500 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.384+0000 7fee2985d640 1 --2- 172.21.15.16:0/2479801856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee0c0039e0 0x7fee240cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.384+0000 7fee2985d640 1 -- 172.21.15.16:0/2479801856 >> 172.21.15.16:0/2479801856 conn(0x7fee240726c0 msgr2=0x7fee2415bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.386+0000 7fee2985d640 1 -- 172.21.15.16:0/2479801856 shutdown_connections 2024-04-25T10:14:27.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.386+0000 7fee2985d640 1 -- 172.21.15.16:0/2479801856 wait complete. 2024-04-25T10:14:27.419 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:27.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.519+0000 7f0264df5640 1 Processor -- start 2024-04-25T10:14:27.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.519+0000 7f0264df5640 1 -- start start 2024-04-25T10:14:27.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.520+0000 7f0264df5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02600d35b0 0x7f02600b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.520+0000 7f0264df5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f02600b5990 0x7f02600b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.520+0000 7f0264df5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026015e0c0 con 0x7f026015dce0 2024-04-25T10:14:27.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.520+0000 7f0264df5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02600b5de0 con 0x7f02600b5990 2024-04-25T10:14:27.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.520+0000 7f0264df5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02600d2510 con 0x7f02600d35b0 2024-04-25T10:14:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.521+0000 7f025dd9b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02600d35b0 0x7f02600b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.521+0000 7f025ed9d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f02600b5990 0x7f02600b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.521+0000 7f025e59c640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f026015dce0 0x7f02600d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:45730/0 (socket says 172.21.15.16:45730) 2024-04-25T10:14:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.521+0000 7f025e59c640 1 -- 172.21.15.16:0/4015205214 learned_addr learned my addr 172.21.15.16:0/4015205214 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.521+0000 7f025d59a640 1 -- 172.21.15.16:0/4015205214 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364529771 0 0) 0x7f026015e0c0 con 0x7f026015dce0 2024-04-25T10:14:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.521+0000 7f025d59a640 1 -- 172.21.15.16:0/4015205214 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f024c003520 con 0x7f026015dce0 2024-04-25T10:14:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.521+0000 7f025dd9b640 1 -- 172.21.15.16:0/4015205214 >> v2:172.21.15.186:3300/0 conn(0x7f02600b5990 msgr2=0x7f02600b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.521+0000 7f025dd9b640 1 --2- 172.21.15.16:0/4015205214 >> v2:172.21.15.186:3300/0 conn(0x7f02600b5990 0x7f02600b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.521+0000 7f025dd9b640 1 -- 172.21.15.16:0/4015205214 >> v1:172.21.15.186:6789/0 conn(0x7f026015dce0 legacy=0x7f02600d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.521+0000 7f025dd9b640 1 -- 172.21.15.16:0/4015205214 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02600d27f0 con 0x7f02600d35b0 2024-04-25T10:14:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.521+0000 7f025ed9d640 1 --2- 172.21.15.16:0/4015205214 >> v2:172.21.15.186:3300/0 conn(0x7f02600b5990 0x7f02600b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.522+0000 7f025dd9b640 1 --2- 172.21.15.16:0/4015205214 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02600d35b0 0x7f02600b5450 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f02480097a0 tx=0x7f024802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1444629b5a3a69e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.522+0000 7f025d59a640 1 -- 172.21.15.16:0/4015205214 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248037070 con 0x7f02600d35b0 2024-04-25T10:14:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.522+0000 7f025d59a640 1 -- 172.21.15.16:0/4015205214 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0248032440 con 0x7f02600d35b0 2024-04-25T10:14:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.522+0000 7f025d59a640 1 -- 172.21.15.16:0/4015205214 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248032810 con 0x7f02600d35b0 2024-04-25T10:14:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.522+0000 7f0264df5640 1 -- 172.21.15.16:0/4015205214 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02600d35b0 msgr2=0x7f02600b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.522+0000 7f0264df5640 1 --2- 172.21.15.16:0/4015205214 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02600d35b0 0x7f02600b5450 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f02480097a0 tx=0x7f024802a180 comp rx=0 tx=0).stop 2024-04-25T10:14:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.523+0000 7f0264df5640 1 -- 172.21.15.16:0/4015205214 shutdown_connections 2024-04-25T10:14:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.523+0000 7f0264df5640 1 --2- 172.21.15.16:0/4015205214 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02600d35b0 0x7f02600b5450 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.523+0000 7f0264df5640 1 --2- 172.21.15.16:0/4015205214 >> v2:172.21.15.186:3300/0 conn(0x7f02600b5990 0x7f02600b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.523+0000 7f0264df5640 1 -- 172.21.15.16:0/4015205214 >> 172.21.15.16:0/4015205214 conn(0x7f0260072900 msgr2=0x7f02600d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.523+0000 7f0264df5640 1 -- 172.21.15.16:0/4015205214 shutdown_connections 2024-04-25T10:14:27.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.523+0000 7f0264df5640 1 -- 172.21.15.16:0/4015205214 wait complete. 2024-04-25T10:14:27.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.524+0000 7f0264df5640 1 Processor -- start 2024-04-25T10:14:27.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.524+0000 7f0264df5640 1 -- start start 2024-04-25T10:14:27.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.525+0000 7f0264df5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02600b5990 0x7f0260156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.525+0000 7f0264df5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02600d35b0 0x7f026014bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.525+0000 7f0264df5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f026015dce0 0x7f026015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.525+0000 7f0264df5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02600d3090 con 0x7f02600b5990 2024-04-25T10:14:27.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.525+0000 7f0264df5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02600d2db0 con 0x7f02600d35b0 2024-04-25T10:14:27.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.525+0000 7f0264df5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02600d2160 con 0x7f026015dce0 2024-04-25T10:14:27.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.525+0000 7f025e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02600b5990 0x7f0260156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.525+0000 7f025e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02600b5990 0x7f0260156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39080/0 (socket says 172.21.15.16:39080) 2024-04-25T10:14:27.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.525+0000 7f025e59c640 1 -- 172.21.15.16:0/2441326784 learned_addr learned my addr 172.21.15.16:0/2441326784 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:27.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.525+0000 7f025dd9b640 1 --2- 172.21.15.16:0/2441326784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02600d35b0 0x7f026014bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.525+0000 7f025ed9d640 1 --2- 172.21.15.16:0/2441326784 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f026015dce0 0x7f026015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.525+0000 7f025e59c640 1 -- 172.21.15.16:0/2441326784 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f026015dce0 msgr2=0x7f026015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.525+0000 7f025e59c640 1 --2- 172.21.15.16:0/2441326784 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f026015dce0 0x7f026015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.525+0000 7f025e59c640 1 -- 172.21.15.16:0/2441326784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02600d35b0 msgr2=0x7f026014bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.525+0000 7f025e59c640 1 --2- 172.21.15.16:0/2441326784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02600d35b0 0x7f026014bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.525+0000 7f025e59c640 1 -- 172.21.15.16:0/2441326784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02600cabd0 con 0x7f02600b5990 2024-04-25T10:14:27.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.525+0000 7f025e59c640 1 --2- 172.21.15.16:0/2441326784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02600b5990 0x7f0260156600 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f025000cad0 tx=0x7f025000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.526+0000 7f0256ffd640 1 -- 172.21.15.16:0/2441326784 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250017070 con 0x7f02600b5990 2024-04-25T10:14:27.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.526+0000 7f0256ffd640 1 -- 172.21.15.16:0/2441326784 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025000ac30 con 0x7f02600b5990 2024-04-25T10:14:27.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.526+0000 7f0264df5640 1 -- 172.21.15.16:0/2441326784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02600caeb0 con 0x7f02600b5990 2024-04-25T10:14:27.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.526+0000 7f0256ffd640 1 -- 172.21.15.16:0/2441326784 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250012400 con 0x7f02600b5990 2024-04-25T10:14:27.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.526+0000 7f0264df5640 1 -- 172.21.15.16:0/2441326784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02600cb470 con 0x7f02600b5990 2024-04-25T10:14:27.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.527+0000 7f0264df5640 1 -- 172.21.15.16:0/2441326784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02600749b0 con 0x7f02600b5990 2024-04-25T10:14:27.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.527+0000 7f0256ffd640 1 -- 172.21.15.16:0/2441326784 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f025001a070 con 0x7f02600b5990 2024-04-25T10:14:27.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.528+0000 7f0256ffd640 1 --2- 172.21.15.16:0/2441326784 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f024006b4d0 0x7f024006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.528+0000 7f0256ffd640 1 -- 172.21.15.16:0/2441326784 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0250089ed0 con 0x7f02600b5990 2024-04-25T10:14:27.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.528+0000 7f025dd9b640 1 --2- 172.21.15.16:0/2441326784 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f024006b4d0 0x7f024006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.528+0000 7f025dd9b640 1 --2- 172.21.15.16:0/2441326784 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f024006b4d0 0x7f024006d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0248034000 tx=0x7f024802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.532+0000 7f0256ffd640 1 -- 172.21.15.16:0/2441326784 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0250058790 con 0x7f02600b5990 2024-04-25T10:14:27.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.712+0000 7f0264df5640 1 -- 172.21.15.16:0/2441326784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02600d0420 con 0x7f02600b5990 2024-04-25T10:14:27.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.713+0000 7f0256ffd640 1 -- 172.21.15.16:0/2441326784 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+16696 (secure 0 0 0) 0x7f025005c830 con 0x7f02600b5990 2024-04-25T10:14:27.718 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:27.718 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":114,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:14:26.750759+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:25.025480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:25.726747+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.715+0000 7f0264df5640 1 -- 172.21.15.16:0/2441326784 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f024006b4d0 msgr2=0x7f024006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.715+0000 7f0264df5640 1 --2- 172.21.15.16:0/2441326784 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f024006b4d0 0x7f024006d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0248034000 tx=0x7f024802e000 comp rx=0 tx=0).stop 2024-04-25T10:14:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.715+0000 7f0264df5640 1 -- 172.21.15.16:0/2441326784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02600b5990 msgr2=0x7f0260156600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.715+0000 7f0264df5640 1 --2- 172.21.15.16:0/2441326784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02600b5990 0x7f0260156600 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f025000cad0 tx=0x7f025000cf90 comp rx=0 tx=0).stop 2024-04-25T10:14:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.715+0000 7f0264df5640 1 -- 172.21.15.16:0/2441326784 shutdown_connections 2024-04-25T10:14:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.715+0000 7f0264df5640 1 --2- 172.21.15.16:0/2441326784 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f024006b4d0 0x7f024006d990 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.715+0000 7f0264df5640 1 --2- 172.21.15.16:0/2441326784 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f026015dce0 0x7f026015ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.715+0000 7f0264df5640 1 --2- 172.21.15.16:0/2441326784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02600d35b0 0x7f026014bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.715+0000 7f0264df5640 1 --2- 172.21.15.16:0/2441326784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02600b5990 0x7f0260156600 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.715+0000 7f0264df5640 1 -- 172.21.15.16:0/2441326784 >> 172.21.15.16:0/2441326784 conn(0x7f0260072900 msgr2=0x7f02600d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.715+0000 7f0264df5640 1 -- 172.21.15.16:0/2441326784 shutdown_connections 2024-04-25T10:14:27.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.716+0000 7f0264df5640 1 -- 172.21.15.16:0/2441326784 wait complete. 2024-04-25T10:14:27.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:27.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.827+0000 7fcc1617d640 1 Processor -- start 2024-04-25T10:14:27.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.827+0000 7fcc1617d640 1 -- start start 2024-04-25T10:14:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.828+0000 7fcc1617d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc1015dce0 0x7fcc100d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.828+0000 7fcc1617d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fcc100d35b0 0x7fcc100b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.828+0000 7fcc1617d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc1015e0c0 con 0x7fcc100d1f90 2024-04-25T10:14:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.828+0000 7fcc1617d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc100b7b10 con 0x7fcc1015dce0 2024-04-25T10:14:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.828+0000 7fcc1617d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc100d2510 con 0x7fcc100d35b0 2024-04-25T10:14:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.828+0000 7fcc0f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc1015dce0 0x7fcc100d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.828+0000 7fcc0ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcc100d1f90 0x7fcc1006bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43550/0 (socket says 172.21.15.16:43550) 2024-04-25T10:14:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.828+0000 7fcc0f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc1015dce0 0x7fcc100d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39096/0 (socket says 172.21.15.16:39096) 2024-04-25T10:14:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.828+0000 7fcc0ffff640 1 -- 172.21.15.16:0/2151905909 learned_addr learned my addr 172.21.15.16:0/2151905909 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.829+0000 7fcc0effd640 1 --2- 172.21.15.16:0/2151905909 >> v2:172.21.15.186:3300/0 conn(0x7fcc100d35b0 0x7fcc100b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.829+0000 7fcc0f7fe640 1 -- 172.21.15.16:0/2151905909 >> v2:172.21.15.186:3300/0 conn(0x7fcc100d35b0 msgr2=0x7fcc100b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.829+0000 7fcc0f7fe640 1 --2- 172.21.15.16:0/2151905909 >> v2:172.21.15.186:3300/0 conn(0x7fcc100d35b0 0x7fcc100b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.829+0000 7fcc0f7fe640 1 -- 172.21.15.16:0/2151905909 >> v1:172.21.15.16:6789/0 conn(0x7fcc100d1f90 legacy=0x7fcc1006bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.829+0000 7fcc0f7fe640 1 -- 172.21.15.16:0/2151905909 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc100d27f0 con 0x7fcc1015dce0 2024-04-25T10:14:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.829+0000 7fcc0e7fc640 1 -- 172.21.15.16:0/2151905909 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577705375 0 0) 0x7fcc1015e0c0 con 0x7fcc100d1f90 2024-04-25T10:14:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.829+0000 7fcc0f7fe640 1 --2- 172.21.15.16:0/2151905909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1015dce0 0x7fcc100d1a50 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fcc100cdfa0 tx=0x7fcc0002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b4b12f30e794bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.830+0000 7fcc0e7fc640 1 -- 172.21.15.16:0/2151905909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00037070 con 0x7fcc1015dce0 2024-04-25T10:14:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.830+0000 7fcc0e7fc640 1 -- 172.21.15.16:0/2151905909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc00032440 con 0x7fcc1015dce0 2024-04-25T10:14:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.830+0000 7fcc0e7fc640 1 -- 172.21.15.16:0/2151905909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00032810 con 0x7fcc1015dce0 2024-04-25T10:14:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.830+0000 7fcc1617d640 1 -- 172.21.15.16:0/2151905909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1015dce0 msgr2=0x7fcc100d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.830+0000 7fcc1617d640 1 --2- 172.21.15.16:0/2151905909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1015dce0 0x7fcc100d1a50 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fcc100cdfa0 tx=0x7fcc0002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.830+0000 7fcc1617d640 1 -- 172.21.15.16:0/2151905909 shutdown_connections 2024-04-25T10:14:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.830+0000 7fcc1617d640 1 --2- 172.21.15.16:0/2151905909 >> v2:172.21.15.186:3300/0 conn(0x7fcc100d35b0 0x7fcc100b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.830+0000 7fcc1617d640 1 --2- 172.21.15.16:0/2151905909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1015dce0 0x7fcc100d1a50 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.830+0000 7fcc1617d640 1 -- 172.21.15.16:0/2151905909 >> 172.21.15.16:0/2151905909 conn(0x7fcc10072900 msgr2=0x7fcc100d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.830+0000 7fcc1617d640 1 -- 172.21.15.16:0/2151905909 shutdown_connections 2024-04-25T10:14:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.830+0000 7fcc1617d640 1 -- 172.21.15.16:0/2151905909 wait complete. 2024-04-25T10:14:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.831+0000 7fcc1617d640 1 Processor -- start 2024-04-25T10:14:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.832+0000 7fcc1617d640 1 -- start start 2024-04-25T10:14:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.832+0000 7fcc1617d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc100d1f90 0x7fcc101507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.832+0000 7fcc1617d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc100d35b0 0x7fcc100cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.832+0000 7fcc1617d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc1015dce0 0x7fcc10153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.832+0000 7fcc1617d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc100d3090 con 0x7fcc100d35b0 2024-04-25T10:14:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.832+0000 7fcc1617d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc100d2db0 con 0x7fcc1015dce0 2024-04-25T10:14:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.832+0000 7fcc1617d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc100b9820 con 0x7fcc100d1f90 2024-04-25T10:14:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.832+0000 7fcc0effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc100d35b0 0x7fcc100cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.832+0000 7fcc0effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc100d35b0 0x7fcc100cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39102/0 (socket says 172.21.15.16:39102) 2024-04-25T10:14:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.832+0000 7fcc0effd640 1 -- 172.21.15.16:0/2511930781 learned_addr learned my addr 172.21.15.16:0/2511930781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.832+0000 7fcc0f7fe640 1 --2- 172.21.15.16:0/2511930781 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc100d1f90 0x7fcc101507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.833+0000 7fcc0ffff640 1 --2- 172.21.15.16:0/2511930781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc1015dce0 0x7fcc10153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.833+0000 7fcc0effd640 1 -- 172.21.15.16:0/2511930781 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc100d1f90 msgr2=0x7fcc101507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.833+0000 7fcc0effd640 1 --2- 172.21.15.16:0/2511930781 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc100d1f90 0x7fcc101507c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.833+0000 7fcc0effd640 1 -- 172.21.15.16:0/2511930781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc1015dce0 msgr2=0x7fcc10153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.833+0000 7fcc0effd640 1 --2- 172.21.15.16:0/2511930781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc1015dce0 0x7fcc10153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.833+0000 7fcc0effd640 1 -- 172.21.15.16:0/2511930781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc100cc600 con 0x7fcc100d35b0 2024-04-25T10:14:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.833+0000 7fcc0f7fe640 1 --2- 172.21.15.16:0/2511930781 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc100d1f90 0x7fcc101507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.833+0000 7fcc0effd640 1 --2- 172.21.15.16:0/2511930781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc100d35b0 0x7fcc100cbc20 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400d7d0 tx=0x7fcc0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.833+0000 7fcbf7fff640 1 -- 172.21.15.16:0/2511930781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0400b6c0 con 0x7fcc100d35b0 2024-04-25T10:14:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.833+0000 7fcbf7fff640 1 -- 172.21.15.16:0/2511930781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc04019070 con 0x7fcc100d35b0 2024-04-25T10:14:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.833+0000 7fcc1617d640 1 -- 172.21.15.16:0/2511930781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc100cc8e0 con 0x7fcc100d35b0 2024-04-25T10:14:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.833+0000 7fcbf7fff640 1 -- 172.21.15.16:0/2511930781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0400bac0 con 0x7fcc100d35b0 2024-04-25T10:14:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.833+0000 7fcc1617d640 1 -- 172.21.15.16:0/2511930781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc100ccea0 con 0x7fcc100d35b0 2024-04-25T10:14:27.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.834+0000 7fcc1617d640 1 -- 172.21.15.16:0/2511930781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc100749b0 con 0x7fcc100d35b0 2024-04-25T10:14:27.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.835+0000 7fcbf7fff640 1 -- 172.21.15.16:0/2511930781 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc0400bc60 con 0x7fcc100d35b0 2024-04-25T10:14:27.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.835+0000 7fcbf7fff640 1 --2- 172.21.15.16:0/2511930781 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcbfc06b4d0 0x7fcbfc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.835+0000 7fcbf7fff640 1 -- 172.21.15.16:0/2511930781 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc0408a660 con 0x7fcc100d35b0 2024-04-25T10:14:27.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.835+0000 7fcc0f7fe640 1 --2- 172.21.15.16:0/2511930781 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcbfc06b4d0 0x7fcbfc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.836+0000 7fcc0f7fe640 1 --2- 172.21.15.16:0/2511930781 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcbfc06b4d0 0x7fcbfc06d990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fcc00034000 tx=0x7fcc0002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:27.839+0000 7fcbf7fff640 1 -- 172.21.15.16:0/2511930781 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc040594f0 con 0x7fcc100d35b0 2024-04-25T10:14:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.022+0000 7fcc1617d640 1 -- 172.21.15.16:0/2511930781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc1015d170 con 0x7fcc100d35b0 2024-04-25T10:14:28.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.023+0000 7fcbf7fff640 1 -- 172.21.15.16:0/2511930781 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+16696 (secure 0 0 0) 0x7fcc04058df0 con 0x7fcc100d35b0 2024-04-25T10:14:28.028 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:28.028 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":114,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:14:26.750759+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:25.025480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:25.726747+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:28.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.025+0000 7fcc1617d640 1 -- 172.21.15.16:0/2511930781 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcbfc06b4d0 msgr2=0x7fcbfc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.025+0000 7fcc1617d640 1 --2- 172.21.15.16:0/2511930781 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcbfc06b4d0 0x7fcbfc06d990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fcc00034000 tx=0x7fcc0002e000 comp rx=0 tx=0).stop 2024-04-25T10:14:28.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.025+0000 7fcc1617d640 1 -- 172.21.15.16:0/2511930781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc100d35b0 msgr2=0x7fcc100cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.025+0000 7fcc1617d640 1 --2- 172.21.15.16:0/2511930781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc100d35b0 0x7fcc100cbc20 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400d7d0 tx=0x7fcc0400dc90 comp rx=0 tx=0).stop 2024-04-25T10:14:28.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.025+0000 7fcc1617d640 1 -- 172.21.15.16:0/2511930781 shutdown_connections 2024-04-25T10:14:28.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.025+0000 7fcc1617d640 1 --2- 172.21.15.16:0/2511930781 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcbfc06b4d0 0x7fcbfc06d990 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.025+0000 7fcc1617d640 1 --2- 172.21.15.16:0/2511930781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc1015dce0 0x7fcc10153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.025+0000 7fcc1617d640 1 --2- 172.21.15.16:0/2511930781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc100d35b0 0x7fcc100cbc20 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.025+0000 7fcc1617d640 1 --2- 172.21.15.16:0/2511930781 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc100d1f90 0x7fcc101507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.025+0000 7fcc1617d640 1 -- 172.21.15.16:0/2511930781 >> 172.21.15.16:0/2511930781 conn(0x7fcc10072900 msgr2=0x7fcc100b8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:28.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.025+0000 7fcc1617d640 1 -- 172.21.15.16:0/2511930781 shutdown_connections 2024-04-25T10:14:28.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.025+0000 7fcc1617d640 1 -- 172.21.15.16:0/2511930781 wait complete. 2024-04-25T10:14:28.040 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:14:28.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.141+0000 7f6e5dc5d640 1 Processor -- start 2024-04-25T10:14:28.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.141+0000 7f6e5dc5d640 1 -- start start 2024-04-25T10:14:28.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.141+0000 7f6e5dc5d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e5815dca0 0x7f6e580ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.141+0000 7f6e5dc5d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6e580d3b90 0x7f6e580d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.141+0000 7f6e5dc5d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580cee80 0x7f6e580d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.141+0000 7f6e5dc5d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e580b6b20 con 0x7f6e5815dca0 2024-04-25T10:14:28.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.141+0000 7f6e5dc5d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e580d4090 con 0x7f6e580d3b90 2024-04-25T10:14:28.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.141+0000 7f6e5dc5d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580d4370 con 0x7f6e580cee80 2024-04-25T10:14:28.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.142+0000 7f6e577fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e5815dca0 0x7f6e580ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.142+0000 7f6e577fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e5815dca0 0x7f6e580ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39118/0 (socket says 172.21.15.16:39118) 2024-04-25T10:14:28.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.142+0000 7f6e577fe640 1 -- 172.21.15.16:0/4208012986 learned_addr learned my addr 172.21.15.16:0/4208012986 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:28.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.142+0000 7f6e577fe640 1 -- 172.21.15.16:0/4208012986 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580cee80 msgr2=0x7f6e580d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:28.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.142+0000 7f6e577fe640 1 --2- 172.21.15.16:0/4208012986 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580cee80 0x7f6e580d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.142+0000 7f6e56ffd640 1 --2- 172.21.15.16:0/4208012986 >> v2:172.21.15.186:3300/0 conn(0x7f6e580d3b90 0x7f6e580d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.142+0000 7f6e577fe640 1 -- 172.21.15.16:0/4208012986 >> v2:172.21.15.186:3300/0 conn(0x7f6e580d3b90 msgr2=0x7f6e580d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:28.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.142+0000 7f6e577fe640 1 --2- 172.21.15.16:0/4208012986 >> v2:172.21.15.186:3300/0 conn(0x7f6e580d3b90 0x7f6e580d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.142+0000 7f6e577fe640 1 -- 172.21.15.16:0/4208012986 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e580d4650 con 0x7f6e5815dca0 2024-04-25T10:14:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.143+0000 7f6e577fe640 1 --2- 172.21.15.16:0/4208012986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e5815dca0 0x7f6e580ce940 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f6e580b6560 tx=0x7f6e4802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad1e82e8a7f42e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.143+0000 7f6e567fc640 1 -- 172.21.15.16:0/4208012986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48037070 con 0x7f6e5815dca0 2024-04-25T10:14:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.143+0000 7f6e567fc640 1 -- 172.21.15.16:0/4208012986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e48032440 con 0x7f6e5815dca0 2024-04-25T10:14:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.143+0000 7f6e567fc640 1 -- 172.21.15.16:0/4208012986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48032810 con 0x7f6e5815dca0 2024-04-25T10:14:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.143+0000 7f6e5dc5d640 1 -- 172.21.15.16:0/4208012986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e5815dca0 msgr2=0x7f6e580ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.143+0000 7f6e5dc5d640 1 --2- 172.21.15.16:0/4208012986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e5815dca0 0x7f6e580ce940 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f6e580b6560 tx=0x7f6e4802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.143+0000 7f6e5dc5d640 1 -- 172.21.15.16:0/4208012986 shutdown_connections 2024-04-25T10:14:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.143+0000 7f6e5dc5d640 1 --2- 172.21.15.16:0/4208012986 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580cee80 0x7f6e580d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.143+0000 7f6e5dc5d640 1 --2- 172.21.15.16:0/4208012986 >> v2:172.21.15.186:3300/0 conn(0x7f6e580d3b90 0x7f6e580d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.143+0000 7f6e5dc5d640 1 --2- 172.21.15.16:0/4208012986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e5815dca0 0x7f6e580ce940 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.143+0000 7f6e5dc5d640 1 -- 172.21.15.16:0/4208012986 >> 172.21.15.16:0/4208012986 conn(0x7f6e580726c0 msgr2=0x7f6e58072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.143+0000 7f6e5dc5d640 1 -- 172.21.15.16:0/4208012986 shutdown_connections 2024-04-25T10:14:28.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.143+0000 7f6e5dc5d640 1 -- 172.21.15.16:0/4208012986 wait complete. 2024-04-25T10:14:28.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.145+0000 7f6e5dc5d640 1 Processor -- start 2024-04-25T10:14:28.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.145+0000 7f6e5dc5d640 1 -- start start 2024-04-25T10:14:28.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.145+0000 7f6e5dc5d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e580cee80 0x7f6e580cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.145+0000 7f6e5dc5d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580d3b90 0x7f6e5814df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.145+0000 7f6e5dc5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e5815dca0 0x7f6e58159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.145+0000 7f6e5dc5d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5815e200 con 0x7f6e5815dca0 2024-04-25T10:14:28.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.145+0000 7f6e5dc5d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5815e080 con 0x7f6e580cee80 2024-04-25T10:14:28.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.145+0000 7f6e5dc5d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580b2500 con 0x7f6e580d3b90 2024-04-25T10:14:28.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.146+0000 7f6e57fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e5815dca0 0x7f6e58159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.146+0000 7f6e57fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e5815dca0 0x7f6e58159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39130/0 (socket says 172.21.15.16:39130) 2024-04-25T10:14:28.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.146+0000 7f6e57fff640 1 -- 172.21.15.16:0/4290089696 learned_addr learned my addr 172.21.15.16:0/4290089696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:28.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.146+0000 7f6e56ffd640 1 --2- 172.21.15.16:0/4290089696 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580d3b90 0x7f6e5814df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.146+0000 7f6e577fe640 1 --2- 172.21.15.16:0/4290089696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e580cee80 0x7f6e580cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.146+0000 7f6e57fff640 1 -- 172.21.15.16:0/4290089696 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580d3b90 msgr2=0x7f6e5814df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.146+0000 7f6e57fff640 1 --2- 172.21.15.16:0/4290089696 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580d3b90 0x7f6e5814df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.146+0000 7f6e57fff640 1 -- 172.21.15.16:0/4290089696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e580cee80 msgr2=0x7f6e580cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.146+0000 7f6e57fff640 1 --2- 172.21.15.16:0/4290089696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e580cee80 0x7f6e580cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.146+0000 7f6e57fff640 1 -- 172.21.15.16:0/4290089696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e580cb0d0 con 0x7f6e5815dca0 2024-04-25T10:14:28.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.146+0000 7f6e56ffd640 1 --2- 172.21.15.16:0/4290089696 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580d3b90 0x7f6e5814df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:28.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.146+0000 7f6e57fff640 1 --2- 172.21.15.16:0/4290089696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e5815dca0 0x7f6e58159610 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000b9a0 tx=0x7f6e5000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.146+0000 7f6e3ffff640 1 -- 172.21.15.16:0/4290089696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50019070 con 0x7f6e5815dca0 2024-04-25T10:14:28.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.146+0000 7f6e3ffff640 1 -- 172.21.15.16:0/4290089696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5000cbc0 con 0x7f6e5815dca0 2024-04-25T10:14:28.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.146+0000 7f6e5dc5d640 1 -- 172.21.15.16:0/4290089696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e580cb3b0 con 0x7f6e5815dca0 2024-04-25T10:14:28.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.146+0000 7f6e3ffff640 1 -- 172.21.15.16:0/4290089696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50014440 con 0x7f6e5815dca0 2024-04-25T10:14:28.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.146+0000 7f6e5dc5d640 1 -- 172.21.15.16:0/4290089696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e580cb970 con 0x7f6e5815dca0 2024-04-25T10:14:28.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.147+0000 7f6e5dc5d640 1 -- 172.21.15.16:0/4290089696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e580750b0 con 0x7f6e5815dca0 2024-04-25T10:14:28.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.148+0000 7f6e3ffff640 1 -- 172.21.15.16:0/4290089696 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e5001c070 con 0x7f6e5815dca0 2024-04-25T10:14:28.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.148+0000 7f6e3ffff640 1 --2- 172.21.15.16:0/4290089696 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6e4406b500 0x7f6e4406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.148+0000 7f6e3ffff640 1 -- 172.21.15.16:0/4290089696 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e50089ed0 con 0x7f6e5815dca0 2024-04-25T10:14:28.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.148+0000 7f6e577fe640 1 --2- 172.21.15.16:0/4290089696 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6e4406b500 0x7f6e4406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.149+0000 7f6e577fe640 1 --2- 172.21.15.16:0/4290089696 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6e4406b500 0x7f6e4406d9c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6e48034000 tx=0x7f6e4802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.152+0000 7f6e3ffff640 1 -- 172.21.15.16:0/4290089696 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e50058d50 con 0x7f6e5815dca0 2024-04-25T10:14:28.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.379+0000 7f6e5dc5d640 1 -- 172.21.15.16:0/4290089696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6e580d0eb0 con 0x7f6e5815dca0 2024-04-25T10:14:28.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.396+0000 7f6e3ffff640 1 -- 172.21.15.16:0/4290089696 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v83) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6e50058650 con 0x7f6e5815dca0 2024-04-25T10:14:28.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.399+0000 7f6e5dc5d640 1 -- 172.21.15.16:0/4290089696 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6e4406b500 msgr2=0x7f6e4406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.399+0000 7f6e5dc5d640 1 --2- 172.21.15.16:0/4290089696 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6e4406b500 0x7f6e4406d9c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6e48034000 tx=0x7f6e4802e000 comp rx=0 tx=0).stop 2024-04-25T10:14:28.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.399+0000 7f6e5dc5d640 1 -- 172.21.15.16:0/4290089696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e5815dca0 msgr2=0x7f6e58159610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.399+0000 7f6e5dc5d640 1 --2- 172.21.15.16:0/4290089696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e5815dca0 0x7f6e58159610 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000b9a0 tx=0x7f6e5000be60 comp rx=0 tx=0).stop 2024-04-25T10:14:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.399+0000 7f6e5dc5d640 1 -- 172.21.15.16:0/4290089696 shutdown_connections 2024-04-25T10:14:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.399+0000 7f6e5dc5d640 1 --2- 172.21.15.16:0/4290089696 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6e4406b500 0x7f6e4406d9c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.399+0000 7f6e5dc5d640 1 --2- 172.21.15.16:0/4290089696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e5815dca0 0x7f6e58159610 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.399+0000 7f6e5dc5d640 1 --2- 172.21.15.16:0/4290089696 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e580d3b90 0x7f6e5814df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.399+0000 7f6e5dc5d640 1 --2- 172.21.15.16:0/4290089696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e580cee80 0x7f6e580cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.399+0000 7f6e5dc5d640 1 -- 172.21.15.16:0/4290089696 >> 172.21.15.16:0/4290089696 conn(0x7f6e580726c0 msgr2=0x7f6e580d2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.399+0000 7f6e5dc5d640 1 -- 172.21.15.16:0/4290089696 shutdown_connections 2024-04-25T10:14:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.399+0000 7f6e5dc5d640 1 -- 172.21.15.16:0/4290089696 wait complete. 2024-04-25T10:14:28.414 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:14:28.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:14:28.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.514+0000 7f6eb2e9e640 1 Processor -- start 2024-04-25T10:14:28.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.514+0000 7f6eb2e9e640 1 -- start start 2024-04-25T10:14:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.515+0000 7f6eb2e9e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6eac15ba70 0x7f6eac0d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.515+0000 7f6eb2e9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6eac0ce940 0x7f6eac0b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.515+0000 7f6eb2e9e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eac0cfcf0 con 0x7f6eac0b7030 2024-04-25T10:14:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.515+0000 7f6eb2e9e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eac0cffd0 con 0x7f6eac15ba70 2024-04-25T10:14:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.515+0000 7f6eb2e9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac0d02b0 con 0x7f6eac0ce940 2024-04-25T10:14:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.515+0000 7f6eabfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6eac0b7030 0x7f6eac0ce230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43554/0 (socket says 172.21.15.16:43554) 2024-04-25T10:14:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.515+0000 7f6eabfff640 1 -- 172.21.15.16:0/863017015 learned_addr learned my addr 172.21.15.16:0/863017015 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.515+0000 7f6eb143b640 1 --2- 172.21.15.16:0/863017015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6eac0ce940 0x7f6eac0b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.515+0000 7f6eb0c3a640 1 --2- 172.21.15.16:0/863017015 >> v2:172.21.15.186:3300/0 conn(0x7f6eac15ba70 0x7f6eac0d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.515+0000 7f6eab7fe640 1 -- 172.21.15.16:0/863017015 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503700824 0 0) 0x7f6eac0cfcf0 con 0x7f6eac0b7030 2024-04-25T10:14:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.515+0000 7f6eab7fe640 1 -- 172.21.15.16:0/863017015 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e98003520 con 0x7f6eac0b7030 2024-04-25T10:14:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eab7fe640 1 -- 172.21.15.16:0/863017015 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 211220340 0 0) 0x7f6e98003520 con 0x7f6eac0b7030 2024-04-25T10:14:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eab7fe640 1 -- 172.21.15.16:0/863017015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6eac0ce940 msgr2=0x7f6eac0b74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eab7fe640 1 --2- 172.21.15.16:0/863017015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6eac0ce940 0x7f6eac0b74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eab7fe640 1 -- 172.21.15.16:0/863017015 >> v2:172.21.15.186:3300/0 conn(0x7f6eac15ba70 msgr2=0x7f6eac0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eab7fe640 1 --2- 172.21.15.16:0/863017015 >> v2:172.21.15.186:3300/0 conn(0x7f6eac15ba70 0x7f6eac0d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eab7fe640 1 -- 172.21.15.16:0/863017015 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac0d0590 con 0x7f6eac0b7030 2024-04-25T10:14:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eab7fe640 1 -- 172.21.15.16:0/863017015 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f6e94001690 con 0x7f6eac0b7030 2024-04-25T10:14:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eab7fe640 1 -- 172.21.15.16:0/863017015 >> v1:172.21.15.16:6789/0 conn(0x7f6eac0b7030 legacy=0x7f6eac0ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eab7fe640 1 --2- 172.21.15.16:0/863017015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e980039b0 0x7f6e98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eab7fe640 1 --2- 172.21.15.16:0/863017015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e98004800 0x7f6e98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eab7fe640 1 --2- 172.21.15.16:0/863017015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6eac0ce940 0x7f6e980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eab7fe640 1 -- 172.21.15.16:0/863017015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac0cffd0 con 0x7f6e98004800 2024-04-25T10:14:28.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eab7fe640 1 -- 172.21.15.16:0/863017015 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac0d02b0 con 0x7f6e980039b0 2024-04-25T10:14:28.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eab7fe640 1 -- 172.21.15.16:0/863017015 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9800b9c0 con 0x7f6eac0ce940 2024-04-25T10:14:28.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eabfff640 1 --2- 172.21.15.16:0/863017015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e98004800 0x7f6e98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eb0c3a640 1 --2- 172.21.15.16:0/863017015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e980039b0 0x7f6e98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eb143b640 1 --2- 172.21.15.16:0/863017015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6eac0ce940 0x7f6e980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eabfff640 1 -- 172.21.15.16:0/863017015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6eac0ce940 msgr2=0x7f6e980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:28.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eabfff640 1 --2- 172.21.15.16:0/863017015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6eac0ce940 0x7f6e980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eabfff640 1 -- 172.21.15.16:0/863017015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e980039b0 msgr2=0x7f6e98003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:28.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eabfff640 1 --2- 172.21.15.16:0/863017015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e980039b0 0x7f6e98003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.516+0000 7f6eabfff640 1 -- 172.21.15.16:0/863017015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e98003520 con 0x7f6e98004800 2024-04-25T10:14:28.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.517+0000 7f6eabfff640 1 --2- 172.21.15.16:0/863017015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e98004800 0x7f6e98004c80 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400c240 tx=0x7f6e98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49e6db867f4395b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.517+0000 7f6eab7fe640 1 -- 172.21.15.16:0/863017015 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94007d70 con 0x7f6e98004800 2024-04-25T10:14:28.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.517+0000 7f6eab7fe640 1 -- 172.21.15.16:0/863017015 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e94005ad0 con 0x7f6e98004800 2024-04-25T10:14:28.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.517+0000 7f6eab7fe640 1 -- 172.21.15.16:0/863017015 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94007070 con 0x7f6e98004800 2024-04-25T10:14:28.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.518+0000 7f6eb2e9e640 1 -- 172.21.15.16:0/863017015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e98004800 msgr2=0x7f6e98004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.518+0000 7f6eb2e9e640 1 --2- 172.21.15.16:0/863017015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e98004800 0x7f6e98004c80 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400c240 tx=0x7f6e98005830 comp rx=0 tx=0).stop 2024-04-25T10:14:28.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.518+0000 7f6eb143b640 1 -- 172.21.15.16:0/863017015 reap_dead start 2024-04-25T10:14:28.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.518+0000 7f6eb2e9e640 1 -- 172.21.15.16:0/863017015 shutdown_connections 2024-04-25T10:14:28.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.518+0000 7f6eb2e9e640 1 -- 172.21.15.16:0/863017015 >> 172.21.15.16:0/863017015 conn(0x7f6eac06f4a0 msgr2=0x7f6eac06f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:28.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.518+0000 7f6eb2e9e640 1 -- 172.21.15.16:0/863017015 shutdown_connections 2024-04-25T10:14:28.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.518+0000 7f6eb2e9e640 1 -- 172.21.15.16:0/863017015 wait complete. 2024-04-25T10:14:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.519+0000 7f6eb2e9e640 1 Processor -- start 2024-04-25T10:14:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.519+0000 7f6eb2e9e640 1 -- start start 2024-04-25T10:14:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.520+0000 7f6eb2e9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e98004800 0x7f6eac0c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.520+0000 7f6eb2e9e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eac157750 0x7f6eac157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.520+0000 7f6eb2e9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6eac0adbb0 0x7f6eac0ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.520+0000 7f6eb2e9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac15be50 con 0x7f6e98004800 2024-04-25T10:14:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.520+0000 7f6eb2e9e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac0cf980 con 0x7f6eac157750 2024-04-25T10:14:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.520+0000 7f6eb2e9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac0ca420 con 0x7f6eac0adbb0 2024-04-25T10:14:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.520+0000 7f6eb0c3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e98004800 0x7f6eac0c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.520+0000 7f6eb0c3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e98004800 0x7f6eac0c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39156/0 (socket says 172.21.15.16:39156) 2024-04-25T10:14:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.520+0000 7f6eb0c3a640 1 -- 172.21.15.16:0/2981736740 learned_addr learned my addr 172.21.15.16:0/2981736740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.520+0000 7f6eabfff640 1 --2- 172.21.15.16:0/2981736740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eac157750 0x7f6eac157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.520+0000 7f6eb143b640 1 --2- 172.21.15.16:0/2981736740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6eac0adbb0 0x7f6eac0ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.520+0000 7f6eb0c3a640 1 -- 172.21.15.16:0/2981736740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6eac0adbb0 msgr2=0x7f6eac0ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.520+0000 7f6eb0c3a640 1 --2- 172.21.15.16:0/2981736740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6eac0adbb0 0x7f6eac0ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.520+0000 7f6eb0c3a640 1 -- 172.21.15.16:0/2981736740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eac157750 msgr2=0x7f6eac157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.520+0000 7f6eb0c3a640 1 --2- 172.21.15.16:0/2981736740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eac157750 0x7f6eac157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.520+0000 7f6eb0c3a640 1 -- 172.21.15.16:0/2981736740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac0c9470 con 0x7f6e98004800 2024-04-25T10:14:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.520+0000 7f6eabfff640 1 --2- 172.21.15.16:0/2981736740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eac157750 0x7f6eac157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.520+0000 7f6eb0c3a640 1 --2- 172.21.15.16:0/2981736740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e98004800 0x7f6eac0c9ee0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009f90 tx=0x7f6ea400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.521+0000 7f6ea97fa640 1 -- 172.21.15.16:0/2981736740 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea400fea0 con 0x7f6e98004800 2024-04-25T10:14:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.521+0000 7f6ea97fa640 1 -- 172.21.15.16:0/2981736740 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea4015630 con 0x7f6e98004800 2024-04-25T10:14:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.521+0000 7f6eb2e9e640 1 -- 172.21.15.16:0/2981736740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eac158260 con 0x7f6e98004800 2024-04-25T10:14:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.521+0000 7f6ea97fa640 1 -- 172.21.15.16:0/2981736740 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4015a20 con 0x7f6e98004800 2024-04-25T10:14:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.521+0000 7f6eb2e9e640 1 -- 172.21.15.16:0/2981736740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eac158540 con 0x7f6e98004800 2024-04-25T10:14:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.522+0000 7f6eb2e9e640 1 -- 172.21.15.16:0/2981736740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eac0c97f0 con 0x7f6e98004800 2024-04-25T10:14:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.522+0000 7f6ea97fa640 1 -- 172.21.15.16:0/2981736740 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ea4015bc0 con 0x7f6e98004800 2024-04-25T10:14:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.523+0000 7f6ea97fa640 1 --2- 172.21.15.16:0/2981736740 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6e8c06b4d0 0x7f6e8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.523+0000 7f6ea97fa640 1 -- 172.21.15.16:0/2981736740 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ea408b370 con 0x7f6e98004800 2024-04-25T10:14:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.523+0000 7f6eabfff640 1 --2- 172.21.15.16:0/2981736740 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6e8c06b4d0 0x7f6e8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.524+0000 7f6eabfff640 1 --2- 172.21.15.16:0/2981736740 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6e8c06b4d0 0x7f6e8c06d990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6e94033000 tx=0x7f6e9402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.527+0000 7f6ea97fa640 1 -- 172.21.15.16:0/2981736740 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ea4019030 con 0x7f6e98004800 2024-04-25T10:14:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:28.747+0000 7f6eb2e9e640 1 -- 172.21.15.16:0/2981736740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6eac0750b0 con 0x7f6e98004800 2024-04-25T10:14:29.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.401+0000 7f6ea97fa640 1 -- 172.21.15.16:0/2981736740 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v84) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6ea4010040 con 0x7f6e98004800 2024-04-25T10:14:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.403+0000 7f6eb2e9e640 1 -- 172.21.15.16:0/2981736740 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6e8c06b4d0 msgr2=0x7f6e8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.403+0000 7f6eb2e9e640 1 --2- 172.21.15.16:0/2981736740 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6e8c06b4d0 0x7f6e8c06d990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6e94033000 tx=0x7f6e9402e000 comp rx=0 tx=0).stop 2024-04-25T10:14:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.404+0000 7f6eb2e9e640 1 -- 172.21.15.16:0/2981736740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e98004800 msgr2=0x7f6eac0c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.404+0000 7f6eb2e9e640 1 --2- 172.21.15.16:0/2981736740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e98004800 0x7f6eac0c9ee0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009f90 tx=0x7f6ea400f580 comp rx=0 tx=0).stop 2024-04-25T10:14:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.404+0000 7f6eb2e9e640 1 -- 172.21.15.16:0/2981736740 shutdown_connections 2024-04-25T10:14:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.404+0000 7f6eb2e9e640 1 --2- 172.21.15.16:0/2981736740 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6e8c06b4d0 0x7f6e8c06d990 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.404+0000 7f6eb2e9e640 1 --2- 172.21.15.16:0/2981736740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6eac0adbb0 0x7f6eac0ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.404+0000 7f6eb2e9e640 1 --2- 172.21.15.16:0/2981736740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eac157750 0x7f6eac157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.404+0000 7f6eb2e9e640 1 --2- 172.21.15.16:0/2981736740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e98004800 0x7f6eac0c9ee0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.404+0000 7f6eb2e9e640 1 -- 172.21.15.16:0/2981736740 >> 172.21.15.16:0/2981736740 conn(0x7f6eac06f4a0 msgr2=0x7f6eac0b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:29.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.404+0000 7f6eb2e9e640 1 -- 172.21.15.16:0/2981736740 shutdown_connections 2024-04-25T10:14:29.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.404+0000 7f6eb2e9e640 1 -- 172.21.15.16:0/2981736740 wait complete. 2024-04-25T10:14:29.421 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:14:29.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:29.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.524+0000 7f767e970640 1 Processor -- start 2024-04-25T10:14:29.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.525+0000 7f767e970640 1 -- start start 2024-04-25T10:14:29.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.525+0000 7f767e970640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f767806cdd0 0x7f76780b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.525+0000 7f767e970640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767806c1f0 con 0x7f767815dc30 2024-04-25T10:14:29.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.525+0000 7f767e970640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76780b1dd0 con 0x7f76780b1960 2024-04-25T10:14:29.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.525+0000 7f767e970640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76780cfb40 con 0x7f767806cdd0 2024-04-25T10:14:29.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.525+0000 7f7677fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f767815dc30 0x7f767806c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43558/0 (socket says 172.21.15.16:43558) 2024-04-25T10:14:29.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.525+0000 7f7677fff640 1 -- 172.21.15.16:0/4169232108 learned_addr learned my addr 172.21.15.16:0/4169232108 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:29.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f7676ffd640 1 -- 172.21.15.16:0/4169232108 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960282658 0 0) 0x7f7668001110 con 0x7f767815dc30 2024-04-25T10:14:29.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f76777fe640 1 --2- 172.21.15.16:0/4169232108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f767806cdd0 0x7f76780b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f7676ffd640 1 -- 172.21.15.16:0/4169232108 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7660003520 con 0x7f767815dc30 2024-04-25T10:14:29.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f7676ffd640 1 -- 172.21.15.16:0/4169232108 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3521842215 0 0) 0x7f7660003520 con 0x7f767815dc30 2024-04-25T10:14:29.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f7676ffd640 1 -- 172.21.15.16:0/4169232108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f767806cdd0 msgr2=0x7f76780b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f7676ffd640 1 --2- 172.21.15.16:0/4169232108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f767806cdd0 0x7f76780b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f7676ffd640 1 -- 172.21.15.16:0/4169232108 >> v1:172.21.15.186:6789/0 conn(0x7f76780b1960 legacy=0x7f76780d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f7676ffd640 1 -- 172.21.15.16:0/4169232108 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76780b2060 con 0x7f767815dc30 2024-04-25T10:14:29.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f7676ffd640 1 -- 172.21.15.16:0/4169232108 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f7668001840 con 0x7f767815dc30 2024-04-25T10:14:29.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f7676ffd640 1 -- 172.21.15.16:0/4169232108 >> v1:172.21.15.16:6789/0 conn(0x7f767815dc30 legacy=0x7f767806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f7676ffd640 1 --2- 172.21.15.16:0/4169232108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76600039e0 0x7f7660003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f7676ffd640 1 --2- 172.21.15.16:0/4169232108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f767806cdd0 0x7f7660004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f7676ffd640 1 --2- 172.21.15.16:0/4169232108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7660004d40 0x7f76600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f7676ffd640 1 -- 172.21.15.16:0/4169232108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76780cfb40 con 0x7f76600039e0 2024-04-25T10:14:29.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f7676ffd640 1 -- 172.21.15.16:0/4169232108 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766000ba70 con 0x7f7660004d40 2024-04-25T10:14:29.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f7676ffd640 1 -- 172.21.15.16:0/4169232108 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766000bc70 con 0x7f767806cdd0 2024-04-25T10:14:29.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f7677fff640 1 --2- 172.21.15.16:0/4169232108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76600039e0 0x7f7660003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f76777fe640 1 --2- 172.21.15.16:0/4169232108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f767806cdd0 0x7f7660004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.526+0000 7f767cf0d640 1 --2- 172.21.15.16:0/4169232108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7660004d40 0x7f76600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.527+0000 7f7677fff640 1 -- 172.21.15.16:0/4169232108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f767806cdd0 msgr2=0x7f7660004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.527+0000 7f7677fff640 1 --2- 172.21.15.16:0/4169232108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f767806cdd0 0x7f7660004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.527+0000 7f7677fff640 1 -- 172.21.15.16:0/4169232108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7660004d40 msgr2=0x7f76600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.527+0000 7f7677fff640 1 --2- 172.21.15.16:0/4169232108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7660004d40 0x7f76600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.527+0000 7f7677fff640 1 -- 172.21.15.16:0/4169232108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7660003520 con 0x7f76600039e0 2024-04-25T10:14:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.527+0000 7f7677fff640 1 --2- 172.21.15.16:0/4169232108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76600039e0 0x7f7660003e00 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f766800c490 tx=0x7f7660005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9172cc83c9d90ae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.527+0000 7f7676ffd640 1 -- 172.21.15.16:0/4169232108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766800c0e0 con 0x7f76600039e0 2024-04-25T10:14:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.527+0000 7f7676ffd640 1 -- 172.21.15.16:0/4169232108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7668034070 con 0x7f76600039e0 2024-04-25T10:14:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.527+0000 7f7676ffd640 1 -- 172.21.15.16:0/4169232108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7668005d30 con 0x7f76600039e0 2024-04-25T10:14:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.528+0000 7f767e970640 1 -- 172.21.15.16:0/4169232108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76600039e0 msgr2=0x7f7660003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.528+0000 7f767e970640 1 --2- 172.21.15.16:0/4169232108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76600039e0 0x7f7660003e00 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f766800c490 tx=0x7f7660005830 comp rx=0 tx=0).stop 2024-04-25T10:14:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.528+0000 7f767cf0d640 1 -- 172.21.15.16:0/4169232108 reap_dead start 2024-04-25T10:14:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.528+0000 7f767e970640 1 -- 172.21.15.16:0/4169232108 shutdown_connections 2024-04-25T10:14:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.528+0000 7f767e970640 1 -- 172.21.15.16:0/4169232108 >> 172.21.15.16:0/4169232108 conn(0x7f76780d1900 msgr2=0x7f76780d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:29.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.528+0000 7f767e970640 1 -- 172.21.15.16:0/4169232108 shutdown_connections 2024-04-25T10:14:29.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.528+0000 7f767e970640 1 -- 172.21.15.16:0/4169232108 wait complete. 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.529+0000 7f767e970640 1 Processor -- start 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.530+0000 7f767e970640 1 -- start start 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.530+0000 7f767e970640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76600039e0 0x7f7678158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.530+0000 7f767e970640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76780cac70 0x7f76780b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.530+0000 7f767e970640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7678160760 0x7f7678160be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.530+0000 7f767e970640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76780cfde0 con 0x7f76600039e0 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.530+0000 7f767e970640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76780b2620 con 0x7f7678160760 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.530+0000 7f767e970640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767815e010 con 0x7f76780cac70 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.530+0000 7f7677fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76600039e0 0x7f7678158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.530+0000 7f7677fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76600039e0 0x7f7678158020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39176/0 (socket says 172.21.15.16:39176) 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.530+0000 7f7677fff640 1 -- 172.21.15.16:0/2248709262 learned_addr learned my addr 172.21.15.16:0/2248709262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.531+0000 7f76777fe640 1 --2- 172.21.15.16:0/2248709262 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76780cac70 0x7f76780b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.531+0000 7f7677fff640 1 -- 172.21.15.16:0/2248709262 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76780cac70 msgr2=0x7f76780b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.531+0000 7f767cf0d640 1 --2- 172.21.15.16:0/2248709262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7678160760 0x7f7678160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.531+0000 7f7677fff640 1 --2- 172.21.15.16:0/2248709262 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76780cac70 0x7f76780b44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.531+0000 7f7677fff640 1 -- 172.21.15.16:0/2248709262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7678160760 msgr2=0x7f7678160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.531+0000 7f7677fff640 1 --2- 172.21.15.16:0/2248709262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7678160760 0x7f7678160be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.531+0000 7f7677fff640 1 -- 172.21.15.16:0/2248709262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76780cb7d0 con 0x7f76600039e0 2024-04-25T10:14:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.531+0000 7f7677fff640 1 --2- 172.21.15.16:0/2248709262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76600039e0 0x7f7678158020 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f7668031040 tx=0x7f76780b2230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.531+0000 7f7674ff9640 1 -- 172.21.15.16:0/2248709262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7668004cb0 con 0x7f76600039e0 2024-04-25T10:14:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.531+0000 7f7674ff9640 1 -- 172.21.15.16:0/2248709262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7668004e50 con 0x7f76600039e0 2024-04-25T10:14:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.531+0000 7f7674ff9640 1 -- 172.21.15.16:0/2248709262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7668007290 con 0x7f76600039e0 2024-04-25T10:14:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.531+0000 7f767e970640 1 -- 172.21.15.16:0/2248709262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76780b4b80 con 0x7f76600039e0 2024-04-25T10:14:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.531+0000 7f767e970640 1 -- 172.21.15.16:0/2248709262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76780b5140 con 0x7f76600039e0 2024-04-25T10:14:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.532+0000 7f767e970640 1 -- 172.21.15.16:0/2248709262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7678067850 con 0x7f76600039e0 2024-04-25T10:14:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.533+0000 7f7674ff9640 1 -- 172.21.15.16:0/2248709262 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7668007430 con 0x7f76600039e0 2024-04-25T10:14:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.533+0000 7f7674ff9640 1 --2- 172.21.15.16:0/2248709262 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f766406b4d0 0x7f766406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.533+0000 7f7674ff9640 1 -- 172.21.15.16:0/2248709262 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76680aa500 con 0x7f76600039e0 2024-04-25T10:14:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.533+0000 7f76777fe640 1 --2- 172.21.15.16:0/2248709262 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f766406b4d0 0x7f766406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.534+0000 7f76777fe640 1 --2- 172.21.15.16:0/2248709262 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f766406b4d0 0x7f766406d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7678160520 tx=0x7f766c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.537+0000 7f7674ff9640 1 -- 172.21.15.16:0/2248709262 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7668049030 con 0x7f76600039e0 2024-04-25T10:14:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.755+0000 7f767e970640 1 -- 172.21.15.16:0/2248709262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76780749b0 con 0x7f76600039e0 2024-04-25T10:14:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.755+0000 7f7674ff9640 1 -- 172.21.15.16:0/2248709262 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f767806c1f0 con 0x7f76600039e0 2024-04-25T10:14:29.760 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:29.760 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2871061122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2871061122},{"type":"v1","addr":"172.21.15.186:6836","nonce":2871061122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1871507206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1871507206},{"type":"v1","addr":"172.21.15.186:6837","nonce":1871507206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4064412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4064412586},{"type":"v1","addr":"172.21.15.16:6834","nonce":4064412586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:26.751060+0000","modified":"2024-04-25T10:14:28.748797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8850":{"gid":8850,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/123396308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":123396308},{"type":"v1","addr":"172.21.15.16:6835","nonce":123396308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:29.760 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 84 2024-04-25T10:14:29.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.757+0000 7f767e970640 1 -- 172.21.15.16:0/2248709262 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f766406b4d0 msgr2=0x7f766406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.757+0000 7f767e970640 1 --2- 172.21.15.16:0/2248709262 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f766406b4d0 0x7f766406d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7678160520 tx=0x7f766c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:29.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.757+0000 7f767e970640 1 -- 172.21.15.16:0/2248709262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76600039e0 msgr2=0x7f7678158020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.757+0000 7f767e970640 1 --2- 172.21.15.16:0/2248709262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76600039e0 0x7f7678158020 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f7668031040 tx=0x7f76780b2230 comp rx=0 tx=0).stop 2024-04-25T10:14:29.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.757+0000 7f767e970640 1 -- 172.21.15.16:0/2248709262 shutdown_connections 2024-04-25T10:14:29.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.757+0000 7f767e970640 1 --2- 172.21.15.16:0/2248709262 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f766406b4d0 0x7f766406d990 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.757+0000 7f767e970640 1 --2- 172.21.15.16:0/2248709262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7678160760 0x7f7678160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.757+0000 7f767e970640 1 --2- 172.21.15.16:0/2248709262 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76780cac70 0x7f76780b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.757+0000 7f767e970640 1 --2- 172.21.15.16:0/2248709262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76600039e0 0x7f7678158020 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.757+0000 7f767e970640 1 -- 172.21.15.16:0/2248709262 >> 172.21.15.16:0/2248709262 conn(0x7f76780d1900 msgr2=0x7f76780d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:29.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.757+0000 7f767e970640 1 -- 172.21.15.16:0/2248709262 shutdown_connections 2024-04-25T10:14:29.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.757+0000 7f767e970640 1 -- 172.21.15.16:0/2248709262 wait complete. 2024-04-25T10:14:29.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.870+0000 7f145141f640 1 Processor -- start 2024-04-25T10:14:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.870+0000 7f145141f640 1 -- start start 2024-04-25T10:14:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.871+0000 7f145141f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144c0d35b0 0x7f144c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.871+0000 7f145141f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f144c0b5990 0x7f144c0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.871+0000 7f145141f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f144c15e0c0 con 0x7f144c15dce0 2024-04-25T10:14:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.871+0000 7f145141f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144c0b5de0 con 0x7f144c0b5990 2024-04-25T10:14:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.871+0000 7f145141f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144c0d2510 con 0x7f144c0d35b0 2024-04-25T10:14:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.871+0000 7f144affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f144c15dce0 0x7f144c0d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43560/0 (socket says 172.21.15.16:43560) 2024-04-25T10:14:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.871+0000 7f144affd640 1 -- 172.21.15.16:0/4180242074 learned_addr learned my addr 172.21.15.16:0/4180242074 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.871+0000 7f1443fff640 1 --2- 172.21.15.16:0/4180242074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144c0d35b0 0x7f144c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.871+0000 7f144b7fe640 1 --2- 172.21.15.16:0/4180242074 >> v2:172.21.15.186:3300/0 conn(0x7f144c0b5990 0x7f144c0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.871+0000 7f144a7fc640 1 -- 172.21.15.16:0/4180242074 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792258432 0 0) 0x7f144c15e0c0 con 0x7f144c15dce0 2024-04-25T10:14:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.871+0000 7f144a7fc640 1 -- 172.21.15.16:0/4180242074 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1438003520 con 0x7f144c15dce0 2024-04-25T10:14:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.871+0000 7f1443fff640 1 -- 172.21.15.16:0/4180242074 >> v2:172.21.15.186:3300/0 conn(0x7f144c0b5990 msgr2=0x7f144c0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.871+0000 7f1443fff640 1 --2- 172.21.15.16:0/4180242074 >> v2:172.21.15.186:3300/0 conn(0x7f144c0b5990 0x7f144c0b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.871+0000 7f1443fff640 1 -- 172.21.15.16:0/4180242074 >> v1:172.21.15.16:6789/0 conn(0x7f144c15dce0 legacy=0x7f144c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.871+0000 7f1443fff640 1 -- 172.21.15.16:0/4180242074 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144c0d27f0 con 0x7f144c0d35b0 2024-04-25T10:14:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.872+0000 7f1443fff640 1 --2- 172.21.15.16:0/4180242074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144c0d35b0 0x7f144c0b5450 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f14340097a0 tx=0x7f143402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9fa4c79177f760e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.872+0000 7f144a7fc640 1 -- 172.21.15.16:0/4180242074 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434037070 con 0x7f144c0d35b0 2024-04-25T10:14:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.872+0000 7f144a7fc640 1 -- 172.21.15.16:0/4180242074 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1434032440 con 0x7f144c0d35b0 2024-04-25T10:14:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.872+0000 7f144a7fc640 1 -- 172.21.15.16:0/4180242074 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14340327f0 con 0x7f144c0d35b0 2024-04-25T10:14:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.873+0000 7f145141f640 1 -- 172.21.15.16:0/4180242074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144c0d35b0 msgr2=0x7f144c0b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.873+0000 7f145141f640 1 --2- 172.21.15.16:0/4180242074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144c0d35b0 0x7f144c0b5450 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f14340097a0 tx=0x7f143402a180 comp rx=0 tx=0).stop 2024-04-25T10:14:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.873+0000 7f145141f640 1 -- 172.21.15.16:0/4180242074 shutdown_connections 2024-04-25T10:14:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.873+0000 7f145141f640 1 --2- 172.21.15.16:0/4180242074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144c0d35b0 0x7f144c0b5450 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.873+0000 7f145141f640 1 --2- 172.21.15.16:0/4180242074 >> v2:172.21.15.186:3300/0 conn(0x7f144c0b5990 0x7f144c0b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.873+0000 7f145141f640 1 -- 172.21.15.16:0/4180242074 >> 172.21.15.16:0/4180242074 conn(0x7f144c072900 msgr2=0x7f144c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.873+0000 7f145141f640 1 -- 172.21.15.16:0/4180242074 shutdown_connections 2024-04-25T10:14:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.873+0000 7f145141f640 1 -- 172.21.15.16:0/4180242074 wait complete. 2024-04-25T10:14:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.874+0000 7f145141f640 1 Processor -- start 2024-04-25T10:14:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.874+0000 7f145141f640 1 -- start start 2024-04-25T10:14:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.875+0000 7f145141f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144c0b5990 0x7f144c0cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.875+0000 7f145141f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144c0d35b0 0x7f144c15feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.875+0000 7f145141f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f144c15dce0 0x7f144c154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.875+0000 7f145141f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c0d3090 con 0x7f144c0b5990 2024-04-25T10:14:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.875+0000 7f145141f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c0d2db0 con 0x7f144c15dce0 2024-04-25T10:14:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.875+0000 7f145141f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144c0d2160 con 0x7f144c0d35b0 2024-04-25T10:14:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.875+0000 7f144affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144c0b5990 0x7f144c0cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.875+0000 7f144affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144c0b5990 0x7f144c0cd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39184/0 (socket says 172.21.15.16:39184) 2024-04-25T10:14:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.875+0000 7f144affd640 1 -- 172.21.15.16:0/3959796861 learned_addr learned my addr 172.21.15.16:0/3959796861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.875+0000 7f144b7fe640 1 --2- 172.21.15.16:0/3959796861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f144c15dce0 0x7f144c154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.875+0000 7f1443fff640 1 --2- 172.21.15.16:0/3959796861 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144c0d35b0 0x7f144c15feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.875+0000 7f144affd640 1 -- 172.21.15.16:0/3959796861 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144c0d35b0 msgr2=0x7f144c15feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.875+0000 7f144affd640 1 --2- 172.21.15.16:0/3959796861 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144c0d35b0 0x7f144c15feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.875+0000 7f144affd640 1 -- 172.21.15.16:0/3959796861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f144c15dce0 msgr2=0x7f144c154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.875+0000 7f144affd640 1 --2- 172.21.15.16:0/3959796861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f144c15dce0 0x7f144c154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.875+0000 7f144affd640 1 -- 172.21.15.16:0/3959796861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144c1605e0 con 0x7f144c0b5990 2024-04-25T10:14:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.875+0000 7f144b7fe640 1 --2- 172.21.15.16:0/3959796861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f144c15dce0 0x7f144c154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.875+0000 7f1443fff640 1 --2- 172.21.15.16:0/3959796861 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144c0d35b0 0x7f144c15feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.876+0000 7f144affd640 1 --2- 172.21.15.16:0/3959796861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144c0b5990 0x7f144c0cd460 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f143c00c390 tx=0x7f143c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.876+0000 7f14437fe640 1 -- 172.21.15.16:0/3959796861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c017070 con 0x7f144c0b5990 2024-04-25T10:14:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.876+0000 7f14437fe640 1 -- 172.21.15.16:0/3959796861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143c005ce0 con 0x7f144c0b5990 2024-04-25T10:14:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.876+0000 7f14437fe640 1 -- 172.21.15.16:0/3959796861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c0072f0 con 0x7f144c0b5990 2024-04-25T10:14:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.876+0000 7f145141f640 1 -- 172.21.15.16:0/3959796861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144c0c76d0 con 0x7f144c0b5990 2024-04-25T10:14:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.876+0000 7f145141f640 1 -- 172.21.15.16:0/3959796861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144c0c7c90 con 0x7f144c0b5990 2024-04-25T10:14:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.877+0000 7f145141f640 1 -- 172.21.15.16:0/3959796861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144c0749b0 con 0x7f144c0b5990 2024-04-25T10:14:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.877+0000 7f14437fe640 1 -- 172.21.15.16:0/3959796861 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f143c01a070 con 0x7f144c0b5990 2024-04-25T10:14:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.878+0000 7f14437fe640 1 --2- 172.21.15.16:0/3959796861 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f142c06b500 0x7f142c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.878+0000 7f14437fe640 1 -- 172.21.15.16:0/3959796861 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f143c0895e0 con 0x7f144c0b5990 2024-04-25T10:14:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.878+0000 7f1443fff640 1 --2- 172.21.15.16:0/3959796861 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f142c06b500 0x7f142c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.878+0000 7f1443fff640 1 --2- 172.21.15.16:0/3959796861 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f142c06b500 0x7f142c06d9c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1434034000 tx=0x7f143402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:29.882+0000 7f14437fe640 1 -- 172.21.15.16:0/3959796861 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f143c058450 con 0x7f144c0b5990 2024-04-25T10:14:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.065+0000 7f145141f640 1 -- 172.21.15.16:0/3959796861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f144c0c7e60 con 0x7f144c0b5990 2024-04-25T10:14:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.066+0000 7f14437fe640 1 -- 172.21.15.16:0/3959796861 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+16683 (secure 0 0 0) 0x7f143c057d50 con 0x7f144c0b5990 2024-04-25T10:14:30.071 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:30.071 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":115,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:14:28.393191+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:25.025480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:25.726747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.068+0000 7f145141f640 1 -- 172.21.15.16:0/3959796861 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f142c06b500 msgr2=0x7f142c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.068+0000 7f145141f640 1 --2- 172.21.15.16:0/3959796861 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f142c06b500 0x7f142c06d9c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1434034000 tx=0x7f143402e000 comp rx=0 tx=0).stop 2024-04-25T10:14:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.068+0000 7f145141f640 1 -- 172.21.15.16:0/3959796861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144c0b5990 msgr2=0x7f144c0cd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.068+0000 7f145141f640 1 --2- 172.21.15.16:0/3959796861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144c0b5990 0x7f144c0cd460 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f143c00c390 tx=0x7f143c00c850 comp rx=0 tx=0).stop 2024-04-25T10:14:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.068+0000 7f145141f640 1 -- 172.21.15.16:0/3959796861 shutdown_connections 2024-04-25T10:14:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.068+0000 7f145141f640 1 --2- 172.21.15.16:0/3959796861 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f142c06b500 0x7f142c06d9c0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.068+0000 7f145141f640 1 --2- 172.21.15.16:0/3959796861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f144c15dce0 0x7f144c154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.068+0000 7f145141f640 1 --2- 172.21.15.16:0/3959796861 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f144c0d35b0 0x7f144c15feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.068+0000 7f145141f640 1 --2- 172.21.15.16:0/3959796861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144c0b5990 0x7f144c0cd460 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.068+0000 7f145141f640 1 -- 172.21.15.16:0/3959796861 >> 172.21.15.16:0/3959796861 conn(0x7f144c072900 msgr2=0x7f144c0d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:30.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.069+0000 7f145141f640 1 -- 172.21.15.16:0/3959796861 shutdown_connections 2024-04-25T10:14:30.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.069+0000 7f145141f640 1 -- 172.21.15.16:0/3959796861 wait complete. 2024-04-25T10:14:30.084 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:14:30.084 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:14:30.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.181+0000 7f4118bc8640 1 Processor -- start 2024-04-25T10:14:30.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.181+0000 7f4118bc8640 1 -- start start 2024-04-25T10:14:30.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.182+0000 7f4118bc8640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f41140d35b0 0x7f41140b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.182+0000 7f4118bc8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f41140b5990 0x7f41140b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.182+0000 7f4118bc8640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411415e0c0 con 0x7f411415dce0 2024-04-25T10:14:30.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.182+0000 7f4118bc8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41140d2510 con 0x7f41140b5990 2024-04-25T10:14:30.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.182+0000 7f4118bc8640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41140d27f0 con 0x7f41140d35b0 2024-04-25T10:14:30.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.182+0000 7f4112d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f41140b5990 0x7f41140b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.182+0000 7f4112d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f41140b5990 0x7f41140b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39188/0 (socket says 172.21.15.16:39188) 2024-04-25T10:14:30.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.182+0000 7f4112d9d640 1 -- 172.21.15.16:0/104832785 learned_addr learned my addr 172.21.15.16:0/104832785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:30.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.182+0000 7f4111d9b640 1 --2- 172.21.15.16:0/104832785 >> v2:172.21.15.186:3300/0 conn(0x7f41140d35b0 0x7f41140b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.182+0000 7f4112d9d640 1 -- 172.21.15.16:0/104832785 >> v2:172.21.15.186:3300/0 conn(0x7f41140d35b0 msgr2=0x7f41140b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.182+0000 7f4112d9d640 1 --2- 172.21.15.16:0/104832785 >> v2:172.21.15.186:3300/0 conn(0x7f41140d35b0 0x7f41140b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.183+0000 7f4112d9d640 1 -- 172.21.15.16:0/104832785 >> v1:172.21.15.186:6789/0 conn(0x7f411415dce0 legacy=0x7f41140d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.183+0000 7f4112d9d640 1 -- 172.21.15.16:0/104832785 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41140d2ad0 con 0x7f41140b5990 2024-04-25T10:14:30.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.183+0000 7f4112d9d640 1 --2- 172.21.15.16:0/104832785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41140b5990 0x7f41140b6b00 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f41140cdfa0 tx=0x7f410c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ebb0c6e396e9507 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.183+0000 7f411159a640 1 -- 172.21.15.16:0/104832785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c036070 con 0x7f41140b5990 2024-04-25T10:14:30.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.183+0000 7f411159a640 1 -- 172.21.15.16:0/104832785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c031600 con 0x7f41140b5990 2024-04-25T10:14:30.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.183+0000 7f411159a640 1 -- 172.21.15.16:0/104832785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c0319d0 con 0x7f41140b5990 2024-04-25T10:14:30.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.184+0000 7f4118bc8640 1 -- 172.21.15.16:0/104832785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41140b5990 msgr2=0x7f41140b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.184+0000 7f4118bc8640 1 --2- 172.21.15.16:0/104832785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41140b5990 0x7f41140b6b00 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f41140cdfa0 tx=0x7f410c02a730 comp rx=0 tx=0).stop 2024-04-25T10:14:30.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.184+0000 7f4118bc8640 1 -- 172.21.15.16:0/104832785 shutdown_connections 2024-04-25T10:14:30.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.184+0000 7f4118bc8640 1 --2- 172.21.15.16:0/104832785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41140b5990 0x7f41140b6b00 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.184+0000 7f4118bc8640 1 --2- 172.21.15.16:0/104832785 >> v2:172.21.15.186:3300/0 conn(0x7f41140d35b0 0x7f41140b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.184+0000 7f4118bc8640 1 -- 172.21.15.16:0/104832785 >> 172.21.15.16:0/104832785 conn(0x7f4114072900 msgr2=0x7f41140d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:30.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.184+0000 7f4118bc8640 1 -- 172.21.15.16:0/104832785 shutdown_connections 2024-04-25T10:14:30.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.184+0000 7f4118bc8640 1 -- 172.21.15.16:0/104832785 wait complete. 2024-04-25T10:14:30.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.185+0000 7f4118bc8640 1 Processor -- start 2024-04-25T10:14:30.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.185+0000 7f4118bc8640 1 -- start start 2024-04-25T10:14:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.186+0000 7f4118bc8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41140b5990 0x7f41141f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.186+0000 7f4118bc8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41140d35b0 0x7f41140c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.186+0000 7f4118bc8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f411415dce0 0x7f41140ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.186+0000 7f4118bc8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41140d2160 con 0x7f41140b5990 2024-04-25T10:14:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.186+0000 7f4118bc8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41140d3090 con 0x7f41140d35b0 2024-04-25T10:14:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.186+0000 7f4118bc8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41140ca360 con 0x7f411415dce0 2024-04-25T10:14:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.186+0000 7f411259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41140b5990 0x7f41141f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.186+0000 7f411259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41140b5990 0x7f41141f3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39200/0 (socket says 172.21.15.16:39200) 2024-04-25T10:14:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.186+0000 7f411259c640 1 -- 172.21.15.16:0/1043848317 learned_addr learned my addr 172.21.15.16:0/1043848317 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.186+0000 7f4112d9d640 1 --2- 172.21.15.16:0/1043848317 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f411415dce0 0x7f41140ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.186+0000 7f4111d9b640 1 --2- 172.21.15.16:0/1043848317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41140d35b0 0x7f41140c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.187+0000 7f411259c640 1 -- 172.21.15.16:0/1043848317 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f411415dce0 msgr2=0x7f41140ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.187+0000 7f411259c640 1 --2- 172.21.15.16:0/1043848317 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f411415dce0 0x7f41140ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.187+0000 7f411259c640 1 -- 172.21.15.16:0/1043848317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41140d35b0 msgr2=0x7f41140c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.187+0000 7f411259c640 1 --2- 172.21.15.16:0/1043848317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41140d35b0 0x7f41140c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.187+0000 7f411259c640 1 -- 172.21.15.16:0/1043848317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411415e870 con 0x7f41140b5990 2024-04-25T10:14:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.187+0000 7f4111d9b640 1 --2- 172.21.15.16:0/1043848317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41140d35b0 0x7f41140c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.187+0000 7f411259c640 1 --2- 172.21.15.16:0/1043848317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41140b5990 0x7f41141f3f60 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f410400c3f0 tx=0x7f410400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.187+0000 7f4102ffd640 1 -- 172.21.15.16:0/1043848317 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41040052d0 con 0x7f41140b5990 2024-04-25T10:14:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.187+0000 7f4102ffd640 1 -- 172.21.15.16:0/1043848317 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4104005490 con 0x7f41140b5990 2024-04-25T10:14:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.187+0000 7f4118bc8640 1 -- 172.21.15.16:0/1043848317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411415eb50 con 0x7f41140b5990 2024-04-25T10:14:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.187+0000 7f4102ffd640 1 -- 172.21.15.16:0/1043848317 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104007260 con 0x7f41140b5990 2024-04-25T10:14:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.187+0000 7f4118bc8640 1 -- 172.21.15.16:0/1043848317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411415f110 con 0x7f41140b5990 2024-04-25T10:14:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.188+0000 7f4118bc8640 1 -- 172.21.15.16:0/1043848317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41140749b0 con 0x7f41140b5990 2024-04-25T10:14:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.189+0000 7f4102ffd640 1 -- 172.21.15.16:0/1043848317 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4104007400 con 0x7f41140b5990 2024-04-25T10:14:30.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.189+0000 7f4102ffd640 1 --2- 172.21.15.16:0/1043848317 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f40f406b4d0 0x7f40f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.189+0000 7f4102ffd640 1 -- 172.21.15.16:0/1043848317 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4104089630 con 0x7f41140b5990 2024-04-25T10:14:30.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.189+0000 7f4111d9b640 1 --2- 172.21.15.16:0/1043848317 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f40f406b4d0 0x7f40f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.190+0000 7f4111d9b640 1 --2- 172.21.15.16:0/1043848317 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f40f406b4d0 0x7f40f406d990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f41140b7ae0 tx=0x7f4108008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.193+0000 7f4102ffd640 1 -- 172.21.15.16:0/1043848317 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4104057e60 con 0x7f41140b5990 2024-04-25T10:14:30.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.369+0000 7f4118bc8640 1 -- 172.21.15.16:0/1043848317 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f411406b080 con 0x7f40f406b4d0 2024-04-25T10:14:30.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.375+0000 7f4102ffd640 1 -- 172.21.15.16:0/1043848317 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252415 (secure 0 0 0) 0x7f411406b080 con 0x7f40f406b4d0 2024-04-25T10:14:30.382 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:30.384 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:14:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.378+0000 7f4118bc8640 1 -- 172.21.15.16:0/1043848317 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f40f406b4d0 msgr2=0x7f40f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.378+0000 7f4118bc8640 1 --2- 172.21.15.16:0/1043848317 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f40f406b4d0 0x7f40f406d990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f41140b7ae0 tx=0x7f4108008040 comp rx=0 tx=0).stop 2024-04-25T10:14:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.378+0000 7f4118bc8640 1 -- 172.21.15.16:0/1043848317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41140b5990 msgr2=0x7f41141f3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.378+0000 7f4118bc8640 1 --2- 172.21.15.16:0/1043848317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41140b5990 0x7f41141f3f60 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f410400c3f0 tx=0x7f410400c8b0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.378+0000 7f4118bc8640 1 -- 172.21.15.16:0/1043848317 shutdown_connections 2024-04-25T10:14:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.378+0000 7f4118bc8640 1 --2- 172.21.15.16:0/1043848317 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f40f406b4d0 0x7f40f406d990 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.378+0000 7f4118bc8640 1 --2- 172.21.15.16:0/1043848317 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f411415dce0 0x7f41140ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.378+0000 7f4118bc8640 1 --2- 172.21.15.16:0/1043848317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41140d35b0 0x7f41140c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.378+0000 7f4118bc8640 1 --2- 172.21.15.16:0/1043848317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41140b5990 0x7f41141f3f60 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.378+0000 7f4118bc8640 1 -- 172.21.15.16:0/1043848317 >> 172.21.15.16:0/1043848317 conn(0x7f4114072900 msgr2=0x7f41140d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.378+0000 7f4118bc8640 1 -- 172.21.15.16:0/1043848317 shutdown_connections 2024-04-25T10:14:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.378+0000 7f4118bc8640 1 -- 172.21.15.16:0/1043848317 wait complete. 2024-04-25T10:14:30.393 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-04-25T10:14:29.919313+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":274112,"kb_used_data":12148,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754700608,"statfs":{"total":773094113280,"available":772813422592,"internally_reserved":0,"allocated":12439552,"data_stored":6040032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.539548"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.407209+0000","last_change":"2024-04-25T10:14:27.400799+0000","last_active":"2024-04-25T10:14:27.407209+0000","last_peered":"2024-04-25T10:14:27.407209+0000","last_clean":"2024-04-25T10:14:27.407209+0000","last_became_active":"2024-04-25T10:14:27.400409+0000","last_became_peered":"2024-04-25T10:14:27.400409+0000","last_unstale":"2024-04-25T10:14:27.407209+0000","last_undegraded":"2024-04-25T10:14:27.407209+0000","last_fullsized":"2024-04-25T10:14:27.407209+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:45.905737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.411635+0000","last_change":"2024-04-25T10:14:27.408444+0000","last_active":"2024-04-25T10:14:27.411635+0000","last_peered":"2024-04-25T10:14:27.411635+0000","last_clean":"2024-04-25T10:14:27.411635+0000","last_became_active":"2024-04-25T10:14:27.408163+0000","last_became_peered":"2024-04-25T10:14:27.408163+0000","last_unstale":"2024-04-25T10:14:27.411635+0000","last_undegraded":"2024-04-25T10:14:27.411635+0000","last_fullsized":"2024-04-25T10:14:27.411635+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:36.716791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408446+0000","last_change":"2024-04-25T10:14:27.402639+0000","last_active":"2024-04-25T10:14:27.408446+0000","last_peered":"2024-04-25T10:14:27.408446+0000","last_clean":"2024-04-25T10:14:27.408446+0000","last_became_active":"2024-04-25T10:14:27.402387+0000","last_became_peered":"2024-04-25T10:14:27.402387+0000","last_unstale":"2024-04-25T10:14:27.408446+0000","last_undegraded":"2024-04-25T10:14:27.408446+0000","last_fullsized":"2024-04-25T10:14:27.408446+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:56.837933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.396299+0000","last_change":"2024-04-25T10:14:27.393304+0000","last_active":"2024-04-25T10:14:27.396299+0000","last_peered":"2024-04-25T10:14:27.396299+0000","last_clean":"2024-04-25T10:14:27.396299+0000","last_became_active":"2024-04-25T10:14:27.393003+0000","last_became_peered":"2024-04-25T10:14:27.393003+0000","last_unstale":"2024-04-25T10:14:27.396299+0000","last_undegraded":"2024-04-25T10:14:27.396299+0000","last_fullsized":"2024-04-25T10:14:27.396299+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:45.211167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.409647+0000","last_change":"2024-04-25T10:14:27.407044+0000","last_active":"2024-04-25T10:14:27.409647+0000","last_peered":"2024-04-25T10:14:27.409647+0000","last_clean":"2024-04-25T10:14:27.409647+0000","last_became_active":"2024-04-25T10:14:27.405699+0000","last_became_peered":"2024-04-25T10:14:27.405699+0000","last_unstale":"2024-04-25T10:14:27.409647+0000","last_undegraded":"2024-04-25T10:14:27.409647+0000","last_fullsized":"2024-04-25T10:14:27.409647+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:18.842796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.264756+0000","last_change":"2024-04-25T10:14:27.403414+0000","last_active":"2024-04-25T10:14:29.264756+0000","last_peered":"2024-04-25T10:14:29.264756+0000","last_clean":"2024-04-25T10:14:29.264756+0000","last_became_active":"2024-04-25T10:14:27.403093+0000","last_became_peered":"2024-04-25T10:14:27.403093+0000","last_unstale":"2024-04-25T10:14:29.264756+0000","last_undegraded":"2024-04-25T10:14:29.264756+0000","last_fullsized":"2024-04-25T10:14:29.264756+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:09.915978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.404949+0000","last_change":"2024-04-25T10:14:27.403269+0000","last_active":"2024-04-25T10:14:27.404949+0000","last_peered":"2024-04-25T10:14:27.404949+0000","last_clean":"2024-04-25T10:14:27.404949+0000","last_became_active":"2024-04-25T10:14:27.402988+0000","last_became_peered":"2024-04-25T10:14:27.402988+0000","last_unstale":"2024-04-25T10:14:27.404949+0000","last_undegraded":"2024-04-25T10:14:27.404949+0000","last_fullsized":"2024-04-25T10:14:27.404949+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:10.557283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.409848+0000","last_change":"2024-04-25T10:14:27.408306+0000","last_active":"2024-04-25T10:14:27.409848+0000","last_peered":"2024-04-25T10:14:27.409848+0000","last_clean":"2024-04-25T10:14:27.409848+0000","last_became_active":"2024-04-25T10:14:27.408011+0000","last_became_peered":"2024-04-25T10:14:27.408011+0000","last_unstale":"2024-04-25T10:14:27.409848+0000","last_undegraded":"2024-04-25T10:14:27.409848+0000","last_fullsized":"2024-04-25T10:14:27.409848+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:22.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.407952+0000","last_change":"2024-04-25T10:14:27.406151+0000","last_active":"2024-04-25T10:14:27.407952+0000","last_peered":"2024-04-25T10:14:27.407952+0000","last_clean":"2024-04-25T10:14:27.407952+0000","last_became_active":"2024-04-25T10:14:27.405897+0000","last_became_peered":"2024-04-25T10:14:27.405897+0000","last_unstale":"2024-04-25T10:14:27.407952+0000","last_undegraded":"2024-04-25T10:14:27.407952+0000","last_fullsized":"2024-04-25T10:14:27.407952+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:17.554024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.401731+0000","last_change":"2024-04-25T10:14:27.397952+0000","last_active":"2024-04-25T10:14:27.401731+0000","last_peered":"2024-04-25T10:14:27.401731+0000","last_clean":"2024-04-25T10:14:27.401731+0000","last_became_active":"2024-04-25T10:14:27.397657+0000","last_became_peered":"2024-04-25T10:14:27.397657+0000","last_unstale":"2024-04-25T10:14:27.401731+0000","last_undegraded":"2024-04-25T10:14:27.401731+0000","last_fullsized":"2024-04-25T10:14:27.401731+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:33.637080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.404226+0000","last_change":"2024-04-25T10:14:27.397757+0000","last_active":"2024-04-25T10:14:27.404226+0000","last_peered":"2024-04-25T10:14:27.404226+0000","last_clean":"2024-04-25T10:14:27.404226+0000","last_became_active":"2024-04-25T10:14:27.397495+0000","last_became_peered":"2024-04-25T10:14:27.397495+0000","last_unstale":"2024-04-25T10:14:27.404226+0000","last_undegraded":"2024-04-25T10:14:27.404226+0000","last_fullsized":"2024-04-25T10:14:27.404226+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:58.449267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.398264+0000","last_change":"2024-04-25T10:14:27.397378+0000","last_active":"2024-04-25T10:14:27.398264+0000","last_peered":"2024-04-25T10:14:27.398264+0000","last_clean":"2024-04-25T10:14:27.398264+0000","last_became_active":"2024-04-25T10:14:27.397088+0000","last_became_peered":"2024-04-25T10:14:27.397088+0000","last_unstale":"2024-04-25T10:14:27.398264+0000","last_undegraded":"2024-04-25T10:14:27.398264+0000","last_fullsized":"2024-04-25T10:14:27.398264+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:06.765935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.401974+0000","last_change":"2024-04-25T10:14:27.394831+0000","last_active":"2024-04-25T10:14:27.401974+0000","last_peered":"2024-04-25T10:14:27.401974+0000","last_clean":"2024-04-25T10:14:27.401974+0000","last_became_active":"2024-04-25T10:14:27.393997+0000","last_became_peered":"2024-04-25T10:14:27.393997+0000","last_unstale":"2024-04-25T10:14:27.401974+0000","last_undegraded":"2024-04-25T10:14:27.401974+0000","last_fullsized":"2024-04-25T10:14:27.401974+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:42.494044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.405827+0000","last_change":"2024-04-25T10:14:27.398094+0000","last_active":"2024-04-25T10:14:27.405827+0000","last_peered":"2024-04-25T10:14:27.405827+0000","last_clean":"2024-04-25T10:14:27.405827+0000","last_became_active":"2024-04-25T10:14:27.397805+0000","last_became_peered":"2024-04-25T10:14:27.397805+0000","last_unstale":"2024-04-25T10:14:27.405827+0000","last_undegraded":"2024-04-25T10:14:27.405827+0000","last_fullsized":"2024-04-25T10:14:27.405827+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:02.970569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408202+0000","last_change":"2024-04-25T10:14:27.404203+0000","last_active":"2024-04-25T10:14:27.408202+0000","last_peered":"2024-04-25T10:14:27.408202+0000","last_clean":"2024-04-25T10:14:27.408202+0000","last_became_active":"2024-04-25T10:14:27.403913+0000","last_became_peered":"2024-04-25T10:14:27.403913+0000","last_unstale":"2024-04-25T10:14:27.408202+0000","last_undegraded":"2024-04-25T10:14:27.408202+0000","last_fullsized":"2024-04-25T10:14:27.408202+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:26.494609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.403791+0000","last_change":"2024-04-25T10:14:27.401097+0000","last_active":"2024-04-25T10:14:27.403791+0000","last_peered":"2024-04-25T10:14:27.403791+0000","last_clean":"2024-04-25T10:14:27.403791+0000","last_became_active":"2024-04-25T10:14:27.400760+0000","last_became_peered":"2024-04-25T10:14:27.400760+0000","last_unstale":"2024-04-25T10:14:27.403791+0000","last_undegraded":"2024-04-25T10:14:27.403791+0000","last_fullsized":"2024-04-25T10:14:27.403791+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:14.560458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.401827+0000","last_change":"2024-04-25T10:14:27.399801+0000","last_active":"2024-04-25T10:14:27.401827+0000","last_peered":"2024-04-25T10:14:27.401827+0000","last_clean":"2024-04-25T10:14:27.401827+0000","last_became_active":"2024-04-25T10:14:27.399508+0000","last_became_peered":"2024-04-25T10:14:27.399508+0000","last_unstale":"2024-04-25T10:14:27.401827+0000","last_undegraded":"2024-04-25T10:14:27.401827+0000","last_fullsized":"2024-04-25T10:14:27.401827+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:45.653282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412921+0000","last_change":"2024-04-25T10:14:27.411850+0000","last_active":"2024-04-25T10:14:27.412921+0000","last_peered":"2024-04-25T10:14:27.412921+0000","last_clean":"2024-04-25T10:14:27.412921+0000","last_became_active":"2024-04-25T10:14:27.411545+0000","last_became_peered":"2024-04-25T10:14:27.411545+0000","last_unstale":"2024-04-25T10:14:27.412921+0000","last_undegraded":"2024-04-25T10:14:27.412921+0000","last_fullsized":"2024-04-25T10:14:27.412921+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:11.105430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408465+0000","last_change":"2024-04-25T10:14:27.402249+0000","last_active":"2024-04-25T10:14:27.408465+0000","last_peered":"2024-04-25T10:14:27.408465+0000","last_clean":"2024-04-25T10:14:27.408465+0000","last_became_active":"2024-04-25T10:14:27.401947+0000","last_became_peered":"2024-04-25T10:14:27.401947+0000","last_unstale":"2024-04-25T10:14:27.408465+0000","last_undegraded":"2024-04-25T10:14:27.408465+0000","last_fullsized":"2024-04-25T10:14:27.408465+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:02.776562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.401761+0000","last_change":"2024-04-25T10:14:27.399340+0000","last_active":"2024-04-25T10:14:27.401761+0000","last_peered":"2024-04-25T10:14:27.401761+0000","last_clean":"2024-04-25T10:14:27.401761+0000","last_became_active":"2024-04-25T10:14:27.399054+0000","last_became_peered":"2024-04-25T10:14:27.399054+0000","last_unstale":"2024-04-25T10:14:27.401761+0000","last_undegraded":"2024-04-25T10:14:27.401761+0000","last_fullsized":"2024-04-25T10:14:27.401761+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:34.465544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.403405+0000","last_change":"2024-04-25T10:14:27.396410+0000","last_active":"2024-04-25T10:14:27.403405+0000","last_peered":"2024-04-25T10:14:27.403405+0000","last_clean":"2024-04-25T10:14:27.403405+0000","last_became_active":"2024-04-25T10:14:27.395475+0000","last_became_peered":"2024-04-25T10:14:27.395475+0000","last_unstale":"2024-04-25T10:14:27.403405+0000","last_undegraded":"2024-04-25T10:14:27.403405+0000","last_fullsized":"2024-04-25T10:14:27.403405+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:08.179273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.265072+0000","last_change":"2024-04-25T10:14:27.399279+0000","last_active":"2024-04-25T10:14:29.265072+0000","last_peered":"2024-04-25T10:14:29.265072+0000","last_clean":"2024-04-25T10:14:29.265072+0000","last_became_active":"2024-04-25T10:14:27.399011+0000","last_became_peered":"2024-04-25T10:14:27.399011+0000","last_unstale":"2024-04-25T10:14:29.265072+0000","last_undegraded":"2024-04-25T10:14:29.265072+0000","last_fullsized":"2024-04-25T10:14:29.265072+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:24.527238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412751+0000","last_change":"2024-04-25T10:14:27.411194+0000","last_active":"2024-04-25T10:14:27.412751+0000","last_peered":"2024-04-25T10:14:27.412751+0000","last_clean":"2024-04-25T10:14:27.412751+0000","last_became_active":"2024-04-25T10:14:27.410923+0000","last_became_peered":"2024-04-25T10:14:27.410923+0000","last_unstale":"2024-04-25T10:14:27.412751+0000","last_undegraded":"2024-04-25T10:14:27.412751+0000","last_fullsized":"2024-04-25T10:14:27.412751+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:45.905737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.411261+0000","last_change":"2024-04-25T10:14:27.408747+0000","last_active":"2024-04-25T10:14:27.411261+0000","last_peered":"2024-04-25T10:14:27.411261+0000","last_clean":"2024-04-25T10:14:27.411261+0000","last_became_active":"2024-04-25T10:14:27.407960+0000","last_became_peered":"2024-04-25T10:14:27.407960+0000","last_unstale":"2024-04-25T10:14:27.411261+0000","last_undegraded":"2024-04-25T10:14:27.411261+0000","last_fullsized":"2024-04-25T10:14:27.411261+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:38.602823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.404900+0000","last_change":"2024-04-25T10:14:27.402897+0000","last_active":"2024-04-25T10:14:27.404900+0000","last_peered":"2024-04-25T10:14:27.404900+0000","last_clean":"2024-04-25T10:14:27.404900+0000","last_became_active":"2024-04-25T10:14:27.402613+0000","last_became_peered":"2024-04-25T10:14:27.402613+0000","last_unstale":"2024-04-25T10:14:27.404900+0000","last_undegraded":"2024-04-25T10:14:27.404900+0000","last_fullsized":"2024-04-25T10:14:27.404900+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:10.245804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.265369+0000","last_change":"2024-04-25T10:14:27.401114+0000","last_active":"2024-04-25T10:14:29.265369+0000","last_peered":"2024-04-25T10:14:29.265369+0000","last_clean":"2024-04-25T10:14:29.265369+0000","last_became_active":"2024-04-25T10:14:27.400762+0000","last_became_peered":"2024-04-25T10:14:27.400762+0000","last_unstale":"2024-04-25T10:14:29.265369+0000","last_undegraded":"2024-04-25T10:14:29.265369+0000","last_fullsized":"2024-04-25T10:14:29.265369+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:13.125496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412095+0000","last_change":"2024-04-25T10:14:27.409296+0000","last_active":"2024-04-25T10:14:27.412095+0000","last_peered":"2024-04-25T10:14:27.412095+0000","last_clean":"2024-04-25T10:14:27.412095+0000","last_became_active":"2024-04-25T10:14:27.406648+0000","last_became_peered":"2024-04-25T10:14:27.406648+0000","last_unstale":"2024-04-25T10:14:27.412095+0000","last_undegraded":"2024-04-25T10:14:27.412095+0000","last_fullsized":"2024-04-25T10:14:27.412095+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:48.523576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.265146+0000","last_change":"2024-04-25T10:14:27.407877+0000","last_active":"2024-04-25T10:14:29.265146+0000","last_peered":"2024-04-25T10:14:29.265146+0000","last_clean":"2024-04-25T10:14:29.265146+0000","last_became_active":"2024-04-25T10:14:27.407578+0000","last_became_peered":"2024-04-25T10:14:27.407578+0000","last_unstale":"2024-04-25T10:14:29.265146+0000","last_undegraded":"2024-04-25T10:14:29.265146+0000","last_fullsized":"2024-04-25T10:14:29.265146+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.616799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.405965+0000","last_change":"2024-04-25T10:14:27.399018+0000","last_active":"2024-04-25T10:14:27.405965+0000","last_peered":"2024-04-25T10:14:27.405965+0000","last_clean":"2024-04-25T10:14:27.405965+0000","last_became_active":"2024-04-25T10:14:27.398721+0000","last_became_peered":"2024-04-25T10:14:27.398721+0000","last_unstale":"2024-04-25T10:14:27.405965+0000","last_undegraded":"2024-04-25T10:14:27.405965+0000","last_fullsized":"2024-04-25T10:14:27.405965+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:12.030595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.404175+0000","last_change":"2024-04-25T10:14:27.396648+0000","last_active":"2024-04-25T10:14:27.404175+0000","last_peered":"2024-04-25T10:14:27.404175+0000","last_clean":"2024-04-25T10:14:27.404175+0000","last_became_active":"2024-04-25T10:14:27.396271+0000","last_became_peered":"2024-04-25T10:14:27.396271+0000","last_unstale":"2024-04-25T10:14:27.404175+0000","last_undegraded":"2024-04-25T10:14:27.404175+0000","last_fullsized":"2024-04-25T10:14:27.404175+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:44.732911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408221+0000","last_change":"2024-04-25T10:14:27.405661+0000","last_active":"2024-04-25T10:14:27.408221+0000","last_peered":"2024-04-25T10:14:27.408221+0000","last_clean":"2024-04-25T10:14:27.408221+0000","last_became_active":"2024-04-25T10:14:27.405363+0000","last_became_peered":"2024-04-25T10:14:27.405363+0000","last_unstale":"2024-04-25T10:14:27.408221+0000","last_undegraded":"2024-04-25T10:14:27.408221+0000","last_fullsized":"2024-04-25T10:14:27.408221+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:23.505083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.405545+0000","last_change":"2024-04-25T10:14:27.400722+0000","last_active":"2024-04-25T10:14:27.405545+0000","last_peered":"2024-04-25T10:14:27.405545+0000","last_clean":"2024-04-25T10:14:27.405545+0000","last_became_active":"2024-04-25T10:14:27.400369+0000","last_became_peered":"2024-04-25T10:14:27.400369+0000","last_unstale":"2024-04-25T10:14:27.405545+0000","last_undegraded":"2024-04-25T10:14:27.405545+0000","last_fullsized":"2024-04-25T10:14:27.405545+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:34.465544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.414501+0000","last_change":"2024-04-25T10:14:27.412826+0000","last_active":"2024-04-25T10:14:27.414501+0000","last_peered":"2024-04-25T10:14:27.414501+0000","last_clean":"2024-04-25T10:14:27.414501+0000","last_became_active":"2024-04-25T10:14:27.412521+0000","last_became_peered":"2024-04-25T10:14:27.412521+0000","last_unstale":"2024-04-25T10:14:27.414501+0000","last_undegraded":"2024-04-25T10:14:27.414501+0000","last_fullsized":"2024-04-25T10:14:27.414501+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:12.595508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"114'4","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.414691+0000","last_change":"2024-04-25T10:14:26.394128+0000","last_active":"2024-04-25T10:14:27.414691+0000","last_peered":"2024-04-25T10:14:27.414691+0000","last_clean":"2024-04-25T10:14:27.414691+0000","last_became_active":"2024-04-25T10:14:26.393815+0000","last_became_peered":"2024-04-25T10:14:26.393815+0000","last_unstale":"2024-04-25T10:14:27.414691+0000","last_undegraded":"2024-04-25T10:14:27.414691+0000","last_fullsized":"2024-04-25T10:14:27.414691+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:53.877050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412787+0000","last_change":"2024-04-25T10:14:27.411129+0000","last_active":"2024-04-25T10:14:27.412787+0000","last_peered":"2024-04-25T10:14:27.412787+0000","last_clean":"2024-04-25T10:14:27.412787+0000","last_became_active":"2024-04-25T10:14:27.410834+0000","last_became_peered":"2024-04-25T10:14:27.410834+0000","last_unstale":"2024-04-25T10:14:27.412787+0000","last_undegraded":"2024-04-25T10:14:27.412787+0000","last_fullsized":"2024-04-25T10:14:27.412787+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:45.211167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"114'1","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412419+0000","last_change":"2024-04-25T10:14:26.387502+0000","last_active":"2024-04-25T10:14:27.412419+0000","last_peered":"2024-04-25T10:14:27.412419+0000","last_clean":"2024-04-25T10:14:27.412419+0000","last_became_active":"2024-04-25T10:14:26.387228+0000","last_became_peered":"2024-04-25T10:14:26.387228+0000","last_unstale":"2024-04-25T10:14:27.412419+0000","last_undegraded":"2024-04-25T10:14:27.412419+0000","last_fullsized":"2024-04-25T10:14:27.412419+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:54.436383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.265197+0000","last_change":"2024-04-25T10:14:27.403563+0000","last_active":"2024-04-25T10:14:29.265197+0000","last_peered":"2024-04-25T10:14:29.265197+0000","last_clean":"2024-04-25T10:14:29.265197+0000","last_became_active":"2024-04-25T10:14:27.403237+0000","last_became_peered":"2024-04-25T10:14:27.403237+0000","last_unstale":"2024-04-25T10:14:29.265197+0000","last_undegraded":"2024-04-25T10:14:29.265197+0000","last_fullsized":"2024-04-25T10:14:29.265197+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:09.422709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"114'2","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412043+0000","last_change":"2024-04-25T10:14:26.401039+0000","last_active":"2024-04-25T10:14:27.412043+0000","last_peered":"2024-04-25T10:14:27.412043+0000","last_clean":"2024-04-25T10:14:27.412043+0000","last_became_active":"2024-04-25T10:14:26.400766+0000","last_became_peered":"2024-04-25T10:14:26.400766+0000","last_unstale":"2024-04-25T10:14:27.412043+0000","last_undegraded":"2024-04-25T10:14:27.412043+0000","last_fullsized":"2024-04-25T10:14:27.412043+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:11.028136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408424+0000","last_change":"2024-04-25T10:14:27.402058+0000","last_active":"2024-04-25T10:14:27.408424+0000","last_peered":"2024-04-25T10:14:27.408424+0000","last_clean":"2024-04-25T10:14:27.408424+0000","last_became_active":"2024-04-25T10:14:27.401570+0000","last_became_peered":"2024-04-25T10:14:27.401570+0000","last_unstale":"2024-04-25T10:14:27.408424+0000","last_undegraded":"2024-04-25T10:14:27.408424+0000","last_fullsized":"2024-04-25T10:14:27.408424+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:18.842796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"114'2","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.413228+0000","last_change":"2024-04-25T10:14:26.401991+0000","last_active":"2024-04-25T10:14:27.413228+0000","last_peered":"2024-04-25T10:14:27.413228+0000","last_clean":"2024-04-25T10:14:27.413228+0000","last_became_active":"2024-04-25T10:14:26.401650+0000","last_became_peered":"2024-04-25T10:14:26.401650+0000","last_unstale":"2024-04-25T10:14:27.413228+0000","last_undegraded":"2024-04-25T10:14:27.413228+0000","last_fullsized":"2024-04-25T10:14:27.413228+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:43.623099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.411712+0000","last_change":"2024-04-25T10:14:27.409672+0000","last_active":"2024-04-25T10:14:27.411712+0000","last_peered":"2024-04-25T10:14:27.411712+0000","last_clean":"2024-04-25T10:14:27.411712+0000","last_became_active":"2024-04-25T10:14:27.408297+0000","last_became_peered":"2024-04-25T10:14:27.408297+0000","last_unstale":"2024-04-25T10:14:27.411712+0000","last_undegraded":"2024-04-25T10:14:27.411712+0000","last_fullsized":"2024-04-25T10:14:27.411712+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:23.505083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"114'1","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.407126+0000","last_change":"2024-04-25T10:14:26.396449+0000","last_active":"2024-04-25T10:14:27.407126+0000","last_peered":"2024-04-25T10:14:27.407126+0000","last_clean":"2024-04-25T10:14:27.407126+0000","last_became_active":"2024-04-25T10:14:26.396146+0000","last_became_peered":"2024-04-25T10:14:26.396146+0000","last_unstale":"2024-04-25T10:14:27.407126+0000","last_undegraded":"2024-04-25T10:14:27.407126+0000","last_fullsized":"2024-04-25T10:14:27.407126+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:22.079207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.404943+0000","last_change":"2024-04-25T10:14:27.403529+0000","last_active":"2024-04-25T10:14:27.404943+0000","last_peered":"2024-04-25T10:14:27.404943+0000","last_clean":"2024-04-25T10:14:27.404943+0000","last_became_active":"2024-04-25T10:14:27.403204+0000","last_became_peered":"2024-04-25T10:14:27.403204+0000","last_unstale":"2024-04-25T10:14:27.404943+0000","last_undegraded":"2024-04-25T10:14:27.404943+0000","last_fullsized":"2024-04-25T10:14:27.404943+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:44.353580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"114'1","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.415451+0000","last_change":"2024-04-25T10:14:26.395516+0000","last_active":"2024-04-25T10:14:27.415451+0000","last_peered":"2024-04-25T10:14:27.415451+0000","last_clean":"2024-04-25T10:14:27.415451+0000","last_became_active":"2024-04-25T10:14:26.395240+0000","last_became_peered":"2024-04-25T10:14:26.395240+0000","last_unstale":"2024-04-25T10:14:27.415451+0000","last_undegraded":"2024-04-25T10:14:27.415451+0000","last_fullsized":"2024-04-25T10:14:27.415451+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:35.714332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.403773+0000","last_change":"2024-04-25T10:14:27.398047+0000","last_active":"2024-04-25T10:14:27.403773+0000","last_peered":"2024-04-25T10:14:27.403773+0000","last_clean":"2024-04-25T10:14:27.403773+0000","last_became_active":"2024-04-25T10:14:27.397779+0000","last_became_peered":"2024-04-25T10:14:27.397779+0000","last_unstale":"2024-04-25T10:14:27.403773+0000","last_undegraded":"2024-04-25T10:14:27.403773+0000","last_fullsized":"2024-04-25T10:14:27.403773+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:23.081666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.388084+0000","last_change":"2024-04-25T10:14:26.399334+0000","last_active":"2024-04-25T10:14:27.388084+0000","last_peered":"2024-04-25T10:14:27.388084+0000","last_clean":"2024-04-25T10:14:27.388084+0000","last_became_active":"2024-04-25T10:14:26.399009+0000","last_became_peered":"2024-04-25T10:14:26.399009+0000","last_unstale":"2024-04-25T10:14:27.388084+0000","last_undegraded":"2024-04-25T10:14:27.388084+0000","last_fullsized":"2024-04-25T10:14:27.388084+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:48.181442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.404967+0000","last_change":"2024-04-25T10:14:27.403852+0000","last_active":"2024-04-25T10:14:27.404967+0000","last_peered":"2024-04-25T10:14:27.404967+0000","last_clean":"2024-04-25T10:14:27.404967+0000","last_became_active":"2024-04-25T10:14:27.403594+0000","last_became_peered":"2024-04-25T10:14:27.403594+0000","last_unstale":"2024-04-25T10:14:27.404967+0000","last_undegraded":"2024-04-25T10:14:27.404967+0000","last_fullsized":"2024-04-25T10:14:27.404967+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:30.054392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.390987+0000","last_change":"2024-04-25T10:14:26.403085+0000","last_active":"2024-04-25T10:14:27.390987+0000","last_peered":"2024-04-25T10:14:27.390987+0000","last_clean":"2024-04-25T10:14:27.390987+0000","last_became_active":"2024-04-25T10:14:26.402789+0000","last_became_peered":"2024-04-25T10:14:26.402789+0000","last_unstale":"2024-04-25T10:14:27.390987+0000","last_undegraded":"2024-04-25T10:14:27.390987+0000","last_fullsized":"2024-04-25T10:14:27.390987+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:40.375833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.411662+0000","last_change":"2024-04-25T10:14:27.408461+0000","last_active":"2024-04-25T10:14:27.411662+0000","last_peered":"2024-04-25T10:14:27.411662+0000","last_clean":"2024-04-25T10:14:27.411662+0000","last_became_active":"2024-04-25T10:14:27.408180+0000","last_became_peered":"2024-04-25T10:14:27.408180+0000","last_unstale":"2024-04-25T10:14:27.411662+0000","last_undegraded":"2024-04-25T10:14:27.411662+0000","last_fullsized":"2024-04-25T10:14:27.411662+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:45.211167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.387604+0000","last_change":"2024-04-25T10:14:26.398102+0000","last_active":"2024-04-25T10:14:27.387604+0000","last_peered":"2024-04-25T10:14:27.387604+0000","last_clean":"2024-04-25T10:14:27.387604+0000","last_became_active":"2024-04-25T10:14:26.397805+0000","last_became_peered":"2024-04-25T10:14:26.397805+0000","last_unstale":"2024-04-25T10:14:27.387604+0000","last_undegraded":"2024-04-25T10:14:27.387604+0000","last_fullsized":"2024-04-25T10:14:27.387604+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:55.437165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.401719+0000","last_change":"2024-04-25T10:14:27.398503+0000","last_active":"2024-04-25T10:14:27.401719+0000","last_peered":"2024-04-25T10:14:27.401719+0000","last_clean":"2024-04-25T10:14:27.401719+0000","last_became_active":"2024-04-25T10:14:27.397940+0000","last_became_peered":"2024-04-25T10:14:27.397940+0000","last_unstale":"2024-04-25T10:14:27.401719+0000","last_undegraded":"2024-04-25T10:14:27.401719+0000","last_fullsized":"2024-04-25T10:14:27.401719+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:21.027596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.393191+0000","last_change":"2024-04-25T10:14:26.401312+0000","last_active":"2024-04-25T10:14:27.393191+0000","last_peered":"2024-04-25T10:14:27.393191+0000","last_clean":"2024-04-25T10:14:27.393191+0000","last_became_active":"2024-04-25T10:14:26.401047+0000","last_became_peered":"2024-04-25T10:14:26.401047+0000","last_unstale":"2024-04-25T10:14:27.393191+0000","last_undegraded":"2024-04-25T10:14:27.393191+0000","last_fullsized":"2024-04-25T10:14:27.393191+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:55.080445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.265340+0000","last_change":"2024-04-25T10:14:27.404853+0000","last_active":"2024-04-25T10:14:29.265340+0000","last_peered":"2024-04-25T10:14:29.265340+0000","last_clean":"2024-04-25T10:14:29.265340+0000","last_became_active":"2024-04-25T10:14:27.404538+0000","last_became_peered":"2024-04-25T10:14:27.404538+0000","last_unstale":"2024-04-25T10:14:29.265340+0000","last_undegraded":"2024-04-25T10:14:29.265340+0000","last_fullsized":"2024-04-25T10:14:29.265340+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:48.945878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"114'3","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.414040+0000","last_change":"2024-04-25T10:14:26.402072+0000","last_active":"2024-04-25T10:14:27.414040+0000","last_peered":"2024-04-25T10:14:27.414040+0000","last_clean":"2024-04-25T10:14:27.414040+0000","last_became_active":"2024-04-25T10:14:26.401803+0000","last_became_peered":"2024-04-25T10:14:26.401803+0000","last_unstale":"2024-04-25T10:14:27.414040+0000","last_undegraded":"2024-04-25T10:14:27.414040+0000","last_fullsized":"2024-04-25T10:14:27.414040+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:13.625722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.404717+0000","last_change":"2024-04-25T10:14:27.402474+0000","last_active":"2024-04-25T10:14:27.404717+0000","last_peered":"2024-04-25T10:14:27.404717+0000","last_clean":"2024-04-25T10:14:27.404717+0000","last_became_active":"2024-04-25T10:14:27.402196+0000","last_became_peered":"2024-04-25T10:14:27.402196+0000","last_unstale":"2024-04-25T10:14:27.404717+0000","last_undegraded":"2024-04-25T10:14:27.404717+0000","last_fullsized":"2024-04-25T10:14:27.404717+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:10.484587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"114'6","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.416746+0000","last_change":"2024-04-25T10:14:26.398379+0000","last_active":"2024-04-25T10:14:27.416746+0000","last_peered":"2024-04-25T10:14:27.416746+0000","last_clean":"2024-04-25T10:14:27.416746+0000","last_became_active":"2024-04-25T10:14:26.398058+0000","last_became_peered":"2024-04-25T10:14:26.398058+0000","last_unstale":"2024-04-25T10:14:27.416746+0000","last_undegraded":"2024-04-25T10:14:27.416746+0000","last_fullsized":"2024-04-25T10:14:27.416746+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:35.714332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408552+0000","last_change":"2024-04-25T10:14:27.400833+0000","last_active":"2024-04-25T10:14:27.408552+0000","last_peered":"2024-04-25T10:14:27.408552+0000","last_clean":"2024-04-25T10:14:27.408552+0000","last_became_active":"2024-04-25T10:14:27.400548+0000","last_became_peered":"2024-04-25T10:14:27.400548+0000","last_unstale":"2024-04-25T10:14:27.408552+0000","last_undegraded":"2024-04-25T10:14:27.408552+0000","last_fullsized":"2024-04-25T10:14:27.408552+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:45.653282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"114'2","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412390+0000","last_change":"2024-04-25T10:14:26.401794+0000","last_active":"2024-04-25T10:14:27.412390+0000","last_peered":"2024-04-25T10:14:27.412390+0000","last_clean":"2024-04-25T10:14:27.412390+0000","last_became_active":"2024-04-25T10:14:26.401457+0000","last_became_peered":"2024-04-25T10:14:26.401457+0000","last_unstale":"2024-04-25T10:14:27.412390+0000","last_undegraded":"2024-04-25T10:14:27.412390+0000","last_fullsized":"2024-04-25T10:14:27.412390+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:42.589728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.406174+0000","last_change":"2024-04-25T10:14:27.398829+0000","last_active":"2024-04-25T10:14:27.406174+0000","last_peered":"2024-04-25T10:14:27.406174+0000","last_clean":"2024-04-25T10:14:27.406174+0000","last_became_active":"2024-04-25T10:14:27.398570+0000","last_became_peered":"2024-04-25T10:14:27.398570+0000","last_unstale":"2024-04-25T10:14:27.406174+0000","last_undegraded":"2024-04-25T10:14:27.406174+0000","last_fullsized":"2024-04-25T10:14:27.406174+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:26.494609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.389889+0000","last_change":"2024-04-25T10:14:26.394187+0000","last_active":"2024-04-25T10:14:27.389889+0000","last_peered":"2024-04-25T10:14:27.389889+0000","last_clean":"2024-04-25T10:14:27.389889+0000","last_became_active":"2024-04-25T10:14:26.393904+0000","last_became_peered":"2024-04-25T10:14:26.393904+0000","last_unstale":"2024-04-25T10:14:27.389889+0000","last_undegraded":"2024-04-25T10:14:27.389889+0000","last_fullsized":"2024-04-25T10:14:27.389889+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:20.769896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.413037+0000","last_change":"2024-04-25T10:14:27.412112+0000","last_active":"2024-04-25T10:14:27.413037+0000","last_peered":"2024-04-25T10:14:27.413037+0000","last_clean":"2024-04-25T10:14:27.413037+0000","last_became_active":"2024-04-25T10:14:27.411847+0000","last_became_peered":"2024-04-25T10:14:27.411847+0000","last_unstale":"2024-04-25T10:14:27.413037+0000","last_undegraded":"2024-04-25T10:14:27.413037+0000","last_fullsized":"2024-04-25T10:14:27.413037+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:17.554024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"114'8","reported_seq":20,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.419427+0000","last_change":"2024-04-25T10:14:26.396288+0000","last_active":"2024-04-25T10:14:27.419427+0000","last_peered":"2024-04-25T10:14:27.419427+0000","last_clean":"2024-04-25T10:14:27.419427+0000","last_became_active":"2024-04-25T10:14:26.396004+0000","last_became_peered":"2024-04-25T10:14:26.396004+0000","last_unstale":"2024-04-25T10:14:27.419427+0000","last_undegraded":"2024-04-25T10:14:27.419427+0000","last_fullsized":"2024-04-25T10:14:27.419427+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:41.491585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.411574+0000","last_change":"2024-04-25T10:14:27.408290+0000","last_active":"2024-04-25T10:14:27.411574+0000","last_peered":"2024-04-25T10:14:27.411574+0000","last_clean":"2024-04-25T10:14:27.411574+0000","last_became_active":"2024-04-25T10:14:27.407717+0000","last_became_peered":"2024-04-25T10:14:27.407717+0000","last_unstale":"2024-04-25T10:14:27.411574+0000","last_undegraded":"2024-04-25T10:14:27.411574+0000","last_fullsized":"2024-04-25T10:14:27.411574+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:45.369856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"114'1","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.265452+0000","last_change":"2024-04-25T10:14:26.400227+0000","last_active":"2024-04-25T10:14:29.265452+0000","last_peered":"2024-04-25T10:14:29.265452+0000","last_clean":"2024-04-25T10:14:29.265452+0000","last_became_active":"2024-04-25T10:14:26.399515+0000","last_became_peered":"2024-04-25T10:14:26.399515+0000","last_unstale":"2024-04-25T10:14:29.265452+0000","last_undegraded":"2024-04-25T10:14:29.265452+0000","last_fullsized":"2024-04-25T10:14:29.265452+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:10.867163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.407969+0000","last_change":"2024-04-25T10:14:27.406595+0000","last_active":"2024-04-25T10:14:27.407969+0000","last_peered":"2024-04-25T10:14:27.407969+0000","last_clean":"2024-04-25T10:14:27.407969+0000","last_became_active":"2024-04-25T10:14:27.406314+0000","last_became_peered":"2024-04-25T10:14:27.406314+0000","last_unstale":"2024-04-25T10:14:27.407969+0000","last_undegraded":"2024-04-25T10:14:27.407969+0000","last_fullsized":"2024-04-25T10:14:27.407969+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.623749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"114'2","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412345+0000","last_change":"2024-04-25T10:14:26.401790+0000","last_active":"2024-04-25T10:14:27.412345+0000","last_peered":"2024-04-25T10:14:27.412345+0000","last_clean":"2024-04-25T10:14:27.412345+0000","last_became_active":"2024-04-25T10:14:26.401472+0000","last_became_peered":"2024-04-25T10:14:26.401472+0000","last_unstale":"2024-04-25T10:14:27.412345+0000","last_undegraded":"2024-04-25T10:14:27.412345+0000","last_fullsized":"2024-04-25T10:14:27.412345+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:52.318817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412894+0000","last_change":"2024-04-25T10:14:27.411144+0000","last_active":"2024-04-25T10:14:27.412894+0000","last_peered":"2024-04-25T10:14:27.412894+0000","last_clean":"2024-04-25T10:14:27.412894+0000","last_became_active":"2024-04-25T10:14:27.410656+0000","last_became_peered":"2024-04-25T10:14:27.410656+0000","last_unstale":"2024-04-25T10:14:27.412894+0000","last_undegraded":"2024-04-25T10:14:27.412894+0000","last_fullsized":"2024-04-25T10:14:27.412894+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:12.498391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"114'1","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.406577+0000","last_change":"2024-04-25T10:14:26.396174+0000","last_active":"2024-04-25T10:14:27.406577+0000","last_peered":"2024-04-25T10:14:27.406577+0000","last_clean":"2024-04-25T10:14:27.406577+0000","last_became_active":"2024-04-25T10:14:26.395916+0000","last_became_peered":"2024-04-25T10:14:26.395916+0000","last_unstale":"2024-04-25T10:14:27.406577+0000","last_undegraded":"2024-04-25T10:14:27.406577+0000","last_fullsized":"2024-04-25T10:14:27.406577+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:55.437165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408361+0000","last_change":"2024-04-25T10:14:27.406814+0000","last_active":"2024-04-25T10:14:27.408361+0000","last_peered":"2024-04-25T10:14:27.408361+0000","last_clean":"2024-04-25T10:14:27.408361+0000","last_became_active":"2024-04-25T10:14:27.406324+0000","last_became_peered":"2024-04-25T10:14:27.406324+0000","last_unstale":"2024-04-25T10:14:27.408361+0000","last_undegraded":"2024-04-25T10:14:27.408361+0000","last_fullsized":"2024-04-25T10:14:27.408361+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:55.438842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.391803+0000","last_change":"2024-04-25T10:14:26.398044+0000","last_active":"2024-04-25T10:14:27.391803+0000","last_peered":"2024-04-25T10:14:27.391803+0000","last_clean":"2024-04-25T10:14:27.391803+0000","last_became_active":"2024-04-25T10:14:26.397764+0000","last_became_peered":"2024-04-25T10:14:26.397764+0000","last_unstale":"2024-04-25T10:14:27.391803+0000","last_undegraded":"2024-04-25T10:14:27.391803+0000","last_fullsized":"2024-04-25T10:14:27.391803+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:00.039314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408386+0000","last_change":"2024-04-25T10:14:27.403930+0000","last_active":"2024-04-25T10:14:27.408386+0000","last_peered":"2024-04-25T10:14:27.408386+0000","last_clean":"2024-04-25T10:14:27.408386+0000","last_became_active":"2024-04-25T10:14:27.403622+0000","last_became_peered":"2024-04-25T10:14:27.403622+0000","last_unstale":"2024-04-25T10:14:27.408386+0000","last_undegraded":"2024-04-25T10:14:27.408386+0000","last_fullsized":"2024-04-25T10:14:27.408386+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:07.652434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"114'2","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.415478+0000","last_change":"2024-04-25T10:14:26.401084+0000","last_active":"2024-04-25T10:14:27.415478+0000","last_peered":"2024-04-25T10:14:27.415478+0000","last_clean":"2024-04-25T10:14:27.415478+0000","last_became_active":"2024-04-25T10:14:26.400795+0000","last_became_peered":"2024-04-25T10:14:26.400795+0000","last_unstale":"2024-04-25T10:14:27.415478+0000","last_undegraded":"2024-04-25T10:14:27.415478+0000","last_fullsized":"2024-04-25T10:14:27.415478+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:48.084204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.407878+0000","last_change":"2024-04-25T10:14:27.405035+0000","last_active":"2024-04-25T10:14:27.407878+0000","last_peered":"2024-04-25T10:14:27.407878+0000","last_clean":"2024-04-25T10:14:27.407878+0000","last_became_active":"2024-04-25T10:14:27.404763+0000","last_became_peered":"2024-04-25T10:14:27.404763+0000","last_unstale":"2024-04-25T10:14:27.407878+0000","last_undegraded":"2024-04-25T10:14:27.407878+0000","last_fullsized":"2024-04-25T10:14:27.407878+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:48.523576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.395083+0000","last_change":"2024-04-25T10:14:26.398905+0000","last_active":"2024-04-25T10:14:27.395083+0000","last_peered":"2024-04-25T10:14:27.395083+0000","last_clean":"2024-04-25T10:14:27.395083+0000","last_became_active":"2024-04-25T10:14:26.398627+0000","last_became_peered":"2024-04-25T10:14:26.398627+0000","last_unstale":"2024-04-25T10:14:27.395083+0000","last_undegraded":"2024-04-25T10:14:27.395083+0000","last_fullsized":"2024-04-25T10:14:27.395083+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:44.208708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408901+0000","last_change":"2024-04-25T10:14:27.407507+0000","last_active":"2024-04-25T10:14:27.408901+0000","last_peered":"2024-04-25T10:14:27.408901+0000","last_clean":"2024-04-25T10:14:27.408901+0000","last_became_active":"2024-04-25T10:14:27.407236+0000","last_became_peered":"2024-04-25T10:14:27.407236+0000","last_unstale":"2024-04-25T10:14:27.408901+0000","last_undegraded":"2024-04-25T10:14:27.408901+0000","last_fullsized":"2024-04-25T10:14:27.408901+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:42.494044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.387758+0000","last_change":"2024-04-25T10:14:26.402011+0000","last_active":"2024-04-25T10:14:27.387758+0000","last_peered":"2024-04-25T10:14:27.387758+0000","last_clean":"2024-04-25T10:14:27.387758+0000","last_became_active":"2024-04-25T10:14:26.401697+0000","last_became_peered":"2024-04-25T10:14:26.401697+0000","last_unstale":"2024-04-25T10:14:27.387758+0000","last_undegraded":"2024-04-25T10:14:27.387758+0000","last_fullsized":"2024-04-25T10:14:27.387758+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:01.968111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.403741+0000","last_change":"2024-04-25T10:14:27.398679+0000","last_active":"2024-04-25T10:14:27.403741+0000","last_peered":"2024-04-25T10:14:27.403741+0000","last_clean":"2024-04-25T10:14:27.403741+0000","last_became_active":"2024-04-25T10:14:27.398426+0000","last_became_peered":"2024-04-25T10:14:27.398426+0000","last_unstale":"2024-04-25T10:14:27.403741+0000","last_undegraded":"2024-04-25T10:14:27.403741+0000","last_fullsized":"2024-04-25T10:14:27.403741+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:21.772355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.388473+0000","last_change":"2024-04-25T10:14:26.401127+0000","last_active":"2024-04-25T10:14:27.388473+0000","last_peered":"2024-04-25T10:14:27.388473+0000","last_clean":"2024-04-25T10:14:27.388473+0000","last_became_active":"2024-04-25T10:14:26.400828+0000","last_became_peered":"2024-04-25T10:14:26.400828+0000","last_unstale":"2024-04-25T10:14:27.388473+0000","last_undegraded":"2024-04-25T10:14:27.388473+0000","last_fullsized":"2024-04-25T10:14:27.388473+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:41.491585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.410306+0000","last_change":"2024-04-25T10:14:27.408674+0000","last_active":"2024-04-25T10:14:27.410306+0000","last_peered":"2024-04-25T10:14:27.410306+0000","last_clean":"2024-04-25T10:14:27.410306+0000","last_became_active":"2024-04-25T10:14:27.408391+0000","last_became_peered":"2024-04-25T10:14:27.408391+0000","last_unstale":"2024-04-25T10:14:27.410306+0000","last_undegraded":"2024-04-25T10:14:27.410306+0000","last_fullsized":"2024-04-25T10:14:27.410306+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:21.027596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"114'2","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.413498+0000","last_change":"2024-04-25T10:14:26.398102+0000","last_active":"2024-04-25T10:14:27.413498+0000","last_peered":"2024-04-25T10:14:27.413498+0000","last_clean":"2024-04-25T10:14:27.413498+0000","last_became_active":"2024-04-25T10:14:26.397796+0000","last_became_peered":"2024-04-25T10:14:26.397796+0000","last_unstale":"2024-04-25T10:14:27.413498+0000","last_undegraded":"2024-04-25T10:14:27.413498+0000","last_fullsized":"2024-04-25T10:14:27.413498+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:51.749378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.265414+0000","last_change":"2024-04-25T10:14:27.404769+0000","last_active":"2024-04-25T10:14:29.265414+0000","last_peered":"2024-04-25T10:14:29.265414+0000","last_clean":"2024-04-25T10:14:29.265414+0000","last_became_active":"2024-04-25T10:14:27.404474+0000","last_became_peered":"2024-04-25T10:14:27.404474+0000","last_unstale":"2024-04-25T10:14:29.265414+0000","last_undegraded":"2024-04-25T10:14:29.265414+0000","last_fullsized":"2024-04-25T10:14:29.265414+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:05.792549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.390313+0000","last_change":"2024-04-25T10:14:26.396893+0000","last_active":"2024-04-25T10:14:27.390313+0000","last_peered":"2024-04-25T10:14:27.390313+0000","last_clean":"2024-04-25T10:14:27.390313+0000","last_became_active":"2024-04-25T10:14:26.396593+0000","last_became_peered":"2024-04-25T10:14:26.396593+0000","last_unstale":"2024-04-25T10:14:27.390313+0000","last_undegraded":"2024-04-25T10:14:27.390313+0000","last_fullsized":"2024-04-25T10:14:27.390313+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:28.082901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.407863+0000","last_change":"2024-04-25T10:14:27.404209+0000","last_active":"2024-04-25T10:14:27.407863+0000","last_peered":"2024-04-25T10:14:27.407863+0000","last_clean":"2024-04-25T10:14:27.407863+0000","last_became_active":"2024-04-25T10:14:27.403931+0000","last_became_peered":"2024-04-25T10:14:27.403931+0000","last_unstale":"2024-04-25T10:14:27.407863+0000","last_undegraded":"2024-04-25T10:14:27.407863+0000","last_fullsized":"2024-04-25T10:14:27.407863+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:38.602823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.388436+0000","last_change":"2024-04-25T10:14:26.397428+0000","last_active":"2024-04-25T10:14:27.388436+0000","last_peered":"2024-04-25T10:14:27.388436+0000","last_clean":"2024-04-25T10:14:27.388436+0000","last_became_active":"2024-04-25T10:14:26.394137+0000","last_became_peered":"2024-04-25T10:14:26.394137+0000","last_unstale":"2024-04-25T10:14:27.388436+0000","last_undegraded":"2024-04-25T10:14:27.388436+0000","last_fullsized":"2024-04-25T10:14:27.388436+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:54.436383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408339+0000","last_change":"2024-04-25T10:14:27.406626+0000","last_active":"2024-04-25T10:14:27.408339+0000","last_peered":"2024-04-25T10:14:27.408339+0000","last_clean":"2024-04-25T10:14:27.408339+0000","last_became_active":"2024-04-25T10:14:27.406232+0000","last_became_peered":"2024-04-25T10:14:27.406232+0000","last_unstale":"2024-04-25T10:14:27.408339+0000","last_undegraded":"2024-04-25T10:14:27.408339+0000","last_fullsized":"2024-04-25T10:14:27.408339+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:44.625558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.388542+0000","last_change":"2024-04-25T10:14:26.398067+0000","last_active":"2024-04-25T10:14:27.388542+0000","last_peered":"2024-04-25T10:14:27.388542+0000","last_clean":"2024-04-25T10:14:27.388542+0000","last_became_active":"2024-04-25T10:14:26.397760+0000","last_became_peered":"2024-04-25T10:14:26.397760+0000","last_unstale":"2024-04-25T10:14:27.388542+0000","last_undegraded":"2024-04-25T10:14:27.388542+0000","last_fullsized":"2024-04-25T10:14:27.388542+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:20.025137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.401889+0000","last_change":"2024-04-25T10:14:27.399956+0000","last_active":"2024-04-25T10:14:27.401889+0000","last_peered":"2024-04-25T10:14:27.401889+0000","last_clean":"2024-04-25T10:14:27.401889+0000","last_became_active":"2024-04-25T10:14:27.399627+0000","last_became_peered":"2024-04-25T10:14:27.399627+0000","last_unstale":"2024-04-25T10:14:27.401889+0000","last_undegraded":"2024-04-25T10:14:27.401889+0000","last_fullsized":"2024-04-25T10:14:27.401889+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:14.560458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.388648+0000","last_change":"2024-04-25T10:14:26.395028+0000","last_active":"2024-04-25T10:14:27.388648+0000","last_peered":"2024-04-25T10:14:27.388648+0000","last_clean":"2024-04-25T10:14:27.388648+0000","last_became_active":"2024-04-25T10:14:26.394720+0000","last_became_peered":"2024-04-25T10:14:26.394720+0000","last_unstale":"2024-04-25T10:14:27.388648+0000","last_undegraded":"2024-04-25T10:14:27.388648+0000","last_fullsized":"2024-04-25T10:14:27.388648+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:57.509448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412162+0000","last_change":"2024-04-25T10:14:27.409830+0000","last_active":"2024-04-25T10:14:27.412162+0000","last_peered":"2024-04-25T10:14:27.412162+0000","last_clean":"2024-04-25T10:14:27.412162+0000","last_became_active":"2024-04-25T10:14:27.409560+0000","last_became_peered":"2024-04-25T10:14:27.409560+0000","last_unstale":"2024-04-25T10:14:27.412162+0000","last_undegraded":"2024-04-25T10:14:27.412162+0000","last_fullsized":"2024-04-25T10:14:27.412162+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:36.716791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.393187+0000","last_change":"2024-04-25T10:14:26.396177+0000","last_active":"2024-04-25T10:14:27.393187+0000","last_peered":"2024-04-25T10:14:27.393187+0000","last_clean":"2024-04-25T10:14:27.393187+0000","last_became_active":"2024-04-25T10:14:26.395882+0000","last_became_peered":"2024-04-25T10:14:26.395882+0000","last_unstale":"2024-04-25T10:14:27.393187+0000","last_undegraded":"2024-04-25T10:14:27.393187+0000","last_fullsized":"2024-04-25T10:14:27.393187+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:51.749378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.413049+0000","last_change":"2024-04-25T10:14:27.411853+0000","last_active":"2024-04-25T10:14:27.413049+0000","last_peered":"2024-04-25T10:14:27.413049+0000","last_clean":"2024-04-25T10:14:27.413049+0000","last_became_active":"2024-04-25T10:14:27.411550+0000","last_became_peered":"2024-04-25T10:14:27.411550+0000","last_unstale":"2024-04-25T10:14:27.413049+0000","last_undegraded":"2024-04-25T10:14:27.413049+0000","last_fullsized":"2024-04-25T10:14:27.413049+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:26.494609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.394356+0000","last_change":"2024-04-25T10:14:26.396537+0000","last_active":"2024-04-25T10:14:27.394356+0000","last_peered":"2024-04-25T10:14:27.394356+0000","last_clean":"2024-04-25T10:14:27.394356+0000","last_became_active":"2024-04-25T10:14:26.396221+0000","last_became_peered":"2024-04-25T10:14:26.396221+0000","last_unstale":"2024-04-25T10:14:27.394356+0000","last_undegraded":"2024-04-25T10:14:27.394356+0000","last_fullsized":"2024-04-25T10:14:27.394356+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:20.025137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.264693+0000","last_change":"2024-04-25T10:14:27.410338+0000","last_active":"2024-04-25T10:14:29.264693+0000","last_peered":"2024-04-25T10:14:29.264693+0000","last_clean":"2024-04-25T10:14:29.264693+0000","last_became_active":"2024-04-25T10:14:27.410050+0000","last_became_peered":"2024-04-25T10:14:27.410050+0000","last_unstale":"2024-04-25T10:14:29.264693+0000","last_undegraded":"2024-04-25T10:14:29.264693+0000","last_fullsized":"2024-04-25T10:14:29.264693+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:28.594949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.264499+0000","last_change":"2024-04-25T10:14:26.400437+0000","last_active":"2024-04-25T10:14:29.264499+0000","last_peered":"2024-04-25T10:14:29.264499+0000","last_clean":"2024-04-25T10:14:29.264499+0000","last_became_active":"2024-04-25T10:14:26.399347+0000","last_became_peered":"2024-04-25T10:14:26.399347+0000","last_unstale":"2024-04-25T10:14:29.264499+0000","last_undegraded":"2024-04-25T10:14:29.264499+0000","last_fullsized":"2024-04-25T10:14:29.264499+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:38.469650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.396538+0000","last_change":"2024-04-25T10:14:27.394580+0000","last_active":"2024-04-25T10:14:27.396538+0000","last_peered":"2024-04-25T10:14:27.396538+0000","last_clean":"2024-04-25T10:14:27.396538+0000","last_became_active":"2024-04-25T10:14:27.394296+0000","last_became_peered":"2024-04-25T10:14:27.394296+0000","last_unstale":"2024-04-25T10:14:27.396538+0000","last_undegraded":"2024-04-25T10:14:27.396538+0000","last_fullsized":"2024-04-25T10:14:27.396538+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:45.905737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.384360+0000","last_change":"2024-04-25T10:14:26.395283+0000","last_active":"2024-04-25T10:14:27.384360+0000","last_peered":"2024-04-25T10:14:27.384360+0000","last_clean":"2024-04-25T10:14:27.384360+0000","last_became_active":"2024-04-25T10:14:26.394981+0000","last_became_peered":"2024-04-25T10:14:26.394981+0000","last_unstale":"2024-04-25T10:14:27.384360+0000","last_undegraded":"2024-04-25T10:14:27.384360+0000","last_fullsized":"2024-04-25T10:14:27.384360+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:43.623099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":358,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.388879+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:14:27.388879+0000","last_peered":"2024-04-25T10:14:27.388879+0000","last_clean":"2024-04-25T10:14:27.388879+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:14:27.388879+0000","last_undegraded":"2024-04-25T10:14:27.388879+0000","last_fullsized":"2024-04-25T10:14:27.388879+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":603144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1396736,"data_stored":603144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":603144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509558,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36724,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335116,"statfs":{"total":96636764160,"available":96599158784,"internally_reserved":0,"allocated":1409024,"data_stored":609919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36712,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335128,"statfs":{"total":96636764160,"available":96599171072,"internally_reserved":0,"allocated":1396736,"data_stored":603144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509557,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37296,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334544,"statfs":{"total":96636764160,"available":96598573056,"internally_reserved":0,"allocated":2002944,"data_stored":1200287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1417216,"data_stored":610571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33160,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338680,"statfs":{"total":96636764160,"available":96602808320,"internally_reserved":0,"allocated":2023424,"data_stored":1206679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:14:30.398 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:14:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.530+0000 7f2042177640 1 Processor -- start 2024-04-25T10:14:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.530+0000 7f2042177640 1 -- start start 2024-04-25T10:14:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.531+0000 7f2042177640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f203c0d35b0 0x7f203c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.531+0000 7f2042177640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f203c0b5990 0x7f203c0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.531+0000 7f2042177640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203c15e0c0 con 0x7f203c15dce0 2024-04-25T10:14:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.531+0000 7f2042177640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f203c0d2510 con 0x7f203c0d35b0 2024-04-25T10:14:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.531+0000 7f2042177640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0d27f0 con 0x7f203c0b5990 2024-04-25T10:14:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.531+0000 7f203affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f203c0d35b0 0x7f203c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.531+0000 7f203affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f203c0d35b0 0x7f203c0b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39214/0 (socket says 172.21.15.16:39214) 2024-04-25T10:14:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.531+0000 7f203affd640 1 -- 172.21.15.16:0/3671695984 learned_addr learned my addr 172.21.15.16:0/3671695984 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.531+0000 7f203affd640 1 -- 172.21.15.16:0/3671695984 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f203c0b5990 msgr2=0x7f203c0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.531+0000 7f203bfff640 1 --2- 172.21.15.16:0/3671695984 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f203c0b5990 0x7f203c0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.531+0000 7f203affd640 1 --2- 172.21.15.16:0/3671695984 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f203c0b5990 0x7f203c0b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.531+0000 7f203affd640 1 -- 172.21.15.16:0/3671695984 >> v1:172.21.15.186:6789/0 conn(0x7f203c15dce0 legacy=0x7f203c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.531+0000 7f203affd640 1 -- 172.21.15.16:0/3671695984 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c0d2ad0 con 0x7f203c0d35b0 2024-04-25T10:14:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.532+0000 7f203affd640 1 --2- 172.21.15.16:0/3671695984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f203c0d35b0 0x7f203c0b5450 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f2030009870 tx=0x7f203002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988283605ef6c360 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.532+0000 7f203a7fc640 1 -- 172.21.15.16:0/3671695984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030037070 con 0x7f203c0d35b0 2024-04-25T10:14:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.532+0000 7f203a7fc640 1 -- 172.21.15.16:0/3671695984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2030032440 con 0x7f203c0d35b0 2024-04-25T10:14:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.532+0000 7f203a7fc640 1 -- 172.21.15.16:0/3671695984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030032810 con 0x7f203c0d35b0 2024-04-25T10:14:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.533+0000 7f2042177640 1 -- 172.21.15.16:0/3671695984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f203c0d35b0 msgr2=0x7f203c0b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.533+0000 7f2042177640 1 --2- 172.21.15.16:0/3671695984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f203c0d35b0 0x7f203c0b5450 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f2030009870 tx=0x7f203002a210 comp rx=0 tx=0).stop 2024-04-25T10:14:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.533+0000 7f2042177640 1 -- 172.21.15.16:0/3671695984 shutdown_connections 2024-04-25T10:14:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.533+0000 7f2042177640 1 --2- 172.21.15.16:0/3671695984 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f203c0b5990 0x7f203c0b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.533+0000 7f2042177640 1 --2- 172.21.15.16:0/3671695984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f203c0d35b0 0x7f203c0b5450 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.533+0000 7f2042177640 1 -- 172.21.15.16:0/3671695984 >> 172.21.15.16:0/3671695984 conn(0x7f203c072900 msgr2=0x7f203c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.533+0000 7f2042177640 1 -- 172.21.15.16:0/3671695984 shutdown_connections 2024-04-25T10:14:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.533+0000 7f2042177640 1 -- 172.21.15.16:0/3671695984 wait complete. 2024-04-25T10:14:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.534+0000 7f2042177640 1 Processor -- start 2024-04-25T10:14:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.534+0000 7f2042177640 1 -- start start 2024-04-25T10:14:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.535+0000 7f2042177640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f203c0b5990 0x7f203c158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.535+0000 7f2042177640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f203c0d35b0 0x7f203c1606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.535+0000 7f2042177640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f203c15dce0 0x7f203c0caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.535+0000 7f2042177640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0d2160 con 0x7f203c0d35b0 2024-04-25T10:14:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.535+0000 7f2042177640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0d3090 con 0x7f203c0b5990 2024-04-25T10:14:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.535+0000 7f2042177640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c159470 con 0x7f203c15dce0 2024-04-25T10:14:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.535+0000 7f203affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f203c0d35b0 0x7f203c1606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.535+0000 7f203affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f203c0d35b0 0x7f203c1606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39224/0 (socket says 172.21.15.16:39224) 2024-04-25T10:14:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.535+0000 7f203affd640 1 -- 172.21.15.16:0/3478557226 learned_addr learned my addr 172.21.15.16:0/3478557226 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.535+0000 7f203bfff640 1 --2- 172.21.15.16:0/3478557226 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f203c15dce0 0x7f203c0caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.535+0000 7f203b7fe640 1 --2- 172.21.15.16:0/3478557226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f203c0b5990 0x7f203c158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.535+0000 7f203affd640 1 -- 172.21.15.16:0/3478557226 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f203c15dce0 msgr2=0x7f203c0caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.535+0000 7f203affd640 1 --2- 172.21.15.16:0/3478557226 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f203c15dce0 0x7f203c0caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.535+0000 7f203affd640 1 -- 172.21.15.16:0/3478557226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f203c0b5990 msgr2=0x7f203c158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.535+0000 7f203affd640 1 --2- 172.21.15.16:0/3478557226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f203c0b5990 0x7f203c158f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.536+0000 7f203affd640 1 -- 172.21.15.16:0/3478557226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c0cd2a0 con 0x7f203c0d35b0 2024-04-25T10:14:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.536+0000 7f203affd640 1 --2- 172.21.15.16:0/3478557226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f203c0d35b0 0x7f203c1606a0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f2030009870 tx=0x7f20300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.536+0000 7f2023fff640 1 -- 172.21.15.16:0/3478557226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030037040 con 0x7f203c0d35b0 2024-04-25T10:14:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.536+0000 7f2023fff640 1 -- 172.21.15.16:0/3478557226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2030042070 con 0x7f203c0d35b0 2024-04-25T10:14:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.536+0000 7f2042177640 1 -- 172.21.15.16:0/3478557226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203c0cd580 con 0x7f203c0d35b0 2024-04-25T10:14:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.536+0000 7f2023fff640 1 -- 172.21.15.16:0/3478557226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030030bb0 con 0x7f203c0d35b0 2024-04-25T10:14:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.536+0000 7f2042177640 1 -- 172.21.15.16:0/3478557226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203c15fea0 con 0x7f203c0d35b0 2024-04-25T10:14:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.537+0000 7f2042177640 1 -- 172.21.15.16:0/3478557226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203c0749b0 con 0x7f203c0d35b0 2024-04-25T10:14:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.537+0000 7f2023fff640 1 -- 172.21.15.16:0/3478557226 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2030047460 con 0x7f203c0d35b0 2024-04-25T10:14:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.538+0000 7f2023fff640 1 --2- 172.21.15.16:0/3478557226 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f202806b4d0 0x7f202806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.538+0000 7f2023fff640 1 -- 172.21.15.16:0/3478557226 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20300aae30 con 0x7f203c0d35b0 2024-04-25T10:14:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.538+0000 7f203b7fe640 1 --2- 172.21.15.16:0/3478557226 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f202806b4d0 0x7f202806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.539+0000 7f203b7fe640 1 --2- 172.21.15.16:0/3478557226 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f202806b4d0 0x7f202806d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f202c006fd0 tx=0x7f202c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.542+0000 7f2023fff640 1 -- 172.21.15.16:0/3478557226 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2030045360 con 0x7f203c0d35b0 2024-04-25T10:14:30.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.720+0000 7f2042177640 1 -- 172.21.15.16:0/3478557226 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f203c0d15a0 con 0x7f202806b4d0 2024-04-25T10:14:30.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.727+0000 7f2023fff640 1 -- 172.21.15.16:0/3478557226 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252415 (secure 0 0 0) 0x7f203c0d15a0 con 0x7f202806b4d0 2024-04-25T10:14:30.733 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:30.735 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:14:30.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.730+0000 7f2042177640 1 -- 172.21.15.16:0/3478557226 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f202806b4d0 msgr2=0x7f202806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.730+0000 7f2042177640 1 --2- 172.21.15.16:0/3478557226 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f202806b4d0 0x7f202806d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f202c006fd0 tx=0x7f202c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:30.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.730+0000 7f2042177640 1 -- 172.21.15.16:0/3478557226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f203c0d35b0 msgr2=0x7f203c1606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.730+0000 7f2042177640 1 --2- 172.21.15.16:0/3478557226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f203c0d35b0 0x7f203c1606a0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f2030009870 tx=0x7f20300046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.730+0000 7f2042177640 1 -- 172.21.15.16:0/3478557226 shutdown_connections 2024-04-25T10:14:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.730+0000 7f2042177640 1 --2- 172.21.15.16:0/3478557226 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f202806b4d0 0x7f202806d990 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.730+0000 7f2042177640 1 --2- 172.21.15.16:0/3478557226 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f203c15dce0 0x7f203c0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.730+0000 7f2042177640 1 --2- 172.21.15.16:0/3478557226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f203c0d35b0 0x7f203c1606a0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.730+0000 7f2042177640 1 --2- 172.21.15.16:0/3478557226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f203c0b5990 0x7f203c158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.730+0000 7f2042177640 1 -- 172.21.15.16:0/3478557226 >> 172.21.15.16:0/3478557226 conn(0x7f203c072900 msgr2=0x7f203c0b7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.730+0000 7f2042177640 1 -- 172.21.15.16:0/3478557226 shutdown_connections 2024-04-25T10:14:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.730+0000 7f2042177640 1 -- 172.21.15.16:0/3478557226 wait complete. 2024-04-25T10:14:30.745 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-04-25T10:14:29.919313+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":274112,"kb_used_data":12148,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754700608,"statfs":{"total":773094113280,"available":772813422592,"internally_reserved":0,"allocated":12439552,"data_stored":6040032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.539548"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.407209+0000","last_change":"2024-04-25T10:14:27.400799+0000","last_active":"2024-04-25T10:14:27.407209+0000","last_peered":"2024-04-25T10:14:27.407209+0000","last_clean":"2024-04-25T10:14:27.407209+0000","last_became_active":"2024-04-25T10:14:27.400409+0000","last_became_peered":"2024-04-25T10:14:27.400409+0000","last_unstale":"2024-04-25T10:14:27.407209+0000","last_undegraded":"2024-04-25T10:14:27.407209+0000","last_fullsized":"2024-04-25T10:14:27.407209+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:45.905737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.411635+0000","last_change":"2024-04-25T10:14:27.408444+0000","last_active":"2024-04-25T10:14:27.411635+0000","last_peered":"2024-04-25T10:14:27.411635+0000","last_clean":"2024-04-25T10:14:27.411635+0000","last_became_active":"2024-04-25T10:14:27.408163+0000","last_became_peered":"2024-04-25T10:14:27.408163+0000","last_unstale":"2024-04-25T10:14:27.411635+0000","last_undegraded":"2024-04-25T10:14:27.411635+0000","last_fullsized":"2024-04-25T10:14:27.411635+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:36.716791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408446+0000","last_change":"2024-04-25T10:14:27.402639+0000","last_active":"2024-04-25T10:14:27.408446+0000","last_peered":"2024-04-25T10:14:27.408446+0000","last_clean":"2024-04-25T10:14:27.408446+0000","last_became_active":"2024-04-25T10:14:27.402387+0000","last_became_peered":"2024-04-25T10:14:27.402387+0000","last_unstale":"2024-04-25T10:14:27.408446+0000","last_undegraded":"2024-04-25T10:14:27.408446+0000","last_fullsized":"2024-04-25T10:14:27.408446+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:56.837933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.396299+0000","last_change":"2024-04-25T10:14:27.393304+0000","last_active":"2024-04-25T10:14:27.396299+0000","last_peered":"2024-04-25T10:14:27.396299+0000","last_clean":"2024-04-25T10:14:27.396299+0000","last_became_active":"2024-04-25T10:14:27.393003+0000","last_became_peered":"2024-04-25T10:14:27.393003+0000","last_unstale":"2024-04-25T10:14:27.396299+0000","last_undegraded":"2024-04-25T10:14:27.396299+0000","last_fullsized":"2024-04-25T10:14:27.396299+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:45.211167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.409647+0000","last_change":"2024-04-25T10:14:27.407044+0000","last_active":"2024-04-25T10:14:27.409647+0000","last_peered":"2024-04-25T10:14:27.409647+0000","last_clean":"2024-04-25T10:14:27.409647+0000","last_became_active":"2024-04-25T10:14:27.405699+0000","last_became_peered":"2024-04-25T10:14:27.405699+0000","last_unstale":"2024-04-25T10:14:27.409647+0000","last_undegraded":"2024-04-25T10:14:27.409647+0000","last_fullsized":"2024-04-25T10:14:27.409647+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:18.842796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.264756+0000","last_change":"2024-04-25T10:14:27.403414+0000","last_active":"2024-04-25T10:14:29.264756+0000","last_peered":"2024-04-25T10:14:29.264756+0000","last_clean":"2024-04-25T10:14:29.264756+0000","last_became_active":"2024-04-25T10:14:27.403093+0000","last_became_peered":"2024-04-25T10:14:27.403093+0000","last_unstale":"2024-04-25T10:14:29.264756+0000","last_undegraded":"2024-04-25T10:14:29.264756+0000","last_fullsized":"2024-04-25T10:14:29.264756+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:09.915978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.404949+0000","last_change":"2024-04-25T10:14:27.403269+0000","last_active":"2024-04-25T10:14:27.404949+0000","last_peered":"2024-04-25T10:14:27.404949+0000","last_clean":"2024-04-25T10:14:27.404949+0000","last_became_active":"2024-04-25T10:14:27.402988+0000","last_became_peered":"2024-04-25T10:14:27.402988+0000","last_unstale":"2024-04-25T10:14:27.404949+0000","last_undegraded":"2024-04-25T10:14:27.404949+0000","last_fullsized":"2024-04-25T10:14:27.404949+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:10.557283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.409848+0000","last_change":"2024-04-25T10:14:27.408306+0000","last_active":"2024-04-25T10:14:27.409848+0000","last_peered":"2024-04-25T10:14:27.409848+0000","last_clean":"2024-04-25T10:14:27.409848+0000","last_became_active":"2024-04-25T10:14:27.408011+0000","last_became_peered":"2024-04-25T10:14:27.408011+0000","last_unstale":"2024-04-25T10:14:27.409848+0000","last_undegraded":"2024-04-25T10:14:27.409848+0000","last_fullsized":"2024-04-25T10:14:27.409848+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:22.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.407952+0000","last_change":"2024-04-25T10:14:27.406151+0000","last_active":"2024-04-25T10:14:27.407952+0000","last_peered":"2024-04-25T10:14:27.407952+0000","last_clean":"2024-04-25T10:14:27.407952+0000","last_became_active":"2024-04-25T10:14:27.405897+0000","last_became_peered":"2024-04-25T10:14:27.405897+0000","last_unstale":"2024-04-25T10:14:27.407952+0000","last_undegraded":"2024-04-25T10:14:27.407952+0000","last_fullsized":"2024-04-25T10:14:27.407952+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:17.554024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.401731+0000","last_change":"2024-04-25T10:14:27.397952+0000","last_active":"2024-04-25T10:14:27.401731+0000","last_peered":"2024-04-25T10:14:27.401731+0000","last_clean":"2024-04-25T10:14:27.401731+0000","last_became_active":"2024-04-25T10:14:27.397657+0000","last_became_peered":"2024-04-25T10:14:27.397657+0000","last_unstale":"2024-04-25T10:14:27.401731+0000","last_undegraded":"2024-04-25T10:14:27.401731+0000","last_fullsized":"2024-04-25T10:14:27.401731+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:33.637080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.404226+0000","last_change":"2024-04-25T10:14:27.397757+0000","last_active":"2024-04-25T10:14:27.404226+0000","last_peered":"2024-04-25T10:14:27.404226+0000","last_clean":"2024-04-25T10:14:27.404226+0000","last_became_active":"2024-04-25T10:14:27.397495+0000","last_became_peered":"2024-04-25T10:14:27.397495+0000","last_unstale":"2024-04-25T10:14:27.404226+0000","last_undegraded":"2024-04-25T10:14:27.404226+0000","last_fullsized":"2024-04-25T10:14:27.404226+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:58.449267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.398264+0000","last_change":"2024-04-25T10:14:27.397378+0000","last_active":"2024-04-25T10:14:27.398264+0000","last_peered":"2024-04-25T10:14:27.398264+0000","last_clean":"2024-04-25T10:14:27.398264+0000","last_became_active":"2024-04-25T10:14:27.397088+0000","last_became_peered":"2024-04-25T10:14:27.397088+0000","last_unstale":"2024-04-25T10:14:27.398264+0000","last_undegraded":"2024-04-25T10:14:27.398264+0000","last_fullsized":"2024-04-25T10:14:27.398264+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:06.765935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.401974+0000","last_change":"2024-04-25T10:14:27.394831+0000","last_active":"2024-04-25T10:14:27.401974+0000","last_peered":"2024-04-25T10:14:27.401974+0000","last_clean":"2024-04-25T10:14:27.401974+0000","last_became_active":"2024-04-25T10:14:27.393997+0000","last_became_peered":"2024-04-25T10:14:27.393997+0000","last_unstale":"2024-04-25T10:14:27.401974+0000","last_undegraded":"2024-04-25T10:14:27.401974+0000","last_fullsized":"2024-04-25T10:14:27.401974+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:42.494044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.405827+0000","last_change":"2024-04-25T10:14:27.398094+0000","last_active":"2024-04-25T10:14:27.405827+0000","last_peered":"2024-04-25T10:14:27.405827+0000","last_clean":"2024-04-25T10:14:27.405827+0000","last_became_active":"2024-04-25T10:14:27.397805+0000","last_became_peered":"2024-04-25T10:14:27.397805+0000","last_unstale":"2024-04-25T10:14:27.405827+0000","last_undegraded":"2024-04-25T10:14:27.405827+0000","last_fullsized":"2024-04-25T10:14:27.405827+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:02.970569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408202+0000","last_change":"2024-04-25T10:14:27.404203+0000","last_active":"2024-04-25T10:14:27.408202+0000","last_peered":"2024-04-25T10:14:27.408202+0000","last_clean":"2024-04-25T10:14:27.408202+0000","last_became_active":"2024-04-25T10:14:27.403913+0000","last_became_peered":"2024-04-25T10:14:27.403913+0000","last_unstale":"2024-04-25T10:14:27.408202+0000","last_undegraded":"2024-04-25T10:14:27.408202+0000","last_fullsized":"2024-04-25T10:14:27.408202+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:26.494609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.403791+0000","last_change":"2024-04-25T10:14:27.401097+0000","last_active":"2024-04-25T10:14:27.403791+0000","last_peered":"2024-04-25T10:14:27.403791+0000","last_clean":"2024-04-25T10:14:27.403791+0000","last_became_active":"2024-04-25T10:14:27.400760+0000","last_became_peered":"2024-04-25T10:14:27.400760+0000","last_unstale":"2024-04-25T10:14:27.403791+0000","last_undegraded":"2024-04-25T10:14:27.403791+0000","last_fullsized":"2024-04-25T10:14:27.403791+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:14.560458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.401827+0000","last_change":"2024-04-25T10:14:27.399801+0000","last_active":"2024-04-25T10:14:27.401827+0000","last_peered":"2024-04-25T10:14:27.401827+0000","last_clean":"2024-04-25T10:14:27.401827+0000","last_became_active":"2024-04-25T10:14:27.399508+0000","last_became_peered":"2024-04-25T10:14:27.399508+0000","last_unstale":"2024-04-25T10:14:27.401827+0000","last_undegraded":"2024-04-25T10:14:27.401827+0000","last_fullsized":"2024-04-25T10:14:27.401827+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:45.653282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412921+0000","last_change":"2024-04-25T10:14:27.411850+0000","last_active":"2024-04-25T10:14:27.412921+0000","last_peered":"2024-04-25T10:14:27.412921+0000","last_clean":"2024-04-25T10:14:27.412921+0000","last_became_active":"2024-04-25T10:14:27.411545+0000","last_became_peered":"2024-04-25T10:14:27.411545+0000","last_unstale":"2024-04-25T10:14:27.412921+0000","last_undegraded":"2024-04-25T10:14:27.412921+0000","last_fullsized":"2024-04-25T10:14:27.412921+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:11.105430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408465+0000","last_change":"2024-04-25T10:14:27.402249+0000","last_active":"2024-04-25T10:14:27.408465+0000","last_peered":"2024-04-25T10:14:27.408465+0000","last_clean":"2024-04-25T10:14:27.408465+0000","last_became_active":"2024-04-25T10:14:27.401947+0000","last_became_peered":"2024-04-25T10:14:27.401947+0000","last_unstale":"2024-04-25T10:14:27.408465+0000","last_undegraded":"2024-04-25T10:14:27.408465+0000","last_fullsized":"2024-04-25T10:14:27.408465+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:02.776562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.401761+0000","last_change":"2024-04-25T10:14:27.399340+0000","last_active":"2024-04-25T10:14:27.401761+0000","last_peered":"2024-04-25T10:14:27.401761+0000","last_clean":"2024-04-25T10:14:27.401761+0000","last_became_active":"2024-04-25T10:14:27.399054+0000","last_became_peered":"2024-04-25T10:14:27.399054+0000","last_unstale":"2024-04-25T10:14:27.401761+0000","last_undegraded":"2024-04-25T10:14:27.401761+0000","last_fullsized":"2024-04-25T10:14:27.401761+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:34.465544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.403405+0000","last_change":"2024-04-25T10:14:27.396410+0000","last_active":"2024-04-25T10:14:27.403405+0000","last_peered":"2024-04-25T10:14:27.403405+0000","last_clean":"2024-04-25T10:14:27.403405+0000","last_became_active":"2024-04-25T10:14:27.395475+0000","last_became_peered":"2024-04-25T10:14:27.395475+0000","last_unstale":"2024-04-25T10:14:27.403405+0000","last_undegraded":"2024-04-25T10:14:27.403405+0000","last_fullsized":"2024-04-25T10:14:27.403405+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:08.179273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.265072+0000","last_change":"2024-04-25T10:14:27.399279+0000","last_active":"2024-04-25T10:14:29.265072+0000","last_peered":"2024-04-25T10:14:29.265072+0000","last_clean":"2024-04-25T10:14:29.265072+0000","last_became_active":"2024-04-25T10:14:27.399011+0000","last_became_peered":"2024-04-25T10:14:27.399011+0000","last_unstale":"2024-04-25T10:14:29.265072+0000","last_undegraded":"2024-04-25T10:14:29.265072+0000","last_fullsized":"2024-04-25T10:14:29.265072+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:24.527238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412751+0000","last_change":"2024-04-25T10:14:27.411194+0000","last_active":"2024-04-25T10:14:27.412751+0000","last_peered":"2024-04-25T10:14:27.412751+0000","last_clean":"2024-04-25T10:14:27.412751+0000","last_became_active":"2024-04-25T10:14:27.410923+0000","last_became_peered":"2024-04-25T10:14:27.410923+0000","last_unstale":"2024-04-25T10:14:27.412751+0000","last_undegraded":"2024-04-25T10:14:27.412751+0000","last_fullsized":"2024-04-25T10:14:27.412751+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:45.905737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.411261+0000","last_change":"2024-04-25T10:14:27.408747+0000","last_active":"2024-04-25T10:14:27.411261+0000","last_peered":"2024-04-25T10:14:27.411261+0000","last_clean":"2024-04-25T10:14:27.411261+0000","last_became_active":"2024-04-25T10:14:27.407960+0000","last_became_peered":"2024-04-25T10:14:27.407960+0000","last_unstale":"2024-04-25T10:14:27.411261+0000","last_undegraded":"2024-04-25T10:14:27.411261+0000","last_fullsized":"2024-04-25T10:14:27.411261+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:38.602823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.404900+0000","last_change":"2024-04-25T10:14:27.402897+0000","last_active":"2024-04-25T10:14:27.404900+0000","last_peered":"2024-04-25T10:14:27.404900+0000","last_clean":"2024-04-25T10:14:27.404900+0000","last_became_active":"2024-04-25T10:14:27.402613+0000","last_became_peered":"2024-04-25T10:14:27.402613+0000","last_unstale":"2024-04-25T10:14:27.404900+0000","last_undegraded":"2024-04-25T10:14:27.404900+0000","last_fullsized":"2024-04-25T10:14:27.404900+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:10.245804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.265369+0000","last_change":"2024-04-25T10:14:27.401114+0000","last_active":"2024-04-25T10:14:29.265369+0000","last_peered":"2024-04-25T10:14:29.265369+0000","last_clean":"2024-04-25T10:14:29.265369+0000","last_became_active":"2024-04-25T10:14:27.400762+0000","last_became_peered":"2024-04-25T10:14:27.400762+0000","last_unstale":"2024-04-25T10:14:29.265369+0000","last_undegraded":"2024-04-25T10:14:29.265369+0000","last_fullsized":"2024-04-25T10:14:29.265369+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:13.125496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412095+0000","last_change":"2024-04-25T10:14:27.409296+0000","last_active":"2024-04-25T10:14:27.412095+0000","last_peered":"2024-04-25T10:14:27.412095+0000","last_clean":"2024-04-25T10:14:27.412095+0000","last_became_active":"2024-04-25T10:14:27.406648+0000","last_became_peered":"2024-04-25T10:14:27.406648+0000","last_unstale":"2024-04-25T10:14:27.412095+0000","last_undegraded":"2024-04-25T10:14:27.412095+0000","last_fullsized":"2024-04-25T10:14:27.412095+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:48.523576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.265146+0000","last_change":"2024-04-25T10:14:27.407877+0000","last_active":"2024-04-25T10:14:29.265146+0000","last_peered":"2024-04-25T10:14:29.265146+0000","last_clean":"2024-04-25T10:14:29.265146+0000","last_became_active":"2024-04-25T10:14:27.407578+0000","last_became_peered":"2024-04-25T10:14:27.407578+0000","last_unstale":"2024-04-25T10:14:29.265146+0000","last_undegraded":"2024-04-25T10:14:29.265146+0000","last_fullsized":"2024-04-25T10:14:29.265146+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.616799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.405965+0000","last_change":"2024-04-25T10:14:27.399018+0000","last_active":"2024-04-25T10:14:27.405965+0000","last_peered":"2024-04-25T10:14:27.405965+0000","last_clean":"2024-04-25T10:14:27.405965+0000","last_became_active":"2024-04-25T10:14:27.398721+0000","last_became_peered":"2024-04-25T10:14:27.398721+0000","last_unstale":"2024-04-25T10:14:27.405965+0000","last_undegraded":"2024-04-25T10:14:27.405965+0000","last_fullsized":"2024-04-25T10:14:27.405965+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:12.030595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.404175+0000","last_change":"2024-04-25T10:14:27.396648+0000","last_active":"2024-04-25T10:14:27.404175+0000","last_peered":"2024-04-25T10:14:27.404175+0000","last_clean":"2024-04-25T10:14:27.404175+0000","last_became_active":"2024-04-25T10:14:27.396271+0000","last_became_peered":"2024-04-25T10:14:27.396271+0000","last_unstale":"2024-04-25T10:14:27.404175+0000","last_undegraded":"2024-04-25T10:14:27.404175+0000","last_fullsized":"2024-04-25T10:14:27.404175+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:44.732911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408221+0000","last_change":"2024-04-25T10:14:27.405661+0000","last_active":"2024-04-25T10:14:27.408221+0000","last_peered":"2024-04-25T10:14:27.408221+0000","last_clean":"2024-04-25T10:14:27.408221+0000","last_became_active":"2024-04-25T10:14:27.405363+0000","last_became_peered":"2024-04-25T10:14:27.405363+0000","last_unstale":"2024-04-25T10:14:27.408221+0000","last_undegraded":"2024-04-25T10:14:27.408221+0000","last_fullsized":"2024-04-25T10:14:27.408221+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:23.505083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.405545+0000","last_change":"2024-04-25T10:14:27.400722+0000","last_active":"2024-04-25T10:14:27.405545+0000","last_peered":"2024-04-25T10:14:27.405545+0000","last_clean":"2024-04-25T10:14:27.405545+0000","last_became_active":"2024-04-25T10:14:27.400369+0000","last_became_peered":"2024-04-25T10:14:27.400369+0000","last_unstale":"2024-04-25T10:14:27.405545+0000","last_undegraded":"2024-04-25T10:14:27.405545+0000","last_fullsized":"2024-04-25T10:14:27.405545+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:34.465544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.414501+0000","last_change":"2024-04-25T10:14:27.412826+0000","last_active":"2024-04-25T10:14:27.414501+0000","last_peered":"2024-04-25T10:14:27.414501+0000","last_clean":"2024-04-25T10:14:27.414501+0000","last_became_active":"2024-04-25T10:14:27.412521+0000","last_became_peered":"2024-04-25T10:14:27.412521+0000","last_unstale":"2024-04-25T10:14:27.414501+0000","last_undegraded":"2024-04-25T10:14:27.414501+0000","last_fullsized":"2024-04-25T10:14:27.414501+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:12.595508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"114'4","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.414691+0000","last_change":"2024-04-25T10:14:26.394128+0000","last_active":"2024-04-25T10:14:27.414691+0000","last_peered":"2024-04-25T10:14:27.414691+0000","last_clean":"2024-04-25T10:14:27.414691+0000","last_became_active":"2024-04-25T10:14:26.393815+0000","last_became_peered":"2024-04-25T10:14:26.393815+0000","last_unstale":"2024-04-25T10:14:27.414691+0000","last_undegraded":"2024-04-25T10:14:27.414691+0000","last_fullsized":"2024-04-25T10:14:27.414691+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:53.877050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412787+0000","last_change":"2024-04-25T10:14:27.411129+0000","last_active":"2024-04-25T10:14:27.412787+0000","last_peered":"2024-04-25T10:14:27.412787+0000","last_clean":"2024-04-25T10:14:27.412787+0000","last_became_active":"2024-04-25T10:14:27.410834+0000","last_became_peered":"2024-04-25T10:14:27.410834+0000","last_unstale":"2024-04-25T10:14:27.412787+0000","last_undegraded":"2024-04-25T10:14:27.412787+0000","last_fullsized":"2024-04-25T10:14:27.412787+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:45.211167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"114'1","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412419+0000","last_change":"2024-04-25T10:14:26.387502+0000","last_active":"2024-04-25T10:14:27.412419+0000","last_peered":"2024-04-25T10:14:27.412419+0000","last_clean":"2024-04-25T10:14:27.412419+0000","last_became_active":"2024-04-25T10:14:26.387228+0000","last_became_peered":"2024-04-25T10:14:26.387228+0000","last_unstale":"2024-04-25T10:14:27.412419+0000","last_undegraded":"2024-04-25T10:14:27.412419+0000","last_fullsized":"2024-04-25T10:14:27.412419+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:54.436383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.265197+0000","last_change":"2024-04-25T10:14:27.403563+0000","last_active":"2024-04-25T10:14:29.265197+0000","last_peered":"2024-04-25T10:14:29.265197+0000","last_clean":"2024-04-25T10:14:29.265197+0000","last_became_active":"2024-04-25T10:14:27.403237+0000","last_became_peered":"2024-04-25T10:14:27.403237+0000","last_unstale":"2024-04-25T10:14:29.265197+0000","last_undegraded":"2024-04-25T10:14:29.265197+0000","last_fullsized":"2024-04-25T10:14:29.265197+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:09.422709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"114'2","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412043+0000","last_change":"2024-04-25T10:14:26.401039+0000","last_active":"2024-04-25T10:14:27.412043+0000","last_peered":"2024-04-25T10:14:27.412043+0000","last_clean":"2024-04-25T10:14:27.412043+0000","last_became_active":"2024-04-25T10:14:26.400766+0000","last_became_peered":"2024-04-25T10:14:26.400766+0000","last_unstale":"2024-04-25T10:14:27.412043+0000","last_undegraded":"2024-04-25T10:14:27.412043+0000","last_fullsized":"2024-04-25T10:14:27.412043+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:11.028136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408424+0000","last_change":"2024-04-25T10:14:27.402058+0000","last_active":"2024-04-25T10:14:27.408424+0000","last_peered":"2024-04-25T10:14:27.408424+0000","last_clean":"2024-04-25T10:14:27.408424+0000","last_became_active":"2024-04-25T10:14:27.401570+0000","last_became_peered":"2024-04-25T10:14:27.401570+0000","last_unstale":"2024-04-25T10:14:27.408424+0000","last_undegraded":"2024-04-25T10:14:27.408424+0000","last_fullsized":"2024-04-25T10:14:27.408424+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:18.842796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"114'2","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.413228+0000","last_change":"2024-04-25T10:14:26.401991+0000","last_active":"2024-04-25T10:14:27.413228+0000","last_peered":"2024-04-25T10:14:27.413228+0000","last_clean":"2024-04-25T10:14:27.413228+0000","last_became_active":"2024-04-25T10:14:26.401650+0000","last_became_peered":"2024-04-25T10:14:26.401650+0000","last_unstale":"2024-04-25T10:14:27.413228+0000","last_undegraded":"2024-04-25T10:14:27.413228+0000","last_fullsized":"2024-04-25T10:14:27.413228+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:43.623099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.411712+0000","last_change":"2024-04-25T10:14:27.409672+0000","last_active":"2024-04-25T10:14:27.411712+0000","last_peered":"2024-04-25T10:14:27.411712+0000","last_clean":"2024-04-25T10:14:27.411712+0000","last_became_active":"2024-04-25T10:14:27.408297+0000","last_became_peered":"2024-04-25T10:14:27.408297+0000","last_unstale":"2024-04-25T10:14:27.411712+0000","last_undegraded":"2024-04-25T10:14:27.411712+0000","last_fullsized":"2024-04-25T10:14:27.411712+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:23.505083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"114'1","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.407126+0000","last_change":"2024-04-25T10:14:26.396449+0000","last_active":"2024-04-25T10:14:27.407126+0000","last_peered":"2024-04-25T10:14:27.407126+0000","last_clean":"2024-04-25T10:14:27.407126+0000","last_became_active":"2024-04-25T10:14:26.396146+0000","last_became_peered":"2024-04-25T10:14:26.396146+0000","last_unstale":"2024-04-25T10:14:27.407126+0000","last_undegraded":"2024-04-25T10:14:27.407126+0000","last_fullsized":"2024-04-25T10:14:27.407126+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:22.079207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.404943+0000","last_change":"2024-04-25T10:14:27.403529+0000","last_active":"2024-04-25T10:14:27.404943+0000","last_peered":"2024-04-25T10:14:27.404943+0000","last_clean":"2024-04-25T10:14:27.404943+0000","last_became_active":"2024-04-25T10:14:27.403204+0000","last_became_peered":"2024-04-25T10:14:27.403204+0000","last_unstale":"2024-04-25T10:14:27.404943+0000","last_undegraded":"2024-04-25T10:14:27.404943+0000","last_fullsized":"2024-04-25T10:14:27.404943+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:44.353580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"114'1","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.415451+0000","last_change":"2024-04-25T10:14:26.395516+0000","last_active":"2024-04-25T10:14:27.415451+0000","last_peered":"2024-04-25T10:14:27.415451+0000","last_clean":"2024-04-25T10:14:27.415451+0000","last_became_active":"2024-04-25T10:14:26.395240+0000","last_became_peered":"2024-04-25T10:14:26.395240+0000","last_unstale":"2024-04-25T10:14:27.415451+0000","last_undegraded":"2024-04-25T10:14:27.415451+0000","last_fullsized":"2024-04-25T10:14:27.415451+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:35.714332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.403773+0000","last_change":"2024-04-25T10:14:27.398047+0000","last_active":"2024-04-25T10:14:27.403773+0000","last_peered":"2024-04-25T10:14:27.403773+0000","last_clean":"2024-04-25T10:14:27.403773+0000","last_became_active":"2024-04-25T10:14:27.397779+0000","last_became_peered":"2024-04-25T10:14:27.397779+0000","last_unstale":"2024-04-25T10:14:27.403773+0000","last_undegraded":"2024-04-25T10:14:27.403773+0000","last_fullsized":"2024-04-25T10:14:27.403773+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:23.081666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.388084+0000","last_change":"2024-04-25T10:14:26.399334+0000","last_active":"2024-04-25T10:14:27.388084+0000","last_peered":"2024-04-25T10:14:27.388084+0000","last_clean":"2024-04-25T10:14:27.388084+0000","last_became_active":"2024-04-25T10:14:26.399009+0000","last_became_peered":"2024-04-25T10:14:26.399009+0000","last_unstale":"2024-04-25T10:14:27.388084+0000","last_undegraded":"2024-04-25T10:14:27.388084+0000","last_fullsized":"2024-04-25T10:14:27.388084+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:48.181442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.404967+0000","last_change":"2024-04-25T10:14:27.403852+0000","last_active":"2024-04-25T10:14:27.404967+0000","last_peered":"2024-04-25T10:14:27.404967+0000","last_clean":"2024-04-25T10:14:27.404967+0000","last_became_active":"2024-04-25T10:14:27.403594+0000","last_became_peered":"2024-04-25T10:14:27.403594+0000","last_unstale":"2024-04-25T10:14:27.404967+0000","last_undegraded":"2024-04-25T10:14:27.404967+0000","last_fullsized":"2024-04-25T10:14:27.404967+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:30.054392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.390987+0000","last_change":"2024-04-25T10:14:26.403085+0000","last_active":"2024-04-25T10:14:27.390987+0000","last_peered":"2024-04-25T10:14:27.390987+0000","last_clean":"2024-04-25T10:14:27.390987+0000","last_became_active":"2024-04-25T10:14:26.402789+0000","last_became_peered":"2024-04-25T10:14:26.402789+0000","last_unstale":"2024-04-25T10:14:27.390987+0000","last_undegraded":"2024-04-25T10:14:27.390987+0000","last_fullsized":"2024-04-25T10:14:27.390987+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:40.375833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.411662+0000","last_change":"2024-04-25T10:14:27.408461+0000","last_active":"2024-04-25T10:14:27.411662+0000","last_peered":"2024-04-25T10:14:27.411662+0000","last_clean":"2024-04-25T10:14:27.411662+0000","last_became_active":"2024-04-25T10:14:27.408180+0000","last_became_peered":"2024-04-25T10:14:27.408180+0000","last_unstale":"2024-04-25T10:14:27.411662+0000","last_undegraded":"2024-04-25T10:14:27.411662+0000","last_fullsized":"2024-04-25T10:14:27.411662+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:45.211167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.387604+0000","last_change":"2024-04-25T10:14:26.398102+0000","last_active":"2024-04-25T10:14:27.387604+0000","last_peered":"2024-04-25T10:14:27.387604+0000","last_clean":"2024-04-25T10:14:27.387604+0000","last_became_active":"2024-04-25T10:14:26.397805+0000","last_became_peered":"2024-04-25T10:14:26.397805+0000","last_unstale":"2024-04-25T10:14:27.387604+0000","last_undegraded":"2024-04-25T10:14:27.387604+0000","last_fullsized":"2024-04-25T10:14:27.387604+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:55.437165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.401719+0000","last_change":"2024-04-25T10:14:27.398503+0000","last_active":"2024-04-25T10:14:27.401719+0000","last_peered":"2024-04-25T10:14:27.401719+0000","last_clean":"2024-04-25T10:14:27.401719+0000","last_became_active":"2024-04-25T10:14:27.397940+0000","last_became_peered":"2024-04-25T10:14:27.397940+0000","last_unstale":"2024-04-25T10:14:27.401719+0000","last_undegraded":"2024-04-25T10:14:27.401719+0000","last_fullsized":"2024-04-25T10:14:27.401719+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:21.027596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.393191+0000","last_change":"2024-04-25T10:14:26.401312+0000","last_active":"2024-04-25T10:14:27.393191+0000","last_peered":"2024-04-25T10:14:27.393191+0000","last_clean":"2024-04-25T10:14:27.393191+0000","last_became_active":"2024-04-25T10:14:26.401047+0000","last_became_peered":"2024-04-25T10:14:26.401047+0000","last_unstale":"2024-04-25T10:14:27.393191+0000","last_undegraded":"2024-04-25T10:14:27.393191+0000","last_fullsized":"2024-04-25T10:14:27.393191+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:55.080445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.265340+0000","last_change":"2024-04-25T10:14:27.404853+0000","last_active":"2024-04-25T10:14:29.265340+0000","last_peered":"2024-04-25T10:14:29.265340+0000","last_clean":"2024-04-25T10:14:29.265340+0000","last_became_active":"2024-04-25T10:14:27.404538+0000","last_became_peered":"2024-04-25T10:14:27.404538+0000","last_unstale":"2024-04-25T10:14:29.265340+0000","last_undegraded":"2024-04-25T10:14:29.265340+0000","last_fullsized":"2024-04-25T10:14:29.265340+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:48.945878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"114'3","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.414040+0000","last_change":"2024-04-25T10:14:26.402072+0000","last_active":"2024-04-25T10:14:27.414040+0000","last_peered":"2024-04-25T10:14:27.414040+0000","last_clean":"2024-04-25T10:14:27.414040+0000","last_became_active":"2024-04-25T10:14:26.401803+0000","last_became_peered":"2024-04-25T10:14:26.401803+0000","last_unstale":"2024-04-25T10:14:27.414040+0000","last_undegraded":"2024-04-25T10:14:27.414040+0000","last_fullsized":"2024-04-25T10:14:27.414040+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:13.625722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.404717+0000","last_change":"2024-04-25T10:14:27.402474+0000","last_active":"2024-04-25T10:14:27.404717+0000","last_peered":"2024-04-25T10:14:27.404717+0000","last_clean":"2024-04-25T10:14:27.404717+0000","last_became_active":"2024-04-25T10:14:27.402196+0000","last_became_peered":"2024-04-25T10:14:27.402196+0000","last_unstale":"2024-04-25T10:14:27.404717+0000","last_undegraded":"2024-04-25T10:14:27.404717+0000","last_fullsized":"2024-04-25T10:14:27.404717+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:10.484587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"114'6","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.416746+0000","last_change":"2024-04-25T10:14:26.398379+0000","last_active":"2024-04-25T10:14:27.416746+0000","last_peered":"2024-04-25T10:14:27.416746+0000","last_clean":"2024-04-25T10:14:27.416746+0000","last_became_active":"2024-04-25T10:14:26.398058+0000","last_became_peered":"2024-04-25T10:14:26.398058+0000","last_unstale":"2024-04-25T10:14:27.416746+0000","last_undegraded":"2024-04-25T10:14:27.416746+0000","last_fullsized":"2024-04-25T10:14:27.416746+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:35.714332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408552+0000","last_change":"2024-04-25T10:14:27.400833+0000","last_active":"2024-04-25T10:14:27.408552+0000","last_peered":"2024-04-25T10:14:27.408552+0000","last_clean":"2024-04-25T10:14:27.408552+0000","last_became_active":"2024-04-25T10:14:27.400548+0000","last_became_peered":"2024-04-25T10:14:27.400548+0000","last_unstale":"2024-04-25T10:14:27.408552+0000","last_undegraded":"2024-04-25T10:14:27.408552+0000","last_fullsized":"2024-04-25T10:14:27.408552+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:45.653282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"114'2","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412390+0000","last_change":"2024-04-25T10:14:26.401794+0000","last_active":"2024-04-25T10:14:27.412390+0000","last_peered":"2024-04-25T10:14:27.412390+0000","last_clean":"2024-04-25T10:14:27.412390+0000","last_became_active":"2024-04-25T10:14:26.401457+0000","last_became_peered":"2024-04-25T10:14:26.401457+0000","last_unstale":"2024-04-25T10:14:27.412390+0000","last_undegraded":"2024-04-25T10:14:27.412390+0000","last_fullsized":"2024-04-25T10:14:27.412390+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:42.589728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.406174+0000","last_change":"2024-04-25T10:14:27.398829+0000","last_active":"2024-04-25T10:14:27.406174+0000","last_peered":"2024-04-25T10:14:27.406174+0000","last_clean":"2024-04-25T10:14:27.406174+0000","last_became_active":"2024-04-25T10:14:27.398570+0000","last_became_peered":"2024-04-25T10:14:27.398570+0000","last_unstale":"2024-04-25T10:14:27.406174+0000","last_undegraded":"2024-04-25T10:14:27.406174+0000","last_fullsized":"2024-04-25T10:14:27.406174+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:26.494609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.389889+0000","last_change":"2024-04-25T10:14:26.394187+0000","last_active":"2024-04-25T10:14:27.389889+0000","last_peered":"2024-04-25T10:14:27.389889+0000","last_clean":"2024-04-25T10:14:27.389889+0000","last_became_active":"2024-04-25T10:14:26.393904+0000","last_became_peered":"2024-04-25T10:14:26.393904+0000","last_unstale":"2024-04-25T10:14:27.389889+0000","last_undegraded":"2024-04-25T10:14:27.389889+0000","last_fullsized":"2024-04-25T10:14:27.389889+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:20.769896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.413037+0000","last_change":"2024-04-25T10:14:27.412112+0000","last_active":"2024-04-25T10:14:27.413037+0000","last_peered":"2024-04-25T10:14:27.413037+0000","last_clean":"2024-04-25T10:14:27.413037+0000","last_became_active":"2024-04-25T10:14:27.411847+0000","last_became_peered":"2024-04-25T10:14:27.411847+0000","last_unstale":"2024-04-25T10:14:27.413037+0000","last_undegraded":"2024-04-25T10:14:27.413037+0000","last_fullsized":"2024-04-25T10:14:27.413037+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:17.554024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"114'8","reported_seq":20,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.419427+0000","last_change":"2024-04-25T10:14:26.396288+0000","last_active":"2024-04-25T10:14:27.419427+0000","last_peered":"2024-04-25T10:14:27.419427+0000","last_clean":"2024-04-25T10:14:27.419427+0000","last_became_active":"2024-04-25T10:14:26.396004+0000","last_became_peered":"2024-04-25T10:14:26.396004+0000","last_unstale":"2024-04-25T10:14:27.419427+0000","last_undegraded":"2024-04-25T10:14:27.419427+0000","last_fullsized":"2024-04-25T10:14:27.419427+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:41.491585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.411574+0000","last_change":"2024-04-25T10:14:27.408290+0000","last_active":"2024-04-25T10:14:27.411574+0000","last_peered":"2024-04-25T10:14:27.411574+0000","last_clean":"2024-04-25T10:14:27.411574+0000","last_became_active":"2024-04-25T10:14:27.407717+0000","last_became_peered":"2024-04-25T10:14:27.407717+0000","last_unstale":"2024-04-25T10:14:27.411574+0000","last_undegraded":"2024-04-25T10:14:27.411574+0000","last_fullsized":"2024-04-25T10:14:27.411574+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:45.369856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"114'1","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.265452+0000","last_change":"2024-04-25T10:14:26.400227+0000","last_active":"2024-04-25T10:14:29.265452+0000","last_peered":"2024-04-25T10:14:29.265452+0000","last_clean":"2024-04-25T10:14:29.265452+0000","last_became_active":"2024-04-25T10:14:26.399515+0000","last_became_peered":"2024-04-25T10:14:26.399515+0000","last_unstale":"2024-04-25T10:14:29.265452+0000","last_undegraded":"2024-04-25T10:14:29.265452+0000","last_fullsized":"2024-04-25T10:14:29.265452+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:10.867163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.407969+0000","last_change":"2024-04-25T10:14:27.406595+0000","last_active":"2024-04-25T10:14:27.407969+0000","last_peered":"2024-04-25T10:14:27.407969+0000","last_clean":"2024-04-25T10:14:27.407969+0000","last_became_active":"2024-04-25T10:14:27.406314+0000","last_became_peered":"2024-04-25T10:14:27.406314+0000","last_unstale":"2024-04-25T10:14:27.407969+0000","last_undegraded":"2024-04-25T10:14:27.407969+0000","last_fullsized":"2024-04-25T10:14:27.407969+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:01.623749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"114'2","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412345+0000","last_change":"2024-04-25T10:14:26.401790+0000","last_active":"2024-04-25T10:14:27.412345+0000","last_peered":"2024-04-25T10:14:27.412345+0000","last_clean":"2024-04-25T10:14:27.412345+0000","last_became_active":"2024-04-25T10:14:26.401472+0000","last_became_peered":"2024-04-25T10:14:26.401472+0000","last_unstale":"2024-04-25T10:14:27.412345+0000","last_undegraded":"2024-04-25T10:14:27.412345+0000","last_fullsized":"2024-04-25T10:14:27.412345+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:52.318817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412894+0000","last_change":"2024-04-25T10:14:27.411144+0000","last_active":"2024-04-25T10:14:27.412894+0000","last_peered":"2024-04-25T10:14:27.412894+0000","last_clean":"2024-04-25T10:14:27.412894+0000","last_became_active":"2024-04-25T10:14:27.410656+0000","last_became_peered":"2024-04-25T10:14:27.410656+0000","last_unstale":"2024-04-25T10:14:27.412894+0000","last_undegraded":"2024-04-25T10:14:27.412894+0000","last_fullsized":"2024-04-25T10:14:27.412894+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:12.498391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"114'1","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.406577+0000","last_change":"2024-04-25T10:14:26.396174+0000","last_active":"2024-04-25T10:14:27.406577+0000","last_peered":"2024-04-25T10:14:27.406577+0000","last_clean":"2024-04-25T10:14:27.406577+0000","last_became_active":"2024-04-25T10:14:26.395916+0000","last_became_peered":"2024-04-25T10:14:26.395916+0000","last_unstale":"2024-04-25T10:14:27.406577+0000","last_undegraded":"2024-04-25T10:14:27.406577+0000","last_fullsized":"2024-04-25T10:14:27.406577+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:55.437165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408361+0000","last_change":"2024-04-25T10:14:27.406814+0000","last_active":"2024-04-25T10:14:27.408361+0000","last_peered":"2024-04-25T10:14:27.408361+0000","last_clean":"2024-04-25T10:14:27.408361+0000","last_became_active":"2024-04-25T10:14:27.406324+0000","last_became_peered":"2024-04-25T10:14:27.406324+0000","last_unstale":"2024-04-25T10:14:27.408361+0000","last_undegraded":"2024-04-25T10:14:27.408361+0000","last_fullsized":"2024-04-25T10:14:27.408361+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:55.438842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.391803+0000","last_change":"2024-04-25T10:14:26.398044+0000","last_active":"2024-04-25T10:14:27.391803+0000","last_peered":"2024-04-25T10:14:27.391803+0000","last_clean":"2024-04-25T10:14:27.391803+0000","last_became_active":"2024-04-25T10:14:26.397764+0000","last_became_peered":"2024-04-25T10:14:26.397764+0000","last_unstale":"2024-04-25T10:14:27.391803+0000","last_undegraded":"2024-04-25T10:14:27.391803+0000","last_fullsized":"2024-04-25T10:14:27.391803+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:00.039314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408386+0000","last_change":"2024-04-25T10:14:27.403930+0000","last_active":"2024-04-25T10:14:27.408386+0000","last_peered":"2024-04-25T10:14:27.408386+0000","last_clean":"2024-04-25T10:14:27.408386+0000","last_became_active":"2024-04-25T10:14:27.403622+0000","last_became_peered":"2024-04-25T10:14:27.403622+0000","last_unstale":"2024-04-25T10:14:27.408386+0000","last_undegraded":"2024-04-25T10:14:27.408386+0000","last_fullsized":"2024-04-25T10:14:27.408386+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:07.652434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"114'2","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.415478+0000","last_change":"2024-04-25T10:14:26.401084+0000","last_active":"2024-04-25T10:14:27.415478+0000","last_peered":"2024-04-25T10:14:27.415478+0000","last_clean":"2024-04-25T10:14:27.415478+0000","last_became_active":"2024-04-25T10:14:26.400795+0000","last_became_peered":"2024-04-25T10:14:26.400795+0000","last_unstale":"2024-04-25T10:14:27.415478+0000","last_undegraded":"2024-04-25T10:14:27.415478+0000","last_fullsized":"2024-04-25T10:14:27.415478+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:48.084204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.407878+0000","last_change":"2024-04-25T10:14:27.405035+0000","last_active":"2024-04-25T10:14:27.407878+0000","last_peered":"2024-04-25T10:14:27.407878+0000","last_clean":"2024-04-25T10:14:27.407878+0000","last_became_active":"2024-04-25T10:14:27.404763+0000","last_became_peered":"2024-04-25T10:14:27.404763+0000","last_unstale":"2024-04-25T10:14:27.407878+0000","last_undegraded":"2024-04-25T10:14:27.407878+0000","last_fullsized":"2024-04-25T10:14:27.407878+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:48.523576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.395083+0000","last_change":"2024-04-25T10:14:26.398905+0000","last_active":"2024-04-25T10:14:27.395083+0000","last_peered":"2024-04-25T10:14:27.395083+0000","last_clean":"2024-04-25T10:14:27.395083+0000","last_became_active":"2024-04-25T10:14:26.398627+0000","last_became_peered":"2024-04-25T10:14:26.398627+0000","last_unstale":"2024-04-25T10:14:27.395083+0000","last_undegraded":"2024-04-25T10:14:27.395083+0000","last_fullsized":"2024-04-25T10:14:27.395083+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:44.208708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408901+0000","last_change":"2024-04-25T10:14:27.407507+0000","last_active":"2024-04-25T10:14:27.408901+0000","last_peered":"2024-04-25T10:14:27.408901+0000","last_clean":"2024-04-25T10:14:27.408901+0000","last_became_active":"2024-04-25T10:14:27.407236+0000","last_became_peered":"2024-04-25T10:14:27.407236+0000","last_unstale":"2024-04-25T10:14:27.408901+0000","last_undegraded":"2024-04-25T10:14:27.408901+0000","last_fullsized":"2024-04-25T10:14:27.408901+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:42.494044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.387758+0000","last_change":"2024-04-25T10:14:26.402011+0000","last_active":"2024-04-25T10:14:27.387758+0000","last_peered":"2024-04-25T10:14:27.387758+0000","last_clean":"2024-04-25T10:14:27.387758+0000","last_became_active":"2024-04-25T10:14:26.401697+0000","last_became_peered":"2024-04-25T10:14:26.401697+0000","last_unstale":"2024-04-25T10:14:27.387758+0000","last_undegraded":"2024-04-25T10:14:27.387758+0000","last_fullsized":"2024-04-25T10:14:27.387758+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:01.968111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.403741+0000","last_change":"2024-04-25T10:14:27.398679+0000","last_active":"2024-04-25T10:14:27.403741+0000","last_peered":"2024-04-25T10:14:27.403741+0000","last_clean":"2024-04-25T10:14:27.403741+0000","last_became_active":"2024-04-25T10:14:27.398426+0000","last_became_peered":"2024-04-25T10:14:27.398426+0000","last_unstale":"2024-04-25T10:14:27.403741+0000","last_undegraded":"2024-04-25T10:14:27.403741+0000","last_fullsized":"2024-04-25T10:14:27.403741+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:21.772355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.388473+0000","last_change":"2024-04-25T10:14:26.401127+0000","last_active":"2024-04-25T10:14:27.388473+0000","last_peered":"2024-04-25T10:14:27.388473+0000","last_clean":"2024-04-25T10:14:27.388473+0000","last_became_active":"2024-04-25T10:14:26.400828+0000","last_became_peered":"2024-04-25T10:14:26.400828+0000","last_unstale":"2024-04-25T10:14:27.388473+0000","last_undegraded":"2024-04-25T10:14:27.388473+0000","last_fullsized":"2024-04-25T10:14:27.388473+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:41.491585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.410306+0000","last_change":"2024-04-25T10:14:27.408674+0000","last_active":"2024-04-25T10:14:27.410306+0000","last_peered":"2024-04-25T10:14:27.410306+0000","last_clean":"2024-04-25T10:14:27.410306+0000","last_became_active":"2024-04-25T10:14:27.408391+0000","last_became_peered":"2024-04-25T10:14:27.408391+0000","last_unstale":"2024-04-25T10:14:27.410306+0000","last_undegraded":"2024-04-25T10:14:27.410306+0000","last_fullsized":"2024-04-25T10:14:27.410306+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:21.027596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"114'2","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.413498+0000","last_change":"2024-04-25T10:14:26.398102+0000","last_active":"2024-04-25T10:14:27.413498+0000","last_peered":"2024-04-25T10:14:27.413498+0000","last_clean":"2024-04-25T10:14:27.413498+0000","last_became_active":"2024-04-25T10:14:26.397796+0000","last_became_peered":"2024-04-25T10:14:26.397796+0000","last_unstale":"2024-04-25T10:14:27.413498+0000","last_undegraded":"2024-04-25T10:14:27.413498+0000","last_fullsized":"2024-04-25T10:14:27.413498+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:51.749378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.265414+0000","last_change":"2024-04-25T10:14:27.404769+0000","last_active":"2024-04-25T10:14:29.265414+0000","last_peered":"2024-04-25T10:14:29.265414+0000","last_clean":"2024-04-25T10:14:29.265414+0000","last_became_active":"2024-04-25T10:14:27.404474+0000","last_became_peered":"2024-04-25T10:14:27.404474+0000","last_unstale":"2024-04-25T10:14:29.265414+0000","last_undegraded":"2024-04-25T10:14:29.265414+0000","last_fullsized":"2024-04-25T10:14:29.265414+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:05.792549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.390313+0000","last_change":"2024-04-25T10:14:26.396893+0000","last_active":"2024-04-25T10:14:27.390313+0000","last_peered":"2024-04-25T10:14:27.390313+0000","last_clean":"2024-04-25T10:14:27.390313+0000","last_became_active":"2024-04-25T10:14:26.396593+0000","last_became_peered":"2024-04-25T10:14:26.396593+0000","last_unstale":"2024-04-25T10:14:27.390313+0000","last_undegraded":"2024-04-25T10:14:27.390313+0000","last_fullsized":"2024-04-25T10:14:27.390313+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:28.082901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.407863+0000","last_change":"2024-04-25T10:14:27.404209+0000","last_active":"2024-04-25T10:14:27.407863+0000","last_peered":"2024-04-25T10:14:27.407863+0000","last_clean":"2024-04-25T10:14:27.407863+0000","last_became_active":"2024-04-25T10:14:27.403931+0000","last_became_peered":"2024-04-25T10:14:27.403931+0000","last_unstale":"2024-04-25T10:14:27.407863+0000","last_undegraded":"2024-04-25T10:14:27.407863+0000","last_fullsized":"2024-04-25T10:14:27.407863+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:38.602823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.388436+0000","last_change":"2024-04-25T10:14:26.397428+0000","last_active":"2024-04-25T10:14:27.388436+0000","last_peered":"2024-04-25T10:14:27.388436+0000","last_clean":"2024-04-25T10:14:27.388436+0000","last_became_active":"2024-04-25T10:14:26.394137+0000","last_became_peered":"2024-04-25T10:14:26.394137+0000","last_unstale":"2024-04-25T10:14:27.388436+0000","last_undegraded":"2024-04-25T10:14:27.388436+0000","last_fullsized":"2024-04-25T10:14:27.388436+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:54.436383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.408339+0000","last_change":"2024-04-25T10:14:27.406626+0000","last_active":"2024-04-25T10:14:27.408339+0000","last_peered":"2024-04-25T10:14:27.408339+0000","last_clean":"2024-04-25T10:14:27.408339+0000","last_became_active":"2024-04-25T10:14:27.406232+0000","last_became_peered":"2024-04-25T10:14:27.406232+0000","last_unstale":"2024-04-25T10:14:27.408339+0000","last_undegraded":"2024-04-25T10:14:27.408339+0000","last_fullsized":"2024-04-25T10:14:27.408339+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:44.625558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.388542+0000","last_change":"2024-04-25T10:14:26.398067+0000","last_active":"2024-04-25T10:14:27.388542+0000","last_peered":"2024-04-25T10:14:27.388542+0000","last_clean":"2024-04-25T10:14:27.388542+0000","last_became_active":"2024-04-25T10:14:26.397760+0000","last_became_peered":"2024-04-25T10:14:26.397760+0000","last_unstale":"2024-04-25T10:14:27.388542+0000","last_undegraded":"2024-04-25T10:14:27.388542+0000","last_fullsized":"2024-04-25T10:14:27.388542+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:20.025137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.401889+0000","last_change":"2024-04-25T10:14:27.399956+0000","last_active":"2024-04-25T10:14:27.401889+0000","last_peered":"2024-04-25T10:14:27.401889+0000","last_clean":"2024-04-25T10:14:27.401889+0000","last_became_active":"2024-04-25T10:14:27.399627+0000","last_became_peered":"2024-04-25T10:14:27.399627+0000","last_unstale":"2024-04-25T10:14:27.401889+0000","last_undegraded":"2024-04-25T10:14:27.401889+0000","last_fullsized":"2024-04-25T10:14:27.401889+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:14.560458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.388648+0000","last_change":"2024-04-25T10:14:26.395028+0000","last_active":"2024-04-25T10:14:27.388648+0000","last_peered":"2024-04-25T10:14:27.388648+0000","last_clean":"2024-04-25T10:14:27.388648+0000","last_became_active":"2024-04-25T10:14:26.394720+0000","last_became_peered":"2024-04-25T10:14:26.394720+0000","last_unstale":"2024-04-25T10:14:27.388648+0000","last_undegraded":"2024-04-25T10:14:27.388648+0000","last_fullsized":"2024-04-25T10:14:27.388648+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:57.509448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.412162+0000","last_change":"2024-04-25T10:14:27.409830+0000","last_active":"2024-04-25T10:14:27.412162+0000","last_peered":"2024-04-25T10:14:27.412162+0000","last_clean":"2024-04-25T10:14:27.412162+0000","last_became_active":"2024-04-25T10:14:27.409560+0000","last_became_peered":"2024-04-25T10:14:27.409560+0000","last_unstale":"2024-04-25T10:14:27.412162+0000","last_undegraded":"2024-04-25T10:14:27.412162+0000","last_fullsized":"2024-04-25T10:14:27.412162+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:36.716791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.393187+0000","last_change":"2024-04-25T10:14:26.396177+0000","last_active":"2024-04-25T10:14:27.393187+0000","last_peered":"2024-04-25T10:14:27.393187+0000","last_clean":"2024-04-25T10:14:27.393187+0000","last_became_active":"2024-04-25T10:14:26.395882+0000","last_became_peered":"2024-04-25T10:14:26.395882+0000","last_unstale":"2024-04-25T10:14:27.393187+0000","last_undegraded":"2024-04-25T10:14:27.393187+0000","last_fullsized":"2024-04-25T10:14:27.393187+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:51.749378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.413049+0000","last_change":"2024-04-25T10:14:27.411853+0000","last_active":"2024-04-25T10:14:27.413049+0000","last_peered":"2024-04-25T10:14:27.413049+0000","last_clean":"2024-04-25T10:14:27.413049+0000","last_became_active":"2024-04-25T10:14:27.411550+0000","last_became_peered":"2024-04-25T10:14:27.411550+0000","last_unstale":"2024-04-25T10:14:27.413049+0000","last_undegraded":"2024-04-25T10:14:27.413049+0000","last_fullsized":"2024-04-25T10:14:27.413049+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:26.494609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.394356+0000","last_change":"2024-04-25T10:14:26.396537+0000","last_active":"2024-04-25T10:14:27.394356+0000","last_peered":"2024-04-25T10:14:27.394356+0000","last_clean":"2024-04-25T10:14:27.394356+0000","last_became_active":"2024-04-25T10:14:26.396221+0000","last_became_peered":"2024-04-25T10:14:26.396221+0000","last_unstale":"2024-04-25T10:14:27.394356+0000","last_undegraded":"2024-04-25T10:14:27.394356+0000","last_fullsized":"2024-04-25T10:14:27.394356+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:20.025137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.264693+0000","last_change":"2024-04-25T10:14:27.410338+0000","last_active":"2024-04-25T10:14:29.264693+0000","last_peered":"2024-04-25T10:14:29.264693+0000","last_clean":"2024-04-25T10:14:29.264693+0000","last_became_active":"2024-04-25T10:14:27.410050+0000","last_became_peered":"2024-04-25T10:14:27.410050+0000","last_unstale":"2024-04-25T10:14:29.264693+0000","last_undegraded":"2024-04-25T10:14:29.264693+0000","last_fullsized":"2024-04-25T10:14:29.264693+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:28.594949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.264499+0000","last_change":"2024-04-25T10:14:26.400437+0000","last_active":"2024-04-25T10:14:29.264499+0000","last_peered":"2024-04-25T10:14:29.264499+0000","last_clean":"2024-04-25T10:14:29.264499+0000","last_became_active":"2024-04-25T10:14:26.399347+0000","last_became_peered":"2024-04-25T10:14:26.399347+0000","last_unstale":"2024-04-25T10:14:29.264499+0000","last_undegraded":"2024-04-25T10:14:29.264499+0000","last_fullsized":"2024-04-25T10:14:29.264499+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:38.469650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.396538+0000","last_change":"2024-04-25T10:14:27.394580+0000","last_active":"2024-04-25T10:14:27.396538+0000","last_peered":"2024-04-25T10:14:27.396538+0000","last_clean":"2024-04-25T10:14:27.396538+0000","last_became_active":"2024-04-25T10:14:27.394296+0000","last_became_peered":"2024-04-25T10:14:27.394296+0000","last_unstale":"2024-04-25T10:14:27.396538+0000","last_undegraded":"2024-04-25T10:14:27.396538+0000","last_fullsized":"2024-04-25T10:14:27.396538+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.374504+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.374504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:45.905737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.384360+0000","last_change":"2024-04-25T10:14:26.395283+0000","last_active":"2024-04-25T10:14:27.384360+0000","last_peered":"2024-04-25T10:14:27.384360+0000","last_clean":"2024-04-25T10:14:27.384360+0000","last_became_active":"2024-04-25T10:14:26.394981+0000","last_became_peered":"2024-04-25T10:14:26.394981+0000","last_unstale":"2024-04-25T10:14:27.384360+0000","last_undegraded":"2024-04-25T10:14:27.384360+0000","last_fullsized":"2024-04-25T10:14:27.384360+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:25.372045+0000","last_clean_scrub_stamp":"2024-04-25T10:14:25.372045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:43.623099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":358,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-25T10:14:27.388879+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:14:27.388879+0000","last_peered":"2024-04-25T10:14:27.388879+0000","last_clean":"2024-04-25T10:14:27.388879+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:14:27.388879+0000","last_undegraded":"2024-04-25T10:14:27.388879+0000","last_fullsized":"2024-04-25T10:14:27.388879+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":603144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1396736,"data_stored":603144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":603144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509558,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36724,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335116,"statfs":{"total":96636764160,"available":96599158784,"internally_reserved":0,"allocated":1409024,"data_stored":609919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36712,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335128,"statfs":{"total":96636764160,"available":96599171072,"internally_reserved":0,"allocated":1396736,"data_stored":603144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509557,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37296,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334544,"statfs":{"total":96636764160,"available":96598573056,"internally_reserved":0,"allocated":2002944,"data_stored":1200287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1417216,"data_stored":610571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33160,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338680,"statfs":{"total":96636764160,"available":96602808320,"internally_reserved":0,"allocated":2023424,"data_stored":1206679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:14:30.749 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:14:30.749 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:14:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.889+0000 7fc5d345d640 1 Processor -- start 2024-04-25T10:14:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.890+0000 7fc5d345d640 1 -- start start 2024-04-25T10:14:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.890+0000 7fc5d345d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5cc15daf0 0x7fc5cc162270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.890+0000 7fc5d345d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc5cc0b6e00 0x7fc5cc0b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.890+0000 7fc5d345d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc5cc15ded0 0x7fc5cc0b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.890+0000 7fc5d345d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5cc1627b0 con 0x7fc5cc15ded0 2024-04-25T10:14:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.890+0000 7fc5d345d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5cc0cf400 con 0x7fc5cc0b6e00 2024-04-25T10:14:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.890+0000 7fc5d345d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0cf120 con 0x7fc5cc15daf0 2024-04-25T10:14:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.890+0000 7fc5d19fa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc5cc15ded0 0x7fc5cc0b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.890+0000 7fc5d19fa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc5cc15ded0 0x7fc5cc0b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39226/0 (socket says 172.21.15.16:39226) 2024-04-25T10:14:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.890+0000 7fc5d19fa640 1 -- 172.21.15.16:0/1638162257 learned_addr learned my addr 172.21.15.16:0/1638162257 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.891+0000 7fc5d09f8640 1 --2- 172.21.15.16:0/1638162257 >> v2:172.21.15.186:3300/0 conn(0x7fc5cc0b6e00 0x7fc5cc0b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.891+0000 7fc5d11f9640 1 --2- 172.21.15.16:0/1638162257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5cc15daf0 0x7fc5cc162270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.891+0000 7fc5d19fa640 1 -- 172.21.15.16:0/1638162257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5cc15daf0 msgr2=0x7fc5cc162270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.891+0000 7fc5d19fa640 1 --2- 172.21.15.16:0/1638162257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5cc15daf0 0x7fc5cc162270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.891+0000 7fc5d19fa640 1 -- 172.21.15.16:0/1638162257 >> v2:172.21.15.186:3300/0 conn(0x7fc5cc0b6e00 msgr2=0x7fc5cc0b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.891+0000 7fc5d19fa640 1 --2- 172.21.15.16:0/1638162257 >> v2:172.21.15.186:3300/0 conn(0x7fc5cc0b6e00 0x7fc5cc0b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.891+0000 7fc5d19fa640 1 -- 172.21.15.16:0/1638162257 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc0d1e90 con 0x7fc5cc15ded0 2024-04-25T10:14:30.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.891+0000 7fc5d19fa640 1 --2- 172.21.15.16:0/1638162257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5cc15ded0 0x7fc5cc0b5ae0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc073210 tx=0x7fc5c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6174babce74e1c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.892+0000 7fc5bffff640 1 -- 172.21.15.16:0/1638162257 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4037070 con 0x7fc5cc15ded0 2024-04-25T10:14:30.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.892+0000 7fc5bffff640 1 -- 172.21.15.16:0/1638162257 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c4032440 con 0x7fc5cc15ded0 2024-04-25T10:14:30.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.892+0000 7fc5bffff640 1 -- 172.21.15.16:0/1638162257 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4032810 con 0x7fc5cc15ded0 2024-04-25T10:14:30.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.892+0000 7fc5d345d640 1 -- 172.21.15.16:0/1638162257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5cc15ded0 msgr2=0x7fc5cc0b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.892+0000 7fc5d345d640 1 --2- 172.21.15.16:0/1638162257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5cc15ded0 0x7fc5cc0b5ae0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc073210 tx=0x7fc5c402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.892+0000 7fc5d345d640 1 -- 172.21.15.16:0/1638162257 shutdown_connections 2024-04-25T10:14:30.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.892+0000 7fc5d345d640 1 --2- 172.21.15.16:0/1638162257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5cc15ded0 0x7fc5cc0b5ae0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.892+0000 7fc5d345d640 1 --2- 172.21.15.16:0/1638162257 >> v2:172.21.15.186:3300/0 conn(0x7fc5cc0b6e00 0x7fc5cc0b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.892+0000 7fc5d345d640 1 --2- 172.21.15.16:0/1638162257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5cc15daf0 0x7fc5cc162270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.892+0000 7fc5d345d640 1 -- 172.21.15.16:0/1638162257 >> 172.21.15.16:0/1638162257 conn(0x7fc5cc0d05a0 msgr2=0x7fc5cc0d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:30.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.892+0000 7fc5d345d640 1 -- 172.21.15.16:0/1638162257 shutdown_connections 2024-04-25T10:14:30.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.892+0000 7fc5d345d640 1 -- 172.21.15.16:0/1638162257 wait complete. 2024-04-25T10:14:30.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.893+0000 7fc5d345d640 1 Processor -- start 2024-04-25T10:14:30.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.894+0000 7fc5d345d640 1 -- start start 2024-04-25T10:14:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.894+0000 7fc5d345d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5cc0b6e00 0x7fc5cc161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.894+0000 7fc5d345d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5cc15daf0 0x7fc5cc149d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.894+0000 7fc5d345d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5cc15ded0 0x7fc5cc0cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.894+0000 7fc5d345d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0ceb60 con 0x7fc5cc0b6e00 2024-04-25T10:14:30.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.894+0000 7fc5d345d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0d2450 con 0x7fc5cc15ded0 2024-04-25T10:14:30.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.894+0000 7fc5d345d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0cece0 con 0x7fc5cc15daf0 2024-04-25T10:14:30.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.894+0000 7fc5d11f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5cc0b6e00 0x7fc5cc161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.894+0000 7fc5d11f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5cc0b6e00 0x7fc5cc161100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39228/0 (socket says 172.21.15.16:39228) 2024-04-25T10:14:30.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.894+0000 7fc5d11f9640 1 -- 172.21.15.16:0/2545538304 learned_addr learned my addr 172.21.15.16:0/2545538304 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:30.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.895+0000 7fc5d11f9640 1 -- 172.21.15.16:0/2545538304 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5cc15daf0 msgr2=0x7fc5cc149d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.895+0000 7fc5d11f9640 1 --2- 172.21.15.16:0/2545538304 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5cc15daf0 0x7fc5cc149d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.895+0000 7fc5d11f9640 1 -- 172.21.15.16:0/2545538304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5cc15ded0 msgr2=0x7fc5cc0cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.895+0000 7fc5d11f9640 1 --2- 172.21.15.16:0/2545538304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5cc15ded0 0x7fc5cc0cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.895+0000 7fc5d11f9640 1 -- 172.21.15.16:0/2545538304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc160060 con 0x7fc5cc0b6e00 2024-04-25T10:14:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.895+0000 7fc5d11f9640 1 --2- 172.21.15.16:0/2545538304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5cc0b6e00 0x7fc5cc161100 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800d830 tx=0x7fc5c800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.895+0000 7fc5bdffb640 1 -- 172.21.15.16:0/2545538304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800b660 con 0x7fc5cc0b6e00 2024-04-25T10:14:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.895+0000 7fc5bdffb640 1 -- 172.21.15.16:0/2545538304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c8019070 con 0x7fc5cc0b6e00 2024-04-25T10:14:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.895+0000 7fc5bdffb640 1 -- 172.21.15.16:0/2545538304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800ba20 con 0x7fc5cc0b6e00 2024-04-25T10:14:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.895+0000 7fc5d345d640 1 -- 172.21.15.16:0/2545538304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5cc160340 con 0x7fc5cc0b6e00 2024-04-25T10:14:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.895+0000 7fc5d345d640 1 -- 172.21.15.16:0/2545538304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5cc160900 con 0x7fc5cc0b6e00 2024-04-25T10:14:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.896+0000 7fc5d345d640 1 -- 172.21.15.16:0/2545538304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5cc067c00 con 0x7fc5cc0b6e00 2024-04-25T10:14:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.897+0000 7fc5bdffb640 1 -- 172.21.15.16:0/2545538304 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5c801e460 con 0x7fc5cc0b6e00 2024-04-25T10:14:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.897+0000 7fc5bdffb640 1 --2- 172.21.15.16:0/2545538304 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc5b006b4d0 0x7fc5b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.897+0000 7fc5bdffb640 1 -- 172.21.15.16:0/2545538304 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5c808a8a0 con 0x7fc5cc0b6e00 2024-04-25T10:14:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.897+0000 7fc5d09f8640 1 --2- 172.21.15.16:0/2545538304 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc5b006b4d0 0x7fc5b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.898+0000 7fc5d09f8640 1 --2- 172.21.15.16:0/2545538304 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc5b006b4d0 0x7fc5b006d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc06d140 tx=0x7fc5c0009450 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:30.901+0000 7fc5bdffb640 1 -- 172.21.15.16:0/2545538304 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5cc067c00 con 0x7fc5cc0b6e00 2024-04-25T10:14:31.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.148+0000 7fc5d345d640 1 -- 172.21.15.16:0/2545538304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc5cc0ce290 con 0x7fc5cc0b6e00 2024-04-25T10:14:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.150+0000 7fc5bdffb640 1 -- 172.21.15.16:0/2545538304 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc5cc0ce290 con 0x7fc5cc0b6e00 2024-04-25T10:14:31.154 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-04-25T10:14:31.154 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCZKipmAph/GhAAG3x1zC98yHQ13fjpmrlPkg== 2024-04-25T10:14:31.155 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-04-25T10:14:31.155 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-04-25T10:14:31.155 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:14:31.155 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-04-25T10:14:31.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.152+0000 7fc5d345d640 1 -- 172.21.15.16:0/2545538304 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc5b006b4d0 msgr2=0x7fc5b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.152+0000 7fc5d345d640 1 --2- 172.21.15.16:0/2545538304 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc5b006b4d0 0x7fc5b006d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc06d140 tx=0x7fc5c0009450 comp rx=0 tx=0).stop 2024-04-25T10:14:31.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.152+0000 7fc5d345d640 1 -- 172.21.15.16:0/2545538304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5cc0b6e00 msgr2=0x7fc5cc161100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.152+0000 7fc5d345d640 1 --2- 172.21.15.16:0/2545538304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5cc0b6e00 0x7fc5cc161100 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800d830 tx=0x7fc5c800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.152+0000 7fc5d345d640 1 -- 172.21.15.16:0/2545538304 shutdown_connections 2024-04-25T10:14:31.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.152+0000 7fc5d345d640 1 --2- 172.21.15.16:0/2545538304 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fc5b006b4d0 0x7fc5b006d990 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.152+0000 7fc5d345d640 1 --2- 172.21.15.16:0/2545538304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5cc15ded0 0x7fc5cc0cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.152+0000 7fc5d345d640 1 --2- 172.21.15.16:0/2545538304 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5cc15daf0 0x7fc5cc149d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.152+0000 7fc5d345d640 1 --2- 172.21.15.16:0/2545538304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5cc0b6e00 0x7fc5cc161100 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.152+0000 7fc5d345d640 1 -- 172.21.15.16:0/2545538304 >> 172.21.15.16:0/2545538304 conn(0x7fc5cc0d05a0 msgr2=0x7fc5cc0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.152+0000 7fc5d345d640 1 -- 172.21.15.16:0/2545538304 shutdown_connections 2024-04-25T10:14:31.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.153+0000 7fc5d345d640 1 -- 172.21.15.16:0/2545538304 wait complete. 2024-04-25T10:14:31.168 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:31.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.264+0000 7f9ff7929640 1 Processor -- start 2024-04-25T10:14:31.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.265+0000 7f9ff7929640 1 -- start start 2024-04-25T10:14:31.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.265+0000 7f9ff7929640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ff015dce0 0x7f9ff00d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.265+0000 7f9ff7929640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ff00d35b0 0x7f9ff00b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.265+0000 7f9ff7929640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff015e0c0 con 0x7f9ff00d1f90 2024-04-25T10:14:31.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.265+0000 7f9ff7929640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff00b7b10 con 0x7f9ff015dce0 2024-04-25T10:14:31.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.265+0000 7f9ff7929640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff00d2510 con 0x7f9ff00d35b0 2024-04-25T10:14:31.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.265+0000 7f9ff56c5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ff015dce0 0x7f9ff00d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.265+0000 7f9ff5ec6640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9ff00d1f90 0x7f9ff006bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43574/0 (socket says 172.21.15.16:43574) 2024-04-25T10:14:31.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.266+0000 7f9ff5ec6640 1 -- 172.21.15.16:0/1314590343 learned_addr learned my addr 172.21.15.16:0/1314590343 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:31.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.266+0000 7f9ff4ec4640 1 --2- 172.21.15.16:0/1314590343 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ff00d35b0 0x7f9ff00b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.266+0000 7f9ff56c5640 1 -- 172.21.15.16:0/1314590343 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ff00d35b0 msgr2=0x7f9ff00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.266+0000 7f9ff56c5640 1 --2- 172.21.15.16:0/1314590343 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ff00d35b0 0x7f9ff00b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.266+0000 7f9ff56c5640 1 -- 172.21.15.16:0/1314590343 >> v1:172.21.15.16:6789/0 conn(0x7f9ff00d1f90 legacy=0x7f9ff006bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.266+0000 7f9feffff640 1 -- 172.21.15.16:0/1314590343 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731966268 0 0) 0x7f9ff015e0c0 con 0x7f9ff00d1f90 2024-04-25T10:14:31.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.266+0000 7f9ff56c5640 1 -- 172.21.15.16:0/1314590343 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff00d27f0 con 0x7f9ff015dce0 2024-04-25T10:14:31.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.266+0000 7f9ff4ec4640 1 --2- 172.21.15.16:0/1314590343 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ff00d35b0 0x7f9ff00b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:31.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.267+0000 7f9ff56c5640 1 --2- 172.21.15.16:0/1314590343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff015dce0 0x7f9ff00d1a50 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80097a0 tx=0x7f9fe802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce541fbe3092cb0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.267+0000 7f9feffff640 1 -- 172.21.15.16:0/1314590343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8037070 con 0x7f9ff015dce0 2024-04-25T10:14:31.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.267+0000 7f9feffff640 1 -- 172.21.15.16:0/1314590343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe802ae50 con 0x7f9ff015dce0 2024-04-25T10:14:31.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.267+0000 7f9feffff640 1 -- 172.21.15.16:0/1314590343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8032660 con 0x7f9ff015dce0 2024-04-25T10:14:31.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.267+0000 7f9ff7929640 1 -- 172.21.15.16:0/1314590343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff015dce0 msgr2=0x7f9ff00d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.267+0000 7f9ff7929640 1 --2- 172.21.15.16:0/1314590343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff015dce0 0x7f9ff00d1a50 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80097a0 tx=0x7f9fe802a100 comp rx=0 tx=0).stop 2024-04-25T10:14:31.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.267+0000 7f9ff7929640 1 -- 172.21.15.16:0/1314590343 shutdown_connections 2024-04-25T10:14:31.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.267+0000 7f9ff7929640 1 --2- 172.21.15.16:0/1314590343 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ff00d35b0 0x7f9ff00b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.267+0000 7f9ff7929640 1 --2- 172.21.15.16:0/1314590343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff015dce0 0x7f9ff00d1a50 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.267+0000 7f9ff7929640 1 -- 172.21.15.16:0/1314590343 >> 172.21.15.16:0/1314590343 conn(0x7f9ff0072900 msgr2=0x7f9ff00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.267+0000 7f9ff7929640 1 -- 172.21.15.16:0/1314590343 shutdown_connections 2024-04-25T10:14:31.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.267+0000 7f9ff7929640 1 -- 172.21.15.16:0/1314590343 wait complete. 2024-04-25T10:14:31.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.268+0000 7f9ff7929640 1 Processor -- start 2024-04-25T10:14:31.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.268+0000 7f9ff7929640 1 -- start start 2024-04-25T10:14:31.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.269+0000 7f9ff7929640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff00d1f90 0x7f9ff01475c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.269+0000 7f9ff7929640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ff00d35b0 0x7f9ff00b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.269+0000 7f9ff7929640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ff015dce0 0x7f9ff00c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.269+0000 7f9ff7929640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff00d3090 con 0x7f9ff00d1f90 2024-04-25T10:14:31.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.269+0000 7f9ff7929640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff00d2db0 con 0x7f9ff00d35b0 2024-04-25T10:14:31.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.269+0000 7f9ff7929640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff00b9820 con 0x7f9ff015dce0 2024-04-25T10:14:31.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.269+0000 7f9ff56c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff00d1f90 0x7f9ff01475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.269+0000 7f9ff56c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff00d1f90 0x7f9ff01475c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39246/0 (socket says 172.21.15.16:39246) 2024-04-25T10:14:31.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.269+0000 7f9ff56c5640 1 -- 172.21.15.16:0/1795422458 learned_addr learned my addr 172.21.15.16:0/1795422458 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:31.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.269+0000 7f9ff4ec4640 1 --2- 172.21.15.16:0/1795422458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ff00d35b0 0x7f9ff00b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.269+0000 7f9ff5ec6640 1 --2- 172.21.15.16:0/1795422458 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ff015dce0 0x7f9ff00c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.270+0000 7f9ff56c5640 1 -- 172.21.15.16:0/1795422458 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ff015dce0 msgr2=0x7f9ff00c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.270+0000 7f9ff56c5640 1 --2- 172.21.15.16:0/1795422458 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ff015dce0 0x7f9ff00c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.270+0000 7f9ff56c5640 1 -- 172.21.15.16:0/1795422458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ff00d35b0 msgr2=0x7f9ff00b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.270+0000 7f9ff56c5640 1 --2- 172.21.15.16:0/1795422458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ff00d35b0 0x7f9ff00b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.270+0000 7f9ff56c5640 1 -- 172.21.15.16:0/1795422458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff00c8070 con 0x7f9ff00d1f90 2024-04-25T10:14:31.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.270+0000 7f9ff4ec4640 1 --2- 172.21.15.16:0/1795422458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ff00d35b0 0x7f9ff00b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:31.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.270+0000 7f9ff56c5640 1 --2- 172.21.15.16:0/1795422458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff00d1f90 0x7f9ff01475c0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8035040 tx=0x7f9fe80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.270+0000 7f9fedffb640 1 -- 172.21.15.16:0/1795422458 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8037040 con 0x7f9ff00d1f90 2024-04-25T10:14:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.270+0000 7f9fedffb640 1 -- 172.21.15.16:0/1795422458 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe8004250 con 0x7f9ff00d1f90 2024-04-25T10:14:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.270+0000 7f9ff7929640 1 -- 172.21.15.16:0/1795422458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff00c8350 con 0x7f9ff00d1f90 2024-04-25T10:14:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.270+0000 7f9fedffb640 1 -- 172.21.15.16:0/1795422458 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8003610 con 0x7f9ff00d1f90 2024-04-25T10:14:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.270+0000 7f9ff7929640 1 -- 172.21.15.16:0/1795422458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff00c8910 con 0x7f9ff00d1f90 2024-04-25T10:14:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.271+0000 7f9ff7929640 1 -- 172.21.15.16:0/1795422458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff00749b0 con 0x7f9ff00d1f90 2024-04-25T10:14:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.272+0000 7f9fedffb640 1 -- 172.21.15.16:0/1795422458 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9fe802ad10 con 0x7f9ff00d1f90 2024-04-25T10:14:31.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.272+0000 7f9fedffb640 1 --2- 172.21.15.16:0/1795422458 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9fdc06b4d0 0x7f9fdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.272+0000 7f9fedffb640 1 -- 172.21.15.16:0/1795422458 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fe80aae80 con 0x7f9ff00d1f90 2024-04-25T10:14:31.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.272+0000 7f9ff4ec4640 1 --2- 172.21.15.16:0/1795422458 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9fdc06b4d0 0x7f9fdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.273+0000 7f9ff4ec4640 1 --2- 172.21.15.16:0/1795422458 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9fdc06b4d0 0x7f9fdc06d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0068010 tx=0x7f9fe4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.276+0000 7f9fedffb640 1 -- 172.21.15.16:0/1795422458 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9fe804c070 con 0x7f9ff00d1f90 2024-04-25T10:14:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.496+0000 7f9ff7929640 1 -- 172.21.15.16:0/1795422458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ff00b5080 con 0x7f9ff00d1f90 2024-04-25T10:14:31.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.496+0000 7f9fedffb640 1 -- 172.21.15.16:0/1795422458 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f9fe8079eb0 con 0x7f9ff00d1f90 2024-04-25T10:14:31.501 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:31.501 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2871061122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2871061122},{"type":"v1","addr":"172.21.15.186:6836","nonce":2871061122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1871507206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1871507206},{"type":"v1","addr":"172.21.15.186:6837","nonce":1871507206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4064412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4064412586},{"type":"v1","addr":"172.21.15.16:6834","nonce":4064412586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:26.751060+0000","modified":"2024-04-25T10:14:28.748797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8850":{"gid":8850,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/123396308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":123396308},{"type":"v1","addr":"172.21.15.16:6835","nonce":123396308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:31.501 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 84 2024-04-25T10:14:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.498+0000 7f9ff7929640 1 -- 172.21.15.16:0/1795422458 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9fdc06b4d0 msgr2=0x7f9fdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.498+0000 7f9ff7929640 1 --2- 172.21.15.16:0/1795422458 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9fdc06b4d0 0x7f9fdc06d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0068010 tx=0x7f9fe4008040 comp rx=0 tx=0).stop 2024-04-25T10:14:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.498+0000 7f9ff7929640 1 -- 172.21.15.16:0/1795422458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff00d1f90 msgr2=0x7f9ff01475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.498+0000 7f9ff7929640 1 --2- 172.21.15.16:0/1795422458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff00d1f90 0x7f9ff01475c0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8035040 tx=0x7f9fe80046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.498+0000 7f9ff7929640 1 -- 172.21.15.16:0/1795422458 shutdown_connections 2024-04-25T10:14:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.498+0000 7f9ff7929640 1 --2- 172.21.15.16:0/1795422458 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9fdc06b4d0 0x7f9fdc06d990 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.498+0000 7f9ff7929640 1 --2- 172.21.15.16:0/1795422458 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ff015dce0 0x7f9ff00c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.498+0000 7f9ff7929640 1 --2- 172.21.15.16:0/1795422458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ff00d35b0 0x7f9ff00b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.498+0000 7f9ff7929640 1 --2- 172.21.15.16:0/1795422458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ff00d1f90 0x7f9ff01475c0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.498+0000 7f9ff7929640 1 -- 172.21.15.16:0/1795422458 >> 172.21.15.16:0/1795422458 conn(0x7f9ff0072900 msgr2=0x7f9ff00d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.499+0000 7f9ff7929640 1 -- 172.21.15.16:0/1795422458 shutdown_connections 2024-04-25T10:14:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:31.499+0000 7f9ff7929640 1 -- 172.21.15.16:0/1795422458 wait complete. 2024-04-25T10:14:31.916 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:14:26.751060+0000', 'modified': '2024-04-25T10:14:28.748797+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8850}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8850': {'gid': 8850, 'name': 'c', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6835/123396308', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 123396308}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 123396308}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:14:31.916 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:14:31.917 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:14:32.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.017+0000 7f633331b640 1 Processor -- start 2024-04-25T10:14:32.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.018+0000 7f633331b640 1 -- start start 2024-04-25T10:14:32.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.018+0000 7f633331b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f632c15dbe0 0x7f632c15ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.018+0000 7f633331b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f632c0d3580 0x7f632c0b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.018+0000 7f633331b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f632c0b9410 0x7f632c0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.018+0000 7f633331b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f632c15f210 con 0x7f632c0b9410 2024-04-25T10:14:32.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.018+0000 7f633331b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f632c0b9980 con 0x7f632c0d3580 2024-04-25T10:14:32.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.018+0000 7f633331b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632c0ce140 con 0x7f632c15dbe0 2024-04-25T10:14:32.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.018+0000 7f63318b8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f632c0b9410 0x7f632c0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.018+0000 7f63318b8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f632c0b9410 0x7f632c0b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39254/0 (socket says 172.21.15.16:39254) 2024-04-25T10:14:32.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.018+0000 7f63318b8640 1 -- 172.21.15.16:0/2645385979 learned_addr learned my addr 172.21.15.16:0/2645385979 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:32.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.019+0000 7f63308b6640 1 --2- 172.21.15.16:0/2645385979 >> v2:172.21.15.186:3300/0 conn(0x7f632c0d3580 0x7f632c0b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.019+0000 7f63310b7640 1 --2- 172.21.15.16:0/2645385979 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f632c15dbe0 0x7f632c15ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.019+0000 7f63318b8640 1 -- 172.21.15.16:0/2645385979 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f632c15dbe0 msgr2=0x7f632c15ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.019+0000 7f63318b8640 1 --2- 172.21.15.16:0/2645385979 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f632c15dbe0 0x7f632c15ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.019+0000 7f63318b8640 1 -- 172.21.15.16:0/2645385979 >> v2:172.21.15.186:3300/0 conn(0x7f632c0d3580 msgr2=0x7f632c0b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.019+0000 7f63318b8640 1 --2- 172.21.15.16:0/2645385979 >> v2:172.21.15.186:3300/0 conn(0x7f632c0d3580 0x7f632c0b5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.019+0000 7f63318b8640 1 -- 172.21.15.16:0/2645385979 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c0ce420 con 0x7f632c0b9410 2024-04-25T10:14:32.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.019+0000 7f63318b8640 1 --2- 172.21.15.16:0/2645385979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f632c0b9410 0x7f632c0b5ab0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f632c0d4b80 tx=0x7f631c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965bf70485ff1a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.020+0000 7f632bfff640 1 -- 172.21.15.16:0/2645385979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c036070 con 0x7f632c0b9410 2024-04-25T10:14:32.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.020+0000 7f632bfff640 1 -- 172.21.15.16:0/2645385979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631c031600 con 0x7f632c0b9410 2024-04-25T10:14:32.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.020+0000 7f632bfff640 1 -- 172.21.15.16:0/2645385979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c0319d0 con 0x7f632c0b9410 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.020+0000 7f633331b640 1 -- 172.21.15.16:0/2645385979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f632c0b9410 msgr2=0x7f632c0b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.020+0000 7f633331b640 1 --2- 172.21.15.16:0/2645385979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f632c0b9410 0x7f632c0b5ab0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f632c0d4b80 tx=0x7f631c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.020+0000 7f633331b640 1 -- 172.21.15.16:0/2645385979 shutdown_connections 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.020+0000 7f633331b640 1 --2- 172.21.15.16:0/2645385979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f632c0b9410 0x7f632c0b5ab0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.020+0000 7f633331b640 1 --2- 172.21.15.16:0/2645385979 >> v2:172.21.15.186:3300/0 conn(0x7f632c0d3580 0x7f632c0b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.020+0000 7f633331b640 1 --2- 172.21.15.16:0/2645385979 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f632c15dbe0 0x7f632c15ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.020+0000 7f633331b640 1 -- 172.21.15.16:0/2645385979 >> 172.21.15.16:0/2645385979 conn(0x7f632c072900 msgr2=0x7f632c0d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.020+0000 7f633331b640 1 -- 172.21.15.16:0/2645385979 shutdown_connections 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.020+0000 7f633331b640 1 -- 172.21.15.16:0/2645385979 wait complete. 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.021+0000 7f633331b640 1 Processor -- start 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.022+0000 7f633331b640 1 -- start start 2024-04-25T10:14:32.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.022+0000 7f633331b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f632c0b9410 0x7f632c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.022+0000 7f633331b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f632c0d3580 0x7f632c1579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.022+0000 7f633331b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f632c15dbe0 0x7f632c157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.022+0000 7f633331b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c15dfc0 con 0x7f632c15dbe0 2024-04-25T10:14:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.022+0000 7f633331b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c0d3060 con 0x7f632c0b9410 2024-04-25T10:14:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.022+0000 7f633331b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632c15e140 con 0x7f632c0d3580 2024-04-25T10:14:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.022+0000 7f63318b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f632c15dbe0 0x7f632c157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.022+0000 7f63318b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f632c15dbe0 0x7f632c157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39266/0 (socket says 172.21.15.16:39266) 2024-04-25T10:14:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.022+0000 7f63318b8640 1 -- 172.21.15.16:0/1558434120 learned_addr learned my addr 172.21.15.16:0/1558434120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.023+0000 7f63310b7640 1 --2- 172.21.15.16:0/1558434120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f632c0b9410 0x7f632c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.023+0000 7f63308b6640 1 --2- 172.21.15.16:0/1558434120 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f632c0d3580 0x7f632c1579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.023+0000 7f63318b8640 1 -- 172.21.15.16:0/1558434120 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f632c0d3580 msgr2=0x7f632c1579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.023+0000 7f63318b8640 1 --2- 172.21.15.16:0/1558434120 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f632c0d3580 0x7f632c1579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.023+0000 7f63318b8640 1 -- 172.21.15.16:0/1558434120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f632c0b9410 msgr2=0x7f632c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.023+0000 7f63318b8640 1 --2- 172.21.15.16:0/1558434120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f632c0b9410 0x7f632c15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.023+0000 7f63318b8640 1 -- 172.21.15.16:0/1558434120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c0c7a90 con 0x7f632c15dbe0 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.023+0000 7f63308b6640 1 --2- 172.21.15.16:0/1558434120 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f632c0d3580 0x7f632c1579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.023+0000 7f63310b7640 1 --2- 172.21.15.16:0/1558434120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f632c0b9410 0x7f632c15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.023+0000 7f63318b8640 1 --2- 172.21.15.16:0/1558434120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f632c15dbe0 0x7f632c157ee0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f631c009f40 tx=0x7f631c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.023+0000 7f6329ffb640 1 -- 172.21.15.16:0/1558434120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c036050 con 0x7f632c15dbe0 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.023+0000 7f6329ffb640 1 -- 172.21.15.16:0/1558434120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631c042070 con 0x7f632c15dbe0 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.023+0000 7f633331b640 1 -- 172.21.15.16:0/1558434120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632c0c7d70 con 0x7f632c15dbe0 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.023+0000 7f6329ffb640 1 -- 172.21.15.16:0/1558434120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c009510 con 0x7f632c15dbe0 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.023+0000 7f633331b640 1 -- 172.21.15.16:0/1558434120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632c0c8610 con 0x7f632c15dbe0 2024-04-25T10:14:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.025+0000 7f6329ffb640 1 -- 172.21.15.16:0/1558434120 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f631c007bd0 con 0x7f632c15dbe0 2024-04-25T10:14:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.025+0000 7f6329ffb640 1 --2- 172.21.15.16:0/1558434120 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f631806b4d0 0x7f631806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.025+0000 7f6329ffb640 1 -- 172.21.15.16:0/1558434120 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f631c0ab190 con 0x7f632c15dbe0 2024-04-25T10:14:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.025+0000 7f63310b7640 1 --2- 172.21.15.16:0/1558434120 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f631806b4d0 0x7f631806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.026+0000 7f63310b7640 1 --2- 172.21.15.16:0/1558434120 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f631806b4d0 0x7f631806d990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6324006fd0 tx=0x7f6324008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.026+0000 7f633601d740 1 Processor -- start 2024-04-25T10:14:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.026+0000 7f633601d740 1 -- start start 2024-04-25T10:14:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.027+0000 7f633601d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0348230 0x55c2d0348610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.027+0000 7f633601d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0365ad0 0x55c2d0366020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.027+0000 7f633601d740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d031df10 0x55c2d0364190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.027+0000 7f633601d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2d0280a10 con 0x55c2d0348230 2024-04-25T10:14:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.027+0000 7f633601d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2d0229b30 con 0x55c2d0365ad0 2024-04-25T10:14:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.027+0000 7f633601d740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c2d027cc50 con 0x55c2d031df10 2024-04-25T10:14:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.027+0000 7f63310b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0348230 0x55c2d0348610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.027+0000 7f63310b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0348230 0x55c2d0348610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39276/0 (socket says 172.21.15.16:39276) 2024-04-25T10:14:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.027+0000 7f63310b7640 1 -- 172.21.15.16:0/2050222949 learned_addr learned my addr 172.21.15.16:0/2050222949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.027+0000 7f63308b6640 1 --2- 172.21.15.16:0/2050222949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0365ad0 0x55c2d0366020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.027+0000 7f63318b8640 1 --2- 172.21.15.16:0/2050222949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d031df10 0x55c2d0364190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.027+0000 7f63310b7640 1 -- 172.21.15.16:0/2050222949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d031df10 msgr2=0x55c2d0364190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.027+0000 7f63310b7640 1 --2- 172.21.15.16:0/2050222949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d031df10 0x55c2d0364190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.027+0000 7f63310b7640 1 -- 172.21.15.16:0/2050222949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0365ad0 msgr2=0x55c2d0366020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.027+0000 7f63310b7640 1 --2- 172.21.15.16:0/2050222949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0365ad0 0x55c2d0366020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.027+0000 7f63310b7640 1 -- 172.21.15.16:0/2050222949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2d023ee10 con 0x55c2d0348230 2024-04-25T10:14:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.027+0000 7f63310b7640 1 --2- 172.21.15.16:0/2050222949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0348230 0x55c2d0348610 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f632400eba0 tx=0x7f6324011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6856c79a916e3be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.028+0000 7f6312ffd640 1 -- 172.21.15.16:0/2050222949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632401a050 con 0x55c2d0348230 2024-04-25T10:14:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.028+0000 7f6312ffd640 1 -- 172.21.15.16:0/2050222949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6324024070 con 0x55c2d0348230 2024-04-25T10:14:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.028+0000 7f6312ffd640 1 -- 172.21.15.16:0/2050222949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632401f440 con 0x55c2d0348230 2024-04-25T10:14:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.028+0000 7f633601d740 1 -- 172.21.15.16:0/2050222949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0348230 msgr2=0x55c2d0348610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.028+0000 7f633601d740 1 --2- 172.21.15.16:0/2050222949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0348230 0x55c2d0348610 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f632400eba0 tx=0x7f6324011000 comp rx=0 tx=0).stop 2024-04-25T10:14:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.028+0000 7f633601d740 1 -- 172.21.15.16:0/2050222949 shutdown_connections 2024-04-25T10:14:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.028+0000 7f633601d740 1 --2- 172.21.15.16:0/2050222949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d031df10 0x55c2d0364190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.028+0000 7f633601d740 1 --2- 172.21.15.16:0/2050222949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0365ad0 0x55c2d0366020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.028+0000 7f633601d740 1 --2- 172.21.15.16:0/2050222949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0348230 0x55c2d0348610 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.028+0000 7f633601d740 1 -- 172.21.15.16:0/2050222949 >> 172.21.15.16:0/2050222949 conn(0x55c2d031a840 msgr2=0x55c2d0363040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.028+0000 7f633601d740 1 -- 172.21.15.16:0/2050222949 shutdown_connections 2024-04-25T10:14:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.028+0000 7f633601d740 1 -- 172.21.15.16:0/2050222949 wait complete. 2024-04-25T10:14:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.029+0000 7f633601d740 1 -- start start 2024-04-25T10:14:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.029+0000 7f633601d740 1 Processor -- start 2024-04-25T10:14:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f633601d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0348230 0x55c2d034a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f633601d740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d0365ad0 0x55c2d0346d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f633601d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0343f30 0x55c2d03472d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f633601d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2d031ecb0 con 0x55c2d0348230 2024-04-25T10:14:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f633601d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2d023d0c0 con 0x55c2d0343f30 2024-04-25T10:14:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f633601d740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c2d0318770 con 0x55c2d0365ad0 2024-04-25T10:14:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f633601d740 10 client.0 _ll_register_callbacks cb 0x55c2d0230ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f63310b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0348230 0x55c2d034a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f63310b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0348230 0x55c2d034a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39290/0 (socket says 172.21.15.16:39290) 2024-04-25T10:14:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f63310b7640 1 -- 172.21.15.16:0/341491537 learned_addr learned my addr 172.21.15.16:0/341491537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f63318b8640 1 --2- 172.21.15.16:0/341491537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0343f30 0x55c2d03472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f63308b6640 1 --2- 172.21.15.16:0/341491537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d0365ad0 0x55c2d0346d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f63310b7640 1 -- 172.21.15.16:0/341491537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d0365ad0 msgr2=0x55c2d0346d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f63310b7640 1 --2- 172.21.15.16:0/341491537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d0365ad0 0x55c2d0346d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f63310b7640 1 -- 172.21.15.16:0/341491537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0343f30 msgr2=0x55c2d03472d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f63310b7640 1 --2- 172.21.15.16:0/341491537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0343f30 0x55c2d03472d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f63310b7640 1 -- 172.21.15.16:0/341491537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2cff952d0 con 0x55c2d0348230 2024-04-25T10:14:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f63318b8640 1 --2- 172.21.15.16:0/341491537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0343f30 0x55c2d03472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f63310b7640 1 --2- 172.21.15.16:0/341491537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0348230 0x55c2d034a710 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f632400ffd0 tx=0x7f6324004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1f7716fb0bffce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.030+0000 7f630bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:14:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.031+0000 7f630bfff640 1 -- 172.21.15.16:0/341491537 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63240061b0 con 0x55c2d0348230 2024-04-25T10:14:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.031+0000 7f630bfff640 1 -- 172.21.15.16:0/341491537 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6324006350 con 0x55c2d0348230 2024-04-25T10:14:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.031+0000 7f633601d740 1 -- 172.21.15.16:0/341491537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6324024070 con 0x55c2d0348230 2024-04-25T10:14:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.031+0000 7f630bfff640 1 -- 172.21.15.16:0/341491537 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f632401a020 con 0x55c2d0348230 2024-04-25T10:14:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.031+0000 7f633601d740 10 client.8985 fetch_fsmap learned FSMap version 84 2024-04-25T10:14:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.031+0000 7f633601d740 1 -- 172.21.15.16:0/341491537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) v3 -- 0x55c2d005f6b0 con 0x55c2d0348230 2024-04-25T10:14:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.031+0000 7f630bfff640 1 -- 172.21.15.16:0/341491537 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6324024070 con 0x55c2d0348230 2024-04-25T10:14:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.031+0000 7f630bfff640 1 -- 172.21.15.16:0/341491537 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 84) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6324006550 con 0x55c2d0348230 2024-04-25T10:14:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.031+0000 7f633601d740 10 client.8985 fetch_fsmap finished waiting for FSMap version 84 2024-04-25T10:14:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.031+0000 7f633601d740 10 client.8985 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:14:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.031+0000 7f633601d740 20 client.8985 populate_metadata read hostname 'smithi016' 2024-04-25T10:14:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.031+0000 7f633601d740 1 --2- 172.21.15.16:0/341491537 >> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] conn(0x55c2d0369a00 0x55c2d0369de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.031+0000 7f633601d740 4 client.8985 mds_command: new command op to 8850 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.031+0000 7f633601d740 1 -- 172.21.15.16:0/341491537 --> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2d02df920 con 0x55c2d0369a00 2024-04-25T10:14:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.031+0000 7f63308b6640 1 --2- 172.21.15.16:0/341491537 >> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] conn(0x55c2d0369a00 0x55c2d0369de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.032+0000 7f63308b6640 1 --2- 172.21.15.16:0/341491537 >> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] conn(0x55c2d0369a00 0x55c2d0369de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f39b3feadc9986f2 server_cookie=8092d5a84dcb58e in_seq=0 out_seq=0 2024-04-25T10:14:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.032+0000 7f630bfff640 10 client.8985 ms_handle_connect on v2:172.21.15.16:6833/123396308 2024-04-25T10:14:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.032+0000 7f630bfff640 1 -- 172.21.15.16:0/341491537 <== mds.0 v2:172.21.15.16:6833/123396308 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c2d02df920 con 0x55c2d0369a00 2024-04-25T10:14:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.032+0000 7f630bfff640 10 client.8985 handle_command_reply: tid=0 2024-04-25T10:14:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.032+0000 7f633601d740 1 client.8985 shutdown 2024-04-25T10:14:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.032+0000 7f633601d740 1 -- 172.21.15.16:0/341491537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0348230 msgr2=0x55c2d034a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.032+0000 7f633601d740 1 --2- 172.21.15.16:0/341491537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0348230 0x55c2d034a710 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f632400ffd0 tx=0x7f6324004730 comp rx=0 tx=0).stop 2024-04-25T10:14:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.032+0000 7f633601d740 1 -- 172.21.15.16:0/341491537 shutdown_connections 2024-04-25T10:14:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.032+0000 7f633601d740 1 --2- 172.21.15.16:0/341491537 >> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] conn(0x55c2d0369a00 0x55c2d0369de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.032+0000 7f633601d740 1 --2- 172.21.15.16:0/341491537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0343f30 0x55c2d03472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.032+0000 7f633601d740 1 --2- 172.21.15.16:0/341491537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d0365ad0 0x55c2d0346d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.032+0000 7f630bfff640 0 client.8985 ms_handle_reset on v2:172.21.15.16:6833/123396308 2024-04-25T10:14:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.032+0000 7f633601d740 1 --2- 172.21.15.16:0/341491537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0348230 0x55c2d034a710 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.032+0000 7f633601d740 1 -- 172.21.15.16:0/341491537 >> 172.21.15.16:0/341491537 conn(0x55c2d031df10 msgr2=0x55c2d03665c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.032+0000 7f633601d740 1 -- 172.21.15.16:0/341491537 shutdown_connections 2024-04-25T10:14:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.032+0000 7f633601d740 1 -- 172.21.15.16:0/341491537 wait complete. 2024-04-25T10:14:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.033+0000 7f633601d740 20 client.8985 trim_cache size 0 max 16384 2024-04-25T10:14:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.048+0000 7f633601d740 1 Processor -- start 2024-04-25T10:14:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.048+0000 7f633601d740 1 -- start start 2024-04-25T10:14:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.049+0000 7f633601d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0343f30 0x55c2d0317c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.049+0000 7f633601d740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d0348230 0x55c2d03864e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.049+0000 7f633601d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0365ad0 0x55c2d034a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.049+0000 7f633601d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c1ff290 con 0x55c2d0343f30 2024-04-25T10:14:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.049+0000 7f633601d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c15a480 con 0x55c2d0365ad0 2024-04-25T10:14:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.049+0000 7f633601d740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632c0cb5a0 con 0x55c2d0348230 2024-04-25T10:14:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.049+0000 7f63310b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0343f30 0x55c2d0317c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.049+0000 7f63310b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0343f30 0x55c2d0317c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39296/0 (socket says 172.21.15.16:39296) 2024-04-25T10:14:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.049+0000 7f63310b7640 1 -- 172.21.15.16:0/2761038816 learned_addr learned my addr 172.21.15.16:0/2761038816 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.049+0000 7f63310b7640 1 -- 172.21.15.16:0/2761038816 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d0348230 msgr2=0x55c2d03864e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.049+0000 7f63310b7640 1 --2- 172.21.15.16:0/2761038816 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d0348230 0x55c2d03864e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.050+0000 7f63310b7640 1 -- 172.21.15.16:0/2761038816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0365ad0 msgr2=0x55c2d034a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.050+0000 7f63310b7640 1 --2- 172.21.15.16:0/2761038816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0365ad0 0x55c2d034a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.050+0000 7f63310b7640 1 -- 172.21.15.16:0/2761038816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2d0319880 con 0x55c2d0343f30 2024-04-25T10:14:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.050+0000 7f63310b7640 1 --2- 172.21.15.16:0/2761038816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0343f30 0x55c2d0317c60 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f6324008010 tx=0x7f632402a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd500262aa83a6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.050+0000 7f63127fc640 1 -- 172.21.15.16:0/2761038816 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63240061b0 con 0x55c2d0343f30 2024-04-25T10:14:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.050+0000 7f63127fc640 1 -- 172.21.15.16:0/2761038816 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6324024040 con 0x55c2d0343f30 2024-04-25T10:14:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.050+0000 7f63127fc640 1 -- 172.21.15.16:0/2761038816 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324006570 con 0x55c2d0343f30 2024-04-25T10:14:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.050+0000 7f633601d740 1 -- 172.21.15.16:0/2761038816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0343f30 msgr2=0x55c2d0317c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.050+0000 7f633601d740 1 --2- 172.21.15.16:0/2761038816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0343f30 0x55c2d0317c60 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f6324008010 tx=0x7f632402a000 comp rx=0 tx=0).stop 2024-04-25T10:14:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.050+0000 7f633601d740 1 -- 172.21.15.16:0/2761038816 shutdown_connections 2024-04-25T10:14:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.050+0000 7f633601d740 1 --2- 172.21.15.16:0/2761038816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0365ad0 0x55c2d034a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.050+0000 7f633601d740 1 --2- 172.21.15.16:0/2761038816 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d0348230 0x55c2d03864e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.050+0000 7f633601d740 1 --2- 172.21.15.16:0/2761038816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0343f30 0x55c2d0317c60 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.050+0000 7f633601d740 1 -- 172.21.15.16:0/2761038816 >> 172.21.15.16:0/2761038816 conn(0x55c2d031df10 msgr2=0x55c2d034bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.050+0000 7f633601d740 1 -- 172.21.15.16:0/2761038816 shutdown_connections 2024-04-25T10:14:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.050+0000 7f633601d740 1 -- 172.21.15.16:0/2761038816 wait complete. 2024-04-25T10:14:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.051+0000 7f633601d740 1 -- start start 2024-04-25T10:14:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.051+0000 7f633601d740 1 Processor -- start 2024-04-25T10:14:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f633601d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0343f30 0x55c2d031cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f633601d740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d0348230 0x55c2d031e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f633601d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0365ad0 0x55c2d031e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f633601d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2d0377b80 con 0x55c2d0343f30 2024-04-25T10:14:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f633601d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2d0367d70 con 0x55c2d0365ad0 2024-04-25T10:14:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f633601d740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c2d0377230 con 0x55c2d0348230 2024-04-25T10:14:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f633601d740 10 client.0 _ll_register_callbacks cb 0x55c2d0318c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f63310b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0343f30 0x55c2d031cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f63310b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0343f30 0x55c2d031cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39308/0 (socket says 172.21.15.16:39308) 2024-04-25T10:14:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f63310b7640 1 -- 172.21.15.16:0/1910953565 learned_addr learned my addr 172.21.15.16:0/1910953565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f63308b6640 1 --2- 172.21.15.16:0/1910953565 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d0348230 0x55c2d031e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f63318b8640 1 --2- 172.21.15.16:0/1910953565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0365ad0 0x55c2d031e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f63310b7640 1 -- 172.21.15.16:0/1910953565 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d0348230 msgr2=0x55c2d031e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f63310b7640 1 --2- 172.21.15.16:0/1910953565 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d0348230 0x55c2d031e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f63310b7640 1 -- 172.21.15.16:0/1910953565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0365ad0 msgr2=0x55c2d031e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f63310b7640 1 --2- 172.21.15.16:0/1910953565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0365ad0 0x55c2d031e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f63310b7640 1 -- 172.21.15.16:0/1910953565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2d036a6e0 con 0x55c2d0343f30 2024-04-25T10:14:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f63318b8640 1 --2- 172.21.15.16:0/1910953565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0365ad0 0x55c2d031e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.052+0000 7f63310b7640 1 --2- 172.21.15.16:0/1910953565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0343f30 0x55c2d031cc10 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f6324006e60 tx=0x7f6324014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f27b085d0cf00f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.053+0000 7f630bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:14:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.053+0000 7f630bfff640 1 -- 172.21.15.16:0/1910953565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63240138e0 con 0x55c2d0343f30 2024-04-25T10:14:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.053+0000 7f630bfff640 1 -- 172.21.15.16:0/1910953565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f632401f6a0 con 0x55c2d0343f30 2024-04-25T10:14:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.053+0000 7f633601d740 1 -- 172.21.15.16:0/1910953565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6324024040 con 0x55c2d0343f30 2024-04-25T10:14:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.053+0000 7f630bfff640 1 -- 172.21.15.16:0/1910953565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6324055050 con 0x55c2d0343f30 2024-04-25T10:14:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.053+0000 7f633601d740 10 client.8991 fetch_fsmap learned FSMap version 84 2024-04-25T10:14:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.053+0000 7f633601d740 1 -- 172.21.15.16:0/1910953565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) v3 -- 0x55c2d03cdea0 con 0x55c2d0343f30 2024-04-25T10:14:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.053+0000 7f630bfff640 1 -- 172.21.15.16:0/1910953565 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6324024040 con 0x55c2d0343f30 2024-04-25T10:14:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.053+0000 7f630bfff640 1 -- 172.21.15.16:0/1910953565 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 84) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6324058020 con 0x55c2d0343f30 2024-04-25T10:14:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.053+0000 7f633601d740 10 client.8991 fetch_fsmap finished waiting for FSMap version 84 2024-04-25T10:14:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.053+0000 7f633601d740 10 client.8991 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:14:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.053+0000 7f633601d740 20 client.8991 populate_metadata read hostname 'smithi016' 2024-04-25T10:14:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.053+0000 7f633601d740 1 --2- 172.21.15.16:0/1910953565 >> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] conn(0x55c2d03646d0 0x55c2d0341f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.053+0000 7f633601d740 4 client.8991 mds_command: new command op to 8850 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:14:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.053+0000 7f633601d740 1 -- 172.21.15.16:0/1910953565 --> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c2d028df50 con 0x55c2d03646d0 2024-04-25T10:14:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.053+0000 7f63308b6640 1 --2- 172.21.15.16:0/1910953565 >> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] conn(0x55c2d03646d0 0x55c2d0341f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.054+0000 7f630bfff640 10 client.8991 ms_handle_connect on v2:172.21.15.16:6833/123396308 2024-04-25T10:14:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.054+0000 7f63308b6640 1 --2- 172.21.15.16:0/1910953565 >> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] conn(0x55c2d03646d0 0x55c2d0341f00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa8c49b816493351 server_cookie=c60001fb3d075f3 in_seq=0 out_seq=0 2024-04-25T10:14:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.054+0000 7f630bfff640 1 -- 172.21.15.16:0/1910953565 <== mds.0 v2:172.21.15.16:6833/123396308 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55c2d028df50 con 0x55c2d03646d0 2024-04-25T10:14:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.054+0000 7f630bfff640 10 client.8991 handle_command_reply: tid=0 2024-04-25T10:14:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.054+0000 7f633601d740 1 client.8991 shutdown 2024-04-25T10:14:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.054+0000 7f633601d740 1 -- 172.21.15.16:0/1910953565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0343f30 msgr2=0x55c2d031cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.054+0000 7f633601d740 1 --2- 172.21.15.16:0/1910953565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0343f30 0x55c2d031cc10 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f6324006e60 tx=0x7f6324014000 comp rx=0 tx=0).stop 2024-04-25T10:14:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.054+0000 7f633601d740 1 -- 172.21.15.16:0/1910953565 shutdown_connections 2024-04-25T10:14:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.054+0000 7f633601d740 1 --2- 172.21.15.16:0/1910953565 >> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] conn(0x55c2d03646d0 0x55c2d0341f00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.054+0000 7f633601d740 1 --2- 172.21.15.16:0/1910953565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c2d0365ad0 0x55c2d031e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.054+0000 7f633601d740 1 --2- 172.21.15.16:0/1910953565 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c2d0348230 0x55c2d031e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.054+0000 7f630bfff640 0 client.8991 ms_handle_reset on v2:172.21.15.16:6833/123396308 2024-04-25T10:14:32.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.054+0000 7f633601d740 1 --2- 172.21.15.16:0/1910953565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2d0343f30 0x55c2d031cc10 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.054+0000 7f633601d740 1 -- 172.21.15.16:0/1910953565 >> 172.21.15.16:0/1910953565 conn(0x55c2d031df10 msgr2=0x55c2d0368ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.054+0000 7f633601d740 1 -- 172.21.15.16:0/1910953565 shutdown_connections 2024-04-25T10:14:32.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.054+0000 7f633601d740 1 -- 172.21.15.16:0/1910953565 wait complete. 2024-04-25T10:14:32.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.054+0000 7f633601d740 20 client.8991 trim_cache size 0 max 16384 2024-04-25T10:14:32.065 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:14:32.065 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "34bc5eca-3a21-42a6-a26f-07da0905e8bc", 2024-04-25T10:14:32.065 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-04-25T10:14:32.065 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 8850, 2024-04-25T10:14:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-04-25T10:14:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-04-25T10:14:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-04-25T10:14:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 4.6733397129999998, 2024-04-25T10:14:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 84, 2024-04-25T10:14:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 115, 2024-04-25T10:14:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 115, 2024-04-25T10:14:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 10.006118107000001 2024-04-25T10:14:32.066 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:14:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.055+0000 7f633331b640 1 -- 172.21.15.16:0/1558434120 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f631806b4d0 msgr2=0x7f631806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.055+0000 7f633331b640 1 --2- 172.21.15.16:0/1558434120 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f631806b4d0 0x7f631806d990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6324006fd0 tx=0x7f6324008040 comp rx=0 tx=0).stop 2024-04-25T10:14:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.055+0000 7f633331b640 1 -- 172.21.15.16:0/1558434120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f632c15dbe0 msgr2=0x7f632c157ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.055+0000 7f633331b640 1 --2- 172.21.15.16:0/1558434120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f632c15dbe0 0x7f632c157ee0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f631c009f40 tx=0x7f631c004a40 comp rx=0 tx=0).stop 2024-04-25T10:14:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.055+0000 7f633331b640 1 -- 172.21.15.16:0/1558434120 shutdown_connections 2024-04-25T10:14:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.055+0000 7f633331b640 1 --2- 172.21.15.16:0/1558434120 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f631806b4d0 0x7f631806d990 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.055+0000 7f633331b640 1 --2- 172.21.15.16:0/1558434120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f632c15dbe0 0x7f632c157ee0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.055+0000 7f633331b640 1 --2- 172.21.15.16:0/1558434120 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f632c0d3580 0x7f632c1579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.055+0000 7f633331b640 1 --2- 172.21.15.16:0/1558434120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f632c0b9410 0x7f632c15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.055+0000 7f633331b640 1 -- 172.21.15.16:0/1558434120 >> 172.21.15.16:0/1558434120 conn(0x7f632c072900 msgr2=0x7f632c0b5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.055+0000 7f633331b640 1 -- 172.21.15.16:0/1558434120 shutdown_connections 2024-04-25T10:14:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.055+0000 7f633331b640 1 -- 172.21.15.16:0/1558434120 wait complete. 2024-04-25T10:14:32.070 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:14:32.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.166+0000 7f06045d5640 1 Processor -- start 2024-04-25T10:14:32.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.166+0000 7f06045d5640 1 -- start start 2024-04-25T10:14:32.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.167+0000 7f06045d5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f05fc15dce0 0x7f05fc0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.167+0000 7f06045d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc0d35b0 0x7f05fc0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.167+0000 7f06045d5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05fc0d1f90 0x7f05fc0b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.167+0000 7f06045d5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05fc15e0c0 con 0x7f05fc0d1f90 2024-04-25T10:14:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.167+0000 7f06045d5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05fc0d2510 con 0x7f05fc15dce0 2024-04-25T10:14:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.167+0000 7f06045d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0d27f0 con 0x7f05fc0d35b0 2024-04-25T10:14:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.167+0000 7f0602b72640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05fc0d1f90 0x7f05fc0b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.167+0000 7f0602b72640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05fc0d1f90 0x7f05fc0b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39314/0 (socket says 172.21.15.16:39314) 2024-04-25T10:14:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.167+0000 7f0602b72640 1 -- 172.21.15.16:0/2444744502 learned_addr learned my addr 172.21.15.16:0/2444744502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.167+0000 7f0602371640 1 --2- 172.21.15.16:0/2444744502 >> v2:172.21.15.186:3300/0 conn(0x7f05fc15dce0 0x7f05fc0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.167+0000 7f0601b70640 1 --2- 172.21.15.16:0/2444744502 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc0d35b0 0x7f05fc0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.167+0000 7f0602b72640 1 -- 172.21.15.16:0/2444744502 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc0d35b0 msgr2=0x7f05fc0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.167+0000 7f0602b72640 1 --2- 172.21.15.16:0/2444744502 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc0d35b0 0x7f05fc0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.167+0000 7f0602b72640 1 -- 172.21.15.16:0/2444744502 >> v2:172.21.15.186:3300/0 conn(0x7f05fc15dce0 msgr2=0x7f05fc0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.167+0000 7f0602b72640 1 --2- 172.21.15.16:0/2444744502 >> v2:172.21.15.186:3300/0 conn(0x7f05fc15dce0 0x7f05fc0d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.167+0000 7f0602b72640 1 -- 172.21.15.16:0/2444744502 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc0d2ad0 con 0x7f05fc0d1f90 2024-04-25T10:14:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.167+0000 7f0602371640 1 --2- 172.21.15.16:0/2444744502 >> v2:172.21.15.186:3300/0 conn(0x7f05fc15dce0 0x7f05fc0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.168+0000 7f0602b72640 1 --2- 172.21.15.16:0/2444744502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05fc0d1f90 0x7f05fc0b5990 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0ce220 tx=0x7f05f402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47dec401c4ed76c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.168+0000 7f060136f640 1 -- 172.21.15.16:0/2444744502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4036070 con 0x7f05fc0d1f90 2024-04-25T10:14:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.168+0000 7f060136f640 1 -- 172.21.15.16:0/2444744502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f4031600 con 0x7f05fc0d1f90 2024-04-25T10:14:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.168+0000 7f060136f640 1 -- 172.21.15.16:0/2444744502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f40319d0 con 0x7f05fc0d1f90 2024-04-25T10:14:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.169+0000 7f06045d5640 1 -- 172.21.15.16:0/2444744502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05fc0d1f90 msgr2=0x7f05fc0b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.169+0000 7f06045d5640 1 --2- 172.21.15.16:0/2444744502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05fc0d1f90 0x7f05fc0b5990 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0ce220 tx=0x7f05f402a610 comp rx=0 tx=0).stop 2024-04-25T10:14:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.169+0000 7f06045d5640 1 -- 172.21.15.16:0/2444744502 shutdown_connections 2024-04-25T10:14:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.169+0000 7f06045d5640 1 --2- 172.21.15.16:0/2444744502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05fc0d1f90 0x7f05fc0b5990 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.169+0000 7f06045d5640 1 --2- 172.21.15.16:0/2444744502 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc0d35b0 0x7f05fc0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.169+0000 7f06045d5640 1 --2- 172.21.15.16:0/2444744502 >> v2:172.21.15.186:3300/0 conn(0x7f05fc15dce0 0x7f05fc0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.169+0000 7f06045d5640 1 -- 172.21.15.16:0/2444744502 >> 172.21.15.16:0/2444744502 conn(0x7f05fc072900 msgr2=0x7f05fc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.169+0000 7f06045d5640 1 -- 172.21.15.16:0/2444744502 shutdown_connections 2024-04-25T10:14:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.169+0000 7f06045d5640 1 -- 172.21.15.16:0/2444744502 wait complete. 2024-04-25T10:14:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.170+0000 7f06045d5640 1 Processor -- start 2024-04-25T10:14:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.170+0000 7f06045d5640 1 -- start start 2024-04-25T10:14:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.171+0000 7f06045d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc0d1f90 0x7f05fc0c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.171+0000 7f06045d5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05fc0d35b0 0x7f05fc145ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.171+0000 7f06045d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05fc15dce0 0x7f05fc1461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.171+0000 7f06045d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0b9820 con 0x7f05fc15dce0 2024-04-25T10:14:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.171+0000 7f06045d5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0d3090 con 0x7f05fc0d35b0 2024-04-25T10:14:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.171+0000 7f06045d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0b99a0 con 0x7f05fc0d1f90 2024-04-25T10:14:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.171+0000 7f0602b72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05fc15dce0 0x7f05fc1461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.171+0000 7f0602b72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05fc15dce0 0x7f05fc1461e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39320/0 (socket says 172.21.15.16:39320) 2024-04-25T10:14:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.171+0000 7f0602b72640 1 -- 172.21.15.16:0/3879617149 learned_addr learned my addr 172.21.15.16:0/3879617149 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.171+0000 7f0601b70640 1 --2- 172.21.15.16:0/3879617149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05fc0d35b0 0x7f05fc145ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.171+0000 7f0602371640 1 --2- 172.21.15.16:0/3879617149 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc0d1f90 0x7f05fc0c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.171+0000 7f0602b72640 1 -- 172.21.15.16:0/3879617149 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc0d1f90 msgr2=0x7f05fc0c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.171+0000 7f0602b72640 1 --2- 172.21.15.16:0/3879617149 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc0d1f90 0x7f05fc0c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.171+0000 7f0602b72640 1 -- 172.21.15.16:0/3879617149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05fc0d35b0 msgr2=0x7f05fc145ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.171+0000 7f0602b72640 1 --2- 172.21.15.16:0/3879617149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05fc0d35b0 0x7f05fc145ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.171+0000 7f0602b72640 1 -- 172.21.15.16:0/3879617149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc0cbac0 con 0x7f05fc15dce0 2024-04-25T10:14:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.171+0000 7f0602371640 1 --2- 172.21.15.16:0/3879617149 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc0d1f90 0x7f05fc0c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.171+0000 7f0602b72640 1 --2- 172.21.15.16:0/3879617149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05fc15dce0 0x7f05fc1461e0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009f40 tx=0x7f05f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.172+0000 7f05f2ffd640 1 -- 172.21.15.16:0/3879617149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4036050 con 0x7f05fc15dce0 2024-04-25T10:14:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.172+0000 7f05f2ffd640 1 -- 172.21.15.16:0/3879617149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f4042070 con 0x7f05fc15dce0 2024-04-25T10:14:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.172+0000 7f06045d5640 1 -- 172.21.15.16:0/3879617149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc0cbd80 con 0x7f05fc15dce0 2024-04-25T10:14:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.172+0000 7f05f2ffd640 1 -- 172.21.15.16:0/3879617149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4009510 con 0x7f05fc15dce0 2024-04-25T10:14:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.172+0000 7f06045d5640 1 -- 172.21.15.16:0/3879617149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc0cc340 con 0x7f05fc15dce0 2024-04-25T10:14:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.173+0000 7f06045d5640 1 -- 172.21.15.16:0/3879617149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05fc0749b0 con 0x7f05fc15dce0 2024-04-25T10:14:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.173+0000 7f05f2ffd640 1 -- 172.21.15.16:0/3879617149 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05f4007bf0 con 0x7f05fc15dce0 2024-04-25T10:14:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.174+0000 7f05f2ffd640 1 --2- 172.21.15.16:0/3879617149 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f05e006b4d0 0x7f05e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.174+0000 7f05f2ffd640 1 -- 172.21.15.16:0/3879617149 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05f40ab040 con 0x7f05fc15dce0 2024-04-25T10:14:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.174+0000 7f0602371640 1 --2- 172.21.15.16:0/3879617149 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f05e006b4d0 0x7f05e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.174+0000 7f0602371640 1 --2- 172.21.15.16:0/3879617149 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f05e006b4d0 0x7f05e006d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f05f8006fd0 tx=0x7f05f8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.178+0000 7f05f2ffd640 1 -- 172.21.15.16:0/3879617149 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f05f4079ec0 con 0x7f05fc15dce0 2024-04-25T10:14:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.393+0000 7f06045d5640 1 -- 172.21.15.16:0/3879617149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f05fc15ca70 con 0x7f05fc15dce0 2024-04-25T10:14:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.394+0000 7f05f2ffd640 1 -- 172.21.15.16:0/3879617149 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) v1 ==== 53+0+73 (secure 0 0 0) 0x7f05f40797c0 con 0x7f05fc15dce0 2024-04-25T10:14:32.398 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:14:32.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.396+0000 7f06045d5640 1 -- 172.21.15.16:0/3879617149 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f05e006b4d0 msgr2=0x7f05e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.396+0000 7f06045d5640 1 --2- 172.21.15.16:0/3879617149 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f05e006b4d0 0x7f05e006d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f05f8006fd0 tx=0x7f05f8008040 comp rx=0 tx=0).stop 2024-04-25T10:14:32.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.396+0000 7f06045d5640 1 -- 172.21.15.16:0/3879617149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05fc15dce0 msgr2=0x7f05fc1461e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.396+0000 7f06045d5640 1 --2- 172.21.15.16:0/3879617149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05fc15dce0 0x7f05fc1461e0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009f40 tx=0x7f05f4004a40 comp rx=0 tx=0).stop 2024-04-25T10:14:32.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.396+0000 7f06045d5640 1 -- 172.21.15.16:0/3879617149 shutdown_connections 2024-04-25T10:14:32.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.396+0000 7f06045d5640 1 --2- 172.21.15.16:0/3879617149 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f05e006b4d0 0x7f05e006d990 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.396+0000 7f06045d5640 1 --2- 172.21.15.16:0/3879617149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05fc15dce0 0x7f05fc1461e0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.396+0000 7f06045d5640 1 --2- 172.21.15.16:0/3879617149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05fc0d35b0 0x7f05fc145ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.396+0000 7f06045d5640 1 --2- 172.21.15.16:0/3879617149 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f05fc0d1f90 0x7f05fc0c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.396+0000 7f06045d5640 1 -- 172.21.15.16:0/3879617149 >> 172.21.15.16:0/3879617149 conn(0x7f05fc072900 msgr2=0x7f05fc0b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.396+0000 7f06045d5640 1 -- 172.21.15.16:0/3879617149 shutdown_connections 2024-04-25T10:14:32.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:32.396+0000 7f06045d5640 1 -- 172.21.15.16:0/3879617149 wait complete. 2024-04-25T10:14:32.410 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:14:32.410 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:14:32.410 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-04-25T10:14:32.411 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:14:32.411 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:32.411 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:14:32.411 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:14:32.411 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:14:32.411 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:32.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:32.411 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-04-25T10:14:32.427 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:14:32.427 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:14:32.427 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:14:32.427 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:32.427 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:14:32.427 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:32.427 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:14:32.427 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:32.427 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-04-25T10:14:32.428 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:14:32.428 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:32.428 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-04-25T10:14:32.428 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:14:32.428 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:32.428 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-04-25T10:14:32.428 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:14:32.428 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42088sec preferred_lft 42088sec 2024-04-25T10:14:32.428 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-25T10:14:32.428 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:32.428 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:14:32.428 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:32.428 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-04-25T10:14:32.428 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:14:32.429 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:32.429 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:14:32.429 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:32.429 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f06b:fdff:fed4:a335/64 scope link 2024-04-25T10:14:32.429 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:32.429 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:32.429 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:32.429 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::90ad:4fff:fe91:3c33/64 scope link 2024-04-25T10:14:32.429 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:32.429 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:32.429 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether be:14:4a:3c:16:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:32.429 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::bc14:4aff:fe3c:1605/64 scope link 2024-04-25T10:14:32.429 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:32.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:32.430 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:14:32.490 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:14:32.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:32.492 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-04-25T10:14:32.547 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-04-25T10:14:32.548 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:14:32.548 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:14:32.548 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:14:32.548 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:14:32.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:32.548 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:14:32.548 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:14:32.548 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:14:32.548 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:14:32.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:14:32.549 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:14:32.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:32.633 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:14:32.648 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:14:32.648 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:14:32.648 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:32.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:32.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-04-25T10:14:32.712 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:32.713 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:32.768 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:32.769 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:32.769 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:32.823 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:14:32.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:32.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:32.906 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:32.906 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:14:32.906 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:14:32.914 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:32.915 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:14:32.915 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:14:32.915 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:14:32.915 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:14:32.921 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:32.921 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:14:32.921 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:14:32.921 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:14:32.921 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:14:32.921 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:14:32.921 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:14:32.921 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:14:32.922 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:14:32.922 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:14:32.922 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:14:32.922 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:14:32.922 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:14:32.923 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:14:32.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:32.924 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:32.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:32.989 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:14:33.049 INFO:teuthology.orchestra.run.smithi016.stdout:client36842 2024-04-25T10:14:33.049 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:14:33.049 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:33.058 INFO:tasks.ceph.mds.c.smithi016.stderr:2024-04-25T10:14:33.054+0000 7f3ec1f21640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:14:33.073 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:33.073 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:33.073 DEBUG:teuthology.orchestra.run.smithi016:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:14:33.073 DEBUG:teuthology.orchestra.run.smithi016:> parent = os.path.join(mount_point, "parent") 2024-04-25T10:14:33.073 DEBUG:teuthology.orchestra.run.smithi016:> os.mkdir(parent) 2024-04-25T10:14:33.074 DEBUG:teuthology.orchestra.run.smithi016:> flushed = os.path.join(parent, "flushed") 2024-04-25T10:14:33.074 DEBUG:teuthology.orchestra.run.smithi016:> os.mkdir(flushed) 2024-04-25T10:14:33.074 DEBUG:teuthology.orchestra.run.smithi016:> for f in ["alpha", "bravo", "charlie"]: 2024-04-25T10:14:33.074 DEBUG:teuthology.orchestra.run.smithi016:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-04-25T10:14:33.074 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:33.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:33.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:33.196 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:14:33.257 INFO:teuthology.orchestra.run.smithi016.stdout:./ 2024-04-25T10:14:33.257 INFO:teuthology.orchestra.run.smithi016.stdout:./parent 2024-04-25T10:14:33.257 INFO:teuthology.orchestra.run.smithi016.stdout:./parent/flushed 2024-04-25T10:14:33.257 INFO:teuthology.orchestra.run.smithi016.stdout:./parent/flushed/charlie 2024-04-25T10:14:33.257 INFO:teuthology.orchestra.run.smithi016.stdout:./parent/flushed/alpha 2024-04-25T10:14:33.258 INFO:teuthology.orchestra.run.smithi016.stdout:./parent/flushed/bravo 2024-04-25T10:14:33.258 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:33.258 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:33.258 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:33.258 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:33.258 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:14:33.258 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:33.364 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:14:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:33.374 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:33.374 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:33.374 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:33.374 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:33.374 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T10:14:33.374 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:33.481 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627776 2024-04-25T10:14:33.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:33.489 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:33.489 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:33.489 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:33.489 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:33.489 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T10:14:33.489 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:33.597 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627777 2024-04-25T10:14:33.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:33.606 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:33.606 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:33.606 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:33.606 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:33.606 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T10:14:33.606 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:33.713 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627780 2024-04-25T10:14:33.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:33.722 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:33.722 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:33.722 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:33.722 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:33.723 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T10:14:33.723 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:33.830 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627778 2024-04-25T10:14:33.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:33.839 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:33.839 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:33.839 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:33.839 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:33.839 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-25T10:14:33.839 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:33.945 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627779 2024-04-25T10:14:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:33.954 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:14:33.954 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:34.011 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:14:34.011 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:34.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:34.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.202+0000 7f9cac438640 1 Processor -- start 2024-04-25T10:14:34.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.202+0000 7f9cac438640 1 -- start start 2024-04-25T10:14:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.203+0000 7f9cac438640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9ca409a150 0x7f9ca406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.203+0000 7f9cac438640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca406c1f0 0x7f9ca40cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.203+0000 7f9cac438640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca406c680 con 0x7f9ca415ba10 2024-04-25T10:14:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.203+0000 7f9cac438640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca40cbe50 con 0x7f9ca409a150 2024-04-25T10:14:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.203+0000 7f9cac438640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca40cfe80 con 0x7f9ca406c1f0 2024-04-25T10:14:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.203+0000 7f9caa1d4640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9ca415ba10 0x7f9ca4099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45158/0 (socket says 172.21.15.16:45158) 2024-04-25T10:14:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.203+0000 7f9caa1d4640 1 -- 172.21.15.16:0/371010976 learned_addr learned my addr 172.21.15.16:0/371010976 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.203+0000 7f9ca99d3640 1 --2- 172.21.15.16:0/371010976 >> v2:172.21.15.186:3300/0 conn(0x7f9ca409a150 0x7f9ca406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.203+0000 7f9caa9d5640 1 --2- 172.21.15.16:0/371010976 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca406c1f0 0x7f9ca40cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.203+0000 7f9ca91d2640 1 -- 172.21.15.16:0/371010976 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582901913 0 0) 0x7f9ca0001110 con 0x7f9ca415ba10 2024-04-25T10:14:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.203+0000 7f9ca91d2640 1 -- 172.21.15.16:0/371010976 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c90003520 con 0x7f9ca415ba10 2024-04-25T10:14:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca91d2640 1 -- 172.21.15.16:0/371010976 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1485640932 0 0) 0x7f9c90003520 con 0x7f9ca415ba10 2024-04-25T10:14:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca91d2640 1 -- 172.21.15.16:0/371010976 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca406c1f0 msgr2=0x7f9ca40cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca91d2640 1 --2- 172.21.15.16:0/371010976 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca406c1f0 0x7f9ca40cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca91d2640 1 -- 172.21.15.16:0/371010976 >> v2:172.21.15.186:3300/0 conn(0x7f9ca409a150 msgr2=0x7f9ca406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca91d2640 1 --2- 172.21.15.16:0/371010976 >> v2:172.21.15.186:3300/0 conn(0x7f9ca409a150 0x7f9ca406bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca91d2640 1 -- 172.21.15.16:0/371010976 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca406c900 con 0x7f9ca415ba10 2024-04-25T10:14:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca91d2640 1 -- 172.21.15.16:0/371010976 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f9ca0001840 con 0x7f9ca415ba10 2024-04-25T10:14:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca91d2640 1 -- 172.21.15.16:0/371010976 >> v1:172.21.15.16:6789/0 conn(0x7f9ca415ba10 legacy=0x7f9ca4099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca91d2640 1 --2- 172.21.15.16:0/371010976 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca406c1f0 0x7f9c900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca91d2640 1 --2- 172.21.15.16:0/371010976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c90003f30 0x7f9c90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca91d2640 1 --2- 172.21.15.16:0/371010976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c90004d40 0x7f9c900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca91d2640 1 -- 172.21.15.16:0/371010976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca40cfe80 con 0x7f9c90003f30 2024-04-25T10:14:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca91d2640 1 -- 172.21.15.16:0/371010976 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9000b9c0 con 0x7f9c90004d40 2024-04-25T10:14:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca91d2640 1 -- 172.21.15.16:0/371010976 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9000bb40 con 0x7f9ca406c1f0 2024-04-25T10:14:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca99d3640 1 --2- 172.21.15.16:0/371010976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c90003f30 0x7f9c90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9caa1d4640 1 --2- 172.21.15.16:0/371010976 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca406c1f0 0x7f9c900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9caa9d5640 1 --2- 172.21.15.16:0/371010976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c90004d40 0x7f9c900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca99d3640 1 -- 172.21.15.16:0/371010976 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca406c1f0 msgr2=0x7f9c900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca99d3640 1 --2- 172.21.15.16:0/371010976 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca406c1f0 0x7f9c900039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca99d3640 1 -- 172.21.15.16:0/371010976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c90004d40 msgr2=0x7f9c900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca99d3640 1 --2- 172.21.15.16:0/371010976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c90004d40 0x7f9c900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9ca99d3640 1 -- 172.21.15.16:0/371010976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c90003520 con 0x7f9c90003f30 2024-04-25T10:14:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.204+0000 7f9caa9d5640 1 --2- 172.21.15.16:0/371010976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c90004d40 0x7f9c900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.205+0000 7f9ca99d3640 1 --2- 172.21.15.16:0/371010976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c90003f30 0x7f9c90004800 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009ac0 tx=0x7f9c90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa946e0ee33d5081 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.205+0000 7f9ca91d2640 1 -- 172.21.15.16:0/371010976 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94036070 con 0x7f9c90003f30 2024-04-25T10:14:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.205+0000 7f9ca91d2640 1 -- 172.21.15.16:0/371010976 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c94031400 con 0x7f9c90003f30 2024-04-25T10:14:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.205+0000 7f9ca91d2640 1 -- 172.21.15.16:0/371010976 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94009690 con 0x7f9c90003f30 2024-04-25T10:14:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.205+0000 7f9cac438640 1 -- 172.21.15.16:0/371010976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c90003f30 msgr2=0x7f9c90004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.205+0000 7f9cac438640 1 --2- 172.21.15.16:0/371010976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c90003f30 0x7f9c90004800 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009ac0 tx=0x7f9c90005830 comp rx=0 tx=0).stop 2024-04-25T10:14:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.205+0000 7f9caa9d5640 1 -- 172.21.15.16:0/371010976 reap_dead start 2024-04-25T10:14:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.206+0000 7f9cac438640 1 -- 172.21.15.16:0/371010976 shutdown_connections 2024-04-25T10:14:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.206+0000 7f9cac438640 1 -- 172.21.15.16:0/371010976 >> 172.21.15.16:0/371010976 conn(0x7f9ca40b32f0 msgr2=0x7f9ca40b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.206+0000 7f9cac438640 1 -- 172.21.15.16:0/371010976 shutdown_connections 2024-04-25T10:14:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.206+0000 7f9cac438640 1 -- 172.21.15.16:0/371010976 wait complete. 2024-04-25T10:14:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.207+0000 7f9cac438640 1 Processor -- start 2024-04-25T10:14:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.207+0000 7f9cac438640 1 -- start start 2024-04-25T10:14:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9cac438640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c90003f30 0x7f9ca41f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9cac438640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca4159fd0 0x7f9ca40c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9cac438640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ca40ce280 0x7f9ca40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9cac438640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca40cd220 con 0x7f9c90003f30 2024-04-25T10:14:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9cac438640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca406cec0 con 0x7f9ca40ce280 2024-04-25T10:14:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9cac438640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca40cd3a0 con 0x7f9ca4159fd0 2024-04-25T10:14:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9caa1d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c90003f30 0x7f9ca41f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9caa1d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c90003f30 0x7f9ca41f53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37854/0 (socket says 172.21.15.16:37854) 2024-04-25T10:14:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9caa1d4640 1 -- 172.21.15.16:0/3178059400 learned_addr learned my addr 172.21.15.16:0/3178059400 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9ca99d3640 1 --2- 172.21.15.16:0/3178059400 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca4159fd0 0x7f9ca40c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9caa9d5640 1 --2- 172.21.15.16:0/3178059400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ca40ce280 0x7f9ca40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9caa1d4640 1 -- 172.21.15.16:0/3178059400 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca4159fd0 msgr2=0x7f9ca40c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9caa1d4640 1 --2- 172.21.15.16:0/3178059400 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca4159fd0 0x7f9ca40c6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9caa1d4640 1 -- 172.21.15.16:0/3178059400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ca40ce280 msgr2=0x7f9ca40ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9caa1d4640 1 --2- 172.21.15.16:0/3178059400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ca40ce280 0x7f9ca40ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9caa1d4640 1 -- 172.21.15.16:0/3178059400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca40b4550 con 0x7f9c90003f30 2024-04-25T10:14:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9caa9d5640 1 --2- 172.21.15.16:0/3178059400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ca40ce280 0x7f9ca40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9caa1d4640 1 --2- 172.21.15.16:0/3178059400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c90003f30 0x7f9ca41f53f0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0007290 tx=0x7f9ca0007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9c9affd640 1 -- 172.21.15.16:0/3178059400 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca000f9c0 con 0x7f9c90003f30 2024-04-25T10:14:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9c9affd640 1 -- 172.21.15.16:0/3178059400 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca0018070 con 0x7f9c90003f30 2024-04-25T10:14:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9c9affd640 1 -- 172.21.15.16:0/3178059400 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca000fde0 con 0x7f9c90003f30 2024-04-25T10:14:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9cac438640 1 -- 172.21.15.16:0/3178059400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca40b4830 con 0x7f9c90003f30 2024-04-25T10:14:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.208+0000 7f9cac438640 1 -- 172.21.15.16:0/3178059400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca40b4df0 con 0x7f9c90003f30 2024-04-25T10:14:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.209+0000 7f9cac438640 1 -- 172.21.15.16:0/3178059400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca40749b0 con 0x7f9c90003f30 2024-04-25T10:14:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.210+0000 7f9c9affd640 1 -- 172.21.15.16:0/3178059400 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ca001d460 con 0x7f9c90003f30 2024-04-25T10:14:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.210+0000 7f9c9affd640 1 --2- 172.21.15.16:0/3178059400 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9c8806b500 0x7f9c8806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.210+0000 7f9c9affd640 1 -- 172.21.15.16:0/3178059400 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ca008a360 con 0x7f9c90003f30 2024-04-25T10:14:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.210+0000 7f9ca99d3640 1 --2- 172.21.15.16:0/3178059400 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9c8806b500 0x7f9c8806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.211+0000 7f9ca99d3640 1 --2- 172.21.15.16:0/3178059400 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9c8806b500 0x7f9c8806d9c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9c94033000 tx=0x7f9c94008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.214+0000 7f9c9affd640 1 -- 172.21.15.16:0/3178059400 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9ca406c680 con 0x7f9c90003f30 2024-04-25T10:14:34.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.396+0000 7f9cac438640 1 -- 172.21.15.16:0/3178059400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ca409a570 con 0x7f9c90003f30 2024-04-25T10:14:34.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.397+0000 7f9c9affd640 1 -- 172.21.15.16:0/3178059400 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+16683 (secure 0 0 0) 0x7f9ca0059270 con 0x7f9c90003f30 2024-04-25T10:14:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:34.401 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":115,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:14:28.393191+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:25.025480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:25.726747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:34.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.399+0000 7f9cac438640 1 -- 172.21.15.16:0/3178059400 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9c8806b500 msgr2=0x7f9c8806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.399+0000 7f9cac438640 1 --2- 172.21.15.16:0/3178059400 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9c8806b500 0x7f9c8806d9c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9c94033000 tx=0x7f9c94008030 comp rx=0 tx=0).stop 2024-04-25T10:14:34.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.399+0000 7f9cac438640 1 -- 172.21.15.16:0/3178059400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c90003f30 msgr2=0x7f9ca41f53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.399+0000 7f9cac438640 1 --2- 172.21.15.16:0/3178059400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c90003f30 0x7f9ca41f53f0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0007290 tx=0x7f9ca0007750 comp rx=0 tx=0).stop 2024-04-25T10:14:34.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.399+0000 7f9cac438640 1 -- 172.21.15.16:0/3178059400 shutdown_connections 2024-04-25T10:14:34.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.399+0000 7f9cac438640 1 --2- 172.21.15.16:0/3178059400 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9c8806b500 0x7f9c8806d9c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.399+0000 7f9cac438640 1 --2- 172.21.15.16:0/3178059400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ca40ce280 0x7f9ca40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.399+0000 7f9cac438640 1 --2- 172.21.15.16:0/3178059400 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca4159fd0 0x7f9ca40c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.399+0000 7f9cac438640 1 --2- 172.21.15.16:0/3178059400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c90003f30 0x7f9ca41f53f0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.399+0000 7f9cac438640 1 -- 172.21.15.16:0/3178059400 >> 172.21.15.16:0/3178059400 conn(0x7f9ca40b32f0 msgr2=0x7f9ca415ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:34.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.399+0000 7f9cac438640 1 -- 172.21.15.16:0/3178059400 shutdown_connections 2024-04-25T10:14:34.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.399+0000 7f9cac438640 1 -- 172.21.15.16:0/3178059400 wait complete. 2024-04-25T10:14:34.414 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:14:25.025480+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:14:25.726747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:34.415 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:34.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.515+0000 7fb2d886f640 1 Processor -- start 2024-04-25T10:14:34.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.515+0000 7fb2d886f640 1 -- start start 2024-04-25T10:14:34.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.516+0000 7fb2d886f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb2d40ce9d0 0x7fb2d40cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.516+0000 7fb2d886f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb2d40cc300 0x7fb2d40cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.516+0000 7fb2d886f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d406dcb0 con 0x7fb2d4144e70 2024-04-25T10:14:34.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.516+0000 7fb2d886f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d415bfe0 con 0x7fb2d40ce9d0 2024-04-25T10:14:34.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.516+0000 7fb2d886f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d406df90 con 0x7fb2d40cc300 2024-04-25T10:14:34.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.516+0000 7fb2cbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb2d40ce9d0 0x7fb2d40cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.516+0000 7fb2cbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb2d40ce9d0 0x7fb2d40cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37862/0 (socket says 172.21.15.16:37862) 2024-04-25T10:14:34.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.516+0000 7fb2cbfff640 1 -- 172.21.15.16:0/2880266985 learned_addr learned my addr 172.21.15.16:0/2880266985 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:34.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.516+0000 7fb2cbfff640 1 -- 172.21.15.16:0/2880266985 >> v2:172.21.15.186:3300/0 conn(0x7fb2d40cc300 msgr2=0x7fb2d40cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.516+0000 7fb2cbfff640 1 --2- 172.21.15.16:0/2880266985 >> v2:172.21.15.186:3300/0 conn(0x7fb2d40cc300 0x7fb2d40cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.516+0000 7fb2cbfff640 1 -- 172.21.15.16:0/2880266985 >> v1:172.21.15.186:6789/0 conn(0x7fb2d4144e70 legacy=0x7fb2d406d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.516+0000 7fb2cbfff640 1 -- 172.21.15.16:0/2880266985 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d406e270 con 0x7fb2d40ce9d0 2024-04-25T10:14:34.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.517+0000 7fb2cbfff640 1 --2- 172.21.15.16:0/2880266985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d40ce9d0 0x7fb2d40cee60 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fb2d41458f0 tx=0x7fb2bc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e079fdacd01651c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.517+0000 7fb2d1d9b640 1 -- 172.21.15.16:0/2880266985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc037070 con 0x7fb2d40ce9d0 2024-04-25T10:14:34.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.517+0000 7fb2d1d9b640 1 -- 172.21.15.16:0/2880266985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc032440 con 0x7fb2d40ce9d0 2024-04-25T10:14:34.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.517+0000 7fb2d1d9b640 1 -- 172.21.15.16:0/2880266985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc032810 con 0x7fb2d40ce9d0 2024-04-25T10:14:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.518+0000 7fb2d886f640 1 -- 172.21.15.16:0/2880266985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d40ce9d0 msgr2=0x7fb2d40cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.518+0000 7fb2d886f640 1 --2- 172.21.15.16:0/2880266985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d40ce9d0 0x7fb2d40cee60 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fb2d41458f0 tx=0x7fb2bc02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.518+0000 7fb2d886f640 1 -- 172.21.15.16:0/2880266985 shutdown_connections 2024-04-25T10:14:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.518+0000 7fb2d886f640 1 --2- 172.21.15.16:0/2880266985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d40ce9d0 0x7fb2d40cee60 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.518+0000 7fb2d886f640 1 --2- 172.21.15.16:0/2880266985 >> v2:172.21.15.186:3300/0 conn(0x7fb2d40cc300 0x7fb2d40cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.518+0000 7fb2d886f640 1 -- 172.21.15.16:0/2880266985 >> 172.21.15.16:0/2880266985 conn(0x7fb2d406aba0 msgr2=0x7fb2d406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.518+0000 7fb2d886f640 1 -- 172.21.15.16:0/2880266985 shutdown_connections 2024-04-25T10:14:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.518+0000 7fb2d886f640 1 -- 172.21.15.16:0/2880266985 wait complete. 2024-04-25T10:14:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.519+0000 7fb2d886f640 1 Processor -- start 2024-04-25T10:14:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.519+0000 7fb2d886f640 1 -- start start 2024-04-25T10:14:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2d886f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d40cc300 0x7fb2d413c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2d886f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d40ce9d0 0x7fb2d41f2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2d886f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2d4144e70 0x7fb2d41f1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2d886f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d406d970 con 0x7fb2d40ce9d0 2024-04-25T10:14:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2d886f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d40cf3a0 con 0x7fb2d4144e70 2024-04-25T10:14:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2d886f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d41454b0 con 0x7fb2d40cc300 2024-04-25T10:14:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d40ce9d0 0x7fb2d41f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d40ce9d0 0x7fb2d41f2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37870/0 (socket says 172.21.15.16:37870) 2024-04-25T10:14:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2cbfff640 1 -- 172.21.15.16:0/3597056599 learned_addr learned my addr 172.21.15.16:0/3597056599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2d2d9d640 1 --2- 172.21.15.16:0/3597056599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2d4144e70 0x7fb2d41f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2d259c640 1 --2- 172.21.15.16:0/3597056599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d40cc300 0x7fb2d413c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2cbfff640 1 -- 172.21.15.16:0/3597056599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d40cc300 msgr2=0x7fb2d413c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2cbfff640 1 --2- 172.21.15.16:0/3597056599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d40cc300 0x7fb2d413c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2cbfff640 1 -- 172.21.15.16:0/3597056599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2d4144e70 msgr2=0x7fb2d41f1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2cbfff640 1 --2- 172.21.15.16:0/3597056599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2d4144e70 0x7fb2d41f1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2cbfff640 1 -- 172.21.15.16:0/3597056599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d413c760 con 0x7fb2d40ce9d0 2024-04-25T10:14:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2cbfff640 1 --2- 172.21.15.16:0/3597056599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d40ce9d0 0x7fb2d41f2920 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc000c00 tx=0x7fb2bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2caffd640 1 -- 172.21.15.16:0/3597056599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc041070 con 0x7fb2d40ce9d0 2024-04-25T10:14:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2caffd640 1 -- 172.21.15.16:0/3597056599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc03c870 con 0x7fb2d40ce9d0 2024-04-25T10:14:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2caffd640 1 -- 172.21.15.16:0/3597056599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc037040 con 0x7fb2d40ce9d0 2024-04-25T10:14:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.520+0000 7fb2d886f640 1 -- 172.21.15.16:0/3597056599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d41f2200 con 0x7fb2d40ce9d0 2024-04-25T10:14:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.521+0000 7fb2d886f640 1 -- 172.21.15.16:0/3597056599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d413cb10 con 0x7fb2d40ce9d0 2024-04-25T10:14:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.521+0000 7fb2d886f640 1 -- 172.21.15.16:0/3597056599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d41f2720 con 0x7fb2d40ce9d0 2024-04-25T10:14:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.522+0000 7fb2caffd640 1 -- 172.21.15.16:0/3597056599 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2bc044070 con 0x7fb2d40ce9d0 2024-04-25T10:14:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.522+0000 7fb2caffd640 1 --2- 172.21.15.16:0/3597056599 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb2b406b4d0 0x7fb2b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.522+0000 7fb2caffd640 1 -- 172.21.15.16:0/3597056599 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2bc0aadb0 con 0x7fb2d40ce9d0 2024-04-25T10:14:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.523+0000 7fb2d259c640 1 --2- 172.21.15.16:0/3597056599 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb2b406b4d0 0x7fb2b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.523+0000 7fb2d259c640 1 --2- 172.21.15.16:0/3597056599 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb2b406b4d0 0x7fb2b406d990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4006fd0 tx=0x7fb2c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.526+0000 7fb2caffd640 1 -- 172.21.15.16:0/3597056599 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb2d41f2720 con 0x7fb2d40ce9d0 2024-04-25T10:14:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.705+0000 7fb2d886f640 1 -- 172.21.15.16:0/3597056599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb2d40750b0 con 0x7fb2d40ce9d0 2024-04-25T10:14:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.705+0000 7fb2caffd640 1 -- 172.21.15.16:0/3597056599 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb2bc079de0 con 0x7fb2d40ce9d0 2024-04-25T10:14:34.709 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:14:34.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.707+0000 7fb2d886f640 1 -- 172.21.15.16:0/3597056599 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb2b406b4d0 msgr2=0x7fb2b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.707+0000 7fb2d886f640 1 --2- 172.21.15.16:0/3597056599 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb2b406b4d0 0x7fb2b406d990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4006fd0 tx=0x7fb2c4008040 comp rx=0 tx=0).stop 2024-04-25T10:14:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.707+0000 7fb2d886f640 1 -- 172.21.15.16:0/3597056599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d40ce9d0 msgr2=0x7fb2d41f2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.707+0000 7fb2d886f640 1 --2- 172.21.15.16:0/3597056599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d40ce9d0 0x7fb2d41f2920 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc000c00 tx=0x7fb2bc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.707+0000 7fb2d886f640 1 -- 172.21.15.16:0/3597056599 shutdown_connections 2024-04-25T10:14:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.707+0000 7fb2d886f640 1 --2- 172.21.15.16:0/3597056599 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb2b406b4d0 0x7fb2b406d990 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.707+0000 7fb2d886f640 1 --2- 172.21.15.16:0/3597056599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2d4144e70 0x7fb2d41f1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.707+0000 7fb2d886f640 1 --2- 172.21.15.16:0/3597056599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d40ce9d0 0x7fb2d41f2920 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.707+0000 7fb2d886f640 1 --2- 172.21.15.16:0/3597056599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d40cc300 0x7fb2d413c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.707+0000 7fb2d886f640 1 -- 172.21.15.16:0/3597056599 >> 172.21.15.16:0/3597056599 conn(0x7fb2d406aba0 msgr2=0x7fb2d40cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:34.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.707+0000 7fb2d886f640 1 -- 172.21.15.16:0/3597056599 shutdown_connections 2024-04-25T10:14:34.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.707+0000 7fb2d886f640 1 -- 172.21.15.16:0/3597056599 wait complete. 2024-04-25T10:14:34.722 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.821+0000 7efd2cf9e640 1 Processor -- start 2024-04-25T10:14:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.821+0000 7efd2cf9e640 1 -- start start 2024-04-25T10:14:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.822+0000 7efd2cf9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd280cf510 0x7efd280cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.822+0000 7efd2cf9e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efd280cff00 0x7efd280d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.822+0000 7efd2cf9e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd280cc830 con 0x7efd2815bbe0 2024-04-25T10:14:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.822+0000 7efd2cf9e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd280ccb10 con 0x7efd280cff00 2024-04-25T10:14:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.822+0000 7efd2cf9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd280ccdf0 con 0x7efd280cf510 2024-04-25T10:14:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.822+0000 7efd26d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efd280cff00 0x7efd280d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.823+0000 7efd26d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efd280cff00 0x7efd280d03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37874/0 (socket says 172.21.15.16:37874) 2024-04-25T10:14:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.823+0000 7efd26d9d640 1 -- 172.21.15.16:0/534932336 learned_addr learned my addr 172.21.15.16:0/534932336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.823+0000 7efd25d9b640 1 --2- 172.21.15.16:0/534932336 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd280cf510 0x7efd280cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.823+0000 7efd26d9d640 1 -- 172.21.15.16:0/534932336 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd280cf510 msgr2=0x7efd280cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.823+0000 7efd26d9d640 1 --2- 172.21.15.16:0/534932336 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd280cf510 0x7efd280cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.823+0000 7efd26d9d640 1 -- 172.21.15.16:0/534932336 >> v1:172.21.15.186:6789/0 conn(0x7efd2815bbe0 legacy=0x7efd280cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.823+0000 7efd26d9d640 1 -- 172.21.15.16:0/534932336 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd280cd0d0 con 0x7efd280cff00 2024-04-25T10:14:34.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.823+0000 7efd25d9b640 1 --2- 172.21.15.16:0/534932336 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd280cf510 0x7efd280cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:34.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.823+0000 7efd26d9d640 1 --2- 172.21.15.16:0/534932336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd280cff00 0x7efd280d03b0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7efd18009e50 tx=0x7efd1802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b16daea66a3a180 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.824+0000 7efd2559a640 1 -- 172.21.15.16:0/534932336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd18036070 con 0x7efd280cff00 2024-04-25T10:14:34.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.824+0000 7efd2559a640 1 -- 172.21.15.16:0/534932336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd18031600 con 0x7efd280cff00 2024-04-25T10:14:34.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.824+0000 7efd2559a640 1 -- 172.21.15.16:0/534932336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd180319d0 con 0x7efd280cff00 2024-04-25T10:14:34.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.824+0000 7efd2cf9e640 1 -- 172.21.15.16:0/534932336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd280cff00 msgr2=0x7efd280d03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.824+0000 7efd2cf9e640 1 --2- 172.21.15.16:0/534932336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd280cff00 0x7efd280d03b0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7efd18009e50 tx=0x7efd1802a6d0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.824+0000 7efd2cf9e640 1 -- 172.21.15.16:0/534932336 shutdown_connections 2024-04-25T10:14:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.824+0000 7efd2cf9e640 1 --2- 172.21.15.16:0/534932336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd280cff00 0x7efd280d03b0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.824+0000 7efd2cf9e640 1 --2- 172.21.15.16:0/534932336 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd280cf510 0x7efd280cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.824+0000 7efd2cf9e640 1 -- 172.21.15.16:0/534932336 >> 172.21.15.16:0/534932336 conn(0x7efd280b3430 msgr2=0x7efd280b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.824+0000 7efd2cf9e640 1 -- 172.21.15.16:0/534932336 shutdown_connections 2024-04-25T10:14:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.824+0000 7efd2cf9e640 1 -- 172.21.15.16:0/534932336 wait complete. 2024-04-25T10:14:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.826+0000 7efd2cf9e640 1 Processor -- start 2024-04-25T10:14:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.826+0000 7efd2cf9e640 1 -- start start 2024-04-25T10:14:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.826+0000 7efd2cf9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd280cf510 0x7efd280cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.826+0000 7efd2cf9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd280cff00 0x7efd2806ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.826+0000 7efd2cf9e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd2815bbe0 0x7efd280c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.826+0000 7efd2cf9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd280cc450 con 0x7efd280cff00 2024-04-25T10:14:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.826+0000 7efd2cf9e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2815bfc0 con 0x7efd2815bbe0 2024-04-25T10:14:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.826+0000 7efd2cf9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd280cc5d0 con 0x7efd280cf510 2024-04-25T10:14:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.826+0000 7efd25d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd280cff00 0x7efd2806ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.826+0000 7efd25d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd280cff00 0x7efd2806ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37878/0 (socket says 172.21.15.16:37878) 2024-04-25T10:14:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.827+0000 7efd25d9b640 1 -- 172.21.15.16:0/2994487617 learned_addr learned my addr 172.21.15.16:0/2994487617 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.827+0000 7efd2659c640 1 --2- 172.21.15.16:0/2994487617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd280cf510 0x7efd280cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.827+0000 7efd26d9d640 1 --2- 172.21.15.16:0/2994487617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd2815bbe0 0x7efd280c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.827+0000 7efd25d9b640 1 -- 172.21.15.16:0/2994487617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd280cf510 msgr2=0x7efd280cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.827+0000 7efd25d9b640 1 --2- 172.21.15.16:0/2994487617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd280cf510 0x7efd280cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.827+0000 7efd25d9b640 1 -- 172.21.15.16:0/2994487617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd2815bbe0 msgr2=0x7efd280c42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.827+0000 7efd25d9b640 1 --2- 172.21.15.16:0/2994487617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd2815bbe0 0x7efd280c42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.827+0000 7efd25d9b640 1 -- 172.21.15.16:0/2994487617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2806d390 con 0x7efd280cff00 2024-04-25T10:14:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.827+0000 7efd2659c640 1 --2- 172.21.15.16:0/2994487617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd280cf510 0x7efd280cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.827+0000 7efd25d9b640 1 --2- 172.21.15.16:0/2994487617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd280cff00 0x7efd2806ccb0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00c820 tx=0x7efd1c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.827+0000 7efd16ffd640 1 -- 172.21.15.16:0/2994487617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c004230 con 0x7efd280cff00 2024-04-25T10:14:34.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.827+0000 7efd16ffd640 1 -- 172.21.15.16:0/2994487617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1c0043d0 con 0x7efd280cff00 2024-04-25T10:14:34.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.827+0000 7efd2cf9e640 1 -- 172.21.15.16:0/2994487617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd281e7120 con 0x7efd280cff00 2024-04-25T10:14:34.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.827+0000 7efd16ffd640 1 -- 172.21.15.16:0/2994487617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c007a70 con 0x7efd280cff00 2024-04-25T10:14:34.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.827+0000 7efd2cf9e640 1 -- 172.21.15.16:0/2994487617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd281e76e0 con 0x7efd280cff00 2024-04-25T10:14:34.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.828+0000 7efd2cf9e640 1 -- 172.21.15.16:0/2994487617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd280750b0 con 0x7efd280cff00 2024-04-25T10:14:34.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.829+0000 7efd16ffd640 1 -- 172.21.15.16:0/2994487617 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd1c007c10 con 0x7efd280cff00 2024-04-25T10:14:34.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.829+0000 7efd16ffd640 1 --2- 172.21.15.16:0/2994487617 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efd0806b500 0x7efd0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.829+0000 7efd16ffd640 1 -- 172.21.15.16:0/2994487617 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd1c08a210 con 0x7efd280cff00 2024-04-25T10:14:34.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.829+0000 7efd2659c640 1 --2- 172.21.15.16:0/2994487617 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efd0806b500 0x7efd0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.830+0000 7efd2659c640 1 --2- 172.21.15.16:0/2994487617 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efd0806b500 0x7efd0806d9c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7efd20006fd0 tx=0x7efd20008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:34.833+0000 7efd16ffd640 1 -- 172.21.15.16:0/2994487617 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd1c015020 con 0x7efd280cff00 2024-04-25T10:14:35.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.016+0000 7efd2cf9e640 1 -- 172.21.15.16:0/2994487617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efd280c6ca0 con 0x7efd280cff00 2024-04-25T10:14:35.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.016+0000 7efd16ffd640 1 -- 172.21.15.16:0/2994487617 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) v1 ==== 104+0+11 (secure 0 0 0) 0x7efd1c058a40 con 0x7efd280cff00 2024-04-25T10:14:35.020 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:14:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.018+0000 7efd2cf9e640 1 -- 172.21.15.16:0/2994487617 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efd0806b500 msgr2=0x7efd0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.018+0000 7efd2cf9e640 1 --2- 172.21.15.16:0/2994487617 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efd0806b500 0x7efd0806d9c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7efd20006fd0 tx=0x7efd20008040 comp rx=0 tx=0).stop 2024-04-25T10:14:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.018+0000 7efd2cf9e640 1 -- 172.21.15.16:0/2994487617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd280cff00 msgr2=0x7efd2806ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.018+0000 7efd2cf9e640 1 --2- 172.21.15.16:0/2994487617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd280cff00 0x7efd2806ccb0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00c820 tx=0x7efd1c00cce0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.018+0000 7efd2cf9e640 1 -- 172.21.15.16:0/2994487617 shutdown_connections 2024-04-25T10:14:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.018+0000 7efd2cf9e640 1 --2- 172.21.15.16:0/2994487617 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7efd0806b500 0x7efd0806d9c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.018+0000 7efd2cf9e640 1 --2- 172.21.15.16:0/2994487617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd2815bbe0 0x7efd280c42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.018+0000 7efd2cf9e640 1 --2- 172.21.15.16:0/2994487617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd280cff00 0x7efd2806ccb0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.018+0000 7efd2cf9e640 1 --2- 172.21.15.16:0/2994487617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd280cf510 0x7efd280cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.018+0000 7efd2cf9e640 1 -- 172.21.15.16:0/2994487617 >> 172.21.15.16:0/2994487617 conn(0x7efd280b3430 msgr2=0x7efd280c4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.018+0000 7efd2cf9e640 1 -- 172.21.15.16:0/2994487617 shutdown_connections 2024-04-25T10:14:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.019+0000 7efd2cf9e640 1 -- 172.21.15.16:0/2994487617 wait complete. 2024-04-25T10:14:35.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:35.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.133+0000 7f3a9e83a640 1 Processor -- start 2024-04-25T10:14:35.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.133+0000 7f3a9e83a640 1 -- start start 2024-04-25T10:14:35.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.133+0000 7f3a9e83a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3a980b5c00 0x7f3a98168c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.133+0000 7f3a9e83a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a981627a0 con 0x7f3a9815dc80 2024-04-25T10:14:35.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.133+0000 7f3a9e83a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a981621e0 con 0x7f3a980b97c0 2024-04-25T10:14:35.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.133+0000 7f3a9e83a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a98161f80 con 0x7f3a980b5c00 2024-04-25T10:14:35.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.133+0000 7f3a97fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3a9815dc80 0x7f3a98161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45166/0 (socket says 172.21.15.16:45166) 2024-04-25T10:14:35.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.133+0000 7f3a97fff640 1 -- 172.21.15.16:0/3140248008 learned_addr learned my addr 172.21.15.16:0/3140248008 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:35.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a9cdd7640 1 --2- 172.21.15.16:0/3140248008 >> v2:172.21.15.186:3300/0 conn(0x7f3a980b5c00 0x7f3a98168c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a96ffd640 1 -- 172.21.15.16:0/3140248008 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3007838409 0 0) 0x7f3a981627a0 con 0x7f3a9815dc80 2024-04-25T10:14:35.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a96ffd640 1 -- 172.21.15.16:0/3140248008 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a84003520 con 0x7f3a9815dc80 2024-04-25T10:14:35.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a96ffd640 1 -- 172.21.15.16:0/3140248008 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3556557805 0 0) 0x7f3a84003520 con 0x7f3a9815dc80 2024-04-25T10:14:35.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a96ffd640 1 -- 172.21.15.16:0/3140248008 >> v2:172.21.15.186:3300/0 conn(0x7f3a980b5c00 msgr2=0x7f3a98168c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a96ffd640 1 --2- 172.21.15.16:0/3140248008 >> v2:172.21.15.186:3300/0 conn(0x7f3a980b5c00 0x7f3a98168c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a96ffd640 1 -- 172.21.15.16:0/3140248008 >> v1:172.21.15.186:6789/0 conn(0x7f3a980b97c0 legacy=0x7f3a980b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a96ffd640 1 -- 172.21.15.16:0/3140248008 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a981624c0 con 0x7f3a9815dc80 2024-04-25T10:14:35.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a96ffd640 1 -- 172.21.15.16:0/3140248008 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f3a80001670 con 0x7f3a9815dc80 2024-04-25T10:14:35.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a96ffd640 1 -- 172.21.15.16:0/3140248008 >> v1:172.21.15.16:6789/0 conn(0x7f3a9815dc80 legacy=0x7f3a98161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a96ffd640 1 --2- 172.21.15.16:0/3140248008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a840039b0 0x7f3a84003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a96ffd640 1 --2- 172.21.15.16:0/3140248008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a840043b0 0x7f3a84004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a96ffd640 1 --2- 172.21.15.16:0/3140248008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a84005080 0x7f3a8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a96ffd640 1 -- 172.21.15.16:0/3140248008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a98161f80 con 0x7f3a840043b0 2024-04-25T10:14:35.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a96ffd640 1 -- 172.21.15.16:0/3140248008 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8400bfe0 con 0x7f3a84005080 2024-04-25T10:14:35.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a96ffd640 1 -- 172.21.15.16:0/3140248008 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8400c1e0 con 0x7f3a840039b0 2024-04-25T10:14:35.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a977fe640 1 --2- 172.21.15.16:0/3140248008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a840043b0 0x7f3a84004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a9cdd7640 1 --2- 172.21.15.16:0/3140248008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a84005080 0x7f3a8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.134+0000 7f3a97fff640 1 --2- 172.21.15.16:0/3140248008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a840039b0 0x7f3a84003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.135+0000 7f3a977fe640 1 -- 172.21.15.16:0/3140248008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a840039b0 msgr2=0x7f3a84003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.135+0000 7f3a977fe640 1 --2- 172.21.15.16:0/3140248008 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a840039b0 0x7f3a84003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.135+0000 7f3a977fe640 1 -- 172.21.15.16:0/3140248008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a84005080 msgr2=0x7f3a8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.135+0000 7f3a977fe640 1 --2- 172.21.15.16:0/3140248008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a84005080 0x7f3a8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.135+0000 7f3a977fe640 1 -- 172.21.15.16:0/3140248008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a84003520 con 0x7f3a840043b0 2024-04-25T10:14:35.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.135+0000 7f3a9cdd7640 1 -- 172.21.15.16:0/3140248008 reap_dead start 2024-04-25T10:14:35.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.135+0000 7f3a977fe640 1 --2- 172.21.15.16:0/3140248008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a840043b0 0x7f3a84004880 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f3a88001860 tx=0x7f3a84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce55e98a1b5a8407 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.135+0000 7f3a96ffd640 1 -- 172.21.15.16:0/3140248008 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88037070 con 0x7f3a840043b0 2024-04-25T10:14:35.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.136+0000 7f3a96ffd640 1 -- 172.21.15.16:0/3140248008 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8802abe0 con 0x7f3a840043b0 2024-04-25T10:14:35.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.136+0000 7f3a96ffd640 1 -- 172.21.15.16:0/3140248008 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88001430 con 0x7f3a840043b0 2024-04-25T10:14:35.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.136+0000 7f3a9e83a640 1 -- 172.21.15.16:0/3140248008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a840043b0 msgr2=0x7f3a84004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.136+0000 7f3a9e83a640 1 --2- 172.21.15.16:0/3140248008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a840043b0 0x7f3a84004880 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f3a88001860 tx=0x7f3a84005830 comp rx=0 tx=0).stop 2024-04-25T10:14:35.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.136+0000 7f3a9e83a640 1 -- 172.21.15.16:0/3140248008 shutdown_connections 2024-04-25T10:14:35.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.136+0000 7f3a9e83a640 1 --2- 172.21.15.16:0/3140248008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a840043b0 0x7f3a84004880 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.136+0000 7f3a9e83a640 1 -- 172.21.15.16:0/3140248008 >> 172.21.15.16:0/3140248008 conn(0x7f3a980d04f0 msgr2=0x7f3a980d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:35.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.136+0000 7f3a9e83a640 1 -- 172.21.15.16:0/3140248008 shutdown_connections 2024-04-25T10:14:35.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.136+0000 7f3a9e83a640 1 -- 172.21.15.16:0/3140248008 wait complete. 2024-04-25T10:14:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.137+0000 7f3a9e83a640 1 Processor -- start 2024-04-25T10:14:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.137+0000 7f3a9e83a640 1 -- start start 2024-04-25T10:14:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.138+0000 7f3a9e83a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a840043b0 0x7f3a9815a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.138+0000 7f3a9e83a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a980b6e80 0x7f3a980b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.138+0000 7f3a9e83a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a980aaf10 0x7f3a980ab390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.138+0000 7f3a9e83a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a980b54f0 con 0x7f3a980b6e80 2024-04-25T10:14:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.138+0000 7f3a9e83a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9815e060 con 0x7f3a840043b0 2024-04-25T10:14:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.138+0000 7f3a9e83a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a980b5670 con 0x7f3a980aaf10 2024-04-25T10:14:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.138+0000 7f3a977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a980b6e80 0x7f3a980b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.138+0000 7f3a977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a980b6e80 0x7f3a980b7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37884/0 (socket says 172.21.15.16:37884) 2024-04-25T10:14:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.138+0000 7f3a977fe640 1 -- 172.21.15.16:0/3270771032 learned_addr learned my addr 172.21.15.16:0/3270771032 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.138+0000 7f3a97fff640 1 --2- 172.21.15.16:0/3270771032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a840043b0 0x7f3a9815a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.138+0000 7f3a9cdd7640 1 --2- 172.21.15.16:0/3270771032 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a980aaf10 0x7f3a980ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.139+0000 7f3a977fe640 1 -- 172.21.15.16:0/3270771032 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a980aaf10 msgr2=0x7f3a980ab390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.139+0000 7f3a977fe640 1 --2- 172.21.15.16:0/3270771032 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a980aaf10 0x7f3a980ab390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.139+0000 7f3a977fe640 1 -- 172.21.15.16:0/3270771032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a840043b0 msgr2=0x7f3a9815a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.139+0000 7f3a977fe640 1 --2- 172.21.15.16:0/3270771032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a840043b0 0x7f3a9815a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.139+0000 7f3a977fe640 1 -- 172.21.15.16:0/3270771032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a980ab8d0 con 0x7f3a980b6e80 2024-04-25T10:14:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.139+0000 7f3a977fe640 1 --2- 172.21.15.16:0/3270771032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a980b6e80 0x7f3a980b7300 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f3a8802a670 tx=0x7f3a88001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.139+0000 7f3a94ff9640 1 -- 172.21.15.16:0/3270771032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88040020 con 0x7f3a980b6e80 2024-04-25T10:14:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.139+0000 7f3a94ff9640 1 -- 172.21.15.16:0/3270771032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a88038040 con 0x7f3a980b6e80 2024-04-25T10:14:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.139+0000 7f3a94ff9640 1 -- 172.21.15.16:0/3270771032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88037070 con 0x7f3a980b6e80 2024-04-25T10:14:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.139+0000 7f3a9e83a640 1 -- 172.21.15.16:0/3270771032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9815e310 con 0x7f3a980b6e80 2024-04-25T10:14:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.139+0000 7f3a9e83a640 1 -- 172.21.15.16:0/3270771032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9815e8d0 con 0x7f3a980b6e80 2024-04-25T10:14:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.140+0000 7f3a9e83a640 1 -- 172.21.15.16:0/3270771032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a980b7840 con 0x7f3a980b6e80 2024-04-25T10:14:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.140+0000 7f3a94ff9640 1 -- 172.21.15.16:0/3270771032 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a88043020 con 0x7f3a980b6e80 2024-04-25T10:14:35.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.141+0000 7f3a94ff9640 1 --2- 172.21.15.16:0/3270771032 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a7806b500 0x7f3a7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.141+0000 7f3a94ff9640 1 -- 172.21.15.16:0/3270771032 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a880aa1c0 con 0x7f3a980b6e80 2024-04-25T10:14:35.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.141+0000 7f3a97fff640 1 --2- 172.21.15.16:0/3270771032 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a7806b500 0x7f3a7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.142+0000 7f3a97fff640 1 --2- 172.21.15.16:0/3270771032 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a7806b500 0x7f3a7806d9c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3a80000f60 tx=0x7f3a80000ed0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.145+0000 7f3a94ff9640 1 -- 172.21.15.16:0/3270771032 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a8804a030 con 0x7f3a980b6e80 2024-04-25T10:14:35.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.329+0000 7f3a9e83a640 1 -- 172.21.15.16:0/3270771032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3a980750b0 con 0x7f3a980b6e80 2024-04-25T10:14:35.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.330+0000 7f3a94ff9640 1 -- 172.21.15.16:0/3270771032 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3a88079190 con 0x7f3a980b6e80 2024-04-25T10:14:35.334 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:14:35.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.331+0000 7f3a9e83a640 1 -- 172.21.15.16:0/3270771032 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a7806b500 msgr2=0x7f3a7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.331+0000 7f3a9e83a640 1 --2- 172.21.15.16:0/3270771032 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a7806b500 0x7f3a7806d9c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3a80000f60 tx=0x7f3a80000ed0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.331+0000 7f3a9e83a640 1 -- 172.21.15.16:0/3270771032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a980b6e80 msgr2=0x7f3a980b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.331+0000 7f3a9e83a640 1 --2- 172.21.15.16:0/3270771032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a980b6e80 0x7f3a980b7300 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f3a8802a670 tx=0x7f3a88001230 comp rx=0 tx=0).stop 2024-04-25T10:14:35.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.331+0000 7f3a9e83a640 1 -- 172.21.15.16:0/3270771032 shutdown_connections 2024-04-25T10:14:35.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.331+0000 7f3a9e83a640 1 --2- 172.21.15.16:0/3270771032 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3a7806b500 0x7f3a7806d9c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.331+0000 7f3a9e83a640 1 --2- 172.21.15.16:0/3270771032 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a980aaf10 0x7f3a980ab390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.331+0000 7f3a9e83a640 1 --2- 172.21.15.16:0/3270771032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a980b6e80 0x7f3a980b7300 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.331+0000 7f3a9e83a640 1 --2- 172.21.15.16:0/3270771032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a840043b0 0x7f3a9815a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.331+0000 7f3a9e83a640 1 -- 172.21.15.16:0/3270771032 >> 172.21.15.16:0/3270771032 conn(0x7f3a980d04f0 msgr2=0x7f3a980b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:35.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.332+0000 7f3a9e83a640 1 -- 172.21.15.16:0/3270771032 shutdown_connections 2024-04-25T10:14:35.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.332+0000 7f3a9e83a640 1 -- 172.21.15.16:0/3270771032 wait complete. 2024-04-25T10:14:35.347 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:14:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.447+0000 7f1286d70640 1 Processor -- start 2024-04-25T10:14:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.447+0000 7f1286d70640 1 -- start start 2024-04-25T10:14:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.447+0000 7f1286d70640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f12801610b0 0x7f12800b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.447+0000 7f1286d70640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12800cef40 0x7f12800d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.447+0000 7f1286d70640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1280162040 con 0x7f12801618d0 2024-04-25T10:14:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.447+0000 7f1286d70640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12800d1ea0 con 0x7f12801610b0 2024-04-25T10:14:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.447+0000 7f1286d70640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1280162320 con 0x7f12800cef40 2024-04-25T10:14:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f127ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f12801618d0 0x7f12800ce830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45182/0 (socket says 172.21.15.16:45182) 2024-04-25T10:14:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f127ffff640 1 -- 172.21.15.16:0/2948352006 learned_addr learned my addr 172.21.15.16:0/2948352006 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f1284b0c640 1 --2- 172.21.15.16:0/2948352006 >> v2:172.21.15.186:3300/0 conn(0x7f12801610b0 0x7f12800b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f128530d640 1 --2- 172.21.15.16:0/2948352006 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12800cef40 0x7f12800d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f127f7fe640 1 -- 172.21.15.16:0/2948352006 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106646259 0 0) 0x7f1280162040 con 0x7f12801618d0 2024-04-25T10:14:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f127f7fe640 1 -- 172.21.15.16:0/2948352006 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f126c003520 con 0x7f12801618d0 2024-04-25T10:14:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f127f7fe640 1 -- 172.21.15.16:0/2948352006 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1148901526 0 0) 0x7f126c003520 con 0x7f12801618d0 2024-04-25T10:14:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f127f7fe640 1 -- 172.21.15.16:0/2948352006 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12800cef40 msgr2=0x7f12800d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f127f7fe640 1 --2- 172.21.15.16:0/2948352006 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12800cef40 0x7f12800d2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f127f7fe640 1 -- 172.21.15.16:0/2948352006 >> v2:172.21.15.186:3300/0 conn(0x7f12801610b0 msgr2=0x7f12800b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f127f7fe640 1 --2- 172.21.15.16:0/2948352006 >> v2:172.21.15.186:3300/0 conn(0x7f12801610b0 0x7f12800b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f127f7fe640 1 -- 172.21.15.16:0/2948352006 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1280162600 con 0x7f12801618d0 2024-04-25T10:14:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f127f7fe640 1 -- 172.21.15.16:0/2948352006 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f1268001690 con 0x7f12801618d0 2024-04-25T10:14:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f127f7fe640 1 -- 172.21.15.16:0/2948352006 >> v1:172.21.15.16:6789/0 conn(0x7f12801618d0 legacy=0x7f12800ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f127f7fe640 1 --2- 172.21.15.16:0/2948352006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f126c0039b0 0x7f126c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f127f7fe640 1 --2- 172.21.15.16:0/2948352006 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12800cef40 0x7f126c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f127f7fe640 1 --2- 172.21.15.16:0/2948352006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c004d40 0x7f126c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f127f7fe640 1 -- 172.21.15.16:0/2948352006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12800d1ea0 con 0x7f126c004d40 2024-04-25T10:14:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f127f7fe640 1 -- 172.21.15.16:0/2948352006 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1280162320 con 0x7f126c0039b0 2024-04-25T10:14:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f127f7fe640 1 -- 172.21.15.16:0/2948352006 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f126c00ba10 con 0x7f12800cef40 2024-04-25T10:14:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.448+0000 7f128530d640 1 --2- 172.21.15.16:0/2948352006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c004d40 0x7f126c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.449+0000 7f1284b0c640 1 --2- 172.21.15.16:0/2948352006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f126c0039b0 0x7f126c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.449+0000 7f127ffff640 1 --2- 172.21.15.16:0/2948352006 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12800cef40 0x7f126c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.449+0000 7f128530d640 1 -- 172.21.15.16:0/2948352006 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12800cef40 msgr2=0x7f126c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.449+0000 7f128530d640 1 --2- 172.21.15.16:0/2948352006 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12800cef40 0x7f126c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.449+0000 7f128530d640 1 -- 172.21.15.16:0/2948352006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f126c0039b0 msgr2=0x7f126c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.449+0000 7f128530d640 1 --2- 172.21.15.16:0/2948352006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f126c0039b0 0x7f126c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.449+0000 7f128530d640 1 -- 172.21.15.16:0/2948352006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126c003520 con 0x7f126c004d40 2024-04-25T10:14:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.449+0000 7f128530d640 1 --2- 172.21.15.16:0/2948352006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c004d40 0x7f126c0051c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f1278009f90 tx=0x7f126c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75956cad73cff39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.450+0000 7f127f7fe640 1 -- 172.21.15.16:0/2948352006 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278009c10 con 0x7f126c004d40 2024-04-25T10:14:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.450+0000 7f127f7fe640 1 -- 172.21.15.16:0/2948352006 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1278036070 con 0x7f126c004d40 2024-04-25T10:14:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.450+0000 7f127f7fe640 1 -- 172.21.15.16:0/2948352006 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278031850 con 0x7f126c004d40 2024-04-25T10:14:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.450+0000 7f1286d70640 1 -- 172.21.15.16:0/2948352006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c004d40 msgr2=0x7f126c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.450+0000 7f1286d70640 1 --2- 172.21.15.16:0/2948352006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c004d40 0x7f126c0051c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f1278009f90 tx=0x7f126c005830 comp rx=0 tx=0).stop 2024-04-25T10:14:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.450+0000 7f128530d640 1 -- 172.21.15.16:0/2948352006 reap_dead start 2024-04-25T10:14:35.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.450+0000 7f1286d70640 1 -- 172.21.15.16:0/2948352006 shutdown_connections 2024-04-25T10:14:35.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.450+0000 7f1286d70640 1 -- 172.21.15.16:0/2948352006 >> 172.21.15.16:0/2948352006 conn(0x7f12800b5570 msgr2=0x7f12800b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:35.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.450+0000 7f1286d70640 1 -- 172.21.15.16:0/2948352006 shutdown_connections 2024-04-25T10:14:35.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.450+0000 7f1286d70640 1 -- 172.21.15.16:0/2948352006 wait complete. 2024-04-25T10:14:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.451+0000 7f1286d70640 1 Processor -- start 2024-04-25T10:14:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.451+0000 7f1286d70640 1 -- start start 2024-04-25T10:14:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.452+0000 7f1286d70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c004d40 0x7f1280155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.452+0000 7f1286d70640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1280158420 0x7f128015f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.452+0000 7f1286d70640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12801f3bc0 0x7f12801f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.452+0000 7f1286d70640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1280161d10 con 0x7f126c004d40 2024-04-25T10:14:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.452+0000 7f1286d70640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12800b6c00 con 0x7f1280158420 2024-04-25T10:14:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.452+0000 7f1286d70640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1280161490 con 0x7f12801f3bc0 2024-04-25T10:14:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.452+0000 7f1284b0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c004d40 0x7f1280155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.452+0000 7f1284b0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c004d40 0x7f1280155b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37898/0 (socket says 172.21.15.16:37898) 2024-04-25T10:14:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.452+0000 7f1284b0c640 1 -- 172.21.15.16:0/892453949 learned_addr learned my addr 172.21.15.16:0/892453949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.453+0000 7f127ffff640 1 --2- 172.21.15.16:0/892453949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1280158420 0x7f128015f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.453+0000 7f128530d640 1 --2- 172.21.15.16:0/892453949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12801f3bc0 0x7f12801f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.453+0000 7f1284b0c640 1 -- 172.21.15.16:0/892453949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12801f3bc0 msgr2=0x7f12801f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.453+0000 7f1284b0c640 1 --2- 172.21.15.16:0/892453949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12801f3bc0 0x7f12801f4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.453+0000 7f1284b0c640 1 -- 172.21.15.16:0/892453949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1280158420 msgr2=0x7f128015f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.453+0000 7f1284b0c640 1 --2- 172.21.15.16:0/892453949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1280158420 0x7f128015f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.453+0000 7f1284b0c640 1 -- 172.21.15.16:0/892453949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1280156160 con 0x7f126c004d40 2024-04-25T10:14:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.453+0000 7f127ffff640 1 --2- 172.21.15.16:0/892453949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1280158420 0x7f128015f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.453+0000 7f1284b0c640 1 --2- 172.21.15.16:0/892453949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c004d40 0x7f1280155b80 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f1270011a70 tx=0x7f1270011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.453+0000 7f127d7fa640 1 -- 172.21.15.16:0/892453949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127000fb50 con 0x7f126c004d40 2024-04-25T10:14:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.453+0000 7f127d7fa640 1 -- 172.21.15.16:0/892453949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127000fcf0 con 0x7f126c004d40 2024-04-25T10:14:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.453+0000 7f1286d70640 1 -- 172.21.15.16:0/892453949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1280156440 con 0x7f126c004d40 2024-04-25T10:14:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.453+0000 7f127d7fa640 1 -- 172.21.15.16:0/892453949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127000fea0 con 0x7f126c004d40 2024-04-25T10:14:35.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.453+0000 7f1286d70640 1 -- 172.21.15.16:0/892453949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1280156a00 con 0x7f126c004d40 2024-04-25T10:14:35.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.454+0000 7f1286d70640 1 -- 172.21.15.16:0/892453949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1280156e40 con 0x7f126c004d40 2024-04-25T10:14:35.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.455+0000 7f127d7fa640 1 -- 172.21.15.16:0/892453949 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1270012040 con 0x7f126c004d40 2024-04-25T10:14:35.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.455+0000 7f127d7fa640 1 --2- 172.21.15.16:0/892453949 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f126006b4d0 0x7f126006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.455+0000 7f127d7fa640 1 -- 172.21.15.16:0/892453949 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f127008a850 con 0x7f126c004d40 2024-04-25T10:14:35.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.455+0000 7f127ffff640 1 --2- 172.21.15.16:0/892453949 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f126006b4d0 0x7f126006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.456+0000 7f127ffff640 1 --2- 172.21.15.16:0/892453949 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f126006b4d0 0x7f126006d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f128015fd60 tx=0x7f1268009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.459+0000 7f127d7fa640 1 -- 172.21.15.16:0/892453949 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1270019030 con 0x7f126c004d40 2024-04-25T10:14:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.653+0000 7f1286d70640 1 -- 172.21.15.16:0/892453949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f12800750b0 con 0x7f126c004d40 2024-04-25T10:14:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.653+0000 7f127d7fa640 1 -- 172.21.15.16:0/892453949 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v115) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1270059110 con 0x7f126c004d40 2024-04-25T10:14:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-04-25T10:14:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.655+0000 7f1286d70640 1 -- 172.21.15.16:0/892453949 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f126006b4d0 msgr2=0x7f126006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.655+0000 7f1286d70640 1 --2- 172.21.15.16:0/892453949 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f126006b4d0 0x7f126006d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f128015fd60 tx=0x7f1268009040 comp rx=0 tx=0).stop 2024-04-25T10:14:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.655+0000 7f1286d70640 1 -- 172.21.15.16:0/892453949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c004d40 msgr2=0x7f1280155b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.655+0000 7f1286d70640 1 --2- 172.21.15.16:0/892453949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c004d40 0x7f1280155b80 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f1270011a70 tx=0x7f1270011f30 comp rx=0 tx=0).stop 2024-04-25T10:14:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.655+0000 7f1286d70640 1 -- 172.21.15.16:0/892453949 shutdown_connections 2024-04-25T10:14:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.655+0000 7f1286d70640 1 --2- 172.21.15.16:0/892453949 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f126006b4d0 0x7f126006d990 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.655+0000 7f1286d70640 1 --2- 172.21.15.16:0/892453949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12801f3bc0 0x7f12801f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.655+0000 7f1286d70640 1 --2- 172.21.15.16:0/892453949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1280158420 0x7f128015f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.655+0000 7f1286d70640 1 --2- 172.21.15.16:0/892453949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f126c004d40 0x7f1280155b80 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.655+0000 7f1286d70640 1 -- 172.21.15.16:0/892453949 >> 172.21.15.16:0/892453949 conn(0x7f12800b5570 msgr2=0x7f12800b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.655+0000 7f1286d70640 1 -- 172.21.15.16:0/892453949 shutdown_connections 2024-04-25T10:14:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.655+0000 7f1286d70640 1 -- 172.21.15.16:0/892453949 wait complete. 2024-04-25T10:14:35.670 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:14:35.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:35.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:35.716 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:14:35.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:35.716 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:14:35.744 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:14:35.744 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-25T10:14:35.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.882+0000 7f85a4b3b640 1 Processor -- start 2024-04-25T10:14:35.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.882+0000 7f85a4b3b640 1 -- start start 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.883+0000 7f85a4b3b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f85a015deb0 0x7f85a00b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.883+0000 7f85a4b3b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85a00b1d30 0x7f85a006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.883+0000 7f85a4b3b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a00cd150 con 0x7f85a006b0e0 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.883+0000 7f85a4b3b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a006b530 con 0x7f85a015deb0 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.883+0000 7f85a4b3b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00b21d0 con 0x7f85a00b1d30 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859ed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f85a006b0e0 0x7f85a0161b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45192/0 (socket says 172.21.15.16:45192) 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859ed9d640 1 -- 172.21.15.16:0/2610156076 learned_addr learned my addr 172.21.15.16:0/2610156076 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859e59c640 1 --2- 172.21.15.16:0/2610156076 >> v2:172.21.15.186:3300/0 conn(0x7f85a015deb0 0x7f85a00b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859dd9b640 1 --2- 172.21.15.16:0/2610156076 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85a00b1d30 0x7f85a006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859d59a640 1 -- 172.21.15.16:0/2610156076 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3368378303 0 0) 0x7f85a00cd150 con 0x7f85a006b0e0 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859d59a640 1 -- 172.21.15.16:0/2610156076 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f858c003520 con 0x7f85a006b0e0 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859d59a640 1 -- 172.21.15.16:0/2610156076 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 206396686 0 0) 0x7f858c003520 con 0x7f85a006b0e0 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859d59a640 1 -- 172.21.15.16:0/2610156076 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85a00b1d30 msgr2=0x7f85a006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859d59a640 1 --2- 172.21.15.16:0/2610156076 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85a00b1d30 0x7f85a006aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859d59a640 1 -- 172.21.15.16:0/2610156076 >> v2:172.21.15.186:3300/0 conn(0x7f85a015deb0 msgr2=0x7f85a00b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859d59a640 1 --2- 172.21.15.16:0/2610156076 >> v2:172.21.15.186:3300/0 conn(0x7f85a015deb0 0x7f85a00b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859d59a640 1 -- 172.21.15.16:0/2610156076 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a00cd430 con 0x7f85a006b0e0 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859d59a640 1 -- 172.21.15.16:0/2610156076 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f8598001670 con 0x7f85a006b0e0 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859e59c640 1 --2- 172.21.15.16:0/2610156076 >> v2:172.21.15.186:3300/0 conn(0x7f85a015deb0 0x7f85a00b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:35.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859dd9b640 1 --2- 172.21.15.16:0/2610156076 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85a00b1d30 0x7f85a006aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:35.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859d59a640 1 -- 172.21.15.16:0/2610156076 >> v1:172.21.15.16:6789/0 conn(0x7f85a006b0e0 legacy=0x7f85a0161b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859d59a640 1 --2- 172.21.15.16:0/2610156076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f858c0039b0 0x7f858c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859d59a640 1 --2- 172.21.15.16:0/2610156076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f858c004800 0x7f858c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859d59a640 1 --2- 172.21.15.16:0/2610156076 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85a00b1d30 0x7f858c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859d59a640 1 -- 172.21.15.16:0/2610156076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a006b530 con 0x7f858c0039b0 2024-04-25T10:14:35.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859d59a640 1 -- 172.21.15.16:0/2610156076 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00b21d0 con 0x7f858c004800 2024-04-25T10:14:35.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859d59a640 1 -- 172.21.15.16:0/2610156076 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858c00b9c0 con 0x7f85a00b1d30 2024-04-25T10:14:35.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859e59c640 1 --2- 172.21.15.16:0/2610156076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f858c0039b0 0x7f858c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.884+0000 7f859dd9b640 1 --2- 172.21.15.16:0/2610156076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f858c004800 0x7f858c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.885+0000 7f859ed9d640 1 --2- 172.21.15.16:0/2610156076 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85a00b1d30 0x7f858c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.885+0000 7f859e59c640 1 -- 172.21.15.16:0/2610156076 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85a00b1d30 msgr2=0x7f858c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.885+0000 7f859e59c640 1 --2- 172.21.15.16:0/2610156076 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85a00b1d30 0x7f858c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.885+0000 7f859e59c640 1 -- 172.21.15.16:0/2610156076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f858c004800 msgr2=0x7f858c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.885+0000 7f859e59c640 1 --2- 172.21.15.16:0/2610156076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f858c004800 0x7f858c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.885+0000 7f859e59c640 1 -- 172.21.15.16:0/2610156076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c003520 con 0x7f858c0039b0 2024-04-25T10:14:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.885+0000 7f859e59c640 1 --2- 172.21.15.16:0/2610156076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f858c0039b0 0x7f858c003dd0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f859000e350 tx=0x7f858c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c167a8edf73e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.885+0000 7f859d59a640 1 -- 172.21.15.16:0/2610156076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590009de0 con 0x7f858c0039b0 2024-04-25T10:14:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.885+0000 7f859d59a640 1 -- 172.21.15.16:0/2610156076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859002e660 con 0x7f858c0039b0 2024-04-25T10:14:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.885+0000 7f859d59a640 1 -- 172.21.15.16:0/2610156076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859002ea10 con 0x7f858c0039b0 2024-04-25T10:14:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.886+0000 7f85a4b3b640 1 -- 172.21.15.16:0/2610156076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f858c0039b0 msgr2=0x7f858c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.886+0000 7f85a4b3b640 1 --2- 172.21.15.16:0/2610156076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f858c0039b0 0x7f858c003dd0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f859000e350 tx=0x7f858c005830 comp rx=0 tx=0).stop 2024-04-25T10:14:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.886+0000 7f859ed9d640 1 -- 172.21.15.16:0/2610156076 reap_dead start 2024-04-25T10:14:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.886+0000 7f85a4b3b640 1 -- 172.21.15.16:0/2610156076 shutdown_connections 2024-04-25T10:14:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.886+0000 7f85a4b3b640 1 -- 172.21.15.16:0/2610156076 >> 172.21.15.16:0/2610156076 conn(0x7f85a00b2390 msgr2=0x7f85a00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.886+0000 7f85a4b3b640 1 -- 172.21.15.16:0/2610156076 shutdown_connections 2024-04-25T10:14:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.886+0000 7f85a4b3b640 1 -- 172.21.15.16:0/2610156076 wait complete. 2024-04-25T10:14:35.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.887+0000 7f85a4b3b640 1 Processor -- start 2024-04-25T10:14:35.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.888+0000 7f85a4b3b640 1 -- start start 2024-04-25T10:14:35.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.888+0000 7f85a4b3b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f858c0039b0 0x7f85a0159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.888+0000 7f85a4b3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85a00b39d0 0x7f85a01f6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.888+0000 7f85a4b3b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85a01f71f0 0x7f85a00c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.888+0000 7f85a4b3b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a015e290 con 0x7f85a00b39d0 2024-04-25T10:14:35.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.888+0000 7f85a4b3b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00cd9f0 con 0x7f85a01f71f0 2024-04-25T10:14:35.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.888+0000 7f85a4b3b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a015e410 con 0x7f858c0039b0 2024-04-25T10:14:35.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.888+0000 7f859dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85a00b39d0 0x7f85a01f6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.888+0000 7f859dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85a00b39d0 0x7f85a01f6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37912/0 (socket says 172.21.15.16:37912) 2024-04-25T10:14:35.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.888+0000 7f859dd9b640 1 -- 172.21.15.16:0/3116479395 learned_addr learned my addr 172.21.15.16:0/3116479395 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:35.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.888+0000 7f859ed9d640 1 --2- 172.21.15.16:0/3116479395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85a01f71f0 0x7f85a00c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.888+0000 7f859e59c640 1 --2- 172.21.15.16:0/3116479395 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f858c0039b0 0x7f85a0159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.889+0000 7f859dd9b640 1 -- 172.21.15.16:0/3116479395 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f858c0039b0 msgr2=0x7f85a0159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.889+0000 7f859dd9b640 1 --2- 172.21.15.16:0/3116479395 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f858c0039b0 0x7f85a0159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.889+0000 7f859dd9b640 1 -- 172.21.15.16:0/3116479395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85a01f71f0 msgr2=0x7f85a00c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.889+0000 7f859dd9b640 1 --2- 172.21.15.16:0/3116479395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85a01f71f0 0x7f85a00c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.889+0000 7f859dd9b640 1 -- 172.21.15.16:0/3116479395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a00c7a40 con 0x7f85a00b39d0 2024-04-25T10:14:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.889+0000 7f859dd9b640 1 --2- 172.21.15.16:0/3116479395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85a00b39d0 0x7f85a01f6cb0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f85880119d0 tx=0x7f8588011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.889+0000 7f8596ffd640 1 -- 172.21.15.16:0/3116479395 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858800fa90 con 0x7f85a00b39d0 2024-04-25T10:14:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.889+0000 7f8596ffd640 1 -- 172.21.15.16:0/3116479395 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858800fc30 con 0x7f85a00b39d0 2024-04-25T10:14:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.889+0000 7f8596ffd640 1 -- 172.21.15.16:0/3116479395 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588004030 con 0x7f85a00b39d0 2024-04-25T10:14:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.889+0000 7f85a4b3b640 1 -- 172.21.15.16:0/3116479395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a00c7d20 con 0x7f85a00b39d0 2024-04-25T10:14:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.889+0000 7f85a4b3b640 1 -- 172.21.15.16:0/3116479395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a00c82e0 con 0x7f85a00b39d0 2024-04-25T10:14:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.891+0000 7f8596ffd640 1 -- 172.21.15.16:0/3116479395 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8588012040 con 0x7f85a00b39d0 2024-04-25T10:14:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.891+0000 7f8596ffd640 1 --2- 172.21.15.16:0/3116479395 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f858006b4d0 0x7f858006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.891+0000 7f8596ffd640 1 -- 172.21.15.16:0/3116479395 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f858808a3f0 con 0x7f85a00b39d0 2024-04-25T10:14:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.891+0000 7f859e59c640 1 --2- 172.21.15.16:0/3116479395 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f858006b4d0 0x7f858006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.892+0000 7f859e59c640 1 --2- 172.21.15.16:0/3116479395 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f858006b4d0 0x7f858006d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8590035000 tx=0x7f859002f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.892+0000 7f85a7840740 1 Processor -- start 2024-04-25T10:14:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.892+0000 7f85a7840740 1 -- start start 2024-04-25T10:14:35.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.893+0000 7f85a7840740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 0x55f397cb8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.893+0000 7f85a7840740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cb5ef0 0x55f397cb8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.893+0000 7f85a7840740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397c88770 0x55f397c8bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.893+0000 7f85a7840740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f397bf1da0 con 0x55f397cb5b10 2024-04-25T10:14:35.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.893+0000 7f85a7840740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f397b9ab80 con 0x55f397c88770 2024-04-25T10:14:35.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.893+0000 7f85a7840740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f397bede40 con 0x55f397cb5ef0 2024-04-25T10:14:35.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.893+0000 7f859e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 0x55f397cb8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.893+0000 7f859e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 0x55f397cb8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37914/0 (socket says 172.21.15.16:37914) 2024-04-25T10:14:35.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.893+0000 7f859e59c640 1 -- 172.21.15.16:0/364506558 learned_addr learned my addr 172.21.15.16:0/364506558 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:35.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.893+0000 7f859ed9d640 1 --2- 172.21.15.16:0/364506558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397c88770 0x55f397c8bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.893+0000 7f859dd9b640 1 --2- 172.21.15.16:0/364506558 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cb5ef0 0x55f397cb8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.893+0000 7f859e59c640 1 -- 172.21.15.16:0/364506558 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cb5ef0 msgr2=0x55f397cb8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.893+0000 7f859e59c640 1 --2- 172.21.15.16:0/364506558 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cb5ef0 0x55f397cb8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.893+0000 7f859e59c640 1 -- 172.21.15.16:0/364506558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397c88770 msgr2=0x55f397c8bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.893+0000 7f859e59c640 1 --2- 172.21.15.16:0/364506558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397c88770 0x55f397c8bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.893+0000 7f859e59c640 1 -- 172.21.15.16:0/364506558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f397ba4f10 con 0x55f397cb5b10 2024-04-25T10:14:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.893+0000 7f859e59c640 1 --2- 172.21.15.16:0/364506558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 0x55f397cb8320 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f859002b0e0 tx=0x7f859002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5f3d20f9d050ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.894+0000 7f857ffff640 1 -- 172.21.15.16:0/364506558 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590054070 con 0x55f397cb5b10 2024-04-25T10:14:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.894+0000 7f857ffff640 1 -- 172.21.15.16:0/364506558 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85900449c0 con 0x55f397cb5b10 2024-04-25T10:14:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.894+0000 7f857ffff640 1 -- 172.21.15.16:0/364506558 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590044d70 con 0x55f397cb5b10 2024-04-25T10:14:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.894+0000 7f85a7840740 1 -- 172.21.15.16:0/364506558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 msgr2=0x55f397cb8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.894+0000 7f85a7840740 1 --2- 172.21.15.16:0/364506558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 0x55f397cb8320 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f859002b0e0 tx=0x7f859002d000 comp rx=0 tx=0).stop 2024-04-25T10:14:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.894+0000 7f85a7840740 1 -- 172.21.15.16:0/364506558 shutdown_connections 2024-04-25T10:14:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.894+0000 7f85a7840740 1 --2- 172.21.15.16:0/364506558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397c88770 0x55f397c8bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.894+0000 7f85a7840740 1 --2- 172.21.15.16:0/364506558 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cb5ef0 0x55f397cb8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.894+0000 7f85a7840740 1 --2- 172.21.15.16:0/364506558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 0x55f397cb8320 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.894+0000 7f85a7840740 1 -- 172.21.15.16:0/364506558 >> 172.21.15.16:0/364506558 conn(0x55f397c8a9e0 msgr2=0x55f397cd3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.894+0000 7f85a7840740 1 -- 172.21.15.16:0/364506558 shutdown_connections 2024-04-25T10:14:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.894+0000 7f85a7840740 1 -- 172.21.15.16:0/364506558 wait complete. 2024-04-25T10:14:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.895+0000 7f85a7840740 1 -- start start 2024-04-25T10:14:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.895+0000 7f85a7840740 1 Processor -- start 2024-04-25T10:14:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.896+0000 7f85a7840740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 0x55f397cd4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.896+0000 7f85a7840740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397cb5ef0 0x55f397cb7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.896+0000 7f85a7840740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cd7450 0x55f397cac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.896+0000 7f85a7840740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f397c8db90 con 0x55f397cb5b10 2024-04-25T10:14:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.896+0000 7f85a7840740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f397b904b0 con 0x55f397cb5ef0 2024-04-25T10:14:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.896+0000 7f85a7840740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f397c8de70 con 0x55f397cd7450 2024-04-25T10:14:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.896+0000 7f85a7840740 10 client.0 _ll_register_callbacks cb 0x55f397bd0180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.896+0000 7f859e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 0x55f397cd4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.896+0000 7f859e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 0x55f397cd4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37924/0 (socket says 172.21.15.16:37924) 2024-04-25T10:14:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.896+0000 7f859e59c640 1 -- 172.21.15.16:0/3529903365 learned_addr learned my addr 172.21.15.16:0/3529903365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.896+0000 7f859dd9b640 1 --2- 172.21.15.16:0/3529903365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397cb5ef0 0x55f397cb7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.896+0000 7f859ed9d640 1 --2- 172.21.15.16:0/3529903365 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cd7450 0x55f397cac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.896+0000 7f859e59c640 1 -- 172.21.15.16:0/3529903365 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cd7450 msgr2=0x55f397cac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.896+0000 7f859e59c640 1 --2- 172.21.15.16:0/3529903365 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cd7450 0x55f397cac7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.896+0000 7f859e59c640 1 -- 172.21.15.16:0/3529903365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397cb5ef0 msgr2=0x55f397cb7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.896+0000 7f859e59c640 1 --2- 172.21.15.16:0/3529903365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397cb5ef0 0x55f397cb7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.896+0000 7f859e59c640 1 -- 172.21.15.16:0/3529903365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3979d07d0 con 0x55f397cb5b10 2024-04-25T10:14:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.897+0000 7f859e59c640 1 --2- 172.21.15.16:0/3529903365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 0x55f397cd4f60 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f859000e320 tx=0x7f8590004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b2d54836d7ce5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.897+0000 7f857d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:14:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.897+0000 7f857d7fa640 1 -- 172.21.15.16:0/3529903365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590054040 con 0x55f397cb5b10 2024-04-25T10:14:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.897+0000 7f857d7fa640 1 -- 172.21.15.16:0/3529903365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8590044d70 con 0x55f397cb5b10 2024-04-25T10:14:35.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.897+0000 7f85a7840740 1 -- 172.21.15.16:0/3529903365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85900449c0 con 0x55f397cb5b10 2024-04-25T10:14:35.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.897+0000 7f857d7fa640 1 -- 172.21.15.16:0/3529903365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8590066050 con 0x55f397cb5b10 2024-04-25T10:14:35.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.897+0000 7f85a7840740 10 client.9039 fetch_fsmap learned FSMap version 84 2024-04-25T10:14:35.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.897+0000 7f85a7840740 1 -- 172.21.15.16:0/3529903365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) v3 -- 0x55f3979063f0 con 0x55f397cb5b10 2024-04-25T10:14:35.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.897+0000 7f857d7fa640 1 -- 172.21.15.16:0/3529903365 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85900449c0 con 0x55f397cb5b10 2024-04-25T10:14:35.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.897+0000 7f857d7fa640 1 -- 172.21.15.16:0/3529903365 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 84) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8590069020 con 0x55f397cb5b10 2024-04-25T10:14:35.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.897+0000 7f85a7840740 10 client.9039 fetch_fsmap finished waiting for FSMap version 84 2024-04-25T10:14:35.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.897+0000 7f85a7840740 10 client.9039 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:14:35.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.897+0000 7f85a7840740 20 client.9039 populate_metadata read hostname 'smithi016' 2024-04-25T10:14:35.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.897+0000 7f85a7840740 1 --2- 172.21.15.16:0/3529903365 >> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] conn(0x55f397ce3280 0x55f397ce3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.897+0000 7f85a7840740 4 client.9039 mds_command: new command op to 8850 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:35.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.897+0000 7f85a7840740 1 -- 172.21.15.16:0/3529903365 --> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f397bff2e0 con 0x55f397ce3280 2024-04-25T10:14:35.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.897+0000 7f859dd9b640 1 --2- 172.21.15.16:0/3529903365 >> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] conn(0x55f397ce3280 0x55f397ce3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.898+0000 7f857d7fa640 10 client.9039 ms_handle_connect on v2:172.21.15.16:6833/123396308 2024-04-25T10:14:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.898+0000 7f859dd9b640 1 --2- 172.21.15.16:0/3529903365 >> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] conn(0x55f397ce3280 0x55f397ce3660 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dfeb9cb24b54828 server_cookie=1daf5d2d515f97c1 in_seq=0 out_seq=0 2024-04-25T10:14:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.898+0000 7f857d7fa640 1 -- 172.21.15.16:0/3529903365 <== mds.0 v2:172.21.15.16:6833/123396308 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f397bff2e0 con 0x55f397ce3280 2024-04-25T10:14:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.898+0000 7f857d7fa640 10 client.9039 handle_command_reply: tid=0 2024-04-25T10:14:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.898+0000 7f85a7840740 1 client.9039 shutdown 2024-04-25T10:14:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.898+0000 7f85a7840740 1 -- 172.21.15.16:0/3529903365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 msgr2=0x55f397cd4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.898+0000 7f85a7840740 1 --2- 172.21.15.16:0/3529903365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 0x55f397cd4f60 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f859000e320 tx=0x7f8590004ab0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.898+0000 7f85a7840740 1 -- 172.21.15.16:0/3529903365 shutdown_connections 2024-04-25T10:14:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.898+0000 7f85a7840740 1 --2- 172.21.15.16:0/3529903365 >> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] conn(0x55f397ce3280 0x55f397ce3660 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.898+0000 7f85a7840740 1 --2- 172.21.15.16:0/3529903365 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cd7450 0x55f397cac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.898+0000 7f85a7840740 1 --2- 172.21.15.16:0/3529903365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397cb5ef0 0x55f397cb7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.898+0000 7f857d7fa640 0 client.9039 ms_handle_reset on v2:172.21.15.16:6833/123396308 2024-04-25T10:14:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.898+0000 7f85a7840740 1 --2- 172.21.15.16:0/3529903365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 0x55f397cd4f60 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.898+0000 7f85a7840740 1 -- 172.21.15.16:0/3529903365 >> 172.21.15.16:0/3529903365 conn(0x55f397c88770 msgr2=0x55f397c8d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:35.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.899+0000 7f85a7840740 1 -- 172.21.15.16:0/3529903365 shutdown_connections 2024-04-25T10:14:35.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.899+0000 7f85a7840740 1 -- 172.21.15.16:0/3529903365 wait complete. 2024-04-25T10:14:35.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.899+0000 7f85a7840740 20 client.9039 trim_cache size 0 max 16384 2024-04-25T10:14:35.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.912+0000 7f85a7840740 1 Processor -- start 2024-04-25T10:14:35.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.912+0000 7f85a7840740 1 -- start start 2024-04-25T10:14:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f85a7840740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397cb5b10 0x55f397cb3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f85a7840740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5ef0 0x55f397c88e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f85a7840740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cd7450 0x55f397c89350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f85a7840740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a02007e0 con 0x55f397cb5ef0 2024-04-25T10:14:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f85a7840740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a01ff960 con 0x55f397cb5b10 2024-04-25T10:14:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f85a7840740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00c5c00 con 0x55f397cd7450 2024-04-25T10:14:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f859dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5ef0 0x55f397c88e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f859dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5ef0 0x55f397c88e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37936/0 (socket says 172.21.15.16:37936) 2024-04-25T10:14:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f859dd9b640 1 -- 172.21.15.16:0/346477179 learned_addr learned my addr 172.21.15.16:0/346477179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f859e59c640 1 --2- 172.21.15.16:0/346477179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397cb5b10 0x55f397cb3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f859ed9d640 1 --2- 172.21.15.16:0/346477179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cd7450 0x55f397c89350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f859dd9b640 1 -- 172.21.15.16:0/346477179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cd7450 msgr2=0x55f397c89350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f859dd9b640 1 --2- 172.21.15.16:0/346477179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cd7450 0x55f397c89350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f859dd9b640 1 -- 172.21.15.16:0/346477179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397cb5b10 msgr2=0x55f397cb3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f859dd9b640 1 --2- 172.21.15.16:0/346477179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397cb5b10 0x55f397cb3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f859dd9b640 1 -- 172.21.15.16:0/346477179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3978c60f0 con 0x55f397cb5ef0 2024-04-25T10:14:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f859e59c640 1 --2- 172.21.15.16:0/346477179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397cb5b10 0x55f397cb3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f859dd9b640 1 --2- 172.21.15.16:0/346477179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5ef0 0x55f397c88e10 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f858801b000 tx=0x7f8588055080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e130e9251f77df0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f857f7fe640 1 -- 172.21.15.16:0/346477179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588022040 con 0x55f397cb5ef0 2024-04-25T10:14:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f857f7fe640 1 -- 172.21.15.16:0/346477179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8588070070 con 0x55f397cb5ef0 2024-04-25T10:14:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.913+0000 7f857f7fe640 1 -- 172.21.15.16:0/346477179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588055320 con 0x55f397cb5ef0 2024-04-25T10:14:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.914+0000 7f85a7840740 1 -- 172.21.15.16:0/346477179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5ef0 msgr2=0x55f397c88e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.914+0000 7f85a7840740 1 --2- 172.21.15.16:0/346477179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5ef0 0x55f397c88e10 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f858801b000 tx=0x7f8588055080 comp rx=0 tx=0).stop 2024-04-25T10:14:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.914+0000 7f85a7840740 1 -- 172.21.15.16:0/346477179 shutdown_connections 2024-04-25T10:14:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.914+0000 7f85a7840740 1 --2- 172.21.15.16:0/346477179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cd7450 0x55f397c89350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.914+0000 7f85a7840740 1 --2- 172.21.15.16:0/346477179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5ef0 0x55f397c88e10 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.914+0000 7f85a7840740 1 --2- 172.21.15.16:0/346477179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397cb5b10 0x55f397cb3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.914+0000 7f85a7840740 1 -- 172.21.15.16:0/346477179 >> 172.21.15.16:0/346477179 conn(0x55f397c88770 msgr2=0x55f397c8c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.914+0000 7f85a7840740 1 -- 172.21.15.16:0/346477179 shutdown_connections 2024-04-25T10:14:35.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.914+0000 7f85a7840740 1 -- 172.21.15.16:0/346477179 wait complete. 2024-04-25T10:14:35.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.914+0000 7f85a7840740 1 -- start start 2024-04-25T10:14:35.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.915+0000 7f85a7840740 1 Processor -- start 2024-04-25T10:14:35.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.915+0000 7f85a7840740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 0x55f397cd3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.915+0000 7f85a7840740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397cb5ef0 0x55f397cb95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.915+0000 7f85a7840740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cd7450 0x55f397cb9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.915+0000 7f85a7840740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f397cd59b0 con 0x55f397cb5b10 2024-04-25T10:14:35.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.915+0000 7f85a7840740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a01ff360 con 0x55f397cb5ef0 2024-04-25T10:14:35.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.915+0000 7f85a7840740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f397cb7590 con 0x55f397cd7450 2024-04-25T10:14:35.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.915+0000 7f85a7840740 10 client.0 _ll_register_callbacks cb 0x7f8588009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.915+0000 7f859e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 0x55f397cd3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f859e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 0x55f397cd3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37940/0 (socket says 172.21.15.16:37940) 2024-04-25T10:14:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f859e59c640 1 -- 172.21.15.16:0/3042248124 learned_addr learned my addr 172.21.15.16:0/3042248124 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f859dd9b640 1 --2- 172.21.15.16:0/3042248124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397cb5ef0 0x55f397cb95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f859ed9d640 1 --2- 172.21.15.16:0/3042248124 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cd7450 0x55f397cb9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f859e59c640 1 -- 172.21.15.16:0/3042248124 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cd7450 msgr2=0x55f397cb9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f859e59c640 1 --2- 172.21.15.16:0/3042248124 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cd7450 0x55f397cb9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f859e59c640 1 -- 172.21.15.16:0/3042248124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397cb5ef0 msgr2=0x55f397cb95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f859e59c640 1 --2- 172.21.15.16:0/3042248124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397cb5ef0 0x55f397cb95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f859e59c640 1 -- 172.21.15.16:0/3042248124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f397c84040 con 0x55f397cb5b10 2024-04-25T10:14:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f859ed9d640 1 --2- 172.21.15.16:0/3042248124 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cd7450 0x55f397cb9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f859e59c640 1 --2- 172.21.15.16:0/3042248124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 0x55f397cd3d70 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f8590044a70 tx=0x7f8590030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734bf0fd3046dea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f857d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:14:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f857d7fa640 1 -- 172.21.15.16:0/3042248124 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859006f060 con 0x55f397cb5b10 2024-04-25T10:14:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f857d7fa640 1 -- 172.21.15.16:0/3042248124 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8590046620 con 0x55f397cb5b10 2024-04-25T10:14:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f85a7840740 1 -- 172.21.15.16:0/3042248124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8588070070 con 0x55f397cb5b10 2024-04-25T10:14:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f857d7fa640 1 -- 172.21.15.16:0/3042248124 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8590075050 con 0x55f397cb5b10 2024-04-25T10:14:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f85a7840740 10 client.9045 fetch_fsmap learned FSMap version 84 2024-04-25T10:14:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f85a7840740 1 -- 172.21.15.16:0/3042248124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) v3 -- 0x55f397cd3570 con 0x55f397cb5b10 2024-04-25T10:14:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.916+0000 7f857d7fa640 1 -- 172.21.15.16:0/3042248124 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8588070070 con 0x55f397cb5b10 2024-04-25T10:14:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.917+0000 7f857d7fa640 1 -- 172.21.15.16:0/3042248124 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 84) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8590078020 con 0x55f397cb5b10 2024-04-25T10:14:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.917+0000 7f85a7840740 10 client.9045 fetch_fsmap finished waiting for FSMap version 84 2024-04-25T10:14:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.917+0000 7f85a7840740 10 client.9045 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:14:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.917+0000 7f85a7840740 20 client.9045 populate_metadata read hostname 'smithi016' 2024-04-25T10:14:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.917+0000 7f85a7840740 1 --2- 172.21.15.16:0/3042248124 >> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] conn(0x55f397d403b0 0x55f397cd4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.917+0000 7f85a7840740 4 client.9045 mds_command: new command op to 8850 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:14:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.917+0000 7f85a7840740 1 -- 172.21.15.16:0/3042248124 --> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f397bf9630 con 0x55f397d403b0 2024-04-25T10:14:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.917+0000 7f859dd9b640 1 --2- 172.21.15.16:0/3042248124 >> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] conn(0x55f397d403b0 0x55f397cd4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.917+0000 7f857d7fa640 10 client.9045 ms_handle_connect on v2:172.21.15.16:6833/123396308 2024-04-25T10:14:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.917+0000 7f859dd9b640 1 --2- 172.21.15.16:0/3042248124 >> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] conn(0x55f397d403b0 0x55f397cd4810 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7d37cf63c16ba72 server_cookie=cc47e48656bc4469 in_seq=0 out_seq=0 2024-04-25T10:14:35.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.926+0000 7f857d7fa640 1 -- 172.21.15.16:0/3042248124 <== mds.0 v2:172.21.15.16:6833/123396308 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f397bf9630 con 0x55f397d403b0 2024-04-25T10:14:35.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.926+0000 7f857d7fa640 10 client.9045 handle_command_reply: tid=0 2024-04-25T10:14:35.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.926+0000 7f85a7840740 1 client.9045 shutdown 2024-04-25T10:14:35.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.926+0000 7f85a7840740 1 -- 172.21.15.16:0/3042248124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 msgr2=0x55f397cd3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.926+0000 7f85a7840740 1 --2- 172.21.15.16:0/3042248124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 0x55f397cd3d70 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f8590044a70 tx=0x7f8590030000 comp rx=0 tx=0).stop 2024-04-25T10:14:35.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.926+0000 7f85a7840740 1 -- 172.21.15.16:0/3042248124 shutdown_connections 2024-04-25T10:14:35.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.926+0000 7f85a7840740 1 --2- 172.21.15.16:0/3042248124 >> [v2:172.21.15.16:6833/123396308,v1:172.21.15.16:6835/123396308] conn(0x55f397d403b0 0x55f397cd4810 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.926+0000 7f85a7840740 1 --2- 172.21.15.16:0/3042248124 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f397cd7450 0x55f397cb9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.926+0000 7f85a7840740 1 --2- 172.21.15.16:0/3042248124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f397cb5ef0 0x55f397cb95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.926+0000 7f85a7840740 1 --2- 172.21.15.16:0/3042248124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f397cb5b10 0x55f397cd3d70 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.926+0000 7f857d7fa640 0 client.9045 ms_handle_reset on v2:172.21.15.16:6833/123396308 2024-04-25T10:14:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.926+0000 7f85a7840740 1 -- 172.21.15.16:0/3042248124 >> 172.21.15.16:0/3042248124 conn(0x55f397c88770 msgr2=0x55f397cb7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.926+0000 7f85a7840740 1 -- 172.21.15.16:0/3042248124 shutdown_connections 2024-04-25T10:14:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.926+0000 7f85a7840740 1 -- 172.21.15.16:0/3042248124 wait complete. 2024-04-25T10:14:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.926+0000 7f85a7840740 20 client.9045 trim_cache size 0 max 16384 2024-04-25T10:14:35.932 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:14:35.932 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "", 2024-04-25T10:14:35.932 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0 2024-04-25T10:14:35.932 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:14:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.927+0000 7f85a4b3b640 1 -- 172.21.15.16:0/3116479395 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f858006b4d0 msgr2=0x7f858006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.927+0000 7f85a4b3b640 1 --2- 172.21.15.16:0/3116479395 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f858006b4d0 0x7f858006d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8590035000 tx=0x7f859002f000 comp rx=0 tx=0).stop 2024-04-25T10:14:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.927+0000 7f85a4b3b640 1 -- 172.21.15.16:0/3116479395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85a00b39d0 msgr2=0x7f85a01f6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.927+0000 7f85a4b3b640 1 --2- 172.21.15.16:0/3116479395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85a00b39d0 0x7f85a01f6cb0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f85880119d0 tx=0x7f8588011dd0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.927+0000 7f85a4b3b640 1 -- 172.21.15.16:0/3116479395 shutdown_connections 2024-04-25T10:14:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.927+0000 7f85a4b3b640 1 --2- 172.21.15.16:0/3116479395 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f858006b4d0 0x7f858006d990 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.927+0000 7f85a4b3b640 1 --2- 172.21.15.16:0/3116479395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85a01f71f0 0x7f85a00c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.927+0000 7f85a4b3b640 1 --2- 172.21.15.16:0/3116479395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85a00b39d0 0x7f85a01f6cb0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.927+0000 7f85a4b3b640 1 --2- 172.21.15.16:0/3116479395 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f858c0039b0 0x7f85a0159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.927+0000 7f85a4b3b640 1 -- 172.21.15.16:0/3116479395 >> 172.21.15.16:0/3116479395 conn(0x7f85a00b2390 msgr2=0x7f85a00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.927+0000 7f85a4b3b640 1 -- 172.21.15.16:0/3116479395 shutdown_connections 2024-04-25T10:14:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:35.927+0000 7f85a4b3b640 1 -- 172.21.15.16:0/3116479395 wait complete. 2024-04-25T10:14:35.941 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:14:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.038+0000 7f4c3b6bc640 1 Processor -- start 2024-04-25T10:14:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.038+0000 7f4c3b6bc640 1 -- start start 2024-04-25T10:14:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.039+0000 7f4c3b6bc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c3415dce0 0x7f4c340d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.039+0000 7f4c3b6bc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c340d35b0 0x7f4c340b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.039+0000 7f4c3b6bc640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3415e0c0 con 0x7f4c340d1f90 2024-04-25T10:14:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.039+0000 7f4c3b6bc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c340b7b10 con 0x7f4c3415dce0 2024-04-25T10:14:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.039+0000 7f4c3b6bc640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c340d2510 con 0x7f4c340d35b0 2024-04-25T10:14:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.039+0000 7f4c39458640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c3415dce0 0x7f4c340d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.039+0000 7f4c39c59640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4c340d1f90 0x7f4c3406bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45198/0 (socket says 172.21.15.16:45198) 2024-04-25T10:14:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.039+0000 7f4c39458640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c3415dce0 0x7f4c340d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37950/0 (socket says 172.21.15.16:37950) 2024-04-25T10:14:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.039+0000 7f4c39c59640 1 -- 172.21.15.16:0/3131083989 learned_addr learned my addr 172.21.15.16:0/3131083989 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.039+0000 7f4c38c57640 1 --2- 172.21.15.16:0/3131083989 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c340d35b0 0x7f4c340b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.039+0000 7f4c39458640 1 -- 172.21.15.16:0/3131083989 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c340d35b0 msgr2=0x7f4c340b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.039+0000 7f4c39458640 1 --2- 172.21.15.16:0/3131083989 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c340d35b0 0x7f4c340b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.039+0000 7f4c39458640 1 -- 172.21.15.16:0/3131083989 >> v1:172.21.15.16:6789/0 conn(0x7f4c340d1f90 legacy=0x7f4c3406bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.039+0000 7f4c2bfff640 1 -- 172.21.15.16:0/3131083989 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989087858 0 0) 0x7f4c3415e0c0 con 0x7f4c340d1f90 2024-04-25T10:14:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.039+0000 7f4c39458640 1 -- 172.21.15.16:0/3131083989 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c340d27f0 con 0x7f4c3415dce0 2024-04-25T10:14:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.040+0000 7f4c39458640 1 --2- 172.21.15.16:0/3131083989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c3415dce0 0x7f4c340d1a50 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f4c300097a0 tx=0x7f4c3002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebfa10e52d25e41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.040+0000 7f4c2bfff640 1 -- 172.21.15.16:0/3131083989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30037070 con 0x7f4c3415dce0 2024-04-25T10:14:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.040+0000 7f4c2bfff640 1 -- 172.21.15.16:0/3131083989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3002ae50 con 0x7f4c3415dce0 2024-04-25T10:14:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.040+0000 7f4c2bfff640 1 -- 172.21.15.16:0/3131083989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30032660 con 0x7f4c3415dce0 2024-04-25T10:14:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.041+0000 7f4c3b6bc640 1 -- 172.21.15.16:0/3131083989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c3415dce0 msgr2=0x7f4c340d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.041+0000 7f4c3b6bc640 1 --2- 172.21.15.16:0/3131083989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c3415dce0 0x7f4c340d1a50 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f4c300097a0 tx=0x7f4c3002a100 comp rx=0 tx=0).stop 2024-04-25T10:14:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.041+0000 7f4c3b6bc640 1 -- 172.21.15.16:0/3131083989 shutdown_connections 2024-04-25T10:14:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.041+0000 7f4c3b6bc640 1 --2- 172.21.15.16:0/3131083989 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c340d35b0 0x7f4c340b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.041+0000 7f4c3b6bc640 1 --2- 172.21.15.16:0/3131083989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c3415dce0 0x7f4c340d1a50 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.041+0000 7f4c3b6bc640 1 -- 172.21.15.16:0/3131083989 >> 172.21.15.16:0/3131083989 conn(0x7f4c34072900 msgr2=0x7f4c340d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.041+0000 7f4c3b6bc640 1 -- 172.21.15.16:0/3131083989 shutdown_connections 2024-04-25T10:14:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.041+0000 7f4c3b6bc640 1 -- 172.21.15.16:0/3131083989 wait complete. 2024-04-25T10:14:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.042+0000 7f4c3b6bc640 1 Processor -- start 2024-04-25T10:14:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.042+0000 7f4c3b6bc640 1 -- start start 2024-04-25T10:14:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.043+0000 7f4c3b6bc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c340d1f90 0x7f4c34160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.043+0000 7f4c3b6bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c340d35b0 0x7f4c340cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.043+0000 7f4c3b6bc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c3415dce0 0x7f4c3415e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.043+0000 7f4c3b6bc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c340d3090 con 0x7f4c340d35b0 2024-04-25T10:14:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.043+0000 7f4c3b6bc640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c340d2db0 con 0x7f4c3415dce0 2024-04-25T10:14:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.043+0000 7f4c3b6bc640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c340b9820 con 0x7f4c340d1f90 2024-04-25T10:14:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.043+0000 7f4c38c57640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c340d35b0 0x7f4c340cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.043+0000 7f4c38c57640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c340d35b0 0x7f4c340cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37964/0 (socket says 172.21.15.16:37964) 2024-04-25T10:14:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.043+0000 7f4c38c57640 1 -- 172.21.15.16:0/131509559 learned_addr learned my addr 172.21.15.16:0/131509559 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.043+0000 7f4c39458640 1 --2- 172.21.15.16:0/131509559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c340d1f90 0x7f4c34160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.043+0000 7f4c39c59640 1 --2- 172.21.15.16:0/131509559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c3415dce0 0x7f4c3415e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.043+0000 7f4c38c57640 1 -- 172.21.15.16:0/131509559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c340d1f90 msgr2=0x7f4c34160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.043+0000 7f4c38c57640 1 --2- 172.21.15.16:0/131509559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c340d1f90 0x7f4c34160dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.043+0000 7f4c38c57640 1 -- 172.21.15.16:0/131509559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c3415dce0 msgr2=0x7f4c3415e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.043+0000 7f4c38c57640 1 --2- 172.21.15.16:0/131509559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c3415dce0 0x7f4c3415e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.043+0000 7f4c38c57640 1 -- 172.21.15.16:0/131509559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c340cc540 con 0x7f4c340d35b0 2024-04-25T10:14:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.043+0000 7f4c38c57640 1 --2- 172.21.15.16:0/131509559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c340d35b0 0x7f4c340cbaf0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400d800 tx=0x7f4c2400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.043+0000 7f4c29ffb640 1 -- 172.21.15.16:0/131509559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400b6c0 con 0x7f4c340d35b0 2024-04-25T10:14:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.044+0000 7f4c29ffb640 1 -- 172.21.15.16:0/131509559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c24019070 con 0x7f4c340d35b0 2024-04-25T10:14:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.044+0000 7f4c3b6bc640 1 -- 172.21.15.16:0/131509559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c340cc820 con 0x7f4c340d35b0 2024-04-25T10:14:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.044+0000 7f4c29ffb640 1 -- 172.21.15.16:0/131509559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400ba80 con 0x7f4c340d35b0 2024-04-25T10:14:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.044+0000 7f4c3b6bc640 1 -- 172.21.15.16:0/131509559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c340ccde0 con 0x7f4c340d35b0 2024-04-25T10:14:36.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.045+0000 7f4c3b6bc640 1 -- 172.21.15.16:0/131509559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c340749b0 con 0x7f4c340d35b0 2024-04-25T10:14:36.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.045+0000 7f4c29ffb640 1 -- 172.21.15.16:0/131509559 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c2400bc20 con 0x7f4c340d35b0 2024-04-25T10:14:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.045+0000 7f4c29ffb640 1 --2- 172.21.15.16:0/131509559 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4c1806b4d0 0x7f4c1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.045+0000 7f4c29ffb640 1 -- 172.21.15.16:0/131509559 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c2408a650 con 0x7f4c340d35b0 2024-04-25T10:14:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.046+0000 7f4c39458640 1 --2- 172.21.15.16:0/131509559 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4c1806b4d0 0x7f4c1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.046+0000 7f4c39458640 1 --2- 172.21.15.16:0/131509559 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4c1806b4d0 0x7f4c1806d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4c30034000 tx=0x7f4c3002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:36.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.050+0000 7f4c29ffb640 1 -- 172.21.15.16:0/131509559 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c24059560 con 0x7f4c340d35b0 2024-04-25T10:14:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.269+0000 7f4c3b6bc640 1 -- 172.21.15.16:0/131509559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4c340d42c0 con 0x7f4c340d35b0 2024-04-25T10:14:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.269+0000 7f4c29ffb640 1 -- 172.21.15.16:0/131509559 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4c24058e60 con 0x7f4c340d35b0 2024-04-25T10:14:36.273 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:14:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.271+0000 7f4c3b6bc640 1 -- 172.21.15.16:0/131509559 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4c1806b4d0 msgr2=0x7f4c1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.271+0000 7f4c3b6bc640 1 --2- 172.21.15.16:0/131509559 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4c1806b4d0 0x7f4c1806d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4c30034000 tx=0x7f4c3002e000 comp rx=0 tx=0).stop 2024-04-25T10:14:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.271+0000 7f4c3b6bc640 1 -- 172.21.15.16:0/131509559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c340d35b0 msgr2=0x7f4c340cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.271+0000 7f4c3b6bc640 1 --2- 172.21.15.16:0/131509559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c340d35b0 0x7f4c340cbaf0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400d800 tx=0x7f4c2400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.271+0000 7f4c3b6bc640 1 -- 172.21.15.16:0/131509559 shutdown_connections 2024-04-25T10:14:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.271+0000 7f4c3b6bc640 1 --2- 172.21.15.16:0/131509559 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4c1806b4d0 0x7f4c1806d990 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.271+0000 7f4c3b6bc640 1 --2- 172.21.15.16:0/131509559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c3415dce0 0x7f4c3415e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.271+0000 7f4c3b6bc640 1 --2- 172.21.15.16:0/131509559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c340d35b0 0x7f4c340cbaf0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.271+0000 7f4c3b6bc640 1 --2- 172.21.15.16:0/131509559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c340d1f90 0x7f4c34160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.271+0000 7f4c3b6bc640 1 -- 172.21.15.16:0/131509559 >> 172.21.15.16:0/131509559 conn(0x7f4c34072900 msgr2=0x7f4c3415d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.271+0000 7f4c3b6bc640 1 -- 172.21.15.16:0/131509559 shutdown_connections 2024-04-25T10:14:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:36.271+0000 7f4c3b6bc640 1 -- 172.21.15.16:0/131509559 wait complete. 2024-04-25T10:14:36.285 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:14:36.285 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:14:36.285 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-04-25T10:14:36.286 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:14:36.286 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:36.286 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:14:36.286 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:14:36.286 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:14:36.286 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:36.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:36.286 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-04-25T10:14:36.302 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:14:36.302 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:14:36.302 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:14:36.302 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:36.302 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:14:36.302 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:36.302 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:14:36.302 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:36.303 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-04-25T10:14:36.303 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:14:36.303 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:36.303 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-04-25T10:14:36.303 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:14:36.303 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:36.303 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-04-25T10:14:36.303 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:14:36.303 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42084sec preferred_lft 42084sec 2024-04-25T10:14:36.303 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-25T10:14:36.303 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:36.303 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:14:36.303 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:36.304 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-04-25T10:14:36.304 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:14:36.304 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:36.304 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:14:36.304 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:36.304 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f06b:fdff:fed4:a335/64 scope link 2024-04-25T10:14:36.304 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:36.304 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:36.304 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:36.304 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::90ad:4fff:fe91:3c33/64 scope link 2024-04-25T10:14:36.304 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:36.304 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:36.304 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether be:14:4a:3c:16:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:36.304 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::bc14:4aff:fe3c:1605/64 scope link 2024-04-25T10:14:36.305 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:36.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:36.305 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:14:36.366 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:14:36.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:36.367 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-04-25T10:14:36.423 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-04-25T10:14:36.424 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:14:36.424 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:14:36.424 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:14:36.424 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:14:36.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:36.424 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:14:36.424 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:14:36.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:14:36.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:14:36.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:14:36.425 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:14:36.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:36.510 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:14:36.525 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:14:36.525 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:14:36.525 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:36.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:36.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-04-25T10:14:36.590 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:36.591 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:36.646 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:36.647 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:36.647 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:36.701 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:14:36.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:36.702 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:36.784 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:36.784 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:14:36.784 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:14:36.792 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:36.792 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:14:36.793 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:14:36.793 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:14:36.793 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:14:36.797 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:36.798 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:14:36.798 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:14:36.798 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:14:36.798 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:14:36.798 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:14:36.798 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:14:36.798 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:14:36.799 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:14:36.799 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:14:36.799 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:14:36.799 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:14:36.799 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:14:36.800 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:14:36.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:36.800 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:36.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:36.866 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:14:36.922 INFO:teuthology.orchestra.run.smithi016.stdout:client7852 2024-04-25T10:14:36.923 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:14:36.923 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:36.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:36.978 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-04-25T10:14:37.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:37.037 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-04-25T10:14:37.118 INFO:teuthology.orchestra.run.smithi016.stderr:8+0 records in 2024-04-25T10:14:37.118 INFO:teuthology.orchestra.run.smithi016.stderr:8+0 records out 2024-04-25T10:14:37.118 INFO:teuthology.orchestra.run.smithi016.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0233816 s, 359 MB/s 2024-04-25T10:14:37.119 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:37.119 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:37.119 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:37.119 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:37.119 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T10:14:37.119 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:37.224 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627781 2024-04-25T10:14:37.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:37.233 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:37.233 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:37.234 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:37.234 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:37.234 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T10:14:37.234 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:37.340 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627782 2024-04-25T10:14:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:37.350 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:14:37.350 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:37.405 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:14:37.405 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:37.462 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.599+0000 7fe8d32bd640 1 Processor -- start 2024-04-25T10:14:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.599+0000 7fe8d32bd640 1 -- start start 2024-04-25T10:14:37.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8d32bd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc071190 0x7fe8cc0ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:37.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8d32bd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8cc15e910 con 0x7fe8cc071570 2024-04-25T10:14:37.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8d32bd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8cc06d550 con 0x7fe8cc0b3730 2024-04-25T10:14:37.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8d32bd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc0aeee0 con 0x7fe8cc071190 2024-04-25T10:14:37.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8d0858640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe8cc071570 0x7fe8cc15daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45200/0 (socket says 172.21.15.16:45200) 2024-04-25T10:14:37.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8d0858640 1 -- 172.21.15.16:0/1135547789 learned_addr learned my addr 172.21.15.16:0/1135547789 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:37.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8d1059640 1 --2- 172.21.15.16:0/1135547789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc071190 0x7fe8cc0ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:37.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8c3fff640 1 -- 172.21.15.16:0/1135547789 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986467977 0 0) 0x7fe8cc15e910 con 0x7fe8cc071570 2024-04-25T10:14:37.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8c3fff640 1 -- 172.21.15.16:0/1135547789 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8b8004630 con 0x7fe8cc071570 2024-04-25T10:14:37.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8c3fff640 1 -- 172.21.15.16:0/1135547789 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 799760857 0 0) 0x7fe8b8004630 con 0x7fe8cc071570 2024-04-25T10:14:37.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8c3fff640 1 -- 172.21.15.16:0/1135547789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc071190 msgr2=0x7fe8cc0ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:37.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8c3fff640 1 --2- 172.21.15.16:0/1135547789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc071190 0x7fe8cc0ae9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8c3fff640 1 -- 172.21.15.16:0/1135547789 >> v1:172.21.15.186:6789/0 conn(0x7fe8cc0b3730 legacy=0x7fe8cc15e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:37.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8c3fff640 1 -- 172.21.15.16:0/1135547789 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc06d830 con 0x7fe8cc071570 2024-04-25T10:14:37.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8c3fff640 1 -- 172.21.15.16:0/1135547789 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fe8bc001670 con 0x7fe8cc071570 2024-04-25T10:14:37.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8c3fff640 1 -- 172.21.15.16:0/1135547789 >> v1:172.21.15.16:6789/0 conn(0x7fe8cc071570 legacy=0x7fe8cc15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:37.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8c3fff640 1 --2- 172.21.15.16:0/1135547789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8b8004af0 0x7fe8b8004f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:37.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8c3fff640 1 --2- 172.21.15.16:0/1135547789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc071190 0x7fe8b8005910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:37.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8c3fff640 1 --2- 172.21.15.16:0/1135547789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8b8005e50 0x7fe8b80062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:37.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8c3fff640 1 -- 172.21.15.16:0/1135547789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc0aeee0 con 0x7fe8b8004af0 2024-04-25T10:14:37.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8c3fff640 1 -- 172.21.15.16:0/1135547789 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b800cb80 con 0x7fe8b8005e50 2024-04-25T10:14:37.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.600+0000 7fe8c3fff640 1 -- 172.21.15.16:0/1135547789 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b80034c0 con 0x7fe8cc071190 2024-04-25T10:14:37.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.601+0000 7fe8d1059640 1 --2- 172.21.15.16:0/1135547789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8b8004af0 0x7fe8b8004f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:37.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.601+0000 7fe8d0858640 1 --2- 172.21.15.16:0/1135547789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc071190 0x7fe8b8005910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:37.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.601+0000 7fe8d185a640 1 --2- 172.21.15.16:0/1135547789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8b8005e50 0x7fe8b80062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:37.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.601+0000 7fe8d1059640 1 -- 172.21.15.16:0/1135547789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc071190 msgr2=0x7fe8b8005910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:37.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.601+0000 7fe8d1059640 1 --2- 172.21.15.16:0/1135547789 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8cc071190 0x7fe8b8005910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.601+0000 7fe8d1059640 1 -- 172.21.15.16:0/1135547789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8b8005e50 msgr2=0x7fe8b80062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:37.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.601+0000 7fe8d1059640 1 --2- 172.21.15.16:0/1135547789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8b8005e50 0x7fe8b80062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.601+0000 7fe8d1059640 1 -- 172.21.15.16:0/1135547789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b8004630 con 0x7fe8b8004af0 2024-04-25T10:14:37.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.602+0000 7fe8d1059640 1 --2- 172.21.15.16:0/1135547789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8b8004af0 0x7fe8b8004f10 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009f90 tx=0x7fe8b8006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e0225b1c5f35c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:37.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.602+0000 7fe8c3fff640 1 -- 172.21.15.16:0/1135547789 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8009c10 con 0x7fe8b8004af0 2024-04-25T10:14:37.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.602+0000 7fe8c3fff640 1 -- 172.21.15.16:0/1135547789 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c8036070 con 0x7fe8b8004af0 2024-04-25T10:14:37.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.602+0000 7fe8c3fff640 1 -- 172.21.15.16:0/1135547789 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8031850 con 0x7fe8b8004af0 2024-04-25T10:14:37.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.602+0000 7fe8d32bd640 1 -- 172.21.15.16:0/1135547789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8b8004af0 msgr2=0x7fe8b8004f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:37.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.602+0000 7fe8d32bd640 1 --2- 172.21.15.16:0/1135547789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8b8004af0 0x7fe8b8004f10 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009f90 tx=0x7fe8b8006940 comp rx=0 tx=0).stop 2024-04-25T10:14:37.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.602+0000 7fe8d185a640 1 -- 172.21.15.16:0/1135547789 reap_dead start 2024-04-25T10:14:37.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.602+0000 7fe8d32bd640 1 -- 172.21.15.16:0/1135547789 shutdown_connections 2024-04-25T10:14:37.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.602+0000 7fe8d32bd640 1 -- 172.21.15.16:0/1135547789 >> 172.21.15.16:0/1135547789 conn(0x7fe8cc06aba0 msgr2=0x7fe8cc06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:37.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.602+0000 7fe8d32bd640 1 -- 172.21.15.16:0/1135547789 shutdown_connections 2024-04-25T10:14:37.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.602+0000 7fe8d32bd640 1 -- 172.21.15.16:0/1135547789 wait complete. 2024-04-25T10:14:37.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.603+0000 7fe8d32bd640 1 Processor -- start 2024-04-25T10:14:37.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.603+0000 7fe8d32bd640 1 -- start start 2024-04-25T10:14:37.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.604+0000 7fe8d32bd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8b8004af0 0x7fe8cc1f01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:37.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.604+0000 7fe8d32bd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc0ced70 0x7fe8cc0cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:37.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.604+0000 7fe8d32bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc154f70 0x7fe8cc1553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:37.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.604+0000 7fe8d32bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc06e0d0 con 0x7fe8cc154f70 2024-04-25T10:14:37.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.604+0000 7fe8d32bd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc06ddf0 con 0x7fe8cc0ced70 2024-04-25T10:14:37.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.604+0000 7fe8d32bd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc06e8b0 con 0x7fe8b8004af0 2024-04-25T10:14:37.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.604+0000 7fe8d185a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc154f70 0x7fe8cc1553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:37.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.604+0000 7fe8d185a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc154f70 0x7fe8cc1553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37984/0 (socket says 172.21.15.16:37984) 2024-04-25T10:14:37.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.604+0000 7fe8d185a640 1 -- 172.21.15.16:0/851967104 learned_addr learned my addr 172.21.15.16:0/851967104 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:37.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.604+0000 7fe8d1059640 1 --2- 172.21.15.16:0/851967104 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8b8004af0 0x7fe8cc1f01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:37.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.604+0000 7fe8d0858640 1 --2- 172.21.15.16:0/851967104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc0ced70 0x7fe8cc0cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:37.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.605+0000 7fe8d185a640 1 -- 172.21.15.16:0/851967104 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8b8004af0 msgr2=0x7fe8cc1f01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:37.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.605+0000 7fe8d185a640 1 --2- 172.21.15.16:0/851967104 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8b8004af0 0x7fe8cc1f01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.605+0000 7fe8d185a640 1 -- 172.21.15.16:0/851967104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc0ced70 msgr2=0x7fe8cc0cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:37.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.605+0000 7fe8d185a640 1 --2- 172.21.15.16:0/851967104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc0ced70 0x7fe8cc0cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.605+0000 7fe8d185a640 1 -- 172.21.15.16:0/851967104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc1f06f0 con 0x7fe8cc154f70 2024-04-25T10:14:37.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.605+0000 7fe8d185a640 1 --2- 172.21.15.16:0/851967104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc154f70 0x7fe8cc1553f0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0b3bc0 tx=0x7fe8c400f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:37.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.605+0000 7fe8c1ffb640 1 -- 172.21.15.16:0/851967104 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40053a0 con 0x7fe8cc154f70 2024-04-25T10:14:37.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.605+0000 7fe8c1ffb640 1 -- 172.21.15.16:0/851967104 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c4018070 con 0x7fe8cc154f70 2024-04-25T10:14:37.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.605+0000 7fe8d32bd640 1 -- 172.21.15.16:0/851967104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8cc155930 con 0x7fe8cc154f70 2024-04-25T10:14:37.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.605+0000 7fe8c1ffb640 1 -- 172.21.15.16:0/851967104 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4003830 con 0x7fe8cc154f70 2024-04-25T10:14:37.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.605+0000 7fe8d32bd640 1 -- 172.21.15.16:0/851967104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8cc155ed0 con 0x7fe8cc154f70 2024-04-25T10:14:37.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.606+0000 7fe8d32bd640 1 -- 172.21.15.16:0/851967104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8cc0749b0 con 0x7fe8cc154f70 2024-04-25T10:14:37.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.607+0000 7fe8c1ffb640 1 -- 172.21.15.16:0/851967104 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8c40039d0 con 0x7fe8cc154f70 2024-04-25T10:14:37.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.607+0000 7fe8c1ffb640 1 --2- 172.21.15.16:0/851967104 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe8b006b500 0x7fe8b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:37.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.607+0000 7fe8c1ffb640 1 -- 172.21.15.16:0/851967104 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8c4089b20 con 0x7fe8cc154f70 2024-04-25T10:14:37.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.607+0000 7fe8d1059640 1 --2- 172.21.15.16:0/851967104 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe8b006b500 0x7fe8b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:37.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.608+0000 7fe8d1059640 1 --2- 172.21.15.16:0/851967104 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe8b006b500 0x7fe8b006d9c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8035000 tx=0x7fe8c802f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:37.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.611+0000 7fe8c1ffb640 1 -- 172.21.15.16:0/851967104 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8c40583e0 con 0x7fe8cc154f70 2024-04-25T10:14:37.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.793+0000 7fe8d32bd640 1 -- 172.21.15.16:0/851967104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8cc06e470 con 0x7fe8cc154f70 2024-04-25T10:14:37.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.794+0000 7fe8c1ffb640 1 -- 172.21.15.16:0/851967104 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+16683 (secure 0 0 0) 0x7fe8c405c480 con 0x7fe8cc154f70 2024-04-25T10:14:37.798 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:37.798 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":115,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:14:28.393191+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:25.025480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:25.726747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.796+0000 7fe8d32bd640 1 -- 172.21.15.16:0/851967104 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe8b006b500 msgr2=0x7fe8b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.796+0000 7fe8d32bd640 1 --2- 172.21.15.16:0/851967104 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe8b006b500 0x7fe8b006d9c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8035000 tx=0x7fe8c802f000 comp rx=0 tx=0).stop 2024-04-25T10:14:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.796+0000 7fe8d32bd640 1 -- 172.21.15.16:0/851967104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc154f70 msgr2=0x7fe8cc1553f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.796+0000 7fe8d32bd640 1 --2- 172.21.15.16:0/851967104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc154f70 0x7fe8cc1553f0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0b3bc0 tx=0x7fe8c400f920 comp rx=0 tx=0).stop 2024-04-25T10:14:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.796+0000 7fe8d32bd640 1 -- 172.21.15.16:0/851967104 shutdown_connections 2024-04-25T10:14:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.796+0000 7fe8d32bd640 1 --2- 172.21.15.16:0/851967104 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe8b006b500 0x7fe8b006d9c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.796+0000 7fe8d32bd640 1 --2- 172.21.15.16:0/851967104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc154f70 0x7fe8cc1553f0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.796+0000 7fe8d32bd640 1 --2- 172.21.15.16:0/851967104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc0ced70 0x7fe8cc0cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.796+0000 7fe8d32bd640 1 --2- 172.21.15.16:0/851967104 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8b8004af0 0x7fe8cc1f01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.796+0000 7fe8d32bd640 1 -- 172.21.15.16:0/851967104 >> 172.21.15.16:0/851967104 conn(0x7fe8cc06aba0 msgr2=0x7fe8cc14e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.796+0000 7fe8d32bd640 1 -- 172.21.15.16:0/851967104 shutdown_connections 2024-04-25T10:14:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.796+0000 7fe8d32bd640 1 -- 172.21.15.16:0/851967104 wait complete. 2024-04-25T10:14:37.811 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:14:25.025480+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:14:25.726747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:37.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:37.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.911+0000 7f2cf0b14640 1 Processor -- start 2024-04-25T10:14:37.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.911+0000 7f2cf0b14640 1 -- start start 2024-04-25T10:14:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.912+0000 7f2cf0b14640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2cec15dca0 0x7f2cec0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.912+0000 7f2cf0b14640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cec0d3b90 0x7f2cec0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.912+0000 7f2cf0b14640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cec0d4090 con 0x7f2cec0cee80 2024-04-25T10:14:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.912+0000 7f2cf0b14640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cec0b6b20 con 0x7f2cec0d3b90 2024-04-25T10:14:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.912+0000 7f2cf0b14640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cec0d4370 con 0x7f2cec15dca0 2024-04-25T10:14:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.912+0000 7f2ce3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cec0d3b90 0x7f2cec0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.912+0000 7f2ce3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cec0d3b90 0x7f2cec0d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37994/0 (socket says 172.21.15.16:37994) 2024-04-25T10:14:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.912+0000 7f2ce3fff640 1 -- 172.21.15.16:0/1995604535 learned_addr learned my addr 172.21.15.16:0/1995604535 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.912+0000 7f2cea59c640 1 --2- 172.21.15.16:0/1995604535 >> v2:172.21.15.186:3300/0 conn(0x7f2cec15dca0 0x7f2cec0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.912+0000 7f2ce3fff640 1 -- 172.21.15.16:0/1995604535 >> v2:172.21.15.186:3300/0 conn(0x7f2cec15dca0 msgr2=0x7f2cec0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.912+0000 7f2ce3fff640 1 --2- 172.21.15.16:0/1995604535 >> v2:172.21.15.186:3300/0 conn(0x7f2cec15dca0 0x7f2cec0ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.912+0000 7f2ce3fff640 1 -- 172.21.15.16:0/1995604535 >> v1:172.21.15.186:6789/0 conn(0x7f2cec0cee80 legacy=0x7f2cec0b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.912+0000 7f2ce3fff640 1 -- 172.21.15.16:0/1995604535 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec0d4650 con 0x7f2cec0d3b90 2024-04-25T10:14:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.913+0000 7f2ce3fff640 1 --2- 172.21.15.16:0/1995604535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec0d3b90 0x7f2cec0d0a20 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f2cec0b6340 tx=0x7f2cd402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2eb8418dce39c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.913+0000 7f2ce9d9b640 1 -- 172.21.15.16:0/1995604535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4037070 con 0x7f2cec0d3b90 2024-04-25T10:14:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.913+0000 7f2ce9d9b640 1 -- 172.21.15.16:0/1995604535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd4032440 con 0x7f2cec0d3b90 2024-04-25T10:14:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.913+0000 7f2ce9d9b640 1 -- 172.21.15.16:0/1995604535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4032810 con 0x7f2cec0d3b90 2024-04-25T10:14:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.914+0000 7f2cf0b14640 1 -- 172.21.15.16:0/1995604535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec0d3b90 msgr2=0x7f2cec0d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.914+0000 7f2cf0b14640 1 --2- 172.21.15.16:0/1995604535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec0d3b90 0x7f2cec0d0a20 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f2cec0b6340 tx=0x7f2cd402a270 comp rx=0 tx=0).stop 2024-04-25T10:14:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.914+0000 7f2cf0b14640 1 -- 172.21.15.16:0/1995604535 shutdown_connections 2024-04-25T10:14:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.914+0000 7f2cf0b14640 1 --2- 172.21.15.16:0/1995604535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec0d3b90 0x7f2cec0d0a20 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.914+0000 7f2cf0b14640 1 --2- 172.21.15.16:0/1995604535 >> v2:172.21.15.186:3300/0 conn(0x7f2cec15dca0 0x7f2cec0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.914+0000 7f2cf0b14640 1 -- 172.21.15.16:0/1995604535 >> 172.21.15.16:0/1995604535 conn(0x7f2cec0726c0 msgr2=0x7f2cec072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.914+0000 7f2cf0b14640 1 -- 172.21.15.16:0/1995604535 shutdown_connections 2024-04-25T10:14:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.914+0000 7f2cf0b14640 1 -- 172.21.15.16:0/1995604535 wait complete. 2024-04-25T10:14:37.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.915+0000 7f2cf0b14640 1 Processor -- start 2024-04-25T10:14:37.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.915+0000 7f2cf0b14640 1 -- start start 2024-04-25T10:14:37.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.916+0000 7f2cf0b14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec0cee80 0x7f2cec15a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:37.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.916+0000 7f2cf0b14640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cec0d3b90 0x7f2cec1f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:37.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.916+0000 7f2cf0b14640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cec15dca0 0x7f2cec1f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:37.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.916+0000 7f2cf0b14640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec15e200 con 0x7f2cec0cee80 2024-04-25T10:14:37.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.916+0000 7f2cf0b14640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec15e080 con 0x7f2cec15dca0 2024-04-25T10:14:37.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.916+0000 7f2cf0b14640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec0d0f60 con 0x7f2cec0d3b90 2024-04-25T10:14:37.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.916+0000 7f2cea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec0cee80 0x7f2cec15a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:37.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.916+0000 7f2cea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec0cee80 0x7f2cec15a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38008/0 (socket says 172.21.15.16:38008) 2024-04-25T10:14:37.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.916+0000 7f2cea59c640 1 -- 172.21.15.16:0/2858055756 learned_addr learned my addr 172.21.15.16:0/2858055756 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:37.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.916+0000 7f2cea59c640 1 -- 172.21.15.16:0/2858055756 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cec0d3b90 msgr2=0x7f2cec1f70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:37.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.916+0000 7f2cea59c640 1 --2- 172.21.15.16:0/2858055756 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cec0d3b90 0x7f2cec1f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.916+0000 7f2cea59c640 1 -- 172.21.15.16:0/2858055756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cec15dca0 msgr2=0x7f2cec1f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:37.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.916+0000 7f2cead9d640 1 --2- 172.21.15.16:0/2858055756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cec15dca0 0x7f2cec1f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:37.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.916+0000 7f2cea59c640 1 --2- 172.21.15.16:0/2858055756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cec15dca0 0x7f2cec1f75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.916+0000 7f2cea59c640 1 -- 172.21.15.16:0/2858055756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec1591d0 con 0x7f2cec0cee80 2024-04-25T10:14:37.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.916+0000 7f2cead9d640 1 --2- 172.21.15.16:0/2858055756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cec15dca0 0x7f2cec1f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:37.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.916+0000 7f2cea59c640 1 --2- 172.21.15.16:0/2858055756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec0cee80 0x7f2cec15a510 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00d5f0 tx=0x7f2cdc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:37.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.917+0000 7f2ce2ffd640 1 -- 172.21.15.16:0/2858055756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc0125f0 con 0x7f2cec0cee80 2024-04-25T10:14:37.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.917+0000 7f2ce2ffd640 1 -- 172.21.15.16:0/2858055756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc019070 con 0x7f2cec0cee80 2024-04-25T10:14:37.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.917+0000 7f2cf0b14640 1 -- 172.21.15.16:0/2858055756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cec159470 con 0x7f2cec0cee80 2024-04-25T10:14:37.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.917+0000 7f2ce2ffd640 1 -- 172.21.15.16:0/2858055756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc0129d0 con 0x7f2cec0cee80 2024-04-25T10:14:37.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.917+0000 7f2cf0b14640 1 -- 172.21.15.16:0/2858055756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cec159a30 con 0x7f2cec0cee80 2024-04-25T10:14:37.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.918+0000 7f2cf0b14640 1 -- 172.21.15.16:0/2858055756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cec0750b0 con 0x7f2cec0cee80 2024-04-25T10:14:37.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.918+0000 7f2ce2ffd640 1 -- 172.21.15.16:0/2858055756 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2cdc012b70 con 0x7f2cec0cee80 2024-04-25T10:14:37.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.919+0000 7f2ce2ffd640 1 --2- 172.21.15.16:0/2858055756 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ccc06b4d0 0x7f2ccc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:37.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.919+0000 7f2ce2ffd640 1 -- 172.21.15.16:0/2858055756 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2cdc08a600 con 0x7f2cec0cee80 2024-04-25T10:14:37.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.919+0000 7f2ce3fff640 1 --2- 172.21.15.16:0/2858055756 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ccc06b4d0 0x7f2ccc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:37.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.919+0000 7f2ce3fff640 1 --2- 172.21.15.16:0/2858055756 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ccc06b4d0 0x7f2ccc06d990 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4034000 tx=0x7f2cd402c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:37.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:37.923+0000 7f2ce2ffd640 1 -- 172.21.15.16:0/2858055756 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2cdc059480 con 0x7f2cec0cee80 2024-04-25T10:14:38.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.101+0000 7f2cf0b14640 1 -- 172.21.15.16:0/2858055756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2cec0b84a0 con 0x7f2cec0cee80 2024-04-25T10:14:38.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.102+0000 7f2ce2ffd640 1 -- 172.21.15.16:0/2858055756 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2cdc058d80 con 0x7f2cec0cee80 2024-04-25T10:14:38.106 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:14:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.103+0000 7f2cf0b14640 1 -- 172.21.15.16:0/2858055756 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ccc06b4d0 msgr2=0x7f2ccc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.103+0000 7f2cf0b14640 1 --2- 172.21.15.16:0/2858055756 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ccc06b4d0 0x7f2ccc06d990 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4034000 tx=0x7f2cd402c000 comp rx=0 tx=0).stop 2024-04-25T10:14:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.104+0000 7f2cf0b14640 1 -- 172.21.15.16:0/2858055756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec0cee80 msgr2=0x7f2cec15a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.104+0000 7f2cf0b14640 1 --2- 172.21.15.16:0/2858055756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec0cee80 0x7f2cec15a510 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00d5f0 tx=0x7f2cdc00dab0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.104+0000 7f2cf0b14640 1 -- 172.21.15.16:0/2858055756 shutdown_connections 2024-04-25T10:14:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.104+0000 7f2cf0b14640 1 --2- 172.21.15.16:0/2858055756 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ccc06b4d0 0x7f2ccc06d990 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.104+0000 7f2cf0b14640 1 --2- 172.21.15.16:0/2858055756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cec15dca0 0x7f2cec1f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.104+0000 7f2cf0b14640 1 --2- 172.21.15.16:0/2858055756 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2cec0d3b90 0x7f2cec1f70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.104+0000 7f2cf0b14640 1 --2- 172.21.15.16:0/2858055756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec0cee80 0x7f2cec15a510 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.104+0000 7f2cf0b14640 1 -- 172.21.15.16:0/2858055756 >> 172.21.15.16:0/2858055756 conn(0x7f2cec0726c0 msgr2=0x7f2cec0b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:38.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.104+0000 7f2cf0b14640 1 -- 172.21.15.16:0/2858055756 shutdown_connections 2024-04-25T10:14:38.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.104+0000 7f2cf0b14640 1 -- 172.21.15.16:0/2858055756 wait complete. 2024-04-25T10:14:38.119 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.218+0000 7fa49821f640 1 Processor -- start 2024-04-25T10:14:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.218+0000 7fa49821f640 1 -- start start 2024-04-25T10:14:38.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.219+0000 7fa49821f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49015ec90 0x7fa4900ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.219+0000 7fa49821f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49006c150 con 0x7fa4900b2740 2024-04-25T10:14:38.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.219+0000 7fa49821f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49006c430 con 0x7fa4900d0490 2024-04-25T10:14:38.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.219+0000 7fa49821f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4900d2000 con 0x7fa49015ec90 2024-04-25T10:14:38.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.219+0000 7fa4957ba640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa4900b2740 0x7fa4900ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45206/0 (socket says 172.21.15.16:45206) 2024-04-25T10:14:38.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.219+0000 7fa4957ba640 1 -- 172.21.15.16:0/2480346133 learned_addr learned my addr 172.21.15.16:0/2480346133 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:38.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.219+0000 7fa4967bc640 1 --2- 172.21.15.16:0/2480346133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49015ec90 0x7fa4900ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.219+0000 7fa494fb9640 1 -- 172.21.15.16:0/2480346133 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321474952 0 0) 0x7fa49006c150 con 0x7fa4900b2740 2024-04-25T10:14:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa494fb9640 1 -- 172.21.15.16:0/2480346133 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa478003520 con 0x7fa4900b2740 2024-04-25T10:14:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa494fb9640 1 -- 172.21.15.16:0/2480346133 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 549532936 0 0) 0x7fa478003520 con 0x7fa4900b2740 2024-04-25T10:14:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa494fb9640 1 -- 172.21.15.16:0/2480346133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49015ec90 msgr2=0x7fa4900ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa494fb9640 1 --2- 172.21.15.16:0/2480346133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49015ec90 0x7fa4900ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa494fb9640 1 -- 172.21.15.16:0/2480346133 >> v1:172.21.15.186:6789/0 conn(0x7fa4900d0490 legacy=0x7fa4900d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa494fb9640 1 -- 172.21.15.16:0/2480346133 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49006c710 con 0x7fa4900b2740 2024-04-25T10:14:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa494fb9640 1 -- 172.21.15.16:0/2480346133 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fa484001670 con 0x7fa4900b2740 2024-04-25T10:14:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa494fb9640 1 -- 172.21.15.16:0/2480346133 >> v1:172.21.15.16:6789/0 conn(0x7fa4900b2740 legacy=0x7fa4900ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa494fb9640 1 --2- 172.21.15.16:0/2480346133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4780039e0 0x7fa478003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa494fb9640 1 --2- 172.21.15.16:0/2480346133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49015ec90 0x7fa478004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa494fb9640 1 --2- 172.21.15.16:0/2480346133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa478004d40 0x7fa4780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa494fb9640 1 -- 172.21.15.16:0/2480346133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47800baf0 con 0x7fa478004d40 2024-04-25T10:14:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa494fb9640 1 -- 172.21.15.16:0/2480346133 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47800bca0 con 0x7fa4780039e0 2024-04-25T10:14:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa494fb9640 1 -- 172.21.15.16:0/2480346133 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47800bea0 con 0x7fa49015ec90 2024-04-25T10:14:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa4967bc640 1 --2- 172.21.15.16:0/2480346133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa478004d40 0x7fa4780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa495fbb640 1 --2- 172.21.15.16:0/2480346133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4780039e0 0x7fa478003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa4957ba640 1 --2- 172.21.15.16:0/2480346133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49015ec90 0x7fa478004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa4967bc640 1 -- 172.21.15.16:0/2480346133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49015ec90 msgr2=0x7fa478004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa4967bc640 1 --2- 172.21.15.16:0/2480346133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49015ec90 0x7fa478004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa4967bc640 1 -- 172.21.15.16:0/2480346133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4780039e0 msgr2=0x7fa478003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa4967bc640 1 --2- 172.21.15.16:0/2480346133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4780039e0 0x7fa478003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa4967bc640 1 -- 172.21.15.16:0/2480346133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa478003520 con 0x7fa478004d40 2024-04-25T10:14:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.220+0000 7fa4957ba640 1 --2- 172.21.15.16:0/2480346133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49015ec90 0x7fa478004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.221+0000 7fa4967bc640 1 --2- 172.21.15.16:0/2480346133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa478004d40 0x7fa4780051c0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fa480009fc0 tx=0x7fa478005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10915dafacf635e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.221+0000 7fa494fb9640 1 -- 172.21.15.16:0/2480346133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480009c10 con 0x7fa478004d40 2024-04-25T10:14:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.221+0000 7fa494fb9640 1 -- 172.21.15.16:0/2480346133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa480036070 con 0x7fa478004d40 2024-04-25T10:14:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.221+0000 7fa494fb9640 1 -- 172.21.15.16:0/2480346133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480031880 con 0x7fa478004d40 2024-04-25T10:14:38.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.222+0000 7fa49821f640 1 -- 172.21.15.16:0/2480346133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa478004d40 msgr2=0x7fa4780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.222+0000 7fa49821f640 1 --2- 172.21.15.16:0/2480346133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa478004d40 0x7fa4780051c0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fa480009fc0 tx=0x7fa478005830 comp rx=0 tx=0).stop 2024-04-25T10:14:38.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.222+0000 7fa4967bc640 1 -- 172.21.15.16:0/2480346133 reap_dead start 2024-04-25T10:14:38.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.222+0000 7fa49821f640 1 -- 172.21.15.16:0/2480346133 shutdown_connections 2024-04-25T10:14:38.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.222+0000 7fa49821f640 1 -- 172.21.15.16:0/2480346133 >> 172.21.15.16:0/2480346133 conn(0x7fa4900b5660 msgr2=0x7fa4900b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:38.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.222+0000 7fa49821f640 1 -- 172.21.15.16:0/2480346133 shutdown_connections 2024-04-25T10:14:38.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.222+0000 7fa49821f640 1 -- 172.21.15.16:0/2480346133 wait complete. 2024-04-25T10:14:38.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.223+0000 7fa49821f640 1 Processor -- start 2024-04-25T10:14:38.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.223+0000 7fa49821f640 1 -- start start 2024-04-25T10:14:38.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa49821f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa49014b790 0x7fa490147bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa49821f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49014bb70 0x7fa4901480f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa49821f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4900cbda0 0x7fa4900cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa49821f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49015f120 con 0x7fa4900cbda0 2024-04-25T10:14:38.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa49821f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49015e020 con 0x7fa49014b790 2024-04-25T10:14:38.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa49821f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4900ce8b0 con 0x7fa49014bb70 2024-04-25T10:14:38.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa4967bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4900cbda0 0x7fa4900cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa4967bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4900cbda0 0x7fa4900cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38024/0 (socket says 172.21.15.16:38024) 2024-04-25T10:14:38.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa4967bc640 1 -- 172.21.15.16:0/4205306526 learned_addr learned my addr 172.21.15.16:0/4205306526 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:38.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa495fbb640 1 --2- 172.21.15.16:0/4205306526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa49014b790 0x7fa490147bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa4957ba640 1 --2- 172.21.15.16:0/4205306526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49014bb70 0x7fa4901480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa4967bc640 1 -- 172.21.15.16:0/4205306526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49014bb70 msgr2=0x7fa4901480f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa4967bc640 1 --2- 172.21.15.16:0/4205306526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49014bb70 0x7fa4901480f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa4967bc640 1 -- 172.21.15.16:0/4205306526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa49014b790 msgr2=0x7fa490147bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa4967bc640 1 --2- 172.21.15.16:0/4205306526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa49014b790 0x7fa490147bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa4967bc640 1 -- 172.21.15.16:0/4205306526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4900ccab0 con 0x7fa4900cbda0 2024-04-25T10:14:38.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa4957ba640 1 --2- 172.21.15.16:0/4205306526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49014bb70 0x7fa4901480f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:38.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa495fbb640 1 --2- 172.21.15.16:0/4205306526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa49014b790 0x7fa490147bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:38.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.224+0000 7fa4967bc640 1 --2- 172.21.15.16:0/4205306526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4900cbda0 0x7fa4900cc230 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fa48000e230 tx=0x7fa480009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:38.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.225+0000 7fa48e7fc640 1 -- 172.21.15.16:0/4205306526 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480009c10 con 0x7fa4900cbda0 2024-04-25T10:14:38.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.225+0000 7fa48e7fc640 1 -- 172.21.15.16:0/4205306526 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa480031df0 con 0x7fa4900cbda0 2024-04-25T10:14:38.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.225+0000 7fa49821f640 1 -- 172.21.15.16:0/4205306526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4900ccd90 con 0x7fa4900cbda0 2024-04-25T10:14:38.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.225+0000 7fa48e7fc640 1 -- 172.21.15.16:0/4205306526 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480003bd0 con 0x7fa4900cbda0 2024-04-25T10:14:38.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.225+0000 7fa49821f640 1 -- 172.21.15.16:0/4205306526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4900cd350 con 0x7fa4900cbda0 2024-04-25T10:14:38.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.226+0000 7fa49821f640 1 -- 172.21.15.16:0/4205306526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa490153e50 con 0x7fa4900cbda0 2024-04-25T10:14:38.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.226+0000 7fa48e7fc640 1 -- 172.21.15.16:0/4205306526 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa480005d10 con 0x7fa4900cbda0 2024-04-25T10:14:38.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.227+0000 7fa48e7fc640 1 --2- 172.21.15.16:0/4205306526 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa47c06b4d0 0x7fa47c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.227+0000 7fa48e7fc640 1 -- 172.21.15.16:0/4205306526 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4800ab520 con 0x7fa4900cbda0 2024-04-25T10:14:38.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.227+0000 7fa495fbb640 1 --2- 172.21.15.16:0/4205306526 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa47c06b4d0 0x7fa47c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.227+0000 7fa495fbb640 1 --2- 172.21.15.16:0/4205306526 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa47c06b4d0 0x7fa47c06d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa4880036b0 tx=0x7fa488009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:38.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.231+0000 7fa48e7fc640 1 -- 172.21.15.16:0/4205306526 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa490153e50 con 0x7fa4900cbda0 2024-04-25T10:14:38.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.410+0000 7fa49821f640 1 -- 172.21.15.16:0/4205306526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4900750b0 con 0x7fa4900cbda0 2024-04-25T10:14:38.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.410+0000 7fa48e7fc640 1 -- 172.21.15.16:0/4205306526 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa48004a060 con 0x7fa4900cbda0 2024-04-25T10:14:38.414 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:14:38.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.412+0000 7fa49821f640 1 -- 172.21.15.16:0/4205306526 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa47c06b4d0 msgr2=0x7fa47c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.412+0000 7fa49821f640 1 --2- 172.21.15.16:0/4205306526 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa47c06b4d0 0x7fa47c06d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa4880036b0 tx=0x7fa488009040 comp rx=0 tx=0).stop 2024-04-25T10:14:38.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.412+0000 7fa49821f640 1 -- 172.21.15.16:0/4205306526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4900cbda0 msgr2=0x7fa4900cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.412+0000 7fa49821f640 1 --2- 172.21.15.16:0/4205306526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4900cbda0 0x7fa4900cc230 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fa48000e230 tx=0x7fa480009750 comp rx=0 tx=0).stop 2024-04-25T10:14:38.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.412+0000 7fa49821f640 1 -- 172.21.15.16:0/4205306526 shutdown_connections 2024-04-25T10:14:38.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.412+0000 7fa49821f640 1 --2- 172.21.15.16:0/4205306526 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa47c06b4d0 0x7fa47c06d990 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.412+0000 7fa49821f640 1 --2- 172.21.15.16:0/4205306526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4900cbda0 0x7fa4900cc230 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.412+0000 7fa49821f640 1 --2- 172.21.15.16:0/4205306526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa49014bb70 0x7fa4901480f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.412+0000 7fa49821f640 1 --2- 172.21.15.16:0/4205306526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa49014b790 0x7fa490147bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.412+0000 7fa49821f640 1 -- 172.21.15.16:0/4205306526 >> 172.21.15.16:0/4205306526 conn(0x7fa4900b5660 msgr2=0x7fa4900ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:38.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.412+0000 7fa49821f640 1 -- 172.21.15.16:0/4205306526 shutdown_connections 2024-04-25T10:14:38.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.412+0000 7fa49821f640 1 -- 172.21.15.16:0/4205306526 wait complete. 2024-04-25T10:14:38.427 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:38.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.526+0000 7f8c0d238640 1 Processor -- start 2024-04-25T10:14:38.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.526+0000 7f8c0d238640 1 -- start start 2024-04-25T10:14:38.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.527+0000 7f8c0d238640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c0815dc80 0x7f8c08161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.527+0000 7f8c0d238640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c081627a0 con 0x7f8c080b97c0 2024-04-25T10:14:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.527+0000 7f8c0d238640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c081621e0 con 0x7f8c080b5c00 2024-04-25T10:14:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.527+0000 7f8c0d238640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c081624c0 con 0x7f8c0815dc80 2024-04-25T10:14:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.527+0000 7f8c0659c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8c080b97c0 0x7f8c080b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45210/0 (socket says 172.21.15.16:45210) 2024-04-25T10:14:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.527+0000 7f8c0659c640 1 -- 172.21.15.16:0/3319239113 learned_addr learned my addr 172.21.15.16:0/3319239113 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.528+0000 7f8c06d9d640 1 --2- 172.21.15.16:0/3319239113 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c0815dc80 0x7f8c08161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.528+0000 7f8c05d9b640 1 -- 172.21.15.16:0/3319239113 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1597414703 0 0) 0x7f8c081627a0 con 0x7f8c080b97c0 2024-04-25T10:14:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.528+0000 7f8c05d9b640 1 -- 172.21.15.16:0/3319239113 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bf4003520 con 0x7f8c080b97c0 2024-04-25T10:14:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.528+0000 7f8c05d9b640 1 -- 172.21.15.16:0/3319239113 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405166391 0 0) 0x7f8c081621e0 con 0x7f8c080b5c00 2024-04-25T10:14:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.528+0000 7f8c05d9b640 1 -- 172.21.15.16:0/3319239113 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c081627a0 con 0x7f8c080b5c00 2024-04-25T10:14:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.528+0000 7f8c06d9d640 1 -- 172.21.15.16:0/3319239113 >> v1:172.21.15.186:6789/0 conn(0x7f8c080b5c00 legacy=0x7f8c080d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.528+0000 7f8c06d9d640 1 -- 172.21.15.16:0/3319239113 >> v1:172.21.15.16:6789/0 conn(0x7f8c080b97c0 legacy=0x7f8c080b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.528+0000 7f8c05d9b640 1 -- 172.21.15.16:0/3319239113 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3079519985 0 0) 0x7f8bf4003520 con 0x7f8c080b97c0 2024-04-25T10:14:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.528+0000 7f8c06d9d640 1 -- 172.21.15.16:0/3319239113 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c080d1c70 con 0x7f8c0815dc80 2024-04-25T10:14:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.529+0000 7f8c06d9d640 1 --2- 172.21.15.16:0/3319239113 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c0815dc80 0x7f8c08161870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8c000097d0 tx=0x7f8c0002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=416102078a527e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.529+0000 7f8c05d9b640 1 -- 172.21.15.16:0/3319239113 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00037070 con 0x7f8c0815dc80 2024-04-25T10:14:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.529+0000 7f8c05d9b640 1 -- 172.21.15.16:0/3319239113 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0002ac30 con 0x7f8c0815dc80 2024-04-25T10:14:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.529+0000 7f8c05d9b640 1 -- 172.21.15.16:0/3319239113 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0001670 con 0x7f8c0815dc80 2024-04-25T10:14:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.529+0000 7f8c0d238640 1 -- 172.21.15.16:0/3319239113 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c0815dc80 msgr2=0x7f8c08161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.529+0000 7f8c0d238640 1 --2- 172.21.15.16:0/3319239113 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c0815dc80 0x7f8c08161870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8c000097d0 tx=0x7f8c0002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.529+0000 7f8c0d238640 1 -- 172.21.15.16:0/3319239113 shutdown_connections 2024-04-25T10:14:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.529+0000 7f8c0d238640 1 --2- 172.21.15.16:0/3319239113 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c0815dc80 0x7f8c08161870 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.529+0000 7f8c0d238640 1 -- 172.21.15.16:0/3319239113 >> 172.21.15.16:0/3319239113 conn(0x7f8c080d04f0 msgr2=0x7f8c080d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.529+0000 7f8c0d238640 1 -- 172.21.15.16:0/3319239113 shutdown_connections 2024-04-25T10:14:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.529+0000 7f8c0d238640 1 -- 172.21.15.16:0/3319239113 wait complete. 2024-04-25T10:14:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.531+0000 7f8c0d238640 1 Processor -- start 2024-04-25T10:14:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.531+0000 7f8c0d238640 1 -- start start 2024-04-25T10:14:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c0d238640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c080b5c00 0x7f8c080b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c0d238640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c080b97c0 0x7f8c080c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c0d238640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c0815dc80 0x7f8c0815e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c0d238640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c08161f30 con 0x7f8c080b5c00 2024-04-25T10:14:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c0d238640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c08161db0 con 0x7f8c0815dc80 2024-04-25T10:14:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c0d238640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c080d2b80 con 0x7f8c080b97c0 2024-04-25T10:14:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c06d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c080b5c00 0x7f8c080b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c06d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c080b5c00 0x7f8c080b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38034/0 (socket says 172.21.15.16:38034) 2024-04-25T10:14:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c06d9d640 1 -- 172.21.15.16:0/1856860572 learned_addr learned my addr 172.21.15.16:0/1856860572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c0659c640 1 --2- 172.21.15.16:0/1856860572 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c080b97c0 0x7f8c080c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c0759e640 1 --2- 172.21.15.16:0/1856860572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c0815dc80 0x7f8c0815e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c06d9d640 1 -- 172.21.15.16:0/1856860572 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c080b97c0 msgr2=0x7f8c080c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c06d9d640 1 --2- 172.21.15.16:0/1856860572 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c080b97c0 0x7f8c080c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c06d9d640 1 -- 172.21.15.16:0/1856860572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c0815dc80 msgr2=0x7f8c0815e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c06d9d640 1 --2- 172.21.15.16:0/1856860572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c0815dc80 0x7f8c0815e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c06d9d640 1 -- 172.21.15.16:0/1856860572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c080cb540 con 0x7f8c080b5c00 2024-04-25T10:14:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c0759e640 1 --2- 172.21.15.16:0/1856860572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c0815dc80 0x7f8c0815e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:38.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c06d9d640 1 --2- 172.21.15.16:0/1856860572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c080b5c00 0x7f8c080b1bf0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f8c000097d0 tx=0x7f8c000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:38.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8bff7fe640 1 -- 172.21.15.16:0/1856860572 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00037040 con 0x7f8c080b5c00 2024-04-25T10:14:38.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8bff7fe640 1 -- 172.21.15.16:0/1856860572 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c00004230 con 0x7f8c080b5c00 2024-04-25T10:14:38.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8c0d238640 1 -- 172.21.15.16:0/1856860572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c080cb820 con 0x7f8c080b5c00 2024-04-25T10:14:38.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.532+0000 7f8bff7fe640 1 -- 172.21.15.16:0/1856860572 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00006e10 con 0x7f8c080b5c00 2024-04-25T10:14:38.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.533+0000 7f8c0d238640 1 -- 172.21.15.16:0/1856860572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c080cbde0 con 0x7f8c080b5c00 2024-04-25T10:14:38.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.533+0000 7f8c0d238640 1 -- 172.21.15.16:0/1856860572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c080750b0 con 0x7f8c080b5c00 2024-04-25T10:14:38.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.534+0000 7f8bff7fe640 1 -- 172.21.15.16:0/1856860572 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c00003610 con 0x7f8c080b5c00 2024-04-25T10:14:38.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.534+0000 7f8bff7fe640 1 --2- 172.21.15.16:0/1856860572 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8be806b500 0x7f8be806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.534+0000 7f8bff7fe640 1 -- 172.21.15.16:0/1856860572 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c000ab030 con 0x7f8c080b5c00 2024-04-25T10:14:38.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.534+0000 7f8c0659c640 1 --2- 172.21.15.16:0/1856860572 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8be806b500 0x7f8be806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.535+0000 7f8c0659c640 1 --2- 172.21.15.16:0/1856860572 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8be806b500 0x7f8be806d9c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8c080c7530 tx=0x7f8bf0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:38.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.538+0000 7f8bff7fe640 1 -- 172.21.15.16:0/1856860572 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c00079eb0 con 0x7f8c080b5c00 2024-04-25T10:14:38.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.718+0000 7f8c0d238640 1 -- 172.21.15.16:0/1856860572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c080d1780 con 0x7f8c080b5c00 2024-04-25T10:14:38.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.718+0000 7f8bff7fe640 1 -- 172.21.15.16:0/1856860572 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c000797b0 con 0x7f8c080b5c00 2024-04-25T10:14:38.722 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:14:38.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.720+0000 7f8c0d238640 1 -- 172.21.15.16:0/1856860572 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8be806b500 msgr2=0x7f8be806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.720+0000 7f8c0d238640 1 --2- 172.21.15.16:0/1856860572 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8be806b500 0x7f8be806d9c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8c080c7530 tx=0x7f8bf0009040 comp rx=0 tx=0).stop 2024-04-25T10:14:38.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.720+0000 7f8c0d238640 1 -- 172.21.15.16:0/1856860572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c080b5c00 msgr2=0x7f8c080b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.720+0000 7f8c0d238640 1 --2- 172.21.15.16:0/1856860572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c080b5c00 0x7f8c080b1bf0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f8c000097d0 tx=0x7f8c000046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.720+0000 7f8c0d238640 1 -- 172.21.15.16:0/1856860572 shutdown_connections 2024-04-25T10:14:38.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.720+0000 7f8c0d238640 1 --2- 172.21.15.16:0/1856860572 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8be806b500 0x7f8be806d9c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.720+0000 7f8c0d238640 1 --2- 172.21.15.16:0/1856860572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c0815dc80 0x7f8c0815e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.720+0000 7f8c0d238640 1 --2- 172.21.15.16:0/1856860572 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c080b97c0 0x7f8c080c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.720+0000 7f8c0d238640 1 --2- 172.21.15.16:0/1856860572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c080b5c00 0x7f8c080b1bf0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.720+0000 7f8c0d238640 1 -- 172.21.15.16:0/1856860572 >> 172.21.15.16:0/1856860572 conn(0x7f8c080d04f0 msgr2=0x7f8c080b6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:38.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.720+0000 7f8c0d238640 1 -- 172.21.15.16:0/1856860572 shutdown_connections 2024-04-25T10:14:38.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.720+0000 7f8c0d238640 1 -- 172.21.15.16:0/1856860572 wait complete. 2024-04-25T10:14:38.735 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:14:38.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.832+0000 7f92f4a38640 1 Processor -- start 2024-04-25T10:14:38.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.832+0000 7f92f4a38640 1 -- start start 2024-04-25T10:14:38.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.833+0000 7f92f4a38640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92f00b3f80 0x7f92f0162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.833+0000 7f92f4a38640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f92f0161400 0x7f92f00cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.833+0000 7f92f4a38640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f00cfed0 con 0x7f92f00b35e0 2024-04-25T10:14:38.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.833+0000 7f92f4a38640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f0162890 con 0x7f92f0161400 2024-04-25T10:14:38.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.833+0000 7f92f4a38640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f00cf850 con 0x7f92f00b3f80 2024-04-25T10:14:38.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.833+0000 7f92eed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f92f0161400 0x7f92f00cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.833+0000 7f92ee59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f92f00b35e0 0x7f92f0160cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45224/0 (socket says 172.21.15.16:45224) 2024-04-25T10:14:38.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.833+0000 7f92ee59c640 1 -- 172.21.15.16:0/1181667260 learned_addr learned my addr 172.21.15.16:0/1181667260 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:38.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.834+0000 7f92eed9d640 1 -- 172.21.15.16:0/1181667260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92f00b3f80 msgr2=0x7f92f0162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:38.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.834+0000 7f92edd9b640 1 --2- 172.21.15.16:0/1181667260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92f00b3f80 0x7f92f0162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.834+0000 7f92eed9d640 1 --2- 172.21.15.16:0/1181667260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92f00b3f80 0x7f92f0162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.834+0000 7f92eed9d640 1 -- 172.21.15.16:0/1181667260 >> v1:172.21.15.16:6789/0 conn(0x7f92f00b35e0 legacy=0x7f92f0160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.834+0000 7f92eed9d640 1 -- 172.21.15.16:0/1181667260 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f00cfb30 con 0x7f92f0161400 2024-04-25T10:14:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.834+0000 7f92ed59a640 1 -- 172.21.15.16:0/1181667260 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166704274 0 0) 0x7f92f00cfed0 con 0x7f92f00b35e0 2024-04-25T10:14:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.834+0000 7f92edd9b640 1 --2- 172.21.15.16:0/1181667260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92f00b3f80 0x7f92f0162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.834+0000 7f92eed9d640 1 --2- 172.21.15.16:0/1181667260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92f0161400 0x7f92f00cc500 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009d90 tx=0x7f92e002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9838ecc821a17325 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.835+0000 7f92ed59a640 1 -- 172.21.15.16:0/1181667260 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e0031600 con 0x7f92f0161400 2024-04-25T10:14:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.835+0000 7f92ed59a640 1 -- 172.21.15.16:0/1181667260 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e00317a0 con 0x7f92f0161400 2024-04-25T10:14:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.835+0000 7f92ed59a640 1 -- 172.21.15.16:0/1181667260 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e0031b70 con 0x7f92f0161400 2024-04-25T10:14:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.835+0000 7f92f4a38640 1 -- 172.21.15.16:0/1181667260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92f0161400 msgr2=0x7f92f00cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.835+0000 7f92f4a38640 1 --2- 172.21.15.16:0/1181667260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92f0161400 0x7f92f00cc500 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009d90 tx=0x7f92e002a5d0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.835+0000 7f92f4a38640 1 -- 172.21.15.16:0/1181667260 shutdown_connections 2024-04-25T10:14:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.835+0000 7f92f4a38640 1 --2- 172.21.15.16:0/1181667260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92f0161400 0x7f92f00cc500 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.835+0000 7f92f4a38640 1 --2- 172.21.15.16:0/1181667260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92f00b3f80 0x7f92f0162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.835+0000 7f92f4a38640 1 -- 172.21.15.16:0/1181667260 >> 172.21.15.16:0/1181667260 conn(0x7f92f006f4c0 msgr2=0x7f92f006f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.835+0000 7f92f4a38640 1 -- 172.21.15.16:0/1181667260 shutdown_connections 2024-04-25T10:14:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.835+0000 7f92f4a38640 1 -- 172.21.15.16:0/1181667260 wait complete. 2024-04-25T10:14:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.837+0000 7f92f4a38640 1 Processor -- start 2024-04-25T10:14:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.837+0000 7f92f4a38640 1 -- start start 2024-04-25T10:14:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.837+0000 7f92f4a38640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92f00b35e0 0x7f92f00cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.837+0000 7f92f4a38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92f00b3f80 0x7f92f01f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.837+0000 7f92f4a38640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92f0161400 0x7f92f014e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.837+0000 7f92f4a38640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f00d05d0 con 0x7f92f00b3f80 2024-04-25T10:14:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.837+0000 7f92f4a38640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f00d0450 con 0x7f92f0161400 2024-04-25T10:14:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.837+0000 7f92f4a38640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f00b39c0 con 0x7f92f00b35e0 2024-04-25T10:14:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.837+0000 7f92edd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92f00b3f80 0x7f92f01f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.837+0000 7f92edd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92f00b3f80 0x7f92f01f63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38058/0 (socket says 172.21.15.16:38058) 2024-04-25T10:14:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.837+0000 7f92edd9b640 1 -- 172.21.15.16:0/447560201 learned_addr learned my addr 172.21.15.16:0/447560201 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.838+0000 7f92edd9b640 1 -- 172.21.15.16:0/447560201 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92f00b35e0 msgr2=0x7f92f00cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.838+0000 7f92edd9b640 1 --2- 172.21.15.16:0/447560201 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92f00b35e0 0x7f92f00cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.838+0000 7f92edd9b640 1 -- 172.21.15.16:0/447560201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92f0161400 msgr2=0x7f92f014e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.838+0000 7f92edd9b640 1 --2- 172.21.15.16:0/447560201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92f0161400 0x7f92f014e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.838+0000 7f92edd9b640 1 -- 172.21.15.16:0/447560201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f014ec30 con 0x7f92f00b3f80 2024-04-25T10:14:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.838+0000 7f92edd9b640 1 --2- 172.21.15.16:0/447560201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92f00b3f80 0x7f92f01f63a0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f92d800c820 tx=0x7f92d800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.838+0000 7f92e6ffd640 1 -- 172.21.15.16:0/447560201 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d8015070 con 0x7f92f00b3f80 2024-04-25T10:14:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.838+0000 7f92e6ffd640 1 -- 172.21.15.16:0/447560201 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d8007a30 con 0x7f92f00b3f80 2024-04-25T10:14:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.838+0000 7f92f4a38640 1 -- 172.21.15.16:0/447560201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f00ca750 con 0x7f92f00b3f80 2024-04-25T10:14:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.838+0000 7f92e6ffd640 1 -- 172.21.15.16:0/447560201 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d8007e00 con 0x7f92f00b3f80 2024-04-25T10:14:38.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.838+0000 7f92f4a38640 1 -- 172.21.15.16:0/447560201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f00cad10 con 0x7f92f00b3f80 2024-04-25T10:14:38.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.839+0000 7f92f4a38640 1 -- 172.21.15.16:0/447560201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f00750b0 con 0x7f92f00b3f80 2024-04-25T10:14:38.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.840+0000 7f92e6ffd640 1 -- 172.21.15.16:0/447560201 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f92d8018070 con 0x7f92f00b3f80 2024-04-25T10:14:38.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.840+0000 7f92e6ffd640 1 --2- 172.21.15.16:0/447560201 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f92d006b4d0 0x7f92d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.840+0000 7f92e6ffd640 1 -- 172.21.15.16:0/447560201 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92d808a0e0 con 0x7f92f00b3f80 2024-04-25T10:14:38.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.840+0000 7f92ee59c640 1 --2- 172.21.15.16:0/447560201 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f92d006b4d0 0x7f92d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.841+0000 7f92ee59c640 1 --2- 172.21.15.16:0/447560201 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f92d006b4d0 0x7f92d006d990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f92e8005880 tx=0x7f92e8007950 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:38.844+0000 7f92e6ffd640 1 -- 172.21.15.16:0/447560201 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f92f014ec30 con 0x7f92f00b3f80 2024-04-25T10:14:38.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:38.870 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:39.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.035+0000 7f92f4a38640 1 -- 172.21.15.16:0/447560201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f92f00b7870 con 0x7f92f00b3f80 2024-04-25T10:14:39.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.035+0000 7f92e6ffd640 1 -- 172.21.15.16:0/447560201 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v115) v1 ==== 80+0+0 (secure 0 0 0) 0x7f92d8058ff0 con 0x7f92f00b3f80 2024-04-25T10:14:39.040 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-04-25T10:14:39.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.037+0000 7f92f4a38640 1 -- 172.21.15.16:0/447560201 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f92d006b4d0 msgr2=0x7f92d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.037+0000 7f92f4a38640 1 --2- 172.21.15.16:0/447560201 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f92d006b4d0 0x7f92d006d990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f92e8005880 tx=0x7f92e8007950 comp rx=0 tx=0).stop 2024-04-25T10:14:39.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.037+0000 7f92f4a38640 1 -- 172.21.15.16:0/447560201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92f00b3f80 msgr2=0x7f92f01f63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.037+0000 7f92f4a38640 1 --2- 172.21.15.16:0/447560201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92f00b3f80 0x7f92f01f63a0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f92d800c820 tx=0x7f92d800cce0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.037+0000 7f92f4a38640 1 -- 172.21.15.16:0/447560201 shutdown_connections 2024-04-25T10:14:39.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.037+0000 7f92f4a38640 1 --2- 172.21.15.16:0/447560201 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f92d006b4d0 0x7f92d006d990 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.037+0000 7f92f4a38640 1 --2- 172.21.15.16:0/447560201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92f0161400 0x7f92f014e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.037+0000 7f92f4a38640 1 --2- 172.21.15.16:0/447560201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92f00b3f80 0x7f92f01f63a0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.037+0000 7f92f4a38640 1 --2- 172.21.15.16:0/447560201 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92f00b35e0 0x7f92f00cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.037+0000 7f92f4a38640 1 -- 172.21.15.16:0/447560201 >> 172.21.15.16:0/447560201 conn(0x7f92f006f4c0 msgr2=0x7f92f00cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.037+0000 7f92f4a38640 1 -- 172.21.15.16:0/447560201 shutdown_connections 2024-04-25T10:14:39.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.037+0000 7f92f4a38640 1 -- 172.21.15.16:0/447560201 wait complete. 2024-04-25T10:14:39.052 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:14:39.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:39.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:39.139 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:14:39.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:39.139 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:14:39.159 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:14:39.159 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:39.214 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:14:39.215 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:14:39.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:39.215 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:14:39.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:39.270 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T10:14:39.270 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:14:39.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.409+0000 7fe9a74b3640 1 Processor -- start 2024-04-25T10:14:39.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.409+0000 7fe9a74b3640 1 -- start start 2024-04-25T10:14:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.410+0000 7fe9a74b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9a015dc50 0x7fe9a0070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.410+0000 7fe9a74b3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe9a00cd940 0x7fe9a00713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.410+0000 7fe9a74b3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a006c220 con 0x7fe9a00cf600 2024-04-25T10:14:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.410+0000 7fe9a74b3640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a00cfe00 con 0x7fe9a00cd940 2024-04-25T10:14:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.410+0000 7fe9a74b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a00cec10 con 0x7fe9a015dc50 2024-04-25T10:14:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.410+0000 7fe9a5a50640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe9a00cf600 0x7fe9a0161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45232/0 (socket says 172.21.15.16:45232) 2024-04-25T10:14:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.410+0000 7fe9a5a50640 1 -- 172.21.15.16:0/616178066 learned_addr learned my addr 172.21.15.16:0/616178066 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.410+0000 7fe9a4a4e640 1 --2- 172.21.15.16:0/616178066 >> v2:172.21.15.186:3300/0 conn(0x7fe9a00cd940 0x7fe9a00713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.410+0000 7fe9a524f640 1 --2- 172.21.15.16:0/616178066 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9a015dc50 0x7fe9a0070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.410+0000 7fe997fff640 1 -- 172.21.15.16:0/616178066 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144531752 0 0) 0x7fe9a006c220 con 0x7fe9a00cf600 2024-04-25T10:14:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.410+0000 7fe997fff640 1 -- 172.21.15.16:0/616178066 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe98c003520 con 0x7fe9a00cf600 2024-04-25T10:14:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.411+0000 7fe9a4a4e640 1 -- 172.21.15.16:0/616178066 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9a015dc50 msgr2=0x7fe9a0070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.411+0000 7fe9a4a4e640 1 --2- 172.21.15.16:0/616178066 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9a015dc50 0x7fe9a0070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.411+0000 7fe997fff640 1 -- 172.21.15.16:0/616178066 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 535335522 0 0) 0x7fe98c003520 con 0x7fe9a00cf600 2024-04-25T10:14:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.411+0000 7fe9a4a4e640 1 -- 172.21.15.16:0/616178066 >> v1:172.21.15.16:6789/0 conn(0x7fe9a00cf600 legacy=0x7fe9a0161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.411+0000 7fe9a4a4e640 1 -- 172.21.15.16:0/616178066 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a00d0050 con 0x7fe9a00cd940 2024-04-25T10:14:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.411+0000 7fe9a524f640 1 --2- 172.21.15.16:0/616178066 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9a015dc50 0x7fe9a0070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.411+0000 7fe9a4a4e640 1 --2- 172.21.15.16:0/616178066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9a00cd940 0x7fe9a00713c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe9900097a0 tx=0x7fe99002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc87d1ba7f98e494 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.412+0000 7fe997fff640 1 -- 172.21.15.16:0/616178066 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe990037070 con 0x7fe9a00cd940 2024-04-25T10:14:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.412+0000 7fe997fff640 1 -- 172.21.15.16:0/616178066 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99002aba0 con 0x7fe9a00cd940 2024-04-25T10:14:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.412+0000 7fe997fff640 1 -- 172.21.15.16:0/616178066 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9980019f0 con 0x7fe9a00cd940 2024-04-25T10:14:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.412+0000 7fe9a74b3640 1 -- 172.21.15.16:0/616178066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9a00cd940 msgr2=0x7fe9a00713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.412+0000 7fe9a74b3640 1 --2- 172.21.15.16:0/616178066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9a00cd940 0x7fe9a00713c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe9900097a0 tx=0x7fe99002a110 comp rx=0 tx=0).stop 2024-04-25T10:14:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.412+0000 7fe9a74b3640 1 -- 172.21.15.16:0/616178066 shutdown_connections 2024-04-25T10:14:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.412+0000 7fe9a74b3640 1 --2- 172.21.15.16:0/616178066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9a00cd940 0x7fe9a00713c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.412+0000 7fe9a74b3640 1 --2- 172.21.15.16:0/616178066 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9a015dc50 0x7fe9a0070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.412+0000 7fe9a74b3640 1 -- 172.21.15.16:0/616178066 >> 172.21.15.16:0/616178066 conn(0x7fe9a006f430 msgr2=0x7fe9a006f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.412+0000 7fe9a74b3640 1 -- 172.21.15.16:0/616178066 shutdown_connections 2024-04-25T10:14:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.412+0000 7fe9a74b3640 1 -- 172.21.15.16:0/616178066 wait complete. 2024-04-25T10:14:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.413+0000 7fe9a74b3640 1 Processor -- start 2024-04-25T10:14:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.414+0000 7fe9a74b3640 1 -- start start 2024-04-25T10:14:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.414+0000 7fe9a74b3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9a00cd940 0x7fe9a0160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.414+0000 7fe9a74b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9a00cf600 0x7fe9a015e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.414+0000 7fe9a74b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9a015dc50 0x7fe9a015e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.414+0000 7fe9a74b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a00ce8a0 con 0x7fe9a015dc50 2024-04-25T10:14:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.414+0000 7fe9a74b3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a00cc0c0 con 0x7fe9a00cd940 2024-04-25T10:14:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.414+0000 7fe9a74b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a00cd620 con 0x7fe9a00cf600 2024-04-25T10:14:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.414+0000 7fe9a5a50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9a015dc50 0x7fe9a015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.414+0000 7fe9a5a50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9a015dc50 0x7fe9a015e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38072/0 (socket says 172.21.15.16:38072) 2024-04-25T10:14:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.414+0000 7fe9a5a50640 1 -- 172.21.15.16:0/2816244074 learned_addr learned my addr 172.21.15.16:0/2816244074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.415+0000 7fe9a524f640 1 --2- 172.21.15.16:0/2816244074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9a00cd940 0x7fe9a0160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.415+0000 7fe9a4a4e640 1 --2- 172.21.15.16:0/2816244074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9a00cf600 0x7fe9a015e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.415+0000 7fe9a5a50640 1 -- 172.21.15.16:0/2816244074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9a00cf600 msgr2=0x7fe9a015e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.415+0000 7fe9a5a50640 1 --2- 172.21.15.16:0/2816244074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9a00cf600 0x7fe9a015e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.415+0000 7fe9a5a50640 1 -- 172.21.15.16:0/2816244074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9a00cd940 msgr2=0x7fe9a0160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.415+0000 7fe9a5a50640 1 --2- 172.21.15.16:0/2816244074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9a00cd940 0x7fe9a0160060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.415+0000 7fe9a5a50640 1 -- 172.21.15.16:0/2816244074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a00c6440 con 0x7fe9a015dc50 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.415+0000 7fe9a5a50640 1 --2- 172.21.15.16:0/2816244074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9a015dc50 0x7fe9a015e570 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fe998001b60 tx=0x7fe99800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.415+0000 7fe995ffb640 1 -- 172.21.15.16:0/2816244074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe998017070 con 0x7fe9a015dc50 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.415+0000 7fe995ffb640 1 -- 172.21.15.16:0/2816244074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe998012400 con 0x7fe9a015dc50 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.415+0000 7fe995ffb640 1 -- 172.21.15.16:0/2816244074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9980127b0 con 0x7fe9a015dc50 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.415+0000 7fe9a74b3640 1 -- 172.21.15.16:0/2816244074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a01608e0 con 0x7fe9a015dc50 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.415+0000 7fe9a74b3640 1 -- 172.21.15.16:0/2816244074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a0160ea0 con 0x7fe9a015dc50 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.416+0000 7fe9a74b3640 1 -- 172.21.15.16:0/2816244074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a00750b0 con 0x7fe9a015dc50 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.417+0000 7fe995ffb640 1 -- 172.21.15.16:0/2816244074 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe99801a070 con 0x7fe9a015dc50 2024-04-25T10:14:39.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.417+0000 7fe995ffb640 1 --2- 172.21.15.16:0/2816244074 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe98406b4d0 0x7fe98406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.417+0000 7fe995ffb640 1 -- 172.21.15.16:0/2816244074 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe998089fe0 con 0x7fe9a015dc50 2024-04-25T10:14:39.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.417+0000 7fe9a524f640 1 --2- 172.21.15.16:0/2816244074 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe98406b4d0 0x7fe98406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.418+0000 7fe9a524f640 1 --2- 172.21.15.16:0/2816244074 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe98406b4d0 0x7fe98406d990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009560 tx=0x7fe99c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.421+0000 7fe995ffb640 1 -- 172.21.15.16:0/2816244074 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9980588a0 con 0x7fe9a015dc50 2024-04-25T10:14:39.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.638+0000 7fe9a74b3640 1 -- 172.21.15.16:0/2816244074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe9a00b77a0 con 0x7fe9a015dc50 2024-04-25T10:14:39.642 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-04-25T10:14:39.638+0000 7ff13f9bb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:14:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.640+0000 7fe995ffb640 1 -- 172.21.15.16:0/2816244074 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v85) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe99805c940 con 0x7fe9a015dc50 2024-04-25T10:14:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:14:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.643+0000 7fe9a74b3640 1 -- 172.21.15.16:0/2816244074 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe98406b4d0 msgr2=0x7fe98406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.643+0000 7fe9a74b3640 1 --2- 172.21.15.16:0/2816244074 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe98406b4d0 0x7fe98406d990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009560 tx=0x7fe99c009190 comp rx=0 tx=0).stop 2024-04-25T10:14:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.643+0000 7fe9a74b3640 1 -- 172.21.15.16:0/2816244074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9a015dc50 msgr2=0x7fe9a015e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.643+0000 7fe9a74b3640 1 --2- 172.21.15.16:0/2816244074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9a015dc50 0x7fe9a015e570 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fe998001b60 tx=0x7fe99800a430 comp rx=0 tx=0).stop 2024-04-25T10:14:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.643+0000 7fe9a74b3640 1 -- 172.21.15.16:0/2816244074 shutdown_connections 2024-04-25T10:14:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.643+0000 7fe9a74b3640 1 --2- 172.21.15.16:0/2816244074 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe98406b4d0 0x7fe98406d990 secure :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009560 tx=0x7fe99c009190 comp rx=0 tx=0).stop 2024-04-25T10:14:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.643+0000 7fe9a74b3640 1 --2- 172.21.15.16:0/2816244074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9a015dc50 0x7fe9a015e570 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.643+0000 7fe9a74b3640 1 --2- 172.21.15.16:0/2816244074 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe9a00cf600 0x7fe9a015e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.643+0000 7fe9a74b3640 1 --2- 172.21.15.16:0/2816244074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9a00cd940 0x7fe9a0160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.643+0000 7fe9a74b3640 1 -- 172.21.15.16:0/2816244074 >> 172.21.15.16:0/2816244074 conn(0x7fe9a006f430 msgr2=0x7fe9a00b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.643+0000 7fe9a74b3640 1 -- 172.21.15.16:0/2816244074 shutdown_connections 2024-04-25T10:14:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.643+0000 7fe9a74b3640 1 -- 172.21.15.16:0/2816244074 wait complete. 2024-04-25T10:14:39.651 INFO:tasks.ceph.mds.c.smithi016.stderr: -6847> 2024-04-25T10:14:33.054+0000 7f3ec1f21640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:14:39.659 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:14:39.662 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:14:39.682 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-04-25T10:14:39.702 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-04-25T10:14:39.702 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2024-04-25T10:14:39.702 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:14:39.702 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2024-04-25T10:14:39.702 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.702 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2024-04-25T10:14:39.702 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.702 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2024-04-25T10:14:39.703 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.703 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2024-04-25T10:14:39.703 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2024-04-25T10:14:39.703 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.703 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2024-04-25T10:14:39.703 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2024-04-25T10:14:39.703 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.703 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:14:39.704 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2024-04-25T10:14:39.704 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.704 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:14:39.704 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:14:39.704 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.704 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2024-04-25T10:14:39.704 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.704 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2024-04-25T10:14:39.704 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2024-04-25T10:14:39.704 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:14:39.704 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:14:39.704 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2024-04-25T10:14:39.705 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:14:39.705 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2024-04-25T10:14:39.705 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:14:39.705 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:14:39.705 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2024-04-25T10:14:39.705 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.705 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:14:39.705 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.705 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2024-04-25T10:14:39.705 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.705 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2024-04-25T10:14:39.705 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.706 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2024-04-25T10:14:39.706 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2024-04-25T10:14:39.706 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2024-04-25T10:14:39.706 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]" 2024-04-25T10:14:39.706 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2024-04-25T10:14:39.706 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 34bc5eca-3a21-42a6-a26f-07da0905e8bc 2024-04-25T10:14:39.706 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.706 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2024-04-25T10:14:39.706 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2024-04-25T10:14:39.706 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.706 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2024-04-25T10:14:39.706 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2024-04-25T10:14:39.706 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:14:39.707 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.707 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2024-04-25T10:14:39.707 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2024-04-25T10:14:39.707 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.707 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2024-04-25T10:14:39.707 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2024-04-25T10:14:39.707 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2024-04-25T10:14:39.707 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2024-04-25T10:14:39.707 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.707 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2024-04-25T10:14:39.707 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2024-04-25T10:14:39.707 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.708 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:14:39.708 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.708 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2024-04-25T10:14:39.708 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.708 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2024-04-25T10:14:39.708 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.708 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:14:39.708 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2024-04-25T10:14:39.708 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.708 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2024-04-25T10:14:39.708 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2024-04-25T10:14:39.708 INFO:teuthology.orchestra.run.smithi016.stdout: bdev async discard = True 2024-04-25T10:14:39.708 INFO:teuthology.orchestra.run.smithi016.stdout: bdev enable discard = True 2024-04-25T10:14:39.709 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore allocator = bitmap 2024-04-25T10:14:39.709 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2024-04-25T10:14:39.709 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2024-04-25T10:14:39.709 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 1/20 2024-04-25T10:14:39.709 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 1/20 2024-04-25T10:14:39.709 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:39.709 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2024-04-25T10:14:39.709 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 4/10 2024-04-25T10:14:39.709 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:14:39.709 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2024-04-25T10:14:39.709 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:14:39.709 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:14:39.709 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2024-04-25T10:14:39.710 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-04-25T10:14:39.710 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.710 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2024-04-25T10:14:39.710 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:39.710 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2024-04-25T10:14:39.710 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-04-25T10:14:39.710 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-04-25T10:14:39.710 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:39.710 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:39.710 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2024-04-25T10:14:39.710 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.710 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2024-04-25T10:14:39.710 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:39.710 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-04-25T10:14:39.711 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2024-04-25T10:14:39.711 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-04-25T10:14:39.711 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2024-04-25T10:14:39.711 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2024-04-25T10:14:39.711 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:39.711 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2024-04-25T10:14:39.711 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2024-04-25T10:14:39.711 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:39.711 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.711 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:14:39.711 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:14:39.711 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:14:39.711 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.711 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:14:39.711 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:14:39.712 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:14:39.712 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.712 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2024-04-25T10:14:39.712 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:14:39.712 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.712 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:14:39.712 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2024-04-25T10:14:39.712 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.712 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2024-04-25T10:14:39.712 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2024-04-25T10:14:39.712 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2024-04-25T10:14:39.712 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2024-04-25T10:14:39.712 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:14:39.712 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:14:39.712 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2024-04-25T10:14:39.713 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-04-25T10:14:39.713 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:39.713 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2024-04-25T10:14:39.713 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-04-25T10:14:39.713 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-04-25T10:14:39.713 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2024-04-25T10:14:39.713 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2024-04-25T10:14:39.713 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2024-04-25T10:14:39.713 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-04-25T10:14:39.713 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-04-25T10:14:39.713 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-04-25T10:14:39.713 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-04-25T10:14:39.713 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2024-04-25T10:14:39.713 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2024-04-25T10:14:39.713 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2024-04-25T10:14:39.713 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:39.714 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2024-04-25T10:14:39.714 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2024-04-25T10:14:39.714 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2024-04-25T10:14:39.714 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = False 2024-04-25T10:14:39.714 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-04-25T10:14:39.714 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-04-25T10:14:39.714 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-04-25T10:14:39.715 INFO:teuthology.orchestra.run.smithi186.stdout:[global] 2024-04-25T10:14:39.715 INFO:teuthology.orchestra.run.smithi186.stdout: chdir = "" 2024-04-25T10:14:39.715 INFO:teuthology.orchestra.run.smithi186.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:14:39.715 INFO:teuthology.orchestra.run.smithi186.stdout: auth supported = cephx 2024-04-25T10:14:39.715 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.715 INFO:teuthology.orchestra.run.smithi186.stdout: filestore xattr use omap = true 2024-04-25T10:14:39.715 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.715 INFO:teuthology.orchestra.run.smithi186.stdout: mon clock drift allowed = 1.000 2024-04-25T10:14:39.715 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.715 INFO:teuthology.orchestra.run.smithi186.stdout: osd crush chooseleaf type = 0 2024-04-25T10:14:39.715 INFO:teuthology.orchestra.run.smithi186.stdout: auth debug = true 2024-04-25T10:14:39.715 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.716 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on old message = true 2024-04-25T10:14:39.716 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on bug = true 2024-04-25T10:14:39.716 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.716 INFO:teuthology.orchestra.run.smithi186.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:14:39.716 INFO:teuthology.orchestra.run.smithi186.stdout: mon pg warn max object skew = 0 2024-04-25T10:14:39.716 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.716 INFO:teuthology.orchestra.run.smithi186.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:14:39.716 INFO:teuthology.orchestra.run.smithi186.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:14:39.716 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.716 INFO:teuthology.orchestra.run.smithi186.stdout: osd pool default size = 2 2024-04-25T10:14:39.716 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.716 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd allow primary affinity = true 2024-04-25T10:14:39.716 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd allow pg remap = true 2024-04-25T10:14:39.716 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:14:39.716 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:14:39.716 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on no sortbitwise = false 2024-04-25T10:14:39.717 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:14:39.717 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on too few osds = false 2024-04-25T10:14:39.717 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:14:39.717 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:14:39.717 INFO:teuthology.orchestra.run.smithi186.stdout: mon_allow_pool_size_one = true 2024-04-25T10:14:39.717 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.717 INFO:teuthology.orchestra.run.smithi186.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:14:39.717 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.717 INFO:teuthology.orchestra.run.smithi186.stdout: osd default data pool replay window = 5 2024-04-25T10:14:39.717 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.717 INFO:teuthology.orchestra.run.smithi186.stdout: mon allow pool delete = true 2024-04-25T10:14:39.717 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.717 INFO:teuthology.orchestra.run.smithi186.stdout: mon cluster log file level = debug 2024-04-25T10:14:39.717 INFO:teuthology.orchestra.run.smithi186.stdout: debug asserts on shutdown = true 2024-04-25T10:14:39.718 INFO:teuthology.orchestra.run.smithi186.stdout: mon health detail to clog = false 2024-04-25T10:14:39.718 INFO:teuthology.orchestra.run.smithi186.stdout: mon host = "172.21.15.16,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]" 2024-04-25T10:14:39.718 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on skipped message = False 2024-04-25T10:14:39.718 INFO:teuthology.orchestra.run.smithi186.stdout: fsid = 34bc5eca-3a21-42a6-a26f-07da0905e8bc 2024-04-25T10:14:39.718 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.718 INFO:teuthology.orchestra.run.smithi186.stdout:[osd] 2024-04-25T10:14:39.718 INFO:teuthology.orchestra.run.smithi186.stdout: osd journal size = 100 2024-04-25T10:14:39.718 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.718 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub load threshold = 5.0 2024-04-25T10:14:39.718 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub max interval = 600 2024-04-25T10:14:39.718 INFO:teuthology.orchestra.run.smithi186.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:14:39.718 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.718 INFO:teuthology.orchestra.run.smithi186.stdout: osd recover clone overlap = true 2024-04-25T10:14:39.718 INFO:teuthology.orchestra.run.smithi186.stdout: osd recovery max chunk = 1048576 2024-04-25T10:14:39.719 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.719 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug shutdown = true 2024-04-25T10:14:39.719 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug op order = true 2024-04-25T10:14:39.719 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug verify stray on activate = true 2024-04-25T10:14:39.719 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug trim objects = true 2024-04-25T10:14:39.719 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.719 INFO:teuthology.orchestra.run.smithi186.stdout: osd open classes on start = true 2024-04-25T10:14:39.719 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug pg log writeout = true 2024-04-25T10:14:39.719 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.719 INFO:teuthology.orchestra.run.smithi186.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:14:39.719 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.719 INFO:teuthology.orchestra.run.smithi186.stdout: osd map max advance = 10 2024-04-25T10:14:39.719 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.720 INFO:teuthology.orchestra.run.smithi186.stdout: journal zero on create = true 2024-04-25T10:14:39.720 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.720 INFO:teuthology.orchestra.run.smithi186.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:14:39.720 INFO:teuthology.orchestra.run.smithi186.stdout: filestore apply finisher threads = 3 2024-04-25T10:14:39.720 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.720 INFO:teuthology.orchestra.run.smithi186.stdout: bdev debug aio = true 2024-04-25T10:14:39.720 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug misdirected ops = true 2024-04-25T10:14:39.720 INFO:teuthology.orchestra.run.smithi186.stdout: bdev async discard = True 2024-04-25T10:14:39.720 INFO:teuthology.orchestra.run.smithi186.stdout: bdev enable discard = True 2024-04-25T10:14:39.720 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore allocator = bitmap 2024-04-25T10:14:39.720 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore block size = 96636764160 2024-04-25T10:14:39.720 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore fsck on mount = True 2024-04-25T10:14:39.720 INFO:teuthology.orchestra.run.smithi186.stdout: debug bluefs = 1/20 2024-04-25T10:14:39.721 INFO:teuthology.orchestra.run.smithi186.stdout: debug bluestore = 1/20 2024-04-25T10:14:39.721 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:39.721 INFO:teuthology.orchestra.run.smithi186.stdout: debug osd = 20 2024-04-25T10:14:39.721 INFO:teuthology.orchestra.run.smithi186.stdout: debug rocksdb = 4/10 2024-04-25T10:14:39.721 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:14:39.721 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd full ratio = 0.9 2024-04-25T10:14:39.721 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:14:39.721 INFO:teuthology.orchestra.run.smithi186.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:14:39.721 INFO:teuthology.orchestra.run.smithi186.stdout: osd objectstore = bluestore 2024-04-25T10:14:39.721 INFO:teuthology.orchestra.run.smithi186.stdout: osd op complaint time = 180 2024-04-25T10:14:39.721 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.721 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr] 2024-04-25T10:14:39.722 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:39.722 INFO:teuthology.orchestra.run.smithi186.stdout: debug mgr = 20 2024-04-25T10:14:39.722 INFO:teuthology.orchestra.run.smithi186.stdout: debug mon = 20 2024-04-25T10:14:39.722 INFO:teuthology.orchestra.run.smithi186.stdout: debug auth = 20 2024-04-25T10:14:39.722 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:39.722 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:39.722 INFO:teuthology.orchestra.run.smithi186.stdout: mgr/telemetry/nag = false 2024-04-25T10:14:39.722 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.722 INFO:teuthology.orchestra.run.smithi186.stdout:[mon] 2024-04-25T10:14:39.722 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:39.722 INFO:teuthology.orchestra.run.smithi186.stdout: debug mon = 20 2024-04-25T10:14:39.722 INFO:teuthology.orchestra.run.smithi186.stdout: debug paxos = 20 2024-04-25T10:14:39.722 INFO:teuthology.orchestra.run.smithi186.stdout: debug auth = 20 2024-04-25T10:14:39.722 INFO:teuthology.orchestra.run.smithi186.stdout: mon data avail warn = 5 2024-04-25T10:14:39.723 INFO:teuthology.orchestra.run.smithi186.stdout: mon mgr mkfs grace = 240 2024-04-25T10:14:39.723 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:39.723 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd reporter subtree level = osd 2024-04-25T10:14:39.723 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd prime pg temp = true 2024-04-25T10:14:39.723 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:39.723 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.723 INFO:teuthology.orchestra.run.smithi186.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:14:39.723 INFO:teuthology.orchestra.run.smithi186.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:14:39.723 INFO:teuthology.orchestra.run.smithi186.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:14:39.723 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.723 INFO:teuthology.orchestra.run.smithi186.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:14:39.723 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:14:39.723 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:14:39.723 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.724 INFO:teuthology.orchestra.run.smithi186.stdout: # 1m isn't quite enough 2024-04-25T10:14:39.724 INFO:teuthology.orchestra.run.smithi186.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:14:39.724 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.724 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:14:39.724 INFO:teuthology.orchestra.run.smithi186.stdout: mon op complaint time = 120 2024-04-25T10:14:39.724 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.724 INFO:teuthology.orchestra.run.smithi186.stdout:[client] 2024-04-25T10:14:39.724 INFO:teuthology.orchestra.run.smithi186.stdout: rgw cache enabled = true 2024-04-25T10:14:39.724 INFO:teuthology.orchestra.run.smithi186.stdout: rgw enable ops log = true 2024-04-25T10:14:39.724 INFO:teuthology.orchestra.run.smithi186.stdout: rgw enable usage log = true 2024-04-25T10:14:39.724 INFO:teuthology.orchestra.run.smithi186.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:14:39.724 INFO:teuthology.orchestra.run.smithi186.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:14:39.724 INFO:teuthology.orchestra.run.smithi186.stdout: client mount timeout = 600 2024-04-25T10:14:39.725 INFO:teuthology.orchestra.run.smithi186.stdout: debug client = 20 2024-04-25T10:14:39.725 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:39.725 INFO:teuthology.orchestra.run.smithi186.stdout: pid file = "" 2024-04-25T10:14:39.725 INFO:teuthology.orchestra.run.smithi186.stdout: rados mon op timeout = 900 2024-04-25T10:14:39.725 INFO:teuthology.orchestra.run.smithi186.stdout: rados osd op timeout = 900 2024-04-25T10:14:39.725 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.a] 2024-04-25T10:14:39.725 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.b] 2024-04-25T10:14:39.725 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.c] 2024-04-25T10:14:39.725 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.a] 2024-04-25T10:14:39.725 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.c] 2024-04-25T10:14:39.725 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.b] 2024-04-25T10:14:39.725 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.d] 2024-04-25T10:14:39.725 INFO:teuthology.orchestra.run.smithi186.stdout:[mds] 2024-04-25T10:14:39.725 INFO:teuthology.orchestra.run.smithi186.stdout: debug mds = 20 2024-04-25T10:14:39.725 INFO:teuthology.orchestra.run.smithi186.stdout: debug mds balancer = 20 2024-04-25T10:14:39.725 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:39.725 INFO:teuthology.orchestra.run.smithi186.stdout: mds debug frag = True 2024-04-25T10:14:39.726 INFO:teuthology.orchestra.run.smithi186.stdout: mds debug scatterstat = True 2024-04-25T10:14:39.726 INFO:teuthology.orchestra.run.smithi186.stdout: mds op complaint time = 180 2024-04-25T10:14:39.726 INFO:teuthology.orchestra.run.smithi186.stdout: mds verify scatter = False 2024-04-25T10:14:39.726 INFO:teuthology.orchestra.run.smithi186.stdout: osd op complaint time = 180 2024-04-25T10:14:39.726 INFO:teuthology.orchestra.run.smithi186.stdout: rados mon op timeout = 900 2024-04-25T10:14:39.726 INFO:teuthology.orchestra.run.smithi186.stdout: rados osd op timeout = 900 2024-04-25T10:14:39.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:14:39.769 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:14:39.809 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-04-25T10:14:39.809 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2024-04-25T10:14:39.809 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:14:39.809 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2024-04-25T10:14:39.809 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.809 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2024-04-25T10:14:39.809 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.809 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2024-04-25T10:14:39.809 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.809 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2024-04-25T10:14:39.809 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2024-04-25T10:14:39.809 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2024-04-25T10:14:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2024-04-25T10:14:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:14:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2024-04-25T10:14:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:14:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:14:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2024-04-25T10:14:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2024-04-25T10:14:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2024-04-25T10:14:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:14:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:14:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2024-04-25T10:14:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:14:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2024-04-25T10:14:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:14:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:14:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2024-04-25T10:14:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:14:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2024-04-25T10:14:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2024-04-25T10:14:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2024-04-25T10:14:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2024-04-25T10:14:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2024-04-25T10:14:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]" 2024-04-25T10:14:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2024-04-25T10:14:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 34bc5eca-3a21-42a6-a26f-07da0905e8bc 2024-04-25T10:14:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.812 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2024-04-25T10:14:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2024-04-25T10:14:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2024-04-25T10:14:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2024-04-25T10:14:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:14:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2024-04-25T10:14:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2024-04-25T10:14:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2024-04-25T10:14:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2024-04-25T10:14:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2024-04-25T10:14:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2024-04-25T10:14:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2024-04-25T10:14:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2024-04-25T10:14:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:14:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2024-04-25T10:14:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2024-04-25T10:14:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:14:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2024-04-25T10:14:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2024-04-25T10:14:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2024-04-25T10:14:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: bdev async discard = True 2024-04-25T10:14:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: bdev enable discard = True 2024-04-25T10:14:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore allocator = bitmap 2024-04-25T10:14:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2024-04-25T10:14:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2024-04-25T10:14:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 1/20 2024-04-25T10:14:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 1/20 2024-04-25T10:14:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:39.815 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2024-04-25T10:14:39.815 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 4/10 2024-04-25T10:14:39.815 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:14:39.815 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2024-04-25T10:14:39.815 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:14:39.815 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:14:39.815 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2024-04-25T10:14:39.815 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-04-25T10:14:39.815 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.815 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2024-04-25T10:14:39.815 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:39.815 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2024-04-25T10:14:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-04-25T10:14:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-04-25T10:14:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2024-04-25T10:14:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.816 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2024-04-25T10:14:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-04-25T10:14:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2024-04-25T10:14:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-04-25T10:14:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2024-04-25T10:14:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2024-04-25T10:14:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2024-04-25T10:14:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2024-04-25T10:14:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:14:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:14:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:14:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:14:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:14:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:14:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2024-04-25T10:14:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:14:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.818 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:14:39.818 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2024-04-25T10:14:39.818 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:39.818 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2024-04-25T10:14:39.818 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2024-04-25T10:14:39.818 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2024-04-25T10:14:39.818 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2024-04-25T10:14:39.818 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:14:39.818 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:14:39.818 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2024-04-25T10:14:39.818 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-04-25T10:14:39.818 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:39.818 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2024-04-25T10:14:39.818 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-04-25T10:14:39.819 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-04-25T10:14:39.819 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2024-04-25T10:14:39.819 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2024-04-25T10:14:39.819 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2024-04-25T10:14:39.819 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-04-25T10:14:39.819 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-04-25T10:14:39.819 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-04-25T10:14:39.819 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-04-25T10:14:39.819 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2024-04-25T10:14:39.819 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2024-04-25T10:14:39.819 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2024-04-25T10:14:39.819 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:39.819 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2024-04-25T10:14:39.819 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = False 2024-04-25T10:14:39.819 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2024-04-25T10:14:39.819 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = False 2024-04-25T10:14:39.820 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-04-25T10:14:39.820 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-04-25T10:14:39.820 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-04-25T10:14:39.821 INFO:teuthology.orchestra.run.smithi186.stdout:[global] 2024-04-25T10:14:39.821 INFO:teuthology.orchestra.run.smithi186.stdout: chdir = "" 2024-04-25T10:14:39.821 INFO:teuthology.orchestra.run.smithi186.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:14:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: auth supported = cephx 2024-04-25T10:14:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: filestore xattr use omap = true 2024-04-25T10:14:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: mon clock drift allowed = 1.000 2024-04-25T10:14:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: osd crush chooseleaf type = 0 2024-04-25T10:14:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: auth debug = true 2024-04-25T10:14:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on old message = true 2024-04-25T10:14:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on bug = true 2024-04-25T10:14:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:14:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: mon pg warn max object skew = 0 2024-04-25T10:14:39.823 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.823 INFO:teuthology.orchestra.run.smithi186.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:14:39.823 INFO:teuthology.orchestra.run.smithi186.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:14:39.823 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.823 INFO:teuthology.orchestra.run.smithi186.stdout: osd pool default size = 2 2024-04-25T10:14:39.823 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.823 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd allow primary affinity = true 2024-04-25T10:14:39.823 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd allow pg remap = true 2024-04-25T10:14:39.823 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:14:39.823 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:14:39.823 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on no sortbitwise = false 2024-04-25T10:14:39.823 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:14:39.823 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on too few osds = false 2024-04-25T10:14:39.823 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:14:39.824 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:14:39.824 INFO:teuthology.orchestra.run.smithi186.stdout: mon_allow_pool_size_one = true 2024-04-25T10:14:39.824 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.824 INFO:teuthology.orchestra.run.smithi186.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:14:39.824 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.824 INFO:teuthology.orchestra.run.smithi186.stdout: osd default data pool replay window = 5 2024-04-25T10:14:39.824 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.824 INFO:teuthology.orchestra.run.smithi186.stdout: mon allow pool delete = true 2024-04-25T10:14:39.824 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.824 INFO:teuthology.orchestra.run.smithi186.stdout: mon cluster log file level = debug 2024-04-25T10:14:39.824 INFO:teuthology.orchestra.run.smithi186.stdout: debug asserts on shutdown = true 2024-04-25T10:14:39.824 INFO:teuthology.orchestra.run.smithi186.stdout: mon health detail to clog = false 2024-04-25T10:14:39.824 INFO:teuthology.orchestra.run.smithi186.stdout: mon host = "172.21.15.16,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]" 2024-04-25T10:14:39.824 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on skipped message = False 2024-04-25T10:14:39.825 INFO:teuthology.orchestra.run.smithi186.stdout: fsid = 34bc5eca-3a21-42a6-a26f-07da0905e8bc 2024-04-25T10:14:39.825 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.825 INFO:teuthology.orchestra.run.smithi186.stdout:[osd] 2024-04-25T10:14:39.825 INFO:teuthology.orchestra.run.smithi186.stdout: osd journal size = 100 2024-04-25T10:14:39.825 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.825 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub load threshold = 5.0 2024-04-25T10:14:39.825 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub max interval = 600 2024-04-25T10:14:39.825 INFO:teuthology.orchestra.run.smithi186.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:14:39.825 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.825 INFO:teuthology.orchestra.run.smithi186.stdout: osd recover clone overlap = true 2024-04-25T10:14:39.825 INFO:teuthology.orchestra.run.smithi186.stdout: osd recovery max chunk = 1048576 2024-04-25T10:14:39.825 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.825 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug shutdown = true 2024-04-25T10:14:39.825 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug op order = true 2024-04-25T10:14:39.825 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug verify stray on activate = true 2024-04-25T10:14:39.826 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug trim objects = true 2024-04-25T10:14:39.826 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.826 INFO:teuthology.orchestra.run.smithi186.stdout: osd open classes on start = true 2024-04-25T10:14:39.826 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug pg log writeout = true 2024-04-25T10:14:39.826 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.826 INFO:teuthology.orchestra.run.smithi186.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:14:39.826 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.826 INFO:teuthology.orchestra.run.smithi186.stdout: osd map max advance = 10 2024-04-25T10:14:39.826 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.826 INFO:teuthology.orchestra.run.smithi186.stdout: journal zero on create = true 2024-04-25T10:14:39.826 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.826 INFO:teuthology.orchestra.run.smithi186.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:14:39.826 INFO:teuthology.orchestra.run.smithi186.stdout: filestore apply finisher threads = 3 2024-04-25T10:14:39.826 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.826 INFO:teuthology.orchestra.run.smithi186.stdout: bdev debug aio = true 2024-04-25T10:14:39.827 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug misdirected ops = true 2024-04-25T10:14:39.827 INFO:teuthology.orchestra.run.smithi186.stdout: bdev async discard = True 2024-04-25T10:14:39.827 INFO:teuthology.orchestra.run.smithi186.stdout: bdev enable discard = True 2024-04-25T10:14:39.827 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore allocator = bitmap 2024-04-25T10:14:39.827 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore block size = 96636764160 2024-04-25T10:14:39.827 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore fsck on mount = True 2024-04-25T10:14:39.827 INFO:teuthology.orchestra.run.smithi186.stdout: debug bluefs = 1/20 2024-04-25T10:14:39.827 INFO:teuthology.orchestra.run.smithi186.stdout: debug bluestore = 1/20 2024-04-25T10:14:39.827 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:39.827 INFO:teuthology.orchestra.run.smithi186.stdout: debug osd = 20 2024-04-25T10:14:39.827 INFO:teuthology.orchestra.run.smithi186.stdout: debug rocksdb = 4/10 2024-04-25T10:14:39.827 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:14:39.827 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd full ratio = 0.9 2024-04-25T10:14:39.827 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:14:39.827 INFO:teuthology.orchestra.run.smithi186.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:14:39.828 INFO:teuthology.orchestra.run.smithi186.stdout: osd objectstore = bluestore 2024-04-25T10:14:39.828 INFO:teuthology.orchestra.run.smithi186.stdout: osd op complaint time = 180 2024-04-25T10:14:39.828 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.828 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr] 2024-04-25T10:14:39.828 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:39.828 INFO:teuthology.orchestra.run.smithi186.stdout: debug mgr = 20 2024-04-25T10:14:39.828 INFO:teuthology.orchestra.run.smithi186.stdout: debug mon = 20 2024-04-25T10:14:39.828 INFO:teuthology.orchestra.run.smithi186.stdout: debug auth = 20 2024-04-25T10:14:39.828 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:39.828 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:39.828 INFO:teuthology.orchestra.run.smithi186.stdout: mgr/telemetry/nag = false 2024-04-25T10:14:39.828 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.828 INFO:teuthology.orchestra.run.smithi186.stdout:[mon] 2024-04-25T10:14:39.828 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:39.828 INFO:teuthology.orchestra.run.smithi186.stdout: debug mon = 20 2024-04-25T10:14:39.829 INFO:teuthology.orchestra.run.smithi186.stdout: debug paxos = 20 2024-04-25T10:14:39.829 INFO:teuthology.orchestra.run.smithi186.stdout: debug auth = 20 2024-04-25T10:14:39.829 INFO:teuthology.orchestra.run.smithi186.stdout: mon data avail warn = 5 2024-04-25T10:14:39.829 INFO:teuthology.orchestra.run.smithi186.stdout: mon mgr mkfs grace = 240 2024-04-25T10:14:39.829 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:39.829 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd reporter subtree level = osd 2024-04-25T10:14:39.829 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd prime pg temp = true 2024-04-25T10:14:39.829 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:39.829 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.829 INFO:teuthology.orchestra.run.smithi186.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:14:39.829 INFO:teuthology.orchestra.run.smithi186.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:14:39.829 INFO:teuthology.orchestra.run.smithi186.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:14:39.829 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.829 INFO:teuthology.orchestra.run.smithi186.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:14:39.830 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:14:39.830 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:14:39.830 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.830 INFO:teuthology.orchestra.run.smithi186.stdout: # 1m isn't quite enough 2024-04-25T10:14:39.830 INFO:teuthology.orchestra.run.smithi186.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:14:39.830 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.830 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:14:39.830 INFO:teuthology.orchestra.run.smithi186.stdout: mon op complaint time = 120 2024-04-25T10:14:39.830 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:39.830 INFO:teuthology.orchestra.run.smithi186.stdout:[client] 2024-04-25T10:14:39.830 INFO:teuthology.orchestra.run.smithi186.stdout: rgw cache enabled = true 2024-04-25T10:14:39.830 INFO:teuthology.orchestra.run.smithi186.stdout: rgw enable ops log = true 2024-04-25T10:14:39.830 INFO:teuthology.orchestra.run.smithi186.stdout: rgw enable usage log = true 2024-04-25T10:14:39.830 INFO:teuthology.orchestra.run.smithi186.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:14:39.830 INFO:teuthology.orchestra.run.smithi186.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:14:39.831 INFO:teuthology.orchestra.run.smithi186.stdout: client mount timeout = 600 2024-04-25T10:14:39.831 INFO:teuthology.orchestra.run.smithi186.stdout: debug client = 20 2024-04-25T10:14:39.831 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:39.831 INFO:teuthology.orchestra.run.smithi186.stdout: pid file = "" 2024-04-25T10:14:39.831 INFO:teuthology.orchestra.run.smithi186.stdout: rados mon op timeout = 900 2024-04-25T10:14:39.831 INFO:teuthology.orchestra.run.smithi186.stdout: rados osd op timeout = 900 2024-04-25T10:14:39.831 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.a] 2024-04-25T10:14:39.831 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.b] 2024-04-25T10:14:39.831 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.c] 2024-04-25T10:14:39.831 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.a] 2024-04-25T10:14:39.831 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.c] 2024-04-25T10:14:39.831 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.b] 2024-04-25T10:14:39.831 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.d] 2024-04-25T10:14:39.831 INFO:teuthology.orchestra.run.smithi186.stdout:[mds] 2024-04-25T10:14:39.831 INFO:teuthology.orchestra.run.smithi186.stdout: debug mds = 20 2024-04-25T10:14:39.832 INFO:teuthology.orchestra.run.smithi186.stdout: debug mds balancer = 20 2024-04-25T10:14:39.832 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:39.832 INFO:teuthology.orchestra.run.smithi186.stdout: mds debug frag = True 2024-04-25T10:14:39.832 INFO:teuthology.orchestra.run.smithi186.stdout: mds debug scatterstat = False 2024-04-25T10:14:39.832 INFO:teuthology.orchestra.run.smithi186.stdout: mds op complaint time = 180 2024-04-25T10:14:39.832 INFO:teuthology.orchestra.run.smithi186.stdout: mds verify scatter = False 2024-04-25T10:14:39.832 INFO:teuthology.orchestra.run.smithi186.stdout: osd op complaint time = 180 2024-04-25T10:14:39.832 INFO:teuthology.orchestra.run.smithi186.stdout: rados mon op timeout = 900 2024-04-25T10:14:39.832 INFO:teuthology.orchestra.run.smithi186.stdout: rados osd op timeout = 900 2024-04-25T10:14:39.832 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-04-25T10:14:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.901+0000 7f848b11ef40 1 Processor -- start 2024-04-25T10:14:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.901+0000 7f848b11ef40 1 -- start start 2024-04-25T10:14:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.901+0000 7f848b11ef40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5a6b8c5d0 0x55c5a6c13d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.902+0000 7f848b11ef40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5a6ae31f0 0x55c5a6c14c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.902+0000 7f848b11ef40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c5a6c142c0 0x55c5a6c15190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.902+0000 7f848b11ef40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a6a5edf0 con 0x55c5a6b8c5d0 2024-04-25T10:14:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.902+0000 7f848b11ef40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a6a62d50 con 0x55c5a6ae31f0 2024-04-25T10:14:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.902+0000 7f848b11ef40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5a6b8d540 con 0x55c5a6c142c0 2024-04-25T10:14:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.902+0000 7f84881b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5a6b8c5d0 0x55c5a6c13d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.902+0000 7f84881b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5a6b8c5d0 0x55c5a6c13d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38104/0 (socket says 172.21.15.16:38104) 2024-04-25T10:14:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.902+0000 7f84881b9640 1 -- 172.21.15.16:0/1177418558 learned_addr learned my addr 172.21.15.16:0/1177418558 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.902+0000 7f84881b9640 1 -- 172.21.15.16:0/1177418558 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c5a6c142c0 msgr2=0x55c5a6c15190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.902+0000 7f84881b9640 1 --2- 172.21.15.16:0/1177418558 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c5a6c142c0 0x55c5a6c15190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.902+0000 7f84881b9640 1 -- 172.21.15.16:0/1177418558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5a6ae31f0 msgr2=0x55c5a6c14c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.902+0000 7f84879b8640 1 --2- 172.21.15.16:0/1177418558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5a6ae31f0 0x55c5a6c14c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.902+0000 7f84881b9640 1 --2- 172.21.15.16:0/1177418558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5a6ae31f0 0x55c5a6c14c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.902+0000 7f84881b9640 1 -- 172.21.15.16:0/1177418558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5a6b81ce0 con 0x55c5a6b8c5d0 2024-04-25T10:14:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.902+0000 7f84881b9640 1 --2- 172.21.15.16:0/1177418558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5a6b8c5d0 0x55c5a6c13d80 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f848000c6e0 tx=0x7f848000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c162a724620f1342 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.902+0000 7f84869b6640 1 -- 172.21.15.16:0/1177418558 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480015020 con 0x55c5a6b8c5d0 2024-04-25T10:14:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.902+0000 7f84869b6640 1 -- 172.21.15.16:0/1177418558 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848000cdf0 con 0x55c5a6b8c5d0 2024-04-25T10:14:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.902+0000 7f84869b6640 1 -- 172.21.15.16:0/1177418558 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848000ea30 con 0x55c5a6b8c5d0 2024-04-25T10:14:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.903+0000 7f848b11ef40 1 -- 172.21.15.16:0/1177418558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5a6b8c5d0 msgr2=0x55c5a6c13d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.903+0000 7f848b11ef40 1 --2- 172.21.15.16:0/1177418558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5a6b8c5d0 0x55c5a6c13d80 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f848000c6e0 tx=0x7f848000cba0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.903+0000 7f848b11ef40 1 -- 172.21.15.16:0/1177418558 shutdown_connections 2024-04-25T10:14:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.903+0000 7f848b11ef40 1 --2- 172.21.15.16:0/1177418558 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c5a6c142c0 0x55c5a6c15190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.903+0000 7f848b11ef40 1 --2- 172.21.15.16:0/1177418558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5a6ae31f0 0x55c5a6c14c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.903+0000 7f848b11ef40 1 --2- 172.21.15.16:0/1177418558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5a6b8c5d0 0x55c5a6c13d80 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.903+0000 7f848b11ef40 1 -- 172.21.15.16:0/1177418558 >> 172.21.15.16:0/1177418558 conn(0x55c5a6ae23d0 msgr2=0x55c5a6b00d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.903+0000 7f848b11ef40 1 -- 172.21.15.16:0/1177418558 shutdown_connections 2024-04-25T10:14:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.903+0000 7f848b11ef40 1 -- 172.21.15.16:0/1177418558 wait complete. 2024-04-25T10:14:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.903+0000 7f848b11ef40 1 Processor -- start 2024-04-25T10:14:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.903+0000 7f848b11ef40 1 -- start start 2024-04-25T10:14:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.904+0000 7f848b11ef40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5a6b8c5d0 0x55c5a6c18430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.904+0000 7f848b11ef40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5a6c142c0 0x55c5a6afc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.904+0000 7f848b11ef40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c5a6b6ec80 0x55c5a6ab88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.904+0000 7f848b11ef40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a6c156d0 con 0x55c5a6c142c0 2024-04-25T10:14:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.904+0000 7f848b11ef40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a6ae1fe0 con 0x55c5a6b8c5d0 2024-04-25T10:14:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.904+0000 7f848b11ef40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5a6ae1cb0 con 0x55c5a6b6ec80 2024-04-25T10:14:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.904+0000 7f84879b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5a6c142c0 0x55c5a6afc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.904+0000 7f84879b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5a6c142c0 0x55c5a6afc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38106/0 (socket says 172.21.15.16:38106) 2024-04-25T10:14:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.904+0000 7f84879b8640 1 -- 172.21.15.16:0/2071760406 learned_addr learned my addr 172.21.15.16:0/2071760406 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.904+0000 7f84881b9640 1 --2- 172.21.15.16:0/2071760406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5a6b8c5d0 0x55c5a6c18430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.904+0000 7f84889ba640 1 --2- 172.21.15.16:0/2071760406 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c5a6b6ec80 0x55c5a6ab88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.904+0000 7f84879b8640 1 -- 172.21.15.16:0/2071760406 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c5a6b6ec80 msgr2=0x55c5a6ab88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.904+0000 7f84879b8640 1 --2- 172.21.15.16:0/2071760406 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c5a6b6ec80 0x55c5a6ab88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.904+0000 7f84879b8640 1 -- 172.21.15.16:0/2071760406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5a6b8c5d0 msgr2=0x55c5a6c18430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.904+0000 7f84879b8640 1 --2- 172.21.15.16:0/2071760406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c5a6b8c5d0 0x55c5a6c18430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.904+0000 7f84879b8640 1 -- 172.21.15.16:0/2071760406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5a6c18a10 con 0x55c5a6c142c0 2024-04-25T10:14:39.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.905+0000 7f84879b8640 1 --2- 172.21.15.16:0/2071760406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5a6c142c0 0x55c5a6afc960 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f84700262c0 tx=0x55c5a6b000d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.905+0000 7f84859b4640 1 -- 172.21.15.16:0/2071760406 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470065070 con 0x55c5a6c142c0 2024-04-25T10:14:39.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.905+0000 7f84859b4640 1 -- 172.21.15.16:0/2071760406 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8470001470 con 0x55c5a6c142c0 2024-04-25T10:14:39.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.905+0000 7f848b11ef40 1 -- 172.21.15.16:0/2071760406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c5a6c18cf0 con 0x55c5a6c142c0 2024-04-25T10:14:39.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.905+0000 7f84859b4640 1 -- 172.21.15.16:0/2071760406 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470061650 con 0x55c5a6c142c0 2024-04-25T10:14:39.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.905+0000 7f848b11ef40 1 -- 172.21.15.16:0/2071760406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5a6c192b0 con 0x55c5a6c142c0 2024-04-25T10:14:39.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.906+0000 7f84859b4640 1 -- 172.21.15.16:0/2071760406 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8470068070 con 0x55c5a6c142c0 2024-04-25T10:14:39.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.907+0000 7f84859b4640 1 --2- 172.21.15.16:0/2071760406 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8468069ee0 0x7f846806c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.907+0000 7f84859b4640 1 -- 172.21.15.16:0/2071760406 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f84700cd790 con 0x55c5a6c142c0 2024-04-25T10:14:39.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.907+0000 7f84881b9640 1 --2- 172.21.15.16:0/2071760406 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8468069ee0 0x7f846806c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.907+0000 7f848b11ef40 1 --2- 172.21.15.16:0/2071760406 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x55c5a6b6f090 0x55c5a6afbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.907+0000 7f848b11ef40 1 -- 172.21.15.16:0/2071760406 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v9 -- 0x55c5a6afc210 con 0x55c5a6b6f090 2024-04-25T10:14:39.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.907+0000 7f84889ba640 1 --2- 172.21.15.16:0/2071760406 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x55c5a6b6f090 0x55c5a6afbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.907+0000 7f84881b9640 1 --2- 172.21.15.16:0/2071760406 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8468069ee0 0x7f846806c3a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8480014040 tx=0x7f8480009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.908+0000 7f84889ba640 1 --2- 172.21.15.16:0/2071760406 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x55c5a6b6f090 0x55c5a6afbcd0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.909+0000 7f84889ba640 1 -- 172.21.15.16:0/2071760406 <== osd.4 v2:172.21.15.186:6824/3091472889 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v116'5 uv5 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8478011750 con 0x55c5a6b6f090 2024-04-25T10:14:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.909+0000 7f848b11ef40 1 -- 172.21.15.16:0/2071760406 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x55c5a6b6f090 msgr2=0x55c5a6afbcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.909+0000 7f848b11ef40 1 --2- 172.21.15.16:0/2071760406 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x55c5a6b6f090 0x55c5a6afbcd0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.909+0000 7f848b11ef40 1 -- 172.21.15.16:0/2071760406 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8468069ee0 msgr2=0x7f846806c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.909+0000 7f848b11ef40 1 --2- 172.21.15.16:0/2071760406 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8468069ee0 0x7f846806c3a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8480014040 tx=0x7f8480009000 comp rx=0 tx=0).stop 2024-04-25T10:14:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.909+0000 7f848b11ef40 1 -- 172.21.15.16:0/2071760406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5a6c142c0 msgr2=0x55c5a6afc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.909+0000 7f848b11ef40 1 --2- 172.21.15.16:0/2071760406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5a6c142c0 0x55c5a6afc960 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f84700262c0 tx=0x55c5a6b000d0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.909+0000 7f84889ba640 1 -- 172.21.15.16:0/2071760406 reap_dead start 2024-04-25T10:14:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.909+0000 7f848b11ef40 1 -- 172.21.15.16:0/2071760406 shutdown_connections 2024-04-25T10:14:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.909+0000 7f848b11ef40 1 -- 172.21.15.16:0/2071760406 >> 172.21.15.16:0/2071760406 conn(0x55c5a6ae23d0 msgr2=0x55c5a6ae1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.909+0000 7f848b11ef40 1 -- 172.21.15.16:0/2071760406 shutdown_connections 2024-04-25T10:14:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:39.909+0000 7f848b11ef40 1 -- 172.21.15.16:0/2071760406 wait complete. 2024-04-25T10:14:39.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:14:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.056+0000 7f34f2bd5640 1 Processor -- start 2024-04-25T10:14:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.056+0000 7f34f2bd5640 1 -- start start 2024-04-25T10:14:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.057+0000 7f34f2bd5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f34ec15dc80 0x7f34ec161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.057+0000 7f34f2bd5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34ec0b5c00 0x7f34ec0d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.057+0000 7f34f2bd5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ec1627a0 con 0x7f34ec0b97c0 2024-04-25T10:14:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.057+0000 7f34f2bd5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ec1621e0 con 0x7f34ec15dc80 2024-04-25T10:14:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.057+0000 7f34f2bd5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec1624c0 con 0x7f34ec0b5c00 2024-04-25T10:14:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.057+0000 7f34ebfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f34ec0b97c0 0x7f34ec0b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45246/0 (socket says 172.21.15.16:45246) 2024-04-25T10:14:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.057+0000 7f34ebfff640 1 -- 172.21.15.16:0/155833089 learned_addr learned my addr 172.21.15.16:0/155833089 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.058+0000 7f34f1172640 1 --2- 172.21.15.16:0/155833089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34ec0b5c00 0x7f34ec0d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.058+0000 7f34f0971640 1 --2- 172.21.15.16:0/155833089 >> v2:172.21.15.186:3300/0 conn(0x7f34ec15dc80 0x7f34ec161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.058+0000 7f34eb7fe640 1 -- 172.21.15.16:0/155833089 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580627401 0 0) 0x7f34ec1627a0 con 0x7f34ec0b97c0 2024-04-25T10:14:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.058+0000 7f34eb7fe640 1 -- 172.21.15.16:0/155833089 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34d8003520 con 0x7f34ec0b97c0 2024-04-25T10:14:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.058+0000 7f34f1172640 1 -- 172.21.15.16:0/155833089 >> v2:172.21.15.186:3300/0 conn(0x7f34ec15dc80 msgr2=0x7f34ec161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.058+0000 7f34f1172640 1 --2- 172.21.15.16:0/155833089 >> v2:172.21.15.186:3300/0 conn(0x7f34ec15dc80 0x7f34ec161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.058+0000 7f34f1172640 1 -- 172.21.15.16:0/155833089 >> v1:172.21.15.16:6789/0 conn(0x7f34ec0b97c0 legacy=0x7f34ec0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.058+0000 7f34f1172640 1 -- 172.21.15.16:0/155833089 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec0d1c70 con 0x7f34ec0b5c00 2024-04-25T10:14:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.058+0000 7f34f0971640 1 --2- 172.21.15.16:0/155833089 >> v2:172.21.15.186:3300/0 conn(0x7f34ec15dc80 0x7f34ec161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.059+0000 7f34f1172640 1 --2- 172.21.15.16:0/155833089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34ec0b5c00 0x7f34ec0d2470 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f34d40097a0 tx=0x7f34d402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66b3931b94314a0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.059+0000 7f34eb7fe640 1 -- 172.21.15.16:0/155833089 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d4037070 con 0x7f34ec0b5c00 2024-04-25T10:14:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.059+0000 7f34eb7fe640 1 -- 172.21.15.16:0/155833089 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d4032440 con 0x7f34ec0b5c00 2024-04-25T10:14:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.059+0000 7f34eb7fe640 1 -- 172.21.15.16:0/155833089 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d4032810 con 0x7f34ec0b5c00 2024-04-25T10:14:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.059+0000 7f34f2bd5640 1 -- 172.21.15.16:0/155833089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34ec0b5c00 msgr2=0x7f34ec0d2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.059+0000 7f34f2bd5640 1 --2- 172.21.15.16:0/155833089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34ec0b5c00 0x7f34ec0d2470 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f34d40097a0 tx=0x7f34d402a180 comp rx=0 tx=0).stop 2024-04-25T10:14:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.059+0000 7f34f2bd5640 1 -- 172.21.15.16:0/155833089 shutdown_connections 2024-04-25T10:14:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.059+0000 7f34f2bd5640 1 --2- 172.21.15.16:0/155833089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34ec0b5c00 0x7f34ec0d2470 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.059+0000 7f34f2bd5640 1 --2- 172.21.15.16:0/155833089 >> v2:172.21.15.186:3300/0 conn(0x7f34ec15dc80 0x7f34ec161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.059+0000 7f34f2bd5640 1 -- 172.21.15.16:0/155833089 >> 172.21.15.16:0/155833089 conn(0x7f34ec0d04f0 msgr2=0x7f34ec0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:40.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.059+0000 7f34f2bd5640 1 -- 172.21.15.16:0/155833089 shutdown_connections 2024-04-25T10:14:40.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.059+0000 7f34f2bd5640 1 -- 172.21.15.16:0/155833089 wait complete. 2024-04-25T10:14:40.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.060+0000 7f34f2bd5640 1 Processor -- start 2024-04-25T10:14:40.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.060+0000 7f34f2bd5640 1 -- start start 2024-04-25T10:14:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.061+0000 7f34f2bd5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34ec0b5c00 0x7f34ec0c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.061+0000 7f34f2bd5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34ec0b97c0 0x7f34ec0c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.061+0000 7f34f2bd5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34ec15dc80 0x7f34ec14e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.061+0000 7f34f2bd5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec161f30 con 0x7f34ec0b97c0 2024-04-25T10:14:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.061+0000 7f34f2bd5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec161db0 con 0x7f34ec15dc80 2024-04-25T10:14:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.061+0000 7f34f2bd5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec0d29b0 con 0x7f34ec0b5c00 2024-04-25T10:14:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.061+0000 7f34ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34ec0b97c0 0x7f34ec0c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.062+0000 7f34ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34ec0b97c0 0x7f34ec0c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38114/0 (socket says 172.21.15.16:38114) 2024-04-25T10:14:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.062+0000 7f34ebfff640 1 -- 172.21.15.16:0/3435164459 learned_addr learned my addr 172.21.15.16:0/3435164459 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.062+0000 7f34f0971640 1 --2- 172.21.15.16:0/3435164459 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34ec0b5c00 0x7f34ec0c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.062+0000 7f34f1172640 1 --2- 172.21.15.16:0/3435164459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34ec15dc80 0x7f34ec14e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.062+0000 7f34ebfff640 1 -- 172.21.15.16:0/3435164459 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34ec0b5c00 msgr2=0x7f34ec0c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.062+0000 7f34ebfff640 1 --2- 172.21.15.16:0/3435164459 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34ec0b5c00 0x7f34ec0c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.062+0000 7f34ebfff640 1 -- 172.21.15.16:0/3435164459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34ec15dc80 msgr2=0x7f34ec14e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.062+0000 7f34ebfff640 1 --2- 172.21.15.16:0/3435164459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34ec15dc80 0x7f34ec14e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.062+0000 7f34ebfff640 1 -- 172.21.15.16:0/3435164459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec15e6d0 con 0x7f34ec0b97c0 2024-04-25T10:14:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.062+0000 7f34f0971640 1 --2- 172.21.15.16:0/3435164459 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34ec0b5c00 0x7f34ec0c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.062+0000 7f34ebfff640 1 --2- 172.21.15.16:0/3435164459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34ec0b97c0 0x7f34ec0c95b0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00cb70 tx=0x7f34dc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.062+0000 7f34e97fa640 1 -- 172.21.15.16:0/3435164459 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc017070 con 0x7f34ec0b97c0 2024-04-25T10:14:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.062+0000 7f34e97fa640 1 -- 172.21.15.16:0/3435164459 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34dc00ad40 con 0x7f34ec0b97c0 2024-04-25T10:14:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.062+0000 7f34f2bd5640 1 -- 172.21.15.16:0/3435164459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ec0c9af0 con 0x7f34ec0b97c0 2024-04-25T10:14:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.062+0000 7f34e97fa640 1 -- 172.21.15.16:0/3435164459 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc012620 con 0x7f34ec0b97c0 2024-04-25T10:14:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.062+0000 7f34f2bd5640 1 -- 172.21.15.16:0/3435164459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ec0c9dc0 con 0x7f34ec0b97c0 2024-04-25T10:14:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.063+0000 7f34f2bd5640 1 -- 172.21.15.16:0/3435164459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ec0750b0 con 0x7f34ec0b97c0 2024-04-25T10:14:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.064+0000 7f34e97fa640 1 -- 172.21.15.16:0/3435164459 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34dc01a070 con 0x7f34ec0b97c0 2024-04-25T10:14:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.064+0000 7f34e97fa640 1 --2- 172.21.15.16:0/3435164459 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f34cc06b4d0 0x7f34cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.064+0000 7f34e97fa640 1 -- 172.21.15.16:0/3435164459 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f34dc089f90 con 0x7f34ec0b97c0 2024-04-25T10:14:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.064+0000 7f34f0971640 1 --2- 172.21.15.16:0/3435164459 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f34cc06b4d0 0x7f34cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.065+0000 7f34f0971640 1 --2- 172.21.15.16:0/3435164459 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f34cc06b4d0 0x7f34cc06d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009560 tx=0x7f34e4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:40.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.068+0000 7f34e97fa640 1 -- 172.21.15.16:0/3435164459 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34ec15e6d0 con 0x7f34ec0b97c0 2024-04-25T10:14:40.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.294+0000 7f34f2bd5640 1 -- 172.21.15.16:0/3435164459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f34ec0d0d30 con 0x7f34ec0b97c0 2024-04-25T10:14:40.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.488+0000 7f34e97fa640 1 -- 172.21.15.16:0/3435164459 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v86) v1 ==== 162+0+0 (secure 0 0 0) 0x7f34dc0587f0 con 0x7f34ec0b97c0 2024-04-25T10:14:40.492 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:14:40.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.490+0000 7f34f2bd5640 1 -- 172.21.15.16:0/3435164459 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f34cc06b4d0 msgr2=0x7f34cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.490+0000 7f34f2bd5640 1 --2- 172.21.15.16:0/3435164459 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f34cc06b4d0 0x7f34cc06d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009560 tx=0x7f34e4009190 comp rx=0 tx=0).stop 2024-04-25T10:14:40.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.490+0000 7f34f2bd5640 1 -- 172.21.15.16:0/3435164459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34ec0b97c0 msgr2=0x7f34ec0c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.490+0000 7f34f2bd5640 1 --2- 172.21.15.16:0/3435164459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34ec0b97c0 0x7f34ec0c95b0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00cb70 tx=0x7f34dc00a430 comp rx=0 tx=0).stop 2024-04-25T10:14:40.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.490+0000 7f34f2bd5640 1 -- 172.21.15.16:0/3435164459 shutdown_connections 2024-04-25T10:14:40.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.490+0000 7f34f2bd5640 1 --2- 172.21.15.16:0/3435164459 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f34cc06b4d0 0x7f34cc06d990 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.490+0000 7f34f2bd5640 1 --2- 172.21.15.16:0/3435164459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34ec15dc80 0x7f34ec14e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.490+0000 7f34f2bd5640 1 --2- 172.21.15.16:0/3435164459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34ec0b97c0 0x7f34ec0c95b0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.490+0000 7f34f2bd5640 1 --2- 172.21.15.16:0/3435164459 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34ec0b5c00 0x7f34ec0c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.490+0000 7f34f2bd5640 1 -- 172.21.15.16:0/3435164459 >> 172.21.15.16:0/3435164459 conn(0x7f34ec0d04f0 msgr2=0x7f34ec0ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:40.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.491+0000 7f34f2bd5640 1 -- 172.21.15.16:0/3435164459 shutdown_connections 2024-04-25T10:14:40.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.491+0000 7f34f2bd5640 1 -- 172.21.15.16:0/3435164459 wait complete. 2024-04-25T10:14:40.507 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:40.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.604+0000 7fbbda22a640 1 Processor -- start 2024-04-25T10:14:40.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.604+0000 7fbbda22a640 1 -- start start 2024-04-25T10:14:40.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.605+0000 7fbbda22a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbbd409a150 0x7fbbd406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.605+0000 7fbbda22a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd406c660 con 0x7fbbd406c1f0 2024-04-25T10:14:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.605+0000 7fbbda22a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd406c900 con 0x7fbbd415ba10 2024-04-25T10:14:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.605+0000 7fbbda22a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbd40cbe50 con 0x7fbbd409a150 2024-04-25T10:14:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.605+0000 7fbbd3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbbd406c1f0 0x7fbbd40cd220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45254/0 (socket says 172.21.15.16:45254) 2024-04-25T10:14:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.605+0000 7fbbd3fff640 1 -- 172.21.15.16:0/2340069856 learned_addr learned my addr 172.21.15.16:0/2340069856 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.605+0000 7fbbd2ffd640 1 --2- 172.21.15.16:0/2340069856 >> v2:172.21.15.186:3300/0 conn(0x7fbbd409a150 0x7fbbd406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.606+0000 7fbbd27fc640 1 -- 172.21.15.16:0/2340069856 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 552639037 0 0) 0x7fbbd406c660 con 0x7fbbd406c1f0 2024-04-25T10:14:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.606+0000 7fbbd27fc640 1 -- 172.21.15.16:0/2340069856 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbbc003520 con 0x7fbbd406c1f0 2024-04-25T10:14:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.606+0000 7fbbd27fc640 1 -- 172.21.15.16:0/2340069856 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3652058307 0 0) 0x7fbbd406c900 con 0x7fbbd415ba10 2024-04-25T10:14:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.606+0000 7fbbd27fc640 1 -- 172.21.15.16:0/2340069856 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbd406c660 con 0x7fbbd415ba10 2024-04-25T10:14:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.606+0000 7fbbd2ffd640 1 -- 172.21.15.16:0/2340069856 >> v1:172.21.15.186:6789/0 conn(0x7fbbd415ba10 legacy=0x7fbbd4099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.606+0000 7fbbd2ffd640 1 -- 172.21.15.16:0/2340069856 >> v1:172.21.15.16:6789/0 conn(0x7fbbd406c1f0 legacy=0x7fbbd40cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.606+0000 7fbbd2ffd640 1 -- 172.21.15.16:0/2340069856 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd406cbe0 con 0x7fbbd409a150 2024-04-25T10:14:40.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.607+0000 7fbbd2ffd640 1 --2- 172.21.15.16:0/2340069856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd409a150 0x7fbbd406bcb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80097a0 tx=0x7fbbc802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e37f651bf3b457b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:40.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.607+0000 7fbbd27fc640 1 -- 172.21.15.16:0/2340069856 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8037070 con 0x7fbbd409a150 2024-04-25T10:14:40.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.607+0000 7fbbd27fc640 1 -- 172.21.15.16:0/2340069856 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc8032440 con 0x7fbbd409a150 2024-04-25T10:14:40.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.607+0000 7fbbd27fc640 1 -- 172.21.15.16:0/2340069856 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc80327f0 con 0x7fbbd409a150 2024-04-25T10:14:40.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.607+0000 7fbbda22a640 1 -- 172.21.15.16:0/2340069856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd409a150 msgr2=0x7fbbd406bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.607+0000 7fbbda22a640 1 --2- 172.21.15.16:0/2340069856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd409a150 0x7fbbd406bcb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80097a0 tx=0x7fbbc802a230 comp rx=0 tx=0).stop 2024-04-25T10:14:40.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.607+0000 7fbbda22a640 1 -- 172.21.15.16:0/2340069856 shutdown_connections 2024-04-25T10:14:40.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.607+0000 7fbbda22a640 1 --2- 172.21.15.16:0/2340069856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd409a150 0x7fbbd406bcb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.607+0000 7fbbda22a640 1 -- 172.21.15.16:0/2340069856 >> 172.21.15.16:0/2340069856 conn(0x7fbbd40b32f0 msgr2=0x7fbbd40b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:40.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.607+0000 7fbbda22a640 1 -- 172.21.15.16:0/2340069856 shutdown_connections 2024-04-25T10:14:40.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.607+0000 7fbbda22a640 1 -- 172.21.15.16:0/2340069856 wait complete. 2024-04-25T10:14:40.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.608+0000 7fbbda22a640 1 Processor -- start 2024-04-25T10:14:40.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.609+0000 7fbbda22a640 1 -- start start 2024-04-25T10:14:40.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.609+0000 7fbbda22a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbd406c1f0 0x7fbbd40c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.609+0000 7fbbda22a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbd409a150 0x7fbbd4159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.609+0000 7fbbda22a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd415ba10 0x7fbbd40b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.609+0000 7fbbda22a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd40d1320 con 0x7fbbd406c1f0 2024-04-25T10:14:40.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.609+0000 7fbbda22a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd40cfe80 con 0x7fbbd415ba10 2024-04-25T10:14:40.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.609+0000 7fbbda22a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd40d14a0 con 0x7fbbd409a150 2024-04-25T10:14:40.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.609+0000 7fbbd37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbd406c1f0 0x7fbbd40c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.610+0000 7fbbd37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbd406c1f0 0x7fbbd40c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38120/0 (socket says 172.21.15.16:38120) 2024-04-25T10:14:40.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.610+0000 7fbbd37fe640 1 -- 172.21.15.16:0/4178384258 learned_addr learned my addr 172.21.15.16:0/4178384258 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:40.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.610+0000 7fbbd3fff640 1 --2- 172.21.15.16:0/4178384258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd415ba10 0x7fbbd40b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.610+0000 7fbbd37fe640 1 -- 172.21.15.16:0/4178384258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbd409a150 msgr2=0x7fbbd4159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.610+0000 7fbbd37fe640 1 --2- 172.21.15.16:0/4178384258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbd409a150 0x7fbbd4159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.610+0000 7fbbd37fe640 1 -- 172.21.15.16:0/4178384258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd415ba10 msgr2=0x7fbbd40b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.610+0000 7fbbd37fe640 1 --2- 172.21.15.16:0/4178384258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd415ba10 0x7fbbd40b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.610+0000 7fbbd37fe640 1 -- 172.21.15.16:0/4178384258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd40b4e00 con 0x7fbbd406c1f0 2024-04-25T10:14:40.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.610+0000 7fbbd37fe640 1 --2- 172.21.15.16:0/4178384258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbd406c1f0 0x7fbbd40c5330 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400c500 tx=0x7fbbc400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:40.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.610+0000 7fbbbbfff640 1 -- 172.21.15.16:0/4178384258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc40052d0 con 0x7fbbd406c1f0 2024-04-25T10:14:40.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.610+0000 7fbbbbfff640 1 -- 172.21.15.16:0/4178384258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc4005490 con 0x7fbbd406c1f0 2024-04-25T10:14:40.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.610+0000 7fbbda22a640 1 -- 172.21.15.16:0/4178384258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd40cc950 con 0x7fbbd406c1f0 2024-04-25T10:14:40.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.610+0000 7fbbbbfff640 1 -- 172.21.15.16:0/4178384258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4012870 con 0x7fbbd406c1f0 2024-04-25T10:14:40.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.610+0000 7fbbda22a640 1 -- 172.21.15.16:0/4178384258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd40b50e0 con 0x7fbbd406c1f0 2024-04-25T10:14:40.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.611+0000 7fbbda22a640 1 -- 172.21.15.16:0/4178384258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd40749b0 con 0x7fbbd406c1f0 2024-04-25T10:14:40.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.612+0000 7fbbbbfff640 1 -- 172.21.15.16:0/4178384258 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbbc4012a10 con 0x7fbbd406c1f0 2024-04-25T10:14:40.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.612+0000 7fbbbbfff640 1 --2- 172.21.15.16:0/4178384258 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbbc006b4d0 0x7fbbc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.612+0000 7fbbbbfff640 1 -- 172.21.15.16:0/4178384258 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbbc4089eb0 con 0x7fbbd406c1f0 2024-04-25T10:14:40.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.612+0000 7fbbd2ffd640 1 --2- 172.21.15.16:0/4178384258 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbbc006b4d0 0x7fbbc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.613+0000 7fbbd2ffd640 1 --2- 172.21.15.16:0/4178384258 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbbc006b4d0 0x7fbbc006d990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8034000 tx=0x7fbbc802c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:40.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.616+0000 7fbbbbfff640 1 -- 172.21.15.16:0/4178384258 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbbc4058ce0 con 0x7fbbd406c1f0 2024-04-25T10:14:40.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.836+0000 7fbbda22a640 1 -- 172.21.15.16:0/4178384258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbd409a570 con 0x7fbbd406c1f0 2024-04-25T10:14:40.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.837+0000 7fbbbbfff640 1 -- 172.21.15.16:0/4178384258 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fbbc40585e0 con 0x7fbbd406c1f0 2024-04-25T10:14:40.841 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:40.841 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2871061122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2871061122},{"type":"v1","addr":"172.21.15.186:6836","nonce":2871061122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1871507206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1871507206},{"type":"v1","addr":"172.21.15.186:6837","nonce":1871507206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4064412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4064412586},{"type":"v1","addr":"172.21.15.16:6834","nonce":4064412586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:26.751060+0000","modified":"2024-04-25T10:14:40.487689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9096":{"gid":9096,"name":"c","rank":0,"incarnation":86,"state":"up:replay","state_seq":1,"addr":"172.21.15.16:6835/2697110279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2697110279},{"type":"v1","addr":"172.21.15.16:6835","nonce":2697110279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:40.842 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 86 2024-04-25T10:14:40.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.839+0000 7fbbda22a640 1 -- 172.21.15.16:0/4178384258 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbbc006b4d0 msgr2=0x7fbbc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.839+0000 7fbbda22a640 1 --2- 172.21.15.16:0/4178384258 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbbc006b4d0 0x7fbbc006d990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8034000 tx=0x7fbbc802c000 comp rx=0 tx=0).stop 2024-04-25T10:14:40.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.839+0000 7fbbda22a640 1 -- 172.21.15.16:0/4178384258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbd406c1f0 msgr2=0x7fbbd40c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.839+0000 7fbbda22a640 1 --2- 172.21.15.16:0/4178384258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbd406c1f0 0x7fbbd40c5330 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400c500 tx=0x7fbbc400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.839+0000 7fbbda22a640 1 -- 172.21.15.16:0/4178384258 shutdown_connections 2024-04-25T10:14:40.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.839+0000 7fbbda22a640 1 --2- 172.21.15.16:0/4178384258 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbbc006b4d0 0x7fbbc006d990 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.839+0000 7fbbda22a640 1 --2- 172.21.15.16:0/4178384258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd415ba10 0x7fbbd40b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.839+0000 7fbbda22a640 1 --2- 172.21.15.16:0/4178384258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbd409a150 0x7fbbd4159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.839+0000 7fbbda22a640 1 --2- 172.21.15.16:0/4178384258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbd406c1f0 0x7fbbd40c5330 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.839+0000 7fbbda22a640 1 -- 172.21.15.16:0/4178384258 >> 172.21.15.16:0/4178384258 conn(0x7fbbd40b32f0 msgr2=0x7fbbd40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:40.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.839+0000 7fbbda22a640 1 -- 172.21.15.16:0/4178384258 shutdown_connections 2024-04-25T10:14:40.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:40.839+0000 7fbbda22a640 1 -- 172.21.15.16:0/4178384258 wait complete. 2024-04-25T10:14:41.257 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:14:26.751060+0000', 'modified': '2024-04-25T10:14:40.487689+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 116, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9096}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9096': {'gid': 9096, 'name': 'c', 'rank': 0, 'incarnation': 86, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.16:6835/2697110279', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 2697110279}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 2697110279}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:14:41.340 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9096:up:replay 2024-04-25T10:14:42.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.452+0000 7f65cf7cf640 1 Processor -- start 2024-04-25T10:14:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.453+0000 7f65cf7cf640 1 -- start start 2024-04-25T10:14:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.453+0000 7f65cf7cf640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65c8144450 0x7f65c80c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.453+0000 7f65cf7cf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c80c75c0 con 0x7f65c815eaa0 2024-04-25T10:14:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.453+0000 7f65cf7cf640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c80cd2c0 con 0x7f65c80ccdc0 2024-04-25T10:14:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.453+0000 7f65cf7cf640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80cdb60 con 0x7f65c8144450 2024-04-25T10:14:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65cdd6c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f65c815eaa0 0x7f65c80af6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45266/0 (socket says 172.21.15.16:45266) 2024-04-25T10:14:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65cdd6c640 1 -- 172.21.15.16:0/141982193 learned_addr learned my addr 172.21.15.16:0/141982193 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65cd56b640 1 --2- 172.21.15.16:0/141982193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65c8144450 0x7f65c80c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65bbfff640 1 -- 172.21.15.16:0/141982193 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332887422 0 0) 0x7f65c80c75c0 con 0x7f65c815eaa0 2024-04-25T10:14:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65bbfff640 1 -- 172.21.15.16:0/141982193 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65b4003520 con 0x7f65c815eaa0 2024-04-25T10:14:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65bbfff640 1 -- 172.21.15.16:0/141982193 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1265631356 0 0) 0x7f65b4003520 con 0x7f65c815eaa0 2024-04-25T10:14:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65bbfff640 1 -- 172.21.15.16:0/141982193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65c8144450 msgr2=0x7f65c80c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65bbfff640 1 --2- 172.21.15.16:0/141982193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65c8144450 0x7f65c80c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65bbfff640 1 -- 172.21.15.16:0/141982193 >> v1:172.21.15.186:6789/0 conn(0x7f65c80ccdc0 legacy=0x7f65c80722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65bbfff640 1 -- 172.21.15.16:0/141982193 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80cd5a0 con 0x7f65c815eaa0 2024-04-25T10:14:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65bbfff640 1 -- 172.21.15.16:0/141982193 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f65c00019f0 con 0x7f65c815eaa0 2024-04-25T10:14:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65bbfff640 1 -- 172.21.15.16:0/141982193 >> v1:172.21.15.16:6789/0 conn(0x7f65c815eaa0 legacy=0x7f65c80af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65bbfff640 1 --2- 172.21.15.16:0/141982193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65b40039e0 0x7f65b4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65bbfff640 1 --2- 172.21.15.16:0/141982193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65b4004390 0x7f65b4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65bbfff640 1 --2- 172.21.15.16:0/141982193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65c8144450 0x7f65b4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65bbfff640 1 -- 172.21.15.16:0/141982193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b400ba70 con 0x7f65b40039e0 2024-04-25T10:14:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65bbfff640 1 -- 172.21.15.16:0/141982193 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b400bbf0 con 0x7f65b4004390 2024-04-25T10:14:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65bbfff640 1 -- 172.21.15.16:0/141982193 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b400bdf0 con 0x7f65c8144450 2024-04-25T10:14:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65cd56b640 1 --2- 172.21.15.16:0/141982193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65b40039e0 0x7f65b4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.454+0000 7f65ccd6a640 1 --2- 172.21.15.16:0/141982193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65b4004390 0x7f65b4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.455+0000 7f65cdd6c640 1 --2- 172.21.15.16:0/141982193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65c8144450 0x7f65b4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.455+0000 7f65cd56b640 1 -- 172.21.15.16:0/141982193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65c8144450 msgr2=0x7f65b4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.455+0000 7f65cd56b640 1 --2- 172.21.15.16:0/141982193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65c8144450 0x7f65b4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.455+0000 7f65cd56b640 1 -- 172.21.15.16:0/141982193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65b4004390 msgr2=0x7f65b4004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.455+0000 7f65cd56b640 1 --2- 172.21.15.16:0/141982193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65b4004390 0x7f65b4004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.455+0000 7f65cd56b640 1 -- 172.21.15.16:0/141982193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b4003520 con 0x7f65b40039e0 2024-04-25T10:14:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.455+0000 7f65cdd6c640 1 --2- 172.21.15.16:0/141982193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65c8144450 0x7f65b4005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.455+0000 7f65cd56b640 1 --2- 172.21.15.16:0/141982193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65b40039e0 0x7f65b4003dc0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009ac0 tx=0x7f65b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dba393f4345cb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.456+0000 7f65bbfff640 1 -- 172.21.15.16:0/141982193 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc036070 con 0x7f65b40039e0 2024-04-25T10:14:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.456+0000 7f65bbfff640 1 -- 172.21.15.16:0/141982193 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65bc031400 con 0x7f65b40039e0 2024-04-25T10:14:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.456+0000 7f65bbfff640 1 -- 172.21.15.16:0/141982193 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc009690 con 0x7f65b40039e0 2024-04-25T10:14:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.456+0000 7f65cf7cf640 1 -- 172.21.15.16:0/141982193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65b40039e0 msgr2=0x7f65b4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.456+0000 7f65cf7cf640 1 --2- 172.21.15.16:0/141982193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65b40039e0 0x7f65b4003dc0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009ac0 tx=0x7f65b4005830 comp rx=0 tx=0).stop 2024-04-25T10:14:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.456+0000 7f65cdd6c640 1 -- 172.21.15.16:0/141982193 reap_dead start 2024-04-25T10:14:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.456+0000 7f65cf7cf640 1 -- 172.21.15.16:0/141982193 shutdown_connections 2024-04-25T10:14:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.456+0000 7f65cf7cf640 1 -- 172.21.15.16:0/141982193 >> 172.21.15.16:0/141982193 conn(0x7f65c80b2370 msgr2=0x7f65c80b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.456+0000 7f65cf7cf640 1 -- 172.21.15.16:0/141982193 shutdown_connections 2024-04-25T10:14:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.456+0000 7f65cf7cf640 1 -- 172.21.15.16:0/141982193 wait complete. 2024-04-25T10:14:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.457+0000 7f65cf7cf640 1 Processor -- start 2024-04-25T10:14:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.458+0000 7f65cf7cf640 1 -- start start 2024-04-25T10:14:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.458+0000 7f65cf7cf640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65b40039e0 0x7f65c80b6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.458+0000 7f65cf7cf640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65c80c7c10 0x7f65c80c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.458+0000 7f65cf7cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c8153070 0x7f65c81404c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.458+0000 7f65cf7cf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80729b0 con 0x7f65c8153070 2024-04-25T10:14:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.458+0000 7f65cf7cf640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c815e580 con 0x7f65c80c7c10 2024-04-25T10:14:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.458+0000 7f65cf7cf640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c8072b30 con 0x7f65b40039e0 2024-04-25T10:14:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.458+0000 7f65cdd6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c8153070 0x7f65c81404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.458+0000 7f65cdd6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c8153070 0x7f65c81404c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38124/0 (socket says 172.21.15.16:38124) 2024-04-25T10:14:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.458+0000 7f65cdd6c640 1 -- 172.21.15.16:0/1857258112 learned_addr learned my addr 172.21.15.16:0/1857258112 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.459+0000 7f65ccd6a640 1 --2- 172.21.15.16:0/1857258112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65c80c7c10 0x7f65c80c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.459+0000 7f65cd56b640 1 --2- 172.21.15.16:0/1857258112 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65b40039e0 0x7f65c80b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.459+0000 7f65cdd6c640 1 -- 172.21.15.16:0/1857258112 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65b40039e0 msgr2=0x7f65c80b6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.459+0000 7f65cdd6c640 1 --2- 172.21.15.16:0/1857258112 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65b40039e0 0x7f65c80b6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.459+0000 7f65cdd6c640 1 -- 172.21.15.16:0/1857258112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65c80c7c10 msgr2=0x7f65c80c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.459+0000 7f65cdd6c640 1 --2- 172.21.15.16:0/1857258112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65c80c7c10 0x7f65c80c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.459+0000 7f65cdd6c640 1 -- 172.21.15.16:0/1857258112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80c8090 con 0x7f65c8153070 2024-04-25T10:14:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.459+0000 7f65ccd6a640 1 --2- 172.21.15.16:0/1857258112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65c80c7c10 0x7f65c80c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.459+0000 7f65cd56b640 1 --2- 172.21.15.16:0/1857258112 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65b40039e0 0x7f65c80b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.459+0000 7f65cdd6c640 1 --2- 172.21.15.16:0/1857258112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c8153070 0x7f65c81404c0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f65c8073410 tx=0x7f65c000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.459+0000 7f65b9ffb640 1 -- 172.21.15.16:0/1857258112 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c000eb90 con 0x7f65c8153070 2024-04-25T10:14:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.459+0000 7f65b9ffb640 1 -- 172.21.15.16:0/1857258112 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c000ed30 con 0x7f65c8153070 2024-04-25T10:14:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.459+0000 7f65cf7cf640 1 -- 172.21.15.16:0/1857258112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c80c8310 con 0x7f65c8153070 2024-04-25T10:14:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.459+0000 7f65b9ffb640 1 -- 172.21.15.16:0/1857258112 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0014670 con 0x7f65c8153070 2024-04-25T10:14:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.459+0000 7f65cf7cf640 1 -- 172.21.15.16:0/1857258112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c80c88d0 con 0x7f65c8153070 2024-04-25T10:14:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.460+0000 7f65cf7cf640 1 -- 172.21.15.16:0/1857258112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c80c8ee0 con 0x7f65c8153070 2024-04-25T10:14:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.461+0000 7f65b9ffb640 1 -- 172.21.15.16:0/1857258112 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65c0014810 con 0x7f65c8153070 2024-04-25T10:14:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.461+0000 7f65b9ffb640 1 --2- 172.21.15.16:0/1857258112 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f65ac06b4d0 0x7f65ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.461+0000 7f65b9ffb640 1 -- 172.21.15.16:0/1857258112 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f65c008a0f0 con 0x7f65c8153070 2024-04-25T10:14:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.461+0000 7f65cd56b640 1 --2- 172.21.15.16:0/1857258112 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f65ac06b4d0 0x7f65ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.462+0000 7f65cd56b640 1 --2- 172.21.15.16:0/1857258112 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f65ac06b4d0 0x7f65ac06d990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f65bc033000 tx=0x7f65bc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.465+0000 7f65b9ffb640 1 -- 172.21.15.16:0/1857258112 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65c80c8090 con 0x7f65c8153070 2024-04-25T10:14:42.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.683+0000 7f65cf7cf640 1 -- 172.21.15.16:0/1857258112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65c80749b0 con 0x7f65c8153070 2024-04-25T10:14:42.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.684+0000 7f65b9ffb640 1 -- 172.21.15.16:0/1857258112 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f65c0058f10 con 0x7f65c8153070 2024-04-25T10:14:42.688 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:42.688 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2871061122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2871061122},{"type":"v1","addr":"172.21.15.186:6836","nonce":2871061122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1871507206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1871507206},{"type":"v1","addr":"172.21.15.186:6837","nonce":1871507206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4064412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4064412586},{"type":"v1","addr":"172.21.15.16:6834","nonce":4064412586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:26.751060+0000","modified":"2024-04-25T10:14:41.492234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9096":{"gid":9096,"name":"c","rank":0,"incarnation":86,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.16:6835/2697110279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2697110279},{"type":"v1","addr":"172.21.15.16:6835","nonce":2697110279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:42.688 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 88 2024-04-25T10:14:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.686+0000 7f65cf7cf640 1 -- 172.21.15.16:0/1857258112 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f65ac06b4d0 msgr2=0x7f65ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.686+0000 7f65cf7cf640 1 --2- 172.21.15.16:0/1857258112 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f65ac06b4d0 0x7f65ac06d990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f65bc033000 tx=0x7f65bc008030 comp rx=0 tx=0).stop 2024-04-25T10:14:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.686+0000 7f65cf7cf640 1 -- 172.21.15.16:0/1857258112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c8153070 msgr2=0x7f65c81404c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.686+0000 7f65cf7cf640 1 --2- 172.21.15.16:0/1857258112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c8153070 0x7f65c81404c0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f65c8073410 tx=0x7f65c000e530 comp rx=0 tx=0).stop 2024-04-25T10:14:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.686+0000 7f65cf7cf640 1 -- 172.21.15.16:0/1857258112 shutdown_connections 2024-04-25T10:14:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.686+0000 7f65cf7cf640 1 --2- 172.21.15.16:0/1857258112 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f65ac06b4d0 0x7f65ac06d990 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.686+0000 7f65cf7cf640 1 --2- 172.21.15.16:0/1857258112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c8153070 0x7f65c81404c0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.686+0000 7f65cf7cf640 1 --2- 172.21.15.16:0/1857258112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65c80c7c10 0x7f65c80c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.686+0000 7f65cf7cf640 1 --2- 172.21.15.16:0/1857258112 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65b40039e0 0x7f65c80b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.686+0000 7f65cf7cf640 1 -- 172.21.15.16:0/1857258112 >> 172.21.15.16:0/1857258112 conn(0x7f65c80b2370 msgr2=0x7f65c80c5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.686+0000 7f65cf7cf640 1 -- 172.21.15.16:0/1857258112 shutdown_connections 2024-04-25T10:14:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:42.686+0000 7f65cf7cf640 1 -- 172.21.15.16:0/1857258112 wait complete. 2024-04-25T10:14:43.104 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:14:26.751060+0000', 'modified': '2024-04-25T10:14:41.492234+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 116, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9096}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9096': {'gid': 9096, 'name': 'c', 'rank': 0, 'incarnation': 86, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.16:6835/2697110279', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 2697110279}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 2697110279}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:14:43.104 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9096:up:rejoin 2024-04-25T10:14:44.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:44.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.215+0000 7f3634db0640 1 Processor -- start 2024-04-25T10:14:44.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.215+0000 7f3634db0640 1 -- start start 2024-04-25T10:14:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.216+0000 7f3634db0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f363015dce0 0x7f36300d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.216+0000 7f3634db0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36300d35b0 0x7f36300b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.216+0000 7f3634db0640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363015e0c0 con 0x7f36300d1f90 2024-04-25T10:14:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.216+0000 7f3634db0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36300b7b10 con 0x7f363015dce0 2024-04-25T10:14:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.216+0000 7f3634db0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36300d2510 con 0x7f36300d35b0 2024-04-25T10:14:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.216+0000 7f362e59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f363015dce0 0x7f36300d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.216+0000 7f362e59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f363015dce0 0x7f36300d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58446/0 (socket says 172.21.15.16:58446) 2024-04-25T10:14:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.216+0000 7f362e59c640 1 -- 172.21.15.16:0/4132257718 learned_addr learned my addr 172.21.15.16:0/4132257718 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.216+0000 7f3627fff640 1 --2- 172.21.15.16:0/4132257718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36300d35b0 0x7f36300b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.216+0000 7f362e59c640 1 -- 172.21.15.16:0/4132257718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36300d35b0 msgr2=0x7f36300b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.216+0000 7f362e59c640 1 --2- 172.21.15.16:0/4132257718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36300d35b0 0x7f36300b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.216+0000 7f362e59c640 1 -- 172.21.15.16:0/4132257718 >> v1:172.21.15.186:6789/0 conn(0x7f36300d1f90 legacy=0x7f363006bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.216+0000 7f362e59c640 1 -- 172.21.15.16:0/4132257718 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36300d27f0 con 0x7f363015dce0 2024-04-25T10:14:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.217+0000 7f362e59c640 1 --2- 172.21.15.16:0/4132257718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363015dce0 0x7f36300d1a50 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f3620009870 tx=0x7f362002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27e6460c5ea71ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:44.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.217+0000 7f362dd9b640 1 -- 172.21.15.16:0/4132257718 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620037070 con 0x7f363015dce0 2024-04-25T10:14:44.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.217+0000 7f362dd9b640 1 -- 172.21.15.16:0/4132257718 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3620032440 con 0x7f363015dce0 2024-04-25T10:14:44.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.217+0000 7f362dd9b640 1 -- 172.21.15.16:0/4132257718 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620032810 con 0x7f363015dce0 2024-04-25T10:14:44.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.218+0000 7f3634db0640 1 -- 172.21.15.16:0/4132257718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363015dce0 msgr2=0x7f36300d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:44.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.218+0000 7f3634db0640 1 --2- 172.21.15.16:0/4132257718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363015dce0 0x7f36300d1a50 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f3620009870 tx=0x7f362002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.218+0000 7f3634db0640 1 -- 172.21.15.16:0/4132257718 shutdown_connections 2024-04-25T10:14:44.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.218+0000 7f3634db0640 1 --2- 172.21.15.16:0/4132257718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36300d35b0 0x7f36300b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.218+0000 7f3634db0640 1 --2- 172.21.15.16:0/4132257718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f363015dce0 0x7f36300d1a50 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.218+0000 7f3634db0640 1 -- 172.21.15.16:0/4132257718 >> 172.21.15.16:0/4132257718 conn(0x7f3630072900 msgr2=0x7f36300d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:44.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.218+0000 7f3634db0640 1 -- 172.21.15.16:0/4132257718 shutdown_connections 2024-04-25T10:14:44.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.218+0000 7f3634db0640 1 -- 172.21.15.16:0/4132257718 wait complete. 2024-04-25T10:14:44.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.219+0000 7f3634db0640 1 Processor -- start 2024-04-25T10:14:44.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.219+0000 7f3634db0640 1 -- start start 2024-04-25T10:14:44.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.220+0000 7f3634db0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36300d1f90 0x7f3630160d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:44.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.220+0000 7f3634db0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36300d35b0 0x7f36300c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:44.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.220+0000 7f3634db0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f363015dce0 0x7f36300c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:44.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.220+0000 7f3634db0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36300d3090 con 0x7f36300d1f90 2024-04-25T10:14:44.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.220+0000 7f3634db0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36300d2db0 con 0x7f36300d35b0 2024-04-25T10:14:44.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.220+0000 7f3634db0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36300b9820 con 0x7f363015dce0 2024-04-25T10:14:44.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.220+0000 7f362e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36300d1f90 0x7f3630160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:44.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.220+0000 7f362e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36300d1f90 0x7f3630160d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58454/0 (socket says 172.21.15.16:58454) 2024-04-25T10:14:44.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.220+0000 7f362e59c640 1 -- 172.21.15.16:0/2342140608 learned_addr learned my addr 172.21.15.16:0/2342140608 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:44.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.220+0000 7f3627fff640 1 --2- 172.21.15.16:0/2342140608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36300d35b0 0x7f36300c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:44.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.220+0000 7f362ed9d640 1 --2- 172.21.15.16:0/2342140608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f363015dce0 0x7f36300c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:44.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.220+0000 7f362e59c640 1 -- 172.21.15.16:0/2342140608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f363015dce0 msgr2=0x7f36300c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:44.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.220+0000 7f362e59c640 1 --2- 172.21.15.16:0/2342140608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f363015dce0 0x7f36300c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.220+0000 7f362e59c640 1 -- 172.21.15.16:0/2342140608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36300d35b0 msgr2=0x7f36300c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:44.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.220+0000 7f362e59c640 1 --2- 172.21.15.16:0/2342140608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36300d35b0 0x7f36300c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.220+0000 7f362e59c640 1 -- 172.21.15.16:0/2342140608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36300c8b70 con 0x7f36300d1f90 2024-04-25T10:14:44.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.220+0000 7f362ed9d640 1 --2- 172.21.15.16:0/2342140608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f363015dce0 0x7f36300c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:44.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.221+0000 7f362e59c640 1 --2- 172.21.15.16:0/2342140608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36300d1f90 0x7f3630160d60 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f362002a200 tx=0x7f36200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:44.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.221+0000 7f3626ffd640 1 -- 172.21.15.16:0/2342140608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620041020 con 0x7f36300d1f90 2024-04-25T10:14:44.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.221+0000 7f3626ffd640 1 -- 172.21.15.16:0/2342140608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36200306c0 con 0x7f36300d1f90 2024-04-25T10:14:44.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.221+0000 7f3626ffd640 1 -- 172.21.15.16:0/2342140608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620037040 con 0x7f36300d1f90 2024-04-25T10:14:44.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.221+0000 7f3634db0640 1 -- 172.21.15.16:0/2342140608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36300c8e10 con 0x7f36300d1f90 2024-04-25T10:14:44.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.221+0000 7f3634db0640 1 -- 172.21.15.16:0/2342140608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36300c93d0 con 0x7f36300d1f90 2024-04-25T10:14:44.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.222+0000 7f3634db0640 1 -- 172.21.15.16:0/2342140608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36300749b0 con 0x7f36300d1f90 2024-04-25T10:14:44.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.222+0000 7f3626ffd640 1 -- 172.21.15.16:0/2342140608 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3620044020 con 0x7f36300d1f90 2024-04-25T10:14:44.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.223+0000 7f3626ffd640 1 --2- 172.21.15.16:0/2342140608 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f361006b4d0 0x7f361006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:44.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.223+0000 7f3626ffd640 1 -- 172.21.15.16:0/2342140608 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f36200aa970 con 0x7f36300d1f90 2024-04-25T10:14:44.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.223+0000 7f3627fff640 1 --2- 172.21.15.16:0/2342140608 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f361006b4d0 0x7f361006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:44.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.223+0000 7f3627fff640 1 --2- 172.21.15.16:0/2342140608 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f361006b4d0 0x7f361006d990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f36300c85c0 tx=0x7f3618008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:44.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.227+0000 7f3626ffd640 1 -- 172.21.15.16:0/2342140608 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f362004b030 con 0x7f36300d1f90 2024-04-25T10:14:44.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.444+0000 7f3634db0640 1 -- 172.21.15.16:0/2342140608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36300d14e0 con 0x7f36300d1f90 2024-04-25T10:14:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.444+0000 7f3626ffd640 1 -- 172.21.15.16:0/2342140608 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f3620079790 con 0x7f36300d1f90 2024-04-25T10:14:44.448 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:44.449 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2871061122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2871061122},{"type":"v1","addr":"172.21.15.186:6836","nonce":2871061122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1871507206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1871507206},{"type":"v1","addr":"172.21.15.186:6837","nonce":1871507206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4064412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4064412586},{"type":"v1","addr":"172.21.15.16:6834","nonce":4064412586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:26.751060+0000","modified":"2024-04-25T10:14:42.496103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9096":{"gid":9096,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6835/2697110279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2697110279},{"type":"v1","addr":"172.21.15.16:6835","nonce":2697110279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:44.449 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 89 2024-04-25T10:14:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.446+0000 7f3634db0640 1 -- 172.21.15.16:0/2342140608 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f361006b4d0 msgr2=0x7f361006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.446+0000 7f3634db0640 1 --2- 172.21.15.16:0/2342140608 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f361006b4d0 0x7f361006d990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f36300c85c0 tx=0x7f3618008040 comp rx=0 tx=0).stop 2024-04-25T10:14:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.446+0000 7f3634db0640 1 -- 172.21.15.16:0/2342140608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36300d1f90 msgr2=0x7f3630160d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.446+0000 7f3634db0640 1 --2- 172.21.15.16:0/2342140608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36300d1f90 0x7f3630160d60 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f362002a200 tx=0x7f36200046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.446+0000 7f3634db0640 1 -- 172.21.15.16:0/2342140608 shutdown_connections 2024-04-25T10:14:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.446+0000 7f3634db0640 1 --2- 172.21.15.16:0/2342140608 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f361006b4d0 0x7f361006d990 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.446+0000 7f3634db0640 1 --2- 172.21.15.16:0/2342140608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f363015dce0 0x7f36300c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.446+0000 7f3634db0640 1 --2- 172.21.15.16:0/2342140608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36300d35b0 0x7f36300c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.447+0000 7f3634db0640 1 --2- 172.21.15.16:0/2342140608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36300d1f90 0x7f3630160d60 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.447+0000 7f3634db0640 1 -- 172.21.15.16:0/2342140608 >> 172.21.15.16:0/2342140608 conn(0x7f3630072900 msgr2=0x7f36300b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.447+0000 7f3634db0640 1 -- 172.21.15.16:0/2342140608 shutdown_connections 2024-04-25T10:14:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.447+0000 7f3634db0640 1 -- 172.21.15.16:0/2342140608 wait complete. 2024-04-25T10:14:44.865 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:14:26.751060+0000', 'modified': '2024-04-25T10:14:42.496103+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 116, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9096}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9096': {'gid': 9096, 'name': 'c', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.16:6835/2697110279', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 2697110279}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 2697110279}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:14:44.865 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:14:44.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:14:44.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.967+0000 7f0c02a9e640 1 Processor -- start 2024-04-25T10:14:44.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.968+0000 7f0c02a9e640 1 -- start start 2024-04-25T10:14:44.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.968+0000 7f0c02a9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bfc1610c0 0x7f0bfc0acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:44.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.968+0000 7f0c02a9e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0bfc06bac0 0x7f0bfc06bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.968+0000 7f0c02a9e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bfc06c580 con 0x7f0bfc0cdb30 2024-04-25T10:14:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.968+0000 7f0c02a9e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bfc1614a0 con 0x7f0bfc06bac0 2024-04-25T10:14:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.968+0000 7f0c02a9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc06dde0 con 0x7f0bfc1610c0 2024-04-25T10:14:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.969+0000 7f0c0083a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bfc1610c0 0x7f0bfc0acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.969+0000 7f0bfbfff640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0bfc06bac0 0x7f0bfc06bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.969+0000 7f0c0083a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bfc1610c0 0x7f0bfc0acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:44054/0 (socket says 172.21.15.16:44054) 2024-04-25T10:14:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.969+0000 7f0c0103b640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f0bfc0cdb30 0x7f0bfc06eba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:55416/0 (socket says 172.21.15.16:55416) 2024-04-25T10:14:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.969+0000 7f0c0083a640 1 -- 172.21.15.16:0/3198856196 learned_addr learned my addr 172.21.15.16:0/3198856196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.969+0000 7f0bfbfff640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0bfc06bac0 0x7f0bfc06bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:52510/0 (socket says 172.21.15.16:52510) 2024-04-25T10:14:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.969+0000 7f0c0083a640 1 -- 172.21.15.16:0/3198856196 >> v2:172.21.15.186:3300/0 conn(0x7f0bfc06bac0 msgr2=0x7f0bfc06bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.969+0000 7f0c0083a640 1 --2- 172.21.15.16:0/3198856196 >> v2:172.21.15.186:3300/0 conn(0x7f0bfc06bac0 0x7f0bfc06bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.969+0000 7f0c0083a640 1 -- 172.21.15.16:0/3198856196 >> v1:172.21.15.186:6789/0 conn(0x7f0bfc0cdb30 legacy=0x7f0bfc06eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.969+0000 7f0c0083a640 1 -- 172.21.15.16:0/3198856196 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc06e0c0 con 0x7f0bfc1610c0 2024-04-25T10:14:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.969+0000 7f0bfbfff640 1 --2- 172.21.15.16:0/3198856196 >> v2:172.21.15.186:3300/0 conn(0x7f0bfc06bac0 0x7f0bfc06bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.970+0000 7f0c0083a640 1 --2- 172.21.15.16:0/3198856196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bfc1610c0 0x7f0bfc0acdf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009870 tx=0x7f0bec02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=480c412818e39e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.970+0000 7f0bfb7fe640 1 -- 172.21.15.16:0/3198856196 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec037070 con 0x7f0bfc1610c0 2024-04-25T10:14:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.970+0000 7f0bfb7fe640 1 -- 172.21.15.16:0/3198856196 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bec032440 con 0x7f0bfc1610c0 2024-04-25T10:14:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.970+0000 7f0bfb7fe640 1 -- 172.21.15.16:0/3198856196 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec032810 con 0x7f0bfc1610c0 2024-04-25T10:14:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.971+0000 7f0c02a9e640 1 -- 172.21.15.16:0/3198856196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bfc1610c0 msgr2=0x7f0bfc0acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.971+0000 7f0c02a9e640 1 --2- 172.21.15.16:0/3198856196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bfc1610c0 0x7f0bfc0acdf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009870 tx=0x7f0bec02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.971+0000 7f0c02a9e640 1 -- 172.21.15.16:0/3198856196 shutdown_connections 2024-04-25T10:14:44.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.971+0000 7f0c02a9e640 1 --2- 172.21.15.16:0/3198856196 >> v2:172.21.15.186:3300/0 conn(0x7f0bfc06bac0 0x7f0bfc06bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.971+0000 7f0c02a9e640 1 --2- 172.21.15.16:0/3198856196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bfc1610c0 0x7f0bfc0acdf0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.971+0000 7f0c02a9e640 1 -- 172.21.15.16:0/3198856196 >> 172.21.15.16:0/3198856196 conn(0x7f0bfc0cc9e0 msgr2=0x7f0bfc0ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:44.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.971+0000 7f0c02a9e640 1 -- 172.21.15.16:0/3198856196 shutdown_connections 2024-04-25T10:14:44.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.971+0000 7f0c02a9e640 1 -- 172.21.15.16:0/3198856196 wait complete. 2024-04-25T10:14:44.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.972+0000 7f0c02a9e640 1 Processor -- start 2024-04-25T10:14:44.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.972+0000 7f0c02a9e640 1 -- start start 2024-04-25T10:14:44.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.973+0000 7f0c02a9e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bfc06bac0 0x7f0bfc0c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:44.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.973+0000 7f0c02a9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bfc0cdb30 0x7f0bfc0ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:44.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.973+0000 7f0c02a9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bfc1610c0 0x7f0bfc0b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:44.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.973+0000 7f0c02a9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc06add0 con 0x7f0bfc1610c0 2024-04-25T10:14:44.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.973+0000 7f0c02a9e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc06e680 con 0x7f0bfc06bac0 2024-04-25T10:14:44.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.973+0000 7f0c02a9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc0cdf60 con 0x7f0bfc0cdb30 2024-04-25T10:14:44.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.973+0000 7f0c0103b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bfc1610c0 0x7f0bfc0b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:44.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.973+0000 7f0c0103b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bfc1610c0 0x7f0bfc0b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58466/0 (socket says 172.21.15.16:58466) 2024-04-25T10:14:44.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.973+0000 7f0c0103b640 1 -- 172.21.15.16:0/1768278432 learned_addr learned my addr 172.21.15.16:0/1768278432 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:44.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.973+0000 7f0bfbfff640 1 --2- 172.21.15.16:0/1768278432 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bfc0cdb30 0x7f0bfc0ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.973+0000 7f0c0083a640 1 --2- 172.21.15.16:0/1768278432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bfc06bac0 0x7f0bfc0c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.973+0000 7f0c0103b640 1 -- 172.21.15.16:0/1768278432 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bfc0cdb30 msgr2=0x7f0bfc0ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.973+0000 7f0c0103b640 1 --2- 172.21.15.16:0/1768278432 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bfc0cdb30 0x7f0bfc0ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.973+0000 7f0c0103b640 1 -- 172.21.15.16:0/1768278432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bfc06bac0 msgr2=0x7f0bfc0c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.973+0000 7f0c0103b640 1 --2- 172.21.15.16:0/1768278432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bfc06bac0 0x7f0bfc0c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.973+0000 7f0c0103b640 1 -- 172.21.15.16:0/1768278432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc160310 con 0x7f0bfc1610c0 2024-04-25T10:14:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.974+0000 7f0c0103b640 1 --2- 172.21.15.16:0/1768278432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bfc1610c0 0x7f0bfc0b4060 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400c710 tx=0x7f0bf400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.974+0000 7f0bf97fa640 1 -- 172.21.15.16:0/1768278432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4017070 con 0x7f0bfc1610c0 2024-04-25T10:14:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.974+0000 7f0bf97fa640 1 -- 172.21.15.16:0/1768278432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf400ce20 con 0x7f0bfc1610c0 2024-04-25T10:14:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.974+0000 7f0bf97fa640 1 -- 172.21.15.16:0/1768278432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf400a860 con 0x7f0bfc1610c0 2024-04-25T10:14:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.974+0000 7f0c02a9e640 1 -- 172.21.15.16:0/1768278432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bfc0c8250 con 0x7f0bfc1610c0 2024-04-25T10:14:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.974+0000 7f0c02a9e640 1 -- 172.21.15.16:0/1768278432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bfc0c8810 con 0x7f0bfc1610c0 2024-04-25T10:14:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.975+0000 7f0bf97fa640 1 -- 172.21.15.16:0/1768278432 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0bf401a070 con 0x7f0bfc1610c0 2024-04-25T10:14:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.976+0000 7f0bf97fa640 1 --2- 172.21.15.16:0/1768278432 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0be806b500 0x7f0be806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:44.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.976+0000 7f0bf97fa640 1 -- 172.21.15.16:0/1768278432 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0bf4089740 con 0x7f0bfc1610c0 2024-04-25T10:14:44.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.976+0000 7f0c0083a640 1 --2- 172.21.15.16:0/1768278432 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0be806b500 0x7f0be806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:44.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.977+0000 7f0c0083a640 1 --2- 172.21.15.16:0/1768278432 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0be806b500 0x7f0be806d9c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0bec034000 tx=0x7f0bec02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:44.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.977+0000 7f0c057a6740 1 Processor -- start 2024-04-25T10:14:44.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.977+0000 7f0c057a6740 1 -- start start 2024-04-25T10:14:44.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.978+0000 7f0c057a6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f49230 0x55ca27f49610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:44.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.978+0000 7f0c057a6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f66ad0 0x55ca27f67020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:44.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.978+0000 7f0c057a6740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f1ef10 0x55ca27f65190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:44.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.978+0000 7f0c057a6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca27e81a10 con 0x55ca27f66ad0 2024-04-25T10:14:44.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.978+0000 7f0c057a6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca27e2ab30 con 0x55ca27f49230 2024-04-25T10:14:44.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.978+0000 7f0c057a6740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca27e7dc50 con 0x55ca27f1ef10 2024-04-25T10:14:44.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.978+0000 7f0bfbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f66ad0 0x55ca27f67020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:44.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.978+0000 7f0c0083a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f49230 0x55ca27f49610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:44.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.978+0000 7f0bfbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f66ad0 0x55ca27f67020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58480/0 (socket says 172.21.15.16:58480) 2024-04-25T10:14:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.978+0000 7f0bfbfff640 1 -- 172.21.15.16:0/1740820480 learned_addr learned my addr 172.21.15.16:0/1740820480 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.978+0000 7f0c0103b640 1 --2- 172.21.15.16:0/1740820480 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f1ef10 0x55ca27f65190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.978+0000 7f0bfbfff640 1 -- 172.21.15.16:0/1740820480 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f1ef10 msgr2=0x55ca27f65190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.978+0000 7f0bfbfff640 1 --2- 172.21.15.16:0/1740820480 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f1ef10 0x55ca27f65190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.978+0000 7f0bfbfff640 1 -- 172.21.15.16:0/1740820480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f49230 msgr2=0x55ca27f49610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.978+0000 7f0bfbfff640 1 --2- 172.21.15.16:0/1740820480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f49230 0x55ca27f49610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.978+0000 7f0bfbfff640 1 -- 172.21.15.16:0/1740820480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca27e3fe10 con 0x55ca27f66ad0 2024-04-25T10:14:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.979+0000 7f0bfbfff640 1 --2- 172.21.15.16:0/1740820480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f66ad0 0x55ca27f67020 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009bf0 tx=0x7f0bf0011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86516bd97dbbceb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.979+0000 7f0be2ffd640 1 -- 172.21.15.16:0/1740820480 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0019070 con 0x55ca27f66ad0 2024-04-25T10:14:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.979+0000 7f0be2ffd640 1 -- 172.21.15.16:0/1740820480 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf0004030 con 0x55ca27f66ad0 2024-04-25T10:14:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.979+0000 7f0be2ffd640 1 -- 172.21.15.16:0/1740820480 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0004420 con 0x55ca27f66ad0 2024-04-25T10:14:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.979+0000 7f0c057a6740 1 -- 172.21.15.16:0/1740820480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f66ad0 msgr2=0x55ca27f67020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.979+0000 7f0c057a6740 1 --2- 172.21.15.16:0/1740820480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f66ad0 0x55ca27f67020 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009bf0 tx=0x7f0bf0011690 comp rx=0 tx=0).stop 2024-04-25T10:14:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.979+0000 7f0c057a6740 1 -- 172.21.15.16:0/1740820480 shutdown_connections 2024-04-25T10:14:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.979+0000 7f0c057a6740 1 --2- 172.21.15.16:0/1740820480 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f1ef10 0x55ca27f65190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.979+0000 7f0c057a6740 1 --2- 172.21.15.16:0/1740820480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f66ad0 0x55ca27f67020 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.979+0000 7f0c057a6740 1 --2- 172.21.15.16:0/1740820480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f49230 0x55ca27f49610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.979+0000 7f0c057a6740 1 -- 172.21.15.16:0/1740820480 >> 172.21.15.16:0/1740820480 conn(0x55ca27f1b840 msgr2=0x55ca27f64040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.979+0000 7f0c057a6740 1 -- 172.21.15.16:0/1740820480 shutdown_connections 2024-04-25T10:14:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.979+0000 7f0c057a6740 1 -- 172.21.15.16:0/1740820480 wait complete. 2024-04-25T10:14:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.980+0000 7f0c057a6740 1 -- start start 2024-04-25T10:14:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.980+0000 7f0c057a6740 1 Processor -- start 2024-04-25T10:14:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0c057a6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f49230 0x55ca27f328c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0c057a6740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f66ad0 0x55ca27f4aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0c057a6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f35a90 0x55ca27f34670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0c057a6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca27f1fcb0 con 0x55ca27f35a90 2024-04-25T10:14:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0c057a6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca27e3e0c0 con 0x55ca27f49230 2024-04-25T10:14:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0c057a6740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca27f19770 con 0x55ca27f66ad0 2024-04-25T10:14:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0c057a6740 10 client.0 _ll_register_callbacks cb 0x55ca27e31ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0c0103b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f35a90 0x55ca27f34670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0c0103b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f35a90 0x55ca27f34670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58482/0 (socket says 172.21.15.16:58482) 2024-04-25T10:14:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0c0103b640 1 -- 172.21.15.16:0/2643027782 learned_addr learned my addr 172.21.15.16:0/2643027782 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0c0083a640 1 --2- 172.21.15.16:0/2643027782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f49230 0x55ca27f328c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0bfbfff640 1 --2- 172.21.15.16:0/2643027782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f66ad0 0x55ca27f4aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0c0103b640 1 -- 172.21.15.16:0/2643027782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f66ad0 msgr2=0x55ca27f4aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0c0103b640 1 --2- 172.21.15.16:0/2643027782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f66ad0 0x55ca27f4aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0c0103b640 1 -- 172.21.15.16:0/2643027782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f49230 msgr2=0x55ca27f328c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0c0103b640 1 --2- 172.21.15.16:0/2643027782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f49230 0x55ca27f328c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0c0103b640 1 -- 172.21.15.16:0/2643027782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca27b962d0 con 0x55ca27f35a90 2024-04-25T10:14:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0bfbfff640 1 --2- 172.21.15.16:0/2643027782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f66ad0 0x55ca27f4aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0c0103b640 1 --2- 172.21.15.16:0/2643027782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f35a90 0x55ca27f34670 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4057f30 tx=0x7f0bf400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc329cf98d4947 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0bdb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:14:45.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0bdb7fe640 1 -- 172.21.15.16:0/2643027782 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4086060 con 0x55ca27f35a90 2024-04-25T10:14:45.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.981+0000 7f0bdb7fe640 1 -- 172.21.15.16:0/2643027782 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0bf4087910 con 0x55ca27f35a90 2024-04-25T10:14:45.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.982+0000 7f0c057a6740 1 -- 172.21.15.16:0/2643027782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bf0004030 con 0x55ca27f35a90 2024-04-25T10:14:45.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.982+0000 7f0bdb7fe640 1 -- 172.21.15.16:0/2643027782 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bf406f070 con 0x55ca27f35a90 2024-04-25T10:14:45.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.982+0000 7f0c057a6740 10 client.9138 fetch_fsmap learned FSMap version 89 2024-04-25T10:14:45.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.982+0000 7f0c057a6740 1 -- 172.21.15.16:0/2643027782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55ca27c606b0 con 0x55ca27f35a90 2024-04-25T10:14:45.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.982+0000 7f0bdb7fe640 1 -- 172.21.15.16:0/2643027782 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf0004030 con 0x55ca27f35a90 2024-04-25T10:14:45.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.982+0000 7f0bdb7fe640 1 -- 172.21.15.16:0/2643027782 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0bf4060170 con 0x55ca27f35a90 2024-04-25T10:14:45.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.982+0000 7f0c057a6740 10 client.9138 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:14:45.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.982+0000 7f0c057a6740 10 client.9138 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:14:45.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.982+0000 7f0c057a6740 20 client.9138 populate_metadata read hostname 'smithi016' 2024-04-25T10:14:45.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.982+0000 7f0c057a6740 1 --2- 172.21.15.16:0/2643027782 >> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] conn(0x55ca27f478a0 0x55ca27f47c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.982+0000 7f0c057a6740 4 client.9138 mds_command: new command op to 9096 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:45.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.982+0000 7f0c057a6740 1 -- 172.21.15.16:0/2643027782 --> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca27ee0920 con 0x55ca27f478a0 2024-04-25T10:14:45.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.982+0000 7f0c0083a640 1 --2- 172.21.15.16:0/2643027782 >> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] conn(0x55ca27f478a0 0x55ca27f47c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.982+0000 7f0bdb7fe640 10 client.9138 ms_handle_connect on v2:172.21.15.16:6833/2697110279 2024-04-25T10:14:45.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.982+0000 7f0c0083a640 1 --2- 172.21.15.16:0/2643027782 >> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] conn(0x55ca27f478a0 0x55ca27f47c80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ec61d3486385a9b server_cookie=187fe7de089ffaf4 in_seq=0 out_seq=0 2024-04-25T10:14:45.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.983+0000 7f0bdb7fe640 1 -- 172.21.15.16:0/2643027782 <== mds.0 v2:172.21.15.16:6833/2697110279 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ca27ee0920 con 0x55ca27f478a0 2024-04-25T10:14:45.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.983+0000 7f0bdb7fe640 10 client.9138 handle_command_reply: tid=0 2024-04-25T10:14:45.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.983+0000 7f0c057a6740 1 client.9138 shutdown 2024-04-25T10:14:45.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.983+0000 7f0c057a6740 1 -- 172.21.15.16:0/2643027782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f35a90 msgr2=0x55ca27f34670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.983+0000 7f0c057a6740 1 --2- 172.21.15.16:0/2643027782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f35a90 0x55ca27f34670 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4057f30 tx=0x7f0bf400f000 comp rx=0 tx=0).stop 2024-04-25T10:14:45.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.983+0000 7f0c057a6740 1 -- 172.21.15.16:0/2643027782 shutdown_connections 2024-04-25T10:14:45.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.983+0000 7f0c057a6740 1 --2- 172.21.15.16:0/2643027782 >> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] conn(0x55ca27f478a0 0x55ca27f47c80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.983+0000 7f0c057a6740 1 --2- 172.21.15.16:0/2643027782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f35a90 0x55ca27f34670 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.983+0000 7f0c057a6740 1 --2- 172.21.15.16:0/2643027782 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f66ad0 0x55ca27f4aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.983+0000 7f0bdb7fe640 0 client.9138 ms_handle_reset on v2:172.21.15.16:6833/2697110279 2024-04-25T10:14:45.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.983+0000 7f0c057a6740 1 --2- 172.21.15.16:0/2643027782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f49230 0x55ca27f328c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.983+0000 7f0c057a6740 1 -- 172.21.15.16:0/2643027782 >> 172.21.15.16:0/2643027782 conn(0x55ca27f1ef10 msgr2=0x55ca27f675c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:45.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.983+0000 7f0c057a6740 1 -- 172.21.15.16:0/2643027782 shutdown_connections 2024-04-25T10:14:45.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.983+0000 7f0c057a6740 1 -- 172.21.15.16:0/2643027782 wait complete. 2024-04-25T10:14:45.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.983+0000 7f0c057a6740 20 client.9138 trim_cache size 0 max 16384 2024-04-25T10:14:45.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.999+0000 7f0c057a6740 1 Processor -- start 2024-04-25T10:14:45.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:44.999+0000 7f0c057a6740 1 -- start start 2024-04-25T10:14:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.000+0000 7f0c057a6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f35a90 0x55ca27f48590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.000+0000 7f0c057a6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f49230 0x55ca27f599d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.000+0000 7f0c057a6740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f66ad0 0x55ca27f59f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.000+0000 7f0c057a6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc1f4ab0 con 0x55ca27f49230 2024-04-25T10:14:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.000+0000 7f0c057a6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc1f3eb0 con 0x55ca27f35a90 2024-04-25T10:14:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.000+0000 7f0c057a6740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc1f44b0 con 0x55ca27f66ad0 2024-04-25T10:14:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.000+0000 7f0bfbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f49230 0x55ca27f599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.000+0000 7f0bfbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f49230 0x55ca27f599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58494/0 (socket says 172.21.15.16:58494) 2024-04-25T10:14:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.000+0000 7f0bfbfff640 1 -- 172.21.15.16:0/3267864689 learned_addr learned my addr 172.21.15.16:0/3267864689 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.000+0000 7f0c0083a640 1 --2- 172.21.15.16:0/3267864689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f35a90 0x55ca27f48590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.000+0000 7f0c0103b640 1 --2- 172.21.15.16:0/3267864689 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f66ad0 0x55ca27f59f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.000+0000 7f0bfbfff640 1 -- 172.21.15.16:0/3267864689 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f66ad0 msgr2=0x55ca27f59f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.000+0000 7f0bfbfff640 1 --2- 172.21.15.16:0/3267864689 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f66ad0 0x55ca27f59f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.000+0000 7f0bfbfff640 1 -- 172.21.15.16:0/3267864689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f35a90 msgr2=0x55ca27f48590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.000+0000 7f0bfbfff640 1 --2- 172.21.15.16:0/3267864689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f35a90 0x55ca27f48590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.000+0000 7f0bfbfff640 1 -- 172.21.15.16:0/3267864689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca27f1b520 con 0x55ca27f49230 2024-04-25T10:14:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.000+0000 7f0c0083a640 1 --2- 172.21.15.16:0/3267864689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f35a90 0x55ca27f48590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.000+0000 7f0bfbfff640 1 --2- 172.21.15.16:0/3267864689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f49230 0x55ca27f599d0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0011b90 tx=0x7f0bf0008070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160d77a5441e9e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.001+0000 7f0be27fc640 1 -- 172.21.15.16:0/3267864689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0019040 con 0x55ca27f49230 2024-04-25T10:14:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.001+0000 7f0be27fc640 1 -- 172.21.15.16:0/3267864689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf0004230 con 0x55ca27f49230 2024-04-25T10:14:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.001+0000 7f0be27fc640 1 -- 172.21.15.16:0/3267864689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf00097c0 con 0x55ca27f49230 2024-04-25T10:14:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.001+0000 7f0c057a6740 1 -- 172.21.15.16:0/3267864689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f49230 msgr2=0x55ca27f599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.001+0000 7f0c057a6740 1 --2- 172.21.15.16:0/3267864689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f49230 0x55ca27f599d0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0011b90 tx=0x7f0bf0008070 comp rx=0 tx=0).stop 2024-04-25T10:14:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.001+0000 7f0c057a6740 1 -- 172.21.15.16:0/3267864689 shutdown_connections 2024-04-25T10:14:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.001+0000 7f0c057a6740 1 --2- 172.21.15.16:0/3267864689 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f66ad0 0x55ca27f59f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.001+0000 7f0c057a6740 1 --2- 172.21.15.16:0/3267864689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f49230 0x55ca27f599d0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.001+0000 7f0c057a6740 1 --2- 172.21.15.16:0/3267864689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f35a90 0x55ca27f48590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.001+0000 7f0c057a6740 1 -- 172.21.15.16:0/3267864689 >> 172.21.15.16:0/3267864689 conn(0x55ca27f1ef10 msgr2=0x55ca27f326d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.001+0000 7f0c057a6740 1 -- 172.21.15.16:0/3267864689 shutdown_connections 2024-04-25T10:14:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.001+0000 7f0c057a6740 1 -- 172.21.15.16:0/3267864689 wait complete. 2024-04-25T10:14:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.002+0000 7f0c057a6740 1 -- start start 2024-04-25T10:14:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.002+0000 7f0c057a6740 1 Processor -- start 2024-04-25T10:14:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.002+0000 7f0c057a6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f35a90 0x55ca27f72760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.002+0000 7f0c057a6740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f49230 0x55ca27f72ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.002+0000 7f0c057a6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f66ad0 0x55ca27f18510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.002+0000 7f0c057a6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca27f4b290 con 0x55ca27f66ad0 2024-04-25T10:14:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.002+0000 7f0c057a6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca27f33e70 con 0x55ca27f35a90 2024-04-25T10:14:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.002+0000 7f0c057a6740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca27f5b5c0 con 0x55ca27f49230 2024-04-25T10:14:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0c057a6740 10 client.0 _ll_register_callbacks cb 0x55ca27f19c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0c0103b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f66ad0 0x55ca27f18510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0c0103b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f66ad0 0x55ca27f18510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58506/0 (socket says 172.21.15.16:58506) 2024-04-25T10:14:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0c0103b640 1 -- 172.21.15.16:0/3877418637 learned_addr learned my addr 172.21.15.16:0/3877418637 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0c0083a640 1 --2- 172.21.15.16:0/3877418637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f35a90 0x55ca27f72760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0bfbfff640 1 --2- 172.21.15.16:0/3877418637 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f49230 0x55ca27f72ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0c0103b640 1 -- 172.21.15.16:0/3877418637 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f49230 msgr2=0x55ca27f72ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0c0103b640 1 --2- 172.21.15.16:0/3877418637 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f49230 0x55ca27f72ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0c0103b640 1 -- 172.21.15.16:0/3877418637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f35a90 msgr2=0x55ca27f72760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0c0103b640 1 --2- 172.21.15.16:0/3877418637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f35a90 0x55ca27f72760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0c0103b640 1 -- 172.21.15.16:0/3877418637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca27f310f0 con 0x55ca27f66ad0 2024-04-25T10:14:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0c0083a640 1 --2- 172.21.15.16:0/3877418637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f35a90 0x55ca27f72760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0c0103b640 1 --2- 172.21.15.16:0/3877418637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f66ad0 0x55ca27f18510 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40600d0 tx=0x7f0bf4087b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13a0d688fa8f8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0bdb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:14:45.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0bdb7fe640 1 -- 172.21.15.16:0/3877418637 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4086040 con 0x55ca27f66ad0 2024-04-25T10:14:45.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0bdb7fe640 1 -- 172.21.15.16:0/3877418637 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0bf40575f0 con 0x55ca27f66ad0 2024-04-25T10:14:45.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0c057a6740 1 -- 172.21.15.16:0/3877418637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bf0004230 con 0x55ca27f66ad0 2024-04-25T10:14:45.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0bdb7fe640 1 -- 172.21.15.16:0/3877418637 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bf4054240 con 0x55ca27f66ad0 2024-04-25T10:14:45.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0c057a6740 10 client.9144 fetch_fsmap learned FSMap version 89 2024-04-25T10:14:45.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0c057a6740 1 -- 172.21.15.16:0/3877418637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55ca27f310f0 con 0x55ca27f66ad0 2024-04-25T10:14:45.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0bdb7fe640 1 -- 172.21.15.16:0/3877418637 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf0004230 con 0x55ca27f66ad0 2024-04-25T10:14:45.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0bdb7fe640 1 -- 172.21.15.16:0/3877418637 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0bf4076040 con 0x55ca27f66ad0 2024-04-25T10:14:45.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.003+0000 7f0c057a6740 10 client.9144 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:14:45.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c057a6740 10 client.9144 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:14:45.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c057a6740 20 client.9144 populate_metadata read hostname 'smithi016' 2024-04-25T10:14:45.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c057a6740 1 --2- 172.21.15.16:0/3877418637 >> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] conn(0x55ca27f65690 0x55ca27fcf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c057a6740 4 client.9144 mds_command: new command op to 9096 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:14:45.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c057a6740 1 -- 172.21.15.16:0/3877418637 --> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ca27e8ef50 con 0x55ca27f65690 2024-04-25T10:14:45.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c0083a640 1 --2- 172.21.15.16:0/3877418637 >> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] conn(0x55ca27f65690 0x55ca27fcf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0bdb7fe640 10 client.9144 ms_handle_connect on v2:172.21.15.16:6833/2697110279 2024-04-25T10:14:45.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c0083a640 1 --2- 172.21.15.16:0/3877418637 >> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] conn(0x55ca27f65690 0x55ca27fcf3e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4070e5d8667bb90 server_cookie=36c59f72f5b8abb9 in_seq=0 out_seq=0 2024-04-25T10:14:45.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0bdb7fe640 1 -- 172.21.15.16:0/3877418637 <== mds.0 v2:172.21.15.16:6833/2697110279 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55ca27e8ef50 con 0x55ca27f65690 2024-04-25T10:14:45.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0bdb7fe640 10 client.9144 handle_command_reply: tid=0 2024-04-25T10:14:45.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c057a6740 1 client.9144 shutdown 2024-04-25T10:14:45.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c057a6740 1 -- 172.21.15.16:0/3877418637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f66ad0 msgr2=0x55ca27f18510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c057a6740 1 --2- 172.21.15.16:0/3877418637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f66ad0 0x55ca27f18510 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40600d0 tx=0x7f0bf4087b70 comp rx=0 tx=0).stop 2024-04-25T10:14:45.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c057a6740 1 -- 172.21.15.16:0/3877418637 shutdown_connections 2024-04-25T10:14:45.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c057a6740 1 --2- 172.21.15.16:0/3877418637 >> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] conn(0x55ca27f65690 0x55ca27fcf3e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c057a6740 1 --2- 172.21.15.16:0/3877418637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca27f66ad0 0x55ca27f18510 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c057a6740 1 --2- 172.21.15.16:0/3877418637 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ca27f49230 0x55ca27f72ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0bdb7fe640 0 client.9144 ms_handle_reset on v2:172.21.15.16:6833/2697110279 2024-04-25T10:14:45.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c057a6740 1 --2- 172.21.15.16:0/3877418637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca27f35a90 0x55ca27f72760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c057a6740 1 -- 172.21.15.16:0/3877418637 >> 172.21.15.16:0/3877418637 conn(0x55ca27f1ef10 msgr2=0x55ca27f326d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:45.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c057a6740 1 -- 172.21.15.16:0/3877418637 shutdown_connections 2024-04-25T10:14:45.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c057a6740 1 -- 172.21.15.16:0/3877418637 wait complete. 2024-04-25T10:14:45.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.004+0000 7f0c057a6740 20 client.9144 trim_cache size 0 max 16384 2024-04-25T10:14:45.017 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:14:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "34bc5eca-3a21-42a6-a26f-07da0905e8bc", 2024-04-25T10:14:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-04-25T10:14:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 9096, 2024-04-25T10:14:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-04-25T10:14:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-04-25T10:14:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-04-25T10:14:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 4.5159296610000004, 2024-04-25T10:14:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 89, 2024-04-25T10:14:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 116, 2024-04-25T10:14:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 116, 2024-04-25T10:14:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 5.324161395 2024-04-25T10:14:45.018 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:14:45.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.005+0000 7f0c02a9e640 1 -- 172.21.15.16:0/1768278432 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0be806b500 msgr2=0x7f0be806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.005+0000 7f0c02a9e640 1 --2- 172.21.15.16:0/1768278432 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0be806b500 0x7f0be806d9c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0bec034000 tx=0x7f0bec02e000 comp rx=0 tx=0).stop 2024-04-25T10:14:45.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.005+0000 7f0c02a9e640 1 -- 172.21.15.16:0/1768278432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bfc1610c0 msgr2=0x7f0bfc0b4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.005+0000 7f0c02a9e640 1 --2- 172.21.15.16:0/1768278432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bfc1610c0 0x7f0bfc0b4060 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400c710 tx=0x7f0bf400cbd0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.005+0000 7f0c02a9e640 1 -- 172.21.15.16:0/1768278432 shutdown_connections 2024-04-25T10:14:45.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.005+0000 7f0c02a9e640 1 --2- 172.21.15.16:0/1768278432 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0be806b500 0x7f0be806d9c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.005+0000 7f0c02a9e640 1 --2- 172.21.15.16:0/1768278432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bfc1610c0 0x7f0bfc0b4060 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.005+0000 7f0c02a9e640 1 --2- 172.21.15.16:0/1768278432 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0bfc0cdb30 0x7f0bfc0ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.005+0000 7f0c02a9e640 1 --2- 172.21.15.16:0/1768278432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bfc06bac0 0x7f0bfc0c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.005+0000 7f0c02a9e640 1 -- 172.21.15.16:0/1768278432 >> 172.21.15.16:0/1768278432 conn(0x7f0bfc0cc9e0 msgr2=0x7f0bfc0b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:45.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.005+0000 7f0c02a9e640 1 -- 172.21.15.16:0/1768278432 shutdown_connections 2024-04-25T10:14:45.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.005+0000 7f0c02a9e640 1 -- 172.21.15.16:0/1768278432 wait complete. 2024-04-25T10:14:45.021 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:14:45.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.117+0000 7fbc19f9e640 1 Processor -- start 2024-04-25T10:14:45.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.117+0000 7fbc19f9e640 1 -- start start 2024-04-25T10:14:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.118+0000 7fbc19f9e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbc140afc20 0x7fbc140cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.118+0000 7fbc19f9e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc140d0280 con 0x7fbc14161080 2024-04-25T10:14:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.118+0000 7fbc19f9e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc140cddf0 con 0x7fbc140cfdd0 2024-04-25T10:14:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.118+0000 7fbc19f9e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc140cdbd0 con 0x7fbc140afc20 2024-04-25T10:14:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.118+0000 7fbc18f9c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbc14161080 0x7fbc140cd4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56386/0 (socket says 172.21.15.16:56386) 2024-04-25T10:14:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.118+0000 7fbc18f9c640 1 -- 172.21.15.16:0/3687134961 learned_addr learned my addr 172.21.15.16:0/3687134961 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc13fff640 1 --2- 172.21.15.16:0/3687134961 >> v2:172.21.15.186:3300/0 conn(0x7fbc140afc20 0x7fbc140cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc137fe640 1 -- 172.21.15.16:0/3687134961 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3279616286 0 0) 0x7fbc140d0280 con 0x7fbc14161080 2024-04-25T10:14:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc137fe640 1 -- 172.21.15.16:0/3687134961 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbfc003520 con 0x7fbc14161080 2024-04-25T10:14:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc137fe640 1 -- 172.21.15.16:0/3687134961 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 513134518 0 0) 0x7fbbfc003520 con 0x7fbc14161080 2024-04-25T10:14:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc137fe640 1 -- 172.21.15.16:0/3687134961 >> v2:172.21.15.186:3300/0 conn(0x7fbc140afc20 msgr2=0x7fbc140cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc137fe640 1 --2- 172.21.15.16:0/3687134961 >> v2:172.21.15.186:3300/0 conn(0x7fbc140afc20 0x7fbc140cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc137fe640 1 -- 172.21.15.16:0/3687134961 >> v1:172.21.15.186:6789/0 conn(0x7fbc140cfdd0 legacy=0x7fbc140cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc137fe640 1 -- 172.21.15.16:0/3687134961 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc141618b0 con 0x7fbc14161080 2024-04-25T10:14:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc137fe640 1 -- 172.21.15.16:0/3687134961 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fbc0c0019f0 con 0x7fbc14161080 2024-04-25T10:14:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc137fe640 1 -- 172.21.15.16:0/3687134961 >> v1:172.21.15.16:6789/0 conn(0x7fbc14161080 legacy=0x7fbc140cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc137fe640 1 --2- 172.21.15.16:0/3687134961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbfc003a20 0x7fbbfc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc137fe640 1 --2- 172.21.15.16:0/3687134961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbfc004800 0x7fbbfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc137fe640 1 --2- 172.21.15.16:0/3687134961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbfc0051c0 0x7fbbfc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc137fe640 1 -- 172.21.15.16:0/3687134961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc140cdbd0 con 0x7fbbfc003a20 2024-04-25T10:14:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc137fe640 1 -- 172.21.15.16:0/3687134961 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc00c0f0 con 0x7fbbfc0051c0 2024-04-25T10:14:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc137fe640 1 -- 172.21.15.16:0/3687134961 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc00c2f0 con 0x7fbbfc004800 2024-04-25T10:14:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc18f9c640 1 --2- 172.21.15.16:0/3687134961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbfc003a20 0x7fbbfc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc13fff640 1 --2- 172.21.15.16:0/3687134961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbfc004800 0x7fbbfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc1979d640 1 --2- 172.21.15.16:0/3687134961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbfc0051c0 0x7fbbfc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc18f9c640 1 -- 172.21.15.16:0/3687134961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbfc004800 msgr2=0x7fbbfc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc18f9c640 1 --2- 172.21.15.16:0/3687134961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbfc004800 0x7fbbfc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc18f9c640 1 -- 172.21.15.16:0/3687134961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbfc0051c0 msgr2=0x7fbbfc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc18f9c640 1 --2- 172.21.15.16:0/3687134961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbfc0051c0 0x7fbbfc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc18f9c640 1 -- 172.21.15.16:0/3687134961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc003520 con 0x7fbbfc003a20 2024-04-25T10:14:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc1979d640 1 -- 172.21.15.16:0/3687134961 reap_dead start 2024-04-25T10:14:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.119+0000 7fbc13fff640 1 --2- 172.21.15.16:0/3687134961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbfc004800 0x7fbbfc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.120+0000 7fbc18f9c640 1 --2- 172.21.15.16:0/3687134961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbfc003a20 0x7fbbfc003e40 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00a840 tx=0x7fbbfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88583d4091157ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.120+0000 7fbc137fe640 1 -- 172.21.15.16:0/3687134961 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c00a460 con 0x7fbbfc003a20 2024-04-25T10:14:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.120+0000 7fbc137fe640 1 -- 172.21.15.16:0/3687134961 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c02f070 con 0x7fbbfc003a20 2024-04-25T10:14:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.120+0000 7fbc137fe640 1 -- 172.21.15.16:0/3687134961 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c02bcf0 con 0x7fbbfc003a20 2024-04-25T10:14:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.121+0000 7fbc19f9e640 1 -- 172.21.15.16:0/3687134961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbfc003a20 msgr2=0x7fbbfc003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.121+0000 7fbc19f9e640 1 --2- 172.21.15.16:0/3687134961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbfc003a20 0x7fbbfc003e40 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00a840 tx=0x7fbbfc005830 comp rx=0 tx=0).stop 2024-04-25T10:14:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.121+0000 7fbc19f9e640 1 -- 172.21.15.16:0/3687134961 shutdown_connections 2024-04-25T10:14:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.121+0000 7fbc19f9e640 1 --2- 172.21.15.16:0/3687134961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbfc003a20 0x7fbbfc003e40 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.121+0000 7fbc19f9e640 1 -- 172.21.15.16:0/3687134961 >> 172.21.15.16:0/3687134961 conn(0x7fbc14072640 msgr2=0x7fbc14099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.121+0000 7fbc19f9e640 1 -- 172.21.15.16:0/3687134961 shutdown_connections 2024-04-25T10:14:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.121+0000 7fbc19f9e640 1 -- 172.21.15.16:0/3687134961 wait complete. 2024-04-25T10:14:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.122+0000 7fbc19f9e640 1 Processor -- start 2024-04-25T10:14:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.122+0000 7fbc19f9e640 1 -- start start 2024-04-25T10:14:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.123+0000 7fbc19f9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbfc003a20 0x7fbc141558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.123+0000 7fbc19f9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc140c6010 0x7fbc14154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.123+0000 7fbc19f9e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc140ae900 0x7fbc140aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.123+0000 7fbc19f9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc14161660 con 0x7fbbfc003a20 2024-04-25T10:14:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.123+0000 7fbc19f9e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc14161f00 con 0x7fbc140ae900 2024-04-25T10:14:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.123+0000 7fbc19f9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc14162390 con 0x7fbc140c6010 2024-04-25T10:14:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.123+0000 7fbc18f9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbfc003a20 0x7fbc141558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.123+0000 7fbc18f9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbfc003a20 0x7fbc141558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58534/0 (socket says 172.21.15.16:58534) 2024-04-25T10:14:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.123+0000 7fbc18f9c640 1 -- 172.21.15.16:0/1822991225 learned_addr learned my addr 172.21.15.16:0/1822991225 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.124+0000 7fbc13fff640 1 --2- 172.21.15.16:0/1822991225 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc140c6010 0x7fbc14154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.124+0000 7fbc1979d640 1 --2- 172.21.15.16:0/1822991225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc140ae900 0x7fbc140aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.124+0000 7fbc18f9c640 1 -- 172.21.15.16:0/1822991225 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc140c6010 msgr2=0x7fbc14154650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.124+0000 7fbc18f9c640 1 --2- 172.21.15.16:0/1822991225 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc140c6010 0x7fbc14154650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.124+0000 7fbc18f9c640 1 -- 172.21.15.16:0/1822991225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc140ae900 msgr2=0x7fbc140aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.124+0000 7fbc18f9c640 1 --2- 172.21.15.16:0/1822991225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc140ae900 0x7fbc140aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.124+0000 7fbc18f9c640 1 -- 172.21.15.16:0/1822991225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc140b3220 con 0x7fbbfc003a20 2024-04-25T10:14:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.124+0000 7fbc13fff640 1 --2- 172.21.15.16:0/1822991225 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc140c6010 0x7fbc14154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.124+0000 7fbc1979d640 1 --2- 172.21.15.16:0/1822991225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc140ae900 0x7fbc140aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.124+0000 7fbc18f9c640 1 --2- 172.21.15.16:0/1822991225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbfc003a20 0x7fbc141558c0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00a840 tx=0x7fbc14161e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.124+0000 7fbc117fa640 1 -- 172.21.15.16:0/1822991225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c00a460 con 0x7fbbfc003a20 2024-04-25T10:14:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.124+0000 7fbc117fa640 1 -- 172.21.15.16:0/1822991225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c034670 con 0x7fbbfc003a20 2024-04-25T10:14:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.124+0000 7fbc117fa640 1 -- 172.21.15.16:0/1822991225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c001fb0 con 0x7fbbfc003a20 2024-04-25T10:14:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.124+0000 7fbc19f9e640 1 -- 172.21.15.16:0/1822991225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc14156110 con 0x7fbbfc003a20 2024-04-25T10:14:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.124+0000 7fbc19f9e640 1 -- 172.21.15.16:0/1822991225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc140b37c0 con 0x7fbbfc003a20 2024-04-25T10:14:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.125+0000 7fbc19f9e640 1 -- 172.21.15.16:0/1822991225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc140749b0 con 0x7fbbfc003a20 2024-04-25T10:14:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.125+0000 7fbc117fa640 1 -- 172.21.15.16:0/1822991225 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc0c034a20 con 0x7fbbfc003a20 2024-04-25T10:14:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.126+0000 7fbc117fa640 1 --2- 172.21.15.16:0/1822991225 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbc0006b500 0x7fbc0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.126+0000 7fbc117fa640 1 -- 172.21.15.16:0/1822991225 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbc0c0aa800 con 0x7fbbfc003a20 2024-04-25T10:14:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.126+0000 7fbc13fff640 1 --2- 172.21.15.16:0/1822991225 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbc0006b500 0x7fbc0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.127+0000 7fbc13fff640 1 --2- 172.21.15.16:0/1822991225 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbc0006b500 0x7fbc0006d9c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbc140ce5b0 tx=0x7fbc08008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.130+0000 7fbc117fa640 1 -- 172.21.15.16:0/1822991225 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc0c079590 con 0x7fbbfc003a20 2024-04-25T10:14:45.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.345+0000 7fbc19f9e640 1 -- 172.21.15.16:0/1822991225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbc140749b0 con 0x7fbbfc003a20 2024-04-25T10:14:45.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.346+0000 7fbc117fa640 1 -- 172.21.15.16:0/1822991225 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v89) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbc0c078e90 con 0x7fbbfc003a20 2024-04-25T10:14:45.350 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:14:45.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.348+0000 7fbc19f9e640 1 -- 172.21.15.16:0/1822991225 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbc0006b500 msgr2=0x7fbc0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.348+0000 7fbc19f9e640 1 --2- 172.21.15.16:0/1822991225 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbc0006b500 0x7fbc0006d9c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbc140ce5b0 tx=0x7fbc08008040 comp rx=0 tx=0).stop 2024-04-25T10:14:45.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.348+0000 7fbc19f9e640 1 -- 172.21.15.16:0/1822991225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbfc003a20 msgr2=0x7fbc141558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.348+0000 7fbc19f9e640 1 --2- 172.21.15.16:0/1822991225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbfc003a20 0x7fbc141558c0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00a840 tx=0x7fbc14161e70 comp rx=0 tx=0).stop 2024-04-25T10:14:45.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.348+0000 7fbc19f9e640 1 -- 172.21.15.16:0/1822991225 shutdown_connections 2024-04-25T10:14:45.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.348+0000 7fbc19f9e640 1 --2- 172.21.15.16:0/1822991225 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbc0006b500 0x7fbc0006d9c0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.348+0000 7fbc19f9e640 1 --2- 172.21.15.16:0/1822991225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc140ae900 0x7fbc140aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.348+0000 7fbc19f9e640 1 --2- 172.21.15.16:0/1822991225 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbc140c6010 0x7fbc14154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.348+0000 7fbc19f9e640 1 --2- 172.21.15.16:0/1822991225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbfc003a20 0x7fbc141558c0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.348+0000 7fbc19f9e640 1 -- 172.21.15.16:0/1822991225 >> 172.21.15.16:0/1822991225 conn(0x7fbc14072640 msgr2=0x7fbc1406aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:45.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.348+0000 7fbc19f9e640 1 -- 172.21.15.16:0/1822991225 shutdown_connections 2024-04-25T10:14:45.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:45.348+0000 7fbc19f9e640 1 -- 172.21.15.16:0/1822991225 wait complete. 2024-04-25T10:14:45.363 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:14:45.363 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:14:45.363 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-04-25T10:14:45.363 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:14:45.363 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:45.363 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:14:45.363 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:14:45.363 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:14:45.363 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:45.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:45.364 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-04-25T10:14:45.379 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:14:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:14:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:14:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:14:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:45.380 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:14:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-04-25T10:14:45.380 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:14:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-04-25T10:14:45.380 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:14:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-04-25T10:14:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:14:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42075sec preferred_lft 42075sec 2024-04-25T10:14:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-25T10:14:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:45.380 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:14:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-04-25T10:14:45.381 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:14:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:14:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f06b:fdff:fed4:a335/64 scope link 2024-04-25T10:14:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:45.381 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::90ad:4fff:fe91:3c33/64 scope link 2024-04-25T10:14:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:45.381 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether be:14:4a:3c:16:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::bc14:4aff:fe3c:1605/64 scope link 2024-04-25T10:14:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:45.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:45.382 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:14:45.442 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:14:45.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:45.444 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-04-25T10:14:45.497 INFO:tasks.ceph.mds.c.smithi016.stderr:2024-04-25T10:14:45.494+0000 7fde56bdb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:14:45.500 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-04-25T10:14:45.500 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:14:45.500 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:14:45.500 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:14:45.500 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:14:45.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:45.501 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:14:45.501 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:14:45.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:14:45.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:14:45.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:14:45.501 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:14:45.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:45.586 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:14:45.601 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:14:45.601 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:14:45.602 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:45.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:45.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-04-25T10:14:45.666 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:45.666 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:45.721 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:45.721 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:45.721 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:45.777 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:14:45.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:45.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:45.859 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:45.859 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:14:45.859 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:14:45.868 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:45.868 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:14:45.868 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:14:45.868 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:14:45.868 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:14:45.873 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:45.873 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:14:45.873 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:14:45.873 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:14:45.873 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:14:45.873 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:14:45.873 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:14:45.874 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:14:45.874 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:14:45.874 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:14:45.874 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:14:45.874 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:14:45.874 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:14:45.876 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:14:45.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:45.876 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:45.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:45.940 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:14:45.996 INFO:teuthology.orchestra.run.smithi016.stdout:client7948 2024-04-25T10:14:45.996 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:14:45.997 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:46.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:46.052 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:14:46.112 INFO:teuthology.orchestra.run.smithi016.stdout:./ 2024-04-25T10:14:46.112 INFO:teuthology.orchestra.run.smithi016.stdout:./parent 2024-04-25T10:14:46.112 INFO:teuthology.orchestra.run.smithi016.stdout:./parent/flushed 2024-04-25T10:14:46.112 INFO:teuthology.orchestra.run.smithi016.stdout:./parent/flushed/charlie 2024-04-25T10:14:46.112 INFO:teuthology.orchestra.run.smithi016.stdout:./parent/flushed/alpha 2024-04-25T10:14:46.112 INFO:teuthology.orchestra.run.smithi016.stdout:./parent/unflushed 2024-04-25T10:14:46.112 INFO:teuthology.orchestra.run.smithi016.stdout:./parent/unflushed/jfile 2024-04-25T10:14:46.112 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:46.113 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:46.113 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:46.113 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:46.113 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:14:46.113 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:46.218 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:14:46.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:46.227 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:46.227 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:46.227 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:46.227 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:46.227 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T10:14:46.227 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:46.333 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627776 2024-04-25T10:14:46.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:46.342 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:46.342 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:46.342 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:46.342 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:46.342 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T10:14:46.342 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:46.450 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627777 2024-04-25T10:14:46.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:46.459 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:46.459 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:46.459 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:46.459 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:46.459 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T10:14:46.459 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:46.565 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627780 2024-04-25T10:14:46.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:46.574 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:46.574 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:46.574 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:46.574 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:46.574 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T10:14:46.574 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:46.680 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627778 2024-04-25T10:14:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:46.689 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:46.689 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:46.689 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:46.689 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:46.689 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T10:14:46.690 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:46.796 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627781 2024-04-25T10:14:46.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:46.806 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:46.806 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:46.806 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:46.806 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:46.806 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T10:14:46.806 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:46.912 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627782 2024-04-25T10:14:46.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:46.922 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:14:46.922 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:46.977 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:14:46.977 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:47.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:47.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.169+0000 7fed4ea4a640 1 Processor -- start 2024-04-25T10:14:47.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.169+0000 7fed4ea4a640 1 -- start start 2024-04-25T10:14:47.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.170+0000 7fed4ea4a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fed4815ba30 0x7fed4806eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.170+0000 7fed4ea4a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fed4815fed0 0x7fed4806f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.170+0000 7fed4ea4a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed480717d0 con 0x7fed480caa90 2024-04-25T10:14:47.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.170+0000 7fed4ea4a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed48160400 con 0x7fed4815fed0 2024-04-25T10:14:47.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.170+0000 7fed4ea4a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed4815be10 con 0x7fed4815ba30 2024-04-25T10:14:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.170+0000 7fed477fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fed4815fed0 0x7fed4806f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.170+0000 7fed477fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fed4815fed0 0x7fed4806f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58542/0 (socket says 172.21.15.16:58542) 2024-04-25T10:14:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.170+0000 7fed477fe640 1 -- 172.21.15.16:0/4231718832 learned_addr learned my addr 172.21.15.16:0/4231718832 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.171+0000 7fed47fff640 1 --2- 172.21.15.16:0/4231718832 >> v2:172.21.15.186:3300/0 conn(0x7fed4815ba30 0x7fed4806eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.171+0000 7fed477fe640 1 -- 172.21.15.16:0/4231718832 >> v2:172.21.15.186:3300/0 conn(0x7fed4815ba30 msgr2=0x7fed4806eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.171+0000 7fed477fe640 1 --2- 172.21.15.16:0/4231718832 >> v2:172.21.15.186:3300/0 conn(0x7fed4815ba30 0x7fed4806eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.171+0000 7fed477fe640 1 -- 172.21.15.16:0/4231718832 >> v1:172.21.15.186:6789/0 conn(0x7fed480caa90 legacy=0x7fed4806d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.171+0000 7fed477fe640 1 -- 172.21.15.16:0/4231718832 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed48071a70 con 0x7fed4815fed0 2024-04-25T10:14:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.171+0000 7fed477fe640 1 --2- 172.21.15.16:0/4231718832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed4815fed0 0x7fed4806f0a0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fed48072f70 tx=0x7fed3802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf0114dc043c129 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.172+0000 7fed46ffd640 1 -- 172.21.15.16:0/4231718832 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38037070 con 0x7fed4815fed0 2024-04-25T10:14:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.172+0000 7fed46ffd640 1 -- 172.21.15.16:0/4231718832 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed38032440 con 0x7fed4815fed0 2024-04-25T10:14:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.172+0000 7fed46ffd640 1 -- 172.21.15.16:0/4231718832 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38032810 con 0x7fed4815fed0 2024-04-25T10:14:47.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.172+0000 7fed4ea4a640 1 -- 172.21.15.16:0/4231718832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed4815fed0 msgr2=0x7fed4806f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.172+0000 7fed4ea4a640 1 --2- 172.21.15.16:0/4231718832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed4815fed0 0x7fed4806f0a0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fed48072f70 tx=0x7fed3802a230 comp rx=0 tx=0).stop 2024-04-25T10:14:47.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.172+0000 7fed4ea4a640 1 -- 172.21.15.16:0/4231718832 shutdown_connections 2024-04-25T10:14:47.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.172+0000 7fed4ea4a640 1 --2- 172.21.15.16:0/4231718832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed4815fed0 0x7fed4806f0a0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.172+0000 7fed4ea4a640 1 --2- 172.21.15.16:0/4231718832 >> v2:172.21.15.186:3300/0 conn(0x7fed4815ba30 0x7fed4806eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.172+0000 7fed4ea4a640 1 -- 172.21.15.16:0/4231718832 >> 172.21.15.16:0/4231718832 conn(0x7fed480cc540 msgr2=0x7fed480cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:47.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.172+0000 7fed4ea4a640 1 -- 172.21.15.16:0/4231718832 shutdown_connections 2024-04-25T10:14:47.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.172+0000 7fed4ea4a640 1 -- 172.21.15.16:0/4231718832 wait complete. 2024-04-25T10:14:47.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.173+0000 7fed4ea4a640 1 Processor -- start 2024-04-25T10:14:47.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.174+0000 7fed4ea4a640 1 -- start start 2024-04-25T10:14:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.174+0000 7fed4ea4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed480caa90 0x7fed481566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.174+0000 7fed4ea4a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed4815ba30 0x7fed48159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.174+0000 7fed4ea4a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed4815fed0 0x7fed48144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.174+0000 7fed4ea4a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed480ca2a0 con 0x7fed480caa90 2024-04-25T10:14:47.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.174+0000 7fed4ea4a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed48072030 con 0x7fed4815ba30 2024-04-25T10:14:47.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.174+0000 7fed4ea4a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed480ca420 con 0x7fed4815fed0 2024-04-25T10:14:47.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.174+0000 7fed47fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed480caa90 0x7fed481566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.174+0000 7fed47fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed480caa90 0x7fed481566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58546/0 (socket says 172.21.15.16:58546) 2024-04-25T10:14:47.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.174+0000 7fed47fff640 1 -- 172.21.15.16:0/2486936083 learned_addr learned my addr 172.21.15.16:0/2486936083 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:47.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.175+0000 7fed477fe640 1 --2- 172.21.15.16:0/2486936083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed4815ba30 0x7fed48159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.175+0000 7fed47fff640 1 -- 172.21.15.16:0/2486936083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed4815fed0 msgr2=0x7fed48144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.175+0000 7fed47fff640 1 --2- 172.21.15.16:0/2486936083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed4815fed0 0x7fed48144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.175+0000 7fed47fff640 1 -- 172.21.15.16:0/2486936083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed4815ba30 msgr2=0x7fed48159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.175+0000 7fed47fff640 1 --2- 172.21.15.16:0/2486936083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed4815ba30 0x7fed48159bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.175+0000 7fed47fff640 1 -- 172.21.15.16:0/2486936083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed48156f20 con 0x7fed480caa90 2024-04-25T10:14:47.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.175+0000 7fed47fff640 1 --2- 172.21.15.16:0/2486936083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed480caa90 0x7fed481566d0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00d800 tx=0x7fed3c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.175+0000 7fed44ff9640 1 -- 172.21.15.16:0/2486936083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c00b6c0 con 0x7fed480caa90 2024-04-25T10:14:47.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.175+0000 7fed44ff9640 1 -- 172.21.15.16:0/2486936083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3c019070 con 0x7fed480caa90 2024-04-25T10:14:47.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.175+0000 7fed4ea4a640 1 -- 172.21.15.16:0/2486936083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed48157200 con 0x7fed480caa90 2024-04-25T10:14:47.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.175+0000 7fed44ff9640 1 -- 172.21.15.16:0/2486936083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c00ba80 con 0x7fed480caa90 2024-04-25T10:14:47.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.175+0000 7fed4ea4a640 1 -- 172.21.15.16:0/2486936083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed481577c0 con 0x7fed480caa90 2024-04-25T10:14:47.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.176+0000 7fed4ea4a640 1 -- 172.21.15.16:0/2486936083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed48157b00 con 0x7fed480caa90 2024-04-25T10:14:47.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.176+0000 7fed44ff9640 1 -- 172.21.15.16:0/2486936083 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed3c00bc20 con 0x7fed480caa90 2024-04-25T10:14:47.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.177+0000 7fed44ff9640 1 --2- 172.21.15.16:0/2486936083 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed3406b4d0 0x7fed3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.177+0000 7fed44ff9640 1 -- 172.21.15.16:0/2486936083 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fed3c08a740 con 0x7fed480caa90 2024-04-25T10:14:47.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.177+0000 7fed477fe640 1 --2- 172.21.15.16:0/2486936083 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed3406b4d0 0x7fed3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.178+0000 7fed477fe640 1 --2- 172.21.15.16:0/2486936083 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed3406b4d0 0x7fed3406d990 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fed38034000 tx=0x7fed3802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.181+0000 7fed44ff9640 1 -- 172.21.15.16:0/2486936083 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed48157b00 con 0x7fed480caa90 2024-04-25T10:14:47.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.366+0000 7fed4ea4a640 1 -- 172.21.15.16:0/2486936083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed480749b0 con 0x7fed480caa90 2024-04-25T10:14:47.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.368+0000 7fed44ff9640 1 -- 172.21.15.16:0/2486936083 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16810 (secure 0 0 0) 0x7fed3c01c360 con 0x7fed480caa90 2024-04-25T10:14:47.372 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:47.372 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":116,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:14:39.639531+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:25.025480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:25.726747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/123396308":"2024-04-26T10:14:39.639508+0000","172.21.15.16:6833/123396308":"2024-04-26T10:14:39.639508+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:47.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.370+0000 7fed4ea4a640 1 -- 172.21.15.16:0/2486936083 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed3406b4d0 msgr2=0x7fed3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.370+0000 7fed4ea4a640 1 --2- 172.21.15.16:0/2486936083 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed3406b4d0 0x7fed3406d990 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fed38034000 tx=0x7fed3802e000 comp rx=0 tx=0).stop 2024-04-25T10:14:47.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.370+0000 7fed4ea4a640 1 -- 172.21.15.16:0/2486936083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed480caa90 msgr2=0x7fed481566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.370+0000 7fed4ea4a640 1 --2- 172.21.15.16:0/2486936083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed480caa90 0x7fed481566d0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00d800 tx=0x7fed3c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.370+0000 7fed4ea4a640 1 -- 172.21.15.16:0/2486936083 shutdown_connections 2024-04-25T10:14:47.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.370+0000 7fed4ea4a640 1 --2- 172.21.15.16:0/2486936083 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fed3406b4d0 0x7fed3406d990 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.370+0000 7fed4ea4a640 1 --2- 172.21.15.16:0/2486936083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed4815fed0 0x7fed48144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.370+0000 7fed4ea4a640 1 --2- 172.21.15.16:0/2486936083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed4815ba30 0x7fed48159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.370+0000 7fed4ea4a640 1 --2- 172.21.15.16:0/2486936083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed480caa90 0x7fed481566d0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.370+0000 7fed4ea4a640 1 -- 172.21.15.16:0/2486936083 >> 172.21.15.16:0/2486936083 conn(0x7fed480cc540 msgr2=0x7fed480c94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:47.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.370+0000 7fed4ea4a640 1 -- 172.21.15.16:0/2486936083 shutdown_connections 2024-04-25T10:14:47.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.370+0000 7fed4ea4a640 1 -- 172.21.15.16:0/2486936083 wait complete. 2024-04-25T10:14:47.385 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:14:25.025480+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:14:25.726747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:47.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.484+0000 7f434784a640 1 Processor -- start 2024-04-25T10:14:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.484+0000 7f434784a640 1 -- start start 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.485+0000 7f434784a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f434015dca0 0x7f43400ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.485+0000 7f434784a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43400d1130 0x7f43400b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.485+0000 7f434784a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43400d4090 con 0x7f43400d3b90 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.485+0000 7f434784a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43400b6b20 con 0x7f434015dca0 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.485+0000 7f434784a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43400d4370 con 0x7f43400d1130 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.485+0000 7f43455e6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f434015dca0 0x7f43400ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.485+0000 7f4345de7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43400d1130 0x7f43400b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.485+0000 7f4345de7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43400d1130 0x7f43400b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:44152/0 (socket says 172.21.15.16:44152) 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.485+0000 7f4345de7640 1 -- 172.21.15.16:0/424186441 learned_addr learned my addr 172.21.15.16:0/424186441 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.486+0000 7f43455e6640 1 -- 172.21.15.16:0/424186441 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43400d1130 msgr2=0x7f43400b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.486+0000 7f43455e6640 1 --2- 172.21.15.16:0/424186441 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43400d1130 0x7f43400b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.486+0000 7f43455e6640 1 -- 172.21.15.16:0/424186441 >> v1:172.21.15.186:6789/0 conn(0x7f43400d3b90 legacy=0x7f43400d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.486+0000 7f43455e6640 1 -- 172.21.15.16:0/424186441 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43400d4650 con 0x7f434015dca0 2024-04-25T10:14:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.486+0000 7f4337fff640 1 -- 172.21.15.16:0/424186441 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2585621439 0 0) 0x7f43400d4090 con 0x7f43400d3b90 2024-04-25T10:14:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.486+0000 7f4345de7640 1 --2- 172.21.15.16:0/424186441 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43400d1130 0x7f43400b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.487+0000 7f43455e6640 1 --2- 172.21.15.16:0/424186441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f434015dca0 0x7f43400ce940 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f433c0097a0 tx=0x7f433c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70066d2d2bbb5129 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.487+0000 7f4337fff640 1 -- 172.21.15.16:0/424186441 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c037070 con 0x7f434015dca0 2024-04-25T10:14:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.487+0000 7f4337fff640 1 -- 172.21.15.16:0/424186441 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433c032440 con 0x7f434015dca0 2024-04-25T10:14:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.487+0000 7f4337fff640 1 -- 172.21.15.16:0/424186441 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c032810 con 0x7f434015dca0 2024-04-25T10:14:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.487+0000 7f434784a640 1 -- 172.21.15.16:0/424186441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f434015dca0 msgr2=0x7f43400ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.487+0000 7f434784a640 1 --2- 172.21.15.16:0/424186441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f434015dca0 0x7f43400ce940 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f433c0097a0 tx=0x7f433c02a180 comp rx=0 tx=0).stop 2024-04-25T10:14:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.487+0000 7f434784a640 1 -- 172.21.15.16:0/424186441 shutdown_connections 2024-04-25T10:14:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.487+0000 7f434784a640 1 --2- 172.21.15.16:0/424186441 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43400d1130 0x7f43400b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.487+0000 7f434784a640 1 --2- 172.21.15.16:0/424186441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f434015dca0 0x7f43400ce940 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.487+0000 7f434784a640 1 -- 172.21.15.16:0/424186441 >> 172.21.15.16:0/424186441 conn(0x7f43400726c0 msgr2=0x7f4340072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.487+0000 7f434784a640 1 -- 172.21.15.16:0/424186441 shutdown_connections 2024-04-25T10:14:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.487+0000 7f434784a640 1 -- 172.21.15.16:0/424186441 wait complete. 2024-04-25T10:14:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.489+0000 7f434784a640 1 Processor -- start 2024-04-25T10:14:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.489+0000 7f434784a640 1 -- start start 2024-04-25T10:14:47.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.489+0000 7f434784a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43400d1130 0x7f43400cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.489+0000 7f434784a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43400d3b90 0x7f43400ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.489+0000 7f434784a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f434015dca0 0x7f43400ad720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.489+0000 7f434784a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434015e200 con 0x7f434015dca0 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.489+0000 7f434784a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434015e080 con 0x7f43400d3b90 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.489+0000 7f434784a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43400cee80 con 0x7f43400d1130 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.489+0000 7f4345de7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f434015dca0 0x7f43400ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.489+0000 7f4345de7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f434015dca0 0x7f43400ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58556/0 (socket says 172.21.15.16:58556) 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.489+0000 7f4345de7640 1 -- 172.21.15.16:0/4008313094 learned_addr learned my addr 172.21.15.16:0/4008313094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.490+0000 7f4344de5640 1 --2- 172.21.15.16:0/4008313094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43400d3b90 0x7f43400ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.490+0000 7f43455e6640 1 --2- 172.21.15.16:0/4008313094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43400d1130 0x7f43400cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.490+0000 7f4345de7640 1 -- 172.21.15.16:0/4008313094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43400d1130 msgr2=0x7f43400cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.490+0000 7f4345de7640 1 --2- 172.21.15.16:0/4008313094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43400d1130 0x7f43400cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.490+0000 7f4345de7640 1 -- 172.21.15.16:0/4008313094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43400d3b90 msgr2=0x7f43400ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.490+0000 7f4345de7640 1 --2- 172.21.15.16:0/4008313094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43400d3b90 0x7f43400ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.490+0000 7f4345de7640 1 -- 172.21.15.16:0/4008313094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43400cd160 con 0x7f434015dca0 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.490+0000 7f4345de7640 1 --2- 172.21.15.16:0/4008313094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f434015dca0 0x7f43400ad720 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f433800efc0 tx=0x7f433800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.490+0000 7f4335ffb640 1 -- 172.21.15.16:0/4008313094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433800cbe0 con 0x7f434015dca0 2024-04-25T10:14:47.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.490+0000 7f4335ffb640 1 -- 172.21.15.16:0/4008313094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433800cd80 con 0x7f434015dca0 2024-04-25T10:14:47.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.490+0000 7f4335ffb640 1 -- 172.21.15.16:0/4008313094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4338003bb0 con 0x7f434015dca0 2024-04-25T10:14:47.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.490+0000 7f434784a640 1 -- 172.21.15.16:0/4008313094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43400cd440 con 0x7f434015dca0 2024-04-25T10:14:47.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.490+0000 7f434784a640 1 -- 172.21.15.16:0/4008313094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434014d230 con 0x7f434015dca0 2024-04-25T10:14:47.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.491+0000 7f434784a640 1 -- 172.21.15.16:0/4008313094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f434008ba50 con 0x7f434015dca0 2024-04-25T10:14:47.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.492+0000 7f4335ffb640 1 -- 172.21.15.16:0/4008313094 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f433801b030 con 0x7f434015dca0 2024-04-25T10:14:47.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.492+0000 7f4335ffb640 1 --2- 172.21.15.16:0/4008313094 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f432406b4d0 0x7f432406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.492+0000 7f4335ffb640 1 -- 172.21.15.16:0/4008313094 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f433808a640 con 0x7f434015dca0 2024-04-25T10:14:47.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.493+0000 7f43455e6640 1 --2- 172.21.15.16:0/4008313094 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f432406b4d0 0x7f432406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.493+0000 7f43455e6640 1 --2- 172.21.15.16:0/4008313094 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f432406b4d0 0x7f432406d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f433c034000 tx=0x7f433c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.496+0000 7f4335ffb640 1 -- 172.21.15.16:0/4008313094 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f434008ba50 con 0x7f434015dca0 2024-04-25T10:14:47.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.677+0000 7f434784a640 1 -- 172.21.15.16:0/4008313094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43400750b0 con 0x7f434015dca0 2024-04-25T10:14:47.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.678+0000 7f4335ffb640 1 -- 172.21.15.16:0/4008313094 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4338022020 con 0x7f434015dca0 2024-04-25T10:14:47.682 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:14:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.680+0000 7f434784a640 1 -- 172.21.15.16:0/4008313094 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f432406b4d0 msgr2=0x7f432406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.680+0000 7f434784a640 1 --2- 172.21.15.16:0/4008313094 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f432406b4d0 0x7f432406d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f433c034000 tx=0x7f433c02e000 comp rx=0 tx=0).stop 2024-04-25T10:14:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.680+0000 7f434784a640 1 -- 172.21.15.16:0/4008313094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f434015dca0 msgr2=0x7f43400ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.680+0000 7f434784a640 1 --2- 172.21.15.16:0/4008313094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f434015dca0 0x7f43400ad720 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f433800efc0 tx=0x7f433800c490 comp rx=0 tx=0).stop 2024-04-25T10:14:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.680+0000 7f434784a640 1 -- 172.21.15.16:0/4008313094 shutdown_connections 2024-04-25T10:14:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.680+0000 7f434784a640 1 --2- 172.21.15.16:0/4008313094 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f432406b4d0 0x7f432406d990 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.680+0000 7f434784a640 1 --2- 172.21.15.16:0/4008313094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f434015dca0 0x7f43400ad720 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.680+0000 7f434784a640 1 --2- 172.21.15.16:0/4008313094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43400d3b90 0x7f43400ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.680+0000 7f434784a640 1 --2- 172.21.15.16:0/4008313094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f43400d1130 0x7f43400cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.680+0000 7f434784a640 1 -- 172.21.15.16:0/4008313094 >> 172.21.15.16:0/4008313094 conn(0x7f43400726c0 msgr2=0x7f43400d2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.680+0000 7f434784a640 1 -- 172.21.15.16:0/4008313094 shutdown_connections 2024-04-25T10:14:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.680+0000 7f434784a640 1 -- 172.21.15.16:0/4008313094 wait complete. 2024-04-25T10:14:47.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:47.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.794+0000 7f5668eb3640 1 Processor -- start 2024-04-25T10:14:47.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.794+0000 7f5668eb3640 1 -- start start 2024-04-25T10:14:47.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.795+0000 7f5668eb3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f56640d3b90 0x7f56640d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.795+0000 7f5668eb3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56640d4090 con 0x7f56640d0f60 2024-04-25T10:14:47.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.795+0000 7f5668eb3640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56640d4370 con 0x7f566415dca0 2024-04-25T10:14:47.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.795+0000 7f5668eb3640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56640b6b20 con 0x7f56640d3b90 2024-04-25T10:14:47.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.795+0000 7f5662d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f56640d0f60 0x7f56640b2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56398/0 (socket says 172.21.15.16:56398) 2024-04-25T10:14:47.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.795+0000 7f5662d9d640 1 -- 172.21.15.16:0/1900265226 learned_addr learned my addr 172.21.15.16:0/1900265226 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:47.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.795+0000 7f565bfff640 1 --2- 172.21.15.16:0/1900265226 >> v2:172.21.15.186:3300/0 conn(0x7f56640d3b90 0x7f56640d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.795+0000 7f5661d9b640 1 -- 172.21.15.16:0/1900265226 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165106026 0 0) 0x7f56640d4090 con 0x7f56640d0f60 2024-04-25T10:14:47.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.795+0000 7f5661d9b640 1 -- 172.21.15.16:0/1900265226 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5650003520 con 0x7f56640d0f60 2024-04-25T10:14:47.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.796+0000 7f565bfff640 1 -- 172.21.15.16:0/1900265226 >> v1:172.21.15.186:6789/0 conn(0x7f566415dca0 legacy=0x7f56640ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.796+0000 7f5661d9b640 1 -- 172.21.15.16:0/1900265226 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2451895648 0 0) 0x7f5650003520 con 0x7f56640d0f60 2024-04-25T10:14:47.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.796+0000 7f565bfff640 1 -- 172.21.15.16:0/1900265226 >> v1:172.21.15.16:6789/0 conn(0x7f56640d0f60 legacy=0x7f56640b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.796+0000 7f565bfff640 1 -- 172.21.15.16:0/1900265226 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56640d4650 con 0x7f56640d3b90 2024-04-25T10:14:47.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.796+0000 7f565bfff640 1 --2- 172.21.15.16:0/1900265226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56640d3b90 0x7f56640d0a20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f564c0097a0 tx=0x7f564c02a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e1c8dcff0d320b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.796+0000 7f5661d9b640 1 -- 172.21.15.16:0/1900265226 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c037070 con 0x7f56640d3b90 2024-04-25T10:14:47.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.796+0000 7f5661d9b640 1 -- 172.21.15.16:0/1900265226 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c02ab60 con 0x7f56640d3b90 2024-04-25T10:14:47.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.797+0000 7f5661d9b640 1 -- 172.21.15.16:0/1900265226 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c0019f0 con 0x7f56640d3b90 2024-04-25T10:14:47.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.797+0000 7f5668eb3640 1 -- 172.21.15.16:0/1900265226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56640d3b90 msgr2=0x7f56640d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.797+0000 7f5668eb3640 1 --2- 172.21.15.16:0/1900265226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56640d3b90 0x7f56640d0a20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f564c0097a0 tx=0x7f564c02a0d0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.797+0000 7f5668eb3640 1 -- 172.21.15.16:0/1900265226 shutdown_connections 2024-04-25T10:14:47.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.797+0000 7f5668eb3640 1 --2- 172.21.15.16:0/1900265226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56640d3b90 0x7f56640d0a20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.797+0000 7f5668eb3640 1 -- 172.21.15.16:0/1900265226 >> 172.21.15.16:0/1900265226 conn(0x7f56640726c0 msgr2=0x7f5664072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:47.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.797+0000 7f5668eb3640 1 -- 172.21.15.16:0/1900265226 shutdown_connections 2024-04-25T10:14:47.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.797+0000 7f5668eb3640 1 -- 172.21.15.16:0/1900265226 wait complete. 2024-04-25T10:14:47.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.798+0000 7f5668eb3640 1 Processor -- start 2024-04-25T10:14:47.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.798+0000 7f5668eb3640 1 -- start start 2024-04-25T10:14:47.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.799+0000 7f5668eb3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56640d0f60 0x7f5664154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.799+0000 7f5668eb3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56640d3b90 0x7f566415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.799+0000 7f5668eb3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f566415dca0 0x7f56640ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.799+0000 7f5668eb3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56640cf050 con 0x7f566415dca0 2024-04-25T10:14:47.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.799+0000 7f5668eb3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56640d1390 con 0x7f56640d3b90 2024-04-25T10:14:47.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.799+0000 7f5668eb3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566415e080 con 0x7f56640d0f60 2024-04-25T10:14:47.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.799+0000 7f5662d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f566415dca0 0x7f56640ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.799+0000 7f5662d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f566415dca0 0x7f56640ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58566/0 (socket says 172.21.15.16:58566) 2024-04-25T10:14:47.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.799+0000 7f5662d9d640 1 -- 172.21.15.16:0/1700150808 learned_addr learned my addr 172.21.15.16:0/1700150808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:47.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.799+0000 7f565bfff640 1 --2- 172.21.15.16:0/1700150808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56640d3b90 0x7f566415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.799+0000 7f566259c640 1 --2- 172.21.15.16:0/1700150808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56640d0f60 0x7f5664154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.800+0000 7f5662d9d640 1 -- 172.21.15.16:0/1700150808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56640d0f60 msgr2=0x7f5664154970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.800+0000 7f5662d9d640 1 --2- 172.21.15.16:0/1700150808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56640d0f60 0x7f5664154970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.800+0000 7f5662d9d640 1 -- 172.21.15.16:0/1700150808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56640d3b90 msgr2=0x7f566415aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.800+0000 7f5662d9d640 1 --2- 172.21.15.16:0/1700150808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56640d3b90 0x7f566415aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.800+0000 7f5662d9d640 1 -- 172.21.15.16:0/1700150808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5664159c00 con 0x7f566415dca0 2024-04-25T10:14:47.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.800+0000 7f566259c640 1 --2- 172.21.15.16:0/1700150808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56640d0f60 0x7f5664154970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:47.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.800+0000 7f5662d9d640 1 --2- 172.21.15.16:0/1700150808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f566415dca0 0x7f56640ca2f0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f565c001560 tx=0x7f565c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.800+0000 7f565affd640 1 -- 172.21.15.16:0/1700150808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c018070 con 0x7f566415dca0 2024-04-25T10:14:47.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.800+0000 7f565affd640 1 -- 172.21.15.16:0/1700150808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565c00a640 con 0x7f566415dca0 2024-04-25T10:14:47.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.800+0000 7f565affd640 1 -- 172.21.15.16:0/1700150808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c00aa30 con 0x7f566415dca0 2024-04-25T10:14:47.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.800+0000 7f5668eb3640 1 -- 172.21.15.16:0/1700150808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5664159ee0 con 0x7f566415dca0 2024-04-25T10:14:47.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.800+0000 7f5668eb3640 1 -- 172.21.15.16:0/1700150808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f566415a4a0 con 0x7f566415dca0 2024-04-25T10:14:47.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.801+0000 7f5668eb3640 1 -- 172.21.15.16:0/1700150808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56640750b0 con 0x7f566415dca0 2024-04-25T10:14:47.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.801+0000 7f565affd640 1 -- 172.21.15.16:0/1700150808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f565c01b070 con 0x7f566415dca0 2024-04-25T10:14:47.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.802+0000 7f565affd640 1 --2- 172.21.15.16:0/1700150808 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f564406b4d0 0x7f564406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.802+0000 7f565affd640 1 -- 172.21.15.16:0/1700150808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f565c089900 con 0x7f566415dca0 2024-04-25T10:14:47.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.802+0000 7f566259c640 1 --2- 172.21.15.16:0/1700150808 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f564406b4d0 0x7f564406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.803+0000 7f566259c640 1 --2- 172.21.15.16:0/1700150808 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f564406b4d0 0x7f564406d990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5654007fd0 tx=0x7f5654009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.806+0000 7f565affd640 1 -- 172.21.15.16:0/1700150808 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f565c0580d0 con 0x7f566415dca0 2024-04-25T10:14:47.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.985+0000 7f5668eb3640 1 -- 172.21.15.16:0/1700150808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f566415e4d0 con 0x7f566415dca0 2024-04-25T10:14:47.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.985+0000 7f565affd640 1 -- 172.21.15.16:0/1700150808 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) v1 ==== 104+0+11 (secure 0 0 0) 0x7f565c05c170 con 0x7f566415dca0 2024-04-25T10:14:47.989 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:14:47.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.987+0000 7f5668eb3640 1 -- 172.21.15.16:0/1700150808 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f564406b4d0 msgr2=0x7f564406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.987+0000 7f5668eb3640 1 --2- 172.21.15.16:0/1700150808 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f564406b4d0 0x7f564406d990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5654007fd0 tx=0x7f5654009040 comp rx=0 tx=0).stop 2024-04-25T10:14:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.987+0000 7f5668eb3640 1 -- 172.21.15.16:0/1700150808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f566415dca0 msgr2=0x7f56640ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.987+0000 7f5668eb3640 1 --2- 172.21.15.16:0/1700150808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f566415dca0 0x7f56640ca2f0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f565c001560 tx=0x7f565c00cd20 comp rx=0 tx=0).stop 2024-04-25T10:14:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.987+0000 7f5668eb3640 1 -- 172.21.15.16:0/1700150808 shutdown_connections 2024-04-25T10:14:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.987+0000 7f5668eb3640 1 --2- 172.21.15.16:0/1700150808 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f564406b4d0 0x7f564406d990 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.987+0000 7f5668eb3640 1 --2- 172.21.15.16:0/1700150808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f566415dca0 0x7f56640ca2f0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.987+0000 7f5668eb3640 1 --2- 172.21.15.16:0/1700150808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56640d3b90 0x7f566415aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.987+0000 7f5668eb3640 1 --2- 172.21.15.16:0/1700150808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56640d0f60 0x7f5664154970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.987+0000 7f5668eb3640 1 -- 172.21.15.16:0/1700150808 >> 172.21.15.16:0/1700150808 conn(0x7f56640726c0 msgr2=0x7f5664160a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.987+0000 7f5668eb3640 1 -- 172.21.15.16:0/1700150808 shutdown_connections 2024-04-25T10:14:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:47.987+0000 7f5668eb3640 1 -- 172.21.15.16:0/1700150808 wait complete. 2024-04-25T10:14:48.002 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:48.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.101+0000 7fd32d763640 1 Processor -- start 2024-04-25T10:14:48.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.102+0000 7fd32d763640 1 -- start start 2024-04-25T10:14:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.102+0000 7fd32d763640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd32815dca0 0x7fd3280ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.102+0000 7fd32d763640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3280d3b90 0x7fd3280d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.102+0000 7fd32d763640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3280cee80 0x7fd3280d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.102+0000 7fd32d763640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3280b6b20 con 0x7fd3280d3b90 2024-04-25T10:14:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.102+0000 7fd32d763640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3280d4090 con 0x7fd32815dca0 2024-04-25T10:14:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.102+0000 7fd32d763640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3280d4370 con 0x7fd3280cee80 2024-04-25T10:14:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.102+0000 7fd3267fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3280d3b90 0x7fd3280d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.103+0000 7fd3267fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3280d3b90 0x7fd3280d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58572/0 (socket says 172.21.15.16:58572) 2024-04-25T10:14:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.103+0000 7fd3267fc640 1 -- 172.21.15.16:0/1575448930 learned_addr learned my addr 172.21.15.16:0/1575448930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.103+0000 7fd3277fe640 1 --2- 172.21.15.16:0/1575448930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3280cee80 0x7fd3280d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.103+0000 7fd326ffd640 1 --2- 172.21.15.16:0/1575448930 >> v2:172.21.15.186:3300/0 conn(0x7fd32815dca0 0x7fd3280ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.103+0000 7fd3267fc640 1 -- 172.21.15.16:0/1575448930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3280cee80 msgr2=0x7fd3280d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.103+0000 7fd3267fc640 1 --2- 172.21.15.16:0/1575448930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3280cee80 0x7fd3280d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.103+0000 7fd3267fc640 1 -- 172.21.15.16:0/1575448930 >> v2:172.21.15.186:3300/0 conn(0x7fd32815dca0 msgr2=0x7fd3280ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.103+0000 7fd3267fc640 1 --2- 172.21.15.16:0/1575448930 >> v2:172.21.15.186:3300/0 conn(0x7fd32815dca0 0x7fd3280ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.103+0000 7fd3267fc640 1 -- 172.21.15.16:0/1575448930 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3280d4650 con 0x7fd3280d3b90 2024-04-25T10:14:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.104+0000 7fd3267fc640 1 --2- 172.21.15.16:0/1575448930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3280d3b90 0x7fd3280d0a20 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fd3280b6340 tx=0x7fd31c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8fd15a65b79666 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.104+0000 7fd325ffb640 1 -- 172.21.15.16:0/1575448930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c037070 con 0x7fd3280d3b90 2024-04-25T10:14:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.104+0000 7fd325ffb640 1 -- 172.21.15.16:0/1575448930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31c032440 con 0x7fd3280d3b90 2024-04-25T10:14:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.104+0000 7fd325ffb640 1 -- 172.21.15.16:0/1575448930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c032810 con 0x7fd3280d3b90 2024-04-25T10:14:48.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.104+0000 7fd32d763640 1 -- 172.21.15.16:0/1575448930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3280d3b90 msgr2=0x7fd3280d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.104+0000 7fd32d763640 1 --2- 172.21.15.16:0/1575448930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3280d3b90 0x7fd3280d0a20 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fd3280b6340 tx=0x7fd31c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.104+0000 7fd32d763640 1 -- 172.21.15.16:0/1575448930 shutdown_connections 2024-04-25T10:14:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.104+0000 7fd32d763640 1 --2- 172.21.15.16:0/1575448930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3280cee80 0x7fd3280d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.104+0000 7fd32d763640 1 --2- 172.21.15.16:0/1575448930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3280d3b90 0x7fd3280d0a20 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.104+0000 7fd32d763640 1 --2- 172.21.15.16:0/1575448930 >> v2:172.21.15.186:3300/0 conn(0x7fd32815dca0 0x7fd3280ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.104+0000 7fd32d763640 1 -- 172.21.15.16:0/1575448930 >> 172.21.15.16:0/1575448930 conn(0x7fd3280726c0 msgr2=0x7fd328072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.104+0000 7fd32d763640 1 -- 172.21.15.16:0/1575448930 shutdown_connections 2024-04-25T10:14:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.104+0000 7fd32d763640 1 -- 172.21.15.16:0/1575448930 wait complete. 2024-04-25T10:14:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.106+0000 7fd32d763640 1 Processor -- start 2024-04-25T10:14:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.106+0000 7fd32d763640 1 -- start start 2024-04-25T10:14:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.106+0000 7fd32d763640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3280cee80 0x7fd3281523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.106+0000 7fd32d763640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3280d3b90 0x7fd328153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.106+0000 7fd32d763640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd32815dca0 0x7fd328156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.106+0000 7fd32d763640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32815e200 con 0x7fd3280d3b90 2024-04-25T10:14:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.106+0000 7fd32d763640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32815e080 con 0x7fd3280cee80 2024-04-25T10:14:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.106+0000 7fd32d763640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3280b2500 con 0x7fd32815dca0 2024-04-25T10:14:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.107+0000 7fd3267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3280d3b90 0x7fd328153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.107+0000 7fd3267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3280d3b90 0x7fd328153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58588/0 (socket says 172.21.15.16:58588) 2024-04-25T10:14:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.107+0000 7fd3267fc640 1 -- 172.21.15.16:0/3958370089 learned_addr learned my addr 172.21.15.16:0/3958370089 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.107+0000 7fd326ffd640 1 --2- 172.21.15.16:0/3958370089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3280cee80 0x7fd3281523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.107+0000 7fd3267fc640 1 -- 172.21.15.16:0/3958370089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd32815dca0 msgr2=0x7fd328156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.107+0000 7fd3277fe640 1 --2- 172.21.15.16:0/3958370089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd32815dca0 0x7fd328156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.107+0000 7fd3267fc640 1 --2- 172.21.15.16:0/3958370089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd32815dca0 0x7fd328156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.107+0000 7fd3267fc640 1 -- 172.21.15.16:0/3958370089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3280cee80 msgr2=0x7fd3281523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.107+0000 7fd3267fc640 1 --2- 172.21.15.16:0/3958370089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3280cee80 0x7fd3281523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.107+0000 7fd3267fc640 1 -- 172.21.15.16:0/3958370089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd328152930 con 0x7fd3280d3b90 2024-04-25T10:14:48.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.107+0000 7fd326ffd640 1 --2- 172.21.15.16:0/3958370089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3280cee80 0x7fd3281523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:48.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.107+0000 7fd3267fc640 1 --2- 172.21.15.16:0/3958370089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3280d3b90 0x7fd328153b90 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fd31c02a220 tx=0x7fd31c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.107+0000 7fd30f7fe640 1 -- 172.21.15.16:0/3958370089 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c041020 con 0x7fd3280d3b90 2024-04-25T10:14:48.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.107+0000 7fd30f7fe640 1 -- 172.21.15.16:0/3958370089 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31c0306c0 con 0x7fd3280d3b90 2024-04-25T10:14:48.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.107+0000 7fd32d763640 1 -- 172.21.15.16:0/3958370089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd328160b90 con 0x7fd3280d3b90 2024-04-25T10:14:48.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.107+0000 7fd30f7fe640 1 -- 172.21.15.16:0/3958370089 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c037040 con 0x7fd3280d3b90 2024-04-25T10:14:48.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.107+0000 7fd32d763640 1 -- 172.21.15.16:0/3958370089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd328161150 con 0x7fd3280d3b90 2024-04-25T10:14:48.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.108+0000 7fd32d763640 1 -- 172.21.15.16:0/3958370089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3280750b0 con 0x7fd3280d3b90 2024-04-25T10:14:48.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.109+0000 7fd30f7fe640 1 -- 172.21.15.16:0/3958370089 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd31c044020 con 0x7fd3280d3b90 2024-04-25T10:14:48.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.109+0000 7fd30f7fe640 1 --2- 172.21.15.16:0/3958370089 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd31406b4d0 0x7fd31406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.109+0000 7fd30f7fe640 1 -- 172.21.15.16:0/3958370089 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd31c0aaa30 con 0x7fd3280d3b90 2024-04-25T10:14:48.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.109+0000 7fd326ffd640 1 --2- 172.21.15.16:0/3958370089 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd31406b4d0 0x7fd31406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.110+0000 7fd326ffd640 1 --2- 172.21.15.16:0/3958370089 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd31406b4d0 0x7fd31406d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd318006fd0 tx=0x7fd318008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.113+0000 7fd30f7fe640 1 -- 172.21.15.16:0/3958370089 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd31c04b030 con 0x7fd3280d3b90 2024-04-25T10:14:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.293+0000 7fd32d763640 1 -- 172.21.15.16:0/3958370089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd3280ce790 con 0x7fd3280d3b90 2024-04-25T10:14:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.293+0000 7fd30f7fe640 1 -- 172.21.15.16:0/3958370089 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd31c079860 con 0x7fd3280d3b90 2024-04-25T10:14:48.299 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:14:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.295+0000 7fd32d763640 1 -- 172.21.15.16:0/3958370089 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd31406b4d0 msgr2=0x7fd31406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.295+0000 7fd32d763640 1 --2- 172.21.15.16:0/3958370089 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd31406b4d0 0x7fd31406d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd318006fd0 tx=0x7fd318008040 comp rx=0 tx=0).stop 2024-04-25T10:14:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.295+0000 7fd32d763640 1 -- 172.21.15.16:0/3958370089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3280d3b90 msgr2=0x7fd328153b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.295+0000 7fd32d763640 1 --2- 172.21.15.16:0/3958370089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3280d3b90 0x7fd328153b90 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fd31c02a220 tx=0x7fd31c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.295+0000 7fd32d763640 1 -- 172.21.15.16:0/3958370089 shutdown_connections 2024-04-25T10:14:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.295+0000 7fd32d763640 1 --2- 172.21.15.16:0/3958370089 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd31406b4d0 0x7fd31406d990 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.295+0000 7fd32d763640 1 --2- 172.21.15.16:0/3958370089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd32815dca0 0x7fd328156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.295+0000 7fd32d763640 1 --2- 172.21.15.16:0/3958370089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3280d3b90 0x7fd328153b90 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.295+0000 7fd32d763640 1 --2- 172.21.15.16:0/3958370089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3280cee80 0x7fd3281523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.295+0000 7fd32d763640 1 -- 172.21.15.16:0/3958370089 >> 172.21.15.16:0/3958370089 conn(0x7fd3280726c0 msgr2=0x7fd3280b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.295+0000 7fd32d763640 1 -- 172.21.15.16:0/3958370089 shutdown_connections 2024-04-25T10:14:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.295+0000 7fd32d763640 1 -- 172.21.15.16:0/3958370089 wait complete. 2024-04-25T10:14:48.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:14:48.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.409+0000 7fe22961f640 1 Processor -- start 2024-04-25T10:14:48.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.409+0000 7fe22961f640 1 -- start start 2024-04-25T10:14:48.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.410+0000 7fe22961f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe2240cc8c0 0x7fe2240cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.410+0000 7fe22961f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe224070440 con 0x7fe22406aba0 2024-04-25T10:14:48.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.410+0000 7fe22961f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2240caed0 con 0x7fe22406f980 2024-04-25T10:14:48.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.410+0000 7fe22961f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22415e280 con 0x7fe2240cc8c0 2024-04-25T10:14:48.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.410+0000 7fe222ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe22406aba0 0x7fe22406af80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56406/0 (socket says 172.21.15.16:56406) 2024-04-25T10:14:48.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.410+0000 7fe222ffd640 1 -- 172.21.15.16:0/588804163 learned_addr learned my addr 172.21.15.16:0/588804163 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:48.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.410+0000 7fe2237fe640 1 --2- 172.21.15.16:0/588804163 >> v2:172.21.15.186:3300/0 conn(0x7fe2240cc8c0 0x7fe2240cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.410+0000 7fe221ffb640 1 -- 172.21.15.16:0/588804163 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3705952753 0 0) 0x7fe224070440 con 0x7fe22406aba0 2024-04-25T10:14:48.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.410+0000 7fe221ffb640 1 -- 172.21.15.16:0/588804163 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe20c003520 con 0x7fe22406aba0 2024-04-25T10:14:48.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.410+0000 7fe221ffb640 1 -- 172.21.15.16:0/588804163 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1149583923 0 0) 0x7fe20c003520 con 0x7fe22406aba0 2024-04-25T10:14:48.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.410+0000 7fe221ffb640 1 -- 172.21.15.16:0/588804163 >> v2:172.21.15.186:3300/0 conn(0x7fe2240cc8c0 msgr2=0x7fe2240cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.410+0000 7fe221ffb640 1 --2- 172.21.15.16:0/588804163 >> v2:172.21.15.186:3300/0 conn(0x7fe2240cc8c0 0x7fe2240cbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.410+0000 7fe221ffb640 1 -- 172.21.15.16:0/588804163 >> v1:172.21.15.186:6789/0 conn(0x7fe22406f980 legacy=0x7fe2240cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.410+0000 7fe221ffb640 1 -- 172.21.15.16:0/588804163 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22406fe80 con 0x7fe22406aba0 2024-04-25T10:14:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.410+0000 7fe221ffb640 1 -- 172.21.15.16:0/588804163 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fe214001670 con 0x7fe22406aba0 2024-04-25T10:14:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.411+0000 7fe221ffb640 1 -- 172.21.15.16:0/588804163 >> v1:172.21.15.16:6789/0 conn(0x7fe22406aba0 legacy=0x7fe22406af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.411+0000 7fe221ffb640 1 --2- 172.21.15.16:0/588804163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c0039b0 0x7fe20c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.411+0000 7fe221ffb640 1 --2- 172.21.15.16:0/588804163 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe20c004800 0x7fe20c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.411+0000 7fe221ffb640 1 --2- 172.21.15.16:0/588804163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe20c0051c0 0x7fe20c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.411+0000 7fe221ffb640 1 -- 172.21.15.16:0/588804163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22415e280 con 0x7fe20c0039b0 2024-04-25T10:14:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.411+0000 7fe221ffb640 1 -- 172.21.15.16:0/588804163 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c00c060 con 0x7fe20c0051c0 2024-04-25T10:14:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.411+0000 7fe221ffb640 1 -- 172.21.15.16:0/588804163 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c00c260 con 0x7fe20c004800 2024-04-25T10:14:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.411+0000 7fe222ffd640 1 --2- 172.21.15.16:0/588804163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c0039b0 0x7fe20c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.411+0000 7fe2227fc640 1 --2- 172.21.15.16:0/588804163 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe20c004800 0x7fe20c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.411+0000 7fe2237fe640 1 --2- 172.21.15.16:0/588804163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe20c0051c0 0x7fe20c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.411+0000 7fe222ffd640 1 -- 172.21.15.16:0/588804163 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe20c004800 msgr2=0x7fe20c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.411+0000 7fe222ffd640 1 --2- 172.21.15.16:0/588804163 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe20c004800 0x7fe20c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.411+0000 7fe222ffd640 1 -- 172.21.15.16:0/588804163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe20c0051c0 msgr2=0x7fe20c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.411+0000 7fe222ffd640 1 --2- 172.21.15.16:0/588804163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe20c0051c0 0x7fe20c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.411+0000 7fe222ffd640 1 -- 172.21.15.16:0/588804163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20c003520 con 0x7fe20c0039b0 2024-04-25T10:14:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.411+0000 7fe2227fc640 1 --2- 172.21.15.16:0/588804163 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe20c004800 0x7fe20c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.411+0000 7fe2237fe640 1 --2- 172.21.15.16:0/588804163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe20c0051c0 0x7fe20c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.411+0000 7fe2237fe640 1 -- 172.21.15.16:0/588804163 reap_dead start 2024-04-25T10:14:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.412+0000 7fe222ffd640 1 --2- 172.21.15.16:0/588804163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c0039b0 0x7fe20c003dd0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fe21400c240 tx=0x7fe20c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc75b58b75f04f73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.412+0000 7fe221ffb640 1 -- 172.21.15.16:0/588804163 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214007d10 con 0x7fe20c0039b0 2024-04-25T10:14:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.412+0000 7fe221ffb640 1 -- 172.21.15.16:0/588804163 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2140058d0 con 0x7fe20c0039b0 2024-04-25T10:14:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.412+0000 7fe221ffb640 1 -- 172.21.15.16:0/588804163 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214005d40 con 0x7fe20c0039b0 2024-04-25T10:14:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.412+0000 7fe22961f640 1 -- 172.21.15.16:0/588804163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c0039b0 msgr2=0x7fe20c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.412+0000 7fe22961f640 1 --2- 172.21.15.16:0/588804163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c0039b0 0x7fe20c003dd0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fe21400c240 tx=0x7fe20c005830 comp rx=0 tx=0).stop 2024-04-25T10:14:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.412+0000 7fe22961f640 1 -- 172.21.15.16:0/588804163 shutdown_connections 2024-04-25T10:14:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.412+0000 7fe22961f640 1 --2- 172.21.15.16:0/588804163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c0039b0 0x7fe20c003dd0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.412+0000 7fe22961f640 1 -- 172.21.15.16:0/588804163 >> 172.21.15.16:0/588804163 conn(0x7fe2240d0750 msgr2=0x7fe2240d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.412+0000 7fe22961f640 1 -- 172.21.15.16:0/588804163 shutdown_connections 2024-04-25T10:14:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.412+0000 7fe22961f640 1 -- 172.21.15.16:0/588804163 wait complete. 2024-04-25T10:14:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.414+0000 7fe22961f640 1 Processor -- start 2024-04-25T10:14:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.414+0000 7fe22961f640 1 -- start start 2024-04-25T10:14:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.414+0000 7fe22961f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c0039b0 0x7fe2240c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.414+0000 7fe22961f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2241512f0 0x7fe22414e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.414+0000 7fe22961f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe224151770 0x7fe22414e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.414+0000 7fe22961f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22406b950 con 0x7fe20c0039b0 2024-04-25T10:14:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.414+0000 7fe22961f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2240cb170 con 0x7fe224151770 2024-04-25T10:14:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.414+0000 7fe22961f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22406bad0 con 0x7fe2241512f0 2024-04-25T10:14:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.415+0000 7fe222ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c0039b0 0x7fe2240c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.415+0000 7fe222ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c0039b0 0x7fe2240c7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58606/0 (socket says 172.21.15.16:58606) 2024-04-25T10:14:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.415+0000 7fe222ffd640 1 -- 172.21.15.16:0/1631572033 learned_addr learned my addr 172.21.15.16:0/1631572033 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.415+0000 7fe2237fe640 1 --2- 172.21.15.16:0/1631572033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe224151770 0x7fe22414e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.415+0000 7fe222ffd640 1 -- 172.21.15.16:0/1631572033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2241512f0 msgr2=0x7fe22414e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.415+0000 7fe2227fc640 1 --2- 172.21.15.16:0/1631572033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2241512f0 0x7fe22414e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.415+0000 7fe222ffd640 1 --2- 172.21.15.16:0/1631572033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2241512f0 0x7fe22414e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.415+0000 7fe222ffd640 1 -- 172.21.15.16:0/1631572033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe224151770 msgr2=0x7fe22414e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.415+0000 7fe222ffd640 1 --2- 172.21.15.16:0/1631572033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe224151770 0x7fe22414e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.415+0000 7fe222ffd640 1 -- 172.21.15.16:0/1631572033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22414ed60 con 0x7fe20c0039b0 2024-04-25T10:14:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.415+0000 7fe2237fe640 1 --2- 172.21.15.16:0/1631572033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe224151770 0x7fe22414e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.415+0000 7fe2227fc640 1 --2- 172.21.15.16:0/1631572033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2241512f0 0x7fe22414e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.415+0000 7fe222ffd640 1 --2- 172.21.15.16:0/1631572033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c0039b0 0x7fe2240c7890 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fe21400c360 tx=0x7fe2240720d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.415+0000 7fe20b7fe640 1 -- 172.21.15.16:0/1631572033 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214040020 con 0x7fe20c0039b0 2024-04-25T10:14:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.415+0000 7fe20b7fe640 1 -- 172.21.15.16:0/1631572033 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe214000ed0 con 0x7fe20c0039b0 2024-04-25T10:14:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.415+0000 7fe22961f640 1 -- 172.21.15.16:0/1631572033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2240c5280 con 0x7fe20c0039b0 2024-04-25T10:14:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.415+0000 7fe20b7fe640 1 -- 172.21.15.16:0/1631572033 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214007d10 con 0x7fe20c0039b0 2024-04-25T10:14:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.415+0000 7fe22961f640 1 -- 172.21.15.16:0/1631572033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2240c5840 con 0x7fe20c0039b0 2024-04-25T10:14:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.416+0000 7fe22961f640 1 -- 172.21.15.16:0/1631572033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2240750b0 con 0x7fe20c0039b0 2024-04-25T10:14:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.417+0000 7fe20b7fe640 1 -- 172.21.15.16:0/1631572033 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe214043020 con 0x7fe20c0039b0 2024-04-25T10:14:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.417+0000 7fe20b7fe640 1 --2- 172.21.15.16:0/1631572033 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe21006b500 0x7fe21006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.417+0000 7fe20b7fe640 1 -- 172.21.15.16:0/1631572033 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe2140aa2a0 con 0x7fe20c0039b0 2024-04-25T10:14:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.418+0000 7fe2227fc640 1 --2- 172.21.15.16:0/1631572033 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe21006b500 0x7fe21006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.418+0000 7fe2227fc640 1 --2- 172.21.15.16:0/1631572033 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe21006b500 0x7fe21006d9c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe218000c00 tx=0x7fe218009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.422+0000 7fe20b7fe640 1 -- 172.21.15.16:0/1631572033 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe21404a020 con 0x7fe20c0039b0 2024-04-25T10:14:48.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.613+0000 7fe22961f640 1 -- 172.21.15.16:0/1631572033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe2240b3050 con 0x7fe20c0039b0 2024-04-25T10:14:48.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.613+0000 7fe20b7fe640 1 -- 172.21.15.16:0/1631572033 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v116) v1 ==== 80+0+120 (secure 0 0 0) 0x7fe214079140 con 0x7fe20c0039b0 2024-04-25T10:14:48.617 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6835/123396308 2024-04-26T10:14:39.639508+0000 2024-04-25T10:14:48.617 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6833/123396308 2024-04-26T10:14:39.639508+0000 2024-04-25T10:14:48.617 INFO:teuthology.orchestra.run.smithi016.stderr:listed 2 entries 2024-04-25T10:14:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.615+0000 7fe22961f640 1 -- 172.21.15.16:0/1631572033 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe21006b500 msgr2=0x7fe21006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.615+0000 7fe22961f640 1 --2- 172.21.15.16:0/1631572033 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe21006b500 0x7fe21006d9c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe218000c00 tx=0x7fe218009040 comp rx=0 tx=0).stop 2024-04-25T10:14:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.615+0000 7fe22961f640 1 -- 172.21.15.16:0/1631572033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c0039b0 msgr2=0x7fe2240c7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.615+0000 7fe22961f640 1 --2- 172.21.15.16:0/1631572033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c0039b0 0x7fe2240c7890 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fe21400c360 tx=0x7fe2240720d0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.615+0000 7fe22961f640 1 -- 172.21.15.16:0/1631572033 shutdown_connections 2024-04-25T10:14:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.615+0000 7fe22961f640 1 --2- 172.21.15.16:0/1631572033 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe21006b500 0x7fe21006d9c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.615+0000 7fe22961f640 1 --2- 172.21.15.16:0/1631572033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe224151770 0x7fe22414e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.615+0000 7fe22961f640 1 --2- 172.21.15.16:0/1631572033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2241512f0 0x7fe22414e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.615+0000 7fe22961f640 1 --2- 172.21.15.16:0/1631572033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c0039b0 0x7fe2240c7890 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.615+0000 7fe22961f640 1 -- 172.21.15.16:0/1631572033 >> 172.21.15.16:0/1631572033 conn(0x7fe2240d0750 msgr2=0x7fe2240b21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.615+0000 7fe22961f640 1 -- 172.21.15.16:0/1631572033 shutdown_connections 2024-04-25T10:14:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.615+0000 7fe22961f640 1 -- 172.21.15.16:0/1631572033 wait complete. 2024-04-25T10:14:48.630 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:14:48.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:48.630 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:48.657 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:14:48.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:48.657 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:14:48.713 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:48.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.850+0000 7fcd14b5c640 1 Processor -- start 2024-04-25T10:14:48.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.850+0000 7fcd14b5c640 1 -- start start 2024-04-25T10:14:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.851+0000 7fcd14b5c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd100d35b0 0x7fcd100b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.851+0000 7fcd14b5c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fcd100b5990 0x7fcd100b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.851+0000 7fcd14b5c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1015e0c0 con 0x7fcd1015dce0 2024-04-25T10:14:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.851+0000 7fcd14b5c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd100d2510 con 0x7fcd100d35b0 2024-04-25T10:14:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.851+0000 7fcd14b5c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd100d27f0 con 0x7fcd100b5990 2024-04-25T10:14:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.851+0000 7fcd0dd9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd100d35b0 0x7fcd100b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.851+0000 7fcd0dd9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd100d35b0 0x7fcd100b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58610/0 (socket says 172.21.15.16:58610) 2024-04-25T10:14:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.851+0000 7fcd0dd9b640 1 -- 172.21.15.16:0/3926516068 learned_addr learned my addr 172.21.15.16:0/3926516068 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.852+0000 7fcd0ed9d640 1 --2- 172.21.15.16:0/3926516068 >> v2:172.21.15.186:3300/0 conn(0x7fcd100b5990 0x7fcd100b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.852+0000 7fcd0dd9b640 1 -- 172.21.15.16:0/3926516068 >> v2:172.21.15.186:3300/0 conn(0x7fcd100b5990 msgr2=0x7fcd100b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.852+0000 7fcd0dd9b640 1 --2- 172.21.15.16:0/3926516068 >> v2:172.21.15.186:3300/0 conn(0x7fcd100b5990 0x7fcd100b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.852+0000 7fcd0dd9b640 1 -- 172.21.15.16:0/3926516068 >> v1:172.21.15.186:6789/0 conn(0x7fcd1015dce0 legacy=0x7fcd100d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.852+0000 7fcd0dd9b640 1 -- 172.21.15.16:0/3926516068 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd100d2ad0 con 0x7fcd100d35b0 2024-04-25T10:14:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.852+0000 7fcd0dd9b640 1 --2- 172.21.15.16:0/3926516068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd100d35b0 0x7fcd100b5450 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fcd100b3390 tx=0x7fccf802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e442afe11b4ab650 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.853+0000 7fcd0d59a640 1 -- 172.21.15.16:0/3926516068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf8037070 con 0x7fcd100d35b0 2024-04-25T10:14:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.853+0000 7fcd0d59a640 1 -- 172.21.15.16:0/3926516068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf8032440 con 0x7fcd100d35b0 2024-04-25T10:14:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.853+0000 7fcd0d59a640 1 -- 172.21.15.16:0/3926516068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf8032810 con 0x7fcd100d35b0 2024-04-25T10:14:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.853+0000 7fcd14b5c640 1 -- 172.21.15.16:0/3926516068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd100d35b0 msgr2=0x7fcd100b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.853+0000 7fcd14b5c640 1 --2- 172.21.15.16:0/3926516068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd100d35b0 0x7fcd100b5450 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fcd100b3390 tx=0x7fccf802a270 comp rx=0 tx=0).stop 2024-04-25T10:14:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.853+0000 7fcd14b5c640 1 -- 172.21.15.16:0/3926516068 shutdown_connections 2024-04-25T10:14:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.853+0000 7fcd14b5c640 1 --2- 172.21.15.16:0/3926516068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd100d35b0 0x7fcd100b5450 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.853+0000 7fcd14b5c640 1 --2- 172.21.15.16:0/3926516068 >> v2:172.21.15.186:3300/0 conn(0x7fcd100b5990 0x7fcd100b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.853+0000 7fcd14b5c640 1 -- 172.21.15.16:0/3926516068 >> 172.21.15.16:0/3926516068 conn(0x7fcd10072900 msgr2=0x7fcd100d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.853+0000 7fcd14b5c640 1 -- 172.21.15.16:0/3926516068 shutdown_connections 2024-04-25T10:14:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.853+0000 7fcd14b5c640 1 -- 172.21.15.16:0/3926516068 wait complete. 2024-04-25T10:14:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.855+0000 7fcd14b5c640 1 Processor -- start 2024-04-25T10:14:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.855+0000 7fcd14b5c640 1 -- start start 2024-04-25T10:14:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.855+0000 7fcd14b5c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd100b5990 0x7fcd1014dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.855+0000 7fcd14b5c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd100d35b0 0x7fcd1015f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.855+0000 7fcd14b5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd1015dce0 0x7fcd100b7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.855+0000 7fcd14b5c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100d2160 con 0x7fcd1015dce0 2024-04-25T10:14:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.855+0000 7fcd14b5c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100d3090 con 0x7fcd100d35b0 2024-04-25T10:14:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.855+0000 7fcd14b5c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd10160c80 con 0x7fcd100b5990 2024-04-25T10:14:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.855+0000 7fcd0ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd1015dce0 0x7fcd100b7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.856+0000 7fcd0ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd1015dce0 0x7fcd100b7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58622/0 (socket says 172.21.15.16:58622) 2024-04-25T10:14:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.856+0000 7fcd0ed9d640 1 -- 172.21.15.16:0/3527657889 learned_addr learned my addr 172.21.15.16:0/3527657889 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.856+0000 7fcd0e59c640 1 --2- 172.21.15.16:0/3527657889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd100b5990 0x7fcd1014dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.856+0000 7fcd0dd9b640 1 --2- 172.21.15.16:0/3527657889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd100d35b0 0x7fcd1015f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.856+0000 7fcd0ed9d640 1 -- 172.21.15.16:0/3527657889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd100b5990 msgr2=0x7fcd1014dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.856+0000 7fcd0ed9d640 1 --2- 172.21.15.16:0/3527657889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd100b5990 0x7fcd1014dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.856+0000 7fcd0ed9d640 1 -- 172.21.15.16:0/3527657889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd100d35b0 msgr2=0x7fcd1015f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.856+0000 7fcd0ed9d640 1 --2- 172.21.15.16:0/3527657889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd100d35b0 0x7fcd1015f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.856+0000 7fcd0ed9d640 1 -- 172.21.15.16:0/3527657889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1014e1b0 con 0x7fcd1015dce0 2024-04-25T10:14:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.856+0000 7fcd0ed9d640 1 --2- 172.21.15.16:0/3527657889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd1015dce0 0x7fcd100b7010 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fcd00007e40 tx=0x7fcd0000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.856+0000 7fcd06ffd640 1 -- 172.21.15.16:0/3527657889 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00014620 con 0x7fcd1015dce0 2024-04-25T10:14:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.856+0000 7fcd06ffd640 1 -- 172.21.15.16:0/3527657889 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd00019070 con 0x7fcd1015dce0 2024-04-25T10:14:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.856+0000 7fcd14b5c640 1 -- 172.21.15.16:0/3527657889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd10160070 con 0x7fcd1015dce0 2024-04-25T10:14:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.856+0000 7fcd06ffd640 1 -- 172.21.15.16:0/3527657889 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00014a20 con 0x7fcd1015dce0 2024-04-25T10:14:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.856+0000 7fcd14b5c640 1 -- 172.21.15.16:0/3527657889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd10160630 con 0x7fcd1015dce0 2024-04-25T10:14:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.857+0000 7fcd14b5c640 1 -- 172.21.15.16:0/3527657889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd100749b0 con 0x7fcd1015dce0 2024-04-25T10:14:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.858+0000 7fcd06ffd640 1 -- 172.21.15.16:0/3527657889 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd00014bc0 con 0x7fcd1015dce0 2024-04-25T10:14:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.858+0000 7fcd06ffd640 1 --2- 172.21.15.16:0/3527657889 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fccf006b4d0 0x7fccf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.858+0000 7fcd06ffd640 1 -- 172.21.15.16:0/3527657889 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcd0008aae0 con 0x7fcd1015dce0 2024-04-25T10:14:48.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.858+0000 7fcd0e59c640 1 --2- 172.21.15.16:0/3527657889 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fccf006b4d0 0x7fccf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.859+0000 7fcd0e59c640 1 --2- 172.21.15.16:0/3527657889 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fccf006b4d0 0x7fccf006d990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fcd08001500 tx=0x7fcd08009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:48.862+0000 7fcd06ffd640 1 -- 172.21.15.16:0/3527657889 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd00059900 con 0x7fcd1015dce0 2024-04-25T10:14:49.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.078+0000 7fcd14b5c640 1 -- 172.21.15.16:0/3527657889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd100c7030 con 0x7fcd1015dce0 2024-04-25T10:14:49.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.079+0000 7fcd06ffd640 1 -- 172.21.15.16:0/3527657889 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fcd00059200 con 0x7fcd1015dce0 2024-04-25T10:14:49.083 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:49.083 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2871061122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2871061122},{"type":"v1","addr":"172.21.15.186:6836","nonce":2871061122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1871507206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1871507206},{"type":"v1","addr":"172.21.15.186:6837","nonce":1871507206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4064412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4064412586},{"type":"v1","addr":"172.21.15.16:6834","nonce":4064412586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:26.751060+0000","modified":"2024-04-25T10:14:42.496103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9096":{"gid":9096,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6835/2697110279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2697110279},{"type":"v1","addr":"172.21.15.16:6835","nonce":2697110279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:49.083 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 89 2024-04-25T10:14:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.081+0000 7fcd14b5c640 1 -- 172.21.15.16:0/3527657889 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fccf006b4d0 msgr2=0x7fccf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.081+0000 7fcd14b5c640 1 --2- 172.21.15.16:0/3527657889 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fccf006b4d0 0x7fccf006d990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fcd08001500 tx=0x7fcd08009040 comp rx=0 tx=0).stop 2024-04-25T10:14:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.081+0000 7fcd14b5c640 1 -- 172.21.15.16:0/3527657889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd1015dce0 msgr2=0x7fcd100b7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.081+0000 7fcd14b5c640 1 --2- 172.21.15.16:0/3527657889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd1015dce0 0x7fcd100b7010 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fcd00007e40 tx=0x7fcd0000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.081+0000 7fcd14b5c640 1 -- 172.21.15.16:0/3527657889 shutdown_connections 2024-04-25T10:14:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.081+0000 7fcd14b5c640 1 --2- 172.21.15.16:0/3527657889 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fccf006b4d0 0x7fccf006d990 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.081+0000 7fcd14b5c640 1 --2- 172.21.15.16:0/3527657889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd1015dce0 0x7fcd100b7010 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.081+0000 7fcd14b5c640 1 --2- 172.21.15.16:0/3527657889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd100d35b0 0x7fcd1015f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.081+0000 7fcd14b5c640 1 --2- 172.21.15.16:0/3527657889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd100b5990 0x7fcd1014dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.081+0000 7fcd14b5c640 1 -- 172.21.15.16:0/3527657889 >> 172.21.15.16:0/3527657889 conn(0x7fcd10072900 msgr2=0x7fcd100b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.081+0000 7fcd14b5c640 1 -- 172.21.15.16:0/3527657889 shutdown_connections 2024-04-25T10:14:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.081+0000 7fcd14b5c640 1 -- 172.21.15.16:0/3527657889 wait complete. 2024-04-25T10:14:49.096 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parent mytag123 2024-04-25T10:14:49.215 INFO:teuthology.orchestra.run.smithi016.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-04-25T10:14:49.216 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-04-25T10:14:49.216 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000003.00000000 scrub_tag 2024-04-25T10:14:49.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.255+0000 7f3b85d1bf40 1 Processor -- start 2024-04-25T10:14:49.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.255+0000 7f3b85d1bf40 1 -- start start 2024-04-25T10:14:49.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.256+0000 7f3b85d1bf40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b700022a0 0x55ad26b5c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.256+0000 7f3b85d1bf40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ad26a829a0 0x55ad26b50760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.256+0000 7f3b85d1bf40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad26a82e20 0x55ad26b50ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.256+0000 7f3b85d1bf40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad26a2bdf0 con 0x55ad26a82e20 2024-04-25T10:14:49.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.256+0000 7f3b85d1bf40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad26b5b2f0 con 0x7f3b700022a0 2024-04-25T10:14:49.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.256+0000 7f3b85d1bf40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad26b5c0e0 con 0x55ad26a829a0 2024-04-25T10:14:49.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.256+0000 7f3b8401d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad26a82e20 0x55ad26b50ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.256+0000 7f3b8401d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad26a82e20 0x55ad26b50ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58630/0 (socket says 172.21.15.16:58630) 2024-04-25T10:14:49.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.256+0000 7f3b8401d640 1 -- 172.21.15.16:0/332503831 learned_addr learned my addr 172.21.15.16:0/332503831 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:49.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.256+0000 7f3b8301b640 1 --2- 172.21.15.16:0/332503831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ad26a829a0 0x55ad26b50760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.256+0000 7f3b8381c640 1 --2- 172.21.15.16:0/332503831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b700022a0 0x55ad26b5c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.256+0000 7f3b8401d640 1 -- 172.21.15.16:0/332503831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ad26a829a0 msgr2=0x55ad26b50760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.256+0000 7f3b8401d640 1 --2- 172.21.15.16:0/332503831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ad26a829a0 0x55ad26b50760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.256+0000 7f3b8401d640 1 -- 172.21.15.16:0/332503831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b700022a0 msgr2=0x55ad26b5c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.256+0000 7f3b8401d640 1 --2- 172.21.15.16:0/332503831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b700022a0 0x55ad26b5c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.256+0000 7f3b8401d640 1 -- 172.21.15.16:0/332503831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad269b1c50 con 0x55ad26a82e20 2024-04-25T10:14:49.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.257+0000 7f3b8401d640 1 --2- 172.21.15.16:0/332503831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad26a82e20 0x55ad26b50ca0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x55ad26acf420 tx=0x7f3b7c005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719df644acaffa16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.257+0000 7f3b82019640 1 -- 172.21.15.16:0/332503831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c01a070 con 0x55ad26a82e20 2024-04-25T10:14:49.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.257+0000 7f3b82019640 1 -- 172.21.15.16:0/332503831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7c015660 con 0x55ad26a82e20 2024-04-25T10:14:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.257+0000 7f3b82019640 1 -- 172.21.15.16:0/332503831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c015a50 con 0x55ad26a82e20 2024-04-25T10:14:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.257+0000 7f3b85d1bf40 1 -- 172.21.15.16:0/332503831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad26a82e20 msgr2=0x55ad26b50ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.257+0000 7f3b85d1bf40 1 --2- 172.21.15.16:0/332503831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad26a82e20 0x55ad26b50ca0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x55ad26acf420 tx=0x7f3b7c005a00 comp rx=0 tx=0).stop 2024-04-25T10:14:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.257+0000 7f3b85d1bf40 1 -- 172.21.15.16:0/332503831 shutdown_connections 2024-04-25T10:14:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.257+0000 7f3b85d1bf40 1 --2- 172.21.15.16:0/332503831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad26a82e20 0x55ad26b50ca0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.257+0000 7f3b85d1bf40 1 --2- 172.21.15.16:0/332503831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ad26a829a0 0x55ad26b50760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.257+0000 7f3b85d1bf40 1 --2- 172.21.15.16:0/332503831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b700022a0 0x55ad26b5c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.257+0000 7f3b85d1bf40 1 -- 172.21.15.16:0/332503831 >> 172.21.15.16:0/332503831 conn(0x55ad26aaf3d0 msgr2=0x55ad26b5b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.257+0000 7f3b85d1bf40 1 -- 172.21.15.16:0/332503831 shutdown_connections 2024-04-25T10:14:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.257+0000 7f3b85d1bf40 1 -- 172.21.15.16:0/332503831 wait complete. 2024-04-25T10:14:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.258+0000 7f3b85d1bf40 1 Processor -- start 2024-04-25T10:14:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.258+0000 7f3b85d1bf40 1 -- start start 2024-04-25T10:14:49.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:49.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b85d1bf40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad26a829a0 0x55ad26a86b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b85d1bf40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ad26a82e20 0x55ad26bf0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b85d1bf40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b700022a0 0x55ad26b5c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b85d1bf40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad26a85b40 con 0x55ad26a829a0 2024-04-25T10:14:49.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b85d1bf40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad26b5be00 con 0x7f3b700022a0 2024-04-25T10:14:49.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b85d1bf40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad26a85e20 con 0x55ad26a82e20 2024-04-25T10:14:49.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b8381c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad26a829a0 0x55ad26a86b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b8381c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad26a829a0 0x55ad26a86b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58640/0 (socket says 172.21.15.16:58640) 2024-04-25T10:14:49.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b8381c640 1 -- 172.21.15.16:0/2173449883 learned_addr learned my addr 172.21.15.16:0/2173449883 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:49.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b8401d640 1 --2- 172.21.15.16:0/2173449883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b700022a0 0x55ad26b5c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b8301b640 1 --2- 172.21.15.16:0/2173449883 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ad26a82e20 0x55ad26bf0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b8381c640 1 -- 172.21.15.16:0/2173449883 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ad26a82e20 msgr2=0x55ad26bf0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b8381c640 1 --2- 172.21.15.16:0/2173449883 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55ad26a82e20 0x55ad26bf0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b8381c640 1 -- 172.21.15.16:0/2173449883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b700022a0 msgr2=0x55ad26b5c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b8381c640 1 --2- 172.21.15.16:0/2173449883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b700022a0 0x55ad26b5c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b8381c640 1 -- 172.21.15.16:0/2173449883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad26b54110 con 0x55ad26a829a0 2024-04-25T10:14:49.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b8381c640 1 --2- 172.21.15.16:0/2173449883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad26a829a0 0x55ad26a86b40 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f3b74026110 tx=0x55ad26b5b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b81017640 1 -- 172.21.15.16:0/2173449883 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b74001ff0 con 0x55ad26a829a0 2024-04-25T10:14:49.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b81017640 1 -- 172.21.15.16:0/2173449883 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b74006270 con 0x55ad26a829a0 2024-04-25T10:14:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b85d1bf40 1 -- 172.21.15.16:0/2173449883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ad26b543b0 con 0x55ad26a829a0 2024-04-25T10:14:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.259+0000 7f3b81017640 1 -- 172.21.15.16:0/2173449883 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7404db70 con 0x55ad26a829a0 2024-04-25T10:14:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.260+0000 7f3b85d1bf40 1 -- 172.21.15.16:0/2173449883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad26b54970 con 0x55ad26a829a0 2024-04-25T10:14:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.261+0000 7f3b81017640 1 -- 172.21.15.16:0/2173449883 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b7404dd10 con 0x55ad26a829a0 2024-04-25T10:14:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.261+0000 7f3b81017640 1 --2- 172.21.15.16:0/2173449883 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b64069eb0 0x7f3b6406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.261+0000 7f3b81017640 1 -- 172.21.15.16:0/2173449883 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3b740cc710 con 0x55ad26a829a0 2024-04-25T10:14:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.262+0000 7f3b8301b640 1 --2- 172.21.15.16:0/2173449883 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b64069eb0 0x7f3b6406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.262+0000 7f3b85d1bf40 1 --2- 172.21.15.16:0/2173449883 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55ad26b51ae0 0x55ad26b51f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.262+0000 7f3b85d1bf40 1 -- 172.21.15.16:0/2173449883 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:1 17.32 17:4def72d4:::10000000003.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) v9 -- 0x55ad26b524b0 con 0x55ad26b51ae0 2024-04-25T10:14:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.262+0000 7f3b8401d640 1 --2- 172.21.15.16:0/2173449883 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55ad26b51ae0 0x55ad26b51f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.262+0000 7f3b8401d640 1 --2- 172.21.15.16:0/2173449883 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55ad26b51ae0 0x55ad26b51f70 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.262+0000 7f3b8301b640 1 --2- 172.21.15.16:0/2173449883 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b64069eb0 0x7f3b6406c370 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x55ad26acb5e0 tx=0x7f3b6c00c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.262+0000 7f3b8401d640 1 -- 172.21.15.16:0/2173449883 <== osd.0 v2:172.21.15.16:6816/2167750256 1 ==== osd_op_reply(1 10000000003.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3b7c015b20 con 0x55ad26b51ae0 2024-04-25T10:14:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:error getting xattr cephfs_data/10000000003.00000000/scrub_tag: (61) No data available 2024-04-25T10:14:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.262+0000 7f3b85d1bf40 1 -- 172.21.15.16:0/2173449883 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55ad26b51ae0 msgr2=0x55ad26b51f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.262+0000 7f3b85d1bf40 1 --2- 172.21.15.16:0/2173449883 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55ad26b51ae0 0x55ad26b51f70 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.262+0000 7f3b85d1bf40 1 -- 172.21.15.16:0/2173449883 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b64069eb0 msgr2=0x7f3b6406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.262+0000 7f3b85d1bf40 1 --2- 172.21.15.16:0/2173449883 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b64069eb0 0x7f3b6406c370 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x55ad26acb5e0 tx=0x7f3b6c00c040 comp rx=0 tx=0).stop 2024-04-25T10:14:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.262+0000 7f3b85d1bf40 1 -- 172.21.15.16:0/2173449883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad26a829a0 msgr2=0x55ad26a86b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.262+0000 7f3b85d1bf40 1 --2- 172.21.15.16:0/2173449883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad26a829a0 0x55ad26a86b40 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f3b74026110 tx=0x55ad26b5b590 comp rx=0 tx=0).stop 2024-04-25T10:14:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.262+0000 7f3b8401d640 1 -- 172.21.15.16:0/2173449883 reap_dead start 2024-04-25T10:14:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.262+0000 7f3b85d1bf40 1 -- 172.21.15.16:0/2173449883 shutdown_connections 2024-04-25T10:14:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.262+0000 7f3b85d1bf40 1 -- 172.21.15.16:0/2173449883 >> 172.21.15.16:0/2173449883 conn(0x55ad26aaf3d0 msgr2=0x55ad26a6f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.263+0000 7f3b85d1bf40 1 -- 172.21.15.16:0/2173449883 shutdown_connections 2024-04-25T10:14:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.263+0000 7f3b85d1bf40 1 -- 172.21.15.16:0/2173449883 wait complete. 2024-04-25T10:14:49.273 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-25T10:14:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.348+0000 7f06f9c0bf40 1 Processor -- start 2024-04-25T10:14:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.348+0000 7f06f9c0bf40 1 -- start start 2024-04-25T10:14:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.349+0000 7f06f9c0bf40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdc58acb00 0x55bdc594f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.349+0000 7f06f9c0bf40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bdc58acee0 0x55bdc5945f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.349+0000 7f06f9c0bf40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bdc5955e50 0x55bdc594dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.349+0000 7f06f9c0bf40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdc5823df0 con 0x55bdc58acb00 2024-04-25T10:14:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.349+0000 7f06f9c0bf40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdc5827d50 con 0x55bdc58acee0 2024-04-25T10:14:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.349+0000 7f06f9c0bf40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdc5952540 con 0x55bdc5955e50 2024-04-25T10:14:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.349+0000 7f06f6a30640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdc58acb00 0x55bdc594f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.349+0000 7f06f6a30640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdc58acb00 0x55bdc594f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58666/0 (socket says 172.21.15.16:58666) 2024-04-25T10:14:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.349+0000 7f06f6a30640 1 -- 172.21.15.16:0/1386084091 learned_addr learned my addr 172.21.15.16:0/1386084091 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.349+0000 7f06f6a30640 1 -- 172.21.15.16:0/1386084091 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bdc5955e50 msgr2=0x55bdc594dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.349+0000 7f06f6a30640 1 --2- 172.21.15.16:0/1386084091 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bdc5955e50 0x55bdc594dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.349+0000 7f06f6a30640 1 -- 172.21.15.16:0/1386084091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bdc58acee0 msgr2=0x55bdc5945f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.349+0000 7f06f6a30640 1 --2- 172.21.15.16:0/1386084091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bdc58acee0 0x55bdc5945f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.349+0000 7f06f6a30640 1 -- 172.21.15.16:0/1386084091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdc58a7e30 con 0x55bdc58acb00 2024-04-25T10:14:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.349+0000 7f06f6a30640 1 --2- 172.21.15.16:0/1386084091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdc58acb00 0x55bdc594f1e0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f06f0002de0 tx=0x7f06f000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b0b6ce72a24428 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.349+0000 7f06f522d640 1 -- 172.21.15.16:0/1386084091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f000f400 con 0x55bdc58acb00 2024-04-25T10:14:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.349+0000 7f06f522d640 1 -- 172.21.15.16:0/1386084091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f0014070 con 0x55bdc58acb00 2024-04-25T10:14:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.349+0000 7f06f522d640 1 -- 172.21.15.16:0/1386084091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f000f7e0 con 0x55bdc58acb00 2024-04-25T10:14:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.350+0000 7f06f9c0bf40 1 -- 172.21.15.16:0/1386084091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdc58acb00 msgr2=0x55bdc594f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.350+0000 7f06f9c0bf40 1 --2- 172.21.15.16:0/1386084091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdc58acb00 0x55bdc594f1e0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f06f0002de0 tx=0x7f06f000b5b0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.350+0000 7f06f9c0bf40 1 -- 172.21.15.16:0/1386084091 shutdown_connections 2024-04-25T10:14:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.350+0000 7f06f9c0bf40 1 --2- 172.21.15.16:0/1386084091 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bdc5955e50 0x55bdc594dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.350+0000 7f06f9c0bf40 1 --2- 172.21.15.16:0/1386084091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bdc58acee0 0x55bdc5945f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.350+0000 7f06f9c0bf40 1 --2- 172.21.15.16:0/1386084091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdc58acb00 0x55bdc594f1e0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.350+0000 7f06f9c0bf40 1 -- 172.21.15.16:0/1386084091 >> 172.21.15.16:0/1386084091 conn(0x55bdc58a73d0 msgr2=0x55bdc59517f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.350+0000 7f06f9c0bf40 1 -- 172.21.15.16:0/1386084091 shutdown_connections 2024-04-25T10:14:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.350+0000 7f06f9c0bf40 1 -- 172.21.15.16:0/1386084091 wait complete. 2024-04-25T10:14:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.350+0000 7f06f9c0bf40 1 Processor -- start 2024-04-25T10:14:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.351+0000 7f06f9c0bf40 1 -- start start 2024-04-25T10:14:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.351+0000 7f06f9c0bf40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdc58acb00 0x55bdc59437a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.351+0000 7f06f9c0bf40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bdc58acee0 0x55bdc5943ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.351+0000 7f06f9c0bf40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bdc5955e50 0x55bdc5952d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.351+0000 7f06f9c0bf40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdc5952080 con 0x55bdc58acb00 2024-04-25T10:14:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.351+0000 7f06f9c0bf40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdc58c2630 con 0x55bdc5955e50 2024-04-25T10:14:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.351+0000 7f06f9c0bf40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdc58c2b80 con 0x55bdc58acee0 2024-04-25T10:14:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.351+0000 7f06f6a30640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdc58acb00 0x55bdc59437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.351+0000 7f06f6a30640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdc58acb00 0x55bdc59437a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58682/0 (socket says 172.21.15.16:58682) 2024-04-25T10:14:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.351+0000 7f06f6a30640 1 -- 172.21.15.16:0/744308325 learned_addr learned my addr 172.21.15.16:0/744308325 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.351+0000 7f06f622f640 1 --2- 172.21.15.16:0/744308325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bdc58acee0 0x55bdc5943ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.351+0000 7f06f7231640 1 --2- 172.21.15.16:0/744308325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bdc5955e50 0x55bdc5952d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.352+0000 7f06f6a30640 1 -- 172.21.15.16:0/744308325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bdc58acee0 msgr2=0x55bdc5943ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.352+0000 7f06f6a30640 1 --2- 172.21.15.16:0/744308325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bdc58acee0 0x55bdc5943ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.352+0000 7f06f6a30640 1 -- 172.21.15.16:0/744308325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bdc5955e50 msgr2=0x55bdc5952d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.352+0000 7f06f6a30640 1 --2- 172.21.15.16:0/744308325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bdc5955e50 0x55bdc5952d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.352+0000 7f06f6a30640 1 -- 172.21.15.16:0/744308325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdc5944e90 con 0x55bdc58acb00 2024-04-25T10:14:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.352+0000 7f06f7231640 1 --2- 172.21.15.16:0/744308325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bdc5955e50 0x55bdc5952d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.352+0000 7f06f6a30640 1 --2- 172.21.15.16:0/744308325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdc58acb00 0x55bdc59437a0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x55bdc587f220 tx=0x7f06f0001fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.352+0000 7f06effff640 1 -- 172.21.15.16:0/744308325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f0028070 con 0x55bdc58acb00 2024-04-25T10:14:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.352+0000 7f06effff640 1 -- 172.21.15.16:0/744308325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f001be10 con 0x55bdc58acb00 2024-04-25T10:14:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.352+0000 7f06f9c0bf40 1 -- 172.21.15.16:0/744308325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bdc59445b0 con 0x55bdc58acb00 2024-04-25T10:14:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.352+0000 7f06effff640 1 -- 172.21.15.16:0/744308325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f00196c0 con 0x55bdc58acb00 2024-04-25T10:14:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.352+0000 7f06f9c0bf40 1 -- 172.21.15.16:0/744308325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdc587eca0 con 0x55bdc58acb00 2024-04-25T10:14:49.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.353+0000 7f06effff640 1 -- 172.21.15.16:0/744308325 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06f002b070 con 0x55bdc58acb00 2024-04-25T10:14:49.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.354+0000 7f06effff640 1 --2- 172.21.15.16:0/744308325 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f06d8069eb0 0x7f06d806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.354+0000 7f06effff640 1 -- 172.21.15.16:0/744308325 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f06f008a370 con 0x55bdc58acb00 2024-04-25T10:14:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.354+0000 7f06f9c0bf40 1 --2- 172.21.15.16:0/744308325 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55bdc58c60e0 0x55bdc5953290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.354+0000 7f06f622f640 1 --2- 172.21.15.16:0/744308325 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f06d8069eb0 0x7f06d806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.354+0000 7f06f9c0bf40 1 -- 172.21.15.16:0/744308325 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:1 17.12 17:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) v9 -- 0x55bdc59557f0 con 0x55bdc58c60e0 2024-04-25T10:14:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.354+0000 7f06f7231640 1 --2- 172.21.15.16:0/744308325 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55bdc58c60e0 0x55bdc5953290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.354+0000 7f06f622f640 1 --2- 172.21.15.16:0/744308325 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f06d8069eb0 0x7f06d806c370 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f06e0057000 tx=0x7f06e0051000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.355+0000 7f06f7231640 1 --2- 172.21.15.16:0/744308325 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55bdc58c60e0 0x55bdc5953290 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.355+0000 7f06f7231640 1 -- 172.21.15.16:0/744308325 <== osd.7 v2:172.21.15.186:6808/1627307043 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f06e8003450 con 0x55bdc58c60e0 2024-04-25T10:14:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.355+0000 7f06f9c0bf40 1 -- 172.21.15.16:0/744308325 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55bdc58c60e0 msgr2=0x55bdc5953290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.355+0000 7f06f9c0bf40 1 --2- 172.21.15.16:0/744308325 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55bdc58c60e0 0x55bdc5953290 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.355+0000 7f06f9c0bf40 1 -- 172.21.15.16:0/744308325 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f06d8069eb0 msgr2=0x7f06d806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.355+0000 7f06f9c0bf40 1 --2- 172.21.15.16:0/744308325 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f06d8069eb0 0x7f06d806c370 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f06e0057000 tx=0x7f06e0051000 comp rx=0 tx=0).stop 2024-04-25T10:14:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.355+0000 7f06f9c0bf40 1 -- 172.21.15.16:0/744308325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdc58acb00 msgr2=0x55bdc59437a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.355+0000 7f06f9c0bf40 1 --2- 172.21.15.16:0/744308325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdc58acb00 0x55bdc59437a0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x55bdc587f220 tx=0x7f06f0001fe0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.355+0000 7f06f7231640 1 -- 172.21.15.16:0/744308325 reap_dead start 2024-04-25T10:14:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.355+0000 7f06f9c0bf40 1 -- 172.21.15.16:0/744308325 shutdown_connections 2024-04-25T10:14:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.355+0000 7f06f9c0bf40 1 -- 172.21.15.16:0/744308325 >> 172.21.15.16:0/744308325 conn(0x55bdc58a73d0 msgr2=0x55bdc5945b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.355+0000 7f06f9c0bf40 1 -- 172.21.15.16:0/744308325 shutdown_connections 2024-04-25T10:14:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.355+0000 7f06f9c0bf40 1 -- 172.21.15.16:0/744308325 wait complete. 2024-04-25T10:14:49.365 INFO:teuthology.orchestra.run.smithi016.stdout:mytag123 2024-04-25T10:14:49.365 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-04-25T10:14:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.445+0000 7f847d287f40 1 Processor -- start 2024-04-25T10:14:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.445+0000 7f847d287f40 1 -- start start 2024-04-25T10:14:49.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.446+0000 7f847d287f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a7ea2aa570 0x55a7ea3476b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.446+0000 7f847d287f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a7ea350d90 0x55a7ea27a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.446+0000 7f847d287f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ea3516b0 0x55a7ea27aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.446+0000 7f847d287f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ea2ab530 con 0x55a7ea3516b0 2024-04-25T10:14:49.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.446+0000 7f847d287f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ea223df0 con 0x55a7ea350d90 2024-04-25T10:14:49.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.446+0000 7f847d287f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7ea2ab7d0 con 0x55a7ea2aa570 2024-04-25T10:14:49.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.446+0000 7f847b58b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ea3516b0 0x55a7ea27aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.446+0000 7f847b58b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ea3516b0 0x55a7ea27aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58686/0 (socket says 172.21.15.16:58686) 2024-04-25T10:14:49.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.446+0000 7f847b58b640 1 -- 172.21.15.16:0/991312668 learned_addr learned my addr 172.21.15.16:0/991312668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:49.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.446+0000 7f847a589640 1 --2- 172.21.15.16:0/991312668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a7ea350d90 0x55a7ea27a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.446+0000 7f847ad8a640 1 --2- 172.21.15.16:0/991312668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a7ea2aa570 0x55a7ea3476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.446+0000 7f847b58b640 1 -- 172.21.15.16:0/991312668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a7ea2aa570 msgr2=0x55a7ea3476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.446+0000 7f847b58b640 1 --2- 172.21.15.16:0/991312668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a7ea2aa570 0x55a7ea3476b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.446+0000 7f847b58b640 1 -- 172.21.15.16:0/991312668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a7ea350d90 msgr2=0x55a7ea27a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.446+0000 7f847b58b640 1 --2- 172.21.15.16:0/991312668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a7ea350d90 0x55a7ea27a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.446+0000 7f847b58b640 1 -- 172.21.15.16:0/991312668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7ea1a9c50 con 0x55a7ea3516b0 2024-04-25T10:14:49.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.446+0000 7f847b58b640 1 --2- 172.21.15.16:0/991312668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ea3516b0 0x55a7ea27aea0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f846c00a870 tx=0x7f846c00ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f0ab6209456bfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.447+0000 7f8479587640 1 -- 172.21.15.16:0/991312668 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c00d9c0 con 0x55a7ea3516b0 2024-04-25T10:14:49.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.447+0000 7f8479587640 1 -- 172.21.15.16:0/991312668 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846c00db60 con 0x55a7ea3516b0 2024-04-25T10:14:49.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.447+0000 7f8479587640 1 -- 172.21.15.16:0/991312668 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c012470 con 0x55a7ea3516b0 2024-04-25T10:14:49.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.447+0000 7f847d287f40 1 -- 172.21.15.16:0/991312668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ea3516b0 msgr2=0x55a7ea27aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.447+0000 7f847d287f40 1 --2- 172.21.15.16:0/991312668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ea3516b0 0x55a7ea27aea0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f846c00a870 tx=0x7f846c00ad30 comp rx=0 tx=0).stop 2024-04-25T10:14:49.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.447+0000 7f847d287f40 1 -- 172.21.15.16:0/991312668 shutdown_connections 2024-04-25T10:14:49.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.447+0000 7f847d287f40 1 --2- 172.21.15.16:0/991312668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ea3516b0 0x55a7ea27aea0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.447+0000 7f847d287f40 1 --2- 172.21.15.16:0/991312668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a7ea350d90 0x55a7ea27a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.447+0000 7f847d287f40 1 --2- 172.21.15.16:0/991312668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a7ea2aa570 0x55a7ea3476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.447+0000 7f847d287f40 1 -- 172.21.15.16:0/991312668 >> 172.21.15.16:0/991312668 conn(0x55a7ea2a73d0 msgr2=0x55a7ea2c4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.447+0000 7f847d287f40 1 -- 172.21.15.16:0/991312668 shutdown_connections 2024-04-25T10:14:49.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.447+0000 7f847d287f40 1 -- 172.21.15.16:0/991312668 wait complete. 2024-04-25T10:14:49.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.448+0000 7f847d287f40 1 Processor -- start 2024-04-25T10:14:49.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.448+0000 7f847d287f40 1 -- start start 2024-04-25T10:14:49.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.448+0000 7f847d287f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a7ea2aa570 0x55a7ea2a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.448+0000 7f847d287f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ea350d90 0x55a7ea346640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.448+0000 7f847d287f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a7ea3516b0 0x55a7ea341e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.448+0000 7f847d287f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ea34e940 con 0x55a7ea350d90 2024-04-25T10:14:49.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.448+0000 7f847d287f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ea346b80 con 0x55a7ea2aa570 2024-04-25T10:14:49.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.448+0000 7f847d287f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7ea346de0 con 0x55a7ea3516b0 2024-04-25T10:14:49.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.448+0000 7f847a589640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ea350d90 0x55a7ea346640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.448+0000 7f847a589640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ea350d90 0x55a7ea346640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58696/0 (socket says 172.21.15.16:58696) 2024-04-25T10:14:49.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.448+0000 7f847a589640 1 -- 172.21.15.16:0/1182272063 learned_addr learned my addr 172.21.15.16:0/1182272063 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:49.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.449+0000 7f847ad8a640 1 --2- 172.21.15.16:0/1182272063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a7ea2aa570 0x55a7ea2a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.449+0000 7f847b58b640 1 --2- 172.21.15.16:0/1182272063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a7ea3516b0 0x55a7ea341e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.449+0000 7f847a589640 1 -- 172.21.15.16:0/1182272063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a7ea3516b0 msgr2=0x55a7ea341e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.449+0000 7f847a589640 1 --2- 172.21.15.16:0/1182272063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a7ea3516b0 0x55a7ea341e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.449+0000 7f847a589640 1 -- 172.21.15.16:0/1182272063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a7ea2aa570 msgr2=0x55a7ea2a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.449+0000 7f847a589640 1 --2- 172.21.15.16:0/1182272063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a7ea2aa570 0x55a7ea2a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.449+0000 7f847a589640 1 -- 172.21.15.16:0/1182272063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7ea3e1b50 con 0x55a7ea350d90 2024-04-25T10:14:49.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.449+0000 7f847a589640 1 --2- 172.21.15.16:0/1182272063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ea350d90 0x55a7ea346640 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f846400c580 tx=0x7f846400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.449+0000 7f8473fff640 1 -- 172.21.15.16:0/1182272063 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84640104c0 con 0x55a7ea350d90 2024-04-25T10:14:49.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.449+0000 7f8473fff640 1 -- 172.21.15.16:0/1182272063 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8464015020 con 0x55a7ea350d90 2024-04-25T10:14:49.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.449+0000 7f847d287f40 1 -- 172.21.15.16:0/1182272063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a7ea3e1e30 con 0x55a7ea350d90 2024-04-25T10:14:49.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.449+0000 7f8473fff640 1 -- 172.21.15.16:0/1182272063 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464010860 con 0x55a7ea350d90 2024-04-25T10:14:49.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.449+0000 7f847d287f40 1 -- 172.21.15.16:0/1182272063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7ea342380 con 0x55a7ea350d90 2024-04-25T10:14:49.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.450+0000 7f8473fff640 1 -- 172.21.15.16:0/1182272063 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8464010a00 con 0x55a7ea350d90 2024-04-25T10:14:49.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.451+0000 7f8473fff640 1 --2- 172.21.15.16:0/1182272063 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f845c069eb0 0x7f845c06c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.451+0000 7f8473fff640 1 -- 172.21.15.16:0/1182272063 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8464089630 con 0x55a7ea350d90 2024-04-25T10:14:49.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.451+0000 7f847d287f40 1 --2- 172.21.15.16:0/1182272063 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55a7ea3e2630 0x55a7ea3e2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.451+0000 7f847ad8a640 1 --2- 172.21.15.16:0/1182272063 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f845c069eb0 0x7f845c06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.451+0000 7f847d287f40 1 -- 172.21.15.16:0/1182272063 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:1 17.5 17:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) v9 -- 0x55a7ea3e2f80 con 0x55a7ea3e2630 2024-04-25T10:14:49.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.451+0000 7f847b58b640 1 --2- 172.21.15.16:0/1182272063 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55a7ea3e2630 0x55a7ea3e2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.452+0000 7f847ad8a640 1 --2- 172.21.15.16:0/1182272063 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f845c069eb0 0x7f845c06c370 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8474056000 tx=0x7f8474050000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.452+0000 7f847b58b640 1 --2- 172.21.15.16:0/1182272063 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55a7ea3e2630 0x55a7ea3e2a40 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.452+0000 7f847b58b640 1 -- 172.21.15.16:0/1182272063 <== osd.0 v2:172.21.15.16:6816/2167750256 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f846c018020 con 0x55a7ea3e2630 2024-04-25T10:14:49.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.452+0000 7f847d287f40 1 -- 172.21.15.16:0/1182272063 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55a7ea3e2630 msgr2=0x55a7ea3e2a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.452+0000 7f847d287f40 1 --2- 172.21.15.16:0/1182272063 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55a7ea3e2630 0x55a7ea3e2a40 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.452+0000 7f847d287f40 1 -- 172.21.15.16:0/1182272063 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f845c069eb0 msgr2=0x7f845c06c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.452+0000 7f847d287f40 1 --2- 172.21.15.16:0/1182272063 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f845c069eb0 0x7f845c06c370 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8474056000 tx=0x7f8474050000 comp rx=0 tx=0).stop 2024-04-25T10:14:49.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.452+0000 7f847d287f40 1 -- 172.21.15.16:0/1182272063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ea350d90 msgr2=0x55a7ea346640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.452+0000 7f847d287f40 1 --2- 172.21.15.16:0/1182272063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ea350d90 0x55a7ea346640 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f846400c580 tx=0x7f846400ca40 comp rx=0 tx=0).stop 2024-04-25T10:14:49.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.452+0000 7f847b58b640 1 -- 172.21.15.16:0/1182272063 reap_dead start 2024-04-25T10:14:49.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.452+0000 7f847d287f40 1 -- 172.21.15.16:0/1182272063 shutdown_connections 2024-04-25T10:14:49.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.452+0000 7f847d287f40 1 -- 172.21.15.16:0/1182272063 >> 172.21.15.16:0/1182272063 conn(0x55a7ea2a73d0 msgr2=0x55a7ea2c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.452+0000 7f847d287f40 1 -- 172.21.15.16:0/1182272063 shutdown_connections 2024-04-25T10:14:49.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.452+0000 7f847d287f40 1 -- 172.21.15.16:0/1182272063 wait complete. 2024-04-25T10:14:49.462 INFO:teuthology.orchestra.run.smithi016.stdout:mytag123 2024-04-25T10:14:49.462 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000006.00000000 scrub_tag 2024-04-25T10:14:49.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.501+0000 7fd57a920f40 1 Processor -- start 2024-04-25T10:14:49.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.501+0000 7fd57a920f40 1 -- start start 2024-04-25T10:14:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.502+0000 7fd57a920f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557460ec3880 0x557460e96350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.502+0000 7fd57a920f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557460f66b60 0x557460e94ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.502+0000 7fd57a920f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557460f695e0 0x557460f5e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.502+0000 7fd57a920f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557460e3adf0 con 0x557460f66b60 2024-04-25T10:14:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.502+0000 7fd57a920f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557460e3ed50 con 0x557460f695e0 2024-04-25T10:14:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.502+0000 7fd57a920f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x557460f6a860 con 0x557460ec3880 2024-04-25T10:14:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.502+0000 7fd577305640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557460f66b60 0x557460e94ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.502+0000 7fd577305640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557460f66b60 0x557460e94ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58716/0 (socket says 172.21.15.16:58716) 2024-04-25T10:14:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.502+0000 7fd577305640 1 -- 172.21.15.16:0/1625605378 learned_addr learned my addr 172.21.15.16:0/1625605378 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.502+0000 7fd578307640 1 --2- 172.21.15.16:0/1625605378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557460f695e0 0x557460f5e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.502+0000 7fd577b06640 1 --2- 172.21.15.16:0/1625605378 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557460ec3880 0x557460e96350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.502+0000 7fd577305640 1 -- 172.21.15.16:0/1625605378 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557460ec3880 msgr2=0x557460e96350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.502+0000 7fd577305640 1 --2- 172.21.15.16:0/1625605378 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557460ec3880 0x557460e96350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.502+0000 7fd577305640 1 -- 172.21.15.16:0/1625605378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557460f695e0 msgr2=0x557460f5e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.502+0000 7fd577305640 1 --2- 172.21.15.16:0/1625605378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557460f695e0 0x557460f5e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.502+0000 7fd577305640 1 -- 172.21.15.16:0/1625605378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557460e957d0 con 0x557460f66b60 2024-04-25T10:14:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.502+0000 7fd578307640 1 --2- 172.21.15.16:0/1625605378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557460f695e0 0x557460f5e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.502+0000 7fd577b06640 1 --2- 172.21.15.16:0/1625605378 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557460ec3880 0x557460e96350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.503+0000 7fd577305640 1 --2- 172.21.15.16:0/1625605378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557460f66b60 0x557460e94ea0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00a440 tx=0x7fd56c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a5ee446ae5ae39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.503+0000 7fd576303640 1 -- 172.21.15.16:0/1625605378 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c054b10 con 0x557460f66b60 2024-04-25T10:14:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.503+0000 7fd576303640 1 -- 172.21.15.16:0/1625605378 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56c054cb0 con 0x557460f66b60 2024-04-25T10:14:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.503+0000 7fd576303640 1 -- 172.21.15.16:0/1625605378 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c054e60 con 0x557460f66b60 2024-04-25T10:14:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.503+0000 7fd57a920f40 1 -- 172.21.15.16:0/1625605378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557460f66b60 msgr2=0x557460e94ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.503+0000 7fd57a920f40 1 --2- 172.21.15.16:0/1625605378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557460f66b60 0x557460e94ea0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00a440 tx=0x7fd56c051000 comp rx=0 tx=0).stop 2024-04-25T10:14:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.503+0000 7fd57a920f40 1 -- 172.21.15.16:0/1625605378 shutdown_connections 2024-04-25T10:14:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.503+0000 7fd57a920f40 1 --2- 172.21.15.16:0/1625605378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557460f695e0 0x557460f5e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.503+0000 7fd57a920f40 1 --2- 172.21.15.16:0/1625605378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557460f66b60 0x557460e94ea0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.503+0000 7fd57a920f40 1 --2- 172.21.15.16:0/1625605378 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557460ec3880 0x557460e96350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.503+0000 7fd57a920f40 1 -- 172.21.15.16:0/1625605378 >> 172.21.15.16:0/1625605378 conn(0x557460ebe3d0 msgr2=0x557460f6ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.503+0000 7fd57a920f40 1 -- 172.21.15.16:0/1625605378 shutdown_connections 2024-04-25T10:14:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.503+0000 7fd57a920f40 1 -- 172.21.15.16:0/1625605378 wait complete. 2024-04-25T10:14:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.504+0000 7fd57a920f40 1 Processor -- start 2024-04-25T10:14:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.504+0000 7fd57a920f40 1 -- start start 2024-04-25T10:14:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd57a920f40 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557460ec3880 0x557460ebd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd57a920f40 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557460f66b60 0x557460f5cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd57a920f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557460f695e0 0x557460f60b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd57a920f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557460ebdc60 con 0x557460f695e0 2024-04-25T10:14:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd57a920f40 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557460e92630 con 0x557460ec3880 2024-04-25T10:14:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd57a920f40 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x557460e954f0 con 0x557460f66b60 2024-04-25T10:14:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd578307640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557460f695e0 0x557460f60b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd578307640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557460f695e0 0x557460f60b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58732/0 (socket says 172.21.15.16:58732) 2024-04-25T10:14:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd578307640 1 -- 172.21.15.16:0/2445667806 learned_addr learned my addr 172.21.15.16:0/2445667806 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd577b06640 1 --2- 172.21.15.16:0/2445667806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557460ec3880 0x557460ebd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd577305640 1 --2- 172.21.15.16:0/2445667806 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557460f66b60 0x557460f5cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd578307640 1 -- 172.21.15.16:0/2445667806 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557460f66b60 msgr2=0x557460f5cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd578307640 1 --2- 172.21.15.16:0/2445667806 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557460f66b60 0x557460f5cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd578307640 1 -- 172.21.15.16:0/2445667806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557460ec3880 msgr2=0x557460ebd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd578307640 1 --2- 172.21.15.16:0/2445667806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557460ec3880 0x557460ebd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd578307640 1 -- 172.21.15.16:0/2445667806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557460f682d0 con 0x557460f695e0 2024-04-25T10:14:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd577305640 1 --2- 172.21.15.16:0/2445667806 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x557460f66b60 0x557460f5cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd578307640 1 --2- 172.21.15.16:0/2445667806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557460f695e0 0x557460f60b80 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fd570009180 tx=0x7fd570009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd575301640 1 -- 172.21.15.16:0/2445667806 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570014020 con 0x557460f695e0 2024-04-25T10:14:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd575301640 1 -- 172.21.15.16:0/2445667806 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd570010820 con 0x557460f695e0 2024-04-25T10:14:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd57a920f40 1 -- 172.21.15.16:0/2445667806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557460f685b0 con 0x557460f695e0 2024-04-25T10:14:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.505+0000 7fd575301640 1 -- 172.21.15.16:0/2445667806 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570010bf0 con 0x557460f695e0 2024-04-25T10:14:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.506+0000 7fd57a920f40 1 -- 172.21.15.16:0/2445667806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557460ec3e80 con 0x557460f695e0 2024-04-25T10:14:49.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.507+0000 7fd575301640 1 -- 172.21.15.16:0/2445667806 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd570017020 con 0x557460f695e0 2024-04-25T10:14:49.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.507+0000 7fd575301640 1 --2- 172.21.15.16:0/2445667806 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd564069eb0 0x7fd56406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.507+0000 7fd575301640 1 -- 172.21.15.16:0/2445667806 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd570089020 con 0x557460f695e0 2024-04-25T10:14:49.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.507+0000 7fd57a920f40 1 --2- 172.21.15.16:0/2445667806 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x557460f5d2c0 0x557460ec2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.508+0000 7fd577b06640 1 --2- 172.21.15.16:0/2445667806 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd564069eb0 0x7fd56406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.508+0000 7fd57a920f40 1 -- 172.21.15.16:0/2445667806 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:1 17.25 17:a5feb6f3:::10000000006.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) v9 -- 0x557460f5d860 con 0x557460f5d2c0 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.508+0000 7fd577305640 1 --2- 172.21.15.16:0/2445667806 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x557460f5d2c0 0x557460ec2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.508+0000 7fd577b06640 1 --2- 172.21.15.16:0/2445667806 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd564069eb0 0x7fd56406c370 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd568000c00 tx=0x7fd568001880 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.508+0000 7fd577305640 1 --2- 172.21.15.16:0/2445667806 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x557460f5d2c0 0x557460ec2c10 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.509+0000 7fd577305640 1 -- 172.21.15.16:0/2445667806 <== osd.4 v2:172.21.15.186:6824/3091472889 1 ==== osd_op_reply(1 10000000006.00000000 [getxattr (12) out=12b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7fd56c054a10 con 0x557460f5d2c0 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.509+0000 7fd57a920f40 1 -- 172.21.15.16:0/2445667806 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x557460f5d2c0 msgr2=0x557460ec2c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.509+0000 7fd57a920f40 1 --2- 172.21.15.16:0/2445667806 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x557460f5d2c0 0x557460ec2c10 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.509+0000 7fd57a920f40 1 -- 172.21.15.16:0/2445667806 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd564069eb0 msgr2=0x7fd56406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.509+0000 7fd57a920f40 1 --2- 172.21.15.16:0/2445667806 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd564069eb0 0x7fd56406c370 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd568000c00 tx=0x7fd568001880 comp rx=0 tx=0).stop 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.509+0000 7fd57a920f40 1 -- 172.21.15.16:0/2445667806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557460f695e0 msgr2=0x557460f60b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.509+0000 7fd57a920f40 1 --2- 172.21.15.16:0/2445667806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557460f695e0 0x557460f60b80 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fd570009180 tx=0x7fd570009640 comp rx=0 tx=0).stop 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.509+0000 7fd578307640 1 -- 172.21.15.16:0/2445667806 reap_dead start 2024-04-25T10:14:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.509+0000 7fd57a920f40 1 -- 172.21.15.16:0/2445667806 shutdown_connections 2024-04-25T10:14:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.509+0000 7fd57a920f40 1 -- 172.21.15.16:0/2445667806 >> 172.21.15.16:0/2445667806 conn(0x557460ebe3d0 msgr2=0x557460e7e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.509+0000 7fd57a920f40 1 -- 172.21.15.16:0/2445667806 shutdown_connections 2024-04-25T10:14:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.509+0000 7fd57a920f40 1 -- 172.21.15.16:0/2445667806 wait complete. 2024-04-25T10:14:49.519 INFO:teuthology.orchestra.run.smithi016.stdout:mytag123 2024-04-25T10:14:49.519 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:14:49.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-25T10:14:49.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.658+0000 7f779c35a640 1 Processor -- start 2024-04-25T10:14:49.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.658+0000 7f779c35a640 1 -- start start 2024-04-25T10:14:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.658+0000 7f779c35a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f779415b7b0 0x7f77940ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f779c35a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f779415f000 con 0x7f77940cf760 2024-04-25T10:14:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f779c35a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f779415ed20 con 0x7f779415e820 2024-04-25T10:14:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f779c35a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77940ced00 con 0x7f779415b7b0 2024-04-25T10:14:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f77998f5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f77940cf760 0x7f779415e110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56448/0 (socket says 172.21.15.16:56448) 2024-04-25T10:14:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f77998f5640 1 -- 172.21.15.16:0/800568177 learned_addr learned my addr 172.21.15.16:0/800568177 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f779a0f6640 1 --2- 172.21.15.16:0/800568177 >> v2:172.21.15.186:3300/0 conn(0x7f779415b7b0 0x7f77940ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f77990f4640 1 -- 172.21.15.16:0/800568177 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1523573207 0 0) 0x7f779415f000 con 0x7f77940cf760 2024-04-25T10:14:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f77990f4640 1 -- 172.21.15.16:0/800568177 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f777c003520 con 0x7f77940cf760 2024-04-25T10:14:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f77990f4640 1 -- 172.21.15.16:0/800568177 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 65276534 0 0) 0x7f777c003520 con 0x7f77940cf760 2024-04-25T10:14:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f77990f4640 1 -- 172.21.15.16:0/800568177 >> v2:172.21.15.186:3300/0 conn(0x7f779415b7b0 msgr2=0x7f77940ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f77990f4640 1 --2- 172.21.15.16:0/800568177 >> v2:172.21.15.186:3300/0 conn(0x7f779415b7b0 0x7f77940ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f77990f4640 1 -- 172.21.15.16:0/800568177 >> v1:172.21.15.186:6789/0 conn(0x7f779415e820 legacy=0x7f77940cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f77990f4640 1 -- 172.21.15.16:0/800568177 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77940cdce0 con 0x7f77940cf760 2024-04-25T10:14:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f77990f4640 1 -- 172.21.15.16:0/800568177 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f7788001670 con 0x7f77940cf760 2024-04-25T10:14:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f77990f4640 1 -- 172.21.15.16:0/800568177 >> v1:172.21.15.16:6789/0 conn(0x7f77940cf760 legacy=0x7f779415e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f77990f4640 1 --2- 172.21.15.16:0/800568177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c0039b0 0x7f777c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f77990f4640 1 --2- 172.21.15.16:0/800568177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f777c004800 0x7f777c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f77990f4640 1 --2- 172.21.15.16:0/800568177 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f777c0051c0 0x7f777c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f77990f4640 1 -- 172.21.15.16:0/800568177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77940ced00 con 0x7f777c0039b0 2024-04-25T10:14:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f77990f4640 1 -- 172.21.15.16:0/800568177 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c00c060 con 0x7f777c004800 2024-04-25T10:14:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f77990f4640 1 -- 172.21.15.16:0/800568177 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777c00c260 con 0x7f777c0051c0 2024-04-25T10:14:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f779a0f6640 1 --2- 172.21.15.16:0/800568177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c0039b0 0x7f777c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.659+0000 7f77998f5640 1 --2- 172.21.15.16:0/800568177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f777c004800 0x7f777c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.660+0000 7f779a8f7640 1 --2- 172.21.15.16:0/800568177 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f777c0051c0 0x7f777c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.660+0000 7f779a0f6640 1 -- 172.21.15.16:0/800568177 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f777c0051c0 msgr2=0x7f777c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.660+0000 7f779a0f6640 1 --2- 172.21.15.16:0/800568177 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f777c0051c0 0x7f777c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.660+0000 7f779a0f6640 1 -- 172.21.15.16:0/800568177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f777c004800 msgr2=0x7f777c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.660+0000 7f779a0f6640 1 --2- 172.21.15.16:0/800568177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f777c004800 0x7f777c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.660+0000 7f779a0f6640 1 -- 172.21.15.16:0/800568177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777c003520 con 0x7f777c0039b0 2024-04-25T10:14:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.660+0000 7f77998f5640 1 --2- 172.21.15.16:0/800568177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f777c004800 0x7f777c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.660+0000 7f779a8f7640 1 -- 172.21.15.16:0/800568177 reap_dead start 2024-04-25T10:14:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.660+0000 7f779a0f6640 1 --2- 172.21.15.16:0/800568177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c0039b0 0x7f777c003dd0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f778c009ac0 tx=0x7f777c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67aaf1bf71209270 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.660+0000 7f77990f4640 1 -- 172.21.15.16:0/800568177 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c036070 con 0x7f777c0039b0 2024-04-25T10:14:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.660+0000 7f77990f4640 1 -- 172.21.15.16:0/800568177 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778c031400 con 0x7f777c0039b0 2024-04-25T10:14:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.661+0000 7f77990f4640 1 -- 172.21.15.16:0/800568177 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c009690 con 0x7f777c0039b0 2024-04-25T10:14:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.661+0000 7f779c35a640 1 -- 172.21.15.16:0/800568177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c0039b0 msgr2=0x7f777c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.661+0000 7f779c35a640 1 --2- 172.21.15.16:0/800568177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c0039b0 0x7f777c003dd0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f778c009ac0 tx=0x7f777c005830 comp rx=0 tx=0).stop 2024-04-25T10:14:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.661+0000 7f779c35a640 1 -- 172.21.15.16:0/800568177 shutdown_connections 2024-04-25T10:14:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.661+0000 7f779c35a640 1 --2- 172.21.15.16:0/800568177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c0039b0 0x7f777c003dd0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.661+0000 7f779c35a640 1 -- 172.21.15.16:0/800568177 >> 172.21.15.16:0/800568177 conn(0x7f779406f4a0 msgr2=0x7f779406f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.661+0000 7f779c35a640 1 -- 172.21.15.16:0/800568177 shutdown_connections 2024-04-25T10:14:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.661+0000 7f779c35a640 1 -- 172.21.15.16:0/800568177 wait complete. 2024-04-25T10:14:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.662+0000 7f779c35a640 1 Processor -- start 2024-04-25T10:14:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.662+0000 7f779c35a640 1 -- start start 2024-04-25T10:14:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.663+0000 7f779c35a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c0039b0 0x7f7794156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.663+0000 7f779c35a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77940cae40 0x7f77940cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.663+0000 7f779c35a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77940b28e0 0x7f77940b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.663+0000 7f779c35a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77941619f0 con 0x7f777c0039b0 2024-04-25T10:14:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.663+0000 7f779c35a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77940ce330 con 0x7f77940b28e0 2024-04-25T10:14:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.663+0000 7f779c35a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77940ce7c0 con 0x7f77940cae40 2024-04-25T10:14:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.663+0000 7f779a0f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c0039b0 0x7f7794156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.663+0000 7f779a0f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c0039b0 0x7f7794156c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58750/0 (socket says 172.21.15.16:58750) 2024-04-25T10:14:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.663+0000 7f779a0f6640 1 -- 172.21.15.16:0/68586622 learned_addr learned my addr 172.21.15.16:0/68586622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.663+0000 7f77998f5640 1 --2- 172.21.15.16:0/68586622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77940cae40 0x7f77940cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.663+0000 7f779a8f7640 1 --2- 172.21.15.16:0/68586622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77940b28e0 0x7f77940b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.663+0000 7f779a0f6640 1 -- 172.21.15.16:0/68586622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77940cae40 msgr2=0x7f77940cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.663+0000 7f779a0f6640 1 --2- 172.21.15.16:0/68586622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77940cae40 0x7f77940cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.663+0000 7f779a0f6640 1 -- 172.21.15.16:0/68586622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77940b28e0 msgr2=0x7f77940b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.663+0000 7f779a0f6640 1 --2- 172.21.15.16:0/68586622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77940b28e0 0x7f77940b2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.663+0000 7f779a0f6640 1 -- 172.21.15.16:0/68586622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77940b3610 con 0x7f777c0039b0 2024-04-25T10:14:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.664+0000 7f779a0f6640 1 --2- 172.21.15.16:0/68586622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c0039b0 0x7f7794156c40 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f778c009be0 tx=0x7f778c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.664+0000 7f7792ffd640 1 -- 172.21.15.16:0/68586622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c03f020 con 0x7f777c0039b0 2024-04-25T10:14:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.664+0000 7f7792ffd640 1 -- 172.21.15.16:0/68586622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778c037040 con 0x7f777c0039b0 2024-04-25T10:14:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.664+0000 7f779c35a640 1 -- 172.21.15.16:0/68586622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77940b38f0 con 0x7f777c0039b0 2024-04-25T10:14:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.664+0000 7f7792ffd640 1 -- 172.21.15.16:0/68586622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c036070 con 0x7f777c0039b0 2024-04-25T10:14:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.664+0000 7f779c35a640 1 -- 172.21.15.16:0/68586622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77940c8dd0 con 0x7f777c0039b0 2024-04-25T10:14:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.665+0000 7f7792ffd640 1 -- 172.21.15.16:0/68586622 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f778c042020 con 0x7f777c0039b0 2024-04-25T10:14:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.666+0000 7f7792ffd640 1 --2- 172.21.15.16:0/68586622 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f778006b4d0 0x7f778006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.666+0000 7f7792ffd640 1 -- 172.21.15.16:0/68586622 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f778c0aa190 con 0x7f777c0039b0 2024-04-25T10:14:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.666+0000 7f77998f5640 1 --2- 172.21.15.16:0/68586622 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f778006b4d0 0x7f778006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.666+0000 7f77998f5640 1 --2- 172.21.15.16:0/68586622 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f778006b4d0 0x7f778006d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f77941f70f0 tx=0x7f7788009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.667+0000 7f779f058740 1 Processor -- start 2024-04-25T10:14:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.667+0000 7f779f058740 1 -- start start 2024-04-25T10:14:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f779f058740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa1cb10 0x56050aa1f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f779f058740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa1cef0 0x56050aa1f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f779f058740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050a9ef770 0x56050a9f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f779f058740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56050a958da0 con 0x56050aa1cef0 2024-04-25T10:14:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f779f058740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56050a901b80 con 0x56050aa1cb10 2024-04-25T10:14:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f779f058740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56050a954e40 con 0x56050a9ef770 2024-04-25T10:14:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f77998f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa1cef0 0x56050aa1f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f77998f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa1cef0 0x56050aa1f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58756/0 (socket says 172.21.15.16:58756) 2024-04-25T10:14:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f77998f5640 1 -- 172.21.15.16:0/2994953596 learned_addr learned my addr 172.21.15.16:0/2994953596 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f779a0f6640 1 --2- 172.21.15.16:0/2994953596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa1cb10 0x56050aa1f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f779a8f7640 1 --2- 172.21.15.16:0/2994953596 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050a9ef770 0x56050a9f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f77998f5640 1 -- 172.21.15.16:0/2994953596 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050a9ef770 msgr2=0x56050a9f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f77998f5640 1 --2- 172.21.15.16:0/2994953596 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050a9ef770 0x56050a9f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f77998f5640 1 -- 172.21.15.16:0/2994953596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa1cb10 msgr2=0x56050aa1f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f77998f5640 1 --2- 172.21.15.16:0/2994953596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa1cb10 0x56050aa1f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f77998f5640 1 -- 172.21.15.16:0/2994953596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56050a90bf10 con 0x56050aa1cef0 2024-04-25T10:14:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f77998f5640 1 --2- 172.21.15.16:0/2994953596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa1cef0 0x56050aa1f860 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f77880109a0 tx=0x7f778800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219810df8697e4dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f777bfff640 1 -- 172.21.15.16:0/2994953596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788014b80 con 0x56050aa1cef0 2024-04-25T10:14:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f777bfff640 1 -- 172.21.15.16:0/2994953596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7788014d20 con 0x56050aa1cef0 2024-04-25T10:14:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.668+0000 7f777bfff640 1 -- 172.21.15.16:0/2994953596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788022620 con 0x56050aa1cef0 2024-04-25T10:14:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.669+0000 7f779f058740 1 -- 172.21.15.16:0/2994953596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa1cef0 msgr2=0x56050aa1f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.669+0000 7f779f058740 1 --2- 172.21.15.16:0/2994953596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa1cef0 0x56050aa1f860 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f77880109a0 tx=0x7f778800c000 comp rx=0 tx=0).stop 2024-04-25T10:14:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.669+0000 7f779f058740 1 -- 172.21.15.16:0/2994953596 shutdown_connections 2024-04-25T10:14:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.669+0000 7f779f058740 1 --2- 172.21.15.16:0/2994953596 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050a9ef770 0x56050a9f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.669+0000 7f779f058740 1 --2- 172.21.15.16:0/2994953596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa1cef0 0x56050aa1f860 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.669+0000 7f779f058740 1 --2- 172.21.15.16:0/2994953596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa1cb10 0x56050aa1f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.669+0000 7f779f058740 1 -- 172.21.15.16:0/2994953596 >> 172.21.15.16:0/2994953596 conn(0x56050a9f19e0 msgr2=0x56050aa3a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.669+0000 7f779f058740 1 -- 172.21.15.16:0/2994953596 shutdown_connections 2024-04-25T10:14:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.669+0000 7f779f058740 1 -- 172.21.15.16:0/2994953596 wait complete. 2024-04-25T10:14:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.670+0000 7f779f058740 1 -- start start 2024-04-25T10:14:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.670+0000 7f779f058740 1 Processor -- start 2024-04-25T10:14:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f779f058740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050aa1cb10 0x56050aa3e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f779f058740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa1cef0 0x56050a9f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f779f058740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa13750 0x56050a9f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f779f058740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56050a9f4b90 con 0x56050aa13750 2024-04-25T10:14:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f779f058740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56050a8f74b0 con 0x56050aa1cef0 2024-04-25T10:14:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f779f058740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56050a9f4e70 con 0x56050aa1cb10 2024-04-25T10:14:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f779f058740 10 client.0 _ll_register_callbacks cb 0x56050a937180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f779a8f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa13750 0x56050a9f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f779a8f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa13750 0x56050a9f44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58762/0 (socket says 172.21.15.16:58762) 2024-04-25T10:14:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f779a8f7640 1 -- 172.21.15.16:0/711077900 learned_addr learned my addr 172.21.15.16:0/711077900 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f779a0f6640 1 --2- 172.21.15.16:0/711077900 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050aa1cb10 0x56050aa3e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f77998f5640 1 --2- 172.21.15.16:0/711077900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa1cef0 0x56050a9f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f779a8f7640 1 -- 172.21.15.16:0/711077900 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050aa1cb10 msgr2=0x56050aa3e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f779a8f7640 1 --2- 172.21.15.16:0/711077900 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050aa1cb10 0x56050aa3e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f779a8f7640 1 -- 172.21.15.16:0/711077900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa1cef0 msgr2=0x56050a9f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f779a8f7640 1 --2- 172.21.15.16:0/711077900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa1cef0 0x56050a9f3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f779a8f7640 1 -- 172.21.15.16:0/711077900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56050a7377d0 con 0x56050aa13750 2024-04-25T10:14:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f779a8f7640 1 --2- 172.21.15.16:0/711077900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa13750 0x56050a9f44d0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f77940d29c0 tx=0x7f77840019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7463ef054834a7b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.671+0000 7f77797fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:14:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.672+0000 7f77797fa640 1 -- 172.21.15.16:0/711077900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784016450 con 0x56050aa13750 2024-04-25T10:14:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.672+0000 7f77797fa640 1 -- 172.21.15.16:0/711077900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f77840165f0 con 0x56050aa13750 2024-04-25T10:14:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.672+0000 7f779f058740 1 -- 172.21.15.16:0/711077900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7788014d20 con 0x56050aa13750 2024-04-25T10:14:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.672+0000 7f77797fa640 1 -- 172.21.15.16:0/711077900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7784023050 con 0x56050aa13750 2024-04-25T10:14:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.672+0000 7f779f058740 10 client.9234 fetch_fsmap learned FSMap version 89 2024-04-25T10:14:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.672+0000 7f779f058740 1 -- 172.21.15.16:0/711077900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x56050a66d3f0 con 0x56050aa13750 2024-04-25T10:14:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.672+0000 7f77797fa640 1 -- 172.21.15.16:0/711077900 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7788014d20 con 0x56050aa13750 2024-04-25T10:14:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.672+0000 7f77797fa640 1 -- 172.21.15.16:0/711077900 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7784026020 con 0x56050aa13750 2024-04-25T10:14:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.672+0000 7f779f058740 10 client.9234 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:14:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.672+0000 7f779f058740 10 client.9234 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:14:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.672+0000 7f779f058740 20 client.9234 populate_metadata read hostname 'smithi016' 2024-04-25T10:14:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.672+0000 7f779f058740 1 --2- 172.21.15.16:0/711077900 >> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] conn(0x56050aa3b9c0 0x56050aa3bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.672+0000 7f779f058740 4 client.9234 mds_command: new command op to 9096 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.672+0000 7f779f058740 1 -- 172.21.15.16:0/711077900 --> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56050a9662e0 con 0x56050aa3b9c0 2024-04-25T10:14:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.672+0000 7f779a0f6640 1 --2- 172.21.15.16:0/711077900 >> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] conn(0x56050aa3b9c0 0x56050aa3bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.673+0000 7f77797fa640 10 client.9234 ms_handle_connect on v2:172.21.15.16:6833/2697110279 2024-04-25T10:14:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.673+0000 7f779a0f6640 1 --2- 172.21.15.16:0/711077900 >> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] conn(0x56050aa3b9c0 0x56050aa3bdd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=903c8085274394f0 server_cookie=d9f4e964d43d9641 in_seq=0 out_seq=0 2024-04-25T10:14:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.673+0000 7f77797fa640 1 -- 172.21.15.16:0/711077900 <== mds.0 v2:172.21.15.16:6833/2697110279 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56050a9662e0 con 0x56050aa3b9c0 2024-04-25T10:14:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.673+0000 7f77797fa640 10 client.9234 handle_command_reply: tid=0 2024-04-25T10:14:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.673+0000 7f779f058740 1 client.9234 shutdown 2024-04-25T10:14:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.673+0000 7f779f058740 1 -- 172.21.15.16:0/711077900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa13750 msgr2=0x56050a9f44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.673+0000 7f779f058740 1 --2- 172.21.15.16:0/711077900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa13750 0x56050a9f44d0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f77940d29c0 tx=0x7f77840019e0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.673+0000 7f779f058740 1 -- 172.21.15.16:0/711077900 shutdown_connections 2024-04-25T10:14:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.673+0000 7f779f058740 1 --2- 172.21.15.16:0/711077900 >> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] conn(0x56050aa3b9c0 0x56050aa3bdd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.673+0000 7f779f058740 1 --2- 172.21.15.16:0/711077900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa13750 0x56050a9f44d0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.673+0000 7f779f058740 1 --2- 172.21.15.16:0/711077900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa1cef0 0x56050a9f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.673+0000 7f779f058740 1 --2- 172.21.15.16:0/711077900 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050aa1cb10 0x56050aa3e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.673+0000 7f77797fa640 0 client.9234 ms_handle_reset on v2:172.21.15.16:6833/2697110279 2024-04-25T10:14:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.673+0000 7f779f058740 1 -- 172.21.15.16:0/711077900 >> 172.21.15.16:0/711077900 conn(0x56050a9ef770 msgr2=0x56050aa1bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.673+0000 7f779f058740 1 -- 172.21.15.16:0/711077900 shutdown_connections 2024-04-25T10:14:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.673+0000 7f779f058740 1 -- 172.21.15.16:0/711077900 wait complete. 2024-04-25T10:14:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.674+0000 7f779f058740 20 client.9234 trim_cache size 0 max 16384 2024-04-25T10:14:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.687+0000 7f779f058740 1 Processor -- start 2024-04-25T10:14:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.687+0000 7f779f058740 1 -- start start 2024-04-25T10:14:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f779f058740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa13750 0x56050aa5c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f779f058740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa1cb10 0x56050aa3a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f779f058740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050aa1cef0 0x56050aa3a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f779f058740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77942007d0 con 0x56050aa1cb10 2024-04-25T10:14:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f779f058740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77941ff9d0 con 0x56050aa13750 2024-04-25T10:14:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f779f058740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77940c9da0 con 0x56050aa1cef0 2024-04-25T10:14:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f77998f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa1cb10 0x56050aa3a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f77998f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa1cb10 0x56050aa3a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58776/0 (socket says 172.21.15.16:58776) 2024-04-25T10:14:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f77998f5640 1 -- 172.21.15.16:0/336710642 learned_addr learned my addr 172.21.15.16:0/336710642 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f779a0f6640 1 --2- 172.21.15.16:0/336710642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa13750 0x56050aa5c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f77998f5640 1 -- 172.21.15.16:0/336710642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050aa1cef0 msgr2=0x56050aa3a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f77998f5640 1 --2- 172.21.15.16:0/336710642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050aa1cef0 0x56050aa3a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f77998f5640 1 -- 172.21.15.16:0/336710642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa13750 msgr2=0x56050aa5c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f77998f5640 1 --2- 172.21.15.16:0/336710642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa13750 0x56050aa5c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f77998f5640 1 -- 172.21.15.16:0/336710642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56050a62d0f0 con 0x56050aa1cb10 2024-04-25T10:14:49.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f77998f5640 1 --2- 172.21.15.16:0/336710642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa1cb10 0x56050aa3a320 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f7788001eb0 tx=0x7f7788015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ceb6e78b596edd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f777b7fe640 1 -- 172.21.15.16:0/336710642 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788033070 con 0x56050aa1cb10 2024-04-25T10:14:49.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f777b7fe640 1 -- 172.21.15.16:0/336710642 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7788014dc0 con 0x56050aa1cb10 2024-04-25T10:14:49.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.688+0000 7f777b7fe640 1 -- 172.21.15.16:0/336710642 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778802f670 con 0x56050aa1cb10 2024-04-25T10:14:49.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.689+0000 7f779f058740 1 -- 172.21.15.16:0/336710642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa1cb10 msgr2=0x56050aa3a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.689+0000 7f779f058740 1 --2- 172.21.15.16:0/336710642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa1cb10 0x56050aa3a320 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f7788001eb0 tx=0x7f7788015000 comp rx=0 tx=0).stop 2024-04-25T10:14:49.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.689+0000 7f779f058740 1 -- 172.21.15.16:0/336710642 shutdown_connections 2024-04-25T10:14:49.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.689+0000 7f779f058740 1 --2- 172.21.15.16:0/336710642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050aa1cef0 0x56050aa3a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.689+0000 7f779f058740 1 --2- 172.21.15.16:0/336710642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa1cb10 0x56050aa3a320 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.689+0000 7f779f058740 1 --2- 172.21.15.16:0/336710642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa13750 0x56050aa5c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.689+0000 7f779f058740 1 -- 172.21.15.16:0/336710642 >> 172.21.15.16:0/336710642 conn(0x56050a9ef770 msgr2=0x56050aa226e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.689+0000 7f779f058740 1 -- 172.21.15.16:0/336710642 shutdown_connections 2024-04-25T10:14:49.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.689+0000 7f779f058740 1 -- 172.21.15.16:0/336710642 wait complete. 2024-04-25T10:14:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.689+0000 7f779f058740 1 -- start start 2024-04-25T10:14:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.690+0000 7f779f058740 1 Processor -- start 2024-04-25T10:14:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.690+0000 7f779f058740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa13750 0x56050aaa4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.690+0000 7f779f058740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050aa1cb10 0x56050aa3a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.690+0000 7f779f058740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa1cef0 0x56050a9ee600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.690+0000 7f779f058740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56050aa3d0c0 con 0x56050aa13750 2024-04-25T10:14:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.690+0000 7f779f058740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77940ca3a0 con 0x56050aa1cef0 2024-04-25T10:14:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.690+0000 7f779f058740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56050a9ea1a0 con 0x56050aa1cb10 2024-04-25T10:14:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.690+0000 7f779f058740 10 client.0 _ll_register_callbacks cb 0x7f7784003ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f779a0f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa13750 0x56050aaa4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f779a0f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa13750 0x56050aaa4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58780/0 (socket says 172.21.15.16:58780) 2024-04-25T10:14:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f779a0f6640 1 -- 172.21.15.16:0/718244348 learned_addr learned my addr 172.21.15.16:0/718244348 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f77998f5640 1 --2- 172.21.15.16:0/718244348 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050aa1cb10 0x56050aa3a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f779a0f6640 1 -- 172.21.15.16:0/718244348 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050aa1cb10 msgr2=0x56050aa3a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f779a0f6640 1 --2- 172.21.15.16:0/718244348 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050aa1cb10 0x56050aa3a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f779a0f6640 1 -- 172.21.15.16:0/718244348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa1cef0 msgr2=0x56050a9ee600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f779a8f7640 1 --2- 172.21.15.16:0/718244348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa1cef0 0x56050a9ee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f779a0f6640 1 --2- 172.21.15.16:0/718244348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa1cef0 0x56050a9ee600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f779a0f6640 1 -- 172.21.15.16:0/718244348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56050a9f43f0 con 0x56050aa13750 2024-04-25T10:14:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f779a0f6640 1 --2- 172.21.15.16:0/718244348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa13750 0x56050aaa4750 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f778c074e60 tx=0x7f778c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5809d1a67951297 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f77797fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:14:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f77797fa640 1 -- 172.21.15.16:0/718244348 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c081110 con 0x56050aa13750 2024-04-25T10:14:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f77797fa640 1 -- 172.21.15.16:0/718244348 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f778c075070 con 0x56050aa13750 2024-04-25T10:14:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f779f058740 1 -- 172.21.15.16:0/718244348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7788014dc0 con 0x56050aa13750 2024-04-25T10:14:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f77797fa640 1 -- 172.21.15.16:0/718244348 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f778c0a8200 con 0x56050aa13750 2024-04-25T10:14:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f779f058740 10 client.9240 fetch_fsmap learned FSMap version 89 2024-04-25T10:14:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f779f058740 1 -- 172.21.15.16:0/718244348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x56050aaa6100 con 0x56050aa13750 2024-04-25T10:14:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.691+0000 7f77797fa640 1 -- 172.21.15.16:0/718244348 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7788014dc0 con 0x56050aa13750 2024-04-25T10:14:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.692+0000 7f77797fa640 1 -- 172.21.15.16:0/718244348 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f778c080c80 con 0x56050aa13750 2024-04-25T10:14:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.692+0000 7f779f058740 10 client.9240 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:14:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.692+0000 7f779f058740 10 client.9240 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:14:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.692+0000 7f779f058740 20 client.9240 populate_metadata read hostname 'smithi016' 2024-04-25T10:14:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.692+0000 7f779f058740 1 --2- 172.21.15.16:0/718244348 >> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] conn(0x56050a9eeb40 0x56050aa32fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.692+0000 7f779f058740 4 client.9240 mds_command: new command op to 9096 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:14:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.692+0000 7f779f058740 1 -- 172.21.15.16:0/718244348 --> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56050aa37620 con 0x56050a9eeb40 2024-04-25T10:14:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.692+0000 7f77998f5640 1 --2- 172.21.15.16:0/718244348 >> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] conn(0x56050a9eeb40 0x56050aa32fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.692+0000 7f77998f5640 1 --2- 172.21.15.16:0/718244348 >> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] conn(0x56050a9eeb40 0x56050aa32fc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=457b67273b761424 server_cookie=ce38462a682fad8 in_seq=0 out_seq=0 2024-04-25T10:14:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.692+0000 7f77797fa640 10 client.9240 ms_handle_connect on v2:172.21.15.16:6833/2697110279 2024-04-25T10:14:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.702+0000 7f77797fa640 1 -- 172.21.15.16:0/718244348 <== mds.0 v2:172.21.15.16:6833/2697110279 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56050aa37620 con 0x56050a9eeb40 2024-04-25T10:14:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.702+0000 7f77797fa640 10 client.9240 handle_command_reply: tid=0 2024-04-25T10:14:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.702+0000 7f779f058740 1 client.9240 shutdown 2024-04-25T10:14:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779f058740 1 -- 172.21.15.16:0/718244348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa13750 msgr2=0x56050aaa4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779f058740 1 --2- 172.21.15.16:0/718244348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa13750 0x56050aaa4750 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f778c074e60 tx=0x7f778c0091d0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779f058740 1 -- 172.21.15.16:0/718244348 shutdown_connections 2024-04-25T10:14:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779f058740 1 --2- 172.21.15.16:0/718244348 >> [v2:172.21.15.16:6833/2697110279,v1:172.21.15.16:6835/2697110279] conn(0x56050a9eeb40 0x56050aa32fc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779f058740 1 --2- 172.21.15.16:0/718244348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56050aa1cef0 0x56050a9ee600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779f058740 1 --2- 172.21.15.16:0/718244348 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56050aa1cb10 0x56050aa3a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779f058740 1 --2- 172.21.15.16:0/718244348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56050aa13750 0x56050aaa4750 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f77797fa640 0 client.9240 ms_handle_reset on v2:172.21.15.16:6833/2697110279 2024-04-25T10:14:49.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779f058740 1 -- 172.21.15.16:0/718244348 >> 172.21.15.16:0/718244348 conn(0x56050a9ef770 msgr2=0x56050aa226e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779f058740 1 -- 172.21.15.16:0/718244348 shutdown_connections 2024-04-25T10:14:49.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779f058740 1 -- 172.21.15.16:0/718244348 wait complete. 2024-04-25T10:14:49.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779f058740 20 client.9240 trim_cache size 0 max 16384 2024-04-25T10:14:49.709 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:14:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "", 2024-04-25T10:14:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0 2024-04-25T10:14:49.709 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:14:49.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779c35a640 1 -- 172.21.15.16:0/68586622 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f778006b4d0 msgr2=0x7f778006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779c35a640 1 --2- 172.21.15.16:0/68586622 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f778006b4d0 0x7f778006d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f77941f70f0 tx=0x7f7788009040 comp rx=0 tx=0).stop 2024-04-25T10:14:49.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779c35a640 1 -- 172.21.15.16:0/68586622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c0039b0 msgr2=0x7f7794156c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779c35a640 1 --2- 172.21.15.16:0/68586622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c0039b0 0x7f7794156c40 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f778c009be0 tx=0x7f778c004730 comp rx=0 tx=0).stop 2024-04-25T10:14:49.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779c35a640 1 -- 172.21.15.16:0/68586622 shutdown_connections 2024-04-25T10:14:49.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779c35a640 1 --2- 172.21.15.16:0/68586622 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f778006b4d0 0x7f778006d990 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779c35a640 1 --2- 172.21.15.16:0/68586622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77940b28e0 0x7f77940b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779c35a640 1 --2- 172.21.15.16:0/68586622 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77940cae40 0x7f77940cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.703+0000 7f779c35a640 1 --2- 172.21.15.16:0/68586622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c0039b0 0x7f7794156c40 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.704+0000 7f779c35a640 1 -- 172.21.15.16:0/68586622 >> 172.21.15.16:0/68586622 conn(0x7f779406f4a0 msgr2=0x7f779406f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.704+0000 7f779c35a640 1 -- 172.21.15.16:0/68586622 shutdown_connections 2024-04-25T10:14:49.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.704+0000 7f779c35a640 1 -- 172.21.15.16:0/68586622 wait complete. 2024-04-25T10:14:49.718 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:14:49.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.816+0000 7fe92e7d5640 1 Processor -- start 2024-04-25T10:14:49.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.816+0000 7fe92e7d5640 1 -- start start 2024-04-25T10:14:49.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.817+0000 7fe92e7d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe92815dc50 0x7fe928070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.817+0000 7fe92e7d5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe9280cf600 0x7fe9280713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.817+0000 7fe92e7d5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92806c220 con 0x7fe9280cd940 2024-04-25T10:14:49.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.817+0000 7fe92e7d5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9280cfe00 con 0x7fe9280cf600 2024-04-25T10:14:49.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.817+0000 7fe92e7d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9280cec10 con 0x7fe92815dc50 2024-04-25T10:14:49.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.817+0000 7fe92cd72640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe9280cf600 0x7fe9280713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.817+0000 7fe92cd72640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe9280cf600 0x7fe9280713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58786/0 (socket says 172.21.15.16:58786) 2024-04-25T10:14:49.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.817+0000 7fe92cd72640 1 -- 172.21.15.16:0/3396417342 learned_addr learned my addr 172.21.15.16:0/3396417342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:49.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.817+0000 7fe927fff640 1 --2- 172.21.15.16:0/3396417342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe92815dc50 0x7fe928070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.817+0000 7fe92cd72640 1 -- 172.21.15.16:0/3396417342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe92815dc50 msgr2=0x7fe928070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.817+0000 7fe92cd72640 1 --2- 172.21.15.16:0/3396417342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe92815dc50 0x7fe928070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.817+0000 7fe92cd72640 1 -- 172.21.15.16:0/3396417342 >> v1:172.21.15.186:6789/0 conn(0x7fe9280cd940 legacy=0x7fe928161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.817+0000 7fe92cd72640 1 -- 172.21.15.16:0/3396417342 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9280d0050 con 0x7fe9280cf600 2024-04-25T10:14:49.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.818+0000 7fe92cd72640 1 --2- 172.21.15.16:0/3396417342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9280cf600 0x7fe9280713c0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fe920009e50 tx=0x7fe92002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8439dae7d3f3dda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.818+0000 7fe926ffd640 1 -- 172.21.15.16:0/3396417342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920036070 con 0x7fe9280cf600 2024-04-25T10:14:49.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.818+0000 7fe926ffd640 1 -- 172.21.15.16:0/3396417342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe920031600 con 0x7fe9280cf600 2024-04-25T10:14:49.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.818+0000 7fe926ffd640 1 -- 172.21.15.16:0/3396417342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9200319d0 con 0x7fe9280cf600 2024-04-25T10:14:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.819+0000 7fe92e7d5640 1 -- 172.21.15.16:0/3396417342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9280cf600 msgr2=0x7fe9280713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.819+0000 7fe92e7d5640 1 --2- 172.21.15.16:0/3396417342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9280cf600 0x7fe9280713c0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fe920009e50 tx=0x7fe92002a710 comp rx=0 tx=0).stop 2024-04-25T10:14:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.819+0000 7fe92e7d5640 1 -- 172.21.15.16:0/3396417342 shutdown_connections 2024-04-25T10:14:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.819+0000 7fe92e7d5640 1 --2- 172.21.15.16:0/3396417342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9280cf600 0x7fe9280713c0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.819+0000 7fe92e7d5640 1 --2- 172.21.15.16:0/3396417342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe92815dc50 0x7fe928070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.819+0000 7fe92e7d5640 1 -- 172.21.15.16:0/3396417342 >> 172.21.15.16:0/3396417342 conn(0x7fe92806f430 msgr2=0x7fe92806f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.819+0000 7fe92e7d5640 1 -- 172.21.15.16:0/3396417342 shutdown_connections 2024-04-25T10:14:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.819+0000 7fe92e7d5640 1 -- 172.21.15.16:0/3396417342 wait complete. 2024-04-25T10:14:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.820+0000 7fe92e7d5640 1 Processor -- start 2024-04-25T10:14:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.820+0000 7fe92e7d5640 1 -- start start 2024-04-25T10:14:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.821+0000 7fe92e7d5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9280cd940 0x7fe92815a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.821+0000 7fe92e7d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9280cf600 0x7fe9280c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.821+0000 7fe92e7d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe92815dc50 0x7fe9280b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.821+0000 7fe92e7d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9280ce8a0 con 0x7fe9280cf600 2024-04-25T10:14:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.821+0000 7fe92e7d5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9280cc0c0 con 0x7fe9280cd940 2024-04-25T10:14:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.821+0000 7fe92e7d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9280cd620 con 0x7fe92815dc50 2024-04-25T10:14:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.821+0000 7fe9277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9280cf600 0x7fe9280c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.821+0000 7fe9277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9280cf600 0x7fe9280c7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58790/0 (socket says 172.21.15.16:58790) 2024-04-25T10:14:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.821+0000 7fe9277fe640 1 -- 172.21.15.16:0/1453048701 learned_addr learned my addr 172.21.15.16:0/1453048701 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.821+0000 7fe927fff640 1 --2- 172.21.15.16:0/1453048701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9280cd940 0x7fe92815a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.821+0000 7fe92cd72640 1 --2- 172.21.15.16:0/1453048701 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe92815dc50 0x7fe9280b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.821+0000 7fe9277fe640 1 -- 172.21.15.16:0/1453048701 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe92815dc50 msgr2=0x7fe9280b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.821+0000 7fe9277fe640 1 --2- 172.21.15.16:0/1453048701 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe92815dc50 0x7fe9280b4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.821+0000 7fe9277fe640 1 -- 172.21.15.16:0/1453048701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9280cd940 msgr2=0x7fe92815a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.821+0000 7fe9277fe640 1 --2- 172.21.15.16:0/1453048701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9280cd940 0x7fe92815a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.821+0000 7fe9277fe640 1 -- 172.21.15.16:0/1453048701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9280c7c60 con 0x7fe9280cf600 2024-04-25T10:14:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.821+0000 7fe9277fe640 1 --2- 172.21.15.16:0/1453048701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9280cf600 0x7fe9280c7560 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fe91800c400 tx=0x7fe91800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.822+0000 7fe924ff9640 1 -- 172.21.15.16:0/1453048701 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9180052d0 con 0x7fe9280cf600 2024-04-25T10:14:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.822+0000 7fe924ff9640 1 -- 172.21.15.16:0/1453048701 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe918005490 con 0x7fe9280cf600 2024-04-25T10:14:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.822+0000 7fe924ff9640 1 -- 172.21.15.16:0/1453048701 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe918007260 con 0x7fe9280cf600 2024-04-25T10:14:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.822+0000 7fe92e7d5640 1 -- 172.21.15.16:0/1453048701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9280c7f40 con 0x7fe9280cf600 2024-04-25T10:14:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.822+0000 7fe92e7d5640 1 -- 172.21.15.16:0/1453048701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9280b4850 con 0x7fe9280cf600 2024-04-25T10:14:49.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.823+0000 7fe92e7d5640 1 -- 172.21.15.16:0/1453048701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9280750b0 con 0x7fe9280cf600 2024-04-25T10:14:49.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.823+0000 7fe924ff9640 1 -- 172.21.15.16:0/1453048701 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe918007400 con 0x7fe9280cf600 2024-04-25T10:14:49.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.824+0000 7fe924ff9640 1 --2- 172.21.15.16:0/1453048701 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe91406b4d0 0x7fe91406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.824+0000 7fe924ff9640 1 -- 172.21.15.16:0/1453048701 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe918089670 con 0x7fe9280cf600 2024-04-25T10:14:49.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.824+0000 7fe927fff640 1 --2- 172.21.15.16:0/1453048701 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe91406b4d0 0x7fe91406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.824+0000 7fe927fff640 1 --2- 172.21.15.16:0/1453048701 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe91406b4d0 0x7fe91406d990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe9280b74d0 tx=0x7fe91c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:49.828+0000 7fe924ff9640 1 -- 172.21.15.16:0/1453048701 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe918057ed0 con 0x7fe9280cf600 2024-04-25T10:14:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.049+0000 7fe92e7d5640 1 -- 172.21.15.16:0/1453048701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe9280b4a20 con 0x7fe9280cf600 2024-04-25T10:14:50.053 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-04-25T10:14:50.049+0000 7ff13f9bb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:14:50.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.051+0000 7fe924ff9640 1 -- 172.21.15.16:0/1453048701 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v90) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe91805bf70 con 0x7fe9280cf600 2024-04-25T10:14:50.055 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:14:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.054+0000 7fe92e7d5640 1 -- 172.21.15.16:0/1453048701 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe91406b4d0 msgr2=0x7fe91406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.054+0000 7fe92e7d5640 1 --2- 172.21.15.16:0/1453048701 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe91406b4d0 0x7fe91406d990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe9280b74d0 tx=0x7fe91c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.054+0000 7fe92e7d5640 1 -- 172.21.15.16:0/1453048701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9280cf600 msgr2=0x7fe9280c7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.054+0000 7fe92e7d5640 1 --2- 172.21.15.16:0/1453048701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9280cf600 0x7fe9280c7560 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fe91800c400 tx=0x7fe91800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.054+0000 7fe92e7d5640 1 -- 172.21.15.16:0/1453048701 shutdown_connections 2024-04-25T10:14:50.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.054+0000 7fe92e7d5640 1 --2- 172.21.15.16:0/1453048701 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe91406b4d0 0x7fe91406d990 secure :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe9280b74d0 tx=0x7fe91c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:50.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.054+0000 7fe92e7d5640 1 --2- 172.21.15.16:0/1453048701 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe92815dc50 0x7fe9280b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.054+0000 7fe92e7d5640 1 --2- 172.21.15.16:0/1453048701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9280cf600 0x7fe9280c7560 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.054+0000 7fe92e7d5640 1 --2- 172.21.15.16:0/1453048701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9280cd940 0x7fe92815a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.054+0000 7fe92e7d5640 1 -- 172.21.15.16:0/1453048701 >> 172.21.15.16:0/1453048701 conn(0x7fe92806f430 msgr2=0x7fe92806cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.054+0000 7fe92e7d5640 1 -- 172.21.15.16:0/1453048701 shutdown_connections 2024-04-25T10:14:50.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.054+0000 7fe92e7d5640 1 -- 172.21.15.16:0/1453048701 wait complete. 2024-04-25T10:14:50.061 INFO:tasks.ceph.mds.c.smithi016.stderr: -1720> 2024-04-25T10:14:45.494+0000 7fde56bdb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:14:50.071 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-04-25T10:14:50.088 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-04-25T10:14:50.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.109+0000 7facb000c280 1 -- start start 2024-04-25T10:14:50.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.109+0000 7facb000c280 1 Processor -- start 2024-04-25T10:14:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.109+0000 7facb000c280 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f58830ec00 0x55f58837e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facb000c280 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f58830f800 0x55f58837eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facb000c280 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f58830fc00 0x55f58837f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facb000c280 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58909e600 con 0x55f58830ec00 2024-04-25T10:14:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facb000c280 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58909e480 con 0x55f58830f800 2024-04-25T10:14:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facb000c280 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f58909e300 con 0x55f58830fc00 2024-04-25T10:14:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facada5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f58830ec00 0x55f58837e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facada5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f58830ec00 0x55f58837e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58846/0 (socket says 172.21.15.16:58846) 2024-04-25T10:14:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facada5c640 1 -- 172.21.15.16:0/2059304354 learned_addr learned my addr 172.21.15.16:0/2059304354 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facae25d640 1 --2- 172.21.15.16:0/2059304354 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f58830fc00 0x55f58837f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facad25b640 1 --2- 172.21.15.16:0/2059304354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f58830f800 0x55f58837eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facada5c640 1 -- 172.21.15.16:0/2059304354 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f58830fc00 msgr2=0x55f58837f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facada5c640 1 --2- 172.21.15.16:0/2059304354 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f58830fc00 0x55f58837f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facada5c640 1 -- 172.21.15.16:0/2059304354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f58830f800 msgr2=0x55f58837eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facada5c640 1 --2- 172.21.15.16:0/2059304354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f58830f800 0x55f58837eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facada5c640 1 -- 172.21.15.16:0/2059304354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f588344780 con 0x55f58830ec00 2024-04-25T10:14:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facae25d640 1 --2- 172.21.15.16:0/2059304354 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f58830fc00 0x55f58837f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facad25b640 1 --2- 172.21.15.16:0/2059304354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f58830f800 0x55f58837eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facada5c640 1 --2- 172.21.15.16:0/2059304354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f58830ec00 0x55f58837e580 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x55f5891253b0 tx=0x55f5891c4540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e28f68a4e3fab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facab257640 1 -- 172.21.15.16:0/2059304354 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f588334680 con 0x55f58830ec00 2024-04-25T10:14:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facab257640 1 -- 172.21.15.16:0/2059304354 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f58833ae00 con 0x55f58830ec00 2024-04-25T10:14:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facab257640 1 -- 172.21.15.16:0/2059304354 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f588334b60 con 0x55f58830ec00 2024-04-25T10:14:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facb000c280 1 -- 172.21.15.16:0/2059304354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5883443c0 con 0x55f58830ec00 2024-04-25T10:14:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facb000c280 1 -- 172.21.15.16:0/2059304354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55f588345680 con 0x55f58830ec00 2024-04-25T10:14:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.110+0000 7facb000c280 4 waiting for MDS map... 2024-04-25T10:14:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.111+0000 7facab257640 1 -- 172.21.15.16:0/2059304354 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(117..117 src has 1..117) v4 ==== 5881+0+0 (secure 0 0 0) 0x55f5891da600 con 0x55f58830ec00 2024-04-25T10:14:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.111+0000 7facab257640 1 -- 172.21.15.16:0/2059304354 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 90) v1 ==== 2723+0+0 (secure 0 0 0) 0x55f588369180 con 0x55f58830ec00 2024-04-25T10:14:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.111+0000 7facb000c280 4 Got MDS map 90 2024-04-25T10:14:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.111+0000 7facb000c280 10 main: JournalTool::main 2024-04-25T10:14:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.111+0000 7facb000c280 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:14:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.111+0000 7facb000c280 1 Processor -- start 2024-04-25T10:14:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.111+0000 7facb000c280 1 -- start start 2024-04-25T10:14:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.112+0000 7facb000c280 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5891dd000 0x55f58837f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.112+0000 7facb000c280 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5891dd400 0x55f58837fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.112+0000 7facb000c280 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5891dd800 0x55f588380100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.112+0000 7facb000c280 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58909e180 con 0x55f5891dd400 2024-04-25T10:14:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.112+0000 7facb000c280 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f588370480 con 0x55f5891dd000 2024-04-25T10:14:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.112+0000 7facb000c280 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f588370900 con 0x55f5891dd800 2024-04-25T10:14:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.112+0000 7facada5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5891dd400 0x55f58837fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.112+0000 7facada5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5891dd400 0x55f58837fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58850/0 (socket says 172.21.15.16:58850) 2024-04-25T10:14:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.112+0000 7facada5c640 1 -- 172.21.15.16:0/2986278130 learned_addr learned my addr 172.21.15.16:0/2986278130 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.112+0000 7facae25d640 1 --2- 172.21.15.16:0/2986278130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5891dd000 0x55f58837f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.112+0000 7facad25b640 1 --2- 172.21.15.16:0/2986278130 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5891dd800 0x55f588380100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.112+0000 7facada5c640 1 -- 172.21.15.16:0/2986278130 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5891dd800 msgr2=0x55f588380100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.112+0000 7facada5c640 1 --2- 172.21.15.16:0/2986278130 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5891dd800 0x55f588380100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.112+0000 7facada5c640 1 -- 172.21.15.16:0/2986278130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5891dd000 msgr2=0x55f58837f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.112+0000 7facada5c640 1 --2- 172.21.15.16:0/2986278130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5891dd000 0x55f58837f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.112+0000 7facada5c640 1 -- 172.21.15.16:0/2986278130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5883454a0 con 0x55f5891dd400 2024-04-25T10:14:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.112+0000 7facada5c640 1 --2- 172.21.15.16:0/2986278130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5891dd400 0x55f58837fb80 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x55f5891f0660 tx=0x55f5891c4060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456e9b852b8f5143 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.113+0000 7faca9253640 1 -- 172.21.15.16:0/2986278130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f588334820 con 0x55f5891dd400 2024-04-25T10:14:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.113+0000 7faca9253640 1 -- 172.21.15.16:0/2986278130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f58833ac40 con 0x55f5891dd400 2024-04-25T10:14:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.113+0000 7faca9253640 1 -- 172.21.15.16:0/2986278130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f588334680 con 0x55f5891dd400 2024-04-25T10:14:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.113+0000 7facb000c280 1 -- 172.21.15.16:0/2986278130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5891dd400 msgr2=0x55f58837fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.113+0000 7facb000c280 1 --2- 172.21.15.16:0/2986278130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5891dd400 0x55f58837fb80 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x55f5891f0660 tx=0x55f5891c4060 comp rx=0 tx=0).stop 2024-04-25T10:14:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.113+0000 7facb000c280 1 -- 172.21.15.16:0/2986278130 shutdown_connections 2024-04-25T10:14:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.113+0000 7facb000c280 1 --2- 172.21.15.16:0/2986278130 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5891dd800 0x55f588380100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.113+0000 7facb000c280 1 --2- 172.21.15.16:0/2986278130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5891dd400 0x55f58837fb80 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.113+0000 7facb000c280 1 --2- 172.21.15.16:0/2986278130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5891dd000 0x55f58837f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.113+0000 7facb000c280 1 -- 172.21.15.16:0/2986278130 >> 172.21.15.16:0/2986278130 conn(0x55f5891dcc00 msgr2=0x55f5883f4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.113+0000 7facb000c280 1 -- 172.21.15.16:0/2986278130 shutdown_connections 2024-04-25T10:14:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.113+0000 7facb000c280 1 -- 172.21.15.16:0/2986278130 wait complete. 2024-04-25T10:14:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.113+0000 7facb000c280 1 Processor -- start 2024-04-25T10:14:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.114+0000 7facb000c280 1 -- start start 2024-04-25T10:14:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.114+0000 7facb000c280 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5891dd000 0x55f588381180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.114+0000 7facb000c280 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5891dd400 0x55f588380c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.114+0000 7facb000c280 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5891dd800 0x55f588380680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.114+0000 7facb000c280 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58909f380 con 0x55f5891dd400 2024-04-25T10:14:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.114+0000 7facb000c280 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58909f500 con 0x55f5891dd800 2024-04-25T10:14:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.114+0000 7facb000c280 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f58909e180 con 0x55f5891dd000 2024-04-25T10:14:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.114+0000 7facada5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5891dd400 0x55f588380c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.114+0000 7facada5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5891dd400 0x55f588380c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58854/0 (socket says 172.21.15.16:58854) 2024-04-25T10:14:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.114+0000 7facada5c640 1 -- 172.21.15.16:0/678208314 learned_addr learned my addr 172.21.15.16:0/678208314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.114+0000 7facae25d640 1 --2- 172.21.15.16:0/678208314 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5891dd000 0x55f588381180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.114+0000 7facad25b640 1 --2- 172.21.15.16:0/678208314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5891dd800 0x55f588380680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.115+0000 7facada5c640 1 -- 172.21.15.16:0/678208314 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5891dd000 msgr2=0x55f588381180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.115+0000 7facada5c640 1 --2- 172.21.15.16:0/678208314 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5891dd000 0x55f588381180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.115+0000 7facada5c640 1 -- 172.21.15.16:0/678208314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5891dd800 msgr2=0x55f588380680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.115+0000 7facada5c640 1 --2- 172.21.15.16:0/678208314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5891dd800 0x55f588380680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.115+0000 7facada5c640 1 -- 172.21.15.16:0/678208314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f588344780 con 0x55f5891dd400 2024-04-25T10:14:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.115+0000 7facad25b640 1 --2- 172.21.15.16:0/678208314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5891dd800 0x55f588380680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.115+0000 7facada5c640 1 --2- 172.21.15.16:0/678208314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5891dd400 0x55f588380c00 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x55f5891f1320 tx=0x55f5891d2300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.115+0000 7faca7a50640 1 -- 172.21.15.16:0/678208314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f588334b60 con 0x55f5891dd400 2024-04-25T10:14:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.115+0000 7faca7a50640 1 -- 172.21.15.16:0/678208314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f58833a1c0 con 0x55f5891dd400 2024-04-25T10:14:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.115+0000 7facb000c280 1 -- 172.21.15.16:0/678208314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f5883443c0 con 0x55f5891dd400 2024-04-25T10:14:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.115+0000 7faca7a50640 1 -- 172.21.15.16:0/678208314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f5883344e0 con 0x55f5891dd400 2024-04-25T10:14:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.115+0000 7facb000c280 1 -- 172.21.15.16:0/678208314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f588344000 con 0x55f5891dd400 2024-04-25T10:14:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.115+0000 7facb000c280 4 main: JournalTool: resolving pool 16 2024-04-25T10:14:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.116+0000 7faca7a50640 1 -- 172.21.15.16:0/678208314 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x55f5891d4c00 con 0x55f5891dd400 2024-04-25T10:14:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.116+0000 7faca7a50640 1 --2- 172.21.15.16:0/678208314 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55f588670000 0x55f588381700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.117+0000 7faca7a50640 1 -- 172.21.15.16:0/678208314 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5881+0+0 (secure 0 0 0) 0x55f58924c600 con 0x55f5891dd400 2024-04-25T10:14:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.117+0000 7facb000c280 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:14:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.117+0000 7facb000c280 4 main: Executing for rank 0 2024-04-25T10:14:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.117+0000 7facae25d640 1 --2- 172.21.15.16:0/678208314 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55f588670000 0x55f588381700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.117+0000 7facb000c280 1 -- start start 2024-04-25T10:14:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.117+0000 7facb000c280 1 Processor -- start 2024-04-25T10:14:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.117+0000 7facae25d640 1 --2- 172.21.15.16:0/678208314 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55f588670000 0x55f588381700 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x55f58908e840 tx=0x55f588680c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.118+0000 7facb000c280 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f588670800 0x55f588380100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.118+0000 7facb000c280 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f588670c00 0x55f58837fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.118+0000 7facb000c280 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f588671000 0x55f58837f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.118+0000 7facad25b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f588671000 0x55f58837f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.118+0000 7facad25b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f588671000 0x55f58837f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58870/0 (socket says 172.21.15.16:58870) 2024-04-25T10:14:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.118+0000 7facad25b640 1 -- 172.21.15.16:0/3945558090 learned_addr learned my addr 172.21.15.16:0/3945558090 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.118+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58909f800 con 0x55f588671000 2024-04-25T10:14:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.118+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58909f980 con 0x55f588670800 2024-04-25T10:14:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.118+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f588370900 con 0x55f588670c00 2024-04-25T10:14:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.118+0000 7facae25d640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f588670800 0x55f588380100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.118+0000 7facada5c640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f588670c00 0x55f58837fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.118+0000 7facae25d640 1 -- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f588670c00 msgr2=0x55f58837fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.118+0000 7facae25d640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f588670c00 0x55f58837fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.118+0000 7facae25d640 1 -- 172.21.15.16:0/3945558090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f588671000 msgr2=0x55f58837f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.118+0000 7facae25d640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f588671000 0x55f58837f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.118+0000 7facae25d640 1 -- 172.21.15.16:0/3945558090 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f588344960 con 0x55f588670800 2024-04-25T10:14:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.119+0000 7facae25d640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f588670800 0x55f588380100 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x55f5886238c0 tx=0x55f5891c4120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=309d2ef5fc782a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.119+0000 7faca4a4a640 1 -- 172.21.15.16:0/3945558090 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f58927a9c0 con 0x55f588670800 2024-04-25T10:14:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.119+0000 7faca4a4a640 1 -- 172.21.15.16:0/3945558090 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f58833aa80 con 0x55f588670800 2024-04-25T10:14:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.119+0000 7faca4a4a640 1 -- 172.21.15.16:0/3945558090 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f58927a820 con 0x55f588670800 2024-04-25T10:14:50.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.119+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f588345680 con 0x55f588670800 2024-04-25T10:14:50.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.119+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55f5883454a0 con 0x55f588670800 2024-04-25T10:14:50.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.119+0000 7facb000c280 4 waiting for MDS map... 2024-04-25T10:14:50.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.119+0000 7faca4a4a640 1 -- 172.21.15.16:0/3945558090 <== mon.1 v2:172.21.15.186:3300/0 4 ==== osd_map(117..117 src has 1..117) v4 ==== 5881+0+0 (secure 0 0 0) 0x55f5886e0600 con 0x55f588670800 2024-04-25T10:14:50.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.119+0000 7faca4a4a640 1 -- 172.21.15.16:0/3945558090 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 90) v1 ==== 2723+0+0 (secure 0 0 0) 0x55f5886ec000 con 0x55f588670800 2024-04-25T10:14:50.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.119+0000 7facb000c280 4 Got MDS map 90 2024-04-25T10:14:50.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.120+0000 7facb000c280 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55f588671400 0x55f5886f8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.120+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:1 16.f 16:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v9 -- 0x55f5886df000 con 0x55f588671400 2024-04-25T10:14:50.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.120+0000 7facada5c640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55f588671400 0x55f5886f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.120+0000 7facada5c640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55f588671400 0x55f5886f8000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=e4cfb76aeeea82c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.123+0000 7facada5c640 1 -- 172.21.15.16:0/3945558090 <== osd.5 v2:172.21.15.186:6800/1827108421 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v117'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55f58840a000 con 0x55f588671400 2024-04-25T10:14:50.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.124+0000 7facb000c280 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x55f5886df400 0x55f5886f8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.124+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:2 16.14 16:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v9 -- 0x55f5886df800 con 0x55f5886df400 2024-04-25T10:14:50.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.124+0000 7facad25b640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x55f5886df400 0x55f5886f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.124+0000 7facad25b640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x55f5886df400 0x55f5886f8580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=42b13aa9f8413e26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facad25b640 1 -- 172.21.15.16:0/3945558090 <== osd.4 v2:172.21.15.186:6824/3091472889 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v117'39 uv39 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55f58840a500 con 0x55f5886df400 2024-04-25T10:14:50.134 INFO:teuthology.orchestra.run.smithi016.stdout:writing EResetJournal entry 2024-04-25T10:14:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:3 16.14 16:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v9 -- 0x55f5886dfc00 con 0x55f5886df400 2024-04-25T10:14:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facb000c280 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55f5886df800 0x55f5886f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:4 16.1b 16:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v9 -- 0x55f5890d9400 con 0x55f5886df800 2024-04-25T10:14:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facb000c280 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55f5890d9000 0x55f5886f9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:5 16.b 16:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v9 -- 0x55f5890d8800 con 0x55f5890d9000 2024-04-25T10:14:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facb000c280 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55f5890d8c00 0x55f5886f9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facae25d640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55f5886df800 0x55f5886f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:6 16.1f 16:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v9 -- 0x55f58870a000 con 0x55f5890d8c00 2024-04-25T10:14:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facad25b640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55f5890d8c00 0x55f5886f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facb000c280 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55f58870a400 0x55f5886f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:7 16.1a 16:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v9 -- 0x55f58870a800 con 0x55f58870a400 2024-04-25T10:14:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facada5c640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55f5890d9000 0x55f5886f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facb000c280 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55f58870ac00 0x55f5886fa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:8 16.1e 16:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v9 -- 0x55f58870b000 con 0x55f58870ac00 2024-04-25T10:14:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facada5c640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55f58870ac00 0x55f5886fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facae25d640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55f5886df800 0x55f5886f8b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=681768f4117ae58c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facad25b640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55f5890d8c00 0x55f5886f9600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=3efeeb3ca810fa8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.126+0000 7facae25d640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55f58870a400 0x55f5886f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.127+0000 7facada5c640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55f58870ac00 0x55f5886fa100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=a60c4b9e4a426d2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.127+0000 7facae25d640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55f58870a400 0x55f5886f9b80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=e08ee09a863fb4a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.127+0000 7facada5c640 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55f5890d9000 0x55f5886f9080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=118272d9d07b14a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.127+0000 7facad25b640 1 -- 172.21.15.16:0/3945558090 <== osd.4 v2:172.21.15.186:6824/3091472889 2 ==== osd_op_reply(3 200.00000001 [delete] v117'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55f58840a500 con 0x55f5886df400 2024-04-25T10:14:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.128+0000 7facae25d640 1 -- 172.21.15.16:0/3945558090 <== osd.1 v2:172.21.15.16:6808/1916704159 1 ==== osd_op_reply(4 200.00000002 [delete] v117'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55f58840a780 con 0x55f5886df800 2024-04-25T10:14:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.128+0000 7faca3247640 1 -- 172.21.15.16:0/3945558090 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:9 16.14 16:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v9 -- 0x55f588720000 con 0x55f5886df400 2024-04-25T10:14:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.128+0000 7facada5c640 1 -- 172.21.15.16:0/3945558090 <== osd.2 v2:172.21.15.16:6800/2094356376 1 ==== osd_op_reply(8 200.00000006 [delete] v117'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55f58840a000 con 0x55f58870ac00 2024-04-25T10:14:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.129+0000 7facada5c640 1 -- 172.21.15.16:0/3945558090 <== osd.7 v2:172.21.15.186:6808/1627307043 1 ==== osd_op_reply(5 200.00000003 [delete] v117'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55f58840a000 con 0x55f5890d9000 2024-04-25T10:14:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.129+0000 7facad25b640 1 -- 172.21.15.16:0/3945558090 <== osd.4 v2:172.21.15.186:6824/3091472889 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v117'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55f58840a500 con 0x55f5886df400 2024-04-25T10:14:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facad25b640 1 -- 172.21.15.16:0/3945558090 <== osd.0 v2:172.21.15.16:6816/2167750256 1 ==== osd_op_reply(6 200.00000004 [delete] v117'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55f58840a500 con 0x55f5890d8c00 2024-04-25T10:14:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facae25d640 1 -- 172.21.15.16:0/3945558090 <== osd.6 v2:172.21.15.186:6816/1123185664 1 ==== osd_op_reply(7 200.00000005 [delete] v117'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55f58840a780 con 0x55f58870a400 2024-04-25T10:14:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facb000c280 4 Successfully wrote new journal pointer and header for rank 8:0 2024-04-25T10:14:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55f5890d8c00 msgr2=0x55f5886f9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facb000c280 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55f5890d8c00 0x55f5886f9600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55f5886df800 msgr2=0x55f5886f8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facb000c280 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55f5886df800 0x55f5886f8b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55f58870ac00 msgr2=0x55f5886fa100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facb000c280 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55f58870ac00 0x55f5886fa100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x55f5886df400 msgr2=0x55f5886f8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facb000c280 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x55f5886df400 0x55f5886f8580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55f588671400 msgr2=0x55f5886f8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facb000c280 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55f588671400 0x55f5886f8000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facad25b640 1 -- 172.21.15.16:0/3945558090 reap_dead start 2024-04-25T10:14:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55f58870a400 msgr2=0x55f5886f9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facad25b640 1 -- 172.21.15.16:0/3945558090 reap_dead start 2024-04-25T10:14:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facad25b640 1 -- 172.21.15.16:0/3945558090 reap_dead start 2024-04-25T10:14:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facb000c280 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55f58870a400 0x55f5886f9b80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55f5890d9000 msgr2=0x55f5886f9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facb000c280 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55f5890d9000 0x55f5886f9080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f588670800 msgr2=0x55f588380100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.130+0000 7facb000c280 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f588670800 0x55f588380100 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x55f5886238c0 tx=0x55f5891c4120 comp rx=0 tx=0).stop 2024-04-25T10:14:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 shutdown_connections 2024-04-25T10:14:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55f58870a400 0x55f5886f9b80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55f5890d9000 0x55f5886f9080 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 --2- 172.21.15.16:0/3945558090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f588670800 0x55f588380100 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 >> 172.21.15.16:0/3945558090 conn(0x55f588670400 msgr2=0x55f5883f4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 shutdown_connections 2024-04-25T10:14:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 -- 172.21.15.16:0/3945558090 wait complete. 2024-04-25T10:14:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 -- 172.21.15.16:0/678208314 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55f588670000 msgr2=0x55f588381700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 --2- 172.21.15.16:0/678208314 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55f588670000 0x55f588381700 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x55f58908e840 tx=0x55f588680c00 comp rx=0 tx=0).stop 2024-04-25T10:14:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 -- 172.21.15.16:0/678208314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5891dd400 msgr2=0x55f588380c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 --2- 172.21.15.16:0/678208314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5891dd400 0x55f588380c00 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x55f5891f1320 tx=0x55f5891d2300 comp rx=0 tx=0).stop 2024-04-25T10:14:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 -- 172.21.15.16:0/678208314 shutdown_connections 2024-04-25T10:14:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 --2- 172.21.15.16:0/678208314 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55f588670000 0x55f588381700 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 --2- 172.21.15.16:0/678208314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5891dd800 0x55f588380680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 --2- 172.21.15.16:0/678208314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5891dd400 0x55f588380c00 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 --2- 172.21.15.16:0/678208314 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5891dd000 0x55f588381180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 -- 172.21.15.16:0/678208314 >> 172.21.15.16:0/678208314 conn(0x55f5891dcc00 msgr2=0x55f5883f4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 -- 172.21.15.16:0/678208314 shutdown_connections 2024-04-25T10:14:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 -- 172.21.15.16:0/678208314 wait complete. 2024-04-25T10:14:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 -- 172.21.15.16:0/2059304354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f58830ec00 msgr2=0x55f58837e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 --2- 172.21.15.16:0/2059304354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f58830ec00 0x55f58837e580 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x55f5891253b0 tx=0x55f5891c4540 comp rx=0 tx=0).stop 2024-04-25T10:14:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 -- 172.21.15.16:0/2059304354 shutdown_connections 2024-04-25T10:14:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 --2- 172.21.15.16:0/2059304354 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f58830fc00 0x55f58837f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 --2- 172.21.15.16:0/2059304354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f58830f800 0x55f58837eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 --2- 172.21.15.16:0/2059304354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f58830ec00 0x55f58837e580 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 -- 172.21.15.16:0/2059304354 >> 172.21.15.16:0/2059304354 conn(0x55f58830f000 msgr2=0x55f5883f5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 -- 172.21.15.16:0/2059304354 shutdown_connections 2024-04-25T10:14:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.131+0000 7facb000c280 1 -- 172.21.15.16:0/2059304354 wait complete. 2024-04-25T10:14:50.149 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.077921, result: writing EResetJournal entry 2024-04-25T10:14:50.149 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-25T10:14:50.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.219+0000 7f10cb22d740 1 -- start start 2024-04-25T10:14:50.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.219+0000 7f10cb22d740 1 Processor -- start 2024-04-25T10:14:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.220+0000 7f10cb22d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab1c52c00 0x563ab1cc2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.220+0000 7f10cb22d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab1c53800 0x563ab1cc2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.220+0000 7f10cb22d740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563ab1c53c00 0x563ab1cc3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.220+0000 7f10cb22d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab2b0e180 con 0x563ab1c53800 2024-04-25T10:14:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.220+0000 7f10cb22d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab2b0e300 con 0x563ab1c52c00 2024-04-25T10:14:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.220+0000 7f10cb22d740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ab2b0e480 con 0x563ab1c53c00 2024-04-25T10:14:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.220+0000 7f10c833e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab1c53800 0x563ab1cc2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.220+0000 7f10c833e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab1c53800 0x563ab1cc2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58896/0 (socket says 172.21.15.16:58896) 2024-04-25T10:14:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.220+0000 7f10c833e640 1 -- 172.21.15.16:0/57886160 learned_addr learned my addr 172.21.15.16:0/57886160 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.220+0000 7f10c8b3f640 1 --2- 172.21.15.16:0/57886160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab1c52c00 0x563ab1cc2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.220+0000 7f10c9340640 1 --2- 172.21.15.16:0/57886160 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563ab1c53c00 0x563ab1cc3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.221+0000 7f10c833e640 1 -- 172.21.15.16:0/57886160 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563ab1c53c00 msgr2=0x563ab1cc3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.221+0000 7f10c833e640 1 --2- 172.21.15.16:0/57886160 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563ab1c53c00 0x563ab1cc3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.221+0000 7f10c833e640 1 -- 172.21.15.16:0/57886160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab1c52c00 msgr2=0x563ab1cc2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.221+0000 7f10c833e640 1 --2- 172.21.15.16:0/57886160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab1c52c00 0x563ab1cc2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.221+0000 7f10c833e640 1 -- 172.21.15.16:0/57886160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ab1c7f680 con 0x563ab1c53800 2024-04-25T10:14:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.221+0000 7f10c9340640 1 --2- 172.21.15.16:0/57886160 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563ab1c53c00 0x563ab1cc3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.221+0000 7f10c833e640 1 --2- 172.21.15.16:0/57886160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab1c53800 0x563ab1cc2b00 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x563ab2a6b200 tx=0x563ab2b16120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3aee375aef64a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.221+0000 7f10c633a640 1 -- 172.21.15.16:0/57886160 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563ab1c789c0 con 0x563ab1c53800 2024-04-25T10:14:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.221+0000 7f10c633a640 1 -- 172.21.15.16:0/57886160 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563ab1c9ce00 con 0x563ab1c53800 2024-04-25T10:14:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.221+0000 7f10c633a640 1 -- 172.21.15.16:0/57886160 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563ab1c78680 con 0x563ab1c53800 2024-04-25T10:14:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.221+0000 7f10cb22d740 1 -- 172.21.15.16:0/57886160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ab1c7f860 con 0x563ab1c53800 2024-04-25T10:14:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.221+0000 7f10cb22d740 1 -- 172.21.15.16:0/57886160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x563ab1c7f2c0 con 0x563ab1c53800 2024-04-25T10:14:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.221+0000 7f10cb22d740 4 waiting for MDS map... 2024-04-25T10:14:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.221+0000 7f10c633a640 1 -- 172.21.15.16:0/57886160 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(117..117 src has 1..117) v4 ==== 5881+0+0 (secure 0 0 0) 0x563ab2b28200 con 0x563ab1c53800 2024-04-25T10:14:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.221+0000 7f10c633a640 1 -- 172.21.15.16:0/57886160 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 90) v1 ==== 2723+0+0 (secure 0 0 0) 0x563ab1cab180 con 0x563ab1c53800 2024-04-25T10:14:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.221+0000 7f10cb22d740 4 Got MDS map 90 2024-04-25T10:14:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.221+0000 7f10cb22d740 4 datascan.main: Using metadata pool output 2024-04-25T10:14:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.221+0000 7f10cb22d740 4 datascan.main: connecting to RADOS... 2024-04-25T10:14:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.222+0000 7f10cb22d740 1 Processor -- start 2024-04-25T10:14:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.222+0000 7f10cb22d740 1 -- start start 2024-04-25T10:14:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.222+0000 7f10cb22d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2a2fc00 0x563ab1cc3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.222+0000 7f10cb22d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2b74000 0x563ab1cc3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.222+0000 7f10cb22d740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563ab2b74400 0x563ab1cc4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.222+0000 7f10cb22d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab2b0e600 con 0x563ab2a2fc00 2024-04-25T10:14:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.222+0000 7f10cb22d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab2b0e780 con 0x563ab2b74000 2024-04-25T10:14:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.222+0000 7f10cb22d740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ab2b0e900 con 0x563ab2b74400 2024-04-25T10:14:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.222+0000 7f10c9340640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2a2fc00 0x563ab1cc3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.222+0000 7f10c9340640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2a2fc00 0x563ab1cc3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58904/0 (socket says 172.21.15.16:58904) 2024-04-25T10:14:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.222+0000 7f10c9340640 1 -- 172.21.15.16:0/1764862026 learned_addr learned my addr 172.21.15.16:0/1764862026 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10c8b3f640 1 --2- 172.21.15.16:0/1764862026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2b74000 0x563ab1cc3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10c833e640 1 --2- 172.21.15.16:0/1764862026 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563ab2b74400 0x563ab1cc4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10c9340640 1 -- 172.21.15.16:0/1764862026 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563ab2b74400 msgr2=0x563ab1cc4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10c9340640 1 --2- 172.21.15.16:0/1764862026 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563ab2b74400 0x563ab1cc4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10c9340640 1 -- 172.21.15.16:0/1764862026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2b74000 msgr2=0x563ab1cc3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10c9340640 1 --2- 172.21.15.16:0/1764862026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2b74000 0x563ab1cc3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10c9340640 1 -- 172.21.15.16:0/1764862026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ab1c7f4a0 con 0x563ab2a2fc00 2024-04-25T10:14:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10c8b3f640 1 --2- 172.21.15.16:0/1764862026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2b74000 0x563ab1cc3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10c9340640 1 --2- 172.21.15.16:0/1764862026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2a2fc00 0x563ab1cc3600 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x563ab2a4c930 tx=0x563ab2b88000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed27b72fe03d919 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10c4336640 1 -- 172.21.15.16:0/1764862026 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563ab1c789c0 con 0x563ab2a2fc00 2024-04-25T10:14:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10c4336640 1 -- 172.21.15.16:0/1764862026 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563ab1c9ca80 con 0x563ab2a2fc00 2024-04-25T10:14:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10c4336640 1 -- 172.21.15.16:0/1764862026 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563ab1c78680 con 0x563ab2a2fc00 2024-04-25T10:14:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10cb22d740 1 -- 172.21.15.16:0/1764862026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2a2fc00 msgr2=0x563ab1cc3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10cb22d740 1 --2- 172.21.15.16:0/1764862026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2a2fc00 0x563ab1cc3600 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x563ab2a4c930 tx=0x563ab2b88000 comp rx=0 tx=0).stop 2024-04-25T10:14:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10cb22d740 1 -- 172.21.15.16:0/1764862026 shutdown_connections 2024-04-25T10:14:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10cb22d740 1 --2- 172.21.15.16:0/1764862026 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563ab2b74400 0x563ab1cc4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10cb22d740 1 --2- 172.21.15.16:0/1764862026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2b74000 0x563ab1cc3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10cb22d740 1 --2- 172.21.15.16:0/1764862026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2a2fc00 0x563ab1cc3600 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.223+0000 7f10cb22d740 1 -- 172.21.15.16:0/1764862026 >> 172.21.15.16:0/1764862026 conn(0x563ab2a2f800 msgr2=0x563ab1d3b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.224+0000 7f10cb22d740 1 -- 172.21.15.16:0/1764862026 shutdown_connections 2024-04-25T10:14:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.224+0000 7f10cb22d740 1 -- 172.21.15.16:0/1764862026 wait complete. 2024-04-25T10:14:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.224+0000 7f10cb22d740 1 Processor -- start 2024-04-25T10:14:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.224+0000 7f10cb22d740 1 -- start start 2024-04-25T10:14:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10cb22d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2a2f800 0x563ab1cc5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10cb22d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2b75000 0x563ab1cc4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10cb22d740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563ab2b74c00 0x563ab1cc4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10cb22d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab2b0fc80 con 0x563ab2a2f800 2024-04-25T10:14:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10cb22d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab2b0fb00 con 0x563ab2b75000 2024-04-25T10:14:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10cb22d740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ab2b0f980 con 0x563ab2b74c00 2024-04-25T10:14:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10c9340640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2a2f800 0x563ab1cc5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10c9340640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2a2f800 0x563ab1cc5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58918/0 (socket says 172.21.15.16:58918) 2024-04-25T10:14:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 learned_addr learned my addr 172.21.15.16:0/589051477 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10c8b3f640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2b75000 0x563ab1cc4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10c833e640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563ab2b74c00 0x563ab1cc4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563ab2b74c00 msgr2=0x563ab1cc4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10c9340640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563ab2b74c00 0x563ab1cc4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2b75000 msgr2=0x563ab1cc4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10c9340640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2b75000 0x563ab1cc4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ab1c7f0e0 con 0x563ab2a2f800 2024-04-25T10:14:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10c9340640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2a2f800 0x563ab1cc5180 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x563ab2a4dbf0 tx=0x563ab2b16420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10c2b33640 1 -- 172.21.15.16:0/589051477 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563ab1c784e0 con 0x563ab2a2f800 2024-04-25T10:14:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10c2b33640 1 -- 172.21.15.16:0/589051477 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563ab1c9d340 con 0x563ab2a2f800 2024-04-25T10:14:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563ab1c7fa40 con 0x563ab2a2f800 2024-04-25T10:14:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10c2b33640 1 -- 172.21.15.16:0/589051477 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563ab1c789c0 con 0x563ab2a2f800 2024-04-25T10:14:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ab1c7e3c0 con 0x563ab2a2f800 2024-04-25T10:14:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.225+0000 7f10cb22d740 4 datascan.init: resolving metadata pool 16 2024-04-25T10:14:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.227+0000 7f10c2b33640 1 -- 172.21.15.16:0/589051477 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x563ab2b8ec00 con 0x563ab2a2f800 2024-04-25T10:14:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.227+0000 7f10c2b33640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x563ab1fb8000 0x563ab1cc5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.227+0000 7f10c2b33640 1 -- 172.21.15.16:0/589051477 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5881+0+0 (secure 0 0 0) 0x563ab2b7b400 con 0x563ab2a2f800 2024-04-25T10:14:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.227+0000 7f10cb22d740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:14:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.227+0000 7f10cb22d740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-04-25T10:14:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.227+0000 7f10cb22d740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:14:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.227+0000 7f10cb22d740 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x563ab2b74800 0x563ab1cc4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.227+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab2b74400 con 0x563ab2b74800 2024-04-25T10:14:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.227+0000 7f10c8b3f640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x563ab1fb8000 0x563ab1cc5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.227+0000 7f10c833e640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x563ab2b74800 0x563ab1cc4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.228+0000 7f10c833e640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x563ab2b74800 0x563ab1cc4100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.228+0000 7f10c8b3f640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x563ab1fb8000 0x563ab1cc5700 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x563ab2a6bb90 tx=0x563ab2b882a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.228+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.0 v2:172.21.15.16:6816/2167750256 1 ==== osd_op_reply(1 [pgnls start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab2b74800 2024-04-25T10:14:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.228+0000 7f10c3b35640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x563ab1fb9000 0x563ab1fca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.228+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:2 17.20 17:04000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fb9400 con 0x563ab1fb9000 2024-04-25T10:14:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.228+0000 7f10c9340640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x563ab1fb9000 0x563ab1fca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.229+0000 7f10c9340640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x563ab1fb9000 0x563ab1fca000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.229+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.7 v2:172.21.15.186:6808/1627307043 1 ==== osd_op_reply(2 [pgnls start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fb9000 2024-04-25T10:14:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.229+0000 7f10c4336640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x563ab1fb9c00 0x563ab1fca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.229+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:3 17.10 17:08000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fe0000 con 0x563ab1fb9c00 2024-04-25T10:14:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.229+0000 7f10c8b3f640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x563ab1fb9c00 0x563ab1fca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.230+0000 7f10c8b3f640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x563ab1fb9c00 0x563ab1fca580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.230+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 <== osd.4 v2:172.21.15.186:6824/3091472889 1 ==== osd_op_reply(3 [pgnls start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40780 con 0x563ab1fb9c00 2024-04-25T10:14:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.230+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:4 17.30 17:0c000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fb9800 con 0x563ab1fb9000 2024-04-25T10:14:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.230+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.7 v2:172.21.15.186:6808/1627307043 2 ==== osd_op_reply(4 [pgnls start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fb9000 2024-04-25T10:14:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.230+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:5 17.8 17:10000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fe0400 con 0x563ab2b74800 2024-04-25T10:14:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.230+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.0 v2:172.21.15.16:6816/2167750256 2 ==== osd_op_reply(5 [pgnls start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab2b74800 2024-04-25T10:14:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.230+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:6 17.28 17:14000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fe0800 con 0x563ab1fb9c00 2024-04-25T10:14:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.231+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 <== osd.4 v2:172.21.15.186:6824/3091472889 2 ==== osd_op_reply(6 [pgnls start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40780 con 0x563ab1fb9c00 2024-04-25T10:14:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.231+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:7 17.18 17:18000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fe1400 con 0x563ab1fb9000 2024-04-25T10:14:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.231+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.7 v2:172.21.15.186:6808/1627307043 3 ==== osd_op_reply(7 [pgnls start_epoch 117 out=81b] v115'17 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fb9000 2024-04-25T10:14:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.231+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:8 17.18 17:1ad8d55b:::10000000006.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab2b74000 con 0x563ab1fb9000 2024-04-25T10:14:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.231+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.7 v2:172.21.15.186:6808/1627307043 4 ==== osd_op_reply(8 10000000006.00000001 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fb9000 2024-04-25T10:14:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.231+0000 7f10cb22d740 10 datascan.operator(): handling object 1099511627782.1 2024-04-25T10:14:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.231+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:9 17.25 17:a5feb6f3:::10000000006.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab2b74000 con 0x563ab1fb9c00 2024-04-25T10:14:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.233+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 <== osd.4 v2:172.21.15.186:6824/3091472889 3 ==== osd_op_reply(9 10000000006.00000000 [call] v117'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563ab1d40780 con 0x563ab1fb9c00 2024-04-25T10:14:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.233+0000 7f10cb22d740 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x563ab1fe1400 0x563ab1cc3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.233+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:10 17.38 17:1c000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fb9400 con 0x563ab1fe1400 2024-04-25T10:14:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.233+0000 7f10c833e640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x563ab1fe1400 0x563ab1cc3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.233+0000 7f10c833e640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x563ab1fe1400 0x563ab1cc3b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.233+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.3 v2:172.21.15.16:6824/256577261 1 ==== osd_op_reply(10 [pgnls start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab1fe1400 2024-04-25T10:14:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.234+0000 7f10c4336640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x563ab1fe1800 0x563ab1fcab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.234+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:11 17.4 17:20000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fe1c00 con 0x563ab1fe1800 2024-04-25T10:14:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.234+0000 7f10c9340640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x563ab1fe1800 0x563ab1fcab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.234+0000 7f10c9340640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x563ab1fe1800 0x563ab1fcab00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.234+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.6 v2:172.21.15.186:6816/1123185664 1 ==== osd_op_reply(11 [pgnls start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fe1800 2024-04-25T10:14:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.234+0000 7f10c3b35640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x563ab1fe0c00 0x563ab1fcb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.234+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:12 17.24 17:24000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fe1000 con 0x563ab1fe0c00 2024-04-25T10:14:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.235+0000 7f10c8b3f640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x563ab1fe0c00 0x563ab1fcb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.235+0000 7f10c8b3f640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x563ab1fe0c00 0x563ab1fcb600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.235+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 <== osd.5 v2:172.21.15.186:6800/1827108421 1 ==== osd_op_reply(12 [pgnls start_epoch 117 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40780 con 0x563ab1fe0c00 2024-04-25T10:14:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.235+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:13 17.14 17:28000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab2b74000 con 0x563ab1fe0c00 2024-04-25T10:14:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.236+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 <== osd.5 v2:172.21.15.186:6800/1827108421 2 ==== osd_op_reply(13 [pgnls start_epoch 117 out=49b] v115'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40780 con 0x563ab1fe0c00 2024-04-25T10:14:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.236+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:14 17.34 17:2c000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ff6000 con 0x563ab1fb9000 2024-04-25T10:14:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.236+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.7 v2:172.21.15.186:6808/1627307043 5 ==== osd_op_reply(14 [pgnls start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fb9000 2024-04-25T10:14:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.236+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:15 17.c 17:30000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fe0000 con 0x563ab1fe1400 2024-04-25T10:14:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.236+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.3 v2:172.21.15.16:6824/256577261 2 ==== osd_op_reply(15 [pgnls start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab1fe1400 2024-04-25T10:14:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.236+0000 7f10c3b35640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x563ab1ff6400 0x563ab1fcbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.236+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:16 17.2c 17:34000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ff6800 con 0x563ab1ff6400 2024-04-25T10:14:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.236+0000 7f10c833e640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x563ab1ff6400 0x563ab1fcbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.237+0000 7f10c833e640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x563ab1ff6400 0x563ab1fcbb80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.237+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.1 v2:172.21.15.16:6808/1916704159 1 ==== osd_op_reply(16 [pgnls start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab1ff6400 2024-04-25T10:14:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.237+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:17 17.1c 17:38000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fe0800 con 0x563ab1fe0c00 2024-04-25T10:14:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.237+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 <== osd.5 v2:172.21.15.186:6800/1827108421 3 ==== osd_op_reply(17 [pgnls start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40780 con 0x563ab1fe0c00 2024-04-25T10:14:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.237+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:18 17.3c 17:3c000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ff6c00 con 0x563ab1fb9c00 2024-04-25T10:14:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.237+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 <== osd.4 v2:172.21.15.186:6824/3091472889 4 ==== osd_op_reply(18 [pgnls start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40780 con 0x563ab1fb9c00 2024-04-25T10:14:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.238+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:19 17.2 17:40000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fb8c00 con 0x563ab1ff6400 2024-04-25T10:14:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.238+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.1 v2:172.21.15.16:6808/1916704159 2 ==== osd_op_reply(19 [pgnls start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab1ff6400 2024-04-25T10:14:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.238+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:20 17.22 17:44000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fe0400 con 0x563ab1fe1400 2024-04-25T10:14:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.238+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.3 v2:172.21.15.16:6824/256577261 3 ==== osd_op_reply(20 [pgnls start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab1fe1400 2024-04-25T10:14:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.238+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:21 17.12 17:48000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ff7000 con 0x563ab1fb9000 2024-04-25T10:14:50.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.238+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.7 v2:172.21.15.186:6808/1627307043 6 ==== osd_op_reply(21 [pgnls start_epoch 117 out=81b] v116'22 uv22 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fb9000 2024-04-25T10:14:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.238+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:22 17.12 17:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fb9800 con 0x563ab1fb9000 2024-04-25T10:14:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.239+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.7 v2:172.21.15.186:6808/1627307043 7 ==== osd_op_reply(22 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fb9000 2024-04-25T10:14:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.239+0000 7f10cb22d740 10 datascan.operator(): handling object 1099511627778.0 2024-04-25T10:14:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.239+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:23 17.12 17:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffc000 con 0x563ab1fb9000 2024-04-25T10:14:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.240+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.7 v2:172.21.15.186:6808/1627307043 8 ==== osd_op_reply(23 10000000002.00000000 [call] v117'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fb9000 2024-04-25T10:14:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.240+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:24 17.32 17:4c000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffc400 con 0x563ab2b74800 2024-04-25T10:14:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.240+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.0 v2:172.21.15.16:6816/2167750256 3 ==== osd_op_reply(24 [pgnls start_epoch 117 out=81b] v115'16 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563ab1d40000 con 0x563ab2b74800 2024-04-25T10:14:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.240+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:25 17.32 17:4def72d4:::10000000003.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffc800 con 0x563ab2b74800 2024-04-25T10:14:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.241+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.0 v2:172.21.15.16:6816/2167750256 4 ==== osd_op_reply(25 10000000003.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563ab1d40000 con 0x563ab2b74800 2024-04-25T10:14:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.241+0000 7f10cb22d740 10 datascan.operator(): handling object 1099511627779.0 2024-04-25T10:14:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.241+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:26 17.32 17:4def72d4:::10000000003.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffcc00 con 0x563ab2b74800 2024-04-25T10:14:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.242+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.0 v2:172.21.15.16:6816/2167750256 5 ==== osd_op_reply(26 10000000003.00000000 [call] v117'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563ab1d40000 con 0x563ab2b74800 2024-04-25T10:14:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.242+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:27 17.a 17:50000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffd000 con 0x563ab1ff6400 2024-04-25T10:14:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.242+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.1 v2:172.21.15.16:6808/1916704159 3 ==== osd_op_reply(27 [pgnls start_epoch 117 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab1ff6400 2024-04-25T10:14:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.242+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:28 17.2a 17:54000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fe0000 con 0x563ab1fe0c00 2024-04-25T10:14:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.242+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 <== osd.5 v2:172.21.15.186:6800/1827108421 4 ==== osd_op_reply(28 [pgnls start_epoch 117 out=49b] v115'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40780 con 0x563ab1fe0c00 2024-04-25T10:14:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.242+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:29 17.1a 17:58000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ff7400 con 0x563ab1fb9c00 2024-04-25T10:14:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.243+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 <== osd.4 v2:172.21.15.186:6824/3091472889 5 ==== osd_op_reply(29 [pgnls start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40780 con 0x563ab1fb9c00 2024-04-25T10:14:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.243+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:30 17.3a 17:5c000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fb9400 con 0x563ab1fe0c00 2024-04-25T10:14:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.243+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 <== osd.5 v2:172.21.15.186:6800/1827108421 5 ==== osd_op_reply(30 [pgnls start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40780 con 0x563ab1fe0c00 2024-04-25T10:14:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.243+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:31 17.6 17:60000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ff7800 con 0x563ab2b74800 2024-04-25T10:14:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.243+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.0 v2:172.21.15.16:6816/2167750256 6 ==== osd_op_reply(31 [pgnls start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab2b74800 2024-04-25T10:14:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.243+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:32 17.26 17:64000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab2b74400 con 0x563ab1fe0c00 2024-04-25T10:14:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.244+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 <== osd.5 v2:172.21.15.186:6800/1827108421 6 ==== osd_op_reply(32 [pgnls start_epoch 117 out=49b] v115'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40780 con 0x563ab1fe0c00 2024-04-25T10:14:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.244+0000 7f10c4336640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x563ab1ff7c00 0x563ab1fcb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.244+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:33 17.16 17:68000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fe1000 con 0x563ab1ff7c00 2024-04-25T10:14:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.244+0000 7f10c9340640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x563ab1ff7c00 0x563ab1fcb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.244+0000 7f10c9340640 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x563ab1ff7c00 0x563ab1fcb080 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.244+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.2 v2:172.21.15.16:6800/2094356376 1 ==== osd_op_reply(33 [pgnls start_epoch 117 out=49b] v115'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1ff7c00 2024-04-25T10:14:50.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.244+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:34 17.36 17:6c000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab2a2f400 con 0x563ab1fb9000 2024-04-25T10:14:50.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.245+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.7 v2:172.21.15.186:6808/1627307043 9 ==== osd_op_reply(34 [pgnls start_epoch 117 out=49b] v115'32 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fb9000 2024-04-25T10:14:50.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.245+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:35 17.e 17:70000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fe0800 con 0x563ab1fe1800 2024-04-25T10:14:50.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.245+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.6 v2:172.21.15.186:6816/1123185664 2 ==== osd_op_reply(35 [pgnls start_epoch 117 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fe1800 2024-04-25T10:14:50.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.245+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:36 17.2e 17:74000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fb8c00 con 0x563ab1fe1400 2024-04-25T10:14:50.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.245+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.3 v2:172.21.15.16:6824/256577261 4 ==== osd_op_reply(36 [pgnls start_epoch 117 out=49b] v115'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab1fe1400 2024-04-25T10:14:50.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.245+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:37 17.1e 17:78000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fe0000 con 0x563ab1fe1800 2024-04-25T10:14:50.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.246+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.6 v2:172.21.15.186:6816/1123185664 3 ==== osd_op_reply(37 [pgnls start_epoch 117 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fe1800 2024-04-25T10:14:50.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.246+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:38 17.3e 17:7c000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fb9800 con 0x563ab1ff7c00 2024-04-25T10:14:50.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.246+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.2 v2:172.21.15.16:6800/2094356376 2 ==== osd_op_reply(38 [pgnls start_epoch 117 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1ff7c00 2024-04-25T10:14:50.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.246+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:39 17.1 17:80000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fe0400 con 0x563ab1fe0c00 2024-04-25T10:14:50.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.246+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 <== osd.5 v2:172.21.15.186:6800/1827108421 7 ==== osd_op_reply(39 [pgnls start_epoch 117 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40780 con 0x563ab1fe0c00 2024-04-25T10:14:50.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.246+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:40 17.21 17:84000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ff6c00 con 0x563ab1fe1800 2024-04-25T10:14:50.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.247+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.6 v2:172.21.15.186:6816/1123185664 4 ==== osd_op_reply(40 [pgnls start_epoch 117 out=49b] v115'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fe1800 2024-04-25T10:14:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.247+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:41 17.11 17:88000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fe1c00 con 0x563ab1fb9c00 2024-04-25T10:14:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.247+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 <== osd.4 v2:172.21.15.186:6824/3091472889 6 ==== osd_op_reply(41 [pgnls start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40780 con 0x563ab1fb9c00 2024-04-25T10:14:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.247+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:42 17.31 17:8c000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ff6800 con 0x563ab1fe1800 2024-04-25T10:14:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.247+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.6 v2:172.21.15.186:6816/1123185664 5 ==== osd_op_reply(42 [pgnls start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fe1800 2024-04-25T10:14:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.247+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:43 17.9 17:90000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab2b74000 con 0x563ab1fb9000 2024-04-25T10:14:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.248+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.7 v2:172.21.15.186:6808/1627307043 10 ==== osd_op_reply(43 [pgnls start_epoch 117 out=49b] v115'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fb9000 2024-04-25T10:14:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.248+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:44 17.29 17:94000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ff6000 con 0x563ab2b74800 2024-04-25T10:14:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.248+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.0 v2:172.21.15.16:6816/2167750256 7 ==== osd_op_reply(44 [pgnls start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab2b74800 2024-04-25T10:14:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.248+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:45 17.19 17:98000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffc400 con 0x563ab1fe1400 2024-04-25T10:14:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.248+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.3 v2:172.21.15.16:6824/256577261 5 ==== osd_op_reply(45 [pgnls start_epoch 117 out=49b] v115'21 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab1fe1400 2024-04-25T10:14:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.248+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:46 17.39 17:9c000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ff7000 con 0x563ab1fe1400 2024-04-25T10:14:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.248+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.3 v2:172.21.15.16:6824/256577261 6 ==== osd_op_reply(46 [pgnls start_epoch 117 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab1fe1400 2024-04-25T10:14:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.248+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:47 17.5 17:a0000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ff6c00 con 0x563ab2b74800 2024-04-25T10:14:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.249+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.0 v2:172.21.15.16:6816/2167750256 8 ==== osd_op_reply(47 [pgnls start_epoch 117 out=81b] v116'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563ab1d40000 con 0x563ab2b74800 2024-04-25T10:14:50.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.249+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:48 17.5 17:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffd400 con 0x563ab2b74800 2024-04-25T10:14:50.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.249+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.0 v2:172.21.15.16:6816/2167750256 9 ==== osd_op_reply(48 10000000004.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563ab1d40000 con 0x563ab2b74800 2024-04-25T10:14:50.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.249+0000 7f10cb22d740 10 datascan.operator(): handling object 1099511627780.0 2024-04-25T10:14:50.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.249+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:49 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffd800 con 0x563ab2b74800 2024-04-25T10:14:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.250+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.0 v2:172.21.15.16:6816/2167750256 10 ==== osd_op_reply(49 10000000004.00000000 [call] v117'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563ab1d40000 con 0x563ab2b74800 2024-04-25T10:14:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.250+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:50 17.25 17:a4000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffdc00 con 0x563ab1fb9c00 2024-04-25T10:14:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.251+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 <== osd.4 v2:172.21.15.186:6824/3091472889 7 ==== osd_op_reply(50 [pgnls start_epoch 117 out=81b] v117'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563ab1d40780 con 0x563ab1fb9c00 2024-04-25T10:14:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.251+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:51 17.25 17:a5feb6f3:::10000000006.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ff7400 con 0x563ab1fb9c00 2024-04-25T10:14:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.251+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 <== osd.4 v2:172.21.15.186:6824/3091472889 8 ==== osd_op_reply(51 10000000006.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563ab1d40780 con 0x563ab1fb9c00 2024-04-25T10:14:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.251+0000 7f10cb22d740 10 datascan.operator(): handling object 1099511627782.0 2024-04-25T10:14:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.251+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:52 17.25 17:a5feb6f3:::10000000006.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ff7800 con 0x563ab1fb9c00 2024-04-25T10:14:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.252+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 <== osd.4 v2:172.21.15.186:6824/3091472889 9 ==== osd_op_reply(52 10000000006.00000000 [call] v117'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563ab1d40780 con 0x563ab1fb9c00 2024-04-25T10:14:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.252+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:53 17.15 17:a8000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ff6000 con 0x563ab1fe1400 2024-04-25T10:14:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.253+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.3 v2:172.21.15.16:6824/256577261 7 ==== osd_op_reply(53 [pgnls start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab1fe1400 2024-04-25T10:14:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.253+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:54 17.35 17:ac000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffdc00 con 0x563ab1fe1400 2024-04-25T10:14:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.253+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.3 v2:172.21.15.16:6824/256577261 8 ==== osd_op_reply(54 [pgnls start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab1fe1400 2024-04-25T10:14:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.253+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:55 17.d 17:b0000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fb9800 con 0x563ab2b74800 2024-04-25T10:14:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.253+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.0 v2:172.21.15.16:6816/2167750256 11 ==== osd_op_reply(55 [pgnls start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab2b74800 2024-04-25T10:14:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.253+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:56 17.2d 17:b4000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffd000 con 0x563ab1ff6400 2024-04-25T10:14:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.253+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.1 v2:172.21.15.16:6808/1916704159 4 ==== osd_op_reply(56 [pgnls start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab1ff6400 2024-04-25T10:14:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.253+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:57 17.1d 17:b8000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fe1000 con 0x563ab1ff6400 2024-04-25T10:14:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.254+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.1 v2:172.21.15.16:6808/1916704159 5 ==== osd_op_reply(57 [pgnls start_epoch 117 out=49b] v115'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab1ff6400 2024-04-25T10:14:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.254+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:58 17.3d 17:bc000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffc400 con 0x563ab1ff6400 2024-04-25T10:14:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.254+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.1 v2:172.21.15.16:6808/1916704159 6 ==== osd_op_reply(58 [pgnls start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab1ff6400 2024-04-25T10:14:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.254+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:59 17.3 17:c0000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fe0800 con 0x563ab2b74800 2024-04-25T10:14:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.254+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.0 v2:172.21.15.16:6816/2167750256 12 ==== osd_op_reply(59 [pgnls start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab2b74800 2024-04-25T10:14:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.254+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:60 17.23 17:c4000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffc800 con 0x563ab2b74800 2024-04-25T10:14:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.254+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.0 v2:172.21.15.16:6816/2167750256 13 ==== osd_op_reply(60 [pgnls start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab2b74800 2024-04-25T10:14:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.254+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:61 17.13 17:c8000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1fe0000 con 0x563ab1fb9c00 2024-04-25T10:14:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.255+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 <== osd.4 v2:172.21.15.186:6824/3091472889 10 ==== osd_op_reply(61 [pgnls start_epoch 117 out=49b] v115'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40780 con 0x563ab1fb9c00 2024-04-25T10:14:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.255+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:62 17.33 17:cc000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffcc00 con 0x563ab1ff7c00 2024-04-25T10:14:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.255+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.2 v2:172.21.15.16:6800/2094356376 3 ==== osd_op_reply(62 [pgnls start_epoch 117 out=49b] v115'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1ff7c00 2024-04-25T10:14:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.255+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:63 17.b 17:d0000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab2a2f400 con 0x563ab1fe1800 2024-04-25T10:14:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.256+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.6 v2:172.21.15.186:6816/1123185664 6 ==== osd_op_reply(63 [pgnls start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fe1800 2024-04-25T10:14:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.256+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:64 17.2b 17:d4000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffc000 con 0x563ab1fe1400 2024-04-25T10:14:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.256+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.3 v2:172.21.15.16:6824/256577261 9 ==== osd_op_reply(64 [pgnls start_epoch 117 out=49b] v115'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab1fe1400 2024-04-25T10:14:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.256+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:65 17.1b 17:d8000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffd400 con 0x563ab1fe1400 2024-04-25T10:14:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.256+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.3 v2:172.21.15.16:6824/256577261 10 ==== osd_op_reply(65 [pgnls start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab1fe1400 2024-04-25T10:14:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.256+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:66 17.3b 17:dc000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab200c000 con 0x563ab1fb9000 2024-04-25T10:14:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.257+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.7 v2:172.21.15.186:6808/1627307043 11 ==== osd_op_reply(66 [pgnls start_epoch 117 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fb9000 2024-04-25T10:14:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.257+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:67 17.7 17:e0000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffd800 con 0x563ab1fe0c00 2024-04-25T10:14:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.257+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 <== osd.5 v2:172.21.15.186:6800/1827108421 8 ==== osd_op_reply(67 [pgnls start_epoch 117 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40780 con 0x563ab1fe0c00 2024-04-25T10:14:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.257+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:68 17.27 17:e4000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab200c400 con 0x563ab1fe1400 2024-04-25T10:14:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.257+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.3 v2:172.21.15.16:6824/256577261 11 ==== osd_op_reply(68 [pgnls start_epoch 117 out=49b] v115'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab1fe1400 2024-04-25T10:14:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.257+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:69 17.17 17:e8000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffdc00 con 0x563ab1ff7c00 2024-04-25T10:14:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.257+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.2 v2:172.21.15.16:6800/2094356376 4 ==== osd_op_reply(69 [pgnls start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1ff7c00 2024-04-25T10:14:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.257+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:70 17.37 17:ec000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab200c800 con 0x563ab1fe1800 2024-04-25T10:14:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.258+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.6 v2:172.21.15.186:6816/1123185664 7 ==== osd_op_reply(70 [pgnls start_epoch 117 out=49b] v115'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fe1800 2024-04-25T10:14:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.258+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:71 17.f 17:f0000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffd000 con 0x563ab1fe1800 2024-04-25T10:14:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.258+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.6 v2:172.21.15.186:6816/1123185664 8 ==== osd_op_reply(71 [pgnls start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fe1800 2024-04-25T10:14:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.258+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:72 17.2f 17:f4000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab200cc00 con 0x563ab1ff6400 2024-04-25T10:14:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.258+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.1 v2:172.21.15.16:6808/1916704159 7 ==== osd_op_reply(72 [pgnls start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab1ff6400 2024-04-25T10:14:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.258+0000 7f10c4336640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:73 17.1f 17:f8000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab1ffc400 con 0x563ab2b74800 2024-04-25T10:14:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10c833e640 1 -- 172.21.15.16:0/589051477 <== osd.0 v2:172.21.15.16:6816/2167750256 14 ==== osd_op_reply(73 [pgnls start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40000 con 0x563ab2b74800 2024-04-25T10:14:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10c3b35640 1 -- 172.21.15.16:0/589051477 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:74 17.3f 17:fc000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x563ab200d000 con 0x563ab1fe1800 2024-04-25T10:14:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10c9340640 1 -- 172.21.15.16:0/589051477 <== osd.6 v2:172.21.15.186:6816/1123185664 9 ==== osd_op_reply(74 [pgnls start_epoch 117 out=49b] v115'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563ab1d40500 con 0x563ab1fe1800 2024-04-25T10:14:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x563ab2b74800 msgr2=0x563ab1cc4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x563ab2b74800 0x563ab1cc4100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x563ab1ff6400 msgr2=0x563ab1fcbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x563ab1ff6400 0x563ab1fcbb80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x563ab1ff7c00 msgr2=0x563ab1fcb080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x563ab1ff7c00 0x563ab1fcb080 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x563ab1fe1400 msgr2=0x563ab1cc3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x563ab1fe1400 0x563ab1cc3b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x563ab1fb9c00 msgr2=0x563ab1fca580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x563ab1fb9c00 0x563ab1fca580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 reap_dead start 2024-04-25T10:14:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x563ab1fe0c00 msgr2=0x563ab1fcb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x563ab1fe0c00 0x563ab1fcb600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 reap_dead start 2024-04-25T10:14:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x563ab1fe1800 msgr2=0x563ab1fcab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x563ab1fe1800 0x563ab1fcab00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x563ab1fb9000 msgr2=0x563ab1fca000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x563ab1fb9000 0x563ab1fca000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x563ab1fb8000 msgr2=0x563ab1cc5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x563ab1fb8000 0x563ab1cc5700 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x563ab2a6bb90 tx=0x563ab2b882a0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2a2f800 msgr2=0x563ab1cc5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 --2- 172.21.15.16:0/589051477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2a2f800 0x563ab1cc5180 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x563ab2a4dbf0 tx=0x563ab2b16420 comp rx=0 tx=0).stop 2024-04-25T10:14:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10c8b3f640 1 -- 172.21.15.16:0/589051477 reap_dead start 2024-04-25T10:14:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 shutdown_connections 2024-04-25T10:14:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 >> 172.21.15.16:0/589051477 conn(0x563ab2a2fc00 msgr2=0x563ab1d3b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 shutdown_connections 2024-04-25T10:14:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.259+0000 7f10cb22d740 1 -- 172.21.15.16:0/589051477 wait complete. 2024-04-25T10:14:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.260+0000 7f10cb22d740 1 -- 172.21.15.16:0/57886160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab1c53800 msgr2=0x563ab1cc2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.260+0000 7f10cb22d740 1 --2- 172.21.15.16:0/57886160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab1c53800 0x563ab1cc2b00 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x563ab2a6b200 tx=0x563ab2b16120 comp rx=0 tx=0).stop 2024-04-25T10:14:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.260+0000 7f10cb22d740 1 -- 172.21.15.16:0/57886160 shutdown_connections 2024-04-25T10:14:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.260+0000 7f10cb22d740 1 --2- 172.21.15.16:0/57886160 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x563ab1c53c00 0x563ab1cc3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.260+0000 7f10cb22d740 1 --2- 172.21.15.16:0/57886160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab1c53800 0x563ab1cc2b00 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.260+0000 7f10cb22d740 1 --2- 172.21.15.16:0/57886160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab1c52c00 0x563ab1cc2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.260+0000 7f10cb22d740 1 -- 172.21.15.16:0/57886160 >> 172.21.15.16:0/57886160 conn(0x563ab1c53000 msgr2=0x563ab1d3a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.260+0000 7f10cb22d740 1 -- 172.21.15.16:0/57886160 shutdown_connections 2024-04-25T10:14:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.260+0000 7f10cb22d740 1 -- 172.21.15.16:0/57886160 wait complete. 2024-04-25T10:14:50.278 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.128386, result: 2024-04-25T10:14:50.278 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-04-25T10:14:50.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.348+0000 7fcd41828740 1 -- start start 2024-04-25T10:14:50.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.348+0000 7fcd41828740 1 Processor -- start 2024-04-25T10:14:50.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.349+0000 7fcd41828740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555eb5360000 0x555eb45f5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.349+0000 7fcd41828740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5361c00 0x555eb45f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.349+0000 7fcd41828740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555eb544e000 0x555eb45f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.349+0000 7fcd41828740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb5454000 con 0x555eb5361c00 2024-04-25T10:14:50.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.349+0000 7fcd41828740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb5454180 con 0x555eb544e000 2024-04-25T10:14:50.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.349+0000 7fcd41828740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x555eb5454300 con 0x555eb5360000 2024-04-25T10:14:50.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.349+0000 7fcd3eccd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5361c00 0x555eb45f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.349+0000 7fcd3eccd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5361c00 0x555eb45f4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58932/0 (socket says 172.21.15.16:58932) 2024-04-25T10:14:50.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.349+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2611541338 learned_addr learned my addr 172.21.15.16:0/2611541338 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:50.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.349+0000 7fcd3fccf640 1 --2- 172.21.15.16:0/2611541338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555eb544e000 0x555eb45f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.349+0000 7fcd3f4ce640 1 --2- 172.21.15.16:0/2611541338 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555eb5360000 0x555eb45f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.349+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2611541338 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555eb5360000 msgr2=0x555eb45f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.349+0000 7fcd3eccd640 1 --2- 172.21.15.16:0/2611541338 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555eb5360000 0x555eb45f5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.349+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2611541338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555eb544e000 msgr2=0x555eb45f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.349+0000 7fcd3eccd640 1 --2- 172.21.15.16:0/2611541338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555eb544e000 0x555eb45f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.349+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2611541338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eb45b1680 con 0x555eb5361c00 2024-04-25T10:14:50.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.349+0000 7fcd3eccd640 1 --2- 172.21.15.16:0/2611541338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5361c00 0x555eb45f4b00 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x555eb53a77a0 tx=0x555eb5468000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773dc3a892fb8ac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.350+0000 7fcd3ccc9640 1 -- 172.21.15.16:0/2611541338 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555eb45aad00 con 0x555eb5361c00 2024-04-25T10:14:50.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.350+0000 7fcd3ccc9640 1 -- 172.21.15.16:0/2611541338 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555eb45ce1c0 con 0x555eb5361c00 2024-04-25T10:14:50.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.350+0000 7fcd3ccc9640 1 -- 172.21.15.16:0/2611541338 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555eb45aa680 con 0x555eb5361c00 2024-04-25T10:14:50.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.350+0000 7fcd41828740 1 -- 172.21.15.16:0/2611541338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555eb45b12c0 con 0x555eb5361c00 2024-04-25T10:14:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.350+0000 7fcd41828740 1 -- 172.21.15.16:0/2611541338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x555eb45b1a40 con 0x555eb5361c00 2024-04-25T10:14:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.350+0000 7fcd41828740 4 waiting for MDS map... 2024-04-25T10:14:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.350+0000 7fcd3ccc9640 1 -- 172.21.15.16:0/2611541338 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(117..117 src has 1..117) v4 ==== 5881+0+0 (secure 0 0 0) 0x555eb546c600 con 0x555eb5361c00 2024-04-25T10:14:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.350+0000 7fcd3ccc9640 1 -- 172.21.15.16:0/2611541338 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 90) v1 ==== 2723+0+0 (secure 0 0 0) 0x555eb45dd180 con 0x555eb5361c00 2024-04-25T10:14:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.350+0000 7fcd41828740 4 Got MDS map 90 2024-04-25T10:14:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.350+0000 7fcd41828740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-04-25T10:14:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.350+0000 7fcd41828740 4 datascan.main: Using metadata pool output 2024-04-25T10:14:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.350+0000 7fcd41828740 4 datascan.main: connecting to RADOS... 2024-04-25T10:14:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.350+0000 7fcd41828740 1 Processor -- start 2024-04-25T10:14:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.351+0000 7fcd41828740 1 -- start start 2024-04-25T10:14:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.351+0000 7fcd41828740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555eb544e800 0x555eb45f4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.351+0000 7fcd41828740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb544ec00 0x555eb45f6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.351+0000 7fcd41828740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555eb544f000 0x555eb45f6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.351+0000 7fcd41828740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb5454480 con 0x555eb544ec00 2024-04-25T10:14:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.351+0000 7fcd41828740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb5454600 con 0x555eb544e800 2024-04-25T10:14:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.351+0000 7fcd41828740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x555eb5454780 con 0x555eb544f000 2024-04-25T10:14:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.351+0000 7fcd3f4ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb544ec00 0x555eb45f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.351+0000 7fcd3f4ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb544ec00 0x555eb45f6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58946/0 (socket says 172.21.15.16:58946) 2024-04-25T10:14:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.351+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/1746934210 learned_addr learned my addr 172.21.15.16:0/1746934210 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.351+0000 7fcd3fccf640 1 --2- 172.21.15.16:0/1746934210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555eb544e800 0x555eb45f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.351+0000 7fcd3eccd640 1 --2- 172.21.15.16:0/1746934210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555eb544f000 0x555eb45f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.351+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/1746934210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555eb544f000 msgr2=0x555eb45f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.351+0000 7fcd3f4ce640 1 --2- 172.21.15.16:0/1746934210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555eb544f000 0x555eb45f6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.351+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/1746934210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555eb544e800 msgr2=0x555eb45f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.351+0000 7fcd3f4ce640 1 --2- 172.21.15.16:0/1746934210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555eb544e800 0x555eb45f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.351+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/1746934210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eb45b03c0 con 0x555eb544ec00 2024-04-25T10:14:50.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.351+0000 7fcd3eccd640 1 --2- 172.21.15.16:0/1746934210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555eb544f000 0x555eb45f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:50.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.352+0000 7fcd3f4ce640 1 --2- 172.21.15.16:0/1746934210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb544ec00 0x555eb45f6100 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x555eb5476c90 tx=0x555eb53175c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474197b4299845b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.352+0000 7fcd3acc5640 1 -- 172.21.15.16:0/1746934210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555eb45aad00 con 0x555eb544ec00 2024-04-25T10:14:50.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.352+0000 7fcd3acc5640 1 -- 172.21.15.16:0/1746934210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555eb45cec40 con 0x555eb544ec00 2024-04-25T10:14:50.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.352+0000 7fcd3acc5640 1 -- 172.21.15.16:0/1746934210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555eb45aa680 con 0x555eb544ec00 2024-04-25T10:14:50.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.352+0000 7fcd41828740 1 -- 172.21.15.16:0/1746934210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb544ec00 msgr2=0x555eb45f6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.352+0000 7fcd41828740 1 --2- 172.21.15.16:0/1746934210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb544ec00 0x555eb45f6100 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x555eb5476c90 tx=0x555eb53175c0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.352+0000 7fcd41828740 1 -- 172.21.15.16:0/1746934210 shutdown_connections 2024-04-25T10:14:50.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.352+0000 7fcd41828740 1 --2- 172.21.15.16:0/1746934210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555eb544f000 0x555eb45f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.352+0000 7fcd41828740 1 --2- 172.21.15.16:0/1746934210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb544ec00 0x555eb45f6100 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.352+0000 7fcd41828740 1 --2- 172.21.15.16:0/1746934210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555eb544e800 0x555eb45f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.352+0000 7fcd41828740 1 -- 172.21.15.16:0/1746934210 >> 172.21.15.16:0/1746934210 conn(0x555eb544e400 msgr2=0x555eb466d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.352+0000 7fcd41828740 1 -- 172.21.15.16:0/1746934210 shutdown_connections 2024-04-25T10:14:50.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.352+0000 7fcd41828740 1 -- 172.21.15.16:0/1746934210 wait complete. 2024-04-25T10:14:50.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.353+0000 7fcd41828740 1 Processor -- start 2024-04-25T10:14:50.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.353+0000 7fcd41828740 1 -- start start 2024-04-25T10:14:50.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.353+0000 7fcd41828740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555eb544e800 0x555eb45f7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.353+0000 7fcd41828740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb544ec00 0x555eb45f7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.353+0000 7fcd41828740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555eb544f000 0x555eb45f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.353+0000 7fcd41828740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb5455c80 con 0x555eb544ec00 2024-04-25T10:14:50.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.353+0000 7fcd41828740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb5455b00 con 0x555eb544e800 2024-04-25T10:14:50.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.353+0000 7fcd41828740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x555eb5455980 con 0x555eb544f000 2024-04-25T10:14:50.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.353+0000 7fcd3f4ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb544ec00 0x555eb45f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.353+0000 7fcd3f4ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb544ec00 0x555eb45f7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58950/0 (socket says 172.21.15.16:58950) 2024-04-25T10:14:50.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.353+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 learned_addr learned my addr 172.21.15.16:0/2194945898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:50.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.354+0000 7fcd3fccf640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555eb544e800 0x555eb45f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.354+0000 7fcd3eccd640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555eb544f000 0x555eb45f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.354+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555eb544f000 msgr2=0x555eb45f6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.354+0000 7fcd3f4ce640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555eb544f000 0x555eb45f6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.354+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555eb544e800 msgr2=0x555eb45f7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.354+0000 7fcd3f4ce640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555eb544e800 0x555eb45f7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.354+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eb45b1860 con 0x555eb544ec00 2024-04-25T10:14:50.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.354+0000 7fcd3f4ce640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb544ec00 0x555eb45f7180 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x555eb5477950 tx=0x555eb5309e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.354+0000 7fcd394c2640 1 -- 172.21.15.16:0/2194945898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555eb45aa820 con 0x555eb544ec00 2024-04-25T10:14:50.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.354+0000 7fcd394c2640 1 -- 172.21.15.16:0/2194945898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555eb45cea80 con 0x555eb544ec00 2024-04-25T10:14:50.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.354+0000 7fcd394c2640 1 -- 172.21.15.16:0/2194945898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555eb45aad00 con 0x555eb544ec00 2024-04-25T10:14:50.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.354+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555eb45b10e0 con 0x555eb544ec00 2024-04-25T10:14:50.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.354+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555eb45b14a0 con 0x555eb544ec00 2024-04-25T10:14:50.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.354+0000 7fcd41828740 4 datascan.init: resolving metadata pool 16 2024-04-25T10:14:50.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.355+0000 7fcd394c2640 1 -- 172.21.15.16:0/2194945898 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x555eb54c6c00 con 0x555eb544ec00 2024-04-25T10:14:50.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.356+0000 7fcd394c2640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x555eb48f6000 0x555eb48fc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.356+0000 7fcd394c2640 1 -- 172.21.15.16:0/2194945898 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5881+0+0 (secure 0 0 0) 0x555eb54c4a00 con 0x555eb544ec00 2024-04-25T10:14:50.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.356+0000 7fcd41828740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:14:50.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.356+0000 7fcd41828740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-04-25T10:14:50.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.356+0000 7fcd41828740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:14:50.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.356+0000 7fcd41828740 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x555eb48f6400 0x555eb45f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.356+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:1 16.14 16:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb48f6800 con 0x555eb48f6400 2024-04-25T10:14:50.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.356+0000 7fcd3fccf640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x555eb48f6000 0x555eb48fc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.356+0000 7fcd3eccd640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x555eb48f6400 0x555eb45f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.356+0000 7fcd3fccf640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x555eb48f6000 0x555eb48fc000 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x555eb45c5470 tx=0x555eb54d05a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.356+0000 7fcd3eccd640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x555eb48f6400 0x555eb45f6680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.357+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.4 v2:172.21.15.186:6824/3091472889 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv34 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f6400 2024-04-25T10:14:50.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.357+0000 7fcd41828740 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x555eb48f6c00 0x555eb45f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.357+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:2 16.1f 16:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb48f7000 con 0x555eb48f6c00 2024-04-25T10:14:50.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.357+0000 7fcd3fccf640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x555eb48f6c00 0x555eb45f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.357+0000 7fcd3fccf640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x555eb48f6c00 0x555eb45f6100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.358+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.0 v2:172.21.15.16:6816/2167750256 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x555eb4672500 con 0x555eb48f6c00 2024-04-25T10:14:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.358+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:3 17.0 17:00000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb48f7400 con 0x555eb48f6c00 2024-04-25T10:14:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.358+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.0 v2:172.21.15.16:6816/2167750256 2 ==== osd_op_reply(3 [pgnls-filter start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb48f6c00 2024-04-25T10:14:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.358+0000 7fcd3a4c4640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x555eb48f7c00 0x555eb48fc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.358+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:4 17.20 17:04000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491a000 con 0x555eb48f7c00 2024-04-25T10:14:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.358+0000 7fcd3f4ce640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x555eb48f7c00 0x555eb48fc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.359+0000 7fcd3f4ce640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x555eb48f7c00 0x555eb48fc580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.359+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.7 v2:172.21.15.186:6808/1627307043 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb48f7c00 2024-04-25T10:14:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.359+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:5 17.10 17:08000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491a800 con 0x555eb48f6400 2024-04-25T10:14:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.359+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.4 v2:172.21.15.186:6824/3091472889 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f6400 2024-04-25T10:14:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.359+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:6 17.30 17:0c000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491a400 con 0x555eb48f7c00 2024-04-25T10:14:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.360+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.7 v2:172.21.15.186:6808/1627307043 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb48f7c00 2024-04-25T10:14:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.360+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:7 17.8 17:10000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491ac00 con 0x555eb48f6c00 2024-04-25T10:14:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.360+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.0 v2:172.21.15.16:6816/2167750256 3 ==== osd_op_reply(7 [pgnls-filter start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb48f6c00 2024-04-25T10:14:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.360+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:8 17.28 17:14000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491ac00 con 0x555eb48f6400 2024-04-25T10:14:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.360+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.4 v2:172.21.15.186:6824/3091472889 3 ==== osd_op_reply(8 [pgnls-filter start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f6400 2024-04-25T10:14:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.360+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:9 17.18 17:18000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491b000 con 0x555eb48f7c00 2024-04-25T10:14:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.360+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.7 v2:172.21.15.186:6808/1627307043 3 ==== osd_op_reply(9 [pgnls-filter start_epoch 117 out=49b] v115'17 uv9 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb48f7c00 2024-04-25T10:14:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.361+0000 7fcd3a4c4640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x555eb48f7000 0x555eb48fcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.361+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:10 17.38 17:1c000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb48f7400 con 0x555eb48f7000 2024-04-25T10:14:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.361+0000 7fcd3eccd640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x555eb48f7000 0x555eb48fcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.361+0000 7fcd3eccd640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x555eb48f7000 0x555eb48fcb00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.361+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.3 v2:172.21.15.16:6824/256577261 1 ==== osd_op_reply(10 [pgnls-filter start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f7000 2024-04-25T10:14:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.361+0000 7fcd3acc5640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x555eb491a800 0x555eb48fd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.361+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:11 17.4 17:20000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb48f6800 con 0x555eb491a800 2024-04-25T10:14:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.362+0000 7fcd3fccf640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x555eb491a800 0x555eb48fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.362+0000 7fcd3fccf640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x555eb491a800 0x555eb48fd600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.362+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.6 v2:172.21.15.186:6816/1123185664 1 ==== osd_op_reply(11 [pgnls-filter start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb491a800 2024-04-25T10:14:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.362+0000 7fcd3a4c4640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x555eb491ac00 0x555eb48fd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.362+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:12 17.24 17:24000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491b400 con 0x555eb491ac00 2024-04-25T10:14:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.362+0000 7fcd3f4ce640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x555eb491ac00 0x555eb48fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.363+0000 7fcd3f4ce640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x555eb491ac00 0x555eb48fd080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.363+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.5 v2:172.21.15.186:6800/1827108421 1 ==== osd_op_reply(12 [pgnls-filter start_epoch 117 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb491ac00 2024-04-25T10:14:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.363+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:13 17.14 17:28000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb544fc00 con 0x555eb491ac00 2024-04-25T10:14:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.363+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.5 v2:172.21.15.186:6800/1827108421 2 ==== osd_op_reply(13 [pgnls-filter start_epoch 117 out=49b] v115'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb491ac00 2024-04-25T10:14:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.363+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:14 17.34 17:2c000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491b800 con 0x555eb48f7c00 2024-04-25T10:14:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.364+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.7 v2:172.21.15.186:6808/1627307043 4 ==== osd_op_reply(14 [pgnls-filter start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb48f7c00 2024-04-25T10:14:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.364+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:15 17.c 17:30000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491a000 con 0x555eb48f7000 2024-04-25T10:14:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.364+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.3 v2:172.21.15.16:6824/256577261 2 ==== osd_op_reply(15 [pgnls-filter start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f7000 2024-04-25T10:14:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.364+0000 7fcd3a4c4640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x555eb491bc00 0x555eb48fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.364+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:16 17.2c 17:34000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491b400 con 0x555eb491bc00 2024-04-25T10:14:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.364+0000 7fcd3eccd640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x555eb491bc00 0x555eb48fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.364+0000 7fcd3eccd640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x555eb491bc00 0x555eb48fdb80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.365+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.1 v2:172.21.15.16:6808/1916704159 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb491bc00 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.365+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:17 17.1c 17:38000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491a400 con 0x555eb491ac00 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.365+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.5 v2:172.21.15.186:6800/1827108421 3 ==== osd_op_reply(17 [pgnls-filter start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb491ac00 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.365+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:18 17.3c 17:3c000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4932000 con 0x555eb48f6400 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.365+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.4 v2:172.21.15.186:6824/3091472889 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f6400 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.365+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:19 17.2 17:40000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491b000 con 0x555eb491bc00 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.366+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.1 v2:172.21.15.16:6808/1916704159 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb491bc00 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.366+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:20 17.22 17:44000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4932400 con 0x555eb48f7000 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.366+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.3 v2:172.21.15.16:6824/256577261 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f7000 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.366+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:21 17.12 17:48000000::::head [pgnls-filter start_epoch 117 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb5361800 con 0x555eb48f7c00 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.366+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.7 v2:172.21.15.186:6808/1627307043 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 117 out=81b] v117'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555eb4672780 con 0x555eb48f7c00 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.366+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:22 17.0 17:00000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb48f7800 con 0x555eb48f6c00 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.367+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.0 v2:172.21.15.16:6816/2167750256 4 ==== osd_op_reply(22 [pgnls-filter start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb48f6c00 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.367+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:23 17.20 17:04000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4932000 con 0x555eb48f7c00 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.367+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.7 v2:172.21.15.186:6808/1627307043 6 ==== osd_op_reply(23 [pgnls-filter start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb48f7c00 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.367+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:24 17.10 17:08000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4932800 con 0x555eb48f6400 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.367+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.4 v2:172.21.15.186:6824/3091472889 5 ==== osd_op_reply(24 [pgnls-filter start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f6400 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.367+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:25 17.30 17:0c000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491a000 con 0x555eb48f7c00 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.368+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.7 v2:172.21.15.186:6808/1627307043 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb48f7c00 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.368+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:26 17.8 17:10000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4932c00 con 0x555eb48f6c00 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.368+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.0 v2:172.21.15.16:6816/2167750256 5 ==== osd_op_reply(26 [pgnls-filter start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb48f6c00 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.368+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:27 17.28 17:14000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491b400 con 0x555eb48f6400 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.368+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.4 v2:172.21.15.186:6824/3091472889 6 ==== osd_op_reply(27 [pgnls-filter start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f6400 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.368+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:28 17.18 17:18000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4933000 con 0x555eb48f7c00 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.369+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.7 v2:172.21.15.186:6808/1627307043 8 ==== osd_op_reply(28 [pgnls-filter start_epoch 117 out=49b] v115'17 uv9 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb48f7c00 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.369+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:29 17.38 17:1c000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491b000 con 0x555eb48f7000 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.369+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.3 v2:172.21.15.16:6824/256577261 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f7000 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.369+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:30 17.4 17:20000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4933400 con 0x555eb491a800 2024-04-25T10:14:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.369+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.6 v2:172.21.15.186:6816/1123185664 2 ==== osd_op_reply(30 [pgnls-filter start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb491a800 2024-04-25T10:14:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.369+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:31 17.24 17:24000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb48f7400 con 0x555eb491ac00 2024-04-25T10:14:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.369+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.5 v2:172.21.15.186:6800/1827108421 4 ==== osd_op_reply(31 [pgnls-filter start_epoch 117 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb491ac00 2024-04-25T10:14:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.370+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:32 17.14 17:28000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491b800 con 0x555eb491ac00 2024-04-25T10:14:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.370+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.5 v2:172.21.15.186:6800/1827108421 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 117 out=49b] v115'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb491ac00 2024-04-25T10:14:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.370+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:33 17.34 17:2c000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4932c00 con 0x555eb48f7c00 2024-04-25T10:14:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.370+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.7 v2:172.21.15.186:6808/1627307043 9 ==== osd_op_reply(33 [pgnls-filter start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb48f7c00 2024-04-25T10:14:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.370+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:34 17.c 17:30000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491a400 con 0x555eb48f7000 2024-04-25T10:14:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.370+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.3 v2:172.21.15.16:6824/256577261 5 ==== osd_op_reply(34 [pgnls-filter start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f7000 2024-04-25T10:14:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.370+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:35 17.2c 17:34000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4933400 con 0x555eb491bc00 2024-04-25T10:14:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.371+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.1 v2:172.21.15.16:6808/1916704159 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb491bc00 2024-04-25T10:14:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.371+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:36 17.1c 17:38000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491a000 con 0x555eb491ac00 2024-04-25T10:14:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.371+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.5 v2:172.21.15.186:6800/1827108421 6 ==== osd_op_reply(36 [pgnls-filter start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb491ac00 2024-04-25T10:14:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.371+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:37 17.3c 17:3c000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4932000 con 0x555eb48f6400 2024-04-25T10:14:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.371+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.4 v2:172.21.15.186:6824/3091472889 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f6400 2024-04-25T10:14:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.371+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:38 17.2 17:40000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb5361800 con 0x555eb491bc00 2024-04-25T10:14:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.372+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.1 v2:172.21.15.16:6808/1916704159 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb491bc00 2024-04-25T10:14:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.372+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:39 17.22 17:44000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4933800 con 0x555eb48f7000 2024-04-25T10:14:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.372+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.3 v2:172.21.15.16:6824/256577261 6 ==== osd_op_reply(39 [pgnls-filter start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f7000 2024-04-25T10:14:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.372+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:40 17.12 17:48000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb544fc00 con 0x555eb48f7c00 2024-04-25T10:14:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.372+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.7 v2:172.21.15.186:6808/1627307043 10 ==== osd_op_reply(40 [pgnls-filter start_epoch 117 out=49b] v117'23 uv23 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb48f7c00 2024-04-25T10:14:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.372+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:41 17.32 17:4c000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4933c00 con 0x555eb48f6c00 2024-04-25T10:14:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.373+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.0 v2:172.21.15.16:6816/2167750256 6 ==== osd_op_reply(41 [pgnls-filter start_epoch 117 out=81b] v117'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555eb4672500 con 0x555eb48f6c00 2024-04-25T10:14:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.373+0000 7fcd41828740 20 datascan.forall_objects: OSD matched oid 10000000003.00000000 2024-04-25T10:14:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.373+0000 7fcd41828740 10 datascan.operator(): handling object 10000000003.0 2024-04-25T10:14:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.373+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:42 17.32 17:4def72d4:::10000000003.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4932800 con 0x555eb48f6c00 2024-04-25T10:14:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.373+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.0 v2:172.21.15.16:6816/2167750256 7 ==== osd_op_reply(42 10000000003.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+194 (crc 0 0 0) 0x555eb4672500 con 0x555eb48f6c00 2024-04-25T10:14:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.373+0000 7fcd41828740 10 datascan.inject_with_backtrace: inode: 0x0x10000000003 2024-04-25T10:14:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.373+0000 7fcd41828740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-04-25T10:14:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.373+0000 7fcd41828740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-04-25T10:14:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.373+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:43 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4933400 con 0x555eb48f6400 2024-04-25T10:14:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.373+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.4 v2:172.21.15.186:6824/3091472889 8 ==== osd_op_reply(43 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f6400 2024-04-25T10:14:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.373+0000 7fcd41828740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-04-25T10:14:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.373+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:44 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491b400 con 0x555eb491a800 2024-04-25T10:14:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.374+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.6 v2:172.21.15.186:6816/1123185664 3 ==== osd_op_reply(44 10000000000.00000000 [omap-get-vals-by-keys out=585b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+585 (crc 0 0 0) 0x555eb4672500 con 0x555eb491a800 2024-04-25T10:14:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.374+0000 7fcd41828740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-04-25T10:14:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.374+0000 7fcd41828740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-04-25T10:14:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.374+0000 7fcd41828740 20 datascan.get_frag_of: frag is * 2024-04-25T10:14:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.374+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:45 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491b000 con 0x555eb48f6400 2024-04-25T10:14:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.374+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.4 v2:172.21.15.186:6824/3091472889 9 ==== osd_op_reply(45 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f6400 2024-04-25T10:14:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.374+0000 7fcd41828740 10 datascan.find_or_create_dirfrag: read_version = 8 2024-04-25T10:14:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.374+0000 7fcd41828740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-04-25T10:14:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.374+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:46 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491a400 con 0x555eb48f6400 2024-04-25T10:14:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.374+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.4 v2:172.21.15.186:6824/3091472889 10 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f6400 2024-04-25T10:14:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.374+0000 7fcd41828740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-25T10:14:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.374+0000 7fcd41828740 20 datascan.read_dentry: bravo_head not found in result 2024-04-25T10:14:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.374+0000 7fcd41828740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000003 at 0x0x10000000001/bravo with size=5 bytes 2024-04-25T10:14:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.374+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:47 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb493c000 con 0x555eb48f6400 2024-04-25T10:14:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.376+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.4 v2:172.21.15.186:6824/3091472889 11 ==== osd_op_reply(47 10000000001.00000000 [omap-set-vals] v117'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f6400 2024-04-25T10:14:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.376+0000 7fcd41828740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x10000000003 2024-04-25T10:14:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.376+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:48 17.a 17:50000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb493c400 con 0x555eb491bc00 2024-04-25T10:14:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.376+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.1 v2:172.21.15.16:6808/1916704159 5 ==== osd_op_reply(48 [pgnls-filter start_epoch 117 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb491bc00 2024-04-25T10:14:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.376+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:49 17.2a 17:54000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb493c000 con 0x555eb491ac00 2024-04-25T10:14:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.376+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.5 v2:172.21.15.186:6800/1827108421 7 ==== osd_op_reply(49 [pgnls-filter start_epoch 117 out=49b] v115'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb491ac00 2024-04-25T10:14:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.376+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:50 17.1a 17:58000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb48f6800 con 0x555eb48f6400 2024-04-25T10:14:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.377+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.4 v2:172.21.15.186:6824/3091472889 12 ==== osd_op_reply(50 [pgnls-filter start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f6400 2024-04-25T10:14:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.377+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:51 17.3a 17:5c000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491b000 con 0x555eb491ac00 2024-04-25T10:14:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.377+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.5 v2:172.21.15.186:6800/1827108421 8 ==== osd_op_reply(51 [pgnls-filter start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb491ac00 2024-04-25T10:14:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.377+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:52 17.6 17:60000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb48f7800 con 0x555eb48f6c00 2024-04-25T10:14:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.378+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.0 v2:172.21.15.16:6816/2167750256 8 ==== osd_op_reply(52 [pgnls-filter start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb48f6c00 2024-04-25T10:14:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.378+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:53 17.26 17:64000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491a400 con 0x555eb491ac00 2024-04-25T10:14:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.378+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.5 v2:172.21.15.186:6800/1827108421 9 ==== osd_op_reply(53 [pgnls-filter start_epoch 117 out=49b] v115'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb491ac00 2024-04-25T10:14:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.378+0000 7fcd3a4c4640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x555eb491b400 0x555eb48fe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.378+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:54 17.16 17:68000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4933c00 con 0x555eb491b400 2024-04-25T10:14:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.378+0000 7fcd3fccf640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x555eb491b400 0x555eb48fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.378+0000 7fcd3fccf640 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x555eb491b400 0x555eb48fe100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.379+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.2 v2:172.21.15.16:6800/2094356376 1 ==== osd_op_reply(54 [pgnls-filter start_epoch 117 out=49b] v115'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb491b400 2024-04-25T10:14:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.379+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:55 17.36 17:6c000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4932000 con 0x555eb48f7c00 2024-04-25T10:14:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.379+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.7 v2:172.21.15.186:6808/1627307043 11 ==== osd_op_reply(55 [pgnls-filter start_epoch 117 out=49b] v115'32 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb48f7c00 2024-04-25T10:14:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.379+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:56 17.e 17:70000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4932800 con 0x555eb491a800 2024-04-25T10:14:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.379+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.6 v2:172.21.15.186:6816/1123185664 4 ==== osd_op_reply(56 [pgnls-filter start_epoch 117 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb491a800 2024-04-25T10:14:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.379+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:57 17.2e 17:74000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4933800 con 0x555eb48f7000 2024-04-25T10:14:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.380+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.3 v2:172.21.15.16:6824/256577261 7 ==== osd_op_reply(57 [pgnls-filter start_epoch 117 out=49b] v115'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f7000 2024-04-25T10:14:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.380+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:58 17.1e 17:78000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4932400 con 0x555eb491a800 2024-04-25T10:14:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.380+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.6 v2:172.21.15.186:6816/1123185664 5 ==== osd_op_reply(58 [pgnls-filter start_epoch 117 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb491a800 2024-04-25T10:14:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.380+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:59 17.3e 17:7c000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4933400 con 0x555eb491b400 2024-04-25T10:14:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.380+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.2 v2:172.21.15.16:6800/2094356376 2 ==== osd_op_reply(59 [pgnls-filter start_epoch 117 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb491b400 2024-04-25T10:14:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.380+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:60 17.1 17:80000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4933000 con 0x555eb491ac00 2024-04-25T10:14:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.381+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.5 v2:172.21.15.186:6800/1827108421 10 ==== osd_op_reply(60 [pgnls-filter start_epoch 117 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb491ac00 2024-04-25T10:14:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.381+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:61 17.21 17:84000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb5361800 con 0x555eb491a800 2024-04-25T10:14:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.381+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.6 v2:172.21.15.186:6816/1123185664 6 ==== osd_op_reply(61 [pgnls-filter start_epoch 117 out=49b] v115'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb491a800 2024-04-25T10:14:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.381+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:62 17.11 17:88000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4932c00 con 0x555eb48f6400 2024-04-25T10:14:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.381+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.4 v2:172.21.15.186:6824/3091472889 13 ==== osd_op_reply(62 [pgnls-filter start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f6400 2024-04-25T10:14:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.381+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:63 17.31 17:8c000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb493c000 con 0x555eb491a800 2024-04-25T10:14:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.381+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.6 v2:172.21.15.186:6816/1123185664 7 ==== osd_op_reply(63 [pgnls-filter start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb491a800 2024-04-25T10:14:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.381+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:64 17.9 17:90000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb48f7800 con 0x555eb48f7c00 2024-04-25T10:14:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.382+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.7 v2:172.21.15.186:6808/1627307043 12 ==== osd_op_reply(64 [pgnls-filter start_epoch 117 out=49b] v115'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb48f7c00 2024-04-25T10:14:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.382+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:65 17.29 17:94000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb544fc00 con 0x555eb48f6c00 2024-04-25T10:14:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.382+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.0 v2:172.21.15.16:6816/2167750256 9 ==== osd_op_reply(65 [pgnls-filter start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb48f6c00 2024-04-25T10:14:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.382+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:66 17.19 17:98000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb48f7400 con 0x555eb48f7000 2024-04-25T10:14:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.382+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.3 v2:172.21.15.16:6824/256577261 8 ==== osd_op_reply(66 [pgnls-filter start_epoch 117 out=49b] v115'21 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f7000 2024-04-25T10:14:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.382+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:67 17.39 17:9c000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491b800 con 0x555eb48f7000 2024-04-25T10:14:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.383+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.3 v2:172.21.15.16:6824/256577261 9 ==== osd_op_reply(67 [pgnls-filter start_epoch 117 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f7000 2024-04-25T10:14:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.383+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb493c000 con 0x555eb48f6c00 2024-04-25T10:14:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.383+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.0 v2:172.21.15.16:6816/2167750256 10 ==== osd_op_reply(68 [pgnls-filter start_epoch 117 out=49b] v117'12 uv12 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb48f6c00 2024-04-25T10:14:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.383+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:69 17.25 17:a4000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491a000 con 0x555eb48f6400 2024-04-25T10:14:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.383+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.4 v2:172.21.15.186:6824/3091472889 14 ==== osd_op_reply(69 [pgnls-filter start_epoch 117 out=49b] v117'17 uv17 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f6400 2024-04-25T10:14:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.383+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:70 17.15 17:a8000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb5361800 con 0x555eb48f7000 2024-04-25T10:14:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.384+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.3 v2:172.21.15.16:6824/256577261 10 ==== osd_op_reply(70 [pgnls-filter start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f7000 2024-04-25T10:14:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.384+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:71 17.35 17:ac000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb5361800 con 0x555eb48f7000 2024-04-25T10:14:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.384+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.3 v2:172.21.15.16:6824/256577261 11 ==== osd_op_reply(71 [pgnls-filter start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f7000 2024-04-25T10:14:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.384+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:72 17.d 17:b0000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4933c00 con 0x555eb48f6c00 2024-04-25T10:14:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.384+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.0 v2:172.21.15.16:6816/2167750256 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb48f6c00 2024-04-25T10:14:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.384+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:73 17.2d 17:b4000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491b800 con 0x555eb491bc00 2024-04-25T10:14:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.384+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.1 v2:172.21.15.16:6808/1916704159 6 ==== osd_op_reply(73 [pgnls-filter start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb491bc00 2024-04-25T10:14:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.384+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:74 17.1d 17:b8000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4932800 con 0x555eb491bc00 2024-04-25T10:14:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.385+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.1 v2:172.21.15.16:6808/1916704159 7 ==== osd_op_reply(74 [pgnls-filter start_epoch 117 out=49b] v115'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb491bc00 2024-04-25T10:14:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.385+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:75 17.3d 17:bc000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb491a000 con 0x555eb491bc00 2024-04-25T10:14:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.385+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.1 v2:172.21.15.16:6808/1916704159 8 ==== osd_op_reply(75 [pgnls-filter start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb491bc00 2024-04-25T10:14:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.385+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:76 17.3 17:c0000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4932400 con 0x555eb48f6c00 2024-04-25T10:14:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.385+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.0 v2:172.21.15.16:6816/2167750256 12 ==== osd_op_reply(76 [pgnls-filter start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb48f6c00 2024-04-25T10:14:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.385+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:77 17.23 17:c4000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb48f6800 con 0x555eb48f6c00 2024-04-25T10:14:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.385+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.0 v2:172.21.15.16:6816/2167750256 13 ==== osd_op_reply(77 [pgnls-filter start_epoch 117 out=49b] v115'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb48f6c00 2024-04-25T10:14:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.385+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:78 17.13 17:c8000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4933400 con 0x555eb48f6400 2024-04-25T10:14:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.386+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.4 v2:172.21.15.186:6824/3091472889 15 ==== osd_op_reply(78 [pgnls-filter start_epoch 117 out=49b] v115'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f6400 2024-04-25T10:14:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.386+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:79 17.33 17:cc000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb48f7400 con 0x555eb491b400 2024-04-25T10:14:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.386+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.2 v2:172.21.15.16:6800/2094356376 3 ==== osd_op_reply(79 [pgnls-filter start_epoch 117 out=49b] v115'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb491b400 2024-04-25T10:14:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.386+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:80 17.b 17:d0000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4944000 con 0x555eb491a800 2024-04-25T10:14:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.386+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.6 v2:172.21.15.186:6816/1123185664 8 ==== osd_op_reply(80 [pgnls-filter start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb491a800 2024-04-25T10:14:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.387+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:81 17.2b 17:d4000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4933800 con 0x555eb48f7000 2024-04-25T10:14:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.387+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.3 v2:172.21.15.16:6824/256577261 12 ==== osd_op_reply(81 [pgnls-filter start_epoch 117 out=49b] v115'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f7000 2024-04-25T10:14:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.387+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:82 17.1b 17:d8000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4944400 con 0x555eb48f7000 2024-04-25T10:14:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.387+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.3 v2:172.21.15.16:6824/256577261 13 ==== osd_op_reply(82 [pgnls-filter start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f7000 2024-04-25T10:14:50.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.387+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:83 17.3b 17:dc000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4932c00 con 0x555eb48f7c00 2024-04-25T10:14:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.387+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.7 v2:172.21.15.186:6808/1627307043 13 ==== osd_op_reply(83 [pgnls-filter start_epoch 117 out=49b] v115'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb48f7c00 2024-04-25T10:14:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.387+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:84 17.7 17:e0000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4944800 con 0x555eb491ac00 2024-04-25T10:14:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.388+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 <== osd.5 v2:172.21.15.186:6800/1827108421 11 ==== osd_op_reply(84 [pgnls-filter start_epoch 117 out=49b] v115'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672780 con 0x555eb491ac00 2024-04-25T10:14:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.388+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:85 17.27 17:e4000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb493c400 con 0x555eb48f7000 2024-04-25T10:14:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.388+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.3 v2:172.21.15.16:6824/256577261 14 ==== osd_op_reply(85 [pgnls-filter start_epoch 117 out=49b] v115'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb48f7000 2024-04-25T10:14:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.388+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:86 17.17 17:e8000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4944c00 con 0x555eb491b400 2024-04-25T10:14:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.388+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.2 v2:172.21.15.16:6800/2094356376 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb491b400 2024-04-25T10:14:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.388+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:87 17.37 17:ec000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4944000 con 0x555eb491a800 2024-04-25T10:14:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.389+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.6 v2:172.21.15.186:6816/1123185664 9 ==== osd_op_reply(87 [pgnls-filter start_epoch 117 out=49b] v115'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb491a800 2024-04-25T10:14:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.389+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:88 17.f 17:f0000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4945000 con 0x555eb491a800 2024-04-25T10:14:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.389+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.6 v2:172.21.15.186:6816/1123185664 10 ==== osd_op_reply(88 [pgnls-filter start_epoch 117 out=49b] v115'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb491a800 2024-04-25T10:14:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.389+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:89 17.2f 17:f4000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb48f6800 con 0x555eb491bc00 2024-04-25T10:14:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.389+0000 7fcd3eccd640 1 -- 172.21.15.16:0/2194945898 <== osd.1 v2:172.21.15.16:6808/1916704159 9 ==== osd_op_reply(89 [pgnls-filter start_epoch 117 out=49b] v115'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672000 con 0x555eb491bc00 2024-04-25T10:14:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.389+0000 7fcd3a4c4640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:90 17.1f 17:f8000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb4945400 con 0x555eb48f6c00 2024-04-25T10:14:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.389+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.0 v2:172.21.15.16:6816/2167750256 14 ==== osd_op_reply(90 [pgnls-filter start_epoch 117 out=49b] v115'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb48f6c00 2024-04-25T10:14:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.389+0000 7fcd3acc5640 1 -- 172.21.15.16:0/2194945898 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:91 17.3f 17:fc000000::::head [pgnls-filter start_epoch 117 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x555eb48f7400 con 0x555eb491a800 2024-04-25T10:14:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd3fccf640 1 -- 172.21.15.16:0/2194945898 <== osd.6 v2:172.21.15.186:6816/1123185664 11 ==== osd_op_reply(91 [pgnls-filter start_epoch 117 out=49b] v115'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555eb4672500 con 0x555eb491a800 2024-04-25T10:14:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x555eb48f6c00 msgr2=0x555eb45f6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x555eb48f6c00 0x555eb45f6100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x555eb491bc00 msgr2=0x555eb48fdb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x555eb491bc00 0x555eb48fdb80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x555eb491b400 msgr2=0x555eb48fe100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x555eb491b400 0x555eb48fe100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x555eb48f7000 msgr2=0x555eb48fcb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x555eb48f7000 0x555eb48fcb00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 reap_dead start 2024-04-25T10:14:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x555eb48f6400 msgr2=0x555eb45f6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x555eb48f6400 0x555eb45f6680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x555eb491ac00 msgr2=0x555eb48fd080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 reap_dead start 2024-04-25T10:14:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x555eb491ac00 0x555eb48fd080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x555eb491a800 msgr2=0x555eb48fd600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x555eb491a800 0x555eb48fd600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x555eb48f7c00 msgr2=0x555eb48fc580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x555eb48f7c00 0x555eb48fc580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x555eb48f6000 msgr2=0x555eb48fc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x555eb48f6000 0x555eb48fc000 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x555eb45c5470 tx=0x555eb54d05a0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb544ec00 msgr2=0x555eb45f7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 --2- 172.21.15.16:0/2194945898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb544ec00 0x555eb45f7180 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x555eb5477950 tx=0x555eb5309e00 comp rx=0 tx=0).stop 2024-04-25T10:14:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd3f4ce640 1 -- 172.21.15.16:0/2194945898 reap_dead start 2024-04-25T10:14:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 shutdown_connections 2024-04-25T10:14:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 >> 172.21.15.16:0/2194945898 conn(0x555eb544e400 msgr2=0x555eb466d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 shutdown_connections 2024-04-25T10:14:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2194945898 wait complete. 2024-04-25T10:14:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2611541338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5361c00 msgr2=0x555eb45f4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 --2- 172.21.15.16:0/2611541338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5361c00 0x555eb45f4b00 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x555eb53a77a0 tx=0x555eb5468000 comp rx=0 tx=0).stop 2024-04-25T10:14:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2611541338 shutdown_connections 2024-04-25T10:14:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 --2- 172.21.15.16:0/2611541338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555eb544e000 0x555eb45f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 --2- 172.21.15.16:0/2611541338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5361c00 0x555eb45f4b00 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 --2- 172.21.15.16:0/2611541338 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555eb5360000 0x555eb45f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2611541338 >> 172.21.15.16:0/2611541338 conn(0x555eb4585000 msgr2=0x555eb466c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2611541338 shutdown_connections 2024-04-25T10:14:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.390+0000 7fcd41828740 1 -- 172.21.15.16:0/2611541338 wait complete. 2024-04-25T10:14:50.415 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.137635, result: 2024-04-25T10:14:50.416 DEBUG:teuthology.orchestra.run.smithi016:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-25T10:14:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.490+0000 7f6beb922740 1 -- start start 2024-04-25T10:14:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.490+0000 7f6beb922740 1 Processor -- start 2024-04-25T10:14:50.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.490+0000 7f6beb922740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c8182c6c00 0x55c818336580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.490+0000 7f6beb922740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8182c7800 0x55c818336b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.490+0000 7f6beb922740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8182c7c00 0x55c818337080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.490+0000 7f6beb922740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8183a2180 con 0x55c8182c7800 2024-04-25T10:14:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.490+0000 7f6beb922740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8183a2300 con 0x55c8182c7c00 2024-04-25T10:14:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.490+0000 7f6beb922740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8183a2480 con 0x55c8182c6c00 2024-04-25T10:14:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.491+0000 7f6be9014640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8182c7800 0x55c818336b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.491+0000 7f6be9014640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8182c7800 0x55c818336b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58958/0 (socket says 172.21.15.16:58958) 2024-04-25T10:14:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.491+0000 7f6be9014640 1 -- 172.21.15.16:0/841742349 learned_addr learned my addr 172.21.15.16:0/841742349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.491+0000 7f6be9815640 1 --2- 172.21.15.16:0/841742349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c8182c6c00 0x55c818336580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.491+0000 7f6bea016640 1 --2- 172.21.15.16:0/841742349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8182c7c00 0x55c818337080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.491+0000 7f6be9014640 1 -- 172.21.15.16:0/841742349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c8182c6c00 msgr2=0x55c818336580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.491+0000 7f6be9014640 1 --2- 172.21.15.16:0/841742349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c8182c6c00 0x55c818336580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.491+0000 7f6be9014640 1 -- 172.21.15.16:0/841742349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8182c7c00 msgr2=0x55c818337080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.491+0000 7f6be9014640 1 --2- 172.21.15.16:0/841742349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8182c7c00 0x55c818337080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.491+0000 7f6be9014640 1 -- 172.21.15.16:0/841742349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8182f3680 con 0x55c8182c7800 2024-04-25T10:14:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.491+0000 7f6be9014640 1 --2- 172.21.15.16:0/841742349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8182c7800 0x55c818336b00 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x55c8190dbcb0 tx=0x55c8191924e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af8f0efcf41e9b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.491+0000 7f6be7010640 1 -- 172.21.15.16:0/841742349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c8182ec680 con 0x55c8182c7800 2024-04-25T10:14:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.491+0000 7f6be7010640 1 -- 172.21.15.16:0/841742349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c8183101c0 con 0x55c8182c7800 2024-04-25T10:14:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.491+0000 7f6be7010640 1 -- 172.21.15.16:0/841742349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c8182ecb60 con 0x55c8182c7800 2024-04-25T10:14:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.491+0000 7f6beb922740 1 -- 172.21.15.16:0/841742349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8182f3860 con 0x55c8182c7800 2024-04-25T10:14:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.491+0000 7f6beb922740 1 -- 172.21.15.16:0/841742349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55c8182f32c0 con 0x55c8182c7800 2024-04-25T10:14:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.491+0000 7f6beb922740 4 waiting for MDS map... 2024-04-25T10:14:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.491+0000 7f6be7010640 1 -- 172.21.15.16:0/841742349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== osd_map(117..117 src has 1..117) v4 ==== 5881+0+0 (secure 0 0 0) 0x55c8191a8600 con 0x55c8182c7800 2024-04-25T10:14:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.492+0000 7f6be7010640 1 -- 172.21.15.16:0/841742349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 90) v1 ==== 2723+0+0 (secure 0 0 0) 0x55c81831f180 con 0x55c8182c7800 2024-04-25T10:14:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.492+0000 7f6beb922740 4 Got MDS map 90 2024-04-25T10:14:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.492+0000 7f6beb922740 4 datascan.main: Using metadata pool output 2024-04-25T10:14:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.492+0000 7f6beb922740 4 datascan.main: connecting to RADOS... 2024-04-25T10:14:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.492+0000 7f6beb922740 1 Processor -- start 2024-04-25T10:14:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.492+0000 7f6beb922740 1 -- start start 2024-04-25T10:14:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6beb922740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c8191a5000 0x55c818337600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6beb922740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8191a5400 0x55c818337b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6beb922740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8191a5800 0x55c818338100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6beb922740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8183a2600 con 0x55c8191a5400 2024-04-25T10:14:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6beb922740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8183a2780 con 0x55c8191a5800 2024-04-25T10:14:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6beb922740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8183a2900 con 0x55c8191a5000 2024-04-25T10:14:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6be9815640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8191a5400 0x55c818337b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6be9815640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8191a5400 0x55c818337b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58966/0 (socket says 172.21.15.16:58966) 2024-04-25T10:14:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6be9815640 1 -- 172.21.15.16:0/44885010 learned_addr learned my addr 172.21.15.16:0/44885010 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6bea016640 1 --2- 172.21.15.16:0/44885010 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c8191a5000 0x55c818337600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6be9014640 1 --2- 172.21.15.16:0/44885010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8191a5800 0x55c818338100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6be9815640 1 -- 172.21.15.16:0/44885010 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c8191a5000 msgr2=0x55c818337600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6be9815640 1 --2- 172.21.15.16:0/44885010 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c8191a5000 0x55c818337600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6be9815640 1 -- 172.21.15.16:0/44885010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8191a5800 msgr2=0x55c818338100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6be9815640 1 --2- 172.21.15.16:0/44885010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8191a5800 0x55c818338100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6be9815640 1 -- 172.21.15.16:0/44885010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8182f34a0 con 0x55c8191a5400 2024-04-25T10:14:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6be9815640 1 --2- 172.21.15.16:0/44885010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8191a5400 0x55c818337b80 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x55c8190db1a0 tx=0x55c8191921e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929b5f35e2a48989 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6be500c640 1 -- 172.21.15.16:0/44885010 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c8182ec9c0 con 0x55c8191a5400 2024-04-25T10:14:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6be500c640 1 -- 172.21.15.16:0/44885010 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c818310e00 con 0x55c8191a5400 2024-04-25T10:14:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.493+0000 7f6be500c640 1 -- 172.21.15.16:0/44885010 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c8182ec680 con 0x55c8191a5400 2024-04-25T10:14:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.494+0000 7f6beb922740 1 -- 172.21.15.16:0/44885010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8191a5400 msgr2=0x55c818337b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.494+0000 7f6beb922740 1 --2- 172.21.15.16:0/44885010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8191a5400 0x55c818337b80 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x55c8190db1a0 tx=0x55c8191921e0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.494+0000 7f6beb922740 1 -- 172.21.15.16:0/44885010 shutdown_connections 2024-04-25T10:14:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.494+0000 7f6beb922740 1 --2- 172.21.15.16:0/44885010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8191a5800 0x55c818338100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.494+0000 7f6beb922740 1 --2- 172.21.15.16:0/44885010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8191a5400 0x55c818337b80 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.494+0000 7f6beb922740 1 --2- 172.21.15.16:0/44885010 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c8191a5000 0x55c818337600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.494+0000 7f6beb922740 1 -- 172.21.15.16:0/44885010 >> 172.21.15.16:0/44885010 conn(0x55c8191a4c00 msgr2=0x55c8183af000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.494+0000 7f6beb922740 1 -- 172.21.15.16:0/44885010 shutdown_connections 2024-04-25T10:14:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.494+0000 7f6beb922740 1 -- 172.21.15.16:0/44885010 wait complete. 2024-04-25T10:14:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.494+0000 7f6beb922740 1 Processor -- start 2024-04-25T10:14:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.494+0000 7f6beb922740 1 -- start start 2024-04-25T10:14:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.495+0000 7f6beb922740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c8191a5000 0x55c818339180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.495+0000 7f6beb922740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8191a5400 0x55c818338c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.495+0000 7f6beb922740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8191a5800 0x55c818338680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.495+0000 7f6beb922740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8183a3980 con 0x55c8191a5400 2024-04-25T10:14:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.495+0000 7f6beb922740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8183a3800 con 0x55c8191a5800 2024-04-25T10:14:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.495+0000 7f6beb922740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8183a3680 con 0x55c8191a5000 2024-04-25T10:14:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.495+0000 7f6be9815640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8191a5400 0x55c818338c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.495+0000 7f6be9815640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8191a5400 0x55c818338c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58972/0 (socket says 172.21.15.16:58972) 2024-04-25T10:14:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.495+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 learned_addr learned my addr 172.21.15.16:0/1856076095 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.495+0000 7f6bea016640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c8191a5000 0x55c818339180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.495+0000 7f6be9014640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8191a5800 0x55c818338680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.495+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c8191a5000 msgr2=0x55c818339180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.495+0000 7f6be9815640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c8191a5000 0x55c818339180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.495+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8191a5800 msgr2=0x55c818338680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.495+0000 7f6be9815640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8191a5800 0x55c818338680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.495+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8182f3680 con 0x55c8191a5400 2024-04-25T10:14:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.496+0000 7f6be9815640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8191a5400 0x55c818338c00 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x55c81919fa40 tx=0x55c819192180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.496+0000 7f6be3809640 1 -- 172.21.15.16:0/1856076095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c8182ecb60 con 0x55c8191a5400 2024-04-25T10:14:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.496+0000 7f6be3809640 1 -- 172.21.15.16:0/1856076095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c818310c40 con 0x55c8191a5400 2024-04-25T10:14:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.496+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c8182f3860 con 0x55c8191a5400 2024-04-25T10:14:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.496+0000 7f6be3809640 1 -- 172.21.15.16:0/1856076095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c8182ec4e0 con 0x55c8191a5400 2024-04-25T10:14:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.496+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8182f30e0 con 0x55c8191a5400 2024-04-25T10:14:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.496+0000 7f6beb922740 4 datascan.init: resolving metadata pool 16 2024-04-25T10:14:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.497+0000 7f6be3809640 1 -- 172.21.15.16:0/1856076095 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x55c819206c00 con 0x55c8191a5400 2024-04-25T10:14:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.497+0000 7f6be3809640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55c8191a5c00 0x55c818339700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.497+0000 7f6be3809640 1 -- 172.21.15.16:0/1856076095 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5881+0+0 (secure 0 0 0) 0x55c81920a600 con 0x55c8191a5400 2024-04-25T10:14:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.497+0000 7f6beb922740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:14:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.497+0000 7f6beb922740 4 datascan.main: resolving metadata pool 16 2024-04-25T10:14:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.498+0000 7f6beb922740 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55c818628000 0x55c818338100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.498+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:1 16.0 16:00000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818628400 con 0x55c818628000 2024-04-25T10:14:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.498+0000 7f6bea016640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55c8191a5c00 0x55c818339700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.498+0000 7f6be9014640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55c818628000 0x55c818338100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.498+0000 7f6bea016640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55c8191a5c00 0x55c818339700 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x55c8190dbb90 tx=0x55c8192025a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.498+0000 7f6be9014640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55c818628000 0x55c818338100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.499+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.5 v2:172.21.15.186:6800/1827108421 1 ==== osd_op_reply(1 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4000 con 0x55c818628000 2024-04-25T10:14:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.499+0000 7f6be9014640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55c818628400 0x55c818634000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.499+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:2 16.10 16:08000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4400 con 0x55c818628400 2024-04-25T10:14:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.499+0000 7f6bea016640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55c818628400 0x55c818634000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.499+0000 7f6bea016640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55c818628400 0x55c818634000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.499+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.6 v2:172.21.15.186:6816/1123185664 1 ==== osd_op_reply(2 [pgnls start_epoch 117 out=177b] v116'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55c8183b4500 con 0x55c818628400 2024-04-25T10:14:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.499+0000 7f6bea016640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x55c8191a4400 0x55c818634580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.499+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:3 16.8 16:10000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818629c00 con 0x55c8191a4400 2024-04-25T10:14:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.500+0000 7f6be9815640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x55c8191a4400 0x55c818634580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.500+0000 7f6be9815640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x55c8191a4400 0x55c818634580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.500+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 1 ==== osd_op_reply(3 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.500+0000 7f6be9815640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55c818629c00 0x55c818634b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.500+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:4 16.18 16:18000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8190a7400 con 0x55c818629c00 2024-04-25T10:14:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.500+0000 7f6be9014640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55c818629c00 0x55c818634b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.501+0000 7f6be9014640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55c818629c00 0x55c818634b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.501+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 1 ==== osd_op_reply(4 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.501+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:5 16.4 16:20000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8190a7400 con 0x55c8191a4400 2024-04-25T10:14:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.501+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 2 ==== osd_op_reply(5 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.501+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:6 16.14 16:28000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8190a7400 con 0x55c8191a4400 2024-04-25T10:14:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.502+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 3 ==== osd_op_reply(6 [pgnls start_epoch 117 out=149b] v117'41 uv41 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.502+0000 7f6be9815640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55c8190a7400 0x55c818635080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.502+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:7 16.c 16:30000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818650000 con 0x55c8190a7400 2024-04-25T10:14:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.502+0000 7f6bea016640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55c8190a7400 0x55c818635080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.502+0000 7f6bea016640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55c8190a7400 0x55c818635080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.502+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.0 v2:172.21.15.16:6816/2167750256 1 ==== osd_op_reply(7 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4500 con 0x55c8190a7400 2024-04-25T10:14:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.502+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:8 16.1c 16:38000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818650000 con 0x55c8191a4400 2024-04-25T10:14:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.503+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 4 ==== osd_op_reply(8 [pgnls start_epoch 117 out=73b] v116'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.503+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:9 16.2 16:40000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818650000 con 0x55c8190a7400 2024-04-25T10:14:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.503+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.0 v2:172.21.15.16:6816/2167750256 2 ==== osd_op_reply(9 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4500 con 0x55c8190a7400 2024-04-25T10:14:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.503+0000 7f6bea016640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55c818650000 0x55c818635b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.503+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:10 16.12 16:48000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818651000 con 0x55c818650000 2024-04-25T10:14:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.503+0000 7f6be9815640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55c818650000 0x55c818635b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.503+0000 7f6be9815640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55c818650000 0x55c818635b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.504+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.1 v2:172.21.15.16:6808/1916704159 1 ==== osd_op_reply(10 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4780 con 0x55c818650000 2024-04-25T10:14:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.504+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:11 16.a 16:50000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818651000 con 0x55c818628400 2024-04-25T10:14:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.504+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.6 v2:172.21.15.186:6816/1123185664 2 ==== osd_op_reply(11 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4500 con 0x55c818628400 2024-04-25T10:14:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.504+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:12 16.1a 16:58000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818651000 con 0x55c818628400 2024-04-25T10:14:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.504+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.6 v2:172.21.15.186:6816/1123185664 3 ==== osd_op_reply(12 [pgnls start_epoch 117 out=49b] v117'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4500 con 0x55c818628400 2024-04-25T10:14:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.505+0000 7f6bea016640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55c818651000 0x55c818636100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.505+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:13 16.6 16:60000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818651400 con 0x55c818651000 2024-04-25T10:14:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.505+0000 7f6be9014640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55c818651000 0x55c818636100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.505+0000 7f6be9014640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55c818651000 0x55c818636100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.505+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.3 v2:172.21.15.16:6824/256577261 1 ==== osd_op_reply(13 [pgnls start_epoch 117 out=77b] v116'9 uv9 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55c8183b4000 con 0x55c818651000 2024-04-25T10:14:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.505+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:14 16.16 16:68000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818651400 con 0x55c818629c00 2024-04-25T10:14:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.506+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 2 ==== osd_op_reply(14 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.506+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:15 16.e 16:70000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818651400 con 0x55c818629c00 2024-04-25T10:14:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.506+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 3 ==== osd_op_reply(15 [pgnls start_epoch 117 out=73b] v116'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.506+0000 7f6be9014640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55c818651400 0x55c818636680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.506+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:16 16.1e 16:78000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c818651400 2024-04-25T10:14:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.506+0000 7f6bea016640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55c818651400 0x55c818636680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.506+0000 7f6bea016640 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55c818651400 0x55c818636680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.506+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.2 v2:172.21.15.16:6800/2094356376 1 ==== osd_op_reply(16 [pgnls start_epoch 117 out=49b] v117'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4500 con 0x55c818651400 2024-04-25T10:14:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.506+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:17 16.1 16:80000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c818628400 2024-04-25T10:14:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.507+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.6 v2:172.21.15.186:6816/1123185664 4 ==== osd_op_reply(17 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4500 con 0x55c818628400 2024-04-25T10:14:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.507+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:18 16.11 16:88000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c818651400 2024-04-25T10:14:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.507+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.2 v2:172.21.15.16:6800/2094356376 2 ==== osd_op_reply(18 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4500 con 0x55c818651400 2024-04-25T10:14:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.507+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:19 16.9 16:90000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c8191a4400 2024-04-25T10:14:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.507+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 5 ==== osd_op_reply(19 [pgnls start_epoch 117 out=81b] v116'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.507+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:20 16.19 16:98000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c818651000 2024-04-25T10:14:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.508+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.3 v2:172.21.15.16:6824/256577261 2 ==== osd_op_reply(20 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4000 con 0x55c818651000 2024-04-25T10:14:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.508+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:21 16.5 16:a0000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c8191a4400 2024-04-25T10:14:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.508+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 6 ==== osd_op_reply(21 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.508+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:22 16.15 16:a8000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c818629c00 2024-04-25T10:14:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.508+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 4 ==== osd_op_reply(22 [pgnls start_epoch 117 out=97b] v116'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.508+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:23 16.d 16:b0000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c8190a7400 2024-04-25T10:14:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.509+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.0 v2:172.21.15.16:6816/2167750256 3 ==== osd_op_reply(23 [pgnls start_epoch 117 out=74b] v114'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55c8183b4500 con 0x55c8190a7400 2024-04-25T10:14:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.509+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:24 16.1d 16:b8000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c8191a4400 2024-04-25T10:14:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.509+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 7 ==== osd_op_reply(24 [pgnls start_epoch 117 out=73b] v116'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.509+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:25 16.3 16:c0000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c818651000 2024-04-25T10:14:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.509+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.3 v2:172.21.15.16:6824/256577261 3 ==== osd_op_reply(25 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4000 con 0x55c818651000 2024-04-25T10:14:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.509+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:26 16.13 16:c8000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c818629c00 2024-04-25T10:14:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.510+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 5 ==== osd_op_reply(26 [pgnls start_epoch 117 out=98b] v116'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.510+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:27 16.b 16:d0000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c818629c00 2024-04-25T10:14:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.510+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 6 ==== osd_op_reply(27 [pgnls start_epoch 117 out=76b] v117'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.510+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:28 16.1b 16:d8000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c818650000 2024-04-25T10:14:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.510+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.1 v2:172.21.15.16:6808/1916704159 2 ==== osd_op_reply(28 [pgnls start_epoch 117 out=49b] v117'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4780 con 0x55c818650000 2024-04-25T10:14:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.510+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:29 16.7 16:e0000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c8191a4400 2024-04-25T10:14:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.511+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 8 ==== osd_op_reply(29 [pgnls start_epoch 117 out=105b] v117'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.511+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:30 16.17 16:e8000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c8191a4400 2024-04-25T10:14:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.511+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 9 ==== osd_op_reply(30 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.511+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:31 16.f 16:f0000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c818628000 2024-04-25T10:14:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.511+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.5 v2:172.21.15.186:6800/1827108421 2 ==== osd_op_reply(31 [pgnls start_epoch 117 out=73b] v117'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c8183b4000 con 0x55c818628000 2024-04-25T10:14:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.512+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:32 16.1f 16:f8000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c8190a7400 2024-04-25T10:14:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.512+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.0 v2:172.21.15.16:6816/2167750256 4 ==== osd_op_reply(32 [pgnls start_epoch 117 out=101b] v117'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55c8183b4500 con 0x55c8190a7400 2024-04-25T10:14:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.512+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-25T10:14:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.512+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:33 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818628800 con 0x55c818628400 2024-04-25T10:14:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.512+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.6 v2:172.21.15.186:6816/1123185664 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4500 con 0x55c818628400 2024-04-25T10:14:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.512+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-25T10:14:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.512+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:34 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818628c00 con 0x55c818628400 2024-04-25T10:14:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.513+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.6 v2:172.21.15.186:6816/1123185664 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4500 con 0x55c818628400 2024-04-25T10:14:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.513+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-25T10:14:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.513+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:35 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818629000 con 0x55c818628400 2024-04-25T10:14:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.513+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.6 v2:172.21.15.186:6816/1123185664 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1169 (crc 0 0 0) 0x55c8183b4500 con 0x55c818628400 2024-04-25T10:14:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.513+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-25T10:14:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.513+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:36 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818651800 con 0x55c818628400 2024-04-25T10:14:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.513+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.6 v2:172.21.15.186:6816/1123185664 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4500 con 0x55c818628400 2024-04-25T10:14:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.513+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-25T10:14:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.513+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:37 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818651c00 con 0x55c818628400 2024-04-25T10:14:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.514+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.6 v2:172.21.15.186:6816/1123185664 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4500 con 0x55c818628400 2024-04-25T10:14:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.514+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-25T10:14:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.514+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:14:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.514+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-25T10:14:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.514+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:14:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.514+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-25T10:14:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.514+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:14:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.514+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-25T10:14:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.514+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:38 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818628800 con 0x55c8191a4400 2024-04-25T10:14:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.514+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 10 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.514+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-25T10:14:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.514+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:39 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818628c00 con 0x55c8191a4400 2024-04-25T10:14:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.515+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 11 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.515+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-25T10:14:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.515+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:14:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.515+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-25T10:14:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.515+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:40 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818629000 con 0x55c818629c00 2024-04-25T10:14:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.515+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 7 ==== osd_op_reply(40 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.515+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 10000000005.00000000 2024-04-25T10:14:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.515+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:41 16.9 16:9258b67d:::10000000005.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c8191a4400 2024-04-25T10:14:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.515+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 12 ==== osd_op_reply(41 10000000005.00000000 [omap-get-vals out=584b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+584 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.516+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-25T10:14:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.516+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:42 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c8191a4800 con 0x55c818629c00 2024-04-25T10:14:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.516+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 8 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.516+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-25T10:14:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.516+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:14:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.516+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-25T10:14:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.516+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:14:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.516+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-25T10:14:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.516+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:43 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818628800 con 0x55c8191a4400 2024-04-25T10:14:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.516+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 13 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.516+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-25T10:14:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.516+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:14:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.516+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-25T10:14:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.516+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:44 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818628c00 con 0x55c818629c00 2024-04-25T10:14:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.517+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 9 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.517+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-25T10:14:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.517+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:14:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.517+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-25T10:14:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.517+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:45 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818650c00 con 0x55c8191a4400 2024-04-25T10:14:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.517+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 14 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.517+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-04-25T10:14:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.517+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:46 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818650800 con 0x55c8191a4400 2024-04-25T10:14:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.518+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 15 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1667b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1667 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.518+0000 7f6beb922740 20 datascan.scan_links: injected ino detected 2024-04-25T10:14:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.518+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-25T10:14:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.518+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:14:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.518+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-25T10:14:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.518+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:14:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.518+0000 7f6beb922740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-25T10:14:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.518+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:47 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818650400 con 0x55c8190a7400 2024-04-25T10:14:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.518+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.0 v2:172.21.15.16:6816/2167750256 5 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+585 (crc 0 0 0) 0x55c8183b4500 con 0x55c8190a7400 2024-04-25T10:14:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.518+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:48 16.0 16:00000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818628000 2024-04-25T10:14:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.518+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.5 v2:172.21.15.186:6800/1827108421 3 ==== osd_op_reply(48 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4000 con 0x55c818628000 2024-04-25T10:14:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.518+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:49 16.10 16:08000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818628400 2024-04-25T10:14:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.519+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.6 v2:172.21.15.186:6816/1123185664 10 ==== osd_op_reply(49 [pgnls start_epoch 117 out=177b] v116'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55c8183b4500 con 0x55c818628400 2024-04-25T10:14:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.519+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:50 16.8 16:10000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c8191a4400 2024-04-25T10:14:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.519+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 16 ==== osd_op_reply(50 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.519+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:51 16.18 16:18000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818629c00 2024-04-25T10:14:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.519+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 10 ==== osd_op_reply(51 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.519+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:52 16.4 16:20000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c8191a4400 2024-04-25T10:14:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.520+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 17 ==== osd_op_reply(52 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.520+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:53 16.14 16:28000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c8191a4400 2024-04-25T10:14:50.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.520+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 18 ==== osd_op_reply(53 [pgnls start_epoch 117 out=149b] v117'41 uv41 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.520+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:54 16.c 16:30000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c8190a7400 2024-04-25T10:14:50.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.520+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.0 v2:172.21.15.16:6816/2167750256 6 ==== osd_op_reply(54 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4500 con 0x55c8190a7400 2024-04-25T10:14:50.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.520+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:55 16.1c 16:38000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c8191a4400 2024-04-25T10:14:50.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.521+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 19 ==== osd_op_reply(55 [pgnls start_epoch 117 out=73b] v116'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.521+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:56 16.2 16:40000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c8190a7400 2024-04-25T10:14:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.521+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.0 v2:172.21.15.16:6816/2167750256 7 ==== osd_op_reply(56 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4500 con 0x55c8190a7400 2024-04-25T10:14:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.521+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:57 16.12 16:48000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818650000 2024-04-25T10:14:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.521+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.1 v2:172.21.15.16:6808/1916704159 3 ==== osd_op_reply(57 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4780 con 0x55c818650000 2024-04-25T10:14:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.521+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:58 16.a 16:50000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818628400 2024-04-25T10:14:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.522+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.6 v2:172.21.15.186:6816/1123185664 11 ==== osd_op_reply(58 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4500 con 0x55c818628400 2024-04-25T10:14:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.522+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:59 16.1a 16:58000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818628400 2024-04-25T10:14:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.522+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.6 v2:172.21.15.186:6816/1123185664 12 ==== osd_op_reply(59 [pgnls start_epoch 117 out=49b] v117'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4500 con 0x55c818628400 2024-04-25T10:14:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.522+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:60 16.6 16:60000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818651000 2024-04-25T10:14:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.522+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.3 v2:172.21.15.16:6824/256577261 4 ==== osd_op_reply(60 [pgnls start_epoch 117 out=77b] v116'9 uv9 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55c8183b4000 con 0x55c818651000 2024-04-25T10:14:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.522+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:61 16.16 16:68000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818629c00 2024-04-25T10:14:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.523+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 11 ==== osd_op_reply(61 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.523+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:62 16.e 16:70000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818629c00 2024-04-25T10:14:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.523+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 12 ==== osd_op_reply(62 [pgnls start_epoch 117 out=73b] v116'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.523+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:63 16.1e 16:78000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818651400 2024-04-25T10:14:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.523+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.2 v2:172.21.15.16:6800/2094356376 3 ==== osd_op_reply(63 [pgnls start_epoch 117 out=49b] v117'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4500 con 0x55c818651400 2024-04-25T10:14:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.523+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:64 16.1 16:80000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818628400 2024-04-25T10:14:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.524+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.6 v2:172.21.15.186:6816/1123185664 13 ==== osd_op_reply(64 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4500 con 0x55c818628400 2024-04-25T10:14:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.524+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- osd_op(unknown.0.0:65 16.11 16:88000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818651400 2024-04-25T10:14:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.524+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.2 v2:172.21.15.16:6800/2094356376 4 ==== osd_op_reply(65 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4500 con 0x55c818651400 2024-04-25T10:14:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.524+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:66 16.9 16:90000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c8191a4400 2024-04-25T10:14:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.524+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 20 ==== osd_op_reply(66 [pgnls start_epoch 117 out=81b] v116'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.524+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:67 16.19 16:98000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818651000 2024-04-25T10:14:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.524+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.3 v2:172.21.15.16:6824/256577261 5 ==== osd_op_reply(67 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4000 con 0x55c818651000 2024-04-25T10:14:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.524+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:68 16.5 16:a0000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c8191a4400 2024-04-25T10:14:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.525+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 21 ==== osd_op_reply(68 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.525+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:69 16.15 16:a8000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818629c00 2024-04-25T10:14:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.525+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 13 ==== osd_op_reply(69 [pgnls start_epoch 117 out=97b] v116'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.525+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:70 16.d 16:b0000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c8190a7400 2024-04-25T10:14:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.525+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.0 v2:172.21.15.16:6816/2167750256 8 ==== osd_op_reply(70 [pgnls start_epoch 117 out=74b] v114'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55c8183b4500 con 0x55c8190a7400 2024-04-25T10:14:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.525+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:71 16.1d 16:b8000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c8191a4400 2024-04-25T10:14:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.526+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 22 ==== osd_op_reply(71 [pgnls start_epoch 117 out=73b] v116'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.526+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- osd_op(unknown.0.0:72 16.3 16:c0000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818651000 2024-04-25T10:14:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.526+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.3 v2:172.21.15.16:6824/256577261 6 ==== osd_op_reply(72 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4000 con 0x55c818651000 2024-04-25T10:14:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.526+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:73 16.13 16:c8000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818629c00 2024-04-25T10:14:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.526+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 14 ==== osd_op_reply(73 [pgnls start_epoch 117 out=98b] v116'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.526+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:74 16.b 16:d0000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818629c00 2024-04-25T10:14:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.527+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 15 ==== osd_op_reply(74 [pgnls start_epoch 117 out=76b] v117'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.527+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- osd_op(unknown.0.0:75 16.1b 16:d8000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818650000 2024-04-25T10:14:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.527+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.1 v2:172.21.15.16:6808/1916704159 4 ==== osd_op_reply(75 [pgnls start_epoch 117 out=49b] v117'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4780 con 0x55c818650000 2024-04-25T10:14:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.527+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:76 16.7 16:e0000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c8191a4400 2024-04-25T10:14:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.527+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 23 ==== osd_op_reply(76 [pgnls start_epoch 117 out=105b] v117'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.527+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:77 16.17 16:e8000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c8191a4400 2024-04-25T10:14:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.527+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 24 ==== osd_op_reply(77 [pgnls start_epoch 117 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.527+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- osd_op(unknown.0.0:78 16.f 16:f0000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818628000 2024-04-25T10:14:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.528+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.5 v2:172.21.15.186:6800/1827108421 4 ==== osd_op_reply(78 [pgnls start_epoch 117 out=73b] v117'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c8183b4000 con 0x55c818628000 2024-04-25T10:14:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.528+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:79 16.1f 16:f8000000::::head [pgnls start_epoch 117 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c8190a7400 2024-04-25T10:14:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.528+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.0 v2:172.21.15.16:6816/2167750256 9 ==== osd_op_reply(79 [pgnls start_epoch 117 out=101b] v117'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55c8183b4500 con 0x55c8190a7400 2024-04-25T10:14:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.528+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-25T10:14:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.528+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:80 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666400 con 0x55c818628400 2024-04-25T10:14:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.528+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.6 v2:172.21.15.186:6816/1123185664 14 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4500 con 0x55c818628400 2024-04-25T10:14:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.528+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-25T10:14:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.528+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:81 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666800 con 0x55c818628400 2024-04-25T10:14:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.529+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.6 v2:172.21.15.186:6816/1123185664 15 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4500 con 0x55c818628400 2024-04-25T10:14:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.529+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-25T10:14:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.529+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:82 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666c00 con 0x55c818628400 2024-04-25T10:14:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.529+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.6 v2:172.21.15.186:6816/1123185664 16 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1169 (crc 0 0 0) 0x55c8183b4500 con 0x55c818628400 2024-04-25T10:14:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.529+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-25T10:14:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.529+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:83 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818667000 con 0x55c818628400 2024-04-25T10:14:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.530+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.6 v2:172.21.15.186:6816/1123185664 17 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4500 con 0x55c818628400 2024-04-25T10:14:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.530+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-25T10:14:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.530+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- osd_op(unknown.0.0:84 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818667400 con 0x55c818628400 2024-04-25T10:14:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.530+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.6 v2:172.21.15.186:6816/1123185664 18 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4500 con 0x55c818628400 2024-04-25T10:14:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.530+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-25T10:14:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.530+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:14:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.530+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-25T10:14:50.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.530+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:14:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.530+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-25T10:14:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.530+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:14:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.530+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-25T10:14:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.530+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:85 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818667800 con 0x55c8191a4400 2024-04-25T10:14:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.531+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 25 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.531+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-25T10:14:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.531+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:86 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818667c00 con 0x55c8191a4400 2024-04-25T10:14:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.531+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 26 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.532+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-25T10:14:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.532+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:14:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.532+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-25T10:14:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.532+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:87 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666000 con 0x55c818629c00 2024-04-25T10:14:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.532+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 16 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.532+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 10000000005.00000000 2024-04-25T10:14:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.532+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:88 16.9 16:9258b67d:::10000000005.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666400 con 0x55c8191a4400 2024-04-25T10:14:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.533+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 27 ==== osd_op_reply(88 10000000005.00000000 [omap-get-vals out=584b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+584 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.533+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-25T10:14:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.533+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:89 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666800 con 0x55c818629c00 2024-04-25T10:14:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.533+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 17 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.533+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-25T10:14:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.533+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:14:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.533+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-25T10:14:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.533+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:14:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.533+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-25T10:14:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.533+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:90 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818651800 con 0x55c8191a4400 2024-04-25T10:14:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.534+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 28 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.534+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-25T10:14:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.534+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:14:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.534+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-25T10:14:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.534+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:91 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818651c00 con 0x55c818629c00 2024-04-25T10:14:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.534+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 18 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.534+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-25T10:14:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.534+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:14:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.534+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-25T10:14:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.534+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:92 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818650400 con 0x55c8191a4400 2024-04-25T10:14:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.535+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 29 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.535+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-04-25T10:14:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.535+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:93 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818667800 con 0x55c8191a4400 2024-04-25T10:14:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.535+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 30 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1667b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1667 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.535+0000 7f6beb922740 20 datascan.scan_links: injected ino detected 2024-04-25T10:14:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.535+0000 7f6beb922740 20 datascan.scan_links: adding 0x10000000003 for future processing to fix dnfirst 2024-04-25T10:14:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.535+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-25T10:14:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.535+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:14:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.535+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-25T10:14:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.535+0000 7f6beb922740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:14:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.535+0000 7f6beb922740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-25T10:14:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.535+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:94 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818667c00 con 0x55c8190a7400 2024-04-25T10:14:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.536+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.0 v2:172.21.15.16:6816/2167750256 10 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+585 (crc 0 0 0) 0x55c8183b4500 con 0x55c8190a7400 2024-04-25T10:14:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.536+0000 7f6beb922740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-04-25T10:14:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.536+0000 7f6beb922740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-25T10:14:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.536+0000 7f6beb922740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-04-25T10:14:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.536+0000 7f6beb922740 10 datascan.scan_links: processing 1 injected_inos 2024-04-25T10:14:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.536+0000 7f6beb922740 10 datascan.scan_links: handling injected_ino 0x10000000003 2024-04-25T10:14:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.536+0000 7f6beb922740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-04-25T10:14:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.536+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:95 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818666400 con 0x55c8191a4400 2024-04-25T10:14:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.536+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 31 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.536+0000 7f6beb922740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-25T10:14:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.536+0000 7f6beb922740 20 datascan.scan_links: first is now 2 2024-04-25T10:14:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.536+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- osd_op(unknown.0.0:96 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818628800 con 0x55c8191a4400 2024-04-25T10:14:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.537+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 <== osd.4 v2:172.21.15.186:6824/3091472889 32 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v117'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c8183b4780 con 0x55c8191a4400 2024-04-25T10:14:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.537+0000 7f6beb922740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x10000000003 2024-04-25T10:14:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.537+0000 7f6beb922740 10 datascan.scan_links: updating inotable 2024-04-25T10:14:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.537+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- osd_op(unknown.0.0:97 16.13 16:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818629000 con 0x55c818629c00 2024-04-25T10:14:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6be9014640 1 -- 172.21.15.16:0/1856076095 <== osd.7 v2:172.21.15.186:6808/1627307043 19 ==== osd_op_reply(97 mds0_inotable [read 0~34 out=34b] v0'0 uv4 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x55c8183b4000 con 0x55c818629c00 2024-04-25T10:14:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 10 datascan.scan_links: updating snaptable 2024-04-25T10:14:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- osd_op(unknown.0.0:98 16.d 16:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) v9 -- 0x55c818628c00 con 0x55c8190a7400 2024-04-25T10:14:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6bea016640 1 -- 172.21.15.16:0/1856076095 <== osd.0 v2:172.21.15.16:6816/2167750256 11 ==== osd_op_reply(98 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55c8183b4500 con 0x55c8190a7400 2024-04-25T10:14:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55c8190a7400 msgr2=0x55c818635080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x55c8190a7400 0x55c818635080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55c818650000 msgr2=0x55c818635b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x55c818650000 0x55c818635b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55c818651400 msgr2=0x55c818636680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x55c818651400 0x55c818636680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55c818651000 msgr2=0x55c818636100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x55c818651000 0x55c818636100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x55c8191a4400 msgr2=0x55c818634580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x55c8191a4400 0x55c818634580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55c818628000 msgr2=0x55c818338100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 reap_dead start 2024-04-25T10:14:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x55c818628000 0x55c818338100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 reap_dead start 2024-04-25T10:14:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55c818628400 msgr2=0x55c818634000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x55c818628400 0x55c818634000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55c818629c00 msgr2=0x55c818634b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x55c818629c00 0x55c818634b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55c8191a5c00 msgr2=0x55c818339700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x55c8191a5c00 0x55c818339700 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x55c8190dbb90 tx=0x55c8192025a0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8191a5400 msgr2=0x55c818338c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 --2- 172.21.15.16:0/1856076095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8191a5400 0x55c818338c00 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x55c81919fa40 tx=0x55c819192180 comp rx=0 tx=0).stop 2024-04-25T10:14:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6be9815640 1 -- 172.21.15.16:0/1856076095 reap_dead start 2024-04-25T10:14:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 shutdown_connections 2024-04-25T10:14:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.538+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 >> 172.21.15.16:0/1856076095 conn(0x55c8191a4c00 msgr2=0x55c8183af000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.539+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 shutdown_connections 2024-04-25T10:14:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.539+0000 7f6beb922740 1 -- 172.21.15.16:0/1856076095 wait complete. 2024-04-25T10:14:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.539+0000 7f6beb922740 1 -- 172.21.15.16:0/841742349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8182c7800 msgr2=0x55c818336b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.539+0000 7f6beb922740 1 --2- 172.21.15.16:0/841742349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8182c7800 0x55c818336b00 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x55c8190dbcb0 tx=0x55c8191924e0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.539+0000 7f6beb922740 1 -- 172.21.15.16:0/841742349 shutdown_connections 2024-04-25T10:14:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.539+0000 7f6beb922740 1 --2- 172.21.15.16:0/841742349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c8182c7c00 0x55c818337080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.539+0000 7f6beb922740 1 --2- 172.21.15.16:0/841742349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8182c7800 0x55c818336b00 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.539+0000 7f6beb922740 1 --2- 172.21.15.16:0/841742349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c8182c6c00 0x55c818336580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.539+0000 7f6beb922740 1 -- 172.21.15.16:0/841742349 >> 172.21.15.16:0/841742349 conn(0x55c8182c7000 msgr2=0x55c8183ae800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.539+0000 7f6beb922740 1 -- 172.21.15.16:0/841742349 shutdown_connections 2024-04-25T10:14:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.539+0000 7f6beb922740 1 -- 172.21.15.16:0/841742349 wait complete. 2024-04-25T10:14:50.562 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.146793, result: 2024-04-25T10:14:50.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:14:50.592 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:14:50.631 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-04-25T10:14:50.632 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2024-04-25T10:14:50.632 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:14:50.632 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2024-04-25T10:14:50.632 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.632 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2024-04-25T10:14:50.632 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.632 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2024-04-25T10:14:50.632 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.632 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2024-04-25T10:14:50.632 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2024-04-25T10:14:50.632 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.632 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2024-04-25T10:14:50.632 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2024-04-25T10:14:50.633 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.633 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:14:50.633 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2024-04-25T10:14:50.633 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.633 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:14:50.633 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:14:50.633 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.633 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2024-04-25T10:14:50.633 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.633 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2024-04-25T10:14:50.633 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2024-04-25T10:14:50.633 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:14:50.634 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:14:50.634 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2024-04-25T10:14:50.634 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:14:50.634 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2024-04-25T10:14:50.634 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:14:50.634 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:14:50.634 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2024-04-25T10:14:50.634 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.634 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:14:50.634 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.634 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2024-04-25T10:14:50.634 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.634 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2024-04-25T10:14:50.634 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.635 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2024-04-25T10:14:50.635 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2024-04-25T10:14:50.635 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2024-04-25T10:14:50.635 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]" 2024-04-25T10:14:50.635 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2024-04-25T10:14:50.635 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 34bc5eca-3a21-42a6-a26f-07da0905e8bc 2024-04-25T10:14:50.635 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.635 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2024-04-25T10:14:50.635 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2024-04-25T10:14:50.635 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.635 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2024-04-25T10:14:50.635 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2024-04-25T10:14:50.635 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:14:50.635 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.635 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2024-04-25T10:14:50.636 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2024-04-25T10:14:50.636 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.636 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2024-04-25T10:14:50.636 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2024-04-25T10:14:50.636 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2024-04-25T10:14:50.636 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2024-04-25T10:14:50.636 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.636 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2024-04-25T10:14:50.636 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2024-04-25T10:14:50.636 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.636 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:14:50.636 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.636 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2024-04-25T10:14:50.637 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.637 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2024-04-25T10:14:50.637 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.637 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:14:50.637 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2024-04-25T10:14:50.637 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.637 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2024-04-25T10:14:50.637 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2024-04-25T10:14:50.637 INFO:teuthology.orchestra.run.smithi016.stdout: bdev async discard = True 2024-04-25T10:14:50.637 INFO:teuthology.orchestra.run.smithi016.stdout: bdev enable discard = True 2024-04-25T10:14:50.637 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore allocator = bitmap 2024-04-25T10:14:50.637 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2024-04-25T10:14:50.637 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2024-04-25T10:14:50.637 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 1/20 2024-04-25T10:14:50.638 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 1/20 2024-04-25T10:14:50.638 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:50.638 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2024-04-25T10:14:50.638 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 4/10 2024-04-25T10:14:50.638 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:14:50.638 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2024-04-25T10:14:50.638 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:14:50.638 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:14:50.638 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2024-04-25T10:14:50.638 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-04-25T10:14:50.638 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.638 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2024-04-25T10:14:50.638 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:50.639 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2024-04-25T10:14:50.639 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-04-25T10:14:50.639 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-04-25T10:14:50.639 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:50.639 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:50.639 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2024-04-25T10:14:50.639 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.639 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2024-04-25T10:14:50.639 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:50.639 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-04-25T10:14:50.639 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2024-04-25T10:14:50.639 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-04-25T10:14:50.639 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2024-04-25T10:14:50.639 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2024-04-25T10:14:50.639 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:50.640 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2024-04-25T10:14:50.640 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2024-04-25T10:14:50.640 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:50.640 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.640 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:14:50.640 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:14:50.640 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:14:50.640 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.640 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:14:50.640 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:14:50.640 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:14:50.640 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.640 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2024-04-25T10:14:50.641 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:14:50.641 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.641 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:14:50.641 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2024-04-25T10:14:50.641 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.641 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2024-04-25T10:14:50.641 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2024-04-25T10:14:50.641 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2024-04-25T10:14:50.641 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2024-04-25T10:14:50.641 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:14:50.641 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:14:50.641 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2024-04-25T10:14:50.641 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-04-25T10:14:50.641 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:50.641 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2024-04-25T10:14:50.642 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-04-25T10:14:50.642 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-04-25T10:14:50.642 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2024-04-25T10:14:50.642 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2024-04-25T10:14:50.642 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2024-04-25T10:14:50.642 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-04-25T10:14:50.642 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-04-25T10:14:50.642 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-04-25T10:14:50.642 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-04-25T10:14:50.642 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2024-04-25T10:14:50.642 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2024-04-25T10:14:50.642 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2024-04-25T10:14:50.643 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:50.643 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2024-04-25T10:14:50.643 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = False 2024-04-25T10:14:50.643 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2024-04-25T10:14:50.643 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2024-04-25T10:14:50.643 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-04-25T10:14:50.643 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-04-25T10:14:50.643 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-04-25T10:14:50.644 INFO:teuthology.orchestra.run.smithi186.stdout:[global] 2024-04-25T10:14:50.644 INFO:teuthology.orchestra.run.smithi186.stdout: chdir = "" 2024-04-25T10:14:50.644 INFO:teuthology.orchestra.run.smithi186.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:14:50.644 INFO:teuthology.orchestra.run.smithi186.stdout: auth supported = cephx 2024-04-25T10:14:50.644 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.644 INFO:teuthology.orchestra.run.smithi186.stdout: filestore xattr use omap = true 2024-04-25T10:14:50.644 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.644 INFO:teuthology.orchestra.run.smithi186.stdout: mon clock drift allowed = 1.000 2024-04-25T10:14:50.644 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.644 INFO:teuthology.orchestra.run.smithi186.stdout: osd crush chooseleaf type = 0 2024-04-25T10:14:50.644 INFO:teuthology.orchestra.run.smithi186.stdout: auth debug = true 2024-04-25T10:14:50.644 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.644 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on old message = true 2024-04-25T10:14:50.645 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on bug = true 2024-04-25T10:14:50.645 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.645 INFO:teuthology.orchestra.run.smithi186.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:14:50.645 INFO:teuthology.orchestra.run.smithi186.stdout: mon pg warn max object skew = 0 2024-04-25T10:14:50.645 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.645 INFO:teuthology.orchestra.run.smithi186.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:14:50.645 INFO:teuthology.orchestra.run.smithi186.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:14:50.645 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.645 INFO:teuthology.orchestra.run.smithi186.stdout: osd pool default size = 2 2024-04-25T10:14:50.645 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.645 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd allow primary affinity = true 2024-04-25T10:14:50.645 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd allow pg remap = true 2024-04-25T10:14:50.645 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:14:50.645 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:14:50.646 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on no sortbitwise = false 2024-04-25T10:14:50.646 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:14:50.646 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on too few osds = false 2024-04-25T10:14:50.646 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:14:50.646 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:14:50.646 INFO:teuthology.orchestra.run.smithi186.stdout: mon_allow_pool_size_one = true 2024-04-25T10:14:50.646 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.646 INFO:teuthology.orchestra.run.smithi186.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:14:50.646 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.646 INFO:teuthology.orchestra.run.smithi186.stdout: osd default data pool replay window = 5 2024-04-25T10:14:50.646 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.646 INFO:teuthology.orchestra.run.smithi186.stdout: mon allow pool delete = true 2024-04-25T10:14:50.646 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.647 INFO:teuthology.orchestra.run.smithi186.stdout: mon cluster log file level = debug 2024-04-25T10:14:50.647 INFO:teuthology.orchestra.run.smithi186.stdout: debug asserts on shutdown = true 2024-04-25T10:14:50.647 INFO:teuthology.orchestra.run.smithi186.stdout: mon health detail to clog = false 2024-04-25T10:14:50.647 INFO:teuthology.orchestra.run.smithi186.stdout: mon host = "172.21.15.16,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]" 2024-04-25T10:14:50.647 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on skipped message = False 2024-04-25T10:14:50.647 INFO:teuthology.orchestra.run.smithi186.stdout: fsid = 34bc5eca-3a21-42a6-a26f-07da0905e8bc 2024-04-25T10:14:50.647 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.647 INFO:teuthology.orchestra.run.smithi186.stdout:[osd] 2024-04-25T10:14:50.647 INFO:teuthology.orchestra.run.smithi186.stdout: osd journal size = 100 2024-04-25T10:14:50.647 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.647 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub load threshold = 5.0 2024-04-25T10:14:50.647 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub max interval = 600 2024-04-25T10:14:50.647 INFO:teuthology.orchestra.run.smithi186.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:14:50.647 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.647 INFO:teuthology.orchestra.run.smithi186.stdout: osd recover clone overlap = true 2024-04-25T10:14:50.648 INFO:teuthology.orchestra.run.smithi186.stdout: osd recovery max chunk = 1048576 2024-04-25T10:14:50.648 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.648 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug shutdown = true 2024-04-25T10:14:50.648 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug op order = true 2024-04-25T10:14:50.648 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug verify stray on activate = true 2024-04-25T10:14:50.648 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug trim objects = true 2024-04-25T10:14:50.648 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.648 INFO:teuthology.orchestra.run.smithi186.stdout: osd open classes on start = true 2024-04-25T10:14:50.648 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug pg log writeout = true 2024-04-25T10:14:50.648 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.648 INFO:teuthology.orchestra.run.smithi186.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:14:50.648 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.648 INFO:teuthology.orchestra.run.smithi186.stdout: osd map max advance = 10 2024-04-25T10:14:50.648 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.649 INFO:teuthology.orchestra.run.smithi186.stdout: journal zero on create = true 2024-04-25T10:14:50.649 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.649 INFO:teuthology.orchestra.run.smithi186.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:14:50.649 INFO:teuthology.orchestra.run.smithi186.stdout: filestore apply finisher threads = 3 2024-04-25T10:14:50.649 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.649 INFO:teuthology.orchestra.run.smithi186.stdout: bdev debug aio = true 2024-04-25T10:14:50.649 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug misdirected ops = true 2024-04-25T10:14:50.649 INFO:teuthology.orchestra.run.smithi186.stdout: bdev async discard = True 2024-04-25T10:14:50.649 INFO:teuthology.orchestra.run.smithi186.stdout: bdev enable discard = True 2024-04-25T10:14:50.649 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore allocator = bitmap 2024-04-25T10:14:50.649 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore block size = 96636764160 2024-04-25T10:14:50.649 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore fsck on mount = True 2024-04-25T10:14:50.649 INFO:teuthology.orchestra.run.smithi186.stdout: debug bluefs = 1/20 2024-04-25T10:14:50.649 INFO:teuthology.orchestra.run.smithi186.stdout: debug bluestore = 1/20 2024-04-25T10:14:50.649 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:50.650 INFO:teuthology.orchestra.run.smithi186.stdout: debug osd = 20 2024-04-25T10:14:50.650 INFO:teuthology.orchestra.run.smithi186.stdout: debug rocksdb = 4/10 2024-04-25T10:14:50.650 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:14:50.650 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd full ratio = 0.9 2024-04-25T10:14:50.650 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:14:50.650 INFO:teuthology.orchestra.run.smithi186.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:14:50.650 INFO:teuthology.orchestra.run.smithi186.stdout: osd objectstore = bluestore 2024-04-25T10:14:50.650 INFO:teuthology.orchestra.run.smithi186.stdout: osd op complaint time = 180 2024-04-25T10:14:50.650 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.650 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr] 2024-04-25T10:14:50.650 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:50.650 INFO:teuthology.orchestra.run.smithi186.stdout: debug mgr = 20 2024-04-25T10:14:50.650 INFO:teuthology.orchestra.run.smithi186.stdout: debug mon = 20 2024-04-25T10:14:50.651 INFO:teuthology.orchestra.run.smithi186.stdout: debug auth = 20 2024-04-25T10:14:50.651 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:50.651 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:50.651 INFO:teuthology.orchestra.run.smithi186.stdout: mgr/telemetry/nag = false 2024-04-25T10:14:50.651 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.651 INFO:teuthology.orchestra.run.smithi186.stdout:[mon] 2024-04-25T10:14:50.651 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:50.651 INFO:teuthology.orchestra.run.smithi186.stdout: debug mon = 20 2024-04-25T10:14:50.651 INFO:teuthology.orchestra.run.smithi186.stdout: debug paxos = 20 2024-04-25T10:14:50.651 INFO:teuthology.orchestra.run.smithi186.stdout: debug auth = 20 2024-04-25T10:14:50.651 INFO:teuthology.orchestra.run.smithi186.stdout: mon data avail warn = 5 2024-04-25T10:14:50.651 INFO:teuthology.orchestra.run.smithi186.stdout: mon mgr mkfs grace = 240 2024-04-25T10:14:50.651 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:50.651 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd reporter subtree level = osd 2024-04-25T10:14:50.651 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd prime pg temp = true 2024-04-25T10:14:50.652 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:50.652 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.652 INFO:teuthology.orchestra.run.smithi186.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:14:50.652 INFO:teuthology.orchestra.run.smithi186.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:14:50.652 INFO:teuthology.orchestra.run.smithi186.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:14:50.652 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.652 INFO:teuthology.orchestra.run.smithi186.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:14:50.652 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:14:50.652 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:14:50.652 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.652 INFO:teuthology.orchestra.run.smithi186.stdout: # 1m isn't quite enough 2024-04-25T10:14:50.652 INFO:teuthology.orchestra.run.smithi186.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:14:50.652 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.652 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:14:50.652 INFO:teuthology.orchestra.run.smithi186.stdout: mon op complaint time = 120 2024-04-25T10:14:50.653 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.653 INFO:teuthology.orchestra.run.smithi186.stdout:[client] 2024-04-25T10:14:50.653 INFO:teuthology.orchestra.run.smithi186.stdout: rgw cache enabled = true 2024-04-25T10:14:50.653 INFO:teuthology.orchestra.run.smithi186.stdout: rgw enable ops log = true 2024-04-25T10:14:50.653 INFO:teuthology.orchestra.run.smithi186.stdout: rgw enable usage log = true 2024-04-25T10:14:50.653 INFO:teuthology.orchestra.run.smithi186.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:14:50.653 INFO:teuthology.orchestra.run.smithi186.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:14:50.653 INFO:teuthology.orchestra.run.smithi186.stdout: client mount timeout = 600 2024-04-25T10:14:50.653 INFO:teuthology.orchestra.run.smithi186.stdout: debug client = 20 2024-04-25T10:14:50.653 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:50.653 INFO:teuthology.orchestra.run.smithi186.stdout: pid file = "" 2024-04-25T10:14:50.653 INFO:teuthology.orchestra.run.smithi186.stdout: rados mon op timeout = 900 2024-04-25T10:14:50.653 INFO:teuthology.orchestra.run.smithi186.stdout: rados osd op timeout = 900 2024-04-25T10:14:50.653 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.a] 2024-04-25T10:14:50.653 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.b] 2024-04-25T10:14:50.654 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.c] 2024-04-25T10:14:50.654 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.a] 2024-04-25T10:14:50.654 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.c] 2024-04-25T10:14:50.654 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.b] 2024-04-25T10:14:50.654 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.d] 2024-04-25T10:14:50.654 INFO:teuthology.orchestra.run.smithi186.stdout:[mds] 2024-04-25T10:14:50.654 INFO:teuthology.orchestra.run.smithi186.stdout: debug mds = 20 2024-04-25T10:14:50.654 INFO:teuthology.orchestra.run.smithi186.stdout: debug mds balancer = 20 2024-04-25T10:14:50.654 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:50.654 INFO:teuthology.orchestra.run.smithi186.stdout: mds debug frag = True 2024-04-25T10:14:50.654 INFO:teuthology.orchestra.run.smithi186.stdout: mds debug scatterstat = False 2024-04-25T10:14:50.654 INFO:teuthology.orchestra.run.smithi186.stdout: mds op complaint time = 180 2024-04-25T10:14:50.654 INFO:teuthology.orchestra.run.smithi186.stdout: mds verify scatter = True 2024-04-25T10:14:50.654 INFO:teuthology.orchestra.run.smithi186.stdout: osd op complaint time = 180 2024-04-25T10:14:50.654 INFO:teuthology.orchestra.run.smithi186.stdout: rados mon op timeout = 900 2024-04-25T10:14:50.655 INFO:teuthology.orchestra.run.smithi186.stdout: rados osd op timeout = 900 2024-04-25T10:14:50.657 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:14:50.698 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:14:50.737 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-04-25T10:14:50.737 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2024-04-25T10:14:50.738 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:14:50.738 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2024-04-25T10:14:50.738 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.738 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2024-04-25T10:14:50.738 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.738 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2024-04-25T10:14:50.738 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.738 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2024-04-25T10:14:50.738 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2024-04-25T10:14:50.738 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.738 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2024-04-25T10:14:50.738 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2024-04-25T10:14:50.738 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.739 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:14:50.739 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2024-04-25T10:14:50.739 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.739 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:14:50.739 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:14:50.739 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.739 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2024-04-25T10:14:50.739 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.739 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2024-04-25T10:14:50.739 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2024-04-25T10:14:50.739 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:14:50.739 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:14:50.739 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2024-04-25T10:14:50.739 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:14:50.740 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2024-04-25T10:14:50.740 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:14:50.740 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:14:50.740 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2024-04-25T10:14:50.740 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.740 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:14:50.740 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.740 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2024-04-25T10:14:50.740 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.740 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2024-04-25T10:14:50.740 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.740 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2024-04-25T10:14:50.740 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2024-04-25T10:14:50.740 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2024-04-25T10:14:50.741 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]" 2024-04-25T10:14:50.741 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2024-04-25T10:14:50.741 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 34bc5eca-3a21-42a6-a26f-07da0905e8bc 2024-04-25T10:14:50.741 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.741 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2024-04-25T10:14:50.741 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2024-04-25T10:14:50.741 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.741 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2024-04-25T10:14:50.741 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2024-04-25T10:14:50.741 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:14:50.741 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.741 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2024-04-25T10:14:50.741 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2024-04-25T10:14:50.742 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.742 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2024-04-25T10:14:50.742 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2024-04-25T10:14:50.742 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2024-04-25T10:14:50.742 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2024-04-25T10:14:50.742 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.742 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2024-04-25T10:14:50.742 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2024-04-25T10:14:50.742 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.742 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:14:50.742 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.742 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2024-04-25T10:14:50.742 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.743 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2024-04-25T10:14:50.743 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.743 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:14:50.743 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2024-04-25T10:14:50.743 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.743 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2024-04-25T10:14:50.743 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2024-04-25T10:14:50.743 INFO:teuthology.orchestra.run.smithi016.stdout: bdev async discard = True 2024-04-25T10:14:50.743 INFO:teuthology.orchestra.run.smithi016.stdout: bdev enable discard = True 2024-04-25T10:14:50.743 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore allocator = bitmap 2024-04-25T10:14:50.743 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2024-04-25T10:14:50.743 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2024-04-25T10:14:50.743 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 1/20 2024-04-25T10:14:50.744 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 1/20 2024-04-25T10:14:50.744 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:50.744 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2024-04-25T10:14:50.744 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 4/10 2024-04-25T10:14:50.744 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:14:50.744 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2024-04-25T10:14:50.744 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:14:50.744 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:14:50.744 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2024-04-25T10:14:50.744 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-04-25T10:14:50.744 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.744 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2024-04-25T10:14:50.744 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:50.745 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2024-04-25T10:14:50.745 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-04-25T10:14:50.745 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-04-25T10:14:50.745 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:50.745 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:50.745 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2024-04-25T10:14:50.745 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.745 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2024-04-25T10:14:50.745 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:50.745 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-04-25T10:14:50.745 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2024-04-25T10:14:50.745 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-04-25T10:14:50.745 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2024-04-25T10:14:50.745 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2024-04-25T10:14:50.746 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:50.746 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2024-04-25T10:14:50.746 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2024-04-25T10:14:50.746 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:50.746 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.746 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:14:50.746 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:14:50.746 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:14:50.746 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.746 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:14:50.746 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:14:50.746 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2024-04-25T10:14:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2024-04-25T10:14:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2024-04-25T10:14:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-04-25T10:14:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2024-04-25T10:14:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2024-04-25T10:14:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2024-04-25T10:14:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2024-04-25T10:14:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-04-25T10:14:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-04-25T10:14:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-04-25T10:14:50.754 INFO:teuthology.orchestra.run.smithi186.stdout:[global] 2024-04-25T10:14:50.754 INFO:teuthology.orchestra.run.smithi186.stdout: chdir = "" 2024-04-25T10:14:50.754 INFO:teuthology.orchestra.run.smithi186.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:14:50.754 INFO:teuthology.orchestra.run.smithi186.stdout: auth supported = cephx 2024-04-25T10:14:50.754 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.754 INFO:teuthology.orchestra.run.smithi186.stdout: filestore xattr use omap = true 2024-04-25T10:14:50.754 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.754 INFO:teuthology.orchestra.run.smithi186.stdout: mon clock drift allowed = 1.000 2024-04-25T10:14:50.754 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.754 INFO:teuthology.orchestra.run.smithi186.stdout: osd crush chooseleaf type = 0 2024-04-25T10:14:50.754 INFO:teuthology.orchestra.run.smithi186.stdout: auth debug = true 2024-04-25T10:14:50.754 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.755 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on old message = true 2024-04-25T10:14:50.755 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on bug = true 2024-04-25T10:14:50.755 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.755 INFO:teuthology.orchestra.run.smithi186.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:14:50.755 INFO:teuthology.orchestra.run.smithi186.stdout: mon pg warn max object skew = 0 2024-04-25T10:14:50.755 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.755 INFO:teuthology.orchestra.run.smithi186.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:14:50.755 INFO:teuthology.orchestra.run.smithi186.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:14:50.755 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.755 INFO:teuthology.orchestra.run.smithi186.stdout: osd pool default size = 2 2024-04-25T10:14:50.755 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.755 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd allow primary affinity = true 2024-04-25T10:14:50.755 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd allow pg remap = true 2024-04-25T10:14:50.755 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:14:50.755 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:14:50.756 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on no sortbitwise = false 2024-04-25T10:14:50.756 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:14:50.756 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on too few osds = false 2024-04-25T10:14:50.756 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:14:50.756 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:14:50.756 INFO:teuthology.orchestra.run.smithi186.stdout: mon_allow_pool_size_one = true 2024-04-25T10:14:50.756 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.756 INFO:teuthology.orchestra.run.smithi186.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:14:50.756 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.756 INFO:teuthology.orchestra.run.smithi186.stdout: osd default data pool replay window = 5 2024-04-25T10:14:50.756 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.757 INFO:teuthology.orchestra.run.smithi186.stdout: mon allow pool delete = true 2024-04-25T10:14:50.757 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.757 INFO:teuthology.orchestra.run.smithi186.stdout: mon cluster log file level = debug 2024-04-25T10:14:50.757 INFO:teuthology.orchestra.run.smithi186.stdout: debug asserts on shutdown = true 2024-04-25T10:14:50.757 INFO:teuthology.orchestra.run.smithi186.stdout: mon health detail to clog = false 2024-04-25T10:14:50.757 INFO:teuthology.orchestra.run.smithi186.stdout: mon host = "172.21.15.16,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]" 2024-04-25T10:14:50.757 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on skipped message = False 2024-04-25T10:14:50.757 INFO:teuthology.orchestra.run.smithi186.stdout: fsid = 34bc5eca-3a21-42a6-a26f-07da0905e8bc 2024-04-25T10:14:50.757 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.757 INFO:teuthology.orchestra.run.smithi186.stdout:[osd] 2024-04-25T10:14:50.757 INFO:teuthology.orchestra.run.smithi186.stdout: osd journal size = 100 2024-04-25T10:14:50.757 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.757 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub load threshold = 5.0 2024-04-25T10:14:50.758 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub max interval = 600 2024-04-25T10:14:50.758 INFO:teuthology.orchestra.run.smithi186.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:14:50.758 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.758 INFO:teuthology.orchestra.run.smithi186.stdout: osd recover clone overlap = true 2024-04-25T10:14:50.758 INFO:teuthology.orchestra.run.smithi186.stdout: osd recovery max chunk = 1048576 2024-04-25T10:14:50.758 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.758 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug shutdown = true 2024-04-25T10:14:50.758 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug op order = true 2024-04-25T10:14:50.758 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug verify stray on activate = true 2024-04-25T10:14:50.758 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug trim objects = true 2024-04-25T10:14:50.758 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.758 INFO:teuthology.orchestra.run.smithi186.stdout: osd open classes on start = true 2024-04-25T10:14:50.758 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug pg log writeout = true 2024-04-25T10:14:50.758 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.759 INFO:teuthology.orchestra.run.smithi186.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:14:50.759 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.759 INFO:teuthology.orchestra.run.smithi186.stdout: osd map max advance = 10 2024-04-25T10:14:50.759 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.759 INFO:teuthology.orchestra.run.smithi186.stdout: journal zero on create = true 2024-04-25T10:14:50.759 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.759 INFO:teuthology.orchestra.run.smithi186.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:14:50.759 INFO:teuthology.orchestra.run.smithi186.stdout: filestore apply finisher threads = 3 2024-04-25T10:14:50.759 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.759 INFO:teuthology.orchestra.run.smithi186.stdout: bdev debug aio = true 2024-04-25T10:14:50.759 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug misdirected ops = true 2024-04-25T10:14:50.759 INFO:teuthology.orchestra.run.smithi186.stdout: bdev async discard = True 2024-04-25T10:14:50.759 INFO:teuthology.orchestra.run.smithi186.stdout: bdev enable discard = True 2024-04-25T10:14:50.759 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore allocator = bitmap 2024-04-25T10:14:50.759 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore block size = 96636764160 2024-04-25T10:14:50.760 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore fsck on mount = True 2024-04-25T10:14:50.760 INFO:teuthology.orchestra.run.smithi186.stdout: debug bluefs = 1/20 2024-04-25T10:14:50.760 INFO:teuthology.orchestra.run.smithi186.stdout: debug bluestore = 1/20 2024-04-25T10:14:50.760 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:50.760 INFO:teuthology.orchestra.run.smithi186.stdout: debug osd = 20 2024-04-25T10:14:50.760 INFO:teuthology.orchestra.run.smithi186.stdout: debug rocksdb = 4/10 2024-04-25T10:14:50.760 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:14:50.760 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd full ratio = 0.9 2024-04-25T10:14:50.760 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:14:50.760 INFO:teuthology.orchestra.run.smithi186.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:14:50.760 INFO:teuthology.orchestra.run.smithi186.stdout: osd objectstore = bluestore 2024-04-25T10:14:50.760 INFO:teuthology.orchestra.run.smithi186.stdout: osd op complaint time = 180 2024-04-25T10:14:50.760 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.760 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr] 2024-04-25T10:14:50.760 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:50.761 INFO:teuthology.orchestra.run.smithi186.stdout: debug mgr = 20 2024-04-25T10:14:50.761 INFO:teuthology.orchestra.run.smithi186.stdout: debug mon = 20 2024-04-25T10:14:50.761 INFO:teuthology.orchestra.run.smithi186.stdout: debug auth = 20 2024-04-25T10:14:50.761 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:50.761 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:50.761 INFO:teuthology.orchestra.run.smithi186.stdout: mgr/telemetry/nag = false 2024-04-25T10:14:50.761 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.761 INFO:teuthology.orchestra.run.smithi186.stdout:[mon] 2024-04-25T10:14:50.761 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:50.761 INFO:teuthology.orchestra.run.smithi186.stdout: debug mon = 20 2024-04-25T10:14:50.761 INFO:teuthology.orchestra.run.smithi186.stdout: debug paxos = 20 2024-04-25T10:14:50.761 INFO:teuthology.orchestra.run.smithi186.stdout: debug auth = 20 2024-04-25T10:14:50.761 INFO:teuthology.orchestra.run.smithi186.stdout: mon data avail warn = 5 2024-04-25T10:14:50.761 INFO:teuthology.orchestra.run.smithi186.stdout: mon mgr mkfs grace = 240 2024-04-25T10:14:50.761 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:50.762 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd reporter subtree level = osd 2024-04-25T10:14:50.762 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd prime pg temp = true 2024-04-25T10:14:50.762 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:50.762 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.762 INFO:teuthology.orchestra.run.smithi186.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:14:50.762 INFO:teuthology.orchestra.run.smithi186.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:14:50.762 INFO:teuthology.orchestra.run.smithi186.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:14:50.762 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.762 INFO:teuthology.orchestra.run.smithi186.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:14:50.762 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:14:50.762 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:14:50.762 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.762 INFO:teuthology.orchestra.run.smithi186.stdout: # 1m isn't quite enough 2024-04-25T10:14:50.762 INFO:teuthology.orchestra.run.smithi186.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:14:50.763 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.763 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:14:50.763 INFO:teuthology.orchestra.run.smithi186.stdout: mon op complaint time = 120 2024-04-25T10:14:50.763 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-25T10:14:50.763 INFO:teuthology.orchestra.run.smithi186.stdout:[client] 2024-04-25T10:14:50.763 INFO:teuthology.orchestra.run.smithi186.stdout: rgw cache enabled = true 2024-04-25T10:14:50.763 INFO:teuthology.orchestra.run.smithi186.stdout: rgw enable ops log = true 2024-04-25T10:14:50.763 INFO:teuthology.orchestra.run.smithi186.stdout: rgw enable usage log = true 2024-04-25T10:14:50.763 INFO:teuthology.orchestra.run.smithi186.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:14:50.763 INFO:teuthology.orchestra.run.smithi186.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:14:50.763 INFO:teuthology.orchestra.run.smithi186.stdout: client mount timeout = 600 2024-04-25T10:14:50.763 INFO:teuthology.orchestra.run.smithi186.stdout: debug client = 20 2024-04-25T10:14:50.763 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:50.763 INFO:teuthology.orchestra.run.smithi186.stdout: pid file = "" 2024-04-25T10:14:50.763 INFO:teuthology.orchestra.run.smithi186.stdout: rados mon op timeout = 900 2024-04-25T10:14:50.764 INFO:teuthology.orchestra.run.smithi186.stdout: rados osd op timeout = 900 2024-04-25T10:14:50.764 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.a] 2024-04-25T10:14:50.764 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.b] 2024-04-25T10:14:50.764 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.c] 2024-04-25T10:14:50.764 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.a] 2024-04-25T10:14:50.764 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.c] 2024-04-25T10:14:50.764 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.b] 2024-04-25T10:14:50.764 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.d] 2024-04-25T10:14:50.764 INFO:teuthology.orchestra.run.smithi186.stdout:[mds] 2024-04-25T10:14:50.764 INFO:teuthology.orchestra.run.smithi186.stdout: debug mds = 20 2024-04-25T10:14:50.764 INFO:teuthology.orchestra.run.smithi186.stdout: debug mds balancer = 20 2024-04-25T10:14:50.764 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2024-04-25T10:14:50.764 INFO:teuthology.orchestra.run.smithi186.stdout: mds debug frag = True 2024-04-25T10:14:50.764 INFO:teuthology.orchestra.run.smithi186.stdout: mds debug scatterstat = True 2024-04-25T10:14:50.765 INFO:teuthology.orchestra.run.smithi186.stdout: mds op complaint time = 180 2024-04-25T10:14:50.765 INFO:teuthology.orchestra.run.smithi186.stdout: mds verify scatter = True 2024-04-25T10:14:50.765 INFO:teuthology.orchestra.run.smithi186.stdout: osd op complaint time = 180 2024-04-25T10:14:50.765 INFO:teuthology.orchestra.run.smithi186.stdout: rados mon op timeout = 900 2024-04-25T10:14:50.765 INFO:teuthology.orchestra.run.smithi186.stdout: rados osd op timeout = 900 2024-04-25T10:14:50.765 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:14:50.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.891+0000 7fdc6e770640 1 Processor -- start 2024-04-25T10:14:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.891+0000 7fdc6e770640 1 -- start start 2024-04-25T10:14:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.892+0000 7fdc6e770640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdc680cf590 0x7fdc680cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.892+0000 7fdc6e770640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc68099b20 0x7fdc68099fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.892+0000 7fdc6e770640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc680cc820 con 0x7fdc6815bb60 2024-04-25T10:14:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.892+0000 7fdc6e770640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc680cd430 con 0x7fdc680cf590 2024-04-25T10:14:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.892+0000 7fdc6e770640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc680ccb00 con 0x7fdc68099b20 2024-04-25T10:14:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.892+0000 7fdc677fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdc680cf590 0x7fdc680cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.892+0000 7fdc677fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdc680cf590 0x7fdc680cd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58982/0 (socket says 172.21.15.16:58982) 2024-04-25T10:14:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.892+0000 7fdc677fe640 1 -- 172.21.15.16:0/3991165975 learned_addr learned my addr 172.21.15.16:0/3991165975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.893+0000 7fdc6cd0d640 1 --2- 172.21.15.16:0/3991165975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc68099b20 0x7fdc68099fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.893+0000 7fdc677fe640 1 -- 172.21.15.16:0/3991165975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc68099b20 msgr2=0x7fdc68099fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.893+0000 7fdc677fe640 1 --2- 172.21.15.16:0/3991165975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc68099b20 0x7fdc68099fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.893+0000 7fdc677fe640 1 -- 172.21.15.16:0/3991165975 >> v1:172.21.15.186:6789/0 conn(0x7fdc6815bb60 legacy=0x7fdc680d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.893+0000 7fdc677fe640 1 -- 172.21.15.16:0/3991165975 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc680ccde0 con 0x7fdc680cf590 2024-04-25T10:14:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.893+0000 7fdc677fe640 1 --2- 172.21.15.16:0/3991165975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc680cf590 0x7fdc680cd8c0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009870 tx=0x7fdc5c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3b574fbcd797e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.894+0000 7fdc66ffd640 1 -- 172.21.15.16:0/3991165975 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c037070 con 0x7fdc680cf590 2024-04-25T10:14:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.894+0000 7fdc66ffd640 1 -- 172.21.15.16:0/3991165975 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5c032440 con 0x7fdc680cf590 2024-04-25T10:14:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.894+0000 7fdc66ffd640 1 -- 172.21.15.16:0/3991165975 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c032810 con 0x7fdc680cf590 2024-04-25T10:14:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.894+0000 7fdc6e770640 1 -- 172.21.15.16:0/3991165975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc680cf590 msgr2=0x7fdc680cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.894+0000 7fdc6e770640 1 --2- 172.21.15.16:0/3991165975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc680cf590 0x7fdc680cd8c0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009870 tx=0x7fdc5c02a210 comp rx=0 tx=0).stop 2024-04-25T10:14:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.894+0000 7fdc6e770640 1 -- 172.21.15.16:0/3991165975 shutdown_connections 2024-04-25T10:14:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.894+0000 7fdc6e770640 1 --2- 172.21.15.16:0/3991165975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc68099b20 0x7fdc68099fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.894+0000 7fdc6e770640 1 --2- 172.21.15.16:0/3991165975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc680cf590 0x7fdc680cd8c0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.894+0000 7fdc6e770640 1 -- 172.21.15.16:0/3991165975 >> 172.21.15.16:0/3991165975 conn(0x7fdc680b33b0 msgr2=0x7fdc680b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.894+0000 7fdc6e770640 1 -- 172.21.15.16:0/3991165975 shutdown_connections 2024-04-25T10:14:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.894+0000 7fdc6e770640 1 -- 172.21.15.16:0/3991165975 wait complete. 2024-04-25T10:14:50.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.895+0000 7fdc6e770640 1 Processor -- start 2024-04-25T10:14:50.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.896+0000 7fdc6e770640 1 -- start start 2024-04-25T10:14:50.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.896+0000 7fdc6e770640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc68099b20 0x7fdc6814d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.896+0000 7fdc6e770640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc680cf590 0x7fdc6814dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.896+0000 7fdc6e770640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc6815bb60 0x7fdc681f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.896+0000 7fdc6e770640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc680cc440 con 0x7fdc68099b20 2024-04-25T10:14:50.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.896+0000 7fdc6e770640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc680cde00 con 0x7fdc6815bb60 2024-04-25T10:14:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.896+0000 7fdc6e770640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc680cc5c0 con 0x7fdc680cf590 2024-04-25T10:14:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.896+0000 7fdc67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc68099b20 0x7fdc6814d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.896+0000 7fdc67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc68099b20 0x7fdc6814d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58998/0 (socket says 172.21.15.16:58998) 2024-04-25T10:14:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.896+0000 7fdc67fff640 1 -- 172.21.15.16:0/921026431 learned_addr learned my addr 172.21.15.16:0/921026431 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.896+0000 7fdc6cd0d640 1 --2- 172.21.15.16:0/921026431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc6815bb60 0x7fdc681f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.896+0000 7fdc677fe640 1 --2- 172.21.15.16:0/921026431 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc680cf590 0x7fdc6814dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.897+0000 7fdc67fff640 1 -- 172.21.15.16:0/921026431 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc680cf590 msgr2=0x7fdc6814dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.897+0000 7fdc67fff640 1 --2- 172.21.15.16:0/921026431 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc680cf590 0x7fdc6814dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.897+0000 7fdc67fff640 1 -- 172.21.15.16:0/921026431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc6815bb60 msgr2=0x7fdc681f5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.897+0000 7fdc67fff640 1 --2- 172.21.15.16:0/921026431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc6815bb60 0x7fdc681f5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.897+0000 7fdc67fff640 1 -- 172.21.15.16:0/921026431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6814e270 con 0x7fdc68099b20 2024-04-25T10:14:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.897+0000 7fdc67fff640 1 --2- 172.21.15.16:0/921026431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc68099b20 0x7fdc6814d7f0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800b3f0 tx=0x7fdc5800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.897+0000 7fdc64ff9640 1 -- 172.21.15.16:0/921026431 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58015020 con 0x7fdc68099b20 2024-04-25T10:14:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.897+0000 7fdc64ff9640 1 -- 172.21.15.16:0/921026431 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc58011830 con 0x7fdc68099b20 2024-04-25T10:14:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.897+0000 7fdc6e770640 1 -- 172.21.15.16:0/921026431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc680c8f20 con 0x7fdc68099b20 2024-04-25T10:14:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.897+0000 7fdc64ff9640 1 -- 172.21.15.16:0/921026431 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58011c20 con 0x7fdc68099b20 2024-04-25T10:14:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.897+0000 7fdc6e770640 1 -- 172.21.15.16:0/921026431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc680c94e0 con 0x7fdc68099b20 2024-04-25T10:14:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.898+0000 7fdc6e770640 1 -- 172.21.15.16:0/921026431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc680750b0 con 0x7fdc68099b20 2024-04-25T10:14:50.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.899+0000 7fdc64ff9640 1 -- 172.21.15.16:0/921026431 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc58018020 con 0x7fdc68099b20 2024-04-25T10:14:50.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.899+0000 7fdc64ff9640 1 --2- 172.21.15.16:0/921026431 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdc5406b500 0x7fdc5406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.899+0000 7fdc64ff9640 1 -- 172.21.15.16:0/921026431 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fdc58089c90 con 0x7fdc68099b20 2024-04-25T10:14:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.900+0000 7fdc677fe640 1 --2- 172.21.15.16:0/921026431 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdc5406b500 0x7fdc5406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.900+0000 7fdc677fe640 1 --2- 172.21.15.16:0/921026431 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdc5406b500 0x7fdc5406d9c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c034000 tx=0x7fdc5c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:50.904+0000 7fdc64ff9640 1 -- 172.21.15.16:0/921026431 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdc5801f030 con 0x7fdc68099b20 2024-04-25T10:14:51.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:51.122+0000 7fdc6e770640 1 -- 172.21.15.16:0/921026431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fdc68070d70 con 0x7fdc68099b20 2024-04-25T10:14:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.053+0000 7fdc64ff9640 1 -- 172.21.15.16:0/921026431 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v92) v1 ==== 162+0+0 (secure 0 0 0) 0x7fdc580584a0 con 0x7fdc68099b20 2024-04-25T10:14:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:14:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.056+0000 7fdc6e770640 1 -- 172.21.15.16:0/921026431 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdc5406b500 msgr2=0x7fdc5406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.056+0000 7fdc6e770640 1 --2- 172.21.15.16:0/921026431 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdc5406b500 0x7fdc5406d9c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c034000 tx=0x7fdc5c02e000 comp rx=0 tx=0).stop 2024-04-25T10:14:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.056+0000 7fdc6e770640 1 -- 172.21.15.16:0/921026431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc68099b20 msgr2=0x7fdc6814d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.056+0000 7fdc6e770640 1 --2- 172.21.15.16:0/921026431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc68099b20 0x7fdc6814d7f0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800b3f0 tx=0x7fdc5800b8b0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.056+0000 7fdc6e770640 1 -- 172.21.15.16:0/921026431 shutdown_connections 2024-04-25T10:14:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.056+0000 7fdc6e770640 1 --2- 172.21.15.16:0/921026431 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdc5406b500 0x7fdc5406d9c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.056+0000 7fdc6e770640 1 --2- 172.21.15.16:0/921026431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc6815bb60 0x7fdc681f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.056+0000 7fdc6e770640 1 --2- 172.21.15.16:0/921026431 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc680cf590 0x7fdc6814dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.056+0000 7fdc6e770640 1 --2- 172.21.15.16:0/921026431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc68099b20 0x7fdc6814d7f0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.056+0000 7fdc6e770640 1 -- 172.21.15.16:0/921026431 >> 172.21.15.16:0/921026431 conn(0x7fdc680b33b0 msgr2=0x7fdc68070240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.057+0000 7fdc6e770640 1 -- 172.21.15.16:0/921026431 shutdown_connections 2024-04-25T10:14:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.057+0000 7fdc6e770640 1 -- 172.21.15.16:0/921026431 wait complete. 2024-04-25T10:14:52.076 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:52.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.178+0000 7f62f0f89640 1 Processor -- start 2024-04-25T10:14:52.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.178+0000 7f62f0f89640 1 -- start start 2024-04-25T10:14:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.179+0000 7f62f0f89640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62ec15dce0 0x7f62ec0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.179+0000 7f62f0f89640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f62ec0d35b0 0x7f62ec0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.179+0000 7f62f0f89640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ec15e0c0 con 0x7f62ec0d1f90 2024-04-25T10:14:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.179+0000 7f62f0f89640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ec0b7b10 con 0x7f62ec0d35b0 2024-04-25T10:14:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.179+0000 7f62f0f89640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0d2510 con 0x7f62ec15dce0 2024-04-25T10:14:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62ead9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f62ec0d1f90 0x7f62ec06bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56484/0 (socket says 172.21.15.16:56484) 2024-04-25T10:14:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62ead9d640 1 -- 172.21.15.16:0/1038392002 learned_addr learned my addr 172.21.15.16:0/1038392002 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62e9d9b640 1 --2- 172.21.15.16:0/1038392002 >> v2:172.21.15.186:3300/0 conn(0x7f62ec0d35b0 0x7f62ec0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62e959a640 1 -- 172.21.15.16:0/1038392002 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206463 0 0) 0x7f62ec15e0c0 con 0x7f62ec0d1f90 2024-04-25T10:14:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62ea59c640 1 --2- 172.21.15.16:0/1038392002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62ec15dce0 0x7f62ec0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62e959a640 1 -- 172.21.15.16:0/1038392002 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62d8003520 con 0x7f62ec0d1f90 2024-04-25T10:14:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62e959a640 1 -- 172.21.15.16:0/1038392002 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4220972109 0 0) 0x7f62d8003520 con 0x7f62ec0d1f90 2024-04-25T10:14:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62e959a640 1 -- 172.21.15.16:0/1038392002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62ec15dce0 msgr2=0x7f62ec0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62e959a640 1 --2- 172.21.15.16:0/1038392002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62ec15dce0 0x7f62ec0d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62e959a640 1 -- 172.21.15.16:0/1038392002 >> v2:172.21.15.186:3300/0 conn(0x7f62ec0d35b0 msgr2=0x7f62ec0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62e959a640 1 --2- 172.21.15.16:0/1038392002 >> v2:172.21.15.186:3300/0 conn(0x7f62ec0d35b0 0x7f62ec0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62e959a640 1 -- 172.21.15.16:0/1038392002 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec0d27f0 con 0x7f62ec0d1f90 2024-04-25T10:14:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62e959a640 1 -- 172.21.15.16:0/1038392002 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f62dc0019f0 con 0x7f62ec0d1f90 2024-04-25T10:14:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62e959a640 1 -- 172.21.15.16:0/1038392002 >> v1:172.21.15.16:6789/0 conn(0x7f62ec0d1f90 legacy=0x7f62ec06bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62e959a640 1 --2- 172.21.15.16:0/1038392002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62d80039b0 0x7f62d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62e959a640 1 --2- 172.21.15.16:0/1038392002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62d8004800 0x7f62d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62e959a640 1 --2- 172.21.15.16:0/1038392002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62ec15dce0 0x7f62d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62e959a640 1 -- 172.21.15.16:0/1038392002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0b7b10 con 0x7f62d8004800 2024-04-25T10:14:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62e959a640 1 -- 172.21.15.16:0/1038392002 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0d2510 con 0x7f62d80039b0 2024-04-25T10:14:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62e959a640 1 -- 172.21.15.16:0/1038392002 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d800b9c0 con 0x7f62ec15dce0 2024-04-25T10:14:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.180+0000 7f62e9d9b640 1 --2- 172.21.15.16:0/1038392002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62d8004800 0x7f62d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.181+0000 7f62ea59c640 1 --2- 172.21.15.16:0/1038392002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62d80039b0 0x7f62d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.181+0000 7f62ead9d640 1 --2- 172.21.15.16:0/1038392002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62ec15dce0 0x7f62d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.181+0000 7f62e9d9b640 1 -- 172.21.15.16:0/1038392002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62ec15dce0 msgr2=0x7f62d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.181+0000 7f62e9d9b640 1 --2- 172.21.15.16:0/1038392002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62ec15dce0 0x7f62d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.181+0000 7f62e9d9b640 1 -- 172.21.15.16:0/1038392002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62d80039b0 msgr2=0x7f62d8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.181+0000 7f62e9d9b640 1 --2- 172.21.15.16:0/1038392002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62d80039b0 0x7f62d8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.181+0000 7f62e9d9b640 1 -- 172.21.15.16:0/1038392002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d8003520 con 0x7f62d8004800 2024-04-25T10:14:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.181+0000 7f62e9d9b640 1 --2- 172.21.15.16:0/1038392002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62d8004800 0x7f62d8004c80 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f62d4009af0 tx=0x7f62d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3e5947f1b80236 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.182+0000 7f62e959a640 1 -- 172.21.15.16:0/1038392002 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4036070 con 0x7f62d8004800 2024-04-25T10:14:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.182+0000 7f62e959a640 1 -- 172.21.15.16:0/1038392002 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d4031400 con 0x7f62d8004800 2024-04-25T10:14:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.182+0000 7f62e959a640 1 -- 172.21.15.16:0/1038392002 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4009690 con 0x7f62d8004800 2024-04-25T10:14:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.182+0000 7f62f0f89640 1 -- 172.21.15.16:0/1038392002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62d8004800 msgr2=0x7f62d8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.182+0000 7f62f0f89640 1 --2- 172.21.15.16:0/1038392002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62d8004800 0x7f62d8004c80 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f62d4009af0 tx=0x7f62d8005830 comp rx=0 tx=0).stop 2024-04-25T10:14:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.182+0000 7f62ead9d640 1 -- 172.21.15.16:0/1038392002 reap_dead start 2024-04-25T10:14:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.182+0000 7f62f0f89640 1 -- 172.21.15.16:0/1038392002 shutdown_connections 2024-04-25T10:14:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.182+0000 7f62f0f89640 1 -- 172.21.15.16:0/1038392002 >> 172.21.15.16:0/1038392002 conn(0x7f62ec072900 msgr2=0x7f62ec0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.182+0000 7f62f0f89640 1 -- 172.21.15.16:0/1038392002 shutdown_connections 2024-04-25T10:14:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.182+0000 7f62f0f89640 1 -- 172.21.15.16:0/1038392002 wait complete. 2024-04-25T10:14:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.184+0000 7f62f0f89640 1 Processor -- start 2024-04-25T10:14:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.184+0000 7f62f0f89640 1 -- start start 2024-04-25T10:14:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.184+0000 7f62f0f89640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62d8004800 0x7f62ec0c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.184+0000 7f62f0f89640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62ec1527f0 0x7f62ec157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.184+0000 7f62f0f89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62ec152d40 0x7f62ec157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.184+0000 7f62f0f89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0d3090 con 0x7f62ec152d40 2024-04-25T10:14:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.184+0000 7f62f0f89640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0d2db0 con 0x7f62ec1527f0 2024-04-25T10:14:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.184+0000 7f62f0f89640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0b9820 con 0x7f62d8004800 2024-04-25T10:14:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.184+0000 7f62ead9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62ec152d40 0x7f62ec157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.185+0000 7f62ead9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62ec152d40 0x7f62ec157590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59004/0 (socket says 172.21.15.16:59004) 2024-04-25T10:14:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.185+0000 7f62ead9d640 1 -- 172.21.15.16:0/3655712220 learned_addr learned my addr 172.21.15.16:0/3655712220 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.185+0000 7f62ea59c640 1 --2- 172.21.15.16:0/3655712220 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62d8004800 0x7f62ec0c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.185+0000 7f62e9d9b640 1 --2- 172.21.15.16:0/3655712220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62ec1527f0 0x7f62ec157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.185+0000 7f62ead9d640 1 -- 172.21.15.16:0/3655712220 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62d8004800 msgr2=0x7f62ec0c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.185+0000 7f62ead9d640 1 --2- 172.21.15.16:0/3655712220 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62d8004800 0x7f62ec0c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.185+0000 7f62ead9d640 1 -- 172.21.15.16:0/3655712220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62ec1527f0 msgr2=0x7f62ec157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.185+0000 7f62ead9d640 1 --2- 172.21.15.16:0/3655712220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62ec1527f0 0x7f62ec157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.185+0000 7f62ead9d640 1 -- 172.21.15.16:0/3655712220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec1557f0 con 0x7f62ec152d40 2024-04-25T10:14:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.185+0000 7f62ead9d640 1 --2- 172.21.15.16:0/3655712220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62ec152d40 0x7f62ec157590 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0b33c0 tx=0x7f62dc007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.185+0000 7f62e2ffd640 1 -- 172.21.15.16:0/3655712220 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc00ac40 con 0x7f62ec152d40 2024-04-25T10:14:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.185+0000 7f62e2ffd640 1 -- 172.21.15.16:0/3655712220 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62dc018070 con 0x7f62ec152d40 2024-04-25T10:14:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.185+0000 7f62e2ffd640 1 -- 172.21.15.16:0/3655712220 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc0133f0 con 0x7f62ec152d40 2024-04-25T10:14:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.185+0000 7f62f0f89640 1 -- 172.21.15.16:0/3655712220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec155ad0 con 0x7f62ec152d40 2024-04-25T10:14:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.185+0000 7f62f0f89640 1 -- 172.21.15.16:0/3655712220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec156090 con 0x7f62ec152d40 2024-04-25T10:14:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.186+0000 7f62f0f89640 1 -- 172.21.15.16:0/3655712220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec075100 con 0x7f62ec152d40 2024-04-25T10:14:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.187+0000 7f62e2ffd640 1 -- 172.21.15.16:0/3655712220 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62dc0135b0 con 0x7f62ec152d40 2024-04-25T10:14:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.187+0000 7f62e2ffd640 1 --2- 172.21.15.16:0/3655712220 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f62cc06b500 0x7f62cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.188+0000 7f62e2ffd640 1 -- 172.21.15.16:0/3655712220 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f62dc08a930 con 0x7f62ec152d40 2024-04-25T10:14:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.188+0000 7f62ea59c640 1 --2- 172.21.15.16:0/3655712220 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f62cc06b500 0x7f62cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.188+0000 7f62ea59c640 1 --2- 172.21.15.16:0/3655712220 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f62cc06b500 0x7f62cc06d9c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f62e40099c0 tx=0x7f62e4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.192+0000 7f62e2ffd640 1 -- 172.21.15.16:0/3655712220 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62ec075100 con 0x7f62ec152d40 2024-04-25T10:14:52.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.412+0000 7f62f0f89640 1 -- 172.21.15.16:0/3655712220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62ec0749b0 con 0x7f62ec152d40 2024-04-25T10:14:52.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.413+0000 7f62e2ffd640 1 -- 172.21.15.16:0/3655712220 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f62dc059700 con 0x7f62ec152d40 2024-04-25T10:14:52.417 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:52.417 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2871061122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2871061122},{"type":"v1","addr":"172.21.15.186:6836","nonce":2871061122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1871507206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1871507206},{"type":"v1","addr":"172.21.15.186:6837","nonce":1871507206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4064412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4064412586},{"type":"v1","addr":"172.21.15.16:6834","nonce":4064412586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:26.751060+0000","modified":"2024-04-25T10:14:52.056922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9252":{"gid":9252,"name":"c","rank":0,"incarnation":93,"state":"up:replay","state_seq":1,"addr":"172.21.15.16:6835/3763439204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":3763439204},{"type":"v1","addr":"172.21.15.16:6835","nonce":3763439204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:52.418 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 93 2024-04-25T10:14:52.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.415+0000 7f62f0f89640 1 -- 172.21.15.16:0/3655712220 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f62cc06b500 msgr2=0x7f62cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.415+0000 7f62f0f89640 1 --2- 172.21.15.16:0/3655712220 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f62cc06b500 0x7f62cc06d9c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f62e40099c0 tx=0x7f62e4008040 comp rx=0 tx=0).stop 2024-04-25T10:14:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.415+0000 7f62f0f89640 1 -- 172.21.15.16:0/3655712220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62ec152d40 msgr2=0x7f62ec157590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.415+0000 7f62f0f89640 1 --2- 172.21.15.16:0/3655712220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62ec152d40 0x7f62ec157590 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0b33c0 tx=0x7f62dc007bd0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.415+0000 7f62f0f89640 1 -- 172.21.15.16:0/3655712220 shutdown_connections 2024-04-25T10:14:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.415+0000 7f62f0f89640 1 --2- 172.21.15.16:0/3655712220 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f62cc06b500 0x7f62cc06d9c0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.415+0000 7f62f0f89640 1 --2- 172.21.15.16:0/3655712220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62ec152d40 0x7f62ec157590 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.415+0000 7f62f0f89640 1 --2- 172.21.15.16:0/3655712220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62ec1527f0 0x7f62ec157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.415+0000 7f62f0f89640 1 --2- 172.21.15.16:0/3655712220 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62d8004800 0x7f62ec0c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.415+0000 7f62f0f89640 1 -- 172.21.15.16:0/3655712220 >> 172.21.15.16:0/3655712220 conn(0x7f62ec072900 msgr2=0x7f62ec0b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.415+0000 7f62f0f89640 1 -- 172.21.15.16:0/3655712220 shutdown_connections 2024-04-25T10:14:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:52.415+0000 7f62f0f89640 1 -- 172.21.15.16:0/3655712220 wait complete. 2024-04-25T10:14:52.833 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 93, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:14:26.751060+0000', 'modified': '2024-04-25T10:14:52.056922+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9252}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9252': {'gid': 9252, 'name': 'c', 'rank': 0, 'incarnation': 93, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.16:6835/3763439204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 3763439204}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 3763439204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:14:52.833 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9252:up:replay 2024-04-25T10:14:53.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.944+0000 7f6b887e7640 1 Processor -- start 2024-04-25T10:14:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.944+0000 7f6b887e7640 1 -- start start 2024-04-25T10:14:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.945+0000 7f6b887e7640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6b800d35b0 0x7f6b800b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.945+0000 7f6b887e7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6b800b5990 0x7f6b800b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.945+0000 7f6b887e7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b8015e0c0 con 0x7f6b8015dce0 2024-04-25T10:14:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.945+0000 7f6b887e7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b800d2510 con 0x7f6b800b5990 2024-04-25T10:14:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.945+0000 7f6b887e7640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b800d27f0 con 0x7f6b800d35b0 2024-04-25T10:14:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.945+0000 7f6b86d84640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6b800b5990 0x7f6b800b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.945+0000 7f6b86583640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6b8015dce0 0x7f6b800d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55668/0 (socket says 172.21.15.16:55668) 2024-04-25T10:14:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.945+0000 7f6b86583640 1 -- 172.21.15.16:0/121691360 learned_addr learned my addr 172.21.15.16:0/121691360 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.945+0000 7f6b85d82640 1 --2- 172.21.15.16:0/121691360 >> v2:172.21.15.186:3300/0 conn(0x7f6b800d35b0 0x7f6b800b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.946+0000 7f6b86d84640 1 -- 172.21.15.16:0/121691360 >> v2:172.21.15.186:3300/0 conn(0x7f6b800d35b0 msgr2=0x7f6b800b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.946+0000 7f6b86d84640 1 --2- 172.21.15.16:0/121691360 >> v2:172.21.15.186:3300/0 conn(0x7f6b800d35b0 0x7f6b800b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.946+0000 7f6b86d84640 1 -- 172.21.15.16:0/121691360 >> v1:172.21.15.16:6789/0 conn(0x7f6b8015dce0 legacy=0x7f6b800d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.946+0000 7f6b86d84640 1 -- 172.21.15.16:0/121691360 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b800d2ad0 con 0x7f6b800b5990 2024-04-25T10:14:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.946+0000 7f6b85581640 1 -- 172.21.15.16:0/121691360 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3930696276 0 0) 0x7f6b8015e0c0 con 0x7f6b8015dce0 2024-04-25T10:14:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.946+0000 7f6b86d84640 1 --2- 172.21.15.16:0/121691360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b800b5990 0x7f6b800b6b00 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f6b800cdfa0 tx=0x7f6b7802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd513589e80f19d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.946+0000 7f6b85581640 1 -- 172.21.15.16:0/121691360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b78007740 con 0x7f6b800b5990 2024-04-25T10:14:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.947+0000 7f6b85581640 1 -- 172.21.15.16:0/121691360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b780078e0 con 0x7f6b800b5990 2024-04-25T10:14:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.947+0000 7f6b85581640 1 -- 172.21.15.16:0/121691360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b78007cb0 con 0x7f6b800b5990 2024-04-25T10:14:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.947+0000 7f6b887e7640 1 -- 172.21.15.16:0/121691360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b800b5990 msgr2=0x7f6b800b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.947+0000 7f6b887e7640 1 --2- 172.21.15.16:0/121691360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b800b5990 0x7f6b800b6b00 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f6b800cdfa0 tx=0x7f6b7802a680 comp rx=0 tx=0).stop 2024-04-25T10:14:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.947+0000 7f6b887e7640 1 -- 172.21.15.16:0/121691360 shutdown_connections 2024-04-25T10:14:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.947+0000 7f6b887e7640 1 --2- 172.21.15.16:0/121691360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b800b5990 0x7f6b800b6b00 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.947+0000 7f6b887e7640 1 --2- 172.21.15.16:0/121691360 >> v2:172.21.15.186:3300/0 conn(0x7f6b800d35b0 0x7f6b800b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.947+0000 7f6b887e7640 1 -- 172.21.15.16:0/121691360 >> 172.21.15.16:0/121691360 conn(0x7f6b80072900 msgr2=0x7f6b800d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.947+0000 7f6b887e7640 1 -- 172.21.15.16:0/121691360 shutdown_connections 2024-04-25T10:14:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.947+0000 7f6b887e7640 1 -- 172.21.15.16:0/121691360 wait complete. 2024-04-25T10:14:53.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.948+0000 7f6b887e7640 1 Processor -- start 2024-04-25T10:14:53.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.948+0000 7f6b887e7640 1 -- start start 2024-04-25T10:14:53.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.949+0000 7f6b887e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b800b5990 0x7f6b80156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.949+0000 7f6b887e7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b800d35b0 0x7f6b800cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.949+0000 7f6b887e7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b8015dce0 0x7f6b800ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.949+0000 7f6b887e7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b800d2160 con 0x7f6b800b5990 2024-04-25T10:14:53.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.949+0000 7f6b887e7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b800d3090 con 0x7f6b800d35b0 2024-04-25T10:14:53.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.949+0000 7f6b887e7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8015e3c0 con 0x7f6b8015dce0 2024-04-25T10:14:53.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.949+0000 7f6b86583640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b800b5990 0x7f6b80156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.949+0000 7f6b86583640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b800b5990 0x7f6b80156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35466/0 (socket says 172.21.15.16:35466) 2024-04-25T10:14:53.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.949+0000 7f6b86583640 1 -- 172.21.15.16:0/3800919657 learned_addr learned my addr 172.21.15.16:0/3800919657 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:53.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.949+0000 7f6b85d82640 1 --2- 172.21.15.16:0/3800919657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b800d35b0 0x7f6b800cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.949+0000 7f6b86d84640 1 --2- 172.21.15.16:0/3800919657 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b8015dce0 0x7f6b800ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.950+0000 7f6b86583640 1 -- 172.21.15.16:0/3800919657 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b8015dce0 msgr2=0x7f6b800ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.950+0000 7f6b86583640 1 --2- 172.21.15.16:0/3800919657 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b8015dce0 0x7f6b800ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.950+0000 7f6b86583640 1 -- 172.21.15.16:0/3800919657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b800d35b0 msgr2=0x7f6b800cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.950+0000 7f6b86583640 1 --2- 172.21.15.16:0/3800919657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b800d35b0 0x7f6b800cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.950+0000 7f6b86583640 1 -- 172.21.15.16:0/3800919657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8015e6a0 con 0x7f6b800b5990 2024-04-25T10:14:53.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.950+0000 7f6b85d82640 1 --2- 172.21.15.16:0/3800919657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b800d35b0 0x7f6b800cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:53.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.950+0000 7f6b86d84640 1 --2- 172.21.15.16:0/3800919657 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b8015dce0 0x7f6b800ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:53.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.950+0000 7f6b86583640 1 --2- 172.21.15.16:0/3800919657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b800b5990 0x7f6b80156ef0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00c430 tx=0x7f6b7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.950+0000 7f6b76ffd640 1 -- 172.21.15.16:0/3800919657 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c0074e0 con 0x7f6b800b5990 2024-04-25T10:14:53.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.950+0000 7f6b76ffd640 1 -- 172.21.15.16:0/3800919657 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7c0047e0 con 0x7f6b800b5990 2024-04-25T10:14:53.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.950+0000 7f6b887e7640 1 -- 172.21.15.16:0/3800919657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8015e980 con 0x7f6b800b5990 2024-04-25T10:14:53.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.950+0000 7f6b76ffd640 1 -- 172.21.15.16:0/3800919657 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c004bd0 con 0x7f6b800b5990 2024-04-25T10:14:53.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.950+0000 7f6b887e7640 1 -- 172.21.15.16:0/3800919657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b80157430 con 0x7f6b800b5990 2024-04-25T10:14:53.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.951+0000 7f6b887e7640 1 -- 172.21.15.16:0/3800919657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b800cf8d0 con 0x7f6b800b5990 2024-04-25T10:14:53.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.952+0000 7f6b76ffd640 1 -- 172.21.15.16:0/3800919657 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b7c011070 con 0x7f6b800b5990 2024-04-25T10:14:53.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.952+0000 7f6b76ffd640 1 --2- 172.21.15.16:0/3800919657 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6b6406b500 0x7f6b6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.952+0000 7f6b76ffd640 1 -- 172.21.15.16:0/3800919657 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f6b7c089f60 con 0x7f6b800b5990 2024-04-25T10:14:53.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.952+0000 7f6b85d82640 1 --2- 172.21.15.16:0/3800919657 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6b6406b500 0x7f6b6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.953+0000 7f6b85d82640 1 --2- 172.21.15.16:0/3800919657 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6b6406b500 0x7f6b6406d9c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6b800cd1d0 tx=0x7f6b70008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:53.956+0000 7f6b76ffd640 1 -- 172.21.15.16:0/3800919657 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b800cf8d0 con 0x7f6b800b5990 2024-04-25T10:14:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:54.174+0000 7f6b887e7640 1 -- 172.21.15.16:0/3800919657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b800749b0 con 0x7f6b800b5990 2024-04-25T10:14:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:54.174+0000 7f6b76ffd640 1 -- 172.21.15.16:0/3800919657 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f6b7c058cf0 con 0x7f6b800b5990 2024-04-25T10:14:54.178 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:54.179 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":95,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2871061122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2871061122},{"type":"v1","addr":"172.21.15.186:6836","nonce":2871061122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1871507206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1871507206},{"type":"v1","addr":"172.21.15.186:6837","nonce":1871507206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4064412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4064412586},{"type":"v1","addr":"172.21.15.16:6834","nonce":4064412586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:26.751060+0000","modified":"2024-04-25T10:14:53.061252+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9252":{"gid":9252,"name":"c","rank":0,"incarnation":93,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.16:6835/3763439204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":3763439204},{"type":"v1","addr":"172.21.15.16:6835","nonce":3763439204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 95 2024-04-25T10:14:54.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:54.176+0000 7f6b887e7640 1 -- 172.21.15.16:0/3800919657 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6b6406b500 msgr2=0x7f6b6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:54.176+0000 7f6b887e7640 1 --2- 172.21.15.16:0/3800919657 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6b6406b500 0x7f6b6406d9c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6b800cd1d0 tx=0x7f6b70008040 comp rx=0 tx=0).stop 2024-04-25T10:14:54.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:54.176+0000 7f6b887e7640 1 -- 172.21.15.16:0/3800919657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b800b5990 msgr2=0x7f6b80156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:54.176+0000 7f6b887e7640 1 --2- 172.21.15.16:0/3800919657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b800b5990 0x7f6b80156ef0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00c430 tx=0x7f6b7c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:54.176+0000 7f6b887e7640 1 -- 172.21.15.16:0/3800919657 shutdown_connections 2024-04-25T10:14:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:54.176+0000 7f6b887e7640 1 --2- 172.21.15.16:0/3800919657 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f6b6406b500 0x7f6b6406d9c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:54.176+0000 7f6b887e7640 1 --2- 172.21.15.16:0/3800919657 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b8015dce0 0x7f6b800ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:54.176+0000 7f6b887e7640 1 --2- 172.21.15.16:0/3800919657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b800d35b0 0x7f6b800cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:54.176+0000 7f6b887e7640 1 --2- 172.21.15.16:0/3800919657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b800b5990 0x7f6b80156ef0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:54.176+0000 7f6b887e7640 1 -- 172.21.15.16:0/3800919657 >> 172.21.15.16:0/3800919657 conn(0x7f6b80072900 msgr2=0x7f6b8015ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:54.176+0000 7f6b887e7640 1 -- 172.21.15.16:0/3800919657 shutdown_connections 2024-04-25T10:14:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:54.176+0000 7f6b887e7640 1 -- 172.21.15.16:0/3800919657 wait complete. 2024-04-25T10:14:54.594 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 95, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:14:26.751060+0000', 'modified': '2024-04-25T10:14:53.061252+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9252}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9252': {'gid': 9252, 'name': 'c', 'rank': 0, 'incarnation': 93, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.16:6835/3763439204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 3763439204}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 3763439204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:14:54.594 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9252:up:rejoin 2024-04-25T10:14:55.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:55.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.706+0000 7f3524015640 1 Processor -- start 2024-04-25T10:14:55.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.706+0000 7f3524015640 1 -- start start 2024-04-25T10:14:55.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.707+0000 7f3524015640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f351c0d35b0 0x7f351c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.707+0000 7f3524015640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f351c0b5990 0x7f351c0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.707+0000 7f3524015640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351c15e0c0 con 0x7f351c15dce0 2024-04-25T10:14:55.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.707+0000 7f3524015640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f351c0b5de0 con 0x7f351c0b5990 2024-04-25T10:14:55.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.707+0000 7f3524015640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351c0d2510 con 0x7f351c0d35b0 2024-04-25T10:14:55.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.707+0000 7f35225b2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f351c0b5990 0x7f351c0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.707+0000 7f3521db1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f351c15dce0 0x7f351c0d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55674/0 (socket says 172.21.15.16:55674) 2024-04-25T10:14:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.707+0000 7f3521db1640 1 -- 172.21.15.16:0/2215622465 learned_addr learned my addr 172.21.15.16:0/2215622465 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.707+0000 7f35215b0640 1 --2- 172.21.15.16:0/2215622465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f351c0d35b0 0x7f351c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.707+0000 7f35225b2640 1 -- 172.21.15.16:0/2215622465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f351c0d35b0 msgr2=0x7f351c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.707+0000 7f35225b2640 1 --2- 172.21.15.16:0/2215622465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f351c0d35b0 0x7f351c0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.707+0000 7f35225b2640 1 -- 172.21.15.16:0/2215622465 >> v1:172.21.15.16:6789/0 conn(0x7f351c15dce0 legacy=0x7f351c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.707+0000 7f35225b2640 1 -- 172.21.15.16:0/2215622465 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351c0d27f0 con 0x7f351c0b5990 2024-04-25T10:14:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.707+0000 7f3520daf640 1 -- 172.21.15.16:0/2215622465 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3526331105 0 0) 0x7f351c15e0c0 con 0x7f351c15dce0 2024-04-25T10:14:55.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.708+0000 7f35225b2640 1 --2- 172.21.15.16:0/2215622465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0b5990 0x7f351c0b6b00 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f3514009d90 tx=0x7f351402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818a6daa4824018f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.708+0000 7f3520daf640 1 -- 172.21.15.16:0/2215622465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3514007740 con 0x7f351c0b5990 2024-04-25T10:14:55.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.708+0000 7f3520daf640 1 -- 172.21.15.16:0/2215622465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35140078e0 con 0x7f351c0b5990 2024-04-25T10:14:55.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.708+0000 7f3520daf640 1 -- 172.21.15.16:0/2215622465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3514007cb0 con 0x7f351c0b5990 2024-04-25T10:14:55.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.709+0000 7f3524015640 1 -- 172.21.15.16:0/2215622465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0b5990 msgr2=0x7f351c0b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.709+0000 7f3524015640 1 --2- 172.21.15.16:0/2215622465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0b5990 0x7f351c0b6b00 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f3514009d90 tx=0x7f351402a660 comp rx=0 tx=0).stop 2024-04-25T10:14:55.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.709+0000 7f3524015640 1 -- 172.21.15.16:0/2215622465 shutdown_connections 2024-04-25T10:14:55.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.709+0000 7f3524015640 1 --2- 172.21.15.16:0/2215622465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0b5990 0x7f351c0b6b00 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.709+0000 7f3524015640 1 --2- 172.21.15.16:0/2215622465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f351c0d35b0 0x7f351c0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.709+0000 7f3524015640 1 -- 172.21.15.16:0/2215622465 >> 172.21.15.16:0/2215622465 conn(0x7f351c072900 msgr2=0x7f351c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.709+0000 7f3524015640 1 -- 172.21.15.16:0/2215622465 shutdown_connections 2024-04-25T10:14:55.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.709+0000 7f3524015640 1 -- 172.21.15.16:0/2215622465 wait complete. 2024-04-25T10:14:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.710+0000 7f3524015640 1 Processor -- start 2024-04-25T10:14:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.710+0000 7f3524015640 1 -- start start 2024-04-25T10:14:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.711+0000 7f3524015640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0b5990 0x7f351c0c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.711+0000 7f3524015640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f351c0d35b0 0x7f351c0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.711+0000 7f3524015640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f351c15dce0 0x7f351c14a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.711+0000 7f3524015640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351c0d3090 con 0x7f351c0b5990 2024-04-25T10:14:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.711+0000 7f3524015640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351c0d2db0 con 0x7f351c15dce0 2024-04-25T10:14:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.711+0000 7f3524015640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351c0d2160 con 0x7f351c0d35b0 2024-04-25T10:14:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.711+0000 7f3521db1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0b5990 0x7f351c0c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.711+0000 7f3521db1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0b5990 0x7f351c0c9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35494/0 (socket says 172.21.15.16:35494) 2024-04-25T10:14:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.711+0000 7f3521db1640 1 -- 172.21.15.16:0/2200371749 learned_addr learned my addr 172.21.15.16:0/2200371749 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.711+0000 7f35215b0640 1 --2- 172.21.15.16:0/2200371749 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f351c0d35b0 0x7f351c0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.711+0000 7f35225b2640 1 --2- 172.21.15.16:0/2200371749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f351c15dce0 0x7f351c14a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.711+0000 7f3521db1640 1 -- 172.21.15.16:0/2200371749 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f351c0d35b0 msgr2=0x7f351c0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.711+0000 7f3521db1640 1 --2- 172.21.15.16:0/2200371749 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f351c0d35b0 0x7f351c0cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.711+0000 7f3521db1640 1 -- 172.21.15.16:0/2200371749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f351c15dce0 msgr2=0x7f351c14a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.711+0000 7f3521db1640 1 --2- 172.21.15.16:0/2200371749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f351c15dce0 0x7f351c14a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.711+0000 7f3521db1640 1 -- 172.21.15.16:0/2200371749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351c0c8cb0 con 0x7f351c0b5990 2024-04-25T10:14:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.711+0000 7f35225b2640 1 --2- 172.21.15.16:0/2200371749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f351c15dce0 0x7f351c14a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.711+0000 7f3521db1640 1 --2- 172.21.15.16:0/2200371749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0b5990 0x7f351c0c9d50 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f350c00c460 tx=0x7f350c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.712+0000 7f35127fc640 1 -- 172.21.15.16:0/2200371749 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c018070 con 0x7f351c0b5990 2024-04-25T10:14:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.712+0000 7f35127fc640 1 -- 172.21.15.16:0/2200371749 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350c007500 con 0x7f351c0b5990 2024-04-25T10:14:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.712+0000 7f3524015640 1 -- 172.21.15.16:0/2200371749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351c0c8f90 con 0x7f351c0b5990 2024-04-25T10:14:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.712+0000 7f35127fc640 1 -- 172.21.15.16:0/2200371749 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c004a20 con 0x7f351c0b5990 2024-04-25T10:14:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.712+0000 7f3524015640 1 -- 172.21.15.16:0/2200371749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351c0c9550 con 0x7f351c0b5990 2024-04-25T10:14:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.713+0000 7f3524015640 1 -- 172.21.15.16:0/2200371749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f351c0c9a40 con 0x7f351c0b5990 2024-04-25T10:14:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.713+0000 7f35127fc640 1 -- 172.21.15.16:0/2200371749 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f350c01b070 con 0x7f351c0b5990 2024-04-25T10:14:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.714+0000 7f35127fc640 1 --2- 172.21.15.16:0/2200371749 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f350006b4d0 0x7f350006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.714+0000 7f35127fc640 1 -- 172.21.15.16:0/2200371749 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f350c089530 con 0x7f351c0b5990 2024-04-25T10:14:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.714+0000 7f35215b0640 1 --2- 172.21.15.16:0/2200371749 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f350006b4d0 0x7f350006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.715+0000 7f35215b0640 1 --2- 172.21.15.16:0/2200371749 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f350006b4d0 0x7f350006d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f351c1f6c60 tx=0x7f3518008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.718+0000 7f35127fc640 1 -- 172.21.15.16:0/2200371749 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f350c058350 con 0x7f351c0b5990 2024-04-25T10:14:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.938+0000 7f3524015640 1 -- 172.21.15.16:0/2200371749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f351c0b3430 con 0x7f351c0b5990 2024-04-25T10:14:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.939+0000 7f35127fc640 1 -- 172.21.15.16:0/2200371749 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f350c057c50 con 0x7f351c0b5990 2024-04-25T10:14:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:55.944 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":96,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2871061122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2871061122},{"type":"v1","addr":"172.21.15.186:6836","nonce":2871061122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1871507206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1871507206},{"type":"v1","addr":"172.21.15.186:6837","nonce":1871507206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4064412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4064412586},{"type":"v1","addr":"172.21.15.16:6834","nonce":4064412586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:26.751060+0000","modified":"2024-04-25T10:14:54.069182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9252":{"gid":9252,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6835/3763439204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":3763439204},{"type":"v1","addr":"172.21.15.16:6835","nonce":3763439204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:55.946 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 96 2024-04-25T10:14:55.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.941+0000 7f3524015640 1 -- 172.21.15.16:0/2200371749 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f350006b4d0 msgr2=0x7f350006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.941+0000 7f3524015640 1 --2- 172.21.15.16:0/2200371749 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f350006b4d0 0x7f350006d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f351c1f6c60 tx=0x7f3518008040 comp rx=0 tx=0).stop 2024-04-25T10:14:55.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.941+0000 7f3524015640 1 -- 172.21.15.16:0/2200371749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0b5990 msgr2=0x7f351c0c9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.941+0000 7f3524015640 1 --2- 172.21.15.16:0/2200371749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0b5990 0x7f351c0c9d50 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f350c00c460 tx=0x7f350c00c920 comp rx=0 tx=0).stop 2024-04-25T10:14:55.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.941+0000 7f3524015640 1 -- 172.21.15.16:0/2200371749 shutdown_connections 2024-04-25T10:14:55.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.941+0000 7f3524015640 1 --2- 172.21.15.16:0/2200371749 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f350006b4d0 0x7f350006d990 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.941+0000 7f3524015640 1 --2- 172.21.15.16:0/2200371749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f351c15dce0 0x7f351c14a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.941+0000 7f3524015640 1 --2- 172.21.15.16:0/2200371749 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f351c0d35b0 0x7f351c0cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.941+0000 7f3524015640 1 --2- 172.21.15.16:0/2200371749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0b5990 0x7f351c0c9d50 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.941+0000 7f3524015640 1 -- 172.21.15.16:0/2200371749 >> 172.21.15.16:0/2200371749 conn(0x7f351c072900 msgr2=0x7f351c0b8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.941+0000 7f3524015640 1 -- 172.21.15.16:0/2200371749 shutdown_connections 2024-04-25T10:14:55.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:55.941+0000 7f3524015640 1 -- 172.21.15.16:0/2200371749 wait complete. 2024-04-25T10:14:56.358 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:14:26.751060+0000', 'modified': '2024-04-25T10:14:54.069182+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9252}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9252': {'gid': 9252, 'name': 'c', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.16:6835/3763439204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 3763439204}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 3763439204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:14:56.358 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:14:56.358 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:14:56.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.459+0000 7f0bee6e4640 1 Processor -- start 2024-04-25T10:14:56.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.459+0000 7f0bee6e4640 1 -- start start 2024-04-25T10:14:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.460+0000 7f0bee6e4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0be815dec0 0x7f0be80d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.460+0000 7f0bee6e4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0be80d0b60 0x7f0be80b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.460+0000 7f0bee6e4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be80d2760 con 0x7f0be80d22e0 2024-04-25T10:14:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.460+0000 7f0bee6e4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be80d2a20 con 0x7f0be80d0b60 2024-04-25T10:14:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.460+0000 7f0bee6e4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be80d2d00 con 0x7f0be815dec0 2024-04-25T10:14:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.460+0000 7f0becc81640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0be80d0b60 0x7f0be80b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.460+0000 7f0be77fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0be80d22e0 0x7f0be80d0450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55678/0 (socket says 172.21.15.16:55678) 2024-04-25T10:14:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.460+0000 7f0be77fe640 1 -- 172.21.15.16:0/2934855596 learned_addr learned my addr 172.21.15.16:0/2934855596 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.460+0000 7f0be7fff640 1 --2- 172.21.15.16:0/2934855596 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0be815dec0 0x7f0be80d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.460+0000 7f0becc81640 1 -- 172.21.15.16:0/2934855596 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0be815dec0 msgr2=0x7f0be80d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.460+0000 7f0becc81640 1 --2- 172.21.15.16:0/2934855596 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0be815dec0 0x7f0be80d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.460+0000 7f0becc81640 1 -- 172.21.15.16:0/2934855596 >> v1:172.21.15.16:6789/0 conn(0x7f0be80d22e0 legacy=0x7f0be80d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.460+0000 7f0becc81640 1 -- 172.21.15.16:0/2934855596 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be80d2fe0 con 0x7f0be80d0b60 2024-04-25T10:14:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.460+0000 7f0be6ffd640 1 -- 172.21.15.16:0/2934855596 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375073891 0 0) 0x7f0be80d2760 con 0x7f0be80d22e0 2024-04-25T10:14:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.460+0000 7f0be7fff640 1 --2- 172.21.15.16:0/2934855596 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0be815dec0 0x7f0be80d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.461+0000 7f0becc81640 1 --2- 172.21.15.16:0/2934855596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be80d0b60 0x7f0be80b6e90 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009d90 tx=0x7f0be002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b2a04917a961b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.461+0000 7f0be6ffd640 1 -- 172.21.15.16:0/2934855596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0007740 con 0x7f0be80d0b60 2024-04-25T10:14:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.461+0000 7f0be6ffd640 1 -- 172.21.15.16:0/2934855596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be00078e0 con 0x7f0be80d0b60 2024-04-25T10:14:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.461+0000 7f0be6ffd640 1 -- 172.21.15.16:0/2934855596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0007cb0 con 0x7f0be80d0b60 2024-04-25T10:14:56.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.462+0000 7f0bee6e4640 1 -- 172.21.15.16:0/2934855596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be80d0b60 msgr2=0x7f0be80b6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.462+0000 7f0bee6e4640 1 --2- 172.21.15.16:0/2934855596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be80d0b60 0x7f0be80b6e90 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009d90 tx=0x7f0be002a660 comp rx=0 tx=0).stop 2024-04-25T10:14:56.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.462+0000 7f0bee6e4640 1 -- 172.21.15.16:0/2934855596 shutdown_connections 2024-04-25T10:14:56.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.462+0000 7f0bee6e4640 1 --2- 172.21.15.16:0/2934855596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be80d0b60 0x7f0be80b6e90 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.462+0000 7f0bee6e4640 1 --2- 172.21.15.16:0/2934855596 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0be815dec0 0x7f0be80d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.462+0000 7f0bee6e4640 1 -- 172.21.15.16:0/2934855596 >> 172.21.15.16:0/2934855596 conn(0x7f0be8072900 msgr2=0x7f0be80b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.462+0000 7f0bee6e4640 1 -- 172.21.15.16:0/2934855596 shutdown_connections 2024-04-25T10:14:56.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.462+0000 7f0bee6e4640 1 -- 172.21.15.16:0/2934855596 wait complete. 2024-04-25T10:14:56.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.463+0000 7f0bee6e4640 1 Processor -- start 2024-04-25T10:14:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.463+0000 7f0bee6e4640 1 -- start start 2024-04-25T10:14:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0bee6e4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0be80d0b60 0x7f0be80cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0bee6e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be80d22e0 0x7f0be8151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0bee6e4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0be815dec0 0x7f0be80c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0bee6e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be80b73d0 con 0x7f0be80d22e0 2024-04-25T10:14:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0bee6e4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be80d35a0 con 0x7f0be815dec0 2024-04-25T10:14:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0bee6e4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be80b7550 con 0x7f0be80d0b60 2024-04-25T10:14:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0be77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be80d22e0 0x7f0be8151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0be77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be80d22e0 0x7f0be8151920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35504/0 (socket says 172.21.15.16:35504) 2024-04-25T10:14:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0be77fe640 1 -- 172.21.15.16:0/2553476135 learned_addr learned my addr 172.21.15.16:0/2553476135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0be7fff640 1 --2- 172.21.15.16:0/2553476135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0be80d0b60 0x7f0be80cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0becc81640 1 --2- 172.21.15.16:0/2553476135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0be815dec0 0x7f0be80c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0be77fe640 1 -- 172.21.15.16:0/2553476135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0be80d0b60 msgr2=0x7f0be80cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0be77fe640 1 --2- 172.21.15.16:0/2553476135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0be80d0b60 0x7f0be80cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0be77fe640 1 -- 172.21.15.16:0/2553476135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0be815dec0 msgr2=0x7f0be80c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0be77fe640 1 --2- 172.21.15.16:0/2553476135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0be815dec0 0x7f0be80c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0be77fe640 1 -- 172.21.15.16:0/2553476135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be80cd900 con 0x7f0be80d22e0 2024-04-25T10:14:56.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0be77fe640 1 --2- 172.21.15.16:0/2553476135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be80d22e0 0x7f0be8151920 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800c4d0 tx=0x7f0bd800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0be4ff9640 1 -- 172.21.15.16:0/2553476135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8018070 con 0x7f0be80d22e0 2024-04-25T10:14:56.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0be4ff9640 1 -- 172.21.15.16:0/2553476135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd8007500 con 0x7f0be80d22e0 2024-04-25T10:14:56.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0bee6e4640 1 -- 172.21.15.16:0/2553476135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be80cc920 con 0x7f0be80d22e0 2024-04-25T10:14:56.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.464+0000 7f0be4ff9640 1 -- 172.21.15.16:0/2553476135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8004a20 con 0x7f0be80d22e0 2024-04-25T10:14:56.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.465+0000 7f0bee6e4640 1 -- 172.21.15.16:0/2553476135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be80ccbc0 con 0x7f0be80d22e0 2024-04-25T10:14:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.466+0000 7f0be4ff9640 1 -- 172.21.15.16:0/2553476135 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0bd801b070 con 0x7f0be80d22e0 2024-04-25T10:14:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.466+0000 7f0be4ff9640 1 --2- 172.21.15.16:0/2553476135 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0bd406b500 0x7f0bd406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.466+0000 7f0be4ff9640 1 -- 172.21.15.16:0/2553476135 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f0bd80895b0 con 0x7f0be80d22e0 2024-04-25T10:14:56.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.467+0000 7f0be7fff640 1 --2- 172.21.15.16:0/2553476135 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0bd406b500 0x7f0bd406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.467+0000 7f0be7fff640 1 --2- 172.21.15.16:0/2553476135 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0bd406b500 0x7f0bd406d9c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00a950 tx=0x7f0bdc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.467+0000 7f0bf13df740 1 Processor -- start 2024-04-25T10:14:56.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.467+0000 7f0bf13df740 1 -- start start 2024-04-25T10:14:56.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.468+0000 7f0bf13df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5d5230 0x55bc0a5d5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.468+0000 7f0bf13df740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5f2ad0 0x55bc0a5f3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.468+0000 7f0bf13df740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5aaf10 0x55bc0a5f1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.468+0000 7f0bf13df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0a50da10 con 0x55bc0a5d5230 2024-04-25T10:14:56.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.468+0000 7f0bf13df740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0a4b6b30 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.468+0000 7f0bf13df740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc0a509c50 con 0x55bc0a5aaf10 2024-04-25T10:14:56.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.468+0000 7f0be7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5d5230 0x55bc0a5d5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.468+0000 7f0be7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5d5230 0x55bc0a5d5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35510/0 (socket says 172.21.15.16:35510) 2024-04-25T10:14:56.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.468+0000 7f0be7fff640 1 -- 172.21.15.16:0/2363345246 learned_addr learned my addr 172.21.15.16:0/2363345246 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.468+0000 7f0be77fe640 1 --2- 172.21.15.16:0/2363345246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5f2ad0 0x55bc0a5f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.468+0000 7f0becc81640 1 --2- 172.21.15.16:0/2363345246 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5aaf10 0x55bc0a5f1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.468+0000 7f0be7fff640 1 -- 172.21.15.16:0/2363345246 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5aaf10 msgr2=0x55bc0a5f1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.468+0000 7f0be7fff640 1 --2- 172.21.15.16:0/2363345246 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5aaf10 0x55bc0a5f1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.469+0000 7f0be7fff640 1 -- 172.21.15.16:0/2363345246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5f2ad0 msgr2=0x55bc0a5f3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.469+0000 7f0be7fff640 1 --2- 172.21.15.16:0/2363345246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5f2ad0 0x55bc0a5f3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.469+0000 7f0be7fff640 1 -- 172.21.15.16:0/2363345246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0a4cbe10 con 0x55bc0a5d5230 2024-04-25T10:14:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.469+0000 7f0be77fe640 1 --2- 172.21.15.16:0/2363345246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5f2ad0 0x55bc0a5f3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.469+0000 7f0be7fff640 1 --2- 172.21.15.16:0/2363345246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5d5230 0x55bc0a5d5610 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00a160 tx=0x7f0bdc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b336b15eca9f662 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.469+0000 7f0bcdffb640 1 -- 172.21.15.16:0/2363345246 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc01a050 con 0x55bc0a5d5230 2024-04-25T10:14:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.469+0000 7f0bcdffb640 1 -- 172.21.15.16:0/2363345246 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc024070 con 0x55bc0a5d5230 2024-04-25T10:14:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.469+0000 7f0bcdffb640 1 -- 172.21.15.16:0/2363345246 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc01f440 con 0x55bc0a5d5230 2024-04-25T10:14:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.469+0000 7f0bf13df740 1 -- 172.21.15.16:0/2363345246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5d5230 msgr2=0x55bc0a5d5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.469+0000 7f0bf13df740 1 --2- 172.21.15.16:0/2363345246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5d5230 0x55bc0a5d5610 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00a160 tx=0x7f0bdc011000 comp rx=0 tx=0).stop 2024-04-25T10:14:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.469+0000 7f0bf13df740 1 -- 172.21.15.16:0/2363345246 shutdown_connections 2024-04-25T10:14:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.469+0000 7f0bf13df740 1 --2- 172.21.15.16:0/2363345246 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5aaf10 0x55bc0a5f1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.469+0000 7f0bf13df740 1 --2- 172.21.15.16:0/2363345246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5f2ad0 0x55bc0a5f3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.469+0000 7f0bf13df740 1 --2- 172.21.15.16:0/2363345246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5d5230 0x55bc0a5d5610 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.469+0000 7f0bf13df740 1 -- 172.21.15.16:0/2363345246 >> 172.21.15.16:0/2363345246 conn(0x55bc0a5a7840 msgr2=0x55bc0a5f0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.469+0000 7f0bf13df740 1 -- 172.21.15.16:0/2363345246 shutdown_connections 2024-04-25T10:14:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.469+0000 7f0bf13df740 1 -- 172.21.15.16:0/2363345246 wait complete. 2024-04-25T10:14:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.470+0000 7f0bf13df740 1 -- start start 2024-04-25T10:14:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.470+0000 7f0bf13df740 1 Processor -- start 2024-04-25T10:14:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.471+0000 7f0bf13df740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5d5230 0x55bc0a5d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.471+0000 7f0bf13df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5f2ad0 0x55bc0a5d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.471+0000 7f0bf13df740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5d4350 0x55bc0a5d7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.471+0000 7f0bf13df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0a5abcb0 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.471+0000 7f0bf13df740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0a4ca0c0 con 0x55bc0a5d5230 2024-04-25T10:14:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.471+0000 7f0bf13df740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc0a5a5770 con 0x55bc0a5d4350 2024-04-25T10:14:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.471+0000 7f0bf13df740 10 client.0 _ll_register_callbacks cb 0x55bc0a4bdec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.471+0000 7f0be77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5f2ad0 0x55bc0a5d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.471+0000 7f0be77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5f2ad0 0x55bc0a5d3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35518/0 (socket says 172.21.15.16:35518) 2024-04-25T10:14:56.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.471+0000 7f0be77fe640 1 -- 172.21.15.16:0/1134511600 learned_addr learned my addr 172.21.15.16:0/1134511600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:56.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.471+0000 7f0be7fff640 1 --2- 172.21.15.16:0/1134511600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5d5230 0x55bc0a5d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.471+0000 7f0becc81640 1 --2- 172.21.15.16:0/1134511600 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5d4350 0x55bc0a5d7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.471+0000 7f0be77fe640 1 -- 172.21.15.16:0/1134511600 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5d4350 msgr2=0x55bc0a5d7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.471+0000 7f0be77fe640 1 --2- 172.21.15.16:0/1134511600 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5d4350 0x55bc0a5d7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.471+0000 7f0be77fe640 1 -- 172.21.15.16:0/1134511600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5d5230 msgr2=0x55bc0a5d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.471+0000 7f0be77fe640 1 --2- 172.21.15.16:0/1134511600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5d5230 0x55bc0a5d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.471+0000 7f0be77fe640 1 -- 172.21.15.16:0/1134511600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc0a2222d0 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.472+0000 7f0be77fe640 1 --2- 172.21.15.16:0/1134511600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5f2ad0 0x55bc0a5d3e10 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f0bd805fe20 tx=0x7f0bd8058400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498ce2f75adf9416 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.472+0000 7f0bc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:14:56.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.472+0000 7f0bc6ffd640 1 -- 172.21.15.16:0/1134511600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8004dc0 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.472+0000 7f0bc6ffd640 1 -- 172.21.15.16:0/1134511600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f0bd8057090 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.472+0000 7f0bf13df740 1 -- 172.21.15.16:0/1134511600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bdc024070 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.472+0000 7f0bc6ffd640 1 -- 172.21.15.16:0/1134511600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bd805e070 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.472+0000 7f0bf13df740 10 client.9354 fetch_fsmap learned FSMap version 96 2024-04-25T10:14:56.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.472+0000 7f0bf13df740 1 -- 172.21.15.16:0/1134511600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x55bc0a2ec6b0 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.472+0000 7f0bc6ffd640 1 -- 172.21.15.16:0/1134511600 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc024070 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.472+0000 7f0bc6ffd640 1 -- 172.21.15.16:0/1134511600 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 96) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0bd80890b0 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.472+0000 7f0bf13df740 10 client.9354 fetch_fsmap finished waiting for FSMap version 96 2024-04-25T10:14:56.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.472+0000 7f0bf13df740 10 client.9354 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:14:56.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.472+0000 7f0bf13df740 20 client.9354 populate_metadata read hostname 'smithi016' 2024-04-25T10:14:56.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.472+0000 7f0bf13df740 1 --2- 172.21.15.16:0/1134511600 >> [v2:172.21.15.16:6833/3763439204,v1:172.21.15.16:6835/3763439204] conn(0x55bc0a5d64a0 0x55bc0a5d6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.472+0000 7f0bf13df740 4 client.9354 mds_command: new command op to 9252 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:56.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.472+0000 7f0bf13df740 1 -- 172.21.15.16:0/1134511600 --> [v2:172.21.15.16:6833/3763439204,v1:172.21.15.16:6835/3763439204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc0a56c920 con 0x55bc0a5d64a0 2024-04-25T10:14:56.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.472+0000 7f0be7fff640 1 --2- 172.21.15.16:0/1134511600 >> [v2:172.21.15.16:6833/3763439204,v1:172.21.15.16:6835/3763439204] conn(0x55bc0a5d64a0 0x55bc0a5d6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.473+0000 7f0be7fff640 1 --2- 172.21.15.16:0/1134511600 >> [v2:172.21.15.16:6833/3763439204,v1:172.21.15.16:6835/3763439204] conn(0x55bc0a5d64a0 0x55bc0a5d6880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cddaafd4693850a server_cookie=a1435580923a2b40 in_seq=0 out_seq=0 2024-04-25T10:14:56.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.473+0000 7f0bc6ffd640 10 client.9354 ms_handle_connect on v2:172.21.15.16:6833/3763439204 2024-04-25T10:14:56.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.473+0000 7f0bc6ffd640 1 -- 172.21.15.16:0/1134511600 <== mds.0 v2:172.21.15.16:6833/3763439204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc0a56c920 con 0x55bc0a5d64a0 2024-04-25T10:14:56.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.473+0000 7f0bc6ffd640 10 client.9354 handle_command_reply: tid=0 2024-04-25T10:14:56.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.473+0000 7f0bf13df740 1 client.9354 shutdown 2024-04-25T10:14:56.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.473+0000 7f0bf13df740 1 -- 172.21.15.16:0/1134511600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5f2ad0 msgr2=0x55bc0a5d3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.474+0000 7f0bf13df740 1 --2- 172.21.15.16:0/1134511600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5f2ad0 0x55bc0a5d3e10 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f0bd805fe20 tx=0x7f0bd8058400 comp rx=0 tx=0).stop 2024-04-25T10:14:56.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.474+0000 7f0bf13df740 1 -- 172.21.15.16:0/1134511600 shutdown_connections 2024-04-25T10:14:56.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.474+0000 7f0bf13df740 1 --2- 172.21.15.16:0/1134511600 >> [v2:172.21.15.16:6833/3763439204,v1:172.21.15.16:6835/3763439204] conn(0x55bc0a5d64a0 0x55bc0a5d6880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.474+0000 7f0bf13df740 1 --2- 172.21.15.16:0/1134511600 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5d4350 0x55bc0a5d7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.474+0000 7f0bf13df740 1 --2- 172.21.15.16:0/1134511600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5f2ad0 0x55bc0a5d3e10 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.474+0000 7f0bc6ffd640 0 client.9354 ms_handle_reset on v2:172.21.15.16:6833/3763439204 2024-04-25T10:14:56.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.474+0000 7f0bf13df740 1 --2- 172.21.15.16:0/1134511600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5d5230 0x55bc0a5d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.474+0000 7f0bf13df740 1 -- 172.21.15.16:0/1134511600 >> 172.21.15.16:0/1134511600 conn(0x55bc0a5aaf10 msgr2=0x55bc0a5f35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.474+0000 7f0bf13df740 1 -- 172.21.15.16:0/1134511600 shutdown_connections 2024-04-25T10:14:56.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.474+0000 7f0bf13df740 1 -- 172.21.15.16:0/1134511600 wait complete. 2024-04-25T10:14:56.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.474+0000 7f0bf13df740 20 client.9354 trim_cache size 0 max 16384 2024-04-25T10:14:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.490+0000 7f0bf13df740 1 Processor -- start 2024-04-25T10:14:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.490+0000 7f0bf13df740 1 -- start start 2024-04-25T10:14:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.490+0000 7f0bf13df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5d4350 0x55bc0a5f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.490+0000 7f0bf13df740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5d5230 0x55bc0a5a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.490+0000 7f0bf13df740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5f2ad0 0x55bc0a5d8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.490+0000 7f0bf13df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be80c96b0 con 0x55bc0a5d4350 2024-04-25T10:14:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.490+0000 7f0bf13df740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be80c8ab0 con 0x55bc0a5d5230 2024-04-25T10:14:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.490+0000 7f0bf13df740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be80c90b0 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.490+0000 7f0be7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5d4350 0x55bc0a5f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.490+0000 7f0be7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5d4350 0x55bc0a5f1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35534/0 (socket says 172.21.15.16:35534) 2024-04-25T10:14:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.490+0000 7f0be7fff640 1 -- 172.21.15.16:0/1638026244 learned_addr learned my addr 172.21.15.16:0/1638026244 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.491+0000 7f0be7fff640 1 -- 172.21.15.16:0/1638026244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5f2ad0 msgr2=0x55bc0a5d8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.491+0000 7f0be7fff640 1 --2- 172.21.15.16:0/1638026244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5f2ad0 0x55bc0a5d8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.491+0000 7f0be7fff640 1 -- 172.21.15.16:0/1638026244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5d5230 msgr2=0x55bc0a5a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.491+0000 7f0be7fff640 1 --2- 172.21.15.16:0/1638026244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5d5230 0x55bc0a5a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.491+0000 7f0be7fff640 1 -- 172.21.15.16:0/1638026244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0a5e6f20 con 0x55bc0a5d4350 2024-04-25T10:14:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.491+0000 7f0be7fff640 1 --2- 172.21.15.16:0/1638026244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5d4350 0x55bc0a5f1bb0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc021040 tx=0x7f0bdc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc12bb73f37a2cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.491+0000 7f0bcd7fa640 1 -- 172.21.15.16:0/1638026244 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc01a040 con 0x55bc0a5d4350 2024-04-25T10:14:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.491+0000 7f0bcd7fa640 1 -- 172.21.15.16:0/1638026244 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc024040 con 0x55bc0a5d4350 2024-04-25T10:14:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.491+0000 7f0bcd7fa640 1 -- 172.21.15.16:0/1638026244 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc00e850 con 0x55bc0a5d4350 2024-04-25T10:14:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.491+0000 7f0bf13df740 1 -- 172.21.15.16:0/1638026244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5d4350 msgr2=0x55bc0a5f1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.491+0000 7f0bf13df740 1 --2- 172.21.15.16:0/1638026244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5d4350 0x55bc0a5f1bb0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc021040 tx=0x7f0bdc013000 comp rx=0 tx=0).stop 2024-04-25T10:14:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.491+0000 7f0bf13df740 1 -- 172.21.15.16:0/1638026244 shutdown_connections 2024-04-25T10:14:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.491+0000 7f0bf13df740 1 --2- 172.21.15.16:0/1638026244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5f2ad0 0x55bc0a5d8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.491+0000 7f0bf13df740 1 --2- 172.21.15.16:0/1638026244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5d5230 0x55bc0a5a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.491+0000 7f0bf13df740 1 --2- 172.21.15.16:0/1638026244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5d4350 0x55bc0a5f1bb0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.491+0000 7f0bf13df740 1 -- 172.21.15.16:0/1638026244 >> 172.21.15.16:0/1638026244 conn(0x55bc0a5aaf10 msgr2=0x55bc0a5d7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.492+0000 7f0bf13df740 1 -- 172.21.15.16:0/1638026244 shutdown_connections 2024-04-25T10:14:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.492+0000 7f0bf13df740 1 -- 172.21.15.16:0/1638026244 wait complete. 2024-04-25T10:14:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.492+0000 7f0bf13df740 1 -- start start 2024-04-25T10:14:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.492+0000 7f0bf13df740 1 Processor -- start 2024-04-25T10:14:56.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.493+0000 7f0bf13df740 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5d4350 0x55bc0a601460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.493+0000 7f0bf13df740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5d5230 0x55bc0a5d8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.493+0000 7f0bf13df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5f2ad0 0x55bc0a5d8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.493+0000 7f0bf13df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0a601a00 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.493+0000 7f0bf13df740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0a5e7d10 con 0x55bc0a5d5230 2024-04-25T10:14:56.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.493+0000 7f0bf13df740 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc0a5e8a20 con 0x55bc0a5d4350 2024-04-25T10:14:56.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.493+0000 7f0bf13df740 10 client.0 _ll_register_callbacks cb 0x55bc0a5a5c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.493+0000 7f0becc81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5f2ad0 0x55bc0a5d8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.493+0000 7f0becc81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5f2ad0 0x55bc0a5d8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35550/0 (socket says 172.21.15.16:35550) 2024-04-25T10:14:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.493+0000 7f0becc81640 1 -- 172.21.15.16:0/2932969828 learned_addr learned my addr 172.21.15.16:0/2932969828 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.493+0000 7f0be77fe640 1 --2- 172.21.15.16:0/2932969828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5d5230 0x55bc0a5d8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.493+0000 7f0be7fff640 1 --2- 172.21.15.16:0/2932969828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5d4350 0x55bc0a601460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.493+0000 7f0becc81640 1 -- 172.21.15.16:0/2932969828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5d4350 msgr2=0x55bc0a601460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.493+0000 7f0becc81640 1 --2- 172.21.15.16:0/2932969828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5d4350 0x55bc0a601460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.493+0000 7f0becc81640 1 -- 172.21.15.16:0/2932969828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5d5230 msgr2=0x55bc0a5d8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.493+0000 7f0becc81640 1 --2- 172.21.15.16:0/2932969828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5d5230 0x55bc0a5d8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.493+0000 7f0becc81640 1 -- 172.21.15.16:0/2932969828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc0a5e9d10 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.493+0000 7f0be77fe640 1 --2- 172.21.15.16:0/2932969828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5d5230 0x55bc0a5d8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.494+0000 7f0becc81640 1 --2- 172.21.15.16:0/2932969828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5f2ad0 0x55bc0a5d8d90 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009e80 tx=0x7f0be0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69b70f5d3c05834 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.494+0000 7f0bc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-25T10:14:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.494+0000 7f0bc6ffd640 1 -- 172.21.15.16:0/2932969828 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0007e40 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.494+0000 7f0bc6ffd640 1 -- 172.21.15.16:0/2932969828 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f0be003d640 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.494+0000 7f0bf13df740 1 -- 172.21.15.16:0/2932969828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bdc024040 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.494+0000 7f0bc6ffd640 1 -- 172.21.15.16:0/2932969828 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0be0051050 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.494+0000 7f0bf13df740 10 client.9360 fetch_fsmap learned FSMap version 96 2024-04-25T10:14:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.494+0000 7f0bf13df740 1 -- 172.21.15.16:0/2932969828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x55bc0a65f410 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.494+0000 7f0bc6ffd640 1 -- 172.21.15.16:0/2932969828 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc024040 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.494+0000 7f0bc6ffd640 1 -- 172.21.15.16:0/2932969828 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 96) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0be0054020 con 0x55bc0a5f2ad0 2024-04-25T10:14:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.494+0000 7f0bf13df740 10 client.9360 fetch_fsmap finished waiting for FSMap version 96 2024-04-25T10:14:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.494+0000 7f0bf13df740 10 client.9360 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:14:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.494+0000 7f0bf13df740 20 client.9360 populate_metadata read hostname 'smithi016' 2024-04-25T10:14:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.494+0000 7f0bf13df740 1 --2- 172.21.15.16:0/2932969828 >> [v2:172.21.15.16:6833/3763439204,v1:172.21.15.16:6835/3763439204] conn(0x55bc0a65eeb0 0x55bc0a5a8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.494+0000 7f0bf13df740 4 client.9360 mds_command: new command op to 9252 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:14:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.494+0000 7f0bf13df740 1 -- 172.21.15.16:0/2932969828 --> [v2:172.21.15.16:6833/3763439204,v1:172.21.15.16:6835/3763439204] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bc0a51af50 con 0x55bc0a65eeb0 2024-04-25T10:14:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.495+0000 7f0be7fff640 1 --2- 172.21.15.16:0/2932969828 >> [v2:172.21.15.16:6833/3763439204,v1:172.21.15.16:6835/3763439204] conn(0x55bc0a65eeb0 0x55bc0a5a8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.495+0000 7f0be7fff640 1 --2- 172.21.15.16:0/2932969828 >> [v2:172.21.15.16:6833/3763439204,v1:172.21.15.16:6835/3763439204] conn(0x55bc0a65eeb0 0x55bc0a5a8de0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c0a4ac35f3e129d server_cookie=fb19eb7558ff042b in_seq=0 out_seq=0 2024-04-25T10:14:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.495+0000 7f0bc6ffd640 10 client.9360 ms_handle_connect on v2:172.21.15.16:6833/3763439204 2024-04-25T10:14:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.495+0000 7f0bc6ffd640 1 -- 172.21.15.16:0/2932969828 <== mds.0 v2:172.21.15.16:6833/3763439204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55bc0a51af50 con 0x55bc0a65eeb0 2024-04-25T10:14:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.495+0000 7f0bc6ffd640 10 client.9360 handle_command_reply: tid=0 2024-04-25T10:14:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.495+0000 7f0bf13df740 1 client.9360 shutdown 2024-04-25T10:14:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.495+0000 7f0bf13df740 1 -- 172.21.15.16:0/2932969828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5f2ad0 msgr2=0x55bc0a5d8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.495+0000 7f0bf13df740 1 --2- 172.21.15.16:0/2932969828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5f2ad0 0x55bc0a5d8d90 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009e80 tx=0x7f0be0004a40 comp rx=0 tx=0).stop 2024-04-25T10:14:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.495+0000 7f0bf13df740 1 -- 172.21.15.16:0/2932969828 shutdown_connections 2024-04-25T10:14:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.495+0000 7f0bf13df740 1 --2- 172.21.15.16:0/2932969828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc0a5f2ad0 0x55bc0a5d8d90 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.495+0000 7f0bf13df740 1 --2- 172.21.15.16:0/2932969828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc0a5d5230 0x55bc0a5d8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.495+0000 7f0bf13df740 1 --2- 172.21.15.16:0/2932969828 >> [v2:172.21.15.16:6833/3763439204,v1:172.21.15.16:6835/3763439204] conn(0x55bc0a65eeb0 0x55bc0a5a8de0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.495+0000 7f0bf13df740 1 --2- 172.21.15.16:0/2932969828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bc0a5d4350 0x55bc0a601460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.495+0000 7f0bc6ffd640 0 client.9360 ms_handle_reset on v2:172.21.15.16:6833/3763439204 2024-04-25T10:14:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.495+0000 7f0bf13df740 1 -- 172.21.15.16:0/2932969828 >> 172.21.15.16:0/2932969828 conn(0x55bc0a5aaf10 msgr2=0x55bc0a5eea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.495+0000 7f0bf13df740 1 -- 172.21.15.16:0/2932969828 shutdown_connections 2024-04-25T10:14:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.495+0000 7f0bf13df740 1 -- 172.21.15.16:0/2932969828 wait complete. 2024-04-25T10:14:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.496+0000 7f0bf13df740 20 client.9360 trim_cache size 0 max 16384 2024-04-25T10:14:56.506 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:14:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "34bc5eca-3a21-42a6-a26f-07da0905e8bc", 2024-04-25T10:14:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-04-25T10:14:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 9252, 2024-04-25T10:14:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-04-25T10:14:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-04-25T10:14:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-04-25T10:14:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 4.4376156839999998, 2024-04-25T10:14:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 96, 2024-04-25T10:14:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 119, 2024-04-25T10:14:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 119, 2024-04-25T10:14:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 6.4087736099999999 2024-04-25T10:14:56.507 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:14:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.496+0000 7f0bee6e4640 1 -- 172.21.15.16:0/2553476135 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0bd406b500 msgr2=0x7f0bd406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.496+0000 7f0bee6e4640 1 --2- 172.21.15.16:0/2553476135 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0bd406b500 0x7f0bd406d9c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00a950 tx=0x7f0bdc008040 comp rx=0 tx=0).stop 2024-04-25T10:14:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.496+0000 7f0bee6e4640 1 -- 172.21.15.16:0/2553476135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be80d22e0 msgr2=0x7f0be8151920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.496+0000 7f0bee6e4640 1 --2- 172.21.15.16:0/2553476135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be80d22e0 0x7f0be8151920 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800c4d0 tx=0x7f0bd800c990 comp rx=0 tx=0).stop 2024-04-25T10:14:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.496+0000 7f0bee6e4640 1 -- 172.21.15.16:0/2553476135 shutdown_connections 2024-04-25T10:14:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.496+0000 7f0bee6e4640 1 --2- 172.21.15.16:0/2553476135 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f0bd406b500 0x7f0bd406d9c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.496+0000 7f0bee6e4640 1 --2- 172.21.15.16:0/2553476135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0be815dec0 0x7f0be80c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.496+0000 7f0bee6e4640 1 --2- 172.21.15.16:0/2553476135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0be80d22e0 0x7f0be8151920 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.496+0000 7f0bee6e4640 1 --2- 172.21.15.16:0/2553476135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0be80d0b60 0x7f0be80cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.496+0000 7f0bee6e4640 1 -- 172.21.15.16:0/2553476135 >> 172.21.15.16:0/2553476135 conn(0x7f0be8072900 msgr2=0x7f0be8154f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.496+0000 7f0bee6e4640 1 -- 172.21.15.16:0/2553476135 shutdown_connections 2024-04-25T10:14:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.496+0000 7f0bee6e4640 1 -- 172.21.15.16:0/2553476135 wait complete. 2024-04-25T10:14:56.511 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:14:56.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.607+0000 7fca5fce4640 1 Processor -- start 2024-04-25T10:14:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.608+0000 7fca5fce4640 1 -- start start 2024-04-25T10:14:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.608+0000 7fca5fce4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca5815dd50 0x7fca580d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.608+0000 7fca5fce4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca580d2440 con 0x7fca580ceeb0 2024-04-25T10:14:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.608+0000 7fca5fce4640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca580d2720 con 0x7fca580b9460 2024-04-25T10:14:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.608+0000 7fca5fce4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca580d2a00 con 0x7fca5815dd50 2024-04-25T10:14:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5e281640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fca580ceeb0 0x7fca580b6c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55688/0 (socket says 172.21.15.16:55688) 2024-04-25T10:14:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5e281640 1 -- 172.21.15.16:0/3634015035 learned_addr learned my addr 172.21.15.16:0/3634015035 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5da80640 1 --2- 172.21.15.16:0/3634015035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca5815dd50 0x7fca580d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5ca7e640 1 -- 172.21.15.16:0/3634015035 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052366592 0 0) 0x7fca580d2440 con 0x7fca580ceeb0 2024-04-25T10:14:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5ca7e640 1 -- 172.21.15.16:0/3634015035 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca44003520 con 0x7fca580ceeb0 2024-04-25T10:14:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5ca7e640 1 -- 172.21.15.16:0/3634015035 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1580584996 0 0) 0x7fca580d2720 con 0x7fca580b9460 2024-04-25T10:14:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5ca7e640 1 -- 172.21.15.16:0/3634015035 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca580d2440 con 0x7fca580b9460 2024-04-25T10:14:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5ca7e640 1 -- 172.21.15.16:0/3634015035 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2583198721 0 0) 0x7fca44003520 con 0x7fca580ceeb0 2024-04-25T10:14:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5ca7e640 1 -- 172.21.15.16:0/3634015035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca5815dd50 msgr2=0x7fca580d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5ca7e640 1 --2- 172.21.15.16:0/3634015035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca5815dd50 0x7fca580d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5ca7e640 1 -- 172.21.15.16:0/3634015035 >> v1:172.21.15.186:6789/0 conn(0x7fca580b9460 legacy=0x7fca580ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5ca7e640 1 -- 172.21.15.16:0/3634015035 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca580d2ce0 con 0x7fca580ceeb0 2024-04-25T10:14:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5ca7e640 1 -- 172.21.15.16:0/3634015035 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fca500019f0 con 0x7fca580ceeb0 2024-04-25T10:14:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5ca7e640 1 -- 172.21.15.16:0/3634015035 >> v1:172.21.15.16:6789/0 conn(0x7fca580ceeb0 legacy=0x7fca580b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5ca7e640 1 --2- 172.21.15.16:0/3634015035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca44003b70 0x7fca44003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5ca7e640 1 --2- 172.21.15.16:0/3634015035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca44004800 0x7fca44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5ca7e640 1 --2- 172.21.15.16:0/3634015035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca5815dd50 0x7fca440051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5ca7e640 1 -- 172.21.15.16:0/3634015035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca580d2a00 con 0x7fca44004800 2024-04-25T10:14:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5ca7e640 1 -- 172.21.15.16:0/3634015035 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4400bad0 con 0x7fca44003b70 2024-04-25T10:14:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5ca7e640 1 -- 172.21.15.16:0/3634015035 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4400bcd0 con 0x7fca5815dd50 2024-04-25T10:14:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5d27f640 1 --2- 172.21.15.16:0/3634015035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca44004800 0x7fca44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5da80640 1 --2- 172.21.15.16:0/3634015035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca44003b70 0x7fca44003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.609+0000 7fca5e281640 1 --2- 172.21.15.16:0/3634015035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca5815dd50 0x7fca440051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.610+0000 7fca5d27f640 1 -- 172.21.15.16:0/3634015035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca5815dd50 msgr2=0x7fca440051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.610+0000 7fca5d27f640 1 --2- 172.21.15.16:0/3634015035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca5815dd50 0x7fca440051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.610+0000 7fca5d27f640 1 -- 172.21.15.16:0/3634015035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca44003b70 msgr2=0x7fca44003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.610+0000 7fca5d27f640 1 --2- 172.21.15.16:0/3634015035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca44003b70 0x7fca44003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.610+0000 7fca5d27f640 1 -- 172.21.15.16:0/3634015035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca44003520 con 0x7fca44004800 2024-04-25T10:14:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.610+0000 7fca5e281640 1 --2- 172.21.15.16:0/3634015035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca5815dd50 0x7fca440051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.610+0000 7fca5d27f640 1 --2- 172.21.15.16:0/3634015035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca44004800 0x7fca44004c80 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fca480018d0 tx=0x7fca44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5ddcdd79288889 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.610+0000 7fca5ca7e640 1 -- 172.21.15.16:0/3634015035 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca480014f0 con 0x7fca44004800 2024-04-25T10:14:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.610+0000 7fca5ca7e640 1 -- 172.21.15.16:0/3634015035 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca480052d0 con 0x7fca44004800 2024-04-25T10:14:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.610+0000 7fca5ca7e640 1 -- 172.21.15.16:0/3634015035 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4802aca0 con 0x7fca44004800 2024-04-25T10:14:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.611+0000 7fca5fce4640 1 -- 172.21.15.16:0/3634015035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca44004800 msgr2=0x7fca44004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.611+0000 7fca5fce4640 1 --2- 172.21.15.16:0/3634015035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca44004800 0x7fca44004c80 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fca480018d0 tx=0x7fca44005830 comp rx=0 tx=0).stop 2024-04-25T10:14:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.611+0000 7fca5e281640 1 -- 172.21.15.16:0/3634015035 reap_dead start 2024-04-25T10:14:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.611+0000 7fca5fce4640 1 -- 172.21.15.16:0/3634015035 shutdown_connections 2024-04-25T10:14:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.611+0000 7fca5fce4640 1 -- 172.21.15.16:0/3634015035 >> 172.21.15.16:0/3634015035 conn(0x7fca580722d0 msgr2=0x7fca580726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.611+0000 7fca5fce4640 1 -- 172.21.15.16:0/3634015035 shutdown_connections 2024-04-25T10:14:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.611+0000 7fca5fce4640 1 -- 172.21.15.16:0/3634015035 wait complete. 2024-04-25T10:14:56.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.612+0000 7fca5fce4640 1 Processor -- start 2024-04-25T10:14:56.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.612+0000 7fca5fce4640 1 -- start start 2024-04-25T10:14:56.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.613+0000 7fca5fce4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca44004800 0x7fca5815e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.613+0000 7fca5fce4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca580ccb30 0x7fca58148110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.613+0000 7fca5fce4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca580ccfb0 0x7fca58148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.613+0000 7fca5fce4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca580d3660 con 0x7fca580ccfb0 2024-04-25T10:14:56.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.613+0000 7fca5fce4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca580d34e0 con 0x7fca44004800 2024-04-25T10:14:56.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.613+0000 7fca5fce4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca580d37e0 con 0x7fca580ccb30 2024-04-25T10:14:56.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.613+0000 7fca5e281640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca580ccfb0 0x7fca58148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.613+0000 7fca5e281640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca580ccfb0 0x7fca58148650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35570/0 (socket says 172.21.15.16:35570) 2024-04-25T10:14:56.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.613+0000 7fca5e281640 1 -- 172.21.15.16:0/2776878465 learned_addr learned my addr 172.21.15.16:0/2776878465 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:56.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.613+0000 7fca5da80640 1 --2- 172.21.15.16:0/2776878465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca44004800 0x7fca5815e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.613+0000 7fca5d27f640 1 --2- 172.21.15.16:0/2776878465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca580ccb30 0x7fca58148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.613+0000 7fca5e281640 1 -- 172.21.15.16:0/2776878465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca580ccb30 msgr2=0x7fca58148110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.613+0000 7fca5e281640 1 --2- 172.21.15.16:0/2776878465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca580ccb30 0x7fca58148110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.613+0000 7fca5e281640 1 -- 172.21.15.16:0/2776878465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca44004800 msgr2=0x7fca5815e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.613+0000 7fca5e281640 1 --2- 172.21.15.16:0/2776878465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca44004800 0x7fca5815e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.614+0000 7fca5e281640 1 -- 172.21.15.16:0/2776878465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca580cadf0 con 0x7fca580ccfb0 2024-04-25T10:14:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.614+0000 7fca5da80640 1 --2- 172.21.15.16:0/2776878465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca44004800 0x7fca5815e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.614+0000 7fca5d27f640 1 --2- 172.21.15.16:0/2776878465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca580ccb30 0x7fca58148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.614+0000 7fca5e281640 1 --2- 172.21.15.16:0/2776878465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca580ccfb0 0x7fca58148650 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fca50000f80 tx=0x7fca50007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.614+0000 7fca4e7fc640 1 -- 172.21.15.16:0/2776878465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5000e9e0 con 0x7fca580ccfb0 2024-04-25T10:14:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.614+0000 7fca4e7fc640 1 -- 172.21.15.16:0/2776878465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca50018070 con 0x7fca580ccfb0 2024-04-25T10:14:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.614+0000 7fca5fce4640 1 -- 172.21.15.16:0/2776878465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca580cb0d0 con 0x7fca580ccfb0 2024-04-25T10:14:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.614+0000 7fca4e7fc640 1 -- 172.21.15.16:0/2776878465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5000ede0 con 0x7fca580ccfb0 2024-04-25T10:14:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.614+0000 7fca5fce4640 1 -- 172.21.15.16:0/2776878465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca580cda10 con 0x7fca580ccfb0 2024-04-25T10:14:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.615+0000 7fca5fce4640 1 -- 172.21.15.16:0/2776878465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca58067850 con 0x7fca580ccfb0 2024-04-25T10:14:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.615+0000 7fca4e7fc640 1 -- 172.21.15.16:0/2776878465 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca5001d460 con 0x7fca580ccfb0 2024-04-25T10:14:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.616+0000 7fca4e7fc640 1 --2- 172.21.15.16:0/2776878465 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fca3c06b500 0x7fca3c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.616+0000 7fca4e7fc640 1 -- 172.21.15.16:0/2776878465 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fca5008a420 con 0x7fca580ccfb0 2024-04-25T10:14:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.616+0000 7fca5da80640 1 --2- 172.21.15.16:0/2776878465 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fca3c06b500 0x7fca3c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.617+0000 7fca5da80640 1 --2- 172.21.15.16:0/2776878465 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fca3c06b500 0x7fca3c06d9c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fca540099e0 tx=0x7fca54008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.620+0000 7fca4e7fc640 1 -- 172.21.15.16:0/2776878465 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca50059240 con 0x7fca580ccfb0 2024-04-25T10:14:56.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.844+0000 7fca5fce4640 1 -- 172.21.15.16:0/2776878465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fca580749b0 con 0x7fca580ccfb0 2024-04-25T10:14:56.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.844+0000 7fca4e7fc640 1 -- 172.21.15.16:0/2776878465 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v96) v1 ==== 53+0+73 (secure 0 0 0) 0x7fca50058b40 con 0x7fca580ccfb0 2024-04-25T10:14:56.848 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:14:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.846+0000 7fca5fce4640 1 -- 172.21.15.16:0/2776878465 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fca3c06b500 msgr2=0x7fca3c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.846+0000 7fca5fce4640 1 --2- 172.21.15.16:0/2776878465 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fca3c06b500 0x7fca3c06d9c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fca540099e0 tx=0x7fca54008040 comp rx=0 tx=0).stop 2024-04-25T10:14:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.846+0000 7fca5fce4640 1 -- 172.21.15.16:0/2776878465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca580ccfb0 msgr2=0x7fca58148650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.846+0000 7fca5fce4640 1 --2- 172.21.15.16:0/2776878465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca580ccfb0 0x7fca58148650 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fca50000f80 tx=0x7fca50007b00 comp rx=0 tx=0).stop 2024-04-25T10:14:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.846+0000 7fca5fce4640 1 -- 172.21.15.16:0/2776878465 shutdown_connections 2024-04-25T10:14:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.847+0000 7fca5fce4640 1 --2- 172.21.15.16:0/2776878465 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fca3c06b500 0x7fca3c06d9c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.847+0000 7fca5fce4640 1 --2- 172.21.15.16:0/2776878465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca580ccfb0 0x7fca58148650 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.847+0000 7fca5fce4640 1 --2- 172.21.15.16:0/2776878465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca580ccb30 0x7fca58148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.847+0000 7fca5fce4640 1 --2- 172.21.15.16:0/2776878465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca44004800 0x7fca5815e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.847+0000 7fca5fce4640 1 -- 172.21.15.16:0/2776878465 >> 172.21.15.16:0/2776878465 conn(0x7fca580722d0 msgr2=0x7fca5815c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.847+0000 7fca5fce4640 1 -- 172.21.15.16:0/2776878465 shutdown_connections 2024-04-25T10:14:56.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:56.847+0000 7fca5fce4640 1 -- 172.21.15.16:0/2776878465 wait complete. 2024-04-25T10:14:56.861 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:14:56.862 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:14:56.862 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-04-25T10:14:56.862 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:14:56.862 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:56.862 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:14:56.862 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:14:56.862 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:14:56.862 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:56.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:56.862 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-04-25T10:14:56.878 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:14:56.878 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:14:56.878 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:14:56.878 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:56.878 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-25T10:14:56.879 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:56.879 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:14:56.879 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:56.879 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-04-25T10:14:56.879 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:14:56.879 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:56.879 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-04-25T10:14:56.879 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:14:56.879 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:56.879 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-04-25T10:14:56.879 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:14:56.879 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42063sec preferred_lft 42063sec 2024-04-25T10:14:56.879 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-25T10:14:56.879 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:56.880 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:14:56.880 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:56.880 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-04-25T10:14:56.880 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:14:56.880 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:56.880 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:14:56.880 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:56.880 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f06b:fdff:fed4:a335/64 scope link 2024-04-25T10:14:56.880 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:56.880 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:56.880 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 92:ad:4f:91:3c:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:56.880 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::90ad:4fff:fe91:3c33/64 scope link 2024-04-25T10:14:56.880 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:56.880 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:56.880 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether be:14:4a:3c:16:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:56.881 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::bc14:4aff:fe3c:1605/64 scope link 2024-04-25T10:14:56.881 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:56.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:56.881 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:14:56.941 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:14:56.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:56.943 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-04-25T10:14:57.000 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-04-25T10:14:57.000 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:14:57.000 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:14:57.000 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:14:57.000 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:14:57.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:57.000 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-04-25T10:14:57.000 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-25T10:14:57.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:14:57.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:14:57.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:14:57.001 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-25T10:14:57.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:57.086 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:14:57.101 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:14:57.101 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:14:57.101 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:57.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:57.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-04-25T10:14:57.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:57.165 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:57.221 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:57.221 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:57.221 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:57.275 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:14:57.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:57.276 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:57.357 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:57.358 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:14:57.358 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:14:57.366 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:57.366 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:14:57.366 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:14:57.366 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:14:57.366 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:14:57.372 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:57.372 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:14:57.372 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-04-25T10:14:57.372 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.186:3300,172.21.15.186:3301" 2024-04-25T10:14:57.372 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:14:57.372 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790" 2024-04-25T10:14:57.372 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2024-04-25T10:14:57.372 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=34bc5eca-3a21-42a6-a26f-07da0905e8bc" will pass to kernel 2024-04-25T10:14:57.373 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:14:57.373 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:14:57.373 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:14:57.373 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:14:57.373 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:14:57.374 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:14:57.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:57.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:57.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:57.439 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:14:57.495 INFO:teuthology.orchestra.run.smithi016.stdout:client8113 2024-04-25T10:14:57.496 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:14:57.496 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:57.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:57.551 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:14:57.611 INFO:teuthology.orchestra.run.smithi016.stdout:./ 2024-04-25T10:14:57.612 INFO:teuthology.orchestra.run.smithi016.stdout:./parent 2024-04-25T10:14:57.612 INFO:teuthology.orchestra.run.smithi016.stdout:./parent/flushed 2024-04-25T10:14:57.612 INFO:teuthology.orchestra.run.smithi016.stdout:./parent/flushed/charlie 2024-04-25T10:14:57.612 INFO:teuthology.orchestra.run.smithi016.stdout:./parent/flushed/alpha 2024-04-25T10:14:57.612 INFO:teuthology.orchestra.run.smithi016.stdout:./parent/flushed/bravo 2024-04-25T10:14:57.612 INFO:teuthology.orchestra.run.smithi016.stdout:./parent/unflushed 2024-04-25T10:14:57.612 INFO:teuthology.orchestra.run.smithi016.stdout:./parent/unflushed/jfile 2024-04-25T10:14:57.612 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:57.612 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:57.613 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:57.613 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:57.613 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:14:57.613 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:57.717 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-25T10:14:57.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:57.726 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:57.726 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:57.727 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:57.727 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:57.727 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T10:14:57.727 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:57.834 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627776 2024-04-25T10:14:57.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:57.843 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:57.843 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:57.843 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:57.843 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:57.843 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T10:14:57.843 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:57.950 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627777 2024-04-25T10:14:57.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:57.959 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:57.959 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:57.959 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:57.959 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:57.959 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T10:14:57.959 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:58.066 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627780 2024-04-25T10:14:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:58.075 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:58.075 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:58.075 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:58.075 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:58.075 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T10:14:58.075 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:58.181 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627778 2024-04-25T10:14:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:58.190 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:58.190 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:58.190 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:58.190 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:58.190 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-25T10:14:58.190 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:58.297 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627779 2024-04-25T10:14:58.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:58.307 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:58.307 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:58.307 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:58.307 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:58.307 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T10:14:58.307 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:58.414 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627781 2024-04-25T10:14:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:58.423 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-04-25T10:14:58.423 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-25T10:14:58.423 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-04-25T10:14:58.423 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-25T10:14:58.423 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T10:14:58.423 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-25T10:14:58.529 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627782 2024-04-25T10:14:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:58.538 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:14:58.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:58.621 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:58.663 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:14:58.663 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:14:58.663 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:58.685 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:14:58.685 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:58.741 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.880+0000 7f46e2542640 1 Processor -- start 2024-04-25T10:14:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.881+0000 7f46e2542640 1 -- start start 2024-04-25T10:14:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.881+0000 7f46e2542640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f46dc15dce0 0x7f46dc0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.881+0000 7f46e2542640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46dc0d35b0 0x7f46dc0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.881+0000 7f46e2542640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f46dc0d1f90 0x7f46dc0b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.881+0000 7f46e2542640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46dc15e0c0 con 0x7f46dc0d1f90 2024-04-25T10:14:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.881+0000 7f46e2542640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46dc0d2510 con 0x7f46dc15dce0 2024-04-25T10:14:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.881+0000 7f46e2542640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc0d27f0 con 0x7f46dc0d35b0 2024-04-25T10:14:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.881+0000 7f46e0adf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f46dc0d1f90 0x7f46dc0b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.881+0000 7f46e0adf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f46dc0d1f90 0x7f46dc0b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35572/0 (socket says 172.21.15.16:35572) 2024-04-25T10:14:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.881+0000 7f46e0adf640 1 -- 172.21.15.16:0/2361100571 learned_addr learned my addr 172.21.15.16:0/2361100571 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.882+0000 7f46e0adf640 1 -- 172.21.15.16:0/2361100571 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46dc0d35b0 msgr2=0x7f46dc0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.882+0000 7f46e0adf640 1 --2- 172.21.15.16:0/2361100571 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46dc0d35b0 0x7f46dc0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.882+0000 7f46e0adf640 1 -- 172.21.15.16:0/2361100571 >> v2:172.21.15.186:3300/0 conn(0x7f46dc15dce0 msgr2=0x7f46dc0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.882+0000 7f46e0adf640 1 --2- 172.21.15.16:0/2361100571 >> v2:172.21.15.186:3300/0 conn(0x7f46dc15dce0 0x7f46dc0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.882+0000 7f46e0adf640 1 -- 172.21.15.16:0/2361100571 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46dc0d2ad0 con 0x7f46dc0d1f90 2024-04-25T10:14:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.882+0000 7f46e0adf640 1 --2- 172.21.15.16:0/2361100571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46dc0d1f90 0x7f46dc0b5990 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f46dc0ce220 tx=0x7f46d402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aef75b2d274029e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.883+0000 7f46daffd640 1 -- 172.21.15.16:0/2361100571 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d4036070 con 0x7f46dc0d1f90 2024-04-25T10:14:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.883+0000 7f46daffd640 1 -- 172.21.15.16:0/2361100571 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d4031600 con 0x7f46dc0d1f90 2024-04-25T10:14:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.883+0000 7f46daffd640 1 -- 172.21.15.16:0/2361100571 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d40319d0 con 0x7f46dc0d1f90 2024-04-25T10:14:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.883+0000 7f46e2542640 1 -- 172.21.15.16:0/2361100571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46dc0d1f90 msgr2=0x7f46dc0b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.883+0000 7f46e2542640 1 --2- 172.21.15.16:0/2361100571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46dc0d1f90 0x7f46dc0b5990 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f46dc0ce220 tx=0x7f46d402a610 comp rx=0 tx=0).stop 2024-04-25T10:14:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.883+0000 7f46e2542640 1 -- 172.21.15.16:0/2361100571 shutdown_connections 2024-04-25T10:14:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.883+0000 7f46e2542640 1 --2- 172.21.15.16:0/2361100571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46dc0d1f90 0x7f46dc0b5990 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.883+0000 7f46e2542640 1 --2- 172.21.15.16:0/2361100571 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46dc0d35b0 0x7f46dc0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.883+0000 7f46e2542640 1 --2- 172.21.15.16:0/2361100571 >> v2:172.21.15.186:3300/0 conn(0x7f46dc15dce0 0x7f46dc0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.883+0000 7f46e2542640 1 -- 172.21.15.16:0/2361100571 >> 172.21.15.16:0/2361100571 conn(0x7f46dc072900 msgr2=0x7f46dc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.883+0000 7f46e2542640 1 -- 172.21.15.16:0/2361100571 shutdown_connections 2024-04-25T10:14:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.883+0000 7f46e2542640 1 -- 172.21.15.16:0/2361100571 wait complete. 2024-04-25T10:14:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.884+0000 7f46e2542640 1 Processor -- start 2024-04-25T10:14:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.885+0000 7f46e2542640 1 -- start start 2024-04-25T10:14:58.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.885+0000 7f46e2542640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46dc0d1f90 0x7f46dc0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.885+0000 7f46e2542640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46dc0d35b0 0x7f46dc14d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.885+0000 7f46e2542640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46dc15dce0 0x7f46dc14dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.885+0000 7f46e2542640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc0b9820 con 0x7f46dc15dce0 2024-04-25T10:14:58.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.885+0000 7f46e2542640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc0d3090 con 0x7f46dc0d1f90 2024-04-25T10:14:58.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.885+0000 7f46e2542640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc0b99a0 con 0x7f46dc0d35b0 2024-04-25T10:14:58.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.885+0000 7f46e0adf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46dc15dce0 0x7f46dc14dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.885+0000 7f46e0adf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46dc15dce0 0x7f46dc14dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35588/0 (socket says 172.21.15.16:35588) 2024-04-25T10:14:58.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.885+0000 7f46e0adf640 1 -- 172.21.15.16:0/277623909 learned_addr learned my addr 172.21.15.16:0/277623909 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.886+0000 7f46dbfff640 1 --2- 172.21.15.16:0/277623909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46dc0d1f90 0x7f46dc0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.886+0000 7f46db7fe640 1 --2- 172.21.15.16:0/277623909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46dc0d35b0 0x7f46dc14d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.886+0000 7f46e0adf640 1 -- 172.21.15.16:0/277623909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46dc0d35b0 msgr2=0x7f46dc14d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.886+0000 7f46e0adf640 1 --2- 172.21.15.16:0/277623909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46dc0d35b0 0x7f46dc14d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.886+0000 7f46e0adf640 1 -- 172.21.15.16:0/277623909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46dc0d1f90 msgr2=0x7f46dc0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.886+0000 7f46e0adf640 1 --2- 172.21.15.16:0/277623909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46dc0d1f90 0x7f46dc0ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.886+0000 7f46e0adf640 1 -- 172.21.15.16:0/277623909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46dc14d2e0 con 0x7f46dc15dce0 2024-04-25T10:14:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.886+0000 7f46dbfff640 1 --2- 172.21.15.16:0/277623909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46dc0d1f90 0x7f46dc0ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.886+0000 7f46db7fe640 1 --2- 172.21.15.16:0/277623909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46dc0d35b0 0x7f46dc14d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.886+0000 7f46e0adf640 1 --2- 172.21.15.16:0/277623909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46dc15dce0 0x7f46dc14dd90 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f46d4034040 tx=0x7f46d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.886+0000 7f46d8ff9640 1 -- 172.21.15.16:0/277623909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d4041020 con 0x7f46dc15dce0 2024-04-25T10:14:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.886+0000 7f46d8ff9640 1 -- 172.21.15.16:0/277623909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d4007780 con 0x7f46dc15dce0 2024-04-25T10:14:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.886+0000 7f46d8ff9640 1 -- 172.21.15.16:0/277623909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d4036040 con 0x7f46dc15dce0 2024-04-25T10:14:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.886+0000 7f46e2542640 1 -- 172.21.15.16:0/277623909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46dc0cbd30 con 0x7f46dc15dce0 2024-04-25T10:14:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.886+0000 7f46e2542640 1 -- 172.21.15.16:0/277623909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46dc0cc2f0 con 0x7f46dc15dce0 2024-04-25T10:14:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.887+0000 7f46e2542640 1 -- 172.21.15.16:0/277623909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46dc0749b0 con 0x7f46dc15dce0 2024-04-25T10:14:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.887+0000 7f46d8ff9640 1 -- 172.21.15.16:0/277623909 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f46d4044020 con 0x7f46dc15dce0 2024-04-25T10:14:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.888+0000 7f46d8ff9640 1 --2- 172.21.15.16:0/277623909 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f46c806b4d0 0x7f46c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.888+0000 7f46d8ff9640 1 -- 172.21.15.16:0/277623909 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f46d40ab0e0 con 0x7f46dc15dce0 2024-04-25T10:14:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.888+0000 7f46dbfff640 1 --2- 172.21.15.16:0/277623909 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f46c806b4d0 0x7f46c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.889+0000 7f46dbfff640 1 --2- 172.21.15.16:0/277623909 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f46c806b4d0 0x7f46c806d990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f46cc006fd0 tx=0x7f46cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:58.892+0000 7f46d8ff9640 1 -- 172.21.15.16:0/277623909 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f46d404b030 con 0x7f46dc15dce0 2024-04-25T10:14:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.072+0000 7f46e2542640 1 -- 172.21.15.16:0/277623909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46dc06b080 con 0x7f46dc15dce0 2024-04-25T10:14:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.074+0000 7f46d8ff9640 1 -- 172.21.15.16:0/277623909 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+17026 (secure 0 0 0) 0x7f46d4079f30 con 0x7f46dc15dce0 2024-04-25T10:14:59.078 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:14:59.078 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":119,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:14:54.519874+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:25.025480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:25.726747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1d","mappings":[{"from":4,"to":3}]},{"pgid":"17.1e","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/2697110279":"2024-04-26T10:14:50.050260+0000","172.21.15.16:6833/2697110279":"2024-04-26T10:14:50.050260+0000","172.21.15.16:6835/123396308":"2024-04-26T10:14:39.639508+0000","172.21.15.16:6833/123396308":"2024-04-26T10:14:39.639508+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.076+0000 7f46e2542640 1 -- 172.21.15.16:0/277623909 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f46c806b4d0 msgr2=0x7f46c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.076+0000 7f46e2542640 1 --2- 172.21.15.16:0/277623909 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f46c806b4d0 0x7f46c806d990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f46cc006fd0 tx=0x7f46cc008040 comp rx=0 tx=0).stop 2024-04-25T10:14:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.076+0000 7f46e2542640 1 -- 172.21.15.16:0/277623909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46dc15dce0 msgr2=0x7f46dc14dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.076+0000 7f46e2542640 1 --2- 172.21.15.16:0/277623909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46dc15dce0 0x7f46dc14dd90 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f46d4034040 tx=0x7f46d4004a40 comp rx=0 tx=0).stop 2024-04-25T10:14:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.076+0000 7f46e2542640 1 -- 172.21.15.16:0/277623909 shutdown_connections 2024-04-25T10:14:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.076+0000 7f46e2542640 1 --2- 172.21.15.16:0/277623909 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f46c806b4d0 0x7f46c806d990 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.076+0000 7f46e2542640 1 --2- 172.21.15.16:0/277623909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46dc15dce0 0x7f46dc14dd90 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.076+0000 7f46e2542640 1 --2- 172.21.15.16:0/277623909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46dc0d35b0 0x7f46dc14d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.076+0000 7f46e2542640 1 --2- 172.21.15.16:0/277623909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46dc0d1f90 0x7f46dc0ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.076+0000 7f46e2542640 1 -- 172.21.15.16:0/277623909 >> 172.21.15.16:0/277623909 conn(0x7f46dc072900 msgr2=0x7f46dc0d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.076+0000 7f46e2542640 1 -- 172.21.15.16:0/277623909 shutdown_connections 2024-04-25T10:14:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.076+0000 7f46e2542640 1 -- 172.21.15.16:0/277623909 wait complete. 2024-04-25T10:14:59.091 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:14:25.025480+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:14:25.726747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:59.091 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.191+0000 7f319694c640 1 Processor -- start 2024-04-25T10:14:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.191+0000 7f319694c640 1 -- start start 2024-04-25T10:14:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.192+0000 7f319694c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31900d3b90 0x7f31900d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.192+0000 7f319694c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f31900d0f60 0x7f31900b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.192+0000 7f319694c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31900d4090 con 0x7f319015dca0 2024-04-25T10:14:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.192+0000 7f319694c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31900b6b20 con 0x7f31900d0f60 2024-04-25T10:14:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.192+0000 7f319694c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31900d4370 con 0x7f31900d3b90 2024-04-25T10:14:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.192+0000 7f3194ee9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f31900d0f60 0x7f31900b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.192+0000 7f3194ee9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f31900d0f60 0x7f31900b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35598/0 (socket says 172.21.15.16:35598) 2024-04-25T10:14:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.192+0000 7f3194ee9640 1 -- 172.21.15.16:0/1303044815 learned_addr learned my addr 172.21.15.16:0/1303044815 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.192+0000 7f318f7fe640 1 --2- 172.21.15.16:0/1303044815 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31900d3b90 0x7f31900d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.193+0000 7f3194ee9640 1 -- 172.21.15.16:0/1303044815 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31900d3b90 msgr2=0x7f31900d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.193+0000 7f3194ee9640 1 --2- 172.21.15.16:0/1303044815 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31900d3b90 0x7f31900d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.193+0000 7f3194ee9640 1 -- 172.21.15.16:0/1303044815 >> v1:172.21.15.186:6789/0 conn(0x7f319015dca0 legacy=0x7f31900ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.193+0000 7f3194ee9640 1 -- 172.21.15.16:0/1303044815 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31900d4650 con 0x7f31900d0f60 2024-04-25T10:14:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.193+0000 7f3194ee9640 1 --2- 172.21.15.16:0/1303044815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31900d0f60 0x7f31900b2500 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f3188009e50 tx=0x7f318802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76af947c27fb6bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.193+0000 7f318effd640 1 -- 172.21.15.16:0/1303044815 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188036070 con 0x7f31900d0f60 2024-04-25T10:14:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.193+0000 7f318effd640 1 -- 172.21.15.16:0/1303044815 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3188031600 con 0x7f31900d0f60 2024-04-25T10:14:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.194+0000 7f318effd640 1 -- 172.21.15.16:0/1303044815 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31880319d0 con 0x7f31900d0f60 2024-04-25T10:14:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.194+0000 7f319694c640 1 -- 172.21.15.16:0/1303044815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31900d0f60 msgr2=0x7f31900b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.194+0000 7f319694c640 1 --2- 172.21.15.16:0/1303044815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31900d0f60 0x7f31900b2500 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f3188009e50 tx=0x7f318802a710 comp rx=0 tx=0).stop 2024-04-25T10:14:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.194+0000 7f319694c640 1 -- 172.21.15.16:0/1303044815 shutdown_connections 2024-04-25T10:14:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.194+0000 7f319694c640 1 --2- 172.21.15.16:0/1303044815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31900d0f60 0x7f31900b2500 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.194+0000 7f319694c640 1 --2- 172.21.15.16:0/1303044815 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31900d3b90 0x7f31900d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.194+0000 7f319694c640 1 -- 172.21.15.16:0/1303044815 >> 172.21.15.16:0/1303044815 conn(0x7f31900726c0 msgr2=0x7f3190072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.194+0000 7f319694c640 1 -- 172.21.15.16:0/1303044815 shutdown_connections 2024-04-25T10:14:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.194+0000 7f319694c640 1 -- 172.21.15.16:0/1303044815 wait complete. 2024-04-25T10:14:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.195+0000 7f319694c640 1 Processor -- start 2024-04-25T10:14:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.195+0000 7f319694c640 1 -- start start 2024-04-25T10:14:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.196+0000 7f319694c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31900d0f60 0x7f31900aada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.196+0000 7f319694c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31900d3b90 0x7f319014c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.196+0000 7f319694c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f319015dca0 0x7f31900ad730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.196+0000 7f319694c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319015e080 con 0x7f319015dca0 2024-04-25T10:14:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.196+0000 7f319694c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31900cf050 con 0x7f31900d0f60 2024-04-25T10:14:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.196+0000 7f319694c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319015e200 con 0x7f31900d3b90 2024-04-25T10:14:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.196+0000 7f3194ee9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f319015dca0 0x7f31900ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.196+0000 7f3194ee9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f319015dca0 0x7f31900ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35606/0 (socket says 172.21.15.16:35606) 2024-04-25T10:14:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.196+0000 7f3194ee9640 1 -- 172.21.15.16:0/1014030114 learned_addr learned my addr 172.21.15.16:0/1014030114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.196+0000 7f318f7fe640 1 --2- 172.21.15.16:0/1014030114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31900d3b90 0x7f319014c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.196+0000 7f318ffff640 1 --2- 172.21.15.16:0/1014030114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31900d0f60 0x7f31900aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.197+0000 7f3194ee9640 1 -- 172.21.15.16:0/1014030114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31900d3b90 msgr2=0x7f319014c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.197+0000 7f3194ee9640 1 --2- 172.21.15.16:0/1014030114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31900d3b90 0x7f319014c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.197+0000 7f3194ee9640 1 -- 172.21.15.16:0/1014030114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31900d0f60 msgr2=0x7f31900aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.197+0000 7f3194ee9640 1 --2- 172.21.15.16:0/1014030114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31900d0f60 0x7f31900aada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.197+0000 7f3194ee9640 1 -- 172.21.15.16:0/1014030114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319014cbe0 con 0x7f319015dca0 2024-04-25T10:14:59.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.197+0000 7f318ffff640 1 --2- 172.21.15.16:0/1014030114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31900d0f60 0x7f31900aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:59.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.197+0000 7f3194ee9640 1 --2- 172.21.15.16:0/1014030114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f319015dca0 0x7f31900ad730 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f3188034040 tx=0x7f3188004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.197+0000 7f318cff9640 1 -- 172.21.15.16:0/1014030114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188036050 con 0x7f319015dca0 2024-04-25T10:14:59.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.197+0000 7f318cff9640 1 -- 172.21.15.16:0/1014030114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3188042070 con 0x7f319015dca0 2024-04-25T10:14:59.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.197+0000 7f319694c640 1 -- 172.21.15.16:0/1014030114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319014cec0 con 0x7f319015dca0 2024-04-25T10:14:59.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.197+0000 7f318cff9640 1 -- 172.21.15.16:0/1014030114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188009510 con 0x7f319015dca0 2024-04-25T10:14:59.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.197+0000 7f319694c640 1 -- 172.21.15.16:0/1014030114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f319014d480 con 0x7f319015dca0 2024-04-25T10:14:59.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.198+0000 7f319694c640 1 -- 172.21.15.16:0/1014030114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31900750b0 con 0x7f319015dca0 2024-04-25T10:14:59.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.199+0000 7f318cff9640 1 -- 172.21.15.16:0/1014030114 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3188007b90 con 0x7f319015dca0 2024-04-25T10:14:59.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.199+0000 7f318cff9640 1 --2- 172.21.15.16:0/1014030114 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f317006b4d0 0x7f317006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.199+0000 7f318cff9640 1 -- 172.21.15.16:0/1014030114 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f31880ab0d0 con 0x7f319015dca0 2024-04-25T10:14:59.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.199+0000 7f318ffff640 1 --2- 172.21.15.16:0/1014030114 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f317006b4d0 0x7f317006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.200+0000 7f318ffff640 1 --2- 172.21.15.16:0/1014030114 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f317006b4d0 0x7f317006d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3180001660 tx=0x7f3180009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.203+0000 7f318cff9640 1 -- 172.21.15.16:0/1014030114 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3188079e60 con 0x7f319015dca0 2024-04-25T10:14:59.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.384+0000 7f319694c640 1 -- 172.21.15.16:0/1014030114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f319006cb20 con 0x7f319015dca0 2024-04-25T10:14:59.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.385+0000 7f318cff9640 1 -- 172.21.15.16:0/1014030114 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3188079760 con 0x7f319015dca0 2024-04-25T10:14:59.389 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:14:59.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.387+0000 7f319694c640 1 -- 172.21.15.16:0/1014030114 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f317006b4d0 msgr2=0x7f317006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.387+0000 7f319694c640 1 --2- 172.21.15.16:0/1014030114 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f317006b4d0 0x7f317006d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3180001660 tx=0x7f3180009040 comp rx=0 tx=0).stop 2024-04-25T10:14:59.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.387+0000 7f319694c640 1 -- 172.21.15.16:0/1014030114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f319015dca0 msgr2=0x7f31900ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.387+0000 7f319694c640 1 --2- 172.21.15.16:0/1014030114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f319015dca0 0x7f31900ad730 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f3188034040 tx=0x7f3188004a40 comp rx=0 tx=0).stop 2024-04-25T10:14:59.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.387+0000 7f319694c640 1 -- 172.21.15.16:0/1014030114 shutdown_connections 2024-04-25T10:14:59.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.387+0000 7f319694c640 1 --2- 172.21.15.16:0/1014030114 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f317006b4d0 0x7f317006d990 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.387+0000 7f319694c640 1 --2- 172.21.15.16:0/1014030114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f319015dca0 0x7f31900ad730 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.387+0000 7f319694c640 1 --2- 172.21.15.16:0/1014030114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31900d3b90 0x7f319014c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.387+0000 7f319694c640 1 --2- 172.21.15.16:0/1014030114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31900d0f60 0x7f31900aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.387+0000 7f319694c640 1 -- 172.21.15.16:0/1014030114 >> 172.21.15.16:0/1014030114 conn(0x7f31900726c0 msgr2=0x7f31900d1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.387+0000 7f319694c640 1 -- 172.21.15.16:0/1014030114 shutdown_connections 2024-04-25T10:14:59.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.387+0000 7f319694c640 1 -- 172.21.15.16:0/1014030114 wait complete. 2024-04-25T10:14:59.402 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:59.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.502+0000 7f78caecb640 1 Processor -- start 2024-04-25T10:14:59.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.502+0000 7f78caecb640 1 -- start start 2024-04-25T10:14:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.503+0000 7f78caecb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78c415b7b0 0x7f78c40ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.503+0000 7f78caecb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c415f000 con 0x7f78c40cf760 2024-04-25T10:14:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.503+0000 7f78caecb640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c415ed20 con 0x7f78c415e820 2024-04-25T10:14:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.503+0000 7f78caecb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c40ced00 con 0x7f78c415b7b0 2024-04-25T10:14:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.503+0000 7f78c3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f78c40cf760 0x7f78c415e110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55702/0 (socket says 172.21.15.16:55702) 2024-04-25T10:14:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.503+0000 7f78c3fff640 1 -- 172.21.15.16:0/734759926 learned_addr learned my addr 172.21.15.16:0/734759926 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:14:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c37fe640 1 -- 172.21.15.16:0/734759926 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4085861260 0 0) 0x7f78c415f000 con 0x7f78c40cf760 2024-04-25T10:14:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c37fe640 1 -- 172.21.15.16:0/734759926 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78ac003520 con 0x7f78c40cf760 2024-04-25T10:14:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c8c67640 1 --2- 172.21.15.16:0/734759926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78c415b7b0 0x7f78c40ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c37fe640 1 -- 172.21.15.16:0/734759926 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3017660411 0 0) 0x7f78ac003520 con 0x7f78c40cf760 2024-04-25T10:14:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c37fe640 1 -- 172.21.15.16:0/734759926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78c415b7b0 msgr2=0x7f78c40ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c37fe640 1 --2- 172.21.15.16:0/734759926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78c415b7b0 0x7f78c40ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c37fe640 1 -- 172.21.15.16:0/734759926 >> v1:172.21.15.186:6789/0 conn(0x7f78c415e820 legacy=0x7f78c40cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c37fe640 1 -- 172.21.15.16:0/734759926 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c40cdce0 con 0x7f78c40cf760 2024-04-25T10:14:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c37fe640 1 -- 172.21.15.16:0/734759926 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f78b8001670 con 0x7f78c40cf760 2024-04-25T10:14:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c37fe640 1 -- 172.21.15.16:0/734759926 >> v1:172.21.15.16:6789/0 conn(0x7f78c40cf760 legacy=0x7f78c415e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c37fe640 1 --2- 172.21.15.16:0/734759926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78ac0039e0 0x7f78ac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c37fe640 1 --2- 172.21.15.16:0/734759926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ac004800 0x7f78ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c37fe640 1 --2- 172.21.15.16:0/734759926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78c415b7b0 0x7f78ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c37fe640 1 -- 172.21.15.16:0/734759926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c40ced00 con 0x7f78ac004800 2024-04-25T10:14:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c37fe640 1 -- 172.21.15.16:0/734759926 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac00ba70 con 0x7f78ac0039e0 2024-04-25T10:14:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c37fe640 1 -- 172.21.15.16:0/734759926 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac00bc70 con 0x7f78c415b7b0 2024-04-25T10:14:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c3fff640 1 --2- 172.21.15.16:0/734759926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ac004800 0x7f78ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c8c67640 1 --2- 172.21.15.16:0/734759926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78ac0039e0 0x7f78ac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c9468640 1 --2- 172.21.15.16:0/734759926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78c415b7b0 0x7f78ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c3fff640 1 -- 172.21.15.16:0/734759926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78c415b7b0 msgr2=0x7f78ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c3fff640 1 --2- 172.21.15.16:0/734759926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78c415b7b0 0x7f78ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c3fff640 1 -- 172.21.15.16:0/734759926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78ac0039e0 msgr2=0x7f78ac003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c3fff640 1 --2- 172.21.15.16:0/734759926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78ac0039e0 0x7f78ac003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c3fff640 1 -- 172.21.15.16:0/734759926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac003520 con 0x7f78ac004800 2024-04-25T10:14:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.504+0000 7f78c9468640 1 --2- 172.21.15.16:0/734759926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78c415b7b0 0x7f78ac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.505+0000 7f78c3fff640 1 --2- 172.21.15.16:0/734759926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ac004800 0x7f78ac004c80 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f78b800c210 tx=0x7f78ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e984d0b47748f74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.505+0000 7f78c37fe640 1 -- 172.21.15.16:0/734759926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8007cf0 con 0x7f78ac004800 2024-04-25T10:14:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.505+0000 7f78c37fe640 1 -- 172.21.15.16:0/734759926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b8005ad0 con 0x7f78ac004800 2024-04-25T10:14:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.505+0000 7f78c37fe640 1 -- 172.21.15.16:0/734759926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8007070 con 0x7f78ac004800 2024-04-25T10:14:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.506+0000 7f78caecb640 1 -- 172.21.15.16:0/734759926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ac004800 msgr2=0x7f78ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.506+0000 7f78caecb640 1 --2- 172.21.15.16:0/734759926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ac004800 0x7f78ac004c80 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f78b800c210 tx=0x7f78ac005830 comp rx=0 tx=0).stop 2024-04-25T10:14:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.506+0000 7f78c9468640 1 -- 172.21.15.16:0/734759926 reap_dead start 2024-04-25T10:14:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.506+0000 7f78caecb640 1 -- 172.21.15.16:0/734759926 shutdown_connections 2024-04-25T10:14:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.506+0000 7f78caecb640 1 -- 172.21.15.16:0/734759926 >> 172.21.15.16:0/734759926 conn(0x7f78c406f4a0 msgr2=0x7f78c406f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.506+0000 7f78caecb640 1 -- 172.21.15.16:0/734759926 shutdown_connections 2024-04-25T10:14:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.506+0000 7f78caecb640 1 -- 172.21.15.16:0/734759926 wait complete. 2024-04-25T10:14:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.507+0000 7f78caecb640 1 Processor -- start 2024-04-25T10:14:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.507+0000 7f78caecb640 1 -- start start 2024-04-25T10:14:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.508+0000 7f78caecb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78ac004800 0x7f78c414bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.508+0000 7f78caecb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c40b5630 0x7f78c40b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.508+0000 7f78caecb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78c40b38c0 0x7f78c4158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.508+0000 7f78caecb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c41619f0 con 0x7f78c40b5630 2024-04-25T10:14:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.508+0000 7f78caecb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c40ce330 con 0x7f78c40b38c0 2024-04-25T10:14:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.508+0000 7f78caecb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c40ce7c0 con 0x7f78ac004800 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.508+0000 7f78c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c40b5630 0x7f78c40b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.508+0000 7f78c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c40b5630 0x7f78c40b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35622/0 (socket says 172.21.15.16:35622) 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.508+0000 7f78c3fff640 1 -- 172.21.15.16:0/2764131308 learned_addr learned my addr 172.21.15.16:0/2764131308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.508+0000 7f78c8c67640 1 --2- 172.21.15.16:0/2764131308 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78ac004800 0x7f78c414bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.508+0000 7f78c9468640 1 --2- 172.21.15.16:0/2764131308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78c40b38c0 0x7f78c4158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.508+0000 7f78c3fff640 1 -- 172.21.15.16:0/2764131308 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78ac004800 msgr2=0x7f78c414bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.508+0000 7f78c3fff640 1 --2- 172.21.15.16:0/2764131308 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78ac004800 0x7f78c414bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.508+0000 7f78c3fff640 1 -- 172.21.15.16:0/2764131308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78c40b38c0 msgr2=0x7f78c4158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.508+0000 7f78c3fff640 1 --2- 172.21.15.16:0/2764131308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78c40b38c0 0x7f78c4158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.508+0000 7f78c3fff640 1 -- 172.21.15.16:0/2764131308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c40b4590 con 0x7f78c40b5630 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.509+0000 7f78c3fff640 1 --2- 172.21.15.16:0/2764131308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c40b5630 0x7f78c40b3380 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f78b800c210 tx=0x7f78c415eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.509+0000 7f78c17fa640 1 -- 172.21.15.16:0/2764131308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8001630 con 0x7f78c40b5630 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.509+0000 7f78c17fa640 1 -- 172.21.15.16:0/2764131308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b8038040 con 0x7f78c40b5630 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.509+0000 7f78caecb640 1 -- 172.21.15.16:0/2764131308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c40b4870 con 0x7f78c40b5630 2024-04-25T10:14:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.509+0000 7f78c17fa640 1 -- 172.21.15.16:0/2764131308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b80072b0 con 0x7f78c40b5630 2024-04-25T10:14:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.509+0000 7f78caecb640 1 -- 172.21.15.16:0/2764131308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c40b4e30 con 0x7f78c40b5630 2024-04-25T10:14:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.510+0000 7f78caecb640 1 -- 172.21.15.16:0/2764131308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c40750b0 con 0x7f78c40b5630 2024-04-25T10:14:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.511+0000 7f78c17fa640 1 -- 172.21.15.16:0/2764131308 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78b8007450 con 0x7f78c40b5630 2024-04-25T10:14:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.511+0000 7f78c17fa640 1 --2- 172.21.15.16:0/2764131308 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f78b006b4d0 0x7f78b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.511+0000 7f78c17fa640 1 -- 172.21.15.16:0/2764131308 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f78b80aa360 con 0x7f78c40b5630 2024-04-25T10:14:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.511+0000 7f78c8c67640 1 --2- 172.21.15.16:0/2764131308 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f78b006b4d0 0x7f78b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.512+0000 7f78c8c67640 1 --2- 172.21.15.16:0/2764131308 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f78b006b4d0 0x7f78b006d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f78bc014510 tx=0x7f78bc00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.515+0000 7f78c17fa640 1 -- 172.21.15.16:0/2764131308 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78b8079180 con 0x7f78c40b5630 2024-04-25T10:14:59.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.693+0000 7f78caecb640 1 -- 172.21.15.16:0/2764131308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78c40cd390 con 0x7f78c40b5630 2024-04-25T10:14:59.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.693+0000 7f78c17fa640 1 -- 172.21.15.16:0/2764131308 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) v1 ==== 104+0+11 (secure 0 0 0) 0x7f78b8005aa0 con 0x7f78c40b5630 2024-04-25T10:14:59.698 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:14:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.695+0000 7f78caecb640 1 -- 172.21.15.16:0/2764131308 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f78b006b4d0 msgr2=0x7f78b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.695+0000 7f78caecb640 1 --2- 172.21.15.16:0/2764131308 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f78b006b4d0 0x7f78b006d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f78bc014510 tx=0x7f78bc00b040 comp rx=0 tx=0).stop 2024-04-25T10:14:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.695+0000 7f78caecb640 1 -- 172.21.15.16:0/2764131308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c40b5630 msgr2=0x7f78c40b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.695+0000 7f78caecb640 1 --2- 172.21.15.16:0/2764131308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c40b5630 0x7f78c40b3380 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f78b800c210 tx=0x7f78c415eef0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.695+0000 7f78caecb640 1 -- 172.21.15.16:0/2764131308 shutdown_connections 2024-04-25T10:14:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.695+0000 7f78caecb640 1 --2- 172.21.15.16:0/2764131308 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f78b006b4d0 0x7f78b006d990 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.695+0000 7f78caecb640 1 --2- 172.21.15.16:0/2764131308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78c40b38c0 0x7f78c4158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.695+0000 7f78caecb640 1 --2- 172.21.15.16:0/2764131308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c40b5630 0x7f78c40b3380 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.695+0000 7f78caecb640 1 --2- 172.21.15.16:0/2764131308 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78ac004800 0x7f78c414bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.695+0000 7f78caecb640 1 -- 172.21.15.16:0/2764131308 >> 172.21.15.16:0/2764131308 conn(0x7f78c406f4a0 msgr2=0x7f78c40cc040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.695+0000 7f78caecb640 1 -- 172.21.15.16:0/2764131308 shutdown_connections 2024-04-25T10:14:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.696+0000 7f78caecb640 1 -- 172.21.15.16:0/2764131308 wait complete. 2024-04-25T10:14:59.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:59.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.810+0000 7fadc6f77640 1 Processor -- start 2024-04-25T10:14:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.810+0000 7fadc6f77640 1 -- start start 2024-04-25T10:14:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.811+0000 7fadc6f77640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fadc00b97c0 0x7fadc00b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.811+0000 7fadc6f77640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadc00b5a30 0x7fadc00d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.811+0000 7fadc6f77640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc01627a0 con 0x7fadc015dc80 2024-04-25T10:14:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.811+0000 7fadc6f77640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc0161f80 con 0x7fadc00b97c0 2024-04-25T10:14:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.811+0000 7fadc6f77640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc00b5e60 con 0x7fadc00b5a30 2024-04-25T10:14:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.811+0000 7fadbffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fadc00b97c0 0x7fadc00b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.811+0000 7fadbffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fadc00b97c0 0x7fadc00b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35638/0 (socket says 172.21.15.16:35638) 2024-04-25T10:14:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.811+0000 7fadbffff640 1 -- 172.21.15.16:0/1365778009 learned_addr learned my addr 172.21.15.16:0/1365778009 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.811+0000 7fadc5514640 1 --2- 172.21.15.16:0/1365778009 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadc00b5a30 0x7fadc00d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.811+0000 7fadbffff640 1 -- 172.21.15.16:0/1365778009 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadc00b5a30 msgr2=0x7fadc00d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.811+0000 7fadbffff640 1 --2- 172.21.15.16:0/1365778009 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadc00b5a30 0x7fadc00d2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.811+0000 7fadbffff640 1 -- 172.21.15.16:0/1365778009 >> v1:172.21.15.186:6789/0 conn(0x7fadc015dc80 legacy=0x7fadc0161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.811+0000 7fadbffff640 1 -- 172.21.15.16:0/1365778009 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc01621e0 con 0x7fadc00b97c0 2024-04-25T10:14:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.812+0000 7fadbffff640 1 --2- 172.21.15.16:0/1365778009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadc00b97c0 0x7fadc00b54f0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fadb40098a0 tx=0x7fadb402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffec3c49dd5dc33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.812+0000 7fadbf7fe640 1 -- 172.21.15.16:0/1365778009 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4037070 con 0x7fadc00b97c0 2024-04-25T10:14:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.812+0000 7fadbf7fe640 1 -- 172.21.15.16:0/1365778009 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb4032440 con 0x7fadc00b97c0 2024-04-25T10:14:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.812+0000 7fadbf7fe640 1 -- 172.21.15.16:0/1365778009 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4032810 con 0x7fadc00b97c0 2024-04-25T10:14:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.813+0000 7fadc6f77640 1 -- 172.21.15.16:0/1365778009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadc00b97c0 msgr2=0x7fadc00b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.813+0000 7fadc6f77640 1 --2- 172.21.15.16:0/1365778009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadc00b97c0 0x7fadc00b54f0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fadb40098a0 tx=0x7fadb402a250 comp rx=0 tx=0).stop 2024-04-25T10:14:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.813+0000 7fadc6f77640 1 -- 172.21.15.16:0/1365778009 shutdown_connections 2024-04-25T10:14:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.813+0000 7fadc6f77640 1 --2- 172.21.15.16:0/1365778009 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadc00b5a30 0x7fadc00d2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.813+0000 7fadc6f77640 1 --2- 172.21.15.16:0/1365778009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadc00b97c0 0x7fadc00b54f0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.813+0000 7fadc6f77640 1 -- 172.21.15.16:0/1365778009 >> 172.21.15.16:0/1365778009 conn(0x7fadc00d04f0 msgr2=0x7fadc00d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.813+0000 7fadc6f77640 1 -- 172.21.15.16:0/1365778009 shutdown_connections 2024-04-25T10:14:59.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.813+0000 7fadc6f77640 1 -- 172.21.15.16:0/1365778009 wait complete. 2024-04-25T10:14:59.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.814+0000 7fadc6f77640 1 Processor -- start 2024-04-25T10:14:59.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.814+0000 7fadc6f77640 1 -- start start 2024-04-25T10:14:59.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.815+0000 7fadc6f77640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadc00b5a30 0x7fadc014eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.815+0000 7fadc6f77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadc00b97c0 0x7fadc00cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.815+0000 7fadc6f77640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadc015dc80 0x7fadc015fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.815+0000 7fadc6f77640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc015e060 con 0x7fadc00b97c0 2024-04-25T10:14:59.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.815+0000 7fadc6f77640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc00d1c70 con 0x7fadc015dc80 2024-04-25T10:14:59.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.815+0000 7fadc6f77640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc00d29b0 con 0x7fadc00b5a30 2024-04-25T10:14:59.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.815+0000 7fadbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadc00b97c0 0x7fadc00cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.815+0000 7fadbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadc00b97c0 0x7fadc00cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35646/0 (socket says 172.21.15.16:35646) 2024-04-25T10:14:59.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.815+0000 7fadbffff640 1 -- 172.21.15.16:0/2203587411 learned_addr learned my addr 172.21.15.16:0/2203587411 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:14:59.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.815+0000 7fadc4d13640 1 --2- 172.21.15.16:0/2203587411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadc00b5a30 0x7fadc014eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.815+0000 7fadc5514640 1 --2- 172.21.15.16:0/2203587411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadc015dc80 0x7fadc015fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.815+0000 7fadbffff640 1 -- 172.21.15.16:0/2203587411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadc00b5a30 msgr2=0x7fadc014eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.815+0000 7fadbffff640 1 --2- 172.21.15.16:0/2203587411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadc00b5a30 0x7fadc014eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.815+0000 7fadbffff640 1 -- 172.21.15.16:0/2203587411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadc015dc80 msgr2=0x7fadc015fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.815+0000 7fadbffff640 1 --2- 172.21.15.16:0/2203587411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadc015dc80 0x7fadc015fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.816+0000 7fadbffff640 1 -- 172.21.15.16:0/2203587411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc014f3f0 con 0x7fadc00b97c0 2024-04-25T10:14:59.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.816+0000 7fadc5514640 1 --2- 172.21.15.16:0/2203587411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadc015dc80 0x7fadc015fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:59.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.816+0000 7fadbffff640 1 --2- 172.21.15.16:0/2203587411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadc00b97c0 0x7fadc00cb4c0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fadb4000c00 tx=0x7fadb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.816+0000 7fadbd7fa640 1 -- 172.21.15.16:0/2203587411 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4037040 con 0x7fadc00b97c0 2024-04-25T10:14:59.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.816+0000 7fadbd7fa640 1 -- 172.21.15.16:0/2203587411 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb4004230 con 0x7fadc00b97c0 2024-04-25T10:14:59.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.816+0000 7fadc6f77640 1 -- 172.21.15.16:0/2203587411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc00c7740 con 0x7fadc00b97c0 2024-04-25T10:14:59.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.816+0000 7fadbd7fa640 1 -- 172.21.15.16:0/2203587411 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4030620 con 0x7fadc00b97c0 2024-04-25T10:14:59.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.816+0000 7fadc6f77640 1 -- 172.21.15.16:0/2203587411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc00c7cc0 con 0x7fadc00b97c0 2024-04-25T10:14:59.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.817+0000 7fadc6f77640 1 -- 172.21.15.16:0/2203587411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc00750b0 con 0x7fadc00b97c0 2024-04-25T10:14:59.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.817+0000 7fadbd7fa640 1 -- 172.21.15.16:0/2203587411 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fadb40307c0 con 0x7fadc00b97c0 2024-04-25T10:14:59.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.818+0000 7fadbd7fa640 1 --2- 172.21.15.16:0/2203587411 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fadac06b4d0 0x7fadac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.818+0000 7fadbd7fa640 1 -- 172.21.15.16:0/2203587411 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fadb40ab150 con 0x7fadc00b97c0 2024-04-25T10:14:59.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.818+0000 7fadc4d13640 1 --2- 172.21.15.16:0/2203587411 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fadac06b4d0 0x7fadac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.818+0000 7fadc4d13640 1 --2- 172.21.15.16:0/2203587411 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fadac06b4d0 0x7fadac06d990 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fadb0006fd0 tx=0x7fadb0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:14:59.822+0000 7fadbd7fa640 1 -- 172.21.15.16:0/2203587411 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fadb407a090 con 0x7fadc00b97c0 2024-04-25T10:15:00.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.002+0000 7fadc6f77640 1 -- 172.21.15.16:0/2203587411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fadc00cde30 con 0x7fadc00b97c0 2024-04-25T10:15:00.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.003+0000 7fadbd7fa640 1 -- 172.21.15.16:0/2203587411 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) v1 ==== 100+0+11 (secure 0 0 0) 0x7fadb4079990 con 0x7fadc00b97c0 2024-04-25T10:15:00.007 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:15:00.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.005+0000 7fadc6f77640 1 -- 172.21.15.16:0/2203587411 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fadac06b4d0 msgr2=0x7fadac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.005+0000 7fadc6f77640 1 --2- 172.21.15.16:0/2203587411 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fadac06b4d0 0x7fadac06d990 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fadb0006fd0 tx=0x7fadb0008040 comp rx=0 tx=0).stop 2024-04-25T10:15:00.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.005+0000 7fadc6f77640 1 -- 172.21.15.16:0/2203587411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadc00b97c0 msgr2=0x7fadc00cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.005+0000 7fadc6f77640 1 --2- 172.21.15.16:0/2203587411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadc00b97c0 0x7fadc00cb4c0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fadb4000c00 tx=0x7fadb40046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.005+0000 7fadc6f77640 1 -- 172.21.15.16:0/2203587411 shutdown_connections 2024-04-25T10:15:00.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.005+0000 7fadc6f77640 1 --2- 172.21.15.16:0/2203587411 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fadac06b4d0 0x7fadac06d990 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.005+0000 7fadc6f77640 1 --2- 172.21.15.16:0/2203587411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadc015dc80 0x7fadc015fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.005+0000 7fadc6f77640 1 --2- 172.21.15.16:0/2203587411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadc00b97c0 0x7fadc00cb4c0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.005+0000 7fadc6f77640 1 --2- 172.21.15.16:0/2203587411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadc00b5a30 0x7fadc014eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.005+0000 7fadc6f77640 1 -- 172.21.15.16:0/2203587411 >> 172.21.15.16:0/2203587411 conn(0x7fadc00d04f0 msgr2=0x7fadc00b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:00.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.005+0000 7fadc6f77640 1 -- 172.21.15.16:0/2203587411 shutdown_connections 2024-04-25T10:15:00.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.005+0000 7fadc6f77640 1 -- 172.21.15.16:0/2203587411 wait complete. 2024-04-25T10:15:00.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:15:00.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.119+0000 7f519df5c640 1 Processor -- start 2024-04-25T10:15:00.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.119+0000 7f519df5c640 1 -- start start 2024-04-25T10:15:00.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.120+0000 7f519df5c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f51980d0b60 0x7f5198168950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.120+0000 7f519df5c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51980d2a20 con 0x7f519815dec0 2024-04-25T10:15:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.120+0000 7f519df5c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51980d2d00 con 0x7f51980d24b0 2024-04-25T10:15:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.120+0000 7f519df5c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51980d2fe0 con 0x7f51980d0b60 2024-04-25T10:15:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.120+0000 7f5197fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f51980d0b60 0x7f5198168950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.120+0000 7f51977fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f519815dec0 0x7f51980d1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55718/0 (socket says 172.21.15.16:55718) 2024-04-25T10:15:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.120+0000 7f51977fe640 1 -- 172.21.15.16:0/142245984 learned_addr learned my addr 172.21.15.16:0/142245984 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.121+0000 7f5197fff640 1 -- 172.21.15.16:0/142245984 >> v1:172.21.15.186:6789/0 conn(0x7f51980d24b0 legacy=0x7f51980d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.121+0000 7f5197fff640 1 -- 172.21.15.16:0/142245984 >> v1:172.21.15.16:6789/0 conn(0x7f519815dec0 legacy=0x7f51980d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.121+0000 7f51967fc640 1 -- 172.21.15.16:0/142245984 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925895253 0 0) 0x7f51980d2a20 con 0x7f519815dec0 2024-04-25T10:15:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.121+0000 7f5197fff640 1 -- 172.21.15.16:0/142245984 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51980d32c0 con 0x7f51980d0b60 2024-04-25T10:15:00.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.122+0000 7f5197fff640 1 --2- 172.21.15.16:0/142245984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51980d0b60 0x7f5198168950 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f5190009dc0 tx=0x7f519002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d07e3b7edd167ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.122+0000 7f51967fc640 1 -- 172.21.15.16:0/142245984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190007740 con 0x7f51980d0b60 2024-04-25T10:15:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.122+0000 7f51967fc640 1 -- 172.21.15.16:0/142245984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51900078e0 con 0x7f51980d0b60 2024-04-25T10:15:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.122+0000 7f51967fc640 1 -- 172.21.15.16:0/142245984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190007cb0 con 0x7f51980d0b60 2024-04-25T10:15:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.122+0000 7f519df5c640 1 -- 172.21.15.16:0/142245984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51980d0b60 msgr2=0x7f5198168950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.122+0000 7f519df5c640 1 --2- 172.21.15.16:0/142245984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51980d0b60 0x7f5198168950 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f5190009dc0 tx=0x7f519002a730 comp rx=0 tx=0).stop 2024-04-25T10:15:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.122+0000 7f519df5c640 1 -- 172.21.15.16:0/142245984 shutdown_connections 2024-04-25T10:15:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.122+0000 7f519df5c640 1 --2- 172.21.15.16:0/142245984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51980d0b60 0x7f5198168950 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.122+0000 7f519df5c640 1 -- 172.21.15.16:0/142245984 >> 172.21.15.16:0/142245984 conn(0x7f5198072900 msgr2=0x7f51980b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.122+0000 7f519df5c640 1 -- 172.21.15.16:0/142245984 shutdown_connections 2024-04-25T10:15:00.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.122+0000 7f519df5c640 1 -- 172.21.15.16:0/142245984 wait complete. 2024-04-25T10:15:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.123+0000 7f519df5c640 1 Processor -- start 2024-04-25T10:15:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.124+0000 7f519df5c640 1 -- start start 2024-04-25T10:15:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.124+0000 7f519df5c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51980d0b60 0x7f51981f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.124+0000 7f519df5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51980d24b0 0x7f5198153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.124+0000 7f519df5c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f519815dec0 0x7f519815e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.124+0000 7f519df5c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51980b8020 con 0x7f51980d24b0 2024-04-25T10:15:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.124+0000 7f519df5c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51980b7ea0 con 0x7f519815dec0 2024-04-25T10:15:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.124+0000 7f519df5c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51980b81a0 con 0x7f51980d0b60 2024-04-25T10:15:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.124+0000 7f5196ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51980d24b0 0x7f5198153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.124+0000 7f5196ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51980d24b0 0x7f5198153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35652/0 (socket says 172.21.15.16:35652) 2024-04-25T10:15:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.124+0000 7f5196ffd640 1 -- 172.21.15.16:0/3850959872 learned_addr learned my addr 172.21.15.16:0/3850959872 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.125+0000 7f5197fff640 1 --2- 172.21.15.16:0/3850959872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f519815dec0 0x7f519815e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.125+0000 7f51977fe640 1 --2- 172.21.15.16:0/3850959872 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51980d0b60 0x7f51981f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.125+0000 7f5196ffd640 1 -- 172.21.15.16:0/3850959872 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51980d0b60 msgr2=0x7f51981f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.125+0000 7f5196ffd640 1 --2- 172.21.15.16:0/3850959872 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51980d0b60 0x7f51981f3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.125+0000 7f5196ffd640 1 -- 172.21.15.16:0/3850959872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f519815dec0 msgr2=0x7f519815e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.125+0000 7f5196ffd640 1 --2- 172.21.15.16:0/3850959872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f519815dec0 0x7f519815e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.125+0000 7f5196ffd640 1 -- 172.21.15.16:0/3850959872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519815e7e0 con 0x7f51980d24b0 2024-04-25T10:15:00.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.125+0000 7f5196ffd640 1 --2- 172.21.15.16:0/3850959872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51980d24b0 0x7f5198153dd0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f518c00c480 tx=0x7f518c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.125+0000 7f517ffff640 1 -- 172.21.15.16:0/3850959872 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0052d0 con 0x7f51980d24b0 2024-04-25T10:15:00.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.125+0000 7f517ffff640 1 -- 172.21.15.16:0/3850959872 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c005470 con 0x7f51980d24b0 2024-04-25T10:15:00.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.125+0000 7f517ffff640 1 -- 172.21.15.16:0/3850959872 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c012870 con 0x7f51980d24b0 2024-04-25T10:15:00.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.125+0000 7f519df5c640 1 -- 172.21.15.16:0/3850959872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519815ea80 con 0x7f51980d24b0 2024-04-25T10:15:00.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.125+0000 7f519df5c640 1 -- 172.21.15.16:0/3850959872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519815f040 con 0x7f51980d24b0 2024-04-25T10:15:00.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.126+0000 7f519df5c640 1 -- 172.21.15.16:0/3850959872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51980750b0 con 0x7f51980d24b0 2024-04-25T10:15:00.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.127+0000 7f517ffff640 1 -- 172.21.15.16:0/3850959872 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f518c012a10 con 0x7f51980d24b0 2024-04-25T10:15:00.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.127+0000 7f517ffff640 1 --2- 172.21.15.16:0/3850959872 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f518406b4d0 0x7f518406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.127+0000 7f517ffff640 1 -- 172.21.15.16:0/3850959872 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f518c089db0 con 0x7f51980d24b0 2024-04-25T10:15:00.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.127+0000 7f51977fe640 1 --2- 172.21.15.16:0/3850959872 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f518406b4d0 0x7f518406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.128+0000 7f51977fe640 1 --2- 172.21.15.16:0/3850959872 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f518406b4d0 0x7f518406d990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5188000f60 tx=0x7f5188007e40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.131+0000 7f517ffff640 1 -- 172.21.15.16:0/3850959872 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f518c058580 con 0x7f51980d24b0 2024-04-25T10:15:00.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.323+0000 7f519df5c640 1 -- 172.21.15.16:0/3850959872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f51980d0610 con 0x7f51980d24b0 2024-04-25T10:15:00.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.323+0000 7f517ffff640 1 -- 172.21.15.16:0/3850959872 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v119) v1 ==== 80+0+242 (secure 0 0 0) 0x7f518c05c620 con 0x7f51980d24b0 2024-04-25T10:15:00.328 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6835/2697110279 2024-04-26T10:14:50.050260+0000 2024-04-25T10:15:00.328 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6833/2697110279 2024-04-26T10:14:50.050260+0000 2024-04-25T10:15:00.328 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6835/123396308 2024-04-26T10:14:39.639508+0000 2024-04-25T10:15:00.328 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6833/123396308 2024-04-26T10:14:39.639508+0000 2024-04-25T10:15:00.328 INFO:teuthology.orchestra.run.smithi016.stderr:listed 4 entries 2024-04-25T10:15:00.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.325+0000 7f519df5c640 1 -- 172.21.15.16:0/3850959872 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f518406b4d0 msgr2=0x7f518406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.325+0000 7f519df5c640 1 --2- 172.21.15.16:0/3850959872 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f518406b4d0 0x7f518406d990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5188000f60 tx=0x7f5188007e40 comp rx=0 tx=0).stop 2024-04-25T10:15:00.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.325+0000 7f519df5c640 1 -- 172.21.15.16:0/3850959872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51980d24b0 msgr2=0x7f5198153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.325+0000 7f519df5c640 1 --2- 172.21.15.16:0/3850959872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51980d24b0 0x7f5198153dd0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f518c00c480 tx=0x7f518c00c940 comp rx=0 tx=0).stop 2024-04-25T10:15:00.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.325+0000 7f519df5c640 1 -- 172.21.15.16:0/3850959872 shutdown_connections 2024-04-25T10:15:00.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.325+0000 7f519df5c640 1 --2- 172.21.15.16:0/3850959872 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f518406b4d0 0x7f518406d990 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.325+0000 7f519df5c640 1 --2- 172.21.15.16:0/3850959872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f519815dec0 0x7f519815e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.325+0000 7f519df5c640 1 --2- 172.21.15.16:0/3850959872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51980d24b0 0x7f5198153dd0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.325+0000 7f519df5c640 1 --2- 172.21.15.16:0/3850959872 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51980d0b60 0x7f51981f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.325+0000 7f519df5c640 1 -- 172.21.15.16:0/3850959872 >> 172.21.15.16:0/3850959872 conn(0x7f5198072900 msgr2=0x7f51980b8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:00.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.326+0000 7f519df5c640 1 -- 172.21.15.16:0/3850959872 shutdown_connections 2024-04-25T10:15:00.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.326+0000 7f519df5c640 1 -- 172.21.15.16:0/3850959872 wait complete. 2024-04-25T10:15:00.340 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:15:00.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:00.340 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:00.368 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:15:00.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:00.368 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:15:00.423 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:15:00.423 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:00.442 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:00.443 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:15:00.443 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:00.480 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:15:00.480 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:15:00.481 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:00.502 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:00.503 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.617+0000 7f80ebc58640 1 Processor -- start 2024-04-25T10:15:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.617+0000 7f80ebc58640 1 -- start start 2024-04-25T10:15:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.618+0000 7f80ebc58640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f80e415dd50 0x7f80e40d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.618+0000 7f80ebc58640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e40d2440 con 0x7f80e40ceeb0 2024-04-25T10:15:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.618+0000 7f80ebc58640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e40d2720 con 0x7f80e40b9460 2024-04-25T10:15:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.618+0000 7f80ebc58640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e40d2a00 con 0x7f80e415dd50 2024-04-25T10:15:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.618+0000 7f80ea1f5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f80e40ceeb0 0x7f80e40b6c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55722/0 (socket says 172.21.15.16:55722) 2024-04-25T10:15:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.618+0000 7f80ea1f5640 1 -- 172.21.15.16:0/2442578763 learned_addr learned my addr 172.21.15.16:0/2442578763 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.618+0000 7f80e89f2640 1 -- 172.21.15.16:0/2442578763 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951948560 0 0) 0x7f80e40d2440 con 0x7f80e40ceeb0 2024-04-25T10:15:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.618+0000 7f80e89f2640 1 -- 172.21.15.16:0/2442578763 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80d0003520 con 0x7f80e40ceeb0 2024-04-25T10:15:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.618+0000 7f80e99f4640 1 --2- 172.21.15.16:0/2442578763 >> v2:172.21.15.186:3300/0 conn(0x7f80e415dd50 0x7f80e40d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e89f2640 1 -- 172.21.15.16:0/2442578763 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2392346773 0 0) 0x7f80d0003520 con 0x7f80e40ceeb0 2024-04-25T10:15:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e89f2640 1 -- 172.21.15.16:0/2442578763 >> v2:172.21.15.186:3300/0 conn(0x7f80e415dd50 msgr2=0x7f80e40d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e89f2640 1 --2- 172.21.15.16:0/2442578763 >> v2:172.21.15.186:3300/0 conn(0x7f80e415dd50 0x7f80e40d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e89f2640 1 -- 172.21.15.16:0/2442578763 >> v1:172.21.15.186:6789/0 conn(0x7f80e40b9460 legacy=0x7f80e40ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e89f2640 1 -- 172.21.15.16:0/2442578763 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e40d2ce0 con 0x7f80e40ceeb0 2024-04-25T10:15:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e89f2640 1 -- 172.21.15.16:0/2442578763 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f80dc0019f0 con 0x7f80e40ceeb0 2024-04-25T10:15:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e89f2640 1 -- 172.21.15.16:0/2442578763 >> v1:172.21.15.16:6789/0 conn(0x7f80e40ceeb0 legacy=0x7f80e40b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e89f2640 1 --2- 172.21.15.16:0/2442578763 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80d00039b0 0x7f80d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e89f2640 1 --2- 172.21.15.16:0/2442578763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d0004800 0x7f80d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e89f2640 1 --2- 172.21.15.16:0/2442578763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80d00051c0 0x7f80d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e89f2640 1 -- 172.21.15.16:0/2442578763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e40d2a00 con 0x7f80d0004800 2024-04-25T10:15:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e89f2640 1 -- 172.21.15.16:0/2442578763 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d000c160 con 0x7f80d00051c0 2024-04-25T10:15:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e89f2640 1 -- 172.21.15.16:0/2442578763 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d000c360 con 0x7f80d00039b0 2024-04-25T10:15:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e91f3640 1 --2- 172.21.15.16:0/2442578763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d0004800 0x7f80d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e99f4640 1 --2- 172.21.15.16:0/2442578763 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80d00039b0 0x7f80d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80ea1f5640 1 --2- 172.21.15.16:0/2442578763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80d00051c0 0x7f80d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e91f3640 1 -- 172.21.15.16:0/2442578763 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80d00039b0 msgr2=0x7f80d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e91f3640 1 --2- 172.21.15.16:0/2442578763 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80d00039b0 0x7f80d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e91f3640 1 -- 172.21.15.16:0/2442578763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80d00051c0 msgr2=0x7f80d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e91f3640 1 --2- 172.21.15.16:0/2442578763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80d00051c0 0x7f80d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80e91f3640 1 -- 172.21.15.16:0/2442578763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d0003520 con 0x7f80d0004800 2024-04-25T10:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.619+0000 7f80ea1f5640 1 -- 172.21.15.16:0/2442578763 reap_dead start 2024-04-25T10:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.620+0000 7f80e91f3640 1 --2- 172.21.15.16:0/2442578763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d0004800 0x7f80d0004c80 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f80d4001750 tx=0x7f80d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5249ee321e43cb75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.620+0000 7f80e89f2640 1 -- 172.21.15.16:0/2442578763 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d4001370 con 0x7f80d0004800 2024-04-25T10:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.620+0000 7f80e89f2640 1 -- 172.21.15.16:0/2442578763 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d40052d0 con 0x7f80d0004800 2024-04-25T10:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.620+0000 7f80e89f2640 1 -- 172.21.15.16:0/2442578763 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d402abc0 con 0x7f80d0004800 2024-04-25T10:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.621+0000 7f80ebc58640 1 -- 172.21.15.16:0/2442578763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d0004800 msgr2=0x7f80d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.621+0000 7f80ebc58640 1 --2- 172.21.15.16:0/2442578763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d0004800 0x7f80d0004c80 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f80d4001750 tx=0x7f80d0005830 comp rx=0 tx=0).stop 2024-04-25T10:15:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.621+0000 7f80ebc58640 1 -- 172.21.15.16:0/2442578763 shutdown_connections 2024-04-25T10:15:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.621+0000 7f80ebc58640 1 --2- 172.21.15.16:0/2442578763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d0004800 0x7f80d0004c80 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.621+0000 7f80ebc58640 1 -- 172.21.15.16:0/2442578763 >> 172.21.15.16:0/2442578763 conn(0x7f80e40722d0 msgr2=0x7f80e40726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.621+0000 7f80ebc58640 1 -- 172.21.15.16:0/2442578763 shutdown_connections 2024-04-25T10:15:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.621+0000 7f80ebc58640 1 -- 172.21.15.16:0/2442578763 wait complete. 2024-04-25T10:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.622+0000 7f80ebc58640 1 Processor -- start 2024-04-25T10:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.622+0000 7f80ebc58640 1 -- start start 2024-04-25T10:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80ebc58640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80d0004800 0x7f80e40b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80ebc58640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80e4150510 0x7f80e40cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80ebc58640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e40cbe70 0x7f80e40cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80ebc58640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e40d3660 con 0x7f80e40cbe70 2024-04-25T10:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80ebc58640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e40d34e0 con 0x7f80d0004800 2024-04-25T10:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80ebc58640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e40d37e0 con 0x7f80e4150510 2024-04-25T10:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80ea1f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e40cbe70 0x7f80e40cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80ea1f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e40cbe70 0x7f80e40cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35670/0 (socket says 172.21.15.16:35670) 2024-04-25T10:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80ea1f5640 1 -- 172.21.15.16:0/1866766463 learned_addr learned my addr 172.21.15.16:0/1866766463 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:00.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80e91f3640 1 --2- 172.21.15.16:0/1866766463 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80e4150510 0x7f80e40cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80e99f4640 1 --2- 172.21.15.16:0/1866766463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80d0004800 0x7f80e40b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80ea1f5640 1 -- 172.21.15.16:0/1866766463 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80e4150510 msgr2=0x7f80e40cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80ea1f5640 1 --2- 172.21.15.16:0/1866766463 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80e4150510 0x7f80e40cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80ea1f5640 1 -- 172.21.15.16:0/1866766463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80d0004800 msgr2=0x7f80e40b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80ea1f5640 1 --2- 172.21.15.16:0/1866766463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80d0004800 0x7f80e40b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80ea1f5640 1 -- 172.21.15.16:0/1866766463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e40cc860 con 0x7f80e40cbe70 2024-04-25T10:15:00.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80ea1f5640 1 --2- 172.21.15.16:0/1866766463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e40cbe70 0x7f80e40cc320 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f80e408bba0 tx=0x7f80dc007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80da7fc640 1 -- 172.21.15.16:0/1866766463 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc00abc0 con 0x7f80e40cbe70 2024-04-25T10:15:00.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80da7fc640 1 -- 172.21.15.16:0/1866766463 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80dc018070 con 0x7f80e40cbe70 2024-04-25T10:15:00.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80da7fc640 1 -- 172.21.15.16:0/1866766463 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc0133f0 con 0x7f80e40cbe70 2024-04-25T10:15:00.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.623+0000 7f80ebc58640 1 -- 172.21.15.16:0/1866766463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e4150990 con 0x7f80e40cbe70 2024-04-25T10:15:00.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.624+0000 7f80ebc58640 1 -- 172.21.15.16:0/1866766463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e40b4f90 con 0x7f80e40cbe70 2024-04-25T10:15:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.625+0000 7f80ebc58640 1 -- 172.21.15.16:0/1866766463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e4067850 con 0x7f80e40cbe70 2024-04-25T10:15:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.625+0000 7f80da7fc640 1 -- 172.21.15.16:0/1866766463 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f80dc013590 con 0x7f80e40cbe70 2024-04-25T10:15:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.625+0000 7f80da7fc640 1 --2- 172.21.15.16:0/1866766463 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f80c806b500 0x7f80c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.626+0000 7f80da7fc640 1 -- 172.21.15.16:0/1866766463 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f80dc08a750 con 0x7f80e40cbe70 2024-04-25T10:15:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.626+0000 7f80e99f4640 1 --2- 172.21.15.16:0/1866766463 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f80c806b500 0x7f80c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.626+0000 7f80e99f4640 1 --2- 172.21.15.16:0/1866766463 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f80c806b500 0x7f80c806d9c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009b00 tx=0x7f80e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.630+0000 7f80da7fc640 1 -- 172.21.15.16:0/1866766463 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f80dc0594e0 con 0x7f80e40cbe70 2024-04-25T10:15:00.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.845+0000 7f80ebc58640 1 -- 172.21.15.16:0/1866766463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80e40c7880 con 0x7f80e40cbe70 2024-04-25T10:15:00.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.846+0000 7f80da7fc640 1 -- 172.21.15.16:0/1866766463 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f80dc058de0 con 0x7f80e40cbe70 2024-04-25T10:15:00.850 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:15:00.850 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":96,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2871061122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2871061122},{"type":"v1","addr":"172.21.15.186:6836","nonce":2871061122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1871507206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1871507206},{"type":"v1","addr":"172.21.15.186:6837","nonce":1871507206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4064412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4064412586},{"type":"v1","addr":"172.21.15.16:6834","nonce":4064412586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:26.751060+0000","modified":"2024-04-25T10:14:54.069182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9252":{"gid":9252,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6835/3763439204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":3763439204},{"type":"v1","addr":"172.21.15.16:6835","nonce":3763439204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:15:00.850 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 96 2024-04-25T10:15:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.847+0000 7f80ebc58640 1 -- 172.21.15.16:0/1866766463 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f80c806b500 msgr2=0x7f80c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.847+0000 7f80ebc58640 1 --2- 172.21.15.16:0/1866766463 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f80c806b500 0x7f80c806d9c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009b00 tx=0x7f80e0008040 comp rx=0 tx=0).stop 2024-04-25T10:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.847+0000 7f80ebc58640 1 -- 172.21.15.16:0/1866766463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e40cbe70 msgr2=0x7f80e40cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.847+0000 7f80ebc58640 1 --2- 172.21.15.16:0/1866766463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e40cbe70 0x7f80e40cc320 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f80e408bba0 tx=0x7f80dc007a90 comp rx=0 tx=0).stop 2024-04-25T10:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.847+0000 7f80ebc58640 1 -- 172.21.15.16:0/1866766463 shutdown_connections 2024-04-25T10:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.847+0000 7f80ebc58640 1 --2- 172.21.15.16:0/1866766463 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f80c806b500 0x7f80c806d9c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.847+0000 7f80ebc58640 1 --2- 172.21.15.16:0/1866766463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e40cbe70 0x7f80e40cc320 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.847+0000 7f80ebc58640 1 --2- 172.21.15.16:0/1866766463 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80e4150510 0x7f80e40cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.847+0000 7f80ebc58640 1 --2- 172.21.15.16:0/1866766463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80d0004800 0x7f80e40b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.847+0000 7f80ebc58640 1 -- 172.21.15.16:0/1866766463 >> 172.21.15.16:0/1866766463 conn(0x7f80e40722d0 msgr2=0x7f80e40c9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.847+0000 7f80ebc58640 1 -- 172.21.15.16:0/1866766463 shutdown_connections 2024-04-25T10:15:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.848+0000 7f80ebc58640 1 -- 172.21.15.16:0/1866766463 wait complete. 2024-04-25T10:15:00.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.959+0000 7f619d1e2640 1 Processor -- start 2024-04-25T10:15:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.959+0000 7f619d1e2640 1 -- start start 2024-04-25T10:15:00.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.960+0000 7f619d1e2640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f619815dce0 0x7f61980d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.960+0000 7f619d1e2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f619815e0c0 con 0x7f61980d35b0 2024-04-25T10:15:00.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.960+0000 7f619d1e2640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61980d2510 con 0x7f61980b5b60 2024-04-25T10:15:00.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.960+0000 7f619d1e2640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61980d27f0 con 0x7f619815dce0 2024-04-25T10:15:00.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.960+0000 7f619659c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f61980d35b0 0x7f61980b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55736/0 (socket says 172.21.15.16:55736) 2024-04-25T10:15:00.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.960+0000 7f619659c640 1 -- 172.21.15.16:0/3087419013 learned_addr learned my addr 172.21.15.16:0/3087419013 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:00.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.960+0000 7f6195d9b640 1 -- 172.21.15.16:0/3087419013 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330321053 0 0) 0x7f619815e0c0 con 0x7f61980d35b0 2024-04-25T10:15:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.960+0000 7f6195d9b640 1 -- 172.21.15.16:0/3087419013 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6180003520 con 0x7f61980d35b0 2024-04-25T10:15:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.960+0000 7f6196d9d640 1 --2- 172.21.15.16:0/3087419013 >> v2:172.21.15.186:3300/0 conn(0x7f619815dce0 0x7f61980d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f6195d9b640 1 -- 172.21.15.16:0/3087419013 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1338336472 0 0) 0x7f6180003520 con 0x7f61980d35b0 2024-04-25T10:15:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f6195d9b640 1 -- 172.21.15.16:0/3087419013 >> v2:172.21.15.186:3300/0 conn(0x7f619815dce0 msgr2=0x7f61980d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f6195d9b640 1 --2- 172.21.15.16:0/3087419013 >> v2:172.21.15.186:3300/0 conn(0x7f619815dce0 0x7f61980d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f6195d9b640 1 -- 172.21.15.16:0/3087419013 >> v1:172.21.15.186:6789/0 conn(0x7f61980b5b60 legacy=0x7f61980b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f6195d9b640 1 -- 172.21.15.16:0/3087419013 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61980d2ad0 con 0x7f61980d35b0 2024-04-25T10:15:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f6195d9b640 1 -- 172.21.15.16:0/3087419013 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f618c001670 con 0x7f61980d35b0 2024-04-25T10:15:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f6195d9b640 1 -- 172.21.15.16:0/3087419013 >> v1:172.21.15.16:6789/0 conn(0x7f61980d35b0 legacy=0x7f61980b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f6195d9b640 1 --2- 172.21.15.16:0/3087419013 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61800039b0 0x7f6180003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f6195d9b640 1 --2- 172.21.15.16:0/3087419013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6180004800 0x7f6180004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f6195d9b640 1 --2- 172.21.15.16:0/3087419013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800051c0 0x7f618000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f6195d9b640 1 -- 172.21.15.16:0/3087419013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61980d27f0 con 0x7f61800051c0 2024-04-25T10:15:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f6195d9b640 1 -- 172.21.15.16:0/3087419013 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618000c060 con 0x7f6180004800 2024-04-25T10:15:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f6195d9b640 1 -- 172.21.15.16:0/3087419013 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618000c260 con 0x7f61800039b0 2024-04-25T10:15:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f619759e640 1 --2- 172.21.15.16:0/3087419013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800051c0 0x7f618000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f619659c640 1 --2- 172.21.15.16:0/3087419013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6180004800 0x7f6180004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f6196d9d640 1 --2- 172.21.15.16:0/3087419013 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61800039b0 0x7f6180003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f619759e640 1 -- 172.21.15.16:0/3087419013 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61800039b0 msgr2=0x7f6180003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f619759e640 1 --2- 172.21.15.16:0/3087419013 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61800039b0 0x7f6180003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f619759e640 1 -- 172.21.15.16:0/3087419013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6180004800 msgr2=0x7f6180004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f619759e640 1 --2- 172.21.15.16:0/3087419013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6180004800 0x7f6180004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.961+0000 7f619759e640 1 -- 172.21.15.16:0/3087419013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6180003520 con 0x7f61800051c0 2024-04-25T10:15:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.962+0000 7f619759e640 1 -- 172.21.15.16:0/3087419013 reap_dead start 2024-04-25T10:15:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.962+0000 7f619759e640 1 --2- 172.21.15.16:0/3087419013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800051c0 0x7f618000b9c0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f6188001e50 tx=0x7f6180005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6648a818bf06d1be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.962+0000 7f6195d9b640 1 -- 172.21.15.16:0/3087419013 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6188001a70 con 0x7f61800051c0 2024-04-25T10:15:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.962+0000 7f6195d9b640 1 -- 172.21.15.16:0/3087419013 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6188003970 con 0x7f61800051c0 2024-04-25T10:15:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.962+0000 7f6195d9b640 1 -- 172.21.15.16:0/3087419013 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6188003d60 con 0x7f61800051c0 2024-04-25T10:15:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.963+0000 7f619d1e2640 1 -- 172.21.15.16:0/3087419013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800051c0 msgr2=0x7f618000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.963+0000 7f619d1e2640 1 --2- 172.21.15.16:0/3087419013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800051c0 0x7f618000b9c0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f6188001e50 tx=0x7f6180005830 comp rx=0 tx=0).stop 2024-04-25T10:15:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.963+0000 7f619d1e2640 1 -- 172.21.15.16:0/3087419013 shutdown_connections 2024-04-25T10:15:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.963+0000 7f619d1e2640 1 --2- 172.21.15.16:0/3087419013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800051c0 0x7f618000b9c0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.963+0000 7f619d1e2640 1 -- 172.21.15.16:0/3087419013 >> 172.21.15.16:0/3087419013 conn(0x7f6198072900 msgr2=0x7f61980d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.963+0000 7f619d1e2640 1 -- 172.21.15.16:0/3087419013 shutdown_connections 2024-04-25T10:15:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.963+0000 7f619d1e2640 1 -- 172.21.15.16:0/3087419013 wait complete. 2024-04-25T10:15:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.964+0000 7f619d1e2640 1 Processor -- start 2024-04-25T10:15:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.964+0000 7f619d1e2640 1 -- start start 2024-04-25T10:15:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.965+0000 7f619d1e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800051c0 0x7f619815ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.965+0000 7f619d1e2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f619815eff0 0x7f6198153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.965+0000 7f619d1e2640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61980ccc30 0x7f61980cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.965+0000 7f619d1e2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61980b9820 con 0x7f61800051c0 2024-04-25T10:15:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.965+0000 7f619d1e2640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61980d3090 con 0x7f619815eff0 2024-04-25T10:15:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.965+0000 7f619d1e2640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61980b99a0 con 0x7f61980ccc30 2024-04-25T10:15:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.965+0000 7f6196d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800051c0 0x7f619815ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.965+0000 7f6196d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800051c0 0x7f619815ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35680/0 (socket says 172.21.15.16:35680) 2024-04-25T10:15:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.965+0000 7f6196d9d640 1 -- 172.21.15.16:0/1626241998 learned_addr learned my addr 172.21.15.16:0/1626241998 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.965+0000 7f619659c640 1 --2- 172.21.15.16:0/1626241998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f619815eff0 0x7f6198153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.965+0000 7f619759e640 1 --2- 172.21.15.16:0/1626241998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61980ccc30 0x7f61980cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.965+0000 7f6196d9d640 1 -- 172.21.15.16:0/1626241998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61980ccc30 msgr2=0x7f61980cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.965+0000 7f6196d9d640 1 --2- 172.21.15.16:0/1626241998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61980ccc30 0x7f61980cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.965+0000 7f6196d9d640 1 -- 172.21.15.16:0/1626241998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f619815eff0 msgr2=0x7f6198153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.965+0000 7f6196d9d640 1 --2- 172.21.15.16:0/1626241998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f619815eff0 0x7f6198153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.965+0000 7f6196d9d640 1 -- 172.21.15.16:0/1626241998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6198160450 con 0x7f61800051c0 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.965+0000 7f619759e640 1 --2- 172.21.15.16:0/1626241998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61980ccc30 0x7f61980cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.965+0000 7f619659c640 1 --2- 172.21.15.16:0/1626241998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f619815eff0 0x7f6198153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.966+0000 7f6196d9d640 1 --2- 172.21.15.16:0/1626241998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800051c0 0x7f619815ff10 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f61900118e0 tx=0x7f6190011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.966+0000 7f617f7fe640 1 -- 172.21.15.16:0/1626241998 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619000faf0 con 0x7f61800051c0 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.966+0000 7f617f7fe640 1 -- 172.21.15.16:0/1626241998 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619000fc90 con 0x7f61800051c0 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.966+0000 7f619d1e2640 1 -- 172.21.15.16:0/1626241998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61980cbb90 con 0x7f61800051c0 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.966+0000 7f617f7fe640 1 -- 172.21.15.16:0/1626241998 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190014440 con 0x7f61800051c0 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.966+0000 7f619d1e2640 1 -- 172.21.15.16:0/1626241998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61980cc150 con 0x7f61800051c0 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.967+0000 7f619d1e2640 1 -- 172.21.15.16:0/1626241998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61980749b0 con 0x7f61800051c0 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.967+0000 7f617f7fe640 1 -- 172.21.15.16:0/1626241998 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6190012040 con 0x7f61800051c0 2024-04-25T10:15:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.968+0000 7f617f7fe640 1 --2- 172.21.15.16:0/1626241998 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f618406b500 0x7f618406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.968+0000 7f617f7fe640 1 -- 172.21.15.16:0/1626241998 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f619008a990 con 0x7f61800051c0 2024-04-25T10:15:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.968+0000 7f619659c640 1 --2- 172.21.15.16:0/1626241998 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f618406b500 0x7f618406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.968+0000 7f619659c640 1 --2- 172.21.15.16:0/1626241998 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f618406b500 0x7f618406d9c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f61980cdaa0 tx=0x7f618c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:00.972+0000 7f617f7fe640 1 -- 172.21.15.16:0/1626241998 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6190019030 con 0x7f61800051c0 2024-04-25T10:15:01.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.156+0000 7f619d1e2640 1 -- 172.21.15.16:0/1626241998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f619815d410 con 0x7f61800051c0 2024-04-25T10:15:01.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.157+0000 7f617f7fe640 1 -- 172.21.15.16:0/1626241998 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+17026 (secure 0 0 0) 0x7f61900590c0 con 0x7f61800051c0 2024-04-25T10:15:01.161 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:15:01.161 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":119,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:14:54.519874+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:25.025480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:25.726747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1d","mappings":[{"from":4,"to":3}]},{"pgid":"17.1e","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/2697110279":"2024-04-26T10:14:50.050260+0000","172.21.15.16:6833/2697110279":"2024-04-26T10:14:50.050260+0000","172.21.15.16:6835/123396308":"2024-04-26T10:14:39.639508+0000","172.21.15.16:6833/123396308":"2024-04-26T10:14:39.639508+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.159+0000 7f619d1e2640 1 -- 172.21.15.16:0/1626241998 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f618406b500 msgr2=0x7f618406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.159+0000 7f619d1e2640 1 --2- 172.21.15.16:0/1626241998 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f618406b500 0x7f618406d9c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f61980cdaa0 tx=0x7f618c009040 comp rx=0 tx=0).stop 2024-04-25T10:15:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.159+0000 7f619d1e2640 1 -- 172.21.15.16:0/1626241998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800051c0 msgr2=0x7f619815ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.159+0000 7f619d1e2640 1 --2- 172.21.15.16:0/1626241998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800051c0 0x7f619815ff10 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f61900118e0 tx=0x7f6190011da0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.159+0000 7f619d1e2640 1 -- 172.21.15.16:0/1626241998 shutdown_connections 2024-04-25T10:15:01.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.159+0000 7f619d1e2640 1 --2- 172.21.15.16:0/1626241998 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f618406b500 0x7f618406d9c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.159+0000 7f619d1e2640 1 --2- 172.21.15.16:0/1626241998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61980ccc30 0x7f61980cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.159+0000 7f619d1e2640 1 --2- 172.21.15.16:0/1626241998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f619815eff0 0x7f6198153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.159+0000 7f619d1e2640 1 --2- 172.21.15.16:0/1626241998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61800051c0 0x7f619815ff10 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.159+0000 7f619d1e2640 1 -- 172.21.15.16:0/1626241998 >> 172.21.15.16:0/1626241998 conn(0x7f6198072900 msgr2=0x7f61980b48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:01.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.159+0000 7f619d1e2640 1 -- 172.21.15.16:0/1626241998 shutdown_connections 2024-04-25T10:15:01.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.159+0000 7f619d1e2640 1 -- 172.21.15.16:0/1626241998 wait complete. 2024-04-25T10:15:01.175 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:04:57.825688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:14:25.025480+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:14:25.726747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:15:01.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:15:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.275+0000 7f2499812640 1 Processor -- start 2024-04-25T10:15:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.275+0000 7f2499812640 1 -- start start 2024-04-25T10:15:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.275+0000 7f2499812640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24940b42c0 0x7f249415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.275+0000 7f2499812640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f24940b46a0 0x7f24941549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.275+0000 7f2499812640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249415b370 con 0x7f249415aef0 2024-04-25T10:15:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.275+0000 7f2499812640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24940d05d0 con 0x7f24940b42c0 2024-04-25T10:15:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.275+0000 7f2499812640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f249415b650 con 0x7f24940b46a0 2024-04-25T10:15:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.276+0000 7f2492ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24940b42c0 0x7f249415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.276+0000 7f2492ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24940b42c0 0x7f249415ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35684/0 (socket says 172.21.15.16:35684) 2024-04-25T10:15:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.276+0000 7f2492ffd640 1 -- 172.21.15.16:0/154810544 learned_addr learned my addr 172.21.15.16:0/154810544 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.276+0000 7f2492ffd640 1 -- 172.21.15.16:0/154810544 >> v2:172.21.15.186:3300/0 conn(0x7f24940b46a0 msgr2=0x7f24941549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.276+0000 7f24937fe640 1 --2- 172.21.15.16:0/154810544 >> v2:172.21.15.186:3300/0 conn(0x7f24940b46a0 0x7f24941549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.276+0000 7f2492ffd640 1 --2- 172.21.15.16:0/154810544 >> v2:172.21.15.186:3300/0 conn(0x7f24940b46a0 0x7f24941549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.276+0000 7f2492ffd640 1 -- 172.21.15.16:0/154810544 >> v1:172.21.15.186:6789/0 conn(0x7f249415aef0 legacy=0x7f24940cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.276+0000 7f2492ffd640 1 -- 172.21.15.16:0/154810544 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249415b930 con 0x7f24940b42c0 2024-04-25T10:15:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.277+0000 7f2492ffd640 1 --2- 172.21.15.16:0/154810544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24940b42c0 0x7f249415ef40 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f249406e6b0 tx=0x7f248402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1200efa1aee53d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.277+0000 7f2491ffb640 1 -- 172.21.15.16:0/154810544 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484038070 con 0x7f24940b42c0 2024-04-25T10:15:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.277+0000 7f2491ffb640 1 -- 172.21.15.16:0/154810544 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2484009600 con 0x7f24940b42c0 2024-04-25T10:15:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.277+0000 7f2491ffb640 1 -- 172.21.15.16:0/154810544 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24840099d0 con 0x7f24940b42c0 2024-04-25T10:15:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.277+0000 7f2499812640 1 -- 172.21.15.16:0/154810544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24940b42c0 msgr2=0x7f249415ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.277+0000 7f2499812640 1 --2- 172.21.15.16:0/154810544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24940b42c0 0x7f249415ef40 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f249406e6b0 tx=0x7f248402c7e0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.277+0000 7f2499812640 1 -- 172.21.15.16:0/154810544 shutdown_connections 2024-04-25T10:15:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.277+0000 7f2499812640 1 --2- 172.21.15.16:0/154810544 >> v2:172.21.15.186:3300/0 conn(0x7f24940b46a0 0x7f24941549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.277+0000 7f2499812640 1 --2- 172.21.15.16:0/154810544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24940b42c0 0x7f249415ef40 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.277+0000 7f2499812640 1 -- 172.21.15.16:0/154810544 >> 172.21.15.16:0/154810544 conn(0x7f249406aba0 msgr2=0x7f249406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.277+0000 7f2499812640 1 -- 172.21.15.16:0/154810544 shutdown_connections 2024-04-25T10:15:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.277+0000 7f2499812640 1 -- 172.21.15.16:0/154810544 wait complete. 2024-04-25T10:15:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.279+0000 7f2499812640 1 Processor -- start 2024-04-25T10:15:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.279+0000 7f2499812640 1 -- start start 2024-04-25T10:15:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.279+0000 7f2499812640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24940b42c0 0x7f2494141180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.279+0000 7f2499812640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24940b46a0 0x7f249415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.279+0000 7f2499812640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f249415aef0 0x7f249414c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.279+0000 7f2499812640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249406ec60 con 0x7f24940b46a0 2024-04-25T10:15:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.279+0000 7f2499812640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249406eae0 con 0x7f24940b42c0 2024-04-25T10:15:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.279+0000 7f2499812640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2494154f10 con 0x7f249415aef0 2024-04-25T10:15:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.280+0000 7f24927fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24940b46a0 0x7f249415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.280+0000 7f24927fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24940b46a0 0x7f249415a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35690/0 (socket says 172.21.15.16:35690) 2024-04-25T10:15:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.280+0000 7f24927fc640 1 -- 172.21.15.16:0/2625734812 learned_addr learned my addr 172.21.15.16:0/2625734812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.280+0000 7f24937fe640 1 --2- 172.21.15.16:0/2625734812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f249415aef0 0x7f249414c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.280+0000 7f2492ffd640 1 --2- 172.21.15.16:0/2625734812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24940b42c0 0x7f2494141180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.280+0000 7f24927fc640 1 -- 172.21.15.16:0/2625734812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f249415aef0 msgr2=0x7f249414c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.280+0000 7f24927fc640 1 --2- 172.21.15.16:0/2625734812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f249415aef0 0x7f249414c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.280+0000 7f24927fc640 1 -- 172.21.15.16:0/2625734812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24940b42c0 msgr2=0x7f2494141180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.280+0000 7f24927fc640 1 --2- 172.21.15.16:0/2625734812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24940b42c0 0x7f2494141180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.280+0000 7f24927fc640 1 -- 172.21.15.16:0/2625734812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249414b740 con 0x7f24940b46a0 2024-04-25T10:15:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.280+0000 7f24927fc640 1 --2- 172.21.15.16:0/2625734812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24940b46a0 0x7f249415a290 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f248800b3c0 tx=0x7f248800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.280+0000 7f247bfff640 1 -- 172.21.15.16:0/2625734812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2488011830 con 0x7f24940b46a0 2024-04-25T10:15:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.280+0000 7f247bfff640 1 -- 172.21.15.16:0/2625734812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2488016070 con 0x7f24940b46a0 2024-04-25T10:15:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.280+0000 7f2499812640 1 -- 172.21.15.16:0/2625734812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249414ba20 con 0x7f24940b46a0 2024-04-25T10:15:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.280+0000 7f247bfff640 1 -- 172.21.15.16:0/2625734812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2488011c30 con 0x7f24940b46a0 2024-04-25T10:15:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.280+0000 7f2499812640 1 -- 172.21.15.16:0/2625734812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249414bfe0 con 0x7f24940b46a0 2024-04-25T10:15:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.281+0000 7f2499812640 1 -- 172.21.15.16:0/2625734812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24940750b0 con 0x7f24940b46a0 2024-04-25T10:15:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.282+0000 7f247bfff640 1 -- 172.21.15.16:0/2625734812 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f248801b460 con 0x7f24940b46a0 2024-04-25T10:15:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.282+0000 7f247bfff640 1 --2- 172.21.15.16:0/2625734812 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f248006b4d0 0x7f248006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.282+0000 7f247bfff640 1 -- 172.21.15.16:0/2625734812 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f248808a040 con 0x7f24940b46a0 2024-04-25T10:15:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.283+0000 7f2492ffd640 1 --2- 172.21.15.16:0/2625734812 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f248006b4d0 0x7f248006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.283+0000 7f2492ffd640 1 --2- 172.21.15.16:0/2625734812 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f248006b4d0 0x7f248006d990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2484035000 tx=0x7f248400b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.287+0000 7f247bfff640 1 -- 172.21.15.16:0/2625734812 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2488058dd0 con 0x7f24940b46a0 2024-04-25T10:15:01.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.472+0000 7f2499812640 1 -- 172.21.15.16:0/2625734812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f249406e700 con 0x7f24940b46a0 2024-04-25T10:15:01.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.472+0000 7f247bfff640 1 -- 172.21.15.16:0/2625734812 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7f24880586d0 con 0x7f24940b46a0 2024-04-25T10:15:01.476 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.474+0000 7f2499812640 1 -- 172.21.15.16:0/2625734812 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f248006b4d0 msgr2=0x7f248006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.474+0000 7f2499812640 1 --2- 172.21.15.16:0/2625734812 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f248006b4d0 0x7f248006d990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2484035000 tx=0x7f248400b000 comp rx=0 tx=0).stop 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.474+0000 7f2499812640 1 -- 172.21.15.16:0/2625734812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24940b46a0 msgr2=0x7f249415a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.474+0000 7f2499812640 1 --2- 172.21.15.16:0/2625734812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24940b46a0 0x7f249415a290 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f248800b3c0 tx=0x7f248800b880 comp rx=0 tx=0).stop 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.474+0000 7f2499812640 1 -- 172.21.15.16:0/2625734812 shutdown_connections 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.474+0000 7f2499812640 1 --2- 172.21.15.16:0/2625734812 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f248006b4d0 0x7f248006d990 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.474+0000 7f2499812640 1 --2- 172.21.15.16:0/2625734812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f249415aef0 0x7f249414c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.474+0000 7f2499812640 1 --2- 172.21.15.16:0/2625734812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24940b46a0 0x7f249415a290 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.474+0000 7f2499812640 1 --2- 172.21.15.16:0/2625734812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24940b42c0 0x7f2494141180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.474+0000 7f2499812640 1 -- 172.21.15.16:0/2625734812 >> 172.21.15.16:0/2625734812 conn(0x7f249406aba0 msgr2=0x7f249415c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.474+0000 7f2499812640 1 -- 172.21.15.16:0/2625734812 shutdown_connections 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.474+0000 7f2499812640 1 -- 172.21.15.16:0/2625734812 wait complete. 2024-04-25T10:15:01.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:15:01.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.588+0000 7f3b32953640 1 Processor -- start 2024-04-25T10:15:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.588+0000 7f3b32953640 1 -- start start 2024-04-25T10:15:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.589+0000 7f3b32953640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3b2c15e040 0x7f3b2c0cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.589+0000 7f3b32953640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2c0cc7e0 con 0x7f3b2c0cee20 2024-04-25T10:15:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.589+0000 7f3b32953640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2c0cb350 con 0x7f3b2c0ce410 2024-04-25T10:15:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.589+0000 7f3b32953640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b2c0ce9d0 con 0x7f3b2c15e040 2024-04-25T10:15:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.589+0000 7f3b2bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3b2c15e040 0x7f3b2c0cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.589+0000 7f3b30ef0640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3b2c0cee20 0x7f3b2c0ad600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55748/0 (socket says 172.21.15.16:55748) 2024-04-25T10:15:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.589+0000 7f3b30ef0640 1 -- 172.21.15.16:0/1674290347 learned_addr learned my addr 172.21.15.16:0/1674290347 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.590+0000 7f3b2bfff640 1 -- 172.21.15.16:0/1674290347 >> v1:172.21.15.186:6789/0 conn(0x7f3b2c0ce410 legacy=0x7f3b2c0cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.590+0000 7f3b2bfff640 1 -- 172.21.15.16:0/1674290347 >> v1:172.21.15.16:6789/0 conn(0x7f3b2c0cee20 legacy=0x7f3b2c0ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.590+0000 7f3b2bfff640 1 -- 172.21.15.16:0/1674290347 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2c0cb5f0 con 0x7f3b2c15e040 2024-04-25T10:15:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.590+0000 7f3b2bfff640 1 --2- 172.21.15.16:0/1674290347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b2c15e040 0x7f3b2c0cdbb0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0097d0 tx=0x7f3b1c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0a116b8bd67ecb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.590+0000 7f3b2affd640 1 -- 172.21.15.16:0/1674290347 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c037070 con 0x7f3b2c15e040 2024-04-25T10:15:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.590+0000 7f3b2affd640 1 -- 172.21.15.16:0/1674290347 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c032440 con 0x7f3b2c15e040 2024-04-25T10:15:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.591+0000 7f3b2affd640 1 -- 172.21.15.16:0/1674290347 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c032810 con 0x7f3b2c15e040 2024-04-25T10:15:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.591+0000 7f3b32953640 1 -- 172.21.15.16:0/1674290347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b2c15e040 msgr2=0x7f3b2c0cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.591+0000 7f3b32953640 1 --2- 172.21.15.16:0/1674290347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b2c15e040 0x7f3b2c0cdbb0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0097d0 tx=0x7f3b1c02a210 comp rx=0 tx=0).stop 2024-04-25T10:15:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.591+0000 7f3b32953640 1 -- 172.21.15.16:0/1674290347 shutdown_connections 2024-04-25T10:15:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.591+0000 7f3b32953640 1 --2- 172.21.15.16:0/1674290347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b2c15e040 0x7f3b2c0cdbb0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.591+0000 7f3b32953640 1 -- 172.21.15.16:0/1674290347 >> 172.21.15.16:0/1674290347 conn(0x7f3b2c0b2390 msgr2=0x7f3b2c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.591+0000 7f3b32953640 1 -- 172.21.15.16:0/1674290347 shutdown_connections 2024-04-25T10:15:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.591+0000 7f3b32953640 1 -- 172.21.15.16:0/1674290347 wait complete. 2024-04-25T10:15:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.592+0000 7f3b32953640 1 Processor -- start 2024-04-25T10:15:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.592+0000 7f3b32953640 1 -- start start 2024-04-25T10:15:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.593+0000 7f3b32953640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3b2c0ce410 0x7f3b2c158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.593+0000 7f3b32953640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b2c0cee20 0x7f3b2c0c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.593+0000 7f3b32953640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b2c15e040 0x7f3b2c0b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.593+0000 7f3b32953640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c15e420 con 0x7f3b2c0cee20 2024-04-25T10:15:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.593+0000 7f3b32953640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c0cbbb0 con 0x7f3b2c15e040 2024-04-25T10:15:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.593+0000 7f3b32953640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c0ce0f0 con 0x7f3b2c0ce410 2024-04-25T10:15:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.593+0000 7f3b2b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b2c0cee20 0x7f3b2c0c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.593+0000 7f3b2b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b2c0cee20 0x7f3b2c0c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35706/0 (socket says 172.21.15.16:35706) 2024-04-25T10:15:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.593+0000 7f3b2b7fe640 1 -- 172.21.15.16:0/1629702052 learned_addr learned my addr 172.21.15.16:0/1629702052 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.593+0000 7f3b30ef0640 1 --2- 172.21.15.16:0/1629702052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b2c15e040 0x7f3b2c0b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.593+0000 7f3b2bfff640 1 --2- 172.21.15.16:0/1629702052 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3b2c0ce410 0x7f3b2c158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.594+0000 7f3b2b7fe640 1 -- 172.21.15.16:0/1629702052 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3b2c0ce410 msgr2=0x7f3b2c158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.594+0000 7f3b2b7fe640 1 --2- 172.21.15.16:0/1629702052 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3b2c0ce410 0x7f3b2c158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.594+0000 7f3b2b7fe640 1 -- 172.21.15.16:0/1629702052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b2c15e040 msgr2=0x7f3b2c0b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.594+0000 7f3b2b7fe640 1 --2- 172.21.15.16:0/1629702052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b2c15e040 0x7f3b2c0b2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.594+0000 7f3b2b7fe640 1 -- 172.21.15.16:0/1629702052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2c0c9ae0 con 0x7f3b2c0cee20 2024-04-25T10:15:01.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.594+0000 7f3b2bfff640 1 --2- 172.21.15.16:0/1629702052 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3b2c0ce410 0x7f3b2c158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:01.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.594+0000 7f3b2b7fe640 1 --2- 172.21.15.16:0/1629702052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b2c0cee20 0x7f3b2c0c9370 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f3b2000b3f0 tx=0x7f3b2000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.594+0000 7f3b28ff9640 1 -- 172.21.15.16:0/1629702052 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20015020 con 0x7f3b2c0cee20 2024-04-25T10:15:01.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.594+0000 7f3b28ff9640 1 -- 172.21.15.16:0/1629702052 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b20011830 con 0x7f3b2c0cee20 2024-04-25T10:15:01.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.594+0000 7f3b32953640 1 -- 172.21.15.16:0/1629702052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2c0c9dc0 con 0x7f3b2c0cee20 2024-04-25T10:15:01.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.594+0000 7f3b28ff9640 1 -- 172.21.15.16:0/1629702052 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20011c20 con 0x7f3b2c0cee20 2024-04-25T10:15:01.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.594+0000 7f3b32953640 1 -- 172.21.15.16:0/1629702052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2c15ad50 con 0x7f3b2c0cee20 2024-04-25T10:15:01.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.595+0000 7f3b32953640 1 -- 172.21.15.16:0/1629702052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2c0750b0 con 0x7f3b2c0cee20 2024-04-25T10:15:01.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.595+0000 7f3b28ff9640 1 -- 172.21.15.16:0/1629702052 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b20018020 con 0x7f3b2c0cee20 2024-04-25T10:15:01.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.596+0000 7f3b28ff9640 1 --2- 172.21.15.16:0/1629702052 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b1806b4d0 0x7f3b1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.596+0000 7f3b28ff9640 1 -- 172.21.15.16:0/1629702052 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f3b2008a060 con 0x7f3b2c0cee20 2024-04-25T10:15:01.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.596+0000 7f3b2bfff640 1 --2- 172.21.15.16:0/1629702052 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b1806b4d0 0x7f3b1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.597+0000 7f3b2bfff640 1 --2- 172.21.15.16:0/1629702052 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b1806b4d0 0x7f3b1806d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c034000 tx=0x7f3b1c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.600+0000 7f3b28ff9640 1 -- 172.21.15.16:0/1629702052 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b2001f030 con 0x7f3b2c0cee20 2024-04-25T10:15:01.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.778+0000 7f3b32953640 1 -- 172.21.15.16:0/1629702052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3b2c15af20 con 0x7f3b2c0cee20 2024-04-25T10:15:01.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.778+0000 7f3b28ff9640 1 -- 172.21.15.16:0/1629702052 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3b200172c0 con 0x7f3b2c0cee20 2024-04-25T10:15:01.782 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-25T10:15:01.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.780+0000 7f3b32953640 1 -- 172.21.15.16:0/1629702052 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b1806b4d0 msgr2=0x7f3b1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.780+0000 7f3b32953640 1 --2- 172.21.15.16:0/1629702052 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b1806b4d0 0x7f3b1806d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c034000 tx=0x7f3b1c02e000 comp rx=0 tx=0).stop 2024-04-25T10:15:01.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.780+0000 7f3b32953640 1 -- 172.21.15.16:0/1629702052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b2c0cee20 msgr2=0x7f3b2c0c9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.780+0000 7f3b32953640 1 --2- 172.21.15.16:0/1629702052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b2c0cee20 0x7f3b2c0c9370 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f3b2000b3f0 tx=0x7f3b2000b8b0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.780+0000 7f3b32953640 1 -- 172.21.15.16:0/1629702052 shutdown_connections 2024-04-25T10:15:01.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.780+0000 7f3b32953640 1 --2- 172.21.15.16:0/1629702052 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f3b1806b4d0 0x7f3b1806d990 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.780+0000 7f3b32953640 1 --2- 172.21.15.16:0/1629702052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b2c15e040 0x7f3b2c0b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.780+0000 7f3b32953640 1 --2- 172.21.15.16:0/1629702052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b2c0cee20 0x7f3b2c0c9370 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.780+0000 7f3b32953640 1 --2- 172.21.15.16:0/1629702052 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3b2c0ce410 0x7f3b2c158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.780+0000 7f3b32953640 1 -- 172.21.15.16:0/1629702052 >> 172.21.15.16:0/1629702052 conn(0x7f3b2c0b2390 msgr2=0x7f3b2c0c6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.780+0000 7f3b32953640 1 -- 172.21.15.16:0/1629702052 shutdown_connections 2024-04-25T10:15:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.780+0000 7f3b32953640 1 -- 172.21.15.16:0/1629702052 wait complete. 2024-04-25T10:15:01.795 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:15:01.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.894+0000 7f1b12bf5640 1 Processor -- start 2024-04-25T10:15:01.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.894+0000 7f1b12bf5640 1 -- start start 2024-04-25T10:15:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.895+0000 7f1b12bf5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b0c15b7b0 0x7f1b0c0ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.895+0000 7f1b12bf5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1b0c0cf760 0x7f1b0c0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.895+0000 7f1b12bf5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b0c15f000 con 0x7f1b0c15e110 2024-04-25T10:15:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.895+0000 7f1b12bf5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0c0ced00 con 0x7f1b0c15b7b0 2024-04-25T10:15:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.895+0000 7f1b12bf5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0c0ce330 con 0x7f1b0c0cf760 2024-04-25T10:15:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.895+0000 7f1b10991640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b0c15b7b0 0x7f1b0c0ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.895+0000 7f1b11192640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1b0c15e110 0x7f1b0c15e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55760/0 (socket says 172.21.15.16:55760) 2024-04-25T10:15:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.895+0000 7f1b11192640 1 -- 172.21.15.16:0/3315142675 learned_addr learned my addr 172.21.15.16:0/3315142675 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.896+0000 7f1b10991640 1 -- 172.21.15.16:0/3315142675 >> v2:172.21.15.186:3300/0 conn(0x7f1b0c0cf760 msgr2=0x7f1b0c0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.896+0000 7f1b0bfff640 1 --2- 172.21.15.16:0/3315142675 >> v2:172.21.15.186:3300/0 conn(0x7f1b0c0cf760 0x7f1b0c0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.896+0000 7f1b10991640 1 --2- 172.21.15.16:0/3315142675 >> v2:172.21.15.186:3300/0 conn(0x7f1b0c0cf760 0x7f1b0c0cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.896+0000 7f1b0b7fe640 1 -- 172.21.15.16:0/3315142675 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3678162425 0 0) 0x7f1b0c15f000 con 0x7f1b0c15e110 2024-04-25T10:15:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.896+0000 7f1b10991640 1 -- 172.21.15.16:0/3315142675 >> v1:172.21.15.16:6789/0 conn(0x7f1b0c15e110 legacy=0x7f1b0c15e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.896+0000 7f1b10991640 1 -- 172.21.15.16:0/3315142675 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c15ed20 con 0x7f1b0c15b7b0 2024-04-25T10:15:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.896+0000 7f1b0bfff640 1 --2- 172.21.15.16:0/3315142675 >> v2:172.21.15.186:3300/0 conn(0x7f1b0c0cf760 0x7f1b0c0cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.896+0000 7f1b10991640 1 --2- 172.21.15.16:0/3315142675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b0c15b7b0 0x7f1b0c0ce7c0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0097a0 tx=0x7f1afc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6304769be7fd8942 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.896+0000 7f1b0b7fe640 1 -- 172.21.15.16:0/3315142675 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc037070 con 0x7f1b0c15b7b0 2024-04-25T10:15:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.896+0000 7f1b0b7fe640 1 -- 172.21.15.16:0/3315142675 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1afc032440 con 0x7f1b0c15b7b0 2024-04-25T10:15:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.896+0000 7f1b0b7fe640 1 -- 172.21.15.16:0/3315142675 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc032810 con 0x7f1b0c15b7b0 2024-04-25T10:15:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.897+0000 7f1b12bf5640 1 -- 172.21.15.16:0/3315142675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b0c15b7b0 msgr2=0x7f1b0c0ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.897+0000 7f1b12bf5640 1 --2- 172.21.15.16:0/3315142675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b0c15b7b0 0x7f1b0c0ce7c0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0097a0 tx=0x7f1afc02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.897+0000 7f1b12bf5640 1 -- 172.21.15.16:0/3315142675 shutdown_connections 2024-04-25T10:15:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.897+0000 7f1b12bf5640 1 --2- 172.21.15.16:0/3315142675 >> v2:172.21.15.186:3300/0 conn(0x7f1b0c0cf760 0x7f1b0c0cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.897+0000 7f1b12bf5640 1 --2- 172.21.15.16:0/3315142675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b0c15b7b0 0x7f1b0c0ce7c0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.897+0000 7f1b12bf5640 1 -- 172.21.15.16:0/3315142675 >> 172.21.15.16:0/3315142675 conn(0x7f1b0c06f4a0 msgr2=0x7f1b0c06f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.897+0000 7f1b12bf5640 1 -- 172.21.15.16:0/3315142675 shutdown_connections 2024-04-25T10:15:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.897+0000 7f1b12bf5640 1 -- 172.21.15.16:0/3315142675 wait complete. 2024-04-25T10:15:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.898+0000 7f1b12bf5640 1 Processor -- start 2024-04-25T10:15:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.898+0000 7f1b12bf5640 1 -- start start 2024-04-25T10:15:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.899+0000 7f1b12bf5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b0c0cf760 0x7f1b0c0c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.899+0000 7f1b12bf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b0c15b7b0 0x7f1b0c1565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.899+0000 7f1b12bf5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0c15e110 0x7f1b0c0cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.899+0000 7f1b12bf5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c1619f0 con 0x7f1b0c15b7b0 2024-04-25T10:15:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.899+0000 7f1b12bf5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0cdfc0 con 0x7f1b0c0cf760 2024-04-25T10:15:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.899+0000 7f1b12bf5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0cc080 con 0x7f1b0c15e110 2024-04-25T10:15:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.899+0000 7f1b0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b0c15b7b0 0x7f1b0c1565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.899+0000 7f1b0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b0c15b7b0 0x7f1b0c1565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35722/0 (socket says 172.21.15.16:35722) 2024-04-25T10:15:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.899+0000 7f1b0bfff640 1 -- 172.21.15.16:0/2144510368 learned_addr learned my addr 172.21.15.16:0/2144510368 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.899+0000 7f1b10991640 1 --2- 172.21.15.16:0/2144510368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b0c0cf760 0x7f1b0c0c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.899+0000 7f1b11192640 1 --2- 172.21.15.16:0/2144510368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0c15e110 0x7f1b0c0cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.900+0000 7f1b0bfff640 1 -- 172.21.15.16:0/2144510368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0c15e110 msgr2=0x7f1b0c0cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.900+0000 7f1b0bfff640 1 --2- 172.21.15.16:0/2144510368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0c15e110 0x7f1b0c0cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.900+0000 7f1b0bfff640 1 -- 172.21.15.16:0/2144510368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b0c0cf760 msgr2=0x7f1b0c0c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.900+0000 7f1b0bfff640 1 --2- 172.21.15.16:0/2144510368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b0c0cf760 0x7f1b0c0c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.900+0000 7f1b0bfff640 1 -- 172.21.15.16:0/2144510368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c0b33a0 con 0x7f1b0c15b7b0 2024-04-25T10:15:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.900+0000 7f1b0bfff640 1 --2- 172.21.15.16:0/2144510368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b0c15b7b0 0x7f1b0c1565d0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f1b0000c7f0 tx=0x7f1b0000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.900+0000 7f1b097fa640 1 -- 172.21.15.16:0/2144510368 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b00015070 con 0x7f1b0c15b7b0 2024-04-25T10:15:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.900+0000 7f1b097fa640 1 -- 172.21.15.16:0/2144510368 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b00007a30 con 0x7f1b0c15b7b0 2024-04-25T10:15:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.900+0000 7f1b12bf5640 1 -- 172.21.15.16:0/2144510368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0c0b3680 con 0x7f1b0c15b7b0 2024-04-25T10:15:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.900+0000 7f1b097fa640 1 -- 172.21.15.16:0/2144510368 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b00007e00 con 0x7f1b0c15b7b0 2024-04-25T10:15:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.900+0000 7f1b12bf5640 1 -- 172.21.15.16:0/2144510368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0c0b5310 con 0x7f1b0c15b7b0 2024-04-25T10:15:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.901+0000 7f1b12bf5640 1 -- 172.21.15.16:0/2144510368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b0c0750b0 con 0x7f1b0c15b7b0 2024-04-25T10:15:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.902+0000 7f1b097fa640 1 -- 172.21.15.16:0/2144510368 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b00018070 con 0x7f1b0c15b7b0 2024-04-25T10:15:01.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.902+0000 7f1b097fa640 1 --2- 172.21.15.16:0/2144510368 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1af806b4d0 0x7f1af806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.902+0000 7f1b097fa640 1 -- 172.21.15.16:0/2144510368 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f1b0008a120 con 0x7f1b0c15b7b0 2024-04-25T10:15:01.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.902+0000 7f1b10991640 1 --2- 172.21.15.16:0/2144510368 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1af806b4d0 0x7f1af806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.903+0000 7f1b10991640 1 --2- 172.21.15.16:0/2144510368 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1af806b4d0 0x7f1af806d990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1afc034000 tx=0x7f1afc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:01.906+0000 7f1b097fa640 1 -- 172.21.15.16:0/2144510368 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b000588f0 con 0x7f1b0c15b7b0 2024-04-25T10:15:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.088+0000 7f1b12bf5640 1 -- 172.21.15.16:0/2144510368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b0c0b54e0 con 0x7f1b0c15b7b0 2024-04-25T10:15:02.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.090+0000 7f1b097fa640 1 -- 172.21.15.16:0/2144510368 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b0005c990 con 0x7f1b0c15b7b0 2024-04-25T10:15:02.094 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-04-25T10:15:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.092+0000 7f1b12bf5640 1 -- 172.21.15.16:0/2144510368 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1af806b4d0 msgr2=0x7f1af806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.092+0000 7f1b12bf5640 1 --2- 172.21.15.16:0/2144510368 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1af806b4d0 0x7f1af806d990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1afc034000 tx=0x7f1afc02e000 comp rx=0 tx=0).stop 2024-04-25T10:15:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.092+0000 7f1b12bf5640 1 -- 172.21.15.16:0/2144510368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b0c15b7b0 msgr2=0x7f1b0c1565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.092+0000 7f1b12bf5640 1 --2- 172.21.15.16:0/2144510368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b0c15b7b0 0x7f1b0c1565d0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f1b0000c7f0 tx=0x7f1b0000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.092+0000 7f1b12bf5640 1 -- 172.21.15.16:0/2144510368 shutdown_connections 2024-04-25T10:15:02.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.092+0000 7f1b12bf5640 1 --2- 172.21.15.16:0/2144510368 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f1af806b4d0 0x7f1af806d990 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.092+0000 7f1b12bf5640 1 --2- 172.21.15.16:0/2144510368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1b0c15e110 0x7f1b0c0cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.092+0000 7f1b12bf5640 1 --2- 172.21.15.16:0/2144510368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b0c15b7b0 0x7f1b0c1565d0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.092+0000 7f1b12bf5640 1 --2- 172.21.15.16:0/2144510368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b0c0cf760 0x7f1b0c0c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.092+0000 7f1b12bf5640 1 -- 172.21.15.16:0/2144510368 >> 172.21.15.16:0/2144510368 conn(0x7f1b0c06f4a0 msgr2=0x7f1b0c0cc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:02.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.092+0000 7f1b12bf5640 1 -- 172.21.15.16:0/2144510368 shutdown_connections 2024-04-25T10:15:02.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.092+0000 7f1b12bf5640 1 -- 172.21.15.16:0/2144510368 wait complete. 2024-04-25T10:15:02.107 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.204+0000 7f0093570640 1 Processor -- start 2024-04-25T10:15:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.204+0000 7f0093570640 1 -- start start 2024-04-25T10:15:02.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.205+0000 7f0093570640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f008c0d35b0 0x7f008c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.205+0000 7f0093570640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f008c0b5990 0x7f008c0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.205+0000 7f0093570640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f008c15e0c0 con 0x7f008c15dce0 2024-04-25T10:15:02.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.205+0000 7f0093570640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008c0d2510 con 0x7f008c0d35b0 2024-04-25T10:15:02.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.205+0000 7f0093570640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008c0d27f0 con 0x7f008c0b5990 2024-04-25T10:15:02.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.205+0000 7f0090b0b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f008c0d35b0 0x7f008c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.205+0000 7f0090b0b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f008c0d35b0 0x7f008c0b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35724/0 (socket says 172.21.15.16:35724) 2024-04-25T10:15:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.205+0000 7f0090b0b640 1 -- 172.21.15.16:0/858092639 learned_addr learned my addr 172.21.15.16:0/858092639 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.205+0000 7f0090b0b640 1 -- 172.21.15.16:0/858092639 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f008c0b5990 msgr2=0x7f008c0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.205+0000 7f0090b0b640 1 --2- 172.21.15.16:0/858092639 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f008c0b5990 0x7f008c0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.205+0000 7f0090b0b640 1 -- 172.21.15.16:0/858092639 >> v1:172.21.15.186:6789/0 conn(0x7f008c15dce0 legacy=0x7f008c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.205+0000 7f0090b0b640 1 -- 172.21.15.16:0/858092639 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008c0d2ad0 con 0x7f008c0d35b0 2024-04-25T10:15:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.206+0000 7f0090b0b640 1 --2- 172.21.15.16:0/858092639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f008c0d35b0 0x7f008c0b5450 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f007c009870 tx=0x7f007c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb9444f5187aaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.206+0000 7f0083fff640 1 -- 172.21.15.16:0/858092639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c037070 con 0x7f008c0d35b0 2024-04-25T10:15:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.206+0000 7f0083fff640 1 -- 172.21.15.16:0/858092639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007c032440 con 0x7f008c0d35b0 2024-04-25T10:15:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.206+0000 7f0083fff640 1 -- 172.21.15.16:0/858092639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c032810 con 0x7f008c0d35b0 2024-04-25T10:15:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.207+0000 7f0093570640 1 -- 172.21.15.16:0/858092639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f008c0d35b0 msgr2=0x7f008c0b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.207+0000 7f0093570640 1 --2- 172.21.15.16:0/858092639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f008c0d35b0 0x7f008c0b5450 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f007c009870 tx=0x7f007c02a210 comp rx=0 tx=0).stop 2024-04-25T10:15:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.207+0000 7f0093570640 1 -- 172.21.15.16:0/858092639 shutdown_connections 2024-04-25T10:15:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.207+0000 7f0093570640 1 --2- 172.21.15.16:0/858092639 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f008c0b5990 0x7f008c0b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.207+0000 7f0093570640 1 --2- 172.21.15.16:0/858092639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f008c0d35b0 0x7f008c0b5450 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.207+0000 7f0093570640 1 -- 172.21.15.16:0/858092639 >> 172.21.15.16:0/858092639 conn(0x7f008c072900 msgr2=0x7f008c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.207+0000 7f0093570640 1 -- 172.21.15.16:0/858092639 shutdown_connections 2024-04-25T10:15:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.207+0000 7f0093570640 1 -- 172.21.15.16:0/858092639 wait complete. 2024-04-25T10:15:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.208+0000 7f0093570640 1 Processor -- start 2024-04-25T10:15:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.208+0000 7f0093570640 1 -- start start 2024-04-25T10:15:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f0093570640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f008c0b5990 0x7f008c154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f0093570640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f008c0d35b0 0x7f008c15e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f0093570640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f008c15dce0 0x7f008c0c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f0093570640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c0d2160 con 0x7f008c0b5990 2024-04-25T10:15:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f0093570640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c0d3090 con 0x7f008c0d35b0 2024-04-25T10:15:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f0093570640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008c1611d0 con 0x7f008c15dce0 2024-04-25T10:15:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f009130c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f008c0b5990 0x7f008c154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f009130c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f008c0b5990 0x7f008c154a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35732/0 (socket says 172.21.15.16:35732) 2024-04-25T10:15:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f009130c640 1 -- 172.21.15.16:0/877736952 learned_addr learned my addr 172.21.15.16:0/877736952 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f009130c640 1 -- 172.21.15.16:0/877736952 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f008c15dce0 msgr2=0x7f008c0c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f009130c640 1 --2- 172.21.15.16:0/877736952 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f008c15dce0 0x7f008c0c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f009130c640 1 -- 172.21.15.16:0/877736952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f008c0d35b0 msgr2=0x7f008c15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f009130c640 1 --2- 172.21.15.16:0/877736952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f008c0d35b0 0x7f008c15e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f009130c640 1 -- 172.21.15.16:0/877736952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008c15e7d0 con 0x7f008c0b5990 2024-04-25T10:15:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f009130c640 1 --2- 172.21.15.16:0/877736952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f008c0b5990 0x7f008c154a30 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f008800b3f0 tx=0x7f008800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f0081ffb640 1 -- 172.21.15.16:0/877736952 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088015020 con 0x7f008c0b5990 2024-04-25T10:15:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f0081ffb640 1 -- 172.21.15.16:0/877736952 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0088011830 con 0x7f008c0b5990 2024-04-25T10:15:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f0093570640 1 -- 172.21.15.16:0/877736952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008c160c10 con 0x7f008c0b5990 2024-04-25T10:15:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.209+0000 7f0081ffb640 1 -- 172.21.15.16:0/877736952 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088011c20 con 0x7f008c0b5990 2024-04-25T10:15:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.210+0000 7f0093570640 1 -- 172.21.15.16:0/877736952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008c0c7650 con 0x7f008c0b5990 2024-04-25T10:15:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.210+0000 7f0093570640 1 -- 172.21.15.16:0/877736952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008c0749b0 con 0x7f008c0b5990 2024-04-25T10:15:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.211+0000 7f0081ffb640 1 -- 172.21.15.16:0/877736952 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0088018020 con 0x7f008c0b5990 2024-04-25T10:15:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.211+0000 7f0081ffb640 1 --2- 172.21.15.16:0/877736952 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f007006b4d0 0x7f007006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.212+0000 7f0081ffb640 1 -- 172.21.15.16:0/877736952 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f0088089f20 con 0x7f008c0b5990 2024-04-25T10:15:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.212+0000 7f0090b0b640 1 --2- 172.21.15.16:0/877736952 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f007006b4d0 0x7f007006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.212+0000 7f0090b0b640 1 --2- 172.21.15.16:0/877736952 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f007006b4d0 0x7f007006d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f007c034000 tx=0x7f007c02c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.216+0000 7f0081ffb640 1 -- 172.21.15.16:0/877736952 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f008801f030 con 0x7f008c0b5990 2024-04-25T10:15:02.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.431+0000 7f0093570640 1 -- 172.21.15.16:0/877736952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f008c0d0b10 con 0x7f008c0b5990 2024-04-25T10:15:02.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.432+0000 7f0081ffb640 1 -- 172.21.15.16:0/877736952 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f00880172c0 con 0x7f008c0b5990 2024-04-25T10:15:02.439 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:15:02.439 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":96,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2871061122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2871061122},{"type":"v1","addr":"172.21.15.186:6836","nonce":2871061122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1871507206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1871507206},{"type":"v1","addr":"172.21.15.186:6837","nonce":1871507206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4064412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4064412586},{"type":"v1","addr":"172.21.15.16:6834","nonce":4064412586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:26.751060+0000","modified":"2024-04-25T10:14:54.069182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9252":{"gid":9252,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6835/3763439204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":3763439204},{"type":"v1","addr":"172.21.15.16:6835","nonce":3763439204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:15:02.440 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 96 2024-04-25T10:15:02.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.434+0000 7f0093570640 1 -- 172.21.15.16:0/877736952 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f007006b4d0 msgr2=0x7f007006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.434+0000 7f0093570640 1 --2- 172.21.15.16:0/877736952 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f007006b4d0 0x7f007006d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f007c034000 tx=0x7f007c02c000 comp rx=0 tx=0).stop 2024-04-25T10:15:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.434+0000 7f0093570640 1 -- 172.21.15.16:0/877736952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f008c0b5990 msgr2=0x7f008c154a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.434+0000 7f0093570640 1 --2- 172.21.15.16:0/877736952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f008c0b5990 0x7f008c154a30 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f008800b3f0 tx=0x7f008800b8b0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.434+0000 7f0093570640 1 -- 172.21.15.16:0/877736952 shutdown_connections 2024-04-25T10:15:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.434+0000 7f0093570640 1 --2- 172.21.15.16:0/877736952 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f007006b4d0 0x7f007006d990 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.434+0000 7f0093570640 1 --2- 172.21.15.16:0/877736952 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f008c15dce0 0x7f008c0c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.434+0000 7f0093570640 1 --2- 172.21.15.16:0/877736952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f008c0d35b0 0x7f008c15e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.434+0000 7f0093570640 1 --2- 172.21.15.16:0/877736952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f008c0b5990 0x7f008c154a30 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.434+0000 7f0093570640 1 -- 172.21.15.16:0/877736952 >> 172.21.15.16:0/877736952 conn(0x7f008c072900 msgr2=0x7f008c0b6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.434+0000 7f0093570640 1 -- 172.21.15.16:0/877736952 shutdown_connections 2024-04-25T10:15:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.434+0000 7f0093570640 1 -- 172.21.15.16:0/877736952 wait complete. 2024-04-25T10:15:02.449 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:02.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.545+0000 7fae29acb640 1 Processor -- start 2024-04-25T10:15:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.546+0000 7fae29acb640 1 -- start start 2024-04-25T10:15:02.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.546+0000 7fae29acb640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fae240cfdb0 0x7fae24166720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.546+0000 7fae29acb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae240cdfe0 con 0x7fae240cea00 2024-04-25T10:15:02.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.546+0000 7fae29acb640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae240d0430 con 0x7fae2415f120 2024-04-25T10:15:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.546+0000 7fae29acb640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae240d09f0 con 0x7fae240cfdb0 2024-04-25T10:15:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae22ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fae240cea00 0x7fae240cf6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55776/0 (socket says 172.21.15.16:55776) 2024-04-25T10:15:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae22ffd640 1 -- 172.21.15.16:0/1368727151 learned_addr learned my addr 172.21.15.16:0/1368727151 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae23fff640 1 --2- 172.21.15.16:0/1368727151 >> v2:172.21.15.186:3300/0 conn(0x7fae240cfdb0 0x7fae24166720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae227fc640 1 -- 172.21.15.16:0/1368727151 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083774638 0 0) 0x7fae240cdfe0 con 0x7fae240cea00 2024-04-25T10:15:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae227fc640 1 -- 172.21.15.16:0/1368727151 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae0c003520 con 0x7fae240cea00 2024-04-25T10:15:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae227fc640 1 -- 172.21.15.16:0/1368727151 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4034700241 0 0) 0x7fae0c003520 con 0x7fae240cea00 2024-04-25T10:15:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae227fc640 1 -- 172.21.15.16:0/1368727151 >> v2:172.21.15.186:3300/0 conn(0x7fae240cfdb0 msgr2=0x7fae24166720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae227fc640 1 --2- 172.21.15.16:0/1368727151 >> v2:172.21.15.186:3300/0 conn(0x7fae240cfdb0 0x7fae24166720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae227fc640 1 -- 172.21.15.16:0/1368727151 >> v1:172.21.15.186:6789/0 conn(0x7fae2415f120 legacy=0x7fae240ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae227fc640 1 -- 172.21.15.16:0/1368727151 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae240d0710 con 0x7fae240cea00 2024-04-25T10:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae227fc640 1 -- 172.21.15.16:0/1368727151 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fae18001690 con 0x7fae240cea00 2024-04-25T10:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae227fc640 1 -- 172.21.15.16:0/1368727151 >> v1:172.21.15.16:6789/0 conn(0x7fae240cea00 legacy=0x7fae240cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae227fc640 1 --2- 172.21.15.16:0/1368727151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae0c0039b0 0x7fae0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae227fc640 1 --2- 172.21.15.16:0/1368727151 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae0c004800 0x7fae0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae227fc640 1 --2- 172.21.15.16:0/1368727151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae0c0051c0 0x7fae0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae227fc640 1 -- 172.21.15.16:0/1368727151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae240d09f0 con 0x7fae0c0051c0 2024-04-25T10:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae227fc640 1 -- 172.21.15.16:0/1368727151 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c00c070 con 0x7fae0c0039b0 2024-04-25T10:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae227fc640 1 -- 172.21.15.16:0/1368727151 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c00c270 con 0x7fae0c004800 2024-04-25T10:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae23fff640 1 --2- 172.21.15.16:0/1368727151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae0c0051c0 0x7fae0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae237fe640 1 --2- 172.21.15.16:0/1368727151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae0c0039b0 0x7fae0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.547+0000 7fae22ffd640 1 --2- 172.21.15.16:0/1368727151 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae0c004800 0x7fae0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.548+0000 7fae23fff640 1 -- 172.21.15.16:0/1368727151 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae0c004800 msgr2=0x7fae0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.548+0000 7fae23fff640 1 --2- 172.21.15.16:0/1368727151 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae0c004800 0x7fae0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.548+0000 7fae23fff640 1 -- 172.21.15.16:0/1368727151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae0c0039b0 msgr2=0x7fae0c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.548+0000 7fae23fff640 1 --2- 172.21.15.16:0/1368727151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae0c0039b0 0x7fae0c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.548+0000 7fae23fff640 1 -- 172.21.15.16:0/1368727151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c003520 con 0x7fae0c0051c0 2024-04-25T10:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.548+0000 7fae23fff640 1 -- 172.21.15.16:0/1368727151 reap_dead start 2024-04-25T10:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.548+0000 7fae23fff640 1 --2- 172.21.15.16:0/1368727151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae0c0051c0 0x7fae0c00b9c0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009fc0 tx=0x7fae0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b947aee93d3cd73d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.548+0000 7fae227fc640 1 -- 172.21.15.16:0/1368727151 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c035020 con 0x7fae0c0051c0 2024-04-25T10:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.548+0000 7fae227fc640 1 -- 172.21.15.16:0/1368727151 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1c003bd0 con 0x7fae0c0051c0 2024-04-25T10:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.549+0000 7fae227fc640 1 -- 172.21.15.16:0/1368727151 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c009c10 con 0x7fae0c0051c0 2024-04-25T10:15:02.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.549+0000 7fae29acb640 1 -- 172.21.15.16:0/1368727151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae0c0051c0 msgr2=0x7fae0c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.549+0000 7fae29acb640 1 --2- 172.21.15.16:0/1368727151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae0c0051c0 0x7fae0c00b9c0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009fc0 tx=0x7fae0c005830 comp rx=0 tx=0).stop 2024-04-25T10:15:02.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.549+0000 7fae29acb640 1 -- 172.21.15.16:0/1368727151 shutdown_connections 2024-04-25T10:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.549+0000 7fae29acb640 1 --2- 172.21.15.16:0/1368727151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae0c0051c0 0x7fae0c00b9c0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.549+0000 7fae29acb640 1 -- 172.21.15.16:0/1368727151 >> 172.21.15.16:0/1368727151 conn(0x7fae240d1900 msgr2=0x7fae240d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.549+0000 7fae29acb640 1 -- 172.21.15.16:0/1368727151 shutdown_connections 2024-04-25T10:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.549+0000 7fae29acb640 1 -- 172.21.15.16:0/1368727151 wait complete. 2024-04-25T10:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.550+0000 7fae29acb640 1 Processor -- start 2024-04-25T10:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.550+0000 7fae29acb640 1 -- start start 2024-04-25T10:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.551+0000 7fae29acb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae0c0051c0 0x7fae240b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.551+0000 7fae29acb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae2415b040 0x7fae24153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.551+0000 7fae29acb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae240b4520 0x7fae240b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.551+0000 7fae29acb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae240cc480 con 0x7fae0c0051c0 2024-04-25T10:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.551+0000 7fae29acb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae240ccab0 con 0x7fae240b4520 2024-04-25T10:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.551+0000 7fae29acb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae240c5450 con 0x7fae2415b040 2024-04-25T10:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.551+0000 7fae237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae0c0051c0 0x7fae240b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.551+0000 7fae237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae0c0051c0 0x7fae240b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35748/0 (socket says 172.21.15.16:35748) 2024-04-25T10:15:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.551+0000 7fae237fe640 1 -- 172.21.15.16:0/1372687961 learned_addr learned my addr 172.21.15.16:0/1372687961 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.551+0000 7fae22ffd640 1 --2- 172.21.15.16:0/1372687961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae2415b040 0x7fae24153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.552+0000 7fae23fff640 1 --2- 172.21.15.16:0/1372687961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae240b4520 0x7fae240b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.552+0000 7fae237fe640 1 -- 172.21.15.16:0/1372687961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae2415b040 msgr2=0x7fae24153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.552+0000 7fae237fe640 1 --2- 172.21.15.16:0/1372687961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae2415b040 0x7fae24153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.552+0000 7fae237fe640 1 -- 172.21.15.16:0/1372687961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae240b4520 msgr2=0x7fae240b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.552+0000 7fae237fe640 1 --2- 172.21.15.16:0/1372687961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae240b4520 0x7fae240b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.552+0000 7fae237fe640 1 -- 172.21.15.16:0/1372687961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae240c9ea0 con 0x7fae0c0051c0 2024-04-25T10:15:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.552+0000 7fae23fff640 1 --2- 172.21.15.16:0/1372687961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae240b4520 0x7fae240b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.552+0000 7fae237fe640 1 --2- 172.21.15.16:0/1372687961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae0c0051c0 0x7fae240b0c00 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fae1400f5f0 tx=0x7fae1400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.552+0000 7fae0bfff640 1 -- 172.21.15.16:0/1372687961 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae140052e0 con 0x7fae0c0051c0 2024-04-25T10:15:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.552+0000 7fae0bfff640 1 -- 172.21.15.16:0/1372687961 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae14018070 con 0x7fae0c0051c0 2024-04-25T10:15:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.552+0000 7fae0bfff640 1 -- 172.21.15.16:0/1372687961 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14003810 con 0x7fae0c0051c0 2024-04-25T10:15:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.552+0000 7fae29acb640 1 -- 172.21.15.16:0/1372687961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae240ca180 con 0x7fae0c0051c0 2024-04-25T10:15:02.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.552+0000 7fae29acb640 1 -- 172.21.15.16:0/1372687961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae240ca740 con 0x7fae0c0051c0 2024-04-25T10:15:02.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.553+0000 7fae29acb640 1 -- 172.21.15.16:0/1372687961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae24067850 con 0x7fae0c0051c0 2024-04-25T10:15:02.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.553+0000 7fae0bfff640 1 -- 172.21.15.16:0/1372687961 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae140039b0 con 0x7fae0c0051c0 2024-04-25T10:15:02.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.554+0000 7fae0bfff640 1 --2- 172.21.15.16:0/1372687961 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae1006b500 0x7fae1006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.554+0000 7fae0bfff640 1 -- 172.21.15.16:0/1372687961 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fae1408a1f0 con 0x7fae0c0051c0 2024-04-25T10:15:02.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.554+0000 7fae22ffd640 1 --2- 172.21.15.16:0/1372687961 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae1006b500 0x7fae1006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.555+0000 7fae22ffd640 1 --2- 172.21.15.16:0/1372687961 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae1006b500 0x7fae1006d9c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fae241f7100 tx=0x7fae18009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.558+0000 7fae0bfff640 1 -- 172.21.15.16:0/1372687961 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae14058e50 con 0x7fae0c0051c0 2024-04-25T10:15:02.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.738+0000 7fae29acb640 1 -- 172.21.15.16:0/1372687961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae240cde70 con 0x7fae0c0051c0 2024-04-25T10:15:02.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.739+0000 7fae0bfff640 1 -- 172.21.15.16:0/1372687961 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+17026 (secure 0 0 0) 0x7fae14058750 con 0x7fae0c0051c0 2024-04-25T10:15:02.744 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:15:02.744 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":119,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:14:54.519874+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:25.025480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:25.726747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1d","mappings":[{"from":4,"to":3}]},{"pgid":"17.1e","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/2697110279":"2024-04-26T10:14:50.050260+0000","172.21.15.16:6833/2697110279":"2024-04-26T10:14:50.050260+0000","172.21.15.16:6835/123396308":"2024-04-26T10:14:39.639508+0000","172.21.15.16:6833/123396308":"2024-04-26T10:14:39.639508+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.741+0000 7fae29acb640 1 -- 172.21.15.16:0/1372687961 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae1006b500 msgr2=0x7fae1006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.741+0000 7fae29acb640 1 --2- 172.21.15.16:0/1372687961 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae1006b500 0x7fae1006d9c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fae241f7100 tx=0x7fae18009040 comp rx=0 tx=0).stop 2024-04-25T10:15:02.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.741+0000 7fae29acb640 1 -- 172.21.15.16:0/1372687961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae0c0051c0 msgr2=0x7fae240b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.741+0000 7fae29acb640 1 --2- 172.21.15.16:0/1372687961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae0c0051c0 0x7fae240b0c00 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fae1400f5f0 tx=0x7fae1400fab0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.741+0000 7fae29acb640 1 -- 172.21.15.16:0/1372687961 shutdown_connections 2024-04-25T10:15:02.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.741+0000 7fae29acb640 1 --2- 172.21.15.16:0/1372687961 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fae1006b500 0x7fae1006d9c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.741+0000 7fae29acb640 1 --2- 172.21.15.16:0/1372687961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae240b4520 0x7fae240b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.741+0000 7fae29acb640 1 --2- 172.21.15.16:0/1372687961 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae2415b040 0x7fae24153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.741+0000 7fae29acb640 1 --2- 172.21.15.16:0/1372687961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae0c0051c0 0x7fae240b0c00 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.741+0000 7fae29acb640 1 -- 172.21.15.16:0/1372687961 >> 172.21.15.16:0/1372687961 conn(0x7fae240d1900 msgr2=0x7fae24070b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:02.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.741+0000 7fae29acb640 1 -- 172.21.15.16:0/1372687961 shutdown_connections 2024-04-25T10:15:02.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.741+0000 7fae29acb640 1 -- 172.21.15.16:0/1372687961 wait complete. 2024-04-25T10:15:02.756 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:15:02.757 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.853+0000 7fd3f0c3f640 1 Processor -- start 2024-04-25T10:15:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.853+0000 7fd3f0c3f640 1 -- start start 2024-04-25T10:15:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.854+0000 7fd3f0c3f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3ec15dce0 0x7fd3ec0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.854+0000 7fd3f0c3f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd3ec0d35b0 0x7fd3ec0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.854+0000 7fd3f0c3f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3ec15e0c0 con 0x7fd3ec0d1f90 2024-04-25T10:15:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.854+0000 7fd3f0c3f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ec0b7b10 con 0x7fd3ec15dce0 2024-04-25T10:15:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.854+0000 7fd3f0c3f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ec0d2510 con 0x7fd3ec0d35b0 2024-04-25T10:15:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.854+0000 7fd3ea59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3ec15dce0 0x7fd3ec0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.854+0000 7fd3ead9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd3ec0d1f90 0x7fd3ec06bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59356/0 (socket says 172.21.15.16:59356) 2024-04-25T10:15:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.854+0000 7fd3ead9d640 1 -- 172.21.15.16:0/3820918031 learned_addr learned my addr 172.21.15.16:0/3820918031 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.854+0000 7fd3e3fff640 1 --2- 172.21.15.16:0/3820918031 >> v2:172.21.15.186:3300/0 conn(0x7fd3ec0d35b0 0x7fd3ec0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.854+0000 7fd3e9d9b640 1 -- 172.21.15.16:0/3820918031 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069929762 0 0) 0x7fd3ec15e0c0 con 0x7fd3ec0d1f90 2024-04-25T10:15:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.854+0000 7fd3e9d9b640 1 -- 172.21.15.16:0/3820918031 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3d8003520 con 0x7fd3ec0d1f90 2024-04-25T10:15:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.854+0000 7fd3ea59c640 1 -- 172.21.15.16:0/3820918031 >> v2:172.21.15.186:3300/0 conn(0x7fd3ec0d35b0 msgr2=0x7fd3ec0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.854+0000 7fd3ea59c640 1 --2- 172.21.15.16:0/3820918031 >> v2:172.21.15.186:3300/0 conn(0x7fd3ec0d35b0 0x7fd3ec0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.854+0000 7fd3ea59c640 1 -- 172.21.15.16:0/3820918031 >> v1:172.21.15.16:6789/0 conn(0x7fd3ec0d1f90 legacy=0x7fd3ec06bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.854+0000 7fd3ea59c640 1 -- 172.21.15.16:0/3820918031 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec0d27f0 con 0x7fd3ec15dce0 2024-04-25T10:15:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.855+0000 7fd3ea59c640 1 --2- 172.21.15.16:0/3820918031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3ec15dce0 0x7fd3ec0d1a50 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec0cdfa0 tx=0x7fd3dc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d95fb3f8d1293b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.855+0000 7fd3e9d9b640 1 -- 172.21.15.16:0/3820918031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc037070 con 0x7fd3ec15dce0 2024-04-25T10:15:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.855+0000 7fd3e9d9b640 1 -- 172.21.15.16:0/3820918031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc032440 con 0x7fd3ec15dce0 2024-04-25T10:15:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.855+0000 7fd3e9d9b640 1 -- 172.21.15.16:0/3820918031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc0327f0 con 0x7fd3ec15dce0 2024-04-25T10:15:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.856+0000 7fd3f0c3f640 1 -- 172.21.15.16:0/3820918031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3ec15dce0 msgr2=0x7fd3ec0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.856+0000 7fd3f0c3f640 1 --2- 172.21.15.16:0/3820918031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3ec15dce0 0x7fd3ec0d1a50 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec0cdfa0 tx=0x7fd3dc02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.856+0000 7fd3f0c3f640 1 -- 172.21.15.16:0/3820918031 shutdown_connections 2024-04-25T10:15:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.856+0000 7fd3f0c3f640 1 --2- 172.21.15.16:0/3820918031 >> v2:172.21.15.186:3300/0 conn(0x7fd3ec0d35b0 0x7fd3ec0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.856+0000 7fd3f0c3f640 1 --2- 172.21.15.16:0/3820918031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3ec15dce0 0x7fd3ec0d1a50 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.856+0000 7fd3f0c3f640 1 -- 172.21.15.16:0/3820918031 >> 172.21.15.16:0/3820918031 conn(0x7fd3ec072900 msgr2=0x7fd3ec0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.856+0000 7fd3f0c3f640 1 -- 172.21.15.16:0/3820918031 shutdown_connections 2024-04-25T10:15:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.856+0000 7fd3f0c3f640 1 -- 172.21.15.16:0/3820918031 wait complete. 2024-04-25T10:15:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.857+0000 7fd3f0c3f640 1 Processor -- start 2024-04-25T10:15:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.857+0000 7fd3f0c3f640 1 -- start start 2024-04-25T10:15:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.858+0000 7fd3f0c3f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3ec0d1f90 0x7fd3ec146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.858+0000 7fd3f0c3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3ec0d35b0 0x7fd3ec149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.858+0000 7fd3f0c3f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3ec15dce0 0x7fd3ec153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.858+0000 7fd3f0c3f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec0d3090 con 0x7fd3ec0d35b0 2024-04-25T10:15:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.858+0000 7fd3f0c3f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec0d2db0 con 0x7fd3ec15dce0 2024-04-25T10:15:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.858+0000 7fd3f0c3f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec0b9820 con 0x7fd3ec0d1f90 2024-04-25T10:15:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.858+0000 7fd3e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3ec0d35b0 0x7fd3ec149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.858+0000 7fd3e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3ec0d35b0 0x7fd3ec149350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56256/0 (socket says 172.21.15.16:56256) 2024-04-25T10:15:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.858+0000 7fd3e3fff640 1 -- 172.21.15.16:0/3875954169 learned_addr learned my addr 172.21.15.16:0/3875954169 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.858+0000 7fd3ead9d640 1 --2- 172.21.15.16:0/3875954169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3ec15dce0 0x7fd3ec153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.858+0000 7fd3ea59c640 1 --2- 172.21.15.16:0/3875954169 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3ec0d1f90 0x7fd3ec146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.858+0000 7fd3e3fff640 1 -- 172.21.15.16:0/3875954169 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3ec0d1f90 msgr2=0x7fd3ec146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.858+0000 7fd3e3fff640 1 --2- 172.21.15.16:0/3875954169 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3ec0d1f90 0x7fd3ec146490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.858+0000 7fd3e3fff640 1 -- 172.21.15.16:0/3875954169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3ec15dce0 msgr2=0x7fd3ec153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.858+0000 7fd3e3fff640 1 --2- 172.21.15.16:0/3875954169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3ec15dce0 0x7fd3ec153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.858+0000 7fd3e3fff640 1 -- 172.21.15.16:0/3875954169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec1469d0 con 0x7fd3ec0d35b0 2024-04-25T10:15:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.858+0000 7fd3ea59c640 1 --2- 172.21.15.16:0/3875954169 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3ec0d1f90 0x7fd3ec146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.859+0000 7fd3e3fff640 1 --2- 172.21.15.16:0/3875954169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3ec0d35b0 0x7fd3ec149350 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400e980 tx=0x7fd3d400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.859+0000 7fd3e2ffd640 1 -- 172.21.15.16:0/3875954169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4017070 con 0x7fd3ec0d35b0 2024-04-25T10:15:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.859+0000 7fd3e2ffd640 1 -- 172.21.15.16:0/3875954169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d400ca00 con 0x7fd3ec0d35b0 2024-04-25T10:15:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.859+0000 7fd3f0c3f640 1 -- 172.21.15.16:0/3875954169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ec149890 con 0x7fd3ec0d35b0 2024-04-25T10:15:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.859+0000 7fd3e2ffd640 1 -- 172.21.15.16:0/3875954169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d400cdd0 con 0x7fd3ec0d35b0 2024-04-25T10:15:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.859+0000 7fd3f0c3f640 1 -- 172.21.15.16:0/3875954169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ec1560f0 con 0x7fd3ec0d35b0 2024-04-25T10:15:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.860+0000 7fd3f0c3f640 1 -- 172.21.15.16:0/3875954169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3ec0749b0 con 0x7fd3ec0d35b0 2024-04-25T10:15:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.860+0000 7fd3e2ffd640 1 -- 172.21.15.16:0/3875954169 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd3d401a070 con 0x7fd3ec0d35b0 2024-04-25T10:15:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.861+0000 7fd3e2ffd640 1 --2- 172.21.15.16:0/3875954169 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd3cc06b4d0 0x7fd3cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.861+0000 7fd3e2ffd640 1 -- 172.21.15.16:0/3875954169 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fd3d408a240 con 0x7fd3ec0d35b0 2024-04-25T10:15:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.861+0000 7fd3ea59c640 1 --2- 172.21.15.16:0/3875954169 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd3cc06b4d0 0x7fd3cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.862+0000 7fd3ea59c640 1 --2- 172.21.15.16:0/3875954169 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd3cc06b4d0 0x7fd3cc06d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc034000 tx=0x7fd3dc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:02.865+0000 7fd3e2ffd640 1 -- 172.21.15.16:0/3875954169 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd3ec149890 con 0x7fd3ec0d35b0 2024-04-25T10:15:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.081+0000 7fd3f0c3f640 1 -- 172.21.15.16:0/3875954169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3ec15d0c0 con 0x7fd3ec0d35b0 2024-04-25T10:15:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.082+0000 7fd3e2ffd640 1 -- 172.21.15.16:0/3875954169 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fd3ec1469d0 con 0x7fd3ec0d35b0 2024-04-25T10:15:03.086 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:15:03.086 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":96,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2871061122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2871061122},{"type":"v1","addr":"172.21.15.186:6836","nonce":2871061122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1871507206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1871507206},{"type":"v1","addr":"172.21.15.186:6837","nonce":1871507206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4064412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4064412586},{"type":"v1","addr":"172.21.15.16:6834","nonce":4064412586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:26.751060+0000","modified":"2024-04-25T10:14:54.069182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9252":{"gid":9252,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6835/3763439204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":3763439204},{"type":"v1","addr":"172.21.15.16:6835","nonce":3763439204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:15:03.086 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 96 2024-04-25T10:15:03.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.083+0000 7fd3f0c3f640 1 -- 172.21.15.16:0/3875954169 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd3cc06b4d0 msgr2=0x7fd3cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.083+0000 7fd3f0c3f640 1 --2- 172.21.15.16:0/3875954169 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd3cc06b4d0 0x7fd3cc06d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc034000 tx=0x7fd3dc02e000 comp rx=0 tx=0).stop 2024-04-25T10:15:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.083+0000 7fd3f0c3f640 1 -- 172.21.15.16:0/3875954169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3ec0d35b0 msgr2=0x7fd3ec149350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.083+0000 7fd3f0c3f640 1 --2- 172.21.15.16:0/3875954169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3ec0d35b0 0x7fd3ec149350 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400e980 tx=0x7fd3d400ee40 comp rx=0 tx=0).stop 2024-04-25T10:15:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.083+0000 7fd3f0c3f640 1 -- 172.21.15.16:0/3875954169 shutdown_connections 2024-04-25T10:15:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.083+0000 7fd3f0c3f640 1 --2- 172.21.15.16:0/3875954169 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fd3cc06b4d0 0x7fd3cc06d990 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.083+0000 7fd3f0c3f640 1 --2- 172.21.15.16:0/3875954169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3ec15dce0 0x7fd3ec153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.083+0000 7fd3f0c3f640 1 --2- 172.21.15.16:0/3875954169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3ec0d35b0 0x7fd3ec149350 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.083+0000 7fd3f0c3f640 1 --2- 172.21.15.16:0/3875954169 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3ec0d1f90 0x7fd3ec146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.083+0000 7fd3f0c3f640 1 -- 172.21.15.16:0/3875954169 >> 172.21.15.16:0/3875954169 conn(0x7fd3ec072900 msgr2=0x7fd3ec0d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.083+0000 7fd3f0c3f640 1 -- 172.21.15.16:0/3875954169 shutdown_connections 2024-04-25T10:15:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.084+0000 7fd3f0c3f640 1 -- 172.21.15.16:0/3875954169 wait complete. 2024-04-25T10:15:03.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.196+0000 7f8c299e0640 1 Processor -- start 2024-04-25T10:15:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.196+0000 7f8c299e0640 1 -- start start 2024-04-25T10:15:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.197+0000 7f8c299e0640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8c2415baf0 0x7f8c240ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.197+0000 7f8c299e0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c24070a10 0x7f8c24072b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.197+0000 7f8c299e0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c240ce810 con 0x7f8c240ce380 2024-04-25T10:15:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.197+0000 7f8c299e0640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c24073210 con 0x7f8c2415baf0 2024-04-25T10:15:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.197+0000 7f8c299e0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c240cecc0 con 0x7f8c24070a10 2024-04-25T10:15:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.197+0000 7f8c237fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8c240ce380 0x7f8c240cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59358/0 (socket says 172.21.15.16:59358) 2024-04-25T10:15:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.197+0000 7f8c237fe640 1 -- 172.21.15.16:0/3776599767 learned_addr learned my addr 172.21.15.16:0/3776599767 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.197+0000 7f8c227fc640 1 --2- 172.21.15.16:0/3776599767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c24070a10 0x7f8c24072b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.197+0000 7f8c22ffd640 1 --2- 172.21.15.16:0/3776599767 >> v2:172.21.15.186:3300/0 conn(0x7f8c2415baf0 0x7f8c240ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.197+0000 7f8c21ffb640 1 -- 172.21.15.16:0/3776599767 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667359452 0 0) 0x7f8c1c001490 con 0x7f8c240ce380 2024-04-25T10:15:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.197+0000 7f8c21ffb640 1 -- 172.21.15.16:0/3776599767 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c0c003520 con 0x7f8c240ce380 2024-04-25T10:15:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.197+0000 7f8c22ffd640 1 -- 172.21.15.16:0/3776599767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c24070a10 msgr2=0x7f8c24072b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.197+0000 7f8c22ffd640 1 --2- 172.21.15.16:0/3776599767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c24070a10 0x7f8c24072b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.197+0000 7f8c22ffd640 1 -- 172.21.15.16:0/3776599767 >> v1:172.21.15.16:6789/0 conn(0x7f8c240ce380 legacy=0x7f8c240cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.197+0000 7f8c22ffd640 1 -- 172.21.15.16:0/3776599767 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c240cefa0 con 0x7f8c2415baf0 2024-04-25T10:15:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.197+0000 7f8c227fc640 1 --2- 172.21.15.16:0/3776599767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c24070a10 0x7f8c24072b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.198+0000 7f8c22ffd640 1 --2- 172.21.15.16:0/3776599767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c2415baf0 0x7f8c240ccb90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8c140097a0 tx=0x7f8c1402a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4abbd3fbb7c64c74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.198+0000 7f8c21ffb640 1 -- 172.21.15.16:0/3776599767 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c14037070 con 0x7f8c2415baf0 2024-04-25T10:15:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.198+0000 7f8c21ffb640 1 -- 172.21.15.16:0/3776599767 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1402adb0 con 0x7f8c2415baf0 2024-04-25T10:15:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.198+0000 7f8c21ffb640 1 -- 172.21.15.16:0/3776599767 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c14032640 con 0x7f8c2415baf0 2024-04-25T10:15:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.199+0000 7f8c299e0640 1 -- 172.21.15.16:0/3776599767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c2415baf0 msgr2=0x7f8c240ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.199+0000 7f8c299e0640 1 --2- 172.21.15.16:0/3776599767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c2415baf0 0x7f8c240ccb90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8c140097a0 tx=0x7f8c1402a060 comp rx=0 tx=0).stop 2024-04-25T10:15:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.199+0000 7f8c299e0640 1 -- 172.21.15.16:0/3776599767 shutdown_connections 2024-04-25T10:15:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.199+0000 7f8c299e0640 1 --2- 172.21.15.16:0/3776599767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c24070a10 0x7f8c24072b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.199+0000 7f8c299e0640 1 --2- 172.21.15.16:0/3776599767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c2415baf0 0x7f8c240ccb90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.199+0000 7f8c299e0640 1 -- 172.21.15.16:0/3776599767 >> 172.21.15.16:0/3776599767 conn(0x7f8c240b2370 msgr2=0x7f8c240b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.199+0000 7f8c299e0640 1 -- 172.21.15.16:0/3776599767 shutdown_connections 2024-04-25T10:15:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.199+0000 7f8c299e0640 1 -- 172.21.15.16:0/3776599767 wait complete. 2024-04-25T10:15:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.200+0000 7f8c299e0640 1 Processor -- start 2024-04-25T10:15:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.200+0000 7f8c299e0640 1 -- start start 2024-04-25T10:15:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.201+0000 7f8c299e0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c24070a10 0x7f8c24145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.201+0000 7f8c299e0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c240ce380 0x7f8c24157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.201+0000 7f8c299e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c2415baf0 0x7f8c24158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.201+0000 7f8c299e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c240cc0a0 con 0x7f8c2415baf0 2024-04-25T10:15:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.201+0000 7f8c299e0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c240704f0 con 0x7f8c24070a10 2024-04-25T10:15:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.201+0000 7f8c299e0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c240cc220 con 0x7f8c240ce380 2024-04-25T10:15:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.201+0000 7f8c237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c2415baf0 0x7f8c24158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.201+0000 7f8c237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c2415baf0 0x7f8c24158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56262/0 (socket says 172.21.15.16:56262) 2024-04-25T10:15:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.201+0000 7f8c237fe640 1 -- 172.21.15.16:0/980294428 learned_addr learned my addr 172.21.15.16:0/980294428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.201+0000 7f8c22ffd640 1 --2- 172.21.15.16:0/980294428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c24070a10 0x7f8c24145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.201+0000 7f8c227fc640 1 --2- 172.21.15.16:0/980294428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c240ce380 0x7f8c24157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.202+0000 7f8c237fe640 1 -- 172.21.15.16:0/980294428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c240ce380 msgr2=0x7f8c24157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.202+0000 7f8c237fe640 1 --2- 172.21.15.16:0/980294428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c240ce380 0x7f8c24157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.202+0000 7f8c237fe640 1 -- 172.21.15.16:0/980294428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c24070a10 msgr2=0x7f8c24145460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.202+0000 7f8c237fe640 1 --2- 172.21.15.16:0/980294428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c24070a10 0x7f8c24145460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.202+0000 7f8c237fe640 1 -- 172.21.15.16:0/980294428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c24152770 con 0x7f8c2415baf0 2024-04-25T10:15:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.202+0000 7f8c22ffd640 1 --2- 172.21.15.16:0/980294428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c24070a10 0x7f8c24145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.202+0000 7f8c237fe640 1 --2- 172.21.15.16:0/980294428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c2415baf0 0x7f8c24158460 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00cb10 tx=0x7f8c1c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.202+0000 7f8c0bfff640 1 -- 172.21.15.16:0/980294428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c017070 con 0x7f8c2415baf0 2024-04-25T10:15:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.202+0000 7f8c299e0640 1 -- 172.21.15.16:0/980294428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c2415b300 con 0x7f8c2415baf0 2024-04-25T10:15:03.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.202+0000 7f8c0bfff640 1 -- 172.21.15.16:0/980294428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1c00ac10 con 0x7f8c2415baf0 2024-04-25T10:15:03.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.202+0000 7f8c0bfff640 1 -- 172.21.15.16:0/980294428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c012400 con 0x7f8c2415baf0 2024-04-25T10:15:03.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.202+0000 7f8c299e0640 1 -- 172.21.15.16:0/980294428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2415b020 con 0x7f8c2415baf0 2024-04-25T10:15:03.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.203+0000 7f8c299e0640 1 -- 172.21.15.16:0/980294428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c240749b0 con 0x7f8c2415baf0 2024-04-25T10:15:03.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.203+0000 7f8c0bfff640 1 -- 172.21.15.16:0/980294428 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c1c01a070 con 0x7f8c2415baf0 2024-04-25T10:15:03.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.204+0000 7f8c0bfff640 1 --2- 172.21.15.16:0/980294428 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8c1006b500 0x7f8c1006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.204+0000 7f8c0bfff640 1 -- 172.21.15.16:0/980294428 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f8c1c08a0c0 con 0x7f8c2415baf0 2024-04-25T10:15:03.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.204+0000 7f8c22ffd640 1 --2- 172.21.15.16:0/980294428 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8c1006b500 0x7f8c1006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.204+0000 7f8c22ffd640 1 --2- 172.21.15.16:0/980294428 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8c1006b500 0x7f8c1006d9c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8c14034000 tx=0x7f8c1402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.208+0000 7f8c0bfff640 1 -- 172.21.15.16:0/980294428 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c24152770 con 0x7f8c2415baf0 2024-04-25T10:15:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.426+0000 7f8c299e0640 1 -- 172.21.15.16:0/980294428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c240b1100 con 0x7f8c2415baf0 2024-04-25T10:15:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.427+0000 7f8c0bfff640 1 -- 172.21.15.16:0/980294428 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f8c240ce810 con 0x7f8c2415baf0 2024-04-25T10:15:03.431 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:15:03.431 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":96,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2871061122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2871061122},{"type":"v1","addr":"172.21.15.186:6836","nonce":2871061122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1871507206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1871507206},{"type":"v1","addr":"172.21.15.186:6837","nonce":1871507206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4064412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4064412586},{"type":"v1","addr":"172.21.15.16:6834","nonce":4064412586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:26.751060+0000","modified":"2024-04-25T10:14:54.069182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9252":{"gid":9252,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6835/3763439204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":3763439204},{"type":"v1","addr":"172.21.15.16:6835","nonce":3763439204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:15:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 96 2024-04-25T10:15:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.429+0000 7f8c299e0640 1 -- 172.21.15.16:0/980294428 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8c1006b500 msgr2=0x7f8c1006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.429+0000 7f8c299e0640 1 --2- 172.21.15.16:0/980294428 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8c1006b500 0x7f8c1006d9c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8c14034000 tx=0x7f8c1402e000 comp rx=0 tx=0).stop 2024-04-25T10:15:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.429+0000 7f8c299e0640 1 -- 172.21.15.16:0/980294428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c2415baf0 msgr2=0x7f8c24158460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.429+0000 7f8c299e0640 1 --2- 172.21.15.16:0/980294428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c2415baf0 0x7f8c24158460 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00cb10 tx=0x7f8c1c00a060 comp rx=0 tx=0).stop 2024-04-25T10:15:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.429+0000 7f8c299e0640 1 -- 172.21.15.16:0/980294428 shutdown_connections 2024-04-25T10:15:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.429+0000 7f8c299e0640 1 --2- 172.21.15.16:0/980294428 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8c1006b500 0x7f8c1006d9c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.429+0000 7f8c299e0640 1 --2- 172.21.15.16:0/980294428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c2415baf0 0x7f8c24158460 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.429+0000 7f8c299e0640 1 --2- 172.21.15.16:0/980294428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c240ce380 0x7f8c24157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.429+0000 7f8c299e0640 1 --2- 172.21.15.16:0/980294428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c24070a10 0x7f8c24145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.429+0000 7f8c299e0640 1 -- 172.21.15.16:0/980294428 >> 172.21.15.16:0/980294428 conn(0x7f8c240b2370 msgr2=0x7f8c24156e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.429+0000 7f8c299e0640 1 -- 172.21.15.16:0/980294428 shutdown_connections 2024-04-25T10:15:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.429+0000 7f8c299e0640 1 -- 172.21.15.16:0/980294428 wait complete. 2024-04-25T10:15:03.444 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:03.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.540+0000 7fbf3b8e1640 1 Processor -- start 2024-04-25T10:15:03.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.540+0000 7fbf3b8e1640 1 -- start start 2024-04-25T10:15:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.541+0000 7fbf3b8e1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbf3415dd50 0x7fbf340d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.541+0000 7fbf3b8e1640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbf340b9460 0x7fbf340ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.541+0000 7fbf3b8e1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf340d34e0 0x7fbf340cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.541+0000 7fbf3b8e1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf340d2440 con 0x7fbf3415dd50 2024-04-25T10:15:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.541+0000 7fbf3b8e1640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf340d2720 con 0x7fbf340b9460 2024-04-25T10:15:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.541+0000 7fbf3b8e1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf340d2a00 con 0x7fbf340d34e0 2024-04-25T10:15:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.541+0000 7fbf3967d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbf3415dd50 0x7fbf340d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.541+0000 7fbf3967d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbf3415dd50 0x7fbf340d1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56278/0 (socket says 172.21.15.16:56278) 2024-04-25T10:15:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.541+0000 7fbf3967d640 1 -- 172.21.15.16:0/3218073013 learned_addr learned my addr 172.21.15.16:0/3218073013 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.542+0000 7fbf38e7c640 1 --2- 172.21.15.16:0/3218073013 >> v2:172.21.15.186:3300/0 conn(0x7fbf340b9460 0x7fbf340ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.542+0000 7fbf39e7e640 1 --2- 172.21.15.16:0/3218073013 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf340d34e0 0x7fbf340cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.542+0000 7fbf3967d640 1 -- 172.21.15.16:0/3218073013 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf340d34e0 msgr2=0x7fbf340cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.542+0000 7fbf3967d640 1 --2- 172.21.15.16:0/3218073013 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf340d34e0 0x7fbf340cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.542+0000 7fbf3967d640 1 -- 172.21.15.16:0/3218073013 >> v2:172.21.15.186:3300/0 conn(0x7fbf340b9460 msgr2=0x7fbf340ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.542+0000 7fbf3967d640 1 --2- 172.21.15.16:0/3218073013 >> v2:172.21.15.186:3300/0 conn(0x7fbf340b9460 0x7fbf340ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.542+0000 7fbf3967d640 1 -- 172.21.15.16:0/3218073013 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf340d2ce0 con 0x7fbf3415dd50 2024-04-25T10:15:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.542+0000 7fbf3967d640 1 --2- 172.21.15.16:0/3218073013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf3415dd50 0x7fbf340d1c10 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fbf3406ac90 tx=0x7fbf3002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe83b27376668717 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.543+0000 7fbf2bfff640 1 -- 172.21.15.16:0/3218073013 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf30037070 con 0x7fbf3415dd50 2024-04-25T10:15:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.543+0000 7fbf2bfff640 1 -- 172.21.15.16:0/3218073013 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf30032440 con 0x7fbf3415dd50 2024-04-25T10:15:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.543+0000 7fbf2bfff640 1 -- 172.21.15.16:0/3218073013 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf30032810 con 0x7fbf3415dd50 2024-04-25T10:15:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.543+0000 7fbf3b8e1640 1 -- 172.21.15.16:0/3218073013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf3415dd50 msgr2=0x7fbf340d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.543+0000 7fbf3b8e1640 1 --2- 172.21.15.16:0/3218073013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf3415dd50 0x7fbf340d1c10 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fbf3406ac90 tx=0x7fbf3002a140 comp rx=0 tx=0).stop 2024-04-25T10:15:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.543+0000 7fbf3b8e1640 1 -- 172.21.15.16:0/3218073013 shutdown_connections 2024-04-25T10:15:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.543+0000 7fbf3b8e1640 1 --2- 172.21.15.16:0/3218073013 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf340d34e0 0x7fbf340cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.543+0000 7fbf3b8e1640 1 --2- 172.21.15.16:0/3218073013 >> v2:172.21.15.186:3300/0 conn(0x7fbf340b9460 0x7fbf340ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.543+0000 7fbf3b8e1640 1 --2- 172.21.15.16:0/3218073013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf3415dd50 0x7fbf340d1c10 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.543+0000 7fbf3b8e1640 1 -- 172.21.15.16:0/3218073013 >> 172.21.15.16:0/3218073013 conn(0x7fbf340722d0 msgr2=0x7fbf340726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.543+0000 7fbf3b8e1640 1 -- 172.21.15.16:0/3218073013 shutdown_connections 2024-04-25T10:15:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.543+0000 7fbf3b8e1640 1 -- 172.21.15.16:0/3218073013 wait complete. 2024-04-25T10:15:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.544+0000 7fbf3b8e1640 1 Processor -- start 2024-04-25T10:15:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.544+0000 7fbf3b8e1640 1 -- start start 2024-04-25T10:15:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.545+0000 7fbf3b8e1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf340b9460 0x7fbf340cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.545+0000 7fbf3b8e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf340d34e0 0x7fbf3414b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.545+0000 7fbf3b8e1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf3415dd50 0x7fbf340caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.545+0000 7fbf3b8e1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf340b5550 con 0x7fbf340d34e0 2024-04-25T10:15:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.545+0000 7fbf3b8e1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf340d2150 con 0x7fbf340b9460 2024-04-25T10:15:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.545+0000 7fbf3b8e1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf340b56d0 con 0x7fbf3415dd50 2024-04-25T10:15:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.545+0000 7fbf38e7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf340d34e0 0x7fbf3414b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.545+0000 7fbf38e7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf340d34e0 0x7fbf3414b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56280/0 (socket says 172.21.15.16:56280) 2024-04-25T10:15:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.545+0000 7fbf38e7c640 1 -- 172.21.15.16:0/3223215164 learned_addr learned my addr 172.21.15.16:0/3223215164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.545+0000 7fbf3967d640 1 --2- 172.21.15.16:0/3223215164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf340b9460 0x7fbf340cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.545+0000 7fbf39e7e640 1 --2- 172.21.15.16:0/3223215164 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf3415dd50 0x7fbf340caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.546+0000 7fbf38e7c640 1 -- 172.21.15.16:0/3223215164 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf3415dd50 msgr2=0x7fbf340caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.546+0000 7fbf38e7c640 1 --2- 172.21.15.16:0/3223215164 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf3415dd50 0x7fbf340caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.546+0000 7fbf38e7c640 1 -- 172.21.15.16:0/3223215164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf340b9460 msgr2=0x7fbf340cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.546+0000 7fbf38e7c640 1 --2- 172.21.15.16:0/3223215164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf340b9460 0x7fbf340cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.546+0000 7fbf38e7c640 1 -- 172.21.15.16:0/3223215164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3414b620 con 0x7fbf340d34e0 2024-04-25T10:15:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.546+0000 7fbf38e7c640 1 --2- 172.21.15.16:0/3223215164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf340d34e0 0x7fbf3414b0e0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400e9b0 tx=0x7fbf2400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.546+0000 7fbf29ffb640 1 -- 172.21.15.16:0/3223215164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24017070 con 0x7fbf340d34e0 2024-04-25T10:15:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.546+0000 7fbf29ffb640 1 -- 172.21.15.16:0/3223215164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2400c9a0 con 0x7fbf340d34e0 2024-04-25T10:15:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.546+0000 7fbf3b8e1640 1 -- 172.21.15.16:0/3223215164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf340cb580 con 0x7fbf340d34e0 2024-04-25T10:15:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.546+0000 7fbf29ffb640 1 -- 172.21.15.16:0/3223215164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2400cd90 con 0x7fbf340d34e0 2024-04-25T10:15:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.546+0000 7fbf3b8e1640 1 -- 172.21.15.16:0/3223215164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf340cbb40 con 0x7fbf340d34e0 2024-04-25T10:15:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.547+0000 7fbf3b8e1640 1 -- 172.21.15.16:0/3223215164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf34067850 con 0x7fbf340d34e0 2024-04-25T10:15:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.548+0000 7fbf29ffb640 1 -- 172.21.15.16:0/3223215164 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf2401a070 con 0x7fbf340d34e0 2024-04-25T10:15:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.548+0000 7fbf29ffb640 1 --2- 172.21.15.16:0/3223215164 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbf1806b4d0 0x7fbf1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.548+0000 7fbf29ffb640 1 -- 172.21.15.16:0/3223215164 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fbf2408a330 con 0x7fbf340d34e0 2024-04-25T10:15:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.548+0000 7fbf3967d640 1 --2- 172.21.15.16:0/3223215164 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbf1806b4d0 0x7fbf1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.549+0000 7fbf3967d640 1 --2- 172.21.15.16:0/3223215164 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbf1806b4d0 0x7fbf1806d990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fbf30034000 tx=0x7fbf3002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.552+0000 7fbf29ffb640 1 -- 172.21.15.16:0/3223215164 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf3414b620 con 0x7fbf340d34e0 2024-04-25T10:15:03.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.733+0000 7fbf3b8e1640 1 -- 172.21.15.16:0/3223215164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf340749b0 con 0x7fbf340d34e0 2024-04-25T10:15:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.734+0000 7fbf29ffb640 1 -- 172.21.15.16:0/3223215164 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+17026 (secure 0 0 0) 0x7fbf240590c0 con 0x7fbf340d34e0 2024-04-25T10:15:03.738 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:15:03.738 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":119,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:14:54.519874+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:25.025480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:25.726747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1d","mappings":[{"from":4,"to":3}]},{"pgid":"17.1e","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/2697110279":"2024-04-26T10:14:50.050260+0000","172.21.15.16:6833/2697110279":"2024-04-26T10:14:50.050260+0000","172.21.15.16:6835/123396308":"2024-04-26T10:14:39.639508+0000","172.21.15.16:6833/123396308":"2024-04-26T10:14:39.639508+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.736+0000 7fbf3b8e1640 1 -- 172.21.15.16:0/3223215164 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbf1806b4d0 msgr2=0x7fbf1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.736+0000 7fbf3b8e1640 1 --2- 172.21.15.16:0/3223215164 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbf1806b4d0 0x7fbf1806d990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fbf30034000 tx=0x7fbf3002e000 comp rx=0 tx=0).stop 2024-04-25T10:15:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.736+0000 7fbf3b8e1640 1 -- 172.21.15.16:0/3223215164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf340d34e0 msgr2=0x7fbf3414b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.736+0000 7fbf3b8e1640 1 --2- 172.21.15.16:0/3223215164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf340d34e0 0x7fbf3414b0e0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400e9b0 tx=0x7fbf2400ee70 comp rx=0 tx=0).stop 2024-04-25T10:15:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.736+0000 7fbf3b8e1640 1 -- 172.21.15.16:0/3223215164 shutdown_connections 2024-04-25T10:15:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.736+0000 7fbf3b8e1640 1 --2- 172.21.15.16:0/3223215164 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fbf1806b4d0 0x7fbf1806d990 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.736+0000 7fbf3b8e1640 1 --2- 172.21.15.16:0/3223215164 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf3415dd50 0x7fbf340caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.736+0000 7fbf3b8e1640 1 --2- 172.21.15.16:0/3223215164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf340d34e0 0x7fbf3414b0e0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.736+0000 7fbf3b8e1640 1 --2- 172.21.15.16:0/3223215164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf340b9460 0x7fbf340cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.736+0000 7fbf3b8e1640 1 -- 172.21.15.16:0/3223215164 >> 172.21.15.16:0/3223215164 conn(0x7fbf340722d0 msgr2=0x7fbf3415bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.736+0000 7fbf3b8e1640 1 -- 172.21.15.16:0/3223215164 shutdown_connections 2024-04-25T10:15:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.736+0000 7fbf3b8e1640 1 -- 172.21.15.16:0/3223215164 wait complete. 2024-04-25T10:15:03.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:15:03.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.849+0000 7f99edab3640 1 Processor -- start 2024-04-25T10:15:03.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.849+0000 7f99edab3640 1 -- start start 2024-04-25T10:15:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.850+0000 7f99edab3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99e80cef40 0x7f99e80b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.850+0000 7f99edab3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f99e81618d0 0x7f99e80d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.850+0000 7f99edab3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e8162040 con 0x7f99e81610b0 2024-04-25T10:15:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.850+0000 7f99edab3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e80d1ea0 con 0x7f99e80cef40 2024-04-25T10:15:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.850+0000 7f99edab3640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e8162320 con 0x7f99e81618d0 2024-04-25T10:15:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.850+0000 7f99e6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99e80cef40 0x7f99e80b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.850+0000 7f99e77fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f99e81610b0 0x7f99e80ce830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59372/0 (socket says 172.21.15.16:59372) 2024-04-25T10:15:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.850+0000 7f99e77fe640 1 -- 172.21.15.16:0/3045776070 learned_addr learned my addr 172.21.15.16:0/3045776070 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.850+0000 7f99e7fff640 1 --2- 172.21.15.16:0/3045776070 >> v2:172.21.15.186:3300/0 conn(0x7f99e81618d0 0x7f99e80d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.850+0000 7f99e6ffd640 1 -- 172.21.15.16:0/3045776070 >> v2:172.21.15.186:3300/0 conn(0x7f99e81618d0 msgr2=0x7f99e80d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.850+0000 7f99e6ffd640 1 --2- 172.21.15.16:0/3045776070 >> v2:172.21.15.186:3300/0 conn(0x7f99e81618d0 0x7f99e80d2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.850+0000 7f99e6ffd640 1 -- 172.21.15.16:0/3045776070 >> v1:172.21.15.16:6789/0 conn(0x7f99e81610b0 legacy=0x7f99e80ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.850+0000 7f99e67fc640 1 -- 172.21.15.16:0/3045776070 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951896737 0 0) 0x7f99e8162040 con 0x7f99e81610b0 2024-04-25T10:15:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.850+0000 7f99e6ffd640 1 -- 172.21.15.16:0/3045776070 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e8162600 con 0x7f99e80cef40 2024-04-25T10:15:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.850+0000 7f99e7fff640 1 --2- 172.21.15.16:0/3045776070 >> v2:172.21.15.186:3300/0 conn(0x7f99e81618d0 0x7f99e80d2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.851+0000 7f99e6ffd640 1 --2- 172.21.15.16:0/3045776070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e80cef40 0x7f99e80b9580 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f99e80b9190 tx=0x7f99dc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04d7b849c9e4a6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.851+0000 7f99e67fc640 1 -- 172.21.15.16:0/3045776070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc037070 con 0x7f99e80cef40 2024-04-25T10:15:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.851+0000 7f99e67fc640 1 -- 172.21.15.16:0/3045776070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99dc032440 con 0x7f99e80cef40 2024-04-25T10:15:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.851+0000 7f99e67fc640 1 -- 172.21.15.16:0/3045776070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc02ae60 con 0x7f99e80cef40 2024-04-25T10:15:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.852+0000 7f99edab3640 1 -- 172.21.15.16:0/3045776070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e80cef40 msgr2=0x7f99e80b9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.852+0000 7f99edab3640 1 --2- 172.21.15.16:0/3045776070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e80cef40 0x7f99e80b9580 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f99e80b9190 tx=0x7f99dc02a110 comp rx=0 tx=0).stop 2024-04-25T10:15:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.852+0000 7f99edab3640 1 -- 172.21.15.16:0/3045776070 shutdown_connections 2024-04-25T10:15:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.852+0000 7f99edab3640 1 --2- 172.21.15.16:0/3045776070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e80cef40 0x7f99e80b9580 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.852+0000 7f99edab3640 1 --2- 172.21.15.16:0/3045776070 >> v2:172.21.15.186:3300/0 conn(0x7f99e81618d0 0x7f99e80d2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.852+0000 7f99edab3640 1 -- 172.21.15.16:0/3045776070 >> 172.21.15.16:0/3045776070 conn(0x7f99e80b5570 msgr2=0x7f99e80b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:03.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.852+0000 7f99edab3640 1 -- 172.21.15.16:0/3045776070 shutdown_connections 2024-04-25T10:15:03.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.852+0000 7f99edab3640 1 -- 172.21.15.16:0/3045776070 wait complete. 2024-04-25T10:15:03.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.853+0000 7f99edab3640 1 Processor -- start 2024-04-25T10:15:03.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.853+0000 7f99edab3640 1 -- start start 2024-04-25T10:15:03.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99edab3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99e80cef40 0x7f99e8151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99edab3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e81610b0 0x7f99e814f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99edab3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99e81618d0 0x7f99e815f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99edab3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e8161d40 con 0x7f99e81610b0 2024-04-25T10:15:03.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99edab3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e80b6c00 con 0x7f99e81618d0 2024-04-25T10:15:03.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99edab3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e8161490 con 0x7f99e80cef40 2024-04-25T10:15:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99e6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e81610b0 0x7f99e814f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99e6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e81610b0 0x7f99e814f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56288/0 (socket says 172.21.15.16:56288) 2024-04-25T10:15:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99e6ffd640 1 -- 172.21.15.16:0/3043059111 learned_addr learned my addr 172.21.15.16:0/3043059111 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99e7fff640 1 --2- 172.21.15.16:0/3043059111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99e81618d0 0x7f99e815f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99e77fe640 1 --2- 172.21.15.16:0/3043059111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99e80cef40 0x7f99e8151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99e6ffd640 1 -- 172.21.15.16:0/3043059111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99e80cef40 msgr2=0x7f99e8151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99e6ffd640 1 --2- 172.21.15.16:0/3043059111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99e80cef40 0x7f99e8151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99e6ffd640 1 -- 172.21.15.16:0/3043059111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99e81618d0 msgr2=0x7f99e815f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99e6ffd640 1 --2- 172.21.15.16:0/3043059111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99e81618d0 0x7f99e815f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99e6ffd640 1 -- 172.21.15.16:0/3043059111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e81522c0 con 0x7f99e81610b0 2024-04-25T10:15:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99e7fff640 1 --2- 172.21.15.16:0/3043059111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99e81618d0 0x7f99e815f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99e77fe640 1 --2- 172.21.15.16:0/3043059111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99e80cef40 0x7f99e8151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99e6ffd640 1 --2- 172.21.15.16:0/3043059111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e81610b0 0x7f99e814f840 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f99dc035040 tx=0x7f99dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99cffff640 1 -- 172.21.15.16:0/3043059111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc037050 con 0x7f99e81610b0 2024-04-25T10:15:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99cffff640 1 -- 172.21.15.16:0/3043059111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99dc038040 con 0x7f99e81610b0 2024-04-25T10:15:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.854+0000 7f99cffff640 1 -- 172.21.15.16:0/3043059111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc004450 con 0x7f99e81610b0 2024-04-25T10:15:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.855+0000 7f99edab3640 1 -- 172.21.15.16:0/3043059111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e81525a0 con 0x7f99e81610b0 2024-04-25T10:15:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.855+0000 7f99edab3640 1 -- 172.21.15.16:0/3043059111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e80cad10 con 0x7f99e81610b0 2024-04-25T10:15:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.855+0000 7f99edab3640 1 -- 172.21.15.16:0/3043059111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e80cab10 con 0x7f99e81610b0 2024-04-25T10:15:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.856+0000 7f99cffff640 1 -- 172.21.15.16:0/3043059111 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f99dc030980 con 0x7f99e81610b0 2024-04-25T10:15:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.856+0000 7f99cffff640 1 --2- 172.21.15.16:0/3043059111 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f99d406b4d0 0x7f99d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.856+0000 7f99cffff640 1 -- 172.21.15.16:0/3043059111 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f99dc0aae70 con 0x7f99e81610b0 2024-04-25T10:15:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.857+0000 7f99e77fe640 1 --2- 172.21.15.16:0/3043059111 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f99d406b4d0 0x7f99d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.857+0000 7f99e77fe640 1 --2- 172.21.15.16:0/3043059111 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f99d406b4d0 0x7f99d406d990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f99d8000f60 tx=0x7f99d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:03.860+0000 7f99cffff640 1 -- 172.21.15.16:0/3043059111 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99dc079db0 con 0x7f99e81610b0 2024-04-25T10:15:04.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.078+0000 7f99edab3640 1 -- 172.21.15.16:0/3043059111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f99e80750b0 con 0x7f99e81610b0 2024-04-25T10:15:04.083 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-04-25T10:15:04.079+0000 7ff13f9bb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:15:04.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.080+0000 7f99cffff640 1 -- 172.21.15.16:0/3043059111 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v97) v1 ==== 161+0+0 (secure 0 0 0) 0x7f99dc0796b0 con 0x7f99e81610b0 2024-04-25T10:15:04.084 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:15:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.083+0000 7f99edab3640 1 -- 172.21.15.16:0/3043059111 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f99d406b4d0 msgr2=0x7f99d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.083+0000 7f99edab3640 1 --2- 172.21.15.16:0/3043059111 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f99d406b4d0 0x7f99d406d990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f99d8000f60 tx=0x7f99d8009040 comp rx=0 tx=0).stop 2024-04-25T10:15:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.083+0000 7f99edab3640 1 -- 172.21.15.16:0/3043059111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e81610b0 msgr2=0x7f99e814f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.083+0000 7f99edab3640 1 --2- 172.21.15.16:0/3043059111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e81610b0 0x7f99e814f840 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f99dc035040 tx=0x7f99dc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.083+0000 7f99edab3640 1 -- 172.21.15.16:0/3043059111 shutdown_connections 2024-04-25T10:15:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.083+0000 7f99edab3640 1 --2- 172.21.15.16:0/3043059111 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f99d406b4d0 0x7f99d406d990 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.083+0000 7f99edab3640 1 --2- 172.21.15.16:0/3043059111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99e81618d0 0x7f99e815f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.083+0000 7f99edab3640 1 --2- 172.21.15.16:0/3043059111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99e81610b0 0x7f99e814f840 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.083+0000 7f99edab3640 1 --2- 172.21.15.16:0/3043059111 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99e80cef40 0x7f99e8151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.083+0000 7f99edab3640 1 -- 172.21.15.16:0/3043059111 >> 172.21.15.16:0/3043059111 conn(0x7f99e80b5570 msgr2=0x7f99e80b6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:04.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.083+0000 7f99edab3640 1 -- 172.21.15.16:0/3043059111 shutdown_connections 2024-04-25T10:15:04.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.083+0000 7f99edab3640 1 -- 172.21.15.16:0/3043059111 wait complete. 2024-04-25T10:15:04.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:15:04.117 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-04-25T10:15:04.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.200+0000 7fe070007640 1 Processor -- start 2024-04-25T10:15:04.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.200+0000 7fe070007640 1 -- start start 2024-04-25T10:15:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.200+0000 7fe070007640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe06815dca0 0x7fe0680ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.200+0000 7fe070007640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0680d4090 con 0x7fe0680d3b90 2024-04-25T10:15:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.200+0000 7fe070007640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0680d4370 con 0x7fe0680d1130 2024-04-25T10:15:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.200+0000 7fe070007640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0680b6b20 con 0x7fe06815dca0 2024-04-25T10:15:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06d5a2640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe0680d3b90 0x7fe0680d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59404/0 (socket says 172.21.15.16:59404) 2024-04-25T10:15:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06d5a2640 1 -- 172.21.15.16:0/3676473007 learned_addr learned my addr 172.21.15.16:0/3676473007 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06dda3640 1 --2- 172.21.15.16:0/3676473007 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe06815dca0 0x7fe0680ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06cda1640 1 -- 172.21.15.16:0/3676473007 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186783769 0 0) 0x7fe0680d4090 con 0x7fe0680d3b90 2024-04-25T10:15:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06cda1640 1 -- 172.21.15.16:0/3676473007 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe050003520 con 0x7fe0680d3b90 2024-04-25T10:15:04.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06cda1640 1 -- 172.21.15.16:0/3676473007 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 989713043 0 0) 0x7fe050003520 con 0x7fe0680d3b90 2024-04-25T10:15:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06cda1640 1 -- 172.21.15.16:0/3676473007 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe06815dca0 msgr2=0x7fe0680ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06cda1640 1 --2- 172.21.15.16:0/3676473007 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe06815dca0 0x7fe0680ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06cda1640 1 -- 172.21.15.16:0/3676473007 >> v1:172.21.15.186:6789/0 conn(0x7fe0680d1130 legacy=0x7fe0680b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06cda1640 1 -- 172.21.15.16:0/3676473007 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0680d4650 con 0x7fe0680d3b90 2024-04-25T10:15:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06cda1640 1 -- 172.21.15.16:0/3676473007 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fe05c001670 con 0x7fe0680d3b90 2024-04-25T10:15:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06cda1640 1 -- 172.21.15.16:0/3676473007 >> v1:172.21.15.16:6789/0 conn(0x7fe0680d3b90 legacy=0x7fe0680d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06cda1640 1 --2- 172.21.15.16:0/3676473007 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe06815dca0 0x7fe0500039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06cda1640 1 --2- 172.21.15.16:0/3676473007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe050003f60 0x7fe050004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06cda1640 1 --2- 172.21.15.16:0/3676473007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe050004d40 0x7fe0500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06cda1640 1 -- 172.21.15.16:0/3676473007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05000ba70 con 0x7fe050004d40 2024-04-25T10:15:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06cda1640 1 -- 172.21.15.16:0/3676473007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05000bc20 con 0x7fe050003f60 2024-04-25T10:15:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06cda1640 1 -- 172.21.15.16:0/3676473007 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe05000be20 con 0x7fe06815dca0 2024-04-25T10:15:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.201+0000 7fe06e5a4640 1 --2- 172.21.15.16:0/3676473007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe050004d40 0x7fe0500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.202+0000 7fe06d5a2640 1 --2- 172.21.15.16:0/3676473007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe050003f60 0x7fe050004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.202+0000 7fe06dda3640 1 --2- 172.21.15.16:0/3676473007 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe06815dca0 0x7fe0500039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.202+0000 7fe06e5a4640 1 -- 172.21.15.16:0/3676473007 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe06815dca0 msgr2=0x7fe0500039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.202+0000 7fe06e5a4640 1 --2- 172.21.15.16:0/3676473007 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe06815dca0 0x7fe0500039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.202+0000 7fe06e5a4640 1 -- 172.21.15.16:0/3676473007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe050003f60 msgr2=0x7fe050004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:04.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.202+0000 7fe06e5a4640 1 --2- 172.21.15.16:0/3676473007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe050003f60 0x7fe050004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.202+0000 7fe06e5a4640 1 -- 172.21.15.16:0/3676473007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe050003520 con 0x7fe050004d40 2024-04-25T10:15:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.203+0000 7fe06e5a4640 1 --2- 172.21.15.16:0/3676473007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe050004d40 0x7fe0500051c0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fe0680d0320 tx=0x7fe050005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20183d4bae9e4f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.203+0000 7fe06cda1640 1 -- 172.21.15.16:0/3676473007 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe060001370 con 0x7fe050004d40 2024-04-25T10:15:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.203+0000 7fe06cda1640 1 -- 172.21.15.16:0/3676473007 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0600052d0 con 0x7fe050004d40 2024-04-25T10:15:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.203+0000 7fe06cda1640 1 -- 172.21.15.16:0/3676473007 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06002ab90 con 0x7fe050004d40 2024-04-25T10:15:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.203+0000 7fe070007640 1 -- 172.21.15.16:0/3676473007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe050004d40 msgr2=0x7fe0500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.203+0000 7fe070007640 1 --2- 172.21.15.16:0/3676473007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe050004d40 0x7fe0500051c0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fe0680d0320 tx=0x7fe050005830 comp rx=0 tx=0).stop 2024-04-25T10:15:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.203+0000 7fe06e5a4640 1 -- 172.21.15.16:0/3676473007 reap_dead start 2024-04-25T10:15:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.203+0000 7fe070007640 1 -- 172.21.15.16:0/3676473007 shutdown_connections 2024-04-25T10:15:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.203+0000 7fe070007640 1 -- 172.21.15.16:0/3676473007 >> 172.21.15.16:0/3676473007 conn(0x7fe0680726c0 msgr2=0x7fe068072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.203+0000 7fe070007640 1 -- 172.21.15.16:0/3676473007 shutdown_connections 2024-04-25T10:15:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.203+0000 7fe070007640 1 -- 172.21.15.16:0/3676473007 wait complete. 2024-04-25T10:15:04.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.204+0000 7fe070007640 1 Processor -- start 2024-04-25T10:15:04.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.205+0000 7fe070007640 1 -- start start 2024-04-25T10:15:04.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.205+0000 7fe070007640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe050004d40 0x7fe06815f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.205+0000 7fe070007640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe068160ac0 0x7fe0681538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.205+0000 7fe070007640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe068153e10 0x7fe068156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.205+0000 7fe070007640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06815e200 con 0x7fe068153e10 2024-04-25T10:15:04.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.205+0000 7fe070007640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06815e080 con 0x7fe050004d40 2024-04-25T10:15:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.205+0000 7fe070007640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0680b8280 con 0x7fe068160ac0 2024-04-25T10:15:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.205+0000 7fe06e5a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe068153e10 0x7fe068156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.205+0000 7fe06e5a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe068153e10 0x7fe068156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56328/0 (socket says 172.21.15.16:56328) 2024-04-25T10:15:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.205+0000 7fe06e5a4640 1 -- 172.21.15.16:0/1516854450 learned_addr learned my addr 172.21.15.16:0/1516854450 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.206+0000 7fe06dda3640 1 --2- 172.21.15.16:0/1516854450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe050004d40 0x7fe06815f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.206+0000 7fe06d5a2640 1 --2- 172.21.15.16:0/1516854450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe068160ac0 0x7fe0681538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.206+0000 7fe06e5a4640 1 -- 172.21.15.16:0/1516854450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe068160ac0 msgr2=0x7fe0681538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.206+0000 7fe06e5a4640 1 --2- 172.21.15.16:0/1516854450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe068160ac0 0x7fe0681538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.206+0000 7fe06e5a4640 1 -- 172.21.15.16:0/1516854450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe050004d40 msgr2=0x7fe06815f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.206+0000 7fe06e5a4640 1 --2- 172.21.15.16:0/1516854450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe050004d40 0x7fe06815f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.206+0000 7fe06e5a4640 1 -- 172.21.15.16:0/1516854450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06815fa20 con 0x7fe068153e10 2024-04-25T10:15:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.206+0000 7fe06dda3640 1 --2- 172.21.15.16:0/1516854450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe050004d40 0x7fe06815f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.206+0000 7fe06e5a4640 1 --2- 172.21.15.16:0/1516854450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe068153e10 0x7fe068156040 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fe06002a540 tx=0x7fe060001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.206+0000 7fe0667fc640 1 -- 172.21.15.16:0/1516854450 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe060037040 con 0x7fe068153e10 2024-04-25T10:15:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.206+0000 7fe0667fc640 1 -- 172.21.15.16:0/1516854450 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe060040070 con 0x7fe068153e10 2024-04-25T10:15:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.206+0000 7fe070007640 1 -- 172.21.15.16:0/1516854450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe06815fd00 con 0x7fe068153e10 2024-04-25T10:15:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.206+0000 7fe0667fc640 1 -- 172.21.15.16:0/1516854450 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe060007a30 con 0x7fe068153e10 2024-04-25T10:15:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.206+0000 7fe070007640 1 -- 172.21.15.16:0/1516854450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0681602c0 con 0x7fe068153e10 2024-04-25T10:15:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.207+0000 7fe070007640 1 -- 172.21.15.16:0/1516854450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0680750b0 con 0x7fe068153e10 2024-04-25T10:15:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.208+0000 7fe0667fc640 1 -- 172.21.15.16:0/1516854450 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0600052d0 con 0x7fe068153e10 2024-04-25T10:15:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.208+0000 7fe0667fc640 1 --2- 172.21.15.16:0/1516854450 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe05406b4d0 0x7fe05406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.208+0000 7fe0667fc640 1 -- 172.21.15.16:0/1516854450 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6025+0+0 (secure 0 0 0) 0x7fe0600a9c40 con 0x7fe068153e10 2024-04-25T10:15:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.208+0000 7fe06dda3640 1 --2- 172.21.15.16:0/1516854450 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe05406b4d0 0x7fe05406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.209+0000 7fe06dda3640 1 --2- 172.21.15.16:0/1516854450 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe05406b4d0 0x7fe05406d990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe058007440 tx=0x7fe05800b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.212+0000 7fe0667fc640 1 -- 172.21.15.16:0/1516854450 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0600788f0 con 0x7fe068153e10 2024-04-25T10:15:04.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:04.430+0000 7fe070007640 1 -- 172.21.15.16:0/1516854450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe068067f20 con 0x7fe068153e10 2024-04-25T10:15:05.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.099+0000 7fe0667fc640 1 -- 172.21.15.16:0/1516854450 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v98) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe0600781f0 con 0x7fe068153e10 2024-04-25T10:15:05.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.102+0000 7fe070007640 1 -- 172.21.15.16:0/1516854450 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe05406b4d0 msgr2=0x7fe05406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.102+0000 7fe070007640 1 --2- 172.21.15.16:0/1516854450 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe05406b4d0 0x7fe05406d990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe058007440 tx=0x7fe05800b040 comp rx=0 tx=0).stop 2024-04-25T10:15:05.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.102+0000 7fe070007640 1 -- 172.21.15.16:0/1516854450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe068153e10 msgr2=0x7fe068156040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.102+0000 7fe070007640 1 --2- 172.21.15.16:0/1516854450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe068153e10 0x7fe068156040 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fe06002a540 tx=0x7fe060001120 comp rx=0 tx=0).stop 2024-04-25T10:15:05.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.102+0000 7fe070007640 1 -- 172.21.15.16:0/1516854450 shutdown_connections 2024-04-25T10:15:05.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.102+0000 7fe070007640 1 --2- 172.21.15.16:0/1516854450 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe05406b4d0 0x7fe05406d990 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.102+0000 7fe070007640 1 --2- 172.21.15.16:0/1516854450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe068153e10 0x7fe068156040 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.102+0000 7fe070007640 1 --2- 172.21.15.16:0/1516854450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe068160ac0 0x7fe0681538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.102+0000 7fe070007640 1 --2- 172.21.15.16:0/1516854450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe050004d40 0x7fe06815f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.102+0000 7fe070007640 1 -- 172.21.15.16:0/1516854450 >> 172.21.15.16:0/1516854450 conn(0x7fe0680726c0 msgr2=0x7fe068072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:05.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.102+0000 7fe070007640 1 -- 172.21.15.16:0/1516854450 shutdown_connections 2024-04-25T10:15:05.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.102+0000 7fe070007640 1 -- 172.21.15.16:0/1516854450 wait complete. 2024-04-25T10:15:05.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:15:05.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.269+0000 7ff0f28dd640 1 Processor -- start 2024-04-25T10:15:05.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.269+0000 7ff0f28dd640 1 -- start start 2024-04-25T10:15:05.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.270+0000 7ff0f28dd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff0ec161090 0x7ff0ec161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.270+0000 7ff0f28dd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ec0b9720 0x7ff0ec0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.270+0000 7ff0f28dd640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0ec161f80 con 0x7ff0ec0cece0 2024-04-25T10:15:05.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.270+0000 7ff0f28dd640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0ec0cf130 con 0x7ff0ec161090 2024-04-25T10:15:05.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.270+0000 7ff0f28dd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec0d21a0 con 0x7ff0ec0b9720 2024-04-25T10:15:05.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.270+0000 7ff0eb7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ec0b9720 0x7ff0ec0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.270+0000 7ff0ebfff640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff0ec161090 0x7ff0ec161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.270+0000 7ff0eb7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ec0b9720 0x7ff0ec0ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:33928/0 (socket says 172.21.15.16:33928) 2024-04-25T10:15:05.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.270+0000 7ff0eb7fe640 1 -- 172.21.15.16:0/22510479 learned_addr learned my addr 172.21.15.16:0/22510479 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:05.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.270+0000 7ff0ebfff640 1 --2- 172.21.15.16:0/22510479 >> v2:172.21.15.186:3300/0 conn(0x7ff0ec161090 0x7ff0ec161730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:32976/0 (socket says 172.21.15.16:32976) 2024-04-25T10:15:05.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.270+0000 7ff0f0e7a640 1 --1- 172.21.15.16:0/22510479 >> v1:172.21.15.186:6789/0 conn(0x7ff0ec0cece0 0x7ff0ec06c0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:54060/0 (socket says 172.21.15.16:54060) 2024-04-25T10:15:05.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.271+0000 7ff0eb7fe640 1 -- 172.21.15.16:0/22510479 >> v2:172.21.15.186:3300/0 conn(0x7ff0ec161090 msgr2=0x7ff0ec161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:05.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.271+0000 7ff0eb7fe640 1 --2- 172.21.15.16:0/22510479 >> v2:172.21.15.186:3300/0 conn(0x7ff0ec161090 0x7ff0ec161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.271+0000 7ff0eb7fe640 1 -- 172.21.15.16:0/22510479 >> v1:172.21.15.186:6789/0 conn(0x7ff0ec0cece0 legacy=0x7ff0ec06c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.271+0000 7ff0eb7fe640 1 -- 172.21.15.16:0/22510479 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec162260 con 0x7ff0ec0b9720 2024-04-25T10:15:05.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.271+0000 7ff0ebfff640 1 --2- 172.21.15.16:0/22510479 >> v2:172.21.15.186:3300/0 conn(0x7ff0ec161090 0x7ff0ec161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.272+0000 7ff0eb7fe640 1 --2- 172.21.15.16:0/22510479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ec0b9720 0x7ff0ec0ce7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009870 tx=0x7ff0dc02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ce313f1ccda624d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.272+0000 7ff0eaffd640 1 -- 172.21.15.16:0/22510479 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc037070 con 0x7ff0ec0b9720 2024-04-25T10:15:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.272+0000 7ff0eaffd640 1 -- 172.21.15.16:0/22510479 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0dc032440 con 0x7ff0ec0b9720 2024-04-25T10:15:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.272+0000 7ff0eaffd640 1 -- 172.21.15.16:0/22510479 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc032810 con 0x7ff0ec0b9720 2024-04-25T10:15:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.272+0000 7ff0f28dd640 1 -- 172.21.15.16:0/22510479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ec0b9720 msgr2=0x7ff0ec0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.272+0000 7ff0f28dd640 1 --2- 172.21.15.16:0/22510479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ec0b9720 0x7ff0ec0ce7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009870 tx=0x7ff0dc02a270 comp rx=0 tx=0).stop 2024-04-25T10:15:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.272+0000 7ff0f28dd640 1 -- 172.21.15.16:0/22510479 shutdown_connections 2024-04-25T10:15:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.272+0000 7ff0f28dd640 1 --2- 172.21.15.16:0/22510479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ec0b9720 0x7ff0ec0ce7a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.272+0000 7ff0f28dd640 1 --2- 172.21.15.16:0/22510479 >> v2:172.21.15.186:3300/0 conn(0x7ff0ec161090 0x7ff0ec161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.272+0000 7ff0f28dd640 1 -- 172.21.15.16:0/22510479 >> 172.21.15.16:0/22510479 conn(0x7ff0ec0b55e0 msgr2=0x7ff0ec0b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.272+0000 7ff0f28dd640 1 -- 172.21.15.16:0/22510479 shutdown_connections 2024-04-25T10:15:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.272+0000 7ff0f28dd640 1 -- 172.21.15.16:0/22510479 wait complete. 2024-04-25T10:15:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.274+0000 7ff0f28dd640 1 Processor -- start 2024-04-25T10:15:05.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.274+0000 7ff0f28dd640 1 -- start start 2024-04-25T10:15:05.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.274+0000 7ff0f28dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0ec0b9720 0x7ff0ec1518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.274+0000 7ff0f28dd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0ec0cece0 0x7ff0ec0c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.274+0000 7ff0f28dd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ec161090 0x7ff0ec0c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.274+0000 7ff0f28dd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec161470 con 0x7ff0ec0b9720 2024-04-25T10:15:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.274+0000 7ff0f28dd640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec0d1e30 con 0x7ff0ec0cece0 2024-04-25T10:15:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.274+0000 7ff0f28dd640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec161c70 con 0x7ff0ec161090 2024-04-25T10:15:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.275+0000 7ff0ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0ec0b9720 0x7ff0ec1518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.275+0000 7ff0ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0ec0b9720 0x7ff0ec1518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56342/0 (socket says 172.21.15.16:56342) 2024-04-25T10:15:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.275+0000 7ff0ebfff640 1 -- 172.21.15.16:0/3356713561 learned_addr learned my addr 172.21.15.16:0/3356713561 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.275+0000 7ff0eb7fe640 1 --2- 172.21.15.16:0/3356713561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0ec0cece0 0x7ff0ec0c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.275+0000 7ff0f0e7a640 1 --2- 172.21.15.16:0/3356713561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ec161090 0x7ff0ec0c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.275+0000 7ff0ebfff640 1 -- 172.21.15.16:0/3356713561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ec161090 msgr2=0x7ff0ec0c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.275+0000 7ff0ebfff640 1 --2- 172.21.15.16:0/3356713561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ec161090 0x7ff0ec0c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.275+0000 7ff0ebfff640 1 -- 172.21.15.16:0/3356713561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0ec0cece0 msgr2=0x7ff0ec0c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.275+0000 7ff0ebfff640 1 --2- 172.21.15.16:0/3356713561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0ec0cece0 0x7ff0ec0c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.275+0000 7ff0ebfff640 1 -- 172.21.15.16:0/3356713561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec1f7780 con 0x7ff0ec0b9720 2024-04-25T10:15:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.275+0000 7ff0eb7fe640 1 --2- 172.21.15.16:0/3356713561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0ec0cece0 0x7ff0ec0c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.275+0000 7ff0ebfff640 1 --2- 172.21.15.16:0/3356713561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0ec0b9720 0x7ff0ec1518d0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400e960 tx=0x7ff0d400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.275+0000 7ff0e8ff9640 1 -- 172.21.15.16:0/3356713561 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4017070 con 0x7ff0ec0b9720 2024-04-25T10:15:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.275+0000 7ff0e8ff9640 1 -- 172.21.15.16:0/3356713561 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d400ca00 con 0x7ff0ec0b9720 2024-04-25T10:15:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.275+0000 7ff0f28dd640 1 -- 172.21.15.16:0/3356713561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ec1f7a60 con 0x7ff0ec0b9720 2024-04-25T10:15:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.275+0000 7ff0e8ff9640 1 -- 172.21.15.16:0/3356713561 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400cdd0 con 0x7ff0ec0b9720 2024-04-25T10:15:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.275+0000 7ff0f28dd640 1 -- 172.21.15.16:0/3356713561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ec1f8020 con 0x7ff0ec0b9720 2024-04-25T10:15:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.276+0000 7ff0f28dd640 1 -- 172.21.15.16:0/3356713561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0ec0750b0 con 0x7ff0ec0b9720 2024-04-25T10:15:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.277+0000 7ff0e8ff9640 1 -- 172.21.15.16:0/3356713561 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff0d401a070 con 0x7ff0ec0b9720 2024-04-25T10:15:05.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.277+0000 7ff0e8ff9640 1 --2- 172.21.15.16:0/3356713561 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff0cc06b4d0 0x7ff0cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.277+0000 7ff0e8ff9640 1 -- 172.21.15.16:0/3356713561 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6025+0+0 (secure 0 0 0) 0x7ff0d408a640 con 0x7ff0ec0b9720 2024-04-25T10:15:05.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.278+0000 7ff0eb7fe640 1 --2- 172.21.15.16:0/3356713561 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff0cc06b4d0 0x7ff0cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.278+0000 7ff0eb7fe640 1 --2- 172.21.15.16:0/3356713561 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff0cc06b4d0 0x7ff0cc06d990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc034000 tx=0x7ff0dc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:05.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.282+0000 7ff0e8ff9640 1 -- 172.21.15.16:0/3356713561 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0d4059300 con 0x7ff0ec0b9720 2024-04-25T10:15:05.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:05.466+0000 7ff0f28dd640 1 -- 172.21.15.16:0/3356713561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff0ec0c71d0 con 0x7ff0ec0b9720 2024-04-25T10:15:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.103+0000 7ff0e8ff9640 1 -- 172.21.15.16:0/3356713561 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v121) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff0d4058c00 con 0x7ff0ec0b9720 2024-04-25T10:15:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:15:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.107+0000 7ff0f28dd640 1 -- 172.21.15.16:0/3356713561 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff0cc06b4d0 msgr2=0x7ff0cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.107+0000 7ff0f28dd640 1 --2- 172.21.15.16:0/3356713561 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff0cc06b4d0 0x7ff0cc06d990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc034000 tx=0x7ff0dc02e000 comp rx=0 tx=0).stop 2024-04-25T10:15:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.107+0000 7ff0f28dd640 1 -- 172.21.15.16:0/3356713561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0ec0b9720 msgr2=0x7ff0ec1518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.107+0000 7ff0f28dd640 1 --2- 172.21.15.16:0/3356713561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0ec0b9720 0x7ff0ec1518d0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400e960 tx=0x7ff0d400ee20 comp rx=0 tx=0).stop 2024-04-25T10:15:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.108+0000 7ff0f28dd640 1 -- 172.21.15.16:0/3356713561 shutdown_connections 2024-04-25T10:15:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.108+0000 7ff0f28dd640 1 --2- 172.21.15.16:0/3356713561 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ff0cc06b4d0 0x7ff0cc06d990 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.108+0000 7ff0f28dd640 1 --2- 172.21.15.16:0/3356713561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ec161090 0x7ff0ec0c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.108+0000 7ff0f28dd640 1 --2- 172.21.15.16:0/3356713561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0ec0cece0 0x7ff0ec0c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.108+0000 7ff0f28dd640 1 --2- 172.21.15.16:0/3356713561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0ec0b9720 0x7ff0ec1518d0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.108+0000 7ff0f28dd640 1 -- 172.21.15.16:0/3356713561 >> 172.21.15.16:0/3356713561 conn(0x7ff0ec0b55e0 msgr2=0x7ff0ec0d3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.108+0000 7ff0f28dd640 1 -- 172.21.15.16:0/3356713561 shutdown_connections 2024-04-25T10:15:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.108+0000 7ff0f28dd640 1 -- 172.21.15.16:0/3356713561 wait complete. 2024-04-25T10:15:06.126 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:15:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.232+0000 7fe0c561c640 1 Processor -- start 2024-04-25T10:15:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.232+0000 7fe0c561c640 1 -- start start 2024-04-25T10:15:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.233+0000 7fe0c561c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe0c0147550 0x7fe0c00b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.233+0000 7fe0c561c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe0c006e900 0x7fe0c006ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.233+0000 7fe0c561c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c006bb70 con 0x7fe0c0161780 2024-04-25T10:15:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.233+0000 7fe0c561c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c006f220 con 0x7fe0c006e900 2024-04-25T10:15:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.233+0000 7fe0c561c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c00b3190 con 0x7fe0c0147550 2024-04-25T10:15:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.233+0000 7fe0bf7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe0c006e900 0x7fe0c006ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.233+0000 7fe0b7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe0c0161780 0x7fe0c00acf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59410/0 (socket says 172.21.15.16:59410) 2024-04-25T10:15:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.233+0000 7fe0b7fff640 1 -- 172.21.15.16:0/2283520341 learned_addr learned my addr 172.21.15.16:0/2283520341 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.233+0000 7fe0be7fc640 1 -- 172.21.15.16:0/2283520341 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 761290059 0 0) 0x7fe0c006bb70 con 0x7fe0c0161780 2024-04-25T10:15:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.233+0000 7fe0be7fc640 1 -- 172.21.15.16:0/2283520341 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0ac003520 con 0x7fe0c0161780 2024-04-25T10:15:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.233+0000 7fe0beffd640 1 --2- 172.21.15.16:0/2283520341 >> v2:172.21.15.186:3300/0 conn(0x7fe0c0147550 0x7fe0c00b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.233+0000 7fe0bf7fe640 1 -- 172.21.15.16:0/2283520341 >> v2:172.21.15.186:3300/0 conn(0x7fe0c0147550 msgr2=0x7fe0c00b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.233+0000 7fe0bf7fe640 1 --2- 172.21.15.16:0/2283520341 >> v2:172.21.15.186:3300/0 conn(0x7fe0c0147550 0x7fe0c00b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.233+0000 7fe0bf7fe640 1 -- 172.21.15.16:0/2283520341 >> v1:172.21.15.16:6789/0 conn(0x7fe0c0161780 legacy=0x7fe0c00acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.233+0000 7fe0bf7fe640 1 -- 172.21.15.16:0/2283520341 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c00b3470 con 0x7fe0c006e900 2024-04-25T10:15:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.233+0000 7fe0beffd640 1 --2- 172.21.15.16:0/2283520341 >> v2:172.21.15.186:3300/0 conn(0x7fe0c0147550 0x7fe0c00b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.234+0000 7fe0bf7fe640 1 --2- 172.21.15.16:0/2283520341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c006e900 0x7fe0c006ece0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fe0c006cf60 tx=0x7fe0b802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7b3d3a29f81823 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.234+0000 7fe0be7fc640 1 -- 172.21.15.16:0/2283520341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8007740 con 0x7fe0c006e900 2024-04-25T10:15:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.234+0000 7fe0be7fc640 1 -- 172.21.15.16:0/2283520341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b80078e0 con 0x7fe0c006e900 2024-04-25T10:15:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.234+0000 7fe0be7fc640 1 -- 172.21.15.16:0/2283520341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8007cb0 con 0x7fe0c006e900 2024-04-25T10:15:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.235+0000 7fe0c561c640 1 -- 172.21.15.16:0/2283520341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c006e900 msgr2=0x7fe0c006ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.235+0000 7fe0c561c640 1 --2- 172.21.15.16:0/2283520341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c006e900 0x7fe0c006ece0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fe0c006cf60 tx=0x7fe0b802a680 comp rx=0 tx=0).stop 2024-04-25T10:15:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.235+0000 7fe0c561c640 1 -- 172.21.15.16:0/2283520341 shutdown_connections 2024-04-25T10:15:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.235+0000 7fe0c561c640 1 --2- 172.21.15.16:0/2283520341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c006e900 0x7fe0c006ece0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.235+0000 7fe0c561c640 1 --2- 172.21.15.16:0/2283520341 >> v2:172.21.15.186:3300/0 conn(0x7fe0c0147550 0x7fe0c00b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.235+0000 7fe0c561c640 1 -- 172.21.15.16:0/2283520341 >> 172.21.15.16:0/2283520341 conn(0x7fe0c006aba0 msgr2=0x7fe0c006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.235+0000 7fe0c561c640 1 -- 172.21.15.16:0/2283520341 shutdown_connections 2024-04-25T10:15:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.235+0000 7fe0c561c640 1 -- 172.21.15.16:0/2283520341 wait complete. 2024-04-25T10:15:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.236+0000 7fe0c561c640 1 Processor -- start 2024-04-25T10:15:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.236+0000 7fe0c561c640 1 -- start start 2024-04-25T10:15:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.237+0000 7fe0c561c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c006e900 0x7fe0c013d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.237+0000 7fe0c561c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0c0147550 0x7fe0c013e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.237+0000 7fe0c561c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0c0161780 0x7fe0c00cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.237+0000 7fe0c561c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00b3d10 con 0x7fe0c006e900 2024-04-25T10:15:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.237+0000 7fe0c561c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00b3a30 con 0x7fe0c0161780 2024-04-25T10:15:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.237+0000 7fe0c561c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00ad6a0 con 0x7fe0c0147550 2024-04-25T10:15:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.237+0000 7fe0beffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c006e900 0x7fe0c013d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.237+0000 7fe0beffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c006e900 0x7fe0c013d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56360/0 (socket says 172.21.15.16:56360) 2024-04-25T10:15:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.237+0000 7fe0beffd640 1 -- 172.21.15.16:0/1478668342 learned_addr learned my addr 172.21.15.16:0/1478668342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.237+0000 7fe0bf7fe640 1 --2- 172.21.15.16:0/1478668342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0c0161780 0x7fe0c00cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.237+0000 7fe0beffd640 1 -- 172.21.15.16:0/1478668342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0c0147550 msgr2=0x7fe0c013e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.237+0000 7fe0beffd640 1 --2- 172.21.15.16:0/1478668342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0c0147550 0x7fe0c013e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.237+0000 7fe0beffd640 1 -- 172.21.15.16:0/1478668342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0c0161780 msgr2=0x7fe0c00cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.237+0000 7fe0beffd640 1 --2- 172.21.15.16:0/1478668342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0c0161780 0x7fe0c00cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.237+0000 7fe0beffd640 1 -- 172.21.15.16:0/1478668342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c013faa0 con 0x7fe0c006e900 2024-04-25T10:15:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.237+0000 7fe0beffd640 1 --2- 172.21.15.16:0/1478668342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c006e900 0x7fe0c013d940 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000c7f0 tx=0x7fe0b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.238+0000 7fe0b77fe640 1 -- 172.21.15.16:0/1478668342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0015070 con 0x7fe0c006e900 2024-04-25T10:15:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.238+0000 7fe0b77fe640 1 -- 172.21.15.16:0/1478668342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b0007820 con 0x7fe0c006e900 2024-04-25T10:15:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.238+0000 7fe0c561c640 1 -- 172.21.15.16:0/1478668342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c013ea00 con 0x7fe0c006e900 2024-04-25T10:15:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.238+0000 7fe0b77fe640 1 -- 172.21.15.16:0/1478668342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0007bd0 con 0x7fe0c006e900 2024-04-25T10:15:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.238+0000 7fe0c561c640 1 -- 172.21.15.16:0/1478668342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c013efc0 con 0x7fe0c006e900 2024-04-25T10:15:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.239+0000 7fe0c561c640 1 -- 172.21.15.16:0/1478668342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c00750b0 con 0x7fe0c006e900 2024-04-25T10:15:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.239+0000 7fe0b77fe640 1 -- 172.21.15.16:0/1478668342 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0b0018070 con 0x7fe0c006e900 2024-04-25T10:15:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.240+0000 7fe0b77fe640 1 --2- 172.21.15.16:0/1478668342 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe0a006b500 0x7fe0a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.240+0000 7fe0b77fe640 1 -- 172.21.15.16:0/1478668342 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5558+0+0 (secure 0 0 0) 0x7fe0b008a280 con 0x7fe0c006e900 2024-04-25T10:15:06.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.240+0000 7fe0b7fff640 1 --2- 172.21.15.16:0/1478668342 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe0a006b500 0x7fe0a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.241+0000 7fe0b7fff640 1 --2- 172.21.15.16:0/1478668342 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe0a006b500 0x7fe0a006d9c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8000f60 tx=0x7fe0a80078f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:06.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.244+0000 7fe0b77fe640 1 -- 172.21.15.16:0/1478668342 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0b0059190 con 0x7fe0c006e900 2024-04-25T10:15:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:06.426+0000 7fe0c561c640 1 -- 172.21.15.16:0/1478668342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe0c0161540 con 0x7fe0c006e900 2024-04-25T10:15:07.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.104+0000 7fe0b77fe640 1 -- 172.21.15.16:0/1478668342 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v122) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe0b0058a90 con 0x7fe0c006e900 2024-04-25T10:15:07.108 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-04-25T10:15:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.109+0000 7fe0c561c640 1 -- 172.21.15.16:0/1478668342 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe0a006b500 msgr2=0x7fe0a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.109+0000 7fe0c561c640 1 --2- 172.21.15.16:0/1478668342 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe0a006b500 0x7fe0a006d9c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8000f60 tx=0x7fe0a80078f0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.109+0000 7fe0c561c640 1 -- 172.21.15.16:0/1478668342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c006e900 msgr2=0x7fe0c013d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.109+0000 7fe0c561c640 1 --2- 172.21.15.16:0/1478668342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c006e900 0x7fe0c013d940 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000c7f0 tx=0x7fe0b000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.110+0000 7fe0c561c640 1 -- 172.21.15.16:0/1478668342 shutdown_connections 2024-04-25T10:15:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.110+0000 7fe0c561c640 1 --2- 172.21.15.16:0/1478668342 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fe0a006b500 0x7fe0a006d9c0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.110+0000 7fe0c561c640 1 --2- 172.21.15.16:0/1478668342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0c0161780 0x7fe0c00cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.110+0000 7fe0c561c640 1 --2- 172.21.15.16:0/1478668342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0c0147550 0x7fe0c013e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.110+0000 7fe0c561c640 1 --2- 172.21.15.16:0/1478668342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c006e900 0x7fe0c013d940 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.111+0000 7fe0c561c640 1 -- 172.21.15.16:0/1478668342 >> 172.21.15.16:0/1478668342 conn(0x7fe0c006aba0 msgr2=0x7fe0c00716e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:07.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.112+0000 7fe0c561c640 1 -- 172.21.15.16:0/1478668342 shutdown_connections 2024-04-25T10:15:07.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.112+0000 7fe0c561c640 1 -- 172.21.15.16:0/1478668342 wait complete. 2024-04-25T10:15:07.133 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:15:07.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.237+0000 7fdb4cdce640 1 Processor -- start 2024-04-25T10:15:07.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.237+0000 7fdb4cdce640 1 -- start start 2024-04-25T10:15:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.238+0000 7fdb4cdce640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb4815dca0 0x7fdb480ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.238+0000 7fdb4cdce640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb480d4090 con 0x7fdb480d1130 2024-04-25T10:15:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.238+0000 7fdb4cdce640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb480d4370 con 0x7fdb480d3b90 2024-04-25T10:15:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.238+0000 7fdb4cdce640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb480b6b20 con 0x7fdb4815dca0 2024-04-25T10:15:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.238+0000 7fdb46d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdb480d1130 0x7fdb480b2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59424/0 (socket says 172.21.15.16:59424) 2024-04-25T10:15:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.238+0000 7fdb46d9d640 1 -- 172.21.15.16:0/4107876032 learned_addr learned my addr 172.21.15.16:0/4107876032 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.238+0000 7fdb4659c640 1 --2- 172.21.15.16:0/4107876032 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb4815dca0 0x7fdb480ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.239+0000 7fdb4559a640 1 -- 172.21.15.16:0/4107876032 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309738776 0 0) 0x7fdb480d4090 con 0x7fdb480d1130 2024-04-25T10:15:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.239+0000 7fdb4559a640 1 -- 172.21.15.16:0/4107876032 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb34003520 con 0x7fdb480d1130 2024-04-25T10:15:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.239+0000 7fdb4559a640 1 -- 172.21.15.16:0/4107876032 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657435139 0 0) 0x7fdb480d4370 con 0x7fdb480d3b90 2024-04-25T10:15:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.239+0000 7fdb4559a640 1 -- 172.21.15.16:0/4107876032 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb480d4090 con 0x7fdb480d3b90 2024-04-25T10:15:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.239+0000 7fdb4559a640 1 -- 172.21.15.16:0/4107876032 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 814277782 0 0) 0x7fdb34003520 con 0x7fdb480d1130 2024-04-25T10:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.239+0000 7fdb4659c640 1 -- 172.21.15.16:0/4107876032 >> v1:172.21.15.186:6789/0 conn(0x7fdb480d3b90 legacy=0x7fdb480d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.239+0000 7fdb4659c640 1 -- 172.21.15.16:0/4107876032 >> v1:172.21.15.16:6789/0 conn(0x7fdb480d1130 legacy=0x7fdb480b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.239+0000 7fdb4659c640 1 -- 172.21.15.16:0/4107876032 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb480d4650 con 0x7fdb4815dca0 2024-04-25T10:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.240+0000 7fdb4659c640 1 --2- 172.21.15.16:0/4107876032 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb4815dca0 0x7fdb480ce940 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdb400097d0 tx=0x7fdb4002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b0927db800d4285 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.240+0000 7fdb4559a640 1 -- 172.21.15.16:0/4107876032 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb40037070 con 0x7fdb4815dca0 2024-04-25T10:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.240+0000 7fdb4559a640 1 -- 172.21.15.16:0/4107876032 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4002ac30 con 0x7fdb4815dca0 2024-04-25T10:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.240+0000 7fdb4559a640 1 -- 172.21.15.16:0/4107876032 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb380019f0 con 0x7fdb4815dca0 2024-04-25T10:15:07.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.240+0000 7fdb4cdce640 1 -- 172.21.15.16:0/4107876032 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb4815dca0 msgr2=0x7fdb480ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.240+0000 7fdb4cdce640 1 --2- 172.21.15.16:0/4107876032 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb4815dca0 0x7fdb480ce940 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdb400097d0 tx=0x7fdb4002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.240+0000 7fdb4cdce640 1 -- 172.21.15.16:0/4107876032 shutdown_connections 2024-04-25T10:15:07.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.240+0000 7fdb4cdce640 1 --2- 172.21.15.16:0/4107876032 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb4815dca0 0x7fdb480ce940 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.240+0000 7fdb4cdce640 1 -- 172.21.15.16:0/4107876032 >> 172.21.15.16:0/4107876032 conn(0x7fdb480726c0 msgr2=0x7fdb48072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.240+0000 7fdb4cdce640 1 -- 172.21.15.16:0/4107876032 shutdown_connections 2024-04-25T10:15:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.241+0000 7fdb4cdce640 1 -- 172.21.15.16:0/4107876032 wait complete. 2024-04-25T10:15:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.242+0000 7fdb4cdce640 1 Processor -- start 2024-04-25T10:15:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.242+0000 7fdb4cdce640 1 -- start start 2024-04-25T10:15:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb4cdce640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb480d1130 0x7fdb480ab120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb4cdce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb480d3b90 0x7fdb480ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb4cdce640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb4815dca0 0x7fdb481f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb4cdce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4815e200 con 0x7fdb480d3b90 2024-04-25T10:15:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb4cdce640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4815e080 con 0x7fdb4815dca0 2024-04-25T10:15:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb4cdce640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb480cee80 con 0x7fdb480d1130 2024-04-25T10:15:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb45d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb480d3b90 0x7fdb480ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb45d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb480d3b90 0x7fdb480ccb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56364/0 (socket says 172.21.15.16:56364) 2024-04-25T10:15:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb45d9b640 1 -- 172.21.15.16:0/1351520472 learned_addr learned my addr 172.21.15.16:0/1351520472 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb46d9d640 1 --2- 172.21.15.16:0/1351520472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb4815dca0 0x7fdb481f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb4659c640 1 --2- 172.21.15.16:0/1351520472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb480d1130 0x7fdb480ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb45d9b640 1 -- 172.21.15.16:0/1351520472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb480d1130 msgr2=0x7fdb480ab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb45d9b640 1 --2- 172.21.15.16:0/1351520472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb480d1130 0x7fdb480ab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb45d9b640 1 -- 172.21.15.16:0/1351520472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb4815dca0 msgr2=0x7fdb481f6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb45d9b640 1 --2- 172.21.15.16:0/1351520472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb4815dca0 0x7fdb481f6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb45d9b640 1 -- 172.21.15.16:0/1351520472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb480cdb80 con 0x7fdb480d3b90 2024-04-25T10:15:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb45d9b640 1 --2- 172.21.15.16:0/1351520472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb480d3b90 0x7fdb480ccb70 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000c500 tx=0x7fdb3000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb3effd640 1 -- 172.21.15.16:0/1351520472 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb300052d0 con 0x7fdb480d3b90 2024-04-25T10:15:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb3effd640 1 -- 172.21.15.16:0/1351520472 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb30005490 con 0x7fdb480d3b90 2024-04-25T10:15:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb3effd640 1 -- 172.21.15.16:0/1351520472 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb30012870 con 0x7fdb480d3b90 2024-04-25T10:15:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb4cdce640 1 -- 172.21.15.16:0/1351520472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb480cd2e0 con 0x7fdb480d3b90 2024-04-25T10:15:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.243+0000 7fdb4cdce640 1 -- 172.21.15.16:0/1351520472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb480cd8a0 con 0x7fdb480d3b90 2024-04-25T10:15:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.245+0000 7fdb4cdce640 1 -- 172.21.15.16:0/1351520472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb480750b0 con 0x7fdb480d3b90 2024-04-25T10:15:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.245+0000 7fdb3effd640 1 -- 172.21.15.16:0/1351520472 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb30012a10 con 0x7fdb480d3b90 2024-04-25T10:15:07.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.245+0000 7fdb3effd640 1 --2- 172.21.15.16:0/1351520472 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb2806b4d0 0x7fdb2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.245+0000 7fdb3effd640 1 -- 172.21.15.16:0/1351520472 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fdb30089b50 con 0x7fdb480d3b90 2024-04-25T10:15:07.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.246+0000 7fdb4659c640 1 --2- 172.21.15.16:0/1351520472 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb2806b4d0 0x7fdb2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.246+0000 7fdb4659c640 1 --2- 172.21.15.16:0/1351520472 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb2806b4d0 0x7fdb2806d990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fdb40034000 tx=0x7fdb4002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.250+0000 7fdb3effd640 1 -- 172.21.15.16:0/1351520472 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb300585e0 con 0x7fdb480d3b90 2024-04-25T10:15:07.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.434+0000 7fdb4cdce640 1 -- 172.21.15.16:0/1351520472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdb480ce750 con 0x7fdb480d3b90 2024-04-25T10:15:07.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.437+0000 7fdb3effd640 1 -- 172.21.15.16:0/1351520472 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdb3005c680 con 0x7fdb480d3b90 2024-04-25T10:15:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.440+0000 7fdb4cdce640 1 -- 172.21.15.16:0/1351520472 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb2806b4d0 msgr2=0x7fdb2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.440+0000 7fdb4cdce640 1 --2- 172.21.15.16:0/1351520472 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb2806b4d0 0x7fdb2806d990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fdb40034000 tx=0x7fdb4002e000 comp rx=0 tx=0).stop 2024-04-25T10:15:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.440+0000 7fdb4cdce640 1 -- 172.21.15.16:0/1351520472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb480d3b90 msgr2=0x7fdb480ccb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.440+0000 7fdb4cdce640 1 --2- 172.21.15.16:0/1351520472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb480d3b90 0x7fdb480ccb70 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000c500 tx=0x7fdb3000c9c0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.440+0000 7fdb4cdce640 1 -- 172.21.15.16:0/1351520472 shutdown_connections 2024-04-25T10:15:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.440+0000 7fdb4cdce640 1 --2- 172.21.15.16:0/1351520472 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fdb2806b4d0 0x7fdb2806d990 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.440+0000 7fdb4cdce640 1 --2- 172.21.15.16:0/1351520472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb4815dca0 0x7fdb481f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.440+0000 7fdb4cdce640 1 --2- 172.21.15.16:0/1351520472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb480d3b90 0x7fdb480ccb70 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.440+0000 7fdb4cdce640 1 --2- 172.21.15.16:0/1351520472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb480d1130 0x7fdb480ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.440+0000 7fdb4cdce640 1 -- 172.21.15.16:0/1351520472 >> 172.21.15.16:0/1351520472 conn(0x7fdb480726c0 msgr2=0x7fdb480d1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.440+0000 7fdb4cdce640 1 -- 172.21.15.16:0/1351520472 shutdown_connections 2024-04-25T10:15:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.440+0000 7fdb4cdce640 1 -- 172.21.15.16:0/1351520472 wait complete. 2024-04-25T10:15:07.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-25T10:15:07.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.551+0000 7ffbabef5640 1 Processor -- start 2024-04-25T10:15:07.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.551+0000 7ffbabef5640 1 -- start start 2024-04-25T10:15:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.552+0000 7ffbabef5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba415dce0 0x7ffba40d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.552+0000 7ffbabef5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ffba40d35b0 0x7ffba40b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.552+0000 7ffbabef5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffba40d1f90 0x7ffba40b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.552+0000 7ffbabef5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba415e0c0 con 0x7ffba40d1f90 2024-04-25T10:15:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.552+0000 7ffbabef5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba40d2510 con 0x7ffba40d35b0 2024-04-25T10:15:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.552+0000 7ffbabef5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba40d27f0 con 0x7ffba415dce0 2024-04-25T10:15:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.552+0000 7ffbaa492640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffba40d1f90 0x7ffba40b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.552+0000 7ffbaa492640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffba40d1f90 0x7ffba40b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56366/0 (socket says 172.21.15.16:56366) 2024-04-25T10:15:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.552+0000 7ffbaa492640 1 -- 172.21.15.16:0/2325369722 learned_addr learned my addr 172.21.15.16:0/2325369722 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.552+0000 7ffba9490640 1 --2- 172.21.15.16:0/2325369722 >> v2:172.21.15.186:3300/0 conn(0x7ffba40d35b0 0x7ffba40b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.552+0000 7ffba9c91640 1 --2- 172.21.15.16:0/2325369722 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba415dce0 0x7ffba40d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.553+0000 7ffbaa492640 1 -- 172.21.15.16:0/2325369722 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba415dce0 msgr2=0x7ffba40d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.553+0000 7ffbaa492640 1 --2- 172.21.15.16:0/2325369722 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba415dce0 0x7ffba40d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.553+0000 7ffbaa492640 1 -- 172.21.15.16:0/2325369722 >> v2:172.21.15.186:3300/0 conn(0x7ffba40d35b0 msgr2=0x7ffba40b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.553+0000 7ffbaa492640 1 --2- 172.21.15.16:0/2325369722 >> v2:172.21.15.186:3300/0 conn(0x7ffba40d35b0 0x7ffba40b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.553+0000 7ffbaa492640 1 -- 172.21.15.16:0/2325369722 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba40d2ad0 con 0x7ffba40d1f90 2024-04-25T10:15:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.553+0000 7ffba9490640 1 --2- 172.21.15.16:0/2325369722 >> v2:172.21.15.186:3300/0 conn(0x7ffba40d35b0 0x7ffba40b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.553+0000 7ffbaa492640 1 --2- 172.21.15.16:0/2325369722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba40d1f90 0x7ffba40b5990 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7ffba406d0d0 tx=0x7ffb9402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8b35f570dc9739 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.553+0000 7ffba8c8f640 1 -- 172.21.15.16:0/2325369722 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94036070 con 0x7ffba40d1f90 2024-04-25T10:15:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.553+0000 7ffba8c8f640 1 -- 172.21.15.16:0/2325369722 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb94031600 con 0x7ffba40d1f90 2024-04-25T10:15:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.553+0000 7ffba8c8f640 1 -- 172.21.15.16:0/2325369722 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb940319d0 con 0x7ffba40d1f90 2024-04-25T10:15:07.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.554+0000 7ffbabef5640 1 -- 172.21.15.16:0/2325369722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba40d1f90 msgr2=0x7ffba40b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.554+0000 7ffbabef5640 1 --2- 172.21.15.16:0/2325369722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba40d1f90 0x7ffba40b5990 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7ffba406d0d0 tx=0x7ffb9402a610 comp rx=0 tx=0).stop 2024-04-25T10:15:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.554+0000 7ffbabef5640 1 -- 172.21.15.16:0/2325369722 shutdown_connections 2024-04-25T10:15:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.554+0000 7ffbabef5640 1 --2- 172.21.15.16:0/2325369722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba40d1f90 0x7ffba40b5990 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.554+0000 7ffbabef5640 1 --2- 172.21.15.16:0/2325369722 >> v2:172.21.15.186:3300/0 conn(0x7ffba40d35b0 0x7ffba40b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.554+0000 7ffbabef5640 1 --2- 172.21.15.16:0/2325369722 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba415dce0 0x7ffba40d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.554+0000 7ffbabef5640 1 -- 172.21.15.16:0/2325369722 >> 172.21.15.16:0/2325369722 conn(0x7ffba4072900 msgr2=0x7ffba40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.554+0000 7ffbabef5640 1 -- 172.21.15.16:0/2325369722 shutdown_connections 2024-04-25T10:15:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.554+0000 7ffbabef5640 1 -- 172.21.15.16:0/2325369722 wait complete. 2024-04-25T10:15:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.555+0000 7ffbabef5640 1 Processor -- start 2024-04-25T10:15:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.555+0000 7ffbabef5640 1 -- start start 2024-04-25T10:15:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.556+0000 7ffbabef5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba40d1f90 0x7ffba4149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.556+0000 7ffbabef5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba40d35b0 0x7ffba40c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.556+0000 7ffbabef5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffba415dce0 0x7ffba40c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.556+0000 7ffbabef5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba40b9820 con 0x7ffba40d1f90 2024-04-25T10:15:07.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.556+0000 7ffbabef5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba40d3090 con 0x7ffba415dce0 2024-04-25T10:15:07.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.556+0000 7ffbabef5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba40b99a0 con 0x7ffba40d35b0 2024-04-25T10:15:07.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.556+0000 7ffba9c91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba40d1f90 0x7ffba4149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.556+0000 7ffba9c91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba40d1f90 0x7ffba4149cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56382/0 (socket says 172.21.15.16:56382) 2024-04-25T10:15:07.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.556+0000 7ffba9c91640 1 -- 172.21.15.16:0/1509093250 learned_addr learned my addr 172.21.15.16:0/1509093250 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:07.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.556+0000 7ffbaa492640 1 --2- 172.21.15.16:0/1509093250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffba415dce0 0x7ffba40c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.557+0000 7ffba9490640 1 --2- 172.21.15.16:0/1509093250 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba40d35b0 0x7ffba40c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.557+0000 7ffba9c91640 1 -- 172.21.15.16:0/1509093250 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba40d35b0 msgr2=0x7ffba40c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.557+0000 7ffba9c91640 1 --2- 172.21.15.16:0/1509093250 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba40d35b0 0x7ffba40c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.557+0000 7ffba9c91640 1 -- 172.21.15.16:0/1509093250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffba415dce0 msgr2=0x7ffba40c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.557+0000 7ffba9c91640 1 --2- 172.21.15.16:0/1509093250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffba415dce0 0x7ffba40c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.557+0000 7ffba9c91640 1 -- 172.21.15.16:0/1509093250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba4160e90 con 0x7ffba40d1f90 2024-04-25T10:15:07.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.557+0000 7ffbaa492640 1 --2- 172.21.15.16:0/1509093250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffba415dce0 0x7ffba40c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:07.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.557+0000 7ffba9490640 1 --2- 172.21.15.16:0/1509093250 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba40d35b0 0x7ffba40c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:07.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.557+0000 7ffba9c91640 1 --2- 172.21.15.16:0/1509093250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba40d1f90 0x7ffba4149cd0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00d830 tx=0x7ffb9c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.557+0000 7ffba27fc640 1 -- 172.21.15.16:0/1509093250 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c00b6c0 con 0x7ffba40d1f90 2024-04-25T10:15:07.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.557+0000 7ffba27fc640 1 -- 172.21.15.16:0/1509093250 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9c019070 con 0x7ffba40d1f90 2024-04-25T10:15:07.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.557+0000 7ffbabef5640 1 -- 172.21.15.16:0/1509093250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba414a490 con 0x7ffba40d1f90 2024-04-25T10:15:07.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.557+0000 7ffba27fc640 1 -- 172.21.15.16:0/1509093250 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c00ba80 con 0x7ffba40d1f90 2024-04-25T10:15:07.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.557+0000 7ffbabef5640 1 -- 172.21.15.16:0/1509093250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba414aa50 con 0x7ffba40d1f90 2024-04-25T10:15:07.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.558+0000 7ffbabef5640 1 -- 172.21.15.16:0/1509093250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba4067490 con 0x7ffba40d1f90 2024-04-25T10:15:07.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.558+0000 7ffba27fc640 1 -- 172.21.15.16:0/1509093250 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb9c01e460 con 0x7ffba40d1f90 2024-04-25T10:15:07.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.559+0000 7ffba27fc640 1 --2- 172.21.15.16:0/1509093250 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffb9006b4d0 0x7ffb9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.559+0000 7ffba27fc640 1 -- 172.21.15.16:0/1509093250 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7ffb9c08a5d0 con 0x7ffba40d1f90 2024-04-25T10:15:07.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.559+0000 7ffba9490640 1 --2- 172.21.15.16:0/1509093250 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffb9006b4d0 0x7ffb9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.560+0000 7ffba9490640 1 --2- 172.21.15.16:0/1509093250 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffb9006b4d0 0x7ffb9006d990 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ffb98006fd0 tx=0x7ffb98008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.563+0000 7ffba27fc640 1 -- 172.21.15.16:0/1509093250 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffba4067490 con 0x7ffba40d1f90 2024-04-25T10:15:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:07.818+0000 7ffbabef5640 1 -- 172.21.15.16:0/1509093250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7ffba40749b0 con 0x7ffba40d1f90 2024-04-25T10:15:08.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:08.112+0000 7ffba27fc640 1 -- 172.21.15.16:0/1509093250 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v480) v1 ==== 144+0+0 (secure 0 0 0) 0x7ffb9c059730 con 0x7ffba40d1f90 2024-04-25T10:15:08.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:08.114+0000 7ffbabef5640 1 -- 172.21.15.16:0/1509093250 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffb9006b4d0 msgr2=0x7ffb9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:08.114+0000 7ffbabef5640 1 --2- 172.21.15.16:0/1509093250 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffb9006b4d0 0x7ffb9006d990 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ffb98006fd0 tx=0x7ffb98008040 comp rx=0 tx=0).stop 2024-04-25T10:15:08.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:08.114+0000 7ffbabef5640 1 -- 172.21.15.16:0/1509093250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba40d1f90 msgr2=0x7ffba4149cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:08.114+0000 7ffbabef5640 1 --2- 172.21.15.16:0/1509093250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba40d1f90 0x7ffba4149cd0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00d830 tx=0x7ffb9c00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:08.114+0000 7ffbabef5640 1 -- 172.21.15.16:0/1509093250 shutdown_connections 2024-04-25T10:15:08.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:08.114+0000 7ffbabef5640 1 --2- 172.21.15.16:0/1509093250 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7ffb9006b4d0 0x7ffb9006d990 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:08.114+0000 7ffbabef5640 1 --2- 172.21.15.16:0/1509093250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffba415dce0 0x7ffba40c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:08.114+0000 7ffbabef5640 1 --2- 172.21.15.16:0/1509093250 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffba40d35b0 0x7ffba40c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:08.114+0000 7ffbabef5640 1 --2- 172.21.15.16:0/1509093250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffba40d1f90 0x7ffba4149cd0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:08.114+0000 7ffbabef5640 1 -- 172.21.15.16:0/1509093250 >> 172.21.15.16:0/1509093250 conn(0x7ffba4072900 msgr2=0x7ffba40b8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:08.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:08.115+0000 7ffbabef5640 1 -- 172.21.15.16:0/1509093250 shutdown_connections 2024-04-25T10:15:08.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:08.115+0000 7ffbabef5640 1 -- 172.21.15.16:0/1509093250 wait complete. 2024-04-25T10:15:08.130 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T10:15:08.130 INFO:tasks.cephfs_test_runner: 2024-04-25T10:15:08.130 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T10:15:08.130 INFO:tasks.cephfs_test_runner:Ran 7 tests in 567.554s 2024-04-25T10:15:08.130 INFO:tasks.cephfs_test_runner: 2024-04-25T10:15:08.131 INFO:tasks.cephfs_test_runner:OK 2024-04-25T10:15:08.131 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T10:15:08.141 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T10:15:08.152 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T10:15:08.152 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:15:08.152 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:08.168 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:15:08.168 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-25T10:15:08.168 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:08.224 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4668 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:15:08.224 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:15:08.224 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:08.243 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:08.244 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-25T10:15:08.244 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:08.302 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16608 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:08.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:08.302 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:15:08.318 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:15:08.318 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:15:08.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:08.318 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:15:08.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:08.383 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:15:08.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:08.450 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-04-25T10:15:08.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:08.546 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-25T10:15:08.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:08.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-04-25T10:15:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-04-25T10:15:08.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:15:08.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:08.632 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2024-04-25T10:15:08.652 INFO:teuthology.orchestra.run.smithi186.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T10:15:08.652 INFO:teuthology.orchestra.run.smithi186.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:15:08.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:08.652 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:15:08.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:08.721 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:15:08.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:08.787 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link delete ceph-brx 2024-04-25T10:15:08.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:08.902 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2024-04-25T10:15:08.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:08.918 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link delete ceph-brx 2024-04-25T10:15:08.929 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:08.932 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:08.986 INFO:teuthology.orchestra.run.smithi186.stderr:Cannot find device "ceph-brx" 2024-04-25T10:15:08.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:15:08.988 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T10:15:08.997 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:15:08.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:15:09.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.096+0000 7f73a1458640 1 Processor -- start 2024-04-25T10:15:09.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.096+0000 7f73a1458640 1 -- start start 2024-04-25T10:15:09.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.097+0000 7f73a1458640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f739c15dce0 0x7f739c0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.097+0000 7f73a1458640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f739c0b5b60 0x7f739c0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.097+0000 7f73a1458640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739c15e0c0 con 0x7f739c0d35b0 2024-04-25T10:15:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.097+0000 7f73a1458640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f739c0d2510 con 0x7f739c0b5b60 2024-04-25T10:15:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.097+0000 7f73a1458640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739c0d27f0 con 0x7f739c15dce0 2024-04-25T10:15:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.097+0000 7f739b7fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f739c0b5b60 0x7f739c0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.097+0000 7f739affd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f739c15dce0 0x7f739c0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.097+0000 7f739a7fc640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f739c0d35b0 0x7f739c0b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:54088/0 (socket says 172.21.15.16:54088) 2024-04-25T10:15:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.097+0000 7f739a7fc640 1 -- 172.21.15.16:0/190548695 learned_addr learned my addr 172.21.15.16:0/190548695 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.097+0000 7f739b7fe640 1 -- 172.21.15.16:0/190548695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f739c15dce0 msgr2=0x7f739c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.097+0000 7f739b7fe640 1 --2- 172.21.15.16:0/190548695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f739c15dce0 0x7f739c0d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.097+0000 7f739b7fe640 1 -- 172.21.15.16:0/190548695 >> v1:172.21.15.186:6789/0 conn(0x7f739c0d35b0 legacy=0x7f739c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.097+0000 7f739b7fe640 1 -- 172.21.15.16:0/190548695 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c0d2ad0 con 0x7f739c0b5b60 2024-04-25T10:15:09.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.097+0000 7f739affd640 1 --2- 172.21.15.16:0/190548695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f739c15dce0 0x7f739c0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:09.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.098+0000 7f739b7fe640 1 --2- 172.21.15.16:0/190548695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f739c0b5b60 0x7f739c0b6b00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7394009d90 tx=0x7f739402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7b38d12fb1a4a1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:09.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.098+0000 7f7399ffb640 1 -- 172.21.15.16:0/190548695 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394007740 con 0x7f739c0b5b60 2024-04-25T10:15:09.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.098+0000 7f7399ffb640 1 -- 172.21.15.16:0/190548695 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73940078e0 con 0x7f739c0b5b60 2024-04-25T10:15:09.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.098+0000 7f7399ffb640 1 -- 172.21.15.16:0/190548695 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394007cb0 con 0x7f739c0b5b60 2024-04-25T10:15:09.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.099+0000 7f73a1458640 1 -- 172.21.15.16:0/190548695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f739c0b5b60 msgr2=0x7f739c0b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.099+0000 7f73a1458640 1 --2- 172.21.15.16:0/190548695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f739c0b5b60 0x7f739c0b6b00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7394009d90 tx=0x7f739402a660 comp rx=0 tx=0).stop 2024-04-25T10:15:09.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.099+0000 7f73a1458640 1 -- 172.21.15.16:0/190548695 shutdown_connections 2024-04-25T10:15:09.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.099+0000 7f73a1458640 1 --2- 172.21.15.16:0/190548695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f739c0b5b60 0x7f739c0b6b00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.099+0000 7f73a1458640 1 --2- 172.21.15.16:0/190548695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f739c15dce0 0x7f739c0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.099+0000 7f73a1458640 1 -- 172.21.15.16:0/190548695 >> 172.21.15.16:0/190548695 conn(0x7f739c072900 msgr2=0x7f739c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:09.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.099+0000 7f73a1458640 1 -- 172.21.15.16:0/190548695 shutdown_connections 2024-04-25T10:15:09.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.099+0000 7f73a1458640 1 -- 172.21.15.16:0/190548695 wait complete. 2024-04-25T10:15:09.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.100+0000 7f73a1458640 1 Processor -- start 2024-04-25T10:15:09.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.100+0000 7f73a1458640 1 -- start start 2024-04-25T10:15:09.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.101+0000 7f73a1458640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f739c0b5b60 0x7f739c152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.101+0000 7f73a1458640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f739c0d35b0 0x7f739c151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.101+0000 7f73a1458640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f739c15dce0 0x7f739c0c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.101+0000 7f73a1458640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c0b9820 con 0x7f739c0b5b60 2024-04-25T10:15:09.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.101+0000 7f73a1458640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c0d3090 con 0x7f739c15dce0 2024-04-25T10:15:09.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.101+0000 7f73a1458640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739c0b99a0 con 0x7f739c0d35b0 2024-04-25T10:15:09.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.101+0000 7f739affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f739c0b5b60 0x7f739c152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.101+0000 7f739affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f739c0b5b60 0x7f739c152190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56392/0 (socket says 172.21.15.16:56392) 2024-04-25T10:15:09.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.101+0000 7f739affd640 1 -- 172.21.15.16:0/351774657 learned_addr learned my addr 172.21.15.16:0/351774657 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:09.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.101+0000 7f739b7fe640 1 --2- 172.21.15.16:0/351774657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f739c15dce0 0x7f739c0c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.101+0000 7f739a7fc640 1 --2- 172.21.15.16:0/351774657 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f739c0d35b0 0x7f739c151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.102+0000 7f739affd640 1 -- 172.21.15.16:0/351774657 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f739c0d35b0 msgr2=0x7f739c151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.102+0000 7f739affd640 1 --2- 172.21.15.16:0/351774657 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f739c0d35b0 0x7f739c151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.102+0000 7f739affd640 1 -- 172.21.15.16:0/351774657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f739c15dce0 msgr2=0x7f739c0c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.102+0000 7f739affd640 1 --2- 172.21.15.16:0/351774657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f739c15dce0 0x7f739c0c9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.102+0000 7f739affd640 1 -- 172.21.15.16:0/351774657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c151f40 con 0x7f739c0b5b60 2024-04-25T10:15:09.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.102+0000 7f739b7fe640 1 --2- 172.21.15.16:0/351774657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f739c15dce0 0x7f739c0c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:09.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.102+0000 7f739affd640 1 --2- 172.21.15.16:0/351774657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f739c0b5b60 0x7f739c152190 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f738c00e990 tx=0x7f738c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:09.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.102+0000 7f73837fe640 1 -- 172.21.15.16:0/351774657 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c017070 con 0x7f739c0b5b60 2024-04-25T10:15:09.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.102+0000 7f73837fe640 1 -- 172.21.15.16:0/351774657 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738c00c9e0 con 0x7f739c0b5b60 2024-04-25T10:15:09.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.102+0000 7f73a1458640 1 -- 172.21.15.16:0/351774657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f739c0c7750 con 0x7f739c0b5b60 2024-04-25T10:15:09.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.102+0000 7f73837fe640 1 -- 172.21.15.16:0/351774657 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c00cdb0 con 0x7f739c0b5b60 2024-04-25T10:15:09.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.102+0000 7f73a1458640 1 -- 172.21.15.16:0/351774657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739c160970 con 0x7f739c0b5b60 2024-04-25T10:15:09.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.103+0000 7f73a1458640 1 -- 172.21.15.16:0/351774657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f739c0749b0 con 0x7f739c0b5b60 2024-04-25T10:15:09.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.104+0000 7f73837fe640 1 -- 172.21.15.16:0/351774657 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f738c01a070 con 0x7f739c0b5b60 2024-04-25T10:15:09.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.104+0000 7f73837fe640 1 --2- 172.21.15.16:0/351774657 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f738806b4d0 0x7f738806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.104+0000 7f73837fe640 1 -- 172.21.15.16:0/351774657 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f738c089d30 con 0x7f739c0b5b60 2024-04-25T10:15:09.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.104+0000 7f739a7fc640 1 --2- 172.21.15.16:0/351774657 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f738806b4d0 0x7f738806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.105+0000 7f739a7fc640 1 --2- 172.21.15.16:0/351774657 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f738806b4d0 0x7f738806d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f739c0cc710 tx=0x7f7390007e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:09.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.108+0000 7f73837fe640 1 -- 172.21.15.16:0/351774657 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f738c058840 con 0x7f739c0b5b60 2024-04-25T10:15:09.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.285+0000 7f73a1458640 1 -- 172.21.15.16:0/351774657 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f739c0c7500 con 0x7f738806b4d0 2024-04-25T10:15:09.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.287+0000 7f73837fe640 1 -- 172.21.15.16:0/351774657 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13483 (secure 0 0 0) 0x7f739c0c7500 con 0x7f738806b4d0 2024-04-25T10:15:09.290 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:15:09.291 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:15:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.288+0000 7f73a1458640 1 -- 172.21.15.16:0/351774657 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f738806b4d0 msgr2=0x7f738806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.288+0000 7f73a1458640 1 --2- 172.21.15.16:0/351774657 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f738806b4d0 0x7f738806d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f739c0cc710 tx=0x7f7390007e80 comp rx=0 tx=0).stop 2024-04-25T10:15:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.288+0000 7f73a1458640 1 -- 172.21.15.16:0/351774657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f739c0b5b60 msgr2=0x7f739c152190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.288+0000 7f73a1458640 1 --2- 172.21.15.16:0/351774657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f739c0b5b60 0x7f739c152190 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f738c00e990 tx=0x7f738c00ee50 comp rx=0 tx=0).stop 2024-04-25T10:15:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.288+0000 7f73a1458640 1 -- 172.21.15.16:0/351774657 shutdown_connections 2024-04-25T10:15:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.288+0000 7f73a1458640 1 --2- 172.21.15.16:0/351774657 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f738806b4d0 0x7f738806d990 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.288+0000 7f73a1458640 1 --2- 172.21.15.16:0/351774657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f739c15dce0 0x7f739c0c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.288+0000 7f73a1458640 1 --2- 172.21.15.16:0/351774657 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f739c0d35b0 0x7f739c151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.288+0000 7f73a1458640 1 --2- 172.21.15.16:0/351774657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f739c0b5b60 0x7f739c152190 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.288+0000 7f73a1458640 1 -- 172.21.15.16:0/351774657 >> 172.21.15.16:0/351774657 conn(0x7f739c072900 msgr2=0x7f739c0b7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.289+0000 7f73a1458640 1 -- 172.21.15.16:0/351774657 shutdown_connections 2024-04-25T10:15:09.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.289+0000 7f73a1458640 1 -- 172.21.15.16:0/351774657 wait complete. 2024-04-25T10:15:09.303 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2024-04-25T10:15:07.929130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":291328,"kb_used_data":25232,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754683392,"statfs":{"total":773094113280,"available":772795793408,"internally_reserved":0,"allocated":25837568,"data_stored":19007365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390399,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-256,"num_read_kb":-266,"num_write":-169,"num_write_kb":-8300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001487"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":375,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T10:15:07.109634+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:15:07.109634+0000","last_peered":"2024-04-25T10:15:07.109634+0000","last_clean":"2024-04-25T10:15:07.109634+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:15:07.109634+0000","last_undegraded":"2024-04-25T10:15:07.109634+0000","last_fullsized":"2024-04-25T10:15:07.109634+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509565,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":5564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":5697536,"data_stored":4847660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509565,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32660,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1515520,"data_stored":659105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509566,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40844,"kb_used_data":9660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330996,"statfs":{"total":96636764160,"available":96594939904,"internally_reserved":0,"allocated":9891840,"data_stored":9041966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509566,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36816,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335024,"statfs":{"total":96636764160,"available":96599064576,"internally_reserved":0,"allocated":1507328,"data_stored":653361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509567,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36824,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":1515520,"data_stored":659105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509565,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37400,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334440,"statfs":{"total":96636764160,"available":96598466560,"internally_reserved":0,"allocated":2109440,"data_stored":1249473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509567,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36820,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":1515520,"data_stored":659105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574973,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33216,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338624,"statfs":{"total":96636764160,"available":96602750976,"internally_reserved":0,"allocated":2084864,"data_stored":1237590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:15:09.304 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:15:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.400+0000 7fb8d1b6a640 1 Processor -- start 2024-04-25T10:15:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.401+0000 7fb8d1b6a640 1 -- start start 2024-04-25T10:15:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.401+0000 7fb8d1b6a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8cc15f120 0x7fb8cc0ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.401+0000 7fb8d1b6a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8cc0cfdb0 0x7fb8cc0ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.401+0000 7fb8d1b6a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8cc0d0430 con 0x7fb8cc0ce830 2024-04-25T10:15:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.401+0000 7fb8d1b6a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8cc0d09f0 con 0x7fb8cc0cfdb0 2024-04-25T10:15:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.401+0000 7fb8d1b6a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc0cec70 con 0x7fb8cc15f120 2024-04-25T10:15:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.401+0000 7fb8cbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8cc0cfdb0 0x7fb8cc0ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.401+0000 7fb8caffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb8cc0ce830 0x7fb8cc0cf6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59438/0 (socket says 172.21.15.16:59438) 2024-04-25T10:15:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.401+0000 7fb8caffd640 1 -- 172.21.15.16:0/751505595 learned_addr learned my addr 172.21.15.16:0/751505595 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.402+0000 7fb8cb7fe640 1 --2- 172.21.15.16:0/751505595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8cc15f120 0x7fb8cc0ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.402+0000 7fb8cbfff640 1 -- 172.21.15.16:0/751505595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8cc15f120 msgr2=0x7fb8cc0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.402+0000 7fb8cbfff640 1 --2- 172.21.15.16:0/751505595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8cc15f120 0x7fb8cc0ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.402+0000 7fb8cbfff640 1 -- 172.21.15.16:0/751505595 >> v1:172.21.15.16:6789/0 conn(0x7fb8cc0ce830 legacy=0x7fb8cc0cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.402+0000 7fb8cbfff640 1 -- 172.21.15.16:0/751505595 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8cc0d0710 con 0x7fb8cc0cfdb0 2024-04-25T10:15:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.402+0000 7fb8ca7fc640 1 -- 172.21.15.16:0/751505595 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129802777 0 0) 0x7fb8cc0d0430 con 0x7fb8cc0ce830 2024-04-25T10:15:09.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.402+0000 7fb8cbfff640 1 --2- 172.21.15.16:0/751505595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc0cfdb0 0x7fb8cc0ccfd0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009d90 tx=0x7fb8c402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3438c0d923178297 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:09.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.403+0000 7fb8ca7fc640 1 -- 172.21.15.16:0/751505595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4007740 con 0x7fb8cc0cfdb0 2024-04-25T10:15:09.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.403+0000 7fb8ca7fc640 1 -- 172.21.15.16:0/751505595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c40078e0 con 0x7fb8cc0cfdb0 2024-04-25T10:15:09.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.403+0000 7fb8ca7fc640 1 -- 172.21.15.16:0/751505595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4007cb0 con 0x7fb8cc0cfdb0 2024-04-25T10:15:09.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.403+0000 7fb8d1b6a640 1 -- 172.21.15.16:0/751505595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc0cfdb0 msgr2=0x7fb8cc0ccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.403+0000 7fb8d1b6a640 1 --2- 172.21.15.16:0/751505595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc0cfdb0 0x7fb8cc0ccfd0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009d90 tx=0x7fb8c402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.403+0000 7fb8d1b6a640 1 -- 172.21.15.16:0/751505595 shutdown_connections 2024-04-25T10:15:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.403+0000 7fb8d1b6a640 1 --2- 172.21.15.16:0/751505595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc0cfdb0 0x7fb8cc0ccfd0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.403+0000 7fb8d1b6a640 1 --2- 172.21.15.16:0/751505595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8cc15f120 0x7fb8cc0ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.403+0000 7fb8d1b6a640 1 -- 172.21.15.16:0/751505595 >> 172.21.15.16:0/751505595 conn(0x7fb8cc0d1900 msgr2=0x7fb8cc0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.403+0000 7fb8d1b6a640 1 -- 172.21.15.16:0/751505595 shutdown_connections 2024-04-25T10:15:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.403+0000 7fb8d1b6a640 1 -- 172.21.15.16:0/751505595 wait complete. 2024-04-25T10:15:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.404+0000 7fb8d1b6a640 1 Processor -- start 2024-04-25T10:15:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.405+0000 7fb8d1b6a640 1 -- start start 2024-04-25T10:15:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.405+0000 7fb8d1b6a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8cc0ce830 0x7fb8cc155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.405+0000 7fb8d1b6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc0cfdb0 0x7fb8cc0affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.405+0000 7fb8d1b6a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8cc15f120 0x7fb8cc0ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.405+0000 7fb8d1b6a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc0cc480 con 0x7fb8cc0cfdb0 2024-04-25T10:15:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.405+0000 7fb8d1b6a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc0ccab0 con 0x7fb8cc0ce830 2024-04-25T10:15:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.405+0000 7fb8d1b6a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc0c5450 con 0x7fb8cc15f120 2024-04-25T10:15:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.405+0000 7fb8caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc0cfdb0 0x7fb8cc0affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.405+0000 7fb8caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc0cfdb0 0x7fb8cc0affe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56400/0 (socket says 172.21.15.16:56400) 2024-04-25T10:15:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.405+0000 7fb8caffd640 1 -- 172.21.15.16:0/542818226 learned_addr learned my addr 172.21.15.16:0/542818226 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.406+0000 7fb8cb7fe640 1 --2- 172.21.15.16:0/542818226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8cc0ce830 0x7fb8cc155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.406+0000 7fb8cbfff640 1 --2- 172.21.15.16:0/542818226 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8cc15f120 0x7fb8cc0ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.406+0000 7fb8caffd640 1 -- 172.21.15.16:0/542818226 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8cc15f120 msgr2=0x7fb8cc0ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.406+0000 7fb8caffd640 1 --2- 172.21.15.16:0/542818226 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8cc15f120 0x7fb8cc0ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.406+0000 7fb8caffd640 1 -- 172.21.15.16:0/542818226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8cc0ce830 msgr2=0x7fb8cc155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.406+0000 7fb8caffd640 1 --2- 172.21.15.16:0/542818226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8cc0ce830 0x7fb8cc155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.406+0000 7fb8caffd640 1 -- 172.21.15.16:0/542818226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8cc0aef40 con 0x7fb8cc0cfdb0 2024-04-25T10:15:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.406+0000 7fb8caffd640 1 --2- 172.21.15.16:0/542818226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc0cfdb0 0x7fb8cc0affe0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000c4d0 tx=0x7fb8c000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.406+0000 7fb8b3fff640 1 -- 172.21.15.16:0/542818226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0018070 con 0x7fb8cc0cfdb0 2024-04-25T10:15:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.406+0000 7fb8b3fff640 1 -- 172.21.15.16:0/542818226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c0007500 con 0x7fb8cc0cfdb0 2024-04-25T10:15:09.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.406+0000 7fb8d1b6a640 1 -- 172.21.15.16:0/542818226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8cc0af220 con 0x7fb8cc0cfdb0 2024-04-25T10:15:09.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.406+0000 7fb8b3fff640 1 -- 172.21.15.16:0/542818226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0004a20 con 0x7fb8cc0cfdb0 2024-04-25T10:15:09.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.406+0000 7fb8d1b6a640 1 -- 172.21.15.16:0/542818226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8cc0af7e0 con 0x7fb8cc0cfdb0 2024-04-25T10:15:09.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.407+0000 7fb8d1b6a640 1 -- 172.21.15.16:0/542818226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8cc067850 con 0x7fb8cc0cfdb0 2024-04-25T10:15:09.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.408+0000 7fb8b3fff640 1 -- 172.21.15.16:0/542818226 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb8c001b070 con 0x7fb8cc0cfdb0 2024-04-25T10:15:09.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.408+0000 7fb8b3fff640 1 --2- 172.21.15.16:0/542818226 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb8b806b4d0 0x7fb8b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.408+0000 7fb8b3fff640 1 -- 172.21.15.16:0/542818226 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fb8c0089180 con 0x7fb8cc0cfdb0 2024-04-25T10:15:09.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.408+0000 7fb8cb7fe640 1 --2- 172.21.15.16:0/542818226 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb8b806b4d0 0x7fb8b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.409+0000 7fb8cb7fe640 1 --2- 172.21.15.16:0/542818226 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb8b806b4d0 0x7fb8b806d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00aa20 tx=0x7fb8bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.412+0000 7fb8b3fff640 1 -- 172.21.15.16:0/542818226 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb8c0057c90 con 0x7fb8cc0cfdb0 2024-04-25T10:15:09.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.590+0000 7fb8d1b6a640 1 -- 172.21.15.16:0/542818226 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb8cc155ed0 con 0x7fb8b806b4d0 2024-04-25T10:15:09.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.591+0000 7fb8b3fff640 1 -- 172.21.15.16:0/542818226 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13483 (secure 0 0 0) 0x7fb8bc00f8c0 con 0x7fb8b806b4d0 2024-04-25T10:15:09.595 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:15:09.595 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:15:09.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.593+0000 7fb8d1b6a640 1 -- 172.21.15.16:0/542818226 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb8b806b4d0 msgr2=0x7fb8b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.593+0000 7fb8d1b6a640 1 --2- 172.21.15.16:0/542818226 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb8b806b4d0 0x7fb8b806d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00aa20 tx=0x7fb8bc008040 comp rx=0 tx=0).stop 2024-04-25T10:15:09.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.593+0000 7fb8d1b6a640 1 -- 172.21.15.16:0/542818226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc0cfdb0 msgr2=0x7fb8cc0affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.593+0000 7fb8d1b6a640 1 --2- 172.21.15.16:0/542818226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc0cfdb0 0x7fb8cc0affe0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000c4d0 tx=0x7fb8c000c990 comp rx=0 tx=0).stop 2024-04-25T10:15:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.593+0000 7fb8d1b6a640 1 -- 172.21.15.16:0/542818226 shutdown_connections 2024-04-25T10:15:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.593+0000 7fb8d1b6a640 1 --2- 172.21.15.16:0/542818226 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fb8b806b4d0 0x7fb8b806d990 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.593+0000 7fb8d1b6a640 1 --2- 172.21.15.16:0/542818226 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8cc15f120 0x7fb8cc0ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.593+0000 7fb8d1b6a640 1 --2- 172.21.15.16:0/542818226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc0cfdb0 0x7fb8cc0affe0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.593+0000 7fb8d1b6a640 1 --2- 172.21.15.16:0/542818226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8cc0ce830 0x7fb8cc155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.593+0000 7fb8d1b6a640 1 -- 172.21.15.16:0/542818226 >> 172.21.15.16:0/542818226 conn(0x7fb8cc0d1900 msgr2=0x7fb8cc15d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.593+0000 7fb8d1b6a640 1 -- 172.21.15.16:0/542818226 shutdown_connections 2024-04-25T10:15:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.593+0000 7fb8d1b6a640 1 -- 172.21.15.16:0/542818226 wait complete. 2024-04-25T10:15:09.608 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2024-04-25T10:15:07.929130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":291328,"kb_used_data":25232,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754683392,"statfs":{"total":773094113280,"available":772795793408,"internally_reserved":0,"allocated":25837568,"data_stored":19007365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390399,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-256,"num_read_kb":-266,"num_write":-169,"num_write_kb":-8300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001487"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":375,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T10:15:07.109634+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:15:07.109634+0000","last_peered":"2024-04-25T10:15:07.109634+0000","last_clean":"2024-04-25T10:15:07.109634+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:15:07.109634+0000","last_undegraded":"2024-04-25T10:15:07.109634+0000","last_fullsized":"2024-04-25T10:15:07.109634+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509565,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":5564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":5697536,"data_stored":4847660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509565,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32660,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1515520,"data_stored":659105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509566,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40844,"kb_used_data":9660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330996,"statfs":{"total":96636764160,"available":96594939904,"internally_reserved":0,"allocated":9891840,"data_stored":9041966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509566,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36816,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335024,"statfs":{"total":96636764160,"available":96599064576,"internally_reserved":0,"allocated":1507328,"data_stored":653361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509567,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36824,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":1515520,"data_stored":659105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509565,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37400,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334440,"statfs":{"total":96636764160,"available":96598466560,"internally_reserved":0,"allocated":2109440,"data_stored":1249473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509567,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36820,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":1515520,"data_stored":659105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574973,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33216,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338624,"statfs":{"total":96636764160,"available":96602750976,"internally_reserved":0,"allocated":2084864,"data_stored":1237590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:15:09.608 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:15:09.609 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:15:09.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.705+0000 7f5f1f41c640 1 Processor -- start 2024-04-25T10:15:09.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.705+0000 7f5f1f41c640 1 -- start start 2024-04-25T10:15:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.706+0000 7f5f1f41c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5f1815dce0 0x7f5f180d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.706+0000 7f5f1f41c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f180d35b0 0x7f5f180b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.706+0000 7f5f1f41c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5f180d1f90 0x7f5f180b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.706+0000 7f5f1f41c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1815e0c0 con 0x7f5f1815dce0 2024-04-25T10:15:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.706+0000 7f5f1f41c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f180d2510 con 0x7f5f180d1f90 2024-04-25T10:15:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.706+0000 7f5f1f41c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f180d27f0 con 0x7f5f180d35b0 2024-04-25T10:15:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.706+0000 7f5f1d1b8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5f1815dce0 0x7f5f180d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.706+0000 7f5f1d1b8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5f1815dce0 0x7f5f180d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56408/0 (socket says 172.21.15.16:56408) 2024-04-25T10:15:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.706+0000 7f5f1d1b8640 1 -- 172.21.15.16:0/2599900568 learned_addr learned my addr 172.21.15.16:0/2599900568 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.706+0000 7f5f1d9b9640 1 --2- 172.21.15.16:0/2599900568 >> v2:172.21.15.186:3300/0 conn(0x7f5f180d1f90 0x7f5f180b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.706+0000 7f5f1c9b7640 1 --2- 172.21.15.16:0/2599900568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f180d35b0 0x7f5f180b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.707+0000 7f5f1d1b8640 1 -- 172.21.15.16:0/2599900568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f180d35b0 msgr2=0x7f5f180b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.707+0000 7f5f1d1b8640 1 --2- 172.21.15.16:0/2599900568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f180d35b0 0x7f5f180b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.707+0000 7f5f1d1b8640 1 -- 172.21.15.16:0/2599900568 >> v2:172.21.15.186:3300/0 conn(0x7f5f180d1f90 msgr2=0x7f5f180b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.707+0000 7f5f1d1b8640 1 --2- 172.21.15.16:0/2599900568 >> v2:172.21.15.186:3300/0 conn(0x7f5f180d1f90 0x7f5f180b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.707+0000 7f5f1d1b8640 1 -- 172.21.15.16:0/2599900568 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f180d2ad0 con 0x7f5f1815dce0 2024-04-25T10:15:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.707+0000 7f5f1d1b8640 1 --2- 172.21.15.16:0/2599900568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f1815dce0 0x7f5f180d1a50 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f5f18161710 tx=0x7f5f1002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd33c5fd6f5460e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:09.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.707+0000 7f5f17fff640 1 -- 172.21.15.16:0/2599900568 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f10037070 con 0x7f5f1815dce0 2024-04-25T10:15:09.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.708+0000 7f5f17fff640 1 -- 172.21.15.16:0/2599900568 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f10032440 con 0x7f5f1815dce0 2024-04-25T10:15:09.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.708+0000 7f5f17fff640 1 -- 172.21.15.16:0/2599900568 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f10032810 con 0x7f5f1815dce0 2024-04-25T10:15:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.708+0000 7f5f1f41c640 1 -- 172.21.15.16:0/2599900568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f1815dce0 msgr2=0x7f5f180d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.708+0000 7f5f1f41c640 1 --2- 172.21.15.16:0/2599900568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f1815dce0 0x7f5f180d1a50 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f5f18161710 tx=0x7f5f1002a140 comp rx=0 tx=0).stop 2024-04-25T10:15:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.708+0000 7f5f1f41c640 1 -- 172.21.15.16:0/2599900568 shutdown_connections 2024-04-25T10:15:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.708+0000 7f5f1f41c640 1 --2- 172.21.15.16:0/2599900568 >> v2:172.21.15.186:3300/0 conn(0x7f5f180d1f90 0x7f5f180b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.708+0000 7f5f1f41c640 1 --2- 172.21.15.16:0/2599900568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f180d35b0 0x7f5f180b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.708+0000 7f5f1f41c640 1 --2- 172.21.15.16:0/2599900568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f1815dce0 0x7f5f180d1a50 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.708+0000 7f5f1f41c640 1 -- 172.21.15.16:0/2599900568 >> 172.21.15.16:0/2599900568 conn(0x7f5f18072900 msgr2=0x7f5f180d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.708+0000 7f5f1f41c640 1 -- 172.21.15.16:0/2599900568 shutdown_connections 2024-04-25T10:15:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.708+0000 7f5f1f41c640 1 -- 172.21.15.16:0/2599900568 wait complete. 2024-04-25T10:15:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.709+0000 7f5f1f41c640 1 Processor -- start 2024-04-25T10:15:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.709+0000 7f5f1f41c640 1 -- start start 2024-04-25T10:15:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.710+0000 7f5f1f41c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f180d1f90 0x7f5f18151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.710+0000 7f5f1f41c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f180d35b0 0x7f5f180c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.710+0000 7f5f1f41c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f1815dce0 0x7f5f18151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.710+0000 7f5f1f41c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f180b9820 con 0x7f5f180d35b0 2024-04-25T10:15:09.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.710+0000 7f5f1f41c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f180d3090 con 0x7f5f1815dce0 2024-04-25T10:15:09.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.710+0000 7f5f1f41c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f180b99a0 con 0x7f5f180d1f90 2024-04-25T10:15:09.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.710+0000 7f5f1c9b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f180d35b0 0x7f5f180c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.710+0000 7f5f1c9b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f180d35b0 0x7f5f180c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56422/0 (socket says 172.21.15.16:56422) 2024-04-25T10:15:09.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.710+0000 7f5f1c9b7640 1 -- 172.21.15.16:0/197297759 learned_addr learned my addr 172.21.15.16:0/197297759 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:09.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.710+0000 7f5f1d9b9640 1 --2- 172.21.15.16:0/197297759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f1815dce0 0x7f5f18151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.710+0000 7f5f1d1b8640 1 --2- 172.21.15.16:0/197297759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f180d1f90 0x7f5f18151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.711+0000 7f5f1c9b7640 1 -- 172.21.15.16:0/197297759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f180d1f90 msgr2=0x7f5f18151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.711+0000 7f5f1c9b7640 1 --2- 172.21.15.16:0/197297759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f180d1f90 0x7f5f18151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.711+0000 7f5f1c9b7640 1 -- 172.21.15.16:0/197297759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f1815dce0 msgr2=0x7f5f18151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.711+0000 7f5f1c9b7640 1 --2- 172.21.15.16:0/197297759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f1815dce0 0x7f5f18151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.711+0000 7f5f1c9b7640 1 -- 172.21.15.16:0/197297759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f180c8570 con 0x7f5f180d35b0 2024-04-25T10:15:09.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.711+0000 7f5f1c9b7640 1 --2- 172.21.15.16:0/197297759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f180d35b0 0x7f5f180c8030 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00d620 tx=0x7f5f0c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:09.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.711+0000 7f5f15ffb640 1 -- 172.21.15.16:0/197297759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c00b620 con 0x7f5f180d35b0 2024-04-25T10:15:09.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.711+0000 7f5f15ffb640 1 -- 172.21.15.16:0/197297759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c019070 con 0x7f5f180d35b0 2024-04-25T10:15:09.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.711+0000 7f5f1f41c640 1 -- 172.21.15.16:0/197297759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f181523f0 con 0x7f5f180d35b0 2024-04-25T10:15:09.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.711+0000 7f5f15ffb640 1 -- 172.21.15.16:0/197297759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c00ba00 con 0x7f5f180d35b0 2024-04-25T10:15:09.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.711+0000 7f5f1f41c640 1 -- 172.21.15.16:0/197297759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f181529b0 con 0x7f5f180d35b0 2024-04-25T10:15:09.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.712+0000 7f5f1f41c640 1 -- 172.21.15.16:0/197297759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f180749b0 con 0x7f5f180d35b0 2024-04-25T10:15:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.712+0000 7f5f15ffb640 1 -- 172.21.15.16:0/197297759 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f0c00bba0 con 0x7f5f180d35b0 2024-04-25T10:15:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.713+0000 7f5f15ffb640 1 --2- 172.21.15.16:0/197297759 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5f0406b4d0 0x7f5f0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.713+0000 7f5f15ffb640 1 -- 172.21.15.16:0/197297759 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f5f0c08a300 con 0x7f5f180d35b0 2024-04-25T10:15:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.713+0000 7f5f1d1b8640 1 --2- 172.21.15.16:0/197297759 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5f0406b4d0 0x7f5f0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.714+0000 7f5f1d1b8640 1 --2- 172.21.15.16:0/197297759 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5f0406b4d0 0x7f5f0406d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5f10034000 tx=0x7f5f1002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.717+0000 7f5f15ffb640 1 -- 172.21.15.16:0/197297759 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5f0c058cf0 con 0x7f5f180d35b0 2024-04-25T10:15:09.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.894+0000 7f5f1f41c640 1 -- 172.21.15.16:0/197297759 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f180c7200 con 0x7f5f0406b4d0 2024-04-25T10:15:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.895+0000 7f5f15ffb640 1 -- 172.21.15.16:0/197297759 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13483 (secure 0 0 0) 0x7f5f180c7200 con 0x7f5f0406b4d0 2024-04-25T10:15:09.899 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:15:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:15:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.897+0000 7f5f1f41c640 1 -- 172.21.15.16:0/197297759 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5f0406b4d0 msgr2=0x7f5f0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.897+0000 7f5f1f41c640 1 --2- 172.21.15.16:0/197297759 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5f0406b4d0 0x7f5f0406d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5f10034000 tx=0x7f5f1002e000 comp rx=0 tx=0).stop 2024-04-25T10:15:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.897+0000 7f5f1f41c640 1 -- 172.21.15.16:0/197297759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f180d35b0 msgr2=0x7f5f180c8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.897+0000 7f5f1f41c640 1 --2- 172.21.15.16:0/197297759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f180d35b0 0x7f5f180c8030 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00d620 tx=0x7f5f0c00dae0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.897+0000 7f5f1f41c640 1 -- 172.21.15.16:0/197297759 shutdown_connections 2024-04-25T10:15:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.897+0000 7f5f1f41c640 1 --2- 172.21.15.16:0/197297759 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f5f0406b4d0 0x7f5f0406d990 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.897+0000 7f5f1f41c640 1 --2- 172.21.15.16:0/197297759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f1815dce0 0x7f5f18151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.897+0000 7f5f1f41c640 1 --2- 172.21.15.16:0/197297759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f180d35b0 0x7f5f180c8030 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.897+0000 7f5f1f41c640 1 --2- 172.21.15.16:0/197297759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f180d1f90 0x7f5f18151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.897+0000 7f5f1f41c640 1 -- 172.21.15.16:0/197297759 >> 172.21.15.16:0/197297759 conn(0x7f5f18072900 msgr2=0x7f5f1806bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.897+0000 7f5f1f41c640 1 -- 172.21.15.16:0/197297759 shutdown_connections 2024-04-25T10:15:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:09.897+0000 7f5f1f41c640 1 -- 172.21.15.16:0/197297759 wait complete. 2024-04-25T10:15:09.912 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2024-04-25T10:15:07.929130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":291328,"kb_used_data":25232,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754683392,"statfs":{"total":773094113280,"available":772795793408,"internally_reserved":0,"allocated":25837568,"data_stored":19007365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390399,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-256,"num_read_kb":-266,"num_write":-169,"num_write_kb":-8300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001487"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":375,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T10:15:07.109634+0000","last_change":"2024-04-25T10:04:59.850466+0000","last_active":"2024-04-25T10:15:07.109634+0000","last_peered":"2024-04-25T10:15:07.109634+0000","last_clean":"2024-04-25T10:15:07.109634+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:15:07.109634+0000","last_undegraded":"2024-04-25T10:15:07.109634+0000","last_fullsized":"2024-04-25T10:15:07.109634+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:58.390788+0000","last_clean_scrub_stamp":"2024-04-25T10:04:58.390788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:41.241431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509565,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":5564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":5697536,"data_stored":4847660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509565,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32660,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1515520,"data_stored":659105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509566,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40844,"kb_used_data":9660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330996,"statfs":{"total":96636764160,"available":96594939904,"internally_reserved":0,"allocated":9891840,"data_stored":9041966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509566,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36816,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335024,"statfs":{"total":96636764160,"available":96599064576,"internally_reserved":0,"allocated":1507328,"data_stored":653361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509567,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36824,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":1515520,"data_stored":659105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509565,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37400,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334440,"statfs":{"total":96636764160,"available":96598466560,"internally_reserved":0,"allocated":2109440,"data_stored":1249473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509567,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36820,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":1515520,"data_stored":659105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574973,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33216,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338624,"statfs":{"total":96636764160,"available":96602750976,"internally_reserved":0,"allocated":2084864,"data_stored":1237590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:15:09.912 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:10.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.009+0000 7fac106b3640 1 Processor -- start 2024-04-25T10:15:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.010+0000 7fac106b3640 1 -- start start 2024-04-25T10:15:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.010+0000 7fac106b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac0815dce0 0x7fac080d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.010+0000 7fac106b3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fac080d35b0 0x7fac080b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.010+0000 7fac106b3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fac080d1f90 0x7fac080b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.010+0000 7fac106b3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac0815e0c0 con 0x7fac080d35b0 2024-04-25T10:15:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.010+0000 7fac106b3640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac080d2510 con 0x7fac080d1f90 2024-04-25T10:15:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.011+0000 7fac106b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac080d27f0 con 0x7fac0815dce0 2024-04-25T10:15:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.011+0000 7fac0dc4e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fac080d35b0 0x7fac080b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.011+0000 7fac0dc4e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fac080d35b0 0x7fac080b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56424/0 (socket says 172.21.15.16:56424) 2024-04-25T10:15:10.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.011+0000 7fac0dc4e640 1 -- 172.21.15.16:0/2854068120 learned_addr learned my addr 172.21.15.16:0/2854068120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:10.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.011+0000 7fac0ec50640 1 --2- 172.21.15.16:0/2854068120 >> v2:172.21.15.186:3300/0 conn(0x7fac080d1f90 0x7fac080b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.011+0000 7fac0e44f640 1 --2- 172.21.15.16:0/2854068120 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac0815dce0 0x7fac080d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.011+0000 7fac0dc4e640 1 -- 172.21.15.16:0/2854068120 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac0815dce0 msgr2=0x7fac080d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:10.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.011+0000 7fac0dc4e640 1 --2- 172.21.15.16:0/2854068120 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac0815dce0 0x7fac080d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.011+0000 7fac0dc4e640 1 -- 172.21.15.16:0/2854068120 >> v2:172.21.15.186:3300/0 conn(0x7fac080d1f90 msgr2=0x7fac080b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:10.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.011+0000 7fac0dc4e640 1 --2- 172.21.15.16:0/2854068120 >> v2:172.21.15.186:3300/0 conn(0x7fac080d1f90 0x7fac080b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.011+0000 7fac0dc4e640 1 -- 172.21.15.16:0/2854068120 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac080d2ad0 con 0x7fac080d35b0 2024-04-25T10:15:10.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.011+0000 7fac0e44f640 1 --2- 172.21.15.16:0/2854068120 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac0815dce0 0x7fac080d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.012+0000 7fac0dc4e640 1 --2- 172.21.15.16:0/2854068120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac080d35b0 0x7fac080b5450 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fac08161710 tx=0x7fabfc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eada16977ca6896 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.012+0000 7fac0d44d640 1 -- 172.21.15.16:0/2854068120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc037070 con 0x7fac080d35b0 2024-04-25T10:15:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.012+0000 7fac0d44d640 1 -- 172.21.15.16:0/2854068120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc032440 con 0x7fac080d35b0 2024-04-25T10:15:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.012+0000 7fac0d44d640 1 -- 172.21.15.16:0/2854068120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc032810 con 0x7fac080d35b0 2024-04-25T10:15:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.012+0000 7fac106b3640 1 -- 172.21.15.16:0/2854068120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac080d35b0 msgr2=0x7fac080b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.012+0000 7fac106b3640 1 --2- 172.21.15.16:0/2854068120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac080d35b0 0x7fac080b5450 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fac08161710 tx=0x7fabfc02a140 comp rx=0 tx=0).stop 2024-04-25T10:15:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.013+0000 7fac106b3640 1 -- 172.21.15.16:0/2854068120 shutdown_connections 2024-04-25T10:15:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.013+0000 7fac106b3640 1 --2- 172.21.15.16:0/2854068120 >> v2:172.21.15.186:3300/0 conn(0x7fac080d1f90 0x7fac080b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.013+0000 7fac106b3640 1 --2- 172.21.15.16:0/2854068120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac080d35b0 0x7fac080b5450 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.013+0000 7fac106b3640 1 --2- 172.21.15.16:0/2854068120 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac0815dce0 0x7fac080d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.013+0000 7fac106b3640 1 -- 172.21.15.16:0/2854068120 >> 172.21.15.16:0/2854068120 conn(0x7fac08072900 msgr2=0x7fac080d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:10.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.013+0000 7fac106b3640 1 -- 172.21.15.16:0/2854068120 shutdown_connections 2024-04-25T10:15:10.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.013+0000 7fac106b3640 1 -- 172.21.15.16:0/2854068120 wait complete. 2024-04-25T10:15:10.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.014+0000 7fac106b3640 1 Processor -- start 2024-04-25T10:15:10.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.014+0000 7fac106b3640 1 -- start start 2024-04-25T10:15:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fac106b3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac080d1f90 0x7fac080cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fac106b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac080d35b0 0x7fac080cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fac106b3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac0815dce0 0x7fac08152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fac106b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac080b9820 con 0x7fac080d35b0 2024-04-25T10:15:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fac106b3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac080d3090 con 0x7fac0815dce0 2024-04-25T10:15:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fac106b3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac080b99a0 con 0x7fac080d1f90 2024-04-25T10:15:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fac0dc4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac080d35b0 0x7fac080cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fac0dc4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac080d35b0 0x7fac080cd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56434/0 (socket says 172.21.15.16:56434) 2024-04-25T10:15:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fac0dc4e640 1 -- 172.21.15.16:0/2520677439 learned_addr learned my addr 172.21.15.16:0/2520677439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fac0dc4e640 1 -- 172.21.15.16:0/2520677439 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac080d1f90 msgr2=0x7fac080cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fac0dc4e640 1 --2- 172.21.15.16:0/2520677439 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac080d1f90 0x7fac080cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fac0dc4e640 1 -- 172.21.15.16:0/2520677439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac0815dce0 msgr2=0x7fac08152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fac0dc4e640 1 --2- 172.21.15.16:0/2520677439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac0815dce0 0x7fac08152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fac0dc4e640 1 -- 172.21.15.16:0/2520677439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac08152640 con 0x7fac080d35b0 2024-04-25T10:15:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fac0dc4e640 1 --2- 172.21.15.16:0/2520677439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac080d35b0 0x7fac080cd3e0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fabfc02a180 tx=0x7fabfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fabfaffd640 1 -- 172.21.15.16:0/2520677439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc037040 con 0x7fac080d35b0 2024-04-25T10:15:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fabfaffd640 1 -- 172.21.15.16:0/2520677439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc042070 con 0x7fac080d35b0 2024-04-25T10:15:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fac106b3640 1 -- 172.21.15.16:0/2520677439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac080cbb40 con 0x7fac080d35b0 2024-04-25T10:15:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fabfaffd640 1 -- 172.21.15.16:0/2520677439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc03b870 con 0x7fac080d35b0 2024-04-25T10:15:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.015+0000 7fac106b3640 1 -- 172.21.15.16:0/2520677439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac080cc100 con 0x7fac080d35b0 2024-04-25T10:15:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.016+0000 7fac106b3640 1 -- 172.21.15.16:0/2520677439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac080b90e0 con 0x7fac080d35b0 2024-04-25T10:15:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.017+0000 7fabfaffd640 1 -- 172.21.15.16:0/2520677439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fabfc03ba10 con 0x7fac080d35b0 2024-04-25T10:15:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.017+0000 7fabfaffd640 1 --2- 172.21.15.16:0/2520677439 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fabec06b4d0 0x7fabec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.017+0000 7fabfaffd640 1 -- 172.21.15.16:0/2520677439 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fabfc0aa780 con 0x7fac080d35b0 2024-04-25T10:15:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.017+0000 7fac0e44f640 1 --2- 172.21.15.16:0/2520677439 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fabec06b4d0 0x7fabec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.018+0000 7fac0e44f640 1 --2- 172.21.15.16:0/2520677439 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fabec06b4d0 0x7fabec06d990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fac04006fd0 tx=0x7fac04008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.021+0000 7fabfaffd640 1 -- 172.21.15.16:0/2520677439 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fac080b90e0 con 0x7fac080d35b0 2024-04-25T10:15:10.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.208+0000 7fac106b3640 1 -- 172.21.15.16:0/2520677439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac080749b0 con 0x7fac080d35b0 2024-04-25T10:15:10.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.209+0000 7fabfaffd640 1 -- 172.21.15.16:0/2520677439 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+13288 (secure 0 0 0) 0x7fabfc0798e0 con 0x7fac080d35b0 2024-04-25T10:15:10.213 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:15:10.213 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":122,"fsid":"34bc5eca-3a21-42a6-a26f-07da0905e8bc","created":"2024-04-25T10:04:45.620943+0000","modified":"2024-04-25T10:15:07.103893+0000","last_up_change":"2024-04-25T10:04:57.388712+0000","last_in_change":"2024-04-25T10:04:48.672971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:04:57.825688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"40dcfc39-064c-4b17-bcf1-3f45d2e26b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6817","nonce":2167750256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6819","nonce":2167750256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6823","nonce":2167750256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2167750256},{"type":"v1","addr":"172.21.15.16:6821","nonce":2167750256}]},"public_addr":"172.21.15.16:6817/2167750256","cluster_addr":"172.21.15.16:6819/2167750256","heartbeat_back_addr":"172.21.15.16:6823/2167750256","heartbeat_front_addr":"172.21.15.16:6821/2167750256","state":["exists","up"]},{"osd":1,"uuid":"eeff61d1-2c4f-4879-ac01-b803de113532","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6809","nonce":1916704159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6811","nonce":1916704159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6815","nonce":1916704159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1916704159},{"type":"v1","addr":"172.21.15.16:6813","nonce":1916704159}]},"public_addr":"172.21.15.16:6809/1916704159","cluster_addr":"172.21.15.16:6811/1916704159","heartbeat_back_addr":"172.21.15.16:6815/1916704159","heartbeat_front_addr":"172.21.15.16:6813/1916704159","state":["exists","up"]},{"osd":2,"uuid":"49fff454-4f8d-46c8-b700-ca1380a49ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6801","nonce":2094356376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6803","nonce":2094356376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6807","nonce":2094356376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2094356376},{"type":"v1","addr":"172.21.15.16:6805","nonce":2094356376}]},"public_addr":"172.21.15.16:6801/2094356376","cluster_addr":"172.21.15.16:6803/2094356376","heartbeat_back_addr":"172.21.15.16:6807/2094356376","heartbeat_front_addr":"172.21.15.16:6805/2094356376","state":["exists","up"]},{"osd":3,"uuid":"6ed9721d-2101-49a5-9ac4-61696c382866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6825","nonce":256577261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6827","nonce":256577261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6831","nonce":256577261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":256577261},{"type":"v1","addr":"172.21.15.16:6829","nonce":256577261}]},"public_addr":"172.21.15.16:6825/256577261","cluster_addr":"172.21.15.16:6827/256577261","heartbeat_back_addr":"172.21.15.16:6831/256577261","heartbeat_front_addr":"172.21.15.16:6829/256577261","state":["exists","up"]},{"osd":4,"uuid":"46edf17f-ad1d-404a-b94a-9ed0cc9a7226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6825","nonce":3091472889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6827","nonce":3091472889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6831","nonce":3091472889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3091472889},{"type":"v1","addr":"172.21.15.186:6829","nonce":3091472889}]},"public_addr":"172.21.15.186:6825/3091472889","cluster_addr":"172.21.15.186:6827/3091472889","heartbeat_back_addr":"172.21.15.186:6831/3091472889","heartbeat_front_addr":"172.21.15.186:6829/3091472889","state":["exists","up"]},{"osd":5,"uuid":"f4824f88-e2a1-4c26-8198-1bc0cedaa1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6801","nonce":1827108421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6803","nonce":1827108421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6807","nonce":1827108421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1827108421},{"type":"v1","addr":"172.21.15.186:6805","nonce":1827108421}]},"public_addr":"172.21.15.186:6801/1827108421","cluster_addr":"172.21.15.186:6803/1827108421","heartbeat_back_addr":"172.21.15.186:6807/1827108421","heartbeat_front_addr":"172.21.15.186:6805/1827108421","state":["exists","up"]},{"osd":6,"uuid":"e75b8ba6-c9f1-49e6-bb20-fb2ac8aa938e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6817","nonce":1123185664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6819","nonce":1123185664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6823","nonce":1123185664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":1123185664},{"type":"v1","addr":"172.21.15.186:6821","nonce":1123185664}]},"public_addr":"172.21.15.186:6817/1123185664","cluster_addr":"172.21.15.186:6819/1123185664","heartbeat_back_addr":"172.21.15.186:6823/1123185664","heartbeat_front_addr":"172.21.15.186:6821/1123185664","state":["exists","up"]},{"osd":7,"uuid":"cb71fc85-240b-40b2-95cb-e39b8570f4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6809","nonce":1627307043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6811","nonce":1627307043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6815","nonce":1627307043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":1627307043},{"type":"v1","addr":"172.21.15.186:6813","nonce":1627307043}]},"public_addr":"172.21.15.186:6809/1627307043","cluster_addr":"172.21.15.186:6811/1627307043","heartbeat_back_addr":"172.21.15.186:6815/1627307043","heartbeat_front_addr":"172.21.15.186:6813/1627307043","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.866131+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.827898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.757606+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.859295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.850318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.710571+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.828035+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:04:53.834583+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/3763439204":"2024-04-26T10:15:04.079898+0000","172.21.15.16:6835/2697110279":"2024-04-26T10:14:50.050260+0000","172.21.15.16:6833/3763439204":"2024-04-26T10:15:04.079898+0000","172.21.15.16:6833/2697110279":"2024-04-26T10:14:50.050260+0000","172.21.15.16:6835/123396308":"2024-04-26T10:14:39.639508+0000","172.21.15.16:6833/123396308":"2024-04-26T10:14:39.639508+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:10.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.211+0000 7fac106b3640 1 -- 172.21.15.16:0/2520677439 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fabec06b4d0 msgr2=0x7fabec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.211+0000 7fac106b3640 1 --2- 172.21.15.16:0/2520677439 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fabec06b4d0 0x7fabec06d990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fac04006fd0 tx=0x7fac04008040 comp rx=0 tx=0).stop 2024-04-25T10:15:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.211+0000 7fac106b3640 1 -- 172.21.15.16:0/2520677439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac080d35b0 msgr2=0x7fac080cd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.211+0000 7fac106b3640 1 --2- 172.21.15.16:0/2520677439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac080d35b0 0x7fac080cd3e0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fabfc02a180 tx=0x7fabfc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.211+0000 7fac106b3640 1 -- 172.21.15.16:0/2520677439 shutdown_connections 2024-04-25T10:15:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.211+0000 7fac106b3640 1 --2- 172.21.15.16:0/2520677439 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fabec06b4d0 0x7fabec06d990 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.211+0000 7fac106b3640 1 --2- 172.21.15.16:0/2520677439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac0815dce0 0x7fac08152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.211+0000 7fac106b3640 1 --2- 172.21.15.16:0/2520677439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac080d35b0 0x7fac080cd3e0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.211+0000 7fac106b3640 1 --2- 172.21.15.16:0/2520677439 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac080d1f90 0x7fac080cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.211+0000 7fac106b3640 1 -- 172.21.15.16:0/2520677439 >> 172.21.15.16:0/2520677439 conn(0x7fac08072900 msgr2=0x7fac080b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.211+0000 7fac106b3640 1 -- 172.21.15.16:0/2520677439 shutdown_connections 2024-04-25T10:15:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:10.211+0000 7fac106b3640 1 -- 172.21.15.16:0/2520677439 wait complete. 2024-04-25T10:15:11.227 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T10:15:11.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:15:11.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.329+0000 7fa70361f640 1 Processor -- start 2024-04-25T10:15:11.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.330+0000 7fa70361f640 1 -- start start 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.330+0000 7fa70361f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa6fc071660 0x7fa6fc0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.330+0000 7fa70361f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc0cc9a0 con 0x7fa6fc145260 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.330+0000 7fa70361f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc0ccc80 con 0x7fa6fc071200 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.330+0000 7fa70361f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6fc0cea00 con 0x7fa6fc071660 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.330+0000 7fa701bbc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa6fc071660 0x7fa6fc0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.331+0000 7fa7013bb640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa6fc145260 0x7fa6fc0cbf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59444/0 (socket says 172.21.15.16:59444) 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.331+0000 7fa701bbc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa6fc071660 0x7fa6fc0cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56444/0 (socket says 172.21.15.16:56444) 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.331+0000 7fa7013bb640 1 -- 172.21.15.16:0/2223211818 learned_addr learned my addr 172.21.15.16:0/2223211818 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.331+0000 7fa701bbc640 1 -- 172.21.15.16:0/2223211818 >> v1:172.21.15.186:6789/0 conn(0x7fa6fc071200 legacy=0x7fa6fc0cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.331+0000 7fa701bbc640 1 -- 172.21.15.16:0/2223211818 >> v1:172.21.15.16:6789/0 conn(0x7fa6fc145260 legacy=0x7fa6fc0cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.331+0000 7fa701bbc640 1 -- 172.21.15.16:0/2223211818 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc0ccf60 con 0x7fa6fc071660 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.331+0000 7fa6f3fff640 1 -- 172.21.15.16:0/2223211818 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3928619384 0 0) 0x7fa6fc0cc9a0 con 0x7fa6fc145260 2024-04-25T10:15:11.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.332+0000 7fa701bbc640 1 --2- 172.21.15.16:0/2223211818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6fc071660 0x7fa6fc0cf5a0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009d90 tx=0x7fa6f402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4d5e4ddba28d93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.332+0000 7fa6f3fff640 1 -- 172.21.15.16:0/2223211818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4007740 con 0x7fa6fc071660 2024-04-25T10:15:11.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.332+0000 7fa6f3fff640 1 -- 172.21.15.16:0/2223211818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f40078e0 con 0x7fa6fc071660 2024-04-25T10:15:11.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.332+0000 7fa6f3fff640 1 -- 172.21.15.16:0/2223211818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4007cb0 con 0x7fa6fc071660 2024-04-25T10:15:11.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.332+0000 7fa70361f640 1 -- 172.21.15.16:0/2223211818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6fc071660 msgr2=0x7fa6fc0cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.332+0000 7fa70361f640 1 --2- 172.21.15.16:0/2223211818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6fc071660 0x7fa6fc0cf5a0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009d90 tx=0x7fa6f402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.332+0000 7fa70361f640 1 -- 172.21.15.16:0/2223211818 shutdown_connections 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.332+0000 7fa70361f640 1 --2- 172.21.15.16:0/2223211818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6fc071660 0x7fa6fc0cf5a0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.332+0000 7fa70361f640 1 -- 172.21.15.16:0/2223211818 >> 172.21.15.16:0/2223211818 conn(0x7fa6fc06aba0 msgr2=0x7fa6fc06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.332+0000 7fa70361f640 1 -- 172.21.15.16:0/2223211818 shutdown_connections 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.332+0000 7fa70361f640 1 -- 172.21.15.16:0/2223211818 wait complete. 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.334+0000 7fa70361f640 1 Processor -- start 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.334+0000 7fa70361f640 1 -- start start 2024-04-25T10:15:11.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.334+0000 7fa70361f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6fc071200 0x7fa6fc144520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.334+0000 7fa70361f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6fc071660 0x7fa6fc15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.334+0000 7fa70361f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6fc145260 0x7fa6fc15e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.334+0000 7fa70361f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc071a90 con 0x7fa6fc145260 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.334+0000 7fa70361f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0cc660 con 0x7fa6fc071200 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.334+0000 7fa70361f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc145640 con 0x7fa6fc071660 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.335+0000 7fa701bbc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6fc145260 0x7fa6fc15e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.335+0000 7fa701bbc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6fc145260 0x7fa6fc15e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56446/0 (socket says 172.21.15.16:56446) 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.335+0000 7fa701bbc640 1 -- 172.21.15.16:0/3824409841 learned_addr learned my addr 172.21.15.16:0/3824409841 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.335+0000 7fa7013bb640 1 --2- 172.21.15.16:0/3824409841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6fc071200 0x7fa6fc144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.335+0000 7fa700bba640 1 --2- 172.21.15.16:0/3824409841 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6fc071660 0x7fa6fc15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.335+0000 7fa701bbc640 1 -- 172.21.15.16:0/3824409841 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6fc071660 msgr2=0x7fa6fc15e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.335+0000 7fa701bbc640 1 --2- 172.21.15.16:0/3824409841 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6fc071660 0x7fa6fc15e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.335+0000 7fa701bbc640 1 -- 172.21.15.16:0/3824409841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6fc071200 msgr2=0x7fa6fc144520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.335+0000 7fa701bbc640 1 --2- 172.21.15.16:0/3824409841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6fc071200 0x7fa6fc144520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.335+0000 7fa701bbc640 1 -- 172.21.15.16:0/3824409841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc15f120 con 0x7fa6fc145260 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.335+0000 7fa701bbc640 1 --2- 172.21.15.16:0/3824409841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6fc145260 0x7fa6fc15e830 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009d90 tx=0x7fa6f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.335+0000 7fa6f1ffb640 1 -- 172.21.15.16:0/3824409841 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4040070 con 0x7fa6fc145260 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.335+0000 7fa6f1ffb640 1 -- 172.21.15.16:0/3824409841 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f4031c90 con 0x7fa6fc145260 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.335+0000 7fa70361f640 1 -- 172.21.15.16:0/3824409841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6fc0b4800 con 0x7fa6fc145260 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.335+0000 7fa6f1ffb640 1 -- 172.21.15.16:0/3824409841 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f403c640 con 0x7fa6fc145260 2024-04-25T10:15:11.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.336+0000 7fa70361f640 1 -- 172.21.15.16:0/3824409841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6fc15ae00 con 0x7fa6fc145260 2024-04-25T10:15:11.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.336+0000 7fa70361f640 1 -- 172.21.15.16:0/3824409841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa6fc0750b0 con 0x7fa6fc145260 2024-04-25T10:15:11.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.337+0000 7fa6f1ffb640 1 -- 172.21.15.16:0/3824409841 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6f4039040 con 0x7fa6fc145260 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.337+0000 7fa6f1ffb640 1 --2- 172.21.15.16:0/3824409841 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa6e006b4d0 0x7fa6e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.338+0000 7fa6f1ffb640 1 -- 172.21.15.16:0/3824409841 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fa6f40ab670 con 0x7fa6fc145260 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.338+0000 7fa7013bb640 1 --2- 172.21.15.16:0/3824409841 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa6e006b4d0 0x7fa6e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.338+0000 7fa6f1ffb640 1 --2- 172.21.15.16:0/3824409841 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x7fa6e00714d0 0x7fa6e00738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.338+0000 7fa6f1ffb640 1 -- 172.21.15.16:0/3824409841 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa6e0073fa0 con 0x7fa6e00714d0 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.338+0000 7fa6f1ffb640 1 -- 172.21.15.16:0/3824409841 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6f40abb00 con 0x7fa6fc145260 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.338+0000 7fa700bba640 1 --2- 172.21.15.16:0/3824409841 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x7fa6e00714d0 0x7fa6e00738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.338+0000 7fa7013bb640 1 --2- 172.21.15.16:0/3824409841 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa6e006b4d0 0x7fa6e006d990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec007f70 tx=0x7fa6ec007f00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.338+0000 7fa700bba640 1 --2- 172.21.15.16:0/3824409841 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x7fa6e00714d0 0x7fa6e00738f0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.339+0000 7fa6f1ffb640 1 -- 172.21.15.16:0/3824409841 <== osd.0 v2:172.21.15.16:6816/2167750256 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa6e0073fa0 con 0x7fa6e00714d0 2024-04-25T10:15:11.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.352+0000 7fa70361f640 1 -- 172.21.15.16:0/3824409841 --> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa6fc142500 con 0x7fa6e00714d0 2024-04-25T10:15:11.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.352+0000 7fa6f1ffb640 1 -- 172.21.15.16:0/3824409841 <== osd.0 v2:172.21.15.16:6816/2167750256 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa6fc142500 con 0x7fa6e00714d0 2024-04-25T10:15:11.356 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:15:11.356 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:15:11.356 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:15:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.352+0000 7fa70361f640 1 -- 172.21.15.16:0/3824409841 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x7fa6e00714d0 msgr2=0x7fa6e00738f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.352+0000 7fa70361f640 1 --2- 172.21.15.16:0/3824409841 >> [v2:172.21.15.16:6816/2167750256,v1:172.21.15.16:6817/2167750256] conn(0x7fa6e00714d0 0x7fa6e00738f0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.353+0000 7fa70361f640 1 -- 172.21.15.16:0/3824409841 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa6e006b4d0 msgr2=0x7fa6e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.353+0000 7fa70361f640 1 --2- 172.21.15.16:0/3824409841 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa6e006b4d0 0x7fa6e006d990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec007f70 tx=0x7fa6ec007f00 comp rx=0 tx=0).stop 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.353+0000 7fa70361f640 1 -- 172.21.15.16:0/3824409841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6fc145260 msgr2=0x7fa6fc15e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.353+0000 7fa70361f640 1 --2- 172.21.15.16:0/3824409841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6fc145260 0x7fa6fc15e830 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009d90 tx=0x7fa6f4004a40 comp rx=0 tx=0).stop 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.353+0000 7fa701bbc640 1 -- 172.21.15.16:0/3824409841 reap_dead start 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.353+0000 7fa70361f640 1 -- 172.21.15.16:0/3824409841 shutdown_connections 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.353+0000 7fa70361f640 1 -- 172.21.15.16:0/3824409841 >> 172.21.15.16:0/3824409841 conn(0x7fa6fc06aba0 msgr2=0x7fa6fc141710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.353+0000 7fa70361f640 1 -- 172.21.15.16:0/3824409841 shutdown_connections 2024-04-25T10:15:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.353+0000 7fa70361f640 1 -- 172.21.15.16:0/3824409841 wait complete. 2024-04-25T10:15:11.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T10:15:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.466+0000 7f83c576a640 1 Processor -- start 2024-04-25T10:15:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.466+0000 7f83c576a640 1 -- start start 2024-04-25T10:15:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.467+0000 7f83c576a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83c00aedc0 0x7f83c015ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.467+0000 7f83c576a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83c006d260 0x7f83c00b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.467+0000 7f83c576a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f83c00b4730 0x7f83c00b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.467+0000 7f83c576a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c00b4ba0 con 0x7f83c006d260 2024-04-25T10:15:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.467+0000 7f83c576a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c015c050 con 0x7f83c00b4730 2024-04-25T10:15:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.467+0000 7f83c576a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c00af1a0 con 0x7f83c00aedc0 2024-04-25T10:15:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.467+0000 7f83b7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83c006d260 0x7f83c00b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.467+0000 7f83b7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83c006d260 0x7f83c00b41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56456/0 (socket says 172.21.15.16:56456) 2024-04-25T10:15:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.467+0000 7f83b7fff640 1 -- 172.21.15.16:0/3186925845 learned_addr learned my addr 172.21.15.16:0/3186925845 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.467+0000 7f83beffd640 1 --2- 172.21.15.16:0/3186925845 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83c00aedc0 0x7f83c015ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.467+0000 7f83bf7fe640 1 --2- 172.21.15.16:0/3186925845 >> v2:172.21.15.186:3300/0 conn(0x7f83c00b4730 0x7f83c00b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.467+0000 7f83b7fff640 1 -- 172.21.15.16:0/3186925845 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83c00aedc0 msgr2=0x7f83c015ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.467+0000 7f83b7fff640 1 --2- 172.21.15.16:0/3186925845 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83c00aedc0 0x7f83c015ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.467+0000 7f83b7fff640 1 -- 172.21.15.16:0/3186925845 >> v2:172.21.15.186:3300/0 conn(0x7f83c00b4730 msgr2=0x7f83c00b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.467+0000 7f83b7fff640 1 --2- 172.21.15.16:0/3186925845 >> v2:172.21.15.186:3300/0 conn(0x7f83c00b4730 0x7f83c00b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.467+0000 7f83b7fff640 1 -- 172.21.15.16:0/3186925845 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c006da40 con 0x7f83c006d260 2024-04-25T10:15:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.468+0000 7f83b7fff640 1 --2- 172.21.15.16:0/3186925845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83c006d260 0x7f83c00b41f0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f83c015bfe0 tx=0x7f83a802c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484b11b64fc4915c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.468+0000 7f83be7fc640 1 -- 172.21.15.16:0/3186925845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8039070 con 0x7f83c006d260 2024-04-25T10:15:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.468+0000 7f83be7fc640 1 -- 172.21.15.16:0/3186925845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a8034640 con 0x7f83c006d260 2024-04-25T10:15:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.468+0000 7f83be7fc640 1 -- 172.21.15.16:0/3186925845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8034a10 con 0x7f83c006d260 2024-04-25T10:15:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.469+0000 7f83c576a640 1 -- 172.21.15.16:0/3186925845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83c006d260 msgr2=0x7f83c00b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.469+0000 7f83c576a640 1 --2- 172.21.15.16:0/3186925845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83c006d260 0x7f83c00b41f0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f83c015bfe0 tx=0x7f83a802c390 comp rx=0 tx=0).stop 2024-04-25T10:15:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.469+0000 7f83c576a640 1 -- 172.21.15.16:0/3186925845 shutdown_connections 2024-04-25T10:15:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.469+0000 7f83c576a640 1 --2- 172.21.15.16:0/3186925845 >> v2:172.21.15.186:3300/0 conn(0x7f83c00b4730 0x7f83c00b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.469+0000 7f83c576a640 1 --2- 172.21.15.16:0/3186925845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83c006d260 0x7f83c00b41f0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.469+0000 7f83c576a640 1 --2- 172.21.15.16:0/3186925845 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83c00aedc0 0x7f83c015ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.469+0000 7f83c576a640 1 -- 172.21.15.16:0/3186925845 >> 172.21.15.16:0/3186925845 conn(0x7f83c006aba0 msgr2=0x7f83c006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.469+0000 7f83c576a640 1 -- 172.21.15.16:0/3186925845 shutdown_connections 2024-04-25T10:15:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.469+0000 7f83c576a640 1 -- 172.21.15.16:0/3186925845 wait complete. 2024-04-25T10:15:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.470+0000 7f83c576a640 1 Processor -- start 2024-04-25T10:15:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.470+0000 7f83c576a640 1 -- start start 2024-04-25T10:15:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.471+0000 7f83c576a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c006d260 0x7f83c0159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.471+0000 7f83c576a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83c00aedc0 0x7f83c0159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.471+0000 7f83c576a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83c00b4730 0x7f83c01f3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.471+0000 7f83c576a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c015c2a0 con 0x7f83c00aedc0 2024-04-25T10:15:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.471+0000 7f83c576a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c006e000 con 0x7f83c006d260 2024-04-25T10:15:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.471+0000 7f83c576a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c006d6c0 con 0x7f83c00b4730 2024-04-25T10:15:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.471+0000 7f83b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83c00aedc0 0x7f83c0159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.471+0000 7f83b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83c00aedc0 0x7f83c0159580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56466/0 (socket says 172.21.15.16:56466) 2024-04-25T10:15:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.471+0000 7f83b7fff640 1 -- 172.21.15.16:0/1618810129 learned_addr learned my addr 172.21.15.16:0/1618810129 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.471+0000 7f83bf7fe640 1 --2- 172.21.15.16:0/1618810129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83c00b4730 0x7f83c01f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.471+0000 7f83beffd640 1 --2- 172.21.15.16:0/1618810129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c006d260 0x7f83c0159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.471+0000 7f83b7fff640 1 -- 172.21.15.16:0/1618810129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83c00b4730 msgr2=0x7f83c01f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.471+0000 7f83b7fff640 1 --2- 172.21.15.16:0/1618810129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83c00b4730 0x7f83c01f3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.471+0000 7f83b7fff640 1 -- 172.21.15.16:0/1618810129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c006d260 msgr2=0x7f83c0159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.471+0000 7f83b7fff640 1 --2- 172.21.15.16:0/1618810129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c006d260 0x7f83c0159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.471+0000 7f83b7fff640 1 -- 172.21.15.16:0/1618810129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c01f6f50 con 0x7f83c00aedc0 2024-04-25T10:15:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.471+0000 7f83b7fff640 1 --2- 172.21.15.16:0/1618810129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83c00aedc0 0x7f83c0159580 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f83a8000c00 tx=0x7f83a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.472+0000 7f83b77fe640 1 -- 172.21.15.16:0/1618810129 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8044070 con 0x7f83c00aedc0 2024-04-25T10:15:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.472+0000 7f83b77fe640 1 -- 172.21.15.16:0/1618810129 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a8032980 con 0x7f83c00aedc0 2024-04-25T10:15:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.472+0000 7f83c576a640 1 -- 172.21.15.16:0/1618810129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c01f7230 con 0x7f83c00aedc0 2024-04-25T10:15:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.472+0000 7f83b77fe640 1 -- 172.21.15.16:0/1618810129 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8039070 con 0x7f83c00aedc0 2024-04-25T10:15:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.472+0000 7f83c576a640 1 -- 172.21.15.16:0/1618810129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c01f77f0 con 0x7f83c00aedc0 2024-04-25T10:15:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.473+0000 7f83c576a640 1 -- 172.21.15.16:0/1618810129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c01f67b0 con 0x7f83c00aedc0 2024-04-25T10:15:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.473+0000 7f83b77fe640 1 -- 172.21.15.16:0/1618810129 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83a8047070 con 0x7f83c00aedc0 2024-04-25T10:15:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.474+0000 7f83b77fe640 1 --2- 172.21.15.16:0/1618810129 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f83a006b4d0 0x7f83a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.474+0000 7f83b77fe640 1 -- 172.21.15.16:0/1618810129 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f83a80aa000 con 0x7f83c00aedc0 2024-04-25T10:15:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.474+0000 7f83beffd640 1 --2- 172.21.15.16:0/1618810129 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f83a006b4d0 0x7f83a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.474+0000 7f83beffd640 1 --2- 172.21.15.16:0/1618810129 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f83a006b4d0 0x7f83a006d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009ac0 tx=0x7f83b0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.478+0000 7f83b77fe640 1 -- 172.21.15.16:0/1618810129 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83c01f67b0 con 0x7f83c00aedc0 2024-04-25T10:15:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.660+0000 7f83c576a640 1 -- 172.21.15.16:0/1618810129 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f83c00750b0 con 0x7f83a006b4d0 2024-04-25T10:15:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.660+0000 7f83b77fe640 1 -- 172.21.15.16:0/1618810129 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f83c00750b0 con 0x7f83a006b4d0 2024-04-25T10:15:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T10:15:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.662+0000 7f83c576a640 1 -- 172.21.15.16:0/1618810129 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f83a006b4d0 msgr2=0x7f83a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.662+0000 7f83c576a640 1 --2- 172.21.15.16:0/1618810129 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f83a006b4d0 0x7f83a006d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009ac0 tx=0x7f83b0009190 comp rx=0 tx=0).stop 2024-04-25T10:15:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.662+0000 7f83c576a640 1 -- 172.21.15.16:0/1618810129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83c00aedc0 msgr2=0x7f83c0159580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.662+0000 7f83c576a640 1 --2- 172.21.15.16:0/1618810129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83c00aedc0 0x7f83c0159580 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f83a8000c00 tx=0x7f83a80046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.662+0000 7f83c576a640 1 -- 172.21.15.16:0/1618810129 shutdown_connections 2024-04-25T10:15:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.662+0000 7f83c576a640 1 --2- 172.21.15.16:0/1618810129 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f83a006b4d0 0x7f83a006d990 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.662+0000 7f83c576a640 1 --2- 172.21.15.16:0/1618810129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83c00b4730 0x7f83c01f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.662+0000 7f83c576a640 1 --2- 172.21.15.16:0/1618810129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83c00aedc0 0x7f83c0159580 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.662+0000 7f83c576a640 1 --2- 172.21.15.16:0/1618810129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c006d260 0x7f83c0159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.662+0000 7f83c576a640 1 -- 172.21.15.16:0/1618810129 >> 172.21.15.16:0/1618810129 conn(0x7f83c006aba0 msgr2=0x7f83c00cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.662+0000 7f83c576a640 1 -- 172.21.15.16:0/1618810129 shutdown_connections 2024-04-25T10:15:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.662+0000 7f83c576a640 1 -- 172.21.15.16:0/1618810129 wait complete. 2024-04-25T10:15:11.677 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T10:15:11.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:15:11.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.778+0000 7f9a11a9e640 1 Processor -- start 2024-04-25T10:15:11.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.778+0000 7f9a11a9e640 1 -- start start 2024-04-25T10:15:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.779+0000 7f9a11a9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a0c0cef40 0x7f9a0c0b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.779+0000 7f9a11a9e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a0c1618d0 0x7f9a0c0d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.779+0000 7f9a11a9e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0c161d80 con 0x7f9a0c1610b0 2024-04-25T10:15:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.779+0000 7f9a11a9e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c0d1ea0 con 0x7f9a0c1618d0 2024-04-25T10:15:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.779+0000 7f9a11a9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c162040 con 0x7f9a0c0cef40 2024-04-25T10:15:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.779+0000 7f9a0bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a0c1618d0 0x7f9a0c0d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.779+0000 7f9a0bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a0c1618d0 0x7f9a0c0d2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56474/0 (socket says 172.21.15.16:56474) 2024-04-25T10:15:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.779+0000 7f9a0bfff640 1 -- 172.21.15.16:0/232708898 learned_addr learned my addr 172.21.15.16:0/232708898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.780+0000 7f9a0affd640 1 --2- 172.21.15.16:0/232708898 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a0c0cef40 0x7f9a0c0b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.780+0000 7f9a0bfff640 1 -- 172.21.15.16:0/232708898 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a0c0cef40 msgr2=0x7f9a0c0b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.780+0000 7f9a0bfff640 1 --2- 172.21.15.16:0/232708898 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a0c0cef40 0x7f9a0c0b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.780+0000 7f9a0bfff640 1 -- 172.21.15.16:0/232708898 >> v1:172.21.15.186:6789/0 conn(0x7f9a0c1610b0 legacy=0x7f9a0c0ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.780+0000 7f9a0bfff640 1 -- 172.21.15.16:0/232708898 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c162320 con 0x7f9a0c1618d0 2024-04-25T10:15:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.780+0000 7f9a0bfff640 1 --2- 172.21.15.16:0/232708898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c1618d0 0x7f9a0c0d2610 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009e50 tx=0x7f9a0402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6b10890a6aed24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.781+0000 7f9a0a7fc640 1 -- 172.21.15.16:0/232708898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04036070 con 0x7f9a0c1618d0 2024-04-25T10:15:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.781+0000 7f9a0a7fc640 1 -- 172.21.15.16:0/232708898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a04031600 con 0x7f9a0c1618d0 2024-04-25T10:15:11.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.781+0000 7f9a0a7fc640 1 -- 172.21.15.16:0/232708898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a040319d0 con 0x7f9a0c1618d0 2024-04-25T10:15:11.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.781+0000 7f9a11a9e640 1 -- 172.21.15.16:0/232708898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c1618d0 msgr2=0x7f9a0c0d2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.781+0000 7f9a11a9e640 1 --2- 172.21.15.16:0/232708898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c1618d0 0x7f9a0c0d2610 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009e50 tx=0x7f9a0402a680 comp rx=0 tx=0).stop 2024-04-25T10:15:11.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.781+0000 7f9a11a9e640 1 -- 172.21.15.16:0/232708898 shutdown_connections 2024-04-25T10:15:11.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.781+0000 7f9a11a9e640 1 --2- 172.21.15.16:0/232708898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c1618d0 0x7f9a0c0d2610 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.781+0000 7f9a11a9e640 1 --2- 172.21.15.16:0/232708898 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a0c0cef40 0x7f9a0c0b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.781+0000 7f9a11a9e640 1 -- 172.21.15.16:0/232708898 >> 172.21.15.16:0/232708898 conn(0x7f9a0c0b5570 msgr2=0x7f9a0c0b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:11.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.781+0000 7f9a11a9e640 1 -- 172.21.15.16:0/232708898 shutdown_connections 2024-04-25T10:15:11.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.781+0000 7f9a11a9e640 1 -- 172.21.15.16:0/232708898 wait complete. 2024-04-25T10:15:11.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.782+0000 7f9a11a9e640 1 Processor -- start 2024-04-25T10:15:11.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.783+0000 7f9a11a9e640 1 -- start start 2024-04-25T10:15:11.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.783+0000 7f9a11a9e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c0cef40 0x7f9a0c152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.783+0000 7f9a11a9e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a0c1610b0 0x7f9a0c156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.783+0000 7f9a11a9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c1618d0 0x7f9a0c156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.783+0000 7f9a11a9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0b6c00 con 0x7f9a0c1618d0 2024-04-25T10:15:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.783+0000 7f9a11a9e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0d20f0 con 0x7f9a0c0cef40 2024-04-25T10:15:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.783+0000 7f9a11a9e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c161490 con 0x7f9a0c1610b0 2024-04-25T10:15:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.783+0000 7f9a0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c1618d0 0x7f9a0c156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.783+0000 7f9a0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c1618d0 0x7f9a0c156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56486/0 (socket says 172.21.15.16:56486) 2024-04-25T10:15:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.783+0000 7f9a0bfff640 1 -- 172.21.15.16:0/204732076 learned_addr learned my addr 172.21.15.16:0/204732076 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.784+0000 7f9a0b7fe640 1 --2- 172.21.15.16:0/204732076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c0cef40 0x7f9a0c152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.784+0000 7f9a0affd640 1 --2- 172.21.15.16:0/204732076 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a0c1610b0 0x7f9a0c156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.784+0000 7f9a0bfff640 1 -- 172.21.15.16:0/204732076 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a0c1610b0 msgr2=0x7f9a0c156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.784+0000 7f9a0bfff640 1 --2- 172.21.15.16:0/204732076 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a0c1610b0 0x7f9a0c156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.784+0000 7f9a0bfff640 1 -- 172.21.15.16:0/204732076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c0cef40 msgr2=0x7f9a0c152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.784+0000 7f9a0bfff640 1 --2- 172.21.15.16:0/204732076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c0cef40 0x7f9a0c152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.784+0000 7f9a0bfff640 1 -- 172.21.15.16:0/204732076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c1f7860 con 0x7f9a0c1618d0 2024-04-25T10:15:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.784+0000 7f9a0bfff640 1 --2- 172.21.15.16:0/204732076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c1618d0 0x7f9a0c156680 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f9a04034040 tx=0x7f9a04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.784+0000 7f99f3fff640 1 -- 172.21.15.16:0/204732076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04036050 con 0x7f9a0c1618d0 2024-04-25T10:15:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.784+0000 7f99f3fff640 1 -- 172.21.15.16:0/204732076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a04042070 con 0x7f9a0c1618d0 2024-04-25T10:15:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.784+0000 7f99f3fff640 1 -- 172.21.15.16:0/204732076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04009510 con 0x7f9a0c1618d0 2024-04-25T10:15:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.784+0000 7f9a11a9e640 1 -- 172.21.15.16:0/204732076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c1f7b40 con 0x7f9a0c1618d0 2024-04-25T10:15:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.784+0000 7f9a11a9e640 1 -- 172.21.15.16:0/204732076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c1f8100 con 0x7f9a0c1618d0 2024-04-25T10:15:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.785+0000 7f9a11a9e640 1 -- 172.21.15.16:0/204732076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9a0c0750b0 con 0x7f9a0c1618d0 2024-04-25T10:15:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.785+0000 7f99f3fff640 1 -- 172.21.15.16:0/204732076 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a04007bd0 con 0x7f9a0c1618d0 2024-04-25T10:15:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.786+0000 7f99f3fff640 1 --2- 172.21.15.16:0/204732076 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f99f806b4d0 0x7f99f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.786+0000 7f99f3fff640 1 -- 172.21.15.16:0/204732076 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f9a040aae60 con 0x7f9a0c1618d0 2024-04-25T10:15:11.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.786+0000 7f99f3fff640 1 --2- 172.21.15.16:0/204732076 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x7f99f80714d0 0x7f99f80738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.786+0000 7f99f3fff640 1 -- 172.21.15.16:0/204732076 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a0404ae80 con 0x7f99f80714d0 2024-04-25T10:15:11.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.786+0000 7f99f3fff640 1 -- 172.21.15.16:0/204732076 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a040ad560 con 0x7f9a0c1618d0 2024-04-25T10:15:11.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.786+0000 7f9a0b7fe640 1 --2- 172.21.15.16:0/204732076 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f99f806b4d0 0x7f99f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.786+0000 7f9a0affd640 1 --2- 172.21.15.16:0/204732076 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x7f99f80714d0 0x7f99f80738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.787+0000 7f9a0affd640 1 --2- 172.21.15.16:0/204732076 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x7f99f80714d0 0x7f99f80738f0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.787+0000 7f9a0b7fe640 1 --2- 172.21.15.16:0/204732076 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f99f806b4d0 0x7f99f806d990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f99fc001640 tx=0x7f99fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.787+0000 7f99f3fff640 1 -- 172.21.15.16:0/204732076 <== osd.1 v2:172.21.15.16:6808/1916704159 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9a0404ae80 con 0x7f99f80714d0 2024-04-25T10:15:11.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.800+0000 7f9a11a9e640 1 -- 172.21.15.16:0/204732076 --> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9a0c06c470 con 0x7f99f80714d0 2024-04-25T10:15:11.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.801+0000 7f99f3fff640 1 -- 172.21.15.16:0/204732076 <== osd.1 v2:172.21.15.16:6808/1916704159 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9a0c06c470 con 0x7f99f80714d0 2024-04-25T10:15:11.805 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:15:11.805 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:15:11.805 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:15:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.801+0000 7f9a11a9e640 1 -- 172.21.15.16:0/204732076 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x7f99f80714d0 msgr2=0x7f99f80738f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.801+0000 7f9a11a9e640 1 --2- 172.21.15.16:0/204732076 >> [v2:172.21.15.16:6808/1916704159,v1:172.21.15.16:6809/1916704159] conn(0x7f99f80714d0 0x7f99f80738f0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.801+0000 7f9a11a9e640 1 -- 172.21.15.16:0/204732076 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f99f806b4d0 msgr2=0x7f99f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.801+0000 7f9a11a9e640 1 --2- 172.21.15.16:0/204732076 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f99f806b4d0 0x7f99f806d990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f99fc001640 tx=0x7f99fc009040 comp rx=0 tx=0).stop 2024-04-25T10:15:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.801+0000 7f9a11a9e640 1 -- 172.21.15.16:0/204732076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c1618d0 msgr2=0x7f9a0c156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.801+0000 7f9a11a9e640 1 --2- 172.21.15.16:0/204732076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c1618d0 0x7f9a0c156680 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f9a04034040 tx=0x7f9a04004a40 comp rx=0 tx=0).stop 2024-04-25T10:15:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.801+0000 7f9a0bfff640 1 -- 172.21.15.16:0/204732076 reap_dead start 2024-04-25T10:15:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.801+0000 7f9a11a9e640 1 -- 172.21.15.16:0/204732076 shutdown_connections 2024-04-25T10:15:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.801+0000 7f9a11a9e640 1 -- 172.21.15.16:0/204732076 >> 172.21.15.16:0/204732076 conn(0x7f9a0c0b5570 msgr2=0x7f9a0c0b8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.801+0000 7f9a11a9e640 1 -- 172.21.15.16:0/204732076 shutdown_connections 2024-04-25T10:15:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.801+0000 7f9a11a9e640 1 -- 172.21.15.16:0/204732076 wait complete. 2024-04-25T10:15:11.815 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T10:15:11.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.913+0000 7f4f0eb28640 1 Processor -- start 2024-04-25T10:15:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.913+0000 7f4f0eb28640 1 -- start start 2024-04-25T10:15:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.914+0000 7f4f0eb28640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4f080acdf0 0x7f4f080cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.914+0000 7f4f0eb28640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f0806c580 con 0x7f4f081611e0 2024-04-25T10:15:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.914+0000 7f4f0eb28640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f080ce470 con 0x7f4f080ce070 2024-04-25T10:15:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.914+0000 7f4f0eb28640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f0806dde0 con 0x7f4f080acdf0 2024-04-25T10:15:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.914+0000 7f4f0c8c4640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4f081611e0 0x7f4f0806ba90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59460/0 (socket says 172.21.15.16:59460) 2024-04-25T10:15:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.914+0000 7f4f0c8c4640 1 -- 172.21.15.16:0/1651185922 learned_addr learned my addr 172.21.15.16:0/1651185922 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.914+0000 7f4f07fff640 1 --2- 172.21.15.16:0/1651185922 >> v2:172.21.15.186:3300/0 conn(0x7f4f080acdf0 0x7f4f080cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.914+0000 7f4f077fe640 1 -- 172.21.15.16:0/1651185922 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370081799 0 0) 0x7f4f0806c580 con 0x7f4f081611e0 2024-04-25T10:15:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.914+0000 7f4f077fe640 1 -- 172.21.15.16:0/1651185922 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ef8003520 con 0x7f4f081611e0 2024-04-25T10:15:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.914+0000 7f4f07fff640 1 -- 172.21.15.16:0/1651185922 >> v1:172.21.15.186:6789/0 conn(0x7f4f080ce070 legacy=0x7f4f0806eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.914+0000 7f4f07fff640 1 -- 172.21.15.16:0/1651185922 >> v1:172.21.15.16:6789/0 conn(0x7f4f081611e0 legacy=0x7f4f0806ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.914+0000 7f4f077fe640 1 -- 172.21.15.16:0/1651185922 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638941052 0 0) 0x7f4f080ce470 con 0x7f4f080ce070 2024-04-25T10:15:12.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.914+0000 7f4f07fff640 1 -- 172.21.15.16:0/1651185922 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0806e0c0 con 0x7f4f080acdf0 2024-04-25T10:15:12.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.915+0000 7f4f07fff640 1 --2- 172.21.15.16:0/1651185922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f080acdf0 0x7f4f080cdb30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40097a0 tx=0x7f4ef402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eabf49ea3e21de91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.915+0000 7f4f077fe640 1 -- 172.21.15.16:0/1651185922 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4037070 con 0x7f4f080acdf0 2024-04-25T10:15:12.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.915+0000 7f4f077fe640 1 -- 172.21.15.16:0/1651185922 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef4032440 con 0x7f4f080acdf0 2024-04-25T10:15:12.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.915+0000 7f4f077fe640 1 -- 172.21.15.16:0/1651185922 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef402ae60 con 0x7f4f080acdf0 2024-04-25T10:15:12.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.916+0000 7f4f0eb28640 1 -- 172.21.15.16:0/1651185922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f080acdf0 msgr2=0x7f4f080cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.916+0000 7f4f0eb28640 1 --2- 172.21.15.16:0/1651185922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f080acdf0 0x7f4f080cdb30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40097a0 tx=0x7f4ef402a110 comp rx=0 tx=0).stop 2024-04-25T10:15:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.916+0000 7f4f0eb28640 1 -- 172.21.15.16:0/1651185922 shutdown_connections 2024-04-25T10:15:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.916+0000 7f4f0eb28640 1 --2- 172.21.15.16:0/1651185922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f080acdf0 0x7f4f080cdb30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.916+0000 7f4f0eb28640 1 -- 172.21.15.16:0/1651185922 >> 172.21.15.16:0/1651185922 conn(0x7f4f080cc9e0 msgr2=0x7f4f080ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.916+0000 7f4f0eb28640 1 -- 172.21.15.16:0/1651185922 shutdown_connections 2024-04-25T10:15:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.916+0000 7f4f0eb28640 1 -- 172.21.15.16:0/1651185922 wait complete. 2024-04-25T10:15:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.917+0000 7f4f0eb28640 1 Processor -- start 2024-04-25T10:15:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.917+0000 7f4f0eb28640 1 -- start start 2024-04-25T10:15:12.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.918+0000 7f4f0eb28640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f080acdf0 0x7f4f0815ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.918+0000 7f4f0eb28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f080ce070 0x7f4f080c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.918+0000 7f4f0eb28640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f081611e0 0x7f4f080c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.918+0000 7f4f0eb28640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0806ae30 con 0x7f4f080ce070 2024-04-25T10:15:12.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.918+0000 7f4f0eb28640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0806e680 con 0x7f4f080acdf0 2024-04-25T10:15:12.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.918+0000 7f4f0eb28640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0806c1a0 con 0x7f4f081611e0 2024-04-25T10:15:12.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.918+0000 7f4f07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f080ce070 0x7f4f080c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.918+0000 7f4f07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f080ce070 0x7f4f080c5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56494/0 (socket says 172.21.15.16:56494) 2024-04-25T10:15:12.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.918+0000 7f4f07fff640 1 -- 172.21.15.16:0/2331098566 learned_addr learned my addr 172.21.15.16:0/2331098566 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:12.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.918+0000 7f4f0c8c4640 1 --2- 172.21.15.16:0/2331098566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f080acdf0 0x7f4f0815ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.918+0000 7f4f0d0c5640 1 --2- 172.21.15.16:0/2331098566 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f081611e0 0x7f4f080c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.918+0000 7f4f07fff640 1 -- 172.21.15.16:0/2331098566 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f081611e0 msgr2=0x7f4f080c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.918+0000 7f4f07fff640 1 --2- 172.21.15.16:0/2331098566 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f081611e0 0x7f4f080c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.918+0000 7f4f07fff640 1 -- 172.21.15.16:0/2331098566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f080acdf0 msgr2=0x7f4f0815ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.918+0000 7f4f07fff640 1 --2- 172.21.15.16:0/2331098566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f080acdf0 0x7f4f0815ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.918+0000 7f4f07fff640 1 -- 172.21.15.16:0/2331098566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f080cb610 con 0x7f4f080ce070 2024-04-25T10:15:12.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.918+0000 7f4f07fff640 1 --2- 172.21.15.16:0/2331098566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f080ce070 0x7f4f080c5ca0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f4ef402a150 tx=0x7f4ef40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.919+0000 7f4f057fa640 1 -- 172.21.15.16:0/2331098566 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4037040 con 0x7f4f080ce070 2024-04-25T10:15:12.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.919+0000 7f4f057fa640 1 -- 172.21.15.16:0/2331098566 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef4042070 con 0x7f4f080ce070 2024-04-25T10:15:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.919+0000 7f4f0eb28640 1 -- 172.21.15.16:0/2331098566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f080cb8f0 con 0x7f4f080ce070 2024-04-25T10:15:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.919+0000 7f4f057fa640 1 -- 172.21.15.16:0/2331098566 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4030900 con 0x7f4f080ce070 2024-04-25T10:15:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.919+0000 7f4f0eb28640 1 -- 172.21.15.16:0/2331098566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f080c7160 con 0x7f4f080ce070 2024-04-25T10:15:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.920+0000 7f4f0eb28640 1 -- 172.21.15.16:0/2331098566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f080750b0 con 0x7f4f080ce070 2024-04-25T10:15:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.920+0000 7f4f057fa640 1 -- 172.21.15.16:0/2331098566 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ef4030aa0 con 0x7f4f080ce070 2024-04-25T10:15:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.921+0000 7f4f057fa640 1 --2- 172.21.15.16:0/2331098566 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4eec06b4d0 0x7f4eec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.921+0000 7f4f057fa640 1 -- 172.21.15.16:0/2331098566 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f4ef40aa710 con 0x7f4f080ce070 2024-04-25T10:15:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.921+0000 7f4f0c8c4640 1 --2- 172.21.15.16:0/2331098566 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4eec06b4d0 0x7f4eec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.921+0000 7f4f0c8c4640 1 --2- 172.21.15.16:0/2331098566 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4eec06b4d0 0x7f4eec06d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4efc005880 tx=0x7f4efc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:11.925+0000 7f4f057fa640 1 -- 172.21.15.16:0/2331098566 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ef4079220 con 0x7f4f080ce070 2024-04-25T10:15:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.104+0000 7f4f0eb28640 1 -- 172.21.15.16:0/2331098566 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f080c7330 con 0x7f4eec06b4d0 2024-04-25T10:15:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.104+0000 7f4f057fa640 1 -- 172.21.15.16:0/2331098566 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4f080c7330 con 0x7f4eec06b4d0 2024-04-25T10:15:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T10:15:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.106+0000 7f4f0eb28640 1 -- 172.21.15.16:0/2331098566 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4eec06b4d0 msgr2=0x7f4eec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.106+0000 7f4f0eb28640 1 --2- 172.21.15.16:0/2331098566 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4eec06b4d0 0x7f4eec06d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4efc005880 tx=0x7f4efc009040 comp rx=0 tx=0).stop 2024-04-25T10:15:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.106+0000 7f4f0eb28640 1 -- 172.21.15.16:0/2331098566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f080ce070 msgr2=0x7f4f080c5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.106+0000 7f4f0eb28640 1 --2- 172.21.15.16:0/2331098566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f080ce070 0x7f4f080c5ca0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f4ef402a150 tx=0x7f4ef40046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.106+0000 7f4f0eb28640 1 -- 172.21.15.16:0/2331098566 shutdown_connections 2024-04-25T10:15:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.106+0000 7f4f0eb28640 1 --2- 172.21.15.16:0/2331098566 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f4eec06b4d0 0x7f4eec06d990 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.106+0000 7f4f0eb28640 1 --2- 172.21.15.16:0/2331098566 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f081611e0 0x7f4f080c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.106+0000 7f4f0eb28640 1 --2- 172.21.15.16:0/2331098566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f080ce070 0x7f4f080c5ca0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.106+0000 7f4f0eb28640 1 --2- 172.21.15.16:0/2331098566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f080acdf0 0x7f4f0815ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.106+0000 7f4f0eb28640 1 -- 172.21.15.16:0/2331098566 >> 172.21.15.16:0/2331098566 conn(0x7f4f080cc9e0 msgr2=0x7f4f080ccdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.106+0000 7f4f0eb28640 1 -- 172.21.15.16:0/2331098566 shutdown_connections 2024-04-25T10:15:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.106+0000 7f4f0eb28640 1 -- 172.21.15.16:0/2331098566 wait complete. 2024-04-25T10:15:12.121 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T10:15:12.121 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:15:12.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.222+0000 7f8c71600640 1 Processor -- start 2024-04-25T10:15:12.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.222+0000 7f8c71600640 1 -- start start 2024-04-25T10:15:12.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.223+0000 7f8c71600640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8c6c0ced30 0x7f8c6c0d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.223+0000 7f8c71600640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c6c0d3330 0x7f8c6c0b9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.223+0000 7f8c71600640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c6c0d2290 con 0x7f8c6c15dbd0 2024-04-25T10:15:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.223+0000 7f8c71600640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c6c15dfb0 con 0x7f8c6c0ced30 2024-04-25T10:15:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.223+0000 7f8c71600640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0b9860 con 0x7f8c6c0d3330 2024-04-25T10:15:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.223+0000 7f8c6b7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c6c0d3330 0x7f8c6c0b9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.223+0000 7f8c63fff640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8c6c0ced30 0x7f8c6c0d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.223+0000 7f8c6b7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c6c0d3330 0x7f8c6c0b9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.16:34096/0 (socket says 172.21.15.16:34096) 2024-04-25T10:15:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.223+0000 7f8c63fff640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8c6c0ced30 0x7f8c6c0d1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.16:33132/0 (socket says 172.21.15.16:33132) 2024-04-25T10:15:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.223+0000 7f8c6b7fe640 1 -- 172.21.15.16:0/3802572307 learned_addr learned my addr 172.21.15.16:0/3802572307 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.223+0000 7f8c6a7fc640 1 -- 172.21.15.16:0/3802572307 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 814342635 0 0) 0x7f8c6c0d2290 con 0x7f8c6c15dbd0 2024-04-25T10:15:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.223+0000 7f8c6a7fc640 1 -- 172.21.15.16:0/3802572307 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c58003520 con 0x7f8c6c15dbd0 2024-04-25T10:15:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.223+0000 7f8c63fff640 1 -- 172.21.15.16:0/3802572307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c6c0d3330 msgr2=0x7f8c6c0b9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.223+0000 7f8c63fff640 1 --2- 172.21.15.16:0/3802572307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c6c0d3330 0x7f8c6c0b9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.223+0000 7f8c63fff640 1 -- 172.21.15.16:0/3802572307 >> v1:172.21.15.186:6789/0 conn(0x7f8c6c15dbd0 legacy=0x7f8c6c0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.223+0000 7f8c63fff640 1 -- 172.21.15.16:0/3802572307 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6c0d2570 con 0x7f8c6c0ced30 2024-04-25T10:15:12.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.223+0000 7f8c6b7fe640 1 --2- 172.21.15.16:0/3802572307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c6c0d3330 0x7f8c6c0b9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:12.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.224+0000 7f8c63fff640 1 --2- 172.21.15.16:0/3802572307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c6c0ced30 0x7f8c6c0d1bb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8c540097a0 tx=0x7f8c5402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=894e8a0e615bac14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.224+0000 7f8c6a7fc640 1 -- 172.21.15.16:0/3802572307 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54037070 con 0x7f8c6c0ced30 2024-04-25T10:15:12.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.224+0000 7f8c6a7fc640 1 -- 172.21.15.16:0/3802572307 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c54032440 con 0x7f8c6c0ced30 2024-04-25T10:15:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.224+0000 7f8c6a7fc640 1 -- 172.21.15.16:0/3802572307 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54032810 con 0x7f8c6c0ced30 2024-04-25T10:15:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.225+0000 7f8c71600640 1 -- 172.21.15.16:0/3802572307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c6c0ced30 msgr2=0x7f8c6c0d1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.225+0000 7f8c71600640 1 --2- 172.21.15.16:0/3802572307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c6c0ced30 0x7f8c6c0d1bb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8c540097a0 tx=0x7f8c5402a180 comp rx=0 tx=0).stop 2024-04-25T10:15:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.225+0000 7f8c71600640 1 -- 172.21.15.16:0/3802572307 shutdown_connections 2024-04-25T10:15:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.225+0000 7f8c71600640 1 --2- 172.21.15.16:0/3802572307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c6c0d3330 0x7f8c6c0b9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.225+0000 7f8c71600640 1 --2- 172.21.15.16:0/3802572307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c6c0ced30 0x7f8c6c0d1bb0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.225+0000 7f8c71600640 1 -- 172.21.15.16:0/3802572307 >> 172.21.15.16:0/3802572307 conn(0x7f8c6c0726f0 msgr2=0x7f8c6c072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.225+0000 7f8c71600640 1 -- 172.21.15.16:0/3802572307 shutdown_connections 2024-04-25T10:15:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.225+0000 7f8c71600640 1 -- 172.21.15.16:0/3802572307 wait complete. 2024-04-25T10:15:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.226+0000 7f8c71600640 1 Processor -- start 2024-04-25T10:15:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.226+0000 7f8c71600640 1 -- start start 2024-04-25T10:15:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.227+0000 7f8c71600640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c6c0ced30 0x7f8c6c156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.227+0000 7f8c71600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c0d3330 0x7f8c6c0cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.227+0000 7f8c71600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c6c15dbd0 0x7f8c6c0b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.227+0000 7f8c71600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0d2e10 con 0x7f8c6c0d3330 2024-04-25T10:15:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.227+0000 7f8c71600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0d2b30 con 0x7f8c6c15dbd0 2024-04-25T10:15:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.227+0000 7f8c71600640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0b6980 con 0x7f8c6c0ced30 2024-04-25T10:15:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.227+0000 7f8c63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c0d3330 0x7f8c6c0cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.227+0000 7f8c63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c0d3330 0x7f8c6c0cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56500/0 (socket says 172.21.15.16:56500) 2024-04-25T10:15:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.227+0000 7f8c63fff640 1 -- 172.21.15.16:0/4152867960 learned_addr learned my addr 172.21.15.16:0/4152867960 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.227+0000 7f8c6b7fe640 1 --2- 172.21.15.16:0/4152867960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c6c15dbd0 0x7f8c6c0b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.227+0000 7f8c6affd640 1 --2- 172.21.15.16:0/4152867960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c6c0ced30 0x7f8c6c156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.228+0000 7f8c63fff640 1 -- 172.21.15.16:0/4152867960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c6c0ced30 msgr2=0x7f8c6c156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.228+0000 7f8c63fff640 1 --2- 172.21.15.16:0/4152867960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c6c0ced30 0x7f8c6c156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.228+0000 7f8c63fff640 1 -- 172.21.15.16:0/4152867960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c6c15dbd0 msgr2=0x7f8c6c0b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.228+0000 7f8c63fff640 1 --2- 172.21.15.16:0/4152867960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c6c15dbd0 0x7f8c6c0b7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.228+0000 7f8c63fff640 1 -- 172.21.15.16:0/4152867960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6c0cd050 con 0x7f8c6c0d3330 2024-04-25T10:15:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.228+0000 7f8c6b7fe640 1 --2- 172.21.15.16:0/4152867960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c6c15dbd0 0x7f8c6c0b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.228+0000 7f8c63fff640 1 --2- 172.21.15.16:0/4152867960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c0d3330 0x7f8c6c0cc600 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f8c54035040 tx=0x7f8c540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.228+0000 7f8c637fe640 1 -- 172.21.15.16:0/4152867960 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54041030 con 0x7f8c6c0d3330 2024-04-25T10:15:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.228+0000 7f8c637fe640 1 -- 172.21.15.16:0/4152867960 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5403d4e0 con 0x7f8c6c0d3330 2024-04-25T10:15:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.228+0000 7f8c71600640 1 -- 172.21.15.16:0/4152867960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6c157930 con 0x7f8c6c0d3330 2024-04-25T10:15:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.228+0000 7f8c637fe640 1 -- 172.21.15.16:0/4152867960 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54037040 con 0x7f8c6c0d3330 2024-04-25T10:15:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.228+0000 7f8c71600640 1 -- 172.21.15.16:0/4152867960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6c157ef0 con 0x7f8c6c0d3330 2024-04-25T10:15:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.229+0000 7f8c71600640 1 -- 172.21.15.16:0/4152867960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8c6c0749b0 con 0x7f8c6c0d3330 2024-04-25T10:15:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.229+0000 7f8c637fe640 1 -- 172.21.15.16:0/4152867960 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c54044070 con 0x7f8c6c0d3330 2024-04-25T10:15:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.230+0000 7f8c637fe640 1 --2- 172.21.15.16:0/4152867960 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8c4c06b4d0 0x7f8c4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.230+0000 7f8c637fe640 1 -- 172.21.15.16:0/4152867960 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f8c540aa4f0 con 0x7f8c6c0d3330 2024-04-25T10:15:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.230+0000 7f8c637fe640 1 --2- 172.21.15.16:0/4152867960 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x7f8c4c0714d0 0x7f8c4c0738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.230+0000 7f8c637fe640 1 -- 172.21.15.16:0/4152867960 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8c54032440 con 0x7f8c4c0714d0 2024-04-25T10:15:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.230+0000 7f8c637fe640 1 -- 172.21.15.16:0/4152867960 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c540b1050 con 0x7f8c6c0d3330 2024-04-25T10:15:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.230+0000 7f8c6affd640 1 --2- 172.21.15.16:0/4152867960 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8c4c06b4d0 0x7f8c4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.230+0000 7f8c6b7fe640 1 --2- 172.21.15.16:0/4152867960 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x7f8c4c0714d0 0x7f8c4c0738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.230+0000 7f8c6affd640 1 --2- 172.21.15.16:0/4152867960 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8c4c06b4d0 0x7f8c4c06d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c007fd0 tx=0x7f8c5c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.230+0000 7f8c6b7fe640 1 --2- 172.21.15.16:0/4152867960 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x7f8c4c0714d0 0x7f8c4c0738f0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.231+0000 7f8c637fe640 1 -- 172.21.15.16:0/4152867960 <== osd.2 v2:172.21.15.16:6800/2094356376 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8c54032440 con 0x7f8c4c0714d0 2024-04-25T10:15:12.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.244+0000 7f8c71600640 1 -- 172.21.15.16:0/4152867960 --> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8c6c06caa0 con 0x7f8c4c0714d0 2024-04-25T10:15:12.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.244+0000 7f8c637fe640 1 -- 172.21.15.16:0/4152867960 <== osd.2 v2:172.21.15.16:6800/2094356376 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8c6c06caa0 con 0x7f8c4c0714d0 2024-04-25T10:15:12.249 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:15:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:15:12.249 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:15:12.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.245+0000 7f8c71600640 1 -- 172.21.15.16:0/4152867960 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x7f8c4c0714d0 msgr2=0x7f8c4c0738f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.245+0000 7f8c71600640 1 --2- 172.21.15.16:0/4152867960 >> [v2:172.21.15.16:6800/2094356376,v1:172.21.15.16:6801/2094356376] conn(0x7f8c4c0714d0 0x7f8c4c0738f0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.245+0000 7f8c71600640 1 -- 172.21.15.16:0/4152867960 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8c4c06b4d0 msgr2=0x7f8c4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.245+0000 7f8c71600640 1 --2- 172.21.15.16:0/4152867960 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f8c4c06b4d0 0x7f8c4c06d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c007fd0 tx=0x7f8c5c009040 comp rx=0 tx=0).stop 2024-04-25T10:15:12.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.245+0000 7f8c71600640 1 -- 172.21.15.16:0/4152867960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c0d3330 msgr2=0x7f8c6c0cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.245+0000 7f8c71600640 1 --2- 172.21.15.16:0/4152867960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c6c0d3330 0x7f8c6c0cc600 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f8c54035040 tx=0x7f8c540046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.245+0000 7f8c6b7fe640 1 -- 172.21.15.16:0/4152867960 reap_dead start 2024-04-25T10:15:12.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.245+0000 7f8c71600640 1 -- 172.21.15.16:0/4152867960 shutdown_connections 2024-04-25T10:15:12.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.245+0000 7f8c71600640 1 -- 172.21.15.16:0/4152867960 >> 172.21.15.16:0/4152867960 conn(0x7f8c6c0726f0 msgr2=0x7f8c6c0b3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.245+0000 7f8c71600640 1 -- 172.21.15.16:0/4152867960 shutdown_connections 2024-04-25T10:15:12.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.245+0000 7f8c71600640 1 -- 172.21.15.16:0/4152867960 wait complete. 2024-04-25T10:15:12.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T10:15:12.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.356+0000 7f9ef955c640 1 Processor -- start 2024-04-25T10:15:12.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.356+0000 7f9ef955c640 1 -- start start 2024-04-25T10:15:12.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.357+0000 7f9ef955c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ef4147550 0x7f9ef40b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.357+0000 7f9ef955c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ef406e900 0x7f9ef406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.357+0000 7f9ef955c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef406bb70 con 0x7f9ef4161780 2024-04-25T10:15:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.357+0000 7f9ef955c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef406f270 con 0x7f9ef406e900 2024-04-25T10:15:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.357+0000 7f9ef955c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40b3190 con 0x7f9ef4147550 2024-04-25T10:15:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.357+0000 7f9ef37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ef406e900 0x7f9ef406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.357+0000 7f9ef27fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9ef4161780 0x7f9ef40acf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59472/0 (socket says 172.21.15.16:59472) 2024-04-25T10:15:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.357+0000 7f9ef27fc640 1 -- 172.21.15.16:0/295130402 learned_addr learned my addr 172.21.15.16:0/295130402 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.357+0000 7f9ef2ffd640 1 --2- 172.21.15.16:0/295130402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ef4147550 0x7f9ef40b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.357+0000 7f9ef37fe640 1 -- 172.21.15.16:0/295130402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ef4147550 msgr2=0x7f9ef40b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.357+0000 7f9ef37fe640 1 --2- 172.21.15.16:0/295130402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ef4147550 0x7f9ef40b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.357+0000 7f9ef37fe640 1 -- 172.21.15.16:0/295130402 >> v1:172.21.15.16:6789/0 conn(0x7f9ef4161780 legacy=0x7f9ef40acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.357+0000 7f9ef1ffb640 1 -- 172.21.15.16:0/295130402 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256360776 0 0) 0x7f9ef406bb70 con 0x7f9ef4161780 2024-04-25T10:15:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.357+0000 7f9ef37fe640 1 -- 172.21.15.16:0/295130402 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef40b3470 con 0x7f9ef406e900 2024-04-25T10:15:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.357+0000 7f9ef2ffd640 1 --2- 172.21.15.16:0/295130402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ef4147550 0x7f9ef40b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.358+0000 7f9ef37fe640 1 --2- 172.21.15.16:0/295130402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef406e900 0x7f9ef406ed30 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009d90 tx=0x7f9ee402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cf9115dd419a10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.358+0000 7f9ef1ffb640 1 -- 172.21.15.16:0/295130402 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4007740 con 0x7f9ef406e900 2024-04-25T10:15:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.358+0000 7f9ef1ffb640 1 -- 172.21.15.16:0/295130402 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee40078e0 con 0x7f9ef406e900 2024-04-25T10:15:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.358+0000 7f9ef1ffb640 1 -- 172.21.15.16:0/295130402 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4007cb0 con 0x7f9ef406e900 2024-04-25T10:15:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.359+0000 7f9ef955c640 1 -- 172.21.15.16:0/295130402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef406e900 msgr2=0x7f9ef406ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.359+0000 7f9ef955c640 1 --2- 172.21.15.16:0/295130402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef406e900 0x7f9ef406ed30 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009d90 tx=0x7f9ee402a660 comp rx=0 tx=0).stop 2024-04-25T10:15:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.359+0000 7f9ef955c640 1 -- 172.21.15.16:0/295130402 shutdown_connections 2024-04-25T10:15:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.359+0000 7f9ef955c640 1 --2- 172.21.15.16:0/295130402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef406e900 0x7f9ef406ed30 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.359+0000 7f9ef955c640 1 --2- 172.21.15.16:0/295130402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ef4147550 0x7f9ef40b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.359+0000 7f9ef955c640 1 -- 172.21.15.16:0/295130402 >> 172.21.15.16:0/295130402 conn(0x7f9ef406aba0 msgr2=0x7f9ef406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.359+0000 7f9ef955c640 1 -- 172.21.15.16:0/295130402 shutdown_connections 2024-04-25T10:15:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.359+0000 7f9ef955c640 1 -- 172.21.15.16:0/295130402 wait complete. 2024-04-25T10:15:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.360+0000 7f9ef955c640 1 Processor -- start 2024-04-25T10:15:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.360+0000 7f9ef955c640 1 -- start start 2024-04-25T10:15:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.361+0000 7f9ef955c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ef406e900 0x7f9ef4153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.361+0000 7f9ef955c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ef4147550 0x7f9ef4145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.361+0000 7f9ef955c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef4161780 0x7f9ef4145e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.361+0000 7f9ef955c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40b3d10 con 0x7f9ef4161780 2024-04-25T10:15:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.361+0000 7f9ef955c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40b3a30 con 0x7f9ef406e900 2024-04-25T10:15:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.361+0000 7f9ef955c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40ad6a0 con 0x7f9ef4147550 2024-04-25T10:15:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.361+0000 7f9ef37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef4161780 0x7f9ef4145e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.361+0000 7f9ef37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef4161780 0x7f9ef4145e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56518/0 (socket says 172.21.15.16:56518) 2024-04-25T10:15:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.361+0000 7f9ef37fe640 1 -- 172.21.15.16:0/2011561808 learned_addr learned my addr 172.21.15.16:0/2011561808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:12.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.361+0000 7f9ef2ffd640 1 --2- 172.21.15.16:0/2011561808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ef406e900 0x7f9ef4153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.361+0000 7f9ef27fc640 1 --2- 172.21.15.16:0/2011561808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ef4147550 0x7f9ef4145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.362+0000 7f9ef37fe640 1 -- 172.21.15.16:0/2011561808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ef4147550 msgr2=0x7f9ef4145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.362+0000 7f9ef37fe640 1 --2- 172.21.15.16:0/2011561808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ef4147550 0x7f9ef4145940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.362+0000 7f9ef37fe640 1 -- 172.21.15.16:0/2011561808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ef406e900 msgr2=0x7f9ef4153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.362+0000 7f9ef37fe640 1 --2- 172.21.15.16:0/2011561808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ef406e900 0x7f9ef4153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.362+0000 7f9ef37fe640 1 -- 172.21.15.16:0/2011561808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef40cdf00 con 0x7f9ef4161780 2024-04-25T10:15:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.362+0000 7f9ef37fe640 1 --2- 172.21.15.16:0/2011561808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef4161780 0x7f9ef4145e80 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009eb0 tx=0x7f9ee4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.362+0000 7f9eeb7fe640 1 -- 172.21.15.16:0/2011561808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee403f070 con 0x7f9ef4161780 2024-04-25T10:15:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.362+0000 7f9eeb7fe640 1 -- 172.21.15.16:0/2011561808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee4031bc0 con 0x7f9ef4161780 2024-04-25T10:15:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.362+0000 7f9ef955c640 1 -- 172.21.15.16:0/2011561808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef40ce1e0 con 0x7f9ef4161780 2024-04-25T10:15:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.362+0000 7f9eeb7fe640 1 -- 172.21.15.16:0/2011561808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4044460 con 0x7f9ef4161780 2024-04-25T10:15:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.362+0000 7f9ef955c640 1 -- 172.21.15.16:0/2011561808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef40ce7a0 con 0x7f9ef4161780 2024-04-25T10:15:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.363+0000 7f9ef955c640 1 -- 172.21.15.16:0/2011561808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef40750b0 con 0x7f9ef4161780 2024-04-25T10:15:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.363+0000 7f9eeb7fe640 1 -- 172.21.15.16:0/2011561808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ee4007790 con 0x7f9ef4161780 2024-04-25T10:15:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.364+0000 7f9eeb7fe640 1 --2- 172.21.15.16:0/2011561808 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9ed406b500 0x7f9ed406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.364+0000 7f9eeb7fe640 1 -- 172.21.15.16:0/2011561808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f9ee40ab690 con 0x7f9ef4161780 2024-04-25T10:15:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.364+0000 7f9ef2ffd640 1 --2- 172.21.15.16:0/2011561808 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9ed406b500 0x7f9ed406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.365+0000 7f9ef2ffd640 1 --2- 172.21.15.16:0/2011561808 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9ed406b500 0x7f9ed406d9c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9eec006e80 tx=0x7f9eec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.369+0000 7f9eeb7fe640 1 -- 172.21.15.16:0/2011561808 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9ee407a110 con 0x7f9ef4161780 2024-04-25T10:15:12.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.549+0000 7f9ef955c640 1 -- 172.21.15.16:0/2011561808 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ef40b2b10 con 0x7f9ed406b500 2024-04-25T10:15:12.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.549+0000 7f9eeb7fe640 1 -- 172.21.15.16:0/2011561808 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9ef40b2b10 con 0x7f9ed406b500 2024-04-25T10:15:12.553 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T10:15:12.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.551+0000 7f9ef955c640 1 -- 172.21.15.16:0/2011561808 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9ed406b500 msgr2=0x7f9ed406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.551+0000 7f9ef955c640 1 --2- 172.21.15.16:0/2011561808 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9ed406b500 0x7f9ed406d9c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9eec006e80 tx=0x7f9eec008040 comp rx=0 tx=0).stop 2024-04-25T10:15:12.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.551+0000 7f9ef955c640 1 -- 172.21.15.16:0/2011561808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef4161780 msgr2=0x7f9ef4145e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.551+0000 7f9ef955c640 1 --2- 172.21.15.16:0/2011561808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef4161780 0x7f9ef4145e80 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009eb0 tx=0x7f9ee4004a40 comp rx=0 tx=0).stop 2024-04-25T10:15:12.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.551+0000 7f9ef955c640 1 -- 172.21.15.16:0/2011561808 shutdown_connections 2024-04-25T10:15:12.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.551+0000 7f9ef955c640 1 --2- 172.21.15.16:0/2011561808 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f9ed406b500 0x7f9ed406d9c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.551+0000 7f9ef955c640 1 --2- 172.21.15.16:0/2011561808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef4161780 0x7f9ef4145e80 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.551+0000 7f9ef955c640 1 --2- 172.21.15.16:0/2011561808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ef4147550 0x7f9ef4145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.551+0000 7f9ef955c640 1 --2- 172.21.15.16:0/2011561808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ef406e900 0x7f9ef4153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.551+0000 7f9ef955c640 1 -- 172.21.15.16:0/2011561808 >> 172.21.15.16:0/2011561808 conn(0x7f9ef406aba0 msgr2=0x7f9ef40af590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.551+0000 7f9ef955c640 1 -- 172.21.15.16:0/2011561808 shutdown_connections 2024-04-25T10:15:12.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.551+0000 7f9ef955c640 1 -- 172.21.15.16:0/2011561808 wait complete. 2024-04-25T10:15:12.566 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T10:15:12.566 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:15:12.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.667+0000 7f85113d5640 1 Processor -- start 2024-04-25T10:15:12.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.667+0000 7f85113d5640 1 -- start start 2024-04-25T10:15:12.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.668+0000 7f85113d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f850c0b9720 0x7f850c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.668+0000 7f85113d5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f850c161f80 con 0x7f850c161090 2024-04-25T10:15:12.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.668+0000 7f85113d5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f850c162260 con 0x7f850c0cece0 2024-04-25T10:15:12.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.668+0000 7f85113d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850c0cf130 con 0x7f850c0b9720 2024-04-25T10:15:12.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.668+0000 7f850affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f850c161090 0x7f850c161730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59476/0 (socket says 172.21.15.16:59476) 2024-04-25T10:15:12.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.668+0000 7f850affd640 1 -- 172.21.15.16:0/450164839 learned_addr learned my addr 172.21.15.16:0/450164839 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:12.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f850a7fc640 1 --2- 172.21.15.16:0/450164839 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f850c0b9720 0x7f850c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f8509ffb640 1 -- 172.21.15.16:0/450164839 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711099226 0 0) 0x7f850c161f80 con 0x7f850c161090 2024-04-25T10:15:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f8509ffb640 1 -- 172.21.15.16:0/450164839 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84f4003520 con 0x7f850c161090 2024-04-25T10:15:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f8509ffb640 1 -- 172.21.15.16:0/450164839 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2663413910 0 0) 0x7f84f4003520 con 0x7f850c161090 2024-04-25T10:15:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f8509ffb640 1 -- 172.21.15.16:0/450164839 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f850c0b9720 msgr2=0x7f850c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f8509ffb640 1 --2- 172.21.15.16:0/450164839 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f850c0b9720 0x7f850c0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f8509ffb640 1 -- 172.21.15.16:0/450164839 >> v1:172.21.15.186:6789/0 conn(0x7f850c0cece0 legacy=0x7f850c0d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f8509ffb640 1 -- 172.21.15.16:0/450164839 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c162540 con 0x7f850c161090 2024-04-25T10:15:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f8509ffb640 1 -- 172.21.15.16:0/450164839 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f84fc0019f0 con 0x7f850c161090 2024-04-25T10:15:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f8509ffb640 1 -- 172.21.15.16:0/450164839 >> v1:172.21.15.16:6789/0 conn(0x7f850c161090 legacy=0x7f850c161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f8509ffb640 1 --2- 172.21.15.16:0/450164839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84f40039e0 0x7f84f4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f8509ffb640 1 --2- 172.21.15.16:0/450164839 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f850c0b9720 0x7f84f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f8509ffb640 1 --2- 172.21.15.16:0/450164839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84f4004d40 0x7f84f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f8509ffb640 1 -- 172.21.15.16:0/450164839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c0cf130 con 0x7f84f4004d40 2024-04-25T10:15:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f8509ffb640 1 -- 172.21.15.16:0/450164839 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f400b9c0 con 0x7f84f40039e0 2024-04-25T10:15:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f8509ffb640 1 -- 172.21.15.16:0/450164839 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f400bbc0 con 0x7f850c0b9720 2024-04-25T10:15:12.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f850b7fe640 1 --2- 172.21.15.16:0/450164839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84f4004d40 0x7f84f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f850affd640 1 --2- 172.21.15.16:0/450164839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84f40039e0 0x7f84f4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.669+0000 7f850a7fc640 1 --2- 172.21.15.16:0/450164839 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f850c0b9720 0x7f84f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.670+0000 7f850b7fe640 1 -- 172.21.15.16:0/450164839 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f850c0b9720 msgr2=0x7f84f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:12.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.670+0000 7f850b7fe640 1 --2- 172.21.15.16:0/450164839 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f850c0b9720 0x7f84f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.670+0000 7f850b7fe640 1 -- 172.21.15.16:0/450164839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84f40039e0 msgr2=0x7f84f4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:12.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.670+0000 7f850b7fe640 1 --2- 172.21.15.16:0/450164839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84f40039e0 0x7f84f4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.670+0000 7f850b7fe640 1 -- 172.21.15.16:0/450164839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f4003520 con 0x7f84f4004d40 2024-04-25T10:15:12.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.670+0000 7f850b7fe640 1 --2- 172.21.15.16:0/450164839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84f4004d40 0x7f84f40051c0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f8504001860 tx=0x7f84f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad82794debf1a72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.670+0000 7f8509ffb640 1 -- 172.21.15.16:0/450164839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504037070 con 0x7f84f4004d40 2024-04-25T10:15:12.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.670+0000 7f8509ffb640 1 -- 172.21.15.16:0/450164839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850402abf0 con 0x7f84f4004d40 2024-04-25T10:15:12.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.670+0000 7f8509ffb640 1 -- 172.21.15.16:0/450164839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504001430 con 0x7f84f4004d40 2024-04-25T10:15:12.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.671+0000 7f85113d5640 1 -- 172.21.15.16:0/450164839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84f4004d40 msgr2=0x7f84f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.671+0000 7f85113d5640 1 --2- 172.21.15.16:0/450164839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84f4004d40 0x7f84f40051c0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f8504001860 tx=0x7f84f4005830 comp rx=0 tx=0).stop 2024-04-25T10:15:12.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.671+0000 7f850b7fe640 1 -- 172.21.15.16:0/450164839 reap_dead start 2024-04-25T10:15:12.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.671+0000 7f85113d5640 1 -- 172.21.15.16:0/450164839 shutdown_connections 2024-04-25T10:15:12.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.671+0000 7f85113d5640 1 -- 172.21.15.16:0/450164839 >> 172.21.15.16:0/450164839 conn(0x7f850c0b55e0 msgr2=0x7f850c0b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.671+0000 7f85113d5640 1 -- 172.21.15.16:0/450164839 shutdown_connections 2024-04-25T10:15:12.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.671+0000 7f85113d5640 1 -- 172.21.15.16:0/450164839 wait complete. 2024-04-25T10:15:12.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.672+0000 7f85113d5640 1 Processor -- start 2024-04-25T10:15:12.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.672+0000 7f85113d5640 1 -- start start 2024-04-25T10:15:12.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.673+0000 7f85113d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f850c0b6e70 0x7f850c0ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.673+0000 7f85113d5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f850c0cd2b0 0x7f850c1f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.673+0000 7f85113d5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f850c0b7250 0x7f850c1f7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.673+0000 7f85113d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c161470 con 0x7f850c0b6e70 2024-04-25T10:15:12.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.673+0000 7f85113d5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c0d21a0 con 0x7f850c0cd2b0 2024-04-25T10:15:12.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.673+0000 7f85113d5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850c0ce7a0 con 0x7f850c0b7250 2024-04-25T10:15:12.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.673+0000 7f850affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f850c0b6e70 0x7f850c0ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.673+0000 7f850affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f850c0b6e70 0x7f850c0ccd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56548/0 (socket says 172.21.15.16:56548) 2024-04-25T10:15:12.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.673+0000 7f850affd640 1 -- 172.21.15.16:0/982517818 learned_addr learned my addr 172.21.15.16:0/982517818 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:12.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.673+0000 7f850b7fe640 1 --2- 172.21.15.16:0/982517818 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f850c0b7250 0x7f850c1f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.673+0000 7f850a7fc640 1 --2- 172.21.15.16:0/982517818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f850c0cd2b0 0x7f850c1f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.674+0000 7f850affd640 1 -- 172.21.15.16:0/982517818 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f850c0b7250 msgr2=0x7f850c1f7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.674+0000 7f850affd640 1 --2- 172.21.15.16:0/982517818 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f850c0b7250 0x7f850c1f7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.674+0000 7f850affd640 1 -- 172.21.15.16:0/982517818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f850c0cd2b0 msgr2=0x7f850c1f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.674+0000 7f850affd640 1 --2- 172.21.15.16:0/982517818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f850c0cd2b0 0x7f850c1f7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.674+0000 7f850affd640 1 -- 172.21.15.16:0/982517818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c15cf00 con 0x7f850c0b6e70 2024-04-25T10:15:12.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.674+0000 7f850affd640 1 --2- 172.21.15.16:0/982517818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f850c0b6e70 0x7f850c0ccd70 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f84fc001f20 tx=0x7f84fc00a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.674+0000 7f84f37fe640 1 -- 172.21.15.16:0/982517818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc017020 con 0x7f850c0b6e70 2024-04-25T10:15:12.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.674+0000 7f84f37fe640 1 -- 172.21.15.16:0/982517818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc0133f0 con 0x7f850c0b6e70 2024-04-25T10:15:12.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.674+0000 7f85113d5640 1 -- 172.21.15.16:0/982517818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850c15d1e0 con 0x7f850c0b6e70 2024-04-25T10:15:12.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.674+0000 7f84f37fe640 1 -- 172.21.15.16:0/982517818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc013800 con 0x7f850c0b6e70 2024-04-25T10:15:12.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.674+0000 7f85113d5640 1 -- 172.21.15.16:0/982517818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850c15d7a0 con 0x7f850c0b6e70 2024-04-25T10:15:12.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.675+0000 7f85113d5640 1 -- 172.21.15.16:0/982517818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f850c0750b0 con 0x7f850c0b6e70 2024-04-25T10:15:12.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.676+0000 7f84f37fe640 1 -- 172.21.15.16:0/982517818 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84fc01a020 con 0x7f850c0b6e70 2024-04-25T10:15:12.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.676+0000 7f84f37fe640 1 --2- 172.21.15.16:0/982517818 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f84f806b4d0 0x7f84f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.676+0000 7f84f37fe640 1 -- 172.21.15.16:0/982517818 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f84fc08aa60 con 0x7f850c0b6e70 2024-04-25T10:15:12.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.676+0000 7f84f37fe640 1 --2- 172.21.15.16:0/982517818 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x7f84f80714d0 0x7f84f80738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.676+0000 7f84f37fe640 1 -- 172.21.15.16:0/982517818 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f84f8073fa0 con 0x7f84f80714d0 2024-04-25T10:15:12.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.676+0000 7f84f37fe640 1 -- 172.21.15.16:0/982517818 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84fc08f2c0 con 0x7f850c0b6e70 2024-04-25T10:15:12.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.676+0000 7f850a7fc640 1 --2- 172.21.15.16:0/982517818 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f84f806b4d0 0x7f84f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.676+0000 7f850b7fe640 1 --2- 172.21.15.16:0/982517818 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x7f84f80714d0 0x7f84f80738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.677+0000 7f850b7fe640 1 --2- 172.21.15.16:0/982517818 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x7f84f80714d0 0x7f84f80738f0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.677+0000 7f850a7fc640 1 --2- 172.21.15.16:0/982517818 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f84f806b4d0 0x7f84f806d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f850c0c9fe0 tx=0x7f8500009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.677+0000 7f84f37fe640 1 -- 172.21.15.16:0/982517818 <== osd.3 v2:172.21.15.16:6824/256577261 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f84f8073fa0 con 0x7f84f80714d0 2024-04-25T10:15:12.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.691+0000 7f85113d5640 1 -- 172.21.15.16:0/982517818 --> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f850c0d1700 con 0x7f84f80714d0 2024-04-25T10:15:12.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.691+0000 7f84f37fe640 1 -- 172.21.15.16:0/982517818 <== osd.3 v2:172.21.15.16:6824/256577261 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f850c0d1700 con 0x7f84f80714d0 2024-04-25T10:15:12.695 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:15:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:15:12.695 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:15:12.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.691+0000 7f85113d5640 1 -- 172.21.15.16:0/982517818 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x7f84f80714d0 msgr2=0x7f84f80738f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.691+0000 7f85113d5640 1 --2- 172.21.15.16:0/982517818 >> [v2:172.21.15.16:6824/256577261,v1:172.21.15.16:6825/256577261] conn(0x7f84f80714d0 0x7f84f80738f0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.691+0000 7f85113d5640 1 -- 172.21.15.16:0/982517818 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f84f806b4d0 msgr2=0x7f84f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.691+0000 7f85113d5640 1 --2- 172.21.15.16:0/982517818 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f84f806b4d0 0x7f84f806d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f850c0c9fe0 tx=0x7f8500009190 comp rx=0 tx=0).stop 2024-04-25T10:15:12.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.691+0000 7f85113d5640 1 -- 172.21.15.16:0/982517818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f850c0b6e70 msgr2=0x7f850c0ccd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.691+0000 7f85113d5640 1 --2- 172.21.15.16:0/982517818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f850c0b6e70 0x7f850c0ccd70 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f84fc001f20 tx=0x7f84fc00a730 comp rx=0 tx=0).stop 2024-04-25T10:15:12.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.691+0000 7f850b7fe640 1 -- 172.21.15.16:0/982517818 reap_dead start 2024-04-25T10:15:12.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.691+0000 7f85113d5640 1 -- 172.21.15.16:0/982517818 shutdown_connections 2024-04-25T10:15:12.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.691+0000 7f85113d5640 1 -- 172.21.15.16:0/982517818 >> 172.21.15.16:0/982517818 conn(0x7f850c0b55e0 msgr2=0x7f850c0d26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.691+0000 7f85113d5640 1 -- 172.21.15.16:0/982517818 shutdown_connections 2024-04-25T10:15:12.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.691+0000 7f85113d5640 1 -- 172.21.15.16:0/982517818 wait complete. 2024-04-25T10:15:12.705 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T10:15:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.803+0000 7fcd5c207640 1 Processor -- start 2024-04-25T10:15:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.803+0000 7fcd5c207640 1 -- start start 2024-04-25T10:15:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.804+0000 7fcd5c207640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd540cd940 0x7fcd540ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.804+0000 7fcd5c207640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5406c220 con 0x7fcd5415dc50 2024-04-25T10:15:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.804+0000 7fcd5c207640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd540cec10 con 0x7fcd54071590 2024-04-25T10:15:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.804+0000 7fcd5c207640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540cfe00 con 0x7fcd540cd940 2024-04-25T10:15:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.804+0000 7fcd59fa3640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcd5415dc50 0x7fcd54070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46482/0 (socket says 172.21.15.16:46482) 2024-04-25T10:15:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.804+0000 7fcd59fa3640 1 -- 172.21.15.16:0/28525114 learned_addr learned my addr 172.21.15.16:0/28525114 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.804+0000 7fcd5a7a4640 1 --2- 172.21.15.16:0/28525114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd540cd940 0x7fcd540ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.804+0000 7fcd58fa1640 1 -- 172.21.15.16:0/28525114 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307444215 0 0) 0x7fcd5406c220 con 0x7fcd5415dc50 2024-04-25T10:15:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.804+0000 7fcd58fa1640 1 -- 172.21.15.16:0/28525114 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd40003520 con 0x7fcd5415dc50 2024-04-25T10:15:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.804+0000 7fcd58fa1640 1 -- 172.21.15.16:0/28525114 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 953326603 0 0) 0x7fcd540cec10 con 0x7fcd54071590 2024-04-25T10:15:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.804+0000 7fcd58fa1640 1 -- 172.21.15.16:0/28525114 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd5406c220 con 0x7fcd54071590 2024-04-25T10:15:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.804+0000 7fcd58fa1640 1 -- 172.21.15.16:0/28525114 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 865423827 0 0) 0x7fcd40003520 con 0x7fcd5415dc50 2024-04-25T10:15:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd58fa1640 1 -- 172.21.15.16:0/28525114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd540cd940 msgr2=0x7fcd540ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd58fa1640 1 --2- 172.21.15.16:0/28525114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd540cd940 0x7fcd540ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd58fa1640 1 -- 172.21.15.16:0/28525114 >> v1:172.21.15.186:6789/0 conn(0x7fcd54071590 legacy=0x7fcd540cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd58fa1640 1 -- 172.21.15.16:0/28525114 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd540d0050 con 0x7fcd5415dc50 2024-04-25T10:15:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd58fa1640 1 -- 172.21.15.16:0/28525114 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7fcd50001670 con 0x7fcd5415dc50 2024-04-25T10:15:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd5a7a4640 1 --2- 172.21.15.16:0/28525114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd540cd940 0x7fcd540ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd58fa1640 1 -- 172.21.15.16:0/28525114 >> v1:172.21.15.16:6789/0 conn(0x7fcd5415dc50 legacy=0x7fcd54070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd58fa1640 1 --2- 172.21.15.16:0/28525114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd40003b70 0x7fcd40003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd58fa1640 1 --2- 172.21.15.16:0/28525114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd540cd940 0x7fcd40004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd58fa1640 1 --2- 172.21.15.16:0/28525114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd40004d40 0x7fcd400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd58fa1640 1 -- 172.21.15.16:0/28525114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540cfe00 con 0x7fcd40003b70 2024-04-25T10:15:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd58fa1640 1 -- 172.21.15.16:0/28525114 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4000baa0 con 0x7fcd40004d40 2024-04-25T10:15:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd58fa1640 1 -- 172.21.15.16:0/28525114 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4000bca0 con 0x7fcd540cd940 2024-04-25T10:15:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd59fa3640 1 --2- 172.21.15.16:0/28525114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd40003b70 0x7fcd40003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd597a2640 1 --2- 172.21.15.16:0/28525114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd540cd940 0x7fcd40004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd5a7a4640 1 --2- 172.21.15.16:0/28525114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd40004d40 0x7fcd400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd59fa3640 1 -- 172.21.15.16:0/28525114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd540cd940 msgr2=0x7fcd40004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd59fa3640 1 --2- 172.21.15.16:0/28525114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd540cd940 0x7fcd40004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd59fa3640 1 -- 172.21.15.16:0/28525114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd40004d40 msgr2=0x7fcd400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd59fa3640 1 --2- 172.21.15.16:0/28525114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd40004d40 0x7fcd400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.805+0000 7fcd59fa3640 1 -- 172.21.15.16:0/28525114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd40003520 con 0x7fcd40003b70 2024-04-25T10:15:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.806+0000 7fcd59fa3640 1 --2- 172.21.15.16:0/28525114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd40003b70 0x7fcd40003f70 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fcd5000c190 tx=0x7fcd40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa184b0b9990643 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.806+0000 7fcd58fa1640 1 -- 172.21.15.16:0/28525114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50007ce0 con 0x7fcd40003b70 2024-04-25T10:15:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.806+0000 7fcd58fa1640 1 -- 172.21.15.16:0/28525114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd50005ad0 con 0x7fcd40003b70 2024-04-25T10:15:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.806+0000 7fcd58fa1640 1 -- 172.21.15.16:0/28525114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50007070 con 0x7fcd40003b70 2024-04-25T10:15:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.806+0000 7fcd5c207640 1 -- 172.21.15.16:0/28525114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd40003b70 msgr2=0x7fcd40003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.806+0000 7fcd5c207640 1 --2- 172.21.15.16:0/28525114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd40003b70 0x7fcd40003f70 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fcd5000c190 tx=0x7fcd40005830 comp rx=0 tx=0).stop 2024-04-25T10:15:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.806+0000 7fcd5a7a4640 1 -- 172.21.15.16:0/28525114 reap_dead start 2024-04-25T10:15:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.806+0000 7fcd5c207640 1 -- 172.21.15.16:0/28525114 shutdown_connections 2024-04-25T10:15:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.806+0000 7fcd5c207640 1 -- 172.21.15.16:0/28525114 >> 172.21.15.16:0/28525114 conn(0x7fcd5406f430 msgr2=0x7fcd5406f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.806+0000 7fcd5c207640 1 -- 172.21.15.16:0/28525114 shutdown_connections 2024-04-25T10:15:12.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.806+0000 7fcd5c207640 1 -- 172.21.15.16:0/28525114 wait complete. 2024-04-25T10:15:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.808+0000 7fcd5c207640 1 Processor -- start 2024-04-25T10:15:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.808+0000 7fcd5c207640 1 -- start start 2024-04-25T10:15:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.808+0000 7fcd5c207640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd40003b70 0x7fcd5415acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.808+0000 7fcd5c207640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd540b6f80 0x7fcd541f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.808+0000 7fcd5c207640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd540c9220 0x7fcd540c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.808+0000 7fcd5c207640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540ce8a0 con 0x7fcd40003b70 2024-04-25T10:15:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.808+0000 7fcd5c207640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540cc0c0 con 0x7fcd540b6f80 2024-04-25T10:15:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.808+0000 7fcd5c207640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5415e030 con 0x7fcd540c9220 2024-04-25T10:15:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.808+0000 7fcd59fa3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd40003b70 0x7fcd5415acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.808+0000 7fcd59fa3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd40003b70 0x7fcd5415acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43154/0 (socket says 172.21.15.16:43154) 2024-04-25T10:15:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.808+0000 7fcd59fa3640 1 -- 172.21.15.16:0/2574864530 learned_addr learned my addr 172.21.15.16:0/2574864530 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.809+0000 7fcd5a7a4640 1 --2- 172.21.15.16:0/2574864530 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd540c9220 0x7fcd540c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.809+0000 7fcd597a2640 1 --2- 172.21.15.16:0/2574864530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd540b6f80 0x7fcd541f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.809+0000 7fcd59fa3640 1 -- 172.21.15.16:0/2574864530 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd540c9220 msgr2=0x7fcd540c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.809+0000 7fcd59fa3640 1 --2- 172.21.15.16:0/2574864530 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd540c9220 0x7fcd540c96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.809+0000 7fcd59fa3640 1 -- 172.21.15.16:0/2574864530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd540b6f80 msgr2=0x7fcd541f6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.809+0000 7fcd59fa3640 1 --2- 172.21.15.16:0/2574864530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd540b6f80 0x7fcd541f6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.809+0000 7fcd59fa3640 1 -- 172.21.15.16:0/2574864530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd541f77f0 con 0x7fcd40003b70 2024-04-25T10:15:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.809+0000 7fcd597a2640 1 --2- 172.21.15.16:0/2574864530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd540b6f80 0x7fcd541f6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.809+0000 7fcd59fa3640 1 --2- 172.21.15.16:0/2574864530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd40003b70 0x7fcd5415acc0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fcd5000c280 tx=0x7fcd500019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.809+0000 7fcd4a7fc640 1 -- 172.21.15.16:0/2574864530 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50001670 con 0x7fcd40003b70 2024-04-25T10:15:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.809+0000 7fcd4a7fc640 1 -- 172.21.15.16:0/2574864530 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd50042070 con 0x7fcd40003b70 2024-04-25T10:15:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.809+0000 7fcd4a7fc640 1 -- 172.21.15.16:0/2574864530 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50007270 con 0x7fcd40003b70 2024-04-25T10:15:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.809+0000 7fcd5c207640 1 -- 172.21.15.16:0/2574864530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd541f7ad0 con 0x7fcd40003b70 2024-04-25T10:15:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.809+0000 7fcd5c207640 1 -- 172.21.15.16:0/2574864530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5415b200 con 0x7fcd40003b70 2024-04-25T10:15:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.810+0000 7fcd5c207640 1 -- 172.21.15.16:0/2574864530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd540b77a0 con 0x7fcd40003b70 2024-04-25T10:15:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.811+0000 7fcd4a7fc640 1 -- 172.21.15.16:0/2574864530 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd50007410 con 0x7fcd40003b70 2024-04-25T10:15:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.811+0000 7fcd4a7fc640 1 --2- 172.21.15.16:0/2574864530 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcd3806b500 0x7fcd3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.811+0000 7fcd4a7fc640 1 -- 172.21.15.16:0/2574864530 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fcd500a9ee0 con 0x7fcd40003b70 2024-04-25T10:15:12.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.811+0000 7fcd597a2640 1 --2- 172.21.15.16:0/2574864530 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcd3806b500 0x7fcd3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.812+0000 7fcd597a2640 1 --2- 172.21.15.16:0/2574864530 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcd3806b500 0x7fcd3806d9c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcd5406cd10 tx=0x7fcd44009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.815+0000 7fcd4a7fc640 1 -- 172.21.15.16:0/2574864530 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd50079160 con 0x7fcd40003b70 2024-04-25T10:15:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.995+0000 7fcd5c207640 1 -- 172.21.15.16:0/2574864530 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd54075090 con 0x7fcd3806b500 2024-04-25T10:15:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.996+0000 7fcd4a7fc640 1 -- 172.21.15.16:0/2574864530 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcd54075090 con 0x7fcd3806b500 2024-04-25T10:15:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T10:15:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.998+0000 7fcd5c207640 1 -- 172.21.15.16:0/2574864530 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcd3806b500 msgr2=0x7fcd3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.998+0000 7fcd5c207640 1 --2- 172.21.15.16:0/2574864530 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcd3806b500 0x7fcd3806d9c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcd5406cd10 tx=0x7fcd44009040 comp rx=0 tx=0).stop 2024-04-25T10:15:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.998+0000 7fcd5c207640 1 -- 172.21.15.16:0/2574864530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd40003b70 msgr2=0x7fcd5415acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.998+0000 7fcd5c207640 1 --2- 172.21.15.16:0/2574864530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd40003b70 0x7fcd5415acc0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fcd5000c280 tx=0x7fcd500019c0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.998+0000 7fcd5c207640 1 -- 172.21.15.16:0/2574864530 shutdown_connections 2024-04-25T10:15:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.998+0000 7fcd5c207640 1 --2- 172.21.15.16:0/2574864530 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fcd3806b500 0x7fcd3806d9c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.998+0000 7fcd5c207640 1 --2- 172.21.15.16:0/2574864530 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd540c9220 0x7fcd540c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.998+0000 7fcd5c207640 1 --2- 172.21.15.16:0/2574864530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd540b6f80 0x7fcd541f6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.998+0000 7fcd5c207640 1 --2- 172.21.15.16:0/2574864530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd40003b70 0x7fcd5415acc0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.998+0000 7fcd5c207640 1 -- 172.21.15.16:0/2574864530 >> 172.21.15.16:0/2574864530 conn(0x7fcd5406f430 msgr2=0x7fcd5415be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.998+0000 7fcd5c207640 1 -- 172.21.15.16:0/2574864530 shutdown_connections 2024-04-25T10:15:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:12.998+0000 7fcd5c207640 1 -- 172.21.15.16:0/2574864530 wait complete. 2024-04-25T10:15:13.013 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T10:15:13.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:15:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.113+0000 7f83917a5640 1 Processor -- start 2024-04-25T10:15:13.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.114+0000 7f83917a5640 1 -- start start 2024-04-25T10:15:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.114+0000 7f83917a5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f838c0cde40 0x7f838c0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.114+0000 7f83917a5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f838c06b8d0 0x7f838c0ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.114+0000 7f83917a5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f838c0cf520 0x7f838c06d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.114+0000 7f83917a5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838c0cf190 con 0x7f838c06b8d0 2024-04-25T10:15:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.114+0000 7f83917a5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838c0b31f0 con 0x7f838c0cde40 2024-04-25T10:15:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.114+0000 7f83917a5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838c0ce760 con 0x7f838c0cf520 2024-04-25T10:15:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.114+0000 7f838a7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f838c06b8d0 0x7f838c0ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.115+0000 7f838a7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f838c06b8d0 0x7f838c0ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43164/0 (socket says 172.21.15.16:43164) 2024-04-25T10:15:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.115+0000 7f838a7fc640 1 -- 172.21.15.16:0/3717705953 learned_addr learned my addr 172.21.15.16:0/3717705953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.115+0000 7f838affd640 1 --2- 172.21.15.16:0/3717705953 >> v2:172.21.15.186:3300/0 conn(0x7f838c0cde40 0x7f838c0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.115+0000 7f838b7fe640 1 --2- 172.21.15.16:0/3717705953 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f838c0cf520 0x7f838c06d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.115+0000 7f838a7fc640 1 -- 172.21.15.16:0/3717705953 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f838c0cf520 msgr2=0x7f838c06d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.115+0000 7f838a7fc640 1 --2- 172.21.15.16:0/3717705953 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f838c0cf520 0x7f838c06d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.115+0000 7f838a7fc640 1 -- 172.21.15.16:0/3717705953 >> v2:172.21.15.186:3300/0 conn(0x7f838c0cde40 msgr2=0x7f838c0cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.115+0000 7f838a7fc640 1 --2- 172.21.15.16:0/3717705953 >> v2:172.21.15.186:3300/0 conn(0x7f838c0cde40 0x7f838c0cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.115+0000 7f838a7fc640 1 -- 172.21.15.16:0/3717705953 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c06fea0 con 0x7f838c06b8d0 2024-04-25T10:15:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.116+0000 7f838a7fc640 1 --2- 172.21.15.16:0/3717705953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838c06b8d0 0x7f838c0ce220 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f838c04ffb0 tx=0x7f838002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9081884c8e32eb62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.116+0000 7f8389ffb640 1 -- 172.21.15.16:0/3717705953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380037070 con 0x7f838c06b8d0 2024-04-25T10:15:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.116+0000 7f8389ffb640 1 -- 172.21.15.16:0/3717705953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8380032440 con 0x7f838c06b8d0 2024-04-25T10:15:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.116+0000 7f8389ffb640 1 -- 172.21.15.16:0/3717705953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380032810 con 0x7f838c06b8d0 2024-04-25T10:15:13.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.116+0000 7f83917a5640 1 -- 172.21.15.16:0/3717705953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838c06b8d0 msgr2=0x7f838c0ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.116+0000 7f83917a5640 1 --2- 172.21.15.16:0/3717705953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838c06b8d0 0x7f838c0ce220 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f838c04ffb0 tx=0x7f838002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.116+0000 7f83917a5640 1 -- 172.21.15.16:0/3717705953 shutdown_connections 2024-04-25T10:15:13.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.116+0000 7f83917a5640 1 --2- 172.21.15.16:0/3717705953 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f838c0cf520 0x7f838c06d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.116+0000 7f83917a5640 1 --2- 172.21.15.16:0/3717705953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838c06b8d0 0x7f838c0ce220 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.116+0000 7f83917a5640 1 --2- 172.21.15.16:0/3717705953 >> v2:172.21.15.186:3300/0 conn(0x7f838c0cde40 0x7f838c0cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.116+0000 7f83917a5640 1 -- 172.21.15.16:0/3717705953 >> 172.21.15.16:0/3717705953 conn(0x7f838c06aba0 msgr2=0x7f838c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.116+0000 7f83917a5640 1 -- 172.21.15.16:0/3717705953 shutdown_connections 2024-04-25T10:15:13.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.116+0000 7f83917a5640 1 -- 172.21.15.16:0/3717705953 wait complete. 2024-04-25T10:15:13.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.118+0000 7f83917a5640 1 Processor -- start 2024-04-25T10:15:13.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.118+0000 7f83917a5640 1 -- start start 2024-04-25T10:15:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.118+0000 7f83917a5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f838c06b8d0 0x7f838c147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.118+0000 7f83917a5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f838c0cde40 0x7f838c1f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.118+0000 7f83917a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838c0cf520 0x7f838c071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.118+0000 7f83917a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c15adb0 con 0x7f838c0cf520 2024-04-25T10:15:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.118+0000 7f83917a5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c0b2f10 con 0x7f838c0cde40 2024-04-25T10:15:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.118+0000 7f83917a5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838c06fb30 con 0x7f838c06b8d0 2024-04-25T10:15:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.119+0000 7f838b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838c0cf520 0x7f838c071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.119+0000 7f838b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838c0cf520 0x7f838c071ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43174/0 (socket says 172.21.15.16:43174) 2024-04-25T10:15:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.119+0000 7f838b7fe640 1 -- 172.21.15.16:0/281963556 learned_addr learned my addr 172.21.15.16:0/281963556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.119+0000 7f838a7fc640 1 --2- 172.21.15.16:0/281963556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f838c0cde40 0x7f838c1f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.119+0000 7f838affd640 1 --2- 172.21.15.16:0/281963556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f838c06b8d0 0x7f838c147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.119+0000 7f838b7fe640 1 -- 172.21.15.16:0/281963556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f838c06b8d0 msgr2=0x7f838c147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.119+0000 7f838b7fe640 1 --2- 172.21.15.16:0/281963556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f838c06b8d0 0x7f838c147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.119+0000 7f838b7fe640 1 -- 172.21.15.16:0/281963556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f838c0cde40 msgr2=0x7f838c1f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.119+0000 7f838b7fe640 1 --2- 172.21.15.16:0/281963556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f838c0cde40 0x7f838c1f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.119+0000 7f838b7fe640 1 -- 172.21.15.16:0/281963556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c148490 con 0x7f838c0cf520 2024-04-25T10:15:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.119+0000 7f838b7fe640 1 --2- 172.21.15.16:0/281963556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838c0cf520 0x7f838c071ef0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f838400efc0 tx=0x7f838400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.119+0000 7f83737fe640 1 -- 172.21.15.16:0/281963556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8384018020 con 0x7f838c0cf520 2024-04-25T10:15:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.119+0000 7f83737fe640 1 -- 172.21.15.16:0/281963556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8384014480 con 0x7f838c0cf520 2024-04-25T10:15:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.119+0000 7f83737fe640 1 -- 172.21.15.16:0/281963556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8384014870 con 0x7f838c0cf520 2024-04-25T10:15:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.119+0000 7f83917a5640 1 -- 172.21.15.16:0/281963556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838c1f5cc0 con 0x7f838c0cf520 2024-04-25T10:15:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.119+0000 7f83917a5640 1 -- 172.21.15.16:0/281963556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838c14f890 con 0x7f838c0cf520 2024-04-25T10:15:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.120+0000 7f83917a5640 1 -- 172.21.15.16:0/281963556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f838c0750b0 con 0x7f838c0cf520 2024-04-25T10:15:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.121+0000 7f83737fe640 1 -- 172.21.15.16:0/281963556 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8384005d10 con 0x7f838c0cf520 2024-04-25T10:15:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.121+0000 7f83737fe640 1 --2- 172.21.15.16:0/281963556 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f837806b4d0 0x7f837806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.121+0000 7f83737fe640 1 -- 172.21.15.16:0/281963556 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f838408a930 con 0x7f838c0cf520 2024-04-25T10:15:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.122+0000 7f838affd640 1 --2- 172.21.15.16:0/281963556 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f837806b4d0 0x7f837806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.122+0000 7f83737fe640 1 --2- 172.21.15.16:0/281963556 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x7f83780714d0 0x7f83780738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.122+0000 7f83737fe640 1 -- 172.21.15.16:0/281963556 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8384020eb0 con 0x7f83780714d0 2024-04-25T10:15:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.122+0000 7f83737fe640 1 -- 172.21.15.16:0/281963556 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f838408f2a0 con 0x7f838c0cf520 2024-04-25T10:15:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.122+0000 7f838a7fc640 1 --2- 172.21.15.16:0/281963556 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x7f83780714d0 0x7f83780738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.122+0000 7f838affd640 1 --2- 172.21.15.16:0/281963556 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f837806b4d0 0x7f837806d990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f837c009ac0 tx=0x7f837c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.122+0000 7f838a7fc640 1 --2- 172.21.15.16:0/281963556 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x7f83780714d0 0x7f83780738f0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.123+0000 7f83737fe640 1 -- 172.21.15.16:0/281963556 <== osd.4 v2:172.21.15.186:6824/3091472889 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8384020eb0 con 0x7f83780714d0 2024-04-25T10:15:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.137+0000 7f83917a5640 1 -- 172.21.15.16:0/281963556 --> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f838c0af930 con 0x7f83780714d0 2024-04-25T10:15:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.137+0000 7f83737fe640 1 -- 172.21.15.16:0/281963556 <== osd.4 v2:172.21.15.186:6824/3091472889 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f838c0af930 con 0x7f83780714d0 2024-04-25T10:15:13.142 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:15:13.142 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:15:13.142 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:15:13.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.138+0000 7f83917a5640 1 -- 172.21.15.16:0/281963556 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x7f83780714d0 msgr2=0x7f83780738f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.138+0000 7f83917a5640 1 --2- 172.21.15.16:0/281963556 >> [v2:172.21.15.186:6824/3091472889,v1:172.21.15.186:6825/3091472889] conn(0x7f83780714d0 0x7f83780738f0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.138+0000 7f83917a5640 1 -- 172.21.15.16:0/281963556 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f837806b4d0 msgr2=0x7f837806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.138+0000 7f83917a5640 1 --2- 172.21.15.16:0/281963556 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f837806b4d0 0x7f837806d990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f837c009ac0 tx=0x7f837c009190 comp rx=0 tx=0).stop 2024-04-25T10:15:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.138+0000 7f83917a5640 1 -- 172.21.15.16:0/281963556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838c0cf520 msgr2=0x7f838c071ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.138+0000 7f83917a5640 1 --2- 172.21.15.16:0/281963556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838c0cf520 0x7f838c071ef0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f838400efc0 tx=0x7f838400c490 comp rx=0 tx=0).stop 2024-04-25T10:15:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.138+0000 7f838b7fe640 1 -- 172.21.15.16:0/281963556 reap_dead start 2024-04-25T10:15:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.138+0000 7f83917a5640 1 -- 172.21.15.16:0/281963556 shutdown_connections 2024-04-25T10:15:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.138+0000 7f83917a5640 1 -- 172.21.15.16:0/281963556 >> 172.21.15.16:0/281963556 conn(0x7f838c06aba0 msgr2=0x7f838c0afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.138+0000 7f83917a5640 1 -- 172.21.15.16:0/281963556 shutdown_connections 2024-04-25T10:15:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.138+0000 7f83917a5640 1 -- 172.21.15.16:0/281963556 wait complete. 2024-04-25T10:15:13.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T10:15:13.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.250+0000 7f49f9b7e640 1 Processor -- start 2024-04-25T10:15:13.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.250+0000 7f49f9b7e640 1 -- start start 2024-04-25T10:15:13.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.251+0000 7f49f9b7e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49f40ce1e0 0x7f49f40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.251+0000 7f49f9b7e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f415f490 con 0x7f49f40cc240 2024-04-25T10:15:13.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.251+0000 7f49f9b7e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f40cf8f0 con 0x7f49f415f0b0 2024-04-25T10:15:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.251+0000 7f49f9b7e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49f40cebb0 con 0x7f49f40ce1e0 2024-04-25T10:15:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.251+0000 7f49f2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49f40ce1e0 0x7f49f40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.251+0000 7f49f3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f49f40cc240 0x7f49f40cc670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46492/0 (socket says 172.21.15.16:46492) 2024-04-25T10:15:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.251+0000 7f49f3fff640 1 -- 172.21.15.16:0/452273313 learned_addr learned my addr 172.21.15.16:0/452273313 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.251+0000 7f49f2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49f40ce1e0 0x7f49f40ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43184/0 (socket says 172.21.15.16:43184) 2024-04-25T10:15:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.251+0000 7f49f2ffd640 1 -- 172.21.15.16:0/452273313 >> v1:172.21.15.186:6789/0 conn(0x7f49f415f0b0 legacy=0x7f49f40d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.251+0000 7f49f2ffd640 1 -- 172.21.15.16:0/452273313 >> v1:172.21.15.16:6789/0 conn(0x7f49f40cc240 legacy=0x7f49f40cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.251+0000 7f49f2ffd640 1 -- 172.21.15.16:0/452273313 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f40cfbd0 con 0x7f49f40ce1e0 2024-04-25T10:15:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.251+0000 7f49f27fc640 1 -- 172.21.15.16:0/452273313 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2508827244 0 0) 0x7f49f415f490 con 0x7f49f40cc240 2024-04-25T10:15:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.252+0000 7f49f2ffd640 1 --2- 172.21.15.16:0/452273313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f40ce1e0 0x7f49f40ce670 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f49f40ced30 tx=0x7f49e802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4a68f1c13a21c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.252+0000 7f49f27fc640 1 -- 172.21.15.16:0/452273313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8037070 con 0x7f49f40ce1e0 2024-04-25T10:15:13.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.252+0000 7f49f27fc640 1 -- 172.21.15.16:0/452273313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e8032640 con 0x7f49f40ce1e0 2024-04-25T10:15:13.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.252+0000 7f49f27fc640 1 -- 172.21.15.16:0/452273313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8032a10 con 0x7f49f40ce1e0 2024-04-25T10:15:13.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.252+0000 7f49f9b7e640 1 -- 172.21.15.16:0/452273313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f40ce1e0 msgr2=0x7f49f40ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.252+0000 7f49f9b7e640 1 --2- 172.21.15.16:0/452273313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f40ce1e0 0x7f49f40ce670 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f49f40ced30 tx=0x7f49e802a2c0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.252+0000 7f49f9b7e640 1 -- 172.21.15.16:0/452273313 shutdown_connections 2024-04-25T10:15:13.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.252+0000 7f49f9b7e640 1 --2- 172.21.15.16:0/452273313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f40ce1e0 0x7f49f40ce670 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.252+0000 7f49f9b7e640 1 -- 172.21.15.16:0/452273313 >> 172.21.15.16:0/452273313 conn(0x7f49f406f430 msgr2=0x7f49f406f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.252+0000 7f49f9b7e640 1 -- 172.21.15.16:0/452273313 shutdown_connections 2024-04-25T10:15:13.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.252+0000 7f49f9b7e640 1 -- 172.21.15.16:0/452273313 wait complete. 2024-04-25T10:15:13.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.254+0000 7f49f9b7e640 1 Processor -- start 2024-04-25T10:15:13.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.254+0000 7f49f9b7e640 1 -- start start 2024-04-25T10:15:13.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.254+0000 7f49f9b7e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49f40cc240 0x7f49f415cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.254+0000 7f49f9b7e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49f40ce1e0 0x7f49f40c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.254+0000 7f49f9b7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f415f0b0 0x7f49f406c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.254+0000 7f49f9b7e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f40d0470 con 0x7f49f415f0b0 2024-04-25T10:15:13.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.254+0000 7f49f9b7e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f40d0190 con 0x7f49f40ce1e0 2024-04-25T10:15:13.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.254+0000 7f49f9b7e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f40b4820 con 0x7f49f40cc240 2024-04-25T10:15:13.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.255+0000 7f49f3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f415f0b0 0x7f49f406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.255+0000 7f49f3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f415f0b0 0x7f49f406c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43196/0 (socket says 172.21.15.16:43196) 2024-04-25T10:15:13.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.255+0000 7f49f3fff640 1 -- 172.21.15.16:0/1838134722 learned_addr learned my addr 172.21.15.16:0/1838134722 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:13.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.255+0000 7f49f2ffd640 1 --2- 172.21.15.16:0/1838134722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49f40ce1e0 0x7f49f40c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.255+0000 7f49f37fe640 1 --2- 172.21.15.16:0/1838134722 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49f40cc240 0x7f49f415cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.255+0000 7f49f3fff640 1 -- 172.21.15.16:0/1838134722 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49f40cc240 msgr2=0x7f49f415cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.255+0000 7f49f3fff640 1 --2- 172.21.15.16:0/1838134722 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49f40cc240 0x7f49f415cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.255+0000 7f49f3fff640 1 -- 172.21.15.16:0/1838134722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49f40ce1e0 msgr2=0x7f49f40c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.255+0000 7f49f3fff640 1 --2- 172.21.15.16:0/1838134722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49f40ce1e0 0x7f49f40c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.255+0000 7f49f3fff640 1 -- 172.21.15.16:0/1838134722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f415bfa0 con 0x7f49f415f0b0 2024-04-25T10:15:13.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.255+0000 7f49f2ffd640 1 --2- 172.21.15.16:0/1838134722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49f40ce1e0 0x7f49f40c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:13.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.255+0000 7f49f3fff640 1 --2- 172.21.15.16:0/1838134722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f415f0b0 0x7f49f406c420 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00c7a0 tx=0x7f49ec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.255+0000 7f49dbfff640 1 -- 172.21.15.16:0/1838134722 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec018070 con 0x7f49f415f0b0 2024-04-25T10:15:13.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.255+0000 7f49dbfff640 1 -- 172.21.15.16:0/1838134722 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec00a640 con 0x7f49f415f0b0 2024-04-25T10:15:13.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.255+0000 7f49dbfff640 1 -- 172.21.15.16:0/1838134722 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec00aab0 con 0x7f49f415f0b0 2024-04-25T10:15:13.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.255+0000 7f49f9b7e640 1 -- 172.21.15.16:0/1838134722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f415c280 con 0x7f49f415f0b0 2024-04-25T10:15:13.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.255+0000 7f49f9b7e640 1 -- 172.21.15.16:0/1838134722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f406bf00 con 0x7f49f415f0b0 2024-04-25T10:15:13.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.256+0000 7f49f9b7e640 1 -- 172.21.15.16:0/1838134722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f40750b0 con 0x7f49f415f0b0 2024-04-25T10:15:13.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.257+0000 7f49dbfff640 1 -- 172.21.15.16:0/1838134722 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49ec01b070 con 0x7f49f415f0b0 2024-04-25T10:15:13.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.257+0000 7f49dbfff640 1 --2- 172.21.15.16:0/1838134722 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f49e006b4d0 0x7f49e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.257+0000 7f49dbfff640 1 -- 172.21.15.16:0/1838134722 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f49ec0895f0 con 0x7f49f415f0b0 2024-04-25T10:15:13.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.258+0000 7f49f37fe640 1 --2- 172.21.15.16:0/1838134722 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f49e006b4d0 0x7f49e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.258+0000 7f49f37fe640 1 --2- 172.21.15.16:0/1838134722 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f49e006b4d0 0x7f49e006d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f49e4000f60 tx=0x7f49e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.262+0000 7f49dbfff640 1 -- 172.21.15.16:0/1838134722 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f49ec0586c0 con 0x7f49f415f0b0 2024-04-25T10:15:13.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.442+0000 7f49f9b7e640 1 -- 172.21.15.16:0/1838134722 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f49f40d0e50 con 0x7f49e006b4d0 2024-04-25T10:15:13.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.443+0000 7f49dbfff640 1 -- 172.21.15.16:0/1838134722 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f49f40d0e50 con 0x7f49e006b4d0 2024-04-25T10:15:13.447 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T10:15:13.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.445+0000 7f49f9b7e640 1 -- 172.21.15.16:0/1838134722 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f49e006b4d0 msgr2=0x7f49e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.445+0000 7f49f9b7e640 1 --2- 172.21.15.16:0/1838134722 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f49e006b4d0 0x7f49e006d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f49e4000f60 tx=0x7f49e4009040 comp rx=0 tx=0).stop 2024-04-25T10:15:13.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.445+0000 7f49f9b7e640 1 -- 172.21.15.16:0/1838134722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f415f0b0 msgr2=0x7f49f406c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.445+0000 7f49f9b7e640 1 --2- 172.21.15.16:0/1838134722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f415f0b0 0x7f49f406c420 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00c7a0 tx=0x7f49ec00cc60 comp rx=0 tx=0).stop 2024-04-25T10:15:13.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.445+0000 7f49f9b7e640 1 -- 172.21.15.16:0/1838134722 shutdown_connections 2024-04-25T10:15:13.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.445+0000 7f49f9b7e640 1 --2- 172.21.15.16:0/1838134722 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f49e006b4d0 0x7f49e006d990 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.445+0000 7f49f9b7e640 1 --2- 172.21.15.16:0/1838134722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f415f0b0 0x7f49f406c420 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.445+0000 7f49f9b7e640 1 --2- 172.21.15.16:0/1838134722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49f40ce1e0 0x7f49f40c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.445+0000 7f49f9b7e640 1 --2- 172.21.15.16:0/1838134722 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49f40cc240 0x7f49f415cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.445+0000 7f49f9b7e640 1 -- 172.21.15.16:0/1838134722 >> 172.21.15.16:0/1838134722 conn(0x7f49f406f430 msgr2=0x7f49f415d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.445+0000 7f49f9b7e640 1 -- 172.21.15.16:0/1838134722 shutdown_connections 2024-04-25T10:15:13.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.445+0000 7f49f9b7e640 1 -- 172.21.15.16:0/1838134722 wait complete. 2024-04-25T10:15:13.459 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T10:15:13.459 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:15:13.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.560+0000 7fa7bf929640 1 Processor -- start 2024-04-25T10:15:13.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.560+0000 7fa7bf929640 1 -- start start 2024-04-25T10:15:13.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.561+0000 7fa7bf929640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7b80aedc0 0x7fa7b815ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.561+0000 7fa7bf929640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa7b80b4900 0x7fa7b815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.561+0000 7fa7bf929640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b806da40 con 0x7fa7b806d260 2024-04-25T10:15:13.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.561+0000 7fa7bf929640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b815c050 con 0x7fa7b80b4900 2024-04-25T10:15:13.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.561+0000 7fa7bf929640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b80af1a0 con 0x7fa7b80aedc0 2024-04-25T10:15:13.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.561+0000 7fa7bdec6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa7b80b4900 0x7fa7b815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.561+0000 7fa7bd6c5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7b80aedc0 0x7fa7b815ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.561+0000 7fa7bcec4640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fa7b806d260 0x7fa7b80b41f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.16:41432/0 (socket says 172.21.15.16:41432) 2024-04-25T10:15:13.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.561+0000 7fa7bcec4640 1 -- 172.21.15.16:0/3263158833 learned_addr learned my addr 172.21.15.16:0/3263158833 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:13.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.561+0000 7fa7bd6c5640 1 -- 172.21.15.16:0/3263158833 >> v2:172.21.15.186:3300/0 conn(0x7fa7b80b4900 msgr2=0x7fa7b815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:13.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.561+0000 7fa7bd6c5640 1 --2- 172.21.15.16:0/3263158833 >> v2:172.21.15.186:3300/0 conn(0x7fa7b80b4900 0x7fa7b815c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.561+0000 7fa7bd6c5640 1 -- 172.21.15.16:0/3263158833 >> v1:172.21.15.186:6789/0 conn(0x7fa7b806d260 legacy=0x7fa7b80b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.561+0000 7fa7bd6c5640 1 -- 172.21.15.16:0/3263158833 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b806dd20 con 0x7fa7b80aedc0 2024-04-25T10:15:13.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.561+0000 7fa7affff640 1 -- 172.21.15.16:0/3263158833 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134343034 0 0) 0x7fa7b806da40 con 0x7fa7b806d260 2024-04-25T10:15:13.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.562+0000 7fa7bdec6640 1 --2- 172.21.15.16:0/3263158833 >> v2:172.21.15.186:3300/0 conn(0x7fa7b80b4900 0x7fa7b815c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:13.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.562+0000 7fa7bd6c5640 1 --2- 172.21.15.16:0/3263158833 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7b80aedc0 0x7fa7b815ae20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa7b40097a0 tx=0x7fa7b402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9015109a6dd5c72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.562+0000 7fa7affff640 1 -- 172.21.15.16:0/3263158833 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4037070 con 0x7fa7b80aedc0 2024-04-25T10:15:13.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.562+0000 7fa7affff640 1 -- 172.21.15.16:0/3263158833 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b402ae50 con 0x7fa7b80aedc0 2024-04-25T10:15:13.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.562+0000 7fa7affff640 1 -- 172.21.15.16:0/3263158833 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4032660 con 0x7fa7b80aedc0 2024-04-25T10:15:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.563+0000 7fa7bf929640 1 -- 172.21.15.16:0/3263158833 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7b80aedc0 msgr2=0x7fa7b815ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.563+0000 7fa7bf929640 1 --2- 172.21.15.16:0/3263158833 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7b80aedc0 0x7fa7b815ae20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa7b40097a0 tx=0x7fa7b402a100 comp rx=0 tx=0).stop 2024-04-25T10:15:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.563+0000 7fa7bf929640 1 -- 172.21.15.16:0/3263158833 shutdown_connections 2024-04-25T10:15:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.563+0000 7fa7bf929640 1 --2- 172.21.15.16:0/3263158833 >> v2:172.21.15.186:3300/0 conn(0x7fa7b80b4900 0x7fa7b815c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.563+0000 7fa7bf929640 1 --2- 172.21.15.16:0/3263158833 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7b80aedc0 0x7fa7b815ae20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.563+0000 7fa7bf929640 1 -- 172.21.15.16:0/3263158833 >> 172.21.15.16:0/3263158833 conn(0x7fa7b806aba0 msgr2=0x7fa7b806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.563+0000 7fa7bf929640 1 -- 172.21.15.16:0/3263158833 shutdown_connections 2024-04-25T10:15:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.563+0000 7fa7bf929640 1 -- 172.21.15.16:0/3263158833 wait complete. 2024-04-25T10:15:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.564+0000 7fa7bf929640 1 Processor -- start 2024-04-25T10:15:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.564+0000 7fa7bf929640 1 -- start start 2024-04-25T10:15:13.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.565+0000 7fa7bf929640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b806d260 0x7fa7b81e5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.565+0000 7fa7bf929640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7b80aedc0 0x7fa7b80a8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.565+0000 7fa7bf929640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7b80b4900 0x7fa7b8154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.565+0000 7fa7bf929640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b806d6c0 con 0x7fa7b806d260 2024-04-25T10:15:13.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.565+0000 7fa7bf929640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b815c2a0 con 0x7fa7b80aedc0 2024-04-25T10:15:13.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.565+0000 7fa7bf929640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b815cd00 con 0x7fa7b80b4900 2024-04-25T10:15:13.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.565+0000 7fa7bd6c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b806d260 0x7fa7b81e5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.565+0000 7fa7bd6c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b806d260 0x7fa7b81e5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43204/0 (socket says 172.21.15.16:43204) 2024-04-25T10:15:13.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.565+0000 7fa7bd6c5640 1 -- 172.21.15.16:0/3922336798 learned_addr learned my addr 172.21.15.16:0/3922336798 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:13.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.565+0000 7fa7bcec4640 1 --2- 172.21.15.16:0/3922336798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7b80aedc0 0x7fa7b80a8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.565+0000 7fa7bdec6640 1 --2- 172.21.15.16:0/3922336798 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7b80b4900 0x7fa7b8154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.565+0000 7fa7bd6c5640 1 -- 172.21.15.16:0/3922336798 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7b80b4900 msgr2=0x7fa7b8154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.565+0000 7fa7bd6c5640 1 --2- 172.21.15.16:0/3922336798 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7b80b4900 0x7fa7b8154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.565+0000 7fa7bd6c5640 1 -- 172.21.15.16:0/3922336798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7b80aedc0 msgr2=0x7fa7b80a8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.565+0000 7fa7bd6c5640 1 --2- 172.21.15.16:0/3922336798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7b80aedc0 0x7fa7b80a8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.565+0000 7fa7bd6c5640 1 -- 172.21.15.16:0/3922336798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b81e63d0 con 0x7fa7b806d260 2024-04-25T10:15:13.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.566+0000 7fa7bd6c5640 1 --2- 172.21.15.16:0/3922336798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b806d260 0x7fa7b81e5df0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4035040 tx=0x7fa7b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.566+0000 7fa7adffb640 1 -- 172.21.15.16:0/3922336798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4041070 con 0x7fa7b806d260 2024-04-25T10:15:13.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.566+0000 7fa7adffb640 1 -- 172.21.15.16:0/3922336798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b403c460 con 0x7fa7b806d260 2024-04-25T10:15:13.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.566+0000 7fa7adffb640 1 -- 172.21.15.16:0/3922336798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4037040 con 0x7fa7b806d260 2024-04-25T10:15:13.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.566+0000 7fa7bf929640 1 -- 172.21.15.16:0/3922336798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b81e66b0 con 0x7fa7b806d260 2024-04-25T10:15:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.566+0000 7fa7bf929640 1 -- 172.21.15.16:0/3922336798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b81e6c70 con 0x7fa7b806d260 2024-04-25T10:15:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.567+0000 7fa7bf929640 1 -- 172.21.15.16:0/3922336798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa7b80750b0 con 0x7fa7b806d260 2024-04-25T10:15:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.567+0000 7fa7adffb640 1 -- 172.21.15.16:0/3922336798 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa7b403a070 con 0x7fa7b806d260 2024-04-25T10:15:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.568+0000 7fa7adffb640 1 --2- 172.21.15.16:0/3922336798 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa79c06b4d0 0x7fa79c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.568+0000 7fa7adffb640 1 -- 172.21.15.16:0/3922336798 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fa7b40aa2b0 con 0x7fa7b806d260 2024-04-25T10:15:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.568+0000 7fa7adffb640 1 --2- 172.21.15.16:0/3922336798 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x7fa79c0714d0 0x7fa79c0738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.568+0000 7fa7adffb640 1 -- 172.21.15.16:0/3922336798 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7b4008eb0 con 0x7fa79c0714d0 2024-04-25T10:15:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.568+0000 7fa7adffb640 1 -- 172.21.15.16:0/3922336798 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7b40b1050 con 0x7fa7b806d260 2024-04-25T10:15:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.568+0000 7fa7bcec4640 1 --2- 172.21.15.16:0/3922336798 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa79c06b4d0 0x7fa79c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.568+0000 7fa7bdec6640 1 --2- 172.21.15.16:0/3922336798 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x7fa79c0714d0 0x7fa79c0738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.569+0000 7fa7bcec4640 1 --2- 172.21.15.16:0/3922336798 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa79c06b4d0 0x7fa79c06d990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa7b81e7b50 tx=0x7fa7a8007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.569+0000 7fa7bdec6640 1 --2- 172.21.15.16:0/3922336798 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x7fa79c0714d0 0x7fa79c0738f0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.570+0000 7fa7adffb640 1 -- 172.21.15.16:0/3922336798 <== osd.5 v2:172.21.15.186:6800/1827108421 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa7b4008eb0 con 0x7fa79c0714d0 2024-04-25T10:15:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.583+0000 7fa7bf929640 1 -- 172.21.15.16:0/3922336798 --> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa7b81e70f0 con 0x7fa79c0714d0 2024-04-25T10:15:13.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.583+0000 7fa7adffb640 1 -- 172.21.15.16:0/3922336798 <== osd.5 v2:172.21.15.186:6800/1827108421 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa7b81e70f0 con 0x7fa79c0714d0 2024-04-25T10:15:13.587 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:15:13.588 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:15:13.588 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:15:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.584+0000 7fa7bf929640 1 -- 172.21.15.16:0/3922336798 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x7fa79c0714d0 msgr2=0x7fa79c0738f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.584+0000 7fa7bf929640 1 --2- 172.21.15.16:0/3922336798 >> [v2:172.21.15.186:6800/1827108421,v1:172.21.15.186:6801/1827108421] conn(0x7fa79c0714d0 0x7fa79c0738f0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.584+0000 7fa7bf929640 1 -- 172.21.15.16:0/3922336798 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa79c06b4d0 msgr2=0x7fa79c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.584+0000 7fa7bf929640 1 --2- 172.21.15.16:0/3922336798 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa79c06b4d0 0x7fa79c06d990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa7b81e7b50 tx=0x7fa7a8007d50 comp rx=0 tx=0).stop 2024-04-25T10:15:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.584+0000 7fa7bf929640 1 -- 172.21.15.16:0/3922336798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b806d260 msgr2=0x7fa7b81e5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.584+0000 7fa7bf929640 1 --2- 172.21.15.16:0/3922336798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b806d260 0x7fa7b81e5df0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4035040 tx=0x7fa7b40046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.584+0000 7fa7bdec6640 1 -- 172.21.15.16:0/3922336798 reap_dead start 2024-04-25T10:15:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.584+0000 7fa7bf929640 1 -- 172.21.15.16:0/3922336798 shutdown_connections 2024-04-25T10:15:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.584+0000 7fa7bf929640 1 -- 172.21.15.16:0/3922336798 >> 172.21.15.16:0/3922336798 conn(0x7fa7b806aba0 msgr2=0x7fa7b815a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.584+0000 7fa7bf929640 1 -- 172.21.15.16:0/3922336798 shutdown_connections 2024-04-25T10:15:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.584+0000 7fa7bf929640 1 -- 172.21.15.16:0/3922336798 wait complete. 2024-04-25T10:15:13.598 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T10:15:13.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.695+0000 7f558cdf9640 1 Processor -- start 2024-04-25T10:15:13.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.695+0000 7f558cdf9640 1 -- start start 2024-04-25T10:15:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.696+0000 7f558cdf9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f558815ae20 0x7f55880b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.696+0000 7f558cdf9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55880b4730 0x7f558815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.696+0000 7f558cdf9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f558806da40 con 0x7f55880aedc0 2024-04-25T10:15:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.696+0000 7f558cdf9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f558815c050 con 0x7f558815ae20 2024-04-25T10:15:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.696+0000 7f558cdf9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55880af1a0 con 0x7f55880b4730 2024-04-25T10:15:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.696+0000 7f558659c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f55880aedc0 0x7f558806d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46496/0 (socket says 172.21.15.16:46496) 2024-04-25T10:15:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.696+0000 7f558659c640 1 -- 172.21.15.16:0/3742566628 learned_addr learned my addr 172.21.15.16:0/3742566628 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.697+0000 7f5585d9b640 1 --2- 172.21.15.16:0/3742566628 >> v2:172.21.15.186:3300/0 conn(0x7f558815ae20 0x7f55880b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.697+0000 7f5586d9d640 1 --2- 172.21.15.16:0/3742566628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55880b4730 0x7f558815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.697+0000 7f558559a640 1 -- 172.21.15.16:0/3742566628 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3534659228 0 0) 0x7f558806da40 con 0x7f55880aedc0 2024-04-25T10:15:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.697+0000 7f558559a640 1 -- 172.21.15.16:0/3742566628 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5574003520 con 0x7f55880aedc0 2024-04-25T10:15:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.697+0000 7f5585d9b640 1 -- 172.21.15.16:0/3742566628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55880b4730 msgr2=0x7f558815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.697+0000 7f5585d9b640 1 --2- 172.21.15.16:0/3742566628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55880b4730 0x7f558815c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.697+0000 7f5585d9b640 1 -- 172.21.15.16:0/3742566628 >> v1:172.21.15.16:6789/0 conn(0x7f55880aedc0 legacy=0x7f558806d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.697+0000 7f558559a640 1 -- 172.21.15.16:0/3742566628 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2082286974 0 0) 0x7f5574003520 con 0x7f55880aedc0 2024-04-25T10:15:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.697+0000 7f5585d9b640 1 -- 172.21.15.16:0/3742566628 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558806dd20 con 0x7f558815ae20 2024-04-25T10:15:13.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.698+0000 7f5585d9b640 1 --2- 172.21.15.16:0/3742566628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f558815ae20 0x7f55880b41f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f55700097a0 tx=0x7f557002a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62c0d65d39717d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.698+0000 7f558559a640 1 -- 172.21.15.16:0/3742566628 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570037070 con 0x7f558815ae20 2024-04-25T10:15:13.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.698+0000 7f558559a640 1 -- 172.21.15.16:0/3742566628 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557002ab10 con 0x7f558815ae20 2024-04-25T10:15:13.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.698+0000 7f558559a640 1 -- 172.21.15.16:0/3742566628 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5580001690 con 0x7f558815ae20 2024-04-25T10:15:13.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.698+0000 7f558cdf9640 1 -- 172.21.15.16:0/3742566628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f558815ae20 msgr2=0x7f55880b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.698+0000 7f558cdf9640 1 --2- 172.21.15.16:0/3742566628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f558815ae20 0x7f55880b41f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f55700097a0 tx=0x7f557002a080 comp rx=0 tx=0).stop 2024-04-25T10:15:13.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.698+0000 7f558cdf9640 1 -- 172.21.15.16:0/3742566628 shutdown_connections 2024-04-25T10:15:13.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.698+0000 7f558cdf9640 1 --2- 172.21.15.16:0/3742566628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55880b4730 0x7f558815c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.698+0000 7f558cdf9640 1 --2- 172.21.15.16:0/3742566628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f558815ae20 0x7f55880b41f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.698+0000 7f558cdf9640 1 -- 172.21.15.16:0/3742566628 >> 172.21.15.16:0/3742566628 conn(0x7f558806aba0 msgr2=0x7f558806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.699+0000 7f558cdf9640 1 -- 172.21.15.16:0/3742566628 shutdown_connections 2024-04-25T10:15:13.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.699+0000 7f558cdf9640 1 -- 172.21.15.16:0/3742566628 wait complete. 2024-04-25T10:15:13.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.700+0000 7f558cdf9640 1 Processor -- start 2024-04-25T10:15:13.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.700+0000 7f558cdf9640 1 -- start start 2024-04-25T10:15:13.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.700+0000 7f558cdf9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55880aedc0 0x7f5588150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.700+0000 7f558cdf9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55880b4730 0x7f55880ace60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f558cdf9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f558815ae20 0x7f55880ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f558cdf9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558815cd00 con 0x7f55880b4730 2024-04-25T10:15:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f558cdf9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558815c2a0 con 0x7f558815ae20 2024-04-25T10:15:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f558cdf9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558815ce80 con 0x7f55880aedc0 2024-04-25T10:15:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f5585d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55880b4730 0x7f55880ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f5585d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55880b4730 0x7f55880ace60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43208/0 (socket says 172.21.15.16:43208) 2024-04-25T10:15:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f5585d9b640 1 -- 172.21.15.16:0/3214958709 learned_addr learned my addr 172.21.15.16:0/3214958709 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f558659c640 1 --2- 172.21.15.16:0/3214958709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55880aedc0 0x7f5588150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f5586d9d640 1 --2- 172.21.15.16:0/3214958709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f558815ae20 0x7f55880ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f5585d9b640 1 -- 172.21.15.16:0/3214958709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55880aedc0 msgr2=0x7f5588150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f5585d9b640 1 --2- 172.21.15.16:0/3214958709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55880aedc0 0x7f5588150050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f5585d9b640 1 -- 172.21.15.16:0/3214958709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f558815ae20 msgr2=0x7f55880ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f5585d9b640 1 --2- 172.21.15.16:0/3214958709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f558815ae20 0x7f55880ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f5585d9b640 1 -- 172.21.15.16:0/3214958709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55880c5550 con 0x7f55880b4730 2024-04-25T10:15:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f558659c640 1 --2- 172.21.15.16:0/3214958709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55880aedc0 0x7f5588150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f5585d9b640 1 --2- 172.21.15.16:0/3214958709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55880b4730 0x7f55880ace60 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f5570035040 tx=0x7f55700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f557effd640 1 -- 172.21.15.16:0/3214958709 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570037050 con 0x7f55880b4730 2024-04-25T10:15:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f557effd640 1 -- 172.21.15.16:0/3214958709 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5570043070 con 0x7f55880b4730 2024-04-25T10:15:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f558cdf9640 1 -- 172.21.15.16:0/3214958709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55880c5830 con 0x7f55880b4730 2024-04-25T10:15:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f557effd640 1 -- 172.21.15.16:0/3214958709 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557003ca70 con 0x7f55880b4730 2024-04-25T10:15:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.701+0000 7f558cdf9640 1 -- 172.21.15.16:0/3214958709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55880c5df0 con 0x7f55880b4730 2024-04-25T10:15:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.702+0000 7f558cdf9640 1 -- 172.21.15.16:0/3214958709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55880750b0 con 0x7f55880b4730 2024-04-25T10:15:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.703+0000 7f557effd640 1 -- 172.21.15.16:0/3214958709 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f557003cc10 con 0x7f55880b4730 2024-04-25T10:15:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.703+0000 7f557effd640 1 --2- 172.21.15.16:0/3214958709 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f556806b4d0 0x7f556806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.704+0000 7f557effd640 1 -- 172.21.15.16:0/3214958709 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f55700aaa10 con 0x7f55880b4730 2024-04-25T10:15:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.704+0000 7f558659c640 1 --2- 172.21.15.16:0/3214958709 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f556806b4d0 0x7f556806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.704+0000 7f558659c640 1 --2- 172.21.15.16:0/3214958709 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f556806b4d0 0x7f556806d990 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5580001eb0 tx=0x7f5580001d80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.708+0000 7f557effd640 1 -- 172.21.15.16:0/3214958709 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5570079c00 con 0x7f55880b4730 2024-04-25T10:15:13.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.887+0000 7f558cdf9640 1 -- 172.21.15.16:0/3214958709 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f55880cfaa0 con 0x7f556806b4d0 2024-04-25T10:15:13.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.888+0000 7f557effd640 1 -- 172.21.15.16:0/3214958709 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f55880cfaa0 con 0x7f556806b4d0 2024-04-25T10:15:13.891 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T10:15:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.890+0000 7f558cdf9640 1 -- 172.21.15.16:0/3214958709 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f556806b4d0 msgr2=0x7f556806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.890+0000 7f558cdf9640 1 --2- 172.21.15.16:0/3214958709 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f556806b4d0 0x7f556806d990 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5580001eb0 tx=0x7f5580001d80 comp rx=0 tx=0).stop 2024-04-25T10:15:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.890+0000 7f558cdf9640 1 -- 172.21.15.16:0/3214958709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55880b4730 msgr2=0x7f55880ace60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.890+0000 7f558cdf9640 1 --2- 172.21.15.16:0/3214958709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55880b4730 0x7f55880ace60 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f5570035040 tx=0x7f55700046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.890+0000 7f558cdf9640 1 -- 172.21.15.16:0/3214958709 shutdown_connections 2024-04-25T10:15:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.890+0000 7f558cdf9640 1 --2- 172.21.15.16:0/3214958709 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f556806b4d0 0x7f556806d990 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.890+0000 7f558cdf9640 1 --2- 172.21.15.16:0/3214958709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f558815ae20 0x7f55880ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.890+0000 7f558cdf9640 1 --2- 172.21.15.16:0/3214958709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55880b4730 0x7f55880ace60 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.890+0000 7f558cdf9640 1 --2- 172.21.15.16:0/3214958709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55880aedc0 0x7f5588150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.890+0000 7f558cdf9640 1 -- 172.21.15.16:0/3214958709 >> 172.21.15.16:0/3214958709 conn(0x7f558806aba0 msgr2=0x7f558806d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.890+0000 7f558cdf9640 1 -- 172.21.15.16:0/3214958709 shutdown_connections 2024-04-25T10:15:13.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:13.890+0000 7f558cdf9640 1 -- 172.21.15.16:0/3214958709 wait complete. 2024-04-25T10:15:13.904 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T10:15:13.904 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:15:14.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.005+0000 7f2a22128640 1 Processor -- start 2024-04-25T10:15:14.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.005+0000 7f2a22128640 1 -- start start 2024-04-25T10:15:14.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.006+0000 7f2a22128640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2a1c0ceeb0 0x7f2a1c168a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.006+0000 7f2a22128640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a1c161f80 con 0x7f2a1c161090 2024-04-25T10:15:14.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.006+0000 7f2a22128640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a1c162260 con 0x7f2a1c0b9720 2024-04-25T10:15:14.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.006+0000 7f2a22128640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a1c0d21a0 con 0x7f2a1c0ceeb0 2024-04-25T10:15:14.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.006+0000 7f2a1bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2a1c0ceeb0 0x7f2a1c168a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.006+0000 7f2a1b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2a1c161090 0x7f2a1c161730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46506/0 (socket says 172.21.15.16:46506) 2024-04-25T10:15:14.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.006+0000 7f2a1b7fe640 1 -- 172.21.15.16:0/2051381844 learned_addr learned my addr 172.21.15.16:0/2051381844 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:14.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.007+0000 7f2a1bfff640 1 -- 172.21.15.16:0/2051381844 >> v1:172.21.15.186:6789/0 conn(0x7f2a1c0b9720 legacy=0x7f2a1c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.007+0000 7f2a1bfff640 1 -- 172.21.15.16:0/2051381844 >> v1:172.21.15.16:6789/0 conn(0x7f2a1c161090 legacy=0x7f2a1c161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.007+0000 7f2a1a7fc640 1 -- 172.21.15.16:0/2051381844 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1633618658 0 0) 0x7f2a1c161f80 con 0x7f2a1c161090 2024-04-25T10:15:14.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.007+0000 7f2a1bfff640 1 -- 172.21.15.16:0/2051381844 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1c162540 con 0x7f2a1c0ceeb0 2024-04-25T10:15:14.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.007+0000 7f2a1bfff640 1 --2- 172.21.15.16:0/2051381844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a1c0ceeb0 0x7f2a1c168a60 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0d2320 tx=0x7f2a1402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e742654bd31fb0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.007+0000 7f2a1a7fc640 1 -- 172.21.15.16:0/2051381844 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a14007740 con 0x7f2a1c0ceeb0 2024-04-25T10:15:14.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.007+0000 7f2a1a7fc640 1 -- 172.21.15.16:0/2051381844 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a140078e0 con 0x7f2a1c0ceeb0 2024-04-25T10:15:14.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.007+0000 7f2a1a7fc640 1 -- 172.21.15.16:0/2051381844 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a14007cb0 con 0x7f2a1c0ceeb0 2024-04-25T10:15:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.008+0000 7f2a22128640 1 -- 172.21.15.16:0/2051381844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a1c0ceeb0 msgr2=0x7f2a1c168a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.008+0000 7f2a22128640 1 --2- 172.21.15.16:0/2051381844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a1c0ceeb0 0x7f2a1c168a60 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0d2320 tx=0x7f2a1402a660 comp rx=0 tx=0).stop 2024-04-25T10:15:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.008+0000 7f2a22128640 1 -- 172.21.15.16:0/2051381844 shutdown_connections 2024-04-25T10:15:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.008+0000 7f2a22128640 1 --2- 172.21.15.16:0/2051381844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a1c0ceeb0 0x7f2a1c168a60 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.008+0000 7f2a22128640 1 -- 172.21.15.16:0/2051381844 >> 172.21.15.16:0/2051381844 conn(0x7f2a1c0b55e0 msgr2=0x7f2a1c0b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.008+0000 7f2a22128640 1 -- 172.21.15.16:0/2051381844 shutdown_connections 2024-04-25T10:15:14.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.008+0000 7f2a22128640 1 -- 172.21.15.16:0/2051381844 wait complete. 2024-04-25T10:15:14.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.009+0000 7f2a22128640 1 Processor -- start 2024-04-25T10:15:14.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.009+0000 7f2a22128640 1 -- start start 2024-04-25T10:15:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.010+0000 7f2a22128640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a1c0b9720 0x7f2a1c1f0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.010+0000 7f2a22128640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a1c0ceeb0 0x7f2a1c1f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.010+0000 7f2a22128640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1c161090 0x7f2a1c0c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.010+0000 7f2a22128640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c0d3640 con 0x7f2a1c0b9720 2024-04-25T10:15:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.010+0000 7f2a22128640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c161470 con 0x7f2a1c161090 2024-04-25T10:15:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.010+0000 7f2a22128640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c0d37c0 con 0x7f2a1c0ceeb0 2024-04-25T10:15:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.010+0000 7f2a1b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a1c0b9720 0x7f2a1c1f0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.010+0000 7f2a1b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a1c0b9720 0x7f2a1c1f0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43222/0 (socket says 172.21.15.16:43222) 2024-04-25T10:15:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.010+0000 7f2a1b7fe640 1 -- 172.21.15.16:0/3965503701 learned_addr learned my addr 172.21.15.16:0/3965503701 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.010+0000 7f2a1b7fe640 1 -- 172.21.15.16:0/3965503701 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a1c0ceeb0 msgr2=0x7f2a1c1f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.010+0000 7f2a1b7fe640 1 --2- 172.21.15.16:0/3965503701 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a1c0ceeb0 0x7f2a1c1f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.010+0000 7f2a1b7fe640 1 -- 172.21.15.16:0/3965503701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1c161090 msgr2=0x7f2a1c0c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.010+0000 7f2a1b7fe640 1 --2- 172.21.15.16:0/3965503701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1c161090 0x7f2a1c0c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.010+0000 7f2a1b7fe640 1 -- 172.21.15.16:0/3965503701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1c1f6fb0 con 0x7f2a1c0b9720 2024-04-25T10:15:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.011+0000 7f2a1b7fe640 1 --2- 172.21.15.16:0/3965503701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a1c0b9720 0x7f2a1c1f0d10 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00c4d0 tx=0x7f2a0c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.011+0000 7f2a03fff640 1 -- 172.21.15.16:0/3965503701 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c018070 con 0x7f2a1c0b9720 2024-04-25T10:15:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.011+0000 7f2a03fff640 1 -- 172.21.15.16:0/3965503701 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0c0074e0 con 0x7f2a1c0b9720 2024-04-25T10:15:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.011+0000 7f2a22128640 1 -- 172.21.15.16:0/3965503701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1c1f7290 con 0x7f2a1c0b9720 2024-04-25T10:15:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.011+0000 7f2a03fff640 1 -- 172.21.15.16:0/3965503701 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c004a20 con 0x7f2a1c0b9720 2024-04-25T10:15:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.011+0000 7f2a22128640 1 -- 172.21.15.16:0/3965503701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1c15ee00 con 0x7f2a1c0b9720 2024-04-25T10:15:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.012+0000 7f2a22128640 1 -- 172.21.15.16:0/3965503701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2a1c1f6d60 con 0x7f2a1c0b9720 2024-04-25T10:15:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.012+0000 7f2a03fff640 1 -- 172.21.15.16:0/3965503701 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a0c01b070 con 0x7f2a1c0b9720 2024-04-25T10:15:14.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.013+0000 7f2a03fff640 1 --2- 172.21.15.16:0/3965503701 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2a0806b500 0x7f2a0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.013+0000 7f2a03fff640 1 -- 172.21.15.16:0/3965503701 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f2a0c089820 con 0x7f2a1c0b9720 2024-04-25T10:15:14.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.013+0000 7f2a03fff640 1 --2- 172.21.15.16:0/3965503701 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x7f2a08071500 0x7f2a08073920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.013+0000 7f2a03fff640 1 -- 172.21.15.16:0/3965503701 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2a08073fd0 con 0x7f2a08071500 2024-04-25T10:15:14.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.013+0000 7f2a03fff640 1 -- 172.21.15.16:0/3965503701 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a0c089cb0 con 0x7f2a1c0b9720 2024-04-25T10:15:14.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.013+0000 7f2a1affd640 1 --2- 172.21.15.16:0/3965503701 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2a0806b500 0x7f2a0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.013+0000 7f2a1bfff640 1 --2- 172.21.15.16:0/3965503701 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x7f2a08071500 0x7f2a08073920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.014+0000 7f2a1affd640 1 --2- 172.21.15.16:0/3965503701 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2a0806b500 0x7f2a0806d9c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2a10000f60 tx=0x7f2a100079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.014+0000 7f2a1bfff640 1 --2- 172.21.15.16:0/3965503701 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x7f2a08071500 0x7f2a08073920 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.014+0000 7f2a03fff640 1 -- 172.21.15.16:0/3965503701 <== osd.6 v2:172.21.15.186:6816/1123185664 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2a08073fd0 con 0x7f2a08071500 2024-04-25T10:15:14.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.028+0000 7f2a22128640 1 -- 172.21.15.16:0/3965503701 --> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2a1c0750b0 con 0x7f2a08071500 2024-04-25T10:15:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.028+0000 7f2a03fff640 1 -- 172.21.15.16:0/3965503701 <== osd.6 v2:172.21.15.186:6816/1123185664 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2a1c0750b0 con 0x7f2a08071500 2024-04-25T10:15:14.032 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:15:14.032 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:15:14.032 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:15:14.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.028+0000 7f2a22128640 1 -- 172.21.15.16:0/3965503701 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x7f2a08071500 msgr2=0x7f2a08073920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.028+0000 7f2a22128640 1 --2- 172.21.15.16:0/3965503701 >> [v2:172.21.15.186:6816/1123185664,v1:172.21.15.186:6817/1123185664] conn(0x7f2a08071500 0x7f2a08073920 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.028+0000 7f2a22128640 1 -- 172.21.15.16:0/3965503701 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2a0806b500 msgr2=0x7f2a0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.028+0000 7f2a22128640 1 --2- 172.21.15.16:0/3965503701 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2a0806b500 0x7f2a0806d9c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2a10000f60 tx=0x7f2a100079b0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.028+0000 7f2a22128640 1 -- 172.21.15.16:0/3965503701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a1c0b9720 msgr2=0x7f2a1c1f0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.028+0000 7f2a22128640 1 --2- 172.21.15.16:0/3965503701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a1c0b9720 0x7f2a1c1f0d10 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00c4d0 tx=0x7f2a0c00c990 comp rx=0 tx=0).stop 2024-04-25T10:15:14.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.029+0000 7f2a1bfff640 1 -- 172.21.15.16:0/3965503701 reap_dead start 2024-04-25T10:15:14.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.029+0000 7f2a22128640 1 -- 172.21.15.16:0/3965503701 shutdown_connections 2024-04-25T10:15:14.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.029+0000 7f2a22128640 1 -- 172.21.15.16:0/3965503701 >> 172.21.15.16:0/3965503701 conn(0x7f2a1c0b55e0 msgr2=0x7f2a1c0d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.029+0000 7f2a22128640 1 -- 172.21.15.16:0/3965503701 shutdown_connections 2024-04-25T10:15:14.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.029+0000 7f2a22128640 1 -- 172.21.15.16:0/3965503701 wait complete. 2024-04-25T10:15:14.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T10:15:14.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.140+0000 7f773bfa5640 1 Processor -- start 2024-04-25T10:15:14.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.141+0000 7f773bfa5640 1 -- start start 2024-04-25T10:15:14.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.141+0000 7f773bfa5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77340ceeb0 0x7f7734168a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.141+0000 7f773bfa5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7734161f80 con 0x7f7734161090 2024-04-25T10:15:14.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.141+0000 7f773bfa5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7734162260 con 0x7f77340b9720 2024-04-25T10:15:14.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.141+0000 7f773bfa5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77340d21a0 con 0x7f77340ceeb0 2024-04-25T10:15:14.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.141+0000 7f7739d41640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7734161090 0x7f7734161730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46512/0 (socket says 172.21.15.16:46512) 2024-04-25T10:15:14.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7739d41640 1 -- 172.21.15.16:0/778697326 learned_addr learned my addr 172.21.15.16:0/778697326 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f773a542640 1 --2- 172.21.15.16:0/778697326 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77340ceeb0 0x7f7734168a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7738d3f640 1 -- 172.21.15.16:0/778697326 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093052232 0 0) 0x7f7734161f80 con 0x7f7734161090 2024-04-25T10:15:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7738d3f640 1 -- 172.21.15.16:0/778697326 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f771c003520 con 0x7f7734161090 2024-04-25T10:15:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7738d3f640 1 -- 172.21.15.16:0/778697326 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256162288 0 0) 0x7f7734162260 con 0x7f77340b9720 2024-04-25T10:15:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7738d3f640 1 -- 172.21.15.16:0/778697326 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7734161f80 con 0x7f77340b9720 2024-04-25T10:15:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7738d3f640 1 -- 172.21.15.16:0/778697326 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2605487307 0 0) 0x7f771c003520 con 0x7f7734161090 2024-04-25T10:15:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7738d3f640 1 -- 172.21.15.16:0/778697326 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77340ceeb0 msgr2=0x7f7734168a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7738d3f640 1 --2- 172.21.15.16:0/778697326 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77340ceeb0 0x7f7734168a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7738d3f640 1 -- 172.21.15.16:0/778697326 >> v1:172.21.15.186:6789/0 conn(0x7f77340b9720 legacy=0x7f77340ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7738d3f640 1 -- 172.21.15.16:0/778697326 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7734162540 con 0x7f7734161090 2024-04-25T10:15:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7738d3f640 1 -- 172.21.15.16:0/778697326 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 413428187 0 0) 0x7f772c001690 con 0x7f7734161090 2024-04-25T10:15:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7738d3f640 1 -- 172.21.15.16:0/778697326 >> v1:172.21.15.16:6789/0 conn(0x7f7734161090 legacy=0x7f7734161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7738d3f640 1 --2- 172.21.15.16:0/778697326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f771c003b70 0x7f771c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7738d3f640 1 --2- 172.21.15.16:0/778697326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f771c004800 0x7f771c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7738d3f640 1 --2- 172.21.15.16:0/778697326 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77340ceeb0 0x7f771c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7738d3f640 1 -- 172.21.15.16:0/778697326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77340d21a0 con 0x7f771c004800 2024-04-25T10:15:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7738d3f640 1 -- 172.21.15.16:0/778697326 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c00bb50 con 0x7f771c003b70 2024-04-25T10:15:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7738d3f640 1 -- 172.21.15.16:0/778697326 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771c00bd50 con 0x7f77340ceeb0 2024-04-25T10:15:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7739540640 1 --2- 172.21.15.16:0/778697326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f771c004800 0x7f771c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f7739d41640 1 --2- 172.21.15.16:0/778697326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f771c003b70 0x7f771c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.142+0000 7f773a542640 1 --2- 172.21.15.16:0/778697326 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77340ceeb0 0x7f771c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.143+0000 7f7739540640 1 -- 172.21.15.16:0/778697326 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77340ceeb0 msgr2=0x7f771c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.143+0000 7f7739540640 1 --2- 172.21.15.16:0/778697326 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77340ceeb0 0x7f771c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.143+0000 7f7739540640 1 -- 172.21.15.16:0/778697326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f771c003b70 msgr2=0x7f771c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.143+0000 7f7739540640 1 --2- 172.21.15.16:0/778697326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f771c003b70 0x7f771c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.143+0000 7f7739540640 1 -- 172.21.15.16:0/778697326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771c003520 con 0x7f771c004800 2024-04-25T10:15:14.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.143+0000 7f7739540640 1 --2- 172.21.15.16:0/778697326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f771c004800 0x7f771c004c80 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f77280018d0 tx=0x7f771c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92196523ba1b09d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.143+0000 7f7738d3f640 1 -- 172.21.15.16:0/778697326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77280014f0 con 0x7f771c004800 2024-04-25T10:15:14.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.143+0000 7f7738d3f640 1 -- 172.21.15.16:0/778697326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77280052d0 con 0x7f771c004800 2024-04-25T10:15:14.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.143+0000 7f7738d3f640 1 -- 172.21.15.16:0/778697326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772802aca0 con 0x7f771c004800 2024-04-25T10:15:14.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.144+0000 7f773bfa5640 1 -- 172.21.15.16:0/778697326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f771c004800 msgr2=0x7f771c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.144+0000 7f773bfa5640 1 --2- 172.21.15.16:0/778697326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f771c004800 0x7f771c004c80 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f77280018d0 tx=0x7f771c005830 comp rx=0 tx=0).stop 2024-04-25T10:15:14.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.144+0000 7f773a542640 1 -- 172.21.15.16:0/778697326 reap_dead start 2024-04-25T10:15:14.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.144+0000 7f773bfa5640 1 -- 172.21.15.16:0/778697326 shutdown_connections 2024-04-25T10:15:14.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.144+0000 7f773bfa5640 1 -- 172.21.15.16:0/778697326 >> 172.21.15.16:0/778697326 conn(0x7f77340b55e0 msgr2=0x7f77340b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.144+0000 7f773bfa5640 1 -- 172.21.15.16:0/778697326 shutdown_connections 2024-04-25T10:15:14.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.144+0000 7f773bfa5640 1 -- 172.21.15.16:0/778697326 wait complete. 2024-04-25T10:15:14.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.145+0000 7f773bfa5640 1 Processor -- start 2024-04-25T10:15:14.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.145+0000 7f773bfa5640 1 -- start start 2024-04-25T10:15:14.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.146+0000 7f773bfa5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f771c004800 0x7f77340ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.146+0000 7f773bfa5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f773415f5f0 0x7f773415dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.146+0000 7f773bfa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f773415e510 0x7f77341f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.146+0000 7f773bfa5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77340d3640 con 0x7f773415e510 2024-04-25T10:15:14.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.146+0000 7f773bfa5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7734161470 con 0x7f771c004800 2024-04-25T10:15:14.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.146+0000 7f773bfa5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77340d37c0 con 0x7f773415f5f0 2024-04-25T10:15:14.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.146+0000 7f773a542640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f773415e510 0x7f77341f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.146+0000 7f773a542640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f773415e510 0x7f77341f7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43228/0 (socket says 172.21.15.16:43228) 2024-04-25T10:15:14.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.146+0000 7f773a542640 1 -- 172.21.15.16:0/2214712468 learned_addr learned my addr 172.21.15.16:0/2214712468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:14.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.146+0000 7f7739d41640 1 --2- 172.21.15.16:0/2214712468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f771c004800 0x7f77340ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.146+0000 7f7739540640 1 --2- 172.21.15.16:0/2214712468 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f773415f5f0 0x7f773415dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.147+0000 7f773a542640 1 -- 172.21.15.16:0/2214712468 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f773415f5f0 msgr2=0x7f773415dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.147+0000 7f773a542640 1 --2- 172.21.15.16:0/2214712468 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f773415f5f0 0x7f773415dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.147+0000 7f773a542640 1 -- 172.21.15.16:0/2214712468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f771c004800 msgr2=0x7f77340ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.147+0000 7f773a542640 1 --2- 172.21.15.16:0/2214712468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f771c004800 0x7f77340ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.147+0000 7f773a542640 1 -- 172.21.15.16:0/2214712468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773415cf30 con 0x7f773415e510 2024-04-25T10:15:14.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.147+0000 7f773a542640 1 --2- 172.21.15.16:0/2214712468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f773415e510 0x7f77341f7340 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f77340cab30 tx=0x7f77240115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.147+0000 7f77327fc640 1 -- 172.21.15.16:0/2214712468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724003990 con 0x7f773415e510 2024-04-25T10:15:14.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.147+0000 7f77327fc640 1 -- 172.21.15.16:0/2214712468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772401b070 con 0x7f773415e510 2024-04-25T10:15:14.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.147+0000 7f773bfa5640 1 -- 172.21.15.16:0/2214712468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773415d210 con 0x7f773415e510 2024-04-25T10:15:14.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.147+0000 7f77327fc640 1 -- 172.21.15.16:0/2214712468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724003b70 con 0x7f773415e510 2024-04-25T10:15:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.147+0000 7f773bfa5640 1 -- 172.21.15.16:0/2214712468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773415d7d0 con 0x7f773415e510 2024-04-25T10:15:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.148+0000 7f773bfa5640 1 -- 172.21.15.16:0/2214712468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77340750b0 con 0x7f773415e510 2024-04-25T10:15:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.148+0000 7f77327fc640 1 -- 172.21.15.16:0/2214712468 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7724005d10 con 0x7f773415e510 2024-04-25T10:15:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.149+0000 7f77327fc640 1 --2- 172.21.15.16:0/2214712468 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f772006b4d0 0x7f772006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.149+0000 7f77327fc640 1 -- 172.21.15.16:0/2214712468 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f772408a3b0 con 0x7f773415e510 2024-04-25T10:15:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.149+0000 7f7739d41640 1 --2- 172.21.15.16:0/2214712468 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f772006b4d0 0x7f772006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.150+0000 7f7739d41640 1 --2- 172.21.15.16:0/2214712468 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f772006b4d0 0x7f772006d990 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f772c001220 tx=0x7f772c007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.153+0000 7f77327fc640 1 -- 172.21.15.16:0/2214712468 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7724025020 con 0x7f773415e510 2024-04-25T10:15:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.334+0000 7f773bfa5640 1 -- 172.21.15.16:0/2214712468 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f77340b3db0 con 0x7f772006b4d0 2024-04-25T10:15:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.336+0000 7f77327fc640 1 -- 172.21.15.16:0/2214712468 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f77340b3db0 con 0x7f772006b4d0 2024-04-25T10:15:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T10:15:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.337+0000 7f773bfa5640 1 -- 172.21.15.16:0/2214712468 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f772006b4d0 msgr2=0x7f772006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.337+0000 7f773bfa5640 1 --2- 172.21.15.16:0/2214712468 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f772006b4d0 0x7f772006d990 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f772c001220 tx=0x7f772c007040 comp rx=0 tx=0).stop 2024-04-25T10:15:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.337+0000 7f773bfa5640 1 -- 172.21.15.16:0/2214712468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f773415e510 msgr2=0x7f77341f7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.337+0000 7f773bfa5640 1 --2- 172.21.15.16:0/2214712468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f773415e510 0x7f77341f7340 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f77340cab30 tx=0x7f77240115a0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.337+0000 7f773bfa5640 1 -- 172.21.15.16:0/2214712468 shutdown_connections 2024-04-25T10:15:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.337+0000 7f773bfa5640 1 --2- 172.21.15.16:0/2214712468 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f772006b4d0 0x7f772006d990 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.337+0000 7f773bfa5640 1 --2- 172.21.15.16:0/2214712468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f773415e510 0x7f77341f7340 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.337+0000 7f773bfa5640 1 --2- 172.21.15.16:0/2214712468 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f773415f5f0 0x7f773415dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.337+0000 7f773bfa5640 1 --2- 172.21.15.16:0/2214712468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f771c004800 0x7f77340ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.337+0000 7f773bfa5640 1 -- 172.21.15.16:0/2214712468 >> 172.21.15.16:0/2214712468 conn(0x7f77340b55e0 msgr2=0x7f77340b6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.337+0000 7f773bfa5640 1 -- 172.21.15.16:0/2214712468 shutdown_connections 2024-04-25T10:15:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.337+0000 7f773bfa5640 1 -- 172.21.15.16:0/2214712468 wait complete. 2024-04-25T10:15:14.352 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T10:15:14.352 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:15:14.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.453+0000 7f2ec4fd5640 1 Processor -- start 2024-04-25T10:15:14.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.453+0000 7f2ec4fd5640 1 -- start start 2024-04-25T10:15:14.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.453+0000 7f2ec4fd5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ec01610b0 0x7f2ec00b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.453+0000 7f2ec4fd5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2ec01618d0 0x7f2ec00ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.453+0000 7f2ec4fd5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec0162040 con 0x7f2ec00ced70 2024-04-25T10:15:14.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.453+0000 7f2ec4fd5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec00d1ea0 con 0x7f2ec01610b0 2024-04-25T10:15:14.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.453+0000 7f2ec4fd5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec0162320 con 0x7f2ec01618d0 2024-04-25T10:15:14.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.454+0000 7f2ebe59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ec01610b0 0x7f2ec00b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.454+0000 7f2ebe59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ec01610b0 0x7f2ec00b9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43232/0 (socket says 172.21.15.16:43232) 2024-04-25T10:15:14.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.454+0000 7f2ebe59c640 1 -- 172.21.15.16:0/3954833475 learned_addr learned my addr 172.21.15.16:0/3954833475 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:14.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.454+0000 7f2eb7fff640 1 --2- 172.21.15.16:0/3954833475 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2ec01618d0 0x7f2ec00ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.454+0000 7f2ebe59c640 1 -- 172.21.15.16:0/3954833475 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2ec01618d0 msgr2=0x7f2ec00ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.454+0000 7f2ebe59c640 1 --2- 172.21.15.16:0/3954833475 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2ec01618d0 0x7f2ec00ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.454+0000 7f2ebe59c640 1 -- 172.21.15.16:0/3954833475 >> v1:172.21.15.186:6789/0 conn(0x7f2ec00ced70 legacy=0x7f2ec006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.454+0000 7f2ebe59c640 1 -- 172.21.15.16:0/3954833475 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec0162600 con 0x7f2ec01610b0 2024-04-25T10:15:14.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.455+0000 7f2ebe59c640 1 --2- 172.21.15.16:0/3954833475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec01610b0 0x7f2ec00b9580 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009870 tx=0x7f2eb002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86c07acd72b2d7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.455+0000 7f2ebdd9b640 1 -- 172.21.15.16:0/3954833475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0037070 con 0x7f2ec01610b0 2024-04-25T10:15:14.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.455+0000 7f2ebdd9b640 1 -- 172.21.15.16:0/3954833475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb0032440 con 0x7f2ec01610b0 2024-04-25T10:15:14.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.455+0000 7f2ebdd9b640 1 -- 172.21.15.16:0/3954833475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0032810 con 0x7f2ec01610b0 2024-04-25T10:15:14.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.455+0000 7f2ec4fd5640 1 -- 172.21.15.16:0/3954833475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec01610b0 msgr2=0x7f2ec00b9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.455+0000 7f2ec4fd5640 1 --2- 172.21.15.16:0/3954833475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec01610b0 0x7f2ec00b9580 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009870 tx=0x7f2eb002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.455+0000 7f2ec4fd5640 1 -- 172.21.15.16:0/3954833475 shutdown_connections 2024-04-25T10:15:14.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.455+0000 7f2ec4fd5640 1 --2- 172.21.15.16:0/3954833475 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2ec01618d0 0x7f2ec00ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.455+0000 7f2ec4fd5640 1 --2- 172.21.15.16:0/3954833475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec01610b0 0x7f2ec00b9580 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.455+0000 7f2ec4fd5640 1 -- 172.21.15.16:0/3954833475 >> 172.21.15.16:0/3954833475 conn(0x7f2ec00b5570 msgr2=0x7f2ec00b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.456+0000 7f2ec4fd5640 1 -- 172.21.15.16:0/3954833475 shutdown_connections 2024-04-25T10:15:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.456+0000 7f2ec4fd5640 1 -- 172.21.15.16:0/3954833475 wait complete. 2024-04-25T10:15:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.457+0000 7f2ec4fd5640 1 Processor -- start 2024-04-25T10:15:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.457+0000 7f2ec4fd5640 1 -- start start 2024-04-25T10:15:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2ec4fd5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ec00ced70 0x7f2ec0141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2ec4fd5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2ec01610b0 0x7f2ec01f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2ec4fd5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec01618d0 0x7f2ec00b7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2ec4fd5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec0161d10 con 0x7f2ec01618d0 2024-04-25T10:15:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2ec4fd5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec00b6c00 con 0x7f2ec00ced70 2024-04-25T10:15:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2ec4fd5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec00d3620 con 0x7f2ec01610b0 2024-04-25T10:15:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2ebed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec01618d0 0x7f2ec00b7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2ebed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec01618d0 0x7f2ec00b7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43246/0 (socket says 172.21.15.16:43246) 2024-04-25T10:15:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2ebed9d640 1 -- 172.21.15.16:0/649866129 learned_addr learned my addr 172.21.15.16:0/649866129 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2eb7fff640 1 --2- 172.21.15.16:0/649866129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2ec01610b0 0x7f2ec01f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2ebe59c640 1 --2- 172.21.15.16:0/649866129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ec00ced70 0x7f2ec0141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2ebed9d640 1 -- 172.21.15.16:0/649866129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2ec01610b0 msgr2=0x7f2ec01f7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2ebed9d640 1 --2- 172.21.15.16:0/649866129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2ec01610b0 0x7f2ec01f7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2ebed9d640 1 -- 172.21.15.16:0/649866129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ec00ced70 msgr2=0x7f2ec0141ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2ebed9d640 1 --2- 172.21.15.16:0/649866129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ec00ced70 0x7f2ec0141ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2ebed9d640 1 -- 172.21.15.16:0/649866129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec01f8530 con 0x7f2ec01618d0 2024-04-25T10:15:14.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2ebed9d640 1 --2- 172.21.15.16:0/649866129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec01618d0 0x7f2ec00b7540 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800c6b0 tx=0x7f2eb800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/649866129 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8003b70 con 0x7f2ec01618d0 2024-04-25T10:15:14.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/649866129 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb8003d30 con 0x7f2ec01618d0 2024-04-25T10:15:14.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/649866129 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8005310 con 0x7f2ec01618d0 2024-04-25T10:15:14.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2ec4fd5640 1 -- 172.21.15.16:0/649866129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec014f2f0 con 0x7f2ec01618d0 2024-04-25T10:15:14.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.458+0000 7f2ec4fd5640 1 -- 172.21.15.16:0/649866129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec014f8b0 con 0x7f2ec01618d0 2024-04-25T10:15:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.459+0000 7f2ec4fd5640 1 -- 172.21.15.16:0/649866129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2ec01420e0 con 0x7f2ec01618d0 2024-04-25T10:15:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.460+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/649866129 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2eb80054b0 con 0x7f2ec01618d0 2024-04-25T10:15:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.460+0000 7f2eb6ffd640 1 --2- 172.21.15.16:0/649866129 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ea006b4d0 0x7f2ea006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.460+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/649866129 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f2eb808a6b0 con 0x7f2ec01618d0 2024-04-25T10:15:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.461+0000 7f2eb6ffd640 1 --2- 172.21.15.16:0/649866129 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x7f2ea00714d0 0x7f2ea00738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.461+0000 7f2ebe59c640 1 --2- 172.21.15.16:0/649866129 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ea006b4d0 0x7f2ea006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.461+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/649866129 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ea0073fa0 con 0x7f2ea00714d0 2024-04-25T10:15:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.461+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/649866129 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2eb808aa60 con 0x7f2ec01618d0 2024-04-25T10:15:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.461+0000 7f2eb7fff640 1 --2- 172.21.15.16:0/649866129 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x7f2ea00714d0 0x7f2ea00738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.461+0000 7f2ebe59c640 1 --2- 172.21.15.16:0/649866129 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ea006b4d0 0x7f2ea006d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0034000 tx=0x7f2eb002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.461+0000 7f2eb7fff640 1 --2- 172.21.15.16:0/649866129 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x7f2ea00714d0 0x7f2ea00738f0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.462+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/649866129 <== osd.7 v2:172.21.15.186:6808/1627307043 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2ea0073fa0 con 0x7f2ea00714d0 2024-04-25T10:15:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.475+0000 7f2ec4fd5640 1 -- 172.21.15.16:0/649866129 --> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2ec014ef80 con 0x7f2ea00714d0 2024-04-25T10:15:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.476+0000 7f2eb6ffd640 1 -- 172.21.15.16:0/649866129 <== osd.7 v2:172.21.15.186:6808/1627307043 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2ec014ef80 con 0x7f2ea00714d0 2024-04-25T10:15:14.480 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-25T10:15:14.480 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:15:14.480 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.476+0000 7f2ec4fd5640 1 -- 172.21.15.16:0/649866129 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x7f2ea00714d0 msgr2=0x7f2ea00738f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.476+0000 7f2ec4fd5640 1 --2- 172.21.15.16:0/649866129 >> [v2:172.21.15.186:6808/1627307043,v1:172.21.15.186:6809/1627307043] conn(0x7f2ea00714d0 0x7f2ea00738f0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.476+0000 7f2ec4fd5640 1 -- 172.21.15.16:0/649866129 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ea006b4d0 msgr2=0x7f2ea006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.476+0000 7f2ec4fd5640 1 --2- 172.21.15.16:0/649866129 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f2ea006b4d0 0x7f2ea006d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0034000 tx=0x7f2eb002e000 comp rx=0 tx=0).stop 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.476+0000 7f2ec4fd5640 1 -- 172.21.15.16:0/649866129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec01618d0 msgr2=0x7f2ec00b7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.476+0000 7f2ec4fd5640 1 --2- 172.21.15.16:0/649866129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec01618d0 0x7f2ec00b7540 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800c6b0 tx=0x7f2eb800cb70 comp rx=0 tx=0).stop 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.476+0000 7f2ebed9d640 1 -- 172.21.15.16:0/649866129 reap_dead start 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.476+0000 7f2ec4fd5640 1 -- 172.21.15.16:0/649866129 shutdown_connections 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.476+0000 7f2ec4fd5640 1 -- 172.21.15.16:0/649866129 >> 172.21.15.16:0/649866129 conn(0x7f2ec00b5570 msgr2=0x7f2ec00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.476+0000 7f2ec4fd5640 1 -- 172.21.15.16:0/649866129 shutdown_connections 2024-04-25T10:15:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.476+0000 7f2ec4fd5640 1 -- 172.21.15.16:0/649866129 wait complete. 2024-04-25T10:15:14.491 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T10:15:14.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.589+0000 7fa16dab3640 1 Processor -- start 2024-04-25T10:15:14.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.589+0000 7fa16dab3640 1 -- start start 2024-04-25T10:15:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.590+0000 7fa16dab3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa1681610b0 0x7fa1680b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.590+0000 7fa16dab3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1680cef40 0x7fa1680d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.590+0000 7fa16dab3640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa168162040 con 0x7fa1681618d0 2024-04-25T10:15:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.590+0000 7fa16dab3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1680d1ea0 con 0x7fa1680cef40 2024-04-25T10:15:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.590+0000 7fa16dab3640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa168162320 con 0x7fa1681610b0 2024-04-25T10:15:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.590+0000 7fa167fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1680cef40 0x7fa1680d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.590+0000 7fa167fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1680cef40 0x7fa1680d2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43260/0 (socket says 172.21.15.16:43260) 2024-04-25T10:15:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.590+0000 7fa167fff640 1 -- 172.21.15.16:0/2468696221 learned_addr learned my addr 172.21.15.16:0/2468696221 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.590+0000 7fa1677fe640 1 --2- 172.21.15.16:0/2468696221 >> v2:172.21.15.186:3300/0 conn(0x7fa1681610b0 0x7fa1680b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.590+0000 7fa167fff640 1 -- 172.21.15.16:0/2468696221 >> v2:172.21.15.186:3300/0 conn(0x7fa1681610b0 msgr2=0x7fa1680b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.590+0000 7fa167fff640 1 --2- 172.21.15.16:0/2468696221 >> v2:172.21.15.186:3300/0 conn(0x7fa1681610b0 0x7fa1680b9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.590+0000 7fa167fff640 1 -- 172.21.15.16:0/2468696221 >> v1:172.21.15.186:6789/0 conn(0x7fa1681618d0 legacy=0x7fa1680ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.590+0000 7fa167fff640 1 -- 172.21.15.16:0/2468696221 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa168162600 con 0x7fa1680cef40 2024-04-25T10:15:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.591+0000 7fa167fff640 1 --2- 172.21.15.16:0/2468696221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1680cef40 0x7fa1680d2610 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fa16808c410 tx=0x7fa16002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844940d5395c0fce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.591+0000 7fa1667fc640 1 -- 172.21.15.16:0/2468696221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160036070 con 0x7fa1680cef40 2024-04-25T10:15:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.591+0000 7fa1667fc640 1 -- 172.21.15.16:0/2468696221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa160031600 con 0x7fa1680cef40 2024-04-25T10:15:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.591+0000 7fa1667fc640 1 -- 172.21.15.16:0/2468696221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1600319d0 con 0x7fa1680cef40 2024-04-25T10:15:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.592+0000 7fa16dab3640 1 -- 172.21.15.16:0/2468696221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1680cef40 msgr2=0x7fa1680d2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.592+0000 7fa16dab3640 1 --2- 172.21.15.16:0/2468696221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1680cef40 0x7fa1680d2610 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fa16808c410 tx=0x7fa16002a660 comp rx=0 tx=0).stop 2024-04-25T10:15:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.592+0000 7fa16dab3640 1 -- 172.21.15.16:0/2468696221 shutdown_connections 2024-04-25T10:15:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.592+0000 7fa16dab3640 1 --2- 172.21.15.16:0/2468696221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1680cef40 0x7fa1680d2610 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.592+0000 7fa16dab3640 1 --2- 172.21.15.16:0/2468696221 >> v2:172.21.15.186:3300/0 conn(0x7fa1681610b0 0x7fa1680b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.592+0000 7fa16dab3640 1 -- 172.21.15.16:0/2468696221 >> 172.21.15.16:0/2468696221 conn(0x7fa1680b5570 msgr2=0x7fa1680b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.592+0000 7fa16dab3640 1 -- 172.21.15.16:0/2468696221 shutdown_connections 2024-04-25T10:15:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.592+0000 7fa16dab3640 1 -- 172.21.15.16:0/2468696221 wait complete. 2024-04-25T10:15:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.593+0000 7fa16dab3640 1 Processor -- start 2024-04-25T10:15:14.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.593+0000 7fa16dab3640 1 -- start start 2024-04-25T10:15:14.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.594+0000 7fa16dab3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1680cef40 0x7fa168158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.594+0000 7fa16dab3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1681610b0 0x7fa16814f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.594+0000 7fa16dab3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1681618d0 0x7fa1680cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.594+0000 7fa16dab3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa168161d10 con 0x7fa1681618d0 2024-04-25T10:15:14.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.594+0000 7fa16dab3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680b6c00 con 0x7fa1680cef40 2024-04-25T10:15:14.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.594+0000 7fa16dab3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa168161490 con 0x7fa1681610b0 2024-04-25T10:15:14.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.594+0000 7fa167fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1681618d0 0x7fa1680cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.594+0000 7fa167fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1681618d0 0x7fa1680cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43270/0 (socket says 172.21.15.16:43270) 2024-04-25T10:15:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.594+0000 7fa167fff640 1 -- 172.21.15.16:0/340821576 learned_addr learned my addr 172.21.15.16:0/340821576 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.594+0000 7fa1677fe640 1 --2- 172.21.15.16:0/340821576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1680cef40 0x7fa168158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.594+0000 7fa166ffd640 1 --2- 172.21.15.16:0/340821576 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1681610b0 0x7fa16814f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.594+0000 7fa167fff640 1 -- 172.21.15.16:0/340821576 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1681610b0 msgr2=0x7fa16814f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.594+0000 7fa167fff640 1 --2- 172.21.15.16:0/340821576 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1681610b0 0x7fa16814f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.594+0000 7fa167fff640 1 -- 172.21.15.16:0/340821576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1680cef40 msgr2=0x7fa168158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.594+0000 7fa167fff640 1 --2- 172.21.15.16:0/340821576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1680cef40 0x7fa168158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.594+0000 7fa167fff640 1 -- 172.21.15.16:0/340821576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa168157bb0 con 0x7fa1681618d0 2024-04-25T10:15:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.594+0000 7fa167fff640 1 --2- 172.21.15.16:0/340821576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1681618d0 0x7fa1680cbfa0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fa160009f40 tx=0x7fa160004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.595+0000 7fa14ffff640 1 -- 172.21.15.16:0/340821576 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160036050 con 0x7fa1681618d0 2024-04-25T10:15:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.595+0000 7fa14ffff640 1 -- 172.21.15.16:0/340821576 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa160042070 con 0x7fa1681618d0 2024-04-25T10:15:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.595+0000 7fa14ffff640 1 -- 172.21.15.16:0/340821576 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160009510 con 0x7fa1681618d0 2024-04-25T10:15:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.595+0000 7fa16dab3640 1 -- 172.21.15.16:0/340821576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa168157e90 con 0x7fa1681618d0 2024-04-25T10:15:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.595+0000 7fa16dab3640 1 -- 172.21.15.16:0/340821576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa168158450 con 0x7fa1681618d0 2024-04-25T10:15:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.596+0000 7fa16dab3640 1 -- 172.21.15.16:0/340821576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1680750b0 con 0x7fa1681618d0 2024-04-25T10:15:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.596+0000 7fa14ffff640 1 -- 172.21.15.16:0/340821576 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa160007bd0 con 0x7fa1681618d0 2024-04-25T10:15:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.597+0000 7fa14ffff640 1 --2- 172.21.15.16:0/340821576 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa15406b4d0 0x7fa15406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.597+0000 7fa14ffff640 1 -- 172.21.15.16:0/340821576 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fa1600aaf70 con 0x7fa1681618d0 2024-04-25T10:15:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.597+0000 7fa1677fe640 1 --2- 172.21.15.16:0/340821576 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa15406b4d0 0x7fa15406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.597+0000 7fa1677fe640 1 --2- 172.21.15.16:0/340821576 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa15406b4d0 0x7fa15406d990 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa15800a970 tx=0x7fa158008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.601+0000 7fa14ffff640 1 -- 172.21.15.16:0/340821576 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa16007a040 con 0x7fa1681618d0 2024-04-25T10:15:14.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.781+0000 7fa16dab3640 1 -- 172.21.15.16:0/340821576 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fa168157960 con 0x7fa15406b4d0 2024-04-25T10:15:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.781+0000 7fa14ffff640 1 -- 172.21.15.16:0/340821576 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa168157960 con 0x7fa15406b4d0 2024-04-25T10:15:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T10:15:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.783+0000 7fa16dab3640 1 -- 172.21.15.16:0/340821576 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa15406b4d0 msgr2=0x7fa15406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.783+0000 7fa16dab3640 1 --2- 172.21.15.16:0/340821576 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa15406b4d0 0x7fa15406d990 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa15800a970 tx=0x7fa158008040 comp rx=0 tx=0).stop 2024-04-25T10:15:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.783+0000 7fa16dab3640 1 -- 172.21.15.16:0/340821576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1681618d0 msgr2=0x7fa1680cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.783+0000 7fa16dab3640 1 --2- 172.21.15.16:0/340821576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1681618d0 0x7fa1680cbfa0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fa160009f40 tx=0x7fa160004a40 comp rx=0 tx=0).stop 2024-04-25T10:15:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.783+0000 7fa16dab3640 1 -- 172.21.15.16:0/340821576 shutdown_connections 2024-04-25T10:15:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.783+0000 7fa16dab3640 1 --2- 172.21.15.16:0/340821576 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7fa15406b4d0 0x7fa15406d990 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.783+0000 7fa16dab3640 1 --2- 172.21.15.16:0/340821576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1681618d0 0x7fa1680cbfa0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.783+0000 7fa16dab3640 1 --2- 172.21.15.16:0/340821576 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa1681610b0 0x7fa16814f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.783+0000 7fa16dab3640 1 --2- 172.21.15.16:0/340821576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1680cef40 0x7fa168158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.783+0000 7fa16dab3640 1 -- 172.21.15.16:0/340821576 >> 172.21.15.16:0/340821576 conn(0x7fa1680b5570 msgr2=0x7fa1680d3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.783+0000 7fa16dab3640 1 -- 172.21.15.16:0/340821576 shutdown_connections 2024-04-25T10:15:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.783+0000 7fa16dab3640 1 -- 172.21.15.16:0/340821576 wait complete. 2024-04-25T10:15:14.798 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:15:14.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.894+0000 7f04f98f5640 1 Processor -- start 2024-04-25T10:15:14.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.894+0000 7f04f98f5640 1 -- start start 2024-04-25T10:15:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.895+0000 7f04f98f5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04f415dd50 0x7f04f40d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.895+0000 7f04f98f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04f40b9460 0x7f04f40ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.895+0000 7f04f98f5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04f40d2440 con 0x7f04f40d34e0 2024-04-25T10:15:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.895+0000 7f04f98f5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04f40d2720 con 0x7f04f415dd50 2024-04-25T10:15:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.895+0000 7f04f98f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f40d2a00 con 0x7f04f40b9460 2024-04-25T10:15:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.895+0000 7f04f2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04f415dd50 0x7f04f40d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.895+0000 7f04f37fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f04f40d34e0 0x7f04f40b5550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46520/0 (socket says 172.21.15.16:46520) 2024-04-25T10:15:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.895+0000 7f04f37fe640 1 -- 172.21.15.16:0/3080401757 learned_addr learned my addr 172.21.15.16:0/3080401757 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.895+0000 7f04f27fc640 1 --2- 172.21.15.16:0/3080401757 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04f40b9460 0x7f04f40ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.896+0000 7f04f2ffd640 1 -- 172.21.15.16:0/3080401757 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04f40b9460 msgr2=0x7f04f40ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.896+0000 7f04f2ffd640 1 --2- 172.21.15.16:0/3080401757 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04f40b9460 0x7f04f40ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.896+0000 7f04f2ffd640 1 -- 172.21.15.16:0/3080401757 >> v1:172.21.15.16:6789/0 conn(0x7f04f40d34e0 legacy=0x7f04f40b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.896+0000 7f04f2ffd640 1 -- 172.21.15.16:0/3080401757 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f40d2ce0 con 0x7f04f415dd50 2024-04-25T10:15:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.896+0000 7f04f2ffd640 1 --2- 172.21.15.16:0/3080401757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f415dd50 0x7f04f40d1c10 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f04e4009d90 tx=0x7f04e402a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfcab8356d03bb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.897+0000 7f04f1ffb640 1 -- 172.21.15.16:0/3080401757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4007740 con 0x7f04f415dd50 2024-04-25T10:15:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.897+0000 7f04f1ffb640 1 -- 172.21.15.16:0/3080401757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e40078e0 con 0x7f04f415dd50 2024-04-25T10:15:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.897+0000 7f04f1ffb640 1 -- 172.21.15.16:0/3080401757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4007cb0 con 0x7f04f415dd50 2024-04-25T10:15:14.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.897+0000 7f04f98f5640 1 -- 172.21.15.16:0/3080401757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f415dd50 msgr2=0x7f04f40d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.897+0000 7f04f98f5640 1 --2- 172.21.15.16:0/3080401757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f415dd50 0x7f04f40d1c10 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f04e4009d90 tx=0x7f04e402a4d0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.897+0000 7f04f98f5640 1 -- 172.21.15.16:0/3080401757 shutdown_connections 2024-04-25T10:15:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.897+0000 7f04f98f5640 1 --2- 172.21.15.16:0/3080401757 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04f40b9460 0x7f04f40ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.897+0000 7f04f98f5640 1 --2- 172.21.15.16:0/3080401757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f415dd50 0x7f04f40d1c10 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.897+0000 7f04f98f5640 1 -- 172.21.15.16:0/3080401757 >> 172.21.15.16:0/3080401757 conn(0x7f04f40722d0 msgr2=0x7f04f40726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.897+0000 7f04f98f5640 1 -- 172.21.15.16:0/3080401757 shutdown_connections 2024-04-25T10:15:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.897+0000 7f04f98f5640 1 -- 172.21.15.16:0/3080401757 wait complete. 2024-04-25T10:15:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.898+0000 7f04f98f5640 1 Processor -- start 2024-04-25T10:15:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.899+0000 7f04f98f5640 1 -- start start 2024-04-25T10:15:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.899+0000 7f04f98f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f40b9460 0x7f04f4160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.899+0000 7f04f98f5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04f40d34e0 0x7f04f41f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.899+0000 7f04f98f5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04f415dd50 0x7f04f40b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.899+0000 7f04f98f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f40cece0 con 0x7f04f40b9460 2024-04-25T10:15:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.899+0000 7f04f98f5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f40d2150 con 0x7f04f40d34e0 2024-04-25T10:15:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.899+0000 7f04f98f5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f40cee60 con 0x7f04f415dd50 2024-04-25T10:15:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.899+0000 7f04f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f40b9460 0x7f04f4160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.899+0000 7f04f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f40b9460 0x7f04f4160a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43290/0 (socket says 172.21.15.16:43290) 2024-04-25T10:15:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.899+0000 7f04f2ffd640 1 -- 172.21.15.16:0/1667462882 learned_addr learned my addr 172.21.15.16:0/1667462882 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.900+0000 7f04f37fe640 1 --2- 172.21.15.16:0/1667462882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04f415dd50 0x7f04f40b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.900+0000 7f04f27fc640 1 --2- 172.21.15.16:0/1667462882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04f40d34e0 0x7f04f41f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.900+0000 7f04f2ffd640 1 -- 172.21.15.16:0/1667462882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04f415dd50 msgr2=0x7f04f40b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.900+0000 7f04f2ffd640 1 --2- 172.21.15.16:0/1667462882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04f415dd50 0x7f04f40b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.900+0000 7f04f2ffd640 1 -- 172.21.15.16:0/1667462882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04f40d34e0 msgr2=0x7f04f41f37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.900+0000 7f04f2ffd640 1 --2- 172.21.15.16:0/1667462882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04f40d34e0 0x7f04f41f37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.900+0000 7f04f2ffd640 1 -- 172.21.15.16:0/1667462882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f40cd5a0 con 0x7f04f40b9460 2024-04-25T10:15:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.900+0000 7f04f37fe640 1 --2- 172.21.15.16:0/1667462882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04f415dd50 0x7f04f40b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.900+0000 7f04f2ffd640 1 --2- 172.21.15.16:0/1667462882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f40b9460 0x7f04f4160a20 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f04e4009d90 tx=0x7f04e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.900+0000 7f04dbfff640 1 -- 172.21.15.16:0/1667462882 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e403f070 con 0x7f04f40b9460 2024-04-25T10:15:14.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.900+0000 7f04dbfff640 1 -- 172.21.15.16:0/1667462882 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e4031bc0 con 0x7f04f40b9460 2024-04-25T10:15:14.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.900+0000 7f04f98f5640 1 -- 172.21.15.16:0/1667462882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f40cd880 con 0x7f04f40b9460 2024-04-25T10:15:14.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.900+0000 7f04dbfff640 1 -- 172.21.15.16:0/1667462882 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4044460 con 0x7f04f40b9460 2024-04-25T10:15:14.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.900+0000 7f04f98f5640 1 -- 172.21.15.16:0/1667462882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f415e1d0 con 0x7f04f40b9460 2024-04-25T10:15:14.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.901+0000 7f04f98f5640 1 -- 172.21.15.16:0/1667462882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f40749b0 con 0x7f04f40b9460 2024-04-25T10:15:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.901+0000 7f04dbfff640 1 -- 172.21.15.16:0/1667462882 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f04e4044600 con 0x7f04f40b9460 2024-04-25T10:15:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.902+0000 7f04dbfff640 1 --2- 172.21.15.16:0/1667462882 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f04e006b4d0 0x7f04e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.902+0000 7f04dbfff640 1 -- 172.21.15.16:0/1667462882 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f04e40ab7f0 con 0x7f04f40b9460 2024-04-25T10:15:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.902+0000 7f04f27fc640 1 --2- 172.21.15.16:0/1667462882 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f04e006b4d0 0x7f04e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.903+0000 7f04f27fc640 1 --2- 172.21.15.16:0/1667462882 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f04e006b4d0 0x7f04e006d990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f04f40d0b50 tx=0x7f04e8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:14.906+0000 7f04dbfff640 1 -- 172.21.15.16:0/1667462882 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f04e407a960 con 0x7f04f40b9460 2024-04-25T10:15:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.082+0000 7f04f98f5640 1 -- 172.21.15.16:0/1667462882 --> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f04f40b8b60 con 0x7f04e006b4d0 2024-04-25T10:15:15.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.083+0000 7f04dbfff640 1 -- 172.21.15.16:0/1667462882 <== mgr.4099 v2:172.21.15.186:6832/4291145543 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13445 (secure 0 0 0) 0x7f04f40b8b60 con 0x7f04e006b4d0 2024-04-25T10:15:15.087 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:15:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-25T10:15:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.085+0000 7f04f98f5640 1 -- 172.21.15.16:0/1667462882 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f04e006b4d0 msgr2=0x7f04e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.085+0000 7f04f98f5640 1 --2- 172.21.15.16:0/1667462882 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f04e006b4d0 0x7f04e006d990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f04f40d0b50 tx=0x7f04e8008040 comp rx=0 tx=0).stop 2024-04-25T10:15:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.085+0000 7f04f98f5640 1 -- 172.21.15.16:0/1667462882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f40b9460 msgr2=0x7f04f4160a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.085+0000 7f04f98f5640 1 --2- 172.21.15.16:0/1667462882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f40b9460 0x7f04f4160a20 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f04e4009d90 tx=0x7f04e4004a40 comp rx=0 tx=0).stop 2024-04-25T10:15:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.085+0000 7f04f98f5640 1 -- 172.21.15.16:0/1667462882 shutdown_connections 2024-04-25T10:15:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.085+0000 7f04f98f5640 1 --2- 172.21.15.16:0/1667462882 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f04e006b4d0 0x7f04e006d990 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.085+0000 7f04f98f5640 1 --2- 172.21.15.16:0/1667462882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04f415dd50 0x7f04f40b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.085+0000 7f04f98f5640 1 --2- 172.21.15.16:0/1667462882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04f40d34e0 0x7f04f41f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.085+0000 7f04f98f5640 1 --2- 172.21.15.16:0/1667462882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f40b9460 0x7f04f4160a20 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.085+0000 7f04f98f5640 1 -- 172.21.15.16:0/1667462882 >> 172.21.15.16:0/1667462882 conn(0x7f04f40722d0 msgr2=0x7f04f415bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.085+0000 7f04f98f5640 1 -- 172.21.15.16:0/1667462882 shutdown_connections 2024-04-25T10:15:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.086+0000 7f04f98f5640 1 -- 172.21.15.16:0/1667462882 wait complete. 2024-04-25T10:15:15.100 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2024-04-25T10:15:13.930193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":279084,"kb_used_data":13000,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754695636,"statfs":{"total":773094113280,"available":772808331264,"internally_reserved":0,"allocated":13312000,"data_stored":6453576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392252,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-256,"num_read_kb":-266,"num_write":-175,"num_write_kb":-8304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001467"},"pg_stats":[{"pgid":"1.0","version":"122'119","reported_seq":386,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T10:15:12.578932+0000","last_change":"2024-04-25T10:15:12.578996+0000","last_active":"2024-04-25T10:15:12.578932+0000","last_peered":"2024-04-25T10:15:12.578932+0000","last_clean":"2024-04-25T10:15:12.578932+0000","last_became_active":"2024-04-25T10:04:59.849442+0000","last_became_peered":"2024-04-25T10:04:59.849442+0000","last_unstale":"2024-04-25T10:15:12.578932+0000","last_undegraded":"2024-04-25T10:15:12.578932+0000","last_fullsized":"2024-04-25T10:15:12.578932+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"122'119","last_scrub_stamp":"2024-04-25T10:15:12.578840+0000","last_deep_scrub":"122'119","last_deep_scrub_stamp":"2024-04-25T10:15:12.578840+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.578840+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:42.643960+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509566,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32660,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1515520,"data_stored":659105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509566,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32660,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1515520,"data_stored":659105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509567,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32660,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1515520,"data_stored":659105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509567,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36820,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":1515520,"data_stored":659105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509568,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36824,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":1515520,"data_stored":659105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509566,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37400,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334440,"statfs":{"total":96636764160,"available":96598466560,"internally_reserved":0,"allocated":2109440,"data_stored":1249473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509568,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36820,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":1515520,"data_stored":659105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574974,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33240,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338600,"statfs":{"total":96636764160,"available":96602726400,"internally_reserved":0,"allocated":2109440,"data_stored":1249473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:15:15.103 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T10:15:15.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.201+0000 7f6624af5640 1 Processor -- start 2024-04-25T10:15:15.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.201+0000 7f6624af5640 1 -- start start 2024-04-25T10:15:15.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.202+0000 7f6624af5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f662015dca0 0x7f66200ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.202+0000 7f6624af5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f66200d3b90 0x7f66200d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.202+0000 7f6624af5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66200d4090 con 0x7f66200cee80 2024-04-25T10:15:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.202+0000 7f6624af5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66200b6b20 con 0x7f66200d3b90 2024-04-25T10:15:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.202+0000 7f6624af5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66200d4370 con 0x7f662015dca0 2024-04-25T10:15:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.202+0000 7f661ed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f66200cee80 0x7f66200b2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46536/0 (socket says 172.21.15.16:46536) 2024-04-25T10:15:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.202+0000 7f661ed9d640 1 -- 172.21.15.16:0/2752157309 learned_addr learned my addr 172.21.15.16:0/2752157309 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-04-25T10:15:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.202+0000 7f661e59c640 1 --2- 172.21.15.16:0/2752157309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f662015dca0 0x7f66200ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.202+0000 7f661dd9b640 1 --2- 172.21.15.16:0/2752157309 >> v2:172.21.15.186:3300/0 conn(0x7f66200d3b90 0x7f66200d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.202+0000 7f661d59a640 1 -- 172.21.15.16:0/2752157309 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194830326 0 0) 0x7f66200d4090 con 0x7f66200cee80 2024-04-25T10:15:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.202+0000 7f661d59a640 1 -- 172.21.15.16:0/2752157309 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f660c003520 con 0x7f66200cee80 2024-04-25T10:15:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.202+0000 7f661e59c640 1 -- 172.21.15.16:0/2752157309 >> v2:172.21.15.186:3300/0 conn(0x7f66200d3b90 msgr2=0x7f66200d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.202+0000 7f661e59c640 1 --2- 172.21.15.16:0/2752157309 >> v2:172.21.15.186:3300/0 conn(0x7f66200d3b90 0x7f66200d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.202+0000 7f661e59c640 1 -- 172.21.15.16:0/2752157309 >> v1:172.21.15.16:6789/0 conn(0x7f66200cee80 legacy=0x7f66200b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.202+0000 7f661d59a640 1 -- 172.21.15.16:0/2752157309 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 59849214 0 0) 0x7f660c003520 con 0x7f66200cee80 2024-04-25T10:15:15.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.202+0000 7f661e59c640 1 -- 172.21.15.16:0/2752157309 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66200d4650 con 0x7f662015dca0 2024-04-25T10:15:15.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.202+0000 7f661dd9b640 1 --2- 172.21.15.16:0/2752157309 >> v2:172.21.15.186:3300/0 conn(0x7f66200d3b90 0x7f66200d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:15.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.203+0000 7f661e59c640 1 --2- 172.21.15.16:0/2752157309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f662015dca0 0x7f66200ce940 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f66180097a0 tx=0x7f661802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=863ec406ec769445 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.203+0000 7f661d59a640 1 -- 172.21.15.16:0/2752157309 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6618037070 con 0x7f662015dca0 2024-04-25T10:15:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.203+0000 7f661d59a640 1 -- 172.21.15.16:0/2752157309 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661802aba0 con 0x7f662015dca0 2024-04-25T10:15:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.203+0000 7f661d59a640 1 -- 172.21.15.16:0/2752157309 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66100019f0 con 0x7f662015dca0 2024-04-25T10:15:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.204+0000 7f6624af5640 1 -- 172.21.15.16:0/2752157309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f662015dca0 msgr2=0x7f66200ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.204+0000 7f6624af5640 1 --2- 172.21.15.16:0/2752157309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f662015dca0 0x7f66200ce940 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f66180097a0 tx=0x7f661802a110 comp rx=0 tx=0).stop 2024-04-25T10:15:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.204+0000 7f6624af5640 1 -- 172.21.15.16:0/2752157309 shutdown_connections 2024-04-25T10:15:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.204+0000 7f6624af5640 1 --2- 172.21.15.16:0/2752157309 >> v2:172.21.15.186:3300/0 conn(0x7f66200d3b90 0x7f66200d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.204+0000 7f6624af5640 1 --2- 172.21.15.16:0/2752157309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f662015dca0 0x7f66200ce940 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.204+0000 7f6624af5640 1 -- 172.21.15.16:0/2752157309 >> 172.21.15.16:0/2752157309 conn(0x7f66200726c0 msgr2=0x7f6620072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:15.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.204+0000 7f6624af5640 1 -- 172.21.15.16:0/2752157309 shutdown_connections 2024-04-25T10:15:15.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.204+0000 7f6624af5640 1 -- 172.21.15.16:0/2752157309 wait complete. 2024-04-25T10:15:15.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.205+0000 7f6624af5640 1 Processor -- start 2024-04-25T10:15:15.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.205+0000 7f6624af5640 1 -- start start 2024-04-25T10:15:15.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.206+0000 7f6624af5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66200cee80 0x7f66200cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.206+0000 7f6624af5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66200d3b90 0x7f662015a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.206+0000 7f6624af5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f662015dca0 0x7f6620156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.206+0000 7f6624af5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662015e200 con 0x7f66200d3b90 2024-04-25T10:15:15.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.206+0000 7f6624af5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662015e080 con 0x7f662015dca0 2024-04-25T10:15:15.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.206+0000 7f6624af5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66200d0f60 con 0x7f66200cee80 2024-04-25T10:15:15.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.206+0000 7f661dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66200d3b90 0x7f662015a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.206+0000 7f661dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66200d3b90 0x7f662015a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43298/0 (socket says 172.21.15.16:43298) 2024-04-25T10:15:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.206+0000 7f661dd9b640 1 -- 172.21.15.16:0/3574977634 learned_addr learned my addr 172.21.15.16:0/3574977634 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.206+0000 7f661ed9d640 1 --2- 172.21.15.16:0/3574977634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f662015dca0 0x7f6620156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.206+0000 7f661e59c640 1 --2- 172.21.15.16:0/3574977634 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66200cee80 0x7f66200cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.206+0000 7f661dd9b640 1 -- 172.21.15.16:0/3574977634 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66200cee80 msgr2=0x7f66200cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.206+0000 7f661dd9b640 1 --2- 172.21.15.16:0/3574977634 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66200cee80 0x7f66200cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.207+0000 7f661dd9b640 1 -- 172.21.15.16:0/3574977634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f662015dca0 msgr2=0x7f6620156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.207+0000 7f661dd9b640 1 --2- 172.21.15.16:0/3574977634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f662015dca0 0x7f6620156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.207+0000 7f661dd9b640 1 -- 172.21.15.16:0/3574977634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66200cc400 con 0x7f66200d3b90 2024-04-25T10:15:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.207+0000 7f661dd9b640 1 --2- 172.21.15.16:0/3574977634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66200d3b90 0x7f662015a140 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f660800ebe0 tx=0x7f660800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.207+0000 7f6616ffd640 1 -- 172.21.15.16:0/3574977634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608017070 con 0x7f66200d3b90 2024-04-25T10:15:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.207+0000 7f6616ffd640 1 -- 172.21.15.16:0/3574977634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660800cc30 con 0x7f66200d3b90 2024-04-25T10:15:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.207+0000 7f6624af5640 1 -- 172.21.15.16:0/3574977634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66200cc6e0 con 0x7f66200d3b90 2024-04-25T10:15:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.207+0000 7f6616ffd640 1 -- 172.21.15.16:0/3574977634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608013470 con 0x7f66200d3b90 2024-04-25T10:15:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.207+0000 7f6624af5640 1 -- 172.21.15.16:0/3574977634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66200ccca0 con 0x7f66200d3b90 2024-04-25T10:15:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.208+0000 7f6624af5640 1 -- 172.21.15.16:0/3574977634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66200750b0 con 0x7f66200d3b90 2024-04-25T10:15:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.208+0000 7f6616ffd640 1 -- 172.21.15.16:0/3574977634 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f660801a070 con 0x7f66200d3b90 2024-04-25T10:15:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.209+0000 7f6616ffd640 1 --2- 172.21.15.16:0/3574977634 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f660006b4d0 0x7f660006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.209+0000 7f6616ffd640 1 -- 172.21.15.16:0/3574977634 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f660808a300 con 0x7f66200d3b90 2024-04-25T10:15:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.209+0000 7f661e59c640 1 --2- 172.21.15.16:0/3574977634 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f660006b4d0 0x7f660006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.210+0000 7f661e59c640 1 --2- 172.21.15.16:0/3574977634 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f660006b4d0 0x7f660006d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6618034000 tx=0x7f661802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:15.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.213+0000 7f6616ffd640 1 -- 172.21.15.16:0/3574977634 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6608058c70 con 0x7f66200d3b90 2024-04-25T10:15:15.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.397+0000 7f6624af5640 1 -- 172.21.15.16:0/3574977634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f66200d2ae0 con 0x7f66200d3b90 2024-04-25T10:15:15.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.400+0000 7f6616ffd640 1 -- 172.21.15.16:0/3574977634 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) v1 ==== 123+0+0 (secure 0 0 0) 0x7f660805cd10 con 0x7f66200d3b90 2024-04-25T10:15:15.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.402+0000 7f6616ffd640 1 -- 172.21.15.16:0/3574977634 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f66080093e0 con 0x7f66200d3b90 2024-04-25T10:15:15.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.403+0000 7f6624af5640 1 -- 172.21.15.16:0/3574977634 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f660006b4d0 msgr2=0x7f660006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.403+0000 7f6624af5640 1 --2- 172.21.15.16:0/3574977634 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f660006b4d0 0x7f660006d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6618034000 tx=0x7f661802e000 comp rx=0 tx=0).stop 2024-04-25T10:15:15.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.403+0000 7f6624af5640 1 -- 172.21.15.16:0/3574977634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66200d3b90 msgr2=0x7f662015a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.403+0000 7f6624af5640 1 --2- 172.21.15.16:0/3574977634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66200d3b90 0x7f662015a140 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f660800ebe0 tx=0x7f660800c520 comp rx=0 tx=0).stop 2024-04-25T10:15:15.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.403+0000 7f6624af5640 1 -- 172.21.15.16:0/3574977634 shutdown_connections 2024-04-25T10:15:15.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.403+0000 7f6624af5640 1 --2- 172.21.15.16:0/3574977634 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f660006b4d0 0x7f660006d990 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.403+0000 7f6624af5640 1 --2- 172.21.15.16:0/3574977634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f662015dca0 0x7f6620156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.403+0000 7f6624af5640 1 --2- 172.21.15.16:0/3574977634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66200d3b90 0x7f662015a140 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.403+0000 7f6624af5640 1 --2- 172.21.15.16:0/3574977634 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66200cee80 0x7f66200cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.403+0000 7f6624af5640 1 -- 172.21.15.16:0/3574977634 >> 172.21.15.16:0/3574977634 conn(0x7f66200726c0 msgr2=0x7f6620160390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:15.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.403+0000 7f6624af5640 1 -- 172.21.15.16:0/3574977634 shutdown_connections 2024-04-25T10:15:15.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.403+0000 7f6624af5640 1 -- 172.21.15.16:0/3574977634 wait complete. 2024-04-25T10:15:15.417 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:15:15.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.514+0000 7f4798d22640 1 Processor -- start 2024-04-25T10:15:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.514+0000 7f4798d22640 1 -- start start 2024-04-25T10:15:15.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.515+0000 7f4798d22640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f479415dc30 0x7f479406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.515+0000 7f4798d22640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f479406cc00 0x7f47940b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.515+0000 7f4798d22640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f47940b1960 0x7f4794099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.515+0000 7f4798d22640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479406c1f0 con 0x7f479406cc00 2024-04-25T10:15:15.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.515+0000 7f4798d22640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47940cfb40 con 0x7f47940b1960 2024-04-25T10:15:15.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.515+0000 7f4798d22640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47940b2060 con 0x7f479415dc30 2024-04-25T10:15:15.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.515+0000 7f4791d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f479406cc00 0x7f47940b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.515+0000 7f4791d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f479406cc00 0x7f47940b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43312/0 (socket says 172.21.15.16:43312) 2024-04-25T10:15:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.515+0000 7f4791d9b640 1 -- 172.21.15.16:0/706466698 learned_addr learned my addr 172.21.15.16:0/706466698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.516+0000 7f4792d9d640 1 --2- 172.21.15.16:0/706466698 >> v2:172.21.15.186:3300/0 conn(0x7f47940b1960 0x7f4794099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.516+0000 7f479259c640 1 --2- 172.21.15.16:0/706466698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f479415dc30 0x7f479406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.516+0000 7f4791d9b640 1 -- 172.21.15.16:0/706466698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f479415dc30 msgr2=0x7f479406c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.516+0000 7f4791d9b640 1 --2- 172.21.15.16:0/706466698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f479415dc30 0x7f479406c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.516+0000 7f4791d9b640 1 -- 172.21.15.16:0/706466698 >> v2:172.21.15.186:3300/0 conn(0x7f47940b1960 msgr2=0x7f4794099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.516+0000 7f4791d9b640 1 --2- 172.21.15.16:0/706466698 >> v2:172.21.15.186:3300/0 conn(0x7f47940b1960 0x7f4794099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.516+0000 7f4791d9b640 1 -- 172.21.15.16:0/706466698 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47940b2340 con 0x7f479406cc00 2024-04-25T10:15:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.516+0000 7f4792d9d640 1 --2- 172.21.15.16:0/706466698 >> v2:172.21.15.186:3300/0 conn(0x7f47940b1960 0x7f4794099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.517+0000 7f4791d9b640 1 --2- 172.21.15.16:0/706466698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f479406cc00 0x7f47940b1420 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f47940d2770 tx=0x7f478802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc76bf3ffe347b21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.517+0000 7f479159a640 1 -- 172.21.15.16:0/706466698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788037070 con 0x7f479406cc00 2024-04-25T10:15:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.517+0000 7f479159a640 1 -- 172.21.15.16:0/706466698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4788032440 con 0x7f479406cc00 2024-04-25T10:15:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.517+0000 7f479159a640 1 -- 172.21.15.16:0/706466698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788032830 con 0x7f479406cc00 2024-04-25T10:15:15.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.517+0000 7f4798d22640 1 -- 172.21.15.16:0/706466698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f479406cc00 msgr2=0x7f47940b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.517+0000 7f4798d22640 1 --2- 172.21.15.16:0/706466698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f479406cc00 0x7f47940b1420 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f47940d2770 tx=0x7f478802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.517+0000 7f4798d22640 1 -- 172.21.15.16:0/706466698 shutdown_connections 2024-04-25T10:15:15.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.517+0000 7f4798d22640 1 --2- 172.21.15.16:0/706466698 >> v2:172.21.15.186:3300/0 conn(0x7f47940b1960 0x7f4794099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.517+0000 7f4798d22640 1 --2- 172.21.15.16:0/706466698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f479406cc00 0x7f47940b1420 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.517+0000 7f4798d22640 1 --2- 172.21.15.16:0/706466698 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f479415dc30 0x7f479406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.517+0000 7f4798d22640 1 -- 172.21.15.16:0/706466698 >> 172.21.15.16:0/706466698 conn(0x7f47940d1900 msgr2=0x7f47940d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:15.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.517+0000 7f4798d22640 1 -- 172.21.15.16:0/706466698 shutdown_connections 2024-04-25T10:15:15.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.517+0000 7f4798d22640 1 -- 172.21.15.16:0/706466698 wait complete. 2024-04-25T10:15:15.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.519+0000 7f4798d22640 1 Processor -- start 2024-04-25T10:15:15.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.519+0000 7f4798d22640 1 -- start start 2024-04-25T10:15:15.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4798d22640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f479406cc00 0x7f47941f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4798d22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47940b1960 0x7f47941f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4798d22640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f479415dc30 0x7f47941f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4798d22640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47940b1d80 con 0x7f47940b1960 2024-04-25T10:15:15.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4798d22640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47940cfde0 con 0x7f479406cc00 2024-04-25T10:15:15.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4798d22640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479415e010 con 0x7f479415dc30 2024-04-25T10:15:15.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4791d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47940b1960 0x7f47941f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4791d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47940b1960 0x7f47941f6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43322/0 (socket says 172.21.15.16:43322) 2024-04-25T10:15:15.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4791d9b640 1 -- 172.21.15.16:0/3068151341 learned_addr learned my addr 172.21.15.16:0/3068151341 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-25T10:15:15.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f479259c640 1 --2- 172.21.15.16:0/3068151341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f479406cc00 0x7f47941f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4792d9d640 1 --2- 172.21.15.16:0/3068151341 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f479415dc30 0x7f47941f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4791d9b640 1 -- 172.21.15.16:0/3068151341 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f479415dc30 msgr2=0x7f47941f7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4791d9b640 1 --2- 172.21.15.16:0/3068151341 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f479415dc30 0x7f47941f7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4791d9b640 1 -- 172.21.15.16:0/3068151341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f479406cc00 msgr2=0x7f47941f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4791d9b640 1 --2- 172.21.15.16:0/3068151341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f479406cc00 0x7f47941f4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4791d9b640 1 -- 172.21.15.16:0/3068151341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479415e410 con 0x7f47940b1960 2024-04-25T10:15:15.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f479259c640 1 --2- 172.21.15.16:0/3068151341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f479406cc00 0x7f47941f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:15.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4792d9d640 1 --2- 172.21.15.16:0/3068151341 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f479415dc30 0x7f47941f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:15.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4791d9b640 1 --2- 172.21.15.16:0/3068151341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47940b1960 0x7f47941f6c30 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f478802a220 tx=0x7f47880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:15.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4782ffd640 1 -- 172.21.15.16:0/3068151341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788037040 con 0x7f47940b1960 2024-04-25T10:15:15.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4782ffd640 1 -- 172.21.15.16:0/3068151341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4788042070 con 0x7f47940b1960 2024-04-25T10:15:15.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4798d22640 1 -- 172.21.15.16:0/3068151341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479415e6f0 con 0x7f47940b1960 2024-04-25T10:15:15.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4782ffd640 1 -- 172.21.15.16:0/3068151341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788030bf0 con 0x7f47940b1960 2024-04-25T10:15:15.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.520+0000 7f4798d22640 1 -- 172.21.15.16:0/3068151341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479415ecb0 con 0x7f47940b1960 2024-04-25T10:15:15.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.521+0000 7f4798d22640 1 -- 172.21.15.16:0/3068151341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4794067850 con 0x7f47940b1960 2024-04-25T10:15:15.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.522+0000 7f4782ffd640 1 -- 172.21.15.16:0/3068151341 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4788047460 con 0x7f47940b1960 2024-04-25T10:15:15.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.522+0000 7f4782ffd640 1 --2- 172.21.15.16:0/3068151341 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f477406b4d0 0x7f477406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.522+0000 7f4782ffd640 1 -- 172.21.15.16:0/3068151341 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f47880aabc0 con 0x7f47940b1960 2024-04-25T10:15:15.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.523+0000 7f479259c640 1 --2- 172.21.15.16:0/3068151341 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f477406b4d0 0x7f477406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.523+0000 7f479259c640 1 --2- 172.21.15.16:0/3068151341 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f477406b4d0 0x7f477406d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4784006fd0 tx=0x7f4784008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:15.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.527+0000 7f4782ffd640 1 -- 172.21.15.16:0/3068151341 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4788032e20 con 0x7f47940b1960 2024-04-25T10:15:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.743+0000 7f4798d22640 1 -- 172.21.15.16:0/3068151341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47940d1500 con 0x7f47940b1960 2024-04-25T10:15:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.744+0000 7f4782ffd640 1 -- 172.21.15.16:0/3068151341 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f4788045360 con 0x7f47940b1960 2024-04-25T10:15:15.748 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-25T10:15:15.748 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":98,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2871061122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2871061122},{"type":"v1","addr":"172.21.15.186:6836","nonce":2871061122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1871507206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1871507206},{"type":"v1","addr":"172.21.15.186:6837","nonce":1871507206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4064412586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4064412586},{"type":"v1","addr":"172.21.15.16:6834","nonce":4064412586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":9483,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3215055221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":3215055221},{"type":"v1","addr":"172.21.15.16:6835","nonce":3215055221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[]} 2024-04-25T10:15:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 98 2024-04-25T10:15:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.745+0000 7f4798d22640 1 -- 172.21.15.16:0/3068151341 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f477406b4d0 msgr2=0x7f477406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.745+0000 7f4798d22640 1 --2- 172.21.15.16:0/3068151341 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f477406b4d0 0x7f477406d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4784006fd0 tx=0x7f4784008040 comp rx=0 tx=0).stop 2024-04-25T10:15:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.745+0000 7f4798d22640 1 -- 172.21.15.16:0/3068151341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47940b1960 msgr2=0x7f47941f6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.745+0000 7f4798d22640 1 --2- 172.21.15.16:0/3068151341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47940b1960 0x7f47941f6c30 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f478802a220 tx=0x7f47880046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.745+0000 7f4798d22640 1 -- 172.21.15.16:0/3068151341 shutdown_connections 2024-04-25T10:15:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.745+0000 7f4798d22640 1 --2- 172.21.15.16:0/3068151341 >> [v2:172.21.15.186:6832/4291145543,v1:172.21.15.186:6833/4291145543] conn(0x7f477406b4d0 0x7f477406d990 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.745+0000 7f4798d22640 1 --2- 172.21.15.16:0/3068151341 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f479415dc30 0x7f47941f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.745+0000 7f4798d22640 1 --2- 172.21.15.16:0/3068151341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47940b1960 0x7f47941f6c30 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.745+0000 7f4798d22640 1 --2- 172.21.15.16:0/3068151341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f479406cc00 0x7f47941f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.745+0000 7f4798d22640 1 -- 172.21.15.16:0/3068151341 >> 172.21.15.16:0/3068151341 conn(0x7f47940d1900 msgr2=0x7f47940d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.746+0000 7f4798d22640 1 -- 172.21.15.16:0/3068151341 shutdown_connections 2024-04-25T10:15:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-25T10:15:15.746+0000 7f4798d22640 1 -- 172.21.15.16:0/3068151341 wait complete. 2024-04-25T10:15:15.760 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T10:15:15.760 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T10:15:15.761 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:15:15.761 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:15.769 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:15:15.769 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:15:15.769 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:15.819 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:15:15.819 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:15:15.819 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:15.831 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:15:15.832 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:15:15.832 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:15.884 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:15:15.884 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T10:15:15.885 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T10:15:15.885 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:15.912 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T10:15:15.912 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T10:15:15.912 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:15.982 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T10:15:15.982 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T10:15:15.982 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:16.051 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T10:15:16.051 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T10:15:16.051 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:16.118 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T10:15:16.119 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T10:15:16.119 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:16.162 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T10:15:16.162 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T10:15:16.162 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:16.196 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T10:15:16.197 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T10:15:16.197 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:16.271 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T10:15:16.271 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T10:15:16.271 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:16.339 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T10:15:16.339 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T10:15:16.340 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T10:15:16.340 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:16.364 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T10:15:16.364 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T10:15:16.364 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:16.413 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T10:15:16.414 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T10:15:16.414 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T10:15:16.414 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:16.426 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T10:15:16.426 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T10:15:16.426 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:16.467 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T10:15:16.467 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T10:15:16.467 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:16.519 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T10:15:16.519 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T10:15:16.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-25T10:15:16.550 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:15:16.550 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T10:15:16.866 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:15:16.866 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T10:15:16.948 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:15:16.948 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T10:15:16.998 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:15:16.998 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T10:15:17.047 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi186.front.sepia.ceph.com 2024-04-25T10:15:17.047 DEBUG:teuthology.orchestra.run.smithi186:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T10:15:17.255 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi186.front.sepia.ceph.com 2024-04-25T10:15:17.255 DEBUG:teuthology.orchestra.run.smithi186:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T10:15:17.347 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi186.front.sepia.ceph.com 2024-04-25T10:15:17.347 DEBUG:teuthology.orchestra.run.smithi186:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T10:15:17.398 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi186.front.sepia.ceph.com 2024-04-25T10:15:17.398 DEBUG:teuthology.orchestra.run.smithi186:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T10:15:17.449 INFO:tasks.ceph:Archiving mon data... 2024-04-25T10:15:17.450 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673050/data/mon.a.tgz 2024-04-25T10:15:17.450 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-04-25T10:15:17.466 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.iZ0iVdeeMy 2024-04-25T10:15:17.466 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.iZ0iVdeeMy 2024-04-25T10:15:17.633 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.iZ0iVdeeMy 2024-04-25T10:15:17.716 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.iZ0iVdeeMy is 460KB 2024-04-25T10:15:17.815 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.iZ0iVdeeMy 2024-04-25T10:15:17.829 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673050/data/mon.b.tgz 2024-04-25T10:15:17.830 DEBUG:teuthology.orchestra.run.smithi186:> mktemp 2024-04-25T10:15:17.848 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.lo5iuj8x1G 2024-04-25T10:15:17.848 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.lo5iuj8x1G 2024-04-25T10:15:18.024 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.lo5iuj8x1G 2024-04-25T10:15:18.106 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.lo5iuj8x1G is 485KB 2024-04-25T10:15:18.209 DEBUG:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.lo5iuj8x1G 2024-04-25T10:15:18.223 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673050/data/mon.c.tgz 2024-04-25T10:15:18.223 DEBUG:teuthology.orchestra.run.smithi186:> mktemp 2024-04-25T10:15:18.277 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.lPGlw5JNmh 2024-04-25T10:15:18.277 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.lPGlw5JNmh 2024-04-25T10:15:18.430 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.lPGlw5JNmh 2024-04-25T10:15:18.511 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.lPGlw5JNmh is 485KB 2024-04-25T10:15:18.611 DEBUG:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.lPGlw5JNmh 2024-04-25T10:15:18.626 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T10:15:18.627 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:15:18.629 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:15:18.710 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:15:18.744 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:15:18.745 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T10:15:18.746 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673050/remote/smithi016/crash 2024-04-25T10:15:18.747 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:15:18.776 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673050/remote/smithi186/crash 2024-04-25T10:15:18.777 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:15:18.807 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T10:15:18.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:18.821 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:18.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:18.921 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:18.944 INFO:tasks.ceph:Compressing logs... 2024-04-25T10:15:18.944 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:15:18.963 DEBUG:teuthology.orchestra.run.smithi186:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:15:19.046 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34826.log 2024-04-25T10:15:19.046 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:15:19.046 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.admin.34826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40918.log 2024-04-25T10:15:19.046 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:15:19.046 INFO:teuthology.orchestra.run.smithi186.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34826.log.gz 2024-04-25T10:15:19.046 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.1.40918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34706.log 2024-04-25T10:15:19.046 INFO:teuthology.orchestra.run.smithi186.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40918.log.gz 2024-04-25T10:15:19.046 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41313.log 2024-04-25T10:15:19.047 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.admin.34706.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T10:15:19.047 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.3.41313.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34706.log.gz 2024-04-25T10:15:19.047 INFO:teuthology.orchestra.run.smithi186.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41313.log.gz 2024-04-25T10:15:19.047 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T10:15:19.047 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T10:15:19.047 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T10:15:19.047 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T10:15:19.047 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34666.log 2024-04-25T10:15:19.047 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T10:15:19.047 INFO:teuthology.orchestra.run.smithi186.stderr: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:15:19.047 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.admin.34666.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34666.log.gz 2024-04-25T10:15:19.047 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34546.log 2024-04-25T10:15:19.047 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T10:15:19.047 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.admin.34546.log: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:15:19.048 INFO:teuthology.orchestra.run.smithi186.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34546.log.gz 2024-04-25T10:15:19.048 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34746.log 2024-04-25T10:15:19.048 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34626.log 2024-04-25T10:15:19.048 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.admin.34746.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34746.log.gz 2024-04-25T10:15:19.048 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34586.log 2024-04-25T10:15:19.048 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.admin.34626.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34626.log.gz 2024-04-25T10:15:19.048 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41317.log 2024-04-25T10:15:19.048 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.admin.34586.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34586.log.gz 2024-04-25T10:15:19.048 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T10:15:19.048 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.3.41317.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41317.log.gz 2024-04-25T10:15:19.048 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T10:15:19.048 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40923.log 2024-04-25T10:15:19.048 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34786.log 2024-04-25T10:15:19.048 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.1.40923.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40923.log.gz 2024-04-25T10:15:19.057 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.admin.34786.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34786.log.gz 2024-04-25T10:15:19.144 INFO:teuthology.orchestra.run.smithi186.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T10:15:19.163 INFO:teuthology.orchestra.run.smithi186.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T10:15:19.217 INFO:teuthology.orchestra.run.smithi186.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T10:15:19.316 INFO:teuthology.orchestra.run.smithi186.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T10:15:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46060.log 2024-04-25T10:15:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65825.log 2024-04-25T10:15:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39596.log 2024-04-25T10:15:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65825.log: gzip 87.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.46060.log.gz 2024-04-25T10:15:19.404 INFO:teuthology.orchestra.run.smithi016.stderr: -- /var/log/ceph/ceph-client.admin.48393.log 2024-04-25T10:15:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39596.log: gzip -5 89.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.65825.log.gz 2024-04-25T10:15:19.404 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.78015.log 2024-04-25T10:15:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71905.log 2024-04-25T10:15:19.405 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39596.log.gz 2024-04-25T10:15:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.78015.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48393.log.gz 2024-04-25T10:15:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58018.log 2024-04-25T10:15:19.405 INFO:teuthology.orchestra.run.smithi016.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.78015.log.gz 2024-04-25T10:15:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51056.log 2024-04-25T10:15:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71386.log 2024-04-25T10:15:19.405 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71905.log.gz 2024-04-25T10:15:19.405 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58018.log.gz 2024-04-25T10:15:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45503.log 2024-04-25T10:15:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40041.log 2024-04-25T10:15:19.405 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51056.log.gz 2024-04-25T10:15:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76871.log 2024-04-25T10:15:19.405 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71386.log.gz 2024-04-25T10:15:19.405 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45503.log.gz 2024-04-25T10:15:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.40041.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.63628.log 2024-04-25T10:15:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48510.log 2024-04-25T10:15:19.406 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40041.log.gz 2024-04-25T10:15:19.406 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.76871.log.gz 2024-04-25T10:15:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57092.log 2024-04-25T10:15:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63910.log 2024-04-25T10:15:19.406 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0%/var/log/ceph/ceph-client.admin.57092.log: -- replaced with /var/log/ceph/ceph-client.admin.63628.log.gz 2024-04-25T10:15:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70127.log 2024-04-25T10:15:19.406 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48510.log.gz 2024-04-25T10:15:19.406 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57092.log.gz 2024-04-25T10:15:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60871.log 2024-04-25T10:15:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41694.log 2024-04-25T10:15:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76432.log 2024-04-25T10:15:19.406 INFO:teuthology.orchestra.run.smithi016.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.63910.log.gz 2024-04-25T10:15:19.407 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70127.log.gz 2024-04-25T10:15:19.407 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60871.log.gz 2024-04-25T10:15:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76348.log 2024-04-25T10:15:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67562.log 2024-04-25T10:15:19.407 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41694.log.gz 2024-04-25T10:15:19.407 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76432.log.gz 2024-04-25T10:15:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74949.log 2024-04-25T10:15:19.407 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.76348.log.gz 2024-04-25T10:15:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61508.log 2024-04-25T10:15:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74949.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67562.log.gz 2024-04-25T10:15:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63193.log 2024-04-25T10:15:19.407 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74949.log.gz 2024-04-25T10:15:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70708.log 2024-04-25T10:15:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63193.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61508.log.gz 2024-04-25T10:15:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40717.log 2024-04-25T10:15:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70708.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63193.log.gz 2024-04-25T10:15:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61178.log 2024-04-25T10:15:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40717.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70708.log.gz 2024-04-25T10:15:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71003.log 2024-04-25T10:15:19.408 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0%/var/log/ceph/ceph-client.admin.61178.log: -- replaced with /var/log/ceph/ceph-client.admin.40717.log.gz 2024-04-25T10:15:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59508.log 2024-04-25T10:15:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71003.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61178.log.gz 2024-04-25T10:15:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41693.log 2024-04-25T10:15:19.408 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71003.log.gz 2024-04-25T10:15:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59616.log 2024-04-25T10:15:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46702.log 2024-04-25T10:15:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59616.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59508.log.gz 2024-04-25T10:15:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48735.log 2024-04-25T10:15:19.409 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41693.log.gz 2024-04-25T10:15:19.409 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59616.log.gz 2024-04-25T10:15:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46786.log 2024-04-25T10:15:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46702.log: /var/log/ceph/ceph-client.admin.48735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55636.log 2024-04-25T10:15:19.409 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46702.log.gz 2024-04-25T10:15:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46786.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48735.log.gz 2024-04-25T10:15:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66045.log 2024-04-25T10:15:19.409 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46786.log.gz 2024-04-25T10:15:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62310.log 2024-04-25T10:15:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55636.log: /var/log/ceph/ceph-client.admin.66045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60642.log 2024-04-25T10:15:19.409 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55636.log.gz 2024-04-25T10:15:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62310.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.54252.log 2024-04-25T10:15:19.410 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66045.log.gz 2024-04-25T10:15:19.410 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62310.log.gz 2024-04-25T10:15:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62118.log 2024-04-25T10:15:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67436.log 2024-04-25T10:15:19.410 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60642.log.gz 2024-04-25T10:15:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63976.log 2024-04-25T10:15:19.410 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54252.log.gz 2024-04-25T10:15:19.410 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62118.log.gz 2024-04-25T10:15:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61638.log 2024-04-25T10:15:19.410 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67436.log.gz 2024-04-25T10:15:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71129.log 2024-04-25T10:15:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72659.log 2024-04-25T10:15:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71129.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61638.log.gz 2024-04-25T10:15:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59031.log 2024-04-25T10:15:19.411 INFO:teuthology.orchestra.run.smithi016.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.63976.log.gz 2024-04-25T10:15:19.411 INFO:teuthology.orchestra.run.smithi016.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.71129.log.gz 2024-04-25T10:15:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41017.log 2024-04-25T10:15:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40255.log 2024-04-25T10:15:19.411 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72659.log.gz 2024-04-25T10:15:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55762.log 2024-04-25T10:15:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40255.log: gzip 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59031.log.gz 2024-04-25T10:15:19.411 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41017.log.gz 2024-04-25T10:15:19.411 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.45892.log 2024-04-25T10:15:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50209.log 2024-04-25T10:15:19.411 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40255.log.gz 2024-04-25T10:15:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74716.log 2024-04-25T10:15:19.411 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55762.log.gz 2024-04-25T10:15:19.412 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45892.log.gz 2024-04-25T10:15:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53614.log 2024-04-25T10:15:19.412 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50209.log.gz 2024-04-25T10:15:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61806.log 2024-04-25T10:15:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41021.log 2024-04-25T10:15:19.412 INFO:teuthology.orchestra.run.smithi016.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.74716.log.gz 2024-04-25T10:15:19.412 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53614.log.gz 2024-04-25T10:15:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44557.log 2024-04-25T10:15:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51378.log 2024-04-25T10:15:19.412 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61806.log.gz 2024-04-25T10:15:19.412 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9%/var/log/ceph/ceph-client.admin.44557.log: -- replaced with /var/log/ceph/ceph-client.admin.41021.log.gz 2024-04-25T10:15:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65157.log 2024-04-25T10:15:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51378.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44557.log.gz 2024-04-25T10:15:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67096.log 2024-04-25T10:15:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.65157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64818.log 2024-04-25T10:15:19.413 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65157.log.gz 2024-04-25T10:15:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67096.log: 91.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.51378.log.gz 2024-04-25T10:15:19.413 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48253.log 2024-04-25T10:15:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64818.log: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.67096.log.gz 2024-04-25T10:15:19.413 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.70498.log 2024-04-25T10:15:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48253.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64818.log.gz 2024-04-25T10:15:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41020.log 2024-04-25T10:15:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70498.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48253.log.gz 2024-04-25T10:15:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72031.log 2024-04-25T10:15:19.413 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70498.log.gz 2024-04-25T10:15:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47772.log 2024-04-25T10:15:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62940.log 2024-04-25T10:15:19.413 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41020.log.gz 2024-04-25T10:15:19.414 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72031.log.gz 2024-04-25T10:15:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.0.47772.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45172.log 2024-04-25T10:15:19.414 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47772.log.gz 2024-04-25T10:15:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41977.log 2024-04-25T10:15:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42061.log 2024-04-25T10:15:19.414 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62940.log.gz 2024-04-25T10:15:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41977.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45172.log.gz 2024-04-25T10:15:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42187.log 2024-04-25T10:15:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68297.log 2024-04-25T10:15:19.414 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41977.log.gz 2024-04-25T10:15:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64662.log 2024-04-25T10:15:19.414 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42061.log.gz 2024-04-25T10:15:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68297.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42187.log.gz 2024-04-25T10:15:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44998.log 2024-04-25T10:15:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64662.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68297.log.gz 2024-04-25T10:15:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62202.log 2024-04-25T10:15:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64904.log 2024-04-25T10:15:19.415 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44998.log.gz/var/log/ceph/ceph-client.admin.62202.log: 2024-04-25T10:15:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63151.log 2024-04-25T10:15:19.415 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62202.log.gz 2024-04-25T10:15:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48436.log 2024-04-25T10:15:19.415 INFO:teuthology.orchestra.run.smithi016.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.64662.log.gz 2024-04-25T10:15:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62737.log 2024-04-25T10:15:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48436.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:15:19.415 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63151.log.gz 2024-04-25T10:15:19.415 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64904.log.gz 2024-04-25T10:15:19.415 INFO:teuthology.orchestra.run.smithi016.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.48436.log.gz 2024-04-25T10:15:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64578.log 2024-04-25T10:15:19.416 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62737.log.gz 2024-04-25T10:15:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44095.log 2024-04-25T10:15:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.64578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51444.log 2024-04-25T10:15:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68686.log 2024-04-25T10:15:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51444.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44095.log.gz 2024-04-25T10:15:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64393.log 2024-04-25T10:15:19.416 INFO:teuthology.orchestra.run.smithi016.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.64578.log.gz 2024-04-25T10:15:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69113.log 2024-04-25T10:15:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.68686.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51444.log.gz 2024-04-25T10:15:19.416 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68686.log.gz 2024-04-25T10:15:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46558.log 2024-04-25T10:15:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69113.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64393.log.gz 2024-04-25T10:15:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47137.log 2024-04-25T10:15:19.416 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69113.log.gz 2024-04-25T10:15:19.417 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77891.log 2024-04-25T10:15:19.417 INFO:teuthology.orchestra.run.smithi016.stderr: 86.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:15:19.417 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46558.log.gz 2024-04-25T10:15:19.417 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74428.log 2024-04-25T10:15:19.417 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55552.log 2024-04-25T10:15:19.417 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47137.log.gz 2024-04-25T10:15:19.417 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77891.log.gz 2024-04-25T10:15:19.417 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44668.log 2024-04-25T10:15:19.417 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74428.log.gz 2024-04-25T10:15:19.417 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77765.log 2024-04-25T10:15:19.417 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1%/var/log/ceph/ceph-client.admin.44668.log: -- replaced with /var/log/ceph/ceph-client.admin.55552.log.gz 2024-04-25T10:15:19.417 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54793.log 2024-04-25T10:15:19.417 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62779.log 2024-04-25T10:15:19.417 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44668.log.gz 2024-04-25T10:15:19.417 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54793.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.77765.log.gz 2024-04-25T10:15:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57471.log 2024-04-25T10:15:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72115.log 2024-04-25T10:15:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.57471.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62779.log.gz 2024-04-25T10:15:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73798.log 2024-04-25T10:15:19.418 INFO:teuthology.orchestra.run.smithi016.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.54793.log.gz 2024-04-25T10:15:19.418 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57471.log.gz 2024-04-25T10:15:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60264.log 2024-04-25T10:15:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42661.log 2024-04-25T10:15:19.418 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73798.log.gz 2024-04-25T10:15:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47179.log 2024-04-25T10:15:19.418 INFO:teuthology.orchestra.run.smithi016.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.72115.log.gz 2024-04-25T10:15:19.418 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60264.log.gz 2024-04-25T10:15:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50335.log 2024-04-25T10:15:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75122.log 2024-04-25T10:15:19.419 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42661.log.gz 2024-04-25T10:15:19.419 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47179.log.gz 2024-04-25T10:15:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41023.log 2024-04-25T10:15:19.419 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50335.log.gz 2024-04-25T10:15:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52927.log 2024-04-25T10:15:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60474.log 2024-04-25T10:15:19.419 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41023.log.gz 2024-04-25T10:15:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49911.log 2024-04-25T10:15:19.419 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52927.log.gz 2024-04-25T10:15:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42313.log 2024-04-25T10:15:19.419 INFO:teuthology.orchestra.run.smithi016.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.75122.log.gz 2024-04-25T10:15:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41361.log 2024-04-25T10:15:19.419 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60474.log.gz 2024-04-25T10:15:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42313.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49911.log.gz 2024-04-25T10:15:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60044.log 2024-04-25T10:15:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41361.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42313.log.gz 2024-04-25T10:15:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51132.log 2024-04-25T10:15:19.420 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41361.log.gz 2024-04-25T10:15:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44914.log 2024-04-25T10:15:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51132.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60044.log.gz 2024-04-25T10:15:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62982.log 2024-04-25T10:15:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44914.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 88.8% 2024-04-25T10:15:19.420 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51132.log.gz 2024-04-25T10:15:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62982.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44914.log.gz 2024-04-25T10:15:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60516.log 2024-04-25T10:15:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.2.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62982.log.gz 2024-04-25T10:15:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52334.log 2024-04-25T10:15:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64513.log 2024-04-25T10:15:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52334.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60516.log.gz 2024-04-25T10:15:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57176.log 2024-04-25T10:15:19.421 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52334.log.gz 2024-04-25T10:15:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55594.log 2024-04-25T10:15:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71863.log 2024-04-25T10:15:19.421 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64513.log.gz 2024-04-25T10:15:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55594.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57176.log.gz 2024-04-25T10:15:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48921.log 2024-04-25T10:15:19.421 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55594.log.gz 2024-04-25T10:15:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57736.log 2024-04-25T10:15:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60171.log 2024-04-25T10:15:19.422 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71863.log.gz 2024-04-25T10:15:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57736.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48921.log.gz 2024-04-25T10:15:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77849.log 2024-04-25T10:15:19.422 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57736.log.gz 2024-04-25T10:15:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56882.log 2024-04-25T10:15:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77849.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.60171.log.gz -5 2024-04-25T10:15:19.422 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.0.74179.log 2024-04-25T10:15:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56882.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77849.log.gz 2024-04-25T10:15:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77555.log 2024-04-25T10:15:19.422 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56882.log.gz 2024-04-25T10:15:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.74179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67688.log 2024-04-25T10:15:19.423 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74179.log.gz 2024-04-25T10:15:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39999.log 2024-04-25T10:15:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67688.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77555.log.gz 2024-04-25T10:15:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71045.log 2024-04-25T10:15:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53824.log 2024-04-25T10:15:19.423 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67688.log.gz 2024-04-25T10:15:19.423 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39999.log.gz 2024-04-25T10:15:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53053.log 2024-04-25T10:15:19.423 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.71045.log.gz 2024-04-25T10:15:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51938.log 2024-04-25T10:15:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50125.log 2024-04-25T10:15:19.423 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53053.log.gz 2024-04-25T10:15:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51938.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.53824.log.gz 2024-04-25T10:15:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49703.log 2024-04-25T10:15:19.424 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68682.log 2024-04-25T10:15:19.424 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9%/var/log/ceph/ceph-client.admin.50125.log: -- replaced with /var/log/ceph/ceph-client.admin.51938.log.gz 2024-04-25T10:15:19.424 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53095.log 2024-04-25T10:15:19.424 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50125.log.gz 2024-04-25T10:15:19.424 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.68682.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49703.log.gz 2024-04-25T10:15:19.424 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56338.log 2024-04-25T10:15:19.424 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68682.log.gz 2024-04-25T10:15:19.424 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55492.log 2024-04-25T10:15:19.424 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56338.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53095.log.gzgzip 2024-04-25T10:15:19.424 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47053.log 2024-04-25T10:15:19.424 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56338.log.gz 2024-04-25T10:15:19.424 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45595.log 2024-04-25T10:15:19.424 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75293.log 2024-04-25T10:15:19.424 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55492.log.gz 2024-04-25T10:15:19.424 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45595.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47053.log.gz 2024-04-25T10:15:19.425 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74827.log 2024-04-25T10:15:19.425 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45595.log.gz 2024-04-25T10:15:19.425 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58879.log 2024-04-25T10:15:19.425 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56756.log 2024-04-25T10:15:19.425 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75293.log.gz 2024-04-25T10:15:19.425 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58879.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74827.log.gz 2024-04-25T10:15:19.425 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66297.log 2024-04-25T10:15:19.425 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59247.log 2024-04-25T10:15:19.425 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58879.log.gz 2024-04-25T10:15:19.425 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66297.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56756.log.gz 2024-04-25T10:15:19.425 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57008.log 2024-04-25T10:15:19.425 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66297.log.gz 2024-04-25T10:15:19.425 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45639.log 2024-04-25T10:15:19.426 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.57008.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.55230.log 2024-04-25T10:15:19.426 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45639.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57008.log.gz 2024-04-25T10:15:19.426 INFO:teuthology.orchestra.run.smithi016.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.59247.log.gz 2024-04-25T10:15:19.426 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41695.log 2024-04-25T10:15:19.426 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:15:19.426 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45639.log.gz 2024-04-25T10:15:19.426 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55230.log: /var/log/ceph/ceph-client.admin.41695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48473.log 2024-04-25T10:15:19.426 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55230.log.gz 2024-04-25T10:15:19.426 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41695.log.gz 2024-04-25T10:15:19.426 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67224.log 2024-04-25T10:15:19.426 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75038.log 2024-04-25T10:15:19.426 INFO:teuthology.orchestra.run.smithi016.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.48473.log.gz 2024-04-25T10:15:19.426 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69029.log 2024-04-25T10:15:19.427 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56033.log 2024-04-25T10:15:19.427 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7%/var/log/ceph/ceph-client.admin.75038.log: -- replaced with /var/log/ceph/ceph-client.admin.67224.log.gz 2024-04-25T10:15:19.427 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70877.log 2024-04-25T10:15:19.427 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5%/var/log/ceph/ceph-client.admin.56033.log: -- replaced with /var/log/ceph/ceph-client.admin.69029.log.gz 2024-04-25T10:15:19.427 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55804.log 2024-04-25T10:15:19.427 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70877.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56033.log.gz 2024-04-25T10:15:19.427 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67987.log 2024-04-25T10:15:19.427 INFO:teuthology.orchestra.run.smithi016.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.75038.log.gz 2024-04-25T10:15:19.427 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47305.log 2024-04-25T10:15:19.427 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70877.log.gz 2024-04-25T10:15:19.427 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55804.log.gz 2024-04-25T10:15:19.427 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.67987.log: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:15:19.427 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55272.log 2024-04-25T10:15:19.427 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67987.log.gz 2024-04-25T10:15:19.427 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57694.log 2024-04-25T10:15:19.428 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47305.log: /var/log/ceph/ceph-client.admin.55272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70456.log 2024-04-25T10:15:19.428 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47305.log.gz 2024-04-25T10:15:19.428 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77123.log 88.8% 2024-04-25T10:15:19.428 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55272.log.gz 2024-04-25T10:15:19.428 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70456.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57694.log.gz 2024-04-25T10:15:19.428 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65304.log 2024-04-25T10:15:19.428 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70456.log.gz 2024-04-25T10:15:19.428 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41893.log 2024-04-25T10:15:19.428 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77123.log.gz 2024-04-25T10:15:19.428 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73884.log 2024-04-25T10:15:19.428 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57652.log 2024-04-25T10:15:19.428 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73884.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41893.log.gz 2024-04-25T10:15:19.428 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- 90.9% /var/log/ceph/ceph-client.admin.43139.log 2024-04-25T10:15:19.429 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65304.log.gz 2024-04-25T10:15:19.429 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73884.log.gz 2024-04-25T10:15:19.429 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68171.log 2024-04-25T10:15:19.429 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57652.log.gz 2024-04-25T10:15:19.429 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67352.log 2024-04-25T10:15:19.429 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68171.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43139.log.gz 2024-04-25T10:15:19.429 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53318.log 2024-04-25T10:15:19.429 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67352.log: gzip -5 88.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.68171.log.gz /var/log/ceph/ceph-client.admin.44263.log 2024-04-25T10:15:19.429 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.429 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4%/var/log/ceph/ceph-client.admin.53318.log: -- replaced with /var/log/ceph/ceph-client.admin.67352.log.gz 2024-04-25T10:15:19.429 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50083.log 2024-04-25T10:15:19.429 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53318.log.gz 2024-04-25T10:15:19.430 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60829.log 2024-04-25T10:15:19.430 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44263.log.gz 2024-04-25T10:15:19.430 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49131.log 2024-04-25T10:15:19.430 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50083.log.gz 2024-04-25T10:15:19.430 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48693.log 2024-04-25T10:15:19.430 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49528.log 2024-04-25T10:15:19.431 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60829.log.gz 2024-04-25T10:15:19.431 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49131.log.gz 2024-04-25T10:15:19.431 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60744.log 2024-04-25T10:15:19.431 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48693.log.gz 2024-04-25T10:15:19.431 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59139.log 2024-04-25T10:15:19.431 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60744.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49528.log.gz 2024-04-25T10:15:19.431 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43657.log 2024-04-25T10:15:19.431 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60744.log.gz 2024-04-25T10:15:19.431 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74883.log 2024-04-25T10:15:19.431 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77429.log 2024-04-25T10:15:19.431 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74883.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43657.log.gz 2024-04-25T10:15:19.431 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60787.log 2024-04-25T10:15:19.432 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59139.log.gz 2024-04-25T10:15:19.432 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46967.log 2024-04-25T10:15:19.432 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5%/var/log/ceph/ceph-client.admin.60787.log: -- replaced with /var/log/ceph/ceph-client.admin.77429.log.gz 2024-04-25T10:15:19.432 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.74883.log.gz 2024-04-25T10:15:19.432 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41025.log 2024-04-25T10:15:19.432 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46967.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60787.log.gz 2024-04-25T10:15:19.432 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60390.log 2024-04-25T10:15:19.432 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41025.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46967.log.gz 2024-04-25T10:15:19.432 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44431.log 2024-04-25T10:15:19.432 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41025.log.gz 2024-04-25T10:15:19.432 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71821.log 2024-04-25T10:15:19.432 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58676.log 2024-04-25T10:15:19.432 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60390.log.gz 2024-04-25T10:15:19.432 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71821.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44431.log.gz 2024-04-25T10:15:19.433 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46874.log 2024-04-25T10:15:19.433 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71821.log.gz 2024-04-25T10:15:19.433 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58676.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T10:15:19.433 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75335.log 2024-04-25T10:15:19.433 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58676.log.gz 2024-04-25T10:15:19.433 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46874.log.gz 2024-04-25T10:15:19.433 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57897.log 2024-04-25T10:15:19.433 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62497.log 2024-04-25T10:15:19.433 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57897.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75335.log.gz 2024-04-25T10:15:19.433 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50671.log 2024-04-25T10:15:19.433 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57897.log.gz 2024-04-25T10:15:19.433 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76558.log 2024-04-25T10:15:19.433 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.62497.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62497.log.gz 2024-04-25T10:15:19.433 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45766.log 2024-04-25T10:15:19.433 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47095.log 2024-04-25T10:15:19.434 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45766.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76558.log.gz 2024-04-25T10:15:19.434 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50671.log.gz 2024-04-25T10:15:19.434 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39677.log 2024-04-25T10:15:19.434 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7%/var/log/ceph/ceph-client.admin.47095.log: -- replaced with /var/log/ceph/ceph-client.admin.45766.log.gz 2024-04-25T10:15:19.434 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71428.log 2024-04-25T10:15:19.434 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47095.log.gz 2024-04-25T10:15:19.434 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63042.log 2024-04-25T10:15:19.434 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39677.log: /var/log/ceph/ceph-client.admin.71428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51634.log 2024-04-25T10:15:19.434 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39677.log.gz 2024-04-25T10:15:19.434 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71428.log.gz 2024-04-25T10:15:19.434 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52502.log 2024-04-25T10:15:19.434 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66670.log 2024-04-25T10:15:19.434 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51634.log: /var/log/ceph/ceph-client.admin.52502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73379.log 2024-04-25T10:15:19.434 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51634.log.gz 2024-04-25T10:15:19.435 INFO:teuthology.orchestra.run.smithi016.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.63042.log.gz 2024-04-25T10:15:19.435 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52502.log.gz 2024-04-25T10:15:19.435 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60955.log 2024-04-25T10:15:19.435 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62695.log 2024-04-25T10:15:19.435 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66670.log.gz 2024-04-25T10:15:19.435 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60955.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73379.log.gz 2024-04-25T10:15:19.435 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42229.log 2024-04-25T10:15:19.435 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60955.log.gz 2024-04-25T10:15:19.435 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53698.log 2024-04-25T10:15:19.435 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62695.log: /var/log/ceph/ceph-client.admin.42229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74512.log 2024-04-25T10:15:19.435 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62695.log.gz 2024-04-25T10:15:19.435 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53698.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42229.log.gz 2024-04-25T10:15:19.435 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46432.log 2024-04-25T10:15:19.435 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53698.log.gz 2024-04-25T10:15:19.436 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54727.log 2024-04-25T10:15:19.436 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74512.log: /var/log/ceph/ceph-client.admin.46432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45298.log 2024-04-25T10:15:19.436 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74512.log.gz 2024-04-25T10:15:19.436 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41349.log 2024-04-25T10:15:19.436 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46432.log.gz 2024-04-25T10:15:19.436 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64435.log 2024-04-25T10:15:19.436 INFO:teuthology.orchestra.run.smithi016.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.54727.log.gz 2024-04-25T10:15:19.436 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41349.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45298.log.gzgzip 2024-04-25T10:15:19.436 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41019.log 2024-04-25T10:15:19.436 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41349.log.gz 2024-04-25T10:15:19.436 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51079.log 2024-04-25T10:15:19.436 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54305.log 2024-04-25T10:15:19.436 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41019.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64435.log.gz 2024-04-25T10:15:19.436 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47515.log 2024-04-25T10:15:19.436 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41019.log.gz 2024-04-25T10:15:19.437 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.51079.log.gz 2024-04-25T10:15:19.437 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61339.log 2024-04-25T10:15:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54305.log: /var/log/ceph/ceph-client.admin.47515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77345.log 2024-04-25T10:15:19.437 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54305.log.gz 2024-04-25T10:15:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44836.log 2024-04-25T10:15:19.437 INFO:teuthology.orchestra.run.smithi016.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.47515.log.gz 2024-04-25T10:15:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59574.log 2024-04-25T10:15:19.437 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61339.log.gz 2024-04-25T10:15:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70666.log 2024-04-25T10:15:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44836.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77345.log.gz 2024-04-25T10:15:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66502.log 2024-04-25T10:15:19.437 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44836.log.gz 2024-04-25T10:15:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46102.log 2024-04-25T10:15:19.437 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59574.log.gz 2024-04-25T10:15:19.437 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70666.log.gz 2024-04-25T10:15:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48777.log 2024-04-25T10:15:19.438 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66502.log.gz 2024-04-25T10:15:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62821.log 2024-04-25T10:15:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46102.log: /var/log/ceph/ceph-client.admin.48777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54077.log 2024-04-25T10:15:19.438 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46102.log.gz 2024-04-25T10:15:19.438 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48777.log.gz 2024-04-25T10:15:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73591.log 2024-04-25T10:15:19.438 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62821.log.gz 2024-04-25T10:15:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52124.log 2024-04-25T10:15:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.54077.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54077.log.gz 2024-04-25T10:15:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71642.log 2024-04-25T10:15:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73591.log: /var/log/ceph/ceph-client.admin.52124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72372.log 2024-04-25T10:15:19.438 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73591.log.gz 2024-04-25T10:15:19.439 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52124.log.gz/var/log/ceph/ceph-client.admin.71642.log: 2024-04-25T10:15:19.439 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44473.log 2024-04-25T10:15:19.439 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69685.log 2024-04-25T10:15:19.439 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.72372.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71642.log.gz 2024-04-25T10:15:19.439 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72372.log.gz 2024-04-25T10:15:19.439 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50928.log 2024-04-25T10:15:19.439 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69685.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44473.log.gz 2024-04-25T10:15:19.440 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72785.log 2024-04-25T10:15:19.440 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69685.log.gz 2024-04-25T10:15:19.440 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66460.log 2024-04-25T10:15:19.440 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.50928.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50928.log.gz 2024-04-25T10:15:19.440 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49868.log 2024-04-25T10:15:19.440 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66460.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.72785.log.gz 2024-04-25T10:15:19.441 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T10:15:19.441 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41356.log 2024-04-25T10:15:19.441 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66460.log.gz 2024-04-25T10:15:19.441 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49868.log: /var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49191.log 2024-04-25T10:15:19.441 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49868.log.gz 2024-04-25T10:15:19.441 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77723.log 2024-04-25T10:15:19.441 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41356.log: /var/log/ceph/ceph-client.admin.49191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57475.log 2024-04-25T10:15:19.441 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41356.log.gz 2024-04-25T10:15:19.441 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5%gzip -5/var/log/ceph/ceph-client.admin.77723.log: --verbose -- /var/log/ceph/ceph-client.admin.44305.log 2024-04-25T10:15:19.441 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49191.log.gz 2024-04-25T10:15:19.441 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.57475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41360.log 2024-04-25T10:15:19.442 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77723.log.gz 2024-04-25T10:15:19.442 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57475.log.gz 2024-04-25T10:15:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52292.log 2024-04-25T10:15:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41360.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.44305.log.gz -5 2024-04-25T10:15:19.442 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52460.log 2024-04-25T10:15:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58945.log 2024-04-25T10:15:19.442 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41360.log.gz 2024-04-25T10:15:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72862.log 2024-04-25T10:15:19.442 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52292.log.gz 2024-04-25T10:15:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57134.log 2024-04-25T10:15:19.442 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52460.log.gz 2024-04-25T10:15:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58945.log: /var/log/ceph/ceph-client.admin.72862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41016.log 2024-04-25T10:15:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56924.log 90.8% 2024-04-25T10:15:19.443 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58945.log.gz 91.1% -- replaced with /var/log/ceph/ceph-client.admin.72862.log.gz 2024-04-25T10:15:19.443 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57134.log.gz 2024-04-25T10:15:19.443 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69769.log 2024-04-25T10:15:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56924.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41016.log.gz 2024-04-25T10:15:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69190.log 2024-04-25T10:15:19.443 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56924.log.gz 2024-04-25T10:15:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63760.log 2024-04-25T10:15:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76003.log 2024-04-25T10:15:19.444 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69190.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.69769.log.gz 2024-04-25T10:15:19.444 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44053.log 2024-04-25T10:15:19.444 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69190.log.gz 2024-04-25T10:15:19.444 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63760.log.gz/var/log/ceph/ceph-client.admin.76003.log: 2024-04-25T10:15:19.444 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70750.log 2024-04-25T10:15:19.444 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44053.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76003.log.gz 2024-04-25T10:15:19.444 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63534.log 2024-04-25T10:15:19.444 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44053.log.gz 2024-04-25T10:15:19.444 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47347.log 2024-04-25T10:15:19.444 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.63534.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63534.log.gz 2024-04-25T10:15:19.444 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70750.log.gz 2024-04-25T10:15:19.444 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67268.log 2024-04-25T10:15:19.444 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40759.log 2024-04-25T10:15:19.444 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47347.log: /var/log/ceph/ceph-client.admin.67268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48963.log 2024-04-25T10:15:19.445 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47347.log.gz 2024-04-25T10:15:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75678.log 2024-04-25T10:15:19.445 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67268.log.gz 2024-04-25T10:15:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48963.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40759.log.gz 2024-04-25T10:15:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50167.log 2024-04-25T10:15:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50629.log 2024-04-25T10:15:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.75678.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48963.log.gz 2024-04-25T10:15:19.445 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75678.log.gz 2024-04-25T10:15:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68945.log 2024-04-25T10:15:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50629.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T10:15:19.445 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50167.log.gz 2024-04-25T10:15:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68945.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50629.log.gz 2024-04-25T10:15:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53183.log 2024-04-25T10:15:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T10:15:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.1.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68945.log.gz 2024-04-25T10:15:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49360.log 2024-04-25T10:15:19.446 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53183.log.gz 2024-04-25T10:15:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74790.log 2024-04-25T10:15:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.49360.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.65783.log 2024-04-25T10:15:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74790.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49360.log.gz 2024-04-25T10:15:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41357.log 2024-04-25T10:15:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65783.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.74790.log.gz 2024-04-25T10:15:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44710.log 2024-04-25T10:15:19.446 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6%/var/log/ceph/ceph-client.admin.41357.log: -- replaced with /var/log/ceph/ceph-client.admin.65783.log.gz 2024-04-25T10:15:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41357.log.gz 2024-04-25T10:15:19.446 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.42523.log 2024-04-25T10:15:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44710.log: gzip -5 --verbose -- 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44710.log.gz 2024-04-25T10:15:19.447 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.55720.log 2024-04-25T10:15:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47768.log 2024-04-25T10:15:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75379.log 2024-04-25T10:15:19.447 INFO:teuthology.orchestra.run.smithi016.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.42523.log.gz 2024-04-25T10:15:19.447 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55720.log.gz 2024-04-25T10:15:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.47768.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47768.log.gz 2024-04-25T10:15:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46474.log 2024-04-25T10:15:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.75379.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.64309.log 2024-04-25T10:15:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46474.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75379.log.gz 2024-04-25T10:15:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74183.log 2024-04-25T10:15:19.447 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46474.log.gz 2024-04-25T10:15:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58802.log 2024-04-25T10:15:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64309.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64309.log.gz 2024-04-25T10:15:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.74183.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74183.log.gz 2024-04-25T10:15:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54511.log 2024-04-25T10:15:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58452.log 2024-04-25T10:15:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54511.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58802.log.gz 2024-04-25T10:15:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44137.log 2024-04-25T10:15:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.58452.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58452.log.gz 2024-04-25T10:15:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T10:15:19.448 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54511.log.gz 2024-04-25T10:15:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45460.log 2024-04-25T10:15:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44137.log: /var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67310.log 2024-04-25T10:15:19.449 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44137.log.gz 2024-04-25T10:15:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52586.log 2024-04-25T10:15:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67310.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45460.log.gz 2024-04-25T10:15:19.449 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67310.log.gz 94.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T10:15:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:gzip 2024-04-25T10:15:19.449 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.67138.log 2024-04-25T10:15:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.52586.log: -5 --verbose -- /var/log/ceph/ceph-client.0.75674.log 2024-04-25T10:15:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67138.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52586.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76129.log 2024-04-25T10:15:19.449 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.450 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67138.log.gz 2024-04-25T10:15:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.75674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45418.log 2024-04-25T10:15:19.450 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75674.log.gz 2024-04-25T10:15:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65238.log 2024-04-25T10:15:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65741.log 2024-04-25T10:15:19.450 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76129.log.gz 2024-04-25T10:15:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45418.log: /var/log/ceph/ceph-client.admin.65238.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45418.log.gz 2024-04-25T10:15:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75487.log 2024-04-25T10:15:19.451 INFO:teuthology.orchestra.run.smithi016.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.65238.log.gz 2024-04-25T10:15:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65741.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65741.log.gz 2024-04-25T10:15:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61466.log 2024-04-25T10:15:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71554.log 2024-04-25T10:15:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61466.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75487.log.gz 2024-04-25T10:15:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54081.log 2024-04-25T10:15:19.452 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61466.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47431.log 2024-04-25T10:15:19.452 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71554.log: /var/log/ceph/ceph-client.0.54081.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54081.log.gz 2024-04-25T10:15:19.452 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71554.log.gz 2024-04-25T10:15:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71693.log 2024-04-25T10:15:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70582.log 2024-04-25T10:15:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71693.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47431.log.gz 2024-04-25T10:15:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56798.log 2024-04-25T10:15:19.452 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71693.log.gz 2024-04-25T10:15:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73119.log 2024-04-25T10:15:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70582.log: /var/log/ceph/ceph-client.admin.56798.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70582.log.gz 2024-04-25T10:15:19.453 INFO:teuthology.orchestra.run.smithi016.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.56798.log.gz 2024-04-25T10:15:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47263.log 2024-04-25T10:15:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.73119.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.0.73119.log.gz 2024-04-25T10:15:19.453 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41359.log 2024-04-25T10:15:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75421.log 2024-04-25T10:15:19.453 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47263.log.gz 2024-04-25T10:15:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51676.log 2024-04-25T10:15:19.453 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0%/var/log/ceph/ceph-client.admin.75421.log: -- replaced with /var/log/ceph/ceph-client.admin.41359.log.gz 2024-04-25T10:15:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70330.log 2024-04-25T10:15:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48879.log 2024-04-25T10:15:19.453 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.75421.log.gz 2024-04-25T10:15:19.453 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51676.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50737.log 2024-04-25T10:15:19.453 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70330.log: /var/log/ceph/ceph-client.admin.48879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76516.log 2024-04-25T10:15:19.453 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1%/var/log/ceph/ceph-client.admin.50737.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48879.log.gz 2024-04-25T10:15:19.454 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70330.log.gz 2024-04-25T10:15:19.454 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50737.log.gz 2024-04-25T10:15:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55949.log 2024-04-25T10:15:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69545.log 2024-04-25T10:15:19.454 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50587.log 2024-04-25T10:15:19.454 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76516.log.gz 2024-04-25T10:15:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55949.log: /var/log/ceph/ceph-client.admin.69545.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55949.log.gz 2024-04-25T10:15:19.454 INFO:teuthology.orchestra.run.smithi016.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.69545.log.gz 2024-04-25T10:15:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77639.log 2024-04-25T10:15:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.50587.log: /var/log/ceph/ceph-client.admin.75961.log 2024-04-25T10:15:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77639.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50587.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34704.log 2024-04-25T10:15:19.454 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.454 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77639.log.gz 2024-04-25T10:15:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.75961.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40169.log 2024-04-25T10:15:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73992.log 2024-04-25T10:15:19.455 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75961.log.gz 2024-04-25T10:15:19.455 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34704.log.gz 2024-04-25T10:15:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.43444.log 2024-04-25T10:15:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72928.log 2024-04-25T10:15:19.455 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40169.log.gz/var/log/ceph/ceph-client.2.43444.log: 2024-04-25T10:15:19.455 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.43444.log.gz 89.3% 2024-04-25T10:15:19.455 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73992.log.gz 2024-04-25T10:15:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69283.log 2024-04-25T10:15:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67180.log 2024-04-25T10:15:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69283.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72928.log.gz 2024-04-25T10:15:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64127.log 2024-04-25T10:15:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67180.log: 91.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66213.log 2024-04-25T10:15:19.455 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69283.log.gz 2024-04-25T10:15:19.456 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67180.log.gz 2024-04-25T10:15:19.456 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5/var/log/ceph/ceph-client.admin.64127.log: --verbose -- /var/log/ceph/ceph-client.0.58456.log 2024-04-25T10:15:19.456 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66171.log 2024-04-25T10:15:19.456 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64127.log.gz 2024-04-25T10:15:19.456 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66213.log.gz 2024-04-25T10:15:19.456 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.58456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64970.log 2024-04-25T10:15:19.456 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58456.log.gz 2024-04-25T10:15:19.456 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46390.log 2024-04-25T10:15:19.456 INFO:teuthology.orchestra.run.smithi016.stderr:gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66171.log.gz 2024-04-25T10:15:19.456 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56075.log 2024-04-25T10:15:19.456 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46390.log: /var/log/ceph/ceph-client.admin.64970.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46390.log.gz 2024-04-25T10:15:19.456 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64970.log.gz 2024-04-25T10:15:19.456 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72701.log 2024-04-25T10:15:19.456 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42103.log 2024-04-25T10:15:19.457 INFO:teuthology.orchestra.run.smithi016.stderr: 87.0%/var/log/ceph/ceph-client.admin.72701.log: -- replaced with /var/log/ceph/ceph-client.admin.56075.log.gz 2024-04-25T10:15:19.457 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57962.log 2024-04-25T10:15:19.457 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72701.log.gz 2024-04-25T10:15:19.457 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55146.log 2024-04-25T10:15:19.457 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42103.log: /var/log/ceph/ceph-client.admin.57962.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42103.log.gz 2024-04-25T10:15:19.457 INFO:teuthology.orchestra.run.smithi016.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.57962.log.gz 2024-04-25T10:15:19.457 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56712.log 2024-04-25T10:15:19.457 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66339.log 2024-04-25T10:15:19.457 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56712.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55146.log.gz 2024-04-25T10:15:19.458 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42439.log 2024-04-25T10:15:19.458 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56712.log.gz 2024-04-25T10:15:19.458 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55450.log 2024-04-25T10:15:19.458 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66339.log: /var/log/ceph/ceph-client.admin.42439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44794.log 2024-04-25T10:15:19.458 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55450.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66339.log.gz 2024-04-25T10:15:19.458 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42439.log.gz 2024-04-25T10:15:19.458 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55450.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48085.log 2024-04-25T10:15:19.458 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.458 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.44794.log: /var/log/ceph/ceph-client.admin.71735.log 2024-04-25T10:15:19.458 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48085.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50251.log 2024-04-25T10:15:19.458 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44794.log.gz 2024-04-25T10:15:19.458 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48085.log.gz 2024-04-25T10:15:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42481.log 2024-04-25T10:15:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61552.log 2024-04-25T10:15:19.459 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71735.log.gz 2024-04-25T10:15:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42481.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53890.log 2024-04-25T10:15:19.459 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50251.log.gz 2024-04-25T10:15:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52670.log 2024-04-25T10:15:19.459 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42481.log.gz 2024-04-25T10:15:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53890.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61552.log.gz 2024-04-25T10:15:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53890.log.gz 2024-04-25T10:15:19.459 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.68255.log 2024-04-25T10:15:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52670.log: gzip -5 --verbose 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52670.log.gz 2024-04-25T10:15:19.459 INFO:teuthology.orchestra.run.smithi016.stderr: -- /var/log/ceph/ceph.tmp-client.admin.28369.log 2024-04-25T10:15:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51718.log 2024-04-25T10:15:19.459 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68255.log.gz 2024-04-25T10:15:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.tmp-client.admin.28369.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28369.log.gz 2024-04-25T10:15:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60129.log 2024-04-25T10:15:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.51718.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43927.log 2024-04-25T10:15:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60129.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51718.log.gz 2024-04-25T10:15:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53139.log 2024-04-25T10:15:19.460 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60129.log.gz 2024-04-25T10:15:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63802.log 2024-04-25T10:15:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43927.log: /var/log/ceph/ceph-client.admin.53139.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43927.log.gz 2024-04-25T10:15:19.460 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53139.log.gz 2024-04-25T10:15:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T10:15:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71087.log 2024-04-25T10:15:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63235.log 2024-04-25T10:15:19.460 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63802.log.gz 2024-04-25T10:15:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46744.log 2024-04-25T10:15:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63235.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71087.log.gz 2024-04-25T10:15:19.461 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63235.log.gz 2024-04-25T10:15:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73840.log 2024-04-25T10:15:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56117.log 2024-04-25T10:15:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73840.log: /var/log/ceph/ceph-client.admin.46744.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46744.log.gz 2024-04-25T10:15:19.461 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73840.log.gz 2024-04-25T10:15:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40423.log 2024-04-25T10:15:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50419.log 2024-04-25T10:15:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56117.log: /var/log/ceph/ceph-client.admin.40423.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56117.log.gz 2024-04-25T10:15:19.461 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40423.log.gz 2024-04-25T10:15:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48819.log 2024-04-25T10:15:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52845.log 2024-04-25T10:15:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48819.log: /var/log/ceph/ceph-client.admin.50419.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50419.log.gz 2024-04-25T10:15:19.461 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48819.log.gz 2024-04-25T10:15:19.462 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76264.log 2024-04-25T10:15:19.462 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49089.log 2024-04-25T10:15:19.462 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52845.log.gz 2024-04-25T10:15:19.462 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52208.log 2024-04-25T10:15:19.462 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49089.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76264.log.gz 2024-04-25T10:15:19.462 INFO:teuthology.orchestra.run.smithi016.stderr:gzip 89.4% -5 --verbose -- /var/log/ceph/ceph-client.admin.65496.log 2024-04-25T10:15:19.462 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49089.log.gz 2024-04-25T10:15:19.462 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56159.log 2024-04-25T10:15:19.462 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65496.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52208.log.gz 2024-04-25T10:15:19.462 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73688.log 2024-04-25T10:15:19.463 INFO:teuthology.orchestra.run.smithi016.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.65496.log.gz 2024-04-25T10:15:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60913.log 2024-04-25T10:15:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73688.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56159.log.gz 2024-04-25T10:15:19.463 INFO:teuthology.orchestra.run.smithi016.stderr: 89.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.73688.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.67796.log 2024-04-25T10:15:19.463 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77165.log 2024-04-25T10:15:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60913.log: /var/log/ceph/ceph-client.admin.67796.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67796.log.gz 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60913.log.gz 2024-04-25T10:15:19.463 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65454.log 2024-04-25T10:15:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.77165.log: /var/log/ceph/ceph-client.admin.77933.log 2024-04-25T10:15:19.463 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77165.log.gz 2024-04-25T10:15:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56668.log 2024-04-25T10:15:19.464 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65454.log.gz 2024-04-25T10:15:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47389.log 2024-04-25T10:15:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56668.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77933.log.gz 2024-04-25T10:15:19.464 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56668.log.gz 2024-04-25T10:15:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50503.log 2024-04-25T10:15:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51760.log 2024-04-25T10:15:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47389.log: /var/log/ceph/ceph-client.admin.50503.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47389.log.gz 2024-04-25T10:15:19.464 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73295.log 2024-04-25T10:15:19.465 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50503.log.gz 2024-04-25T10:15:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51467.log 2024-04-25T10:15:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51760.log: /var/log/ceph/ceph-client.admin.73295.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51760.log.gz 2024-04-25T10:15:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61680.log 2024-04-25T10:15:19.465 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73295.log.gz 2024-04-25T10:15:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71989.log 2024-04-25T10:15:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61680.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51467.log.gz 2024-04-25T10:15:19.465 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61680.log.gz 2024-04-25T10:15:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62016.log 2024-04-25T10:15:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53404.log 2024-04-25T10:15:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71989.log: /var/log/ceph/ceph-client.admin.62016.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71989.log.gz 2024-04-25T10:15:19.465 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62016.log.gz 2024-04-25T10:15:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47221.log 2024-04-25T10:15:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48169.log 2024-04-25T10:15:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53404.log: /var/log/ceph/ceph-client.admin.47221.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53404.log.gz 2024-04-25T10:15:19.466 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47221.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.0.62501.log 2024-04-25T10:15:19.466 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66087.log 2024-04-25T10:15:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.62501.log: /var/log/ceph/ceph-client.admin.48169.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62501.log.gz 2024-04-25T10:15:19.466 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48169.log.gz 2024-04-25T10:15:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71598.log 2024-04-25T10:15:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64351.log 2024-04-25T10:15:19.466 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66087.log.gz 2024-04-25T10:15:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54577.log 2024-04-25T10:15:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64351.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71598.log.gz 2024-04-25T10:15:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45850.log 2024-04-25T10:15:19.466 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64351.log.gz 2024-04-25T10:15:19.467 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41355.log 2024-04-25T10:15:19.467 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45850.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54577.log.gz 2024-04-25T10:15:19.467 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45850.log.gz 2024-04-25T10:15:19.467 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69871.log 2024-04-25T10:15:19.467 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48331.log 2024-04-25T10:15:19.467 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41355.log: /var/log/ceph/ceph-client.admin.69871.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41355.log.gz 2024-04-25T10:15:19.467 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41012.log 2024-04-25T10:15:19.467 INFO:teuthology.orchestra.run.smithi016.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.69871.log.gz 2024-04-25T10:15:19.467 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66255.log 2024-04-25T10:15:19.467 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41012.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48331.log.gz 2024-04-25T10:15:19.467 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67394.log 2024-04-25T10:15:19.467 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41012.log.gz 2024-04-25T10:15:19.467 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63277.log 2024-04-25T10:15:19.467 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67394.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66255.log.gz 2024-04-25T10:15:19.468 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67394.log.gz 2024-04-25T10:15:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76745.log 2024-04-25T10:15:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46618.log 2024-04-25T10:15:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63277.log: /var/log/ceph/ceph-client.admin.76745.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76745.log.gz 2024-04-25T10:15:19.468 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63277.log.gz 2024-04-25T10:15:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54985.log 2024-04-25T10:15:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63868.log 2024-04-25T10:15:19.468 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46618.log.gz 2024-04-25T10:15:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60997.log 2024-04-25T10:15:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63868.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.54985.log.gz 2024-04-25T10:15:19.468 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63868.log.gz 2024-04-25T10:15:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64776.log 2024-04-25T10:15:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55314.log 2024-04-25T10:15:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64776.log: /var/log/ceph/ceph-client.admin.60997.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64776.log.gz 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60997.log.gz 2024-04-25T10:15:19.469 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51980.log 2024-04-25T10:15:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53572.log 2024-04-25T10:15:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55314.log: /var/log/ceph/ceph-client.admin.51980.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55314.log.gz 2024-04-25T10:15:19.469 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51980.log.gz 2024-04-25T10:15:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45256.log 2024-04-25T10:15:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34440.log 2024-04-25T10:15:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45256.log: /var/log/ceph/ceph-client.admin.53572.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45256.log.gz 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53572.log.gz 2024-04-25T10:15:19.469 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40633.log 2024-04-25T10:15:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47473.log 2024-04-25T10:15:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34440.log: /var/log/ceph/ceph-client.admin.40633.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.34440.log.gz 2024-04-25T10:15:19.469 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40633.log.gz 2024-04-25T10:15:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56966.log 2024-04-25T10:15:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59313.log 2024-04-25T10:15:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47473.log: /var/log/ceph/ceph-client.admin.56966.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47473.log.gz 2024-04-25T10:15:19.470 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56966.log.gz 2024-04-25T10:15:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54427.log 2024-04-25T10:15:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57939.log 2024-04-25T10:15:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54427.log: /var/log/ceph/ceph-client.admin.59313.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.54427.log.gz 91.4% -- replaced with /var/log/ceph/ceph-client.admin.59313.log.gz 2024-04-25T10:15:19.470 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52418.log 2024-04-25T10:15:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51592.log 2024-04-25T10:15:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57939.log: /var/log/ceph/ceph-client.admin.52418.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57939.log.gz 2024-04-25T10:15:19.471 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70792.log 2024-04-25T10:15:19.471 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52418.log.gz 2024-04-25T10:15:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71777.log 2024-04-25T10:15:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51592.log: /var/log/ceph/ceph-client.admin.70792.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70792.log.gz 2024-04-25T10:15:19.471 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51592.log.gz 2024-04-25T10:15:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63109.log 2024-04-25T10:15:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70193.log 2024-04-25T10:15:19.471 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71777.log.gz 2024-04-25T10:15:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53446.log 2024-04-25T10:15:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70193.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63109.log.gz 2024-04-25T10:15:19.471 INFO:teuthology.orchestra.run.smithi016.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T10:15:19.472 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70193.log.gz 2024-04-25T10:15:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54619.log 2024-04-25T10:15:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73926.log 2024-04-25T10:15:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69955.log 2024-04-25T10:15:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54619.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53446.log.gz 2024-04-25T10:15:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73926.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.54619.log.gz 2024-04-25T10:15:19.472 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.73926.log.gz 2024-04-25T10:15:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65961.log 2024-04-25T10:15:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67478.log 2024-04-25T10:15:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70414.log 2024-04-25T10:15:19.472 INFO:teuthology.orchestra.run.smithi016.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.69955.log.gz 2024-04-25T10:15:19.472 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65961.log.gz 2024-04-25T10:15:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45808.log 2024-04-25T10:15:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49997.log 2024-04-25T10:15:19.473 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67478.log.gz/var/log/ceph/ceph-client.admin.45808.log: 2024-04-25T10:15:19.473 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70414.log.gz 2024-04-25T10:15:19.473 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77597.log 2024-04-25T10:15:19.473 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45808.log.gz 2024-04-25T10:15:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69456.log 2024-04-25T10:15:19.473 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43699.log 2024-04-25T10:15:19.473 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49997.log.gz 2024-04-25T10:15:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77597.log: /var/log/ceph/ceph-client.admin.69456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56243.log 2024-04-25T10:15:19.473 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77597.log.gz 2024-04-25T10:15:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43699.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.69456.log.gz 2024-04-25T10:15:19.474 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43699.log.gz 2024-04-25T10:15:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64620.log 2024-04-25T10:15:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53782.log 2024-04-25T10:15:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64620.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56243.log.gz 2024-04-25T10:15:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72743.log 2024-04-25T10:15:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77513.log 2024-04-25T10:15:19.474 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53782.log.gz 2024-04-25T10:15:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59205.log 2024-04-25T10:15:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77513.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.64620.log.gz 2024-04-25T10:15:19.474 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72743.log.gz 2024-04-25T10:15:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77513.log.gz 2024-04-25T10:15:19.475 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.59883.log 2024-04-25T10:15:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74972.log 2024-04-25T10:15:19.475 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59205.log.gz 2024-04-25T10:15:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51206.log 2024-04-25T10:15:19.475 INFO:teuthology.orchestra.run.smithi016.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.59883.log.gz/var/log/ceph/ceph-client.admin.74972.log: 2024-04-25T10:15:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70216.log 2024-04-25T10:15:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.51206.log: -- /var/log/ceph/ceph-client.admin.46516.log 2024-04-25T10:15:19.475 INFO:teuthology.orchestra.run.smithi016.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.74972.log.gz 2024-04-25T10:15:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40801.log 91.1% -- replaced with /var/log/ceph/ceph-client.admin.51206.log.gz 2024-04-25T10:15:19.475 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.475 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46516.log: /var/log/ceph/ceph-client.admin.70216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74753.log 2024-04-25T10:15:19.475 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46516.log.gz 2024-04-25T10:15:19.475 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70216.log.gz 2024-04-25T10:15:19.476 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52250.log 2024-04-25T10:15:19.476 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74753.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65562.log 2024-04-25T10:15:19.476 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40801.log.gz 2024-04-25T10:15:19.476 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52250.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74753.log.gz 2024-04-25T10:15:19.476 INFO:teuthology.orchestra.run.smithi016.stderr:gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52250.log.gz 2024-04-25T10:15:19.476 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.64555.log 2024-04-25T10:15:19.476 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65562.log: gzip -5 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65562.log.gz 2024-04-25T10:15:19.476 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52040.log 2024-04-25T10:15:19.476 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60558.log 2024-04-25T10:15:19.476 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64555.log.gz 2024-04-25T10:15:19.476 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66417.log 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52040.log.gz 2024-04-25T10:15:19.476 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.477 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57050.log 2024-04-25T10:15:19.477 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60558.log.gz/var/log/ceph/ceph-client.admin.66417.log: 2024-04-25T10:15:19.477 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45214.log 2024-04-25T10:15:19.477 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66417.log.gz 2024-04-25T10:15:19.477 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66628.log 2024-04-25T10:15:19.477 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57050.log.gz 2024-04-25T10:15:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77207.log 87.4% 2024-04-25T10:15:19.478 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45214.log.gz 2024-04-25T10:15:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40675.log 2024-04-25T10:15:19.478 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66628.log.gz/var/log/ceph/ceph-client.admin.77207.log: 2024-04-25T10:15:19.478 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39773.log 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77207.log.gz 2024-04-25T10:15:19.478 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40675.log: gzip -5 --verbose 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40675.log.gz 2024-04-25T10:15:19.479 INFO:teuthology.orchestra.run.smithi016.stderr: -- /var/log/ceph/ceph-client.admin.68374.log 2024-04-25T10:15:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72073.log 2024-04-25T10:15:19.479 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39773.log.gz/var/log/ceph/ceph-client.admin.68374.log: 2024-04-25T10:15:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58199.log 2024-04-25T10:15:19.480 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68374.log.gz 2024-04-25T10:15:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56201.log 2024-04-25T10:15:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70961.log 2024-04-25T10:15:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58199.log: 88.3%/var/log/ceph/ceph-client.admin.56201.log: -- replaced with /var/log/ceph/ceph-client.admin.72073.log.gz 2024-04-25T10:15:19.480 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56201.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43783.log 2024-04-25T10:15:19.480 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.480 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70961.log: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61040.log 2024-04-25T10:15:19.481 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58199.log.gz/var/log/ceph/ceph-client.admin.43783.log: 2024-04-25T10:15:19.481 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70961.log.gz 2024-04-25T10:15:19.481 INFO:teuthology.orchestra.run.smithi016.stderr:gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43783.log.gz 2024-04-25T10:15:19.481 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49623.log 2024-04-25T10:15:19.481 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77387.log 2024-04-25T10:15:19.481 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61040.log.gz 2024-04-25T10:15:19.481 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50041.log 2024-04-25T10:15:19.481 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49623.log.gz 2024-04-25T10:15:19.481 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44347.log 2024-04-25T10:15:19.481 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77387.log: /var/log/ceph/ceph-client.admin.50041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66754.log 2024-04-25T10:15:19.481 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.77387.log.gz 2024-04-25T10:15:19.481 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44347.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50041.log.gz 2024-04-25T10:15:19.481 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44347.log.gz 2024-04-25T10:15:19.482 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.64267.log 2024-04-25T10:15:19.482 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67054.log 2024-04-25T10:15:19.482 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66754.log.gz 2024-04-25T10:15:19.482 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64267.log: gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64267.log.gz 2024-04-25T10:15:19.482 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.61932.log 2024-04-25T10:15:19.482 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54685.log 2024-04-25T10:15:19.482 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.67054.log.gz/var/log/ceph/ceph-client.admin.61932.log: 2024-04-25T10:15:19.482 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72617.log 2024-04-25T10:15:19.482 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61932.log.gz 2024-04-25T10:15:19.482 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56840.log 2024-04-25T10:15:19.482 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54685.log.gz 2024-04-25T10:15:19.483 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46830.log 2024-04-25T10:15:19.483 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72617.log: /var/log/ceph/ceph-client.admin.56840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61722.log 2024-04-25T10:15:19.483 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46830.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56840.log.gz 2024-04-25T10:15:19.483 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72617.log.gz 2024-04-25T10:15:19.483 INFO:teuthology.orchestra.run.smithi016.stderr:gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46830.log.gz 2024-04-25T10:15:19.483 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60684.log 2024-04-25T10:15:19.484 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41935.log 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61722.log.gz 2024-04-25T10:15:19.484 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.484 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67983.log 2024-04-25T10:15:19.484 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60684.log.gz 2024-04-25T10:15:19.484 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59398.log 2024-04-25T10:15:19.485 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41935.log.gz 2024-04-25T10:15:19.485 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56583.log 2024-04-25T10:15:19.485 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.67983.log: /var/log/ceph/ceph-client.admin.59398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49784.log 2024-04-25T10:15:19.485 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67983.log.gz 2024-04-25T10:15:19.485 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60600.log 2024-04-25T10:15:19.485 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56583.log.gz 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59398.log.gz 2024-04-25T10:15:19.485 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.486 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55904.log 2024-04-25T10:15:19.486 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60600.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52544.log 2024-04-25T10:15:19.486 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49784.log.gz 2024-04-25T10:15:19.486 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60600.log.gz 2024-04-25T10:15:19.486 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55904.log: gzip 87.5% -5 --verbose -- /var/log/ceph/ceph-client.admin.48584.log 2024-04-25T10:15:19.487 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55904.log.gz 2024-04-25T10:15:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60348.log 2024-04-25T10:15:19.487 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52544.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40549.log 2024-04-25T10:15:19.487 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48584.log: /var/log/ceph/ceph-client.admin.60348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64862.log 2024-04-25T10:15:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40549.log: 89.4% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60348.log.gz 2024-04-25T10:15:19.487 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48584.log.gz 2024-04-25T10:15:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40549.log.gz 2024-04-25T10:15:19.487 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40083.log 2024-04-25T10:15:19.488 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47009.log 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64862.log.gz 2024-04-25T10:15:19.488 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.488 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59803.log 2024-04-25T10:15:19.488 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40083.log.gz 2024-04-25T10:15:19.488 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49444.log 2024-04-25T10:15:19.488 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47009.log.gz 2024-04-25T10:15:19.488 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56626.log 2024-04-25T10:15:19.488 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.59803.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59803.log.gz 2024-04-25T10:15:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77471.log 2024-04-25T10:15:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49444.log: /var/log/ceph/ceph-client.admin.56626.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49444.log.gz 2024-04-25T10:15:19.489 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56626.log.gz 2024-04-25T10:15:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77681.log 2024-04-25T10:15:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54943.log 2024-04-25T10:15:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77681.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77471.log.gz 2024-04-25T10:15:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50924.log 2024-04-25T10:15:19.489 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77681.log.gz 2024-04-25T10:15:19.489 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57218.log 2024-04-25T10:15:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56541.log 2024-04-25T10:15:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.50924.log: 0.0% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54943.log.gz 2024-04-25T10:15:19.490 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.0.50924.log.gz 2024-04-25T10:15:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59807.log 2024-04-25T10:15:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50377.log 2024-04-25T10:15:19.490 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.59807.log: 88.9% 0.0% -- replaced with /var/log/ceph/ceph-client.0.59807.log.gz 2024-04-25T10:15:19.490 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56541.log.gz 2024-04-25T10:15:19.491 INFO:teuthology.orchestra.run.smithi016.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.57218.log.gz 2024-04-25T10:15:19.491 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41557.log 2024-04-25T10:15:19.491 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64723.log 2024-04-25T10:15:19.491 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41557.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50377.log.gz 2024-04-25T10:15:19.491 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50461.log 2024-04-25T10:15:19.491 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41557.log.gz 2024-04-25T10:15:19.491 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62058.log 2024-04-25T10:15:19.491 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64723.log: /var/log/ceph/ceph-client.admin.50461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61890.log 2024-04-25T10:15:19.492 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62058.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64723.log.gz 2024-04-25T10:15:19.492 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50461.log.gz 2024-04-25T10:15:19.492 INFO:teuthology.orchestra.run.smithi016.stderr:gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62058.log.gz 2024-04-25T10:15:19.492 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62863.log 2024-04-25T10:15:19.492 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73421.log 2024-04-25T10:15:19.492 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61890.log.gz 2024-04-25T10:15:19.492 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52628.log 2024-04-25T10:15:19.492 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62863.log.gz 2024-04-25T10:15:19.493 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61596.log 2024-04-25T10:15:19.493 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49234.log 2024-04-25T10:15:19.493 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52628.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73421.log.gz 2024-04-25T10:15:19.493 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63005.log 2024-04-25T10:15:19.493 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52628.log.gz 2024-04-25T10:15:19.493 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49234.log: -- replaced with /var/log/ceph/ceph-client.admin.61596.log.gz 2024-04-25T10:15:19.493 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42397.log 2024-04-25T10:15:19.494 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49234.log.gz 2024-04-25T10:15:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55188.log 2024-04-25T10:15:19.494 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63005.log.gz 2024-04-25T10:15:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41696.log 2024-04-25T10:15:19.494 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42397.log.gz 2024-04-25T10:15:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70834.log 2024-04-25T10:15:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55188.log: /var/log/ceph/ceph-client.admin.41696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76702.log 2024-04-25T10:15:19.494 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70834.log: 88.2% 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41696.log.gz 2024-04-25T10:15:19.494 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55188.log.gz 2024-04-25T10:15:19.495 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51896.log 2024-04-25T10:15:19.495 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70834.log.gz 2024-04-25T10:15:19.495 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39896.log 2024-04-25T10:15:19.495 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76390.log 2024-04-25T10:15:19.495 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76702.log.gz 2024-04-25T10:15:19.495 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51896.log: /var/log/ceph/ceph-client.admin.39896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69284.log 2024-04-25T10:15:19.495 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51896.log.gz 2024-04-25T10:15:19.495 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2%/var/log/ceph/ceph-client.admin.76390.log: -- replaced with /var/log/ceph/ceph-client.admin.39896.log.gz 2024-04-25T10:15:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45545.log 2024-04-25T10:15:19.496 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76390.log.gz 2024-04-25T10:15:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55051.log 2024-04-25T10:15:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69284.log: /var/log/ceph/ceph-client.admin.45545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49276.log 2024-04-25T10:15:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55051.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.69284.log.gz 2024-04-25T10:15:19.496 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45545.log.gz 2024-04-25T10:15:19.496 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55051.log.gz 2024-04-25T10:15:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70372.log 2024-04-25T10:15:19.496 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68129.log/var/log/ceph/ceph-client.admin.49276.log: 2024-04-25T10:15:19.497 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43010.log 2024-04-25T10:15:19.497 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49276.log.gz 2024-04-25T10:15:19.497 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70372.log.gz 2024-04-25T10:15:19.497 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55846.log 2024-04-25T10:15:19.497 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68129.log: /var/log/ceph/ceph-client.0.43010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66129.log 2024-04-25T10:15:19.497 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43010.log.gz 2024-04-25T10:15:19.497 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68129.log.gz 2024-04-25T10:15:19.497 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50293.log 2024-04-25T10:15:19.497 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6%/var/log/ceph/ceph-client.admin.66129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48127.log 2024-04-25T10:15:19.498 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55846.log.gz 2024-04-25T10:15:19.498 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69371.log 2024-04-25T10:15:19.498 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66129.log.gz 2024-04-25T10:15:19.498 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76087.log 2024-04-25T10:15:19.498 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50293.log.gz 2024-04-25T10:15:19.498 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48127.log: /var/log/ceph/ceph-client.admin.69371.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48127.log.gz 2024-04-25T10:15:19.498 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43969.log 2024-04-25T10:15:19.499 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.69371.log.gz 2024-04-25T10:15:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63530.log 2024-04-25T10:15:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76087.log: /var/log/ceph/ceph-client.admin.43969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53234.log 2024-04-25T10:15:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.63530.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76087.log.gz 2024-04-25T10:15:19.499 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43969.log.gz 2024-04-25T10:15:19.499 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63530.log.gz 2024-04-25T10:15:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68439.log 2024-04-25T10:15:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57820.log 2024-04-25T10:15:19.499 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68439.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53234.log.gz 2024-04-25T10:15:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44956.log 2024-04-25T10:15:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57820.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68439.log.gz 2024-04-25T10:15:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73337.log 2024-04-25T10:15:19.500 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57820.log.gz 2024-04-25T10:15:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74673.log 2024-04-25T10:15:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44956.log: /var/log/ceph/ceph-client.admin.73337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43573.log 2024-04-25T10:15:19.500 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74673.log: 88.5% 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44956.log.gz 2024-04-25T10:15:19.500 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73337.log.gz 2024-04-25T10:15:19.500 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74673.log.gz 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76045.log 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.43573.log: -5 --verbose -- /var/log/ceph/ceph-client.0.43005.log 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44389.log 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43573.log.gz 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.43005.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76045.log.gz 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43005.log.gz 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57778.log 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76829.log 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57778.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44389.log.gz 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72368.log 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57778.log.gz/var/log/ceph/ceph-client.admin.76829.log: 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72181.log 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76829.log.gz 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.72368.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72368.log.gz 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46660.log 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61381.log 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.72181.log.gz 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44752.log 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46660.log.gz 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5/var/log/ceph/ceph-client.admin.61381.log: --verbose -- /var/log/ceph/ceph-client.admin.63343.log 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69829.log 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61381.log.gz 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44752.log.gz 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52765.log 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63343.log.gzgzip/var/log/ceph/ceph-client.admin.69829.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51207.log 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52765.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69829.log.gz 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52765.log.gz 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45934.log 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51207.log: gzip -5 --verbose -- 86.8% -- replaced with /var/log/ceph/ceph-client.admin.51207.log.gz 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.40591.log 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67646.log 2024-04-25T10:15:19.504 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0%/var/log/ceph/ceph-client.admin.40591.log: -- replaced with /var/log/ceph/ceph-client.admin.45934.log.gz 2024-04-25T10:15:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71304.log 2024-04-25T10:15:19.504 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9%/var/log/ceph/ceph-client.admin.67646.log: -- replaced with /var/log/ceph/ceph-client.admin.40591.log.gz 2024-04-25T10:15:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68495.log 2024-04-25T10:15:19.504 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67646.log.gz 2024-04-25T10:15:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64169.log 2024-04-25T10:15:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71304.log: /var/log/ceph/ceph-client.admin.68495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61764.log 2024-04-25T10:15:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64169.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71304.log.gz 2024-04-25T10:15:19.504 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68495.log.gz 2024-04-25T10:15:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67520.log 2024-04-25T10:15:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61764.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.64169.log.gz 2024-04-25T10:15:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65657.log 2024-04-25T10:15:19.504 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61764.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77081.log 2024-04-25T10:15:19.504 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67520.log: /var/log/ceph/ceph-client.admin.65657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49486.log 2024-04-25T10:15:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77081.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67520.log.gz 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65657.log.gz 2024-04-25T10:15:19.505 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.505 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77081.log.gz 2024-04-25T10:15:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58113.log 2024-04-25T10:15:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62244.log 2024-04-25T10:15:19.505 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49486.log.gz/var/log/ceph/ceph-client.admin.58113.log: 2024-04-25T10:15:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71470.log 2024-04-25T10:15:19.505 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58113.log.gz 2024-04-25T10:15:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58760.log 2024-04-25T10:15:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62244.log: /var/log/ceph/ceph-client.admin.71470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58071.log 2024-04-25T10:15:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58760.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71470.log.gz 2024-04-25T10:15:19.505 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58760.log.gz 90.8% -- replaced with /var/log/ceph/ceph-client.admin.62244.log.gz 2024-04-25T10:15:19.505 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67012.log 2024-04-25T10:15:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70624.log 2024-04-25T10:15:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76306.log 2024-04-25T10:15:19.506 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58071.log.gz 2024-04-25T10:15:19.506 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8%/var/log/ceph/ceph-client.admin.70624.log: -- replaced with /var/log/ceph/ceph-client.admin.67012.log.gz 2024-04-25T10:15:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65919.log 2024-04-25T10:15:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76306.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70624.log.gz 2024-04-25T10:15:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55408.log 2024-04-25T10:15:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65919.log: gzip 88.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.76306.log.gz -- 2024-04-25T10:15:19.506 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.52376.log 2024-04-25T10:15:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60432.log 2024-04-25T10:15:19.506 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2%/var/log/ceph/ceph-client.admin.55408.log: -- replaced with /var/log/ceph/ceph-client.admin.65919.log.gz 2024-04-25T10:15:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52376.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55408.log.gzgzip -5 --verbose -- 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52376.log.gz 2024-04-25T10:15:19.506 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.43615.log 2024-04-25T10:15:19.507 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49953.log 2024-04-25T10:15:19.507 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60432.log.gz 2024-04-25T10:15:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66712.log 2024-04-25T10:15:19.507 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43615.log.gz 2024-04-25T10:15:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74470.log 2024-04-25T10:15:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49953.log: /var/log/ceph/ceph-client.admin.66712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46144.log 2024-04-25T10:15:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74470.log: 88.3% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66712.log.gz 2024-04-25T10:15:19.507 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49953.log.gz 2024-04-25T10:15:19.507 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74470.log.gz 2024-04-25T10:15:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43741.log 2024-04-25T10:15:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68987.log 2024-04-25T10:15:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43741.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46144.log.gz 2024-04-25T10:15:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61082.log 2024-04-25T10:15:19.508 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43741.log.gz 2024-04-25T10:15:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47581.log 2024-04-25T10:15:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68987.log: /var/log/ceph/ceph-client.admin.61082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76787.log 2024-04-25T10:15:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47581.log: 89.4% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61082.log.gz 2024-04-25T10:15:19.508 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68987.log.gz 2024-04-25T10:15:19.508 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47581.log.gz 2024-04-25T10:15:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52082.log 2024-04-25T10:15:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73115.log 2024-04-25T10:15:19.508 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76787.log.gz/var/log/ceph/ceph-client.admin.52082.log: 2024-04-25T10:15:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40213.log 2024-04-25T10:15:19.508 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52082.log.gz/var/log/ceph/ceph-client.0.73115.log: 2024-04-25T10:15:19.509 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.73115.log.gz 2024-04-25T10:15:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53656.log 2024-04-25T10:15:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70919.log 2024-04-25T10:15:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53656.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40213.log.gz 2024-04-25T10:15:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45976.log 2024-04-25T10:15:19.509 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53656.log.gz 2024-04-25T10:15:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52166.log 2024-04-25T10:15:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70919.log: /var/log/ceph/ceph-client.admin.45976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68213.log 2024-04-25T10:15:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52166.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45976.log.gz 2024-04-25T10:15:19.509 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70919.log.gz 2024-04-25T10:15:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52166.log.gz 2024-04-25T10:15:19.509 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55991.log 2024-04-25T10:15:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66003.log 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68213.log.gz 2024-04-25T10:15:19.509 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61848.log 2024-04-25T10:15:19.510 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55991.log.gz 2024-04-25T10:15:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71512.log 2024-04-25T10:15:19.510 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66003.log.gz 2024-04-25T10:15:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53276.log 2024-04-25T10:15:19.510 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61848.log.gz 2024-04-25T10:15:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40465.log 2024-04-25T10:15:19.510 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.71512.log.gz 2024-04-25T10:15:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45040.log 2024-04-25T10:15:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58718.log 2024-04-25T10:15:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40465.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53276.log.gz/var/log/ceph/ceph-client.admin.45040.log: 2024-04-25T10:15:19.510 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40465.log.gz 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45040.log.gz 2024-04-25T10:15:19.510 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55678.log 2024-04-25T10:15:19.511 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49402.log 2024-04-25T10:15:19.511 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58718.log.gz 2024-04-25T10:15:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49047.log 2024-04-25T10:15:19.511 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55678.log.gz 2024-04-25T10:15:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62160.log 2024-04-25T10:15:19.511 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49402.log.gz 2024-04-25T10:15:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66586.log 2024-04-25T10:15:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76474.log 2024-04-25T10:15:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62160.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49047.log.gz 2024-04-25T10:15:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66586.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63694.log 2024-04-25T10:15:19.511 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62160.log.gz 2024-04-25T10:15:19.511 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76474.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66586.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46348.log 2024-04-25T10:15:19.511 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.511 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76474.log.gz 2024-04-25T10:15:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69587.log 2024-04-25T10:15:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63694.log: /var/log/ceph/ceph-client.admin.46348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66849.log 2024-04-25T10:15:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69587.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46348.log.gz 2024-04-25T10:15:19.512 INFO:teuthology.orchestra.run.smithi016.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.63694.log.gz 2024-04-25T10:15:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose 90.5% -- replaced with /var/log/ceph/ceph-client.admin.69587.log.gz 2024-04-25T10:15:19.512 INFO:teuthology.orchestra.run.smithi016.stderr: -- /var/log/ceph/ceph-mds.a.log 2024-04-25T10:15:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51854.log 2024-04-25T10:15:19.512 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66849.log.gz 2024-04-25T10:15:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46018.log 2024-04-25T10:15:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69727.log 2024-04-25T10:15:19.512 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51854.log.gz 2024-04-25T10:15:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59466.log 2024-04-25T10:15:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69727.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46018.log.gz 2024-04-25T10:15:19.512 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69727.log.gz 2024-04-25T10:15:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42019.log 2024-04-25T10:15:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40843.log 2024-04-25T10:15:19.513 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59466.log.gz 2024-04-25T10:15:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66544.log 2024-04-25T10:15:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40843.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42019.log.gz 2024-04-25T10:15:19.513 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40843.log.gz 2024-04-25T10:15:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46306.log 2024-04-25T10:15:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43885.log 2024-04-25T10:15:19.513 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66544.log.gz/var/log/ceph/ceph-client.admin.46306.log: 2024-04-25T10:15:19.513 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70540.log 2024-04-25T10:15:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43885.log: -- replaced with /var/log/ceph/ceph-client.admin.46306.log.gz 2024-04-25T10:15:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45082.log 2024-04-25T10:15:19.513 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43885.log.gz 2024-04-25T10:15:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41697.log 2024-04-25T10:15:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45082.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70540.log.gz 2024-04-25T10:15:19.514 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45082.log.gz 2024-04-25T10:15:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74554.log 2024-04-25T10:15:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73463.log 2024-04-25T10:15:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74554.log: /var/log/ceph/ceph-client.admin.41697.log: 89.4% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41697.log.gz 2024-04-25T10:15:19.514 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.74554.log.gz 2024-04-25T10:15:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77973.log 2024-04-25T10:15:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65699.log 2024-04-25T10:15:19.514 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73463.log.gz/var/log/ceph/ceph-client.admin.77973.log: 2024-04-25T10:15:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53740.log 2024-04-25T10:15:19.514 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77973.log.gz 2024-04-25T10:15:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61258.log 2024-04-25T10:15:19.514 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65699.log.gz/var/log/ceph/ceph-client.admin.53740.log: 2024-04-25T10:15:19.515 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57284.log 2024-04-25T10:15:19.515 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6%/var/log/ceph/ceph-client.admin.61258.log: -- replaced with /var/log/ceph/ceph-client.admin.53740.log.gz 2024-04-25T10:15:19.515 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61974.log 2024-04-25T10:15:19.515 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61258.log.gz 2024-04-25T10:15:19.515 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46925.log 2024-04-25T10:15:19.515 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61974.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57284.log.gz 2024-04-25T10:15:19.515 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61974.log.gz 2024-04-25T10:15:19.515 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50545.log 2024-04-25T10:15:19.515 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45724.log 2024-04-25T10:15:19.515 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46925.log.gz 2024-04-25T10:15:19.515 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60306.log 2024-04-25T10:15:19.515 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45724.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50545.log.gz 2024-04-25T10:15:19.515 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45724.log.gz 2024-04-25T10:15:19.515 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77302.log 2024-04-25T10:15:19.515 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48211.log 2024-04-25T10:15:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60306.log: /var/log/ceph/ceph-client.admin.77302.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60306.log.gz 2024-04-25T10:15:19.516 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64061.log 2024-04-25T10:15:19.516 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77302.log.gz 2024-04-25T10:15:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65388.log 2024-04-25T10:15:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48211.log: /var/log/ceph/ceph-client.admin.64061.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48211.log.gz 2024-04-25T10:15:19.516 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.64061.log.gzgzip 2024-04-25T10:15:19.516 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68416.log 2024-04-25T10:15:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77807.log 2024-04-25T10:15:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68416.log: /var/log/ceph/ceph-client.admin.65388.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68416.log.gz 2024-04-25T10:15:19.516 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70043.log 2024-04-25T10:15:19.516 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65388.log.gz/var/log/ceph/ceph-client.admin.77807.log: 2024-04-25T10:15:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:gzip 88.7% -5 --verbose -- /var/log/ceph/ceph-client.admin.56499.log 2024-04-25T10:15:19.516 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77807.log.gz 2024-04-25T10:15:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40339.log 2024-04-25T10:15:19.517 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70043.log: /var/log/ceph/ceph-client.admin.56499.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56499.log.gz 2024-04-25T10:15:19.517 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69956.log 2024-04-25T10:15:19.517 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40339.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.70043.log.gz 2024-04-25T10:15:19.517 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49318.log 2024-04-25T10:15:19.517 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40339.log.gz 2024-04-25T10:15:19.517 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54877.log 2024-04-25T10:15:19.517 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49318.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.69956.log.gz 2024-04-25T10:15:19.517 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49318.log.gz 2024-04-25T10:15:19.517 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34823.log 2024-04-25T10:15:19.517 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43843.log 2024-04-25T10:15:19.517 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34823.log: /var/log/ceph/ceph-client.admin.54877.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34823.log.gz 2024-04-25T10:15:19.517 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53488.log 2024-04-25T10:15:19.517 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54877.log.gz 2024-04-25T10:15:19.517 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60000.log 2024-04-25T10:15:19.518 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43843.log: /var/log/ceph/ceph-client.admin.53488.log: 88.9% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43843.log.gz 2024-04-25T10:15:19.518 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53488.log.gz 2024-04-25T10:15:19.518 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40297.log 2024-04-25T10:15:19.518 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67604.log 2024-04-25T10:15:19.518 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60000.log.gz 2024-04-25T10:15:19.518 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42355.log 2024-04-25T10:15:19.518 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40297.log.gz 2024-04-25T10:15:19.518 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66929.log 2024-04-25T10:15:19.518 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67604.log.gz/var/log/ceph/ceph-client.admin.42355.log: 2024-04-25T10:15:19.518 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41692.log 2024-04-25T10:15:19.518 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42355.log.gz 2024-04-25T10:15:19.518 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45340.log 2024-04-25T10:15:19.518 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41692.log: 88.8% 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41692.log.gz 2024-04-25T10:15:19.519 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66929.log.gz 2024-04-25T10:15:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52969.log 2024-04-25T10:15:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46186.log 2024-04-25T10:15:19.519 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45340.log.gz 2024-04-25T10:15:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41354.log 2024-04-25T10:15:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46186.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52969.log.gz 2024-04-25T10:15:19.519 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46186.log.gz 2024-04-25T10:15:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69522.log 2024-04-25T10:15:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71947.log 2024-04-25T10:15:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41354.log: /var/log/ceph/ceph-client.admin.69522.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41354.log.gz 2024-04-25T10:15:19.519 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69522.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42145.log 2024-04-25T10:15:19.519 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44221.log 2024-04-25T10:15:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71947.log: gzip -5 --verbose -- 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71947.log.gz 2024-04-25T10:15:19.519 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.77260.log 2024-04-25T10:15:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42145.log: /var/log/ceph/ceph-client.admin.44221.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42145.log.gz 2024-04-25T10:15:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49005.log 2024-04-25T10:15:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77260.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44221.log.gz 2024-04-25T10:15:19.520 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77260.log.gz 2024-04-25T10:15:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39957.log 2024-04-25T10:15:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44515.log 2024-04-25T10:15:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49005.log: /var/log/ceph/ceph-client.admin.39957.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49005.log.gz 2024-04-25T10:15:19.520 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54361.log 2024-04-25T10:15:19.520 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39957.log.gz 2024-04-25T10:15:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75080.log 2024-04-25T10:15:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44515.log: /var/log/ceph/ceph-client.admin.54361.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44515.log.gz 2024-04-25T10:15:19.520 INFO:teuthology.orchestra.run.smithi016.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.54361.log.gz 2024-04-25T10:15:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42271.log 2024-04-25T10:15:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60086.log 2024-04-25T10:15:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42271.log: /var/log/ceph/ceph-client.admin.75080.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42271.log.gz 2024-04-25T10:15:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40507.log 2024-04-25T10:15:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60086.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.75080.log.gz 2024-04-25T10:15:19.521 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60086.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69629.log 2024-04-25T10:15:19.521 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40507.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40507.log.gz 2024-04-25T10:15:19.521 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59097.log 2024-04-25T10:15:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73744.log 2024-04-25T10:15:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59097.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.69629.log.gz 2024-04-25T10:15:19.521 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59097.log.gz 2024-04-25T10:15:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65161.log 2024-04-25T10:15:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54209.log 2024-04-25T10:15:19.521 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73744.log.gz 2024-04-25T10:15:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.65161.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65161.log.gz 2024-04-25T10:15:19.522 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67730.log 2024-04-25T10:15:19.522 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61423.log 2024-04-25T10:15:19.522 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54209.log.gz 2024-04-25T10:15:19.522 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53362.log 2024-04-25T10:15:19.522 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61423.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61423.log.gz 2024-04-25T10:15:19.522 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.67730.log.gz 2024-04-25T10:15:19.522 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49826.log 2024-04-25T10:15:19.522 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39854.log 2024-04-25T10:15:19.522 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53362.log.gz 2024-04-25T10:15:19.522 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40381.log 2024-04-25T10:15:19.522 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39854.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49826.log.gz 2024-04-25T10:15:19.522 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39854.log.gz 2024-04-25T10:15:19.522 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58265.log 2024-04-25T10:15:19.523 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53530.log 2024-04-25T10:15:19.523 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40381.log.gz 2024-04-25T10:15:19.523 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.43440.log 2024-04-25T10:15:19.523 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53530.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58265.log.gz 2024-04-25T10:15:19.523 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53530.log.gz 2024-04-25T10:15:19.523 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58634.log 2024-04-25T10:15:19.523 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.2.43440.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.43440.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41691.log 2024-04-25T10:15:19.523 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.523 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56418.log 2024-04-25T10:15:19.523 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41691.log: /var/log/ceph/ceph-client.admin.58634.log: 88.6% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41691.log.gz 2024-04-25T10:15:19.523 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58634.log.gz 2024-04-25T10:15:19.523 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40125.log 2024-04-25T10:15:19.523 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48547.log 2024-04-25T10:15:19.523 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56418.log.gz 2024-04-25T10:15:19.523 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51294.log 2024-04-25T10:15:19.524 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48547.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40125.log.gz 2024-04-25T10:15:19.524 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69071.log 2024-04-25T10:15:19.524 INFO:teuthology.orchestra.run.smithi016.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.48547.log.gz 2024-04-25T10:15:19.524 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74596.log 2024-04-25T10:15:19.524 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69071.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69071.log.gz 2024-04-25T10:15:19.524 INFO:teuthology.orchestra.run.smithi016.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.51294.log.gz 2024-04-25T10:15:19.524 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76642.log 2024-04-25T10:15:19.524 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45682.log 2024-04-25T10:15:19.524 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74596.log.gz 2024-04-25T10:15:19.524 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76222.log 2024-04-25T10:15:19.524 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76642.log.gz 2024-04-25T10:15:19.524 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44011.log 2024-04-25T10:15:19.524 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45682.log.gz/var/log/ceph/ceph-client.admin.76222.log: 2024-04-25T10:15:19.524 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75239.log 2024-04-25T10:15:19.525 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76222.log.gz 2024-04-25T10:15:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53011.log 2024-04-25T10:15:19.525 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44011.log.gz/var/log/ceph/ceph-client.admin.75239.log: 2024-04-25T10:15:19.525 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58157.log 2024-04-25T10:15:19.525 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75239.log.gz 2024-04-25T10:15:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44179.log 2024-04-25T10:15:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58157.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53011.log.gz 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58157.log.gz 2024-04-25T10:15:19.525 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71224.log 2024-04-25T10:15:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76600.log 2024-04-25T10:15:19.525 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44179.log.gz 2024-04-25T10:15:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71224.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71224.log.gz 2024-04-25T10:15:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76600.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.76600.log.gz 2024-04-25T10:15:19.525 INFO:teuthology.orchestra.run.smithi016.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T10:15:19.526 INFO:teuthology.orchestra.run.smithi186.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T10:15:19.626 INFO:teuthology.orchestra.run.smithi186.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T10:15:19.637 INFO:teuthology.orchestra.run.smithi186.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T10:15:19.660 INFO:teuthology.orchestra.run.smithi016.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T10:15:19.670 INFO:teuthology.orchestra.run.smithi186.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T10:15:19.703 INFO:teuthology.orchestra.run.smithi186.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T10:15:19.705 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-25T10:15:19.705 INFO:teuthology.orchestra.run.smithi186.stderr:real 0m0.702s 2024-04-25T10:15:19.705 INFO:teuthology.orchestra.run.smithi186.stderr:user 0m3.492s 2024-04-25T10:15:19.705 INFO:teuthology.orchestra.run.smithi186.stderr:sys 0m0.135s 2024-04-25T10:15:19.726 INFO:teuthology.orchestra.run.smithi016.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T10:15:19.776 INFO:teuthology.orchestra.run.smithi016.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T10:15:19.801 INFO:teuthology.orchestra.run.smithi016.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T10:15:19.818 INFO:teuthology.orchestra.run.smithi016.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T10:15:19.820 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-25T10:15:19.820 INFO:teuthology.orchestra.run.smithi016.stderr:real 0m0.846s 2024-04-25T10:15:19.820 INFO:teuthology.orchestra.run.smithi016.stderr:user 0m3.989s 2024-04-25T10:15:19.820 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m0.530s 2024-04-25T10:15:19.821 INFO:tasks.ceph:Archiving logs... 2024-04-25T10:15:19.822 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673050/remote/smithi016/log 2024-04-25T10:15:19.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:15:21.013 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673050/remote/smithi186/log 2024-04-25T10:15:21.014 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:15:21.503 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T10:15:21.515 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T10:15:21.516 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:15:21.518 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:15:21.601 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T10:15:21.612 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T10:15:21.612 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:15:21.615 DEBUG:teuthology.orchestra.run.smithi186:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:15:21.628 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:15:21.630 INFO:teuthology.orchestra.run.smithi186.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:15:21.633 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:15:21.633 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-04-25T10:15:21.633 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 +1112us[+1100us] +/- 42ms 2024-04-25T10:15:21.633 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 43 -1253us[-1265us] +/- 68ms 2024-04-25T10:15:21.633 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 -586us[ -598us] +/- 89ms 2024-04-25T10:15:21.633 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:15:21.635 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:15:21.635 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2024-04-25T10:15:21.635 INFO:teuthology.orchestra.run.smithi186.stdout:^* hv01.front.sepia.ceph.com 3 6 377 42 +1080us[+1081us] +/- 42ms 2024-04-25T10:15:21.635 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 43 -1264us[-1263us] +/- 68ms 2024-04-25T10:15:21.635 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 -612us[ -611us] +/- 89ms 2024-04-25T10:15:21.636 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:15:21.636 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T10:15:21.653 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T10:15:21.654 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T10:15:21.669 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:15:21.761 DEBUG:teuthology.orchestra.run.smithi186:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:15:21.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T10:15:21.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:15:21.854 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T10:15:21.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:15:21.880 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T10:15:21.890 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T10:15:21.905 INFO:teuthology.task.internal:Duration was 1035.978252 seconds 2024-04-25T10:15:21.906 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T10:15:21.921 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T10:15:21.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:15:21.924 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:15:21.960 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:15:21.960 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:15:22.159 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T10:15:22.159 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-04-25T10:15:22.159 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:15:22.220 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi186.front.sepia.ceph.com 2024-04-25T10:15:22.220 DEBUG:teuthology.orchestra.run.smithi186:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:15:22.242 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T10:15:22.242 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:15:22.262 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:15:22.307 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T10:15:22.308 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:15:22.328 DEBUG:teuthology.orchestra.run.smithi186:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:15:22.459 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T10:15:22.471 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T10:15:22.471 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:15:22.496 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:15:22.520 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T10:15:22.530 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:15:22.538 DEBUG:teuthology.orchestra.run.smithi186:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:15:22.558 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-04-25T10:15:22.581 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = core 2024-04-25T10:15:22.593 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:15:22.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:15:22.624 DEBUG:teuthology.orchestra.run.smithi186:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:15:22.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:15:22.648 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T10:15:22.659 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T10:15:22.659 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673050/remote/smithi016 2024-04-25T10:15:22.659 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:15:22.700 DEBUG:teuthology.misc:Transferring archived files from smithi186:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673050/remote/smithi186 2024-04-25T10:15:22.700 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:15:22.733 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T10:15:22.733 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:15:22.736 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:15:22.781 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T10:15:22.793 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T10:15:22.794 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T10:15:22.805 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T10:15:22.805 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:15:22.807 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:15:22.820 INFO:teuthology.orchestra.run.smithi016.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:15 /home/ubuntu/cephtest 2024-04-25T10:15:22.836 INFO:teuthology.orchestra.run.smithi186.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:15 /home/ubuntu/cephtest 2024-04-25T10:15:22.837 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T10:15:22.847 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T10:15:22.872 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} duration: 1035.9782524108887 flavor: default owner: scheduled_neesingh@teuthology success: true 2024-04-25T10:15:22.872 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:15:22.957 INFO:teuthology.run:pass